2024-04-24T22:52:50.493 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-24T22:52:50.493 DEBUG:teuthology.run:Teuthology command: teuthology --name teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/mds-full} --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672173 --verbose -- /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672173/orig.config.yaml 2024-04-24T22:52:50.543 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-24T22:52:50.635 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672173 branch: squid description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/mds-full} email: ceph-qa@ceph.com first_in_suite: false job_id: '7672173' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd - disabled conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 client.0: debug client: 20 debug objectcacher: 20 debug objecter: 20 client.1: debug client: 20 debug objectcacher: 20 debug objecter: 20 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon osd backfillfull ratio: 0.6 mon osd full ratio: 0.7 mon osd nearfull ratio: 0.6 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 memstore device bytes: 200000000 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 1.0 osd mon report interval: 5 osd objectstore: memstore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL - PG_RECOVERY_FULL - PG_DEGRADED sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 kclient: syntax: v2 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 8528 sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 sleep_before_teardown: 0 subset: 122/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi032.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHYvII4S0kzwIW0jt40JM7Sembk40wVx3gw7EE3142Uuc/vtmIdcOdyv8dm8skJ8aREs6KWvYuorh9rmVcdC39s= smithi121.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMjX2m8W9izlQuvyTJSgRUGOUlFR0yuouRl55SNghfJ83YhYUHKqDo1sbWUssrqbOSoImMIXTQAFNp6j13d1Kkg= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_full teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-24_21:24:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-24T22:52:50.636 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa; will attempt to use it 2024-04-24T22:52:50.636 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks 2024-04-24T22:52:50.637 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-24T22:52:50.638 INFO:teuthology.task.internal:Checking packages... 2024-04-24T22:52:50.662 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd2c45d977142e024a59b68a8f1dce8f0b9840551' 2024-04-24T22:52:50.662 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-24T22:52:50.663 INFO:teuthology.packaging:ref: None 2024-04-24T22:52:50.663 INFO:teuthology.packaging:tag: None 2024-04-24T22:52:50.663 INFO:teuthology.packaging:branch: squid 2024-04-24T22:52:50.663 INFO:teuthology.packaging:sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:52:50.663 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-04-24T22:52:50.951 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2445.gd2c45d97 2024-04-24T22:52:50.952 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-24T22:52:50.959 INFO:teuthology.task.internal:no buildpackages task found 2024-04-24T22:52:50.959 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-24T22:52:50.980 INFO:teuthology.task.internal:Saving configuration 2024-04-24T22:52:50.998 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-24T22:52:51.005 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-24T22:52:51.032 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi032.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672173', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 22:47:50.921392', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHYvII4S0kzwIW0jt40JM7Sembk40wVx3gw7EE3142Uuc/vtmIdcOdyv8dm8skJ8aREs6KWvYuorh9rmVcdC39s='} 2024-04-24T22:52:51.056 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi121.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672173', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 22:47:50.922904', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMjX2m8W9izlQuvyTJSgRUGOUlFR0yuouRl55SNghfJ83YhYUHKqDo1sbWUssrqbOSoImMIXTQAFNp6j13d1Kkg='} 2024-04-24T22:52:51.056 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-24T22:52:51.062 INFO:teuthology.task.internal:roles: ubuntu@smithi032.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-24T22:52:51.062 INFO:teuthology.task.internal:roles: ubuntu@smithi121.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-24T22:52:51.062 INFO:teuthology.run_tasks:Running task console_log... 2024-04-24T22:52:51.129 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f5f76d120d0>, signals=[15]) 2024-04-24T22:52:51.130 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-24T22:52:51.137 INFO:teuthology.task.internal:Opening connections... 2024-04-24T22:52:51.138 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi032.front.sepia.ceph.com 2024-04-24T22:52:51.140 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T22:52:51.222 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi121.front.sepia.ceph.com 2024-04-24T22:52:51.223 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T22:52:51.298 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-24T22:52:51.304 DEBUG:teuthology.orchestra.run.smithi032:> uname -m 2024-04-24T22:52:51.323 INFO:teuthology.orchestra.run.smithi032.stdout:x86_64 2024-04-24T22:52:51.323 DEBUG:teuthology.orchestra.run.smithi032:> cat /etc/os-release 2024-04-24T22:52:51.380 INFO:teuthology.orchestra.run.smithi032.stdout:NAME="CentOS Stream" 2024-04-24T22:52:51.380 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION="9" 2024-04-24T22:52:51.380 INFO:teuthology.orchestra.run.smithi032.stdout:ID="centos" 2024-04-24T22:52:51.380 INFO:teuthology.orchestra.run.smithi032.stdout:ID_LIKE="rhel fedora" 2024-04-24T22:52:51.380 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION_ID="9" 2024-04-24T22:52:51.380 INFO:teuthology.orchestra.run.smithi032.stdout:PLATFORM_ID="platform:el9" 2024-04-24T22:52:51.380 INFO:teuthology.orchestra.run.smithi032.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T22:52:51.380 INFO:teuthology.orchestra.run.smithi032.stdout:ANSI_COLOR="0;31" 2024-04-24T22:52:51.380 INFO:teuthology.orchestra.run.smithi032.stdout:LOGO="fedora-logo-icon" 2024-04-24T22:52:51.381 INFO:teuthology.orchestra.run.smithi032.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T22:52:51.381 INFO:teuthology.orchestra.run.smithi032.stdout:HOME_URL="https://centos.org/" 2024-04-24T22:52:51.381 INFO:teuthology.orchestra.run.smithi032.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T22:52:51.381 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T22:52:51.381 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T22:52:51.381 INFO:teuthology.lock.ops:Updating smithi032.front.sepia.ceph.com on lock server 2024-04-24T22:52:51.405 DEBUG:teuthology.orchestra.run.smithi121:> uname -m 2024-04-24T22:52:51.423 INFO:teuthology.orchestra.run.smithi121.stdout:x86_64 2024-04-24T22:52:51.424 DEBUG:teuthology.orchestra.run.smithi121:> cat /etc/os-release 2024-04-24T22:52:51.480 INFO:teuthology.orchestra.run.smithi121.stdout:NAME="CentOS Stream" 2024-04-24T22:52:51.480 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION="9" 2024-04-24T22:52:51.480 INFO:teuthology.orchestra.run.smithi121.stdout:ID="centos" 2024-04-24T22:52:51.480 INFO:teuthology.orchestra.run.smithi121.stdout:ID_LIKE="rhel fedora" 2024-04-24T22:52:51.480 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION_ID="9" 2024-04-24T22:52:51.480 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM_ID="platform:el9" 2024-04-24T22:52:51.480 INFO:teuthology.orchestra.run.smithi121.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T22:52:51.481 INFO:teuthology.orchestra.run.smithi121.stdout:ANSI_COLOR="0;31" 2024-04-24T22:52:51.481 INFO:teuthology.orchestra.run.smithi121.stdout:LOGO="fedora-logo-icon" 2024-04-24T22:52:51.481 INFO:teuthology.orchestra.run.smithi121.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T22:52:51.481 INFO:teuthology.orchestra.run.smithi121.stdout:HOME_URL="https://centos.org/" 2024-04-24T22:52:51.481 INFO:teuthology.orchestra.run.smithi121.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T22:52:51.481 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T22:52:51.481 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T22:52:51.481 INFO:teuthology.lock.ops:Updating smithi121.front.sepia.ceph.com on lock server 2024-04-24T22:52:51.505 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-24T22:52:51.515 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-24T22:52:51.521 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-24T22:52:51.521 DEBUG:teuthology.orchestra.run.smithi032:> test '!' -e /home/ubuntu/cephtest 2024-04-24T22:52:51.524 DEBUG:teuthology.orchestra.run.smithi121:> test '!' -e /home/ubuntu/cephtest 2024-04-24T22:52:51.541 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-24T22:52:51.547 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-24T22:52:51.547 DEBUG:teuthology.orchestra.run.smithi032:> test -z $(ls -A /var/lib/ceph) 2024-04-24T22:52:51.582 DEBUG:teuthology.orchestra.run.smithi121:> test -z $(ls -A /var/lib/ceph) 2024-04-24T22:52:51.616 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-24T22:52:51.671 INFO:teuthology.run_tasks:Running task kernel... 2024-04-24T22:52:51.685 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-04-24T22:52:51.685 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2024-04-24T22:52:51.686 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-04-24T22:52:51.686 DEBUG:teuthology.orchestra.run.smithi032:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T22:52:51.686 DEBUG:teuthology.orchestra.run.smithi121:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T22:52:51.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:52:51.702 DEBUG:teuthology.orchestra.run.smithi121:> uname -r 2024-04-24T22:52:51.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:52:51.706 DEBUG:teuthology.orchestra.run.smithi032:> uname -r 2024-04-24T22:52:51.758 INFO:teuthology.orchestra.run.smithi121.stdout:5.14.0-437.el9.x86_64 2024-04-24T22:52:51.758 INFO:teuthology.task.kernel:Running kernel on smithi121: 5.14.0-437.el9.x86_64 2024-04-24T22:52:51.759 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum install -y kernel 2024-04-24T22:52:51.762 INFO:teuthology.orchestra.run.smithi032.stdout:5.14.0-437.el9.x86_64 2024-04-24T22:52:51.762 INFO:teuthology.task.kernel:Running kernel on smithi032: 5.14.0-437.el9.x86_64 2024-04-24T22:52:51.762 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum install -y kernel 2024-04-24T22:52:54.592 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 9 - BaseOS 6.3 MB/s | 8.0 MB 00:01 2024-04-24T22:52:57.327 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 9 - AppStream 12 MB/s | 19 MB 00:01 2024-04-24T22:52:57.516 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 9 - BaseOS 1.9 MB/s | 8.0 MB 00:04 2024-04-24T22:52:59.384 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 9 - AppStream 24 MB/s | 19 MB 00:00 2024-04-24T22:53:02.345 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 9 - CRB 4.8 MB/s | 6.2 MB 00:01 2024-04-24T22:53:03.535 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-04-24T22:53:04.028 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 16 kB 00:00 2024-04-24T22:53:05.940 INFO:teuthology.orchestra.run.smithi032.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-04-24T22:53:06.043 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 9 - Extras packages 14 kB/s | 16 kB 00:01 2024-04-24T22:53:08.105 INFO:teuthology.orchestra.run.smithi121.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-04-24T22:53:11.221 INFO:teuthology.orchestra.run.smithi032.stdout:lab-extras 23 kB/s | 1.7 kB 00:00 2024-04-24T22:53:13.169 INFO:teuthology.orchestra.run.smithi032.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-24T22:53:13.170 INFO:teuthology.orchestra.run.smithi032.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-24T22:53:13.243 INFO:teuthology.orchestra.run.smithi121.stdout:lab-extras 9.1 kB/s | 1.7 kB 00:00 2024-04-24T22:53:13.265 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-04-24T22:53:13.266 INFO:teuthology.orchestra.run.smithi032.stdout:Nothing to do. 2024-04-24T22:53:13.266 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-04-24T22:53:13.365 DEBUG:teuthology.orchestra.run.smithi032:> echo no | sudo yum reinstall kernel || true 2024-04-24T22:53:13.887 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:00:02 ago on Wed 24 Apr 2024 10:53:11 PM UTC. 2024-04-24T22:53:14.010 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-04-24T22:53:14.024 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-04-24T22:53:14.024 INFO:teuthology.orchestra.run.smithi032.stdout: Package Architecture Version Repository Size 2024-04-24T22:53:14.024 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-04-24T22:53:14.025 INFO:teuthology.orchestra.run.smithi032.stdout:Reinstalling: 2024-04-24T22:53:14.025 INFO:teuthology.orchestra.run.smithi032.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-24T22:53:14.025 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:53:14.025 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction Summary 2024-04-24T22:53:14.025 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-04-24T22:53:14.025 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:53:14.025 INFO:teuthology.orchestra.run.smithi032.stdout:Total download size: 255 k 2024-04-24T22:53:14.025 INFO:teuthology.orchestra.run.smithi032.stdout:Installed size: 0 2024-04-24T22:53:14.025 INFO:teuthology.orchestra.run.smithi032.stderr:Operation aborted. 2024-04-24T22:53:14.088 INFO:teuthology.orchestra.run.smithi032.stdout:Is this ok [y/N]: 2024-04-24T22:53:14.089 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum reinstall -y kernel || true 2024-04-24T22:53:14.603 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:00:03 ago on Wed 24 Apr 2024 10:53:11 PM UTC. 2024-04-24T22:53:14.723 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-04-24T22:53:14.724 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-04-24T22:53:14.724 INFO:teuthology.orchestra.run.smithi032.stdout: Package Architecture Version Repository Size 2024-04-24T22:53:14.724 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-04-24T22:53:14.725 INFO:teuthology.orchestra.run.smithi032.stdout:Reinstalling: 2024-04-24T22:53:14.725 INFO:teuthology.orchestra.run.smithi032.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-24T22:53:14.725 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:53:14.725 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction Summary 2024-04-24T22:53:14.725 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-04-24T22:53:14.725 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:53:14.725 INFO:teuthology.orchestra.run.smithi032.stdout:Total download size: 255 k 2024-04-24T22:53:14.725 INFO:teuthology.orchestra.run.smithi032.stdout:Installed size: 0 2024-04-24T22:53:14.725 INFO:teuthology.orchestra.run.smithi032.stdout:Downloading Packages: 2024-04-24T22:53:15.075 INFO:teuthology.orchestra.run.smithi121.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-24T22:53:15.075 INFO:teuthology.orchestra.run.smithi121.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-24T22:53:15.170 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-04-24T22:53:15.171 INFO:teuthology.orchestra.run.smithi121.stdout:Nothing to do. 2024-04-24T22:53:15.172 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-04-24T22:53:15.265 INFO:teuthology.orchestra.run.smithi032.stdout:kernel-5.14.0-437.el9.x86_64.rpm 956 kB/s | 255 kB 00:00 2024-04-24T22:53:15.265 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2024-04-24T22:53:15.265 INFO:teuthology.orchestra.run.smithi032.stdout:Total 472 kB/s | 255 kB 00:00 2024-04-24T22:53:15.284 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction check 2024-04-24T22:53:15.288 DEBUG:teuthology.orchestra.run.smithi121:> echo no | sudo yum reinstall kernel || true 2024-04-24T22:53:15.472 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction check succeeded. 2024-04-24T22:53:15.473 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction test 2024-04-24T22:53:15.478 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction test succeeded. 2024-04-24T22:53:15.478 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction 2024-04-24T22:53:15.634 INFO:teuthology.orchestra.run.smithi032.stdout: Preparing : 1/1 2024-04-24T22:53:15.691 INFO:teuthology.orchestra.run.smithi032.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-24T22:53:15.691 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T22:53:15.829 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:00:02 ago on Wed 24 Apr 2024 10:53:13 PM UTC. 2024-04-24T22:53:15.965 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-04-24T22:53:15.977 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-04-24T22:53:15.977 INFO:teuthology.orchestra.run.smithi121.stdout: Package Architecture Version Repository Size 2024-04-24T22:53:15.977 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-04-24T22:53:15.977 INFO:teuthology.orchestra.run.smithi121.stdout:Reinstalling: 2024-04-24T22:53:15.977 INFO:teuthology.orchestra.run.smithi121.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-24T22:53:15.977 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-24T22:53:15.978 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-04-24T22:53:15.978 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-04-24T22:53:15.978 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-24T22:53:15.978 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 255 k 2024-04-24T22:53:15.978 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 0 2024-04-24T22:53:15.979 INFO:teuthology.orchestra.run.smithi121.stderr:Operation aborted. 2024-04-24T22:53:16.027 INFO:teuthology.orchestra.run.smithi121.stdout:Is this ok [y/N]: 2024-04-24T22:53:16.027 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum reinstall -y kernel || true 2024-04-24T22:53:16.483 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T22:53:16.483 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-24T22:53:16.519 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:00:03 ago on Wed 24 Apr 2024 10:53:13 PM UTC. 2024-04-24T22:53:16.634 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-04-24T22:53:16.635 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-04-24T22:53:16.635 INFO:teuthology.orchestra.run.smithi121.stdout: Package Architecture Version Repository Size 2024-04-24T22:53:16.635 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-04-24T22:53:16.635 INFO:teuthology.orchestra.run.smithi121.stdout:Reinstalling: 2024-04-24T22:53:16.635 INFO:teuthology.orchestra.run.smithi121.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-24T22:53:16.635 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-24T22:53:16.635 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-04-24T22:53:16.635 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-04-24T22:53:16.636 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-24T22:53:16.636 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 255 k 2024-04-24T22:53:16.636 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 0 2024-04-24T22:53:16.636 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-04-24T22:53:16.929 INFO:teuthology.orchestra.run.smithi121.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.4 MB/s | 255 kB 00:00 2024-04-24T22:53:16.929 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-04-24T22:53:16.929 INFO:teuthology.orchestra.run.smithi121.stdout:Total 868 kB/s | 255 kB 00:00 2024-04-24T22:53:16.944 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-04-24T22:53:17.131 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-04-24T22:53:17.132 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-04-24T22:53:17.136 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-04-24T22:53:17.137 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-04-24T22:53:17.160 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T22:53:17.160 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:53:17.160 INFO:teuthology.orchestra.run.smithi032.stdout:Reinstalled: 2024-04-24T22:53:17.160 INFO:teuthology.orchestra.run.smithi032.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-24T22:53:17.160 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:53:17.160 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-04-24T22:53:17.264 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-04-24T22:53:17.330 INFO:teuthology.orchestra.run.smithi121.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-24T22:53:17.330 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T22:53:17.337 DEBUG:teuthology.orchestra.run.smithi032:> rpm -q kernel | sort -rV | head -n 1 2024-04-24T22:53:17.385 INFO:teuthology.orchestra.run.smithi032.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-24T22:53:17.385 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-24T22:53:17.385 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-24T22:53:17.385 DEBUG:teuthology.parallel:result is None 2024-04-24T22:53:18.079 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T22:53:18.079 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-24T22:53:18.792 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T22:53:18.792 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-24T22:53:18.792 INFO:teuthology.orchestra.run.smithi121.stdout:Reinstalled: 2024-04-24T22:53:18.792 INFO:teuthology.orchestra.run.smithi121.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-24T22:53:18.793 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-24T22:53:18.793 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-04-24T22:53:18.968 DEBUG:teuthology.orchestra.run.smithi121:> rpm -q kernel | sort -rV | head -n 1 2024-04-24T22:53:19.015 INFO:teuthology.orchestra.run.smithi121.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-24T22:53:19.015 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-24T22:53:19.016 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-24T22:53:19.016 DEBUG:teuthology.parallel:result is None 2024-04-24T22:53:19.016 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-24T22:53:19.026 INFO:teuthology.task.internal:Creating test directory... 2024-04-24T22:53:19.027 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T22:53:19.030 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T22:53:19.046 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-24T22:53:19.052 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-24T22:53:19.119 INFO:teuthology.task.internal:Creating archive directory... 2024-04-24T22:53:19.119 DEBUG:teuthology.orchestra.run.smithi032:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T22:53:19.122 DEBUG:teuthology.orchestra.run.smithi121:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T22:53:19.153 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-24T22:53:19.160 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-24T22:53:19.160 DEBUG:teuthology.orchestra.run.smithi032:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T22:53:19.186 DEBUG:teuthology.orchestra.run.smithi121:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T22:53:19.215 INFO:teuthology.orchestra.run.smithi121.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T22:53:19.229 INFO:teuthology.orchestra.run.smithi121.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T22:53:19.238 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T22:53:19.257 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T22:53:19.258 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-24T22:53:19.269 INFO:teuthology.task.internal:Configuring sudo... 2024-04-24T22:53:19.270 DEBUG:teuthology.orchestra.run.smithi032:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T22:53:19.272 DEBUG:teuthology.orchestra.run.smithi121:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T22:53:19.297 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-24T22:53:19.306 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-24T22:53:19.306 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T22:53:19.339 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T22:53:19.354 DEBUG:teuthology.orchestra.run.smithi032:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T22:53:19.431 DEBUG:teuthology.orchestra.run.smithi032:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T22:53:19.492 DEBUG:teuthology.orchestra.run.smithi032:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T22:53:19.574 DEBUG:teuthology.orchestra.run.smithi032:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T22:53:19.615 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T22:53:19.616 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T22:53:19.682 DEBUG:teuthology.orchestra.run.smithi121:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T22:53:19.723 DEBUG:teuthology.orchestra.run.smithi121:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T22:53:19.781 DEBUG:teuthology.orchestra.run.smithi121:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T22:53:19.863 DEBUG:teuthology.orchestra.run.smithi121:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T22:53:19.905 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T22:53:19.905 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T22:53:19.969 DEBUG:teuthology.orchestra.run.smithi032:> sudo service rsyslog restart 2024-04-24T22:53:19.972 DEBUG:teuthology.orchestra.run.smithi121:> sudo service rsyslog restart 2024-04-24T22:53:20.004 INFO:teuthology.orchestra.run.smithi032.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T22:53:20.039 INFO:teuthology.orchestra.run.smithi121.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T22:53:20.394 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-24T22:53:20.402 INFO:teuthology.task.internal:Starting timer... 2024-04-24T22:53:20.402 INFO:teuthology.run_tasks:Running task pcp... 2024-04-24T22:53:20.412 INFO:teuthology.run_tasks:Running task selinux... 2024-04-24T22:53:20.422 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-24T22:53:20.423 DEBUG:teuthology.orchestra.run.smithi032:> sudo service auditd rotate 2024-04-24T22:53:20.494 INFO:teuthology.orchestra.run.smithi032.stdout:Rotating logs: 2024-04-24T22:53:20.496 DEBUG:teuthology.orchestra.run.smithi121:> sudo service auditd rotate 2024-04-24T22:53:20.567 INFO:teuthology.orchestra.run.smithi121.stdout:Rotating logs: 2024-04-24T22:53:20.569 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-24T22:53:20.569 DEBUG:teuthology.orchestra.run.smithi032:> /usr/sbin/getenforce 2024-04-24T22:53:20.593 INFO:teuthology.orchestra.run.smithi032.stdout:Permissive 2024-04-24T22:53:20.594 DEBUG:teuthology.orchestra.run.smithi121:> /usr/sbin/getenforce 2024-04-24T22:53:20.613 INFO:teuthology.orchestra.run.smithi121.stdout:Permissive 2024-04-24T22:53:20.613 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi032.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi121.front.sepia.ceph.com': 'permissive'} 2024-04-24T22:53:20.613 DEBUG:teuthology.orchestra.run.smithi032:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T22:53:20.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:53:20.661 DEBUG:teuthology.orchestra.run.smithi121:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T22:53:20.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:53:20.685 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-24T22:53:20.685 DEBUG:teuthology.orchestra.run.smithi032:> sudo /usr/sbin/setenforce permissive 2024-04-24T22:53:20.728 DEBUG:teuthology.orchestra.run.smithi121:> sudo /usr/sbin/setenforce permissive 2024-04-24T22:53:20.752 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-24T22:53:20.762 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-24T22:53:20.781 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-24T22:53:20.782 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi032.front.sepia.ceph.com,smithi121.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-24T22:56:53.711 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi032.front.sepia.ceph.com'), Remote(name='ubuntu@smithi121.front.sepia.ceph.com')] 2024-04-24T22:56:53.712 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-04-24T22:56:53.713 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T22:56:53.793 DEBUG:teuthology.orchestra.run.smithi032:> true 2024-04-24T22:56:53.875 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-04-24T22:56:53.876 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2024-04-24T22:56:53.877 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T22:56:53.951 DEBUG:teuthology.orchestra.run.smithi121:> true 2024-04-24T22:56:54.038 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi121.front.sepia.ceph.com' 2024-04-24T22:56:54.038 INFO:teuthology.run_tasks:Running task clock... 2024-04-24T22:56:54.048 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-24T22:56:54.048 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-24T22:56:54.049 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T22:56:54.051 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-24T22:56:54.051 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T22:56:54.080 INFO:teuthology.orchestra.run.smithi032.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-24T22:56:54.094 INFO:teuthology.orchestra.run.smithi032.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-24T22:56:54.108 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-24T22:56:54.118 INFO:teuthology.orchestra.run.smithi032.stderr:sudo: ntpd: command not found 2024-04-24T22:56:54.121 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-24T22:56:54.129 INFO:teuthology.orchestra.run.smithi032.stdout:506 Cannot talk to daemon 2024-04-24T22:56:54.143 INFO:teuthology.orchestra.run.smithi032.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-24T22:56:54.144 INFO:teuthology.orchestra.run.smithi121.stderr:sudo: ntpd: command not found 2024-04-24T22:56:54.154 INFO:teuthology.orchestra.run.smithi121.stdout:506 Cannot talk to daemon 2024-04-24T22:56:54.156 INFO:teuthology.orchestra.run.smithi032.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-24T22:56:54.168 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-24T22:56:54.181 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-24T22:56:54.213 INFO:teuthology.orchestra.run.smithi032.stderr:bash: line 1: ntpq: command not found 2024-04-24T22:56:54.217 INFO:teuthology.orchestra.run.smithi032.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T22:56:54.217 INFO:teuthology.orchestra.run.smithi032.stdout:=============================================================================== 2024-04-24T22:56:54.217 INFO:teuthology.orchestra.run.smithi032.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:56:54.217 INFO:teuthology.orchestra.run.smithi032.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:56:54.217 INFO:teuthology.orchestra.run.smithi032.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:56:54.217 INFO:teuthology.orchestra.run.smithi032.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:56:54.233 INFO:teuthology.orchestra.run.smithi121.stderr:bash: line 1: ntpq: command not found 2024-04-24T22:56:54.237 INFO:teuthology.orchestra.run.smithi121.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T22:56:54.237 INFO:teuthology.orchestra.run.smithi121.stdout:=============================================================================== 2024-04-24T22:56:54.237 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:56:54.237 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:56:54.237 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:56:54.237 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:56:54.238 INFO:teuthology.run_tasks:Running task install... 2024-04-24T22:56:54.247 DEBUG:teuthology.task.install:project ceph 2024-04-24T22:56:54.247 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}} 2024-04-24T22:56:54.247 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'} 2024-04-24T22:56:54.247 INFO:teuthology.task.install:Using flavor: default 2024-04-24T22:56:54.253 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-24T22:56:54.253 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-24T22:56:54.253 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-24T22:56:54.254 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:56:54.256 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-24T22:56:54.257 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:56:54.402 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/ 2024-04-24T22:56:54.403 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2445.gd2c45d97 2024-04-24T22:56:54.407 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/ 2024-04-24T22:56:54.407 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2445.gd2c45d97 2024-04-24T22:56:54.514 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-24T22:56:54.514 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T22:56:54.514 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-24T22:56:54.522 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-24T22:56:54.522 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T22:56:54.522 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-24T22:56:54.540 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-24T22:56:54.541 DEBUG:teuthology.orchestra.run.smithi032:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2c45d977142e024a59b68a8f1dce8f0b9840551/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-24T22:56:54.547 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-24T22:56:54.547 DEBUG:teuthology.orchestra.run.smithi121:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2c45d977142e024a59b68a8f1dce8f0b9840551/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-24T22:56:54.608 DEBUG:teuthology.orchestra.run.smithi032:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-24T22:56:54.611 DEBUG:teuthology.orchestra.run.smithi121:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-24T22:56:54.683 DEBUG:teuthology.orchestra.run.smithi121:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-24T22:56:54.685 DEBUG:teuthology.orchestra.run.smithi032:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-24T22:56:54.746 INFO:teuthology.orchestra.run.smithi121.stdout:check_obsoletes = 1 2024-04-24T22:56:54.747 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum clean all 2024-04-24T22:56:54.749 INFO:teuthology.orchestra.run.smithi032.stdout:check_obsoletes = 1 2024-04-24T22:56:54.750 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum clean all 2024-04-24T22:56:54.972 INFO:teuthology.orchestra.run.smithi121.stdout:48 files removed 2024-04-24T22:56:54.984 INFO:teuthology.orchestra.run.smithi032.stdout:48 files removed 2024-04-24T22:56:54.999 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-24T22:56:55.011 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-24T22:56:55.608 INFO:teuthology.orchestra.run.smithi121.stdout:ceph packages for x86_64 237 kB/s | 83 kB 00:00 2024-04-24T22:56:55.740 INFO:teuthology.orchestra.run.smithi032.stdout:ceph packages for x86_64 183 kB/s | 83 kB 00:00 2024-04-24T22:56:55.934 INFO:teuthology.orchestra.run.smithi121.stdout:ceph noarch packages 41 kB/s | 11 kB 00:00 2024-04-24T22:56:56.207 INFO:teuthology.orchestra.run.smithi032.stdout:ceph noarch packages 27 kB/s | 11 kB 00:00 2024-04-24T22:56:56.301 INFO:teuthology.orchestra.run.smithi121.stdout:ceph source packages 5.7 kB/s | 1.9 kB 00:00 2024-04-24T22:56:56.524 INFO:teuthology.orchestra.run.smithi032.stdout:ceph source packages 6.7 kB/s | 1.9 kB 00:00 2024-04-24T22:56:57.001 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-04-24T22:56:57.133 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-04-24T22:56:58.785 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 9 - AppStream 27 MB/s | 19 MB 00:00 2024-04-24T22:57:01.900 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 9 - AppStream 5.2 MB/s | 19 MB 00:03 2024-04-24T22:57:03.203 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 9 - CRB 7.1 MB/s | 6.2 MB 00:00 2024-04-24T22:57:04.662 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 9 - Extras packages 50 kB/s | 16 kB 00:00 2024-04-24T22:57:05.888 INFO:teuthology.orchestra.run.smithi121.stdout:Extra Packages for Enterprise Linux 19 MB/s | 21 MB 00:01 2024-04-24T22:57:06.635 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 9 - CRB 5.8 MB/s | 6.2 MB 00:01 2024-04-24T22:57:08.719 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 9 - Extras packages 18 kB/s | 16 kB 00:00 2024-04-24T22:57:09.988 INFO:teuthology.orchestra.run.smithi032.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-04-24T22:57:10.963 INFO:teuthology.orchestra.run.smithi121.stdout:lab-extras 45 kB/s | 1.7 kB 00:00 2024-04-24T22:57:13.089 INFO:teuthology.orchestra.run.smithi121.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T22:57:13.090 INFO:teuthology.orchestra.run.smithi121.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T22:57:13.101 INFO:teuthology.orchestra.run.smithi121.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-24T22:57:13.102 INFO:teuthology.orchestra.run.smithi121.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-24T22:57:13.102 INFO:teuthology.orchestra.run.smithi121.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-24T22:57:13.103 INFO:teuthology.orchestra.run.smithi121.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-24T22:57:13.105 INFO:teuthology.orchestra.run.smithi121.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-24T22:57:13.107 INFO:teuthology.orchestra.run.smithi121.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-24T22:57:13.108 INFO:teuthology.orchestra.run.smithi121.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-24T22:57:13.169 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-04-24T22:57:13.177 INFO:teuthology.orchestra.run.smithi121.stdout:======================================================================================= 2024-04-24T22:57:13.178 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2024-04-24T22:57:13.178 INFO:teuthology.orchestra.run.smithi121.stdout:======================================================================================= 2024-04-24T22:57:13.178 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-04-24T22:57:13.178 INFO:teuthology.orchestra.run.smithi121.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-24T22:57:13.178 INFO:teuthology.orchestra.run.smithi121.stdout: ceph x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 6.6 k 2024-04-24T22:57:13.178 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-base x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.4 M 2024-04-24T22:57:13.178 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 890 k 2024-04-24T22:57:13.178 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 145 k 2024-04-24T22:57:13.178 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 1.5 M 2024-04-24T22:57:13.178 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 141 k 2024-04-24T22:57:13.178 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 3.5 M 2024-04-24T22:57:13.179 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 7.4 M 2024-04-24T22:57:13.179 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-rook noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 49 k 2024-04-24T22:57:13.179 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-radosgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 10 M 2024-04-24T22:57:13.179 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-test x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 48 M 2024-04-24T22:57:13.179 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-volume noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 264 k 2024-04-24T22:57:13.179 INFO:teuthology.orchestra.run.smithi121.stdout: cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 762 k 2024-04-24T22:57:13.179 INFO:teuthology.orchestra.run.smithi121.stdout: cephfs-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 222 k 2024-04-24T22:57:13.179 INFO:teuthology.orchestra.run.smithi121.stdout: cephfs-shell x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 27 k 2024-04-24T22:57:13.179 INFO:teuthology.orchestra.run.smithi121.stdout: cephfs-top noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 18 k 2024-04-24T22:57:13.179 INFO:teuthology.orchestra.run.smithi121.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-24T22:57:13.179 INFO:teuthology.orchestra.run.smithi121.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-24T22:57:13.179 INFO:teuthology.orchestra.run.smithi121.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-24T22:57:13.179 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 32 k 2024-04-24T22:57:13.179 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 745 k 2024-04-24T22:57:13.180 INFO:teuthology.orchestra.run.smithi121.stdout: librados-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 127 k 2024-04-24T22:57:13.180 INFO:teuthology.orchestra.run.smithi121.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-24T22:57:13.180 INFO:teuthology.orchestra.run.smithi121.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-24T22:57:13.180 INFO:teuthology.orchestra.run.smithi121.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-24T22:57:13.180 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-24T22:57:13.180 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-24T22:57:13.180 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-24T22:57:13.180 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cephfs x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 162 k 2024-04-24T22:57:13.180 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rados x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 321 k 2024-04-24T22:57:13.180 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 299 k 2024-04-24T22:57:13.180 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 99 k 2024-04-24T22:57:13.180 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 85 k 2024-04-24T22:57:13.180 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T22:57:13.180 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-nbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 171 k 2024-04-24T22:57:13.181 INFO:teuthology.orchestra.run.smithi121.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-24T22:57:13.181 INFO:teuthology.orchestra.run.smithi121.stdout:Upgrading: 2024-04-24T22:57:13.181 INFO:teuthology.orchestra.run.smithi121.stdout: librados2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.4 M 2024-04-24T22:57:13.181 INFO:teuthology.orchestra.run.smithi121.stdout: librbd1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T22:57:13.181 INFO:teuthology.orchestra.run.smithi121.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-24T22:57:13.181 INFO:teuthology.orchestra.run.smithi121.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-24T22:57:13.181 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2024-04-24T22:57:13.181 INFO:teuthology.orchestra.run.smithi121.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-24T22:57:13.181 INFO:teuthology.orchestra.run.smithi121.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-24T22:57:13.181 INFO:teuthology.orchestra.run.smithi121.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-24T22:57:13.181 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 21 M 2024-04-24T22:57:13.181 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 32 k 2024-04-24T22:57:13.181 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mds x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 2.4 M 2024-04-24T22:57:13.181 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 249 k 2024-04-24T22:57:13.181 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mon x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 4.7 M 2024-04-24T22:57:13.182 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-osd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 17 M 2024-04-24T22:57:13.182 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 15 k 2024-04-24T22:57:13.182 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-selinux x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 25 k 2024-04-24T22:57:13.182 INFO:teuthology.orchestra.run.smithi121.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-24T22:57:13.182 INFO:teuthology.orchestra.run.smithi121.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-24T22:57:13.182 INFO:teuthology.orchestra.run.smithi121.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-24T22:57:13.182 INFO:teuthology.orchestra.run.smithi121.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-24T22:57:13.182 INFO:teuthology.orchestra.run.smithi121.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-24T22:57:13.182 INFO:teuthology.orchestra.run.smithi121.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-24T22:57:13.182 INFO:teuthology.orchestra.run.smithi121.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-24T22:57:13.182 INFO:teuthology.orchestra.run.smithi121.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-24T22:57:13.182 INFO:teuthology.orchestra.run.smithi121.stdout: libcephsqlite x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 163 k 2024-04-24T22:57:13.182 INFO:teuthology.orchestra.run.smithi121.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-24T22:57:13.183 INFO:teuthology.orchestra.run.smithi121.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-24T22:57:13.183 INFO:teuthology.orchestra.run.smithi121.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-24T22:57:13.183 INFO:teuthology.orchestra.run.smithi121.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-24T22:57:13.183 INFO:teuthology.orchestra.run.smithi121.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-24T22:57:13.183 INFO:teuthology.orchestra.run.smithi121.stdout: libradosstriper1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 501 k 2024-04-24T22:57:13.183 INFO:teuthology.orchestra.run.smithi121.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-24T22:57:13.183 INFO:teuthology.orchestra.run.smithi121.stdout: librgw2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.3 M 2024-04-24T22:57:13.183 INFO:teuthology.orchestra.run.smithi121.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-24T22:57:13.183 INFO:teuthology.orchestra.run.smithi121.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-24T22:57:13.183 INFO:teuthology.orchestra.run.smithi121.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-24T22:57:13.183 INFO:teuthology.orchestra.run.smithi121.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-24T22:57:13.183 INFO:teuthology.orchestra.run.smithi121.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-24T22:57:13.183 INFO:teuthology.orchestra.run.smithi121.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-24T22:57:13.183 INFO:teuthology.orchestra.run.smithi121.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-24T22:57:13.183 INFO:teuthology.orchestra.run.smithi121.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-24T22:57:13.184 INFO:teuthology.orchestra.run.smithi121.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-24T22:57:13.184 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-24T22:57:13.184 INFO:teuthology.orchestra.run.smithi121.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-24T22:57:13.184 INFO:teuthology.orchestra.run.smithi121.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-24T22:57:13.184 INFO:teuthology.orchestra.run.smithi121.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-24T22:57:13.184 INFO:teuthology.orchestra.run.smithi121.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-24T22:57:13.184 INFO:teuthology.orchestra.run.smithi121.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-24T22:57:13.184 INFO:teuthology.orchestra.run.smithi121.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-24T22:57:13.184 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-24T22:57:13.184 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-argparse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 45 k 2024-04-24T22:57:13.184 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 133 k 2024-04-24T22:57:13.184 INFO:teuthology.orchestra.run.smithi121.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-24T22:57:13.184 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-24T22:57:13.184 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-24T22:57:13.185 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-24T22:57:13.185 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-24T22:57:13.185 INFO:teuthology.orchestra.run.smithi121.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-24T22:57:13.185 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-24T22:57:13.185 INFO:teuthology.orchestra.run.smithi121.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-24T22:57:13.185 INFO:teuthology.orchestra.run.smithi121.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-24T22:57:13.185 INFO:teuthology.orchestra.run.smithi121.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-24T22:57:13.185 INFO:teuthology.orchestra.run.smithi121.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-24T22:57:13.185 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-24T22:57:13.185 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-24T22:57:13.185 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-24T22:57:13.185 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-24T22:57:13.185 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-24T22:57:13.185 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-24T22:57:13.185 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-24T22:57:13.186 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-24T22:57:13.186 INFO:teuthology.orchestra.run.smithi121.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-24T22:57:13.186 INFO:teuthology.orchestra.run.smithi121.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-24T22:57:13.186 INFO:teuthology.orchestra.run.smithi121.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-24T22:57:13.186 INFO:teuthology.orchestra.run.smithi121.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-24T22:57:13.186 INFO:teuthology.orchestra.run.smithi121.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-24T22:57:13.186 INFO:teuthology.orchestra.run.smithi121.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-24T22:57:13.186 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-24T22:57:13.186 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-24T22:57:13.186 INFO:teuthology.orchestra.run.smithi121.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-24T22:57:13.186 INFO:teuthology.orchestra.run.smithi121.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-24T22:57:13.187 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-24T22:57:13.187 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-24T22:57:13.187 INFO:teuthology.orchestra.run.smithi121.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-24T22:57:13.187 INFO:teuthology.orchestra.run.smithi121.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-24T22:57:13.187 INFO:teuthology.orchestra.run.smithi121.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-24T22:57:13.187 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-24T22:57:13.187 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-24T22:57:13.187 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-24T22:57:13.188 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-24T22:57:13.188 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-24T22:57:13.188 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-24T22:57:13.188 INFO:teuthology.orchestra.run.smithi121.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-24T22:57:13.188 INFO:teuthology.orchestra.run.smithi121.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-24T22:57:13.188 INFO:teuthology.orchestra.run.smithi121.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-24T22:57:13.188 INFO:teuthology.orchestra.run.smithi121.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-24T22:57:13.188 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-24T22:57:13.188 INFO:teuthology.orchestra.run.smithi121.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-24T22:57:13.188 INFO:teuthology.orchestra.run.smithi121.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-24T22:57:13.188 INFO:teuthology.orchestra.run.smithi121.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-24T22:57:13.189 INFO:teuthology.orchestra.run.smithi121.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-24T22:57:13.189 INFO:teuthology.orchestra.run.smithi121.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-24T22:57:13.189 INFO:teuthology.orchestra.run.smithi121.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-24T22:57:13.189 INFO:teuthology.orchestra.run.smithi121.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-24T22:57:13.189 INFO:teuthology.orchestra.run.smithi121.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-24T22:57:13.189 INFO:teuthology.orchestra.run.smithi121.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-24T22:57:13.189 INFO:teuthology.orchestra.run.smithi121.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-24T22:57:13.189 INFO:teuthology.orchestra.run.smithi121.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-24T22:57:13.189 INFO:teuthology.orchestra.run.smithi121.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-24T22:57:13.189 INFO:teuthology.orchestra.run.smithi121.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-24T22:57:13.189 INFO:teuthology.orchestra.run.smithi121.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-24T22:57:13.189 INFO:teuthology.orchestra.run.smithi121.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-24T22:57:13.189 INFO:teuthology.orchestra.run.smithi121.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-24T22:57:13.189 INFO:teuthology.orchestra.run.smithi121.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-24T22:57:13.190 INFO:teuthology.orchestra.run.smithi121.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-24T22:57:13.190 INFO:teuthology.orchestra.run.smithi121.stdout:Installing weak dependencies: 2024-04-24T22:57:13.190 INFO:teuthology.orchestra.run.smithi121.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-24T22:57:13.190 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-24T22:57:13.190 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-04-24T22:57:13.190 INFO:teuthology.orchestra.run.smithi121.stdout:======================================================================================= 2024-04-24T22:57:13.190 INFO:teuthology.orchestra.run.smithi121.stdout:Install 148 Packages 2024-04-24T22:57:13.190 INFO:teuthology.orchestra.run.smithi121.stdout:Upgrade 4 Packages 2024-04-24T22:57:13.190 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-24T22:57:13.191 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 220 M 2024-04-24T22:57:13.191 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-04-24T22:57:14.031 INFO:teuthology.orchestra.run.smithi121.stdout:(1/152): ceph-19.0.0-2445.gd2c45d97.el9.x86_64. 27 kB/s | 6.6 kB 00:00 2024-04-24T22:57:14.247 INFO:teuthology.orchestra.run.smithi121.stdout:(2/152): ceph-fuse-19.0.0-2445.gd2c45d97.el9.x8 4.0 MB/s | 890 kB 00:00 2024-04-24T22:57:14.323 INFO:teuthology.orchestra.run.smithi121.stdout:(3/152): ceph-immutable-object-cache-19.0.0-244 1.9 MB/s | 145 kB 00:00 2024-04-24T22:57:14.664 INFO:teuthology.orchestra.run.smithi121.stdout:(4/152): ceph-mds-19.0.0-2445.gd2c45d97.el9.x86 6.9 MB/s | 2.4 MB 00:00 2024-04-24T22:57:14.898 INFO:teuthology.orchestra.run.smithi121.stdout:(5/152): ceph-mgr-19.0.0-2445.gd2c45d97.el9.x86 6.4 MB/s | 1.5 MB 00:00 2024-04-24T22:57:15.188 INFO:teuthology.orchestra.run.smithi032.stdout:lab-extras 37 kB/s | 1.7 kB 00:00 2024-04-24T22:57:15.482 INFO:teuthology.orchestra.run.smithi121.stdout:(6/152): ceph-mon-19.0.0-2445.gd2c45d97.el9.x86 8.0 MB/s | 4.7 MB 00:00 2024-04-24T22:57:17.390 INFO:teuthology.orchestra.run.smithi032.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T22:57:17.390 INFO:teuthology.orchestra.run.smithi032.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T22:57:17.402 INFO:teuthology.orchestra.run.smithi032.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-24T22:57:17.402 INFO:teuthology.orchestra.run.smithi032.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-24T22:57:17.403 INFO:teuthology.orchestra.run.smithi032.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-24T22:57:17.404 INFO:teuthology.orchestra.run.smithi032.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-24T22:57:17.406 INFO:teuthology.orchestra.run.smithi032.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-24T22:57:17.408 INFO:teuthology.orchestra.run.smithi032.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-24T22:57:17.409 INFO:teuthology.orchestra.run.smithi032.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-24T22:57:17.474 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-04-24T22:57:17.483 INFO:teuthology.orchestra.run.smithi032.stdout:======================================================================================= 2024-04-24T22:57:17.483 INFO:teuthology.orchestra.run.smithi032.stdout: Package Arch Version Repository Size 2024-04-24T22:57:17.483 INFO:teuthology.orchestra.run.smithi032.stdout:======================================================================================= 2024-04-24T22:57:17.483 INFO:teuthology.orchestra.run.smithi032.stdout:Installing: 2024-04-24T22:57:17.483 INFO:teuthology.orchestra.run.smithi032.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-24T22:57:17.483 INFO:teuthology.orchestra.run.smithi032.stdout: ceph x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 6.6 k 2024-04-24T22:57:17.483 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-base x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.4 M 2024-04-24T22:57:17.483 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 890 k 2024-04-24T22:57:17.483 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 145 k 2024-04-24T22:57:17.483 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 1.5 M 2024-04-24T22:57:17.484 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 141 k 2024-04-24T22:57:17.484 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 3.5 M 2024-04-24T22:57:17.484 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 7.4 M 2024-04-24T22:57:17.484 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-rook noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 49 k 2024-04-24T22:57:17.484 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-radosgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 10 M 2024-04-24T22:57:17.484 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-test x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 48 M 2024-04-24T22:57:17.484 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-volume noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 264 k 2024-04-24T22:57:17.484 INFO:teuthology.orchestra.run.smithi032.stdout: cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 762 k 2024-04-24T22:57:17.484 INFO:teuthology.orchestra.run.smithi032.stdout: cephfs-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 222 k 2024-04-24T22:57:17.484 INFO:teuthology.orchestra.run.smithi032.stdout: cephfs-shell x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 27 k 2024-04-24T22:57:17.484 INFO:teuthology.orchestra.run.smithi032.stdout: cephfs-top noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 18 k 2024-04-24T22:57:17.484 INFO:teuthology.orchestra.run.smithi032.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-24T22:57:17.484 INFO:teuthology.orchestra.run.smithi032.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-24T22:57:17.484 INFO:teuthology.orchestra.run.smithi032.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-24T22:57:17.484 INFO:teuthology.orchestra.run.smithi032.stdout: libcephfs-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 32 k 2024-04-24T22:57:17.485 INFO:teuthology.orchestra.run.smithi032.stdout: libcephfs2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 745 k 2024-04-24T22:57:17.485 INFO:teuthology.orchestra.run.smithi032.stdout: librados-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 127 k 2024-04-24T22:57:17.485 INFO:teuthology.orchestra.run.smithi032.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-24T22:57:17.485 INFO:teuthology.orchestra.run.smithi032.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-24T22:57:17.485 INFO:teuthology.orchestra.run.smithi032.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-24T22:57:17.485 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-24T22:57:17.485 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-24T22:57:17.485 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-24T22:57:17.485 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cephfs x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 162 k 2024-04-24T22:57:17.485 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rados x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 321 k 2024-04-24T22:57:17.485 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 299 k 2024-04-24T22:57:17.485 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 99 k 2024-04-24T22:57:17.486 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 85 k 2024-04-24T22:57:17.486 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T22:57:17.486 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-nbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 171 k 2024-04-24T22:57:17.486 INFO:teuthology.orchestra.run.smithi032.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-24T22:57:17.486 INFO:teuthology.orchestra.run.smithi032.stdout:Upgrading: 2024-04-24T22:57:17.486 INFO:teuthology.orchestra.run.smithi032.stdout: librados2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.4 M 2024-04-24T22:57:17.486 INFO:teuthology.orchestra.run.smithi032.stdout: librbd1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T22:57:17.486 INFO:teuthology.orchestra.run.smithi032.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-24T22:57:17.486 INFO:teuthology.orchestra.run.smithi032.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-24T22:57:17.486 INFO:teuthology.orchestra.run.smithi032.stdout:Installing dependencies: 2024-04-24T22:57:17.486 INFO:teuthology.orchestra.run.smithi032.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-24T22:57:17.486 INFO:teuthology.orchestra.run.smithi032.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-24T22:57:17.487 INFO:teuthology.orchestra.run.smithi032.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-24T22:57:17.487 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 21 M 2024-04-24T22:57:17.487 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 32 k 2024-04-24T22:57:17.487 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mds x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 2.4 M 2024-04-24T22:57:17.487 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 249 k 2024-04-24T22:57:17.487 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mon x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 4.7 M 2024-04-24T22:57:17.487 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-osd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 17 M 2024-04-24T22:57:17.487 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 15 k 2024-04-24T22:57:17.487 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-selinux x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 25 k 2024-04-24T22:57:17.487 INFO:teuthology.orchestra.run.smithi032.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-24T22:57:17.487 INFO:teuthology.orchestra.run.smithi032.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-24T22:57:17.487 INFO:teuthology.orchestra.run.smithi032.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-24T22:57:17.487 INFO:teuthology.orchestra.run.smithi032.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-24T22:57:17.487 INFO:teuthology.orchestra.run.smithi032.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-24T22:57:17.488 INFO:teuthology.orchestra.run.smithi032.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-24T22:57:17.488 INFO:teuthology.orchestra.run.smithi032.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-24T22:57:17.488 INFO:teuthology.orchestra.run.smithi032.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-24T22:57:17.488 INFO:teuthology.orchestra.run.smithi032.stdout: libcephsqlite x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 163 k 2024-04-24T22:57:17.488 INFO:teuthology.orchestra.run.smithi032.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-24T22:57:17.488 INFO:teuthology.orchestra.run.smithi032.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-24T22:57:17.488 INFO:teuthology.orchestra.run.smithi032.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-24T22:57:17.488 INFO:teuthology.orchestra.run.smithi032.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-24T22:57:17.488 INFO:teuthology.orchestra.run.smithi032.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-24T22:57:17.488 INFO:teuthology.orchestra.run.smithi032.stdout: libradosstriper1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 501 k 2024-04-24T22:57:17.488 INFO:teuthology.orchestra.run.smithi032.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-24T22:57:17.488 INFO:teuthology.orchestra.run.smithi032.stdout: librgw2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.3 M 2024-04-24T22:57:17.488 INFO:teuthology.orchestra.run.smithi032.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-24T22:57:17.488 INFO:teuthology.orchestra.run.smithi032.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-24T22:57:17.489 INFO:teuthology.orchestra.run.smithi032.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-24T22:57:17.489 INFO:teuthology.orchestra.run.smithi032.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-24T22:57:17.489 INFO:teuthology.orchestra.run.smithi032.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-24T22:57:17.489 INFO:teuthology.orchestra.run.smithi032.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-24T22:57:17.489 INFO:teuthology.orchestra.run.smithi032.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-24T22:57:17.489 INFO:teuthology.orchestra.run.smithi032.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-24T22:57:17.489 INFO:teuthology.orchestra.run.smithi032.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-24T22:57:17.489 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-24T22:57:17.489 INFO:teuthology.orchestra.run.smithi032.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-24T22:57:17.489 INFO:teuthology.orchestra.run.smithi032.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-24T22:57:17.489 INFO:teuthology.orchestra.run.smithi032.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-24T22:57:17.489 INFO:teuthology.orchestra.run.smithi032.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-24T22:57:17.489 INFO:teuthology.orchestra.run.smithi032.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-24T22:57:17.489 INFO:teuthology.orchestra.run.smithi032.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-24T22:57:17.490 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-24T22:57:17.490 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-argparse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 45 k 2024-04-24T22:57:17.490 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 133 k 2024-04-24T22:57:17.490 INFO:teuthology.orchestra.run.smithi032.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-24T22:57:17.490 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-24T22:57:17.490 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-24T22:57:17.490 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-24T22:57:17.490 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-24T22:57:17.490 INFO:teuthology.orchestra.run.smithi032.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-24T22:57:17.490 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-24T22:57:17.490 INFO:teuthology.orchestra.run.smithi032.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-24T22:57:17.491 INFO:teuthology.orchestra.run.smithi032.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-24T22:57:17.491 INFO:teuthology.orchestra.run.smithi032.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-24T22:57:17.491 INFO:teuthology.orchestra.run.smithi032.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-24T22:57:17.491 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-24T22:57:17.491 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-24T22:57:17.491 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-24T22:57:17.491 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-24T22:57:17.491 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-24T22:57:17.491 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-24T22:57:17.491 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-24T22:57:17.491 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-24T22:57:17.491 INFO:teuthology.orchestra.run.smithi032.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-24T22:57:17.491 INFO:teuthology.orchestra.run.smithi032.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-24T22:57:17.492 INFO:teuthology.orchestra.run.smithi032.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-24T22:57:17.492 INFO:teuthology.orchestra.run.smithi032.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-24T22:57:17.492 INFO:teuthology.orchestra.run.smithi032.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-24T22:57:17.492 INFO:teuthology.orchestra.run.smithi032.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-24T22:57:17.492 INFO:teuthology.orchestra.run.smithi032.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-24T22:57:17.492 INFO:teuthology.orchestra.run.smithi032.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-24T22:57:17.492 INFO:teuthology.orchestra.run.smithi032.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-24T22:57:17.492 INFO:teuthology.orchestra.run.smithi032.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-24T22:57:17.492 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-24T22:57:17.492 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-24T22:57:17.492 INFO:teuthology.orchestra.run.smithi032.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-24T22:57:17.492 INFO:teuthology.orchestra.run.smithi032.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-24T22:57:17.492 INFO:teuthology.orchestra.run.smithi032.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-24T22:57:17.492 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-24T22:57:17.493 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-24T22:57:17.493 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-24T22:57:17.493 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-24T22:57:17.493 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-24T22:57:17.493 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-24T22:57:17.493 INFO:teuthology.orchestra.run.smithi032.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-24T22:57:17.493 INFO:teuthology.orchestra.run.smithi032.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-24T22:57:17.493 INFO:teuthology.orchestra.run.smithi032.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-24T22:57:17.493 INFO:teuthology.orchestra.run.smithi032.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-24T22:57:17.493 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-24T22:57:17.493 INFO:teuthology.orchestra.run.smithi032.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-24T22:57:17.493 INFO:teuthology.orchestra.run.smithi032.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-24T22:57:17.493 INFO:teuthology.orchestra.run.smithi032.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-24T22:57:17.493 INFO:teuthology.orchestra.run.smithi032.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-24T22:57:17.494 INFO:teuthology.orchestra.run.smithi032.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-24T22:57:17.494 INFO:teuthology.orchestra.run.smithi032.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-24T22:57:17.494 INFO:teuthology.orchestra.run.smithi032.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-24T22:57:17.494 INFO:teuthology.orchestra.run.smithi032.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-24T22:57:17.494 INFO:teuthology.orchestra.run.smithi032.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-24T22:57:17.494 INFO:teuthology.orchestra.run.smithi032.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-24T22:57:17.494 INFO:teuthology.orchestra.run.smithi032.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-24T22:57:17.494 INFO:teuthology.orchestra.run.smithi032.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-24T22:57:17.494 INFO:teuthology.orchestra.run.smithi032.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-24T22:57:17.494 INFO:teuthology.orchestra.run.smithi032.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-24T22:57:17.494 INFO:teuthology.orchestra.run.smithi032.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-24T22:57:17.494 INFO:teuthology.orchestra.run.smithi032.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-24T22:57:17.494 INFO:teuthology.orchestra.run.smithi032.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-24T22:57:17.495 INFO:teuthology.orchestra.run.smithi032.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-24T22:57:17.495 INFO:teuthology.orchestra.run.smithi032.stdout:Installing weak dependencies: 2024-04-24T22:57:17.495 INFO:teuthology.orchestra.run.smithi032.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-24T22:57:17.495 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:57:17.495 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction Summary 2024-04-24T22:57:17.495 INFO:teuthology.orchestra.run.smithi032.stdout:======================================================================================= 2024-04-24T22:57:17.495 INFO:teuthology.orchestra.run.smithi032.stdout:Install 148 Packages 2024-04-24T22:57:17.495 INFO:teuthology.orchestra.run.smithi032.stdout:Upgrade 4 Packages 2024-04-24T22:57:17.495 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:57:17.496 INFO:teuthology.orchestra.run.smithi032.stdout:Total download size: 220 M 2024-04-24T22:57:17.496 INFO:teuthology.orchestra.run.smithi032.stdout:Downloading Packages: 2024-04-24T22:57:17.748 INFO:teuthology.orchestra.run.smithi121.stdout:(7/152): ceph-osd-19.0.0-2445.gd2c45d97.el9.x86 7.4 MB/s | 17 MB 00:02 2024-04-24T22:57:17.940 INFO:teuthology.orchestra.run.smithi121.stdout:(8/152): ceph-common-19.0.0-2445.gd2c45d97.el9. 5.1 MB/s | 21 MB 00:04 2024-04-24T22:57:17.999 INFO:teuthology.orchestra.run.smithi121.stdout:(9/152): ceph-selinux-19.0.0-2445.gd2c45d97.el9 432 kB/s | 25 kB 00:00 2024-04-24T22:57:18.305 INFO:teuthology.orchestra.run.smithi032.stdout:(1/152): ceph-19.0.0-2445.gd2c45d97.el9.x86_64. 18 kB/s | 6.6 kB 00:00 2024-04-24T22:57:18.424 INFO:teuthology.orchestra.run.smithi121.stdout:(10/152): ceph-base-19.0.0-2445.gd2c45d97.el9.x 1.2 MB/s | 5.4 MB 00:04 2024-04-24T22:57:18.716 INFO:teuthology.orchestra.run.smithi121.stdout:(11/152): cephfs-mirror-19.0.0-2445.gd2c45d97.e 762 kB/s | 222 kB 00:00 2024-04-24T22:57:18.824 INFO:teuthology.orchestra.run.smithi121.stdout:(12/152): cephfs-shell-19.0.0-2445.gd2c45d97.el 250 kB/s | 27 kB 00:00 2024-04-24T22:57:18.967 INFO:teuthology.orchestra.run.smithi121.stdout:(13/152): libcephfs-devel-19.0.0-2445.gd2c45d97 224 kB/s | 32 kB 00:00 2024-04-24T22:57:19.325 INFO:teuthology.orchestra.run.smithi121.stdout:(14/152): ceph-radosgw-19.0.0-2445.gd2c45d97.el 6.5 MB/s | 10 MB 00:01 2024-04-24T22:57:19.347 INFO:teuthology.orchestra.run.smithi032.stdout:(2/152): ceph-fuse-19.0.0-2445.gd2c45d97.el9.x8 854 kB/s | 890 kB 00:01 2024-04-24T22:57:19.392 INFO:teuthology.orchestra.run.smithi121.stdout:(15/152): libcephsqlite-19.0.0-2445.gd2c45d97.e 2.4 MB/s | 163 kB 00:00 2024-04-24T22:57:19.468 INFO:teuthology.orchestra.run.smithi121.stdout:(16/152): librados-devel-19.0.0-2445.gd2c45d97. 1.6 MB/s | 127 kB 00:00 2024-04-24T22:57:19.531 INFO:teuthology.orchestra.run.smithi032.stdout:(3/152): ceph-immutable-object-cache-19.0.0-244 791 kB/s | 145 kB 00:00 2024-04-24T22:57:19.618 INFO:teuthology.orchestra.run.smithi121.stdout:(17/152): libradosstriper1-19.0.0-2445.gd2c45d9 3.3 MB/s | 501 kB 00:00 2024-04-24T22:57:19.727 INFO:teuthology.orchestra.run.smithi121.stdout:(18/152): libcephfs2-19.0.0-2445.gd2c45d97.el9. 981 kB/s | 745 kB 00:00 2024-04-24T22:57:19.860 INFO:teuthology.orchestra.run.smithi121.stdout:(19/152): python3-ceph-argparse-19.0.0-2445.gd2 338 kB/s | 45 kB 00:00 2024-04-24T22:57:20.119 INFO:teuthology.orchestra.run.smithi121.stdout:(20/152): python3-ceph-common-19.0.0-2445.gd2c4 515 kB/s | 133 kB 00:00 2024-04-24T22:57:20.361 INFO:teuthology.orchestra.run.smithi121.stdout:(21/152): python3-cephfs-19.0.0-2445.gd2c45d97. 671 kB/s | 162 kB 00:00 2024-04-24T22:57:20.590 INFO:teuthology.orchestra.run.smithi032.stdout:(4/152): ceph-base-19.0.0-2445.gd2c45d97.el9.x8 2.0 MB/s | 5.4 MB 00:02 2024-04-24T22:57:20.661 INFO:teuthology.orchestra.run.smithi121.stdout:(22/152): librgw2-19.0.0-2445.gd2c45d97.el9.x86 5.1 MB/s | 5.3 MB 00:01 2024-04-24T22:57:20.769 INFO:teuthology.orchestra.run.smithi121.stdout:(23/152): python3-rbd-19.0.0-2445.gd2c45d97.el9 2.7 MB/s | 299 kB 00:00 2024-04-24T22:57:20.811 INFO:teuthology.orchestra.run.smithi121.stdout:(24/152): python3-rados-19.0.0-2445.gd2c45d97.e 713 kB/s | 321 kB 00:00 2024-04-24T22:57:20.862 INFO:teuthology.orchestra.run.smithi121.stdout:(25/152): python3-rgw-19.0.0-2445.gd2c45d97.el9 1.1 MB/s | 99 kB 00:00 2024-04-24T22:57:20.931 INFO:teuthology.orchestra.run.smithi032.stdout:(5/152): ceph-mgr-19.0.0-2445.gd2c45d97.el9.x86 4.4 MB/s | 1.5 MB 00:00 2024-04-24T22:57:21.078 INFO:teuthology.orchestra.run.smithi121.stdout:(26/152): rbd-fuse-19.0.0-2445.gd2c45d97.el9.x8 319 kB/s | 85 kB 00:00 2024-04-24T22:57:21.395 INFO:teuthology.orchestra.run.smithi121.stdout:(27/152): rbd-nbd-19.0.0-2445.gd2c45d97.el9.x86 541 kB/s | 171 kB 00:00 2024-04-24T22:57:21.398 INFO:teuthology.orchestra.run.smithi032.stdout:(6/152): ceph-mds-19.0.0-2445.gd2c45d97.el9.x86 1.3 MB/s | 2.4 MB 00:01 2024-04-24T22:57:21.512 INFO:teuthology.orchestra.run.smithi121.stdout:(28/152): ceph-grafana-dashboards-19.0.0-2445.g 273 kB/s | 32 kB 00:00 2024-04-24T22:57:21.579 INFO:teuthology.orchestra.run.smithi121.stdout:(29/152): rbd-mirror-19.0.0-2445.gd2c45d97.el9. 4.3 MB/s | 3.1 MB 00:00 2024-04-24T22:57:21.730 INFO:teuthology.orchestra.run.smithi121.stdout:(30/152): ceph-mgr-cephadm-19.0.0-2445.gd2c45d9 651 kB/s | 141 kB 00:00 2024-04-24T22:57:22.323 INFO:teuthology.orchestra.run.smithi032.stdout:(7/152): ceph-common-19.0.0-2445.gd2c45d97.el9. 4.8 MB/s | 21 MB 00:04 2024-04-24T22:57:22.355 INFO:teuthology.orchestra.run.smithi121.stdout:(31/152): ceph-mgr-dashboard-19.0.0-2445.gd2c45 4.5 MB/s | 3.5 MB 00:00 2024-04-24T22:57:22.438 INFO:teuthology.orchestra.run.smithi121.stdout:(32/152): ceph-mgr-modules-core-19.0.0-2445.gd2 2.9 MB/s | 249 kB 00:00 2024-04-24T22:57:22.490 INFO:teuthology.orchestra.run.smithi032.stdout:(8/152): ceph-mon-19.0.0-2445.gd2c45d97.el9.x86 3.0 MB/s | 4.7 MB 00:01 2024-04-24T22:57:22.522 INFO:teuthology.orchestra.run.smithi121.stdout:(33/152): ceph-mgr-rook-19.0.0-2445.gd2c45d97.e 592 kB/s | 49 kB 00:00 2024-04-24T22:57:22.574 INFO:teuthology.orchestra.run.smithi032.stdout:(9/152): ceph-selinux-19.0.0-2445.gd2c45d97.el9 302 kB/s | 25 kB 00:00 2024-04-24T22:57:22.606 INFO:teuthology.orchestra.run.smithi121.stdout:(34/152): ceph-prometheus-alerts-19.0.0-2445.gd 184 kB/s | 15 kB 00:00 2024-04-24T22:57:22.714 INFO:teuthology.orchestra.run.smithi121.stdout:(35/152): ceph-volume-19.0.0-2445.gd2c45d97.el9 2.4 MB/s | 264 kB 00:00 2024-04-24T22:57:22.966 INFO:teuthology.orchestra.run.smithi121.stdout:(36/152): cephadm-19.0.0-2445.gd2c45d97.el9.noa 3.0 MB/s | 762 kB 00:00 2024-04-24T22:57:23.049 INFO:teuthology.orchestra.run.smithi121.stdout:(37/152): cephfs-top-19.0.0-2445.gd2c45d97.el9. 219 kB/s | 18 kB 00:00 2024-04-24T22:57:23.224 INFO:teuthology.orchestra.run.smithi121.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 460 kB/s | 80 kB 00:00 2024-04-24T22:57:23.690 INFO:teuthology.orchestra.run.smithi121.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.7 MB/s | 803 kB 00:00 2024-04-24T22:57:23.691 INFO:teuthology.orchestra.run.smithi032.stdout:(10/152): ceph-radosgw-19.0.0-2445.gd2c45d97.el 7.5 MB/s | 10 MB 00:01 2024-04-24T22:57:23.766 INFO:teuthology.orchestra.run.smithi032.stdout:(11/152): cephfs-mirror-19.0.0-2445.gd2c45d97.e 2.9 MB/s | 222 kB 00:00 2024-04-24T22:57:23.816 INFO:teuthology.orchestra.run.smithi121.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.5 MB/s | 195 kB 00:00 2024-04-24T22:57:23.825 INFO:teuthology.orchestra.run.smithi032.stdout:(12/152): cephfs-shell-19.0.0-2445.gd2c45d97.el 463 kB/s | 27 kB 00:00 2024-04-24T22:57:23.892 INFO:teuthology.orchestra.run.smithi032.stdout:(13/152): libcephfs-devel-19.0.0-2445.gd2c45d97 477 kB/s | 32 kB 00:00 2024-04-24T22:57:23.941 INFO:teuthology.orchestra.run.smithi121.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.0 MB/s | 253 kB 00:00 2024-04-24T22:57:24.034 INFO:teuthology.orchestra.run.smithi032.stdout:(14/152): libcephfs2-19.0.0-2445.gd2c45d97.el9. 5.1 MB/s | 745 kB 00:00 2024-04-24T22:57:24.109 INFO:teuthology.orchestra.run.smithi032.stdout:(15/152): libcephsqlite-19.0.0-2445.gd2c45d97.e 2.1 MB/s | 163 kB 00:00 2024-04-24T22:57:24.185 INFO:teuthology.orchestra.run.smithi032.stdout:(16/152): librados-devel-19.0.0-2445.gd2c45d97. 1.7 MB/s | 127 kB 00:00 2024-04-24T22:57:24.258 INFO:teuthology.orchestra.run.smithi121.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 3.9 MB/s | 1.2 MB 00:00 2024-04-24T22:57:24.316 INFO:teuthology.orchestra.run.smithi121.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-24T22:57:24.318 INFO:teuthology.orchestra.run.smithi032.stdout:(17/152): libradosstriper1-19.0.0-2445.gd2c45d9 3.7 MB/s | 501 kB 00:00 2024-04-24T22:57:24.375 INFO:teuthology.orchestra.run.smithi121.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-04-24T22:57:24.792 INFO:teuthology.orchestra.run.smithi121.stdout:(45/152): ceph-test-19.0.0-2445.gd2c45d97.el9.x 7.1 MB/s | 48 MB 00:06 2024-04-24T22:57:24.817 INFO:teuthology.orchestra.run.smithi121.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 282 kB/s | 125 kB 00:00 2024-04-24T22:57:24.952 INFO:teuthology.orchestra.run.smithi032.stdout:(18/152): librgw2-19.0.0-2445.gd2c45d97.el9.x86 8.4 MB/s | 5.3 MB 00:00 2024-04-24T22:57:24.984 INFO:teuthology.orchestra.run.smithi121.stdout:(47/152): automake-1.16.2-8.el9.noarch.rpm 4.0 MB/s | 689 kB 00:00 2024-04-24T22:57:25.011 INFO:teuthology.orchestra.run.smithi032.stdout:(19/152): python3-ceph-argparse-19.0.0-2445.gd2 778 kB/s | 45 kB 00:00 2024-04-24T22:57:25.059 INFO:teuthology.orchestra.run.smithi121.stdout:(48/152): bison-3.7.4-5.el9.x86_64.rpm 12 MB/s | 943 kB 00:00 2024-04-24T22:57:25.069 INFO:teuthology.orchestra.run.smithi032.stdout:(20/152): python3-ceph-common-19.0.0-2445.gd2c4 2.2 MB/s | 133 kB 00:00 2024-04-24T22:57:25.085 INFO:teuthology.orchestra.run.smithi121.stdout:(49/152): python3-urllib3-1.26.5-5.el9.noarch.r 735 kB/s | 215 kB 00:00 2024-04-24T22:57:25.111 INFO:teuthology.orchestra.run.smithi121.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-04-24T22:57:25.136 INFO:teuthology.orchestra.run.smithi032.stdout:(21/152): python3-cephfs-19.0.0-2445.gd2c45d97. 2.4 MB/s | 162 kB 00:00 2024-04-24T22:57:25.152 INFO:teuthology.orchestra.run.smithi121.stdout:(51/152): flex-2.6.4-9.el9.x86_64.rpm 7.4 MB/s | 313 kB 00:00 2024-04-24T22:57:25.177 INFO:teuthology.orchestra.run.smithi121.stdout:(52/152): elfutils-libelf-devel-0.190-2.el9.x86 233 kB/s | 21 kB 00:00 2024-04-24T22:57:25.203 INFO:teuthology.orchestra.run.smithi121.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 650 kB/s | 33 kB 00:00 2024-04-24T22:57:25.203 INFO:teuthology.orchestra.run.smithi032.stdout:(22/152): python3-rados-19.0.0-2445.gd2c45d97.e 4.7 MB/s | 321 kB 00:00 2024-04-24T22:57:25.244 INFO:teuthology.orchestra.run.smithi121.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 440 kB/s | 18 kB 00:00 2024-04-24T22:57:25.270 INFO:teuthology.orchestra.run.smithi032.stdout:(23/152): python3-rbd-19.0.0-2445.gd2c45d97.el9 4.4 MB/s | 299 kB 00:00 2024-04-24T22:57:25.287 INFO:teuthology.orchestra.run.smithi121.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 267 kB/s | 11 kB 00:00 2024-04-24T22:57:25.328 INFO:teuthology.orchestra.run.smithi121.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.8 MB/s | 160 kB 00:00 2024-04-24T22:57:25.329 INFO:teuthology.orchestra.run.smithi032.stdout:(24/152): python3-rgw-19.0.0-2445.gd2c45d97.el9 1.7 MB/s | 99 kB 00:00 2024-04-24T22:57:25.387 INFO:teuthology.orchestra.run.smithi121.stdout:(57/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 14 MB/s | 3.0 MB 00:00 2024-04-24T22:57:25.388 INFO:teuthology.orchestra.run.smithi032.stdout:(25/152): rbd-fuse-19.0.0-2445.gd2c45d97.el9.x8 1.4 MB/s | 85 kB 00:00 2024-04-24T22:57:25.412 INFO:teuthology.orchestra.run.smithi121.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 541 kB/s | 45 kB 00:00 2024-04-24T22:57:25.446 INFO:teuthology.orchestra.run.smithi121.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-04-24T22:57:25.472 INFO:teuthology.orchestra.run.smithi121.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 9.7 MB/s | 582 kB 00:00 2024-04-24T22:57:25.513 INFO:teuthology.orchestra.run.smithi121.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 380 kB/s | 25 kB 00:00 2024-04-24T22:57:25.547 INFO:teuthology.orchestra.run.smithi121.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 630 kB/s | 47 kB 00:00 2024-04-24T22:57:25.572 INFO:teuthology.orchestra.run.smithi121.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-04-24T22:57:25.606 INFO:teuthology.orchestra.run.smithi121.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 188 kB 00:00 2024-04-24T22:57:25.631 INFO:teuthology.orchestra.run.smithi121.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 598 kB/s | 35 kB 00:00 2024-04-24T22:57:25.731 INFO:teuthology.orchestra.run.smithi121.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 38 MB/s | 4.7 MB 00:00 2024-04-24T22:57:25.771 INFO:teuthology.orchestra.run.smithi032.stdout:(26/152): rbd-mirror-19.0.0-2445.gd2c45d97.el9. 8.1 MB/s | 3.1 MB 00:00 2024-04-24T22:57:25.798 INFO:teuthology.orchestra.run.smithi121.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 27 MB/s | 4.4 MB 00:00 2024-04-24T22:57:25.830 INFO:teuthology.orchestra.run.smithi032.stdout:(27/152): rbd-nbd-19.0.0-2445.gd2c45d97.el9.x86 2.9 MB/s | 171 kB 00:00 2024-04-24T22:57:25.841 INFO:teuthology.orchestra.run.smithi121.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-04-24T22:57:25.872 INFO:teuthology.orchestra.run.smithi032.stdout:(28/152): ceph-grafana-dashboards-19.0.0-2445.g 765 kB/s | 32 kB 00:00 2024-04-24T22:57:25.874 INFO:teuthology.orchestra.run.smithi121.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 911 kB 00:00 2024-04-24T22:57:25.899 INFO:teuthology.orchestra.run.smithi121.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-04-24T22:57:25.950 INFO:teuthology.orchestra.run.smithi121.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 20 MB/s | 1.0 MB 00:00 2024-04-24T22:57:25.955 INFO:teuthology.orchestra.run.smithi032.stdout:(29/152): ceph-mgr-cephadm-19.0.0-2445.gd2c45d9 1.7 MB/s | 141 kB 00:00 2024-04-24T22:57:26.025 INFO:teuthology.orchestra.run.smithi121.stdout:(72/152): postgresql-server-13.14-1.el9.x86_64. 38 MB/s | 5.8 MB 00:00 2024-04-24T22:57:26.068 INFO:teuthology.orchestra.run.smithi121.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 740 kB/s | 87 kB 00:00 2024-04-24T22:57:26.117 INFO:teuthology.orchestra.run.smithi121.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 4.9 MB/s | 244 kB 00:00 2024-04-24T22:57:26.209 INFO:teuthology.orchestra.run.smithi121.stdout:(75/152): python3-babel-2.9.1-2.el9.noarch.rpm 32 MB/s | 6.0 MB 00:00 2024-04-24T22:57:26.234 INFO:teuthology.orchestra.run.smithi121.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.1 MB/s | 248 kB 00:00 2024-04-24T22:57:26.260 INFO:teuthology.orchestra.run.smithi121.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 513 kB/s | 26 kB 00:00 2024-04-24T22:57:26.285 INFO:teuthology.orchestra.run.smithi121.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 380 kB/s | 19 kB 00:00 2024-04-24T22:57:26.310 INFO:teuthology.orchestra.run.smithi121.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-04-24T22:57:26.335 INFO:teuthology.orchestra.run.smithi121.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 694 kB/s | 35 kB 00:00 2024-04-24T22:57:26.377 INFO:teuthology.orchestra.run.smithi121.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.9 MB/s | 250 kB 00:00 2024-04-24T22:57:26.446 INFO:teuthology.orchestra.run.smithi121.stdout:(82/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 38 MB/s | 5.1 MB 00:00 2024-04-24T22:57:26.456 INFO:teuthology.orchestra.run.smithi032.stdout:(30/152): ceph-mgr-dashboard-19.0.0-2445.gd2c45 7.0 MB/s | 3.5 MB 00:00 2024-04-24T22:57:26.470 INFO:teuthology.orchestra.run.smithi121.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-04-24T22:57:26.495 INFO:teuthology.orchestra.run.smithi121.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-24T22:57:26.520 INFO:teuthology.orchestra.run.smithi121.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 841 kB/s | 42 kB 00:00 2024-04-24T22:57:26.546 INFO:teuthology.orchestra.run.smithi121.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-04-24T22:57:26.571 INFO:teuthology.orchestra.run.smithi121.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-24T22:57:26.596 INFO:teuthology.orchestra.run.smithi121.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-04-24T22:57:26.621 INFO:teuthology.orchestra.run.smithi121.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-24T22:57:26.650 INFO:teuthology.orchestra.run.smithi121.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-24T22:57:26.689 INFO:teuthology.orchestra.run.smithi121.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-04-24T22:57:26.733 INFO:teuthology.orchestra.run.smithi121.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-04-24T22:57:26.775 INFO:teuthology.orchestra.run.smithi121.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 5.2 MB/s | 221 kB 00:00 2024-04-24T22:57:26.817 INFO:teuthology.orchestra.run.smithi121.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 867 kB/s | 36 kB 00:00 2024-04-24T22:57:26.975 INFO:teuthology.orchestra.run.smithi121.stdout:(95/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 46 MB/s | 16 MB 00:00 2024-04-24T22:57:27.000 INFO:teuthology.orchestra.run.smithi121.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 258 kB/s | 47 kB 00:00 2024-04-24T22:57:27.026 INFO:teuthology.orchestra.run.smithi121.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-04-24T22:57:27.051 INFO:teuthology.orchestra.run.smithi121.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-24T22:57:27.076 INFO:teuthology.orchestra.run.smithi121.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-04-24T22:57:27.102 INFO:teuthology.orchestra.run.smithi121.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-24T22:57:27.127 INFO:teuthology.orchestra.run.smithi121.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 892 kB/s | 45 kB 00:00 2024-04-24T22:57:27.311 INFO:teuthology.orchestra.run.smithi121.stdout:(102/152): ceph-mgr-diskprediction-local-19.0.0 1.3 MB/s | 7.4 MB 00:05 2024-04-24T22:57:27.331 INFO:teuthology.orchestra.run.smithi032.stdout:(31/152): ceph-mgr-diskprediction-local-19.0.0- 8.4 MB/s | 7.4 MB 00:00 2024-04-24T22:57:27.398 INFO:teuthology.orchestra.run.smithi032.stdout:(32/152): ceph-mgr-modules-core-19.0.0-2445.gd2 3.7 MB/s | 249 kB 00:00 2024-04-24T22:57:27.427 INFO:teuthology.orchestra.run.smithi121.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 68 kB/s | 22 kB 00:00 2024-04-24T22:57:27.461 INFO:teuthology.orchestra.run.smithi032.stdout:(33/152): ceph-mgr-rook-19.0.0-2445.gd2c45d97.e 788 kB/s | 49 kB 00:00 2024-04-24T22:57:27.524 INFO:teuthology.orchestra.run.smithi032.stdout:(34/152): ceph-prometheus-alerts-19.0.0-2445.gd 246 kB/s | 15 kB 00:00 2024-04-24T22:57:27.599 INFO:teuthology.orchestra.run.smithi032.stdout:(35/152): ceph-volume-19.0.0-2445.gd2c45d97.el9 3.4 MB/s | 264 kB 00:00 2024-04-24T22:57:27.619 INFO:teuthology.orchestra.run.smithi121.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 1.7 MB/s | 863 kB 00:00 2024-04-24T22:57:27.716 INFO:teuthology.orchestra.run.smithi032.stdout:(36/152): cephadm-19.0.0-2445.gd2c45d97.el9.noa 6.4 MB/s | 762 kB 00:00 2024-04-24T22:57:27.761 INFO:teuthology.orchestra.run.smithi121.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.2 MB/s | 548 kB 00:00 2024-04-24T22:57:27.766 INFO:teuthology.orchestra.run.smithi032.stdout:(37/152): cephfs-top-19.0.0-2445.gd2c45d97.el9. 365 kB/s | 18 kB 00:00 2024-04-24T22:57:27.861 INFO:teuthology.orchestra.run.smithi121.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 712 kB/s | 309 kB 00:00 2024-04-24T22:57:27.890 INFO:teuthology.orchestra.run.smithi121.stdout:(107/152): grpc-data-1.46.7-10.el9.noarch.rpm 73 kB/s | 19 kB 00:00 2024-04-24T22:57:27.937 INFO:teuthology.orchestra.run.smithi121.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 345 kB/s | 26 kB 00:00 2024-04-24T22:57:27.958 INFO:teuthology.orchestra.run.smithi032.stdout:(38/152): ceph-osd-19.0.0-2445.gd2c45d97.el9.x8 2.6 MB/s | 17 MB 00:06 2024-04-24T22:57:28.013 INFO:teuthology.orchestra.run.smithi121.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 897 kB/s | 67 kB 00:00 2024-04-24T22:57:28.037 INFO:teuthology.orchestra.run.smithi121.stdout:(110/152): liboath-2.6.7-2.el9.x86_64.rpm 323 kB/s | 49 kB 00:00 2024-04-24T22:57:28.083 INFO:teuthology.orchestra.run.smithi032.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 253 kB/s | 80 kB 00:00 2024-04-24T22:57:28.134 INFO:teuthology.orchestra.run.smithi121.stdout:(111/152): libarrow-9.0.0-10.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-04-24T22:57:28.155 INFO:teuthology.orchestra.run.smithi121.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 1.0 MB/s | 151 kB 00:00 2024-04-24T22:57:28.213 INFO:teuthology.orchestra.run.smithi121.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 6.8 MB/s | 548 kB 00:00 2024-04-24T22:57:28.239 INFO:teuthology.orchestra.run.smithi121.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 520 kB/s | 43 kB 00:00 2024-04-24T22:57:28.242 INFO:teuthology.orchestra.run.smithi032.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 195 kB 00:00 2024-04-24T22:57:28.281 INFO:teuthology.orchestra.run.smithi121.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 482 kB/s | 32 kB 00:00 2024-04-24T22:57:28.306 INFO:teuthology.orchestra.run.smithi121.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 225 kB/s | 15 kB 00:00 2024-04-24T22:57:28.339 INFO:teuthology.orchestra.run.smithi121.stdout:(117/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.7 MB/s | 838 kB 00:00 2024-04-24T22:57:28.364 INFO:teuthology.orchestra.run.smithi121.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-04-24T22:57:28.409 INFO:teuthology.orchestra.run.smithi032.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.5 MB/s | 253 kB 00:00 2024-04-24T22:57:28.423 INFO:teuthology.orchestra.run.smithi121.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-04-24T22:57:28.448 INFO:teuthology.orchestra.run.smithi121.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 422 kB/s | 35 kB 00:00 2024-04-24T22:57:28.474 INFO:teuthology.orchestra.run.smithi121.stdout:(121/152): python3-cherrypy-18.6.1-2.el9.noarch 2.1 MB/s | 358 kB 00:00 2024-04-24T22:57:28.499 INFO:teuthology.orchestra.run.smithi121.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 2.7 MB/s | 206 kB 00:00 2024-04-24T22:57:28.534 INFO:teuthology.orchestra.run.smithi032.stdout:(42/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.4 MB/s | 803 kB 00:00 2024-04-24T22:57:28.566 INFO:teuthology.orchestra.run.smithi121.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 17 MB/s | 2.0 MB 00:00 2024-04-24T22:57:28.591 INFO:teuthology.orchestra.run.smithi121.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 116 kB/s | 11 kB 00:00 2024-04-24T22:57:28.617 INFO:teuthology.orchestra.run.smithi121.stdout:(125/152): python3-grpcio-tools-1.46.7-10.el9.x 1.0 MB/s | 144 kB 00:00 2024-04-24T22:57:28.626 INFO:teuthology.orchestra.run.smithi032.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 106 kB 00:00 2024-04-24T22:57:28.659 INFO:teuthology.orchestra.run.smithi121.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 192 kB/s | 18 kB 00:00 2024-04-24T22:57:28.677 INFO:teuthology.orchestra.run.smithi121.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 276 kB/s | 23 kB 00:00 2024-04-24T22:57:28.701 INFO:teuthology.orchestra.run.smithi121.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 232 kB/s | 19 kB 00:00 2024-04-24T22:57:28.718 INFO:teuthology.orchestra.run.smithi032.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 135 kB 00:00 2024-04-24T22:57:28.726 INFO:teuthology.orchestra.run.smithi121.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 294 kB/s | 20 kB 00:00 2024-04-24T22:57:28.793 INFO:teuthology.orchestra.run.smithi121.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 8.7 MB/s | 1.0 MB 00:00 2024-04-24T22:57:28.810 INFO:teuthology.orchestra.run.smithi032.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 1.3 MB/s | 125 kB 00:00 2024-04-24T22:57:28.818 INFO:teuthology.orchestra.run.smithi121.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 396 kB/s | 46 kB 00:00 2024-04-24T22:57:28.844 INFO:teuthology.orchestra.run.smithi121.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 673 kB/s | 79 kB 00:00 2024-04-24T22:57:28.869 INFO:teuthology.orchestra.run.smithi121.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 767 kB/s | 58 kB 00:00 2024-04-24T22:57:28.894 INFO:teuthology.orchestra.run.smithi121.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-04-24T22:57:28.911 INFO:teuthology.orchestra.run.smithi032.stdout:(46/152): python3-cryptography-36.0.1-4.el9.x86 2.4 MB/s | 1.2 MB 00:00 2024-04-24T22:57:28.919 INFO:teuthology.orchestra.run.smithi121.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 218 kB/s | 16 kB 00:00 2024-04-24T22:57:28.936 INFO:teuthology.orchestra.run.smithi032.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 1.7 MB/s | 215 kB 00:00 2024-04-24T22:57:28.945 INFO:teuthology.orchestra.run.smithi121.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-04-24T22:57:28.970 INFO:teuthology.orchestra.run.smithi121.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 374 kB/s | 28 kB 00:00 2024-04-24T22:57:29.020 INFO:teuthology.orchestra.run.smithi121.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 306 kB/s | 31 kB 00:00 2024-04-24T22:57:29.045 INFO:teuthology.orchestra.run.smithi121.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 787 kB/s | 59 kB 00:00 2024-04-24T22:57:29.079 INFO:teuthology.orchestra.run.smithi121.stdout:(140/152): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2024-04-24T22:57:29.104 INFO:teuthology.orchestra.run.smithi121.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 428 kB/s | 36 kB 00:00 2024-04-24T22:57:29.119 INFO:teuthology.orchestra.run.smithi032.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 3.2 MB/s | 689 kB 00:00 2024-04-24T22:57:29.130 INFO:teuthology.orchestra.run.smithi121.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 608 kB/s | 51 kB 00:00 2024-04-24T22:57:29.162 INFO:teuthology.orchestra.run.smithi032.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 4.1 MB/s | 943 kB 00:00 2024-04-24T22:57:29.171 INFO:teuthology.orchestra.run.smithi121.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-04-24T22:57:29.187 INFO:teuthology.orchestra.run.smithi032.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-04-24T22:57:29.197 INFO:teuthology.orchestra.run.smithi121.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 971 kB/s | 90 kB 00:00 2024-04-24T22:57:29.203 INFO:teuthology.orchestra.run.smithi032.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 516 kB/s | 21 kB 00:00 2024-04-24T22:57:29.222 INFO:teuthology.orchestra.run.smithi121.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 4.5 MB/s | 427 kB 00:00 2024-04-24T22:57:29.237 INFO:teuthology.orchestra.run.smithi032.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-04-24T22:57:29.247 INFO:teuthology.orchestra.run.smithi121.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 265 kB/s | 20 kB 00:00 2024-04-24T22:57:29.254 INFO:teuthology.orchestra.run.smithi032.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 650 kB/s | 33 kB 00:00 2024-04-24T22:57:29.272 INFO:teuthology.orchestra.run.smithi121.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-04-24T22:57:29.288 INFO:teuthology.orchestra.run.smithi032.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 546 kB/s | 18 kB 00:00 2024-04-24T22:57:29.338 INFO:teuthology.orchestra.run.smithi032.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-04-24T22:57:29.364 INFO:teuthology.orchestra.run.smithi121.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-24T22:57:29.389 INFO:teuthology.orchestra.run.smithi032.stdout:(56/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 20 MB/s | 3.0 MB 00:00 2024-04-24T22:57:29.414 INFO:teuthology.orchestra.run.smithi032.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 160 kB 00:00 2024-04-24T22:57:29.431 INFO:teuthology.orchestra.run.smithi032.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-04-24T22:57:29.464 INFO:teuthology.orchestra.run.smithi032.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 662 kB 00:00 2024-04-24T22:57:29.499 INFO:teuthology.orchestra.run.smithi032.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 8.5 MB/s | 582 kB 00:00 2024-04-24T22:57:29.515 INFO:teuthology.orchestra.run.smithi032.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 505 kB/s | 25 kB 00:00 2024-04-24T22:57:29.549 INFO:teuthology.orchestra.run.smithi032.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-04-24T22:57:29.574 INFO:teuthology.orchestra.run.smithi032.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-04-24T22:57:29.599 INFO:teuthology.orchestra.run.smithi032.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-04-24T22:57:29.625 INFO:teuthology.orchestra.run.smithi032.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 704 kB/s | 35 kB 00:00 2024-04-24T22:57:29.690 INFO:teuthology.orchestra.run.smithi121.stdout:(149/152): librados2-19.0.0-2445.gd2c45d97.el9. 7.7 MB/s | 3.4 MB 00:00 2024-04-24T22:57:29.775 INFO:teuthology.orchestra.run.smithi032.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 27 MB/s | 4.7 MB 00:00 2024-04-24T22:57:29.848 INFO:teuthology.orchestra.run.smithi121.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 2.8 MB/s | 1.3 MB 00:00 2024-04-24T22:57:29.850 INFO:teuthology.orchestra.run.smithi032.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-04-24T22:57:29.892 INFO:teuthology.orchestra.run.smithi032.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-04-24T22:57:30.318 INFO:teuthology.orchestra.run.smithi032.stdout:(69/152): ceph-test-19.0.0-2445.gd2c45d97.el9.x 6.2 MB/s | 48 MB 00:07 2024-04-24T22:57:30.343 INFO:teuthology.orchestra.run.smithi032.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 296 kB/s | 133 kB 00:00 2024-04-24T22:57:30.365 INFO:teuthology.orchestra.run.smithi121.stdout:(151/152): librbd1-19.0.0-2445.gd2c45d97.el9.x8 2.9 MB/s | 3.1 MB 00:01 2024-04-24T22:57:30.385 INFO:teuthology.orchestra.run.smithi032.stdout:(71/152): postgresql-contrib-13.14-1.el9.x86_64 1.7 MB/s | 911 kB 00:00 2024-04-24T22:57:30.429 INFO:teuthology.orchestra.run.smithi032.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2024-04-24T22:57:30.454 INFO:teuthology.orchestra.run.smithi032.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.2 MB/s | 87 kB 00:00 2024-04-24T22:57:30.504 INFO:teuthology.orchestra.run.smithi032.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-04-24T22:57:30.555 INFO:teuthology.orchestra.run.smithi032.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-04-24T22:57:30.638 INFO:teuthology.orchestra.run.smithi032.stdout:(76/152): python3-babel-2.9.1-2.el9.noarch.rpm 29 MB/s | 6.0 MB 00:00 2024-04-24T22:57:30.663 INFO:teuthology.orchestra.run.smithi032.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 237 kB/s | 26 kB 00:00 2024-04-24T22:57:30.689 INFO:teuthology.orchestra.run.smithi032.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-04-24T22:57:30.706 INFO:teuthology.orchestra.run.smithi032.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-04-24T22:57:30.739 INFO:teuthology.orchestra.run.smithi032.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 690 kB/s | 35 kB 00:00 2024-04-24T22:57:30.790 INFO:teuthology.orchestra.run.smithi121.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 2.2 MB/s | 2.4 MB 00:01 2024-04-24T22:57:30.797 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-04-24T22:57:30.797 INFO:teuthology.orchestra.run.smithi121.stdout:Total 13 MB/s | 220 MB 00:17 2024-04-24T22:57:30.798 INFO:teuthology.orchestra.run.smithi032.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-04-24T22:57:30.865 INFO:teuthology.orchestra.run.smithi032.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.2 MB/s | 222 kB 00:00 2024-04-24T22:57:30.932 INFO:teuthology.orchestra.run.smithi032.stdout:(83/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 23 MB/s | 5.1 MB 00:00 2024-04-24T22:57:30.957 INFO:teuthology.orchestra.run.smithi032.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 841 kB/s | 77 kB 00:00 2024-04-24T22:57:30.982 INFO:teuthology.orchestra.run.smithi032.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 845 kB/s | 42 kB 00:00 2024-04-24T22:57:31.016 INFO:teuthology.orchestra.run.smithi032.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-04-24T22:57:31.077 INFO:teuthology.orchestra.run.smithi032.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.6 MB/s | 159 kB 00:00 2024-04-24T22:57:31.108 INFO:teuthology.orchestra.run.smithi032.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.0 MB/s | 279 kB 00:00 2024-04-24T22:57:31.125 INFO:teuthology.orchestra.run.smithi032.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-24T22:57:31.201 INFO:teuthology.orchestra.run.smithi032.stdout:(90/152): postgresql-server-13.14-1.el9.x86_64. 6.5 MB/s | 5.8 MB 00:00 2024-04-24T22:57:31.218 INFO:teuthology.orchestra.run.smithi032.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 493 kB/s | 54 kB 00:00 2024-04-24T22:57:31.243 INFO:teuthology.orchestra.run.smithi032.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 995 kB/s | 42 kB 00:00 2024-04-24T22:57:31.268 INFO:teuthology.orchestra.run.smithi032.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 880 kB/s | 44 kB 00:00 2024-04-24T22:57:31.302 INFO:teuthology.orchestra.run.smithi032.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-04-24T22:57:31.319 INFO:teuthology.orchestra.run.smithi032.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 713 kB/s | 36 kB 00:00 2024-04-24T22:57:31.352 INFO:teuthology.orchestra.run.smithi032.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 942 kB/s | 47 kB 00:00 2024-04-24T22:57:31.386 INFO:teuthology.orchestra.run.smithi032.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.4 MB/s | 305 kB 00:00 2024-04-24T22:57:31.411 INFO:teuthology.orchestra.run.smithi032.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 987 kB/s | 58 kB 00:00 2024-04-24T22:57:31.445 INFO:teuthology.orchestra.run.smithi032.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-24T22:57:31.470 INFO:teuthology.orchestra.run.smithi032.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-24T22:57:31.496 INFO:teuthology.orchestra.run.smithi032.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 886 kB/s | 45 kB 00:00 2024-04-24T22:57:31.521 INFO:teuthology.orchestra.run.smithi032.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 445 kB/s | 22 kB 00:00 2024-04-24T22:57:31.623 INFO:teuthology.orchestra.run.smithi032.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 6.7 MB/s | 863 kB 00:00 2024-04-24T22:57:31.797 INFO:teuthology.orchestra.run.smithi032.stdout:(104/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 24 MB/s | 16 MB 00:00 2024-04-24T22:57:31.907 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-04-24T22:57:31.914 INFO:teuthology.orchestra.run.smithi032.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 167 kB/s | 19 kB 00:00 2024-04-24T22:57:32.006 INFO:teuthology.orchestra.run.smithi032.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 807 kB/s | 309 kB 00:00 2024-04-24T22:57:32.048 INFO:teuthology.orchestra.run.smithi032.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.0 MB/s | 548 kB 00:00 2024-04-24T22:57:32.048 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-04-24T22:57:32.048 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-04-24T22:57:32.090 INFO:teuthology.orchestra.run.smithi032.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 310 kB/s | 26 kB 00:00 2024-04-24T22:57:32.115 INFO:teuthology.orchestra.run.smithi032.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 724 kB/s | 49 kB 00:00 2024-04-24T22:57:32.140 INFO:teuthology.orchestra.run.smithi032.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-04-24T22:57:32.174 INFO:teuthology.orchestra.run.smithi032.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-04-24T22:57:32.266 INFO:teuthology.orchestra.run.smithi032.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 5.8 MB/s | 548 kB 00:00 2024-04-24T22:57:32.308 INFO:teuthology.orchestra.run.smithi032.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.9 MB/s | 838 kB 00:00 2024-04-24T22:57:32.333 INFO:teuthology.orchestra.run.smithi032.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 651 kB/s | 43 kB 00:00 2024-04-24T22:57:32.358 INFO:teuthology.orchestra.run.smithi032.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 641 kB/s | 32 kB 00:00 2024-04-24T22:57:32.425 INFO:teuthology.orchestra.run.smithi032.stdout:(116/152): libarrow-9.0.0-10.el9.x86_64.rpm 8.6 MB/s | 4.4 MB 00:00 2024-04-24T22:57:32.451 INFO:teuthology.orchestra.run.smithi032.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 128 kB/s | 15 kB 00:00 2024-04-24T22:57:32.476 INFO:teuthology.orchestra.run.smithi032.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.4 MB/s | 172 kB 00:00 2024-04-24T22:57:32.501 INFO:teuthology.orchestra.run.smithi032.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 4.6 MB/s | 358 kB 00:00 2024-04-24T22:57:32.526 INFO:teuthology.orchestra.run.smithi032.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-04-24T22:57:32.552 INFO:teuthology.orchestra.run.smithi032.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 467 kB/s | 35 kB 00:00 2024-04-24T22:57:32.577 INFO:teuthology.orchestra.run.smithi032.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 2.7 MB/s | 206 kB 00:00 2024-04-24T22:57:32.644 INFO:teuthology.orchestra.run.smithi032.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 17 MB/s | 2.0 MB 00:00 2024-04-24T22:57:32.661 INFO:teuthology.orchestra.run.smithi032.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 128 kB/s | 11 kB 00:00 2024-04-24T22:57:32.678 INFO:teuthology.orchestra.run.smithi032.stdout:(125/152): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-04-24T22:57:32.695 INFO:teuthology.orchestra.run.smithi032.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 351 kB/s | 18 kB 00:00 2024-04-24T22:57:32.712 INFO:teuthology.orchestra.run.smithi032.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 458 kB/s | 23 kB 00:00 2024-04-24T22:57:32.729 INFO:teuthology.orchestra.run.smithi032.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 383 kB/s | 19 kB 00:00 2024-04-24T22:57:32.746 INFO:teuthology.orchestra.run.smithi032.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 386 kB/s | 20 kB 00:00 2024-04-24T22:57:32.780 INFO:teuthology.orchestra.run.smithi032.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 920 kB/s | 46 kB 00:00 2024-04-24T22:57:32.805 INFO:teuthology.orchestra.run.smithi032.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-04-24T22:57:32.866 INFO:teuthology.orchestra.run.smithi032.stdout:(132/152): python3-kubernetes-26.1.0-2.el9.noar 6.6 MB/s | 1.0 MB 00:00 2024-04-24T22:57:32.889 INFO:teuthology.orchestra.run.smithi032.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 530 kB/s | 58 kB 00:00 2024-04-24T22:57:32.914 INFO:teuthology.orchestra.run.smithi032.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-04-24T22:57:32.939 INFO:teuthology.orchestra.run.smithi032.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 225 kB/s | 16 kB 00:00 2024-04-24T22:57:32.965 INFO:teuthology.orchestra.run.smithi032.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-04-24T22:57:32.990 INFO:teuthology.orchestra.run.smithi032.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 374 kB/s | 28 kB 00:00 2024-04-24T22:57:33.015 INFO:teuthology.orchestra.run.smithi032.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 407 kB/s | 31 kB 00:00 2024-04-24T22:57:33.049 INFO:teuthology.orchestra.run.smithi032.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-04-24T22:57:33.074 INFO:teuthology.orchestra.run.smithi032.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 611 kB/s | 36 kB 00:00 2024-04-24T22:57:33.099 INFO:teuthology.orchestra.run.smithi032.stdout:(141/152): python3-rsa-4.9-2.el9.noarch.rpm 541 kB/s | 59 kB 00:00 2024-04-24T22:57:33.125 INFO:teuthology.orchestra.run.smithi032.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 676 kB/s | 51 kB 00:00 2024-04-24T22:57:33.150 INFO:teuthology.orchestra.run.smithi032.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-24T22:57:33.175 INFO:teuthology.orchestra.run.smithi032.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-24T22:57:33.201 INFO:teuthology.orchestra.run.smithi032.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-04-24T22:57:33.226 INFO:teuthology.orchestra.run.smithi032.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 264 kB/s | 20 kB 00:00 2024-04-24T22:57:33.251 INFO:teuthology.orchestra.run.smithi032.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-04-24T22:57:33.326 INFO:teuthology.orchestra.run.smithi032.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-04-24T22:57:33.602 INFO:teuthology.orchestra.run.smithi032.stdout:(149/152): librbd1-19.0.0-2445.gd2c45d97.el9.x8 9.0 MB/s | 3.1 MB 00:00 2024-04-24T22:57:33.808 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-04-24T22:57:33.808 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-04-24T22:57:33.830 INFO:teuthology.orchestra.run.smithi032.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 2.7 MB/s | 1.3 MB 00:00 2024-04-24T22:57:34.085 INFO:teuthology.orchestra.run.smithi032.stdout:(151/152): librados2-19.0.0-2445.gd2c45d97.el9. 3.9 MB/s | 3.4 MB 00:00 2024-04-24T22:57:34.394 INFO:teuthology.orchestra.run.smithi032.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 3.0 MB/s | 2.4 MB 00:00 2024-04-24T22:57:34.397 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2024-04-24T22:57:34.397 INFO:teuthology.orchestra.run.smithi032.stdout:Total 13 MB/s | 220 MB 00:16 2024-04-24T22:57:35.530 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction check 2024-04-24T22:57:35.691 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction check succeeded. 2024-04-24T22:57:35.691 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction test 2024-04-24T22:57:37.556 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction test succeeded. 2024-04-24T22:57:37.556 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction 2024-04-24T22:57:39.856 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-04-24T22:57:39.944 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-24T22:57:39.982 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-24T22:57:40.044 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-24T22:57:40.077 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-24T22:57:40.103 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-24T22:57:40.146 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-24T22:57:40.467 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-24T22:57:40.579 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T22:57:40.700 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T22:57:40.722 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T22:57:40.813 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T22:57:40.851 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T22:57:41.001 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-24T22:57:41.056 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-24T22:57:41.084 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T22:57:41.187 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T22:57:41.210 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T22:57:41.273 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T22:57:41.295 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T22:57:41.319 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T22:57:41.520 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T22:57:41.553 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T22:57:41.659 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-24T22:57:41.757 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-24T22:57:41.785 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-24T22:57:41.820 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-24T22:57:41.885 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-24T22:57:41.919 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-24T22:57:41.962 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-24T22:57:41.995 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-24T22:57:42.023 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-24T22:57:42.091 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-24T22:57:42.153 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-24T22:57:42.184 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-24T22:57:42.219 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-24T22:57:42.295 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-24T22:57:42.330 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 31/156 2024-04-24T22:57:42.373 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 32/156 2024-04-24T22:57:42.402 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 33/156 2024-04-24T22:57:42.471 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-24T22:57:42.517 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-24T22:57:42.591 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-24T22:57:42.733 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-24T22:57:42.785 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-24T22:57:43.313 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-24T22:57:43.407 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-24T22:57:43.444 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-24T22:57:43.486 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-24T22:57:43.524 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 43/156 2024-04-24T22:57:43.566 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-24T22:57:43.724 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-24T22:57:43.777 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-24T22:57:43.789 INFO:teuthology.orchestra.run.smithi032.stdout: Preparing : 1/1 2024-04-24T22:57:43.832 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-24T22:57:43.873 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-24T22:57:43.905 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-24T22:57:43.920 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-24T22:57:43.944 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-24T22:57:43.974 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-24T22:57:43.980 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-24T22:57:44.016 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-24T22:57:44.027 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-24T22:57:44.041 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-24T22:57:44.064 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-24T22:57:44.085 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-24T22:57:44.107 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-24T22:57:44.149 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-24T22:57:44.287 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-24T22:57:44.330 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-24T22:57:44.359 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-24T22:57:44.401 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-24T22:57:44.433 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-24T22:57:44.496 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-24T22:57:44.534 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T22:57:44.534 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-24T22:57:44.574 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-24T22:57:44.600 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-24T22:57:44.625 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-24T22:57:44.641 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T22:57:44.677 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T22:57:44.699 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-24T22:57:44.760 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T22:57:44.789 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T22:57:44.827 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-24T22:57:44.865 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-24T22:57:44.893 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-24T22:57:44.927 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-24T22:57:44.956 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-24T22:57:44.967 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-24T22:57:45.012 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-24T22:57:45.040 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T22:57:45.098 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-24T22:57:45.144 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T22:57:45.174 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T22:57:45.236 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T22:57:45.258 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T22:57:45.271 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-24T22:57:45.279 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T22:57:45.305 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-24T22:57:45.339 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-24T22:57:45.381 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-24T22:57:45.423 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-24T22:57:45.467 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T22:57:45.500 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T22:57:45.512 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-24T22:57:45.561 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-24T22:57:45.592 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-24T22:57:45.611 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-24T22:57:45.639 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-24T22:57:45.718 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-24T22:57:45.737 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-24T22:57:45.749 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-24T22:57:45.785 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-24T22:57:45.785 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-24T22:57:45.862 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-24T22:57:45.901 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-24T22:57:45.944 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-24T22:57:45.976 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-24T22:57:46.004 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-24T22:57:46.077 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-24T22:57:46.129 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-24T22:57:46.157 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-24T22:57:46.202 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-24T22:57:46.280 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-24T22:57:46.312 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 31/156 2024-04-24T22:57:46.356 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 32/156 2024-04-24T22:57:46.384 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 33/156 2024-04-24T22:57:46.460 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-24T22:57:46.508 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-24T22:57:46.515 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-24T22:57:46.565 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-24T22:57:46.586 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-24T22:57:46.599 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-24T22:57:46.714 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-24T22:57:46.732 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-24T22:57:46.765 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-24T22:57:46.794 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-24T22:57:46.853 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-24T22:57:46.892 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-24T22:57:47.069 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-24T22:57:47.107 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-24T22:57:47.192 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-24T22:57:47.354 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-24T22:57:47.434 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-24T22:57:47.468 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-24T22:57:47.511 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-24T22:57:47.541 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 43/156 2024-04-24T22:57:47.576 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-24T22:57:47.732 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-24T22:57:47.794 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-24T22:57:47.839 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-24T22:57:47.850 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-24T22:57:47.924 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-24T22:57:47.948 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-24T22:57:47.962 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-24T22:57:47.998 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-24T22:57:48.046 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-24T22:57:48.090 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-24T22:57:48.133 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-24T22:57:48.176 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-24T22:57:48.319 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-24T22:57:48.364 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-24T22:57:48.394 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-24T22:57:48.436 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-24T22:57:48.542 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-24T22:57:48.577 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-24T22:57:48.618 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-24T22:57:48.643 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-24T22:57:48.667 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-24T22:57:48.737 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-24T22:57:48.874 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-24T22:57:48.917 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-24T22:57:48.945 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-24T22:57:49.180 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-24T22:57:49.180 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-24T22:57:49.229 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-24T22:57:49.261 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-24T22:57:49.268 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-24T22:57:49.289 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-24T22:57:49.297 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-24T22:57:49.322 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-24T22:57:49.332 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-24T22:57:49.365 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-24T22:57:49.408 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-24T22:57:49.441 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-24T22:57:49.533 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-24T22:57:49.581 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-24T22:57:49.611 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-24T22:57:49.632 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-24T22:57:49.659 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-24T22:57:49.677 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T22:57:49.767 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T22:57:49.778 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-24T22:57:49.815 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 100/156 2024-04-24T22:57:49.830 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-24T22:57:49.850 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-24T22:57:50.320 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-24T22:57:50.563 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T22:57:50.594 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-24T22:57:50.634 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T22:57:50.635 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-24T22:57:50.664 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 104/156 2024-04-24T22:57:50.669 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-24T22:57:50.780 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-24T22:57:50.836 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-24T22:57:50.934 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-24T22:57:50.971 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-24T22:57:51.155 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-24T22:57:51.185 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-24T22:57:51.265 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-24T22:57:51.946 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-24T22:57:52.062 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-24T22:57:52.904 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T22:57:52.936 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T22:57:53.370 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-24T22:57:53.432 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-24T22:57:53.460 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-24T22:57:53.478 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T22:57:53.493 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-24T22:57:53.514 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T22:57:53.612 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T22:57:53.612 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-24T22:57:53.612 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-24T22:57:53.613 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-24T22:57:53.649 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T22:57:53.829 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-24T22:57:53.864 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T22:57:53.948 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T22:57:53.994 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 100/156 2024-04-24T22:57:54.031 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-24T22:57:54.527 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-24T22:57:54.559 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T22:57:54.614 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T22:57:54.644 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 104/156 2024-04-24T22:57:57.022 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T22:57:57.057 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T22:57:57.629 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T22:57:57.670 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T22:57:57.760 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T22:57:57.760 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-24T22:57:57.760 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-24T22:57:57.760 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:57:57.787 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T22:58:05.018 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T22:58:05.019 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /sys 2024-04-24T22:58:05.019 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /proc 2024-04-24T22:58:05.019 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /mnt 2024-04-24T22:58:05.019 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /var/tmp 2024-04-24T22:58:05.019 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /home 2024-04-24T22:58:05.019 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /root 2024-04-24T22:58:05.019 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /tmp 2024-04-24T22:58:05.019 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-24T22:58:05.187 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T22:58:05.202 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T22:58:05.202 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:58:05.202 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-24T22:58:05.202 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-24T22:58:06.673 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T22:58:06.698 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T22:58:06.698 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:58:06.698 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-24T22:58:06.698 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T22:58:06.699 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T22:58:06.699 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-24T22:58:06.923 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T22:58:06.946 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T22:58:06.946 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:58:06.946 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-24T22:58:06.946 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T22:58:06.946 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T22:58:06.946 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-24T22:58:07.390 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T22:58:07.413 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T22:58:07.413 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:58:07.413 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-24T22:58:07.413 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T22:58:07.413 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T22:58:07.413 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-24T22:58:07.512 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-24T22:58:07.570 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-24T22:58:07.624 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-24T22:58:07.810 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-24T22:58:07.858 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-24T22:58:07.937 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-24T22:58:08.034 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T22:58:08.173 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T22:58:08.208 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-24T22:58:08.240 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-24T22:58:08.268 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-24T22:58:08.667 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T22:58:08.700 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T22:58:08.731 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 124/156 2024-04-24T22:58:08.794 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 125/156 2024-04-24T22:58:08.831 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T22:58:09.636 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T22:58:09.636 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /sys 2024-04-24T22:58:09.636 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /proc 2024-04-24T22:58:09.636 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /mnt 2024-04-24T22:58:09.636 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /var/tmp 2024-04-24T22:58:09.636 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /home 2024-04-24T22:58:09.636 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /root 2024-04-24T22:58:09.636 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /tmp 2024-04-24T22:58:09.637 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:58:09.819 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T22:58:09.835 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T22:58:09.835 INFO:teuthology.orchestra.run.smithi032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:58:09.835 INFO:teuthology.orchestra.run.smithi032.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-24T22:58:09.835 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:58:09.856 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T22:58:09.891 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T22:58:10.851 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T22:58:10.875 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T22:58:10.976 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T22:58:11.138 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 129/156 2024-04-24T22:58:11.169 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T22:58:11.192 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T22:58:11.192 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:58:11.192 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-24T22:58:11.192 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T22:58:11.192 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T22:58:11.192 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-24T22:58:11.236 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T22:58:11.247 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T22:58:11.346 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 132/156 2024-04-24T22:58:11.397 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T22:58:11.400 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 133/156 2024-04-24T22:58:11.424 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T22:58:11.425 INFO:teuthology.orchestra.run.smithi032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:58:11.425 INFO:teuthology.orchestra.run.smithi032.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-24T22:58:11.425 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T22:58:11.425 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T22:58:11.425 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:58:11.656 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T22:58:11.680 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T22:58:11.680 INFO:teuthology.orchestra.run.smithi032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:58:11.680 INFO:teuthology.orchestra.run.smithi032.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-24T22:58:11.680 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T22:58:11.680 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T22:58:11.681 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:58:12.140 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T22:58:12.164 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T22:58:12.165 INFO:teuthology.orchestra.run.smithi032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:58:12.165 INFO:teuthology.orchestra.run.smithi032.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-24T22:58:12.165 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T22:58:12.165 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T22:58:12.165 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:58:12.279 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-24T22:58:12.317 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-24T22:58:12.338 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-24T22:58:12.347 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T22:58:12.369 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T22:58:12.370 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:58:12.370 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-24T22:58:12.370 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T22:58:12.370 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T22:58:12.370 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-24T22:58:12.403 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-24T22:58:12.406 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T22:58:12.427 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T22:58:12.428 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:58:12.428 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-24T22:58:12.428 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-24T22:58:12.473 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T22:58:12.495 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T22:58:12.495 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:58:12.495 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-24T22:58:12.495 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T22:58:12.496 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T22:58:12.496 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-24T22:58:12.599 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-24T22:58:12.659 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-24T22:58:12.740 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-24T22:58:12.799 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T22:58:12.821 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T22:58:12.822 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:58:12.822 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-24T22:58:12.822 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T22:58:12.822 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T22:58:12.822 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-24T22:58:12.838 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T22:58:12.982 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T22:58:13.017 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-24T22:58:13.049 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-24T22:58:13.068 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-24T22:58:13.520 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T22:58:13.550 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T22:58:13.583 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 124/156 2024-04-24T22:58:13.648 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 125/156 2024-04-24T22:58:13.682 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T22:58:14.781 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T22:58:14.817 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T22:58:15.840 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T22:58:15.867 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T22:58:15.975 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T22:58:16.146 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 129/156 2024-04-24T22:58:16.177 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T22:58:16.202 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T22:58:16.202 INFO:teuthology.orchestra.run.smithi032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:58:16.202 INFO:teuthology.orchestra.run.smithi032.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-24T22:58:16.202 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T22:58:16.202 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T22:58:16.202 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:58:16.253 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T22:58:16.265 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T22:58:16.367 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 132/156 2024-04-24T22:58:16.419 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 133/156 2024-04-24T22:58:17.311 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 139/156 2024-04-24T22:58:17.352 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 140/156 2024-04-24T22:58:17.369 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-24T22:58:17.392 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 141/156 2024-04-24T22:58:17.398 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T22:58:17.422 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T22:58:17.422 INFO:teuthology.orchestra.run.smithi032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:58:17.422 INFO:teuthology.orchestra.run.smithi032.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-24T22:58:17.422 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T22:58:17.423 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T22:58:17.423 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:58:17.424 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-24T22:58:17.452 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-24T22:58:17.465 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T22:58:17.488 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T22:58:17.489 INFO:teuthology.orchestra.run.smithi032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:58:17.489 INFO:teuthology.orchestra.run.smithi032.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-24T22:58:17.489 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:58:17.541 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T22:58:17.564 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T22:58:17.564 INFO:teuthology.orchestra.run.smithi032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:58:17.564 INFO:teuthology.orchestra.run.smithi032.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-24T22:58:17.564 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T22:58:17.564 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T22:58:17.564 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:58:17.633 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 144/156 2024-04-24T22:58:17.673 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-24T22:58:17.703 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 146/156 2024-04-24T22:58:17.883 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T22:58:17.907 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T22:58:17.907 INFO:teuthology.orchestra.run.smithi032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:58:17.907 INFO:teuthology.orchestra.run.smithi032.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-24T22:58:17.907 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T22:58:17.907 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T22:58:17.907 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:58:18.314 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T22:58:18.351 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T22:58:18.371 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T22:58:19.196 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 148/156 2024-04-24T22:58:19.289 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-24T22:58:19.321 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-24T22:58:19.412 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-24T22:58:19.443 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-24T22:58:19.443 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T22:58:19.536 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T22:58:19.569 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T22:58:19.569 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T22:58:19.659 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T22:58:19.725 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T22:58:22.696 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 139/156 2024-04-24T22:58:22.738 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 140/156 2024-04-24T22:58:22.779 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 141/156 2024-04-24T22:58:22.935 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-24T22:58:22.962 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-24T22:58:23.158 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 144/156 2024-04-24T22:58:23.200 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-24T22:58:23.230 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 146/156 2024-04-24T22:58:23.886 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T22:58:23.928 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T22:58:23.949 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T22:58:24.879 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 148/156 2024-04-24T22:58:24.967 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-24T22:58:25.009 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-24T22:58:25.093 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-24T22:58:25.155 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-24T22:58:25.155 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T22:58:25.239 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T22:58:25.271 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T22:58:25.272 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T22:58:25.372 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T22:58:25.440 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T22:58:25.606 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T22:58:25.607 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/156 2024-04-24T22:58:25.607 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2/156 2024-04-24T22:58:25.607 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 3/156 2024-04-24T22:58:25.607 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 4/156 2024-04-24T22:58:25.607 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 5/156 2024-04-24T22:58:25.607 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 6/156 2024-04-24T22:58:25.607 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 7/156 2024-04-24T22:58:25.607 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T22:58:25.607 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T22:58:25.607 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T22:58:25.607 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 11/156 2024-04-24T22:58:25.608 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 12/156 2024-04-24T22:58:25.608 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T22:58:25.608 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 14/156 2024-04-24T22:58:25.608 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 15/156 2024-04-24T22:58:25.608 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 16/156 2024-04-24T22:58:25.608 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 17/156 2024-04-24T22:58:25.608 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 18/156 2024-04-24T22:58:25.608 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 19/156 2024-04-24T22:58:25.608 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 20/156 2024-04-24T22:58:25.608 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 21/156 2024-04-24T22:58:25.608 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 22/156 2024-04-24T22:58:25.608 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 23/156 2024-04-24T22:58:25.609 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 24/156 2024-04-24T22:58:25.609 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 25/156 2024-04-24T22:58:25.609 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 26/156 2024-04-24T22:58:25.609 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 27/156 2024-04-24T22:58:25.609 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 28/156 2024-04-24T22:58:25.609 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 29/156 2024-04-24T22:58:25.609 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 30/156 2024-04-24T22:58:25.609 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 31/156 2024-04-24T22:58:25.609 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 32/156 2024-04-24T22:58:25.609 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 33/156 2024-04-24T22:58:25.609 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 34/156 2024-04-24T22:58:25.609 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 35/156 2024-04-24T22:58:25.609 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 36/156 2024-04-24T22:58:25.609 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 37/156 2024-04-24T22:58:25.610 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 38/156 2024-04-24T22:58:25.610 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 39/156 2024-04-24T22:58:25.610 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-24T22:58:25.610 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-24T22:58:25.610 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-24T22:58:25.610 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-24T22:58:25.610 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-24T22:58:25.610 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-24T22:58:25.610 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-24T22:58:25.610 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-24T22:58:25.610 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-24T22:58:25.611 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-24T22:58:25.611 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-24T22:58:25.611 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-24T22:58:25.611 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-24T22:58:25.611 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-24T22:58:25.611 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-24T22:58:25.611 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-24T22:58:25.611 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-24T22:58:25.611 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-24T22:58:25.611 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-24T22:58:25.611 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-24T22:58:25.611 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-24T22:58:25.611 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-24T22:58:25.612 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-24T22:58:25.612 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-24T22:58:25.612 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-24T22:58:25.612 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-24T22:58:25.612 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-24T22:58:25.612 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-24T22:58:25.612 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-24T22:58:25.612 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-24T22:58:25.612 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-24T22:58:25.612 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-24T22:58:25.612 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-24T22:58:25.612 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-24T22:58:25.612 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-24T22:58:25.612 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-24T22:58:25.613 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-24T22:58:25.613 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-24T22:58:25.613 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-24T22:58:25.613 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-24T22:58:25.613 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-24T22:58:25.613 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-24T22:58:25.613 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-24T22:58:25.613 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-24T22:58:25.613 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-24T22:58:25.613 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-24T22:58:25.613 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-24T22:58:25.613 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-24T22:58:25.614 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-24T22:58:25.614 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-24T22:58:25.614 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-24T22:58:25.614 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-24T22:58:25.614 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-24T22:58:25.614 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-24T22:58:25.614 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-24T22:58:25.614 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-24T22:58:25.614 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-24T22:58:25.614 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-24T22:58:25.614 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-24T22:58:25.614 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-24T22:58:25.614 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-24T22:58:25.614 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-24T22:58:25.615 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-24T22:58:25.615 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-24T22:58:25.615 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-24T22:58:25.615 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-24T22:58:25.615 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-24T22:58:25.615 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-24T22:58:25.615 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-24T22:58:25.615 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-24T22:58:25.615 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-24T22:58:25.615 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-24T22:58:25.615 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-24T22:58:25.615 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-24T22:58:25.615 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-24T22:58:25.616 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-24T22:58:25.616 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-24T22:58:25.616 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-24T22:58:25.616 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T22:58:25.616 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T22:58:25.616 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-24T22:58:25.616 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-24T22:58:25.616 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-24T22:58:25.616 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-24T22:58:25.616 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-24T22:58:25.616 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-24T22:58:25.616 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-24T22:58:25.616 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-24T22:58:25.616 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-24T22:58:25.616 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-24T22:58:25.617 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-24T22:58:25.617 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-24T22:58:25.617 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-24T22:58:25.617 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-24T22:58:25.617 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-24T22:58:25.617 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-24T22:58:25.617 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-24T22:58:25.617 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-24T22:58:25.617 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-24T22:58:25.617 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-24T22:58:25.617 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-24T22:58:25.617 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-24T22:58:25.617 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-24T22:58:25.617 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-24T22:58:25.618 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-24T22:58:25.618 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-24T22:58:25.618 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-24T22:58:25.618 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-24T22:58:25.618 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-24T22:58:25.618 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 149/156 2024-04-24T22:58:25.618 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-24T22:58:25.618 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 151/156 2024-04-24T22:58:25.618 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-24T22:58:25.618 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-24T22:58:25.618 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T22:58:25.618 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-24T22:58:27.689 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T22:58:27.690 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-24T22:58:27.690 INFO:teuthology.orchestra.run.smithi121.stdout:Upgraded: 2024-04-24T22:58:27.690 INFO:teuthology.orchestra.run.smithi121.stdout: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:27.690 INFO:teuthology.orchestra.run.smithi121.stdout: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:27.690 INFO:teuthology.orchestra.run.smithi121.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-24T22:58:27.690 INFO:teuthology.orchestra.run.smithi121.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-24T22:58:27.690 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-04-24T22:58:27.690 INFO:teuthology.orchestra.run.smithi121.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-24T22:58:27.690 INFO:teuthology.orchestra.run.smithi121.stdout: automake-1.16.2-8.el9.noarch 2024-04-24T22:58:27.690 INFO:teuthology.orchestra.run.smithi121.stdout: bison-3.7.4-5.el9.x86_64 2024-04-24T22:58:27.690 INFO:teuthology.orchestra.run.smithi121.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-24T22:58:27.690 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:27.691 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:27.691 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:27.691 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:27.691 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:58:27.691 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:27.691 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:27.691 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:27.691 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:58:27.691 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:58:27.691 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:58:27.691 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:58:27.691 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:58:27.691 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:27.691 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:27.692 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:58:27.692 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:27.692 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:27.692 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:27.692 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:58:27.692 INFO:teuthology.orchestra.run.smithi121.stdout: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:58:27.692 INFO:teuthology.orchestra.run.smithi121.stdout: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:27.692 INFO:teuthology.orchestra.run.smithi121.stdout: cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:27.692 INFO:teuthology.orchestra.run.smithi121.stdout: cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:58:27.692 INFO:teuthology.orchestra.run.smithi121.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-24T22:58:27.692 INFO:teuthology.orchestra.run.smithi121.stdout: flex-2.6.4-9.el9.x86_64 2024-04-24T22:58:27.692 INFO:teuthology.orchestra.run.smithi121.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-24T22:58:27.692 INFO:teuthology.orchestra.run.smithi121.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-24T22:58:27.693 INFO:teuthology.orchestra.run.smithi121.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-24T22:58:27.693 INFO:teuthology.orchestra.run.smithi121.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-24T22:58:27.693 INFO:teuthology.orchestra.run.smithi121.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-24T22:58:27.693 INFO:teuthology.orchestra.run.smithi121.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-24T22:58:27.693 INFO:teuthology.orchestra.run.smithi121.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-24T22:58:27.693 INFO:teuthology.orchestra.run.smithi121.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-24T22:58:27.693 INFO:teuthology.orchestra.run.smithi121.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-24T22:58:27.693 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:27.693 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:27.693 INFO:teuthology.orchestra.run.smithi121.stdout: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:27.693 INFO:teuthology.orchestra.run.smithi121.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-24T22:58:27.693 INFO:teuthology.orchestra.run.smithi121.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-24T22:58:27.693 INFO:teuthology.orchestra.run.smithi121.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-24T22:58:27.693 INFO:teuthology.orchestra.run.smithi121.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-24T22:58:27.694 INFO:teuthology.orchestra.run.smithi121.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-24T22:58:27.694 INFO:teuthology.orchestra.run.smithi121.stdout: librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:27.694 INFO:teuthology.orchestra.run.smithi121.stdout: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:27.694 INFO:teuthology.orchestra.run.smithi121.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-24T22:58:27.694 INFO:teuthology.orchestra.run.smithi121.stdout: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:27.694 INFO:teuthology.orchestra.run.smithi121.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-24T22:58:27.694 INFO:teuthology.orchestra.run.smithi121.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-24T22:58:27.694 INFO:teuthology.orchestra.run.smithi121.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-24T22:58:27.694 INFO:teuthology.orchestra.run.smithi121.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-24T22:58:27.694 INFO:teuthology.orchestra.run.smithi121.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-24T22:58:27.694 INFO:teuthology.orchestra.run.smithi121.stdout: lua-5.4.4-4.el9.x86_64 2024-04-24T22:58:27.694 INFO:teuthology.orchestra.run.smithi121.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-24T22:58:27.694 INFO:teuthology.orchestra.run.smithi121.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-24T22:58:27.694 INFO:teuthology.orchestra.run.smithi121.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-24T22:58:27.695 INFO:teuthology.orchestra.run.smithi121.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-24T22:58:27.695 INFO:teuthology.orchestra.run.smithi121.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-24T22:58:27.695 INFO:teuthology.orchestra.run.smithi121.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-24T22:58:27.695 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql-13.14-1.el9.x86_64 2024-04-24T22:58:27.695 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-24T22:58:27.695 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-24T22:58:27.695 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-24T22:58:27.695 INFO:teuthology.orchestra.run.smithi121.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-24T22:58:27.695 INFO:teuthology.orchestra.run.smithi121.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-24T22:58:27.695 INFO:teuthology.orchestra.run.smithi121.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-24T22:58:27.695 INFO:teuthology.orchestra.run.smithi121.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-24T22:58:27.695 INFO:teuthology.orchestra.run.smithi121.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-24T22:58:27.695 INFO:teuthology.orchestra.run.smithi121.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-24T22:58:27.695 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-24T22:58:27.695 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:27.696 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:27.696 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:27.696 INFO:teuthology.orchestra.run.smithi121.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-24T22:58:27.696 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-24T22:58:27.696 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-24T22:58:27.696 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-24T22:58:27.696 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-24T22:58:27.696 INFO:teuthology.orchestra.run.smithi121.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-24T22:58:27.696 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-24T22:58:27.696 INFO:teuthology.orchestra.run.smithi121.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-24T22:58:27.696 INFO:teuthology.orchestra.run.smithi121.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-24T22:58:27.696 INFO:teuthology.orchestra.run.smithi121.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-24T22:58:27.696 INFO:teuthology.orchestra.run.smithi121.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-24T22:58:27.697 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-24T22:58:27.697 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-24T22:58:27.697 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-24T22:58:27.697 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-24T22:58:27.697 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-24T22:58:27.697 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-24T22:58:27.697 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-24T22:58:27.697 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-24T22:58:27.697 INFO:teuthology.orchestra.run.smithi121.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-24T22:58:27.697 INFO:teuthology.orchestra.run.smithi121.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-24T22:58:27.697 INFO:teuthology.orchestra.run.smithi121.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-24T22:58:27.697 INFO:teuthology.orchestra.run.smithi121.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-24T22:58:27.698 INFO:teuthology.orchestra.run.smithi121.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-24T22:58:27.698 INFO:teuthology.orchestra.run.smithi121.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-24T22:58:27.698 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-24T22:58:27.698 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-24T22:58:27.698 INFO:teuthology.orchestra.run.smithi121.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-24T22:58:27.698 INFO:teuthology.orchestra.run.smithi121.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-24T22:58:27.698 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-24T22:58:27.698 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ply-3.11-14.el9.noarch 2024-04-24T22:58:27.698 INFO:teuthology.orchestra.run.smithi121.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-24T22:58:27.698 INFO:teuthology.orchestra.run.smithi121.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-24T22:58:27.698 INFO:teuthology.orchestra.run.smithi121.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-24T22:58:27.698 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-24T22:58:27.698 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-24T22:58:27.698 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-24T22:58:27.699 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-24T22:58:27.699 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-24T22:58:27.699 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-24T22:58:27.699 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:27.699 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:27.699 INFO:teuthology.orchestra.run.smithi121.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-24T22:58:27.699 INFO:teuthology.orchestra.run.smithi121.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-24T22:58:27.699 INFO:teuthology.orchestra.run.smithi121.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-24T22:58:27.699 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:27.699 INFO:teuthology.orchestra.run.smithi121.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-24T22:58:27.699 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-24T22:58:27.699 INFO:teuthology.orchestra.run.smithi121.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-24T22:58:27.699 INFO:teuthology.orchestra.run.smithi121.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-24T22:58:27.699 INFO:teuthology.orchestra.run.smithi121.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-24T22:58:27.699 INFO:teuthology.orchestra.run.smithi121.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-24T22:58:27.700 INFO:teuthology.orchestra.run.smithi121.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-24T22:58:27.700 INFO:teuthology.orchestra.run.smithi121.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-24T22:58:27.700 INFO:teuthology.orchestra.run.smithi121.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-24T22:58:27.700 INFO:teuthology.orchestra.run.smithi121.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-24T22:58:27.700 INFO:teuthology.orchestra.run.smithi121.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-24T22:58:27.700 INFO:teuthology.orchestra.run.smithi121.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-24T22:58:27.700 INFO:teuthology.orchestra.run.smithi121.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-24T22:58:27.700 INFO:teuthology.orchestra.run.smithi121.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-24T22:58:27.700 INFO:teuthology.orchestra.run.smithi121.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-24T22:58:27.700 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:27.700 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:27.700 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:27.700 INFO:teuthology.orchestra.run.smithi121.stdout: re2-1:20211101-3.el9.x86_64 2024-04-24T22:58:27.700 INFO:teuthology.orchestra.run.smithi121.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-24T22:58:27.701 INFO:teuthology.orchestra.run.smithi121.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-24T22:58:27.701 INFO:teuthology.orchestra.run.smithi121.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-24T22:58:27.701 INFO:teuthology.orchestra.run.smithi121.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-24T22:58:27.701 INFO:teuthology.orchestra.run.smithi121.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-24T22:58:27.701 INFO:teuthology.orchestra.run.smithi121.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-24T22:58:27.701 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-24T22:58:27.701 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-04-24T22:58:28.094 DEBUG:teuthology.parallel:result is None 2024-04-24T22:58:31.374 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T22:58:31.375 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/156 2024-04-24T22:58:31.375 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2/156 2024-04-24T22:58:31.375 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 3/156 2024-04-24T22:58:31.375 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 4/156 2024-04-24T22:58:31.375 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 5/156 2024-04-24T22:58:31.375 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 6/156 2024-04-24T22:58:31.375 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 7/156 2024-04-24T22:58:31.375 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T22:58:31.375 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T22:58:31.375 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T22:58:31.375 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 11/156 2024-04-24T22:58:31.376 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 12/156 2024-04-24T22:58:31.376 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T22:58:31.376 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 14/156 2024-04-24T22:58:31.376 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 15/156 2024-04-24T22:58:31.376 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 16/156 2024-04-24T22:58:31.376 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 17/156 2024-04-24T22:58:31.376 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 18/156 2024-04-24T22:58:31.376 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 19/156 2024-04-24T22:58:31.376 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 20/156 2024-04-24T22:58:31.376 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 21/156 2024-04-24T22:58:31.376 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 22/156 2024-04-24T22:58:31.376 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 23/156 2024-04-24T22:58:31.376 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 24/156 2024-04-24T22:58:31.376 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 25/156 2024-04-24T22:58:31.376 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 26/156 2024-04-24T22:58:31.377 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 27/156 2024-04-24T22:58:31.377 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 28/156 2024-04-24T22:58:31.377 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 29/156 2024-04-24T22:58:31.377 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 30/156 2024-04-24T22:58:31.388 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 31/156 2024-04-24T22:58:31.388 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 32/156 2024-04-24T22:58:31.389 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 33/156 2024-04-24T22:58:31.389 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 34/156 2024-04-24T22:58:31.389 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 35/156 2024-04-24T22:58:31.389 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 36/156 2024-04-24T22:58:31.389 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 37/156 2024-04-24T22:58:31.389 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 38/156 2024-04-24T22:58:31.389 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 39/156 2024-04-24T22:58:31.389 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-24T22:58:31.389 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-24T22:58:31.389 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-24T22:58:31.389 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-24T22:58:31.389 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-24T22:58:31.389 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-24T22:58:31.390 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-24T22:58:31.390 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-24T22:58:31.390 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-24T22:58:31.390 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-24T22:58:31.390 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-24T22:58:31.390 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-24T22:58:31.390 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-24T22:58:31.390 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-24T22:58:31.390 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-24T22:58:31.390 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-24T22:58:31.390 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-24T22:58:31.390 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-24T22:58:31.390 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-24T22:58:31.390 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-24T22:58:31.391 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-24T22:58:31.391 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-24T22:58:31.391 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-24T22:58:31.391 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-24T22:58:31.391 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-24T22:58:31.391 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-24T22:58:31.391 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-24T22:58:31.391 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-24T22:58:31.391 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-24T22:58:31.391 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-24T22:58:31.391 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-24T22:58:31.391 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-24T22:58:31.391 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-24T22:58:31.391 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-24T22:58:31.391 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-24T22:58:31.392 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-24T22:58:31.392 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-24T22:58:31.392 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-24T22:58:31.392 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-24T22:58:31.392 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-24T22:58:31.392 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-24T22:58:31.392 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-24T22:58:31.392 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-24T22:58:31.392 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-24T22:58:31.392 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-24T22:58:31.392 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-24T22:58:31.392 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-24T22:58:31.392 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-24T22:58:31.393 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-24T22:58:31.393 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-24T22:58:31.393 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-24T22:58:31.393 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-24T22:58:31.393 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-24T22:58:31.393 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-24T22:58:31.393 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-24T22:58:31.393 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-24T22:58:31.393 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-24T22:58:31.393 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-24T22:58:31.393 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-24T22:58:31.393 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-24T22:58:31.393 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-24T22:58:31.393 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-24T22:58:31.394 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-24T22:58:31.394 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-24T22:58:31.394 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-24T22:58:31.394 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-24T22:58:31.394 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-24T22:58:31.394 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-24T22:58:31.394 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-24T22:58:31.394 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-24T22:58:31.394 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-24T22:58:31.394 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-24T22:58:31.394 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-24T22:58:31.394 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-24T22:58:31.395 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-24T22:58:31.395 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-24T22:58:31.395 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-24T22:58:31.395 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-24T22:58:31.395 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T22:58:31.395 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T22:58:31.395 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-24T22:58:31.395 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-24T22:58:31.395 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-24T22:58:31.395 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-24T22:58:31.395 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-24T22:58:31.395 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-24T22:58:31.395 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-24T22:58:31.395 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-24T22:58:31.396 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-24T22:58:31.396 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-24T22:58:31.396 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-24T22:58:31.396 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-24T22:58:31.396 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-24T22:58:31.396 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-24T22:58:31.396 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-24T22:58:31.396 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-24T22:58:31.396 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-24T22:58:31.396 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-24T22:58:31.396 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-24T22:58:31.396 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-24T22:58:31.396 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-24T22:58:31.397 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-24T22:58:31.397 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-24T22:58:31.397 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-24T22:58:31.397 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-24T22:58:31.397 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-24T22:58:31.397 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-24T22:58:31.397 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-24T22:58:31.397 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-24T22:58:31.397 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 149/156 2024-04-24T22:58:31.397 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-24T22:58:31.397 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 151/156 2024-04-24T22:58:31.397 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-24T22:58:31.397 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-24T22:58:31.397 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T22:58:31.398 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-24T22:58:33.462 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T22:58:33.462 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:58:33.463 INFO:teuthology.orchestra.run.smithi032.stdout:Upgraded: 2024-04-24T22:58:33.463 INFO:teuthology.orchestra.run.smithi032.stdout: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:33.463 INFO:teuthology.orchestra.run.smithi032.stdout: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:33.463 INFO:teuthology.orchestra.run.smithi032.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-24T22:58:33.463 INFO:teuthology.orchestra.run.smithi032.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-24T22:58:33.463 INFO:teuthology.orchestra.run.smithi032.stdout:Installed: 2024-04-24T22:58:33.463 INFO:teuthology.orchestra.run.smithi032.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-24T22:58:33.463 INFO:teuthology.orchestra.run.smithi032.stdout: automake-1.16.2-8.el9.noarch 2024-04-24T22:58:33.463 INFO:teuthology.orchestra.run.smithi032.stdout: bison-3.7.4-5.el9.x86_64 2024-04-24T22:58:33.463 INFO:teuthology.orchestra.run.smithi032.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-24T22:58:33.463 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:33.463 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:33.463 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:33.464 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:33.464 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:58:33.464 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:33.464 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:33.464 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:33.464 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:58:33.464 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:58:33.464 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:58:33.464 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:58:33.464 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:58:33.464 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:33.464 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:33.465 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:58:33.465 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:33.465 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:33.465 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:33.465 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:58:33.465 INFO:teuthology.orchestra.run.smithi032.stdout: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:58:33.465 INFO:teuthology.orchestra.run.smithi032.stdout: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:33.465 INFO:teuthology.orchestra.run.smithi032.stdout: cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:33.465 INFO:teuthology.orchestra.run.smithi032.stdout: cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:58:33.465 INFO:teuthology.orchestra.run.smithi032.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-24T22:58:33.465 INFO:teuthology.orchestra.run.smithi032.stdout: flex-2.6.4-9.el9.x86_64 2024-04-24T22:58:33.465 INFO:teuthology.orchestra.run.smithi032.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-24T22:58:33.465 INFO:teuthology.orchestra.run.smithi032.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-24T22:58:33.465 INFO:teuthology.orchestra.run.smithi032.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-24T22:58:33.466 INFO:teuthology.orchestra.run.smithi032.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-24T22:58:33.466 INFO:teuthology.orchestra.run.smithi032.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-24T22:58:33.466 INFO:teuthology.orchestra.run.smithi032.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-24T22:58:33.466 INFO:teuthology.orchestra.run.smithi032.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-24T22:58:33.466 INFO:teuthology.orchestra.run.smithi032.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-24T22:58:33.466 INFO:teuthology.orchestra.run.smithi032.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-24T22:58:33.466 INFO:teuthology.orchestra.run.smithi032.stdout: libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:33.466 INFO:teuthology.orchestra.run.smithi032.stdout: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:33.466 INFO:teuthology.orchestra.run.smithi032.stdout: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:33.466 INFO:teuthology.orchestra.run.smithi032.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-24T22:58:33.466 INFO:teuthology.orchestra.run.smithi032.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-24T22:58:33.466 INFO:teuthology.orchestra.run.smithi032.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-24T22:58:33.466 INFO:teuthology.orchestra.run.smithi032.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-24T22:58:33.467 INFO:teuthology.orchestra.run.smithi032.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-24T22:58:33.467 INFO:teuthology.orchestra.run.smithi032.stdout: librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:33.467 INFO:teuthology.orchestra.run.smithi032.stdout: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:33.467 INFO:teuthology.orchestra.run.smithi032.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-24T22:58:33.467 INFO:teuthology.orchestra.run.smithi032.stdout: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:33.467 INFO:teuthology.orchestra.run.smithi032.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-24T22:58:33.467 INFO:teuthology.orchestra.run.smithi032.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-24T22:58:33.467 INFO:teuthology.orchestra.run.smithi032.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-24T22:58:33.467 INFO:teuthology.orchestra.run.smithi032.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-24T22:58:33.467 INFO:teuthology.orchestra.run.smithi032.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-24T22:58:33.467 INFO:teuthology.orchestra.run.smithi032.stdout: lua-5.4.4-4.el9.x86_64 2024-04-24T22:58:33.467 INFO:teuthology.orchestra.run.smithi032.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-24T22:58:33.467 INFO:teuthology.orchestra.run.smithi032.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-24T22:58:33.468 INFO:teuthology.orchestra.run.smithi032.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-24T22:58:33.468 INFO:teuthology.orchestra.run.smithi032.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-24T22:58:33.468 INFO:teuthology.orchestra.run.smithi032.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-24T22:58:33.468 INFO:teuthology.orchestra.run.smithi032.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-24T22:58:33.468 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-13.14-1.el9.x86_64 2024-04-24T22:58:33.468 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-24T22:58:33.468 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-24T22:58:33.468 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-24T22:58:33.468 INFO:teuthology.orchestra.run.smithi032.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-24T22:58:33.468 INFO:teuthology.orchestra.run.smithi032.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-24T22:58:33.468 INFO:teuthology.orchestra.run.smithi032.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-24T22:58:33.468 INFO:teuthology.orchestra.run.smithi032.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-24T22:58:33.468 INFO:teuthology.orchestra.run.smithi032.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-24T22:58:33.468 INFO:teuthology.orchestra.run.smithi032.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-24T22:58:33.469 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-24T22:58:33.469 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:33.469 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:33.469 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:33.469 INFO:teuthology.orchestra.run.smithi032.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-24T22:58:33.469 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-24T22:58:33.469 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-24T22:58:33.469 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-24T22:58:33.469 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-24T22:58:33.469 INFO:teuthology.orchestra.run.smithi032.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-24T22:58:33.469 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-24T22:58:33.469 INFO:teuthology.orchestra.run.smithi032.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-24T22:58:33.469 INFO:teuthology.orchestra.run.smithi032.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-24T22:58:33.469 INFO:teuthology.orchestra.run.smithi032.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-24T22:58:33.470 INFO:teuthology.orchestra.run.smithi032.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-24T22:58:33.470 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-24T22:58:33.470 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-24T22:58:33.470 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-24T22:58:33.470 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-24T22:58:33.470 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-24T22:58:33.470 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-24T22:58:33.470 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-24T22:58:33.470 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-24T22:58:33.470 INFO:teuthology.orchestra.run.smithi032.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-24T22:58:33.471 INFO:teuthology.orchestra.run.smithi032.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-24T22:58:33.471 INFO:teuthology.orchestra.run.smithi032.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-24T22:58:33.471 INFO:teuthology.orchestra.run.smithi032.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-24T22:58:33.471 INFO:teuthology.orchestra.run.smithi032.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-24T22:58:33.471 INFO:teuthology.orchestra.run.smithi032.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-24T22:58:33.471 INFO:teuthology.orchestra.run.smithi032.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-24T22:58:33.471 INFO:teuthology.orchestra.run.smithi032.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-24T22:58:33.471 INFO:teuthology.orchestra.run.smithi032.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-24T22:58:33.471 INFO:teuthology.orchestra.run.smithi032.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-24T22:58:33.471 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-24T22:58:33.471 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ply-3.11-14.el9.noarch 2024-04-24T22:58:33.471 INFO:teuthology.orchestra.run.smithi032.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-24T22:58:33.471 INFO:teuthology.orchestra.run.smithi032.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-24T22:58:33.472 INFO:teuthology.orchestra.run.smithi032.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-24T22:58:33.472 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-24T22:58:33.472 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-24T22:58:33.472 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-24T22:58:33.472 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-24T22:58:33.472 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-24T22:58:33.472 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-24T22:58:33.472 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:33.472 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:33.472 INFO:teuthology.orchestra.run.smithi032.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-24T22:58:33.472 INFO:teuthology.orchestra.run.smithi032.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-24T22:58:33.472 INFO:teuthology.orchestra.run.smithi032.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-24T22:58:33.472 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:33.472 INFO:teuthology.orchestra.run.smithi032.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-24T22:58:33.472 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-24T22:58:33.473 INFO:teuthology.orchestra.run.smithi032.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-24T22:58:33.473 INFO:teuthology.orchestra.run.smithi032.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-24T22:58:33.473 INFO:teuthology.orchestra.run.smithi032.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-24T22:58:33.473 INFO:teuthology.orchestra.run.smithi032.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-24T22:58:33.473 INFO:teuthology.orchestra.run.smithi032.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-24T22:58:33.473 INFO:teuthology.orchestra.run.smithi032.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-24T22:58:33.473 INFO:teuthology.orchestra.run.smithi032.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-24T22:58:33.473 INFO:teuthology.orchestra.run.smithi032.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-24T22:58:33.473 INFO:teuthology.orchestra.run.smithi032.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-24T22:58:33.473 INFO:teuthology.orchestra.run.smithi032.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-24T22:58:33.473 INFO:teuthology.orchestra.run.smithi032.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-24T22:58:33.473 INFO:teuthology.orchestra.run.smithi032.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-24T22:58:33.473 INFO:teuthology.orchestra.run.smithi032.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-24T22:58:33.474 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:33.474 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:33.474 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:58:33.474 INFO:teuthology.orchestra.run.smithi032.stdout: re2-1:20211101-3.el9.x86_64 2024-04-24T22:58:33.474 INFO:teuthology.orchestra.run.smithi032.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-24T22:58:33.474 INFO:teuthology.orchestra.run.smithi032.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-24T22:58:33.474 INFO:teuthology.orchestra.run.smithi032.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-24T22:58:33.474 INFO:teuthology.orchestra.run.smithi032.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-24T22:58:33.474 INFO:teuthology.orchestra.run.smithi032.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-24T22:58:33.474 INFO:teuthology.orchestra.run.smithi032.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-24T22:58:33.475 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:58:33.475 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-04-24T22:58:33.849 DEBUG:teuthology.parallel:result is None 2024-04-24T22:58:33.850 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:58:34.076 DEBUG:teuthology.orchestra.run.smithi032:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-24T22:58:34.098 INFO:teuthology.orchestra.run.smithi032.stdout:19.0.0-2445.gd2c45d97.el9 2024-04-24T22:58:34.099 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445.gd2c45d97.el9 2024-04-24T22:58:34.099 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445.gd2c45d97 is installed. 2024-04-24T22:58:34.101 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:58:34.247 DEBUG:teuthology.orchestra.run.smithi121:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-24T22:58:34.273 INFO:teuthology.orchestra.run.smithi121.stdout:19.0.0-2445.gd2c45d97.el9 2024-04-24T22:58:34.273 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445.gd2c45d97.el9 2024-04-24T22:58:34.274 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445.gd2c45d97 is installed. 2024-04-24T22:58:34.275 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-24T22:58:34.276 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T22:58:34.276 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-24T22:58:34.303 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T22:58:34.303 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-24T22:58:34.343 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-24T22:58:34.343 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T22:58:34.343 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/daemon-helper 2024-04-24T22:58:34.369 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-24T22:58:34.433 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T22:58:34.433 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/usr/bin/daemon-helper 2024-04-24T22:58:34.461 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-24T22:58:34.529 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-24T22:58:34.529 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T22:58:34.529 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-24T22:58:34.553 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-24T22:58:34.618 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T22:58:34.618 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-24T22:58:34.645 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-24T22:58:34.711 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-24T22:58:34.711 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T22:58:34.711 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/stdin-killer 2024-04-24T22:58:34.735 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-24T22:58:34.801 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T22:58:34.801 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/usr/bin/stdin-killer 2024-04-24T22:58:34.825 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-24T22:58:34.890 INFO:teuthology.run_tasks:Running task ceph... 2024-04-24T22:58:34.992 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-24T22:58:34.992 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 777 /var/log/ceph 2024-04-24T22:58:34.994 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 777 /var/log/ceph 2024-04-24T22:58:35.018 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-24T22:58:35.018 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-24T22:58:35.058 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-24T22:58:35.082 INFO:tasks.ceph:Creating extra log directories... 2024-04-24T22:58:35.083 DEBUG:teuthology.orchestra.run.smithi032:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-24T22:58:35.124 DEBUG:teuthology.orchestra.run.smithi121:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-24T22:58:35.154 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-24T22:58:35.154 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-24T22:58:35.154 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-24T22:58:35.154 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T22:58:35.154 DEBUG:teuthology.orchestra.run.smithi032:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-24T22:58:35.201 DEBUG:teuthology.orchestra.run.smithi032:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-24T22:58:35.310 DEBUG:teuthology.orchestra.run.smithi032:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-24T22:58:35.334 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T22:58:35.335 DEBUG:teuthology.orchestra.run.smithi121:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-24T22:58:35.350 DEBUG:teuthology.orchestra.run.smithi121:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-24T22:58:35.459 DEBUG:teuthology.orchestra.run.smithi121:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-24T22:58:35.503 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-24T22:58:35.503 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL', 'PG_RECOVERY_FULL', 'PG_DEGRADED'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-24T22:58:35.504 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672173', 'branch': 'squid', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/mds-full}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7672173', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd', 'disabled']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL', 'PG_RECOVERY_FULL', 'PG_DEGRADED'], 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 8528, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'sleep_before_teardown': 0, 'subset': '122/512', 'suite': 'fs', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi032.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHYvII4S0kzwIW0jt40JM7Sembk40wVx3gw7EE3142Uuc/vtmIdcOdyv8dm8skJ8aREs6KWvYuorh9rmVcdC39s=', 'smithi121.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMjX2m8W9izlQuvyTJSgRUGOUlFR0yuouRl55SNghfJ83YhYUHKqDo1sbWUssrqbOSoImMIXTQAFNp6j13d1Kkg='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd', 'disabled']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL', 'PG_RECOVERY_FULL', 'PG_DEGRADED'], 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_full']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-24_21:24:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-24T22:58:35.504 DEBUG:teuthology.orchestra.run.smithi032:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-24T22:58:35.506 DEBUG:teuthology.orchestra.run.smithi121:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-24T22:58:35.553 DEBUG:teuthology.orchestra.run.smithi032:> sudo install -d -m0777 -- /var/run/ceph 2024-04-24T22:58:35.562 DEBUG:teuthology.orchestra.run.smithi121:> sudo install -d -m0777 -- /var/run/ceph 2024-04-24T22:58:35.617 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T22:58:35.617 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/scratch_devs of=/dev/stdout 2024-04-24T22:58:35.642 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-24T22:58:35.642 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_1 2024-04-24T22:58:35.698 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-24T22:58:35.739 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T22:58:35.740 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-24T22:58:35.740 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T22:58:35.740 INFO:teuthology.orchestra.run.smithi032.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T22:58:35.740 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2024-04-24 22:58:25.844224225 +0000 2024-04-24T22:58:35.740 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2024-04-24 22:56:35.117689546 +0000 2024-04-24T22:58:35.740 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2024-04-24 22:56:35.117689546 +0000 2024-04-24T22:58:35.740 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: 2024-04-24 22:56:35.117689546 +0000 2024-04-24T22:58:35.740 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-24T22:58:35.763 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2024-04-24T22:58:35.763 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2024-04-24T22:58:35.763 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 9.4206e-05 s, 5.4 MB/s 2024-04-24T22:58:35.764 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-24T22:58:35.821 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_2 2024-04-24T22:58:35.877 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-24T22:58:35.877 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T22:58:35.877 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-24T22:58:35.877 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T22:58:35.877 INFO:teuthology.orchestra.run.smithi032.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T22:58:35.877 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2024-04-24 22:58:25.844224225 +0000 2024-04-24T22:58:35.877 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2024-04-24 22:56:35.455682921 +0000 2024-04-24T22:58:35.877 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2024-04-24 22:56:35.455682921 +0000 2024-04-24T22:58:35.877 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: 2024-04-24 22:56:35.455682921 +0000 2024-04-24T22:58:35.878 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-24T22:58:35.941 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2024-04-24T22:58:35.942 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2024-04-24T22:58:35.942 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000101565 s, 5.0 MB/s 2024-04-24T22:58:35.942 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-24T22:58:35.998 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_3 2024-04-24T22:58:36.054 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-24T22:58:36.054 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T22:58:36.054 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-24T22:58:36.054 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T22:58:36.054 INFO:teuthology.orchestra.run.smithi032.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T22:58:36.054 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2024-04-24 22:58:25.845224205 +0000 2024-04-24T22:58:36.054 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2024-04-24 22:56:35.785676454 +0000 2024-04-24T22:58:36.054 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2024-04-24 22:56:35.785676454 +0000 2024-04-24T22:58:36.054 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: 2024-04-24 22:56:35.785676454 +0000 2024-04-24T22:58:36.055 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-24T22:58:36.117 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2024-04-24T22:58:36.117 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2024-04-24T22:58:36.117 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 9.4352e-05 s, 5.4 MB/s 2024-04-24T22:58:36.118 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-24T22:58:36.175 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_4 2024-04-24T22:58:36.230 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-24T22:58:36.230 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T22:58:36.230 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-24T22:58:36.230 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T22:58:36.230 INFO:teuthology.orchestra.run.smithi032.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T22:58:36.231 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2024-04-24 22:58:25.845224205 +0000 2024-04-24T22:58:36.231 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2024-04-24 22:56:36.130669693 +0000 2024-04-24T22:58:36.231 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2024-04-24 22:56:36.130669693 +0000 2024-04-24T22:58:36.231 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: 2024-04-24 22:56:36.130669693 +0000 2024-04-24T22:58:36.231 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-24T22:58:36.294 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2024-04-24T22:58:36.294 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2024-04-24T22:58:36.295 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 9.614e-05 s, 5.3 MB/s 2024-04-24T22:58:36.295 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-24T22:58:36.350 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-24T22:58:36.350 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T22:58:36.350 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/scratch_devs of=/dev/stdout 2024-04-24T22:58:36.367 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-24T22:58:36.368 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_1 2024-04-24T22:58:36.423 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-24T22:58:36.423 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T22:58:36.424 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-04-24T22:58:36.424 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T22:58:36.424 INFO:teuthology.orchestra.run.smithi121.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T22:58:36.424 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2024-04-24 22:58:20.093610402 +0000 2024-04-24T22:58:36.424 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2024-04-24 22:56:31.499307787 +0000 2024-04-24T22:58:36.424 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2024-04-24 22:56:31.499307787 +0000 2024-04-24T22:58:36.424 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: 2024-04-24 22:56:31.499307787 +0000 2024-04-24T22:58:36.424 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-24T22:58:36.488 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2024-04-24T22:58:36.488 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2024-04-24T22:58:36.488 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.000217 s, 2.4 MB/s 2024-04-24T22:58:36.490 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-24T22:58:36.547 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_2 2024-04-24T22:58:36.604 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-24T22:58:36.604 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T22:58:36.604 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-24T22:58:36.604 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T22:58:36.604 INFO:teuthology.orchestra.run.smithi121.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T22:58:36.604 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2024-04-24 22:58:20.093610402 +0000 2024-04-24T22:58:36.605 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2024-04-24 22:56:31.841303250 +0000 2024-04-24T22:58:36.605 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2024-04-24 22:56:31.841303250 +0000 2024-04-24T22:58:36.605 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: 2024-04-24 22:56:31.841303250 +0000 2024-04-24T22:58:36.605 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-24T22:58:36.667 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2024-04-24T22:58:36.668 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2024-04-24T22:58:36.668 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.000220763 s, 2.3 MB/s 2024-04-24T22:58:36.669 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-24T22:58:36.725 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_3 2024-04-24T22:58:36.781 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-24T22:58:36.781 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T22:58:36.781 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-24T22:58:36.781 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T22:58:36.781 INFO:teuthology.orchestra.run.smithi121.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T22:58:36.782 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2024-04-24 22:58:20.094610389 +0000 2024-04-24T22:58:36.782 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2024-04-24 22:56:32.191298608 +0000 2024-04-24T22:58:36.782 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2024-04-24 22:56:32.191298608 +0000 2024-04-24T22:58:36.782 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: 2024-04-24 22:56:32.191298608 +0000 2024-04-24T22:58:36.782 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-24T22:58:36.843 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2024-04-24T22:58:36.844 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2024-04-24T22:58:36.844 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.000187731 s, 2.7 MB/s 2024-04-24T22:58:36.845 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-24T22:58:36.901 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_4 2024-04-24T22:58:36.957 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-24T22:58:36.957 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T22:58:36.957 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-24T22:58:36.957 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T22:58:36.958 INFO:teuthology.orchestra.run.smithi121.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T22:58:36.958 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2024-04-24 22:58:20.094610389 +0000 2024-04-24T22:58:36.958 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2024-04-24 22:56:32.536294032 +0000 2024-04-24T22:58:36.958 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2024-04-24 22:56:32.536294032 +0000 2024-04-24T22:58:36.958 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: 2024-04-24 22:56:32.536294032 +0000 2024-04-24T22:58:36.959 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-24T22:58:37.019 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2024-04-24T22:58:37.020 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2024-04-24T22:58:37.020 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.000249618 s, 2.1 MB/s 2024-04-24T22:58:37.021 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-24T22:58:37.076 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-24T22:58:37.077 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi032.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi121.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-24T22:58:37.077 INFO:tasks.ceph:Generating config... 2024-04-24T22:58:37.078 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-24T22:58:37.078 INFO:tasks.ceph:[client] debug client = 20 2024-04-24T22:58:37.078 INFO:tasks.ceph:[client] debug ms = 1 2024-04-24T22:58:37.078 INFO:tasks.ceph:[client] pid file = 2024-04-24T22:58:37.078 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-24T22:58:37.078 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-24T22:58:37.078 INFO:tasks.ceph:[client.0] debug client = 20 2024-04-24T22:58:37.078 INFO:tasks.ceph:[client.0] debug objectcacher = 20 2024-04-24T22:58:37.079 INFO:tasks.ceph:[client.0] debug objecter = 20 2024-04-24T22:58:37.079 INFO:tasks.ceph:[client.1] debug client = 20 2024-04-24T22:58:37.079 INFO:tasks.ceph:[client.1] debug objectcacher = 20 2024-04-24T22:58:37.079 INFO:tasks.ceph:[client.1] debug objecter = 20 2024-04-24T22:58:37.079 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-04-24T22:58:37.079 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-24T22:58:37.079 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-24T22:58:37.079 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-24T22:58:37.079 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-24T22:58:37.079 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-24T22:58:37.079 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-24T22:58:37.079 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-24T22:58:37.079 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-24T22:58:37.079 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-24T22:58:37.079 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-24T22:58:37.080 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-24T22:58:37.080 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-24T22:58:37.080 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-24T22:58:37.080 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-24T22:58:37.080 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-24T22:58:37.080 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-24T22:58:37.080 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-24T22:58:37.080 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-24T22:58:37.080 INFO:tasks.ceph:[mon] mon osd backfillfull ratio = 0.6 2024-04-24T22:58:37.080 INFO:tasks.ceph:[mon] mon osd full ratio = 0.7 2024-04-24T22:58:37.080 INFO:tasks.ceph:[mon] mon osd nearfull ratio = 0.6 2024-04-24T22:58:37.080 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-24T22:58:37.080 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-24T22:58:37.080 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-04-24T22:58:37.080 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-04-24T22:58:37.081 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-24T22:58:37.081 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-24T22:58:37.081 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-04-24T22:58:37.081 INFO:tasks.ceph:[osd] memstore device bytes = 200000000 2024-04-24T22:58:37.081 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-24T22:58:37.081 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-24T22:58:37.081 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-24T22:58:37.081 INFO:tasks.ceph:[osd] osd failsafe full ratio = 1.0 2024-04-24T22:58:37.081 INFO:tasks.ceph:[osd] osd mon report interval = 5 2024-04-24T22:58:37.081 INFO:tasks.ceph:[osd] osd objectstore = memstore 2024-04-24T22:58:37.081 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-24T22:58:37.081 INFO:tasks.ceph:Setting up mon.a... 2024-04-24T22:58:37.082 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-24T22:58:37.143 INFO:teuthology.orchestra.run.smithi032.stdout:creating /etc/ceph/ceph.keyring 2024-04-24T22:58:37.146 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-24T22:58:37.189 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-24T22:58:37.255 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.32'), ('mon.b', '172.21.15.121'), ('mon.c', '[v2:172.21.15.121:3301,v1:172.21.15.121:6790]')] 2024-04-24T22:58:37.255 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.32,172.21.15.121,[v2:172.21.15.121:3301,v1:172.21.15.121:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-24T22:58:37.256 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T22:58:37.256 DEBUG:teuthology.orchestra.run.smithi032:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-24T22:58:37.311 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.32 --add b 172.21.15.121 --addv c '[v2:172.21.15.121:3301,v1:172.21.15.121:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-24T22:58:37.397 INFO:teuthology.orchestra.run.smithi032.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-24T22:58:37.397 INFO:teuthology.orchestra.run.smithi032.stdout:monmaptool: generated fsid eb1b7a01-6f65-4534-9a12-83011862ba51 2024-04-24T22:58:37.397 INFO:teuthology.orchestra.run.smithi032.stdout:setting min_mon_release = quincy 2024-04-24T22:58:37.397 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 0 2024-04-24T22:58:37.397 INFO:teuthology.orchestra.run.smithi032.stdout:fsid eb1b7a01-6f65-4534-9a12-83011862ba51 2024-04-24T22:58:37.397 INFO:teuthology.orchestra.run.smithi032.stdout:last_changed 2024-04-24T22:58:37.396824+0000 2024-04-24T22:58:37.397 INFO:teuthology.orchestra.run.smithi032.stdout:created 2024-04-24T22:58:37.396824+0000 2024-04-24T22:58:37.398 INFO:teuthology.orchestra.run.smithi032.stdout:min_mon_release 17 (quincy) 2024-04-24T22:58:37.398 INFO:teuthology.orchestra.run.smithi032.stdout:election_strategy: 1 2024-04-24T22:58:37.398 INFO:teuthology.orchestra.run.smithi032.stdout:0: [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] mon.a 2024-04-24T22:58:37.398 INFO:teuthology.orchestra.run.smithi032.stdout:1: [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] mon.b 2024-04-24T22:58:37.398 INFO:teuthology.orchestra.run.smithi032.stdout:2: [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] mon.c 2024-04-24T22:58:37.398 INFO:teuthology.orchestra.run.smithi032.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-24T22:58:37.399 DEBUG:teuthology.orchestra.run.smithi032:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-24T22:58:37.414 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID eb1b7a01-6f65-4534-9a12-83011862ba51... 2024-04-24T22:58:37.415 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-24T22:58:37.457 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-24T22:58:37.497 INFO:teuthology.orchestra.run.smithi032.stdout:[global] 2024-04-24T22:58:37.497 INFO:teuthology.orchestra.run.smithi032.stdout: chdir = "" 2024-04-24T22:58:37.497 INFO:teuthology.orchestra.run.smithi032.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-24T22:58:37.498 INFO:teuthology.orchestra.run.smithi032.stdout: auth supported = cephx 2024-04-24T22:58:37.498 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:58:37.498 INFO:teuthology.orchestra.run.smithi032.stdout: filestore xattr use omap = true 2024-04-24T22:58:37.498 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:58:37.498 INFO:teuthology.orchestra.run.smithi032.stdout: mon clock drift allowed = 1.000 2024-04-24T22:58:37.498 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:58:37.498 INFO:teuthology.orchestra.run.smithi032.stdout: osd crush chooseleaf type = 0 2024-04-24T22:58:37.498 INFO:teuthology.orchestra.run.smithi032.stdout: auth debug = true 2024-04-24T22:58:37.498 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:58:37.498 INFO:teuthology.orchestra.run.smithi032.stdout: ms die on old message = true 2024-04-24T22:58:37.498 INFO:teuthology.orchestra.run.smithi032.stdout: ms die on bug = true 2024-04-24T22:58:37.498 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:58:37.498 INFO:teuthology.orchestra.run.smithi032.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-24T22:58:37.498 INFO:teuthology.orchestra.run.smithi032.stdout: mon pg warn max object skew = 0 2024-04-24T22:58:37.499 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:58:37.499 INFO:teuthology.orchestra.run.smithi032.stdout: # disable pg_autoscaler by default for new pools 2024-04-24T22:58:37.499 INFO:teuthology.orchestra.run.smithi032.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-24T22:58:37.499 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:58:37.499 INFO:teuthology.orchestra.run.smithi032.stdout: osd pool default size = 2 2024-04-24T22:58:37.499 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:58:37.499 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd allow primary affinity = true 2024-04-24T22:58:37.499 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd allow pg remap = true 2024-04-24T22:58:37.499 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on legacy crush tunables = false 2024-04-24T22:58:37.499 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on crush straw calc version zero = false 2024-04-24T22:58:37.499 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on no sortbitwise = false 2024-04-24T22:58:37.499 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on osd down out interval zero = false 2024-04-24T22:58:37.499 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on too few osds = false 2024-04-24T22:58:37.499 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-24T22:58:37.500 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-24T22:58:37.500 INFO:teuthology.orchestra.run.smithi032.stdout: mon_allow_pool_size_one = true 2024-04-24T22:58:37.500 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:58:37.500 INFO:teuthology.orchestra.run.smithi032.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-24T22:58:37.500 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:58:37.500 INFO:teuthology.orchestra.run.smithi032.stdout: osd default data pool replay window = 5 2024-04-24T22:58:37.500 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:58:37.500 INFO:teuthology.orchestra.run.smithi032.stdout: mon allow pool delete = true 2024-04-24T22:58:37.500 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:58:37.500 INFO:teuthology.orchestra.run.smithi032.stdout: mon cluster log file level = debug 2024-04-24T22:58:37.500 INFO:teuthology.orchestra.run.smithi032.stdout: debug asserts on shutdown = true 2024-04-24T22:58:37.500 INFO:teuthology.orchestra.run.smithi032.stdout: mon health detail to clog = false 2024-04-24T22:58:37.500 INFO:teuthology.orchestra.run.smithi032.stdout: mon host = "172.21.15.32,172.21.15.121,[v2:172.21.15.121:3301,v1:172.21.15.121:6790]" 2024-04-24T22:58:37.500 INFO:teuthology.orchestra.run.smithi032.stdout: ms die on skipped message = False 2024-04-24T22:58:37.500 INFO:teuthology.orchestra.run.smithi032.stdout: fsid = eb1b7a01-6f65-4534-9a12-83011862ba51 2024-04-24T22:58:37.501 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:58:37.501 INFO:teuthology.orchestra.run.smithi032.stdout:[osd] 2024-04-24T22:58:37.501 INFO:teuthology.orchestra.run.smithi032.stdout: osd journal size = 100 2024-04-24T22:58:37.501 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:58:37.501 INFO:teuthology.orchestra.run.smithi032.stdout: osd scrub load threshold = 5.0 2024-04-24T22:58:37.501 INFO:teuthology.orchestra.run.smithi032.stdout: osd scrub max interval = 600 2024-04-24T22:58:37.501 INFO:teuthology.orchestra.run.smithi032.stdout: osd mclock profile = high_recovery_ops 2024-04-24T22:58:37.501 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:58:37.501 INFO:teuthology.orchestra.run.smithi032.stdout: osd recover clone overlap = true 2024-04-24T22:58:37.501 INFO:teuthology.orchestra.run.smithi032.stdout: osd recovery max chunk = 1048576 2024-04-24T22:58:37.501 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:58:37.501 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug shutdown = true 2024-04-24T22:58:37.501 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug op order = true 2024-04-24T22:58:37.501 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug verify stray on activate = true 2024-04-24T22:58:37.501 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug trim objects = true 2024-04-24T22:58:37.502 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:58:37.502 INFO:teuthology.orchestra.run.smithi032.stdout: osd open classes on start = true 2024-04-24T22:58:37.502 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug pg log writeout = true 2024-04-24T22:58:37.502 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:58:37.502 INFO:teuthology.orchestra.run.smithi032.stdout: osd deep scrub update digest min age = 30 2024-04-24T22:58:37.502 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:58:37.502 INFO:teuthology.orchestra.run.smithi032.stdout: osd map max advance = 10 2024-04-24T22:58:37.502 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:58:37.502 INFO:teuthology.orchestra.run.smithi032.stdout: journal zero on create = true 2024-04-24T22:58:37.502 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:58:37.502 INFO:teuthology.orchestra.run.smithi032.stdout: filestore ondisk finisher threads = 3 2024-04-24T22:58:37.502 INFO:teuthology.orchestra.run.smithi032.stdout: filestore apply finisher threads = 3 2024-04-24T22:58:37.502 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:58:37.502 INFO:teuthology.orchestra.run.smithi032.stdout: bdev debug aio = true 2024-04-24T22:58:37.503 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug misdirected ops = true 2024-04-24T22:58:37.503 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore block size = 96636764160 2024-04-24T22:58:37.503 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore fsck on mount = True 2024-04-24T22:58:37.503 INFO:teuthology.orchestra.run.smithi032.stdout: debug bluefs = 20 2024-04-24T22:58:37.503 INFO:teuthology.orchestra.run.smithi032.stdout: debug bluestore = 20 2024-04-24T22:58:37.503 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2024-04-24T22:58:37.503 INFO:teuthology.orchestra.run.smithi032.stdout: debug osd = 20 2024-04-24T22:58:37.503 INFO:teuthology.orchestra.run.smithi032.stdout: debug rocksdb = 10 2024-04-24T22:58:37.503 INFO:teuthology.orchestra.run.smithi032.stdout: memstore device bytes = 200000000 2024-04-24T22:58:37.503 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd backfillfull_ratio = 0.85 2024-04-24T22:58:37.503 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd full ratio = 0.9 2024-04-24T22:58:37.503 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd nearfull ratio = 0.8 2024-04-24T22:58:37.503 INFO:teuthology.orchestra.run.smithi032.stdout: osd failsafe full ratio = 1.0 2024-04-24T22:58:37.503 INFO:teuthology.orchestra.run.smithi032.stdout: osd mon report interval = 5 2024-04-24T22:58:37.504 INFO:teuthology.orchestra.run.smithi032.stdout: osd objectstore = memstore 2024-04-24T22:58:37.504 INFO:teuthology.orchestra.run.smithi032.stdout: osd op complaint time = 180 2024-04-24T22:58:37.504 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:58:37.504 INFO:teuthology.orchestra.run.smithi032.stdout:[mgr] 2024-04-24T22:58:37.504 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2024-04-24T22:58:37.504 INFO:teuthology.orchestra.run.smithi032.stdout: debug mgr = 20 2024-04-24T22:58:37.504 INFO:teuthology.orchestra.run.smithi032.stdout: debug mon = 20 2024-04-24T22:58:37.504 INFO:teuthology.orchestra.run.smithi032.stdout: debug auth = 20 2024-04-24T22:58:37.504 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min pgs per osd = 4 2024-04-24T22:58:37.504 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min bytes per osd = 10 2024-04-24T22:58:37.504 INFO:teuthology.orchestra.run.smithi032.stdout: mgr/telemetry/nag = false 2024-04-24T22:58:37.504 INFO:teuthology.orchestra.run.smithi032.stdout: debug client = 20 2024-04-24T22:58:37.504 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:58:37.504 INFO:teuthology.orchestra.run.smithi032.stdout:[mon] 2024-04-24T22:58:37.504 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2024-04-24T22:58:37.505 INFO:teuthology.orchestra.run.smithi032.stdout: debug mon = 20 2024-04-24T22:58:37.505 INFO:teuthology.orchestra.run.smithi032.stdout: debug paxos = 20 2024-04-24T22:58:37.505 INFO:teuthology.orchestra.run.smithi032.stdout: debug auth = 20 2024-04-24T22:58:37.505 INFO:teuthology.orchestra.run.smithi032.stdout: mon data avail warn = 5 2024-04-24T22:58:37.505 INFO:teuthology.orchestra.run.smithi032.stdout: mon mgr mkfs grace = 240 2024-04-24T22:58:37.505 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min pgs per osd = 4 2024-04-24T22:58:37.505 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd reporter subtree level = osd 2024-04-24T22:58:37.505 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd prime pg temp = true 2024-04-24T22:58:37.505 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min bytes per osd = 10 2024-04-24T22:58:37.505 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:58:37.505 INFO:teuthology.orchestra.run.smithi032.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-24T22:58:37.505 INFO:teuthology.orchestra.run.smithi032.stdout: auth mon ticket ttl = 660 # 11m 2024-04-24T22:58:37.505 INFO:teuthology.orchestra.run.smithi032.stdout: auth service ticket ttl = 240 # 4m 2024-04-24T22:58:37.505 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:58:37.505 INFO:teuthology.orchestra.run.smithi032.stdout: # don't complain about insecure global_id in the test suite 2024-04-24T22:58:37.506 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-24T22:58:37.506 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-24T22:58:37.506 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:58:37.506 INFO:teuthology.orchestra.run.smithi032.stdout: # 1m isn't quite enough 2024-04-24T22:58:37.506 INFO:teuthology.orchestra.run.smithi032.stdout: mon_down_mkfs_grace = 2m 2024-04-24T22:58:37.506 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:58:37.506 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_filestore_osds = false 2024-04-24T22:58:37.506 INFO:teuthology.orchestra.run.smithi032.stdout: mon down mkfs grace = 300 2024-04-24T22:58:37.506 INFO:teuthology.orchestra.run.smithi032.stdout: mon op complaint time = 120 2024-04-24T22:58:37.506 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd backfillfull ratio = 0.6 2024-04-24T22:58:37.506 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd full ratio = 0.7 2024-04-24T22:58:37.506 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd nearfull ratio = 0.6 2024-04-24T22:58:37.506 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:58:37.507 INFO:teuthology.orchestra.run.smithi032.stdout:[client] 2024-04-24T22:58:37.507 INFO:teuthology.orchestra.run.smithi032.stdout: rgw cache enabled = true 2024-04-24T22:58:37.507 INFO:teuthology.orchestra.run.smithi032.stdout: rgw enable ops log = true 2024-04-24T22:58:37.507 INFO:teuthology.orchestra.run.smithi032.stdout: rgw enable usage log = true 2024-04-24T22:58:37.507 INFO:teuthology.orchestra.run.smithi032.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-24T22:58:37.507 INFO:teuthology.orchestra.run.smithi032.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-24T22:58:37.507 INFO:teuthology.orchestra.run.smithi032.stdout: client mount timeout = 600 2024-04-24T22:58:37.507 INFO:teuthology.orchestra.run.smithi032.stdout: debug client = 20 2024-04-24T22:58:37.507 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2024-04-24T22:58:37.507 INFO:teuthology.orchestra.run.smithi032.stdout: pid file = "" 2024-04-24T22:58:37.507 INFO:teuthology.orchestra.run.smithi032.stdout: rados mon op timeout = 900 2024-04-24T22:58:37.507 INFO:teuthology.orchestra.run.smithi032.stdout: rados osd op timeout = 900 2024-04-24T22:58:37.507 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.a] 2024-04-24T22:58:37.507 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.b] 2024-04-24T22:58:37.508 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.c] 2024-04-24T22:58:37.508 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.a] 2024-04-24T22:58:37.508 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.c] 2024-04-24T22:58:37.508 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.b] 2024-04-24T22:58:37.508 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.d] 2024-04-24T22:58:37.508 INFO:teuthology.orchestra.run.smithi032.stdout:[client.0] 2024-04-24T22:58:37.508 INFO:teuthology.orchestra.run.smithi032.stdout: debug client = 20 2024-04-24T22:58:37.508 INFO:teuthology.orchestra.run.smithi032.stdout: debug objectcacher = 20 2024-04-24T22:58:37.508 INFO:teuthology.orchestra.run.smithi032.stdout: debug objecter = 20 2024-04-24T22:58:37.508 INFO:teuthology.orchestra.run.smithi032.stdout:[client.1] 2024-04-24T22:58:37.508 INFO:teuthology.orchestra.run.smithi032.stdout: debug client = 20 2024-04-24T22:58:37.508 INFO:teuthology.orchestra.run.smithi032.stdout: debug objectcacher = 20 2024-04-24T22:58:37.508 INFO:teuthology.orchestra.run.smithi032.stdout: debug objecter = 20 2024-04-24T22:58:37.508 INFO:teuthology.orchestra.run.smithi032.stdout:[mds] 2024-04-24T22:58:37.508 INFO:teuthology.orchestra.run.smithi032.stdout: debug mds = 20 2024-04-24T22:58:37.509 INFO:teuthology.orchestra.run.smithi032.stdout: debug mds balancer = 20 2024-04-24T22:58:37.509 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2024-04-24T22:58:37.509 INFO:teuthology.orchestra.run.smithi032.stdout: mds debug frag = True 2024-04-24T22:58:37.509 INFO:teuthology.orchestra.run.smithi032.stdout: mds debug scatterstat = True 2024-04-24T22:58:37.509 INFO:teuthology.orchestra.run.smithi032.stdout: mds op complaint time = 180 2024-04-24T22:58:37.509 INFO:teuthology.orchestra.run.smithi032.stdout: mds verify scatter = True 2024-04-24T22:58:37.509 INFO:teuthology.orchestra.run.smithi032.stdout: osd op complaint time = 180 2024-04-24T22:58:37.509 INFO:teuthology.orchestra.run.smithi032.stdout: rados mon op timeout = 900 2024-04-24T22:58:37.509 INFO:teuthology.orchestra.run.smithi032.stdout: rados osd op timeout = 900 2024-04-24T22:58:37.511 INFO:teuthology.orchestra.run.smithi121.stdout:[global] 2024-04-24T22:58:37.511 INFO:teuthology.orchestra.run.smithi121.stdout: chdir = "" 2024-04-24T22:58:37.511 INFO:teuthology.orchestra.run.smithi121.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-24T22:58:37.511 INFO:teuthology.orchestra.run.smithi121.stdout: auth supported = cephx 2024-04-24T22:58:37.511 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-24T22:58:37.511 INFO:teuthology.orchestra.run.smithi121.stdout: filestore xattr use omap = true 2024-04-24T22:58:37.511 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-24T22:58:37.511 INFO:teuthology.orchestra.run.smithi121.stdout: mon clock drift allowed = 1.000 2024-04-24T22:58:37.511 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-24T22:58:37.511 INFO:teuthology.orchestra.run.smithi121.stdout: osd crush chooseleaf type = 0 2024-04-24T22:58:37.511 INFO:teuthology.orchestra.run.smithi121.stdout: auth debug = true 2024-04-24T22:58:37.511 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-24T22:58:37.511 INFO:teuthology.orchestra.run.smithi121.stdout: ms die on old message = true 2024-04-24T22:58:37.512 INFO:teuthology.orchestra.run.smithi121.stdout: ms die on bug = true 2024-04-24T22:58:37.512 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-24T22:58:37.512 INFO:teuthology.orchestra.run.smithi121.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-24T22:58:37.512 INFO:teuthology.orchestra.run.smithi121.stdout: mon pg warn max object skew = 0 2024-04-24T22:58:37.512 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-24T22:58:37.512 INFO:teuthology.orchestra.run.smithi121.stdout: # disable pg_autoscaler by default for new pools 2024-04-24T22:58:37.512 INFO:teuthology.orchestra.run.smithi121.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-24T22:58:37.512 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-24T22:58:37.512 INFO:teuthology.orchestra.run.smithi121.stdout: osd pool default size = 2 2024-04-24T22:58:37.512 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-24T22:58:37.512 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd allow primary affinity = true 2024-04-24T22:58:37.512 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd allow pg remap = true 2024-04-24T22:58:37.512 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on legacy crush tunables = false 2024-04-24T22:58:37.512 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on crush straw calc version zero = false 2024-04-24T22:58:37.512 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on no sortbitwise = false 2024-04-24T22:58:37.513 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on osd down out interval zero = false 2024-04-24T22:58:37.513 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on too few osds = false 2024-04-24T22:58:37.513 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-24T22:58:37.513 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-24T22:58:37.513 INFO:teuthology.orchestra.run.smithi121.stdout: mon_allow_pool_size_one = true 2024-04-24T22:58:37.513 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-24T22:58:37.513 INFO:teuthology.orchestra.run.smithi121.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-24T22:58:37.513 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-24T22:58:37.513 INFO:teuthology.orchestra.run.smithi121.stdout: osd default data pool replay window = 5 2024-04-24T22:58:37.513 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-24T22:58:37.513 INFO:teuthology.orchestra.run.smithi121.stdout: mon allow pool delete = true 2024-04-24T22:58:37.513 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-24T22:58:37.513 INFO:teuthology.orchestra.run.smithi121.stdout: mon cluster log file level = debug 2024-04-24T22:58:37.513 INFO:teuthology.orchestra.run.smithi121.stdout: debug asserts on shutdown = true 2024-04-24T22:58:37.513 INFO:teuthology.orchestra.run.smithi121.stdout: mon health detail to clog = false 2024-04-24T22:58:37.514 INFO:teuthology.orchestra.run.smithi121.stdout: mon host = "172.21.15.32,172.21.15.121,[v2:172.21.15.121:3301,v1:172.21.15.121:6790]" 2024-04-24T22:58:37.514 INFO:teuthology.orchestra.run.smithi121.stdout: ms die on skipped message = False 2024-04-24T22:58:37.514 INFO:teuthology.orchestra.run.smithi121.stdout: fsid = eb1b7a01-6f65-4534-9a12-83011862ba51 2024-04-24T22:58:37.514 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-24T22:58:37.514 INFO:teuthology.orchestra.run.smithi121.stdout:[osd] 2024-04-24T22:58:37.514 INFO:teuthology.orchestra.run.smithi121.stdout: osd journal size = 100 2024-04-24T22:58:37.514 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-24T22:58:37.514 INFO:teuthology.orchestra.run.smithi121.stdout: osd scrub load threshold = 5.0 2024-04-24T22:58:37.514 INFO:teuthology.orchestra.run.smithi121.stdout: osd scrub max interval = 600 2024-04-24T22:58:37.514 INFO:teuthology.orchestra.run.smithi121.stdout: osd mclock profile = high_recovery_ops 2024-04-24T22:58:37.514 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-24T22:58:37.514 INFO:teuthology.orchestra.run.smithi121.stdout: osd recover clone overlap = true 2024-04-24T22:58:37.514 INFO:teuthology.orchestra.run.smithi121.stdout: osd recovery max chunk = 1048576 2024-04-24T22:58:37.514 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-24T22:58:37.514 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug shutdown = true 2024-04-24T22:58:37.515 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug op order = true 2024-04-24T22:58:37.515 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug verify stray on activate = true 2024-04-24T22:58:37.515 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug trim objects = true 2024-04-24T22:58:37.515 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-24T22:58:37.515 INFO:teuthology.orchestra.run.smithi121.stdout: osd open classes on start = true 2024-04-24T22:58:37.515 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug pg log writeout = true 2024-04-24T22:58:37.515 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-24T22:58:37.515 INFO:teuthology.orchestra.run.smithi121.stdout: osd deep scrub update digest min age = 30 2024-04-24T22:58:37.515 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-24T22:58:37.515 INFO:teuthology.orchestra.run.smithi121.stdout: osd map max advance = 10 2024-04-24T22:58:37.515 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-24T22:58:37.515 INFO:teuthology.orchestra.run.smithi121.stdout: journal zero on create = true 2024-04-24T22:58:37.515 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-24T22:58:37.515 INFO:teuthology.orchestra.run.smithi121.stdout: filestore ondisk finisher threads = 3 2024-04-24T22:58:37.515 INFO:teuthology.orchestra.run.smithi121.stdout: filestore apply finisher threads = 3 2024-04-24T22:58:37.516 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-24T22:58:37.516 INFO:teuthology.orchestra.run.smithi121.stdout: bdev debug aio = true 2024-04-24T22:58:37.516 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug misdirected ops = true 2024-04-24T22:58:37.516 INFO:teuthology.orchestra.run.smithi121.stdout: bluestore block size = 96636764160 2024-04-24T22:58:37.516 INFO:teuthology.orchestra.run.smithi121.stdout: bluestore fsck on mount = True 2024-04-24T22:58:37.516 INFO:teuthology.orchestra.run.smithi121.stdout: debug bluefs = 20 2024-04-24T22:58:37.516 INFO:teuthology.orchestra.run.smithi121.stdout: debug bluestore = 20 2024-04-24T22:58:37.516 INFO:teuthology.orchestra.run.smithi121.stdout: debug ms = 1 2024-04-24T22:58:37.516 INFO:teuthology.orchestra.run.smithi121.stdout: debug osd = 20 2024-04-24T22:58:37.516 INFO:teuthology.orchestra.run.smithi121.stdout: debug rocksdb = 10 2024-04-24T22:58:37.516 INFO:teuthology.orchestra.run.smithi121.stdout: memstore device bytes = 200000000 2024-04-24T22:58:37.516 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd backfillfull_ratio = 0.85 2024-04-24T22:58:37.516 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd full ratio = 0.9 2024-04-24T22:58:37.516 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd nearfull ratio = 0.8 2024-04-24T22:58:37.516 INFO:teuthology.orchestra.run.smithi121.stdout: osd failsafe full ratio = 1.0 2024-04-24T22:58:37.517 INFO:teuthology.orchestra.run.smithi121.stdout: osd mon report interval = 5 2024-04-24T22:58:37.517 INFO:teuthology.orchestra.run.smithi121.stdout: osd objectstore = memstore 2024-04-24T22:58:37.517 INFO:teuthology.orchestra.run.smithi121.stdout: osd op complaint time = 180 2024-04-24T22:58:37.517 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-24T22:58:37.517 INFO:teuthology.orchestra.run.smithi121.stdout:[mgr] 2024-04-24T22:58:37.517 INFO:teuthology.orchestra.run.smithi121.stdout: debug ms = 1 2024-04-24T22:58:37.517 INFO:teuthology.orchestra.run.smithi121.stdout: debug mgr = 20 2024-04-24T22:58:37.517 INFO:teuthology.orchestra.run.smithi121.stdout: debug mon = 20 2024-04-24T22:58:37.517 INFO:teuthology.orchestra.run.smithi121.stdout: debug auth = 20 2024-04-24T22:58:37.517 INFO:teuthology.orchestra.run.smithi121.stdout: mon reweight min pgs per osd = 4 2024-04-24T22:58:37.517 INFO:teuthology.orchestra.run.smithi121.stdout: mon reweight min bytes per osd = 10 2024-04-24T22:58:37.517 INFO:teuthology.orchestra.run.smithi121.stdout: mgr/telemetry/nag = false 2024-04-24T22:58:37.517 INFO:teuthology.orchestra.run.smithi121.stdout: debug client = 20 2024-04-24T22:58:37.517 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-24T22:58:37.517 INFO:teuthology.orchestra.run.smithi121.stdout:[mon] 2024-04-24T22:58:37.518 INFO:teuthology.orchestra.run.smithi121.stdout: debug ms = 1 2024-04-24T22:58:37.518 INFO:teuthology.orchestra.run.smithi121.stdout: debug mon = 20 2024-04-24T22:58:37.518 INFO:teuthology.orchestra.run.smithi121.stdout: debug paxos = 20 2024-04-24T22:58:37.518 INFO:teuthology.orchestra.run.smithi121.stdout: debug auth = 20 2024-04-24T22:58:37.518 INFO:teuthology.orchestra.run.smithi121.stdout: mon data avail warn = 5 2024-04-24T22:58:37.518 INFO:teuthology.orchestra.run.smithi121.stdout: mon mgr mkfs grace = 240 2024-04-24T22:58:37.518 INFO:teuthology.orchestra.run.smithi121.stdout: mon reweight min pgs per osd = 4 2024-04-24T22:58:37.518 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd reporter subtree level = osd 2024-04-24T22:58:37.518 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd prime pg temp = true 2024-04-24T22:58:37.518 INFO:teuthology.orchestra.run.smithi121.stdout: mon reweight min bytes per osd = 10 2024-04-24T22:58:37.518 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-24T22:58:37.518 INFO:teuthology.orchestra.run.smithi121.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-24T22:58:37.518 INFO:teuthology.orchestra.run.smithi121.stdout: auth mon ticket ttl = 660 # 11m 2024-04-24T22:58:37.518 INFO:teuthology.orchestra.run.smithi121.stdout: auth service ticket ttl = 240 # 4m 2024-04-24T22:58:37.518 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-24T22:58:37.519 INFO:teuthology.orchestra.run.smithi121.stdout: # don't complain about insecure global_id in the test suite 2024-04-24T22:58:37.519 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-24T22:58:37.519 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-24T22:58:37.519 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-24T22:58:37.519 INFO:teuthology.orchestra.run.smithi121.stdout: # 1m isn't quite enough 2024-04-24T22:58:37.519 INFO:teuthology.orchestra.run.smithi121.stdout: mon_down_mkfs_grace = 2m 2024-04-24T22:58:37.519 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-24T22:58:37.519 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_filestore_osds = false 2024-04-24T22:58:37.519 INFO:teuthology.orchestra.run.smithi121.stdout: mon down mkfs grace = 300 2024-04-24T22:58:37.519 INFO:teuthology.orchestra.run.smithi121.stdout: mon op complaint time = 120 2024-04-24T22:58:37.519 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd backfillfull ratio = 0.6 2024-04-24T22:58:37.519 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd full ratio = 0.7 2024-04-24T22:58:37.519 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd nearfull ratio = 0.6 2024-04-24T22:58:37.519 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-24T22:58:37.519 INFO:teuthology.orchestra.run.smithi121.stdout:[client] 2024-04-24T22:58:37.520 INFO:teuthology.orchestra.run.smithi121.stdout: rgw cache enabled = true 2024-04-24T22:58:37.520 INFO:teuthology.orchestra.run.smithi121.stdout: rgw enable ops log = true 2024-04-24T22:58:37.520 INFO:teuthology.orchestra.run.smithi121.stdout: rgw enable usage log = true 2024-04-24T22:58:37.520 INFO:teuthology.orchestra.run.smithi121.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-24T22:58:37.520 INFO:teuthology.orchestra.run.smithi121.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-24T22:58:37.520 INFO:teuthology.orchestra.run.smithi121.stdout: client mount timeout = 600 2024-04-24T22:58:37.520 INFO:teuthology.orchestra.run.smithi121.stdout: debug client = 20 2024-04-24T22:58:37.520 INFO:teuthology.orchestra.run.smithi121.stdout: debug ms = 1 2024-04-24T22:58:37.520 INFO:teuthology.orchestra.run.smithi121.stdout: pid file = "" 2024-04-24T22:58:37.520 INFO:teuthology.orchestra.run.smithi121.stdout: rados mon op timeout = 900 2024-04-24T22:58:37.520 INFO:teuthology.orchestra.run.smithi121.stdout: rados osd op timeout = 900 2024-04-24T22:58:37.520 INFO:teuthology.orchestra.run.smithi121.stdout:[mon.a] 2024-04-24T22:58:37.520 INFO:teuthology.orchestra.run.smithi121.stdout:[mon.b] 2024-04-24T22:58:37.520 INFO:teuthology.orchestra.run.smithi121.stdout:[mon.c] 2024-04-24T22:58:37.521 INFO:teuthology.orchestra.run.smithi121.stdout:[mds.a] 2024-04-24T22:58:37.521 INFO:teuthology.orchestra.run.smithi121.stdout:[mds.c] 2024-04-24T22:58:37.521 INFO:teuthology.orchestra.run.smithi121.stdout:[mds.b] 2024-04-24T22:58:37.521 INFO:teuthology.orchestra.run.smithi121.stdout:[mds.d] 2024-04-24T22:58:37.521 INFO:teuthology.orchestra.run.smithi121.stdout:[client.0] 2024-04-24T22:58:37.521 INFO:teuthology.orchestra.run.smithi121.stdout: debug client = 20 2024-04-24T22:58:37.521 INFO:teuthology.orchestra.run.smithi121.stdout: debug objectcacher = 20 2024-04-24T22:58:37.521 INFO:teuthology.orchestra.run.smithi121.stdout: debug objecter = 20 2024-04-24T22:58:37.521 INFO:teuthology.orchestra.run.smithi121.stdout:[client.1] 2024-04-24T22:58:37.521 INFO:teuthology.orchestra.run.smithi121.stdout: debug client = 20 2024-04-24T22:58:37.521 INFO:teuthology.orchestra.run.smithi121.stdout: debug objectcacher = 20 2024-04-24T22:58:37.521 INFO:teuthology.orchestra.run.smithi121.stdout: debug objecter = 20 2024-04-24T22:58:37.521 INFO:teuthology.orchestra.run.smithi121.stdout:[mds] 2024-04-24T22:58:37.521 INFO:teuthology.orchestra.run.smithi121.stdout: debug mds = 20 2024-04-24T22:58:37.521 INFO:teuthology.orchestra.run.smithi121.stdout: debug mds balancer = 20 2024-04-24T22:58:37.521 INFO:teuthology.orchestra.run.smithi121.stdout: debug ms = 1 2024-04-24T22:58:37.522 INFO:teuthology.orchestra.run.smithi121.stdout: mds debug frag = True 2024-04-24T22:58:37.522 INFO:teuthology.orchestra.run.smithi121.stdout: mds debug scatterstat = True 2024-04-24T22:58:37.522 INFO:teuthology.orchestra.run.smithi121.stdout: mds op complaint time = 180 2024-04-24T22:58:37.522 INFO:teuthology.orchestra.run.smithi121.stdout: mds verify scatter = True 2024-04-24T22:58:37.522 INFO:teuthology.orchestra.run.smithi121.stdout: osd op complaint time = 180 2024-04-24T22:58:37.522 INFO:teuthology.orchestra.run.smithi121.stdout: rados mon op timeout = 900 2024-04-24T22:58:37.522 INFO:teuthology.orchestra.run.smithi121.stdout: rados osd op timeout = 900 2024-04-24T22:58:37.522 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-24T22:58:37.522 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-24T22:58:37.604 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-24T22:58:37.605 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T22:58:37.605 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-24T22:58:37.620 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T22:58:37.620 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-24T22:58:37.675 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi032.front.sepia.ceph.com 2024-04-24T22:58:37.675 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T22:58:37.675 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-24T22:58:37.675 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-24T22:58:37.749 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T22:58:37.749 DEBUG:teuthology.orchestra.run.smithi032:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-24T22:58:37.804 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi121.front.sepia.ceph.com 2024-04-24T22:58:37.804 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T22:58:37.804 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-24T22:58:37.804 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-24T22:58:37.838 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T22:58:37.838 DEBUG:teuthology.orchestra.run.smithi121:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-24T22:58:37.893 INFO:tasks.ceph:Setting up mon nodes... 2024-04-24T22:58:37.893 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-24T22:58:37.893 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-24T22:58:37.943 INFO:teuthology.orchestra.run.smithi032.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-24T22:58:37.946 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-24T22:58:38.005 INFO:teuthology.orchestra.run.smithi121.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-24T22:58:38.008 INFO:tasks.ceph:Setting up mds nodes... 2024-04-24T22:58:38.008 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-24T22:58:38.058 INFO:teuthology.orchestra.run.smithi032.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-24T22:58:38.060 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-24T22:58:38.084 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-24T22:58:38.174 INFO:teuthology.orchestra.run.smithi032.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-24T22:58:38.176 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-24T22:58:38.201 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-24T22:58:38.249 INFO:teuthology.orchestra.run.smithi121.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-24T22:58:38.252 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-24T22:58:38.276 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-24T22:58:38.364 INFO:teuthology.orchestra.run.smithi121.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-24T22:58:38.366 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-24T22:58:38.390 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-24T22:58:38.390 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-24T22:58:38.430 INFO:teuthology.orchestra.run.smithi032.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-24T22:58:38.442 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-24T22:58:38.522 INFO:teuthology.orchestra.run.smithi032.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-24T22:58:38.536 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-24T22:58:38.575 INFO:teuthology.orchestra.run.smithi121.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-24T22:58:38.586 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-24T22:58:38.665 INFO:teuthology.orchestra.run.smithi121.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-24T22:58:38.677 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-24T22:58:38.677 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi032.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi121.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-24T22:58:38.677 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-24T22:58:38.701 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-24T22:58:38.701 INFO:tasks.ceph:role: osd.0 2024-04-24T22:58:38.701 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi032.front.sepia.ceph.com 2024-04-24T22:58:38.701 DEBUG:teuthology.orchestra.run.smithi032:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-24T22:58:38.765 INFO:teuthology.orchestra.run.smithi032.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T22:58:38.766 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T22:58:38.766 INFO:teuthology.orchestra.run.smithi032.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T22:58:38.766 INFO:teuthology.orchestra.run.smithi032.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T22:58:38.766 INFO:teuthology.orchestra.run.smithi032.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T22:58:38.766 INFO:teuthology.orchestra.run.smithi032.stdout: = sunit=0 swidth=0 blks 2024-04-24T22:58:38.767 INFO:teuthology.orchestra.run.smithi032.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T22:58:38.767 INFO:teuthology.orchestra.run.smithi032.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T22:58:38.767 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T22:58:38.767 INFO:teuthology.orchestra.run.smithi032.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T22:58:38.798 INFO:teuthology.orchestra.run.smithi032.stdout:Discarding blocks...Done. 2024-04-24T22:58:38.800 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi032.front.sepia.ceph.com -o noatime 2024-04-24T22:58:38.800 DEBUG:teuthology.orchestra.run.smithi032:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-24T22:58:38.870 DEBUG:teuthology.orchestra.run.smithi032:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-24T22:58:38.939 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-24T22:58:39.004 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-24T22:58:39.004 INFO:tasks.ceph:role: osd.1 2024-04-24T22:58:39.004 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi032.front.sepia.ceph.com 2024-04-24T22:58:39.005 DEBUG:teuthology.orchestra.run.smithi032:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-24T22:58:39.068 INFO:teuthology.orchestra.run.smithi032.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T22:58:39.069 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T22:58:39.069 INFO:teuthology.orchestra.run.smithi032.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T22:58:39.069 INFO:teuthology.orchestra.run.smithi032.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T22:58:39.069 INFO:teuthology.orchestra.run.smithi032.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T22:58:39.069 INFO:teuthology.orchestra.run.smithi032.stdout: = sunit=0 swidth=0 blks 2024-04-24T22:58:39.069 INFO:teuthology.orchestra.run.smithi032.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T22:58:39.069 INFO:teuthology.orchestra.run.smithi032.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T22:58:39.069 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T22:58:39.069 INFO:teuthology.orchestra.run.smithi032.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T22:58:39.101 INFO:teuthology.orchestra.run.smithi032.stdout:Discarding blocks...Done. 2024-04-24T22:58:39.103 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi032.front.sepia.ceph.com -o noatime 2024-04-24T22:58:39.104 DEBUG:teuthology.orchestra.run.smithi032:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-24T22:58:39.176 DEBUG:teuthology.orchestra.run.smithi032:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-24T22:58:39.245 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-24T22:58:39.310 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-24T22:58:39.310 INFO:tasks.ceph:role: osd.2 2024-04-24T22:58:39.311 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi032.front.sepia.ceph.com 2024-04-24T22:58:39.311 DEBUG:teuthology.orchestra.run.smithi032:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-24T22:58:39.375 INFO:teuthology.orchestra.run.smithi032.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T22:58:39.375 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T22:58:39.375 INFO:teuthology.orchestra.run.smithi032.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T22:58:39.375 INFO:teuthology.orchestra.run.smithi032.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T22:58:39.375 INFO:teuthology.orchestra.run.smithi032.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T22:58:39.375 INFO:teuthology.orchestra.run.smithi032.stdout: = sunit=0 swidth=0 blks 2024-04-24T22:58:39.376 INFO:teuthology.orchestra.run.smithi032.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T22:58:39.376 INFO:teuthology.orchestra.run.smithi032.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T22:58:39.376 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T22:58:39.376 INFO:teuthology.orchestra.run.smithi032.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T22:58:39.408 INFO:teuthology.orchestra.run.smithi032.stdout:Discarding blocks...Done. 2024-04-24T22:58:39.410 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi032.front.sepia.ceph.com -o noatime 2024-04-24T22:58:39.411 DEBUG:teuthology.orchestra.run.smithi032:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-24T22:58:39.479 DEBUG:teuthology.orchestra.run.smithi032:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-24T22:58:39.547 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-24T22:58:39.612 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-24T22:58:39.612 INFO:tasks.ceph:role: osd.3 2024-04-24T22:58:39.613 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi032.front.sepia.ceph.com 2024-04-24T22:58:39.613 DEBUG:teuthology.orchestra.run.smithi032:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-24T22:58:39.676 INFO:teuthology.orchestra.run.smithi032.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T22:58:39.676 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T22:58:39.676 INFO:teuthology.orchestra.run.smithi032.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T22:58:39.676 INFO:teuthology.orchestra.run.smithi032.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T22:58:39.677 INFO:teuthology.orchestra.run.smithi032.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T22:58:39.677 INFO:teuthology.orchestra.run.smithi032.stdout: = sunit=0 swidth=0 blks 2024-04-24T22:58:39.677 INFO:teuthology.orchestra.run.smithi032.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T22:58:39.677 INFO:teuthology.orchestra.run.smithi032.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T22:58:39.677 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T22:58:39.677 INFO:teuthology.orchestra.run.smithi032.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T22:58:39.708 INFO:teuthology.orchestra.run.smithi032.stdout:Discarding blocks...Done. 2024-04-24T22:58:39.710 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi032.front.sepia.ceph.com -o noatime 2024-04-24T22:58:39.710 DEBUG:teuthology.orchestra.run.smithi032:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-24T22:58:39.778 DEBUG:teuthology.orchestra.run.smithi032:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-24T22:58:39.847 DEBUG:teuthology.orchestra.run.smithi032:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T22:58:39.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:39.940+0000 7f0ccddd8640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-24T22:58:39.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:39.940+0000 7f0ccddd8640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-24T22:58:39.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:39.940+0000 7f0ccddd8640 -1 memstore(/var/lib/ceph/osd/ceph-0) /var/lib/ceph/osd/ceph-0 2024-04-24T22:58:39.949 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-24T22:58:39.973 DEBUG:teuthology.orchestra.run.smithi032:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T22:58:40.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:40.056+0000 7fabd1557640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-24T22:58:40.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:40.056+0000 7fabd1557640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-24T22:58:40.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:40.057+0000 7fabd1557640 -1 memstore(/var/lib/ceph/osd/ceph-1) /var/lib/ceph/osd/ceph-1 2024-04-24T22:58:40.065 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-24T22:58:40.129 DEBUG:teuthology.orchestra.run.smithi032:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T22:58:40.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:40.210+0000 7fd1ef9cb640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-24T22:58:40.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:40.210+0000 7fd1ef9cb640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-24T22:58:40.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:40.211+0000 7fd1ef9cb640 -1 memstore(/var/lib/ceph/osd/ceph-2) /var/lib/ceph/osd/ceph-2 2024-04-24T22:58:40.219 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-24T22:58:40.283 DEBUG:teuthology.orchestra.run.smithi032:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T22:58:40.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:40.366+0000 7fa03621d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-24T22:58:40.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:40.366+0000 7fa03621d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-24T22:58:40.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:40.366+0000 7fa03621d640 -1 memstore(/var/lib/ceph/osd/ceph-3) /var/lib/ceph/osd/ceph-3 2024-04-24T22:58:40.374 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-24T22:58:40.398 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-24T22:58:40.426 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-24T22:58:40.427 INFO:tasks.ceph:role: osd.4 2024-04-24T22:58:40.427 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi121.front.sepia.ceph.com 2024-04-24T22:58:40.427 DEBUG:teuthology.orchestra.run.smithi121:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-24T22:58:40.502 INFO:teuthology.orchestra.run.smithi121.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T22:58:40.502 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T22:58:40.502 INFO:teuthology.orchestra.run.smithi121.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T22:58:40.502 INFO:teuthology.orchestra.run.smithi121.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T22:58:40.502 INFO:teuthology.orchestra.run.smithi121.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T22:58:40.502 INFO:teuthology.orchestra.run.smithi121.stdout: = sunit=0 swidth=0 blks 2024-04-24T22:58:40.502 INFO:teuthology.orchestra.run.smithi121.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T22:58:40.502 INFO:teuthology.orchestra.run.smithi121.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T22:58:40.503 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T22:58:40.503 INFO:teuthology.orchestra.run.smithi121.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T22:58:41.014 INFO:teuthology.orchestra.run.smithi121.stdout:Discarding blocks...Done. 2024-04-24T22:58:41.017 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi121.front.sepia.ceph.com -o noatime 2024-04-24T22:58:41.017 DEBUG:teuthology.orchestra.run.smithi121:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-24T22:58:41.129 DEBUG:teuthology.orchestra.run.smithi121:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-24T22:58:41.161 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-24T22:58:41.226 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-24T22:58:41.226 INFO:tasks.ceph:role: osd.5 2024-04-24T22:58:41.226 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi121.front.sepia.ceph.com 2024-04-24T22:58:41.227 DEBUG:teuthology.orchestra.run.smithi121:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-24T22:58:41.300 INFO:teuthology.orchestra.run.smithi121.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T22:58:41.300 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T22:58:41.300 INFO:teuthology.orchestra.run.smithi121.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T22:58:41.300 INFO:teuthology.orchestra.run.smithi121.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T22:58:41.300 INFO:teuthology.orchestra.run.smithi121.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T22:58:41.300 INFO:teuthology.orchestra.run.smithi121.stdout: = sunit=0 swidth=0 blks 2024-04-24T22:58:41.300 INFO:teuthology.orchestra.run.smithi121.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T22:58:41.300 INFO:teuthology.orchestra.run.smithi121.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T22:58:41.300 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T22:58:41.300 INFO:teuthology.orchestra.run.smithi121.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T22:58:41.760 INFO:teuthology.orchestra.run.smithi121.stdout:Discarding blocks...Done. 2024-04-24T22:58:41.763 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi121.front.sepia.ceph.com -o noatime 2024-04-24T22:58:41.763 DEBUG:teuthology.orchestra.run.smithi121:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-24T22:58:41.851 DEBUG:teuthology.orchestra.run.smithi121:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-24T22:58:41.878 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-24T22:58:41.942 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-24T22:58:41.942 INFO:tasks.ceph:role: osd.6 2024-04-24T22:58:41.942 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi121.front.sepia.ceph.com 2024-04-24T22:58:41.942 DEBUG:teuthology.orchestra.run.smithi121:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-24T22:58:42.013 INFO:teuthology.orchestra.run.smithi121.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T22:58:42.013 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T22:58:42.014 INFO:teuthology.orchestra.run.smithi121.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T22:58:42.014 INFO:teuthology.orchestra.run.smithi121.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T22:58:42.014 INFO:teuthology.orchestra.run.smithi121.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T22:58:42.014 INFO:teuthology.orchestra.run.smithi121.stdout: = sunit=0 swidth=0 blks 2024-04-24T22:58:42.014 INFO:teuthology.orchestra.run.smithi121.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T22:58:42.014 INFO:teuthology.orchestra.run.smithi121.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T22:58:42.014 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T22:58:42.014 INFO:teuthology.orchestra.run.smithi121.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T22:58:42.505 INFO:teuthology.orchestra.run.smithi121.stdout:Discarding blocks...Done. 2024-04-24T22:58:42.506 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi121.front.sepia.ceph.com -o noatime 2024-04-24T22:58:42.506 DEBUG:teuthology.orchestra.run.smithi121:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-24T22:58:42.641 DEBUG:teuthology.orchestra.run.smithi121:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-24T22:58:42.671 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-24T22:58:42.735 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-24T22:58:42.735 INFO:tasks.ceph:role: osd.7 2024-04-24T22:58:42.735 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi121.front.sepia.ceph.com 2024-04-24T22:58:42.736 DEBUG:teuthology.orchestra.run.smithi121:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-24T22:58:42.807 INFO:teuthology.orchestra.run.smithi121.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T22:58:42.807 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T22:58:42.807 INFO:teuthology.orchestra.run.smithi121.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T22:58:42.807 INFO:teuthology.orchestra.run.smithi121.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T22:58:42.807 INFO:teuthology.orchestra.run.smithi121.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T22:58:42.807 INFO:teuthology.orchestra.run.smithi121.stdout: = sunit=0 swidth=0 blks 2024-04-24T22:58:42.807 INFO:teuthology.orchestra.run.smithi121.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T22:58:42.807 INFO:teuthology.orchestra.run.smithi121.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T22:58:42.807 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T22:58:42.807 INFO:teuthology.orchestra.run.smithi121.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T22:58:43.328 INFO:teuthology.orchestra.run.smithi121.stdout:Discarding blocks...Done. 2024-04-24T22:58:43.330 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi121.front.sepia.ceph.com -o noatime 2024-04-24T22:58:43.330 DEBUG:teuthology.orchestra.run.smithi121:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-24T22:58:43.468 DEBUG:teuthology.orchestra.run.smithi121:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-24T22:58:43.500 DEBUG:teuthology.orchestra.run.smithi121:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T22:58:43.597 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:43.595+0000 7f7e753dc640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-24T22:58:43.597 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:43.595+0000 7f7e753dc640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-24T22:58:43.597 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:43.596+0000 7f7e753dc640 -1 memstore(/var/lib/ceph/osd/ceph-4) /var/lib/ceph/osd/ceph-4 2024-04-24T22:58:43.605 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-24T22:58:43.629 DEBUG:teuthology.orchestra.run.smithi121:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T22:58:43.712 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:43.710+0000 7f3301cb5640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-24T22:58:43.712 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:43.711+0000 7f3301cb5640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-24T22:58:43.713 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:43.711+0000 7f3301cb5640 -1 memstore(/var/lib/ceph/osd/ceph-5) /var/lib/ceph/osd/ceph-5 2024-04-24T22:58:43.718 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-24T22:58:43.782 DEBUG:teuthology.orchestra.run.smithi121:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T22:58:43.866 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:43.864+0000 7f1f7a210640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-24T22:58:43.866 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:43.864+0000 7f1f7a210640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-24T22:58:43.867 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:43.865+0000 7f1f7a210640 -1 memstore(/var/lib/ceph/osd/ceph-6) /var/lib/ceph/osd/ceph-6 2024-04-24T22:58:43.872 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-24T22:58:43.936 DEBUG:teuthology.orchestra.run.smithi121:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T22:58:44.019 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:44.017+0000 7fe12ca0e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-24T22:58:44.019 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:44.017+0000 7fe12ca0e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-24T22:58:44.019 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:44.018+0000 7fe12ca0e640 -1 memstore(/var/lib/ceph/osd/ceph-7) /var/lib/ceph/osd/ceph-7 2024-04-24T22:58:44.026 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-24T22:58:44.090 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-24T22:58:44.090 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T22:58:44.091 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-24T22:58:44.122 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T22:58:44.122 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-24T22:58:44.191 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T22:58:44.192 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-24T22:58:44.260 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T22:58:44.260 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-24T22:58:44.327 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T22:58:44.327 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-24T22:58:44.394 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T22:58:44.394 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-24T22:58:44.461 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T22:58:44.461 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-24T22:58:44.527 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T22:58:44.527 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-24T22:58:44.553 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T22:58:44.553 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-24T22:58:44.618 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T22:58:44.618 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-24T22:58:44.682 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T22:58:44.682 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-24T22:58:44.746 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T22:58:44.747 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-24T22:58:44.811 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T22:58:44.811 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-24T22:58:44.875 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T22:58:44.875 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-24T22:58:44.938 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T22:58:44.939 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-24T22:58:44.958 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T22:58:44.958 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-24T22:58:45.017 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T22:58:45.017 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-24T22:58:45.032 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T22:58:45.032 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-24T22:58:45.087 INFO:tasks.ceph:Adding keys to all mons... 2024-04-24T22:58:45.087 DEBUG:teuthology.orchestra.run.smithi032:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-24T22:58:45.090 DEBUG:teuthology.orchestra.run.smithi121:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-24T22:58:45.130 INFO:teuthology.orchestra.run.smithi032.stdout:[mgr.y] 2024-04-24T22:58:45.130 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQCdjilmMqs2OBAAOe3bLpnCq+HK2EbeXnwY8Q== 2024-04-24T22:58:45.131 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.a] 2024-04-24T22:58:45.131 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQCejilmXRdxAxAA/EYEn1C38XhG7DC+0X4y7A== 2024-04-24T22:58:45.131 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.c] 2024-04-24T22:58:45.131 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQCejilmZXFdChAA8TwwkA5EIXHBzPsQSFvXcQ== 2024-04-24T22:58:45.131 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.0] 2024-04-24T22:58:45.131 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQCfjilmhTwoOBAAig98xa3a2qnv4rriUO9VCQ== 2024-04-24T22:58:45.131 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.1] 2024-04-24T22:58:45.131 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQCgjilmwRaAAxAAUqDOyR9HV6DEnmbyn3WAjA== 2024-04-24T22:58:45.131 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.2] 2024-04-24T22:58:45.131 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQCgjilmDl2qDBAAF1bhTOaSOc612wmFRd8+XA== 2024-04-24T22:58:45.131 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.3] 2024-04-24T22:58:45.131 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQCgjilmAxHwFRAA9a0v/YwngRxq1vmMRsmEkQ== 2024-04-24T22:58:45.131 INFO:teuthology.orchestra.run.smithi032.stdout:[mgr.x] 2024-04-24T22:58:45.131 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQCejilmn3FJABAAGnvNAoNLzK20cQF6beihmw== 2024-04-24T22:58:45.132 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.b] 2024-04-24T22:58:45.132 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQCejilmak7bDhAA0he+Jp/gkkrfBjYKSObjcw== 2024-04-24T22:58:45.132 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.d] 2024-04-24T22:58:45.132 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQCejilmLbGuFRAAe91fZlulziWdWiDdLBLWNA== 2024-04-24T22:58:45.132 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.4] 2024-04-24T22:58:45.132 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQCjjilmDPmJIxAASZlEo1ZJHOhIAFGukIj1tQ== 2024-04-24T22:58:45.132 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.5] 2024-04-24T22:58:45.132 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQCjjilmgcpuKhAAedSq0ubNB30s3bz0fY38hw== 2024-04-24T22:58:45.132 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.6] 2024-04-24T22:58:45.132 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQCjjilmFv2VMxAAM+mUc4I0sjawy2uwuUEKYg== 2024-04-24T22:58:45.132 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.7] 2024-04-24T22:58:45.132 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQCkjilmLvEbARAAnaOvQ5Ee0clN/A9SVoKuBQ== 2024-04-24T22:58:45.132 INFO:teuthology.orchestra.run.smithi032.stdout:[client.0] 2024-04-24T22:58:45.132 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQCejilm5oeiGRAAdLXwwqrYhV3HHo7Dw/2ZqQ== 2024-04-24T22:58:45.132 INFO:teuthology.orchestra.run.smithi032.stdout:[client.2] 2024-04-24T22:58:45.133 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQCejilm4QElHxAAXgE316aRCosxX7XER5gtKQ== 2024-04-24T22:58:45.133 INFO:teuthology.orchestra.run.smithi032.stdout:[client.1] 2024-04-24T22:58:45.133 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQCejilmyWBDIhAARtVP4Unk3HHlQYZ2QZ3GzQ== 2024-04-24T22:58:45.133 INFO:teuthology.orchestra.run.smithi032.stdout:[client.3] 2024-04-24T22:58:45.133 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQCejilmMZihJxAAoWaAAFY3vVfZnC4/GoT0+g== 2024-04-24T22:58:45.149 INFO:teuthology.orchestra.run.smithi121.stdout:[mgr.y] 2024-04-24T22:58:45.149 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQCdjilmMqs2OBAAOe3bLpnCq+HK2EbeXnwY8Q== 2024-04-24T22:58:45.149 INFO:teuthology.orchestra.run.smithi121.stdout:[mds.a] 2024-04-24T22:58:45.149 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQCejilmXRdxAxAA/EYEn1C38XhG7DC+0X4y7A== 2024-04-24T22:58:45.149 INFO:teuthology.orchestra.run.smithi121.stdout:[mds.c] 2024-04-24T22:58:45.149 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQCejilmZXFdChAA8TwwkA5EIXHBzPsQSFvXcQ== 2024-04-24T22:58:45.149 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.0] 2024-04-24T22:58:45.149 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQCfjilmhTwoOBAAig98xa3a2qnv4rriUO9VCQ== 2024-04-24T22:58:45.150 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.1] 2024-04-24T22:58:45.150 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQCgjilmwRaAAxAAUqDOyR9HV6DEnmbyn3WAjA== 2024-04-24T22:58:45.150 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.2] 2024-04-24T22:58:45.150 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQCgjilmDl2qDBAAF1bhTOaSOc612wmFRd8+XA== 2024-04-24T22:58:45.150 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.3] 2024-04-24T22:58:45.150 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQCgjilmAxHwFRAA9a0v/YwngRxq1vmMRsmEkQ== 2024-04-24T22:58:45.150 INFO:teuthology.orchestra.run.smithi121.stdout:[mgr.x] 2024-04-24T22:58:45.150 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQCejilmn3FJABAAGnvNAoNLzK20cQF6beihmw== 2024-04-24T22:58:45.150 INFO:teuthology.orchestra.run.smithi121.stdout:[mds.b] 2024-04-24T22:58:45.153 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQCejilmak7bDhAA0he+Jp/gkkrfBjYKSObjcw== 2024-04-24T22:58:45.153 INFO:teuthology.orchestra.run.smithi121.stdout:[mds.d] 2024-04-24T22:58:45.153 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQCejilmLbGuFRAAe91fZlulziWdWiDdLBLWNA== 2024-04-24T22:58:45.153 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.4] 2024-04-24T22:58:45.153 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQCjjilmDPmJIxAASZlEo1ZJHOhIAFGukIj1tQ== 2024-04-24T22:58:45.153 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.5] 2024-04-24T22:58:45.153 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQCjjilmgcpuKhAAedSq0ubNB30s3bz0fY38hw== 2024-04-24T22:58:45.153 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.6] 2024-04-24T22:58:45.153 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQCjjilmFv2VMxAAM+mUc4I0sjawy2uwuUEKYg== 2024-04-24T22:58:45.153 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.7] 2024-04-24T22:58:45.153 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQCkjilmLvEbARAAnaOvQ5Ee0clN/A9SVoKuBQ== 2024-04-24T22:58:45.153 INFO:teuthology.orchestra.run.smithi121.stdout:[client.0] 2024-04-24T22:58:45.154 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQCejilm5oeiGRAAdLXwwqrYhV3HHo7Dw/2ZqQ== 2024-04-24T22:58:45.154 INFO:teuthology.orchestra.run.smithi121.stdout:[client.2] 2024-04-24T22:58:45.154 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQCejilm4QElHxAAXgE316aRCosxX7XER5gtKQ== 2024-04-24T22:58:45.154 INFO:teuthology.orchestra.run.smithi121.stdout:[client.1] 2024-04-24T22:58:45.154 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQCejilmyWBDIhAARtVP4Unk3HHlQYZ2QZ3GzQ== 2024-04-24T22:58:45.154 INFO:teuthology.orchestra.run.smithi121.stdout:[client.3] 2024-04-24T22:58:45.154 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQCejilmMZihJxAAoWaAAFY3vVfZnC4/GoT0+g== 2024-04-24T22:58:45.155 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-24T22:58:45.181 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-24T22:58:45.244 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T22:58:45.247 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T22:58:45.301 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T22:58:45.303 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T22:58:45.356 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T22:58:45.358 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T22:58:45.407 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T22:58:45.409 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T22:58:45.457 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T22:58:45.459 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T22:58:45.508 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T22:58:45.510 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T22:58:45.559 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-24T22:58:45.561 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-24T22:58:45.610 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T22:58:45.612 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T22:58:45.664 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T22:58:45.666 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T22:58:45.717 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T22:58:45.720 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T22:58:45.768 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T22:58:45.771 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T22:58:45.819 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T22:58:45.821 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T22:58:45.870 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T22:58:45.872 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T22:58:45.921 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-24T22:58:45.924 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-24T22:58:45.975 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-24T22:58:45.977 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-24T22:58:46.028 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-24T22:58:46.030 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-24T22:58:46.081 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-24T22:58:46.084 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-24T22:58:46.135 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-24T22:58:46.135 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-24T22:58:46.159 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-24T22:58:46.265 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-24T22:58:46.289 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-24T22:58:46.313 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-24T22:58:46.408 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-24T22:58:46.432 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-24T22:58:46.495 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-24T22:58:46.588 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-24T22:58:46.611 DEBUG:teuthology.orchestra.run.smithi032:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-24T22:58:46.613 DEBUG:teuthology.orchestra.run.smithi121:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-24T22:58:46.667 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-24T22:58:46.667 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-24T22:58:46.667 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-24T22:58:46.670 INFO:tasks.ceph.mon.a:Started 2024-04-24T22:58:46.670 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-24T22:58:46.670 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-24T22:58:46.709 INFO:tasks.ceph.mon.b:Started 2024-04-24T22:58:46.710 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-24T22:58:46.710 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-24T22:58:46.712 INFO:tasks.ceph.mon.c:Started 2024-04-24T22:58:46.712 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-24T22:58:46.712 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-24T22:58:46.712 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-24T22:58:46.715 INFO:tasks.ceph.mgr.y:Started 2024-04-24T22:58:46.715 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-24T22:58:46.715 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-24T22:58:46.717 INFO:tasks.ceph.mgr.x:Started 2024-04-24T22:58:46.717 INFO:tasks.ceph:Setting crush tunables to default 2024-04-24T22:58:46.717 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --cluster ceph osd crush tunables default 2024-04-24T22:58:46.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.816+0000 7ffbded5a640 1 Processor -- start 2024-04-24T22:58:46.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.816+0000 7ffbded5a640 1 -- start start 2024-04-24T22:58:46.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.817+0000 7ffbded5a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffbd8158db0 0x7ffbd80b6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:46.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.817+0000 7ffbded5a640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbd8159190 con 0x7ffbd80730c0 2024-04-24T22:58:46.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.817+0000 7ffbded5a640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbd80b7230 con 0x7ffbd806e800 2024-04-24T22:58:46.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.817+0000 7ffbded5a640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7ffbd815fd00 con 0x7ffbd8158db0 2024-04-24T22:58:46.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.817+0000 7ffbdcaf6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffbd8158db0 0x7ffbd80b6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:46.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.817+0000 7ffbdcaf6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffbd8158db0 0x7ffbd80b6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35312/0 (socket says 172.21.15.32:35312) 2024-04-24T22:58:46.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.817+0000 7ffbdcaf6640 1 -- 172.21.15.32:0/3667844884 learned_addr learned my addr 172.21.15.32:0/3667844884 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:58:46.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.817+0000 7ffbdcaf6640 1 -- 172.21.15.32:0/3667844884 >> v2:172.21.15.32:3300/0 conn(0x7ffbd8158db0 msgr2=0x7ffbd80b6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-24T22:58:46.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.817+0000 7ffbdcaf6640 1 -- 172.21.15.32:0/3667844884 >> v2:172.21.15.32:3300/0 conn(0x7ffbd8158db0 msgr2=0x7ffbd80b6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T22:58:46.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.817+0000 7ffbdcaf6640 1 --2- 172.21.15.32:0/3667844884 >> v2:172.21.15.32:3300/0 conn(0x7ffbd8158db0 0x7ffbd80b6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T22:58:46.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.817+0000 7ffbdcaf6640 1 --2- 172.21.15.32:0/3667844884 >> v2:172.21.15.32:3300/0 conn(0x7ffbd8158db0 0x7ffbd80b6bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:58:46.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.978+0000 7ffbd77fe640 1 -- 172.21.15.32:0/3667844884 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2458518192 0 0) 0x7ffbd8159190 con 0x7ffbd80730c0 2024-04-24T22:58:46.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.978+0000 7ffbd77fe640 1 -- 172.21.15.32:0/3667844884 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffbc0003520 con 0x7ffbd80730c0 2024-04-24T22:58:46.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.979+0000 7ffbd77fe640 1 -- 172.21.15.32:0/3667844884 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3875941513 0 0) 0x7ffbc0003520 con 0x7ffbd80730c0 2024-04-24T22:58:46.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.979+0000 7ffbd77fe640 1 -- 172.21.15.32:0/3667844884 >> v2:172.21.15.32:3300/0 conn(0x7ffbd8158db0 msgr2=0x7ffbd80b6bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T22:58:46.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.979+0000 7ffbd77fe640 1 --2- 172.21.15.32:0/3667844884 >> v2:172.21.15.32:3300/0 conn(0x7ffbd8158db0 0x7ffbd80b6bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:46.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.979+0000 7ffbd77fe640 1 -- 172.21.15.32:0/3667844884 >> v1:172.21.15.121:6789/0 conn(0x7ffbd806e800 legacy=0x7ffbd806ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:46.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.979+0000 7ffbd77fe640 1 -- 172.21.15.32:0/3667844884 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbd80b7510 con 0x7ffbd80730c0 2024-04-24T22:58:46.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.979+0000 7ffbd77fe640 1 -- 172.21.15.32:0/3667844884 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7ffbcc001670 con 0x7ffbd80730c0 2024-04-24T22:58:46.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.979+0000 7ffbd77fe640 1 -- 172.21.15.32:0/3667844884 >> v1:172.21.15.32:6789/0 conn(0x7ffbd80730c0 legacy=0x7ffbd815f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:46.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.979+0000 7ffbd77fe640 1 --2- 172.21.15.32:0/3667844884 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffbc00039b0 0x7ffbc0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:46.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.979+0000 7ffbd77fe640 1 --2- 172.21.15.32:0/3667844884 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbc0004800 0x7ffbc0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:46.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.979+0000 7ffbd77fe640 1 --2- 172.21.15.32:0/3667844884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffbc00051c0 0x7ffbc000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:46.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.979+0000 7ffbd77fe640 1 -- 172.21.15.32:0/3667844884 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd815fd00 con 0x7ffbc0004800 2024-04-24T22:58:46.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.979+0000 7ffbd77fe640 1 -- 172.21.15.32:0/3667844884 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc000c110 con 0x7ffbc00051c0 2024-04-24T22:58:46.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.979+0000 7ffbd77fe640 1 -- 172.21.15.32:0/3667844884 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ffbc000c310 con 0x7ffbc00039b0 2024-04-24T22:58:46.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.979+0000 7ffbd7fff640 1 --2- 172.21.15.32:0/3667844884 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbc0004800 0x7ffbc0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:46.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.979+0000 7ffbdd2f7640 1 --2- 172.21.15.32:0/3667844884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffbc00051c0 0x7ffbc000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:46.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.979+0000 7ffbdcaf6640 1 --2- 172.21.15.32:0/3667844884 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffbc00039b0 0x7ffbc0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:46.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.980+0000 7ffbd7fff640 1 -- 172.21.15.32:0/3667844884 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffbc00039b0 msgr2=0x7ffbc0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:58:46.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.980+0000 7ffbd7fff640 1 --2- 172.21.15.32:0/3667844884 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffbc00039b0 0x7ffbc0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:46.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.980+0000 7ffbd7fff640 1 -- 172.21.15.32:0/3667844884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffbc00051c0 msgr2=0x7ffbc000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:58:46.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.980+0000 7ffbd7fff640 1 --2- 172.21.15.32:0/3667844884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffbc00051c0 0x7ffbc000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:46.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.980+0000 7ffbd7fff640 1 -- 172.21.15.32:0/3667844884 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbc0003520 con 0x7ffbc0004800 2024-04-24T22:58:46.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.980+0000 7ffbdd2f7640 1 -- 172.21.15.32:0/3667844884 reap_dead start 2024-04-24T22:58:46.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.980+0000 7ffbd7fff640 1 --2- 172.21.15.32:0/3667844884 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbc0004800 0x7ffbc0004c80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc00c160 tx=0x7ffbc0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0e85006841e3ea0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:58:46.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.981+0000 7ffbd77fe640 1 -- 172.21.15.32:0/3667844884 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc036070 con 0x7ffbc0004800 2024-04-24T22:58:46.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.981+0000 7ffbd77fe640 1 -- 172.21.15.32:0/3667844884 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbcc007250 con 0x7ffbc0004800 2024-04-24T22:58:46.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.981+0000 7ffbd77fe640 1 -- 172.21.15.32:0/3667844884 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc007c60 con 0x7ffbc0004800 2024-04-24T22:58:46.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.981+0000 7ffbded5a640 1 -- 172.21.15.32:0/3667844884 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbc0004800 msgr2=0x7ffbc0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:46.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.981+0000 7ffbded5a640 1 --2- 172.21.15.32:0/3667844884 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbc0004800 0x7ffbc0004c80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc00c160 tx=0x7ffbc0005830 comp rx=0 tx=0).stop 2024-04-24T22:58:46.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.981+0000 7ffbded5a640 1 -- 172.21.15.32:0/3667844884 shutdown_connections 2024-04-24T22:58:46.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.981+0000 7ffbded5a640 1 --2- 172.21.15.32:0/3667844884 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbc0004800 0x7ffbc0004c80 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:46.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.981+0000 7ffbded5a640 1 -- 172.21.15.32:0/3667844884 >> 172.21.15.32:0/3667844884 conn(0x7ffbd80d24c0 msgr2=0x7ffbd80d28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:58:46.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.981+0000 7ffbded5a640 1 -- 172.21.15.32:0/3667844884 shutdown_connections 2024-04-24T22:58:46.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.981+0000 7ffbded5a640 1 -- 172.21.15.32:0/3667844884 wait complete. 2024-04-24T22:58:46.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.983+0000 7ffbded5a640 1 Processor -- start 2024-04-24T22:58:46.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.983+0000 7ffbded5a640 1 -- start start 2024-04-24T22:58:46.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.984+0000 7ffbded5a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbc0004800 0x7ffbd8153190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:46.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.984+0000 7ffbded5a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffbd80af2c0 0x7ffbd8162470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:46.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.984+0000 7ffbded5a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffbd81f3950 0x7ffbd81f3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:46.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.984+0000 7ffbded5a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd815ed80 con 0x7ffbc0004800 2024-04-24T22:58:46.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.984+0000 7ffbded5a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd815f0d0 con 0x7ffbd81f3950 2024-04-24T22:58:46.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.984+0000 7ffbded5a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ffbd8073570 con 0x7ffbd80af2c0 2024-04-24T22:58:46.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.984+0000 7ffbdcaf6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbc0004800 0x7ffbd8153190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:46.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.984+0000 7ffbdcaf6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbc0004800 0x7ffbd8153190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35330/0 (socket says 172.21.15.32:35330) 2024-04-24T22:58:46.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.984+0000 7ffbdcaf6640 1 -- 172.21.15.32:0/2296536851 learned_addr learned my addr 172.21.15.32:0/2296536851 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:58:46.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.984+0000 7ffbd7fff640 1 --2- 172.21.15.32:0/2296536851 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffbd80af2c0 0x7ffbd8162470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:46.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.984+0000 7ffbdd2f7640 1 --2- 172.21.15.32:0/2296536851 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffbd81f3950 0x7ffbd81f3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:46.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.984+0000 7ffbdcaf6640 1 -- 172.21.15.32:0/2296536851 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffbd80af2c0 msgr2=0x7ffbd8162470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:46.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.984+0000 7ffbdcaf6640 1 --2- 172.21.15.32:0/2296536851 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffbd80af2c0 0x7ffbd8162470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:46.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.984+0000 7ffbdcaf6640 1 -- 172.21.15.32:0/2296536851 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffbd81f3950 msgr2=0x7ffbd81f3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:46.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.984+0000 7ffbdcaf6640 1 --2- 172.21.15.32:0/2296536851 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffbd81f3950 0x7ffbd81f3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:46.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.984+0000 7ffbdcaf6640 1 -- 172.21.15.32:0/2296536851 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbd81f42f0 con 0x7ffbc0004800 2024-04-24T22:58:46.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.985+0000 7ffbdcaf6640 1 --2- 172.21.15.32:0/2296536851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbc0004800 0x7ffbd8153190 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8010890 tx=0x7ffbc8010c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:58:46.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.985+0000 7ffbd57fa640 1 -- 172.21.15.32:0/2296536851 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc801a020 con 0x7ffbc0004800 2024-04-24T22:58:46.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.985+0000 7ffbd57fa640 1 -- 172.21.15.32:0/2296536851 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbc800cb40 con 0x7ffbc0004800 2024-04-24T22:58:46.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.985+0000 7ffbd57fa640 1 -- 172.21.15.32:0/2296536851 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc800f040 con 0x7ffbc0004800 2024-04-24T22:58:46.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.986+0000 7ffbded5a640 1 -- 172.21.15.32:0/2296536851 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbd80afaa0 con 0x7ffbc0004800 2024-04-24T22:58:46.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.986+0000 7ffbded5a640 1 -- 172.21.15.32:0/2296536851 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbd80b0340 con 0x7ffbc0004800 2024-04-24T22:58:46.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.986+0000 7ffbd57fa640 1 -- 172.21.15.32:0/2296536851 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ffbc801d020 con 0x7ffbc0004800 2024-04-24T22:58:46.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.987+0000 7ffbd57fa640 1 -- 172.21.15.32:0/2296536851 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ffbc80167c0 con 0x7ffbc0004800 2024-04-24T22:58:46.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.987+0000 7ffbded5a640 1 -- 172.21.15.32:0/2296536851 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbd8068430 con 0x7ffbc0004800 2024-04-24T22:58:46.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:46.989+0000 7ffbd57fa640 1 -- 172.21.15.32:0/2296536851 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7ffbc8024020 con 0x7ffbc0004800 2024-04-24T22:58:47.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:47.057+0000 7ffbded5a640 1 -- 172.21.15.32:0/2296536851 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7ffbd8072a40 con 0x7ffbc0004800 2024-04-24T22:58:47.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:47.059+0000 7ffbd57fa640 1 -- 172.21.15.32:0/2296536851 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7ffbc8024020 con 0x7ffbc0004800 2024-04-24T22:58:47.060 INFO:teuthology.orchestra.run.smithi032.stderr:adjusted tunables profile to default 2024-04-24T22:58:47.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:47.059+0000 7ffbded5a640 1 -- 172.21.15.32:0/2296536851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbc0004800 msgr2=0x7ffbd8153190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:47.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:47.059+0000 7ffbded5a640 1 --2- 172.21.15.32:0/2296536851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbc0004800 0x7ffbd8153190 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8010890 tx=0x7ffbc8010c90 comp rx=0 tx=0).stop 2024-04-24T22:58:47.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:47.060+0000 7ffbded5a640 1 -- 172.21.15.32:0/2296536851 shutdown_connections 2024-04-24T22:58:47.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:47.060+0000 7ffbded5a640 1 --2- 172.21.15.32:0/2296536851 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffbd81f3950 0x7ffbd81f3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:47.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:47.060+0000 7ffbded5a640 1 --2- 172.21.15.32:0/2296536851 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffbd80af2c0 0x7ffbd8162470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:47.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:47.060+0000 7ffbded5a640 1 --2- 172.21.15.32:0/2296536851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbc0004800 0x7ffbd8153190 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:47.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:47.060+0000 7ffbded5a640 1 -- 172.21.15.32:0/2296536851 >> 172.21.15.32:0/2296536851 conn(0x7ffbd80d24c0 msgr2=0x7ffbd80d28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:58:47.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:47.060+0000 7ffbded5a640 1 -- 172.21.15.32:0/2296536851 shutdown_connections 2024-04-24T22:58:47.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:47.060+0000 7ffbded5a640 1 -- 172.21.15.32:0/2296536851 wait complete. 2024-04-24T22:58:47.072 INFO:tasks.ceph:check_enable_crimson: False 2024-04-24T22:58:47.072 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-24T22:58:47.072 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T22:58:47.072 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-24T22:58:47.097 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T22:58:47.097 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-24T22:58:47.097 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-24T22:58:47.096+0000 7f2c22579040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T22:58:47.103 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-04-24T22:58:47.102+0000 7f8308c0d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T22:58:47.122 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T22:58:47.122 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-24T22:58:47.187 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T22:58:47.187 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-24T22:58:47.251 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T22:58:47.252 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-24T22:58:47.253 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-24T22:58:47.252+0000 7f2c22579040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T22:58:47.255 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-04-24T22:58:47.254+0000 7f8308c0d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T22:58:47.276 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T22:58:47.276 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-24T22:58:47.310 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-24T22:58:47.308+0000 7f2c22579040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T22:58:47.311 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-04-24T22:58:47.310+0000 7f8308c0d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T22:58:47.334 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T22:58:47.334 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-24T22:58:47.398 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T22:58:47.398 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-24T22:58:47.462 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 0 2024-04-24T22:58:47.559 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-04-24T22:58:47.558+0000 7f8308c0d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T22:58:47.566 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-24T22:58:47.565+0000 7f2c22579040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T22:58:47.599 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.597+0000 7fa1768bd640 1 Processor -- start 2024-04-24T22:58:47.599 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.597+0000 7fa1768bd640 1 -- start start 2024-04-24T22:58:47.600 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.598+0000 7fa1768bd640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa17015ec10 0x7fa17015fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:47.600 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.598+0000 7fa1768bd640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa1700b7090 0x7fa1700ce540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:47.600 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.598+0000 7fa1768bd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa1700d12b0 0x7fa1700d1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:47.600 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.598+0000 7fa1768bd640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fa1700d1c60 con 0x7fa1700d12b0 2024-04-24T22:58:47.600 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.598+0000 7fa1768bd640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fa17015eff0 con 0x7fa17015ec10 2024-04-24T22:58:47.600 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.598+0000 7fa1768bd640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa1700b60c0 con 0x7fa1700b7090 2024-04-24T22:58:47.600 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.598+0000 7fa16f7fe640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa1700b7090 0x7fa1700ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:47.601 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.598+0000 7fa16ffff640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa17015ec10 0x7fa17015fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:47.601 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.598+0000 7fa16ffff640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa17015ec10 0x7fa17015fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:45294/0 (socket says 172.21.15.121:45294) 2024-04-24T22:58:47.601 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.598+0000 7fa16f7fe640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa1700b7090 0x7fa1700ce540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.121:46524/0 (socket says 172.21.15.121:46524) 2024-04-24T22:58:47.601 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.598+0000 7fa16ffff640 1 -- 172.21.15.121:0/3552003397 learned_addr learned my addr 172.21.15.121:0/3552003397 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-24T22:58:47.601 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.598+0000 7fa16ffff640 1 -- 172.21.15.121:0/3552003397 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa1700b7090 msgr2=0x7fa1700ce540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:58:47.601 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.598+0000 7fa16ffff640 1 --2- 172.21.15.121:0/3552003397 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa1700b7090 0x7fa1700ce540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:47.601 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.598+0000 7fa16ffff640 1 -- 172.21.15.121:0/3552003397 >> v2:172.21.15.32:3300/0 conn(0x7fa1700d12b0 msgr2=0x7fa1700d1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:58:47.601 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.598+0000 7fa16ffff640 1 --2- 172.21.15.121:0/3552003397 >> v2:172.21.15.32:3300/0 conn(0x7fa1700d12b0 0x7fa1700d1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:47.601 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.599+0000 7fa16ffff640 1 -- 172.21.15.121:0/3552003397 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1700b6310 con 0x7fa17015ec10 2024-04-24T22:58:47.602 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.599+0000 7fa16ffff640 1 --2- 172.21.15.121:0/3552003397 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa17015ec10 0x7fa17015fc00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa1700720e0 tx=0x7fa16402a140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da39a7c71588c4db server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:58:47.603 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.599+0000 7fa16effd640 1 -- 172.21.15.121:0/3552003397 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa164037070 con 0x7fa17015ec10 2024-04-24T22:58:47.603 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.599+0000 7fa16effd640 1 -- 172.21.15.121:0/3552003397 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa164032440 con 0x7fa17015ec10 2024-04-24T22:58:47.603 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.599+0000 7fa16effd640 1 -- 172.21.15.121:0/3552003397 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa164032810 con 0x7fa17015ec10 2024-04-24T22:58:47.603 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.600+0000 7fa1768bd640 1 -- 172.21.15.121:0/3552003397 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa17015ec10 msgr2=0x7fa17015fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:47.603 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.600+0000 7fa1768bd640 1 --2- 172.21.15.121:0/3552003397 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa17015ec10 0x7fa17015fc00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa1700720e0 tx=0x7fa16402a140 comp rx=0 tx=0).stop 2024-04-24T22:58:47.603 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.600+0000 7fa1768bd640 1 -- 172.21.15.121:0/3552003397 shutdown_connections 2024-04-24T22:58:47.603 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.600+0000 7fa1768bd640 1 --2- 172.21.15.121:0/3552003397 >> v2:172.21.15.32:3300/0 conn(0x7fa1700d12b0 0x7fa1700d1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:47.603 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.600+0000 7fa1768bd640 1 --2- 172.21.15.121:0/3552003397 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa1700b7090 0x7fa1700ce540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:47.603 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.600+0000 7fa1768bd640 1 --2- 172.21.15.121:0/3552003397 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa17015ec10 0x7fa17015fc00 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:47.603 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.600+0000 7fa1768bd640 1 -- 172.21.15.121:0/3552003397 >> 172.21.15.121:0/3552003397 conn(0x7fa1700b4de0 msgr2=0x7fa170072850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:58:47.603 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.600+0000 7fa1768bd640 1 -- 172.21.15.121:0/3552003397 shutdown_connections 2024-04-24T22:58:47.603 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.600+0000 7fa1768bd640 1 -- 172.21.15.121:0/3552003397 wait complete. 2024-04-24T22:58:47.603 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.601+0000 7fa1768bd640 1 Processor -- start 2024-04-24T22:58:47.604 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.601+0000 7fa1768bd640 1 -- start start 2024-04-24T22:58:47.606 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.602+0000 7fa1768bd640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa1700b7090 0x7fa170156ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:47.606 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.602+0000 7fa1768bd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1700d12b0 0x7fa1700afab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:47.606 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.602+0000 7fa1768bd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa17015ec10 0x7fa170158c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:47.606 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.602+0000 7fa1768bd640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa1700cdd40 con 0x7fa17015ec10 2024-04-24T22:58:47.606 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.602+0000 7fa1768bd640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa1700b68d0 con 0x7fa1700d12b0 2024-04-24T22:58:47.606 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.602+0000 7fa1768bd640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa1700ce020 con 0x7fa1700b7090 2024-04-24T22:58:47.606 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.602+0000 7fa16f7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1700d12b0 0x7fa1700afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:47.606 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.602+0000 7fa16ffff640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa1700b7090 0x7fa170156ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:47.606 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.602+0000 7fa16f7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1700d12b0 0x7fa1700afab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:45298/0 (socket says 172.21.15.121:45298) 2024-04-24T22:58:47.607 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.602+0000 7fa16f7fe640 1 -- 172.21.15.121:0/1491224656 learned_addr learned my addr 172.21.15.121:0/1491224656 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-24T22:58:47.607 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.602+0000 7fa174e5a640 1 --2- 172.21.15.121:0/1491224656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa17015ec10 0x7fa170158c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:47.607 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.602+0000 7fa16ffff640 1 -- 172.21.15.121:0/1491224656 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1700d12b0 msgr2=0x7fa1700afab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:47.607 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.602+0000 7fa16ffff640 1 --2- 172.21.15.121:0/1491224656 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1700d12b0 0x7fa1700afab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:47.607 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.602+0000 7fa16ffff640 1 -- 172.21.15.121:0/1491224656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa17015ec10 msgr2=0x7fa170158c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:47.607 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.602+0000 7fa16ffff640 1 --2- 172.21.15.121:0/1491224656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa17015ec10 0x7fa170158c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:47.607 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.602+0000 7fa16ffff640 1 -- 172.21.15.121:0/1491224656 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1700c38b0 con 0x7fa1700b7090 2024-04-24T22:58:47.607 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.602+0000 7fa16f7fe640 1 --2- 172.21.15.121:0/1491224656 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1700d12b0 0x7fa1700afab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:58:47.607 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.603+0000 7fa16ffff640 1 --2- 172.21.15.121:0/1491224656 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa1700b7090 0x7fa170156ae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa164031000 tx=0x7fa1640046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:58:47.607 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.603+0000 7fa16cff9640 1 -- 172.21.15.121:0/1491224656 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa164041070 con 0x7fa1700b7090 2024-04-24T22:58:47.607 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.603+0000 7fa16cff9640 1 -- 172.21.15.121:0/1491224656 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa16403c870 con 0x7fa1700b7090 2024-04-24T22:58:47.607 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.603+0000 7fa1768bd640 1 -- 172.21.15.121:0/1491224656 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa170157020 con 0x7fa1700b7090 2024-04-24T22:58:47.607 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.603+0000 7fa16cff9640 1 -- 172.21.15.121:0/1491224656 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa164037040 con 0x7fa1700b7090 2024-04-24T22:58:47.607 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.603+0000 7fa1768bd640 1 -- 172.21.15.121:0/1491224656 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1700c3b20 con 0x7fa1700b7090 2024-04-24T22:58:47.608 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.603+0000 7fa16cff9640 1 -- 172.21.15.121:0/1491224656 <== mon.2 v2:172.21.15.121:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa164044070 con 0x7fa1700b7090 2024-04-24T22:58:47.608 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.603+0000 7fa16cff9640 1 -- 172.21.15.121:0/1491224656 <== mon.2 v2:172.21.15.121:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7fa164046d20 con 0x7fa1700b7090 2024-04-24T22:58:47.608 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.604+0000 7fa1768bd640 1 -- 172.21.15.121:0/1491224656 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa170068430 con 0x7fa1700b7090 2024-04-24T22:58:47.608 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-04-24T22:58:47.606+0000 7f8308c0d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T22:58:47.609 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.606+0000 7fa16cff9640 1 -- 172.21.15.121:0/1491224656 <== mon.2 v2:172.21.15.121:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fa170157020 con 0x7fa1700b7090 2024-04-24T22:58:47.614 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-24T22:58:47.613+0000 7f2c22579040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T22:58:47.679 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.678+0000 7fa1768bd640 1 -- 172.21.15.121:0/1491224656 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 0} v 0) -- 0x7fa1700d0e90 con 0x7fa1700b7090 2024-04-24T22:58:47.682 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.680+0000 7fa16cff9640 1 -- 172.21.15.121:0/1491224656 <== mon.2 v2:172.21.15.121:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7fa164046450 con 0x7fa1700b7090 2024-04-24T22:58:47.682 INFO:teuthology.orchestra.run.smithi121.stdout:0 2024-04-24T22:58:47.683 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.681+0000 7fa1768bd640 1 -- 172.21.15.121:0/1491224656 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa1700b7090 msgr2=0x7fa170156ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:47.683 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.681+0000 7fa1768bd640 1 --2- 172.21.15.121:0/1491224656 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa1700b7090 0x7fa170156ae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa164031000 tx=0x7fa1640046c0 comp rx=0 tx=0).stop 2024-04-24T22:58:47.683 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.681+0000 7fa1768bd640 1 -- 172.21.15.121:0/1491224656 shutdown_connections 2024-04-24T22:58:47.683 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.681+0000 7fa1768bd640 1 --2- 172.21.15.121:0/1491224656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa17015ec10 0x7fa170158c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:47.683 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.681+0000 7fa1768bd640 1 --2- 172.21.15.121:0/1491224656 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1700d12b0 0x7fa1700afab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:47.683 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.681+0000 7fa1768bd640 1 --2- 172.21.15.121:0/1491224656 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa1700b7090 0x7fa170156ae0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:47.683 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.681+0000 7fa1768bd640 1 -- 172.21.15.121:0/1491224656 >> 172.21.15.121:0/1491224656 conn(0x7fa1700b4de0 msgr2=0x7fa17006bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:58:47.683 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.681+0000 7fa1768bd640 1 -- 172.21.15.121:0/1491224656 shutdown_connections 2024-04-24T22:58:47.683 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.681+0000 7fa1768bd640 1 -- 172.21.15.121:0/1491224656 wait complete. 2024-04-24T22:58:47.693 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 1 2024-04-24T22:58:47.791 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.789+0000 7f43c1438640 1 Processor -- start 2024-04-24T22:58:47.791 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.789+0000 7f43c1438640 1 -- start start 2024-04-24T22:58:47.792 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.790+0000 7f43c1438640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f43bc0b06b0 0x7f43bc06d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:47.792 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.790+0000 7f43c1438640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43bc0d1190 con 0x7f43bc0b6e10 2024-04-24T22:58:47.792 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.790+0000 7f43c1438640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43bc0d1470 con 0x7f43bc074c20 2024-04-24T22:58:47.792 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.790+0000 7f43c1438640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f43bc0d1750 con 0x7f43bc0b06b0 2024-04-24T22:58:47.792 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.790+0000 7f43baffd640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f43bc0b06b0 0x7f43bc06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:47.792 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.790+0000 7f43ba7fc640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f43bc074c20 0x7f43bc0b6700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:53508/0 (socket says 172.21.15.121:53508) 2024-04-24T22:58:47.792 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.790+0000 7f43ba7fc640 1 -- 172.21.15.121:0/3012872226 learned_addr learned my addr 172.21.15.121:0/3012872226 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-24T22:58:47.792 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.790+0000 7f43baffd640 1 --2- 172.21.15.121:0/3012872226 >> v2:172.21.15.121:3300/0 conn(0x7f43bc0b06b0 0x7f43bc06d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:45300/0 (socket says 172.21.15.121:45300) 2024-04-24T22:58:47.792 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.790+0000 7f43baffd640 1 -- 172.21.15.121:0/3012872226 >> v1:172.21.15.121:6789/0 conn(0x7f43bc074c20 legacy=0x7f43bc0b6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:47.793 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.790+0000 7f43b9ffb640 1 -- 172.21.15.121:0/3012872226 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2394859118 0 0) 0x7f43bc0d1470 con 0x7f43bc074c20 2024-04-24T22:58:47.793 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.790+0000 7f43baffd640 1 -- 172.21.15.121:0/3012872226 >> v1:172.21.15.32:6789/0 conn(0x7f43bc0b6e10 legacy=0x7f43bc0cdf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:47.793 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.790+0000 7f43baffd640 1 -- 172.21.15.121:0/3012872226 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43bc0d1a30 con 0x7f43bc0b06b0 2024-04-24T22:58:47.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.791+0000 7f43baffd640 1 --2- 172.21.15.121:0/3012872226 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43bc0b06b0 0x7f43bc06d570 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f43b4009d90 tx=0x7f43b402a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb98c88fe8cf7dd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:58:47.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.791+0000 7f43b9ffb640 1 -- 172.21.15.121:0/3012872226 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b4007740 con 0x7f43bc0b06b0 2024-04-24T22:58:47.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.791+0000 7f43b9ffb640 1 -- 172.21.15.121:0/3012872226 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43b40078e0 con 0x7f43bc0b06b0 2024-04-24T22:58:47.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.791+0000 7f43b9ffb640 1 -- 172.21.15.121:0/3012872226 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b4007cb0 con 0x7f43bc0b06b0 2024-04-24T22:58:47.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.792+0000 7f43c1438640 1 -- 172.21.15.121:0/3012872226 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43bc0b06b0 msgr2=0x7f43bc06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:47.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.792+0000 7f43c1438640 1 --2- 172.21.15.121:0/3012872226 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43bc0b06b0 0x7f43bc06d570 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f43b4009d90 tx=0x7f43b402a6f0 comp rx=0 tx=0).stop 2024-04-24T22:58:47.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.792+0000 7f43c1438640 1 -- 172.21.15.121:0/3012872226 shutdown_connections 2024-04-24T22:58:47.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.792+0000 7f43c1438640 1 --2- 172.21.15.121:0/3012872226 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43bc0b06b0 0x7f43bc06d570 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:47.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.792+0000 7f43c1438640 1 -- 172.21.15.121:0/3012872226 >> 172.21.15.121:0/3012872226 conn(0x7f43bc0b2ed0 msgr2=0x7f43bc0b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:58:47.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.792+0000 7f43c1438640 1 -- 172.21.15.121:0/3012872226 shutdown_connections 2024-04-24T22:58:47.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.792+0000 7f43c1438640 1 -- 172.21.15.121:0/3012872226 wait complete. 2024-04-24T22:58:47.797 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.793+0000 7f43c1438640 1 Processor -- start 2024-04-24T22:58:47.797 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.793+0000 7f43c1438640 1 -- start start 2024-04-24T22:58:47.797 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.794+0000 7f43c1438640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43bc074c20 0x7f43bc0c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:47.797 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.794+0000 7f43c1438640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43bc0b06b0 0x7f43bc158490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:47.797 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.794+0000 7f43c1438640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f43bc0b6e10 0x7f43bc0cef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:47.797 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.794+0000 7f43c1438640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f43bc0750a0 con 0x7f43bc074c20 2024-04-24T22:58:47.797 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.794+0000 7f43c1438640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f43bc0b0a90 con 0x7f43bc0b06b0 2024-04-24T22:58:47.797 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.794+0000 7f43c1438640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f43bc0b3b40 con 0x7f43bc0b6e10 2024-04-24T22:58:47.797 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.794+0000 7f43ba7fc640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43bc0b06b0 0x7f43bc158490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:47.797 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.794+0000 7f43bb7fe640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f43bc0b6e10 0x7f43bc0cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:47.797 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.794+0000 7f43ba7fc640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43bc0b06b0 0x7f43bc158490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:45302/0 (socket says 172.21.15.121:45302) 2024-04-24T22:58:47.797 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.794+0000 7f43ba7fc640 1 -- 172.21.15.121:0/1605326537 learned_addr learned my addr 172.21.15.121:0/1605326537 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-24T22:58:47.797 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.794+0000 7f43baffd640 1 --2- 172.21.15.121:0/1605326537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43bc074c20 0x7f43bc0c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:47.798 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.794+0000 7f43ba7fc640 1 -- 172.21.15.121:0/1605326537 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f43bc0b6e10 msgr2=0x7f43bc0cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:47.798 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.794+0000 7f43ba7fc640 1 --2- 172.21.15.121:0/1605326537 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f43bc0b6e10 0x7f43bc0cef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:47.798 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.794+0000 7f43ba7fc640 1 -- 172.21.15.121:0/1605326537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43bc074c20 msgr2=0x7f43bc0c5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:47.798 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.794+0000 7f43ba7fc640 1 --2- 172.21.15.121:0/1605326537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43bc074c20 0x7f43bc0c5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:47.798 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.794+0000 7f43ba7fc640 1 -- 172.21.15.121:0/1605326537 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43bc0cf4a0 con 0x7f43bc0b06b0 2024-04-24T22:58:47.798 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.795+0000 7f43ba7fc640 1 --2- 172.21.15.121:0/1605326537 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43bc0b06b0 0x7f43bc158490 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f43b000c450 tx=0x7f43b000c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:58:47.798 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.795+0000 7f43a37fe640 1 -- 172.21.15.121:0/1605326537 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b0018070 con 0x7f43bc0b06b0 2024-04-24T22:58:47.798 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.795+0000 7f43a37fe640 1 -- 172.21.15.121:0/1605326537 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43b00074e0 con 0x7f43bc0b06b0 2024-04-24T22:58:47.798 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.795+0000 7f43a37fe640 1 -- 172.21.15.121:0/1605326537 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b0004a20 con 0x7f43bc0b06b0 2024-04-24T22:58:47.798 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.795+0000 7f43c1438640 1 -- 172.21.15.121:0/1605326537 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43bc0cf720 con 0x7f43bc0b06b0 2024-04-24T22:58:47.798 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.795+0000 7f43c1438640 1 -- 172.21.15.121:0/1605326537 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43bc0cfce0 con 0x7f43bc0b06b0 2024-04-24T22:58:47.798 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.795+0000 7f43a37fe640 1 -- 172.21.15.121:0/1605326537 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f43b001b070 con 0x7f43bc0b06b0 2024-04-24T22:58:47.798 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.795+0000 7f43a37fe640 1 -- 172.21.15.121:0/1605326537 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f43b0012400 con 0x7f43bc0b06b0 2024-04-24T22:58:47.798 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.796+0000 7f43c1438640 1 -- 172.21.15.121:0/1605326537 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43bc0b5820 con 0x7f43bc0b06b0 2024-04-24T22:58:47.801 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.799+0000 7f43a37fe640 1 -- 172.21.15.121:0/1605326537 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f43bc0b5820 con 0x7f43bc0b06b0 2024-04-24T22:58:47.873 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.872+0000 7f43c1438640 1 -- 172.21.15.121:0/1605326537 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 1} v 0) -- 0x7f43bc068430 con 0x7f43bc0b06b0 2024-04-24T22:58:47.876 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.875+0000 7f43a37fe640 1 -- 172.21.15.121:0/1605326537 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f43b0022070 con 0x7f43bc0b06b0 2024-04-24T22:58:47.876 INFO:teuthology.orchestra.run.smithi121.stdout:1 2024-04-24T22:58:47.877 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.875+0000 7f43c1438640 1 -- 172.21.15.121:0/1605326537 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43bc0b06b0 msgr2=0x7f43bc158490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:47.877 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.875+0000 7f43c1438640 1 --2- 172.21.15.121:0/1605326537 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43bc0b06b0 0x7f43bc158490 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f43b000c450 tx=0x7f43b000c910 comp rx=0 tx=0).stop 2024-04-24T22:58:47.877 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.876+0000 7f43c1438640 1 -- 172.21.15.121:0/1605326537 shutdown_connections 2024-04-24T22:58:47.877 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.876+0000 7f43c1438640 1 --2- 172.21.15.121:0/1605326537 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f43bc0b6e10 0x7f43bc0cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:47.877 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.876+0000 7f43c1438640 1 --2- 172.21.15.121:0/1605326537 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43bc0b06b0 0x7f43bc158490 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:47.878 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.876+0000 7f43c1438640 1 --2- 172.21.15.121:0/1605326537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43bc074c20 0x7f43bc0c5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:47.878 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.876+0000 7f43c1438640 1 -- 172.21.15.121:0/1605326537 >> 172.21.15.121:0/1605326537 conn(0x7f43bc0b2ed0 msgr2=0x7f43bc0b0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:58:47.878 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.876+0000 7f43c1438640 1 -- 172.21.15.121:0/1605326537 shutdown_connections 2024-04-24T22:58:47.878 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.876+0000 7f43c1438640 1 -- 172.21.15.121:0/1605326537 wait complete. 2024-04-24T22:58:47.887 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 2 2024-04-24T22:58:47.985 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.984+0000 7fa2bf7d5640 1 Processor -- start 2024-04-24T22:58:47.985 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.984+0000 7fa2bf7d5640 1 -- start start 2024-04-24T22:58:47.987 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.985+0000 7fa2bf7d5640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa2b815c490 0x7fa2b815ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:47.987 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.985+0000 7fa2bf7d5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2b80d2f30 0x7fa2b80b5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:47.987 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.985+0000 7fa2bf7d5640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2b815c870 con 0x7fa2b80b59c0 2024-04-24T22:58:47.987 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.985+0000 7fa2bf7d5640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fa2b80b1ea0 con 0x7fa2b815c490 2024-04-24T22:58:47.987 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.985+0000 7fa2bf7d5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa2b8160e40 con 0x7fa2b80d2f30 2024-04-24T22:58:47.987 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.985+0000 7fa2bd571640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa2b815c490 0x7fa2b815ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:47.988 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.985+0000 7fa2bcd70640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2b80d2f30 0x7fa2b80b5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:47.988 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.985+0000 7fa2bd571640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa2b815c490 0x7fa2b815ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:45318/0 (socket says 172.21.15.121:45318) 2024-04-24T22:58:47.988 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.985+0000 7fa2bd571640 1 -- 172.21.15.121:0/2312684468 learned_addr learned my addr 172.21.15.121:0/2312684468 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-24T22:58:47.988 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.985+0000 7fa2bcd70640 1 --2- 172.21.15.121:0/2312684468 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2b80d2f30 0x7fa2b80b5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.121:46548/0 (socket says 172.21.15.121:46548) 2024-04-24T22:58:47.988 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.985+0000 7fa2bd571640 1 -- 172.21.15.121:0/2312684468 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2b80d2f30 msgr2=0x7fa2b80b5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:58:47.988 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.985+0000 7fa2bd571640 1 --2- 172.21.15.121:0/2312684468 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2b80d2f30 0x7fa2b80b5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:47.988 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.985+0000 7fa2bd571640 1 -- 172.21.15.121:0/2312684468 >> v1:172.21.15.32:6789/0 conn(0x7fa2b80b59c0 legacy=0x7fa2b815d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:47.988 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.985+0000 7fa2bd571640 1 -- 172.21.15.121:0/2312684468 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2b81605a0 con 0x7fa2b815c490 2024-04-24T22:58:47.988 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.985+0000 7fa2bcd70640 1 --2- 172.21.15.121:0/2312684468 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2b80d2f30 0x7fa2b80b5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:58:47.988 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.986+0000 7fa2bd571640 1 --2- 172.21.15.121:0/2312684468 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2b815c490 0x7fa2b815ffc0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4009870 tx=0x7fa2b402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37d3e90e93848641 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:58:47.988 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.986+0000 7fa2affff640 1 -- 172.21.15.121:0/2312684468 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b4037070 con 0x7fa2b815c490 2024-04-24T22:58:47.988 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.986+0000 7fa2affff640 1 -- 172.21.15.121:0/2312684468 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2b4032440 con 0x7fa2b815c490 2024-04-24T22:58:47.988 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.986+0000 7fa2affff640 1 -- 172.21.15.121:0/2312684468 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b4032810 con 0x7fa2b815c490 2024-04-24T22:58:47.991 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.986+0000 7fa2bf7d5640 1 -- 172.21.15.121:0/2312684468 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2b815c490 msgr2=0x7fa2b815ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:47.991 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.986+0000 7fa2bf7d5640 1 --2- 172.21.15.121:0/2312684468 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2b815c490 0x7fa2b815ffc0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4009870 tx=0x7fa2b402a270 comp rx=0 tx=0).stop 2024-04-24T22:58:47.992 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.987+0000 7fa2bf7d5640 1 -- 172.21.15.121:0/2312684468 shutdown_connections 2024-04-24T22:58:47.992 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.987+0000 7fa2bf7d5640 1 --2- 172.21.15.121:0/2312684468 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2b80d2f30 0x7fa2b80b5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:47.992 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.987+0000 7fa2bf7d5640 1 --2- 172.21.15.121:0/2312684468 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2b815c490 0x7fa2b815ffc0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:47.992 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.987+0000 7fa2bf7d5640 1 -- 172.21.15.121:0/2312684468 >> 172.21.15.121:0/2312684468 conn(0x7fa2b80d2410 msgr2=0x7fa2b80d2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:58:47.992 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.987+0000 7fa2bf7d5640 1 -- 172.21.15.121:0/2312684468 shutdown_connections 2024-04-24T22:58:47.992 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.987+0000 7fa2bf7d5640 1 -- 172.21.15.121:0/2312684468 wait complete. 2024-04-24T22:58:47.992 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.988+0000 7fa2bf7d5640 1 Processor -- start 2024-04-24T22:58:47.992 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.988+0000 7fa2bf7d5640 1 -- start start 2024-04-24T22:58:47.992 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.988+0000 7fa2bf7d5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2b80b59c0 0x7fa2b81f1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:47.992 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.988+0000 7fa2bf7d5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2b80d2f30 0x7fa2b80d0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:47.992 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.989+0000 7fa2bf7d5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2b815c490 0x7fa2b8142ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:47.993 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.989+0000 7fa2bf7d5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b815cae0 con 0x7fa2b80b59c0 2024-04-24T22:58:47.993 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.989+0000 7fa2bf7d5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b8160b60 con 0x7fa2b815c490 2024-04-24T22:58:47.993 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.989+0000 7fa2bf7d5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa2b815d710 con 0x7fa2b80d2f30 2024-04-24T22:58:47.993 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.989+0000 7fa2bcd70640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2b80d2f30 0x7fa2b80d0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:47.993 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.989+0000 7fa2bcd70640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2b80d2f30 0x7fa2b80d0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.121:46558/0 (socket says 172.21.15.121:46558) 2024-04-24T22:58:47.993 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.989+0000 7fa2bcd70640 1 -- 172.21.15.121:0/2292177595 learned_addr learned my addr 172.21.15.121:0/2292177595 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-24T22:58:47.993 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.989+0000 7fa2bdd72640 1 --2- 172.21.15.121:0/2292177595 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2b815c490 0x7fa2b8142ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:47.993 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.989+0000 7fa2bd571640 1 --2- 172.21.15.121:0/2292177595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2b80b59c0 0x7fa2b81f1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:47.993 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.989+0000 7fa2bcd70640 1 -- 172.21.15.121:0/2292177595 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2b815c490 msgr2=0x7fa2b8142ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:47.993 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.989+0000 7fa2bcd70640 1 --2- 172.21.15.121:0/2292177595 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2b815c490 0x7fa2b8142ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:47.993 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.989+0000 7fa2bcd70640 1 -- 172.21.15.121:0/2292177595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2b80b59c0 msgr2=0x7fa2b81f1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:47.993 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.989+0000 7fa2bcd70640 1 --2- 172.21.15.121:0/2292177595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2b80b59c0 0x7fa2b81f1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:47.993 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.989+0000 7fa2bcd70640 1 -- 172.21.15.121:0/2292177595 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2b8159e30 con 0x7fa2b80d2f30 2024-04-24T22:58:47.994 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.989+0000 7fa2bcd70640 1 --2- 172.21.15.121:0/2292177595 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2b80d2f30 0x7fa2b80d0b20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa2a800ebe0 tx=0x7fa2a800c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:58:47.994 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.989+0000 7fa2adffb640 1 -- 172.21.15.121:0/2292177595 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a8017070 con 0x7fa2b80d2f30 2024-04-24T22:58:47.995 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.989+0000 7fa2adffb640 1 -- 172.21.15.121:0/2292177595 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2a800cc30 con 0x7fa2b80d2f30 2024-04-24T22:58:47.995 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.989+0000 7fa2bf7d5640 1 -- 172.21.15.121:0/2292177595 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2b80d1570 con 0x7fa2b80d2f30 2024-04-24T22:58:47.995 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.989+0000 7fa2adffb640 1 -- 172.21.15.121:0/2292177595 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a8013470 con 0x7fa2b80d2f30 2024-04-24T22:58:47.995 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.989+0000 7fa2bf7d5640 1 -- 172.21.15.121:0/2292177595 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2b815a0b0 con 0x7fa2b80d2f30 2024-04-24T22:58:47.995 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.990+0000 7fa2adffb640 1 -- 172.21.15.121:0/2292177595 <== mon.2 v2:172.21.15.121:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa2a801a070 con 0x7fa2b80d2f30 2024-04-24T22:58:47.995 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.990+0000 7fa2adffb640 1 -- 172.21.15.121:0/2292177595 <== mon.2 v2:172.21.15.121:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7fa2a8013df0 con 0x7fa2b80d2f30 2024-04-24T22:58:47.995 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.991+0000 7fa2bf7d5640 1 -- 172.21.15.121:0/2292177595 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2b8068430 con 0x7fa2b80d2f30 2024-04-24T22:58:47.995 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:47.993+0000 7fa2adffb640 1 -- 172.21.15.121:0/2292177595 <== mon.2 v2:172.21.15.121:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fa2b8159e30 con 0x7fa2b80d2f30 2024-04-24T22:58:48.067 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.065+0000 7fa2bf7d5640 1 -- 172.21.15.121:0/2292177595 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 2} v 0) -- 0x7fa2b815a4b0 con 0x7fa2b80d2f30 2024-04-24T22:58:48.070 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.069+0000 7fa2adffb640 1 -- 172.21.15.121:0/2292177595 <== mon.2 v2:172.21.15.121:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7fa2a80139d0 con 0x7fa2b80d2f30 2024-04-24T22:58:48.070 INFO:teuthology.orchestra.run.smithi121.stdout:2 2024-04-24T22:58:48.071 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.070+0000 7fa2bf7d5640 1 -- 172.21.15.121:0/2292177595 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2b80d2f30 msgr2=0x7fa2b80d0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:48.071 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.070+0000 7fa2bf7d5640 1 --2- 172.21.15.121:0/2292177595 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2b80d2f30 0x7fa2b80d0b20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa2a800ebe0 tx=0x7fa2a800c520 comp rx=0 tx=0).stop 2024-04-24T22:58:48.072 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.070+0000 7fa2bf7d5640 1 -- 172.21.15.121:0/2292177595 shutdown_connections 2024-04-24T22:58:48.072 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.070+0000 7fa2bf7d5640 1 --2- 172.21.15.121:0/2292177595 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2b815c490 0x7fa2b8142ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:48.072 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.070+0000 7fa2bf7d5640 1 --2- 172.21.15.121:0/2292177595 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2b80d2f30 0x7fa2b80d0b20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:48.072 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.070+0000 7fa2bf7d5640 1 --2- 172.21.15.121:0/2292177595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2b80b59c0 0x7fa2b81f1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:48.072 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.070+0000 7fa2bf7d5640 1 -- 172.21.15.121:0/2292177595 >> 172.21.15.121:0/2292177595 conn(0x7fa2b80d2410 msgr2=0x7fa2b80d27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:58:48.072 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.070+0000 7fa2bf7d5640 1 -- 172.21.15.121:0/2292177595 shutdown_connections 2024-04-24T22:58:48.072 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.070+0000 7fa2bf7d5640 1 -- 172.21.15.121:0/2292177595 wait complete. 2024-04-24T22:58:48.081 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 3 2024-04-24T22:58:48.186 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.185+0000 7f06cbf3b640 1 Processor -- start 2024-04-24T22:58:48.187 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.185+0000 7f06cbf3b640 1 -- start start 2024-04-24T22:58:48.188 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.186+0000 7f06cbf3b640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f06c4160320 0x7f06c4074210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:48.188 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.186+0000 7f06cbf3b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06c4160900 con 0x7f06c4074750 2024-04-24T22:58:48.188 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.186+0000 7f06cbf3b640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06c4160be0 con 0x7f06c40b0700 2024-04-24T22:58:48.188 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.186+0000 7f06cbf3b640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f06c4160ec0 con 0x7f06c4160320 2024-04-24T22:58:48.188 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.186+0000 7f06c94d6640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f06c4160320 0x7f06c4074210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:48.188 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.186+0000 7f06c9cd7640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f06c40b0700 0x7f06c40731e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:53516/0 (socket says 172.21.15.121:53516) 2024-04-24T22:58:48.188 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.186+0000 7f06c9cd7640 1 -- 172.21.15.121:0/3292704359 learned_addr learned my addr 172.21.15.121:0/3292704359 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-24T22:58:48.188 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.186+0000 7f06c8cd5640 1 -- 172.21.15.121:0/3292704359 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1677840695 0 0) 0x7f06c4160be0 con 0x7f06c40b0700 2024-04-24T22:58:48.188 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.186+0000 7f06c8cd5640 1 -- 172.21.15.121:0/3292704359 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f06b0003520 con 0x7f06c40b0700 2024-04-24T22:58:48.189 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.186+0000 7f06c94d6640 1 -- 172.21.15.121:0/3292704359 >> v1:172.21.15.121:6789/0 conn(0x7f06c40b0700 legacy=0x7f06c40731e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:48.189 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.186+0000 7f06c94d6640 1 -- 172.21.15.121:0/3292704359 >> v1:172.21.15.32:6789/0 conn(0x7f06c4074750 legacy=0x7f06c40b1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:48.189 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.186+0000 7f06c94d6640 1 -- 172.21.15.121:0/3292704359 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06c41611a0 con 0x7f06c4160320 2024-04-24T22:58:48.190 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.187+0000 7f06c94d6640 1 --2- 172.21.15.121:0/3292704359 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06c4160320 0x7f06c4074210 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f06c00097a0 tx=0x7f06c002a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f3d22e6b0896201 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:58:48.190 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.187+0000 7f06c8cd5640 1 -- 172.21.15.121:0/3292704359 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c0037070 con 0x7f06c4160320 2024-04-24T22:58:48.190 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.188+0000 7f06c8cd5640 1 -- 172.21.15.121:0/3292704359 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06c0032440 con 0x7f06c4160320 2024-04-24T22:58:48.190 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.188+0000 7f06c8cd5640 1 -- 172.21.15.121:0/3292704359 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c0032810 con 0x7f06c4160320 2024-04-24T22:58:48.190 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.188+0000 7f06cbf3b640 1 -- 172.21.15.121:0/3292704359 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06c4160320 msgr2=0x7f06c4074210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:48.190 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.188+0000 7f06cbf3b640 1 --2- 172.21.15.121:0/3292704359 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06c4160320 0x7f06c4074210 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f06c00097a0 tx=0x7f06c002a230 comp rx=0 tx=0).stop 2024-04-24T22:58:48.191 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.188+0000 7f06cbf3b640 1 -- 172.21.15.121:0/3292704359 shutdown_connections 2024-04-24T22:58:48.191 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.188+0000 7f06cbf3b640 1 --2- 172.21.15.121:0/3292704359 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06c4160320 0x7f06c4074210 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:48.191 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.188+0000 7f06cbf3b640 1 -- 172.21.15.121:0/3292704359 >> 172.21.15.121:0/3292704359 conn(0x7f06c40b4de0 msgr2=0x7f06c4072850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:58:48.191 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.188+0000 7f06cbf3b640 1 -- 172.21.15.121:0/3292704359 shutdown_connections 2024-04-24T22:58:48.191 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.188+0000 7f06cbf3b640 1 -- 172.21.15.121:0/3292704359 wait complete. 2024-04-24T22:58:48.195 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.189+0000 7f06cbf3b640 1 Processor -- start 2024-04-24T22:58:48.195 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.189+0000 7f06cbf3b640 1 -- start start 2024-04-24T22:58:48.195 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.190+0000 7f06cbf3b640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f06c4074750 0x7f06c4157c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:48.195 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.190+0000 7f06cbf3b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06c40b0700 0x7f06c41552a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:48.196 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.190+0000 7f06cbf3b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06c4160320 0x7f06c409a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:48.196 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.190+0000 7f06cbf3b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f06c40b1aa0 con 0x7f06c4160320 2024-04-24T22:58:48.196 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.190+0000 7f06cbf3b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f06c40b0ae0 con 0x7f06c40b0700 2024-04-24T22:58:48.196 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.190+0000 7f06cbf3b640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f06c40b1c20 con 0x7f06c4074750 2024-04-24T22:58:48.196 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.190+0000 7f06c9cd7640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f06c4074750 0x7f06c4157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:48.196 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.190+0000 7f06c9cd7640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f06c4074750 0x7f06c4157c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.121:46574/0 (socket says 172.21.15.121:46574) 2024-04-24T22:58:48.196 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.190+0000 7f06c9cd7640 1 -- 172.21.15.121:0/2924442939 learned_addr learned my addr 172.21.15.121:0/2924442939 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-24T22:58:48.196 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.190+0000 7f06c94d6640 1 --2- 172.21.15.121:0/2924442939 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06c40b0700 0x7f06c41552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:48.196 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.190+0000 7f06ca4d8640 1 --2- 172.21.15.121:0/2924442939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06c4160320 0x7f06c409a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:48.196 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.191+0000 7f06c9cd7640 1 -- 172.21.15.121:0/2924442939 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06c40b0700 msgr2=0x7f06c41552a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:48.196 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.191+0000 7f06c9cd7640 1 --2- 172.21.15.121:0/2924442939 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06c40b0700 0x7f06c41552a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:48.196 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.191+0000 7f06c9cd7640 1 -- 172.21.15.121:0/2924442939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06c4160320 msgr2=0x7f06c409a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:48.196 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.191+0000 7f06c9cd7640 1 --2- 172.21.15.121:0/2924442939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06c4160320 0x7f06c409a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:48.196 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.191+0000 7f06c9cd7640 1 -- 172.21.15.121:0/2924442939 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06c40c53c0 con 0x7f06c4074750 2024-04-24T22:58:48.196 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.191+0000 7f06c9cd7640 1 --2- 172.21.15.121:0/2924442939 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f06c4074750 0x7f06c4157c20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f06b400c530 tx=0x7f06b400c9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:58:48.197 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.191+0000 7f06ba7fc640 1 -- 172.21.15.121:0/2924442939 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b4017070 con 0x7f06c4074750 2024-04-24T22:58:48.197 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.191+0000 7f06ba7fc640 1 -- 172.21.15.121:0/2924442939 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06b4005ce0 con 0x7f06c4074750 2024-04-24T22:58:48.197 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.191+0000 7f06ba7fc640 1 -- 172.21.15.121:0/2924442939 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b40072b0 con 0x7f06c4074750 2024-04-24T22:58:48.197 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.191+0000 7f06cbf3b640 1 -- 172.21.15.121:0/2924442939 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06c40c56a0 con 0x7f06c4074750 2024-04-24T22:58:48.197 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.191+0000 7f06cbf3b640 1 -- 172.21.15.121:0/2924442939 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f06c40c5c60 con 0x7f06c4074750 2024-04-24T22:58:48.197 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.191+0000 7f06ba7fc640 1 -- 172.21.15.121:0/2924442939 <== mon.2 v2:172.21.15.121:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f06b401a070 con 0x7f06c4074750 2024-04-24T22:58:48.197 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.191+0000 7f06ba7fc640 1 -- 172.21.15.121:0/2924442939 <== mon.2 v2:172.21.15.121:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f06b4012890 con 0x7f06c4074750 2024-04-24T22:58:48.197 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.192+0000 7f06cbf3b640 1 -- 172.21.15.121:0/2924442939 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06c4068430 con 0x7f06c4074750 2024-04-24T22:58:48.197 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.194+0000 7f06ba7fc640 1 -- 172.21.15.121:0/2924442939 <== mon.2 v2:172.21.15.121:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f06b4021070 con 0x7f06c4074750 2024-04-24T22:58:48.268 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.266+0000 7f06cbf3b640 1 -- 172.21.15.121:0/2924442939 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 3} v 0) -- 0x7f06c4074e80 con 0x7f06c4074750 2024-04-24T22:58:48.270 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.269+0000 7f06ba7fc640 1 -- 172.21.15.121:0/2924442939 <== mon.2 v2:172.21.15.121:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f06b4012400 con 0x7f06c4074750 2024-04-24T22:58:48.271 INFO:teuthology.orchestra.run.smithi121.stdout:3 2024-04-24T22:58:48.272 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.270+0000 7f06cbf3b640 1 -- 172.21.15.121:0/2924442939 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f06c4074750 msgr2=0x7f06c4157c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:48.272 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.270+0000 7f06cbf3b640 1 --2- 172.21.15.121:0/2924442939 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f06c4074750 0x7f06c4157c20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f06b400c530 tx=0x7f06b400c9f0 comp rx=0 tx=0).stop 2024-04-24T22:58:48.272 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.270+0000 7f06cbf3b640 1 -- 172.21.15.121:0/2924442939 shutdown_connections 2024-04-24T22:58:48.272 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.270+0000 7f06cbf3b640 1 --2- 172.21.15.121:0/2924442939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06c4160320 0x7f06c409a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:48.272 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.270+0000 7f06cbf3b640 1 --2- 172.21.15.121:0/2924442939 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06c40b0700 0x7f06c41552a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:48.272 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.270+0000 7f06cbf3b640 1 --2- 172.21.15.121:0/2924442939 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f06c4074750 0x7f06c4157c20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:48.272 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.270+0000 7f06cbf3b640 1 -- 172.21.15.121:0/2924442939 >> 172.21.15.121:0/2924442939 conn(0x7f06c40b4de0 msgr2=0x7f06c415ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:58:48.272 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.270+0000 7f06cbf3b640 1 -- 172.21.15.121:0/2924442939 shutdown_connections 2024-04-24T22:58:48.272 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.270+0000 7f06cbf3b640 1 -- 172.21.15.121:0/2924442939 wait complete. 2024-04-24T22:58:48.285 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 4 2024-04-24T22:58:48.356 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-24T22:58:48.355+0000 7f2c22579040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T22:58:48.360 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-04-24T22:58:48.359+0000 7f8308c0d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T22:58:48.382 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.381+0000 7f7ff4e64640 1 Processor -- start 2024-04-24T22:58:48.382 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.381+0000 7f7ff4e64640 1 -- start start 2024-04-24T22:58:48.384 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.381+0000 7f7ff4e64640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7ff015c5d0 0x7f7ff00b5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:48.384 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.381+0000 7f7ff4e64640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7ff0072e70 0x7f7ff0073300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:48.384 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.381+0000 7f7ff4e64640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ff00b5cf0 con 0x7f7ff00d1c10 2024-04-24T22:58:48.384 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.381+0000 7f7ff4e64640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f7ff0161160 con 0x7f7ff0072e70 2024-04-24T22:58:48.385 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.381+0000 7f7ff4e64640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f7ff00b5fd0 con 0x7f7ff015c5d0 2024-04-24T22:58:48.385 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.382+0000 7f7fee59c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7ff015c5d0 0x7f7ff00b5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:48.385 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.382+0000 7f7feed9d640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7ff0072e70 0x7f7ff0073300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:48.385 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.382+0000 7f7feed9d640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7ff0072e70 0x7f7ff0073300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:45350/0 (socket says 172.21.15.121:45350) 2024-04-24T22:58:48.385 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.382+0000 7f7fee59c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7ff015c5d0 0x7f7ff00b5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.121:46576/0 (socket says 172.21.15.121:46576) 2024-04-24T22:58:48.385 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.382+0000 7f7feed9d640 1 -- 172.21.15.121:0/3827370238 learned_addr learned my addr 172.21.15.121:0/3827370238 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-24T22:58:48.385 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.382+0000 7f7fee59c640 1 -- 172.21.15.121:0/3827370238 >> v2:172.21.15.121:3300/0 conn(0x7f7ff0072e70 msgr2=0x7f7ff0073300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:58:48.385 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.382+0000 7f7fee59c640 1 --2- 172.21.15.121:0/3827370238 >> v2:172.21.15.121:3300/0 conn(0x7f7ff0072e70 0x7f7ff0073300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:48.385 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.382+0000 7f7fee59c640 1 -- 172.21.15.121:0/3827370238 >> v1:172.21.15.32:6789/0 conn(0x7f7ff00d1c10 legacy=0x7f7ff0160a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:48.385 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.382+0000 7f7fee59c640 1 -- 172.21.15.121:0/3827370238 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ff00b62b0 con 0x7f7ff015c5d0 2024-04-24T22:58:48.385 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.382+0000 7f7feed9d640 1 --2- 172.21.15.121:0/3827370238 >> v2:172.21.15.121:3300/0 conn(0x7f7ff0072e70 0x7f7ff0073300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:58:48.385 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.383+0000 7f7fee59c640 1 --2- 172.21.15.121:0/3827370238 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7ff015c5d0 0x7f7ff00b5410 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7fe00097a0 tx=0x7f7fe002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45e52ccefa2e2f21 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:58:48.385 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.383+0000 7f7fedd9b640 1 -- 172.21.15.121:0/3827370238 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe0037070 con 0x7f7ff015c5d0 2024-04-24T22:58:48.385 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.383+0000 7f7fedd9b640 1 -- 172.21.15.121:0/3827370238 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fe0032440 con 0x7f7ff015c5d0 2024-04-24T22:58:48.386 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.383+0000 7f7fedd9b640 1 -- 172.21.15.121:0/3827370238 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe0032810 con 0x7f7ff015c5d0 2024-04-24T22:58:48.389 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.383+0000 7f7ff4e64640 1 -- 172.21.15.121:0/3827370238 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7ff015c5d0 msgr2=0x7f7ff00b5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:48.389 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.383+0000 7f7ff4e64640 1 --2- 172.21.15.121:0/3827370238 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7ff015c5d0 0x7f7ff00b5410 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7fe00097a0 tx=0x7f7fe002a180 comp rx=0 tx=0).stop 2024-04-24T22:58:48.389 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.383+0000 7f7ff4e64640 1 -- 172.21.15.121:0/3827370238 shutdown_connections 2024-04-24T22:58:48.389 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.383+0000 7f7ff4e64640 1 --2- 172.21.15.121:0/3827370238 >> v2:172.21.15.121:3300/0 conn(0x7f7ff0072e70 0x7f7ff0073300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:48.389 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.383+0000 7f7ff4e64640 1 --2- 172.21.15.121:0/3827370238 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7ff015c5d0 0x7f7ff00b5410 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:48.389 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.383+0000 7f7ff4e64640 1 -- 172.21.15.121:0/3827370238 >> 172.21.15.121:0/3827370238 conn(0x7f7ff00d24c0 msgr2=0x7f7ff00d28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:58:48.389 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.383+0000 7f7ff4e64640 1 -- 172.21.15.121:0/3827370238 shutdown_connections 2024-04-24T22:58:48.389 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.383+0000 7f7ff4e64640 1 -- 172.21.15.121:0/3827370238 wait complete. 2024-04-24T22:58:48.389 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.385+0000 7f7ff4e64640 1 Processor -- start 2024-04-24T22:58:48.389 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.385+0000 7f7ff4e64640 1 -- start start 2024-04-24T22:58:48.390 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.385+0000 7f7ff4e64640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7ff0072e70 0x7f7ff0140bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:48.390 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.385+0000 7f7ff4e64640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ff00d1c10 0x7f7ff0143620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:48.390 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.385+0000 7f7ff4e64640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ff015c5d0 0x7f7ff015ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:48.390 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.385+0000 7f7ff4e64640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff00b5950 con 0x7f7ff00d1c10 2024-04-24T22:58:48.390 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.385+0000 7f7ff4e64640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff015c9b0 con 0x7f7ff015c5d0 2024-04-24T22:58:48.390 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.385+0000 7f7ff4e64640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f7ff00b5ad0 con 0x7f7ff0072e70 2024-04-24T22:58:48.390 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.386+0000 7f7fee59c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7ff0072e70 0x7f7ff0140bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:48.390 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.386+0000 7f7fee59c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7ff0072e70 0x7f7ff0140bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.121:46590/0 (socket says 172.21.15.121:46590) 2024-04-24T22:58:48.390 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.386+0000 7f7fee59c640 1 -- 172.21.15.121:0/1667321643 learned_addr learned my addr 172.21.15.121:0/1667321643 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-24T22:58:48.390 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.386+0000 7f7feed9d640 1 --2- 172.21.15.121:0/1667321643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ff015c5d0 0x7f7ff015ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:48.390 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.386+0000 7f7fe7fff640 1 --2- 172.21.15.121:0/1667321643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ff00d1c10 0x7f7ff0143620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:48.390 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.386+0000 7f7fee59c640 1 -- 172.21.15.121:0/1667321643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ff015c5d0 msgr2=0x7f7ff015ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:48.391 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.386+0000 7f7fee59c640 1 --2- 172.21.15.121:0/1667321643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ff015c5d0 0x7f7ff015ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:48.391 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.386+0000 7f7fee59c640 1 -- 172.21.15.121:0/1667321643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ff00d1c10 msgr2=0x7f7ff0143620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:48.391 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.386+0000 7f7fee59c640 1 --2- 172.21.15.121:0/1667321643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ff00d1c10 0x7f7ff0143620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:48.391 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.386+0000 7f7fee59c640 1 -- 172.21.15.121:0/1667321643 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ff0141100 con 0x7f7ff0072e70 2024-04-24T22:58:48.391 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.386+0000 7f7feed9d640 1 --2- 172.21.15.121:0/1667321643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ff015c5d0 0x7f7ff015ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:58:48.391 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.386+0000 7f7fee59c640 1 --2- 172.21.15.121:0/1667321643 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7ff0072e70 0x7f7ff0140bc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0035040 tx=0x7f7fe00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:58:48.391 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.386+0000 7f7fe6ffd640 1 -- 172.21.15.121:0/1667321643 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe0037040 con 0x7f7ff0072e70 2024-04-24T22:58:48.391 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.386+0000 7f7fe6ffd640 1 -- 172.21.15.121:0/1667321643 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fe0030500 con 0x7f7ff0072e70 2024-04-24T22:58:48.391 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.386+0000 7f7ff4e64640 1 -- 172.21.15.121:0/1667321643 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ff0143b60 con 0x7f7ff0072e70 2024-04-24T22:58:48.391 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.386+0000 7f7fe6ffd640 1 -- 172.21.15.121:0/1667321643 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe0030910 con 0x7f7ff0072e70 2024-04-24T22:58:48.391 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.386+0000 7f7ff4e64640 1 -- 172.21.15.121:0/1667321643 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ff0156580 con 0x7f7ff0072e70 2024-04-24T22:58:48.391 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.386+0000 7f7fe6ffd640 1 -- 172.21.15.121:0/1667321643 <== mon.2 v2:172.21.15.121:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7fe0044020 con 0x7f7ff0072e70 2024-04-24T22:58:48.391 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.386+0000 7f7fe6ffd640 1 -- 172.21.15.121:0/1667321643 <== mon.2 v2:172.21.15.121:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f7fe003bb60 con 0x7f7ff0072e70 2024-04-24T22:58:48.391 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.387+0000 7f7ff4e64640 1 -- 172.21.15.121:0/1667321643 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ff0068430 con 0x7f7ff0072e70 2024-04-24T22:58:48.392 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.389+0000 7f7fe6ffd640 1 -- 172.21.15.121:0/1667321643 <== mon.2 v2:172.21.15.121:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f7fe003be00 con 0x7f7ff0072e70 2024-04-24T22:58:48.448 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-24T22:58:48.447+0000 7f2c22579040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T22:58:48.452 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-04-24T22:58:48.451+0000 7f8308c0d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T22:58:48.463 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.462+0000 7f7ff4e64640 1 -- 172.21.15.121:0/1667321643 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 4} v 0) -- 0x7f7ff0072a20 con 0x7f7ff0072e70 2024-04-24T22:58:48.466 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.465+0000 7f7fe6ffd640 1 -- 172.21.15.121:0/1667321643 <== mon.2 v2:172.21.15.121:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f7fe003b430 con 0x7f7ff0072e70 2024-04-24T22:58:48.466 INFO:teuthology.orchestra.run.smithi121.stdout:4 2024-04-24T22:58:48.468 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.466+0000 7f7ff4e64640 1 -- 172.21.15.121:0/1667321643 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7ff0072e70 msgr2=0x7f7ff0140bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:48.468 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.466+0000 7f7ff4e64640 1 --2- 172.21.15.121:0/1667321643 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7ff0072e70 0x7f7ff0140bc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0035040 tx=0x7f7fe00046c0 comp rx=0 tx=0).stop 2024-04-24T22:58:48.468 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.466+0000 7f7ff4e64640 1 -- 172.21.15.121:0/1667321643 shutdown_connections 2024-04-24T22:58:48.468 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.466+0000 7f7ff4e64640 1 --2- 172.21.15.121:0/1667321643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ff00d1c10 0x7f7ff0143620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:48.468 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.466+0000 7f7ff4e64640 1 --2- 172.21.15.121:0/1667321643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ff015c5d0 0x7f7ff015ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:48.468 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.466+0000 7f7ff4e64640 1 --2- 172.21.15.121:0/1667321643 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7ff0072e70 0x7f7ff0140bc0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:48.468 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.466+0000 7f7ff4e64640 1 -- 172.21.15.121:0/1667321643 >> 172.21.15.121:0/1667321643 conn(0x7f7ff00d24c0 msgr2=0x7f7ff00b6730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:58:48.468 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.466+0000 7f7ff4e64640 1 -- 172.21.15.121:0/1667321643 shutdown_connections 2024-04-24T22:58:48.468 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.466+0000 7f7ff4e64640 1 -- 172.21.15.121:0/1667321643 wait complete. 2024-04-24T22:58:48.482 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 5 2024-04-24T22:58:48.493 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-24T22:58:48.492+0000 7f2c22579040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T22:58:48.502 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-04-24T22:58:48.501+0000 7f8308c0d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T22:58:48.578 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.577+0000 7fedae86b640 1 Processor -- start 2024-04-24T22:58:48.579 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.577+0000 7fedae86b640 1 -- start start 2024-04-24T22:58:48.580 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.578+0000 7fedae86b640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feda80ce510 0x7feda80b0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:48.580 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.578+0000 7fedae86b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7feda80b0990 0x7feda80728c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:48.580 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.578+0000 7fedae86b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feda815cf70 con 0x7feda80cd7e0 2024-04-24T22:58:48.581 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.578+0000 7fedae86b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7feda815db80 con 0x7feda80b0990 2024-04-24T22:58:48.581 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.578+0000 7fedae86b640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7feda815d250 con 0x7feda80ce510 2024-04-24T22:58:48.581 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.578+0000 7feda7fff640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feda80ce510 0x7feda80b0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:48.581 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.578+0000 7feda7fff640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feda80ce510 0x7feda80b0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.121:46600/0 (socket says 172.21.15.121:46600) 2024-04-24T22:58:48.581 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.578+0000 7feda7fff640 1 -- 172.21.15.121:0/3573046701 learned_addr learned my addr 172.21.15.121:0/3573046701 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-24T22:58:48.581 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.578+0000 7feda77fe640 1 --2- 172.21.15.121:0/3573046701 >> v2:172.21.15.32:3300/0 conn(0x7feda80b0990 0x7feda80728c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:48.581 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.578+0000 7feda7fff640 1 -- 172.21.15.121:0/3573046701 >> v2:172.21.15.32:3300/0 conn(0x7feda80b0990 msgr2=0x7feda80728c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:58:48.581 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.578+0000 7feda7fff640 1 --2- 172.21.15.121:0/3573046701 >> v2:172.21.15.32:3300/0 conn(0x7feda80b0990 0x7feda80728c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:48.581 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.578+0000 7feda7fff640 1 -- 172.21.15.121:0/3573046701 >> v1:172.21.15.32:6789/0 conn(0x7feda80cd7e0 legacy=0x7feda8074210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:48.581 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.578+0000 7feda7fff640 1 -- 172.21.15.121:0/3573046701 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feda815d530 con 0x7feda80ce510 2024-04-24T22:58:48.581 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.579+0000 7feda7fff640 1 --2- 172.21.15.121:0/3573046701 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feda80ce510 0x7feda80b0450 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fed9c009870 tx=0x7fed9c02a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed4736bc079bcbde server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:58:48.581 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.579+0000 7feda6ffd640 1 -- 172.21.15.121:0/3573046701 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9c037070 con 0x7feda80ce510 2024-04-24T22:58:48.581 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.579+0000 7feda6ffd640 1 -- 172.21.15.121:0/3573046701 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed9c032440 con 0x7feda80ce510 2024-04-24T22:58:48.581 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.579+0000 7feda6ffd640 1 -- 172.21.15.121:0/3573046701 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9c032810 con 0x7feda80ce510 2024-04-24T22:58:48.582 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.580+0000 7fedae86b640 1 -- 172.21.15.121:0/3573046701 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feda80ce510 msgr2=0x7feda80b0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:48.583 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.580+0000 7fedae86b640 1 --2- 172.21.15.121:0/3573046701 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feda80ce510 0x7feda80b0450 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fed9c009870 tx=0x7fed9c02a1c0 comp rx=0 tx=0).stop 2024-04-24T22:58:48.583 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.580+0000 7fedae86b640 1 -- 172.21.15.121:0/3573046701 shutdown_connections 2024-04-24T22:58:48.583 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.580+0000 7fedae86b640 1 --2- 172.21.15.121:0/3573046701 >> v2:172.21.15.32:3300/0 conn(0x7feda80b0990 0x7feda80728c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:48.583 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.580+0000 7fedae86b640 1 --2- 172.21.15.121:0/3573046701 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feda80ce510 0x7feda80b0450 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:48.583 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.580+0000 7fedae86b640 1 -- 172.21.15.121:0/3573046701 >> 172.21.15.121:0/3573046701 conn(0x7feda8071a10 msgr2=0x7feda80b6130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:58:48.583 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.580+0000 7fedae86b640 1 -- 172.21.15.121:0/3573046701 shutdown_connections 2024-04-24T22:58:48.583 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.580+0000 7fedae86b640 1 -- 172.21.15.121:0/3573046701 wait complete. 2024-04-24T22:58:48.583 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.581+0000 7fedae86b640 1 Processor -- start 2024-04-24T22:58:48.583 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.581+0000 7fedae86b640 1 -- start start 2024-04-24T22:58:48.585 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.582+0000 7fedae86b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feda80b0990 0x7feda80d1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:48.585 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.582+0000 7fedae86b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feda80cd7e0 0x7feda80cc920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:48.586 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.582+0000 7fedae86b640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feda80ce510 0x7feda815b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:48.586 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.582+0000 7fedae86b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7feda80ce8f0 con 0x7feda80b0990 2024-04-24T22:58:48.586 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.582+0000 7fedae86b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7feda806cba0 con 0x7feda80cd7e0 2024-04-24T22:58:48.586 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.582+0000 7fedae86b640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7feda8072e00 con 0x7feda80ce510 2024-04-24T22:58:48.586 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.582+0000 7feda77fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feda80cd7e0 0x7feda80cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:48.586 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.582+0000 7fedace08640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feda80ce510 0x7feda815b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:48.586 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.582+0000 7feda77fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feda80cd7e0 0x7feda80cc920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:45366/0 (socket says 172.21.15.121:45366) 2024-04-24T22:58:48.586 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.582+0000 7feda77fe640 1 -- 172.21.15.121:0/3612417492 learned_addr learned my addr 172.21.15.121:0/3612417492 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-24T22:58:48.586 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.582+0000 7fedace08640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feda80ce510 0x7feda815b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.121:46616/0 (socket says 172.21.15.121:46616) 2024-04-24T22:58:48.586 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.582+0000 7feda7fff640 1 --2- 172.21.15.121:0/3612417492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feda80b0990 0x7feda80d1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:48.586 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.582+0000 7fedace08640 1 -- 172.21.15.121:0/3612417492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feda80cd7e0 msgr2=0x7feda80cc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:48.586 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.582+0000 7fedace08640 1 --2- 172.21.15.121:0/3612417492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feda80cd7e0 0x7feda80cc920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:48.586 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.582+0000 7fedace08640 1 -- 172.21.15.121:0/3612417492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feda80b0990 msgr2=0x7feda80d1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:48.586 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.582+0000 7fedace08640 1 --2- 172.21.15.121:0/3612417492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feda80b0990 0x7feda80d1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:48.587 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.582+0000 7fedace08640 1 -- 172.21.15.121:0/3612417492 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feda8152ba0 con 0x7feda80ce510 2024-04-24T22:58:48.587 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.582+0000 7feda77fe640 1 --2- 172.21.15.121:0/3612417492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feda80cd7e0 0x7feda80cc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:58:48.587 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.582+0000 7feda7fff640 1 --2- 172.21.15.121:0/3612417492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feda80b0990 0x7feda80d1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:58:48.587 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.582+0000 7fedace08640 1 --2- 172.21.15.121:0/3612417492 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feda80ce510 0x7feda815b9d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7feda000c690 tx=0x7feda000cb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:58:48.587 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.582+0000 7feda4ff9640 1 -- 172.21.15.121:0/3612417492 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda0003b70 con 0x7feda80ce510 2024-04-24T22:58:48.587 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.582+0000 7feda4ff9640 1 -- 172.21.15.121:0/3612417492 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feda0003d10 con 0x7feda80ce510 2024-04-24T22:58:48.587 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.582+0000 7fedae86b640 1 -- 172.21.15.121:0/3612417492 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feda8152e40 con 0x7feda80ce510 2024-04-24T22:58:48.587 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.582+0000 7feda4ff9640 1 -- 172.21.15.121:0/3612417492 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda0005310 con 0x7feda80ce510 2024-04-24T22:58:48.587 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.583+0000 7fedae86b640 1 -- 172.21.15.121:0/3612417492 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7feda8153400 con 0x7feda80ce510 2024-04-24T22:58:48.587 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.583+0000 7feda4ff9640 1 -- 172.21.15.121:0/3612417492 <== mon.2 v2:172.21.15.121:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7feda00054b0 con 0x7feda80ce510 2024-04-24T22:58:48.588 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.583+0000 7feda4ff9640 1 -- 172.21.15.121:0/3612417492 <== mon.2 v2:172.21.15.121:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7feda00129e0 con 0x7feda80ce510 2024-04-24T22:58:48.588 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.584+0000 7fedae86b640 1 -- 172.21.15.121:0/3612417492 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feda806b110 con 0x7feda80ce510 2024-04-24T22:58:48.588 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.586+0000 7feda4ff9640 1 -- 172.21.15.121:0/3612417492 <== mon.2 v2:172.21.15.121:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7feda806b110 con 0x7feda80ce510 2024-04-24T22:58:48.659 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.658+0000 7fedae86b640 1 -- 172.21.15.121:0/3612417492 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 5} v 0) -- 0x7feda8068430 con 0x7feda80ce510 2024-04-24T22:58:48.662 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.661+0000 7feda4ff9640 1 -- 172.21.15.121:0/3612417492 <== mon.2 v2:172.21.15.121:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7feda0012c80 con 0x7feda80ce510 2024-04-24T22:58:48.662 INFO:teuthology.orchestra.run.smithi121.stdout:5 2024-04-24T22:58:48.663 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.662+0000 7fedae86b640 1 -- 172.21.15.121:0/3612417492 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feda80ce510 msgr2=0x7feda815b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:48.663 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.662+0000 7fedae86b640 1 --2- 172.21.15.121:0/3612417492 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feda80ce510 0x7feda815b9d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7feda000c690 tx=0x7feda000cb50 comp rx=0 tx=0).stop 2024-04-24T22:58:48.663 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.662+0000 7fedae86b640 1 -- 172.21.15.121:0/3612417492 shutdown_connections 2024-04-24T22:58:48.663 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.662+0000 7fedae86b640 1 --2- 172.21.15.121:0/3612417492 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feda80ce510 0x7feda815b9d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:48.663 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.662+0000 7fedae86b640 1 --2- 172.21.15.121:0/3612417492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feda80cd7e0 0x7feda80cc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:48.663 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.662+0000 7fedae86b640 1 --2- 172.21.15.121:0/3612417492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feda80b0990 0x7feda80d1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:48.664 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.662+0000 7fedae86b640 1 -- 172.21.15.121:0/3612417492 >> 172.21.15.121:0/3612417492 conn(0x7feda8071a10 msgr2=0x7feda806bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:58:48.664 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.662+0000 7fedae86b640 1 -- 172.21.15.121:0/3612417492 shutdown_connections 2024-04-24T22:58:48.664 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.662+0000 7fedae86b640 1 -- 172.21.15.121:0/3612417492 wait complete. 2024-04-24T22:58:48.667 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-24T22:58:48.666+0000 7f2c22579040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T22:58:48.673 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 6 2024-04-24T22:58:48.677 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-04-24T22:58:48.676+0000 7f8308c0d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T22:58:48.719 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-24T22:58:48.718+0000 7f2c22579040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T22:58:48.746 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-04-24T22:58:48.745+0000 7f8308c0d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T22:58:48.773 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.772+0000 7f397f1e7640 1 Processor -- start 2024-04-24T22:58:48.774 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.772+0000 7f397f1e7640 1 -- start start 2024-04-24T22:58:48.775 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.773+0000 7f397f1e7640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f39780704b0 0x7f3978160c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:48.775 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.773+0000 7f397f1e7640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f39780d03e0 0x7f39780b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:48.776 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.773+0000 7f397f1e7640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39780d08a0 con 0x7f397815c6e0 2024-04-24T22:58:48.776 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.773+0000 7f397f1e7640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f39780d0b80 con 0x7f39780704b0 2024-04-24T22:58:48.776 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.773+0000 7f397f1e7640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f39780d0e60 con 0x7f39780d03e0 2024-04-24T22:58:48.776 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.773+0000 7f397d784640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f39780d03e0 0x7f39780b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:48.776 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.773+0000 7f397cf83640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f397815c6e0 0x7f397806fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:53518/0 (socket says 172.21.15.121:53518) 2024-04-24T22:58:48.776 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.773+0000 7f397cf83640 1 -- 172.21.15.121:0/487791231 learned_addr learned my addr 172.21.15.121:0/487791231 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-24T22:58:48.776 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.773+0000 7f3977fff640 1 --2- 172.21.15.121:0/487791231 >> v2:172.21.15.32:3300/0 conn(0x7f39780704b0 0x7f3978160c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:48.776 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.773+0000 7f397d784640 1 -- 172.21.15.121:0/487791231 >> v2:172.21.15.32:3300/0 conn(0x7f39780704b0 msgr2=0x7f3978160c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:58:48.776 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.773+0000 7f397d784640 1 --2- 172.21.15.121:0/487791231 >> v2:172.21.15.32:3300/0 conn(0x7f39780704b0 0x7f3978160c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:48.776 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.773+0000 7f397d784640 1 -- 172.21.15.121:0/487791231 >> v1:172.21.15.121:6789/0 conn(0x7f397815c6e0 legacy=0x7f397806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:48.776 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.773+0000 7f39777fe640 1 -- 172.21.15.121:0/487791231 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1317003945 0 0) 0x7f39780d08a0 con 0x7f397815c6e0 2024-04-24T22:58:48.776 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.773+0000 7f397d784640 1 -- 172.21.15.121:0/487791231 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39780d1140 con 0x7f39780d03e0 2024-04-24T22:58:48.776 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.773+0000 7f3977fff640 1 --2- 172.21.15.121:0/487791231 >> v2:172.21.15.32:3300/0 conn(0x7f39780704b0 0x7f3978160c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:58:48.776 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.774+0000 7f397d784640 1 --2- 172.21.15.121:0/487791231 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f39780d03e0 0x7f39780b54f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3970009d90 tx=0x7f397002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85d4702164cba15b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:58:48.777 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.774+0000 7f39777fe640 1 -- 172.21.15.121:0/487791231 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3970007740 con 0x7f39780d03e0 2024-04-24T22:58:48.777 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.774+0000 7f39777fe640 1 -- 172.21.15.121:0/487791231 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39700078e0 con 0x7f39780d03e0 2024-04-24T22:58:48.777 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.774+0000 7f39777fe640 1 -- 172.21.15.121:0/487791231 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3970007cb0 con 0x7f39780d03e0 2024-04-24T22:58:48.777 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-24T22:58:48.773+0000 7f2c22579040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T22:58:48.780 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.775+0000 7f397f1e7640 1 -- 172.21.15.121:0/487791231 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f39780d03e0 msgr2=0x7f39780b54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:48.780 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.775+0000 7f397f1e7640 1 --2- 172.21.15.121:0/487791231 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f39780d03e0 0x7f39780b54f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3970009d90 tx=0x7f397002a660 comp rx=0 tx=0).stop 2024-04-24T22:58:48.780 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.775+0000 7f397f1e7640 1 -- 172.21.15.121:0/487791231 shutdown_connections 2024-04-24T22:58:48.780 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.775+0000 7f397f1e7640 1 --2- 172.21.15.121:0/487791231 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f39780d03e0 0x7f39780b54f0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:48.780 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.775+0000 7f397f1e7640 1 --2- 172.21.15.121:0/487791231 >> v2:172.21.15.32:3300/0 conn(0x7f39780704b0 0x7f3978160c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:48.780 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.775+0000 7f397f1e7640 1 -- 172.21.15.121:0/487791231 >> 172.21.15.121:0/487791231 conn(0x7f397806d320 msgr2=0x7f397806d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:58:48.781 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.775+0000 7f397f1e7640 1 -- 172.21.15.121:0/487791231 shutdown_connections 2024-04-24T22:58:48.781 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.775+0000 7f397f1e7640 1 -- 172.21.15.121:0/487791231 wait complete. 2024-04-24T22:58:48.781 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.776+0000 7f397f1e7640 1 Processor -- start 2024-04-24T22:58:48.781 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.776+0000 7f397f1e7640 1 -- start start 2024-04-24T22:58:48.781 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.777+0000 7f397f1e7640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f39780704b0 0x7f397815cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:48.781 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.777+0000 7f397f1e7640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f39780d03e0 0x7f3978143920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:48.781 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.777+0000 7f397f1e7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f397815c6e0 0x7f39780aa4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:48.781 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.777+0000 7f397f1e7640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f397806e070 con 0x7f397815c6e0 2024-04-24T22:58:48.781 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.777+0000 7f397f1e7640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f397815cac0 con 0x7f39780704b0 2024-04-24T22:58:48.781 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.777+0000 7f397f1e7640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f39780b5a30 con 0x7f39780d03e0 2024-04-24T22:58:48.781 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.777+0000 7f397cf83640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f39780704b0 0x7f397815cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:48.781 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.777+0000 7f3977fff640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f39780d03e0 0x7f3978143920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:48.781 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.777+0000 7f397cf83640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f39780704b0 0x7f397815cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:45368/0 (socket says 172.21.15.121:45368) 2024-04-24T22:58:48.781 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.777+0000 7f397cf83640 1 -- 172.21.15.121:0/1615887476 learned_addr learned my addr 172.21.15.121:0/1615887476 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-24T22:58:48.782 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.777+0000 7f397d784640 1 --2- 172.21.15.121:0/1615887476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f397815c6e0 0x7f39780aa4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:48.782 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.777+0000 7f3977fff640 1 -- 172.21.15.121:0/1615887476 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f39780704b0 msgr2=0x7f397815cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:48.782 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.777+0000 7f3977fff640 1 --2- 172.21.15.121:0/1615887476 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f39780704b0 0x7f397815cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:48.782 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.777+0000 7f3977fff640 1 -- 172.21.15.121:0/1615887476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f397815c6e0 msgr2=0x7f39780aa4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:48.782 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.777+0000 7f3977fff640 1 --2- 172.21.15.121:0/1615887476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f397815c6e0 0x7f39780aa4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:48.782 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.777+0000 7f3977fff640 1 -- 172.21.15.121:0/1615887476 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3978143e60 con 0x7f39780d03e0 2024-04-24T22:58:48.782 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.777+0000 7f3977fff640 1 --2- 172.21.15.121:0/1615887476 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f39780d03e0 0x7f3978143920 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f396400e7e0 tx=0x7f396400eca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:58:48.782 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.778+0000 7f39757fa640 1 -- 172.21.15.121:0/1615887476 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396400f040 con 0x7f39780d03e0 2024-04-24T22:58:48.782 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.778+0000 7f39757fa640 1 -- 172.21.15.121:0/1615887476 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3964004030 con 0x7f39780d03e0 2024-04-24T22:58:48.782 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.778+0000 7f397f1e7640 1 -- 172.21.15.121:0/1615887476 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3978144210 con 0x7f39780d03e0 2024-04-24T22:58:48.782 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.778+0000 7f39757fa640 1 -- 172.21.15.121:0/1615887476 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3964004420 con 0x7f39780d03e0 2024-04-24T22:58:48.782 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.778+0000 7f397f1e7640 1 -- 172.21.15.121:0/1615887476 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f39781447d0 con 0x7f39780d03e0 2024-04-24T22:58:48.782 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.778+0000 7f39757fa640 1 -- 172.21.15.121:0/1615887476 <== mon.2 v2:172.21.15.121:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f396400ca60 con 0x7f39780d03e0 2024-04-24T22:58:48.782 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.778+0000 7f39757fa640 1 -- 172.21.15.121:0/1615887476 <== mon.2 v2:172.21.15.121:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f3964013a50 con 0x7f39780d03e0 2024-04-24T22:58:48.782 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.779+0000 7f397f1e7640 1 -- 172.21.15.121:0/1615887476 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39780690e0 con 0x7f39780d03e0 2024-04-24T22:58:48.783 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.781+0000 7f39757fa640 1 -- 172.21.15.121:0/1615887476 <== mon.2 v2:172.21.15.121:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f39780690e0 con 0x7f39780d03e0 2024-04-24T22:58:48.802 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-04-24T22:58:48.800+0000 7f8308c0d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T22:58:48.853 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.851+0000 7f397f1e7640 1 -- 172.21.15.121:0/1615887476 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 6} v 0) -- 0x7f3978068430 con 0x7f39780d03e0 2024-04-24T22:58:48.855 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.854+0000 7f39757fa640 1 -- 172.21.15.121:0/1615887476 <== mon.2 v2:172.21.15.121:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f3964017020 con 0x7f39780d03e0 2024-04-24T22:58:48.855 INFO:teuthology.orchestra.run.smithi121.stdout:6 2024-04-24T22:58:48.856 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.855+0000 7f397f1e7640 1 -- 172.21.15.121:0/1615887476 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f39780d03e0 msgr2=0x7f3978143920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:48.856 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.855+0000 7f397f1e7640 1 --2- 172.21.15.121:0/1615887476 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f39780d03e0 0x7f3978143920 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f396400e7e0 tx=0x7f396400eca0 comp rx=0 tx=0).stop 2024-04-24T22:58:48.856 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.855+0000 7f397f1e7640 1 -- 172.21.15.121:0/1615887476 shutdown_connections 2024-04-24T22:58:48.857 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.855+0000 7f397f1e7640 1 --2- 172.21.15.121:0/1615887476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f397815c6e0 0x7f39780aa4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:48.857 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.855+0000 7f397f1e7640 1 --2- 172.21.15.121:0/1615887476 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f39780d03e0 0x7f3978143920 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:48.857 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.855+0000 7f397f1e7640 1 --2- 172.21.15.121:0/1615887476 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f39780704b0 0x7f397815cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:48.857 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.855+0000 7f397f1e7640 1 -- 172.21.15.121:0/1615887476 >> 172.21.15.121:0/1615887476 conn(0x7f397806d320 msgr2=0x7f3978158320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:58:48.857 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.855+0000 7f397f1e7640 1 -- 172.21.15.121:0/1615887476 shutdown_connections 2024-04-24T22:58:48.857 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.855+0000 7f397f1e7640 1 -- 172.21.15.121:0/1615887476 wait complete. 2024-04-24T22:58:48.867 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 7 2024-04-24T22:58:48.893 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-24T22:58:48.892+0000 7f2c22579040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T22:58:48.919 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-04-24T22:58:48.918+0000 7f8308c0d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T22:58:48.964 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.963+0000 7fcd19f7e640 1 Processor -- start 2024-04-24T22:58:48.964 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.963+0000 7fcd19f7e640 1 -- start start 2024-04-24T22:58:48.967 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.963+0000 7fcd19f7e640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fcd1406e800 0x7fcd1406ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:48.967 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.963+0000 7fcd19f7e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd1406f150 con 0x7fcd140b6bb0 2024-04-24T22:58:48.967 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.963+0000 7fcd19f7e640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd14159190 con 0x7fcd14158db0 2024-04-24T22:58:48.967 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.963+0000 7fcd19f7e640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fcd1415fd00 con 0x7fcd1406e800 2024-04-24T22:58:48.967 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.964+0000 7fcd13fff640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fcd1406e800 0x7fcd1406ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:48.967 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.964+0000 7fcd13fff640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fcd1406e800 0x7fcd1406ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:45382/0 (socket says 172.21.15.121:45382) 2024-04-24T22:58:48.967 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.964+0000 7fcd13fff640 1 -- 172.21.15.121:0/2037282599 learned_addr learned my addr 172.21.15.121:0/2037282599 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-24T22:58:48.967 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.964+0000 7fcd137fe640 1 --1- 172.21.15.121:0/2037282599 >> v1:172.21.15.121:6789/0 conn(0x7fcd14158db0 0x7fcd140730c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:53530/0 (socket says 172.21.15.121:53530) 2024-04-24T22:58:48.967 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.964+0000 7fcd13fff640 1 -- 172.21.15.121:0/2037282599 >> v1:172.21.15.121:6789/0 conn(0x7fcd14158db0 legacy=0x7fcd140730c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:48.967 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.964+0000 7fcd13fff640 1 -- 172.21.15.121:0/2037282599 >> v1:172.21.15.32:6789/0 conn(0x7fcd140b6bb0 legacy=0x7fcd1415f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:48.967 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.964+0000 7fcd127fc640 1 -- 172.21.15.121:0/2037282599 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3861125951 0 0) 0x7fcd14159190 con 0x7fcd14158db0 2024-04-24T22:58:48.968 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.964+0000 7fcd13fff640 1 -- 172.21.15.121:0/2037282599 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd140b7230 con 0x7fcd1406e800 2024-04-24T22:58:48.968 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.965+0000 7fcd13fff640 1 --2- 172.21.15.121:0/2037282599 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd1406e800 0x7fcd1406ec10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcd1415fe80 tx=0x7fcd0c02a7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b82e490eecca2c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:58:48.968 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.965+0000 7fcd127fc640 1 -- 172.21.15.121:0/2037282599 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c036070 con 0x7fcd1406e800 2024-04-24T22:58:48.968 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.965+0000 7fcd127fc640 1 -- 172.21.15.121:0/2037282599 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd0c031600 con 0x7fcd1406e800 2024-04-24T22:58:48.968 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.965+0000 7fcd127fc640 1 -- 172.21.15.121:0/2037282599 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c0319d0 con 0x7fcd1406e800 2024-04-24T22:58:48.968 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.965+0000 7fcd19f7e640 1 -- 172.21.15.121:0/2037282599 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd1406e800 msgr2=0x7fcd1406ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:48.968 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.965+0000 7fcd19f7e640 1 --2- 172.21.15.121:0/2037282599 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd1406e800 0x7fcd1406ec10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcd1415fe80 tx=0x7fcd0c02a7a0 comp rx=0 tx=0).stop 2024-04-24T22:58:48.968 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.965+0000 7fcd19f7e640 1 -- 172.21.15.121:0/2037282599 shutdown_connections 2024-04-24T22:58:48.968 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.965+0000 7fcd19f7e640 1 --2- 172.21.15.121:0/2037282599 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd1406e800 0x7fcd1406ec10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:48.968 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.965+0000 7fcd19f7e640 1 -- 172.21.15.121:0/2037282599 >> 172.21.15.121:0/2037282599 conn(0x7fcd140d24c0 msgr2=0x7fcd140d28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:58:48.968 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.965+0000 7fcd19f7e640 1 -- 172.21.15.121:0/2037282599 shutdown_connections 2024-04-24T22:58:48.968 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.965+0000 7fcd19f7e640 1 -- 172.21.15.121:0/2037282599 wait complete. 2024-04-24T22:58:48.971 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.966+0000 7fcd19f7e640 1 Processor -- start 2024-04-24T22:58:48.971 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.967+0000 7fcd19f7e640 1 -- start start 2024-04-24T22:58:48.971 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.967+0000 7fcd19f7e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcd1406e800 0x7fcd1415e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:48.971 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.967+0000 7fcd19f7e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd140b6bb0 0x7fcd140ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:48.971 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.967+0000 7fcd19f7e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd14158db0 0x7fcd14154b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:48.971 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.967+0000 7fcd19f7e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1415f0d0 con 0x7fcd14158db0 2024-04-24T22:58:48.971 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.967+0000 7fcd19f7e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fcd140b77f0 con 0x7fcd140b6bb0 2024-04-24T22:58:48.972 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.967+0000 7fcd19f7e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fcd1415ed80 con 0x7fcd1406e800 2024-04-24T22:58:48.972 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.967+0000 7fcd137fe640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcd1406e800 0x7fcd1415e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:48.972 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.967+0000 7fcd12ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd140b6bb0 0x7fcd140ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:48.972 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.968+0000 7fcd137fe640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcd1406e800 0x7fcd1415e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.121:46624/0 (socket says 172.21.15.121:46624) 2024-04-24T22:58:48.972 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.968+0000 7fcd12ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd140b6bb0 0x7fcd140ae8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:45390/0 (socket says 172.21.15.121:45390) 2024-04-24T22:58:48.972 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.968+0000 7fcd137fe640 1 -- 172.21.15.121:0/2905360082 learned_addr learned my addr 172.21.15.121:0/2905360082 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-24T22:58:48.972 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.968+0000 7fcd13fff640 1 --2- 172.21.15.121:0/2905360082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd14158db0 0x7fcd14154b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:48.973 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.968+0000 7fcd137fe640 1 -- 172.21.15.121:0/2905360082 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd140b6bb0 msgr2=0x7fcd140ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:48.973 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.968+0000 7fcd137fe640 1 --2- 172.21.15.121:0/2905360082 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd140b6bb0 0x7fcd140ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:48.973 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.968+0000 7fcd137fe640 1 -- 172.21.15.121:0/2905360082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd14158db0 msgr2=0x7fcd14154b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:48.973 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.968+0000 7fcd137fe640 1 --2- 172.21.15.121:0/2905360082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd14158db0 0x7fcd14154b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:48.973 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.968+0000 7fcd137fe640 1 -- 172.21.15.121:0/2905360082 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd140aedf0 con 0x7fcd1406e800 2024-04-24T22:58:48.973 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.968+0000 7fcd12ffd640 1 --2- 172.21.15.121:0/2905360082 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd140b6bb0 0x7fcd140ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:58:48.973 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.968+0000 7fcd137fe640 1 --2- 172.21.15.121:0/2905360082 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcd1406e800 0x7fcd1415e2f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcd0400c4a0 tx=0x7fcd0400c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:58:48.973 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.968+0000 7fccfbfff640 1 -- 172.21.15.121:0/2905360082 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd04018070 con 0x7fcd1406e800 2024-04-24T22:58:48.973 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.968+0000 7fccfbfff640 1 -- 172.21.15.121:0/2905360082 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd040074e0 con 0x7fcd1406e800 2024-04-24T22:58:48.973 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.968+0000 7fcd19f7e640 1 -- 172.21.15.121:0/2905360082 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd141552e0 con 0x7fcd1406e800 2024-04-24T22:58:48.973 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.968+0000 7fccfbfff640 1 -- 172.21.15.121:0/2905360082 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd04004a20 con 0x7fcd1406e800 2024-04-24T22:58:48.973 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.968+0000 7fcd19f7e640 1 -- 172.21.15.121:0/2905360082 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd140af370 con 0x7fcd1406e800 2024-04-24T22:58:48.973 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.968+0000 7fccfbfff640 1 -- 172.21.15.121:0/2905360082 <== mon.2 v2:172.21.15.121:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcd0401b070 con 0x7fcd1406e800 2024-04-24T22:58:48.973 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.968+0000 7fccfbfff640 1 -- 172.21.15.121:0/2905360082 <== mon.2 v2:172.21.15.121:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7fcd04012a60 con 0x7fcd1406e800 2024-04-24T22:58:48.973 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.969+0000 7fcd19f7e640 1 -- 172.21.15.121:0/2905360082 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd14068430 con 0x7fcd1406e800 2024-04-24T22:58:48.974 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:48.971+0000 7fccfbfff640 1 -- 172.21.15.121:0/2905360082 <== mon.2 v2:172.21.15.121:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fcd0401d440 con 0x7fcd1406e800 2024-04-24T22:58:49.045 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:49.043+0000 7fcd19f7e640 1 -- 172.21.15.121:0/2905360082 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 7} v 0) -- 0x7fcd140b1d80 con 0x7fcd1406e800 2024-04-24T22:58:49.051 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:49.049+0000 7fccfbfff640 1 -- 172.21.15.121:0/2905360082 <== mon.2 v2:172.21.15.121:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7fcd0401d7b0 con 0x7fcd1406e800 2024-04-24T22:58:49.051 INFO:teuthology.orchestra.run.smithi121.stdout:7 2024-04-24T22:58:49.052 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:49.050+0000 7fcd19f7e640 1 -- 172.21.15.121:0/2905360082 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcd1406e800 msgr2=0x7fcd1415e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:49.052 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:49.050+0000 7fcd19f7e640 1 --2- 172.21.15.121:0/2905360082 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcd1406e800 0x7fcd1415e2f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcd0400c4a0 tx=0x7fcd0400c960 comp rx=0 tx=0).stop 2024-04-24T22:58:49.052 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:49.050+0000 7fcd19f7e640 1 -- 172.21.15.121:0/2905360082 shutdown_connections 2024-04-24T22:58:49.052 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:49.050+0000 7fcd19f7e640 1 --2- 172.21.15.121:0/2905360082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd14158db0 0x7fcd14154b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:49.052 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:49.050+0000 7fcd19f7e640 1 --2- 172.21.15.121:0/2905360082 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd140b6bb0 0x7fcd140ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:49.052 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:49.050+0000 7fcd19f7e640 1 --2- 172.21.15.121:0/2905360082 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcd1406e800 0x7fcd1415e2f0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:49.052 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:49.050+0000 7fcd19f7e640 1 -- 172.21.15.121:0/2905360082 >> 172.21.15.121:0/2905360082 conn(0x7fcd140d24c0 msgr2=0x7fcd14162200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:58:49.052 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:49.050+0000 7fcd19f7e640 1 -- 172.21.15.121:0/2905360082 shutdown_connections 2024-04-24T22:58:49.052 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:58:49.050+0000 7fcd19f7e640 1 -- 172.21.15.121:0/2905360082 wait complete. 2024-04-24T22:58:49.063 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-24T22:58:49.063 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-24T22:58:49.065 INFO:tasks.ceph.osd.0:Started 2024-04-24T22:58:49.066 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-24T22:58:49.066 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-24T22:58:49.069 INFO:tasks.ceph.osd.1:Started 2024-04-24T22:58:49.069 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-24T22:58:49.069 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-24T22:58:49.071 INFO:tasks.ceph.osd.2:Started 2024-04-24T22:58:49.071 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-24T22:58:49.071 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-24T22:58:49.073 INFO:tasks.ceph.osd.3:Started 2024-04-24T22:58:49.073 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-24T22:58:49.074 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-24T22:58:49.075 INFO:tasks.ceph.osd.4:Started 2024-04-24T22:58:49.076 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-24T22:58:49.076 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-24T22:58:49.078 INFO:tasks.ceph.osd.5:Started 2024-04-24T22:58:49.078 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-24T22:58:49.078 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-24T22:58:49.080 INFO:tasks.ceph.osd.6:Started 2024-04-24T22:58:49.080 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-24T22:58:49.080 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-24T22:58:49.082 INFO:tasks.ceph.osd.7:Started 2024-04-24T22:58:49.083 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:58:49.153 INFO:tasks.ceph.osd.4.smithi121.stderr:2024-04-24T22:58:49.151+0000 7f4d034af640 -1 Falling back to public interface 2024-04-24T22:58:49.155 INFO:tasks.ceph.osd.5.smithi121.stderr:2024-04-24T22:58:49.154+0000 7f7b4e61b640 -1 Falling back to public interface 2024-04-24T22:58:49.167 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-04-24T22:58:49.165+0000 7f08160dd640 -1 Falling back to public interface 2024-04-24T22:58:49.167 INFO:tasks.ceph.osd.5.smithi121.stderr:2024-04-24T22:58:49.166+0000 7f7b4e61b640 -1 osd.5 0 log_to_monitors true 2024-04-24T22:58:49.171 INFO:tasks.ceph.osd.4.smithi121.stderr:2024-04-24T22:58:49.169+0000 7f4d034af640 -1 osd.4 0 log_to_monitors true 2024-04-24T22:58:49.177 INFO:tasks.ceph.osd.7.smithi121.stderr:2024-04-24T22:58:49.176+0000 7f4eff316640 -1 Falling back to public interface 2024-04-24T22:58:49.180 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-04-24T22:58:49.179+0000 7f96879fc640 -1 Falling back to public interface 2024-04-24T22:58:49.180 INFO:tasks.ceph.osd.6.smithi121.stderr:2024-04-24T22:58:49.179+0000 7f694b21f640 -1 Falling back to public interface 2024-04-24T22:58:49.181 INFO:tasks.ceph.osd.6.smithi121.stderr:2024-04-24T22:58:49.179+0000 7f694a1ce640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6822/0: (98) Address already in use 2024-04-24T22:58:49.185 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-04-24T22:58:49.183+0000 7f9691e0f640 -1 Falling back to public interface 2024-04-24T22:58:49.186 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-04-24T22:58:49.183+0000 7f08160dd640 -1 osd.0 0 log_to_monitors true 2024-04-24T22:58:49.186 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-04-24T22:58:49.184+0000 7ff706c27640 -1 Falling back to public interface 2024-04-24T22:58:49.186 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-04-24T22:58:49.184+0000 7f9690818640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6818/0: (98) Address already in use 2024-04-24T22:58:49.189 INFO:tasks.ceph.osd.7.smithi121.stderr:2024-04-24T22:58:49.188+0000 7f4eff316640 -1 osd.7 0 log_to_monitors true 2024-04-24T22:58:49.192 INFO:tasks.ceph.osd.6.smithi121.stderr:2024-04-24T22:58:49.191+0000 7f694b21f640 -1 osd.6 0 log_to_monitors true 2024-04-24T22:58:49.196 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-04-24T22:58:49.195+0000 7f96879fc640 -1 osd.1 0 log_to_monitors true 2024-04-24T22:58:49.198 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-04-24T22:58:49.197+0000 7f9691e0f640 -1 osd.2 0 log_to_monitors true 2024-04-24T22:58:49.203 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-04-24T22:58:49.201+0000 7ff706c27640 -1 osd.3 0 log_to_monitors true 2024-04-24T22:58:49.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.206+0000 7f7a16cfc640 1 Processor -- start 2024-04-24T22:58:49.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.206+0000 7f7a16cfc640 1 -- start start 2024-04-24T22:58:49.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.207+0000 7f7a16cfc640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7a1015e8d0 0x7f7a1006c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:49.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.207+0000 7f7a16cfc640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7a100d1e70 0x7f7a1006b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:49.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.207+0000 7f7a16cfc640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a1006fd80 con 0x7f7a1006cd40 2024-04-24T22:58:49.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.207+0000 7f7a16cfc640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f7a10070990 con 0x7f7a1015e8d0 2024-04-24T22:58:49.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.207+0000 7f7a16cfc640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f7a10070060 con 0x7f7a100d1e70 2024-04-24T22:58:49.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.207+0000 7f7a0ffff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7a1006cd40 0x7f7a100d1760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46288/0 (socket says 172.21.15.32:46288) 2024-04-24T22:58:49.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.207+0000 7f7a0ffff640 1 -- 172.21.15.32:0/2501489986 learned_addr learned my addr 172.21.15.32:0/2501489986 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:58:49.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.207+0000 7f7a14a98640 1 --2- 172.21.15.32:0/2501489986 >> v2:172.21.15.121:3300/0 conn(0x7f7a1015e8d0 0x7f7a1006c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:49.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.207+0000 7f7a15299640 1 --2- 172.21.15.32:0/2501489986 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7a100d1e70 0x7f7a1006b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:49.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.207+0000 7f7a0f7fe640 1 -- 172.21.15.32:0/2501489986 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2883350867 0 0) 0x7f7a1006fd80 con 0x7f7a1006cd40 2024-04-24T22:58:49.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.207+0000 7f7a14a98640 1 -- 172.21.15.32:0/2501489986 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7a100d1e70 msgr2=0x7f7a1006b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:58:49.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.207+0000 7f7a14a98640 1 --2- 172.21.15.32:0/2501489986 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7a100d1e70 0x7f7a1006b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:49.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.207+0000 7f7a14a98640 1 -- 172.21.15.32:0/2501489986 >> v1:172.21.15.32:6789/0 conn(0x7f7a1006cd40 legacy=0x7f7a100d1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:49.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.207+0000 7f7a14a98640 1 -- 172.21.15.32:0/2501489986 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a10070340 con 0x7f7a1015e8d0 2024-04-24T22:58:49.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.208+0000 7f7a14a98640 1 --2- 172.21.15.32:0/2501489986 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a1015e8d0 0x7f7a1006c800 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7a040097a0 tx=0x7f7a0402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a6581c232be76dd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:58:49.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.208+0000 7f7a0f7fe640 1 -- 172.21.15.32:0/2501489986 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a04037070 con 0x7f7a1015e8d0 2024-04-24T22:58:49.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.208+0000 7f7a0f7fe640 1 -- 172.21.15.32:0/2501489986 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a04032440 con 0x7f7a1015e8d0 2024-04-24T22:58:49.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.208+0000 7f7a0f7fe640 1 -- 172.21.15.32:0/2501489986 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a04032810 con 0x7f7a1015e8d0 2024-04-24T22:58:49.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.209+0000 7f7a16cfc640 1 -- 172.21.15.32:0/2501489986 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a1015e8d0 msgr2=0x7f7a1006c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:49.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.209+0000 7f7a16cfc640 1 --2- 172.21.15.32:0/2501489986 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a1015e8d0 0x7f7a1006c800 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7a040097a0 tx=0x7f7a0402a180 comp rx=0 tx=0).stop 2024-04-24T22:58:49.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.209+0000 7f7a16cfc640 1 -- 172.21.15.32:0/2501489986 shutdown_connections 2024-04-24T22:58:49.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.209+0000 7f7a16cfc640 1 --2- 172.21.15.32:0/2501489986 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7a100d1e70 0x7f7a1006b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:49.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.209+0000 7f7a16cfc640 1 --2- 172.21.15.32:0/2501489986 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a1015e8d0 0x7f7a1006c800 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:49.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.209+0000 7f7a16cfc640 1 -- 172.21.15.32:0/2501489986 >> 172.21.15.32:0/2501489986 conn(0x7f7a100d2410 msgr2=0x7f7a100d2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:58:49.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.209+0000 7f7a16cfc640 1 -- 172.21.15.32:0/2501489986 shutdown_connections 2024-04-24T22:58:49.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.209+0000 7f7a16cfc640 1 -- 172.21.15.32:0/2501489986 wait complete. 2024-04-24T22:58:49.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.210+0000 7f7a16cfc640 1 Processor -- start 2024-04-24T22:58:49.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.210+0000 7f7a16cfc640 1 -- start start 2024-04-24T22:58:49.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.211+0000 7f7a16cfc640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7a1006cd40 0x7f7a100b2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:49.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.211+0000 7f7a16cfc640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a100d1e70 0x7f7a100cc530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:49.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.211+0000 7f7a16cfc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a1015fea0 0x7f7a100cca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:49.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.211+0000 7f7a16cfc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1006b650 con 0x7f7a1015fea0 2024-04-24T22:58:49.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.211+0000 7f7a16cfc640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7a100b7680 con 0x7f7a100d1e70 2024-04-24T22:58:49.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.211+0000 7f7a16cfc640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f7a1006b7d0 con 0x7f7a1006cd40 2024-04-24T22:58:49.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.211+0000 7f7a15299640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a1015fea0 0x7f7a100cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:49.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.211+0000 7f7a15299640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a1015fea0 0x7f7a100cca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35410/0 (socket says 172.21.15.32:35410) 2024-04-24T22:58:49.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.211+0000 7f7a14a98640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7a1006cd40 0x7f7a100b2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:49.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.211+0000 7f7a15299640 1 -- 172.21.15.32:0/1180991246 learned_addr learned my addr 172.21.15.32:0/1180991246 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:58:49.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.211+0000 7f7a0ffff640 1 --2- 172.21.15.32:0/1180991246 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a100d1e70 0x7f7a100cc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:49.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.211+0000 7f7a14a98640 1 -- 172.21.15.32:0/1180991246 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a100d1e70 msgr2=0x7f7a100cc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:49.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.211+0000 7f7a14a98640 1 --2- 172.21.15.32:0/1180991246 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a100d1e70 0x7f7a100cc530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:49.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.211+0000 7f7a14a98640 1 -- 172.21.15.32:0/1180991246 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a1015fea0 msgr2=0x7f7a100cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:49.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.211+0000 7f7a14a98640 1 --2- 172.21.15.32:0/1180991246 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a1015fea0 0x7f7a100cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:49.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.211+0000 7f7a14a98640 1 -- 172.21.15.32:0/1180991246 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a1015ecc0 con 0x7f7a1006cd40 2024-04-24T22:58:49.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.212+0000 7f7a14a98640 1 --2- 172.21.15.32:0/1180991246 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7a1006cd40 0x7f7a100b2360 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7a04031000 tx=0x7f7a040046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:58:49.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.212+0000 7f7a0d7fa640 1 -- 172.21.15.32:0/1180991246 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a04041020 con 0x7f7a1006cd40 2024-04-24T22:58:49.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.212+0000 7f7a0d7fa640 1 -- 172.21.15.32:0/1180991246 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a04030990 con 0x7f7a1006cd40 2024-04-24T22:58:49.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.212+0000 7f7a0d7fa640 1 -- 172.21.15.32:0/1180991246 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a04037040 con 0x7f7a1006cd40 2024-04-24T22:58:49.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.212+0000 7f7a16cfc640 1 -- 172.21.15.32:0/1180991246 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a1015efa0 con 0x7f7a1006cd40 2024-04-24T22:58:49.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.212+0000 7f7a16cfc640 1 -- 172.21.15.32:0/1180991246 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a1015f560 con 0x7f7a1006cd40 2024-04-24T22:58:49.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.213+0000 7f7a0d7fa640 1 -- 172.21.15.32:0/1180991246 <== mon.2 v2:172.21.15.121:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7a04044020 con 0x7f7a1006cd40 2024-04-24T22:58:49.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.213+0000 7f7a0d7fa640 1 -- 172.21.15.32:0/1180991246 <== mon.2 v2:172.21.15.121:3301/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f7a04046ad0 con 0x7f7a1006cd40 2024-04-24T22:58:49.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.213+0000 7f7a16cfc640 1 -- 172.21.15.32:0/1180991246 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a10068430 con 0x7f7a1006cd40 2024-04-24T22:58:49.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.215+0000 7f7a0d7fa640 1 -- 172.21.15.32:0/1180991246 <== mon.2 v2:172.21.15.121:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f7a0404b030 con 0x7f7a1006cd40 2024-04-24T22:58:49.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.277+0000 7f7a16cfc640 1 -- 172.21.15.32:0/1180991246 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7a100b6b90 con 0x7f7a1006cd40 2024-04-24T22:58:49.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.278+0000 7f7a0d7fa640 1 -- 172.21.15.32:0/1180991246 <== mon.2 v2:172.21.15.121:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f7a04030dd0 con 0x7f7a1006cd40 2024-04-24T22:58:49.280 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:58:49.280 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":10,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T22:58:49.045609+0000","last_up_change":"0.000000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:58:49.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.279+0000 7f7a16cfc640 1 -- 172.21.15.32:0/1180991246 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7a1006cd40 msgr2=0x7f7a100b2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:49.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.279+0000 7f7a16cfc640 1 --2- 172.21.15.32:0/1180991246 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7a1006cd40 0x7f7a100b2360 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7a04031000 tx=0x7f7a040046c0 comp rx=0 tx=0).stop 2024-04-24T22:58:49.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.279+0000 7f7a16cfc640 1 -- 172.21.15.32:0/1180991246 shutdown_connections 2024-04-24T22:58:49.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.279+0000 7f7a16cfc640 1 --2- 172.21.15.32:0/1180991246 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a1015fea0 0x7f7a100cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:49.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.279+0000 7f7a16cfc640 1 --2- 172.21.15.32:0/1180991246 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a100d1e70 0x7f7a100cc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:49.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.279+0000 7f7a16cfc640 1 --2- 172.21.15.32:0/1180991246 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7a1006cd40 0x7f7a100b2360 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:49.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.279+0000 7f7a16cfc640 1 -- 172.21.15.32:0/1180991246 >> 172.21.15.32:0/1180991246 conn(0x7f7a100d2410 msgr2=0x7f7a100b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:58:49.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.279+0000 7f7a16cfc640 1 -- 172.21.15.32:0/1180991246 shutdown_connections 2024-04-24T22:58:49.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:49.279+0000 7f7a16cfc640 1 -- 172.21.15.32:0/1180991246 wait complete. 2024-04-24T22:58:49.291 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-24T22:58:49.292 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-24T22:58:49.329 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-04-24T22:58:49.328+0000 7f8308c0d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T22:58:49.330 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-24T22:58:49.329+0000 7f2c22579040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T22:58:49.584 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-04-24T22:58:49.583+0000 7f8308c0d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T22:58:49.602 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-24T22:58:49.601+0000 7f2c22579040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T22:58:49.635 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-04-24T22:58:49.634+0000 7f8308c0d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T22:58:49.657 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-24T22:58:49.656+0000 7f2c22579040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T22:58:49.749 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-04-24T22:58:49.748+0000 7f8308c0d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T22:58:49.783 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-24T22:58:49.782+0000 7f2c22579040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T22:58:50.029 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-04-24T22:58:50.028+0000 7f8308c0d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T22:58:50.078 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-04-24T22:58:50.076+0000 7f8308c0d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T22:58:50.080 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-24T22:58:50.079+0000 7f2c22579040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T22:58:50.099 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-24T22:58:50.132 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-24T22:58:50.130+0000 7f2c22579040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T22:58:50.192 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.190+0000 7f3b65c31640 1 Processor -- start 2024-04-24T22:58:50.192 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.191+0000 7f3b65c31640 1 -- start start 2024-04-24T22:58:50.193 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.191+0000 7f3b65c31640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3b6015e510 0x7f3b600d3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:50.193 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.191+0000 7f3b65c31640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3b600d4cf0 0x7f3b600d2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:50.194 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.191+0000 7f3b65c31640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b600d2d00 con 0x7f3b6015e8f0 2024-04-24T22:58:50.194 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.191+0000 7f3b65c31640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f3b600b77d0 con 0x7f3b6015e510 2024-04-24T22:58:50.194 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.191+0000 7f3b65c31640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f3b600d2fe0 con 0x7f3b600d4cf0 2024-04-24T22:58:50.194 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.191+0000 7f3b5f7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3b6015e510 0x7f3b600d3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:50.194 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.192+0000 7f3b5f7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3b6015e510 0x7f3b600d3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51656/0 (socket says 172.21.15.32:51656) 2024-04-24T22:58:50.194 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.192+0000 7f3b5effd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3b6015e8f0 0x7f3b600d45e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54724/0 (socket says 172.21.15.32:54724) 2024-04-24T22:58:50.194 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.192+0000 7f3b5f7fe640 1 -- 172.21.15.32:0/1998745197 learned_addr learned my addr 172.21.15.32:0/1998745197 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:58:50.194 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.192+0000 7f3b5ffff640 1 --2- 172.21.15.32:0/1998745197 >> v2:172.21.15.121:3300/0 conn(0x7f3b600d4cf0 0x7f3b600d2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:50.194 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.192+0000 7f3b5f7fe640 1 -- 172.21.15.32:0/1998745197 >> v2:172.21.15.121:3300/0 conn(0x7f3b600d4cf0 msgr2=0x7f3b600d2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:58:50.194 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.192+0000 7f3b5f7fe640 1 --2- 172.21.15.32:0/1998745197 >> v2:172.21.15.121:3300/0 conn(0x7f3b600d4cf0 0x7f3b600d2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:50.194 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.192+0000 7f3b5f7fe640 1 -- 172.21.15.32:0/1998745197 >> v1:172.21.15.32:6789/0 conn(0x7f3b6015e8f0 legacy=0x7f3b600d45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:50.194 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.192+0000 7f3b5e7fc640 1 -- 172.21.15.32:0/1998745197 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3274544850 0 0) 0x7f3b600d2d00 con 0x7f3b6015e8f0 2024-04-24T22:58:50.195 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.192+0000 7f3b5f7fe640 1 -- 172.21.15.32:0/1998745197 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b600d32c0 con 0x7f3b6015e510 2024-04-24T22:58:50.196 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.193+0000 7f3b5f7fe640 1 --2- 172.21.15.32:0/1998745197 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b6015e510 0x7f3b600d3a40 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3b600c8150 tx=0x7f3b5002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c93fb57ec64d90fc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:58:50.196 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.193+0000 7f3b5e7fc640 1 -- 172.21.15.32:0/1998745197 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b50037070 con 0x7f3b6015e510 2024-04-24T22:58:50.196 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.193+0000 7f3b5e7fc640 1 -- 172.21.15.32:0/1998745197 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b50032440 con 0x7f3b6015e510 2024-04-24T22:58:50.196 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.193+0000 7f3b5e7fc640 1 -- 172.21.15.32:0/1998745197 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b50032810 con 0x7f3b6015e510 2024-04-24T22:58:50.196 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.194+0000 7f3b65c31640 1 -- 172.21.15.32:0/1998745197 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b6015e510 msgr2=0x7f3b600d3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:50.197 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.194+0000 7f3b65c31640 1 --2- 172.21.15.32:0/1998745197 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b6015e510 0x7f3b600d3a40 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3b600c8150 tx=0x7f3b5002a270 comp rx=0 tx=0).stop 2024-04-24T22:58:50.197 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.194+0000 7f3b65c31640 1 -- 172.21.15.32:0/1998745197 shutdown_connections 2024-04-24T22:58:50.197 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.194+0000 7f3b65c31640 1 --2- 172.21.15.32:0/1998745197 >> v2:172.21.15.121:3300/0 conn(0x7f3b600d4cf0 0x7f3b600d2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:50.197 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.194+0000 7f3b65c31640 1 --2- 172.21.15.32:0/1998745197 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b6015e510 0x7f3b600d3a40 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:50.197 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.194+0000 7f3b65c31640 1 -- 172.21.15.32:0/1998745197 >> 172.21.15.32:0/1998745197 conn(0x7f3b60071b60 msgr2=0x7f3b60071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:58:50.197 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.194+0000 7f3b65c31640 1 -- 172.21.15.32:0/1998745197 shutdown_connections 2024-04-24T22:58:50.197 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.194+0000 7f3b65c31640 1 -- 172.21.15.32:0/1998745197 wait complete. 2024-04-24T22:58:50.197 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.195+0000 7f3b65c31640 1 Processor -- start 2024-04-24T22:58:50.197 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.195+0000 7f3b65c31640 1 -- start start 2024-04-24T22:58:50.199 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.196+0000 7f3b65c31640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b600d4cf0 0x7f3b600cdf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:50.199 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.196+0000 7f3b65c31640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b6015e510 0x7f3b600cc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:50.199 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.196+0000 7f3b65c31640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b6015e8f0 0x7f3b600b4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:50.199 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.196+0000 7f3b65c31640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3b600b33a0 con 0x7f3b600d4cf0 2024-04-24T22:58:50.200 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.196+0000 7f3b65c31640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3b600b3220 con 0x7f3b6015e510 2024-04-24T22:58:50.200 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.196+0000 7f3b65c31640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f3b600b3520 con 0x7f3b6015e8f0 2024-04-24T22:58:50.200 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.196+0000 7f3b5f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b600d4cf0 0x7f3b600cdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:50.200 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.196+0000 7f3b5f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b600d4cf0 0x7f3b600cdf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51666/0 (socket says 172.21.15.32:51666) 2024-04-24T22:58:50.200 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.196+0000 7f3b5f7fe640 1 -- 172.21.15.32:0/4265945463 learned_addr learned my addr 172.21.15.32:0/4265945463 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:58:50.200 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.196+0000 7f3b5effd640 1 --2- 172.21.15.32:0/4265945463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b6015e510 0x7f3b600cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:50.200 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.196+0000 7f3b5ffff640 1 --2- 172.21.15.32:0/4265945463 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b6015e8f0 0x7f3b600b4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:50.200 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.196+0000 7f3b5f7fe640 1 -- 172.21.15.32:0/4265945463 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b6015e8f0 msgr2=0x7f3b600b4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:50.200 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.196+0000 7f3b5f7fe640 1 --2- 172.21.15.32:0/4265945463 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b6015e8f0 0x7f3b600b4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:50.200 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.196+0000 7f3b5f7fe640 1 -- 172.21.15.32:0/4265945463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b6015e510 msgr2=0x7f3b600cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:50.200 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.196+0000 7f3b5f7fe640 1 --2- 172.21.15.32:0/4265945463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b6015e510 0x7f3b600cc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:50.200 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.196+0000 7f3b5f7fe640 1 -- 172.21.15.32:0/4265945463 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b600ccbc0 con 0x7f3b600d4cf0 2024-04-24T22:58:50.200 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.196+0000 7f3b5f7fe640 1 --2- 172.21.15.32:0/4265945463 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b600d4cf0 0x7f3b600cdf10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3b50000c00 tx=0x7f3b500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:58:50.200 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.197+0000 7f3b47fff640 1 -- 172.21.15.32:0/4265945463 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b50041020 con 0x7f3b600d4cf0 2024-04-24T22:58:50.201 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.197+0000 7f3b47fff640 1 -- 172.21.15.32:0/4265945463 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b50030640 con 0x7f3b600d4cf0 2024-04-24T22:58:50.201 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.197+0000 7f3b65c31640 1 -- 172.21.15.32:0/4265945463 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b600cb8c0 con 0x7f3b600d4cf0 2024-04-24T22:58:50.201 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.197+0000 7f3b47fff640 1 -- 172.21.15.32:0/4265945463 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b50037040 con 0x7f3b600d4cf0 2024-04-24T22:58:50.201 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.197+0000 7f3b65c31640 1 -- 172.21.15.32:0/4265945463 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b600cbe80 con 0x7f3b600d4cf0 2024-04-24T22:58:50.201 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.197+0000 7f3b47fff640 1 -- 172.21.15.32:0/4265945463 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3b50044020 con 0x7f3b600d4cf0 2024-04-24T22:58:50.201 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.197+0000 7f3b47fff640 1 -- 172.21.15.32:0/4265945463 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(11..11 src has 1..11) ==== 1883+0+0 (secure 0 0 0) 0x7f3b5003d420 con 0x7f3b600d4cf0 2024-04-24T22:58:50.201 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.198+0000 7f3b65c31640 1 -- 172.21.15.32:0/4265945463 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b60068be0 con 0x7f3b600d4cf0 2024-04-24T22:58:50.201 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.200+0000 7f3b47fff640 1 -- 172.21.15.32:0/4265945463 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f3b5004b070 con 0x7f3b600d4cf0 2024-04-24T22:58:50.234 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-04-24T22:58:50.232+0000 7f8308c0d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T22:58:50.264 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.263+0000 7f3b65c31640 1 -- 172.21.15.32:0/4265945463 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3b600d1fb0 con 0x7f3b600d4cf0 2024-04-24T22:58:50.265 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.263+0000 7f3b47fff640 1 -- 172.21.15.32:0/4265945463 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v11) ==== 74+0+7058 (secure 0 0 0) 0x7f3b600ccbc0 con 0x7f3b600d4cf0 2024-04-24T22:58:50.265 INFO:teuthology.misc.health.smithi032.stdout: 2024-04-24T22:58:50.265 INFO:teuthology.misc.health.smithi032.stdout:{"epoch":11,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T22:58:50.046560+0000","last_up_change":"0.000000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:58:50.266 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.264+0000 7f3b65c31640 1 -- 172.21.15.32:0/4265945463 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b600d4cf0 msgr2=0x7f3b600cdf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:50.266 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.264+0000 7f3b65c31640 1 --2- 172.21.15.32:0/4265945463 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b600d4cf0 0x7f3b600cdf10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3b50000c00 tx=0x7f3b500046c0 comp rx=0 tx=0).stop 2024-04-24T22:58:50.266 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.264+0000 7f3b65c31640 1 -- 172.21.15.32:0/4265945463 shutdown_connections 2024-04-24T22:58:50.266 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.264+0000 7f3b65c31640 1 --2- 172.21.15.32:0/4265945463 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b6015e8f0 0x7f3b600b4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:50.266 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.264+0000 7f3b65c31640 1 --2- 172.21.15.32:0/4265945463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b6015e510 0x7f3b600cc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:50.266 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.264+0000 7f3b65c31640 1 --2- 172.21.15.32:0/4265945463 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b600d4cf0 0x7f3b600cdf10 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:50.266 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.264+0000 7f3b65c31640 1 -- 172.21.15.32:0/4265945463 >> 172.21.15.32:0/4265945463 conn(0x7f3b60071b60 msgr2=0x7f3b60066f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:58:50.266 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.264+0000 7f3b65c31640 1 -- 172.21.15.32:0/4265945463 shutdown_connections 2024-04-24T22:58:50.266 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:50.264+0000 7f3b65c31640 1 -- 172.21.15.32:0/4265945463 wait complete. 2024-04-24T22:58:50.276 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-24T22:58:50.304 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-24T22:58:50.302+0000 7f2c22579040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T22:58:50.368 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-04-24T22:58:50.367+0000 7f8308c0d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T22:58:50.417 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-04-24T22:58:50.415+0000 7f8308c0d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T22:58:50.450 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-24T22:58:50.449+0000 7f2c22579040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T22:58:50.486 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-04-24T22:58:50.484+0000 7f8308c0d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T22:58:50.503 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-24T22:58:50.502+0000 7f2c22579040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T22:58:50.530 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-04-24T22:58:50.529+0000 7f8308c0d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T22:58:50.577 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-24T22:58:50.576+0000 7f2c22579040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T22:58:50.625 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-24T22:58:50.624+0000 7f2c22579040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T22:58:50.999 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-04-24T22:58:50.997+0000 7f8308c0d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T22:58:51.051 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-04-24T22:58:51.049+0000 7ff702bbd640 -1 osd.3 0 waiting for initial osdmap 2024-04-24T22:58:51.051 INFO:tasks.ceph.osd.5.smithi121.stderr:2024-04-24T22:58:51.049+0000 7f7b4a5b1640 -1 osd.5 0 waiting for initial osdmap 2024-04-24T22:58:51.052 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-04-24T22:58:51.051+0000 7ff6fc1f1640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T22:58:51.052 INFO:tasks.ceph.osd.5.smithi121.stderr:2024-04-24T22:58:51.051+0000 7f7b43be5640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T22:58:51.057 INFO:tasks.ceph.osd.7.smithi121.stderr:2024-04-24T22:58:51.056+0000 7f4efb2ac640 -1 osd.7 0 waiting for initial osdmap 2024-04-24T22:58:51.058 INFO:tasks.ceph.osd.6.smithi121.stderr:2024-04-24T22:58:51.056+0000 7f69471b5640 -1 osd.6 0 waiting for initial osdmap 2024-04-24T22:58:51.058 INFO:tasks.ceph.osd.4.smithi121.stderr:2024-04-24T22:58:51.056+0000 7f4cff445640 -1 osd.4 0 waiting for initial osdmap 2024-04-24T22:58:51.059 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-04-24T22:58:51.056+0000 7f9683992640 -1 osd.1 0 waiting for initial osdmap 2024-04-24T22:58:51.059 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-04-24T22:58:51.056+0000 7f968d7ff640 -1 osd.2 0 waiting for initial osdmap 2024-04-24T22:58:51.059 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-04-24T22:58:51.056+0000 7f0812073640 -1 osd.0 0 waiting for initial osdmap 2024-04-24T22:58:51.059 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-04-24T22:58:51.058+0000 7f080b6a7640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T22:58:51.060 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-04-24T22:58:51.058+0000 7f967cfc6640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T22:58:51.060 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-04-24T22:58:51.058+0000 7f9686e33640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T22:58:51.061 INFO:tasks.ceph.osd.7.smithi121.stderr:2024-04-24T22:58:51.058+0000 7f4ef48e0640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T22:58:51.061 INFO:tasks.ceph.osd.6.smithi121.stderr:2024-04-24T22:58:51.058+0000 7f69407e9640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T22:58:51.061 INFO:tasks.ceph.osd.4.smithi121.stderr:2024-04-24T22:58:51.058+0000 7f4cf8a79640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T22:58:51.118 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-04-24T22:58:51.117+0000 7f8308c0d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T22:58:51.133 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-24T22:58:51.131+0000 7f2c22579040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T22:58:51.256 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-24T22:58:51.254+0000 7f2c22579040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T22:58:52.136 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-04-24T22:58:52.134+0000 7f82a94aa640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-24T22:58:52.136 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-04-24T22:58:52.135+0000 7f82a94aa640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-24T22:58:57.083 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-24T22:58:57.189 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.188+0000 7f6f5311d640 1 Processor -- start 2024-04-24T22:58:57.189 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.188+0000 7f6f5311d640 1 -- start start 2024-04-24T22:58:57.190 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.189+0000 7f6f5311d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6f4c0ae870 0x7f6f4c15f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:57.190 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.189+0000 7f6f5311d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f4c06def0 0x7f6f4c06e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:57.190 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.189+0000 7f6f5311d640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f4c0aef10 con 0x7f6f4c0b4920 2024-04-24T22:58:57.191 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.189+0000 7f6f5311d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f6f4c1600d0 con 0x7f6f4c0ae870 2024-04-24T22:58:57.191 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.189+0000 7f6f5311d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f6f4c15fd00 con 0x7f6f4c06def0 2024-04-24T22:58:57.191 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.189+0000 7f6f4bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6f4c0ae870 0x7f6f4c15f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:57.191 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.189+0000 7f6f4bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6f4c0ae870 0x7f6f4c15f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51668/0 (socket says 172.21.15.32:51668) 2024-04-24T22:58:57.191 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.189+0000 7f6f4bfff640 1 -- 172.21.15.32:0/3261155146 learned_addr learned my addr 172.21.15.32:0/3261155146 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:58:57.191 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.189+0000 7f6f516ba640 1 --2- 172.21.15.32:0/3261155146 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f4c06def0 0x7f6f4c06e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:57.191 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.189+0000 7f6f4bfff640 1 -- 172.21.15.32:0/3261155146 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f4c06def0 msgr2=0x7f6f4c06e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:58:57.191 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.189+0000 7f6f4bfff640 1 --2- 172.21.15.32:0/3261155146 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f4c06def0 0x7f6f4c06e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:57.191 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.189+0000 7f6f4bfff640 1 -- 172.21.15.32:0/3261155146 >> v1:172.21.15.121:6789/0 conn(0x7f6f4c0b4920 legacy=0x7f6f4c0b4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:57.191 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.189+0000 7f6f4bfff640 1 -- 172.21.15.32:0/3261155146 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f4c0af1f0 con 0x7f6f4c0ae870 2024-04-24T22:58:57.192 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.190+0000 7f6f4bfff640 1 --2- 172.21.15.32:0/3261155146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f4c0ae870 0x7f6f4c15f7c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6f34009870 tx=0x7f6f3402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ecee430c695779b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:58:57.192 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.190+0000 7f6f4b7fe640 1 -- 172.21.15.32:0/3261155146 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f34037070 con 0x7f6f4c0ae870 2024-04-24T22:58:57.193 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.190+0000 7f6f4b7fe640 1 -- 172.21.15.32:0/3261155146 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f34032440 con 0x7f6f4c0ae870 2024-04-24T22:58:57.193 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.190+0000 7f6f4b7fe640 1 -- 172.21.15.32:0/3261155146 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f34032810 con 0x7f6f4c0ae870 2024-04-24T22:58:57.193 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.190+0000 7f6f5311d640 1 -- 172.21.15.32:0/3261155146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f4c0ae870 msgr2=0x7f6f4c15f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:57.193 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.190+0000 7f6f5311d640 1 --2- 172.21.15.32:0/3261155146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f4c0ae870 0x7f6f4c15f7c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6f34009870 tx=0x7f6f3402a270 comp rx=0 tx=0).stop 2024-04-24T22:58:57.193 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.191+0000 7f6f5311d640 1 -- 172.21.15.32:0/3261155146 shutdown_connections 2024-04-24T22:58:57.193 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.191+0000 7f6f5311d640 1 --2- 172.21.15.32:0/3261155146 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f4c06def0 0x7f6f4c06e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:57.193 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.191+0000 7f6f5311d640 1 --2- 172.21.15.32:0/3261155146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f4c0ae870 0x7f6f4c15f7c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:57.193 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.191+0000 7f6f5311d640 1 -- 172.21.15.32:0/3261155146 >> 172.21.15.32:0/3261155146 conn(0x7f6f4c0b2510 msgr2=0x7f6f4c0b2920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:58:57.193 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.191+0000 7f6f5311d640 1 -- 172.21.15.32:0/3261155146 shutdown_connections 2024-04-24T22:58:57.193 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.191+0000 7f6f5311d640 1 -- 172.21.15.32:0/3261155146 wait complete. 2024-04-24T22:58:57.195 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.192+0000 7f6f5311d640 1 Processor -- start 2024-04-24T22:58:57.196 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.192+0000 7f6f5311d640 1 -- start start 2024-04-24T22:58:57.196 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.193+0000 7f6f5311d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f4c06def0 0x7f6f4c1510c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:57.196 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.193+0000 7f6f5311d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f4c0ae870 0x7f6f4c1553e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:57.196 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.193+0000 7f6f5311d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f4c0b4920 0x7f6f4c155920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:57.196 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.193+0000 7f6f5311d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c0afa90 con 0x7f6f4c0ae870 2024-04-24T22:58:57.196 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.193+0000 7f6f5311d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c0af7b0 con 0x7f6f4c0b4920 2024-04-24T22:58:57.196 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.193+0000 7f6f5311d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f6f4c15c590 con 0x7f6f4c06def0 2024-04-24T22:58:57.196 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.193+0000 7f6f4bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f4c0ae870 0x7f6f4c1553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:57.196 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.193+0000 7f6f4bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f4c0ae870 0x7f6f4c1553e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51672/0 (socket says 172.21.15.32:51672) 2024-04-24T22:58:57.196 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.193+0000 7f6f4bfff640 1 -- 172.21.15.32:0/1094490985 learned_addr learned my addr 172.21.15.32:0/1094490985 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:58:57.196 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.193+0000 7f6f50eb9640 1 --2- 172.21.15.32:0/1094490985 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f4c06def0 0x7f6f4c1510c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:57.196 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.193+0000 7f6f516ba640 1 --2- 172.21.15.32:0/1094490985 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f4c0b4920 0x7f6f4c155920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:57.196 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.193+0000 7f6f4bfff640 1 -- 172.21.15.32:0/1094490985 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f4c06def0 msgr2=0x7f6f4c1510c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:57.197 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.193+0000 7f6f4bfff640 1 --2- 172.21.15.32:0/1094490985 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f4c06def0 0x7f6f4c1510c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:57.197 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.193+0000 7f6f4bfff640 1 -- 172.21.15.32:0/1094490985 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f4c0b4920 msgr2=0x7f6f4c155920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:57.197 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.193+0000 7f6f4bfff640 1 --2- 172.21.15.32:0/1094490985 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f4c0b4920 0x7f6f4c155920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:57.197 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.193+0000 7f6f4bfff640 1 -- 172.21.15.32:0/1094490985 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f4c0cde00 con 0x7f6f4c0ae870 2024-04-24T22:58:57.197 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.193+0000 7f6f50eb9640 1 --2- 172.21.15.32:0/1094490985 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f4c06def0 0x7f6f4c1510c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:58:57.197 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.193+0000 7f6f4bfff640 1 --2- 172.21.15.32:0/1094490985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f4c0ae870 0x7f6f4c1553e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6f34000c00 tx=0x7f6f340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:58:57.197 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.194+0000 7f6f497fa640 1 -- 172.21.15.32:0/1094490985 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f34037040 con 0x7f6f4c0ae870 2024-04-24T22:58:57.197 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.194+0000 7f6f497fa640 1 -- 172.21.15.32:0/1094490985 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f34042070 con 0x7f6f4c0ae870 2024-04-24T22:58:57.197 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.194+0000 7f6f5311d640 1 -- 172.21.15.32:0/1094490985 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f4c0ce0e0 con 0x7f6f4c0ae870 2024-04-24T22:58:57.197 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.194+0000 7f6f497fa640 1 -- 172.21.15.32:0/1094490985 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f34030880 con 0x7f6f4c0ae870 2024-04-24T22:58:57.197 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.194+0000 7f6f5311d640 1 -- 172.21.15.32:0/1094490985 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f4c0ce6a0 con 0x7f6f4c0ae870 2024-04-24T22:58:57.198 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.195+0000 7f6f5311d640 1 -- 172.21.15.32:0/1094490985 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f4c0cea80 con 0x7f6f4c0ae870 2024-04-24T22:58:57.198 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.195+0000 7f6f497fa640 1 -- 172.21.15.32:0/1094490985 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6f34030a20 con 0x7f6f4c0ae870 2024-04-24T22:58:57.198 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.196+0000 7f6f497fa640 1 --2- 172.21.15.32:0/1094490985 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6f2c06be60 0x7f6f2c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:57.198 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.196+0000 7f6f497fa640 1 -- 172.21.15.32:0/1094490985 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f6f340ab320 con 0x7f6f4c0ae870 2024-04-24T22:58:57.198 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.196+0000 7f6f50eb9640 1 --2- 172.21.15.32:0/1094490985 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6f2c06be60 0x7f6f2c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:57.198 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.196+0000 7f6f50eb9640 1 --2- 172.21.15.32:0/1094490985 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6f2c06be60 0x7f6f2c06e320 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6f44001cd0 tx=0x7f6f44009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:58:57.201 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.200+0000 7f6f497fa640 1 -- 172.21.15.32:0/1094490985 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6f4c0cea80 con 0x7f6f4c0ae870 2024-04-24T22:58:57.390 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.389+0000 7f6f5311d640 1 -- 172.21.15.32:0/1094490985 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6f4c15fef0 con 0x7f6f4c0ae870 2024-04-24T22:58:57.391 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.390+0000 7f6f497fa640 1 -- 172.21.15.32:0/1094490985 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12896 (secure 0 0 0) 0x7f6f4c15fef0 con 0x7f6f4c0ae870 2024-04-24T22:58:57.391 INFO:teuthology.misc.health.smithi032.stdout: 2024-04-24T22:58:57.391 INFO:teuthology.misc.health.smithi032.stdout:{"epoch":17,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T22:58:56.136617+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:58:57.394 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.392+0000 7f6f5311d640 1 -- 172.21.15.32:0/1094490985 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6f2c06be60 msgr2=0x7f6f2c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:57.394 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.392+0000 7f6f5311d640 1 --2- 172.21.15.32:0/1094490985 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6f2c06be60 0x7f6f2c06e320 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6f44001cd0 tx=0x7f6f44009040 comp rx=0 tx=0).stop 2024-04-24T22:58:57.394 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.392+0000 7f6f5311d640 1 -- 172.21.15.32:0/1094490985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f4c0ae870 msgr2=0x7f6f4c1553e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:57.394 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.392+0000 7f6f5311d640 1 --2- 172.21.15.32:0/1094490985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f4c0ae870 0x7f6f4c1553e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6f34000c00 tx=0x7f6f340046c0 comp rx=0 tx=0).stop 2024-04-24T22:58:57.394 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.392+0000 7f6f5311d640 1 -- 172.21.15.32:0/1094490985 shutdown_connections 2024-04-24T22:58:57.394 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.392+0000 7f6f5311d640 1 --2- 172.21.15.32:0/1094490985 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6f2c06be60 0x7f6f2c06e320 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:57.395 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.392+0000 7f6f5311d640 1 --2- 172.21.15.32:0/1094490985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f4c0ae870 0x7f6f4c1553e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:57.395 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.392+0000 7f6f5311d640 1 --2- 172.21.15.32:0/1094490985 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f4c0b4920 0x7f6f4c155920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:57.395 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.392+0000 7f6f5311d640 1 --2- 172.21.15.32:0/1094490985 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f4c06def0 0x7f6f4c1510c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:57.395 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.392+0000 7f6f5311d640 1 -- 172.21.15.32:0/1094490985 >> 172.21.15.32:0/1094490985 conn(0x7f6f4c0b2510 msgr2=0x7f6f4c0b0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:58:57.395 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.392+0000 7f6f5311d640 1 -- 172.21.15.32:0/1094490985 shutdown_connections 2024-04-24T22:58:57.395 INFO:teuthology.misc.health.smithi032.stderr:2024-04-24T22:58:57.392+0000 7f6f5311d640 1 -- 172.21.15.32:0/1094490985 wait complete. 2024-04-24T22:58:57.403 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-24T22:58:57.403 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-24T22:58:57.404 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-24T22:58:57.404 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-24T22:58:57.406 INFO:tasks.ceph.mds.a:Started 2024-04-24T22:58:57.406 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-24T22:58:57.406 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-24T22:58:57.408 INFO:tasks.ceph.mds.c:Started 2024-04-24T22:58:57.408 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-24T22:58:57.408 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-24T22:58:57.410 INFO:tasks.ceph.mds.b:Started 2024-04-24T22:58:57.411 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-24T22:58:57.411 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-24T22:58:57.413 INFO:tasks.ceph.mds.d:Started 2024-04-24T22:58:57.413 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-24T22:58:57.414 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:58:57.480 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2024-04-24T22:58:57.481 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2024-04-24T22:58:57.489 INFO:tasks.ceph.mds.d.smithi121.stdout:starting mds.d at 2024-04-24T22:58:57.489 INFO:tasks.ceph.mds.b.smithi121.stdout:starting mds.b at 2024-04-24T22:58:57.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.516+0000 7f2d75c12640 1 Processor -- start 2024-04-24T22:58:57.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.517+0000 7f2d75c12640 1 -- start start 2024-04-24T22:58:57.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.517+0000 7f2d75c12640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2d7015cc40 0x7f2d7015d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:57.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.517+0000 7f2d75c12640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2d700d05b0 0x7f2d7015d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:57.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.517+0000 7f2d75c12640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d700b4ee0 con 0x7f2d700cec30 2024-04-24T22:58:57.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.517+0000 7f2d75c12640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f2d7006cca0 con 0x7f2d7015cc40 2024-04-24T22:58:57.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.517+0000 7f2d75c12640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f2d700ce5f0 con 0x7f2d700d05b0 2024-04-24T22:58:57.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.518+0000 7f2d6ffff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2d700cec30 0x7f2d700cf0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54732/0 (socket says 172.21.15.32:54732) 2024-04-24T22:58:57.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.518+0000 7f2d6ffff640 1 -- 172.21.15.32:0/3413284046 learned_addr learned my addr 172.21.15.32:0/3413284046 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:58:57.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.518+0000 7f2d6effd640 1 --2- 172.21.15.32:0/3413284046 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2d700d05b0 0x7f2d7015d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:57.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.518+0000 7f2d6f7fe640 1 --2- 172.21.15.32:0/3413284046 >> v2:172.21.15.121:3300/0 conn(0x7f2d7015cc40 0x7f2d7015d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:57.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.518+0000 7f2d6e7fc640 1 -- 172.21.15.32:0/3413284046 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3448899979 0 0) 0x7f2d700b4ee0 con 0x7f2d700cec30 2024-04-24T22:58:57.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.518+0000 7f2d6e7fc640 1 -- 172.21.15.32:0/3413284046 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d58003520 con 0x7f2d700cec30 2024-04-24T22:58:57.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.518+0000 7f2d6effd640 1 -- 172.21.15.32:0/3413284046 >> v2:172.21.15.121:3300/0 conn(0x7f2d7015cc40 msgr2=0x7f2d7015d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:58:57.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.518+0000 7f2d6effd640 1 --2- 172.21.15.32:0/3413284046 >> v2:172.21.15.121:3300/0 conn(0x7f2d7015cc40 0x7f2d7015d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:57.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.518+0000 7f2d6effd640 1 -- 172.21.15.32:0/3413284046 >> v1:172.21.15.32:6789/0 conn(0x7f2d700cec30 legacy=0x7f2d700cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:57.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.518+0000 7f2d6effd640 1 -- 172.21.15.32:0/3413284046 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d700b51c0 con 0x7f2d700d05b0 2024-04-24T22:58:57.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.518+0000 7f2d6f7fe640 1 --2- 172.21.15.32:0/3413284046 >> v2:172.21.15.121:3300/0 conn(0x7f2d7015cc40 0x7f2d7015d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:58:57.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.519+0000 7f2d6effd640 1 --2- 172.21.15.32:0/3413284046 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2d700d05b0 0x7f2d7015d560 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2d640097a0 tx=0x7f2d6402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e010e627eaac75e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:58:57.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.519+0000 7f2d6e7fc640 1 -- 172.21.15.32:0/3413284046 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d64037070 con 0x7f2d700d05b0 2024-04-24T22:58:57.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.519+0000 7f2d6e7fc640 1 -- 172.21.15.32:0/3413284046 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d64032440 con 0x7f2d700d05b0 2024-04-24T22:58:57.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.519+0000 7f2d6e7fc640 1 -- 172.21.15.32:0/3413284046 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d64032810 con 0x7f2d700d05b0 2024-04-24T22:58:57.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.519+0000 7f2d75c12640 1 -- 172.21.15.32:0/3413284046 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2d700d05b0 msgr2=0x7f2d7015d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:57.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.519+0000 7f2d75c12640 1 --2- 172.21.15.32:0/3413284046 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2d700d05b0 0x7f2d7015d560 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2d640097a0 tx=0x7f2d6402a180 comp rx=0 tx=0).stop 2024-04-24T22:58:57.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.519+0000 7f2d75c12640 1 -- 172.21.15.32:0/3413284046 shutdown_connections 2024-04-24T22:58:57.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.519+0000 7f2d75c12640 1 --2- 172.21.15.32:0/3413284046 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2d700d05b0 0x7f2d7015d560 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:57.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.519+0000 7f2d75c12640 1 --2- 172.21.15.32:0/3413284046 >> v2:172.21.15.121:3300/0 conn(0x7f2d7015cc40 0x7f2d7015d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:57.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.519+0000 7f2d75c12640 1 -- 172.21.15.32:0/3413284046 >> 172.21.15.32:0/3413284046 conn(0x7f2d700d0ae0 msgr2=0x7f2d700d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:58:57.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.520+0000 7f2d75c12640 1 -- 172.21.15.32:0/3413284046 shutdown_connections 2024-04-24T22:58:57.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.520+0000 7f2d75c12640 1 -- 172.21.15.32:0/3413284046 wait complete. 2024-04-24T22:58:57.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.521+0000 7f2d75c12640 1 Processor -- start 2024-04-24T22:58:57.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.521+0000 7f2d75c12640 1 -- start start 2024-04-24T22:58:57.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.522+0000 7f2d75c12640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2d700cec30 0x7f2d701f6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:57.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.522+0000 7f2d75c12640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d700d05b0 0x7f2d701462b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:57.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.522+0000 7f2d75c12640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d7015cc40 0x7f2d7015b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:57.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.522+0000 7f2d75c12640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2d700b5780 con 0x7f2d700d05b0 2024-04-24T22:58:57.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.522+0000 7f2d75c12640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7015daa0 con 0x7f2d7015cc40 2024-04-24T22:58:57.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.522+0000 7f2d75c12640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f2d700b5a60 con 0x7f2d700cec30 2024-04-24T22:58:57.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.522+0000 7f2d6f7fe640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2d700cec30 0x7f2d701f6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:57.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.522+0000 7f2d6f7fe640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2d700cec30 0x7f2d701f6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.32:43466/0 (socket says 172.21.15.32:43466) 2024-04-24T22:58:57.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.522+0000 7f2d6f7fe640 1 -- 172.21.15.32:0/2167537004 learned_addr learned my addr 172.21.15.32:0/2167537004 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:58:57.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.522+0000 7f2d6effd640 1 --2- 172.21.15.32:0/2167537004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d700d05b0 0x7f2d701462b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:57.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.522+0000 7f2d6ffff640 1 --2- 172.21.15.32:0/2167537004 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d7015cc40 0x7f2d7015b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:57.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.522+0000 7f2d6f7fe640 1 -- 172.21.15.32:0/2167537004 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d7015cc40 msgr2=0x7f2d7015b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:57.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.522+0000 7f2d6f7fe640 1 --2- 172.21.15.32:0/2167537004 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d7015cc40 0x7f2d7015b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:57.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.522+0000 7f2d6f7fe640 1 -- 172.21.15.32:0/2167537004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d700d05b0 msgr2=0x7f2d701462b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:57.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.522+0000 7f2d6f7fe640 1 --2- 172.21.15.32:0/2167537004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d700d05b0 0x7f2d701462b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:57.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.522+0000 7f2d6f7fe640 1 -- 172.21.15.32:0/2167537004 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d7015a810 con 0x7f2d700cec30 2024-04-24T22:58:57.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.522+0000 7f2d6effd640 1 --2- 172.21.15.32:0/2167537004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d700d05b0 0x7f2d701462b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:58:57.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.522+0000 7f2d6f7fe640 1 --2- 172.21.15.32:0/2167537004 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2d700cec30 0x7f2d701f6380 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2d6000ebe0 tx=0x7f2d6000c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:58:57.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.522+0000 7f2d57fff640 1 -- 172.21.15.32:0/2167537004 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d60017070 con 0x7f2d700cec30 2024-04-24T22:58:57.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.522+0000 7f2d57fff640 1 -- 172.21.15.32:0/2167537004 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d6000cc30 con 0x7f2d700cec30 2024-04-24T22:58:57.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.522+0000 7f2d75c12640 1 -- 172.21.15.32:0/2167537004 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d7015aaf0 con 0x7f2d700cec30 2024-04-24T22:58:57.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.522+0000 7f2d75c12640 1 -- 172.21.15.32:0/2167537004 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d701f68c0 con 0x7f2d700cec30 2024-04-24T22:58:57.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.523+0000 7f2d57fff640 1 -- 172.21.15.32:0/2167537004 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d60013470 con 0x7f2d700cec30 2024-04-24T22:58:57.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.523+0000 7f2d75c12640 1 -- 172.21.15.32:0/2167537004 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d70068430 con 0x7f2d700cec30 2024-04-24T22:58:57.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.524+0000 7f2d57fff640 1 -- 172.21.15.32:0/2167537004 <== mon.2 v2:172.21.15.121:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2d6001a070 con 0x7f2d700cec30 2024-04-24T22:58:57.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.524+0000 7f2d57fff640 1 --2- 172.21.15.32:0/2167537004 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2d5c06be30 0x7f2d5c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:57.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.524+0000 7f2d57fff640 1 -- 172.21.15.32:0/2167537004 <== mon.2 v2:172.21.15.121:3301/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f2d6008ae00 con 0x7f2d700cec30 2024-04-24T22:58:57.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.524+0000 7f2d6effd640 1 --2- 172.21.15.32:0/2167537004 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2d5c06be30 0x7f2d5c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:57.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.525+0000 7f2d6effd640 1 --2- 172.21.15.32:0/2167537004 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2d5c06be30 0x7f2d5c06e2f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2d64034000 tx=0x7f2d6402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:58:57.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.529+0000 7f2d57fff640 1 -- 172.21.15.32:0/2167537004 <== mon.2 v2:172.21.15.121:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2d60059410 con 0x7f2d700cec30 2024-04-24T22:58:57.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.711+0000 7f2d75c12640 1 -- 172.21.15.32:0/2167537004 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2d70152620 con 0x7f2d700cec30 2024-04-24T22:58:57.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.712+0000 7f2d57fff640 1 -- 172.21.15.32:0/2167537004 <== mon.2 v2:172.21.15.121:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12896 (secure 0 0 0) 0x7f2d6005d710 con 0x7f2d700cec30 2024-04-24T22:58:57.714 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:58:57.714 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":17,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T22:58:56.136617+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:58:57.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.714+0000 7f2d75c12640 1 -- 172.21.15.32:0/2167537004 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2d5c06be30 msgr2=0x7f2d5c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:57.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.714+0000 7f2d75c12640 1 --2- 172.21.15.32:0/2167537004 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2d5c06be30 0x7f2d5c06e2f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2d64034000 tx=0x7f2d6402e000 comp rx=0 tx=0).stop 2024-04-24T22:58:57.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.714+0000 7f2d75c12640 1 -- 172.21.15.32:0/2167537004 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2d700cec30 msgr2=0x7f2d701f6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:57.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.714+0000 7f2d75c12640 1 --2- 172.21.15.32:0/2167537004 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2d700cec30 0x7f2d701f6380 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2d6000ebe0 tx=0x7f2d6000c520 comp rx=0 tx=0).stop 2024-04-24T22:58:57.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.714+0000 7f2d75c12640 1 -- 172.21.15.32:0/2167537004 shutdown_connections 2024-04-24T22:58:57.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.714+0000 7f2d75c12640 1 --2- 172.21.15.32:0/2167537004 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2d5c06be30 0x7f2d5c06e2f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:57.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.714+0000 7f2d75c12640 1 --2- 172.21.15.32:0/2167537004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d700d05b0 0x7f2d701462b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:57.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.714+0000 7f2d75c12640 1 --2- 172.21.15.32:0/2167537004 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d7015cc40 0x7f2d7015b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:57.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.714+0000 7f2d75c12640 1 --2- 172.21.15.32:0/2167537004 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2d700cec30 0x7f2d701f6380 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:57.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.714+0000 7f2d75c12640 1 -- 172.21.15.32:0/2167537004 >> 172.21.15.32:0/2167537004 conn(0x7f2d700d0ae0 msgr2=0x7f2d7015c510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:58:57.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.714+0000 7f2d75c12640 1 -- 172.21.15.32:0/2167537004 shutdown_connections 2024-04-24T22:58:57.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.714+0000 7f2d75c12640 1 -- 172.21.15.32:0/2167537004 wait complete. 2024-04-24T22:58:57.726 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T22:58:57.727 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T22:58:57.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.828+0000 7f84dd0f5640 1 Processor -- start 2024-04-24T22:58:57.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.828+0000 7f84dd0f5640 1 -- start start 2024-04-24T22:58:57.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.829+0000 7f84dd0f5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84d815e9b0 0x7f84d80d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:57.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.829+0000 7f84dd0f5640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f84d806b820 0x7f84d80b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:57.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.829+0000 7f84dd0f5640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84d80d3170 con 0x7f84d80d4210 2024-04-24T22:58:57.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.829+0000 7f84dd0f5640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f84d80d3450 con 0x7f84d806b820 2024-04-24T22:58:57.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.829+0000 7f84dd0f5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f84d80d3730 con 0x7f84d815e9b0 2024-04-24T22:58:57.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.829+0000 7f84d659c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f84d80d4210 0x7f84d806b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54744/0 (socket says 172.21.15.32:54744) 2024-04-24T22:58:57.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.829+0000 7f84d659c640 1 -- 172.21.15.32:0/454902176 learned_addr learned my addr 172.21.15.32:0/454902176 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:58:57.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.829+0000 7f84d759e640 1 --2- 172.21.15.32:0/454902176 >> v2:172.21.15.121:3300/0 conn(0x7f84d806b820 0x7f84d80b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:57.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.829+0000 7f84d6d9d640 1 --2- 172.21.15.32:0/454902176 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84d815e9b0 0x7f84d80d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:57.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.829+0000 7f84d5d9b640 1 -- 172.21.15.32:0/454902176 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3218843529 0 0) 0x7f84d80d3170 con 0x7f84d80d4210 2024-04-24T22:58:57.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.830+0000 7f84d5d9b640 1 -- 172.21.15.32:0/454902176 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f84c4003520 con 0x7f84d80d4210 2024-04-24T22:58:57.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.830+0000 7f84d759e640 1 -- 172.21.15.32:0/454902176 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84d815e9b0 msgr2=0x7f84d80d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:58:57.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.830+0000 7f84d759e640 1 --2- 172.21.15.32:0/454902176 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84d815e9b0 0x7f84d80d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:57.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.830+0000 7f84d759e640 1 -- 172.21.15.32:0/454902176 >> v1:172.21.15.32:6789/0 conn(0x7f84d80d4210 legacy=0x7f84d806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:57.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.830+0000 7f84d759e640 1 -- 172.21.15.32:0/454902176 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84d80d3a10 con 0x7f84d806b820 2024-04-24T22:58:57.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.830+0000 7f84d6d9d640 1 --2- 172.21.15.32:0/454902176 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84d815e9b0 0x7f84d80d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:58:57.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.830+0000 7f84d759e640 1 --2- 172.21.15.32:0/454902176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84d806b820 0x7f84d80b78f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f84c8009d90 tx=0x7f84c802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fa0f970c42db23b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:58:57.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.831+0000 7f84d5d9b640 1 -- 172.21.15.32:0/454902176 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c8007740 con 0x7f84d806b820 2024-04-24T22:58:57.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.831+0000 7f84d5d9b640 1 -- 172.21.15.32:0/454902176 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84c80078e0 con 0x7f84d806b820 2024-04-24T22:58:57.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.831+0000 7f84d5d9b640 1 -- 172.21.15.32:0/454902176 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c8007cb0 con 0x7f84d806b820 2024-04-24T22:58:57.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.831+0000 7f84dd0f5640 1 -- 172.21.15.32:0/454902176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84d806b820 msgr2=0x7f84d80b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:57.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.831+0000 7f84dd0f5640 1 --2- 172.21.15.32:0/454902176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84d806b820 0x7f84d80b78f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f84c8009d90 tx=0x7f84c802a660 comp rx=0 tx=0).stop 2024-04-24T22:58:57.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.831+0000 7f84dd0f5640 1 -- 172.21.15.32:0/454902176 shutdown_connections 2024-04-24T22:58:57.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.831+0000 7f84dd0f5640 1 --2- 172.21.15.32:0/454902176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84d806b820 0x7f84d80b78f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:57.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.831+0000 7f84dd0f5640 1 --2- 172.21.15.32:0/454902176 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84d815e9b0 0x7f84d80d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:57.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.831+0000 7f84dd0f5640 1 -- 172.21.15.32:0/454902176 >> 172.21.15.32:0/454902176 conn(0x7f84d80d11d0 msgr2=0x7f84d80d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:58:57.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.831+0000 7f84dd0f5640 1 -- 172.21.15.32:0/454902176 shutdown_connections 2024-04-24T22:58:57.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.831+0000 7f84dd0f5640 1 -- 172.21.15.32:0/454902176 wait complete. 2024-04-24T22:58:57.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.833+0000 7f84dd0f5640 1 Processor -- start 2024-04-24T22:58:57.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.833+0000 7f84dd0f5640 1 -- start start 2024-04-24T22:58:57.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.833+0000 7f84dd0f5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84d806b820 0x7f84d80cc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:57.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.833+0000 7f84dd0f5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84d80d4210 0x7f84d80cc770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:57.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.833+0000 7f84dd0f5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84d815e9b0 0x7f84d8153d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:57.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.833+0000 7f84dd0f5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f84d80ba0e0 con 0x7f84d80d4210 2024-04-24T22:58:57.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.833+0000 7f84dd0f5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f84d80d2400 con 0x7f84d806b820 2024-04-24T22:58:57.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.833+0000 7f84dd0f5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f84d80ba260 con 0x7f84d815e9b0 2024-04-24T22:58:57.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.833+0000 7f84d659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84d80d4210 0x7f84d80cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:57.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.833+0000 7f84d659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84d80d4210 0x7f84d80cc770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51718/0 (socket says 172.21.15.32:51718) 2024-04-24T22:58:57.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.833+0000 7f84d659c640 1 -- 172.21.15.32:0/3238612341 learned_addr learned my addr 172.21.15.32:0/3238612341 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:58:57.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.834+0000 7f84d759e640 1 --2- 172.21.15.32:0/3238612341 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84d815e9b0 0x7f84d8153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:57.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.834+0000 7f84d6d9d640 1 --2- 172.21.15.32:0/3238612341 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84d806b820 0x7f84d80cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:57.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.834+0000 7f84d659c640 1 -- 172.21.15.32:0/3238612341 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84d815e9b0 msgr2=0x7f84d8153d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:57.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.834+0000 7f84d659c640 1 --2- 172.21.15.32:0/3238612341 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84d815e9b0 0x7f84d8153d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:57.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.834+0000 7f84d659c640 1 -- 172.21.15.32:0/3238612341 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84d806b820 msgr2=0x7f84d80cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:57.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.834+0000 7f84d659c640 1 --2- 172.21.15.32:0/3238612341 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84d806b820 0x7f84d80cc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:57.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.834+0000 7f84d659c640 1 -- 172.21.15.32:0/3238612341 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84d80ca8e0 con 0x7f84d80d4210 2024-04-24T22:58:57.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.834+0000 7f84d6d9d640 1 --2- 172.21.15.32:0/3238612341 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84d806b820 0x7f84d80cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:58:57.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.834+0000 7f84d759e640 1 --2- 172.21.15.32:0/3238612341 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84d815e9b0 0x7f84d8153d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:58:57.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.834+0000 7f84d659c640 1 --2- 172.21.15.32:0/3238612341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84d80d4210 0x7f84d80cc770 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f84c000c530 tx=0x7f84c000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:58:57.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.834+0000 7f84cf7fe640 1 -- 172.21.15.32:0/3238612341 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c0017070 con 0x7f84d80d4210 2024-04-24T22:58:57.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.834+0000 7f84cf7fe640 1 -- 172.21.15.32:0/3238612341 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84c0005ce0 con 0x7f84d80d4210 2024-04-24T22:58:57.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.834+0000 7f84cf7fe640 1 -- 172.21.15.32:0/3238612341 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c00072b0 con 0x7f84d80d4210 2024-04-24T22:58:57.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.834+0000 7f84dd0f5640 1 -- 172.21.15.32:0/3238612341 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84d80ccda0 con 0x7f84d80d4210 2024-04-24T22:58:57.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.834+0000 7f84dd0f5640 1 -- 172.21.15.32:0/3238612341 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84d80cd360 con 0x7f84d80d4210 2024-04-24T22:58:57.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.835+0000 7f84dd0f5640 1 -- 172.21.15.32:0/3238612341 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84d8068430 con 0x7f84d80d4210 2024-04-24T22:58:57.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.836+0000 7f84cf7fe640 1 -- 172.21.15.32:0/3238612341 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f84c001a070 con 0x7f84d80d4210 2024-04-24T22:58:57.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.836+0000 7f84cf7fe640 1 --2- 172.21.15.32:0/3238612341 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f84b806be30 0x7f84b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:57.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.836+0000 7f84cf7fe640 1 -- 172.21.15.32:0/3238612341 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f84c0089f30 con 0x7f84d80d4210 2024-04-24T22:58:57.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.836+0000 7f84d6d9d640 1 --2- 172.21.15.32:0/3238612341 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f84b806be30 0x7f84b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:57.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.837+0000 7f84d6d9d640 1 --2- 172.21.15.32:0/3238612341 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f84b806be30 0x7f84b806e2f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f84d0009a20 tx=0x7f84d0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:58:57.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:57.840+0000 7f84cf7fe640 1 -- 172.21.15.32:0/3238612341 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f84c0058cf0 con 0x7f84d80d4210 2024-04-24T22:58:58.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.026+0000 7f84dd0f5640 1 -- 172.21.15.32:0/3238612341 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f84d806b110 con 0x7f84d80d4210 2024-04-24T22:58:58.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.026+0000 7f84cf7fe640 1 -- 172.21.15.32:0/3238612341 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) ==== 93+0+10 (secure 0 0 0) 0x7f84c00585f0 con 0x7f84d80d4210 2024-04-24T22:58:58.028 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T22:58:58.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.029+0000 7f84dd0f5640 1 -- 172.21.15.32:0/3238612341 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f84b806be30 msgr2=0x7f84b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:58.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.029+0000 7f84dd0f5640 1 --2- 172.21.15.32:0/3238612341 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f84b806be30 0x7f84b806e2f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f84d0009a20 tx=0x7f84d0008040 comp rx=0 tx=0).stop 2024-04-24T22:58:58.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.029+0000 7f84dd0f5640 1 -- 172.21.15.32:0/3238612341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84d80d4210 msgr2=0x7f84d80cc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:58.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.029+0000 7f84dd0f5640 1 --2- 172.21.15.32:0/3238612341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84d80d4210 0x7f84d80cc770 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f84c000c530 tx=0x7f84c000c9f0 comp rx=0 tx=0).stop 2024-04-24T22:58:58.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.029+0000 7f84dd0f5640 1 -- 172.21.15.32:0/3238612341 shutdown_connections 2024-04-24T22:58:58.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.029+0000 7f84dd0f5640 1 --2- 172.21.15.32:0/3238612341 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f84b806be30 0x7f84b806e2f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:58.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.029+0000 7f84dd0f5640 1 --2- 172.21.15.32:0/3238612341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84d80d4210 0x7f84d80cc770 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:58.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.029+0000 7f84dd0f5640 1 --2- 172.21.15.32:0/3238612341 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84d815e9b0 0x7f84d8153d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:58.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.029+0000 7f84dd0f5640 1 --2- 172.21.15.32:0/3238612341 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84d806b820 0x7f84d80cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:58.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.029+0000 7f84dd0f5640 1 -- 172.21.15.32:0/3238612341 >> 172.21.15.32:0/3238612341 conn(0x7f84d80d11d0 msgr2=0x7f84d8155d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:58:58.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.029+0000 7f84dd0f5640 1 -- 172.21.15.32:0/3238612341 shutdown_connections 2024-04-24T22:58:58.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.029+0000 7f84dd0f5640 1 -- 172.21.15.32:0/3238612341 wait complete. 2024-04-24T22:58:58.041 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T22:58:58.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.140+0000 7fd5cfea5640 1 Processor -- start 2024-04-24T22:58:58.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.140+0000 7fd5cfea5640 1 -- start start 2024-04-24T22:58:58.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.141+0000 7fd5cfea5640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd5c8073390 0x7fd5c806d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:58.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.141+0000 7fd5cfea5640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5c815ccf0 con 0x7fd5c806d400 2024-04-24T22:58:58.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.141+0000 7fd5cfea5640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5c815cfb0 con 0x7fd5c8156bb0 2024-04-24T22:58:58.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.141+0000 7fd5cfea5640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fd5c815d290 con 0x7fd5c8073390 2024-04-24T22:58:58.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.141+0000 7fd5cd440640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd5c806d400 0x7fd5c8072c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54758/0 (socket says 172.21.15.32:54758) 2024-04-24T22:58:58.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.141+0000 7fd5cd440640 1 -- 172.21.15.32:0/98672041 learned_addr learned my addr 172.21.15.32:0/98672041 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:58:58.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.141+0000 7fd5ce442640 1 --2- 172.21.15.32:0/98672041 >> v2:172.21.15.121:3300/0 conn(0x7fd5c8073390 0x7fd5c806d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:58.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.141+0000 7fd5ccc3f640 1 -- 172.21.15.32:0/98672041 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1603233396 0 0) 0x7fd5c815ccf0 con 0x7fd5c806d400 2024-04-24T22:58:58.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.141+0000 7fd5ccc3f640 1 -- 172.21.15.32:0/98672041 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5b4003520 con 0x7fd5c806d400 2024-04-24T22:58:58.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.141+0000 7fd5ccc3f640 1 -- 172.21.15.32:0/98672041 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1697888210 0 0) 0x7fd5b4003520 con 0x7fd5c806d400 2024-04-24T22:58:58.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.142+0000 7fd5ccc3f640 1 -- 172.21.15.32:0/98672041 >> v2:172.21.15.121:3300/0 conn(0x7fd5c8073390 msgr2=0x7fd5c806d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:58:58.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.142+0000 7fd5ccc3f640 1 --2- 172.21.15.32:0/98672041 >> v2:172.21.15.121:3300/0 conn(0x7fd5c8073390 0x7fd5c806d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:58.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.142+0000 7fd5ccc3f640 1 -- 172.21.15.32:0/98672041 >> v1:172.21.15.121:6789/0 conn(0x7fd5c8156bb0 legacy=0x7fd5c815dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:58.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.142+0000 7fd5ccc3f640 1 -- 172.21.15.32:0/98672041 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5c815d570 con 0x7fd5c806d400 2024-04-24T22:58:58.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.142+0000 7fd5ccc3f640 1 -- 172.21.15.32:0/98672041 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7fd5b8001670 con 0x7fd5c806d400 2024-04-24T22:58:58.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.142+0000 7fd5ccc3f640 1 -- 172.21.15.32:0/98672041 >> v1:172.21.15.32:6789/0 conn(0x7fd5c806d400 legacy=0x7fd5c8072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:58.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.142+0000 7fd5ccc3f640 1 --2- 172.21.15.32:0/98672041 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5b40039b0 0x7fd5b4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:58.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.142+0000 7fd5ccc3f640 1 --2- 172.21.15.32:0/98672041 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5b40043b0 0x7fd5b4004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:58.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.142+0000 7fd5ccc3f640 1 --2- 172.21.15.32:0/98672041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5b4005120 0x7fd5b400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:58.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.142+0000 7fd5ccc3f640 1 -- 172.21.15.32:0/98672041 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c815d290 con 0x7fd5b4005120 2024-04-24T22:58:58.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.142+0000 7fd5ccc3f640 1 -- 172.21.15.32:0/98672041 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b400c060 con 0x7fd5b40043b0 2024-04-24T22:58:58.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.142+0000 7fd5ccc3f640 1 -- 172.21.15.32:0/98672041 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd5b400c260 con 0x7fd5b40039b0 2024-04-24T22:58:58.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.142+0000 7fd5ce442640 1 --2- 172.21.15.32:0/98672041 >> v2:172.21.15.121:3300/0 conn(0x7fd5c8073390 0x7fd5c806d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:58:58.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.142+0000 7fd5cd440640 1 --2- 172.21.15.32:0/98672041 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5b40043b0 0x7fd5b4004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:58.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.142+0000 7fd5cdc41640 1 --2- 172.21.15.32:0/98672041 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5b40039b0 0x7fd5b4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:58.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.142+0000 7fd5ce442640 1 --2- 172.21.15.32:0/98672041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5b4005120 0x7fd5b400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:58.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.142+0000 7fd5cd440640 1 -- 172.21.15.32:0/98672041 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5b40039b0 msgr2=0x7fd5b4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:58:58.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.142+0000 7fd5cd440640 1 --2- 172.21.15.32:0/98672041 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5b40039b0 0x7fd5b4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:58.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.142+0000 7fd5cd440640 1 -- 172.21.15.32:0/98672041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5b4005120 msgr2=0x7fd5b400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:58:58.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.142+0000 7fd5cd440640 1 --2- 172.21.15.32:0/98672041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5b4005120 0x7fd5b400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:58.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.142+0000 7fd5cd440640 1 -- 172.21.15.32:0/98672041 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5b4003520 con 0x7fd5b40043b0 2024-04-24T22:58:58.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.142+0000 7fd5ce442640 1 --2- 172.21.15.32:0/98672041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5b4005120 0x7fd5b400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:58:58.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.142+0000 7fd5cdc41640 1 --2- 172.21.15.32:0/98672041 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5b40039b0 0x7fd5b4003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:58:58.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.142+0000 7fd5ce442640 1 -- 172.21.15.32:0/98672041 reap_dead start 2024-04-24T22:58:58.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.143+0000 7fd5cd440640 1 --2- 172.21.15.32:0/98672041 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5b40043b0 0x7fd5b4004880 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd5b800c240 tx=0x7fd5b4005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=906c8ff8cd1068aa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:58:58.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.143+0000 7fd5ccc3f640 1 -- 172.21.15.32:0/98672041 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8007d30 con 0x7fd5b40043b0 2024-04-24T22:58:58.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.143+0000 7fd5ccc3f640 1 -- 172.21.15.32:0/98672041 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5b8005af0 con 0x7fd5b40043b0 2024-04-24T22:58:58.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.143+0000 7fd5ccc3f640 1 -- 172.21.15.32:0/98672041 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8007070 con 0x7fd5b40043b0 2024-04-24T22:58:58.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.144+0000 7fd5cfea5640 1 -- 172.21.15.32:0/98672041 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5b40043b0 msgr2=0x7fd5b4004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:58.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.144+0000 7fd5cfea5640 1 --2- 172.21.15.32:0/98672041 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5b40043b0 0x7fd5b4004880 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd5b800c240 tx=0x7fd5b4005830 comp rx=0 tx=0).stop 2024-04-24T22:58:58.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.144+0000 7fd5cfea5640 1 -- 172.21.15.32:0/98672041 shutdown_connections 2024-04-24T22:58:58.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.144+0000 7fd5cfea5640 1 --2- 172.21.15.32:0/98672041 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5b40043b0 0x7fd5b4004880 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:58.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.144+0000 7fd5cfea5640 1 -- 172.21.15.32:0/98672041 >> 172.21.15.32:0/98672041 conn(0x7fd5c806e800 msgr2=0x7fd5c806ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:58:58.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.144+0000 7fd5cfea5640 1 -- 172.21.15.32:0/98672041 shutdown_connections 2024-04-24T22:58:58.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.144+0000 7fd5cfea5640 1 -- 172.21.15.32:0/98672041 wait complete. 2024-04-24T22:58:58.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.145+0000 7fd5cfea5640 1 Processor -- start 2024-04-24T22:58:58.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.145+0000 7fd5cfea5640 1 -- start start 2024-04-24T22:58:58.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.146+0000 7fd5cfea5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5b40043b0 0x7fd5c815c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:58.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.146+0000 7fd5cfea5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5c813eb30 0x7fd5c81f7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:58.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.146+0000 7fd5cfea5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5c8141980 0x7fd5c8141e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:58.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.146+0000 7fd5cfea5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c8157110 con 0x7fd5c813eb30 2024-04-24T22:58:58.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.146+0000 7fd5cfea5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c8156f90 con 0x7fd5c8141980 2024-04-24T22:58:58.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.146+0000 7fd5cfea5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd5c806d8b0 con 0x7fd5b40043b0 2024-04-24T22:58:58.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.146+0000 7fd5cd440640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5c813eb30 0x7fd5c81f7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:58.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.146+0000 7fd5cdc41640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5b40043b0 0x7fd5c815c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:58.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.146+0000 7fd5cd440640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5c813eb30 0x7fd5c81f7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51734/0 (socket says 172.21.15.32:51734) 2024-04-24T22:58:58.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.146+0000 7fd5cd440640 1 -- 172.21.15.32:0/1478252654 learned_addr learned my addr 172.21.15.32:0/1478252654 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:58:58.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.146+0000 7fd5ce442640 1 --2- 172.21.15.32:0/1478252654 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5c8141980 0x7fd5c8141e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:58.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.146+0000 7fd5cd440640 1 -- 172.21.15.32:0/1478252654 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5b40043b0 msgr2=0x7fd5c815c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:58.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.146+0000 7fd5cd440640 1 --2- 172.21.15.32:0/1478252654 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5b40043b0 0x7fd5c815c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:58.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.146+0000 7fd5cd440640 1 -- 172.21.15.32:0/1478252654 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5c8141980 msgr2=0x7fd5c8141e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:58.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.146+0000 7fd5cd440640 1 --2- 172.21.15.32:0/1478252654 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5c8141980 0x7fd5c8141e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:58.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.146+0000 7fd5cd440640 1 -- 172.21.15.32:0/1478252654 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5c81f81e0 con 0x7fd5c813eb30 2024-04-24T22:58:58.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.146+0000 7fd5cdc41640 1 --2- 172.21.15.32:0/1478252654 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5b40043b0 0x7fd5c815c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:58:58.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.146+0000 7fd5ce442640 1 --2- 172.21.15.32:0/1478252654 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5c8141980 0x7fd5c8141e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:58:58.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.146+0000 7fd5cd440640 1 --2- 172.21.15.32:0/1478252654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5c813eb30 0x7fd5c81f7b00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd5b800c240 tx=0x7fd5c80cfb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:58:58.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.148+0000 7fd5be7fc640 1 -- 172.21.15.32:0/1478252654 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8037040 con 0x7fd5c813eb30 2024-04-24T22:58:58.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.148+0000 7fd5be7fc640 1 -- 172.21.15.32:0/1478252654 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5b8041070 con 0x7fd5c813eb30 2024-04-24T22:58:58.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.148+0000 7fd5cfea5640 1 -- 172.21.15.32:0/1478252654 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5c81f84c0 con 0x7fd5c813eb30 2024-04-24T22:58:58.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.148+0000 7fd5be7fc640 1 -- 172.21.15.32:0/1478252654 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b80047e0 con 0x7fd5c813eb30 2024-04-24T22:58:58.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.148+0000 7fd5cfea5640 1 -- 172.21.15.32:0/1478252654 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5c813f250 con 0x7fd5c813eb30 2024-04-24T22:58:58.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.149+0000 7fd5cfea5640 1 -- 172.21.15.32:0/1478252654 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5c8141290 con 0x7fd5c813eb30 2024-04-24T22:58:58.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.150+0000 7fd5be7fc640 1 -- 172.21.15.32:0/1478252654 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd5b8004980 con 0x7fd5c813eb30 2024-04-24T22:58:58.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.150+0000 7fd5be7fc640 1 --2- 172.21.15.32:0/1478252654 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd5ac06be30 0x7fd5ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:58:58.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.150+0000 7fd5be7fc640 1 -- 172.21.15.32:0/1478252654 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fd5b80aa620 con 0x7fd5c813eb30 2024-04-24T22:58:58.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.150+0000 7fd5cdc41640 1 --2- 172.21.15.32:0/1478252654 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd5ac06be30 0x7fd5ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:58:58.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.151+0000 7fd5cdc41640 1 --2- 172.21.15.32:0/1478252654 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd5ac06be30 0x7fd5ac06e2f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4001400 tx=0x7fd5c4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:58:58.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.154+0000 7fd5be7fc640 1 -- 172.21.15.32:0/1478252654 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd5b8078cd0 con 0x7fd5c813eb30 2024-04-24T22:58:58.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.374+0000 7fd5cfea5640 1 -- 172.21.15.32:0/1478252654 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5c8068430 con 0x7fd5c813eb30 2024-04-24T22:58:58.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.374+0000 7fd5be7fc640 1 -- 172.21.15.32:0/1478252654 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1389 (secure 0 0 0) 0x7fd5b807cfd0 con 0x7fd5c813eb30 2024-04-24T22:58:58.376 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:58:58.376 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/923934104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":923934104},{"type":"v1","addr":"172.21.15.32:6833","nonce":923934104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-24T22:58:58.377 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 2 2024-04-24T22:58:58.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.376+0000 7fd5cfea5640 1 -- 172.21.15.32:0/1478252654 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd5ac06be30 msgr2=0x7fd5ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:58.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.376+0000 7fd5cfea5640 1 --2- 172.21.15.32:0/1478252654 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd5ac06be30 0x7fd5ac06e2f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4001400 tx=0x7fd5c4009040 comp rx=0 tx=0).stop 2024-04-24T22:58:58.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.376+0000 7fd5cfea5640 1 -- 172.21.15.32:0/1478252654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5c813eb30 msgr2=0x7fd5c81f7b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:58:58.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.376+0000 7fd5cfea5640 1 --2- 172.21.15.32:0/1478252654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5c813eb30 0x7fd5c81f7b00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd5b800c240 tx=0x7fd5c80cfb20 comp rx=0 tx=0).stop 2024-04-24T22:58:58.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.376+0000 7fd5cfea5640 1 -- 172.21.15.32:0/1478252654 shutdown_connections 2024-04-24T22:58:58.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.376+0000 7fd5cfea5640 1 --2- 172.21.15.32:0/1478252654 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd5ac06be30 0x7fd5ac06e2f0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:58.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.376+0000 7fd5cfea5640 1 --2- 172.21.15.32:0/1478252654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5c813eb30 0x7fd5c81f7b00 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:58.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.376+0000 7fd5cfea5640 1 --2- 172.21.15.32:0/1478252654 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5c8141980 0x7fd5c8141e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:58.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.376+0000 7fd5cfea5640 1 --2- 172.21.15.32:0/1478252654 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5b40043b0 0x7fd5c815c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:58:58.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.376+0000 7fd5cfea5640 1 -- 172.21.15.32:0/1478252654 >> 172.21.15.32:0/1478252654 conn(0x7fd5c806e800 msgr2=0x7fd5c815f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:58:58.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.376+0000 7fd5cfea5640 1 -- 172.21.15.32:0/1478252654 shutdown_connections 2024-04-24T22:58:58.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:58:58.376+0000 7fd5cfea5640 1 -- 172.21.15.32:0/1478252654 wait complete. 2024-04-24T22:59:00.389 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T22:59:00.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.501+0000 7f7fa9909640 1 Processor -- start 2024-04-24T22:59:00.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.502+0000 7f7fa9909640 1 -- start start 2024-04-24T22:59:00.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.502+0000 7f7fa9909640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7fa4160320 0x7f7fa4074210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:00.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.502+0000 7f7fa9909640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7fa4074750 0x7f7fa40b1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:00.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.502+0000 7f7fa9909640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fa4160900 con 0x7f7fa40b0700 2024-04-24T22:59:00.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.502+0000 7f7fa9909640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f7fa4160be0 con 0x7f7fa4074750 2024-04-24T22:59:00.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.502+0000 7f7fa9909640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f7fa4160ec0 con 0x7f7fa4160320 2024-04-24T22:59:00.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.503+0000 7f7fa27fc640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7fa4160320 0x7f7fa4074210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:00.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.503+0000 7f7fa27fc640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7fa4160320 0x7f7fa4074210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.32:58710/0 (socket says 172.21.15.32:58710) 2024-04-24T22:59:00.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.503+0000 7f7fa37fe640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7fa4074750 0x7f7fa40b1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:00.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.503+0000 7f7fa27fc640 1 -- 172.21.15.32:0/3390860158 learned_addr learned my addr 172.21.15.32:0/3390860158 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:00.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.503+0000 7f7fa1ffb640 1 -- 172.21.15.32:0/3390860158 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2532987224 0 0) 0x7f7fa4160900 con 0x7f7fa40b0700 2024-04-24T22:59:00.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.503+0000 7f7fa1ffb640 1 -- 172.21.15.32:0/3390860158 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f8c003520 con 0x7f7fa40b0700 2024-04-24T22:59:00.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.503+0000 7f7fa27fc640 1 -- 172.21.15.32:0/3390860158 >> v2:172.21.15.121:3300/0 conn(0x7f7fa4074750 msgr2=0x7f7fa40b1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:00.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.503+0000 7f7fa27fc640 1 --2- 172.21.15.32:0/3390860158 >> v2:172.21.15.121:3300/0 conn(0x7f7fa4074750 0x7f7fa40b1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:00.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.503+0000 7f7fa27fc640 1 -- 172.21.15.32:0/3390860158 >> v1:172.21.15.121:6789/0 conn(0x7f7fa40b0700 legacy=0x7f7fa40731e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:00.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.503+0000 7f7fa27fc640 1 -- 172.21.15.32:0/3390860158 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fa41611a0 con 0x7f7fa4160320 2024-04-24T22:59:00.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.503+0000 7f7fa37fe640 1 --2- 172.21.15.32:0/3390860158 >> v2:172.21.15.121:3300/0 conn(0x7f7fa4074750 0x7f7fa40b1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:59:00.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.504+0000 7f7fa27fc640 1 --2- 172.21.15.32:0/3390860158 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7fa4160320 0x7f7fa4074210 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7f98009870 tx=0x7f7f9802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fd39e19bd13c649 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:00.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.504+0000 7f7fa1ffb640 1 -- 172.21.15.32:0/3390860158 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f98037070 con 0x7f7fa4160320 2024-04-24T22:59:00.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.504+0000 7f7fa1ffb640 1 -- 172.21.15.32:0/3390860158 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f98032440 con 0x7f7fa4160320 2024-04-24T22:59:00.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.504+0000 7f7fa1ffb640 1 -- 172.21.15.32:0/3390860158 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f98032810 con 0x7f7fa4160320 2024-04-24T22:59:00.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.504+0000 7f7fa9909640 1 -- 172.21.15.32:0/3390860158 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7fa4160320 msgr2=0x7f7fa4074210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:00.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.504+0000 7f7fa9909640 1 --2- 172.21.15.32:0/3390860158 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7fa4160320 0x7f7fa4074210 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7f98009870 tx=0x7f7f9802a270 comp rx=0 tx=0).stop 2024-04-24T22:59:00.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.504+0000 7f7fa9909640 1 -- 172.21.15.32:0/3390860158 shutdown_connections 2024-04-24T22:59:00.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.504+0000 7f7fa9909640 1 --2- 172.21.15.32:0/3390860158 >> v2:172.21.15.121:3300/0 conn(0x7f7fa4074750 0x7f7fa40b1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:00.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.504+0000 7f7fa9909640 1 --2- 172.21.15.32:0/3390860158 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7fa4160320 0x7f7fa4074210 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:00.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.504+0000 7f7fa9909640 1 -- 172.21.15.32:0/3390860158 >> 172.21.15.32:0/3390860158 conn(0x7f7fa40b4de0 msgr2=0x7f7fa4072850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:00.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.505+0000 7f7fa9909640 1 -- 172.21.15.32:0/3390860158 shutdown_connections 2024-04-24T22:59:00.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.505+0000 7f7fa9909640 1 -- 172.21.15.32:0/3390860158 wait complete. 2024-04-24T22:59:00.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.506+0000 7f7fa9909640 1 Processor -- start 2024-04-24T22:59:00.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.506+0000 7f7fa9909640 1 -- start start 2024-04-24T22:59:00.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.507+0000 7f7fa9909640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7fa4074750 0x7f7fa409aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:00.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.507+0000 7f7fa9909640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7fa40b0700 0x7f7fa415a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:00.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.507+0000 7f7fa9909640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7fa4160320 0x7f7fa415a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:00.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.507+0000 7f7fa9909640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa40b18d0 con 0x7f7fa40b0700 2024-04-24T22:59:00.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.507+0000 7f7fa9909640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa40b0ae0 con 0x7f7fa4160320 2024-04-24T22:59:00.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.507+0000 7f7fa9909640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f7fa40b1a50 con 0x7f7fa4074750 2024-04-24T22:59:00.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.507+0000 7f7fa27fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7fa40b0700 0x7f7fa415a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:00.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.507+0000 7f7fa27fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7fa40b0700 0x7f7fa415a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44456/0 (socket says 172.21.15.32:44456) 2024-04-24T22:59:00.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.507+0000 7f7fa27fc640 1 -- 172.21.15.32:0/4111688201 learned_addr learned my addr 172.21.15.32:0/4111688201 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:00.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.507+0000 7f7fa2ffd640 1 --2- 172.21.15.32:0/4111688201 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7fa4074750 0x7f7fa409aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:00.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.507+0000 7f7fa37fe640 1 --2- 172.21.15.32:0/4111688201 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7fa4160320 0x7f7fa415a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:00.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.507+0000 7f7fa27fc640 1 -- 172.21.15.32:0/4111688201 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7fa4074750 msgr2=0x7f7fa409aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:00.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.507+0000 7f7fa27fc640 1 --2- 172.21.15.32:0/4111688201 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7fa4074750 0x7f7fa409aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:00.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.507+0000 7f7fa27fc640 1 -- 172.21.15.32:0/4111688201 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7fa4160320 msgr2=0x7f7fa415a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:00.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.507+0000 7f7fa27fc640 1 --2- 172.21.15.32:0/4111688201 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7fa4160320 0x7f7fa415a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:00.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.507+0000 7f7fa27fc640 1 -- 172.21.15.32:0/4111688201 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fa40c5aa0 con 0x7f7fa40b0700 2024-04-24T22:59:00.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.507+0000 7f7fa27fc640 1 --2- 172.21.15.32:0/4111688201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7fa40b0700 0x7f7fa415a180 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7f98000c00 tx=0x7f7f980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:00.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.508+0000 7f7f8bfff640 1 -- 172.21.15.32:0/4111688201 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f98041020 con 0x7f7fa40b0700 2024-04-24T22:59:00.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.508+0000 7f7f8bfff640 1 -- 172.21.15.32:0/4111688201 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f9803d620 con 0x7f7fa40b0700 2024-04-24T22:59:00.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.508+0000 7f7fa9909640 1 -- 172.21.15.32:0/4111688201 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fa40c5d80 con 0x7f7fa40b0700 2024-04-24T22:59:00.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.508+0000 7f7f8bfff640 1 -- 172.21.15.32:0/4111688201 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f98037040 con 0x7f7fa40b0700 2024-04-24T22:59:00.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.508+0000 7f7fa9909640 1 -- 172.21.15.32:0/4111688201 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fa415e090 con 0x7f7fa40b0700 2024-04-24T22:59:00.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.509+0000 7f7fa9909640 1 -- 172.21.15.32:0/4111688201 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fa4068430 con 0x7f7fa40b0700 2024-04-24T22:59:00.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.509+0000 7f7f8bfff640 1 -- 172.21.15.32:0/4111688201 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7f98044020 con 0x7f7fa40b0700 2024-04-24T22:59:00.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.510+0000 7f7f8bfff640 1 --2- 172.21.15.32:0/4111688201 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7f9006be30 0x7f7f9006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:00.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.510+0000 7f7f8bfff640 1 -- 172.21.15.32:0/4111688201 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f7f980ab1a0 con 0x7f7fa40b0700 2024-04-24T22:59:00.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.510+0000 7f7fa2ffd640 1 --2- 172.21.15.32:0/4111688201 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7f9006be30 0x7f7f9006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:00.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.510+0000 7f7fa2ffd640 1 --2- 172.21.15.32:0/4111688201 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7f9006be30 0x7f7f9006e2f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c0011e0 tx=0x7f7f9c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:00.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.514+0000 7f7f8bfff640 1 -- 172.21.15.32:0/4111688201 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7f9804b030 con 0x7f7fa40b0700 2024-04-24T22:59:00.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.735+0000 7f7fa9909640 1 -- 172.21.15.32:0/4111688201 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7fa415e260 con 0x7f7fa40b0700 2024-04-24T22:59:00.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.736+0000 7f7f8bfff640 1 -- 172.21.15.32:0/4111688201 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3867 (secure 0 0 0) 0x7f7f98079e20 con 0x7f7fa40b0700 2024-04-24T22:59:00.737 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:59:00.738 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1485884060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1485884060},{"type":"v1","addr":"172.21.15.121:6837","nonce":1485884060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/923934104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":923934104},{"type":"v1","addr":"172.21.15.32:6833","nonce":923934104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3229115835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3229115835},{"type":"v1","addr":"172.21.15.32:6835","nonce":3229115835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/2644625343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2644625343},{"type":"v1","addr":"172.21.15.121:6836","nonce":2644625343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-24T22:59:00.738 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 3 2024-04-24T22:59:00.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.738+0000 7f7fa9909640 1 -- 172.21.15.32:0/4111688201 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7f9006be30 msgr2=0x7f7f9006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:00.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.738+0000 7f7fa9909640 1 --2- 172.21.15.32:0/4111688201 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7f9006be30 0x7f7f9006e2f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c0011e0 tx=0x7f7f9c009040 comp rx=0 tx=0).stop 2024-04-24T22:59:00.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.738+0000 7f7fa9909640 1 -- 172.21.15.32:0/4111688201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7fa40b0700 msgr2=0x7f7fa415a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:00.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.738+0000 7f7fa9909640 1 --2- 172.21.15.32:0/4111688201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7fa40b0700 0x7f7fa415a180 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7f98000c00 tx=0x7f7f980046c0 comp rx=0 tx=0).stop 2024-04-24T22:59:00.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.738+0000 7f7fa9909640 1 -- 172.21.15.32:0/4111688201 shutdown_connections 2024-04-24T22:59:00.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.738+0000 7f7fa9909640 1 --2- 172.21.15.32:0/4111688201 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7f9006be30 0x7f7f9006e2f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:00.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.738+0000 7f7fa9909640 1 --2- 172.21.15.32:0/4111688201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7fa40b0700 0x7f7fa415a180 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:00.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.738+0000 7f7fa9909640 1 --2- 172.21.15.32:0/4111688201 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7fa4160320 0x7f7fa415a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:00.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.738+0000 7f7fa9909640 1 --2- 172.21.15.32:0/4111688201 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7fa4074750 0x7f7fa409aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:00.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.738+0000 7f7fa9909640 1 -- 172.21.15.32:0/4111688201 >> 172.21.15.32:0/4111688201 conn(0x7f7fa40b4de0 msgr2=0x7f7fa40b35d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:00.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.738+0000 7f7fa9909640 1 -- 172.21.15.32:0/4111688201 shutdown_connections 2024-04-24T22:59:00.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.738+0000 7f7fa9909640 1 -- 172.21.15.32:0/4111688201 wait complete. 2024-04-24T22:59:00.751 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:59:00.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.850+0000 7f7123bb0640 1 Processor -- start 2024-04-24T22:59:00.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.851+0000 7f7123bb0640 1 -- start start 2024-04-24T22:59:00.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.851+0000 7f7123bb0640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f711c15e9b0 0x7f711c0d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:00.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.851+0000 7f7123bb0640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f711c06b820 0x7f711c0b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:00.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.851+0000 7f7123bb0640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f711c0d3170 con 0x7f711c0d4210 2024-04-24T22:59:00.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.851+0000 7f7123bb0640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f711c0d3450 con 0x7f711c06b820 2024-04-24T22:59:00.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.851+0000 7f7123bb0640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f711c0d3730 con 0x7f711c15e9b0 2024-04-24T22:59:00.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.851+0000 7f712214d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f711c06b820 0x7f711c0b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:00.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.851+0000 7f712214d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f711c06b820 0x7f711c0b78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44464/0 (socket says 172.21.15.32:44464) 2024-04-24T22:59:00.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.852+0000 7f712214d640 1 -- 172.21.15.32:0/635627510 learned_addr learned my addr 172.21.15.32:0/635627510 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:00.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.852+0000 7f712194c640 1 --2- 172.21.15.32:0/635627510 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f711c15e9b0 0x7f711c0d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:00.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.852+0000 7f712214d640 1 -- 172.21.15.32:0/635627510 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f711c15e9b0 msgr2=0x7f711c0d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:00.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.852+0000 7f712214d640 1 --2- 172.21.15.32:0/635627510 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f711c15e9b0 0x7f711c0d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:00.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.852+0000 7f712214d640 1 -- 172.21.15.32:0/635627510 >> v1:172.21.15.121:6789/0 conn(0x7f711c0d4210 legacy=0x7f711c06b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:00.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.852+0000 7f712214d640 1 -- 172.21.15.32:0/635627510 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f711c0d3a10 con 0x7f711c06b820 2024-04-24T22:59:00.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.852+0000 7f712194c640 1 --2- 172.21.15.32:0/635627510 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f711c15e9b0 0x7f711c0d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:00.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.852+0000 7f712214d640 1 --2- 172.21.15.32:0/635627510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f711c06b820 0x7f711c0b78f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7114009e50 tx=0x7f711402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8681f2a760e7bb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:00.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.853+0000 7f712094a640 1 -- 172.21.15.32:0/635627510 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7114036070 con 0x7f711c06b820 2024-04-24T22:59:00.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.853+0000 7f712094a640 1 -- 172.21.15.32:0/635627510 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7114031600 con 0x7f711c06b820 2024-04-24T22:59:00.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.853+0000 7f712094a640 1 -- 172.21.15.32:0/635627510 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71140319d0 con 0x7f711c06b820 2024-04-24T22:59:00.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.853+0000 7f7123bb0640 1 -- 172.21.15.32:0/635627510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f711c06b820 msgr2=0x7f711c0b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:00.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.853+0000 7f7123bb0640 1 --2- 172.21.15.32:0/635627510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f711c06b820 0x7f711c0b78f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7114009e50 tx=0x7f711402a710 comp rx=0 tx=0).stop 2024-04-24T22:59:00.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.853+0000 7f7123bb0640 1 -- 172.21.15.32:0/635627510 shutdown_connections 2024-04-24T22:59:00.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.853+0000 7f7123bb0640 1 --2- 172.21.15.32:0/635627510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f711c06b820 0x7f711c0b78f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:00.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.853+0000 7f7123bb0640 1 --2- 172.21.15.32:0/635627510 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f711c15e9b0 0x7f711c0d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:00.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.853+0000 7f7123bb0640 1 -- 172.21.15.32:0/635627510 >> 172.21.15.32:0/635627510 conn(0x7f711c0d11d0 msgr2=0x7f711c0d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:00.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.853+0000 7f7123bb0640 1 -- 172.21.15.32:0/635627510 shutdown_connections 2024-04-24T22:59:00.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.853+0000 7f7123bb0640 1 -- 172.21.15.32:0/635627510 wait complete. 2024-04-24T22:59:00.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.854+0000 7f7123bb0640 1 Processor -- start 2024-04-24T22:59:00.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.854+0000 7f7123bb0640 1 -- start start 2024-04-24T22:59:00.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.855+0000 7f7123bb0640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f711c06b820 0x7f711c153d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:00.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.855+0000 7f7123bb0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f711c0d4210 0x7f711c0ce240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:00.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.855+0000 7f7123bb0640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f711c15e9b0 0x7f711c0ce780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:00.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.855+0000 7f7123bb0640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f711c0ba0e0 con 0x7f711c0d4210 2024-04-24T22:59:00.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.855+0000 7f7123bb0640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f711c0d2400 con 0x7f711c15e9b0 2024-04-24T22:59:00.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.855+0000 7f7123bb0640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f711c0ba260 con 0x7f711c06b820 2024-04-24T22:59:00.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.855+0000 7f712114b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f711c0d4210 0x7f711c0ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:00.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.855+0000 7f712114b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f711c0d4210 0x7f711c0ce240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44476/0 (socket says 172.21.15.32:44476) 2024-04-24T22:59:00.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.855+0000 7f712114b640 1 -- 172.21.15.32:0/3136059474 learned_addr learned my addr 172.21.15.32:0/3136059474 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:00.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.855+0000 7f712194c640 1 --2- 172.21.15.32:0/3136059474 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f711c06b820 0x7f711c153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:00.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.855+0000 7f712214d640 1 --2- 172.21.15.32:0/3136059474 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f711c15e9b0 0x7f711c0ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:00.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.856+0000 7f712114b640 1 -- 172.21.15.32:0/3136059474 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f711c06b820 msgr2=0x7f711c153d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:00.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.856+0000 7f712114b640 1 --2- 172.21.15.32:0/3136059474 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f711c06b820 0x7f711c153d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:00.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.856+0000 7f712114b640 1 -- 172.21.15.32:0/3136059474 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f711c15e9b0 msgr2=0x7f711c0ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:00.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.856+0000 7f712114b640 1 --2- 172.21.15.32:0/3136059474 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f711c15e9b0 0x7f711c0ce780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:00.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.856+0000 7f712114b640 1 -- 172.21.15.32:0/3136059474 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f711c15eec0 con 0x7f711c0d4210 2024-04-24T22:59:00.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.856+0000 7f712194c640 1 --2- 172.21.15.32:0/3136059474 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f711c06b820 0x7f711c153d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:59:00.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.856+0000 7f712114b640 1 --2- 172.21.15.32:0/3136059474 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f711c0d4210 0x7f711c0ce240 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f710c00c400 tx=0x7f710c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:00.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.856+0000 7f71127fc640 1 -- 172.21.15.32:0/3136059474 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710c0052d0 con 0x7f711c0d4210 2024-04-24T22:59:00.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.856+0000 7f71127fc640 1 -- 172.21.15.32:0/3136059474 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f710c005490 con 0x7f711c0d4210 2024-04-24T22:59:00.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.856+0000 7f7123bb0640 1 -- 172.21.15.32:0/3136059474 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f711c15f1a0 con 0x7f711c0d4210 2024-04-24T22:59:00.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.856+0000 7f71127fc640 1 -- 172.21.15.32:0/3136059474 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710c007260 con 0x7f711c0d4210 2024-04-24T22:59:00.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.856+0000 7f7123bb0640 1 -- 172.21.15.32:0/3136059474 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f711c15f760 con 0x7f711c0d4210 2024-04-24T22:59:00.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.857+0000 7f7123bb0640 1 -- 172.21.15.32:0/3136059474 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f711c068430 con 0x7f711c0d4210 2024-04-24T22:59:00.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.857+0000 7f71127fc640 1 -- 172.21.15.32:0/3136059474 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f710c007400 con 0x7f711c0d4210 2024-04-24T22:59:00.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.858+0000 7f71127fc640 1 --2- 172.21.15.32:0/3136059474 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f710006be30 0x7f710006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:00.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.858+0000 7f71127fc640 1 -- 172.21.15.32:0/3136059474 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f710c08a010 con 0x7f711c0d4210 2024-04-24T22:59:00.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.858+0000 7f712194c640 1 --2- 172.21.15.32:0/3136059474 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f710006be30 0x7f710006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:00.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.859+0000 7f712194c640 1 --2- 172.21.15.32:0/3136059474 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f710006be30 0x7f710006e2f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f711c072440 tx=0x7f7118008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:00.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:00.862+0000 7f71127fc640 1 -- 172.21.15.32:0/3136059474 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f710c058da0 con 0x7f711c0d4210 2024-04-24T22:59:01.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.048+0000 7f7123bb0640 1 -- 172.21.15.32:0/3136059474 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f711c0d2680 con 0x7f711c0d4210 2024-04-24T22:59:01.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.048+0000 7f71127fc640 1 -- 172.21.15.32:0/3136059474 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12896 (secure 0 0 0) 0x7f710c0586a0 con 0x7f711c0d4210 2024-04-24T22:59:01.050 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:59:01.050 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":17,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T22:58:56.136617+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:59:01.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.050+0000 7f7123bb0640 1 -- 172.21.15.32:0/3136059474 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f710006be30 msgr2=0x7f710006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:01.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.050+0000 7f7123bb0640 1 --2- 172.21.15.32:0/3136059474 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f710006be30 0x7f710006e2f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f711c072440 tx=0x7f7118008040 comp rx=0 tx=0).stop 2024-04-24T22:59:01.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.050+0000 7f7123bb0640 1 -- 172.21.15.32:0/3136059474 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f711c0d4210 msgr2=0x7f711c0ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:01.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.050+0000 7f7123bb0640 1 --2- 172.21.15.32:0/3136059474 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f711c0d4210 0x7f711c0ce240 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f710c00c400 tx=0x7f710c00c8c0 comp rx=0 tx=0).stop 2024-04-24T22:59:01.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.050+0000 7f7123bb0640 1 -- 172.21.15.32:0/3136059474 shutdown_connections 2024-04-24T22:59:01.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.050+0000 7f7123bb0640 1 --2- 172.21.15.32:0/3136059474 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f710006be30 0x7f710006e2f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:01.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.050+0000 7f7123bb0640 1 --2- 172.21.15.32:0/3136059474 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f711c0d4210 0x7f711c0ce240 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:01.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.050+0000 7f7123bb0640 1 --2- 172.21.15.32:0/3136059474 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f711c15e9b0 0x7f711c0ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:01.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.050+0000 7f7123bb0640 1 --2- 172.21.15.32:0/3136059474 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f711c06b820 0x7f711c153d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:01.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.050+0000 7f7123bb0640 1 -- 172.21.15.32:0/3136059474 >> 172.21.15.32:0/3136059474 conn(0x7f711c0d11d0 msgr2=0x7f711c0b78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:01.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.050+0000 7f7123bb0640 1 -- 172.21.15.32:0/3136059474 shutdown_connections 2024-04-24T22:59:01.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.050+0000 7f7123bb0640 1 -- 172.21.15.32:0/3136059474 wait complete. 2024-04-24T22:59:01.063 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T22:59:01.063 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T22:59:01.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.164+0000 7fa1de6b3640 1 Processor -- start 2024-04-24T22:59:01.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.164+0000 7fa1de6b3640 1 -- start start 2024-04-24T22:59:01.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.165+0000 7fa1de6b3640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa1d815e9b0 0x7fa1d80d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:01.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.165+0000 7fa1de6b3640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa1d80d4210 0x7fa1d806b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:01.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.165+0000 7fa1de6b3640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1d80d3170 con 0x7fa1d80ba0e0 2024-04-24T22:59:01.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.165+0000 7fa1de6b3640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fa1d80b8900 con 0x7fa1d815e9b0 2024-04-24T22:59:01.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.165+0000 7fa1de6b3640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fa1d80d3450 con 0x7fa1d80d4210 2024-04-24T22:59:01.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.165+0000 7fa1d7fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa1d815e9b0 0x7fa1d80d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:01.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.165+0000 7fa1dcc50640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa1d80ba0e0 0x7fa1d8071730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44788/0 (socket says 172.21.15.32:44788) 2024-04-24T22:59:01.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.165+0000 7fa1d7fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa1d815e9b0 0x7fa1d80d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44490/0 (socket says 172.21.15.32:44490) 2024-04-24T22:59:01.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.165+0000 7fa1dcc50640 1 -- 172.21.15.32:0/3329717247 learned_addr learned my addr 172.21.15.32:0/3329717247 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:01.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.165+0000 7fa1d77fe640 1 --2- 172.21.15.32:0/3329717247 >> v2:172.21.15.121:3300/0 conn(0x7fa1d80d4210 0x7fa1d806b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:01.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.166+0000 7fa1d6ffd640 1 -- 172.21.15.32:0/3329717247 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3236463658 0 0) 0x7fa1d80d3170 con 0x7fa1d80ba0e0 2024-04-24T22:59:01.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.166+0000 7fa1d6ffd640 1 -- 172.21.15.32:0/3329717247 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa1c4003520 con 0x7fa1d80ba0e0 2024-04-24T22:59:01.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.166+0000 7fa1d7fff640 1 -- 172.21.15.32:0/3329717247 >> v2:172.21.15.121:3300/0 conn(0x7fa1d80d4210 msgr2=0x7fa1d806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:01.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.166+0000 7fa1d7fff640 1 --2- 172.21.15.32:0/3329717247 >> v2:172.21.15.121:3300/0 conn(0x7fa1d80d4210 0x7fa1d806b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:01.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.166+0000 7fa1d7fff640 1 -- 172.21.15.32:0/3329717247 >> v1:172.21.15.32:6789/0 conn(0x7fa1d80ba0e0 legacy=0x7fa1d8071730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:01.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.166+0000 7fa1d7fff640 1 -- 172.21.15.32:0/3329717247 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1d80d3730 con 0x7fa1d815e9b0 2024-04-24T22:59:01.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.166+0000 7fa1d7fff640 1 --2- 172.21.15.32:0/3329717247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1d815e9b0 0x7fa1d80d2b90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa1d80730f0 tx=0x7fa1c002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76815ddef4700143 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:01.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.166+0000 7fa1d6ffd640 1 -- 172.21.15.32:0/3329717247 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c0037070 con 0x7fa1d815e9b0 2024-04-24T22:59:01.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.166+0000 7fa1d6ffd640 1 -- 172.21.15.32:0/3329717247 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1c0032440 con 0x7fa1d815e9b0 2024-04-24T22:59:01.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.167+0000 7fa1d6ffd640 1 -- 172.21.15.32:0/3329717247 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c002ae60 con 0x7fa1d815e9b0 2024-04-24T22:59:01.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.167+0000 7fa1de6b3640 1 -- 172.21.15.32:0/3329717247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1d815e9b0 msgr2=0x7fa1d80d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:01.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.167+0000 7fa1de6b3640 1 --2- 172.21.15.32:0/3329717247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1d815e9b0 0x7fa1d80d2b90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa1d80730f0 tx=0x7fa1c002a110 comp rx=0 tx=0).stop 2024-04-24T22:59:01.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.167+0000 7fa1de6b3640 1 -- 172.21.15.32:0/3329717247 shutdown_connections 2024-04-24T22:59:01.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.167+0000 7fa1de6b3640 1 --2- 172.21.15.32:0/3329717247 >> v2:172.21.15.121:3300/0 conn(0x7fa1d80d4210 0x7fa1d806b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:01.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.167+0000 7fa1de6b3640 1 --2- 172.21.15.32:0/3329717247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1d815e9b0 0x7fa1d80d2b90 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:01.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.167+0000 7fa1de6b3640 1 -- 172.21.15.32:0/3329717247 >> 172.21.15.32:0/3329717247 conn(0x7fa1d80d11d0 msgr2=0x7fa1d80d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:01.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.167+0000 7fa1de6b3640 1 -- 172.21.15.32:0/3329717247 shutdown_connections 2024-04-24T22:59:01.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.167+0000 7fa1de6b3640 1 -- 172.21.15.32:0/3329717247 wait complete. 2024-04-24T22:59:01.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.168+0000 7fa1de6b3640 1 Processor -- start 2024-04-24T22:59:01.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.168+0000 7fa1de6b3640 1 -- start start 2024-04-24T22:59:01.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.169+0000 7fa1de6b3640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa1d80ba0e0 0x7fa1d8150060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:01.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.169+0000 7fa1de6b3640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1d80d4210 0x7fa1d81564b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:01.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.169+0000 7fa1de6b3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1d815e9b0 0x7fa1d81569f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:01.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.169+0000 7fa1de6b3640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d80d2400 con 0x7fa1d815e9b0 2024-04-24T22:59:01.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.169+0000 7fa1de6b3640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d80d3cf0 con 0x7fa1d80d4210 2024-04-24T22:59:01.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.169+0000 7fa1de6b3640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa1d806b650 con 0x7fa1d80ba0e0 2024-04-24T22:59:01.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.169+0000 7fa1dcc50640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1d815e9b0 0x7fa1d81569f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:01.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.169+0000 7fa1dcc50640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1d815e9b0 0x7fa1d81569f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44496/0 (socket says 172.21.15.32:44496) 2024-04-24T22:59:01.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.169+0000 7fa1dcc50640 1 -- 172.21.15.32:0/4136344471 learned_addr learned my addr 172.21.15.32:0/4136344471 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:01.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.169+0000 7fa1d7fff640 1 --2- 172.21.15.32:0/4136344471 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa1d80ba0e0 0x7fa1d8150060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:01.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.169+0000 7fa1d77fe640 1 --2- 172.21.15.32:0/4136344471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1d80d4210 0x7fa1d81564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:01.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.170+0000 7fa1dcc50640 1 -- 172.21.15.32:0/4136344471 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa1d80ba0e0 msgr2=0x7fa1d8150060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:01.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.170+0000 7fa1dcc50640 1 --2- 172.21.15.32:0/4136344471 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa1d80ba0e0 0x7fa1d8150060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:01.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.170+0000 7fa1dcc50640 1 -- 172.21.15.32:0/4136344471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1d80d4210 msgr2=0x7fa1d81564b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:01.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.170+0000 7fa1dcc50640 1 --2- 172.21.15.32:0/4136344471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1d80d4210 0x7fa1d81564b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:01.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.170+0000 7fa1dcc50640 1 -- 172.21.15.32:0/4136344471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1d81570c0 con 0x7fa1d815e9b0 2024-04-24T22:59:01.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.170+0000 7fa1d77fe640 1 --2- 172.21.15.32:0/4136344471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1d80d4210 0x7fa1d81564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:01.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.170+0000 7fa1dcc50640 1 --2- 172.21.15.32:0/4136344471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1d815e9b0 0x7fa1d81569f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa1d000cb40 tx=0x7fa1d000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:01.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.170+0000 7fa1d4ff9640 1 -- 172.21.15.32:0/4136344471 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d0017070 con 0x7fa1d815e9b0 2024-04-24T22:59:01.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.170+0000 7fa1d4ff9640 1 -- 172.21.15.32:0/4136344471 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1d000aca0 con 0x7fa1d815e9b0 2024-04-24T22:59:01.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.170+0000 7fa1de6b3640 1 -- 172.21.15.32:0/4136344471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1d81573a0 con 0x7fa1d815e9b0 2024-04-24T22:59:01.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.170+0000 7fa1d4ff9640 1 -- 172.21.15.32:0/4136344471 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d0012620 con 0x7fa1d815e9b0 2024-04-24T22:59:01.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.170+0000 7fa1de6b3640 1 -- 172.21.15.32:0/4136344471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1d8157960 con 0x7fa1d815e9b0 2024-04-24T22:59:01.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.171+0000 7fa1de6b3640 1 -- 172.21.15.32:0/4136344471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1d8068430 con 0x7fa1d815e9b0 2024-04-24T22:59:01.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.171+0000 7fa1d4ff9640 1 -- 172.21.15.32:0/4136344471 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa1d001a070 con 0x7fa1d815e9b0 2024-04-24T22:59:01.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.172+0000 7fa1d4ff9640 1 --2- 172.21.15.32:0/4136344471 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa1b806be30 0x7fa1b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:01.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.172+0000 7fa1d4ff9640 1 -- 172.21.15.32:0/4136344471 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fa1d008a8a0 con 0x7fa1d815e9b0 2024-04-24T22:59:01.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.172+0000 7fa1d7fff640 1 --2- 172.21.15.32:0/4136344471 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa1b806be30 0x7fa1b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:01.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.173+0000 7fa1d7fff640 1 --2- 172.21.15.32:0/4136344471 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa1b806be30 0x7fa1b806e2f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0034000 tx=0x7fa1c002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:01.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.176+0000 7fa1d4ff9640 1 -- 172.21.15.32:0/4136344471 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa1d0058f50 con 0x7fa1d815e9b0 2024-04-24T22:59:01.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.358+0000 7fa1de6b3640 1 -- 172.21.15.32:0/4136344471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa1d815bd10 con 0x7fa1d815e9b0 2024-04-24T22:59:01.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.358+0000 7fa1d4ff9640 1 -- 172.21.15.32:0/4136344471 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) ==== 93+0+10 (secure 0 0 0) 0x7fa1d005d250 con 0x7fa1d815e9b0 2024-04-24T22:59:01.360 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T22:59:01.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.360+0000 7fa1de6b3640 1 -- 172.21.15.32:0/4136344471 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa1b806be30 msgr2=0x7fa1b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:01.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.360+0000 7fa1de6b3640 1 --2- 172.21.15.32:0/4136344471 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa1b806be30 0x7fa1b806e2f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0034000 tx=0x7fa1c002e000 comp rx=0 tx=0).stop 2024-04-24T22:59:01.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.360+0000 7fa1de6b3640 1 -- 172.21.15.32:0/4136344471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1d815e9b0 msgr2=0x7fa1d81569f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:01.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.360+0000 7fa1de6b3640 1 --2- 172.21.15.32:0/4136344471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1d815e9b0 0x7fa1d81569f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa1d000cb40 tx=0x7fa1d000a060 comp rx=0 tx=0).stop 2024-04-24T22:59:01.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.360+0000 7fa1de6b3640 1 -- 172.21.15.32:0/4136344471 shutdown_connections 2024-04-24T22:59:01.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.360+0000 7fa1de6b3640 1 --2- 172.21.15.32:0/4136344471 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa1b806be30 0x7fa1b806e2f0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:01.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.360+0000 7fa1de6b3640 1 --2- 172.21.15.32:0/4136344471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1d815e9b0 0x7fa1d81569f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:01.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.360+0000 7fa1de6b3640 1 --2- 172.21.15.32:0/4136344471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1d80d4210 0x7fa1d81564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:01.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.360+0000 7fa1de6b3640 1 --2- 172.21.15.32:0/4136344471 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa1d80ba0e0 0x7fa1d8150060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:01.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.360+0000 7fa1de6b3640 1 -- 172.21.15.32:0/4136344471 >> 172.21.15.32:0/4136344471 conn(0x7fa1d80d11d0 msgr2=0x7fa1d815bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:01.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.360+0000 7fa1de6b3640 1 -- 172.21.15.32:0/4136344471 shutdown_connections 2024-04-24T22:59:01.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.360+0000 7fa1de6b3640 1 -- 172.21.15.32:0/4136344471 wait complete. 2024-04-24T22:59:01.373 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-24T22:59:01.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.473+0000 7fb923542640 1 Processor -- start 2024-04-24T22:59:01.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.473+0000 7fb923542640 1 -- start start 2024-04-24T22:59:01.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.474+0000 7fb923542640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb91c15fd60 0x7fb91c06c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:01.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.474+0000 7fb923542640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb91c0b46c0 con 0x7fb91c06cb30 2024-04-24T22:59:01.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.474+0000 7fb923542640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb91c0b49a0 con 0x7fb91c15b480 2024-04-24T22:59:01.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.474+0000 7fb923542640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fb91c0b4ff0 con 0x7fb91c15fd60 2024-04-24T22:59:01.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.474+0000 7fb920add640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb91c15fd60 0x7fb91c06c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:01.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.474+0000 7fb921adf640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb91c06cb30 0x7fb91c0cdf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44798/0 (socket says 172.21.15.32:44798) 2024-04-24T22:59:01.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.474+0000 7fb921adf640 1 -- 172.21.15.32:0/3217208778 learned_addr learned my addr 172.21.15.32:0/3217208778 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:01.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.474+0000 7fb920add640 1 -- 172.21.15.32:0/3217208778 >> v1:172.21.15.121:6789/0 conn(0x7fb91c15b480 legacy=0x7fb91c0b5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:01.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.474+0000 7fb920add640 1 -- 172.21.15.32:0/3217208778 >> v1:172.21.15.32:6789/0 conn(0x7fb91c06cb30 legacy=0x7fb91c0cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:01.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.474+0000 7fb920add640 1 -- 172.21.15.32:0/3217208778 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb91c0b4c80 con 0x7fb91c15fd60 2024-04-24T22:59:01.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.475+0000 7fb920add640 1 --2- 172.21.15.32:0/3217208778 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb91c15fd60 0x7fb91c06c5f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb90c0097a0 tx=0x7fb90c02c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd86c261de6fb9a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:01.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.475+0000 7fb913fff640 1 -- 172.21.15.32:0/3217208778 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90c038070 con 0x7fb91c15fd60 2024-04-24T22:59:01.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.475+0000 7fb913fff640 1 -- 172.21.15.32:0/3217208778 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb90c031710 con 0x7fb91c15fd60 2024-04-24T22:59:01.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.475+0000 7fb913fff640 1 -- 172.21.15.32:0/3217208778 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90c031ae0 con 0x7fb91c15fd60 2024-04-24T22:59:01.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.475+0000 7fb923542640 1 -- 172.21.15.32:0/3217208778 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb91c15fd60 msgr2=0x7fb91c06c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:01.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.475+0000 7fb923542640 1 --2- 172.21.15.32:0/3217208778 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb91c15fd60 0x7fb91c06c5f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb90c0097a0 tx=0x7fb90c02c440 comp rx=0 tx=0).stop 2024-04-24T22:59:01.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.475+0000 7fb923542640 1 -- 172.21.15.32:0/3217208778 shutdown_connections 2024-04-24T22:59:01.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.475+0000 7fb923542640 1 --2- 172.21.15.32:0/3217208778 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb91c15fd60 0x7fb91c06c5f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:01.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.475+0000 7fb923542640 1 -- 172.21.15.32:0/3217208778 >> 172.21.15.32:0/3217208778 conn(0x7fb91c06b110 msgr2=0x7fb91c06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:01.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.475+0000 7fb923542640 1 -- 172.21.15.32:0/3217208778 shutdown_connections 2024-04-24T22:59:01.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.476+0000 7fb923542640 1 -- 172.21.15.32:0/3217208778 wait complete. 2024-04-24T22:59:01.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.477+0000 7fb923542640 1 Processor -- start 2024-04-24T22:59:01.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.477+0000 7fb923542640 1 -- start start 2024-04-24T22:59:01.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.477+0000 7fb923542640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb91c06cb30 0x7fb91c146c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:01.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.477+0000 7fb923542640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb91c15b480 0x7fb91c070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:01.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.477+0000 7fb923542640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb91c15fd60 0x7fb91c072580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:01.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.477+0000 7fb923542640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb91c15b860 con 0x7fb91c06cb30 2024-04-24T22:59:01.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.477+0000 7fb923542640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb91c0ce690 con 0x7fb91c15b480 2024-04-24T22:59:01.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.477+0000 7fb923542640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fb91c0714e0 con 0x7fb91c15fd60 2024-04-24T22:59:01.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.478+0000 7fb9212de640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb91c06cb30 0x7fb91c146c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:01.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.478+0000 7fb9212de640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb91c06cb30 0x7fb91c146c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44500/0 (socket says 172.21.15.32:44500) 2024-04-24T22:59:01.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.478+0000 7fb9212de640 1 -- 172.21.15.32:0/530337927 learned_addr learned my addr 172.21.15.32:0/530337927 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:01.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.478+0000 7fb921adf640 1 --2- 172.21.15.32:0/530337927 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb91c15fd60 0x7fb91c072580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:01.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.478+0000 7fb920add640 1 --2- 172.21.15.32:0/530337927 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb91c15b480 0x7fb91c070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:01.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.478+0000 7fb9212de640 1 -- 172.21.15.32:0/530337927 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb91c15fd60 msgr2=0x7fb91c072580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:01.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.478+0000 7fb9212de640 1 --2- 172.21.15.32:0/530337927 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb91c15fd60 0x7fb91c072580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:01.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.478+0000 7fb9212de640 1 -- 172.21.15.32:0/530337927 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb91c15b480 msgr2=0x7fb91c070b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:01.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.478+0000 7fb9212de640 1 --2- 172.21.15.32:0/530337927 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb91c15b480 0x7fb91c070b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:01.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.478+0000 7fb9212de640 1 -- 172.21.15.32:0/530337927 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb91c0717c0 con 0x7fb91c06cb30 2024-04-24T22:59:01.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.478+0000 7fb921adf640 1 --2- 172.21.15.32:0/530337927 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb91c15fd60 0x7fb91c072580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:59:01.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.478+0000 7fb9212de640 1 --2- 172.21.15.32:0/530337927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb91c06cb30 0x7fb91c146c60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb91800c430 tx=0x7fb91800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:01.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.478+0000 7fb911ffb640 1 -- 172.21.15.32:0/530337927 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9180052d0 con 0x7fb91c06cb30 2024-04-24T22:59:01.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.478+0000 7fb911ffb640 1 -- 172.21.15.32:0/530337927 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb918005490 con 0x7fb91c06cb30 2024-04-24T22:59:01.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.478+0000 7fb923542640 1 -- 172.21.15.32:0/530337927 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb91c071aa0 con 0x7fb91c06cb30 2024-04-24T22:59:01.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.478+0000 7fb911ffb640 1 -- 172.21.15.32:0/530337927 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb918007260 con 0x7fb91c06cb30 2024-04-24T22:59:01.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.478+0000 7fb923542640 1 -- 172.21.15.32:0/530337927 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb91c072060 con 0x7fb91c06cb30 2024-04-24T22:59:01.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.479+0000 7fb923542640 1 -- 172.21.15.32:0/530337927 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb91c068430 con 0x7fb91c06cb30 2024-04-24T22:59:01.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.480+0000 7fb911ffb640 1 -- 172.21.15.32:0/530337927 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb918007400 con 0x7fb91c06cb30 2024-04-24T22:59:01.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.480+0000 7fb911ffb640 1 --2- 172.21.15.32:0/530337927 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb90006be30 0x7fb90006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:01.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.480+0000 7fb911ffb640 1 -- 172.21.15.32:0/530337927 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fb91808a0c0 con 0x7fb91c06cb30 2024-04-24T22:59:01.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.480+0000 7fb920add640 1 --2- 172.21.15.32:0/530337927 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb90006be30 0x7fb90006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:01.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.481+0000 7fb920add640 1 --2- 172.21.15.32:0/530337927 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb90006be30 0x7fb90006e2f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb90c035000 tx=0x7fb90c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:01.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.484+0000 7fb911ffb640 1 -- 172.21.15.32:0/530337927 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb918058d30 con 0x7fb91c06cb30 2024-04-24T22:59:01.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.673+0000 7fb923542640 1 -- 172.21.15.32:0/530337927 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fb91c06c000 con 0x7fb91c06cb30 2024-04-24T22:59:01.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.673+0000 7fb911ffb640 1 -- 172.21.15.32:0/530337927 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) ==== 84+0+65 (secure 0 0 0) 0x7fb918058630 con 0x7fb91c06cb30 2024-04-24T22:59:01.675 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:59:01.675 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2024-04-24T22:59:01.675 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T22:59:01.676 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2024-04-24T22:59:01.676 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": ".mgr" 2024-04-24T22:59:01.676 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T22:59:01.676 INFO:teuthology.orchestra.run.smithi032.stdout:] 2024-04-24T22:59:01.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.675+0000 7fb923542640 1 -- 172.21.15.32:0/530337927 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb90006be30 msgr2=0x7fb90006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:01.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.675+0000 7fb923542640 1 --2- 172.21.15.32:0/530337927 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb90006be30 0x7fb90006e2f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb90c035000 tx=0x7fb90c008030 comp rx=0 tx=0).stop 2024-04-24T22:59:01.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.675+0000 7fb923542640 1 -- 172.21.15.32:0/530337927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb91c06cb30 msgr2=0x7fb91c146c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:01.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.675+0000 7fb923542640 1 --2- 172.21.15.32:0/530337927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb91c06cb30 0x7fb91c146c60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb91800c430 tx=0x7fb91800c8f0 comp rx=0 tx=0).stop 2024-04-24T22:59:01.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.675+0000 7fb923542640 1 -- 172.21.15.32:0/530337927 shutdown_connections 2024-04-24T22:59:01.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.675+0000 7fb923542640 1 --2- 172.21.15.32:0/530337927 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb90006be30 0x7fb90006e2f0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:01.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.675+0000 7fb923542640 1 --2- 172.21.15.32:0/530337927 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb91c15fd60 0x7fb91c072580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:01.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.675+0000 7fb923542640 1 --2- 172.21.15.32:0/530337927 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb91c15b480 0x7fb91c070b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:01.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.675+0000 7fb923542640 1 --2- 172.21.15.32:0/530337927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb91c06cb30 0x7fb91c146c60 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:01.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.675+0000 7fb923542640 1 -- 172.21.15.32:0/530337927 >> 172.21.15.32:0/530337927 conn(0x7fb91c06b110 msgr2=0x7fb91c0cd410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:01.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.675+0000 7fb923542640 1 -- 172.21.15.32:0/530337927 shutdown_connections 2024-04-24T22:59:01.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.675+0000 7fb923542640 1 -- 172.21.15.32:0/530337927 wait complete. 2024-04-24T22:59:01.687 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-24T22:59:01.688 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-24T22:59:01.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.790+0000 7f57e36d4640 1 Processor -- start 2024-04-24T22:59:01.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.790+0000 7f57e36d4640 1 -- start start 2024-04-24T22:59:01.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.791+0000 7f57e36d4640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f57dc15e6e0 0x7f57dc0b9f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:01.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.791+0000 7f57e36d4640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f57dc0cfa40 0x7f57dc0d29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:01.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.791+0000 7f57e36d4640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57dc0b6e60 con 0x7f57dc0b3870 2024-04-24T22:59:01.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.791+0000 7f57e36d4640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f57dc0d12e0 con 0x7f57dc0cfa40 2024-04-24T22:59:01.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.791+0000 7f57e36d4640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f57dc0d15c0 con 0x7f57dc15e6e0 2024-04-24T22:59:01.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.791+0000 7f57e1c71640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f57dc0cfa40 0x7f57dc0d29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:01.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.791+0000 7f57e1c71640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f57dc0cfa40 0x7f57dc0d29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44510/0 (socket says 172.21.15.32:44510) 2024-04-24T22:59:01.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.791+0000 7f57e1c71640 1 -- 172.21.15.32:0/2767703509 learned_addr learned my addr 172.21.15.32:0/2767703509 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:01.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.791+0000 7f57e1470640 1 --2- 172.21.15.32:0/2767703509 >> v2:172.21.15.121:3300/0 conn(0x7f57dc15e6e0 0x7f57dc0b9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:01.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.791+0000 7f57e1c71640 1 -- 172.21.15.32:0/2767703509 >> v2:172.21.15.121:3300/0 conn(0x7f57dc15e6e0 msgr2=0x7f57dc0b9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:01.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.791+0000 7f57e1c71640 1 --2- 172.21.15.32:0/2767703509 >> v2:172.21.15.121:3300/0 conn(0x7f57dc15e6e0 0x7f57dc0b9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:01.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.791+0000 7f57e1c71640 1 -- 172.21.15.32:0/2767703509 >> v1:172.21.15.121:6789/0 conn(0x7f57dc0b3870 legacy=0x7f57dc0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:01.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.791+0000 7f57e1c71640 1 -- 172.21.15.32:0/2767703509 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57dc0d18a0 con 0x7f57dc0cfa40 2024-04-24T22:59:01.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.791+0000 7f57e1470640 1 --2- 172.21.15.32:0/2767703509 >> v2:172.21.15.121:3300/0 conn(0x7f57dc15e6e0 0x7f57dc0b9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:01.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.792+0000 7f57e1c71640 1 --2- 172.21.15.32:0/2767703509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57dc0cfa40 0x7f57dc0d29c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f57dc0d5650 tx=0x7f57d402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5883d4240bab1fcc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:01.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.792+0000 7f57d3fff640 1 -- 172.21.15.32:0/2767703509 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d4036070 con 0x7f57dc0cfa40 2024-04-24T22:59:01.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.792+0000 7f57d3fff640 1 -- 172.21.15.32:0/2767703509 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57d4031600 con 0x7f57dc0cfa40 2024-04-24T22:59:01.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.792+0000 7f57d3fff640 1 -- 172.21.15.32:0/2767703509 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d40319d0 con 0x7f57dc0cfa40 2024-04-24T22:59:01.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.793+0000 7f57e36d4640 1 -- 172.21.15.32:0/2767703509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57dc0cfa40 msgr2=0x7f57dc0d29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:01.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.793+0000 7f57e36d4640 1 --2- 172.21.15.32:0/2767703509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57dc0cfa40 0x7f57dc0d29c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f57dc0d5650 tx=0x7f57d402a730 comp rx=0 tx=0).stop 2024-04-24T22:59:01.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.793+0000 7f57e36d4640 1 -- 172.21.15.32:0/2767703509 shutdown_connections 2024-04-24T22:59:01.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.793+0000 7f57e36d4640 1 --2- 172.21.15.32:0/2767703509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57dc0cfa40 0x7f57dc0d29c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:01.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.793+0000 7f57e36d4640 1 --2- 172.21.15.32:0/2767703509 >> v2:172.21.15.121:3300/0 conn(0x7f57dc15e6e0 0x7f57dc0b9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:01.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.793+0000 7f57e36d4640 1 -- 172.21.15.32:0/2767703509 >> 172.21.15.32:0/2767703509 conn(0x7f57dc071910 msgr2=0x7f57dc071d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:01.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.793+0000 7f57e36d4640 1 -- 172.21.15.32:0/2767703509 shutdown_connections 2024-04-24T22:59:01.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.793+0000 7f57e36d4640 1 -- 172.21.15.32:0/2767703509 wait complete. 2024-04-24T22:59:01.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.794+0000 7f57e36d4640 1 Processor -- start 2024-04-24T22:59:01.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.794+0000 7f57e36d4640 1 -- start start 2024-04-24T22:59:01.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.795+0000 7f57e36d4640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57dc0b3870 0x7f57dc15abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:01.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.795+0000 7f57e36d4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57dc0cfa40 0x7f57dc0ca090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:01.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.795+0000 7f57e36d4640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f57dc15e6e0 0x7f57dc160e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:01.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.795+0000 7f57e36d4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f57dc0d0fd0 con 0x7f57dc0cfa40 2024-04-24T22:59:01.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.795+0000 7f57e36d4640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f57dc0ba450 con 0x7f57dc0b3870 2024-04-24T22:59:01.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.795+0000 7f57e36d4640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f57dc15eac0 con 0x7f57dc15e6e0 2024-04-24T22:59:01.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.795+0000 7f57e0c6f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57dc0cfa40 0x7f57dc0ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:01.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.795+0000 7f57e0c6f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57dc0cfa40 0x7f57dc0ca090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44524/0 (socket says 172.21.15.32:44524) 2024-04-24T22:59:01.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.795+0000 7f57e0c6f640 1 -- 172.21.15.32:0/1541392734 learned_addr learned my addr 172.21.15.32:0/1541392734 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:01.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.795+0000 7f57e1470640 1 --2- 172.21.15.32:0/1541392734 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57dc0b3870 0x7f57dc15abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:01.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.795+0000 7f57e1c71640 1 --2- 172.21.15.32:0/1541392734 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f57dc15e6e0 0x7f57dc160e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:01.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.795+0000 7f57e0c6f640 1 -- 172.21.15.32:0/1541392734 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f57dc15e6e0 msgr2=0x7f57dc160e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:01.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.795+0000 7f57e0c6f640 1 --2- 172.21.15.32:0/1541392734 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f57dc15e6e0 0x7f57dc160e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:01.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.795+0000 7f57e0c6f640 1 -- 172.21.15.32:0/1541392734 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57dc0b3870 msgr2=0x7f57dc15abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:01.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.795+0000 7f57e0c6f640 1 --2- 172.21.15.32:0/1541392734 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57dc0b3870 0x7f57dc15abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:01.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.795+0000 7f57e0c6f640 1 -- 172.21.15.32:0/1541392734 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57dc158350 con 0x7f57dc0cfa40 2024-04-24T22:59:01.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.795+0000 7f57e1470640 1 --2- 172.21.15.32:0/1541392734 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57dc0b3870 0x7f57dc15abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:01.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.795+0000 7f57e0c6f640 1 --2- 172.21.15.32:0/1541392734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57dc0cfa40 0x7f57dc0ca090 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f57cc00c450 tx=0x7f57cc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:01.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.795+0000 7f57d1ffb640 1 -- 172.21.15.32:0/1541392734 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc0052d0 con 0x7f57dc0cfa40 2024-04-24T22:59:01.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.796+0000 7f57d1ffb640 1 -- 172.21.15.32:0/1541392734 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57cc005470 con 0x7f57dc0cfa40 2024-04-24T22:59:01.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.796+0000 7f57e36d4640 1 -- 172.21.15.32:0/1541392734 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57dc155dd0 con 0x7f57dc0cfa40 2024-04-24T22:59:01.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.796+0000 7f57d1ffb640 1 -- 172.21.15.32:0/1541392734 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc012870 con 0x7f57dc0cfa40 2024-04-24T22:59:01.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.796+0000 7f57e36d4640 1 -- 172.21.15.32:0/1541392734 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57dc156390 con 0x7f57dc0cfa40 2024-04-24T22:59:01.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.797+0000 7f57e36d4640 1 -- 172.21.15.32:0/1541392734 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57dc15cac0 con 0x7f57dc0cfa40 2024-04-24T22:59:01.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.797+0000 7f57d1ffb640 1 -- 172.21.15.32:0/1541392734 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f57cc012a10 con 0x7f57dc0cfa40 2024-04-24T22:59:01.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.797+0000 7f57d1ffb640 1 --2- 172.21.15.32:0/1541392734 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f57c006be60 0x7f57c006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:01.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.798+0000 7f57d1ffb640 1 -- 172.21.15.32:0/1541392734 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f57cc08a6d0 con 0x7f57dc0cfa40 2024-04-24T22:59:01.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.798+0000 7f57e1470640 1 --2- 172.21.15.32:0/1541392734 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f57c006be60 0x7f57c006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:01.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.798+0000 7f57e1470640 1 --2- 172.21.15.32:0/1541392734 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f57c006be60 0x7f57c006e320 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f57d8006fd0 tx=0x7f57d8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:01.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.802+0000 7f57d1ffb640 1 -- 172.21.15.32:0/1541392734 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f57dc15cac0 con 0x7f57dc0cfa40 2024-04-24T22:59:01.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:01.985+0000 7f57e36d4640 1 -- 172.21.15.32:0/1541392734 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f57dc068430 con 0x7f57dc0cfa40 2024-04-24T22:59:02.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.494+0000 7f57d1ffb640 1 -- 172.21.15.32:0/1541392734 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) ==== 138+0+0 (secure 0 0 0) 0x7f57cc0593d0 con 0x7f57dc0cfa40 2024-04-24T22:59:02.495 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2024-04-24T22:59:02.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.497+0000 7f57e36d4640 1 -- 172.21.15.32:0/1541392734 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f57c006be60 msgr2=0x7f57c006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:02.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.497+0000 7f57e36d4640 1 --2- 172.21.15.32:0/1541392734 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f57c006be60 0x7f57c006e320 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f57d8006fd0 tx=0x7f57d8008040 comp rx=0 tx=0).stop 2024-04-24T22:59:02.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.497+0000 7f57e36d4640 1 -- 172.21.15.32:0/1541392734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57dc0cfa40 msgr2=0x7f57dc0ca090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:02.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.497+0000 7f57e36d4640 1 --2- 172.21.15.32:0/1541392734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57dc0cfa40 0x7f57dc0ca090 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f57cc00c450 tx=0x7f57cc00c910 comp rx=0 tx=0).stop 2024-04-24T22:59:02.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.497+0000 7f57e36d4640 1 -- 172.21.15.32:0/1541392734 shutdown_connections 2024-04-24T22:59:02.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.497+0000 7f57e36d4640 1 --2- 172.21.15.32:0/1541392734 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f57c006be60 0x7f57c006e320 secure :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f57d8006fd0 tx=0x7f57d8008040 comp rx=0 tx=0).stop 2024-04-24T22:59:02.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.497+0000 7f57e36d4640 1 --2- 172.21.15.32:0/1541392734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57dc0cfa40 0x7f57dc0ca090 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:02.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.497+0000 7f57e36d4640 1 --2- 172.21.15.32:0/1541392734 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f57dc15e6e0 0x7f57dc160e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:02.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.497+0000 7f57e36d4640 1 --2- 172.21.15.32:0/1541392734 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57dc0b3870 0x7f57dc15abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:02.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.497+0000 7f57e36d4640 1 -- 172.21.15.32:0/1541392734 >> 172.21.15.32:0/1541392734 conn(0x7f57dc071910 msgr2=0x7f57dc0d29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:02.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.497+0000 7f57e36d4640 1 -- 172.21.15.32:0/1541392734 shutdown_connections 2024-04-24T22:59:02.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.498+0000 7f57e36d4640 1 -- 172.21.15.32:0/1541392734 wait complete. 2024-04-24T22:59:02.513 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-24T22:59:02.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.621+0000 7fd0ac415640 1 Processor -- start 2024-04-24T22:59:02.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.621+0000 7fd0ac415640 1 -- start start 2024-04-24T22:59:02.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.622+0000 7fd0ac415640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd0a40d4210 0x7fd0a40d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:02.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.622+0000 7fd0ac415640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0a40d3170 con 0x7fd0a406b820 2024-04-24T22:59:02.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.622+0000 7fd0ac415640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0a40d3450 con 0x7fd0a415e9b0 2024-04-24T22:59:02.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.622+0000 7fd0ac415640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fd0a40d3730 con 0x7fd0a40d4210 2024-04-24T22:59:02.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.622+0000 7fd0a99b0640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd0a406b820 0x7fd0a40b78f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44800/0 (socket says 172.21.15.32:44800) 2024-04-24T22:59:02.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.622+0000 7fd0a99b0640 1 -- 172.21.15.32:0/2723599384 learned_addr learned my addr 172.21.15.32:0/2723599384 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:02.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.623+0000 7fd0aa9b2640 1 --2- 172.21.15.32:0/2723599384 >> v2:172.21.15.121:3300/0 conn(0x7fd0a40d4210 0x7fd0a40d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:02.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.623+0000 7fd0a91af640 1 -- 172.21.15.32:0/2723599384 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1518316483 0 0) 0x7fd0a40d3170 con 0x7fd0a406b820 2024-04-24T22:59:02.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.623+0000 7fd0a91af640 1 -- 172.21.15.32:0/2723599384 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd08c003520 con 0x7fd0a406b820 2024-04-24T22:59:02.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.623+0000 7fd0a91af640 1 -- 172.21.15.32:0/2723599384 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1818784096 0 0) 0x7fd0a40d3450 con 0x7fd0a415e9b0 2024-04-24T22:59:02.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.623+0000 7fd0a91af640 1 -- 172.21.15.32:0/2723599384 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd0a40d3170 con 0x7fd0a415e9b0 2024-04-24T22:59:02.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.623+0000 7fd0aa9b2640 1 -- 172.21.15.32:0/2723599384 >> v1:172.21.15.121:6789/0 conn(0x7fd0a415e9b0 legacy=0x7fd0a406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:02.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.623+0000 7fd0a91af640 1 -- 172.21.15.32:0/2723599384 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4106803750 0 0) 0x7fd08c003520 con 0x7fd0a406b820 2024-04-24T22:59:02.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.623+0000 7fd0aa9b2640 1 -- 172.21.15.32:0/2723599384 >> v1:172.21.15.32:6789/0 conn(0x7fd0a406b820 legacy=0x7fd0a40b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:02.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.623+0000 7fd0aa9b2640 1 -- 172.21.15.32:0/2723599384 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a40d3a10 con 0x7fd0a40d4210 2024-04-24T22:59:02.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.624+0000 7fd0aa9b2640 1 --2- 172.21.15.32:0/2723599384 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0a40d4210 0x7fd0a40d2b90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd09c0097a0 tx=0x7fd09c02a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e8e1ebe0ed1849 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:02.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.624+0000 7fd0a91af640 1 -- 172.21.15.32:0/2723599384 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09c037070 con 0x7fd0a40d4210 2024-04-24T22:59:02.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.624+0000 7fd0a91af640 1 -- 172.21.15.32:0/2723599384 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd09c02abc0 con 0x7fd0a40d4210 2024-04-24T22:59:02.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.624+0000 7fd0a91af640 1 -- 172.21.15.32:0/2723599384 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd098001670 con 0x7fd0a40d4210 2024-04-24T22:59:02.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.624+0000 7fd0ac415640 1 -- 172.21.15.32:0/2723599384 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0a40d4210 msgr2=0x7fd0a40d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:02.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.624+0000 7fd0ac415640 1 --2- 172.21.15.32:0/2723599384 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0a40d4210 0x7fd0a40d2b90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd09c0097a0 tx=0x7fd09c02a130 comp rx=0 tx=0).stop 2024-04-24T22:59:02.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.624+0000 7fd0ac415640 1 -- 172.21.15.32:0/2723599384 shutdown_connections 2024-04-24T22:59:02.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.624+0000 7fd0ac415640 1 --2- 172.21.15.32:0/2723599384 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0a40d4210 0x7fd0a40d2b90 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:02.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.625+0000 7fd0ac415640 1 -- 172.21.15.32:0/2723599384 >> 172.21.15.32:0/2723599384 conn(0x7fd0a40d11d0 msgr2=0x7fd0a40d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:02.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.625+0000 7fd0ac415640 1 -- 172.21.15.32:0/2723599384 shutdown_connections 2024-04-24T22:59:02.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.625+0000 7fd0ac415640 1 -- 172.21.15.32:0/2723599384 wait complete. 2024-04-24T22:59:02.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.626+0000 7fd0ac415640 1 Processor -- start 2024-04-24T22:59:02.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.626+0000 7fd0ac415640 1 -- start start 2024-04-24T22:59:02.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.626+0000 7fd0ac415640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0a406b820 0x7fd0a40caab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:02.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.626+0000 7fd0ac415640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd0a40d4210 0x7fd0a4159a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:02.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.626+0000 7fd0ac415640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0a415e9b0 0x7fd0a4159f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:02.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.626+0000 7fd0ac415640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a40ba0e0 con 0x7fd0a406b820 2024-04-24T22:59:02.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.626+0000 7fd0ac415640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a40d2400 con 0x7fd0a415e9b0 2024-04-24T22:59:02.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.626+0000 7fd0ac415640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd0a40ba260 con 0x7fd0a40d4210 2024-04-24T22:59:02.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.627+0000 7fd0aa1b1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0a406b820 0x7fd0a40caab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:02.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.627+0000 7fd0aa1b1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0a406b820 0x7fd0a40caab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44536/0 (socket says 172.21.15.32:44536) 2024-04-24T22:59:02.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.627+0000 7fd0aa1b1640 1 -- 172.21.15.32:0/3792145773 learned_addr learned my addr 172.21.15.32:0/3792145773 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:02.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.627+0000 7fd0aa9b2640 1 --2- 172.21.15.32:0/3792145773 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0a415e9b0 0x7fd0a4159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:02.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.627+0000 7fd0a99b0640 1 --2- 172.21.15.32:0/3792145773 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd0a40d4210 0x7fd0a4159a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:02.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.627+0000 7fd0aa1b1640 1 -- 172.21.15.32:0/3792145773 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd0a40d4210 msgr2=0x7fd0a4159a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:02.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.627+0000 7fd0aa1b1640 1 --2- 172.21.15.32:0/3792145773 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd0a40d4210 0x7fd0a4159a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:02.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.627+0000 7fd0aa1b1640 1 -- 172.21.15.32:0/3792145773 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0a415e9b0 msgr2=0x7fd0a4159f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:02.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.627+0000 7fd0aa1b1640 1 --2- 172.21.15.32:0/3792145773 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0a415e9b0 0x7fd0a4159f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:02.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.627+0000 7fd0aa1b1640 1 -- 172.21.15.32:0/3792145773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a41e60d0 con 0x7fd0a406b820 2024-04-24T22:59:02.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.627+0000 7fd0aa1b1640 1 --2- 172.21.15.32:0/3792145773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0a406b820 0x7fd0a40caab0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd09400c7a0 tx=0x7fd09400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:02.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.627+0000 7fd0a2ffd640 1 -- 172.21.15.32:0/3792145773 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd094003b70 con 0x7fd0a406b820 2024-04-24T22:59:02.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.627+0000 7fd0a2ffd640 1 -- 172.21.15.32:0/3792145773 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd094003d10 con 0x7fd0a406b820 2024-04-24T22:59:02.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.627+0000 7fd0ac415640 1 -- 172.21.15.32:0/3792145773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0a40cc910 con 0x7fd0a406b820 2024-04-24T22:59:02.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.627+0000 7fd0a2ffd640 1 -- 172.21.15.32:0/3792145773 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09400aa90 con 0x7fd0a406b820 2024-04-24T22:59:02.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.627+0000 7fd0ac415640 1 -- 172.21.15.32:0/3792145773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0a40cced0 con 0x7fd0a406b820 2024-04-24T22:59:02.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.628+0000 7fd0ac415640 1 -- 172.21.15.32:0/3792145773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0a4068430 con 0x7fd0a406b820 2024-04-24T22:59:02.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.629+0000 7fd0a2ffd640 1 -- 172.21.15.32:0/3792145773 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd09400ac30 con 0x7fd0a406b820 2024-04-24T22:59:02.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.629+0000 7fd0a2ffd640 1 --2- 172.21.15.32:0/3792145773 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd09006be30 0x7fd09006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:02.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.629+0000 7fd0a2ffd640 1 -- 172.21.15.32:0/3792145773 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5227+0+0 (secure 0 0 0) 0x7fd09408ab30 con 0x7fd0a406b820 2024-04-24T22:59:02.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.629+0000 7fd0a99b0640 1 --2- 172.21.15.32:0/3792145773 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd09006be30 0x7fd09006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:02.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.630+0000 7fd0a99b0640 1 --2- 172.21.15.32:0/3792145773 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd09006be30 0x7fd09006e2f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd0a40ced70 tx=0x7fd098001610 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:02.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.634+0000 7fd0a2ffd640 1 -- 172.21.15.32:0/3792145773 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd094058fd0 con 0x7fd0a406b820 2024-04-24T22:59:02.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:02.815+0000 7fd0ac415640 1 -- 172.21.15.32:0/3792145773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fd0a415d1e0 con 0x7fd0a406b820 2024-04-24T22:59:03.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.496+0000 7fd0a2ffd640 1 -- 172.21.15.32:0/3792145773 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) ==== 170+0+0 (secure 0 0 0) 0x7fd09405d2d0 con 0x7fd0a406b820 2024-04-24T22:59:03.498 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2024-04-24T22:59:03.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.503+0000 7fd0ac415640 1 -- 172.21.15.32:0/3792145773 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd09006be30 msgr2=0x7fd09006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:03.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.503+0000 7fd0ac415640 1 --2- 172.21.15.32:0/3792145773 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd09006be30 0x7fd09006e2f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd0a40ced70 tx=0x7fd098001610 comp rx=0 tx=0).stop 2024-04-24T22:59:03.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.503+0000 7fd0ac415640 1 -- 172.21.15.32:0/3792145773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0a406b820 msgr2=0x7fd0a40caab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:03.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.503+0000 7fd0ac415640 1 --2- 172.21.15.32:0/3792145773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0a406b820 0x7fd0a40caab0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd09400c7a0 tx=0x7fd09400cc60 comp rx=0 tx=0).stop 2024-04-24T22:59:03.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.509+0000 7fd0ac415640 1 -- 172.21.15.32:0/3792145773 shutdown_connections 2024-04-24T22:59:03.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.509+0000 7fd0ac415640 1 --2- 172.21.15.32:0/3792145773 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd09006be30 0x7fd09006e2f0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:03.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.509+0000 7fd0ac415640 1 --2- 172.21.15.32:0/3792145773 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0a415e9b0 0x7fd0a4159f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:03.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.509+0000 7fd0ac415640 1 --2- 172.21.15.32:0/3792145773 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd0a40d4210 0x7fd0a4159a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:03.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.509+0000 7fd0ac415640 1 --2- 172.21.15.32:0/3792145773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0a406b820 0x7fd0a40caab0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:03.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.509+0000 7fd0ac415640 1 -- 172.21.15.32:0/3792145773 >> 172.21.15.32:0/3792145773 conn(0x7fd0a40d11d0 msgr2=0x7fd0a40d15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:03.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.513+0000 7fd0ac415640 1 -- 172.21.15.32:0/3792145773 shutdown_connections 2024-04-24T22:59:03.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.513+0000 7fd0ac415640 1 -- 172.21.15.32:0/3792145773 wait complete. 2024-04-24T22:59:03.528 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-24T22:59:03.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.636+0000 7f7c9a86b640 1 Processor -- start 2024-04-24T22:59:03.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.636+0000 7f7c9a86b640 1 -- start start 2024-04-24T22:59:03.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.636+0000 7f7c9a86b640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c9415b380 0x7f7c9406c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:03.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.636+0000 7f7c9a86b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c940b4520 con 0x7f7c940ceb40 2024-04-24T22:59:03.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.636+0000 7f7c9a86b640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c940b4800 con 0x7f7c9415cb00 2024-04-24T22:59:03.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.636+0000 7f7c9a86b640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f7c940cef90 con 0x7f7c9415b380 2024-04-24T22:59:03.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.637+0000 7f7c98e08640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7c940ceb40 0x7f7c940b55c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44816/0 (socket says 172.21.15.32:44816) 2024-04-24T22:59:03.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.637+0000 7f7c98e08640 1 -- 172.21.15.32:0/826780098 learned_addr learned my addr 172.21.15.32:0/826780098 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:03.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.637+0000 7f7c93fff640 1 --2- 172.21.15.32:0/826780098 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c9415b380 0x7f7c9406c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:03.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.637+0000 7f7c92ffd640 1 -- 172.21.15.32:0/826780098 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3153264542 0 0) 0x7f7c940b4520 con 0x7f7c940ceb40 2024-04-24T22:59:03.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.637+0000 7f7c92ffd640 1 -- 172.21.15.32:0/826780098 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c7c003520 con 0x7f7c940ceb40 2024-04-24T22:59:03.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.637+0000 7f7c92ffd640 1 -- 172.21.15.32:0/826780098 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2613102243 0 0) 0x7f7c940b4800 con 0x7f7c9415cb00 2024-04-24T22:59:03.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.637+0000 7f7c92ffd640 1 -- 172.21.15.32:0/826780098 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c940b4520 con 0x7f7c9415cb00 2024-04-24T22:59:03.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.637+0000 7f7c92ffd640 1 -- 172.21.15.32:0/826780098 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2756413765 0 0) 0x7f7c7c003520 con 0x7f7c940ceb40 2024-04-24T22:59:03.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.637+0000 7f7c92ffd640 1 -- 172.21.15.32:0/826780098 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c9415b380 msgr2=0x7f7c9406c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:03.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.637+0000 7f7c92ffd640 1 --2- 172.21.15.32:0/826780098 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c9415b380 0x7f7c9406c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:03.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.637+0000 7f7c92ffd640 1 -- 172.21.15.32:0/826780098 >> v1:172.21.15.121:6789/0 conn(0x7f7c9415cb00 legacy=0x7f7c940ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:03.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.637+0000 7f7c92ffd640 1 -- 172.21.15.32:0/826780098 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c940b4ae0 con 0x7f7c940ceb40 2024-04-24T22:59:03.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.637+0000 7f7c92ffd640 1 -- 172.21.15.32:0/826780098 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f7c8c0019f0 con 0x7f7c940ceb40 2024-04-24T22:59:03.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.637+0000 7f7c92ffd640 1 -- 172.21.15.32:0/826780098 >> v1:172.21.15.32:6789/0 conn(0x7f7c940ceb40 legacy=0x7f7c940b55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:03.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.637+0000 7f7c92ffd640 1 --2- 172.21.15.32:0/826780098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c7c003b70 0x7f7c7c003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:03.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.637+0000 7f7c92ffd640 1 --2- 172.21.15.32:0/826780098 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c7c004800 0x7f7c7c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:03.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.637+0000 7f7c92ffd640 1 --2- 172.21.15.32:0/826780098 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c7c0051c0 0x7f7c7c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:03.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.637+0000 7f7c92ffd640 1 -- 172.21.15.32:0/826780098 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7c940cef90 con 0x7f7c7c003b70 2024-04-24T22:59:03.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.637+0000 7f7c92ffd640 1 -- 172.21.15.32:0/826780098 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7c7c00c030 con 0x7f7c7c0051c0 2024-04-24T22:59:03.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.637+0000 7f7c92ffd640 1 -- 172.21.15.32:0/826780098 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f7c7c00c230 con 0x7f7c7c004800 2024-04-24T22:59:03.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.637+0000 7f7c93fff640 1 --2- 172.21.15.32:0/826780098 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c9415b380 0x7f7c9406c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:59:03.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.637+0000 7f7c98e08640 1 --2- 172.21.15.32:0/826780098 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c7c0051c0 0x7f7c7c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:03.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.637+0000 7f7c93fff640 1 --2- 172.21.15.32:0/826780098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c7c003b70 0x7f7c7c003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:03.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.637+0000 7f7c937fe640 1 --2- 172.21.15.32:0/826780098 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c7c004800 0x7f7c7c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:03.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.638+0000 7f7c93fff640 1 -- 172.21.15.32:0/826780098 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c7c004800 msgr2=0x7f7c7c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:03.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.638+0000 7f7c93fff640 1 --2- 172.21.15.32:0/826780098 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c7c004800 0x7f7c7c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:03.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.638+0000 7f7c93fff640 1 -- 172.21.15.32:0/826780098 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c7c0051c0 msgr2=0x7f7c7c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:03.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.638+0000 7f7c93fff640 1 --2- 172.21.15.32:0/826780098 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c7c0051c0 0x7f7c7c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:03.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.638+0000 7f7c93fff640 1 -- 172.21.15.32:0/826780098 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c7c003520 con 0x7f7c7c003b70 2024-04-24T22:59:03.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.638+0000 7f7c93fff640 1 --2- 172.21.15.32:0/826780098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c7c003b70 0x7f7c7c003f70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7c9406ed80 tx=0x7f7c7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41f04de28752b535 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:03.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.638+0000 7f7c92ffd640 1 -- 172.21.15.32:0/826780098 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c84036070 con 0x7f7c7c003b70 2024-04-24T22:59:03.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.639+0000 7f7c92ffd640 1 -- 172.21.15.32:0/826780098 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c84004030 con 0x7f7c7c003b70 2024-04-24T22:59:03.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.639+0000 7f7c92ffd640 1 -- 172.21.15.32:0/826780098 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c840097d0 con 0x7f7c7c003b70 2024-04-24T22:59:03.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.639+0000 7f7c9a86b640 1 -- 172.21.15.32:0/826780098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c7c003b70 msgr2=0x7f7c7c003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:03.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.639+0000 7f7c9a86b640 1 --2- 172.21.15.32:0/826780098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c7c003b70 0x7f7c7c003f70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7c9406ed80 tx=0x7f7c7c005830 comp rx=0 tx=0).stop 2024-04-24T22:59:03.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.639+0000 7f7c98e08640 1 -- 172.21.15.32:0/826780098 reap_dead start 2024-04-24T22:59:03.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.639+0000 7f7c9a86b640 1 -- 172.21.15.32:0/826780098 shutdown_connections 2024-04-24T22:59:03.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.639+0000 7f7c9a86b640 1 -- 172.21.15.32:0/826780098 >> 172.21.15.32:0/826780098 conn(0x7f7c9406b110 msgr2=0x7f7c9406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:03.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.639+0000 7f7c9a86b640 1 -- 172.21.15.32:0/826780098 shutdown_connections 2024-04-24T22:59:03.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.639+0000 7f7c9a86b640 1 -- 172.21.15.32:0/826780098 wait complete. 2024-04-24T22:59:03.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.640+0000 7f7c9a86b640 1 Processor -- start 2024-04-24T22:59:03.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.641+0000 7f7c9a86b640 1 -- start start 2024-04-24T22:59:03.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.641+0000 7f7c9a86b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c7c003b70 0x7f7c9414bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:03.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.641+0000 7f7c9a86b640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c941f6800 0x7f7c94150810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:03.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.641+0000 7f7c9a86b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c94153830 0x7f7c94153cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:03.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.641+0000 7f7c9a86b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9415b760 con 0x7f7c94153830 2024-04-24T22:59:03.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.641+0000 7f7c9a86b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7c940b50a0 con 0x7f7c7c003b70 2024-04-24T22:59:03.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.641+0000 7f7c9a86b640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f7c940b5cd0 con 0x7f7c941f6800 2024-04-24T22:59:03.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.641+0000 7f7c98e08640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c94153830 0x7f7c94153cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:03.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.641+0000 7f7c98e08640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c94153830 0x7f7c94153cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44542/0 (socket says 172.21.15.32:44542) 2024-04-24T22:59:03.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.641+0000 7f7c98e08640 1 -- 172.21.15.32:0/2351465286 learned_addr learned my addr 172.21.15.32:0/2351465286 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:03.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.642+0000 7f7c98e08640 1 -- 172.21.15.32:0/2351465286 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c941f6800 msgr2=0x7f7c94150810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:03.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.642+0000 7f7c98e08640 1 --2- 172.21.15.32:0/2351465286 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c941f6800 0x7f7c94150810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:03.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.642+0000 7f7c98e08640 1 -- 172.21.15.32:0/2351465286 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c7c003b70 msgr2=0x7f7c9414bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:03.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.642+0000 7f7c93fff640 1 --2- 172.21.15.32:0/2351465286 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c7c003b70 0x7f7c9414bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:03.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.642+0000 7f7c98e08640 1 --2- 172.21.15.32:0/2351465286 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c7c003b70 0x7f7c9414bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:03.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.642+0000 7f7c98e08640 1 -- 172.21.15.32:0/2351465286 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c9414c410 con 0x7f7c94153830 2024-04-24T22:59:03.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.642+0000 7f7c98e08640 1 --2- 172.21.15.32:0/2351465286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c94153830 0x7f7c94153cb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7c94070490 tx=0x7f7c8c007b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:03.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.642+0000 7f7c90ff9640 1 -- 172.21.15.32:0/2351465286 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c00ab00 con 0x7f7c94153830 2024-04-24T22:59:03.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.642+0000 7f7c90ff9640 1 -- 172.21.15.32:0/2351465286 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c8c018070 con 0x7f7c94153830 2024-04-24T22:59:03.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.642+0000 7f7c9a86b640 1 -- 172.21.15.32:0/2351465286 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c9414ff60 con 0x7f7c94153830 2024-04-24T22:59:03.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.642+0000 7f7c90ff9640 1 -- 172.21.15.32:0/2351465286 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c0133f0 con 0x7f7c94153830 2024-04-24T22:59:03.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.642+0000 7f7c9a86b640 1 -- 172.21.15.32:0/2351465286 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c941f6000 con 0x7f7c94153830 2024-04-24T22:59:03.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.643+0000 7f7c9a86b640 1 -- 172.21.15.32:0/2351465286 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c94068430 con 0x7f7c94153830 2024-04-24T22:59:03.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.644+0000 7f7c90ff9640 1 -- 172.21.15.32:0/2351465286 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7c8c013590 con 0x7f7c94153830 2024-04-24T22:59:03.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.644+0000 7f7c90ff9640 1 --2- 172.21.15.32:0/2351465286 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7c8006be30 0x7f7c8006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:03.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.644+0000 7f7c90ff9640 1 -- 172.21.15.32:0/2351465286 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5609+0+0 (secure 0 0 0) 0x7f7c8c08b210 con 0x7f7c94153830 2024-04-24T22:59:03.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.644+0000 7f7c93fff640 1 --2- 172.21.15.32:0/2351465286 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7c8006be30 0x7f7c8006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:03.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.645+0000 7f7c93fff640 1 --2- 172.21.15.32:0/2351465286 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7c8006be30 0x7f7c8006e2f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7c84033000 tx=0x7f7c84008070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:03.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.649+0000 7f7c90ff9640 1 -- 172.21.15.32:0/2351465286 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7c8c059cb0 con 0x7f7c94153830 2024-04-24T22:59:03.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:03.874+0000 7f7c9a86b640 1 -- 172.21.15.32:0/2351465286 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f7c9406c000 con 0x7f7c94153830 2024-04-24T22:59:04.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.516+0000 7f7c90ff9640 1 -- 172.21.15.32:0/2351465286 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f7c8c0595b0 con 0x7f7c94153830 2024-04-24T22:59:04.518 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 2 and data pool 3 2024-04-24T22:59:04.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.535+0000 7f7c9a86b640 1 -- 172.21.15.32:0/2351465286 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7c8006be30 msgr2=0x7f7c8006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:04.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.535+0000 7f7c9a86b640 1 --2- 172.21.15.32:0/2351465286 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7c8006be30 0x7f7c8006e2f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7c84033000 tx=0x7f7c84008070 comp rx=0 tx=0).stop 2024-04-24T22:59:04.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.535+0000 7f7c9a86b640 1 -- 172.21.15.32:0/2351465286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c94153830 msgr2=0x7f7c94153cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:04.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.535+0000 7f7c9a86b640 1 --2- 172.21.15.32:0/2351465286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c94153830 0x7f7c94153cb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7c94070490 tx=0x7f7c8c007b00 comp rx=0 tx=0).stop 2024-04-24T22:59:04.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.535+0000 7f7c9a86b640 1 -- 172.21.15.32:0/2351465286 shutdown_connections 2024-04-24T22:59:04.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.535+0000 7f7c9a86b640 1 --2- 172.21.15.32:0/2351465286 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7c8006be30 0x7f7c8006e2f0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:04.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.535+0000 7f7c9a86b640 1 --2- 172.21.15.32:0/2351465286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c94153830 0x7f7c94153cb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:04.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.535+0000 7f7c9a86b640 1 --2- 172.21.15.32:0/2351465286 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c941f6800 0x7f7c94150810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:04.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.535+0000 7f7c9a86b640 1 --2- 172.21.15.32:0/2351465286 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c7c003b70 0x7f7c9414bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:04.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.535+0000 7f7c9a86b640 1 -- 172.21.15.32:0/2351465286 >> 172.21.15.32:0/2351465286 conn(0x7f7c9406b110 msgr2=0x7f7c9415b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:04.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.537+0000 7f7c9a86b640 1 -- 172.21.15.32:0/2351465286 shutdown_connections 2024-04-24T22:59:04.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.537+0000 7f7c9a86b640 1 -- 172.21.15.32:0/2351465286 wait complete. 2024-04-24T22:59:04.554 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:59:04.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.657+0000 7f5ab663f640 1 Processor -- start 2024-04-24T22:59:04.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.657+0000 7f5ab663f640 1 -- start start 2024-04-24T22:59:04.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.658+0000 7f5ab663f640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5ab006ba30 0x7f5ab006d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:04.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.658+0000 7f5ab663f640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5ab006da90 0x7f5ab00b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:04.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.658+0000 7f5ab663f640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ab015ef10 con 0x7f5ab0158dd0 2024-04-24T22:59:04.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.658+0000 7f5ab663f640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f5ab015f1f0 con 0x7f5ab006ba30 2024-04-24T22:59:04.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.658+0000 7f5ab663f640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f5ab015f4d0 con 0x7f5ab006da90 2024-04-24T22:59:04.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.658+0000 7f5aaf7fe640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5ab006ba30 0x7f5ab006d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:04.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.658+0000 7f5ab4bdc640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5ab006da90 0x7f5ab00b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:04.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.658+0000 7f5aaf7fe640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5ab006ba30 0x7f5ab006d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.32:45096/0 (socket says 172.21.15.32:45096) 2024-04-24T22:59:04.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.658+0000 7f5aaf7fe640 1 -- 172.21.15.32:0/2717807049 learned_addr learned my addr 172.21.15.32:0/2717807049 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:04.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.658+0000 7f5aaffff640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f5ab0158dd0 0x7f5ab006b320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:57322/0 (socket says 172.21.15.32:57322) 2024-04-24T22:59:04.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.658+0000 7f5ab4bdc640 1 -- 172.21.15.32:0/2717807049 >> v2:172.21.15.121:3300/0 conn(0x7f5ab006ba30 msgr2=0x7f5ab006d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:04.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.658+0000 7f5ab4bdc640 1 --2- 172.21.15.32:0/2717807049 >> v2:172.21.15.121:3300/0 conn(0x7f5ab006ba30 0x7f5ab006d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:04.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.658+0000 7f5ab4bdc640 1 -- 172.21.15.32:0/2717807049 >> v1:172.21.15.121:6789/0 conn(0x7f5ab0158dd0 legacy=0x7f5ab006b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:04.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.658+0000 7f5ab4bdc640 1 -- 172.21.15.32:0/2717807049 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab015f7b0 con 0x7f5ab006da90 2024-04-24T22:59:04.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.658+0000 7f5aaf7fe640 1 --2- 172.21.15.32:0/2717807049 >> v2:172.21.15.121:3300/0 conn(0x7f5ab006ba30 0x7f5ab006d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:59:04.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.659+0000 7f5ab4bdc640 1 --2- 172.21.15.32:0/2717807049 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5ab006da90 0x7f5ab00b5d00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8009d90 tx=0x7f5aa802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b41a4cf004fbd34e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:04.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.659+0000 7f5aaeffd640 1 -- 172.21.15.32:0/2717807049 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa8007740 con 0x7f5ab006da90 2024-04-24T22:59:04.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.659+0000 7f5aaeffd640 1 -- 172.21.15.32:0/2717807049 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5aa80078e0 con 0x7f5ab006da90 2024-04-24T22:59:04.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.659+0000 7f5aaeffd640 1 -- 172.21.15.32:0/2717807049 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa8007cb0 con 0x7f5ab006da90 2024-04-24T22:59:04.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.660+0000 7f5ab663f640 1 -- 172.21.15.32:0/2717807049 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5ab006da90 msgr2=0x7f5ab00b5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:04.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.660+0000 7f5ab663f640 1 --2- 172.21.15.32:0/2717807049 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5ab006da90 0x7f5ab00b5d00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8009d90 tx=0x7f5aa802a660 comp rx=0 tx=0).stop 2024-04-24T22:59:04.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.660+0000 7f5ab663f640 1 -- 172.21.15.32:0/2717807049 shutdown_connections 2024-04-24T22:59:04.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.660+0000 7f5ab663f640 1 --2- 172.21.15.32:0/2717807049 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5ab006da90 0x7f5ab00b5d00 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:04.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.660+0000 7f5ab663f640 1 --2- 172.21.15.32:0/2717807049 >> v2:172.21.15.121:3300/0 conn(0x7f5ab006ba30 0x7f5ab006d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:04.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.660+0000 7f5ab663f640 1 -- 172.21.15.32:0/2717807049 >> 172.21.15.32:0/2717807049 conn(0x7f5ab006e800 msgr2=0x7f5ab006ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:04.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.660+0000 7f5ab663f640 1 -- 172.21.15.32:0/2717807049 shutdown_connections 2024-04-24T22:59:04.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.660+0000 7f5ab663f640 1 -- 172.21.15.32:0/2717807049 wait complete. 2024-04-24T22:59:04.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.661+0000 7f5ab663f640 1 Processor -- start 2024-04-24T22:59:04.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.661+0000 7f5ab663f640 1 -- start start 2024-04-24T22:59:04.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.662+0000 7f5ab663f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ab006ba30 0x7f5ab0157270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:04.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.662+0000 7f5ab663f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ab006da90 0x7f5ab0155ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:04.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.662+0000 7f5ab663f640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5ab0158dd0 0x7f5ab0156430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:04.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.662+0000 7f5ab663f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab006bfe0 con 0x7f5ab006ba30 2024-04-24T22:59:04.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.662+0000 7f5ab663f640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab006ded0 con 0x7f5ab006da90 2024-04-24T22:59:04.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.662+0000 7f5ab663f640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f5ab01591b0 con 0x7f5ab0158dd0 2024-04-24T22:59:04.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.662+0000 7f5aaffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ab006ba30 0x7f5ab0157270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:04.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.662+0000 7f5aaffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ab006ba30 0x7f5ab0157270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44554/0 (socket says 172.21.15.32:44554) 2024-04-24T22:59:04.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.662+0000 7f5aaffff640 1 -- 172.21.15.32:0/1265936515 learned_addr learned my addr 172.21.15.32:0/1265936515 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:04.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.662+0000 7f5aaf7fe640 1 --2- 172.21.15.32:0/1265936515 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ab006da90 0x7f5ab0155ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:04.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.662+0000 7f5ab4bdc640 1 --2- 172.21.15.32:0/1265936515 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5ab0158dd0 0x7f5ab0156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:04.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.663+0000 7f5aaffff640 1 -- 172.21.15.32:0/1265936515 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5ab0158dd0 msgr2=0x7f5ab0156430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:04.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.663+0000 7f5aaffff640 1 --2- 172.21.15.32:0/1265936515 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5ab0158dd0 0x7f5ab0156430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:04.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.663+0000 7f5aaffff640 1 -- 172.21.15.32:0/1265936515 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ab006da90 msgr2=0x7f5ab0155ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:04.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.663+0000 7f5aaffff640 1 --2- 172.21.15.32:0/1265936515 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ab006da90 0x7f5ab0155ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:04.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.663+0000 7f5aaffff640 1 -- 172.21.15.32:0/1265936515 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab0159450 con 0x7f5ab006ba30 2024-04-24T22:59:04.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.663+0000 7f5aaf7fe640 1 --2- 172.21.15.32:0/1265936515 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ab006da90 0x7f5ab0155ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:04.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.663+0000 7f5aaffff640 1 --2- 172.21.15.32:0/1265936515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ab006ba30 0x7f5ab0157270 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5aa400c3c0 tx=0x7f5aa400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:04.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.663+0000 7f5aacff9640 1 -- 172.21.15.32:0/1265936515 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa40074e0 con 0x7f5ab006ba30 2024-04-24T22:59:04.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.663+0000 7f5aacff9640 1 -- 172.21.15.32:0/1265936515 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5aa4007680 con 0x7f5ab006ba30 2024-04-24T22:59:04.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.663+0000 7f5ab663f640 1 -- 172.21.15.32:0/1265936515 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ab0159730 con 0x7f5ab006ba30 2024-04-24T22:59:04.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.663+0000 7f5aacff9640 1 -- 172.21.15.32:0/1265936515 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa4004a00 con 0x7f5ab006ba30 2024-04-24T22:59:04.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.663+0000 7f5ab663f640 1 -- 172.21.15.32:0/1265936515 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ab013ede0 con 0x7f5ab006ba30 2024-04-24T22:59:04.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.664+0000 7f5ab663f640 1 -- 172.21.15.32:0/1265936515 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ab013ec00 con 0x7f5ab006ba30 2024-04-24T22:59:04.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.664+0000 7f5aacff9640 1 -- 172.21.15.32:0/1265936515 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5aa401a020 con 0x7f5ab006ba30 2024-04-24T22:59:04.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.665+0000 7f5aacff9640 1 --2- 172.21.15.32:0/1265936515 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5a9c06be30 0x7f5a9c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:04.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.665+0000 7f5aacff9640 1 -- 172.21.15.32:0/1265936515 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5aa408a5a0 con 0x7f5ab006ba30 2024-04-24T22:59:04.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.665+0000 7f5aaf7fe640 1 --2- 172.21.15.32:0/1265936515 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5a9c06be30 0x7f5a9c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:04.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.666+0000 7f5aaf7fe640 1 --2- 172.21.15.32:0/1265936515 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5a9c06be30 0x7f5a9c06e2f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0009630 tx=0x7f5aa0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:04.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.669+0000 7f5aacff9640 1 -- 172.21.15.32:0/1265936515 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5ab013ec00 con 0x7f5ab006ba30 2024-04-24T22:59:04.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.852+0000 7f5ab663f640 1 -- 172.21.15.32:0/1265936515 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5ab00d2df0 con 0x7f5ab006ba30 2024-04-24T22:59:04.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.853+0000 7f5aacff9640 1 -- 172.21.15.32:0/1265936515 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16634 (secure 0 0 0) 0x7f5aa4058f60 con 0x7f5ab006ba30 2024-04-24T22:59:04.856 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:59:04.856 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":20,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T22:59:03.876011+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:59:02.016639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T22:59:02.846087+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:59:04.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.855+0000 7f5ab663f640 1 -- 172.21.15.32:0/1265936515 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5a9c06be30 msgr2=0x7f5a9c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:04.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.855+0000 7f5ab663f640 1 --2- 172.21.15.32:0/1265936515 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5a9c06be30 0x7f5a9c06e2f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0009630 tx=0x7f5aa0008040 comp rx=0 tx=0).stop 2024-04-24T22:59:04.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.855+0000 7f5ab663f640 1 -- 172.21.15.32:0/1265936515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ab006ba30 msgr2=0x7f5ab0157270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:04.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.855+0000 7f5ab663f640 1 --2- 172.21.15.32:0/1265936515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ab006ba30 0x7f5ab0157270 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5aa400c3c0 tx=0x7f5aa400c880 comp rx=0 tx=0).stop 2024-04-24T22:59:04.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.855+0000 7f5ab663f640 1 -- 172.21.15.32:0/1265936515 shutdown_connections 2024-04-24T22:59:04.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.855+0000 7f5ab663f640 1 --2- 172.21.15.32:0/1265936515 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5a9c06be30 0x7f5a9c06e2f0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:04.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.855+0000 7f5ab663f640 1 --2- 172.21.15.32:0/1265936515 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5ab0158dd0 0x7f5ab0156430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:04.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.855+0000 7f5ab663f640 1 --2- 172.21.15.32:0/1265936515 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ab006da90 0x7f5ab0155ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:04.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.855+0000 7f5ab663f640 1 --2- 172.21.15.32:0/1265936515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ab006ba30 0x7f5ab0157270 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:04.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.855+0000 7f5ab663f640 1 -- 172.21.15.32:0/1265936515 >> 172.21.15.32:0/1265936515 conn(0x7f5ab006e800 msgr2=0x7f5ab0162350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:04.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.856+0000 7f5ab663f640 1 -- 172.21.15.32:0/1265936515 shutdown_connections 2024-04-24T22:59:04.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.856+0000 7f5ab663f640 1 -- 172.21.15.32:0/1265936515 wait complete. 2024-04-24T22:59:04.870 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:59:04.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.969+0000 7f9bb9600640 1 Processor -- start 2024-04-24T22:59:04.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.969+0000 7f9bb9600640 1 -- start start 2024-04-24T22:59:04.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.970+0000 7f9bb9600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9bb40cfee0 0x7f9bb415f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:04.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.970+0000 7f9bb9600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9bb415c880 0x7f9bb406b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:04.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.970+0000 7f9bb9600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bb415fc00 con 0x7f9bb406b650 2024-04-24T22:59:04.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.970+0000 7f9bb9600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f9bb406baa0 con 0x7f9bb415c880 2024-04-24T22:59:04.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.970+0000 7f9bb9600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f9bb4160110 con 0x7f9bb40cfee0 2024-04-24T22:59:04.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.970+0000 7f9bb27fc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9bb415c880 0x7f9bb406b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:04.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.970+0000 7f9bb27fc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9bb415c880 0x7f9bb406b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44564/0 (socket says 172.21.15.32:44564) 2024-04-24T22:59:04.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.970+0000 7f9bb27fc640 1 -- 172.21.15.32:0/1226766512 learned_addr learned my addr 172.21.15.32:0/1226766512 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:04.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.970+0000 7f9bb37fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9bb406b650 0x7f9bb41611b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44818/0 (socket says 172.21.15.32:44818) 2024-04-24T22:59:04.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.970+0000 7f9bb2ffd640 1 --2- 172.21.15.32:0/1226766512 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9bb40cfee0 0x7f9bb415f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:04.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.970+0000 7f9bb27fc640 1 -- 172.21.15.32:0/1226766512 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9bb40cfee0 msgr2=0x7f9bb415f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:04.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.970+0000 7f9bb27fc640 1 --2- 172.21.15.32:0/1226766512 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9bb40cfee0 0x7f9bb415f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:04.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.970+0000 7f9bb1ffb640 1 -- 172.21.15.32:0/1226766512 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3611399952 0 0) 0x7f9ba4001490 con 0x7f9bb406b650 2024-04-24T22:59:04.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.970+0000 7f9bb27fc640 1 -- 172.21.15.32:0/1226766512 >> v1:172.21.15.32:6789/0 conn(0x7f9bb406b650 legacy=0x7f9bb41611b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:04.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.970+0000 7f9bb27fc640 1 -- 172.21.15.32:0/1226766512 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bb41603f0 con 0x7f9bb415c880 2024-04-24T22:59:04.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.971+0000 7f9bb27fc640 1 --2- 172.21.15.32:0/1226766512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bb415c880 0x7f9bb406b110 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c009870 tx=0x7f9b9c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb7ada60af45e29a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:04.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.971+0000 7f9bb1ffb640 1 -- 172.21.15.32:0/1226766512 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c037070 con 0x7f9bb415c880 2024-04-24T22:59:04.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.971+0000 7f9bb1ffb640 1 -- 172.21.15.32:0/1226766512 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b9c032440 con 0x7f9bb415c880 2024-04-24T22:59:04.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.971+0000 7f9bb1ffb640 1 -- 172.21.15.32:0/1226766512 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c032810 con 0x7f9bb415c880 2024-04-24T22:59:04.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.972+0000 7f9bb9600640 1 -- 172.21.15.32:0/1226766512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bb415c880 msgr2=0x7f9bb406b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:04.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.972+0000 7f9bb9600640 1 --2- 172.21.15.32:0/1226766512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bb415c880 0x7f9bb406b110 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c009870 tx=0x7f9b9c02a1c0 comp rx=0 tx=0).stop 2024-04-24T22:59:04.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.972+0000 7f9bb9600640 1 -- 172.21.15.32:0/1226766512 shutdown_connections 2024-04-24T22:59:04.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.972+0000 7f9bb9600640 1 --2- 172.21.15.32:0/1226766512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bb415c880 0x7f9bb406b110 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:04.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.972+0000 7f9bb9600640 1 --2- 172.21.15.32:0/1226766512 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9bb40cfee0 0x7f9bb415f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:04.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.972+0000 7f9bb9600640 1 -- 172.21.15.32:0/1226766512 >> 172.21.15.32:0/1226766512 conn(0x7f9bb40743f0 msgr2=0x7f9bb40b7620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:04.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.972+0000 7f9bb9600640 1 -- 172.21.15.32:0/1226766512 shutdown_connections 2024-04-24T22:59:04.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.972+0000 7f9bb9600640 1 -- 172.21.15.32:0/1226766512 wait complete. 2024-04-24T22:59:04.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.973+0000 7f9bb9600640 1 Processor -- start 2024-04-24T22:59:04.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.973+0000 7f9bb9600640 1 -- start start 2024-04-24T22:59:04.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.974+0000 7f9bb9600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9bb406b650 0x7f9bb40d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:04.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.974+0000 7f9bb9600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bb40cfee0 0x7f9bb40c9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:04.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.974+0000 7f9bb9600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bb415c880 0x7f9bb40cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:04.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.974+0000 7f9bb9600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb4160c90 con 0x7f9bb40cfee0 2024-04-24T22:59:04.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.974+0000 7f9bb9600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb41609b0 con 0x7f9bb415c880 2024-04-24T22:59:04.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.974+0000 7f9bb9600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f9bb415fe00 con 0x7f9bb406b650 2024-04-24T22:59:04.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.974+0000 7f9bb27fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bb40cfee0 0x7f9bb40c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:04.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.974+0000 7f9bb27fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bb40cfee0 0x7f9bb40c9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44568/0 (socket says 172.21.15.32:44568) 2024-04-24T22:59:04.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.974+0000 7f9bb27fc640 1 -- 172.21.15.32:0/3773964576 learned_addr learned my addr 172.21.15.32:0/3773964576 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:04.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.974+0000 7f9bb2ffd640 1 --2- 172.21.15.32:0/3773964576 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9bb406b650 0x7f9bb40d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:04.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.974+0000 7f9bb37fe640 1 --2- 172.21.15.32:0/3773964576 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bb415c880 0x7f9bb40cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:04.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.974+0000 7f9bb27fc640 1 -- 172.21.15.32:0/3773964576 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9bb406b650 msgr2=0x7f9bb40d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:04.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.974+0000 7f9bb27fc640 1 --2- 172.21.15.32:0/3773964576 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9bb406b650 0x7f9bb40d2bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:04.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.975+0000 7f9bb27fc640 1 -- 172.21.15.32:0/3773964576 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bb415c880 msgr2=0x7f9bb40cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:04.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.975+0000 7f9bb27fc640 1 --2- 172.21.15.32:0/3773964576 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bb415c880 0x7f9bb40cebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:04.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.975+0000 7f9bb27fc640 1 -- 172.21.15.32:0/3773964576 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bb40d3360 con 0x7f9bb40cfee0 2024-04-24T22:59:04.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.975+0000 7f9bb27fc640 1 --2- 172.21.15.32:0/3773964576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bb40cfee0 0x7f9bb40c9770 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c02a200 tx=0x7f9b9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:04.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.975+0000 7f9bab7fe640 1 -- 172.21.15.32:0/3773964576 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c041020 con 0x7f9bb40cfee0 2024-04-24T22:59:04.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.975+0000 7f9bab7fe640 1 -- 172.21.15.32:0/3773964576 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b9c0306e0 con 0x7f9bb40cfee0 2024-04-24T22:59:04.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.975+0000 7f9bb9600640 1 -- 172.21.15.32:0/3773964576 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bb40d3640 con 0x7f9bb40cfee0 2024-04-24T22:59:04.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.975+0000 7f9bab7fe640 1 -- 172.21.15.32:0/3773964576 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c037040 con 0x7f9bb40cfee0 2024-04-24T22:59:04.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.975+0000 7f9bb9600640 1 -- 172.21.15.32:0/3773964576 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bb40d3c00 con 0x7f9bb40cfee0 2024-04-24T22:59:04.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.976+0000 7f9bb9600640 1 -- 172.21.15.32:0/3773964576 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bb40682b0 con 0x7f9bb40cfee0 2024-04-24T22:59:04.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.976+0000 7f9bab7fe640 1 -- 172.21.15.32:0/3773964576 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9b9c044020 con 0x7f9bb40cfee0 2024-04-24T22:59:04.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.977+0000 7f9bab7fe640 1 --2- 172.21.15.32:0/3773964576 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9b9406be30 0x7f9b9406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:04.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.977+0000 7f9bab7fe640 1 -- 172.21.15.32:0/3773964576 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9b9c0ab740 con 0x7f9bb40cfee0 2024-04-24T22:59:04.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.977+0000 7f9bb2ffd640 1 --2- 172.21.15.32:0/3773964576 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9b9406be30 0x7f9b9406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:04.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.977+0000 7f9bb2ffd640 1 --2- 172.21.15.32:0/3773964576 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9b9406be30 0x7f9b9406e2f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9bac006fd0 tx=0x7f9bac008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:04.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:04.981+0000 7f9bab7fe640 1 -- 172.21.15.32:0/3773964576 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9b9c04b030 con 0x7f9bb40cfee0 2024-04-24T22:59:05.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.165+0000 7f9bb9600640 1 -- 172.21.15.32:0/3773964576 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9bb415f7b0 con 0x7f9bb40cfee0 2024-04-24T22:59:05.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.166+0000 7f9bab7fe640 1 -- 172.21.15.32:0/3773964576 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16634 (secure 0 0 0) 0x7f9b9c07a050 con 0x7f9bb40cfee0 2024-04-24T22:59:05.168 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:59:05.168 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":20,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T22:59:03.876011+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:59:02.016639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T22:59:02.846087+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:59:05.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.168+0000 7f9bb9600640 1 -- 172.21.15.32:0/3773964576 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9b9406be30 msgr2=0x7f9b9406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:05.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.168+0000 7f9bb9600640 1 --2- 172.21.15.32:0/3773964576 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9b9406be30 0x7f9b9406e2f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9bac006fd0 tx=0x7f9bac008040 comp rx=0 tx=0).stop 2024-04-24T22:59:05.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.168+0000 7f9bb9600640 1 -- 172.21.15.32:0/3773964576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bb40cfee0 msgr2=0x7f9bb40c9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:05.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.168+0000 7f9bb9600640 1 --2- 172.21.15.32:0/3773964576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bb40cfee0 0x7f9bb40c9770 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c02a200 tx=0x7f9b9c0046c0 comp rx=0 tx=0).stop 2024-04-24T22:59:05.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.168+0000 7f9bb9600640 1 -- 172.21.15.32:0/3773964576 shutdown_connections 2024-04-24T22:59:05.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.168+0000 7f9bb9600640 1 --2- 172.21.15.32:0/3773964576 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9b9406be30 0x7f9b9406e2f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:05.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.168+0000 7f9bb9600640 1 --2- 172.21.15.32:0/3773964576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bb40cfee0 0x7f9bb40c9770 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:05.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.168+0000 7f9bb9600640 1 --2- 172.21.15.32:0/3773964576 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bb415c880 0x7f9bb40cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:05.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.168+0000 7f9bb9600640 1 --2- 172.21.15.32:0/3773964576 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9bb406b650 0x7f9bb40d2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:05.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.168+0000 7f9bb9600640 1 -- 172.21.15.32:0/3773964576 >> 172.21.15.32:0/3773964576 conn(0x7f9bb40743f0 msgr2=0x7f9bb40d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:05.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.168+0000 7f9bb9600640 1 -- 172.21.15.32:0/3773964576 shutdown_connections 2024-04-24T22:59:05.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.168+0000 7f9bb9600640 1 -- 172.21.15.32:0/3773964576 wait complete. 2024-04-24T22:59:05.181 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-24T22:59:05.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.283+0000 7fb7fcd9d640 1 Processor -- start 2024-04-24T22:59:05.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.283+0000 7fb7fcd9d640 1 -- start start 2024-04-24T22:59:05.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.284+0000 7fb7fcd9d640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb7f80d4210 0x7fb7f80d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:05.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.284+0000 7fb7fcd9d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7f80d3170 con 0x7fb7f806b820 2024-04-24T22:59:05.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.284+0000 7fb7fcd9d640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7f80d3450 con 0x7fb7f815e9b0 2024-04-24T22:59:05.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.284+0000 7fb7fcd9d640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fb7f80d3730 con 0x7fb7f80d4210 2024-04-24T22:59:05.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.284+0000 7fb7f5d9b640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb7f806b820 0x7fb7f80b78f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44822/0 (socket says 172.21.15.32:44822) 2024-04-24T22:59:05.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.284+0000 7fb7f5d9b640 1 -- 172.21.15.32:0/2508561360 learned_addr learned my addr 172.21.15.32:0/2508561360 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:05.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.285+0000 7fb7f6d9d640 1 --2- 172.21.15.32:0/2508561360 >> v2:172.21.15.121:3300/0 conn(0x7fb7f80d4210 0x7fb7f80d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:05.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.285+0000 7fb7f559a640 1 -- 172.21.15.32:0/2508561360 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2580190842 0 0) 0x7fb7f80d3170 con 0x7fb7f806b820 2024-04-24T22:59:05.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.285+0000 7fb7f559a640 1 -- 172.21.15.32:0/2508561360 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb7e0003520 con 0x7fb7f806b820 2024-04-24T22:59:05.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.285+0000 7fb7f559a640 1 -- 172.21.15.32:0/2508561360 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3382176851 0 0) 0x7fb7f80d3450 con 0x7fb7f815e9b0 2024-04-24T22:59:05.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.285+0000 7fb7f559a640 1 -- 172.21.15.32:0/2508561360 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb7f80d3170 con 0x7fb7f815e9b0 2024-04-24T22:59:05.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.285+0000 7fb7f6d9d640 1 -- 172.21.15.32:0/2508561360 >> v1:172.21.15.121:6789/0 conn(0x7fb7f815e9b0 legacy=0x7fb7f806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:05.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.285+0000 7fb7f6d9d640 1 -- 172.21.15.32:0/2508561360 >> v1:172.21.15.32:6789/0 conn(0x7fb7f806b820 legacy=0x7fb7f80b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:05.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.285+0000 7fb7f559a640 1 -- 172.21.15.32:0/2508561360 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1678397344 0 0) 0x7fb7e0003520 con 0x7fb7f806b820 2024-04-24T22:59:05.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.285+0000 7fb7f6d9d640 1 -- 172.21.15.32:0/2508561360 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7f80d3a10 con 0x7fb7f80d4210 2024-04-24T22:59:05.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.286+0000 7fb7f6d9d640 1 --2- 172.21.15.32:0/2508561360 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7f80d4210 0x7fb7f80d2b90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8009d90 tx=0x7fb7e802a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc8039751a7a8b03 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:05.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.286+0000 7fb7f559a640 1 -- 172.21.15.32:0/2508561360 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec001670 con 0x7fb7f80d4210 2024-04-24T22:59:05.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.286+0000 7fb7f559a640 1 -- 172.21.15.32:0/2508561360 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7e8007540 con 0x7fb7f80d4210 2024-04-24T22:59:05.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.286+0000 7fb7f559a640 1 -- 172.21.15.32:0/2508561360 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e8007910 con 0x7fb7f80d4210 2024-04-24T22:59:05.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.286+0000 7fb7fcd9d640 1 -- 172.21.15.32:0/2508561360 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7f80d4210 msgr2=0x7fb7f80d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:05.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.286+0000 7fb7fcd9d640 1 --2- 172.21.15.32:0/2508561360 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7f80d4210 0x7fb7f80d2b90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8009d90 tx=0x7fb7e802a5f0 comp rx=0 tx=0).stop 2024-04-24T22:59:05.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.287+0000 7fb7fcd9d640 1 -- 172.21.15.32:0/2508561360 shutdown_connections 2024-04-24T22:59:05.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.287+0000 7fb7fcd9d640 1 --2- 172.21.15.32:0/2508561360 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7f80d4210 0x7fb7f80d2b90 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:05.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.287+0000 7fb7fcd9d640 1 -- 172.21.15.32:0/2508561360 >> 172.21.15.32:0/2508561360 conn(0x7fb7f80d11d0 msgr2=0x7fb7f80d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:05.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.287+0000 7fb7fcd9d640 1 -- 172.21.15.32:0/2508561360 shutdown_connections 2024-04-24T22:59:05.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.287+0000 7fb7fcd9d640 1 -- 172.21.15.32:0/2508561360 wait complete. 2024-04-24T22:59:05.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.288+0000 7fb7fcd9d640 1 Processor -- start 2024-04-24T22:59:05.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.288+0000 7fb7fcd9d640 1 -- start start 2024-04-24T22:59:05.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.289+0000 7fb7fcd9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7f806b820 0x7fb7f8154300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:05.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.289+0000 7fb7fcd9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7f80d4210 0x7fb7f80cd210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:05.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.289+0000 7fb7fcd9d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb7f815e9b0 0x7fb7f8150b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:05.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.289+0000 7fb7fcd9d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f80ba0e0 con 0x7fb7f806b820 2024-04-24T22:59:05.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.289+0000 7fb7fcd9d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f80d2400 con 0x7fb7f80d4210 2024-04-24T22:59:05.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.289+0000 7fb7fcd9d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fb7f80ba260 con 0x7fb7f815e9b0 2024-04-24T22:59:05.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.289+0000 7fb7f659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7f806b820 0x7fb7f8154300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:05.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.289+0000 7fb7f659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7f806b820 0x7fb7f8154300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44580/0 (socket says 172.21.15.32:44580) 2024-04-24T22:59:05.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.289+0000 7fb7f659c640 1 -- 172.21.15.32:0/2914462157 learned_addr learned my addr 172.21.15.32:0/2914462157 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:05.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.289+0000 7fb7f6d9d640 1 --2- 172.21.15.32:0/2914462157 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb7f815e9b0 0x7fb7f8150b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:05.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.289+0000 7fb7f5d9b640 1 --2- 172.21.15.32:0/2914462157 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7f80d4210 0x7fb7f80cd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:05.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.289+0000 7fb7f659c640 1 -- 172.21.15.32:0/2914462157 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb7f815e9b0 msgr2=0x7fb7f8150b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:05.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.289+0000 7fb7f659c640 1 --2- 172.21.15.32:0/2914462157 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb7f815e9b0 0x7fb7f8150b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:05.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.289+0000 7fb7f659c640 1 -- 172.21.15.32:0/2914462157 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7f80d4210 msgr2=0x7fb7f80cd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:05.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.289+0000 7fb7f659c640 1 --2- 172.21.15.32:0/2914462157 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7f80d4210 0x7fb7f80cd210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:05.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.289+0000 7fb7f659c640 1 -- 172.21.15.32:0/2914462157 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7f80ceb20 con 0x7fb7f806b820 2024-04-24T22:59:05.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.289+0000 7fb7f659c640 1 --2- 172.21.15.32:0/2914462157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7f806b820 0x7fb7f8154300 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb7f000c430 tx=0x7fb7f000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:05.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.289+0000 7fb7e6ffd640 1 -- 172.21.15.32:0/2914462157 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f0017070 con 0x7fb7f806b820 2024-04-24T22:59:05.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.289+0000 7fb7e6ffd640 1 -- 172.21.15.32:0/2914462157 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7f00052f0 con 0x7fb7f806b820 2024-04-24T22:59:05.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.290+0000 7fb7fcd9d640 1 -- 172.21.15.32:0/2914462157 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7f80cda80 con 0x7fb7f806b820 2024-04-24T22:59:05.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.290+0000 7fb7e6ffd640 1 -- 172.21.15.32:0/2914462157 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f0003830 con 0x7fb7f806b820 2024-04-24T22:59:05.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.290+0000 7fb7fcd9d640 1 -- 172.21.15.32:0/2914462157 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7f80ce040 con 0x7fb7f806b820 2024-04-24T22:59:05.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.290+0000 7fb7fcd9d640 1 -- 172.21.15.32:0/2914462157 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7f8068430 con 0x7fb7f806b820 2024-04-24T22:59:05.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.291+0000 7fb7e6ffd640 1 -- 172.21.15.32:0/2914462157 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb7f001a070 con 0x7fb7f806b820 2024-04-24T22:59:05.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.291+0000 7fb7e6ffd640 1 --2- 172.21.15.32:0/2914462157 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb7d806be30 0x7fb7d806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:05.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.291+0000 7fb7e6ffd640 1 -- 172.21.15.32:0/2914462157 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb7f008a380 con 0x7fb7f806b820 2024-04-24T22:59:05.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.292+0000 7fb7f5d9b640 1 --2- 172.21.15.32:0/2914462157 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb7d806be30 0x7fb7d806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:05.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.292+0000 7fb7f5d9b640 1 --2- 172.21.15.32:0/2914462157 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb7d806be30 0x7fb7d806e2f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb7f815b3b0 tx=0x7fb7ec009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:05.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.296+0000 7fb7e6ffd640 1 -- 172.21.15.32:0/2914462157 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb7f0058c90 con 0x7fb7f806b820 2024-04-24T22:59:05.505 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T22:59:05.508 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T22:59:05.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.522+0000 7fb7fcd9d640 1 -- 172.21.15.32:0/2914462157 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fb7f80008d0 con 0x7fb7f806b820 2024-04-24T22:59:05.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.532+0000 7fb7e6ffd640 1 -- 172.21.15.32:0/2914462157 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) ==== 118+0+0 (secure 0 0 0) 0x7fb7f0058590 con 0x7fb7f806b820 2024-04-24T22:59:05.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.534+0000 7fb7fcd9d640 1 -- 172.21.15.32:0/2914462157 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb7d806be30 msgr2=0x7fb7d806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:05.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.534+0000 7fb7fcd9d640 1 --2- 172.21.15.32:0/2914462157 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb7d806be30 0x7fb7d806e2f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb7f815b3b0 tx=0x7fb7ec009040 comp rx=0 tx=0).stop 2024-04-24T22:59:05.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.534+0000 7fb7fcd9d640 1 -- 172.21.15.32:0/2914462157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7f806b820 msgr2=0x7fb7f8154300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:05.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.534+0000 7fb7fcd9d640 1 --2- 172.21.15.32:0/2914462157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7f806b820 0x7fb7f8154300 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb7f000c430 tx=0x7fb7f000c8f0 comp rx=0 tx=0).stop 2024-04-24T22:59:05.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.534+0000 7fb7fcd9d640 1 -- 172.21.15.32:0/2914462157 shutdown_connections 2024-04-24T22:59:05.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.534+0000 7fb7fcd9d640 1 --2- 172.21.15.32:0/2914462157 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb7d806be30 0x7fb7d806e2f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:05.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.534+0000 7fb7fcd9d640 1 --2- 172.21.15.32:0/2914462157 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb7f815e9b0 0x7fb7f8150b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:05.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.534+0000 7fb7fcd9d640 1 --2- 172.21.15.32:0/2914462157 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7f80d4210 0x7fb7f80cd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:05.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.534+0000 7fb7fcd9d640 1 --2- 172.21.15.32:0/2914462157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7f806b820 0x7fb7f8154300 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:05.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.534+0000 7fb7fcd9d640 1 -- 172.21.15.32:0/2914462157 >> 172.21.15.32:0/2914462157 conn(0x7fb7f80d11d0 msgr2=0x7fb7f80d15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:05.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.534+0000 7fb7fcd9d640 1 -- 172.21.15.32:0/2914462157 shutdown_connections 2024-04-24T22:59:05.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.534+0000 7fb7fcd9d640 1 -- 172.21.15.32:0/2914462157 wait complete. 2024-04-24T22:59:05.546 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd', 'disabled']} 2024-04-24T22:59:05.546 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-24T22:59:05.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.695+0000 7f68c8714640 1 Processor -- start 2024-04-24T22:59:05.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.695+0000 7f68c8714640 1 -- start start 2024-04-24T22:59:05.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.696+0000 7f68c8714640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f68c0158dd0 0x7f68c006b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:05.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.696+0000 7f68c8714640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68c006dc60 0x7f68c00b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:05.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.696+0000 7f68c8714640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68c006bd10 con 0x7f68c006b860 2024-04-24T22:59:05.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.696+0000 7f68c8714640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f68c015ef10 con 0x7f68c0158dd0 2024-04-24T22:59:05.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.696+0000 7f68c8714640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f68c015f1f0 con 0x7f68c006dc60 2024-04-24T22:59:05.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.696+0000 7f68c64b0640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f68c0158dd0 0x7f68c006b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:05.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.696+0000 7f68c5caf640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f68c006b860 0x7f68c006d550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44824/0 (socket says 172.21.15.32:44824) 2024-04-24T22:59:05.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.696+0000 7f68c64b0640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f68c0158dd0 0x7f68c006b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44584/0 (socket says 172.21.15.32:44584) 2024-04-24T22:59:05.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.696+0000 7f68c5caf640 1 -- 172.21.15.32:0/2319688693 learned_addr learned my addr 172.21.15.32:0/2319688693 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:05.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.697+0000 7f68c6cb1640 1 --2- 172.21.15.32:0/2319688693 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68c006dc60 0x7f68c00b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:05.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.697+0000 7f68c64b0640 1 -- 172.21.15.32:0/2319688693 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68c006dc60 msgr2=0x7f68c00b5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:05.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.697+0000 7f68c64b0640 1 --2- 172.21.15.32:0/2319688693 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68c006dc60 0x7f68c00b5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:05.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.697+0000 7f68c64b0640 1 -- 172.21.15.32:0/2319688693 >> v1:172.21.15.32:6789/0 conn(0x7f68c006b860 legacy=0x7f68c006d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:05.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.697+0000 7f68c54ae640 1 -- 172.21.15.32:0/2319688693 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3344596990 0 0) 0x7f68c006bd10 con 0x7f68c006b860 2024-04-24T22:59:05.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.697+0000 7f68c64b0640 1 -- 172.21.15.32:0/2319688693 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68c015f4d0 con 0x7f68c0158dd0 2024-04-24T22:59:05.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.697+0000 7f68c64b0640 1 --2- 172.21.15.32:0/2319688693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68c0158dd0 0x7f68c006b320 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f68b80097a0 tx=0x7f68b802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7398a35085e6f2a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:05.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.697+0000 7f68c54ae640 1 -- 172.21.15.32:0/2319688693 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b8037070 con 0x7f68c0158dd0 2024-04-24T22:59:05.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.697+0000 7f68c54ae640 1 -- 172.21.15.32:0/2319688693 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68b8032440 con 0x7f68c0158dd0 2024-04-24T22:59:05.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.697+0000 7f68c54ae640 1 -- 172.21.15.32:0/2319688693 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b8032810 con 0x7f68c0158dd0 2024-04-24T22:59:05.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.698+0000 7f68c8714640 1 -- 172.21.15.32:0/2319688693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68c0158dd0 msgr2=0x7f68c006b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:05.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.698+0000 7f68c8714640 1 --2- 172.21.15.32:0/2319688693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68c0158dd0 0x7f68c006b320 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f68b80097a0 tx=0x7f68b802a180 comp rx=0 tx=0).stop 2024-04-24T22:59:05.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.698+0000 7f68c8714640 1 -- 172.21.15.32:0/2319688693 shutdown_connections 2024-04-24T22:59:05.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.698+0000 7f68c8714640 1 --2- 172.21.15.32:0/2319688693 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68c006dc60 0x7f68c00b5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:05.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.698+0000 7f68c8714640 1 --2- 172.21.15.32:0/2319688693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68c0158dd0 0x7f68c006b320 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:05.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.698+0000 7f68c8714640 1 -- 172.21.15.32:0/2319688693 >> 172.21.15.32:0/2319688693 conn(0x7f68c006e800 msgr2=0x7f68c006ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:05.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.698+0000 7f68c8714640 1 -- 172.21.15.32:0/2319688693 shutdown_connections 2024-04-24T22:59:05.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.698+0000 7f68c8714640 1 -- 172.21.15.32:0/2319688693 wait complete. 2024-04-24T22:59:05.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.699+0000 7f68c8714640 1 Processor -- start 2024-04-24T22:59:05.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.699+0000 7f68c8714640 1 -- start start 2024-04-24T22:59:05.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.700+0000 7f68c8714640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68c006b860 0x7f68c014f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:05.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.700+0000 7f68c8714640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68c006dc60 0x7f68c015ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:05.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.700+0000 7f68c8714640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68c0158dd0 0x7f68c013fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:05.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.700+0000 7f68c8714640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f68c006bfe0 con 0x7f68c006dc60 2024-04-24T22:59:05.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.700+0000 7f68c8714640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f68c015fcd0 con 0x7f68c0158dd0 2024-04-24T22:59:05.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.700+0000 7f68c8714640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f68c01591b0 con 0x7f68c006b860 2024-04-24T22:59:05.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.700+0000 7f68c5caf640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68c006dc60 0x7f68c015ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:05.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.700+0000 7f68c5caf640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68c006dc60 0x7f68c015ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44586/0 (socket says 172.21.15.32:44586) 2024-04-24T22:59:05.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.700+0000 7f68c5caf640 1 -- 172.21.15.32:0/3210029722 learned_addr learned my addr 172.21.15.32:0/3210029722 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:05.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.700+0000 7f68c64b0640 1 --2- 172.21.15.32:0/3210029722 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68c006b860 0x7f68c014f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:05.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.700+0000 7f68c6cb1640 1 --2- 172.21.15.32:0/3210029722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68c0158dd0 0x7f68c013fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:05.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.700+0000 7f68c5caf640 1 -- 172.21.15.32:0/3210029722 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68c006b860 msgr2=0x7f68c014f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:05.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.700+0000 7f68c5caf640 1 --2- 172.21.15.32:0/3210029722 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68c006b860 0x7f68c014f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:05.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.700+0000 7f68c5caf640 1 -- 172.21.15.32:0/3210029722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68c0158dd0 msgr2=0x7f68c013fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:05.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.700+0000 7f68c5caf640 1 --2- 172.21.15.32:0/3210029722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68c0158dd0 0x7f68c013fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:05.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.700+0000 7f68c5caf640 1 -- 172.21.15.32:0/3210029722 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68c015e560 con 0x7f68c006dc60 2024-04-24T22:59:05.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.701+0000 7f68c5caf640 1 --2- 172.21.15.32:0/3210029722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68c006dc60 0x7f68c015ddc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f68b400c4d0 tx=0x7f68b400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:05.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.701+0000 7f68beffd640 1 -- 172.21.15.32:0/3210029722 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b4018070 con 0x7f68c006dc60 2024-04-24T22:59:05.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.701+0000 7f68beffd640 1 -- 172.21.15.32:0/3210029722 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68b40074e0 con 0x7f68c006dc60 2024-04-24T22:59:05.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.701+0000 7f68c8714640 1 -- 172.21.15.32:0/3210029722 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68c015c250 con 0x7f68c006dc60 2024-04-24T22:59:05.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.701+0000 7f68beffd640 1 -- 172.21.15.32:0/3210029722 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b4004a20 con 0x7f68c006dc60 2024-04-24T22:59:05.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.701+0000 7f68c8714640 1 -- 172.21.15.32:0/3210029722 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68c015c810 con 0x7f68c006dc60 2024-04-24T22:59:05.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.702+0000 7f68c8714640 1 -- 172.21.15.32:0/3210029722 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68c0068430 con 0x7f68c006dc60 2024-04-24T22:59:05.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.702+0000 7f68beffd640 1 -- 172.21.15.32:0/3210029722 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f68b401b070 con 0x7f68c006dc60 2024-04-24T22:59:05.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.702+0000 7f68beffd640 1 --2- 172.21.15.32:0/3210029722 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f68ac06be30 0x7f68ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:05.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.703+0000 7f68beffd640 1 -- 172.21.15.32:0/3210029722 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f68b408a190 con 0x7f68c006dc60 2024-04-24T22:59:05.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.703+0000 7f68c64b0640 1 --2- 172.21.15.32:0/3210029722 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f68ac06be30 0x7f68ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:05.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.703+0000 7f68c64b0640 1 --2- 172.21.15.32:0/3210029722 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f68ac06be30 0x7f68ac06e2f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f68b8034000 tx=0x7f68b802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:05.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.707+0000 7f68beffd640 1 -- 172.21.15.32:0/3210029722 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f68b4058b30 con 0x7f68c006dc60 2024-04-24T22:59:05.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:05.930+0000 7f68c8714640 1 -- 172.21.15.32:0/3210029722 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f68c0162d90 con 0x7f68c006dc60 2024-04-24T22:59:06.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.539+0000 7f68beffd640 1 -- 172.21.15.32:0/3210029722 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v6) ==== 122+0+0 (secure 0 0 0) 0x7f68b4058430 con 0x7f68c006dc60 2024-04-24T22:59:06.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.542+0000 7f68c8714640 1 -- 172.21.15.32:0/3210029722 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f68ac06be30 msgr2=0x7f68ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:06.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.542+0000 7f68c8714640 1 --2- 172.21.15.32:0/3210029722 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f68ac06be30 0x7f68ac06e2f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f68b8034000 tx=0x7f68b802e000 comp rx=0 tx=0).stop 2024-04-24T22:59:06.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.542+0000 7f68c8714640 1 -- 172.21.15.32:0/3210029722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68c006dc60 msgr2=0x7f68c015ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:06.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.542+0000 7f68c8714640 1 --2- 172.21.15.32:0/3210029722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68c006dc60 0x7f68c015ddc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f68b400c4d0 tx=0x7f68b400c990 comp rx=0 tx=0).stop 2024-04-24T22:59:06.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.542+0000 7f68c8714640 1 -- 172.21.15.32:0/3210029722 shutdown_connections 2024-04-24T22:59:06.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.542+0000 7f68c8714640 1 --2- 172.21.15.32:0/3210029722 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f68ac06be30 0x7f68ac06e2f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:06.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.542+0000 7f68c8714640 1 --2- 172.21.15.32:0/3210029722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68c006dc60 0x7f68c015ddc0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:06.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.542+0000 7f68c8714640 1 --2- 172.21.15.32:0/3210029722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68c0158dd0 0x7f68c013fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:06.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.542+0000 7f68c8714640 1 --2- 172.21.15.32:0/3210029722 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68c006b860 0x7f68c014f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:06.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.542+0000 7f68c8714640 1 -- 172.21.15.32:0/3210029722 >> 172.21.15.32:0/3210029722 conn(0x7f68c006e800 msgr2=0x7f68c0160980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:06.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.542+0000 7f68c8714640 1 -- 172.21.15.32:0/3210029722 shutdown_connections 2024-04-24T22:59:06.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.542+0000 7f68c8714640 1 -- 172.21.15.32:0/3210029722 wait complete. 2024-04-24T22:59:06.557 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-24T22:59:06.557 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T22:59:06.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.663+0000 7efeb87ce640 1 Processor -- start 2024-04-24T22:59:06.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.663+0000 7efeb87ce640 1 -- start start 2024-04-24T22:59:06.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.664+0000 7efeb87ce640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efeb0158dd0 0x7efeb006b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:06.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.664+0000 7efeb87ce640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efeb006dc60 0x7efeb00b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:06.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.664+0000 7efeb87ce640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efeb006bd10 con 0x7efeb006b860 2024-04-24T22:59:06.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.664+0000 7efeb87ce640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7efeb015ef10 con 0x7efeb0158dd0 2024-04-24T22:59:06.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.664+0000 7efeb87ce640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7efeb015f1f0 con 0x7efeb006dc60 2024-04-24T22:59:06.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.664+0000 7efeb656a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efeb0158dd0 0x7efeb006b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:06.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.664+0000 7efeb5d69640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7efeb006b860 0x7efeb006d550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44840/0 (socket says 172.21.15.32:44840) 2024-04-24T22:59:06.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.664+0000 7efeb656a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efeb0158dd0 0x7efeb006b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44594/0 (socket says 172.21.15.32:44594) 2024-04-24T22:59:06.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.664+0000 7efeb5d69640 1 -- 172.21.15.32:0/1321803843 learned_addr learned my addr 172.21.15.32:0/1321803843 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:06.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.664+0000 7efeb656a640 1 -- 172.21.15.32:0/1321803843 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efeb006dc60 msgr2=0x7efeb00b5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:06.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.664+0000 7efeb6d6b640 1 --2- 172.21.15.32:0/1321803843 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efeb006dc60 0x7efeb00b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:06.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.664+0000 7efeb656a640 1 --2- 172.21.15.32:0/1321803843 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efeb006dc60 0x7efeb00b5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:06.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.664+0000 7efeb656a640 1 -- 172.21.15.32:0/1321803843 >> v1:172.21.15.32:6789/0 conn(0x7efeb006b860 legacy=0x7efeb006d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:06.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.664+0000 7efeb656a640 1 -- 172.21.15.32:0/1321803843 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeb015f4d0 con 0x7efeb0158dd0 2024-04-24T22:59:06.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.664+0000 7efeb5568640 1 -- 172.21.15.32:0/1321803843 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 172247796 0 0) 0x7efeb006bd10 con 0x7efeb006b860 2024-04-24T22:59:06.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.665+0000 7efeb656a640 1 --2- 172.21.15.32:0/1321803843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efeb0158dd0 0x7efeb006b320 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7efea80097a0 tx=0x7efea802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad20d50c5885b9e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:06.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.665+0000 7efeb5568640 1 -- 172.21.15.32:0/1321803843 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea8037070 con 0x7efeb0158dd0 2024-04-24T22:59:06.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.665+0000 7efeb5568640 1 -- 172.21.15.32:0/1321803843 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efea8032440 con 0x7efeb0158dd0 2024-04-24T22:59:06.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.665+0000 7efeb5568640 1 -- 172.21.15.32:0/1321803843 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea8032810 con 0x7efeb0158dd0 2024-04-24T22:59:06.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.666+0000 7efeb87ce640 1 -- 172.21.15.32:0/1321803843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efeb0158dd0 msgr2=0x7efeb006b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:06.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.666+0000 7efeb87ce640 1 --2- 172.21.15.32:0/1321803843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efeb0158dd0 0x7efeb006b320 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7efea80097a0 tx=0x7efea802a180 comp rx=0 tx=0).stop 2024-04-24T22:59:06.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.666+0000 7efeb87ce640 1 -- 172.21.15.32:0/1321803843 shutdown_connections 2024-04-24T22:59:06.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.666+0000 7efeb87ce640 1 --2- 172.21.15.32:0/1321803843 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efeb006dc60 0x7efeb00b5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:06.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.666+0000 7efeb87ce640 1 --2- 172.21.15.32:0/1321803843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efeb0158dd0 0x7efeb006b320 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:06.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.666+0000 7efeb87ce640 1 -- 172.21.15.32:0/1321803843 >> 172.21.15.32:0/1321803843 conn(0x7efeb006e800 msgr2=0x7efeb006ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:06.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.666+0000 7efeb87ce640 1 -- 172.21.15.32:0/1321803843 shutdown_connections 2024-04-24T22:59:06.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.666+0000 7efeb87ce640 1 -- 172.21.15.32:0/1321803843 wait complete. 2024-04-24T22:59:06.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.667+0000 7efeb87ce640 1 Processor -- start 2024-04-24T22:59:06.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.667+0000 7efeb87ce640 1 -- start start 2024-04-24T22:59:06.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.668+0000 7efeb87ce640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efeb006b860 0x7efeb01f4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:06.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.668+0000 7efeb87ce640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efeb006dc60 0x7efeb01f53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:06.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.668+0000 7efeb87ce640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efeb0158dd0 0x7efeb01f6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:06.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.668+0000 7efeb87ce640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7efeb006bfe0 con 0x7efeb0158dd0 2024-04-24T22:59:06.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.668+0000 7efeb87ce640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7efeb015fcd0 con 0x7efeb006b860 2024-04-24T22:59:06.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.668+0000 7efeb87ce640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7efeb01591b0 con 0x7efeb006dc60 2024-04-24T22:59:06.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.668+0000 7efeb6d6b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efeb0158dd0 0x7efeb01f6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:06.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.668+0000 7efeb6d6b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efeb0158dd0 0x7efeb01f6ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44602/0 (socket says 172.21.15.32:44602) 2024-04-24T22:59:06.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.668+0000 7efeb6d6b640 1 -- 172.21.15.32:0/3010411618 learned_addr learned my addr 172.21.15.32:0/3010411618 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:06.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.668+0000 7efeb5d69640 1 --2- 172.21.15.32:0/3010411618 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efeb006dc60 0x7efeb01f53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:06.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.668+0000 7efeb656a640 1 --2- 172.21.15.32:0/3010411618 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efeb006b860 0x7efeb01f4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:06.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.668+0000 7efeb6d6b640 1 -- 172.21.15.32:0/3010411618 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efeb006dc60 msgr2=0x7efeb01f53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:06.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.668+0000 7efeb6d6b640 1 --2- 172.21.15.32:0/3010411618 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efeb006dc60 0x7efeb01f53b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:06.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.668+0000 7efeb6d6b640 1 -- 172.21.15.32:0/3010411618 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efeb006b860 msgr2=0x7efeb01f4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:06.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.668+0000 7efeb6d6b640 1 --2- 172.21.15.32:0/3010411618 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efeb006b860 0x7efeb01f4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:06.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.668+0000 7efeb6d6b640 1 -- 172.21.15.32:0/3010411618 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeb01f5180 con 0x7efeb0158dd0 2024-04-24T22:59:06.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.669+0000 7efeb6d6b640 1 --2- 172.21.15.32:0/3010411618 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efeb0158dd0 0x7efeb01f6ec0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efea000cd70 tx=0x7efea000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:06.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.669+0000 7efeaeffd640 1 -- 172.21.15.32:0/3010411618 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea0003b90 con 0x7efeb0158dd0 2024-04-24T22:59:06.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.669+0000 7efeaeffd640 1 -- 172.21.15.32:0/3010411618 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efea0003d30 con 0x7efeb0158dd0 2024-04-24T22:59:06.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.669+0000 7efeb87ce640 1 -- 172.21.15.32:0/3010411618 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efeb013e150 con 0x7efeb0158dd0 2024-04-24T22:59:06.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.669+0000 7efeaeffd640 1 -- 172.21.15.32:0/3010411618 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea0004240 con 0x7efeb0158dd0 2024-04-24T22:59:06.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.669+0000 7efeb87ce640 1 -- 172.21.15.32:0/3010411618 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efeb013e710 con 0x7efeb0158dd0 2024-04-24T22:59:06.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.670+0000 7efeb87ce640 1 -- 172.21.15.32:0/3010411618 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efeb0068430 con 0x7efeb0158dd0 2024-04-24T22:59:06.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.670+0000 7efeaeffd640 1 -- 172.21.15.32:0/3010411618 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efea00043e0 con 0x7efeb0158dd0 2024-04-24T22:59:06.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.671+0000 7efeaeffd640 1 --2- 172.21.15.32:0/3010411618 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7efe9c06be60 0x7efe9c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:06.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.671+0000 7efeaeffd640 1 -- 172.21.15.32:0/3010411618 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7efea008b930 con 0x7efeb0158dd0 2024-04-24T22:59:06.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.671+0000 7efeb656a640 1 --2- 172.21.15.32:0/3010411618 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7efe9c06be60 0x7efe9c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:06.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.671+0000 7efeb656a640 1 --2- 172.21.15.32:0/3010411618 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7efe9c06be60 0x7efe9c06e320 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7efea8034000 tx=0x7efea802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:06.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.675+0000 7efeaeffd640 1 -- 172.21.15.32:0/3010411618 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7efea005a2e0 con 0x7efeb0158dd0 2024-04-24T22:59:06.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.895+0000 7efeb87ce640 1 -- 172.21.15.32:0/3010411618 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efeb0068ca0 con 0x7efeb0158dd0 2024-04-24T22:59:06.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.895+0000 7efeaeffd640 1 -- 172.21.15.32:0/3010411618 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5178 (secure 0 0 0) 0x7efea0059be0 con 0x7efeb0158dd0 2024-04-24T22:59:06.897 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:59:06.897 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1485884060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1485884060},{"type":"v1","addr":"172.21.15.121:6837","nonce":1485884060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/923934104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":923934104},{"type":"v1","addr":"172.21.15.32:6833","nonce":923934104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3229115835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3229115835},{"type":"v1","addr":"172.21.15.32:6835","nonce":3229115835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T22:59:03.876305+0000","modified":"2024-04-24T22:59:05.932147+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6836/2644625343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2644625343},{"type":"v1","addr":"172.21.15.121:6836","nonce":2644625343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4276]},"id":1}]} 2024-04-24T22:59:06.898 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 6 2024-04-24T22:59:06.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.897+0000 7efeb87ce640 1 -- 172.21.15.32:0/3010411618 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7efe9c06be60 msgr2=0x7efe9c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:06.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.897+0000 7efeb87ce640 1 --2- 172.21.15.32:0/3010411618 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7efe9c06be60 0x7efe9c06e320 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7efea8034000 tx=0x7efea802e000 comp rx=0 tx=0).stop 2024-04-24T22:59:06.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.897+0000 7efeb87ce640 1 -- 172.21.15.32:0/3010411618 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efeb0158dd0 msgr2=0x7efeb01f6ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:06.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.897+0000 7efeb87ce640 1 --2- 172.21.15.32:0/3010411618 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efeb0158dd0 0x7efeb01f6ec0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efea000cd70 tx=0x7efea000a660 comp rx=0 tx=0).stop 2024-04-24T22:59:06.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.897+0000 7efeb87ce640 1 -- 172.21.15.32:0/3010411618 shutdown_connections 2024-04-24T22:59:06.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.897+0000 7efeb87ce640 1 --2- 172.21.15.32:0/3010411618 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7efe9c06be60 0x7efe9c06e320 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:06.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.897+0000 7efeb87ce640 1 --2- 172.21.15.32:0/3010411618 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efeb0158dd0 0x7efeb01f6ec0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:06.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.897+0000 7efeb87ce640 1 --2- 172.21.15.32:0/3010411618 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efeb006dc60 0x7efeb01f53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:06.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.897+0000 7efeb87ce640 1 --2- 172.21.15.32:0/3010411618 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efeb006b860 0x7efeb01f4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:06.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.897+0000 7efeb87ce640 1 -- 172.21.15.32:0/3010411618 >> 172.21.15.32:0/3010411618 conn(0x7efeb006e800 msgr2=0x7efeb0161840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:06.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.897+0000 7efeb87ce640 1 -- 172.21.15.32:0/3010411618 shutdown_connections 2024-04-24T22:59:06.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:06.897+0000 7efeb87ce640 1 -- 172.21.15.32:0/3010411618 wait complete. 2024-04-24T22:59:06.912 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:59:07.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.011+0000 7f71818bd640 1 Processor -- start 2024-04-24T22:59:07.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.011+0000 7f71818bd640 1 -- start start 2024-04-24T22:59:07.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.012+0000 7f71818bd640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f717c15fd60 0x7f717c06c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:07.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.012+0000 7f71818bd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f717c06cb30 0x7f717c0cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:07.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.012+0000 7f71818bd640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f717c0b46c0 con 0x7f717c15b480 2024-04-24T22:59:07.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.012+0000 7f71818bd640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f717c0b4ff0 con 0x7f717c06cb30 2024-04-24T22:59:07.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.012+0000 7f71818bd640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f717c0b49a0 con 0x7f717c15fd60 2024-04-24T22:59:07.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.012+0000 7f717b7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f717c06cb30 0x7f717c0cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:07.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.012+0000 7f717b7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f717c06cb30 0x7f717c0cdf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44612/0 (socket says 172.21.15.32:44612) 2024-04-24T22:59:07.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.012+0000 7f717b7fe640 1 -- 172.21.15.32:0/2355138081 learned_addr learned my addr 172.21.15.32:0/2355138081 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:07.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.012+0000 7f717a7fc640 1 --2- 172.21.15.32:0/2355138081 >> v2:172.21.15.121:3300/0 conn(0x7f717c15fd60 0x7f717c06c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:07.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.012+0000 7f717b7fe640 1 -- 172.21.15.32:0/2355138081 >> v2:172.21.15.121:3300/0 conn(0x7f717c15fd60 msgr2=0x7f717c06c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:07.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.012+0000 7f717b7fe640 1 --2- 172.21.15.32:0/2355138081 >> v2:172.21.15.121:3300/0 conn(0x7f717c15fd60 0x7f717c06c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:07.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.012+0000 7f717b7fe640 1 -- 172.21.15.32:0/2355138081 >> v1:172.21.15.121:6789/0 conn(0x7f717c15b480 legacy=0x7f717c0b5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:07.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.012+0000 7f717b7fe640 1 -- 172.21.15.32:0/2355138081 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f717c0b4c80 con 0x7f717c06cb30 2024-04-24T22:59:07.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.013+0000 7f717b7fe640 1 --2- 172.21.15.32:0/2355138081 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f717c06cb30 0x7f717c0cdf80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f717c06ee10 tx=0x7f716c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d6acc91c78e4871 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:07.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.013+0000 7f7179ffb640 1 -- 172.21.15.32:0/2355138081 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716c036070 con 0x7f717c06cb30 2024-04-24T22:59:07.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.013+0000 7f7179ffb640 1 -- 172.21.15.32:0/2355138081 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f716c031600 con 0x7f717c06cb30 2024-04-24T22:59:07.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.013+0000 7f7179ffb640 1 -- 172.21.15.32:0/2355138081 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716c0319d0 con 0x7f717c06cb30 2024-04-24T22:59:07.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.014+0000 7f71818bd640 1 -- 172.21.15.32:0/2355138081 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f717c06cb30 msgr2=0x7f717c0cdf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:07.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.014+0000 7f71818bd640 1 --2- 172.21.15.32:0/2355138081 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f717c06cb30 0x7f717c0cdf80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f717c06ee10 tx=0x7f716c02a730 comp rx=0 tx=0).stop 2024-04-24T22:59:07.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.014+0000 7f71818bd640 1 -- 172.21.15.32:0/2355138081 shutdown_connections 2024-04-24T22:59:07.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.014+0000 7f71818bd640 1 --2- 172.21.15.32:0/2355138081 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f717c06cb30 0x7f717c0cdf80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:07.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.014+0000 7f71818bd640 1 --2- 172.21.15.32:0/2355138081 >> v2:172.21.15.121:3300/0 conn(0x7f717c15fd60 0x7f717c06c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:07.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.014+0000 7f71818bd640 1 -- 172.21.15.32:0/2355138081 >> 172.21.15.32:0/2355138081 conn(0x7f717c06b110 msgr2=0x7f717c06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:07.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.014+0000 7f71818bd640 1 -- 172.21.15.32:0/2355138081 shutdown_connections 2024-04-24T22:59:07.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.014+0000 7f71818bd640 1 -- 172.21.15.32:0/2355138081 wait complete. 2024-04-24T22:59:07.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.015+0000 7f71818bd640 1 Processor -- start 2024-04-24T22:59:07.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.015+0000 7f71818bd640 1 -- start start 2024-04-24T22:59:07.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.016+0000 7f71818bd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f717c06cb30 0x7f717c1402d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:07.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.016+0000 7f71818bd640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f717c15b480 0x7f717c15d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:07.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.016+0000 7f71818bd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f717c15fd60 0x7f717c15d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:07.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.016+0000 7f71818bd640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f717c15b860 con 0x7f717c15fd60 2024-04-24T22:59:07.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.016+0000 7f71818bd640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f717c0ce4c0 con 0x7f717c06cb30 2024-04-24T22:59:07.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.016+0000 7f71818bd640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f717c1439b0 con 0x7f717c15b480 2024-04-24T22:59:07.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.016+0000 7f717b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f717c15fd60 0x7f717c15d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:07.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.016+0000 7f717b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f717c15fd60 0x7f717c15d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44628/0 (socket says 172.21.15.32:44628) 2024-04-24T22:59:07.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.016+0000 7f717b7fe640 1 -- 172.21.15.32:0/421204695 learned_addr learned my addr 172.21.15.32:0/421204695 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.016+0000 7f717a7fc640 1 --2- 172.21.15.32:0/421204695 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f717c15b480 0x7f717c15d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.016+0000 7f717affd640 1 --2- 172.21.15.32:0/421204695 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f717c06cb30 0x7f717c1402d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.016+0000 7f717b7fe640 1 -- 172.21.15.32:0/421204695 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f717c15b480 msgr2=0x7f717c15d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.016+0000 7f717b7fe640 1 --2- 172.21.15.32:0/421204695 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f717c15b480 0x7f717c15d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.016+0000 7f717b7fe640 1 -- 172.21.15.32:0/421204695 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f717c06cb30 msgr2=0x7f717c1402d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.016+0000 7f717b7fe640 1 --2- 172.21.15.32:0/421204695 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f717c06cb30 0x7f717c1402d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.016+0000 7f717b7fe640 1 -- 172.21.15.32:0/421204695 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f717c140b00 con 0x7f717c15fd60 2024-04-24T22:59:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.016+0000 7f717a7fc640 1 --2- 172.21.15.32:0/421204695 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f717c15b480 0x7f717c15d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.016+0000 7f717b7fe640 1 --2- 172.21.15.32:0/421204695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f717c15fd60 0x7f717c15d540 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f716c009f40 tx=0x7f716c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.017+0000 7f7173fff640 1 -- 172.21.15.32:0/421204695 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716c036050 con 0x7f717c15fd60 2024-04-24T22:59:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.017+0000 7f7173fff640 1 -- 172.21.15.32:0/421204695 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f716c042070 con 0x7f717c15fd60 2024-04-24T22:59:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.017+0000 7f71818bd640 1 -- 172.21.15.32:0/421204695 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f717c140de0 con 0x7f717c15fd60 2024-04-24T22:59:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.017+0000 7f7173fff640 1 -- 172.21.15.32:0/421204695 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716c009510 con 0x7f717c15fd60 2024-04-24T22:59:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.017+0000 7f71818bd640 1 -- 172.21.15.32:0/421204695 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f717c1413a0 con 0x7f717c15fd60 2024-04-24T22:59:07.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.018+0000 7f71818bd640 1 -- 172.21.15.32:0/421204695 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f717c068430 con 0x7f717c15fd60 2024-04-24T22:59:07.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.018+0000 7f7173fff640 1 -- 172.21.15.32:0/421204695 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f716c007bf0 con 0x7f717c15fd60 2024-04-24T22:59:07.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.019+0000 7f7173fff640 1 --2- 172.21.15.32:0/421204695 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f715c06be30 0x7f715c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:07.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.019+0000 7f7173fff640 1 -- 172.21.15.32:0/421204695 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f716c0abb10 con 0x7f717c15fd60 2024-04-24T22:59:07.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.019+0000 7f717affd640 1 --2- 172.21.15.32:0/421204695 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f715c06be30 0x7f715c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:07.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.019+0000 7f717affd640 1 --2- 172.21.15.32:0/421204695 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f715c06be30 0x7f715c06e2f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7174000f60 tx=0x7f7174009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:07.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.023+0000 7f7173fff640 1 -- 172.21.15.32:0/421204695 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f716c07a4c0 con 0x7f717c15fd60 2024-04-24T22:59:07.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.208+0000 7f71818bd640 1 -- 172.21.15.32:0/421204695 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f717c15fb40 con 0x7f717c15fd60 2024-04-24T22:59:07.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.210+0000 7f7173fff640 1 -- 172.21.15.32:0/421204695 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16621 (secure 0 0 0) 0x7f716c079dc0 con 0x7f717c15fd60 2024-04-24T22:59:07.211 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:59:07.211 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T22:59:05.517554+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:59:02.016639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T22:59:02.846087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:59:07.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.212+0000 7f71818bd640 1 -- 172.21.15.32:0/421204695 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f715c06be30 msgr2=0x7f715c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:07.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.212+0000 7f71818bd640 1 --2- 172.21.15.32:0/421204695 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f715c06be30 0x7f715c06e2f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7174000f60 tx=0x7f7174009040 comp rx=0 tx=0).stop 2024-04-24T22:59:07.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.212+0000 7f71818bd640 1 -- 172.21.15.32:0/421204695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f717c15fd60 msgr2=0x7f717c15d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:07.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.212+0000 7f71818bd640 1 --2- 172.21.15.32:0/421204695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f717c15fd60 0x7f717c15d540 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f716c009f40 tx=0x7f716c004a40 comp rx=0 tx=0).stop 2024-04-24T22:59:07.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.212+0000 7f71818bd640 1 -- 172.21.15.32:0/421204695 shutdown_connections 2024-04-24T22:59:07.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.212+0000 7f71818bd640 1 --2- 172.21.15.32:0/421204695 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f715c06be30 0x7f715c06e2f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:07.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.212+0000 7f71818bd640 1 --2- 172.21.15.32:0/421204695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f717c15fd60 0x7f717c15d540 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:07.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.212+0000 7f71818bd640 1 --2- 172.21.15.32:0/421204695 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f717c15b480 0x7f717c15d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:07.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.212+0000 7f71818bd640 1 --2- 172.21.15.32:0/421204695 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f717c06cb30 0x7f717c1402d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:07.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.212+0000 7f71818bd640 1 -- 172.21.15.32:0/421204695 >> 172.21.15.32:0/421204695 conn(0x7f717c06b110 msgr2=0x7f717c0b0760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:07.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.212+0000 7f71818bd640 1 -- 172.21.15.32:0/421204695 shutdown_connections 2024-04-24T22:59:07.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.212+0000 7f71818bd640 1 -- 172.21.15.32:0/421204695 wait complete. 2024-04-24T22:59:07.225 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-24T22:59:07.225 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T22:59:07.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.325+0000 7f716dd29640 1 Processor -- start 2024-04-24T22:59:07.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.325+0000 7f716dd29640 1 -- start start 2024-04-24T22:59:07.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.325+0000 7f716dd29640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f716815e9b0 0x7f71680d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:07.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.325+0000 7f716dd29640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f716806b820 0x7f71680b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:07.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.325+0000 7f716dd29640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71680d3170 con 0x7f71680d4210 2024-04-24T22:59:07.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.325+0000 7f716dd29640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f71680d3450 con 0x7f716806b820 2024-04-24T22:59:07.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.325+0000 7f716dd29640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f71680d3730 con 0x7f716815e9b0 2024-04-24T22:59:07.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.326+0000 7f7167fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f716806b820 0x7f71680b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:07.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.326+0000 7f7167fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f716806b820 0x7f71680b78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44644/0 (socket says 172.21.15.32:44644) 2024-04-24T22:59:07.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.326+0000 7f7167fff640 1 -- 172.21.15.32:0/888691176 learned_addr learned my addr 172.21.15.32:0/888691176 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:07.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.326+0000 7f7166ffd640 1 --1- 172.21.15.32:0/888691176 >> v1:172.21.15.32:6789/0 conn(0x7f71680d4210 0x7f716806b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44850/0 (socket says 172.21.15.32:44850) 2024-04-24T22:59:07.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.326+0000 7f71677fe640 1 --2- 172.21.15.32:0/888691176 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f716815e9b0 0x7f71680d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:07.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.326+0000 7f7167fff640 1 -- 172.21.15.32:0/888691176 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f716815e9b0 msgr2=0x7f71680d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:07.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.326+0000 7f7167fff640 1 --2- 172.21.15.32:0/888691176 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f716815e9b0 0x7f71680d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:07.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.326+0000 7f7167fff640 1 -- 172.21.15.32:0/888691176 >> v1:172.21.15.32:6789/0 conn(0x7f71680d4210 legacy=0x7f716806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:07.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.326+0000 7f7167fff640 1 -- 172.21.15.32:0/888691176 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71680d3a10 con 0x7f716806b820 2024-04-24T22:59:07.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.327+0000 7f7167fff640 1 --2- 172.21.15.32:0/888691176 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f716806b820 0x7f71680b78f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7160009e50 tx=0x7f716002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=526408cc816c8bb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:07.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.327+0000 7f71667fc640 1 -- 172.21.15.32:0/888691176 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7160036070 con 0x7f716806b820 2024-04-24T22:59:07.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.327+0000 7f71667fc640 1 -- 172.21.15.32:0/888691176 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7160031600 con 0x7f716806b820 2024-04-24T22:59:07.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.327+0000 7f71667fc640 1 -- 172.21.15.32:0/888691176 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71600319d0 con 0x7f716806b820 2024-04-24T22:59:07.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.327+0000 7f716dd29640 1 -- 172.21.15.32:0/888691176 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f716806b820 msgr2=0x7f71680b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:07.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.327+0000 7f716dd29640 1 --2- 172.21.15.32:0/888691176 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f716806b820 0x7f71680b78f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7160009e50 tx=0x7f716002a710 comp rx=0 tx=0).stop 2024-04-24T22:59:07.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.327+0000 7f716dd29640 1 -- 172.21.15.32:0/888691176 shutdown_connections 2024-04-24T22:59:07.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.327+0000 7f716dd29640 1 --2- 172.21.15.32:0/888691176 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f716806b820 0x7f71680b78f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:07.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.327+0000 7f716dd29640 1 --2- 172.21.15.32:0/888691176 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f716815e9b0 0x7f71680d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:07.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.327+0000 7f716dd29640 1 -- 172.21.15.32:0/888691176 >> 172.21.15.32:0/888691176 conn(0x7f71680d11d0 msgr2=0x7f71680d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:07.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.327+0000 7f716dd29640 1 -- 172.21.15.32:0/888691176 shutdown_connections 2024-04-24T22:59:07.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.327+0000 7f716dd29640 1 -- 172.21.15.32:0/888691176 wait complete. 2024-04-24T22:59:07.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.328+0000 7f716dd29640 1 Processor -- start 2024-04-24T22:59:07.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.329+0000 7f716dd29640 1 -- start start 2024-04-24T22:59:07.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.329+0000 7f716dd29640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f716806b820 0x7f71680cb820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:07.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.329+0000 7f716dd29640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71680d4210 0x7f71681f4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:07.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.329+0000 7f716dd29640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f716815e9b0 0x7f71681f4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:07.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.329+0000 7f716dd29640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f71680ba0e0 con 0x7f716806b820 2024-04-24T22:59:07.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.329+0000 7f716dd29640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f71680d2400 con 0x7f71680d4210 2024-04-24T22:59:07.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.329+0000 7f716dd29640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f71680ba260 con 0x7f716815e9b0 2024-04-24T22:59:07.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.329+0000 7f71677fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f716806b820 0x7f71680cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:07.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.329+0000 7f71677fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f716806b820 0x7f71680cb820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44658/0 (socket says 172.21.15.32:44658) 2024-04-24T22:59:07.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.329+0000 7f71677fe640 1 -- 172.21.15.32:0/1803308705 learned_addr learned my addr 172.21.15.32:0/1803308705 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:07.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.330+0000 7f7167fff640 1 --2- 172.21.15.32:0/1803308705 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f716815e9b0 0x7f71681f4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:07.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.330+0000 7f7166ffd640 1 --2- 172.21.15.32:0/1803308705 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71680d4210 0x7f71681f4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:07.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.330+0000 7f71677fe640 1 -- 172.21.15.32:0/1803308705 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f716815e9b0 msgr2=0x7f71681f4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:07.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.330+0000 7f71677fe640 1 --2- 172.21.15.32:0/1803308705 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f716815e9b0 0x7f71681f4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:07.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.330+0000 7f71677fe640 1 -- 172.21.15.32:0/1803308705 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71680d4210 msgr2=0x7f71681f4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:07.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.330+0000 7f71677fe640 1 --2- 172.21.15.32:0/1803308705 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71680d4210 0x7f71681f4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:07.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.330+0000 7f71677fe640 1 -- 172.21.15.32:0/1803308705 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71680cc190 con 0x7f716806b820 2024-04-24T22:59:07.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.330+0000 7f7166ffd640 1 --2- 172.21.15.32:0/1803308705 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71680d4210 0x7f71681f4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:59:07.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.330+0000 7f71677fe640 1 --2- 172.21.15.32:0/1803308705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f716806b820 0x7f71680cb820 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f715800d620 tx=0x7f715800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:07.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.330+0000 7f714ffff640 1 -- 172.21.15.32:0/1803308705 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715800b620 con 0x7f716806b820 2024-04-24T22:59:07.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.330+0000 7f714ffff640 1 -- 172.21.15.32:0/1803308705 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7158019070 con 0x7f716806b820 2024-04-24T22:59:07.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.330+0000 7f716dd29640 1 -- 172.21.15.32:0/1803308705 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71680cc470 con 0x7f716806b820 2024-04-24T22:59:07.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.330+0000 7f714ffff640 1 -- 172.21.15.32:0/1803308705 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715800ba00 con 0x7f716806b820 2024-04-24T22:59:07.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.330+0000 7f716dd29640 1 -- 172.21.15.32:0/1803308705 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71680cca30 con 0x7f716806b820 2024-04-24T22:59:07.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.331+0000 7f716dd29640 1 -- 172.21.15.32:0/1803308705 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7168068430 con 0x7f716806b820 2024-04-24T22:59:07.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.331+0000 7f714ffff640 1 -- 172.21.15.32:0/1803308705 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f715800bba0 con 0x7f716806b820 2024-04-24T22:59:07.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.332+0000 7f714ffff640 1 --2- 172.21.15.32:0/1803308705 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f715406be30 0x7f715406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:07.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.332+0000 7f714ffff640 1 -- 172.21.15.32:0/1803308705 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f715808b260 con 0x7f716806b820 2024-04-24T22:59:07.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.332+0000 7f7166ffd640 1 --2- 172.21.15.32:0/1803308705 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f715406be30 0x7f715406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:07.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.333+0000 7f7166ffd640 1 --2- 172.21.15.32:0/1803308705 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f715406be30 0x7f715406e2f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7168161f50 tx=0x7f715c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:07.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.336+0000 7f714ffff640 1 -- 172.21.15.32:0/1803308705 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7158059c10 con 0x7f716806b820 2024-04-24T22:59:07.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.515+0000 7f716dd29640 1 -- 172.21.15.32:0/1803308705 --> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f71680ccda0 con 0x7f715406be30 2024-04-24T22:59:07.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.521+0000 7f714ffff640 1 -- 172.21.15.32:0/1803308705 <== mgr.4108 v2:172.21.15.121:6832/1848157943 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+236279 (secure 0 0 0) 0x7f71680ccda0 con 0x7f715406be30 2024-04-24T22:59:07.528 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:59:07.530 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-24T22:59:07.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.524+0000 7f716dd29640 1 -- 172.21.15.32:0/1803308705 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f715406be30 msgr2=0x7f715406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:07.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.524+0000 7f716dd29640 1 --2- 172.21.15.32:0/1803308705 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f715406be30 0x7f715406e2f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7168161f50 tx=0x7f715c009040 comp rx=0 tx=0).stop 2024-04-24T22:59:07.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.524+0000 7f716dd29640 1 -- 172.21.15.32:0/1803308705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f716806b820 msgr2=0x7f71680cb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:07.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.524+0000 7f716dd29640 1 --2- 172.21.15.32:0/1803308705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f716806b820 0x7f71680cb820 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f715800d620 tx=0x7f715800dae0 comp rx=0 tx=0).stop 2024-04-24T22:59:07.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.524+0000 7f716dd29640 1 -- 172.21.15.32:0/1803308705 shutdown_connections 2024-04-24T22:59:07.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.524+0000 7f716dd29640 1 --2- 172.21.15.32:0/1803308705 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f715406be30 0x7f715406e2f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:07.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.524+0000 7f716dd29640 1 --2- 172.21.15.32:0/1803308705 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f716815e9b0 0x7f71681f4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:07.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.524+0000 7f716dd29640 1 --2- 172.21.15.32:0/1803308705 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71680d4210 0x7f71681f4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:07.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.524+0000 7f716dd29640 1 --2- 172.21.15.32:0/1803308705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f716806b820 0x7f71680cb820 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:07.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.524+0000 7f716dd29640 1 -- 172.21.15.32:0/1803308705 >> 172.21.15.32:0/1803308705 conn(0x7f71680d11d0 msgr2=0x7f71680b8c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:07.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.524+0000 7f716dd29640 1 -- 172.21.15.32:0/1803308705 shutdown_connections 2024-04-24T22:59:07.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.524+0000 7f716dd29640 1 -- 172.21.15.32:0/1803308705 wait complete. 2024-04-24T22:59:07.536 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-04-24T22:59:07.128866+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1545,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560954,"statfs":{"total":1600000000,"available":1598417040,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.633234"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":162,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-24T22:58:57.295146+0000","last_change":"2024-04-24T22:58:55.138810+0000","last_active":"2024-04-24T22:58:57.295146+0000","last_peered":"2024-04-24T22:58:57.295146+0000","last_clean":"2024-04-24T22:58:57.295146+0000","last_became_active":"2024-04-24T22:58:55.138627+0000","last_became_peered":"2024-04-24T22:58:55.138627+0000","last_unstale":"2024-04-24T22:58:57.295146+0000","last_undegraded":"2024-04-24T22:58:57.295146+0000","last_fullsized":"2024-04-24T22:58:57.295146+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_clean_scrub_stamp":"2024-04-24T22:58:54.127235+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:24.010002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195263,"statfs":{"total":200000000,"available":199949722,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195263,"statfs":{"total":200000000,"available":199949722,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":625,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194686,"statfs":{"total":200000000,"available":199359354,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195263,"statfs":{"total":200000000,"available":199949722,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195263,"statfs":{"total":200000000,"available":199949722,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195263,"statfs":{"total":200000000,"available":199949722,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":625,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194686,"statfs":{"total":200000000,"available":199359354,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195263,"statfs":{"total":200000000,"available":199949722,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-24T22:59:07.541 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T22:59:07.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.675+0000 7f843ea93640 1 Processor -- start 2024-04-24T22:59:07.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.675+0000 7f843ea93640 1 -- start start 2024-04-24T22:59:07.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.676+0000 7f843ea93640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f843815b380 0x7f843806c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:07.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.676+0000 7f843ea93640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f843815cb00 0x7f843806cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:07.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.676+0000 7f843ea93640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84380cef50 con 0x7f84380ce430 2024-04-24T22:59:07.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.676+0000 7f843ea93640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f84380b4520 con 0x7f843815b380 2024-04-24T22:59:07.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.676+0000 7f843ea93640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f84380b4800 con 0x7f843815cb00 2024-04-24T22:59:07.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.676+0000 7f843c82f640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f843815b380 0x7f843806c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:07.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.676+0000 7f843d030640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f84380ce430 0x7f84380ce840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44854/0 (socket says 172.21.15.32:44854) 2024-04-24T22:59:07.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.676+0000 7f843d030640 1 -- 172.21.15.32:0/242164128 learned_addr learned my addr 172.21.15.32:0/242164128 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:07.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.676+0000 7f843c82f640 1 -- 172.21.15.32:0/242164128 >> v2:172.21.15.121:3300/0 conn(0x7f843815cb00 msgr2=0x7f843806cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:07.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.676+0000 7f8437fff640 1 --2- 172.21.15.32:0/242164128 >> v2:172.21.15.121:3300/0 conn(0x7f843815cb00 0x7f843806cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:07.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.676+0000 7f84377fe640 1 -- 172.21.15.32:0/242164128 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1404998199 0 0) 0x7f8430001490 con 0x7f84380ce430 2024-04-24T22:59:07.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.676+0000 7f843c82f640 1 --2- 172.21.15.32:0/242164128 >> v2:172.21.15.121:3300/0 conn(0x7f843815cb00 0x7f843806cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:07.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.676+0000 7f843c82f640 1 -- 172.21.15.32:0/242164128 >> v1:172.21.15.32:6789/0 conn(0x7f84380ce430 legacy=0x7f84380ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:07.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.676+0000 7f843c82f640 1 -- 172.21.15.32:0/242164128 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84380b4ae0 con 0x7f843815b380 2024-04-24T22:59:07.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.676+0000 7f8437fff640 1 --2- 172.21.15.32:0/242164128 >> v2:172.21.15.121:3300/0 conn(0x7f843815cb00 0x7f843806cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:59:07.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.677+0000 7f843c82f640 1 --2- 172.21.15.32:0/242164128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f843815b380 0x7f843806c5f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f84380d12e0 tx=0x7f842c02a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d9ae71553072cbc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:07.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.677+0000 7f84377fe640 1 -- 172.21.15.32:0/242164128 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842c037070 con 0x7f843815b380 2024-04-24T22:59:07.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.677+0000 7f84377fe640 1 -- 172.21.15.32:0/242164128 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f842c02add0 con 0x7f843815b380 2024-04-24T22:59:07.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.677+0000 7f84377fe640 1 -- 172.21.15.32:0/242164128 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842c032660 con 0x7f843815b380 2024-04-24T22:59:07.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.678+0000 7f843ea93640 1 -- 172.21.15.32:0/242164128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f843815b380 msgr2=0x7f843806c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:07.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.678+0000 7f843ea93640 1 --2- 172.21.15.32:0/242164128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f843815b380 0x7f843806c5f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f84380d12e0 tx=0x7f842c02a080 comp rx=0 tx=0).stop 2024-04-24T22:59:07.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.678+0000 7f843ea93640 1 -- 172.21.15.32:0/242164128 shutdown_connections 2024-04-24T22:59:07.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.678+0000 7f843ea93640 1 --2- 172.21.15.32:0/242164128 >> v2:172.21.15.121:3300/0 conn(0x7f843815cb00 0x7f843806cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:07.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.678+0000 7f843ea93640 1 --2- 172.21.15.32:0/242164128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f843815b380 0x7f843806c5f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:07.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.678+0000 7f843ea93640 1 -- 172.21.15.32:0/242164128 >> 172.21.15.32:0/242164128 conn(0x7f843806b110 msgr2=0x7f843806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:07.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.678+0000 7f843ea93640 1 -- 172.21.15.32:0/242164128 shutdown_connections 2024-04-24T22:59:07.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.678+0000 7f843ea93640 1 -- 172.21.15.32:0/242164128 wait complete. 2024-04-24T22:59:07.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.679+0000 7f843ea93640 1 Processor -- start 2024-04-24T22:59:07.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.679+0000 7f843ea93640 1 -- start start 2024-04-24T22:59:07.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.680+0000 7f843ea93640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84380ce430 0x7f8438147680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:07.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.680+0000 7f843ea93640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f843815b380 0x7f84381516b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:07.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.680+0000 7f843ea93640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f843815cb00 0x7f8438155f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:07.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.680+0000 7f843ea93640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f843815b760 con 0x7f843815b380 2024-04-24T22:59:07.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.680+0000 7f843ea93640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f84380b50a0 con 0x7f843815cb00 2024-04-24T22:59:07.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.680+0000 7f843ea93640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f84380b55c0 con 0x7f84380ce430 2024-04-24T22:59:07.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.680+0000 7f8437fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f843815b380 0x7f84381516b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:07.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.680+0000 7f8437fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f843815b380 0x7f84381516b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44674/0 (socket says 172.21.15.32:44674) 2024-04-24T22:59:07.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.680+0000 7f8437fff640 1 -- 172.21.15.32:0/967030414 learned_addr learned my addr 172.21.15.32:0/967030414 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:07.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.680+0000 7f843c82f640 1 --2- 172.21.15.32:0/967030414 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84380ce430 0x7f8438147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:07.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.680+0000 7f843d030640 1 --2- 172.21.15.32:0/967030414 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f843815cb00 0x7f8438155f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:07.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.680+0000 7f8437fff640 1 -- 172.21.15.32:0/967030414 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84380ce430 msgr2=0x7f8438147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:07.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.680+0000 7f8437fff640 1 --2- 172.21.15.32:0/967030414 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84380ce430 0x7f8438147680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:07.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.680+0000 7f8437fff640 1 -- 172.21.15.32:0/967030414 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f843815cb00 msgr2=0x7f8438155f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:07.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.680+0000 7f8437fff640 1 --2- 172.21.15.32:0/967030414 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f843815cb00 0x7f8438155f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:07.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.680+0000 7f8437fff640 1 -- 172.21.15.32:0/967030414 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84381f2aa0 con 0x7f843815b380 2024-04-24T22:59:07.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.680+0000 7f843d030640 1 --2- 172.21.15.32:0/967030414 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f843815cb00 0x7f8438155f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:07.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.680+0000 7f843c82f640 1 --2- 172.21.15.32:0/967030414 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84380ce430 0x7f8438147680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:07.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.680+0000 7f8437fff640 1 --2- 172.21.15.32:0/967030414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f843815b380 0x7f84381516b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f842400c7f0 tx=0x7f842400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:07.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.680+0000 7f84357fa640 1 -- 172.21.15.32:0/967030414 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8424015070 con 0x7f843815b380 2024-04-24T22:59:07.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.680+0000 7f84357fa640 1 -- 172.21.15.32:0/967030414 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8424007820 con 0x7f843815b380 2024-04-24T22:59:07.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.680+0000 7f843ea93640 1 -- 172.21.15.32:0/967030414 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8438151bf0 con 0x7f843815b380 2024-04-24T22:59:07.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.681+0000 7f84357fa640 1 -- 172.21.15.32:0/967030414 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8424007bd0 con 0x7f843815b380 2024-04-24T22:59:07.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.681+0000 7f843ea93640 1 -- 172.21.15.32:0/967030414 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8438155160 con 0x7f843815b380 2024-04-24T22:59:07.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.682+0000 7f843ea93640 1 -- 172.21.15.32:0/967030414 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84380af760 con 0x7f843815b380 2024-04-24T22:59:07.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.682+0000 7f84357fa640 1 -- 172.21.15.32:0/967030414 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8424018070 con 0x7f843815b380 2024-04-24T22:59:07.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.682+0000 7f84357fa640 1 --2- 172.21.15.32:0/967030414 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f841c06be30 0x7f841c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:07.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.683+0000 7f84357fa640 1 -- 172.21.15.32:0/967030414 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f842408af00 con 0x7f843815b380 2024-04-24T22:59:07.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.683+0000 7f843c82f640 1 --2- 172.21.15.32:0/967030414 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f841c06be30 0x7f841c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:07.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.683+0000 7f843c82f640 1 --2- 172.21.15.32:0/967030414 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f841c06be30 0x7f841c06e2f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f842c034000 tx=0x7f842c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:07.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.687+0000 7f84357fa640 1 -- 172.21.15.32:0/967030414 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f84380af760 con 0x7f843815b380 2024-04-24T22:59:07.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.865+0000 7f843ea93640 1 -- 172.21.15.32:0/967030414 --> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8438154c30 con 0x7f841c06be30 2024-04-24T22:59:07.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.871+0000 7f84357fa640 1 -- 172.21.15.32:0/967030414 <== mgr.4108 v2:172.21.15.121:6832/1848157943 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+236279 (secure 0 0 0) 0x7f8438154c30 con 0x7f841c06be30 2024-04-24T22:59:07.875 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:59:07.876 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-24T22:59:07.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.873+0000 7f843ea93640 1 -- 172.21.15.32:0/967030414 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f841c06be30 msgr2=0x7f841c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:07.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.874+0000 7f843ea93640 1 --2- 172.21.15.32:0/967030414 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f841c06be30 0x7f841c06e2f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f842c034000 tx=0x7f842c02e000 comp rx=0 tx=0).stop 2024-04-24T22:59:07.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.874+0000 7f843ea93640 1 -- 172.21.15.32:0/967030414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f843815b380 msgr2=0x7f84381516b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:07.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.874+0000 7f843ea93640 1 --2- 172.21.15.32:0/967030414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f843815b380 0x7f84381516b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f842400c7f0 tx=0x7f842400ccb0 comp rx=0 tx=0).stop 2024-04-24T22:59:07.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.874+0000 7f843ea93640 1 -- 172.21.15.32:0/967030414 shutdown_connections 2024-04-24T22:59:07.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.874+0000 7f843ea93640 1 --2- 172.21.15.32:0/967030414 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f841c06be30 0x7f841c06e2f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:07.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.874+0000 7f843ea93640 1 --2- 172.21.15.32:0/967030414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f843815b380 0x7f84381516b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:07.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.874+0000 7f843ea93640 1 --2- 172.21.15.32:0/967030414 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f843815cb00 0x7f8438155f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:07.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.874+0000 7f843ea93640 1 --2- 172.21.15.32:0/967030414 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84380ce430 0x7f8438147680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:07.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.874+0000 7f843ea93640 1 -- 172.21.15.32:0/967030414 >> 172.21.15.32:0/967030414 conn(0x7f843806b110 msgr2=0x7f843806c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:07.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.874+0000 7f843ea93640 1 -- 172.21.15.32:0/967030414 shutdown_connections 2024-04-24T22:59:07.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:07.874+0000 7f843ea93640 1 -- 172.21.15.32:0/967030414 wait complete. 2024-04-24T22:59:07.886 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-04-24T22:59:07.128866+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1545,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560954,"statfs":{"total":1600000000,"available":1598417040,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.633234"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":162,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-24T22:58:57.295146+0000","last_change":"2024-04-24T22:58:55.138810+0000","last_active":"2024-04-24T22:58:57.295146+0000","last_peered":"2024-04-24T22:58:57.295146+0000","last_clean":"2024-04-24T22:58:57.295146+0000","last_became_active":"2024-04-24T22:58:55.138627+0000","last_became_peered":"2024-04-24T22:58:55.138627+0000","last_unstale":"2024-04-24T22:58:57.295146+0000","last_undegraded":"2024-04-24T22:58:57.295146+0000","last_fullsized":"2024-04-24T22:58:57.295146+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_clean_scrub_stamp":"2024-04-24T22:58:54.127235+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:24.010002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195263,"statfs":{"total":200000000,"available":199949722,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195263,"statfs":{"total":200000000,"available":199949722,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":625,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194686,"statfs":{"total":200000000,"available":199359354,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195263,"statfs":{"total":200000000,"available":199949722,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195263,"statfs":{"total":200000000,"available":199949722,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195263,"statfs":{"total":200000000,"available":199949722,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":625,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194686,"statfs":{"total":200000000,"available":199359354,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195263,"statfs":{"total":200000000,"available":199949722,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-24T22:59:07.889 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T22:59:08.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.025+0000 7f74a3b1b640 1 Processor -- start 2024-04-24T22:59:08.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.025+0000 7f74a3b1b640 1 -- start start 2024-04-24T22:59:08.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.026+0000 7f74a3b1b640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f749c0afae0 0x7f749c06c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:08.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.026+0000 7f74a3b1b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f749c0cfe50 con 0x7f749c1673e0 2024-04-24T22:59:08.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.026+0000 7f74a3b1b640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f749c166900 con 0x7f749c162da0 2024-04-24T22:59:08.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.026+0000 7f74a3b1b640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f749c166be0 con 0x7f749c0afae0 2024-04-24T22:59:08.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.026+0000 7f74a20b8640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f749c1673e0 0x7f749c0b3310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44858/0 (socket says 172.21.15.32:44858) 2024-04-24T22:59:08.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.026+0000 7f74a20b8640 1 -- 172.21.15.32:0/1451496563 learned_addr learned my addr 172.21.15.32:0/1451496563 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:08.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.026+0000 7f74a10b6640 1 --2- 172.21.15.32:0/1451496563 >> v2:172.21.15.121:3300/0 conn(0x7f749c0afae0 0x7f749c06c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:08.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.026+0000 7f74a08b5640 1 -- 172.21.15.32:0/1451496563 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2119451196 0 0) 0x7f749c0cfe50 con 0x7f749c1673e0 2024-04-24T22:59:08.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.026+0000 7f74a08b5640 1 -- 172.21.15.32:0/1451496563 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7488003520 con 0x7f749c1673e0 2024-04-24T22:59:08.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.027+0000 7f74a08b5640 1 -- 172.21.15.32:0/1451496563 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3239390513 0 0) 0x7f749c166900 con 0x7f749c162da0 2024-04-24T22:59:08.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.027+0000 7f74a08b5640 1 -- 172.21.15.32:0/1451496563 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f749c0cfe50 con 0x7f749c162da0 2024-04-24T22:59:08.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.027+0000 7f74a08b5640 1 -- 172.21.15.32:0/1451496563 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1845332499 0 0) 0x7f7488003520 con 0x7f749c1673e0 2024-04-24T22:59:08.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.027+0000 7f74a08b5640 1 -- 172.21.15.32:0/1451496563 >> v2:172.21.15.121:3300/0 conn(0x7f749c0afae0 msgr2=0x7f749c06c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:08.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.027+0000 7f74a08b5640 1 --2- 172.21.15.32:0/1451496563 >> v2:172.21.15.121:3300/0 conn(0x7f749c0afae0 0x7f749c06c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:08.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.027+0000 7f74a08b5640 1 -- 172.21.15.32:0/1451496563 >> v1:172.21.15.121:6789/0 conn(0x7f749c162da0 legacy=0x7f749c0cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:08.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.027+0000 7f74a08b5640 1 -- 172.21.15.32:0/1451496563 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f749c166ec0 con 0x7f749c1673e0 2024-04-24T22:59:08.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.027+0000 7f74a08b5640 1 -- 172.21.15.32:0/1451496563 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f7498001670 con 0x7f749c1673e0 2024-04-24T22:59:08.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.027+0000 7f74a08b5640 1 -- 172.21.15.32:0/1451496563 >> v1:172.21.15.32:6789/0 conn(0x7f749c1673e0 legacy=0x7f749c0b3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:08.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.027+0000 7f74a08b5640 1 --2- 172.21.15.32:0/1451496563 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7488003b40 0x7f7488003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:08.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.027+0000 7f74a08b5640 1 --2- 172.21.15.32:0/1451496563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7488004800 0x7f7488004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:08.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.027+0000 7f74a08b5640 1 --2- 172.21.15.32:0/1451496563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74880051c0 0x7f748800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:08.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.027+0000 7f74a08b5640 1 -- 172.21.15.32:0/1451496563 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f749c166be0 con 0x7f74880051c0 2024-04-24T22:59:08.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.027+0000 7f74a08b5640 1 -- 172.21.15.32:0/1451496563 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f748800c040 con 0x7f7488004800 2024-04-24T22:59:08.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.027+0000 7f74a08b5640 1 -- 172.21.15.32:0/1451496563 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f748800c240 con 0x7f7488003b40 2024-04-24T22:59:08.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.027+0000 7f74a20b8640 1 --2- 172.21.15.32:0/1451496563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74880051c0 0x7f748800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:08.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.027+0000 7f74a10b6640 1 --2- 172.21.15.32:0/1451496563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7488004800 0x7f7488004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:08.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.027+0000 7f74a18b7640 1 --2- 172.21.15.32:0/1451496563 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7488003b40 0x7f7488003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:08.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.027+0000 7f74a20b8640 1 -- 172.21.15.32:0/1451496563 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7488003b40 msgr2=0x7f7488003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:08.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.027+0000 7f74a20b8640 1 --2- 172.21.15.32:0/1451496563 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7488003b40 0x7f7488003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:08.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.027+0000 7f74a20b8640 1 -- 172.21.15.32:0/1451496563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7488004800 msgr2=0x7f7488004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:08.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.027+0000 7f74a20b8640 1 --2- 172.21.15.32:0/1451496563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7488004800 0x7f7488004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:08.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.027+0000 7f74a20b8640 1 -- 172.21.15.32:0/1451496563 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7488003520 con 0x7f74880051c0 2024-04-24T22:59:08.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.028+0000 7f74a20b8640 1 -- 172.21.15.32:0/1451496563 reap_dead start 2024-04-24T22:59:08.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.028+0000 7f74a20b8640 1 --2- 172.21.15.32:0/1451496563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74880051c0 0x7f748800b9c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7498007fd0 tx=0x7f7488005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7585a1ca5a6d24d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:08.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.028+0000 7f74a08b5640 1 -- 172.21.15.32:0/1451496563 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7498007ca0 con 0x7f74880051c0 2024-04-24T22:59:08.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.028+0000 7f74a08b5640 1 -- 172.21.15.32:0/1451496563 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7498005af0 con 0x7f74880051c0 2024-04-24T22:59:08.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.028+0000 7f74a08b5640 1 -- 172.21.15.32:0/1451496563 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7498007070 con 0x7f74880051c0 2024-04-24T22:59:08.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.028+0000 7f74a3b1b640 1 -- 172.21.15.32:0/1451496563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74880051c0 msgr2=0x7f748800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:08.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.028+0000 7f74a3b1b640 1 --2- 172.21.15.32:0/1451496563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74880051c0 0x7f748800b9c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7498007fd0 tx=0x7f7488005830 comp rx=0 tx=0).stop 2024-04-24T22:59:08.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.029+0000 7f74a3b1b640 1 -- 172.21.15.32:0/1451496563 shutdown_connections 2024-04-24T22:59:08.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.029+0000 7f74a3b1b640 1 --2- 172.21.15.32:0/1451496563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74880051c0 0x7f748800b9c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:08.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.029+0000 7f74a3b1b640 1 -- 172.21.15.32:0/1451496563 >> 172.21.15.32:0/1451496563 conn(0x7f749c06b110 msgr2=0x7f749c06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:08.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.029+0000 7f74a3b1b640 1 -- 172.21.15.32:0/1451496563 shutdown_connections 2024-04-24T22:59:08.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.029+0000 7f74a3b1b640 1 -- 172.21.15.32:0/1451496563 wait complete. 2024-04-24T22:59:08.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.030+0000 7f74a3b1b640 1 Processor -- start 2024-04-24T22:59:08.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.030+0000 7f74a3b1b640 1 -- start start 2024-04-24T22:59:08.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.031+0000 7f74a3b1b640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f74880051c0 0x7f749c0aef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:08.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.031+0000 7f74a3b1b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f749c163b90 0x7f749c15a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:08.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.031+0000 7f74a3b1b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f749c161090 0x7f749c161520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:08.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.031+0000 7f74a3b1b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f749c0cf7e0 con 0x7f749c163b90 2024-04-24T22:59:08.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.031+0000 7f74a3b1b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f749c0cf660 con 0x7f749c161090 2024-04-24T22:59:08.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.031+0000 7f74a3b1b640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f749c0cf960 con 0x7f74880051c0 2024-04-24T22:59:08.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.031+0000 7f74a10b6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f749c163b90 0x7f749c15a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:08.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.031+0000 7f74a10b6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f749c163b90 0x7f749c15a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44702/0 (socket says 172.21.15.32:44702) 2024-04-24T22:59:08.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.031+0000 7f74a10b6640 1 -- 172.21.15.32:0/3554292164 learned_addr learned my addr 172.21.15.32:0/3554292164 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:08.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.031+0000 7f74a18b7640 1 --2- 172.21.15.32:0/3554292164 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f74880051c0 0x7f749c0aef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:08.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.031+0000 7f74a20b8640 1 --2- 172.21.15.32:0/3554292164 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f749c161090 0x7f749c161520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:08.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.031+0000 7f74a10b6640 1 -- 172.21.15.32:0/3554292164 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f74880051c0 msgr2=0x7f749c0aef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:08.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.031+0000 7f74a10b6640 1 --2- 172.21.15.32:0/3554292164 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f74880051c0 0x7f749c0aef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:08.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.031+0000 7f74a10b6640 1 -- 172.21.15.32:0/3554292164 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f749c161090 msgr2=0x7f749c161520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:08.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.031+0000 7f74a10b6640 1 --2- 172.21.15.32:0/3554292164 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f749c161090 0x7f749c161520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:08.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.031+0000 7f74a10b6640 1 -- 172.21.15.32:0/3554292164 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f749c164090 con 0x7f749c163b90 2024-04-24T22:59:08.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.031+0000 7f74a10b6640 1 --2- 172.21.15.32:0/3554292164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f749c163b90 0x7f749c15a520 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f748c0118e0 tx=0x7f748c011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:08.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.031+0000 7f74927fc640 1 -- 172.21.15.32:0/3554292164 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f748c00fa30 con 0x7f749c163b90 2024-04-24T22:59:08.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.031+0000 7f74927fc640 1 -- 172.21.15.32:0/3554292164 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f748c00fbd0 con 0x7f749c163b90 2024-04-24T22:59:08.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.031+0000 7f74a3b1b640 1 -- 172.21.15.32:0/3554292164 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f749c164370 con 0x7f749c163b90 2024-04-24T22:59:08.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.031+0000 7f74927fc640 1 -- 172.21.15.32:0/3554292164 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f748c014440 con 0x7f749c163b90 2024-04-24T22:59:08.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.031+0000 7f74a3b1b640 1 -- 172.21.15.32:0/3554292164 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f749c164930 con 0x7f749c163b90 2024-04-24T22:59:08.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.032+0000 7f74a3b1b640 1 -- 172.21.15.32:0/3554292164 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f749c068430 con 0x7f749c163b90 2024-04-24T22:59:08.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.033+0000 7f74927fc640 1 -- 172.21.15.32:0/3554292164 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f748c012040 con 0x7f749c163b90 2024-04-24T22:59:08.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.033+0000 7f74927fc640 1 --2- 172.21.15.32:0/3554292164 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f748006be60 0x7f748006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:08.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.033+0000 7f74927fc640 1 -- 172.21.15.32:0/3554292164 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f748c08b470 con 0x7f749c163b90 2024-04-24T22:59:08.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.033+0000 7f74a18b7640 1 --2- 172.21.15.32:0/3554292164 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f748006be60 0x7f748006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:08.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.034+0000 7f74a18b7640 1 --2- 172.21.15.32:0/3554292164 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f748006be60 0x7f748006e320 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7494001f50 tx=0x7f7494009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:08.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.037+0000 7f74927fc640 1 -- 172.21.15.32:0/3554292164 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f748c019030 con 0x7f749c163b90 2024-04-24T22:59:08.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.216+0000 7f74a3b1b640 1 -- 172.21.15.32:0/3554292164 --> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f749c1625c0 con 0x7f748006be60 2024-04-24T22:59:08.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.221+0000 7f74927fc640 1 -- 172.21.15.32:0/3554292164 <== mgr.4108 v2:172.21.15.121:6832/1848157943 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+236279 (secure 0 0 0) 0x7f749c1625c0 con 0x7f748006be60 2024-04-24T22:59:08.225 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:59:08.227 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-24T22:59:08.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.224+0000 7f74a3b1b640 1 -- 172.21.15.32:0/3554292164 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f748006be60 msgr2=0x7f748006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:08.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.224+0000 7f74a3b1b640 1 --2- 172.21.15.32:0/3554292164 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f748006be60 0x7f748006e320 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7494001f50 tx=0x7f7494009040 comp rx=0 tx=0).stop 2024-04-24T22:59:08.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.224+0000 7f74a3b1b640 1 -- 172.21.15.32:0/3554292164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f749c163b90 msgr2=0x7f749c15a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:08.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.224+0000 7f74a3b1b640 1 --2- 172.21.15.32:0/3554292164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f749c163b90 0x7f749c15a520 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f748c0118e0 tx=0x7f748c011ce0 comp rx=0 tx=0).stop 2024-04-24T22:59:08.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.224+0000 7f74a3b1b640 1 -- 172.21.15.32:0/3554292164 shutdown_connections 2024-04-24T22:59:08.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.224+0000 7f74a3b1b640 1 --2- 172.21.15.32:0/3554292164 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f748006be60 0x7f748006e320 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:08.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.224+0000 7f74a3b1b640 1 --2- 172.21.15.32:0/3554292164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f749c163b90 0x7f749c15a520 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:08.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.224+0000 7f74a3b1b640 1 --2- 172.21.15.32:0/3554292164 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f749c161090 0x7f749c161520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:08.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.224+0000 7f74a3b1b640 1 --2- 172.21.15.32:0/3554292164 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f74880051c0 0x7f749c0aef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:08.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.224+0000 7f74a3b1b640 1 -- 172.21.15.32:0/3554292164 >> 172.21.15.32:0/3554292164 conn(0x7f749c06b110 msgr2=0x7f749c0b5db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:08.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.224+0000 7f74a3b1b640 1 -- 172.21.15.32:0/3554292164 shutdown_connections 2024-04-24T22:59:08.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.224+0000 7f74a3b1b640 1 -- 172.21.15.32:0/3554292164 wait complete. 2024-04-24T22:59:08.236 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-04-24T22:59:07.128866+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1545,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560954,"statfs":{"total":1600000000,"available":1598417040,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.633234"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":162,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-24T22:58:57.295146+0000","last_change":"2024-04-24T22:58:55.138810+0000","last_active":"2024-04-24T22:58:57.295146+0000","last_peered":"2024-04-24T22:58:57.295146+0000","last_clean":"2024-04-24T22:58:57.295146+0000","last_became_active":"2024-04-24T22:58:55.138627+0000","last_became_peered":"2024-04-24T22:58:55.138627+0000","last_unstale":"2024-04-24T22:58:57.295146+0000","last_undegraded":"2024-04-24T22:58:57.295146+0000","last_fullsized":"2024-04-24T22:58:57.295146+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_clean_scrub_stamp":"2024-04-24T22:58:54.127235+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:24.010002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195263,"statfs":{"total":200000000,"available":199949722,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195263,"statfs":{"total":200000000,"available":199949722,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":625,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194686,"statfs":{"total":200000000,"available":199359354,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195263,"statfs":{"total":200000000,"available":199949722,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195263,"statfs":{"total":200000000,"available":199949722,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195263,"statfs":{"total":200000000,"available":199949722,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":625,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194686,"statfs":{"total":200000000,"available":199359354,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195263,"statfs":{"total":200000000,"available":199949722,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-24T22:59:08.240 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3f is not active+clean 2024-04-24T22:59:08.240 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.240 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3e is not active+clean 2024-04-24T22:59:08.240 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.241 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3d is not active+clean 2024-04-24T22:59:08.241 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.241 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3c is not active+clean 2024-04-24T22:59:08.241 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.241 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3b is not active+clean 2024-04-24T22:59:08.241 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.241 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3a is not active+clean 2024-04-24T22:59:08.241 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.241 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.39 is not active+clean 2024-04-24T22:59:08.241 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.241 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.38 is not active+clean 2024-04-24T22:59:08.242 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.242 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.37 is not active+clean 2024-04-24T22:59:08.242 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.242 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.36 is not active+clean 2024-04-24T22:59:08.242 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.242 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.35 is not active+clean 2024-04-24T22:59:08.242 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.242 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.34 is not active+clean 2024-04-24T22:59:08.242 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.242 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.33 is not active+clean 2024-04-24T22:59:08.242 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.243 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.32 is not active+clean 2024-04-24T22:59:08.243 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.243 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.31 is not active+clean 2024-04-24T22:59:08.243 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.243 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.30 is not active+clean 2024-04-24T22:59:08.243 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.243 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2f is not active+clean 2024-04-24T22:59:08.243 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.243 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2e is not active+clean 2024-04-24T22:59:08.243 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.243 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2d is not active+clean 2024-04-24T22:59:08.243 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.244 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2c is not active+clean 2024-04-24T22:59:08.244 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.244 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2b is not active+clean 2024-04-24T22:59:08.244 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.244 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2a is not active+clean 2024-04-24T22:59:08.244 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.244 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.29 is not active+clean 2024-04-24T22:59:08.244 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.244 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.28 is not active+clean 2024-04-24T22:59:08.244 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.244 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.27 is not active+clean 2024-04-24T22:59:08.244 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.245 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.26 is not active+clean 2024-04-24T22:59:08.245 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.245 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.25 is not active+clean 2024-04-24T22:59:08.245 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.245 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.24 is not active+clean 2024-04-24T22:59:08.245 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.245 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.23 is not active+clean 2024-04-24T22:59:08.245 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.245 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.22 is not active+clean 2024-04-24T22:59:08.245 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.245 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.21 is not active+clean 2024-04-24T22:59:08.245 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.246 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.20 is not active+clean 2024-04-24T22:59:08.246 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.246 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1a is not active+clean 2024-04-24T22:59:08.246 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.246 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1b is not active+clean 2024-04-24T22:59:08.246 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:02.493149+0000', 'last_change': '2024-04-24T22:59:02.493149+0000', 'last_active': '2024-04-24T22:59:02.493149+0000', 'last_peered': '2024-04-24T22:59:02.493149+0000', 'last_clean': '2024-04-24T22:59:02.493149+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:02.493149+0000', 'last_undegraded': '2024-04-24T22:59:02.493149+0000', 'last_fullsized': '2024-04-24T22:59:02.493149+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.246 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1b is not active+clean 2024-04-24T22:59:08.246 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.246 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1a is not active+clean 2024-04-24T22:59:08.246 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:02.493149+0000', 'last_change': '2024-04-24T22:59:02.493149+0000', 'last_active': '2024-04-24T22:59:02.493149+0000', 'last_peered': '2024-04-24T22:59:02.493149+0000', 'last_clean': '2024-04-24T22:59:02.493149+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:02.493149+0000', 'last_undegraded': '2024-04-24T22:59:02.493149+0000', 'last_fullsized': '2024-04-24T22:59:02.493149+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.246 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.19 is not active+clean 2024-04-24T22:59:08.246 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:02.493149+0000', 'last_change': '2024-04-24T22:59:02.493149+0000', 'last_active': '2024-04-24T22:59:02.493149+0000', 'last_peered': '2024-04-24T22:59:02.493149+0000', 'last_clean': '2024-04-24T22:59:02.493149+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:02.493149+0000', 'last_undegraded': '2024-04-24T22:59:02.493149+0000', 'last_fullsized': '2024-04-24T22:59:02.493149+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.247 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.18 is not active+clean 2024-04-24T22:59:08.247 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.247 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.18 is not active+clean 2024-04-24T22:59:08.247 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:02.493149+0000', 'last_change': '2024-04-24T22:59:02.493149+0000', 'last_active': '2024-04-24T22:59:02.493149+0000', 'last_peered': '2024-04-24T22:59:02.493149+0000', 'last_clean': '2024-04-24T22:59:02.493149+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:02.493149+0000', 'last_undegraded': '2024-04-24T22:59:02.493149+0000', 'last_fullsized': '2024-04-24T22:59:02.493149+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.247 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.19 is not active+clean 2024-04-24T22:59:08.247 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.247 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.17 is not active+clean 2024-04-24T22:59:08.247 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:02.493149+0000', 'last_change': '2024-04-24T22:59:02.493149+0000', 'last_active': '2024-04-24T22:59:02.493149+0000', 'last_peered': '2024-04-24T22:59:02.493149+0000', 'last_clean': '2024-04-24T22:59:02.493149+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:02.493149+0000', 'last_undegraded': '2024-04-24T22:59:02.493149+0000', 'last_fullsized': '2024-04-24T22:59:02.493149+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.248 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.16 is not active+clean 2024-04-24T22:59:08.248 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.248 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.16 is not active+clean 2024-04-24T22:59:08.248 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:02.493149+0000', 'last_change': '2024-04-24T22:59:02.493149+0000', 'last_active': '2024-04-24T22:59:02.493149+0000', 'last_peered': '2024-04-24T22:59:02.493149+0000', 'last_clean': '2024-04-24T22:59:02.493149+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:02.493149+0000', 'last_undegraded': '2024-04-24T22:59:02.493149+0000', 'last_fullsized': '2024-04-24T22:59:02.493149+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.248 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.17 is not active+clean 2024-04-24T22:59:08.248 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.248 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.15 is not active+clean 2024-04-24T22:59:08.248 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:02.493149+0000', 'last_change': '2024-04-24T22:59:02.493149+0000', 'last_active': '2024-04-24T22:59:02.493149+0000', 'last_peered': '2024-04-24T22:59:02.493149+0000', 'last_clean': '2024-04-24T22:59:02.493149+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:02.493149+0000', 'last_undegraded': '2024-04-24T22:59:02.493149+0000', 'last_fullsized': '2024-04-24T22:59:02.493149+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.248 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.14 is not active+clean 2024-04-24T22:59:08.248 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.248 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.14 is not active+clean 2024-04-24T22:59:08.248 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:02.493149+0000', 'last_change': '2024-04-24T22:59:02.493149+0000', 'last_active': '2024-04-24T22:59:02.493149+0000', 'last_peered': '2024-04-24T22:59:02.493149+0000', 'last_clean': '2024-04-24T22:59:02.493149+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:02.493149+0000', 'last_undegraded': '2024-04-24T22:59:02.493149+0000', 'last_fullsized': '2024-04-24T22:59:02.493149+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.249 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.15 is not active+clean 2024-04-24T22:59:08.249 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.249 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.13 is not active+clean 2024-04-24T22:59:08.249 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:02.493149+0000', 'last_change': '2024-04-24T22:59:02.493149+0000', 'last_active': '2024-04-24T22:59:02.493149+0000', 'last_peered': '2024-04-24T22:59:02.493149+0000', 'last_clean': '2024-04-24T22:59:02.493149+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:02.493149+0000', 'last_undegraded': '2024-04-24T22:59:02.493149+0000', 'last_fullsized': '2024-04-24T22:59:02.493149+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.249 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.12 is not active+clean 2024-04-24T22:59:08.249 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.249 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.12 is not active+clean 2024-04-24T22:59:08.249 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:02.493149+0000', 'last_change': '2024-04-24T22:59:02.493149+0000', 'last_active': '2024-04-24T22:59:02.493149+0000', 'last_peered': '2024-04-24T22:59:02.493149+0000', 'last_clean': '2024-04-24T22:59:02.493149+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:02.493149+0000', 'last_undegraded': '2024-04-24T22:59:02.493149+0000', 'last_fullsized': '2024-04-24T22:59:02.493149+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.249 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.13 is not active+clean 2024-04-24T22:59:08.250 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.250 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.11 is not active+clean 2024-04-24T22:59:08.250 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:02.493149+0000', 'last_change': '2024-04-24T22:59:02.493149+0000', 'last_active': '2024-04-24T22:59:02.493149+0000', 'last_peered': '2024-04-24T22:59:02.493149+0000', 'last_clean': '2024-04-24T22:59:02.493149+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:02.493149+0000', 'last_undegraded': '2024-04-24T22:59:02.493149+0000', 'last_fullsized': '2024-04-24T22:59:02.493149+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.250 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.10 is not active+clean 2024-04-24T22:59:08.250 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.250 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.10 is not active+clean 2024-04-24T22:59:08.250 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:02.493149+0000', 'last_change': '2024-04-24T22:59:02.493149+0000', 'last_active': '2024-04-24T22:59:02.493149+0000', 'last_peered': '2024-04-24T22:59:02.493149+0000', 'last_clean': '2024-04-24T22:59:02.493149+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:02.493149+0000', 'last_undegraded': '2024-04-24T22:59:02.493149+0000', 'last_fullsized': '2024-04-24T22:59:02.493149+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.250 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.11 is not active+clean 2024-04-24T22:59:08.250 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.250 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.f is not active+clean 2024-04-24T22:59:08.250 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:02.493149+0000', 'last_change': '2024-04-24T22:59:02.493149+0000', 'last_active': '2024-04-24T22:59:02.493149+0000', 'last_peered': '2024-04-24T22:59:02.493149+0000', 'last_clean': '2024-04-24T22:59:02.493149+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:02.493149+0000', 'last_undegraded': '2024-04-24T22:59:02.493149+0000', 'last_fullsized': '2024-04-24T22:59:02.493149+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.250 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.e is not active+clean 2024-04-24T22:59:08.251 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.251 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.e is not active+clean 2024-04-24T22:59:08.251 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:02.493149+0000', 'last_change': '2024-04-24T22:59:02.493149+0000', 'last_active': '2024-04-24T22:59:02.493149+0000', 'last_peered': '2024-04-24T22:59:02.493149+0000', 'last_clean': '2024-04-24T22:59:02.493149+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:02.493149+0000', 'last_undegraded': '2024-04-24T22:59:02.493149+0000', 'last_fullsized': '2024-04-24T22:59:02.493149+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.251 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.f is not active+clean 2024-04-24T22:59:08.251 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.251 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.d is not active+clean 2024-04-24T22:59:08.251 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:02.493149+0000', 'last_change': '2024-04-24T22:59:02.493149+0000', 'last_active': '2024-04-24T22:59:02.493149+0000', 'last_peered': '2024-04-24T22:59:02.493149+0000', 'last_clean': '2024-04-24T22:59:02.493149+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:02.493149+0000', 'last_undegraded': '2024-04-24T22:59:02.493149+0000', 'last_fullsized': '2024-04-24T22:59:02.493149+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.251 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.c is not active+clean 2024-04-24T22:59:08.251 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.251 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.c is not active+clean 2024-04-24T22:59:08.251 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:02.493149+0000', 'last_change': '2024-04-24T22:59:02.493149+0000', 'last_active': '2024-04-24T22:59:02.493149+0000', 'last_peered': '2024-04-24T22:59:02.493149+0000', 'last_clean': '2024-04-24T22:59:02.493149+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:02.493149+0000', 'last_undegraded': '2024-04-24T22:59:02.493149+0000', 'last_fullsized': '2024-04-24T22:59:02.493149+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.252 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.d is not active+clean 2024-04-24T22:59:08.252 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.252 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.3 is not active+clean 2024-04-24T22:59:08.252 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:02.493149+0000', 'last_change': '2024-04-24T22:59:02.493149+0000', 'last_active': '2024-04-24T22:59:02.493149+0000', 'last_peered': '2024-04-24T22:59:02.493149+0000', 'last_clean': '2024-04-24T22:59:02.493149+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:02.493149+0000', 'last_undegraded': '2024-04-24T22:59:02.493149+0000', 'last_fullsized': '2024-04-24T22:59:02.493149+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.252 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2 is not active+clean 2024-04-24T22:59:08.252 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.252 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.0 is not active+clean 2024-04-24T22:59:08.252 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:02.493149+0000', 'last_change': '2024-04-24T22:59:02.493149+0000', 'last_active': '2024-04-24T22:59:02.493149+0000', 'last_peered': '2024-04-24T22:59:02.493149+0000', 'last_clean': '2024-04-24T22:59:02.493149+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:02.493149+0000', 'last_undegraded': '2024-04-24T22:59:02.493149+0000', 'last_fullsized': '2024-04-24T22:59:02.493149+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.252 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1 is not active+clean 2024-04-24T22:59:08.252 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.252 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1 is not active+clean 2024-04-24T22:59:08.252 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:02.493149+0000', 'last_change': '2024-04-24T22:59:02.493149+0000', 'last_active': '2024-04-24T22:59:02.493149+0000', 'last_peered': '2024-04-24T22:59:02.493149+0000', 'last_clean': '2024-04-24T22:59:02.493149+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:02.493149+0000', 'last_undegraded': '2024-04-24T22:59:02.493149+0000', 'last_fullsized': '2024-04-24T22:59:02.493149+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.253 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.0 is not active+clean 2024-04-24T22:59:08.253 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.253 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.2 is not active+clean 2024-04-24T22:59:08.253 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:02.493149+0000', 'last_change': '2024-04-24T22:59:02.493149+0000', 'last_active': '2024-04-24T22:59:02.493149+0000', 'last_peered': '2024-04-24T22:59:02.493149+0000', 'last_clean': '2024-04-24T22:59:02.493149+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:02.493149+0000', 'last_undegraded': '2024-04-24T22:59:02.493149+0000', 'last_fullsized': '2024-04-24T22:59:02.493149+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.253 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3 is not active+clean 2024-04-24T22:59:08.253 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.253 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.4 is not active+clean 2024-04-24T22:59:08.253 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:02.493149+0000', 'last_change': '2024-04-24T22:59:02.493149+0000', 'last_active': '2024-04-24T22:59:02.493149+0000', 'last_peered': '2024-04-24T22:59:02.493149+0000', 'last_clean': '2024-04-24T22:59:02.493149+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:02.493149+0000', 'last_undegraded': '2024-04-24T22:59:02.493149+0000', 'last_fullsized': '2024-04-24T22:59:02.493149+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.253 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.5 is not active+clean 2024-04-24T22:59:08.253 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.255 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.5 is not active+clean 2024-04-24T22:59:08.255 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:02.493149+0000', 'last_change': '2024-04-24T22:59:02.493149+0000', 'last_active': '2024-04-24T22:59:02.493149+0000', 'last_peered': '2024-04-24T22:59:02.493149+0000', 'last_clean': '2024-04-24T22:59:02.493149+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:02.493149+0000', 'last_undegraded': '2024-04-24T22:59:02.493149+0000', 'last_fullsized': '2024-04-24T22:59:02.493149+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.255 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.4 is not active+clean 2024-04-24T22:59:08.255 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.255 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.6 is not active+clean 2024-04-24T22:59:08.255 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:02.493149+0000', 'last_change': '2024-04-24T22:59:02.493149+0000', 'last_active': '2024-04-24T22:59:02.493149+0000', 'last_peered': '2024-04-24T22:59:02.493149+0000', 'last_clean': '2024-04-24T22:59:02.493149+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:02.493149+0000', 'last_undegraded': '2024-04-24T22:59:02.493149+0000', 'last_fullsized': '2024-04-24T22:59:02.493149+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.255 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.7 is not active+clean 2024-04-24T22:59:08.255 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.255 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.7 is not active+clean 2024-04-24T22:59:08.255 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:02.493149+0000', 'last_change': '2024-04-24T22:59:02.493149+0000', 'last_active': '2024-04-24T22:59:02.493149+0000', 'last_peered': '2024-04-24T22:59:02.493149+0000', 'last_clean': '2024-04-24T22:59:02.493149+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:02.493149+0000', 'last_undegraded': '2024-04-24T22:59:02.493149+0000', 'last_fullsized': '2024-04-24T22:59:02.493149+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.256 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.6 is not active+clean 2024-04-24T22:59:08.256 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.256 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.8 is not active+clean 2024-04-24T22:59:08.256 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:02.493149+0000', 'last_change': '2024-04-24T22:59:02.493149+0000', 'last_active': '2024-04-24T22:59:02.493149+0000', 'last_peered': '2024-04-24T22:59:02.493149+0000', 'last_clean': '2024-04-24T22:59:02.493149+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:02.493149+0000', 'last_undegraded': '2024-04-24T22:59:02.493149+0000', 'last_fullsized': '2024-04-24T22:59:02.493149+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.256 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.9 is not active+clean 2024-04-24T22:59:08.256 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.256 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.9 is not active+clean 2024-04-24T22:59:08.256 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:02.493149+0000', 'last_change': '2024-04-24T22:59:02.493149+0000', 'last_active': '2024-04-24T22:59:02.493149+0000', 'last_peered': '2024-04-24T22:59:02.493149+0000', 'last_clean': '2024-04-24T22:59:02.493149+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:02.493149+0000', 'last_undegraded': '2024-04-24T22:59:02.493149+0000', 'last_fullsized': '2024-04-24T22:59:02.493149+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.256 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.8 is not active+clean 2024-04-24T22:59:08.256 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.256 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.a is not active+clean 2024-04-24T22:59:08.257 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:02.493149+0000', 'last_change': '2024-04-24T22:59:02.493149+0000', 'last_active': '2024-04-24T22:59:02.493149+0000', 'last_peered': '2024-04-24T22:59:02.493149+0000', 'last_clean': '2024-04-24T22:59:02.493149+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:02.493149+0000', 'last_undegraded': '2024-04-24T22:59:02.493149+0000', 'last_fullsized': '2024-04-24T22:59:02.493149+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.257 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.b is not active+clean 2024-04-24T22:59:08.257 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.257 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.b is not active+clean 2024-04-24T22:59:08.257 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:02.493149+0000', 'last_change': '2024-04-24T22:59:02.493149+0000', 'last_active': '2024-04-24T22:59:02.493149+0000', 'last_peered': '2024-04-24T22:59:02.493149+0000', 'last_clean': '2024-04-24T22:59:02.493149+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:02.493149+0000', 'last_undegraded': '2024-04-24T22:59:02.493149+0000', 'last_fullsized': '2024-04-24T22:59:02.493149+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.257 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.a is not active+clean 2024-04-24T22:59:08.257 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.257 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1d is not active+clean 2024-04-24T22:59:08.257 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.257 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1c is not active+clean 2024-04-24T22:59:08.257 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:02.493149+0000', 'last_change': '2024-04-24T22:59:02.493149+0000', 'last_active': '2024-04-24T22:59:02.493149+0000', 'last_peered': '2024-04-24T22:59:02.493149+0000', 'last_clean': '2024-04-24T22:59:02.493149+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:02.493149+0000', 'last_undegraded': '2024-04-24T22:59:02.493149+0000', 'last_fullsized': '2024-04-24T22:59:02.493149+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.258 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1c is not active+clean 2024-04-24T22:59:08.258 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.258 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1d is not active+clean 2024-04-24T22:59:08.258 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:02.493149+0000', 'last_change': '2024-04-24T22:59:02.493149+0000', 'last_active': '2024-04-24T22:59:02.493149+0000', 'last_peered': '2024-04-24T22:59:02.493149+0000', 'last_clean': '2024-04-24T22:59:02.493149+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:02.493149+0000', 'last_undegraded': '2024-04-24T22:59:02.493149+0000', 'last_fullsized': '2024-04-24T22:59:02.493149+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.258 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1f is not active+clean 2024-04-24T22:59:08.258 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.258 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1e is not active+clean 2024-04-24T22:59:08.258 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:02.493149+0000', 'last_change': '2024-04-24T22:59:02.493149+0000', 'last_active': '2024-04-24T22:59:02.493149+0000', 'last_peered': '2024-04-24T22:59:02.493149+0000', 'last_clean': '2024-04-24T22:59:02.493149+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:02.493149+0000', 'last_undegraded': '2024-04-24T22:59:02.493149+0000', 'last_fullsized': '2024-04-24T22:59:02.493149+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.258 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1e is not active+clean 2024-04-24T22:59:08.258 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:03.494874+0000', 'last_change': '2024-04-24T22:59:03.494874+0000', 'last_active': '2024-04-24T22:59:03.494874+0000', 'last_peered': '2024-04-24T22:59:03.494874+0000', 'last_clean': '2024-04-24T22:59:03.494874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:03.494874+0000', 'last_undegraded': '2024-04-24T22:59:03.494874+0000', 'last_fullsized': '2024-04-24T22:59:03.494874+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:03.494874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.258 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1f is not active+clean 2024-04-24T22:59:08.258 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T22:59:02.493149+0000', 'last_change': '2024-04-24T22:59:02.493149+0000', 'last_active': '2024-04-24T22:59:02.493149+0000', 'last_peered': '2024-04-24T22:59:02.493149+0000', 'last_clean': '2024-04-24T22:59:02.493149+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:02.493149+0000', 'last_undegraded': '2024-04-24T22:59:02.493149+0000', 'last_fullsized': '2024-04-24T22:59:02.493149+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:02.493149+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T22:59:08.259 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-24T22:59:08.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.375+0000 7f7089d4f640 1 Processor -- start 2024-04-24T22:59:08.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.375+0000 7f7089d4f640 1 -- start start 2024-04-24T22:59:08.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.376+0000 7f7089d4f640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7084156bb0 0x7f708415dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:08.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.376+0000 7f7089d4f640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7084073390 0x7f708406d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:08.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.376+0000 7f7089d4f640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f708415ccf0 con 0x7f708406d400 2024-04-24T22:59:08.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.376+0000 7f7089d4f640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f708415cfb0 con 0x7f7084156bb0 2024-04-24T22:59:08.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.376+0000 7f7089d4f640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f708415d290 con 0x7f7084073390 2024-04-24T22:59:08.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.376+0000 7f70837fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7084156bb0 0x7f708415dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:08.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.376+0000 7f70837fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7084156bb0 0x7f708415dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44708/0 (socket says 172.21.15.32:44708) 2024-04-24T22:59:08.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.376+0000 7f70837fe640 1 -- 172.21.15.32:0/2223162142 learned_addr learned my addr 172.21.15.32:0/2223162142 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:08.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.376+0000 7f7083fff640 1 --2- 172.21.15.32:0/2223162142 >> v2:172.21.15.121:3300/0 conn(0x7f7084073390 0x7f708406d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:08.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.376+0000 7f70837fe640 1 -- 172.21.15.32:0/2223162142 >> v2:172.21.15.121:3300/0 conn(0x7f7084073390 msgr2=0x7f708406d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:08.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.376+0000 7f70837fe640 1 --2- 172.21.15.32:0/2223162142 >> v2:172.21.15.121:3300/0 conn(0x7f7084073390 0x7f708406d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:08.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.376+0000 7f70837fe640 1 -- 172.21.15.32:0/2223162142 >> v1:172.21.15.121:6789/0 conn(0x7f708406d400 legacy=0x7f7084072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:08.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.376+0000 7f70837fe640 1 -- 172.21.15.32:0/2223162142 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f708415d570 con 0x7f7084156bb0 2024-04-24T22:59:08.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.376+0000 7f7083fff640 1 --2- 172.21.15.32:0/2223162142 >> v2:172.21.15.121:3300/0 conn(0x7f7084073390 0x7f708406d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:08.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.377+0000 7f70837fe640 1 --2- 172.21.15.32:0/2223162142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7084156bb0 0x7f708415dbc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f708415db30 tx=0x7f707402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49e2e41ef6532059 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:08.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.377+0000 7f70827fc640 1 -- 172.21.15.32:0/2223162142 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7074037070 con 0x7f7084156bb0 2024-04-24T22:59:08.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.377+0000 7f70827fc640 1 -- 172.21.15.32:0/2223162142 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7074032440 con 0x7f7084156bb0 2024-04-24T22:59:08.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.377+0000 7f70827fc640 1 -- 172.21.15.32:0/2223162142 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7074032810 con 0x7f7084156bb0 2024-04-24T22:59:08.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.378+0000 7f7089d4f640 1 -- 172.21.15.32:0/2223162142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7084156bb0 msgr2=0x7f708415dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:08.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.378+0000 7f7089d4f640 1 --2- 172.21.15.32:0/2223162142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7084156bb0 0x7f708415dbc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f708415db30 tx=0x7f707402a1e0 comp rx=0 tx=0).stop 2024-04-24T22:59:08.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.378+0000 7f7089d4f640 1 -- 172.21.15.32:0/2223162142 shutdown_connections 2024-04-24T22:59:08.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.378+0000 7f7089d4f640 1 --2- 172.21.15.32:0/2223162142 >> v2:172.21.15.121:3300/0 conn(0x7f7084073390 0x7f708406d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:08.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.378+0000 7f7089d4f640 1 --2- 172.21.15.32:0/2223162142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7084156bb0 0x7f708415dbc0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:08.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.378+0000 7f7089d4f640 1 -- 172.21.15.32:0/2223162142 >> 172.21.15.32:0/2223162142 conn(0x7f708406e800 msgr2=0x7f708406ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:08.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.378+0000 7f7089d4f640 1 -- 172.21.15.32:0/2223162142 shutdown_connections 2024-04-24T22:59:08.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.378+0000 7f7089d4f640 1 -- 172.21.15.32:0/2223162142 wait complete. 2024-04-24T22:59:08.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.379+0000 7f7089d4f640 1 Processor -- start 2024-04-24T22:59:08.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.379+0000 7f7089d4f640 1 -- start start 2024-04-24T22:59:08.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.380+0000 7f7089d4f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f708406d400 0x7f70840b4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:08.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.380+0000 7f7089d4f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7084073390 0x7f7084145700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:08.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.380+0000 7f7089d4f640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7084156bb0 0x7f70841e7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:08.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.380+0000 7f7089d4f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7084156f90 con 0x7f7084073390 2024-04-24T22:59:08.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.380+0000 7f7089d4f640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f708415e100 con 0x7f708406d400 2024-04-24T22:59:08.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.380+0000 7f7089d4f640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f7084157110 con 0x7f7084156bb0 2024-04-24T22:59:08.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.380+0000 7f7082ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7084073390 0x7f7084145700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:08.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.380+0000 7f7082ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7084073390 0x7f7084145700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44714/0 (socket says 172.21.15.32:44714) 2024-04-24T22:59:08.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.380+0000 7f7082ffd640 1 -- 172.21.15.32:0/2858073632 learned_addr learned my addr 172.21.15.32:0/2858073632 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:08.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.380+0000 7f70837fe640 1 --2- 172.21.15.32:0/2858073632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f708406d400 0x7f70840b4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:08.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.380+0000 7f7083fff640 1 --2- 172.21.15.32:0/2858073632 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7084156bb0 0x7f70841e7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:08.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.380+0000 7f7082ffd640 1 -- 172.21.15.32:0/2858073632 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7084156bb0 msgr2=0x7f70841e7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:08.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.380+0000 7f7082ffd640 1 --2- 172.21.15.32:0/2858073632 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7084156bb0 0x7f70841e7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:08.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.380+0000 7f7082ffd640 1 -- 172.21.15.32:0/2858073632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f708406d400 msgr2=0x7f70840b4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:08.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.380+0000 7f7082ffd640 1 --2- 172.21.15.32:0/2858073632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f708406d400 0x7f70840b4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:08.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.380+0000 7f7082ffd640 1 -- 172.21.15.32:0/2858073632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7084145c40 con 0x7f7084073390 2024-04-24T22:59:08.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.381+0000 7f7082ffd640 1 --2- 172.21.15.32:0/2858073632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7084073390 0x7f7084145700 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f707800c440 tx=0x7f707800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:08.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.381+0000 7f706bfff640 1 -- 172.21.15.32:0/2858073632 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70780052d0 con 0x7f7084073390 2024-04-24T22:59:08.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.381+0000 7f706bfff640 1 -- 172.21.15.32:0/2858073632 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7078005470 con 0x7f7084073390 2024-04-24T22:59:08.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.381+0000 7f706bfff640 1 -- 172.21.15.32:0/2858073632 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7078012870 con 0x7f7084073390 2024-04-24T22:59:08.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.381+0000 7f7089d4f640 1 -- 172.21.15.32:0/2858073632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f708406b440 con 0x7f7084073390 2024-04-24T22:59:08.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.381+0000 7f7089d4f640 1 -- 172.21.15.32:0/2858073632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f708406ba00 con 0x7f7084073390 2024-04-24T22:59:08.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.382+0000 7f7089d4f640 1 -- 172.21.15.32:0/2858073632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7084068430 con 0x7f7084073390 2024-04-24T22:59:08.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.382+0000 7f706bfff640 1 -- 172.21.15.32:0/2858073632 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7078012a10 con 0x7f7084073390 2024-04-24T22:59:08.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.383+0000 7f706bfff640 1 --2- 172.21.15.32:0/2858073632 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f707006be30 0x7f707006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:08.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.383+0000 7f706bfff640 1 -- 172.21.15.32:0/2858073632 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f707808a930 con 0x7f7084073390 2024-04-24T22:59:08.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.383+0000 7f70837fe640 1 --2- 172.21.15.32:0/2858073632 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f707006be30 0x7f707006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:08.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.383+0000 7f70837fe640 1 --2- 172.21.15.32:0/2858073632 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f707006be30 0x7f707006e2f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7074034000 tx=0x7f707402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:08.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.387+0000 7f706bfff640 1 -- 172.21.15.32:0/2858073632 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7078058c90 con 0x7f7084073390 2024-04-24T22:59:08.493 INFO:tasks.ceph.mds.d.smithi121.stderr:2024-04-24T22:59:08.492+0000 7f349f396640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T22:59:08.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.640+0000 7f7089d4f640 1 -- 172.21.15.32:0/2858073632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f70840d1990 con 0x7f7084073390 2024-04-24T22:59:08.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.640+0000 7f706bfff640 1 -- 172.21.15.32:0/2858073632 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1015 (secure 0 0 0) 0x7f707805cf90 con 0x7f7084073390 2024-04-24T22:59:08.642 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:59:08.642 INFO:teuthology.orchestra.run.smithi032.stdout:{"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":21,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":21,"num_osds":8,"num_up_osds":8,"osd_up_since":1713999532,"num_in_osds":8,"osd_in_since":1713999529,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":96},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":1582960,"bytes_avail":1598417040,"bytes_total":1600000000,"unknown_pgs_ratio":0.98969072103500366},"fsmap":{"epoch":6,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"d","status":"up:active","gid":4276}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-04-24T22:58:46.965982+0000","services":{}},"progress_events":{}} 2024-04-24T22:59:08.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.642+0000 7f7089d4f640 1 -- 172.21.15.32:0/2858073632 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f707006be30 msgr2=0x7f707006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:08.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.642+0000 7f7089d4f640 1 --2- 172.21.15.32:0/2858073632 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f707006be30 0x7f707006e2f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7074034000 tx=0x7f707402e000 comp rx=0 tx=0).stop 2024-04-24T22:59:08.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.642+0000 7f7089d4f640 1 -- 172.21.15.32:0/2858073632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7084073390 msgr2=0x7f7084145700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:08.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.642+0000 7f7089d4f640 1 --2- 172.21.15.32:0/2858073632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7084073390 0x7f7084145700 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f707800c440 tx=0x7f707800c900 comp rx=0 tx=0).stop 2024-04-24T22:59:08.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.642+0000 7f7089d4f640 1 -- 172.21.15.32:0/2858073632 shutdown_connections 2024-04-24T22:59:08.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.642+0000 7f7089d4f640 1 --2- 172.21.15.32:0/2858073632 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f707006be30 0x7f707006e2f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:08.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.642+0000 7f7089d4f640 1 --2- 172.21.15.32:0/2858073632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7084073390 0x7f7084145700 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:08.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.642+0000 7f7089d4f640 1 --2- 172.21.15.32:0/2858073632 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7084156bb0 0x7f70841e7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:08.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.642+0000 7f7089d4f640 1 --2- 172.21.15.32:0/2858073632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f708406d400 0x7f70840b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:08.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.642+0000 7f7089d4f640 1 -- 172.21.15.32:0/2858073632 >> 172.21.15.32:0/2858073632 conn(0x7f708406e800 msgr2=0x7f708415e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:08.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.642+0000 7f7089d4f640 1 -- 172.21.15.32:0/2858073632 shutdown_connections 2024-04-24T22:59:08.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.642+0000 7f7089d4f640 1 -- 172.21.15.32:0/2858073632 wait complete. 2024-04-24T22:59:08.656 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-24T22:59:08.656 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T22:59:08.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.755+0000 7f5250ba5640 1 Processor -- start 2024-04-24T22:59:08.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.755+0000 7f5250ba5640 1 -- start start 2024-04-24T22:59:08.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.756+0000 7f5250ba5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f524c06ba30 0x7f524c06d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:08.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.756+0000 7f5250ba5640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f524c06da90 0x7f524c0b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:08.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.756+0000 7f5250ba5640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f524c15ef10 con 0x7f524c158dd0 2024-04-24T22:59:08.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.756+0000 7f5250ba5640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f524c15f1f0 con 0x7f524c06da90 2024-04-24T22:59:08.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.756+0000 7f5250ba5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f524c15f4d0 con 0x7f524c06ba30 2024-04-24T22:59:08.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.756+0000 7f524a59c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f524c158dd0 0x7f524c06b320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44860/0 (socket says 172.21.15.32:44860) 2024-04-24T22:59:08.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.756+0000 7f524a59c640 1 -- 172.21.15.32:0/1667564178 learned_addr learned my addr 172.21.15.32:0/1667564178 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:08.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.756+0000 7f5243fff640 1 --2- 172.21.15.32:0/1667564178 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f524c06ba30 0x7f524c06d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:08.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.756+0000 7f524ad9d640 1 --2- 172.21.15.32:0/1667564178 >> v2:172.21.15.121:3300/0 conn(0x7f524c06da90 0x7f524c0b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:08.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.756+0000 7f5249d9b640 1 -- 172.21.15.32:0/1667564178 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4032492229 0 0) 0x7f524c15ef10 con 0x7f524c158dd0 2024-04-24T22:59:08.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.756+0000 7f5249d9b640 1 -- 172.21.15.32:0/1667564178 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5238003520 con 0x7f524c158dd0 2024-04-24T22:59:08.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.756+0000 7f5249d9b640 1 -- 172.21.15.32:0/1667564178 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 379111419 0 0) 0x7f5238003520 con 0x7f524c158dd0 2024-04-24T22:59:08.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.756+0000 7f5249d9b640 1 -- 172.21.15.32:0/1667564178 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f524c06ba30 msgr2=0x7f524c06d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:08.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.756+0000 7f5249d9b640 1 --2- 172.21.15.32:0/1667564178 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f524c06ba30 0x7f524c06d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:08.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.756+0000 7f5249d9b640 1 -- 172.21.15.32:0/1667564178 >> v2:172.21.15.121:3300/0 conn(0x7f524c06da90 msgr2=0x7f524c0b5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:08.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.756+0000 7f5249d9b640 1 --2- 172.21.15.32:0/1667564178 >> v2:172.21.15.121:3300/0 conn(0x7f524c06da90 0x7f524c0b5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:08.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.756+0000 7f5249d9b640 1 -- 172.21.15.32:0/1667564178 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f524c15f7b0 con 0x7f524c158dd0 2024-04-24T22:59:08.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.756+0000 7f5249d9b640 1 -- 172.21.15.32:0/1667564178 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f523c001670 con 0x7f524c158dd0 2024-04-24T22:59:08.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.756+0000 7f5243fff640 1 --2- 172.21.15.32:0/1667564178 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f524c06ba30 0x7f524c06d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:59:08.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.756+0000 7f524ad9d640 1 --2- 172.21.15.32:0/1667564178 >> v2:172.21.15.121:3300/0 conn(0x7f524c06da90 0x7f524c0b5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:59:08.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.756+0000 7f5249d9b640 1 -- 172.21.15.32:0/1667564178 >> v1:172.21.15.32:6789/0 conn(0x7f524c158dd0 legacy=0x7f524c06b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:08.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.756+0000 7f5249d9b640 1 --2- 172.21.15.32:0/1667564178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52380039b0 0x7f5238003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:08.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.756+0000 7f5249d9b640 1 --2- 172.21.15.32:0/1667564178 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f524c06ba30 0x7f5238004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:08.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.756+0000 7f5249d9b640 1 --2- 172.21.15.32:0/1667564178 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5238004d40 0x7f52380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:08.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.756+0000 7f5249d9b640 1 -- 172.21.15.32:0/1667564178 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f524c15f1f0 con 0x7f52380039b0 2024-04-24T22:59:08.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.756+0000 7f5249d9b640 1 -- 172.21.15.32:0/1667564178 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f524c15f4d0 con 0x7f5238004d40 2024-04-24T22:59:08.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.757+0000 7f5249d9b640 1 -- 172.21.15.32:0/1667564178 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f523800b9c0 con 0x7f524c06ba30 2024-04-24T22:59:08.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.757+0000 7f524a59c640 1 --2- 172.21.15.32:0/1667564178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52380039b0 0x7f5238003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:08.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.757+0000 7f5243fff640 1 --2- 172.21.15.32:0/1667564178 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f524c06ba30 0x7f5238004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:08.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.757+0000 7f524ad9d640 1 --2- 172.21.15.32:0/1667564178 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5238004d40 0x7f52380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:08.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.757+0000 7f524a59c640 1 -- 172.21.15.32:0/1667564178 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f524c06ba30 msgr2=0x7f5238004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:08.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.757+0000 7f524a59c640 1 --2- 172.21.15.32:0/1667564178 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f524c06ba30 0x7f5238004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:08.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.757+0000 7f524a59c640 1 -- 172.21.15.32:0/1667564178 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5238004d40 msgr2=0x7f52380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:08.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.757+0000 7f524a59c640 1 --2- 172.21.15.32:0/1667564178 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5238004d40 0x7f52380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:08.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.757+0000 7f524a59c640 1 -- 172.21.15.32:0/1667564178 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5238003520 con 0x7f52380039b0 2024-04-24T22:59:08.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.758+0000 7f524a59c640 1 --2- 172.21.15.32:0/1667564178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52380039b0 0x7f5238003dd0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f523c00c210 tx=0x7f5238005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e57c033acb99267 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:08.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.758+0000 7f5249d9b640 1 -- 172.21.15.32:0/1667564178 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523c007d50 con 0x7f52380039b0 2024-04-24T22:59:08.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.758+0000 7f5249d9b640 1 -- 172.21.15.32:0/1667564178 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f523c02e070 con 0x7f52380039b0 2024-04-24T22:59:08.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.758+0000 7f5249d9b640 1 -- 172.21.15.32:0/1667564178 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523c005d30 con 0x7f52380039b0 2024-04-24T22:59:08.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.758+0000 7f5250ba5640 1 -- 172.21.15.32:0/1667564178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52380039b0 msgr2=0x7f5238003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:08.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.758+0000 7f5250ba5640 1 --2- 172.21.15.32:0/1667564178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52380039b0 0x7f5238003dd0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f523c00c210 tx=0x7f5238005830 comp rx=0 tx=0).stop 2024-04-24T22:59:08.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.758+0000 7f524ad9d640 1 -- 172.21.15.32:0/1667564178 reap_dead start 2024-04-24T22:59:08.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.758+0000 7f5250ba5640 1 -- 172.21.15.32:0/1667564178 shutdown_connections 2024-04-24T22:59:08.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.758+0000 7f5250ba5640 1 -- 172.21.15.32:0/1667564178 >> 172.21.15.32:0/1667564178 conn(0x7f524c06e800 msgr2=0x7f524c06ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:08.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.758+0000 7f5250ba5640 1 -- 172.21.15.32:0/1667564178 shutdown_connections 2024-04-24T22:59:08.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.758+0000 7f5250ba5640 1 -- 172.21.15.32:0/1667564178 wait complete. 2024-04-24T22:59:08.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.760+0000 7f5250ba5640 1 Processor -- start 2024-04-24T22:59:08.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.760+0000 7f5250ba5640 1 -- start start 2024-04-24T22:59:08.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.760+0000 7f5250ba5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f52380039b0 0x7f524c14a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:08.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.760+0000 7f5250ba5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f524c14c210 0x7f524c1490b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:08.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.760+0000 7f5250ba5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f524c0b5bf0 0x7f524c0b6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:08.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.760+0000 7f5250ba5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f524c06bfe0 con 0x7f524c14c210 2024-04-24T22:59:08.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.760+0000 7f5250ba5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f524c06ded0 con 0x7f524c0b5bf0 2024-04-24T22:59:08.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.760+0000 7f5250ba5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f524c1591b0 con 0x7f52380039b0 2024-04-24T22:59:08.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.761+0000 7f5243fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f524c14c210 0x7f524c1490b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:08.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.761+0000 7f5243fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f524c14c210 0x7f524c1490b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44720/0 (socket says 172.21.15.32:44720) 2024-04-24T22:59:08.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.761+0000 7f5243fff640 1 -- 172.21.15.32:0/1523314272 learned_addr learned my addr 172.21.15.32:0/1523314272 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:08.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.761+0000 7f524a59c640 1 --2- 172.21.15.32:0/1523314272 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f52380039b0 0x7f524c14a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:08.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.761+0000 7f524ad9d640 1 --2- 172.21.15.32:0/1523314272 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f524c0b5bf0 0x7f524c0b6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:08.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.761+0000 7f5243fff640 1 -- 172.21.15.32:0/1523314272 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f52380039b0 msgr2=0x7f524c14a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:08.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.761+0000 7f5243fff640 1 --2- 172.21.15.32:0/1523314272 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f52380039b0 0x7f524c14a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:08.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.761+0000 7f5243fff640 1 -- 172.21.15.32:0/1523314272 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f524c0b5bf0 msgr2=0x7f524c0b6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:08.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.761+0000 7f5243fff640 1 --2- 172.21.15.32:0/1523314272 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f524c0b5bf0 0x7f524c0b6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:08.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.761+0000 7f5243fff640 1 -- 172.21.15.32:0/1523314272 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f524c14ac50 con 0x7f524c14c210 2024-04-24T22:59:08.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.761+0000 7f524ad9d640 1 --2- 172.21.15.32:0/1523314272 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f524c0b5bf0 0x7f524c0b6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:08.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.761+0000 7f5243fff640 1 --2- 172.21.15.32:0/1523314272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f524c14c210 0x7f524c1490b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5234011930 tx=0x7f5234011df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:08.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.761+0000 7f5242ffd640 1 -- 172.21.15.32:0/1523314272 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523400faf0 con 0x7f524c14c210 2024-04-24T22:59:08.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.761+0000 7f5242ffd640 1 -- 172.21.15.32:0/1523314272 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f523400fc90 con 0x7f524c14c210 2024-04-24T22:59:08.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.761+0000 7f5250ba5640 1 -- 172.21.15.32:0/1523314272 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f524c15a3d0 con 0x7f524c14c210 2024-04-24T22:59:08.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.761+0000 7f5242ffd640 1 -- 172.21.15.32:0/1523314272 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5234004030 con 0x7f524c14c210 2024-04-24T22:59:08.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.761+0000 7f5250ba5640 1 -- 172.21.15.32:0/1523314272 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f524c15a990 con 0x7f524c14c210 2024-04-24T22:59:08.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.762+0000 7f5250ba5640 1 -- 172.21.15.32:0/1523314272 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f524c0b65b0 con 0x7f524c14c210 2024-04-24T22:59:08.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.763+0000 7f5242ffd640 1 -- 172.21.15.32:0/1523314272 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5234012040 con 0x7f524c14c210 2024-04-24T22:59:08.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.763+0000 7f5242ffd640 1 --2- 172.21.15.32:0/1523314272 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f522c06be60 0x7f522c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:08.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.763+0000 7f5242ffd640 1 -- 172.21.15.32:0/1523314272 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f523408b0a0 con 0x7f524c14c210 2024-04-24T22:59:08.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.763+0000 7f524a59c640 1 --2- 172.21.15.32:0/1523314272 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f522c06be60 0x7f522c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:08.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.764+0000 7f524a59c640 1 --2- 172.21.15.32:0/1523314272 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f522c06be60 0x7f522c06e320 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f523c033000 tx=0x7f523c009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:08.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.767+0000 7f5242ffd640 1 -- 172.21.15.32:0/1523314272 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5234019070 con 0x7f524c14c210 2024-04-24T22:59:08.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.947+0000 7f5250ba5640 1 -- 172.21.15.32:0/1523314272 --> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f524c068430 con 0x7f522c06be60 2024-04-24T22:59:08.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.953+0000 7f5242ffd640 1 -- 172.21.15.32:0/1523314272 <== mgr.4108 v2:172.21.15.121:6832/1848157943 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+236279 (secure 0 0 0) 0x7f524c068430 con 0x7f522c06be60 2024-04-24T22:59:08.957 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:59:08.961 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-24T22:59:08.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.956+0000 7f5250ba5640 1 -- 172.21.15.32:0/1523314272 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f522c06be60 msgr2=0x7f522c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:08.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.956+0000 7f5250ba5640 1 --2- 172.21.15.32:0/1523314272 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f522c06be60 0x7f522c06e320 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f523c033000 tx=0x7f523c009030 comp rx=0 tx=0).stop 2024-04-24T22:59:08.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.956+0000 7f5250ba5640 1 -- 172.21.15.32:0/1523314272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f524c14c210 msgr2=0x7f524c1490b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:08.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.956+0000 7f5250ba5640 1 --2- 172.21.15.32:0/1523314272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f524c14c210 0x7f524c1490b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5234011930 tx=0x7f5234011df0 comp rx=0 tx=0).stop 2024-04-24T22:59:08.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.956+0000 7f5250ba5640 1 -- 172.21.15.32:0/1523314272 shutdown_connections 2024-04-24T22:59:08.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.956+0000 7f5250ba5640 1 --2- 172.21.15.32:0/1523314272 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f522c06be60 0x7f522c06e320 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:08.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.956+0000 7f5250ba5640 1 --2- 172.21.15.32:0/1523314272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f524c14c210 0x7f524c1490b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:08.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.956+0000 7f5250ba5640 1 --2- 172.21.15.32:0/1523314272 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f524c0b5bf0 0x7f524c0b6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:08.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.956+0000 7f5250ba5640 1 --2- 172.21.15.32:0/1523314272 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f52380039b0 0x7f524c14a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:08.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.956+0000 7f5250ba5640 1 -- 172.21.15.32:0/1523314272 >> 172.21.15.32:0/1523314272 conn(0x7f524c06e800 msgr2=0x7f524c0b2ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:08.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.956+0000 7f5250ba5640 1 -- 172.21.15.32:0/1523314272 shutdown_connections 2024-04-24T22:59:08.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:08.956+0000 7f5250ba5640 1 -- 172.21.15.32:0/1523314272 wait complete. 2024-04-24T22:59:08.968 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-04-24T22:59:07.128866+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1545,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560954,"statfs":{"total":1600000000,"available":1598417040,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.633234"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":162,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-24T22:58:57.295146+0000","last_change":"2024-04-24T22:58:55.138810+0000","last_active":"2024-04-24T22:58:57.295146+0000","last_peered":"2024-04-24T22:58:57.295146+0000","last_clean":"2024-04-24T22:58:57.295146+0000","last_became_active":"2024-04-24T22:58:55.138627+0000","last_became_peered":"2024-04-24T22:58:55.138627+0000","last_unstale":"2024-04-24T22:58:57.295146+0000","last_undegraded":"2024-04-24T22:58:57.295146+0000","last_fullsized":"2024-04-24T22:58:57.295146+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_clean_scrub_stamp":"2024-04-24T22:58:54.127235+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:24.010002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:03.494874+0000","last_change":"2024-04-24T22:59:03.494874+0000","last_active":"2024-04-24T22:59:03.494874+0000","last_peered":"2024-04-24T22:59:03.494874+0000","last_clean":"2024-04-24T22:59:03.494874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:03.494874+0000","last_undegraded":"2024-04-24T22:59:03.494874+0000","last_fullsized":"2024-04-24T22:59:03.494874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T22:59:02.493149+0000","last_change":"2024-04-24T22:59:02.493149+0000","last_active":"2024-04-24T22:59:02.493149+0000","last_peered":"2024-04-24T22:59:02.493149+0000","last_clean":"2024-04-24T22:59:02.493149+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:02.493149+0000","last_undegraded":"2024-04-24T22:59:02.493149+0000","last_fullsized":"2024-04-24T22:59:02.493149+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195263,"statfs":{"total":200000000,"available":199949722,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195263,"statfs":{"total":200000000,"available":199949722,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":625,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194686,"statfs":{"total":200000000,"available":199359354,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195263,"statfs":{"total":200000000,"available":199949722,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195263,"statfs":{"total":200000000,"available":199949722,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195263,"statfs":{"total":200000000,"available":199949722,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":625,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194686,"statfs":{"total":200000000,"available":199359354,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195263,"statfs":{"total":200000000,"available":199949722,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-24T22:59:11.973 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T22:59:12.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.086+0000 7f011bb4f640 1 Processor -- start 2024-04-24T22:59:12.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.086+0000 7f011bb4f640 1 -- start start 2024-04-24T22:59:12.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.087+0000 7f011bb4f640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f011406ba30 0x7f011406d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:12.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.087+0000 7f011bb4f640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f011415ef10 con 0x7f011406da90 2024-04-24T22:59:12.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.087+0000 7f011bb4f640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f011415f1f0 con 0x7f0114158dd0 2024-04-24T22:59:12.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.087+0000 7f011bb4f640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f011415f4d0 con 0x7f011406ba30 2024-04-24T22:59:12.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.087+0000 7f01190ea640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f011406ba30 0x7f011406d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:12.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.087+0000 7f011a0ec640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f011406da90 0x7f01140b5d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47382/0 (socket says 172.21.15.32:47382) 2024-04-24T22:59:12.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.087+0000 7f011a0ec640 1 -- 172.21.15.32:0/1516850495 learned_addr learned my addr 172.21.15.32:0/1516850495 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:12.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.087+0000 7f01190ea640 1 --2- 172.21.15.32:0/1516850495 >> v2:172.21.15.32:3300/0 conn(0x7f011406ba30 0x7f011406d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38364/0 (socket says 172.21.15.32:38364) 2024-04-24T22:59:12.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.087+0000 7f01190ea640 1 -- 172.21.15.32:0/1516850495 >> v1:172.21.15.121:6789/0 conn(0x7f0114158dd0 legacy=0x7f011406b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:12.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.087+0000 7f01190ea640 1 -- 172.21.15.32:0/1516850495 >> v1:172.21.15.32:6789/0 conn(0x7f011406da90 legacy=0x7f01140b5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:12.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.087+0000 7f01188e9640 1 -- 172.21.15.32:0/1516850495 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3603464326 0 0) 0x7f011415ef10 con 0x7f011406da90 2024-04-24T22:59:12.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.087+0000 7f01190ea640 1 -- 172.21.15.32:0/1516850495 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f011415f7b0 con 0x7f011406ba30 2024-04-24T22:59:12.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.088+0000 7f01190ea640 1 --2- 172.21.15.32:0/1516850495 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f011406ba30 0x7f011406d550 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f01040097a0 tx=0x7f010402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d40dbf98d508173b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:12.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.088+0000 7f01188e9640 1 -- 172.21.15.32:0/1516850495 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0104037070 con 0x7f011406ba30 2024-04-24T22:59:12.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.088+0000 7f01188e9640 1 -- 172.21.15.32:0/1516850495 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0104032440 con 0x7f011406ba30 2024-04-24T22:59:12.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.088+0000 7f01188e9640 1 -- 172.21.15.32:0/1516850495 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0104032810 con 0x7f011406ba30 2024-04-24T22:59:12.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.089+0000 7f011bb4f640 1 -- 172.21.15.32:0/1516850495 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f011406ba30 msgr2=0x7f011406d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:12.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.089+0000 7f011bb4f640 1 --2- 172.21.15.32:0/1516850495 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f011406ba30 0x7f011406d550 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f01040097a0 tx=0x7f010402a230 comp rx=0 tx=0).stop 2024-04-24T22:59:12.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.089+0000 7f011bb4f640 1 -- 172.21.15.32:0/1516850495 shutdown_connections 2024-04-24T22:59:12.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.089+0000 7f011bb4f640 1 --2- 172.21.15.32:0/1516850495 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f011406ba30 0x7f011406d550 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:12.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.089+0000 7f011bb4f640 1 -- 172.21.15.32:0/1516850495 >> 172.21.15.32:0/1516850495 conn(0x7f011406e800 msgr2=0x7f011406ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:12.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.089+0000 7f011bb4f640 1 -- 172.21.15.32:0/1516850495 shutdown_connections 2024-04-24T22:59:12.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.089+0000 7f011bb4f640 1 -- 172.21.15.32:0/1516850495 wait complete. 2024-04-24T22:59:12.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.090+0000 7f011bb4f640 1 Processor -- start 2024-04-24T22:59:12.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.090+0000 7f011bb4f640 1 -- start start 2024-04-24T22:59:12.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.091+0000 7f011bb4f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f011406ba30 0x7f01140b4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:12.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.091+0000 7f011bb4f640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f011406da90 0x7f0114148f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:12.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.091+0000 7f011bb4f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0114158dd0 0x7f01140b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:12.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.091+0000 7f011bb4f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f011406bfe0 con 0x7f0114158dd0 2024-04-24T22:59:12.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.091+0000 7f011bb4f640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f011406ded0 con 0x7f011406ba30 2024-04-24T22:59:12.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.091+0000 7f011bb4f640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f01141591b0 con 0x7f011406da90 2024-04-24T22:59:12.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.091+0000 7f011a0ec640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0114158dd0 0x7f01140b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:12.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.091+0000 7f011a0ec640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0114158dd0 0x7f01140b2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38376/0 (socket says 172.21.15.32:38376) 2024-04-24T22:59:12.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.091+0000 7f011a0ec640 1 -- 172.21.15.32:0/1041443664 learned_addr learned my addr 172.21.15.32:0/1041443664 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:12.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.091+0000 7f01190ea640 1 --2- 172.21.15.32:0/1041443664 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f011406da90 0x7f0114148f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:12.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.091+0000 7f01198eb640 1 --2- 172.21.15.32:0/1041443664 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f011406ba30 0x7f01140b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:12.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.091+0000 7f011a0ec640 1 -- 172.21.15.32:0/1041443664 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f011406da90 msgr2=0x7f0114148f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:12.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.091+0000 7f011a0ec640 1 --2- 172.21.15.32:0/1041443664 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f011406da90 0x7f0114148f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:12.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.091+0000 7f011a0ec640 1 -- 172.21.15.32:0/1041443664 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f011406ba30 msgr2=0x7f01140b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:12.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.091+0000 7f011a0ec640 1 --2- 172.21.15.32:0/1041443664 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f011406ba30 0x7f01140b4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:12.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.091+0000 7f011a0ec640 1 -- 172.21.15.32:0/1041443664 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01140b38a0 con 0x7f0114158dd0 2024-04-24T22:59:12.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.092+0000 7f011a0ec640 1 --2- 172.21.15.32:0/1041443664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0114158dd0 0x7f01140b2e50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f010c00c7a0 tx=0x7f010c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:12.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.092+0000 7f010a7fc640 1 -- 172.21.15.32:0/1041443664 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010c018070 con 0x7f0114158dd0 2024-04-24T22:59:12.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.092+0000 7f010a7fc640 1 -- 172.21.15.32:0/1041443664 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f010c00a640 con 0x7f0114158dd0 2024-04-24T22:59:12.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.092+0000 7f011bb4f640 1 -- 172.21.15.32:0/1041443664 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01140b3b80 con 0x7f0114158dd0 2024-04-24T22:59:12.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.092+0000 7f010a7fc640 1 -- 172.21.15.32:0/1041443664 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010c00aab0 con 0x7f0114158dd0 2024-04-24T22:59:12.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.092+0000 7f011bb4f640 1 -- 172.21.15.32:0/1041443664 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01140b4420 con 0x7f0114158dd0 2024-04-24T22:59:12.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.093+0000 7f011bb4f640 1 -- 172.21.15.32:0/1041443664 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01140b3f70 con 0x7f0114158dd0 2024-04-24T22:59:12.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.093+0000 7f010a7fc640 1 -- 172.21.15.32:0/1041443664 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f010c01b070 con 0x7f0114158dd0 2024-04-24T22:59:12.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.094+0000 7f010a7fc640 1 --2- 172.21.15.32:0/1041443664 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f00f806be30 0x7f00f806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:12.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.094+0000 7f010a7fc640 1 -- 172.21.15.32:0/1041443664 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f010c08a3f0 con 0x7f0114158dd0 2024-04-24T22:59:12.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.094+0000 7f01198eb640 1 --2- 172.21.15.32:0/1041443664 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f00f806be30 0x7f00f806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:12.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.094+0000 7f01198eb640 1 --2- 172.21.15.32:0/1041443664 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f00f806be30 0x7f00f806e2f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0110000f60 tx=0x7f0110009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:12.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.098+0000 7f010a7fc640 1 -- 172.21.15.32:0/1041443664 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f010c058750 con 0x7f0114158dd0 2024-04-24T22:59:12.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.277+0000 7f011bb4f640 1 -- 172.21.15.32:0/1041443664 --> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f01140b4660 con 0x7f00f806be30 2024-04-24T22:59:12.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.284+0000 7f010a7fc640 1 -- 172.21.15.32:0/1041443664 <== mgr.4108 v2:172.21.15.121:6832/1848157943 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247483 (secure 0 0 0) 0x7f0110001d10 con 0x7f00f806be30 2024-04-24T22:59:12.288 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:59:12.289 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-24T22:59:12.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.287+0000 7f011bb4f640 1 -- 172.21.15.32:0/1041443664 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f00f806be30 msgr2=0x7f00f806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:12.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.287+0000 7f011bb4f640 1 --2- 172.21.15.32:0/1041443664 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f00f806be30 0x7f00f806e2f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0110000f60 tx=0x7f0110009040 comp rx=0 tx=0).stop 2024-04-24T22:59:12.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.287+0000 7f011bb4f640 1 -- 172.21.15.32:0/1041443664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0114158dd0 msgr2=0x7f01140b2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:12.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.287+0000 7f011bb4f640 1 --2- 172.21.15.32:0/1041443664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0114158dd0 0x7f01140b2e50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f010c00c7a0 tx=0x7f010c00cc60 comp rx=0 tx=0).stop 2024-04-24T22:59:12.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.287+0000 7f011bb4f640 1 -- 172.21.15.32:0/1041443664 shutdown_connections 2024-04-24T22:59:12.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.287+0000 7f011bb4f640 1 --2- 172.21.15.32:0/1041443664 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f00f806be30 0x7f00f806e2f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:12.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.287+0000 7f011bb4f640 1 --2- 172.21.15.32:0/1041443664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0114158dd0 0x7f01140b2e50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:12.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.287+0000 7f011bb4f640 1 --2- 172.21.15.32:0/1041443664 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f011406da90 0x7f0114148f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:12.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.287+0000 7f011bb4f640 1 --2- 172.21.15.32:0/1041443664 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f011406ba30 0x7f01140b4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:12.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.287+0000 7f011bb4f640 1 -- 172.21.15.32:0/1041443664 >> 172.21.15.32:0/1041443664 conn(0x7f011406e800 msgr2=0x7f0114066ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:12.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.287+0000 7f011bb4f640 1 -- 172.21.15.32:0/1041443664 shutdown_connections 2024-04-24T22:59:12.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.287+0000 7f011bb4f640 1 -- 172.21.15.32:0/1041443664 wait complete. 2024-04-24T22:59:12.299 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-04-24T22:59:11.129948+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":144,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1747,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560752,"statfs":{"total":1600000000,"available":1598210884,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.631626"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072128+0000","last_change":"2024-04-24T22:59:04.533635+0000","last_active":"2024-04-24T22:59:06.072128+0000","last_peered":"2024-04-24T22:59:06.072128+0000","last_clean":"2024-04-24T22:59:06.072128+0000","last_became_active":"2024-04-24T22:59:04.533181+0000","last_became_peered":"2024-04-24T22:59:04.533181+0000","last_unstale":"2024-04-24T22:59:06.072128+0000","last_undegraded":"2024-04-24T22:59:06.072128+0000","last_fullsized":"2024-04-24T22:59:06.072128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:19:41.674286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.571341+0000","last_change":"2024-04-24T22:59:04.532345+0000","last_active":"2024-04-24T22:59:05.571341+0000","last_peered":"2024-04-24T22:59:05.571341+0000","last_clean":"2024-04-24T22:59:05.571341+0000","last_became_active":"2024-04-24T22:59:04.532126+0000","last_became_peered":"2024-04-24T22:59:04.532126+0000","last_unstale":"2024-04-24T22:59:05.571341+0000","last_undegraded":"2024-04-24T22:59:05.571341+0000","last_fullsized":"2024-04-24T22:59:05.571341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:40:58.083354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.571343+0000","last_change":"2024-04-24T22:59:04.535851+0000","last_active":"2024-04-24T22:59:05.571343+0000","last_peered":"2024-04-24T22:59:05.571343+0000","last_clean":"2024-04-24T22:59:05.571343+0000","last_became_active":"2024-04-24T22:59:04.535290+0000","last_became_peered":"2024-04-24T22:59:04.535290+0000","last_unstale":"2024-04-24T22:59:05.571343+0000","last_undegraded":"2024-04-24T22:59:05.571343+0000","last_fullsized":"2024-04-24T22:59:05.571343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:57:53.010612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.521146+0000","last_change":"2024-04-24T22:59:04.529616+0000","last_active":"2024-04-24T22:59:05.521146+0000","last_peered":"2024-04-24T22:59:05.521146+0000","last_clean":"2024-04-24T22:59:05.521146+0000","last_became_active":"2024-04-24T22:59:04.529312+0000","last_became_peered":"2024-04-24T22:59:04.529312+0000","last_unstale":"2024-04-24T22:59:05.521146+0000","last_undegraded":"2024-04-24T22:59:05.521146+0000","last_fullsized":"2024-04-24T22:59:05.521146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:55:40.732173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072449+0000","last_change":"2024-04-24T22:59:04.531771+0000","last_active":"2024-04-24T22:59:06.072449+0000","last_peered":"2024-04-24T22:59:06.072449+0000","last_clean":"2024-04-24T22:59:06.072449+0000","last_became_active":"2024-04-24T22:59:04.529534+0000","last_became_peered":"2024-04-24T22:59:04.529534+0000","last_unstale":"2024-04-24T22:59:06.072449+0000","last_undegraded":"2024-04-24T22:59:06.072449+0000","last_fullsized":"2024-04-24T22:59:06.072449+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:35:46.694552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.521763+0000","last_change":"2024-04-24T22:59:04.529880+0000","last_active":"2024-04-24T22:59:05.521763+0000","last_peered":"2024-04-24T22:59:05.521763+0000","last_clean":"2024-04-24T22:59:05.521763+0000","last_became_active":"2024-04-24T22:59:04.529596+0000","last_became_peered":"2024-04-24T22:59:04.529596+0000","last_unstale":"2024-04-24T22:59:05.521763+0000","last_undegraded":"2024-04-24T22:59:05.521763+0000","last_fullsized":"2024-04-24T22:59:05.521763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:45:47.457534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072563+0000","last_change":"2024-04-24T22:59:04.535003+0000","last_active":"2024-04-24T22:59:06.072563+0000","last_peered":"2024-04-24T22:59:06.072563+0000","last_clean":"2024-04-24T22:59:06.072563+0000","last_became_active":"2024-04-24T22:59:04.532802+0000","last_became_peered":"2024-04-24T22:59:04.532802+0000","last_unstale":"2024-04-24T22:59:06.072563+0000","last_undegraded":"2024-04-24T22:59:06.072563+0000","last_fullsized":"2024-04-24T22:59:06.072563+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:17.932086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.571441+0000","last_change":"2024-04-24T22:59:04.537484+0000","last_active":"2024-04-24T22:59:05.571441+0000","last_peered":"2024-04-24T22:59:05.571441+0000","last_clean":"2024-04-24T22:59:05.571441+0000","last_became_active":"2024-04-24T22:59:04.537018+0000","last_became_peered":"2024-04-24T22:59:04.537018+0000","last_unstale":"2024-04-24T22:59:05.571441+0000","last_undegraded":"2024-04-24T22:59:05.571441+0000","last_fullsized":"2024-04-24T22:59:05.571441+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:50:51.056976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.802481+0000","last_change":"2024-04-24T22:59:04.524856+0000","last_active":"2024-04-24T22:59:05.802481+0000","last_peered":"2024-04-24T22:59:05.802481+0000","last_clean":"2024-04-24T22:59:05.802481+0000","last_became_active":"2024-04-24T22:59:04.524572+0000","last_became_peered":"2024-04-24T22:59:04.524572+0000","last_unstale":"2024-04-24T22:59:05.802481+0000","last_undegraded":"2024-04-24T22:59:05.802481+0000","last_fullsized":"2024-04-24T22:59:05.802481+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:41.537600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072207+0000","last_change":"2024-04-24T22:59:04.533184+0000","last_active":"2024-04-24T22:59:06.072207+0000","last_peered":"2024-04-24T22:59:06.072207+0000","last_clean":"2024-04-24T22:59:06.072207+0000","last_became_active":"2024-04-24T22:59:04.532176+0000","last_became_peered":"2024-04-24T22:59:04.532176+0000","last_unstale":"2024-04-24T22:59:06.072207+0000","last_undegraded":"2024-04-24T22:59:06.072207+0000","last_fullsized":"2024-04-24T22:59:06.072207+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:43:05.960134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072278+0000","last_change":"2024-04-24T22:59:04.533428+0000","last_active":"2024-04-24T22:59:06.072278+0000","last_peered":"2024-04-24T22:59:06.072278+0000","last_clean":"2024-04-24T22:59:06.072278+0000","last_became_active":"2024-04-24T22:59:04.532740+0000","last_became_peered":"2024-04-24T22:59:04.532740+0000","last_unstale":"2024-04-24T22:59:06.072278+0000","last_undegraded":"2024-04-24T22:59:06.072278+0000","last_fullsized":"2024-04-24T22:59:06.072278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:45:08.272236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072073+0000","last_change":"2024-04-24T22:59:04.538772+0000","last_active":"2024-04-24T22:59:06.072073+0000","last_peered":"2024-04-24T22:59:06.072073+0000","last_clean":"2024-04-24T22:59:06.072073+0000","last_became_active":"2024-04-24T22:59:04.537512+0000","last_became_peered":"2024-04-24T22:59:04.537512+0000","last_unstale":"2024-04-24T22:59:06.072073+0000","last_undegraded":"2024-04-24T22:59:06.072073+0000","last_fullsized":"2024-04-24T22:59:06.072073+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:17.297900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520613+0000","last_change":"2024-04-24T22:59:04.534720+0000","last_active":"2024-04-24T22:59:05.520613+0000","last_peered":"2024-04-24T22:59:05.520613+0000","last_clean":"2024-04-24T22:59:05.520613+0000","last_became_active":"2024-04-24T22:59:04.533802+0000","last_became_peered":"2024-04-24T22:59:04.533802+0000","last_unstale":"2024-04-24T22:59:05.520613+0000","last_undegraded":"2024-04-24T22:59:05.520613+0000","last_fullsized":"2024-04-24T22:59:05.520613+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:21:36.150878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520463+0000","last_change":"2024-04-24T22:59:04.532980+0000","last_active":"2024-04-24T22:59:05.520463+0000","last_peered":"2024-04-24T22:59:05.520463+0000","last_clean":"2024-04-24T22:59:05.520463+0000","last_became_active":"2024-04-24T22:59:04.532721+0000","last_became_peered":"2024-04-24T22:59:04.532721+0000","last_unstale":"2024-04-24T22:59:05.520463+0000","last_undegraded":"2024-04-24T22:59:05.520463+0000","last_fullsized":"2024-04-24T22:59:05.520463+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:07:57.327727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.571539+0000","last_change":"2024-04-24T22:59:04.529907+0000","last_active":"2024-04-24T22:59:05.571539+0000","last_peered":"2024-04-24T22:59:05.571539+0000","last_clean":"2024-04-24T22:59:05.571539+0000","last_became_active":"2024-04-24T22:59:04.529501+0000","last_became_peered":"2024-04-24T22:59:04.529501+0000","last_unstale":"2024-04-24T22:59:05.571539+0000","last_undegraded":"2024-04-24T22:59:05.571539+0000","last_fullsized":"2024-04-24T22:59:05.571539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:44:58.795998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520176+0000","last_change":"2024-04-24T22:59:04.531489+0000","last_active":"2024-04-24T22:59:05.520176+0000","last_peered":"2024-04-24T22:59:05.520176+0000","last_clean":"2024-04-24T22:59:05.520176+0000","last_became_active":"2024-04-24T22:59:04.531251+0000","last_became_peered":"2024-04-24T22:59:04.531251+0000","last_unstale":"2024-04-24T22:59:05.520176+0000","last_undegraded":"2024-04-24T22:59:05.520176+0000","last_fullsized":"2024-04-24T22:59:05.520176+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:04:17.107583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.519872+0000","last_change":"2024-04-24T22:59:04.533695+0000","last_active":"2024-04-24T22:59:05.519872+0000","last_peered":"2024-04-24T22:59:05.519872+0000","last_clean":"2024-04-24T22:59:05.519872+0000","last_became_active":"2024-04-24T22:59:04.533446+0000","last_became_peered":"2024-04-24T22:59:04.533446+0000","last_unstale":"2024-04-24T22:59:05.519872+0000","last_undegraded":"2024-04-24T22:59:05.519872+0000","last_fullsized":"2024-04-24T22:59:05.519872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:38.140275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072156+0000","last_change":"2024-04-24T22:59:04.534240+0000","last_active":"2024-04-24T22:59:06.072156+0000","last_peered":"2024-04-24T22:59:06.072156+0000","last_clean":"2024-04-24T22:59:06.072156+0000","last_became_active":"2024-04-24T22:59:04.533710+0000","last_became_peered":"2024-04-24T22:59:04.533710+0000","last_unstale":"2024-04-24T22:59:06.072156+0000","last_undegraded":"2024-04-24T22:59:06.072156+0000","last_fullsized":"2024-04-24T22:59:06.072156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:58:18.235537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520204+0000","last_change":"2024-04-24T22:59:04.529089+0000","last_active":"2024-04-24T22:59:05.520204+0000","last_peered":"2024-04-24T22:59:05.520204+0000","last_clean":"2024-04-24T22:59:05.520204+0000","last_became_active":"2024-04-24T22:59:04.528080+0000","last_became_peered":"2024-04-24T22:59:04.528080+0000","last_unstale":"2024-04-24T22:59:05.520204+0000","last_undegraded":"2024-04-24T22:59:05.520204+0000","last_fullsized":"2024-04-24T22:59:05.520204+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:26:32.275407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072288+0000","last_change":"2024-04-24T22:59:04.537977+0000","last_active":"2024-04-24T22:59:06.072288+0000","last_peered":"2024-04-24T22:59:06.072288+0000","last_clean":"2024-04-24T22:59:06.072288+0000","last_became_active":"2024-04-24T22:59:04.537758+0000","last_became_peered":"2024-04-24T22:59:04.537758+0000","last_unstale":"2024-04-24T22:59:06.072288+0000","last_undegraded":"2024-04-24T22:59:06.072288+0000","last_fullsized":"2024-04-24T22:59:06.072288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:02:57.321693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072591+0000","last_change":"2024-04-24T22:59:04.527359+0000","last_active":"2024-04-24T22:59:06.072591+0000","last_peered":"2024-04-24T22:59:06.072591+0000","last_clean":"2024-04-24T22:59:06.072591+0000","last_became_active":"2024-04-24T22:59:04.527093+0000","last_became_peered":"2024-04-24T22:59:04.527093+0000","last_unstale":"2024-04-24T22:59:06.072591+0000","last_undegraded":"2024-04-24T22:59:06.072591+0000","last_fullsized":"2024-04-24T22:59:06.072591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:51:52.321592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.802409+0000","last_change":"2024-04-24T22:59:04.531189+0000","last_active":"2024-04-24T22:59:05.802409+0000","last_peered":"2024-04-24T22:59:05.802409+0000","last_clean":"2024-04-24T22:59:05.802409+0000","last_became_active":"2024-04-24T22:59:04.530929+0000","last_became_peered":"2024-04-24T22:59:04.530929+0000","last_unstale":"2024-04-24T22:59:05.802409+0000","last_undegraded":"2024-04-24T22:59:05.802409+0000","last_fullsized":"2024-04-24T22:59:05.802409+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:36.401052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.571534+0000","last_change":"2024-04-24T22:59:04.537735+0000","last_active":"2024-04-24T22:59:05.571534+0000","last_peered":"2024-04-24T22:59:05.571534+0000","last_clean":"2024-04-24T22:59:05.571534+0000","last_became_active":"2024-04-24T22:59:04.537287+0000","last_became_peered":"2024-04-24T22:59:04.537287+0000","last_unstale":"2024-04-24T22:59:05.571534+0000","last_undegraded":"2024-04-24T22:59:05.571534+0000","last_fullsized":"2024-04-24T22:59:05.571534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:25.003190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.519754+0000","last_change":"2024-04-24T22:59:04.530883+0000","last_active":"2024-04-24T22:59:05.519754+0000","last_peered":"2024-04-24T22:59:05.519754+0000","last_clean":"2024-04-24T22:59:05.519754+0000","last_became_active":"2024-04-24T22:59:04.530647+0000","last_became_peered":"2024-04-24T22:59:04.530647+0000","last_unstale":"2024-04-24T22:59:05.519754+0000","last_undegraded":"2024-04-24T22:59:05.519754+0000","last_fullsized":"2024-04-24T22:59:05.519754+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:54:21.680277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.571676+0000","last_change":"2024-04-24T22:59:04.535874+0000","last_active":"2024-04-24T22:59:05.571676+0000","last_peered":"2024-04-24T22:59:05.571676+0000","last_clean":"2024-04-24T22:59:05.571676+0000","last_became_active":"2024-04-24T22:59:04.535623+0000","last_became_peered":"2024-04-24T22:59:04.535623+0000","last_unstale":"2024-04-24T22:59:05.571676+0000","last_undegraded":"2024-04-24T22:59:05.571676+0000","last_fullsized":"2024-04-24T22:59:05.571676+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:34:41.433939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.521847+0000","last_change":"2024-04-24T22:59:04.534070+0000","last_active":"2024-04-24T22:59:05.521847+0000","last_peered":"2024-04-24T22:59:05.521847+0000","last_clean":"2024-04-24T22:59:05.521847+0000","last_became_active":"2024-04-24T22:59:04.533801+0000","last_became_peered":"2024-04-24T22:59:04.533801+0000","last_unstale":"2024-04-24T22:59:05.521847+0000","last_undegraded":"2024-04-24T22:59:05.521847+0000","last_fullsized":"2024-04-24T22:59:05.521847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:51:46.927874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520818+0000","last_change":"2024-04-24T22:59:04.531614+0000","last_active":"2024-04-24T22:59:05.520818+0000","last_peered":"2024-04-24T22:59:05.520818+0000","last_clean":"2024-04-24T22:59:05.520818+0000","last_became_active":"2024-04-24T22:59:04.530752+0000","last_became_peered":"2024-04-24T22:59:04.530752+0000","last_unstale":"2024-04-24T22:59:05.520818+0000","last_undegraded":"2024-04-24T22:59:05.520818+0000","last_fullsized":"2024-04-24T22:59:05.520818+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:15:43.008089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520758+0000","last_change":"2024-04-24T22:59:04.534458+0000","last_active":"2024-04-24T22:59:05.520758+0000","last_peered":"2024-04-24T22:59:05.520758+0000","last_clean":"2024-04-24T22:59:05.520758+0000","last_became_active":"2024-04-24T22:59:04.533333+0000","last_became_peered":"2024-04-24T22:59:04.533333+0000","last_unstale":"2024-04-24T22:59:05.520758+0000","last_undegraded":"2024-04-24T22:59:05.520758+0000","last_fullsized":"2024-04-24T22:59:05.520758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:16:16.466196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.571278+0000","last_change":"2024-04-24T22:59:04.533761+0000","last_active":"2024-04-24T22:59:05.571278+0000","last_peered":"2024-04-24T22:59:05.571278+0000","last_clean":"2024-04-24T22:59:05.571278+0000","last_became_active":"2024-04-24T22:59:04.533321+0000","last_became_peered":"2024-04-24T22:59:04.533321+0000","last_unstale":"2024-04-24T22:59:05.571278+0000","last_undegraded":"2024-04-24T22:59:05.571278+0000","last_fullsized":"2024-04-24T22:59:05.571278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:19:02.943510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.519970+0000","last_change":"2024-04-24T22:59:04.528638+0000","last_active":"2024-04-24T22:59:05.519970+0000","last_peered":"2024-04-24T22:59:05.519970+0000","last_clean":"2024-04-24T22:59:05.519970+0000","last_became_active":"2024-04-24T22:59:04.528413+0000","last_became_peered":"2024-04-24T22:59:04.528413+0000","last_unstale":"2024-04-24T22:59:05.519970+0000","last_undegraded":"2024-04-24T22:59:05.519970+0000","last_fullsized":"2024-04-24T22:59:05.519970+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:29:38.333908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072338+0000","last_change":"2024-04-24T22:59:04.537062+0000","last_active":"2024-04-24T22:59:06.072338+0000","last_peered":"2024-04-24T22:59:06.072338+0000","last_clean":"2024-04-24T22:59:06.072338+0000","last_became_active":"2024-04-24T22:59:04.536623+0000","last_became_peered":"2024-04-24T22:59:04.536623+0000","last_unstale":"2024-04-24T22:59:06.072338+0000","last_undegraded":"2024-04-24T22:59:06.072338+0000","last_fullsized":"2024-04-24T22:59:06.072338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:42:07.950747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.521248+0000","last_change":"2024-04-24T22:59:04.528631+0000","last_active":"2024-04-24T22:59:05.521248+0000","last_peered":"2024-04-24T22:59:05.521248+0000","last_clean":"2024-04-24T22:59:05.521248+0000","last_became_active":"2024-04-24T22:59:04.527962+0000","last_became_peered":"2024-04-24T22:59:04.527962+0000","last_unstale":"2024-04-24T22:59:05.521248+0000","last_undegraded":"2024-04-24T22:59:05.521248+0000","last_fullsized":"2024-04-24T22:59:05.521248+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:55:25.352217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.802375+0000","last_change":"2024-04-24T22:59:04.536016+0000","last_active":"2024-04-24T22:59:05.802375+0000","last_peered":"2024-04-24T22:59:05.802375+0000","last_clean":"2024-04-24T22:59:05.802375+0000","last_became_active":"2024-04-24T22:59:04.535746+0000","last_became_peered":"2024-04-24T22:59:04.535746+0000","last_unstale":"2024-04-24T22:59:05.802375+0000","last_undegraded":"2024-04-24T22:59:05.802375+0000","last_fullsized":"2024-04-24T22:59:05.802375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:02:30.961716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.571464+0000","last_change":"2024-04-24T22:59:03.508028+0000","last_active":"2024-04-24T22:59:05.571464+0000","last_peered":"2024-04-24T22:59:05.571464+0000","last_clean":"2024-04-24T22:59:05.571464+0000","last_became_active":"2024-04-24T22:59:03.507627+0000","last_became_peered":"2024-04-24T22:59:03.507627+0000","last_unstale":"2024-04-24T22:59:05.571464+0000","last_undegraded":"2024-04-24T22:59:05.571464+0000","last_fullsized":"2024-04-24T22:59:05.571464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:40:21.857692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072150+0000","last_change":"2024-04-24T22:59:04.534868+0000","last_active":"2024-04-24T22:59:06.072150+0000","last_peered":"2024-04-24T22:59:06.072150+0000","last_clean":"2024-04-24T22:59:06.072150+0000","last_became_active":"2024-04-24T22:59:04.534232+0000","last_became_peered":"2024-04-24T22:59:04.534232+0000","last_unstale":"2024-04-24T22:59:06.072150+0000","last_undegraded":"2024-04-24T22:59:06.072150+0000","last_fullsized":"2024-04-24T22:59:06.072150+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:29.091605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.519708+0000","last_change":"2024-04-24T22:59:03.511649+0000","last_active":"2024-04-24T22:59:05.519708+0000","last_peered":"2024-04-24T22:59:05.519708+0000","last_clean":"2024-04-24T22:59:05.519708+0000","last_became_active":"2024-04-24T22:59:03.511079+0000","last_became_peered":"2024-04-24T22:59:03.511079+0000","last_unstale":"2024-04-24T22:59:05.519708+0000","last_undegraded":"2024-04-24T22:59:05.519708+0000","last_fullsized":"2024-04-24T22:59:05.519708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:08.640377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520592+0000","last_change":"2024-04-24T22:59:03.512099+0000","last_active":"2024-04-24T22:59:05.520592+0000","last_peered":"2024-04-24T22:59:05.520592+0000","last_clean":"2024-04-24T22:59:05.520592+0000","last_became_active":"2024-04-24T22:59:03.511903+0000","last_became_peered":"2024-04-24T22:59:03.511903+0000","last_unstale":"2024-04-24T22:59:05.520592+0000","last_undegraded":"2024-04-24T22:59:05.520592+0000","last_fullsized":"2024-04-24T22:59:05.520592+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:06.046861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.571807+0000","last_change":"2024-04-24T22:59:04.529177+0000","last_active":"2024-04-24T22:59:05.571807+0000","last_peered":"2024-04-24T22:59:05.571807+0000","last_clean":"2024-04-24T22:59:05.571807+0000","last_became_active":"2024-04-24T22:59:04.528730+0000","last_became_peered":"2024-04-24T22:59:04.528730+0000","last_unstale":"2024-04-24T22:59:05.571807+0000","last_undegraded":"2024-04-24T22:59:05.571807+0000","last_fullsized":"2024-04-24T22:59:05.571807+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:41:20.670964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520440+0000","last_change":"2024-04-24T22:59:03.510748+0000","last_active":"2024-04-24T22:59:05.520440+0000","last_peered":"2024-04-24T22:59:05.520440+0000","last_clean":"2024-04-24T22:59:05.520440+0000","last_became_active":"2024-04-24T22:59:03.510515+0000","last_became_peered":"2024-04-24T22:59:03.510515+0000","last_unstale":"2024-04-24T22:59:05.520440+0000","last_undegraded":"2024-04-24T22:59:05.520440+0000","last_fullsized":"2024-04-24T22:59:05.520440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:03:30.220338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072114+0000","last_change":"2024-04-24T22:59:04.523319+0000","last_active":"2024-04-24T22:59:06.072114+0000","last_peered":"2024-04-24T22:59:06.072114+0000","last_clean":"2024-04-24T22:59:06.072114+0000","last_became_active":"2024-04-24T22:59:04.523086+0000","last_became_peered":"2024-04-24T22:59:04.523086+0000","last_unstale":"2024-04-24T22:59:06.072114+0000","last_undegraded":"2024-04-24T22:59:06.072114+0000","last_fullsized":"2024-04-24T22:59:06.072114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:27.103005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.802709+0000","last_change":"2024-04-24T22:59:03.510379+0000","last_active":"2024-04-24T22:59:05.802709+0000","last_peered":"2024-04-24T22:59:05.802709+0000","last_clean":"2024-04-24T22:59:05.802709+0000","last_became_active":"2024-04-24T22:59:03.510070+0000","last_became_peered":"2024-04-24T22:59:03.510070+0000","last_unstale":"2024-04-24T22:59:05.802709+0000","last_undegraded":"2024-04-24T22:59:05.802709+0000","last_fullsized":"2024-04-24T22:59:05.802709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:21:24.085269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.521658+0000","last_change":"2024-04-24T22:59:04.534163+0000","last_active":"2024-04-24T22:59:05.521658+0000","last_peered":"2024-04-24T22:59:05.521658+0000","last_clean":"2024-04-24T22:59:05.521658+0000","last_became_active":"2024-04-24T22:59:04.533861+0000","last_became_peered":"2024-04-24T22:59:04.533861+0000","last_unstale":"2024-04-24T22:59:05.521658+0000","last_undegraded":"2024-04-24T22:59:05.521658+0000","last_fullsized":"2024-04-24T22:59:05.521658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:02:58.886576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072490+0000","last_change":"2024-04-24T22:59:03.512403+0000","last_active":"2024-04-24T22:59:06.072490+0000","last_peered":"2024-04-24T22:59:06.072490+0000","last_clean":"2024-04-24T22:59:06.072490+0000","last_became_active":"2024-04-24T22:59:03.511538+0000","last_became_peered":"2024-04-24T22:59:03.511538+0000","last_unstale":"2024-04-24T22:59:06.072490+0000","last_undegraded":"2024-04-24T22:59:06.072490+0000","last_fullsized":"2024-04-24T22:59:06.072490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:08:00.814274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.571343+0000","last_change":"2024-04-24T22:59:04.532532+0000","last_active":"2024-04-24T22:59:05.571343+0000","last_peered":"2024-04-24T22:59:05.571343+0000","last_clean":"2024-04-24T22:59:05.571343+0000","last_became_active":"2024-04-24T22:59:04.532308+0000","last_became_peered":"2024-04-24T22:59:04.532308+0000","last_unstale":"2024-04-24T22:59:05.571343+0000","last_undegraded":"2024-04-24T22:59:05.571343+0000","last_fullsized":"2024-04-24T22:59:05.571343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:07:10.769633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072409+0000","last_change":"2024-04-24T22:59:03.509245+0000","last_active":"2024-04-24T22:59:06.072409+0000","last_peered":"2024-04-24T22:59:06.072409+0000","last_clean":"2024-04-24T22:59:06.072409+0000","last_became_active":"2024-04-24T22:59:03.508959+0000","last_became_peered":"2024-04-24T22:59:03.508959+0000","last_unstale":"2024-04-24T22:59:06.072409+0000","last_undegraded":"2024-04-24T22:59:06.072409+0000","last_fullsized":"2024-04-24T22:59:06.072409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:28:24.443582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.519926+0000","last_change":"2024-04-24T22:59:04.537036+0000","last_active":"2024-04-24T22:59:05.519926+0000","last_peered":"2024-04-24T22:59:05.519926+0000","last_clean":"2024-04-24T22:59:05.519926+0000","last_became_active":"2024-04-24T22:59:04.536728+0000","last_became_peered":"2024-04-24T22:59:04.536728+0000","last_unstale":"2024-04-24T22:59:05.519926+0000","last_undegraded":"2024-04-24T22:59:05.519926+0000","last_fullsized":"2024-04-24T22:59:05.519926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:14:39.754057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.521704+0000","last_change":"2024-04-24T22:59:03.508470+0000","last_active":"2024-04-24T22:59:05.521704+0000","last_peered":"2024-04-24T22:59:05.521704+0000","last_clean":"2024-04-24T22:59:05.521704+0000","last_became_active":"2024-04-24T22:59:03.508200+0000","last_became_peered":"2024-04-24T22:59:03.508200+0000","last_unstale":"2024-04-24T22:59:05.521704+0000","last_undegraded":"2024-04-24T22:59:05.521704+0000","last_fullsized":"2024-04-24T22:59:05.521704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:54.077458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072276+0000","last_change":"2024-04-24T22:59:04.532595+0000","last_active":"2024-04-24T22:59:06.072276+0000","last_peered":"2024-04-24T22:59:06.072276+0000","last_clean":"2024-04-24T22:59:06.072276+0000","last_became_active":"2024-04-24T22:59:04.532203+0000","last_became_peered":"2024-04-24T22:59:04.532203+0000","last_unstale":"2024-04-24T22:59:06.072276+0000","last_undegraded":"2024-04-24T22:59:06.072276+0000","last_fullsized":"2024-04-24T22:59:06.072276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:27:13.861775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072047+0000","last_change":"2024-04-24T22:59:03.509839+0000","last_active":"2024-04-24T22:59:06.072047+0000","last_peered":"2024-04-24T22:59:06.072047+0000","last_clean":"2024-04-24T22:59:06.072047+0000","last_became_active":"2024-04-24T22:59:03.509598+0000","last_became_peered":"2024-04-24T22:59:03.509598+0000","last_unstale":"2024-04-24T22:59:06.072047+0000","last_undegraded":"2024-04-24T22:59:06.072047+0000","last_fullsized":"2024-04-24T22:59:06.072047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:26:55.328076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520508+0000","last_change":"2024-04-24T22:59:04.531989+0000","last_active":"2024-04-24T22:59:05.520508+0000","last_peered":"2024-04-24T22:59:05.520508+0000","last_clean":"2024-04-24T22:59:05.520508+0000","last_became_active":"2024-04-24T22:59:04.531739+0000","last_became_peered":"2024-04-24T22:59:04.531739+0000","last_unstale":"2024-04-24T22:59:05.520508+0000","last_undegraded":"2024-04-24T22:59:05.520508+0000","last_fullsized":"2024-04-24T22:59:05.520508+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:38:53.496201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520273+0000","last_change":"2024-04-24T22:59:03.504141+0000","last_active":"2024-04-24T22:59:05.520273+0000","last_peered":"2024-04-24T22:59:05.520273+0000","last_clean":"2024-04-24T22:59:05.520273+0000","last_became_active":"2024-04-24T22:59:03.503898+0000","last_became_peered":"2024-04-24T22:59:03.503898+0000","last_unstale":"2024-04-24T22:59:05.520273+0000","last_undegraded":"2024-04-24T22:59:05.520273+0000","last_fullsized":"2024-04-24T22:59:05.520273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:51:28.639755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520233+0000","last_change":"2024-04-24T22:59:04.535653+0000","last_active":"2024-04-24T22:59:05.520233+0000","last_peered":"2024-04-24T22:59:05.520233+0000","last_clean":"2024-04-24T22:59:05.520233+0000","last_became_active":"2024-04-24T22:59:04.535211+0000","last_became_peered":"2024-04-24T22:59:04.535211+0000","last_unstale":"2024-04-24T22:59:05.520233+0000","last_undegraded":"2024-04-24T22:59:05.520233+0000","last_fullsized":"2024-04-24T22:59:05.520233+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:14.793006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072608+0000","last_change":"2024-04-24T22:59:03.514558+0000","last_active":"2024-04-24T22:59:06.072608+0000","last_peered":"2024-04-24T22:59:06.072608+0000","last_clean":"2024-04-24T22:59:06.072608+0000","last_became_active":"2024-04-24T22:59:03.514168+0000","last_became_peered":"2024-04-24T22:59:03.514168+0000","last_unstale":"2024-04-24T22:59:06.072608+0000","last_undegraded":"2024-04-24T22:59:06.072608+0000","last_fullsized":"2024-04-24T22:59:06.072608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:42:06.949022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.802534+0000","last_change":"2024-04-24T22:59:04.536979+0000","last_active":"2024-04-24T22:59:05.802534+0000","last_peered":"2024-04-24T22:59:05.802534+0000","last_clean":"2024-04-24T22:59:05.802534+0000","last_became_active":"2024-04-24T22:59:04.536749+0000","last_became_peered":"2024-04-24T22:59:04.536749+0000","last_unstale":"2024-04-24T22:59:05.802534+0000","last_undegraded":"2024-04-24T22:59:05.802534+0000","last_fullsized":"2024-04-24T22:59:05.802534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:06:11.147097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072411+0000","last_change":"2024-04-24T22:59:03.507780+0000","last_active":"2024-04-24T22:59:06.072411+0000","last_peered":"2024-04-24T22:59:06.072411+0000","last_clean":"2024-04-24T22:59:06.072411+0000","last_became_active":"2024-04-24T22:59:03.507476+0000","last_became_peered":"2024-04-24T22:59:03.507476+0000","last_unstale":"2024-04-24T22:59:06.072411+0000","last_undegraded":"2024-04-24T22:59:06.072411+0000","last_fullsized":"2024-04-24T22:59:06.072411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:02.130252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520033+0000","last_change":"2024-04-24T22:59:04.532342+0000","last_active":"2024-04-24T22:59:05.520033+0000","last_peered":"2024-04-24T22:59:05.520033+0000","last_clean":"2024-04-24T22:59:05.520033+0000","last_became_active":"2024-04-24T22:59:04.531830+0000","last_became_peered":"2024-04-24T22:59:04.531830+0000","last_unstale":"2024-04-24T22:59:05.520033+0000","last_undegraded":"2024-04-24T22:59:05.520033+0000","last_fullsized":"2024-04-24T22:59:05.520033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:26.483089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072081+0000","last_change":"2024-04-24T22:59:03.511265+0000","last_active":"2024-04-24T22:59:06.072081+0000","last_peered":"2024-04-24T22:59:06.072081+0000","last_clean":"2024-04-24T22:59:06.072081+0000","last_became_active":"2024-04-24T22:59:03.510105+0000","last_became_peered":"2024-04-24T22:59:03.510105+0000","last_unstale":"2024-04-24T22:59:06.072081+0000","last_undegraded":"2024-04-24T22:59:06.072081+0000","last_fullsized":"2024-04-24T22:59:06.072081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:14:10.398082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.521760+0000","last_change":"2024-04-24T22:59:04.535582+0000","last_active":"2024-04-24T22:59:05.521760+0000","last_peered":"2024-04-24T22:59:05.521760+0000","last_clean":"2024-04-24T22:59:05.521760+0000","last_became_active":"2024-04-24T22:59:04.535184+0000","last_became_peered":"2024-04-24T22:59:04.535184+0000","last_unstale":"2024-04-24T22:59:05.521760+0000","last_undegraded":"2024-04-24T22:59:05.521760+0000","last_fullsized":"2024-04-24T22:59:05.521760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:14:13.409713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072466+0000","last_change":"2024-04-24T22:59:03.511495+0000","last_active":"2024-04-24T22:59:06.072466+0000","last_peered":"2024-04-24T22:59:06.072466+0000","last_clean":"2024-04-24T22:59:06.072466+0000","last_became_active":"2024-04-24T22:59:03.511067+0000","last_became_peered":"2024-04-24T22:59:03.511067+0000","last_unstale":"2024-04-24T22:59:06.072466+0000","last_undegraded":"2024-04-24T22:59:06.072466+0000","last_fullsized":"2024-04-24T22:59:06.072466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:25.766950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.071949+0000","last_change":"2024-04-24T22:59:04.538030+0000","last_active":"2024-04-24T22:59:06.071949+0000","last_peered":"2024-04-24T22:59:06.071949+0000","last_clean":"2024-04-24T22:59:06.071949+0000","last_became_active":"2024-04-24T22:59:04.537558+0000","last_became_peered":"2024-04-24T22:59:04.537558+0000","last_unstale":"2024-04-24T22:59:06.071949+0000","last_undegraded":"2024-04-24T22:59:06.071949+0000","last_fullsized":"2024-04-24T22:59:06.071949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:49:31.717151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.571432+0000","last_change":"2024-04-24T22:59:03.510195+0000","last_active":"2024-04-24T22:59:05.571432+0000","last_peered":"2024-04-24T22:59:05.571432+0000","last_clean":"2024-04-24T22:59:05.571432+0000","last_became_active":"2024-04-24T22:59:03.509950+0000","last_became_peered":"2024-04-24T22:59:03.509950+0000","last_unstale":"2024-04-24T22:59:05.571432+0000","last_undegraded":"2024-04-24T22:59:05.571432+0000","last_fullsized":"2024-04-24T22:59:05.571432+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:25:40.587061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072293+0000","last_change":"2024-04-24T22:59:04.535501+0000","last_active":"2024-04-24T22:59:06.072293+0000","last_peered":"2024-04-24T22:59:06.072293+0000","last_clean":"2024-04-24T22:59:06.072293+0000","last_became_active":"2024-04-24T22:59:04.534547+0000","last_became_peered":"2024-04-24T22:59:04.534547+0000","last_unstale":"2024-04-24T22:59:06.072293+0000","last_undegraded":"2024-04-24T22:59:06.072293+0000","last_fullsized":"2024-04-24T22:59:06.072293+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:49:20.073672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520927+0000","last_change":"2024-04-24T22:59:03.512241+0000","last_active":"2024-04-24T22:59:05.520927+0000","last_peered":"2024-04-24T22:59:05.520927+0000","last_clean":"2024-04-24T22:59:05.520927+0000","last_became_active":"2024-04-24T22:59:03.511991+0000","last_became_peered":"2024-04-24T22:59:03.511991+0000","last_unstale":"2024-04-24T22:59:05.520927+0000","last_undegraded":"2024-04-24T22:59:05.520927+0000","last_fullsized":"2024-04-24T22:59:05.520927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:00:24.103638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520418+0000","last_change":"2024-04-24T22:59:04.524264+0000","last_active":"2024-04-24T22:59:05.520418+0000","last_peered":"2024-04-24T22:59:05.520418+0000","last_clean":"2024-04-24T22:59:05.520418+0000","last_became_active":"2024-04-24T22:59:04.523991+0000","last_became_peered":"2024-04-24T22:59:04.523991+0000","last_unstale":"2024-04-24T22:59:05.520418+0000","last_undegraded":"2024-04-24T22:59:05.520418+0000","last_fullsized":"2024-04-24T22:59:05.520418+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:50.932498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.802551+0000","last_change":"2024-04-24T22:58:55.138810+0000","last_active":"2024-04-24T22:59:05.802551+0000","last_peered":"2024-04-24T22:59:05.802551+0000","last_clean":"2024-04-24T22:59:05.802551+0000","last_became_active":"2024-04-24T22:58:55.138627+0000","last_became_peered":"2024-04-24T22:58:55.138627+0000","last_unstale":"2024-04-24T22:59:05.802551+0000","last_undegraded":"2024-04-24T22:59:05.802551+0000","last_fullsized":"2024-04-24T22:59:05.802551+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_clean_scrub_stamp":"2024-04-24T22:58:54.127235+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:24.010002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520503+0000","last_change":"2024-04-24T22:59:03.501901+0000","last_active":"2024-04-24T22:59:05.520503+0000","last_peered":"2024-04-24T22:59:05.520503+0000","last_clean":"2024-04-24T22:59:05.520503+0000","last_became_active":"2024-04-24T22:59:03.501383+0000","last_became_peered":"2024-04-24T22:59:03.501383+0000","last_unstale":"2024-04-24T22:59:05.520503+0000","last_undegraded":"2024-04-24T22:59:05.520503+0000","last_fullsized":"2024-04-24T22:59:05.520503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:35:55.503771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072518+0000","last_change":"2024-04-24T22:59:04.531995+0000","last_active":"2024-04-24T22:59:06.072518+0000","last_peered":"2024-04-24T22:59:06.072518+0000","last_clean":"2024-04-24T22:59:06.072518+0000","last_became_active":"2024-04-24T22:59:04.531730+0000","last_became_peered":"2024-04-24T22:59:04.531730+0000","last_unstale":"2024-04-24T22:59:06.072518+0000","last_undegraded":"2024-04-24T22:59:06.072518+0000","last_fullsized":"2024-04-24T22:59:06.072518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:51:53.153038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520341+0000","last_change":"2024-04-24T22:59:03.509668+0000","last_active":"2024-04-24T22:59:05.520341+0000","last_peered":"2024-04-24T22:59:05.520341+0000","last_clean":"2024-04-24T22:59:05.520341+0000","last_became_active":"2024-04-24T22:59:03.509450+0000","last_became_peered":"2024-04-24T22:59:03.509450+0000","last_unstale":"2024-04-24T22:59:05.520341+0000","last_undegraded":"2024-04-24T22:59:05.520341+0000","last_fullsized":"2024-04-24T22:59:05.520341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:06:37.228507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520854+0000","last_change":"2024-04-24T22:59:04.530536+0000","last_active":"2024-04-24T22:59:05.520854+0000","last_peered":"2024-04-24T22:59:05.520854+0000","last_clean":"2024-04-24T22:59:05.520854+0000","last_became_active":"2024-04-24T22:59:04.530302+0000","last_became_peered":"2024-04-24T22:59:04.530302+0000","last_unstale":"2024-04-24T22:59:05.520854+0000","last_undegraded":"2024-04-24T22:59:05.520854+0000","last_fullsized":"2024-04-24T22:59:05.520854+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:38:05.122875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072508+0000","last_change":"2024-04-24T22:59:03.508258+0000","last_active":"2024-04-24T22:59:06.072508+0000","last_peered":"2024-04-24T22:59:06.072508+0000","last_clean":"2024-04-24T22:59:06.072508+0000","last_became_active":"2024-04-24T22:59:03.508013+0000","last_became_peered":"2024-04-24T22:59:03.508013+0000","last_unstale":"2024-04-24T22:59:06.072508+0000","last_undegraded":"2024-04-24T22:59:06.072508+0000","last_fullsized":"2024-04-24T22:59:06.072508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:04:08.375090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.571574+0000","last_change":"2024-04-24T22:59:04.533925+0000","last_active":"2024-04-24T22:59:05.571574+0000","last_peered":"2024-04-24T22:59:05.571574+0000","last_clean":"2024-04-24T22:59:05.571574+0000","last_became_active":"2024-04-24T22:59:04.533571+0000","last_became_peered":"2024-04-24T22:59:04.533571+0000","last_unstale":"2024-04-24T22:59:05.571574+0000","last_undegraded":"2024-04-24T22:59:05.571574+0000","last_fullsized":"2024-04-24T22:59:05.571574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:23:08.983760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520235+0000","last_change":"2024-04-24T22:59:03.503558+0000","last_active":"2024-04-24T22:59:05.520235+0000","last_peered":"2024-04-24T22:59:05.520235+0000","last_clean":"2024-04-24T22:59:05.520235+0000","last_became_active":"2024-04-24T22:59:03.503199+0000","last_became_peered":"2024-04-24T22:59:03.503199+0000","last_unstale":"2024-04-24T22:59:05.520235+0000","last_undegraded":"2024-04-24T22:59:05.520235+0000","last_fullsized":"2024-04-24T22:59:05.520235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:23:49.375509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072420+0000","last_change":"2024-04-24T22:59:04.531235+0000","last_active":"2024-04-24T22:59:06.072420+0000","last_peered":"2024-04-24T22:59:06.072420+0000","last_clean":"2024-04-24T22:59:06.072420+0000","last_became_active":"2024-04-24T22:59:04.530974+0000","last_became_peered":"2024-04-24T22:59:04.530974+0000","last_unstale":"2024-04-24T22:59:06.072420+0000","last_undegraded":"2024-04-24T22:59:06.072420+0000","last_fullsized":"2024-04-24T22:59:06.072420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:45:58.398513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.571979+0000","last_change":"2024-04-24T22:59:03.509319+0000","last_active":"2024-04-24T22:59:05.571979+0000","last_peered":"2024-04-24T22:59:05.571979+0000","last_clean":"2024-04-24T22:59:05.571979+0000","last_became_active":"2024-04-24T22:59:03.508493+0000","last_became_peered":"2024-04-24T22:59:03.508493+0000","last_unstale":"2024-04-24T22:59:05.571979+0000","last_undegraded":"2024-04-24T22:59:05.571979+0000","last_fullsized":"2024-04-24T22:59:05.571979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:49:50.756565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.521375+0000","last_change":"2024-04-24T22:59:04.527591+0000","last_active":"2024-04-24T22:59:05.521375+0000","last_peered":"2024-04-24T22:59:05.521375+0000","last_clean":"2024-04-24T22:59:05.521375+0000","last_became_active":"2024-04-24T22:59:04.527329+0000","last_became_peered":"2024-04-24T22:59:04.527329+0000","last_unstale":"2024-04-24T22:59:05.521375+0000","last_undegraded":"2024-04-24T22:59:05.521375+0000","last_fullsized":"2024-04-24T22:59:05.521375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:19:52.022880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520291+0000","last_change":"2024-04-24T22:59:03.512317+0000","last_active":"2024-04-24T22:59:05.520291+0000","last_peered":"2024-04-24T22:59:05.520291+0000","last_clean":"2024-04-24T22:59:05.520291+0000","last_became_active":"2024-04-24T22:59:03.512056+0000","last_became_peered":"2024-04-24T22:59:03.512056+0000","last_unstale":"2024-04-24T22:59:05.520291+0000","last_undegraded":"2024-04-24T22:59:05.520291+0000","last_fullsized":"2024-04-24T22:59:05.520291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:48:30.411017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520699+0000","last_change":"2024-04-24T22:59:04.530095+0000","last_active":"2024-04-24T22:59:05.520699+0000","last_peered":"2024-04-24T22:59:05.520699+0000","last_clean":"2024-04-24T22:59:05.520699+0000","last_became_active":"2024-04-24T22:59:04.529730+0000","last_became_peered":"2024-04-24T22:59:04.529730+0000","last_unstale":"2024-04-24T22:59:05.520699+0000","last_undegraded":"2024-04-24T22:59:05.520699+0000","last_fullsized":"2024-04-24T22:59:05.520699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:34.933698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.525549+0000","last_change":"2024-04-24T22:59:03.510515+0000","last_active":"2024-04-24T22:59:06.525549+0000","last_peered":"2024-04-24T22:59:06.525549+0000","last_clean":"2024-04-24T22:59:06.525549+0000","last_became_active":"2024-04-24T22:59:03.510016+0000","last_became_peered":"2024-04-24T22:59:03.510016+0000","last_unstale":"2024-04-24T22:59:06.525549+0000","last_undegraded":"2024-04-24T22:59:06.525549+0000","last_fullsized":"2024-04-24T22:59:06.525549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:51:23.939098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072377+0000","last_change":"2024-04-24T22:59:04.528509+0000","last_active":"2024-04-24T22:59:06.072377+0000","last_peered":"2024-04-24T22:59:06.072377+0000","last_clean":"2024-04-24T22:59:06.072377+0000","last_became_active":"2024-04-24T22:59:04.527604+0000","last_became_peered":"2024-04-24T22:59:04.527604+0000","last_unstale":"2024-04-24T22:59:06.072377+0000","last_undegraded":"2024-04-24T22:59:06.072377+0000","last_fullsized":"2024-04-24T22:59:06.072377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:13:55.192974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520140+0000","last_change":"2024-04-24T22:59:03.513495+0000","last_active":"2024-04-24T22:59:05.520140+0000","last_peered":"2024-04-24T22:59:05.520140+0000","last_clean":"2024-04-24T22:59:05.520140+0000","last_became_active":"2024-04-24T22:59:03.512772+0000","last_became_peered":"2024-04-24T22:59:03.512772+0000","last_unstale":"2024-04-24T22:59:05.520140+0000","last_undegraded":"2024-04-24T22:59:05.520140+0000","last_fullsized":"2024-04-24T22:59:05.520140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:49:12.718741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072288+0000","last_change":"2024-04-24T22:59:04.528601+0000","last_active":"2024-04-24T22:59:06.072288+0000","last_peered":"2024-04-24T22:59:06.072288+0000","last_clean":"2024-04-24T22:59:06.072288+0000","last_became_active":"2024-04-24T22:59:04.527812+0000","last_became_peered":"2024-04-24T22:59:04.527812+0000","last_unstale":"2024-04-24T22:59:06.072288+0000","last_undegraded":"2024-04-24T22:59:06.072288+0000","last_fullsized":"2024-04-24T22:59:06.072288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:19.252521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072624+0000","last_change":"2024-04-24T22:59:03.510200+0000","last_active":"2024-04-24T22:59:06.072624+0000","last_peered":"2024-04-24T22:59:06.072624+0000","last_clean":"2024-04-24T22:59:06.072624+0000","last_became_active":"2024-04-24T22:59:03.509783+0000","last_became_peered":"2024-04-24T22:59:03.509783+0000","last_unstale":"2024-04-24T22:59:06.072624+0000","last_undegraded":"2024-04-24T22:59:06.072624+0000","last_fullsized":"2024-04-24T22:59:06.072624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:47.755911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520361+0000","last_change":"2024-04-24T22:59:04.525874+0000","last_active":"2024-04-24T22:59:05.520361+0000","last_peered":"2024-04-24T22:59:05.520361+0000","last_clean":"2024-04-24T22:59:05.520361+0000","last_became_active":"2024-04-24T22:59:04.525094+0000","last_became_peered":"2024-04-24T22:59:04.525094+0000","last_unstale":"2024-04-24T22:59:05.520361+0000","last_undegraded":"2024-04-24T22:59:05.520361+0000","last_fullsized":"2024-04-24T22:59:05.520361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:06:23.254812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072568+0000","last_change":"2024-04-24T22:59:03.511287+0000","last_active":"2024-04-24T22:59:06.072568+0000","last_peered":"2024-04-24T22:59:06.072568+0000","last_clean":"2024-04-24T22:59:06.072568+0000","last_became_active":"2024-04-24T22:59:03.510784+0000","last_became_peered":"2024-04-24T22:59:03.510784+0000","last_unstale":"2024-04-24T22:59:06.072568+0000","last_undegraded":"2024-04-24T22:59:06.072568+0000","last_fullsized":"2024-04-24T22:59:06.072568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:19:48.671658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072736+0000","last_change":"2024-04-24T22:59:04.529663+0000","last_active":"2024-04-24T22:59:06.072736+0000","last_peered":"2024-04-24T22:59:06.072736+0000","last_clean":"2024-04-24T22:59:06.072736+0000","last_became_active":"2024-04-24T22:59:04.527858+0000","last_became_peered":"2024-04-24T22:59:04.527858+0000","last_unstale":"2024-04-24T22:59:06.072736+0000","last_undegraded":"2024-04-24T22:59:06.072736+0000","last_fullsized":"2024-04-24T22:59:06.072736+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:32:03.630786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520703+0000","last_change":"2024-04-24T22:59:03.505841+0000","last_active":"2024-04-24T22:59:05.520703+0000","last_peered":"2024-04-24T22:59:05.520703+0000","last_clean":"2024-04-24T22:59:05.520703+0000","last_became_active":"2024-04-24T22:59:03.505048+0000","last_became_peered":"2024-04-24T22:59:03.505048+0000","last_unstale":"2024-04-24T22:59:05.520703+0000","last_undegraded":"2024-04-24T22:59:05.520703+0000","last_fullsized":"2024-04-24T22:59:05.520703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:34:16.021505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072303+0000","last_change":"2024-04-24T22:59:04.532670+0000","last_active":"2024-04-24T22:59:06.072303+0000","last_peered":"2024-04-24T22:59:06.072303+0000","last_clean":"2024-04-24T22:59:06.072303+0000","last_became_active":"2024-04-24T22:59:04.532437+0000","last_became_peered":"2024-04-24T22:59:04.532437+0000","last_unstale":"2024-04-24T22:59:06.072303+0000","last_undegraded":"2024-04-24T22:59:06.072303+0000","last_fullsized":"2024-04-24T22:59:06.072303+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:08:01.815998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.519807+0000","last_change":"2024-04-24T22:59:03.509614+0000","last_active":"2024-04-24T22:59:05.519807+0000","last_peered":"2024-04-24T22:59:05.519807+0000","last_clean":"2024-04-24T22:59:05.519807+0000","last_became_active":"2024-04-24T22:59:03.509379+0000","last_became_peered":"2024-04-24T22:59:03.509379+0000","last_unstale":"2024-04-24T22:59:05.519807+0000","last_undegraded":"2024-04-24T22:59:05.519807+0000","last_fullsized":"2024-04-24T22:59:05.519807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:46:21.150240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520066+0000","last_change":"2024-04-24T22:59:04.526425+0000","last_active":"2024-04-24T22:59:05.520066+0000","last_peered":"2024-04-24T22:59:05.520066+0000","last_clean":"2024-04-24T22:59:05.520066+0000","last_became_active":"2024-04-24T22:59:04.524467+0000","last_became_peered":"2024-04-24T22:59:04.524467+0000","last_unstale":"2024-04-24T22:59:05.520066+0000","last_undegraded":"2024-04-24T22:59:05.520066+0000","last_fullsized":"2024-04-24T22:59:05.520066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:23:05.155059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520105+0000","last_change":"2024-04-24T22:59:04.535456+0000","last_active":"2024-04-24T22:59:05.520105+0000","last_peered":"2024-04-24T22:59:05.520105+0000","last_clean":"2024-04-24T22:59:05.520105+0000","last_became_active":"2024-04-24T22:59:04.534881+0000","last_became_peered":"2024-04-24T22:59:04.534881+0000","last_unstale":"2024-04-24T22:59:05.520105+0000","last_undegraded":"2024-04-24T22:59:05.520105+0000","last_fullsized":"2024-04-24T22:59:05.520105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:01:15.740234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072535+0000","last_change":"2024-04-24T22:59:03.512029+0000","last_active":"2024-04-24T22:59:06.072535+0000","last_peered":"2024-04-24T22:59:06.072535+0000","last_clean":"2024-04-24T22:59:06.072535+0000","last_became_active":"2024-04-24T22:59:03.511808+0000","last_became_peered":"2024-04-24T22:59:03.511808+0000","last_unstale":"2024-04-24T22:59:06.072535+0000","last_undegraded":"2024-04-24T22:59:06.072535+0000","last_fullsized":"2024-04-24T22:59:06.072535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:02:26.617300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.802478+0000","last_change":"2024-04-24T22:59:04.533430+0000","last_active":"2024-04-24T22:59:05.802478+0000","last_peered":"2024-04-24T22:59:05.802478+0000","last_clean":"2024-04-24T22:59:05.802478+0000","last_became_active":"2024-04-24T22:59:04.533192+0000","last_became_peered":"2024-04-24T22:59:04.533192+0000","last_unstale":"2024-04-24T22:59:05.802478+0000","last_undegraded":"2024-04-24T22:59:05.802478+0000","last_fullsized":"2024-04-24T22:59:05.802478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:50:28.246419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072505+0000","last_change":"2024-04-24T22:59:03.512319+0000","last_active":"2024-04-24T22:59:06.072505+0000","last_peered":"2024-04-24T22:59:06.072505+0000","last_clean":"2024-04-24T22:59:06.072505+0000","last_became_active":"2024-04-24T22:59:03.511788+0000","last_became_peered":"2024-04-24T22:59:03.511788+0000","last_unstale":"2024-04-24T22:59:06.072505+0000","last_undegraded":"2024-04-24T22:59:06.072505+0000","last_fullsized":"2024-04-24T22:59:06.072505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:58:17.233812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.571660+0000","last_change":"2024-04-24T22:59:04.536036+0000","last_active":"2024-04-24T22:59:05.571660+0000","last_peered":"2024-04-24T22:59:05.571660+0000","last_clean":"2024-04-24T22:59:05.571660+0000","last_became_active":"2024-04-24T22:59:04.535591+0000","last_became_peered":"2024-04-24T22:59:04.535591+0000","last_unstale":"2024-04-24T22:59:05.571660+0000","last_undegraded":"2024-04-24T22:59:05.571660+0000","last_fullsized":"2024-04-24T22:59:05.571660+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:59.369829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.571777+0000","last_change":"2024-04-24T22:59:03.508163+0000","last_active":"2024-04-24T22:59:05.571777+0000","last_peered":"2024-04-24T22:59:05.571777+0000","last_clean":"2024-04-24T22:59:05.571777+0000","last_became_active":"2024-04-24T22:59:03.507645+0000","last_became_peered":"2024-04-24T22:59:03.507645+0000","last_unstale":"2024-04-24T22:59:05.571777+0000","last_undegraded":"2024-04-24T22:59:05.571777+0000","last_fullsized":"2024-04-24T22:59:05.571777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:28:40.818581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.571435+0000","last_change":"2024-04-24T22:59:04.538784+0000","last_active":"2024-04-24T22:59:05.571435+0000","last_peered":"2024-04-24T22:59:05.571435+0000","last_clean":"2024-04-24T22:59:05.571435+0000","last_became_active":"2024-04-24T22:59:04.537912+0000","last_became_peered":"2024-04-24T22:59:04.537912+0000","last_unstale":"2024-04-24T22:59:05.571435+0000","last_undegraded":"2024-04-24T22:59:05.571435+0000","last_fullsized":"2024-04-24T22:59:05.571435+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:44.470849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072107+0000","last_change":"2024-04-24T22:59:03.513331+0000","last_active":"2024-04-24T22:59:06.072107+0000","last_peered":"2024-04-24T22:59:06.072107+0000","last_clean":"2024-04-24T22:59:06.072107+0000","last_became_active":"2024-04-24T22:59:03.513198+0000","last_became_peered":"2024-04-24T22:59:03.513198+0000","last_unstale":"2024-04-24T22:59:06.072107+0000","last_undegraded":"2024-04-24T22:59:06.072107+0000","last_fullsized":"2024-04-24T22:59:06.072107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:34:31.185473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":9,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195238,"statfs":{"total":200000000,"available":199924431,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195238,"statfs":{"total":200000000,"available":199924555,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":650,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194662,"statfs":{"total":200000000,"available":199334095,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":75,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195236,"statfs":{"total":200000000,"available":199922271,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":9,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":74,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195238,"statfs":{"total":200000000,"available":199923973,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":11,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":75,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195237,"statfs":{"total":200000000,"available":199922943,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":650,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194662,"statfs":{"total":200000000,"available":199334117,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195238,"statfs":{"total":200000000,"available":199924499,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-24T22:59:12.306 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-24T22:59:12.306 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-24T22:59:12.306 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-24T22:59:12.307 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-24T22:59:12.307 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-24T22:59:12.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.438+0000 7fdaf214f640 1 Processor -- start 2024-04-24T22:59:12.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.438+0000 7fdaf214f640 1 -- start start 2024-04-24T22:59:12.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.439+0000 7fdaf214f640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fdaec162320 0x7fdaec0ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:12.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.439+0000 7fdaf214f640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdaec0cd270 con 0x7fdaec162e10 2024-04-24T22:59:12.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.439+0000 7fdaf214f640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdaec071610 con 0x7fdaec0b5af0 2024-04-24T22:59:12.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.439+0000 7fdaf214f640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fdaec0b5660 con 0x7fdaec162320 2024-04-24T22:59:12.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.439+0000 7fdaebfff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fdaec162e10 0x7fdaec15ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47394/0 (socket says 172.21.15.32:47394) 2024-04-24T22:59:12.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.439+0000 7fdaebfff640 1 -- 172.21.15.32:0/1628842745 learned_addr learned my addr 172.21.15.32:0/1628842745 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:12.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.439+0000 7fdaeb7fe640 1 --2- 172.21.15.32:0/1628842745 >> v2:172.21.15.121:3300/0 conn(0x7fdaec162320 0x7fdaec0ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:12.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.439+0000 7fdaea7fc640 1 -- 172.21.15.32:0/1628842745 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1730593275 0 0) 0x7fdaec0cd270 con 0x7fdaec162e10 2024-04-24T22:59:12.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.439+0000 7fdaea7fc640 1 -- 172.21.15.32:0/1628842745 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdad4003520 con 0x7fdaec162e10 2024-04-24T22:59:12.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.439+0000 7fdaea7fc640 1 -- 172.21.15.32:0/1628842745 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1605735220 0 0) 0x7fdaec071610 con 0x7fdaec0b5af0 2024-04-24T22:59:12.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.439+0000 7fdaea7fc640 1 -- 172.21.15.32:0/1628842745 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdaec0cd270 con 0x7fdaec0b5af0 2024-04-24T22:59:12.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.440+0000 7fdaea7fc640 1 -- 172.21.15.32:0/1628842745 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1009919095 0 0) 0x7fdad4003520 con 0x7fdaec162e10 2024-04-24T22:59:12.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.440+0000 7fdaeb7fe640 1 -- 172.21.15.32:0/1628842745 >> v1:172.21.15.121:6789/0 conn(0x7fdaec0b5af0 legacy=0x7fdaec162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:12.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.440+0000 7fdaeb7fe640 1 -- 172.21.15.32:0/1628842745 >> v1:172.21.15.32:6789/0 conn(0x7fdaec162e10 legacy=0x7fdaec15ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:12.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.440+0000 7fdaeb7fe640 1 -- 172.21.15.32:0/1628842745 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaec0718f0 con 0x7fdaec162320 2024-04-24T22:59:12.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.440+0000 7fdaeb7fe640 1 --2- 172.21.15.32:0/1628842745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdaec162320 0x7fdaec0ccc90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdadc0097a0 tx=0x7fdadc02a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e266c75609bdb0b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:12.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.440+0000 7fdaea7fc640 1 -- 172.21.15.32:0/1628842745 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdadc037070 con 0x7fdaec162320 2024-04-24T22:59:12.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.440+0000 7fdaea7fc640 1 -- 172.21.15.32:0/1628842745 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdadc02abc0 con 0x7fdaec162320 2024-04-24T22:59:12.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.441+0000 7fdaea7fc640 1 -- 172.21.15.32:0/1628842745 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae40019f0 con 0x7fdaec162320 2024-04-24T22:59:12.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.441+0000 7fdaf214f640 1 -- 172.21.15.32:0/1628842745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdaec162320 msgr2=0x7fdaec0ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:12.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.441+0000 7fdaf214f640 1 --2- 172.21.15.32:0/1628842745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdaec162320 0x7fdaec0ccc90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdadc0097a0 tx=0x7fdadc02a130 comp rx=0 tx=0).stop 2024-04-24T22:59:12.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.441+0000 7fdaf214f640 1 -- 172.21.15.32:0/1628842745 shutdown_connections 2024-04-24T22:59:12.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.441+0000 7fdaf214f640 1 --2- 172.21.15.32:0/1628842745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdaec162320 0x7fdaec0ccc90 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:12.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.441+0000 7fdaf214f640 1 -- 172.21.15.32:0/1628842745 >> 172.21.15.32:0/1628842745 conn(0x7fdaec0d0ae0 msgr2=0x7fdaec0d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:12.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.441+0000 7fdaf214f640 1 -- 172.21.15.32:0/1628842745 shutdown_connections 2024-04-24T22:59:12.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.441+0000 7fdaf214f640 1 -- 172.21.15.32:0/1628842745 wait complete. 2024-04-24T22:59:12.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.442+0000 7fdaf214f640 1 Processor -- start 2024-04-24T22:59:12.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.442+0000 7fdaf214f640 1 -- start start 2024-04-24T22:59:12.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.443+0000 7fdaf214f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaec0b5af0 0x7fdaec1e9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:12.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.443+0000 7fdaf214f640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdaec162320 0x7fdaec0cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:12.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.443+0000 7fdaf214f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdaec162e10 0x7fdaec157750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:12.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.443+0000 7fdaf214f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdaec071bd0 con 0x7fdaec0b5af0 2024-04-24T22:59:12.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.443+0000 7fdaf214f640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdaec067950 con 0x7fdaec162e10 2024-04-24T22:59:12.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.443+0000 7fdaf214f640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fdaec071d50 con 0x7fdaec162320 2024-04-24T22:59:12.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.443+0000 7fdaeb7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaec0b5af0 0x7fdaec1e9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:12.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.443+0000 7fdaeb7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaec0b5af0 0x7fdaec1e9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38384/0 (socket says 172.21.15.32:38384) 2024-04-24T22:59:12.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.443+0000 7fdaeb7fe640 1 -- 172.21.15.32:0/1828646958 learned_addr learned my addr 172.21.15.32:0/1828646958 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:12.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.443+0000 7fdaeaffd640 1 --2- 172.21.15.32:0/1828646958 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdaec162320 0x7fdaec0cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:12.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.443+0000 7fdaebfff640 1 --2- 172.21.15.32:0/1828646958 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdaec162e10 0x7fdaec157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:12.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.443+0000 7fdaeb7fe640 1 -- 172.21.15.32:0/1828646958 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdaec162320 msgr2=0x7fdaec0cefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:12.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.443+0000 7fdaeb7fe640 1 --2- 172.21.15.32:0/1828646958 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdaec162320 0x7fdaec0cefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:12.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.443+0000 7fdaeb7fe640 1 -- 172.21.15.32:0/1828646958 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdaec162e10 msgr2=0x7fdaec157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:12.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.443+0000 7fdaeb7fe640 1 --2- 172.21.15.32:0/1828646958 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdaec162e10 0x7fdaec157750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:12.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.443+0000 7fdaeb7fe640 1 -- 172.21.15.32:0/1828646958 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaec1f4790 con 0x7fdaec0b5af0 2024-04-24T22:59:12.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.443+0000 7fdaeaffd640 1 --2- 172.21.15.32:0/1828646958 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdaec162320 0x7fdaec0cefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:59:12.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.444+0000 7fdaebfff640 1 --2- 172.21.15.32:0/1828646958 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdaec162e10 0x7fdaec157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:59:12.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.444+0000 7fdaeb7fe640 1 --2- 172.21.15.32:0/1828646958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaec0b5af0 0x7fdaec1e9460 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdadc0098c0 tx=0x7fdadc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:12.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.444+0000 7fdad3fff640 1 -- 172.21.15.32:0/1828646958 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdadc037050 con 0x7fdaec0b5af0 2024-04-24T22:59:12.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.444+0000 7fdad3fff640 1 -- 172.21.15.32:0/1828646958 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdadc043070 con 0x7fdaec0b5af0 2024-04-24T22:59:12.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.444+0000 7fdaf214f640 1 -- 172.21.15.32:0/1828646958 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaec0cf520 con 0x7fdaec0b5af0 2024-04-24T22:59:12.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.444+0000 7fdad3fff640 1 -- 172.21.15.32:0/1828646958 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdadc006e10 con 0x7fdaec0b5af0 2024-04-24T22:59:12.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.444+0000 7fdaf214f640 1 -- 172.21.15.32:0/1828646958 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaec0cfaa0 con 0x7fdaec0b5af0 2024-04-24T22:59:12.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.445+0000 7fdaf214f640 1 -- 172.21.15.32:0/1828646958 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaec068430 con 0x7fdaec0b5af0 2024-04-24T22:59:12.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.445+0000 7fdad3fff640 1 -- 172.21.15.32:0/1828646958 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdadc03ca10 con 0x7fdaec0b5af0 2024-04-24T22:59:12.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.446+0000 7fdad3fff640 1 --2- 172.21.15.32:0/1828646958 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdad806be30 0x7fdad806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:12.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.446+0000 7fdad3fff640 1 -- 172.21.15.32:0/1828646958 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fdadc0abb00 con 0x7fdaec0b5af0 2024-04-24T22:59:12.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.446+0000 7fdaeaffd640 1 --2- 172.21.15.32:0/1828646958 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdad806be30 0x7fdad806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:12.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.446+0000 7fdaeaffd640 1 --2- 172.21.15.32:0/1828646958 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdad806be30 0x7fdad806e2f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdaec06f000 tx=0x7fdae0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:12.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.450+0000 7fdad3fff640 1 -- 172.21.15.32:0/1828646958 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdaec1f4790 con 0x7fdaec0b5af0 2024-04-24T22:59:12.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.685+0000 7fdaf214f640 1 -- 172.21.15.32:0/1828646958 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fdaec0d0850 con 0x7fdaec0b5af0 2024-04-24T22:59:12.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.688+0000 7fdad3fff640 1 -- 172.21.15.32:0/1828646958 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+172944 (secure 0 0 0) 0x7fdadc07a550 con 0x7fdaec0b5af0 2024-04-24T22:59:12.691 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:59:12.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.690+0000 7fdaf214f640 1 -- 172.21.15.32:0/1828646958 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdad806be30 msgr2=0x7fdad806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:12.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.690+0000 7fdaf214f640 1 --2- 172.21.15.32:0/1828646958 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdad806be30 0x7fdad806e2f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdaec06f000 tx=0x7fdae0009040 comp rx=0 tx=0).stop 2024-04-24T22:59:12.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.690+0000 7fdaf214f640 1 -- 172.21.15.32:0/1828646958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaec0b5af0 msgr2=0x7fdaec1e9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:12.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.690+0000 7fdaf214f640 1 --2- 172.21.15.32:0/1828646958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaec0b5af0 0x7fdaec1e9460 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdadc0098c0 tx=0x7fdadc0046c0 comp rx=0 tx=0).stop 2024-04-24T22:59:12.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.691+0000 7fdaf214f640 1 -- 172.21.15.32:0/1828646958 shutdown_connections 2024-04-24T22:59:12.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.691+0000 7fdaf214f640 1 --2- 172.21.15.32:0/1828646958 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdad806be30 0x7fdad806e2f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:12.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.691+0000 7fdaf214f640 1 --2- 172.21.15.32:0/1828646958 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdaec162e10 0x7fdaec157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:12.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.691+0000 7fdaf214f640 1 --2- 172.21.15.32:0/1828646958 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdaec162320 0x7fdaec0cefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:12.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.691+0000 7fdaf214f640 1 --2- 172.21.15.32:0/1828646958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaec0b5af0 0x7fdaec1e9460 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:12.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.691+0000 7fdaf214f640 1 -- 172.21.15.32:0/1828646958 >> 172.21.15.32:0/1828646958 conn(0x7fdaec0d0ae0 msgr2=0x7fdaec15d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:12.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.691+0000 7fdaf214f640 1 -- 172.21.15.32:0/1828646958 shutdown_connections 2024-04-24T22:59:12.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.691+0000 7fdaf214f640 1 -- 172.21.15.32:0/1828646958 wait complete. 2024-04-24T22:59:12.703 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":5,"flags":0,"active_gid":4108,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":1848157943},{"type":"v1","addr":"172.21.15.121:6833","nonce":1848157943}]},"active_addr":"172.21.15.121:6833/1848157943","active_change":"2024-04-24T22:58:51.119926+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4107,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.121:0","nonce":2385568997}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.121:0","nonce":1115959223}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.121:0","nonce":341493145}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.121:0","nonce":419162824}]}]} 2024-04-24T22:59:12.706 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-24T22:59:12.706 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-24T22:59:12.706 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:59:12.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.805+0000 7f0d5667d640 1 Processor -- start 2024-04-24T22:59:12.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.805+0000 7f0d5667d640 1 -- start start 2024-04-24T22:59:12.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.806+0000 7f0d5667d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0d50158dd0 0x7f0d5006b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:12.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.806+0000 7f0d5667d640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0d5006b860 0x7f0d5006d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:12.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.806+0000 7f0d5667d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d500b6d10 con 0x7f0d5006da90 2024-04-24T22:59:12.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.806+0000 7f0d5667d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f0d5006bd10 con 0x7f0d50158dd0 2024-04-24T22:59:12.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.806+0000 7f0d5667d640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f0d5015ef10 con 0x7f0d5006b860 2024-04-24T22:59:12.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.806+0000 7f0d4ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0d50158dd0 0x7f0d5006b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:12.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.806+0000 7f0d4ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0d50158dd0 0x7f0d5006b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38388/0 (socket says 172.21.15.32:38388) 2024-04-24T22:59:12.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.806+0000 7f0d54c1a640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0d5006da90 0x7f0d50072580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47400/0 (socket says 172.21.15.32:47400) 2024-04-24T22:59:12.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.806+0000 7f0d4ffff640 1 -- 172.21.15.32:0/3958900150 learned_addr learned my addr 172.21.15.32:0/3958900150 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:12.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.806+0000 7f0d4ffff640 1 -- 172.21.15.32:0/3958900150 >> v2:172.21.15.121:3300/0 conn(0x7f0d5006b860 msgr2=0x7f0d5006d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:12.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.806+0000 7f0d4ffff640 1 --2- 172.21.15.32:0/3958900150 >> v2:172.21.15.121:3300/0 conn(0x7f0d5006b860 0x7f0d5006d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:12.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.806+0000 7f0d4ffff640 1 -- 172.21.15.32:0/3958900150 >> v1:172.21.15.32:6789/0 conn(0x7f0d5006da90 legacy=0x7f0d50072580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:12.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.806+0000 7f0d4ffff640 1 -- 172.21.15.32:0/3958900150 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d5015f1f0 con 0x7f0d50158dd0 2024-04-24T22:59:12.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.806+0000 7f0d4effd640 1 -- 172.21.15.32:0/3958900150 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 716157140 0 0) 0x7f0d500b6d10 con 0x7f0d5006da90 2024-04-24T22:59:12.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.807+0000 7f0d4ffff640 1 --2- 172.21.15.32:0/3958900150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d50158dd0 0x7f0d5006b320 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0d50070460 tx=0x7f0d4002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db7a93782ae8bbf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:12.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.807+0000 7f0d4effd640 1 -- 172.21.15.32:0/3958900150 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d40037070 con 0x7f0d50158dd0 2024-04-24T22:59:12.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.807+0000 7f0d4effd640 1 -- 172.21.15.32:0/3958900150 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d40032440 con 0x7f0d50158dd0 2024-04-24T22:59:12.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.807+0000 7f0d4effd640 1 -- 172.21.15.32:0/3958900150 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d40032810 con 0x7f0d50158dd0 2024-04-24T22:59:12.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.808+0000 7f0d5667d640 1 -- 172.21.15.32:0/3958900150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d50158dd0 msgr2=0x7f0d5006b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:12.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.808+0000 7f0d5667d640 1 --2- 172.21.15.32:0/3958900150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d50158dd0 0x7f0d5006b320 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0d50070460 tx=0x7f0d4002a270 comp rx=0 tx=0).stop 2024-04-24T22:59:12.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.808+0000 7f0d5667d640 1 -- 172.21.15.32:0/3958900150 shutdown_connections 2024-04-24T22:59:12.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.808+0000 7f0d5667d640 1 --2- 172.21.15.32:0/3958900150 >> v2:172.21.15.121:3300/0 conn(0x7f0d5006b860 0x7f0d5006d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:12.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.808+0000 7f0d5667d640 1 --2- 172.21.15.32:0/3958900150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d50158dd0 0x7f0d5006b320 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:12.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.808+0000 7f0d5667d640 1 -- 172.21.15.32:0/3958900150 >> 172.21.15.32:0/3958900150 conn(0x7f0d5006e800 msgr2=0x7f0d5006ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:12.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.808+0000 7f0d5667d640 1 -- 172.21.15.32:0/3958900150 shutdown_connections 2024-04-24T22:59:12.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.808+0000 7f0d5667d640 1 -- 172.21.15.32:0/3958900150 wait complete. 2024-04-24T22:59:12.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.809+0000 7f0d5667d640 1 Processor -- start 2024-04-24T22:59:12.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.809+0000 7f0d5667d640 1 -- start start 2024-04-24T22:59:12.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.810+0000 7f0d5667d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d5006b860 0x7f0d5014c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:12.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.810+0000 7f0d5667d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d5006da90 0x7f0d5015b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:12.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.810+0000 7f0d5667d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d50158dd0 0x7f0d500ab680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:12.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.810+0000 7f0d5667d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5015fcd0 con 0x7f0d50158dd0 2024-04-24T22:59:12.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.810+0000 7f0d5667d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5015f7b0 con 0x7f0d5006b860 2024-04-24T22:59:12.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.810+0000 7f0d5667d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f0d5006ded0 con 0x7f0d5006da90 2024-04-24T22:59:12.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.810+0000 7f0d54c1a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d50158dd0 0x7f0d500ab680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:12.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.810+0000 7f0d54c1a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d50158dd0 0x7f0d500ab680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38390/0 (socket says 172.21.15.32:38390) 2024-04-24T22:59:12.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.810+0000 7f0d54c1a640 1 -- 172.21.15.32:0/2912619379 learned_addr learned my addr 172.21.15.32:0/2912619379 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:12.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.810+0000 7f0d4ffff640 1 --2- 172.21.15.32:0/2912619379 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d5006b860 0x7f0d5014c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:12.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.810+0000 7f0d4f7fe640 1 --2- 172.21.15.32:0/2912619379 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d5006da90 0x7f0d5015b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:12.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.810+0000 7f0d54c1a640 1 -- 172.21.15.32:0/2912619379 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d5006da90 msgr2=0x7f0d5015b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:12.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.810+0000 7f0d54c1a640 1 --2- 172.21.15.32:0/2912619379 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d5006da90 0x7f0d5015b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:12.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.810+0000 7f0d54c1a640 1 -- 172.21.15.32:0/2912619379 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d5006b860 msgr2=0x7f0d5014c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:12.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.810+0000 7f0d54c1a640 1 --2- 172.21.15.32:0/2912619379 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d5006b860 0x7f0d5014c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:12.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.810+0000 7f0d54c1a640 1 -- 172.21.15.32:0/2912619379 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d5015a570 con 0x7f0d50158dd0 2024-04-24T22:59:12.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.810+0000 7f0d54c1a640 1 --2- 172.21.15.32:0/2912619379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d50158dd0 0x7f0d500ab680 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0d4800c7d0 tx=0x7f0d4800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:12.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.811+0000 7f0d4cff9640 1 -- 172.21.15.32:0/2912619379 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d48018070 con 0x7f0d50158dd0 2024-04-24T22:59:12.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.811+0000 7f0d4cff9640 1 -- 172.21.15.32:0/2912619379 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d4800a640 con 0x7f0d50158dd0 2024-04-24T22:59:12.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.811+0000 7f0d5667d640 1 -- 172.21.15.32:0/2912619379 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d5015a850 con 0x7f0d50158dd0 2024-04-24T22:59:12.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.811+0000 7f0d4cff9640 1 -- 172.21.15.32:0/2912619379 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4800aa70 con 0x7f0d50158dd0 2024-04-24T22:59:12.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.811+0000 7f0d5667d640 1 -- 172.21.15.32:0/2912619379 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d5015ae10 con 0x7f0d50158dd0 2024-04-24T22:59:12.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.812+0000 7f0d5667d640 1 -- 172.21.15.32:0/2912619379 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d50068430 con 0x7f0d50158dd0 2024-04-24T22:59:12.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.812+0000 7f0d4cff9640 1 -- 172.21.15.32:0/2912619379 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0d4801b070 con 0x7f0d50158dd0 2024-04-24T22:59:12.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.812+0000 7f0d4cff9640 1 --2- 172.21.15.32:0/2912619379 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0d3c06be60 0x7f0d3c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:12.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.813+0000 7f0d4cff9640 1 -- 172.21.15.32:0/2912619379 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0d4808a4e0 con 0x7f0d50158dd0 2024-04-24T22:59:12.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.813+0000 7f0d4ffff640 1 --2- 172.21.15.32:0/2912619379 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0d3c06be60 0x7f0d3c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:12.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.813+0000 7f0d4ffff640 1 --2- 172.21.15.32:0/2912619379 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0d3c06be60 0x7f0d3c06e320 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0d40034000 tx=0x7f0d4002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:12.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:12.817+0000 7f0d4cff9640 1 -- 172.21.15.32:0/2912619379 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0d48058e00 con 0x7f0d50158dd0 2024-04-24T22:59:13.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.002+0000 7f0d5667d640 1 -- 172.21.15.32:0/2912619379 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0d5014e4f0 con 0x7f0d50158dd0 2024-04-24T22:59:13.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.003+0000 7f0d4cff9640 1 -- 172.21.15.32:0/2912619379 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16621 (secure 0 0 0) 0x7f0d48058700 con 0x7f0d50158dd0 2024-04-24T22:59:13.004 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:59:13.005 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T22:59:05.517554+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:59:02.016639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T22:59:02.846087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:59:13.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.005+0000 7f0d5667d640 1 -- 172.21.15.32:0/2912619379 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0d3c06be60 msgr2=0x7f0d3c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.005+0000 7f0d5667d640 1 --2- 172.21.15.32:0/2912619379 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0d3c06be60 0x7f0d3c06e320 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0d40034000 tx=0x7f0d4002e000 comp rx=0 tx=0).stop 2024-04-24T22:59:13.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.005+0000 7f0d5667d640 1 -- 172.21.15.32:0/2912619379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d50158dd0 msgr2=0x7f0d500ab680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.005+0000 7f0d5667d640 1 --2- 172.21.15.32:0/2912619379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d50158dd0 0x7f0d500ab680 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0d4800c7d0 tx=0x7f0d4800cc90 comp rx=0 tx=0).stop 2024-04-24T22:59:13.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.005+0000 7f0d5667d640 1 -- 172.21.15.32:0/2912619379 shutdown_connections 2024-04-24T22:59:13.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.005+0000 7f0d5667d640 1 --2- 172.21.15.32:0/2912619379 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0d3c06be60 0x7f0d3c06e320 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.005+0000 7f0d5667d640 1 --2- 172.21.15.32:0/2912619379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d50158dd0 0x7f0d500ab680 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.005+0000 7f0d5667d640 1 --2- 172.21.15.32:0/2912619379 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d5006da90 0x7f0d5015b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.005+0000 7f0d5667d640 1 --2- 172.21.15.32:0/2912619379 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d5006b860 0x7f0d5014c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.005+0000 7f0d5667d640 1 -- 172.21.15.32:0/2912619379 >> 172.21.15.32:0/2912619379 conn(0x7f0d5006e800 msgr2=0x7f0d5014d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:13.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.005+0000 7f0d5667d640 1 -- 172.21.15.32:0/2912619379 shutdown_connections 2024-04-24T22:59:13.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.005+0000 7f0d5667d640 1 -- 172.21.15.32:0/2912619379 wait complete. 2024-04-24T22:59:13.017 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-24T22:59:13.018 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:59:13.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.117+0000 7f014d48c640 1 Processor -- start 2024-04-24T22:59:13.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.117+0000 7f014d48c640 1 -- start start 2024-04-24T22:59:13.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.118+0000 7f014d48c640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f014806ba30 0x7f014806d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.118+0000 7f014d48c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f014806da90 0x7f01480b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.118+0000 7f014d48c640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f014815ef10 con 0x7f0148158dd0 2024-04-24T22:59:13.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.118+0000 7f014d48c640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f014815f1f0 con 0x7f014806ba30 2024-04-24T22:59:13.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.118+0000 7f014d48c640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f014815f4d0 con 0x7f014806da90 2024-04-24T22:59:13.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.118+0000 7f0146ffd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0148158dd0 0x7f014806b320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47402/0 (socket says 172.21.15.32:47402) 2024-04-24T22:59:13.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.118+0000 7f0146ffd640 1 -- 172.21.15.32:0/901438355 learned_addr learned my addr 172.21.15.32:0/901438355 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:13.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.118+0000 7f013ffff640 1 --2- 172.21.15.32:0/901438355 >> v2:172.21.15.121:3300/0 conn(0x7f014806ba30 0x7f014806d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.118+0000 7f01477fe640 1 --2- 172.21.15.32:0/901438355 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f014806da90 0x7f01480b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.118+0000 7f01467fc640 1 -- 172.21.15.32:0/901438355 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1676719627 0 0) 0x7f014815ef10 con 0x7f0148158dd0 2024-04-24T22:59:13.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.118+0000 7f01467fc640 1 -- 172.21.15.32:0/901438355 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0134003520 con 0x7f0148158dd0 2024-04-24T22:59:13.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.118+0000 7f013ffff640 1 -- 172.21.15.32:0/901438355 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f014806da90 msgr2=0x7f01480b5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:13.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.118+0000 7f013ffff640 1 --2- 172.21.15.32:0/901438355 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f014806da90 0x7f01480b5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.118+0000 7f013ffff640 1 -- 172.21.15.32:0/901438355 >> v1:172.21.15.32:6789/0 conn(0x7f0148158dd0 legacy=0x7f014806b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.118+0000 7f013ffff640 1 -- 172.21.15.32:0/901438355 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f014815f7b0 con 0x7f014806ba30 2024-04-24T22:59:13.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.119+0000 7f013ffff640 1 --2- 172.21.15.32:0/901438355 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f014806ba30 0x7f014806d550 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f01300097a0 tx=0x7f013002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c7d054c35da0c7d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.119+0000 7f01467fc640 1 -- 172.21.15.32:0/901438355 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0130037070 con 0x7f014806ba30 2024-04-24T22:59:13.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.119+0000 7f01467fc640 1 -- 172.21.15.32:0/901438355 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0130032440 con 0x7f014806ba30 2024-04-24T22:59:13.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.119+0000 7f01467fc640 1 -- 172.21.15.32:0/901438355 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01300327f0 con 0x7f014806ba30 2024-04-24T22:59:13.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.120+0000 7f014d48c640 1 -- 172.21.15.32:0/901438355 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f014806ba30 msgr2=0x7f014806d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.120+0000 7f014d48c640 1 --2- 172.21.15.32:0/901438355 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f014806ba30 0x7f014806d550 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f01300097a0 tx=0x7f013002a180 comp rx=0 tx=0).stop 2024-04-24T22:59:13.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.120+0000 7f014d48c640 1 -- 172.21.15.32:0/901438355 shutdown_connections 2024-04-24T22:59:13.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.120+0000 7f014d48c640 1 --2- 172.21.15.32:0/901438355 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f014806ba30 0x7f014806d550 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.120+0000 7f014d48c640 1 --2- 172.21.15.32:0/901438355 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f014806da90 0x7f01480b5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.120+0000 7f014d48c640 1 -- 172.21.15.32:0/901438355 >> 172.21.15.32:0/901438355 conn(0x7f014806e800 msgr2=0x7f014806ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:13.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.120+0000 7f014d48c640 1 -- 172.21.15.32:0/901438355 shutdown_connections 2024-04-24T22:59:13.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.120+0000 7f014d48c640 1 -- 172.21.15.32:0/901438355 wait complete. 2024-04-24T22:59:13.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.121+0000 7f014d48c640 1 Processor -- start 2024-04-24T22:59:13.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.121+0000 7f014d48c640 1 -- start start 2024-04-24T22:59:13.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.122+0000 7f014d48c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f014806ba30 0x7f01481625c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.122+0000 7f014d48c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f014806da90 0x7f01480b4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.122+0000 7f014d48c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0148158dd0 0x7f01480b49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.122+0000 7f014d48c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f014806bfe0 con 0x7f014806da90 2024-04-24T22:59:13.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.122+0000 7f014d48c640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f014806ded0 con 0x7f0148158dd0 2024-04-24T22:59:13.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.122+0000 7f014d48c640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f01481591b0 con 0x7f014806ba30 2024-04-24T22:59:13.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.122+0000 7f013ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f014806da90 0x7f01480b4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.122+0000 7f013ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f014806da90 0x7f01480b4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38394/0 (socket says 172.21.15.32:38394) 2024-04-24T22:59:13.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.122+0000 7f013ffff640 1 -- 172.21.15.32:0/2866151744 learned_addr learned my addr 172.21.15.32:0/2866151744 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:13.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.122+0000 7f01477fe640 1 --2- 172.21.15.32:0/2866151744 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0148158dd0 0x7f01480b49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.122+0000 7f0146ffd640 1 --2- 172.21.15.32:0/2866151744 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f014806ba30 0x7f01481625c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.122+0000 7f013ffff640 1 -- 172.21.15.32:0/2866151744 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f014806ba30 msgr2=0x7f01481625c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.122+0000 7f013ffff640 1 --2- 172.21.15.32:0/2866151744 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f014806ba30 0x7f01481625c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.122+0000 7f013ffff640 1 -- 172.21.15.32:0/2866151744 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0148158dd0 msgr2=0x7f01480b49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.122+0000 7f013ffff640 1 --2- 172.21.15.32:0/2866151744 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0148158dd0 0x7f01480b49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.122+0000 7f013ffff640 1 -- 172.21.15.32:0/2866151744 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f014814fb10 con 0x7f014806da90 2024-04-24T22:59:13.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.122+0000 7f0146ffd640 1 --2- 172.21.15.32:0/2866151744 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f014806ba30 0x7f01481625c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:13.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.122+0000 7f013ffff640 1 --2- 172.21.15.32:0/2866151744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f014806da90 0x7f01480b4470 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0130035040 tx=0x7f01300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.122+0000 7f013f7fe640 1 -- 172.21.15.32:0/2866151744 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0130037040 con 0x7f014806da90 2024-04-24T22:59:13.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.123+0000 7f013f7fe640 1 -- 172.21.15.32:0/2866151744 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0130042070 con 0x7f014806da90 2024-04-24T22:59:13.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.123+0000 7f014d48c640 1 -- 172.21.15.32:0/2866151744 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f014814fdf0 con 0x7f014806da90 2024-04-24T22:59:13.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.123+0000 7f013f7fe640 1 -- 172.21.15.32:0/2866151744 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013003b930 con 0x7f014806da90 2024-04-24T22:59:13.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.123+0000 7f014d48c640 1 -- 172.21.15.32:0/2866151744 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01481503b0 con 0x7f014806da90 2024-04-24T22:59:13.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.124+0000 7f014d48c640 1 -- 172.21.15.32:0/2866151744 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f014814f8a0 con 0x7f014806da90 2024-04-24T22:59:13.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.124+0000 7f013f7fe640 1 -- 172.21.15.32:0/2866151744 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f013003bad0 con 0x7f014806da90 2024-04-24T22:59:13.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.124+0000 7f013f7fe640 1 --2- 172.21.15.32:0/2866151744 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f012806be60 0x7f012806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.125+0000 7f013f7fe640 1 -- 172.21.15.32:0/2866151744 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f01300ab6c0 con 0x7f014806da90 2024-04-24T22:59:13.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.125+0000 7f0146ffd640 1 --2- 172.21.15.32:0/2866151744 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f012806be60 0x7f012806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.125+0000 7f0146ffd640 1 --2- 172.21.15.32:0/2866151744 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f012806be60 0x7f012806e320 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0138007fd0 tx=0x7f0138009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.129+0000 7f013f7fe640 1 -- 172.21.15.32:0/2866151744 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0130079fd0 con 0x7f014806da90 2024-04-24T22:59:13.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.312+0000 7f014d48c640 1 -- 172.21.15.32:0/2866151744 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0148068430 con 0x7f014806da90 2024-04-24T22:59:13.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.313+0000 7f013f7fe640 1 -- 172.21.15.32:0/2866151744 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16621 (secure 0 0 0) 0x7f01300798d0 con 0x7f014806da90 2024-04-24T22:59:13.315 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:59:13.315 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T22:59:05.517554+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:59:02.016639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T22:59:02.846087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:59:13.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.315+0000 7f014d48c640 1 -- 172.21.15.32:0/2866151744 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f012806be60 msgr2=0x7f012806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.315+0000 7f014d48c640 1 --2- 172.21.15.32:0/2866151744 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f012806be60 0x7f012806e320 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0138007fd0 tx=0x7f0138009040 comp rx=0 tx=0).stop 2024-04-24T22:59:13.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.315+0000 7f014d48c640 1 -- 172.21.15.32:0/2866151744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f014806da90 msgr2=0x7f01480b4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.315+0000 7f014d48c640 1 --2- 172.21.15.32:0/2866151744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f014806da90 0x7f01480b4470 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0130035040 tx=0x7f01300046c0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.315+0000 7f014d48c640 1 -- 172.21.15.32:0/2866151744 shutdown_connections 2024-04-24T22:59:13.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.315+0000 7f014d48c640 1 --2- 172.21.15.32:0/2866151744 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f012806be60 0x7f012806e320 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.315+0000 7f014d48c640 1 --2- 172.21.15.32:0/2866151744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f014806da90 0x7f01480b4470 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.315+0000 7f014d48c640 1 --2- 172.21.15.32:0/2866151744 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0148158dd0 0x7f01480b49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.315+0000 7f014d48c640 1 --2- 172.21.15.32:0/2866151744 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f014806ba30 0x7f01481625c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.315+0000 7f014d48c640 1 -- 172.21.15.32:0/2866151744 >> 172.21.15.32:0/2866151744 conn(0x7f014806e800 msgr2=0x7f014814eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:13.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.315+0000 7f014d48c640 1 -- 172.21.15.32:0/2866151744 shutdown_connections 2024-04-24T22:59:13.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.315+0000 7f014d48c640 1 -- 172.21.15.32:0/2866151744 wait complete. 2024-04-24T22:59:13.328 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-24T22:59:13.328 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-24T22:59:13.329 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-24T22:59:13.329 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-24T22:59:13.329 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-24T22:59:13.330 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-24T22:59:13.330 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-24T22:59:13.330 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-24T22:59:13.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.501+0000 7f597e6b3640 1 Processor -- start 2024-04-24T22:59:13.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.501+0000 7f597e6b3640 1 -- start start 2024-04-24T22:59:13.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.503+0000 7f597e6b3640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f59780cd6c0 0x7f597808c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.503+0000 7f597e6b3640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f59780b3c00 0x7f59780cf2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.503+0000 7f597e6b3640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59780cf810 con 0x7f59780b1ea0 2024-04-24T22:59:13.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.503+0000 7f597e6b3640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f597808cc50 con 0x7f59780b3c00 2024-04-24T22:59:13.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.503+0000 7f597e6b3640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f59780cdaa0 con 0x7f59780cd6c0 2024-04-24T22:59:13.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.503+0000 7f59777fe640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f59780b3c00 0x7f59780cf2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.503+0000 7f597cc50640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f59780b1ea0 0x7f5978070600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:52206/0 (socket says 172.21.15.32:52206) 2024-04-24T22:59:13.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.503+0000 7f597cc50640 1 -- 172.21.15.32:0/2501583729 learned_addr learned my addr 172.21.15.32:0/2501583729 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:13.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.503+0000 7f5977fff640 1 --2- 172.21.15.32:0/2501583729 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f59780cd6c0 0x7f597808c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.503+0000 7f21497c1640 1 Processor -- start 2024-04-24T22:59:13.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.504+0000 7f5976ffd640 1 -- 172.21.15.32:0/2501583729 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1651919229 0 0) 0x7f59780cf810 con 0x7f59780b1ea0 2024-04-24T22:59:13.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.504+0000 7f5976ffd640 1 -- 172.21.15.32:0/2501583729 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5960003520 con 0x7f59780b1ea0 2024-04-24T22:59:13.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.504+0000 7f21497c1640 1 -- start start 2024-04-24T22:59:13.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.504+0000 7fe509dc1640 1 Processor -- start 2024-04-24T22:59:13.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.504+0000 7f59777fe640 1 -- 172.21.15.32:0/2501583729 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f59780cd6c0 msgr2=0x7f597808c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:13.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.504+0000 7f5976ffd640 1 -- 172.21.15.32:0/2501583729 <== mon.1 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3168361673 0 0) 0x7f5960003520 con 0x7f59780b1ea0 2024-04-24T22:59:13.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.504+0000 7f59777fe640 1 --2- 172.21.15.32:0/2501583729 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f59780cd6c0 0x7f597808c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.504+0000 7f59777fe640 1 -- 172.21.15.32:0/2501583729 >> v1:172.21.15.121:6789/0 conn(0x7f59780b1ea0 legacy=0x7f5978070600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.504+0000 7f59777fe640 1 -- 172.21.15.32:0/2501583729 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59780ce7f0 con 0x7f59780b3c00 2024-04-24T22:59:13.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.504+0000 7fb73359e640 1 Processor -- start 2024-04-24T22:59:13.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.504+0000 7fb73359e640 1 -- start start 2024-04-24T22:59:13.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.504+0000 7fe509dc1640 1 -- start start 2024-04-24T22:59:13.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.505+0000 7fdacd3fb640 1 Processor -- start 2024-04-24T22:59:13.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.505+0000 7fd464a93640 1 Processor -- start 2024-04-24T22:59:13.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.505+0000 7f21497c1640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f21440b7100 0x7f214406d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.505+0000 7f2142ffd640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f214415b5d0 0x7f214406c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:52210/0 (socket says 172.21.15.32:52210) 2024-04-24T22:59:13.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.505+0000 7f2142ffd640 1 -- 172.21.15.32:0/1692750568 learned_addr learned my addr 172.21.15.32:0/1692750568 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:13.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.505+0000 7f21427fc640 1 --2- 172.21.15.32:0/1692750568 >> v2:172.21.15.32:3300/0 conn(0x7f21440b7100 0x7f214406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.505+0000 7fd464a93640 1 -- start start 2024-04-24T22:59:13.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.506+0000 7f21497c1640 1 --2- 172.21.15.32:0/1692750568 >> v2:172.21.15.121:3300/0 conn(0x7f214415efd0 0x7f214406dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.505+0000 7fe509dc1640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe4fc053e50 0x7fe4fc0542e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.505+0000 7fe509dc1640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe4fc04f930 0x7fe4fc04fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.505+0000 7fe509dc1640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4fc050280 con 0x7fe4fc048100 2024-04-24T22:59:13.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.505+0000 7fe509dc1640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fe4fc0e7b90 con 0x7fe4fc04f930 2024-04-24T22:59:13.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.505+0000 7fe509dc1640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fe4fc006a90 con 0x7fe4fc053e50 2024-04-24T22:59:13.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.505+0000 7fe5037fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe4fc048100 0x7fe4fc006f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47404/0 (socket says 172.21.15.32:47404) 2024-04-24T22:59:13.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.505+0000 7fe5037fe640 1 -- 172.21.15.32:0/1759822647 learned_addr learned my addr 172.21.15.32:0/1759822647 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:13.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.506+0000 7fe502ffd640 1 --2- 172.21.15.32:0/1759822647 >> v2:172.21.15.121:3300/0 conn(0x7fe4fc053e50 0x7fe4fc0542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.506+0000 7f21497c1640 1 -- 172.21.15.32:0/1692750568 --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21440d12c0 con 0x7f214415b5d0 2024-04-24T22:59:13.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.506+0000 7fb73359e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb72c0e31d0 0x7fb72c0071e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.506+0000 7fb73359e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb72c054630 con 0x7fb72c0df210 2024-04-24T22:59:13.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.506+0000 7f21437fe640 1 --2- 172.21.15.32:0/1692750568 >> v2:172.21.15.121:3300/0 conn(0x7f214415efd0 0x7f214406dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.506+0000 7f21497c1640 1 -- 172.21.15.32:0/1692750568 --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f214406d3a0 con 0x7f21440b7100 2024-04-24T22:59:13.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.506+0000 7f21497c1640 1 -- 172.21.15.32:0/1692750568 --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f21440d1eb0 con 0x7f214415efd0 2024-04-24T22:59:13.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.506+0000 7fe3af59e640 1 Processor -- start 2024-04-24T22:59:13.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.506+0000 7fe3af59e640 1 -- start start 2024-04-24T22:59:13.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.506+0000 7fb732d9d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb72c0e31d0 0x7fb72c0071e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.506+0000 7f2141ffb640 1 -- 172.21.15.32:0/1692750568 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1003153836 0 0) 0x7f21440d12c0 con 0x7f214415b5d0 2024-04-24T22:59:13.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.506+0000 7fb732d9d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb72c0e31d0 0x7fb72c0071e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.32:41906/0 (socket says 172.21.15.32:41906) 2024-04-24T22:59:13.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.507+0000 7f2141ffb640 1 -- 172.21.15.32:0/1692750568 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f212c003520 con 0x7f214415b5d0 2024-04-24T22:59:13.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.506+0000 7fb732d9d640 1 -- 172.21.15.32:0/3989898001 learned_addr learned my addr 172.21.15.32:0/3989898001 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:13.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.505+0000 7fdacd3fb640 1 -- start start 2024-04-24T22:59:13.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.507+0000 7fdacd3fb640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdac80cdd30 0x7fdac80cfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.507+0000 7fb73359e640 1 -- 172.21.15.32:0/3989898001 --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb72c054910 con 0x7fb72c0e2d70 2024-04-24T22:59:13.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.507+0000 7fb73359e640 1 -- 172.21.15.32:0/3989898001 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fb72c0543e0 con 0x7fb72c0e31d0 2024-04-24T22:59:13.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.507+0000 7f59777fe640 1 --2- 172.21.15.32:0/2501583729 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59780b3c00 0x7f59780cf2d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5968014340 tx=0x7f5968033160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=670999b4b26f7d9c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.507+0000 7f2141ffb640 1 -- 172.21.15.32:0/1692750568 <== mon.1 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2615213399 0 0) 0x7f212c003520 con 0x7f214415b5d0 2024-04-24T22:59:13.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.507+0000 7f2141ffb640 1 -- 172.21.15.32:0/1692750568 >> v2:172.21.15.121:3300/0 conn(0x7f214415efd0 msgr2=0x7f214406dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:13.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.507+0000 7f2141ffb640 1 --2- 172.21.15.32:0/1692750568 >> v2:172.21.15.121:3300/0 conn(0x7f214415efd0 0x7f214406dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.507+0000 7f2141ffb640 1 -- 172.21.15.32:0/1692750568 >> v2:172.21.15.32:3300/0 conn(0x7f21440b7100 msgr2=0x7f214406d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:13.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.507+0000 7f2141ffb640 1 --2- 172.21.15.32:0/1692750568 >> v2:172.21.15.32:3300/0 conn(0x7f21440b7100 0x7f214406d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.507+0000 7f2141ffb640 1 -- 172.21.15.32:0/1692750568 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21440d1580 con 0x7f214415b5d0 2024-04-24T22:59:13.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.507+0000 7f2141ffb640 1 -- 172.21.15.32:0/1692750568 <== mon.1 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f2138001670 con 0x7f214415b5d0 2024-04-24T22:59:13.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.507+0000 7f2141ffb640 1 -- 172.21.15.32:0/1692750568 >> v1:172.21.15.121:6789/0 conn(0x7f214415b5d0 legacy=0x7f214406c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.507+0000 7f2141ffb640 1 --2- 172.21.15.32:0/1692750568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f212c0039b0 0x7f212c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.507+0000 7f2141ffb640 1 --2- 172.21.15.32:0/1692750568 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f212c004800 0x7f212c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.507+0000 7f2141ffb640 1 --2- 172.21.15.32:0/1692750568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f212c0051c0 0x7f212c00fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.506+0000 7fe502ffd640 1 -- 172.21.15.32:0/1759822647 >> v2:172.21.15.32:3300/0 conn(0x7fe4fc04f930 msgr2=0x7fe4fc04fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:13.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.506+0000 7fe502ffd640 1 --2- 172.21.15.32:0/1759822647 >> v2:172.21.15.32:3300/0 conn(0x7fe4fc04f930 0x7fe4fc04fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.506+0000 7fe502ffd640 1 -- 172.21.15.32:0/1759822647 >> v1:172.21.15.32:6789/0 conn(0x7fe4fc048100 legacy=0x7fe4fc006f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.506+0000 7fe502ffd640 1 -- 172.21.15.32:0/1759822647 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4fc005ec0 con 0x7fe4fc053e50 2024-04-24T22:59:13.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.507+0000 7fd464a93640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd46006c5f0 0x7fd4600b4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.507+0000 7fd464a93640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd4600b24d0 0x7fd4600b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.507+0000 7fd464a93640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4600cce60 con 0x7fd4600cca00 2024-04-24T22:59:13.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.507+0000 7fd464a93640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fd4600b3c90 con 0x7fd4600b24d0 2024-04-24T22:59:13.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.507+0000 7fd464a93640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd4600b3f70 con 0x7fd46006c5f0 2024-04-24T22:59:13.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.507+0000 7fdacd3fb640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdac80d01a0 0x7fdac806c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.507+0000 7fdacd3fb640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdac80b3490 con 0x7fdac80ce110 2024-04-24T22:59:13.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.507+0000 7fdacd3fb640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fdac815b480 con 0x7fdac80cdd30 2024-04-24T22:59:13.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.507+0000 7fdacd3fb640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fdac815b760 con 0x7fdac80d01a0 2024-04-24T22:59:13.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.507+0000 7fdac67fc640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fdac80ce110 0x7fdac815aca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:52220/0 (socket says 172.21.15.32:52220) 2024-04-24T22:59:13.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.507+0000 7fdac67fc640 1 -- 172.21.15.32:0/2764502890 learned_addr learned my addr 172.21.15.32:0/2764502890 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:13.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.507+0000 7f5976ffd640 1 -- 172.21.15.32:0/2501583729 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5968040070 con 0x7f59780b3c00 2024-04-24T22:59:13.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.507+0000 7f5976ffd640 1 -- 172.21.15.32:0/2501583729 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5968033bf0 con 0x7f59780b3c00 2024-04-24T22:59:13.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.507+0000 7f5976ffd640 1 -- 172.21.15.32:0/2501583729 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59700019f0 con 0x7f59780b3c00 2024-04-24T22:59:13.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.507+0000 7fe502ffd640 1 --2- 172.21.15.32:0/1759822647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe4fc053e50 0x7fe4fc0542e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc052e70 tx=0x7fe4f002a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d531fcabb4c278e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.507+0000 7f2141ffb640 1 -- 172.21.15.32:0/1692750568 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f21440d1eb0 con 0x7f212c0051c0 2024-04-24T22:59:13.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.507+0000 7f2141ffb640 1 -- 172.21.15.32:0/1692750568 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f212c010310 con 0x7f212c0039b0 2024-04-24T22:59:13.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.507+0000 7f2141ffb640 1 -- 172.21.15.32:0/1692750568 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f212c0104e0 con 0x7f212c004800 2024-04-24T22:59:13.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.507+0000 7f21437fe640 1 --2- 172.21.15.32:0/1692750568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f212c0051c0 0x7f212c00fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.508+0000 7f2142ffd640 1 --2- 172.21.15.32:0/1692750568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f212c0039b0 0x7f212c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.507+0000 7f796f59e640 1 Processor -- start 2024-04-24T22:59:13.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.507+0000 7f796f59e640 1 -- start start 2024-04-24T22:59:13.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.508+0000 7fdac77fe640 1 --2- 172.21.15.32:0/2764502890 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdac80d01a0 0x7fdac806c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.507+0000 7fd45dd9b640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fd4600cca00 0x7fd4600b1dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:52230/0 (socket says 172.21.15.32:52230) 2024-04-24T22:59:13.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.508+0000 7fd45dd9b640 1 -- 172.21.15.32:0/768828882 learned_addr learned my addr 172.21.15.32:0/768828882 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:13.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.508+0000 7fe3af59e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3b00b5860 0x7fe3b00cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.508+0000 7fe3af59e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3b00cea50 con 0x7fe3b00b4760 2024-04-24T22:59:13.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.508+0000 7fe3af59e640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3b00b36c0 con 0x7fe3b00b3230 2024-04-24T22:59:13.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.508+0000 7fe3af59e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe3b00cda90 con 0x7fe3b00b5860 2024-04-24T22:59:13.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.508+0000 7fe5027fc640 1 -- 172.21.15.32:0/1759822647 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f0037070 con 0x7fe4fc053e50 2024-04-24T22:59:13.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.508+0000 7fe5027fc640 1 -- 172.21.15.32:0/1759822647 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4f0032440 con 0x7fe4fc053e50 2024-04-24T22:59:13.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.508+0000 7fb732d9d640 1 -- 172.21.15.32:0/3989898001 >> v1:172.21.15.121:6789/0 conn(0x7fb72c0e2d70 legacy=0x7fb72c006ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T22:59:13.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.508+0000 7fb732d9d640 1 -- 172.21.15.32:0/3989898001 >> v1:172.21.15.32:6789/0 conn(0x7fb72c0df210 legacy=0x7fb72c03a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:13.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.508+0000 7fb732d9d640 1 -- 172.21.15.32:0/3989898001 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb72c054bf0 con 0x7fb72c0e31d0 2024-04-24T22:59:13.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.508+0000 7f21437fe640 1 -- 172.21.15.32:0/1692750568 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f212c004800 msgr2=0x7f212c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:13.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.508+0000 7f21437fe640 1 --2- 172.21.15.32:0/1692750568 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f212c004800 0x7f212c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.508+0000 7f21437fe640 1 -- 172.21.15.32:0/1692750568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f212c0039b0 msgr2=0x7f212c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:13.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.508+0000 7f21437fe640 1 --2- 172.21.15.32:0/1692750568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f212c0039b0 0x7f212c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.508+0000 7fdac6ffd640 1 --2- 172.21.15.32:0/2764502890 >> v2:172.21.15.32:3300/0 conn(0x7fdac80cdd30 0x7fdac80cfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.508+0000 7f21437fe640 1 -- 172.21.15.32:0/1692750568 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21440d1580 con 0x7f212c0051c0 2024-04-24T22:59:13.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.508+0000 7fd45e59c640 1 --2- 172.21.15.32:0/768828882 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd46006c5f0 0x7fd4600b4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.508+0000 7fdac77fe640 1 -- 172.21.15.32:0/2764502890 >> v2:172.21.15.32:3300/0 conn(0x7fdac80cdd30 msgr2=0x7fdac80cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:13.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.508+0000 7fdac77fe640 1 --2- 172.21.15.32:0/2764502890 >> v2:172.21.15.32:3300/0 conn(0x7fdac80cdd30 0x7fdac80cfc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.508+0000 7fdac77fe640 1 -- 172.21.15.32:0/2764502890 >> v1:172.21.15.121:6789/0 conn(0x7fdac80ce110 legacy=0x7fdac815aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.508+0000 7fdac77fe640 1 -- 172.21.15.32:0/2764502890 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdac815ba40 con 0x7fdac80d01a0 2024-04-24T22:59:13.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.508+0000 7f796f59e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f797014c340 0x7f79700704e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.508+0000 7f796f59e640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7970162e50 0x7f79701632b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.508+0000 7f796f59e640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79701639d0 con 0x7f797006dc00 2024-04-24T22:59:13.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.508+0000 7f796f59e640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f7970163cb0 con 0x7f797014c340 2024-04-24T22:59:13.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.508+0000 7f796f59e640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f79700cebe0 con 0x7f7970162e50 2024-04-24T22:59:13.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.508+0000 7fe5027fc640 1 -- 172.21.15.32:0/1759822647 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f0032810 con 0x7fe4fc053e50 2024-04-24T22:59:13.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.509+0000 7f796dd9b640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7970162e50 0x7f79701632b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.509+0000 7f597e6b3640 1 -- 172.21.15.32:0/2501583729 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59780b3c00 msgr2=0x7f59780cf2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.509+0000 7f597e6b3640 1 --2- 172.21.15.32:0/2501583729 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59780b3c00 0x7f59780cf2d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5968014340 tx=0x7f5968033160 comp rx=0 tx=0).stop 2024-04-24T22:59:13.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.508+0000 7fe3add9b640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fe3b00b3230 0x7fe3b00b5c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:52236/0 (socket says 172.21.15.32:52236) 2024-04-24T22:59:13.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.508+0000 7fe3add9b640 1 -- 172.21.15.32:0/3204850122 learned_addr learned my addr 172.21.15.32:0/3204850122 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:13.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.508+0000 7fe3ae59c640 1 --2- 172.21.15.32:0/3204850122 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3b00b5860 0x7fe3b00cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.509+0000 7fe3ae59c640 1 -- 172.21.15.32:0/3204850122 >> v1:172.21.15.121:6789/0 conn(0x7fe3b00b3230 legacy=0x7fe3b00b5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.509+0000 7fe3ae59c640 1 -- 172.21.15.32:0/3204850122 >> v1:172.21.15.32:6789/0 conn(0x7fe3b00b4760 legacy=0x7fe3b0070600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:13.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.509+0000 7f796dd9b640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7970162e50 0x7f79701632b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.32:48718/0 (socket says 172.21.15.32:48718) 2024-04-24T22:59:13.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.509+0000 7f796dd9b640 1 -- 172.21.15.32:0/169146569 learned_addr learned my addr 172.21.15.32:0/169146569 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:13.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.508+0000 7fd45d59a640 1 -- 172.21.15.32:0/768828882 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 422948780 0 0) 0x7fd4600cce60 con 0x7fd4600cca00 2024-04-24T22:59:13.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.508+0000 7fd45ed9d640 1 --2- 172.21.15.32:0/768828882 >> v2:172.21.15.32:3300/0 conn(0x7fd4600b24d0 0x7fd4600b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.508+0000 7f2142ffd640 1 --2- 172.21.15.32:0/1692750568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f212c0039b0 0x7f212c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:59:13.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.509+0000 7fd45d59a640 1 -- 172.21.15.32:0/768828882 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd448003520 con 0x7fd4600cca00 2024-04-24T22:59:13.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.509+0000 7fd45e59c640 1 -- 172.21.15.32:0/768828882 >> v2:172.21.15.32:3300/0 conn(0x7fd4600b24d0 msgr2=0x7fd4600b51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:13.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.509+0000 7fd45e59c640 1 --2- 172.21.15.32:0/768828882 >> v2:172.21.15.32:3300/0 conn(0x7fd4600b24d0 0x7fd4600b51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.509+0000 7fd45e59c640 1 -- 172.21.15.32:0/768828882 >> v1:172.21.15.121:6789/0 conn(0x7fd4600cca00 legacy=0x7fd4600b1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.509+0000 7fd45e59c640 1 -- 172.21.15.32:0/768828882 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4600b4250 con 0x7fd46006c5f0 2024-04-24T22:59:13.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.509+0000 7fe3ae59c640 1 -- 172.21.15.32:0/3204850122 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3b00b39a0 con 0x7fe3b00b5860 2024-04-24T22:59:13.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.509+0000 7f21437fe640 1 -- 172.21.15.32:0/1692750568 reap_dead start 2024-04-24T22:59:13.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.509+0000 7f597e6b3640 1 -- 172.21.15.32:0/2501583729 shutdown_connections 2024-04-24T22:59:13.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.509+0000 7f597e6b3640 1 --2- 172.21.15.32:0/2501583729 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59780b3c00 0x7f59780cf2d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.509+0000 7f597e6b3640 1 --2- 172.21.15.32:0/2501583729 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f59780cd6c0 0x7f597808c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.509+0000 7f597e6b3640 1 -- 172.21.15.32:0/2501583729 >> 172.21.15.32:0/2501583729 conn(0x7f59780743f0 msgr2=0x7f597806b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:13.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.509+0000 7fdac6ffd640 1 --2- 172.21.15.32:0/2764502890 >> v2:172.21.15.32:3300/0 conn(0x7fdac80cdd30 0x7fdac80cfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:59:13.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.509+0000 7f21437fe640 1 --2- 172.21.15.32:0/1692750568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f212c0051c0 0x7f212c00fdd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f213c007460 tx=0x7f212c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11db7fbe7bd1bc89 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.509+0000 7f796e59c640 1 --2- 172.21.15.32:0/169146569 >> v2:172.21.15.32:3300/0 conn(0x7f797014c340 0x7f79700704e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.509+0000 7f796d59a640 1 -- 172.21.15.32:0/169146569 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 308605341 0 0) 0x7f79701639d0 con 0x7f797006dc00 2024-04-24T22:59:13.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.510+0000 7fb732d9d640 1 --2- 172.21.15.32:0/3989898001 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb72c0e31d0 0x7fb72c0071e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb73406aa40 tx=0x7fb7340a8330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=259a92ab77bcd1f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.510+0000 7fdac77fe640 1 --2- 172.21.15.32:0/2764502890 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdac80d01a0 0x7fdac806c5f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdabc0097d0 tx=0x7fdabc02a1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5179ae90d1b5d02 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.510+0000 7f796d59a640 1 -- 172.21.15.32:0/169146569 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7958003520 con 0x7f797006dc00 2024-04-24T22:59:13.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.510+0000 7f796dd9b640 1 -- 172.21.15.32:0/169146569 >> v2:172.21.15.32:3300/0 conn(0x7f797014c340 msgr2=0x7f79700704e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:13.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.510+0000 7f796dd9b640 1 --2- 172.21.15.32:0/169146569 >> v2:172.21.15.32:3300/0 conn(0x7f797014c340 0x7f79700704e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.510+0000 7f796dd9b640 1 -- 172.21.15.32:0/169146569 >> v1:172.21.15.121:6789/0 conn(0x7f797006dc00 legacy=0x7f79700cf3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.510+0000 7fd45e59c640 1 --2- 172.21.15.32:0/768828882 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd46006c5f0 0x7fd4600b4770 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd458009d90 tx=0x7fd45802a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=baa934c38aec17ab server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.510+0000 7f796dd9b640 1 -- 172.21.15.32:0/169146569 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79700ceec0 con 0x7f7970162e50 2024-04-24T22:59:13.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.510+0000 7f2141ffb640 1 -- 172.21.15.32:0/1692750568 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2138007d70 con 0x7f212c0051c0 2024-04-24T22:59:13.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.510+0000 7f2141ffb640 1 -- 172.21.15.32:0/1692750568 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f213c02bc00 con 0x7f212c0051c0 2024-04-24T22:59:13.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.510+0000 7f796e59c640 1 --2- 172.21.15.32:0/169146569 >> v2:172.21.15.32:3300/0 conn(0x7f797014c340 0x7f79700704e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:59:13.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.510+0000 7fe509dc1640 1 -- 172.21.15.32:0/1759822647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe4fc053e50 msgr2=0x7fe4fc0542e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.510+0000 7fe509dc1640 1 --2- 172.21.15.32:0/1759822647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe4fc053e50 0x7fe4fc0542e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc052e70 tx=0x7fe4f002a250 comp rx=0 tx=0).stop 2024-04-24T22:59:13.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.510+0000 7f597e6b3640 1 -- 172.21.15.32:0/2501583729 shutdown_connections 2024-04-24T22:59:13.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.510+0000 7f2141ffb640 1 -- 172.21.15.32:0/1692750568 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f213c031070 con 0x7f212c0051c0 2024-04-24T22:59:13.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.510+0000 7f597e6b3640 1 -- 172.21.15.32:0/2501583729 wait complete. 2024-04-24T22:59:13.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.510+0000 7fe3ae59c640 1 --2- 172.21.15.32:0/3204850122 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3b00b5860 0x7fe3b00cd550 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe3a800bd50 tx=0x7fe3a802e9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=362c2cc6c3af8164 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.510+0000 7fe509dc1640 1 -- 172.21.15.32:0/1759822647 shutdown_connections 2024-04-24T22:59:13.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.510+0000 7fe509dc1640 1 --2- 172.21.15.32:0/1759822647 >> v2:172.21.15.32:3300/0 conn(0x7fe4fc04f930 0x7fe4fc04fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.510+0000 7fe509dc1640 1 --2- 172.21.15.32:0/1759822647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe4fc053e50 0x7fe4fc0542e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.510+0000 7fe509dc1640 1 -- 172.21.15.32:0/1759822647 >> 172.21.15.32:0/1759822647 conn(0x7fe4fc009fe0 msgr2=0x7fe4fc00a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:13.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.510+0000 7fe3ad59a640 1 -- 172.21.15.32:0/3204850122 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a8039020 con 0x7fe3b00b5860 2024-04-24T22:59:13.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.510+0000 7fe3ad59a640 1 -- 172.21.15.32:0/3204850122 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3a8035850 con 0x7fe3b00b5860 2024-04-24T22:59:13.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.510+0000 7fe3ad59a640 1 -- 172.21.15.32:0/3204850122 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a8035c40 con 0x7fe3b00b5860 2024-04-24T22:59:13.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.510+0000 7fb73159a640 1 -- 172.21.15.32:0/3989898001 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7340b5070 con 0x7fb72c0e31d0 2024-04-24T22:59:13.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.510+0000 7fb73159a640 1 -- 172.21.15.32:0/3989898001 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7340b0680 con 0x7fb72c0e31d0 2024-04-24T22:59:13.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.511+0000 7fb73159a640 1 -- 172.21.15.32:0/3989898001 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7340b0af0 con 0x7fb72c0e31d0 2024-04-24T22:59:13.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.510+0000 7fdac5ffb640 1 -- 172.21.15.32:0/2764502890 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc037070 con 0x7fdac80d01a0 2024-04-24T22:59:13.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.510+0000 7fdac5ffb640 1 -- 172.21.15.32:0/2764502890 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdabc032440 con 0x7fdac80d01a0 2024-04-24T22:59:13.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.510+0000 7fdac5ffb640 1 -- 172.21.15.32:0/2764502890 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc032850 con 0x7fdac80d01a0 2024-04-24T22:59:13.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.511+0000 7fd45d59a640 1 -- 172.21.15.32:0/768828882 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd458007740 con 0x7fd46006c5f0 2024-04-24T22:59:13.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.511+0000 7fd45d59a640 1 -- 172.21.15.32:0/768828882 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4580078e0 con 0x7fd46006c5f0 2024-04-24T22:59:13.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.511+0000 7fd45d59a640 1 -- 172.21.15.32:0/768828882 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd458007cf0 con 0x7fd46006c5f0 2024-04-24T22:59:13.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.511+0000 7fe509dc1640 1 -- 172.21.15.32:0/1759822647 shutdown_connections 2024-04-24T22:59:13.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.511+0000 7fdacd3fb640 1 -- 172.21.15.32:0/2764502890 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdac80d01a0 msgr2=0x7fdac806c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.511+0000 7fdacd3fb640 1 --2- 172.21.15.32:0/2764502890 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdac80d01a0 0x7fdac806c5f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdabc0097d0 tx=0x7fdabc02a1e0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.511+0000 7f21497c1640 1 -- 172.21.15.32:0/1692750568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f212c0051c0 msgr2=0x7f212c00fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.511+0000 7f21497c1640 1 --2- 172.21.15.32:0/1692750568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f212c0051c0 0x7f212c00fdd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f213c007460 tx=0x7f212c005830 comp rx=0 tx=0).stop 2024-04-24T22:59:13.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.511+0000 7fdacd3fb640 1 -- 172.21.15.32:0/2764502890 shutdown_connections 2024-04-24T22:59:13.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.511+0000 7fdacd3fb640 1 --2- 172.21.15.32:0/2764502890 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdac80d01a0 0x7fdac806c5f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.511+0000 7fdacd3fb640 1 --2- 172.21.15.32:0/2764502890 >> v2:172.21.15.32:3300/0 conn(0x7fdac80cdd30 0x7fdac80cfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.511+0000 7fdacd3fb640 1 -- 172.21.15.32:0/2764502890 >> 172.21.15.32:0/2764502890 conn(0x7fdac80b2400 msgr2=0x7fdac80b2810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:13.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.511+0000 7f21497c1640 1 -- 172.21.15.32:0/1692750568 shutdown_connections 2024-04-24T22:59:13.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.511+0000 7f21497c1640 1 --2- 172.21.15.32:0/1692750568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f212c0051c0 0x7f212c00fdd0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.511+0000 7f21497c1640 1 -- 172.21.15.32:0/1692750568 >> 172.21.15.32:0/1692750568 conn(0x7f21440b60c0 msgr2=0x7f21440b6780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:13.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.511+0000 7f796dd9b640 1 --2- 172.21.15.32:0/169146569 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7970162e50 0x7f79701632b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7970068370 tx=0x7f796802a590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79b6b1332b7e587b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.511+0000 7fe3af59e640 1 -- 172.21.15.32:0/3204850122 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3b00b5860 msgr2=0x7fe3b00cd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.511+0000 7fe3af59e640 1 --2- 172.21.15.32:0/3204850122 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3b00b5860 0x7fe3b00cd550 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe3a800bd50 tx=0x7fe3a802e9e0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.511+0000 7fdacd3fb640 1 -- 172.21.15.32:0/2764502890 shutdown_connections 2024-04-24T22:59:13.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.511+0000 7f796d59a640 1 -- 172.21.15.32:0/169146569 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7964001670 con 0x7f7970162e50 2024-04-24T22:59:13.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.511+0000 7fe509dc1640 1 -- 172.21.15.32:0/1759822647 wait complete. 2024-04-24T22:59:13.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.511+0000 7f796d59a640 1 -- 172.21.15.32:0/169146569 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7968031400 con 0x7f7970162e50 2024-04-24T22:59:13.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.511+0000 7f796d59a640 1 -- 172.21.15.32:0/169146569 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79680317d0 con 0x7f7970162e50 2024-04-24T22:59:13.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.511+0000 7fe3af59e640 1 -- 172.21.15.32:0/3204850122 shutdown_connections 2024-04-24T22:59:13.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.511+0000 7fe3af59e640 1 --2- 172.21.15.32:0/3204850122 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3b00b5860 0x7fe3b00cd550 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.511+0000 7fe3af59e640 1 -- 172.21.15.32:0/3204850122 >> 172.21.15.32:0/3204850122 conn(0x7fe3b00b1f30 msgr2=0x7fe3b00b2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:13.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.511+0000 7fd464a93640 1 -- 172.21.15.32:0/768828882 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd46006c5f0 msgr2=0x7fd4600b4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.511+0000 7fd464a93640 1 --2- 172.21.15.32:0/768828882 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd46006c5f0 0x7fd4600b4770 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd458009d90 tx=0x7fd45802a710 comp rx=0 tx=0).stop 2024-04-24T22:59:13.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.512+0000 7fb73359e640 1 -- 172.21.15.32:0/3989898001 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb72c0e31d0 msgr2=0x7fb72c0071e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.512+0000 7fb73359e640 1 --2- 172.21.15.32:0/3989898001 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb72c0e31d0 0x7fb72c0071e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb73406aa40 tx=0x7fb7340a8330 comp rx=0 tx=0).stop 2024-04-24T22:59:13.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.512+0000 7fe3af59e640 1 -- 172.21.15.32:0/3204850122 shutdown_connections 2024-04-24T22:59:13.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.512+0000 7fd464a93640 1 -- 172.21.15.32:0/768828882 shutdown_connections 2024-04-24T22:59:13.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.512+0000 7fd464a93640 1 --2- 172.21.15.32:0/768828882 >> v2:172.21.15.32:3300/0 conn(0x7fd4600b24d0 0x7fd4600b51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.512+0000 7fd464a93640 1 --2- 172.21.15.32:0/768828882 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd46006c5f0 0x7fd4600b4770 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.512+0000 7fd464a93640 1 -- 172.21.15.32:0/768828882 >> 172.21.15.32:0/768828882 conn(0x7fd460074210 msgr2=0x7fd460074620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:13.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.512+0000 7fb73359e640 1 -- 172.21.15.32:0/3989898001 shutdown_connections 2024-04-24T22:59:13.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.512+0000 7fb73359e640 1 --2- 172.21.15.32:0/3989898001 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb72c0e31d0 0x7fb72c0071e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.512+0000 7fb73359e640 1 -- 172.21.15.32:0/3989898001 >> 172.21.15.32:0/3989898001 conn(0x7fb72c00a3f0 msgr2=0x7fb72c00a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:13.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.512+0000 7fe3af59e640 1 -- 172.21.15.32:0/3204850122 wait complete. 2024-04-24T22:59:13.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.511+0000 7fdacd3fb640 1 -- 172.21.15.32:0/2764502890 wait complete. 2024-04-24T22:59:13.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.512+0000 7f597e6b3640 1 Processor -- start 2024-04-24T22:59:13.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.512+0000 7f597e6b3640 1 -- start start 2024-04-24T22:59:13.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.512+0000 7fd464a93640 1 -- 172.21.15.32:0/768828882 shutdown_connections 2024-04-24T22:59:13.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.512+0000 7fd464a93640 1 -- 172.21.15.32:0/768828882 wait complete. 2024-04-24T22:59:13.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.512+0000 7fb73359e640 1 -- 172.21.15.32:0/3989898001 shutdown_connections 2024-04-24T22:59:13.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.512+0000 7fb73359e640 1 -- 172.21.15.32:0/3989898001 wait complete. 2024-04-24T22:59:13.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.512+0000 7f796f59e640 1 -- 172.21.15.32:0/169146569 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7970162e50 msgr2=0x7f79701632b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.512+0000 7f796f59e640 1 --2- 172.21.15.32:0/169146569 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7970162e50 0x7f79701632b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7970068370 tx=0x7f796802a590 comp rx=0 tx=0).stop 2024-04-24T22:59:13.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.512+0000 7f796f59e640 1 -- 172.21.15.32:0/169146569 shutdown_connections 2024-04-24T22:59:13.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.512+0000 7f796f59e640 1 --2- 172.21.15.32:0/169146569 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7970162e50 0x7f79701632b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.512+0000 7f796f59e640 1 --2- 172.21.15.32:0/169146569 >> v2:172.21.15.32:3300/0 conn(0x7f797014c340 0x7f79700704e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.512+0000 7f796f59e640 1 -- 172.21.15.32:0/169146569 >> 172.21.15.32:0/169146569 conn(0x7f797006c5f0 msgr2=0x7f797006ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:13.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.513+0000 7f796f59e640 1 -- 172.21.15.32:0/169146569 shutdown_connections 2024-04-24T22:59:13.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.513+0000 7f796f59e640 1 -- 172.21.15.32:0/169146569 wait complete. 2024-04-24T22:59:13.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.513+0000 7fe509dc1640 1 Processor -- start 2024-04-24T22:59:13.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.513+0000 7fdacd3fb640 1 Processor -- start 2024-04-24T22:59:13.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.513+0000 7f21497c1640 1 -- 172.21.15.32:0/1692750568 shutdown_connections 2024-04-24T22:59:13.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.513+0000 7f21497c1640 1 -- 172.21.15.32:0/1692750568 wait complete. 2024-04-24T22:59:13.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.513+0000 7fe509dc1640 1 -- start start 2024-04-24T22:59:13.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.513+0000 7f597e6b3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59780b1ea0 0x7f5978164d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.514+0000 7fe3af59e640 1 Processor -- start 2024-04-24T22:59:13.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.513+0000 7f597e6b3640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f59780b3c00 0x7f5978156670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.513+0000 7f597e6b3640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59780cd6c0 0x7f59781639a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.513+0000 7f597e6b3640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f59780b4680 con 0x7f59780b1ea0 2024-04-24T22:59:13.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.514+0000 7f59777fe640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f59780b3c00 0x7f5978156670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.513+0000 7f597e6b3640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f59780b4060 con 0x7f59780cd6c0 2024-04-24T22:59:13.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.514+0000 7f597e6b3640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f59780b4800 con 0x7f59780b3c00 2024-04-24T22:59:13.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.514+0000 7f597cc50640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59780cd6c0 0x7f59781639a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.514+0000 7f597cc50640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59780cd6c0 0x7f59781639a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.32:48720/0 (socket says 172.21.15.32:48720) 2024-04-24T22:59:13.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.514+0000 7f597cc50640 1 -- 172.21.15.32:0/2132117252 learned_addr learned my addr 172.21.15.32:0/2132117252 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:13.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.514+0000 7f59777fe640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f59780b3c00 0x7f5978156670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.32:41966/0 (socket says 172.21.15.32:41966) 2024-04-24T22:59:13.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.515+0000 7fd464a93640 1 Processor -- start 2024-04-24T22:59:13.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.515+0000 7fe3af59e640 1 -- start start 2024-04-24T22:59:13.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.515+0000 7fd464a93640 1 -- start start 2024-04-24T22:59:13.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.514+0000 7fdacd3fb640 1 -- start start 2024-04-24T22:59:13.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.515+0000 7fdacd3fb640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdac80cdd30 0x7fdac81f9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.515+0000 7fb73359e640 1 Processor -- start 2024-04-24T22:59:13.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.515+0000 7fdacd3fb640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdac80ce110 0x7fdac80ad960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.514+0000 7f5977fff640 1 --2- 172.21.15.32:0/2132117252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59780b1ea0 0x7f5978164d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.514+0000 7f597cc50640 1 -- 172.21.15.32:0/2132117252 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f59780b3c00 msgr2=0x7f5978156670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.515+0000 7f597cc50640 1 --2- 172.21.15.32:0/2132117252 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f59780b3c00 0x7f5978156670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.515+0000 7f597cc50640 1 -- 172.21.15.32:0/2132117252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59780b1ea0 msgr2=0x7f5978164d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.515+0000 7f597cc50640 1 --2- 172.21.15.32:0/2132117252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59780b1ea0 0x7f5978164d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.515+0000 7f597cc50640 1 -- 172.21.15.32:0/2132117252 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5978153ac0 con 0x7f59780cd6c0 2024-04-24T22:59:13.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.515+0000 7f5977fff640 1 --2- 172.21.15.32:0/2132117252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59780b1ea0 0x7f5978164d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:59:13.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.515+0000 7fdac6ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdac80cdd30 0x7fdac81f9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.515+0000 7fdac6ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdac80cdd30 0x7fdac81f9af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.32:48734/0 (socket says 172.21.15.32:48734) 2024-04-24T22:59:13.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.515+0000 7fe509dc1640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe4fc048100 0x7fe4fc0e5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.515+0000 7fdacd3fb640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdac80d01a0 0x7fdac8156690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.515+0000 7f597cc50640 1 --2- 172.21.15.32:0/2132117252 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59780cd6c0 0x7f59781639a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5970001700 tx=0x7f597000a430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.515+0000 7fdac67fc640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdac80ce110 0x7fdac80ad960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.515+0000 7fdac67fc640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdac80ce110 0x7fdac80ad960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.32:41972/0 (socket says 172.21.15.32:41972) 2024-04-24T22:59:13.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.515+0000 7fe509dc1640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe4fc04f930 0x7fe4fc17d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.515+0000 7fdac6ffd640 1 -- 172.21.15.32:0/2902318319 learned_addr learned my addr 172.21.15.32:0/2902318319 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:13.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.515+0000 7fb73359e640 1 -- start start 2024-04-24T22:59:13.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.515+0000 7fdacd3fb640 1 -- 172.21.15.32:0/2902318319 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdac806cec0 con 0x7fdac80d01a0 2024-04-24T22:59:13.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.515+0000 7fdacd3fb640 1 -- 172.21.15.32:0/2902318319 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdac806cd40 con 0x7fdac80cdd30 2024-04-24T22:59:13.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.515+0000 7fdacd3fb640 1 -- 172.21.15.32:0/2902318319 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fdac80b4c40 con 0x7fdac80ce110 2024-04-24T22:59:13.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.515+0000 7fdac77fe640 1 --2- 172.21.15.32:0/2902318319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdac80d01a0 0x7fdac8156690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.515+0000 7f21497c1640 1 Processor -- start 2024-04-24T22:59:13.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.516+0000 7f21497c1640 1 -- start start 2024-04-24T22:59:13.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.515+0000 7fe5037fe640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe4fc048100 0x7fe4fc0e5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.515+0000 7fe502ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe4fc04f930 0x7fe4fc17d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.515+0000 7fe502ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe4fc04f930 0x7fe4fc17d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.32:48750/0 (socket says 172.21.15.32:48750) 2024-04-24T22:59:13.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.515+0000 7fe5037fe640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe4fc048100 0x7fe4fc0e5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.32:41984/0 (socket says 172.21.15.32:41984) 2024-04-24T22:59:13.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.516+0000 7fe502ffd640 1 -- 172.21.15.32:0/3610651407 learned_addr learned my addr 172.21.15.32:0/3610651407 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:13.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.516+0000 7fe509dc1640 1 --2- 172.21.15.32:0/3610651407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4fc053e50 0x7fe4fc038510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.516+0000 7fdac6ffd640 1 -- 172.21.15.32:0/2902318319 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdac80ce110 msgr2=0x7fdac80ad960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.516+0000 7fdac6ffd640 1 --2- 172.21.15.32:0/2902318319 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdac80ce110 0x7fdac80ad960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.516+0000 7fdac6ffd640 1 -- 172.21.15.32:0/2902318319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdac80d01a0 msgr2=0x7fdac8156690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.516+0000 7fdac6ffd640 1 --2- 172.21.15.32:0/2902318319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdac80d01a0 0x7fdac8156690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.516+0000 7fdac6ffd640 1 -- 172.21.15.32:0/2902318319 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdac81fce80 con 0x7fdac80cdd30 2024-04-24T22:59:13.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.516+0000 7fe509dc1640 1 -- 172.21.15.32:0/3610651407 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe4fc006440 con 0x7fe4fc053e50 2024-04-24T22:59:13.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.516+0000 7fe509dc1640 1 -- 172.21.15.32:0/3610651407 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe4fc006160 con 0x7fe4fc04f930 2024-04-24T22:59:13.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.516+0000 7fe509dc1640 1 -- 172.21.15.32:0/3610651407 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe4fc006720 con 0x7fe4fc048100 2024-04-24T22:59:13.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.516+0000 7fe503fff640 1 --2- 172.21.15.32:0/3610651407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4fc053e50 0x7fe4fc038510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.516+0000 7fdac77fe640 1 --2- 172.21.15.32:0/2902318319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdac80d01a0 0x7fdac8156690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:13.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.516+0000 7f5974ff9640 1 -- 172.21.15.32:0/2132117252 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5970017070 con 0x7f59780cd6c0 2024-04-24T22:59:13.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.516+0000 7fe3af59e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3b00b4760 0x7fe3b0150170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.516+0000 7fe3af59e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3b00b5860 0x7fe3b0152c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.516+0000 7fd464a93640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd46006c5f0 0x7fd460147ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.516+0000 7fd464a93640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd4600b24d0 0x7fd460070d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.516+0000 7fe3af59e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3b02010a0 0x7fe3b01518d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.516+0000 7fdac6ffd640 1 --2- 172.21.15.32:0/2902318319 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdac80cdd30 0x7fdac81f9af0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdac000c7f0 tx=0x7fdac000cbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.516+0000 7fd464a93640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4600cca00 0x7fd460148b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.516+0000 7f5974ff9640 1 -- 172.21.15.32:0/2132117252 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f597000ce50 con 0x7f59780cd6c0 2024-04-24T22:59:13.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.516+0000 7f5974ff9640 1 -- 172.21.15.32:0/2132117252 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5970012620 con 0x7f59780cd6c0 2024-04-24T22:59:13.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.516+0000 7f597e6b3640 1 -- 172.21.15.32:0/2132117252 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5978153da0 con 0x7f59780cd6c0 2024-04-24T22:59:13.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.516+0000 7f597e6b3640 1 -- 172.21.15.32:0/2132117252 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5978154360 con 0x7f59780cd6c0 2024-04-24T22:59:13.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.516+0000 7fd464a93640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd4600d0c50 con 0x7fd4600cca00 2024-04-24T22:59:13.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.517+0000 7fd464a93640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd4600d0ad0 con 0x7fd46006c5f0 2024-04-24T22:59:13.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.517+0000 7fd464a93640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd4600d0dd0 con 0x7fd4600b24d0 2024-04-24T22:59:13.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.516+0000 7fe3add9b640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3b00b5860 0x7fe3b0152c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.516+0000 7fe3add9b640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3b00b5860 0x7fe3b0152c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.32:41988/0 (socket says 172.21.15.32:41988) 2024-04-24T22:59:13.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.517+0000 7fe3add9b640 1 -- 172.21.15.32:0/3621777815 learned_addr learned my addr 172.21.15.32:0/3621777815 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:13.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.517+0000 7fe3aed9d640 1 --2- 172.21.15.32:0/3621777815 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3b02010a0 0x7fe3b01518d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.517+0000 7fe3af59e640 1 -- 172.21.15.32:0/3621777815 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b00b3f60 con 0x7fe3b00b4760 2024-04-24T22:59:13.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.517+0000 7fe3af59e640 1 -- 172.21.15.32:0/3621777815 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b00b42d0 con 0x7fe3b02010a0 2024-04-24T22:59:13.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.517+0000 7fb73359e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb72c0df210 0x7fb72c0cec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.517+0000 7fe3af59e640 1 -- 172.21.15.32:0/3621777815 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe3b016b6c0 con 0x7fe3b00b5860 2024-04-24T22:59:13.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.517+0000 7f21497c1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f212c0051c0 0x7f2144156a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.517+0000 7fd45ed9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4600cca00 0x7fd460148b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.515+0000 7f796f59e640 1 Processor -- start 2024-04-24T22:59:13.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.515+0000 7f796f59e640 1 -- start start 2024-04-24T22:59:13.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.517+0000 7fb73359e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb72c0e2d70 0x7fb72c0ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.516+0000 7f796f59e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f797006dc00 0x7f7970201b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.516+0000 7f796f59e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f797014c340 0x7f797014a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.516+0000 7f796f59e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7970162e50 0x7f79700b4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.516+0000 7f796f59e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f79700cd1c0 con 0x7f797014c340 2024-04-24T22:59:13.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.516+0000 7f796f59e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f79700cd040 con 0x7f797006dc00 2024-04-24T22:59:13.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.516+0000 7f796f59e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f797006e090 con 0x7f7970162e50 2024-04-24T22:59:13.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.517+0000 7f796dd9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f797014c340 0x7f797014a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.517+0000 7fd45ed9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4600cca00 0x7fd460148b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38484/0 (socket says 172.21.15.32:38484) 2024-04-24T22:59:13.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.517+0000 7f796dd9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f797014c340 0x7f797014a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38474/0 (socket says 172.21.15.32:38474) 2024-04-24T22:59:13.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.517+0000 7f796dd9b640 1 -- 172.21.15.32:0/25392069 learned_addr learned my addr 172.21.15.32:0/25392069 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:13.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.517+0000 7fd45ed9d640 1 -- 172.21.15.32:0/3356141677 learned_addr learned my addr 172.21.15.32:0/3356141677 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:13.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.517+0000 7fb73359e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb72c0e31d0 0x7fb72c04ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.517+0000 7fb73259c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb72c0df210 0x7fb72c0cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.517+0000 7fb73259c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb72c0df210 0x7fb72c0cec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.32:48780/0 (socket says 172.21.15.32:48780) 2024-04-24T22:59:13.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.517+0000 7f21497c1640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f214415a7e0 0x7f2144159500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.517+0000 7f21497c1640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f214415ac60 0x7f2144159a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.517+0000 7f21497c1640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f214406c5c0 con 0x7f212c0051c0 2024-04-24T22:59:13.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.517+0000 7f21497c1640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f21440d1b40 con 0x7f214415ac60 2024-04-24T22:59:13.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.517+0000 7f21497c1640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f214406c740 con 0x7f214415a7e0 2024-04-24T22:59:13.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.517+0000 7fb73259c640 1 -- 172.21.15.32:0/3277234912 learned_addr learned my addr 172.21.15.32:0/3277234912 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:13.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.517+0000 7fb73359e640 1 -- 172.21.15.32:0/3277234912 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb72c0e3660 con 0x7fb72c0e31d0 2024-04-24T22:59:13.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.517+0000 7fb73359e640 1 -- 172.21.15.32:0/3277234912 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb72c0065b0 con 0x7fb72c0df210 2024-04-24T22:59:13.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.517+0000 7fb73359e640 1 -- 172.21.15.32:0/3277234912 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fb72c0df5f0 con 0x7fb72c0e2d70 2024-04-24T22:59:13.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.517+0000 7f21427fc640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f214415a7e0 0x7f2144159500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.517+0000 7f21427fc640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f214415a7e0 0x7f2144159500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.32:42012/0 (socket says 172.21.15.32:42012) 2024-04-24T22:59:13.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.517+0000 7f21427fc640 1 -- 172.21.15.32:0/3904672589 learned_addr learned my addr 172.21.15.32:0/3904672589 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:13.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.517+0000 7fdaaf7fe640 1 -- 172.21.15.32:0/2902318319 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac0015070 con 0x7fdac80cdd30 2024-04-24T22:59:13.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.517+0000 7fb732d9d640 1 --2- 172.21.15.32:0/3277234912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb72c0e31d0 0x7fb72c04ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.517+0000 7fd45e59c640 1 --2- 172.21.15.32:0/3356141677 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd46006c5f0 0x7fd460147ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.517+0000 7fe3add9b640 1 -- 172.21.15.32:0/3621777815 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3b02010a0 msgr2=0x7fe3b01518d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.517+0000 7fe3add9b640 1 --2- 172.21.15.32:0/3621777815 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3b02010a0 0x7fe3b01518d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.517+0000 7fe3add9b640 1 -- 172.21.15.32:0/3621777815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3b00b4760 msgr2=0x7fe3b0150170 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T22:59:13.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.517+0000 7fe3add9b640 1 --2- 172.21.15.32:0/3621777815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3b00b4760 0x7fe3b0150170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.517+0000 7fe3add9b640 1 -- 172.21.15.32:0/3621777815 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3b0150830 con 0x7fe3b00b5860 2024-04-24T22:59:13.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.518+0000 7fe3aed9d640 1 --2- 172.21.15.32:0/3621777815 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3b02010a0 0x7fe3b01518d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:13.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.517+0000 7fdaaf7fe640 1 -- 172.21.15.32:0/2902318319 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdac0007a30 con 0x7fdac80cdd30 2024-04-24T22:59:13.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.518+0000 7fb73259c640 1 -- 172.21.15.32:0/3277234912 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb72c0e2d70 msgr2=0x7fb72c0ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.518+0000 7fb73259c640 1 --2- 172.21.15.32:0/3277234912 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb72c0e2d70 0x7fb72c0ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.518+0000 7fb73259c640 1 -- 172.21.15.32:0/3277234912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb72c0e31d0 msgr2=0x7fb72c04ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.518+0000 7fb73259c640 1 --2- 172.21.15.32:0/3277234912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb72c0e31d0 0x7fb72c04ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.517+0000 7fe503fff640 1 -- 172.21.15.32:0/3610651407 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe4fc048100 msgr2=0x7fe4fc0e5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.517+0000 7fe503fff640 1 --2- 172.21.15.32:0/3610651407 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe4fc048100 0x7fe4fc0e5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.517+0000 7fe503fff640 1 -- 172.21.15.32:0/3610651407 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe4fc04f930 msgr2=0x7fe4fc17d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.517+0000 7fe503fff640 1 --2- 172.21.15.32:0/3610651407 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe4fc04f930 0x7fe4fc17d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.517+0000 7fe503fff640 1 -- 172.21.15.32:0/3610651407 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4fc17dd00 con 0x7fe4fc053e50 2024-04-24T22:59:13.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.518+0000 7fb73259c640 1 -- 172.21.15.32:0/3277234912 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb72c04d2b0 con 0x7fb72c0df210 2024-04-24T22:59:13.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.518+0000 7f21427fc640 1 -- 172.21.15.32:0/3904672589 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f214415ac60 msgr2=0x7f2144159a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.518+0000 7f21427fc640 1 --2- 172.21.15.32:0/3904672589 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f214415ac60 0x7f2144159a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.518+0000 7f21427fc640 1 -- 172.21.15.32:0/3904672589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f212c0051c0 msgr2=0x7f2144156a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.518+0000 7f21427fc640 1 --2- 172.21.15.32:0/3904672589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f212c0051c0 0x7f2144156a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.518+0000 7f21427fc640 1 -- 172.21.15.32:0/3904672589 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2144159f80 con 0x7f214415a7e0 2024-04-24T22:59:13.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.518+0000 7fe502ffd640 1 --2- 172.21.15.32:0/3610651407 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe4fc04f930 0x7fe4fc17d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:13.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.518+0000 7fb73259c640 1 --2- 172.21.15.32:0/3277234912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb72c0df210 0x7fb72c0cec60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb72400c460 tx=0x7fb72400c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.518+0000 7f21427fc640 1 --2- 172.21.15.32:0/3904672589 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f214415a7e0 0x7f2144159500 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f213400e7d0 tx=0x7f213400ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.517+0000 7f796dd9b640 1 -- 172.21.15.32:0/25392069 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7970162e50 msgr2=0x7f79700b4450 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-24T22:59:13.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.517+0000 7f796dd9b640 1 --2- 172.21.15.32:0/25392069 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7970162e50 0x7f79700b4450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.517+0000 7f796dd9b640 1 -- 172.21.15.32:0/25392069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f797006dc00 msgr2=0x7f7970201b60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-24T22:59:13.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.517+0000 7f796dd9b640 1 --2- 172.21.15.32:0/25392069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f797006dc00 0x7f7970201b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.517+0000 7f796dd9b640 1 -- 172.21.15.32:0/25392069 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f797014abc0 con 0x7f797014c340 2024-04-24T22:59:13.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.518+0000 7f796dd9b640 1 --2- 172.21.15.32:0/25392069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f797014c340 0x7f797014a680 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7968009f40 tx=0x7f7968004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.518+0000 7fe3add9b640 1 --2- 172.21.15.32:0/3621777815 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3b00b5860 0x7fe3b0152c30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe3a400c4b0 tx=0x7fe3a400c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.519+0000 7fb722ffd640 1 -- 172.21.15.32:0/3277234912 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb724016020 con 0x7fb72c0df210 2024-04-24T22:59:13.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.519+0000 7fb722ffd640 1 -- 172.21.15.32:0/3277234912 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb724012870 con 0x7fb72c0df210 2024-04-24T22:59:13.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.519+0000 7fb722ffd640 1 -- 172.21.15.32:0/3277234912 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb724012c60 con 0x7fb72c0df210 2024-04-24T22:59:13.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.518+0000 7fdacd3fb640 1 -- 172.21.15.32:0/2902318319 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdac81fc300 con 0x7fdac80cdd30 2024-04-24T22:59:13.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.518+0000 7fdacd3fb640 1 -- 172.21.15.32:0/2902318319 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdac81fc8c0 con 0x7fdac80cdd30 2024-04-24T22:59:13.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.518+0000 7fdaaf7fe640 1 -- 172.21.15.32:0/2902318319 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac0007de0 con 0x7fdac80cdd30 2024-04-24T22:59:13.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.519+0000 7fdacd3fb640 1 -- 172.21.15.32:0/2902318319 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdac81fa030 con 0x7fdac80cdd30 2024-04-24T22:59:13.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.518+0000 7f597e6b3640 1 -- 172.21.15.32:0/2132117252 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f59780b4fa0 con 0x7f59780cd6c0 2024-04-24T22:59:13.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.519+0000 7f5974ff9640 1 -- 172.21.15.32:0/2132117252 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f597001a070 con 0x7f59780cd6c0 2024-04-24T22:59:13.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.519+0000 7f5974ff9640 1 --2- 172.21.15.32:0/2132117252 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f596406bf00 0x7f596406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.519+0000 7f212b7fe640 1 -- 172.21.15.32:0/3904672589 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2134017070 con 0x7f214415a7e0 2024-04-24T22:59:13.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.519+0000 7f212b7fe640 1 -- 172.21.15.32:0/3904672589 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f213400c940 con 0x7f214415a7e0 2024-04-24T22:59:13.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.519+0000 7f212b7fe640 1 -- 172.21.15.32:0/3904672589 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f213400ccf0 con 0x7f214415a7e0 2024-04-24T22:59:13.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.519+0000 7f21497c1640 1 -- 172.21.15.32:0/3904672589 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21441f3100 con 0x7f214415a7e0 2024-04-24T22:59:13.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.519+0000 7f21497c1640 1 -- 172.21.15.32:0/3904672589 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f21441f36c0 con 0x7f214415a7e0 2024-04-24T22:59:13.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.518+0000 7fd45ed9d640 1 -- 172.21.15.32:0/3356141677 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd4600b24d0 msgr2=0x7fd460070d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.518+0000 7fd45ed9d640 1 --2- 172.21.15.32:0/3356141677 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd4600b24d0 0x7fd460070d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.518+0000 7fd45ed9d640 1 -- 172.21.15.32:0/3356141677 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd46006c5f0 msgr2=0x7fd460147ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.518+0000 7fd45ed9d640 1 --2- 172.21.15.32:0/3356141677 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd46006c5f0 0x7fd460147ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.518+0000 7fd45ed9d640 1 -- 172.21.15.32:0/3356141677 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd460071b80 con 0x7fd4600cca00 2024-04-24T22:59:13.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.518+0000 7fd45e59c640 1 --2- 172.21.15.32:0/3356141677 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd46006c5f0 0x7fd460147ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:13.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.518+0000 7fe39effd640 1 -- 172.21.15.32:0/3621777815 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a4017070 con 0x7fe3b00b5860 2024-04-24T22:59:13.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.518+0000 7fe3af59e640 1 -- 172.21.15.32:0/3621777815 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3b0150b10 con 0x7fe3b00b5860 2024-04-24T22:59:13.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.519+0000 7fe3af59e640 1 -- 172.21.15.32:0/3621777815 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3b01510d0 con 0x7fe3b00b5860 2024-04-24T22:59:13.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.519+0000 7fe39effd640 1 -- 172.21.15.32:0/3621777815 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3a4005ce0 con 0x7fe3b00b5860 2024-04-24T22:59:13.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.518+0000 7f795effd640 1 -- 172.21.15.32:0/25392069 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7968003b90 con 0x7f797014c340 2024-04-24T22:59:13.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.518+0000 7f795effd640 1 -- 172.21.15.32:0/25392069 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7968003d30 con 0x7f797014c340 2024-04-24T22:59:13.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.519+0000 7fb73359e640 1 -- 172.21.15.32:0/3277234912 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb72c04d590 con 0x7fb72c0df210 2024-04-24T22:59:13.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.520+0000 7fb73359e640 1 -- 172.21.15.32:0/3277234912 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb72c0ce7e0 con 0x7fb72c0df210 2024-04-24T22:59:13.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.519+0000 7f796f59e640 1 -- 172.21.15.32:0/25392069 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79700b3690 con 0x7f797014c340 2024-04-24T22:59:13.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.519+0000 7f796f59e640 1 -- 172.21.15.32:0/25392069 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79700b3c50 con 0x7f797014c340 2024-04-24T22:59:13.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.520+0000 7f795effd640 1 -- 172.21.15.32:0/25392069 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796803d8a0 con 0x7f797014c340 2024-04-24T22:59:13.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.520+0000 7f5974ff9640 1 -- 172.21.15.32:0/2132117252 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f597008ad30 con 0x7f59780cd6c0 2024-04-24T22:59:13.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.520+0000 7fe39effd640 1 -- 172.21.15.32:0/3621777815 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a4007270 con 0x7fe3b00b5860 2024-04-24T22:59:13.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.520+0000 7f5974ff9640 1 --2- 172.21.15.32:0/2132117252 >> [v2:172.21.15.32:6816/3405899334,v1:172.21.15.32:6817/3405899334] conn(0x7f59640740a0 0x7f59640764c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.520+0000 7f5974ff9640 1 -- 172.21.15.32:0/2132117252 --> [v2:172.21.15.32:6816/3405899334,v1:172.21.15.32:6817/3405899334] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5970010e50 con 0x7f59640740a0 2024-04-24T22:59:13.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.520+0000 7f5974ff9640 1 -- 172.21.15.32:0/2132117252 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5970055310 con 0x7f59780cd6c0 2024-04-24T22:59:13.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.520+0000 7fe503fff640 1 --2- 172.21.15.32:0/3610651407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4fc053e50 0x7fe4fc038510 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe504050ce0 tx=0x7fe5040694b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.520+0000 7f5977fff640 1 --2- 172.21.15.32:0/2132117252 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f596406bf00 0x7f596406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.520+0000 7f59777fe640 1 --2- 172.21.15.32:0/2132117252 >> [v2:172.21.15.32:6816/3405899334,v1:172.21.15.32:6817/3405899334] conn(0x7f59640740a0 0x7f59640764c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.521+0000 7f59777fe640 1 --2- 172.21.15.32:0/2132117252 >> [v2:172.21.15.32:6816/3405899334,v1:172.21.15.32:6817/3405899334] conn(0x7f59640740a0 0x7f59640764c0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.521+0000 7f796f59e640 1 -- 172.21.15.32:0/25392069 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f797014ae90 con 0x7f797014c340 2024-04-24T22:59:13.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.521+0000 7f5977fff640 1 --2- 172.21.15.32:0/2132117252 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f596406bf00 0x7f596406e3c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f596c006fd0 tx=0x7f596c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.521+0000 7fdaaf7fe640 1 -- 172.21.15.32:0/2902318319 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdac0018070 con 0x7fdac80cdd30 2024-04-24T22:59:13.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.521+0000 7f212b7fe640 1 -- 172.21.15.32:0/3904672589 <== mon.2 v2:172.21.15.121:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f213401a070 con 0x7f214415a7e0 2024-04-24T22:59:13.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.522+0000 7f5974ff9640 1 -- 172.21.15.32:0/2132117252 <== osd.2 v2:172.21.15.32:6816/3405899334 1 ==== command_reply(tid 1: 0 ) ==== 8+0+25033 (crc 0 0 0) 0x7f5970010e50 con 0x7f59640740a0 2024-04-24T22:59:13.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.522+0000 7fb722ffd640 1 -- 172.21.15.32:0/3277234912 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb724019020 con 0x7fb72c0df210 2024-04-24T22:59:13.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.522+0000 7fdaaf7fe640 1 --2- 172.21.15.32:0/2902318319 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdab406bf00 0x7fdab406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.522+0000 7f212b7fe640 1 --2- 172.21.15.32:0/3904672589 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f213006bf00 0x7f213006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.522+0000 7fdaaf7fe640 1 -- 172.21.15.32:0/2902318319 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fdac008b030 con 0x7fdac80cdd30 2024-04-24T22:59:13.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.522+0000 7fd45ed9d640 1 --2- 172.21.15.32:0/3356141677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4600cca00 0x7fd460148b40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd45000d600 tx=0x7fd45000dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.522+0000 7f212b7fe640 1 -- 172.21.15.32:0/3904672589 <== mon.2 v2:172.21.15.121:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f213408acb0 con 0x7f214415a7e0 2024-04-24T22:59:13.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.522+0000 7fdaaf7fe640 1 --2- 172.21.15.32:0/2902318319 >> [v2:172.21.15.32:6808/1340633718,v1:172.21.15.32:6809/1340633718] conn(0x7fdab4074120 0x7fdab4076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.522+0000 7fdaaf7fe640 1 -- 172.21.15.32:0/2902318319 --> [v2:172.21.15.32:6808/1340633718,v1:172.21.15.32:6809/1340633718] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdac003aec0 con 0x7fdab4074120 2024-04-24T22:59:13.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.522+0000 7fdaaf7fe640 1 -- 172.21.15.32:0/2902318319 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fdac008b420 con 0x7fdac80cdd30 2024-04-24T22:59:13.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.522+0000 7f2142ffd640 1 --2- 172.21.15.32:0/3904672589 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f213006bf00 0x7f213006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.522+0000 7fb73359e640 1 -- 172.21.15.32:0/3277234912 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb72c0057d0 con 0x7fb72c0df210 2024-04-24T22:59:13.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.521+0000 7fe39effd640 1 -- 172.21.15.32:0/3621777815 <== mon.2 v2:172.21.15.121:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe3a401a070 con 0x7fe3b00b5860 2024-04-24T22:59:13.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.521+0000 7fe3af59e640 1 -- 172.21.15.32:0/3621777815 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe394001030 con 0x7fe3b00b5860 2024-04-24T22:59:13.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.522+0000 7fe39effd640 1 --2- 172.21.15.32:0/3621777815 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe39006be80 0x7fe39006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.522+0000 7fe39effd640 1 -- 172.21.15.32:0/3621777815 <== mon.2 v2:172.21.15.121:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe3a4089f50 con 0x7fe3b00b5860 2024-04-24T22:59:13.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.522+0000 7fe39effd640 1 --2- 172.21.15.32:0/3621777815 >> [v2:172.21.15.121:6808/3794084091,v1:172.21.15.121:6809/3794084091] conn(0x7fe390074100 0x7fe390076520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.522+0000 7fe39effd640 1 -- 172.21.15.32:0/3621777815 --> [v2:172.21.15.121:6808/3794084091,v1:172.21.15.121:6809/3794084091] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe3a4015e70 con 0x7fe390074100 2024-04-24T22:59:13.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.522+0000 7fe39effd640 1 -- 172.21.15.32:0/3621777815 <== mon.2 v2:172.21.15.121:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe3a4091050 con 0x7fe3b00b5860 2024-04-24T22:59:13.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.522+0000 7fdac77fe640 1 --2- 172.21.15.32:0/2902318319 >> [v2:172.21.15.32:6808/1340633718,v1:172.21.15.32:6809/1340633718] conn(0x7fdab4074120 0x7fdab4076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.523+0000 7fe3aed9d640 1 --2- 172.21.15.32:0/3621777815 >> [v2:172.21.15.121:6808/3794084091,v1:172.21.15.121:6809/3794084091] conn(0x7fe390074100 0x7fe390076520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.523+0000 7fe3ae59c640 1 --2- 172.21.15.32:0/3621777815 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe39006be80 0x7fe39006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.523+0000 7fb722ffd640 1 --2- 172.21.15.32:0/3277234912 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb71406bf00 0x7fb71406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.523+0000 7f2142ffd640 1 --2- 172.21.15.32:0/3904672589 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f213006bf00 0x7f213006e3c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2138001050 tx=0x7f2138000bc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.523+0000 7fdac67fc640 1 --2- 172.21.15.32:0/2902318319 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdab406bf00 0x7fdab406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.523+0000 7f795effd640 1 -- 172.21.15.32:0/25392069 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f796803da40 con 0x7f797014c340 2024-04-24T22:59:13.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.523+0000 7fe4effff640 1 -- 172.21.15.32:0/3610651407 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe504090070 con 0x7fe4fc053e50 2024-04-24T22:59:13.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.523+0000 7fe4effff640 1 -- 172.21.15.32:0/3610651407 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe504073310 con 0x7fe4fc053e50 2024-04-24T22:59:13.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.523+0000 7fe4effff640 1 -- 172.21.15.32:0/3610651407 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe504073780 con 0x7fe4fc053e50 2024-04-24T22:59:13.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.523+0000 7fdac77fe640 1 --2- 172.21.15.32:0/2902318319 >> [v2:172.21.15.32:6808/1340633718,v1:172.21.15.32:6809/1340633718] conn(0x7fdab4074120 0x7fdab4076540 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.523+0000 7f21497c1640 1 --2- 172.21.15.32:0/3904672589 >> [v2:172.21.15.32:6818/3381057190,v1:172.21.15.32:6821/3381057190] conn(0x7f21441f3890 0x7f21441efea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.523+0000 7f21497c1640 1 -- 172.21.15.32:0/3904672589 --> [v2:172.21.15.32:6818/3381057190,v1:172.21.15.32:6821/3381057190] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f214406d250 con 0x7f21441f3890 2024-04-24T22:59:13.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.523+0000 7f21437fe640 1 --2- 172.21.15.32:0/3904672589 >> [v2:172.21.15.32:6818/3381057190,v1:172.21.15.32:6821/3381057190] conn(0x7f21441f3890 0x7f21441efea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.523+0000 7fd44effd640 1 -- 172.21.15.32:0/3356141677 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4500125f0 con 0x7fd4600cca00 2024-04-24T22:59:13.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.523+0000 7fd44effd640 1 -- 172.21.15.32:0/3356141677 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd450019070 con 0x7fd4600cca00 2024-04-24T22:59:13.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.523+0000 7fd44effd640 1 -- 172.21.15.32:0/3356141677 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4500129d0 con 0x7fd4600cca00 2024-04-24T22:59:13.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.523+0000 7fd464a93640 1 -- 172.21.15.32:0/3356141677 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4601fa0c0 con 0x7fd4600cca00 2024-04-24T22:59:13.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.523+0000 7fe3aed9d640 1 --2- 172.21.15.32:0/3621777815 >> [v2:172.21.15.121:6808/3794084091,v1:172.21.15.121:6809/3794084091] conn(0x7fe390074100 0x7fe390076520 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.523+0000 7f795effd640 1 --2- 172.21.15.32:0/25392069 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f795006be80 0x7f795006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.523+0000 7f795effd640 1 -- 172.21.15.32:0/25392069 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f79680acc20 con 0x7f797014c340 2024-04-24T22:59:13.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.523+0000 7f795effd640 1 --2- 172.21.15.32:0/25392069 >> [v2:172.21.15.121:6823/1348640258,v1:172.21.15.121:6825/1348640258] conn(0x7f79500740a0 0x7f79500764c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.523+0000 7f795effd640 1 -- 172.21.15.32:0/25392069 --> [v2:172.21.15.121:6823/1348640258,v1:172.21.15.121:6825/1348640258] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f79680020b0 con 0x7f79500740a0 2024-04-24T22:59:13.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.523+0000 7f795effd640 1 -- 172.21.15.32:0/25392069 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f79680ad100 con 0x7f797014c340 2024-04-24T22:59:13.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.524+0000 7f796e59c640 1 --2- 172.21.15.32:0/25392069 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f795006be80 0x7f795006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.524+0000 7f796ed9d640 1 --2- 172.21.15.32:0/25392069 >> [v2:172.21.15.121:6823/1348640258,v1:172.21.15.121:6825/1348640258] conn(0x7f79500740a0 0x7f79500764c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.523+0000 7fe509dc1640 1 -- 172.21.15.32:0/3610651407 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4fc17dfe0 con 0x7fe4fc053e50 2024-04-24T22:59:13.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.523+0000 7fe509dc1640 1 -- 172.21.15.32:0/3610651407 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4fc04b2b0 con 0x7fe4fc053e50 2024-04-24T22:59:13.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.523+0000 7fdac67fc640 1 --2- 172.21.15.32:0/2902318319 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdab406bf00 0x7fdab406e3c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdac815c850 tx=0x7fdab8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.524+0000 7f21437fe640 1 --2- 172.21.15.32:0/3904672589 >> [v2:172.21.15.32:6818/3381057190,v1:172.21.15.32:6821/3381057190] conn(0x7f21441f3890 0x7f21441efea0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.524+0000 7fdaaf7fe640 1 -- 172.21.15.32:0/2902318319 <== osd.1 v2:172.21.15.32:6808/1340633718 1 ==== command_reply(tid 1: 0 ) ==== 8+0+25033 (crc 0 0 0) 0x7fdac003aec0 con 0x7fdab4074120 2024-04-24T22:59:13.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.525+0000 7f796e59c640 1 --2- 172.21.15.32:0/25392069 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f795006be80 0x7f795006e340 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f796000a8b0 tx=0x7f7960008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.525+0000 7fe39effd640 1 -- 172.21.15.32:0/3621777815 <== osd.5 v2:172.21.15.121:6808/3794084091 1 ==== command_reply(tid 1: 0 ) ==== 8+0+25033 (crc 0 0 0) 0x7fe3a4015e70 con 0x7fe390074100 2024-04-24T22:59:13.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.524+0000 7fd464a93640 1 -- 172.21.15.32:0/3356141677 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4601fa680 con 0x7fd4600cca00 2024-04-24T22:59:13.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.525+0000 7f796ed9d640 1 --2- 172.21.15.32:0/25392069 >> [v2:172.21.15.121:6823/1348640258,v1:172.21.15.121:6825/1348640258] conn(0x7f79500740a0 0x7f79500764c0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.523+0000 7fb722ffd640 1 -- 172.21.15.32:0/3277234912 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb72408ad90 con 0x7fb72c0df210 2024-04-24T22:59:13.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.523+0000 7fb731d9b640 1 --2- 172.21.15.32:0/3277234912 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb71406bf00 0x7fb71406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.524+0000 7fb722ffd640 1 --2- 172.21.15.32:0/3277234912 >> [v2:172.21.15.32:6800/1479395426,v1:172.21.15.32:6801/1479395426] conn(0x7fb714074100 0x7fb714076520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.524+0000 7fb731d9b640 1 --2- 172.21.15.32:0/3277234912 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb71406bf00 0x7fb71406e3c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb72c04cbd0 tx=0x7fb728006c40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.525+0000 7fb732d9d640 1 --2- 172.21.15.32:0/3277234912 >> [v2:172.21.15.32:6800/1479395426,v1:172.21.15.32:6801/1479395426] conn(0x7fb714074100 0x7fb714076520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.525+0000 7f212b7fe640 1 -- 172.21.15.32:0/3904672589 <== osd.3 v2:172.21.15.32:6818/3381057190 1 ==== command_reply(tid 1: 0 ) ==== 8+0+25033 (crc 0 0 0) 0x7f214406d250 con 0x7f21441f3890 2024-04-24T22:59:13.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.526+0000 7fd44effd640 1 -- 172.21.15.32:0/3356141677 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd450012b70 con 0x7fd4600cca00 2024-04-24T22:59:13.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.526+0000 7fd464a93640 1 -- 172.21.15.32:0/3356141677 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd4601480e0 con 0x7fd4600cca00 2024-04-24T22:59:13.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.526+0000 7fe3ae59c640 1 --2- 172.21.15.32:0/3621777815 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe39006be80 0x7fe39006e340 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe3a803a000 tx=0x7fe3a8033000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.525+0000 7f795effd640 1 -- 172.21.15.32:0/25392069 <== osd.6 v2:172.21.15.121:6823/1348640258 1 ==== command_reply(tid 1: 0 ) ==== 8+0+25033 (crc 0 0 0) 0x7f79680020b0 con 0x7f79500740a0 2024-04-24T22:59:13.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.526+0000 7fe509dc1640 1 -- 172.21.15.32:0/3610651407 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe4fc17e220 con 0x7fe4fc053e50 2024-04-24T22:59:13.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.526+0000 7fb722ffd640 1 -- 172.21.15.32:0/3277234912 --> [v2:172.21.15.32:6800/1479395426,v1:172.21.15.32:6801/1479395426] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb724021ec0 con 0x7fb714074100 2024-04-24T22:59:13.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.527+0000 7fd44effd640 1 --2- 172.21.15.32:0/3356141677 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd44006be30 0x7fd44006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.527+0000 7fd44effd640 1 -- 172.21.15.32:0/3356141677 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd45008b350 con 0x7fd4600cca00 2024-04-24T22:59:13.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.527+0000 7fb722ffd640 1 -- 172.21.15.32:0/3277234912 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fb724055180 con 0x7fb72c0df210 2024-04-24T22:59:13.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.527+0000 7fd45e59c640 1 --2- 172.21.15.32:0/3356141677 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd44006be30 0x7fd44006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.527+0000 7fe4effff640 1 -- 172.21.15.32:0/3610651407 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe504093070 con 0x7fe4fc053e50 2024-04-24T22:59:13.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.527+0000 7fd44effd640 1 --2- 172.21.15.32:0/3356141677 >> [v2:172.21.15.121:6800/4010553979,v1:172.21.15.121:6801/4010553979] conn(0x7fd440074030 0x7fd440076450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.527+0000 7fd44effd640 1 -- 172.21.15.32:0/3356141677 --> [v2:172.21.15.121:6800/4010553979,v1:172.21.15.121:6801/4010553979] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd440076b00 con 0x7fd440074030 2024-04-24T22:59:13.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.527+0000 7fd44effd640 1 -- 172.21.15.32:0/3356141677 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fd450092050 con 0x7fd4600cca00 2024-04-24T22:59:13.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.528+0000 7fb732d9d640 1 --2- 172.21.15.32:0/3277234912 >> [v2:172.21.15.32:6800/1479395426,v1:172.21.15.32:6801/1479395426] conn(0x7fb714074100 0x7fb714076520 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.527+0000 7fd45dd9b640 1 --2- 172.21.15.32:0/3356141677 >> [v2:172.21.15.121:6800/4010553979,v1:172.21.15.121:6801/4010553979] conn(0x7fd440074030 0x7fd440076450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.528+0000 7fe4effff640 1 --2- 172.21.15.32:0/3610651407 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe4e806bf00 0x7fe4e806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.528+0000 7fe4effff640 1 -- 172.21.15.32:0/3610651407 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe5041069f0 con 0x7fe4fc053e50 2024-04-24T22:59:13.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.528+0000 7fe5037fe640 1 --2- 172.21.15.32:0/3610651407 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe4e806bf00 0x7fe4e806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.528+0000 7fe4effff640 1 --2- 172.21.15.32:0/3610651407 >> [v2:172.21.15.121:6816/3250005122,v1:172.21.15.121:6817/3250005122] conn(0x7fe4e8074180 0x7fe4e80765a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.528+0000 7fe4effff640 1 -- 172.21.15.32:0/3610651407 --> [v2:172.21.15.121:6816/3250005122,v1:172.21.15.121:6817/3250005122] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe4e8076c50 con 0x7fe4e8074180 2024-04-24T22:59:13.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.528+0000 7fe4effff640 1 -- 172.21.15.32:0/3610651407 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe50410b2a0 con 0x7fe4fc053e50 2024-04-24T22:59:13.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.528+0000 7fd45dd9b640 1 --2- 172.21.15.32:0/3356141677 >> [v2:172.21.15.121:6800/4010553979,v1:172.21.15.121:6801/4010553979] conn(0x7fd440074030 0x7fd440076450 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.528+0000 7fe502ffd640 1 --2- 172.21.15.32:0/3610651407 >> [v2:172.21.15.121:6816/3250005122,v1:172.21.15.121:6817/3250005122] conn(0x7fe4e8074180 0x7fe4e80765a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.528+0000 7fd45e59c640 1 --2- 172.21.15.32:0/3356141677 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd44006be30 0x7fd44006e2f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd458034000 tx=0x7fd45802f000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.529+0000 7fe5037fe640 1 --2- 172.21.15.32:0/3610651407 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe4e806bf00 0x7fe4e806e3c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc17e420 tx=0x7fe4f8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.529+0000 7fe502ffd640 1 --2- 172.21.15.32:0/3610651407 >> [v2:172.21.15.121:6816/3250005122,v1:172.21.15.121:6817/3250005122] conn(0x7fe4e8074180 0x7fe4e80765a0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.529+0000 7fd44effd640 1 -- 172.21.15.32:0/3356141677 <== osd.4 v2:172.21.15.121:6800/4010553979 1 ==== command_reply(tid 1: 0 ) ==== 8+0+25033 (crc 0 0 0) 0x7fd440076b00 con 0x7fd440074030 2024-04-24T22:59:13.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.529+0000 7fb722ffd640 1 -- 172.21.15.32:0/3277234912 <== osd.0 v2:172.21.15.32:6800/1479395426 1 ==== command_reply(tid 1: 0 ) ==== 8+0+25033 (crc 0 0 0) 0x7fb724021ec0 con 0x7fb714074100 2024-04-24T22:59:13.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.529+0000 7fe4effff640 1 -- 172.21.15.32:0/3610651407 <== osd.7 v2:172.21.15.121:6816/3250005122 1 ==== command_reply(tid 1: 0 ) ==== 8+0+25033 (crc 0 0 0) 0x7fe4e8076c50 con 0x7fe4e8074180 2024-04-24T22:59:13.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.552+0000 7f597e6b3640 1 -- 172.21.15.32:0/2132117252 --> [v2:172.21.15.32:6816/3405899334,v1:172.21.15.32:6817/3405899334] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f59781547f0 con 0x7f59640740a0 2024-04-24T22:59:13.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.552+0000 7f5974ff9640 1 -- 172.21.15.32:0/2132117252 <== osd.2 v2:172.21.15.32:6816/3405899334 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f59781547f0 con 0x7f59640740a0 2024-04-24T22:59:13.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.553+0000 7f595e7fc640 1 -- 172.21.15.32:0/2132117252 >> [v2:172.21.15.32:6816/3405899334,v1:172.21.15.32:6817/3405899334] conn(0x7f59640740a0 msgr2=0x7f59640764c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.553+0000 7f595e7fc640 1 --2- 172.21.15.32:0/2132117252 >> [v2:172.21.15.32:6816/3405899334,v1:172.21.15.32:6817/3405899334] conn(0x7f59640740a0 0x7f59640764c0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.553+0000 7f595e7fc640 1 -- 172.21.15.32:0/2132117252 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f596406bf00 msgr2=0x7f596406e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.553+0000 7f595e7fc640 1 --2- 172.21.15.32:0/2132117252 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f596406bf00 0x7f596406e3c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f596c006fd0 tx=0x7f596c008040 comp rx=0 tx=0).stop 2024-04-24T22:59:13.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.553+0000 7f595e7fc640 1 -- 172.21.15.32:0/2132117252 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59780cd6c0 msgr2=0x7f59781639a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.554+0000 7f595e7fc640 1 --2- 172.21.15.32:0/2132117252 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59780cd6c0 0x7f59781639a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5970001700 tx=0x7f597000a430 comp rx=0 tx=0).stop 2024-04-24T22:59:13.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.554+0000 7f597cc50640 1 -- 172.21.15.32:0/2132117252 reap_dead start 2024-04-24T22:59:13.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.554+0000 7f595e7fc640 1 -- 172.21.15.32:0/2132117252 shutdown_connections 2024-04-24T22:59:13.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.554+0000 7f595e7fc640 1 -- 172.21.15.32:0/2132117252 >> 172.21.15.32:0/2132117252 conn(0x7f59780743f0 msgr2=0x7f597816bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:13.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.554+0000 7f595e7fc640 1 -- 172.21.15.32:0/2132117252 shutdown_connections 2024-04-24T22:59:13.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.554+0000 7fdacd3fb640 1 -- 172.21.15.32:0/2902318319 --> [v2:172.21.15.32:6808/1340633718,v1:172.21.15.32:6809/1340633718] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdac8068430 con 0x7fdab4074120 2024-04-24T22:59:13.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.554+0000 7f595e7fc640 1 -- 172.21.15.32:0/2132117252 wait complete. 2024-04-24T22:59:13.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.554+0000 7f796f59e640 1 -- 172.21.15.32:0/25392069 --> [v2:172.21.15.121:6823/1348640258,v1:172.21.15.121:6825/1348640258] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f797014ae90 con 0x7f79500740a0 2024-04-24T22:59:13.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.555+0000 7fdaaf7fe640 1 -- 172.21.15.32:0/2902318319 <== osd.1 v2:172.21.15.32:6808/1340633718 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fdac8068430 con 0x7fdab4074120 2024-04-24T22:59:13.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.555+0000 7fdacd3fb640 1 -- 172.21.15.32:0/2902318319 >> [v2:172.21.15.32:6808/1340633718,v1:172.21.15.32:6809/1340633718] conn(0x7fdab4074120 msgr2=0x7fdab4076540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.555+0000 7fdacd3fb640 1 --2- 172.21.15.32:0/2902318319 >> [v2:172.21.15.32:6808/1340633718,v1:172.21.15.32:6809/1340633718] conn(0x7fdab4074120 0x7fdab4076540 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.555+0000 7fdacd3fb640 1 -- 172.21.15.32:0/2902318319 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdab406bf00 msgr2=0x7fdab406e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.555+0000 7fdacd3fb640 1 --2- 172.21.15.32:0/2902318319 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdab406bf00 0x7fdab406e3c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdac815c850 tx=0x7fdab8009040 comp rx=0 tx=0).stop 2024-04-24T22:59:13.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.555+0000 7fdacd3fb640 1 -- 172.21.15.32:0/2902318319 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdac80cdd30 msgr2=0x7fdac81f9af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.556+0000 7fdacd3fb640 1 --2- 172.21.15.32:0/2902318319 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdac80cdd30 0x7fdac81f9af0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdac000c7f0 tx=0x7fdac000cbf0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.555+0000 7f795effd640 1 -- 172.21.15.32:0/25392069 <== osd.6 v2:172.21.15.121:6823/1348640258 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7964001bb0 con 0x7f79500740a0 2024-04-24T22:59:13.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.556+0000 7fdac77fe640 1 -- 172.21.15.32:0/2902318319 reap_dead start 2024-04-24T22:59:13.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.556+0000 7f21497c1640 1 -- 172.21.15.32:0/3904672589 --> [v2:172.21.15.32:6818/3381057190,v1:172.21.15.32:6821/3381057190] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f21440b69d0 con 0x7f21441f3890 2024-04-24T22:59:13.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.556+0000 7fdacd3fb640 1 -- 172.21.15.32:0/2902318319 shutdown_connections 2024-04-24T22:59:13.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.556+0000 7fdacd3fb640 1 -- 172.21.15.32:0/2902318319 >> 172.21.15.32:0/2902318319 conn(0x7fdac80b2400 msgr2=0x7fdac80afee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:13.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.556+0000 7f212b7fe640 1 -- 172.21.15.32:0/3904672589 <== osd.3 v2:172.21.15.32:6818/3381057190 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f21440b69d0 con 0x7f21441f3890 2024-04-24T22:59:13.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.556+0000 7fe3af59e640 1 -- 172.21.15.32:0/3621777815 --> [v2:172.21.15.121:6808/3794084091,v1:172.21.15.121:6809/3794084091] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe394002d50 con 0x7fe390074100 2024-04-24T22:59:13.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.556+0000 7fdacd3fb640 1 -- 172.21.15.32:0/2902318319 shutdown_connections 2024-04-24T22:59:13.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.557+0000 7f21497c1640 1 -- 172.21.15.32:0/3904672589 >> [v2:172.21.15.32:6818/3381057190,v1:172.21.15.32:6821/3381057190] conn(0x7f21441f3890 msgr2=0x7f21441efea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.557+0000 7fdacd3fb640 1 -- 172.21.15.32:0/2902318319 wait complete. 2024-04-24T22:59:13.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.557+0000 7fe39effd640 1 -- 172.21.15.32:0/3621777815 <== osd.5 v2:172.21.15.121:6808/3794084091 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe394002d50 con 0x7fe390074100 2024-04-24T22:59:13.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.557+0000 7f21497c1640 1 --2- 172.21.15.32:0/3904672589 >> [v2:172.21.15.32:6818/3381057190,v1:172.21.15.32:6821/3381057190] conn(0x7f21441f3890 0x7f21441efea0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.557+0000 7f21497c1640 1 -- 172.21.15.32:0/3904672589 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f213006bf00 msgr2=0x7f213006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.557+0000 7f21497c1640 1 --2- 172.21.15.32:0/3904672589 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f213006bf00 0x7f213006e3c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2138001050 tx=0x7f2138000bc0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.557+0000 7f21497c1640 1 -- 172.21.15.32:0/3904672589 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f214415a7e0 msgr2=0x7f2144159500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.557+0000 7f21497c1640 1 --2- 172.21.15.32:0/3904672589 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f214415a7e0 0x7f2144159500 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f213400e7d0 tx=0x7f213400ec90 comp rx=0 tx=0).stop 2024-04-24T22:59:13.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.557+0000 7f21437fe640 1 -- 172.21.15.32:0/3904672589 reap_dead start 2024-04-24T22:59:13.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.557+0000 7f21497c1640 1 -- 172.21.15.32:0/3904672589 shutdown_connections 2024-04-24T22:59:13.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.557+0000 7f21497c1640 1 -- 172.21.15.32:0/3904672589 >> 172.21.15.32:0/3904672589 conn(0x7f21440b60c0 msgr2=0x7f214415d770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:13.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.557+0000 7f796f59e640 1 -- 172.21.15.32:0/25392069 >> [v2:172.21.15.121:6823/1348640258,v1:172.21.15.121:6825/1348640258] conn(0x7f79500740a0 msgr2=0x7f79500764c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.557+0000 7f796f59e640 1 --2- 172.21.15.32:0/25392069 >> [v2:172.21.15.121:6823/1348640258,v1:172.21.15.121:6825/1348640258] conn(0x7f79500740a0 0x7f79500764c0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.557+0000 7f796f59e640 1 -- 172.21.15.32:0/25392069 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f795006be80 msgr2=0x7f795006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.557+0000 7f796f59e640 1 --2- 172.21.15.32:0/25392069 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f795006be80 0x7f795006e340 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f796000a8b0 tx=0x7f7960008040 comp rx=0 tx=0).stop 2024-04-24T22:59:13.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.557+0000 7f796f59e640 1 -- 172.21.15.32:0/25392069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f797014c340 msgr2=0x7f797014a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.557+0000 7f796f59e640 1 --2- 172.21.15.32:0/25392069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f797014c340 0x7f797014a680 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7968009f40 tx=0x7f7968004a40 comp rx=0 tx=0).stop 2024-04-24T22:59:13.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.557+0000 7f796ed9d640 1 -- 172.21.15.32:0/25392069 reap_dead start 2024-04-24T22:59:13.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.557+0000 7f21497c1640 1 -- 172.21.15.32:0/3904672589 shutdown_connections 2024-04-24T22:59:13.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.557+0000 7f21497c1640 1 -- 172.21.15.32:0/3904672589 wait complete. 2024-04-24T22:59:13.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.557+0000 7f796f59e640 1 -- 172.21.15.32:0/25392069 shutdown_connections 2024-04-24T22:59:13.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.557+0000 7f796f59e640 1 -- 172.21.15.32:0/25392069 >> 172.21.15.32:0/25392069 conn(0x7f797006c5f0 msgr2=0x7f7970071a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:13.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.558+0000 7fe3af59e640 1 -- 172.21.15.32:0/3621777815 >> [v2:172.21.15.121:6808/3794084091,v1:172.21.15.121:6809/3794084091] conn(0x7fe390074100 msgr2=0x7fe390076520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.558+0000 7fe3af59e640 1 --2- 172.21.15.32:0/3621777815 >> [v2:172.21.15.121:6808/3794084091,v1:172.21.15.121:6809/3794084091] conn(0x7fe390074100 0x7fe390076520 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.558+0000 7fe509dc1640 1 -- 172.21.15.32:0/3610651407 --> [v2:172.21.15.121:6816/3250005122,v1:172.21.15.121:6817/3250005122] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe4fc0053c0 con 0x7fe4e8074180 2024-04-24T22:59:13.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.558+0000 7f796f59e640 1 -- 172.21.15.32:0/25392069 shutdown_connections 2024-04-24T22:59:13.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.558+0000 7f796f59e640 1 -- 172.21.15.32:0/25392069 wait complete. 2024-04-24T22:59:13.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.558+0000 7fe3af59e640 1 -- 172.21.15.32:0/3621777815 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe39006be80 msgr2=0x7fe39006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.558+0000 7fe3af59e640 1 --2- 172.21.15.32:0/3621777815 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe39006be80 0x7fe39006e340 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe3a803a000 tx=0x7fe3a8033000 comp rx=0 tx=0).stop 2024-04-24T22:59:13.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.558+0000 7fe3af59e640 1 -- 172.21.15.32:0/3621777815 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3b00b5860 msgr2=0x7fe3b0152c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.558+0000 7fe3af59e640 1 --2- 172.21.15.32:0/3621777815 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3b00b5860 0x7fe3b0152c30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe3a400c4b0 tx=0x7fe3a400c970 comp rx=0 tx=0).stop 2024-04-24T22:59:13.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.558+0000 7fe3aed9d640 1 -- 172.21.15.32:0/3621777815 reap_dead start 2024-04-24T22:59:13.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.558+0000 7fe3af59e640 1 -- 172.21.15.32:0/3621777815 shutdown_connections 2024-04-24T22:59:13.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.558+0000 7fe3af59e640 1 -- 172.21.15.32:0/3621777815 >> 172.21.15.32:0/3621777815 conn(0x7fe3b00b1f30 msgr2=0x7fe3b0073060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:13.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.558+0000 7fe4effff640 1 -- 172.21.15.32:0/3610651407 <== osd.7 v2:172.21.15.121:6816/3250005122 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe4fc0053c0 con 0x7fe4e8074180 2024-04-24T22:59:13.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.559+0000 7fb73359e640 1 -- 172.21.15.32:0/3277234912 --> [v2:172.21.15.32:6800/1479395426,v1:172.21.15.32:6801/1479395426] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb72c048700 con 0x7fb714074100 2024-04-24T22:59:13.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.559+0000 7fe509dc1640 1 -- 172.21.15.32:0/3610651407 >> [v2:172.21.15.121:6816/3250005122,v1:172.21.15.121:6817/3250005122] conn(0x7fe4e8074180 msgr2=0x7fe4e80765a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.559+0000 7fe509dc1640 1 --2- 172.21.15.32:0/3610651407 >> [v2:172.21.15.121:6816/3250005122,v1:172.21.15.121:6817/3250005122] conn(0x7fe4e8074180 0x7fe4e80765a0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.559+0000 7fb722ffd640 1 -- 172.21.15.32:0/3277234912 <== osd.0 v2:172.21.15.32:6800/1479395426 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb72c048700 con 0x7fb714074100 2024-04-24T22:59:13.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.559+0000 7fe3af59e640 1 -- 172.21.15.32:0/3621777815 shutdown_connections 2024-04-24T22:59:13.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.559+0000 7fe3af59e640 1 -- 172.21.15.32:0/3621777815 wait complete. 2024-04-24T22:59:13.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.559+0000 7fe509dc1640 1 -- 172.21.15.32:0/3610651407 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe4e806bf00 msgr2=0x7fe4e806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.559+0000 7fe509dc1640 1 --2- 172.21.15.32:0/3610651407 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe4e806bf00 0x7fe4e806e3c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc17e420 tx=0x7fe4f8009040 comp rx=0 tx=0).stop 2024-04-24T22:59:13.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.559+0000 7fe509dc1640 1 -- 172.21.15.32:0/3610651407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4fc053e50 msgr2=0x7fe4fc038510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.559+0000 7fe509dc1640 1 --2- 172.21.15.32:0/3610651407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4fc053e50 0x7fe4fc038510 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe504050ce0 tx=0x7fe5040694b0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.559+0000 7fd464a93640 1 -- 172.21.15.32:0/3356141677 --> [v2:172.21.15.121:6800/4010553979,v1:172.21.15.121:6801/4010553979] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd4600682b0 con 0x7fd440074030 2024-04-24T22:59:13.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.559+0000 7fe503fff640 1 -- 172.21.15.32:0/3610651407 reap_dead start 2024-04-24T22:59:13.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.560+0000 7fd44effd640 1 -- 172.21.15.32:0/3356141677 <== osd.4 v2:172.21.15.121:6800/4010553979 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd4600682b0 con 0x7fd440074030 2024-04-24T22:59:13.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.560+0000 7fe509dc1640 1 -- 172.21.15.32:0/3610651407 shutdown_connections 2024-04-24T22:59:13.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.560+0000 7fe509dc1640 1 -- 172.21.15.32:0/3610651407 >> 172.21.15.32:0/3610651407 conn(0x7fe4fc009fe0 msgr2=0x7fe4fc00a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:13.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.560+0000 7fb73359e640 1 -- 172.21.15.32:0/3277234912 >> [v2:172.21.15.32:6800/1479395426,v1:172.21.15.32:6801/1479395426] conn(0x7fb714074100 msgr2=0x7fb714076520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.560+0000 7fe509dc1640 1 -- 172.21.15.32:0/3610651407 shutdown_connections 2024-04-24T22:59:13.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.560+0000 7fe509dc1640 1 -- 172.21.15.32:0/3610651407 wait complete. 2024-04-24T22:59:13.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.560+0000 7fb73359e640 1 --2- 172.21.15.32:0/3277234912 >> [v2:172.21.15.32:6800/1479395426,v1:172.21.15.32:6801/1479395426] conn(0x7fb714074100 0x7fb714076520 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.561+0000 7fb73359e640 1 -- 172.21.15.32:0/3277234912 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb71406bf00 msgr2=0x7fb71406e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.561+0000 7fb73359e640 1 --2- 172.21.15.32:0/3277234912 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb71406bf00 0x7fb71406e3c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb72c04cbd0 tx=0x7fb728006c40 comp rx=0 tx=0).stop 2024-04-24T22:59:13.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.561+0000 7fb73359e640 1 -- 172.21.15.32:0/3277234912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb72c0df210 msgr2=0x7fb72c0cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.561+0000 7fb73359e640 1 --2- 172.21.15.32:0/3277234912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb72c0df210 0x7fb72c0cec60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb72400c460 tx=0x7fb72400c920 comp rx=0 tx=0).stop 2024-04-24T22:59:13.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.561+0000 7fb732d9d640 1 -- 172.21.15.32:0/3277234912 reap_dead start 2024-04-24T22:59:13.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.561+0000 7fd464a93640 1 -- 172.21.15.32:0/3356141677 >> [v2:172.21.15.121:6800/4010553979,v1:172.21.15.121:6801/4010553979] conn(0x7fd440074030 msgr2=0x7fd440076450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.561+0000 7fd464a93640 1 --2- 172.21.15.32:0/3356141677 >> [v2:172.21.15.121:6800/4010553979,v1:172.21.15.121:6801/4010553979] conn(0x7fd440074030 0x7fd440076450 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.561+0000 7fb73359e640 1 -- 172.21.15.32:0/3277234912 shutdown_connections 2024-04-24T22:59:13.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.561+0000 7fb73359e640 1 -- 172.21.15.32:0/3277234912 >> 172.21.15.32:0/3277234912 conn(0x7fb72c00a3f0 msgr2=0x7fb72c048b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:13.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.562+0000 7fb73359e640 1 -- 172.21.15.32:0/3277234912 shutdown_connections 2024-04-24T22:59:13.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.562+0000 7fd464a93640 1 -- 172.21.15.32:0/3356141677 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd44006be30 msgr2=0x7fd44006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.562+0000 7fd464a93640 1 --2- 172.21.15.32:0/3356141677 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd44006be30 0x7fd44006e2f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd458034000 tx=0x7fd45802f000 comp rx=0 tx=0).stop 2024-04-24T22:59:13.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.562+0000 7fd464a93640 1 -- 172.21.15.32:0/3356141677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4600cca00 msgr2=0x7fd460148b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.562+0000 7fd464a93640 1 --2- 172.21.15.32:0/3356141677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4600cca00 0x7fd460148b40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd45000d600 tx=0x7fd45000dac0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.562+0000 7fb73359e640 1 -- 172.21.15.32:0/3277234912 wait complete. 2024-04-24T22:59:13.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.562+0000 7fd45ed9d640 1 -- 172.21.15.32:0/3356141677 reap_dead start 2024-04-24T22:59:13.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.563+0000 7fd464a93640 1 -- 172.21.15.32:0/3356141677 shutdown_connections 2024-04-24T22:59:13.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.563+0000 7fd464a93640 1 -- 172.21.15.32:0/3356141677 >> 172.21.15.32:0/3356141677 conn(0x7fd460074210 msgr2=0x7fd4600b0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:13.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.567+0000 7fd464a93640 1 -- 172.21.15.32:0/3356141677 shutdown_connections 2024-04-24T22:59:13.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.567+0000 7fd464a93640 1 -- 172.21.15.32:0/3356141677 wait complete. 2024-04-24T22:59:13.621 INFO:teuthology.orchestra.run.smithi032.stdout:55834574854 2024-04-24T22:59:13.622 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-24T22:59:13.623 INFO:teuthology.orchestra.run.smithi032.stdout:55834574854 2024-04-24T22:59:13.623 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-24T22:59:13.624 INFO:teuthology.orchestra.run.smithi032.stdout:55834574854 2024-04-24T22:59:13.624 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-24T22:59:13.625 INFO:teuthology.orchestra.run.smithi032.stdout:55834574854 2024-04-24T22:59:13.625 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-24T22:59:13.626 INFO:teuthology.orchestra.run.smithi032.stdout:55834574854 2024-04-24T22:59:13.626 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-24T22:59:13.627 INFO:teuthology.orchestra.run.smithi032.stdout:55834574854 2024-04-24T22:59:13.627 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-24T22:59:13.628 INFO:teuthology.orchestra.run.smithi032.stdout:55834574854 2024-04-24T22:59:13.628 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-24T22:59:13.629 INFO:teuthology.orchestra.run.smithi032.stdout:55834574854 2024-04-24T22:59:13.629 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-24T22:59:13.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.834+0000 7f9a55610640 1 Processor -- start 2024-04-24T22:59:13.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.834+0000 7f9a55610640 1 -- start start 2024-04-24T22:59:13.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.835+0000 7f9a55610640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9a500b25b0 0x7f9a5015ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.835+0000 7f9a55610640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9a5015edf0 0x7f9a5015b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.835+0000 7f9a55610640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a5015bbe0 con 0x7f9a500734a0 2024-04-24T22:59:13.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.835+0000 7f9a55610640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f9a501586a0 con 0x7f9a5015edf0 2024-04-24T22:59:13.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.835+0000 7f9a55610640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f9a5015b900 con 0x7f9a500b25b0 2024-04-24T22:59:13.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.836+0000 7f9a4effd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9a500734a0 0x7f9a500b1ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47426/0 (socket says 172.21.15.32:47426) 2024-04-24T22:59:13.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.836+0000 7f9a4effd640 1 -- 172.21.15.32:0/2819182253 learned_addr learned my addr 172.21.15.32:0/2819182253 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:13.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.836+0000 7f9a4dffb640 1 -- 172.21.15.32:0/2819182253 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1048659738 0 0) 0x7f9a5015bbe0 con 0x7f9a500734a0 2024-04-24T22:59:13.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.836+0000 7f9a4dffb640 1 -- 172.21.15.32:0/2819182253 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a38003520 con 0x7f9a500734a0 2024-04-24T22:59:13.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.836+0000 7f9a4e7fc640 1 --2- 172.21.15.32:0/2819182253 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9a500b25b0 0x7f9a5015ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.837+0000 7f9a4dffb640 1 -- 172.21.15.32:0/2819182253 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2104475909 0 0) 0x7f9a38003520 con 0x7f9a500734a0 2024-04-24T22:59:13.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.837+0000 7f9a4dffb640 1 -- 172.21.15.32:0/2819182253 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9a500b25b0 msgr2=0x7f9a5015ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:13.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.837+0000 7f9a4dffb640 1 --2- 172.21.15.32:0/2819182253 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9a500b25b0 0x7f9a5015ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.837+0000 7f9a4dffb640 1 -- 172.21.15.32:0/2819182253 >> v2:172.21.15.121:3300/0 conn(0x7f9a5015edf0 msgr2=0x7f9a5015b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:13.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.837+0000 7f9a4dffb640 1 --2- 172.21.15.32:0/2819182253 >> v2:172.21.15.121:3300/0 conn(0x7f9a5015edf0 0x7f9a5015b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.837+0000 7f9a4dffb640 1 -- 172.21.15.32:0/2819182253 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a5015b690 con 0x7f9a500734a0 2024-04-24T22:59:13.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.837+0000 7f9a4dffb640 1 -- 172.21.15.32:0/2819182253 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f9a40001670 con 0x7f9a500734a0 2024-04-24T22:59:13.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.837+0000 7f9a4dffb640 1 -- 172.21.15.32:0/2819182253 >> v1:172.21.15.32:6789/0 conn(0x7f9a500734a0 legacy=0x7f9a500b1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.837+0000 7f9a4dffb640 1 --2- 172.21.15.32:0/2819182253 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a380039b0 0x7f9a38003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.837+0000 7f9a4dffb640 1 --2- 172.21.15.32:0/2819182253 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9a38004800 0x7f9a38004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.837+0000 7f9a4dffb640 1 --2- 172.21.15.32:0/2819182253 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a380051c0 0x7f9a3800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.837+0000 7f9a4dffb640 1 -- 172.21.15.32:0/2819182253 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5015b900 con 0x7f9a380039b0 2024-04-24T22:59:13.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.837+0000 7f9a4dffb640 1 -- 172.21.15.32:0/2819182253 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9a3800c090 con 0x7f9a380051c0 2024-04-24T22:59:13.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.837+0000 7f9a4dffb640 1 -- 172.21.15.32:0/2819182253 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f9a3800c290 con 0x7f9a38004800 2024-04-24T22:59:13.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.836+0000 7f818656a640 1 Processor -- start 2024-04-24T22:59:13.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.836+0000 7f818656a640 1 -- start start 2024-04-24T22:59:13.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.837+0000 7fb237cfc640 1 Processor -- start 2024-04-24T22:59:13.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.837+0000 7fb237cfc640 1 -- start start 2024-04-24T22:59:13.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.837+0000 7ff4a7d22640 1 Processor -- start 2024-04-24T22:59:13.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.837+0000 7ff4a7d22640 1 -- start start 2024-04-24T22:59:13.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.838+0000 7f9a4effd640 1 --2- 172.21.15.32:0/2819182253 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a380039b0 0x7f9a38003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.838+0000 7f9a4e7fc640 1 --2- 172.21.15.32:0/2819182253 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9a38004800 0x7f9a38004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.838+0000 7f9a4f7fe640 1 --2- 172.21.15.32:0/2819182253 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a380051c0 0x7f9a3800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.839+0000 7f9a4e7fc640 1 -- 172.21.15.32:0/2819182253 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a380051c0 msgr2=0x7f9a3800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:13.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.839+0000 7f9a4e7fc640 1 --2- 172.21.15.32:0/2819182253 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a380051c0 0x7f9a3800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.839+0000 7f9a4e7fc640 1 -- 172.21.15.32:0/2819182253 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a380039b0 msgr2=0x7f9a38003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:13.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.839+0000 7f9a4e7fc640 1 --2- 172.21.15.32:0/2819182253 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a380039b0 0x7f9a38003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.839+0000 7f9a4e7fc640 1 -- 172.21.15.32:0/2819182253 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a38003520 con 0x7f9a38004800 2024-04-24T22:59:13.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.839+0000 7f9a4effd640 1 --2- 172.21.15.32:0/2819182253 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a380039b0 0x7f9a38003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:59:13.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.838+0000 7ff4a7d22640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff4980df030 0x7ff498038170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.838+0000 7ff4a7d22640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff498050140 0x7ff498006a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.838+0000 7ff4a7d22640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff498005f60 con 0x7ff4980df410 2024-04-24T22:59:13.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.838+0000 7ff4a7d22640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7ff4980e38e0 con 0x7ff4980df030 2024-04-24T22:59:13.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.838+0000 7ff4a7d22640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7ff498006200 con 0x7ff498050140 2024-04-24T22:59:13.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.839+0000 7ff4a52bd640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7ff4980df410 0x7ff49804fa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:52254/0 (socket says 172.21.15.32:52254) 2024-04-24T22:59:13.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.839+0000 7ff4a52bd640 1 -- 172.21.15.32:0/2330052393 learned_addr learned my addr 172.21.15.32:0/2330052393 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:13.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.839+0000 7ff4a62bf640 1 --2- 172.21.15.32:0/2330052393 >> v2:172.21.15.121:3300/0 conn(0x7ff498050140 0x7ff498006a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.839+0000 7ff4a4abc640 1 -- 172.21.15.32:0/2330052393 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1360262967 0 0) 0x7ff498005f60 con 0x7ff4980df410 2024-04-24T22:59:13.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.839+0000 7ff4a4abc640 1 -- 172.21.15.32:0/2330052393 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff48c003440 con 0x7ff4980df410 2024-04-24T22:59:13.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.840+0000 7ff4a5abe640 1 --2- 172.21.15.32:0/2330052393 >> v2:172.21.15.32:3300/0 conn(0x7ff4980df030 0x7ff498038170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.840+0000 7ff4a62bf640 1 -- 172.21.15.32:0/2330052393 >> v2:172.21.15.32:3300/0 conn(0x7ff4980df030 msgr2=0x7ff498038170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:13.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.840+0000 7ff4a62bf640 1 --2- 172.21.15.32:0/2330052393 >> v2:172.21.15.32:3300/0 conn(0x7ff4980df030 0x7ff498038170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.840+0000 7ff4a4abc640 1 -- 172.21.15.32:0/2330052393 <== mon.1 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3890547356 0 0) 0x7ff48c003440 con 0x7ff4980df410 2024-04-24T22:59:13.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.840+0000 7ff4a62bf640 1 -- 172.21.15.32:0/2330052393 >> v1:172.21.15.121:6789/0 conn(0x7ff4980df410 legacy=0x7ff49804fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.840+0000 7ff4a62bf640 1 -- 172.21.15.32:0/2330052393 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4980064e0 con 0x7ff498050140 2024-04-24T22:59:13.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.839+0000 7fb237cfc640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb23015b510 0x7fb23006c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.839+0000 7fb237cfc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb2300cd3c0 0x7fb2300b2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.839+0000 7fb237cfc640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb2300b6e00 0x7fb2300b33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.839+0000 7fb235a98640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb23015b510 0x7fb23006c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.839+0000 7fb235a98640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb23015b510 0x7fb23006c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.32:42058/0 (socket says 172.21.15.32:42058) 2024-04-24T22:59:13.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.839+0000 7fb235a98640 1 -- 172.21.15.32:0/4043944318 learned_addr learned my addr 172.21.15.32:0/4043944318 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:13.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.839+0000 7fb237cfc640 1 -- 172.21.15.32:0/4043944318 --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fb2300cd8a0 con 0x7fb2300cd3c0 2024-04-24T22:59:13.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.839+0000 7fb237cfc640 1 -- 172.21.15.32:0/4043944318 --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fb23006b240 con 0x7fb2300b6e00 2024-04-24T22:59:13.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.839+0000 7fb237cfc640 1 -- 172.21.15.32:0/4043944318 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fb23006b520 con 0x7fb23015b510 2024-04-24T22:59:13.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.839+0000 7fb236299640 1 --2- 172.21.15.32:0/4043944318 >> v2:172.21.15.121:3300/0 conn(0x7fb2300b6e00 0x7fb2300b33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.838+0000 7f4c73381640 1 Processor -- start 2024-04-24T22:59:13.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.838+0000 7f4c73381640 1 -- start start 2024-04-24T22:59:13.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.840+0000 7fe76ffff640 1 Processor -- start 2024-04-24T22:59:13.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.839+0000 7fb235a98640 1 -- 172.21.15.32:0/4043944318 >> v2:172.21.15.121:3300/0 conn(0x7fb2300b6e00 msgr2=0x7fb2300b33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:13.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.839+0000 7fb235a98640 1 --2- 172.21.15.32:0/4043944318 >> v2:172.21.15.121:3300/0 conn(0x7fb2300b6e00 0x7fb2300b33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.839+0000 7fb235a98640 1 -- 172.21.15.32:0/4043944318 >> v2:172.21.15.32:3300/0 conn(0x7fb2300cd3c0 msgr2=0x7fb2300b2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:13.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.839+0000 7fb235a98640 1 --2- 172.21.15.32:0/4043944318 >> v2:172.21.15.32:3300/0 conn(0x7fb2300cd3c0 0x7fb2300b2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.839+0000 7fb235a98640 1 -- 172.21.15.32:0/4043944318 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb23006b800 con 0x7fb23015b510 2024-04-24T22:59:13.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.839+0000 7fb236299640 1 --2- 172.21.15.32:0/4043944318 >> v2:172.21.15.121:3300/0 conn(0x7fb2300b6e00 0x7fb2300b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:59:13.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.840+0000 7f4c73381640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4c640e30c0 0x7f4c64057d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.840+0000 7fe76ffff640 1 -- start start 2024-04-24T22:59:13.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.841+0000 7f9a4e7fc640 1 --2- 172.21.15.32:0/2819182253 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9a38004800 0x7f9a38004c80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9a48009ed0 tx=0x7f9a38005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a11f2078009f7d26 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.841+0000 7f4c73381640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4c6403e840 0x7f4c64053bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.838+0000 7f818656a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8178104970 0x7f8178104e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.838+0000 7f818656a640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8178054090 con 0x7f8178054370 2024-04-24T22:59:13.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.838+0000 7f818656a640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8178006220 con 0x7f8178039a80 2024-04-24T22:59:13.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.838+0000 7f818656a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f8178006590 con 0x7f8178104970 2024-04-24T22:59:13.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.841+0000 7f03bc91d640 1 Processor -- start 2024-04-24T22:59:13.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.841+0000 7f03bc91d640 1 -- start start 2024-04-24T22:59:13.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.839+0000 7f817f7fe640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f8178039a80 0x7f817803a3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:52256/0 (socket says 172.21.15.32:52256) 2024-04-24T22:59:13.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.839+0000 7f817f7fe640 1 -- 172.21.15.32:0/3457511594 learned_addr learned my addr 172.21.15.32:0/3457511594 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:13.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.840+0000 7f817effd640 1 -- 172.21.15.32:0/3457511594 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3202696300 0 0) 0x7f8178006220 con 0x7f8178039a80 2024-04-24T22:59:13.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.840+0000 7f817effd640 1 -- 172.21.15.32:0/3457511594 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8170003520 con 0x7f8178039a80 2024-04-24T22:59:13.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.841+0000 7f4c7237f640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4c640e30c0 0x7f4c64057d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.840+0000 7f817effd640 1 -- 172.21.15.32:0/3457511594 <== mon.1 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 455574834 0 0) 0x7f8170003520 con 0x7f8178039a80 2024-04-24T22:59:13.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.840+0000 7f817effd640 1 -- 172.21.15.32:0/3457511594 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8178104970 msgr2=0x7f8178104e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:13.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.840+0000 7f817effd640 1 --2- 172.21.15.32:0/3457511594 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8178104970 0x7f8178104e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.840+0000 7f817effd640 1 -- 172.21.15.32:0/3457511594 >> v1:172.21.15.32:6789/0 conn(0x7f8178054370 legacy=0x7f8178037b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.840+0000 7f817effd640 1 -- 172.21.15.32:0/3457511594 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8178108ca0 con 0x7f8178039a80 2024-04-24T22:59:13.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.840+0000 7f817effd640 1 -- 172.21.15.32:0/3457511594 <== mon.1 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f816c001670 con 0x7f8178039a80 2024-04-24T22:59:13.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.840+0000 7f817effd640 1 -- 172.21.15.32:0/3457511594 >> v1:172.21.15.121:6789/0 conn(0x7f8178039a80 legacy=0x7f817803a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.840+0000 7f817effd640 1 --2- 172.21.15.32:0/3457511594 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81700039b0 0x7f8170003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.840+0000 7f817effd640 1 --2- 172.21.15.32:0/3457511594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8170004800 0x7f8170004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.840+0000 7f817effd640 1 --2- 172.21.15.32:0/3457511594 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8178104970 0x7f81700052a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.840+0000 7f817effd640 1 -- 172.21.15.32:0/3457511594 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8178006590 con 0x7f8170004800 2024-04-24T22:59:13.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.840+0000 7f817effd640 1 -- 172.21.15.32:0/3457511594 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f817000bbd0 con 0x7f81700039b0 2024-04-24T22:59:13.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.840+0000 7f817effd640 1 -- 172.21.15.32:0/3457511594 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f817000bdd0 con 0x7f8178104970 2024-04-24T22:59:13.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.841+0000 7f8184b07640 1 --2- 172.21.15.32:0/3457511594 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8178104970 0x7f81700052a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.841+0000 7f817ffff640 1 --2- 172.21.15.32:0/3457511594 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81700039b0 0x7f8170003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.841+0000 7f4c7237f640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4c640e30c0 0x7f4c64057d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.32:48830/0 (socket says 172.21.15.32:48830) 2024-04-24T22:59:13.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.841+0000 7f817f7fe640 1 --2- 172.21.15.32:0/3457511594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8170004800 0x7f8170004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.841+0000 7f4c7237f640 1 -- 172.21.15.32:0/3447078204 learned_addr learned my addr 172.21.15.32:0/3447078204 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:13.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.841+0000 7f4c73381640 1 --2- 172.21.15.32:0/3447078204 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4c64054130 0x7f4c640e6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.841+0000 7f9a4dffb640 1 -- 172.21.15.32:0/2819182253 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4803c070 con 0x7f9a38004800 2024-04-24T22:59:13.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.841+0000 7f4c73381640 1 -- 172.21.15.32:0/3447078204 --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f4c640545a0 con 0x7f4c6403e840 2024-04-24T22:59:13.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.841+0000 7f4c73381640 1 -- 172.21.15.32:0/3447078204 --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f4c640582d0 con 0x7f4c640e30c0 2024-04-24T22:59:13.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.841+0000 7f4c72b80640 1 --2- 172.21.15.32:0/3447078204 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4c64054130 0x7f4c640e6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.841+0000 7fb235a98640 1 --2- 172.21.15.32:0/4043944318 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb23015b510 0x7fb23006c000 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb23006d0d0 tx=0x7fb22c02e970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca62f8b725e4e9fc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.841+0000 7f4c73381640 1 -- 172.21.15.32:0/3447078204 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f4c64058570 con 0x7f4c64054130 2024-04-24T22:59:13.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.841+0000 7f9a4dffb640 1 -- 172.21.15.32:0/2819182253 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a4802f8d0 con 0x7f9a38004800 2024-04-24T22:59:13.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.841+0000 7ff4a62bf640 1 --2- 172.21.15.32:0/2330052393 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff498050140 0x7ff498006a00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff498034500 tx=0x7ff49002a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8117dd54072a463e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.841+0000 7fb234a96640 1 -- 172.21.15.32:0/4043944318 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22c038020 con 0x7fb23015b510 2024-04-24T22:59:13.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.842+0000 7fb234a96640 1 -- 172.21.15.32:0/4043944318 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb22c034870 con 0x7fb23015b510 2024-04-24T22:59:13.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.842+0000 7fb234a96640 1 -- 172.21.15.32:0/4043944318 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22c034c40 con 0x7fb23015b510 2024-04-24T22:59:13.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.841+0000 7f8184b07640 1 -- 172.21.15.32:0/3457511594 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81700039b0 msgr2=0x7f8170003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:13.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.841+0000 7f8184b07640 1 --2- 172.21.15.32:0/3457511594 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81700039b0 0x7f8170003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.841+0000 7f8184b07640 1 -- 172.21.15.32:0/3457511594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8170004800 msgr2=0x7f8170004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:13.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.842+0000 7f8184b07640 1 --2- 172.21.15.32:0/3457511594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8170004800 0x7f8170004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.842+0000 7f8184b07640 1 -- 172.21.15.32:0/3457511594 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8170003520 con 0x7f8178104970 2024-04-24T22:59:13.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.842+0000 7f817f7fe640 1 --2- 172.21.15.32:0/3457511594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8170004800 0x7f8170004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:59:13.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.842+0000 7ff4a4abc640 1 -- 172.21.15.32:0/2330052393 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff490037070 con 0x7ff498050140 2024-04-24T22:59:13.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.842+0000 7f4c7237f640 1 -- 172.21.15.32:0/3447078204 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4c64054130 msgr2=0x7f4c640e6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:13.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.842+0000 7f4c7237f640 1 --2- 172.21.15.32:0/3447078204 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4c64054130 0x7f4c640e6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.842+0000 7f4c7237f640 1 -- 172.21.15.32:0/3447078204 >> v2:172.21.15.32:3300/0 conn(0x7f4c6403e840 msgr2=0x7f4c64053bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:13.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.842+0000 7f4c7237f640 1 --2- 172.21.15.32:0/3447078204 >> v2:172.21.15.32:3300/0 conn(0x7f4c6403e840 0x7f4c64053bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.842+0000 7f4c7237f640 1 -- 172.21.15.32:0/3447078204 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c64058850 con 0x7f4c640e30c0 2024-04-24T22:59:13.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.842+0000 7fe76ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe7680df800 0x7fe768055d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.842+0000 7fe76ffff640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe768050150 con 0x7fe76803a0a0 2024-04-24T22:59:13.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.842+0000 7fe76ffff640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe768006c30 con 0x7fe7680366a0 2024-04-24T22:59:13.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.842+0000 7fe76ffff640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fe768006f10 con 0x7fe7680df800 2024-04-24T22:59:13.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.842+0000 7f03bc91d640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f03b815e5e0 0x7f03b806cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.843+0000 7f03b659c640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f03b815e5e0 0x7f03b806cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.842+0000 7f9a55610640 1 -- 172.21.15.32:0/2819182253 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9a38004800 msgr2=0x7f9a38004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.842+0000 7f9a55610640 1 --2- 172.21.15.32:0/2819182253 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9a38004800 0x7f9a38004c80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9a48009ed0 tx=0x7f9a38005830 comp rx=0 tx=0).stop 2024-04-24T22:59:13.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.842+0000 7f9a4f7fe640 1 -- 172.21.15.32:0/2819182253 reap_dead start 2024-04-24T22:59:13.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.842+0000 7f9a55610640 1 -- 172.21.15.32:0/2819182253 shutdown_connections 2024-04-24T22:59:13.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.842+0000 7f9a55610640 1 -- 172.21.15.32:0/2819182253 >> 172.21.15.32:0/2819182253 conn(0x7f9a500743f0 msgr2=0x7f9a5006b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:13.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.843+0000 7fb237cfc640 1 -- 172.21.15.32:0/4043944318 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb23015b510 msgr2=0x7fb23006c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.843+0000 7fb237cfc640 1 --2- 172.21.15.32:0/4043944318 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb23015b510 0x7fb23006c000 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb23006d0d0 tx=0x7fb22c02e970 comp rx=0 tx=0).stop 2024-04-24T22:59:13.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.843+0000 7fb237cfc640 1 -- 172.21.15.32:0/4043944318 shutdown_connections 2024-04-24T22:59:13.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.843+0000 7fb237cfc640 1 --2- 172.21.15.32:0/4043944318 >> v2:172.21.15.121:3300/0 conn(0x7fb2300b6e00 0x7fb2300b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.843+0000 7fb237cfc640 1 --2- 172.21.15.32:0/4043944318 >> v2:172.21.15.32:3300/0 conn(0x7fb2300cd3c0 0x7fb2300b2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.843+0000 7f03b659c640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f03b815e5e0 0x7f03b806cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.32:48838/0 (socket says 172.21.15.32:48838) 2024-04-24T22:59:13.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.843+0000 7f03b5d9b640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f03b80cfd20 0x7f03b81621c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47432/0 (socket says 172.21.15.32:47432) 2024-04-24T22:59:13.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.843+0000 7f03bc91d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f03b81628d0 0x7f03b80d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.843+0000 7f03bc91d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03b80aea50 con 0x7f03b80cfd20 2024-04-24T22:59:13.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.843+0000 7f03bc91d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f03b8163190 con 0x7f03b81628d0 2024-04-24T22:59:13.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.842+0000 7ff4a4abc640 1 -- 172.21.15.32:0/2330052393 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff49002ab90 con 0x7ff498050140 2024-04-24T22:59:13.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.843+0000 7ff4a7d22640 1 -- 172.21.15.32:0/2330052393 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff498050140 msgr2=0x7ff498006a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.843+0000 7fb237cfc640 1 --2- 172.21.15.32:0/4043944318 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb23015b510 0x7fb23006c000 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.843+0000 7f03bc91d640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f03b80d06a0 con 0x7f03b815e5e0 2024-04-24T22:59:13.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.843+0000 7f03b659c640 1 -- 172.21.15.32:0/2319546268 learned_addr learned my addr 172.21.15.32:0/2319546268 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:13.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.843+0000 7fb237cfc640 1 -- 172.21.15.32:0/4043944318 >> 172.21.15.32:0/4043944318 conn(0x7fb2300b6030 msgr2=0x7fb2300b66f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:13.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.843+0000 7f4c72b80640 1 --2- 172.21.15.32:0/3447078204 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4c64054130 0x7f4c640e6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:59:13.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.843+0000 7ff4a4abc640 1 -- 172.21.15.32:0/2330052393 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff490037070 con 0x7ff498050140 2024-04-24T22:59:13.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.841+0000 7f4d07370640 1 Processor -- start 2024-04-24T22:59:13.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.841+0000 7f4d07370640 1 -- start start 2024-04-24T22:59:13.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.843+0000 7f4d07370640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4d000cd160 0x7f4d0015ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.843+0000 7f4d07370640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d000cdf50 con 0x7f4d000cf840 2024-04-24T22:59:13.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.843+0000 7f4d07370640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d000ce230 con 0x7f4d00154e20 2024-04-24T22:59:13.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.843+0000 7f4d07370640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f4d0015e8e0 con 0x7f4d000cd160 2024-04-24T22:59:13.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.843+0000 7f03b6d9d640 1 --2- 172.21.15.32:0/2319546268 >> v2:172.21.15.32:3300/0 conn(0x7f03b81628d0 0x7f03b80d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.843+0000 7f9a55610640 1 -- 172.21.15.32:0/2819182253 shutdown_connections 2024-04-24T22:59:13.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.843+0000 7ff4a7d22640 1 --2- 172.21.15.32:0/2330052393 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff498050140 0x7ff498006a00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff498034500 tx=0x7ff49002a100 comp rx=0 tx=0).stop 2024-04-24T22:59:13.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.843+0000 7ff4a7d22640 1 -- 172.21.15.32:0/2330052393 shutdown_connections 2024-04-24T22:59:13.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.843+0000 7ff4a7d22640 1 --2- 172.21.15.32:0/2330052393 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff498050140 0x7ff498006a00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.843+0000 7ff4a7d22640 1 --2- 172.21.15.32:0/2330052393 >> v2:172.21.15.32:3300/0 conn(0x7ff4980df030 0x7ff498038170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.843+0000 7ff4a7d22640 1 -- 172.21.15.32:0/2330052393 >> 172.21.15.32:0/2330052393 conn(0x7ff498009fe0 msgr2=0x7ff49800a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:13.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.842+0000 7fe76effd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe7680df800 0x7fe768055d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.843+0000 7fe76effd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe7680df800 0x7fe768055d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38556/0 (socket says 172.21.15.32:38556) 2024-04-24T22:59:13.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.843+0000 7fe76effd640 1 -- 172.21.15.32:0/1640510275 learned_addr learned my addr 172.21.15.32:0/1640510275 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:13.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.842+0000 7fe76e7fc640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fe7680366a0 0x7fe76804fa40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:52260/0 (socket says 172.21.15.32:52260) 2024-04-24T22:59:13.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.843+0000 7f4c7237f640 1 --2- 172.21.15.32:0/3447078204 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4c640e30c0 0x7f4c64057d90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4c64056960 tx=0x7f4c6802a280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=737a38d96c6d7a11 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.843+0000 7f4d0510c640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f4d00154e20 0x7f4d000cd770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:52266/0 (socket says 172.21.15.32:52266) 2024-04-24T22:59:13.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.843+0000 7f4d0510c640 1 -- 172.21.15.32:0/3515713297 learned_addr learned my addr 172.21.15.32:0/3515713297 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:13.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.843+0000 7f03b659c640 1 -- 172.21.15.32:0/2319546268 >> v2:172.21.15.32:3300/0 conn(0x7f03b81628d0 msgr2=0x7f03b80d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:13.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.843+0000 7f03b659c640 1 --2- 172.21.15.32:0/2319546268 >> v2:172.21.15.32:3300/0 conn(0x7f03b81628d0 0x7f03b80d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.843+0000 7f03b659c640 1 -- 172.21.15.32:0/2319546268 >> v1:172.21.15.32:6789/0 conn(0x7f03b80cfd20 legacy=0x7f03b81621c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:13.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.843+0000 7f03b659c640 1 -- 172.21.15.32:0/2319546268 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03b8162eb0 con 0x7f03b815e5e0 2024-04-24T22:59:13.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.843+0000 7fe76f7fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe76803a0a0 0x7fe76803a4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47430/0 (socket says 172.21.15.32:47430) 2024-04-24T22:59:13.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.843+0000 7f4d0590d640 1 --2- 172.21.15.32:0/3515713297 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4d000cd160 0x7f4d0015ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.843+0000 7f8184b07640 1 --2- 172.21.15.32:0/3457511594 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8178104970 0x7f81700052a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8180050ce0 tx=0x7f8170005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32cbe79d0678087 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.843+0000 7fe76effd640 1 -- 172.21.15.32:0/1640510275 >> v1:172.21.15.121:6789/0 conn(0x7fe7680366a0 legacy=0x7fe76804fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.843+0000 7fe76effd640 1 -- 172.21.15.32:0/1640510275 >> v1:172.21.15.32:6789/0 conn(0x7fe76803a0a0 legacy=0x7fe76803a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:13.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.843+0000 7fe76effd640 1 -- 172.21.15.32:0/1640510275 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7680071f0 con 0x7fe7680df800 2024-04-24T22:59:13.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.843+0000 7ff4a7d22640 1 -- 172.21.15.32:0/2330052393 shutdown_connections 2024-04-24T22:59:13.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.843+0000 7ff4a7d22640 1 -- 172.21.15.32:0/2330052393 wait complete. 2024-04-24T22:59:13.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.844+0000 7fb237cfc640 1 -- 172.21.15.32:0/4043944318 shutdown_connections 2024-04-24T22:59:13.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.844+0000 7fb237cfc640 1 -- 172.21.15.32:0/4043944318 wait complete. 2024-04-24T22:59:13.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.843+0000 7f9a55610640 1 -- 172.21.15.32:0/2819182253 wait complete. 2024-04-24T22:59:13.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.844+0000 7f4d0590d640 1 -- 172.21.15.32:0/3515713297 >> v1:172.21.15.121:6789/0 conn(0x7f4d00154e20 legacy=0x7f4d000cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.844+0000 7f4d0590d640 1 -- 172.21.15.32:0/3515713297 >> v1:172.21.15.32:6789/0 conn(0x7f4d000cf840 legacy=0x7f4d000cca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:13.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.844+0000 7f817effd640 1 -- 172.21.15.32:0/3457511594 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81800a2070 con 0x7f8178104970 2024-04-24T22:59:13.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.844+0000 7f817effd640 1 -- 172.21.15.32:0/3457511594 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8180095bf0 con 0x7f8178104970 2024-04-24T22:59:13.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.844+0000 7f4d0590d640 1 -- 172.21.15.32:0/3515713297 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d0015e290 con 0x7f4d000cd160 2024-04-24T22:59:13.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.844+0000 7fe76effd640 1 --2- 172.21.15.32:0/1640510275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7680df800 0x7fe768055d10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe76000bb00 tx=0x7fe76002c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caf05c30a8aad82c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.845+0000 7f817effd640 1 -- 172.21.15.32:0/3457511594 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8180067eb0 con 0x7f8178104970 2024-04-24T22:59:13.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.845+0000 7f03b659c640 1 --2- 172.21.15.32:0/2319546268 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03b815e5e0 0x7f03b806cad0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f03b8073410 tx=0x7f03ac02a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf935ebf4a18338e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.845+0000 7fe76dffb640 1 -- 172.21.15.32:0/1640510275 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe760032070 con 0x7fe7680df800 2024-04-24T22:59:13.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.845+0000 7fe76dffb640 1 -- 172.21.15.32:0/1640510275 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe760032620 con 0x7fe7680df800 2024-04-24T22:59:13.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.845+0000 7f4d0590d640 1 --2- 172.21.15.32:0/3515713297 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4d000cd160 0x7f4d0015ed70 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4d000b4540 tx=0x7f4cf802c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=add2f4f33255e6bb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.845+0000 7fe76dffb640 1 -- 172.21.15.32:0/1640510275 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7600329f0 con 0x7fe7680df800 2024-04-24T22:59:13.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.845+0000 7f03b559a640 1 -- 172.21.15.32:0/2319546268 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ac037070 con 0x7f03b815e5e0 2024-04-24T22:59:13.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.845+0000 7f03b559a640 1 -- 172.21.15.32:0/2319546268 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03ac032440 con 0x7f03b815e5e0 2024-04-24T22:59:13.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.845+0000 7ff4a7d22640 1 Processor -- start 2024-04-24T22:59:13.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.845+0000 7f9a55610640 1 Processor -- start 2024-04-24T22:59:13.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.845+0000 7ff4a7d22640 1 -- start start 2024-04-24T22:59:13.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.845+0000 7f818656a640 1 -- 172.21.15.32:0/3457511594 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8178104970 msgr2=0x7f81700052a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.845+0000 7f818656a640 1 --2- 172.21.15.32:0/3457511594 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8178104970 0x7f81700052a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8180050ce0 tx=0x7f8170005830 comp rx=0 tx=0).stop 2024-04-24T22:59:13.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.845+0000 7f8184b07640 1 -- 172.21.15.32:0/3457511594 reap_dead start 2024-04-24T22:59:13.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.846+0000 7f818656a640 1 -- 172.21.15.32:0/3457511594 shutdown_connections 2024-04-24T22:59:13.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.846+0000 7f818656a640 1 -- 172.21.15.32:0/3457511594 >> 172.21.15.32:0/3457511594 conn(0x7f8178009fe0 msgr2=0x7f817800a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:13.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.846+0000 7f03b559a640 1 -- 172.21.15.32:0/2319546268 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ac032810 con 0x7f03b815e5e0 2024-04-24T22:59:13.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.845+0000 7f4c7137d640 1 -- 172.21.15.32:0/3447078204 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c68037070 con 0x7f4c640e30c0 2024-04-24T22:59:13.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.845+0000 7f4c7137d640 1 -- 172.21.15.32:0/3447078204 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c68032440 con 0x7f4c640e30c0 2024-04-24T22:59:13.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.845+0000 7f4c7137d640 1 -- 172.21.15.32:0/3447078204 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c68032850 con 0x7f4c640e30c0 2024-04-24T22:59:13.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.846+0000 7f4c73381640 1 -- 172.21.15.32:0/3447078204 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4c640e30c0 msgr2=0x7f4c64057d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.846+0000 7f4c73381640 1 --2- 172.21.15.32:0/3447078204 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4c640e30c0 0x7f4c64057d90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4c64056960 tx=0x7f4c6802a280 comp rx=0 tx=0).stop 2024-04-24T22:59:13.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.846+0000 7f4c73381640 1 -- 172.21.15.32:0/3447078204 shutdown_connections 2024-04-24T22:59:13.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.846+0000 7f4c73381640 1 --2- 172.21.15.32:0/3447078204 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4c64054130 0x7f4c640e6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.846+0000 7f4c73381640 1 --2- 172.21.15.32:0/3447078204 >> v2:172.21.15.32:3300/0 conn(0x7f4c6403e840 0x7f4c64053bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.846+0000 7f4c73381640 1 --2- 172.21.15.32:0/3447078204 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4c640e30c0 0x7f4c64057d90 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.846+0000 7f4c73381640 1 -- 172.21.15.32:0/3447078204 >> 172.21.15.32:0/3447078204 conn(0x7f4c640559e0 msgr2=0x7f4c64055df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:13.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.846+0000 7f4cf7fff640 1 -- 172.21.15.32:0/3515713297 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf803a070 con 0x7f4d000cd160 2024-04-24T22:59:13.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.846+0000 7f4cf7fff640 1 -- 172.21.15.32:0/3515713297 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cf8032480 con 0x7f4d000cd160 2024-04-24T22:59:13.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.846+0000 7f4cf7fff640 1 -- 172.21.15.32:0/3515713297 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf8032850 con 0x7f4d000cd160 2024-04-24T22:59:13.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.845+0000 7fb237cfc640 1 Processor -- start 2024-04-24T22:59:13.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.845+0000 7fb237cfc640 1 -- start start 2024-04-24T22:59:13.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.846+0000 7fe76ffff640 1 -- 172.21.15.32:0/1640510275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7680df800 msgr2=0x7fe768055d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.846+0000 7fe76ffff640 1 --2- 172.21.15.32:0/1640510275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7680df800 0x7fe768055d10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe76000bb00 tx=0x7fe76002c7c0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.846+0000 7f03bc91d640 1 -- 172.21.15.32:0/2319546268 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03b815e5e0 msgr2=0x7f03b806cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.846+0000 7f03bc91d640 1 --2- 172.21.15.32:0/2319546268 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03b815e5e0 0x7f03b806cad0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f03b8073410 tx=0x7f03ac02a230 comp rx=0 tx=0).stop 2024-04-24T22:59:13.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.846+0000 7f9a55610640 1 -- start start 2024-04-24T22:59:13.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.846+0000 7fb237cfc640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb2300b6e00 0x7fb2301f0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.846+0000 7f818656a640 1 -- 172.21.15.32:0/3457511594 shutdown_connections 2024-04-24T22:59:13.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.846+0000 7f818656a640 1 -- 172.21.15.32:0/3457511594 wait complete. 2024-04-24T22:59:13.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.846+0000 7f03bc91d640 1 -- 172.21.15.32:0/2319546268 shutdown_connections 2024-04-24T22:59:13.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.846+0000 7f03bc91d640 1 --2- 172.21.15.32:0/2319546268 >> v2:172.21.15.32:3300/0 conn(0x7f03b81628d0 0x7f03b80d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.846+0000 7f03bc91d640 1 --2- 172.21.15.32:0/2319546268 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03b815e5e0 0x7f03b806cad0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.846+0000 7f03bc91d640 1 -- 172.21.15.32:0/2319546268 >> 172.21.15.32:0/2319546268 conn(0x7f03b80cd3f0 msgr2=0x7f03b80cd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:13.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.846+0000 7fb237cfc640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2300cd3c0 0x7fb23009a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.846+0000 7f03bc91d640 1 -- 172.21.15.32:0/2319546268 shutdown_connections 2024-04-24T22:59:13.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.847+0000 7f03bc91d640 1 -- 172.21.15.32:0/2319546268 wait complete. 2024-04-24T22:59:13.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.847+0000 7fb237cfc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb23015b510 0x7fb2300ae570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.847+0000 7fb235a98640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb2300b6e00 0x7fb2301f0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.847+0000 7fb235a98640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb2300b6e00 0x7fb2301f0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.32:42106/0 (socket says 172.21.15.32:42106) 2024-04-24T22:59:13.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.847+0000 7fb235297640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2300cd3c0 0x7fb23009a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.847+0000 7fb235297640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2300cd3c0 0x7fb23009a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.32:48842/0 (socket says 172.21.15.32:48842) 2024-04-24T22:59:13.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.847+0000 7f9a55610640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a5014de60 0x7f9a5015a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.847+0000 7f9a55610640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a5014e240 0x7f9a50071af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.847+0000 7ff4a7d22640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff498050140 0x7ff4980302c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.847+0000 7fe76ffff640 1 -- 172.21.15.32:0/1640510275 shutdown_connections 2024-04-24T22:59:13.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.847+0000 7fe76ffff640 1 --2- 172.21.15.32:0/1640510275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7680df800 0x7fe768055d10 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.847+0000 7fe76ffff640 1 -- 172.21.15.32:0/1640510275 >> 172.21.15.32:0/1640510275 conn(0x7fe76801e6c0 msgr2=0x7fe7680553f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:13.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.847+0000 7f4d07370640 1 -- 172.21.15.32:0/3515713297 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4d000cd160 msgr2=0x7f4d0015ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.847+0000 7f4d07370640 1 --2- 172.21.15.32:0/3515713297 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4d000cd160 0x7f4d0015ed70 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4d000b4540 tx=0x7f4cf802c6a0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.847+0000 7fb235a98640 1 -- 172.21.15.32:0/3627226877 learned_addr learned my addr 172.21.15.32:0/3627226877 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:13.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.847+0000 7ff4a7d22640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4980df030 0x7ff4980d5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.847+0000 7f9a4effd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a5014de60 0x7f9a5015a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.847+0000 7fb237cfc640 1 -- 172.21.15.32:0/3627226877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb23006ca20 con 0x7fb23015b510 2024-04-24T22:59:13.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.847+0000 7fb237cfc640 1 -- 172.21.15.32:0/3627226877 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb23006c8a0 con 0x7fb2300cd3c0 2024-04-24T22:59:13.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.847+0000 7f9a4e7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a5014e240 0x7f9a50071af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.847+0000 7fb236299640 1 --2- 172.21.15.32:0/3627226877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb23015b510 0x7fb2300ae570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.847+0000 7f9a4e7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a5014e240 0x7f9a50071af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38578/0 (socket says 172.21.15.32:38578) 2024-04-24T22:59:13.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.847+0000 7ff4a5abe640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff498050140 0x7ff4980302c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.847+0000 7ff4a5abe640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff498050140 0x7ff4980302c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.32:42120/0 (socket says 172.21.15.32:42120) 2024-04-24T22:59:13.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.847+0000 7fb237cfc640 1 -- 172.21.15.32:0/3627226877 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fb2300ae1b0 con 0x7fb2300b6e00 2024-04-24T22:59:13.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.847+0000 7f9a4e7fc640 1 -- 172.21.15.32:0/1504777339 learned_addr learned my addr 172.21.15.32:0/1504777339 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:13.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.847+0000 7f9a55610640 1 --2- 172.21.15.32:0/1504777339 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9a501f8b20 0x7f9a50072030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.847+0000 7fe76ffff640 1 -- 172.21.15.32:0/1640510275 shutdown_connections 2024-04-24T22:59:13.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.847+0000 7ff4a5abe640 1 -- 172.21.15.32:0/2397776407 learned_addr learned my addr 172.21.15.32:0/2397776407 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:13.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.847+0000 7f4c73381640 1 -- 172.21.15.32:0/3447078204 shutdown_connections 2024-04-24T22:59:13.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.847+0000 7ff4a52bd640 1 --2- 172.21.15.32:0/2397776407 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4980df030 0x7ff4980d5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.847+0000 7f4c73381640 1 -- 172.21.15.32:0/3447078204 wait complete. 2024-04-24T22:59:13.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.847+0000 7ff4a7d22640 1 --2- 172.21.15.32:0/2397776407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4980df410 0x7ff498038710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.847+0000 7f4d07370640 1 -- 172.21.15.32:0/3515713297 shutdown_connections 2024-04-24T22:59:13.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.847+0000 7f4d07370640 1 --2- 172.21.15.32:0/3515713297 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4d000cd160 0x7f4d0015ed70 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.847+0000 7f4d07370640 1 -- 172.21.15.32:0/3515713297 >> 172.21.15.32:0/3515713297 conn(0x7f4d0006b110 msgr2=0x7f4d0006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:13.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.848+0000 7f9a55610640 1 -- 172.21.15.32:0/1504777339 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9a50074800 con 0x7f9a5014e240 2024-04-24T22:59:13.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.847+0000 7fe76ffff640 1 -- 172.21.15.32:0/1640510275 wait complete. 2024-04-24T22:59:13.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.848+0000 7f9a55610640 1 -- 172.21.15.32:0/1504777339 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5015e5f0 con 0x7f9a5014de60 2024-04-24T22:59:13.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.848+0000 7f9a55610640 1 -- 172.21.15.32:0/1504777339 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f9a50074980 con 0x7f9a501f8b20 2024-04-24T22:59:13.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.848+0000 7fb235a98640 1 -- 172.21.15.32:0/3627226877 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2300cd3c0 msgr2=0x7fb23009a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.848+0000 7ff4a7d22640 1 -- 172.21.15.32:0/2397776407 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff4980507c0 con 0x7ff4980df410 2024-04-24T22:59:13.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.848+0000 7fb235a98640 1 --2- 172.21.15.32:0/3627226877 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2300cd3c0 0x7fb23009a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.848+0000 7fb235a98640 1 -- 172.21.15.32:0/3627226877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb23015b510 msgr2=0x7fb2300ae570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.848+0000 7fb235a98640 1 --2- 172.21.15.32:0/3627226877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb23015b510 0x7fb2300ae570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.848+0000 7fb235a98640 1 -- 172.21.15.32:0/3627226877 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2300b1380 con 0x7fb2300b6e00 2024-04-24T22:59:13.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.848+0000 7f9a4f7fe640 1 --2- 172.21.15.32:0/1504777339 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9a501f8b20 0x7f9a50072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.848+0000 7ff4a7d22640 1 -- 172.21.15.32:0/2397776407 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff498050640 con 0x7ff4980df030 2024-04-24T22:59:13.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.848+0000 7ff4a7d22640 1 -- 172.21.15.32:0/2397776407 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff4980546c0 con 0x7ff498050140 2024-04-24T22:59:13.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.848+0000 7ff4a62bf640 1 --2- 172.21.15.32:0/2397776407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4980df410 0x7ff498038710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.848+0000 7f9a4e7fc640 1 -- 172.21.15.32:0/1504777339 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9a501f8b20 msgr2=0x7f9a50072030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.848+0000 7fb235a98640 1 --2- 172.21.15.32:0/3627226877 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb2300b6e00 0x7fb2301f0c40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb22c00b2c0 tx=0x7fb22c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.848+0000 7f9a4e7fc640 1 --2- 172.21.15.32:0/1504777339 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9a501f8b20 0x7f9a50072030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.848+0000 7f9a4e7fc640 1 -- 172.21.15.32:0/1504777339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a5014de60 msgr2=0x7f9a5015a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.848+0000 7f9a4e7fc640 1 --2- 172.21.15.32:0/1504777339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a5014de60 0x7f9a5015a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.848+0000 7f9a4e7fc640 1 -- 172.21.15.32:0/1504777339 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a501f9020 con 0x7f9a5014e240 2024-04-24T22:59:13.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.849+0000 7fb2267fc640 1 -- 172.21.15.32:0/3627226877 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22c038040 con 0x7fb2300b6e00 2024-04-24T22:59:13.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.849+0000 7fb2267fc640 1 -- 172.21.15.32:0/3627226877 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb22c03dd60 con 0x7fb2300b6e00 2024-04-24T22:59:13.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.849+0000 7fb2267fc640 1 -- 172.21.15.32:0/3627226877 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22c047610 con 0x7fb2300b6e00 2024-04-24T22:59:13.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.848+0000 7f9a4effd640 1 --2- 172.21.15.32:0/1504777339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a5014de60 0x7f9a5015a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:59:13.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.848+0000 7f9a4f7fe640 1 --2- 172.21.15.32:0/1504777339 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9a501f8b20 0x7f9a50072030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:13.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.849+0000 7f9a4e7fc640 1 --2- 172.21.15.32:0/1504777339 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a5014e240 0x7f9a50071af0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9a4802ea20 tx=0x7f9a48004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.849+0000 7f03bc91d640 1 Processor -- start 2024-04-24T22:59:13.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.849+0000 7f03bc91d640 1 -- start start 2024-04-24T22:59:13.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.849+0000 7ff4a62bf640 1 -- 172.21.15.32:0/2397776407 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff498050140 msgr2=0x7ff4980302c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.849+0000 7ff4a62bf640 1 --2- 172.21.15.32:0/2397776407 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff498050140 0x7ff4980302c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.849+0000 7ff4a62bf640 1 -- 172.21.15.32:0/2397776407 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4980df030 msgr2=0x7ff4980d5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.849+0000 7ff4a62bf640 1 --2- 172.21.15.32:0/2397776407 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4980df030 0x7ff4980d5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.849+0000 7ff4a62bf640 1 -- 172.21.15.32:0/2397776407 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff49804d530 con 0x7ff4980df410 2024-04-24T22:59:13.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.849+0000 7ff4a52bd640 1 --2- 172.21.15.32:0/2397776407 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4980df030 0x7ff4980d5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:59:13.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.849+0000 7fb237cfc640 1 -- 172.21.15.32:0/3627226877 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb230157780 con 0x7fb2300b6e00 2024-04-24T22:59:13.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.849+0000 7fb237cfc640 1 -- 172.21.15.32:0/3627226877 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb230157d40 con 0x7fb2300b6e00 2024-04-24T22:59:13.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.848+0000 7f818656a640 1 Processor -- start 2024-04-24T22:59:13.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.848+0000 7f818656a640 1 -- start start 2024-04-24T22:59:13.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.849+0000 7ff4a62bf640 1 --2- 172.21.15.32:0/2397776407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4980df410 0x7ff498038710 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff49800bad0 tx=0x7ff4900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.849+0000 7f818656a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8178104970 0x7f817804e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.849+0000 7f818656a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8178107920 0x7f81780fa0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.849+0000 7f4c73381640 1 Processor -- start 2024-04-24T22:59:13.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.849+0000 7fe76ffff640 1 Processor -- start 2024-04-24T22:59:13.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.850+0000 7fe76ffff640 1 -- start start 2024-04-24T22:59:13.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.850+0000 7f817ffff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8178104970 0x7f817804e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.850+0000 7f817ffff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8178104970 0x7f817804e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.32:48872/0 (socket says 172.21.15.32:48872) 2024-04-24T22:59:13.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.850+0000 7f818656a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8178038730 0x7f8178038bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.850+0000 7f817ffff640 1 -- 172.21.15.32:0/3577223218 learned_addr learned my addr 172.21.15.32:0/3577223218 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:13.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.850+0000 7f818656a640 1 -- 172.21.15.32:0/3577223218 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8178039ee0 con 0x7f8178107920 2024-04-24T22:59:13.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.850+0000 7f818656a640 1 -- 172.21.15.32:0/3577223218 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f817803aae0 con 0x7f8178104970 2024-04-24T22:59:13.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.850+0000 7f818656a640 1 -- 172.21.15.32:0/3577223218 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f8178053cf0 con 0x7f8178038730 2024-04-24T22:59:13.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.850+0000 7f4c73381640 1 -- start start 2024-04-24T22:59:13.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.850+0000 7f4d07370640 1 -- 172.21.15.32:0/3515713297 shutdown_connections 2024-04-24T22:59:13.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.850+0000 7f4d07370640 1 -- 172.21.15.32:0/3515713297 wait complete. 2024-04-24T22:59:13.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.850+0000 7f9a377fe640 1 -- 172.21.15.32:0/1504777339 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4803c030 con 0x7f9a5014e240 2024-04-24T22:59:13.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.850+0000 7f9a377fe640 1 -- 172.21.15.32:0/1504777339 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a4803e070 con 0x7f9a5014e240 2024-04-24T22:59:13.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.850+0000 7ff4967fc640 1 -- 172.21.15.32:0/2397776407 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff490037050 con 0x7ff4980df410 2024-04-24T22:59:13.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.850+0000 7ff4967fc640 1 -- 172.21.15.32:0/2397776407 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff490043070 con 0x7ff4980df410 2024-04-24T22:59:13.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.850+0000 7ff4a7d22640 1 -- 172.21.15.32:0/2397776407 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff49804d810 con 0x7ff4980df410 2024-04-24T22:59:13.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.850+0000 7ff4a7d22640 1 -- 172.21.15.32:0/2397776407 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff49804ddd0 con 0x7ff4980df410 2024-04-24T22:59:13.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.851+0000 7f4c73381640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c6403e840 0x7f4c640d7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.851+0000 7f4c73381640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4c64054130 0x7f4c640525b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.851+0000 7f4c73381640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4c640e30c0 0x7f4c640e5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.851+0000 7f4c73381640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4c6403bfc0 con 0x7f4c6403e840 2024-04-24T22:59:13.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.851+0000 7f4c73381640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4c640063b0 con 0x7f4c640e30c0 2024-04-24T22:59:13.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.851+0000 7f4c73381640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f4c640e34a0 con 0x7f4c64054130 2024-04-24T22:59:13.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.850+0000 7f03bc91d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03b80cfd20 0x7f03b815c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.851+0000 7f03bc91d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f03b815e5e0 0x7f03b81e9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.851+0000 7f03b659c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03b80cfd20 0x7f03b815c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.851+0000 7f03b659c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03b80cfd20 0x7f03b815c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.32:48880/0 (socket says 172.21.15.32:48880) 2024-04-24T22:59:13.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.851+0000 7f03b659c640 1 -- 172.21.15.32:0/2707028841 learned_addr learned my addr 172.21.15.32:0/2707028841 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:13.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.851+0000 7f03bc91d640 1 --2- 172.21.15.32:0/2707028841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03b81628d0 0x7f03b81e95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.851+0000 7f03b5d9b640 1 --2- 172.21.15.32:0/2707028841 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f03b815e5e0 0x7f03b81e9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.851+0000 7f03bc91d640 1 -- 172.21.15.32:0/2707028841 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f03b815e9c0 con 0x7f03b81628d0 2024-04-24T22:59:13.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.851+0000 7f03bc91d640 1 -- 172.21.15.32:0/2707028841 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f03b80aefd0 con 0x7f03b80cfd20 2024-04-24T22:59:13.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.851+0000 7f03bc91d640 1 -- 172.21.15.32:0/2707028841 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f03b80af4f0 con 0x7f03b815e5e0 2024-04-24T22:59:13.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.850+0000 7ff4967fc640 1 -- 172.21.15.32:0/2397776407 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a004fa20 con 0x7ff4980df410 2024-04-24T22:59:13.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.851+0000 7fe76ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7680366a0 0x7fe768178070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.851+0000 7fe76ffff640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe76803a0a0 0x7fe7680dd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.851+0000 7fe76ffff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe7680df800 0x7fe768167d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.851+0000 7fe76ffff640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe7680dfbe0 con 0x7fe7680366a0 2024-04-24T22:59:13.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.851+0000 7fe76ffff640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe768036b10 con 0x7fe7680df800 2024-04-24T22:59:13.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.851+0000 7fe76ffff640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe768006220 con 0x7fe76803a0a0 2024-04-24T22:59:13.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.851+0000 7fe76e7fc640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe76803a0a0 0x7fe7680dd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.851+0000 7fe76e7fc640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe76803a0a0 0x7fe7680dd8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.32:42154/0 (socket says 172.21.15.32:42154) 2024-04-24T22:59:13.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.851+0000 7fe76e7fc640 1 -- 172.21.15.32:0/2898210307 learned_addr learned my addr 172.21.15.32:0/2898210307 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:13.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.851+0000 7f9a55610640 1 -- 172.21.15.32:0/1504777339 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a501f9300 con 0x7f9a5014e240 2024-04-24T22:59:13.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.851+0000 7f4c71b7e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4c64054130 0x7f4c640525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.852+0000 7f9a55610640 1 -- 172.21.15.32:0/1504777339 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a501f98c0 con 0x7f9a5014e240 2024-04-24T22:59:13.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.851+0000 7f4c7237f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c6403e840 0x7f4c640d7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.851+0000 7f4c7237f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c6403e840 0x7f4c640d7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38638/0 (socket says 172.21.15.32:38638) 2024-04-24T22:59:13.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.851+0000 7f4c7237f640 1 -- 172.21.15.32:0/1486441939 learned_addr learned my addr 172.21.15.32:0/1486441939 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:13.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.851+0000 7f4c71b7e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4c64054130 0x7f4c640525b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.32:42148/0 (socket says 172.21.15.32:42148) 2024-04-24T22:59:13.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.852+0000 7f4c72b80640 1 --2- 172.21.15.32:0/1486441939 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4c640e30c0 0x7f4c640e5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.852+0000 7f03b6d9d640 1 --2- 172.21.15.32:0/2707028841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03b81628d0 0x7f03b81e95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.852+0000 7fe76effd640 1 --2- 172.21.15.32:0/2898210307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7680366a0 0x7fe768178070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.852+0000 7fe76e7fc640 1 -- 172.21.15.32:0/2898210307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe7680df800 msgr2=0x7fe768167d90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-24T22:59:13.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.852+0000 7fe76e7fc640 1 --2- 172.21.15.32:0/2898210307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe7680df800 0x7fe768167d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.852+0000 7fe76e7fc640 1 -- 172.21.15.32:0/2898210307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7680366a0 msgr2=0x7fe768178070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.852+0000 7fe76e7fc640 1 --2- 172.21.15.32:0/2898210307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7680366a0 0x7fe768178070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.852+0000 7fe76e7fc640 1 -- 172.21.15.32:0/2898210307 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe768178670 con 0x7fe76803a0a0 2024-04-24T22:59:13.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.852+0000 7fe76effd640 1 --2- 172.21.15.32:0/2898210307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7680366a0 0x7fe768178070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:59:13.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.852+0000 7f4d07370640 1 Processor -- start 2024-04-24T22:59:13.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.852+0000 7f4d07370640 1 -- start start 2024-04-24T22:59:13.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.852+0000 7f4c71b7e640 1 -- 172.21.15.32:0/1486441939 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4c640e30c0 msgr2=0x7f4c640e5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.852+0000 7f4c71b7e640 1 --2- 172.21.15.32:0/1486441939 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4c640e30c0 0x7f4c640e5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.852+0000 7f4c71b7e640 1 -- 172.21.15.32:0/1486441939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c6403e840 msgr2=0x7f4c640d7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.852+0000 7f4c71b7e640 1 --2- 172.21.15.32:0/1486441939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c6403e840 0x7f4c640d7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.852+0000 7f4c71b7e640 1 -- 172.21.15.32:0/1486441939 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c6417dc00 con 0x7f4c64054130 2024-04-24T22:59:13.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.850+0000 7f8184b07640 1 --2- 172.21.15.32:0/3577223218 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8178038730 0x7f8178038bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.850+0000 7f8184b07640 1 -- 172.21.15.32:0/3577223218 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8178104970 msgr2=0x7f817804e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.850+0000 7f8184b07640 1 --2- 172.21.15.32:0/3577223218 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8178104970 0x7f817804e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.850+0000 7f8184b07640 1 -- 172.21.15.32:0/3577223218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8178107920 msgr2=0x7f81780fa0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.850+0000 7f8184b07640 1 --2- 172.21.15.32:0/3577223218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8178107920 0x7f81780fa0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.850+0000 7f8184b07640 1 -- 172.21.15.32:0/3577223218 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8178107f90 con 0x7f8178038730 2024-04-24T22:59:13.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.851+0000 7f8184b07640 1 --2- 172.21.15.32:0/3577223218 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8178038730 0x7f8178038bb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8180050770 tx=0x7f8180069030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.852+0000 7ff4a7d22640 1 -- 172.21.15.32:0/2397776407 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff488005290 con 0x7ff4980df410 2024-04-24T22:59:13.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.853+0000 7f4c71b7e640 1 --2- 172.21.15.32:0/1486441939 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4c64054130 0x7f4c640525b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c00b410 tx=0x7f4c5c00b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.853+0000 7f4c62ffd640 1 -- 172.21.15.32:0/1486441939 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c015020 con 0x7f4c64054130 2024-04-24T22:59:13.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.853+0000 7f4c73381640 1 -- 172.21.15.32:0/1486441939 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c64052fc0 con 0x7f4c64054130 2024-04-24T22:59:13.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.853+0000 7f4c73381640 1 -- 172.21.15.32:0/1486441939 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c6417e180 con 0x7f4c64054130 2024-04-24T22:59:13.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.853+0000 7f9a377fe640 1 -- 172.21.15.32:0/1504777339 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a480132e0 con 0x7f9a5014e240 2024-04-24T22:59:13.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.853+0000 7f4c62ffd640 1 -- 172.21.15.32:0/1486441939 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c5c011850 con 0x7f4c64054130 2024-04-24T22:59:13.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.853+0000 7f4c62ffd640 1 -- 172.21.15.32:0/1486441939 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c011c40 con 0x7f4c64054130 2024-04-24T22:59:13.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.853+0000 7f4d07370640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d000cd160 0x7f4d000b0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.853+0000 7f4d07370640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4d000cf840 0x7f4d00200540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.853+0000 7f4d07370640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d00154e20 0x7f4d00200a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.853+0000 7f4d07370640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4d0006e5e0 con 0x7f4d00154e20 2024-04-24T22:59:13.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.853+0000 7f4d07370640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4d0006e460 con 0x7f4d000cd160 2024-04-24T22:59:13.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.853+0000 7f4d07370640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f4d0015ac50 con 0x7f4d000cf840 2024-04-24T22:59:13.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.853+0000 7ff4967fc640 1 -- 172.21.15.32:0/2397776407 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff49003e890 con 0x7ff4980df410 2024-04-24T22:59:13.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.853+0000 7ff4967fc640 1 --2- 172.21.15.32:0/2397776407 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff48406bf00 0x7ff48406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.853+0000 7ff4967fc640 1 -- 172.21.15.32:0/2397776407 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff4900ab840 con 0x7ff4980df410 2024-04-24T22:59:13.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.853+0000 7f4d0510c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d000cd160 0x7f4d000b0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.852+0000 7fe76e7fc640 1 --2- 172.21.15.32:0/2898210307 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe76803a0a0 0x7fe7680dd8a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe76400c500 tx=0x7fe76400c9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.854+0000 7f4d0510c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d000cd160 0x7f4d000b0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.32:48904/0 (socket says 172.21.15.32:48904) 2024-04-24T22:59:13.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.854+0000 7f4d0510c640 1 -- 172.21.15.32:0/934933624 learned_addr learned my addr 172.21.15.32:0/934933624 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:13.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.854+0000 7f4d0490b640 1 --2- 172.21.15.32:0/934933624 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4d000cf840 0x7f4d00200540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.854+0000 7ff4a5abe640 1 --2- 172.21.15.32:0/2397776407 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff48406bf00 0x7ff48406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.853+0000 7f817cff9640 1 -- 172.21.15.32:0/3577223218 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81800af070 con 0x7f8178038730 2024-04-24T22:59:13.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.854+0000 7f817cff9640 1 -- 172.21.15.32:0/3577223218 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81800a3040 con 0x7f8178038730 2024-04-24T22:59:13.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.854+0000 7f817cff9640 1 -- 172.21.15.32:0/3577223218 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81800a2050 con 0x7f8178038730 2024-04-24T22:59:13.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.853+0000 7fb237cfc640 1 -- 172.21.15.32:0/3627226877 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2300689c0 con 0x7fb2300b6e00 2024-04-24T22:59:13.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.853+0000 7fb2267fc640 1 -- 172.21.15.32:0/3627226877 <== mon.2 v2:172.21.15.121:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb22c04e060 con 0x7fb2300b6e00 2024-04-24T22:59:13.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.853+0000 7fb2267fc640 1 --2- 172.21.15.32:0/3627226877 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb21406bf00 0x7fb21406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.854+0000 7fb2267fc640 1 -- 172.21.15.32:0/3627226877 <== mon.2 v2:172.21.15.121:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb22c0afe90 con 0x7fb2300b6e00 2024-04-24T22:59:13.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.854+0000 7f4d0510c640 1 -- 172.21.15.32:0/934933624 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4d000cf840 msgr2=0x7f4d00200540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.854+0000 7f4d0510c640 1 --2- 172.21.15.32:0/934933624 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4d000cf840 0x7f4d00200540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.854+0000 7f4d0510c640 1 -- 172.21.15.32:0/934933624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d00154e20 msgr2=0x7f4d00200a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.854+0000 7f4d0510c640 1 --2- 172.21.15.32:0/934933624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d00154e20 0x7f4d00200a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.854+0000 7f4d0510c640 1 -- 172.21.15.32:0/934933624 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d00200fc0 con 0x7f4d000cd160 2024-04-24T22:59:13.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.854+0000 7f4c73381640 1 -- 172.21.15.32:0/1486441939 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c640057e0 con 0x7f4c64054130 2024-04-24T22:59:13.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.854+0000 7f4d0510c640 1 --2- 172.21.15.32:0/934933624 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d000cd160 0x7f4d000b0b30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc00cbc0 tx=0x7f4cfc00a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.853+0000 7f03b659c640 1 -- 172.21.15.32:0/2707028841 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f03b815e5e0 msgr2=0x7f03b81e9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.853+0000 7f03b659c640 1 --2- 172.21.15.32:0/2707028841 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f03b815e5e0 0x7f03b81e9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.853+0000 7f03b659c640 1 -- 172.21.15.32:0/2707028841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03b81628d0 msgr2=0x7f03b81e95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:13.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.853+0000 7f03b659c640 1 --2- 172.21.15.32:0/2707028841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03b81628d0 0x7f03b81e95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:13.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.853+0000 7f03b659c640 1 -- 172.21.15.32:0/2707028841 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03b815fb70 con 0x7f03b80cfd20 2024-04-24T22:59:13.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.853+0000 7f03b659c640 1 --2- 172.21.15.32:0/2707028841 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03b80cfd20 0x7f03b815c8a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f03ac000c00 tx=0x7f03ac0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.854+0000 7f03a6ffd640 1 -- 172.21.15.32:0/2707028841 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ac041020 con 0x7f03b80cfd20 2024-04-24T22:59:13.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.854+0000 7f03bc91d640 1 -- 172.21.15.32:0/2707028841 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03b815fe50 con 0x7f03b80cfd20 2024-04-24T22:59:13.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.854+0000 7f03bc91d640 1 -- 172.21.15.32:0/2707028841 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03b8160410 con 0x7f03b80cfd20 2024-04-24T22:59:13.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.854+0000 7f03a6ffd640 1 -- 172.21.15.32:0/2707028841 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03ac030780 con 0x7f03b80cfd20 2024-04-24T22:59:13.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.855+0000 7ff4a5abe640 1 --2- 172.21.15.32:0/2397776407 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff48406bf00 0x7ff48406e3c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff49c006fd0 tx=0x7ff49c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.855+0000 7fb235297640 1 --2- 172.21.15.32:0/3627226877 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb21406bf00 0x7fb21406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.855+0000 7f03bc91d640 1 -- 172.21.15.32:0/2707028841 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f039c005290 con 0x7f03b80cfd20 2024-04-24T22:59:13.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.855+0000 7f4cf5ffb640 1 -- 172.21.15.32:0/934933624 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc017070 con 0x7f4d000cd160 2024-04-24T22:59:13.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.855+0000 7f4cf5ffb640 1 -- 172.21.15.32:0/934933624 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cfc00aca0 con 0x7f4d000cd160 2024-04-24T22:59:13.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.855+0000 7f4cf5ffb640 1 -- 172.21.15.32:0/934933624 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc012600 con 0x7f4d000cd160 2024-04-24T22:59:13.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.855+0000 7f4d07370640 1 -- 172.21.15.32:0/934933624 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d000731c0 con 0x7f4d000cd160 2024-04-24T22:59:13.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.856+0000 7f4d07370640 1 -- 172.21.15.32:0/934933624 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d000aa780 con 0x7f4d000cd160 2024-04-24T22:59:13.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.855+0000 7f818656a640 1 -- 172.21.15.32:0/3577223218 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8178108270 con 0x7f8178038730 2024-04-24T22:59:13.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.855+0000 7f818656a640 1 -- 172.21.15.32:0/3577223218 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8178048aa0 con 0x7f8178038730 2024-04-24T22:59:13.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.855+0000 7fe7577fe640 1 -- 172.21.15.32:0/2898210307 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe764018070 con 0x7fe76803a0a0 2024-04-24T22:59:13.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.855+0000 7fe7577fe640 1 -- 172.21.15.32:0/2898210307 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe764007500 con 0x7fe76803a0a0 2024-04-24T22:59:13.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.855+0000 7fe7577fe640 1 -- 172.21.15.32:0/2898210307 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7640049e0 con 0x7fe76803a0a0 2024-04-24T22:59:13.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.855+0000 7fe76ffff640 1 -- 172.21.15.32:0/2898210307 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe768178950 con 0x7fe76803a0a0 2024-04-24T22:59:13.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.855+0000 7fe76ffff640 1 -- 172.21.15.32:0/2898210307 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe768178f10 con 0x7fe76803a0a0 2024-04-24T22:59:13.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.856+0000 7f03a6ffd640 1 -- 172.21.15.32:0/2707028841 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ac037040 con 0x7f03b80cfd20 2024-04-24T22:59:13.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.856+0000 7f03a6ffd640 1 -- 172.21.15.32:0/2707028841 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f03ac044020 con 0x7f03b80cfd20 2024-04-24T22:59:13.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.856+0000 7f9a377fe640 1 -- 172.21.15.32:0/1504777339 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9a48005d10 con 0x7f9a5014e240 2024-04-24T22:59:13.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.856+0000 7f9a377fe640 1 --2- 172.21.15.32:0/1504777339 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9a3c06beb0 0x7f9a3c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.856+0000 7f9a4effd640 1 --2- 172.21.15.32:0/1504777339 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9a3c06beb0 0x7f9a3c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.857+0000 7fb235297640 1 --2- 172.21.15.32:0/3627226877 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb21406bf00 0x7fb21406e3c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb2300b05a0 tx=0x7fb220009110 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.857+0000 7fe76ffff640 1 -- 172.21.15.32:0/2898210307 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe750005290 con 0x7fe76803a0a0 2024-04-24T22:59:13.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.857+0000 7f818656a640 1 -- 172.21.15.32:0/3577223218 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8178008a00 con 0x7f8178038730 2024-04-24T22:59:13.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.857+0000 7f03a6ffd640 1 --2- 172.21.15.32:0/2707028841 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f039806bf00 0x7f039806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.857+0000 7f9a55610640 1 -- 172.21.15.32:0/1504777339 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a500682b0 con 0x7f9a5014e240 2024-04-24T22:59:13.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.857+0000 7f03b5d9b640 1 --2- 172.21.15.32:0/2707028841 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f039806bf00 0x7f039806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.857+0000 7f9a4effd640 1 --2- 172.21.15.32:0/1504777339 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9a3c06beb0 0x7f9a3c06e370 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9a501565d0 tx=0x7f9a40001d10 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.857+0000 7f03a6ffd640 1 -- 172.21.15.32:0/2707028841 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f03ac0ab6a0 con 0x7f03b80cfd20 2024-04-24T22:59:13.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.858+0000 7f4d07370640 1 -- 172.21.15.32:0/934933624 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d00068430 con 0x7f4d000cd160 2024-04-24T22:59:13.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.858+0000 7f03b5d9b640 1 --2- 172.21.15.32:0/2707028841 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f039806bf00 0x7f039806e3c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f03b80cc6a0 tx=0x7f03b0004f60 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.858+0000 7f4cf5ffb640 1 -- 172.21.15.32:0/934933624 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4cfc01a070 con 0x7f4d000cd160 2024-04-24T22:59:13.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.858+0000 7f4cf5ffb640 1 --2- 172.21.15.32:0/934933624 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4ce406be30 0x7f4ce406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.858+0000 7f4cf5ffb640 1 -- 172.21.15.32:0/934933624 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f4cfc08a9b0 con 0x7f4d000cd160 2024-04-24T22:59:13.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.858+0000 7f4d0490b640 1 --2- 172.21.15.32:0/934933624 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4ce406be30 0x7f4ce406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.859+0000 7f4d0490b640 1 --2- 172.21.15.32:0/934933624 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4ce406be30 0x7f4ce406e2f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4d00205790 tx=0x7f4cf0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.860+0000 7ff4967fc640 1 -- 172.21.15.32:0/2397776407 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff49007a1d0 con 0x7ff4980df410 2024-04-24T22:59:13.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.860+0000 7f9a377fe640 1 -- 172.21.15.32:0/1504777339 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f9a480774e0 con 0x7f9a5014e240 2024-04-24T22:59:13.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.861+0000 7fb2267fc640 1 -- 172.21.15.32:0/3627226877 <== mon.2 v2:172.21.15.121:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb22c042070 con 0x7fb2300b6e00 2024-04-24T22:59:13.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.863+0000 7f4c62ffd640 1 -- 172.21.15.32:0/1486441939 <== mon.2 v2:172.21.15.121:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4c5c018020 con 0x7f4c64054130 2024-04-24T22:59:13.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.863+0000 7f4c62ffd640 1 --2- 172.21.15.32:0/1486441939 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4c5006be80 0x7f4c5006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.863+0000 7fe7577fe640 1 -- 172.21.15.32:0/2898210307 <== mon.2 v2:172.21.15.121:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe76401b070 con 0x7fe76803a0a0 2024-04-24T22:59:13.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.863+0000 7f4c62ffd640 1 -- 172.21.15.32:0/1486441939 <== mon.2 v2:172.21.15.121:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f4c5c08b0b0 con 0x7f4c64054130 2024-04-24T22:59:13.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.863+0000 7f4c7237f640 1 --2- 172.21.15.32:0/1486441939 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4c5006be80 0x7f4c5006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.863+0000 7f817cff9640 1 -- 172.21.15.32:0/3577223218 <== mon.2 v2:172.21.15.121:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f81800b2070 con 0x7f8178038730 2024-04-24T22:59:13.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.864+0000 7fe7577fe640 1 --2- 172.21.15.32:0/2898210307 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe75c06be80 0x7fe75c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.864+0000 7f4c7237f640 1 --2- 172.21.15.32:0/1486441939 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4c5006be80 0x7f4c5006e340 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4c68009630 tx=0x7f4c6802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.864+0000 7fe7577fe640 1 -- 172.21.15.32:0/2898210307 <== mon.2 v2:172.21.15.121:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe76408a670 con 0x7fe76803a0a0 2024-04-24T22:59:13.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.864+0000 7f817cff9640 1 --2- 172.21.15.32:0/3577223218 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f816406bf30 0x7f816406e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:13.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.864+0000 7fe76effd640 1 --2- 172.21.15.32:0/2898210307 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe75c06be80 0x7fe75c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.864+0000 7f817cff9640 1 -- 172.21.15.32:0/3577223218 <== mon.2 v2:172.21.15.121:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8180115b00 con 0x7f8178038730 2024-04-24T22:59:13.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.864+0000 7f817ffff640 1 --2- 172.21.15.32:0/3577223218 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f816406bf30 0x7f816406e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:13.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.864+0000 7fe76effd640 1 --2- 172.21.15.32:0/2898210307 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe75c06be80 0x7fe75c06e340 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe760037000 tx=0x7fe760032000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.865+0000 7f817ffff640 1 --2- 172.21.15.32:0/3577223218 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f816406bf30 0x7f816406e3f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f81780399a0 tx=0x7f8174009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:13.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.866+0000 7f9a377fe640 1 -- 172.21.15.32:0/1504777339 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9a48079440 con 0x7f9a5014e240 2024-04-24T22:59:13.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.868+0000 7f03a6ffd640 1 -- 172.21.15.32:0/2707028841 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f03ac04b030 con 0x7f03b80cfd20 2024-04-24T22:59:13.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.872+0000 7f4c62ffd640 1 -- 172.21.15.32:0/1486441939 <== mon.2 v2:172.21.15.121:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4c5c01f020 con 0x7f4c64054130 2024-04-24T22:59:13.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.877+0000 7f4cf5ffb640 1 -- 172.21.15.32:0/934933624 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4cfc058da0 con 0x7f4d000cd160 2024-04-24T22:59:13.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.879+0000 7fe7577fe640 1 -- 172.21.15.32:0/2898210307 <== mon.2 v2:172.21.15.121:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe7640589b0 con 0x7fe76803a0a0 2024-04-24T22:59:13.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:13.887+0000 7f817cff9640 1 -- 172.21.15.32:0/3577223218 <== mon.2 v2:172.21.15.121:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f81800ba080 con 0x7f8178038730 2024-04-24T22:59:14.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.264+0000 7fb237cfc640 1 -- 172.21.15.32:0/3627226877 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fb230150a20 con 0x7fb2300b6e00 2024-04-24T22:59:14.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.265+0000 7fb2267fc640 1 -- 172.21.15.32:0/3627226877 <== mon.2 v2:172.21.15.121:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb22c07e8b0 con 0x7fb2300b6e00 2024-04-24T22:59:14.266 INFO:teuthology.orchestra.run.smithi032.stdout:55834574853 2024-04-24T22:59:14.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.268+0000 7fb213fff640 1 -- 172.21.15.32:0/3627226877 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb21406bf00 msgr2=0x7fb21406e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:14.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.268+0000 7fb213fff640 1 --2- 172.21.15.32:0/3627226877 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb21406bf00 0x7fb21406e3c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb2300b05a0 tx=0x7fb220009110 comp rx=0 tx=0).stop 2024-04-24T22:59:14.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.268+0000 7fb213fff640 1 -- 172.21.15.32:0/3627226877 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb2300b6e00 msgr2=0x7fb2301f0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:14.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.268+0000 7fb213fff640 1 --2- 172.21.15.32:0/3627226877 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb2300b6e00 0x7fb2301f0c40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb22c00b2c0 tx=0x7fb22c0046c0 comp rx=0 tx=0).stop 2024-04-24T22:59:14.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.268+0000 7fb213fff640 1 -- 172.21.15.32:0/3627226877 shutdown_connections 2024-04-24T22:59:14.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.269+0000 7fb213fff640 1 --2- 172.21.15.32:0/3627226877 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb21406bf00 0x7fb21406e3c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:14.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.269+0000 7fb213fff640 1 --2- 172.21.15.32:0/3627226877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb23015b510 0x7fb2300ae570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:14.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.269+0000 7fb213fff640 1 --2- 172.21.15.32:0/3627226877 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2300cd3c0 0x7fb23009a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:14.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.269+0000 7fb213fff640 1 --2- 172.21.15.32:0/3627226877 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb2300b6e00 0x7fb2301f0c40 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:14.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.269+0000 7fb213fff640 1 -- 172.21.15.32:0/3627226877 >> 172.21.15.32:0/3627226877 conn(0x7fb2300b6030 msgr2=0x7fb23015d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:14.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.269+0000 7fb213fff640 1 -- 172.21.15.32:0/3627226877 shutdown_connections 2024-04-24T22:59:14.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.269+0000 7fb213fff640 1 -- 172.21.15.32:0/3627226877 wait complete. 2024-04-24T22:59:14.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.272+0000 7ff4a7d22640 1 -- 172.21.15.32:0/2397776407 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7ff488005580 con 0x7ff4980df410 2024-04-24T22:59:14.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.273+0000 7ff4967fc640 1 -- 172.21.15.32:0/2397776407 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff490079ad0 con 0x7ff4980df410 2024-04-24T22:59:14.275 INFO:teuthology.orchestra.run.smithi032.stdout:55834574853 2024-04-24T22:59:14.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.277+0000 7ff483fff640 1 -- 172.21.15.32:0/2397776407 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff48406bf00 msgr2=0x7ff48406e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:14.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.277+0000 7ff483fff640 1 --2- 172.21.15.32:0/2397776407 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff48406bf00 0x7ff48406e3c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff49c006fd0 tx=0x7ff49c008040 comp rx=0 tx=0).stop 2024-04-24T22:59:14.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.278+0000 7ff483fff640 1 -- 172.21.15.32:0/2397776407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4980df410 msgr2=0x7ff498038710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:14.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.278+0000 7ff483fff640 1 --2- 172.21.15.32:0/2397776407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4980df410 0x7ff498038710 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff49800bad0 tx=0x7ff4900046c0 comp rx=0 tx=0).stop 2024-04-24T22:59:14.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.278+0000 7ff483fff640 1 -- 172.21.15.32:0/2397776407 shutdown_connections 2024-04-24T22:59:14.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.278+0000 7ff483fff640 1 --2- 172.21.15.32:0/2397776407 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff48406bf00 0x7ff48406e3c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:14.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.278+0000 7ff483fff640 1 --2- 172.21.15.32:0/2397776407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4980df410 0x7ff498038710 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:14.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.278+0000 7ff483fff640 1 --2- 172.21.15.32:0/2397776407 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4980df030 0x7ff4980d5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:14.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.278+0000 7ff483fff640 1 --2- 172.21.15.32:0/2397776407 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff498050140 0x7ff4980302c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:14.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.278+0000 7ff483fff640 1 -- 172.21.15.32:0/2397776407 >> 172.21.15.32:0/2397776407 conn(0x7ff498009fe0 msgr2=0x7ff4980ddfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:14.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.279+0000 7ff483fff640 1 -- 172.21.15.32:0/2397776407 shutdown_connections 2024-04-24T22:59:14.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.279+0000 7ff483fff640 1 -- 172.21.15.32:0/2397776407 wait complete. 2024-04-24T22:59:14.291 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.1 2024-04-24T22:59:14.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.288+0000 7f9a55610640 1 -- 172.21.15.32:0/1504777339 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f9a500b42d0 con 0x7f9a5014e240 2024-04-24T22:59:14.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.288+0000 7f9a377fe640 1 -- 172.21.15.32:0/1504777339 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9a4807d740 con 0x7f9a5014e240 2024-04-24T22:59:14.292 INFO:teuthology.orchestra.run.smithi032.stdout:55834574853 2024-04-24T22:59:14.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.289+0000 7f03bc91d640 1 -- 172.21.15.32:0/2707028841 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f039c005580 con 0x7f03b80cfd20 2024-04-24T22:59:14.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.290+0000 7f03a6ffd640 1 -- 172.21.15.32:0/2707028841 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f03ac07a0f0 con 0x7f03b80cfd20 2024-04-24T22:59:14.293 INFO:teuthology.orchestra.run.smithi032.stdout:55834574853 2024-04-24T22:59:14.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.291+0000 7f9a55610640 1 -- 172.21.15.32:0/1504777339 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9a3c06beb0 msgr2=0x7f9a3c06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:14.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.291+0000 7f9a55610640 1 --2- 172.21.15.32:0/1504777339 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9a3c06beb0 0x7f9a3c06e370 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9a501565d0 tx=0x7f9a40001d10 comp rx=0 tx=0).stop 2024-04-24T22:59:14.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.291+0000 7f9a55610640 1 -- 172.21.15.32:0/1504777339 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a5014e240 msgr2=0x7f9a50071af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:14.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.291+0000 7f9a55610640 1 --2- 172.21.15.32:0/1504777339 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a5014e240 0x7f9a50071af0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9a4802ea20 tx=0x7f9a48004ab0 comp rx=0 tx=0).stop 2024-04-24T22:59:14.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.291+0000 7f9a55610640 1 -- 172.21.15.32:0/1504777339 shutdown_connections 2024-04-24T22:59:14.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.291+0000 7f9a55610640 1 --2- 172.21.15.32:0/1504777339 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9a3c06beb0 0x7f9a3c06e370 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:14.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.291+0000 7f9a55610640 1 --2- 172.21.15.32:0/1504777339 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a5014e240 0x7f9a50071af0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:14.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.291+0000 7f9a55610640 1 --2- 172.21.15.32:0/1504777339 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9a501f8b20 0x7f9a50072030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:14.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.291+0000 7f9a55610640 1 --2- 172.21.15.32:0/1504777339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a5014de60 0x7f9a5015a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:14.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.291+0000 7f9a55610640 1 -- 172.21.15.32:0/1504777339 >> 172.21.15.32:0/1504777339 conn(0x7f9a500743f0 msgr2=0x7f9a50156b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:14.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.291+0000 7f9a55610640 1 -- 172.21.15.32:0/1504777339 shutdown_connections 2024-04-24T22:59:14.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.291+0000 7f9a55610640 1 -- 172.21.15.32:0/1504777339 wait complete. 2024-04-24T22:59:14.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.294+0000 7f03bc91d640 1 -- 172.21.15.32:0/2707028841 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f039806bf00 msgr2=0x7f039806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:14.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.294+0000 7f03bc91d640 1 --2- 172.21.15.32:0/2707028841 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f039806bf00 0x7f039806e3c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f03b80cc6a0 tx=0x7f03b0004f60 comp rx=0 tx=0).stop 2024-04-24T22:59:14.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.294+0000 7f03bc91d640 1 -- 172.21.15.32:0/2707028841 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03b80cfd20 msgr2=0x7f03b815c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:14.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.294+0000 7f03bc91d640 1 --2- 172.21.15.32:0/2707028841 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03b80cfd20 0x7f03b815c8a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f03ac000c00 tx=0x7f03ac0046c0 comp rx=0 tx=0).stop 2024-04-24T22:59:14.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.294+0000 7f03bc91d640 1 -- 172.21.15.32:0/2707028841 shutdown_connections 2024-04-24T22:59:14.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.294+0000 7f03bc91d640 1 --2- 172.21.15.32:0/2707028841 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f039806bf00 0x7f039806e3c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:14.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.294+0000 7f03bc91d640 1 --2- 172.21.15.32:0/2707028841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03b81628d0 0x7f03b81e95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:14.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.294+0000 7f03bc91d640 1 --2- 172.21.15.32:0/2707028841 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f03b815e5e0 0x7f03b81e9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:14.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.294+0000 7f03bc91d640 1 --2- 172.21.15.32:0/2707028841 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03b80cfd20 0x7f03b815c8a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:14.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.294+0000 7f03bc91d640 1 -- 172.21.15.32:0/2707028841 >> 172.21.15.32:0/2707028841 conn(0x7f03b80cd3f0 msgr2=0x7f03b806d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:14.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.294+0000 7f03bc91d640 1 -- 172.21.15.32:0/2707028841 shutdown_connections 2024-04-24T22:59:14.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.294+0000 7f03bc91d640 1 -- 172.21.15.32:0/2707028841 wait complete. 2024-04-24T22:59:14.298 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.6 2024-04-24T22:59:14.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.300+0000 7fe76ffff640 1 -- 172.21.15.32:0/2898210307 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fe750005850 con 0x7fe76803a0a0 2024-04-24T22:59:14.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.301+0000 7fe7577fe640 1 -- 172.21.15.32:0/2898210307 <== mon.2 v2:172.21.15.121:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe76405ccb0 con 0x7fe76803a0a0 2024-04-24T22:59:14.302 INFO:teuthology.orchestra.run.smithi032.stdout:55834574853 2024-04-24T22:59:14.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.303+0000 7fe76ffff640 1 -- 172.21.15.32:0/2898210307 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe75c06be80 msgr2=0x7fe75c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:14.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.303+0000 7fe76ffff640 1 --2- 172.21.15.32:0/2898210307 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe75c06be80 0x7fe75c06e340 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe760037000 tx=0x7fe760032000 comp rx=0 tx=0).stop 2024-04-24T22:59:14.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.303+0000 7fe76ffff640 1 -- 172.21.15.32:0/2898210307 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe76803a0a0 msgr2=0x7fe7680dd8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:14.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.303+0000 7fe76ffff640 1 --2- 172.21.15.32:0/2898210307 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe76803a0a0 0x7fe7680dd8a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe76400c500 tx=0x7fe76400c9c0 comp rx=0 tx=0).stop 2024-04-24T22:59:14.306 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.2 2024-04-24T22:59:14.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.304+0000 7fe76ffff640 1 -- 172.21.15.32:0/2898210307 shutdown_connections 2024-04-24T22:59:14.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.304+0000 7fe76ffff640 1 --2- 172.21.15.32:0/2898210307 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe75c06be80 0x7fe75c06e340 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:14.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.304+0000 7fe76ffff640 1 --2- 172.21.15.32:0/2898210307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe7680df800 0x7fe768167d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:14.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.304+0000 7fe76ffff640 1 --2- 172.21.15.32:0/2898210307 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe76803a0a0 0x7fe7680dd8a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:14.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.304+0000 7fe76ffff640 1 --2- 172.21.15.32:0/2898210307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7680366a0 0x7fe768178070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:14.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.304+0000 7fe76ffff640 1 -- 172.21.15.32:0/2898210307 >> 172.21.15.32:0/2898210307 conn(0x7fe76801e6c0 msgr2=0x7fe7680de680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:14.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.304+0000 7fe76ffff640 1 -- 172.21.15.32:0/2898210307 shutdown_connections 2024-04-24T22:59:14.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.304+0000 7fe76ffff640 1 -- 172.21.15.32:0/2898210307 wait complete. 2024-04-24T22:59:14.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.304+0000 7f4c73381640 1 -- 172.21.15.32:0/1486441939 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f4c6403e3d0 con 0x7f4c64054130 2024-04-24T22:59:14.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.305+0000 7f4d07370640 1 -- 172.21.15.32:0/934933624 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f4d000b3f30 con 0x7f4d000cd160 2024-04-24T22:59:14.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.305+0000 7f4c62ffd640 1 -- 172.21.15.32:0/1486441939 <== mon.2 v2:172.21.15.121:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4c5c059bd0 con 0x7f4c64054130 2024-04-24T22:59:14.308 INFO:teuthology.orchestra.run.smithi032.stdout:55834574853 2024-04-24T22:59:14.309 INFO:teuthology.orchestra.run.smithi032.stdout:55834574853 2024-04-24T22:59:14.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.305+0000 7f4cf5ffb640 1 -- 172.21.15.32:0/934933624 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4cfc05d0a0 con 0x7f4d000cd160 2024-04-24T22:59:14.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.307+0000 7f4c73381640 1 -- 172.21.15.32:0/1486441939 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4c5006be80 msgr2=0x7f4c5006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:14.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.307+0000 7f4c73381640 1 --2- 172.21.15.32:0/1486441939 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4c5006be80 0x7f4c5006e340 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4c68009630 tx=0x7f4c6802e000 comp rx=0 tx=0).stop 2024-04-24T22:59:14.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.307+0000 7f4c73381640 1 -- 172.21.15.32:0/1486441939 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4c64054130 msgr2=0x7f4c640525b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:14.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.307+0000 7f4c73381640 1 --2- 172.21.15.32:0/1486441939 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4c64054130 0x7f4c640525b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c00b410 tx=0x7f4c5c00b8d0 comp rx=0 tx=0).stop 2024-04-24T22:59:14.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.307+0000 7f4c73381640 1 -- 172.21.15.32:0/1486441939 shutdown_connections 2024-04-24T22:59:14.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.307+0000 7f4c73381640 1 --2- 172.21.15.32:0/1486441939 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4c5006be80 0x7f4c5006e340 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:14.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.307+0000 7f4c73381640 1 --2- 172.21.15.32:0/1486441939 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4c640e30c0 0x7f4c640e5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:14.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.307+0000 7f4c73381640 1 --2- 172.21.15.32:0/1486441939 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4c64054130 0x7f4c640525b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:14.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.307+0000 7f4c73381640 1 --2- 172.21.15.32:0/1486441939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c6403e840 0x7f4c640d7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:14.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.307+0000 7f4c73381640 1 -- 172.21.15.32:0/1486441939 >> 172.21.15.32:0/1486441939 conn(0x7f4c640559e0 msgr2=0x7f4c640e7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:14.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.308+0000 7f4c73381640 1 -- 172.21.15.32:0/1486441939 shutdown_connections 2024-04-24T22:59:14.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.308+0000 7f4c73381640 1 -- 172.21.15.32:0/1486441939 wait complete. 2024-04-24T22:59:14.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.308+0000 7f4d07370640 1 -- 172.21.15.32:0/934933624 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4ce406be30 msgr2=0x7f4ce406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:14.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.308+0000 7f4d07370640 1 --2- 172.21.15.32:0/934933624 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4ce406be30 0x7f4ce406e2f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4d00205790 tx=0x7f4cf0009040 comp rx=0 tx=0).stop 2024-04-24T22:59:14.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.308+0000 7f4d07370640 1 -- 172.21.15.32:0/934933624 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d000cd160 msgr2=0x7f4d000b0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:14.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.308+0000 7f4d07370640 1 --2- 172.21.15.32:0/934933624 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d000cd160 0x7f4d000b0b30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc00cbc0 tx=0x7f4cfc00a060 comp rx=0 tx=0).stop 2024-04-24T22:59:14.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.308+0000 7f4d07370640 1 -- 172.21.15.32:0/934933624 shutdown_connections 2024-04-24T22:59:14.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.308+0000 7f4d07370640 1 --2- 172.21.15.32:0/934933624 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4ce406be30 0x7f4ce406e2f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:14.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.308+0000 7f4d07370640 1 --2- 172.21.15.32:0/934933624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d00154e20 0x7f4d00200a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:14.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.309+0000 7f4d07370640 1 --2- 172.21.15.32:0/934933624 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4d000cf840 0x7f4d00200540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:14.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.309+0000 7f4d07370640 1 --2- 172.21.15.32:0/934933624 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d000cd160 0x7f4d000b0b30 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:14.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.309+0000 7f4d07370640 1 -- 172.21.15.32:0/934933624 >> 172.21.15.32:0/934933624 conn(0x7f4d0006b110 msgr2=0x7f4d000d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:14.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.309+0000 7f4d07370640 1 -- 172.21.15.32:0/934933624 shutdown_connections 2024-04-24T22:59:14.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.309+0000 7f4d07370640 1 -- 172.21.15.32:0/934933624 wait complete. 2024-04-24T22:59:14.313 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.4 2024-04-24T22:59:14.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.312+0000 7f818656a640 1 -- 172.21.15.32:0/3577223218 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f81780053c0 con 0x7f8178038730 2024-04-24T22:59:14.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.313+0000 7f817cff9640 1 -- 172.21.15.32:0/3577223218 <== mon.2 v2:172.21.15.121:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8180070040 con 0x7f8178038730 2024-04-24T22:59:14.314 INFO:teuthology.orchestra.run.smithi032.stdout:55834574853 2024-04-24T22:59:14.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.315+0000 7f818656a640 1 -- 172.21.15.32:0/3577223218 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f816406bf30 msgr2=0x7f816406e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:14.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.315+0000 7f818656a640 1 --2- 172.21.15.32:0/3577223218 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f816406bf30 0x7f816406e3f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f81780399a0 tx=0x7f8174009040 comp rx=0 tx=0).stop 2024-04-24T22:59:14.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.315+0000 7f818656a640 1 -- 172.21.15.32:0/3577223218 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8178038730 msgr2=0x7f8178038bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:14.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.315+0000 7f818656a640 1 --2- 172.21.15.32:0/3577223218 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8178038730 0x7f8178038bb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8180050770 tx=0x7f8180069030 comp rx=0 tx=0).stop 2024-04-24T22:59:14.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.316+0000 7f818656a640 1 -- 172.21.15.32:0/3577223218 shutdown_connections 2024-04-24T22:59:14.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.316+0000 7f818656a640 1 --2- 172.21.15.32:0/3577223218 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f816406bf30 0x7f816406e3f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:14.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.316+0000 7f818656a640 1 --2- 172.21.15.32:0/3577223218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8178107920 0x7f81780fa0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:14.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.316+0000 7f818656a640 1 --2- 172.21.15.32:0/3577223218 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8178038730 0x7f8178038bb0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:14.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.316+0000 7f818656a640 1 --2- 172.21.15.32:0/3577223218 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8178104970 0x7f817804e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:14.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.316+0000 7f818656a640 1 -- 172.21.15.32:0/3577223218 >> 172.21.15.32:0/3577223218 conn(0x7f8178009fe0 msgr2=0x7f817803a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:14.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.316+0000 7f818656a640 1 -- 172.21.15.32:0/3577223218 shutdown_connections 2024-04-24T22:59:14.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:14.316+0000 7f818656a640 1 -- 172.21.15.32:0/3577223218 wait complete. 2024-04-24T22:59:14.323 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.7 2024-04-24T22:59:14.324 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.5 2024-04-24T22:59:14.325 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.0 2024-04-24T22:59:14.329 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.3 2024-04-24T22:59:15.292 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-24T22:59:15.299 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-24T22:59:15.306 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-24T22:59:15.315 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-24T22:59:15.323 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-24T22:59:15.325 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-24T22:59:15.326 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-24T22:59:15.330 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-24T22:59:15.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.461+0000 7fbf897e7640 1 Processor -- start 2024-04-24T22:59:15.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.461+0000 7fbf897e7640 1 -- start start 2024-04-24T22:59:15.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.462+0000 7fbf897e7640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fbf7c0e7690 0x7fbf7c052c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:15.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.462+0000 7fbf897e7640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbf7c038e60 0x7fbf7c007be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:15.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.462+0000 7fbf897e7640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf7c051bc0 con 0x7fbf7c04a3e0 2024-04-24T22:59:15.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.462+0000 7fbf897e7640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fbf7c0527d0 con 0x7fbf7c038e60 2024-04-24T22:59:15.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.462+0000 7fbf897e7640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fbf7c051ea0 con 0x7fbf7c0e7690 2024-04-24T22:59:15.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.463+0000 7fbf837fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbf7c038e60 0x7fbf7c007be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:15.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.463+0000 7fbf837fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbf7c038e60 0x7fbf7c007be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38656/0 (socket says 172.21.15.32:38656) 2024-04-24T22:59:15.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.463+0000 7fbf837fe640 1 -- 172.21.15.32:0/1977650762 learned_addr learned my addr 172.21.15.32:0/1977650762 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:15.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.463+0000 7fbf837fe640 1 -- 172.21.15.32:0/1977650762 >> v2:172.21.15.121:3300/0 conn(0x7fbf7c0e7690 msgr2=0x7fbf7c052c60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T22:59:15.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.463+0000 7fbf837fe640 1 --2- 172.21.15.32:0/1977650762 >> v2:172.21.15.121:3300/0 conn(0x7fbf7c0e7690 0x7fbf7c052c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.463+0000 7fbf837fe640 1 -- 172.21.15.32:0/1977650762 >> v1:172.21.15.32:6789/0 conn(0x7fbf7c04a3e0 legacy=0x7fbf7c038750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T22:59:15.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.463+0000 7fbf837fe640 1 -- 172.21.15.32:0/1977650762 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf7c052180 con 0x7fbf7c038e60 2024-04-24T22:59:15.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.465+0000 7fbf837fe640 1 --2- 172.21.15.32:0/1977650762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf7c038e60 0x7fbf7c007be0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c052950 tx=0x7fbf840a62d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa4fea80401d5d8a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:15.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.465+0000 7fbf81ffb640 1 -- 172.21.15.32:0/1977650762 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf840b3070 con 0x7fbf7c038e60 2024-04-24T22:59:15.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.465+0000 7fbf81ffb640 1 -- 172.21.15.32:0/1977650762 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf840ae440 con 0x7fbf7c038e60 2024-04-24T22:59:15.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.466+0000 7fbf897e7640 1 -- 172.21.15.32:0/1977650762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf7c038e60 msgr2=0x7fbf7c007be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:15.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.466+0000 7fbf897e7640 1 --2- 172.21.15.32:0/1977650762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf7c038e60 0x7fbf7c007be0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c052950 tx=0x7fbf840a62d0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.467+0000 7fbf897e7640 1 -- 172.21.15.32:0/1977650762 shutdown_connections 2024-04-24T22:59:15.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.467+0000 7fbf897e7640 1 --2- 172.21.15.32:0/1977650762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf7c038e60 0x7fbf7c007be0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.467+0000 7fbf897e7640 1 --2- 172.21.15.32:0/1977650762 >> v2:172.21.15.121:3300/0 conn(0x7fbf7c0e7690 0x7fbf7c052c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.467+0000 7fbf897e7640 1 -- 172.21.15.32:0/1977650762 >> 172.21.15.32:0/1977650762 conn(0x7fbf7c053750 msgr2=0x7fbf7c053b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:15.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.467+0000 7fbf897e7640 1 -- 172.21.15.32:0/1977650762 shutdown_connections 2024-04-24T22:59:15.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.467+0000 7fbf897e7640 1 -- 172.21.15.32:0/1977650762 wait complete. 2024-04-24T22:59:15.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.469+0000 7fbf897e7640 1 Processor -- start 2024-04-24T22:59:15.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.469+0000 7fbf897e7640 1 -- start start 2024-04-24T22:59:15.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.470+0000 7fbf897e7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf7c038e60 0x7fbf7c0d5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:15.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.470+0000 7fbf897e7640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf7c04a3e0 0x7fbf7c0399f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:15.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.470+0000 7fbf897e7640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf7c0e7690 0x7fbf7c039f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:15.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.470+0000 7fbf897e7640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbf7c0531a0 con 0x7fbf7c038e60 2024-04-24T22:59:15.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.470+0000 7fbf897e7640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fbf7c055bf0 con 0x7fbf7c04a3e0 2024-04-24T22:59:15.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.470+0000 7fbf897e7640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fbf7c0517e0 con 0x7fbf7c0e7690 2024-04-24T22:59:15.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.471+0000 7fbf837fe640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf7c0e7690 0x7fbf7c039f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:15.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.471+0000 7fbf837fe640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf7c0e7690 0x7fbf7c039f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.32:42172/0 (socket says 172.21.15.32:42172) 2024-04-24T22:59:15.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.471+0000 7fbf837fe640 1 -- 172.21.15.32:0/1453054098 learned_addr learned my addr 172.21.15.32:0/1453054098 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:15.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.471+0000 7fbf827fc640 1 --2- 172.21.15.32:0/1453054098 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf7c04a3e0 0x7fbf7c0399f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:15.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.471+0000 7fbf837fe640 1 -- 172.21.15.32:0/1453054098 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf7c04a3e0 msgr2=0x7fbf7c0399f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:15.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.471+0000 7fbf837fe640 1 --2- 172.21.15.32:0/1453054098 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf7c04a3e0 0x7fbf7c0399f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.471+0000 7fbf837fe640 1 -- 172.21.15.32:0/1453054098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf7c038e60 msgr2=0x7fbf7c0d5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:15.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.471+0000 7fbf82ffd640 1 --2- 172.21.15.32:0/1453054098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf7c038e60 0x7fbf7c0d5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:15.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.471+0000 7fbf837fe640 1 --2- 172.21.15.32:0/1453054098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf7c038e60 0x7fbf7c0d5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.471+0000 7fbf837fe640 1 -- 172.21.15.32:0/1453054098 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf7c03aa40 con 0x7fbf7c0e7690 2024-04-24T22:59:15.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.471+0000 7fbf82ffd640 1 --2- 172.21.15.32:0/1453054098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf7c038e60 0x7fbf7c0d5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:59:15.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.472+0000 7fbf837fe640 1 --2- 172.21.15.32:0/1453054098 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf7c0e7690 0x7fbf7c039f30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbf8402d700 tx=0x7fbf84069030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:15.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.472+0000 7fbf737fe640 1 -- 172.21.15.32:0/1453054098 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf840b3040 con 0x7fbf7c0e7690 2024-04-24T22:59:15.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.472+0000 7fbf897e7640 1 -- 172.21.15.32:0/1453054098 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf7c03ad20 con 0x7fbf7c0e7690 2024-04-24T22:59:15.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.472+0000 7fbf897e7640 1 -- 172.21.15.32:0/1453054098 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf7c03a470 con 0x7fbf7c0e7690 2024-04-24T22:59:15.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.472+0000 7fbf737fe640 1 -- 172.21.15.32:0/1453054098 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf840b9470 con 0x7fbf7c0e7690 2024-04-24T22:59:15.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.472+0000 7fbf737fe640 1 -- 172.21.15.32:0/1453054098 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf840b9820 con 0x7fbf7c0e7690 2024-04-24T22:59:15.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.474+0000 7fbf737fe640 1 -- 172.21.15.32:0/1453054098 <== mon.2 v2:172.21.15.121:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbf840ae680 con 0x7fbf7c0e7690 2024-04-24T22:59:15.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.475+0000 7fbf737fe640 1 --2- 172.21.15.32:0/1453054098 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fbf6806bf00 0x7fbf6806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:15.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.475+0000 7fbf737fe640 1 -- 172.21.15.32:0/1453054098 <== mon.2 v2:172.21.15.121:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fbf84116ba0 con 0x7fbf7c0e7690 2024-04-24T22:59:15.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.475+0000 7fbf82ffd640 1 --2- 172.21.15.32:0/1453054098 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fbf6806bf00 0x7fbf6806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:15.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.476+0000 7fbf82ffd640 1 --2- 172.21.15.32:0/1453054098 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fbf6806bf00 0x7fbf6806e3c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c038e30 tx=0x7fbf78006cc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:15.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.480+0000 7fbf897e7640 1 -- 172.21.15.32:0/1453054098 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf60005290 con 0x7fbf7c0e7690 2024-04-24T22:59:15.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.485+0000 7fbf737fe640 1 -- 172.21.15.32:0/1453054098 <== mon.2 v2:172.21.15.121:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbf84089020 con 0x7fbf7c0e7690 2024-04-24T22:59:15.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.514+0000 7f9fcf30e640 1 Processor -- start 2024-04-24T22:59:15.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.514+0000 7f9fcf30e640 1 -- start start 2024-04-24T22:59:15.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.516+0000 7f9fcf30e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9fc80b3230 0x7f9fc816b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:15.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.516+0000 7f9fcf30e640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9fc80cd550 0x7f9fc80b5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:15.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.516+0000 7f8163fff640 1 Processor -- start 2024-04-24T22:59:15.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.516+0000 7f8163fff640 1 -- start start 2024-04-24T22:59:15.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.517+0000 7f8163fff640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f815c053a90 0x7f815c0502a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:15.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.517+0000 7f8163fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f815c053e70 0x7f815c054250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:15.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.517+0000 7f8163fff640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f815c050d50 con 0x7f815c054980 2024-04-24T22:59:15.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.517+0000 7f8163fff640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f815c009500 con 0x7f815c053e70 2024-04-24T22:59:15.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.517+0000 7f8163fff640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f815c050ff0 con 0x7f815c053a90 2024-04-24T22:59:15.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.516+0000 7f1e23ba2640 1 Processor -- start 2024-04-24T22:59:15.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.516+0000 7f1e23ba2640 1 -- start start 2024-04-24T22:59:15.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.518+0000 7f81637fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f815c053e70 0x7f815c054250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:15.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.518+0000 7f1e23ba2640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1e140e3290 0x7f1e140586d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:15.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.518+0000 7f1e23ba2640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1e14053760 0x7f1e14053bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:15.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.518+0000 7f1e23ba2640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1e140e6cf0 0x7f1e140e7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:15.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.518+0000 7f1e23ba2640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f1e140e3670 con 0x7f1e140e3290 2024-04-24T22:59:15.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.518+0000 7f1e23ba2640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f1e1403edf0 con 0x7f1e14053760 2024-04-24T22:59:15.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.518+0000 7f1e23ba2640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f1e1403e830 con 0x7f1e140e6cf0 2024-04-24T22:59:15.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.516+0000 7f9fcf30e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fc80b36c0 con 0x7f9fc80ce360 2024-04-24T22:59:15.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.518+0000 7f1e2193e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1e140e3290 0x7f1e140586d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:15.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.518+0000 7f1e2193e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1e140e3290 0x7f1e140586d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38682/0 (socket says 172.21.15.32:38682) 2024-04-24T22:59:15.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.518+0000 7f1e2193e640 1 -- 172.21.15.32:0/1478479445 learned_addr learned my addr 172.21.15.32:0/1478479445 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:15.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.518+0000 7f1e2113d640 1 --2- 172.21.15.32:0/1478479445 >> v2:172.21.15.121:3300/0 conn(0x7f1e14053760 0x7f1e14053bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:15.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.518+0000 7f1e2213f640 1 --2- 172.21.15.32:0/1478479445 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1e140e6cf0 0x7f1e140e7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:15.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.518+0000 7f81637fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f815c053e70 0x7f815c054250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38670/0 (socket says 172.21.15.32:38670) 2024-04-24T22:59:15.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.518+0000 7f81637fe640 1 -- 172.21.15.32:0/3984526181 learned_addr learned my addr 172.21.15.32:0/3984526181 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:15.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.518+0000 7f81637fe640 1 -- 172.21.15.32:0/3984526181 >> v2:172.21.15.121:3300/0 conn(0x7f815c053a90 msgr2=0x7f815c0502a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T22:59:15.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.518+0000 7f81637fe640 1 --2- 172.21.15.32:0/3984526181 >> v2:172.21.15.121:3300/0 conn(0x7f815c053a90 0x7f815c0502a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.518+0000 7f81637fe640 1 -- 172.21.15.32:0/3984526181 >> v1:172.21.15.32:6789/0 conn(0x7f815c054980 legacy=0x7f815c008df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:15.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.519+0000 7f81637fe640 1 -- 172.21.15.32:0/3984526181 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f815c0085f0 con 0x7f815c053e70 2024-04-24T22:59:15.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.516+0000 7f9fcf30e640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f9fc80b42d0 con 0x7f9fc80b3230 2024-04-24T22:59:15.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.516+0000 7f9fcf30e640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f9fc80b39a0 con 0x7f9fc80cd550 2024-04-24T22:59:15.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.516+0000 7f9fcdb0b640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9fc80ce360 0x7f9fc80b4760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47454/0 (socket says 172.21.15.32:47454) 2024-04-24T22:59:15.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.516+0000 7f9fcdb0b640 1 -- 172.21.15.32:0/1357234131 learned_addr learned my addr 172.21.15.32:0/1357234131 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:15.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.516+0000 7f9fceb0d640 1 --2- 172.21.15.32:0/1357234131 >> v2:172.21.15.121:3300/0 conn(0x7f9fc80cd550 0x7f9fc80b5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:15.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.517+0000 7f9fceb0d640 1 -- 172.21.15.32:0/1357234131 >> v2:172.21.15.32:3300/0 conn(0x7f9fc80b3230 msgr2=0x7f9fc816b450 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T22:59:15.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.517+0000 7f9fceb0d640 1 --2- 172.21.15.32:0/1357234131 >> v2:172.21.15.32:3300/0 conn(0x7f9fc80b3230 0x7f9fc816b450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.517+0000 7f9fceb0d640 1 -- 172.21.15.32:0/1357234131 >> v1:172.21.15.32:6789/0 conn(0x7f9fc80ce360 legacy=0x7f9fc80b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:15.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.517+0000 7f9fceb0d640 1 -- 172.21.15.32:0/1357234131 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fc80b3c80 con 0x7f9fc80cd550 2024-04-24T22:59:15.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.519+0000 7f9fceb0d640 1 --2- 172.21.15.32:0/1357234131 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9fc80cd550 0x7f9fc80b5860 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9fc80b6440 tx=0x7f9fc002a750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=331ee38095593c61 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:15.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.519+0000 7f1e2193e640 1 -- 172.21.15.32:0/1478479445 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1e140e6cf0 msgr2=0x7f1e140e7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:15.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.519+0000 7f1e2193e640 1 --2- 172.21.15.32:0/1478479445 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1e140e6cf0 0x7f1e140e7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.518+0000 7f8e2f9f4640 1 Processor -- start 2024-04-24T22:59:15.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.519+0000 7f8e2f9f4640 1 -- start start 2024-04-24T22:59:15.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.519+0000 7f1e2193e640 1 -- 172.21.15.32:0/1478479445 >> v2:172.21.15.121:3300/0 conn(0x7f1e14053760 msgr2=0x7f1e14053bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:15.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.519+0000 7f1e2193e640 1 --2- 172.21.15.32:0/1478479445 >> v2:172.21.15.121:3300/0 conn(0x7f1e14053760 0x7f1e14053bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.519+0000 7f1e2193e640 1 -- 172.21.15.32:0/1478479445 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e14057ad0 con 0x7f1e140e3290 2024-04-24T22:59:15.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.520+0000 7f1e2113d640 1 --2- 172.21.15.32:0/1478479445 >> v2:172.21.15.121:3300/0 conn(0x7f1e14053760 0x7f1e14053bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:59:15.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.519+0000 7f9fcd30a640 1 -- 172.21.15.32:0/1357234131 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc0007740 con 0x7f9fc80cd550 2024-04-24T22:59:15.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.519+0000 7f9fcd30a640 1 -- 172.21.15.32:0/1357234131 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fc00078e0 con 0x7f9fc80cd550 2024-04-24T22:59:15.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.520+0000 7f06a082b640 1 Processor -- start 2024-04-24T22:59:15.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.520+0000 7f06a082b640 1 -- start start 2024-04-24T22:59:15.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.520+0000 7f1e2193e640 1 --2- 172.21.15.32:0/1478479445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e140e3290 0x7f1e140586d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c064c60 tx=0x7f1e1c0a5f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f198e94f15bd466 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:15.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.520+0000 7f81637fe640 1 --2- 172.21.15.32:0/3984526181 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f815c053e70 0x7f815c054250 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8164064c60 tx=0x7f81640aa4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=720c661dcfd89be7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:15.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.520+0000 7f9fcf30e640 1 -- 172.21.15.32:0/1357234131 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9fc80cd550 msgr2=0x7f9fc80b5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:15.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.520+0000 7f9fcf30e640 1 --2- 172.21.15.32:0/1357234131 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9fc80cd550 0x7f9fc80b5860 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9fc80b6440 tx=0x7f9fc002a750 comp rx=0 tx=0).stop 2024-04-24T22:59:15.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.520+0000 7f8e2f9f4640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8e280cd5c0 0x7f8e280b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:15.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.520+0000 7f8e2f9f4640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8e280b23e0 0x7f8e28070600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:15.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.520+0000 7f8e2f9f4640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e280b2800 con 0x7f8e2816b430 2024-04-24T22:59:15.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.520+0000 7f8e2f9f4640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f8e280b4830 con 0x7f8e280cd5c0 2024-04-24T22:59:15.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.520+0000 7f8e2f9f4640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f8e280cebb0 con 0x7f8e280b23e0 2024-04-24T22:59:15.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.520+0000 7f1e2213f640 1 --2- 172.21.15.32:0/1478479445 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1e140e6cf0 0x7f1e140e7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:15.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.520+0000 7f8e2cf8f640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8e280cd5c0 0x7f8e280b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:15.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.520+0000 7f8e2d790640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8e2816b430 0x7f8e280b3d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47464/0 (socket says 172.21.15.32:47464) 2024-04-24T22:59:15.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.520+0000 7f8e2d790640 1 -- 172.21.15.32:0/1909059908 learned_addr learned my addr 172.21.15.32:0/1909059908 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:15.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.520+0000 7f8e2df91640 1 --2- 172.21.15.32:0/1909059908 >> v2:172.21.15.121:3300/0 conn(0x7f8e280b23e0 0x7f8e28070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:15.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.520+0000 7f9fcf30e640 1 -- 172.21.15.32:0/1357234131 shutdown_connections 2024-04-24T22:59:15.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.520+0000 7f9fcf30e640 1 --2- 172.21.15.32:0/1357234131 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9fc80cd550 0x7f9fc80b5860 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.520+0000 7f9fcf30e640 1 --2- 172.21.15.32:0/1357234131 >> v2:172.21.15.32:3300/0 conn(0x7f9fc80b3230 0x7f9fc816b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.520+0000 7f9fcf30e640 1 -- 172.21.15.32:0/1357234131 >> 172.21.15.32:0/1357234131 conn(0x7f9fc80b1f30 msgr2=0x7f9fc80b2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:15.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.520+0000 7f3928934640 1 Processor -- start 2024-04-24T22:59:15.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.521+0000 7f3928934640 1 -- start start 2024-04-24T22:59:15.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.520+0000 7f54c0142640 1 Processor -- start 2024-04-24T22:59:15.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.521+0000 7f54c0142640 1 -- start start 2024-04-24T22:59:15.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.521+0000 7f8161ffb640 1 -- 172.21.15.32:0/3984526181 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816406f870 con 0x7f815c053e70 2024-04-24T22:59:15.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.521+0000 7f8161ffb640 1 -- 172.21.15.32:0/3984526181 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f816406fa50 con 0x7f815c053e70 2024-04-24T22:59:15.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.521+0000 7f8161ffb640 1 -- 172.21.15.32:0/3984526181 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81640b1400 con 0x7f815c053e70 2024-04-24T22:59:15.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.521+0000 7f9fcf30e640 1 -- 172.21.15.32:0/1357234131 shutdown_connections 2024-04-24T22:59:15.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.521+0000 7f9fcf30e640 1 -- 172.21.15.32:0/1357234131 wait complete. 2024-04-24T22:59:15.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.522+0000 7f8163fff640 1 -- 172.21.15.32:0/3984526181 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f815c053e70 msgr2=0x7f815c054250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:15.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.522+0000 7f8163fff640 1 --2- 172.21.15.32:0/3984526181 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f815c053e70 0x7f815c054250 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8164064c60 tx=0x7f81640aa4a0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.522+0000 7f8163fff640 1 -- 172.21.15.32:0/3984526181 shutdown_connections 2024-04-24T22:59:15.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.522+0000 7f8163fff640 1 --2- 172.21.15.32:0/3984526181 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f815c053e70 0x7f815c054250 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.522+0000 7f8163fff640 1 --2- 172.21.15.32:0/3984526181 >> v2:172.21.15.121:3300/0 conn(0x7f815c053a90 0x7f815c0502a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.522+0000 7f8163fff640 1 -- 172.21.15.32:0/3984526181 >> 172.21.15.32:0/3984526181 conn(0x7f815c055c50 msgr2=0x7f815c056060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:15.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.521+0000 7f1e2093c640 1 -- 172.21.15.32:0/1478479445 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c0b3070 con 0x7f1e140e3290 2024-04-24T22:59:15.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.521+0000 7f1e2093c640 1 -- 172.21.15.32:0/1478479445 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e1c0a6da0 con 0x7f1e140e3290 2024-04-24T22:59:15.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.521+0000 7f1e2093c640 1 -- 172.21.15.32:0/1478479445 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c0ae640 con 0x7f1e140e3290 2024-04-24T22:59:15.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.521+0000 7f06a082b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f069c15e700 0x7f069c0ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:15.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.521+0000 7f06a082b640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f069c0718a0 0x7f069c0cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:15.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.521+0000 7f06a082b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f069c0af830 con 0x7f069c0cde20 2024-04-24T22:59:15.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.521+0000 7f06a082b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f069c15eae0 con 0x7f069c15e700 2024-04-24T22:59:15.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.521+0000 7f06a082b640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f069c0aec20 con 0x7f069c0718a0 2024-04-24T22:59:15.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.522+0000 7f54c0142640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f54b815c690 0x7f54b80707c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:15.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.522+0000 7f54c0142640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f54b80b3f70 0x7f54b806ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:15.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.521+0000 7f8e2cf8f640 1 -- 172.21.15.32:0/1909059908 >> v2:172.21.15.121:3300/0 conn(0x7f8e280b23e0 msgr2=0x7f8e28070600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:15.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.521+0000 7f8e2cf8f640 1 --2- 172.21.15.32:0/1909059908 >> v2:172.21.15.121:3300/0 conn(0x7f8e280b23e0 0x7f8e28070600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.521+0000 7f8e2cf8f640 1 -- 172.21.15.32:0/1909059908 >> v1:172.21.15.32:6789/0 conn(0x7f8e2816b430 legacy=0x7f8e280b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:15.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.521+0000 7f8e2cf8f640 1 -- 172.21.15.32:0/1909059908 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e280cee90 con 0x7f8e280cd5c0 2024-04-24T22:59:15.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.522+0000 7f54c0142640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54b806deb0 con 0x7f54b806d350 2024-04-24T22:59:15.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.522+0000 7f54c0142640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f54b806e120 con 0x7f54b80b3f70 2024-04-24T22:59:15.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.522+0000 7f54c0142640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f54b806e400 con 0x7f54b815c690 2024-04-24T22:59:15.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.522+0000 7f1e23ba2640 1 -- 172.21.15.32:0/1478479445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e140e3290 msgr2=0x7f1e140586d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:15.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.522+0000 7f1e23ba2640 1 --2- 172.21.15.32:0/1478479445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e140e3290 0x7f1e140586d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c064c60 tx=0x7f1e1c0a5f50 comp rx=0 tx=0).stop 2024-04-24T22:59:15.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.522+0000 7f54bd6dd640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f54b80b3f70 0x7f54b806ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:15.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.522+0000 7f54be6df640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f54b806d350 0x7f54b8160060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:52276/0 (socket says 172.21.15.32:52276) 2024-04-24T22:59:15.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.522+0000 7f54be6df640 1 -- 172.21.15.32:0/1188720565 learned_addr learned my addr 172.21.15.32:0/1188720565 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:15.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.522+0000 7f8163fff640 1 -- 172.21.15.32:0/3984526181 shutdown_connections 2024-04-24T22:59:15.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.522+0000 7f8163fff640 1 -- 172.21.15.32:0/3984526181 wait complete. 2024-04-24T22:59:15.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.523+0000 7f1e23ba2640 1 -- 172.21.15.32:0/1478479445 shutdown_connections 2024-04-24T22:59:15.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.523+0000 7f1e23ba2640 1 --2- 172.21.15.32:0/1478479445 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1e140e6cf0 0x7f1e140e7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.523+0000 7f1e23ba2640 1 --2- 172.21.15.32:0/1478479445 >> v2:172.21.15.121:3300/0 conn(0x7f1e14053760 0x7f1e14053bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.523+0000 7f1e23ba2640 1 --2- 172.21.15.32:0/1478479445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e140e3290 0x7f1e140586d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.523+0000 7f1e23ba2640 1 -- 172.21.15.32:0/1478479445 >> 172.21.15.32:0/1478479445 conn(0x7f1e14055990 msgr2=0x7f1e14055d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:15.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.522+0000 7f069b59e640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f069c0718a0 0x7f069c0cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:15.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.522+0000 7f069b59e640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f069c0718a0 0x7f069c0cd320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.32:48954/0 (socket says 172.21.15.32:48954) 2024-04-24T22:59:15.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.522+0000 7f069b59e640 1 -- 172.21.15.32:0/2169363957 learned_addr learned my addr 172.21.15.32:0/2169363957 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:15.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.522+0000 7f392259c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f392415e9b0 0x7f392406b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47480/0 (socket says 172.21.15.32:47480) 2024-04-24T22:59:15.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.522+0000 7f392259c640 1 -- 172.21.15.32:0/342831716 learned_addr learned my addr 172.21.15.32:0/342831716 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:15.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.522+0000 7f54bdede640 1 --2- 172.21.15.32:0/1188720565 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f54b815c690 0x7f54b80707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:15.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.523+0000 7f54bd6dd640 1 -- 172.21.15.32:0/1188720565 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f54b815c690 msgr2=0x7f54b80707c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:15.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.523+0000 7f54bd6dd640 1 --2- 172.21.15.32:0/1188720565 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f54b815c690 0x7f54b80707c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.523+0000 7f54bd6dd640 1 -- 172.21.15.32:0/1188720565 >> v1:172.21.15.121:6789/0 conn(0x7f54b806d350 legacy=0x7f54b8160060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:15.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.523+0000 7f54bd6dd640 1 -- 172.21.15.32:0/1188720565 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54b806c8f0 con 0x7f54b80b3f70 2024-04-24T22:59:15.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.523+0000 7f3928934640 1 --2- 172.21.15.32:0/342831716 >> v2:172.21.15.32:3300/0 conn(0x7f39240d4210 0x7f39240d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:15.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.523+0000 7f8e2cf8f640 1 --2- 172.21.15.32:0/1909059908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e280cd5c0 0x7f8e280b1ea0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8e280ceab0 tx=0x7f8e1802a030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8295307a1728688 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:15.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.523+0000 7f8e1ffff640 1 -- 172.21.15.32:0/1909059908 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e18037070 con 0x7f8e280cd5c0 2024-04-24T22:59:15.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.523+0000 7f8e1ffff640 1 -- 172.21.15.32:0/1909059908 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e1802acc0 con 0x7f8e280cd5c0 2024-04-24T22:59:15.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.523+0000 7f8e1ffff640 1 -- 172.21.15.32:0/1909059908 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e18032660 con 0x7f8e280cd5c0 2024-04-24T22:59:15.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.523+0000 7f1e23ba2640 1 -- 172.21.15.32:0/1478479445 shutdown_connections 2024-04-24T22:59:15.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.523+0000 7f9fcf30e640 1 Processor -- start 2024-04-24T22:59:15.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.523+0000 7f9fcf30e640 1 -- start start 2024-04-24T22:59:15.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.523+0000 7f069ad9d640 1 --2- 172.21.15.32:0/2169363957 >> v2:172.21.15.32:3300/0 conn(0x7f069c15e700 0x7f069c0ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:15.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.523+0000 7f069b59e640 1 -- 172.21.15.32:0/2169363957 >> v2:172.21.15.32:3300/0 conn(0x7f069c15e700 msgr2=0x7f069c0ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:15.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.523+0000 7f0699d9b640 1 -- 172.21.15.32:0/2169363957 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 987660811 0 0) 0x7f069c0af830 con 0x7f069c0cde20 2024-04-24T22:59:15.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.523+0000 7f069b59e640 1 --2- 172.21.15.32:0/2169363957 >> v2:172.21.15.32:3300/0 conn(0x7f069c15e700 0x7f069c0ccde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.523+0000 7f069b59e640 1 -- 172.21.15.32:0/2169363957 >> v1:172.21.15.32:6789/0 conn(0x7f069c0cde20 legacy=0x7f069c1621e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:15.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.523+0000 7f069b59e640 1 -- 172.21.15.32:0/2169363957 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f069c0aef00 con 0x7f069c0718a0 2024-04-24T22:59:15.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.523+0000 7f069ad9d640 1 --2- 172.21.15.32:0/2169363957 >> v2:172.21.15.32:3300/0 conn(0x7f069c15e700 0x7f069c0ccde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:59:15.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.523+0000 7f3928934640 1 -- 172.21.15.32:0/342831716 --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39240d3170 con 0x7f392415e9b0 2024-04-24T22:59:15.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.523+0000 7f3928934640 1 -- 172.21.15.32:0/342831716 --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39240d3450 con 0x7f392406b820 2024-04-24T22:59:15.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.523+0000 7f3928934640 1 -- 172.21.15.32:0/342831716 --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f39240d3730 con 0x7f39240d4210 2024-04-24T22:59:15.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.523+0000 7f392159a640 1 -- 172.21.15.32:0/342831716 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2795994764 0 0) 0x7f39240d3450 con 0x7f392406b820 2024-04-24T22:59:15.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.523+0000 7f392159a640 1 -- 172.21.15.32:0/342831716 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f390c003520 con 0x7f392406b820 2024-04-24T22:59:15.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.524+0000 7f54bd6dd640 1 --2- 172.21.15.32:0/1188720565 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54b80b3f70 0x7f54b806ce10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f54a80097a0 tx=0x7f54a802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6530b36b2be54df8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:15.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.523+0000 7f1e23ba2640 1 -- 172.21.15.32:0/1478479445 wait complete. 2024-04-24T22:59:15.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.524+0000 7f54bcedc640 1 -- 172.21.15.32:0/1188720565 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a8037070 con 0x7f54b80b3f70 2024-04-24T22:59:15.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.524+0000 7f54bcedc640 1 -- 172.21.15.32:0/1188720565 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54a8032440 con 0x7f54b80b3f70 2024-04-24T22:59:15.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.524+0000 7f54bcedc640 1 -- 172.21.15.32:0/1188720565 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a8037070 con 0x7f54b80b3f70 2024-04-24T22:59:15.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.524+0000 7f9fcf30e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9fc80cd550 0x7f9fc8158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:15.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.524+0000 7f9fcf30e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fc80ce360 0x7f9fc8072790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:15.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.524+0000 7f9fcf30e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9fc80733a0 0x7f9fc8072cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:15.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.524+0000 7f9fcf30e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc80b4e70 con 0x7f9fc80ce360 2024-04-24T22:59:15.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.524+0000 7f9fcf30e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc80cd980 con 0x7f9fc80733a0 2024-04-24T22:59:15.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.524+0000 7f9fcf30e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f9fc80b4ff0 con 0x7f9fc80cd550 2024-04-24T22:59:15.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.524+0000 7f8163fff640 1 Processor -- start 2024-04-24T22:59:15.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.524+0000 7f9fce30c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9fc80cd550 0x7f9fc8158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:15.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.524+0000 7f8e2f9f4640 1 -- 172.21.15.32:0/1909059908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e280cd5c0 msgr2=0x7f8e280b1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:15.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.524+0000 7f8e2f9f4640 1 --2- 172.21.15.32:0/1909059908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e280cd5c0 0x7f8e280b1ea0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8e280ceab0 tx=0x7f8e1802a030 comp rx=0 tx=0).stop 2024-04-24T22:59:15.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.524+0000 7f8e2f9f4640 1 -- 172.21.15.32:0/1909059908 shutdown_connections 2024-04-24T22:59:15.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.524+0000 7f8e2f9f4640 1 --2- 172.21.15.32:0/1909059908 >> v2:172.21.15.121:3300/0 conn(0x7f8e280b23e0 0x7f8e28070600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.525+0000 7f8e2f9f4640 1 --2- 172.21.15.32:0/1909059908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e280cd5c0 0x7f8e280b1ea0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.525+0000 7f8e2f9f4640 1 -- 172.21.15.32:0/1909059908 >> 172.21.15.32:0/1909059908 conn(0x7f8e280743f0 msgr2=0x7f8e2806b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:15.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.524+0000 7f9fceb0d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9fc80733a0 0x7f9fc8072cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:15.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.524+0000 7f9fce30c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9fc80cd550 0x7f9fc8158e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.32:42184/0 (socket says 172.21.15.32:42184) 2024-04-24T22:59:15.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.524+0000 7f9fce30c640 1 -- 172.21.15.32:0/1248836844 learned_addr learned my addr 172.21.15.32:0/1248836844 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:15.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.525+0000 7f069b59e640 1 --2- 172.21.15.32:0/2169363957 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f069c0718a0 0x7f069c0cd320 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f069c162120 tx=0x7f069402a6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7a93cd61110d79f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:15.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.525+0000 7f392159a640 1 -- 172.21.15.32:0/342831716 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2901555789 0 0) 0x7f39240d3170 con 0x7f392415e9b0 2024-04-24T22:59:15.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.525+0000 7f392159a640 1 -- 172.21.15.32:0/342831716 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f39240d3450 con 0x7f392415e9b0 2024-04-24T22:59:15.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.525+0000 7f392159a640 1 -- 172.21.15.32:0/342831716 <== mon.1 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1393356980 0 0) 0x7f390c003520 con 0x7f392406b820 2024-04-24T22:59:15.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.525+0000 7f392159a640 1 -- 172.21.15.32:0/342831716 >> v2:172.21.15.32:3300/0 conn(0x7f39240d4210 msgr2=0x7f39240d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:15.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.525+0000 7f3922d9d640 1 --2- 172.21.15.32:0/342831716 >> v2:172.21.15.32:3300/0 conn(0x7f39240d4210 0x7f39240d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:15.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.525+0000 7f9fcdb0b640 1 --2- 172.21.15.32:0/1248836844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fc80ce360 0x7f9fc8072790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:15.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.525+0000 7f392159a640 1 --2- 172.21.15.32:0/342831716 >> v2:172.21.15.32:3300/0 conn(0x7f39240d4210 0x7f39240d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.525+0000 7f8e2f9f4640 1 -- 172.21.15.32:0/1909059908 shutdown_connections 2024-04-24T22:59:15.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.525+0000 7f392159a640 1 -- 172.21.15.32:0/342831716 >> v1:172.21.15.32:6789/0 conn(0x7f392415e9b0 legacy=0x7f392406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:15.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.525+0000 7f392159a640 1 -- 172.21.15.32:0/342831716 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39240d3a10 con 0x7f392406b820 2024-04-24T22:59:15.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.525+0000 7f392159a640 1 -- 172.21.15.32:0/342831716 <== mon.1 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f39180015f0 con 0x7f392406b820 2024-04-24T22:59:15.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.525+0000 7f392159a640 1 -- 172.21.15.32:0/342831716 >> v1:172.21.15.121:6789/0 conn(0x7f392406b820 legacy=0x7f39240b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:15.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.525+0000 7f392159a640 1 --2- 172.21.15.32:0/342831716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f390c003b40 0x7f390c003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:15.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.525+0000 7f392159a640 1 --2- 172.21.15.32:0/342831716 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f390c004800 0x7f390c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:15.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.525+0000 7f392159a640 1 --2- 172.21.15.32:0/342831716 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f390c0053c0 0x7f390c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:15.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.525+0000 7f392159a640 1 -- 172.21.15.32:0/342831716 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f39240d3730 con 0x7f390c003b40 2024-04-24T22:59:15.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.525+0000 7f392159a640 1 -- 172.21.15.32:0/342831716 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f390c00c260 con 0x7f390c004800 2024-04-24T22:59:15.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.525+0000 7f392159a640 1 -- 172.21.15.32:0/342831716 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f390c00c460 con 0x7f390c0053c0 2024-04-24T22:59:15.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.525+0000 7f3922d9d640 1 --2- 172.21.15.32:0/342831716 >> v2:172.21.15.32:3300/0 conn(0x7f39240d4210 0x7f39240d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:59:15.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.525+0000 7f8163fff640 1 -- start start 2024-04-24T22:59:15.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.525+0000 7f9fceb0d640 1 -- 172.21.15.32:0/1248836844 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9fc80cd550 msgr2=0x7f9fc8158e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:15.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.525+0000 7f9fceb0d640 1 --2- 172.21.15.32:0/1248836844 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9fc80cd550 0x7f9fc8158e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.525+0000 7f9fceb0d640 1 -- 172.21.15.32:0/1248836844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fc80ce360 msgr2=0x7f9fc8072790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:15.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.525+0000 7f9fceb0d640 1 --2- 172.21.15.32:0/1248836844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fc80ce360 0x7f9fc8072790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.525+0000 7f9fceb0d640 1 -- 172.21.15.32:0/1248836844 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fc8157d60 con 0x7f9fc80733a0 2024-04-24T22:59:15.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.525+0000 7f1e23ba2640 1 Processor -- start 2024-04-24T22:59:15.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.525+0000 7f392259c640 1 --2- 172.21.15.32:0/342831716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f390c003b40 0x7f390c003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:15.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.525+0000 7f0699d9b640 1 -- 172.21.15.32:0/2169363957 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0694036070 con 0x7f069c0718a0 2024-04-24T22:59:15.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.525+0000 7f3922d9d640 1 --2- 172.21.15.32:0/342831716 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f390c0053c0 0x7f390c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:15.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.525+0000 7f54c0142640 1 -- 172.21.15.32:0/1188720565 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54b80b3f70 msgr2=0x7f54b806ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:15.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.525+0000 7f54c0142640 1 --2- 172.21.15.32:0/1188720565 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54b80b3f70 0x7f54b806ce10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f54a80097a0 tx=0x7f54a802a180 comp rx=0 tx=0).stop 2024-04-24T22:59:15.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.526+0000 7f0699d9b640 1 -- 172.21.15.32:0/2169363957 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0694031600 con 0x7f069c0718a0 2024-04-24T22:59:15.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.526+0000 7f8163fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f815c053a90 0x7f815c0d3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:15.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.526+0000 7f8163fff640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f815c053e70 0x7f815c0d23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:15.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.526+0000 7f8162ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f815c053a90 0x7f815c0d3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:15.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.526+0000 7f8162ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f815c053a90 0x7f815c0d3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38724/0 (socket says 172.21.15.32:38724) 2024-04-24T22:59:15.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.526+0000 7f8162ffd640 1 -- 172.21.15.32:0/3649358836 learned_addr learned my addr 172.21.15.32:0/3649358836 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:15.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.525+0000 7f8e2f9f4640 1 -- 172.21.15.32:0/1909059908 wait complete. 2024-04-24T22:59:15.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.526+0000 7f1e23ba2640 1 -- start start 2024-04-24T22:59:15.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.526+0000 7f0699d9b640 1 -- 172.21.15.32:0/2169363957 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06940319d0 con 0x7f069c0718a0 2024-04-24T22:59:15.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.526+0000 7f3921d9b640 1 --2- 172.21.15.32:0/342831716 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f390c004800 0x7f390c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:15.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.526+0000 7f3922d9d640 1 -- 172.21.15.32:0/342831716 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f390c004800 msgr2=0x7f390c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:15.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.526+0000 7f3922d9d640 1 --2- 172.21.15.32:0/342831716 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f390c004800 0x7f390c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.526+0000 7f3922d9d640 1 -- 172.21.15.32:0/342831716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f390c003b40 msgr2=0x7f390c003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:15.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.526+0000 7f3922d9d640 1 --2- 172.21.15.32:0/342831716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f390c003b40 0x7f390c003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.526+0000 7f3922d9d640 1 -- 172.21.15.32:0/342831716 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39240d3a10 con 0x7f390c0053c0 2024-04-24T22:59:15.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.526+0000 7f3921d9b640 1 --2- 172.21.15.32:0/342831716 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f390c004800 0x7f390c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:59:15.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.526+0000 7f8163fff640 1 --2- 172.21.15.32:0/3649358836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f815c054980 0x7f815c0e2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:15.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.526+0000 7f392259c640 1 --2- 172.21.15.32:0/342831716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f390c003b40 0x7f390c003f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:59:15.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.526+0000 7f81627fc640 1 --2- 172.21.15.32:0/3649358836 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f815c053e70 0x7f815c0d23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:15.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.526+0000 7f8163fff640 1 -- 172.21.15.32:0/3649358836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f815c0507e0 con 0x7f815c053a90 2024-04-24T22:59:15.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.526+0000 7f8163fff640 1 -- 172.21.15.32:0/3649358836 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f815c0512d0 con 0x7f815c054980 2024-04-24T22:59:15.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.526+0000 7f8163fff640 1 -- 172.21.15.32:0/3649358836 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f815c038d80 con 0x7f815c053e70 2024-04-24T22:59:15.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.527+0000 7f06a082b640 1 -- 172.21.15.32:0/2169363957 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f069c0718a0 msgr2=0x7f069c0cd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:15.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.527+0000 7f06a082b640 1 --2- 172.21.15.32:0/2169363957 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f069c0718a0 0x7f069c0cd320 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f069c162120 tx=0x7f069402a6a0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.527+0000 7f06a082b640 1 -- 172.21.15.32:0/2169363957 shutdown_connections 2024-04-24T22:59:15.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.527+0000 7f06a082b640 1 --2- 172.21.15.32:0/2169363957 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f069c0718a0 0x7f069c0cd320 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.527+0000 7f06a082b640 1 --2- 172.21.15.32:0/2169363957 >> v2:172.21.15.32:3300/0 conn(0x7f069c15e700 0x7f069c0ccde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.527+0000 7f06a082b640 1 -- 172.21.15.32:0/2169363957 >> 172.21.15.32:0/2169363957 conn(0x7f069c0b2160 msgr2=0x7f069c0b2570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:15.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.527+0000 7f54c0142640 1 -- 172.21.15.32:0/1188720565 shutdown_connections 2024-04-24T22:59:15.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.527+0000 7f54c0142640 1 --2- 172.21.15.32:0/1188720565 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54b80b3f70 0x7f54b806ce10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.527+0000 7f54c0142640 1 --2- 172.21.15.32:0/1188720565 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f54b815c690 0x7f54b80707c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.527+0000 7f54c0142640 1 -- 172.21.15.32:0/1188720565 >> 172.21.15.32:0/1188720565 conn(0x7f54b806b110 msgr2=0x7f54b806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:15.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.527+0000 7f81637fe640 1 --2- 172.21.15.32:0/3649358836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f815c054980 0x7f815c0e2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:15.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.527+0000 7f1e23ba2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e14053760 0x7f1e14178990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:15.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.527+0000 7f3922d9d640 1 -- 172.21.15.32:0/342831716 reap_dead start 2024-04-24T22:59:15.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.527+0000 7f54c0142640 1 -- 172.21.15.32:0/1188720565 shutdown_connections 2024-04-24T22:59:15.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.527+0000 7f8e2f9f4640 1 Processor -- start 2024-04-24T22:59:15.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.527+0000 7f8e2f9f4640 1 -- start start 2024-04-24T22:59:15.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.527+0000 7f06a082b640 1 -- 172.21.15.32:0/2169363957 shutdown_connections 2024-04-24T22:59:15.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.527+0000 7f06a082b640 1 -- 172.21.15.32:0/2169363957 wait complete. 2024-04-24T22:59:15.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.527+0000 7f8162ffd640 1 -- 172.21.15.32:0/3649358836 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f815c053e70 msgr2=0x7f815c0d23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:15.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.527+0000 7f8162ffd640 1 --2- 172.21.15.32:0/3649358836 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f815c053e70 0x7f815c0d23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.527+0000 7f8162ffd640 1 -- 172.21.15.32:0/3649358836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f815c054980 msgr2=0x7f815c0e2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:15.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.527+0000 7f8162ffd640 1 --2- 172.21.15.32:0/3649358836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f815c054980 0x7f815c0e2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.527+0000 7f8162ffd640 1 -- 172.21.15.32:0/3649358836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f815c0d8210 con 0x7f815c053a90 2024-04-24T22:59:15.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.528+0000 7f8162ffd640 1 --2- 172.21.15.32:0/3649358836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f815c053a90 0x7f815c0d3b00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f815400b450 tx=0x7f815400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:15.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.528+0000 7f1e23ba2640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1e140e3290 0x7f1e1404dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:15.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.528+0000 7f1e23ba2640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e140e6cf0 0x7f1e1404dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:15.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.528+0000 7f1e23ba2640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1403ea80 con 0x7f1e14053760 2024-04-24T22:59:15.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.528+0000 7f1e23ba2640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1e14054100 con 0x7f1e140e6cf0 2024-04-24T22:59:15.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.528+0000 7f1e23ba2640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f1e14058c10 con 0x7f1e140e3290 2024-04-24T22:59:15.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.528+0000 7f3922d9d640 1 --2- 172.21.15.32:0/342831716 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f390c0053c0 0x7f390c00b9c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f391c00c490 tx=0x7f390c005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14535f4b89dd84d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:15.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.528+0000 7f1e2193e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e14053760 0x7f1e14178990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:15.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.528+0000 7f814b7fe640 1 -- 172.21.15.32:0/3649358836 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8154015020 con 0x7f815c053a90 2024-04-24T22:59:15.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.528+0000 7f8e2f9f4640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8e280b23e0 0x7f8e2815d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:15.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.528+0000 7f1e2193e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e14053760 0x7f1e14178990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38728/0 (socket says 172.21.15.32:38728) 2024-04-24T22:59:15.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.528+0000 7f1e2193e640 1 -- 172.21.15.32:0/2380961624 learned_addr learned my addr 172.21.15.32:0/2380961624 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:15.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.528+0000 7f392159a640 1 -- 172.21.15.32:0/342831716 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391c038070 con 0x7f390c0053c0 2024-04-24T22:59:15.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.528+0000 7f392159a640 1 -- 172.21.15.32:0/342831716 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f391c02bbb0 con 0x7f390c0053c0 2024-04-24T22:59:15.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.528+0000 7f392159a640 1 -- 172.21.15.32:0/342831716 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391c007c10 con 0x7f390c0053c0 2024-04-24T22:59:15.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.529+0000 7f814b7fe640 1 -- 172.21.15.32:0/3649358836 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81540118b0 con 0x7f815c053a90 2024-04-24T22:59:15.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.528+0000 7f8e2f9f4640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e2816b430 0x7f8e28159f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:15.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.528+0000 7f8e2f9f4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e2815c430 0x7f8e2815a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:15.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.529+0000 7f8e2d790640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8e280b23e0 0x7f8e2815d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:15.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.527+0000 7f54c0142640 1 -- 172.21.15.32:0/1188720565 wait complete. 2024-04-24T22:59:15.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.529+0000 7f8e2d790640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8e280b23e0 0x7f8e2815d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.32:42216/0 (socket says 172.21.15.32:42216) 2024-04-24T22:59:15.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.529+0000 7f8e2cf8f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e2816b430 0x7f8e28159f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:15.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.529+0000 7f8e2d790640 1 -- 172.21.15.32:0/2544406087 learned_addr learned my addr 172.21.15.32:0/2544406087 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:15.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.529+0000 7f8e2f9f4640 1 -- 172.21.15.32:0/2544406087 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8e280cf690 con 0x7f8e2815c430 2024-04-24T22:59:15.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.529+0000 7f8e2f9f4640 1 -- 172.21.15.32:0/2544406087 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8e280cda30 con 0x7f8e2816b430 2024-04-24T22:59:15.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.529+0000 7f8e2f9f4640 1 -- 172.21.15.32:0/2544406087 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f8e2815dea0 con 0x7f8e280b23e0 2024-04-24T22:59:15.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.529+0000 7f54c0142640 1 Processor -- start 2024-04-24T22:59:15.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.527+0000 7f9fceb0d640 1 --2- 172.21.15.32:0/1248836844 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9fc80733a0 0x7f9fc8072cd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0031000 tx=0x7f9fc0004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:15.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.527+0000 7f9fbeffd640 1 -- 172.21.15.32:0/1248836844 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc0040070 con 0x7f9fc80733a0 2024-04-24T22:59:15.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.527+0000 7f9fbeffd640 1 -- 172.21.15.32:0/1248836844 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fc0045660 con 0x7f9fc80733a0 2024-04-24T22:59:15.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.527+0000 7f9fcf30e640 1 -- 172.21.15.32:0/1248836844 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fc8158040 con 0x7f9fc80733a0 2024-04-24T22:59:15.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.527+0000 7f9fcf30e640 1 -- 172.21.15.32:0/1248836844 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fc8158600 con 0x7f9fc80733a0 2024-04-24T22:59:15.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.529+0000 7f1e2193e640 1 -- 172.21.15.32:0/2380961624 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1e140e3290 msgr2=0x7f1e1404dab0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T22:59:15.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.528+0000 7f9fbeffd640 1 -- 172.21.15.32:0/1248836844 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc0045a10 con 0x7f9fc80733a0 2024-04-24T22:59:15.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.529+0000 7f1e2193e640 1 --2- 172.21.15.32:0/2380961624 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1e140e3290 0x7f1e1404dab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.529+0000 7f1e2193e640 1 -- 172.21.15.32:0/2380961624 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e140e6cf0 msgr2=0x7f1e1404dff0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-24T22:59:15.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.529+0000 7f1e2193e640 1 --2- 172.21.15.32:0/2380961624 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e140e6cf0 0x7f1e1404dff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.529+0000 7f1e2193e640 1 -- 172.21.15.32:0/2380961624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e140db0e0 con 0x7f1e14053760 2024-04-24T22:59:15.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.529+0000 7f8e2d790640 1 -- 172.21.15.32:0/2544406087 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e2816b430 msgr2=0x7f8e28159f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:15.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.529+0000 7f814b7fe640 1 -- 172.21.15.32:0/3649358836 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8154011ca0 con 0x7f815c053a90 2024-04-24T22:59:15.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.529+0000 7f8e2d790640 1 --2- 172.21.15.32:0/2544406087 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e2816b430 0x7f8e28159f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.529+0000 7f1e2193e640 1 --2- 172.21.15.32:0/2380961624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e14053760 0x7f1e14178990 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c0699e0 tx=0x7f1e1c069d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:15.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.529+0000 7f8163fff640 1 -- 172.21.15.32:0/3649358836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f815c0d84b0 con 0x7f815c053a90 2024-04-24T22:59:15.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.530+0000 7f8163fff640 1 -- 172.21.15.32:0/3649358836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f815c0d8a70 con 0x7f815c053a90 2024-04-24T22:59:15.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.529+0000 7f8e2d790640 1 -- 172.21.15.32:0/2544406087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e2815c430 msgr2=0x7f8e2815a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:15.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.529+0000 7f8e2d790640 1 --2- 172.21.15.32:0/2544406087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e2815c430 0x7f8e2815a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.529+0000 7f8e2d790640 1 -- 172.21.15.32:0/2544406087 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e2815c8c0 con 0x7f8e280b23e0 2024-04-24T22:59:15.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.529+0000 7f8e2d790640 1 --2- 172.21.15.32:0/2544406087 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8e280b23e0 0x7f8e2815d960 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8e2400c570 tx=0x7f8e2400ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:15.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.530+0000 7f1e127fc640 1 -- 172.21.15.32:0/2380961624 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c0be020 con 0x7f1e14053760 2024-04-24T22:59:15.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.530+0000 7f1e127fc640 1 -- 172.21.15.32:0/2380961624 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e1c0b9470 con 0x7f1e14053760 2024-04-24T22:59:15.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.530+0000 7f1e23ba2640 1 -- 172.21.15.32:0/2380961624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e140db3c0 con 0x7f1e14053760 2024-04-24T22:59:15.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.530+0000 7f1e23ba2640 1 -- 172.21.15.32:0/2380961624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e140db980 con 0x7f1e14053760 2024-04-24T22:59:15.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.529+0000 7f9fbeffd640 1 -- 172.21.15.32:0/1248836844 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9fc0039040 con 0x7f9fc80733a0 2024-04-24T22:59:15.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.530+0000 7f9fbeffd640 1 --2- 172.21.15.32:0/1248836844 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9fac06be30 0x7f9fac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:15.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.530+0000 7f9fbeffd640 1 -- 172.21.15.32:0/1248836844 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f9fc00acd20 con 0x7f9fc80733a0 2024-04-24T22:59:15.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.530+0000 7f9fce30c640 1 --2- 172.21.15.32:0/1248836844 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9fac06be30 0x7f9fac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:15.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.530+0000 7f06a082b640 1 Processor -- start 2024-04-24T22:59:15.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.529+0000 7f3928934640 1 -- 172.21.15.32:0/342831716 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f390c0053c0 msgr2=0x7f390c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:15.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.530+0000 7f3928934640 1 --2- 172.21.15.32:0/342831716 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f390c0053c0 0x7f390c00b9c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f391c00c490 tx=0x7f390c005830 comp rx=0 tx=0).stop 2024-04-24T22:59:15.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.530+0000 7f3928934640 1 -- 172.21.15.32:0/342831716 shutdown_connections 2024-04-24T22:59:15.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.530+0000 7f3928934640 1 --2- 172.21.15.32:0/342831716 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f390c0053c0 0x7f390c00b9c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.530+0000 7f3928934640 1 -- 172.21.15.32:0/342831716 >> 172.21.15.32:0/342831716 conn(0x7f39240d11d0 msgr2=0x7f39240d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:15.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.529+0000 7f54c0142640 1 -- start start 2024-04-24T22:59:15.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.530+0000 7f54c0142640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f54b806d350 0x7f54b8150fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:15.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.530+0000 7f54c0142640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54b80b3f70 0x7f54b8155020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:15.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.530+0000 7f54c0142640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54b815c690 0x7f54b814a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:15.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.530+0000 7f54bdede640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f54b806d350 0x7f54b8150fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:15.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.530+0000 7f54bdede640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f54b806d350 0x7f54b8150fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.32:42224/0 (socket says 172.21.15.32:42224) 2024-04-24T22:59:15.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.531+0000 7f54bd6dd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54b80b3f70 0x7f54b8155020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:15.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.531+0000 7f54bd6dd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54b80b3f70 0x7f54b8155020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.32:49010/0 (socket says 172.21.15.32:49010) 2024-04-24T22:59:15.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.531+0000 7f54c0142640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f54b80725c0 con 0x7f54b815c690 2024-04-24T22:59:15.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.531+0000 7f54c0142640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f54b8072440 con 0x7f54b80b3f70 2024-04-24T22:59:15.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.531+0000 7f54c0142640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f54b80b44e0 con 0x7f54b806d350 2024-04-24T22:59:15.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.531+0000 7f54bdede640 1 -- 172.21.15.32:0/4258578206 learned_addr learned my addr 172.21.15.32:0/4258578206 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:15.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.530+0000 7f06a082b640 1 -- start start 2024-04-24T22:59:15.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.531+0000 7f8e1dffb640 1 -- 172.21.15.32:0/2544406087 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e24018070 con 0x7f8e280b23e0 2024-04-24T22:59:15.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.531+0000 7f8e1dffb640 1 -- 172.21.15.32:0/2544406087 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e240074e0 con 0x7f8e280b23e0 2024-04-24T22:59:15.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.531+0000 7f8e1dffb640 1 -- 172.21.15.32:0/2544406087 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e24007690 con 0x7f8e280b23e0 2024-04-24T22:59:15.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.531+0000 7f8e2f9f4640 1 -- 172.21.15.32:0/2544406087 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e2815cba0 con 0x7f8e280b23e0 2024-04-24T22:59:15.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.531+0000 7f8e2f9f4640 1 -- 172.21.15.32:0/2544406087 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e2815d160 con 0x7f8e280b23e0 2024-04-24T22:59:15.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.531+0000 7f3928934640 1 -- 172.21.15.32:0/342831716 shutdown_connections 2024-04-24T22:59:15.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.531+0000 7f54bdede640 1 -- 172.21.15.32:0/4258578206 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54b80b3f70 msgr2=0x7f54b8155020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:15.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.531+0000 7f54bdede640 1 --2- 172.21.15.32:0/4258578206 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54b80b3f70 0x7f54b8155020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.531+0000 7f54bdede640 1 -- 172.21.15.32:0/4258578206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54b815c690 msgr2=0x7f54b814a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:15.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.531+0000 7f54bdede640 1 --2- 172.21.15.32:0/4258578206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54b815c690 0x7f54b814a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.531+0000 7f54bdede640 1 -- 172.21.15.32:0/4258578206 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54b8151650 con 0x7f54b806d350 2024-04-24T22:59:15.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.531+0000 7f9fce30c640 1 --2- 172.21.15.32:0/1248836844 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9fac06be30 0x7f9fac06e2f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8009a30 tx=0x7f9fb8009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:15.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.531+0000 7f9fcf30e640 1 -- 172.21.15.32:0/1248836844 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fc80682b0 con 0x7f9fc80733a0 2024-04-24T22:59:15.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.532+0000 7f8163fff640 1 -- 172.21.15.32:0/3649358836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f815c0057e0 con 0x7f815c053a90 2024-04-24T22:59:15.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.533+0000 7f8e2f9f4640 1 -- 172.21.15.32:0/2544406087 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e280682b0 con 0x7f8e280b23e0 2024-04-24T22:59:15.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.531+0000 7f1e127fc640 1 -- 172.21.15.32:0/2380961624 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c0b3040 con 0x7f1e14053760 2024-04-24T22:59:15.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.533+0000 7f1e07fff640 1 -- 172.21.15.32:0/2380961624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e140053c0 con 0x7f1e14053760 2024-04-24T22:59:15.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.531+0000 7f06a082b640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f069c0718a0 0x7f069c150310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:15.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.531+0000 7f06a082b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f069c0cde20 0x7f069c150ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:15.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.531+0000 7f06a082b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f069c15e700 0x7f069c1530f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:15.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.531+0000 7f06a082b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f069c0ae8b0 con 0x7f069c15e700 2024-04-24T22:59:15.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.531+0000 7f06a082b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f069c0af4c0 con 0x7f069c0cde20 2024-04-24T22:59:15.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.531+0000 7f06a082b640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f069c0b3460 con 0x7f069c0718a0 2024-04-24T22:59:15.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.533+0000 7f069a59c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f069c0cde20 0x7f069c150ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:15.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.533+0000 7f069a59c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f069c0cde20 0x7f069c150ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.32:49020/0 (socket says 172.21.15.32:49020) 2024-04-24T22:59:15.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.533+0000 7f069a59c640 1 -- 172.21.15.32:0/2381135675 learned_addr learned my addr 172.21.15.32:0/2381135675 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:15.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.533+0000 7f069ad9d640 1 --2- 172.21.15.32:0/2381135675 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f069c0718a0 0x7f069c150310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:15.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.531+0000 7f3928934640 1 -- 172.21.15.32:0/342831716 wait complete. 2024-04-24T22:59:15.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.533+0000 7f54bdede640 1 --2- 172.21.15.32:0/4258578206 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f54b806d350 0x7f54b8150fe0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f54b400c820 tx=0x7f54b400cce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:15.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.533+0000 7f069b59e640 1 --2- 172.21.15.32:0/2381135675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f069c15e700 0x7f069c1530f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:15.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.533+0000 7f069a59c640 1 -- 172.21.15.32:0/2381135675 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f069c0718a0 msgr2=0x7f069c150310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:15.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.533+0000 7f069a59c640 1 --2- 172.21.15.32:0/2381135675 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f069c0718a0 0x7f069c150310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.533+0000 7f069a59c640 1 -- 172.21.15.32:0/2381135675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f069c15e700 msgr2=0x7f069c1530f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:15.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.533+0000 7f069a59c640 1 --2- 172.21.15.32:0/2381135675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f069c15e700 0x7f069c1530f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.533+0000 7f54ae7fc640 1 -- 172.21.15.32:0/4258578206 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b4015070 con 0x7f54b806d350 2024-04-24T22:59:15.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.533+0000 7f54ae7fc640 1 -- 172.21.15.32:0/4258578206 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54b4004240 con 0x7f54b806d350 2024-04-24T22:59:15.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.533+0000 7f54ae7fc640 1 -- 172.21.15.32:0/4258578206 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b400ce10 con 0x7f54b806d350 2024-04-24T22:59:15.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.533+0000 7f8e1dffb640 1 -- 172.21.15.32:0/2544406087 <== mon.2 v2:172.21.15.121:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8e2401b070 con 0x7f8e280b23e0 2024-04-24T22:59:15.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.533+0000 7f54c0142640 1 -- 172.21.15.32:0/4258578206 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54b8151930 con 0x7f54b806d350 2024-04-24T22:59:15.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.534+0000 7f54c0142640 1 -- 172.21.15.32:0/4258578206 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f54b8151ef0 con 0x7f54b806d350 2024-04-24T22:59:15.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.533+0000 7f069a59c640 1 -- 172.21.15.32:0/2381135675 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f069c1521c0 con 0x7f069c0cde20 2024-04-24T22:59:15.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.534+0000 7f069b59e640 1 --2- 172.21.15.32:0/2381135675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f069c15e700 0x7f069c1530f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:59:15.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.534+0000 7f1e127fc640 1 -- 172.21.15.32:0/2380961624 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1e1c0c1020 con 0x7f1e14053760 2024-04-24T22:59:15.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.534+0000 7f069a59c640 1 --2- 172.21.15.32:0/2381135675 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f069c0cde20 0x7f069c150ed0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f069000c430 tx=0x7f069000c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:15.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.534+0000 7f8e1dffb640 1 --2- 172.21.15.32:0/2544406087 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8e0c06bf00 0x7f8e0c06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:15.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.534+0000 7f3928934640 1 Processor -- start 2024-04-24T22:59:15.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.534+0000 7f8e1dffb640 1 -- 172.21.15.32:0/2544406087 <== mon.2 v2:172.21.15.121:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8e2408a340 con 0x7f8e280b23e0 2024-04-24T22:59:15.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.534+0000 7f1e127fc640 1 --2- 172.21.15.32:0/2380961624 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1e0006bf00 0x7f1e0006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:15.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.534+0000 7f3928934640 1 -- start start 2024-04-24T22:59:15.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.534+0000 7f8e2cf8f640 1 --2- 172.21.15.32:0/2544406087 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8e0c06bf00 0x7f8e0c06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:15.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.535+0000 7f1e2113d640 1 --2- 172.21.15.32:0/2380961624 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1e0006bf00 0x7f1e0006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:15.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.535+0000 7f1e127fc640 1 -- 172.21.15.32:0/2380961624 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1e1c1272f0 con 0x7f1e14053760 2024-04-24T22:59:15.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.535+0000 7f1e2113d640 1 --2- 172.21.15.32:0/2380961624 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1e0006bf00 0x7f1e0006e3c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1e14050c20 tx=0x7f1e0c009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:15.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.535+0000 7f8e2cf8f640 1 --2- 172.21.15.32:0/2544406087 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8e0c06bf00 0x7f8e0c06e3c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8e18034000 tx=0x7f8e1802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:15.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.535+0000 7f54c0142640 1 -- 172.21.15.32:0/4258578206 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54b80689c0 con 0x7f54b806d350 2024-04-24T22:59:15.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.536+0000 7f814b7fe640 1 -- 172.21.15.32:0/3649358836 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8154018020 con 0x7f815c053a90 2024-04-24T22:59:15.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.536+0000 7f3928934640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f390c0053c0 0x7f39240ca350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:15.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.536+0000 7f3928934640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3924148340 0x7f39241487a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:15.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.536+0000 7f3928934640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f39241f34a0 0x7f39241f3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:15.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.536+0000 7f814b7fe640 1 --2- 172.21.15.32:0/3649358836 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f815006be80 0x7f815006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:15.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.536+0000 7f3928934640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f39240ba0e0 con 0x7f3924148340 2024-04-24T22:59:15.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.536+0000 7f392259c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f390c0053c0 0x7f39240ca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:15.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.536+0000 7f3921d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3924148340 0x7f39241487a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:15.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.536+0000 7f392259c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f390c0053c0 0x7f39240ca350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.32:42238/0 (socket says 172.21.15.32:42238) 2024-04-24T22:59:15.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.536+0000 7f392259c640 1 -- 172.21.15.32:0/1392388926 learned_addr learned my addr 172.21.15.32:0/1392388926 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:15.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.536+0000 7f81627fc640 1 --2- 172.21.15.32:0/3649358836 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f815006be80 0x7f815006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:15.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.536+0000 7f3928934640 1 -- 172.21.15.32:0/1392388926 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f39240d2400 con 0x7f39241f34a0 2024-04-24T22:59:15.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.536+0000 7f814b7fe640 1 -- 172.21.15.32:0/3649358836 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f815408b470 con 0x7f815c053a90 2024-04-24T22:59:15.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.536+0000 7f3922d9d640 1 --2- 172.21.15.32:0/1392388926 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f39241f34a0 0x7f39241f3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:15.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.536+0000 7f3928934640 1 -- 172.21.15.32:0/1392388926 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f39240ba260 con 0x7f390c0053c0 2024-04-24T22:59:15.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.537+0000 7f06837fe640 1 -- 172.21.15.32:0/2381135675 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0690017070 con 0x7f069c0cde20 2024-04-24T22:59:15.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.537+0000 7f06a082b640 1 -- 172.21.15.32:0/2381135675 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f069c151920 con 0x7f069c0cde20 2024-04-24T22:59:15.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.537+0000 7f9fbeffd640 1 -- 172.21.15.32:0/1248836844 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9fc007b100 con 0x7f9fc80733a0 2024-04-24T22:59:15.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.537+0000 7f06a082b640 1 -- 172.21.15.32:0/2381135675 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f069c151ee0 con 0x7f069c0cde20 2024-04-24T22:59:15.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.537+0000 7f06837fe640 1 -- 172.21.15.32:0/2381135675 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06900035f0 con 0x7f069c0cde20 2024-04-24T22:59:15.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.537+0000 7f06837fe640 1 -- 172.21.15.32:0/2381135675 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0690003a20 con 0x7f069c0cde20 2024-04-24T22:59:15.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.537+0000 7f3921d9b640 1 -- 172.21.15.32:0/1392388926 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f390c0053c0 msgr2=0x7f39240ca350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:15.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.537+0000 7f3921d9b640 1 --2- 172.21.15.32:0/1392388926 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f390c0053c0 0x7f39240ca350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.537+0000 7f3921d9b640 1 -- 172.21.15.32:0/1392388926 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f39241f34a0 msgr2=0x7f39241f3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:15.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.537+0000 7f3921d9b640 1 --2- 172.21.15.32:0/1392388926 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f39241f34a0 0x7f39241f3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.537+0000 7f3921d9b640 1 -- 172.21.15.32:0/1392388926 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3924148fc0 con 0x7f3924148340 2024-04-24T22:59:15.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.537+0000 7f3921d9b640 1 --2- 172.21.15.32:0/1392388926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3924148340 0x7f39241487a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3918001050 tx=0x7f391800dc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:15.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.537+0000 7f81627fc640 1 --2- 172.21.15.32:0/3649358836 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f815006be80 0x7f815006e340 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f815c04fc20 tx=0x7f8158009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:15.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.538+0000 7f06a082b640 1 -- 172.21.15.32:0/2381135675 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f067c005290 con 0x7f069c0cde20 2024-04-24T22:59:15.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.539+0000 7f8e1dffb640 1 -- 172.21.15.32:0/2544406087 <== mon.2 v2:172.21.15.121:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8e24058d60 con 0x7f8e280b23e0 2024-04-24T22:59:15.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.539+0000 7f06837fe640 1 -- 172.21.15.32:0/2381135675 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f069001a070 con 0x7f069c0cde20 2024-04-24T22:59:15.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.539+0000 7f06837fe640 1 --2- 172.21.15.32:0/2381135675 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f068806be30 0x7f068806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:15.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.539+0000 7f06837fe640 1 -- 172.21.15.32:0/2381135675 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f069008a120 con 0x7f069c0cde20 2024-04-24T22:59:15.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.539+0000 7f069ad9d640 1 --2- 172.21.15.32:0/2381135675 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f068806be30 0x7f068806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:15.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.540+0000 7f069ad9d640 1 --2- 172.21.15.32:0/2381135675 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f068806be30 0x7f068806e2f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f069c0b1980 tx=0x7f068c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:15.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.540+0000 7f814b7fe640 1 -- 172.21.15.32:0/3649358836 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f815401f020 con 0x7f815c053a90 2024-04-24T22:59:15.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.541+0000 7f54ae7fc640 1 -- 172.21.15.32:0/4258578206 <== mon.2 v2:172.21.15.121:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f54b4007830 con 0x7f54b806d350 2024-04-24T22:59:15.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.541+0000 7f54ae7fc640 1 --2- 172.21.15.32:0/4258578206 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f549c06bf00 0x7f549c06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:15.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.541+0000 7f54ae7fc640 1 -- 172.21.15.32:0/4258578206 <== mon.2 v2:172.21.15.121:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f54b408b130 con 0x7f54b806d350 2024-04-24T22:59:15.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.542+0000 7f54bd6dd640 1 --2- 172.21.15.32:0/4258578206 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f549c06bf00 0x7f549c06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:15.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.542+0000 7f54bd6dd640 1 --2- 172.21.15.32:0/4258578206 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f549c06bf00 0x7f549c06e3c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f54a8034000 tx=0x7f54a802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:15.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.544+0000 7f06837fe640 1 -- 172.21.15.32:0/2381135675 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0690058480 con 0x7f069c0cde20 2024-04-24T22:59:15.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.544+0000 7f54ae7fc640 1 -- 172.21.15.32:0/4258578206 <== mon.2 v2:172.21.15.121:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f54b4059500 con 0x7f54b806d350 2024-04-24T22:59:15.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.545+0000 7f3912ffd640 1 -- 172.21.15.32:0/1392388926 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3918005480 con 0x7f3924148340 2024-04-24T22:59:15.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.545+0000 7f3912ffd640 1 -- 172.21.15.32:0/1392388926 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3918017070 con 0x7f3924148340 2024-04-24T22:59:15.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.545+0000 7f3912ffd640 1 -- 172.21.15.32:0/1392388926 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391800b850 con 0x7f3924148340 2024-04-24T22:59:15.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.545+0000 7f3928934640 1 -- 172.21.15.32:0/1392388926 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39241492a0 con 0x7f3924148340 2024-04-24T22:59:15.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.546+0000 7f1e127fc640 1 -- 172.21.15.32:0/2380961624 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1e1c0c8030 con 0x7f1e14053760 2024-04-24T22:59:15.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.545+0000 7f3928934640 1 -- 172.21.15.32:0/1392388926 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3924149860 con 0x7f3924148340 2024-04-24T22:59:15.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.547+0000 7f3910ff9640 1 -- 172.21.15.32:0/1392388926 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3924068430 con 0x7f3924148340 2024-04-24T22:59:15.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.558+0000 7f3912ffd640 1 -- 172.21.15.32:0/1392388926 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f391800b9f0 con 0x7f3924148340 2024-04-24T22:59:15.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.558+0000 7f3912ffd640 1 --2- 172.21.15.32:0/1392388926 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f390406bfd0 0x7f390406e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:15.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.559+0000 7f3912ffd640 1 -- 172.21.15.32:0/1392388926 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f391808b9b0 con 0x7f3924148340 2024-04-24T22:59:15.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.559+0000 7f3912ffd640 1 -- 172.21.15.32:0/1392388926 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f39180b9370 con 0x7f3924148340 2024-04-24T22:59:15.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.566+0000 7f392259c640 1 --2- 172.21.15.32:0/1392388926 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f390406bfd0 0x7f390406e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:15.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.575+0000 7f392259c640 1 --2- 172.21.15.32:0/1392388926 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f390406bfd0 0x7f390406e490 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3914001580 tx=0x7f3914007d90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:15.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.902+0000 7fbf897e7640 1 -- 172.21.15.32:0/1453054098 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fbf60005580 con 0x7fbf7c0e7690 2024-04-24T22:59:15.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.903+0000 7fbf737fe640 1 -- 172.21.15.32:0/1453054098 <== mon.2 v2:172.21.15.121:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbf84070040 con 0x7fbf7c0e7690 2024-04-24T22:59:15.904 INFO:teuthology.orchestra.run.smithi032.stdout:55834574854 2024-04-24T22:59:15.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.906+0000 7fbf717fa640 1 -- 172.21.15.32:0/1453054098 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fbf6806bf00 msgr2=0x7fbf6806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:15.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.906+0000 7fbf717fa640 1 --2- 172.21.15.32:0/1453054098 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fbf6806bf00 0x7fbf6806e3c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c038e30 tx=0x7fbf78006cc0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.906+0000 7fbf717fa640 1 -- 172.21.15.32:0/1453054098 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf7c0e7690 msgr2=0x7fbf7c039f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:15.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.906+0000 7fbf717fa640 1 --2- 172.21.15.32:0/1453054098 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf7c0e7690 0x7fbf7c039f30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbf8402d700 tx=0x7fbf84069030 comp rx=0 tx=0).stop 2024-04-24T22:59:15.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.906+0000 7fbf717fa640 1 -- 172.21.15.32:0/1453054098 shutdown_connections 2024-04-24T22:59:15.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.907+0000 7fbf717fa640 1 --2- 172.21.15.32:0/1453054098 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fbf6806bf00 0x7fbf6806e3c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.907+0000 7fbf717fa640 1 --2- 172.21.15.32:0/1453054098 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf7c0e7690 0x7fbf7c039f30 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.907+0000 7fbf717fa640 1 --2- 172.21.15.32:0/1453054098 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf7c04a3e0 0x7fbf7c0399f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.907+0000 7fbf717fa640 1 --2- 172.21.15.32:0/1453054098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf7c038e60 0x7fbf7c0d5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.907+0000 7fbf717fa640 1 -- 172.21.15.32:0/1453054098 >> 172.21.15.32:0/1453054098 conn(0x7fbf7c053750 msgr2=0x7fbf7c04b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:15.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.907+0000 7fbf717fa640 1 -- 172.21.15.32:0/1453054098 shutdown_connections 2024-04-24T22:59:15.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.907+0000 7fbf717fa640 1 -- 172.21.15.32:0/1453054098 wait complete. 2024-04-24T22:59:15.925 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.1 2024-04-24T22:59:15.926 DEBUG:teuthology.parallel:result is None 2024-04-24T22:59:15.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.943+0000 7f54c0142640 1 -- 172.21.15.32:0/4258578206 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f54b81585e0 con 0x7f54b806d350 2024-04-24T22:59:15.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.944+0000 7f54ae7fc640 1 -- 172.21.15.32:0/4258578206 <== mon.2 v2:172.21.15.121:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f54b405d800 con 0x7f54b806d350 2024-04-24T22:59:15.946 INFO:teuthology.orchestra.run.smithi032.stdout:55834574854 2024-04-24T22:59:15.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.946+0000 7f54c0142640 1 -- 172.21.15.32:0/4258578206 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f549c06bf00 msgr2=0x7f549c06e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:15.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.946+0000 7f54c0142640 1 --2- 172.21.15.32:0/4258578206 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f549c06bf00 0x7f549c06e3c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f54a8034000 tx=0x7f54a802e000 comp rx=0 tx=0).stop 2024-04-24T22:59:15.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.946+0000 7f54c0142640 1 -- 172.21.15.32:0/4258578206 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f54b806d350 msgr2=0x7f54b8150fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:15.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.946+0000 7f54c0142640 1 --2- 172.21.15.32:0/4258578206 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f54b806d350 0x7f54b8150fe0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f54b400c820 tx=0x7f54b400cce0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.946+0000 7f54c0142640 1 -- 172.21.15.32:0/4258578206 shutdown_connections 2024-04-24T22:59:15.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.946+0000 7f54c0142640 1 --2- 172.21.15.32:0/4258578206 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f549c06bf00 0x7f549c06e3c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.946+0000 7f54c0142640 1 --2- 172.21.15.32:0/4258578206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54b815c690 0x7f54b814a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.946+0000 7f54c0142640 1 --2- 172.21.15.32:0/4258578206 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54b80b3f70 0x7f54b8155020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.946+0000 7f54c0142640 1 --2- 172.21.15.32:0/4258578206 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f54b806d350 0x7f54b8150fe0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.946+0000 7f54c0142640 1 -- 172.21.15.32:0/4258578206 >> 172.21.15.32:0/4258578206 conn(0x7f54b806b110 msgr2=0x7f54b815b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:15.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.946+0000 7f54c0142640 1 -- 172.21.15.32:0/4258578206 shutdown_connections 2024-04-24T22:59:15.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.946+0000 7f54c0142640 1 -- 172.21.15.32:0/4258578206 wait complete. 2024-04-24T22:59:15.959 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2024-04-24T22:59:15.959 DEBUG:teuthology.parallel:result is None 2024-04-24T22:59:15.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.970+0000 7f8e2f9f4640 1 -- 172.21.15.32:0/2544406087 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f8e280ce210 con 0x7f8e280b23e0 2024-04-24T22:59:15.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.970+0000 7f8e1dffb640 1 -- 172.21.15.32:0/2544406087 <== mon.2 v2:172.21.15.121:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8e24058660 con 0x7f8e280b23e0 2024-04-24T22:59:15.972 INFO:teuthology.orchestra.run.smithi032.stdout:55834574854 2024-04-24T22:59:15.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.971+0000 7f9fcf30e640 1 -- 172.21.15.32:0/1248836844 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f9fc80b5bd0 con 0x7f9fc80733a0 2024-04-24T22:59:15.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.971+0000 7f8163fff640 1 -- 172.21.15.32:0/3649358836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f815c0dc380 con 0x7f815c053a90 2024-04-24T22:59:15.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.972+0000 7f9fbeffd640 1 -- 172.21.15.32:0/1248836844 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9fc007f400 con 0x7f9fc80733a0 2024-04-24T22:59:15.973 INFO:teuthology.orchestra.run.smithi032.stdout:55834574854 2024-04-24T22:59:15.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.972+0000 7f814b7fe640 1 -- 172.21.15.32:0/3649358836 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f815400d040 con 0x7f815c053a90 2024-04-24T22:59:15.974 INFO:teuthology.orchestra.run.smithi032.stdout:55834574854 2024-04-24T22:59:15.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.973+0000 7f8e2f9f4640 1 -- 172.21.15.32:0/2544406087 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8e0c06bf00 msgr2=0x7f8e0c06e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:15.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.973+0000 7f8e2f9f4640 1 --2- 172.21.15.32:0/2544406087 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8e0c06bf00 0x7f8e0c06e3c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8e18034000 tx=0x7f8e1802e000 comp rx=0 tx=0).stop 2024-04-24T22:59:15.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.973+0000 7f8e2f9f4640 1 -- 172.21.15.32:0/2544406087 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8e280b23e0 msgr2=0x7f8e2815d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:15.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.973+0000 7f8e2f9f4640 1 --2- 172.21.15.32:0/2544406087 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8e280b23e0 0x7f8e2815d960 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8e2400c570 tx=0x7f8e2400ca30 comp rx=0 tx=0).stop 2024-04-24T22:59:15.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.974+0000 7f8e2f9f4640 1 -- 172.21.15.32:0/2544406087 shutdown_connections 2024-04-24T22:59:15.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.974+0000 7f8e2f9f4640 1 --2- 172.21.15.32:0/2544406087 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8e0c06bf00 0x7f8e0c06e3c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.974+0000 7f8e2f9f4640 1 --2- 172.21.15.32:0/2544406087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e2815c430 0x7f8e2815a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.974+0000 7f8e2f9f4640 1 --2- 172.21.15.32:0/2544406087 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e2816b430 0x7f8e28159f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.974+0000 7f8e2f9f4640 1 --2- 172.21.15.32:0/2544406087 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8e280b23e0 0x7f8e2815d960 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.974+0000 7f8e2f9f4640 1 -- 172.21.15.32:0/2544406087 >> 172.21.15.32:0/2544406087 conn(0x7f8e280743f0 msgr2=0x7f8e2816dce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:15.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.974+0000 7f8e2f9f4640 1 -- 172.21.15.32:0/2544406087 shutdown_connections 2024-04-24T22:59:15.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.974+0000 7f8e2f9f4640 1 -- 172.21.15.32:0/2544406087 wait complete. 2024-04-24T22:59:15.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.975+0000 7f9fcf30e640 1 -- 172.21.15.32:0/1248836844 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9fac06be30 msgr2=0x7f9fac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:15.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.975+0000 7f9fcf30e640 1 --2- 172.21.15.32:0/1248836844 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9fac06be30 0x7f9fac06e2f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8009a30 tx=0x7f9fb8009190 comp rx=0 tx=0).stop 2024-04-24T22:59:15.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.975+0000 7f8163fff640 1 -- 172.21.15.32:0/3649358836 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f815006be80 msgr2=0x7f815006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:15.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.975+0000 7f8163fff640 1 --2- 172.21.15.32:0/3649358836 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f815006be80 0x7f815006e340 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f815c04fc20 tx=0x7f8158009040 comp rx=0 tx=0).stop 2024-04-24T22:59:15.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.975+0000 7f8163fff640 1 -- 172.21.15.32:0/3649358836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f815c053a90 msgr2=0x7f815c0d3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:15.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.975+0000 7f8163fff640 1 --2- 172.21.15.32:0/3649358836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f815c053a90 0x7f815c0d3b00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f815400b450 tx=0x7f815400b910 comp rx=0 tx=0).stop 2024-04-24T22:59:15.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.975+0000 7f8163fff640 1 -- 172.21.15.32:0/3649358836 shutdown_connections 2024-04-24T22:59:15.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.975+0000 7f9fcf30e640 1 -- 172.21.15.32:0/1248836844 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9fc80733a0 msgr2=0x7f9fc8072cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:15.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.975+0000 7f8163fff640 1 --2- 172.21.15.32:0/3649358836 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f815006be80 0x7f815006e340 secure :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f815c04fc20 tx=0x7f8158009040 comp rx=0 tx=0).stop 2024-04-24T22:59:15.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.975+0000 7f8163fff640 1 --2- 172.21.15.32:0/3649358836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f815c054980 0x7f815c0e2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.975+0000 7f8163fff640 1 --2- 172.21.15.32:0/3649358836 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f815c053e70 0x7f815c0d23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.975+0000 7f8163fff640 1 --2- 172.21.15.32:0/3649358836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f815c053a90 0x7f815c0d3b00 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.975+0000 7f8163fff640 1 -- 172.21.15.32:0/3649358836 >> 172.21.15.32:0/3649358836 conn(0x7f815c055c50 msgr2=0x7f815c0da830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:15.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.975+0000 7f9fcf30e640 1 --2- 172.21.15.32:0/1248836844 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9fc80733a0 0x7f9fc8072cd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0031000 tx=0x7f9fc0004a40 comp rx=0 tx=0).stop 2024-04-24T22:59:15.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.975+0000 7f9fcf30e640 1 -- 172.21.15.32:0/1248836844 shutdown_connections 2024-04-24T22:59:15.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.975+0000 7f9fcf30e640 1 --2- 172.21.15.32:0/1248836844 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9fac06be30 0x7f9fac06e2f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.975+0000 7f9fcf30e640 1 --2- 172.21.15.32:0/1248836844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fc80ce360 0x7f9fc8072790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.975+0000 7f9fcf30e640 1 --2- 172.21.15.32:0/1248836844 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9fc80733a0 0x7f9fc8072cd0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.975+0000 7f9fcf30e640 1 --2- 172.21.15.32:0/1248836844 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9fc80cd550 0x7f9fc8158e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.975+0000 7f9fcf30e640 1 -- 172.21.15.32:0/1248836844 >> 172.21.15.32:0/1248836844 conn(0x7f9fc80b1f30 msgr2=0x7f9fc816bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:15.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.975+0000 7f9fcf30e640 1 -- 172.21.15.32:0/1248836844 shutdown_connections 2024-04-24T22:59:15.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.975+0000 7f9fcf30e640 1 -- 172.21.15.32:0/1248836844 wait complete. 2024-04-24T22:59:15.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.975+0000 7f8163fff640 1 -- 172.21.15.32:0/3649358836 shutdown_connections 2024-04-24T22:59:15.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.976+0000 7f8163fff640 1 -- 172.21.15.32:0/3649358836 wait complete. 2024-04-24T22:59:15.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.978+0000 7f1e07fff640 1 -- 172.21.15.32:0/2380961624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f1e14056c90 con 0x7f1e14053760 2024-04-24T22:59:15.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.979+0000 7f06a082b640 1 -- 172.21.15.32:0/2381135675 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f067c005580 con 0x7f069c0cde20 2024-04-24T22:59:15.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.979+0000 7f1e127fc640 1 -- 172.21.15.32:0/2380961624 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1e1c070040 con 0x7f1e14053760 2024-04-24T22:59:15.983 INFO:teuthology.orchestra.run.smithi032.stdout:55834574854 2024-04-24T22:59:15.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.979+0000 7f06837fe640 1 -- 172.21.15.32:0/2381135675 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f069005c780 con 0x7f069c0cde20 2024-04-24T22:59:15.984 INFO:teuthology.orchestra.run.smithi032.stdout:55834574854 2024-04-24T22:59:15.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.982+0000 7f1e07fff640 1 -- 172.21.15.32:0/2380961624 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1e0006bf00 msgr2=0x7f1e0006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:15.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.982+0000 7f1e07fff640 1 --2- 172.21.15.32:0/2380961624 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1e0006bf00 0x7f1e0006e3c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1e14050c20 tx=0x7f1e0c009190 comp rx=0 tx=0).stop 2024-04-24T22:59:15.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.982+0000 7f1e07fff640 1 -- 172.21.15.32:0/2380961624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e14053760 msgr2=0x7f1e14178990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:15.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.982+0000 7f1e07fff640 1 --2- 172.21.15.32:0/2380961624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e14053760 0x7f1e14178990 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c0699e0 tx=0x7f1e1c069d50 comp rx=0 tx=0).stop 2024-04-24T22:59:15.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.982+0000 7f1e07fff640 1 -- 172.21.15.32:0/2380961624 shutdown_connections 2024-04-24T22:59:15.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.982+0000 7f1e07fff640 1 --2- 172.21.15.32:0/2380961624 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1e0006bf00 0x7f1e0006e3c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.982+0000 7f1e07fff640 1 --2- 172.21.15.32:0/2380961624 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e140e6cf0 0x7f1e1404dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.982+0000 7f1e07fff640 1 --2- 172.21.15.32:0/2380961624 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1e140e3290 0x7f1e1404dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.982+0000 7f1e07fff640 1 --2- 172.21.15.32:0/2380961624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e14053760 0x7f1e14178990 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.982+0000 7f1e07fff640 1 -- 172.21.15.32:0/2380961624 >> 172.21.15.32:0/2380961624 conn(0x7f1e14055990 msgr2=0x7f1e14055d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:15.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.982+0000 7f1e07fff640 1 -- 172.21.15.32:0/2380961624 shutdown_connections 2024-04-24T22:59:15.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.983+0000 7f1e07fff640 1 -- 172.21.15.32:0/2380961624 wait complete. 2024-04-24T22:59:15.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.983+0000 7f06a082b640 1 -- 172.21.15.32:0/2381135675 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f068806be30 msgr2=0x7f068806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:15.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.983+0000 7f06a082b640 1 --2- 172.21.15.32:0/2381135675 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f068806be30 0x7f068806e2f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f069c0b1980 tx=0x7f068c008040 comp rx=0 tx=0).stop 2024-04-24T22:59:15.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.983+0000 7f06a082b640 1 -- 172.21.15.32:0/2381135675 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f069c0cde20 msgr2=0x7f069c150ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:15.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.983+0000 7f06a082b640 1 --2- 172.21.15.32:0/2381135675 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f069c0cde20 0x7f069c150ed0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f069000c430 tx=0x7f069000c8f0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.983+0000 7f06a082b640 1 -- 172.21.15.32:0/2381135675 shutdown_connections 2024-04-24T22:59:15.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.983+0000 7f06a082b640 1 --2- 172.21.15.32:0/2381135675 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f068806be30 0x7f068806e2f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.983+0000 7f06a082b640 1 --2- 172.21.15.32:0/2381135675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f069c15e700 0x7f069c1530f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.983+0000 7f06a082b640 1 --2- 172.21.15.32:0/2381135675 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f069c0cde20 0x7f069c150ed0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.983+0000 7f06a082b640 1 --2- 172.21.15.32:0/2381135675 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f069c0718a0 0x7f069c150310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.983+0000 7f06a082b640 1 -- 172.21.15.32:0/2381135675 >> 172.21.15.32:0/2381135675 conn(0x7f069c0b2160 msgr2=0x7f069c160cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:15.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.983+0000 7f06a082b640 1 -- 172.21.15.32:0/2381135675 shutdown_connections 2024-04-24T22:59:15.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.983+0000 7f06a082b640 1 -- 172.21.15.32:0/2381135675 wait complete. 2024-04-24T22:59:15.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.986+0000 7f3910ff9640 1 -- 172.21.15.32:0/1392388926 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f3924148ce0 con 0x7f3924148340 2024-04-24T22:59:15.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.986+0000 7f3912ffd640 1 -- 172.21.15.32:0/1392388926 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3918059be0 con 0x7f3924148340 2024-04-24T22:59:15.988 INFO:teuthology.orchestra.run.smithi032.stdout:55834574854 2024-04-24T22:59:15.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.989+0000 7f3910ff9640 1 -- 172.21.15.32:0/1392388926 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f390406bfd0 msgr2=0x7f390406e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:15.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.989+0000 7f3910ff9640 1 --2- 172.21.15.32:0/1392388926 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f390406bfd0 0x7f390406e490 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3914001580 tx=0x7f3914007d90 comp rx=0 tx=0).stop 2024-04-24T22:59:15.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.989+0000 7f3910ff9640 1 -- 172.21.15.32:0/1392388926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3924148340 msgr2=0x7f39241487a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:15.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.989+0000 7f3910ff9640 1 --2- 172.21.15.32:0/1392388926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3924148340 0x7f39241487a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3918001050 tx=0x7f391800dc50 comp rx=0 tx=0).stop 2024-04-24T22:59:15.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.990+0000 7f3910ff9640 1 -- 172.21.15.32:0/1392388926 shutdown_connections 2024-04-24T22:59:15.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.990+0000 7f3910ff9640 1 --2- 172.21.15.32:0/1392388926 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f390406bfd0 0x7f390406e490 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.990+0000 7f3910ff9640 1 --2- 172.21.15.32:0/1392388926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3924148340 0x7f39241487a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.990+0000 7f3910ff9640 1 --2- 172.21.15.32:0/1392388926 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f39241f34a0 0x7f39241f3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.990+0000 7f3910ff9640 1 --2- 172.21.15.32:0/1392388926 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f390c0053c0 0x7f39240ca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:15.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.990+0000 7f3910ff9640 1 -- 172.21.15.32:0/1392388926 >> 172.21.15.32:0/1392388926 conn(0x7f39240d11d0 msgr2=0x7f392415c600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:15.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.990+0000 7f3910ff9640 1 -- 172.21.15.32:0/1392388926 shutdown_connections 2024-04-24T22:59:15.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:15.990+0000 7f3910ff9640 1 -- 172.21.15.32:0/1392388926 wait complete. 2024-04-24T22:59:15.994 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.2 2024-04-24T22:59:15.994 DEBUG:teuthology.parallel:result is None 2024-04-24T22:59:15.997 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.7 2024-04-24T22:59:15.998 DEBUG:teuthology.parallel:result is None 2024-04-24T22:59:15.998 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.6 2024-04-24T22:59:15.998 DEBUG:teuthology.parallel:result is None 2024-04-24T22:59:16.003 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.5 2024-04-24T22:59:16.003 DEBUG:teuthology.parallel:result is None 2024-04-24T22:59:16.003 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.4 2024-04-24T22:59:16.004 DEBUG:teuthology.parallel:result is None 2024-04-24T22:59:16.004 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2024-04-24T22:59:16.004 DEBUG:teuthology.parallel:result is None 2024-04-24T22:59:16.005 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-24T22:59:16.005 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T22:59:16.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.143+0000 7f6ba55e7640 1 Processor -- start 2024-04-24T22:59:16.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.143+0000 7f6ba55e7640 1 -- start start 2024-04-24T22:59:16.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.144+0000 7f6ba55e7640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6ba0158db0 0x7f6ba00b6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:16.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.144+0000 7f6ba55e7640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ba00730c0 0x7f6ba015f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:16.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.144+0000 7f6ba55e7640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ba0159190 con 0x7f6ba006e800 2024-04-24T22:59:16.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.144+0000 7f6ba55e7640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f6ba00b7230 con 0x7f6ba0158db0 2024-04-24T22:59:16.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.144+0000 7f6ba55e7640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f6ba00b7510 con 0x7f6ba00730c0 2024-04-24T22:59:16.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.144+0000 7f6b9effd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6ba0158db0 0x7f6ba00b6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:16.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.144+0000 7f6b9effd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6ba0158db0 0x7f6ba00b6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38768/0 (socket says 172.21.15.32:38768) 2024-04-24T22:59:16.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.144+0000 7f6b9effd640 1 -- 172.21.15.32:0/3457025687 learned_addr learned my addr 172.21.15.32:0/3457025687 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:16.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.144+0000 7f6b9f7fe640 1 --1- 172.21.15.32:0/3457025687 >> v1:172.21.15.32:6789/0 conn(0x7f6ba006e800 0x7f6ba0162370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47482/0 (socket says 172.21.15.32:47482) 2024-04-24T22:59:16.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.144+0000 7f6b9e7fc640 1 --2- 172.21.15.32:0/3457025687 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ba00730c0 0x7f6ba015f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:16.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.145+0000 7f6b9effd640 1 -- 172.21.15.32:0/3457025687 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ba00730c0 msgr2=0x7f6ba015f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:16.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.145+0000 7f6b9effd640 1 --2- 172.21.15.32:0/3457025687 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ba00730c0 0x7f6ba015f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:16.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.145+0000 7f6b9effd640 1 -- 172.21.15.32:0/3457025687 >> v1:172.21.15.32:6789/0 conn(0x7f6ba006e800 legacy=0x7f6ba0162370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:16.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.145+0000 7f6b9effd640 1 -- 172.21.15.32:0/3457025687 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba00b77f0 con 0x7f6ba0158db0 2024-04-24T22:59:16.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.145+0000 7f6b9dffb640 1 -- 172.21.15.32:0/3457025687 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3055427059 0 0) 0x7f6ba0159190 con 0x7f6ba006e800 2024-04-24T22:59:16.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.145+0000 7f6b9effd640 1 --2- 172.21.15.32:0/3457025687 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ba0158db0 0x7f6ba00b6bb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6b98009870 tx=0x7f6b9802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b9b699fa1fadd3a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:16.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.146+0000 7f6b9dffb640 1 -- 172.21.15.32:0/3457025687 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98037070 con 0x7f6ba0158db0 2024-04-24T22:59:16.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.146+0000 7f6b9dffb640 1 -- 172.21.15.32:0/3457025687 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b98032440 con 0x7f6ba0158db0 2024-04-24T22:59:16.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.146+0000 7f6b9dffb640 1 -- 172.21.15.32:0/3457025687 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98032810 con 0x7f6ba0158db0 2024-04-24T22:59:16.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.146+0000 7f6ba55e7640 1 -- 172.21.15.32:0/3457025687 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ba0158db0 msgr2=0x7f6ba00b6bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:16.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.146+0000 7f6ba55e7640 1 --2- 172.21.15.32:0/3457025687 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ba0158db0 0x7f6ba00b6bb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6b98009870 tx=0x7f6b9802a270 comp rx=0 tx=0).stop 2024-04-24T22:59:16.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.146+0000 7f6ba55e7640 1 -- 172.21.15.32:0/3457025687 shutdown_connections 2024-04-24T22:59:16.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.146+0000 7f6ba55e7640 1 --2- 172.21.15.32:0/3457025687 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ba00730c0 0x7f6ba015f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:16.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.146+0000 7f6ba55e7640 1 --2- 172.21.15.32:0/3457025687 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ba0158db0 0x7f6ba00b6bb0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:16.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.146+0000 7f6ba55e7640 1 -- 172.21.15.32:0/3457025687 >> 172.21.15.32:0/3457025687 conn(0x7f6ba00d24c0 msgr2=0x7f6ba00d28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:16.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.146+0000 7f6ba55e7640 1 -- 172.21.15.32:0/3457025687 shutdown_connections 2024-04-24T22:59:16.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.146+0000 7f6ba55e7640 1 -- 172.21.15.32:0/3457025687 wait complete. 2024-04-24T22:59:16.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.147+0000 7f6ba55e7640 1 Processor -- start 2024-04-24T22:59:16.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.147+0000 7f6ba55e7640 1 -- start start 2024-04-24T22:59:16.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.148+0000 7f6ba55e7640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ba006e800 0x7f6ba01f3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:16.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.148+0000 7f6ba55e7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ba00730c0 0x7f6ba0160e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:16.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.148+0000 7f6ba55e7640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ba0158db0 0x7f6ba01613a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:16.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.148+0000 7f6ba55e7640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba0073540 con 0x7f6ba00730c0 2024-04-24T22:59:16.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.148+0000 7f6ba55e7640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba015ed80 con 0x7f6ba0158db0 2024-04-24T22:59:16.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.148+0000 7f6ba55e7640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f6ba015fb30 con 0x7f6ba006e800 2024-04-24T22:59:16.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.148+0000 7f6b9e7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ba00730c0 0x7f6ba0160e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:16.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.148+0000 7f6b9e7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ba00730c0 0x7f6ba0160e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38778/0 (socket says 172.21.15.32:38778) 2024-04-24T22:59:16.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.148+0000 7f6b9e7fc640 1 -- 172.21.15.32:0/1134964773 learned_addr learned my addr 172.21.15.32:0/1134964773 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:16.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.148+0000 7f6b9effd640 1 --2- 172.21.15.32:0/1134964773 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ba006e800 0x7f6ba01f3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:16.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.149+0000 7f6b9e7fc640 1 -- 172.21.15.32:0/1134964773 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ba006e800 msgr2=0x7f6ba01f3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:16.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.149+0000 7f6b9f7fe640 1 --2- 172.21.15.32:0/1134964773 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ba0158db0 0x7f6ba01613a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:16.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.149+0000 7f6b9e7fc640 1 --2- 172.21.15.32:0/1134964773 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ba006e800 0x7f6ba01f3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:16.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.149+0000 7f6b9e7fc640 1 -- 172.21.15.32:0/1134964773 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ba0158db0 msgr2=0x7f6ba01613a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:16.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.149+0000 7f6b9e7fc640 1 --2- 172.21.15.32:0/1134964773 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ba0158db0 0x7f6ba01613a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:16.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.149+0000 7f6b9e7fc640 1 -- 172.21.15.32:0/1134964773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba0153430 con 0x7f6ba00730c0 2024-04-24T22:59:16.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.149+0000 7f6b9effd640 1 --2- 172.21.15.32:0/1134964773 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ba006e800 0x7f6ba01f3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:59:16.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.149+0000 7f6b9e7fc640 1 --2- 172.21.15.32:0/1134964773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ba00730c0 0x7f6ba0160e60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6b8800e9e0 tx=0x7f6b8800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:16.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.149+0000 7f6b977fe640 1 -- 172.21.15.32:0/1134964773 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b88017070 con 0x7f6ba00730c0 2024-04-24T22:59:16.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.149+0000 7f6b977fe640 1 -- 172.21.15.32:0/1134964773 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b8800c9d0 con 0x7f6ba00730c0 2024-04-24T22:59:16.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.149+0000 7f6ba55e7640 1 -- 172.21.15.32:0/1134964773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ba0153710 con 0x7f6ba00730c0 2024-04-24T22:59:16.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.149+0000 7f6b977fe640 1 -- 172.21.15.32:0/1134964773 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8800cdc0 con 0x7f6ba00730c0 2024-04-24T22:59:16.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.149+0000 7f6ba55e7640 1 -- 172.21.15.32:0/1134964773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ba0153cd0 con 0x7f6ba00730c0 2024-04-24T22:59:16.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.150+0000 7f6ba55e7640 1 -- 172.21.15.32:0/1134964773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ba0068430 con 0x7f6ba00730c0 2024-04-24T22:59:16.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.150+0000 7f6b977fe640 1 -- 172.21.15.32:0/1134964773 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6b8801a070 con 0x7f6ba00730c0 2024-04-24T22:59:16.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.151+0000 7f6b977fe640 1 --2- 172.21.15.32:0/1134964773 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6b8006be30 0x7f6b8006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:16.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.151+0000 7f6b977fe640 1 -- 172.21.15.32:0/1134964773 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f6b8808b1e0 con 0x7f6ba00730c0 2024-04-24T22:59:16.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.151+0000 7f6b9effd640 1 --2- 172.21.15.32:0/1134964773 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6b8006be30 0x7f6b8006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:16.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.151+0000 7f6b9effd640 1 --2- 172.21.15.32:0/1134964773 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6b8006be30 0x7f6b8006e2f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6b98034000 tx=0x7f6b9802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:16.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.155+0000 7f6b977fe640 1 -- 172.21.15.32:0/1134964773 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6b88059b90 con 0x7f6ba00730c0 2024-04-24T22:59:16.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.335+0000 7f6ba55e7640 1 -- 172.21.15.32:0/1134964773 --> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6ba0072a40 con 0x7f6b8006be30 2024-04-24T22:59:16.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.341+0000 7f6b977fe640 1 -- 172.21.15.32:0/1134964773 <== mgr.4108 v2:172.21.15.121:6832/1848157943 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247486 (secure 0 0 0) 0x7f6ba0072a40 con 0x7f6b8006be30 2024-04-24T22:59:16.345 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:59:16.346 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-24T22:59:16.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.344+0000 7f6ba55e7640 1 -- 172.21.15.32:0/1134964773 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6b8006be30 msgr2=0x7f6b8006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:16.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.344+0000 7f6ba55e7640 1 --2- 172.21.15.32:0/1134964773 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6b8006be30 0x7f6b8006e2f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6b98034000 tx=0x7f6b9802e000 comp rx=0 tx=0).stop 2024-04-24T22:59:16.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.344+0000 7f6ba55e7640 1 -- 172.21.15.32:0/1134964773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ba00730c0 msgr2=0x7f6ba0160e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:16.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.344+0000 7f6ba55e7640 1 --2- 172.21.15.32:0/1134964773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ba00730c0 0x7f6ba0160e60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6b8800e9e0 tx=0x7f6b8800eea0 comp rx=0 tx=0).stop 2024-04-24T22:59:16.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.344+0000 7f6ba55e7640 1 -- 172.21.15.32:0/1134964773 shutdown_connections 2024-04-24T22:59:16.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.344+0000 7f6ba55e7640 1 --2- 172.21.15.32:0/1134964773 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6b8006be30 0x7f6b8006e2f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:16.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.344+0000 7f6ba55e7640 1 --2- 172.21.15.32:0/1134964773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ba00730c0 0x7f6ba0160e60 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:16.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.344+0000 7f6ba55e7640 1 --2- 172.21.15.32:0/1134964773 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ba0158db0 0x7f6ba01613a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:16.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.344+0000 7f6ba55e7640 1 --2- 172.21.15.32:0/1134964773 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ba006e800 0x7f6ba01f3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:16.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.344+0000 7f6ba55e7640 1 -- 172.21.15.32:0/1134964773 >> 172.21.15.32:0/1134964773 conn(0x7f6ba00d24c0 msgr2=0x7f6ba006db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:16.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.344+0000 7f6ba55e7640 1 -- 172.21.15.32:0/1134964773 shutdown_connections 2024-04-24T22:59:16.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.344+0000 7f6ba55e7640 1 -- 172.21.15.32:0/1134964773 wait complete. 2024-04-24T22:59:16.357 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-24T22:59:15.130992+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1747,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560752,"statfs":{"total":1600000000,"available":1598210884,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002549"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072128+0000","last_change":"2024-04-24T22:59:04.533635+0000","last_active":"2024-04-24T22:59:06.072128+0000","last_peered":"2024-04-24T22:59:06.072128+0000","last_clean":"2024-04-24T22:59:06.072128+0000","last_became_active":"2024-04-24T22:59:04.533181+0000","last_became_peered":"2024-04-24T22:59:04.533181+0000","last_unstale":"2024-04-24T22:59:06.072128+0000","last_undegraded":"2024-04-24T22:59:06.072128+0000","last_fullsized":"2024-04-24T22:59:06.072128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:19:41.674286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.571341+0000","last_change":"2024-04-24T22:59:04.532345+0000","last_active":"2024-04-24T22:59:05.571341+0000","last_peered":"2024-04-24T22:59:05.571341+0000","last_clean":"2024-04-24T22:59:05.571341+0000","last_became_active":"2024-04-24T22:59:04.532126+0000","last_became_peered":"2024-04-24T22:59:04.532126+0000","last_unstale":"2024-04-24T22:59:05.571341+0000","last_undegraded":"2024-04-24T22:59:05.571341+0000","last_fullsized":"2024-04-24T22:59:05.571341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:40:58.083354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.571343+0000","last_change":"2024-04-24T22:59:04.535851+0000","last_active":"2024-04-24T22:59:05.571343+0000","last_peered":"2024-04-24T22:59:05.571343+0000","last_clean":"2024-04-24T22:59:05.571343+0000","last_became_active":"2024-04-24T22:59:04.535290+0000","last_became_peered":"2024-04-24T22:59:04.535290+0000","last_unstale":"2024-04-24T22:59:05.571343+0000","last_undegraded":"2024-04-24T22:59:05.571343+0000","last_fullsized":"2024-04-24T22:59:05.571343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:57:53.010612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.521146+0000","last_change":"2024-04-24T22:59:04.529616+0000","last_active":"2024-04-24T22:59:05.521146+0000","last_peered":"2024-04-24T22:59:05.521146+0000","last_clean":"2024-04-24T22:59:05.521146+0000","last_became_active":"2024-04-24T22:59:04.529312+0000","last_became_peered":"2024-04-24T22:59:04.529312+0000","last_unstale":"2024-04-24T22:59:05.521146+0000","last_undegraded":"2024-04-24T22:59:05.521146+0000","last_fullsized":"2024-04-24T22:59:05.521146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:55:40.732173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072449+0000","last_change":"2024-04-24T22:59:04.531771+0000","last_active":"2024-04-24T22:59:06.072449+0000","last_peered":"2024-04-24T22:59:06.072449+0000","last_clean":"2024-04-24T22:59:06.072449+0000","last_became_active":"2024-04-24T22:59:04.529534+0000","last_became_peered":"2024-04-24T22:59:04.529534+0000","last_unstale":"2024-04-24T22:59:06.072449+0000","last_undegraded":"2024-04-24T22:59:06.072449+0000","last_fullsized":"2024-04-24T22:59:06.072449+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:35:46.694552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.521763+0000","last_change":"2024-04-24T22:59:04.529880+0000","last_active":"2024-04-24T22:59:05.521763+0000","last_peered":"2024-04-24T22:59:05.521763+0000","last_clean":"2024-04-24T22:59:05.521763+0000","last_became_active":"2024-04-24T22:59:04.529596+0000","last_became_peered":"2024-04-24T22:59:04.529596+0000","last_unstale":"2024-04-24T22:59:05.521763+0000","last_undegraded":"2024-04-24T22:59:05.521763+0000","last_fullsized":"2024-04-24T22:59:05.521763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:45:47.457534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072563+0000","last_change":"2024-04-24T22:59:04.535003+0000","last_active":"2024-04-24T22:59:06.072563+0000","last_peered":"2024-04-24T22:59:06.072563+0000","last_clean":"2024-04-24T22:59:06.072563+0000","last_became_active":"2024-04-24T22:59:04.532802+0000","last_became_peered":"2024-04-24T22:59:04.532802+0000","last_unstale":"2024-04-24T22:59:06.072563+0000","last_undegraded":"2024-04-24T22:59:06.072563+0000","last_fullsized":"2024-04-24T22:59:06.072563+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:17.932086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.571441+0000","last_change":"2024-04-24T22:59:04.537484+0000","last_active":"2024-04-24T22:59:05.571441+0000","last_peered":"2024-04-24T22:59:05.571441+0000","last_clean":"2024-04-24T22:59:05.571441+0000","last_became_active":"2024-04-24T22:59:04.537018+0000","last_became_peered":"2024-04-24T22:59:04.537018+0000","last_unstale":"2024-04-24T22:59:05.571441+0000","last_undegraded":"2024-04-24T22:59:05.571441+0000","last_fullsized":"2024-04-24T22:59:05.571441+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:50:51.056976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.802481+0000","last_change":"2024-04-24T22:59:04.524856+0000","last_active":"2024-04-24T22:59:05.802481+0000","last_peered":"2024-04-24T22:59:05.802481+0000","last_clean":"2024-04-24T22:59:05.802481+0000","last_became_active":"2024-04-24T22:59:04.524572+0000","last_became_peered":"2024-04-24T22:59:04.524572+0000","last_unstale":"2024-04-24T22:59:05.802481+0000","last_undegraded":"2024-04-24T22:59:05.802481+0000","last_fullsized":"2024-04-24T22:59:05.802481+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:41.537600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072207+0000","last_change":"2024-04-24T22:59:04.533184+0000","last_active":"2024-04-24T22:59:06.072207+0000","last_peered":"2024-04-24T22:59:06.072207+0000","last_clean":"2024-04-24T22:59:06.072207+0000","last_became_active":"2024-04-24T22:59:04.532176+0000","last_became_peered":"2024-04-24T22:59:04.532176+0000","last_unstale":"2024-04-24T22:59:06.072207+0000","last_undegraded":"2024-04-24T22:59:06.072207+0000","last_fullsized":"2024-04-24T22:59:06.072207+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:43:05.960134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072278+0000","last_change":"2024-04-24T22:59:04.533428+0000","last_active":"2024-04-24T22:59:06.072278+0000","last_peered":"2024-04-24T22:59:06.072278+0000","last_clean":"2024-04-24T22:59:06.072278+0000","last_became_active":"2024-04-24T22:59:04.532740+0000","last_became_peered":"2024-04-24T22:59:04.532740+0000","last_unstale":"2024-04-24T22:59:06.072278+0000","last_undegraded":"2024-04-24T22:59:06.072278+0000","last_fullsized":"2024-04-24T22:59:06.072278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:45:08.272236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072073+0000","last_change":"2024-04-24T22:59:04.538772+0000","last_active":"2024-04-24T22:59:06.072073+0000","last_peered":"2024-04-24T22:59:06.072073+0000","last_clean":"2024-04-24T22:59:06.072073+0000","last_became_active":"2024-04-24T22:59:04.537512+0000","last_became_peered":"2024-04-24T22:59:04.537512+0000","last_unstale":"2024-04-24T22:59:06.072073+0000","last_undegraded":"2024-04-24T22:59:06.072073+0000","last_fullsized":"2024-04-24T22:59:06.072073+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:17.297900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520613+0000","last_change":"2024-04-24T22:59:04.534720+0000","last_active":"2024-04-24T22:59:05.520613+0000","last_peered":"2024-04-24T22:59:05.520613+0000","last_clean":"2024-04-24T22:59:05.520613+0000","last_became_active":"2024-04-24T22:59:04.533802+0000","last_became_peered":"2024-04-24T22:59:04.533802+0000","last_unstale":"2024-04-24T22:59:05.520613+0000","last_undegraded":"2024-04-24T22:59:05.520613+0000","last_fullsized":"2024-04-24T22:59:05.520613+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:21:36.150878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520463+0000","last_change":"2024-04-24T22:59:04.532980+0000","last_active":"2024-04-24T22:59:05.520463+0000","last_peered":"2024-04-24T22:59:05.520463+0000","last_clean":"2024-04-24T22:59:05.520463+0000","last_became_active":"2024-04-24T22:59:04.532721+0000","last_became_peered":"2024-04-24T22:59:04.532721+0000","last_unstale":"2024-04-24T22:59:05.520463+0000","last_undegraded":"2024-04-24T22:59:05.520463+0000","last_fullsized":"2024-04-24T22:59:05.520463+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:07:57.327727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.571539+0000","last_change":"2024-04-24T22:59:04.529907+0000","last_active":"2024-04-24T22:59:05.571539+0000","last_peered":"2024-04-24T22:59:05.571539+0000","last_clean":"2024-04-24T22:59:05.571539+0000","last_became_active":"2024-04-24T22:59:04.529501+0000","last_became_peered":"2024-04-24T22:59:04.529501+0000","last_unstale":"2024-04-24T22:59:05.571539+0000","last_undegraded":"2024-04-24T22:59:05.571539+0000","last_fullsized":"2024-04-24T22:59:05.571539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:44:58.795998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520176+0000","last_change":"2024-04-24T22:59:04.531489+0000","last_active":"2024-04-24T22:59:05.520176+0000","last_peered":"2024-04-24T22:59:05.520176+0000","last_clean":"2024-04-24T22:59:05.520176+0000","last_became_active":"2024-04-24T22:59:04.531251+0000","last_became_peered":"2024-04-24T22:59:04.531251+0000","last_unstale":"2024-04-24T22:59:05.520176+0000","last_undegraded":"2024-04-24T22:59:05.520176+0000","last_fullsized":"2024-04-24T22:59:05.520176+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:04:17.107583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.519872+0000","last_change":"2024-04-24T22:59:04.533695+0000","last_active":"2024-04-24T22:59:05.519872+0000","last_peered":"2024-04-24T22:59:05.519872+0000","last_clean":"2024-04-24T22:59:05.519872+0000","last_became_active":"2024-04-24T22:59:04.533446+0000","last_became_peered":"2024-04-24T22:59:04.533446+0000","last_unstale":"2024-04-24T22:59:05.519872+0000","last_undegraded":"2024-04-24T22:59:05.519872+0000","last_fullsized":"2024-04-24T22:59:05.519872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:38.140275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072156+0000","last_change":"2024-04-24T22:59:04.534240+0000","last_active":"2024-04-24T22:59:06.072156+0000","last_peered":"2024-04-24T22:59:06.072156+0000","last_clean":"2024-04-24T22:59:06.072156+0000","last_became_active":"2024-04-24T22:59:04.533710+0000","last_became_peered":"2024-04-24T22:59:04.533710+0000","last_unstale":"2024-04-24T22:59:06.072156+0000","last_undegraded":"2024-04-24T22:59:06.072156+0000","last_fullsized":"2024-04-24T22:59:06.072156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:58:18.235537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520204+0000","last_change":"2024-04-24T22:59:04.529089+0000","last_active":"2024-04-24T22:59:05.520204+0000","last_peered":"2024-04-24T22:59:05.520204+0000","last_clean":"2024-04-24T22:59:05.520204+0000","last_became_active":"2024-04-24T22:59:04.528080+0000","last_became_peered":"2024-04-24T22:59:04.528080+0000","last_unstale":"2024-04-24T22:59:05.520204+0000","last_undegraded":"2024-04-24T22:59:05.520204+0000","last_fullsized":"2024-04-24T22:59:05.520204+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:26:32.275407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072288+0000","last_change":"2024-04-24T22:59:04.537977+0000","last_active":"2024-04-24T22:59:06.072288+0000","last_peered":"2024-04-24T22:59:06.072288+0000","last_clean":"2024-04-24T22:59:06.072288+0000","last_became_active":"2024-04-24T22:59:04.537758+0000","last_became_peered":"2024-04-24T22:59:04.537758+0000","last_unstale":"2024-04-24T22:59:06.072288+0000","last_undegraded":"2024-04-24T22:59:06.072288+0000","last_fullsized":"2024-04-24T22:59:06.072288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:02:57.321693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072591+0000","last_change":"2024-04-24T22:59:04.527359+0000","last_active":"2024-04-24T22:59:06.072591+0000","last_peered":"2024-04-24T22:59:06.072591+0000","last_clean":"2024-04-24T22:59:06.072591+0000","last_became_active":"2024-04-24T22:59:04.527093+0000","last_became_peered":"2024-04-24T22:59:04.527093+0000","last_unstale":"2024-04-24T22:59:06.072591+0000","last_undegraded":"2024-04-24T22:59:06.072591+0000","last_fullsized":"2024-04-24T22:59:06.072591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:51:52.321592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.802409+0000","last_change":"2024-04-24T22:59:04.531189+0000","last_active":"2024-04-24T22:59:05.802409+0000","last_peered":"2024-04-24T22:59:05.802409+0000","last_clean":"2024-04-24T22:59:05.802409+0000","last_became_active":"2024-04-24T22:59:04.530929+0000","last_became_peered":"2024-04-24T22:59:04.530929+0000","last_unstale":"2024-04-24T22:59:05.802409+0000","last_undegraded":"2024-04-24T22:59:05.802409+0000","last_fullsized":"2024-04-24T22:59:05.802409+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:36.401052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.571534+0000","last_change":"2024-04-24T22:59:04.537735+0000","last_active":"2024-04-24T22:59:05.571534+0000","last_peered":"2024-04-24T22:59:05.571534+0000","last_clean":"2024-04-24T22:59:05.571534+0000","last_became_active":"2024-04-24T22:59:04.537287+0000","last_became_peered":"2024-04-24T22:59:04.537287+0000","last_unstale":"2024-04-24T22:59:05.571534+0000","last_undegraded":"2024-04-24T22:59:05.571534+0000","last_fullsized":"2024-04-24T22:59:05.571534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:25.003190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.519754+0000","last_change":"2024-04-24T22:59:04.530883+0000","last_active":"2024-04-24T22:59:05.519754+0000","last_peered":"2024-04-24T22:59:05.519754+0000","last_clean":"2024-04-24T22:59:05.519754+0000","last_became_active":"2024-04-24T22:59:04.530647+0000","last_became_peered":"2024-04-24T22:59:04.530647+0000","last_unstale":"2024-04-24T22:59:05.519754+0000","last_undegraded":"2024-04-24T22:59:05.519754+0000","last_fullsized":"2024-04-24T22:59:05.519754+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:54:21.680277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.571676+0000","last_change":"2024-04-24T22:59:04.535874+0000","last_active":"2024-04-24T22:59:05.571676+0000","last_peered":"2024-04-24T22:59:05.571676+0000","last_clean":"2024-04-24T22:59:05.571676+0000","last_became_active":"2024-04-24T22:59:04.535623+0000","last_became_peered":"2024-04-24T22:59:04.535623+0000","last_unstale":"2024-04-24T22:59:05.571676+0000","last_undegraded":"2024-04-24T22:59:05.571676+0000","last_fullsized":"2024-04-24T22:59:05.571676+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:34:41.433939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.521847+0000","last_change":"2024-04-24T22:59:04.534070+0000","last_active":"2024-04-24T22:59:05.521847+0000","last_peered":"2024-04-24T22:59:05.521847+0000","last_clean":"2024-04-24T22:59:05.521847+0000","last_became_active":"2024-04-24T22:59:04.533801+0000","last_became_peered":"2024-04-24T22:59:04.533801+0000","last_unstale":"2024-04-24T22:59:05.521847+0000","last_undegraded":"2024-04-24T22:59:05.521847+0000","last_fullsized":"2024-04-24T22:59:05.521847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:51:46.927874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520818+0000","last_change":"2024-04-24T22:59:04.531614+0000","last_active":"2024-04-24T22:59:05.520818+0000","last_peered":"2024-04-24T22:59:05.520818+0000","last_clean":"2024-04-24T22:59:05.520818+0000","last_became_active":"2024-04-24T22:59:04.530752+0000","last_became_peered":"2024-04-24T22:59:04.530752+0000","last_unstale":"2024-04-24T22:59:05.520818+0000","last_undegraded":"2024-04-24T22:59:05.520818+0000","last_fullsized":"2024-04-24T22:59:05.520818+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:15:43.008089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520758+0000","last_change":"2024-04-24T22:59:04.534458+0000","last_active":"2024-04-24T22:59:05.520758+0000","last_peered":"2024-04-24T22:59:05.520758+0000","last_clean":"2024-04-24T22:59:05.520758+0000","last_became_active":"2024-04-24T22:59:04.533333+0000","last_became_peered":"2024-04-24T22:59:04.533333+0000","last_unstale":"2024-04-24T22:59:05.520758+0000","last_undegraded":"2024-04-24T22:59:05.520758+0000","last_fullsized":"2024-04-24T22:59:05.520758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:16:16.466196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.571278+0000","last_change":"2024-04-24T22:59:04.533761+0000","last_active":"2024-04-24T22:59:05.571278+0000","last_peered":"2024-04-24T22:59:05.571278+0000","last_clean":"2024-04-24T22:59:05.571278+0000","last_became_active":"2024-04-24T22:59:04.533321+0000","last_became_peered":"2024-04-24T22:59:04.533321+0000","last_unstale":"2024-04-24T22:59:05.571278+0000","last_undegraded":"2024-04-24T22:59:05.571278+0000","last_fullsized":"2024-04-24T22:59:05.571278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:19:02.943510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.519970+0000","last_change":"2024-04-24T22:59:04.528638+0000","last_active":"2024-04-24T22:59:05.519970+0000","last_peered":"2024-04-24T22:59:05.519970+0000","last_clean":"2024-04-24T22:59:05.519970+0000","last_became_active":"2024-04-24T22:59:04.528413+0000","last_became_peered":"2024-04-24T22:59:04.528413+0000","last_unstale":"2024-04-24T22:59:05.519970+0000","last_undegraded":"2024-04-24T22:59:05.519970+0000","last_fullsized":"2024-04-24T22:59:05.519970+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:29:38.333908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072338+0000","last_change":"2024-04-24T22:59:04.537062+0000","last_active":"2024-04-24T22:59:06.072338+0000","last_peered":"2024-04-24T22:59:06.072338+0000","last_clean":"2024-04-24T22:59:06.072338+0000","last_became_active":"2024-04-24T22:59:04.536623+0000","last_became_peered":"2024-04-24T22:59:04.536623+0000","last_unstale":"2024-04-24T22:59:06.072338+0000","last_undegraded":"2024-04-24T22:59:06.072338+0000","last_fullsized":"2024-04-24T22:59:06.072338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:42:07.950747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.521248+0000","last_change":"2024-04-24T22:59:04.528631+0000","last_active":"2024-04-24T22:59:05.521248+0000","last_peered":"2024-04-24T22:59:05.521248+0000","last_clean":"2024-04-24T22:59:05.521248+0000","last_became_active":"2024-04-24T22:59:04.527962+0000","last_became_peered":"2024-04-24T22:59:04.527962+0000","last_unstale":"2024-04-24T22:59:05.521248+0000","last_undegraded":"2024-04-24T22:59:05.521248+0000","last_fullsized":"2024-04-24T22:59:05.521248+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:55:25.352217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.802375+0000","last_change":"2024-04-24T22:59:04.536016+0000","last_active":"2024-04-24T22:59:05.802375+0000","last_peered":"2024-04-24T22:59:05.802375+0000","last_clean":"2024-04-24T22:59:05.802375+0000","last_became_active":"2024-04-24T22:59:04.535746+0000","last_became_peered":"2024-04-24T22:59:04.535746+0000","last_unstale":"2024-04-24T22:59:05.802375+0000","last_undegraded":"2024-04-24T22:59:05.802375+0000","last_fullsized":"2024-04-24T22:59:05.802375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:02:30.961716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.571464+0000","last_change":"2024-04-24T22:59:03.508028+0000","last_active":"2024-04-24T22:59:05.571464+0000","last_peered":"2024-04-24T22:59:05.571464+0000","last_clean":"2024-04-24T22:59:05.571464+0000","last_became_active":"2024-04-24T22:59:03.507627+0000","last_became_peered":"2024-04-24T22:59:03.507627+0000","last_unstale":"2024-04-24T22:59:05.571464+0000","last_undegraded":"2024-04-24T22:59:05.571464+0000","last_fullsized":"2024-04-24T22:59:05.571464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:40:21.857692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072150+0000","last_change":"2024-04-24T22:59:04.534868+0000","last_active":"2024-04-24T22:59:06.072150+0000","last_peered":"2024-04-24T22:59:06.072150+0000","last_clean":"2024-04-24T22:59:06.072150+0000","last_became_active":"2024-04-24T22:59:04.534232+0000","last_became_peered":"2024-04-24T22:59:04.534232+0000","last_unstale":"2024-04-24T22:59:06.072150+0000","last_undegraded":"2024-04-24T22:59:06.072150+0000","last_fullsized":"2024-04-24T22:59:06.072150+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:29.091605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.519708+0000","last_change":"2024-04-24T22:59:03.511649+0000","last_active":"2024-04-24T22:59:05.519708+0000","last_peered":"2024-04-24T22:59:05.519708+0000","last_clean":"2024-04-24T22:59:05.519708+0000","last_became_active":"2024-04-24T22:59:03.511079+0000","last_became_peered":"2024-04-24T22:59:03.511079+0000","last_unstale":"2024-04-24T22:59:05.519708+0000","last_undegraded":"2024-04-24T22:59:05.519708+0000","last_fullsized":"2024-04-24T22:59:05.519708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:08.640377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520592+0000","last_change":"2024-04-24T22:59:03.512099+0000","last_active":"2024-04-24T22:59:05.520592+0000","last_peered":"2024-04-24T22:59:05.520592+0000","last_clean":"2024-04-24T22:59:05.520592+0000","last_became_active":"2024-04-24T22:59:03.511903+0000","last_became_peered":"2024-04-24T22:59:03.511903+0000","last_unstale":"2024-04-24T22:59:05.520592+0000","last_undegraded":"2024-04-24T22:59:05.520592+0000","last_fullsized":"2024-04-24T22:59:05.520592+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:06.046861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.571807+0000","last_change":"2024-04-24T22:59:04.529177+0000","last_active":"2024-04-24T22:59:05.571807+0000","last_peered":"2024-04-24T22:59:05.571807+0000","last_clean":"2024-04-24T22:59:05.571807+0000","last_became_active":"2024-04-24T22:59:04.528730+0000","last_became_peered":"2024-04-24T22:59:04.528730+0000","last_unstale":"2024-04-24T22:59:05.571807+0000","last_undegraded":"2024-04-24T22:59:05.571807+0000","last_fullsized":"2024-04-24T22:59:05.571807+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:41:20.670964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520440+0000","last_change":"2024-04-24T22:59:03.510748+0000","last_active":"2024-04-24T22:59:05.520440+0000","last_peered":"2024-04-24T22:59:05.520440+0000","last_clean":"2024-04-24T22:59:05.520440+0000","last_became_active":"2024-04-24T22:59:03.510515+0000","last_became_peered":"2024-04-24T22:59:03.510515+0000","last_unstale":"2024-04-24T22:59:05.520440+0000","last_undegraded":"2024-04-24T22:59:05.520440+0000","last_fullsized":"2024-04-24T22:59:05.520440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:03:30.220338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072114+0000","last_change":"2024-04-24T22:59:04.523319+0000","last_active":"2024-04-24T22:59:06.072114+0000","last_peered":"2024-04-24T22:59:06.072114+0000","last_clean":"2024-04-24T22:59:06.072114+0000","last_became_active":"2024-04-24T22:59:04.523086+0000","last_became_peered":"2024-04-24T22:59:04.523086+0000","last_unstale":"2024-04-24T22:59:06.072114+0000","last_undegraded":"2024-04-24T22:59:06.072114+0000","last_fullsized":"2024-04-24T22:59:06.072114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:27.103005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.802709+0000","last_change":"2024-04-24T22:59:03.510379+0000","last_active":"2024-04-24T22:59:05.802709+0000","last_peered":"2024-04-24T22:59:05.802709+0000","last_clean":"2024-04-24T22:59:05.802709+0000","last_became_active":"2024-04-24T22:59:03.510070+0000","last_became_peered":"2024-04-24T22:59:03.510070+0000","last_unstale":"2024-04-24T22:59:05.802709+0000","last_undegraded":"2024-04-24T22:59:05.802709+0000","last_fullsized":"2024-04-24T22:59:05.802709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:21:24.085269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.521658+0000","last_change":"2024-04-24T22:59:04.534163+0000","last_active":"2024-04-24T22:59:05.521658+0000","last_peered":"2024-04-24T22:59:05.521658+0000","last_clean":"2024-04-24T22:59:05.521658+0000","last_became_active":"2024-04-24T22:59:04.533861+0000","last_became_peered":"2024-04-24T22:59:04.533861+0000","last_unstale":"2024-04-24T22:59:05.521658+0000","last_undegraded":"2024-04-24T22:59:05.521658+0000","last_fullsized":"2024-04-24T22:59:05.521658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:02:58.886576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072490+0000","last_change":"2024-04-24T22:59:03.512403+0000","last_active":"2024-04-24T22:59:06.072490+0000","last_peered":"2024-04-24T22:59:06.072490+0000","last_clean":"2024-04-24T22:59:06.072490+0000","last_became_active":"2024-04-24T22:59:03.511538+0000","last_became_peered":"2024-04-24T22:59:03.511538+0000","last_unstale":"2024-04-24T22:59:06.072490+0000","last_undegraded":"2024-04-24T22:59:06.072490+0000","last_fullsized":"2024-04-24T22:59:06.072490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:08:00.814274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.571343+0000","last_change":"2024-04-24T22:59:04.532532+0000","last_active":"2024-04-24T22:59:05.571343+0000","last_peered":"2024-04-24T22:59:05.571343+0000","last_clean":"2024-04-24T22:59:05.571343+0000","last_became_active":"2024-04-24T22:59:04.532308+0000","last_became_peered":"2024-04-24T22:59:04.532308+0000","last_unstale":"2024-04-24T22:59:05.571343+0000","last_undegraded":"2024-04-24T22:59:05.571343+0000","last_fullsized":"2024-04-24T22:59:05.571343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:07:10.769633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072409+0000","last_change":"2024-04-24T22:59:03.509245+0000","last_active":"2024-04-24T22:59:06.072409+0000","last_peered":"2024-04-24T22:59:06.072409+0000","last_clean":"2024-04-24T22:59:06.072409+0000","last_became_active":"2024-04-24T22:59:03.508959+0000","last_became_peered":"2024-04-24T22:59:03.508959+0000","last_unstale":"2024-04-24T22:59:06.072409+0000","last_undegraded":"2024-04-24T22:59:06.072409+0000","last_fullsized":"2024-04-24T22:59:06.072409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:28:24.443582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.519926+0000","last_change":"2024-04-24T22:59:04.537036+0000","last_active":"2024-04-24T22:59:05.519926+0000","last_peered":"2024-04-24T22:59:05.519926+0000","last_clean":"2024-04-24T22:59:05.519926+0000","last_became_active":"2024-04-24T22:59:04.536728+0000","last_became_peered":"2024-04-24T22:59:04.536728+0000","last_unstale":"2024-04-24T22:59:05.519926+0000","last_undegraded":"2024-04-24T22:59:05.519926+0000","last_fullsized":"2024-04-24T22:59:05.519926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:14:39.754057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.521704+0000","last_change":"2024-04-24T22:59:03.508470+0000","last_active":"2024-04-24T22:59:05.521704+0000","last_peered":"2024-04-24T22:59:05.521704+0000","last_clean":"2024-04-24T22:59:05.521704+0000","last_became_active":"2024-04-24T22:59:03.508200+0000","last_became_peered":"2024-04-24T22:59:03.508200+0000","last_unstale":"2024-04-24T22:59:05.521704+0000","last_undegraded":"2024-04-24T22:59:05.521704+0000","last_fullsized":"2024-04-24T22:59:05.521704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:54.077458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072276+0000","last_change":"2024-04-24T22:59:04.532595+0000","last_active":"2024-04-24T22:59:06.072276+0000","last_peered":"2024-04-24T22:59:06.072276+0000","last_clean":"2024-04-24T22:59:06.072276+0000","last_became_active":"2024-04-24T22:59:04.532203+0000","last_became_peered":"2024-04-24T22:59:04.532203+0000","last_unstale":"2024-04-24T22:59:06.072276+0000","last_undegraded":"2024-04-24T22:59:06.072276+0000","last_fullsized":"2024-04-24T22:59:06.072276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:27:13.861775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072047+0000","last_change":"2024-04-24T22:59:03.509839+0000","last_active":"2024-04-24T22:59:06.072047+0000","last_peered":"2024-04-24T22:59:06.072047+0000","last_clean":"2024-04-24T22:59:06.072047+0000","last_became_active":"2024-04-24T22:59:03.509598+0000","last_became_peered":"2024-04-24T22:59:03.509598+0000","last_unstale":"2024-04-24T22:59:06.072047+0000","last_undegraded":"2024-04-24T22:59:06.072047+0000","last_fullsized":"2024-04-24T22:59:06.072047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:26:55.328076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520508+0000","last_change":"2024-04-24T22:59:04.531989+0000","last_active":"2024-04-24T22:59:05.520508+0000","last_peered":"2024-04-24T22:59:05.520508+0000","last_clean":"2024-04-24T22:59:05.520508+0000","last_became_active":"2024-04-24T22:59:04.531739+0000","last_became_peered":"2024-04-24T22:59:04.531739+0000","last_unstale":"2024-04-24T22:59:05.520508+0000","last_undegraded":"2024-04-24T22:59:05.520508+0000","last_fullsized":"2024-04-24T22:59:05.520508+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:38:53.496201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520273+0000","last_change":"2024-04-24T22:59:03.504141+0000","last_active":"2024-04-24T22:59:05.520273+0000","last_peered":"2024-04-24T22:59:05.520273+0000","last_clean":"2024-04-24T22:59:05.520273+0000","last_became_active":"2024-04-24T22:59:03.503898+0000","last_became_peered":"2024-04-24T22:59:03.503898+0000","last_unstale":"2024-04-24T22:59:05.520273+0000","last_undegraded":"2024-04-24T22:59:05.520273+0000","last_fullsized":"2024-04-24T22:59:05.520273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:51:28.639755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520233+0000","last_change":"2024-04-24T22:59:04.535653+0000","last_active":"2024-04-24T22:59:05.520233+0000","last_peered":"2024-04-24T22:59:05.520233+0000","last_clean":"2024-04-24T22:59:05.520233+0000","last_became_active":"2024-04-24T22:59:04.535211+0000","last_became_peered":"2024-04-24T22:59:04.535211+0000","last_unstale":"2024-04-24T22:59:05.520233+0000","last_undegraded":"2024-04-24T22:59:05.520233+0000","last_fullsized":"2024-04-24T22:59:05.520233+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:14.793006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072608+0000","last_change":"2024-04-24T22:59:03.514558+0000","last_active":"2024-04-24T22:59:06.072608+0000","last_peered":"2024-04-24T22:59:06.072608+0000","last_clean":"2024-04-24T22:59:06.072608+0000","last_became_active":"2024-04-24T22:59:03.514168+0000","last_became_peered":"2024-04-24T22:59:03.514168+0000","last_unstale":"2024-04-24T22:59:06.072608+0000","last_undegraded":"2024-04-24T22:59:06.072608+0000","last_fullsized":"2024-04-24T22:59:06.072608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:42:06.949022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.802534+0000","last_change":"2024-04-24T22:59:04.536979+0000","last_active":"2024-04-24T22:59:05.802534+0000","last_peered":"2024-04-24T22:59:05.802534+0000","last_clean":"2024-04-24T22:59:05.802534+0000","last_became_active":"2024-04-24T22:59:04.536749+0000","last_became_peered":"2024-04-24T22:59:04.536749+0000","last_unstale":"2024-04-24T22:59:05.802534+0000","last_undegraded":"2024-04-24T22:59:05.802534+0000","last_fullsized":"2024-04-24T22:59:05.802534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:06:11.147097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072411+0000","last_change":"2024-04-24T22:59:03.507780+0000","last_active":"2024-04-24T22:59:06.072411+0000","last_peered":"2024-04-24T22:59:06.072411+0000","last_clean":"2024-04-24T22:59:06.072411+0000","last_became_active":"2024-04-24T22:59:03.507476+0000","last_became_peered":"2024-04-24T22:59:03.507476+0000","last_unstale":"2024-04-24T22:59:06.072411+0000","last_undegraded":"2024-04-24T22:59:06.072411+0000","last_fullsized":"2024-04-24T22:59:06.072411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:02.130252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520033+0000","last_change":"2024-04-24T22:59:04.532342+0000","last_active":"2024-04-24T22:59:05.520033+0000","last_peered":"2024-04-24T22:59:05.520033+0000","last_clean":"2024-04-24T22:59:05.520033+0000","last_became_active":"2024-04-24T22:59:04.531830+0000","last_became_peered":"2024-04-24T22:59:04.531830+0000","last_unstale":"2024-04-24T22:59:05.520033+0000","last_undegraded":"2024-04-24T22:59:05.520033+0000","last_fullsized":"2024-04-24T22:59:05.520033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:26.483089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072081+0000","last_change":"2024-04-24T22:59:03.511265+0000","last_active":"2024-04-24T22:59:06.072081+0000","last_peered":"2024-04-24T22:59:06.072081+0000","last_clean":"2024-04-24T22:59:06.072081+0000","last_became_active":"2024-04-24T22:59:03.510105+0000","last_became_peered":"2024-04-24T22:59:03.510105+0000","last_unstale":"2024-04-24T22:59:06.072081+0000","last_undegraded":"2024-04-24T22:59:06.072081+0000","last_fullsized":"2024-04-24T22:59:06.072081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:14:10.398082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.521760+0000","last_change":"2024-04-24T22:59:04.535582+0000","last_active":"2024-04-24T22:59:05.521760+0000","last_peered":"2024-04-24T22:59:05.521760+0000","last_clean":"2024-04-24T22:59:05.521760+0000","last_became_active":"2024-04-24T22:59:04.535184+0000","last_became_peered":"2024-04-24T22:59:04.535184+0000","last_unstale":"2024-04-24T22:59:05.521760+0000","last_undegraded":"2024-04-24T22:59:05.521760+0000","last_fullsized":"2024-04-24T22:59:05.521760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:14:13.409713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072466+0000","last_change":"2024-04-24T22:59:03.511495+0000","last_active":"2024-04-24T22:59:06.072466+0000","last_peered":"2024-04-24T22:59:06.072466+0000","last_clean":"2024-04-24T22:59:06.072466+0000","last_became_active":"2024-04-24T22:59:03.511067+0000","last_became_peered":"2024-04-24T22:59:03.511067+0000","last_unstale":"2024-04-24T22:59:06.072466+0000","last_undegraded":"2024-04-24T22:59:06.072466+0000","last_fullsized":"2024-04-24T22:59:06.072466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:25.766950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.071949+0000","last_change":"2024-04-24T22:59:04.538030+0000","last_active":"2024-04-24T22:59:06.071949+0000","last_peered":"2024-04-24T22:59:06.071949+0000","last_clean":"2024-04-24T22:59:06.071949+0000","last_became_active":"2024-04-24T22:59:04.537558+0000","last_became_peered":"2024-04-24T22:59:04.537558+0000","last_unstale":"2024-04-24T22:59:06.071949+0000","last_undegraded":"2024-04-24T22:59:06.071949+0000","last_fullsized":"2024-04-24T22:59:06.071949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:49:31.717151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.571432+0000","last_change":"2024-04-24T22:59:03.510195+0000","last_active":"2024-04-24T22:59:05.571432+0000","last_peered":"2024-04-24T22:59:05.571432+0000","last_clean":"2024-04-24T22:59:05.571432+0000","last_became_active":"2024-04-24T22:59:03.509950+0000","last_became_peered":"2024-04-24T22:59:03.509950+0000","last_unstale":"2024-04-24T22:59:05.571432+0000","last_undegraded":"2024-04-24T22:59:05.571432+0000","last_fullsized":"2024-04-24T22:59:05.571432+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:25:40.587061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072293+0000","last_change":"2024-04-24T22:59:04.535501+0000","last_active":"2024-04-24T22:59:06.072293+0000","last_peered":"2024-04-24T22:59:06.072293+0000","last_clean":"2024-04-24T22:59:06.072293+0000","last_became_active":"2024-04-24T22:59:04.534547+0000","last_became_peered":"2024-04-24T22:59:04.534547+0000","last_unstale":"2024-04-24T22:59:06.072293+0000","last_undegraded":"2024-04-24T22:59:06.072293+0000","last_fullsized":"2024-04-24T22:59:06.072293+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:49:20.073672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520927+0000","last_change":"2024-04-24T22:59:03.512241+0000","last_active":"2024-04-24T22:59:05.520927+0000","last_peered":"2024-04-24T22:59:05.520927+0000","last_clean":"2024-04-24T22:59:05.520927+0000","last_became_active":"2024-04-24T22:59:03.511991+0000","last_became_peered":"2024-04-24T22:59:03.511991+0000","last_unstale":"2024-04-24T22:59:05.520927+0000","last_undegraded":"2024-04-24T22:59:05.520927+0000","last_fullsized":"2024-04-24T22:59:05.520927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:00:24.103638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520418+0000","last_change":"2024-04-24T22:59:04.524264+0000","last_active":"2024-04-24T22:59:05.520418+0000","last_peered":"2024-04-24T22:59:05.520418+0000","last_clean":"2024-04-24T22:59:05.520418+0000","last_became_active":"2024-04-24T22:59:04.523991+0000","last_became_peered":"2024-04-24T22:59:04.523991+0000","last_unstale":"2024-04-24T22:59:05.520418+0000","last_undegraded":"2024-04-24T22:59:05.520418+0000","last_fullsized":"2024-04-24T22:59:05.520418+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:50.932498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.802551+0000","last_change":"2024-04-24T22:58:55.138810+0000","last_active":"2024-04-24T22:59:05.802551+0000","last_peered":"2024-04-24T22:59:05.802551+0000","last_clean":"2024-04-24T22:59:05.802551+0000","last_became_active":"2024-04-24T22:58:55.138627+0000","last_became_peered":"2024-04-24T22:58:55.138627+0000","last_unstale":"2024-04-24T22:59:05.802551+0000","last_undegraded":"2024-04-24T22:59:05.802551+0000","last_fullsized":"2024-04-24T22:59:05.802551+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_clean_scrub_stamp":"2024-04-24T22:58:54.127235+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:24.010002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520503+0000","last_change":"2024-04-24T22:59:03.501901+0000","last_active":"2024-04-24T22:59:05.520503+0000","last_peered":"2024-04-24T22:59:05.520503+0000","last_clean":"2024-04-24T22:59:05.520503+0000","last_became_active":"2024-04-24T22:59:03.501383+0000","last_became_peered":"2024-04-24T22:59:03.501383+0000","last_unstale":"2024-04-24T22:59:05.520503+0000","last_undegraded":"2024-04-24T22:59:05.520503+0000","last_fullsized":"2024-04-24T22:59:05.520503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:35:55.503771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072518+0000","last_change":"2024-04-24T22:59:04.531995+0000","last_active":"2024-04-24T22:59:06.072518+0000","last_peered":"2024-04-24T22:59:06.072518+0000","last_clean":"2024-04-24T22:59:06.072518+0000","last_became_active":"2024-04-24T22:59:04.531730+0000","last_became_peered":"2024-04-24T22:59:04.531730+0000","last_unstale":"2024-04-24T22:59:06.072518+0000","last_undegraded":"2024-04-24T22:59:06.072518+0000","last_fullsized":"2024-04-24T22:59:06.072518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:51:53.153038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520341+0000","last_change":"2024-04-24T22:59:03.509668+0000","last_active":"2024-04-24T22:59:05.520341+0000","last_peered":"2024-04-24T22:59:05.520341+0000","last_clean":"2024-04-24T22:59:05.520341+0000","last_became_active":"2024-04-24T22:59:03.509450+0000","last_became_peered":"2024-04-24T22:59:03.509450+0000","last_unstale":"2024-04-24T22:59:05.520341+0000","last_undegraded":"2024-04-24T22:59:05.520341+0000","last_fullsized":"2024-04-24T22:59:05.520341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:06:37.228507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520854+0000","last_change":"2024-04-24T22:59:04.530536+0000","last_active":"2024-04-24T22:59:05.520854+0000","last_peered":"2024-04-24T22:59:05.520854+0000","last_clean":"2024-04-24T22:59:05.520854+0000","last_became_active":"2024-04-24T22:59:04.530302+0000","last_became_peered":"2024-04-24T22:59:04.530302+0000","last_unstale":"2024-04-24T22:59:05.520854+0000","last_undegraded":"2024-04-24T22:59:05.520854+0000","last_fullsized":"2024-04-24T22:59:05.520854+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:38:05.122875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072508+0000","last_change":"2024-04-24T22:59:03.508258+0000","last_active":"2024-04-24T22:59:06.072508+0000","last_peered":"2024-04-24T22:59:06.072508+0000","last_clean":"2024-04-24T22:59:06.072508+0000","last_became_active":"2024-04-24T22:59:03.508013+0000","last_became_peered":"2024-04-24T22:59:03.508013+0000","last_unstale":"2024-04-24T22:59:06.072508+0000","last_undegraded":"2024-04-24T22:59:06.072508+0000","last_fullsized":"2024-04-24T22:59:06.072508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:04:08.375090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.571574+0000","last_change":"2024-04-24T22:59:04.533925+0000","last_active":"2024-04-24T22:59:05.571574+0000","last_peered":"2024-04-24T22:59:05.571574+0000","last_clean":"2024-04-24T22:59:05.571574+0000","last_became_active":"2024-04-24T22:59:04.533571+0000","last_became_peered":"2024-04-24T22:59:04.533571+0000","last_unstale":"2024-04-24T22:59:05.571574+0000","last_undegraded":"2024-04-24T22:59:05.571574+0000","last_fullsized":"2024-04-24T22:59:05.571574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:23:08.983760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520235+0000","last_change":"2024-04-24T22:59:03.503558+0000","last_active":"2024-04-24T22:59:05.520235+0000","last_peered":"2024-04-24T22:59:05.520235+0000","last_clean":"2024-04-24T22:59:05.520235+0000","last_became_active":"2024-04-24T22:59:03.503199+0000","last_became_peered":"2024-04-24T22:59:03.503199+0000","last_unstale":"2024-04-24T22:59:05.520235+0000","last_undegraded":"2024-04-24T22:59:05.520235+0000","last_fullsized":"2024-04-24T22:59:05.520235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:23:49.375509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072420+0000","last_change":"2024-04-24T22:59:04.531235+0000","last_active":"2024-04-24T22:59:06.072420+0000","last_peered":"2024-04-24T22:59:06.072420+0000","last_clean":"2024-04-24T22:59:06.072420+0000","last_became_active":"2024-04-24T22:59:04.530974+0000","last_became_peered":"2024-04-24T22:59:04.530974+0000","last_unstale":"2024-04-24T22:59:06.072420+0000","last_undegraded":"2024-04-24T22:59:06.072420+0000","last_fullsized":"2024-04-24T22:59:06.072420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:45:58.398513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.571979+0000","last_change":"2024-04-24T22:59:03.509319+0000","last_active":"2024-04-24T22:59:05.571979+0000","last_peered":"2024-04-24T22:59:05.571979+0000","last_clean":"2024-04-24T22:59:05.571979+0000","last_became_active":"2024-04-24T22:59:03.508493+0000","last_became_peered":"2024-04-24T22:59:03.508493+0000","last_unstale":"2024-04-24T22:59:05.571979+0000","last_undegraded":"2024-04-24T22:59:05.571979+0000","last_fullsized":"2024-04-24T22:59:05.571979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:49:50.756565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.521375+0000","last_change":"2024-04-24T22:59:04.527591+0000","last_active":"2024-04-24T22:59:05.521375+0000","last_peered":"2024-04-24T22:59:05.521375+0000","last_clean":"2024-04-24T22:59:05.521375+0000","last_became_active":"2024-04-24T22:59:04.527329+0000","last_became_peered":"2024-04-24T22:59:04.527329+0000","last_unstale":"2024-04-24T22:59:05.521375+0000","last_undegraded":"2024-04-24T22:59:05.521375+0000","last_fullsized":"2024-04-24T22:59:05.521375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:19:52.022880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520291+0000","last_change":"2024-04-24T22:59:03.512317+0000","last_active":"2024-04-24T22:59:05.520291+0000","last_peered":"2024-04-24T22:59:05.520291+0000","last_clean":"2024-04-24T22:59:05.520291+0000","last_became_active":"2024-04-24T22:59:03.512056+0000","last_became_peered":"2024-04-24T22:59:03.512056+0000","last_unstale":"2024-04-24T22:59:05.520291+0000","last_undegraded":"2024-04-24T22:59:05.520291+0000","last_fullsized":"2024-04-24T22:59:05.520291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:48:30.411017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520699+0000","last_change":"2024-04-24T22:59:04.530095+0000","last_active":"2024-04-24T22:59:05.520699+0000","last_peered":"2024-04-24T22:59:05.520699+0000","last_clean":"2024-04-24T22:59:05.520699+0000","last_became_active":"2024-04-24T22:59:04.529730+0000","last_became_peered":"2024-04-24T22:59:04.529730+0000","last_unstale":"2024-04-24T22:59:05.520699+0000","last_undegraded":"2024-04-24T22:59:05.520699+0000","last_fullsized":"2024-04-24T22:59:05.520699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:34.933698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.525549+0000","last_change":"2024-04-24T22:59:03.510515+0000","last_active":"2024-04-24T22:59:06.525549+0000","last_peered":"2024-04-24T22:59:06.525549+0000","last_clean":"2024-04-24T22:59:06.525549+0000","last_became_active":"2024-04-24T22:59:03.510016+0000","last_became_peered":"2024-04-24T22:59:03.510016+0000","last_unstale":"2024-04-24T22:59:06.525549+0000","last_undegraded":"2024-04-24T22:59:06.525549+0000","last_fullsized":"2024-04-24T22:59:06.525549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:51:23.939098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072377+0000","last_change":"2024-04-24T22:59:04.528509+0000","last_active":"2024-04-24T22:59:06.072377+0000","last_peered":"2024-04-24T22:59:06.072377+0000","last_clean":"2024-04-24T22:59:06.072377+0000","last_became_active":"2024-04-24T22:59:04.527604+0000","last_became_peered":"2024-04-24T22:59:04.527604+0000","last_unstale":"2024-04-24T22:59:06.072377+0000","last_undegraded":"2024-04-24T22:59:06.072377+0000","last_fullsized":"2024-04-24T22:59:06.072377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:13:55.192974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520140+0000","last_change":"2024-04-24T22:59:03.513495+0000","last_active":"2024-04-24T22:59:05.520140+0000","last_peered":"2024-04-24T22:59:05.520140+0000","last_clean":"2024-04-24T22:59:05.520140+0000","last_became_active":"2024-04-24T22:59:03.512772+0000","last_became_peered":"2024-04-24T22:59:03.512772+0000","last_unstale":"2024-04-24T22:59:05.520140+0000","last_undegraded":"2024-04-24T22:59:05.520140+0000","last_fullsized":"2024-04-24T22:59:05.520140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:49:12.718741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072288+0000","last_change":"2024-04-24T22:59:04.528601+0000","last_active":"2024-04-24T22:59:06.072288+0000","last_peered":"2024-04-24T22:59:06.072288+0000","last_clean":"2024-04-24T22:59:06.072288+0000","last_became_active":"2024-04-24T22:59:04.527812+0000","last_became_peered":"2024-04-24T22:59:04.527812+0000","last_unstale":"2024-04-24T22:59:06.072288+0000","last_undegraded":"2024-04-24T22:59:06.072288+0000","last_fullsized":"2024-04-24T22:59:06.072288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:19.252521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072624+0000","last_change":"2024-04-24T22:59:03.510200+0000","last_active":"2024-04-24T22:59:06.072624+0000","last_peered":"2024-04-24T22:59:06.072624+0000","last_clean":"2024-04-24T22:59:06.072624+0000","last_became_active":"2024-04-24T22:59:03.509783+0000","last_became_peered":"2024-04-24T22:59:03.509783+0000","last_unstale":"2024-04-24T22:59:06.072624+0000","last_undegraded":"2024-04-24T22:59:06.072624+0000","last_fullsized":"2024-04-24T22:59:06.072624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:47.755911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520361+0000","last_change":"2024-04-24T22:59:04.525874+0000","last_active":"2024-04-24T22:59:05.520361+0000","last_peered":"2024-04-24T22:59:05.520361+0000","last_clean":"2024-04-24T22:59:05.520361+0000","last_became_active":"2024-04-24T22:59:04.525094+0000","last_became_peered":"2024-04-24T22:59:04.525094+0000","last_unstale":"2024-04-24T22:59:05.520361+0000","last_undegraded":"2024-04-24T22:59:05.520361+0000","last_fullsized":"2024-04-24T22:59:05.520361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:06:23.254812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072568+0000","last_change":"2024-04-24T22:59:03.511287+0000","last_active":"2024-04-24T22:59:06.072568+0000","last_peered":"2024-04-24T22:59:06.072568+0000","last_clean":"2024-04-24T22:59:06.072568+0000","last_became_active":"2024-04-24T22:59:03.510784+0000","last_became_peered":"2024-04-24T22:59:03.510784+0000","last_unstale":"2024-04-24T22:59:06.072568+0000","last_undegraded":"2024-04-24T22:59:06.072568+0000","last_fullsized":"2024-04-24T22:59:06.072568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:19:48.671658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072736+0000","last_change":"2024-04-24T22:59:04.529663+0000","last_active":"2024-04-24T22:59:06.072736+0000","last_peered":"2024-04-24T22:59:06.072736+0000","last_clean":"2024-04-24T22:59:06.072736+0000","last_became_active":"2024-04-24T22:59:04.527858+0000","last_became_peered":"2024-04-24T22:59:04.527858+0000","last_unstale":"2024-04-24T22:59:06.072736+0000","last_undegraded":"2024-04-24T22:59:06.072736+0000","last_fullsized":"2024-04-24T22:59:06.072736+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:32:03.630786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520703+0000","last_change":"2024-04-24T22:59:03.505841+0000","last_active":"2024-04-24T22:59:05.520703+0000","last_peered":"2024-04-24T22:59:05.520703+0000","last_clean":"2024-04-24T22:59:05.520703+0000","last_became_active":"2024-04-24T22:59:03.505048+0000","last_became_peered":"2024-04-24T22:59:03.505048+0000","last_unstale":"2024-04-24T22:59:05.520703+0000","last_undegraded":"2024-04-24T22:59:05.520703+0000","last_fullsized":"2024-04-24T22:59:05.520703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:34:16.021505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072303+0000","last_change":"2024-04-24T22:59:04.532670+0000","last_active":"2024-04-24T22:59:06.072303+0000","last_peered":"2024-04-24T22:59:06.072303+0000","last_clean":"2024-04-24T22:59:06.072303+0000","last_became_active":"2024-04-24T22:59:04.532437+0000","last_became_peered":"2024-04-24T22:59:04.532437+0000","last_unstale":"2024-04-24T22:59:06.072303+0000","last_undegraded":"2024-04-24T22:59:06.072303+0000","last_fullsized":"2024-04-24T22:59:06.072303+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:08:01.815998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.519807+0000","last_change":"2024-04-24T22:59:03.509614+0000","last_active":"2024-04-24T22:59:05.519807+0000","last_peered":"2024-04-24T22:59:05.519807+0000","last_clean":"2024-04-24T22:59:05.519807+0000","last_became_active":"2024-04-24T22:59:03.509379+0000","last_became_peered":"2024-04-24T22:59:03.509379+0000","last_unstale":"2024-04-24T22:59:05.519807+0000","last_undegraded":"2024-04-24T22:59:05.519807+0000","last_fullsized":"2024-04-24T22:59:05.519807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:46:21.150240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520066+0000","last_change":"2024-04-24T22:59:04.526425+0000","last_active":"2024-04-24T22:59:05.520066+0000","last_peered":"2024-04-24T22:59:05.520066+0000","last_clean":"2024-04-24T22:59:05.520066+0000","last_became_active":"2024-04-24T22:59:04.524467+0000","last_became_peered":"2024-04-24T22:59:04.524467+0000","last_unstale":"2024-04-24T22:59:05.520066+0000","last_undegraded":"2024-04-24T22:59:05.520066+0000","last_fullsized":"2024-04-24T22:59:05.520066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:23:05.155059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520105+0000","last_change":"2024-04-24T22:59:04.535456+0000","last_active":"2024-04-24T22:59:05.520105+0000","last_peered":"2024-04-24T22:59:05.520105+0000","last_clean":"2024-04-24T22:59:05.520105+0000","last_became_active":"2024-04-24T22:59:04.534881+0000","last_became_peered":"2024-04-24T22:59:04.534881+0000","last_unstale":"2024-04-24T22:59:05.520105+0000","last_undegraded":"2024-04-24T22:59:05.520105+0000","last_fullsized":"2024-04-24T22:59:05.520105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:01:15.740234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072535+0000","last_change":"2024-04-24T22:59:03.512029+0000","last_active":"2024-04-24T22:59:06.072535+0000","last_peered":"2024-04-24T22:59:06.072535+0000","last_clean":"2024-04-24T22:59:06.072535+0000","last_became_active":"2024-04-24T22:59:03.511808+0000","last_became_peered":"2024-04-24T22:59:03.511808+0000","last_unstale":"2024-04-24T22:59:06.072535+0000","last_undegraded":"2024-04-24T22:59:06.072535+0000","last_fullsized":"2024-04-24T22:59:06.072535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:02:26.617300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.802478+0000","last_change":"2024-04-24T22:59:04.533430+0000","last_active":"2024-04-24T22:59:05.802478+0000","last_peered":"2024-04-24T22:59:05.802478+0000","last_clean":"2024-04-24T22:59:05.802478+0000","last_became_active":"2024-04-24T22:59:04.533192+0000","last_became_peered":"2024-04-24T22:59:04.533192+0000","last_unstale":"2024-04-24T22:59:05.802478+0000","last_undegraded":"2024-04-24T22:59:05.802478+0000","last_fullsized":"2024-04-24T22:59:05.802478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:50:28.246419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072505+0000","last_change":"2024-04-24T22:59:03.512319+0000","last_active":"2024-04-24T22:59:06.072505+0000","last_peered":"2024-04-24T22:59:06.072505+0000","last_clean":"2024-04-24T22:59:06.072505+0000","last_became_active":"2024-04-24T22:59:03.511788+0000","last_became_peered":"2024-04-24T22:59:03.511788+0000","last_unstale":"2024-04-24T22:59:06.072505+0000","last_undegraded":"2024-04-24T22:59:06.072505+0000","last_fullsized":"2024-04-24T22:59:06.072505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:58:17.233812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.571660+0000","last_change":"2024-04-24T22:59:04.536036+0000","last_active":"2024-04-24T22:59:05.571660+0000","last_peered":"2024-04-24T22:59:05.571660+0000","last_clean":"2024-04-24T22:59:05.571660+0000","last_became_active":"2024-04-24T22:59:04.535591+0000","last_became_peered":"2024-04-24T22:59:04.535591+0000","last_unstale":"2024-04-24T22:59:05.571660+0000","last_undegraded":"2024-04-24T22:59:05.571660+0000","last_fullsized":"2024-04-24T22:59:05.571660+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:59.369829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.571777+0000","last_change":"2024-04-24T22:59:03.508163+0000","last_active":"2024-04-24T22:59:05.571777+0000","last_peered":"2024-04-24T22:59:05.571777+0000","last_clean":"2024-04-24T22:59:05.571777+0000","last_became_active":"2024-04-24T22:59:03.507645+0000","last_became_peered":"2024-04-24T22:59:03.507645+0000","last_unstale":"2024-04-24T22:59:05.571777+0000","last_undegraded":"2024-04-24T22:59:05.571777+0000","last_fullsized":"2024-04-24T22:59:05.571777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:28:40.818581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.571435+0000","last_change":"2024-04-24T22:59:04.538784+0000","last_active":"2024-04-24T22:59:05.571435+0000","last_peered":"2024-04-24T22:59:05.571435+0000","last_clean":"2024-04-24T22:59:05.571435+0000","last_became_active":"2024-04-24T22:59:04.537912+0000","last_became_peered":"2024-04-24T22:59:04.537912+0000","last_unstale":"2024-04-24T22:59:05.571435+0000","last_undegraded":"2024-04-24T22:59:05.571435+0000","last_fullsized":"2024-04-24T22:59:05.571435+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:44.470849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072107+0000","last_change":"2024-04-24T22:59:03.513331+0000","last_active":"2024-04-24T22:59:06.072107+0000","last_peered":"2024-04-24T22:59:06.072107+0000","last_clean":"2024-04-24T22:59:06.072107+0000","last_became_active":"2024-04-24T22:59:03.513198+0000","last_became_peered":"2024-04-24T22:59:03.513198+0000","last_unstale":"2024-04-24T22:59:06.072107+0000","last_undegraded":"2024-04-24T22:59:06.072107+0000","last_fullsized":"2024-04-24T22:59:06.072107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:34:31.185473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":16,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195238,"statfs":{"total":200000000,"available":199924431,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195238,"statfs":{"total":200000000,"available":199924555,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":650,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194662,"statfs":{"total":200000000,"available":199334095,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":75,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195236,"statfs":{"total":200000000,"available":199922271,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":74,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195238,"statfs":{"total":200000000,"available":199923973,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":75,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195237,"statfs":{"total":200000000,"available":199922943,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":650,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194662,"statfs":{"total":200000000,"available":199334117,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":35,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195238,"statfs":{"total":200000000,"available":199924499,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-24T22:59:16.360 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T22:59:16.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.497+0000 7fecaa990640 1 Processor -- start 2024-04-24T22:59:16.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.497+0000 7fecaa990640 1 -- start start 2024-04-24T22:59:16.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.497+0000 7fecaa990640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feca406dc60 0x7feca4169570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:16.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.497+0000 7fecaa990640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feca40b6d10 con 0x7feca406ba30 2024-04-24T22:59:16.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.497+0000 7fecaa990640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feca415ef10 con 0x7feca4158dd0 2024-04-24T22:59:16.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.497+0000 7fecaa990640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7feca415f1f0 con 0x7feca406dc60 2024-04-24T22:59:16.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.498+0000 7feca37fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7feca406ba30 0x7feca406d550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47484/0 (socket says 172.21.15.32:47484) 2024-04-24T22:59:16.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.498+0000 7feca37fe640 1 -- 172.21.15.32:0/3418185105 learned_addr learned my addr 172.21.15.32:0/3418185105 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:16.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.498+0000 7feca2ffd640 1 -- 172.21.15.32:0/3418185105 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3815458379 0 0) 0x7feca40b6d10 con 0x7feca406ba30 2024-04-24T22:59:16.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.498+0000 7feca2ffd640 1 -- 172.21.15.32:0/3418185105 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec8c003520 con 0x7feca406ba30 2024-04-24T22:59:16.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.498+0000 7feca8f2d640 1 --2- 172.21.15.32:0/3418185105 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feca406dc60 0x7feca4169570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:16.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.498+0000 7feca2ffd640 1 -- 172.21.15.32:0/3418185105 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2306919581 0 0) 0x7fec8c003520 con 0x7feca406ba30 2024-04-24T22:59:16.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.498+0000 7feca2ffd640 1 -- 172.21.15.32:0/3418185105 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feca406dc60 msgr2=0x7feca4169570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:16.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.498+0000 7feca2ffd640 1 --2- 172.21.15.32:0/3418185105 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feca406dc60 0x7feca4169570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:16.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.498+0000 7feca2ffd640 1 -- 172.21.15.32:0/3418185105 >> v1:172.21.15.121:6789/0 conn(0x7feca4158dd0 legacy=0x7feca406b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:16.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.498+0000 7feca2ffd640 1 -- 172.21.15.32:0/3418185105 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feca415f4d0 con 0x7feca406ba30 2024-04-24T22:59:16.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.498+0000 7feca2ffd640 1 -- 172.21.15.32:0/3418185105 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7fec98001690 con 0x7feca406ba30 2024-04-24T22:59:16.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.498+0000 7feca2ffd640 1 -- 172.21.15.32:0/3418185105 >> v1:172.21.15.32:6789/0 conn(0x7feca406ba30 legacy=0x7feca406d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:16.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.498+0000 7feca2ffd640 1 --2- 172.21.15.32:0/3418185105 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec8c0039e0 0x7fec8c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:16.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.498+0000 7feca2ffd640 1 --2- 172.21.15.32:0/3418185105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec8c004800 0x7fec8c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:16.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.498+0000 7feca2ffd640 1 --2- 172.21.15.32:0/3418185105 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feca406dc60 0x7fec8c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:16.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.498+0000 7feca2ffd640 1 -- 172.21.15.32:0/3418185105 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7feca415f1f0 con 0x7fec8c004800 2024-04-24T22:59:16.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.498+0000 7feca2ffd640 1 -- 172.21.15.32:0/3418185105 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fec8c00baa0 con 0x7fec8c0039e0 2024-04-24T22:59:16.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.498+0000 7feca2ffd640 1 -- 172.21.15.32:0/3418185105 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fec8c00bca0 con 0x7feca406dc60 2024-04-24T22:59:16.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.498+0000 7feca37fe640 1 --2- 172.21.15.32:0/3418185105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec8c004800 0x7fec8c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:16.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.498+0000 7feca3fff640 1 --2- 172.21.15.32:0/3418185105 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec8c0039e0 0x7fec8c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:16.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.498+0000 7feca8f2d640 1 --2- 172.21.15.32:0/3418185105 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feca406dc60 0x7fec8c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:16.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.499+0000 7feca37fe640 1 -- 172.21.15.32:0/3418185105 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feca406dc60 msgr2=0x7fec8c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:16.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.499+0000 7feca37fe640 1 --2- 172.21.15.32:0/3418185105 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feca406dc60 0x7fec8c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:16.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.499+0000 7feca37fe640 1 -- 172.21.15.32:0/3418185105 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec8c0039e0 msgr2=0x7fec8c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:16.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.499+0000 7feca37fe640 1 --2- 172.21.15.32:0/3418185105 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec8c0039e0 0x7fec8c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:16.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.499+0000 7feca37fe640 1 -- 172.21.15.32:0/3418185105 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec8c003520 con 0x7fec8c004800 2024-04-24T22:59:16.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.499+0000 7feca37fe640 1 --2- 172.21.15.32:0/3418185105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec8c004800 0x7fec8c004c80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7feca406bf10 tx=0x7fec8c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5c5f15c75bd7598 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:16.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.499+0000 7feca2ffd640 1 -- 172.21.15.32:0/3418185105 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec98007d50 con 0x7fec8c004800 2024-04-24T22:59:16.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.499+0000 7feca2ffd640 1 -- 172.21.15.32:0/3418185105 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec98005af0 con 0x7fec8c004800 2024-04-24T22:59:16.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.499+0000 7feca2ffd640 1 -- 172.21.15.32:0/3418185105 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec98007070 con 0x7fec8c004800 2024-04-24T22:59:16.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.500+0000 7fecaa990640 1 -- 172.21.15.32:0/3418185105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec8c004800 msgr2=0x7fec8c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:16.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.500+0000 7fecaa990640 1 --2- 172.21.15.32:0/3418185105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec8c004800 0x7fec8c004c80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7feca406bf10 tx=0x7fec8c005830 comp rx=0 tx=0).stop 2024-04-24T22:59:16.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.500+0000 7feca8f2d640 1 -- 172.21.15.32:0/3418185105 reap_dead start 2024-04-24T22:59:16.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.500+0000 7fecaa990640 1 -- 172.21.15.32:0/3418185105 shutdown_connections 2024-04-24T22:59:16.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.500+0000 7fecaa990640 1 -- 172.21.15.32:0/3418185105 >> 172.21.15.32:0/3418185105 conn(0x7feca406e800 msgr2=0x7feca406ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:16.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.500+0000 7fecaa990640 1 -- 172.21.15.32:0/3418185105 shutdown_connections 2024-04-24T22:59:16.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.500+0000 7fecaa990640 1 -- 172.21.15.32:0/3418185105 wait complete. 2024-04-24T22:59:16.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.501+0000 7fecaa990640 1 Processor -- start 2024-04-24T22:59:16.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.501+0000 7fecaa990640 1 -- start start 2024-04-24T22:59:16.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.502+0000 7fecaa990640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec8c004800 0x7feca414ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:16.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.502+0000 7fecaa990640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feca41517f0 0x7feca41520a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:16.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.502+0000 7fecaa990640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feca4072480 0x7feca4072910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:16.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.502+0000 7fecaa990640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7feca406bfe0 con 0x7feca41517f0 2024-04-24T22:59:16.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.502+0000 7fecaa990640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7feca415fcd0 con 0x7fec8c004800 2024-04-24T22:59:16.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.502+0000 7fecaa990640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7feca41591b0 con 0x7feca4072480 2024-04-24T22:59:16.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.502+0000 7feca37fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feca41517f0 0x7feca41520a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:16.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.502+0000 7feca37fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feca41517f0 0x7feca41520a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38804/0 (socket says 172.21.15.32:38804) 2024-04-24T22:59:16.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.502+0000 7feca37fe640 1 -- 172.21.15.32:0/2832471910 learned_addr learned my addr 172.21.15.32:0/2832471910 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:16.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.502+0000 7feca8f2d640 1 --2- 172.21.15.32:0/2832471910 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feca4072480 0x7feca4072910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:16.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.502+0000 7feca3fff640 1 --2- 172.21.15.32:0/2832471910 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec8c004800 0x7feca414ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:16.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.502+0000 7feca37fe640 1 -- 172.21.15.32:0/2832471910 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feca4072480 msgr2=0x7feca4072910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:16.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.502+0000 7feca37fe640 1 --2- 172.21.15.32:0/2832471910 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feca4072480 0x7feca4072910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:16.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.502+0000 7feca37fe640 1 -- 172.21.15.32:0/2832471910 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec8c004800 msgr2=0x7feca414ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:16.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.502+0000 7feca37fe640 1 --2- 172.21.15.32:0/2832471910 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec8c004800 0x7feca414ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:16.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.502+0000 7feca37fe640 1 -- 172.21.15.32:0/2832471910 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feca41529f0 con 0x7feca41517f0 2024-04-24T22:59:16.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.503+0000 7feca8f2d640 1 --2- 172.21.15.32:0/2832471910 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feca4072480 0x7feca4072910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:16.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.503+0000 7feca37fe640 1 --2- 172.21.15.32:0/2832471910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feca41517f0 0x7feca41520a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fec9802add0 tx=0x7fec98001a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:16.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.503+0000 7feca0ff9640 1 -- 172.21.15.32:0/2832471910 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec98001650 con 0x7feca41517f0 2024-04-24T22:59:16.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.503+0000 7feca0ff9640 1 -- 172.21.15.32:0/2832471910 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec98042070 con 0x7feca41517f0 2024-04-24T22:59:16.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.503+0000 7fecaa990640 1 -- 172.21.15.32:0/2832471910 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feca4152cd0 con 0x7feca41517f0 2024-04-24T22:59:16.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.503+0000 7feca0ff9640 1 -- 172.21.15.32:0/2832471910 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec980072b0 con 0x7feca41517f0 2024-04-24T22:59:16.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.503+0000 7fecaa990640 1 -- 172.21.15.32:0/2832471910 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feca4153290 con 0x7feca41517f0 2024-04-24T22:59:16.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.504+0000 7fecaa990640 1 -- 172.21.15.32:0/2832471910 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feca41f49b0 con 0x7feca41517f0 2024-04-24T22:59:16.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.504+0000 7feca0ff9640 1 -- 172.21.15.32:0/2832471910 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fec98007450 con 0x7feca41517f0 2024-04-24T22:59:16.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.505+0000 7feca0ff9640 1 --2- 172.21.15.32:0/2832471910 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fec9006be30 0x7fec9006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:16.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.505+0000 7feca0ff9640 1 -- 172.21.15.32:0/2832471910 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fec980aaf90 con 0x7feca41517f0 2024-04-24T22:59:16.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.505+0000 7feca3fff640 1 --2- 172.21.15.32:0/2832471910 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fec9006be30 0x7fec9006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:16.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.505+0000 7feca3fff640 1 --2- 172.21.15.32:0/2832471910 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fec9006be30 0x7fec9006e2f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fec94005590 tx=0x7fec94009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:16.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.509+0000 7feca0ff9640 1 -- 172.21.15.32:0/2832471910 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fec98045370 con 0x7feca41517f0 2024-04-24T22:59:16.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.689+0000 7fecaa990640 1 -- 172.21.15.32:0/2832471910 --> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7feca4151e80 con 0x7fec9006be30 2024-04-24T22:59:16.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.696+0000 7feca0ff9640 1 -- 172.21.15.32:0/2832471910 <== mgr.4108 v2:172.21.15.121:6832/1848157943 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247486 (secure 0 0 0) 0x7feca4151e80 con 0x7fec9006be30 2024-04-24T22:59:16.699 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:59:16.701 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-24T22:59:16.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.698+0000 7fecaa990640 1 -- 172.21.15.32:0/2832471910 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fec9006be30 msgr2=0x7fec9006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:16.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.698+0000 7fecaa990640 1 --2- 172.21.15.32:0/2832471910 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fec9006be30 0x7fec9006e2f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fec94005590 tx=0x7fec94009040 comp rx=0 tx=0).stop 2024-04-24T22:59:16.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.698+0000 7fecaa990640 1 -- 172.21.15.32:0/2832471910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feca41517f0 msgr2=0x7feca41520a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:16.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.698+0000 7fecaa990640 1 --2- 172.21.15.32:0/2832471910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feca41517f0 0x7feca41520a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fec9802add0 tx=0x7fec98001a60 comp rx=0 tx=0).stop 2024-04-24T22:59:16.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.698+0000 7fecaa990640 1 -- 172.21.15.32:0/2832471910 shutdown_connections 2024-04-24T22:59:16.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.698+0000 7fecaa990640 1 --2- 172.21.15.32:0/2832471910 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fec9006be30 0x7fec9006e2f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:16.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.698+0000 7fecaa990640 1 --2- 172.21.15.32:0/2832471910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feca41517f0 0x7feca41520a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:16.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.698+0000 7fecaa990640 1 --2- 172.21.15.32:0/2832471910 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feca4072480 0x7feca4072910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:16.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.698+0000 7fecaa990640 1 --2- 172.21.15.32:0/2832471910 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec8c004800 0x7feca414ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:16.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.698+0000 7fecaa990640 1 -- 172.21.15.32:0/2832471910 >> 172.21.15.32:0/2832471910 conn(0x7feca406e800 msgr2=0x7feca4066fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:16.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.698+0000 7fecaa990640 1 -- 172.21.15.32:0/2832471910 shutdown_connections 2024-04-24T22:59:16.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.698+0000 7fecaa990640 1 -- 172.21.15.32:0/2832471910 wait complete. 2024-04-24T22:59:16.711 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-24T22:59:15.130992+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1747,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560752,"statfs":{"total":1600000000,"available":1598210884,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002549"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072128+0000","last_change":"2024-04-24T22:59:04.533635+0000","last_active":"2024-04-24T22:59:06.072128+0000","last_peered":"2024-04-24T22:59:06.072128+0000","last_clean":"2024-04-24T22:59:06.072128+0000","last_became_active":"2024-04-24T22:59:04.533181+0000","last_became_peered":"2024-04-24T22:59:04.533181+0000","last_unstale":"2024-04-24T22:59:06.072128+0000","last_undegraded":"2024-04-24T22:59:06.072128+0000","last_fullsized":"2024-04-24T22:59:06.072128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:19:41.674286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.571341+0000","last_change":"2024-04-24T22:59:04.532345+0000","last_active":"2024-04-24T22:59:05.571341+0000","last_peered":"2024-04-24T22:59:05.571341+0000","last_clean":"2024-04-24T22:59:05.571341+0000","last_became_active":"2024-04-24T22:59:04.532126+0000","last_became_peered":"2024-04-24T22:59:04.532126+0000","last_unstale":"2024-04-24T22:59:05.571341+0000","last_undegraded":"2024-04-24T22:59:05.571341+0000","last_fullsized":"2024-04-24T22:59:05.571341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:40:58.083354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.571343+0000","last_change":"2024-04-24T22:59:04.535851+0000","last_active":"2024-04-24T22:59:05.571343+0000","last_peered":"2024-04-24T22:59:05.571343+0000","last_clean":"2024-04-24T22:59:05.571343+0000","last_became_active":"2024-04-24T22:59:04.535290+0000","last_became_peered":"2024-04-24T22:59:04.535290+0000","last_unstale":"2024-04-24T22:59:05.571343+0000","last_undegraded":"2024-04-24T22:59:05.571343+0000","last_fullsized":"2024-04-24T22:59:05.571343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:57:53.010612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.521146+0000","last_change":"2024-04-24T22:59:04.529616+0000","last_active":"2024-04-24T22:59:05.521146+0000","last_peered":"2024-04-24T22:59:05.521146+0000","last_clean":"2024-04-24T22:59:05.521146+0000","last_became_active":"2024-04-24T22:59:04.529312+0000","last_became_peered":"2024-04-24T22:59:04.529312+0000","last_unstale":"2024-04-24T22:59:05.521146+0000","last_undegraded":"2024-04-24T22:59:05.521146+0000","last_fullsized":"2024-04-24T22:59:05.521146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:55:40.732173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072449+0000","last_change":"2024-04-24T22:59:04.531771+0000","last_active":"2024-04-24T22:59:06.072449+0000","last_peered":"2024-04-24T22:59:06.072449+0000","last_clean":"2024-04-24T22:59:06.072449+0000","last_became_active":"2024-04-24T22:59:04.529534+0000","last_became_peered":"2024-04-24T22:59:04.529534+0000","last_unstale":"2024-04-24T22:59:06.072449+0000","last_undegraded":"2024-04-24T22:59:06.072449+0000","last_fullsized":"2024-04-24T22:59:06.072449+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:35:46.694552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.521763+0000","last_change":"2024-04-24T22:59:04.529880+0000","last_active":"2024-04-24T22:59:05.521763+0000","last_peered":"2024-04-24T22:59:05.521763+0000","last_clean":"2024-04-24T22:59:05.521763+0000","last_became_active":"2024-04-24T22:59:04.529596+0000","last_became_peered":"2024-04-24T22:59:04.529596+0000","last_unstale":"2024-04-24T22:59:05.521763+0000","last_undegraded":"2024-04-24T22:59:05.521763+0000","last_fullsized":"2024-04-24T22:59:05.521763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:45:47.457534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072563+0000","last_change":"2024-04-24T22:59:04.535003+0000","last_active":"2024-04-24T22:59:06.072563+0000","last_peered":"2024-04-24T22:59:06.072563+0000","last_clean":"2024-04-24T22:59:06.072563+0000","last_became_active":"2024-04-24T22:59:04.532802+0000","last_became_peered":"2024-04-24T22:59:04.532802+0000","last_unstale":"2024-04-24T22:59:06.072563+0000","last_undegraded":"2024-04-24T22:59:06.072563+0000","last_fullsized":"2024-04-24T22:59:06.072563+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:17.932086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.571441+0000","last_change":"2024-04-24T22:59:04.537484+0000","last_active":"2024-04-24T22:59:05.571441+0000","last_peered":"2024-04-24T22:59:05.571441+0000","last_clean":"2024-04-24T22:59:05.571441+0000","last_became_active":"2024-04-24T22:59:04.537018+0000","last_became_peered":"2024-04-24T22:59:04.537018+0000","last_unstale":"2024-04-24T22:59:05.571441+0000","last_undegraded":"2024-04-24T22:59:05.571441+0000","last_fullsized":"2024-04-24T22:59:05.571441+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:50:51.056976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.802481+0000","last_change":"2024-04-24T22:59:04.524856+0000","last_active":"2024-04-24T22:59:05.802481+0000","last_peered":"2024-04-24T22:59:05.802481+0000","last_clean":"2024-04-24T22:59:05.802481+0000","last_became_active":"2024-04-24T22:59:04.524572+0000","last_became_peered":"2024-04-24T22:59:04.524572+0000","last_unstale":"2024-04-24T22:59:05.802481+0000","last_undegraded":"2024-04-24T22:59:05.802481+0000","last_fullsized":"2024-04-24T22:59:05.802481+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:41.537600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072207+0000","last_change":"2024-04-24T22:59:04.533184+0000","last_active":"2024-04-24T22:59:06.072207+0000","last_peered":"2024-04-24T22:59:06.072207+0000","last_clean":"2024-04-24T22:59:06.072207+0000","last_became_active":"2024-04-24T22:59:04.532176+0000","last_became_peered":"2024-04-24T22:59:04.532176+0000","last_unstale":"2024-04-24T22:59:06.072207+0000","last_undegraded":"2024-04-24T22:59:06.072207+0000","last_fullsized":"2024-04-24T22:59:06.072207+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:43:05.960134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072278+0000","last_change":"2024-04-24T22:59:04.533428+0000","last_active":"2024-04-24T22:59:06.072278+0000","last_peered":"2024-04-24T22:59:06.072278+0000","last_clean":"2024-04-24T22:59:06.072278+0000","last_became_active":"2024-04-24T22:59:04.532740+0000","last_became_peered":"2024-04-24T22:59:04.532740+0000","last_unstale":"2024-04-24T22:59:06.072278+0000","last_undegraded":"2024-04-24T22:59:06.072278+0000","last_fullsized":"2024-04-24T22:59:06.072278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:45:08.272236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072073+0000","last_change":"2024-04-24T22:59:04.538772+0000","last_active":"2024-04-24T22:59:06.072073+0000","last_peered":"2024-04-24T22:59:06.072073+0000","last_clean":"2024-04-24T22:59:06.072073+0000","last_became_active":"2024-04-24T22:59:04.537512+0000","last_became_peered":"2024-04-24T22:59:04.537512+0000","last_unstale":"2024-04-24T22:59:06.072073+0000","last_undegraded":"2024-04-24T22:59:06.072073+0000","last_fullsized":"2024-04-24T22:59:06.072073+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:17.297900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520613+0000","last_change":"2024-04-24T22:59:04.534720+0000","last_active":"2024-04-24T22:59:05.520613+0000","last_peered":"2024-04-24T22:59:05.520613+0000","last_clean":"2024-04-24T22:59:05.520613+0000","last_became_active":"2024-04-24T22:59:04.533802+0000","last_became_peered":"2024-04-24T22:59:04.533802+0000","last_unstale":"2024-04-24T22:59:05.520613+0000","last_undegraded":"2024-04-24T22:59:05.520613+0000","last_fullsized":"2024-04-24T22:59:05.520613+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:21:36.150878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520463+0000","last_change":"2024-04-24T22:59:04.532980+0000","last_active":"2024-04-24T22:59:05.520463+0000","last_peered":"2024-04-24T22:59:05.520463+0000","last_clean":"2024-04-24T22:59:05.520463+0000","last_became_active":"2024-04-24T22:59:04.532721+0000","last_became_peered":"2024-04-24T22:59:04.532721+0000","last_unstale":"2024-04-24T22:59:05.520463+0000","last_undegraded":"2024-04-24T22:59:05.520463+0000","last_fullsized":"2024-04-24T22:59:05.520463+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:07:57.327727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.571539+0000","last_change":"2024-04-24T22:59:04.529907+0000","last_active":"2024-04-24T22:59:05.571539+0000","last_peered":"2024-04-24T22:59:05.571539+0000","last_clean":"2024-04-24T22:59:05.571539+0000","last_became_active":"2024-04-24T22:59:04.529501+0000","last_became_peered":"2024-04-24T22:59:04.529501+0000","last_unstale":"2024-04-24T22:59:05.571539+0000","last_undegraded":"2024-04-24T22:59:05.571539+0000","last_fullsized":"2024-04-24T22:59:05.571539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:44:58.795998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520176+0000","last_change":"2024-04-24T22:59:04.531489+0000","last_active":"2024-04-24T22:59:05.520176+0000","last_peered":"2024-04-24T22:59:05.520176+0000","last_clean":"2024-04-24T22:59:05.520176+0000","last_became_active":"2024-04-24T22:59:04.531251+0000","last_became_peered":"2024-04-24T22:59:04.531251+0000","last_unstale":"2024-04-24T22:59:05.520176+0000","last_undegraded":"2024-04-24T22:59:05.520176+0000","last_fullsized":"2024-04-24T22:59:05.520176+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:04:17.107583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.519872+0000","last_change":"2024-04-24T22:59:04.533695+0000","last_active":"2024-04-24T22:59:05.519872+0000","last_peered":"2024-04-24T22:59:05.519872+0000","last_clean":"2024-04-24T22:59:05.519872+0000","last_became_active":"2024-04-24T22:59:04.533446+0000","last_became_peered":"2024-04-24T22:59:04.533446+0000","last_unstale":"2024-04-24T22:59:05.519872+0000","last_undegraded":"2024-04-24T22:59:05.519872+0000","last_fullsized":"2024-04-24T22:59:05.519872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:38.140275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072156+0000","last_change":"2024-04-24T22:59:04.534240+0000","last_active":"2024-04-24T22:59:06.072156+0000","last_peered":"2024-04-24T22:59:06.072156+0000","last_clean":"2024-04-24T22:59:06.072156+0000","last_became_active":"2024-04-24T22:59:04.533710+0000","last_became_peered":"2024-04-24T22:59:04.533710+0000","last_unstale":"2024-04-24T22:59:06.072156+0000","last_undegraded":"2024-04-24T22:59:06.072156+0000","last_fullsized":"2024-04-24T22:59:06.072156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:58:18.235537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520204+0000","last_change":"2024-04-24T22:59:04.529089+0000","last_active":"2024-04-24T22:59:05.520204+0000","last_peered":"2024-04-24T22:59:05.520204+0000","last_clean":"2024-04-24T22:59:05.520204+0000","last_became_active":"2024-04-24T22:59:04.528080+0000","last_became_peered":"2024-04-24T22:59:04.528080+0000","last_unstale":"2024-04-24T22:59:05.520204+0000","last_undegraded":"2024-04-24T22:59:05.520204+0000","last_fullsized":"2024-04-24T22:59:05.520204+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:26:32.275407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072288+0000","last_change":"2024-04-24T22:59:04.537977+0000","last_active":"2024-04-24T22:59:06.072288+0000","last_peered":"2024-04-24T22:59:06.072288+0000","last_clean":"2024-04-24T22:59:06.072288+0000","last_became_active":"2024-04-24T22:59:04.537758+0000","last_became_peered":"2024-04-24T22:59:04.537758+0000","last_unstale":"2024-04-24T22:59:06.072288+0000","last_undegraded":"2024-04-24T22:59:06.072288+0000","last_fullsized":"2024-04-24T22:59:06.072288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:02:57.321693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072591+0000","last_change":"2024-04-24T22:59:04.527359+0000","last_active":"2024-04-24T22:59:06.072591+0000","last_peered":"2024-04-24T22:59:06.072591+0000","last_clean":"2024-04-24T22:59:06.072591+0000","last_became_active":"2024-04-24T22:59:04.527093+0000","last_became_peered":"2024-04-24T22:59:04.527093+0000","last_unstale":"2024-04-24T22:59:06.072591+0000","last_undegraded":"2024-04-24T22:59:06.072591+0000","last_fullsized":"2024-04-24T22:59:06.072591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:51:52.321592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.802409+0000","last_change":"2024-04-24T22:59:04.531189+0000","last_active":"2024-04-24T22:59:05.802409+0000","last_peered":"2024-04-24T22:59:05.802409+0000","last_clean":"2024-04-24T22:59:05.802409+0000","last_became_active":"2024-04-24T22:59:04.530929+0000","last_became_peered":"2024-04-24T22:59:04.530929+0000","last_unstale":"2024-04-24T22:59:05.802409+0000","last_undegraded":"2024-04-24T22:59:05.802409+0000","last_fullsized":"2024-04-24T22:59:05.802409+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:36.401052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.571534+0000","last_change":"2024-04-24T22:59:04.537735+0000","last_active":"2024-04-24T22:59:05.571534+0000","last_peered":"2024-04-24T22:59:05.571534+0000","last_clean":"2024-04-24T22:59:05.571534+0000","last_became_active":"2024-04-24T22:59:04.537287+0000","last_became_peered":"2024-04-24T22:59:04.537287+0000","last_unstale":"2024-04-24T22:59:05.571534+0000","last_undegraded":"2024-04-24T22:59:05.571534+0000","last_fullsized":"2024-04-24T22:59:05.571534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:25.003190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.519754+0000","last_change":"2024-04-24T22:59:04.530883+0000","last_active":"2024-04-24T22:59:05.519754+0000","last_peered":"2024-04-24T22:59:05.519754+0000","last_clean":"2024-04-24T22:59:05.519754+0000","last_became_active":"2024-04-24T22:59:04.530647+0000","last_became_peered":"2024-04-24T22:59:04.530647+0000","last_unstale":"2024-04-24T22:59:05.519754+0000","last_undegraded":"2024-04-24T22:59:05.519754+0000","last_fullsized":"2024-04-24T22:59:05.519754+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:54:21.680277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.571676+0000","last_change":"2024-04-24T22:59:04.535874+0000","last_active":"2024-04-24T22:59:05.571676+0000","last_peered":"2024-04-24T22:59:05.571676+0000","last_clean":"2024-04-24T22:59:05.571676+0000","last_became_active":"2024-04-24T22:59:04.535623+0000","last_became_peered":"2024-04-24T22:59:04.535623+0000","last_unstale":"2024-04-24T22:59:05.571676+0000","last_undegraded":"2024-04-24T22:59:05.571676+0000","last_fullsized":"2024-04-24T22:59:05.571676+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:34:41.433939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.521847+0000","last_change":"2024-04-24T22:59:04.534070+0000","last_active":"2024-04-24T22:59:05.521847+0000","last_peered":"2024-04-24T22:59:05.521847+0000","last_clean":"2024-04-24T22:59:05.521847+0000","last_became_active":"2024-04-24T22:59:04.533801+0000","last_became_peered":"2024-04-24T22:59:04.533801+0000","last_unstale":"2024-04-24T22:59:05.521847+0000","last_undegraded":"2024-04-24T22:59:05.521847+0000","last_fullsized":"2024-04-24T22:59:05.521847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:51:46.927874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520818+0000","last_change":"2024-04-24T22:59:04.531614+0000","last_active":"2024-04-24T22:59:05.520818+0000","last_peered":"2024-04-24T22:59:05.520818+0000","last_clean":"2024-04-24T22:59:05.520818+0000","last_became_active":"2024-04-24T22:59:04.530752+0000","last_became_peered":"2024-04-24T22:59:04.530752+0000","last_unstale":"2024-04-24T22:59:05.520818+0000","last_undegraded":"2024-04-24T22:59:05.520818+0000","last_fullsized":"2024-04-24T22:59:05.520818+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:15:43.008089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520758+0000","last_change":"2024-04-24T22:59:04.534458+0000","last_active":"2024-04-24T22:59:05.520758+0000","last_peered":"2024-04-24T22:59:05.520758+0000","last_clean":"2024-04-24T22:59:05.520758+0000","last_became_active":"2024-04-24T22:59:04.533333+0000","last_became_peered":"2024-04-24T22:59:04.533333+0000","last_unstale":"2024-04-24T22:59:05.520758+0000","last_undegraded":"2024-04-24T22:59:05.520758+0000","last_fullsized":"2024-04-24T22:59:05.520758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:16:16.466196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.571278+0000","last_change":"2024-04-24T22:59:04.533761+0000","last_active":"2024-04-24T22:59:05.571278+0000","last_peered":"2024-04-24T22:59:05.571278+0000","last_clean":"2024-04-24T22:59:05.571278+0000","last_became_active":"2024-04-24T22:59:04.533321+0000","last_became_peered":"2024-04-24T22:59:04.533321+0000","last_unstale":"2024-04-24T22:59:05.571278+0000","last_undegraded":"2024-04-24T22:59:05.571278+0000","last_fullsized":"2024-04-24T22:59:05.571278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:19:02.943510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.519970+0000","last_change":"2024-04-24T22:59:04.528638+0000","last_active":"2024-04-24T22:59:05.519970+0000","last_peered":"2024-04-24T22:59:05.519970+0000","last_clean":"2024-04-24T22:59:05.519970+0000","last_became_active":"2024-04-24T22:59:04.528413+0000","last_became_peered":"2024-04-24T22:59:04.528413+0000","last_unstale":"2024-04-24T22:59:05.519970+0000","last_undegraded":"2024-04-24T22:59:05.519970+0000","last_fullsized":"2024-04-24T22:59:05.519970+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:29:38.333908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072338+0000","last_change":"2024-04-24T22:59:04.537062+0000","last_active":"2024-04-24T22:59:06.072338+0000","last_peered":"2024-04-24T22:59:06.072338+0000","last_clean":"2024-04-24T22:59:06.072338+0000","last_became_active":"2024-04-24T22:59:04.536623+0000","last_became_peered":"2024-04-24T22:59:04.536623+0000","last_unstale":"2024-04-24T22:59:06.072338+0000","last_undegraded":"2024-04-24T22:59:06.072338+0000","last_fullsized":"2024-04-24T22:59:06.072338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:42:07.950747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.521248+0000","last_change":"2024-04-24T22:59:04.528631+0000","last_active":"2024-04-24T22:59:05.521248+0000","last_peered":"2024-04-24T22:59:05.521248+0000","last_clean":"2024-04-24T22:59:05.521248+0000","last_became_active":"2024-04-24T22:59:04.527962+0000","last_became_peered":"2024-04-24T22:59:04.527962+0000","last_unstale":"2024-04-24T22:59:05.521248+0000","last_undegraded":"2024-04-24T22:59:05.521248+0000","last_fullsized":"2024-04-24T22:59:05.521248+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:55:25.352217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.802375+0000","last_change":"2024-04-24T22:59:04.536016+0000","last_active":"2024-04-24T22:59:05.802375+0000","last_peered":"2024-04-24T22:59:05.802375+0000","last_clean":"2024-04-24T22:59:05.802375+0000","last_became_active":"2024-04-24T22:59:04.535746+0000","last_became_peered":"2024-04-24T22:59:04.535746+0000","last_unstale":"2024-04-24T22:59:05.802375+0000","last_undegraded":"2024-04-24T22:59:05.802375+0000","last_fullsized":"2024-04-24T22:59:05.802375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:02:30.961716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.571464+0000","last_change":"2024-04-24T22:59:03.508028+0000","last_active":"2024-04-24T22:59:05.571464+0000","last_peered":"2024-04-24T22:59:05.571464+0000","last_clean":"2024-04-24T22:59:05.571464+0000","last_became_active":"2024-04-24T22:59:03.507627+0000","last_became_peered":"2024-04-24T22:59:03.507627+0000","last_unstale":"2024-04-24T22:59:05.571464+0000","last_undegraded":"2024-04-24T22:59:05.571464+0000","last_fullsized":"2024-04-24T22:59:05.571464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:40:21.857692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072150+0000","last_change":"2024-04-24T22:59:04.534868+0000","last_active":"2024-04-24T22:59:06.072150+0000","last_peered":"2024-04-24T22:59:06.072150+0000","last_clean":"2024-04-24T22:59:06.072150+0000","last_became_active":"2024-04-24T22:59:04.534232+0000","last_became_peered":"2024-04-24T22:59:04.534232+0000","last_unstale":"2024-04-24T22:59:06.072150+0000","last_undegraded":"2024-04-24T22:59:06.072150+0000","last_fullsized":"2024-04-24T22:59:06.072150+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:29.091605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.519708+0000","last_change":"2024-04-24T22:59:03.511649+0000","last_active":"2024-04-24T22:59:05.519708+0000","last_peered":"2024-04-24T22:59:05.519708+0000","last_clean":"2024-04-24T22:59:05.519708+0000","last_became_active":"2024-04-24T22:59:03.511079+0000","last_became_peered":"2024-04-24T22:59:03.511079+0000","last_unstale":"2024-04-24T22:59:05.519708+0000","last_undegraded":"2024-04-24T22:59:05.519708+0000","last_fullsized":"2024-04-24T22:59:05.519708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:08.640377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520592+0000","last_change":"2024-04-24T22:59:03.512099+0000","last_active":"2024-04-24T22:59:05.520592+0000","last_peered":"2024-04-24T22:59:05.520592+0000","last_clean":"2024-04-24T22:59:05.520592+0000","last_became_active":"2024-04-24T22:59:03.511903+0000","last_became_peered":"2024-04-24T22:59:03.511903+0000","last_unstale":"2024-04-24T22:59:05.520592+0000","last_undegraded":"2024-04-24T22:59:05.520592+0000","last_fullsized":"2024-04-24T22:59:05.520592+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:06.046861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.571807+0000","last_change":"2024-04-24T22:59:04.529177+0000","last_active":"2024-04-24T22:59:05.571807+0000","last_peered":"2024-04-24T22:59:05.571807+0000","last_clean":"2024-04-24T22:59:05.571807+0000","last_became_active":"2024-04-24T22:59:04.528730+0000","last_became_peered":"2024-04-24T22:59:04.528730+0000","last_unstale":"2024-04-24T22:59:05.571807+0000","last_undegraded":"2024-04-24T22:59:05.571807+0000","last_fullsized":"2024-04-24T22:59:05.571807+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:41:20.670964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520440+0000","last_change":"2024-04-24T22:59:03.510748+0000","last_active":"2024-04-24T22:59:05.520440+0000","last_peered":"2024-04-24T22:59:05.520440+0000","last_clean":"2024-04-24T22:59:05.520440+0000","last_became_active":"2024-04-24T22:59:03.510515+0000","last_became_peered":"2024-04-24T22:59:03.510515+0000","last_unstale":"2024-04-24T22:59:05.520440+0000","last_undegraded":"2024-04-24T22:59:05.520440+0000","last_fullsized":"2024-04-24T22:59:05.520440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:03:30.220338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072114+0000","last_change":"2024-04-24T22:59:04.523319+0000","last_active":"2024-04-24T22:59:06.072114+0000","last_peered":"2024-04-24T22:59:06.072114+0000","last_clean":"2024-04-24T22:59:06.072114+0000","last_became_active":"2024-04-24T22:59:04.523086+0000","last_became_peered":"2024-04-24T22:59:04.523086+0000","last_unstale":"2024-04-24T22:59:06.072114+0000","last_undegraded":"2024-04-24T22:59:06.072114+0000","last_fullsized":"2024-04-24T22:59:06.072114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:27.103005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.802709+0000","last_change":"2024-04-24T22:59:03.510379+0000","last_active":"2024-04-24T22:59:05.802709+0000","last_peered":"2024-04-24T22:59:05.802709+0000","last_clean":"2024-04-24T22:59:05.802709+0000","last_became_active":"2024-04-24T22:59:03.510070+0000","last_became_peered":"2024-04-24T22:59:03.510070+0000","last_unstale":"2024-04-24T22:59:05.802709+0000","last_undegraded":"2024-04-24T22:59:05.802709+0000","last_fullsized":"2024-04-24T22:59:05.802709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:21:24.085269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.521658+0000","last_change":"2024-04-24T22:59:04.534163+0000","last_active":"2024-04-24T22:59:05.521658+0000","last_peered":"2024-04-24T22:59:05.521658+0000","last_clean":"2024-04-24T22:59:05.521658+0000","last_became_active":"2024-04-24T22:59:04.533861+0000","last_became_peered":"2024-04-24T22:59:04.533861+0000","last_unstale":"2024-04-24T22:59:05.521658+0000","last_undegraded":"2024-04-24T22:59:05.521658+0000","last_fullsized":"2024-04-24T22:59:05.521658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:02:58.886576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072490+0000","last_change":"2024-04-24T22:59:03.512403+0000","last_active":"2024-04-24T22:59:06.072490+0000","last_peered":"2024-04-24T22:59:06.072490+0000","last_clean":"2024-04-24T22:59:06.072490+0000","last_became_active":"2024-04-24T22:59:03.511538+0000","last_became_peered":"2024-04-24T22:59:03.511538+0000","last_unstale":"2024-04-24T22:59:06.072490+0000","last_undegraded":"2024-04-24T22:59:06.072490+0000","last_fullsized":"2024-04-24T22:59:06.072490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:08:00.814274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.571343+0000","last_change":"2024-04-24T22:59:04.532532+0000","last_active":"2024-04-24T22:59:05.571343+0000","last_peered":"2024-04-24T22:59:05.571343+0000","last_clean":"2024-04-24T22:59:05.571343+0000","last_became_active":"2024-04-24T22:59:04.532308+0000","last_became_peered":"2024-04-24T22:59:04.532308+0000","last_unstale":"2024-04-24T22:59:05.571343+0000","last_undegraded":"2024-04-24T22:59:05.571343+0000","last_fullsized":"2024-04-24T22:59:05.571343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:07:10.769633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072409+0000","last_change":"2024-04-24T22:59:03.509245+0000","last_active":"2024-04-24T22:59:06.072409+0000","last_peered":"2024-04-24T22:59:06.072409+0000","last_clean":"2024-04-24T22:59:06.072409+0000","last_became_active":"2024-04-24T22:59:03.508959+0000","last_became_peered":"2024-04-24T22:59:03.508959+0000","last_unstale":"2024-04-24T22:59:06.072409+0000","last_undegraded":"2024-04-24T22:59:06.072409+0000","last_fullsized":"2024-04-24T22:59:06.072409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:28:24.443582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.519926+0000","last_change":"2024-04-24T22:59:04.537036+0000","last_active":"2024-04-24T22:59:05.519926+0000","last_peered":"2024-04-24T22:59:05.519926+0000","last_clean":"2024-04-24T22:59:05.519926+0000","last_became_active":"2024-04-24T22:59:04.536728+0000","last_became_peered":"2024-04-24T22:59:04.536728+0000","last_unstale":"2024-04-24T22:59:05.519926+0000","last_undegraded":"2024-04-24T22:59:05.519926+0000","last_fullsized":"2024-04-24T22:59:05.519926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:14:39.754057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.521704+0000","last_change":"2024-04-24T22:59:03.508470+0000","last_active":"2024-04-24T22:59:05.521704+0000","last_peered":"2024-04-24T22:59:05.521704+0000","last_clean":"2024-04-24T22:59:05.521704+0000","last_became_active":"2024-04-24T22:59:03.508200+0000","last_became_peered":"2024-04-24T22:59:03.508200+0000","last_unstale":"2024-04-24T22:59:05.521704+0000","last_undegraded":"2024-04-24T22:59:05.521704+0000","last_fullsized":"2024-04-24T22:59:05.521704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:54.077458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072276+0000","last_change":"2024-04-24T22:59:04.532595+0000","last_active":"2024-04-24T22:59:06.072276+0000","last_peered":"2024-04-24T22:59:06.072276+0000","last_clean":"2024-04-24T22:59:06.072276+0000","last_became_active":"2024-04-24T22:59:04.532203+0000","last_became_peered":"2024-04-24T22:59:04.532203+0000","last_unstale":"2024-04-24T22:59:06.072276+0000","last_undegraded":"2024-04-24T22:59:06.072276+0000","last_fullsized":"2024-04-24T22:59:06.072276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:27:13.861775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072047+0000","last_change":"2024-04-24T22:59:03.509839+0000","last_active":"2024-04-24T22:59:06.072047+0000","last_peered":"2024-04-24T22:59:06.072047+0000","last_clean":"2024-04-24T22:59:06.072047+0000","last_became_active":"2024-04-24T22:59:03.509598+0000","last_became_peered":"2024-04-24T22:59:03.509598+0000","last_unstale":"2024-04-24T22:59:06.072047+0000","last_undegraded":"2024-04-24T22:59:06.072047+0000","last_fullsized":"2024-04-24T22:59:06.072047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:26:55.328076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520508+0000","last_change":"2024-04-24T22:59:04.531989+0000","last_active":"2024-04-24T22:59:05.520508+0000","last_peered":"2024-04-24T22:59:05.520508+0000","last_clean":"2024-04-24T22:59:05.520508+0000","last_became_active":"2024-04-24T22:59:04.531739+0000","last_became_peered":"2024-04-24T22:59:04.531739+0000","last_unstale":"2024-04-24T22:59:05.520508+0000","last_undegraded":"2024-04-24T22:59:05.520508+0000","last_fullsized":"2024-04-24T22:59:05.520508+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:38:53.496201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520273+0000","last_change":"2024-04-24T22:59:03.504141+0000","last_active":"2024-04-24T22:59:05.520273+0000","last_peered":"2024-04-24T22:59:05.520273+0000","last_clean":"2024-04-24T22:59:05.520273+0000","last_became_active":"2024-04-24T22:59:03.503898+0000","last_became_peered":"2024-04-24T22:59:03.503898+0000","last_unstale":"2024-04-24T22:59:05.520273+0000","last_undegraded":"2024-04-24T22:59:05.520273+0000","last_fullsized":"2024-04-24T22:59:05.520273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:51:28.639755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520233+0000","last_change":"2024-04-24T22:59:04.535653+0000","last_active":"2024-04-24T22:59:05.520233+0000","last_peered":"2024-04-24T22:59:05.520233+0000","last_clean":"2024-04-24T22:59:05.520233+0000","last_became_active":"2024-04-24T22:59:04.535211+0000","last_became_peered":"2024-04-24T22:59:04.535211+0000","last_unstale":"2024-04-24T22:59:05.520233+0000","last_undegraded":"2024-04-24T22:59:05.520233+0000","last_fullsized":"2024-04-24T22:59:05.520233+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:14.793006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072608+0000","last_change":"2024-04-24T22:59:03.514558+0000","last_active":"2024-04-24T22:59:06.072608+0000","last_peered":"2024-04-24T22:59:06.072608+0000","last_clean":"2024-04-24T22:59:06.072608+0000","last_became_active":"2024-04-24T22:59:03.514168+0000","last_became_peered":"2024-04-24T22:59:03.514168+0000","last_unstale":"2024-04-24T22:59:06.072608+0000","last_undegraded":"2024-04-24T22:59:06.072608+0000","last_fullsized":"2024-04-24T22:59:06.072608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:42:06.949022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.802534+0000","last_change":"2024-04-24T22:59:04.536979+0000","last_active":"2024-04-24T22:59:05.802534+0000","last_peered":"2024-04-24T22:59:05.802534+0000","last_clean":"2024-04-24T22:59:05.802534+0000","last_became_active":"2024-04-24T22:59:04.536749+0000","last_became_peered":"2024-04-24T22:59:04.536749+0000","last_unstale":"2024-04-24T22:59:05.802534+0000","last_undegraded":"2024-04-24T22:59:05.802534+0000","last_fullsized":"2024-04-24T22:59:05.802534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:06:11.147097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072411+0000","last_change":"2024-04-24T22:59:03.507780+0000","last_active":"2024-04-24T22:59:06.072411+0000","last_peered":"2024-04-24T22:59:06.072411+0000","last_clean":"2024-04-24T22:59:06.072411+0000","last_became_active":"2024-04-24T22:59:03.507476+0000","last_became_peered":"2024-04-24T22:59:03.507476+0000","last_unstale":"2024-04-24T22:59:06.072411+0000","last_undegraded":"2024-04-24T22:59:06.072411+0000","last_fullsized":"2024-04-24T22:59:06.072411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:02.130252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520033+0000","last_change":"2024-04-24T22:59:04.532342+0000","last_active":"2024-04-24T22:59:05.520033+0000","last_peered":"2024-04-24T22:59:05.520033+0000","last_clean":"2024-04-24T22:59:05.520033+0000","last_became_active":"2024-04-24T22:59:04.531830+0000","last_became_peered":"2024-04-24T22:59:04.531830+0000","last_unstale":"2024-04-24T22:59:05.520033+0000","last_undegraded":"2024-04-24T22:59:05.520033+0000","last_fullsized":"2024-04-24T22:59:05.520033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:26.483089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072081+0000","last_change":"2024-04-24T22:59:03.511265+0000","last_active":"2024-04-24T22:59:06.072081+0000","last_peered":"2024-04-24T22:59:06.072081+0000","last_clean":"2024-04-24T22:59:06.072081+0000","last_became_active":"2024-04-24T22:59:03.510105+0000","last_became_peered":"2024-04-24T22:59:03.510105+0000","last_unstale":"2024-04-24T22:59:06.072081+0000","last_undegraded":"2024-04-24T22:59:06.072081+0000","last_fullsized":"2024-04-24T22:59:06.072081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:14:10.398082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.521760+0000","last_change":"2024-04-24T22:59:04.535582+0000","last_active":"2024-04-24T22:59:05.521760+0000","last_peered":"2024-04-24T22:59:05.521760+0000","last_clean":"2024-04-24T22:59:05.521760+0000","last_became_active":"2024-04-24T22:59:04.535184+0000","last_became_peered":"2024-04-24T22:59:04.535184+0000","last_unstale":"2024-04-24T22:59:05.521760+0000","last_undegraded":"2024-04-24T22:59:05.521760+0000","last_fullsized":"2024-04-24T22:59:05.521760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:14:13.409713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072466+0000","last_change":"2024-04-24T22:59:03.511495+0000","last_active":"2024-04-24T22:59:06.072466+0000","last_peered":"2024-04-24T22:59:06.072466+0000","last_clean":"2024-04-24T22:59:06.072466+0000","last_became_active":"2024-04-24T22:59:03.511067+0000","last_became_peered":"2024-04-24T22:59:03.511067+0000","last_unstale":"2024-04-24T22:59:06.072466+0000","last_undegraded":"2024-04-24T22:59:06.072466+0000","last_fullsized":"2024-04-24T22:59:06.072466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:25.766950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.071949+0000","last_change":"2024-04-24T22:59:04.538030+0000","last_active":"2024-04-24T22:59:06.071949+0000","last_peered":"2024-04-24T22:59:06.071949+0000","last_clean":"2024-04-24T22:59:06.071949+0000","last_became_active":"2024-04-24T22:59:04.537558+0000","last_became_peered":"2024-04-24T22:59:04.537558+0000","last_unstale":"2024-04-24T22:59:06.071949+0000","last_undegraded":"2024-04-24T22:59:06.071949+0000","last_fullsized":"2024-04-24T22:59:06.071949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:49:31.717151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.571432+0000","last_change":"2024-04-24T22:59:03.510195+0000","last_active":"2024-04-24T22:59:05.571432+0000","last_peered":"2024-04-24T22:59:05.571432+0000","last_clean":"2024-04-24T22:59:05.571432+0000","last_became_active":"2024-04-24T22:59:03.509950+0000","last_became_peered":"2024-04-24T22:59:03.509950+0000","last_unstale":"2024-04-24T22:59:05.571432+0000","last_undegraded":"2024-04-24T22:59:05.571432+0000","last_fullsized":"2024-04-24T22:59:05.571432+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:25:40.587061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072293+0000","last_change":"2024-04-24T22:59:04.535501+0000","last_active":"2024-04-24T22:59:06.072293+0000","last_peered":"2024-04-24T22:59:06.072293+0000","last_clean":"2024-04-24T22:59:06.072293+0000","last_became_active":"2024-04-24T22:59:04.534547+0000","last_became_peered":"2024-04-24T22:59:04.534547+0000","last_unstale":"2024-04-24T22:59:06.072293+0000","last_undegraded":"2024-04-24T22:59:06.072293+0000","last_fullsized":"2024-04-24T22:59:06.072293+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:49:20.073672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520927+0000","last_change":"2024-04-24T22:59:03.512241+0000","last_active":"2024-04-24T22:59:05.520927+0000","last_peered":"2024-04-24T22:59:05.520927+0000","last_clean":"2024-04-24T22:59:05.520927+0000","last_became_active":"2024-04-24T22:59:03.511991+0000","last_became_peered":"2024-04-24T22:59:03.511991+0000","last_unstale":"2024-04-24T22:59:05.520927+0000","last_undegraded":"2024-04-24T22:59:05.520927+0000","last_fullsized":"2024-04-24T22:59:05.520927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:00:24.103638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520418+0000","last_change":"2024-04-24T22:59:04.524264+0000","last_active":"2024-04-24T22:59:05.520418+0000","last_peered":"2024-04-24T22:59:05.520418+0000","last_clean":"2024-04-24T22:59:05.520418+0000","last_became_active":"2024-04-24T22:59:04.523991+0000","last_became_peered":"2024-04-24T22:59:04.523991+0000","last_unstale":"2024-04-24T22:59:05.520418+0000","last_undegraded":"2024-04-24T22:59:05.520418+0000","last_fullsized":"2024-04-24T22:59:05.520418+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:50.932498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.802551+0000","last_change":"2024-04-24T22:58:55.138810+0000","last_active":"2024-04-24T22:59:05.802551+0000","last_peered":"2024-04-24T22:59:05.802551+0000","last_clean":"2024-04-24T22:59:05.802551+0000","last_became_active":"2024-04-24T22:58:55.138627+0000","last_became_peered":"2024-04-24T22:58:55.138627+0000","last_unstale":"2024-04-24T22:59:05.802551+0000","last_undegraded":"2024-04-24T22:59:05.802551+0000","last_fullsized":"2024-04-24T22:59:05.802551+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_clean_scrub_stamp":"2024-04-24T22:58:54.127235+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:24.010002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520503+0000","last_change":"2024-04-24T22:59:03.501901+0000","last_active":"2024-04-24T22:59:05.520503+0000","last_peered":"2024-04-24T22:59:05.520503+0000","last_clean":"2024-04-24T22:59:05.520503+0000","last_became_active":"2024-04-24T22:59:03.501383+0000","last_became_peered":"2024-04-24T22:59:03.501383+0000","last_unstale":"2024-04-24T22:59:05.520503+0000","last_undegraded":"2024-04-24T22:59:05.520503+0000","last_fullsized":"2024-04-24T22:59:05.520503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:35:55.503771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072518+0000","last_change":"2024-04-24T22:59:04.531995+0000","last_active":"2024-04-24T22:59:06.072518+0000","last_peered":"2024-04-24T22:59:06.072518+0000","last_clean":"2024-04-24T22:59:06.072518+0000","last_became_active":"2024-04-24T22:59:04.531730+0000","last_became_peered":"2024-04-24T22:59:04.531730+0000","last_unstale":"2024-04-24T22:59:06.072518+0000","last_undegraded":"2024-04-24T22:59:06.072518+0000","last_fullsized":"2024-04-24T22:59:06.072518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:51:53.153038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520341+0000","last_change":"2024-04-24T22:59:03.509668+0000","last_active":"2024-04-24T22:59:05.520341+0000","last_peered":"2024-04-24T22:59:05.520341+0000","last_clean":"2024-04-24T22:59:05.520341+0000","last_became_active":"2024-04-24T22:59:03.509450+0000","last_became_peered":"2024-04-24T22:59:03.509450+0000","last_unstale":"2024-04-24T22:59:05.520341+0000","last_undegraded":"2024-04-24T22:59:05.520341+0000","last_fullsized":"2024-04-24T22:59:05.520341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:06:37.228507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520854+0000","last_change":"2024-04-24T22:59:04.530536+0000","last_active":"2024-04-24T22:59:05.520854+0000","last_peered":"2024-04-24T22:59:05.520854+0000","last_clean":"2024-04-24T22:59:05.520854+0000","last_became_active":"2024-04-24T22:59:04.530302+0000","last_became_peered":"2024-04-24T22:59:04.530302+0000","last_unstale":"2024-04-24T22:59:05.520854+0000","last_undegraded":"2024-04-24T22:59:05.520854+0000","last_fullsized":"2024-04-24T22:59:05.520854+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:38:05.122875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072508+0000","last_change":"2024-04-24T22:59:03.508258+0000","last_active":"2024-04-24T22:59:06.072508+0000","last_peered":"2024-04-24T22:59:06.072508+0000","last_clean":"2024-04-24T22:59:06.072508+0000","last_became_active":"2024-04-24T22:59:03.508013+0000","last_became_peered":"2024-04-24T22:59:03.508013+0000","last_unstale":"2024-04-24T22:59:06.072508+0000","last_undegraded":"2024-04-24T22:59:06.072508+0000","last_fullsized":"2024-04-24T22:59:06.072508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:04:08.375090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.571574+0000","last_change":"2024-04-24T22:59:04.533925+0000","last_active":"2024-04-24T22:59:05.571574+0000","last_peered":"2024-04-24T22:59:05.571574+0000","last_clean":"2024-04-24T22:59:05.571574+0000","last_became_active":"2024-04-24T22:59:04.533571+0000","last_became_peered":"2024-04-24T22:59:04.533571+0000","last_unstale":"2024-04-24T22:59:05.571574+0000","last_undegraded":"2024-04-24T22:59:05.571574+0000","last_fullsized":"2024-04-24T22:59:05.571574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:23:08.983760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520235+0000","last_change":"2024-04-24T22:59:03.503558+0000","last_active":"2024-04-24T22:59:05.520235+0000","last_peered":"2024-04-24T22:59:05.520235+0000","last_clean":"2024-04-24T22:59:05.520235+0000","last_became_active":"2024-04-24T22:59:03.503199+0000","last_became_peered":"2024-04-24T22:59:03.503199+0000","last_unstale":"2024-04-24T22:59:05.520235+0000","last_undegraded":"2024-04-24T22:59:05.520235+0000","last_fullsized":"2024-04-24T22:59:05.520235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:23:49.375509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072420+0000","last_change":"2024-04-24T22:59:04.531235+0000","last_active":"2024-04-24T22:59:06.072420+0000","last_peered":"2024-04-24T22:59:06.072420+0000","last_clean":"2024-04-24T22:59:06.072420+0000","last_became_active":"2024-04-24T22:59:04.530974+0000","last_became_peered":"2024-04-24T22:59:04.530974+0000","last_unstale":"2024-04-24T22:59:06.072420+0000","last_undegraded":"2024-04-24T22:59:06.072420+0000","last_fullsized":"2024-04-24T22:59:06.072420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:45:58.398513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.571979+0000","last_change":"2024-04-24T22:59:03.509319+0000","last_active":"2024-04-24T22:59:05.571979+0000","last_peered":"2024-04-24T22:59:05.571979+0000","last_clean":"2024-04-24T22:59:05.571979+0000","last_became_active":"2024-04-24T22:59:03.508493+0000","last_became_peered":"2024-04-24T22:59:03.508493+0000","last_unstale":"2024-04-24T22:59:05.571979+0000","last_undegraded":"2024-04-24T22:59:05.571979+0000","last_fullsized":"2024-04-24T22:59:05.571979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:49:50.756565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.521375+0000","last_change":"2024-04-24T22:59:04.527591+0000","last_active":"2024-04-24T22:59:05.521375+0000","last_peered":"2024-04-24T22:59:05.521375+0000","last_clean":"2024-04-24T22:59:05.521375+0000","last_became_active":"2024-04-24T22:59:04.527329+0000","last_became_peered":"2024-04-24T22:59:04.527329+0000","last_unstale":"2024-04-24T22:59:05.521375+0000","last_undegraded":"2024-04-24T22:59:05.521375+0000","last_fullsized":"2024-04-24T22:59:05.521375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:19:52.022880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520291+0000","last_change":"2024-04-24T22:59:03.512317+0000","last_active":"2024-04-24T22:59:05.520291+0000","last_peered":"2024-04-24T22:59:05.520291+0000","last_clean":"2024-04-24T22:59:05.520291+0000","last_became_active":"2024-04-24T22:59:03.512056+0000","last_became_peered":"2024-04-24T22:59:03.512056+0000","last_unstale":"2024-04-24T22:59:05.520291+0000","last_undegraded":"2024-04-24T22:59:05.520291+0000","last_fullsized":"2024-04-24T22:59:05.520291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:48:30.411017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520699+0000","last_change":"2024-04-24T22:59:04.530095+0000","last_active":"2024-04-24T22:59:05.520699+0000","last_peered":"2024-04-24T22:59:05.520699+0000","last_clean":"2024-04-24T22:59:05.520699+0000","last_became_active":"2024-04-24T22:59:04.529730+0000","last_became_peered":"2024-04-24T22:59:04.529730+0000","last_unstale":"2024-04-24T22:59:05.520699+0000","last_undegraded":"2024-04-24T22:59:05.520699+0000","last_fullsized":"2024-04-24T22:59:05.520699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:34.933698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.525549+0000","last_change":"2024-04-24T22:59:03.510515+0000","last_active":"2024-04-24T22:59:06.525549+0000","last_peered":"2024-04-24T22:59:06.525549+0000","last_clean":"2024-04-24T22:59:06.525549+0000","last_became_active":"2024-04-24T22:59:03.510016+0000","last_became_peered":"2024-04-24T22:59:03.510016+0000","last_unstale":"2024-04-24T22:59:06.525549+0000","last_undegraded":"2024-04-24T22:59:06.525549+0000","last_fullsized":"2024-04-24T22:59:06.525549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:51:23.939098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072377+0000","last_change":"2024-04-24T22:59:04.528509+0000","last_active":"2024-04-24T22:59:06.072377+0000","last_peered":"2024-04-24T22:59:06.072377+0000","last_clean":"2024-04-24T22:59:06.072377+0000","last_became_active":"2024-04-24T22:59:04.527604+0000","last_became_peered":"2024-04-24T22:59:04.527604+0000","last_unstale":"2024-04-24T22:59:06.072377+0000","last_undegraded":"2024-04-24T22:59:06.072377+0000","last_fullsized":"2024-04-24T22:59:06.072377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:13:55.192974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520140+0000","last_change":"2024-04-24T22:59:03.513495+0000","last_active":"2024-04-24T22:59:05.520140+0000","last_peered":"2024-04-24T22:59:05.520140+0000","last_clean":"2024-04-24T22:59:05.520140+0000","last_became_active":"2024-04-24T22:59:03.512772+0000","last_became_peered":"2024-04-24T22:59:03.512772+0000","last_unstale":"2024-04-24T22:59:05.520140+0000","last_undegraded":"2024-04-24T22:59:05.520140+0000","last_fullsized":"2024-04-24T22:59:05.520140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:49:12.718741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072288+0000","last_change":"2024-04-24T22:59:04.528601+0000","last_active":"2024-04-24T22:59:06.072288+0000","last_peered":"2024-04-24T22:59:06.072288+0000","last_clean":"2024-04-24T22:59:06.072288+0000","last_became_active":"2024-04-24T22:59:04.527812+0000","last_became_peered":"2024-04-24T22:59:04.527812+0000","last_unstale":"2024-04-24T22:59:06.072288+0000","last_undegraded":"2024-04-24T22:59:06.072288+0000","last_fullsized":"2024-04-24T22:59:06.072288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:19.252521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072624+0000","last_change":"2024-04-24T22:59:03.510200+0000","last_active":"2024-04-24T22:59:06.072624+0000","last_peered":"2024-04-24T22:59:06.072624+0000","last_clean":"2024-04-24T22:59:06.072624+0000","last_became_active":"2024-04-24T22:59:03.509783+0000","last_became_peered":"2024-04-24T22:59:03.509783+0000","last_unstale":"2024-04-24T22:59:06.072624+0000","last_undegraded":"2024-04-24T22:59:06.072624+0000","last_fullsized":"2024-04-24T22:59:06.072624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:47.755911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520361+0000","last_change":"2024-04-24T22:59:04.525874+0000","last_active":"2024-04-24T22:59:05.520361+0000","last_peered":"2024-04-24T22:59:05.520361+0000","last_clean":"2024-04-24T22:59:05.520361+0000","last_became_active":"2024-04-24T22:59:04.525094+0000","last_became_peered":"2024-04-24T22:59:04.525094+0000","last_unstale":"2024-04-24T22:59:05.520361+0000","last_undegraded":"2024-04-24T22:59:05.520361+0000","last_fullsized":"2024-04-24T22:59:05.520361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:06:23.254812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072568+0000","last_change":"2024-04-24T22:59:03.511287+0000","last_active":"2024-04-24T22:59:06.072568+0000","last_peered":"2024-04-24T22:59:06.072568+0000","last_clean":"2024-04-24T22:59:06.072568+0000","last_became_active":"2024-04-24T22:59:03.510784+0000","last_became_peered":"2024-04-24T22:59:03.510784+0000","last_unstale":"2024-04-24T22:59:06.072568+0000","last_undegraded":"2024-04-24T22:59:06.072568+0000","last_fullsized":"2024-04-24T22:59:06.072568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:19:48.671658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072736+0000","last_change":"2024-04-24T22:59:04.529663+0000","last_active":"2024-04-24T22:59:06.072736+0000","last_peered":"2024-04-24T22:59:06.072736+0000","last_clean":"2024-04-24T22:59:06.072736+0000","last_became_active":"2024-04-24T22:59:04.527858+0000","last_became_peered":"2024-04-24T22:59:04.527858+0000","last_unstale":"2024-04-24T22:59:06.072736+0000","last_undegraded":"2024-04-24T22:59:06.072736+0000","last_fullsized":"2024-04-24T22:59:06.072736+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:32:03.630786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520703+0000","last_change":"2024-04-24T22:59:03.505841+0000","last_active":"2024-04-24T22:59:05.520703+0000","last_peered":"2024-04-24T22:59:05.520703+0000","last_clean":"2024-04-24T22:59:05.520703+0000","last_became_active":"2024-04-24T22:59:03.505048+0000","last_became_peered":"2024-04-24T22:59:03.505048+0000","last_unstale":"2024-04-24T22:59:05.520703+0000","last_undegraded":"2024-04-24T22:59:05.520703+0000","last_fullsized":"2024-04-24T22:59:05.520703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:34:16.021505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072303+0000","last_change":"2024-04-24T22:59:04.532670+0000","last_active":"2024-04-24T22:59:06.072303+0000","last_peered":"2024-04-24T22:59:06.072303+0000","last_clean":"2024-04-24T22:59:06.072303+0000","last_became_active":"2024-04-24T22:59:04.532437+0000","last_became_peered":"2024-04-24T22:59:04.532437+0000","last_unstale":"2024-04-24T22:59:06.072303+0000","last_undegraded":"2024-04-24T22:59:06.072303+0000","last_fullsized":"2024-04-24T22:59:06.072303+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:08:01.815998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.519807+0000","last_change":"2024-04-24T22:59:03.509614+0000","last_active":"2024-04-24T22:59:05.519807+0000","last_peered":"2024-04-24T22:59:05.519807+0000","last_clean":"2024-04-24T22:59:05.519807+0000","last_became_active":"2024-04-24T22:59:03.509379+0000","last_became_peered":"2024-04-24T22:59:03.509379+0000","last_unstale":"2024-04-24T22:59:05.519807+0000","last_undegraded":"2024-04-24T22:59:05.519807+0000","last_fullsized":"2024-04-24T22:59:05.519807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:46:21.150240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520066+0000","last_change":"2024-04-24T22:59:04.526425+0000","last_active":"2024-04-24T22:59:05.520066+0000","last_peered":"2024-04-24T22:59:05.520066+0000","last_clean":"2024-04-24T22:59:05.520066+0000","last_became_active":"2024-04-24T22:59:04.524467+0000","last_became_peered":"2024-04-24T22:59:04.524467+0000","last_unstale":"2024-04-24T22:59:05.520066+0000","last_undegraded":"2024-04-24T22:59:05.520066+0000","last_fullsized":"2024-04-24T22:59:05.520066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:23:05.155059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.520105+0000","last_change":"2024-04-24T22:59:04.535456+0000","last_active":"2024-04-24T22:59:05.520105+0000","last_peered":"2024-04-24T22:59:05.520105+0000","last_clean":"2024-04-24T22:59:05.520105+0000","last_became_active":"2024-04-24T22:59:04.534881+0000","last_became_peered":"2024-04-24T22:59:04.534881+0000","last_unstale":"2024-04-24T22:59:05.520105+0000","last_undegraded":"2024-04-24T22:59:05.520105+0000","last_fullsized":"2024-04-24T22:59:05.520105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:01:15.740234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072535+0000","last_change":"2024-04-24T22:59:03.512029+0000","last_active":"2024-04-24T22:59:06.072535+0000","last_peered":"2024-04-24T22:59:06.072535+0000","last_clean":"2024-04-24T22:59:06.072535+0000","last_became_active":"2024-04-24T22:59:03.511808+0000","last_became_peered":"2024-04-24T22:59:03.511808+0000","last_unstale":"2024-04-24T22:59:06.072535+0000","last_undegraded":"2024-04-24T22:59:06.072535+0000","last_fullsized":"2024-04-24T22:59:06.072535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:02:26.617300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.802478+0000","last_change":"2024-04-24T22:59:04.533430+0000","last_active":"2024-04-24T22:59:05.802478+0000","last_peered":"2024-04-24T22:59:05.802478+0000","last_clean":"2024-04-24T22:59:05.802478+0000","last_became_active":"2024-04-24T22:59:04.533192+0000","last_became_peered":"2024-04-24T22:59:04.533192+0000","last_unstale":"2024-04-24T22:59:05.802478+0000","last_undegraded":"2024-04-24T22:59:05.802478+0000","last_fullsized":"2024-04-24T22:59:05.802478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:50:28.246419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072505+0000","last_change":"2024-04-24T22:59:03.512319+0000","last_active":"2024-04-24T22:59:06.072505+0000","last_peered":"2024-04-24T22:59:06.072505+0000","last_clean":"2024-04-24T22:59:06.072505+0000","last_became_active":"2024-04-24T22:59:03.511788+0000","last_became_peered":"2024-04-24T22:59:03.511788+0000","last_unstale":"2024-04-24T22:59:06.072505+0000","last_undegraded":"2024-04-24T22:59:06.072505+0000","last_fullsized":"2024-04-24T22:59:06.072505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:58:17.233812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.571660+0000","last_change":"2024-04-24T22:59:04.536036+0000","last_active":"2024-04-24T22:59:05.571660+0000","last_peered":"2024-04-24T22:59:05.571660+0000","last_clean":"2024-04-24T22:59:05.571660+0000","last_became_active":"2024-04-24T22:59:04.535591+0000","last_became_peered":"2024-04-24T22:59:04.535591+0000","last_unstale":"2024-04-24T22:59:05.571660+0000","last_undegraded":"2024-04-24T22:59:05.571660+0000","last_fullsized":"2024-04-24T22:59:05.571660+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:59.369829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.571777+0000","last_change":"2024-04-24T22:59:03.508163+0000","last_active":"2024-04-24T22:59:05.571777+0000","last_peered":"2024-04-24T22:59:05.571777+0000","last_clean":"2024-04-24T22:59:05.571777+0000","last_became_active":"2024-04-24T22:59:03.507645+0000","last_became_peered":"2024-04-24T22:59:03.507645+0000","last_unstale":"2024-04-24T22:59:05.571777+0000","last_undegraded":"2024-04-24T22:59:05.571777+0000","last_fullsized":"2024-04-24T22:59:05.571777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:28:40.818581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:05.571435+0000","last_change":"2024-04-24T22:59:04.538784+0000","last_active":"2024-04-24T22:59:05.571435+0000","last_peered":"2024-04-24T22:59:05.571435+0000","last_clean":"2024-04-24T22:59:05.571435+0000","last_became_active":"2024-04-24T22:59:04.537912+0000","last_became_peered":"2024-04-24T22:59:04.537912+0000","last_unstale":"2024-04-24T22:59:05.571435+0000","last_undegraded":"2024-04-24T22:59:05.571435+0000","last_fullsized":"2024-04-24T22:59:05.571435+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:03.494874+0000","last_clean_scrub_stamp":"2024-04-24T22:59:03.494874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:44.470849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:59:06.072107+0000","last_change":"2024-04-24T22:59:03.513331+0000","last_active":"2024-04-24T22:59:06.072107+0000","last_peered":"2024-04-24T22:59:06.072107+0000","last_clean":"2024-04-24T22:59:06.072107+0000","last_became_active":"2024-04-24T22:59:03.513198+0000","last_became_peered":"2024-04-24T22:59:03.513198+0000","last_unstale":"2024-04-24T22:59:06.072107+0000","last_undegraded":"2024-04-24T22:59:06.072107+0000","last_fullsized":"2024-04-24T22:59:06.072107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:02.493149+0000","last_clean_scrub_stamp":"2024-04-24T22:59:02.493149+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:34:31.185473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":16,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195238,"statfs":{"total":200000000,"available":199924431,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195238,"statfs":{"total":200000000,"available":199924555,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":650,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194662,"statfs":{"total":200000000,"available":199334095,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":75,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195236,"statfs":{"total":200000000,"available":199922271,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":74,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195238,"statfs":{"total":200000000,"available":199923973,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":75,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195237,"statfs":{"total":200000000,"available":199922943,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":650,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194662,"statfs":{"total":200000000,"available":199334117,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":35,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195238,"statfs":{"total":200000000,"available":199924499,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-24T22:59:16.714 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-24T22:59:16.714 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-24T22:59:16.715 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-24T22:59:16.715 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-24T22:59:16.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.849+0000 7f1471038640 1 Processor -- start 2024-04-24T22:59:16.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.849+0000 7f1471038640 1 -- start start 2024-04-24T22:59:16.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.850+0000 7f1471038640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f146c0d4210 0x7f146c0d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:16.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.850+0000 7f1471038640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f146c0d3170 con 0x7f146c06b820 2024-04-24T22:59:16.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.850+0000 7f1471038640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f146c0d3450 con 0x7f146c15e9b0 2024-04-24T22:59:16.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.850+0000 7f1471038640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f146c0d3730 con 0x7f146c0d4210 2024-04-24T22:59:16.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.850+0000 7f146a59c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f146c06b820 0x7f146c0b78f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47498/0 (socket says 172.21.15.32:47498) 2024-04-24T22:59:16.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.850+0000 7f146a59c640 1 -- 172.21.15.32:0/369228322 learned_addr learned my addr 172.21.15.32:0/369228322 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:16.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.851+0000 7f146b59e640 1 --2- 172.21.15.32:0/369228322 >> v2:172.21.15.121:3300/0 conn(0x7f146c0d4210 0x7f146c0d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:16.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.851+0000 7f1469d9b640 1 -- 172.21.15.32:0/369228322 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 235801495 0 0) 0x7f146c0d3170 con 0x7f146c06b820 2024-04-24T22:59:16.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.851+0000 7f1469d9b640 1 -- 172.21.15.32:0/369228322 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1458003520 con 0x7f146c06b820 2024-04-24T22:59:16.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.851+0000 7f1469d9b640 1 -- 172.21.15.32:0/369228322 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4156929874 0 0) 0x7f1458003520 con 0x7f146c06b820 2024-04-24T22:59:16.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.851+0000 7f1469d9b640 1 -- 172.21.15.32:0/369228322 >> v2:172.21.15.121:3300/0 conn(0x7f146c0d4210 msgr2=0x7f146c0d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:16.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.851+0000 7f1469d9b640 1 --2- 172.21.15.32:0/369228322 >> v2:172.21.15.121:3300/0 conn(0x7f146c0d4210 0x7f146c0d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:16.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.851+0000 7f1469d9b640 1 -- 172.21.15.32:0/369228322 >> v1:172.21.15.121:6789/0 conn(0x7f146c15e9b0 legacy=0x7f146c06b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:16.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.851+0000 7f1469d9b640 1 -- 172.21.15.32:0/369228322 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f146c0d3a10 con 0x7f146c06b820 2024-04-24T22:59:16.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.851+0000 7f1469d9b640 1 -- 172.21.15.32:0/369228322 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f1454001670 con 0x7f146c06b820 2024-04-24T22:59:16.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.851+0000 7f1469d9b640 1 -- 172.21.15.32:0/369228322 >> v1:172.21.15.32:6789/0 conn(0x7f146c06b820 legacy=0x7f146c0b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:16.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.851+0000 7f1469d9b640 1 --2- 172.21.15.32:0/369228322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14580039b0 0x7f1458003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:16.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.851+0000 7f1469d9b640 1 --2- 172.21.15.32:0/369228322 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1458004800 0x7f1458004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:16.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.851+0000 7f1469d9b640 1 --2- 172.21.15.32:0/369228322 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14580051c0 0x7f145800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:16.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.851+0000 7f1469d9b640 1 -- 172.21.15.32:0/369228322 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f146c0d3730 con 0x7f14580039b0 2024-04-24T22:59:16.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.851+0000 7f1469d9b640 1 -- 172.21.15.32:0/369228322 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f145800c160 con 0x7f1458004800 2024-04-24T22:59:16.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.851+0000 7f1469d9b640 1 -- 172.21.15.32:0/369228322 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f145800c360 con 0x7f14580051c0 2024-04-24T22:59:16.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.851+0000 7f146ad9d640 1 --2- 172.21.15.32:0/369228322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14580039b0 0x7f1458003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:16.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.851+0000 7f146a59c640 1 --2- 172.21.15.32:0/369228322 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1458004800 0x7f1458004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:16.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.851+0000 7f146b59e640 1 --2- 172.21.15.32:0/369228322 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14580051c0 0x7f145800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:16.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.851+0000 7f146ad9d640 1 -- 172.21.15.32:0/369228322 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14580051c0 msgr2=0x7f145800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:16.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.851+0000 7f146ad9d640 1 --2- 172.21.15.32:0/369228322 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14580051c0 0x7f145800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:16.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.851+0000 7f146ad9d640 1 -- 172.21.15.32:0/369228322 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1458004800 msgr2=0x7f1458004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:16.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.851+0000 7f146ad9d640 1 --2- 172.21.15.32:0/369228322 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1458004800 0x7f1458004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:16.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.851+0000 7f146ad9d640 1 -- 172.21.15.32:0/369228322 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1458003520 con 0x7f14580039b0 2024-04-24T22:59:16.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.851+0000 7f146b59e640 1 -- 172.21.15.32:0/369228322 reap_dead start 2024-04-24T22:59:16.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.852+0000 7f146ad9d640 1 --2- 172.21.15.32:0/369228322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14580039b0 0x7f1458003dd0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1464001750 tx=0x7f1458005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3674d4eb86bac9bb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:16.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.852+0000 7f1469d9b640 1 -- 172.21.15.32:0/369228322 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1464001370 con 0x7f14580039b0 2024-04-24T22:59:16.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.852+0000 7f1469d9b640 1 -- 172.21.15.32:0/369228322 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14640052d0 con 0x7f14580039b0 2024-04-24T22:59:16.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.852+0000 7f1469d9b640 1 -- 172.21.15.32:0/369228322 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f146402abe0 con 0x7f14580039b0 2024-04-24T22:59:16.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.853+0000 7f1471038640 1 -- 172.21.15.32:0/369228322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14580039b0 msgr2=0x7f1458003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:16.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.853+0000 7f1471038640 1 --2- 172.21.15.32:0/369228322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14580039b0 0x7f1458003dd0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1464001750 tx=0x7f1458005830 comp rx=0 tx=0).stop 2024-04-24T22:59:16.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.853+0000 7f1471038640 1 -- 172.21.15.32:0/369228322 shutdown_connections 2024-04-24T22:59:16.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.853+0000 7f1471038640 1 --2- 172.21.15.32:0/369228322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14580039b0 0x7f1458003dd0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:16.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.853+0000 7f1471038640 1 -- 172.21.15.32:0/369228322 >> 172.21.15.32:0/369228322 conn(0x7f146c0d11d0 msgr2=0x7f146c0d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:16.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.853+0000 7f1471038640 1 -- 172.21.15.32:0/369228322 shutdown_connections 2024-04-24T22:59:16.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.853+0000 7f1471038640 1 -- 172.21.15.32:0/369228322 wait complete. 2024-04-24T22:59:16.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.854+0000 7f1471038640 1 Processor -- start 2024-04-24T22:59:16.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.854+0000 7f1471038640 1 -- start start 2024-04-24T22:59:16.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.855+0000 7f1471038640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14580039b0 0x7f146c1f8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:16.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.855+0000 7f1471038640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f146c15ed90 0x7f146c159e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:16.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.855+0000 7f1471038640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f146c14b6c0 0x7f146c14bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:16.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.855+0000 7f1471038640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f146c0ba0e0 con 0x7f14580039b0 2024-04-24T22:59:16.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.855+0000 7f1471038640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f146c0d2400 con 0x7f146c14b6c0 2024-04-24T22:59:16.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.855+0000 7f1471038640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f146c0ba260 con 0x7f146c15ed90 2024-04-24T22:59:16.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.855+0000 7f146ad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14580039b0 0x7f146c1f8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:16.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.855+0000 7f146ad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14580039b0 0x7f146c1f8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38814/0 (socket says 172.21.15.32:38814) 2024-04-24T22:59:16.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.855+0000 7f146ad9d640 1 -- 172.21.15.32:0/924094148 learned_addr learned my addr 172.21.15.32:0/924094148 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:16.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.855+0000 7f146a59c640 1 --2- 172.21.15.32:0/924094148 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f146c15ed90 0x7f146c159e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:16.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.855+0000 7f146b59e640 1 --2- 172.21.15.32:0/924094148 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f146c14b6c0 0x7f146c14bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:16.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.855+0000 7f146ad9d640 1 -- 172.21.15.32:0/924094148 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f146c15ed90 msgr2=0x7f146c159e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:16.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.855+0000 7f146ad9d640 1 --2- 172.21.15.32:0/924094148 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f146c15ed90 0x7f146c159e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:16.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.855+0000 7f146ad9d640 1 -- 172.21.15.32:0/924094148 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f146c14b6c0 msgr2=0x7f146c14bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:16.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.855+0000 7f146ad9d640 1 --2- 172.21.15.32:0/924094148 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f146c14b6c0 0x7f146c14bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:16.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.855+0000 7f146ad9d640 1 -- 172.21.15.32:0/924094148 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f146c14c0b0 con 0x7f14580039b0 2024-04-24T22:59:16.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.856+0000 7f146ad9d640 1 --2- 172.21.15.32:0/924094148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14580039b0 0x7f146c1f8430 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1464001870 tx=0x7f1464001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:16.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.856+0000 7f14637fe640 1 -- 172.21.15.32:0/924094148 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1464037040 con 0x7f14580039b0 2024-04-24T22:59:16.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.856+0000 7f14637fe640 1 -- 172.21.15.32:0/924094148 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1464040070 con 0x7f14580039b0 2024-04-24T22:59:16.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.856+0000 7f1471038640 1 -- 172.21.15.32:0/924094148 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f146c158670 con 0x7f14580039b0 2024-04-24T22:59:16.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.856+0000 7f14637fe640 1 -- 172.21.15.32:0/924094148 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1464001370 con 0x7f14580039b0 2024-04-24T22:59:16.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.856+0000 7f1471038640 1 -- 172.21.15.32:0/924094148 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f146c15f530 con 0x7f14580039b0 2024-04-24T22:59:16.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.857+0000 7f1471038640 1 -- 172.21.15.32:0/924094148 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f146c068430 con 0x7f14580039b0 2024-04-24T22:59:16.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.857+0000 7f14637fe640 1 -- 172.21.15.32:0/924094148 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f14640052d0 con 0x7f14580039b0 2024-04-24T22:59:16.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.858+0000 7f14637fe640 1 --2- 172.21.15.32:0/924094148 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f144c06be30 0x7f144c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:16.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.858+0000 7f14637fe640 1 -- 172.21.15.32:0/924094148 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f14640aaad0 con 0x7f14580039b0 2024-04-24T22:59:16.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.858+0000 7f146a59c640 1 --2- 172.21.15.32:0/924094148 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f144c06be30 0x7f144c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:16.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.858+0000 7f146a59c640 1 --2- 172.21.15.32:0/924094148 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f144c06be30 0x7f144c06e2f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f146c158030 tx=0x7f14540078f0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:16.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:16.862+0000 7f14637fe640 1 -- 172.21.15.32:0/924094148 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1464078d00 con 0x7f14580039b0 2024-04-24T22:59:17.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.081+0000 7f1471038640 1 -- 172.21.15.32:0/924094148 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f146c1f8970 con 0x7f14580039b0 2024-04-24T22:59:17.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.082+0000 7f14637fe640 1 -- 172.21.15.32:0/924094148 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f146407d000 con 0x7f14580039b0 2024-04-24T22:59:17.083 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:59:17.083 INFO:teuthology.orchestra.run.smithi032.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-24T22:59:17.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.083+0000 7f1471038640 1 -- 172.21.15.32:0/924094148 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f144c06be30 msgr2=0x7f144c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:17.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.083+0000 7f1471038640 1 --2- 172.21.15.32:0/924094148 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f144c06be30 0x7f144c06e2f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f146c158030 tx=0x7f14540078f0 comp rx=0 tx=0).stop 2024-04-24T22:59:17.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.083+0000 7f1471038640 1 -- 172.21.15.32:0/924094148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14580039b0 msgr2=0x7f146c1f8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:17.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.083+0000 7f1471038640 1 --2- 172.21.15.32:0/924094148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14580039b0 0x7f146c1f8430 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1464001870 tx=0x7f1464001120 comp rx=0 tx=0).stop 2024-04-24T22:59:17.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.083+0000 7f1471038640 1 -- 172.21.15.32:0/924094148 shutdown_connections 2024-04-24T22:59:17.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.083+0000 7f1471038640 1 --2- 172.21.15.32:0/924094148 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f144c06be30 0x7f144c06e2f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:17.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.083+0000 7f1471038640 1 --2- 172.21.15.32:0/924094148 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f146c14b6c0 0x7f146c14bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:17.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.083+0000 7f1471038640 1 --2- 172.21.15.32:0/924094148 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f146c15ed90 0x7f146c159e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:17.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.083+0000 7f1471038640 1 --2- 172.21.15.32:0/924094148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14580039b0 0x7f146c1f8430 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:17.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.084+0000 7f1471038640 1 -- 172.21.15.32:0/924094148 >> 172.21.15.32:0/924094148 conn(0x7f146c0d11d0 msgr2=0x7f146c06b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:17.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.084+0000 7f1471038640 1 -- 172.21.15.32:0/924094148 shutdown_connections 2024-04-24T22:59:17.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.084+0000 7f1471038640 1 -- 172.21.15.32:0/924094148 wait complete. 2024-04-24T22:59:17.096 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-24T22:59:17.096 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:59:17.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.196+0000 7fb9e2932640 1 Processor -- start 2024-04-24T22:59:17.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.196+0000 7fb9e2932640 1 -- start start 2024-04-24T22:59:17.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.196+0000 7fb9e2932640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb9dc15e9b0 0x7fb9dc0d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:17.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.196+0000 7fb9e2932640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb9dc0d4210 0x7fb9dc06b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:17.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.196+0000 7fb9e2932640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9dc0d3170 con 0x7fb9dc0ba0e0 2024-04-24T22:59:17.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.196+0000 7fb9e2932640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fb9dc0b8900 con 0x7fb9dc15e9b0 2024-04-24T22:59:17.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.196+0000 7fb9e2932640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fb9dc0d3450 con 0x7fb9dc0d4210 2024-04-24T22:59:17.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.197+0000 7fb9dbfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb9dc15e9b0 0x7fb9dc0d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:17.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.197+0000 7fb9dbfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb9dc15e9b0 0x7fb9dc0d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38830/0 (socket says 172.21.15.32:38830) 2024-04-24T22:59:17.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.197+0000 7fb9dbfff640 1 -- 172.21.15.32:0/86094723 learned_addr learned my addr 172.21.15.32:0/86094723 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:17.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.197+0000 7fb9db7fe640 1 --2- 172.21.15.32:0/86094723 >> v2:172.21.15.121:3300/0 conn(0x7fb9dc0d4210 0x7fb9dc06b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:17.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.197+0000 7fb9dbfff640 1 -- 172.21.15.32:0/86094723 >> v2:172.21.15.121:3300/0 conn(0x7fb9dc0d4210 msgr2=0x7fb9dc06b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:17.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.197+0000 7fb9dbfff640 1 --2- 172.21.15.32:0/86094723 >> v2:172.21.15.121:3300/0 conn(0x7fb9dc0d4210 0x7fb9dc06b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:17.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.197+0000 7fb9dbfff640 1 -- 172.21.15.32:0/86094723 >> v1:172.21.15.121:6789/0 conn(0x7fb9dc0ba0e0 legacy=0x7fb9dc071730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:17.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.197+0000 7fb9dbfff640 1 -- 172.21.15.32:0/86094723 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9dc0d3730 con 0x7fb9dc15e9b0 2024-04-24T22:59:17.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.198+0000 7fb9dbfff640 1 --2- 172.21.15.32:0/86094723 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9dc15e9b0 0x7fb9dc0d2b90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc0730f0 tx=0x7fb9cc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4f5752629e9b9ea server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:17.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.198+0000 7fb9daffd640 1 -- 172.21.15.32:0/86094723 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9cc037070 con 0x7fb9dc15e9b0 2024-04-24T22:59:17.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.198+0000 7fb9daffd640 1 -- 172.21.15.32:0/86094723 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9cc032440 con 0x7fb9dc15e9b0 2024-04-24T22:59:17.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.198+0000 7fb9daffd640 1 -- 172.21.15.32:0/86094723 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9cc032810 con 0x7fb9dc15e9b0 2024-04-24T22:59:17.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.198+0000 7fb9e2932640 1 -- 172.21.15.32:0/86094723 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9dc15e9b0 msgr2=0x7fb9dc0d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:17.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.198+0000 7fb9e2932640 1 --2- 172.21.15.32:0/86094723 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9dc15e9b0 0x7fb9dc0d2b90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc0730f0 tx=0x7fb9cc02a1e0 comp rx=0 tx=0).stop 2024-04-24T22:59:17.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.198+0000 7fb9e2932640 1 -- 172.21.15.32:0/86094723 shutdown_connections 2024-04-24T22:59:17.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.198+0000 7fb9e2932640 1 --2- 172.21.15.32:0/86094723 >> v2:172.21.15.121:3300/0 conn(0x7fb9dc0d4210 0x7fb9dc06b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:17.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.198+0000 7fb9e2932640 1 --2- 172.21.15.32:0/86094723 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9dc15e9b0 0x7fb9dc0d2b90 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:17.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.198+0000 7fb9e2932640 1 -- 172.21.15.32:0/86094723 >> 172.21.15.32:0/86094723 conn(0x7fb9dc0d11d0 msgr2=0x7fb9dc0d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:17.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.198+0000 7fb9e2932640 1 -- 172.21.15.32:0/86094723 shutdown_connections 2024-04-24T22:59:17.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.198+0000 7fb9e2932640 1 -- 172.21.15.32:0/86094723 wait complete. 2024-04-24T22:59:17.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.200+0000 7fb9e2932640 1 Processor -- start 2024-04-24T22:59:17.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.200+0000 7fb9e2932640 1 -- start start 2024-04-24T22:59:17.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.200+0000 7fb9e2932640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9dc0ba0e0 0x7fb9dc1f74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:17.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.200+0000 7fb9e2932640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9dc0d4210 0x7fb9dc14c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:17.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.201+0000 7fb9e2932640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb9dc15e9b0 0x7fb9dc1564b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:17.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.201+0000 7fb9e2932640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb9dc0d2400 con 0x7fb9dc0ba0e0 2024-04-24T22:59:17.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.201+0000 7fb9e2932640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb9dc0d3cf0 con 0x7fb9dc0d4210 2024-04-24T22:59:17.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.201+0000 7fb9e2932640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fb9dc06b650 con 0x7fb9dc15e9b0 2024-04-24T22:59:17.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.201+0000 7fb9dbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9dc0ba0e0 0x7fb9dc1f74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:17.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.201+0000 7fb9dbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9dc0ba0e0 0x7fb9dc1f74d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38836/0 (socket says 172.21.15.32:38836) 2024-04-24T22:59:17.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.201+0000 7fb9dbfff640 1 -- 172.21.15.32:0/2386411043 learned_addr learned my addr 172.21.15.32:0/2386411043 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:17.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.201+0000 7fb9db7fe640 1 --2- 172.21.15.32:0/2386411043 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9dc0d4210 0x7fb9dc14c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:17.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.201+0000 7fb9e0ecf640 1 --2- 172.21.15.32:0/2386411043 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb9dc15e9b0 0x7fb9dc1564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:17.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.201+0000 7fb9dbfff640 1 -- 172.21.15.32:0/2386411043 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb9dc15e9b0 msgr2=0x7fb9dc1564b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:17.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.201+0000 7fb9dbfff640 1 --2- 172.21.15.32:0/2386411043 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb9dc15e9b0 0x7fb9dc1564b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:17.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.201+0000 7fb9dbfff640 1 -- 172.21.15.32:0/2386411043 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9dc0d4210 msgr2=0x7fb9dc14c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:17.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.201+0000 7fb9dbfff640 1 --2- 172.21.15.32:0/2386411043 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9dc0d4210 0x7fb9dc14c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:17.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.201+0000 7fb9dbfff640 1 -- 172.21.15.32:0/2386411043 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9dc1f83a0 con 0x7fb9dc0ba0e0 2024-04-24T22:59:17.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.201+0000 7fb9db7fe640 1 --2- 172.21.15.32:0/2386411043 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9dc0d4210 0x7fb9dc14c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:17.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.201+0000 7fb9e0ecf640 1 --2- 172.21.15.32:0/2386411043 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb9dc15e9b0 0x7fb9dc1564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:17.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.201+0000 7fb9dbfff640 1 --2- 172.21.15.32:0/2386411043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9dc0ba0e0 0x7fb9dc1f74d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc02a220 tx=0x7fb9cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:17.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.201+0000 7fb9d8ff9640 1 -- 172.21.15.32:0/2386411043 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9cc041020 con 0x7fb9dc0ba0e0 2024-04-24T22:59:17.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.201+0000 7fb9d8ff9640 1 -- 172.21.15.32:0/2386411043 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9cc0306c0 con 0x7fb9dc0ba0e0 2024-04-24T22:59:17.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.201+0000 7fb9e2932640 1 -- 172.21.15.32:0/2386411043 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9dc0cda10 con 0x7fb9dc0ba0e0 2024-04-24T22:59:17.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.201+0000 7fb9d8ff9640 1 -- 172.21.15.32:0/2386411043 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9cc037040 con 0x7fb9dc0ba0e0 2024-04-24T22:59:17.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.201+0000 7fb9e2932640 1 -- 172.21.15.32:0/2386411043 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9dc1f7de0 con 0x7fb9dc0ba0e0 2024-04-24T22:59:17.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.202+0000 7fb9e2932640 1 -- 172.21.15.32:0/2386411043 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9dc068430 con 0x7fb9dc0ba0e0 2024-04-24T22:59:17.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.203+0000 7fb9d8ff9640 1 -- 172.21.15.32:0/2386411043 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb9cc044020 con 0x7fb9dc0ba0e0 2024-04-24T22:59:17.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.203+0000 7fb9d8ff9640 1 --2- 172.21.15.32:0/2386411043 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb9c806be60 0x7fb9c806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:17.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.203+0000 7fb9d8ff9640 1 -- 172.21.15.32:0/2386411043 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb9cc0ab570 con 0x7fb9dc0ba0e0 2024-04-24T22:59:17.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.204+0000 7fb9db7fe640 1 --2- 172.21.15.32:0/2386411043 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb9c806be60 0x7fb9c806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:17.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.204+0000 7fb9db7fe640 1 --2- 172.21.15.32:0/2386411043 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb9c806be60 0x7fb9c806e320 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc0b61a0 tx=0x7fb9d0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:17.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.207+0000 7fb9d8ff9640 1 -- 172.21.15.32:0/2386411043 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb9cc04b030 con 0x7fb9dc0ba0e0 2024-04-24T22:59:17.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.390+0000 7fb9e2932640 1 -- 172.21.15.32:0/2386411043 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb9dc072260 con 0x7fb9dc0ba0e0 2024-04-24T22:59:17.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.391+0000 7fb9d8ff9640 1 -- 172.21.15.32:0/2386411043 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16621 (secure 0 0 0) 0x7fb9cc079fb0 con 0x7fb9dc0ba0e0 2024-04-24T22:59:17.393 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:59:17.394 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T22:59:05.517554+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:59:02.016639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T22:59:02.846087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:59:17.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.393+0000 7fb9e2932640 1 -- 172.21.15.32:0/2386411043 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb9c806be60 msgr2=0x7fb9c806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:17.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.393+0000 7fb9e2932640 1 --2- 172.21.15.32:0/2386411043 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb9c806be60 0x7fb9c806e320 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc0b61a0 tx=0x7fb9d0008040 comp rx=0 tx=0).stop 2024-04-24T22:59:17.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.393+0000 7fb9e2932640 1 -- 172.21.15.32:0/2386411043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9dc0ba0e0 msgr2=0x7fb9dc1f74d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:17.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.394+0000 7fb9e2932640 1 --2- 172.21.15.32:0/2386411043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9dc0ba0e0 0x7fb9dc1f74d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc02a220 tx=0x7fb9cc0046c0 comp rx=0 tx=0).stop 2024-04-24T22:59:17.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.394+0000 7fb9e2932640 1 -- 172.21.15.32:0/2386411043 shutdown_connections 2024-04-24T22:59:17.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.394+0000 7fb9e2932640 1 --2- 172.21.15.32:0/2386411043 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb9c806be60 0x7fb9c806e320 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:17.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.394+0000 7fb9e2932640 1 --2- 172.21.15.32:0/2386411043 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb9dc15e9b0 0x7fb9dc1564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:17.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.394+0000 7fb9e2932640 1 --2- 172.21.15.32:0/2386411043 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9dc0d4210 0x7fb9dc14c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:17.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.394+0000 7fb9e2932640 1 --2- 172.21.15.32:0/2386411043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9dc0ba0e0 0x7fb9dc1f74d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:17.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.394+0000 7fb9e2932640 1 -- 172.21.15.32:0/2386411043 >> 172.21.15.32:0/2386411043 conn(0x7fb9dc0d11d0 msgr2=0x7fb9dc0cf6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:17.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.394+0000 7fb9e2932640 1 -- 172.21.15.32:0/2386411043 shutdown_connections 2024-04-24T22:59:17.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.394+0000 7fb9e2932640 1 -- 172.21.15.32:0/2386411043 wait complete. 2024-04-24T22:59:17.407 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T22:59:02.016639+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T22:59:02.846087+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T22:59:17.408 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T22:59:17.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.510+0000 7f4b36246640 1 Processor -- start 2024-04-24T22:59:17.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.510+0000 7f4b36246640 1 -- start start 2024-04-24T22:59:17.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.511+0000 7f4b36246640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4b300b0b80 0x7f4b3006c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:17.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.511+0000 7f4b36246640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4b300ae140 0x7f4b300b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:17.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.511+0000 7f4b36246640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b300b0770 con 0x7f4b30074210 2024-04-24T22:59:17.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.511+0000 7f4b36246640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f4b300b1c20 con 0x7f4b300ae140 2024-04-24T22:59:17.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.511+0000 7f4b36246640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f4b300b1010 con 0x7f4b300b0b80 2024-04-24T22:59:17.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.511+0000 7f4b2f7fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4b30074210 0x7f4b300745f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47514/0 (socket says 172.21.15.32:47514) 2024-04-24T22:59:17.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.511+0000 7f4b2f7fe640 1 -- 172.21.15.32:0/441929879 learned_addr learned my addr 172.21.15.32:0/441929879 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:17.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.511+0000 7f4b2ffff640 1 --2- 172.21.15.32:0/441929879 >> v2:172.21.15.121:3300/0 conn(0x7f4b300ae140 0x7f4b300b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:17.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.511+0000 7f4b2effd640 1 --2- 172.21.15.32:0/441929879 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4b300b0b80 0x7f4b3006c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:17.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.511+0000 7f4b2e7fc640 1 -- 172.21.15.32:0/441929879 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2176000774 0 0) 0x7f4b300b0770 con 0x7f4b30074210 2024-04-24T22:59:17.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.511+0000 7f4b2e7fc640 1 -- 172.21.15.32:0/441929879 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b18003520 con 0x7f4b30074210 2024-04-24T22:59:17.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.511+0000 7f4b2e7fc640 1 -- 172.21.15.32:0/441929879 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2728489274 0 0) 0x7f4b18003520 con 0x7f4b30074210 2024-04-24T22:59:17.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.511+0000 7f4b2e7fc640 1 -- 172.21.15.32:0/441929879 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4b300b0b80 msgr2=0x7f4b3006c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:17.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.511+0000 7f4b2e7fc640 1 --2- 172.21.15.32:0/441929879 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4b300b0b80 0x7f4b3006c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:17.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.511+0000 7f4b2e7fc640 1 -- 172.21.15.32:0/441929879 >> v2:172.21.15.121:3300/0 conn(0x7f4b300ae140 msgr2=0x7f4b300b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:17.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.511+0000 7f4b2e7fc640 1 --2- 172.21.15.32:0/441929879 >> v2:172.21.15.121:3300/0 conn(0x7f4b300ae140 0x7f4b300b20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:17.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.511+0000 7f4b2e7fc640 1 -- 172.21.15.32:0/441929879 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b300b12f0 con 0x7f4b30074210 2024-04-24T22:59:17.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.511+0000 7f4b2e7fc640 1 -- 172.21.15.32:0/441929879 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f4b20001670 con 0x7f4b30074210 2024-04-24T22:59:17.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.511+0000 7f4b2e7fc640 1 -- 172.21.15.32:0/441929879 >> v1:172.21.15.32:6789/0 conn(0x7f4b30074210 legacy=0x7f4b300745f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:17.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.511+0000 7f4b2e7fc640 1 --2- 172.21.15.32:0/441929879 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b180039b0 0x7f4b18003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:17.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.511+0000 7f4b2e7fc640 1 --2- 172.21.15.32:0/441929879 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b18004800 0x7f4b18004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:17.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.512+0000 7f4b2e7fc640 1 --2- 172.21.15.32:0/441929879 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4b180051c0 0x7f4b1800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:17.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.512+0000 7f4b2e7fc640 1 -- 172.21.15.32:0/441929879 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4b300b1c20 con 0x7f4b180039b0 2024-04-24T22:59:17.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.512+0000 7f4b2e7fc640 1 -- 172.21.15.32:0/441929879 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4b300b1010 con 0x7f4b18004800 2024-04-24T22:59:17.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.512+0000 7f4b2e7fc640 1 -- 172.21.15.32:0/441929879 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f4b180045a0 con 0x7f4b180051c0 2024-04-24T22:59:17.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.512+0000 7f4b2f7fe640 1 --2- 172.21.15.32:0/441929879 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b180039b0 0x7f4b18003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:17.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.512+0000 7f4b2ffff640 1 --2- 172.21.15.32:0/441929879 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4b180051c0 0x7f4b1800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:17.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.512+0000 7f4b2effd640 1 --2- 172.21.15.32:0/441929879 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b18004800 0x7f4b18004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:17.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.512+0000 7f4b2f7fe640 1 -- 172.21.15.32:0/441929879 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4b180051c0 msgr2=0x7f4b1800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:17.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.512+0000 7f4b2f7fe640 1 --2- 172.21.15.32:0/441929879 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4b180051c0 0x7f4b1800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:17.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.512+0000 7f4b2f7fe640 1 -- 172.21.15.32:0/441929879 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b18004800 msgr2=0x7f4b18004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:17.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.512+0000 7f4b2f7fe640 1 --2- 172.21.15.32:0/441929879 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b18004800 0x7f4b18004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:17.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.512+0000 7f4b2f7fe640 1 -- 172.21.15.32:0/441929879 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b18003520 con 0x7f4b180039b0 2024-04-24T22:59:17.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.513+0000 7f4b2f7fe640 1 --2- 172.21.15.32:0/441929879 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b180039b0 0x7f4b18003dd0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4b2000c2c0 tx=0x7f4b18005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72d5b69c8e3a9cc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:17.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.513+0000 7f4b2e7fc640 1 -- 172.21.15.32:0/441929879 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b20007d50 con 0x7f4b180039b0 2024-04-24T22:59:17.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.513+0000 7f4b2e7fc640 1 -- 172.21.15.32:0/441929879 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b2002e070 con 0x7f4b180039b0 2024-04-24T22:59:17.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.513+0000 7f4b2e7fc640 1 -- 172.21.15.32:0/441929879 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b20005d30 con 0x7f4b180039b0 2024-04-24T22:59:17.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.513+0000 7f4b36246640 1 -- 172.21.15.32:0/441929879 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b180039b0 msgr2=0x7f4b18003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:17.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.513+0000 7f4b36246640 1 --2- 172.21.15.32:0/441929879 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b180039b0 0x7f4b18003dd0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4b2000c2c0 tx=0x7f4b18005830 comp rx=0 tx=0).stop 2024-04-24T22:59:17.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.513+0000 7f4b2ffff640 1 -- 172.21.15.32:0/441929879 reap_dead start 2024-04-24T22:59:17.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.513+0000 7f4b36246640 1 -- 172.21.15.32:0/441929879 shutdown_connections 2024-04-24T22:59:17.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.513+0000 7f4b36246640 1 -- 172.21.15.32:0/441929879 >> 172.21.15.32:0/441929879 conn(0x7f4b300b2e60 msgr2=0x7f4b300b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:17.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.513+0000 7f4b36246640 1 -- 172.21.15.32:0/441929879 shutdown_connections 2024-04-24T22:59:17.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.513+0000 7f4b36246640 1 -- 172.21.15.32:0/441929879 wait complete. 2024-04-24T22:59:17.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.515+0000 7f4b36246640 1 Processor -- start 2024-04-24T22:59:17.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.515+0000 7f4b36246640 1 -- start start 2024-04-24T22:59:17.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.515+0000 7f4b36246640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b180039b0 0x7f4b3009a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:17.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.515+0000 7f4b36246640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b300af6d0 0x7f4b300afb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:17.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.515+0000 7f4b36246640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4b3009bc60 0x7f4b3009c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:17.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.515+0000 7f4b36246640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4b300b3ba0 con 0x7f4b180039b0 2024-04-24T22:59:17.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.515+0000 7f4b36246640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4b300b18b0 con 0x7f4b300af6d0 2024-04-24T22:59:17.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.515+0000 7f4b36246640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f4b300b3d20 con 0x7f4b3009bc60 2024-04-24T22:59:17.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.516+0000 7f4b2f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b180039b0 0x7f4b3009a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:17.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.516+0000 7f4b2f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b180039b0 0x7f4b3009a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38854/0 (socket says 172.21.15.32:38854) 2024-04-24T22:59:17.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.516+0000 7f4b2f7fe640 1 -- 172.21.15.32:0/3932859549 learned_addr learned my addr 172.21.15.32:0/3932859549 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:17.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.516+0000 7f4b2effd640 1 --2- 172.21.15.32:0/3932859549 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b300af6d0 0x7f4b300afb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:17.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.516+0000 7f4b2ffff640 1 --2- 172.21.15.32:0/3932859549 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4b3009bc60 0x7f4b3009c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:17.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.516+0000 7f4b2f7fe640 1 -- 172.21.15.32:0/3932859549 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4b3009bc60 msgr2=0x7f4b3009c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:17.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.516+0000 7f4b2f7fe640 1 --2- 172.21.15.32:0/3932859549 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4b3009bc60 0x7f4b3009c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:17.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.516+0000 7f4b2f7fe640 1 -- 172.21.15.32:0/3932859549 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b300af6d0 msgr2=0x7f4b300afb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:17.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.516+0000 7f4b2f7fe640 1 --2- 172.21.15.32:0/3932859549 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b300af6d0 0x7f4b300afb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:17.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.516+0000 7f4b2f7fe640 1 -- 172.21.15.32:0/3932859549 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b3009e820 con 0x7f4b180039b0 2024-04-24T22:59:17.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.516+0000 7f4b2f7fe640 1 --2- 172.21.15.32:0/3932859549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b180039b0 0x7f4b3009a860 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4b20000c00 tx=0x7f4b300b1b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:17.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.516+0000 7f4b17fff640 1 -- 172.21.15.32:0/3932859549 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b20041070 con 0x7f4b180039b0 2024-04-24T22:59:17.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.516+0000 7f4b17fff640 1 -- 172.21.15.32:0/3932859549 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b20007280 con 0x7f4b180039b0 2024-04-24T22:59:17.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.516+0000 7f4b36246640 1 -- 172.21.15.32:0/3932859549 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b3009eac0 con 0x7f4b180039b0 2024-04-24T22:59:17.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.516+0000 7f4b17fff640 1 -- 172.21.15.32:0/3932859549 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b20007d50 con 0x7f4b180039b0 2024-04-24T22:59:17.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.516+0000 7f4b36246640 1 -- 172.21.15.32:0/3932859549 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b3009f080 con 0x7f4b180039b0 2024-04-24T22:59:17.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.517+0000 7f4b36246640 1 -- 172.21.15.32:0/3932859549 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b300b0070 con 0x7f4b180039b0 2024-04-24T22:59:17.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.518+0000 7f4b17fff640 1 -- 172.21.15.32:0/3932859549 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4b20044070 con 0x7f4b180039b0 2024-04-24T22:59:17.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.518+0000 7f4b17fff640 1 --2- 172.21.15.32:0/3932859549 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4b1c06be60 0x7f4b1c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:17.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.518+0000 7f4b17fff640 1 -- 172.21.15.32:0/3932859549 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f4b200aadd0 con 0x7f4b180039b0 2024-04-24T22:59:17.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.518+0000 7f4b2effd640 1 --2- 172.21.15.32:0/3932859549 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4b1c06be60 0x7f4b1c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:17.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.519+0000 7f4b2effd640 1 --2- 172.21.15.32:0/3932859549 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4b1c06be60 0x7f4b1c06e320 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4b300d0f00 tx=0x7f4b24008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:17.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.523+0000 7f4b17fff640 1 -- 172.21.15.32:0/3932859549 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4b2004c020 con 0x7f4b180039b0 2024-04-24T22:59:17.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.704+0000 7f4b36246640 1 -- 172.21.15.32:0/3932859549 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4b3006b220 con 0x7f4b180039b0 2024-04-24T22:59:17.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.705+0000 7f4b17fff640 1 -- 172.21.15.32:0/3932859549 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f4b20079780 con 0x7f4b180039b0 2024-04-24T22:59:17.706 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T22:59:17.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.706+0000 7f4b36246640 1 -- 172.21.15.32:0/3932859549 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4b1c06be60 msgr2=0x7f4b1c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:17.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.706+0000 7f4b36246640 1 --2- 172.21.15.32:0/3932859549 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4b1c06be60 0x7f4b1c06e320 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4b300d0f00 tx=0x7f4b24008040 comp rx=0 tx=0).stop 2024-04-24T22:59:17.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.706+0000 7f4b36246640 1 -- 172.21.15.32:0/3932859549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b180039b0 msgr2=0x7f4b3009a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:17.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.706+0000 7f4b36246640 1 --2- 172.21.15.32:0/3932859549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b180039b0 0x7f4b3009a860 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4b20000c00 tx=0x7f4b300b1b90 comp rx=0 tx=0).stop 2024-04-24T22:59:17.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.706+0000 7f4b36246640 1 -- 172.21.15.32:0/3932859549 shutdown_connections 2024-04-24T22:59:17.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.706+0000 7f4b36246640 1 --2- 172.21.15.32:0/3932859549 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4b1c06be60 0x7f4b1c06e320 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:17.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.706+0000 7f4b36246640 1 --2- 172.21.15.32:0/3932859549 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4b3009bc60 0x7f4b3009c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:17.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.706+0000 7f4b36246640 1 --2- 172.21.15.32:0/3932859549 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b300af6d0 0x7f4b300afb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:17.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.707+0000 7f4b36246640 1 --2- 172.21.15.32:0/3932859549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b180039b0 0x7f4b3009a860 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:17.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.707+0000 7f4b36246640 1 -- 172.21.15.32:0/3932859549 >> 172.21.15.32:0/3932859549 conn(0x7f4b300b2e60 msgr2=0x7f4b300745f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:17.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.707+0000 7f4b36246640 1 -- 172.21.15.32:0/3932859549 shutdown_connections 2024-04-24T22:59:17.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.707+0000 7f4b36246640 1 -- 172.21.15.32:0/3932859549 wait complete. 2024-04-24T22:59:17.719 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T22:59:17.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.821+0000 7fe3d84e7640 1 Processor -- start 2024-04-24T22:59:17.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.821+0000 7fe3d84e7640 1 -- start start 2024-04-24T22:59:17.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.822+0000 7fe3d84e7640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe3d015e9b0 0x7fe3d00d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:17.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.822+0000 7fe3d84e7640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3d006b820 0x7fe3d00b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:17.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.822+0000 7fe3d84e7640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3d00d3170 con 0x7fe3d00d4210 2024-04-24T22:59:17.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.822+0000 7fe3d84e7640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fe3d00d3450 con 0x7fe3d015e9b0 2024-04-24T22:59:17.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.822+0000 7fe3d84e7640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe3d00d3730 con 0x7fe3d006b820 2024-04-24T22:59:17.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.822+0000 7fe3d6283640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe3d015e9b0 0x7fe3d00d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:17.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.822+0000 7fe3d6283640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe3d015e9b0 0x7fe3d00d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38868/0 (socket says 172.21.15.32:38868) 2024-04-24T22:59:17.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.822+0000 7fe3d6283640 1 -- 172.21.15.32:0/2152917835 learned_addr learned my addr 172.21.15.32:0/2152917835 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:17.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.823+0000 7fe3d6a84640 1 --2- 172.21.15.32:0/2152917835 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3d006b820 0x7fe3d00b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:17.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.823+0000 7fe3d6283640 1 -- 172.21.15.32:0/2152917835 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3d006b820 msgr2=0x7fe3d00b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:17.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.823+0000 7fe3d6283640 1 --2- 172.21.15.32:0/2152917835 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3d006b820 0x7fe3d00b78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:17.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.823+0000 7fe3d6283640 1 -- 172.21.15.32:0/2152917835 >> v1:172.21.15.121:6789/0 conn(0x7fe3d00d4210 legacy=0x7fe3d006b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:17.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.823+0000 7fe3d6283640 1 -- 172.21.15.32:0/2152917835 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3d00d3a10 con 0x7fe3d015e9b0 2024-04-24T22:59:17.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.823+0000 7fe3d6283640 1 --2- 172.21.15.32:0/2152917835 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3d015e9b0 0x7fe3d00d2b90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc009870 tx=0x7fe3cc02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0099048279c44c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:17.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.824+0000 7fe3d5281640 1 -- 172.21.15.32:0/2152917835 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc037070 con 0x7fe3d015e9b0 2024-04-24T22:59:17.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.824+0000 7fe3d5281640 1 -- 172.21.15.32:0/2152917835 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3cc032440 con 0x7fe3d015e9b0 2024-04-24T22:59:17.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.824+0000 7fe3d5281640 1 -- 172.21.15.32:0/2152917835 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc032810 con 0x7fe3d015e9b0 2024-04-24T22:59:17.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.824+0000 7fe3d84e7640 1 -- 172.21.15.32:0/2152917835 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3d015e9b0 msgr2=0x7fe3d00d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:17.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.824+0000 7fe3d84e7640 1 --2- 172.21.15.32:0/2152917835 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3d015e9b0 0x7fe3d00d2b90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc009870 tx=0x7fe3cc02a210 comp rx=0 tx=0).stop 2024-04-24T22:59:17.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.824+0000 7fe3d84e7640 1 -- 172.21.15.32:0/2152917835 shutdown_connections 2024-04-24T22:59:17.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.824+0000 7fe3d84e7640 1 --2- 172.21.15.32:0/2152917835 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3d006b820 0x7fe3d00b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:17.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.824+0000 7fe3d84e7640 1 --2- 172.21.15.32:0/2152917835 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3d015e9b0 0x7fe3d00d2b90 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:17.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.824+0000 7fe3d84e7640 1 -- 172.21.15.32:0/2152917835 >> 172.21.15.32:0/2152917835 conn(0x7fe3d00d11d0 msgr2=0x7fe3d00d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:17.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.824+0000 7fe3d84e7640 1 -- 172.21.15.32:0/2152917835 shutdown_connections 2024-04-24T22:59:17.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.824+0000 7fe3d84e7640 1 -- 172.21.15.32:0/2152917835 wait complete. 2024-04-24T22:59:17.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.825+0000 7fe3d84e7640 1 Processor -- start 2024-04-24T22:59:17.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.825+0000 7fe3d84e7640 1 -- start start 2024-04-24T22:59:17.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.826+0000 7fe3d84e7640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3d006b820 0x7fe3d01f59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:17.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.826+0000 7fe3d84e7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3d00d4210 0x7fe3d00ce160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:17.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.826+0000 7fe3d84e7640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3d015e9b0 0x7fe3d00ce6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:17.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.826+0000 7fe3d84e7640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d00ba0e0 con 0x7fe3d00d4210 2024-04-24T22:59:17.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.826+0000 7fe3d84e7640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d00d2400 con 0x7fe3d006b820 2024-04-24T22:59:17.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.826+0000 7fe3d84e7640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe3d00ba260 con 0x7fe3d015e9b0 2024-04-24T22:59:17.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.826+0000 7fe3d5a82640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3d00d4210 0x7fe3d00ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:17.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.826+0000 7fe3d5a82640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3d00d4210 0x7fe3d00ce160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38876/0 (socket says 172.21.15.32:38876) 2024-04-24T22:59:17.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.826+0000 7fe3d5a82640 1 -- 172.21.15.32:0/1805983179 learned_addr learned my addr 172.21.15.32:0/1805983179 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:17.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.827+0000 7fe3d6a84640 1 --2- 172.21.15.32:0/1805983179 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3d015e9b0 0x7fe3d00ce6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:17.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.827+0000 7fe3d6283640 1 --2- 172.21.15.32:0/1805983179 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3d006b820 0x7fe3d01f59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:17.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.827+0000 7fe3d5a82640 1 -- 172.21.15.32:0/1805983179 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3d015e9b0 msgr2=0x7fe3d00ce6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:17.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.827+0000 7fe3d5a82640 1 --2- 172.21.15.32:0/1805983179 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3d015e9b0 0x7fe3d00ce6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:17.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.827+0000 7fe3d5a82640 1 -- 172.21.15.32:0/1805983179 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3d006b820 msgr2=0x7fe3d01f59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:17.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.827+0000 7fe3d5a82640 1 --2- 172.21.15.32:0/1805983179 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3d006b820 0x7fe3d01f59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:17.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.827+0000 7fe3d5a82640 1 -- 172.21.15.32:0/1805983179 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3d00cb940 con 0x7fe3d00d4210 2024-04-24T22:59:17.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.827+0000 7fe3d6283640 1 --2- 172.21.15.32:0/1805983179 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3d006b820 0x7fe3d01f59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:59:17.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.827+0000 7fe3d5a82640 1 --2- 172.21.15.32:0/1805983179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3d00d4210 0x7fe3d00ce160 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe3c000c430 tx=0x7fe3c000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:17.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.827+0000 7fe3c6ffd640 1 -- 172.21.15.32:0/1805983179 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c0011790 con 0x7fe3d00d4210 2024-04-24T22:59:17.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.827+0000 7fe3c6ffd640 1 -- 172.21.15.32:0/1805983179 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3c0017070 con 0x7fe3d00d4210 2024-04-24T22:59:17.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.827+0000 7fe3c6ffd640 1 -- 172.21.15.32:0/1805983179 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c0011bb0 con 0x7fe3d00d4210 2024-04-24T22:59:17.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.827+0000 7fe3d84e7640 1 -- 172.21.15.32:0/1805983179 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3d00cbc20 con 0x7fe3d00d4210 2024-04-24T22:59:17.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.827+0000 7fe3d84e7640 1 -- 172.21.15.32:0/1805983179 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3d00cc1e0 con 0x7fe3d00d4210 2024-04-24T22:59:17.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.828+0000 7fe3d84e7640 1 -- 172.21.15.32:0/1805983179 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3d0068430 con 0x7fe3d00d4210 2024-04-24T22:59:17.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.828+0000 7fe3c6ffd640 1 -- 172.21.15.32:0/1805983179 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe3c001c460 con 0x7fe3d00d4210 2024-04-24T22:59:17.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.829+0000 7fe3c6ffd640 1 --2- 172.21.15.32:0/1805983179 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe3b406be30 0x7fe3b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:17.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.829+0000 7fe3c6ffd640 1 -- 172.21.15.32:0/1805983179 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe3c008acb0 con 0x7fe3d00d4210 2024-04-24T22:59:17.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.829+0000 7fe3d6283640 1 --2- 172.21.15.32:0/1805983179 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe3b406be30 0x7fe3b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:17.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.829+0000 7fe3d6283640 1 --2- 172.21.15.32:0/1805983179 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe3b406be30 0x7fe3b406e2f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc034000 tx=0x7fe3cc02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:17.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:17.833+0000 7fe3c6ffd640 1 -- 172.21.15.32:0/1805983179 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe3c00596f0 con 0x7fe3d00d4210 2024-04-24T22:59:18.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.018+0000 7fe3d84e7640 1 -- 172.21.15.32:0/1805983179 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe3d00b8e40 con 0x7fe3d00d4210 2024-04-24T22:59:18.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.018+0000 7fe3c6ffd640 1 -- 172.21.15.32:0/1805983179 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7fe3c0058ff0 con 0x7fe3d00d4210 2024-04-24T22:59:18.020 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-24T22:59:18.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.020+0000 7fe3d84e7640 1 -- 172.21.15.32:0/1805983179 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe3b406be30 msgr2=0x7fe3b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:18.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.020+0000 7fe3d84e7640 1 --2- 172.21.15.32:0/1805983179 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe3b406be30 0x7fe3b406e2f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc034000 tx=0x7fe3cc02e000 comp rx=0 tx=0).stop 2024-04-24T22:59:18.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.020+0000 7fe3d84e7640 1 -- 172.21.15.32:0/1805983179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3d00d4210 msgr2=0x7fe3d00ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:18.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.020+0000 7fe3d84e7640 1 --2- 172.21.15.32:0/1805983179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3d00d4210 0x7fe3d00ce160 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe3c000c430 tx=0x7fe3c000c8f0 comp rx=0 tx=0).stop 2024-04-24T22:59:18.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.020+0000 7fe3d84e7640 1 -- 172.21.15.32:0/1805983179 shutdown_connections 2024-04-24T22:59:18.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.020+0000 7fe3d84e7640 1 --2- 172.21.15.32:0/1805983179 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe3b406be30 0x7fe3b406e2f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:18.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.020+0000 7fe3d84e7640 1 --2- 172.21.15.32:0/1805983179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3d00d4210 0x7fe3d00ce160 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:18.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.020+0000 7fe3d84e7640 1 --2- 172.21.15.32:0/1805983179 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3d015e9b0 0x7fe3d00ce6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:18.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.020+0000 7fe3d84e7640 1 --2- 172.21.15.32:0/1805983179 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3d006b820 0x7fe3d01f59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:18.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.020+0000 7fe3d84e7640 1 -- 172.21.15.32:0/1805983179 >> 172.21.15.32:0/1805983179 conn(0x7fe3d00d11d0 msgr2=0x7fe3d00b9360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:18.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.020+0000 7fe3d84e7640 1 -- 172.21.15.32:0/1805983179 shutdown_connections 2024-04-24T22:59:18.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.020+0000 7fe3d84e7640 1 -- 172.21.15.32:0/1805983179 wait complete. 2024-04-24T22:59:18.033 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T22:59:18.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.134+0000 7f032df29640 1 Processor -- start 2024-04-24T22:59:18.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.135+0000 7f032df29640 1 -- start start 2024-04-24T22:59:18.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.135+0000 7f032df29640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f032815e9b0 0x7f03280d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:18.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.135+0000 7f032df29640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f03280d4210 0x7f032806b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:18.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.135+0000 7f032df29640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03280d3170 con 0x7f03280ba0e0 2024-04-24T22:59:18.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.135+0000 7f032df29640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f03280b8900 con 0x7f032815e9b0 2024-04-24T22:59:18.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.135+0000 7f032df29640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f03280d3450 con 0x7f03280d4210 2024-04-24T22:59:18.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.135+0000 7f03277fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f032815e9b0 0x7f03280d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:18.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.135+0000 7f0327fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f03280ba0e0 0x7f0328071730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47520/0 (socket says 172.21.15.32:47520) 2024-04-24T22:59:18.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.135+0000 7f0327fff640 1 -- 172.21.15.32:0/68477311 learned_addr learned my addr 172.21.15.32:0/68477311 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:18.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.136+0000 7f0326ffd640 1 --2- 172.21.15.32:0/68477311 >> v2:172.21.15.121:3300/0 conn(0x7f03280d4210 0x7f032806b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:18.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.136+0000 7f03277fe640 1 -- 172.21.15.32:0/68477311 >> v2:172.21.15.121:3300/0 conn(0x7f03280d4210 msgr2=0x7f032806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:18.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.136+0000 7f03277fe640 1 --2- 172.21.15.32:0/68477311 >> v2:172.21.15.121:3300/0 conn(0x7f03280d4210 0x7f032806b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:18.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.136+0000 7f03277fe640 1 -- 172.21.15.32:0/68477311 >> v1:172.21.15.32:6789/0 conn(0x7f03280ba0e0 legacy=0x7f0328071730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:18.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.136+0000 7f03277fe640 1 -- 172.21.15.32:0/68477311 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03280d3730 con 0x7f032815e9b0 2024-04-24T22:59:18.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.136+0000 7f03267fc640 1 -- 172.21.15.32:0/68477311 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 108034217 0 0) 0x7f03280d3170 con 0x7f03280ba0e0 2024-04-24T22:59:18.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.136+0000 7f03277fe640 1 --2- 172.21.15.32:0/68477311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f032815e9b0 0x7f03280d2b90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f03280730f0 tx=0x7f031802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20c893c81d9e4be server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:18.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.137+0000 7f03267fc640 1 -- 172.21.15.32:0/68477311 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0318037070 con 0x7f032815e9b0 2024-04-24T22:59:18.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.137+0000 7f03267fc640 1 -- 172.21.15.32:0/68477311 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0318032440 con 0x7f032815e9b0 2024-04-24T22:59:18.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.137+0000 7f03267fc640 1 -- 172.21.15.32:0/68477311 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031802ae60 con 0x7f032815e9b0 2024-04-24T22:59:18.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.137+0000 7f032df29640 1 -- 172.21.15.32:0/68477311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f032815e9b0 msgr2=0x7f03280d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:18.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.137+0000 7f032df29640 1 --2- 172.21.15.32:0/68477311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f032815e9b0 0x7f03280d2b90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f03280730f0 tx=0x7f031802a110 comp rx=0 tx=0).stop 2024-04-24T22:59:18.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.137+0000 7f032df29640 1 -- 172.21.15.32:0/68477311 shutdown_connections 2024-04-24T22:59:18.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.137+0000 7f032df29640 1 --2- 172.21.15.32:0/68477311 >> v2:172.21.15.121:3300/0 conn(0x7f03280d4210 0x7f032806b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:18.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.137+0000 7f032df29640 1 --2- 172.21.15.32:0/68477311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f032815e9b0 0x7f03280d2b90 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:18.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.137+0000 7f032df29640 1 -- 172.21.15.32:0/68477311 >> 172.21.15.32:0/68477311 conn(0x7f03280d11d0 msgr2=0x7f03280d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:18.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.137+0000 7f032df29640 1 -- 172.21.15.32:0/68477311 shutdown_connections 2024-04-24T22:59:18.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.137+0000 7f032df29640 1 -- 172.21.15.32:0/68477311 wait complete. 2024-04-24T22:59:18.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.138+0000 7f032df29640 1 Processor -- start 2024-04-24T22:59:18.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.139+0000 7f032df29640 1 -- start start 2024-04-24T22:59:18.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.139+0000 7f032df29640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f03280ba0e0 0x7f0328145b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:18.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.139+0000 7f032df29640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03280d4210 0x7f03280cc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:18.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.139+0000 7f032df29640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f032815e9b0 0x7f03281564b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:18.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.139+0000 7f032df29640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f03280d2400 con 0x7f03280d4210 2024-04-24T22:59:18.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.139+0000 7f032df29640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f03280d3cf0 con 0x7f032815e9b0 2024-04-24T22:59:18.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.139+0000 7f032df29640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f032806b650 con 0x7f03280ba0e0 2024-04-24T22:59:18.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.139+0000 7f0326ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03280d4210 0x7f03280cc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:18.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.139+0000 7f0326ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03280d4210 0x7f03280cc0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38902/0 (socket says 172.21.15.32:38902) 2024-04-24T22:59:18.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.139+0000 7f0326ffd640 1 -- 172.21.15.32:0/1853660396 learned_addr learned my addr 172.21.15.32:0/1853660396 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:18.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.140+0000 7f0326ffd640 1 -- 172.21.15.32:0/1853660396 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f03280ba0e0 msgr2=0x7f0328145b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:18.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.140+0000 7f0326ffd640 1 --2- 172.21.15.32:0/1853660396 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f03280ba0e0 0x7f0328145b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:18.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.140+0000 7f0326ffd640 1 -- 172.21.15.32:0/1853660396 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f032815e9b0 msgr2=0x7f03281564b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:18.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.140+0000 7f0327fff640 1 --2- 172.21.15.32:0/1853660396 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f032815e9b0 0x7f03281564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:18.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.140+0000 7f0326ffd640 1 --2- 172.21.15.32:0/1853660396 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f032815e9b0 0x7f03281564b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:18.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.140+0000 7f0326ffd640 1 -- 172.21.15.32:0/1853660396 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03281f4320 con 0x7f03280d4210 2024-04-24T22:59:18.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.140+0000 7f0326ffd640 1 --2- 172.21.15.32:0/1853660396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03280d4210 0x7f03280cc0f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f031c00d830 tx=0x7f031c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:18.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.140+0000 7f030ffff640 1 -- 172.21.15.32:0/1853660396 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031c00b6c0 con 0x7f03280d4210 2024-04-24T22:59:18.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.140+0000 7f030ffff640 1 -- 172.21.15.32:0/1853660396 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f031c019070 con 0x7f03280d4210 2024-04-24T22:59:18.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.140+0000 7f032df29640 1 -- 172.21.15.32:0/1853660396 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0328160c80 con 0x7f03280d4210 2024-04-24T22:59:18.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.140+0000 7f030ffff640 1 -- 172.21.15.32:0/1853660396 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031c00ba80 con 0x7f03280d4210 2024-04-24T22:59:18.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.140+0000 7f032df29640 1 -- 172.21.15.32:0/1853660396 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0328161240 con 0x7f03280d4210 2024-04-24T22:59:18.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.141+0000 7f032df29640 1 -- 172.21.15.32:0/1853660396 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0328146330 con 0x7f03280d4210 2024-04-24T22:59:18.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.142+0000 7f030ffff640 1 -- 172.21.15.32:0/1853660396 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f031c01e460 con 0x7f03280d4210 2024-04-24T22:59:18.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.142+0000 7f030ffff640 1 --2- 172.21.15.32:0/1853660396 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f031406be30 0x7f031406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:18.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.142+0000 7f030ffff640 1 -- 172.21.15.32:0/1853660396 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f031c08b590 con 0x7f03280d4210 2024-04-24T22:59:18.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.142+0000 7f03277fe640 1 --2- 172.21.15.32:0/1853660396 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f031406be30 0x7f031406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:18.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.143+0000 7f03277fe640 1 --2- 172.21.15.32:0/1853660396 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f031406be30 0x7f031406e2f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0318034000 tx=0x7f031802c000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:18.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.146+0000 7f030ffff640 1 -- 172.21.15.32:0/1853660396 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f031c059f40 con 0x7f03280d4210 2024-04-24T22:59:18.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.327+0000 7f032df29640 1 -- 172.21.15.32:0/1853660396 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0328068430 con 0x7f03280d4210 2024-04-24T22:59:18.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.328+0000 7f030ffff640 1 -- 172.21.15.32:0/1853660396 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f031c059840 con 0x7f03280d4210 2024-04-24T22:59:18.330 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-24T22:59:18.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.330+0000 7f032df29640 1 -- 172.21.15.32:0/1853660396 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f031406be30 msgr2=0x7f031406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:18.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.330+0000 7f032df29640 1 --2- 172.21.15.32:0/1853660396 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f031406be30 0x7f031406e2f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0318034000 tx=0x7f031802c000 comp rx=0 tx=0).stop 2024-04-24T22:59:18.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.330+0000 7f032df29640 1 -- 172.21.15.32:0/1853660396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03280d4210 msgr2=0x7f03280cc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:18.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.330+0000 7f032df29640 1 --2- 172.21.15.32:0/1853660396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03280d4210 0x7f03280cc0f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f031c00d830 tx=0x7f031c00dcf0 comp rx=0 tx=0).stop 2024-04-24T22:59:18.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.330+0000 7f032df29640 1 -- 172.21.15.32:0/1853660396 shutdown_connections 2024-04-24T22:59:18.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.330+0000 7f032df29640 1 --2- 172.21.15.32:0/1853660396 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f031406be30 0x7f031406e2f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:18.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.330+0000 7f032df29640 1 --2- 172.21.15.32:0/1853660396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03280d4210 0x7f03280cc0f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:18.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.330+0000 7f032df29640 1 --2- 172.21.15.32:0/1853660396 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f032815e9b0 0x7f03281564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:18.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.330+0000 7f032df29640 1 --2- 172.21.15.32:0/1853660396 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f03280ba0e0 0x7f0328145b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:18.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.330+0000 7f032df29640 1 -- 172.21.15.32:0/1853660396 >> 172.21.15.32:0/1853660396 conn(0x7f03280d11d0 msgr2=0x7f032815c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:18.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.330+0000 7f032df29640 1 -- 172.21.15.32:0/1853660396 shutdown_connections 2024-04-24T22:59:18.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.330+0000 7f032df29640 1 -- 172.21.15.32:0/1853660396 wait complete. 2024-04-24T22:59:18.342 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T22:59:18.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.441+0000 7f8fd07ce640 1 Processor -- start 2024-04-24T22:59:18.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.441+0000 7f8fd07ce640 1 -- start start 2024-04-24T22:59:18.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.442+0000 7f8fd07ce640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8fc806ba30 0x7f8fc806d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:18.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.442+0000 7f8fd07ce640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8fc806da90 0x7f8fc80b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:18.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.442+0000 7f8fd07ce640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fc815ef10 con 0x7f8fc8158dd0 2024-04-24T22:59:18.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.442+0000 7f8fd07ce640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f8fc815f1f0 con 0x7f8fc806ba30 2024-04-24T22:59:18.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.442+0000 7f8fd07ce640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f8fc815f4d0 con 0x7f8fc806da90 2024-04-24T22:59:18.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.442+0000 7f8fce56a640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8fc8158dd0 0x7f8fc806b320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47532/0 (socket says 172.21.15.32:47532) 2024-04-24T22:59:18.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.442+0000 7f8fcdd69640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8fc806ba30 0x7f8fc806d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:18.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.442+0000 7f8fce56a640 1 -- 172.21.15.32:0/2689568444 learned_addr learned my addr 172.21.15.32:0/2689568444 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:18.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.442+0000 7f8fced6b640 1 --2- 172.21.15.32:0/2689568444 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8fc806da90 0x7f8fc80b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:18.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.442+0000 7f8fcdd69640 1 -- 172.21.15.32:0/2689568444 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8fc806da90 msgr2=0x7f8fc80b5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:18.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.442+0000 7f8fcdd69640 1 --2- 172.21.15.32:0/2689568444 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8fc806da90 0x7f8fc80b5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:18.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.442+0000 7f8fcdd69640 1 -- 172.21.15.32:0/2689568444 >> v1:172.21.15.32:6789/0 conn(0x7f8fc8158dd0 legacy=0x7f8fc806b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:18.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.442+0000 7f8fcd568640 1 -- 172.21.15.32:0/2689568444 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1360794312 0 0) 0x7f8fc815ef10 con 0x7f8fc8158dd0 2024-04-24T22:59:18.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.442+0000 7f8fcdd69640 1 -- 172.21.15.32:0/2689568444 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fc815f7b0 con 0x7f8fc806ba30 2024-04-24T22:59:18.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.442+0000 7f8fced6b640 1 --2- 172.21.15.32:0/2689568444 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8fc806da90 0x7f8fc80b5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:18.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.443+0000 7f8fcdd69640 1 --2- 172.21.15.32:0/2689568444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fc806ba30 0x7f8fc806d550 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8fb80097a0 tx=0x7f8fb802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f6bf58f93df7693 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:18.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.443+0000 7f8fcd568640 1 -- 172.21.15.32:0/2689568444 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb8037070 con 0x7f8fc806ba30 2024-04-24T22:59:18.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.443+0000 7f8fcd568640 1 -- 172.21.15.32:0/2689568444 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fb8032440 con 0x7f8fc806ba30 2024-04-24T22:59:18.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.443+0000 7f8fcd568640 1 -- 172.21.15.32:0/2689568444 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb8032810 con 0x7f8fc806ba30 2024-04-24T22:59:18.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.444+0000 7f8fd07ce640 1 -- 172.21.15.32:0/2689568444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fc806ba30 msgr2=0x7f8fc806d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:18.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.444+0000 7f8fd07ce640 1 --2- 172.21.15.32:0/2689568444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fc806ba30 0x7f8fc806d550 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8fb80097a0 tx=0x7f8fb802a180 comp rx=0 tx=0).stop 2024-04-24T22:59:18.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.444+0000 7f8fd07ce640 1 -- 172.21.15.32:0/2689568444 shutdown_connections 2024-04-24T22:59:18.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.444+0000 7f8fd07ce640 1 --2- 172.21.15.32:0/2689568444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fc806ba30 0x7f8fc806d550 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:18.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.444+0000 7f8fd07ce640 1 --2- 172.21.15.32:0/2689568444 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8fc806da90 0x7f8fc80b5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:18.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.444+0000 7f8fd07ce640 1 -- 172.21.15.32:0/2689568444 >> 172.21.15.32:0/2689568444 conn(0x7f8fc806e800 msgr2=0x7f8fc806ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:18.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.444+0000 7f8fd07ce640 1 -- 172.21.15.32:0/2689568444 shutdown_connections 2024-04-24T22:59:18.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.444+0000 7f8fd07ce640 1 -- 172.21.15.32:0/2689568444 wait complete. 2024-04-24T22:59:18.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.445+0000 7f8fd07ce640 1 Processor -- start 2024-04-24T22:59:18.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.445+0000 7f8fd07ce640 1 -- start start 2024-04-24T22:59:18.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.446+0000 7f8fd07ce640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fc806ba30 0x7f8fc80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:18.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.446+0000 7f8fd07ce640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fc806da90 0x7f8fc80b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:18.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.446+0000 7f8fd07ce640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8fc8158dd0 0x7f8fc814bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:18.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.446+0000 7f8fd07ce640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc806bfe0 con 0x7f8fc806da90 2024-04-24T22:59:18.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.446+0000 7f8fd07ce640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc806ded0 con 0x7f8fc806ba30 2024-04-24T22:59:18.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.446+0000 7f8fd07ce640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f8fc81591b0 con 0x7f8fc8158dd0 2024-04-24T22:59:18.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.446+0000 7f8fcdd69640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fc806da90 0x7f8fc80b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:18.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.446+0000 7f8fcdd69640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fc806da90 0x7f8fc80b31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38920/0 (socket says 172.21.15.32:38920) 2024-04-24T22:59:18.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.446+0000 7f8fcdd69640 1 -- 172.21.15.32:0/3065314328 learned_addr learned my addr 172.21.15.32:0/3065314328 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:18.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.446+0000 7f8fce56a640 1 --2- 172.21.15.32:0/3065314328 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fc806ba30 0x7f8fc80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:18.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.446+0000 7f8fcdd69640 1 -- 172.21.15.32:0/3065314328 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8fc8158dd0 msgr2=0x7f8fc814bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:18.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.446+0000 7f8fcdd69640 1 --2- 172.21.15.32:0/3065314328 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8fc8158dd0 0x7f8fc814bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:18.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.446+0000 7f8fcdd69640 1 -- 172.21.15.32:0/3065314328 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fc806ba30 msgr2=0x7f8fc80b4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:18.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.446+0000 7f8fcdd69640 1 --2- 172.21.15.32:0/3065314328 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fc806ba30 0x7f8fc80b4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:18.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.446+0000 7f8fcdd69640 1 -- 172.21.15.32:0/3065314328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fc80b4b40 con 0x7f8fc806da90 2024-04-24T22:59:18.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.446+0000 7f8fcdd69640 1 --2- 172.21.15.32:0/3065314328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fc806da90 0x7f8fc80b31e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8fb80097a0 tx=0x7f8fb80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:18.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.446+0000 7f8fbeffd640 1 -- 172.21.15.32:0/3065314328 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb8037050 con 0x7f8fc806da90 2024-04-24T22:59:18.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.446+0000 7f8fbeffd640 1 -- 172.21.15.32:0/3065314328 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fb8038040 con 0x7f8fc806da90 2024-04-24T22:59:18.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.446+0000 7f8fd07ce640 1 -- 172.21.15.32:0/3065314328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fc80b3870 con 0x7f8fc806da90 2024-04-24T22:59:18.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.446+0000 7f8fbeffd640 1 -- 172.21.15.32:0/3065314328 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb8004470 con 0x7f8fc806da90 2024-04-24T22:59:18.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.447+0000 7f8fd07ce640 1 -- 172.21.15.32:0/3065314328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fc80b3e30 con 0x7f8fc806da90 2024-04-24T22:59:18.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.447+0000 7f8fd07ce640 1 -- 172.21.15.32:0/3065314328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fc8068430 con 0x7f8fc806da90 2024-04-24T22:59:18.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.448+0000 7f8fbeffd640 1 -- 172.21.15.32:0/3065314328 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8fb8030900 con 0x7f8fc806da90 2024-04-24T22:59:18.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.448+0000 7f8fbeffd640 1 --2- 172.21.15.32:0/3065314328 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8fac06be30 0x7f8fac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:18.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.448+0000 7f8fbeffd640 1 -- 172.21.15.32:0/3065314328 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8fb80ab9f0 con 0x7f8fc806da90 2024-04-24T22:59:18.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.448+0000 7f8fce56a640 1 --2- 172.21.15.32:0/3065314328 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8fac06be30 0x7f8fac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:18.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.449+0000 7f8fce56a640 1 --2- 172.21.15.32:0/3065314328 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8fac06be30 0x7f8fac06e2f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4000f60 tx=0x7f8fc4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:18.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.452+0000 7f8fbeffd640 1 -- 172.21.15.32:0/3065314328 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8fb807a3a0 con 0x7f8fc806da90 2024-04-24T22:59:18.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.677+0000 7f8fd07ce640 1 -- 172.21.15.32:0/3065314328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8fc80d2a40 con 0x7f8fc806da90 2024-04-24T22:59:18.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.677+0000 7f8fbeffd640 1 -- 172.21.15.32:0/3065314328 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5178 (secure 0 0 0) 0x7f8fb8079ca0 con 0x7f8fc806da90 2024-04-24T22:59:18.679 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:59:18.679 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1485884060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1485884060},{"type":"v1","addr":"172.21.15.121:6837","nonce":1485884060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/923934104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":923934104},{"type":"v1","addr":"172.21.15.32:6833","nonce":923934104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3229115835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3229115835},{"type":"v1","addr":"172.21.15.32:6835","nonce":3229115835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T22:59:03.876305+0000","modified":"2024-04-24T22:59:05.932147+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6836/2644625343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2644625343},{"type":"v1","addr":"172.21.15.121:6836","nonce":2644625343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4276]},"id":1}]} 2024-04-24T22:59:18.679 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 6 2024-04-24T22:59:18.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.679+0000 7f8fd07ce640 1 -- 172.21.15.32:0/3065314328 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8fac06be30 msgr2=0x7f8fac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:18.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.679+0000 7f8fd07ce640 1 --2- 172.21.15.32:0/3065314328 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8fac06be30 0x7f8fac06e2f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4000f60 tx=0x7f8fc4009040 comp rx=0 tx=0).stop 2024-04-24T22:59:18.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.679+0000 7f8fd07ce640 1 -- 172.21.15.32:0/3065314328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fc806da90 msgr2=0x7f8fc80b31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:18.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.679+0000 7f8fd07ce640 1 --2- 172.21.15.32:0/3065314328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fc806da90 0x7f8fc80b31e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8fb80097a0 tx=0x7f8fb80046c0 comp rx=0 tx=0).stop 2024-04-24T22:59:18.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.679+0000 7f8fd07ce640 1 -- 172.21.15.32:0/3065314328 shutdown_connections 2024-04-24T22:59:18.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.679+0000 7f8fd07ce640 1 --2- 172.21.15.32:0/3065314328 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8fac06be30 0x7f8fac06e2f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:18.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.679+0000 7f8fd07ce640 1 --2- 172.21.15.32:0/3065314328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fc806da90 0x7f8fc80b31e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:18.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.679+0000 7f8fd07ce640 1 --2- 172.21.15.32:0/3065314328 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8fc8158dd0 0x7f8fc814bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:18.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.679+0000 7f8fd07ce640 1 --2- 172.21.15.32:0/3065314328 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fc806ba30 0x7f8fc80b4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:18.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.679+0000 7f8fd07ce640 1 -- 172.21.15.32:0/3065314328 >> 172.21.15.32:0/3065314328 conn(0x7f8fc806e800 msgr2=0x7f8fc81616f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:18.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.679+0000 7f8fd07ce640 1 -- 172.21.15.32:0/3065314328 shutdown_connections 2024-04-24T22:59:18.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.679+0000 7f8fd07ce640 1 -- 172.21.15.32:0/3065314328 wait complete. 2024-04-24T22:59:18.692 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:59:18.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.791+0000 7f89d1d42640 1 Processor -- start 2024-04-24T22:59:18.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.791+0000 7f89d1d42640 1 -- start start 2024-04-24T22:59:18.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.792+0000 7f89d1d42640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f89cc162e10 0x7f89cc0ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:18.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.792+0000 7f89d1d42640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89cc0cd270 con 0x7f89cc0b5af0 2024-04-24T22:59:18.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.792+0000 7f89d1d42640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89cc071610 con 0x7f89cc162320 2024-04-24T22:59:18.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.792+0000 7f89d1d42640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f89cc0b5660 con 0x7f89cc162e10 2024-04-24T22:59:18.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.792+0000 7f89caffd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f89cc0b5af0 0x7f89cc15ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47548/0 (socket says 172.21.15.32:47548) 2024-04-24T22:59:18.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.792+0000 7f89caffd640 1 -- 172.21.15.32:0/3338634781 learned_addr learned my addr 172.21.15.32:0/3338634781 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:18.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.792+0000 7f89cbfff640 1 --2- 172.21.15.32:0/3338634781 >> v2:172.21.15.32:3300/0 conn(0x7f89cc162e10 0x7f89cc0ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:18.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.792+0000 7f89cbfff640 1 -- 172.21.15.32:0/3338634781 >> v1:172.21.15.121:6789/0 conn(0x7f89cc162320 legacy=0x7f89cc162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:18.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.792+0000 7f89cbfff640 1 -- 172.21.15.32:0/3338634781 >> v1:172.21.15.32:6789/0 conn(0x7f89cc0b5af0 legacy=0x7f89cc15ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:18.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.792+0000 7f89cbfff640 1 -- 172.21.15.32:0/3338634781 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89cc0718f0 con 0x7f89cc162e10 2024-04-24T22:59:18.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.793+0000 7f89cbfff640 1 --2- 172.21.15.32:0/3338634781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89cc162e10 0x7f89cc0ccc90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f89c4009d90 tx=0x7f89c402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d891a32120a7e95 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:18.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.793+0000 7f89ca7fc640 1 -- 172.21.15.32:0/3338634781 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c4007740 con 0x7f89cc162e10 2024-04-24T22:59:18.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.793+0000 7f89ca7fc640 1 -- 172.21.15.32:0/3338634781 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89c40078e0 con 0x7f89cc162e10 2024-04-24T22:59:18.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.793+0000 7f89ca7fc640 1 -- 172.21.15.32:0/3338634781 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c4007cb0 con 0x7f89cc162e10 2024-04-24T22:59:18.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.794+0000 7f89d1d42640 1 -- 172.21.15.32:0/3338634781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89cc162e10 msgr2=0x7f89cc0ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:18.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.794+0000 7f89d1d42640 1 --2- 172.21.15.32:0/3338634781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89cc162e10 0x7f89cc0ccc90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f89c4009d90 tx=0x7f89c402a660 comp rx=0 tx=0).stop 2024-04-24T22:59:18.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.794+0000 7f89d1d42640 1 -- 172.21.15.32:0/3338634781 shutdown_connections 2024-04-24T22:59:18.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.794+0000 7f89d1d42640 1 --2- 172.21.15.32:0/3338634781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89cc162e10 0x7f89cc0ccc90 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:18.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.794+0000 7f89d1d42640 1 -- 172.21.15.32:0/3338634781 >> 172.21.15.32:0/3338634781 conn(0x7f89cc0d0ae0 msgr2=0x7f89cc0d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:18.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.794+0000 7f89d1d42640 1 -- 172.21.15.32:0/3338634781 shutdown_connections 2024-04-24T22:59:18.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.794+0000 7f89d1d42640 1 -- 172.21.15.32:0/3338634781 wait complete. 2024-04-24T22:59:18.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.795+0000 7f89d1d42640 1 Processor -- start 2024-04-24T22:59:18.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.795+0000 7f89d1d42640 1 -- start start 2024-04-24T22:59:18.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.796+0000 7f89d1d42640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89cc0b5af0 0x7f89cc0cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:18.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.796+0000 7f89d1d42640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89cc162320 0x7f89cc15b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:18.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.796+0000 7f89d1d42640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f89cc162e10 0x7f89cc149460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:18.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.796+0000 7f89d1d42640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f89cc071bd0 con 0x7f89cc0b5af0 2024-04-24T22:59:18.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.796+0000 7f89d1d42640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f89cc067950 con 0x7f89cc162320 2024-04-24T22:59:18.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.796+0000 7f89d1d42640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f89cc071d50 con 0x7f89cc162e10 2024-04-24T22:59:18.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.796+0000 7f89cb7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89cc0b5af0 0x7f89cc0cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:18.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.796+0000 7f89cb7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89cc0b5af0 0x7f89cc0cd4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38938/0 (socket says 172.21.15.32:38938) 2024-04-24T22:59:18.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.796+0000 7f89cb7fe640 1 -- 172.21.15.32:0/1885143945 learned_addr learned my addr 172.21.15.32:0/1885143945 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:18.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.796+0000 7f89caffd640 1 --2- 172.21.15.32:0/1885143945 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89cc162320 0x7f89cc15b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:18.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.796+0000 7f89cbfff640 1 --2- 172.21.15.32:0/1885143945 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f89cc162e10 0x7f89cc149460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:18.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.796+0000 7f89cb7fe640 1 -- 172.21.15.32:0/1885143945 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f89cc162e10 msgr2=0x7f89cc149460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:18.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.796+0000 7f89cb7fe640 1 --2- 172.21.15.32:0/1885143945 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f89cc162e10 0x7f89cc149460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:18.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.796+0000 7f89cb7fe640 1 -- 172.21.15.32:0/1885143945 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89cc162320 msgr2=0x7f89cc15b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:18.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.796+0000 7f89cb7fe640 1 --2- 172.21.15.32:0/1885143945 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89cc162320 0x7f89cc15b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:18.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.796+0000 7f89cb7fe640 1 -- 172.21.15.32:0/1885143945 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89cc0cdad0 con 0x7f89cc0b5af0 2024-04-24T22:59:18.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.796+0000 7f89cb7fe640 1 --2- 172.21.15.32:0/1885143945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89cc0b5af0 0x7f89cc0cd4f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f89bc00c450 tx=0x7f89bc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:18.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.796+0000 7f89b3fff640 1 -- 172.21.15.32:0/1885143945 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89bc0052d0 con 0x7f89cc0b5af0 2024-04-24T22:59:18.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.796+0000 7f89b3fff640 1 -- 172.21.15.32:0/1885143945 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89bc005470 con 0x7f89cc0b5af0 2024-04-24T22:59:18.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.796+0000 7f89b3fff640 1 -- 172.21.15.32:0/1885143945 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89bc012870 con 0x7f89cc0b5af0 2024-04-24T22:59:18.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.796+0000 7f89d1d42640 1 -- 172.21.15.32:0/1885143945 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89cc0cddb0 con 0x7f89cc0b5af0 2024-04-24T22:59:18.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.797+0000 7f89d1d42640 1 -- 172.21.15.32:0/1885143945 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89cc0ce370 con 0x7f89cc0b5af0 2024-04-24T22:59:18.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.797+0000 7f89d1d42640 1 -- 172.21.15.32:0/1885143945 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89cc068430 con 0x7f89cc0b5af0 2024-04-24T22:59:18.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.798+0000 7f89b3fff640 1 -- 172.21.15.32:0/1885143945 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f89bc012a10 con 0x7f89cc0b5af0 2024-04-24T22:59:18.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.798+0000 7f89b3fff640 1 --2- 172.21.15.32:0/1885143945 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f89b806be30 0x7f89b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:18.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.799+0000 7f89b3fff640 1 -- 172.21.15.32:0/1885143945 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f89bc08a980 con 0x7f89cc0b5af0 2024-04-24T22:59:18.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.799+0000 7f89caffd640 1 --2- 172.21.15.32:0/1885143945 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f89b806be30 0x7f89b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:18.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.799+0000 7f89caffd640 1 --2- 172.21.15.32:0/1885143945 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f89b806be30 0x7f89b806e2f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f89cc0b5170 tx=0x7f89c0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:18.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.803+0000 7f89b3fff640 1 -- 172.21.15.32:0/1885143945 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f89bc058c40 con 0x7f89cc0b5af0 2024-04-24T22:59:18.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.986+0000 7f89d1d42640 1 -- 172.21.15.32:0/1885143945 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f89cc067730 con 0x7f89cc0b5af0 2024-04-24T22:59:18.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.987+0000 7f89b3fff640 1 -- 172.21.15.32:0/1885143945 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16621 (secure 0 0 0) 0x7f89bc05cf40 con 0x7f89cc0b5af0 2024-04-24T22:59:18.989 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:59:18.989 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T22:59:05.517554+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:59:02.016639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T22:59:02.846087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:59:18.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.989+0000 7f89d1d42640 1 -- 172.21.15.32:0/1885143945 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f89b806be30 msgr2=0x7f89b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:18.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.989+0000 7f89d1d42640 1 --2- 172.21.15.32:0/1885143945 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f89b806be30 0x7f89b806e2f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f89cc0b5170 tx=0x7f89c0009040 comp rx=0 tx=0).stop 2024-04-24T22:59:18.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.989+0000 7f89d1d42640 1 -- 172.21.15.32:0/1885143945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89cc0b5af0 msgr2=0x7f89cc0cd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:18.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.989+0000 7f89d1d42640 1 --2- 172.21.15.32:0/1885143945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89cc0b5af0 0x7f89cc0cd4f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f89bc00c450 tx=0x7f89bc00c910 comp rx=0 tx=0).stop 2024-04-24T22:59:18.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.989+0000 7f89d1d42640 1 -- 172.21.15.32:0/1885143945 shutdown_connections 2024-04-24T22:59:18.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.989+0000 7f89d1d42640 1 --2- 172.21.15.32:0/1885143945 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f89b806be30 0x7f89b806e2f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:18.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.989+0000 7f89d1d42640 1 --2- 172.21.15.32:0/1885143945 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f89cc162e10 0x7f89cc149460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:18.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.989+0000 7f89d1d42640 1 --2- 172.21.15.32:0/1885143945 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89cc162320 0x7f89cc15b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:18.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.989+0000 7f89d1d42640 1 --2- 172.21.15.32:0/1885143945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89cc0b5af0 0x7f89cc0cd4f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:18.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.989+0000 7f89d1d42640 1 -- 172.21.15.32:0/1885143945 >> 172.21.15.32:0/1885143945 conn(0x7f89cc0d0ae0 msgr2=0x7f89cc0b1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:18.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.989+0000 7f89d1d42640 1 -- 172.21.15.32:0/1885143945 shutdown_connections 2024-04-24T22:59:18.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:18.989+0000 7f89d1d42640 1 -- 172.21.15.32:0/1885143945 wait complete. 2024-04-24T22:59:19.002 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T22:59:02.016639+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T22:59:02.846087+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T22:59:19.002 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T22:59:19.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.104+0000 7f3c70961640 1 Processor -- start 2024-04-24T22:59:19.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.104+0000 7f3c70961640 1 -- start start 2024-04-24T22:59:19.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.105+0000 7f3c70961640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3c6c0b44d0 0x7f3c6c0b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:19.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.105+0000 7f3c70961640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3c6c0af890 0x7f3c6c15cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:19.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.105+0000 7f3c70961640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c6c0b6430 con 0x7f3c6c06c5f0 2024-04-24T22:59:19.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.105+0000 7f3c70961640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f3c6c156f80 con 0x7f3c6c0b44d0 2024-04-24T22:59:19.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.105+0000 7f3c70961640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f3c6c0b6150 con 0x7f3c6c0af890 2024-04-24T22:59:19.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.105+0000 7f3c63fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3c6c0b44d0 0x7f3c6c0b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:19.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.105+0000 7f3c6a59c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3c6c06c5f0 0x7f3c6c06c9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47564/0 (socket says 172.21.15.32:47564) 2024-04-24T22:59:19.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.105+0000 7f3c6a59c640 1 -- 172.21.15.32:0/1844387404 learned_addr learned my addr 172.21.15.32:0/1844387404 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:19.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.105+0000 7f3c6ad9d640 1 --2- 172.21.15.32:0/1844387404 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3c6c0af890 0x7f3c6c15cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:19.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.105+0000 7f3c63fff640 1 -- 172.21.15.32:0/1844387404 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3c6c0af890 msgr2=0x7f3c6c15cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:19.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.105+0000 7f3c63fff640 1 --2- 172.21.15.32:0/1844387404 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3c6c0af890 0x7f3c6c15cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:19.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.105+0000 7f3c63fff640 1 -- 172.21.15.32:0/1844387404 >> v1:172.21.15.32:6789/0 conn(0x7f3c6c06c5f0 legacy=0x7f3c6c06c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:19.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.105+0000 7f3c63fff640 1 -- 172.21.15.32:0/1844387404 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c6c0b39f0 con 0x7f3c6c0b44d0 2024-04-24T22:59:19.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.105+0000 7f3c69d9b640 1 -- 172.21.15.32:0/1844387404 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1221154671 0 0) 0x7f3c6c0b6430 con 0x7f3c6c06c5f0 2024-04-24T22:59:19.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.106+0000 7f3c63fff640 1 --2- 172.21.15.32:0/1844387404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c6c0b44d0 0x7f3c6c0b5890 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3c540097a0 tx=0x7f3c5402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d937ed0c6896478 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:19.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.106+0000 7f3c69d9b640 1 -- 172.21.15.32:0/1844387404 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c54037070 con 0x7f3c6c0b44d0 2024-04-24T22:59:19.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.106+0000 7f3c69d9b640 1 -- 172.21.15.32:0/1844387404 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c5402ae50 con 0x7f3c6c0b44d0 2024-04-24T22:59:19.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.106+0000 7f3c69d9b640 1 -- 172.21.15.32:0/1844387404 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c54032660 con 0x7f3c6c0b44d0 2024-04-24T22:59:19.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.107+0000 7f3c70961640 1 -- 172.21.15.32:0/1844387404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c6c0b44d0 msgr2=0x7f3c6c0b5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:19.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.107+0000 7f3c70961640 1 --2- 172.21.15.32:0/1844387404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c6c0b44d0 0x7f3c6c0b5890 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3c540097a0 tx=0x7f3c5402a100 comp rx=0 tx=0).stop 2024-04-24T22:59:19.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.107+0000 7f3c70961640 1 -- 172.21.15.32:0/1844387404 shutdown_connections 2024-04-24T22:59:19.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.107+0000 7f3c70961640 1 --2- 172.21.15.32:0/1844387404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c6c0b44d0 0x7f3c6c0b5890 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:19.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.107+0000 7f3c70961640 1 --2- 172.21.15.32:0/1844387404 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3c6c0af890 0x7f3c6c15cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:19.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.107+0000 7f3c70961640 1 -- 172.21.15.32:0/1844387404 >> 172.21.15.32:0/1844387404 conn(0x7f3c6c06b110 msgr2=0x7f3c6c06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:19.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.107+0000 7f3c70961640 1 -- 172.21.15.32:0/1844387404 shutdown_connections 2024-04-24T22:59:19.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.107+0000 7f3c70961640 1 -- 172.21.15.32:0/1844387404 wait complete. 2024-04-24T22:59:19.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.108+0000 7f3c70961640 1 Processor -- start 2024-04-24T22:59:19.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.108+0000 7f3c70961640 1 -- start start 2024-04-24T22:59:19.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.109+0000 7f3c70961640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c6c06c5f0 0x7f3c6c153b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:19.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.109+0000 7f3c70961640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c6c0af890 0x7f3c6c1f4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:19.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.109+0000 7f3c70961640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3c6c0b44d0 0x7f3c6c146f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:19.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.109+0000 7f3c70961640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3c6c0b5dd0 con 0x7f3c6c06c5f0 2024-04-24T22:59:19.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.109+0000 7f3c70961640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3c6c0b3fb0 con 0x7f3c6c0af890 2024-04-24T22:59:19.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.109+0000 7f3c70961640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f3c6c15d000 con 0x7f3c6c0b44d0 2024-04-24T22:59:19.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.109+0000 7f3c6a59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c6c06c5f0 0x7f3c6c153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:19.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.109+0000 7f3c6a59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c6c06c5f0 0x7f3c6c153b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38954/0 (socket says 172.21.15.32:38954) 2024-04-24T22:59:19.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.109+0000 7f3c6a59c640 1 -- 172.21.15.32:0/2449570757 learned_addr learned my addr 172.21.15.32:0/2449570757 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:19.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.109+0000 7f3c6ad9d640 1 --2- 172.21.15.32:0/2449570757 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3c6c0b44d0 0x7f3c6c146f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:19.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.109+0000 7f3c63fff640 1 --2- 172.21.15.32:0/2449570757 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c6c0af890 0x7f3c6c1f4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:19.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.109+0000 7f3c6a59c640 1 -- 172.21.15.32:0/2449570757 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3c6c0b44d0 msgr2=0x7f3c6c146f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:19.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.109+0000 7f3c6a59c640 1 --2- 172.21.15.32:0/2449570757 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3c6c0b44d0 0x7f3c6c146f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:19.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.109+0000 7f3c6a59c640 1 -- 172.21.15.32:0/2449570757 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c6c0af890 msgr2=0x7f3c6c1f4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:19.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.109+0000 7f3c6a59c640 1 --2- 172.21.15.32:0/2449570757 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c6c0af890 0x7f3c6c1f4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:19.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.109+0000 7f3c6a59c640 1 -- 172.21.15.32:0/2449570757 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c6c147680 con 0x7f3c6c06c5f0 2024-04-24T22:59:19.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.109+0000 7f3c63fff640 1 --2- 172.21.15.32:0/2449570757 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c6c0af890 0x7f3c6c1f4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:19.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.109+0000 7f3c6a59c640 1 --2- 172.21.15.32:0/2449570757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c6c06c5f0 0x7f3c6c153b70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c00c4a0 tx=0x7f3c5c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:19.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.109+0000 7f3c62ffd640 1 -- 172.21.15.32:0/2449570757 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c017070 con 0x7f3c6c06c5f0 2024-04-24T22:59:19.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.109+0000 7f3c62ffd640 1 -- 172.21.15.32:0/2449570757 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c5c005ce0 con 0x7f3c6c06c5f0 2024-04-24T22:59:19.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.109+0000 7f3c70961640 1 -- 172.21.15.32:0/2449570757 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c6c147960 con 0x7f3c6c06c5f0 2024-04-24T22:59:19.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.110+0000 7f3c62ffd640 1 -- 172.21.15.32:0/2449570757 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c0072b0 con 0x7f3c6c06c5f0 2024-04-24T22:59:19.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.110+0000 7f3c70961640 1 -- 172.21.15.32:0/2449570757 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c6c147f20 con 0x7f3c6c06c5f0 2024-04-24T22:59:19.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.110+0000 7f3c70961640 1 -- 172.21.15.32:0/2449570757 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c6c148250 con 0x7f3c6c06c5f0 2024-04-24T22:59:19.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.111+0000 7f3c62ffd640 1 -- 172.21.15.32:0/2449570757 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3c5c01a070 con 0x7f3c6c06c5f0 2024-04-24T22:59:19.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.111+0000 7f3c62ffd640 1 --2- 172.21.15.32:0/2449570757 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3c4c06be30 0x7f3c4c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:19.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.112+0000 7f3c62ffd640 1 -- 172.21.15.32:0/2449570757 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3c5c08a170 con 0x7f3c6c06c5f0 2024-04-24T22:59:19.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.112+0000 7f3c63fff640 1 --2- 172.21.15.32:0/2449570757 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3c4c06be30 0x7f3c4c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:19.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.112+0000 7f3c63fff640 1 --2- 172.21.15.32:0/2449570757 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3c4c06be30 0x7f3c4c06e2f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3c54034000 tx=0x7f3c5402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:19.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.116+0000 7f3c62ffd640 1 -- 172.21.15.32:0/2449570757 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3c5c0584d0 con 0x7f3c6c06c5f0 2024-04-24T22:59:19.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.299+0000 7f3c70961640 1 -- 172.21.15.32:0/2449570757 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3c6c068430 con 0x7f3c6c06c5f0 2024-04-24T22:59:19.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.299+0000 7f3c62ffd640 1 -- 172.21.15.32:0/2449570757 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f3c5c05c7d0 con 0x7f3c6c06c5f0 2024-04-24T22:59:19.301 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T22:59:19.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.301+0000 7f3c70961640 1 -- 172.21.15.32:0/2449570757 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3c4c06be30 msgr2=0x7f3c4c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:19.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.301+0000 7f3c70961640 1 --2- 172.21.15.32:0/2449570757 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3c4c06be30 0x7f3c4c06e2f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3c54034000 tx=0x7f3c5402e000 comp rx=0 tx=0).stop 2024-04-24T22:59:19.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.301+0000 7f3c70961640 1 -- 172.21.15.32:0/2449570757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c6c06c5f0 msgr2=0x7f3c6c153b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:19.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.301+0000 7f3c70961640 1 --2- 172.21.15.32:0/2449570757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c6c06c5f0 0x7f3c6c153b70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c00c4a0 tx=0x7f3c5c00c960 comp rx=0 tx=0).stop 2024-04-24T22:59:19.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.301+0000 7f3c70961640 1 -- 172.21.15.32:0/2449570757 shutdown_connections 2024-04-24T22:59:19.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.301+0000 7f3c70961640 1 --2- 172.21.15.32:0/2449570757 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3c4c06be30 0x7f3c4c06e2f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:19.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.301+0000 7f3c70961640 1 --2- 172.21.15.32:0/2449570757 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3c6c0b44d0 0x7f3c6c146f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:19.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.301+0000 7f3c70961640 1 --2- 172.21.15.32:0/2449570757 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c6c0af890 0x7f3c6c1f4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:19.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.301+0000 7f3c70961640 1 --2- 172.21.15.32:0/2449570757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c6c06c5f0 0x7f3c6c153b70 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:19.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.301+0000 7f3c70961640 1 -- 172.21.15.32:0/2449570757 >> 172.21.15.32:0/2449570757 conn(0x7f3c6c06b110 msgr2=0x7f3c6c071790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:19.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.301+0000 7f3c70961640 1 -- 172.21.15.32:0/2449570757 shutdown_connections 2024-04-24T22:59:19.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.301+0000 7f3c70961640 1 -- 172.21.15.32:0/2449570757 wait complete. 2024-04-24T22:59:19.314 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T22:59:19.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.416+0000 7fc46a638640 1 Processor -- start 2024-04-24T22:59:19.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.416+0000 7fc46a638640 1 -- start start 2024-04-24T22:59:19.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.417+0000 7fc46a638640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc464158dd0 0x7fc46406b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:19.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.417+0000 7fc46a638640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc46406bd10 con 0x7fc46406b860 2024-04-24T22:59:19.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.417+0000 7fc46a638640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc46415ef10 con 0x7fc46406dc60 2024-04-24T22:59:19.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.417+0000 7fc46a638640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fc46415f1f0 con 0x7fc464158dd0 2024-04-24T22:59:19.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.417+0000 7fc463fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc464158dd0 0x7fc46406b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:19.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.417+0000 7fc463fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc464158dd0 0x7fc46406b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38966/0 (socket says 172.21.15.32:38966) 2024-04-24T22:59:19.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.417+0000 7fc4637fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc46406b860 0x7fc46406d550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47574/0 (socket says 172.21.15.32:47574) 2024-04-24T22:59:19.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.417+0000 7fc463fff640 1 -- 172.21.15.32:0/4082145450 learned_addr learned my addr 172.21.15.32:0/4082145450 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:19.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.417+0000 7fc463fff640 1 -- 172.21.15.32:0/4082145450 >> v1:172.21.15.121:6789/0 conn(0x7fc46406dc60 legacy=0x7fc4640b5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:19.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.417+0000 7fc463fff640 1 -- 172.21.15.32:0/4082145450 >> v1:172.21.15.32:6789/0 conn(0x7fc46406b860 legacy=0x7fc46406d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:19.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.417+0000 7fc463fff640 1 -- 172.21.15.32:0/4082145450 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc46415f4d0 con 0x7fc464158dd0 2024-04-24T22:59:19.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.417+0000 7fc462ffd640 1 -- 172.21.15.32:0/4082145450 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2269869503 0 0) 0x7fc46406bd10 con 0x7fc46406b860 2024-04-24T22:59:19.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.418+0000 7fc463fff640 1 --2- 172.21.15.32:0/4082145450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc464158dd0 0x7fc46406b320 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc458009870 tx=0x7fc45802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58096ce4c5740a19 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:19.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.418+0000 7fc462ffd640 1 -- 172.21.15.32:0/4082145450 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc458037070 con 0x7fc464158dd0 2024-04-24T22:59:19.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.418+0000 7fc462ffd640 1 -- 172.21.15.32:0/4082145450 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc458032440 con 0x7fc464158dd0 2024-04-24T22:59:19.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.418+0000 7fc462ffd640 1 -- 172.21.15.32:0/4082145450 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc458032810 con 0x7fc464158dd0 2024-04-24T22:59:19.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.419+0000 7fc46a638640 1 -- 172.21.15.32:0/4082145450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc464158dd0 msgr2=0x7fc46406b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:19.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.419+0000 7fc46a638640 1 --2- 172.21.15.32:0/4082145450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc464158dd0 0x7fc46406b320 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc458009870 tx=0x7fc45802a2a0 comp rx=0 tx=0).stop 2024-04-24T22:59:19.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.419+0000 7fc46a638640 1 -- 172.21.15.32:0/4082145450 shutdown_connections 2024-04-24T22:59:19.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.419+0000 7fc46a638640 1 --2- 172.21.15.32:0/4082145450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc464158dd0 0x7fc46406b320 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:19.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.419+0000 7fc46a638640 1 -- 172.21.15.32:0/4082145450 >> 172.21.15.32:0/4082145450 conn(0x7fc46406e800 msgr2=0x7fc46406ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:19.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.419+0000 7fc46a638640 1 -- 172.21.15.32:0/4082145450 shutdown_connections 2024-04-24T22:59:19.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.419+0000 7fc46a638640 1 -- 172.21.15.32:0/4082145450 wait complete. 2024-04-24T22:59:19.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.420+0000 7fc46a638640 1 Processor -- start 2024-04-24T22:59:19.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.420+0000 7fc46a638640 1 -- start start 2024-04-24T22:59:19.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.421+0000 7fc46a638640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc46406b860 0x7fc46414c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:19.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.421+0000 7fc46a638640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc46406dc60 0x7fc464146d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:19.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.421+0000 7fc46a638640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc464158dd0 0x7fc4641472d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:19.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.421+0000 7fc46a638640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc46406bfe0 con 0x7fc46406dc60 2024-04-24T22:59:19.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.421+0000 7fc46a638640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc46415fcd0 con 0x7fc46406b860 2024-04-24T22:59:19.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.421+0000 7fc46a638640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc4641591b0 con 0x7fc464158dd0 2024-04-24T22:59:19.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.421+0000 7fc4637fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc46406dc60 0x7fc464146d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:19.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.421+0000 7fc4637fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc46406dc60 0x7fc464146d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38974/0 (socket says 172.21.15.32:38974) 2024-04-24T22:59:19.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.421+0000 7fc4637fe640 1 -- 172.21.15.32:0/2038813094 learned_addr learned my addr 172.21.15.32:0/2038813094 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:19.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.421+0000 7fc463fff640 1 --2- 172.21.15.32:0/2038813094 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc46406b860 0x7fc46414c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:19.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.421+0000 7fc468bd5640 1 --2- 172.21.15.32:0/2038813094 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc464158dd0 0x7fc4641472d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:19.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.421+0000 7fc4637fe640 1 -- 172.21.15.32:0/2038813094 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc464158dd0 msgr2=0x7fc4641472d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:19.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.421+0000 7fc4637fe640 1 --2- 172.21.15.32:0/2038813094 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc464158dd0 0x7fc4641472d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:19.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.421+0000 7fc4637fe640 1 -- 172.21.15.32:0/2038813094 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc46406b860 msgr2=0x7fc46414c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:19.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.421+0000 7fc4637fe640 1 --2- 172.21.15.32:0/2038813094 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc46406b860 0x7fc46414c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:19.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.421+0000 7fc4637fe640 1 -- 172.21.15.32:0/2038813094 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4641f4370 con 0x7fc46406dc60 2024-04-24T22:59:19.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.421+0000 7fc468bd5640 1 --2- 172.21.15.32:0/2038813094 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc464158dd0 0x7fc4641472d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:19.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.422+0000 7fc4637fe640 1 --2- 172.21.15.32:0/2038813094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc46406dc60 0x7fc464146d90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc45400c460 tx=0x7fc45400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:19.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.422+0000 7fc460ff9640 1 -- 172.21.15.32:0/2038813094 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc454018070 con 0x7fc46406dc60 2024-04-24T22:59:19.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.422+0000 7fc460ff9640 1 -- 172.21.15.32:0/2038813094 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc454007500 con 0x7fc46406dc60 2024-04-24T22:59:19.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.422+0000 7fc460ff9640 1 -- 172.21.15.32:0/2038813094 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc454004a20 con 0x7fc46406dc60 2024-04-24T22:59:19.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.422+0000 7fc46a638640 1 -- 172.21.15.32:0/2038813094 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4641f45f0 con 0x7fc46406dc60 2024-04-24T22:59:19.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.422+0000 7fc46a638640 1 -- 172.21.15.32:0/2038813094 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4641f4bb0 con 0x7fc46406dc60 2024-04-24T22:59:19.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.423+0000 7fc46a638640 1 -- 172.21.15.32:0/2038813094 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc464068430 con 0x7fc46406dc60 2024-04-24T22:59:19.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.423+0000 7fc460ff9640 1 -- 172.21.15.32:0/2038813094 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc45401b070 con 0x7fc46406dc60 2024-04-24T22:59:19.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.424+0000 7fc460ff9640 1 --2- 172.21.15.32:0/2038813094 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc45006be30 0x7fc45006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:19.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.424+0000 7fc460ff9640 1 -- 172.21.15.32:0/2038813094 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc45408a060 con 0x7fc46406dc60 2024-04-24T22:59:19.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.424+0000 7fc463fff640 1 --2- 172.21.15.32:0/2038813094 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc45006be30 0x7fc45006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:19.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.424+0000 7fc463fff640 1 --2- 172.21.15.32:0/2038813094 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc45006be30 0x7fc45006e2f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc458034000 tx=0x7fc45802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:19.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.428+0000 7fc460ff9640 1 -- 172.21.15.32:0/2038813094 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc4641f4370 con 0x7fc46406dc60 2024-04-24T22:59:19.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.613+0000 7fc46a638640 1 -- 172.21.15.32:0/2038813094 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc46414f210 con 0x7fc46406dc60 2024-04-24T22:59:19.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.613+0000 7fc460ff9640 1 -- 172.21.15.32:0/2038813094 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7fc4540583c0 con 0x7fc46406dc60 2024-04-24T22:59:19.615 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-24T22:59:19.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.616+0000 7fc46a638640 1 -- 172.21.15.32:0/2038813094 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc45006be30 msgr2=0x7fc45006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:19.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.616+0000 7fc46a638640 1 --2- 172.21.15.32:0/2038813094 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc45006be30 0x7fc45006e2f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc458034000 tx=0x7fc45802e000 comp rx=0 tx=0).stop 2024-04-24T22:59:19.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.616+0000 7fc46a638640 1 -- 172.21.15.32:0/2038813094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc46406dc60 msgr2=0x7fc464146d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:19.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.616+0000 7fc46a638640 1 --2- 172.21.15.32:0/2038813094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc46406dc60 0x7fc464146d90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc45400c460 tx=0x7fc45400c920 comp rx=0 tx=0).stop 2024-04-24T22:59:19.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.616+0000 7fc46a638640 1 -- 172.21.15.32:0/2038813094 shutdown_connections 2024-04-24T22:59:19.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.616+0000 7fc46a638640 1 --2- 172.21.15.32:0/2038813094 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc45006be30 0x7fc45006e2f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:19.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.616+0000 7fc46a638640 1 --2- 172.21.15.32:0/2038813094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc46406dc60 0x7fc464146d90 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:19.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.616+0000 7fc46a638640 1 --2- 172.21.15.32:0/2038813094 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc464158dd0 0x7fc4641472d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:19.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.616+0000 7fc46a638640 1 --2- 172.21.15.32:0/2038813094 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc46406b860 0x7fc46414c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:19.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.616+0000 7fc46a638640 1 -- 172.21.15.32:0/2038813094 >> 172.21.15.32:0/2038813094 conn(0x7fc46406e800 msgr2=0x7fc464155220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:19.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.616+0000 7fc46a638640 1 -- 172.21.15.32:0/2038813094 shutdown_connections 2024-04-24T22:59:19.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.616+0000 7fc46a638640 1 -- 172.21.15.32:0/2038813094 wait complete. 2024-04-24T22:59:19.628 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T22:59:19.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.730+0000 7fe285238640 1 Processor -- start 2024-04-24T22:59:19.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.730+0000 7fe285238640 1 -- start start 2024-04-24T22:59:19.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.731+0000 7fe285238640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe280074210 0x7fe28006c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:19.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.731+0000 7fe285238640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe2800ae140 0x7fe2800b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:19.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.731+0000 7fe285238640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2800b0770 con 0x7fe2800b0b80 2024-04-24T22:59:19.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.731+0000 7fe285238640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fe2800b1c20 con 0x7fe280074210 2024-04-24T22:59:19.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.731+0000 7fe285238640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe2800b1010 con 0x7fe2800ae140 2024-04-24T22:59:19.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.731+0000 7fe27ed9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe280074210 0x7fe28006c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:19.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.731+0000 7fe27ed9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe280074210 0x7fe28006c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59712/0 (socket says 172.21.15.32:59712) 2024-04-24T22:59:19.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.731+0000 7fe27ed9d640 1 -- 172.21.15.32:0/3504160345 learned_addr learned my addr 172.21.15.32:0/3504160345 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:19.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.731+0000 7fe27ed9d640 1 -- 172.21.15.32:0/3504160345 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe2800ae140 msgr2=0x7fe2800b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:19.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.731+0000 7fe27f59e640 1 --2- 172.21.15.32:0/3504160345 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe2800ae140 0x7fe2800b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:19.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.731+0000 7fe27ed9d640 1 --2- 172.21.15.32:0/3504160345 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe2800ae140 0x7fe2800b20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:19.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.731+0000 7fe27ed9d640 1 -- 172.21.15.32:0/3504160345 >> v1:172.21.15.121:6789/0 conn(0x7fe2800b0b80 legacy=0x7fe2800745f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:19.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.731+0000 7fe27ed9d640 1 -- 172.21.15.32:0/3504160345 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2800b12f0 con 0x7fe280074210 2024-04-24T22:59:19.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.731+0000 7fe27f59e640 1 --2- 172.21.15.32:0/3504160345 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe2800ae140 0x7fe2800b20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:59:19.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.732+0000 7fe27ed9d640 1 --2- 172.21.15.32:0/3504160345 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe280074210 0x7fe28006c480 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe270009e50 tx=0x7fe27002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5b368933463313e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:19.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.732+0000 7fe27dd9b640 1 -- 172.21.15.32:0/3504160345 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe270036070 con 0x7fe280074210 2024-04-24T22:59:19.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.732+0000 7fe27dd9b640 1 -- 172.21.15.32:0/3504160345 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe270031600 con 0x7fe280074210 2024-04-24T22:59:19.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.732+0000 7fe27dd9b640 1 -- 172.21.15.32:0/3504160345 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2700319d0 con 0x7fe280074210 2024-04-24T22:59:19.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.733+0000 7fe285238640 1 -- 172.21.15.32:0/3504160345 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe280074210 msgr2=0x7fe28006c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:19.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.733+0000 7fe285238640 1 --2- 172.21.15.32:0/3504160345 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe280074210 0x7fe28006c480 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe270009e50 tx=0x7fe27002a710 comp rx=0 tx=0).stop 2024-04-24T22:59:19.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.733+0000 7fe285238640 1 -- 172.21.15.32:0/3504160345 shutdown_connections 2024-04-24T22:59:19.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.733+0000 7fe285238640 1 --2- 172.21.15.32:0/3504160345 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe2800ae140 0x7fe2800b20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:19.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.733+0000 7fe285238640 1 --2- 172.21.15.32:0/3504160345 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe280074210 0x7fe28006c480 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:19.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.733+0000 7fe285238640 1 -- 172.21.15.32:0/3504160345 >> 172.21.15.32:0/3504160345 conn(0x7fe2800b2e60 msgr2=0x7fe2800b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:19.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.733+0000 7fe285238640 1 -- 172.21.15.32:0/3504160345 shutdown_connections 2024-04-24T22:59:19.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.733+0000 7fe285238640 1 -- 172.21.15.32:0/3504160345 wait complete. 2024-04-24T22:59:19.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.734+0000 7fe285238640 1 Processor -- start 2024-04-24T22:59:19.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.734+0000 7fe285238640 1 -- start start 2024-04-24T22:59:19.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.735+0000 7fe285238640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe280074210 0x7fe28006f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:19.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.735+0000 7fe285238640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe2800ae140 0x7fe28015cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:19.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.735+0000 7fe285238640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2800ce380 0x7fe2800ca590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:19.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.735+0000 7fe285238640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe2800b3ba0 con 0x7fe280074210 2024-04-24T22:59:19.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.735+0000 7fe285238640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe2800b18b0 con 0x7fe2800ce380 2024-04-24T22:59:19.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.735+0000 7fe285238640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe2800b3d20 con 0x7fe2800ae140 2024-04-24T22:59:19.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.735+0000 7fe27ed9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe280074210 0x7fe28006f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:19.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.735+0000 7fe27ed9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe280074210 0x7fe28006f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59720/0 (socket says 172.21.15.32:59720) 2024-04-24T22:59:19.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.735+0000 7fe27ed9d640 1 -- 172.21.15.32:0/3091345508 learned_addr learned my addr 172.21.15.32:0/3091345508 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:19.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.735+0000 7fe27e59c640 1 --2- 172.21.15.32:0/3091345508 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe2800ae140 0x7fe28015cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:19.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.735+0000 7fe27f59e640 1 --2- 172.21.15.32:0/3091345508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2800ce380 0x7fe2800ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:19.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.735+0000 7fe27ed9d640 1 -- 172.21.15.32:0/3091345508 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe2800ae140 msgr2=0x7fe28015cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:19.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.735+0000 7fe27ed9d640 1 --2- 172.21.15.32:0/3091345508 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe2800ae140 0x7fe28015cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:19.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.735+0000 7fe27ed9d640 1 -- 172.21.15.32:0/3091345508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2800ce380 msgr2=0x7fe2800ca590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:19.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.735+0000 7fe27ed9d640 1 --2- 172.21.15.32:0/3091345508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2800ce380 0x7fe2800ca590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:19.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.735+0000 7fe27ed9d640 1 -- 172.21.15.32:0/3091345508 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe28015ec10 con 0x7fe280074210 2024-04-24T22:59:19.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.736+0000 7fe27ed9d640 1 --2- 172.21.15.32:0/3091345508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe280074210 0x7fe28006f050 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe270034040 tx=0x7fe270004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:19.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.736+0000 7fe2777fe640 1 -- 172.21.15.32:0/3091345508 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe270036050 con 0x7fe280074210 2024-04-24T22:59:19.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.736+0000 7fe2777fe640 1 -- 172.21.15.32:0/3091345508 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe270042070 con 0x7fe280074210 2024-04-24T22:59:19.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.736+0000 7fe2777fe640 1 -- 172.21.15.32:0/3091345508 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe270009510 con 0x7fe280074210 2024-04-24T22:59:19.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.736+0000 7fe285238640 1 -- 172.21.15.32:0/3091345508 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe28015eeb0 con 0x7fe280074210 2024-04-24T22:59:19.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.736+0000 7fe285238640 1 -- 172.21.15.32:0/3091345508 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe28015f470 con 0x7fe280074210 2024-04-24T22:59:19.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.737+0000 7fe285238640 1 -- 172.21.15.32:0/3091345508 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe280068430 con 0x7fe280074210 2024-04-24T22:59:19.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.737+0000 7fe2777fe640 1 -- 172.21.15.32:0/3091345508 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe270007c50 con 0x7fe280074210 2024-04-24T22:59:19.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.738+0000 7fe2777fe640 1 --2- 172.21.15.32:0/3091345508 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe26006be30 0x7fe26006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:19.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.738+0000 7fe2777fe640 1 -- 172.21.15.32:0/3091345508 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe2700abd30 con 0x7fe280074210 2024-04-24T22:59:19.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.738+0000 7fe27e59c640 1 --2- 172.21.15.32:0/3091345508 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe26006be30 0x7fe26006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:19.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.738+0000 7fe27e59c640 1 --2- 172.21.15.32:0/3091345508 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe26006be30 0x7fe26006e2f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe28006f590 tx=0x7fe268009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:19.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.742+0000 7fe2777fe640 1 -- 172.21.15.32:0/3091345508 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe270079ff0 con 0x7fe280074210 2024-04-24T22:59:19.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.925+0000 7fe285238640 1 -- 172.21.15.32:0/3091345508 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe2800d20e0 con 0x7fe280074210 2024-04-24T22:59:19.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.926+0000 7fe2777fe640 1 -- 172.21.15.32:0/3091345508 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7fe27007e2f0 con 0x7fe280074210 2024-04-24T22:59:19.927 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-24T22:59:19.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.927+0000 7fe285238640 1 -- 172.21.15.32:0/3091345508 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe26006be30 msgr2=0x7fe26006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:19.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.927+0000 7fe285238640 1 --2- 172.21.15.32:0/3091345508 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe26006be30 0x7fe26006e2f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe28006f590 tx=0x7fe268009040 comp rx=0 tx=0).stop 2024-04-24T22:59:19.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.927+0000 7fe285238640 1 -- 172.21.15.32:0/3091345508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe280074210 msgr2=0x7fe28006f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:19.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.927+0000 7fe285238640 1 --2- 172.21.15.32:0/3091345508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe280074210 0x7fe28006f050 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe270034040 tx=0x7fe270004a40 comp rx=0 tx=0).stop 2024-04-24T22:59:19.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.928+0000 7fe285238640 1 -- 172.21.15.32:0/3091345508 shutdown_connections 2024-04-24T22:59:19.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.928+0000 7fe285238640 1 --2- 172.21.15.32:0/3091345508 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe26006be30 0x7fe26006e2f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:19.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.928+0000 7fe285238640 1 --2- 172.21.15.32:0/3091345508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2800ce380 0x7fe2800ca590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:19.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.928+0000 7fe285238640 1 --2- 172.21.15.32:0/3091345508 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe2800ae140 0x7fe28015cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:19.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.928+0000 7fe285238640 1 --2- 172.21.15.32:0/3091345508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe280074210 0x7fe28006f050 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:19.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.928+0000 7fe285238640 1 -- 172.21.15.32:0/3091345508 >> 172.21.15.32:0/3091345508 conn(0x7fe2800b2e60 msgr2=0x7fe2800b69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:19.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.928+0000 7fe285238640 1 -- 172.21.15.32:0/3091345508 shutdown_connections 2024-04-24T22:59:19.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:19.928+0000 7fe285238640 1 -- 172.21.15.32:0/3091345508 wait complete. 2024-04-24T22:59:19.941 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T22:59:20.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.040+0000 7f016f9c1640 1 Processor -- start 2024-04-24T22:59:20.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.040+0000 7f016f9c1640 1 -- start start 2024-04-24T22:59:20.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.041+0000 7f016f9c1640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f016806b820 0x7f01680d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:20.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.041+0000 7f016f9c1640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f01680d4210 0x7f01680b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:20.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.041+0000 7f016f9c1640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01680d3170 con 0x7f016815e9b0 2024-04-24T22:59:20.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.041+0000 7f016f9c1640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f01680d3450 con 0x7f01680d4210 2024-04-24T22:59:20.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.041+0000 7f016f9c1640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f01680d3730 con 0x7f016806b820 2024-04-24T22:59:20.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.041+0000 7f016df5e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f01680d4210 0x7f01680b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:20.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.041+0000 7f016df5e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f01680d4210 0x7f01680b78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59726/0 (socket says 172.21.15.32:59726) 2024-04-24T22:59:20.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.041+0000 7f016d75d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f016815e9b0 0x7f016806b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47728/0 (socket says 172.21.15.32:47728) 2024-04-24T22:59:20.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.041+0000 7f016df5e640 1 -- 172.21.15.32:0/4074303837 learned_addr learned my addr 172.21.15.32:0/4074303837 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:20.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.042+0000 7f016cf5c640 1 --2- 172.21.15.32:0/4074303837 >> v2:172.21.15.121:3300/0 conn(0x7f016806b820 0x7f01680d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:20.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.042+0000 7f016df5e640 1 -- 172.21.15.32:0/4074303837 >> v2:172.21.15.121:3300/0 conn(0x7f016806b820 msgr2=0x7f01680d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:20.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.042+0000 7f016df5e640 1 --2- 172.21.15.32:0/4074303837 >> v2:172.21.15.121:3300/0 conn(0x7f016806b820 0x7f01680d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:20.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.042+0000 7f016df5e640 1 -- 172.21.15.32:0/4074303837 >> v1:172.21.15.32:6789/0 conn(0x7f016815e9b0 legacy=0x7f016806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:20.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.042+0000 7f0167fff640 1 -- 172.21.15.32:0/4074303837 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 589164685 0 0) 0x7f01680d3170 con 0x7f016815e9b0 2024-04-24T22:59:20.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.042+0000 7f016df5e640 1 -- 172.21.15.32:0/4074303837 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01680d3a10 con 0x7f01680d4210 2024-04-24T22:59:20.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.042+0000 7f016cf5c640 1 --2- 172.21.15.32:0/4074303837 >> v2:172.21.15.121:3300/0 conn(0x7f016806b820 0x7f01680d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:20.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.042+0000 7f016df5e640 1 --2- 172.21.15.32:0/4074303837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01680d4210 0x7f01680b78f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f01680730f0 tx=0x7f015802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f065628c5ec02ad6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:20.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.043+0000 7f0167fff640 1 -- 172.21.15.32:0/4074303837 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0158036070 con 0x7f01680d4210 2024-04-24T22:59:20.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.043+0000 7f0167fff640 1 -- 172.21.15.32:0/4074303837 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0158031600 con 0x7f01680d4210 2024-04-24T22:59:20.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.043+0000 7f0167fff640 1 -- 172.21.15.32:0/4074303837 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01580319d0 con 0x7f01680d4210 2024-04-24T22:59:20.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.043+0000 7f016f9c1640 1 -- 172.21.15.32:0/4074303837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01680d4210 msgr2=0x7f01680b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:20.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.043+0000 7f016f9c1640 1 --2- 172.21.15.32:0/4074303837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01680d4210 0x7f01680b78f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f01680730f0 tx=0x7f015802a6a0 comp rx=0 tx=0).stop 2024-04-24T22:59:20.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.043+0000 7f016f9c1640 1 -- 172.21.15.32:0/4074303837 shutdown_connections 2024-04-24T22:59:20.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.043+0000 7f016f9c1640 1 --2- 172.21.15.32:0/4074303837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01680d4210 0x7f01680b78f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:20.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.043+0000 7f016f9c1640 1 --2- 172.21.15.32:0/4074303837 >> v2:172.21.15.121:3300/0 conn(0x7f016806b820 0x7f01680d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:20.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.043+0000 7f016f9c1640 1 -- 172.21.15.32:0/4074303837 >> 172.21.15.32:0/4074303837 conn(0x7f01680d11d0 msgr2=0x7f01680d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:20.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.043+0000 7f016f9c1640 1 -- 172.21.15.32:0/4074303837 shutdown_connections 2024-04-24T22:59:20.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.043+0000 7f016f9c1640 1 -- 172.21.15.32:0/4074303837 wait complete. 2024-04-24T22:59:20.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.044+0000 7f016f9c1640 1 Processor -- start 2024-04-24T22:59:20.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.044+0000 7f016f9c1640 1 -- start start 2024-04-24T22:59:20.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.045+0000 7f016f9c1640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f016806b820 0x7f01681f4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:20.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.045+0000 7f016f9c1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01680d4210 0x7f01680cd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:20.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.045+0000 7f016f9c1640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f016815e9b0 0x7f01680cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:20.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.045+0000 7f016f9c1640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f01680ba0e0 con 0x7f01680d4210 2024-04-24T22:59:20.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.045+0000 7f016f9c1640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f01680d2400 con 0x7f016806b820 2024-04-24T22:59:20.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.045+0000 7f016f9c1640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f01680ba260 con 0x7f016815e9b0 2024-04-24T22:59:20.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.045+0000 7f016cf5c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01680d4210 0x7f01680cd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:20.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.045+0000 7f016cf5c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01680d4210 0x7f01680cd9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59732/0 (socket says 172.21.15.32:59732) 2024-04-24T22:59:20.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.045+0000 7f016cf5c640 1 -- 172.21.15.32:0/451034217 learned_addr learned my addr 172.21.15.32:0/451034217 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:20.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.045+0000 7f016d75d640 1 --2- 172.21.15.32:0/451034217 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f016806b820 0x7f01681f4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:20.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.045+0000 7f016cf5c640 1 -- 172.21.15.32:0/451034217 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f016815e9b0 msgr2=0x7f01680cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:20.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.045+0000 7f016cf5c640 1 --2- 172.21.15.32:0/451034217 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f016815e9b0 0x7f01680cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:20.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.045+0000 7f016cf5c640 1 -- 172.21.15.32:0/451034217 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f016806b820 msgr2=0x7f01681f4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:20.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.045+0000 7f016cf5c640 1 --2- 172.21.15.32:0/451034217 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f016806b820 0x7f01681f4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:20.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.045+0000 7f016cf5c640 1 -- 172.21.15.32:0/451034217 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01681485b0 con 0x7f01680d4210 2024-04-24T22:59:20.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.046+0000 7f016cf5c640 1 --2- 172.21.15.32:0/451034217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01680d4210 0x7f01680cd9f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f015c00e7e0 tx=0x7f015c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:20.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.046+0000 7f0165ffb640 1 -- 172.21.15.32:0/451034217 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015c00f040 con 0x7f01680d4210 2024-04-24T22:59:20.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.046+0000 7f0165ffb640 1 -- 172.21.15.32:0/451034217 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f015c004030 con 0x7f01680d4210 2024-04-24T22:59:20.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.046+0000 7f016f9c1640 1 -- 172.21.15.32:0/451034217 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0168148890 con 0x7f01680d4210 2024-04-24T22:59:20.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.046+0000 7f0165ffb640 1 -- 172.21.15.32:0/451034217 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015c004400 con 0x7f01680d4210 2024-04-24T22:59:20.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.046+0000 7f016f9c1640 1 -- 172.21.15.32:0/451034217 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0168148e50 con 0x7f01680d4210 2024-04-24T22:59:20.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.047+0000 7f016f9c1640 1 -- 172.21.15.32:0/451034217 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0168068430 con 0x7f01680d4210 2024-04-24T22:59:20.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.047+0000 7f0165ffb640 1 -- 172.21.15.32:0/451034217 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f015c00c6d0 con 0x7f01680d4210 2024-04-24T22:59:20.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.047+0000 7f0165ffb640 1 --2- 172.21.15.32:0/451034217 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f015406be30 0x7f015406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:20.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.048+0000 7f0165ffb640 1 -- 172.21.15.32:0/451034217 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f015c08ac00 con 0x7f01680d4210 2024-04-24T22:59:20.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.048+0000 7f016d75d640 1 --2- 172.21.15.32:0/451034217 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f015406be30 0x7f015406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:20.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.048+0000 7f016d75d640 1 --2- 172.21.15.32:0/451034217 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f015406be30 0x7f015406e2f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0160000f60 tx=0x7f0160009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:20.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.052+0000 7f0165ffb640 1 -- 172.21.15.32:0/451034217 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f015c091080 con 0x7f01680d4210 2024-04-24T22:59:20.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.271+0000 7f016f9c1640 1 -- 172.21.15.32:0/451034217 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0168155fe0 con 0x7f01680d4210 2024-04-24T22:59:20.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.272+0000 7f0165ffb640 1 -- 172.21.15.32:0/451034217 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5178 (secure 0 0 0) 0x7f015c017020 con 0x7f01680d4210 2024-04-24T22:59:20.273 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:59:20.274 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1485884060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1485884060},{"type":"v1","addr":"172.21.15.121:6837","nonce":1485884060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/923934104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":923934104},{"type":"v1","addr":"172.21.15.32:6833","nonce":923934104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3229115835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3229115835},{"type":"v1","addr":"172.21.15.32:6835","nonce":3229115835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T22:59:03.876305+0000","modified":"2024-04-24T22:59:05.932147+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6836/2644625343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2644625343},{"type":"v1","addr":"172.21.15.121:6836","nonce":2644625343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4276]},"id":1}]} 2024-04-24T22:59:20.274 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 6 2024-04-24T22:59:20.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.274+0000 7f016f9c1640 1 -- 172.21.15.32:0/451034217 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f015406be30 msgr2=0x7f015406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:20.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.274+0000 7f016f9c1640 1 --2- 172.21.15.32:0/451034217 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f015406be30 0x7f015406e2f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0160000f60 tx=0x7f0160009040 comp rx=0 tx=0).stop 2024-04-24T22:59:20.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.274+0000 7f016f9c1640 1 -- 172.21.15.32:0/451034217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01680d4210 msgr2=0x7f01680cd9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:20.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.274+0000 7f016f9c1640 1 --2- 172.21.15.32:0/451034217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01680d4210 0x7f01680cd9f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f015c00e7e0 tx=0x7f015c00eca0 comp rx=0 tx=0).stop 2024-04-24T22:59:20.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.274+0000 7f016f9c1640 1 -- 172.21.15.32:0/451034217 shutdown_connections 2024-04-24T22:59:20.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.274+0000 7f016f9c1640 1 --2- 172.21.15.32:0/451034217 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f015406be30 0x7f015406e2f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:20.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.274+0000 7f016f9c1640 1 --2- 172.21.15.32:0/451034217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01680d4210 0x7f01680cd9f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:20.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.274+0000 7f016f9c1640 1 --2- 172.21.15.32:0/451034217 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f016815e9b0 0x7f01680cdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:20.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.274+0000 7f016f9c1640 1 --2- 172.21.15.32:0/451034217 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f016806b820 0x7f01681f4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:20.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.274+0000 7f016f9c1640 1 -- 172.21.15.32:0/451034217 >> 172.21.15.32:0/451034217 conn(0x7f01680d11d0 msgr2=0x7f01680b8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:20.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.274+0000 7f016f9c1640 1 -- 172.21.15.32:0/451034217 shutdown_connections 2024-04-24T22:59:20.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.274+0000 7f016f9c1640 1 -- 172.21.15.32:0/451034217 wait complete. 2024-04-24T22:59:20.286 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:59:20.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.385+0000 7f8f561f9640 1 Processor -- start 2024-04-24T22:59:20.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.385+0000 7f8f561f9640 1 -- start start 2024-04-24T22:59:20.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.386+0000 7f8f561f9640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8f5006b820 0x7f8f500d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:20.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.386+0000 7f8f561f9640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8f500d4210 0x7f8f500b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:20.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.386+0000 7f8f561f9640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f500d3170 con 0x7f8f5015e9b0 2024-04-24T22:59:20.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.386+0000 7f8f561f9640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f8f500d3450 con 0x7f8f500d4210 2024-04-24T22:59:20.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.386+0000 7f8f561f9640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f8f500d3730 con 0x7f8f5006b820 2024-04-24T22:59:20.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.386+0000 7f8f4f7fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8f5015e9b0 0x7f8f5006b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47740/0 (socket says 172.21.15.32:47740) 2024-04-24T22:59:20.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.386+0000 7f8f4f7fe640 1 -- 172.21.15.32:0/1963236019 learned_addr learned my addr 172.21.15.32:0/1963236019 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:20.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.387+0000 7f8f4ffff640 1 --2- 172.21.15.32:0/1963236019 >> v2:172.21.15.121:3300/0 conn(0x7f8f500d4210 0x7f8f500b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:20.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.387+0000 7f8f4e7fc640 1 -- 172.21.15.32:0/1963236019 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2216094186 0 0) 0x7f8f500d3170 con 0x7f8f5015e9b0 2024-04-24T22:59:20.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.387+0000 7f8f4effd640 1 --2- 172.21.15.32:0/1963236019 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8f5006b820 0x7f8f500d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:20.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.387+0000 7f8f4e7fc640 1 -- 172.21.15.32:0/1963236019 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f38003520 con 0x7f8f5015e9b0 2024-04-24T22:59:20.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.387+0000 7f8f4e7fc640 1 -- 172.21.15.32:0/1963236019 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1752643562 0 0) 0x7f8f38003520 con 0x7f8f5015e9b0 2024-04-24T22:59:20.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.387+0000 7f8f4e7fc640 1 -- 172.21.15.32:0/1963236019 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8f5006b820 msgr2=0x7f8f500d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:20.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.387+0000 7f8f4e7fc640 1 --2- 172.21.15.32:0/1963236019 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8f5006b820 0x7f8f500d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:20.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.387+0000 7f8f4e7fc640 1 -- 172.21.15.32:0/1963236019 >> v2:172.21.15.121:3300/0 conn(0x7f8f500d4210 msgr2=0x7f8f500b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:20.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.387+0000 7f8f4e7fc640 1 --2- 172.21.15.32:0/1963236019 >> v2:172.21.15.121:3300/0 conn(0x7f8f500d4210 0x7f8f500b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:20.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.387+0000 7f8f4e7fc640 1 -- 172.21.15.32:0/1963236019 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f500d3a10 con 0x7f8f5015e9b0 2024-04-24T22:59:20.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.387+0000 7f8f4e7fc640 1 -- 172.21.15.32:0/1963236019 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f8f40001670 con 0x7f8f5015e9b0 2024-04-24T22:59:20.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.387+0000 7f8f4e7fc640 1 -- 172.21.15.32:0/1963236019 >> v1:172.21.15.32:6789/0 conn(0x7f8f5015e9b0 legacy=0x7f8f5006b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:20.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.387+0000 7f8f4e7fc640 1 --2- 172.21.15.32:0/1963236019 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8f5006b820 0x7f8f380039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:20.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.387+0000 7f8f4e7fc640 1 --2- 172.21.15.32:0/1963236019 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f38003f30 0x7f8f38004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:20.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.387+0000 7f8f4e7fc640 1 --2- 172.21.15.32:0/1963236019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f38004d40 0x7f8f380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:20.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.387+0000 7f8f4e7fc640 1 -- 172.21.15.32:0/1963236019 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8f500d3450 con 0x7f8f38004d40 2024-04-24T22:59:20.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.387+0000 7f8f4e7fc640 1 -- 172.21.15.32:0/1963236019 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8f500d3730 con 0x7f8f38003f30 2024-04-24T22:59:20.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.387+0000 7f8f4e7fc640 1 -- 172.21.15.32:0/1963236019 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f8f3800b9c0 con 0x7f8f5006b820 2024-04-24T22:59:20.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.387+0000 7f8f4ffff640 1 --2- 172.21.15.32:0/1963236019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f38004d40 0x7f8f380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:20.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.387+0000 7f8f4f7fe640 1 --2- 172.21.15.32:0/1963236019 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8f5006b820 0x7f8f380039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:20.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.387+0000 7f8f4effd640 1 --2- 172.21.15.32:0/1963236019 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f38003f30 0x7f8f38004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:20.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.387+0000 7f8f4ffff640 1 -- 172.21.15.32:0/1963236019 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8f5006b820 msgr2=0x7f8f380039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:20.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.387+0000 7f8f4ffff640 1 --2- 172.21.15.32:0/1963236019 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8f5006b820 0x7f8f380039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:20.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.387+0000 7f8f4ffff640 1 -- 172.21.15.32:0/1963236019 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f38003f30 msgr2=0x7f8f38004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:20.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.387+0000 7f8f4ffff640 1 --2- 172.21.15.32:0/1963236019 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f38003f30 0x7f8f38004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:20.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.387+0000 7f8f4ffff640 1 -- 172.21.15.32:0/1963236019 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f38003520 con 0x7f8f38004d40 2024-04-24T22:59:20.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.388+0000 7f8f4ffff640 1 --2- 172.21.15.32:0/1963236019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f38004d40 0x7f8f380051c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8f5006bdf0 tx=0x7f8f38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e4a6d4ff77814b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:20.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.388+0000 7f8f4e7fc640 1 -- 172.21.15.32:0/1963236019 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f48009c10 con 0x7f8f38004d40 2024-04-24T22:59:20.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.388+0000 7f8f4e7fc640 1 -- 172.21.15.32:0/1963236019 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f48036070 con 0x7f8f38004d40 2024-04-24T22:59:20.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.388+0000 7f8f4e7fc640 1 -- 172.21.15.32:0/1963236019 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f48031670 con 0x7f8f38004d40 2024-04-24T22:59:20.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.389+0000 7f8f561f9640 1 -- 172.21.15.32:0/1963236019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f38004d40 msgr2=0x7f8f380051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:20.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.389+0000 7f8f561f9640 1 --2- 172.21.15.32:0/1963236019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f38004d40 0x7f8f380051c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8f5006bdf0 tx=0x7f8f38005830 comp rx=0 tx=0).stop 2024-04-24T22:59:20.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.389+0000 7f8f4ffff640 1 -- 172.21.15.32:0/1963236019 reap_dead start 2024-04-24T22:59:20.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.389+0000 7f8f561f9640 1 -- 172.21.15.32:0/1963236019 shutdown_connections 2024-04-24T22:59:20.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.389+0000 7f8f561f9640 1 -- 172.21.15.32:0/1963236019 >> 172.21.15.32:0/1963236019 conn(0x7f8f500d11d0 msgr2=0x7f8f500d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:20.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.389+0000 7f8f561f9640 1 -- 172.21.15.32:0/1963236019 shutdown_connections 2024-04-24T22:59:20.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.389+0000 7f8f561f9640 1 -- 172.21.15.32:0/1963236019 wait complete. 2024-04-24T22:59:20.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.390+0000 7f8f561f9640 1 Processor -- start 2024-04-24T22:59:20.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.390+0000 7f8f561f9640 1 -- start start 2024-04-24T22:59:20.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.391+0000 7f8f561f9640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8f38004d40 0x7f8f50160c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:20.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.391+0000 7f8f561f9640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f50153e00 0x7f8f5015d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:20.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.391+0000 7f8f561f9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f5015d590 0x7f8f5015ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:20.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.391+0000 7f8f561f9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8f500ba0e0 con 0x7f8f5015d590 2024-04-24T22:59:20.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.391+0000 7f8f561f9640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8f500d2400 con 0x7f8f50153e00 2024-04-24T22:59:20.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.391+0000 7f8f561f9640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f8f500ba260 con 0x7f8f38004d40 2024-04-24T22:59:20.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.391+0000 7f8f4ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f5015d590 0x7f8f5015ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:20.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.391+0000 7f8f4ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f5015d590 0x7f8f5015ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59742/0 (socket says 172.21.15.32:59742) 2024-04-24T22:59:20.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.391+0000 7f8f4ffff640 1 -- 172.21.15.32:0/3364803605 learned_addr learned my addr 172.21.15.32:0/3364803605 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:20.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.391+0000 7f8f4effd640 1 --2- 172.21.15.32:0/3364803605 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f50153e00 0x7f8f5015d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:20.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.391+0000 7f8f4f7fe640 1 --2- 172.21.15.32:0/3364803605 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8f38004d40 0x7f8f50160c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:20.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.391+0000 7f8f4ffff640 1 -- 172.21.15.32:0/3364803605 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8f38004d40 msgr2=0x7f8f50160c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:20.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.391+0000 7f8f4ffff640 1 --2- 172.21.15.32:0/3364803605 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8f38004d40 0x7f8f50160c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:20.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.391+0000 7f8f4ffff640 1 -- 172.21.15.32:0/3364803605 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f50153e00 msgr2=0x7f8f5015d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:20.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.391+0000 7f8f4ffff640 1 --2- 172.21.15.32:0/3364803605 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f50153e00 0x7f8f5015d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:20.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.391+0000 7f8f4ffff640 1 -- 172.21.15.32:0/3364803605 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f5015fb70 con 0x7f8f5015d590 2024-04-24T22:59:20.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.391+0000 7f8f4effd640 1 --2- 172.21.15.32:0/3364803605 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f50153e00 0x7f8f5015d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:59:20.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.392+0000 7f8f4ffff640 1 --2- 172.21.15.32:0/3364803605 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f5015d590 0x7f8f5015ee00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8f4800e230 tx=0x7f8f48009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:20.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.392+0000 7f8f37fff640 1 -- 172.21.15.32:0/3364803605 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f48009c10 con 0x7f8f5015d590 2024-04-24T22:59:20.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.392+0000 7f8f37fff640 1 -- 172.21.15.32:0/3364803605 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f48031db0 con 0x7f8f5015d590 2024-04-24T22:59:20.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.392+0000 7f8f37fff640 1 -- 172.21.15.32:0/3364803605 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f48003bd0 con 0x7f8f5015d590 2024-04-24T22:59:20.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.392+0000 7f8f561f9640 1 -- 172.21.15.32:0/3364803605 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f5015fe50 con 0x7f8f5015d590 2024-04-24T22:59:20.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.392+0000 7f8f561f9640 1 -- 172.21.15.32:0/3364803605 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f50160410 con 0x7f8f5015d590 2024-04-24T22:59:20.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.393+0000 7f8f561f9640 1 -- 172.21.15.32:0/3364803605 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f50068430 con 0x7f8f5015d590 2024-04-24T22:59:20.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.393+0000 7f8f37fff640 1 -- 172.21.15.32:0/3364803605 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8f48005d10 con 0x7f8f5015d590 2024-04-24T22:59:20.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.394+0000 7f8f37fff640 1 --2- 172.21.15.32:0/3364803605 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8f3c06be30 0x7f8f3c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:20.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.394+0000 7f8f37fff640 1 -- 172.21.15.32:0/3364803605 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8f480ac1d0 con 0x7f8f5015d590 2024-04-24T22:59:20.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.394+0000 7f8f4f7fe640 1 --2- 172.21.15.32:0/3364803605 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8f3c06be30 0x7f8f3c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:20.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.394+0000 7f8f4f7fe640 1 --2- 172.21.15.32:0/3364803605 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8f3c06be30 0x7f8f3c06e2f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8f50161fc0 tx=0x7f8f40009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:20.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.398+0000 7f8f37fff640 1 -- 172.21.15.32:0/3364803605 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8f48049060 con 0x7f8f5015d590 2024-04-24T22:59:20.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.583+0000 7f8f561f9640 1 -- 172.21.15.32:0/3364803605 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8f50072a00 con 0x7f8f5015d590 2024-04-24T22:59:20.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.584+0000 7f8f37fff640 1 -- 172.21.15.32:0/3364803605 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16621 (secure 0 0 0) 0x7f8f4807ab80 con 0x7f8f5015d590 2024-04-24T22:59:20.586 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:59:20.586 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T22:59:05.517554+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:59:02.016639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T22:59:02.846087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:59:20.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.586+0000 7f8f561f9640 1 -- 172.21.15.32:0/3364803605 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8f3c06be30 msgr2=0x7f8f3c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:20.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.586+0000 7f8f561f9640 1 --2- 172.21.15.32:0/3364803605 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8f3c06be30 0x7f8f3c06e2f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8f50161fc0 tx=0x7f8f40009040 comp rx=0 tx=0).stop 2024-04-24T22:59:20.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.586+0000 7f8f561f9640 1 -- 172.21.15.32:0/3364803605 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f5015d590 msgr2=0x7f8f5015ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:20.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.586+0000 7f8f561f9640 1 --2- 172.21.15.32:0/3364803605 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f5015d590 0x7f8f5015ee00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8f4800e230 tx=0x7f8f48009750 comp rx=0 tx=0).stop 2024-04-24T22:59:20.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.586+0000 7f8f561f9640 1 -- 172.21.15.32:0/3364803605 shutdown_connections 2024-04-24T22:59:20.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.586+0000 7f8f561f9640 1 --2- 172.21.15.32:0/3364803605 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8f3c06be30 0x7f8f3c06e2f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:20.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.586+0000 7f8f561f9640 1 --2- 172.21.15.32:0/3364803605 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f5015d590 0x7f8f5015ee00 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:20.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.586+0000 7f8f561f9640 1 --2- 172.21.15.32:0/3364803605 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f50153e00 0x7f8f5015d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:20.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.586+0000 7f8f561f9640 1 --2- 172.21.15.32:0/3364803605 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8f38004d40 0x7f8f50160c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:20.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.587+0000 7f8f561f9640 1 -- 172.21.15.32:0/3364803605 >> 172.21.15.32:0/3364803605 conn(0x7f8f500d11d0 msgr2=0x7f8f500ca290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:20.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.587+0000 7f8f561f9640 1 -- 172.21.15.32:0/3364803605 shutdown_connections 2024-04-24T22:59:20.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.587+0000 7f8f561f9640 1 -- 172.21.15.32:0/3364803605 wait complete. 2024-04-24T22:59:20.599 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T22:59:20.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.698+0000 7f8075402640 1 Processor -- start 2024-04-24T22:59:20.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.698+0000 7f8075402640 1 -- start start 2024-04-24T22:59:20.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.699+0000 7f8075402640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f80700b1ea0 0x7f80700cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:20.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.699+0000 7f8075402640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8070070c30 0x7f80700b2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:20.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.699+0000 7f8075402640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8070071070 con 0x7f80700ccbf0 2024-04-24T22:59:20.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.699+0000 7f8075402640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f80700737b0 con 0x7f8070070c30 2024-04-24T22:59:20.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.699+0000 7f8075402640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f80700b27c0 con 0x7f80700b1ea0 2024-04-24T22:59:20.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.699+0000 7f806e7fc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8070070c30 0x7f80700b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:20.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.699+0000 7f806f7fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f80700ccbf0 0x7f80700cd040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47742/0 (socket says 172.21.15.32:47742) 2024-04-24T22:59:20.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.699+0000 7f806f7fe640 1 -- 172.21.15.32:0/46662648 learned_addr learned my addr 172.21.15.32:0/46662648 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:20.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.699+0000 7f806effd640 1 --2- 172.21.15.32:0/46662648 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f80700b1ea0 0x7f80700cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:20.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.699+0000 7f806dffb640 1 -- 172.21.15.32:0/46662648 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1004658243 0 0) 0x7f8070071070 con 0x7f80700ccbf0 2024-04-24T22:59:20.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.699+0000 7f806dffb640 1 -- 172.21.15.32:0/46662648 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f805c003520 con 0x7f80700ccbf0 2024-04-24T22:59:20.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.699+0000 7f806e7fc640 1 -- 172.21.15.32:0/46662648 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f80700b1ea0 msgr2=0x7f80700cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:20.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.699+0000 7f806e7fc640 1 --2- 172.21.15.32:0/46662648 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f80700b1ea0 0x7f80700cfc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:20.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.699+0000 7f806e7fc640 1 -- 172.21.15.32:0/46662648 >> v1:172.21.15.32:6789/0 conn(0x7f80700ccbf0 legacy=0x7f80700cd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:20.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.699+0000 7f806e7fc640 1 -- 172.21.15.32:0/46662648 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80700cf3e0 con 0x7f8070070c30 2024-04-24T22:59:20.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.700+0000 7f806e7fc640 1 --2- 172.21.15.32:0/46662648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8070070c30 0x7f80700b2280 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f80580097a0 tx=0x7f805802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bdea4fd7cc9c804 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:20.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.700+0000 7f806dffb640 1 -- 172.21.15.32:0/46662648 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8058037070 con 0x7f8070070c30 2024-04-24T22:59:20.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.700+0000 7f806dffb640 1 -- 172.21.15.32:0/46662648 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8058032440 con 0x7f8070070c30 2024-04-24T22:59:20.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.700+0000 7f806dffb640 1 -- 172.21.15.32:0/46662648 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8058032810 con 0x7f8070070c30 2024-04-24T22:59:20.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.701+0000 7f8075402640 1 -- 172.21.15.32:0/46662648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8070070c30 msgr2=0x7f80700b2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:20.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.701+0000 7f8075402640 1 --2- 172.21.15.32:0/46662648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8070070c30 0x7f80700b2280 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f80580097a0 tx=0x7f805802a180 comp rx=0 tx=0).stop 2024-04-24T22:59:20.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.701+0000 7f8075402640 1 -- 172.21.15.32:0/46662648 shutdown_connections 2024-04-24T22:59:20.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.701+0000 7f8075402640 1 --2- 172.21.15.32:0/46662648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8070070c30 0x7f80700b2280 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:20.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.701+0000 7f8075402640 1 --2- 172.21.15.32:0/46662648 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f80700b1ea0 0x7f80700cfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:20.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.701+0000 7f8075402640 1 -- 172.21.15.32:0/46662648 >> 172.21.15.32:0/46662648 conn(0x7f8070074840 msgr2=0x7f807006b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:20.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.701+0000 7f8075402640 1 -- 172.21.15.32:0/46662648 shutdown_connections 2024-04-24T22:59:20.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.701+0000 7f8075402640 1 -- 172.21.15.32:0/46662648 wait complete. 2024-04-24T22:59:20.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.702+0000 7f8075402640 1 Processor -- start 2024-04-24T22:59:20.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.702+0000 7f8075402640 1 -- start start 2024-04-24T22:59:20.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.703+0000 7f8075402640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8070070c30 0x7f807013da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:20.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.703+0000 7f8075402640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80700b1ea0 0x7f8070144960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:20.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.703+0000 7f8075402640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80700ccbf0 0x7f8070144ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:20.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.703+0000 7f8075402640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f807015d140 con 0x7f80700ccbf0 2024-04-24T22:59:20.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.703+0000 7f8075402640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f807015ce60 con 0x7f80700b1ea0 2024-04-24T22:59:20.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.703+0000 7f8075402640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f807015d420 con 0x7f8070070c30 2024-04-24T22:59:20.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.703+0000 7f806f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80700ccbf0 0x7f8070144ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:20.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.703+0000 7f806f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80700ccbf0 0x7f8070144ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59754/0 (socket says 172.21.15.32:59754) 2024-04-24T22:59:20.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.703+0000 7f806f7fe640 1 -- 172.21.15.32:0/3051376921 learned_addr learned my addr 172.21.15.32:0/3051376921 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:20.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.703+0000 7f806effd640 1 --2- 172.21.15.32:0/3051376921 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8070070c30 0x7f807013da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:20.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.703+0000 7f806e7fc640 1 --2- 172.21.15.32:0/3051376921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80700b1ea0 0x7f8070144960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:20.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.703+0000 7f806f7fe640 1 -- 172.21.15.32:0/3051376921 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8070070c30 msgr2=0x7f807013da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:20.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.703+0000 7f806f7fe640 1 --2- 172.21.15.32:0/3051376921 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8070070c30 0x7f807013da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:20.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.703+0000 7f806f7fe640 1 -- 172.21.15.32:0/3051376921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80700b1ea0 msgr2=0x7f8070144960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:20.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.703+0000 7f806f7fe640 1 --2- 172.21.15.32:0/3051376921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80700b1ea0 0x7f8070144960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:20.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.703+0000 7f806f7fe640 1 -- 172.21.15.32:0/3051376921 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80700b4490 con 0x7f80700ccbf0 2024-04-24T22:59:20.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.703+0000 7f806effd640 1 --2- 172.21.15.32:0/3051376921 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8070070c30 0x7f807013da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:20.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.703+0000 7f806f7fe640 1 --2- 172.21.15.32:0/3051376921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80700ccbf0 0x7f8070144ea0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f806000caa0 tx=0x7f806000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:20.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.704+0000 7f80677fe640 1 -- 172.21.15.32:0/3051376921 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8060017070 con 0x7f80700ccbf0 2024-04-24T22:59:20.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.704+0000 7f80677fe640 1 -- 172.21.15.32:0/3051376921 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f806000ac00 con 0x7f80700ccbf0 2024-04-24T22:59:20.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.704+0000 7f8075402640 1 -- 172.21.15.32:0/3051376921 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80700b4770 con 0x7f80700ccbf0 2024-04-24T22:59:20.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.704+0000 7f80677fe640 1 -- 172.21.15.32:0/3051376921 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8060012400 con 0x7f80700ccbf0 2024-04-24T22:59:20.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.704+0000 7f8075402640 1 -- 172.21.15.32:0/3051376921 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80700b4d30 con 0x7f80700ccbf0 2024-04-24T22:59:20.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.705+0000 7f8075402640 1 -- 172.21.15.32:0/3051376921 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80700682b0 con 0x7f80700ccbf0 2024-04-24T22:59:20.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.705+0000 7f80677fe640 1 -- 172.21.15.32:0/3051376921 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f806001a070 con 0x7f80700ccbf0 2024-04-24T22:59:20.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.705+0000 7f80677fe640 1 --2- 172.21.15.32:0/3051376921 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f805006be30 0x7f805006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:20.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.706+0000 7f80677fe640 1 -- 172.21.15.32:0/3051376921 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f806008ac00 con 0x7f80700ccbf0 2024-04-24T22:59:20.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.706+0000 7f806effd640 1 --2- 172.21.15.32:0/3051376921 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f805006be30 0x7f805006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:20.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.706+0000 7f806effd640 1 --2- 172.21.15.32:0/3051376921 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f805006be30 0x7f805006e2f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f807006b410 tx=0x7f8068008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:20.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.710+0000 7f80677fe640 1 -- 172.21.15.32:0/3051376921 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8060091080 con 0x7f80700ccbf0 2024-04-24T22:59:20.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.928+0000 7f8075402640 1 -- 172.21.15.32:0/3051376921 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f80700cc2f0 con 0x7f80700ccbf0 2024-04-24T22:59:20.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.929+0000 7f80677fe640 1 -- 172.21.15.32:0/3051376921 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5178 (secure 0 0 0) 0x7f8060059640 con 0x7f80700ccbf0 2024-04-24T22:59:20.930 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:59:20.931 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1485884060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1485884060},{"type":"v1","addr":"172.21.15.121:6837","nonce":1485884060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/923934104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":923934104},{"type":"v1","addr":"172.21.15.32:6833","nonce":923934104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3229115835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3229115835},{"type":"v1","addr":"172.21.15.32:6835","nonce":3229115835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T22:59:03.876305+0000","modified":"2024-04-24T22:59:05.932147+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6836/2644625343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2644625343},{"type":"v1","addr":"172.21.15.121:6836","nonce":2644625343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4276]},"id":1}]} 2024-04-24T22:59:20.931 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 6 2024-04-24T22:59:20.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.930+0000 7f8075402640 1 -- 172.21.15.32:0/3051376921 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f805006be30 msgr2=0x7f805006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:20.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.930+0000 7f8075402640 1 --2- 172.21.15.32:0/3051376921 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f805006be30 0x7f805006e2f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f807006b410 tx=0x7f8068008040 comp rx=0 tx=0).stop 2024-04-24T22:59:20.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.931+0000 7f8075402640 1 -- 172.21.15.32:0/3051376921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80700ccbf0 msgr2=0x7f8070144ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:20.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.931+0000 7f8075402640 1 --2- 172.21.15.32:0/3051376921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80700ccbf0 0x7f8070144ea0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f806000caa0 tx=0x7f806000cf60 comp rx=0 tx=0).stop 2024-04-24T22:59:20.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.931+0000 7f8075402640 1 -- 172.21.15.32:0/3051376921 shutdown_connections 2024-04-24T22:59:20.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.931+0000 7f8075402640 1 --2- 172.21.15.32:0/3051376921 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f805006be30 0x7f805006e2f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:20.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.931+0000 7f8075402640 1 --2- 172.21.15.32:0/3051376921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80700ccbf0 0x7f8070144ea0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:20.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.931+0000 7f8075402640 1 --2- 172.21.15.32:0/3051376921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80700b1ea0 0x7f8070144960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:20.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.931+0000 7f8075402640 1 --2- 172.21.15.32:0/3051376921 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8070070c30 0x7f807013da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:20.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.931+0000 7f8075402640 1 -- 172.21.15.32:0/3051376921 >> 172.21.15.32:0/3051376921 conn(0x7f8070074840 msgr2=0x7f80700cd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:20.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.931+0000 7f8075402640 1 -- 172.21.15.32:0/3051376921 shutdown_connections 2024-04-24T22:59:20.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:20.931+0000 7f8075402640 1 -- 172.21.15.32:0/3051376921 wait complete. 2024-04-24T22:59:21.346 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T22:59:03.876305+0000', 'modified': '2024-04-24T22:59:05.932147+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4276}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4276': {'gid': 4276, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.121:6836/2644625343', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6834', 'nonce': 2644625343}, {'type': 'v1', 'addr': '172.21.15.121:6836', 'nonce': 2644625343}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4276, 'qdb_cluster': [4276]} 2024-04-24T22:59:21.346 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-24T22:59:21.346 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-24T22:59:21.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.448+0000 7fe8eae6b640 1 Processor -- start 2024-04-24T22:59:21.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.449+0000 7fe8eae6b640 1 -- start start 2024-04-24T22:59:21.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.449+0000 7fe8eae6b640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe8e415e9b0 0x7fe8e40d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:21.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.449+0000 7fe8eae6b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe8e406b820 0x7fe8e40b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:21.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.449+0000 7fe8eae6b640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8e40d3170 con 0x7fe8e40d4210 2024-04-24T22:59:21.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.449+0000 7fe8eae6b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fe8e40d3450 con 0x7fe8e406b820 2024-04-24T22:59:21.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.449+0000 7fe8eae6b640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe8e40d3730 con 0x7fe8e415e9b0 2024-04-24T22:59:21.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.449+0000 7fe8e9408640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe8e406b820 0x7fe8e40b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:21.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.450+0000 7fe8e9408640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe8e406b820 0x7fe8e40b78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59758/0 (socket says 172.21.15.32:59758) 2024-04-24T22:59:21.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.450+0000 7fe8e9408640 1 -- 172.21.15.32:0/2877128776 learned_addr learned my addr 172.21.15.32:0/2877128776 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:21.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.450+0000 7fe8e8c07640 1 --2- 172.21.15.32:0/2877128776 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe8e415e9b0 0x7fe8e40d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:21.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.450+0000 7fe8e9408640 1 -- 172.21.15.32:0/2877128776 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe8e415e9b0 msgr2=0x7fe8e40d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:21.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.450+0000 7fe8e9408640 1 --2- 172.21.15.32:0/2877128776 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe8e415e9b0 0x7fe8e40d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:21.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.450+0000 7fe8e9408640 1 -- 172.21.15.32:0/2877128776 >> v1:172.21.15.121:6789/0 conn(0x7fe8e40d4210 legacy=0x7fe8e406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:21.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.450+0000 7fe8e9408640 1 -- 172.21.15.32:0/2877128776 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e40d3a10 con 0x7fe8e406b820 2024-04-24T22:59:21.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.450+0000 7fe8e9408640 1 --2- 172.21.15.32:0/2877128776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8e406b820 0x7fe8e40b78f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc009e50 tx=0x7fe8dc02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=545eade6d65cd6ea server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:21.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.451+0000 7fe8e37fe640 1 -- 172.21.15.32:0/2877128776 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc036070 con 0x7fe8e406b820 2024-04-24T22:59:21.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.451+0000 7fe8e37fe640 1 -- 172.21.15.32:0/2877128776 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8dc031600 con 0x7fe8e406b820 2024-04-24T22:59:21.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.451+0000 7fe8e37fe640 1 -- 172.21.15.32:0/2877128776 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc0319d0 con 0x7fe8e406b820 2024-04-24T22:59:21.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.451+0000 7fe8eae6b640 1 -- 172.21.15.32:0/2877128776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8e406b820 msgr2=0x7fe8e40b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:21.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.451+0000 7fe8eae6b640 1 --2- 172.21.15.32:0/2877128776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8e406b820 0x7fe8e40b78f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc009e50 tx=0x7fe8dc02a6c0 comp rx=0 tx=0).stop 2024-04-24T22:59:21.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.451+0000 7fe8eae6b640 1 -- 172.21.15.32:0/2877128776 shutdown_connections 2024-04-24T22:59:21.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.451+0000 7fe8eae6b640 1 --2- 172.21.15.32:0/2877128776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8e406b820 0x7fe8e40b78f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:21.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.451+0000 7fe8eae6b640 1 --2- 172.21.15.32:0/2877128776 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe8e415e9b0 0x7fe8e40d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:21.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.451+0000 7fe8eae6b640 1 -- 172.21.15.32:0/2877128776 >> 172.21.15.32:0/2877128776 conn(0x7fe8e40d11d0 msgr2=0x7fe8e40d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:21.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.451+0000 7fe8eae6b640 1 -- 172.21.15.32:0/2877128776 shutdown_connections 2024-04-24T22:59:21.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.451+0000 7fe8eae6b640 1 -- 172.21.15.32:0/2877128776 wait complete. 2024-04-24T22:59:21.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.452+0000 7fe8eae6b640 1 Processor -- start 2024-04-24T22:59:21.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.452+0000 7fe8eae6b640 1 -- start start 2024-04-24T22:59:21.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.453+0000 7fe8eae6b640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe8e406b820 0x7fe8e40b8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:21.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.453+0000 7fe8eae6b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8e40d4210 0x7fe8e40c9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:21.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.453+0000 7fe8eae6b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8e415e9b0 0x7fe8e41f3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:21.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.453+0000 7fe8eae6b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e40ba0e0 con 0x7fe8e415e9b0 2024-04-24T22:59:21.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.453+0000 7fe8eae6b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e40d2400 con 0x7fe8e40d4210 2024-04-24T22:59:21.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.453+0000 7fe8eae6b640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe8e40ba260 con 0x7fe8e406b820 2024-04-24T22:59:21.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.453+0000 7fe8e9408640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8e415e9b0 0x7fe8e41f3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:21.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.453+0000 7fe8e9408640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8e415e9b0 0x7fe8e41f3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59774/0 (socket says 172.21.15.32:59774) 2024-04-24T22:59:21.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.453+0000 7fe8e9408640 1 -- 172.21.15.32:0/590280858 learned_addr learned my addr 172.21.15.32:0/590280858 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:21.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.453+0000 7fe8e8c07640 1 --2- 172.21.15.32:0/590280858 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe8e406b820 0x7fe8e40b8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:21.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.454+0000 7fe8e3fff640 1 --2- 172.21.15.32:0/590280858 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8e40d4210 0x7fe8e40c9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:21.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.454+0000 7fe8e9408640 1 -- 172.21.15.32:0/590280858 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe8e406b820 msgr2=0x7fe8e40b8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:21.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.454+0000 7fe8e9408640 1 --2- 172.21.15.32:0/590280858 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe8e406b820 0x7fe8e40b8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:21.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.454+0000 7fe8e9408640 1 -- 172.21.15.32:0/590280858 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8e40d4210 msgr2=0x7fe8e40c9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:21.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.454+0000 7fe8e9408640 1 --2- 172.21.15.32:0/590280858 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8e40d4210 0x7fe8e40c9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:21.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.454+0000 7fe8e9408640 1 -- 172.21.15.32:0/590280858 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e40ca1f0 con 0x7fe8e415e9b0 2024-04-24T22:59:21.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.454+0000 7fe8e9408640 1 --2- 172.21.15.32:0/590280858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8e415e9b0 0x7fe8e41f3340 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc009e50 tx=0x7fe8dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:21.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.454+0000 7fe8e17fa640 1 -- 172.21.15.32:0/590280858 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc036050 con 0x7fe8e415e9b0 2024-04-24T22:59:21.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.454+0000 7fe8e17fa640 1 -- 172.21.15.32:0/590280858 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8dc042070 con 0x7fe8e415e9b0 2024-04-24T22:59:21.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.454+0000 7fe8eae6b640 1 -- 172.21.15.32:0/590280858 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8e40ca4d0 con 0x7fe8e415e9b0 2024-04-24T22:59:21.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.454+0000 7fe8e17fa640 1 -- 172.21.15.32:0/590280858 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc009510 con 0x7fe8e415e9b0 2024-04-24T22:59:21.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.454+0000 7fe8eae6b640 1 -- 172.21.15.32:0/590280858 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8e40cad70 con 0x7fe8e415e9b0 2024-04-24T22:59:21.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.456+0000 7fe8e17fa640 1 -- 172.21.15.32:0/590280858 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe8dc007c30 con 0x7fe8e415e9b0 2024-04-24T22:59:21.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.456+0000 7fe8e17fa640 1 --2- 172.21.15.32:0/590280858 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe8c806be30 0x7fe8c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:21.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.456+0000 7fe8e17fa640 1 -- 172.21.15.32:0/590280858 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe8dc0abce0 con 0x7fe8e415e9b0 2024-04-24T22:59:21.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.456+0000 7fe8e8c07640 1 --2- 172.21.15.32:0/590280858 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe8c806be30 0x7fe8c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:21.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.457+0000 7fe8e8c07640 1 --2- 172.21.15.32:0/590280858 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe8c806be30 0x7fe8c806e2f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe8d80079a0 tx=0x7fe8d8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:21.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.457+0000 7fe8edb78740 1 Processor -- start 2024-04-24T22:59:21.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.457+0000 7fe8edb78740 1 -- start start 2024-04-24T22:59:21.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.458+0000 7fe8edb78740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d0d4be8b60 0x55d0d4b9b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:21.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.458+0000 7fe8edb78740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d0d4bcbc80 0x55d0d4bcb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:21.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.458+0000 7fe8edb78740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d0d4bcb540 0x55d0d4ba0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:21.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.458+0000 7fe8edb78740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55d0d4b03f00 con 0x55d0d4be8b60 2024-04-24T22:59:21.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.458+0000 7fe8edb78740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d0d4aac750 con 0x55d0d4bcbc80 2024-04-24T22:59:21.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.458+0000 7fe8edb78740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x55d0d4affec0 con 0x55d0d4bcb540 2024-04-24T22:59:21.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.458+0000 7fe8e8c07640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d0d4be8b60 0x55d0d4b9b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:21.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.458+0000 7fe8e8c07640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d0d4be8b60 0x55d0d4b9b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59786/0 (socket says 172.21.15.32:59786) 2024-04-24T22:59:21.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.458+0000 7fe8e8c07640 1 -- 172.21.15.32:0/3474099019 learned_addr learned my addr 172.21.15.32:0/3474099019 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:21.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.458+0000 7fe8e9408640 1 --2- 172.21.15.32:0/3474099019 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d0d4bcb540 0x55d0d4ba0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:21.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.458+0000 7fe8e3fff640 1 --2- 172.21.15.32:0/3474099019 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d0d4bcbc80 0x55d0d4bcb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:21.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.458+0000 7fe8e8c07640 1 -- 172.21.15.32:0/3474099019 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d0d4bcb540 msgr2=0x55d0d4ba0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:21.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.458+0000 7fe8e8c07640 1 --2- 172.21.15.32:0/3474099019 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d0d4bcb540 0x55d0d4ba0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:21.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.458+0000 7fe8e8c07640 1 -- 172.21.15.32:0/3474099019 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d0d4bcbc80 msgr2=0x55d0d4bcb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:21.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.458+0000 7fe8e8c07640 1 --2- 172.21.15.32:0/3474099019 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d0d4bcbc80 0x55d0d4bcb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:21.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.458+0000 7fe8e8c07640 1 -- 172.21.15.32:0/3474099019 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0d48182d0 con 0x55d0d4be8b60 2024-04-24T22:59:21.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.458+0000 7fe8e8c07640 1 --2- 172.21.15.32:0/3474099019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d0d4be8b60 0x55d0d4b9b4f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8010b20 tx=0x7fe8d8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f2859ec27b072aa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:21.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.458+0000 7fe8ce7fc640 1 -- 172.21.15.32:0/3474099019 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d801c050 con 0x55d0d4be8b60 2024-04-24T22:59:21.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.459+0000 7fe8ce7fc640 1 -- 172.21.15.32:0/3474099019 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8d8026070 con 0x55d0d4be8b60 2024-04-24T22:59:21.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.459+0000 7fe8ce7fc640 1 -- 172.21.15.32:0/3474099019 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d8021440 con 0x55d0d4be8b60 2024-04-24T22:59:21.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.459+0000 7fe8edb78740 1 -- 172.21.15.32:0/3474099019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d0d4be8b60 msgr2=0x55d0d4b9b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:21.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.459+0000 7fe8edb78740 1 --2- 172.21.15.32:0/3474099019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d0d4be8b60 0x55d0d4b9b4f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8010b20 tx=0x7fe8d8013000 comp rx=0 tx=0).stop 2024-04-24T22:59:21.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.459+0000 7fe8edb78740 1 -- 172.21.15.32:0/3474099019 shutdown_connections 2024-04-24T22:59:21.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.459+0000 7fe8edb78740 1 --2- 172.21.15.32:0/3474099019 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d0d4bcb540 0x55d0d4ba0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:21.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.459+0000 7fe8edb78740 1 --2- 172.21.15.32:0/3474099019 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d0d4bcbc80 0x55d0d4bcb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:21.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.459+0000 7fe8edb78740 1 --2- 172.21.15.32:0/3474099019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d0d4be8b60 0x55d0d4b9b4f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:21.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.459+0000 7fe8edb78740 1 -- 172.21.15.32:0/3474099019 >> 172.21.15.32:0/3474099019 conn(0x55d0d4b9d760 msgr2=0x55d0d4be6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:21.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.459+0000 7fe8edb78740 1 -- 172.21.15.32:0/3474099019 shutdown_connections 2024-04-24T22:59:21.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.459+0000 7fe8edb78740 1 -- 172.21.15.32:0/3474099019 wait complete. 2024-04-24T22:59:21.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.460+0000 7fe8edb78740 1 -- start start 2024-04-24T22:59:21.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.460+0000 7fe8edb78740 1 Processor -- start 2024-04-24T22:59:21.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.461+0000 7fe8edb78740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d0d4bcbc80 0x55d0d4be7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:21.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.461+0000 7fe8edb78740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d0d4be8b60 0x55d0d4bf6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:21.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.461+0000 7fe8edb78740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d0d4bbb580 0x55d0d4bde3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:21.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.461+0000 7fe8edb78740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55d0d4b9f910 con 0x55d0d4bbb580 2024-04-24T22:59:21.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.461+0000 7fe8edb78740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d0d4abfcf0 con 0x55d0d4bcbc80 2024-04-24T22:59:21.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.461+0000 7fe8edb78740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x55d0d4b9ed00 con 0x55d0d4be8b60 2024-04-24T22:59:21.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.461+0000 7fe8edb78740 10 client.0 _ll_register_callbacks cb 0x55d0d4abf6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:59:21.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.461+0000 7fe8e9408640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d0d4bbb580 0x55d0d4bde3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:21.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.461+0000 7fe8e9408640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d0d4bbb580 0x55d0d4bde3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59800/0 (socket says 172.21.15.32:59800) 2024-04-24T22:59:21.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.461+0000 7fe8e9408640 1 -- 172.21.15.32:0/1072149920 learned_addr learned my addr 172.21.15.32:0/1072149920 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:21.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.461+0000 7fe8e3fff640 1 --2- 172.21.15.32:0/1072149920 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d0d4be8b60 0x55d0d4bf6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:21.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.461+0000 7fe8e9408640 1 -- 172.21.15.32:0/1072149920 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d0d4be8b60 msgr2=0x55d0d4bf6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:21.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.461+0000 7fe8e8c07640 1 --2- 172.21.15.32:0/1072149920 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d0d4bcbc80 0x55d0d4be7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:21.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.461+0000 7fe8e9408640 1 --2- 172.21.15.32:0/1072149920 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d0d4be8b60 0x55d0d4bf6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:21.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.461+0000 7fe8e9408640 1 -- 172.21.15.32:0/1072149920 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d0d4bcbc80 msgr2=0x55d0d4be7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:21.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.461+0000 7fe8e9408640 1 --2- 172.21.15.32:0/1072149920 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d0d4bcbc80 0x55d0d4be7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:21.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.461+0000 7fe8e9408640 1 -- 172.21.15.32:0/1072149920 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0d48e26b0 con 0x55d0d4bbb580 2024-04-24T22:59:21.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.461+0000 7fe8e9408640 1 --2- 172.21.15.32:0/1072149920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d0d4bbb580 0x55d0d4bde3b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc02acb0 tx=0x7fe8dc0316c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8e51db210706c7b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:21.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.461+0000 7fe8c37fe640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-24T22:59:21.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.461+0000 7fe8c37fe640 1 -- 172.21.15.32:0/1072149920 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc0a7020 con 0x55d0d4bbb580 2024-04-24T22:59:21.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.461+0000 7fe8c37fe640 1 -- 172.21.15.32:0/1072149920 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe8dc0a3d70 con 0x55d0d4bbb580 2024-04-24T22:59:21.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.461+0000 7fe8edb78740 1 -- 172.21.15.32:0/1072149920 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8d8026070 con 0x55d0d4bbb580 2024-04-24T22:59:21.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.462+0000 7fe8c37fe640 1 -- 172.21.15.32:0/1072149920 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) ==== 24+0+0 (secure 0 0 0) 0x7fe8dc082b70 con 0x55d0d4bbb580 2024-04-24T22:59:21.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.462+0000 7fe8edb78740 10 client.4671 fetch_fsmap learned FSMap version 6 2024-04-24T22:59:21.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.462+0000 7fe8edb78740 1 -- 172.21.15.32:0/1072149920 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) -- 0x55d0d4bddbb0 con 0x55d0d4bbb580 2024-04-24T22:59:21.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.462+0000 7fe8c37fe640 1 -- 172.21.15.32:0/1072149920 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8d8026070 con 0x55d0d4bbb580 2024-04-24T22:59:21.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.462+0000 7fe8c37fe640 1 -- 172.21.15.32:0/1072149920 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 6) ==== 3494+0+0 (secure 0 0 0) 0x7fe8dc079490 con 0x55d0d4bbb580 2024-04-24T22:59:21.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.462+0000 7fe8edb78740 10 client.4671 fetch_fsmap finished waiting for FSMap version 6 2024-04-24T22:59:21.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.462+0000 7fe8edb78740 10 client.4671 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-24T22:59:21.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.462+0000 7fe8edb78740 20 client.4671 populate_metadata read hostname 'smithi032' 2024-04-24T22:59:21.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.462+0000 7fe8edb78740 1 --2- 172.21.15.32:0/1072149920 >> [v2:172.21.15.121:6834/2644625343,v1:172.21.15.121:6836/2644625343] conn(0x55d0d4bcd5b0 0x55d0d4bcd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:21.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.462+0000 7fe8edb78740 4 client.4671 mds_command: new command op to 4276 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:59:21.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.462+0000 7fe8edb78740 1 -- 172.21.15.32:0/1072149920 --> [v2:172.21.15.121:6834/2644625343,v1:172.21.15.121:6836/2644625343] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0d4b11440 con 0x55d0d4bcd5b0 2024-04-24T22:59:21.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.462+0000 7fe8e8c07640 1 --2- 172.21.15.32:0/1072149920 >> [v2:172.21.15.121:6834/2644625343,v1:172.21.15.121:6836/2644625343] conn(0x55d0d4bcd5b0 0x55d0d4bcd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:21.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.463+0000 7fe8e8c07640 1 --2- 172.21.15.32:0/1072149920 >> [v2:172.21.15.121:6834/2644625343,v1:172.21.15.121:6836/2644625343] conn(0x55d0d4bcd5b0 0x55d0d4bcd990 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36837d468b1e0518 server_cookie=85b40a5cc75aa8ed in_seq=0 out_seq=0 2024-04-24T22:59:21.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.463+0000 7fe8c37fe640 10 client.4671 ms_handle_connect on v2:172.21.15.121:6834/2644625343 2024-04-24T22:59:21.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.463+0000 7fe8c37fe640 1 -- 172.21.15.32:0/1072149920 <== mds.0 v2:172.21.15.121:6834/2644625343 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d0d4b11440 con 0x55d0d4bcd5b0 2024-04-24T22:59:21.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.463+0000 7fe8c37fe640 10 client.4671 handle_command_reply: tid=0 2024-04-24T22:59:21.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.463+0000 7fe8edb78740 1 client.4671 shutdown 2024-04-24T22:59:21.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.464+0000 7fe8edb78740 1 -- 172.21.15.32:0/1072149920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d0d4bbb580 msgr2=0x55d0d4bde3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:21.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.464+0000 7fe8edb78740 1 --2- 172.21.15.32:0/1072149920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d0d4bbb580 0x55d0d4bde3b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc02acb0 tx=0x7fe8dc0316c0 comp rx=0 tx=0).stop 2024-04-24T22:59:21.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.464+0000 7fe8edb78740 1 -- 172.21.15.32:0/1072149920 shutdown_connections 2024-04-24T22:59:21.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.464+0000 7fe8edb78740 1 --2- 172.21.15.32:0/1072149920 >> [v2:172.21.15.121:6834/2644625343,v1:172.21.15.121:6836/2644625343] conn(0x55d0d4bcd5b0 0x55d0d4bcd990 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:21.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.464+0000 7fe8edb78740 1 --2- 172.21.15.32:0/1072149920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d0d4bbb580 0x55d0d4bde3b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:21.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.464+0000 7fe8edb78740 1 --2- 172.21.15.32:0/1072149920 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d0d4be8b60 0x55d0d4bf6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:21.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.464+0000 7fe8c37fe640 0 client.4671 ms_handle_reset on v2:172.21.15.121:6834/2644625343 2024-04-24T22:59:21.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.464+0000 7fe8edb78740 1 --2- 172.21.15.32:0/1072149920 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d0d4bcbc80 0x55d0d4be7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:21.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.464+0000 7fe8edb78740 1 -- 172.21.15.32:0/1072149920 >> 172.21.15.32:0/1072149920 conn(0x55d0d4bcb540 msgr2=0x55d0d4ba13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:21.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.464+0000 7fe8edb78740 1 -- 172.21.15.32:0/1072149920 shutdown_connections 2024-04-24T22:59:21.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.464+0000 7fe8edb78740 1 -- 172.21.15.32:0/1072149920 wait complete. 2024-04-24T22:59:21.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.464+0000 7fe8edb78740 20 client.4671 trim_cache size 0 max 16384 2024-04-24T22:59:21.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.482+0000 7fe8edb78740 1 Processor -- start 2024-04-24T22:59:21.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.482+0000 7fe8edb78740 1 -- start start 2024-04-24T22:59:21.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.483+0000 7fe8edb78740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d0d4bbb580 0x55d0d4bf59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:21.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.483+0000 7fe8edb78740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d0d4bcbc80 0x55d0d4bcafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:21.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.483+0000 7fe8edb78740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d0d4be8b60 0x55d0d4b9c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:21.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.483+0000 7fe8edb78740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e41ffc80 con 0x55d0d4be8b60 2024-04-24T22:59:21.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.483+0000 7fe8edb78740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e40cc630 con 0x55d0d4bbb580 2024-04-24T22:59:21.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.483+0000 7fe8edb78740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe8e41ff660 con 0x55d0d4bcbc80 2024-04-24T22:59:21.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.483+0000 7fe8e9408640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d0d4be8b60 0x55d0d4b9c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:21.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.483+0000 7fe8e9408640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d0d4be8b60 0x55d0d4b9c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59808/0 (socket says 172.21.15.32:59808) 2024-04-24T22:59:21.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.483+0000 7fe8e9408640 1 -- 172.21.15.32:0/1975921001 learned_addr learned my addr 172.21.15.32:0/1975921001 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:21.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.483+0000 7fe8e3fff640 1 --2- 172.21.15.32:0/1975921001 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d0d4bcbc80 0x55d0d4bcafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:21.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.483+0000 7fe8e8c07640 1 --2- 172.21.15.32:0/1975921001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d0d4bbb580 0x55d0d4bf59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:21.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.483+0000 7fe8e9408640 1 -- 172.21.15.32:0/1975921001 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d0d4bcbc80 msgr2=0x55d0d4bcafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:21.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.483+0000 7fe8e9408640 1 --2- 172.21.15.32:0/1975921001 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d0d4bcbc80 0x55d0d4bcafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:21.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.483+0000 7fe8e9408640 1 -- 172.21.15.32:0/1975921001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d0d4bbb580 msgr2=0x55d0d4bf59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:21.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.483+0000 7fe8e9408640 1 --2- 172.21.15.32:0/1975921001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d0d4bbb580 0x55d0d4bf59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:21.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.483+0000 7fe8e9408640 1 -- 172.21.15.32:0/1975921001 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0d4be57b0 con 0x55d0d4be8b60 2024-04-24T22:59:21.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.483+0000 7fe8e8c07640 1 --2- 172.21.15.32:0/1975921001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d0d4bbb580 0x55d0d4bf59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:21.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.483+0000 7fe8e9408640 1 --2- 172.21.15.32:0/1975921001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d0d4be8b60 0x55d0d4b9c230 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc082640 tx=0x7fe8dc091000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=399f1b9d9320526e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:21.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.483+0000 7fe8cdffb640 1 -- 172.21.15.32:0/1975921001 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc09f020 con 0x55d0d4be8b60 2024-04-24T22:59:21.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.483+0000 7fe8cdffb640 1 -- 172.21.15.32:0/1975921001 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8dc037040 con 0x55d0d4be8b60 2024-04-24T22:59:21.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.483+0000 7fe8cdffb640 1 -- 172.21.15.32:0/1975921001 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc0a7070 con 0x55d0d4be8b60 2024-04-24T22:59:21.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.484+0000 7fe8edb78740 1 -- 172.21.15.32:0/1975921001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d0d4be8b60 msgr2=0x55d0d4b9c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:21.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.484+0000 7fe8edb78740 1 --2- 172.21.15.32:0/1975921001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d0d4be8b60 0x55d0d4b9c230 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc082640 tx=0x7fe8dc091000 comp rx=0 tx=0).stop 2024-04-24T22:59:21.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.484+0000 7fe8edb78740 1 -- 172.21.15.32:0/1975921001 shutdown_connections 2024-04-24T22:59:21.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.484+0000 7fe8edb78740 1 --2- 172.21.15.32:0/1975921001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d0d4be8b60 0x55d0d4b9c230 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:21.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.484+0000 7fe8edb78740 1 --2- 172.21.15.32:0/1975921001 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d0d4bcbc80 0x55d0d4bcafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:21.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.484+0000 7fe8edb78740 1 --2- 172.21.15.32:0/1975921001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d0d4bbb580 0x55d0d4bf59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:21.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.484+0000 7fe8edb78740 1 -- 172.21.15.32:0/1975921001 >> 172.21.15.32:0/1975921001 conn(0x55d0d4bcb540 msgr2=0x55d0d4ba13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:21.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.484+0000 7fe8edb78740 1 -- 172.21.15.32:0/1975921001 shutdown_connections 2024-04-24T22:59:21.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.484+0000 7fe8edb78740 1 -- 172.21.15.32:0/1975921001 wait complete. 2024-04-24T22:59:21.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.484+0000 7fe8edb78740 1 -- start start 2024-04-24T22:59:21.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.485+0000 7fe8edb78740 1 Processor -- start 2024-04-24T22:59:21.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.485+0000 7fe8edb78740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d0d4bbb580 0x55d0d4c5f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:21.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.485+0000 7fe8edb78740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d0d4bcbc80 0x55d0d4bcc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:21.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.485+0000 7fe8edb78740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d0d4be8b60 0x55d0d4bdedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:21.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.485+0000 7fe8edb78740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55d0d4bdccc0 con 0x55d0d4bbb580 2024-04-24T22:59:21.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.485+0000 7fe8edb78740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d0d4bdde90 con 0x55d0d4bcbc80 2024-04-24T22:59:21.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.485+0000 7fe8edb78740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x55d0d4b9a510 con 0x55d0d4be8b60 2024-04-24T22:59:21.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.486+0000 7fe8edb78740 10 client.0 _ll_register_callbacks cb 0x7fe8dc079b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:59:21.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.486+0000 7fe8e8c07640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d0d4bbb580 0x55d0d4c5f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:21.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.486+0000 7fe8e8c07640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d0d4bbb580 0x55d0d4c5f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59816/0 (socket says 172.21.15.32:59816) 2024-04-24T22:59:21.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.486+0000 7fe8e8c07640 1 -- 172.21.15.32:0/2702517429 learned_addr learned my addr 172.21.15.32:0/2702517429 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:21.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.486+0000 7fe8e3fff640 1 --2- 172.21.15.32:0/2702517429 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d0d4bcbc80 0x55d0d4bcc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:21.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.486+0000 7fe8e9408640 1 --2- 172.21.15.32:0/2702517429 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d0d4be8b60 0x55d0d4bdedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:21.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.486+0000 7fe8e8c07640 1 -- 172.21.15.32:0/2702517429 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d0d4be8b60 msgr2=0x55d0d4bdedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:21.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.486+0000 7fe8e8c07640 1 --2- 172.21.15.32:0/2702517429 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d0d4be8b60 0x55d0d4bdedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:21.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.486+0000 7fe8e8c07640 1 -- 172.21.15.32:0/2702517429 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d0d4bcbc80 msgr2=0x55d0d4bcc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:21.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.486+0000 7fe8e8c07640 1 --2- 172.21.15.32:0/2702517429 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d0d4bcbc80 0x55d0d4bcc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:21.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.486+0000 7fe8e8c07640 1 -- 172.21.15.32:0/2702517429 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0d4be5420 con 0x55d0d4bbb580 2024-04-24T22:59:21.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.486+0000 7fe8e8c07640 1 --2- 172.21.15.32:0/2702517429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d0d4bbb580 0x55d0d4c5f8a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8023040 tx=0x7fe8d8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d057199acb503f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:21.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.486+0000 7fe8c37fe640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-24T22:59:21.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.486+0000 7fe8c37fe640 1 -- 172.21.15.32:0/2702517429 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d8039070 con 0x55d0d4bbb580 2024-04-24T22:59:21.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.486+0000 7fe8c37fe640 1 -- 172.21.15.32:0/2702517429 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe8d8005da0 con 0x55d0d4bbb580 2024-04-24T22:59:21.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.486+0000 7fe8edb78740 1 -- 172.21.15.32:0/2702517429 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8dc037040 con 0x55d0d4bbb580 2024-04-24T22:59:21.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.486+0000 7fe8c37fe640 1 -- 172.21.15.32:0/2702517429 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) ==== 24+0+0 (secure 0 0 0) 0x7fe8d801c040 con 0x55d0d4bbb580 2024-04-24T22:59:21.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.486+0000 7fe8edb78740 10 client.4677 fetch_fsmap learned FSMap version 6 2024-04-24T22:59:21.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.486+0000 7fe8edb78740 1 -- 172.21.15.32:0/2702517429 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) -- 0x55d0d4b9b9e0 con 0x55d0d4bbb580 2024-04-24T22:59:21.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.486+0000 7fe8c37fe640 1 -- 172.21.15.32:0/2702517429 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8dc037040 con 0x55d0d4bbb580 2024-04-24T22:59:21.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.487+0000 7fe8c37fe640 1 -- 172.21.15.32:0/2702517429 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 6) ==== 3494+0+0 (secure 0 0 0) 0x7fe8d8035300 con 0x55d0d4bbb580 2024-04-24T22:59:21.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.487+0000 7fe8edb78740 10 client.4677 fetch_fsmap finished waiting for FSMap version 6 2024-04-24T22:59:21.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.487+0000 7fe8edb78740 10 client.4677 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-24T22:59:21.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.487+0000 7fe8edb78740 20 client.4677 populate_metadata read hostname 'smithi032' 2024-04-24T22:59:21.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.487+0000 7fe8edb78740 1 --2- 172.21.15.32:0/2702517429 >> [v2:172.21.15.121:6834/2644625343,v1:172.21.15.121:6836/2644625343] conn(0x55d0d4ba02e0 0x55d0d4bdb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:21.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.487+0000 7fe8edb78740 4 client.4677 mds_command: new command op to 4276 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T22:59:21.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.487+0000 7fe8edb78740 1 -- 172.21.15.32:0/2702517429 --> [v2:172.21.15.121:6834/2644625343,v1:172.21.15.121:6836/2644625343] -- command(tid 0: {"prefix": "status"}) -- 0x55d0d4b0b790 con 0x55d0d4ba02e0 2024-04-24T22:59:21.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.487+0000 7fe8e3fff640 1 --2- 172.21.15.32:0/2702517429 >> [v2:172.21.15.121:6834/2644625343,v1:172.21.15.121:6836/2644625343] conn(0x55d0d4ba02e0 0x55d0d4bdb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:21.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.487+0000 7fe8e3fff640 1 --2- 172.21.15.32:0/2702517429 >> [v2:172.21.15.121:6834/2644625343,v1:172.21.15.121:6836/2644625343] conn(0x55d0d4ba02e0 0x55d0d4bdb990 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ace68729f1d04eaf server_cookie=dc3c2f1575810f1c in_seq=0 out_seq=0 2024-04-24T22:59:21.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.487+0000 7fe8c37fe640 10 client.4677 ms_handle_connect on v2:172.21.15.121:6834/2644625343 2024-04-24T22:59:21.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.487+0000 7fe8c37fe640 1 -- 172.21.15.32:0/2702517429 <== mds.0 v2:172.21.15.121:6834/2644625343 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x55d0d4b0b790 con 0x55d0d4ba02e0 2024-04-24T22:59:21.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.487+0000 7fe8c37fe640 10 client.4677 handle_command_reply: tid=0 2024-04-24T22:59:21.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.487+0000 7fe8edb78740 1 client.4677 shutdown 2024-04-24T22:59:21.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.488+0000 7fe8edb78740 1 -- 172.21.15.32:0/2702517429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d0d4bbb580 msgr2=0x55d0d4c5f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:21.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.488+0000 7fe8edb78740 1 --2- 172.21.15.32:0/2702517429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d0d4bbb580 0x55d0d4c5f8a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8023040 tx=0x7fe8d8004730 comp rx=0 tx=0).stop 2024-04-24T22:59:21.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.488+0000 7fe8edb78740 1 -- 172.21.15.32:0/2702517429 shutdown_connections 2024-04-24T22:59:21.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.488+0000 7fe8edb78740 1 --2- 172.21.15.32:0/2702517429 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d0d4be8b60 0x55d0d4bdedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:21.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.488+0000 7fe8edb78740 1 --2- 172.21.15.32:0/2702517429 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d0d4bcbc80 0x55d0d4bcc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:21.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.488+0000 7fe8edb78740 1 --2- 172.21.15.32:0/2702517429 >> [v2:172.21.15.121:6834/2644625343,v1:172.21.15.121:6836/2644625343] conn(0x55d0d4ba02e0 0x55d0d4bdb990 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:21.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.488+0000 7fe8edb78740 1 --2- 172.21.15.32:0/2702517429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d0d4bbb580 0x55d0d4c5f8a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:21.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.488+0000 7fe8c37fe640 0 client.4677 ms_handle_reset on v2:172.21.15.121:6834/2644625343 2024-04-24T22:59:21.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.488+0000 7fe8edb78740 1 -- 172.21.15.32:0/2702517429 >> 172.21.15.32:0/2702517429 conn(0x55d0d4bcb540 msgr2=0x55d0d4bcd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:21.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.488+0000 7fe8edb78740 1 -- 172.21.15.32:0/2702517429 shutdown_connections 2024-04-24T22:59:21.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.488+0000 7fe8edb78740 1 -- 172.21.15.32:0/2702517429 wait complete. 2024-04-24T22:59:21.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.488+0000 7fe8edb78740 20 client.4677 trim_cache size 0 max 16384 2024-04-24T22:59:21.501 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T22:59:21.501 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", 2024-04-24T22:59:21.501 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2024-04-24T22:59:21.501 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 4276, 2024-04-24T22:59:21.501 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-04-24T22:59:21.502 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-04-24T22:59:21.502 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-04-24T22:59:21.502 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 16.969564681000001, 2024-04-24T22:59:21.502 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 6, 2024-04-24T22:59:21.502 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 21, 2024-04-24T22:59:21.502 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 21, 2024-04-24T22:59:21.502 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 23.997917108999999 2024-04-24T22:59:21.502 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T22:59:21.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.488+0000 7fe8eae6b640 1 -- 172.21.15.32:0/590280858 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe8c806be30 msgr2=0x7fe8c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:21.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.488+0000 7fe8eae6b640 1 --2- 172.21.15.32:0/590280858 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe8c806be30 0x7fe8c806e2f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe8d80079a0 tx=0x7fe8d8008040 comp rx=0 tx=0).stop 2024-04-24T22:59:21.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.488+0000 7fe8eae6b640 1 -- 172.21.15.32:0/590280858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8e415e9b0 msgr2=0x7fe8e41f3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:21.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.488+0000 7fe8eae6b640 1 --2- 172.21.15.32:0/590280858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8e415e9b0 0x7fe8e41f3340 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc009e50 tx=0x7fe8dc004a40 comp rx=0 tx=0).stop 2024-04-24T22:59:21.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.488+0000 7fe8eae6b640 1 -- 172.21.15.32:0/590280858 shutdown_connections 2024-04-24T22:59:21.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.488+0000 7fe8eae6b640 1 --2- 172.21.15.32:0/590280858 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe8c806be30 0x7fe8c806e2f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:21.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.488+0000 7fe8eae6b640 1 --2- 172.21.15.32:0/590280858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8e415e9b0 0x7fe8e41f3340 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:21.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.488+0000 7fe8eae6b640 1 --2- 172.21.15.32:0/590280858 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8e40d4210 0x7fe8e40c9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:21.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.488+0000 7fe8eae6b640 1 --2- 172.21.15.32:0/590280858 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe8e406b820 0x7fe8e40b8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:21.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.488+0000 7fe8eae6b640 1 -- 172.21.15.32:0/590280858 >> 172.21.15.32:0/590280858 conn(0x7fe8e40d11d0 msgr2=0x7fe8e41592a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:21.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.488+0000 7fe8eae6b640 1 -- 172.21.15.32:0/590280858 shutdown_connections 2024-04-24T22:59:21.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:21.488+0000 7fe8eae6b640 1 -- 172.21.15.32:0/590280858 wait complete. 2024-04-24T22:59:21.506 INFO:teuthology.run_tasks:Running task sequential... 2024-04-24T22:59:21.516 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-24T22:59:21.516 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-24T22:59:21.517 INFO:teuthology.task.print:Enabling mgr modules 2024-04-24T22:59:21.517 INFO:teuthology.run_tasks:Running task kclient... 2024-04-24T22:59:21.586 INFO:tasks.kclient:Mounting kernel clients... 2024-04-24T22:59:21.587 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-24T22:59:21.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:59:21.587 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-04-24T22:59:21.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:59:21.604 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link delete ceph-brx 2024-04-24T22:59:21.669 INFO:teuthology.orchestra.run.smithi032.stderr:Cannot find device "ceph-brx" 2024-04-24T22:59:21.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:59:21.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:59:21.671 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-04-24T22:59:21.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:59:21.727 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link delete ceph-brx 2024-04-24T22:59:21.793 INFO:teuthology.orchestra.run.smithi032.stderr:Cannot find device "ceph-brx" 2024-04-24T22:59:21.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:59:21.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:59:21.794 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2024-04-24T22:59:21.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:59:21.813 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link delete ceph-brx 2024-04-24T22:59:21.888 INFO:teuthology.orchestra.run.smithi121.stderr:Cannot find device "ceph-brx" 2024-04-24T22:59:21.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:59:21.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:59:21.889 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2024-04-24T22:59:21.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:59:21.946 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link delete ceph-brx 2024-04-24T22:59:22.013 INFO:teuthology.orchestra.run.smithi121.stderr:Cannot find device "ceph-brx" 2024-04-24T22:59:22.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:59:22.014 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-04-24T22:59:22.015 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-24T22:59:22.015 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T22:59:22.015 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T22:59:22.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.115+0000 7faefe33b640 1 Processor -- start 2024-04-24T22:59:22.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.116+0000 7faefe33b640 1 -- start start 2024-04-24T22:59:22.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.116+0000 7faefe33b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faef80b5560 0x7faef815e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:22.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.116+0000 7faefe33b640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faef80b6050 0x7faef815ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:22.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.116+0000 7faefe33b640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faef80cd910 con 0x7faef80cd3c0 2024-04-24T22:59:22.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.116+0000 7faefe33b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7faef80cdbf0 con 0x7faef80b5560 2024-04-24T22:59:22.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.116+0000 7faefe33b640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7faef80cded0 con 0x7faef80b6050 2024-04-24T22:59:22.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.116+0000 7faef7fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faef80b5560 0x7faef815e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:22.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.117+0000 7faef7fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faef80b5560 0x7faef815e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59822/0 (socket says 172.21.15.32:59822) 2024-04-24T22:59:22.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.117+0000 7faef7fff640 1 -- 172.21.15.32:0/1415633298 learned_addr learned my addr 172.21.15.32:0/1415633298 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:22.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.117+0000 7faefc8d8640 1 --2- 172.21.15.32:0/1415633298 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faef80b6050 0x7faef815ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:22.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.117+0000 7faef7fff640 1 -- 172.21.15.32:0/1415633298 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faef80b6050 msgr2=0x7faef815ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:22.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.117+0000 7faef7fff640 1 --2- 172.21.15.32:0/1415633298 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faef80b6050 0x7faef815ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:22.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.117+0000 7faef7fff640 1 -- 172.21.15.32:0/1415633298 >> v1:172.21.15.121:6789/0 conn(0x7faef80cd3c0 legacy=0x7faef80b5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:22.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.117+0000 7faef7fff640 1 -- 172.21.15.32:0/1415633298 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faef80ce1b0 con 0x7faef80b5560 2024-04-24T22:59:22.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.118+0000 7faef7fff640 1 --2- 172.21.15.32:0/1415633298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faef80b5560 0x7faef815e6c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7faee800bb00 tx=0x7faee802c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5c84257a5d94f28 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:22.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.118+0000 7faef6ffd640 1 -- 172.21.15.32:0/1415633298 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee803a070 con 0x7faef80b5560 2024-04-24T22:59:22.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.118+0000 7faef6ffd640 1 -- 172.21.15.32:0/1415633298 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faee8032480 con 0x7faef80b5560 2024-04-24T22:59:22.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.118+0000 7faef6ffd640 1 -- 172.21.15.32:0/1415633298 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee8032850 con 0x7faef80b5560 2024-04-24T22:59:22.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.118+0000 7faefe33b640 1 -- 172.21.15.32:0/1415633298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faef80b5560 msgr2=0x7faef815e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:22.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.118+0000 7faefe33b640 1 --2- 172.21.15.32:0/1415633298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faef80b5560 0x7faef815e6c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7faee800bb00 tx=0x7faee802c6e0 comp rx=0 tx=0).stop 2024-04-24T22:59:22.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.118+0000 7faefe33b640 1 -- 172.21.15.32:0/1415633298 shutdown_connections 2024-04-24T22:59:22.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.118+0000 7faefe33b640 1 --2- 172.21.15.32:0/1415633298 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faef80b6050 0x7faef815ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:22.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.118+0000 7faefe33b640 1 --2- 172.21.15.32:0/1415633298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faef80b5560 0x7faef815e6c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:22.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.118+0000 7faefe33b640 1 -- 172.21.15.32:0/1415633298 >> 172.21.15.32:0/1415633298 conn(0x7faef806b110 msgr2=0x7faef806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:22.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.118+0000 7faefe33b640 1 -- 172.21.15.32:0/1415633298 shutdown_connections 2024-04-24T22:59:22.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.118+0000 7faefe33b640 1 -- 172.21.15.32:0/1415633298 wait complete. 2024-04-24T22:59:22.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.119+0000 7faefe33b640 1 Processor -- start 2024-04-24T22:59:22.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.120+0000 7faefe33b640 1 -- start start 2024-04-24T22:59:22.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.120+0000 7faefe33b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faef80b5560 0x7faef8157e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:22.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.120+0000 7faefe33b640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faef80b6050 0x7faef814c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:22.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.120+0000 7faefe33b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faef80cd3c0 0x7faef814cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:22.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.120+0000 7faefe33b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7faef806c770 con 0x7faef80b5560 2024-04-24T22:59:22.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.120+0000 7faefe33b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7faef806c5f0 con 0x7faef80cd3c0 2024-04-24T22:59:22.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.120+0000 7faefe33b640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7faef806c8f0 con 0x7faef80b6050 2024-04-24T22:59:22.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.120+0000 7faef7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faef80b5560 0x7faef8157e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:22.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.120+0000 7faef7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faef80b5560 0x7faef8157e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59824/0 (socket says 172.21.15.32:59824) 2024-04-24T22:59:22.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.120+0000 7faef7fff640 1 -- 172.21.15.32:0/2507261633 learned_addr learned my addr 172.21.15.32:0/2507261633 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:22.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.121+0000 7faefc8d8640 1 --2- 172.21.15.32:0/2507261633 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faef80cd3c0 0x7faef814cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:22.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.121+0000 7faef77fe640 1 --2- 172.21.15.32:0/2507261633 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faef80b6050 0x7faef814c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:22.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.121+0000 7faef7fff640 1 -- 172.21.15.32:0/2507261633 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faef80b6050 msgr2=0x7faef814c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:22.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.121+0000 7faef7fff640 1 --2- 172.21.15.32:0/2507261633 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faef80b6050 0x7faef814c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:22.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.121+0000 7faef7fff640 1 -- 172.21.15.32:0/2507261633 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faef80cd3c0 msgr2=0x7faef814cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:22.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.121+0000 7faef7fff640 1 --2- 172.21.15.32:0/2507261633 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faef80cd3c0 0x7faef814cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:22.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.121+0000 7faef7fff640 1 -- 172.21.15.32:0/2507261633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faef8156570 con 0x7faef80b5560 2024-04-24T22:59:22.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.121+0000 7faef7fff640 1 --2- 172.21.15.32:0/2507261633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faef80b5560 0x7faef8157e20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7faee8000c00 tx=0x7faee80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:22.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.121+0000 7faef4ff9640 1 -- 172.21.15.32:0/2507261633 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee8044020 con 0x7faef80b5560 2024-04-24T22:59:22.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.121+0000 7faef4ff9640 1 -- 172.21.15.32:0/2507261633 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faee803f670 con 0x7faef80b5560 2024-04-24T22:59:22.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.121+0000 7faefe33b640 1 -- 172.21.15.32:0/2507261633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faef8156850 con 0x7faef80b5560 2024-04-24T22:59:22.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.121+0000 7faef4ff9640 1 -- 172.21.15.32:0/2507261633 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee803a040 con 0x7faef80b5560 2024-04-24T22:59:22.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.121+0000 7faefe33b640 1 -- 172.21.15.32:0/2507261633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faef8156e10 con 0x7faef80b5560 2024-04-24T22:59:22.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.122+0000 7faefe33b640 1 -- 172.21.15.32:0/2507261633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faef8068430 con 0x7faef80b5560 2024-04-24T22:59:22.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.123+0000 7faef4ff9640 1 -- 172.21.15.32:0/2507261633 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faee8047020 con 0x7faef80b5560 2024-04-24T22:59:22.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.123+0000 7faef4ff9640 1 --2- 172.21.15.32:0/2507261633 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7faee406be30 0x7faee406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:22.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.123+0000 7faef4ff9640 1 -- 172.21.15.32:0/2507261633 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7faee80ad540 con 0x7faef80b5560 2024-04-24T22:59:22.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.123+0000 7faef77fe640 1 --2- 172.21.15.32:0/2507261633 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7faee406be30 0x7faee406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:22.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.124+0000 7faef77fe640 1 --2- 172.21.15.32:0/2507261633 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7faee406be30 0x7faee406e2f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7faef814d4b0 tx=0x7faeec009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:22.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.128+0000 7faef4ff9640 1 -- 172.21.15.32:0/2507261633 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7faee804e070 con 0x7faef80b5560 2024-04-24T22:59:22.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.351+0000 7faefe33b640 1 -- 172.21.15.32:0/2507261633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7faef806d5f0 con 0x7faef80b5560 2024-04-24T22:59:22.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.352+0000 7faef4ff9640 1 -- 172.21.15.32:0/2507261633 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) ==== 53+0+73 (secure 0 0 0) 0x7faee80820c0 con 0x7faef80b5560 2024-04-24T22:59:22.353 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T22:59:22.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.353+0000 7faefe33b640 1 -- 172.21.15.32:0/2507261633 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7faee406be30 msgr2=0x7faee406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:22.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.353+0000 7faefe33b640 1 --2- 172.21.15.32:0/2507261633 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7faee406be30 0x7faee406e2f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7faef814d4b0 tx=0x7faeec009040 comp rx=0 tx=0).stop 2024-04-24T22:59:22.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.353+0000 7faefe33b640 1 -- 172.21.15.32:0/2507261633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faef80b5560 msgr2=0x7faef8157e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:22.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.353+0000 7faefe33b640 1 --2- 172.21.15.32:0/2507261633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faef80b5560 0x7faef8157e20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7faee8000c00 tx=0x7faee80046c0 comp rx=0 tx=0).stop 2024-04-24T22:59:22.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.353+0000 7faefe33b640 1 -- 172.21.15.32:0/2507261633 shutdown_connections 2024-04-24T22:59:22.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.353+0000 7faefe33b640 1 --2- 172.21.15.32:0/2507261633 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7faee406be30 0x7faee406e2f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:22.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.353+0000 7faefe33b640 1 --2- 172.21.15.32:0/2507261633 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faef80cd3c0 0x7faef814cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:22.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.353+0000 7faefe33b640 1 --2- 172.21.15.32:0/2507261633 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faef80b6050 0x7faef814c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:22.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.353+0000 7faefe33b640 1 --2- 172.21.15.32:0/2507261633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faef80b5560 0x7faef8157e20 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:22.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.353+0000 7faefe33b640 1 -- 172.21.15.32:0/2507261633 >> 172.21.15.32:0/2507261633 conn(0x7faef806b110 msgr2=0x7faef80b16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:22.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.354+0000 7faefe33b640 1 -- 172.21.15.32:0/2507261633 shutdown_connections 2024-04-24T22:59:22.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22.354+0000 7faefe33b640 1 -- 172.21.15.32:0/2507261633 wait complete. 2024-04-24T22:59:22.366 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-24T22:59:22.366 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T22:59:22.366 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2024-04-24T22:59:22.366 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-24T22:59:22.366 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T22:59:22.366 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-24T22:59:22.366 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T22:59:22.367 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T22:59:22.367 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T22:59:22.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:59:22.367 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2024-04-24T22:59:22.383 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T22:59:22.383 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T22:59:22.383 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T22:59:22.383 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T22:59:22.383 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2024-04-24T22:59:22.383 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T22:59:22.383 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T22:59:22.384 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2024-04-24T22:59:22.384 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2024-04-24T22:59:22.384 INFO:teuthology.orchestra.run.smithi032.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T22:59:22.384 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2024-04-24T22:59:22.384 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2024-04-24T22:59:22.384 INFO:teuthology.orchestra.run.smithi032.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T22:59:22.384 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2024-04-24T22:59:22.384 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2024-04-24T22:59:22.384 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-24T22:59:22.384 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 42746sec preferred_lft 42746sec 2024-04-24T22:59:22.384 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link noprefixroute 2024-04-24T22:59:22.384 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T22:59:22.384 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T22:59:22.384 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2024-04-24T22:59:22.384 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2024-04-24T22:59:22.385 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-24T22:59:22.385 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T22:59:22.385 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-24T22:59:22.385 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link add name ceph-brx type bridge 2024-04-24T22:59:22.385 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip addr flush dev ceph-brx 2024-04-24T22:59:22.385 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set ceph-brx up 2024-04-24T22:59:22.385 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-24T22:59:22.386 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-24T22:59:22.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T22:59:22.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:59:22.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:59:22.569 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T22:59:22.594 INFO:teuthology.orchestra.run.smithi032.stdout:1 2024-04-24T22:59:22.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:59:22.596 DEBUG:teuthology.orchestra.run.smithi032:> route 2024-04-24T22:59:22.662 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2024-04-24T22:59:22.662 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T22:59:22.662 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-24T22:59:22.662 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-24T22:59:22.663 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T22:59:22.663 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T22:59:22.663 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-24T22:59:22.663 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-24T22:59:22.663 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-24T22:59:22.663 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-24T22:59:22.664 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-24T22:59:22.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T22:59:22.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:59:22.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:59:22.942 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-04-24T22:59:22.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:59:22.958 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list-id 2024-04-24T22:59:23.014 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T22:59:23.014 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-24T22:59:23.014 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T22:59:23.014 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-24T22:59:23.014 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-24T22:59:23.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T22:59:23.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:59:23.133 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-24T22:59:23.134 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T22:59:23.134 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-24T22:59:23.134 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-24T22:59:23.134 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-24T22:59:23.134 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-24T22:59:23.134 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-24T22:59:23.134 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-24T22:59:23.134 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-24T22:59:23.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T22:59:23.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:59:23.272 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T22:59:23.273 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-24T22:59:23.273 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2024-04-24T22:59:23.273 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set dev brx.0 master ceph-brx 2024-04-24T22:59:23.273 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-24T22:59:23.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T22:59:23.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:59:23.350 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T22:59:23.350 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-24T22:59:23.365 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-24T22:59:23.365 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T22:59:23.365 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-24T22:59:23.420 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-24T22:59:23.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:59:23.421 DEBUG:teuthology.orchestra.run.smithi032:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T22:59:23.514 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T22:59:23.514 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-24T22:59:23.625 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T22:59:23.626 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-24T22:59:23.626 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.121:3300,172.21.15.121:3301" 2024-04-24T22:59:23.626 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: trying mount with new device syntax: 0@eb1b7a01-6f65-4534-9a12-83011862ba51.cephfs=/ 2024-04-24T22:59:23.626 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.32:3300/172.21.15.121:3300/172.21.15.121:3301" will pass to kernel 2024-04-24T22:59:23.626 INFO:teuthology.orchestra.run.smithi032.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-24T22:59:23.627 INFO:teuthology.orchestra.run.smithi032.stdout: You just mounted a file system that supports labels which does not 2024-04-24T22:59:23.627 INFO:teuthology.orchestra.run.smithi032.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-24T22:59:23.627 INFO:teuthology.orchestra.run.smithi032.stdout: applications will generate AVC messages and not be allowed access to 2024-04-24T22:59:23.627 INFO:teuthology.orchestra.run.smithi032.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-24T22:59:23.628 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-24T22:59:23.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:59:23.628 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-24T22:59:23.653 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T22:59:23.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:23 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T22:59:23.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:59:23.755 INFO:teuthology.orchestra.run.smithi032.stdout:client4487 2024-04-24T22:59:23.755 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T22:59:23.755 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T22:59:23.771 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T22:59:23.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:23 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T22:59:23.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:59:23.875 INFO:teuthology.orchestra.run.smithi032.stdout:client4487 2024-04-24T22:59:23.875 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T22:59:23.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:23 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T22:59:23.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:59:23.937 INFO:teuthology.orchestra.run.smithi032.stdout:client4487 2024-04-24T22:59:23.938 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/eb1b7a01-6f65-4534-9a12-83011862ba51.client4487/status') 2024-04-24T22:59:23.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T22:59:24.000 INFO:teuthology.orchestra.run.smithi032.stdout:instance: client.4487 (3)172.21.15.32:0/1859363539 2024-04-24T22:59:24.000 INFO:teuthology.orchestra.run.smithi032.stdout:blocklisted: false 2024-04-24T22:59:24.001 INFO:teuthology.orchestra.run.smithi032.stderr:0+1 records in 2024-04-24T22:59:24.001 INFO:teuthology.orchestra.run.smithi032.stderr:0+1 records out 2024-04-24T22:59:24.001 INFO:teuthology.orchestra.run.smithi032.stderr:70 bytes copied, 2.4105e-05 s, 2.9 MB/s 2024-04-24T22:59:24.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:59:24.008 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2024-04-24T22:59:24.008 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-24T22:59:24.008 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-24T22:59:24.008 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T22:59:24.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.108+0000 7fccbd73b640 1 Processor -- start 2024-04-24T22:59:24.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.108+0000 7fccbd73b640 1 -- start start 2024-04-24T22:59:24.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.108+0000 7fccbd73b640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fccb815cc40 0x7fccb815d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:24.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.108+0000 7fccbd73b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fccb80d05b0 0x7fccb815d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:24.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.109+0000 7fccbd73b640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccb80b4ee0 con 0x7fccb80cec30 2024-04-24T22:59:24.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.109+0000 7fccbd73b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fccb806cca0 con 0x7fccb80d05b0 2024-04-24T22:59:24.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.109+0000 7fccbd73b640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fccb80ce5f0 con 0x7fccb815cc40 2024-04-24T22:59:24.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.109+0000 7fccaffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fccb80d05b0 0x7fccb815d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:24.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.109+0000 7fccaffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fccb80d05b0 0x7fccb815d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59838/0 (socket says 172.21.15.32:59838) 2024-04-24T22:59:24.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.109+0000 7fccaffff640 1 -- 172.21.15.32:0/2573108237 learned_addr learned my addr 172.21.15.32:0/2573108237 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:24.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.109+0000 7fccb6ffd640 1 --2- 172.21.15.32:0/2573108237 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fccb815cc40 0x7fccb815d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:24.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.109+0000 7fccaffff640 1 -- 172.21.15.32:0/2573108237 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fccb815cc40 msgr2=0x7fccb815d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:24.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.109+0000 7fccaffff640 1 --2- 172.21.15.32:0/2573108237 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fccb815cc40 0x7fccb815d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:24.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.109+0000 7fccaffff640 1 -- 172.21.15.32:0/2573108237 >> v1:172.21.15.121:6789/0 conn(0x7fccb80cec30 legacy=0x7fccb80cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:24.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.109+0000 7fccaffff640 1 -- 172.21.15.32:0/2573108237 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccb80b51c0 con 0x7fccb80d05b0 2024-04-24T22:59:24.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.110+0000 7fccaffff640 1 --2- 172.21.15.32:0/2573108237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccb80d05b0 0x7fccb815d560 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcca0009870 tx=0x7fcca002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd5ab812c8b0ff server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:24.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.110+0000 7fccb67fc640 1 -- 172.21.15.32:0/2573108237 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca0037070 con 0x7fccb80d05b0 2024-04-24T22:59:24.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.110+0000 7fccb67fc640 1 -- 172.21.15.32:0/2573108237 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcca0032440 con 0x7fccb80d05b0 2024-04-24T22:59:24.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.110+0000 7fccb67fc640 1 -- 172.21.15.32:0/2573108237 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca0032810 con 0x7fccb80d05b0 2024-04-24T22:59:24.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.110+0000 7fccbd73b640 1 -- 172.21.15.32:0/2573108237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccb80d05b0 msgr2=0x7fccb815d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:24.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.110+0000 7fccbd73b640 1 --2- 172.21.15.32:0/2573108237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccb80d05b0 0x7fccb815d560 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcca0009870 tx=0x7fcca002a210 comp rx=0 tx=0).stop 2024-04-24T22:59:24.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.111+0000 7fccbd73b640 1 -- 172.21.15.32:0/2573108237 shutdown_connections 2024-04-24T22:59:24.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.111+0000 7fccbd73b640 1 --2- 172.21.15.32:0/2573108237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccb80d05b0 0x7fccb815d560 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:24.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.111+0000 7fccbd73b640 1 --2- 172.21.15.32:0/2573108237 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fccb815cc40 0x7fccb815d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:24.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.111+0000 7fccbd73b640 1 -- 172.21.15.32:0/2573108237 >> 172.21.15.32:0/2573108237 conn(0x7fccb80d0ae0 msgr2=0x7fccb80d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:24.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.111+0000 7fccbd73b640 1 -- 172.21.15.32:0/2573108237 shutdown_connections 2024-04-24T22:59:24.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.111+0000 7fccbd73b640 1 -- 172.21.15.32:0/2573108237 wait complete. 2024-04-24T22:59:24.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.112+0000 7fccbd73b640 1 Processor -- start 2024-04-24T22:59:24.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.112+0000 7fccbd73b640 1 -- start start 2024-04-24T22:59:24.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.113+0000 7fccbd73b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccb80cec30 0x7fccb81f2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:24.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.113+0000 7fccbd73b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fccb80d05b0 0x7fccb814eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:24.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.113+0000 7fccbd73b640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fccb815cc40 0x7fccb814f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:24.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.113+0000 7fccbd73b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fccb80b5780 con 0x7fccb80cec30 2024-04-24T22:59:24.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.113+0000 7fccbd73b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fccb815daa0 con 0x7fccb80d05b0 2024-04-24T22:59:24.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.113+0000 7fccbd73b640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fccb80b5a60 con 0x7fccb815cc40 2024-04-24T22:59:24.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.113+0000 7fccb6ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccb80cec30 0x7fccb81f2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:24.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.113+0000 7fccb6ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccb80cec30 0x7fccb81f2e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59848/0 (socket says 172.21.15.32:59848) 2024-04-24T22:59:24.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.113+0000 7fccb6ffd640 1 -- 172.21.15.32:0/1943527884 learned_addr learned my addr 172.21.15.32:0/1943527884 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:24.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.113+0000 7fccaffff640 1 --2- 172.21.15.32:0/1943527884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fccb80d05b0 0x7fccb814eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:24.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.113+0000 7fccb77fe640 1 --2- 172.21.15.32:0/1943527884 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fccb815cc40 0x7fccb814f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:24.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.113+0000 7fccb6ffd640 1 -- 172.21.15.32:0/1943527884 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fccb815cc40 msgr2=0x7fccb814f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:24.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.113+0000 7fccb6ffd640 1 --2- 172.21.15.32:0/1943527884 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fccb815cc40 0x7fccb814f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:24.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.113+0000 7fccb6ffd640 1 -- 172.21.15.32:0/1943527884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fccb80d05b0 msgr2=0x7fccb814eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:24.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.113+0000 7fccb6ffd640 1 --2- 172.21.15.32:0/1943527884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fccb80d05b0 0x7fccb814eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:24.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.113+0000 7fccb6ffd640 1 -- 172.21.15.32:0/1943527884 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccb813d490 con 0x7fccb80cec30 2024-04-24T22:59:24.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.113+0000 7fccb6ffd640 1 --2- 172.21.15.32:0/1943527884 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccb80cec30 0x7fccb81f2e70 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcca800d800 tx=0x7fcca800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:24.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.113+0000 7fccaf7fe640 1 -- 172.21.15.32:0/1943527884 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca800b6c0 con 0x7fccb80cec30 2024-04-24T22:59:24.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.113+0000 7fccaf7fe640 1 -- 172.21.15.32:0/1943527884 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcca8019070 con 0x7fccb80cec30 2024-04-24T22:59:24.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.113+0000 7fccbd73b640 1 -- 172.21.15.32:0/1943527884 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccb813d770 con 0x7fccb80cec30 2024-04-24T22:59:24.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.113+0000 7fccaf7fe640 1 -- 172.21.15.32:0/1943527884 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca800ba80 con 0x7fccb80cec30 2024-04-24T22:59:24.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.113+0000 7fccbd73b640 1 -- 172.21.15.32:0/1943527884 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccb813dd30 con 0x7fccb80cec30 2024-04-24T22:59:24.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.115+0000 7fccbd73b640 1 -- 172.21.15.32:0/1943527884 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccb8068430 con 0x7fccb80cec30 2024-04-24T22:59:24.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.115+0000 7fccaf7fe640 1 -- 172.21.15.32:0/1943527884 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcca800bc20 con 0x7fccb80cec30 2024-04-24T22:59:24.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.115+0000 7fccaf7fe640 1 --2- 172.21.15.32:0/1943527884 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcc9806be60 0x7fcc9806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:24.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.115+0000 7fccaf7fe640 1 -- 172.21.15.32:0/1943527884 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fcca808b2d0 con 0x7fccb80cec30 2024-04-24T22:59:24.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.116+0000 7fccaffff640 1 --2- 172.21.15.32:0/1943527884 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcc9806be60 0x7fcc9806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:24.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.116+0000 7fccaffff640 1 --2- 172.21.15.32:0/1943527884 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcc9806be60 0x7fcc9806e320 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcca0034000 tx=0x7fcca002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:24.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.120+0000 7fccaf7fe640 1 -- 172.21.15.32:0/1943527884 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcca801c360 con 0x7fccb80cec30 2024-04-24T22:59:24.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.340+0000 7fccbd73b640 1 -- 172.21.15.32:0/1943527884 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fccb80d1600 con 0x7fccb80cec30 2024-04-24T22:59:24.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.341+0000 7fccaf7fe640 1 -- 172.21.15.32:0/1943527884 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) ==== 53+0+73 (secure 0 0 0) 0x7fcca8059c80 con 0x7fccb80cec30 2024-04-24T22:59:24.342 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T22:59:24.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.342+0000 7fccbd73b640 1 -- 172.21.15.32:0/1943527884 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcc9806be60 msgr2=0x7fcc9806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:24.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.342+0000 7fccbd73b640 1 --2- 172.21.15.32:0/1943527884 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcc9806be60 0x7fcc9806e320 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcca0034000 tx=0x7fcca002e000 comp rx=0 tx=0).stop 2024-04-24T22:59:24.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.342+0000 7fccbd73b640 1 -- 172.21.15.32:0/1943527884 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccb80cec30 msgr2=0x7fccb81f2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:24.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.342+0000 7fccbd73b640 1 --2- 172.21.15.32:0/1943527884 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccb80cec30 0x7fccb81f2e70 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcca800d800 tx=0x7fcca800dcc0 comp rx=0 tx=0).stop 2024-04-24T22:59:24.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.343+0000 7fccbd73b640 1 -- 172.21.15.32:0/1943527884 shutdown_connections 2024-04-24T22:59:24.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.343+0000 7fccbd73b640 1 --2- 172.21.15.32:0/1943527884 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcc9806be60 0x7fcc9806e320 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:24.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.343+0000 7fccbd73b640 1 --2- 172.21.15.32:0/1943527884 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fccb815cc40 0x7fccb814f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:24.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.343+0000 7fccbd73b640 1 --2- 172.21.15.32:0/1943527884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fccb80d05b0 0x7fccb814eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:24.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.343+0000 7fccbd73b640 1 --2- 172.21.15.32:0/1943527884 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccb80cec30 0x7fccb81f2e70 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:24.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.343+0000 7fccbd73b640 1 -- 172.21.15.32:0/1943527884 >> 172.21.15.32:0/1943527884 conn(0x7fccb80d0ae0 msgr2=0x7fccb80b1650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:24.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.343+0000 7fccbd73b640 1 -- 172.21.15.32:0/1943527884 shutdown_connections 2024-04-24T22:59:24.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24.343+0000 7fccbd73b640 1 -- 172.21.15.32:0/1943527884 wait complete. 2024-04-24T22:59:24.355 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-24T22:59:24.355 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T22:59:24.355 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2024-04-24T22:59:24.355 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-24T22:59:24.355 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-24T22:59:24.355 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-24T22:59:24.355 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T22:59:24.355 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T22:59:24.355 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-24T22:59:24.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:59:24.355 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2024-04-24T22:59:24.371 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T22:59:24.372 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T22:59:24.372 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T22:59:24.372 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T22:59:24.372 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2024-04-24T22:59:24.372 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T22:59:24.372 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T22:59:24.372 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2024-04-24T22:59:24.372 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2024-04-24T22:59:24.372 INFO:teuthology.orchestra.run.smithi032.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T22:59:24.372 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2024-04-24T22:59:24.372 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2024-04-24T22:59:24.372 INFO:teuthology.orchestra.run.smithi032.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T22:59:24.372 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2024-04-24T22:59:24.373 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2024-04-24T22:59:24.373 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-24T22:59:24.373 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 42744sec preferred_lft 42744sec 2024-04-24T22:59:24.373 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link noprefixroute 2024-04-24T22:59:24.373 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T22:59:24.373 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T22:59:24.373 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2024-04-24T22:59:24.373 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2024-04-24T22:59:24.373 INFO:teuthology.orchestra.run.smithi032.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T22:59:24.373 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 72:d4:96:00:43:8c brd ff:ff:ff:ff:ff:ff 2024-04-24T22:59:24.373 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-24T22:59:24.373 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T22:59:24.373 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::f4c6:20ff:fe22:2647/64 scope link 2024-04-24T22:59:24.373 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T22:59:24.373 INFO:teuthology.orchestra.run.smithi032.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T22:59:24.374 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 72:d4:96:00:43:8c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T22:59:24.374 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::70d4:96ff:fe00:438c/64 scope link tentative 2024-04-24T22:59:24.374 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T22:59:24.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:59:24.374 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T22:59:24.435 INFO:teuthology.orchestra.run.smithi032.stdout:1 2024-04-24T22:59:24.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:59:24.437 DEBUG:teuthology.orchestra.run.smithi032:> route 2024-04-24T22:59:24.494 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2024-04-24T22:59:24.494 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T22:59:24.494 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-24T22:59:24.494 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-24T22:59:24.494 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T22:59:24.494 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T22:59:24.495 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-24T22:59:24.495 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-24T22:59:24.495 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-24T22:59:24.495 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-24T22:59:24.495 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-24T22:59:24.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T22:59:24.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:59:24.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:59:24.621 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-04-24T22:59:24.636 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-24T22:59:24.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:59:24.637 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list-id 2024-04-24T22:59:24.691 INFO:teuthology.orchestra.run.smithi032.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-24T22:59:24.691 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T22:59:24.692 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-24T22:59:24.692 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T22:59:24.692 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-24T22:59:24.692 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-24T22:59:24.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T22:59:24.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:59:24.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:59:24.808 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-24T22:59:24.832 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T22:59:24.832 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T22:59:24.832 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T22:59:24.832 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T22:59:24.833 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2024-04-24T22:59:24.833 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T22:59:24.833 INFO:teuthology.orchestra.run.smithi032.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T22:59:24.833 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 2e:e6:79:3b:62:ab brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-24T22:59:24.833 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-24T22:59:24.833 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T22:59:24.833 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::2ce6:79ff:fe3b:62ab/64 scope link 2024-04-24T22:59:24.833 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T22:59:24.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:59:24.834 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-24T22:59:24.899 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T22:59:24.899 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T22:59:24.899 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T22:59:24.899 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T22:59:24.899 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2024-04-24T22:59:24.899 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T22:59:24.899 INFO:teuthology.orchestra.run.smithi032.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T22:59:24.899 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 2e:e6:79:3b:62:ab brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-24T22:59:24.899 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-24T22:59:24.899 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T22:59:24.899 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::2ce6:79ff:fe3b:62ab/64 scope link 2024-04-24T22:59:24.899 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T22:59:24.901 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-24T22:59:24.901 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T22:59:24.901 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-24T22:59:24.901 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-24T22:59:24.901 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-24T22:59:24.901 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-24T22:59:24.901 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-24T22:59:24.901 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-24T22:59:24.901 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-24T22:59:24.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T22:59:25.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:59:25.053 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T22:59:25.053 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-24T22:59:25.053 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.1 up 2024-04-24T22:59:25.053 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set dev brx.1 master ceph-brx 2024-04-24T22:59:25.053 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-24T22:59:25.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T22:59:25.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:59:25.130 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T22:59:25.130 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-24T22:59:25.145 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-24T22:59:25.146 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T22:59:25.146 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-24T22:59:25.200 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2024-04-24T22:59:25.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:59:25.200 DEBUG:teuthology.orchestra.run.smithi032:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T22:59:25.282 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T22:59:25.283 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-24T22:59:25.293 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T22:59:25.293 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-24T22:59:25.293 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.121:3300,172.21.15.121:3301" 2024-04-24T22:59:25.293 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: trying mount with new device syntax: 2@eb1b7a01-6f65-4534-9a12-83011862ba51.cephfs=/ 2024-04-24T22:59:25.294 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.32:3300/172.21.15.121:3300/172.21.15.121:3301" will pass to kernel 2024-04-24T22:59:25.294 INFO:teuthology.orchestra.run.smithi032.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-04-24T22:59:25.294 INFO:teuthology.orchestra.run.smithi032.stdout: You just mounted a file system that supports labels which does not 2024-04-24T22:59:25.294 INFO:teuthology.orchestra.run.smithi032.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-24T22:59:25.294 INFO:teuthology.orchestra.run.smithi032.stdout: applications will generate AVC messages and not be allowed access to 2024-04-24T22:59:25.294 INFO:teuthology.orchestra.run.smithi032.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-24T22:59:25.295 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-24T22:59:25.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:59:25.295 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-24T22:59:25.359 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T22:59:25.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T22:59:25.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:59:25.458 INFO:teuthology.orchestra.run.smithi032.stdout:client4692 2024-04-24T22:59:25.458 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T22:59:25.458 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T22:59:25.474 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T22:59:25.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T22:59:25.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:59:25.571 INFO:teuthology.orchestra.run.smithi032.stdout:client4692 2024-04-24T22:59:25.571 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T22:59:25.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T22:59:25.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:59:25.628 INFO:teuthology.orchestra.run.smithi032.stdout:client4692 2024-04-24T22:59:25.628 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/eb1b7a01-6f65-4534-9a12-83011862ba51.client4692/status') 2024-04-24T22:59:25.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T22:59:25.686 INFO:teuthology.orchestra.run.smithi032.stdout:instance: client.4692 (3)192.168.0.2:0/647880415 2024-04-24T22:59:25.686 INFO:teuthology.orchestra.run.smithi032.stdout:blocklisted: false 2024-04-24T22:59:25.687 INFO:teuthology.orchestra.run.smithi032.stderr:0+1 records in 2024-04-24T22:59:25.687 INFO:teuthology.orchestra.run.smithi032.stderr:0+1 records out 2024-04-24T22:59:25.687 INFO:teuthology.orchestra.run.smithi032.stderr:68 bytes copied, 2.3761e-05 s, 2.9 MB/s 2024-04-24T22:59:25.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:59:25.694 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2024-04-24T22:59:25.694 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-24T22:59:25.694 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-24T22:59:25.694 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T22:59:25.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.793+0000 7ff232a72640 1 Processor -- start 2024-04-24T22:59:25.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.793+0000 7ff232a72640 1 -- start start 2024-04-24T22:59:25.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.794+0000 7ff232a72640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff22c06cd00 0x7ff22c0ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:25.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.794+0000 7ff232a72640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff22c15cb00 0x7ff22c0ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:25.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.794+0000 7ff232a72640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff22c0b4520 con 0x7ff22c15b380 2024-04-24T22:59:25.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.794+0000 7ff232a72640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7ff22c0b4800 con 0x7ff22c15cb00 2024-04-24T22:59:25.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.794+0000 7ff232a72640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7ff22c0b4ae0 con 0x7ff22c06cd00 2024-04-24T22:59:25.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.794+0000 7ff23100f640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff22c15cb00 0x7ff22c0ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:25.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.794+0000 7ff23080e640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff22c15b380 0x7ff22c06c5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47752/0 (socket says 172.21.15.32:47752) 2024-04-24T22:59:25.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.794+0000 7ff23080e640 1 -- 172.21.15.32:0/2852727518 learned_addr learned my addr 172.21.15.32:0/2852727518 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:25.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.795+0000 7ff22bfff640 1 --2- 172.21.15.32:0/2852727518 >> v2:172.21.15.121:3300/0 conn(0x7ff22c06cd00 0x7ff22c0ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:25.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.795+0000 7ff23100f640 1 -- 172.21.15.32:0/2852727518 >> v2:172.21.15.121:3300/0 conn(0x7ff22c06cd00 msgr2=0x7ff22c0ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:25.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.795+0000 7ff23100f640 1 --2- 172.21.15.32:0/2852727518 >> v2:172.21.15.121:3300/0 conn(0x7ff22c06cd00 0x7ff22c0ce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:25.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.795+0000 7ff23100f640 1 -- 172.21.15.32:0/2852727518 >> v1:172.21.15.32:6789/0 conn(0x7ff22c15b380 legacy=0x7ff22c06c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:25.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.795+0000 7ff22b7fe640 1 -- 172.21.15.32:0/2852727518 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3009925643 0 0) 0x7ff22c0b4520 con 0x7ff22c15b380 2024-04-24T22:59:25.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.795+0000 7ff23100f640 1 -- 172.21.15.32:0/2852727518 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff22c0b4dc0 con 0x7ff22c15cb00 2024-04-24T22:59:25.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.795+0000 7ff22bfff640 1 --2- 172.21.15.32:0/2852727518 >> v2:172.21.15.121:3300/0 conn(0x7ff22c06cd00 0x7ff22c0ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:59:25.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.795+0000 7ff23100f640 1 --2- 172.21.15.32:0/2852727518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff22c15cb00 0x7ff22c0ce970 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff22c0b33c0 tx=0x7ff22402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fff4e9f9d08a68f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:25.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.796+0000 7ff22b7fe640 1 -- 172.21.15.32:0/2852727518 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff224007740 con 0x7ff22c15cb00 2024-04-24T22:59:25.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.796+0000 7ff22b7fe640 1 -- 172.21.15.32:0/2852727518 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2240078e0 con 0x7ff22c15cb00 2024-04-24T22:59:25.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.796+0000 7ff22b7fe640 1 -- 172.21.15.32:0/2852727518 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff224007cb0 con 0x7ff22c15cb00 2024-04-24T22:59:25.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.796+0000 7ff232a72640 1 -- 172.21.15.32:0/2852727518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff22c15cb00 msgr2=0x7ff22c0ce970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:25.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.796+0000 7ff232a72640 1 --2- 172.21.15.32:0/2852727518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff22c15cb00 0x7ff22c0ce970 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff22c0b33c0 tx=0x7ff22402a5f0 comp rx=0 tx=0).stop 2024-04-24T22:59:25.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.796+0000 7ff232a72640 1 -- 172.21.15.32:0/2852727518 shutdown_connections 2024-04-24T22:59:25.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.796+0000 7ff232a72640 1 --2- 172.21.15.32:0/2852727518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff22c15cb00 0x7ff22c0ce970 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:25.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.796+0000 7ff232a72640 1 --2- 172.21.15.32:0/2852727518 >> v2:172.21.15.121:3300/0 conn(0x7ff22c06cd00 0x7ff22c0ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:25.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.796+0000 7ff232a72640 1 -- 172.21.15.32:0/2852727518 >> 172.21.15.32:0/2852727518 conn(0x7ff22c06b110 msgr2=0x7ff22c06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:25.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.796+0000 7ff232a72640 1 -- 172.21.15.32:0/2852727518 shutdown_connections 2024-04-24T22:59:25.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.796+0000 7ff232a72640 1 -- 172.21.15.32:0/2852727518 wait complete. 2024-04-24T22:59:25.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.797+0000 7ff232a72640 1 Processor -- start 2024-04-24T22:59:25.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.797+0000 7ff232a72640 1 -- start start 2024-04-24T22:59:25.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.798+0000 7ff232a72640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff22c06cd00 0x7ff22c15a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:25.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.798+0000 7ff232a72640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff22c15b380 0x7ff22c071ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:25.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.798+0000 7ff232a72640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff22c15cb00 0x7ff22c157110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:25.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.798+0000 7ff232a72640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff22c15b760 con 0x7ff22c15cb00 2024-04-24T22:59:25.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.798+0000 7ff232a72640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff22c0ceeb0 con 0x7ff22c06cd00 2024-04-24T22:59:25.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.798+0000 7ff232a72640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff22c0b55c0 con 0x7ff22c15b380 2024-04-24T22:59:25.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.798+0000 7ff23100f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff22c15cb00 0x7ff22c157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:25.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.798+0000 7ff23100f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff22c15cb00 0x7ff22c157110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59868/0 (socket says 172.21.15.32:59868) 2024-04-24T22:59:25.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.798+0000 7ff23100f640 1 -- 172.21.15.32:0/2231123028 learned_addr learned my addr 172.21.15.32:0/2231123028 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:25.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.799+0000 7ff23080e640 1 --2- 172.21.15.32:0/2231123028 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff22c06cd00 0x7ff22c15a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:25.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.799+0000 7ff22bfff640 1 --2- 172.21.15.32:0/2231123028 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff22c15b380 0x7ff22c071ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:25.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.799+0000 7ff23100f640 1 -- 172.21.15.32:0/2231123028 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff22c15b380 msgr2=0x7ff22c071ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:25.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.799+0000 7ff23100f640 1 --2- 172.21.15.32:0/2231123028 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff22c15b380 0x7ff22c071ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:25.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.799+0000 7ff23100f640 1 -- 172.21.15.32:0/2231123028 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff22c06cd00 msgr2=0x7ff22c15a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:25.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.799+0000 7ff23100f640 1 --2- 172.21.15.32:0/2231123028 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff22c06cd00 0x7ff22c15a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:25.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.799+0000 7ff23100f640 1 -- 172.21.15.32:0/2231123028 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff22c15aa90 con 0x7ff22c15cb00 2024-04-24T22:59:25.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.799+0000 7ff22bfff640 1 --2- 172.21.15.32:0/2231123028 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff22c15b380 0x7ff22c071ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:59:25.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.799+0000 7ff23080e640 1 --2- 172.21.15.32:0/2231123028 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff22c06cd00 0x7ff22c15a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:59:25.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.799+0000 7ff23100f640 1 --2- 172.21.15.32:0/2231123028 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff22c15cb00 0x7ff22c157110 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff224009e50 tx=0x7ff224004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:25.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.799+0000 7ff2297fa640 1 -- 172.21.15.32:0/2231123028 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22403f070 con 0x7ff22c15cb00 2024-04-24T22:59:25.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.799+0000 7ff2297fa640 1 -- 172.21.15.32:0/2231123028 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff224031cc0 con 0x7ff22c15cb00 2024-04-24T22:59:25.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.799+0000 7ff232a72640 1 -- 172.21.15.32:0/2231123028 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff22c152bf0 con 0x7ff22c15cb00 2024-04-24T22:59:25.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.799+0000 7ff2297fa640 1 -- 172.21.15.32:0/2231123028 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff224044680 con 0x7ff22c15cb00 2024-04-24T22:59:25.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.799+0000 7ff232a72640 1 -- 172.21.15.32:0/2231123028 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff22c1531b0 con 0x7ff22c15cb00 2024-04-24T22:59:25.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.800+0000 7ff232a72640 1 -- 172.21.15.32:0/2231123028 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff22c068430 con 0x7ff22c15cb00 2024-04-24T22:59:25.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.800+0000 7ff2297fa640 1 -- 172.21.15.32:0/2231123028 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff22403b410 con 0x7ff22c15cb00 2024-04-24T22:59:25.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.801+0000 7ff2297fa640 1 --2- 172.21.15.32:0/2231123028 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff21006be60 0x7ff21006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:25.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.801+0000 7ff2297fa640 1 -- 172.21.15.32:0/2231123028 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff2240ac6c0 con 0x7ff22c15cb00 2024-04-24T22:59:25.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.801+0000 7ff23080e640 1 --2- 172.21.15.32:0/2231123028 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff21006be60 0x7ff21006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:25.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.802+0000 7ff23080e640 1 --2- 172.21.15.32:0/2231123028 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff21006be60 0x7ff21006e320 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff220000f60 tx=0x7ff220009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:25.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:25.805+0000 7ff2297fa640 1 -- 172.21.15.32:0/2231123028 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff22407b1c0 con 0x7ff22c15cb00 2024-04-24T22:59:26.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:26.026+0000 7ff232a72640 1 -- 172.21.15.32:0/2231123028 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff22c155620 con 0x7ff22c15cb00 2024-04-24T22:59:26.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:26.027+0000 7ff2297fa640 1 -- 172.21.15.32:0/2231123028 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) ==== 53+0+73 (secure 0 0 0) 0x7ff22407aac0 con 0x7ff22c15cb00 2024-04-24T22:59:26.028 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T22:59:26.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:26.029+0000 7ff232a72640 1 -- 172.21.15.32:0/2231123028 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff21006be60 msgr2=0x7ff21006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:26.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:26.029+0000 7ff232a72640 1 --2- 172.21.15.32:0/2231123028 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff21006be60 0x7ff21006e320 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff220000f60 tx=0x7ff220009040 comp rx=0 tx=0).stop 2024-04-24T22:59:26.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:26.029+0000 7ff232a72640 1 -- 172.21.15.32:0/2231123028 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff22c15cb00 msgr2=0x7ff22c157110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:26.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:26.029+0000 7ff232a72640 1 --2- 172.21.15.32:0/2231123028 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff22c15cb00 0x7ff22c157110 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff224009e50 tx=0x7ff224004a40 comp rx=0 tx=0).stop 2024-04-24T22:59:26.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:26.029+0000 7ff232a72640 1 -- 172.21.15.32:0/2231123028 shutdown_connections 2024-04-24T22:59:26.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:26.029+0000 7ff232a72640 1 --2- 172.21.15.32:0/2231123028 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff21006be60 0x7ff21006e320 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:26.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:26.029+0000 7ff232a72640 1 --2- 172.21.15.32:0/2231123028 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff22c15cb00 0x7ff22c157110 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:26.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:26.029+0000 7ff232a72640 1 --2- 172.21.15.32:0/2231123028 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff22c15b380 0x7ff22c071ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:26.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:26.029+0000 7ff232a72640 1 --2- 172.21.15.32:0/2231123028 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff22c06cd00 0x7ff22c15a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:26.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:26.029+0000 7ff232a72640 1 -- 172.21.15.32:0/2231123028 >> 172.21.15.32:0/2231123028 conn(0x7ff22c06b110 msgr2=0x7ff22c0cd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:26.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:26.029+0000 7ff232a72640 1 -- 172.21.15.32:0/2231123028 shutdown_connections 2024-04-24T22:59:26.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:26.029+0000 7ff232a72640 1 -- 172.21.15.32:0/2231123028 wait complete. 2024-04-24T22:59:26.041 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-24T22:59:26.041 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T22:59:26.041 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi121.front.sepia.ceph.com 2024-04-24T22:59:26.041 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-24T22:59:26.041 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-24T22:59:26.041 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-24T22:59:26.041 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T22:59:26.042 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T22:59:26.042 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-24T22:59:26.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:59:26.042 DEBUG:teuthology.orchestra.run.smithi121:> ip addr 2024-04-24T22:59:26.061 INFO:teuthology.orchestra.run.smithi121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T22:59:26.061 INFO:teuthology.orchestra.run.smithi121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T22:59:26.061 INFO:teuthology.orchestra.run.smithi121.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T22:59:26.061 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T22:59:26.061 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 ::1/128 scope host 2024-04-24T22:59:26.061 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T22:59:26.062 INFO:teuthology.orchestra.run.smithi121.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T22:59:26.062 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:82 brd ff:ff:ff:ff:ff:ff 2024-04-24T22:59:26.062 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f0 2024-04-24T22:59:26.062 INFO:teuthology.orchestra.run.smithi121.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T22:59:26.062 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:83 brd ff:ff:ff:ff:ff:ff 2024-04-24T22:59:26.062 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f1 2024-04-24T22:59:26.062 INFO:teuthology.orchestra.run.smithi121.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T22:59:26.062 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a2 brd ff:ff:ff:ff:ff:ff 2024-04-24T22:59:26.062 INFO:teuthology.orchestra.run.smithi121.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T22:59:26.062 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a3 brd ff:ff:ff:ff:ff:ff 2024-04-24T22:59:26.062 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.21.15.121/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-24T22:59:26.062 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft 42765sec preferred_lft 42765sec 2024-04-24T22:59:26.062 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::93dd:8489:4369:e127/64 scope link noprefixroute 2024-04-24T22:59:26.062 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T22:59:26.063 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-24T22:59:26.063 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T22:59:26.063 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2024-04-24T22:59:26.063 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link add name ceph-brx type bridge 2024-04-24T22:59:26.063 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip addr flush dev ceph-brx 2024-04-24T22:59:26.063 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set ceph-brx up 2024-04-24T22:59:26.063 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-24T22:59:26.063 DEBUG:teuthology.orchestra.run.smithi121:> ') 2024-04-24T22:59:26.157 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:59:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T22:59:26.241 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:59:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:59:26.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:59:26.248 DEBUG:teuthology.orchestra.run.smithi121:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T22:59:26.270 INFO:teuthology.orchestra.run.smithi121.stdout:1 2024-04-24T22:59:26.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:59:26.272 DEBUG:teuthology.orchestra.run.smithi121:> route 2024-04-24T22:59:26.350 INFO:teuthology.orchestra.run.smithi121.stdout:Kernel IP routing table 2024-04-24T22:59:26.350 INFO:teuthology.orchestra.run.smithi121.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T22:59:26.350 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-24T22:59:26.351 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-24T22:59:26.351 INFO:teuthology.orchestra.run.smithi121.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T22:59:26.351 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T22:59:26.351 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2024-04-24T22:59:26.351 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-24T22:59:26.351 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-24T22:59:26.351 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-24T22:59:26.351 DEBUG:teuthology.orchestra.run.smithi121:> ') 2024-04-24T22:59:26.438 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:59:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T22:59:26.599 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:59:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:59:26.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:59:26.605 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2024-04-24T22:59:26.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:59:26.620 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list-id 2024-04-24T22:59:26.675 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T22:59:26.675 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2024-04-24T22:59:26.675 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T22:59:26.675 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-24T22:59:26.675 DEBUG:teuthology.orchestra.run.smithi121:> ') 2024-04-24T22:59:26.762 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:59:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T22:59:26.784 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:59:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:59:26.790 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-24T22:59:26.790 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T22:59:26.790 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2024-04-24T22:59:26.790 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-24T22:59:26.790 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-24T22:59:26.791 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-24T22:59:26.791 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-24T22:59:26.791 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-24T22:59:26.791 DEBUG:teuthology.orchestra.run.smithi121:> ') 2024-04-24T22:59:26.839 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:59:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T22:59:26.907 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:59:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:59:26.913 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T22:59:26.913 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2024-04-24T22:59:26.913 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set brx.0 up 2024-04-24T22:59:26.913 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set dev brx.0 master ceph-brx 2024-04-24T22:59:26.913 DEBUG:teuthology.orchestra.run.smithi121:> ') 2024-04-24T22:59:26.961 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:59:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T22:59:26.982 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:59:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:59:26.988 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T22:59:26.988 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-24T22:59:27.003 INFO:teuthology.orchestra.run.smithi121.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-24T22:59:27.003 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T22:59:27.003 DEBUG:teuthology.orchestra.run.smithi121:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-24T22:59:27.057 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-24T22:59:27.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:59:27.058 DEBUG:teuthology.orchestra.run.smithi121:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T22:59:27.154 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T22:59:27.155 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-24T22:59:27.260 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T22:59:27.260 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-24T22:59:27.260 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.121:3300,172.21.15.121:3301" 2024-04-24T22:59:27.260 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: trying mount with new device syntax: 1@eb1b7a01-6f65-4534-9a12-83011862ba51.cephfs=/ 2024-04-24T22:59:27.260 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.32:3300/172.21.15.121:3300/172.21.15.121:3301" will pass to kernel 2024-04-24T22:59:27.261 INFO:teuthology.orchestra.run.smithi121.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-24T22:59:27.261 INFO:teuthology.orchestra.run.smithi121.stdout: You just mounted a file system that supports labels which does not 2024-04-24T22:59:27.261 INFO:teuthology.orchestra.run.smithi121.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-24T22:59:27.261 INFO:teuthology.orchestra.run.smithi121.stdout: applications will generate AVC messages and not be allowed access to 2024-04-24T22:59:27.261 INFO:teuthology.orchestra.run.smithi121.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-24T22:59:27.263 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-24T22:59:27.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:59:27.263 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-24T22:59:27.285 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T22:59:27.374 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:59:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T22:59:27.375 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:59:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:59:27.381 INFO:teuthology.orchestra.run.smithi121.stdout:client4496 2024-04-24T22:59:27.381 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T22:59:27.381 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T22:59:27.396 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T22:59:27.483 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:59:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T22:59:27.484 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:59:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:59:27.490 INFO:teuthology.orchestra.run.smithi121.stdout:client4496 2024-04-24T22:59:27.490 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T22:59:27.538 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:59:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T22:59:27.539 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:59:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:59:27.545 INFO:teuthology.orchestra.run.smithi121.stdout:client4496 2024-04-24T22:59:27.545 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/eb1b7a01-6f65-4534-9a12-83011862ba51.client4496/status') 2024-04-24T22:59:27.592 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:59:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T22:59:27.603 INFO:teuthology.orchestra.run.smithi121.stdout:instance: client.4496 (3)192.168.0.1:0/2044026729 2024-04-24T22:59:27.603 INFO:teuthology.orchestra.run.smithi121.stdout:blocklisted: false 2024-04-24T22:59:27.603 INFO:teuthology.orchestra.run.smithi121.stderr:0+1 records in 2024-04-24T22:59:27.603 INFO:teuthology.orchestra.run.smithi121.stderr:0+1 records out 2024-04-24T22:59:27.603 INFO:teuthology.orchestra.run.smithi121.stderr:69 bytes copied, 2.3007e-05 s, 3.0 MB/s 2024-04-24T22:59:27.604 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:59:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:59:27.609 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2024-04-24T22:59:27.609 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-24T22:59:27.609 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-24T22:59:27.609 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T22:59:27.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.721+0000 7fc17fc38640 1 Processor -- start 2024-04-24T22:59:27.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.722+0000 7fc17fc38640 1 -- start start 2024-04-24T22:59:27.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.722+0000 7fc17fc38640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc17815e9b0 0x7fc1780d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:27.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.722+0000 7fc17fc38640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc17806b820 0x7fc1780b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:27.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.722+0000 7fc17fc38640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1780d3170 con 0x7fc1780d4210 2024-04-24T22:59:27.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.722+0000 7fc17fc38640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fc1780d3450 con 0x7fc17815e9b0 2024-04-24T22:59:27.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.722+0000 7fc17fc38640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fc1780d3730 con 0x7fc17806b820 2024-04-24T22:59:27.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.723+0000 7fc17d9d4640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc17815e9b0 0x7fc1780d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:27.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.723+0000 7fc17d9d4640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc17815e9b0 0x7fc1780d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59872/0 (socket says 172.21.15.32:59872) 2024-04-24T22:59:27.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.723+0000 7fc17d9d4640 1 -- 172.21.15.32:0/3462534896 learned_addr learned my addr 172.21.15.32:0/3462534896 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:27.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.723+0000 7fc17e1d5640 1 --2- 172.21.15.32:0/3462534896 >> v2:172.21.15.121:3300/0 conn(0x7fc17806b820 0x7fc1780b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:27.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.723+0000 7fc17d9d4640 1 -- 172.21.15.32:0/3462534896 >> v2:172.21.15.121:3300/0 conn(0x7fc17806b820 msgr2=0x7fc1780b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:27.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.723+0000 7fc17d9d4640 1 --2- 172.21.15.32:0/3462534896 >> v2:172.21.15.121:3300/0 conn(0x7fc17806b820 0x7fc1780b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:27.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.723+0000 7fc17d9d4640 1 -- 172.21.15.32:0/3462534896 >> v1:172.21.15.121:6789/0 conn(0x7fc1780d4210 legacy=0x7fc17806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:27.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.723+0000 7fc17d9d4640 1 -- 172.21.15.32:0/3462534896 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1780d3a10 con 0x7fc17815e9b0 2024-04-24T22:59:27.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.724+0000 7fc17d9d4640 1 --2- 172.21.15.32:0/3462534896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc17815e9b0 0x7fc1780d2b90 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc1781622f0 tx=0x7fc17002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c4903f34a60124 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:27.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.724+0000 7fc17c9d2640 1 -- 172.21.15.32:0/3462534896 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc170037070 con 0x7fc17815e9b0 2024-04-24T22:59:27.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.724+0000 7fc17c9d2640 1 -- 172.21.15.32:0/3462534896 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc170032440 con 0x7fc17815e9b0 2024-04-24T22:59:27.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.724+0000 7fc17c9d2640 1 -- 172.21.15.32:0/3462534896 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc170032810 con 0x7fc17815e9b0 2024-04-24T22:59:27.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.724+0000 7fc17fc38640 1 -- 172.21.15.32:0/3462534896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc17815e9b0 msgr2=0x7fc1780d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:27.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.724+0000 7fc17fc38640 1 --2- 172.21.15.32:0/3462534896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc17815e9b0 0x7fc1780d2b90 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc1781622f0 tx=0x7fc17002a1e0 comp rx=0 tx=0).stop 2024-04-24T22:59:27.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.724+0000 7fc17fc38640 1 -- 172.21.15.32:0/3462534896 shutdown_connections 2024-04-24T22:59:27.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.724+0000 7fc17fc38640 1 --2- 172.21.15.32:0/3462534896 >> v2:172.21.15.121:3300/0 conn(0x7fc17806b820 0x7fc1780b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:27.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.724+0000 7fc17fc38640 1 --2- 172.21.15.32:0/3462534896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc17815e9b0 0x7fc1780d2b90 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:27.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.724+0000 7fc17fc38640 1 -- 172.21.15.32:0/3462534896 >> 172.21.15.32:0/3462534896 conn(0x7fc1780d11d0 msgr2=0x7fc1780d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:27.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.724+0000 7fc17fc38640 1 -- 172.21.15.32:0/3462534896 shutdown_connections 2024-04-24T22:59:27.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.724+0000 7fc17fc38640 1 -- 172.21.15.32:0/3462534896 wait complete. 2024-04-24T22:59:27.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.726+0000 7fc17fc38640 1 Processor -- start 2024-04-24T22:59:27.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.726+0000 7fc17fc38640 1 -- start start 2024-04-24T22:59:27.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.726+0000 7fc17fc38640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc17806b820 0x7fc1780cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:27.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.726+0000 7fc17fc38640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1780d4210 0x7fc1781f8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:27.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.726+0000 7fc17fc38640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc17815e9b0 0x7fc1781f3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:27.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.726+0000 7fc17fc38640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc1780ba0e0 con 0x7fc17806b820 2024-04-24T22:59:27.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.726+0000 7fc17fc38640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc1780d2400 con 0x7fc1780d4210 2024-04-24T22:59:27.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.726+0000 7fc17fc38640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc1780ba260 con 0x7fc17815e9b0 2024-04-24T22:59:27.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.726+0000 7fc17d9d4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc17806b820 0x7fc1780cddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:27.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.726+0000 7fc17d9d4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc17806b820 0x7fc1780cddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59874/0 (socket says 172.21.15.32:59874) 2024-04-24T22:59:27.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.726+0000 7fc17d9d4640 1 -- 172.21.15.32:0/3562209999 learned_addr learned my addr 172.21.15.32:0/3562209999 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:27.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.727+0000 7fc17d1d3640 1 --2- 172.21.15.32:0/3562209999 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1780d4210 0x7fc1781f8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:27.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.727+0000 7fc17e1d5640 1 --2- 172.21.15.32:0/3562209999 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc17815e9b0 0x7fc1781f3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:27.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.727+0000 7fc17d9d4640 1 -- 172.21.15.32:0/3562209999 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc17815e9b0 msgr2=0x7fc1781f3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:27.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.727+0000 7fc17d9d4640 1 --2- 172.21.15.32:0/3562209999 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc17815e9b0 0x7fc1781f3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:27.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.727+0000 7fc17d9d4640 1 -- 172.21.15.32:0/3562209999 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1780d4210 msgr2=0x7fc1781f8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:27.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.727+0000 7fc17d9d4640 1 --2- 172.21.15.32:0/3562209999 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1780d4210 0x7fc1781f8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:27.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.727+0000 7fc17d9d4640 1 -- 172.21.15.32:0/3562209999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1781f8a90 con 0x7fc17806b820 2024-04-24T22:59:27.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.727+0000 7fc17e1d5640 1 --2- 172.21.15.32:0/3562209999 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc17815e9b0 0x7fc1781f3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:27.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.727+0000 7fc17d9d4640 1 --2- 172.21.15.32:0/3562209999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc17806b820 0x7fc1780cddc0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc17002a220 tx=0x7fc1700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:27.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.727+0000 7fc1767fc640 1 -- 172.21.15.32:0/3562209999 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc170041020 con 0x7fc17806b820 2024-04-24T22:59:27.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.727+0000 7fc1767fc640 1 -- 172.21.15.32:0/3562209999 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1700306c0 con 0x7fc17806b820 2024-04-24T22:59:27.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.727+0000 7fc1767fc640 1 -- 172.21.15.32:0/3562209999 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc170037040 con 0x7fc17806b820 2024-04-24T22:59:27.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.727+0000 7fc17fc38640 1 -- 172.21.15.32:0/3562209999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1780ccd20 con 0x7fc17806b820 2024-04-24T22:59:27.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.727+0000 7fc17fc38640 1 -- 172.21.15.32:0/3562209999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1780cd2e0 con 0x7fc17806b820 2024-04-24T22:59:27.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.728+0000 7fc17fc38640 1 -- 172.21.15.32:0/3562209999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc178068430 con 0x7fc17806b820 2024-04-24T22:59:27.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.729+0000 7fc1767fc640 1 -- 172.21.15.32:0/3562209999 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc170044020 con 0x7fc17806b820 2024-04-24T22:59:27.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.729+0000 7fc1767fc640 1 --2- 172.21.15.32:0/3562209999 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc16406be30 0x7fc16406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:27.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.729+0000 7fc1767fc640 1 -- 172.21.15.32:0/3562209999 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc1700ab5c0 con 0x7fc17806b820 2024-04-24T22:59:27.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.729+0000 7fc17d1d3640 1 --2- 172.21.15.32:0/3562209999 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc16406be30 0x7fc16406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:27.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.730+0000 7fc17d1d3640 1 --2- 172.21.15.32:0/3562209999 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc16406be30 0x7fc16406e2f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc17806b490 tx=0x7fc16c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:27.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.733+0000 7fc1767fc640 1 -- 172.21.15.32:0/3562209999 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc17004b030 con 0x7fc17806b820 2024-04-24T22:59:27.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.955+0000 7fc17fc38640 1 -- 172.21.15.32:0/3562209999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc1780b9a20 con 0x7fc17806b820 2024-04-24T22:59:27.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.955+0000 7fc1767fc640 1 -- 172.21.15.32:0/3562209999 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) ==== 53+0+73 (secure 0 0 0) 0x7fc1781f8a90 con 0x7fc17806b820 2024-04-24T22:59:27.957 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T22:59:27.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.957+0000 7fc17fc38640 1 -- 172.21.15.32:0/3562209999 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc16406be30 msgr2=0x7fc16406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:27.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.957+0000 7fc17fc38640 1 --2- 172.21.15.32:0/3562209999 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc16406be30 0x7fc16406e2f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc17806b490 tx=0x7fc16c009040 comp rx=0 tx=0).stop 2024-04-24T22:59:27.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.957+0000 7fc17fc38640 1 -- 172.21.15.32:0/3562209999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc17806b820 msgr2=0x7fc1780cddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:27.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.957+0000 7fc17fc38640 1 --2- 172.21.15.32:0/3562209999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc17806b820 0x7fc1780cddc0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc17002a220 tx=0x7fc1700046c0 comp rx=0 tx=0).stop 2024-04-24T22:59:27.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.957+0000 7fc17fc38640 1 -- 172.21.15.32:0/3562209999 shutdown_connections 2024-04-24T22:59:27.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.957+0000 7fc17fc38640 1 --2- 172.21.15.32:0/3562209999 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc16406be30 0x7fc16406e2f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:27.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.957+0000 7fc17fc38640 1 --2- 172.21.15.32:0/3562209999 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc17815e9b0 0x7fc1781f3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:27.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.957+0000 7fc17fc38640 1 --2- 172.21.15.32:0/3562209999 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1780d4210 0x7fc1781f8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:27.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.957+0000 7fc17fc38640 1 --2- 172.21.15.32:0/3562209999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc17806b820 0x7fc1780cddc0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:27.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.957+0000 7fc17fc38640 1 -- 172.21.15.32:0/3562209999 >> 172.21.15.32:0/3562209999 conn(0x7fc1780d11d0 msgr2=0x7fc17815bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:27.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.957+0000 7fc17fc38640 1 -- 172.21.15.32:0/3562209999 shutdown_connections 2024-04-24T22:59:27.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:27.957+0000 7fc17fc38640 1 -- 172.21.15.32:0/3562209999 wait complete. 2024-04-24T22:59:27.969 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-24T22:59:27.970 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T22:59:27.970 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi121.front.sepia.ceph.com 2024-04-24T22:59:27.970 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-24T22:59:27.970 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-24T22:59:27.970 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-24T22:59:27.970 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T22:59:27.970 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T22:59:27.970 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-24T22:59:27.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:59:27.970 DEBUG:teuthology.orchestra.run.smithi121:> ip addr 2024-04-24T22:59:27.986 INFO:teuthology.orchestra.run.smithi121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T22:59:27.986 INFO:teuthology.orchestra.run.smithi121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T22:59:27.986 INFO:teuthology.orchestra.run.smithi121.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T22:59:27.986 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T22:59:27.987 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 ::1/128 scope host 2024-04-24T22:59:27.987 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T22:59:27.987 INFO:teuthology.orchestra.run.smithi121.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T22:59:27.987 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:82 brd ff:ff:ff:ff:ff:ff 2024-04-24T22:59:27.987 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f0 2024-04-24T22:59:27.987 INFO:teuthology.orchestra.run.smithi121.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T22:59:27.987 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:83 brd ff:ff:ff:ff:ff:ff 2024-04-24T22:59:27.987 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f1 2024-04-24T22:59:27.987 INFO:teuthology.orchestra.run.smithi121.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T22:59:27.987 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a2 brd ff:ff:ff:ff:ff:ff 2024-04-24T22:59:27.987 INFO:teuthology.orchestra.run.smithi121.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T22:59:27.987 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a3 brd ff:ff:ff:ff:ff:ff 2024-04-24T22:59:27.987 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.21.15.121/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-24T22:59:27.987 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft 42763sec preferred_lft 42763sec 2024-04-24T22:59:27.988 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::93dd:8489:4369:e127/64 scope link noprefixroute 2024-04-24T22:59:27.988 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T22:59:27.988 INFO:teuthology.orchestra.run.smithi121.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T22:59:27.988 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether e2:ca:b9:e9:9e:bb brd ff:ff:ff:ff:ff:ff 2024-04-24T22:59:27.988 INFO:teuthology.orchestra.run.smithi121.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-24T22:59:27.988 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T22:59:27.988 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::80fe:d2ff:fe13:d1/64 scope link 2024-04-24T22:59:27.988 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T22:59:27.988 INFO:teuthology.orchestra.run.smithi121.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T22:59:27.988 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether e2:ca:b9:e9:9e:bb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T22:59:27.988 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::e0ca:b9ff:fee9:9ebb/64 scope link tentative 2024-04-24T22:59:27.988 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T22:59:27.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:59:27.989 DEBUG:teuthology.orchestra.run.smithi121:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T22:59:28.049 INFO:teuthology.orchestra.run.smithi121.stdout:1 2024-04-24T22:59:28.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:59:28.051 DEBUG:teuthology.orchestra.run.smithi121:> route 2024-04-24T22:59:28.106 INFO:teuthology.orchestra.run.smithi121.stdout:Kernel IP routing table 2024-04-24T22:59:28.107 INFO:teuthology.orchestra.run.smithi121.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T22:59:28.107 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-24T22:59:28.107 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-24T22:59:28.107 INFO:teuthology.orchestra.run.smithi121.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T22:59:28.107 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T22:59:28.107 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2024-04-24T22:59:28.107 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-24T22:59:28.107 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-24T22:59:28.107 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-24T22:59:28.107 DEBUG:teuthology.orchestra.run.smithi121:> ') 2024-04-24T22:59:28.195 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:59:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T22:59:28.225 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:59:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:59:28.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:59:28.231 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2024-04-24T22:59:28.247 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-24T22:59:28.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:59:28.248 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list-id 2024-04-24T22:59:28.302 INFO:teuthology.orchestra.run.smithi121.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-24T22:59:28.302 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T22:59:28.303 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2024-04-24T22:59:28.303 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-24T22:59:28.303 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-24T22:59:28.303 DEBUG:teuthology.orchestra.run.smithi121:> ') 2024-04-24T22:59:28.389 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:59:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T22:59:28.411 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:59:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:59:28.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:59:28.417 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-24T22:59:28.440 INFO:teuthology.orchestra.run.smithi121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T22:59:28.440 INFO:teuthology.orchestra.run.smithi121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T22:59:28.440 INFO:teuthology.orchestra.run.smithi121.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T22:59:28.440 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T22:59:28.440 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 ::1/128 scope host 2024-04-24T22:59:28.440 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T22:59:28.440 INFO:teuthology.orchestra.run.smithi121.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T22:59:28.440 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 32:b7:06:ea:d4:84 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-24T22:59:28.440 INFO:teuthology.orchestra.run.smithi121.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-24T22:59:28.440 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T22:59:28.441 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::30b7:6ff:feea:d484/64 scope link 2024-04-24T22:59:28.441 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T22:59:28.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:59:28.443 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-24T22:59:28.506 INFO:teuthology.orchestra.run.smithi121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T22:59:28.506 INFO:teuthology.orchestra.run.smithi121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T22:59:28.506 INFO:teuthology.orchestra.run.smithi121.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T22:59:28.506 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T22:59:28.506 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 ::1/128 scope host 2024-04-24T22:59:28.507 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T22:59:28.507 INFO:teuthology.orchestra.run.smithi121.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T22:59:28.507 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 32:b7:06:ea:d4:84 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-24T22:59:28.507 INFO:teuthology.orchestra.run.smithi121.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-24T22:59:28.507 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T22:59:28.507 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::30b7:6ff:feea:d484/64 scope link 2024-04-24T22:59:28.507 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T22:59:28.508 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-24T22:59:28.508 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T22:59:28.508 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2024-04-24T22:59:28.508 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-24T22:59:28.508 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-24T22:59:28.508 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-24T22:59:28.509 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-24T22:59:28.509 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-24T22:59:28.509 DEBUG:teuthology.orchestra.run.smithi121:> ') 2024-04-24T22:59:28.596 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:59:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T22:59:28.648 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:59:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:59:28.654 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T22:59:28.654 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2024-04-24T22:59:28.654 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set brx.1 up 2024-04-24T22:59:28.655 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set dev brx.1 master ceph-brx 2024-04-24T22:59:28.655 DEBUG:teuthology.orchestra.run.smithi121:> ') 2024-04-24T22:59:28.702 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:59:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T22:59:28.722 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:59:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:59:28.728 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T22:59:28.728 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-24T22:59:28.755 INFO:teuthology.orchestra.run.smithi121.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-24T22:59:28.755 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T22:59:28.755 DEBUG:teuthology.orchestra.run.smithi121:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-24T22:59:28.810 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2024-04-24T22:59:28.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:59:28.810 DEBUG:teuthology.orchestra.run.smithi121:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T22:59:28.892 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T22:59:28.892 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-24T22:59:28.902 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T22:59:28.903 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-24T22:59:28.903 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.121:3300,172.21.15.121:3301" 2024-04-24T22:59:28.903 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: trying mount with new device syntax: 3@eb1b7a01-6f65-4534-9a12-83011862ba51.cephfs=/ 2024-04-24T22:59:28.903 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.32:3300/172.21.15.121:3300/172.21.15.121:3301" will pass to kernel 2024-04-24T22:59:28.905 INFO:teuthology.orchestra.run.smithi121.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-04-24T22:59:28.905 INFO:teuthology.orchestra.run.smithi121.stdout: You just mounted a file system that supports labels which does not 2024-04-24T22:59:28.905 INFO:teuthology.orchestra.run.smithi121.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-24T22:59:28.906 INFO:teuthology.orchestra.run.smithi121.stdout: applications will generate AVC messages and not be allowed access to 2024-04-24T22:59:28.906 INFO:teuthology.orchestra.run.smithi121.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-24T22:59:28.906 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-24T22:59:28.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:59:28.907 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-24T22:59:28.973 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T22:59:29.061 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:59:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T22:59:29.062 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:59:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:59:29.068 INFO:teuthology.orchestra.run.smithi121.stdout:client4621 2024-04-24T22:59:29.068 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T22:59:29.068 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T22:59:29.083 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T22:59:29.171 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:59:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T22:59:29.173 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:59:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:59:29.178 INFO:teuthology.orchestra.run.smithi121.stdout:client4621 2024-04-24T22:59:29.179 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T22:59:29.226 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:59:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T22:59:29.228 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:59:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:59:29.233 INFO:teuthology.orchestra.run.smithi121.stdout:client4621 2024-04-24T22:59:29.233 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/eb1b7a01-6f65-4534-9a12-83011862ba51.client4621/status') 2024-04-24T22:59:29.281 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:59:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T22:59:29.290 INFO:teuthology.orchestra.run.smithi121.stdout:instance: client.4621 (3)192.168.0.2:0/626184158 2024-04-24T22:59:29.290 INFO:teuthology.orchestra.run.smithi121.stdout:blocklisted: false 2024-04-24T22:59:29.290 INFO:teuthology.orchestra.run.smithi121.stderr:0+1 records in 2024-04-24T22:59:29.290 INFO:teuthology.orchestra.run.smithi121.stderr:0+1 records out 2024-04-24T22:59:29.291 INFO:teuthology.orchestra.run.smithi121.stderr:68 bytes copied, 2.1883e-05 s, 3.1 MB/s 2024-04-24T22:59:29.291 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T22:59:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:59:29.297 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-24T22:59:29.313 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:59:29.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.424+0000 7f7a782ba640 1 Processor -- start 2024-04-24T22:59:29.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.424+0000 7f7a782ba640 1 -- start start 2024-04-24T22:59:29.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.425+0000 7f7a782ba640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7a7015b480 0x7f7a700b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:29.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.425+0000 7f7a782ba640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7a7015fd60 0x7f7a7006c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:29.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.425+0000 7f7a782ba640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a700cef90 con 0x7f7a7006cb30 2024-04-24T22:59:29.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.425+0000 7f7a782ba640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f7a700b4ff0 con 0x7f7a7015b480 2024-04-24T22:59:29.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.425+0000 7f7a782ba640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f7a700b46c0 con 0x7f7a7015fd60 2024-04-24T22:59:29.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.425+0000 7f7a76056640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7a7015b480 0x7f7a700b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:29.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.425+0000 7f7a76056640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7a7015b480 0x7f7a700b5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59890/0 (socket says 172.21.15.32:59890) 2024-04-24T22:59:29.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.425+0000 7f7a76056640 1 -- 172.21.15.32:0/3328290816 learned_addr learned my addr 172.21.15.32:0/3328290816 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:29.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.425+0000 7f7a75855640 1 --2- 172.21.15.32:0/3328290816 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7a7015fd60 0x7f7a7006c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:29.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.426+0000 7f7a76056640 1 -- 172.21.15.32:0/3328290816 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7a7015fd60 msgr2=0x7f7a7006c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:29.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.426+0000 7f7a76056640 1 --2- 172.21.15.32:0/3328290816 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7a7015fd60 0x7f7a7006c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:29.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.426+0000 7f7a76056640 1 -- 172.21.15.32:0/3328290816 >> v1:172.21.15.121:6789/0 conn(0x7f7a7006cb30 legacy=0x7f7a700ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:29.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.426+0000 7f7a76056640 1 -- 172.21.15.32:0/3328290816 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a700b49a0 con 0x7f7a7015b480 2024-04-24T22:59:29.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.426+0000 7f7a76056640 1 --2- 172.21.15.32:0/3328290816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a7015b480 0x7f7a700b5c60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7a68009870 tx=0x7f7a6802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26d514974321e3c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:29.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.426+0000 7f7a75054640 1 -- 172.21.15.32:0/3328290816 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a68037070 con 0x7f7a7015b480 2024-04-24T22:59:29.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.426+0000 7f7a75054640 1 -- 172.21.15.32:0/3328290816 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a68032440 con 0x7f7a7015b480 2024-04-24T22:59:29.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.426+0000 7f7a75054640 1 -- 172.21.15.32:0/3328290816 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a68032810 con 0x7f7a7015b480 2024-04-24T22:59:29.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.427+0000 7f7a782ba640 1 -- 172.21.15.32:0/3328290816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a7015b480 msgr2=0x7f7a700b5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:29.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.427+0000 7f7a782ba640 1 --2- 172.21.15.32:0/3328290816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a7015b480 0x7f7a700b5c60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7a68009870 tx=0x7f7a6802a270 comp rx=0 tx=0).stop 2024-04-24T22:59:29.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.427+0000 7f7a782ba640 1 -- 172.21.15.32:0/3328290816 shutdown_connections 2024-04-24T22:59:29.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.427+0000 7f7a782ba640 1 --2- 172.21.15.32:0/3328290816 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7a7015fd60 0x7f7a7006c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:29.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.427+0000 7f7a782ba640 1 --2- 172.21.15.32:0/3328290816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a7015b480 0x7f7a700b5c60 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:29.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.427+0000 7f7a782ba640 1 -- 172.21.15.32:0/3328290816 >> 172.21.15.32:0/3328290816 conn(0x7f7a7006b110 msgr2=0x7f7a7006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:29.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.427+0000 7f7a782ba640 1 -- 172.21.15.32:0/3328290816 shutdown_connections 2024-04-24T22:59:29.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.427+0000 7f7a782ba640 1 -- 172.21.15.32:0/3328290816 wait complete. 2024-04-24T22:59:29.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.428+0000 7f7a782ba640 1 Processor -- start 2024-04-24T22:59:29.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.428+0000 7f7a782ba640 1 -- start start 2024-04-24T22:59:29.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.429+0000 7f7a782ba640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a7006cb30 0x7f7a70153ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:29.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.429+0000 7f7a782ba640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7a7015b480 0x7f7a7013d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:29.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.429+0000 7f7a782ba640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a7015fd60 0x7f7a7013d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:29.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.429+0000 7f7a782ba640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7a700b5480 con 0x7f7a7015fd60 2024-04-24T22:59:29.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.429+0000 7f7a782ba640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7a700b61a0 con 0x7f7a7006cb30 2024-04-24T22:59:29.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.429+0000 7f7a782ba640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f7a7015b860 con 0x7f7a7015b480 2024-04-24T22:59:29.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.429+0000 7f7a76857640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a7015fd60 0x7f7a7013d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:29.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.429+0000 7f7a76857640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a7015fd60 0x7f7a7013d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59900/0 (socket says 172.21.15.32:59900) 2024-04-24T22:59:29.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.429+0000 7f7a76857640 1 -- 172.21.15.32:0/248322306 learned_addr learned my addr 172.21.15.32:0/248322306 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:29.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.429+0000 7f7a76056640 1 --2- 172.21.15.32:0/248322306 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a7006cb30 0x7f7a70153ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:29.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.429+0000 7f7a75855640 1 --2- 172.21.15.32:0/248322306 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7a7015b480 0x7f7a7013d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:29.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.430+0000 7f7a76857640 1 -- 172.21.15.32:0/248322306 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7a7015b480 msgr2=0x7f7a7013d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:29.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.430+0000 7f7a76857640 1 --2- 172.21.15.32:0/248322306 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7a7015b480 0x7f7a7013d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:29.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.430+0000 7f7a76857640 1 -- 172.21.15.32:0/248322306 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a7006cb30 msgr2=0x7f7a70153ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:29.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.430+0000 7f7a76857640 1 --2- 172.21.15.32:0/248322306 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a7006cb30 0x7f7a70153ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:29.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.430+0000 7f7a76857640 1 -- 172.21.15.32:0/248322306 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a7015d860 con 0x7f7a7015fd60 2024-04-24T22:59:29.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.430+0000 7f7a76056640 1 --2- 172.21.15.32:0/248322306 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a7006cb30 0x7f7a70153ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:59:29.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.430+0000 7f7a76857640 1 --2- 172.21.15.32:0/248322306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a7015fd60 0x7f7a7013d8b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7a6000c7a0 tx=0x7f7a6000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:29.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.430+0000 7f7a6effd640 1 -- 172.21.15.32:0/248322306 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a60003b70 con 0x7f7a7015fd60 2024-04-24T22:59:29.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.430+0000 7f7a6effd640 1 -- 172.21.15.32:0/248322306 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a60003d30 con 0x7f7a7015fd60 2024-04-24T22:59:29.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.430+0000 7f7a782ba640 1 -- 172.21.15.32:0/248322306 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a7015db40 con 0x7f7a7015fd60 2024-04-24T22:59:29.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.430+0000 7f7a6effd640 1 -- 172.21.15.32:0/248322306 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6000aa90 con 0x7f7a7015fd60 2024-04-24T22:59:29.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.430+0000 7f7a782ba640 1 -- 172.21.15.32:0/248322306 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a7015e100 con 0x7f7a7015fd60 2024-04-24T22:59:29.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.431+0000 7f7a782ba640 1 -- 172.21.15.32:0/248322306 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a70068430 con 0x7f7a7015fd60 2024-04-24T22:59:29.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.431+0000 7f7a6effd640 1 -- 172.21.15.32:0/248322306 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7a6000ac30 con 0x7f7a7015fd60 2024-04-24T22:59:29.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.432+0000 7f7a6effd640 1 --2- 172.21.15.32:0/248322306 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7a5c06be30 0x7f7a5c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:29.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.432+0000 7f7a6effd640 1 -- 172.21.15.32:0/248322306 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f7a6008ac70 con 0x7f7a7015fd60 2024-04-24T22:59:29.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.432+0000 7f7a76056640 1 --2- 172.21.15.32:0/248322306 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7a5c06be30 0x7f7a5c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:29.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.433+0000 7f7a76056640 1 --2- 172.21.15.32:0/248322306 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7a5c06be30 0x7f7a5c06e2f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7a68034000 tx=0x7f7a6802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:29.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.436+0000 7f7a6effd640 1 -- 172.21.15.32:0/248322306 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7a60058fd0 con 0x7f7a7015fd60 2024-04-24T22:59:29.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.620+0000 7f7a782ba640 1 -- 172.21.15.32:0/248322306 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7a7006c000 con 0x7f7a7015fd60 2024-04-24T22:59:29.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.621+0000 7f7a6effd640 1 -- 172.21.15.32:0/248322306 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16621 (secure 0 0 0) 0x7f7a6005d2d0 con 0x7f7a7015fd60 2024-04-24T22:59:29.624 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:59:29.624 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T22:59:05.517554+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:59:02.016639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T22:59:02.846087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:59:29.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.623+0000 7f7a782ba640 1 -- 172.21.15.32:0/248322306 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7a5c06be30 msgr2=0x7f7a5c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:29.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.623+0000 7f7a782ba640 1 --2- 172.21.15.32:0/248322306 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7a5c06be30 0x7f7a5c06e2f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7a68034000 tx=0x7f7a6802e000 comp rx=0 tx=0).stop 2024-04-24T22:59:29.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.623+0000 7f7a782ba640 1 -- 172.21.15.32:0/248322306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a7015fd60 msgr2=0x7f7a7013d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:29.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.623+0000 7f7a782ba640 1 --2- 172.21.15.32:0/248322306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a7015fd60 0x7f7a7013d8b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7a6000c7a0 tx=0x7f7a6000cc60 comp rx=0 tx=0).stop 2024-04-24T22:59:29.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.623+0000 7f7a782ba640 1 -- 172.21.15.32:0/248322306 shutdown_connections 2024-04-24T22:59:29.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.623+0000 7f7a782ba640 1 --2- 172.21.15.32:0/248322306 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7a5c06be30 0x7f7a5c06e2f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:29.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.623+0000 7f7a782ba640 1 --2- 172.21.15.32:0/248322306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a7015fd60 0x7f7a7013d8b0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:29.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.623+0000 7f7a782ba640 1 --2- 172.21.15.32:0/248322306 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7a7015b480 0x7f7a7013d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:29.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.623+0000 7f7a782ba640 1 --2- 172.21.15.32:0/248322306 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a7006cb30 0x7f7a70153ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:29.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.623+0000 7f7a782ba640 1 -- 172.21.15.32:0/248322306 >> 172.21.15.32:0/248322306 conn(0x7f7a7006b110 msgr2=0x7f7a700b3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:29.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.623+0000 7f7a782ba640 1 -- 172.21.15.32:0/248322306 shutdown_connections 2024-04-24T22:59:29.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.623+0000 7f7a782ba640 1 -- 172.21.15.32:0/248322306 wait complete. 2024-04-24T22:59:29.637 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T22:59:02.016639+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T22:59:02.846087+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T22:59:29.637 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T22:59:29.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.738+0000 7f2a176f5640 1 Processor -- start 2024-04-24T22:59:29.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.738+0000 7f2a176f5640 1 -- start start 2024-04-24T22:59:29.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.739+0000 7f2a176f5640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2a100b5af0 0x7f2a100ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:29.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.739+0000 7f2a176f5640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a100cd270 con 0x7f2a10162320 2024-04-24T22:59:29.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.739+0000 7f2a176f5640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a10071610 con 0x7f2a10162e10 2024-04-24T22:59:29.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.739+0000 7f2a176f5640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f2a100b5660 con 0x7f2a100b5af0 2024-04-24T22:59:29.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.739+0000 7f2a15491640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2a10162320 0x7f2a10162700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34012/0 (socket says 172.21.15.32:34012) 2024-04-24T22:59:29.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.739+0000 7f2a15491640 1 -- 172.21.15.32:0/3546908953 learned_addr learned my addr 172.21.15.32:0/3546908953 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:29.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.740+0000 7f2a0ffff640 1 -- 172.21.15.32:0/3546908953 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1124566105 0 0) 0x7f2a100cd270 con 0x7f2a10162320 2024-04-24T22:59:29.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.740+0000 7f2a14c90640 1 --2- 172.21.15.32:0/3546908953 >> v2:172.21.15.121:3300/0 conn(0x7f2a100b5af0 0x7f2a100ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:29.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.740+0000 7f2a0ffff640 1 -- 172.21.15.32:0/3546908953 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f29f8003520 con 0x7f2a10162320 2024-04-24T22:59:29.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.740+0000 7f2a0ffff640 1 -- 172.21.15.32:0/3546908953 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1913726101 0 0) 0x7f29f8003520 con 0x7f2a10162320 2024-04-24T22:59:29.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.740+0000 7f2a0ffff640 1 -- 172.21.15.32:0/3546908953 >> v2:172.21.15.121:3300/0 conn(0x7f2a100b5af0 msgr2=0x7f2a100ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:29.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.740+0000 7f2a0ffff640 1 --2- 172.21.15.32:0/3546908953 >> v2:172.21.15.121:3300/0 conn(0x7f2a100b5af0 0x7f2a100ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:29.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.740+0000 7f2a0ffff640 1 -- 172.21.15.32:0/3546908953 >> v1:172.21.15.121:6789/0 conn(0x7f2a10162e10 legacy=0x7f2a1015ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:29.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.740+0000 7f2a0ffff640 1 -- 172.21.15.32:0/3546908953 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a100718f0 con 0x7f2a10162320 2024-04-24T22:59:29.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.740+0000 7f2a0ffff640 1 -- 172.21.15.32:0/3546908953 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f2a000019f0 con 0x7f2a10162320 2024-04-24T22:59:29.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.740+0000 7f2a0ffff640 1 -- 172.21.15.32:0/3546908953 >> v1:172.21.15.32:6789/0 conn(0x7f2a10162320 legacy=0x7f2a10162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:29.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.740+0000 7f2a0ffff640 1 --2- 172.21.15.32:0/3546908953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f29f80039b0 0x7f29f8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:29.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.740+0000 7f2a0ffff640 1 --2- 172.21.15.32:0/3546908953 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f29f8004800 0x7f29f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:29.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.740+0000 7f2a0ffff640 1 --2- 172.21.15.32:0/3546908953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29f80051c0 0x7f29f800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:29.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.740+0000 7f2a0ffff640 1 -- 172.21.15.32:0/3546908953 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2a100b5660 con 0x7f29f80051c0 2024-04-24T22:59:29.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.740+0000 7f2a0ffff640 1 -- 172.21.15.32:0/3546908953 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f29f800c160 con 0x7f29f80039b0 2024-04-24T22:59:29.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.740+0000 7f2a0ffff640 1 -- 172.21.15.32:0/3546908953 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f29f800c360 con 0x7f29f8004800 2024-04-24T22:59:29.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.740+0000 7f2a15c92640 1 --2- 172.21.15.32:0/3546908953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29f80051c0 0x7f29f800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:29.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.740+0000 7f2a15491640 1 --2- 172.21.15.32:0/3546908953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f29f80039b0 0x7f29f8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:29.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.740+0000 7f2a14c90640 1 --2- 172.21.15.32:0/3546908953 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f29f8004800 0x7f29f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:29.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.741+0000 7f2a15c92640 1 -- 172.21.15.32:0/3546908953 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f29f8004800 msgr2=0x7f29f8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:29.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.741+0000 7f2a15c92640 1 --2- 172.21.15.32:0/3546908953 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f29f8004800 0x7f29f8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:29.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.741+0000 7f2a15c92640 1 -- 172.21.15.32:0/3546908953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f29f80039b0 msgr2=0x7f29f8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:29.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.741+0000 7f2a15c92640 1 --2- 172.21.15.32:0/3546908953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f29f80039b0 0x7f29f8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:29.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.741+0000 7f2a15c92640 1 -- 172.21.15.32:0/3546908953 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29f8003520 con 0x7f29f80051c0 2024-04-24T22:59:29.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.741+0000 7f2a15491640 1 --2- 172.21.15.32:0/3546908953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f29f80039b0 0x7f29f8003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:59:29.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.741+0000 7f2a15c92640 1 -- 172.21.15.32:0/3546908953 reap_dead start 2024-04-24T22:59:29.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.741+0000 7f2a15c92640 1 --2- 172.21.15.32:0/3546908953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29f80051c0 0x7f29f800b9c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2a08001750 tx=0x7f29f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbc09e1396955faf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:29.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.741+0000 7f2a0ffff640 1 -- 172.21.15.32:0/3546908953 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0000a4a0 con 0x7f29f80051c0 2024-04-24T22:59:29.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.741+0000 7f2a0ffff640 1 -- 172.21.15.32:0/3546908953 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a080052f0 con 0x7f29f80051c0 2024-04-24T22:59:29.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.741+0000 7f2a0ffff640 1 -- 172.21.15.32:0/3546908953 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a08001370 con 0x7f29f80051c0 2024-04-24T22:59:29.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.742+0000 7f2a176f5640 1 -- 172.21.15.32:0/3546908953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29f80051c0 msgr2=0x7f29f800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:29.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.742+0000 7f2a176f5640 1 --2- 172.21.15.32:0/3546908953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29f80051c0 0x7f29f800b9c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2a08001750 tx=0x7f29f8005830 comp rx=0 tx=0).stop 2024-04-24T22:59:29.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.742+0000 7f2a176f5640 1 -- 172.21.15.32:0/3546908953 shutdown_connections 2024-04-24T22:59:29.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.742+0000 7f2a176f5640 1 --2- 172.21.15.32:0/3546908953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29f80051c0 0x7f29f800b9c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:29.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.742+0000 7f2a176f5640 1 -- 172.21.15.32:0/3546908953 >> 172.21.15.32:0/3546908953 conn(0x7f2a100d0ae0 msgr2=0x7f2a100d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:29.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.742+0000 7f2a176f5640 1 -- 172.21.15.32:0/3546908953 shutdown_connections 2024-04-24T22:59:29.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.742+0000 7f2a176f5640 1 -- 172.21.15.32:0/3546908953 wait complete. 2024-04-24T22:59:29.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.743+0000 7f2a176f5640 1 Processor -- start 2024-04-24T22:59:29.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.743+0000 7f2a176f5640 1 -- start start 2024-04-24T22:59:29.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.744+0000 7f2a176f5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29f80051c0 0x7f2a10150820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:29.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.744+0000 7f2a176f5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2a1006e700 0x7f2a1015f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:29.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.744+0000 7f2a176f5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2a10160e80 0x7f2a10161300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:29.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.744+0000 7f2a176f5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2a10071bd0 con 0x7f29f80051c0 2024-04-24T22:59:29.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.744+0000 7f2a176f5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2a10067950 con 0x7f2a1006e700 2024-04-24T22:59:29.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.744+0000 7f2a176f5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f2a10071d50 con 0x7f2a10160e80 2024-04-24T22:59:29.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.744+0000 7f2a15491640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29f80051c0 0x7f2a10150820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:29.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.744+0000 7f2a15491640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29f80051c0 0x7f2a10150820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52024/0 (socket says 172.21.15.32:52024) 2024-04-24T22:59:29.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.744+0000 7f2a15491640 1 -- 172.21.15.32:0/297056721 learned_addr learned my addr 172.21.15.32:0/297056721 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:29.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.744+0000 7f2a14c90640 1 --2- 172.21.15.32:0/297056721 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2a1006e700 0x7f2a1015f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:29.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.744+0000 7f2a15c92640 1 --2- 172.21.15.32:0/297056721 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2a10160e80 0x7f2a10161300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:29.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.744+0000 7f2a15491640 1 -- 172.21.15.32:0/297056721 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2a10160e80 msgr2=0x7f2a10161300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:29.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.744+0000 7f2a15491640 1 --2- 172.21.15.32:0/297056721 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2a10160e80 0x7f2a10161300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:29.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.744+0000 7f2a15491640 1 -- 172.21.15.32:0/297056721 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2a1006e700 msgr2=0x7f2a1015f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:29.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.744+0000 7f2a15491640 1 --2- 172.21.15.32:0/297056721 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2a1006e700 0x7f2a1015f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:29.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.744+0000 7f2a15491640 1 -- 172.21.15.32:0/297056721 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a1015f880 con 0x7f29f80051c0 2024-04-24T22:59:29.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.744+0000 7f2a15c92640 1 --2- 172.21.15.32:0/297056721 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2a10160e80 0x7f2a10161300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:59:29.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.745+0000 7f2a15491640 1 --2- 172.21.15.32:0/297056721 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29f80051c0 0x7f2a10150820 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2a000079a0 tx=0x7f2a0000a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:29.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.745+0000 7f2a0dffb640 1 -- 172.21.15.32:0/297056721 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a00017020 con 0x7f29f80051c0 2024-04-24T22:59:29.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.745+0000 7f2a0dffb640 1 -- 172.21.15.32:0/297056721 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a00007a90 con 0x7f29f80051c0 2024-04-24T22:59:29.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.745+0000 7f2a176f5640 1 -- 172.21.15.32:0/297056721 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a1006f610 con 0x7f29f80051c0 2024-04-24T22:59:29.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.745+0000 7f2a0dffb640 1 -- 172.21.15.32:0/297056721 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a000135f0 con 0x7f29f80051c0 2024-04-24T22:59:29.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.745+0000 7f2a176f5640 1 -- 172.21.15.32:0/297056721 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a1006f050 con 0x7f29f80051c0 2024-04-24T22:59:29.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.746+0000 7f2a176f5640 1 -- 172.21.15.32:0/297056721 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a1006eae0 con 0x7f29f80051c0 2024-04-24T22:59:29.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.746+0000 7f2a0dffb640 1 -- 172.21.15.32:0/297056721 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2a0001a020 con 0x7f29f80051c0 2024-04-24T22:59:29.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.747+0000 7f2a0dffb640 1 --2- 172.21.15.32:0/297056721 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f29fc06be30 0x7f29fc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:29.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.747+0000 7f2a0dffb640 1 -- 172.21.15.32:0/297056721 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f2a0008bb80 con 0x7f29f80051c0 2024-04-24T22:59:29.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.747+0000 7f2a14c90640 1 --2- 172.21.15.32:0/297056721 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f29fc06be30 0x7f29fc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:29.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.747+0000 7f2a14c90640 1 --2- 172.21.15.32:0/297056721 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f29fc06be30 0x7f29fc06e2f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2a1015a5c0 tx=0x7f2a04008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:29.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.751+0000 7f2a0dffb640 1 -- 172.21.15.32:0/297056721 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2a00021020 con 0x7f29f80051c0 2024-04-24T22:59:29.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.934+0000 7f2a176f5640 1 -- 172.21.15.32:0/297056721 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2a10068430 con 0x7f29f80051c0 2024-04-24T22:59:29.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.934+0000 7f2a0dffb640 1 -- 172.21.15.32:0/297056721 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f2a0005a5c0 con 0x7f29f80051c0 2024-04-24T22:59:29.937 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T22:59:29.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.936+0000 7f2a176f5640 1 -- 172.21.15.32:0/297056721 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f29fc06be30 msgr2=0x7f29fc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:29.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.936+0000 7f2a176f5640 1 --2- 172.21.15.32:0/297056721 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f29fc06be30 0x7f29fc06e2f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2a1015a5c0 tx=0x7f2a04008040 comp rx=0 tx=0).stop 2024-04-24T22:59:29.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.936+0000 7f2a176f5640 1 -- 172.21.15.32:0/297056721 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29f80051c0 msgr2=0x7f2a10150820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:29.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.936+0000 7f2a176f5640 1 --2- 172.21.15.32:0/297056721 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29f80051c0 0x7f2a10150820 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2a000079a0 tx=0x7f2a0000a730 comp rx=0 tx=0).stop 2024-04-24T22:59:29.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.936+0000 7f2a176f5640 1 -- 172.21.15.32:0/297056721 shutdown_connections 2024-04-24T22:59:29.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.936+0000 7f2a176f5640 1 --2- 172.21.15.32:0/297056721 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f29fc06be30 0x7f29fc06e2f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:29.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.936+0000 7f2a176f5640 1 --2- 172.21.15.32:0/297056721 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2a10160e80 0x7f2a10161300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:29.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.936+0000 7f2a176f5640 1 --2- 172.21.15.32:0/297056721 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2a1006e700 0x7f2a1015f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:29.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.936+0000 7f2a176f5640 1 --2- 172.21.15.32:0/297056721 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29f80051c0 0x7f2a10150820 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:29.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.936+0000 7f2a176f5640 1 -- 172.21.15.32:0/297056721 >> 172.21.15.32:0/297056721 conn(0x7f2a100d0ae0 msgr2=0x7f2a100b3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:29.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.936+0000 7f2a176f5640 1 -- 172.21.15.32:0/297056721 shutdown_connections 2024-04-24T22:59:29.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:29.936+0000 7f2a176f5640 1 -- 172.21.15.32:0/297056721 wait complete. 2024-04-24T22:59:29.950 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T22:59:30.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.051+0000 7f61eedce640 1 Processor -- start 2024-04-24T22:59:30.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.051+0000 7f61eedce640 1 -- start start 2024-04-24T22:59:30.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.052+0000 7f61eedce640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f61e80b44d0 0x7f61e80b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:30.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.052+0000 7f61eedce640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f61e80af890 0x7f61e815cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:30.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.052+0000 7f61eedce640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61e80b6430 con 0x7f61e806c5f0 2024-04-24T22:59:30.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.052+0000 7f61eedce640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f61e8156f80 con 0x7f61e80af890 2024-04-24T22:59:30.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.052+0000 7f61eedce640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f61e80b6150 con 0x7f61e80b44d0 2024-04-24T22:59:30.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.052+0000 7f61ed36b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f61e80af890 0x7f61e815cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:30.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.052+0000 7f61ed36b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f61e80af890 0x7f61e815cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52032/0 (socket says 172.21.15.32:52032) 2024-04-24T22:59:30.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.052+0000 7f61ed36b640 1 -- 172.21.15.32:0/3411636856 learned_addr learned my addr 172.21.15.32:0/3411636856 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:30.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.052+0000 7f61e7fff640 1 --2- 172.21.15.32:0/3411636856 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f61e80b44d0 0x7f61e80b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:30.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.052+0000 7f61ed36b640 1 -- 172.21.15.32:0/3411636856 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f61e80b44d0 msgr2=0x7f61e80b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:30.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.052+0000 7f61ed36b640 1 --2- 172.21.15.32:0/3411636856 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f61e80b44d0 0x7f61e80b5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:30.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.052+0000 7f61ed36b640 1 -- 172.21.15.32:0/3411636856 >> v1:172.21.15.121:6789/0 conn(0x7f61e806c5f0 legacy=0x7f61e806c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:30.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.052+0000 7f61ed36b640 1 -- 172.21.15.32:0/3411636856 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e80b39f0 con 0x7f61e80af890 2024-04-24T22:59:30.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.052+0000 7f61e7fff640 1 --2- 172.21.15.32:0/3411636856 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f61e80b44d0 0x7f61e80b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:59:30.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.053+0000 7f61ed36b640 1 --2- 172.21.15.32:0/3411636856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61e80af890 0x7f61e815cac0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f61e0009e50 tx=0x7f61e002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9193ff5bd0764577 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:30.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.053+0000 7f61e77fe640 1 -- 172.21.15.32:0/3411636856 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e0036070 con 0x7f61e80af890 2024-04-24T22:59:30.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.053+0000 7f61e77fe640 1 -- 172.21.15.32:0/3411636856 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61e0031600 con 0x7f61e80af890 2024-04-24T22:59:30.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.053+0000 7f61e77fe640 1 -- 172.21.15.32:0/3411636856 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e00319d0 con 0x7f61e80af890 2024-04-24T22:59:30.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.054+0000 7f61eedce640 1 -- 172.21.15.32:0/3411636856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61e80af890 msgr2=0x7f61e815cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:30.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.054+0000 7f61eedce640 1 --2- 172.21.15.32:0/3411636856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61e80af890 0x7f61e815cac0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f61e0009e50 tx=0x7f61e002a680 comp rx=0 tx=0).stop 2024-04-24T22:59:30.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.054+0000 7f61eedce640 1 -- 172.21.15.32:0/3411636856 shutdown_connections 2024-04-24T22:59:30.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.054+0000 7f61eedce640 1 --2- 172.21.15.32:0/3411636856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61e80af890 0x7f61e815cac0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:30.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.054+0000 7f61eedce640 1 --2- 172.21.15.32:0/3411636856 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f61e80b44d0 0x7f61e80b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:30.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.054+0000 7f61eedce640 1 -- 172.21.15.32:0/3411636856 >> 172.21.15.32:0/3411636856 conn(0x7f61e806b110 msgr2=0x7f61e806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:30.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.054+0000 7f61eedce640 1 -- 172.21.15.32:0/3411636856 shutdown_connections 2024-04-24T22:59:30.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.054+0000 7f61eedce640 1 -- 172.21.15.32:0/3411636856 wait complete. 2024-04-24T22:59:30.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.055+0000 7f61eedce640 1 Processor -- start 2024-04-24T22:59:30.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.055+0000 7f61eedce640 1 -- start start 2024-04-24T22:59:30.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.056+0000 7f61eedce640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f61e806c5f0 0x7f61e80ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:30.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.056+0000 7f61eedce640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61e80af890 0x7f61e8145e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:30.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.056+0000 7f61eedce640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f61e80b44d0 0x7f61e80ccd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:30.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.056+0000 7f61eedce640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f61e80b5dd0 con 0x7f61e80af890 2024-04-24T22:59:30.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.056+0000 7f61eedce640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f61e80b3fb0 con 0x7f61e806c5f0 2024-04-24T22:59:30.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.056+0000 7f61eedce640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f61e815d000 con 0x7f61e80b44d0 2024-04-24T22:59:30.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.056+0000 7f61e7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61e80af890 0x7f61e8145e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:30.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.056+0000 7f61e7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61e80af890 0x7f61e8145e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52048/0 (socket says 172.21.15.32:52048) 2024-04-24T22:59:30.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.056+0000 7f61e7fff640 1 -- 172.21.15.32:0/3271721859 learned_addr learned my addr 172.21.15.32:0/3271721859 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:30.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.056+0000 7f61ecb6a640 1 --2- 172.21.15.32:0/3271721859 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f61e806c5f0 0x7f61e80ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:30.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.056+0000 7f61ed36b640 1 --2- 172.21.15.32:0/3271721859 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f61e80b44d0 0x7f61e80ccd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:30.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.056+0000 7f61e7fff640 1 -- 172.21.15.32:0/3271721859 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f61e80b44d0 msgr2=0x7f61e80ccd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:30.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.056+0000 7f61e7fff640 1 --2- 172.21.15.32:0/3271721859 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f61e80b44d0 0x7f61e80ccd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:30.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.056+0000 7f61e7fff640 1 -- 172.21.15.32:0/3271721859 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f61e806c5f0 msgr2=0x7f61e80ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:30.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.056+0000 7f61e7fff640 1 --2- 172.21.15.32:0/3271721859 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f61e806c5f0 0x7f61e80ce9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:30.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.056+0000 7f61e7fff640 1 -- 172.21.15.32:0/3271721859 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e80cd2a0 con 0x7f61e80af890 2024-04-24T22:59:30.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.056+0000 7f61e7fff640 1 --2- 172.21.15.32:0/3271721859 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61e80af890 0x7f61e8145e70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f61d400c820 tx=0x7f61d400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:30.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.057+0000 7f61e57fa640 1 -- 172.21.15.32:0/3271721859 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d4015070 con 0x7f61e80af890 2024-04-24T22:59:30.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.057+0000 7f61e57fa640 1 -- 172.21.15.32:0/3271721859 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61d4007a30 con 0x7f61e80af890 2024-04-24T22:59:30.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.057+0000 7f61eedce640 1 -- 172.21.15.32:0/3271721859 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61e80cd520 con 0x7f61e80af890 2024-04-24T22:59:30.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.057+0000 7f61e57fa640 1 -- 172.21.15.32:0/3271721859 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d4007de0 con 0x7f61e80af890 2024-04-24T22:59:30.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.057+0000 7f61eedce640 1 -- 172.21.15.32:0/3271721859 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61e80cdae0 con 0x7f61e80af890 2024-04-24T22:59:30.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.058+0000 7f61eedce640 1 -- 172.21.15.32:0/3271721859 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61e80cdd20 con 0x7f61e80af890 2024-04-24T22:59:30.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.058+0000 7f61e57fa640 1 -- 172.21.15.32:0/3271721859 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f61d4018070 con 0x7f61e80af890 2024-04-24T22:59:30.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.059+0000 7f61e57fa640 1 --2- 172.21.15.32:0/3271721859 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f61cc06be30 0x7f61cc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:30.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.059+0000 7f61e57fa640 1 -- 172.21.15.32:0/3271721859 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f61d408acf0 con 0x7f61e80af890 2024-04-24T22:59:30.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.059+0000 7f61ecb6a640 1 --2- 172.21.15.32:0/3271721859 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f61cc06be30 0x7f61cc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:30.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.059+0000 7f61ecb6a640 1 --2- 172.21.15.32:0/3271721859 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f61cc06be30 0x7f61cc06e2f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f61dc007fd0 tx=0x7f61dc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:30.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.063+0000 7f61e57fa640 1 -- 172.21.15.32:0/3271721859 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f61e80cdd20 con 0x7f61e80af890 2024-04-24T22:59:30.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.247+0000 7f61eedce640 1 -- 172.21.15.32:0/3271721859 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f61e815a4e0 con 0x7f61e80af890 2024-04-24T22:59:30.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.247+0000 7f61e57fa640 1 -- 172.21.15.32:0/3271721859 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f61d408f070 con 0x7f61e80af890 2024-04-24T22:59:30.250 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-24T22:59:30.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.249+0000 7f61eedce640 1 -- 172.21.15.32:0/3271721859 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f61cc06be30 msgr2=0x7f61cc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:30.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.249+0000 7f61eedce640 1 --2- 172.21.15.32:0/3271721859 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f61cc06be30 0x7f61cc06e2f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f61dc007fd0 tx=0x7f61dc009040 comp rx=0 tx=0).stop 2024-04-24T22:59:30.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.249+0000 7f61eedce640 1 -- 172.21.15.32:0/3271721859 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61e80af890 msgr2=0x7f61e8145e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:30.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.249+0000 7f61eedce640 1 --2- 172.21.15.32:0/3271721859 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61e80af890 0x7f61e8145e70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f61d400c820 tx=0x7f61d400cce0 comp rx=0 tx=0).stop 2024-04-24T22:59:30.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.249+0000 7f61eedce640 1 -- 172.21.15.32:0/3271721859 shutdown_connections 2024-04-24T22:59:30.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.249+0000 7f61eedce640 1 --2- 172.21.15.32:0/3271721859 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f61cc06be30 0x7f61cc06e2f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:30.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.249+0000 7f61eedce640 1 --2- 172.21.15.32:0/3271721859 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61e80af890 0x7f61e8145e70 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:30.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.249+0000 7f61eedce640 1 --2- 172.21.15.32:0/3271721859 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f61e80b44d0 0x7f61e80ccd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:30.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.249+0000 7f61eedce640 1 --2- 172.21.15.32:0/3271721859 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f61e806c5f0 0x7f61e80ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:30.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.249+0000 7f61eedce640 1 -- 172.21.15.32:0/3271721859 >> 172.21.15.32:0/3271721859 conn(0x7f61e806b110 msgr2=0x7f61e806c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:30.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.249+0000 7f61eedce640 1 -- 172.21.15.32:0/3271721859 shutdown_connections 2024-04-24T22:59:30.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.249+0000 7f61eedce640 1 -- 172.21.15.32:0/3271721859 wait complete. 2024-04-24T22:59:30.263 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T22:59:30.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.364+0000 7f7fbafd5640 1 Processor -- start 2024-04-24T22:59:30.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.364+0000 7f7fbafd5640 1 -- start start 2024-04-24T22:59:30.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.365+0000 7f7fbafd5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7fb4158dd0 0x7f7fb406b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:30.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.365+0000 7f7fbafd5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7fb406dc60 0x7f7fb40b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:30.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.365+0000 7f7fbafd5640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fb415ef10 con 0x7f7fb406b860 2024-04-24T22:59:30.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.365+0000 7f7fbafd5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f7fb415f1f0 con 0x7f7fb406dc60 2024-04-24T22:59:30.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.365+0000 7f7fbafd5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f7fb415f4d0 con 0x7f7fb4158dd0 2024-04-24T22:59:30.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.365+0000 7f7fb9572640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7fb406dc60 0x7f7fb40b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:30.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.365+0000 7f7fb3fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7fb406b860 0x7f7fb406d550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34026/0 (socket says 172.21.15.32:34026) 2024-04-24T22:59:30.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.365+0000 7f7fb3fff640 1 -- 172.21.15.32:0/2268496520 learned_addr learned my addr 172.21.15.32:0/2268496520 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:30.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.365+0000 7f7fb9572640 1 -- 172.21.15.32:0/2268496520 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7fb4158dd0 msgr2=0x7f7fb406b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:30.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.365+0000 7f7fb9572640 1 --2- 172.21.15.32:0/2268496520 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7fb4158dd0 0x7f7fb406b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:30.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.365+0000 7f7fb9572640 1 -- 172.21.15.32:0/2268496520 >> v1:172.21.15.32:6789/0 conn(0x7f7fb406b860 legacy=0x7f7fb406d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:30.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.365+0000 7f7fb9572640 1 -- 172.21.15.32:0/2268496520 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fb415f7b0 con 0x7f7fb406dc60 2024-04-24T22:59:30.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.365+0000 7f7fb37fe640 1 -- 172.21.15.32:0/2268496520 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 487702762 0 0) 0x7f7fb415ef10 con 0x7f7fb406b860 2024-04-24T22:59:30.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.366+0000 7f7fb9572640 1 --2- 172.21.15.32:0/2268496520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7fb406dc60 0x7f7fb40b5d00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7fac009d90 tx=0x7f7fac02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c36ad13e261d07 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:30.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.366+0000 7f7fb37fe640 1 -- 172.21.15.32:0/2268496520 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fac007740 con 0x7f7fb406dc60 2024-04-24T22:59:30.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.366+0000 7f7fb37fe640 1 -- 172.21.15.32:0/2268496520 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fac0078e0 con 0x7f7fb406dc60 2024-04-24T22:59:30.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.366+0000 7f7fb37fe640 1 -- 172.21.15.32:0/2268496520 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fac007cb0 con 0x7f7fb406dc60 2024-04-24T22:59:30.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.367+0000 7f7fbafd5640 1 -- 172.21.15.32:0/2268496520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7fb406dc60 msgr2=0x7f7fb40b5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:30.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.367+0000 7f7fbafd5640 1 --2- 172.21.15.32:0/2268496520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7fb406dc60 0x7f7fb40b5d00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7fac009d90 tx=0x7f7fac02a660 comp rx=0 tx=0).stop 2024-04-24T22:59:30.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.367+0000 7f7fbafd5640 1 -- 172.21.15.32:0/2268496520 shutdown_connections 2024-04-24T22:59:30.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.367+0000 7f7fbafd5640 1 --2- 172.21.15.32:0/2268496520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7fb406dc60 0x7f7fb40b5d00 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:30.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.367+0000 7f7fbafd5640 1 --2- 172.21.15.32:0/2268496520 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7fb4158dd0 0x7f7fb406b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:30.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.367+0000 7f7fbafd5640 1 -- 172.21.15.32:0/2268496520 >> 172.21.15.32:0/2268496520 conn(0x7f7fb406e800 msgr2=0x7f7fb406ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:30.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.367+0000 7f7fbafd5640 1 -- 172.21.15.32:0/2268496520 shutdown_connections 2024-04-24T22:59:30.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.367+0000 7f7fbafd5640 1 -- 172.21.15.32:0/2268496520 wait complete. 2024-04-24T22:59:30.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.368+0000 7f7fbafd5640 1 Processor -- start 2024-04-24T22:59:30.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.368+0000 7f7fbafd5640 1 -- start start 2024-04-24T22:59:30.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.369+0000 7f7fbafd5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7fb406b860 0x7f7fb41f7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:30.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.369+0000 7f7fbafd5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7fb406dc60 0x7f7fb41f4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:30.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.369+0000 7f7fbafd5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7fb4158dd0 0x7f7fb414cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:30.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.369+0000 7f7fbafd5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb406bfe0 con 0x7f7fb4158dd0 2024-04-24T22:59:30.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.369+0000 7f7fbafd5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb415fcd0 con 0x7f7fb406dc60 2024-04-24T22:59:30.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.369+0000 7f7fbafd5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f7fb41591b0 con 0x7f7fb406b860 2024-04-24T22:59:30.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.369+0000 7f7fb9572640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7fb4158dd0 0x7f7fb414cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:30.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.369+0000 7f7fb9572640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7fb4158dd0 0x7f7fb414cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52068/0 (socket says 172.21.15.32:52068) 2024-04-24T22:59:30.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.369+0000 7f7fb9572640 1 -- 172.21.15.32:0/2967180295 learned_addr learned my addr 172.21.15.32:0/2967180295 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:30.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.369+0000 7f7fb8d71640 1 --2- 172.21.15.32:0/2967180295 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7fb406b860 0x7f7fb41f7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:30.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.369+0000 7f7fb3fff640 1 --2- 172.21.15.32:0/2967180295 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7fb406dc60 0x7f7fb41f4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:30.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.369+0000 7f7fb9572640 1 -- 172.21.15.32:0/2967180295 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7fb406b860 msgr2=0x7f7fb41f7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:30.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.369+0000 7f7fb9572640 1 --2- 172.21.15.32:0/2967180295 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7fb406b860 0x7f7fb41f7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:30.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.369+0000 7f7fb9572640 1 -- 172.21.15.32:0/2967180295 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7fb406dc60 msgr2=0x7f7fb41f4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:30.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.369+0000 7f7fb9572640 1 --2- 172.21.15.32:0/2967180295 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7fb406dc60 0x7f7fb41f4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:30.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.369+0000 7f7fb9572640 1 -- 172.21.15.32:0/2967180295 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fb41f8020 con 0x7f7fb4158dd0 2024-04-24T22:59:30.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.369+0000 7f7fb9572640 1 --2- 172.21.15.32:0/2967180295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7fb4158dd0 0x7f7fb414cc30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7fac009eb0 tx=0x7f7fac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:30.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.369+0000 7f7fb17fa640 1 -- 172.21.15.32:0/2967180295 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fac040070 con 0x7f7fb4158dd0 2024-04-24T22:59:30.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.369+0000 7f7fb17fa640 1 -- 172.21.15.32:0/2967180295 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fac031c40 con 0x7f7fb4158dd0 2024-04-24T22:59:30.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.369+0000 7f7fbafd5640 1 -- 172.21.15.32:0/2967180295 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fb41f8300 con 0x7f7fb4158dd0 2024-04-24T22:59:30.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.369+0000 7f7fb17fa640 1 -- 172.21.15.32:0/2967180295 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fac003990 con 0x7f7fb4158dd0 2024-04-24T22:59:30.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.370+0000 7f7fbafd5640 1 -- 172.21.15.32:0/2967180295 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fb414d170 con 0x7f7fb4158dd0 2024-04-24T22:59:30.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.370+0000 7f7fbafd5640 1 -- 172.21.15.32:0/2967180295 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fb406c160 con 0x7f7fb4158dd0 2024-04-24T22:59:30.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.371+0000 7f7fb17fa640 1 -- 172.21.15.32:0/2967180295 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7fac039040 con 0x7f7fb4158dd0 2024-04-24T22:59:30.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.371+0000 7f7fb17fa640 1 --2- 172.21.15.32:0/2967180295 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7f9806be30 0x7f7f9806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:30.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.371+0000 7f7fb17fa640 1 -- 172.21.15.32:0/2967180295 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f7fac0abe30 con 0x7f7fb4158dd0 2024-04-24T22:59:30.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.372+0000 7f7fb8d71640 1 --2- 172.21.15.32:0/2967180295 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7f9806be30 0x7f7f9806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:30.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.372+0000 7f7fb8d71640 1 --2- 172.21.15.32:0/2967180295 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7f9806be30 0x7f7f9806e2f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7fa800b810 tx=0x7f7fa8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:30.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.376+0000 7f7fb17fa640 1 -- 172.21.15.32:0/2967180295 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7fb406c160 con 0x7f7fb4158dd0 2024-04-24T22:59:30.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.556+0000 7f7fbafd5640 1 -- 172.21.15.32:0/2967180295 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7fb4068430 con 0x7f7fb4158dd0 2024-04-24T22:59:30.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.557+0000 7f7fb17fa640 1 -- 172.21.15.32:0/2967180295 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f7fac07a870 con 0x7f7fb4158dd0 2024-04-24T22:59:30.559 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-24T22:59:30.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.558+0000 7f7fbafd5640 1 -- 172.21.15.32:0/2967180295 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7f9806be30 msgr2=0x7f7f9806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:30.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.558+0000 7f7fbafd5640 1 --2- 172.21.15.32:0/2967180295 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7f9806be30 0x7f7f9806e2f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7fa800b810 tx=0x7f7fa8008040 comp rx=0 tx=0).stop 2024-04-24T22:59:30.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.558+0000 7f7fbafd5640 1 -- 172.21.15.32:0/2967180295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7fb4158dd0 msgr2=0x7f7fb414cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:30.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.558+0000 7f7fbafd5640 1 --2- 172.21.15.32:0/2967180295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7fb4158dd0 0x7f7fb414cc30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7fac009eb0 tx=0x7f7fac004a40 comp rx=0 tx=0).stop 2024-04-24T22:59:30.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.559+0000 7f7fbafd5640 1 -- 172.21.15.32:0/2967180295 shutdown_connections 2024-04-24T22:59:30.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.559+0000 7f7fbafd5640 1 --2- 172.21.15.32:0/2967180295 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7f9806be30 0x7f7f9806e2f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:30.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.559+0000 7f7fbafd5640 1 --2- 172.21.15.32:0/2967180295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7fb4158dd0 0x7f7fb414cc30 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:30.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.559+0000 7f7fbafd5640 1 --2- 172.21.15.32:0/2967180295 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7fb406dc60 0x7f7fb41f4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:30.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.559+0000 7f7fbafd5640 1 --2- 172.21.15.32:0/2967180295 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7fb406b860 0x7f7fb41f7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:30.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.559+0000 7f7fbafd5640 1 -- 172.21.15.32:0/2967180295 >> 172.21.15.32:0/2967180295 conn(0x7f7fb406e800 msgr2=0x7f7fb414f030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:30.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.559+0000 7f7fbafd5640 1 -- 172.21.15.32:0/2967180295 shutdown_connections 2024-04-24T22:59:30.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.559+0000 7f7fbafd5640 1 -- 172.21.15.32:0/2967180295 wait complete. 2024-04-24T22:59:30.572 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:59:30.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.670+0000 7f1b2c25d640 1 Processor -- start 2024-04-24T22:59:30.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.670+0000 7f1b2c25d640 1 -- start start 2024-04-24T22:59:30.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.671+0000 7f1b2c25d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1b2406c5f0 0x7f1b2406c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:30.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.671+0000 7f1b2c25d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b240b44d0 0x7f1b240b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:30.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.671+0000 7f1b2c25d640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b240b6430 con 0x7f1b240af890 2024-04-24T22:59:30.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.671+0000 7f1b2c25d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f1b24156f80 con 0x7f1b2406c5f0 2024-04-24T22:59:30.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.671+0000 7f1b2c25d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f1b240b6150 con 0x7f1b240b44d0 2024-04-24T22:59:30.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.671+0000 7f1b29ff9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1b2406c5f0 0x7f1b2406c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:30.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.671+0000 7f1b29ff9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1b2406c5f0 0x7f1b2406c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52082/0 (socket says 172.21.15.32:52082) 2024-04-24T22:59:30.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.671+0000 7f1b29ff9640 1 -- 172.21.15.32:0/639447339 learned_addr learned my addr 172.21.15.32:0/639447339 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:30.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.671+0000 7f1b29ff9640 1 -- 172.21.15.32:0/639447339 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b240b44d0 msgr2=0x7f1b240b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:30.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.671+0000 7f1b297f8640 1 --2- 172.21.15.32:0/639447339 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b240b44d0 0x7f1b240b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:30.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.671+0000 7f1b29ff9640 1 --2- 172.21.15.32:0/639447339 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b240b44d0 0x7f1b240b5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:30.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.671+0000 7f1b29ff9640 1 -- 172.21.15.32:0/639447339 >> v1:172.21.15.121:6789/0 conn(0x7f1b240af890 legacy=0x7f1b240cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:30.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.671+0000 7f1b29ff9640 1 -- 172.21.15.32:0/639447339 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b240b39f0 con 0x7f1b2406c5f0 2024-04-24T22:59:30.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.671+0000 7f1b297f8640 1 --2- 172.21.15.32:0/639447339 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b240b44d0 0x7f1b240b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:59:30.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.672+0000 7f1b29ff9640 1 --2- 172.21.15.32:0/639447339 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b2406c5f0 0x7f1b2406c9d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1b20009870 tx=0x7f1b2002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a287ddba81e0d35 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:30.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.672+0000 7f1b28ff7640 1 -- 172.21.15.32:0/639447339 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b20037070 con 0x7f1b2406c5f0 2024-04-24T22:59:30.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.672+0000 7f1b28ff7640 1 -- 172.21.15.32:0/639447339 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b20032440 con 0x7f1b2406c5f0 2024-04-24T22:59:30.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.672+0000 7f1b28ff7640 1 -- 172.21.15.32:0/639447339 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b20032810 con 0x7f1b2406c5f0 2024-04-24T22:59:30.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.673+0000 7f1b2c25d640 1 -- 172.21.15.32:0/639447339 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b2406c5f0 msgr2=0x7f1b2406c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:30.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.673+0000 7f1b2c25d640 1 --2- 172.21.15.32:0/639447339 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b2406c5f0 0x7f1b2406c9d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1b20009870 tx=0x7f1b2002a180 comp rx=0 tx=0).stop 2024-04-24T22:59:30.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.673+0000 7f1b2c25d640 1 -- 172.21.15.32:0/639447339 shutdown_connections 2024-04-24T22:59:30.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.673+0000 7f1b2c25d640 1 --2- 172.21.15.32:0/639447339 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b240b44d0 0x7f1b240b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:30.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.673+0000 7f1b2c25d640 1 --2- 172.21.15.32:0/639447339 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b2406c5f0 0x7f1b2406c9d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:30.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.673+0000 7f1b2c25d640 1 -- 172.21.15.32:0/639447339 >> 172.21.15.32:0/639447339 conn(0x7f1b2406b110 msgr2=0x7f1b2406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:30.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.673+0000 7f1b2c25d640 1 -- 172.21.15.32:0/639447339 shutdown_connections 2024-04-24T22:59:30.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.673+0000 7f1b2c25d640 1 -- 172.21.15.32:0/639447339 wait complete. 2024-04-24T22:59:30.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.674+0000 7f1b2c25d640 1 Processor -- start 2024-04-24T22:59:30.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.674+0000 7f1b2c25d640 1 -- start start 2024-04-24T22:59:30.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.675+0000 7f1b2c25d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b2406c5f0 0x7f1b240ab190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:30.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.675+0000 7f1b2c25d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b240af890 0x7f1b24071f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:30.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.675+0000 7f1b2c25d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b240b44d0 0x7f1b24072450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:30.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.675+0000 7f1b2c25d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2406cf10 con 0x7f1b2406c5f0 2024-04-24T22:59:30.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.675+0000 7f1b2c25d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1b240b3fb0 con 0x7f1b240af890 2024-04-24T22:59:30.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.675+0000 7f1b2c25d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f1b240b5dd0 con 0x7f1b240b44d0 2024-04-24T22:59:30.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.675+0000 7f1b29ff9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b2406c5f0 0x7f1b240ab190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:30.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.675+0000 7f1b29ff9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b2406c5f0 0x7f1b240ab190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52098/0 (socket says 172.21.15.32:52098) 2024-04-24T22:59:30.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.675+0000 7f1b29ff9640 1 -- 172.21.15.32:0/4170242487 learned_addr learned my addr 172.21.15.32:0/4170242487 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:30.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.675+0000 7f1b2a7fa640 1 --2- 172.21.15.32:0/4170242487 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b240b44d0 0x7f1b24072450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:30.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.675+0000 7f1b297f8640 1 --2- 172.21.15.32:0/4170242487 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b240af890 0x7f1b24071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:30.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.675+0000 7f1b29ff9640 1 -- 172.21.15.32:0/4170242487 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b240b44d0 msgr2=0x7f1b24072450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:30.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.675+0000 7f1b29ff9640 1 --2- 172.21.15.32:0/4170242487 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b240b44d0 0x7f1b24072450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:30.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.675+0000 7f1b29ff9640 1 -- 172.21.15.32:0/4170242487 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b240af890 msgr2=0x7f1b24071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:30.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.675+0000 7f1b29ff9640 1 --2- 172.21.15.32:0/4170242487 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b240af890 0x7f1b24071f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:30.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.675+0000 7f1b29ff9640 1 -- 172.21.15.32:0/4170242487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b240ae0a0 con 0x7f1b2406c5f0 2024-04-24T22:59:30.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.675+0000 7f1b29ff9640 1 --2- 172.21.15.32:0/4170242487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b2406c5f0 0x7f1b240ab190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1b20009870 tx=0x7f1b200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:30.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.675+0000 7f1b1a7fc640 1 -- 172.21.15.32:0/4170242487 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b20041020 con 0x7f1b2406c5f0 2024-04-24T22:59:30.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.676+0000 7f1b1a7fc640 1 -- 172.21.15.32:0/4170242487 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b20030970 con 0x7f1b2406c5f0 2024-04-24T22:59:30.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.676+0000 7f1b2c25d640 1 -- 172.21.15.32:0/4170242487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b240aa6b0 con 0x7f1b2406c5f0 2024-04-24T22:59:30.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.676+0000 7f1b1a7fc640 1 -- 172.21.15.32:0/4170242487 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b20037040 con 0x7f1b2406c5f0 2024-04-24T22:59:30.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.676+0000 7f1b2c25d640 1 -- 172.21.15.32:0/4170242487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b240aac70 con 0x7f1b2406c5f0 2024-04-24T22:59:30.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.677+0000 7f1b2c25d640 1 -- 172.21.15.32:0/4170242487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b240aa4a0 con 0x7f1b2406c5f0 2024-04-24T22:59:30.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.677+0000 7f1b1a7fc640 1 -- 172.21.15.32:0/4170242487 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1b20044020 con 0x7f1b2406c5f0 2024-04-24T22:59:30.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.677+0000 7f1b1a7fc640 1 --2- 172.21.15.32:0/4170242487 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1b0806be30 0x7f1b0806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:30.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.678+0000 7f1b1a7fc640 1 -- 172.21.15.32:0/4170242487 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1b200ab7c0 con 0x7f1b2406c5f0 2024-04-24T22:59:30.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.678+0000 7f1b297f8640 1 --2- 172.21.15.32:0/4170242487 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1b0806be30 0x7f1b0806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:30.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.678+0000 7f1b297f8640 1 --2- 172.21.15.32:0/4170242487 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1b0806be30 0x7f1b0806e2f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1b2415c260 tx=0x7f1b14008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:30.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.682+0000 7f1b1a7fc640 1 -- 172.21.15.32:0/4170242487 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1b2004b030 con 0x7f1b2406c5f0 2024-04-24T22:59:30.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.864+0000 7f1b2c25d640 1 -- 172.21.15.32:0/4170242487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1b24068430 con 0x7f1b2406c5f0 2024-04-24T22:59:30.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.865+0000 7f1b1a7fc640 1 -- 172.21.15.32:0/4170242487 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16621 (secure 0 0 0) 0x7f1b2007a200 con 0x7f1b2406c5f0 2024-04-24T22:59:30.868 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:59:30.868 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T22:59:05.517554+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:59:02.016639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T22:59:02.846087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:59:30.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.867+0000 7f1b2c25d640 1 -- 172.21.15.32:0/4170242487 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1b0806be30 msgr2=0x7f1b0806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:30.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.867+0000 7f1b2c25d640 1 --2- 172.21.15.32:0/4170242487 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1b0806be30 0x7f1b0806e2f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1b2415c260 tx=0x7f1b14008040 comp rx=0 tx=0).stop 2024-04-24T22:59:30.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.867+0000 7f1b2c25d640 1 -- 172.21.15.32:0/4170242487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b2406c5f0 msgr2=0x7f1b240ab190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:30.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.867+0000 7f1b2c25d640 1 --2- 172.21.15.32:0/4170242487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b2406c5f0 0x7f1b240ab190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1b20009870 tx=0x7f1b200046c0 comp rx=0 tx=0).stop 2024-04-24T22:59:30.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.867+0000 7f1b2c25d640 1 -- 172.21.15.32:0/4170242487 shutdown_connections 2024-04-24T22:59:30.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.867+0000 7f1b2c25d640 1 --2- 172.21.15.32:0/4170242487 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1b0806be30 0x7f1b0806e2f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:30.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.867+0000 7f1b2c25d640 1 --2- 172.21.15.32:0/4170242487 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b240b44d0 0x7f1b24072450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:30.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.867+0000 7f1b2c25d640 1 --2- 172.21.15.32:0/4170242487 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b240af890 0x7f1b24071f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:30.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.867+0000 7f1b2c25d640 1 --2- 172.21.15.32:0/4170242487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b2406c5f0 0x7f1b240ab190 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:30.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.867+0000 7f1b2c25d640 1 -- 172.21.15.32:0/4170242487 >> 172.21.15.32:0/4170242487 conn(0x7f1b2406b110 msgr2=0x7f1b2415c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:30.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.867+0000 7f1b2c25d640 1 -- 172.21.15.32:0/4170242487 shutdown_connections 2024-04-24T22:59:30.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.867+0000 7f1b2c25d640 1 -- 172.21.15.32:0/4170242487 wait complete. 2024-04-24T22:59:30.881 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T22:59:02.016639+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T22:59:02.846087+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T22:59:30.881 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T22:59:30.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.982+0000 7fdc1fad4640 1 Processor -- start 2024-04-24T22:59:30.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.982+0000 7fdc1fad4640 1 -- start start 2024-04-24T22:59:30.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.983+0000 7fdc1fad4640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdc1815b480 0x7fdc180b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:30.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.983+0000 7fdc1fad4640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc180b46c0 con 0x7fdc1815fd60 2024-04-24T22:59:30.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.983+0000 7fdc1fad4640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc180b49a0 con 0x7fdc1806cd00 2024-04-24T22:59:30.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.983+0000 7fdc1fad4640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fdc180b4ff0 con 0x7fdc1815b480 2024-04-24T22:59:30.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.983+0000 7fdc1d870640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdc1815b480 0x7fdc180b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:30.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.983+0000 7fdc1d06f640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fdc1815fd60 0x7fdc1806c5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34040/0 (socket says 172.21.15.32:34040) 2024-04-24T22:59:30.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.983+0000 7fdc1d06f640 1 -- 172.21.15.32:0/3016106422 learned_addr learned my addr 172.21.15.32:0/3016106422 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:30.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.984+0000 7fdc1d870640 1 -- 172.21.15.32:0/3016106422 >> v1:172.21.15.121:6789/0 conn(0x7fdc1806cd00 legacy=0x7fdc180cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:30.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.984+0000 7fdc1d870640 1 -- 172.21.15.32:0/3016106422 >> v1:172.21.15.32:6789/0 conn(0x7fdc1815fd60 legacy=0x7fdc1806c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:30.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.984+0000 7fdc1d870640 1 -- 172.21.15.32:0/3016106422 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc180b4c80 con 0x7fdc1815b480 2024-04-24T22:59:30.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.984+0000 7fdc1c86e640 1 -- 172.21.15.32:0/3016106422 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 834179107 0 0) 0x7fdc180b46c0 con 0x7fdc1815fd60 2024-04-24T22:59:30.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.984+0000 7fdc1d870640 1 --2- 172.21.15.32:0/3016106422 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc1815b480 0x7fdc180b5c60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdc180b5170 tx=0x7fdc1002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c4d1d0d842aa9dc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:30.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.984+0000 7fdc1c86e640 1 -- 172.21.15.32:0/3016106422 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc10037070 con 0x7fdc1815b480 2024-04-24T22:59:30.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.984+0000 7fdc1c86e640 1 -- 172.21.15.32:0/3016106422 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc10032440 con 0x7fdc1815b480 2024-04-24T22:59:30.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.985+0000 7fdc1c86e640 1 -- 172.21.15.32:0/3016106422 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc10032810 con 0x7fdc1815b480 2024-04-24T22:59:30.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.985+0000 7fdc1fad4640 1 -- 172.21.15.32:0/3016106422 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc1815b480 msgr2=0x7fdc180b5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:30.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.985+0000 7fdc1fad4640 1 --2- 172.21.15.32:0/3016106422 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc1815b480 0x7fdc180b5c60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdc180b5170 tx=0x7fdc1002a210 comp rx=0 tx=0).stop 2024-04-24T22:59:30.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.985+0000 7fdc1fad4640 1 -- 172.21.15.32:0/3016106422 shutdown_connections 2024-04-24T22:59:30.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.985+0000 7fdc1fad4640 1 --2- 172.21.15.32:0/3016106422 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc1815b480 0x7fdc180b5c60 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:30.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.985+0000 7fdc1fad4640 1 -- 172.21.15.32:0/3016106422 >> 172.21.15.32:0/3016106422 conn(0x7fdc1806b110 msgr2=0x7fdc1806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:30.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.985+0000 7fdc1fad4640 1 -- 172.21.15.32:0/3016106422 shutdown_connections 2024-04-24T22:59:30.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.985+0000 7fdc1fad4640 1 -- 172.21.15.32:0/3016106422 wait complete. 2024-04-24T22:59:30.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.986+0000 7fdc1fad4640 1 Processor -- start 2024-04-24T22:59:30.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.986+0000 7fdc1fad4640 1 -- start start 2024-04-24T22:59:30.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.987+0000 7fdc1fad4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc1806cd00 0x7fdc18140d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:30.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.987+0000 7fdc1fad4640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdc1815b480 0x7fdc181515b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:30.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.987+0000 7fdc1fad4640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc1815fd60 0x7fdc181416e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:30.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.987+0000 7fdc1fad4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1815b860 con 0x7fdc1806cd00 2024-04-24T22:59:30.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.987+0000 7fdc1fad4640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdc180b5480 con 0x7fdc1815fd60 2024-04-24T22:59:30.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.987+0000 7fdc1fad4640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fdc18151e70 con 0x7fdc1815b480 2024-04-24T22:59:30.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.987+0000 7fdc1d870640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc1806cd00 0x7fdc18140d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:30.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.987+0000 7fdc1d870640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc1806cd00 0x7fdc18140d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52106/0 (socket says 172.21.15.32:52106) 2024-04-24T22:59:30.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.987+0000 7fdc1d870640 1 -- 172.21.15.32:0/555356913 learned_addr learned my addr 172.21.15.32:0/555356913 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:30.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.987+0000 7fdc1d06f640 1 --2- 172.21.15.32:0/555356913 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdc1815b480 0x7fdc181515b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:30.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.987+0000 7fdc1e071640 1 --2- 172.21.15.32:0/555356913 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc1815fd60 0x7fdc181416e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:30.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.987+0000 7fdc1d870640 1 -- 172.21.15.32:0/555356913 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdc1815b480 msgr2=0x7fdc181515b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:30.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.987+0000 7fdc1d870640 1 --2- 172.21.15.32:0/555356913 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdc1815b480 0x7fdc181515b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:30.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.987+0000 7fdc1d870640 1 -- 172.21.15.32:0/555356913 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc1815fd60 msgr2=0x7fdc181416e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:30.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.987+0000 7fdc1d870640 1 --2- 172.21.15.32:0/555356913 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc1815fd60 0x7fdc181416e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:30.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.987+0000 7fdc1d870640 1 -- 172.21.15.32:0/555356913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc18152150 con 0x7fdc1806cd00 2024-04-24T22:59:30.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.988+0000 7fdc1d870640 1 --2- 172.21.15.32:0/555356913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc1806cd00 0x7fdc18140d00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdc10009890 tx=0x7fdc100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:30.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.988+0000 7fdc167fc640 1 -- 172.21.15.32:0/555356913 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc10041070 con 0x7fdc1806cd00 2024-04-24T22:59:30.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.988+0000 7fdc167fc640 1 -- 172.21.15.32:0/555356913 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc1003c890 con 0x7fdc1806cd00 2024-04-24T22:59:30.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.988+0000 7fdc1fad4640 1 -- 172.21.15.32:0/555356913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc18152430 con 0x7fdc1806cd00 2024-04-24T22:59:30.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.988+0000 7fdc167fc640 1 -- 172.21.15.32:0/555356913 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc10037040 con 0x7fdc1806cd00 2024-04-24T22:59:30.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.988+0000 7fdc1fad4640 1 -- 172.21.15.32:0/555356913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc18151af0 con 0x7fdc1806cd00 2024-04-24T22:59:30.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.989+0000 7fdc1fad4640 1 -- 172.21.15.32:0/555356913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc18068430 con 0x7fdc1806cd00 2024-04-24T22:59:30.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.989+0000 7fdc167fc640 1 -- 172.21.15.32:0/555356913 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdc10044070 con 0x7fdc1806cd00 2024-04-24T22:59:30.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.990+0000 7fdc167fc640 1 --2- 172.21.15.32:0/555356913 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdc0406be30 0x7fdc0406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:30.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.990+0000 7fdc167fc640 1 -- 172.21.15.32:0/555356913 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fdc100abb10 con 0x7fdc1806cd00 2024-04-24T22:59:30.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.990+0000 7fdc1d06f640 1 --2- 172.21.15.32:0/555356913 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdc0406be30 0x7fdc0406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:30.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.990+0000 7fdc1d06f640 1 --2- 172.21.15.32:0/555356913 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdc0406be30 0x7fdc0406e2f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdc1814f0b0 tx=0x7fdc0c007e40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:30.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:30.994+0000 7fdc167fc640 1 -- 172.21.15.32:0/555356913 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdc1007a020 con 0x7fdc1806cd00 2024-04-24T22:59:31.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.173+0000 7fdc1fad4640 1 -- 172.21.15.32:0/555356913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdc1806c000 con 0x7fdc1806cd00 2024-04-24T22:59:31.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.174+0000 7fdc167fc640 1 -- 172.21.15.32:0/555356913 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7fdc1007e320 con 0x7fdc1806cd00 2024-04-24T22:59:31.176 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T22:59:31.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.175+0000 7fdc1fad4640 1 -- 172.21.15.32:0/555356913 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdc0406be30 msgr2=0x7fdc0406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:31.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.175+0000 7fdc1fad4640 1 --2- 172.21.15.32:0/555356913 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdc0406be30 0x7fdc0406e2f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdc1814f0b0 tx=0x7fdc0c007e40 comp rx=0 tx=0).stop 2024-04-24T22:59:31.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.175+0000 7fdc1fad4640 1 -- 172.21.15.32:0/555356913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc1806cd00 msgr2=0x7fdc18140d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:31.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.175+0000 7fdc1fad4640 1 --2- 172.21.15.32:0/555356913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc1806cd00 0x7fdc18140d00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdc10009890 tx=0x7fdc100046c0 comp rx=0 tx=0).stop 2024-04-24T22:59:31.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.175+0000 7fdc1fad4640 1 -- 172.21.15.32:0/555356913 shutdown_connections 2024-04-24T22:59:31.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.175+0000 7fdc1fad4640 1 --2- 172.21.15.32:0/555356913 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdc0406be30 0x7fdc0406e2f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:31.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.175+0000 7fdc1fad4640 1 --2- 172.21.15.32:0/555356913 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc1815fd60 0x7fdc181416e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:31.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.175+0000 7fdc1fad4640 1 --2- 172.21.15.32:0/555356913 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdc1815b480 0x7fdc181515b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:31.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.175+0000 7fdc1fad4640 1 --2- 172.21.15.32:0/555356913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc1806cd00 0x7fdc18140d00 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:31.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.175+0000 7fdc1fad4640 1 -- 172.21.15.32:0/555356913 >> 172.21.15.32:0/555356913 conn(0x7fdc1806b110 msgr2=0x7fdc180b3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:31.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.176+0000 7fdc1fad4640 1 -- 172.21.15.32:0/555356913 shutdown_connections 2024-04-24T22:59:31.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.176+0000 7fdc1fad4640 1 -- 172.21.15.32:0/555356913 wait complete. 2024-04-24T22:59:31.189 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T22:59:31.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.290+0000 7efd47bd5640 1 Processor -- start 2024-04-24T22:59:31.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.290+0000 7efd47bd5640 1 -- start start 2024-04-24T22:59:31.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.291+0000 7efd47bd5640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7efd4015c5b0 0x7efd400b4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:31.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.291+0000 7efd47bd5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efd40072cf0 0x7efd4006c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:31.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.291+0000 7efd47bd5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efd40071770 0x7efd4006cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:31.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.291+0000 7efd47bd5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7efd40072860 con 0x7efd40071770 2024-04-24T22:59:31.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.291+0000 7efd47bd5640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7efd4015c990 con 0x7efd4015c5b0 2024-04-24T22:59:31.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.291+0000 7efd47bd5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7efd40071c50 con 0x7efd40072cf0 2024-04-24T22:59:31.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.291+0000 7efd46172640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efd40071770 0x7efd4006cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:31.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.291+0000 7efd46172640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efd40071770 0x7efd4006cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52112/0 (socket says 172.21.15.32:52112) 2024-04-24T22:59:31.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.291+0000 7efd46172640 1 -- 172.21.15.32:0/3496768523 learned_addr learned my addr 172.21.15.32:0/3496768523 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:31.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.291+0000 7efd45170640 1 --2- 172.21.15.32:0/3496768523 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efd40072cf0 0x7efd4006c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:31.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.291+0000 7efd45971640 1 --2- 172.21.15.32:0/3496768523 >> v2:172.21.15.121:3300/0 conn(0x7efd4015c5b0 0x7efd400b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:31.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.291+0000 7efd46172640 1 -- 172.21.15.32:0/3496768523 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efd40072cf0 msgr2=0x7efd4006c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:31.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.291+0000 7efd46172640 1 --2- 172.21.15.32:0/3496768523 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efd40072cf0 0x7efd4006c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:31.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.291+0000 7efd46172640 1 -- 172.21.15.32:0/3496768523 >> v2:172.21.15.121:3300/0 conn(0x7efd4015c5b0 msgr2=0x7efd400b4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:31.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.291+0000 7efd46172640 1 --2- 172.21.15.32:0/3496768523 >> v2:172.21.15.121:3300/0 conn(0x7efd4015c5b0 0x7efd400b4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:31.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.291+0000 7efd46172640 1 -- 172.21.15.32:0/3496768523 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd40071f30 con 0x7efd40071770 2024-04-24T22:59:31.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.291+0000 7efd45170640 1 --2- 172.21.15.32:0/3496768523 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efd40072cf0 0x7efd4006c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:59:31.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.291+0000 7efd45971640 1 --2- 172.21.15.32:0/3496768523 >> v2:172.21.15.121:3300/0 conn(0x7efd4015c5b0 0x7efd400b4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:59:31.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.292+0000 7efd46172640 1 --2- 172.21.15.32:0/3496768523 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd40071770 0x7efd4006cb30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efd40067850 tx=0x7efd3802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14f7ae54cdce3abb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:31.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.292+0000 7efd4496f640 1 -- 172.21.15.32:0/3496768523 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd38036070 con 0x7efd40071770 2024-04-24T22:59:31.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.292+0000 7efd4496f640 1 -- 172.21.15.32:0/3496768523 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd38031600 con 0x7efd40071770 2024-04-24T22:59:31.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.292+0000 7efd4496f640 1 -- 172.21.15.32:0/3496768523 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd380319d0 con 0x7efd40071770 2024-04-24T22:59:31.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.293+0000 7efd47bd5640 1 -- 172.21.15.32:0/3496768523 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd40071770 msgr2=0x7efd4006cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:31.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.293+0000 7efd47bd5640 1 --2- 172.21.15.32:0/3496768523 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd40071770 0x7efd4006cb30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efd40067850 tx=0x7efd3802a610 comp rx=0 tx=0).stop 2024-04-24T22:59:31.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.293+0000 7efd47bd5640 1 -- 172.21.15.32:0/3496768523 shutdown_connections 2024-04-24T22:59:31.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.293+0000 7efd47bd5640 1 --2- 172.21.15.32:0/3496768523 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd40071770 0x7efd4006cb30 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:31.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.293+0000 7efd47bd5640 1 --2- 172.21.15.32:0/3496768523 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efd40072cf0 0x7efd4006c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:31.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.293+0000 7efd47bd5640 1 --2- 172.21.15.32:0/3496768523 >> v2:172.21.15.121:3300/0 conn(0x7efd4015c5b0 0x7efd400b4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:31.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.293+0000 7efd47bd5640 1 -- 172.21.15.32:0/3496768523 >> 172.21.15.32:0/3496768523 conn(0x7efd4006b110 msgr2=0x7efd4006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:31.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.293+0000 7efd47bd5640 1 -- 172.21.15.32:0/3496768523 shutdown_connections 2024-04-24T22:59:31.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.293+0000 7efd47bd5640 1 -- 172.21.15.32:0/3496768523 wait complete. 2024-04-24T22:59:31.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.294+0000 7efd47bd5640 1 Processor -- start 2024-04-24T22:59:31.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.294+0000 7efd47bd5640 1 -- start start 2024-04-24T22:59:31.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.295+0000 7efd47bd5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efd40071770 0x7efd4014f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:31.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.295+0000 7efd47bd5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd40072cf0 0x7efd400ce4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:31.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.295+0000 7efd47bd5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efd4015c5b0 0x7efd400ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:31.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.295+0000 7efd47bd5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7efd400b4570 con 0x7efd40072cf0 2024-04-24T22:59:31.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.295+0000 7efd47bd5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7efd400724f0 con 0x7efd4015c5b0 2024-04-24T22:59:31.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.295+0000 7efd47bd5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7efd400b46f0 con 0x7efd40071770 2024-04-24T22:59:31.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.295+0000 7efd45170640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd40072cf0 0x7efd400ce4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:31.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.295+0000 7efd45170640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd40072cf0 0x7efd400ce4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52114/0 (socket says 172.21.15.32:52114) 2024-04-24T22:59:31.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.295+0000 7efd45170640 1 -- 172.21.15.32:0/298866687 learned_addr learned my addr 172.21.15.32:0/298866687 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:31.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.295+0000 7efd45971640 1 --2- 172.21.15.32:0/298866687 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efd40071770 0x7efd4014f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:31.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.295+0000 7efd46172640 1 --2- 172.21.15.32:0/298866687 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efd4015c5b0 0x7efd400ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:31.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.295+0000 7efd45170640 1 -- 172.21.15.32:0/298866687 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efd40071770 msgr2=0x7efd4014f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:31.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.295+0000 7efd45170640 1 --2- 172.21.15.32:0/298866687 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efd40071770 0x7efd4014f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:31.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.295+0000 7efd45170640 1 -- 172.21.15.32:0/298866687 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efd4015c5b0 msgr2=0x7efd400ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:31.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.295+0000 7efd45170640 1 --2- 172.21.15.32:0/298866687 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efd4015c5b0 0x7efd400ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:31.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.295+0000 7efd45170640 1 -- 172.21.15.32:0/298866687 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd4014f660 con 0x7efd40072cf0 2024-04-24T22:59:31.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.295+0000 7efd46172640 1 --2- 172.21.15.32:0/298866687 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efd4015c5b0 0x7efd400ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:31.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.295+0000 7efd45170640 1 --2- 172.21.15.32:0/298866687 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd40072cf0 0x7efd400ce4d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efd3000c7c0 tx=0x7efd3000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:31.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.295+0000 7efd367fc640 1 -- 172.21.15.32:0/298866687 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd30015070 con 0x7efd40072cf0 2024-04-24T22:59:31.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.295+0000 7efd367fc640 1 -- 172.21.15.32:0/298866687 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd30007a30 con 0x7efd40072cf0 2024-04-24T22:59:31.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.296+0000 7efd47bd5640 1 -- 172.21.15.32:0/298866687 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd4015cb70 con 0x7efd40072cf0 2024-04-24T22:59:31.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.296+0000 7efd367fc640 1 -- 172.21.15.32:0/298866687 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd30007e00 con 0x7efd40072cf0 2024-04-24T22:59:31.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.296+0000 7efd47bd5640 1 -- 172.21.15.32:0/298866687 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd4015d130 con 0x7efd40072cf0 2024-04-24T22:59:31.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.296+0000 7efd47bd5640 1 -- 172.21.15.32:0/298866687 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd40068430 con 0x7efd40072cf0 2024-04-24T22:59:31.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.297+0000 7efd367fc640 1 -- 172.21.15.32:0/298866687 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efd30018070 con 0x7efd40072cf0 2024-04-24T22:59:31.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.297+0000 7efd367fc640 1 --2- 172.21.15.32:0/298866687 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7efd2406be30 0x7efd2406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:31.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.297+0000 7efd367fc640 1 -- 172.21.15.32:0/298866687 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7efd3008ac70 con 0x7efd40072cf0 2024-04-24T22:59:31.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.298+0000 7efd45971640 1 --2- 172.21.15.32:0/298866687 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7efd2406be30 0x7efd2406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:31.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.298+0000 7efd45971640 1 --2- 172.21.15.32:0/298866687 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7efd2406be30 0x7efd2406e2f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7efd3c00aac0 tx=0x7efd3c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:31.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.301+0000 7efd367fc640 1 -- 172.21.15.32:0/298866687 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7efd30058fd0 con 0x7efd40072cf0 2024-04-24T22:59:31.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.485+0000 7efd47bd5640 1 -- 172.21.15.32:0/298866687 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7efd400b53e0 con 0x7efd40072cf0 2024-04-24T22:59:31.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.485+0000 7efd367fc640 1 -- 172.21.15.32:0/298866687 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7efd3005d2d0 con 0x7efd40072cf0 2024-04-24T22:59:31.488 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-24T22:59:31.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.487+0000 7efd47bd5640 1 -- 172.21.15.32:0/298866687 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7efd2406be30 msgr2=0x7efd2406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:31.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.487+0000 7efd47bd5640 1 --2- 172.21.15.32:0/298866687 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7efd2406be30 0x7efd2406e2f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7efd3c00aac0 tx=0x7efd3c008040 comp rx=0 tx=0).stop 2024-04-24T22:59:31.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.487+0000 7efd47bd5640 1 -- 172.21.15.32:0/298866687 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd40072cf0 msgr2=0x7efd400ce4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:31.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.487+0000 7efd47bd5640 1 --2- 172.21.15.32:0/298866687 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd40072cf0 0x7efd400ce4d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efd3000c7c0 tx=0x7efd3000cc80 comp rx=0 tx=0).stop 2024-04-24T22:59:31.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.487+0000 7efd47bd5640 1 -- 172.21.15.32:0/298866687 shutdown_connections 2024-04-24T22:59:31.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.487+0000 7efd47bd5640 1 --2- 172.21.15.32:0/298866687 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7efd2406be30 0x7efd2406e2f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:31.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.487+0000 7efd47bd5640 1 --2- 172.21.15.32:0/298866687 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd40072cf0 0x7efd400ce4d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:31.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.487+0000 7efd47bd5640 1 --2- 172.21.15.32:0/298866687 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efd4015c5b0 0x7efd400ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:31.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.487+0000 7efd47bd5640 1 --2- 172.21.15.32:0/298866687 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efd40071770 0x7efd4014f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:31.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.487+0000 7efd47bd5640 1 -- 172.21.15.32:0/298866687 >> 172.21.15.32:0/298866687 conn(0x7efd4006b110 msgr2=0x7efd400b5be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:31.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.487+0000 7efd47bd5640 1 -- 172.21.15.32:0/298866687 shutdown_connections 2024-04-24T22:59:31.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.487+0000 7efd47bd5640 1 -- 172.21.15.32:0/298866687 wait complete. 2024-04-24T22:59:31.501 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T22:59:31.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.602+0000 7fb8c48c1640 1 Processor -- start 2024-04-24T22:59:31.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.602+0000 7fb8c48c1640 1 -- start start 2024-04-24T22:59:31.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.603+0000 7fb8c48c1640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb8c00d1d60 0x7fb8c0070720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:31.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.603+0000 7fb8c48c1640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8c015e9e0 0x7fb8c00725c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:31.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.603+0000 7fb8c48c1640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8c006b2b0 con 0x7fb8c015e600 2024-04-24T22:59:31.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.603+0000 7fb8c48c1640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fb8c0070290 con 0x7fb8c00d1d60 2024-04-24T22:59:31.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.603+0000 7fb8c48c1640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fb8c006b590 con 0x7fb8c015e9e0 2024-04-24T22:59:31.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.603+0000 7fb8be59c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb8c015e600 0x7fb8c006bb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34044/0 (socket says 172.21.15.32:34044) 2024-04-24T22:59:31.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.603+0000 7fb8be59c640 1 -- 172.21.15.32:0/1220106348 learned_addr learned my addr 172.21.15.32:0/1220106348 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:31.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.603+0000 7fb8bed9d640 1 --2- 172.21.15.32:0/1220106348 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8c015e9e0 0x7fb8c00725c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:31.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.603+0000 7fb8b7fff640 1 --2- 172.21.15.32:0/1220106348 >> v2:172.21.15.121:3300/0 conn(0x7fb8c00d1d60 0x7fb8c0070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:31.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.603+0000 7fb8bdd9b640 1 -- 172.21.15.32:0/1220106348 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1641181966 0 0) 0x7fb8c006b2b0 con 0x7fb8c015e600 2024-04-24T22:59:31.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.603+0000 7fb8bdd9b640 1 -- 172.21.15.32:0/1220106348 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb8ac003520 con 0x7fb8c015e600 2024-04-24T22:59:31.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.603+0000 7fb8bed9d640 1 -- 172.21.15.32:0/1220106348 >> v2:172.21.15.121:3300/0 conn(0x7fb8c00d1d60 msgr2=0x7fb8c0070720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:31.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.603+0000 7fb8bed9d640 1 --2- 172.21.15.32:0/1220106348 >> v2:172.21.15.121:3300/0 conn(0x7fb8c00d1d60 0x7fb8c0070720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:31.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.603+0000 7fb8bed9d640 1 -- 172.21.15.32:0/1220106348 >> v1:172.21.15.32:6789/0 conn(0x7fb8c015e600 legacy=0x7fb8c006bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:31.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.603+0000 7fb8bdd9b640 1 -- 172.21.15.32:0/1220106348 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 484887148 0 0) 0x7fb8ac003520 con 0x7fb8c015e600 2024-04-24T22:59:31.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.603+0000 7fb8bed9d640 1 -- 172.21.15.32:0/1220106348 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8c006ff20 con 0x7fb8c015e9e0 2024-04-24T22:59:31.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.604+0000 7fb8bed9d640 1 --2- 172.21.15.32:0/1220106348 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8c015e9e0 0x7fb8c00725c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8009d90 tx=0x7fb8b802a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=834783648aeabac1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:31.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.604+0000 7fb8bdd9b640 1 -- 172.21.15.32:0/1220106348 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b0001670 con 0x7fb8c015e9e0 2024-04-24T22:59:31.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.604+0000 7fb8bdd9b640 1 -- 172.21.15.32:0/1220106348 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8b8007540 con 0x7fb8c015e9e0 2024-04-24T22:59:31.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.604+0000 7fb8bdd9b640 1 -- 172.21.15.32:0/1220106348 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b8007910 con 0x7fb8c015e9e0 2024-04-24T22:59:31.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.605+0000 7fb8c48c1640 1 -- 172.21.15.32:0/1220106348 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8c015e9e0 msgr2=0x7fb8c00725c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:31.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.605+0000 7fb8c48c1640 1 --2- 172.21.15.32:0/1220106348 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8c015e9e0 0x7fb8c00725c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8009d90 tx=0x7fb8b802a5f0 comp rx=0 tx=0).stop 2024-04-24T22:59:31.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.605+0000 7fb8c48c1640 1 -- 172.21.15.32:0/1220106348 shutdown_connections 2024-04-24T22:59:31.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.605+0000 7fb8c48c1640 1 --2- 172.21.15.32:0/1220106348 >> v2:172.21.15.121:3300/0 conn(0x7fb8c00d1d60 0x7fb8c0070720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:31.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.605+0000 7fb8c48c1640 1 --2- 172.21.15.32:0/1220106348 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8c015e9e0 0x7fb8c00725c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:31.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.605+0000 7fb8c48c1640 1 -- 172.21.15.32:0/1220106348 >> 172.21.15.32:0/1220106348 conn(0x7fb8c00b41f0 msgr2=0x7fb8c00b45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:31.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.605+0000 7fb8c48c1640 1 -- 172.21.15.32:0/1220106348 shutdown_connections 2024-04-24T22:59:31.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.605+0000 7fb8c48c1640 1 -- 172.21.15.32:0/1220106348 wait complete. 2024-04-24T22:59:31.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.606+0000 7fb8c48c1640 1 Processor -- start 2024-04-24T22:59:31.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.606+0000 7fb8c48c1640 1 -- start start 2024-04-24T22:59:31.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.607+0000 7fb8c48c1640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8c00d1d60 0x7fb8c01612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:31.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.607+0000 7fb8c48c1640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8c015e600 0x7fb8c00cf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:31.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.607+0000 7fb8c48c1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8c015e9e0 0x7fb8c00cfc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:31.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.607+0000 7fb8c48c1640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c0072c80 con 0x7fb8c015e9e0 2024-04-24T22:59:31.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.607+0000 7fb8c48c1640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c0072b00 con 0x7fb8c015e600 2024-04-24T22:59:31.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.607+0000 7fb8c48c1640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fb8c0072e00 con 0x7fb8c00d1d60 2024-04-24T22:59:31.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.607+0000 7fb8bed9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8c015e9e0 0x7fb8c00cfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:31.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.607+0000 7fb8bed9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8c015e9e0 0x7fb8c00cfc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52116/0 (socket says 172.21.15.32:52116) 2024-04-24T22:59:31.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.607+0000 7fb8bed9d640 1 -- 172.21.15.32:0/3038085338 learned_addr learned my addr 172.21.15.32:0/3038085338 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:31.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.607+0000 7fb8b7fff640 1 --2- 172.21.15.32:0/3038085338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8c015e600 0x7fb8c00cf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:31.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.607+0000 7fb8be59c640 1 --2- 172.21.15.32:0/3038085338 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8c00d1d60 0x7fb8c01612a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:31.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.608+0000 7fb8bed9d640 1 -- 172.21.15.32:0/3038085338 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8c00d1d60 msgr2=0x7fb8c01612a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:31.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.608+0000 7fb8bed9d640 1 --2- 172.21.15.32:0/3038085338 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8c00d1d60 0x7fb8c01612a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:31.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.608+0000 7fb8bed9d640 1 -- 172.21.15.32:0/3038085338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8c015e600 msgr2=0x7fb8c00cf6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:31.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.608+0000 7fb8bed9d640 1 --2- 172.21.15.32:0/3038085338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8c015e600 0x7fb8c00cf6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:31.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.608+0000 7fb8bed9d640 1 -- 172.21.15.32:0/3038085338 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8c01f3f00 con 0x7fb8c015e9e0 2024-04-24T22:59:31.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.608+0000 7fb8be59c640 1 --2- 172.21.15.32:0/3038085338 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8c00d1d60 0x7fb8c01612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:31.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.608+0000 7fb8bed9d640 1 --2- 172.21.15.32:0/3038085338 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8c015e9e0 0x7fb8c00cfc30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8009eb0 tx=0x7fb8b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:31.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.608+0000 7fb8b6ffd640 1 -- 172.21.15.32:0/3038085338 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b803f070 con 0x7fb8c015e9e0 2024-04-24T22:59:31.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.608+0000 7fb8b6ffd640 1 -- 172.21.15.32:0/3038085338 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8b8031b40 con 0x7fb8c015e9e0 2024-04-24T22:59:31.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.608+0000 7fb8c48c1640 1 -- 172.21.15.32:0/3038085338 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8c015a9c0 con 0x7fb8c015e9e0 2024-04-24T22:59:31.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.608+0000 7fb8b6ffd640 1 -- 172.21.15.32:0/3038085338 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b8007e90 con 0x7fb8c015e9e0 2024-04-24T22:59:31.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.608+0000 7fb8c48c1640 1 -- 172.21.15.32:0/3038085338 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8c0160aa0 con 0x7fb8c015e9e0 2024-04-24T22:59:31.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.609+0000 7fb8c48c1640 1 -- 172.21.15.32:0/3038085338 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8c01617e0 con 0x7fb8c015e9e0 2024-04-24T22:59:31.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.609+0000 7fb8b6ffd640 1 -- 172.21.15.32:0/3038085338 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb8b803b410 con 0x7fb8c015e9e0 2024-04-24T22:59:31.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.610+0000 7fb8b6ffd640 1 --2- 172.21.15.32:0/3038085338 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb8a006be30 0x7fb8a006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:31.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.610+0000 7fb8b6ffd640 1 -- 172.21.15.32:0/3038085338 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb8b80ac0a0 con 0x7fb8c015e9e0 2024-04-24T22:59:31.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.610+0000 7fb8be59c640 1 --2- 172.21.15.32:0/3038085338 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb8a006be30 0x7fb8a006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:31.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.611+0000 7fb8be59c640 1 --2- 172.21.15.32:0/3038085338 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb8a006be30 0x7fb8a006e2f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb8b0005880 tx=0x7fb8b0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:31.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.614+0000 7fb8b6ffd640 1 -- 172.21.15.32:0/3038085338 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb8b807aae0 con 0x7fb8c015e9e0 2024-04-24T22:59:31.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.796+0000 7fb8c48c1640 1 -- 172.21.15.32:0/3038085338 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb8c00b1b50 con 0x7fb8c015e9e0 2024-04-24T22:59:31.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.797+0000 7fb8b6ffd640 1 -- 172.21.15.32:0/3038085338 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7fb8b807a3e0 con 0x7fb8c015e9e0 2024-04-24T22:59:31.799 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-24T22:59:31.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.798+0000 7fb8c48c1640 1 -- 172.21.15.32:0/3038085338 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb8a006be30 msgr2=0x7fb8a006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:31.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.798+0000 7fb8c48c1640 1 --2- 172.21.15.32:0/3038085338 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb8a006be30 0x7fb8a006e2f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb8b0005880 tx=0x7fb8b0009040 comp rx=0 tx=0).stop 2024-04-24T22:59:31.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.798+0000 7fb8c48c1640 1 -- 172.21.15.32:0/3038085338 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8c015e9e0 msgr2=0x7fb8c00cfc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:31.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.798+0000 7fb8c48c1640 1 --2- 172.21.15.32:0/3038085338 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8c015e9e0 0x7fb8c00cfc30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8009eb0 tx=0x7fb8b8004a40 comp rx=0 tx=0).stop 2024-04-24T22:59:31.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.798+0000 7fb8c48c1640 1 -- 172.21.15.32:0/3038085338 shutdown_connections 2024-04-24T22:59:31.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.798+0000 7fb8c48c1640 1 --2- 172.21.15.32:0/3038085338 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb8a006be30 0x7fb8a006e2f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:31.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.798+0000 7fb8c48c1640 1 --2- 172.21.15.32:0/3038085338 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8c015e9e0 0x7fb8c00cfc30 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:31.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.798+0000 7fb8c48c1640 1 --2- 172.21.15.32:0/3038085338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8c015e600 0x7fb8c00cf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:31.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.798+0000 7fb8c48c1640 1 --2- 172.21.15.32:0/3038085338 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8c00d1d60 0x7fb8c01612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:31.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.798+0000 7fb8c48c1640 1 -- 172.21.15.32:0/3038085338 >> 172.21.15.32:0/3038085338 conn(0x7fb8c00b41f0 msgr2=0x7fb8c00b7200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:31.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.798+0000 7fb8c48c1640 1 -- 172.21.15.32:0/3038085338 shutdown_connections 2024-04-24T22:59:31.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.799+0000 7fb8c48c1640 1 -- 172.21.15.32:0/3038085338 wait complete. 2024-04-24T22:59:31.812 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:59:31.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.909+0000 7fe08391b640 1 Processor -- start 2024-04-24T22:59:31.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.910+0000 7fe08391b640 1 -- start start 2024-04-24T22:59:31.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.910+0000 7fe08391b640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe07c06b820 0x7fe07c0d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:31.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.910+0000 7fe08391b640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe07c0d4210 0x7fe07c0b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:31.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.910+0000 7fe08391b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe07c0d3170 con 0x7fe07c15e9b0 2024-04-24T22:59:31.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.910+0000 7fe08391b640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fe07c0d3450 con 0x7fe07c06b820 2024-04-24T22:59:31.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.910+0000 7fe08391b640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe07c0d3730 con 0x7fe07c0d4210 2024-04-24T22:59:31.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.911+0000 7fe0816b7640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe07c15e9b0 0x7fe07c06b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34058/0 (socket says 172.21.15.32:34058) 2024-04-24T22:59:31.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.911+0000 7fe0816b7640 1 -- 172.21.15.32:0/1543945394 learned_addr learned my addr 172.21.15.32:0/1543945394 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:31.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.911+0000 7fe080eb6640 1 --2- 172.21.15.32:0/1543945394 >> v2:172.21.15.121:3300/0 conn(0x7fe07c06b820 0x7fe07c0d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:31.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.911+0000 7fe081eb8640 1 --2- 172.21.15.32:0/1543945394 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe07c0d4210 0x7fe07c0b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:31.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.911+0000 7fe073fff640 1 -- 172.21.15.32:0/1543945394 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2389795458 0 0) 0x7fe07c0d3170 con 0x7fe07c15e9b0 2024-04-24T22:59:31.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.911+0000 7fe073fff640 1 -- 172.21.15.32:0/1543945394 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe068003520 con 0x7fe07c15e9b0 2024-04-24T22:59:31.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.911+0000 7fe073fff640 1 -- 172.21.15.32:0/1543945394 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 281024953 0 0) 0x7fe068003520 con 0x7fe07c15e9b0 2024-04-24T22:59:31.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.911+0000 7fe073fff640 1 -- 172.21.15.32:0/1543945394 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe07c0d4210 msgr2=0x7fe07c0b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:31.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.911+0000 7fe073fff640 1 --2- 172.21.15.32:0/1543945394 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe07c0d4210 0x7fe07c0b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:31.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.911+0000 7fe073fff640 1 -- 172.21.15.32:0/1543945394 >> v2:172.21.15.121:3300/0 conn(0x7fe07c06b820 msgr2=0x7fe07c0d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:31.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.911+0000 7fe073fff640 1 --2- 172.21.15.32:0/1543945394 >> v2:172.21.15.121:3300/0 conn(0x7fe07c06b820 0x7fe07c0d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:31.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.911+0000 7fe073fff640 1 -- 172.21.15.32:0/1543945394 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe07c0d3a10 con 0x7fe07c15e9b0 2024-04-24T22:59:31.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.911+0000 7fe073fff640 1 -- 172.21.15.32:0/1543945394 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7fe078001670 con 0x7fe07c15e9b0 2024-04-24T22:59:31.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.911+0000 7fe073fff640 1 -- 172.21.15.32:0/1543945394 >> v1:172.21.15.32:6789/0 conn(0x7fe07c15e9b0 legacy=0x7fe07c06b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:31.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.911+0000 7fe073fff640 1 --2- 172.21.15.32:0/1543945394 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0680039b0 0x7fe068003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:31.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.911+0000 7fe073fff640 1 --2- 172.21.15.32:0/1543945394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe068004800 0x7fe068004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:31.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.911+0000 7fe073fff640 1 --2- 172.21.15.32:0/1543945394 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe07c0d4210 0x7fe0680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:31.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.911+0000 7fe073fff640 1 -- 172.21.15.32:0/1543945394 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe07c0d3450 con 0x7fe068004800 2024-04-24T22:59:31.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.911+0000 7fe073fff640 1 -- 172.21.15.32:0/1543945394 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe07c0d3730 con 0x7fe0680039b0 2024-04-24T22:59:31.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.911+0000 7fe073fff640 1 -- 172.21.15.32:0/1543945394 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe06800b9c0 con 0x7fe07c0d4210 2024-04-24T22:59:31.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.911+0000 7fe080eb6640 1 --2- 172.21.15.32:0/1543945394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe068004800 0x7fe068004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:31.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.911+0000 7fe0816b7640 1 --2- 172.21.15.32:0/1543945394 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0680039b0 0x7fe068003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:31.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.912+0000 7fe081eb8640 1 --2- 172.21.15.32:0/1543945394 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe07c0d4210 0x7fe0680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:31.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.912+0000 7fe080eb6640 1 -- 172.21.15.32:0/1543945394 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe07c0d4210 msgr2=0x7fe0680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:31.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.912+0000 7fe080eb6640 1 --2- 172.21.15.32:0/1543945394 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe07c0d4210 0x7fe0680051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:31.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.912+0000 7fe080eb6640 1 -- 172.21.15.32:0/1543945394 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0680039b0 msgr2=0x7fe068003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:31.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.912+0000 7fe080eb6640 1 --2- 172.21.15.32:0/1543945394 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0680039b0 0x7fe068003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:31.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.912+0000 7fe080eb6640 1 -- 172.21.15.32:0/1543945394 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe068003520 con 0x7fe068004800 2024-04-24T22:59:31.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.912+0000 7fe080eb6640 1 --2- 172.21.15.32:0/1543945394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe068004800 0x7fe068004c80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe06c009ac0 tx=0x7fe068005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fa0c097a7063881 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:31.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.913+0000 7fe073fff640 1 -- 172.21.15.32:0/1543945394 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06c036070 con 0x7fe068004800 2024-04-24T22:59:31.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.913+0000 7fe073fff640 1 -- 172.21.15.32:0/1543945394 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe06c031400 con 0x7fe068004800 2024-04-24T22:59:31.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.913+0000 7fe073fff640 1 -- 172.21.15.32:0/1543945394 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06c009690 con 0x7fe068004800 2024-04-24T22:59:31.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.913+0000 7fe08391b640 1 -- 172.21.15.32:0/1543945394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe068004800 msgr2=0x7fe068004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:31.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.913+0000 7fe08391b640 1 --2- 172.21.15.32:0/1543945394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe068004800 0x7fe068004c80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe06c009ac0 tx=0x7fe068005830 comp rx=0 tx=0).stop 2024-04-24T22:59:31.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.913+0000 7fe081eb8640 1 -- 172.21.15.32:0/1543945394 reap_dead start 2024-04-24T22:59:31.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.913+0000 7fe08391b640 1 -- 172.21.15.32:0/1543945394 shutdown_connections 2024-04-24T22:59:31.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.913+0000 7fe08391b640 1 -- 172.21.15.32:0/1543945394 >> 172.21.15.32:0/1543945394 conn(0x7fe07c0d11d0 msgr2=0x7fe07c0d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:31.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.913+0000 7fe08391b640 1 -- 172.21.15.32:0/1543945394 shutdown_connections 2024-04-24T22:59:31.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.913+0000 7fe08391b640 1 -- 172.21.15.32:0/1543945394 wait complete. 2024-04-24T22:59:31.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.914+0000 7fe08391b640 1 Processor -- start 2024-04-24T22:59:31.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.914+0000 7fe08391b640 1 -- start start 2024-04-24T22:59:31.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.915+0000 7fe08391b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe068004800 0x7fe07c1f3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:31.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.915+0000 7fe08391b640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe07c160180 0x7fe07c160600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:31.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.915+0000 7fe08391b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe07c1489c0 0x7fe07c148e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:31.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.915+0000 7fe08391b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe07c0ba0e0 con 0x7fe07c1489c0 2024-04-24T22:59:31.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.915+0000 7fe08391b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe07c0d2400 con 0x7fe068004800 2024-04-24T22:59:31.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.915+0000 7fe08391b640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe07c0ba260 con 0x7fe07c160180 2024-04-24T22:59:31.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.915+0000 7fe081eb8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe07c1489c0 0x7fe07c148e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:31.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.915+0000 7fe081eb8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe07c1489c0 0x7fe07c148e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52128/0 (socket says 172.21.15.32:52128) 2024-04-24T22:59:31.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.915+0000 7fe081eb8640 1 -- 172.21.15.32:0/706209230 learned_addr learned my addr 172.21.15.32:0/706209230 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:31.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.915+0000 7fe0816b7640 1 --2- 172.21.15.32:0/706209230 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe068004800 0x7fe07c1f3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:31.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.915+0000 7fe080eb6640 1 --2- 172.21.15.32:0/706209230 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe07c160180 0x7fe07c160600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:31.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.916+0000 7fe081eb8640 1 -- 172.21.15.32:0/706209230 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe07c160180 msgr2=0x7fe07c160600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:31.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.916+0000 7fe081eb8640 1 --2- 172.21.15.32:0/706209230 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe07c160180 0x7fe07c160600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:31.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.916+0000 7fe081eb8640 1 -- 172.21.15.32:0/706209230 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe068004800 msgr2=0x7fe07c1f3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:31.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.916+0000 7fe081eb8640 1 --2- 172.21.15.32:0/706209230 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe068004800 0x7fe07c1f3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:31.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.916+0000 7fe081eb8640 1 -- 172.21.15.32:0/706209230 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe07c160b40 con 0x7fe07c1489c0 2024-04-24T22:59:31.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.916+0000 7fe081eb8640 1 --2- 172.21.15.32:0/706209230 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe07c1489c0 0x7fe07c148e40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe07c06bdd0 tx=0x7fe07400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:31.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.916+0000 7fe071ffb640 1 -- 172.21.15.32:0/706209230 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe074011e60 con 0x7fe07c1489c0 2024-04-24T22:59:31.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.916+0000 7fe071ffb640 1 -- 172.21.15.32:0/706209230 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe07401c070 con 0x7fe07c1489c0 2024-04-24T22:59:31.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.916+0000 7fe08391b640 1 -- 172.21.15.32:0/706209230 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe07c160e00 con 0x7fe07c1489c0 2024-04-24T22:59:31.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.916+0000 7fe071ffb640 1 -- 172.21.15.32:0/706209230 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe074003bd0 con 0x7fe07c1489c0 2024-04-24T22:59:31.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.916+0000 7fe08391b640 1 -- 172.21.15.32:0/706209230 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe07c1613c0 con 0x7fe07c1489c0 2024-04-24T22:59:31.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.917+0000 7fe08391b640 1 -- 172.21.15.32:0/706209230 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe07c068430 con 0x7fe07c1489c0 2024-04-24T22:59:31.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.918+0000 7fe071ffb640 1 -- 172.21.15.32:0/706209230 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe074004030 con 0x7fe07c1489c0 2024-04-24T22:59:31.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.918+0000 7fe071ffb640 1 --2- 172.21.15.32:0/706209230 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe06006be60 0x7fe06006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:31.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.918+0000 7fe071ffb640 1 -- 172.21.15.32:0/706209230 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe07408b5d0 con 0x7fe07c1489c0 2024-04-24T22:59:31.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.918+0000 7fe0816b7640 1 --2- 172.21.15.32:0/706209230 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe06006be60 0x7fe06006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:31.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.919+0000 7fe0816b7640 1 --2- 172.21.15.32:0/706209230 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe06006be60 0x7fe06006e320 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe0780015d0 tx=0x7fe0780078f0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:31.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:31.922+0000 7fe071ffb640 1 -- 172.21.15.32:0/706209230 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe074010040 con 0x7fe07c1489c0 2024-04-24T22:59:32.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.108+0000 7fe08391b640 1 -- 172.21.15.32:0/706209230 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe07c15f040 con 0x7fe07c1489c0 2024-04-24T22:59:32.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.109+0000 7fe071ffb640 1 -- 172.21.15.32:0/706209230 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16621 (secure 0 0 0) 0x7fe074059f80 con 0x7fe07c1489c0 2024-04-24T22:59:32.112 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:59:32.112 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T22:59:05.517554+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:59:02.016639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T22:59:02.846087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:59:32.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.111+0000 7fe08391b640 1 -- 172.21.15.32:0/706209230 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe06006be60 msgr2=0x7fe06006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:32.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.111+0000 7fe08391b640 1 --2- 172.21.15.32:0/706209230 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe06006be60 0x7fe06006e320 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe0780015d0 tx=0x7fe0780078f0 comp rx=0 tx=0).stop 2024-04-24T22:59:32.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.111+0000 7fe08391b640 1 -- 172.21.15.32:0/706209230 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe07c1489c0 msgr2=0x7fe07c148e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:32.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.111+0000 7fe08391b640 1 --2- 172.21.15.32:0/706209230 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe07c1489c0 0x7fe07c148e40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe07c06bdd0 tx=0x7fe07400f580 comp rx=0 tx=0).stop 2024-04-24T22:59:32.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.111+0000 7fe08391b640 1 -- 172.21.15.32:0/706209230 shutdown_connections 2024-04-24T22:59:32.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.111+0000 7fe08391b640 1 --2- 172.21.15.32:0/706209230 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe06006be60 0x7fe06006e320 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:32.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.111+0000 7fe08391b640 1 --2- 172.21.15.32:0/706209230 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe07c1489c0 0x7fe07c148e40 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:32.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.111+0000 7fe08391b640 1 --2- 172.21.15.32:0/706209230 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe07c160180 0x7fe07c160600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:32.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.111+0000 7fe08391b640 1 --2- 172.21.15.32:0/706209230 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe068004800 0x7fe07c1f3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:32.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.111+0000 7fe08391b640 1 -- 172.21.15.32:0/706209230 >> 172.21.15.32:0/706209230 conn(0x7fe07c0d11d0 msgr2=0x7fe07c15c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:32.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.111+0000 7fe08391b640 1 -- 172.21.15.32:0/706209230 shutdown_connections 2024-04-24T22:59:32.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.111+0000 7fe08391b640 1 -- 172.21.15.32:0/706209230 wait complete. 2024-04-24T22:59:32.125 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T22:59:02.016639+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T22:59:02.846087+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T22:59:32.126 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T22:59:32.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.228+0000 7f065df63640 1 Processor -- start 2024-04-24T22:59:32.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.228+0000 7f065df63640 1 -- start start 2024-04-24T22:59:32.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.228+0000 7f065df63640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0658074210 0x7f065806c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:32.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.228+0000 7f065df63640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f06580b0b80 0x7f06580745f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:32.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.228+0000 7f065df63640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06580b0770 con 0x7f06580ae140 2024-04-24T22:59:32.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.228+0000 7f065df63640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f06580b1c20 con 0x7f0658074210 2024-04-24T22:59:32.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.228+0000 7f065df63640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f0658074b30 con 0x7f06580b0b80 2024-04-24T22:59:32.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.229+0000 7f06577fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0658074210 0x7f065806c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:32.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.229+0000 7f0657fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f06580ae140 0x7f06580b20b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34060/0 (socket says 172.21.15.32:34060) 2024-04-24T22:59:32.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.229+0000 7f0657fff640 1 -- 172.21.15.32:0/2883019144 learned_addr learned my addr 172.21.15.32:0/2883019144 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:32.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.229+0000 7f0656ffd640 1 --2- 172.21.15.32:0/2883019144 >> v2:172.21.15.121:3300/0 conn(0x7f06580b0b80 0x7f06580745f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:32.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.229+0000 7f06577fe640 1 -- 172.21.15.32:0/2883019144 >> v2:172.21.15.121:3300/0 conn(0x7f06580b0b80 msgr2=0x7f06580745f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:32.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.229+0000 7f06577fe640 1 --2- 172.21.15.32:0/2883019144 >> v2:172.21.15.121:3300/0 conn(0x7f06580b0b80 0x7f06580745f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:32.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.229+0000 7f06577fe640 1 -- 172.21.15.32:0/2883019144 >> v1:172.21.15.32:6789/0 conn(0x7f06580ae140 legacy=0x7f06580b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:32.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.229+0000 7f06577fe640 1 -- 172.21.15.32:0/2883019144 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06580b1010 con 0x7f0658074210 2024-04-24T22:59:32.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.229+0000 7f06567fc640 1 -- 172.21.15.32:0/2883019144 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2628382486 0 0) 0x7f06580b0770 con 0x7f06580ae140 2024-04-24T22:59:32.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.229+0000 7f0656ffd640 1 --2- 172.21.15.32:0/2883019144 >> v2:172.21.15.121:3300/0 conn(0x7f06580b0b80 0x7f06580745f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:59:32.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.230+0000 7f06577fe640 1 --2- 172.21.15.32:0/2883019144 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0658074210 0x7f065806c480 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f06580722d0 tx=0x7f064802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e0741ede4d84670 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:32.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.230+0000 7f06567fc640 1 -- 172.21.15.32:0/2883019144 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0648037070 con 0x7f0658074210 2024-04-24T22:59:32.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.230+0000 7f06567fc640 1 -- 172.21.15.32:0/2883019144 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0648032440 con 0x7f0658074210 2024-04-24T22:59:32.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.230+0000 7f06567fc640 1 -- 172.21.15.32:0/2883019144 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0648032810 con 0x7f0658074210 2024-04-24T22:59:32.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.230+0000 7f065df63640 1 -- 172.21.15.32:0/2883019144 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0658074210 msgr2=0x7f065806c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:32.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.230+0000 7f065df63640 1 --2- 172.21.15.32:0/2883019144 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0658074210 0x7f065806c480 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f06580722d0 tx=0x7f064802a1a0 comp rx=0 tx=0).stop 2024-04-24T22:59:32.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.230+0000 7f065df63640 1 -- 172.21.15.32:0/2883019144 shutdown_connections 2024-04-24T22:59:32.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.230+0000 7f065df63640 1 --2- 172.21.15.32:0/2883019144 >> v2:172.21.15.121:3300/0 conn(0x7f06580b0b80 0x7f06580745f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:32.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.230+0000 7f065df63640 1 --2- 172.21.15.32:0/2883019144 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0658074210 0x7f065806c480 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:32.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.230+0000 7f065df63640 1 -- 172.21.15.32:0/2883019144 >> 172.21.15.32:0/2883019144 conn(0x7f06580b2e60 msgr2=0x7f06580b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:32.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.230+0000 7f065df63640 1 -- 172.21.15.32:0/2883019144 shutdown_connections 2024-04-24T22:59:32.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.230+0000 7f065df63640 1 -- 172.21.15.32:0/2883019144 wait complete. 2024-04-24T22:59:32.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.231+0000 7f065df63640 1 Processor -- start 2024-04-24T22:59:32.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.232+0000 7f065df63640 1 -- start start 2024-04-24T22:59:32.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.232+0000 7f065df63640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0658074210 0x7f065806ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:32.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.232+0000 7f065df63640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06580ae140 0x7f06581f5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:32.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.232+0000 7f065df63640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06581f5c70 0x7f06580c7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:32.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.232+0000 7f065df63640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f06580b18b0 con 0x7f06581f5c70 2024-04-24T22:59:32.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.232+0000 7f065df63640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f06580b15d0 con 0x7f06580ae140 2024-04-24T22:59:32.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.232+0000 7f065df63640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f06580b3ba0 con 0x7f0658074210 2024-04-24T22:59:32.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.233+0000 7f0657fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06581f5c70 0x7f06580c7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:32.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.233+0000 7f0657fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06581f5c70 0x7f06580c7360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52148/0 (socket says 172.21.15.32:52148) 2024-04-24T22:59:32.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.233+0000 7f0657fff640 1 -- 172.21.15.32:0/1948479867 learned_addr learned my addr 172.21.15.32:0/1948479867 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:32.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.233+0000 7f0656ffd640 1 --2- 172.21.15.32:0/1948479867 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06580ae140 0x7f06581f5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:32.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.233+0000 7f06577fe640 1 --2- 172.21.15.32:0/1948479867 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0658074210 0x7f065806ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:32.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.233+0000 7f0657fff640 1 -- 172.21.15.32:0/1948479867 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0658074210 msgr2=0x7f065806ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:32.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.233+0000 7f0657fff640 1 --2- 172.21.15.32:0/1948479867 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0658074210 0x7f065806ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:32.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.233+0000 7f0657fff640 1 -- 172.21.15.32:0/1948479867 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06580ae140 msgr2=0x7f06581f5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:32.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.233+0000 7f0657fff640 1 --2- 172.21.15.32:0/1948479867 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06580ae140 0x7f06581f5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:32.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.233+0000 7f0657fff640 1 -- 172.21.15.32:0/1948479867 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06580c65a0 con 0x7f06581f5c70 2024-04-24T22:59:32.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.233+0000 7f0656ffd640 1 --2- 172.21.15.32:0/1948479867 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06580ae140 0x7f06581f5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:32.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.233+0000 7f06577fe640 1 --2- 172.21.15.32:0/1948479867 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0658074210 0x7f065806ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:32.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.233+0000 7f0657fff640 1 --2- 172.21.15.32:0/1948479867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06581f5c70 0x7f06580c7360 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f065000c7d0 tx=0x7f065000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:32.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.233+0000 7f063ffff640 1 -- 172.21.15.32:0/1948479867 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0650018070 con 0x7f06581f5c70 2024-04-24T22:59:32.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.233+0000 7f063ffff640 1 -- 172.21.15.32:0/1948479867 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f065000a640 con 0x7f06581f5c70 2024-04-24T22:59:32.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.233+0000 7f065df63640 1 -- 172.21.15.32:0/1948479867 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06580c6880 con 0x7f06581f5c70 2024-04-24T22:59:32.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.233+0000 7f063ffff640 1 -- 172.21.15.32:0/1948479867 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065000aa70 con 0x7f06581f5c70 2024-04-24T22:59:32.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.233+0000 7f065df63640 1 -- 172.21.15.32:0/1948479867 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06580c6e40 con 0x7f06581f5c70 2024-04-24T22:59:32.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.234+0000 7f065df63640 1 -- 172.21.15.32:0/1948479867 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06580c7040 con 0x7f06581f5c70 2024-04-24T22:59:32.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.235+0000 7f063ffff640 1 -- 172.21.15.32:0/1948479867 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f065001b070 con 0x7f06581f5c70 2024-04-24T22:59:32.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.235+0000 7f063ffff640 1 --2- 172.21.15.32:0/1948479867 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f064406be30 0x7f064406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:32.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.235+0000 7f063ffff640 1 -- 172.21.15.32:0/1948479867 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f065008a490 con 0x7f06581f5c70 2024-04-24T22:59:32.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.236+0000 7f06577fe640 1 --2- 172.21.15.32:0/1948479867 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f064406be30 0x7f064406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:32.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.236+0000 7f06577fe640 1 --2- 172.21.15.32:0/1948479867 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f064406be30 0x7f064406e2f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0648034000 tx=0x7f064802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:32.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.239+0000 7f063ffff640 1 -- 172.21.15.32:0/1948479867 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f06580c7040 con 0x7f06581f5c70 2024-04-24T22:59:32.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.422+0000 7f065df63640 1 -- 172.21.15.32:0/1948479867 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f065806b220 con 0x7f06581f5c70 2024-04-24T22:59:32.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.422+0000 7f063ffff640 1 -- 172.21.15.32:0/1948479867 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f0650058e40 con 0x7f06581f5c70 2024-04-24T22:59:32.425 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T22:59:32.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.424+0000 7f065df63640 1 -- 172.21.15.32:0/1948479867 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f064406be30 msgr2=0x7f064406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:32.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.424+0000 7f065df63640 1 --2- 172.21.15.32:0/1948479867 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f064406be30 0x7f064406e2f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0648034000 tx=0x7f064802e000 comp rx=0 tx=0).stop 2024-04-24T22:59:32.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.424+0000 7f065df63640 1 -- 172.21.15.32:0/1948479867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06581f5c70 msgr2=0x7f06580c7360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:32.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.424+0000 7f065df63640 1 --2- 172.21.15.32:0/1948479867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06581f5c70 0x7f06580c7360 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f065000c7d0 tx=0x7f065000cc90 comp rx=0 tx=0).stop 2024-04-24T22:59:32.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.424+0000 7f065df63640 1 -- 172.21.15.32:0/1948479867 shutdown_connections 2024-04-24T22:59:32.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.424+0000 7f065df63640 1 --2- 172.21.15.32:0/1948479867 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f064406be30 0x7f064406e2f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:32.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.424+0000 7f065df63640 1 --2- 172.21.15.32:0/1948479867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06581f5c70 0x7f06580c7360 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:32.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.424+0000 7f065df63640 1 --2- 172.21.15.32:0/1948479867 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06580ae140 0x7f06581f5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:32.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.424+0000 7f065df63640 1 --2- 172.21.15.32:0/1948479867 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0658074210 0x7f065806ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:32.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.424+0000 7f065df63640 1 -- 172.21.15.32:0/1948479867 >> 172.21.15.32:0/1948479867 conn(0x7f06580b2e60 msgr2=0x7f06580af670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:32.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.424+0000 7f065df63640 1 -- 172.21.15.32:0/1948479867 shutdown_connections 2024-04-24T22:59:32.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.424+0000 7f065df63640 1 -- 172.21.15.32:0/1948479867 wait complete. 2024-04-24T22:59:32.438 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T22:59:32.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.539+0000 7f026f7db640 1 Processor -- start 2024-04-24T22:59:32.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.539+0000 7f026f7db640 1 -- start start 2024-04-24T22:59:32.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.540+0000 7f026f7db640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0268162320 0x7f02680ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:32.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.540+0000 7f026f7db640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f02680b5af0 0x7f0268162700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:32.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.540+0000 7f026f7db640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02680cd270 con 0x7f0268162c40 2024-04-24T22:59:32.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.540+0000 7f026f7db640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f02680b5660 con 0x7f0268162320 2024-04-24T22:59:32.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.540+0000 7f026f7db640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f0268071610 con 0x7f02680b5af0 2024-04-24T22:59:32.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.540+0000 7f026d577640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0268162320 0x7f02680ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:32.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.540+0000 7f026d577640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0268162320 0x7f02680ccc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52156/0 (socket says 172.21.15.32:52156) 2024-04-24T22:59:32.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.540+0000 7f026d577640 1 -- 172.21.15.32:0/1472865685 learned_addr learned my addr 172.21.15.32:0/1472865685 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:32.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.540+0000 7f026cd76640 1 --2- 172.21.15.32:0/1472865685 >> v2:172.21.15.121:3300/0 conn(0x7f02680b5af0 0x7f0268162700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:32.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.540+0000 7f026d577640 1 -- 172.21.15.32:0/1472865685 >> v2:172.21.15.121:3300/0 conn(0x7f02680b5af0 msgr2=0x7f0268162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:32.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.540+0000 7f026d577640 1 --2- 172.21.15.32:0/1472865685 >> v2:172.21.15.121:3300/0 conn(0x7f02680b5af0 0x7f0268162700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:32.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.540+0000 7f026d577640 1 -- 172.21.15.32:0/1472865685 >> v1:172.21.15.121:6789/0 conn(0x7f0268162c40 legacy=0x7f02680ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:32.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.540+0000 7f026d577640 1 -- 172.21.15.32:0/1472865685 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02680718f0 con 0x7f0268162320 2024-04-24T22:59:32.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.541+0000 7f026d577640 1 --2- 172.21.15.32:0/1472865685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0268162320 0x7f02680ccc90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f02680d1bc0 tx=0x7f026402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2de740561ac54d3e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:32.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.541+0000 7f025ffff640 1 -- 172.21.15.32:0/1472865685 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0264037070 con 0x7f0268162320 2024-04-24T22:59:32.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.541+0000 7f025ffff640 1 -- 172.21.15.32:0/1472865685 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0264032440 con 0x7f0268162320 2024-04-24T22:59:32.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.541+0000 7f025ffff640 1 -- 172.21.15.32:0/1472865685 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0264032810 con 0x7f0268162320 2024-04-24T22:59:32.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.542+0000 7f026f7db640 1 -- 172.21.15.32:0/1472865685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0268162320 msgr2=0x7f02680ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:32.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.542+0000 7f026f7db640 1 --2- 172.21.15.32:0/1472865685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0268162320 0x7f02680ccc90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f02680d1bc0 tx=0x7f026402a1e0 comp rx=0 tx=0).stop 2024-04-24T22:59:32.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.542+0000 7f026f7db640 1 -- 172.21.15.32:0/1472865685 shutdown_connections 2024-04-24T22:59:32.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.542+0000 7f026f7db640 1 --2- 172.21.15.32:0/1472865685 >> v2:172.21.15.121:3300/0 conn(0x7f02680b5af0 0x7f0268162700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:32.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.542+0000 7f026f7db640 1 --2- 172.21.15.32:0/1472865685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0268162320 0x7f02680ccc90 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:32.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.542+0000 7f026f7db640 1 -- 172.21.15.32:0/1472865685 >> 172.21.15.32:0/1472865685 conn(0x7f02680d0ae0 msgr2=0x7f02680d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:32.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.542+0000 7f026f7db640 1 -- 172.21.15.32:0/1472865685 shutdown_connections 2024-04-24T22:59:32.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.542+0000 7f026f7db640 1 -- 172.21.15.32:0/1472865685 wait complete. 2024-04-24T22:59:32.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.543+0000 7f026f7db640 1 Processor -- start 2024-04-24T22:59:32.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.543+0000 7f026f7db640 1 -- start start 2024-04-24T22:59:32.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.544+0000 7f026f7db640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02680b5af0 0x7f02681586f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:32.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.544+0000 7f026f7db640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0268162320 0x7f02681f4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:32.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.544+0000 7f026f7db640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0268162c40 0x7f0268161380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:32.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.544+0000 7f026f7db640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0268163070 con 0x7f0268162c40 2024-04-24T22:59:32.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.544+0000 7f026f7db640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0268067950 con 0x7f02680b5af0 2024-04-24T22:59:32.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.544+0000 7f026f7db640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f0268071bd0 con 0x7f0268162320 2024-04-24T22:59:32.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.544+0000 7f026dd78640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0268162c40 0x7f0268161380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:32.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.544+0000 7f026dd78640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0268162c40 0x7f0268161380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52166/0 (socket says 172.21.15.32:52166) 2024-04-24T22:59:32.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.544+0000 7f026dd78640 1 -- 172.21.15.32:0/4050065713 learned_addr learned my addr 172.21.15.32:0/4050065713 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:32.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.544+0000 7f026d577640 1 --2- 172.21.15.32:0/4050065713 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02680b5af0 0x7f02681586f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:32.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.544+0000 7f026cd76640 1 --2- 172.21.15.32:0/4050065713 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0268162320 0x7f02681f4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:32.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.544+0000 7f026dd78640 1 -- 172.21.15.32:0/4050065713 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0268162320 msgr2=0x7f02681f4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:32.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.544+0000 7f026dd78640 1 --2- 172.21.15.32:0/4050065713 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0268162320 0x7f02681f4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:32.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.544+0000 7f026dd78640 1 -- 172.21.15.32:0/4050065713 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02680b5af0 msgr2=0x7f02681586f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:32.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.544+0000 7f026dd78640 1 --2- 172.21.15.32:0/4050065713 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02680b5af0 0x7f02681586f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:32.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.544+0000 7f026dd78640 1 -- 172.21.15.32:0/4050065713 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02681f53c0 con 0x7f0268162c40 2024-04-24T22:59:32.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.544+0000 7f026dd78640 1 --2- 172.21.15.32:0/4050065713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0268162c40 0x7f0268161380 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f026000c730 tx=0x7f026000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:32.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.544+0000 7f025dffb640 1 -- 172.21.15.32:0/4050065713 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0260003b90 con 0x7f0268162c40 2024-04-24T22:59:32.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.545+0000 7f025dffb640 1 -- 172.21.15.32:0/4050065713 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0260003d30 con 0x7f0268162c40 2024-04-24T22:59:32.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.545+0000 7f026f7db640 1 -- 172.21.15.32:0/4050065713 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02681f56a0 con 0x7f0268162c40 2024-04-24T22:59:32.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.545+0000 7f025dffb640 1 -- 172.21.15.32:0/4050065713 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0260005310 con 0x7f0268162c40 2024-04-24T22:59:32.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.545+0000 7f026f7db640 1 -- 172.21.15.32:0/4050065713 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02681f6030 con 0x7f0268162c40 2024-04-24T22:59:32.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.546+0000 7f026f7db640 1 -- 172.21.15.32:0/4050065713 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02681f6230 con 0x7f0268162c40 2024-04-24T22:59:32.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.546+0000 7f025dffb640 1 -- 172.21.15.32:0/4050065713 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f02600056c0 con 0x7f0268162c40 2024-04-24T22:59:32.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.546+0000 7f025dffb640 1 --2- 172.21.15.32:0/4050065713 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f024c06be30 0x7f024c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:32.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.547+0000 7f025dffb640 1 -- 172.21.15.32:0/4050065713 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f026008ad60 con 0x7f0268162c40 2024-04-24T22:59:32.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.547+0000 7f026d577640 1 --2- 172.21.15.32:0/4050065713 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f024c06be30 0x7f024c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:32.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.547+0000 7f026d577640 1 --2- 172.21.15.32:0/4050065713 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f024c06be30 0x7f024c06e2f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0264034000 tx=0x7f026402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:32.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.551+0000 7f025dffb640 1 -- 172.21.15.32:0/4050065713 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f02681f6230 con 0x7f0268162c40 2024-04-24T22:59:32.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.733+0000 7f026f7db640 1 -- 172.21.15.32:0/4050065713 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0268068430 con 0x7f0268162c40 2024-04-24T22:59:32.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.733+0000 7f025dffb640 1 -- 172.21.15.32:0/4050065713 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f02600590c0 con 0x7f0268162c40 2024-04-24T22:59:32.736 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-24T22:59:32.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.735+0000 7f026f7db640 1 -- 172.21.15.32:0/4050065713 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f024c06be30 msgr2=0x7f024c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:32.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.735+0000 7f026f7db640 1 --2- 172.21.15.32:0/4050065713 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f024c06be30 0x7f024c06e2f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0264034000 tx=0x7f026402e000 comp rx=0 tx=0).stop 2024-04-24T22:59:32.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.735+0000 7f026f7db640 1 -- 172.21.15.32:0/4050065713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0268162c40 msgr2=0x7f0268161380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:32.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.735+0000 7f026f7db640 1 --2- 172.21.15.32:0/4050065713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0268162c40 0x7f0268161380 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f026000c730 tx=0x7f026000cbf0 comp rx=0 tx=0).stop 2024-04-24T22:59:32.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.735+0000 7f026f7db640 1 -- 172.21.15.32:0/4050065713 shutdown_connections 2024-04-24T22:59:32.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.735+0000 7f026f7db640 1 --2- 172.21.15.32:0/4050065713 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f024c06be30 0x7f024c06e2f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:32.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.735+0000 7f026f7db640 1 --2- 172.21.15.32:0/4050065713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0268162c40 0x7f0268161380 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:32.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.735+0000 7f026f7db640 1 --2- 172.21.15.32:0/4050065713 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0268162320 0x7f02681f4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:32.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.735+0000 7f026f7db640 1 --2- 172.21.15.32:0/4050065713 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02680b5af0 0x7f02681586f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:32.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.735+0000 7f026f7db640 1 -- 172.21.15.32:0/4050065713 >> 172.21.15.32:0/4050065713 conn(0x7f02680d0ae0 msgr2=0x7f026815c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:32.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.735+0000 7f026f7db640 1 -- 172.21.15.32:0/4050065713 shutdown_connections 2024-04-24T22:59:32.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.735+0000 7f026f7db640 1 -- 172.21.15.32:0/4050065713 wait complete. 2024-04-24T22:59:32.749 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T22:59:32.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.850+0000 7fe12fd34640 1 Processor -- start 2024-04-24T22:59:32.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.850+0000 7fe12fd34640 1 -- start start 2024-04-24T22:59:32.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.851+0000 7fe12fd34640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe12815e9b0 0x7fe1280d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:32.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.851+0000 7fe12fd34640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe12806b820 0x7fe1280b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:32.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.851+0000 7fe12fd34640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1280d3170 con 0x7fe1280d4210 2024-04-24T22:59:32.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.851+0000 7fe12fd34640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fe1280d3450 con 0x7fe12815e9b0 2024-04-24T22:59:32.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.851+0000 7fe12fd34640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe1280d3730 con 0x7fe12806b820 2024-04-24T22:59:32.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.851+0000 7fe12d2cf640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe1280d4210 0x7fe12806b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34072/0 (socket says 172.21.15.32:34072) 2024-04-24T22:59:32.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.851+0000 7fe12d2cf640 1 -- 172.21.15.32:0/2352323576 learned_addr learned my addr 172.21.15.32:0/2352323576 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:32.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.852+0000 7fe12e2d1640 1 --2- 172.21.15.32:0/2352323576 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe12806b820 0x7fe1280b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:32.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.852+0000 7fe12dad0640 1 --2- 172.21.15.32:0/2352323576 >> v2:172.21.15.121:3300/0 conn(0x7fe12815e9b0 0x7fe1280d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:32.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.852+0000 7fe12cace640 1 -- 172.21.15.32:0/2352323576 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1462316854 0 0) 0x7fe1280d3170 con 0x7fe1280d4210 2024-04-24T22:59:32.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.852+0000 7fe12cace640 1 -- 172.21.15.32:0/2352323576 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe110003520 con 0x7fe1280d4210 2024-04-24T22:59:32.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.852+0000 7fe12cace640 1 -- 172.21.15.32:0/2352323576 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1160059791 0 0) 0x7fe110003520 con 0x7fe1280d4210 2024-04-24T22:59:32.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.852+0000 7fe12cace640 1 -- 172.21.15.32:0/2352323576 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe12806b820 msgr2=0x7fe1280b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:32.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.852+0000 7fe12cace640 1 --2- 172.21.15.32:0/2352323576 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe12806b820 0x7fe1280b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:32.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.852+0000 7fe12cace640 1 -- 172.21.15.32:0/2352323576 >> v2:172.21.15.121:3300/0 conn(0x7fe12815e9b0 msgr2=0x7fe1280d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:32.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.852+0000 7fe12cace640 1 --2- 172.21.15.32:0/2352323576 >> v2:172.21.15.121:3300/0 conn(0x7fe12815e9b0 0x7fe1280d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:32.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.852+0000 7fe12cace640 1 -- 172.21.15.32:0/2352323576 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1280d3a10 con 0x7fe1280d4210 2024-04-24T22:59:32.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.852+0000 7fe12cace640 1 -- 172.21.15.32:0/2352323576 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7fe11c001670 con 0x7fe1280d4210 2024-04-24T22:59:32.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.852+0000 7fe12dad0640 1 --2- 172.21.15.32:0/2352323576 >> v2:172.21.15.121:3300/0 conn(0x7fe12815e9b0 0x7fe1280d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:59:32.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.852+0000 7fe12cace640 1 -- 172.21.15.32:0/2352323576 >> v1:172.21.15.32:6789/0 conn(0x7fe1280d4210 legacy=0x7fe12806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:32.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.852+0000 7fe12cace640 1 --2- 172.21.15.32:0/2352323576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1100039b0 0x7fe110003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:32.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.852+0000 7fe12cace640 1 --2- 172.21.15.32:0/2352323576 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe110004800 0x7fe110004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:32.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.852+0000 7fe12cace640 1 --2- 172.21.15.32:0/2352323576 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe12806b820 0x7fe1100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:32.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.852+0000 7fe12cace640 1 -- 172.21.15.32:0/2352323576 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe1280d3450 con 0x7fe1100039b0 2024-04-24T22:59:32.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.852+0000 7fe12cace640 1 -- 172.21.15.32:0/2352323576 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe1280d3730 con 0x7fe110004800 2024-04-24T22:59:32.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.852+0000 7fe12cace640 1 -- 172.21.15.32:0/2352323576 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe11000b9c0 con 0x7fe12806b820 2024-04-24T22:59:32.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.852+0000 7fe12dad0640 1 --2- 172.21.15.32:0/2352323576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1100039b0 0x7fe110003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:32.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.852+0000 7fe12d2cf640 1 --2- 172.21.15.32:0/2352323576 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe110004800 0x7fe110004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:32.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.852+0000 7fe12e2d1640 1 --2- 172.21.15.32:0/2352323576 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe12806b820 0x7fe1100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:32.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.852+0000 7fe12dad0640 1 -- 172.21.15.32:0/2352323576 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe12806b820 msgr2=0x7fe1100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:32.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.852+0000 7fe12dad0640 1 --2- 172.21.15.32:0/2352323576 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe12806b820 0x7fe1100051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:32.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.852+0000 7fe12dad0640 1 -- 172.21.15.32:0/2352323576 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe110004800 msgr2=0x7fe110004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:32.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.852+0000 7fe12dad0640 1 --2- 172.21.15.32:0/2352323576 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe110004800 0x7fe110004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:32.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.852+0000 7fe12dad0640 1 -- 172.21.15.32:0/2352323576 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe110003520 con 0x7fe1100039b0 2024-04-24T22:59:32.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.853+0000 7fe12e2d1640 1 --2- 172.21.15.32:0/2352323576 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe12806b820 0x7fe1100051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:32.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.853+0000 7fe12d2cf640 1 --2- 172.21.15.32:0/2352323576 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe110004800 0x7fe110004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:32.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.853+0000 7fe12dad0640 1 --2- 172.21.15.32:0/2352323576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1100039b0 0x7fe110003dd0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe120009c90 tx=0x7fe110005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aae27e62cc27ac72 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:32.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.853+0000 7fe12cace640 1 -- 172.21.15.32:0/2352323576 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe120036070 con 0x7fe1100039b0 2024-04-24T22:59:32.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.853+0000 7fe12cace640 1 -- 172.21.15.32:0/2352323576 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe120031400 con 0x7fe1100039b0 2024-04-24T22:59:32.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.853+0000 7fe12cace640 1 -- 172.21.15.32:0/2352323576 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe120009860 con 0x7fe1100039b0 2024-04-24T22:59:32.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.854+0000 7fe12fd34640 1 -- 172.21.15.32:0/2352323576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1100039b0 msgr2=0x7fe110003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:32.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.854+0000 7fe12fd34640 1 --2- 172.21.15.32:0/2352323576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1100039b0 0x7fe110003dd0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe120009c90 tx=0x7fe110005830 comp rx=0 tx=0).stop 2024-04-24T22:59:32.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.854+0000 7fe12e2d1640 1 -- 172.21.15.32:0/2352323576 reap_dead start 2024-04-24T22:59:32.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.854+0000 7fe12fd34640 1 -- 172.21.15.32:0/2352323576 shutdown_connections 2024-04-24T22:59:32.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.854+0000 7fe12fd34640 1 -- 172.21.15.32:0/2352323576 >> 172.21.15.32:0/2352323576 conn(0x7fe1280d11d0 msgr2=0x7fe1280d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:32.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.854+0000 7fe12fd34640 1 -- 172.21.15.32:0/2352323576 shutdown_connections 2024-04-24T22:59:32.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.854+0000 7fe12fd34640 1 -- 172.21.15.32:0/2352323576 wait complete. 2024-04-24T22:59:32.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.855+0000 7fe12fd34640 1 Processor -- start 2024-04-24T22:59:32.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.855+0000 7fe12fd34640 1 -- start start 2024-04-24T22:59:32.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.856+0000 7fe12fd34640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1100039b0 0x7fe128157cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:32.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.856+0000 7fe12fd34640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1280c88a0 0x7fe128156770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:32.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.856+0000 7fe12fd34640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe128156cb0 0x7fe128157130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:32.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.856+0000 7fe12fd34640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe1280ba0e0 con 0x7fe1280c88a0 2024-04-24T22:59:32.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.856+0000 7fe12fd34640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe1280d2400 con 0x7fe1100039b0 2024-04-24T22:59:32.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.856+0000 7fe12fd34640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe1280ba260 con 0x7fe128156cb0 2024-04-24T22:59:32.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.856+0000 7fe12d2cf640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1280c88a0 0x7fe128156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:32.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.856+0000 7fe12d2cf640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1280c88a0 0x7fe128156770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52176/0 (socket says 172.21.15.32:52176) 2024-04-24T22:59:32.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.856+0000 7fe12d2cf640 1 -- 172.21.15.32:0/2135456697 learned_addr learned my addr 172.21.15.32:0/2135456697 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:32.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.856+0000 7fe12e2d1640 1 --2- 172.21.15.32:0/2135456697 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe128156cb0 0x7fe128157130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:32.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.856+0000 7fe12dad0640 1 --2- 172.21.15.32:0/2135456697 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1100039b0 0x7fe128157cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:32.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.856+0000 7fe12d2cf640 1 -- 172.21.15.32:0/2135456697 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe128156cb0 msgr2=0x7fe128157130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:32.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.856+0000 7fe12d2cf640 1 --2- 172.21.15.32:0/2135456697 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe128156cb0 0x7fe128157130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:32.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.856+0000 7fe12d2cf640 1 -- 172.21.15.32:0/2135456697 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1100039b0 msgr2=0x7fe128157cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:32.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.856+0000 7fe12d2cf640 1 --2- 172.21.15.32:0/2135456697 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1100039b0 0x7fe128157cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:32.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.856+0000 7fe12d2cf640 1 -- 172.21.15.32:0/2135456697 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe128157670 con 0x7fe1280c88a0 2024-04-24T22:59:32.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.856+0000 7fe12dad0640 1 --2- 172.21.15.32:0/2135456697 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1100039b0 0x7fe128157cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:32.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.856+0000 7fe12d2cf640 1 --2- 172.21.15.32:0/2135456697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1280c88a0 0x7fe128156770 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe11c00e7a0 tx=0x7fe11c00ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:32.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.857+0000 7fe1267fc640 1 -- 172.21.15.32:0/2135456697 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11c00c8d0 con 0x7fe1280c88a0 2024-04-24T22:59:32.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.857+0000 7fe1267fc640 1 -- 172.21.15.32:0/2135456697 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe11c018070 con 0x7fe1280c88a0 2024-04-24T22:59:32.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.857+0000 7fe12fd34640 1 -- 172.21.15.32:0/2135456697 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1280c90b0 con 0x7fe1280c88a0 2024-04-24T22:59:32.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.857+0000 7fe1267fc640 1 -- 172.21.15.32:0/2135456697 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11c00cc70 con 0x7fe1280c88a0 2024-04-24T22:59:32.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.857+0000 7fe12fd34640 1 -- 172.21.15.32:0/2135456697 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1280c9670 con 0x7fe1280c88a0 2024-04-24T22:59:32.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.858+0000 7fe12fd34640 1 -- 172.21.15.32:0/2135456697 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe128068430 con 0x7fe1280c88a0 2024-04-24T22:59:32.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.858+0000 7fe1267fc640 1 -- 172.21.15.32:0/2135456697 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe11c01d460 con 0x7fe1280c88a0 2024-04-24T22:59:32.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.859+0000 7fe1267fc640 1 --2- 172.21.15.32:0/2135456697 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe11406be60 0x7fe11406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:32.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.859+0000 7fe1267fc640 1 -- 172.21.15.32:0/2135456697 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe11c08ac60 con 0x7fe1280c88a0 2024-04-24T22:59:32.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.859+0000 7fe12dad0640 1 --2- 172.21.15.32:0/2135456697 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe11406be60 0x7fe11406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:32.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.859+0000 7fe12dad0640 1 --2- 172.21.15.32:0/2135456697 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe11406be60 0x7fe11406e320 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe120033000 tx=0x7fe120008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:32.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:32.864+0000 7fe1267fc640 1 -- 172.21.15.32:0/2135456697 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe11c059610 con 0x7fe1280c88a0 2024-04-24T22:59:33.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.044+0000 7fe12fd34640 1 -- 172.21.15.32:0/2135456697 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe1280b81f0 con 0x7fe1280c88a0 2024-04-24T22:59:33.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.044+0000 7fe1267fc640 1 -- 172.21.15.32:0/2135456697 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7fe11c058f10 con 0x7fe1280c88a0 2024-04-24T22:59:33.048 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-24T22:59:33.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.046+0000 7fe12fd34640 1 -- 172.21.15.32:0/2135456697 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe11406be60 msgr2=0x7fe11406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:33.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.046+0000 7fe12fd34640 1 --2- 172.21.15.32:0/2135456697 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe11406be60 0x7fe11406e320 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe120033000 tx=0x7fe120008030 comp rx=0 tx=0).stop 2024-04-24T22:59:33.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.046+0000 7fe12fd34640 1 -- 172.21.15.32:0/2135456697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1280c88a0 msgr2=0x7fe128156770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:33.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.046+0000 7fe12fd34640 1 --2- 172.21.15.32:0/2135456697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1280c88a0 0x7fe128156770 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe11c00e7a0 tx=0x7fe11c00ec60 comp rx=0 tx=0).stop 2024-04-24T22:59:33.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.046+0000 7fe12fd34640 1 -- 172.21.15.32:0/2135456697 shutdown_connections 2024-04-24T22:59:33.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.046+0000 7fe12fd34640 1 --2- 172.21.15.32:0/2135456697 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe11406be60 0x7fe11406e320 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:33.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.046+0000 7fe12fd34640 1 --2- 172.21.15.32:0/2135456697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1280c88a0 0x7fe128156770 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:33.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.046+0000 7fe12fd34640 1 --2- 172.21.15.32:0/2135456697 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe128156cb0 0x7fe128157130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:33.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.046+0000 7fe12fd34640 1 --2- 172.21.15.32:0/2135456697 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1100039b0 0x7fe128157cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:33.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.046+0000 7fe12fd34640 1 -- 172.21.15.32:0/2135456697 >> 172.21.15.32:0/2135456697 conn(0x7fe1280d11d0 msgr2=0x7fe12815d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:33.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.046+0000 7fe12fd34640 1 -- 172.21.15.32:0/2135456697 shutdown_connections 2024-04-24T22:59:33.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.046+0000 7fe12fd34640 1 -- 172.21.15.32:0/2135456697 wait complete. 2024-04-24T22:59:33.061 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:59:33.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.160+0000 7fee0fa58640 1 Processor -- start 2024-04-24T22:59:33.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.160+0000 7fee0fa58640 1 -- start start 2024-04-24T22:59:33.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.161+0000 7fee0fa58640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fee08156bb0 0x7fee0815dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:33.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.161+0000 7fee0fa58640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fee0806d400 0x7fee0806d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:33.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.161+0000 7fee0fa58640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee0815ccf0 con 0x7fee08072c80 2024-04-24T22:59:33.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.161+0000 7fee0fa58640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fee0815cfb0 con 0x7fee08156bb0 2024-04-24T22:59:33.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.161+0000 7fee0fa58640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fee0815d290 con 0x7fee0806d400 2024-04-24T22:59:33.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.161+0000 7fee0d7f4640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fee08156bb0 0x7fee0815dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:33.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.161+0000 7fee0d7f4640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fee08156bb0 0x7fee0815dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52192/0 (socket says 172.21.15.32:52192) 2024-04-24T22:59:33.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.161+0000 7fee0d7f4640 1 -- 172.21.15.32:0/820425951 learned_addr learned my addr 172.21.15.32:0/820425951 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:33.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.161+0000 7fee0cff3640 1 --2- 172.21.15.32:0/820425951 >> v2:172.21.15.121:3300/0 conn(0x7fee0806d400 0x7fee0806d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:33.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.161+0000 7fee0d7f4640 1 -- 172.21.15.32:0/820425951 >> v2:172.21.15.121:3300/0 conn(0x7fee0806d400 msgr2=0x7fee0806d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:33.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.161+0000 7fee0d7f4640 1 --2- 172.21.15.32:0/820425951 >> v2:172.21.15.121:3300/0 conn(0x7fee0806d400 0x7fee0806d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:33.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.161+0000 7fee0d7f4640 1 -- 172.21.15.32:0/820425951 >> v1:172.21.15.121:6789/0 conn(0x7fee08072c80 legacy=0x7fee080b5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:33.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.161+0000 7fee0d7f4640 1 -- 172.21.15.32:0/820425951 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee0815d570 con 0x7fee08156bb0 2024-04-24T22:59:33.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.162+0000 7fee0d7f4640 1 --2- 172.21.15.32:0/820425951 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee08156bb0 0x7fee0815dbc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fee080b6b80 tx=0x7fee0402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d1a2164c8a5bc4b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:33.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.162+0000 7fedfffff640 1 -- 172.21.15.32:0/820425951 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee04037070 con 0x7fee08156bb0 2024-04-24T22:59:33.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.162+0000 7fedfffff640 1 -- 172.21.15.32:0/820425951 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee04032440 con 0x7fee08156bb0 2024-04-24T22:59:33.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.162+0000 7fedfffff640 1 -- 172.21.15.32:0/820425951 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee04032810 con 0x7fee08156bb0 2024-04-24T22:59:33.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.163+0000 7fee0fa58640 1 -- 172.21.15.32:0/820425951 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee08156bb0 msgr2=0x7fee0815dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:33.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.163+0000 7fee0fa58640 1 --2- 172.21.15.32:0/820425951 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee08156bb0 0x7fee0815dbc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fee080b6b80 tx=0x7fee0402a270 comp rx=0 tx=0).stop 2024-04-24T22:59:33.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.163+0000 7fee0fa58640 1 -- 172.21.15.32:0/820425951 shutdown_connections 2024-04-24T22:59:33.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.163+0000 7fee0fa58640 1 --2- 172.21.15.32:0/820425951 >> v2:172.21.15.121:3300/0 conn(0x7fee0806d400 0x7fee0806d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:33.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.163+0000 7fee0fa58640 1 --2- 172.21.15.32:0/820425951 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee08156bb0 0x7fee0815dbc0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:33.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.163+0000 7fee0fa58640 1 -- 172.21.15.32:0/820425951 >> 172.21.15.32:0/820425951 conn(0x7fee0806e800 msgr2=0x7fee0806ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:33.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.163+0000 7fee0fa58640 1 -- 172.21.15.32:0/820425951 shutdown_connections 2024-04-24T22:59:33.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.163+0000 7fee0fa58640 1 -- 172.21.15.32:0/820425951 wait complete. 2024-04-24T22:59:33.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.164+0000 7fee0fa58640 1 Processor -- start 2024-04-24T22:59:33.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.164+0000 7fee0fa58640 1 -- start start 2024-04-24T22:59:33.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.165+0000 7fee0fa58640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee0806d400 0x7fee08152790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:33.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.165+0000 7fee0fa58640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee08072c80 0x7fee0815e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:33.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.165+0000 7fee0fa58640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fee08156bb0 0x7fee0815e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:33.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.165+0000 7fee0fa58640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fee08156f90 con 0x7fee08072c80 2024-04-24T22:59:33.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.165+0000 7fee0fa58640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fee0815e100 con 0x7fee0806d400 2024-04-24T22:59:33.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.165+0000 7fee0fa58640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fee08157110 con 0x7fee08156bb0 2024-04-24T22:59:33.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.165+0000 7fee0cff3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee08072c80 0x7fee0815e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:33.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.165+0000 7fee0cff3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee08072c80 0x7fee0815e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52196/0 (socket says 172.21.15.32:52196) 2024-04-24T22:59:33.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.165+0000 7fee0cff3640 1 -- 172.21.15.32:0/996533415 learned_addr learned my addr 172.21.15.32:0/996533415 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:33.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.165+0000 7fee0d7f4640 1 --2- 172.21.15.32:0/996533415 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee0806d400 0x7fee08152790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:33.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.165+0000 7fee0dff5640 1 --2- 172.21.15.32:0/996533415 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fee08156bb0 0x7fee0815e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:33.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.165+0000 7fee0cff3640 1 -- 172.21.15.32:0/996533415 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fee08156bb0 msgr2=0x7fee0815e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:33.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.165+0000 7fee0cff3640 1 --2- 172.21.15.32:0/996533415 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fee08156bb0 0x7fee0815e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:33.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.165+0000 7fee0cff3640 1 -- 172.21.15.32:0/996533415 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee0806d400 msgr2=0x7fee08152790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:33.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.165+0000 7fee0cff3640 1 --2- 172.21.15.32:0/996533415 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee0806d400 0x7fee08152790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:33.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.165+0000 7fee0cff3640 1 -- 172.21.15.32:0/996533415 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee08157800 con 0x7fee08072c80 2024-04-24T22:59:33.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.165+0000 7fee0cff3640 1 --2- 172.21.15.32:0/996533415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee08072c80 0x7fee0815e280 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fedf800d5d0 tx=0x7fedf800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:33.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.165+0000 7fedfdffb640 1 -- 172.21.15.32:0/996533415 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf80125f0 con 0x7fee08072c80 2024-04-24T22:59:33.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.166+0000 7fedfdffb640 1 -- 172.21.15.32:0/996533415 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedf8019070 con 0x7fee08072c80 2024-04-24T22:59:33.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.166+0000 7fee0fa58640 1 -- 172.21.15.32:0/996533415 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee08159a60 con 0x7fee08072c80 2024-04-24T22:59:33.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.166+0000 7fedfdffb640 1 -- 172.21.15.32:0/996533415 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf80129f0 con 0x7fee08072c80 2024-04-24T22:59:33.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.166+0000 7fee0fa58640 1 -- 172.21.15.32:0/996533415 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee0815a020 con 0x7fee08072c80 2024-04-24T22:59:33.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.167+0000 7fee0fa58640 1 -- 172.21.15.32:0/996533415 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee08068430 con 0x7fee08072c80 2024-04-24T22:59:33.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.167+0000 7fedfdffb640 1 -- 172.21.15.32:0/996533415 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fedf8012b90 con 0x7fee08072c80 2024-04-24T22:59:33.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.167+0000 7fedfdffb640 1 --2- 172.21.15.32:0/996533415 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fedec06be30 0x7fedec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:33.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.168+0000 7fedfdffb640 1 -- 172.21.15.32:0/996533415 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fedf808b0f0 con 0x7fee08072c80 2024-04-24T22:59:33.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.168+0000 7fee0d7f4640 1 --2- 172.21.15.32:0/996533415 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fedec06be30 0x7fedec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:33.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.168+0000 7fee0d7f4640 1 --2- 172.21.15.32:0/996533415 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fedec06be30 0x7fedec06e2f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fee04034000 tx=0x7fee0402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:33.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.173+0000 7fedfdffb640 1 -- 172.21.15.32:0/996533415 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fedf80594e0 con 0x7fee08072c80 2024-04-24T22:59:33.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.356+0000 7fee0fa58640 1 -- 172.21.15.32:0/996533415 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fee081532b0 con 0x7fee08072c80 2024-04-24T22:59:33.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.357+0000 7fedfdffb640 1 -- 172.21.15.32:0/996533415 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16621 (secure 0 0 0) 0x7fedf805d7e0 con 0x7fee08072c80 2024-04-24T22:59:33.359 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:59:33.359 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T22:59:05.517554+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:59:02.016639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T22:59:02.846087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:59:33.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.359+0000 7fee0fa58640 1 -- 172.21.15.32:0/996533415 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fedec06be30 msgr2=0x7fedec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:33.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.359+0000 7fee0fa58640 1 --2- 172.21.15.32:0/996533415 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fedec06be30 0x7fedec06e2f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fee04034000 tx=0x7fee0402e000 comp rx=0 tx=0).stop 2024-04-24T22:59:33.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.359+0000 7fee0fa58640 1 -- 172.21.15.32:0/996533415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee08072c80 msgr2=0x7fee0815e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:33.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.359+0000 7fee0fa58640 1 --2- 172.21.15.32:0/996533415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee08072c80 0x7fee0815e280 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fedf800d5d0 tx=0x7fedf800da90 comp rx=0 tx=0).stop 2024-04-24T22:59:33.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.359+0000 7fee0fa58640 1 -- 172.21.15.32:0/996533415 shutdown_connections 2024-04-24T22:59:33.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.359+0000 7fee0fa58640 1 --2- 172.21.15.32:0/996533415 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fedec06be30 0x7fedec06e2f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:33.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.359+0000 7fee0fa58640 1 --2- 172.21.15.32:0/996533415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee08072c80 0x7fee0815e280 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:33.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.359+0000 7fee0fa58640 1 --2- 172.21.15.32:0/996533415 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fee08156bb0 0x7fee0815e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:33.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.359+0000 7fee0fa58640 1 --2- 172.21.15.32:0/996533415 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee0806d400 0x7fee08152790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:33.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.359+0000 7fee0fa58640 1 -- 172.21.15.32:0/996533415 >> 172.21.15.32:0/996533415 conn(0x7fee0806e800 msgr2=0x7fee0815eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:33.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.359+0000 7fee0fa58640 1 -- 172.21.15.32:0/996533415 shutdown_connections 2024-04-24T22:59:33.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.359+0000 7fee0fa58640 1 -- 172.21.15.32:0/996533415 wait complete. 2024-04-24T22:59:33.372 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T22:59:02.016639+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T22:59:02.846087+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T22:59:33.372 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T22:59:33.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.475+0000 7ff7e0ee1640 1 Processor -- start 2024-04-24T22:59:33.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.475+0000 7ff7e0ee1640 1 -- start start 2024-04-24T22:59:33.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.476+0000 7ff7e0ee1640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7dc06b820 0x7ff7dc0d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:33.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.476+0000 7ff7e0ee1640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff7dc0d4210 0x7ff7dc0b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:33.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.476+0000 7ff7e0ee1640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7dc0d3170 con 0x7ff7dc15e9b0 2024-04-24T22:59:33.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.476+0000 7ff7e0ee1640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7ff7dc0d3450 con 0x7ff7dc0d4210 2024-04-24T22:59:33.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.476+0000 7ff7e0ee1640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff7dc0d3730 con 0x7ff7dc06b820 2024-04-24T22:59:33.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.476+0000 7ff7dad9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff7dc0d4210 0x7ff7dc0b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:33.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.476+0000 7ff7da59c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff7dc15e9b0 0x7ff7dc06b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34084/0 (socket says 172.21.15.32:34084) 2024-04-24T22:59:33.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.476+0000 7ff7da59c640 1 -- 172.21.15.32:0/325949209 learned_addr learned my addr 172.21.15.32:0/325949209 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:33.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.477+0000 7ff7d3fff640 1 --2- 172.21.15.32:0/325949209 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7dc06b820 0x7ff7dc0d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:33.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.477+0000 7ff7dad9d640 1 -- 172.21.15.32:0/325949209 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7dc06b820 msgr2=0x7ff7dc0d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:33.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.477+0000 7ff7dad9d640 1 --2- 172.21.15.32:0/325949209 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7dc06b820 0x7ff7dc0d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:33.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.477+0000 7ff7dad9d640 1 -- 172.21.15.32:0/325949209 >> v1:172.21.15.32:6789/0 conn(0x7ff7dc15e9b0 legacy=0x7ff7dc06b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:33.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.477+0000 7ff7d9d9b640 1 -- 172.21.15.32:0/325949209 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3170080949 0 0) 0x7ff7dc0d3170 con 0x7ff7dc15e9b0 2024-04-24T22:59:33.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.477+0000 7ff7dad9d640 1 -- 172.21.15.32:0/325949209 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7dc0d3a10 con 0x7ff7dc0d4210 2024-04-24T22:59:33.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.478+0000 7ff7dad9d640 1 --2- 172.21.15.32:0/325949209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7dc0d4210 0x7ff7dc0b78f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff7d4009d90 tx=0x7ff7d402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d2498daa6cda672 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:33.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.478+0000 7ff7d9d9b640 1 -- 172.21.15.32:0/325949209 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d4007740 con 0x7ff7dc0d4210 2024-04-24T22:59:33.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.478+0000 7ff7d9d9b640 1 -- 172.21.15.32:0/325949209 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7d40078e0 con 0x7ff7dc0d4210 2024-04-24T22:59:33.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.478+0000 7ff7d9d9b640 1 -- 172.21.15.32:0/325949209 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d4007cb0 con 0x7ff7dc0d4210 2024-04-24T22:59:33.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.478+0000 7ff7e0ee1640 1 -- 172.21.15.32:0/325949209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7dc0d4210 msgr2=0x7ff7dc0b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:33.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.478+0000 7ff7e0ee1640 1 --2- 172.21.15.32:0/325949209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7dc0d4210 0x7ff7dc0b78f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff7d4009d90 tx=0x7ff7d402a660 comp rx=0 tx=0).stop 2024-04-24T22:59:33.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.479+0000 7ff7e0ee1640 1 -- 172.21.15.32:0/325949209 shutdown_connections 2024-04-24T22:59:33.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.479+0000 7ff7e0ee1640 1 --2- 172.21.15.32:0/325949209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7dc0d4210 0x7ff7dc0b78f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:33.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.479+0000 7ff7e0ee1640 1 --2- 172.21.15.32:0/325949209 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7dc06b820 0x7ff7dc0d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:33.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.479+0000 7ff7e0ee1640 1 -- 172.21.15.32:0/325949209 >> 172.21.15.32:0/325949209 conn(0x7ff7dc0d11d0 msgr2=0x7ff7dc0d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:33.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.479+0000 7ff7e0ee1640 1 -- 172.21.15.32:0/325949209 shutdown_connections 2024-04-24T22:59:33.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.479+0000 7ff7e0ee1640 1 -- 172.21.15.32:0/325949209 wait complete. 2024-04-24T22:59:33.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.480+0000 7ff7e0ee1640 1 Processor -- start 2024-04-24T22:59:33.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.480+0000 7ff7e0ee1640 1 -- start start 2024-04-24T22:59:33.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.481+0000 7ff7e0ee1640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7dc06b820 0x7ff7dc1f3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:33.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.481+0000 7ff7e0ee1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7dc0d4210 0x7ff7dc160d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:33.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.481+0000 7ff7e0ee1640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7dc15e9b0 0x7ff7dc15b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:33.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.481+0000 7ff7e0ee1640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff7dc0ba0e0 con 0x7ff7dc0d4210 2024-04-24T22:59:33.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.481+0000 7ff7e0ee1640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff7dc0d2400 con 0x7ff7dc15e9b0 2024-04-24T22:59:33.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.481+0000 7ff7e0ee1640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff7dc0ba260 con 0x7ff7dc06b820 2024-04-24T22:59:33.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.481+0000 7ff7d3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7dc0d4210 0x7ff7dc160d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:33.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.481+0000 7ff7d3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7dc0d4210 0x7ff7dc160d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52218/0 (socket says 172.21.15.32:52218) 2024-04-24T22:59:33.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.481+0000 7ff7d3fff640 1 -- 172.21.15.32:0/2325390921 learned_addr learned my addr 172.21.15.32:0/2325390921 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:33.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.481+0000 7ff7dad9d640 1 --2- 172.21.15.32:0/2325390921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7dc15e9b0 0x7ff7dc15b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:33.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.481+0000 7ff7da59c640 1 --2- 172.21.15.32:0/2325390921 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7dc06b820 0x7ff7dc1f3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:33.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.481+0000 7ff7d3fff640 1 -- 172.21.15.32:0/2325390921 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7dc06b820 msgr2=0x7ff7dc1f3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:33.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.481+0000 7ff7d3fff640 1 --2- 172.21.15.32:0/2325390921 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7dc06b820 0x7ff7dc1f3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:33.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.481+0000 7ff7d3fff640 1 -- 172.21.15.32:0/2325390921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7dc15e9b0 msgr2=0x7ff7dc15b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:33.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.481+0000 7ff7d3fff640 1 --2- 172.21.15.32:0/2325390921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7dc15e9b0 0x7ff7dc15b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:33.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.481+0000 7ff7d3fff640 1 -- 172.21.15.32:0/2325390921 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7dc15f7a0 con 0x7ff7dc0d4210 2024-04-24T22:59:33.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.481+0000 7ff7d3fff640 1 --2- 172.21.15.32:0/2325390921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7dc0d4210 0x7ff7dc160d10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff7c400e9b0 tx=0x7ff7c400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:33.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.482+0000 7ff7d2ffd640 1 -- 172.21.15.32:0/2325390921 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c4017070 con 0x7ff7dc0d4210 2024-04-24T22:59:33.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.482+0000 7ff7d2ffd640 1 -- 172.21.15.32:0/2325390921 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7c400c9a0 con 0x7ff7dc0d4210 2024-04-24T22:59:33.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.482+0000 7ff7e0ee1640 1 -- 172.21.15.32:0/2325390921 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7dc15fa80 con 0x7ff7dc0d4210 2024-04-24T22:59:33.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.482+0000 7ff7d2ffd640 1 -- 172.21.15.32:0/2325390921 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c400cd90 con 0x7ff7dc0d4210 2024-04-24T22:59:33.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.482+0000 7ff7e0ee1640 1 -- 172.21.15.32:0/2325390921 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7dc160040 con 0x7ff7dc0d4210 2024-04-24T22:59:33.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.483+0000 7ff7e0ee1640 1 -- 172.21.15.32:0/2325390921 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7dc068430 con 0x7ff7dc0d4210 2024-04-24T22:59:33.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.483+0000 7ff7d2ffd640 1 -- 172.21.15.32:0/2325390921 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff7c401a070 con 0x7ff7dc0d4210 2024-04-24T22:59:33.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.484+0000 7ff7d2ffd640 1 --2- 172.21.15.32:0/2325390921 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff7bc06be30 0x7ff7bc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:33.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.484+0000 7ff7d2ffd640 1 -- 172.21.15.32:0/2325390921 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff7c408b260 con 0x7ff7dc0d4210 2024-04-24T22:59:33.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.484+0000 7ff7da59c640 1 --2- 172.21.15.32:0/2325390921 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff7bc06be30 0x7ff7bc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:33.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.485+0000 7ff7da59c640 1 --2- 172.21.15.32:0/2325390921 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff7bc06be30 0x7ff7bc06e2f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc000f60 tx=0x7ff7cc007c00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:33.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.488+0000 7ff7d2ffd640 1 -- 172.21.15.32:0/2325390921 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff7c4059c10 con 0x7ff7dc0d4210 2024-04-24T22:59:33.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.671+0000 7ff7e0ee1640 1 -- 172.21.15.32:0/2325390921 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff7dc0d2580 con 0x7ff7dc0d4210 2024-04-24T22:59:33.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.672+0000 7ff7d2ffd640 1 -- 172.21.15.32:0/2325390921 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7ff7c4059510 con 0x7ff7dc0d4210 2024-04-24T22:59:33.674 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T22:59:33.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.673+0000 7ff7e0ee1640 1 -- 172.21.15.32:0/2325390921 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff7bc06be30 msgr2=0x7ff7bc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:33.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.673+0000 7ff7e0ee1640 1 --2- 172.21.15.32:0/2325390921 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff7bc06be30 0x7ff7bc06e2f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc000f60 tx=0x7ff7cc007c00 comp rx=0 tx=0).stop 2024-04-24T22:59:33.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.673+0000 7ff7e0ee1640 1 -- 172.21.15.32:0/2325390921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7dc0d4210 msgr2=0x7ff7dc160d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:33.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.673+0000 7ff7e0ee1640 1 --2- 172.21.15.32:0/2325390921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7dc0d4210 0x7ff7dc160d10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff7c400e9b0 tx=0x7ff7c400ee70 comp rx=0 tx=0).stop 2024-04-24T22:59:33.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.673+0000 7ff7e0ee1640 1 -- 172.21.15.32:0/2325390921 shutdown_connections 2024-04-24T22:59:33.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.673+0000 7ff7e0ee1640 1 --2- 172.21.15.32:0/2325390921 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff7bc06be30 0x7ff7bc06e2f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:33.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.673+0000 7ff7e0ee1640 1 --2- 172.21.15.32:0/2325390921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7dc0d4210 0x7ff7dc160d10 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:33.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.673+0000 7ff7e0ee1640 1 --2- 172.21.15.32:0/2325390921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7dc15e9b0 0x7ff7dc15b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:33.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.673+0000 7ff7e0ee1640 1 --2- 172.21.15.32:0/2325390921 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7dc06b820 0x7ff7dc1f3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:33.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.673+0000 7ff7e0ee1640 1 -- 172.21.15.32:0/2325390921 >> 172.21.15.32:0/2325390921 conn(0x7ff7dc0d11d0 msgr2=0x7ff7dc06b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:33.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.674+0000 7ff7e0ee1640 1 -- 172.21.15.32:0/2325390921 shutdown_connections 2024-04-24T22:59:33.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.674+0000 7ff7e0ee1640 1 -- 172.21.15.32:0/2325390921 wait complete. 2024-04-24T22:59:33.686 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T22:59:33.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.788+0000 7f1a2d038640 1 Processor -- start 2024-04-24T22:59:33.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.788+0000 7f1a2d038640 1 -- start start 2024-04-24T22:59:33.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.789+0000 7f1a2d038640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1a28158dd0 0x7f1a2806b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:33.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.789+0000 7f1a2d038640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a2806bd10 con 0x7f1a2806b860 2024-04-24T22:59:33.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.789+0000 7f1a2d038640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a2815ef10 con 0x7f1a2806dc60 2024-04-24T22:59:33.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.789+0000 7f1a2d038640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f1a2815f1f0 con 0x7f1a28158dd0 2024-04-24T22:59:33.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.789+0000 7f1a2659c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1a2806b860 0x7f1a2806d550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34086/0 (socket says 172.21.15.32:34086) 2024-04-24T22:59:33.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.789+0000 7f1a2659c640 1 -- 172.21.15.32:0/2505611010 learned_addr learned my addr 172.21.15.32:0/2505611010 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:33.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.789+0000 7f1a25d9b640 1 -- 172.21.15.32:0/2505611010 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2244699673 0 0) 0x7f1a2806bd10 con 0x7f1a2806b860 2024-04-24T22:59:33.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.789+0000 7f1a25d9b640 1 -- 172.21.15.32:0/2505611010 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a14003520 con 0x7f1a2806b860 2024-04-24T22:59:33.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.789+0000 7f1a26d9d640 1 --2- 172.21.15.32:0/2505611010 >> v2:172.21.15.121:3300/0 conn(0x7f1a28158dd0 0x7f1a2806b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:33.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.790+0000 7f1a25d9b640 1 -- 172.21.15.32:0/2505611010 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1971689578 0 0) 0x7f1a14003520 con 0x7f1a2806b860 2024-04-24T22:59:33.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.790+0000 7f1a25d9b640 1 -- 172.21.15.32:0/2505611010 >> v2:172.21.15.121:3300/0 conn(0x7f1a28158dd0 msgr2=0x7f1a2806b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:33.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.790+0000 7f1a25d9b640 1 --2- 172.21.15.32:0/2505611010 >> v2:172.21.15.121:3300/0 conn(0x7f1a28158dd0 0x7f1a2806b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:33.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.790+0000 7f1a25d9b640 1 -- 172.21.15.32:0/2505611010 >> v1:172.21.15.121:6789/0 conn(0x7f1a2806dc60 legacy=0x7f1a280b5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:33.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.790+0000 7f1a25d9b640 1 -- 172.21.15.32:0/2505611010 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a2815f4d0 con 0x7f1a2806b860 2024-04-24T22:59:33.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.790+0000 7f1a25d9b640 1 -- 172.21.15.32:0/2505611010 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f1a10001670 con 0x7f1a2806b860 2024-04-24T22:59:33.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.790+0000 7f1a25d9b640 1 -- 172.21.15.32:0/2505611010 >> v1:172.21.15.32:6789/0 conn(0x7f1a2806b860 legacy=0x7f1a2806d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:33.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.790+0000 7f1a25d9b640 1 --2- 172.21.15.32:0/2505611010 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1a140039b0 0x7f1a14003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:33.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.790+0000 7f1a25d9b640 1 --2- 172.21.15.32:0/2505611010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a14004800 0x7f1a14004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:33.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.790+0000 7f1a25d9b640 1 --2- 172.21.15.32:0/2505611010 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a140051c0 0x7f1a1400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:33.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.790+0000 7f1a25d9b640 1 -- 172.21.15.32:0/2505611010 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1a2815f1f0 con 0x7f1a14004800 2024-04-24T22:59:33.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.790+0000 7f1a25d9b640 1 -- 172.21.15.32:0/2505611010 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1400c060 con 0x7f1a140051c0 2024-04-24T22:59:33.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.790+0000 7f1a25d9b640 1 -- 172.21.15.32:0/2505611010 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f1a1400c260 con 0x7f1a140039b0 2024-04-24T22:59:33.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.790+0000 7f1a2659c640 1 --2- 172.21.15.32:0/2505611010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a14004800 0x7f1a14004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:33.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.790+0000 7f1a26d9d640 1 --2- 172.21.15.32:0/2505611010 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1a140039b0 0x7f1a14003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:33.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.790+0000 7f1a2759e640 1 --2- 172.21.15.32:0/2505611010 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a140051c0 0x7f1a1400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:33.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.790+0000 7f1a2659c640 1 -- 172.21.15.32:0/2505611010 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1a140039b0 msgr2=0x7f1a14003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:33.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.790+0000 7f1a2659c640 1 --2- 172.21.15.32:0/2505611010 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1a140039b0 0x7f1a14003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:33.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.790+0000 7f1a2659c640 1 -- 172.21.15.32:0/2505611010 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a140051c0 msgr2=0x7f1a1400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:33.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.790+0000 7f1a2659c640 1 --2- 172.21.15.32:0/2505611010 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a140051c0 0x7f1a1400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:33.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.790+0000 7f1a2659c640 1 -- 172.21.15.32:0/2505611010 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a14003520 con 0x7f1a14004800 2024-04-24T22:59:33.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.790+0000 7f1a2759e640 1 --2- 172.21.15.32:0/2505611010 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a140051c0 0x7f1a1400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:33.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.790+0000 7f1a2759e640 1 -- 172.21.15.32:0/2505611010 reap_dead start 2024-04-24T22:59:33.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.791+0000 7f1a2659c640 1 --2- 172.21.15.32:0/2505611010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a14004800 0x7f1a14004c80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1a1000c210 tx=0x7f1a14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=948fa01cf0758350 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:33.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.791+0000 7f1a25d9b640 1 -- 172.21.15.32:0/2505611010 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a10007d30 con 0x7f1a14004800 2024-04-24T22:59:33.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.791+0000 7f1a25d9b640 1 -- 172.21.15.32:0/2505611010 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a10005ad0 con 0x7f1a14004800 2024-04-24T22:59:33.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.791+0000 7f1a25d9b640 1 -- 172.21.15.32:0/2505611010 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a10007070 con 0x7f1a14004800 2024-04-24T22:59:33.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.791+0000 7f1a2d038640 1 -- 172.21.15.32:0/2505611010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a14004800 msgr2=0x7f1a14004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:33.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.791+0000 7f1a2d038640 1 --2- 172.21.15.32:0/2505611010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a14004800 0x7f1a14004c80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1a1000c210 tx=0x7f1a14005830 comp rx=0 tx=0).stop 2024-04-24T22:59:33.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.792+0000 7f1a2d038640 1 -- 172.21.15.32:0/2505611010 shutdown_connections 2024-04-24T22:59:33.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.792+0000 7f1a2d038640 1 --2- 172.21.15.32:0/2505611010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a14004800 0x7f1a14004c80 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:33.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.792+0000 7f1a2d038640 1 -- 172.21.15.32:0/2505611010 >> 172.21.15.32:0/2505611010 conn(0x7f1a2806e800 msgr2=0x7f1a2806ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:33.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.792+0000 7f1a2d038640 1 -- 172.21.15.32:0/2505611010 shutdown_connections 2024-04-24T22:59:33.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.792+0000 7f1a2d038640 1 -- 172.21.15.32:0/2505611010 wait complete. 2024-04-24T22:59:33.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.793+0000 7f1a2d038640 1 Processor -- start 2024-04-24T22:59:33.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.793+0000 7f1a2d038640 1 -- start start 2024-04-24T22:59:33.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.794+0000 7f1a2d038640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1a14004800 0x7f1a281430e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:33.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.794+0000 7f1a2d038640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a281f8590 0x7f1a28145aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:33.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.794+0000 7f1a2d038640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a28159d50 0x7f1a280b4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:33.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.794+0000 7f1a2d038640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1a2806bfe0 con 0x7f1a281f8590 2024-04-24T22:59:33.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.794+0000 7f1a2d038640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1a2815fcd0 con 0x7f1a28159d50 2024-04-24T22:59:33.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.794+0000 7f1a2d038640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f1a281591b0 con 0x7f1a14004800 2024-04-24T22:59:33.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.794+0000 7f1a2659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a281f8590 0x7f1a28145aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:33.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.794+0000 7f1a2659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a281f8590 0x7f1a28145aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52238/0 (socket says 172.21.15.32:52238) 2024-04-24T22:59:33.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.794+0000 7f1a2659c640 1 -- 172.21.15.32:0/2565843695 learned_addr learned my addr 172.21.15.32:0/2565843695 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:33.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.794+0000 7f1a26d9d640 1 --2- 172.21.15.32:0/2565843695 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1a14004800 0x7f1a281430e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:33.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.794+0000 7f1a2759e640 1 --2- 172.21.15.32:0/2565843695 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a28159d50 0x7f1a280b4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:33.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.794+0000 7f1a2659c640 1 -- 172.21.15.32:0/2565843695 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1a14004800 msgr2=0x7f1a281430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:33.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.794+0000 7f1a2659c640 1 --2- 172.21.15.32:0/2565843695 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1a14004800 0x7f1a281430e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:33.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.794+0000 7f1a2659c640 1 -- 172.21.15.32:0/2565843695 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a28159d50 msgr2=0x7f1a280b4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:33.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.794+0000 7f1a2659c640 1 --2- 172.21.15.32:0/2565843695 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a28159d50 0x7f1a280b4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:33.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.794+0000 7f1a2659c640 1 -- 172.21.15.32:0/2565843695 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a28146410 con 0x7f1a281f8590 2024-04-24T22:59:33.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.794+0000 7f1a2659c640 1 --2- 172.21.15.32:0/2565843695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a281f8590 0x7f1a28145aa0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1a10001970 tx=0x7f1a10001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:33.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.795+0000 7f1a1f7fe640 1 -- 172.21.15.32:0/2565843695 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a10041070 con 0x7f1a281f8590 2024-04-24T22:59:33.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.795+0000 7f1a1f7fe640 1 -- 172.21.15.32:0/2565843695 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a10000ed0 con 0x7f1a281f8590 2024-04-24T22:59:33.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.795+0000 7f1a2d038640 1 -- 172.21.15.32:0/2565843695 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a281466f0 con 0x7f1a281f8590 2024-04-24T22:59:33.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.795+0000 7f1a1f7fe640 1 -- 172.21.15.32:0/2565843695 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a10007290 con 0x7f1a281f8590 2024-04-24T22:59:33.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.795+0000 7f1a2d038640 1 -- 172.21.15.32:0/2565843695 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a28143620 con 0x7f1a281f8590 2024-04-24T22:59:33.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.796+0000 7f1a2d038640 1 -- 172.21.15.32:0/2565843695 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a28068430 con 0x7f1a281f8590 2024-04-24T22:59:33.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.796+0000 7f1a1f7fe640 1 -- 172.21.15.32:0/2565843695 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1a10044070 con 0x7f1a281f8590 2024-04-24T22:59:33.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.797+0000 7f1a1f7fe640 1 --2- 172.21.15.32:0/2565843695 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1a0806be60 0x7f1a0806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:33.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.797+0000 7f1a1f7fe640 1 -- 172.21.15.32:0/2565843695 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1a100ab380 con 0x7f1a281f8590 2024-04-24T22:59:33.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.797+0000 7f1a26d9d640 1 --2- 172.21.15.32:0/2565843695 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1a0806be60 0x7f1a0806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:33.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.797+0000 7f1a26d9d640 1 --2- 172.21.15.32:0/2565843695 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1a0806be60 0x7f1a0806e320 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1a20009c40 tx=0x7f1a20009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:33.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.801+0000 7f1a1f7fe640 1 -- 172.21.15.32:0/2565843695 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1a10079dc0 con 0x7f1a281f8590 2024-04-24T22:59:33.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.982+0000 7f1a2d038640 1 -- 172.21.15.32:0/2565843695 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1a28160280 con 0x7f1a281f8590 2024-04-24T22:59:33.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.982+0000 7f1a1f7fe640 1 -- 172.21.15.32:0/2565843695 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f1a100796c0 con 0x7f1a281f8590 2024-04-24T22:59:33.984 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-24T22:59:33.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.985+0000 7f1a2d038640 1 -- 172.21.15.32:0/2565843695 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1a0806be60 msgr2=0x7f1a0806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:33.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.985+0000 7f1a2d038640 1 --2- 172.21.15.32:0/2565843695 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1a0806be60 0x7f1a0806e320 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1a20009c40 tx=0x7f1a20009190 comp rx=0 tx=0).stop 2024-04-24T22:59:33.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.985+0000 7f1a2d038640 1 -- 172.21.15.32:0/2565843695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a281f8590 msgr2=0x7f1a28145aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:33.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.985+0000 7f1a2d038640 1 --2- 172.21.15.32:0/2565843695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a281f8590 0x7f1a28145aa0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1a10001970 tx=0x7f1a10001a40 comp rx=0 tx=0).stop 2024-04-24T22:59:33.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.985+0000 7f1a2d038640 1 -- 172.21.15.32:0/2565843695 shutdown_connections 2024-04-24T22:59:33.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.985+0000 7f1a2d038640 1 --2- 172.21.15.32:0/2565843695 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1a0806be60 0x7f1a0806e320 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:33.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.985+0000 7f1a2d038640 1 --2- 172.21.15.32:0/2565843695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a281f8590 0x7f1a28145aa0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:33.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.985+0000 7f1a2d038640 1 --2- 172.21.15.32:0/2565843695 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a28159d50 0x7f1a280b4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:33.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.985+0000 7f1a2d038640 1 --2- 172.21.15.32:0/2565843695 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1a14004800 0x7f1a281430e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:33.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.985+0000 7f1a2d038640 1 -- 172.21.15.32:0/2565843695 >> 172.21.15.32:0/2565843695 conn(0x7f1a2806e800 msgr2=0x7f1a280b24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:33.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.985+0000 7f1a2d038640 1 -- 172.21.15.32:0/2565843695 shutdown_connections 2024-04-24T22:59:33.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:33.985+0000 7f1a2d038640 1 -- 172.21.15.32:0/2565843695 wait complete. 2024-04-24T22:59:33.997 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T22:59:34.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.102+0000 7f741cfd5640 1 Processor -- start 2024-04-24T22:59:34.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.102+0000 7f741cfd5640 1 -- start start 2024-04-24T22:59:34.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.102+0000 7f741cfd5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7418158dd0 0x7f741806b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:34.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.102+0000 7f741cfd5640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f741806b860 0x7f741806d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:34.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.102+0000 7f741cfd5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f741806da90 0x7f7418072580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:34.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.102+0000 7f741cfd5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f741815ef10 con 0x7f741806da90 2024-04-24T22:59:34.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.102+0000 7f741cfd5640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f741815f1f0 con 0x7f741806b860 2024-04-24T22:59:34.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.102+0000 7f741cfd5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f741815f4d0 con 0x7f7418158dd0 2024-04-24T22:59:34.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.103+0000 7f7416d9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f741806da90 0x7f7418072580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:34.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.103+0000 7f7416d9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f741806da90 0x7f7418072580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52252/0 (socket says 172.21.15.32:52252) 2024-04-24T22:59:34.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.103+0000 7f7416d9d640 1 -- 172.21.15.32:0/3219515776 learned_addr learned my addr 172.21.15.32:0/3219515776 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:34.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.103+0000 7f740ffff640 1 --2- 172.21.15.32:0/3219515776 >> v2:172.21.15.121:3300/0 conn(0x7f741806b860 0x7f741806d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:34.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.103+0000 7f741659c640 1 --2- 172.21.15.32:0/3219515776 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7418158dd0 0x7f741806b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:34.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.103+0000 7f7416d9d640 1 -- 172.21.15.32:0/3219515776 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7418158dd0 msgr2=0x7f741806b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:34.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.103+0000 7f7416d9d640 1 --2- 172.21.15.32:0/3219515776 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7418158dd0 0x7f741806b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:34.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.103+0000 7f7416d9d640 1 -- 172.21.15.32:0/3219515776 >> v2:172.21.15.121:3300/0 conn(0x7f741806b860 msgr2=0x7f741806d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:34.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.103+0000 7f7416d9d640 1 --2- 172.21.15.32:0/3219515776 >> v2:172.21.15.121:3300/0 conn(0x7f741806b860 0x7f741806d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:34.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.103+0000 7f7416d9d640 1 -- 172.21.15.32:0/3219515776 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f741815f7b0 con 0x7f741806da90 2024-04-24T22:59:34.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.103+0000 7f741659c640 1 --2- 172.21.15.32:0/3219515776 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7418158dd0 0x7f741806b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:34.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.103+0000 7f740ffff640 1 --2- 172.21.15.32:0/3219515776 >> v2:172.21.15.121:3300/0 conn(0x7f741806b860 0x7f741806d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:34.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.104+0000 7f7416d9d640 1 --2- 172.21.15.32:0/3219515776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f741806da90 0x7f7418072580 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7418070490 tx=0x7f741002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e13e3007ee9171d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:34.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.104+0000 7f7415d9b640 1 -- 172.21.15.32:0/3219515776 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7410036070 con 0x7f741806da90 2024-04-24T22:59:34.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.104+0000 7f7415d9b640 1 -- 172.21.15.32:0/3219515776 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7410031600 con 0x7f741806da90 2024-04-24T22:59:34.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.104+0000 7f7415d9b640 1 -- 172.21.15.32:0/3219515776 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74100319d0 con 0x7f741806da90 2024-04-24T22:59:34.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.104+0000 7f741cfd5640 1 -- 172.21.15.32:0/3219515776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f741806da90 msgr2=0x7f7418072580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:34.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.104+0000 7f741cfd5640 1 --2- 172.21.15.32:0/3219515776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f741806da90 0x7f7418072580 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7418070490 tx=0x7f741002a6a0 comp rx=0 tx=0).stop 2024-04-24T22:59:34.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.104+0000 7f741cfd5640 1 -- 172.21.15.32:0/3219515776 shutdown_connections 2024-04-24T22:59:34.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.104+0000 7f741cfd5640 1 --2- 172.21.15.32:0/3219515776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f741806da90 0x7f7418072580 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:34.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.104+0000 7f741cfd5640 1 --2- 172.21.15.32:0/3219515776 >> v2:172.21.15.121:3300/0 conn(0x7f741806b860 0x7f741806d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:34.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.104+0000 7f741cfd5640 1 --2- 172.21.15.32:0/3219515776 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7418158dd0 0x7f741806b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:34.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.104+0000 7f741cfd5640 1 -- 172.21.15.32:0/3219515776 >> 172.21.15.32:0/3219515776 conn(0x7f741806e800 msgr2=0x7f741806ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:34.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.104+0000 7f741cfd5640 1 -- 172.21.15.32:0/3219515776 shutdown_connections 2024-04-24T22:59:34.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.104+0000 7f741cfd5640 1 -- 172.21.15.32:0/3219515776 wait complete. 2024-04-24T22:59:34.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.106+0000 7f741cfd5640 1 Processor -- start 2024-04-24T22:59:34.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.106+0000 7f741cfd5640 1 -- start start 2024-04-24T22:59:34.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.107+0000 7f741cfd5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f741806b860 0x7f7418146f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:34.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.107+0000 7f741cfd5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f741806da90 0x7f741815db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:34.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.107+0000 7f741cfd5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7418158dd0 0x7f741815a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:34.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.107+0000 7f741cfd5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f741806ded0 con 0x7f741806b860 2024-04-24T22:59:34.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.107+0000 7f741cfd5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f741815fcd0 con 0x7f7418158dd0 2024-04-24T22:59:34.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.107+0000 7f741cfd5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f741806bfe0 con 0x7f741806da90 2024-04-24T22:59:34.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.107+0000 7f741659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f741806b860 0x7f7418146f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:34.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.107+0000 7f741659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f741806b860 0x7f7418146f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52266/0 (socket says 172.21.15.32:52266) 2024-04-24T22:59:34.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.107+0000 7f741659c640 1 -- 172.21.15.32:0/4085356986 learned_addr learned my addr 172.21.15.32:0/4085356986 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:34.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.107+0000 7f740ffff640 1 --2- 172.21.15.32:0/4085356986 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f741806da90 0x7f741815db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:34.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.107+0000 7f7416d9d640 1 --2- 172.21.15.32:0/4085356986 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7418158dd0 0x7f741815a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:34.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.107+0000 7f741659c640 1 -- 172.21.15.32:0/4085356986 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f741806da90 msgr2=0x7f741815db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:34.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.107+0000 7f741659c640 1 --2- 172.21.15.32:0/4085356986 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f741806da90 0x7f741815db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:34.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.107+0000 7f741659c640 1 -- 172.21.15.32:0/4085356986 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7418158dd0 msgr2=0x7f741815a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:34.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.107+0000 7f741659c640 1 --2- 172.21.15.32:0/4085356986 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7418158dd0 0x7f741815a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:34.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.107+0000 7f741659c640 1 -- 172.21.15.32:0/4085356986 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f741815a1e0 con 0x7f741806b860 2024-04-24T22:59:34.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.107+0000 7f741659c640 1 --2- 172.21.15.32:0/4085356986 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f741806b860 0x7f7418146f70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f740800e7e0 tx=0x7f740800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:34.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.107+0000 7f740effd640 1 -- 172.21.15.32:0/4085356986 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740800f040 con 0x7f741806b860 2024-04-24T22:59:34.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.107+0000 7f740effd640 1 -- 172.21.15.32:0/4085356986 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7408004030 con 0x7f741806b860 2024-04-24T22:59:34.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.107+0000 7f741cfd5640 1 -- 172.21.15.32:0/4085356986 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f741815cd80 con 0x7f741806b860 2024-04-24T22:59:34.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.107+0000 7f740effd640 1 -- 172.21.15.32:0/4085356986 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7408004400 con 0x7f741806b860 2024-04-24T22:59:34.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.108+0000 7f741cfd5640 1 -- 172.21.15.32:0/4085356986 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f741815d340 con 0x7f741806b860 2024-04-24T22:59:34.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.109+0000 7f741cfd5640 1 -- 172.21.15.32:0/4085356986 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7418068430 con 0x7f741806b860 2024-04-24T22:59:34.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.109+0000 7f740effd640 1 -- 172.21.15.32:0/4085356986 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f740800c6d0 con 0x7f741806b860 2024-04-24T22:59:34.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.109+0000 7f740effd640 1 --2- 172.21.15.32:0/4085356986 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f73f806be30 0x7f73f806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:34.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.109+0000 7f740effd640 1 -- 172.21.15.32:0/4085356986 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f740808ab50 con 0x7f741806b860 2024-04-24T22:59:34.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.110+0000 7f740ffff640 1 --2- 172.21.15.32:0/4085356986 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f73f806be30 0x7f73f806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:34.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.110+0000 7f740ffff640 1 --2- 172.21.15.32:0/4085356986 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f73f806be30 0x7f73f806e2f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7418151d90 tx=0x7f7400008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:34.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.114+0000 7f740effd640 1 -- 172.21.15.32:0/4085356986 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7408017020 con 0x7f741806b860 2024-04-24T22:59:34.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.294+0000 7f741cfd5640 1 -- 172.21.15.32:0/4085356986 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f74180d32c0 con 0x7f741806b860 2024-04-24T22:59:34.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.295+0000 7f740effd640 1 -- 172.21.15.32:0/4085356986 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f7408058f40 con 0x7f741806b860 2024-04-24T22:59:34.296 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-24T22:59:34.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.296+0000 7f741cfd5640 1 -- 172.21.15.32:0/4085356986 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f73f806be30 msgr2=0x7f73f806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:34.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.296+0000 7f741cfd5640 1 --2- 172.21.15.32:0/4085356986 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f73f806be30 0x7f73f806e2f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7418151d90 tx=0x7f7400008040 comp rx=0 tx=0).stop 2024-04-24T22:59:34.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.296+0000 7f741cfd5640 1 -- 172.21.15.32:0/4085356986 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f741806b860 msgr2=0x7f7418146f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:34.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.296+0000 7f741cfd5640 1 --2- 172.21.15.32:0/4085356986 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f741806b860 0x7f7418146f70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f740800e7e0 tx=0x7f740800eca0 comp rx=0 tx=0).stop 2024-04-24T22:59:34.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.297+0000 7f741cfd5640 1 -- 172.21.15.32:0/4085356986 shutdown_connections 2024-04-24T22:59:34.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.297+0000 7f741cfd5640 1 --2- 172.21.15.32:0/4085356986 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f73f806be30 0x7f73f806e2f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:34.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.297+0000 7f741cfd5640 1 --2- 172.21.15.32:0/4085356986 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7418158dd0 0x7f741815a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:34.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.297+0000 7f741cfd5640 1 --2- 172.21.15.32:0/4085356986 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f741806da90 0x7f741815db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:34.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.297+0000 7f741cfd5640 1 --2- 172.21.15.32:0/4085356986 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f741806b860 0x7f7418146f70 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:34.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.297+0000 7f741cfd5640 1 -- 172.21.15.32:0/4085356986 >> 172.21.15.32:0/4085356986 conn(0x7f741806e800 msgr2=0x7f7418066ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:34.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.297+0000 7f741cfd5640 1 -- 172.21.15.32:0/4085356986 shutdown_connections 2024-04-24T22:59:34.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.297+0000 7f741cfd5640 1 -- 172.21.15.32:0/4085356986 wait complete. 2024-04-24T22:59:34.331 INFO:tasks.cephfs_test_runner:Starting test: test_barrier (tasks.cephfs.test_full.TestClusterFull) That when an OSD epoch barrier is set on an MDS, subsequently 2024-04-24T22:59:34.332 INFO:tasks.cephfs_test_runner:test_barrier (tasks.cephfs.test_full.TestClusterFull) 2024-04-24T22:59:34.332 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:59:34.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.431+0000 7f09b7110640 1 Processor -- start 2024-04-24T22:59:34.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.431+0000 7f09b7110640 1 -- start start 2024-04-24T22:59:34.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.432+0000 7f09b7110640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f09b015b480 0x7f09b00b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:34.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.432+0000 7f09b7110640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f09b015fd60 0x7f09b006c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:34.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.432+0000 7f09b7110640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09b00cef90 con 0x7f09b006cb30 2024-04-24T22:59:34.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.432+0000 7f09b7110640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f09b00b4ff0 con 0x7f09b015fd60 2024-04-24T22:59:34.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.432+0000 7f09b7110640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f09b00b46c0 con 0x7f09b015b480 2024-04-24T22:59:34.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.432+0000 7f09affff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f09b015fd60 0x7f09b006c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:34.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.432+0000 7f09affff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f09b015fd60 0x7f09b006c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52282/0 (socket says 172.21.15.32:52282) 2024-04-24T22:59:34.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.432+0000 7f09affff640 1 -- 172.21.15.32:0/3792342563 learned_addr learned my addr 172.21.15.32:0/3792342563 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:34.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.432+0000 7f09b4eac640 1 --2- 172.21.15.32:0/3792342563 >> v2:172.21.15.121:3300/0 conn(0x7f09b015b480 0x7f09b00b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:34.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.432+0000 7f09affff640 1 -- 172.21.15.32:0/3792342563 >> v2:172.21.15.121:3300/0 conn(0x7f09b015b480 msgr2=0x7f09b00b5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:34.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.432+0000 7f09affff640 1 --2- 172.21.15.32:0/3792342563 >> v2:172.21.15.121:3300/0 conn(0x7f09b015b480 0x7f09b00b5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:34.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.432+0000 7f09affff640 1 -- 172.21.15.32:0/3792342563 >> v1:172.21.15.121:6789/0 conn(0x7f09b006cb30 legacy=0x7f09b00ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:34.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.432+0000 7f09affff640 1 -- 172.21.15.32:0/3792342563 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09b00b49a0 con 0x7f09b015fd60 2024-04-24T22:59:34.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.433+0000 7f09affff640 1 --2- 172.21.15.32:0/3792342563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09b015fd60 0x7f09b006c5f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f09b006bf90 tx=0x7f099c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a36ac120ddd5cee server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:34.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.433+0000 7f09af7fe640 1 -- 172.21.15.32:0/3792342563 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099c037070 con 0x7f09b015fd60 2024-04-24T22:59:34.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.433+0000 7f09af7fe640 1 -- 172.21.15.32:0/3792342563 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f099c032440 con 0x7f09b015fd60 2024-04-24T22:59:34.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.433+0000 7f09af7fe640 1 -- 172.21.15.32:0/3792342563 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099c032810 con 0x7f09b015fd60 2024-04-24T22:59:34.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.434+0000 7f09b7110640 1 -- 172.21.15.32:0/3792342563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09b015fd60 msgr2=0x7f09b006c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:34.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.434+0000 7f09b7110640 1 --2- 172.21.15.32:0/3792342563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09b015fd60 0x7f09b006c5f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f09b006bf90 tx=0x7f099c02a270 comp rx=0 tx=0).stop 2024-04-24T22:59:34.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.434+0000 7f09b7110640 1 -- 172.21.15.32:0/3792342563 shutdown_connections 2024-04-24T22:59:34.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.434+0000 7f09b7110640 1 --2- 172.21.15.32:0/3792342563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09b015fd60 0x7f09b006c5f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:34.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.434+0000 7f09b7110640 1 --2- 172.21.15.32:0/3792342563 >> v2:172.21.15.121:3300/0 conn(0x7f09b015b480 0x7f09b00b5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:34.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.434+0000 7f09b7110640 1 -- 172.21.15.32:0/3792342563 >> 172.21.15.32:0/3792342563 conn(0x7f09b006b110 msgr2=0x7f09b006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:34.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.434+0000 7f09b7110640 1 -- 172.21.15.32:0/3792342563 shutdown_connections 2024-04-24T22:59:34.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.434+0000 7f09b7110640 1 -- 172.21.15.32:0/3792342563 wait complete. 2024-04-24T22:59:34.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.435+0000 7f09b7110640 1 Processor -- start 2024-04-24T22:59:34.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.435+0000 7f09b7110640 1 -- start start 2024-04-24T22:59:34.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.436+0000 7f09b7110640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09b006cb30 0x7f09b015c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:34.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.436+0000 7f09b7110640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09b015b480 0x7f09b015e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:34.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.436+0000 7f09b7110640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09b015fd60 0x7f09b01efd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:34.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.436+0000 7f09b7110640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f09b00b5480 con 0x7f09b015fd60 2024-04-24T22:59:34.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.436+0000 7f09b7110640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f09b00b61a0 con 0x7f09b015b480 2024-04-24T22:59:34.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.436+0000 7f09b7110640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f09b015b860 con 0x7f09b006cb30 2024-04-24T22:59:34.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.436+0000 7f09b56ad640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09b015fd60 0x7f09b01efd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:34.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.436+0000 7f09b56ad640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09b015fd60 0x7f09b01efd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52292/0 (socket says 172.21.15.32:52292) 2024-04-24T22:59:34.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.436+0000 7f09b56ad640 1 -- 172.21.15.32:0/1981562895 learned_addr learned my addr 172.21.15.32:0/1981562895 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:34.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.436+0000 7f09affff640 1 --2- 172.21.15.32:0/1981562895 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09b015b480 0x7f09b015e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:34.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.436+0000 7f09b4eac640 1 --2- 172.21.15.32:0/1981562895 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09b006cb30 0x7f09b015c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:34.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.436+0000 7f09b56ad640 1 -- 172.21.15.32:0/1981562895 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09b006cb30 msgr2=0x7f09b015c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:34.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.436+0000 7f09b56ad640 1 --2- 172.21.15.32:0/1981562895 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09b006cb30 0x7f09b015c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:34.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.436+0000 7f09b56ad640 1 -- 172.21.15.32:0/1981562895 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09b015b480 msgr2=0x7f09b015e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:34.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.436+0000 7f09b56ad640 1 --2- 172.21.15.32:0/1981562895 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09b015b480 0x7f09b015e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:34.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.436+0000 7f09b56ad640 1 -- 172.21.15.32:0/1981562895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09b015ca20 con 0x7f09b015fd60 2024-04-24T22:59:34.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.437+0000 7f09b56ad640 1 --2- 172.21.15.32:0/1981562895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09b015fd60 0x7f09b01efd90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f09a800c6b0 tx=0x7f09a800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:34.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.437+0000 7f09ad7fa640 1 -- 172.21.15.32:0/1981562895 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a8003b70 con 0x7f09b015fd60 2024-04-24T22:59:34.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.437+0000 7f09ad7fa640 1 -- 172.21.15.32:0/1981562895 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09a8003d30 con 0x7f09b015fd60 2024-04-24T22:59:34.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.437+0000 7f09b7110640 1 -- 172.21.15.32:0/1981562895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09b01e5b30 con 0x7f09b015fd60 2024-04-24T22:59:34.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.437+0000 7f09ad7fa640 1 -- 172.21.15.32:0/1981562895 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a8005310 con 0x7f09b015fd60 2024-04-24T22:59:34.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.437+0000 7f09b7110640 1 -- 172.21.15.32:0/1981562895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09b01e60f0 con 0x7f09b015fd60 2024-04-24T22:59:34.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.438+0000 7f09b7110640 1 -- 172.21.15.32:0/1981562895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09b00730e0 con 0x7f09b015fd60 2024-04-24T22:59:34.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.438+0000 7f09ad7fa640 1 -- 172.21.15.32:0/1981562895 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f09a80054b0 con 0x7f09b015fd60 2024-04-24T22:59:34.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.438+0000 7f09ad7fa640 1 --2- 172.21.15.32:0/1981562895 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f099406be60 0x7f099406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:34.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.439+0000 7f09ad7fa640 1 -- 172.21.15.32:0/1981562895 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f09a808aaa0 con 0x7f09b015fd60 2024-04-24T22:59:34.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.439+0000 7f09b4eac640 1 --2- 172.21.15.32:0/1981562895 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f099406be60 0x7f099406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:34.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.439+0000 7f09b4eac640 1 --2- 172.21.15.32:0/1981562895 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f099406be60 0x7f099406e320 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f09a4006fd0 tx=0x7f09a40094e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:34.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.443+0000 7f09ad7fa640 1 -- 172.21.15.32:0/1981562895 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f09b00730e0 con 0x7f09b015fd60 2024-04-24T22:59:34.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.625+0000 7f09b7110640 1 -- 172.21.15.32:0/1981562895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f09b006c000 con 0x7f09b015fd60 2024-04-24T22:59:34.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.626+0000 7f09ad7fa640 1 -- 172.21.15.32:0/1981562895 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16621 (secure 0 0 0) 0x7f09a8058e90 con 0x7f09b015fd60 2024-04-24T22:59:34.628 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:59:34.628 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T22:59:05.517554+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:59:02.016639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T22:59:02.846087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:59:34.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.628+0000 7f09b7110640 1 -- 172.21.15.32:0/1981562895 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f099406be60 msgr2=0x7f099406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:34.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.628+0000 7f09b7110640 1 --2- 172.21.15.32:0/1981562895 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f099406be60 0x7f099406e320 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f09a4006fd0 tx=0x7f09a40094e0 comp rx=0 tx=0).stop 2024-04-24T22:59:34.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.628+0000 7f09b7110640 1 -- 172.21.15.32:0/1981562895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09b015fd60 msgr2=0x7f09b01efd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:34.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.628+0000 7f09b7110640 1 --2- 172.21.15.32:0/1981562895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09b015fd60 0x7f09b01efd90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f09a800c6b0 tx=0x7f09a800cb70 comp rx=0 tx=0).stop 2024-04-24T22:59:34.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.628+0000 7f09b7110640 1 -- 172.21.15.32:0/1981562895 shutdown_connections 2024-04-24T22:59:34.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.628+0000 7f09b7110640 1 --2- 172.21.15.32:0/1981562895 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f099406be60 0x7f099406e320 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:34.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.628+0000 7f09b7110640 1 --2- 172.21.15.32:0/1981562895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09b015fd60 0x7f09b01efd90 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:34.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.628+0000 7f09b7110640 1 --2- 172.21.15.32:0/1981562895 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09b015b480 0x7f09b015e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:34.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.628+0000 7f09b7110640 1 --2- 172.21.15.32:0/1981562895 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09b006cb30 0x7f09b015c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:34.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.628+0000 7f09b7110640 1 -- 172.21.15.32:0/1981562895 >> 172.21.15.32:0/1981562895 conn(0x7f09b006b110 msgr2=0x7f09b00affd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:34.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.628+0000 7f09b7110640 1 -- 172.21.15.32:0/1981562895 shutdown_connections 2024-04-24T22:59:34.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.628+0000 7f09b7110640 1 -- 172.21.15.32:0/1981562895 wait complete. 2024-04-24T22:59:34.641 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T22:59:02.016639+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T22:59:02.846087+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T22:59:34.642 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T22:59:34.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.744+0000 7fa5c774f640 1 Processor -- start 2024-04-24T22:59:34.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.744+0000 7fa5c774f640 1 -- start start 2024-04-24T22:59:34.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.744+0000 7fa5c774f640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa5c006cd00 0x7fa5c00ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:34.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.745+0000 7fa5c774f640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5c00b4520 con 0x7fa5c015cb00 2024-04-24T22:59:34.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.745+0000 7fa5c774f640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5c00b4800 con 0x7fa5c015b380 2024-04-24T22:59:34.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.745+0000 7fa5c774f640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fa5c00b4ae0 con 0x7fa5c006cd00 2024-04-24T22:59:34.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.745+0000 7fa5c5cec640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa5c015cb00 0x7fa5c00ce970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34092/0 (socket says 172.21.15.32:34092) 2024-04-24T22:59:34.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.745+0000 7fa5c5cec640 1 -- 172.21.15.32:0/2193219477 learned_addr learned my addr 172.21.15.32:0/2193219477 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:34.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.745+0000 7fa5bffff640 1 -- 172.21.15.32:0/2193219477 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2339568666 0 0) 0x7fa5c00b4520 con 0x7fa5c015cb00 2024-04-24T22:59:34.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.745+0000 7fa5c4cea640 1 --2- 172.21.15.32:0/2193219477 >> v2:172.21.15.121:3300/0 conn(0x7fa5c006cd00 0x7fa5c00ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:34.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.745+0000 7fa5bffff640 1 -- 172.21.15.32:0/2193219477 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa5a8003520 con 0x7fa5c015cb00 2024-04-24T22:59:34.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.745+0000 7fa5bffff640 1 -- 172.21.15.32:0/2193219477 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 99501606 0 0) 0x7fa5a8003520 con 0x7fa5c015cb00 2024-04-24T22:59:34.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.745+0000 7fa5bffff640 1 -- 172.21.15.32:0/2193219477 >> v2:172.21.15.121:3300/0 conn(0x7fa5c006cd00 msgr2=0x7fa5c00ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:34.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.745+0000 7fa5bffff640 1 --2- 172.21.15.32:0/2193219477 >> v2:172.21.15.121:3300/0 conn(0x7fa5c006cd00 0x7fa5c00ce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:34.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.745+0000 7fa5bffff640 1 -- 172.21.15.32:0/2193219477 >> v1:172.21.15.121:6789/0 conn(0x7fa5c015b380 legacy=0x7fa5c006c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:34.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.745+0000 7fa5bffff640 1 -- 172.21.15.32:0/2193219477 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5c00b4dc0 con 0x7fa5c015cb00 2024-04-24T22:59:34.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.745+0000 7fa5bffff640 1 -- 172.21.15.32:0/2193219477 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7fa5b8001670 con 0x7fa5c015cb00 2024-04-24T22:59:34.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.745+0000 7fa5bffff640 1 -- 172.21.15.32:0/2193219477 >> v1:172.21.15.32:6789/0 conn(0x7fa5c015cb00 legacy=0x7fa5c00ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:34.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.745+0000 7fa5bffff640 1 --2- 172.21.15.32:0/2193219477 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5a80039b0 0x7fa5a8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:34.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.745+0000 7fa5bffff640 1 --2- 172.21.15.32:0/2193219477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5a8004800 0x7fa5a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:34.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.745+0000 7fa5bffff640 1 --2- 172.21.15.32:0/2193219477 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa5a80051c0 0x7fa5a800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:34.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.745+0000 7fa5bffff640 1 -- 172.21.15.32:0/2193219477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c00b4ae0 con 0x7fa5a8004800 2024-04-24T22:59:34.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.745+0000 7fa5bffff640 1 -- 172.21.15.32:0/2193219477 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a800c160 con 0x7fa5a80039b0 2024-04-24T22:59:34.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.745+0000 7fa5bffff640 1 -- 172.21.15.32:0/2193219477 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa5a800c360 con 0x7fa5a80051c0 2024-04-24T22:59:34.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.745+0000 7fa5c4cea640 1 --2- 172.21.15.32:0/2193219477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5a8004800 0x7fa5a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:34.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.746+0000 7fa5c54eb640 1 --2- 172.21.15.32:0/2193219477 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5a80039b0 0x7fa5a8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:34.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.746+0000 7fa5c5cec640 1 --2- 172.21.15.32:0/2193219477 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa5a80051c0 0x7fa5a800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:34.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.746+0000 7fa5c4cea640 1 -- 172.21.15.32:0/2193219477 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa5a80051c0 msgr2=0x7fa5a800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:34.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.746+0000 7fa5c4cea640 1 --2- 172.21.15.32:0/2193219477 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa5a80051c0 0x7fa5a800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:34.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.746+0000 7fa5c4cea640 1 -- 172.21.15.32:0/2193219477 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5a80039b0 msgr2=0x7fa5a8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:34.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.746+0000 7fa5c4cea640 1 --2- 172.21.15.32:0/2193219477 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5a80039b0 0x7fa5a8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:34.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.746+0000 7fa5c4cea640 1 -- 172.21.15.32:0/2193219477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a8003520 con 0x7fa5a8004800 2024-04-24T22:59:34.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.746+0000 7fa5c5cec640 1 -- 172.21.15.32:0/2193219477 reap_dead start 2024-04-24T22:59:34.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.746+0000 7fa5c4cea640 1 --2- 172.21.15.32:0/2193219477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5a8004800 0x7fa5a8004c80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4009ac0 tx=0x7fa5a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c54c0de5731bcfb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:34.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.747+0000 7fa5bffff640 1 -- 172.21.15.32:0/2193219477 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b4036070 con 0x7fa5a8004800 2024-04-24T22:59:34.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.747+0000 7fa5bffff640 1 -- 172.21.15.32:0/2193219477 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5b4031400 con 0x7fa5a8004800 2024-04-24T22:59:34.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.747+0000 7fa5bffff640 1 -- 172.21.15.32:0/2193219477 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b4009690 con 0x7fa5a8004800 2024-04-24T22:59:34.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.747+0000 7fa5c774f640 1 -- 172.21.15.32:0/2193219477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5a8004800 msgr2=0x7fa5a8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:34.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.747+0000 7fa5c774f640 1 --2- 172.21.15.32:0/2193219477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5a8004800 0x7fa5a8004c80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4009ac0 tx=0x7fa5a8005830 comp rx=0 tx=0).stop 2024-04-24T22:59:34.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.747+0000 7fa5c774f640 1 -- 172.21.15.32:0/2193219477 shutdown_connections 2024-04-24T22:59:34.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.747+0000 7fa5c774f640 1 --2- 172.21.15.32:0/2193219477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5a8004800 0x7fa5a8004c80 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:34.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.747+0000 7fa5c774f640 1 -- 172.21.15.32:0/2193219477 >> 172.21.15.32:0/2193219477 conn(0x7fa5c006b110 msgr2=0x7fa5c006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:34.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.747+0000 7fa5c774f640 1 -- 172.21.15.32:0/2193219477 shutdown_connections 2024-04-24T22:59:34.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.747+0000 7fa5c774f640 1 -- 172.21.15.32:0/2193219477 wait complete. 2024-04-24T22:59:34.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.748+0000 7fa5c774f640 1 Processor -- start 2024-04-24T22:59:34.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.748+0000 7fa5c774f640 1 -- start start 2024-04-24T22:59:34.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.749+0000 7fa5c774f640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa5a8004800 0x7fa5c0152910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:34.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.749+0000 7fa5c774f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5c0155e90 0x7fa5c0156310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:34.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.749+0000 7fa5c774f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5c01409d0 0x7fa5c0140e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:34.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.749+0000 7fa5c774f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c006c5f0 con 0x7fa5c0155e90 2024-04-24T22:59:34.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.749+0000 7fa5c774f640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c015b760 con 0x7fa5c01409d0 2024-04-24T22:59:34.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.749+0000 7fa5c774f640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa5c006c770 con 0x7fa5a8004800 2024-04-24T22:59:34.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.749+0000 7fa5c4cea640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5c0155e90 0x7fa5c0156310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:34.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.749+0000 7fa5c4cea640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5c0155e90 0x7fa5c0156310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52308/0 (socket says 172.21.15.32:52308) 2024-04-24T22:59:34.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.749+0000 7fa5c4cea640 1 -- 172.21.15.32:0/1956306837 learned_addr learned my addr 172.21.15.32:0/1956306837 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:34.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.749+0000 7fa5c4cea640 1 -- 172.21.15.32:0/1956306837 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa5a8004800 msgr2=0x7fa5c0152910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:34.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.749+0000 7fa5c4cea640 1 --2- 172.21.15.32:0/1956306837 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa5a8004800 0x7fa5c0152910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:34.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.749+0000 7fa5c4cea640 1 -- 172.21.15.32:0/1956306837 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5c01409d0 msgr2=0x7fa5c0140e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:34.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.749+0000 7fa5c5cec640 1 --2- 172.21.15.32:0/1956306837 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5c01409d0 0x7fa5c0140e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:34.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.749+0000 7fa5c4cea640 1 --2- 172.21.15.32:0/1956306837 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5c01409d0 0x7fa5c0140e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:34.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.749+0000 7fa5c4cea640 1 -- 172.21.15.32:0/1956306837 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5c0153280 con 0x7fa5c0155e90 2024-04-24T22:59:34.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.750+0000 7fa5c4cea640 1 --2- 172.21.15.32:0/1956306837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5c0155e90 0x7fa5c0156310 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4009be0 tx=0x7fa5b4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:34.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.750+0000 7fa5bdffb640 1 -- 172.21.15.32:0/1956306837 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b403f020 con 0x7fa5c0155e90 2024-04-24T22:59:34.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.750+0000 7fa5bdffb640 1 -- 172.21.15.32:0/1956306837 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5b4037040 con 0x7fa5c0155e90 2024-04-24T22:59:34.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.750+0000 7fa5c774f640 1 -- 172.21.15.32:0/1956306837 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5c01f09d0 con 0x7fa5c0155e90 2024-04-24T22:59:34.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.750+0000 7fa5bdffb640 1 -- 172.21.15.32:0/1956306837 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b4036070 con 0x7fa5c0155e90 2024-04-24T22:59:34.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.750+0000 7fa5c774f640 1 -- 172.21.15.32:0/1956306837 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5c01f0f90 con 0x7fa5c0155e90 2024-04-24T22:59:34.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.751+0000 7fa5c774f640 1 -- 172.21.15.32:0/1956306837 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5c0068430 con 0x7fa5c0155e90 2024-04-24T22:59:34.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.752+0000 7fa5bdffb640 1 -- 172.21.15.32:0/1956306837 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa5b4042020 con 0x7fa5c0155e90 2024-04-24T22:59:34.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.752+0000 7fa5bdffb640 1 --2- 172.21.15.32:0/1956306837 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa5ac06be30 0x7fa5ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:34.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.752+0000 7fa5bdffb640 1 -- 172.21.15.32:0/1956306837 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa5b40aaf60 con 0x7fa5c0155e90 2024-04-24T22:59:34.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.752+0000 7fa5c54eb640 1 --2- 172.21.15.32:0/1956306837 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa5ac06be30 0x7fa5ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:34.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.753+0000 7fa5c54eb640 1 --2- 172.21.15.32:0/1956306837 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa5ac06be30 0x7fa5ac06e2f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0000f80 tx=0x7fa5b0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:34.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.756+0000 7fa5bdffb640 1 -- 172.21.15.32:0/1956306837 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa5b4049030 con 0x7fa5c0155e90 2024-04-24T22:59:34.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.941+0000 7fa5c774f640 1 -- 172.21.15.32:0/1956306837 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa5c006c000 con 0x7fa5c0155e90 2024-04-24T22:59:34.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.941+0000 7fa5bdffb640 1 -- 172.21.15.32:0/1956306837 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7fa5b40799d0 con 0x7fa5c0155e90 2024-04-24T22:59:34.943 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T22:59:34.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.943+0000 7fa5c774f640 1 -- 172.21.15.32:0/1956306837 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa5ac06be30 msgr2=0x7fa5ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:34.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.943+0000 7fa5c774f640 1 --2- 172.21.15.32:0/1956306837 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa5ac06be30 0x7fa5ac06e2f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0000f80 tx=0x7fa5b0009040 comp rx=0 tx=0).stop 2024-04-24T22:59:34.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.943+0000 7fa5c774f640 1 -- 172.21.15.32:0/1956306837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5c0155e90 msgr2=0x7fa5c0156310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:34.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.943+0000 7fa5c774f640 1 --2- 172.21.15.32:0/1956306837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5c0155e90 0x7fa5c0156310 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4009be0 tx=0x7fa5b4004730 comp rx=0 tx=0).stop 2024-04-24T22:59:34.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.943+0000 7fa5c774f640 1 -- 172.21.15.32:0/1956306837 shutdown_connections 2024-04-24T22:59:34.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.943+0000 7fa5c774f640 1 --2- 172.21.15.32:0/1956306837 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa5ac06be30 0x7fa5ac06e2f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:34.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.943+0000 7fa5c774f640 1 --2- 172.21.15.32:0/1956306837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5c0155e90 0x7fa5c0156310 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:34.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.943+0000 7fa5c774f640 1 --2- 172.21.15.32:0/1956306837 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5c01409d0 0x7fa5c0140e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:34.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.943+0000 7fa5c774f640 1 --2- 172.21.15.32:0/1956306837 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa5a8004800 0x7fa5c0152910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:34.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.943+0000 7fa5c774f640 1 -- 172.21.15.32:0/1956306837 >> 172.21.15.32:0/1956306837 conn(0x7fa5c006b110 msgr2=0x7fa5c00ce360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:34.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.943+0000 7fa5c774f640 1 -- 172.21.15.32:0/1956306837 shutdown_connections 2024-04-24T22:59:34.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:34.944+0000 7fa5c774f640 1 -- 172.21.15.32:0/1956306837 wait complete. 2024-04-24T22:59:34.956 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T22:59:35.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.058+0000 7f6f29d1b640 1 Processor -- start 2024-04-24T22:59:35.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.058+0000 7f6f29d1b640 1 -- start start 2024-04-24T22:59:35.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.059+0000 7f6f29d1b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6f240b0b80 0x7f6f2406c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:35.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.059+0000 7f6f29d1b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f240b0770 con 0x7f6f24074210 2024-04-24T22:59:35.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.059+0000 7f6f29d1b640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f240b1010 con 0x7f6f240ae140 2024-04-24T22:59:35.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.059+0000 7f6f29d1b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f6f240b1c20 con 0x7f6f240b0b80 2024-04-24T22:59:35.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.059+0000 7f6f22ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6f240b0b80 0x7f6f2406c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:35.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.059+0000 7f6f237fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6f24074210 0x7f6f240745f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34108/0 (socket says 172.21.15.32:34108) 2024-04-24T22:59:35.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.059+0000 7f6f237fe640 1 -- 172.21.15.32:0/1254819389 learned_addr learned my addr 172.21.15.32:0/1254819389 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:35.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.059+0000 7f6f22ffd640 1 -- 172.21.15.32:0/1254819389 >> v1:172.21.15.121:6789/0 conn(0x7f6f240ae140 legacy=0x7f6f240b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:35.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.059+0000 7f6f22ffd640 1 -- 172.21.15.32:0/1254819389 >> v1:172.21.15.32:6789/0 conn(0x7f6f24074210 legacy=0x7f6f240745f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:35.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.059+0000 7f6f22ffd640 1 -- 172.21.15.32:0/1254819389 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f240b12f0 con 0x7f6f240b0b80 2024-04-24T22:59:35.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.059+0000 7f6f227fc640 1 -- 172.21.15.32:0/1254819389 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3587873788 0 0) 0x7f6f240b0770 con 0x7f6f24074210 2024-04-24T22:59:35.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.060+0000 7f6f22ffd640 1 --2- 172.21.15.32:0/1254819389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f240b0b80 0x7f6f2406c480 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6f180097a0 tx=0x7f6f1802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beadeccc1e75e965 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:35.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.060+0000 7f6f227fc640 1 -- 172.21.15.32:0/1254819389 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f18037070 con 0x7f6f240b0b80 2024-04-24T22:59:35.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.060+0000 7f6f227fc640 1 -- 172.21.15.32:0/1254819389 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f18032440 con 0x7f6f240b0b80 2024-04-24T22:59:35.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.060+0000 7f6f227fc640 1 -- 172.21.15.32:0/1254819389 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f18032810 con 0x7f6f240b0b80 2024-04-24T22:59:35.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.061+0000 7f6f29d1b640 1 -- 172.21.15.32:0/1254819389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f240b0b80 msgr2=0x7f6f2406c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:35.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.061+0000 7f6f29d1b640 1 --2- 172.21.15.32:0/1254819389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f240b0b80 0x7f6f2406c480 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6f180097a0 tx=0x7f6f1802a1e0 comp rx=0 tx=0).stop 2024-04-24T22:59:35.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.061+0000 7f6f29d1b640 1 -- 172.21.15.32:0/1254819389 shutdown_connections 2024-04-24T22:59:35.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.061+0000 7f6f29d1b640 1 --2- 172.21.15.32:0/1254819389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f240b0b80 0x7f6f2406c480 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:35.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.061+0000 7f6f29d1b640 1 -- 172.21.15.32:0/1254819389 >> 172.21.15.32:0/1254819389 conn(0x7f6f240b2e60 msgr2=0x7f6f240b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:35.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.061+0000 7f6f29d1b640 1 -- 172.21.15.32:0/1254819389 shutdown_connections 2024-04-24T22:59:35.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.061+0000 7f6f29d1b640 1 -- 172.21.15.32:0/1254819389 wait complete. 2024-04-24T22:59:35.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.062+0000 7f6f29d1b640 1 Processor -- start 2024-04-24T22:59:35.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.062+0000 7f6f29d1b640 1 -- start start 2024-04-24T22:59:35.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.063+0000 7f6f29d1b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f24074210 0x7f6f240d0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:35.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.063+0000 7f6f29d1b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f240ae140 0x7f6f241550e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:35.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.063+0000 7f6f29d1b640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f240c98e0 0x7f6f2415db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:35.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.063+0000 7f6f29d1b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6f240b3ba0 con 0x7f6f240ae140 2024-04-24T22:59:35.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.063+0000 7f6f29d1b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6f240b18b0 con 0x7f6f24074210 2024-04-24T22:59:35.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.063+0000 7f6f29d1b640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f6f240b3d20 con 0x7f6f240c98e0 2024-04-24T22:59:35.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.063+0000 7f6f22ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f240ae140 0x7f6f241550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:35.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.063+0000 7f6f22ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f240ae140 0x7f6f241550e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52332/0 (socket says 172.21.15.32:52332) 2024-04-24T22:59:35.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.063+0000 7f6f22ffd640 1 -- 172.21.15.32:0/832879844 learned_addr learned my addr 172.21.15.32:0/832879844 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:35.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.063+0000 7f6f237fe640 1 --2- 172.21.15.32:0/832879844 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f24074210 0x7f6f240d0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:35.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.063+0000 7f6f22ffd640 1 -- 172.21.15.32:0/832879844 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f240c98e0 msgr2=0x7f6f2415db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:35.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.063+0000 7f6f23fff640 1 --2- 172.21.15.32:0/832879844 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f240c98e0 0x7f6f2415db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:35.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.063+0000 7f6f22ffd640 1 --2- 172.21.15.32:0/832879844 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f240c98e0 0x7f6f2415db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:35.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.063+0000 7f6f22ffd640 1 -- 172.21.15.32:0/832879844 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f24074210 msgr2=0x7f6f240d0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:35.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.063+0000 7f6f22ffd640 1 --2- 172.21.15.32:0/832879844 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f24074210 0x7f6f240d0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:35.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.063+0000 7f6f22ffd640 1 -- 172.21.15.32:0/832879844 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f24155620 con 0x7f6f240ae140 2024-04-24T22:59:35.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.063+0000 7f6f237fe640 1 --2- 172.21.15.32:0/832879844 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f24074210 0x7f6f240d0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:59:35.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.063+0000 7f6f23fff640 1 --2- 172.21.15.32:0/832879844 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f240c98e0 0x7f6f2415db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:59:35.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.064+0000 7f6f22ffd640 1 --2- 172.21.15.32:0/832879844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f240ae140 0x7f6f241550e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6f180097a0 tx=0x7f6f180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:35.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.064+0000 7f6f0bfff640 1 -- 172.21.15.32:0/832879844 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f18041070 con 0x7f6f240ae140 2024-04-24T22:59:35.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.064+0000 7f6f0bfff640 1 -- 172.21.15.32:0/832879844 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f1803c870 con 0x7f6f240ae140 2024-04-24T22:59:35.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.064+0000 7f6f29d1b640 1 -- 172.21.15.32:0/832879844 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f240ca160 con 0x7f6f240ae140 2024-04-24T22:59:35.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.064+0000 7f6f0bfff640 1 -- 172.21.15.32:0/832879844 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f18037040 con 0x7f6f240ae140 2024-04-24T22:59:35.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.064+0000 7f6f29d1b640 1 -- 172.21.15.32:0/832879844 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f240ca720 con 0x7f6f240ae140 2024-04-24T22:59:35.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.065+0000 7f6f29d1b640 1 -- 172.21.15.32:0/832879844 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f24068430 con 0x7f6f240ae140 2024-04-24T22:59:35.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.065+0000 7f6f0bfff640 1 -- 172.21.15.32:0/832879844 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6f18044070 con 0x7f6f240ae140 2024-04-24T22:59:35.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.065+0000 7f6f0bfff640 1 --2- 172.21.15.32:0/832879844 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6f1006be30 0x7f6f1006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:35.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.066+0000 7f6f0bfff640 1 -- 172.21.15.32:0/832879844 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f6f180abc20 con 0x7f6f240ae140 2024-04-24T22:59:35.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.066+0000 7f6f237fe640 1 --2- 172.21.15.32:0/832879844 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6f1006be30 0x7f6f1006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:35.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.066+0000 7f6f237fe640 1 --2- 172.21.15.32:0/832879844 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6f1006be30 0x7f6f1006e2f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6f14000f60 tx=0x7f6f14009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:35.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.070+0000 7f6f0bfff640 1 -- 172.21.15.32:0/832879844 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6f1807a6f0 con 0x7f6f240ae140 2024-04-24T22:59:35.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.256+0000 7f6f29d1b640 1 -- 172.21.15.32:0/832879844 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6f2406b220 con 0x7f6f240ae140 2024-04-24T22:59:35.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.257+0000 7f6f0bfff640 1 -- 172.21.15.32:0/832879844 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f6f18079ff0 con 0x7f6f240ae140 2024-04-24T22:59:35.259 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-24T22:59:35.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.259+0000 7f6f29d1b640 1 -- 172.21.15.32:0/832879844 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6f1006be30 msgr2=0x7f6f1006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:35.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.259+0000 7f6f29d1b640 1 --2- 172.21.15.32:0/832879844 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6f1006be30 0x7f6f1006e2f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6f14000f60 tx=0x7f6f14009040 comp rx=0 tx=0).stop 2024-04-24T22:59:35.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.259+0000 7f6f29d1b640 1 -- 172.21.15.32:0/832879844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f240ae140 msgr2=0x7f6f241550e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:35.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.259+0000 7f6f29d1b640 1 --2- 172.21.15.32:0/832879844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f240ae140 0x7f6f241550e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6f180097a0 tx=0x7f6f180046c0 comp rx=0 tx=0).stop 2024-04-24T22:59:35.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.259+0000 7f6f29d1b640 1 -- 172.21.15.32:0/832879844 shutdown_connections 2024-04-24T22:59:35.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.259+0000 7f6f29d1b640 1 --2- 172.21.15.32:0/832879844 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6f1006be30 0x7f6f1006e2f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:35.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.259+0000 7f6f29d1b640 1 --2- 172.21.15.32:0/832879844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f240ae140 0x7f6f241550e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:35.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.259+0000 7f6f29d1b640 1 --2- 172.21.15.32:0/832879844 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f240c98e0 0x7f6f2415db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:35.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.259+0000 7f6f29d1b640 1 --2- 172.21.15.32:0/832879844 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f24074210 0x7f6f240d0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:35.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.259+0000 7f6f29d1b640 1 -- 172.21.15.32:0/832879844 >> 172.21.15.32:0/832879844 conn(0x7f6f240b2e60 msgr2=0x7f6f240b20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:35.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.259+0000 7f6f29d1b640 1 -- 172.21.15.32:0/832879844 shutdown_connections 2024-04-24T22:59:35.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.259+0000 7f6f29d1b640 1 -- 172.21.15.32:0/832879844 wait complete. 2024-04-24T22:59:35.271 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T22:59:35.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.373+0000 7fad40e91640 1 Processor -- start 2024-04-24T22:59:35.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.373+0000 7fad40e91640 1 -- start start 2024-04-24T22:59:35.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.374+0000 7fad40e91640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fad3c15c5b0 0x7fad3c0b4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:35.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.374+0000 7fad40e91640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fad3c06cd00 0x7fad3c0b4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:35.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.374+0000 7fad40e91640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad3c15c990 con 0x7fad3c072cf0 2024-04-24T22:59:35.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.374+0000 7fad40e91640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fad3c072860 con 0x7fad3c06cd00 2024-04-24T22:59:35.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.374+0000 7fad40e91640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fad3c071c50 con 0x7fad3c15c5b0 2024-04-24T22:59:35.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.374+0000 7fad3ad9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fad3c06cd00 0x7fad3c0b4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:35.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.374+0000 7fad33fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fad3c072cf0 0x7fad3c06c5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34112/0 (socket says 172.21.15.32:34112) 2024-04-24T22:59:35.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.374+0000 7fad33fff640 1 -- 172.21.15.32:0/1031679960 learned_addr learned my addr 172.21.15.32:0/1031679960 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:35.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.374+0000 7fad3a59c640 1 --2- 172.21.15.32:0/1031679960 >> v2:172.21.15.121:3300/0 conn(0x7fad3c15c5b0 0x7fad3c0b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:35.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.374+0000 7fad3ad9d640 1 -- 172.21.15.32:0/1031679960 >> v2:172.21.15.121:3300/0 conn(0x7fad3c15c5b0 msgr2=0x7fad3c0b4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:35.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.374+0000 7fad3ad9d640 1 --2- 172.21.15.32:0/1031679960 >> v2:172.21.15.121:3300/0 conn(0x7fad3c15c5b0 0x7fad3c0b4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:35.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.374+0000 7fad3ad9d640 1 -- 172.21.15.32:0/1031679960 >> v1:172.21.15.32:6789/0 conn(0x7fad3c072cf0 legacy=0x7fad3c06c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:35.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.374+0000 7fad39d9b640 1 -- 172.21.15.32:0/1031679960 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2782346784 0 0) 0x7fad24001110 con 0x7fad3c072cf0 2024-04-24T22:59:35.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.374+0000 7fad3ad9d640 1 -- 172.21.15.32:0/1031679960 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad3c071f30 con 0x7fad3c06cd00 2024-04-24T22:59:35.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.375+0000 7fad3ad9d640 1 --2- 172.21.15.32:0/1031679960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad3c06cd00 0x7fad3c0b4b40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fad3c067850 tx=0x7fad3402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f43bcd32b34e3667 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:35.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.375+0000 7fad39d9b640 1 -- 172.21.15.32:0/1031679960 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad34007740 con 0x7fad3c06cd00 2024-04-24T22:59:35.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.375+0000 7fad39d9b640 1 -- 172.21.15.32:0/1031679960 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad340078e0 con 0x7fad3c06cd00 2024-04-24T22:59:35.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.375+0000 7fad39d9b640 1 -- 172.21.15.32:0/1031679960 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad34007cb0 con 0x7fad3c06cd00 2024-04-24T22:59:35.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.376+0000 7fad40e91640 1 -- 172.21.15.32:0/1031679960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad3c06cd00 msgr2=0x7fad3c0b4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:35.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.376+0000 7fad40e91640 1 --2- 172.21.15.32:0/1031679960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad3c06cd00 0x7fad3c0b4b40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fad3c067850 tx=0x7fad3402a5f0 comp rx=0 tx=0).stop 2024-04-24T22:59:35.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.376+0000 7fad40e91640 1 -- 172.21.15.32:0/1031679960 shutdown_connections 2024-04-24T22:59:35.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.376+0000 7fad40e91640 1 --2- 172.21.15.32:0/1031679960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad3c06cd00 0x7fad3c0b4b40 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:35.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.376+0000 7fad40e91640 1 --2- 172.21.15.32:0/1031679960 >> v2:172.21.15.121:3300/0 conn(0x7fad3c15c5b0 0x7fad3c0b4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:35.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.376+0000 7fad40e91640 1 -- 172.21.15.32:0/1031679960 >> 172.21.15.32:0/1031679960 conn(0x7fad3c06b110 msgr2=0x7fad3c06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:35.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.376+0000 7fad40e91640 1 -- 172.21.15.32:0/1031679960 shutdown_connections 2024-04-24T22:59:35.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.376+0000 7fad40e91640 1 -- 172.21.15.32:0/1031679960 wait complete. 2024-04-24T22:59:35.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.377+0000 7fad40e91640 1 Processor -- start 2024-04-24T22:59:35.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.377+0000 7fad40e91640 1 -- start start 2024-04-24T22:59:35.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.378+0000 7fad40e91640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fad3c06cd00 0x7fad3c141dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:35.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.378+0000 7fad40e91640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fad3c072cf0 0x7fad3c155920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:35.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.378+0000 7fad40e91640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad3c15c5b0 0x7fad3c0b37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:35.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.378+0000 7fad40e91640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fad3c0b4570 con 0x7fad3c15c5b0 2024-04-24T22:59:35.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.378+0000 7fad40e91640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fad3c0724f0 con 0x7fad3c072cf0 2024-04-24T22:59:35.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.378+0000 7fad40e91640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fad3c0b46f0 con 0x7fad3c06cd00 2024-04-24T22:59:35.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.378+0000 7fad3ad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad3c15c5b0 0x7fad3c0b37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:35.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.378+0000 7fad3ad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad3c15c5b0 0x7fad3c0b37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52356/0 (socket says 172.21.15.32:52356) 2024-04-24T22:59:35.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.378+0000 7fad3ad9d640 1 -- 172.21.15.32:0/2242675288 learned_addr learned my addr 172.21.15.32:0/2242675288 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:35.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.378+0000 7fad3a59c640 1 --2- 172.21.15.32:0/2242675288 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fad3c06cd00 0x7fad3c141dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:35.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.378+0000 7fad33fff640 1 --2- 172.21.15.32:0/2242675288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fad3c072cf0 0x7fad3c155920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:35.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.378+0000 7fad3ad9d640 1 -- 172.21.15.32:0/2242675288 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fad3c06cd00 msgr2=0x7fad3c141dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:35.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.378+0000 7fad3ad9d640 1 --2- 172.21.15.32:0/2242675288 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fad3c06cd00 0x7fad3c141dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:35.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.378+0000 7fad3ad9d640 1 -- 172.21.15.32:0/2242675288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fad3c072cf0 msgr2=0x7fad3c155920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:35.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.378+0000 7fad3ad9d640 1 --2- 172.21.15.32:0/2242675288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fad3c072cf0 0x7fad3c155920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:35.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.378+0000 7fad3ad9d640 1 -- 172.21.15.32:0/2242675288 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad3c154660 con 0x7fad3c15c5b0 2024-04-24T22:59:35.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.378+0000 7fad33fff640 1 --2- 172.21.15.32:0/2242675288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fad3c072cf0 0x7fad3c155920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:35.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.378+0000 7fad3ad9d640 1 --2- 172.21.15.32:0/2242675288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad3c15c5b0 0x7fad3c0b37a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fad34009e50 tx=0x7fad34004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:35.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.378+0000 7fad32ffd640 1 -- 172.21.15.32:0/2242675288 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad3403f070 con 0x7fad3c15c5b0 2024-04-24T22:59:35.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.378+0000 7fad32ffd640 1 -- 172.21.15.32:0/2242675288 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad34031bf0 con 0x7fad3c15c5b0 2024-04-24T22:59:35.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.378+0000 7fad40e91640 1 -- 172.21.15.32:0/2242675288 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad3c154940 con 0x7fad3c15c5b0 2024-04-24T22:59:35.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.378+0000 7fad32ffd640 1 -- 172.21.15.32:0/2242675288 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad34009450 con 0x7fad3c15c5b0 2024-04-24T22:59:35.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.379+0000 7fad40e91640 1 -- 172.21.15.32:0/2242675288 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad3c154f00 con 0x7fad3c15c5b0 2024-04-24T22:59:35.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.379+0000 7fad40e91640 1 -- 172.21.15.32:0/2242675288 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad3c068430 con 0x7fad3c15c5b0 2024-04-24T22:59:35.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.380+0000 7fad32ffd640 1 -- 172.21.15.32:0/2242675288 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fad3403b410 con 0x7fad3c15c5b0 2024-04-24T22:59:35.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.380+0000 7fad32ffd640 1 --2- 172.21.15.32:0/2242675288 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fad1c06be30 0x7fad1c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:35.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.380+0000 7fad32ffd640 1 -- 172.21.15.32:0/2242675288 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fad340ac5d0 con 0x7fad3c15c5b0 2024-04-24T22:59:35.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.381+0000 7fad3a59c640 1 --2- 172.21.15.32:0/2242675288 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fad1c06be30 0x7fad1c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:35.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.381+0000 7fad3a59c640 1 --2- 172.21.15.32:0/2242675288 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fad1c06be30 0x7fad1c06e2f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fad2c009b40 tx=0x7fad2c009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:35.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.385+0000 7fad32ffd640 1 -- 172.21.15.32:0/2242675288 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fad3407aee0 con 0x7fad3c15c5b0 2024-04-24T22:59:35.570 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T22:59:35.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.570+0000 7fad40e91640 1 -- 172.21.15.32:0/2242675288 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fad3c155270 con 0x7fad3c15c5b0 2024-04-24T22:59:35.572 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T22:59:35.573 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-24T22:59:35.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.571+0000 7fad32ffd640 1 -- 172.21.15.32:0/2242675288 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7fad3407a7e0 con 0x7fad3c15c5b0 2024-04-24T22:59:35.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.572+0000 7fad40e91640 1 -- 172.21.15.32:0/2242675288 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fad1c06be30 msgr2=0x7fad1c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:35.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.572+0000 7fad40e91640 1 --2- 172.21.15.32:0/2242675288 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fad1c06be30 0x7fad1c06e2f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fad2c009b40 tx=0x7fad2c009190 comp rx=0 tx=0).stop 2024-04-24T22:59:35.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.572+0000 7fad40e91640 1 -- 172.21.15.32:0/2242675288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad3c15c5b0 msgr2=0x7fad3c0b37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:35.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.572+0000 7fad40e91640 1 --2- 172.21.15.32:0/2242675288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad3c15c5b0 0x7fad3c0b37a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fad34009e50 tx=0x7fad34004a40 comp rx=0 tx=0).stop 2024-04-24T22:59:35.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.573+0000 7fad40e91640 1 -- 172.21.15.32:0/2242675288 shutdown_connections 2024-04-24T22:59:35.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.573+0000 7fad40e91640 1 --2- 172.21.15.32:0/2242675288 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fad1c06be30 0x7fad1c06e2f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:35.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.573+0000 7fad40e91640 1 --2- 172.21.15.32:0/2242675288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad3c15c5b0 0x7fad3c0b37a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:35.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.573+0000 7fad40e91640 1 --2- 172.21.15.32:0/2242675288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fad3c072cf0 0x7fad3c155920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:35.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.573+0000 7fad40e91640 1 --2- 172.21.15.32:0/2242675288 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fad3c06cd00 0x7fad3c141dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:35.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.573+0000 7fad40e91640 1 -- 172.21.15.32:0/2242675288 >> 172.21.15.32:0/2242675288 conn(0x7fad3c06b110 msgr2=0x7fad3c06c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:35.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.573+0000 7fad40e91640 1 -- 172.21.15.32:0/2242675288 shutdown_connections 2024-04-24T22:59:35.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.573+0000 7fad40e91640 1 -- 172.21.15.32:0/2242675288 wait complete. 2024-04-24T22:59:35.585 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_barrier' 2024-04-24T22:59:35.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.724+0000 7fbf86b63640 1 Processor -- start 2024-04-24T22:59:35.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.724+0000 7fbf86b63640 1 -- start start 2024-04-24T22:59:35.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.725+0000 7fbf86b63640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbf8006b820 0x7fbf800d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:35.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.725+0000 7fbf86b63640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf800d4210 0x7fbf800b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:35.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.725+0000 7fbf86b63640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf800d3170 con 0x7fbf8015e9b0 2024-04-24T22:59:35.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.725+0000 7fbf86b63640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fbf800d3450 con 0x7fbf8006b820 2024-04-24T22:59:35.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.725+0000 7fbf86b63640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fbf800d3730 con 0x7fbf800d4210 2024-04-24T22:59:35.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.725+0000 7fbf7ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbf8006b820 0x7fbf800d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:35.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.725+0000 7fbf7ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbf8006b820 0x7fbf800d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52370/0 (socket says 172.21.15.32:52370) 2024-04-24T22:59:35.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.725+0000 7fbf7ffff640 1 -- 172.21.15.32:0/4147267238 learned_addr learned my addr 172.21.15.32:0/4147267238 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:35.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.726+0000 7fbf85100640 1 --2- 172.21.15.32:0/4147267238 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf800d4210 0x7fbf800b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:35.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.726+0000 7fbf7ffff640 1 -- 172.21.15.32:0/4147267238 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf800d4210 msgr2=0x7fbf800b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:35.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.726+0000 7fbf7ffff640 1 --2- 172.21.15.32:0/4147267238 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf800d4210 0x7fbf800b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:35.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.726+0000 7fbf7ffff640 1 -- 172.21.15.32:0/4147267238 >> v1:172.21.15.121:6789/0 conn(0x7fbf8015e9b0 legacy=0x7fbf8006b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:35.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.726+0000 7fbf7ffff640 1 -- 172.21.15.32:0/4147267238 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf800d3a10 con 0x7fbf8006b820 2024-04-24T22:59:35.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.726+0000 7fbf85100640 1 --2- 172.21.15.32:0/4147267238 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf800d4210 0x7fbf800b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:35.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.726+0000 7fbf7ffff640 1 --2- 172.21.15.32:0/4147267238 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf8006b820 0x7fbf800d2b90 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbf74009870 tx=0x7fbf7402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17294220cec7af7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:35.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.727+0000 7fbf7f7fe640 1 -- 172.21.15.32:0/4147267238 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf74037070 con 0x7fbf8006b820 2024-04-24T22:59:35.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.727+0000 7fbf7f7fe640 1 -- 172.21.15.32:0/4147267238 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf74032440 con 0x7fbf8006b820 2024-04-24T22:59:35.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.727+0000 7fbf7f7fe640 1 -- 172.21.15.32:0/4147267238 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf74032810 con 0x7fbf8006b820 2024-04-24T22:59:35.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.727+0000 7fbf86b63640 1 -- 172.21.15.32:0/4147267238 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf8006b820 msgr2=0x7fbf800d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:35.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.727+0000 7fbf86b63640 1 --2- 172.21.15.32:0/4147267238 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf8006b820 0x7fbf800d2b90 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbf74009870 tx=0x7fbf7402a270 comp rx=0 tx=0).stop 2024-04-24T22:59:35.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.727+0000 7fbf86b63640 1 -- 172.21.15.32:0/4147267238 shutdown_connections 2024-04-24T22:59:35.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.727+0000 7fbf86b63640 1 --2- 172.21.15.32:0/4147267238 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf800d4210 0x7fbf800b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:35.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.727+0000 7fbf86b63640 1 --2- 172.21.15.32:0/4147267238 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf8006b820 0x7fbf800d2b90 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:35.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.727+0000 7fbf86b63640 1 -- 172.21.15.32:0/4147267238 >> 172.21.15.32:0/4147267238 conn(0x7fbf800d11d0 msgr2=0x7fbf800d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:35.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.727+0000 7fbf86b63640 1 -- 172.21.15.32:0/4147267238 shutdown_connections 2024-04-24T22:59:35.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.727+0000 7fbf86b63640 1 -- 172.21.15.32:0/4147267238 wait complete. 2024-04-24T22:59:35.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.728+0000 7fbf86b63640 1 Processor -- start 2024-04-24T22:59:35.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.728+0000 7fbf86b63640 1 -- start start 2024-04-24T22:59:35.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.729+0000 7fbf86b63640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf8006b820 0x7fbf801564b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:35.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.729+0000 7fbf86b63640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf800d4210 0x7fbf801611d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:35.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.729+0000 7fbf86b63640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf8015e9b0 0x7fbf800cde30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:35.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.729+0000 7fbf86b63640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbf800ba0e0 con 0x7fbf800d4210 2024-04-24T22:59:35.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.729+0000 7fbf86b63640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fbf800d2400 con 0x7fbf8015e9b0 2024-04-24T22:59:35.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.729+0000 7fbf86b63640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fbf800ba260 con 0x7fbf8006b820 2024-04-24T22:59:35.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.729+0000 7fbf7ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf800d4210 0x7fbf801611d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:35.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.729+0000 7fbf7ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf800d4210 0x7fbf801611d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52378/0 (socket says 172.21.15.32:52378) 2024-04-24T22:59:35.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.729+0000 7fbf7ffff640 1 -- 172.21.15.32:0/3596995414 learned_addr learned my addr 172.21.15.32:0/3596995414 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:35.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.730+0000 7fbf85100640 1 --2- 172.21.15.32:0/3596995414 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf8015e9b0 0x7fbf800cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:35.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.730+0000 7fbf848ff640 1 --2- 172.21.15.32:0/3596995414 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf8006b820 0x7fbf801564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:35.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.730+0000 7fbf7ffff640 1 -- 172.21.15.32:0/3596995414 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf8006b820 msgr2=0x7fbf801564b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:35.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.730+0000 7fbf7ffff640 1 --2- 172.21.15.32:0/3596995414 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf8006b820 0x7fbf801564b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:35.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.730+0000 7fbf7ffff640 1 -- 172.21.15.32:0/3596995414 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf8015e9b0 msgr2=0x7fbf800cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:35.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.730+0000 7fbf7ffff640 1 --2- 172.21.15.32:0/3596995414 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf8015e9b0 0x7fbf800cde30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:35.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.730+0000 7fbf7ffff640 1 -- 172.21.15.32:0/3596995414 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf800ccd90 con 0x7fbf800d4210 2024-04-24T22:59:35.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.730+0000 7fbf848ff640 1 --2- 172.21.15.32:0/3596995414 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf8006b820 0x7fbf801564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:59:35.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.730+0000 7fbf7ffff640 1 --2- 172.21.15.32:0/3596995414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf800d4210 0x7fbf801611d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbf74000c00 tx=0x7fbf740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:35.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.730+0000 7fbf7d7fa640 1 -- 172.21.15.32:0/3596995414 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf74041020 con 0x7fbf800d4210 2024-04-24T22:59:35.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.730+0000 7fbf7d7fa640 1 -- 172.21.15.32:0/3596995414 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf740306c0 con 0x7fbf800d4210 2024-04-24T22:59:35.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.730+0000 7fbf7d7fa640 1 -- 172.21.15.32:0/3596995414 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf74037040 con 0x7fbf800d4210 2024-04-24T22:59:35.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.730+0000 7fbf86b63640 1 -- 172.21.15.32:0/3596995414 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf800cd070 con 0x7fbf800d4210 2024-04-24T22:59:35.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.730+0000 7fbf86b63640 1 -- 172.21.15.32:0/3596995414 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf800cd910 con 0x7fbf800d4210 2024-04-24T22:59:35.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.731+0000 7fbf86b63640 1 -- 172.21.15.32:0/3596995414 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf80068430 con 0x7fbf800d4210 2024-04-24T22:59:35.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.732+0000 7fbf7d7fa640 1 -- 172.21.15.32:0/3596995414 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbf74044020 con 0x7fbf800d4210 2024-04-24T22:59:35.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.732+0000 7fbf7d7fa640 1 --2- 172.21.15.32:0/3596995414 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fbf6c06be30 0x7fbf6c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:35.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.732+0000 7fbf7d7fa640 1 -- 172.21.15.32:0/3596995414 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fbf740ab6c0 con 0x7fbf800d4210 2024-04-24T22:59:35.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.732+0000 7fbf848ff640 1 --2- 172.21.15.32:0/3596995414 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fbf6c06be30 0x7fbf6c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:35.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.733+0000 7fbf848ff640 1 --2- 172.21.15.32:0/3596995414 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fbf6c06be30 0x7fbf6c06e2f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbf78001d70 tx=0x7fbf78009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:35.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.736+0000 7fbf7d7fa640 1 -- 172.21.15.32:0/3596995414 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbf7404b030 con 0x7fbf800d4210 2024-04-24T22:59:35.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:35.987+0000 7fbf86b63640 1 -- 172.21.15.32:0/3596995414 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_barrier"]} v 0) -- 0x7fbf800cd240 con 0x7fbf800d4210 2024-04-24T22:59:36.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.615+0000 7fbf7d7fa640 1 -- 172.21.15.32:0/3596995414 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_barrier"]}]=0 v45) ==== 133+0+0 (secure 0 0 0) 0x7fbf7407a070 con 0x7fbf800d4210 2024-04-24T22:59:36.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.618+0000 7fbf86b63640 1 -- 172.21.15.32:0/3596995414 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fbf6c06be30 msgr2=0x7fbf6c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:36.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.618+0000 7fbf86b63640 1 --2- 172.21.15.32:0/3596995414 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fbf6c06be30 0x7fbf6c06e2f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbf78001d70 tx=0x7fbf78009040 comp rx=0 tx=0).stop 2024-04-24T22:59:36.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.618+0000 7fbf86b63640 1 -- 172.21.15.32:0/3596995414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf800d4210 msgr2=0x7fbf801611d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:36.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.618+0000 7fbf86b63640 1 --2- 172.21.15.32:0/3596995414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf800d4210 0x7fbf801611d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbf74000c00 tx=0x7fbf740046c0 comp rx=0 tx=0).stop 2024-04-24T22:59:36.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.618+0000 7fbf86b63640 1 -- 172.21.15.32:0/3596995414 shutdown_connections 2024-04-24T22:59:36.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.618+0000 7fbf86b63640 1 --2- 172.21.15.32:0/3596995414 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fbf6c06be30 0x7fbf6c06e2f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:36.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.618+0000 7fbf86b63640 1 --2- 172.21.15.32:0/3596995414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf800d4210 0x7fbf801611d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:36.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.618+0000 7fbf86b63640 1 --2- 172.21.15.32:0/3596995414 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf8015e9b0 0x7fbf800cde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:36.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.618+0000 7fbf86b63640 1 --2- 172.21.15.32:0/3596995414 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf8006b820 0x7fbf801564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:36.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.618+0000 7fbf86b63640 1 -- 172.21.15.32:0/3596995414 >> 172.21.15.32:0/3596995414 conn(0x7fbf800d11d0 msgr2=0x7fbf80159e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:36.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.618+0000 7fbf86b63640 1 -- 172.21.15.32:0/3596995414 shutdown_connections 2024-04-24T22:59:36.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.618+0000 7fbf86b63640 1 -- 172.21.15.32:0/3596995414 wait complete. 2024-04-24T22:59:36.633 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2024-04-24T22:59:36.753 INFO:teuthology.orchestra.run.smithi032.stdout:{"osd_objectstore":"memstore"} 2024-04-24T22:59:36.754 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2024-04-24T22:59:36.754 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-24T22:59:36.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.856+0000 7f04cba64640 1 Processor -- start 2024-04-24T22:59:36.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.857+0000 7f04cba64640 1 -- start start 2024-04-24T22:59:36.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.857+0000 7f04cba64640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f04c415e9b0 0x7f04c40d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:36.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.857+0000 7f04cba64640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f04c40d4210 0x7f04c406b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:36.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.857+0000 7f04cba64640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04c40d3170 con 0x7f04c40ba0e0 2024-04-24T22:59:36.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.857+0000 7f04cba64640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f04c40b8900 con 0x7f04c40d4210 2024-04-24T22:59:36.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.857+0000 7f04cba64640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f04c40d3450 con 0x7f04c415e9b0 2024-04-24T22:59:36.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.858+0000 7f04c8fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f04c40d4210 0x7f04c406b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:36.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.858+0000 7f04c8fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f04c40d4210 0x7f04c406b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52386/0 (socket says 172.21.15.32:52386) 2024-04-24T22:59:36.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.858+0000 7f04c8fff640 1 -- 172.21.15.32:0/3441666595 learned_addr learned my addr 172.21.15.32:0/3441666595 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:36.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.858+0000 7f04c9800640 1 --2- 172.21.15.32:0/3441666595 >> v2:172.21.15.121:3300/0 conn(0x7f04c415e9b0 0x7f04c40d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:36.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.858+0000 7f04c8fff640 1 -- 172.21.15.32:0/3441666595 >> v2:172.21.15.121:3300/0 conn(0x7f04c415e9b0 msgr2=0x7f04c40d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:36.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.858+0000 7f04c8fff640 1 --2- 172.21.15.32:0/3441666595 >> v2:172.21.15.121:3300/0 conn(0x7f04c415e9b0 0x7f04c40d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:36.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.858+0000 7f04bbfff640 1 -- 172.21.15.32:0/3441666595 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2366946822 0 0) 0x7f04c40d3170 con 0x7f04c40ba0e0 2024-04-24T22:59:36.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.858+0000 7f04c8fff640 1 -- 172.21.15.32:0/3441666595 >> v1:172.21.15.32:6789/0 conn(0x7f04c40ba0e0 legacy=0x7f04c4071730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:36.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.858+0000 7f04c8fff640 1 -- 172.21.15.32:0/3441666595 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04c40d3730 con 0x7f04c40d4210 2024-04-24T22:59:36.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.859+0000 7f04c8fff640 1 --2- 172.21.15.32:0/3441666595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04c40d4210 0x7f04c406b110 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f04c406be20 tx=0x7f04b402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c252b98200da7d65 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:36.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.859+0000 7f04bbfff640 1 -- 172.21.15.32:0/3441666595 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b4037070 con 0x7f04c40d4210 2024-04-24T22:59:36.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.859+0000 7f04bbfff640 1 -- 172.21.15.32:0/3441666595 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04b4032440 con 0x7f04c40d4210 2024-04-24T22:59:36.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.859+0000 7f04bbfff640 1 -- 172.21.15.32:0/3441666595 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b4032810 con 0x7f04c40d4210 2024-04-24T22:59:36.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.859+0000 7f04cba64640 1 -- 172.21.15.32:0/3441666595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04c40d4210 msgr2=0x7f04c406b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:36.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.859+0000 7f04cba64640 1 --2- 172.21.15.32:0/3441666595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04c40d4210 0x7f04c406b110 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f04c406be20 tx=0x7f04b402a1e0 comp rx=0 tx=0).stop 2024-04-24T22:59:36.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.860+0000 7f04cba64640 1 -- 172.21.15.32:0/3441666595 shutdown_connections 2024-04-24T22:59:36.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.860+0000 7f04cba64640 1 --2- 172.21.15.32:0/3441666595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04c40d4210 0x7f04c406b110 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:36.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.860+0000 7f04cba64640 1 --2- 172.21.15.32:0/3441666595 >> v2:172.21.15.121:3300/0 conn(0x7f04c415e9b0 0x7f04c40d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:36.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.860+0000 7f04cba64640 1 -- 172.21.15.32:0/3441666595 >> 172.21.15.32:0/3441666595 conn(0x7f04c40d11d0 msgr2=0x7f04c40d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:36.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.860+0000 7f04cba64640 1 -- 172.21.15.32:0/3441666595 shutdown_connections 2024-04-24T22:59:36.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.860+0000 7f04cba64640 1 -- 172.21.15.32:0/3441666595 wait complete. 2024-04-24T22:59:36.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.861+0000 7f04cba64640 1 Processor -- start 2024-04-24T22:59:36.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.861+0000 7f04cba64640 1 -- start start 2024-04-24T22:59:36.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.862+0000 7f04cba64640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f04c40ba0e0 0x7f04c40cba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:36.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.862+0000 7f04cba64640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f04c40d4210 0x7f04c415ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:36.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.862+0000 7f04cba64640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04c415e9b0 0x7f04c4155310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:36.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.862+0000 7f04cba64640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f04c40d2400 con 0x7f04c415e9b0 2024-04-24T22:59:36.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.862+0000 7f04cba64640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f04c40d3cf0 con 0x7f04c40ba0e0 2024-04-24T22:59:36.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.862+0000 7f04cba64640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f04c406b650 con 0x7f04c40d4210 2024-04-24T22:59:36.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.862+0000 7f04ca001640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04c415e9b0 0x7f04c4155310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:36.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.862+0000 7f04ca001640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04c415e9b0 0x7f04c4155310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52388/0 (socket says 172.21.15.32:52388) 2024-04-24T22:59:36.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.862+0000 7f04ca001640 1 -- 172.21.15.32:0/2979893963 learned_addr learned my addr 172.21.15.32:0/2979893963 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:36.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.862+0000 7f04c9800640 1 --2- 172.21.15.32:0/2979893963 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f04c40ba0e0 0x7f04c40cba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:36.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.862+0000 7f04c8fff640 1 --2- 172.21.15.32:0/2979893963 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f04c40d4210 0x7f04c415ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:36.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.862+0000 7f04ca001640 1 -- 172.21.15.32:0/2979893963 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f04c40d4210 msgr2=0x7f04c415ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:36.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.862+0000 7f04ca001640 1 --2- 172.21.15.32:0/2979893963 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f04c40d4210 0x7f04c415ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:36.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.862+0000 7f04ca001640 1 -- 172.21.15.32:0/2979893963 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f04c40ba0e0 msgr2=0x7f04c40cba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:36.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.862+0000 7f04ca001640 1 --2- 172.21.15.32:0/2979893963 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f04c40ba0e0 0x7f04c40cba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:36.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.862+0000 7f04ca001640 1 -- 172.21.15.32:0/2979893963 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04c41f7ec0 con 0x7f04c415e9b0 2024-04-24T22:59:36.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.862+0000 7f04ca001640 1 --2- 172.21.15.32:0/2979893963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04c415e9b0 0x7f04c4155310 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f04bc00c480 tx=0x7f04bc00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:36.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.862+0000 7f04b9ffb640 1 -- 172.21.15.32:0/2979893963 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04bc005480 con 0x7f04c415e9b0 2024-04-24T22:59:36.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.862+0000 7f04b9ffb640 1 -- 172.21.15.32:0/2979893963 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04bc0035f0 con 0x7f04c415e9b0 2024-04-24T22:59:36.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.862+0000 7f04b9ffb640 1 -- 172.21.15.32:0/2979893963 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04bc003a00 con 0x7f04c415e9b0 2024-04-24T22:59:36.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.862+0000 7f04cba64640 1 -- 172.21.15.32:0/2979893963 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04c41f81a0 con 0x7f04c415e9b0 2024-04-24T22:59:36.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.863+0000 7f04cba64640 1 -- 172.21.15.32:0/2979893963 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04c41f8760 con 0x7f04c415e9b0 2024-04-24T22:59:36.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.864+0000 7f04cba64640 1 -- 172.21.15.32:0/2979893963 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04c406b9e0 con 0x7f04c415e9b0 2024-04-24T22:59:36.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.864+0000 7f04b9ffb640 1 -- 172.21.15.32:0/2979893963 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f04bc003ba0 con 0x7f04c415e9b0 2024-04-24T22:59:36.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.864+0000 7f04b9ffb640 1 --2- 172.21.15.32:0/2979893963 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f04a806bfd0 0x7f04a806e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:36.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.864+0000 7f04b9ffb640 1 -- 172.21.15.32:0/2979893963 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f04bc08a920 con 0x7f04c415e9b0 2024-04-24T22:59:36.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.865+0000 7f04c9800640 1 --2- 172.21.15.32:0/2979893963 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f04a806bfd0 0x7f04a806e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:36.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.865+0000 7f04c9800640 1 --2- 172.21.15.32:0/2979893963 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f04a806bfd0 0x7f04a806e490 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f04c0011340 tx=0x7f04c000b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:36.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:36.869+0000 7f04b9ffb640 1 -- 172.21.15.32:0/2979893963 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f04c406b9e0 con 0x7f04c415e9b0 2024-04-24T22:59:37.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.054+0000 7f04cba64640 1 -- 172.21.15.32:0/2979893963 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f04c4068430 con 0x7f04c415e9b0 2024-04-24T22:59:37.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.057+0000 7f04b9ffb640 1 -- 172.21.15.32:0/2979893963 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v2)=0 v2) ==== 122+0+0 (secure 0 0 0) 0x7f04bc058d10 con 0x7f04c415e9b0 2024-04-24T22:59:37.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.059+0000 7f04cba64640 1 -- 172.21.15.32:0/2979893963 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f04a806bfd0 msgr2=0x7f04a806e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:37.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.059+0000 7f04cba64640 1 --2- 172.21.15.32:0/2979893963 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f04a806bfd0 0x7f04a806e490 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f04c0011340 tx=0x7f04c000b040 comp rx=0 tx=0).stop 2024-04-24T22:59:37.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.059+0000 7f04cba64640 1 -- 172.21.15.32:0/2979893963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04c415e9b0 msgr2=0x7f04c4155310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:37.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.059+0000 7f04cba64640 1 --2- 172.21.15.32:0/2979893963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04c415e9b0 0x7f04c4155310 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f04bc00c480 tx=0x7f04bc00c940 comp rx=0 tx=0).stop 2024-04-24T22:59:37.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.059+0000 7f04cba64640 1 -- 172.21.15.32:0/2979893963 shutdown_connections 2024-04-24T22:59:37.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.059+0000 7f04cba64640 1 --2- 172.21.15.32:0/2979893963 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f04a806bfd0 0x7f04a806e490 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:37.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.059+0000 7f04cba64640 1 --2- 172.21.15.32:0/2979893963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04c415e9b0 0x7f04c4155310 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:37.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.059+0000 7f04cba64640 1 --2- 172.21.15.32:0/2979893963 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f04c40d4210 0x7f04c415ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:37.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.059+0000 7f04cba64640 1 --2- 172.21.15.32:0/2979893963 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f04c40ba0e0 0x7f04c40cba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:37.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.059+0000 7f04cba64640 1 -- 172.21.15.32:0/2979893963 >> 172.21.15.32:0/2979893963 conn(0x7f04c40d11d0 msgr2=0x7f04c40b9450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:37.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.059+0000 7f04cba64640 1 -- 172.21.15.32:0/2979893963 shutdown_connections 2024-04-24T22:59:37.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.059+0000 7f04cba64640 1 -- 172.21.15.32:0/2979893963 wait complete. 2024-04-24T22:59:37.072 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-24T22:59:37.072 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T22:59:37.112 DEBUG:teuthology.orchestra.run.smithi121:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T22:59:37.159 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-24T22:59:37.161 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T22:59:37.161 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T22:59:37.178 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T22:59:37.179 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T22:59:37.235 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T22:59:37.235 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T22:59:37.291 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:59:37.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.430+0000 7f307dd30640 1 Processor -- start 2024-04-24T22:59:37.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.430+0000 7f307dd30640 1 -- start start 2024-04-24T22:59:37.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.431+0000 7f307dd30640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f30780b5af0 0x7f30780ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:37.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.431+0000 7f307dd30640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30780cd270 con 0x7f3078162320 2024-04-24T22:59:37.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.431+0000 7f307dd30640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3078071610 con 0x7f3078162e10 2024-04-24T22:59:37.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.431+0000 7f307dd30640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f30780b5660 con 0x7f30780b5af0 2024-04-24T22:59:37.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.431+0000 7f3076ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f30780b5af0 0x7f30780ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:37.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.431+0000 7f30777fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3078162320 0x7f3078162700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34128/0 (socket says 172.21.15.32:34128) 2024-04-24T22:59:37.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.431+0000 7f30777fe640 1 -- 172.21.15.32:0/553319337 learned_addr learned my addr 172.21.15.32:0/553319337 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:37.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.431+0000 7f3076ffd640 1 -- 172.21.15.32:0/553319337 >> v1:172.21.15.121:6789/0 conn(0x7f3078162e10 legacy=0x7f307815ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:37.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.431+0000 7f3076ffd640 1 -- 172.21.15.32:0/553319337 >> v1:172.21.15.32:6789/0 conn(0x7f3078162320 legacy=0x7f3078162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:37.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.431+0000 7f3076ffd640 1 -- 172.21.15.32:0/553319337 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30780718f0 con 0x7f30780b5af0 2024-04-24T22:59:37.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.431+0000 7f30767fc640 1 -- 172.21.15.32:0/553319337 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 609259766 0 0) 0x7f30780cd270 con 0x7f3078162320 2024-04-24T22:59:37.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.432+0000 7f3076ffd640 1 --2- 172.21.15.32:0/553319337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30780b5af0 0x7f30780ccc90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f306c0097a0 tx=0x7f306c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd3663d546d28f9d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:37.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.432+0000 7f30767fc640 1 -- 172.21.15.32:0/553319337 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306c037070 con 0x7f30780b5af0 2024-04-24T22:59:37.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.432+0000 7f30767fc640 1 -- 172.21.15.32:0/553319337 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f306c02ae50 con 0x7f30780b5af0 2024-04-24T22:59:37.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.432+0000 7f30767fc640 1 -- 172.21.15.32:0/553319337 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306c032660 con 0x7f30780b5af0 2024-04-24T22:59:37.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.433+0000 7f307dd30640 1 -- 172.21.15.32:0/553319337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30780b5af0 msgr2=0x7f30780ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:37.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.433+0000 7f307dd30640 1 --2- 172.21.15.32:0/553319337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30780b5af0 0x7f30780ccc90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f306c0097a0 tx=0x7f306c02a140 comp rx=0 tx=0).stop 2024-04-24T22:59:37.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.433+0000 7f307dd30640 1 -- 172.21.15.32:0/553319337 shutdown_connections 2024-04-24T22:59:37.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.433+0000 7f307dd30640 1 --2- 172.21.15.32:0/553319337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30780b5af0 0x7f30780ccc90 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:37.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.433+0000 7f307dd30640 1 -- 172.21.15.32:0/553319337 >> 172.21.15.32:0/553319337 conn(0x7f30780d0ae0 msgr2=0x7f30780d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:37.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.433+0000 7f307dd30640 1 -- 172.21.15.32:0/553319337 shutdown_connections 2024-04-24T22:59:37.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.433+0000 7f307dd30640 1 -- 172.21.15.32:0/553319337 wait complete. 2024-04-24T22:59:37.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.434+0000 7f307dd30640 1 Processor -- start 2024-04-24T22:59:37.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.434+0000 7f307dd30640 1 -- start start 2024-04-24T22:59:37.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.435+0000 7f307dd30640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f30780b5af0 0x7f30781464d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:37.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.435+0000 7f307dd30640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3078162320 0x7f30781612f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:37.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.435+0000 7f307dd30640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3078162e10 0x7f3078161830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:37.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.435+0000 7f307dd30640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3078071bd0 con 0x7f3078162320 2024-04-24T22:59:37.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.435+0000 7f307dd30640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3078067950 con 0x7f3078162e10 2024-04-24T22:59:37.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.435+0000 7f307dd30640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f3078071d50 con 0x7f30780b5af0 2024-04-24T22:59:37.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.435+0000 7f3076ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3078162320 0x7f30781612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:37.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.435+0000 7f3076ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3078162320 0x7f30781612f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52416/0 (socket says 172.21.15.32:52416) 2024-04-24T22:59:37.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.435+0000 7f3076ffd640 1 -- 172.21.15.32:0/2957074755 learned_addr learned my addr 172.21.15.32:0/2957074755 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:37.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.435+0000 7f3077fff640 1 --2- 172.21.15.32:0/2957074755 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3078162e10 0x7f3078161830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:37.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.435+0000 7f30777fe640 1 --2- 172.21.15.32:0/2957074755 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f30780b5af0 0x7f30781464d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:37.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.435+0000 7f3076ffd640 1 -- 172.21.15.32:0/2957074755 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f30780b5af0 msgr2=0x7f30781464d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:37.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.435+0000 7f3076ffd640 1 --2- 172.21.15.32:0/2957074755 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f30780b5af0 0x7f30781464d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:37.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.435+0000 7f3076ffd640 1 -- 172.21.15.32:0/2957074755 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3078162e10 msgr2=0x7f3078161830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:37.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.435+0000 7f3076ffd640 1 --2- 172.21.15.32:0/2957074755 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3078162e10 0x7f3078161830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:37.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.435+0000 7f3076ffd640 1 -- 172.21.15.32:0/2957074755 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30781f7ee0 con 0x7f3078162320 2024-04-24T22:59:37.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.435+0000 7f3077fff640 1 --2- 172.21.15.32:0/2957074755 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3078162e10 0x7f3078161830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:37.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.435+0000 7f3076ffd640 1 --2- 172.21.15.32:0/2957074755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3078162320 0x7f30781612f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f306c0097a0 tx=0x7f306c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:37.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.436+0000 7f305ffff640 1 -- 172.21.15.32:0/2957074755 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306c037040 con 0x7f3078162320 2024-04-24T22:59:37.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.436+0000 7f305ffff640 1 -- 172.21.15.32:0/2957074755 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f306c004030 con 0x7f3078162320 2024-04-24T22:59:37.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.436+0000 7f307dd30640 1 -- 172.21.15.32:0/2957074755 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30781f81c0 con 0x7f3078162320 2024-04-24T22:59:37.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.436+0000 7f305ffff640 1 -- 172.21.15.32:0/2957074755 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306c006e10 con 0x7f3078162320 2024-04-24T22:59:37.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.436+0000 7f307dd30640 1 -- 172.21.15.32:0/2957074755 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30781f8780 con 0x7f3078162320 2024-04-24T22:59:37.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.437+0000 7f307dd30640 1 -- 172.21.15.32:0/2957074755 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3078068430 con 0x7f3078162320 2024-04-24T22:59:37.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.437+0000 7f305ffff640 1 -- 172.21.15.32:0/2957074755 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f306c03d650 con 0x7f3078162320 2024-04-24T22:59:37.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.438+0000 7f305ffff640 1 --2- 172.21.15.32:0/2957074755 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f306406be30 0x7f306406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:37.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.438+0000 7f305ffff640 1 -- 172.21.15.32:0/2957074755 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f306c0ab680 con 0x7f3078162320 2024-04-24T22:59:37.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.438+0000 7f30777fe640 1 --2- 172.21.15.32:0/2957074755 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f306406be30 0x7f306406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:37.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.439+0000 7f30777fe640 1 --2- 172.21.15.32:0/2957074755 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f306406be30 0x7f306406e2f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3068000f60 tx=0x7f3068007d50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:37.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.442+0000 7f305ffff640 1 -- 172.21.15.32:0/2957074755 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f306c042020 con 0x7f3078162320 2024-04-24T22:59:37.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.629+0000 7f307dd30640 1 -- 172.21.15.32:0/2957074755 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3078067730 con 0x7f3078162320 2024-04-24T22:59:37.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.630+0000 7f305ffff640 1 -- 172.21.15.32:0/2957074755 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16621 (secure 0 0 0) 0x7f306c079b00 con 0x7f3078162320 2024-04-24T22:59:37.632 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:59:37.632 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T22:59:05.517554+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:59:02.016639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T22:59:02.846087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:59:37.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.633+0000 7f307dd30640 1 -- 172.21.15.32:0/2957074755 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f306406be30 msgr2=0x7f306406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:37.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.633+0000 7f307dd30640 1 --2- 172.21.15.32:0/2957074755 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f306406be30 0x7f306406e2f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3068000f60 tx=0x7f3068007d50 comp rx=0 tx=0).stop 2024-04-24T22:59:37.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.633+0000 7f307dd30640 1 -- 172.21.15.32:0/2957074755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3078162320 msgr2=0x7f30781612f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:37.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.633+0000 7f307dd30640 1 --2- 172.21.15.32:0/2957074755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3078162320 0x7f30781612f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f306c0097a0 tx=0x7f306c0046c0 comp rx=0 tx=0).stop 2024-04-24T22:59:37.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.633+0000 7f307dd30640 1 -- 172.21.15.32:0/2957074755 shutdown_connections 2024-04-24T22:59:37.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.633+0000 7f307dd30640 1 --2- 172.21.15.32:0/2957074755 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f306406be30 0x7f306406e2f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:37.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.633+0000 7f307dd30640 1 --2- 172.21.15.32:0/2957074755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3078162320 0x7f30781612f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:37.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.633+0000 7f307dd30640 1 --2- 172.21.15.32:0/2957074755 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3078162e10 0x7f3078161830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:37.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.633+0000 7f307dd30640 1 --2- 172.21.15.32:0/2957074755 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f30780b5af0 0x7f30781464d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:37.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.633+0000 7f307dd30640 1 -- 172.21.15.32:0/2957074755 >> 172.21.15.32:0/2957074755 conn(0x7f30780d0ae0 msgr2=0x7f307815d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:37.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.633+0000 7f307dd30640 1 -- 172.21.15.32:0/2957074755 shutdown_connections 2024-04-24T22:59:37.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.633+0000 7f307dd30640 1 -- 172.21.15.32:0/2957074755 wait complete. 2024-04-24T22:59:37.645 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T22:59:02.016639+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T22:59:02.846087+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T22:59:37.646 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T22:59:37.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.747+0000 7f791c072640 1 Processor -- start 2024-04-24T22:59:37.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.748+0000 7f791c072640 1 -- start start 2024-04-24T22:59:37.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.748+0000 7f791c072640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f791406b820 0x7f79140d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:37.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.748+0000 7f791c072640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79140d3170 con 0x7f791415e9b0 2024-04-24T22:59:37.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.748+0000 7f791c072640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79140d3450 con 0x7f79140d4210 2024-04-24T22:59:37.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.748+0000 7f791c072640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f79140d3730 con 0x7f791406b820 2024-04-24T22:59:37.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.748+0000 7f7919e0e640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f791415e9b0 0x7f791406b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34138/0 (socket says 172.21.15.32:34138) 2024-04-24T22:59:37.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.749+0000 7f7919e0e640 1 -- 172.21.15.32:0/4292515274 learned_addr learned my addr 172.21.15.32:0/4292515274 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:37.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.749+0000 7f791960d640 1 --2- 172.21.15.32:0/4292515274 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f791406b820 0x7f79140d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:37.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.749+0000 7f7918e0c640 1 -- 172.21.15.32:0/4292515274 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1561131476 0 0) 0x7f79140d3170 con 0x7f791415e9b0 2024-04-24T22:59:37.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.749+0000 7f7918e0c640 1 -- 172.21.15.32:0/4292515274 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7900003520 con 0x7f791415e9b0 2024-04-24T22:59:37.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.749+0000 7f7918e0c640 1 -- 172.21.15.32:0/4292515274 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2717477810 0 0) 0x7f79140d3450 con 0x7f79140d4210 2024-04-24T22:59:37.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.749+0000 7f7918e0c640 1 -- 172.21.15.32:0/4292515274 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f79140d3170 con 0x7f79140d4210 2024-04-24T22:59:37.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.749+0000 7f7918e0c640 1 -- 172.21.15.32:0/4292515274 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3193356203 0 0) 0x7f7900003520 con 0x7f791415e9b0 2024-04-24T22:59:37.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.749+0000 7f7918e0c640 1 -- 172.21.15.32:0/4292515274 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f791406b820 msgr2=0x7f79140d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:37.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.749+0000 7f7918e0c640 1 --2- 172.21.15.32:0/4292515274 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f791406b820 0x7f79140d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:37.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.749+0000 7f7918e0c640 1 -- 172.21.15.32:0/4292515274 >> v1:172.21.15.121:6789/0 conn(0x7f79140d4210 legacy=0x7f79140b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:37.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.749+0000 7f7918e0c640 1 -- 172.21.15.32:0/4292515274 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79140d3a10 con 0x7f791415e9b0 2024-04-24T22:59:37.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.749+0000 7f7918e0c640 1 -- 172.21.15.32:0/4292515274 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f7910001670 con 0x7f791415e9b0 2024-04-24T22:59:37.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.749+0000 7f7918e0c640 1 -- 172.21.15.32:0/4292515274 >> v1:172.21.15.32:6789/0 conn(0x7f791415e9b0 legacy=0x7f791406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:37.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.749+0000 7f7918e0c640 1 --2- 172.21.15.32:0/4292515274 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f791406b820 0x7f7900003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:37.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.749+0000 7f7918e0c640 1 --2- 172.21.15.32:0/4292515274 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79000040d0 0x7f7900004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:37.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.749+0000 7f7918e0c640 1 --2- 172.21.15.32:0/4292515274 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7900004d40 0x7f79000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:37.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.749+0000 7f7918e0c640 1 -- 172.21.15.32:0/4292515274 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f79140d3730 con 0x7f7900004d40 2024-04-24T22:59:37.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.749+0000 7f7918e0c640 1 -- 172.21.15.32:0/4292515274 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f790000baa0 con 0x7f79000040d0 2024-04-24T22:59:37.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.749+0000 7f7918e0c640 1 -- 172.21.15.32:0/4292515274 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f790000bca0 con 0x7f791406b820 2024-04-24T22:59:37.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.749+0000 7f791a60f640 1 --2- 172.21.15.32:0/4292515274 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7900004d40 0x7f79000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:37.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.749+0000 7f791960d640 1 --2- 172.21.15.32:0/4292515274 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79000040d0 0x7f7900004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:37.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.749+0000 7f7919e0e640 1 --2- 172.21.15.32:0/4292515274 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f791406b820 0x7f7900003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:37.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.750+0000 7f791a60f640 1 -- 172.21.15.32:0/4292515274 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f791406b820 msgr2=0x7f7900003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:37.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.750+0000 7f791a60f640 1 --2- 172.21.15.32:0/4292515274 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f791406b820 0x7f7900003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:37.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.750+0000 7f791a60f640 1 -- 172.21.15.32:0/4292515274 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79000040d0 msgr2=0x7f7900004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:37.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.750+0000 7f791a60f640 1 --2- 172.21.15.32:0/4292515274 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79000040d0 0x7f7900004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:37.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.750+0000 7f791a60f640 1 -- 172.21.15.32:0/4292515274 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7900003520 con 0x7f7900004d40 2024-04-24T22:59:37.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.750+0000 7f791a60f640 1 --2- 172.21.15.32:0/4292515274 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7900004d40 0x7f79000051c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f791406bdf0 tx=0x7f7900005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f09ca206b433f252 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:37.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.750+0000 7f7918e0c640 1 -- 172.21.15.32:0/4292515274 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790c001bf0 con 0x7f7900004d40 2024-04-24T22:59:37.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.750+0000 7f7918e0c640 1 -- 172.21.15.32:0/4292515274 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f790c003b70 con 0x7f7900004d40 2024-04-24T22:59:37.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.750+0000 7f7918e0c640 1 -- 172.21.15.32:0/4292515274 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790c0050d0 con 0x7f7900004d40 2024-04-24T22:59:37.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.751+0000 7f791c072640 1 -- 172.21.15.32:0/4292515274 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7900004d40 msgr2=0x7f79000051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:37.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.751+0000 7f791c072640 1 --2- 172.21.15.32:0/4292515274 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7900004d40 0x7f79000051c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f791406bdf0 tx=0x7f7900005830 comp rx=0 tx=0).stop 2024-04-24T22:59:37.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.751+0000 7f791a60f640 1 -- 172.21.15.32:0/4292515274 reap_dead start 2024-04-24T22:59:37.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.751+0000 7f791c072640 1 -- 172.21.15.32:0/4292515274 shutdown_connections 2024-04-24T22:59:37.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.751+0000 7f791c072640 1 -- 172.21.15.32:0/4292515274 >> 172.21.15.32:0/4292515274 conn(0x7f79140d11d0 msgr2=0x7f79140d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:37.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.751+0000 7f791c072640 1 -- 172.21.15.32:0/4292515274 shutdown_connections 2024-04-24T22:59:37.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.751+0000 7f791c072640 1 -- 172.21.15.32:0/4292515274 wait complete. 2024-04-24T22:59:37.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.752+0000 7f791c072640 1 Processor -- start 2024-04-24T22:59:37.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.752+0000 7f791c072640 1 -- start start 2024-04-24T22:59:37.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.753+0000 7f791c072640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7900004d40 0x7f791414fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:37.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.753+0000 7f791c072640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7914161900 0x7f79140ccb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:37.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.753+0000 7f791c072640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7914153c90 0x7f7914154140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:37.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.753+0000 7f791c072640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f79140ba0e0 con 0x7f7914153c90 2024-04-24T22:59:37.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.753+0000 7f791c072640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f79140d2400 con 0x7f7900004d40 2024-04-24T22:59:37.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.753+0000 7f791c072640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f79140ba260 con 0x7f7914161900 2024-04-24T22:59:37.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.753+0000 7f791a60f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7914153c90 0x7f7914154140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:37.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.753+0000 7f791a60f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7914153c90 0x7f7914154140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52424/0 (socket says 172.21.15.32:52424) 2024-04-24T22:59:37.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.753+0000 7f791a60f640 1 -- 172.21.15.32:0/1321668911 learned_addr learned my addr 172.21.15.32:0/1321668911 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:37.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.753+0000 7f7919e0e640 1 --2- 172.21.15.32:0/1321668911 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7900004d40 0x7f791414fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:37.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.753+0000 7f791960d640 1 --2- 172.21.15.32:0/1321668911 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7914161900 0x7f79140ccb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:37.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.753+0000 7f791a60f640 1 -- 172.21.15.32:0/1321668911 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7914161900 msgr2=0x7f79140ccb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:37.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.753+0000 7f791a60f640 1 --2- 172.21.15.32:0/1321668911 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7914161900 0x7f79140ccb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:37.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.754+0000 7f791a60f640 1 -- 172.21.15.32:0/1321668911 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7900004d40 msgr2=0x7f791414fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:37.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.754+0000 7f791a60f640 1 --2- 172.21.15.32:0/1321668911 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7900004d40 0x7f791414fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:37.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.754+0000 7f791a60f640 1 -- 172.21.15.32:0/1321668911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79140cd460 con 0x7f7914153c90 2024-04-24T22:59:37.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.754+0000 7f791a60f640 1 --2- 172.21.15.32:0/1321668911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7914153c90 0x7f7914154140 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f790c033040 tx=0x7f790c001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:37.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.754+0000 7f790a7fc640 1 -- 172.21.15.32:0/1321668911 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790c001bf0 con 0x7f7914153c90 2024-04-24T22:59:37.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.754+0000 7f790a7fc640 1 -- 172.21.15.32:0/1321668911 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f790c043070 con 0x7f7914153c90 2024-04-24T22:59:37.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.754+0000 7f790a7fc640 1 -- 172.21.15.32:0/1321668911 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790c005610 con 0x7f7914153c90 2024-04-24T22:59:37.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.754+0000 7f791c072640 1 -- 172.21.15.32:0/1321668911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79140cd740 con 0x7f7914153c90 2024-04-24T22:59:37.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.754+0000 7f791c072640 1 -- 172.21.15.32:0/1321668911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79140cdd00 con 0x7f7914153c90 2024-04-24T22:59:37.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.755+0000 7f791c072640 1 -- 172.21.15.32:0/1321668911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7914068430 con 0x7f7914153c90 2024-04-24T22:59:37.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.755+0000 7f790a7fc640 1 -- 172.21.15.32:0/1321668911 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f790c03a040 con 0x7f7914153c90 2024-04-24T22:59:37.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.756+0000 7f790a7fc640 1 --2- 172.21.15.32:0/1321668911 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f78f806be30 0x7f78f806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:37.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.756+0000 7f790a7fc640 1 -- 172.21.15.32:0/1321668911 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f790c0ab2a0 con 0x7f7914153c90 2024-04-24T22:59:37.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.756+0000 7f7919e0e640 1 --2- 172.21.15.32:0/1321668911 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f78f806be30 0x7f78f806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:37.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.757+0000 7f7919e0e640 1 --2- 172.21.15.32:0/1321668911 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f78f806be30 0x7f78f806e2f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7910001eb0 tx=0x7f7910001e20 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:37.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.760+0000 7f790a7fc640 1 -- 172.21.15.32:0/1321668911 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f790c079c40 con 0x7f7914153c90 2024-04-24T22:59:37.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.940+0000 7f791c072640 1 -- 172.21.15.32:0/1321668911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7914155f90 con 0x7f7914153c90 2024-04-24T22:59:37.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.941+0000 7f790a7fc640 1 -- 172.21.15.32:0/1321668911 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f790c079540 con 0x7f7914153c90 2024-04-24T22:59:37.943 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T22:59:37.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.943+0000 7f791c072640 1 -- 172.21.15.32:0/1321668911 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f78f806be30 msgr2=0x7f78f806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:37.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.943+0000 7f791c072640 1 --2- 172.21.15.32:0/1321668911 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f78f806be30 0x7f78f806e2f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7910001eb0 tx=0x7f7910001e20 comp rx=0 tx=0).stop 2024-04-24T22:59:37.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.943+0000 7f791c072640 1 -- 172.21.15.32:0/1321668911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7914153c90 msgr2=0x7f7914154140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:37.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.943+0000 7f791c072640 1 --2- 172.21.15.32:0/1321668911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7914153c90 0x7f7914154140 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f790c033040 tx=0x7f790c001650 comp rx=0 tx=0).stop 2024-04-24T22:59:37.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.943+0000 7f791c072640 1 -- 172.21.15.32:0/1321668911 shutdown_connections 2024-04-24T22:59:37.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.943+0000 7f791c072640 1 --2- 172.21.15.32:0/1321668911 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f78f806be30 0x7f78f806e2f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:37.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.943+0000 7f791c072640 1 --2- 172.21.15.32:0/1321668911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7914153c90 0x7f7914154140 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:37.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.943+0000 7f791c072640 1 --2- 172.21.15.32:0/1321668911 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7914161900 0x7f79140ccb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:37.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.943+0000 7f791c072640 1 --2- 172.21.15.32:0/1321668911 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7900004d40 0x7f791414fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:37.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.943+0000 7f791c072640 1 -- 172.21.15.32:0/1321668911 >> 172.21.15.32:0/1321668911 conn(0x7f79140d11d0 msgr2=0x7f79140b8be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:37.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.943+0000 7f791c072640 1 -- 172.21.15.32:0/1321668911 shutdown_connections 2024-04-24T22:59:37.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:37.943+0000 7f791c072640 1 -- 172.21.15.32:0/1321668911 wait complete. 2024-04-24T22:59:37.955 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T22:59:38.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.058+0000 7f3334b28640 1 Processor -- start 2024-04-24T22:59:38.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.058+0000 7f3334b28640 1 -- start start 2024-04-24T22:59:38.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.059+0000 7f3334b28640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f33300d4210 0x7f33300d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:38.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.059+0000 7f3334b28640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33300d3170 con 0x7f333006b820 2024-04-24T22:59:38.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.059+0000 7f3334b28640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33300d3450 con 0x7f333015e9b0 2024-04-24T22:59:38.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.059+0000 7f3334b28640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f33300d3730 con 0x7f33300d4210 2024-04-24T22:59:38.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.059+0000 7f3327fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f333006b820 0x7f33300b78f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34148/0 (socket says 172.21.15.32:34148) 2024-04-24T22:59:38.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.059+0000 7f3327fff640 1 -- 172.21.15.32:0/520674758 learned_addr learned my addr 172.21.15.32:0/520674758 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:38.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.059+0000 7f332ed9d640 1 --2- 172.21.15.32:0/520674758 >> v2:172.21.15.121:3300/0 conn(0x7f33300d4210 0x7f33300d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:38.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.059+0000 7f332dd9b640 1 -- 172.21.15.32:0/520674758 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1054722031 0 0) 0x7f33300d3170 con 0x7f333006b820 2024-04-24T22:59:38.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.059+0000 7f332dd9b640 1 -- 172.21.15.32:0/520674758 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f331c003520 con 0x7f333006b820 2024-04-24T22:59:38.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.059+0000 7f332ed9d640 1 -- 172.21.15.32:0/520674758 >> v1:172.21.15.121:6789/0 conn(0x7f333015e9b0 legacy=0x7f333006b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:38.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.059+0000 7f332ed9d640 1 -- 172.21.15.32:0/520674758 >> v1:172.21.15.32:6789/0 conn(0x7f333006b820 legacy=0x7f33300b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:38.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.059+0000 7f332ed9d640 1 -- 172.21.15.32:0/520674758 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33300d3a10 con 0x7f33300d4210 2024-04-24T22:59:38.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.060+0000 7f332ed9d640 1 --2- 172.21.15.32:0/520674758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33300d4210 0x7f33300d2b90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3328009d90 tx=0x7f332802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=658e9f9d56e92acc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:38.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.062+0000 7f332dd9b640 1 -- 172.21.15.32:0/520674758 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3328007740 con 0x7f33300d4210 2024-04-24T22:59:38.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.062+0000 7f332dd9b640 1 -- 172.21.15.32:0/520674758 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33280078e0 con 0x7f33300d4210 2024-04-24T22:59:38.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.062+0000 7f332dd9b640 1 -- 172.21.15.32:0/520674758 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3328007cb0 con 0x7f33300d4210 2024-04-24T22:59:38.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.063+0000 7f3334b28640 1 -- 172.21.15.32:0/520674758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33300d4210 msgr2=0x7f33300d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:38.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.063+0000 7f3334b28640 1 --2- 172.21.15.32:0/520674758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33300d4210 0x7f33300d2b90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3328009d90 tx=0x7f332802a660 comp rx=0 tx=0).stop 2024-04-24T22:59:38.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.063+0000 7f3334b28640 1 -- 172.21.15.32:0/520674758 shutdown_connections 2024-04-24T22:59:38.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.063+0000 7f3334b28640 1 --2- 172.21.15.32:0/520674758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33300d4210 0x7f33300d2b90 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:38.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.063+0000 7f3334b28640 1 -- 172.21.15.32:0/520674758 >> 172.21.15.32:0/520674758 conn(0x7f33300d11d0 msgr2=0x7f33300d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:38.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.063+0000 7f3334b28640 1 -- 172.21.15.32:0/520674758 shutdown_connections 2024-04-24T22:59:38.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.063+0000 7f3334b28640 1 -- 172.21.15.32:0/520674758 wait complete. 2024-04-24T22:59:38.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.064+0000 7f3334b28640 1 Processor -- start 2024-04-24T22:59:38.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.064+0000 7f3334b28640 1 -- start start 2024-04-24T22:59:38.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.065+0000 7f3334b28640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f333006b820 0x7f33301ee0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:38.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.065+0000 7f3334b28640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f33300d4210 0x7f33300ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:38.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.065+0000 7f3334b28640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f333015e9b0 0x7f33300caa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:38.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.065+0000 7f3334b28640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f33300ba0e0 con 0x7f333006b820 2024-04-24T22:59:38.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.065+0000 7f3334b28640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f33300d2400 con 0x7f333015e9b0 2024-04-24T22:59:38.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.065+0000 7f3334b28640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f33300ba260 con 0x7f33300d4210 2024-04-24T22:59:38.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.065+0000 7f332e59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f333006b820 0x7f33301ee0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:38.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.065+0000 7f332e59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f333006b820 0x7f33301ee0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52432/0 (socket says 172.21.15.32:52432) 2024-04-24T22:59:38.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.065+0000 7f332e59c640 1 -- 172.21.15.32:0/3696587588 learned_addr learned my addr 172.21.15.32:0/3696587588 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:38.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.065+0000 7f332ed9d640 1 --2- 172.21.15.32:0/3696587588 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f333015e9b0 0x7f33300caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:38.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.065+0000 7f3327fff640 1 --2- 172.21.15.32:0/3696587588 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f33300d4210 0x7f33300ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:38.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.065+0000 7f332e59c640 1 -- 172.21.15.32:0/3696587588 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f33300d4210 msgr2=0x7f33300ca4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:38.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.065+0000 7f332e59c640 1 --2- 172.21.15.32:0/3696587588 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f33300d4210 0x7f33300ca4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:38.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.065+0000 7f332e59c640 1 -- 172.21.15.32:0/3696587588 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f333015e9b0 msgr2=0x7f33300caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:38.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.065+0000 7f332e59c640 1 --2- 172.21.15.32:0/3696587588 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f333015e9b0 0x7f33300caa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:38.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.065+0000 7f332e59c640 1 -- 172.21.15.32:0/3696587588 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33301f5100 con 0x7f333006b820 2024-04-24T22:59:38.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.065+0000 7f332e59c640 1 --2- 172.21.15.32:0/3696587588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f333006b820 0x7f33301ee0e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f332000c3c0 tx=0x7f332000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:38.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.065+0000 7f3326ffd640 1 -- 172.21.15.32:0/3696587588 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33200052d0 con 0x7f333006b820 2024-04-24T22:59:38.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.066+0000 7f3326ffd640 1 -- 172.21.15.32:0/3696587588 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3320005490 con 0x7f333006b820 2024-04-24T22:59:38.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.066+0000 7f3334b28640 1 -- 172.21.15.32:0/3696587588 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33301f4340 con 0x7f333006b820 2024-04-24T22:59:38.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.066+0000 7f3326ffd640 1 -- 172.21.15.32:0/3696587588 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3320007260 con 0x7f333006b820 2024-04-24T22:59:38.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.066+0000 7f3334b28640 1 -- 172.21.15.32:0/3696587588 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33301f4900 con 0x7f333006b820 2024-04-24T22:59:38.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.067+0000 7f3334b28640 1 -- 172.21.15.32:0/3696587588 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3330068430 con 0x7f333006b820 2024-04-24T22:59:38.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.067+0000 7f3326ffd640 1 -- 172.21.15.32:0/3696587588 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3320007400 con 0x7f333006b820 2024-04-24T22:59:38.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.068+0000 7f3326ffd640 1 --2- 172.21.15.32:0/3696587588 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f331006be30 0x7f331006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:38.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.068+0000 7f3326ffd640 1 -- 172.21.15.32:0/3696587588 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f332008a290 con 0x7f333006b820 2024-04-24T22:59:38.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.068+0000 7f3327fff640 1 --2- 172.21.15.32:0/3696587588 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f331006be30 0x7f331006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:38.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.068+0000 7f3327fff640 1 --2- 172.21.15.32:0/3696587588 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f331006be30 0x7f331006e2f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f33301f76f0 tx=0x7f3318007d30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:38.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.072+0000 7f3326ffd640 1 -- 172.21.15.32:0/3696587588 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f33200585f0 con 0x7f333006b820 2024-04-24T22:59:38.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.255+0000 7f3334b28640 1 -- 172.21.15.32:0/3696587588 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f33300ce6a0 con 0x7f333006b820 2024-04-24T22:59:38.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.255+0000 7f3326ffd640 1 -- 172.21.15.32:0/3696587588 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f332005c8f0 con 0x7f333006b820 2024-04-24T22:59:38.257 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-24T22:59:38.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.257+0000 7f3334b28640 1 -- 172.21.15.32:0/3696587588 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f331006be30 msgr2=0x7f331006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:38.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.257+0000 7f3334b28640 1 --2- 172.21.15.32:0/3696587588 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f331006be30 0x7f331006e2f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f33301f76f0 tx=0x7f3318007d30 comp rx=0 tx=0).stop 2024-04-24T22:59:38.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.257+0000 7f3334b28640 1 -- 172.21.15.32:0/3696587588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f333006b820 msgr2=0x7f33301ee0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:38.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.257+0000 7f3334b28640 1 --2- 172.21.15.32:0/3696587588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f333006b820 0x7f33301ee0e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f332000c3c0 tx=0x7f332000c880 comp rx=0 tx=0).stop 2024-04-24T22:59:38.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.257+0000 7f3334b28640 1 -- 172.21.15.32:0/3696587588 shutdown_connections 2024-04-24T22:59:38.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.257+0000 7f3334b28640 1 --2- 172.21.15.32:0/3696587588 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f331006be30 0x7f331006e2f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:38.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.257+0000 7f3334b28640 1 --2- 172.21.15.32:0/3696587588 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f333015e9b0 0x7f33300caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:38.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.257+0000 7f3334b28640 1 --2- 172.21.15.32:0/3696587588 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f33300d4210 0x7f33300ca4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:38.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.257+0000 7f3334b28640 1 --2- 172.21.15.32:0/3696587588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f333006b820 0x7f33301ee0e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:38.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.257+0000 7f3334b28640 1 -- 172.21.15.32:0/3696587588 >> 172.21.15.32:0/3696587588 conn(0x7f33300d11d0 msgr2=0x7f33300d15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:38.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.257+0000 7f3334b28640 1 -- 172.21.15.32:0/3696587588 shutdown_connections 2024-04-24T22:59:38.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.257+0000 7f3334b28640 1 -- 172.21.15.32:0/3696587588 wait complete. 2024-04-24T22:59:38.270 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T22:59:38.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.372+0000 7fc1551c1640 1 Processor -- start 2024-04-24T22:59:38.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.372+0000 7fc1551c1640 1 -- start start 2024-04-24T22:59:38.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.373+0000 7fc1551c1640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc15015b380 0x7fc15006c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:38.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.373+0000 7fc1551c1640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1500b4520 con 0x7fc1500ceb40 2024-04-24T22:59:38.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.373+0000 7fc1551c1640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1500b4800 con 0x7fc15015cb00 2024-04-24T22:59:38.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.373+0000 7fc1551c1640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc1500cef90 con 0x7fc15015b380 2024-04-24T22:59:38.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.373+0000 7fc14f59e640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc1500ceb40 0x7fc1500b55c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34156/0 (socket says 172.21.15.32:34156) 2024-04-24T22:59:38.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.373+0000 7fc14f59e640 1 -- 172.21.15.32:0/3940557173 learned_addr learned my addr 172.21.15.32:0/3940557173 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:38.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.373+0000 7fc14ed9d640 1 --2- 172.21.15.32:0/3940557173 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc15015b380 0x7fc15006c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:38.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.373+0000 7fc14e59c640 1 -- 172.21.15.32:0/3940557173 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4224584399 0 0) 0x7fc1500b4520 con 0x7fc1500ceb40 2024-04-24T22:59:38.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.373+0000 7fc14e59c640 1 -- 172.21.15.32:0/3940557173 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc13c003520 con 0x7fc1500ceb40 2024-04-24T22:59:38.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.373+0000 7fc14e59c640 1 -- 172.21.15.32:0/3940557173 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3302959034 0 0) 0x7fc13c003520 con 0x7fc1500ceb40 2024-04-24T22:59:38.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.373+0000 7fc14e59c640 1 -- 172.21.15.32:0/3940557173 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc15015b380 msgr2=0x7fc15006c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:38.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.373+0000 7fc14e59c640 1 --2- 172.21.15.32:0/3940557173 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc15015b380 0x7fc15006c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:38.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.373+0000 7fc14e59c640 1 -- 172.21.15.32:0/3940557173 >> v1:172.21.15.121:6789/0 conn(0x7fc15015cb00 legacy=0x7fc1500ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:38.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.373+0000 7fc14e59c640 1 -- 172.21.15.32:0/3940557173 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1500b4ae0 con 0x7fc1500ceb40 2024-04-24T22:59:38.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.373+0000 7fc14e59c640 1 -- 172.21.15.32:0/3940557173 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7fc1480019f0 con 0x7fc1500ceb40 2024-04-24T22:59:38.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.373+0000 7fc14e59c640 1 -- 172.21.15.32:0/3940557173 >> v1:172.21.15.32:6789/0 conn(0x7fc1500ceb40 legacy=0x7fc1500b55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:38.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.373+0000 7fc14e59c640 1 --2- 172.21.15.32:0/3940557173 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc15015b380 0x7fc13c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:38.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.373+0000 7fc14e59c640 1 --2- 172.21.15.32:0/3940557173 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc13c003f60 0x7fc13c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:38.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.373+0000 7fc14e59c640 1 --2- 172.21.15.32:0/3940557173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc13c004d40 0x7fc13c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:38.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.373+0000 7fc14e59c640 1 -- 172.21.15.32:0/3940557173 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc1500cef90 con 0x7fc13c004d40 2024-04-24T22:59:38.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.373+0000 7fc14e59c640 1 -- 172.21.15.32:0/3940557173 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc13c00bb50 con 0x7fc13c003f60 2024-04-24T22:59:38.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.373+0000 7fc14e59c640 1 -- 172.21.15.32:0/3940557173 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc13c00bd50 con 0x7fc15015b380 2024-04-24T22:59:38.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.374+0000 7fc14f59e640 1 --2- 172.21.15.32:0/3940557173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc13c004d40 0x7fc13c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:38.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.374+0000 7fc14ed9d640 1 --2- 172.21.15.32:0/3940557173 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc15015b380 0x7fc13c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:38.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.374+0000 7fc147fff640 1 --2- 172.21.15.32:0/3940557173 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc13c003f60 0x7fc13c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:38.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.374+0000 7fc14f59e640 1 -- 172.21.15.32:0/3940557173 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc15015b380 msgr2=0x7fc13c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:38.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.374+0000 7fc14f59e640 1 --2- 172.21.15.32:0/3940557173 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc15015b380 0x7fc13c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:38.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.374+0000 7fc14f59e640 1 -- 172.21.15.32:0/3940557173 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc13c003f60 msgr2=0x7fc13c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:38.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.374+0000 7fc14f59e640 1 --2- 172.21.15.32:0/3940557173 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc13c003f60 0x7fc13c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:38.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.374+0000 7fc14f59e640 1 -- 172.21.15.32:0/3940557173 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc13c003520 con 0x7fc13c004d40 2024-04-24T22:59:38.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.375+0000 7fc14f59e640 1 --2- 172.21.15.32:0/3940557173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc13c004d40 0x7fc13c0051c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc14800a830 tx=0x7fc13c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3426a8835b68b94e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:38.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.375+0000 7fc14e59c640 1 -- 172.21.15.32:0/3940557173 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc14800a480 con 0x7fc13c004d40 2024-04-24T22:59:38.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.375+0000 7fc14e59c640 1 -- 172.21.15.32:0/3940557173 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc14802f070 con 0x7fc13c004d40 2024-04-24T22:59:38.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.375+0000 7fc14e59c640 1 -- 172.21.15.32:0/3940557173 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc14802bcb0 con 0x7fc13c004d40 2024-04-24T22:59:38.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.375+0000 7fc1551c1640 1 -- 172.21.15.32:0/3940557173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc13c004d40 msgr2=0x7fc13c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:38.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.375+0000 7fc1551c1640 1 --2- 172.21.15.32:0/3940557173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc13c004d40 0x7fc13c0051c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc14800a830 tx=0x7fc13c005830 comp rx=0 tx=0).stop 2024-04-24T22:59:38.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.375+0000 7fc14f59e640 1 -- 172.21.15.32:0/3940557173 reap_dead start 2024-04-24T22:59:38.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.375+0000 7fc1551c1640 1 -- 172.21.15.32:0/3940557173 shutdown_connections 2024-04-24T22:59:38.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.375+0000 7fc1551c1640 1 -- 172.21.15.32:0/3940557173 >> 172.21.15.32:0/3940557173 conn(0x7fc15006b110 msgr2=0x7fc15006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:38.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.375+0000 7fc1551c1640 1 -- 172.21.15.32:0/3940557173 shutdown_connections 2024-04-24T22:59:38.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.375+0000 7fc1551c1640 1 -- 172.21.15.32:0/3940557173 wait complete. 2024-04-24T22:59:38.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.377+0000 7fc1551c1640 1 Processor -- start 2024-04-24T22:59:38.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.377+0000 7fc1551c1640 1 -- start start 2024-04-24T22:59:38.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.377+0000 7fc1551c1640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc13c004d40 0x7fc150144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:38.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.377+0000 7fc1551c1640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc15014ba60 0x7fc15014db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:38.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.377+0000 7fc1551c1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc15014bee0 0x7fc150141230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:38.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.377+0000 7fc1551c1640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc15015b760 con 0x7fc15014bee0 2024-04-24T22:59:38.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.377+0000 7fc1551c1640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc1500b50a0 con 0x7fc15014ba60 2024-04-24T22:59:38.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.377+0000 7fc1551c1640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc1500b5cd0 con 0x7fc13c004d40 2024-04-24T22:59:38.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.377+0000 7fc14f59e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc15014bee0 0x7fc150141230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:38.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.377+0000 7fc14f59e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc15014bee0 0x7fc150141230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52456/0 (socket says 172.21.15.32:52456) 2024-04-24T22:59:38.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.377+0000 7fc14f59e640 1 -- 172.21.15.32:0/2192485775 learned_addr learned my addr 172.21.15.32:0/2192485775 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:38.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.377+0000 7fc147fff640 1 --2- 172.21.15.32:0/2192485775 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc15014ba60 0x7fc15014db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:38.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.378+0000 7fc14ed9d640 1 --2- 172.21.15.32:0/2192485775 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc13c004d40 0x7fc150144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:38.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.378+0000 7fc14f59e640 1 -- 172.21.15.32:0/2192485775 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc13c004d40 msgr2=0x7fc150144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:38.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.378+0000 7fc14f59e640 1 --2- 172.21.15.32:0/2192485775 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc13c004d40 0x7fc150144cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:38.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.378+0000 7fc14f59e640 1 -- 172.21.15.32:0/2192485775 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc15014ba60 msgr2=0x7fc15014db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:38.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.378+0000 7fc14f59e640 1 --2- 172.21.15.32:0/2192485775 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc15014ba60 0x7fc15014db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:38.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.378+0000 7fc14f59e640 1 -- 172.21.15.32:0/2192485775 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc150145520 con 0x7fc15014bee0 2024-04-24T22:59:38.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.378+0000 7fc14ed9d640 1 --2- 172.21.15.32:0/2192485775 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc13c004d40 0x7fc150144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:38.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.378+0000 7fc14f59e640 1 --2- 172.21.15.32:0/2192485775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc15014bee0 0x7fc150141230 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc14802b500 tx=0x7fc1500b46f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:38.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.378+0000 7fc1477fe640 1 -- 172.21.15.32:0/2192485775 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc14800a480 con 0x7fc15014bee0 2024-04-24T22:59:38.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.378+0000 7fc1477fe640 1 -- 172.21.15.32:0/2192485775 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc148001a40 con 0x7fc15014bee0 2024-04-24T22:59:38.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.378+0000 7fc1551c1640 1 -- 172.21.15.32:0/2192485775 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc15014cd80 con 0x7fc15014bee0 2024-04-24T22:59:38.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.378+0000 7fc1477fe640 1 -- 172.21.15.32:0/2192485775 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc148001ff0 con 0x7fc15014bee0 2024-04-24T22:59:38.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.378+0000 7fc1551c1640 1 -- 172.21.15.32:0/2192485775 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc15014d340 con 0x7fc15014bee0 2024-04-24T22:59:38.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.379+0000 7fc1551c1640 1 -- 172.21.15.32:0/2192485775 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc150068a30 con 0x7fc15014bee0 2024-04-24T22:59:38.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.379+0000 7fc1477fe640 1 -- 172.21.15.32:0/2192485775 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc14802cd10 con 0x7fc15014bee0 2024-04-24T22:59:38.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.380+0000 7fc1477fe640 1 --2- 172.21.15.32:0/2192485775 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc13006be30 0x7fc13006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:38.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.380+0000 7fc1477fe640 1 -- 172.21.15.32:0/2192485775 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc1480ab030 con 0x7fc15014bee0 2024-04-24T22:59:38.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.380+0000 7fc14ed9d640 1 --2- 172.21.15.32:0/2192485775 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc13006be30 0x7fc13006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:38.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.381+0000 7fc14ed9d640 1 --2- 172.21.15.32:0/2192485775 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc13006be30 0x7fc13006e2f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc150072ff0 tx=0x7fc140008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:38.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.384+0000 7fc1477fe640 1 -- 172.21.15.32:0/2192485775 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc150068a30 con 0x7fc15014bee0 2024-04-24T22:59:38.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.569+0000 7fc1551c1640 1 -- 172.21.15.32:0/2192485775 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc150169110 con 0x7fc15014bee0 2024-04-24T22:59:38.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.569+0000 7fc1477fe640 1 -- 172.21.15.32:0/2192485775 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7fc1480792f0 con 0x7fc15014bee0 2024-04-24T22:59:38.571 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-24T22:59:38.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.571+0000 7fc1551c1640 1 -- 172.21.15.32:0/2192485775 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc13006be30 msgr2=0x7fc13006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:38.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.571+0000 7fc1551c1640 1 --2- 172.21.15.32:0/2192485775 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc13006be30 0x7fc13006e2f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc150072ff0 tx=0x7fc140008040 comp rx=0 tx=0).stop 2024-04-24T22:59:38.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.571+0000 7fc1551c1640 1 -- 172.21.15.32:0/2192485775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc15014bee0 msgr2=0x7fc150141230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:38.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.571+0000 7fc1551c1640 1 --2- 172.21.15.32:0/2192485775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc15014bee0 0x7fc150141230 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc14802b500 tx=0x7fc1500b46f0 comp rx=0 tx=0).stop 2024-04-24T22:59:38.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.571+0000 7fc1551c1640 1 -- 172.21.15.32:0/2192485775 shutdown_connections 2024-04-24T22:59:38.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.571+0000 7fc1551c1640 1 --2- 172.21.15.32:0/2192485775 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc13006be30 0x7fc13006e2f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:38.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.571+0000 7fc1551c1640 1 --2- 172.21.15.32:0/2192485775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc15014bee0 0x7fc150141230 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:38.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.571+0000 7fc1551c1640 1 --2- 172.21.15.32:0/2192485775 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc15014ba60 0x7fc15014db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:38.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.571+0000 7fc1551c1640 1 --2- 172.21.15.32:0/2192485775 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc13c004d40 0x7fc150144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:38.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.571+0000 7fc1551c1640 1 -- 172.21.15.32:0/2192485775 >> 172.21.15.32:0/2192485775 conn(0x7fc15006b110 msgr2=0x7fc1500cd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:38.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.571+0000 7fc1551c1640 1 -- 172.21.15.32:0/2192485775 shutdown_connections 2024-04-24T22:59:38.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.571+0000 7fc1551c1640 1 -- 172.21.15.32:0/2192485775 wait complete. 2024-04-24T22:59:38.584 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T22:59:38.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.685+0000 7fa5384dd640 1 Processor -- start 2024-04-24T22:59:38.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.685+0000 7fa5384dd640 1 -- start start 2024-04-24T22:59:38.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.686+0000 7fa5384dd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa53015b480 0x7fa5300b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:38.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.686+0000 7fa5384dd640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa53015fd60 0x7fa53006c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:38.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.686+0000 7fa5384dd640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa53006cb30 0x7fa5300ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:38.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.686+0000 7fa5384dd640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fa5300b4ff0 con 0x7fa53015b480 2024-04-24T22:59:38.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.686+0000 7fa5384dd640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fa5300b46c0 con 0x7fa53015fd60 2024-04-24T22:59:38.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.686+0000 7fa5384dd640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa5300b49a0 con 0x7fa53006cb30 2024-04-24T22:59:38.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.686+0000 7fa536279640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa53015b480 0x7fa5300b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:38.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.686+0000 7fa536279640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa53015b480 0x7fa5300b5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52470/0 (socket says 172.21.15.32:52470) 2024-04-24T22:59:38.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.686+0000 7fa536279640 1 -- 172.21.15.32:0/1054289661 learned_addr learned my addr 172.21.15.32:0/1054289661 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:38.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.686+0000 7fa536a7a640 1 --2- 172.21.15.32:0/1054289661 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa53006cb30 0x7fa5300ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:38.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.686+0000 7fa536279640 1 -- 172.21.15.32:0/1054289661 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa53006cb30 msgr2=0x7fa5300ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:38.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.686+0000 7fa536279640 1 --2- 172.21.15.32:0/1054289661 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa53006cb30 0x7fa5300ccc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:38.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.686+0000 7fa535a78640 1 --2- 172.21.15.32:0/1054289661 >> v2:172.21.15.121:3300/0 conn(0x7fa53015fd60 0x7fa53006c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:38.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.686+0000 7fa536279640 1 -- 172.21.15.32:0/1054289661 >> v2:172.21.15.121:3300/0 conn(0x7fa53015fd60 msgr2=0x7fa53006c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:38.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.686+0000 7fa536279640 1 --2- 172.21.15.32:0/1054289661 >> v2:172.21.15.121:3300/0 conn(0x7fa53015fd60 0x7fa53006c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:38.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.686+0000 7fa536279640 1 -- 172.21.15.32:0/1054289661 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5300b4c80 con 0x7fa53015b480 2024-04-24T22:59:38.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.687+0000 7fa536279640 1 --2- 172.21.15.32:0/1054289661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa53015b480 0x7fa5300b5c60 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa53006ee10 tx=0x7fa52c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f058bbc161be131b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:38.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.687+0000 7fa535277640 1 -- 172.21.15.32:0/1054289661 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52c037070 con 0x7fa53015b480 2024-04-24T22:59:38.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.687+0000 7fa535277640 1 -- 172.21.15.32:0/1054289661 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa52c032440 con 0x7fa53015b480 2024-04-24T22:59:38.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.687+0000 7fa535277640 1 -- 172.21.15.32:0/1054289661 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52c032810 con 0x7fa53015b480 2024-04-24T22:59:38.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.688+0000 7fa5384dd640 1 -- 172.21.15.32:0/1054289661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa53015b480 msgr2=0x7fa5300b5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:38.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.688+0000 7fa5384dd640 1 --2- 172.21.15.32:0/1054289661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa53015b480 0x7fa5300b5c60 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa53006ee10 tx=0x7fa52c02a1e0 comp rx=0 tx=0).stop 2024-04-24T22:59:38.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.688+0000 7fa5384dd640 1 -- 172.21.15.32:0/1054289661 shutdown_connections 2024-04-24T22:59:38.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.688+0000 7fa5384dd640 1 --2- 172.21.15.32:0/1054289661 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa53006cb30 0x7fa5300ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:38.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.688+0000 7fa5384dd640 1 --2- 172.21.15.32:0/1054289661 >> v2:172.21.15.121:3300/0 conn(0x7fa53015fd60 0x7fa53006c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:38.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.688+0000 7fa5384dd640 1 --2- 172.21.15.32:0/1054289661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa53015b480 0x7fa5300b5c60 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:38.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.688+0000 7fa5384dd640 1 -- 172.21.15.32:0/1054289661 >> 172.21.15.32:0/1054289661 conn(0x7fa53006b110 msgr2=0x7fa53006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:38.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.688+0000 7fa5384dd640 1 -- 172.21.15.32:0/1054289661 shutdown_connections 2024-04-24T22:59:38.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.688+0000 7fa5384dd640 1 -- 172.21.15.32:0/1054289661 wait complete. 2024-04-24T22:59:38.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.689+0000 7fa5384dd640 1 Processor -- start 2024-04-24T22:59:38.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.689+0000 7fa5384dd640 1 -- start start 2024-04-24T22:59:38.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.690+0000 7fa5384dd640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa53006cb30 0x7fa5301f54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:38.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.690+0000 7fa5384dd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa53015b480 0x7fa5301f1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:38.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.690+0000 7fa5384dd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa53015fd60 0x7fa5301596a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:38.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.690+0000 7fa5384dd640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa53015b860 con 0x7fa53015b480 2024-04-24T22:59:38.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.690+0000 7fa5384dd640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa5300b5480 con 0x7fa53015fd60 2024-04-24T22:59:38.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.690+0000 7fa5384dd640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa5301f2a60 con 0x7fa53006cb30 2024-04-24T22:59:38.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.690+0000 7fa535a78640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa53015b480 0x7fa5301f1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:38.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.690+0000 7fa535a78640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa53015b480 0x7fa5301f1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52476/0 (socket says 172.21.15.32:52476) 2024-04-24T22:59:38.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.690+0000 7fa535a78640 1 -- 172.21.15.32:0/876866642 learned_addr learned my addr 172.21.15.32:0/876866642 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:38.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.690+0000 7fa536279640 1 --2- 172.21.15.32:0/876866642 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa53006cb30 0x7fa5301f54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:38.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.690+0000 7fa536a7a640 1 --2- 172.21.15.32:0/876866642 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa53015fd60 0x7fa5301596a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:38.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.690+0000 7fa535a78640 1 -- 172.21.15.32:0/876866642 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa53006cb30 msgr2=0x7fa5301f54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:38.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.690+0000 7fa535a78640 1 --2- 172.21.15.32:0/876866642 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa53006cb30 0x7fa5301f54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:38.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.690+0000 7fa535a78640 1 -- 172.21.15.32:0/876866642 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa53015fd60 msgr2=0x7fa5301596a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:38.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.690+0000 7fa535a78640 1 --2- 172.21.15.32:0/876866642 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa53015fd60 0x7fa5301596a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:38.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.690+0000 7fa535a78640 1 -- 172.21.15.32:0/876866642 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5301f5a30 con 0x7fa53015b480 2024-04-24T22:59:38.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.690+0000 7fa536a7a640 1 --2- 172.21.15.32:0/876866642 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa53015fd60 0x7fa5301596a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:38.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.690+0000 7fa535a78640 1 --2- 172.21.15.32:0/876866642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa53015b480 0x7fa5301f1f10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa52000c7f0 tx=0x7fa52000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:38.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.691+0000 7fa526ffd640 1 -- 172.21.15.32:0/876866642 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa520015070 con 0x7fa53015b480 2024-04-24T22:59:38.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.691+0000 7fa526ffd640 1 -- 172.21.15.32:0/876866642 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa520004220 con 0x7fa53015b480 2024-04-24T22:59:38.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.691+0000 7fa526ffd640 1 -- 172.21.15.32:0/876866642 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa520007620 con 0x7fa53015b480 2024-04-24T22:59:38.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.691+0000 7fa5384dd640 1 -- 172.21.15.32:0/876866642 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5301f4470 con 0x7fa53015b480 2024-04-24T22:59:38.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.691+0000 7fa5384dd640 1 -- 172.21.15.32:0/876866642 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5301f4d10 con 0x7fa53015b480 2024-04-24T22:59:38.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.692+0000 7fa5384dd640 1 -- 172.21.15.32:0/876866642 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa530068430 con 0x7fa53015b480 2024-04-24T22:59:38.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.692+0000 7fa526ffd640 1 -- 172.21.15.32:0/876866642 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa5200079d0 con 0x7fa53015b480 2024-04-24T22:59:38.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.693+0000 7fa526ffd640 1 --2- 172.21.15.32:0/876866642 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa51406be30 0x7fa51406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:38.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.693+0000 7fa526ffd640 1 -- 172.21.15.32:0/876866642 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa52008b0d0 con 0x7fa53015b480 2024-04-24T22:59:38.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.693+0000 7fa536279640 1 --2- 172.21.15.32:0/876866642 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa51406be30 0x7fa51406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:38.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.693+0000 7fa536279640 1 --2- 172.21.15.32:0/876866642 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa51406be30 0x7fa51406e2f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa52c034000 tx=0x7fa52c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:38.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.697+0000 7fa526ffd640 1 -- 172.21.15.32:0/876866642 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa520059430 con 0x7fa53015b480 2024-04-24T22:59:38.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.896+0000 7fa5384dd640 1 -- 172.21.15.32:0/876866642 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa53006c000 con 0x7fa53015b480 2024-04-24T22:59:38.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.896+0000 7fa526ffd640 1 -- 172.21.15.32:0/876866642 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7fa52005d730 con 0x7fa53015b480 2024-04-24T22:59:38.898 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2024-04-24T22:59:38.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.898+0000 7fa5384dd640 1 -- 172.21.15.32:0/876866642 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa51406be30 msgr2=0x7fa51406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:38.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.898+0000 7fa5384dd640 1 --2- 172.21.15.32:0/876866642 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa51406be30 0x7fa51406e2f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa52c034000 tx=0x7fa52c02e000 comp rx=0 tx=0).stop 2024-04-24T22:59:38.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.898+0000 7fa5384dd640 1 -- 172.21.15.32:0/876866642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa53015b480 msgr2=0x7fa5301f1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:38.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.898+0000 7fa5384dd640 1 --2- 172.21.15.32:0/876866642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa53015b480 0x7fa5301f1f10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa52000c7f0 tx=0x7fa52000ccb0 comp rx=0 tx=0).stop 2024-04-24T22:59:38.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.898+0000 7fa5384dd640 1 -- 172.21.15.32:0/876866642 shutdown_connections 2024-04-24T22:59:38.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.898+0000 7fa5384dd640 1 --2- 172.21.15.32:0/876866642 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa51406be30 0x7fa51406e2f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:38.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.898+0000 7fa5384dd640 1 --2- 172.21.15.32:0/876866642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa53015b480 0x7fa5301f1f10 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:38.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.898+0000 7fa5384dd640 1 --2- 172.21.15.32:0/876866642 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa53015fd60 0x7fa5301596a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:38.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.898+0000 7fa5384dd640 1 --2- 172.21.15.32:0/876866642 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa53006cb30 0x7fa5301f54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:38.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.898+0000 7fa5384dd640 1 -- 172.21.15.32:0/876866642 >> 172.21.15.32:0/876866642 conn(0x7fa53006b110 msgr2=0x7fa5300b3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:38.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.898+0000 7fa5384dd640 1 -- 172.21.15.32:0/876866642 shutdown_connections 2024-04-24T22:59:38.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:38.898+0000 7fa5384dd640 1 -- 172.21.15.32:0/876866642 wait complete. 2024-04-24T22:59:38.911 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-24T22:59:38.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:59:38.911 DEBUG:teuthology.orchestra.run.smithi032:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-24T22:59:38.937 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2024-04-24T22:59:38.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:59:38.937 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-24T22:59:39.003 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T22:59:39.003 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T22:59:39.022 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T22:59:39.023 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T22:59:39.080 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T22:59:39.080 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T22:59:39.139 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:59:39.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.238+0000 7f234b02a640 1 Processor -- start 2024-04-24T22:59:39.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.238+0000 7f234b02a640 1 -- start start 2024-04-24T22:59:39.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.239+0000 7f234b02a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2344162320 0x7f23440ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:39.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.239+0000 7f234b02a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2344162e10 0x7f234415ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:39.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.239+0000 7f234b02a640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23440cd270 con 0x7f23440b5af0 2024-04-24T22:59:39.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.239+0000 7f234b02a640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f23440b5660 con 0x7f2344162320 2024-04-24T22:59:39.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.239+0000 7f234b02a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f2344071610 con 0x7f2344162e10 2024-04-24T22:59:39.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.239+0000 7f2348dc6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2344162320 0x7f23440ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:39.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.239+0000 7f2343fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f23440b5af0 0x7f2344162700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34172/0 (socket says 172.21.15.32:34172) 2024-04-24T22:59:39.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.239+0000 7f2343fff640 1 -- 172.21.15.32:0/4292667680 learned_addr learned my addr 172.21.15.32:0/4292667680 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:39.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.239+0000 7f23495c7640 1 --2- 172.21.15.32:0/4292667680 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2344162e10 0x7f234415ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:39.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.240+0000 7f2348dc6640 1 -- 172.21.15.32:0/4292667680 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2344162e10 msgr2=0x7f234415ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:39.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.240+0000 7f2348dc6640 1 --2- 172.21.15.32:0/4292667680 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2344162e10 0x7f234415ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:39.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.240+0000 7f2348dc6640 1 -- 172.21.15.32:0/4292667680 >> v1:172.21.15.32:6789/0 conn(0x7f23440b5af0 legacy=0x7f2344162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:39.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.240+0000 7f2348dc6640 1 -- 172.21.15.32:0/4292667680 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23440718f0 con 0x7f2344162320 2024-04-24T22:59:39.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.240+0000 7f2348dc6640 1 --2- 172.21.15.32:0/4292667680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2344162320 0x7f23440ccc90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f23340097a0 tx=0x7f233402a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=312bd90da6bf5c23 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:39.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.240+0000 7f23437fe640 1 -- 172.21.15.32:0/4292667680 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2334037070 con 0x7f2344162320 2024-04-24T22:59:39.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.240+0000 7f23437fe640 1 -- 172.21.15.32:0/4292667680 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f233402adb0 con 0x7f2344162320 2024-04-24T22:59:39.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.240+0000 7f23437fe640 1 -- 172.21.15.32:0/4292667680 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2334032660 con 0x7f2344162320 2024-04-24T22:59:39.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.241+0000 7f234b02a640 1 -- 172.21.15.32:0/4292667680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2344162320 msgr2=0x7f23440ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:39.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.241+0000 7f234b02a640 1 --2- 172.21.15.32:0/4292667680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2344162320 0x7f23440ccc90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f23340097a0 tx=0x7f233402a060 comp rx=0 tx=0).stop 2024-04-24T22:59:39.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.241+0000 7f234b02a640 1 -- 172.21.15.32:0/4292667680 shutdown_connections 2024-04-24T22:59:39.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.241+0000 7f234b02a640 1 --2- 172.21.15.32:0/4292667680 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2344162e10 0x7f234415ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:39.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.241+0000 7f234b02a640 1 --2- 172.21.15.32:0/4292667680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2344162320 0x7f23440ccc90 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:39.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.241+0000 7f234b02a640 1 -- 172.21.15.32:0/4292667680 >> 172.21.15.32:0/4292667680 conn(0x7f23440d0ae0 msgr2=0x7f23440d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:39.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.241+0000 7f234b02a640 1 -- 172.21.15.32:0/4292667680 shutdown_connections 2024-04-24T22:59:39.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.241+0000 7f234b02a640 1 -- 172.21.15.32:0/4292667680 wait complete. 2024-04-24T22:59:39.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.242+0000 7f234b02a640 1 Processor -- start 2024-04-24T22:59:39.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.242+0000 7f234b02a640 1 -- start start 2024-04-24T22:59:39.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.243+0000 7f234b02a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f23440b5af0 0x7f23441e9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:39.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.243+0000 7f234b02a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2344162320 0x7f23440aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:39.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.243+0000 7f234b02a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2344162e10 0x7f23441f81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:39.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.243+0000 7f234b02a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2344071bd0 con 0x7f2344162e10 2024-04-24T22:59:39.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.243+0000 7f234b02a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2344067950 con 0x7f2344162320 2024-04-24T22:59:39.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.243+0000 7f234b02a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f2344071d50 con 0x7f23440b5af0 2024-04-24T22:59:39.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.243+0000 7f23495c7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2344162e10 0x7f23441f81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:39.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.243+0000 7f23495c7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2344162e10 0x7f23441f81e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52492/0 (socket says 172.21.15.32:52492) 2024-04-24T22:59:39.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.243+0000 7f23495c7640 1 -- 172.21.15.32:0/286406762 learned_addr learned my addr 172.21.15.32:0/286406762 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:39.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.243+0000 7f23495c7640 1 -- 172.21.15.32:0/286406762 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f23440b5af0 msgr2=0x7f23441e9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:39.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.243+0000 7f23495c7640 1 --2- 172.21.15.32:0/286406762 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f23440b5af0 0x7f23441e9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:39.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.243+0000 7f23495c7640 1 -- 172.21.15.32:0/286406762 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2344162320 msgr2=0x7f23440aeb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:39.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.243+0000 7f23495c7640 1 --2- 172.21.15.32:0/286406762 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2344162320 0x7f23440aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:39.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.244+0000 7f23495c7640 1 -- 172.21.15.32:0/286406762 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23441e9e00 con 0x7f2344162e10 2024-04-24T22:59:39.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.244+0000 7f23495c7640 1 --2- 172.21.15.32:0/286406762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2344162e10 0x7f23441f81e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f233c007e40 tx=0x7f233c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:39.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.244+0000 7f23417fa640 1 -- 172.21.15.32:0/286406762 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f233c014620 con 0x7f2344162e10 2024-04-24T22:59:39.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.244+0000 7f23417fa640 1 -- 172.21.15.32:0/286406762 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f233c019070 con 0x7f2344162e10 2024-04-24T22:59:39.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.244+0000 7f234b02a640 1 -- 172.21.15.32:0/286406762 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23441ea0e0 con 0x7f2344162e10 2024-04-24T22:59:39.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.244+0000 7f23417fa640 1 -- 172.21.15.32:0/286406762 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f233c014a00 con 0x7f2344162e10 2024-04-24T22:59:39.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.244+0000 7f234b02a640 1 -- 172.21.15.32:0/286406762 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f234414bdd0 con 0x7f2344162e10 2024-04-24T22:59:39.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.245+0000 7f234b02a640 1 -- 172.21.15.32:0/286406762 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2344068430 con 0x7f2344162e10 2024-04-24T22:59:39.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.245+0000 7f23417fa640 1 -- 172.21.15.32:0/286406762 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f233c014ba0 con 0x7f2344162e10 2024-04-24T22:59:39.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.246+0000 7f23417fa640 1 --2- 172.21.15.32:0/286406762 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f233006be30 0x7f233006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:39.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.246+0000 7f23417fa640 1 -- 172.21.15.32:0/286406762 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f233c08b840 con 0x7f2344162e10 2024-04-24T22:59:39.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.246+0000 7f2348dc6640 1 --2- 172.21.15.32:0/286406762 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f233006be30 0x7f233006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:39.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.247+0000 7f2348dc6640 1 --2- 172.21.15.32:0/286406762 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f233006be30 0x7f233006e2f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2334034000 tx=0x7f233402c000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:39.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.250+0000 7f23417fa640 1 -- 172.21.15.32:0/286406762 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f233c05a150 con 0x7f2344162e10 2024-04-24T22:59:39.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.438+0000 7f234b02a640 1 -- 172.21.15.32:0/286406762 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2344067730 con 0x7f2344162e10 2024-04-24T22:59:39.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.439+0000 7f23417fa640 1 -- 172.21.15.32:0/286406762 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16621 (secure 0 0 0) 0x7f233c059a50 con 0x7f2344162e10 2024-04-24T22:59:39.441 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:59:39.441 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T22:59:05.517554+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:59:02.016639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T22:59:02.846087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:59:39.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.441+0000 7f234b02a640 1 -- 172.21.15.32:0/286406762 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f233006be30 msgr2=0x7f233006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:39.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.441+0000 7f234b02a640 1 --2- 172.21.15.32:0/286406762 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f233006be30 0x7f233006e2f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2334034000 tx=0x7f233402c000 comp rx=0 tx=0).stop 2024-04-24T22:59:39.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.441+0000 7f234b02a640 1 -- 172.21.15.32:0/286406762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2344162e10 msgr2=0x7f23441f81e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:39.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.441+0000 7f234b02a640 1 --2- 172.21.15.32:0/286406762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2344162e10 0x7f23441f81e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f233c007e40 tx=0x7f233c00e4b0 comp rx=0 tx=0).stop 2024-04-24T22:59:39.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.442+0000 7f234b02a640 1 -- 172.21.15.32:0/286406762 shutdown_connections 2024-04-24T22:59:39.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.442+0000 7f234b02a640 1 --2- 172.21.15.32:0/286406762 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f233006be30 0x7f233006e2f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:39.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.442+0000 7f234b02a640 1 --2- 172.21.15.32:0/286406762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2344162e10 0x7f23441f81e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:39.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.442+0000 7f234b02a640 1 --2- 172.21.15.32:0/286406762 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2344162320 0x7f23440aeb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:39.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.442+0000 7f234b02a640 1 --2- 172.21.15.32:0/286406762 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f23440b5af0 0x7f23441e9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:39.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.442+0000 7f234b02a640 1 -- 172.21.15.32:0/286406762 >> 172.21.15.32:0/286406762 conn(0x7f23440d0ae0 msgr2=0x7f234406ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:39.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.442+0000 7f234b02a640 1 -- 172.21.15.32:0/286406762 shutdown_connections 2024-04-24T22:59:39.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.442+0000 7f234b02a640 1 -- 172.21.15.32:0/286406762 wait complete. 2024-04-24T22:59:39.455 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T22:59:02.016639+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T22:59:02.846087+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T22:59:39.455 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T22:59:39.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.557+0000 7f871509e640 1 Processor -- start 2024-04-24T22:59:39.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.557+0000 7f871509e640 1 -- start start 2024-04-24T22:59:39.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.558+0000 7f871509e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f871006b820 0x7f87100d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:39.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.558+0000 7f871509e640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f87100d4210 0x7f87100b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:39.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.558+0000 7f871509e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87100d3170 con 0x7f871015e9b0 2024-04-24T22:59:39.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.558+0000 7f871509e640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f87100d3450 con 0x7f871006b820 2024-04-24T22:59:39.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.558+0000 7f871509e640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f87100d3730 con 0x7f87100d4210 2024-04-24T22:59:39.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.558+0000 7f870e59c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f871006b820 0x7f87100d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:39.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.558+0000 7f870ed9d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f871015e9b0 0x7f871006b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34180/0 (socket says 172.21.15.32:34180) 2024-04-24T22:59:39.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.558+0000 7f870ed9d640 1 -- 172.21.15.32:0/3255938694 learned_addr learned my addr 172.21.15.32:0/3255938694 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:39.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.559+0000 7f870f59e640 1 --2- 172.21.15.32:0/3255938694 >> v2:172.21.15.121:3300/0 conn(0x7f87100d4210 0x7f87100b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:39.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.559+0000 7f870dd9b640 1 -- 172.21.15.32:0/3255938694 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1871118906 0 0) 0x7f87100d3170 con 0x7f871015e9b0 2024-04-24T22:59:39.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.559+0000 7f870e59c640 1 -- 172.21.15.32:0/3255938694 >> v2:172.21.15.121:3300/0 conn(0x7f87100d4210 msgr2=0x7f87100b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:39.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.559+0000 7f870e59c640 1 --2- 172.21.15.32:0/3255938694 >> v2:172.21.15.121:3300/0 conn(0x7f87100d4210 0x7f87100b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:39.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.559+0000 7f870e59c640 1 -- 172.21.15.32:0/3255938694 >> v1:172.21.15.32:6789/0 conn(0x7f871015e9b0 legacy=0x7f871006b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:39.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.559+0000 7f870e59c640 1 -- 172.21.15.32:0/3255938694 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87100d3a10 con 0x7f871006b820 2024-04-24T22:59:39.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.559+0000 7f870f59e640 1 --2- 172.21.15.32:0/3255938694 >> v2:172.21.15.121:3300/0 conn(0x7f87100d4210 0x7f87100b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:39.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.559+0000 7f870e59c640 1 --2- 172.21.15.32:0/3255938694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f871006b820 0x7f87100d2b90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f87101622f0 tx=0x7f870402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d8cd2001a2c5c34 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:39.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.560+0000 7f870dd9b640 1 -- 172.21.15.32:0/3255938694 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8704037070 con 0x7f871006b820 2024-04-24T22:59:39.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.560+0000 7f870dd9b640 1 -- 172.21.15.32:0/3255938694 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8704032440 con 0x7f871006b820 2024-04-24T22:59:39.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.560+0000 7f870dd9b640 1 -- 172.21.15.32:0/3255938694 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870402ae60 con 0x7f871006b820 2024-04-24T22:59:39.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.560+0000 7f871509e640 1 -- 172.21.15.32:0/3255938694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f871006b820 msgr2=0x7f87100d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:39.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.560+0000 7f871509e640 1 --2- 172.21.15.32:0/3255938694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f871006b820 0x7f87100d2b90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f87101622f0 tx=0x7f870402a110 comp rx=0 tx=0).stop 2024-04-24T22:59:39.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.560+0000 7f871509e640 1 -- 172.21.15.32:0/3255938694 shutdown_connections 2024-04-24T22:59:39.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.560+0000 7f871509e640 1 --2- 172.21.15.32:0/3255938694 >> v2:172.21.15.121:3300/0 conn(0x7f87100d4210 0x7f87100b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:39.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.560+0000 7f871509e640 1 --2- 172.21.15.32:0/3255938694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f871006b820 0x7f87100d2b90 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:39.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.560+0000 7f871509e640 1 -- 172.21.15.32:0/3255938694 >> 172.21.15.32:0/3255938694 conn(0x7f87100d11d0 msgr2=0x7f87100d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:39.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.560+0000 7f871509e640 1 -- 172.21.15.32:0/3255938694 shutdown_connections 2024-04-24T22:59:39.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.560+0000 7f871509e640 1 -- 172.21.15.32:0/3255938694 wait complete. 2024-04-24T22:59:39.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.562+0000 7f871509e640 1 Processor -- start 2024-04-24T22:59:39.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.562+0000 7f871509e640 1 -- start start 2024-04-24T22:59:39.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.562+0000 7f871509e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f871006b820 0x7f871014e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:39.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.562+0000 7f871509e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87100d4210 0x7f871015ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:39.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.562+0000 7f871509e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f871015e9b0 0x7f87101605b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:39.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.562+0000 7f871509e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f87100ba0e0 con 0x7f87100d4210 2024-04-24T22:59:39.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.562+0000 7f871509e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f87100d2400 con 0x7f871006b820 2024-04-24T22:59:39.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.562+0000 7f871509e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f87100ba260 con 0x7f871015e9b0 2024-04-24T22:59:39.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.562+0000 7f870e59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87100d4210 0x7f871015ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:39.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.562+0000 7f870e59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87100d4210 0x7f871015ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52498/0 (socket says 172.21.15.32:52498) 2024-04-24T22:59:39.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.562+0000 7f870e59c640 1 -- 172.21.15.32:0/3469232249 learned_addr learned my addr 172.21.15.32:0/3469232249 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:39.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.563+0000 7f870ed9d640 1 --2- 172.21.15.32:0/3469232249 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f871006b820 0x7f871014e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:39.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.563+0000 7f870f59e640 1 --2- 172.21.15.32:0/3469232249 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f871015e9b0 0x7f87101605b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:39.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.563+0000 7f870e59c640 1 -- 172.21.15.32:0/3469232249 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f871015e9b0 msgr2=0x7f87101605b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:39.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.563+0000 7f870e59c640 1 --2- 172.21.15.32:0/3469232249 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f871015e9b0 0x7f87101605b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:39.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.563+0000 7f870e59c640 1 -- 172.21.15.32:0/3469232249 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f871006b820 msgr2=0x7f871014e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:39.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.563+0000 7f870e59c640 1 --2- 172.21.15.32:0/3469232249 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f871006b820 0x7f871014e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:39.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.563+0000 7f870e59c640 1 -- 172.21.15.32:0/3469232249 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f871015f770 con 0x7f87100d4210 2024-04-24T22:59:39.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.563+0000 7f870f59e640 1 --2- 172.21.15.32:0/3469232249 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f871015e9b0 0x7f87101605b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:39.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.563+0000 7f870e59c640 1 --2- 172.21.15.32:0/3469232249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87100d4210 0x7f871015ed90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8704009890 tx=0x7f87040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:39.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.563+0000 7f86f77fe640 1 -- 172.21.15.32:0/3469232249 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8704037050 con 0x7f87100d4210 2024-04-24T22:59:39.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.563+0000 7f86f77fe640 1 -- 172.21.15.32:0/3469232249 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8704038040 con 0x7f87100d4210 2024-04-24T22:59:39.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.563+0000 7f871509e640 1 -- 172.21.15.32:0/3469232249 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f871015fa50 con 0x7f87100d4210 2024-04-24T22:59:39.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.563+0000 7f86f77fe640 1 -- 172.21.15.32:0/3469232249 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8704004470 con 0x7f87100d4210 2024-04-24T22:59:39.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.563+0000 7f871509e640 1 -- 172.21.15.32:0/3469232249 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8710160010 con 0x7f87100d4210 2024-04-24T22:59:39.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.564+0000 7f871509e640 1 -- 172.21.15.32:0/3469232249 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8710068430 con 0x7f87100d4210 2024-04-24T22:59:39.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.565+0000 7f86f77fe640 1 -- 172.21.15.32:0/3469232249 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8704030900 con 0x7f87100d4210 2024-04-24T22:59:39.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.565+0000 7f86f77fe640 1 --2- 172.21.15.32:0/3469232249 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f86fc06be30 0x7f86fc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:39.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.565+0000 7f86f77fe640 1 -- 172.21.15.32:0/3469232249 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f87040ab990 con 0x7f87100d4210 2024-04-24T22:59:39.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.565+0000 7f870ed9d640 1 --2- 172.21.15.32:0/3469232249 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f86fc06be30 0x7f86fc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:39.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.566+0000 7f870ed9d640 1 --2- 172.21.15.32:0/3469232249 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f86fc06be30 0x7f86fc06e2f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f87080012e0 tx=0x7f8708001630 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:39.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.569+0000 7f86f77fe640 1 -- 172.21.15.32:0/3469232249 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f870407a4f0 con 0x7f87100d4210 2024-04-24T22:59:39.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.753+0000 7f871509e640 1 -- 172.21.15.32:0/3469232249 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f87100728d0 con 0x7f87100d4210 2024-04-24T22:59:39.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.753+0000 7f86f77fe640 1 -- 172.21.15.32:0/3469232249 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f8704079df0 con 0x7f87100d4210 2024-04-24T22:59:39.755 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T22:59:39.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.755+0000 7f871509e640 1 -- 172.21.15.32:0/3469232249 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f86fc06be30 msgr2=0x7f86fc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:39.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.755+0000 7f871509e640 1 --2- 172.21.15.32:0/3469232249 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f86fc06be30 0x7f86fc06e2f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f87080012e0 tx=0x7f8708001630 comp rx=0 tx=0).stop 2024-04-24T22:59:39.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.755+0000 7f871509e640 1 -- 172.21.15.32:0/3469232249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87100d4210 msgr2=0x7f871015ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:39.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.755+0000 7f871509e640 1 --2- 172.21.15.32:0/3469232249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87100d4210 0x7f871015ed90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8704009890 tx=0x7f87040046c0 comp rx=0 tx=0).stop 2024-04-24T22:59:39.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.755+0000 7f871509e640 1 -- 172.21.15.32:0/3469232249 shutdown_connections 2024-04-24T22:59:39.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.755+0000 7f871509e640 1 --2- 172.21.15.32:0/3469232249 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f86fc06be30 0x7f86fc06e2f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:39.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.755+0000 7f871509e640 1 --2- 172.21.15.32:0/3469232249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87100d4210 0x7f871015ed90 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:39.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.755+0000 7f871509e640 1 --2- 172.21.15.32:0/3469232249 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f871015e9b0 0x7f87101605b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:39.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.755+0000 7f871509e640 1 --2- 172.21.15.32:0/3469232249 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f871006b820 0x7f871014e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:39.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.755+0000 7f871509e640 1 -- 172.21.15.32:0/3469232249 >> 172.21.15.32:0/3469232249 conn(0x7f87100d11d0 msgr2=0x7f87100b7b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:39.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.755+0000 7f871509e640 1 -- 172.21.15.32:0/3469232249 shutdown_connections 2024-04-24T22:59:39.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.755+0000 7f871509e640 1 -- 172.21.15.32:0/3469232249 wait complete. 2024-04-24T22:59:39.768 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T22:59:39.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.869+0000 7fb766753640 1 Processor -- start 2024-04-24T22:59:39.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.869+0000 7fb766753640 1 -- start start 2024-04-24T22:59:39.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.870+0000 7fb766753640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb76015b380 0x7fb76006c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:39.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.870+0000 7fb766753640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb76015cb00 0x7fb76006cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:39.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.870+0000 7fb766753640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7600b4520 con 0x7fb7600ce430 2024-04-24T22:59:39.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.870+0000 7fb766753640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fb7600cef90 con 0x7fb76015b380 2024-04-24T22:59:39.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.870+0000 7fb766753640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fb7600b4800 con 0x7fb76015cb00 2024-04-24T22:59:39.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.870+0000 7fb764cf0640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb7600ce430 0x7fb7600ce880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52028/0 (socket says 172.21.15.32:52028) 2024-04-24T22:59:39.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.870+0000 7fb764cf0640 1 -- 172.21.15.32:0/539494625 learned_addr learned my addr 172.21.15.32:0/539494625 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:39.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.870+0000 7fb75f7fe640 1 --2- 172.21.15.32:0/539494625 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb76015cb00 0x7fb76006cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:39.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.870+0000 7fb75ffff640 1 --2- 172.21.15.32:0/539494625 >> v2:172.21.15.121:3300/0 conn(0x7fb76015b380 0x7fb76006c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:39.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.870+0000 7fb75effd640 1 -- 172.21.15.32:0/539494625 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 526201558 0 0) 0x7fb7600b4520 con 0x7fb7600ce430 2024-04-24T22:59:39.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.870+0000 7fb75effd640 1 -- 172.21.15.32:0/539494625 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb748003520 con 0x7fb7600ce430 2024-04-24T22:59:39.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.870+0000 7fb75effd640 1 -- 172.21.15.32:0/539494625 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1519949486 0 0) 0x7fb748003520 con 0x7fb7600ce430 2024-04-24T22:59:39.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.870+0000 7fb75effd640 1 -- 172.21.15.32:0/539494625 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb76015cb00 msgr2=0x7fb76006cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:39.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.870+0000 7fb75effd640 1 --2- 172.21.15.32:0/539494625 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb76015cb00 0x7fb76006cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:39.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.871+0000 7fb75effd640 1 -- 172.21.15.32:0/539494625 >> v2:172.21.15.121:3300/0 conn(0x7fb76015b380 msgr2=0x7fb76006c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:39.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.871+0000 7fb75effd640 1 --2- 172.21.15.32:0/539494625 >> v2:172.21.15.121:3300/0 conn(0x7fb76015b380 0x7fb76006c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:39.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.871+0000 7fb75effd640 1 -- 172.21.15.32:0/539494625 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7600b4ae0 con 0x7fb7600ce430 2024-04-24T22:59:39.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.871+0000 7fb75effd640 1 -- 172.21.15.32:0/539494625 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7fb7580019f0 con 0x7fb7600ce430 2024-04-24T22:59:39.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.871+0000 7fb75effd640 1 -- 172.21.15.32:0/539494625 >> v1:172.21.15.32:6789/0 conn(0x7fb7600ce430 legacy=0x7fb7600ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:39.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.871+0000 7fb75effd640 1 --2- 172.21.15.32:0/539494625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7480039b0 0x7fb748003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:39.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.871+0000 7fb75effd640 1 --2- 172.21.15.32:0/539494625 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb748004800 0x7fb748004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:39.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.871+0000 7fb75effd640 1 --2- 172.21.15.32:0/539494625 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb76015cb00 0x7fb7480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:39.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.871+0000 7fb75effd640 1 -- 172.21.15.32:0/539494625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb7600cef90 con 0x7fb7480039b0 2024-04-24T22:59:39.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.871+0000 7fb75effd640 1 -- 172.21.15.32:0/539494625 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb7600b4800 con 0x7fb748004800 2024-04-24T22:59:39.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.871+0000 7fb75effd640 1 -- 172.21.15.32:0/539494625 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fb74800b9f0 con 0x7fb76015cb00 2024-04-24T22:59:39.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.871+0000 7fb75ffff640 1 --2- 172.21.15.32:0/539494625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7480039b0 0x7fb748003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:39.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.871+0000 7fb75f7fe640 1 --2- 172.21.15.32:0/539494625 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb748004800 0x7fb748004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:39.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.871+0000 7fb764cf0640 1 --2- 172.21.15.32:0/539494625 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb76015cb00 0x7fb7480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:39.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.871+0000 7fb75ffff640 1 -- 172.21.15.32:0/539494625 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb76015cb00 msgr2=0x7fb7480051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:39.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.871+0000 7fb75ffff640 1 --2- 172.21.15.32:0/539494625 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb76015cb00 0x7fb7480051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:39.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.871+0000 7fb75ffff640 1 -- 172.21.15.32:0/539494625 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb748004800 msgr2=0x7fb748004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:39.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.871+0000 7fb75ffff640 1 --2- 172.21.15.32:0/539494625 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb748004800 0x7fb748004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:39.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.871+0000 7fb75ffff640 1 -- 172.21.15.32:0/539494625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb748003520 con 0x7fb7480039b0 2024-04-24T22:59:39.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.872+0000 7fb75ffff640 1 --2- 172.21.15.32:0/539494625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7480039b0 0x7fb748003dd0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb754009650 tx=0x7fb748005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d03c2e5afd42307 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:39.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.872+0000 7fb75effd640 1 -- 172.21.15.32:0/539494625 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb754038070 con 0x7fb7480039b0 2024-04-24T22:59:39.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.872+0000 7fb75effd640 1 -- 172.21.15.32:0/539494625 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb754033400 con 0x7fb7480039b0 2024-04-24T22:59:39.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.872+0000 7fb75effd640 1 -- 172.21.15.32:0/539494625 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb754009690 con 0x7fb7480039b0 2024-04-24T22:59:39.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.872+0000 7fb766753640 1 -- 172.21.15.32:0/539494625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7480039b0 msgr2=0x7fb748003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:39.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.872+0000 7fb766753640 1 --2- 172.21.15.32:0/539494625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7480039b0 0x7fb748003dd0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb754009650 tx=0x7fb748005830 comp rx=0 tx=0).stop 2024-04-24T22:59:39.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.872+0000 7fb764cf0640 1 -- 172.21.15.32:0/539494625 reap_dead start 2024-04-24T22:59:39.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.872+0000 7fb766753640 1 -- 172.21.15.32:0/539494625 shutdown_connections 2024-04-24T22:59:39.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.873+0000 7fb766753640 1 -- 172.21.15.32:0/539494625 >> 172.21.15.32:0/539494625 conn(0x7fb76006b110 msgr2=0x7fb76006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:39.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.873+0000 7fb766753640 1 -- 172.21.15.32:0/539494625 shutdown_connections 2024-04-24T22:59:39.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.873+0000 7fb766753640 1 -- 172.21.15.32:0/539494625 wait complete. 2024-04-24T22:59:39.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.874+0000 7fb766753640 1 Processor -- start 2024-04-24T22:59:39.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.874+0000 7fb766753640 1 -- start start 2024-04-24T22:59:39.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.875+0000 7fb766753640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb7480039b0 0x7fb760072230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:39.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.875+0000 7fb766753640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb760154340 0x7fb76014de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:39.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.875+0000 7fb766753640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb760153800 0x7fb760153cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:39.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.875+0000 7fb766753640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb76015b760 con 0x7fb760153800 2024-04-24T22:59:39.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.875+0000 7fb766753640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb7600b50a0 con 0x7fb760154340 2024-04-24T22:59:39.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.875+0000 7fb766753640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fb76006cb30 con 0x7fb7480039b0 2024-04-24T22:59:39.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.875+0000 7fb764cf0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb760153800 0x7fb760153cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:39.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.875+0000 7fb764cf0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb760153800 0x7fb760153cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57378/0 (socket says 172.21.15.32:57378) 2024-04-24T22:59:39.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.875+0000 7fb764cf0640 1 -- 172.21.15.32:0/311741032 learned_addr learned my addr 172.21.15.32:0/311741032 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:39.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.875+0000 7fb75ffff640 1 --2- 172.21.15.32:0/311741032 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb7480039b0 0x7fb760072230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:39.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.875+0000 7fb75f7fe640 1 --2- 172.21.15.32:0/311741032 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb760154340 0x7fb76014de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:39.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.875+0000 7fb764cf0640 1 -- 172.21.15.32:0/311741032 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb7480039b0 msgr2=0x7fb760072230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:39.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.875+0000 7fb764cf0640 1 --2- 172.21.15.32:0/311741032 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb7480039b0 0x7fb760072230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:39.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.875+0000 7fb764cf0640 1 -- 172.21.15.32:0/311741032 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb760154340 msgr2=0x7fb76014de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:39.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.875+0000 7fb764cf0640 1 --2- 172.21.15.32:0/311741032 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb760154340 0x7fb76014de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:39.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.875+0000 7fb764cf0640 1 -- 172.21.15.32:0/311741032 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb760152a40 con 0x7fb760153800 2024-04-24T22:59:39.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.875+0000 7fb764cf0640 1 --2- 172.21.15.32:0/311741032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb760153800 0x7fb760153cb0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb760072ea0 tx=0x7fb758007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:39.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.875+0000 7fb75cff9640 1 -- 172.21.15.32:0/311741032 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb758017020 con 0x7fb760153800 2024-04-24T22:59:39.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.875+0000 7fb75cff9640 1 -- 172.21.15.32:0/311741032 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb75800ab20 con 0x7fb760153800 2024-04-24T22:59:39.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.875+0000 7fb766753640 1 -- 172.21.15.32:0/311741032 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb760152d20 con 0x7fb760153800 2024-04-24T22:59:39.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.875+0000 7fb75cff9640 1 -- 172.21.15.32:0/311741032 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7580133f0 con 0x7fb760153800 2024-04-24T22:59:39.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.876+0000 7fb766753640 1 -- 172.21.15.32:0/311741032 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb76014e380 con 0x7fb760153800 2024-04-24T22:59:39.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.877+0000 7fb766753640 1 -- 172.21.15.32:0/311741032 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb760068430 con 0x7fb760153800 2024-04-24T22:59:39.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.877+0000 7fb75cff9640 1 -- 172.21.15.32:0/311741032 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb75801a020 con 0x7fb760153800 2024-04-24T22:59:39.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.877+0000 7fb75cff9640 1 --2- 172.21.15.32:0/311741032 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb74c06be30 0x7fb74c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:39.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.878+0000 7fb75cff9640 1 -- 172.21.15.32:0/311741032 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb75808b490 con 0x7fb760153800 2024-04-24T22:59:39.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.878+0000 7fb75ffff640 1 --2- 172.21.15.32:0/311741032 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb74c06be30 0x7fb74c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:39.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.878+0000 7fb75ffff640 1 --2- 172.21.15.32:0/311741032 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb74c06be30 0x7fb74c06e2f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb754035000 tx=0x7fb754008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:39.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:39.882+0000 7fb75cff9640 1 -- 172.21.15.32:0/311741032 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb758021020 con 0x7fb760153800 2024-04-24T22:59:40.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.063+0000 7fb766753640 1 -- 172.21.15.32:0/311741032 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb76006cf00 con 0x7fb760153800 2024-04-24T22:59:40.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.063+0000 7fb75cff9640 1 -- 172.21.15.32:0/311741032 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7fb758059ed0 con 0x7fb760153800 2024-04-24T22:59:40.065 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-24T22:59:40.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.065+0000 7fb766753640 1 -- 172.21.15.32:0/311741032 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb74c06be30 msgr2=0x7fb74c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:40.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.065+0000 7fb766753640 1 --2- 172.21.15.32:0/311741032 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb74c06be30 0x7fb74c06e2f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb754035000 tx=0x7fb754008030 comp rx=0 tx=0).stop 2024-04-24T22:59:40.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.065+0000 7fb766753640 1 -- 172.21.15.32:0/311741032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb760153800 msgr2=0x7fb760153cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:40.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.065+0000 7fb766753640 1 --2- 172.21.15.32:0/311741032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb760153800 0x7fb760153cb0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb760072ea0 tx=0x7fb758007a20 comp rx=0 tx=0).stop 2024-04-24T22:59:40.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.065+0000 7fb766753640 1 -- 172.21.15.32:0/311741032 shutdown_connections 2024-04-24T22:59:40.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.065+0000 7fb766753640 1 --2- 172.21.15.32:0/311741032 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb74c06be30 0x7fb74c06e2f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:40.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.065+0000 7fb766753640 1 --2- 172.21.15.32:0/311741032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb760153800 0x7fb760153cb0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:40.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.065+0000 7fb766753640 1 --2- 172.21.15.32:0/311741032 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb760154340 0x7fb76014de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:40.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.065+0000 7fb766753640 1 --2- 172.21.15.32:0/311741032 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb7480039b0 0x7fb760072230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:40.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.065+0000 7fb766753640 1 -- 172.21.15.32:0/311741032 >> 172.21.15.32:0/311741032 conn(0x7fb76006b110 msgr2=0x7fb7601554d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:40.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.065+0000 7fb766753640 1 -- 172.21.15.32:0/311741032 shutdown_connections 2024-04-24T22:59:40.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.065+0000 7fb766753640 1 -- 172.21.15.32:0/311741032 wait complete. 2024-04-24T22:59:40.078 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T22:59:40.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.179+0000 7f30d459e640 1 Processor -- start 2024-04-24T22:59:40.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.180+0000 7f30d459e640 1 -- start start 2024-04-24T22:59:40.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.180+0000 7f30d459e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f30cc06dc60 0x7f30cc169570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:40.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.180+0000 7f30d459e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30cc0b6d10 con 0x7f30cc158dd0 2024-04-24T22:59:40.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.180+0000 7f30d459e640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30cc15ef10 con 0x7f30cc06ba30 2024-04-24T22:59:40.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.180+0000 7f30d459e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f30cc15f1f0 con 0x7f30cc06dc60 2024-04-24T22:59:40.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.181+0000 7f30d233a640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f30cc158dd0 0x7f30cc06b320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52042/0 (socket says 172.21.15.32:52042) 2024-04-24T22:59:40.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.181+0000 7f30d233a640 1 -- 172.21.15.32:0/4069579193 learned_addr learned my addr 172.21.15.32:0/4069579193 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:40.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.181+0000 7f30d1338640 1 -- 172.21.15.32:0/4069579193 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3458711331 0 0) 0x7f30cc0b6d10 con 0x7f30cc158dd0 2024-04-24T22:59:40.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.181+0000 7f30d2b3b640 1 --2- 172.21.15.32:0/4069579193 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f30cc06dc60 0x7f30cc169570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:40.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.181+0000 7f30d1338640 1 -- 172.21.15.32:0/4069579193 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f30b8003520 con 0x7f30cc158dd0 2024-04-24T22:59:40.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.181+0000 7f30d1338640 1 -- 172.21.15.32:0/4069579193 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1068084617 0 0) 0x7f30b8003520 con 0x7f30cc158dd0 2024-04-24T22:59:40.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.181+0000 7f30d1338640 1 -- 172.21.15.32:0/4069579193 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f30cc06dc60 msgr2=0x7f30cc169570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:40.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.181+0000 7f30d1338640 1 --2- 172.21.15.32:0/4069579193 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f30cc06dc60 0x7f30cc169570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:40.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.181+0000 7f30d1338640 1 -- 172.21.15.32:0/4069579193 >> v1:172.21.15.121:6789/0 conn(0x7f30cc06ba30 legacy=0x7f30cc06d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:40.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.181+0000 7f30d1338640 1 -- 172.21.15.32:0/4069579193 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30cc15f4d0 con 0x7f30cc158dd0 2024-04-24T22:59:40.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.181+0000 7f30d1338640 1 -- 172.21.15.32:0/4069579193 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f30c8001690 con 0x7f30cc158dd0 2024-04-24T22:59:40.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.181+0000 7f30d1338640 1 -- 172.21.15.32:0/4069579193 >> v1:172.21.15.32:6789/0 conn(0x7f30cc158dd0 legacy=0x7f30cc06b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:40.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.181+0000 7f30d1338640 1 --2- 172.21.15.32:0/4069579193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30b80039e0 0x7f30b8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:40.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.181+0000 7f30d1338640 1 --2- 172.21.15.32:0/4069579193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f30b8004390 0x7f30b8004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:40.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.181+0000 7f30d1338640 1 --2- 172.21.15.32:0/4069579193 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f30cc06dc60 0x7f30b80050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:40.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.181+0000 7f30d1338640 1 -- 172.21.15.32:0/4069579193 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f30cc15f1f0 con 0x7f30b80039e0 2024-04-24T22:59:40.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.181+0000 7f30d1338640 1 -- 172.21.15.32:0/4069579193 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f30b800b9f0 con 0x7f30b8004390 2024-04-24T22:59:40.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.181+0000 7f30d1338640 1 -- 172.21.15.32:0/4069579193 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f30b800bbf0 con 0x7f30cc06dc60 2024-04-24T22:59:40.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.181+0000 7f30d233a640 1 --2- 172.21.15.32:0/4069579193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30b80039e0 0x7f30b8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:40.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.182+0000 7f30d1b39640 1 --2- 172.21.15.32:0/4069579193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f30b8004390 0x7f30b8004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:40.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.182+0000 7f30d2b3b640 1 --2- 172.21.15.32:0/4069579193 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f30cc06dc60 0x7f30b80050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:40.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.182+0000 7f30d233a640 1 -- 172.21.15.32:0/4069579193 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f30cc06dc60 msgr2=0x7f30b80050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:40.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.182+0000 7f30d233a640 1 --2- 172.21.15.32:0/4069579193 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f30cc06dc60 0x7f30b80050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:40.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.182+0000 7f30d233a640 1 -- 172.21.15.32:0/4069579193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f30b8004390 msgr2=0x7f30b8004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:40.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.182+0000 7f30d233a640 1 --2- 172.21.15.32:0/4069579193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f30b8004390 0x7f30b8004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:40.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.182+0000 7f30d233a640 1 -- 172.21.15.32:0/4069579193 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30b8003520 con 0x7f30b80039e0 2024-04-24T22:59:40.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.182+0000 7f30d233a640 1 --2- 172.21.15.32:0/4069579193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30b80039e0 0x7f30b8003dc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f30c800c210 tx=0x7f30b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fc65bc7f04fab1f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:40.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.183+0000 7f30d1338640 1 -- 172.21.15.32:0/4069579193 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c8007ce0 con 0x7f30b80039e0 2024-04-24T22:59:40.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.183+0000 7f30d1338640 1 -- 172.21.15.32:0/4069579193 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30c8005ad0 con 0x7f30b80039e0 2024-04-24T22:59:40.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.183+0000 7f30d1338640 1 -- 172.21.15.32:0/4069579193 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c8007070 con 0x7f30b80039e0 2024-04-24T22:59:40.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.183+0000 7f30d459e640 1 -- 172.21.15.32:0/4069579193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30b80039e0 msgr2=0x7f30b8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:40.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.183+0000 7f30d459e640 1 --2- 172.21.15.32:0/4069579193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30b80039e0 0x7f30b8003dc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f30c800c210 tx=0x7f30b8005830 comp rx=0 tx=0).stop 2024-04-24T22:59:40.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.183+0000 7f30d2b3b640 1 -- 172.21.15.32:0/4069579193 reap_dead start 2024-04-24T22:59:40.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.183+0000 7f30d459e640 1 -- 172.21.15.32:0/4069579193 shutdown_connections 2024-04-24T22:59:40.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.183+0000 7f30d459e640 1 -- 172.21.15.32:0/4069579193 >> 172.21.15.32:0/4069579193 conn(0x7f30cc06e800 msgr2=0x7f30cc06ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:40.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.183+0000 7f30d459e640 1 -- 172.21.15.32:0/4069579193 shutdown_connections 2024-04-24T22:59:40.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.183+0000 7f30d459e640 1 -- 172.21.15.32:0/4069579193 wait complete. 2024-04-24T22:59:40.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.184+0000 7f30d459e640 1 Processor -- start 2024-04-24T22:59:40.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.184+0000 7f30d459e640 1 -- start start 2024-04-24T22:59:40.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.185+0000 7f30d459e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f30b80039e0 0x7f30cc150b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:40.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.185+0000 7f30d459e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30cc0728e0 0x7f30cc1555b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:40.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.185+0000 7f30d459e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f30cc072d60 0x7f30cc155af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:40.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.185+0000 7f30d459e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f30cc06bfe0 con 0x7f30cc0728e0 2024-04-24T22:59:40.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.185+0000 7f30d459e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f30cc15fcd0 con 0x7f30b80039e0 2024-04-24T22:59:40.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.185+0000 7f30d459e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f30cc1591b0 con 0x7f30cc072d60 2024-04-24T22:59:40.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.185+0000 7f30d1b39640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30cc0728e0 0x7f30cc1555b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:40.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.185+0000 7f30d1b39640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30cc0728e0 0x7f30cc1555b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57388/0 (socket says 172.21.15.32:57388) 2024-04-24T22:59:40.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.185+0000 7f30d1b39640 1 -- 172.21.15.32:0/2700899731 learned_addr learned my addr 172.21.15.32:0/2700899731 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:40.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.185+0000 7f30d233a640 1 --2- 172.21.15.32:0/2700899731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f30b80039e0 0x7f30cc150b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:40.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.185+0000 7f30d1b39640 1 -- 172.21.15.32:0/2700899731 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f30cc072d60 msgr2=0x7f30cc155af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:40.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.185+0000 7f30d2b3b640 1 --2- 172.21.15.32:0/2700899731 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f30cc072d60 0x7f30cc155af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:40.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.185+0000 7f30d1b39640 1 --2- 172.21.15.32:0/2700899731 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f30cc072d60 0x7f30cc155af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:40.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.185+0000 7f30d1b39640 1 -- 172.21.15.32:0/2700899731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f30b80039e0 msgr2=0x7f30cc150b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:40.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.185+0000 7f30d1b39640 1 --2- 172.21.15.32:0/2700899731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f30b80039e0 0x7f30cc150b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:40.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.186+0000 7f30d1b39640 1 -- 172.21.15.32:0/2700899731 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30cc151070 con 0x7f30cc0728e0 2024-04-24T22:59:40.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.186+0000 7f30d233a640 1 --2- 172.21.15.32:0/2700899731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f30b80039e0 0x7f30cc150b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:59:40.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.186+0000 7f30d2b3b640 1 --2- 172.21.15.32:0/2700899731 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f30cc072d60 0x7f30cc155af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:59:40.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.186+0000 7f30d1b39640 1 --2- 172.21.15.32:0/2700899731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30cc0728e0 0x7f30cc1555b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f30bc00f870 tx=0x7f30bc00fd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:40.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.186+0000 7f30c2ffd640 1 -- 172.21.15.32:0/2700899731 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30bc01a070 con 0x7f30cc0728e0 2024-04-24T22:59:40.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.186+0000 7f30c2ffd640 1 -- 172.21.15.32:0/2700899731 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30bc005320 con 0x7f30cc0728e0 2024-04-24T22:59:40.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.186+0000 7f30d459e640 1 -- 172.21.15.32:0/2700899731 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30cc15d2f0 con 0x7f30cc0728e0 2024-04-24T22:59:40.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.186+0000 7f30c2ffd640 1 -- 172.21.15.32:0/2700899731 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30bc007270 con 0x7f30cc0728e0 2024-04-24T22:59:40.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.186+0000 7f30d459e640 1 -- 172.21.15.32:0/2700899731 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30cc15d8b0 con 0x7f30cc0728e0 2024-04-24T22:59:40.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.187+0000 7f30d459e640 1 -- 172.21.15.32:0/2700899731 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30cc068430 con 0x7f30cc0728e0 2024-04-24T22:59:40.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.187+0000 7f30c2ffd640 1 -- 172.21.15.32:0/2700899731 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f30bc01d070 con 0x7f30cc0728e0 2024-04-24T22:59:40.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.188+0000 7f30c2ffd640 1 --2- 172.21.15.32:0/2700899731 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f30b006be30 0x7f30b006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:40.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.188+0000 7f30c2ffd640 1 -- 172.21.15.32:0/2700899731 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f30bc08a5b0 con 0x7f30cc0728e0 2024-04-24T22:59:40.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.188+0000 7f30d233a640 1 --2- 172.21.15.32:0/2700899731 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f30b006be30 0x7f30b006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:40.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.188+0000 7f30d233a640 1 --2- 172.21.15.32:0/2700899731 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f30b006be30 0x7f30b006e2f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f30cc06bf10 tx=0x7f30c802d000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:40.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.192+0000 7f30c2ffd640 1 -- 172.21.15.32:0/2700899731 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f30bc058ff0 con 0x7f30cc0728e0 2024-04-24T22:59:40.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.373+0000 7f30d459e640 1 -- 172.21.15.32:0/2700899731 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f30cc160380 con 0x7f30cc0728e0 2024-04-24T22:59:40.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.373+0000 7f30c2ffd640 1 -- 172.21.15.32:0/2700899731 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f30bc0588f0 con 0x7f30cc0728e0 2024-04-24T22:59:40.375 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-24T22:59:40.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.375+0000 7f30d459e640 1 -- 172.21.15.32:0/2700899731 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f30b006be30 msgr2=0x7f30b006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:40.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.375+0000 7f30d459e640 1 --2- 172.21.15.32:0/2700899731 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f30b006be30 0x7f30b006e2f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f30cc06bf10 tx=0x7f30c802d000 comp rx=0 tx=0).stop 2024-04-24T22:59:40.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.375+0000 7f30d459e640 1 -- 172.21.15.32:0/2700899731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30cc0728e0 msgr2=0x7f30cc1555b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:40.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.375+0000 7f30d459e640 1 --2- 172.21.15.32:0/2700899731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30cc0728e0 0x7f30cc1555b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f30bc00f870 tx=0x7f30bc00fd30 comp rx=0 tx=0).stop 2024-04-24T22:59:40.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.375+0000 7f30d459e640 1 -- 172.21.15.32:0/2700899731 shutdown_connections 2024-04-24T22:59:40.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.375+0000 7f30d459e640 1 --2- 172.21.15.32:0/2700899731 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f30b006be30 0x7f30b006e2f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:40.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.375+0000 7f30d459e640 1 --2- 172.21.15.32:0/2700899731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30cc0728e0 0x7f30cc1555b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:40.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.375+0000 7f30d459e640 1 --2- 172.21.15.32:0/2700899731 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f30cc072d60 0x7f30cc155af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:40.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.375+0000 7f30d459e640 1 --2- 172.21.15.32:0/2700899731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f30b80039e0 0x7f30cc150b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:40.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.375+0000 7f30d459e640 1 -- 172.21.15.32:0/2700899731 >> 172.21.15.32:0/2700899731 conn(0x7f30cc06e800 msgr2=0x7f30cc154960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:40.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.375+0000 7f30d459e640 1 -- 172.21.15.32:0/2700899731 shutdown_connections 2024-04-24T22:59:40.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.375+0000 7f30d459e640 1 -- 172.21.15.32:0/2700899731 wait complete. 2024-04-24T22:59:40.387 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T22:59:40.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.488+0000 7fec3f5ce640 1 Processor -- start 2024-04-24T22:59:40.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.488+0000 7fec3f5ce640 1 -- start start 2024-04-24T22:59:40.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.489+0000 7fec3f5ce640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fec38162320 0x7fec380ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:40.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.489+0000 7fec3f5ce640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec380b5af0 0x7fec38162700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:40.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.489+0000 7fec3f5ce640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fec38162c40 0x7fec380ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:40.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.489+0000 7fec3f5ce640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fec380b5660 con 0x7fec38162320 2024-04-24T22:59:40.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.489+0000 7fec3f5ce640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fec380cd270 con 0x7fec38162c40 2024-04-24T22:59:40.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.489+0000 7fec3f5ce640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fec38071610 con 0x7fec380b5af0 2024-04-24T22:59:40.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.489+0000 7fec3d36a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fec38162320 0x7fec380ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:40.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.489+0000 7fec3d36a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fec38162320 0x7fec380ccc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57392/0 (socket says 172.21.15.32:57392) 2024-04-24T22:59:40.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.489+0000 7fec3d36a640 1 -- 172.21.15.32:0/1356805595 learned_addr learned my addr 172.21.15.32:0/1356805595 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:40.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.489+0000 7fec3cb69640 1 --2- 172.21.15.32:0/1356805595 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec380b5af0 0x7fec38162700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:40.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.489+0000 7fec3db6b640 1 --2- 172.21.15.32:0/1356805595 >> v2:172.21.15.121:3300/0 conn(0x7fec38162c40 0x7fec380ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:40.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.489+0000 7fec3d36a640 1 -- 172.21.15.32:0/1356805595 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec380b5af0 msgr2=0x7fec38162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:40.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.489+0000 7fec3d36a640 1 --2- 172.21.15.32:0/1356805595 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec380b5af0 0x7fec38162700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:40.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.489+0000 7fec3d36a640 1 -- 172.21.15.32:0/1356805595 >> v2:172.21.15.121:3300/0 conn(0x7fec38162c40 msgr2=0x7fec380ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:40.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.489+0000 7fec3d36a640 1 --2- 172.21.15.32:0/1356805595 >> v2:172.21.15.121:3300/0 conn(0x7fec38162c40 0x7fec380ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:40.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.489+0000 7fec3d36a640 1 -- 172.21.15.32:0/1356805595 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec380718f0 con 0x7fec38162320 2024-04-24T22:59:40.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.490+0000 7fec3d36a640 1 --2- 172.21.15.32:0/1356805595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec38162320 0x7fec380ccc90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fec380b3ac0 tx=0x7fec3402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb97b231da0be611 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:40.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.490+0000 7fec2ffff640 1 -- 172.21.15.32:0/1356805595 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec34037070 con 0x7fec38162320 2024-04-24T22:59:40.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.490+0000 7fec2ffff640 1 -- 172.21.15.32:0/1356805595 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec34032440 con 0x7fec38162320 2024-04-24T22:59:40.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.490+0000 7fec2ffff640 1 -- 172.21.15.32:0/1356805595 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec34032810 con 0x7fec38162320 2024-04-24T22:59:40.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.490+0000 7fec3f5ce640 1 -- 172.21.15.32:0/1356805595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec38162320 msgr2=0x7fec380ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:40.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.490+0000 7fec3f5ce640 1 --2- 172.21.15.32:0/1356805595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec38162320 0x7fec380ccc90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fec380b3ac0 tx=0x7fec3402a140 comp rx=0 tx=0).stop 2024-04-24T22:59:40.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.490+0000 7fec3f5ce640 1 -- 172.21.15.32:0/1356805595 shutdown_connections 2024-04-24T22:59:40.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.490+0000 7fec3f5ce640 1 --2- 172.21.15.32:0/1356805595 >> v2:172.21.15.121:3300/0 conn(0x7fec38162c40 0x7fec380ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:40.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.490+0000 7fec3f5ce640 1 --2- 172.21.15.32:0/1356805595 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec380b5af0 0x7fec38162700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:40.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.490+0000 7fec3f5ce640 1 --2- 172.21.15.32:0/1356805595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec38162320 0x7fec380ccc90 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:40.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.490+0000 7fec3f5ce640 1 -- 172.21.15.32:0/1356805595 >> 172.21.15.32:0/1356805595 conn(0x7fec380d0ae0 msgr2=0x7fec380d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:40.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.491+0000 7fec3f5ce640 1 -- 172.21.15.32:0/1356805595 shutdown_connections 2024-04-24T22:59:40.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.491+0000 7fec3f5ce640 1 -- 172.21.15.32:0/1356805595 wait complete. 2024-04-24T22:59:40.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.492+0000 7fec3f5ce640 1 Processor -- start 2024-04-24T22:59:40.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.492+0000 7fec3f5ce640 1 -- start start 2024-04-24T22:59:40.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.492+0000 7fec3f5ce640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec380b5af0 0x7fec3815eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:40.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.492+0000 7fec3f5ce640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec38162320 0x7fec381464d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:40.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.492+0000 7fec3f5ce640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec38162c40 0x7fec38160fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:40.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.492+0000 7fec3f5ce640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fec38067950 con 0x7fec380b5af0 2024-04-24T22:59:40.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.492+0000 7fec3f5ce640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fec3815c630 con 0x7fec38162c40 2024-04-24T22:59:40.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.492+0000 7fec3f5ce640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fec38071bd0 con 0x7fec38162320 2024-04-24T22:59:40.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.492+0000 7fec3d36a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec380b5af0 0x7fec3815eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:40.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.492+0000 7fec3d36a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec380b5af0 0x7fec3815eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57404/0 (socket says 172.21.15.32:57404) 2024-04-24T22:59:40.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.493+0000 7fec3d36a640 1 -- 172.21.15.32:0/2073504704 learned_addr learned my addr 172.21.15.32:0/2073504704 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:40.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.493+0000 7fec3db6b640 1 --2- 172.21.15.32:0/2073504704 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec38162c40 0x7fec38160fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:40.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.493+0000 7fec3cb69640 1 --2- 172.21.15.32:0/2073504704 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec38162320 0x7fec381464d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:40.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.493+0000 7fec3d36a640 1 -- 172.21.15.32:0/2073504704 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec38162320 msgr2=0x7fec381464d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:40.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.493+0000 7fec3d36a640 1 --2- 172.21.15.32:0/2073504704 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec38162320 0x7fec381464d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:40.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.493+0000 7fec3d36a640 1 -- 172.21.15.32:0/2073504704 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec38162c40 msgr2=0x7fec38160fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:40.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.493+0000 7fec3d36a640 1 --2- 172.21.15.32:0/2073504704 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec38162c40 0x7fec38160fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:40.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.493+0000 7fec3d36a640 1 -- 172.21.15.32:0/2073504704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec3815f260 con 0x7fec380b5af0 2024-04-24T22:59:40.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.493+0000 7fec3db6b640 1 --2- 172.21.15.32:0/2073504704 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec38162c40 0x7fec38160fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:40.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.493+0000 7fec3d36a640 1 --2- 172.21.15.32:0/2073504704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec380b5af0 0x7fec3815eb10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fec34000c00 tx=0x7fec340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:40.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.493+0000 7fec2dffb640 1 -- 172.21.15.32:0/2073504704 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec34041070 con 0x7fec380b5af0 2024-04-24T22:59:40.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.493+0000 7fec2dffb640 1 -- 172.21.15.32:0/2073504704 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec3403c870 con 0x7fec380b5af0 2024-04-24T22:59:40.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.493+0000 7fec3f5ce640 1 -- 172.21.15.32:0/2073504704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec3815f540 con 0x7fec380b5af0 2024-04-24T22:59:40.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.493+0000 7fec2dffb640 1 -- 172.21.15.32:0/2073504704 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec34037040 con 0x7fec380b5af0 2024-04-24T22:59:40.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.493+0000 7fec3f5ce640 1 -- 172.21.15.32:0/2073504704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec3815fb00 con 0x7fec380b5af0 2024-04-24T22:59:40.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.494+0000 7fec3f5ce640 1 -- 172.21.15.32:0/2073504704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec38068430 con 0x7fec380b5af0 2024-04-24T22:59:40.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.495+0000 7fec2dffb640 1 -- 172.21.15.32:0/2073504704 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fec34044070 con 0x7fec380b5af0 2024-04-24T22:59:40.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.495+0000 7fec2dffb640 1 --2- 172.21.15.32:0/2073504704 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fec1c06be30 0x7fec1c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:40.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.495+0000 7fec2dffb640 1 -- 172.21.15.32:0/2073504704 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fec340ab890 con 0x7fec380b5af0 2024-04-24T22:59:40.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.496+0000 7fec3cb69640 1 --2- 172.21.15.32:0/2073504704 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fec1c06be30 0x7fec1c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:40.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.496+0000 7fec3cb69640 1 --2- 172.21.15.32:0/2073504704 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fec1c06be30 0x7fec1c06e2f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fec28006fd0 tx=0x7fec28008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:40.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.500+0000 7fec2dffb640 1 -- 172.21.15.32:0/2073504704 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fec3407a360 con 0x7fec380b5af0 2024-04-24T22:59:40.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.694+0000 7fec3f5ce640 1 -- 172.21.15.32:0/2073504704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fec38067730 con 0x7fec380b5af0 2024-04-24T22:59:40.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.694+0000 7fec2dffb640 1 -- 172.21.15.32:0/2073504704 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7fec34079c60 con 0x7fec380b5af0 2024-04-24T22:59:40.696 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2024-04-24T22:59:40.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.696+0000 7fec3f5ce640 1 -- 172.21.15.32:0/2073504704 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fec1c06be30 msgr2=0x7fec1c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:40.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.696+0000 7fec3f5ce640 1 --2- 172.21.15.32:0/2073504704 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fec1c06be30 0x7fec1c06e2f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fec28006fd0 tx=0x7fec28008040 comp rx=0 tx=0).stop 2024-04-24T22:59:40.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.696+0000 7fec3f5ce640 1 -- 172.21.15.32:0/2073504704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec380b5af0 msgr2=0x7fec3815eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:40.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.696+0000 7fec3f5ce640 1 --2- 172.21.15.32:0/2073504704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec380b5af0 0x7fec3815eb10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fec34000c00 tx=0x7fec340046c0 comp rx=0 tx=0).stop 2024-04-24T22:59:40.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.696+0000 7fec3f5ce640 1 -- 172.21.15.32:0/2073504704 shutdown_connections 2024-04-24T22:59:40.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.696+0000 7fec3f5ce640 1 --2- 172.21.15.32:0/2073504704 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fec1c06be30 0x7fec1c06e2f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:40.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.696+0000 7fec3f5ce640 1 --2- 172.21.15.32:0/2073504704 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec38162c40 0x7fec38160fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:40.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.696+0000 7fec3f5ce640 1 --2- 172.21.15.32:0/2073504704 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec38162320 0x7fec381464d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:40.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.696+0000 7fec3f5ce640 1 --2- 172.21.15.32:0/2073504704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec380b5af0 0x7fec3815eb10 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:40.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.696+0000 7fec3f5ce640 1 -- 172.21.15.32:0/2073504704 >> 172.21.15.32:0/2073504704 conn(0x7fec380d0ae0 msgr2=0x7fec380afbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:40.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.697+0000 7fec3f5ce640 1 -- 172.21.15.32:0/2073504704 shutdown_connections 2024-04-24T22:59:40.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:40.697+0000 7fec3f5ce640 1 -- 172.21.15.32:0/2073504704 wait complete. 2024-04-24T22:59:40.709 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-24T22:59:40.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:59:40.709 DEBUG:teuthology.orchestra.run.smithi121:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-04-24T22:59:40.740 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2024-04-24T22:59:40.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:59:40.741 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-24T22:59:40.807 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T22:59:40.807 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T22:59:40.823 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T22:59:40.823 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T22:59:40.879 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T22:59:40.879 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T22:59:40.936 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:59:41.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.075+0000 7f2a622a5640 1 Processor -- start 2024-04-24T22:59:41.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.075+0000 7f2a622a5640 1 -- start start 2024-04-24T22:59:41.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.076+0000 7f2a622a5640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2a5c06c5f0 0x7f2a5c06c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:41.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.076+0000 7f2a622a5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2a5c0b44d0 0x7f2a5c0b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:41.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.076+0000 7f2a622a5640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a5c0b6430 con 0x7f2a5c0af890 2024-04-24T22:59:41.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.076+0000 7f2a622a5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f2a5c156f80 con 0x7f2a5c0b44d0 2024-04-24T22:59:41.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.076+0000 7f2a622a5640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f2a5c0b6150 con 0x7f2a5c06c5f0 2024-04-24T22:59:41.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.076+0000 7f2a5b7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2a5c0b44d0 0x7f2a5c0b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:41.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.076+0000 7f2a5b7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2a5c0b44d0 0x7f2a5c0b5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57408/0 (socket says 172.21.15.32:57408) 2024-04-24T22:59:41.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.076+0000 7f2a5b7fe640 1 -- 172.21.15.32:0/812658257 learned_addr learned my addr 172.21.15.32:0/812658257 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:41.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.076+0000 7f2a5bfff640 1 --2- 172.21.15.32:0/812658257 >> v2:172.21.15.121:3300/0 conn(0x7f2a5c06c5f0 0x7f2a5c06c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:41.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.077+0000 7f2a5b7fe640 1 -- 172.21.15.32:0/812658257 >> v2:172.21.15.121:3300/0 conn(0x7f2a5c06c5f0 msgr2=0x7f2a5c06c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:41.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.077+0000 7f2a5b7fe640 1 --2- 172.21.15.32:0/812658257 >> v2:172.21.15.121:3300/0 conn(0x7f2a5c06c5f0 0x7f2a5c06c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:41.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.077+0000 7f2a5b7fe640 1 -- 172.21.15.32:0/812658257 >> v1:172.21.15.121:6789/0 conn(0x7f2a5c0af890 legacy=0x7f2a5c0cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:41.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.077+0000 7f2a5b7fe640 1 -- 172.21.15.32:0/812658257 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a5c0b39f0 con 0x7f2a5c0b44d0 2024-04-24T22:59:41.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.077+0000 7f2a5b7fe640 1 --2- 172.21.15.32:0/812658257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a5c0b44d0 0x7f2a5c0b5890 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c06c1f0 tx=0x7f2a4c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fd78a57201a3d1d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:41.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.078+0000 7f2a5affd640 1 -- 172.21.15.32:0/812658257 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c037070 con 0x7f2a5c0b44d0 2024-04-24T22:59:41.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.078+0000 7f2a5affd640 1 -- 172.21.15.32:0/812658257 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a4c032440 con 0x7f2a5c0b44d0 2024-04-24T22:59:41.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.078+0000 7f2a5affd640 1 -- 172.21.15.32:0/812658257 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c032810 con 0x7f2a5c0b44d0 2024-04-24T22:59:41.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.078+0000 7f2a622a5640 1 -- 172.21.15.32:0/812658257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a5c0b44d0 msgr2=0x7f2a5c0b5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:41.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.078+0000 7f2a622a5640 1 --2- 172.21.15.32:0/812658257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a5c0b44d0 0x7f2a5c0b5890 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c06c1f0 tx=0x7f2a4c02a230 comp rx=0 tx=0).stop 2024-04-24T22:59:41.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.078+0000 7f2a622a5640 1 -- 172.21.15.32:0/812658257 shutdown_connections 2024-04-24T22:59:41.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.078+0000 7f2a622a5640 1 --2- 172.21.15.32:0/812658257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a5c0b44d0 0x7f2a5c0b5890 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:41.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.078+0000 7f2a622a5640 1 --2- 172.21.15.32:0/812658257 >> v2:172.21.15.121:3300/0 conn(0x7f2a5c06c5f0 0x7f2a5c06c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:41.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.078+0000 7f2a622a5640 1 -- 172.21.15.32:0/812658257 >> 172.21.15.32:0/812658257 conn(0x7f2a5c06b110 msgr2=0x7f2a5c06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:41.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.078+0000 7f2a622a5640 1 -- 172.21.15.32:0/812658257 shutdown_connections 2024-04-24T22:59:41.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.078+0000 7f2a622a5640 1 -- 172.21.15.32:0/812658257 wait complete. 2024-04-24T22:59:41.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.079+0000 7f2a622a5640 1 Processor -- start 2024-04-24T22:59:41.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.080+0000 7f2a622a5640 1 -- start start 2024-04-24T22:59:41.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.080+0000 7f2a622a5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a5c06c5f0 0x7f2a5c14ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:41.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.080+0000 7f2a622a5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2a5c0af890 0x7f2a5c0cee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:41.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.080+0000 7f2a622a5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2a5c0b44d0 0x7f2a5c0cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:41.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.080+0000 7f2a622a5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c06cf10 con 0x7f2a5c06c5f0 2024-04-24T22:59:41.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.080+0000 7f2a622a5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c0b3fb0 con 0x7f2a5c0b44d0 2024-04-24T22:59:41.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.080+0000 7f2a622a5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f2a5c0b5dd0 con 0x7f2a5c0af890 2024-04-24T22:59:41.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.080+0000 7f2a5bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a5c06c5f0 0x7f2a5c14ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:41.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.080+0000 7f2a5bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a5c06c5f0 0x7f2a5c14ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57412/0 (socket says 172.21.15.32:57412) 2024-04-24T22:59:41.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.080+0000 7f2a5bfff640 1 -- 172.21.15.32:0/3443980679 learned_addr learned my addr 172.21.15.32:0/3443980679 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:41.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.081+0000 7f2a5bfff640 1 -- 172.21.15.32:0/3443980679 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2a5c0af890 msgr2=0x7f2a5c0cee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:41.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.081+0000 7f2a5bfff640 1 --2- 172.21.15.32:0/3443980679 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2a5c0af890 0x7f2a5c0cee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:41.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.081+0000 7f2a5bfff640 1 -- 172.21.15.32:0/3443980679 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2a5c0b44d0 msgr2=0x7f2a5c0cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:41.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.081+0000 7f2a5bfff640 1 --2- 172.21.15.32:0/3443980679 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2a5c0b44d0 0x7f2a5c0cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:41.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.081+0000 7f2a5bfff640 1 -- 172.21.15.32:0/3443980679 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a5c153b30 con 0x7f2a5c06c5f0 2024-04-24T22:59:41.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.081+0000 7f2a5bfff640 1 --2- 172.21.15.32:0/3443980679 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a5c06c5f0 0x7f2a5c14ed00 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2a4400d5f0 tx=0x7f2a4400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:41.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.081+0000 7f2a58ff9640 1 -- 172.21.15.32:0/3443980679 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a440125f0 con 0x7f2a5c06c5f0 2024-04-24T22:59:41.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.081+0000 7f2a58ff9640 1 -- 172.21.15.32:0/3443980679 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a44019070 con 0x7f2a5c06c5f0 2024-04-24T22:59:41.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.081+0000 7f2a622a5640 1 -- 172.21.15.32:0/3443980679 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a5c153e10 con 0x7f2a5c06c5f0 2024-04-24T22:59:41.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.081+0000 7f2a58ff9640 1 -- 172.21.15.32:0/3443980679 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a440129d0 con 0x7f2a5c06c5f0 2024-04-24T22:59:41.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.081+0000 7f2a622a5640 1 -- 172.21.15.32:0/3443980679 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a5c1543d0 con 0x7f2a5c06c5f0 2024-04-24T22:59:41.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.082+0000 7f2a622a5640 1 -- 172.21.15.32:0/3443980679 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a5c068430 con 0x7f2a5c06c5f0 2024-04-24T22:59:41.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.083+0000 7f2a58ff9640 1 -- 172.21.15.32:0/3443980679 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2a44012b70 con 0x7f2a5c06c5f0 2024-04-24T22:59:41.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.083+0000 7f2a58ff9640 1 --2- 172.21.15.32:0/3443980679 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2a3c06be30 0x7f2a3c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:41.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.083+0000 7f2a58ff9640 1 -- 172.21.15.32:0/3443980679 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f2a4408b1e0 con 0x7f2a5c06c5f0 2024-04-24T22:59:41.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.083+0000 7f2a5b7fe640 1 --2- 172.21.15.32:0/3443980679 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2a3c06be30 0x7f2a3c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:41.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.084+0000 7f2a5b7fe640 1 --2- 172.21.15.32:0/3443980679 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2a3c06be30 0x7f2a3c06e2f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c034000 tx=0x7f2a4c02c000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:41.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.087+0000 7f2a58ff9640 1 -- 172.21.15.32:0/3443980679 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2a44059c20 con 0x7f2a5c06c5f0 2024-04-24T22:59:41.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.271+0000 7f2a622a5640 1 -- 172.21.15.32:0/3443980679 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2a5c067150 con 0x7f2a5c06c5f0 2024-04-24T22:59:41.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.272+0000 7f2a58ff9640 1 -- 172.21.15.32:0/3443980679 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16621 (secure 0 0 0) 0x7f2a44059520 con 0x7f2a5c06c5f0 2024-04-24T22:59:41.273 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:59:41.273 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T22:59:05.517554+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:59:02.016639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T22:59:02.846087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:59:41.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.274+0000 7f2a622a5640 1 -- 172.21.15.32:0/3443980679 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2a3c06be30 msgr2=0x7f2a3c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:41.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.274+0000 7f2a622a5640 1 --2- 172.21.15.32:0/3443980679 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2a3c06be30 0x7f2a3c06e2f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c034000 tx=0x7f2a4c02c000 comp rx=0 tx=0).stop 2024-04-24T22:59:41.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.274+0000 7f2a622a5640 1 -- 172.21.15.32:0/3443980679 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a5c06c5f0 msgr2=0x7f2a5c14ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:41.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.274+0000 7f2a622a5640 1 --2- 172.21.15.32:0/3443980679 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a5c06c5f0 0x7f2a5c14ed00 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2a4400d5f0 tx=0x7f2a4400dab0 comp rx=0 tx=0).stop 2024-04-24T22:59:41.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.274+0000 7f2a622a5640 1 -- 172.21.15.32:0/3443980679 shutdown_connections 2024-04-24T22:59:41.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.274+0000 7f2a622a5640 1 --2- 172.21.15.32:0/3443980679 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2a3c06be30 0x7f2a3c06e2f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:41.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.274+0000 7f2a622a5640 1 --2- 172.21.15.32:0/3443980679 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2a5c0b44d0 0x7f2a5c0cf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:41.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.274+0000 7f2a622a5640 1 --2- 172.21.15.32:0/3443980679 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2a5c0af890 0x7f2a5c0cee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:41.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.274+0000 7f2a622a5640 1 --2- 172.21.15.32:0/3443980679 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a5c06c5f0 0x7f2a5c14ed00 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:41.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.274+0000 7f2a622a5640 1 -- 172.21.15.32:0/3443980679 >> 172.21.15.32:0/3443980679 conn(0x7f2a5c06b110 msgr2=0x7f2a5c0ae8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:41.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.274+0000 7f2a622a5640 1 -- 172.21.15.32:0/3443980679 shutdown_connections 2024-04-24T22:59:41.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.274+0000 7f2a622a5640 1 -- 172.21.15.32:0/3443980679 wait complete. 2024-04-24T22:59:41.287 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T22:59:02.016639+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T22:59:02.846087+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T22:59:41.287 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T22:59:41.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.390+0000 7f49a2570640 1 Processor -- start 2024-04-24T22:59:41.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.390+0000 7f49a2570640 1 -- start start 2024-04-24T22:59:41.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.391+0000 7f49a2570640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f499c162320 0x7f499c0ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:41.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.391+0000 7f49a2570640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f499c0cd270 con 0x7f499c162e10 2024-04-24T22:59:41.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.391+0000 7f49a2570640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f499c071610 con 0x7f499c0b5af0 2024-04-24T22:59:41.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.391+0000 7f49a2570640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f499c0b5660 con 0x7f499c162320 2024-04-24T22:59:41.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.391+0000 7f499bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f499c162320 0x7f499c0ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:41.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.391+0000 7f49a0b0d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f499c162e10 0x7f499c15ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52056/0 (socket says 172.21.15.32:52056) 2024-04-24T22:59:41.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.391+0000 7f49a0b0d640 1 -- 172.21.15.32:0/4089632149 learned_addr learned my addr 172.21.15.32:0/4089632149 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:41.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.391+0000 7f499bfff640 1 -- 172.21.15.32:0/4089632149 >> v1:172.21.15.121:6789/0 conn(0x7f499c0b5af0 legacy=0x7f499c162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:41.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.391+0000 7f499bfff640 1 -- 172.21.15.32:0/4089632149 >> v1:172.21.15.32:6789/0 conn(0x7f499c162e10 legacy=0x7f499c15ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:41.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.391+0000 7f499bfff640 1 -- 172.21.15.32:0/4089632149 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f499c0718f0 con 0x7f499c162320 2024-04-24T22:59:41.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.392+0000 7f499bfff640 1 --2- 172.21.15.32:0/4089632149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f499c162320 0x7f499c0ccc90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f499c06c5b0 tx=0x7f498c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23bcaf996266b52e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:41.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.392+0000 7f499affd640 1 -- 172.21.15.32:0/4089632149 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498c037070 con 0x7f499c162320 2024-04-24T22:59:41.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.392+0000 7f499affd640 1 -- 172.21.15.32:0/4089632149 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f498c02ae20 con 0x7f499c162320 2024-04-24T22:59:41.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.392+0000 7f499affd640 1 -- 172.21.15.32:0/4089632149 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498c032660 con 0x7f499c162320 2024-04-24T22:59:41.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.393+0000 7f49a2570640 1 -- 172.21.15.32:0/4089632149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f499c162320 msgr2=0x7f499c0ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:41.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.393+0000 7f49a2570640 1 --2- 172.21.15.32:0/4089632149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f499c162320 0x7f499c0ccc90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f499c06c5b0 tx=0x7f498c02a110 comp rx=0 tx=0).stop 2024-04-24T22:59:41.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.393+0000 7f49a2570640 1 -- 172.21.15.32:0/4089632149 shutdown_connections 2024-04-24T22:59:41.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.393+0000 7f49a2570640 1 --2- 172.21.15.32:0/4089632149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f499c162320 0x7f499c0ccc90 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:41.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.393+0000 7f49a2570640 1 -- 172.21.15.32:0/4089632149 >> 172.21.15.32:0/4089632149 conn(0x7f499c0d0ae0 msgr2=0x7f499c0d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:41.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.393+0000 7f49a2570640 1 -- 172.21.15.32:0/4089632149 shutdown_connections 2024-04-24T22:59:41.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.393+0000 7f49a2570640 1 -- 172.21.15.32:0/4089632149 wait complete. 2024-04-24T22:59:41.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.394+0000 7f49a2570640 1 Processor -- start 2024-04-24T22:59:41.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.394+0000 7f49a2570640 1 -- start start 2024-04-24T22:59:41.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.395+0000 7f49a2570640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f499c0b5af0 0x7f499c161b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:41.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.395+0000 7f49a2570640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f499c162320 0x7f499c1610f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:41.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.395+0000 7f49a2570640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f499c162e10 0x7f499c1f4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:41.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.395+0000 7f49a2570640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f499c071bd0 con 0x7f499c162320 2024-04-24T22:59:41.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.395+0000 7f49a2570640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f499c067950 con 0x7f499c162e10 2024-04-24T22:59:41.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.395+0000 7f49a2570640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f499c071d50 con 0x7f499c0b5af0 2024-04-24T22:59:41.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.395+0000 7f499b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f499c162320 0x7f499c1610f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:41.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.395+0000 7f499b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f499c162320 0x7f499c1610f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57436/0 (socket says 172.21.15.32:57436) 2024-04-24T22:59:41.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.395+0000 7f499b7fe640 1 -- 172.21.15.32:0/3252634536 learned_addr learned my addr 172.21.15.32:0/3252634536 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:41.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.395+0000 7f49a0b0d640 1 --2- 172.21.15.32:0/3252634536 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f499c162e10 0x7f499c1f4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:41.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.395+0000 7f499bfff640 1 --2- 172.21.15.32:0/3252634536 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f499c0b5af0 0x7f499c161b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:41.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.395+0000 7f499b7fe640 1 -- 172.21.15.32:0/3252634536 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f499c0b5af0 msgr2=0x7f499c161b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:41.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.395+0000 7f499b7fe640 1 --2- 172.21.15.32:0/3252634536 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f499c0b5af0 0x7f499c161b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:41.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.395+0000 7f499b7fe640 1 -- 172.21.15.32:0/3252634536 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f499c162e10 msgr2=0x7f499c1f4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:41.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.395+0000 7f499b7fe640 1 --2- 172.21.15.32:0/3252634536 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f499c162e10 0x7f499c1f4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:41.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.395+0000 7f499b7fe640 1 -- 172.21.15.32:0/3252634536 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f499c0cdab0 con 0x7f499c162320 2024-04-24T22:59:41.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.395+0000 7f499b7fe640 1 --2- 172.21.15.32:0/3252634536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f499c162320 0x7f499c1610f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f499000c430 tx=0x7f499000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:41.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.396+0000 7f4998ff9640 1 -- 172.21.15.32:0/3252634536 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4990011790 con 0x7f499c162320 2024-04-24T22:59:41.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.396+0000 7f4998ff9640 1 -- 172.21.15.32:0/3252634536 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4990017070 con 0x7f499c162320 2024-04-24T22:59:41.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.396+0000 7f4998ff9640 1 -- 172.21.15.32:0/3252634536 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4990011bb0 con 0x7f499c162320 2024-04-24T22:59:41.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.396+0000 7f49a2570640 1 -- 172.21.15.32:0/3252634536 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f499c0cdd90 con 0x7f499c162320 2024-04-24T22:59:41.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.396+0000 7f49a2570640 1 -- 172.21.15.32:0/3252634536 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f499c0ce350 con 0x7f499c162320 2024-04-24T22:59:41.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.397+0000 7f49a2570640 1 -- 172.21.15.32:0/3252634536 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f499c068430 con 0x7f499c162320 2024-04-24T22:59:41.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.397+0000 7f4998ff9640 1 -- 172.21.15.32:0/3252634536 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f499001c460 con 0x7f499c162320 2024-04-24T22:59:41.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.397+0000 7f4998ff9640 1 --2- 172.21.15.32:0/3252634536 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f498806be60 0x7f498806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:41.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.397+0000 7f4998ff9640 1 -- 172.21.15.32:0/3252634536 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f499008abd0 con 0x7f499c162320 2024-04-24T22:59:41.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.398+0000 7f499bfff640 1 --2- 172.21.15.32:0/3252634536 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f498806be60 0x7f498806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:41.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.398+0000 7f499bfff640 1 --2- 172.21.15.32:0/3252634536 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f498806be60 0x7f498806e320 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f498c034000 tx=0x7f498c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:41.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.402+0000 7f4998ff9640 1 -- 172.21.15.32:0/3252634536 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4990058ea0 con 0x7f499c162320 2024-04-24T22:59:41.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.583+0000 7f49a2570640 1 -- 172.21.15.32:0/3252634536 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f499c067730 con 0x7f499c162320 2024-04-24T22:59:41.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.583+0000 7f4998ff9640 1 -- 172.21.15.32:0/3252634536 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f499005d1a0 con 0x7f499c162320 2024-04-24T22:59:41.585 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T22:59:41.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.585+0000 7f49a2570640 1 -- 172.21.15.32:0/3252634536 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f498806be60 msgr2=0x7f498806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:41.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.585+0000 7f49a2570640 1 --2- 172.21.15.32:0/3252634536 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f498806be60 0x7f498806e320 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f498c034000 tx=0x7f498c02e000 comp rx=0 tx=0).stop 2024-04-24T22:59:41.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.585+0000 7f49a2570640 1 -- 172.21.15.32:0/3252634536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f499c162320 msgr2=0x7f499c1610f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:41.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.585+0000 7f49a2570640 1 --2- 172.21.15.32:0/3252634536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f499c162320 0x7f499c1610f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f499000c430 tx=0x7f499000c8f0 comp rx=0 tx=0).stop 2024-04-24T22:59:41.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.585+0000 7f49a2570640 1 -- 172.21.15.32:0/3252634536 shutdown_connections 2024-04-24T22:59:41.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.585+0000 7f49a2570640 1 --2- 172.21.15.32:0/3252634536 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f498806be60 0x7f498806e320 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:41.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.585+0000 7f49a2570640 1 --2- 172.21.15.32:0/3252634536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f499c162320 0x7f499c1610f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:41.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.585+0000 7f49a2570640 1 --2- 172.21.15.32:0/3252634536 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f499c162e10 0x7f499c1f4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:41.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.585+0000 7f49a2570640 1 --2- 172.21.15.32:0/3252634536 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f499c0b5af0 0x7f499c161b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:41.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.585+0000 7f49a2570640 1 -- 172.21.15.32:0/3252634536 >> 172.21.15.32:0/3252634536 conn(0x7f499c0d0ae0 msgr2=0x7f499c06cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:41.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.585+0000 7f49a2570640 1 -- 172.21.15.32:0/3252634536 shutdown_connections 2024-04-24T22:59:41.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.585+0000 7f49a2570640 1 -- 172.21.15.32:0/3252634536 wait complete. 2024-04-24T22:59:41.597 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T22:59:41.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.699+0000 7fb97c55a640 1 Processor -- start 2024-04-24T22:59:41.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.699+0000 7fb97c55a640 1 -- start start 2024-04-24T22:59:41.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.700+0000 7fb97c55a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb97406b820 0x7fb9740d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:41.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.700+0000 7fb97c55a640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9740d3170 con 0x7fb9740d4210 2024-04-24T22:59:41.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.700+0000 7fb97c55a640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9740d3450 con 0x7fb97415e9b0 2024-04-24T22:59:41.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.700+0000 7fb97c55a640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fb9740d3730 con 0x7fb97406b820 2024-04-24T22:59:41.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.700+0000 7fb979af5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb97406b820 0x7fb9740d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:41.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.700+0000 7fb97aaf7640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb9740d4210 0x7fb9740b78f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52060/0 (socket says 172.21.15.32:52060) 2024-04-24T22:59:41.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.700+0000 7fb97aaf7640 1 -- 172.21.15.32:0/3125938283 learned_addr learned my addr 172.21.15.32:0/3125938283 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:41.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.701+0000 7fb979af5640 1 -- 172.21.15.32:0/3125938283 >> v1:172.21.15.121:6789/0 conn(0x7fb97415e9b0 legacy=0x7fb97406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:41.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.701+0000 7fb979af5640 1 -- 172.21.15.32:0/3125938283 >> v1:172.21.15.32:6789/0 conn(0x7fb9740d4210 legacy=0x7fb9740b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:41.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.701+0000 7fb979af5640 1 -- 172.21.15.32:0/3125938283 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9740d3a10 con 0x7fb97406b820 2024-04-24T22:59:41.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.701+0000 7fb9792f4640 1 -- 172.21.15.32:0/3125938283 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4146874988 0 0) 0x7fb9740d3170 con 0x7fb9740d4210 2024-04-24T22:59:41.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.701+0000 7fb979af5640 1 --2- 172.21.15.32:0/3125938283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb97406b820 0x7fb9740d2b90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb9680097a0 tx=0x7fb96802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb3f790d7b4ea9d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:41.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.702+0000 7fb9792f4640 1 -- 172.21.15.32:0/3125938283 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb968037070 con 0x7fb97406b820 2024-04-24T22:59:41.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.702+0000 7fb9792f4640 1 -- 172.21.15.32:0/3125938283 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb968032440 con 0x7fb97406b820 2024-04-24T22:59:41.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.702+0000 7fb9792f4640 1 -- 172.21.15.32:0/3125938283 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb968032810 con 0x7fb97406b820 2024-04-24T22:59:41.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.702+0000 7fb97c55a640 1 -- 172.21.15.32:0/3125938283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb97406b820 msgr2=0x7fb9740d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:41.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.702+0000 7fb97c55a640 1 --2- 172.21.15.32:0/3125938283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb97406b820 0x7fb9740d2b90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb9680097a0 tx=0x7fb96802a1a0 comp rx=0 tx=0).stop 2024-04-24T22:59:41.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.702+0000 7fb97c55a640 1 -- 172.21.15.32:0/3125938283 shutdown_connections 2024-04-24T22:59:41.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.702+0000 7fb97c55a640 1 --2- 172.21.15.32:0/3125938283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb97406b820 0x7fb9740d2b90 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:41.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.702+0000 7fb97c55a640 1 -- 172.21.15.32:0/3125938283 >> 172.21.15.32:0/3125938283 conn(0x7fb9740d11d0 msgr2=0x7fb9740d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:41.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.702+0000 7fb97c55a640 1 -- 172.21.15.32:0/3125938283 shutdown_connections 2024-04-24T22:59:41.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.702+0000 7fb97c55a640 1 -- 172.21.15.32:0/3125938283 wait complete. 2024-04-24T22:59:41.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.703+0000 7fb97c55a640 1 Processor -- start 2024-04-24T22:59:41.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.703+0000 7fb97c55a640 1 -- start start 2024-04-24T22:59:41.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.704+0000 7fb97c55a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb97406b820 0x7fb97415a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:41.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.704+0000 7fb97c55a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9740d4210 0x7fb9740ca620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:41.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.704+0000 7fb97c55a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb97415e9b0 0x7fb9740ccd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:41.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.704+0000 7fb97c55a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb9740ba0e0 con 0x7fb97406b820 2024-04-24T22:59:41.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.704+0000 7fb97c55a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb9740d2400 con 0x7fb9740d4210 2024-04-24T22:59:41.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.704+0000 7fb97c55a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fb9740ba260 con 0x7fb97415e9b0 2024-04-24T22:59:41.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.704+0000 7fb97a2f6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb97406b820 0x7fb97415a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:41.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.704+0000 7fb97a2f6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb97406b820 0x7fb97415a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57444/0 (socket says 172.21.15.32:57444) 2024-04-24T22:59:41.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.704+0000 7fb97a2f6640 1 -- 172.21.15.32:0/1624674268 learned_addr learned my addr 172.21.15.32:0/1624674268 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:41.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.704+0000 7fb97aaf7640 1 --2- 172.21.15.32:0/1624674268 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb97415e9b0 0x7fb9740ccd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:41.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.704+0000 7fb979af5640 1 --2- 172.21.15.32:0/1624674268 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9740d4210 0x7fb9740ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:41.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.704+0000 7fb97a2f6640 1 -- 172.21.15.32:0/1624674268 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb97415e9b0 msgr2=0x7fb9740ccd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:41.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.704+0000 7fb97a2f6640 1 --2- 172.21.15.32:0/1624674268 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb97415e9b0 0x7fb9740ccd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:41.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.704+0000 7fb97a2f6640 1 -- 172.21.15.32:0/1624674268 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9740d4210 msgr2=0x7fb9740ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:41.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.704+0000 7fb97a2f6640 1 --2- 172.21.15.32:0/1624674268 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9740d4210 0x7fb9740ca620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:41.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.704+0000 7fb97a2f6640 1 -- 172.21.15.32:0/1624674268 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9740c8320 con 0x7fb97406b820 2024-04-24T22:59:41.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.705+0000 7fb97a2f6640 1 --2- 172.21.15.32:0/1624674268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb97406b820 0x7fb97415a5d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb96400c730 tx=0x7fb96400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:41.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.705+0000 7fb972ffd640 1 -- 172.21.15.32:0/1624674268 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb964003b70 con 0x7fb97406b820 2024-04-24T22:59:41.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.705+0000 7fb972ffd640 1 -- 172.21.15.32:0/1624674268 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb964003d30 con 0x7fb97406b820 2024-04-24T22:59:41.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.705+0000 7fb972ffd640 1 -- 172.21.15.32:0/1624674268 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96400aa90 con 0x7fb97406b820 2024-04-24T22:59:41.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.705+0000 7fb97c55a640 1 -- 172.21.15.32:0/1624674268 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9740cd2a0 con 0x7fb97406b820 2024-04-24T22:59:41.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.705+0000 7fb97c55a640 1 -- 172.21.15.32:0/1624674268 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9740c85c0 con 0x7fb97406b820 2024-04-24T22:59:41.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.706+0000 7fb97c55a640 1 -- 172.21.15.32:0/1624674268 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb974068430 con 0x7fb97406b820 2024-04-24T22:59:41.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.706+0000 7fb972ffd640 1 -- 172.21.15.32:0/1624674268 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb96400ac30 con 0x7fb97406b820 2024-04-24T22:59:41.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.707+0000 7fb972ffd640 1 --2- 172.21.15.32:0/1624674268 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb96006be30 0x7fb96006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:41.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.707+0000 7fb972ffd640 1 -- 172.21.15.32:0/1624674268 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb96408ac50 con 0x7fb97406b820 2024-04-24T22:59:41.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.707+0000 7fb979af5640 1 --2- 172.21.15.32:0/1624674268 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb96006be30 0x7fb96006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:41.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.707+0000 7fb979af5640 1 --2- 172.21.15.32:0/1624674268 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb96006be30 0x7fb96006e2f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb968034000 tx=0x7fb96802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:41.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.711+0000 7fb972ffd640 1 -- 172.21.15.32:0/1624674268 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb964090020 con 0x7fb97406b820 2024-04-24T22:59:41.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.894+0000 7fb97c55a640 1 -- 172.21.15.32:0/1624674268 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb974074df0 con 0x7fb97406b820 2024-04-24T22:59:41.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.894+0000 7fb972ffd640 1 -- 172.21.15.32:0/1624674268 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7fb964058fb0 con 0x7fb97406b820 2024-04-24T22:59:41.896 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-24T22:59:41.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.896+0000 7fb97c55a640 1 -- 172.21.15.32:0/1624674268 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb96006be30 msgr2=0x7fb96006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:41.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.896+0000 7fb97c55a640 1 --2- 172.21.15.32:0/1624674268 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb96006be30 0x7fb96006e2f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb968034000 tx=0x7fb96802e000 comp rx=0 tx=0).stop 2024-04-24T22:59:41.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.896+0000 7fb97c55a640 1 -- 172.21.15.32:0/1624674268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb97406b820 msgr2=0x7fb97415a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:41.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.896+0000 7fb97c55a640 1 --2- 172.21.15.32:0/1624674268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb97406b820 0x7fb97415a5d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb96400c730 tx=0x7fb96400cbf0 comp rx=0 tx=0).stop 2024-04-24T22:59:41.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.896+0000 7fb97c55a640 1 -- 172.21.15.32:0/1624674268 shutdown_connections 2024-04-24T22:59:41.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.896+0000 7fb97c55a640 1 --2- 172.21.15.32:0/1624674268 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb96006be30 0x7fb96006e2f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:41.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.896+0000 7fb97c55a640 1 --2- 172.21.15.32:0/1624674268 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb97415e9b0 0x7fb9740ccd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:41.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.896+0000 7fb97c55a640 1 --2- 172.21.15.32:0/1624674268 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9740d4210 0x7fb9740ca620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:41.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.896+0000 7fb97c55a640 1 --2- 172.21.15.32:0/1624674268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb97406b820 0x7fb97415a5d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:41.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.896+0000 7fb97c55a640 1 -- 172.21.15.32:0/1624674268 >> 172.21.15.32:0/1624674268 conn(0x7fb9740d11d0 msgr2=0x7fb97406b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:41.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.896+0000 7fb97c55a640 1 -- 172.21.15.32:0/1624674268 shutdown_connections 2024-04-24T22:59:41.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:41.896+0000 7fb97c55a640 1 -- 172.21.15.32:0/1624674268 wait complete. 2024-04-24T22:59:41.908 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T22:59:42.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.011+0000 7fec9c170640 1 Processor -- start 2024-04-24T22:59:42.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.011+0000 7fec9c170640 1 -- start start 2024-04-24T22:59:42.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.012+0000 7fec9c170640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec9406ba30 0x7fec9406d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:42.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.012+0000 7fec9c170640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fec9406da90 0x7fec940b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:42.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.012+0000 7fec9c170640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec9415ef10 con 0x7fec94158dd0 2024-04-24T22:59:42.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.012+0000 7fec9c170640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fec9415f1f0 con 0x7fec9406da90 2024-04-24T22:59:42.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.012+0000 7fec9c170640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fec9415f4d0 con 0x7fec9406ba30 2024-04-24T22:59:42.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.012+0000 7fec9a70d640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fec9406da90 0x7fec940b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:42.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.012+0000 7fec9970b640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec9406ba30 0x7fec9406d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:42.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.012+0000 7fec9a70d640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fec9406da90 0x7fec940b5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.32:53240/0 (socket says 172.21.15.32:53240) 2024-04-24T22:59:42.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.012+0000 7fec9970b640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec9406ba30 0x7fec9406d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.32:42260/0 (socket says 172.21.15.32:42260) 2024-04-24T22:59:42.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.012+0000 7fec9a70d640 1 -- 172.21.15.32:0/2634744120 learned_addr learned my addr 172.21.15.32:0/2634744120 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:42.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.012+0000 7fec99f0c640 1 --1- 172.21.15.32:0/2634744120 >> v1:172.21.15.121:6789/0 conn(0x7fec94158dd0 0x7fec9406b320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:43300/0 (socket says 172.21.15.32:43300) 2024-04-24T22:59:42.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.012+0000 7fec9a70d640 1 -- 172.21.15.32:0/2634744120 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec9406ba30 msgr2=0x7fec9406d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:42.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.013+0000 7fec9a70d640 1 --2- 172.21.15.32:0/2634744120 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec9406ba30 0x7fec9406d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:42.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.013+0000 7fec9a70d640 1 -- 172.21.15.32:0/2634744120 >> v1:172.21.15.121:6789/0 conn(0x7fec94158dd0 legacy=0x7fec9406b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:42.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.013+0000 7fec9a70d640 1 -- 172.21.15.32:0/2634744120 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec9415f7b0 con 0x7fec9406da90 2024-04-24T22:59:42.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.013+0000 7fec98f0a640 1 -- 172.21.15.32:0/2634744120 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1786224583 0 0) 0x7fec9415ef10 con 0x7fec94158dd0 2024-04-24T22:59:42.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.013+0000 7fec9970b640 1 --2- 172.21.15.32:0/2634744120 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec9406ba30 0x7fec9406d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:59:42.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.013+0000 7fec9a70d640 1 --2- 172.21.15.32:0/2634744120 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec9406da90 0x7fec940b5d00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fec8c009e50 tx=0x7fec8c02a6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=710afe84cd6d5da5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:42.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.013+0000 7fec98f0a640 1 -- 172.21.15.32:0/2634744120 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8c036070 con 0x7fec9406da90 2024-04-24T22:59:42.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.013+0000 7fec98f0a640 1 -- 172.21.15.32:0/2634744120 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec8c031600 con 0x7fec9406da90 2024-04-24T22:59:42.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.013+0000 7fec98f0a640 1 -- 172.21.15.32:0/2634744120 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8c0319d0 con 0x7fec9406da90 2024-04-24T22:59:42.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.014+0000 7fec9c170640 1 -- 172.21.15.32:0/2634744120 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec9406da90 msgr2=0x7fec940b5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:42.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.014+0000 7fec9c170640 1 --2- 172.21.15.32:0/2634744120 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec9406da90 0x7fec940b5d00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fec8c009e50 tx=0x7fec8c02a6a0 comp rx=0 tx=0).stop 2024-04-24T22:59:42.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.014+0000 7fec9c170640 1 -- 172.21.15.32:0/2634744120 shutdown_connections 2024-04-24T22:59:42.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.014+0000 7fec9c170640 1 --2- 172.21.15.32:0/2634744120 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec9406da90 0x7fec940b5d00 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:42.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.014+0000 7fec9c170640 1 --2- 172.21.15.32:0/2634744120 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec9406ba30 0x7fec9406d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:42.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.014+0000 7fec9c170640 1 -- 172.21.15.32:0/2634744120 >> 172.21.15.32:0/2634744120 conn(0x7fec9406e800 msgr2=0x7fec9406ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:42.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.014+0000 7fec9c170640 1 -- 172.21.15.32:0/2634744120 shutdown_connections 2024-04-24T22:59:42.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.014+0000 7fec9c170640 1 -- 172.21.15.32:0/2634744120 wait complete. 2024-04-24T22:59:42.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.015+0000 7fec9c170640 1 Processor -- start 2024-04-24T22:59:42.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.015+0000 7fec9c170640 1 -- start start 2024-04-24T22:59:42.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.016+0000 7fec9c170640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec9406ba30 0x7fec9414f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:42.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.016+0000 7fec9c170640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec9406da90 0x7fec940b4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:42.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.016+0000 7fec9c170640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec94158dd0 0x7fec940b4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:42.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.016+0000 7fec9c170640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fec9406bfe0 con 0x7fec9406ba30 2024-04-24T22:59:42.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.016+0000 7fec9c170640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fec9406ded0 con 0x7fec9406da90 2024-04-24T22:59:42.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.016+0000 7fec9c170640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fec941591b0 con 0x7fec94158dd0 2024-04-24T22:59:42.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.016+0000 7fec99f0c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec9406ba30 0x7fec9414f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:42.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.016+0000 7fec99f0c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec9406ba30 0x7fec9414f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57446/0 (socket says 172.21.15.32:57446) 2024-04-24T22:59:42.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.016+0000 7fec99f0c640 1 -- 172.21.15.32:0/344536598 learned_addr learned my addr 172.21.15.32:0/344536598 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:42.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.016+0000 7fec9a70d640 1 --2- 172.21.15.32:0/344536598 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec94158dd0 0x7fec940b4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:42.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.016+0000 7fec9970b640 1 --2- 172.21.15.32:0/344536598 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec9406da90 0x7fec940b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:42.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.016+0000 7fec99f0c640 1 -- 172.21.15.32:0/344536598 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec94158dd0 msgr2=0x7fec940b4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:42.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.016+0000 7fec99f0c640 1 --2- 172.21.15.32:0/344536598 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec94158dd0 0x7fec940b4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:42.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.016+0000 7fec99f0c640 1 -- 172.21.15.32:0/344536598 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec9406da90 msgr2=0x7fec940b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:42.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.016+0000 7fec99f0c640 1 --2- 172.21.15.32:0/344536598 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec9406da90 0x7fec940b4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:42.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.016+0000 7fec99f0c640 1 -- 172.21.15.32:0/344536598 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec941439a0 con 0x7fec9406ba30 2024-04-24T22:59:42.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.017+0000 7fec99f0c640 1 --2- 172.21.15.32:0/344536598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec9406ba30 0x7fec9414f8c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fec9000c430 tx=0x7fec9000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:42.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.017+0000 7fec8a7fc640 1 -- 172.21.15.32:0/344536598 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec900074e0 con 0x7fec9406ba30 2024-04-24T22:59:42.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.017+0000 7fec8a7fc640 1 -- 172.21.15.32:0/344536598 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec900047e0 con 0x7fec9406ba30 2024-04-24T22:59:42.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.017+0000 7fec9c170640 1 -- 172.21.15.32:0/344536598 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec94150940 con 0x7fec9406ba30 2024-04-24T22:59:42.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.017+0000 7fec8a7fc640 1 -- 172.21.15.32:0/344536598 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec90004bd0 con 0x7fec9406ba30 2024-04-24T22:59:42.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.017+0000 7fec9c170640 1 -- 172.21.15.32:0/344536598 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec94150660 con 0x7fec9406ba30 2024-04-24T22:59:42.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.018+0000 7fec9c170640 1 -- 172.21.15.32:0/344536598 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec94068430 con 0x7fec9406ba30 2024-04-24T22:59:42.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.018+0000 7fec8a7fc640 1 -- 172.21.15.32:0/344536598 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fec90011070 con 0x7fec9406ba30 2024-04-24T22:59:42.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.019+0000 7fec8a7fc640 1 --2- 172.21.15.32:0/344536598 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fec7806be30 0x7fec7806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:42.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.019+0000 7fec8a7fc640 1 -- 172.21.15.32:0/344536598 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fec9008a8c0 con 0x7fec9406ba30 2024-04-24T22:59:42.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.019+0000 7fec9970b640 1 --2- 172.21.15.32:0/344536598 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fec7806be30 0x7fec7806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:42.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.019+0000 7fec9970b640 1 --2- 172.21.15.32:0/344536598 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fec7806be30 0x7fec7806e2f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fec84009560 tx=0x7fec84009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:42.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.023+0000 7fec8a7fc640 1 -- 172.21.15.32:0/344536598 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fec90058c90 con 0x7fec9406ba30 2024-04-24T22:59:42.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.204+0000 7fec9c170640 1 -- 172.21.15.32:0/344536598 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fec94068a60 con 0x7fec9406ba30 2024-04-24T22:59:42.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.205+0000 7fec8a7fc640 1 -- 172.21.15.32:0/344536598 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7fec9005cf90 con 0x7fec9406ba30 2024-04-24T22:59:42.207 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-24T22:59:42.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.207+0000 7fec9c170640 1 -- 172.21.15.32:0/344536598 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fec7806be30 msgr2=0x7fec7806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:42.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.207+0000 7fec9c170640 1 --2- 172.21.15.32:0/344536598 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fec7806be30 0x7fec7806e2f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fec84009560 tx=0x7fec84009190 comp rx=0 tx=0).stop 2024-04-24T22:59:42.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.207+0000 7fec9c170640 1 -- 172.21.15.32:0/344536598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec9406ba30 msgr2=0x7fec9414f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:42.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.207+0000 7fec9c170640 1 --2- 172.21.15.32:0/344536598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec9406ba30 0x7fec9414f8c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fec9000c430 tx=0x7fec9000c8f0 comp rx=0 tx=0).stop 2024-04-24T22:59:42.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.207+0000 7fec9c170640 1 -- 172.21.15.32:0/344536598 shutdown_connections 2024-04-24T22:59:42.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.207+0000 7fec9c170640 1 --2- 172.21.15.32:0/344536598 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fec7806be30 0x7fec7806e2f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:42.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.207+0000 7fec9c170640 1 --2- 172.21.15.32:0/344536598 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec94158dd0 0x7fec940b4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:42.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.207+0000 7fec9c170640 1 --2- 172.21.15.32:0/344536598 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec9406da90 0x7fec940b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:42.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.207+0000 7fec9c170640 1 --2- 172.21.15.32:0/344536598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec9406ba30 0x7fec9414f8c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:42.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.207+0000 7fec9c170640 1 -- 172.21.15.32:0/344536598 >> 172.21.15.32:0/344536598 conn(0x7fec9406e800 msgr2=0x7fec941626c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:42.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.207+0000 7fec9c170640 1 -- 172.21.15.32:0/344536598 shutdown_connections 2024-04-24T22:59:42.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.207+0000 7fec9c170640 1 -- 172.21.15.32:0/344536598 wait complete. 2024-04-24T22:59:42.219 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T22:59:42.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.320+0000 7fa0a28ee640 1 Processor -- start 2024-04-24T22:59:42.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.320+0000 7fa0a28ee640 1 -- start start 2024-04-24T22:59:42.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.321+0000 7fa0a28ee640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa09c162320 0x7fa09c0ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:42.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.321+0000 7fa0a28ee640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa09c0b5af0 0x7fa09c162700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:42.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.321+0000 7fa0a28ee640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa09c0cd270 con 0x7fa09c162c40 2024-04-24T22:59:42.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.321+0000 7fa0a28ee640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fa09c0b5660 con 0x7fa09c0b5af0 2024-04-24T22:59:42.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.321+0000 7fa0a28ee640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa09c071610 con 0x7fa09c162320 2024-04-24T22:59:42.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.321+0000 7fa09b7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa09c0b5af0 0x7fa09c162700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:42.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.321+0000 7fa09b7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa09c0b5af0 0x7fa09c162700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57456/0 (socket says 172.21.15.32:57456) 2024-04-24T22:59:42.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.321+0000 7fa09b7fe640 1 -- 172.21.15.32:0/3359307245 learned_addr learned my addr 172.21.15.32:0/3359307245 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:42.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.321+0000 7fa09bfff640 1 --2- 172.21.15.32:0/3359307245 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa09c162320 0x7fa09c0ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:42.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.321+0000 7fa09b7fe640 1 -- 172.21.15.32:0/3359307245 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa09c162320 msgr2=0x7fa09c0ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:42.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.321+0000 7fa09b7fe640 1 --2- 172.21.15.32:0/3359307245 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa09c162320 0x7fa09c0ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:42.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.321+0000 7fa09b7fe640 1 -- 172.21.15.32:0/3359307245 >> v1:172.21.15.121:6789/0 conn(0x7fa09c162c40 legacy=0x7fa09c0ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:42.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.321+0000 7fa09b7fe640 1 -- 172.21.15.32:0/3359307245 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa09c0718f0 con 0x7fa09c0b5af0 2024-04-24T22:59:42.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.322+0000 7fa09b7fe640 1 --2- 172.21.15.32:0/3359307245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa09c0b5af0 0x7fa09c162700 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa08c009870 tx=0x7fa08c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=548decbc912c4151 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:42.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.322+0000 7fa09affd640 1 -- 172.21.15.32:0/3359307245 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08c037070 con 0x7fa09c0b5af0 2024-04-24T22:59:42.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.322+0000 7fa09affd640 1 -- 172.21.15.32:0/3359307245 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa08c032440 con 0x7fa09c0b5af0 2024-04-24T22:59:42.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.322+0000 7fa09affd640 1 -- 172.21.15.32:0/3359307245 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08c032810 con 0x7fa09c0b5af0 2024-04-24T22:59:42.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.323+0000 7fa0a28ee640 1 -- 172.21.15.32:0/3359307245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa09c0b5af0 msgr2=0x7fa09c162700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:42.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.323+0000 7fa0a28ee640 1 --2- 172.21.15.32:0/3359307245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa09c0b5af0 0x7fa09c162700 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa08c009870 tx=0x7fa08c02a1c0 comp rx=0 tx=0).stop 2024-04-24T22:59:42.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.323+0000 7fa0a28ee640 1 -- 172.21.15.32:0/3359307245 shutdown_connections 2024-04-24T22:59:42.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.323+0000 7fa0a28ee640 1 --2- 172.21.15.32:0/3359307245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa09c0b5af0 0x7fa09c162700 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:42.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.323+0000 7fa0a28ee640 1 --2- 172.21.15.32:0/3359307245 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa09c162320 0x7fa09c0ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:42.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.323+0000 7fa0a28ee640 1 -- 172.21.15.32:0/3359307245 >> 172.21.15.32:0/3359307245 conn(0x7fa09c0d0ae0 msgr2=0x7fa09c0d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:42.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.323+0000 7fa0a28ee640 1 -- 172.21.15.32:0/3359307245 shutdown_connections 2024-04-24T22:59:42.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.323+0000 7fa0a28ee640 1 -- 172.21.15.32:0/3359307245 wait complete. 2024-04-24T22:59:42.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.324+0000 7fa0a28ee640 1 Processor -- start 2024-04-24T22:59:42.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.324+0000 7fa0a28ee640 1 -- start start 2024-04-24T22:59:42.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.325+0000 7fa0a28ee640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa09c0b5af0 0x7fa09c15a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:42.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.325+0000 7fa0a28ee640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa09c162320 0x7fa09c0cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:42.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.325+0000 7fa0a28ee640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa09c162c40 0x7fa09c160eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:42.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.325+0000 7fa0a28ee640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa09c067950 con 0x7fa09c162c40 2024-04-24T22:59:42.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.325+0000 7fa0a28ee640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa09c15c630 con 0x7fa09c0b5af0 2024-04-24T22:59:42.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.325+0000 7fa0a28ee640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa09c071bd0 con 0x7fa09c162320 2024-04-24T22:59:42.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.325+0000 7fa0a0e8b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa09c162c40 0x7fa09c160eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:42.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.325+0000 7fa0a0e8b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa09c162c40 0x7fa09c160eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57458/0 (socket says 172.21.15.32:57458) 2024-04-24T22:59:42.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.325+0000 7fa0a0e8b640 1 -- 172.21.15.32:0/4173674107 learned_addr learned my addr 172.21.15.32:0/4173674107 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:42.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.325+0000 7fa09b7fe640 1 --2- 172.21.15.32:0/4173674107 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa09c162320 0x7fa09c0cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:42.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.325+0000 7fa09bfff640 1 --2- 172.21.15.32:0/4173674107 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa09c0b5af0 0x7fa09c15a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:42.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.325+0000 7fa0a0e8b640 1 -- 172.21.15.32:0/4173674107 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa09c162320 msgr2=0x7fa09c0cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:42.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.325+0000 7fa0a0e8b640 1 --2- 172.21.15.32:0/4173674107 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa09c162320 0x7fa09c0cd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:42.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.325+0000 7fa0a0e8b640 1 -- 172.21.15.32:0/4173674107 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa09c0b5af0 msgr2=0x7fa09c15a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:42.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.325+0000 7fa0a0e8b640 1 --2- 172.21.15.32:0/4173674107 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa09c0b5af0 0x7fa09c15a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:42.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.325+0000 7fa0a0e8b640 1 -- 172.21.15.32:0/4173674107 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa09c1e8270 con 0x7fa09c162c40 2024-04-24T22:59:42.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.325+0000 7fa09b7fe640 1 --2- 172.21.15.32:0/4173674107 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa09c162320 0x7fa09c0cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:42.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.325+0000 7fa09bfff640 1 --2- 172.21.15.32:0/4173674107 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa09c0b5af0 0x7fa09c15a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:42.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.326+0000 7fa0a0e8b640 1 --2- 172.21.15.32:0/4173674107 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa09c162c40 0x7fa09c160eb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa09400c6b0 tx=0x7fa09400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:42.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.326+0000 7fa098ff9640 1 -- 172.21.15.32:0/4173674107 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa094003b70 con 0x7fa09c162c40 2024-04-24T22:59:42.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.326+0000 7fa098ff9640 1 -- 172.21.15.32:0/4173674107 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa094003d30 con 0x7fa09c162c40 2024-04-24T22:59:42.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.326+0000 7fa0a28ee640 1 -- 172.21.15.32:0/4173674107 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa09c1e8550 con 0x7fa09c162c40 2024-04-24T22:59:42.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.326+0000 7fa098ff9640 1 -- 172.21.15.32:0/4173674107 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa094005310 con 0x7fa09c162c40 2024-04-24T22:59:42.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.326+0000 7fa0a28ee640 1 -- 172.21.15.32:0/4173674107 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa09c1e8b10 con 0x7fa09c162c40 2024-04-24T22:59:42.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.327+0000 7fa0a28ee640 1 -- 172.21.15.32:0/4173674107 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa09c068430 con 0x7fa09c162c40 2024-04-24T22:59:42.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.327+0000 7fa098ff9640 1 -- 172.21.15.32:0/4173674107 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa0940054b0 con 0x7fa09c162c40 2024-04-24T22:59:42.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.328+0000 7fa098ff9640 1 --2- 172.21.15.32:0/4173674107 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa07c06be30 0x7fa07c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:42.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.328+0000 7fa098ff9640 1 -- 172.21.15.32:0/4173674107 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa09408ac80 con 0x7fa09c162c40 2024-04-24T22:59:42.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.328+0000 7fa09bfff640 1 --2- 172.21.15.32:0/4173674107 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa07c06be30 0x7fa07c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:42.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.328+0000 7fa09bfff640 1 --2- 172.21.15.32:0/4173674107 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa07c06be30 0x7fa07c06e2f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa09c15b8e0 tx=0x7fa084008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:42.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.332+0000 7fa098ff9640 1 -- 172.21.15.32:0/4173674107 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa0940596c0 con 0x7fa09c162c40 2024-04-24T22:59:42.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.525+0000 7fa0a28ee640 1 -- 172.21.15.32:0/4173674107 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa09c0ccfb0 con 0x7fa09c162c40 2024-04-24T22:59:42.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.526+0000 7fa098ff9640 1 -- 172.21.15.32:0/4173674107 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7fa094058fc0 con 0x7fa09c162c40 2024-04-24T22:59:42.527 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2024-04-24T22:59:42.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.528+0000 7fa0a28ee640 1 -- 172.21.15.32:0/4173674107 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa07c06be30 msgr2=0x7fa07c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:42.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.528+0000 7fa0a28ee640 1 --2- 172.21.15.32:0/4173674107 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa07c06be30 0x7fa07c06e2f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa09c15b8e0 tx=0x7fa084008040 comp rx=0 tx=0).stop 2024-04-24T22:59:42.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.528+0000 7fa0a28ee640 1 -- 172.21.15.32:0/4173674107 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa09c162c40 msgr2=0x7fa09c160eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:42.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.528+0000 7fa0a28ee640 1 --2- 172.21.15.32:0/4173674107 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa09c162c40 0x7fa09c160eb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa09400c6b0 tx=0x7fa09400cb70 comp rx=0 tx=0).stop 2024-04-24T22:59:42.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.528+0000 7fa0a28ee640 1 -- 172.21.15.32:0/4173674107 shutdown_connections 2024-04-24T22:59:42.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.528+0000 7fa0a28ee640 1 --2- 172.21.15.32:0/4173674107 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa07c06be30 0x7fa07c06e2f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:42.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.528+0000 7fa0a28ee640 1 --2- 172.21.15.32:0/4173674107 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa09c162c40 0x7fa09c160eb0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:42.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.528+0000 7fa0a28ee640 1 --2- 172.21.15.32:0/4173674107 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa09c162320 0x7fa09c0cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:42.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.528+0000 7fa0a28ee640 1 --2- 172.21.15.32:0/4173674107 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa09c0b5af0 0x7fa09c15a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:42.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.528+0000 7fa0a28ee640 1 -- 172.21.15.32:0/4173674107 >> 172.21.15.32:0/4173674107 conn(0x7fa09c0d0ae0 msgr2=0x7fa09c0b1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:42.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.528+0000 7fa0a28ee640 1 -- 172.21.15.32:0/4173674107 shutdown_connections 2024-04-24T22:59:42.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.528+0000 7fa0a28ee640 1 -- 172.21.15.32:0/4173674107 wait complete. 2024-04-24T22:59:42.540 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-04-24T22:59:42.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:59:42.540 DEBUG:teuthology.orchestra.run.smithi032:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-04-24T22:59:42.566 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2024-04-24T22:59:42.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:59:42.567 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-24T22:59:42.626 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T22:59:42.626 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T22:59:42.644 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T22:59:42.645 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T22:59:42.702 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T22:59:42.702 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T22:59:42.760 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:59:42.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.861+0000 7f50db163640 1 Processor -- start 2024-04-24T22:59:42.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.861+0000 7f50db163640 1 -- start start 2024-04-24T22:59:42.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.861+0000 7f50db163640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f50d4162320 0x7f50d40ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:42.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.861+0000 7f50db163640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f50d4162e10 0x7f50d415ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:42.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.861+0000 7f50db163640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50d40cd270 con 0x7f50d40b5af0 2024-04-24T22:59:42.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.861+0000 7f50db163640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f50d40b5660 con 0x7f50d4162320 2024-04-24T22:59:42.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.861+0000 7f50db163640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f50d4071610 con 0x7f50d4162e10 2024-04-24T22:59:42.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.862+0000 7f50d3fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f50d40b5af0 0x7f50d4162700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52070/0 (socket says 172.21.15.32:52070) 2024-04-24T22:59:42.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.862+0000 7f50d3fff640 1 -- 172.21.15.32:0/2734981693 learned_addr learned my addr 172.21.15.32:0/2734981693 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:42.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.862+0000 7f50d9700640 1 --2- 172.21.15.32:0/2734981693 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f50d4162e10 0x7f50d415ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:42.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.862+0000 7f50d8eff640 1 --2- 172.21.15.32:0/2734981693 >> v2:172.21.15.121:3300/0 conn(0x7f50d4162320 0x7f50d40ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:42.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.862+0000 7f50d37fe640 1 -- 172.21.15.32:0/2734981693 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1547423892 0 0) 0x7f50d40cd270 con 0x7f50d40b5af0 2024-04-24T22:59:42.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.862+0000 7f50d37fe640 1 -- 172.21.15.32:0/2734981693 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f50c4003520 con 0x7f50d40b5af0 2024-04-24T22:59:42.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.862+0000 7f50d37fe640 1 -- 172.21.15.32:0/2734981693 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3640364921 0 0) 0x7f50c4003520 con 0x7f50d40b5af0 2024-04-24T22:59:42.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.862+0000 7f50d37fe640 1 -- 172.21.15.32:0/2734981693 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f50d4162e10 msgr2=0x7f50d415ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:42.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.862+0000 7f50d37fe640 1 --2- 172.21.15.32:0/2734981693 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f50d4162e10 0x7f50d415ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:42.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.862+0000 7f50d37fe640 1 -- 172.21.15.32:0/2734981693 >> v2:172.21.15.121:3300/0 conn(0x7f50d4162320 msgr2=0x7f50d40ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:42.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.862+0000 7f50d37fe640 1 --2- 172.21.15.32:0/2734981693 >> v2:172.21.15.121:3300/0 conn(0x7f50d4162320 0x7f50d40ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:42.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.862+0000 7f50d37fe640 1 -- 172.21.15.32:0/2734981693 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50d40718f0 con 0x7f50d40b5af0 2024-04-24T22:59:42.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.862+0000 7f50d37fe640 1 -- 172.21.15.32:0/2734981693 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f50c0001670 con 0x7f50d40b5af0 2024-04-24T22:59:42.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.862+0000 7f50d37fe640 1 -- 172.21.15.32:0/2734981693 >> v1:172.21.15.32:6789/0 conn(0x7f50d40b5af0 legacy=0x7f50d4162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:42.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.862+0000 7f50d37fe640 1 --2- 172.21.15.32:0/2734981693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50c40039b0 0x7f50c4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:42.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.862+0000 7f50d37fe640 1 --2- 172.21.15.32:0/2734981693 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f50d4162e10 0x7f50c4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:42.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.862+0000 7f50d37fe640 1 --2- 172.21.15.32:0/2734981693 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50c4004d40 0x7f50c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:42.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.862+0000 7f50d37fe640 1 -- 172.21.15.32:0/2734981693 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f50d40b5660 con 0x7f50c40039b0 2024-04-24T22:59:42.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.862+0000 7f50d37fe640 1 -- 172.21.15.32:0/2734981693 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f50d4071610 con 0x7f50c4004d40 2024-04-24T22:59:42.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.862+0000 7f50d37fe640 1 -- 172.21.15.32:0/2734981693 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f50c400ba40 con 0x7f50d4162e10 2024-04-24T22:59:42.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.862+0000 7f50d8eff640 1 --2- 172.21.15.32:0/2734981693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50c40039b0 0x7f50c4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:42.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.863+0000 7f50d3fff640 1 --2- 172.21.15.32:0/2734981693 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f50d4162e10 0x7f50c4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:42.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.863+0000 7f50d9700640 1 --2- 172.21.15.32:0/2734981693 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50c4004d40 0x7f50c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:42.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.863+0000 7f50d8eff640 1 -- 172.21.15.32:0/2734981693 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f50d4162e10 msgr2=0x7f50c4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:42.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.863+0000 7f50d8eff640 1 --2- 172.21.15.32:0/2734981693 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f50d4162e10 0x7f50c4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:42.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.863+0000 7f50d8eff640 1 -- 172.21.15.32:0/2734981693 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50c4004d40 msgr2=0x7f50c40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:42.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.863+0000 7f50d8eff640 1 --2- 172.21.15.32:0/2734981693 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50c4004d40 0x7f50c40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:42.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.863+0000 7f50d8eff640 1 -- 172.21.15.32:0/2734981693 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50c4003520 con 0x7f50c40039b0 2024-04-24T22:59:42.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.863+0000 7f50d9700640 1 --2- 172.21.15.32:0/2734981693 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50c4004d40 0x7f50c40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:42.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.863+0000 7f50d8eff640 1 --2- 172.21.15.32:0/2734981693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50c40039b0 0x7f50c4003dd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f50c8009ac0 tx=0x7f50c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9e8d65a2301b5ac server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:42.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.864+0000 7f50d37fe640 1 -- 172.21.15.32:0/2734981693 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c8036070 con 0x7f50c40039b0 2024-04-24T22:59:42.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.864+0000 7f50d37fe640 1 -- 172.21.15.32:0/2734981693 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50c8031400 con 0x7f50c40039b0 2024-04-24T22:59:42.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.864+0000 7f50d37fe640 1 -- 172.21.15.32:0/2734981693 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c8009690 con 0x7f50c40039b0 2024-04-24T22:59:42.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.864+0000 7f50db163640 1 -- 172.21.15.32:0/2734981693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50c40039b0 msgr2=0x7f50c4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:42.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.864+0000 7f50db163640 1 --2- 172.21.15.32:0/2734981693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50c40039b0 0x7f50c4003dd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f50c8009ac0 tx=0x7f50c4005830 comp rx=0 tx=0).stop 2024-04-24T22:59:42.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.864+0000 7f50d9700640 1 -- 172.21.15.32:0/2734981693 reap_dead start 2024-04-24T22:59:42.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.864+0000 7f50db163640 1 -- 172.21.15.32:0/2734981693 shutdown_connections 2024-04-24T22:59:42.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.864+0000 7f50db163640 1 -- 172.21.15.32:0/2734981693 >> 172.21.15.32:0/2734981693 conn(0x7f50d40d0ae0 msgr2=0x7f50d40d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:42.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.864+0000 7f50db163640 1 -- 172.21.15.32:0/2734981693 shutdown_connections 2024-04-24T22:59:42.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.864+0000 7f50db163640 1 -- 172.21.15.32:0/2734981693 wait complete. 2024-04-24T22:59:42.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.865+0000 7f50db163640 1 Processor -- start 2024-04-24T22:59:42.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.865+0000 7f50db163640 1 -- start start 2024-04-24T22:59:42.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.866+0000 7f50db163640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f50c40039b0 0x7f50d40cf370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:42.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.866+0000 7f50db163640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50d4149690 0x7f50d415a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:42.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.866+0000 7f50db163640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50d415aba0 0x7f50d40cd560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:42.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.866+0000 7f50db163640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f50d4071bd0 con 0x7f50d4149690 2024-04-24T22:59:42.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.866+0000 7f50db163640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f50d4067950 con 0x7f50d415aba0 2024-04-24T22:59:42.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.866+0000 7f50db163640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f50d4071d50 con 0x7f50c40039b0 2024-04-24T22:59:42.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.866+0000 7f50d3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50d4149690 0x7f50d415a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:42.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.866+0000 7f50d3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50d4149690 0x7f50d415a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57474/0 (socket says 172.21.15.32:57474) 2024-04-24T22:59:42.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.866+0000 7f50d3fff640 1 -- 172.21.15.32:0/3920385619 learned_addr learned my addr 172.21.15.32:0/3920385619 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:42.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.866+0000 7f50d9700640 1 --2- 172.21.15.32:0/3920385619 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50d415aba0 0x7f50d40cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:42.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.866+0000 7f50d8eff640 1 --2- 172.21.15.32:0/3920385619 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f50c40039b0 0x7f50d40cf370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:42.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.867+0000 7f50d3fff640 1 -- 172.21.15.32:0/3920385619 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f50c40039b0 msgr2=0x7f50d40cf370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:42.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.867+0000 7f50d3fff640 1 --2- 172.21.15.32:0/3920385619 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f50c40039b0 0x7f50d40cf370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:42.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.867+0000 7f50d3fff640 1 -- 172.21.15.32:0/3920385619 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50d415aba0 msgr2=0x7f50d40cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:42.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.867+0000 7f50d3fff640 1 --2- 172.21.15.32:0/3920385619 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50d415aba0 0x7f50d40cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:42.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.867+0000 7f50d3fff640 1 -- 172.21.15.32:0/3920385619 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50d40ce2d0 con 0x7f50d4149690 2024-04-24T22:59:42.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.867+0000 7f50d8eff640 1 --2- 172.21.15.32:0/3920385619 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f50c40039b0 0x7f50d40cf370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:42.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.867+0000 7f50d9700640 1 --2- 172.21.15.32:0/3920385619 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50d415aba0 0x7f50d40cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:42.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.867+0000 7f50d3fff640 1 --2- 172.21.15.32:0/3920385619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50d4149690 0x7f50d415a660 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f50c0000c00 tx=0x7f50c00077a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:42.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.867+0000 7f50d17fa640 1 -- 172.21.15.32:0/3920385619 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c000f960 con 0x7f50d4149690 2024-04-24T22:59:42.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.867+0000 7f50d17fa640 1 -- 172.21.15.32:0/3920385619 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50c0018070 con 0x7f50d4149690 2024-04-24T22:59:42.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.867+0000 7f50db163640 1 -- 172.21.15.32:0/3920385619 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50d40ce5b0 con 0x7f50d4149690 2024-04-24T22:59:42.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.867+0000 7f50d17fa640 1 -- 172.21.15.32:0/3920385619 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c000fd60 con 0x7f50d4149690 2024-04-24T22:59:42.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.867+0000 7f50db163640 1 -- 172.21.15.32:0/3920385619 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50d40ceb70 con 0x7f50d4149690 2024-04-24T22:59:42.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.868+0000 7f50db163640 1 -- 172.21.15.32:0/3920385619 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50d4068430 con 0x7f50d4149690 2024-04-24T22:59:42.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.868+0000 7f50d17fa640 1 -- 172.21.15.32:0/3920385619 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f50c001d460 con 0x7f50d4149690 2024-04-24T22:59:42.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.869+0000 7f50d17fa640 1 --2- 172.21.15.32:0/3920385619 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f50b806be30 0x7f50b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:42.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.869+0000 7f50d17fa640 1 -- 172.21.15.32:0/3920385619 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f50c008ae50 con 0x7f50d4149690 2024-04-24T22:59:42.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.869+0000 7f50d8eff640 1 --2- 172.21.15.32:0/3920385619 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f50b806be30 0x7f50b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:42.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.870+0000 7f50d8eff640 1 --2- 172.21.15.32:0/3920385619 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f50b806be30 0x7f50b806e2f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f50c8033000 tx=0x7f50c8008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:42.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:42.873+0000 7f50d17fa640 1 -- 172.21.15.32:0/3920385619 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f50c0059890 con 0x7f50d4149690 2024-04-24T22:59:43.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.058+0000 7f50db163640 1 -- 172.21.15.32:0/3920385619 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f50d415af80 con 0x7f50d4149690 2024-04-24T22:59:43.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.059+0000 7f50d17fa640 1 -- 172.21.15.32:0/3920385619 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16621 (secure 0 0 0) 0x7f50c0059190 con 0x7f50d4149690 2024-04-24T22:59:43.060 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:59:43.060 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T22:59:05.517554+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:59:02.016639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T22:59:02.846087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:59:43.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.061+0000 7f50db163640 1 -- 172.21.15.32:0/3920385619 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f50b806be30 msgr2=0x7f50b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:43.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.061+0000 7f50db163640 1 --2- 172.21.15.32:0/3920385619 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f50b806be30 0x7f50b806e2f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f50c8033000 tx=0x7f50c8008030 comp rx=0 tx=0).stop 2024-04-24T22:59:43.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.061+0000 7f50db163640 1 -- 172.21.15.32:0/3920385619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50d4149690 msgr2=0x7f50d415a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:43.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.061+0000 7f50db163640 1 --2- 172.21.15.32:0/3920385619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50d4149690 0x7f50d415a660 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f50c0000c00 tx=0x7f50c00077a0 comp rx=0 tx=0).stop 2024-04-24T22:59:43.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.061+0000 7f50db163640 1 -- 172.21.15.32:0/3920385619 shutdown_connections 2024-04-24T22:59:43.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.061+0000 7f50db163640 1 --2- 172.21.15.32:0/3920385619 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f50b806be30 0x7f50b806e2f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:43.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.061+0000 7f50db163640 1 --2- 172.21.15.32:0/3920385619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50d4149690 0x7f50d415a660 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:43.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.061+0000 7f50db163640 1 --2- 172.21.15.32:0/3920385619 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50d415aba0 0x7f50d40cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:43.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.061+0000 7f50db163640 1 --2- 172.21.15.32:0/3920385619 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f50c40039b0 0x7f50d40cf370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:43.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.061+0000 7f50db163640 1 -- 172.21.15.32:0/3920385619 >> 172.21.15.32:0/3920385619 conn(0x7f50d40d0ae0 msgr2=0x7f50d40b3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:43.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.061+0000 7f50db163640 1 -- 172.21.15.32:0/3920385619 shutdown_connections 2024-04-24T22:59:43.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.061+0000 7f50db163640 1 -- 172.21.15.32:0/3920385619 wait complete. 2024-04-24T22:59:43.073 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T22:59:02.016639+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T22:59:02.846087+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T22:59:43.074 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T22:59:43.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.176+0000 7f0274d89640 1 Processor -- start 2024-04-24T22:59:43.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.176+0000 7f0274d89640 1 -- start start 2024-04-24T22:59:43.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.177+0000 7f0274d89640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0270162320 0x7f02700ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:43.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.177+0000 7f0274d89640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02700b5af0 0x7f0270162700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:43.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.177+0000 7f0274d89640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0270162c40 0x7f02700ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:43.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.177+0000 7f0274d89640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f02700b5660 con 0x7f0270162320 2024-04-24T22:59:43.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.177+0000 7f0274d89640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f02700cd270 con 0x7f0270162c40 2024-04-24T22:59:43.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.177+0000 7f0274d89640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f0270071610 con 0x7f02700b5af0 2024-04-24T22:59:43.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.177+0000 7f026e59c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0270162320 0x7f02700ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:43.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.177+0000 7f026e59c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0270162320 0x7f02700ccc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57484/0 (socket says 172.21.15.32:57484) 2024-04-24T22:59:43.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.177+0000 7f026e59c640 1 -- 172.21.15.32:0/720652660 learned_addr learned my addr 172.21.15.32:0/720652660 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:43.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.177+0000 7f026dd9b640 1 --2- 172.21.15.32:0/720652660 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02700b5af0 0x7f0270162700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:43.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.177+0000 7f026ed9d640 1 --2- 172.21.15.32:0/720652660 >> v2:172.21.15.121:3300/0 conn(0x7f0270162c40 0x7f02700ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:43.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.177+0000 7f026e59c640 1 -- 172.21.15.32:0/720652660 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02700b5af0 msgr2=0x7f0270162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:43.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.177+0000 7f026e59c640 1 --2- 172.21.15.32:0/720652660 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02700b5af0 0x7f0270162700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:43.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.177+0000 7f026e59c640 1 -- 172.21.15.32:0/720652660 >> v2:172.21.15.121:3300/0 conn(0x7f0270162c40 msgr2=0x7f02700ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:43.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.177+0000 7f026e59c640 1 --2- 172.21.15.32:0/720652660 >> v2:172.21.15.121:3300/0 conn(0x7f0270162c40 0x7f02700ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:43.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.177+0000 7f026e59c640 1 -- 172.21.15.32:0/720652660 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02700718f0 con 0x7f0270162320 2024-04-24T22:59:43.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.178+0000 7f026e59c640 1 --2- 172.21.15.32:0/720652660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0270162320 0x7f02700ccc90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f02700b3ac0 tx=0x7f026802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=200e16e2f9c92926 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:43.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.178+0000 7f026d59a640 1 -- 172.21.15.32:0/720652660 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0268037070 con 0x7f0270162320 2024-04-24T22:59:43.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.178+0000 7f026d59a640 1 -- 172.21.15.32:0/720652660 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0268032440 con 0x7f0270162320 2024-04-24T22:59:43.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.178+0000 7f026d59a640 1 -- 172.21.15.32:0/720652660 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0268032810 con 0x7f0270162320 2024-04-24T22:59:43.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.179+0000 7f0274d89640 1 -- 172.21.15.32:0/720652660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0270162320 msgr2=0x7f02700ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:43.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.179+0000 7f0274d89640 1 --2- 172.21.15.32:0/720652660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0270162320 0x7f02700ccc90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f02700b3ac0 tx=0x7f026802a140 comp rx=0 tx=0).stop 2024-04-24T22:59:43.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.179+0000 7f0274d89640 1 -- 172.21.15.32:0/720652660 shutdown_connections 2024-04-24T22:59:43.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.179+0000 7f0274d89640 1 --2- 172.21.15.32:0/720652660 >> v2:172.21.15.121:3300/0 conn(0x7f0270162c40 0x7f02700ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:43.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.179+0000 7f0274d89640 1 --2- 172.21.15.32:0/720652660 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02700b5af0 0x7f0270162700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:43.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.179+0000 7f0274d89640 1 --2- 172.21.15.32:0/720652660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0270162320 0x7f02700ccc90 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:43.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.179+0000 7f0274d89640 1 -- 172.21.15.32:0/720652660 >> 172.21.15.32:0/720652660 conn(0x7f02700d0ae0 msgr2=0x7f02700d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:43.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.179+0000 7f0274d89640 1 -- 172.21.15.32:0/720652660 shutdown_connections 2024-04-24T22:59:43.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.179+0000 7f0274d89640 1 -- 172.21.15.32:0/720652660 wait complete. 2024-04-24T22:59:43.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.180+0000 7f0274d89640 1 Processor -- start 2024-04-24T22:59:43.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.180+0000 7f0274d89640 1 -- start start 2024-04-24T22:59:43.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.181+0000 7f0274d89640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02700b5af0 0x7f027015eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:43.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.181+0000 7f0274d89640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0270162320 0x7f02701464d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:43.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.181+0000 7f0274d89640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0270162c40 0x7f0270160fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:43.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.181+0000 7f0274d89640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0270067950 con 0x7f0270162320 2024-04-24T22:59:43.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.181+0000 7f0274d89640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f027015c630 con 0x7f02700b5af0 2024-04-24T22:59:43.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.181+0000 7f0274d89640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f0270071bd0 con 0x7f0270162c40 2024-04-24T22:59:43.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.181+0000 7f026dd9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0270162320 0x7f02701464d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:43.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.181+0000 7f026dd9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0270162320 0x7f02701464d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57496/0 (socket says 172.21.15.32:57496) 2024-04-24T22:59:43.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.181+0000 7f026dd9b640 1 -- 172.21.15.32:0/1299931653 learned_addr learned my addr 172.21.15.32:0/1299931653 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:43.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.181+0000 7f026ed9d640 1 --2- 172.21.15.32:0/1299931653 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0270162c40 0x7f0270160fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:43.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.181+0000 7f026e59c640 1 --2- 172.21.15.32:0/1299931653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02700b5af0 0x7f027015eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:43.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.181+0000 7f026dd9b640 1 -- 172.21.15.32:0/1299931653 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0270162c40 msgr2=0x7f0270160fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:43.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.181+0000 7f026dd9b640 1 --2- 172.21.15.32:0/1299931653 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0270162c40 0x7f0270160fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:43.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.181+0000 7f026dd9b640 1 -- 172.21.15.32:0/1299931653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02700b5af0 msgr2=0x7f027015eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:43.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.181+0000 7f026dd9b640 1 --2- 172.21.15.32:0/1299931653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02700b5af0 0x7f027015eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:43.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.181+0000 7f026dd9b640 1 -- 172.21.15.32:0/1299931653 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f027015f260 con 0x7f0270162320 2024-04-24T22:59:43.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.181+0000 7f026e59c640 1 --2- 172.21.15.32:0/1299931653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02700b5af0 0x7f027015eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:43.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.182+0000 7f026dd9b640 1 --2- 172.21.15.32:0/1299931653 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0270162320 0x7f02701464d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f026400d830 tx=0x7f026400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:43.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.182+0000 7f025effd640 1 -- 172.21.15.32:0/1299931653 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0264018030 con 0x7f0270162320 2024-04-24T22:59:43.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.182+0000 7f025effd640 1 -- 172.21.15.32:0/1299931653 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f026400be10 con 0x7f0270162320 2024-04-24T22:59:43.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.182+0000 7f025effd640 1 -- 172.21.15.32:0/1299931653 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02640098b0 con 0x7f0270162320 2024-04-24T22:59:43.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.182+0000 7f0274d89640 1 -- 172.21.15.32:0/1299931653 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f027015f540 con 0x7f0270162320 2024-04-24T22:59:43.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.182+0000 7f0274d89640 1 -- 172.21.15.32:0/1299931653 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f027015fb00 con 0x7f0270162320 2024-04-24T22:59:43.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.183+0000 7f0274d89640 1 -- 172.21.15.32:0/1299931653 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0270068430 con 0x7f0270162320 2024-04-24T22:59:43.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.183+0000 7f025effd640 1 -- 172.21.15.32:0/1299931653 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f026401b070 con 0x7f0270162320 2024-04-24T22:59:43.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.184+0000 7f025effd640 1 --2- 172.21.15.32:0/1299931653 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f025006be30 0x7f025006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:43.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.184+0000 7f025effd640 1 -- 172.21.15.32:0/1299931653 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f026408aff0 con 0x7f0270162320 2024-04-24T22:59:43.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.184+0000 7f026e59c640 1 --2- 172.21.15.32:0/1299931653 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f025006be30 0x7f025006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:43.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.184+0000 7f026e59c640 1 --2- 172.21.15.32:0/1299931653 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f025006be30 0x7f025006e2f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0268034000 tx=0x7f026802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:43.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.188+0000 7f025effd640 1 -- 172.21.15.32:0/1299931653 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f02640599a0 con 0x7f0270162320 2024-04-24T22:59:43.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.369+0000 7f0274d89640 1 -- 172.21.15.32:0/1299931653 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0270067730 con 0x7f0270162320 2024-04-24T22:59:43.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.370+0000 7f025effd640 1 -- 172.21.15.32:0/1299931653 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f02640592a0 con 0x7f0270162320 2024-04-24T22:59:43.372 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T22:59:43.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.372+0000 7f0274d89640 1 -- 172.21.15.32:0/1299931653 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f025006be30 msgr2=0x7f025006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:43.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.372+0000 7f0274d89640 1 --2- 172.21.15.32:0/1299931653 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f025006be30 0x7f025006e2f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0268034000 tx=0x7f026802e000 comp rx=0 tx=0).stop 2024-04-24T22:59:43.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.372+0000 7f0274d89640 1 -- 172.21.15.32:0/1299931653 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0270162320 msgr2=0x7f02701464d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:43.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.372+0000 7f0274d89640 1 --2- 172.21.15.32:0/1299931653 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0270162320 0x7f02701464d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f026400d830 tx=0x7f026400dcf0 comp rx=0 tx=0).stop 2024-04-24T22:59:43.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.372+0000 7f0274d89640 1 -- 172.21.15.32:0/1299931653 shutdown_connections 2024-04-24T22:59:43.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.372+0000 7f0274d89640 1 --2- 172.21.15.32:0/1299931653 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f025006be30 0x7f025006e2f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:43.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.372+0000 7f0274d89640 1 --2- 172.21.15.32:0/1299931653 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0270162320 0x7f02701464d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:43.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.372+0000 7f0274d89640 1 --2- 172.21.15.32:0/1299931653 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0270162c40 0x7f0270160fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:43.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.372+0000 7f0274d89640 1 --2- 172.21.15.32:0/1299931653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02700b5af0 0x7f027015eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:43.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.372+0000 7f0274d89640 1 -- 172.21.15.32:0/1299931653 >> 172.21.15.32:0/1299931653 conn(0x7f02700d0ae0 msgr2=0x7f02700afbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:43.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.372+0000 7f0274d89640 1 -- 172.21.15.32:0/1299931653 shutdown_connections 2024-04-24T22:59:43.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.372+0000 7f0274d89640 1 -- 172.21.15.32:0/1299931653 wait complete. 2024-04-24T22:59:43.384 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T22:59:43.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.487+0000 7f1844112640 1 Processor -- start 2024-04-24T22:59:43.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.487+0000 7f1844112640 1 -- start start 2024-04-24T22:59:43.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.488+0000 7f1844112640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f183c0b5560 0x7f183c15e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:43.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.488+0000 7f1844112640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f183c0b6050 0x7f183c15ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:43.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.488+0000 7f1844112640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f183c0cd910 con 0x7f183c0cd3c0 2024-04-24T22:59:43.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.488+0000 7f1844112640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f183c0cdbf0 con 0x7f183c0b5560 2024-04-24T22:59:43.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.488+0000 7f1844112640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f183c0cded0 con 0x7f183c0b6050 2024-04-24T22:59:43.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.488+0000 7f1841eae640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f183c0b5560 0x7f183c15e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:43.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.488+0000 7f1841eae640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f183c0b5560 0x7f183c15e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57500/0 (socket says 172.21.15.32:57500) 2024-04-24T22:59:43.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.488+0000 7f1841eae640 1 -- 172.21.15.32:0/186174394 learned_addr learned my addr 172.21.15.32:0/186174394 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:43.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.488+0000 7f18426af640 1 --2- 172.21.15.32:0/186174394 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f183c0b6050 0x7f183c15ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:43.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.488+0000 7f1841eae640 1 -- 172.21.15.32:0/186174394 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f183c0b6050 msgr2=0x7f183c15ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:43.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.488+0000 7f1841eae640 1 --2- 172.21.15.32:0/186174394 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f183c0b6050 0x7f183c15ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:43.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.488+0000 7f1841eae640 1 -- 172.21.15.32:0/186174394 >> v1:172.21.15.121:6789/0 conn(0x7f183c0cd3c0 legacy=0x7f183c0b5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:43.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.488+0000 7f1841eae640 1 -- 172.21.15.32:0/186174394 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f183c0ce1b0 con 0x7f183c0b5560 2024-04-24T22:59:43.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.488+0000 7f18426af640 1 --2- 172.21.15.32:0/186174394 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f183c0b6050 0x7f183c15ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:43.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.489+0000 7f1841eae640 1 --2- 172.21.15.32:0/186174394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f183c0b5560 0x7f183c15e6c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1838009870 tx=0x7f183802b380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83582b7215ba3739 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:43.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.489+0000 7f1840eac640 1 -- 172.21.15.32:0/186174394 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1838038070 con 0x7f183c0b5560 2024-04-24T22:59:43.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.489+0000 7f1840eac640 1 -- 172.21.15.32:0/186174394 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1838033640 con 0x7f183c0b5560 2024-04-24T22:59:43.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.489+0000 7f1840eac640 1 -- 172.21.15.32:0/186174394 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1838033a10 con 0x7f183c0b5560 2024-04-24T22:59:43.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.489+0000 7f1844112640 1 -- 172.21.15.32:0/186174394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f183c0b5560 msgr2=0x7f183c15e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:43.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.489+0000 7f1844112640 1 --2- 172.21.15.32:0/186174394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f183c0b5560 0x7f183c15e6c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1838009870 tx=0x7f183802b380 comp rx=0 tx=0).stop 2024-04-24T22:59:43.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.490+0000 7f1844112640 1 -- 172.21.15.32:0/186174394 shutdown_connections 2024-04-24T22:59:43.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.490+0000 7f1844112640 1 --2- 172.21.15.32:0/186174394 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f183c0b6050 0x7f183c15ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:43.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.490+0000 7f1844112640 1 --2- 172.21.15.32:0/186174394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f183c0b5560 0x7f183c15e6c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:43.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.490+0000 7f1844112640 1 -- 172.21.15.32:0/186174394 >> 172.21.15.32:0/186174394 conn(0x7f183c06b110 msgr2=0x7f183c06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:43.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.490+0000 7f1844112640 1 -- 172.21.15.32:0/186174394 shutdown_connections 2024-04-24T22:59:43.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.490+0000 7f1844112640 1 -- 172.21.15.32:0/186174394 wait complete. 2024-04-24T22:59:43.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.491+0000 7f1844112640 1 Processor -- start 2024-04-24T22:59:43.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.491+0000 7f1844112640 1 -- start start 2024-04-24T22:59:43.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.491+0000 7f1844112640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f183c0b5560 0x7f183c0713b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:43.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.491+0000 7f1844112640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f183c0b6050 0x7f183c070270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:43.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.491+0000 7f1844112640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f183c0cd3c0 0x7f183c14f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:43.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.491+0000 7f1844112640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f183c06c5f0 con 0x7f183c0cd3c0 2024-04-24T22:59:43.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.491+0000 7f1844112640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f183c0b6460 con 0x7f183c0b6050 2024-04-24T22:59:43.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.491+0000 7f1844112640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f183c06c770 con 0x7f183c0b5560 2024-04-24T22:59:43.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.492+0000 7f18426af640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f183c0cd3c0 0x7f183c14f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:43.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.492+0000 7f18426af640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f183c0cd3c0 0x7f183c14f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57514/0 (socket says 172.21.15.32:57514) 2024-04-24T22:59:43.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.492+0000 7f18426af640 1 -- 172.21.15.32:0/235199739 learned_addr learned my addr 172.21.15.32:0/235199739 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:43.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.492+0000 7f1841eae640 1 --2- 172.21.15.32:0/235199739 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f183c0b5560 0x7f183c0713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:43.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.492+0000 7f18416ad640 1 --2- 172.21.15.32:0/235199739 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f183c0b6050 0x7f183c070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:43.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.492+0000 7f18426af640 1 -- 172.21.15.32:0/235199739 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f183c0b5560 msgr2=0x7f183c0713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:43.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.492+0000 7f18426af640 1 --2- 172.21.15.32:0/235199739 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f183c0b5560 0x7f183c0713b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:43.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.492+0000 7f18426af640 1 -- 172.21.15.32:0/235199739 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f183c0b6050 msgr2=0x7f183c070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:43.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.492+0000 7f18426af640 1 --2- 172.21.15.32:0/235199739 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f183c0b6050 0x7f183c070270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:43.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.492+0000 7f18426af640 1 -- 172.21.15.32:0/235199739 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f183c072b80 con 0x7f183c0cd3c0 2024-04-24T22:59:43.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.492+0000 7f18426af640 1 --2- 172.21.15.32:0/235199739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f183c0cd3c0 0x7f183c14f250 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1834009fa0 tx=0x7f183400d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:43.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.492+0000 7f18327fc640 1 -- 172.21.15.32:0/235199739 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183401a070 con 0x7f183c0cd3c0 2024-04-24T22:59:43.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.492+0000 7f18327fc640 1 -- 172.21.15.32:0/235199739 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1834003bb0 con 0x7f183c0cd3c0 2024-04-24T22:59:43.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.492+0000 7f1844112640 1 -- 172.21.15.32:0/235199739 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f183c072e60 con 0x7f183c0cd3c0 2024-04-24T22:59:43.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.492+0000 7f18327fc640 1 -- 172.21.15.32:0/235199739 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1834004030 con 0x7f183c0cd3c0 2024-04-24T22:59:43.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.492+0000 7f1844112640 1 -- 172.21.15.32:0/235199739 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f183c070eb0 con 0x7f183c0cd3c0 2024-04-24T22:59:43.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.493+0000 7f1844112640 1 -- 172.21.15.32:0/235199739 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f183c068430 con 0x7f183c0cd3c0 2024-04-24T22:59:43.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.494+0000 7f18327fc640 1 -- 172.21.15.32:0/235199739 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f183401d070 con 0x7f183c0cd3c0 2024-04-24T22:59:43.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.494+0000 7f18327fc640 1 --2- 172.21.15.32:0/235199739 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f182006be30 0x7f182006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:43.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.494+0000 7f18327fc640 1 -- 172.21.15.32:0/235199739 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f183408b0d0 con 0x7f183c0cd3c0 2024-04-24T22:59:43.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.495+0000 7f1841eae640 1 --2- 172.21.15.32:0/235199739 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f182006be30 0x7f182006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:43.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.495+0000 7f1841eae640 1 --2- 172.21.15.32:0/235199739 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f182006be30 0x7f182006e2f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1838035000 tx=0x7f183802f000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:43.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.499+0000 7f18327fc640 1 -- 172.21.15.32:0/235199739 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1834010070 con 0x7f183c0cd3c0 2024-04-24T22:59:43.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.685+0000 7f1844112640 1 -- 172.21.15.32:0/235199739 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f183c073030 con 0x7f183c0cd3c0 2024-04-24T22:59:43.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.685+0000 7f18327fc640 1 -- 172.21.15.32:0/235199739 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f1834059430 con 0x7f183c0cd3c0 2024-04-24T22:59:43.687 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-24T22:59:43.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.687+0000 7f1844112640 1 -- 172.21.15.32:0/235199739 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f182006be30 msgr2=0x7f182006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:43.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.687+0000 7f1844112640 1 --2- 172.21.15.32:0/235199739 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f182006be30 0x7f182006e2f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1838035000 tx=0x7f183802f000 comp rx=0 tx=0).stop 2024-04-24T22:59:43.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.687+0000 7f1844112640 1 -- 172.21.15.32:0/235199739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f183c0cd3c0 msgr2=0x7f183c14f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:43.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.687+0000 7f1844112640 1 --2- 172.21.15.32:0/235199739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f183c0cd3c0 0x7f183c14f250 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1834009fa0 tx=0x7f183400d560 comp rx=0 tx=0).stop 2024-04-24T22:59:43.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.687+0000 7f1844112640 1 -- 172.21.15.32:0/235199739 shutdown_connections 2024-04-24T22:59:43.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.687+0000 7f1844112640 1 --2- 172.21.15.32:0/235199739 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f182006be30 0x7f182006e2f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:43.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.687+0000 7f1844112640 1 --2- 172.21.15.32:0/235199739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f183c0cd3c0 0x7f183c14f250 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:43.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.687+0000 7f1844112640 1 --2- 172.21.15.32:0/235199739 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f183c0b6050 0x7f183c070270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:43.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.687+0000 7f1844112640 1 --2- 172.21.15.32:0/235199739 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f183c0b5560 0x7f183c0713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:43.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.687+0000 7f1844112640 1 -- 172.21.15.32:0/235199739 >> 172.21.15.32:0/235199739 conn(0x7f183c06b110 msgr2=0x7f183c14c770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:43.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.688+0000 7f1844112640 1 -- 172.21.15.32:0/235199739 shutdown_connections 2024-04-24T22:59:43.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.688+0000 7f1844112640 1 -- 172.21.15.32:0/235199739 wait complete. 2024-04-24T22:59:43.700 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T22:59:43.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.801+0000 7fbe19bc8640 1 Processor -- start 2024-04-24T22:59:43.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.801+0000 7fbe19bc8640 1 -- start start 2024-04-24T22:59:43.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.802+0000 7fbe19bc8640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbe1406b820 0x7fbe140d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:43.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.802+0000 7fbe19bc8640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fbe140d4210 0x7fbe140b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:43.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.802+0000 7fbe19bc8640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe140d3170 con 0x7fbe1415e9b0 2024-04-24T22:59:43.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.802+0000 7fbe19bc8640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fbe140d3450 con 0x7fbe1406b820 2024-04-24T22:59:43.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.802+0000 7fbe19bc8640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fbe140d3730 con 0x7fbe140d4210 2024-04-24T22:59:43.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.802+0000 7fbe12ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbe1406b820 0x7fbe140d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:43.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.802+0000 7fbe12ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbe1406b820 0x7fbe140d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57526/0 (socket says 172.21.15.32:57526) 2024-04-24T22:59:43.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.802+0000 7fbe12ffd640 1 -- 172.21.15.32:0/666846754 learned_addr learned my addr 172.21.15.32:0/666846754 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:43.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.802+0000 7fbe13fff640 1 --2- 172.21.15.32:0/666846754 >> v2:172.21.15.121:3300/0 conn(0x7fbe140d4210 0x7fbe140b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:43.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.803+0000 7fbe12ffd640 1 -- 172.21.15.32:0/666846754 >> v2:172.21.15.121:3300/0 conn(0x7fbe140d4210 msgr2=0x7fbe140b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:43.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.803+0000 7fbe12ffd640 1 --2- 172.21.15.32:0/666846754 >> v2:172.21.15.121:3300/0 conn(0x7fbe140d4210 0x7fbe140b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:43.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.803+0000 7fbe12ffd640 1 -- 172.21.15.32:0/666846754 >> v1:172.21.15.121:6789/0 conn(0x7fbe1415e9b0 legacy=0x7fbe1406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:43.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.803+0000 7fbe12ffd640 1 -- 172.21.15.32:0/666846754 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe140d3a10 con 0x7fbe1406b820 2024-04-24T22:59:43.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.803+0000 7fbe12ffd640 1 --2- 172.21.15.32:0/666846754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe1406b820 0x7fbe140d2b90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fbe141622f0 tx=0x7fbe0802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb4728aaec6505e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:43.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.804+0000 7fbe127fc640 1 -- 172.21.15.32:0/666846754 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe08037070 con 0x7fbe1406b820 2024-04-24T22:59:43.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.804+0000 7fbe127fc640 1 -- 172.21.15.32:0/666846754 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe08032440 con 0x7fbe1406b820 2024-04-24T22:59:43.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.804+0000 7fbe127fc640 1 -- 172.21.15.32:0/666846754 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe08032810 con 0x7fbe1406b820 2024-04-24T22:59:43.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.804+0000 7fbe19bc8640 1 -- 172.21.15.32:0/666846754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe1406b820 msgr2=0x7fbe140d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:43.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.804+0000 7fbe19bc8640 1 --2- 172.21.15.32:0/666846754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe1406b820 0x7fbe140d2b90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fbe141622f0 tx=0x7fbe0802a1e0 comp rx=0 tx=0).stop 2024-04-24T22:59:43.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.804+0000 7fbe19bc8640 1 -- 172.21.15.32:0/666846754 shutdown_connections 2024-04-24T22:59:43.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.804+0000 7fbe19bc8640 1 --2- 172.21.15.32:0/666846754 >> v2:172.21.15.121:3300/0 conn(0x7fbe140d4210 0x7fbe140b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:43.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.804+0000 7fbe19bc8640 1 --2- 172.21.15.32:0/666846754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe1406b820 0x7fbe140d2b90 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:43.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.804+0000 7fbe19bc8640 1 -- 172.21.15.32:0/666846754 >> 172.21.15.32:0/666846754 conn(0x7fbe140d11d0 msgr2=0x7fbe140d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:43.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.804+0000 7fbe19bc8640 1 -- 172.21.15.32:0/666846754 shutdown_connections 2024-04-24T22:59:43.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.804+0000 7fbe19bc8640 1 -- 172.21.15.32:0/666846754 wait complete. 2024-04-24T22:59:43.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.805+0000 7fbe19bc8640 1 Processor -- start 2024-04-24T22:59:43.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.805+0000 7fbe19bc8640 1 -- start start 2024-04-24T22:59:43.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.806+0000 7fbe19bc8640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbe1406b820 0x7fbe14158fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:43.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.806+0000 7fbe19bc8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe140d4210 0x7fbe140cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:43.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.806+0000 7fbe19bc8640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe1415e9b0 0x7fbe141f3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:43.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.806+0000 7fbe19bc8640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbe140ba0e0 con 0x7fbe140d4210 2024-04-24T22:59:43.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.806+0000 7fbe19bc8640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fbe140d2400 con 0x7fbe1415e9b0 2024-04-24T22:59:43.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.806+0000 7fbe19bc8640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fbe140ba260 con 0x7fbe1406b820 2024-04-24T22:59:43.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.806+0000 7fbe12ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe140d4210 0x7fbe140cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:43.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.806+0000 7fbe12ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe140d4210 0x7fbe140cd0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57532/0 (socket says 172.21.15.32:57532) 2024-04-24T22:59:43.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.806+0000 7fbe12ffd640 1 -- 172.21.15.32:0/1883880309 learned_addr learned my addr 172.21.15.32:0/1883880309 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:43.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.806+0000 7fbe13fff640 1 --2- 172.21.15.32:0/1883880309 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe1415e9b0 0x7fbe141f3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:43.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.806+0000 7fbe137fe640 1 --2- 172.21.15.32:0/1883880309 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbe1406b820 0x7fbe14158fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:43.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.807+0000 7fbe12ffd640 1 -- 172.21.15.32:0/1883880309 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbe1406b820 msgr2=0x7fbe14158fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:43.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.807+0000 7fbe12ffd640 1 --2- 172.21.15.32:0/1883880309 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbe1406b820 0x7fbe14158fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:43.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.807+0000 7fbe12ffd640 1 -- 172.21.15.32:0/1883880309 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe1415e9b0 msgr2=0x7fbe141f3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:43.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.807+0000 7fbe12ffd640 1 --2- 172.21.15.32:0/1883880309 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe1415e9b0 0x7fbe141f3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:43.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.807+0000 7fbe12ffd640 1 -- 172.21.15.32:0/1883880309 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe141581f0 con 0x7fbe140d4210 2024-04-24T22:59:43.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.807+0000 7fbe12ffd640 1 --2- 172.21.15.32:0/1883880309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe140d4210 0x7fbe140cd0a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbe0802a220 tx=0x7fbe080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:43.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.807+0000 7fbdfbfff640 1 -- 172.21.15.32:0/1883880309 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe08037040 con 0x7fbe140d4210 2024-04-24T22:59:43.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.807+0000 7fbdfbfff640 1 -- 172.21.15.32:0/1883880309 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe08042070 con 0x7fbe140d4210 2024-04-24T22:59:43.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.807+0000 7fbe19bc8640 1 -- 172.21.15.32:0/1883880309 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe141584d0 con 0x7fbe140d4210 2024-04-24T22:59:43.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.807+0000 7fbdfbfff640 1 -- 172.21.15.32:0/1883880309 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe08030880 con 0x7fbe140d4210 2024-04-24T22:59:43.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.807+0000 7fbe19bc8640 1 -- 172.21.15.32:0/1883880309 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe14158a90 con 0x7fbe140d4210 2024-04-24T22:59:43.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.808+0000 7fbe19bc8640 1 -- 172.21.15.32:0/1883880309 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe14068430 con 0x7fbe140d4210 2024-04-24T22:59:43.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.809+0000 7fbdfbfff640 1 -- 172.21.15.32:0/1883880309 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbe08030a20 con 0x7fbe140d4210 2024-04-24T22:59:43.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.809+0000 7fbdfbfff640 1 --2- 172.21.15.32:0/1883880309 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fbe0006be30 0x7fbe0006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:43.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.809+0000 7fbdfbfff640 1 -- 172.21.15.32:0/1883880309 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fbe080ab650 con 0x7fbe140d4210 2024-04-24T22:59:43.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.809+0000 7fbe137fe640 1 --2- 172.21.15.32:0/1883880309 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fbe0006be30 0x7fbe0006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:43.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.810+0000 7fbe137fe640 1 --2- 172.21.15.32:0/1883880309 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fbe0006be30 0x7fbe0006e2f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbe040016e0 tx=0x7fbe04009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:43.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.813+0000 7fbdfbfff640 1 -- 172.21.15.32:0/1883880309 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbe08079920 con 0x7fbe140d4210 2024-04-24T22:59:43.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.995+0000 7fbe19bc8640 1 -- 172.21.15.32:0/1883880309 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbe14074df0 con 0x7fbe140d4210 2024-04-24T22:59:43.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.996+0000 7fbdfbfff640 1 -- 172.21.15.32:0/1883880309 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7fbe0807dc20 con 0x7fbe140d4210 2024-04-24T22:59:43.998 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-24T22:59:43.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.998+0000 7fbe19bc8640 1 -- 172.21.15.32:0/1883880309 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fbe0006be30 msgr2=0x7fbe0006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:44.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.998+0000 7fbe19bc8640 1 --2- 172.21.15.32:0/1883880309 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fbe0006be30 0x7fbe0006e2f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbe040016e0 tx=0x7fbe04009040 comp rx=0 tx=0).stop 2024-04-24T22:59:44.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.998+0000 7fbe19bc8640 1 -- 172.21.15.32:0/1883880309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe140d4210 msgr2=0x7fbe140cd0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:44.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.998+0000 7fbe19bc8640 1 --2- 172.21.15.32:0/1883880309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe140d4210 0x7fbe140cd0a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbe0802a220 tx=0x7fbe080046c0 comp rx=0 tx=0).stop 2024-04-24T22:59:44.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.998+0000 7fbe19bc8640 1 -- 172.21.15.32:0/1883880309 shutdown_connections 2024-04-24T22:59:44.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.998+0000 7fbe19bc8640 1 --2- 172.21.15.32:0/1883880309 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fbe0006be30 0x7fbe0006e2f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:44.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.998+0000 7fbe19bc8640 1 --2- 172.21.15.32:0/1883880309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe140d4210 0x7fbe140cd0a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:44.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.998+0000 7fbe19bc8640 1 --2- 172.21.15.32:0/1883880309 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe1415e9b0 0x7fbe141f3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:44.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.998+0000 7fbe19bc8640 1 --2- 172.21.15.32:0/1883880309 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbe1406b820 0x7fbe14158fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:44.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.998+0000 7fbe19bc8640 1 -- 172.21.15.32:0/1883880309 >> 172.21.15.32:0/1883880309 conn(0x7fbe140d11d0 msgr2=0x7fbe140b7bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:44.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.998+0000 7fbe19bc8640 1 -- 172.21.15.32:0/1883880309 shutdown_connections 2024-04-24T22:59:44.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:43.998+0000 7fbe19bc8640 1 -- 172.21.15.32:0/1883880309 wait complete. 2024-04-24T22:59:44.010 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T22:59:44.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.113+0000 7ff050c6d640 1 Processor -- start 2024-04-24T22:59:44.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.113+0000 7ff050c6d640 1 -- start start 2024-04-24T22:59:44.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.113+0000 7ff050c6d640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff04c15e9b0 0x7ff04c0d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:44.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.113+0000 7ff050c6d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff04c06b820 0x7ff04c0b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:44.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.113+0000 7ff050c6d640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff04c0d3170 con 0x7ff04c0d4210 2024-04-24T22:59:44.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.113+0000 7ff050c6d640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7ff04c0d3450 con 0x7ff04c15e9b0 2024-04-24T22:59:44.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.113+0000 7ff050c6d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff04c0d3730 con 0x7ff04c06b820 2024-04-24T22:59:44.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.114+0000 7ff04ad9d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff04c06b820 0x7ff04c0b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:44.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.114+0000 7ff04a59c640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff04c15e9b0 0x7ff04c0d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:44.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.114+0000 7ff043fff640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7ff04c0d4210 0x7ff04c06b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:43336/0 (socket says 172.21.15.32:43336) 2024-04-24T22:59:44.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.114+0000 7ff043fff640 1 -- 172.21.15.32:0/2801500041 learned_addr learned my addr 172.21.15.32:0/2801500041 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:44.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.114+0000 7ff04a59c640 1 -- 172.21.15.32:0/2801500041 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff04c06b820 msgr2=0x7ff04c0b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:44.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.114+0000 7ff04a59c640 1 --2- 172.21.15.32:0/2801500041 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff04c06b820 0x7ff04c0b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:44.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.114+0000 7ff04a59c640 1 -- 172.21.15.32:0/2801500041 >> v1:172.21.15.121:6789/0 conn(0x7ff04c0d4210 legacy=0x7ff04c06b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:44.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.114+0000 7ff04a59c640 1 -- 172.21.15.32:0/2801500041 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff04c0d3a10 con 0x7ff04c15e9b0 2024-04-24T22:59:44.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.114+0000 7ff049d9b640 1 -- 172.21.15.32:0/2801500041 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 270393358 0 0) 0x7ff04c0d3170 con 0x7ff04c0d4210 2024-04-24T22:59:44.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.114+0000 7ff04ad9d640 1 --2- 172.21.15.32:0/2801500041 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff04c06b820 0x7ff04c0b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:59:44.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.115+0000 7ff04a59c640 1 --2- 172.21.15.32:0/2801500041 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff04c15e9b0 0x7ff04c0d2b90 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff03c0097a0 tx=0x7ff03c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b0e7de6bc9df72e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:44.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.115+0000 7ff049d9b640 1 -- 172.21.15.32:0/2801500041 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03c037070 con 0x7ff04c15e9b0 2024-04-24T22:59:44.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.115+0000 7ff049d9b640 1 -- 172.21.15.32:0/2801500041 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff03c032440 con 0x7ff04c15e9b0 2024-04-24T22:59:44.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.115+0000 7ff049d9b640 1 -- 172.21.15.32:0/2801500041 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03c032810 con 0x7ff04c15e9b0 2024-04-24T22:59:44.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.115+0000 7ff050c6d640 1 -- 172.21.15.32:0/2801500041 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff04c15e9b0 msgr2=0x7ff04c0d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:44.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.115+0000 7ff050c6d640 1 --2- 172.21.15.32:0/2801500041 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff04c15e9b0 0x7ff04c0d2b90 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff03c0097a0 tx=0x7ff03c02a180 comp rx=0 tx=0).stop 2024-04-24T22:59:44.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.115+0000 7ff050c6d640 1 -- 172.21.15.32:0/2801500041 shutdown_connections 2024-04-24T22:59:44.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.115+0000 7ff050c6d640 1 --2- 172.21.15.32:0/2801500041 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff04c06b820 0x7ff04c0b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:44.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.115+0000 7ff050c6d640 1 --2- 172.21.15.32:0/2801500041 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff04c15e9b0 0x7ff04c0d2b90 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:44.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.115+0000 7ff050c6d640 1 -- 172.21.15.32:0/2801500041 >> 172.21.15.32:0/2801500041 conn(0x7ff04c0d11d0 msgr2=0x7ff04c0d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:44.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.116+0000 7ff050c6d640 1 -- 172.21.15.32:0/2801500041 shutdown_connections 2024-04-24T22:59:44.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.116+0000 7ff050c6d640 1 -- 172.21.15.32:0/2801500041 wait complete. 2024-04-24T22:59:44.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.117+0000 7ff050c6d640 1 Processor -- start 2024-04-24T22:59:44.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.117+0000 7ff050c6d640 1 -- start start 2024-04-24T22:59:44.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.118+0000 7ff050c6d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff04c06b820 0x7ff04c0cd240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:44.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.118+0000 7ff050c6d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff04c0d4210 0x7ff04c0b7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:44.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.118+0000 7ff050c6d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff04c15e9b0 0x7ff04c14baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:44.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.118+0000 7ff050c6d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff04c0ba0e0 con 0x7ff04c0d4210 2024-04-24T22:59:44.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.118+0000 7ff050c6d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff04c0d2400 con 0x7ff04c06b820 2024-04-24T22:59:44.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.118+0000 7ff050c6d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff04c0ba260 con 0x7ff04c15e9b0 2024-04-24T22:59:44.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.118+0000 7ff043fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff04c0d4210 0x7ff04c0b7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:44.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.118+0000 7ff043fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff04c0d4210 0x7ff04c0b7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57548/0 (socket says 172.21.15.32:57548) 2024-04-24T22:59:44.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.118+0000 7ff043fff640 1 -- 172.21.15.32:0/1236031170 learned_addr learned my addr 172.21.15.32:0/1236031170 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:44.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.118+0000 7ff04a59c640 1 --2- 172.21.15.32:0/1236031170 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff04c06b820 0x7ff04c0cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:44.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.118+0000 7ff04ad9d640 1 --2- 172.21.15.32:0/1236031170 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff04c15e9b0 0x7ff04c14baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:44.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.118+0000 7ff043fff640 1 -- 172.21.15.32:0/1236031170 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff04c15e9b0 msgr2=0x7ff04c14baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:44.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.118+0000 7ff043fff640 1 --2- 172.21.15.32:0/1236031170 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff04c15e9b0 0x7ff04c14baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:44.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.118+0000 7ff043fff640 1 -- 172.21.15.32:0/1236031170 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff04c06b820 msgr2=0x7ff04c0cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:44.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.118+0000 7ff043fff640 1 --2- 172.21.15.32:0/1236031170 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff04c06b820 0x7ff04c0cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:44.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.118+0000 7ff043fff640 1 -- 172.21.15.32:0/1236031170 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff04c0b8290 con 0x7ff04c0d4210 2024-04-24T22:59:44.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.118+0000 7ff043fff640 1 --2- 172.21.15.32:0/1236031170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff04c0d4210 0x7ff04c0b7d50 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff03400c500 tx=0x7ff03400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:44.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.118+0000 7ff042ffd640 1 -- 172.21.15.32:0/1236031170 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff034018070 con 0x7ff04c0d4210 2024-04-24T22:59:44.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.118+0000 7ff042ffd640 1 -- 172.21.15.32:0/1236031170 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff034007580 con 0x7ff04c0d4210 2024-04-24T22:59:44.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.118+0000 7ff050c6d640 1 -- 172.21.15.32:0/1236031170 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff04c0cd8d0 con 0x7ff04c0d4210 2024-04-24T22:59:44.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.118+0000 7ff042ffd640 1 -- 172.21.15.32:0/1236031170 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff034004a20 con 0x7ff04c0d4210 2024-04-24T22:59:44.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.118+0000 7ff050c6d640 1 -- 172.21.15.32:0/1236031170 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff04c0cde90 con 0x7ff04c0d4210 2024-04-24T22:59:44.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.120+0000 7ff050c6d640 1 -- 172.21.15.32:0/1236031170 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff04c068430 con 0x7ff04c0d4210 2024-04-24T22:59:44.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.120+0000 7ff042ffd640 1 -- 172.21.15.32:0/1236031170 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff03401b070 con 0x7ff04c0d4210 2024-04-24T22:59:44.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.120+0000 7ff042ffd640 1 --2- 172.21.15.32:0/1236031170 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff02c06be30 0x7ff02c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:44.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.120+0000 7ff042ffd640 1 -- 172.21.15.32:0/1236031170 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff03408a020 con 0x7ff04c0d4210 2024-04-24T22:59:44.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.121+0000 7ff04a59c640 1 --2- 172.21.15.32:0/1236031170 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff02c06be30 0x7ff02c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:44.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.121+0000 7ff04a59c640 1 --2- 172.21.15.32:0/1236031170 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff02c06be30 0x7ff02c06e2f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff03c034000 tx=0x7ff03c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:44.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.125+0000 7ff042ffd640 1 -- 172.21.15.32:0/1236031170 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff034058410 con 0x7ff04c0d4210 2024-04-24T22:59:44.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.320+0000 7ff050c6d640 1 -- 172.21.15.32:0/1236031170 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff04c0d0510 con 0x7ff04c0d4210 2024-04-24T22:59:44.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.321+0000 7ff042ffd640 1 -- 172.21.15.32:0/1236031170 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7ff03405c710 con 0x7ff04c0d4210 2024-04-24T22:59:44.322 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2024-04-24T22:59:44.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.323+0000 7ff050c6d640 1 -- 172.21.15.32:0/1236031170 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff02c06be30 msgr2=0x7ff02c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:44.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.323+0000 7ff050c6d640 1 --2- 172.21.15.32:0/1236031170 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff02c06be30 0x7ff02c06e2f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff03c034000 tx=0x7ff03c02e000 comp rx=0 tx=0).stop 2024-04-24T22:59:44.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.323+0000 7ff050c6d640 1 -- 172.21.15.32:0/1236031170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff04c0d4210 msgr2=0x7ff04c0b7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:44.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.323+0000 7ff050c6d640 1 --2- 172.21.15.32:0/1236031170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff04c0d4210 0x7ff04c0b7d50 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff03400c500 tx=0x7ff03400c9c0 comp rx=0 tx=0).stop 2024-04-24T22:59:44.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.323+0000 7ff050c6d640 1 -- 172.21.15.32:0/1236031170 shutdown_connections 2024-04-24T22:59:44.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.323+0000 7ff050c6d640 1 --2- 172.21.15.32:0/1236031170 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff02c06be30 0x7ff02c06e2f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:44.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.323+0000 7ff050c6d640 1 --2- 172.21.15.32:0/1236031170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff04c0d4210 0x7ff04c0b7d50 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:44.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.323+0000 7ff050c6d640 1 --2- 172.21.15.32:0/1236031170 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff04c15e9b0 0x7ff04c14baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:44.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.323+0000 7ff050c6d640 1 --2- 172.21.15.32:0/1236031170 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff04c06b820 0x7ff04c0cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:44.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.323+0000 7ff050c6d640 1 -- 172.21.15.32:0/1236031170 >> 172.21.15.32:0/1236031170 conn(0x7ff04c0d11d0 msgr2=0x7ff04c0ca010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:44.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.323+0000 7ff050c6d640 1 -- 172.21.15.32:0/1236031170 shutdown_connections 2024-04-24T22:59:44.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.323+0000 7ff050c6d640 1 -- 172.21.15.32:0/1236031170 wait complete. 2024-04-24T22:59:44.335 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-04-24T22:59:44.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:59:44.335 DEBUG:teuthology.orchestra.run.smithi121:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-04-24T22:59:44.366 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2024-04-24T22:59:44.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:59:44.366 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-24T22:59:44.424 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-24T22:59:44.425 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T22:59:44.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.524+0000 7f6766ba2640 1 Processor -- start 2024-04-24T22:59:44.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.525+0000 7f6766ba2640 1 -- start start 2024-04-24T22:59:44.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.525+0000 7f6766ba2640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6760161df0 0x7f676006c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:44.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.525+0000 7f6766ba2640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f67600b6200 0x7f676006b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:44.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.525+0000 7f6766ba2640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6760162aa0 0x7f6760162f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:44.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.525+0000 7f6766ba2640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f676006cab0 con 0x7f6760161df0 2024-04-24T22:59:44.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.525+0000 7f6766ba2640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f67600d0550 con 0x7f67600b6200 2024-04-24T22:59:44.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.525+0000 7f6766ba2640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f67600d0830 con 0x7f6760162aa0 2024-04-24T22:59:44.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.525+0000 7f676493e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6760161df0 0x7f676006c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:44.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.526+0000 7f676493e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6760161df0 0x7f676006c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57550/0 (socket says 172.21.15.32:57550) 2024-04-24T22:59:44.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.526+0000 7f676493e640 1 -- 172.21.15.32:0/3954200893 learned_addr learned my addr 172.21.15.32:0/3954200893 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:44.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.526+0000 7f675ffff640 1 --2- 172.21.15.32:0/3954200893 >> v2:172.21.15.121:3300/0 conn(0x7f67600b6200 0x7f676006b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:44.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.526+0000 7f676513f640 1 --2- 172.21.15.32:0/3954200893 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6760162aa0 0x7f6760162f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:44.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.526+0000 7f676493e640 1 -- 172.21.15.32:0/3954200893 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6760162aa0 msgr2=0x7f6760162f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:44.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.526+0000 7f676493e640 1 --2- 172.21.15.32:0/3954200893 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6760162aa0 0x7f6760162f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:44.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.526+0000 7f676493e640 1 -- 172.21.15.32:0/3954200893 >> v2:172.21.15.121:3300/0 conn(0x7f67600b6200 msgr2=0x7f676006b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:44.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.526+0000 7f676493e640 1 --2- 172.21.15.32:0/3954200893 >> v2:172.21.15.121:3300/0 conn(0x7f67600b6200 0x7f676006b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:44.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.526+0000 7f676493e640 1 -- 172.21.15.32:0/3954200893 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67600d0b10 con 0x7f6760161df0 2024-04-24T22:59:44.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.526+0000 7f676513f640 1 --2- 172.21.15.32:0/3954200893 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6760162aa0 0x7f6760162f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:59:44.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.526+0000 7f675ffff640 1 --2- 172.21.15.32:0/3954200893 >> v2:172.21.15.121:3300/0 conn(0x7f67600b6200 0x7f676006b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:59:44.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.526+0000 7f676493e640 1 --2- 172.21.15.32:0/3954200893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6760161df0 0x7f676006c570 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6760072fe0 tx=0x7f675402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f147c34cf7b2a4e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:44.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.527+0000 7f675f7fe640 1 -- 172.21.15.32:0/3954200893 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6754037070 con 0x7f6760161df0 2024-04-24T22:59:44.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.527+0000 7f675f7fe640 1 -- 172.21.15.32:0/3954200893 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6754032440 con 0x7f6760161df0 2024-04-24T22:59:44.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.527+0000 7f675f7fe640 1 -- 172.21.15.32:0/3954200893 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6754032810 con 0x7f6760161df0 2024-04-24T22:59:44.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.527+0000 7f6766ba2640 1 -- 172.21.15.32:0/3954200893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6760161df0 msgr2=0x7f676006c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:44.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.527+0000 7f6766ba2640 1 --2- 172.21.15.32:0/3954200893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6760161df0 0x7f676006c570 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6760072fe0 tx=0x7f675402a1e0 comp rx=0 tx=0).stop 2024-04-24T22:59:44.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.527+0000 7f6766ba2640 1 -- 172.21.15.32:0/3954200893 shutdown_connections 2024-04-24T22:59:44.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.527+0000 7f6766ba2640 1 --2- 172.21.15.32:0/3954200893 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6760162aa0 0x7f6760162f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:44.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.527+0000 7f6766ba2640 1 --2- 172.21.15.32:0/3954200893 >> v2:172.21.15.121:3300/0 conn(0x7f67600b6200 0x7f676006b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:44.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.527+0000 7f6766ba2640 1 --2- 172.21.15.32:0/3954200893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6760161df0 0x7f676006c570 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:44.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.527+0000 7f6766ba2640 1 -- 172.21.15.32:0/3954200893 >> 172.21.15.32:0/3954200893 conn(0x7f67600cdf30 msgr2=0x7f67600ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:44.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.527+0000 7f6766ba2640 1 -- 172.21.15.32:0/3954200893 shutdown_connections 2024-04-24T22:59:44.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.527+0000 7f6766ba2640 1 -- 172.21.15.32:0/3954200893 wait complete. 2024-04-24T22:59:44.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.528+0000 7f6766ba2640 1 Processor -- start 2024-04-24T22:59:44.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.529+0000 7f6766ba2640 1 -- start start 2024-04-24T22:59:44.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.530+0000 7f6766ba2640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67600b6200 0x7f67601542a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:44.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.530+0000 7f6766ba2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6760161df0 0x7f6760153190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:44.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.530+0000 7f6766ba2640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6760162aa0 0x7f67601f4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:44.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.530+0000 7f6766ba2640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f67601621d0 con 0x7f6760161df0 2024-04-24T22:59:44.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.530+0000 7f6766ba2640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6760162580 con 0x7f67600b6200 2024-04-24T22:59:44.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.530+0000 7f6766ba2640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f67600d10d0 con 0x7f6760162aa0 2024-04-24T22:59:44.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.530+0000 7f675ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6760161df0 0x7f6760153190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:44.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.530+0000 7f675ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6760161df0 0x7f6760153190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57564/0 (socket says 172.21.15.32:57564) 2024-04-24T22:59:44.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.530+0000 7f675ffff640 1 -- 172.21.15.32:0/1100235632 learned_addr learned my addr 172.21.15.32:0/1100235632 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:44.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.530+0000 7f676493e640 1 --2- 172.21.15.32:0/1100235632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67600b6200 0x7f67601542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:44.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.530+0000 7f676513f640 1 --2- 172.21.15.32:0/1100235632 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6760162aa0 0x7f67601f4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:44.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.530+0000 7f675ffff640 1 -- 172.21.15.32:0/1100235632 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6760162aa0 msgr2=0x7f67601f4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:44.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.530+0000 7f675ffff640 1 --2- 172.21.15.32:0/1100235632 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6760162aa0 0x7f67601f4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:44.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.530+0000 7f675ffff640 1 -- 172.21.15.32:0/1100235632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67600b6200 msgr2=0x7f67601542a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:44.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.530+0000 7f675ffff640 1 --2- 172.21.15.32:0/1100235632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67600b6200 0x7f67601542a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:44.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.530+0000 7f675ffff640 1 -- 172.21.15.32:0/1100235632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67601517a0 con 0x7f6760161df0 2024-04-24T22:59:44.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.530+0000 7f675ffff640 1 --2- 172.21.15.32:0/1100235632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6760161df0 0x7f6760153190 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f674c00c7c0 tx=0x7f674c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:44.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.530+0000 7f675d7fa640 1 -- 172.21.15.32:0/1100235632 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674c015070 con 0x7f6760161df0 2024-04-24T22:59:44.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.531+0000 7f675d7fa640 1 -- 172.21.15.32:0/1100235632 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f674c007820 con 0x7f6760161df0 2024-04-24T22:59:44.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.531+0000 7f675d7fa640 1 -- 172.21.15.32:0/1100235632 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674c007bf0 con 0x7f6760161df0 2024-04-24T22:59:44.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.531+0000 7f6766ba2640 1 -- 172.21.15.32:0/1100235632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67601f5770 con 0x7f6760161df0 2024-04-24T22:59:44.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.531+0000 7f6766ba2640 1 -- 172.21.15.32:0/1100235632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6760151d20 con 0x7f6760161df0 2024-04-24T22:59:44.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.532+0000 7f6766ba2640 1 -- 172.21.15.32:0/1100235632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6760068430 con 0x7f6760161df0 2024-04-24T22:59:44.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.532+0000 7f675d7fa640 1 -- 172.21.15.32:0/1100235632 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f674c018070 con 0x7f6760161df0 2024-04-24T22:59:44.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.533+0000 7f675d7fa640 1 --2- 172.21.15.32:0/1100235632 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f674406be30 0x7f674406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:44.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.533+0000 7f675d7fa640 1 -- 172.21.15.32:0/1100235632 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f674c08ac90 con 0x7f6760161df0 2024-04-24T22:59:44.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.533+0000 7f676493e640 1 --2- 172.21.15.32:0/1100235632 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f674406be30 0x7f674406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:44.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.534+0000 7f676493e640 1 --2- 172.21.15.32:0/1100235632 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f674406be30 0x7f674406e2f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6754034000 tx=0x7f675402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:44.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.538+0000 7f675d7fa640 1 -- 172.21.15.32:0/1100235632 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f674c058ff0 con 0x7f6760161df0 2024-04-24T22:59:44.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.761+0000 7f6766ba2640 1 -- 172.21.15.32:0/1100235632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f67600ceb60 con 0x7f6760161df0 2024-04-24T22:59:44.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.761+0000 7f675d7fa640 1 -- 172.21.15.32:0/1100235632 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5178 (secure 0 0 0) 0x7f674c05d2f0 con 0x7f6760161df0 2024-04-24T22:59:44.763 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:59:44.763 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1485884060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1485884060},{"type":"v1","addr":"172.21.15.121:6837","nonce":1485884060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/923934104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":923934104},{"type":"v1","addr":"172.21.15.32:6833","nonce":923934104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3229115835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3229115835},{"type":"v1","addr":"172.21.15.32:6835","nonce":3229115835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T22:59:03.876305+0000","modified":"2024-04-24T22:59:05.932147+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6836/2644625343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2644625343},{"type":"v1","addr":"172.21.15.121:6836","nonce":2644625343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4276]},"id":1}]} 2024-04-24T22:59:44.763 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 6 2024-04-24T22:59:44.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.763+0000 7f6766ba2640 1 -- 172.21.15.32:0/1100235632 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f674406be30 msgr2=0x7f674406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:44.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.763+0000 7f6766ba2640 1 --2- 172.21.15.32:0/1100235632 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f674406be30 0x7f674406e2f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6754034000 tx=0x7f675402e000 comp rx=0 tx=0).stop 2024-04-24T22:59:44.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.763+0000 7f6766ba2640 1 -- 172.21.15.32:0/1100235632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6760161df0 msgr2=0x7f6760153190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:44.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.763+0000 7f6766ba2640 1 --2- 172.21.15.32:0/1100235632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6760161df0 0x7f6760153190 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f674c00c7c0 tx=0x7f674c00cc80 comp rx=0 tx=0).stop 2024-04-24T22:59:44.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.763+0000 7f6766ba2640 1 -- 172.21.15.32:0/1100235632 shutdown_connections 2024-04-24T22:59:44.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.763+0000 7f6766ba2640 1 --2- 172.21.15.32:0/1100235632 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f674406be30 0x7f674406e2f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:44.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.763+0000 7f6766ba2640 1 --2- 172.21.15.32:0/1100235632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6760161df0 0x7f6760153190 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:44.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.763+0000 7f6766ba2640 1 --2- 172.21.15.32:0/1100235632 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6760162aa0 0x7f67601f4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:44.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.763+0000 7f6766ba2640 1 --2- 172.21.15.32:0/1100235632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67600b6200 0x7f67601542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:44.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.763+0000 7f6766ba2640 1 -- 172.21.15.32:0/1100235632 >> 172.21.15.32:0/1100235632 conn(0x7f67600cdf30 msgr2=0x7f6760160290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:44.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.763+0000 7f6766ba2640 1 -- 172.21.15.32:0/1100235632 shutdown_connections 2024-04-24T22:59:44.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.763+0000 7f6766ba2640 1 -- 172.21.15.32:0/1100235632 wait complete. 2024-04-24T22:59:44.776 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:59:44.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.875+0000 7fa9bb909640 1 Processor -- start 2024-04-24T22:59:44.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.875+0000 7fa9bb909640 1 -- start start 2024-04-24T22:59:44.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.876+0000 7fa9bb909640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa9b4162320 0x7fa9b40ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:44.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.876+0000 7fa9bb909640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9b40b5af0 0x7fa9b4162700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:44.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.876+0000 7fa9bb909640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9b40cd270 con 0x7fa9b4162c40 2024-04-24T22:59:44.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.876+0000 7fa9bb909640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fa9b40b5660 con 0x7fa9b4162320 2024-04-24T22:59:44.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.876+0000 7fa9bb909640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa9b4071610 con 0x7fa9b40b5af0 2024-04-24T22:59:44.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.876+0000 7fa9b9ea6640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa9b4162c40 0x7fa9b40ae8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52080/0 (socket says 172.21.15.32:52080) 2024-04-24T22:59:44.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.876+0000 7fa9b9ea6640 1 -- 172.21.15.32:0/3435050199 learned_addr learned my addr 172.21.15.32:0/3435050199 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:44.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.877+0000 7fa9b96a5640 1 --2- 172.21.15.32:0/3435050199 >> v2:172.21.15.121:3300/0 conn(0x7fa9b4162320 0x7fa9b40ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:44.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.877+0000 7fa9b8ea4640 1 --2- 172.21.15.32:0/3435050199 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9b40b5af0 0x7fa9b4162700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:44.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.877+0000 7fa9b3fff640 1 -- 172.21.15.32:0/3435050199 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4157474998 0 0) 0x7fa9b40cd270 con 0x7fa9b4162c40 2024-04-24T22:59:44.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.877+0000 7fa9b3fff640 1 -- 172.21.15.32:0/3435050199 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa99c003520 con 0x7fa9b4162c40 2024-04-24T22:59:44.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.877+0000 7fa9b3fff640 1 -- 172.21.15.32:0/3435050199 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1325066917 0 0) 0x7fa99c003520 con 0x7fa9b4162c40 2024-04-24T22:59:44.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.877+0000 7fa9b3fff640 1 -- 172.21.15.32:0/3435050199 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9b40b5af0 msgr2=0x7fa9b4162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:44.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.877+0000 7fa9b3fff640 1 --2- 172.21.15.32:0/3435050199 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9b40b5af0 0x7fa9b4162700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:44.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.877+0000 7fa9b3fff640 1 -- 172.21.15.32:0/3435050199 >> v2:172.21.15.121:3300/0 conn(0x7fa9b4162320 msgr2=0x7fa9b40ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:44.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.877+0000 7fa9b3fff640 1 --2- 172.21.15.32:0/3435050199 >> v2:172.21.15.121:3300/0 conn(0x7fa9b4162320 0x7fa9b40ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:44.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.877+0000 7fa9b3fff640 1 -- 172.21.15.32:0/3435050199 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9b40718f0 con 0x7fa9b4162c40 2024-04-24T22:59:44.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.877+0000 7fa9b3fff640 1 -- 172.21.15.32:0/3435050199 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7fa9a40019f0 con 0x7fa9b4162c40 2024-04-24T22:59:44.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.877+0000 7fa9b96a5640 1 --2- 172.21.15.32:0/3435050199 >> v2:172.21.15.121:3300/0 conn(0x7fa9b4162320 0x7fa9b40ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:59:44.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.877+0000 7fa9b3fff640 1 -- 172.21.15.32:0/3435050199 >> v1:172.21.15.32:6789/0 conn(0x7fa9b4162c40 legacy=0x7fa9b40ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:44.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.877+0000 7fa9b3fff640 1 --2- 172.21.15.32:0/3435050199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa99c0039b0 0x7fa99c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:44.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.877+0000 7fa9b3fff640 1 --2- 172.21.15.32:0/3435050199 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa99c004800 0x7fa99c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:44.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.877+0000 7fa9b3fff640 1 --2- 172.21.15.32:0/3435050199 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa99c0051c0 0x7fa99c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:44.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.877+0000 7fa9b3fff640 1 -- 172.21.15.32:0/3435050199 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b40b5660 con 0x7fa99c0039b0 2024-04-24T22:59:44.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.877+0000 7fa9b3fff640 1 -- 172.21.15.32:0/3435050199 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b4071610 con 0x7fa99c0051c0 2024-04-24T22:59:44.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.877+0000 7fa9b3fff640 1 -- 172.21.15.32:0/3435050199 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa99c00bf50 con 0x7fa99c004800 2024-04-24T22:59:44.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.877+0000 7fa9b8ea4640 1 --2- 172.21.15.32:0/3435050199 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9b40b5af0 0x7fa9b4162700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:59:44.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.877+0000 7fa9b96a5640 1 --2- 172.21.15.32:0/3435050199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa99c0039b0 0x7fa99c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:44.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.877+0000 7fa9b9ea6640 1 --2- 172.21.15.32:0/3435050199 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa99c0051c0 0x7fa99c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:44.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.877+0000 7fa9b8ea4640 1 --2- 172.21.15.32:0/3435050199 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa99c004800 0x7fa99c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:44.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.878+0000 7fa9b96a5640 1 -- 172.21.15.32:0/3435050199 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa99c004800 msgr2=0x7fa99c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:44.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.878+0000 7fa9b96a5640 1 --2- 172.21.15.32:0/3435050199 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa99c004800 0x7fa99c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:44.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.878+0000 7fa9b96a5640 1 -- 172.21.15.32:0/3435050199 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa99c0051c0 msgr2=0x7fa99c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:44.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.878+0000 7fa9b96a5640 1 --2- 172.21.15.32:0/3435050199 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa99c0051c0 0x7fa99c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:44.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.878+0000 7fa9b96a5640 1 -- 172.21.15.32:0/3435050199 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa99c003520 con 0x7fa99c0039b0 2024-04-24T22:59:44.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.878+0000 7fa9b8ea4640 1 --2- 172.21.15.32:0/3435050199 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa99c004800 0x7fa99c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:44.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.878+0000 7fa9b96a5640 1 --2- 172.21.15.32:0/3435050199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa99c0039b0 0x7fa99c003dd0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac009cc0 tx=0x7fa99c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d863845f7a4e62 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:44.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.878+0000 7fa9b3fff640 1 -- 172.21.15.32:0/3435050199 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac036070 con 0x7fa99c0039b0 2024-04-24T22:59:44.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.878+0000 7fa9b3fff640 1 -- 172.21.15.32:0/3435050199 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9ac031400 con 0x7fa99c0039b0 2024-04-24T22:59:44.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.878+0000 7fa9b3fff640 1 -- 172.21.15.32:0/3435050199 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac009860 con 0x7fa99c0039b0 2024-04-24T22:59:44.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.879+0000 7fa9bb909640 1 -- 172.21.15.32:0/3435050199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa99c0039b0 msgr2=0x7fa99c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:44.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.879+0000 7fa9bb909640 1 --2- 172.21.15.32:0/3435050199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa99c0039b0 0x7fa99c003dd0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac009cc0 tx=0x7fa99c005830 comp rx=0 tx=0).stop 2024-04-24T22:59:44.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.879+0000 7fa9b9ea6640 1 -- 172.21.15.32:0/3435050199 reap_dead start 2024-04-24T22:59:44.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.879+0000 7fa9bb909640 1 -- 172.21.15.32:0/3435050199 shutdown_connections 2024-04-24T22:59:44.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.879+0000 7fa9bb909640 1 -- 172.21.15.32:0/3435050199 >> 172.21.15.32:0/3435050199 conn(0x7fa9b40d0ae0 msgr2=0x7fa9b40d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:44.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.879+0000 7fa9bb909640 1 -- 172.21.15.32:0/3435050199 shutdown_connections 2024-04-24T22:59:44.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.879+0000 7fa9bb909640 1 -- 172.21.15.32:0/3435050199 wait complete. 2024-04-24T22:59:44.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.880+0000 7fa9bb909640 1 Processor -- start 2024-04-24T22:59:44.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.880+0000 7fa9bb909640 1 -- start start 2024-04-24T22:59:44.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.881+0000 7fa9bb909640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa99c0039b0 0x7fa9b40cf240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:44.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.881+0000 7fa9bb909640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9b40ceb60 0x7fa9b4150f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:44.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.881+0000 7fa9bb909640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9b406dcf0 0x7fa9b406e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:44.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.881+0000 7fa9bb909640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b4067950 con 0x7fa9b406dcf0 2024-04-24T22:59:44.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.881+0000 7fa9bb909640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b415c630 con 0x7fa99c0039b0 2024-04-24T22:59:44.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.881+0000 7fa9bb909640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa9b4071bd0 con 0x7fa9b40ceb60 2024-04-24T22:59:44.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.881+0000 7fa9b9ea6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9b406dcf0 0x7fa9b406e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:44.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.881+0000 7fa9b9ea6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9b406dcf0 0x7fa9b406e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57576/0 (socket says 172.21.15.32:57576) 2024-04-24T22:59:44.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.881+0000 7fa9b9ea6640 1 -- 172.21.15.32:0/605209965 learned_addr learned my addr 172.21.15.32:0/605209965 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:44.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.881+0000 7fa9b8ea4640 1 --2- 172.21.15.32:0/605209965 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9b40ceb60 0x7fa9b4150f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:44.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.881+0000 7fa9b96a5640 1 --2- 172.21.15.32:0/605209965 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa99c0039b0 0x7fa9b40cf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:44.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.881+0000 7fa9b9ea6640 1 -- 172.21.15.32:0/605209965 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9b40ceb60 msgr2=0x7fa9b4150f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:44.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.881+0000 7fa9b9ea6640 1 --2- 172.21.15.32:0/605209965 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9b40ceb60 0x7fa9b4150f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:44.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.881+0000 7fa9b9ea6640 1 -- 172.21.15.32:0/605209965 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa99c0039b0 msgr2=0x7fa9b40cf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:44.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.881+0000 7fa9b9ea6640 1 --2- 172.21.15.32:0/605209965 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa99c0039b0 0x7fa9b40cf240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:44.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.881+0000 7fa9b9ea6640 1 -- 172.21.15.32:0/605209965 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9b4160520 con 0x7fa9b406dcf0 2024-04-24T22:59:44.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.881+0000 7fa9b8ea4640 1 --2- 172.21.15.32:0/605209965 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9b40ceb60 0x7fa9b4150f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:44.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.881+0000 7fa9b9ea6640 1 --2- 172.21.15.32:0/605209965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9b406dcf0 0x7fa9b406e170 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4163020 tx=0x7fa9a4007b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:44.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.882+0000 7fa9b1ffb640 1 -- 172.21.15.32:0/605209965 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a4017020 con 0x7fa9b406dcf0 2024-04-24T22:59:44.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.882+0000 7fa9b1ffb640 1 -- 172.21.15.32:0/605209965 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9a400aca0 con 0x7fa9b406dcf0 2024-04-24T22:59:44.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.882+0000 7fa9bb909640 1 -- 172.21.15.32:0/605209965 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9b40cf890 con 0x7fa9b406dcf0 2024-04-24T22:59:44.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.882+0000 7fa9b1ffb640 1 -- 172.21.15.32:0/605209965 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a4013610 con 0x7fa9b406dcf0 2024-04-24T22:59:44.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.882+0000 7fa9bb909640 1 -- 172.21.15.32:0/605209965 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9b40cfe50 con 0x7fa9b406dcf0 2024-04-24T22:59:44.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.883+0000 7fa9bb909640 1 -- 172.21.15.32:0/605209965 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9b4068430 con 0x7fa9b406dcf0 2024-04-24T22:59:44.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.883+0000 7fa9b1ffb640 1 -- 172.21.15.32:0/605209965 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa9a401a020 con 0x7fa9b406dcf0 2024-04-24T22:59:44.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.884+0000 7fa9b1ffb640 1 --2- 172.21.15.32:0/605209965 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa9a006be30 0x7fa9a006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:44.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.884+0000 7fa9b1ffb640 1 -- 172.21.15.32:0/605209965 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa9a408b720 con 0x7fa9b406dcf0 2024-04-24T22:59:44.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.884+0000 7fa9b96a5640 1 --2- 172.21.15.32:0/605209965 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa9a006be30 0x7fa9a006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:44.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.884+0000 7fa9b96a5640 1 --2- 172.21.15.32:0/605209965 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa9a006be30 0x7fa9a006e2f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac033000 tx=0x7fa9ac008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:44.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:44.888+0000 7fa9b1ffb640 1 -- 172.21.15.32:0/605209965 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa9a4021020 con 0x7fa9b406dcf0 2024-04-24T22:59:45.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.070+0000 7fa9bb909640 1 -- 172.21.15.32:0/605209965 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa9b4067730 con 0x7fa9b406dcf0 2024-04-24T22:59:45.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.071+0000 7fa9b1ffb640 1 -- 172.21.15.32:0/605209965 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16621 (secure 0 0 0) 0x7fa9a405a160 con 0x7fa9b406dcf0 2024-04-24T22:59:45.073 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:59:45.073 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T22:59:05.517554+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:59:02.016639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T22:59:02.846087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:59:45.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.073+0000 7fa9bb909640 1 -- 172.21.15.32:0/605209965 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa9a006be30 msgr2=0x7fa9a006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:45.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.073+0000 7fa9bb909640 1 --2- 172.21.15.32:0/605209965 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa9a006be30 0x7fa9a006e2f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac033000 tx=0x7fa9ac008030 comp rx=0 tx=0).stop 2024-04-24T22:59:45.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.073+0000 7fa9bb909640 1 -- 172.21.15.32:0/605209965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9b406dcf0 msgr2=0x7fa9b406e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:45.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.073+0000 7fa9bb909640 1 --2- 172.21.15.32:0/605209965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9b406dcf0 0x7fa9b406e170 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4163020 tx=0x7fa9a4007b20 comp rx=0 tx=0).stop 2024-04-24T22:59:45.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.073+0000 7fa9bb909640 1 -- 172.21.15.32:0/605209965 shutdown_connections 2024-04-24T22:59:45.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.073+0000 7fa9bb909640 1 --2- 172.21.15.32:0/605209965 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa9a006be30 0x7fa9a006e2f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:45.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.073+0000 7fa9bb909640 1 --2- 172.21.15.32:0/605209965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9b406dcf0 0x7fa9b406e170 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:45.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.073+0000 7fa9bb909640 1 --2- 172.21.15.32:0/605209965 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9b40ceb60 0x7fa9b4150f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:45.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.074+0000 7fa9bb909640 1 --2- 172.21.15.32:0/605209965 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa99c0039b0 0x7fa9b40cf240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:45.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.074+0000 7fa9bb909640 1 -- 172.21.15.32:0/605209965 >> 172.21.15.32:0/605209965 conn(0x7fa9b40d0ae0 msgr2=0x7fa9b415ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:45.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.074+0000 7fa9bb909640 1 -- 172.21.15.32:0/605209965 shutdown_connections 2024-04-24T22:59:45.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.074+0000 7fa9bb909640 1 -- 172.21.15.32:0/605209965 wait complete. 2024-04-24T22:59:45.086 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T22:59:02.016639+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T22:59:02.846087+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T22:59:45.087 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T22:59:45.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.189+0000 7f469431b640 1 Processor -- start 2024-04-24T22:59:45.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.189+0000 7f469431b640 1 -- start start 2024-04-24T22:59:45.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.190+0000 7f469431b640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f468c06c480 0x7f468c0b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:45.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.190+0000 7f469431b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f468c0b0770 con 0x7f468c074210 2024-04-24T22:59:45.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.190+0000 7f469431b640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f468c0b1010 con 0x7f468c0b0b80 2024-04-24T22:59:45.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.190+0000 7f469431b640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f468c06c8d0 con 0x7f468c06c480 2024-04-24T22:59:45.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.190+0000 7f46920b7640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f468c074210 0x7f468c0745f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52086/0 (socket says 172.21.15.32:52086) 2024-04-24T22:59:45.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.190+0000 7f46920b7640 1 -- 172.21.15.32:0/2287078915 learned_addr learned my addr 172.21.15.32:0/2287078915 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:45.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.190+0000 7f46910b5640 1 -- 172.21.15.32:0/2287078915 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 699517441 0 0) 0x7f468c0b0770 con 0x7f468c074210 2024-04-24T22:59:45.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.190+0000 7f46928b8640 1 --2- 172.21.15.32:0/2287078915 >> v2:172.21.15.121:3300/0 conn(0x7f468c06c480 0x7f468c0b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:45.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.190+0000 7f46910b5640 1 -- 172.21.15.32:0/2287078915 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4678003520 con 0x7f468c074210 2024-04-24T22:59:45.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.191+0000 7f46910b5640 1 -- 172.21.15.32:0/2287078915 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2122224947 0 0) 0x7f4678003520 con 0x7f468c074210 2024-04-24T22:59:45.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.191+0000 7f46910b5640 1 -- 172.21.15.32:0/2287078915 >> v2:172.21.15.121:3300/0 conn(0x7f468c06c480 msgr2=0x7f468c0b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:45.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.191+0000 7f46910b5640 1 --2- 172.21.15.32:0/2287078915 >> v2:172.21.15.121:3300/0 conn(0x7f468c06c480 0x7f468c0b20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:45.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.191+0000 7f46910b5640 1 -- 172.21.15.32:0/2287078915 >> v1:172.21.15.121:6789/0 conn(0x7f468c0b0b80 legacy=0x7f468c0ae140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:45.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.191+0000 7f46910b5640 1 -- 172.21.15.32:0/2287078915 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f468c0b12f0 con 0x7f468c074210 2024-04-24T22:59:45.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.191+0000 7f46910b5640 1 -- 172.21.15.32:0/2287078915 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f4688001670 con 0x7f468c074210 2024-04-24T22:59:45.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.191+0000 7f46910b5640 1 -- 172.21.15.32:0/2287078915 >> v1:172.21.15.32:6789/0 conn(0x7f468c074210 legacy=0x7f468c0745f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:45.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.191+0000 7f46910b5640 1 --2- 172.21.15.32:0/2287078915 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46780039b0 0x7f4678003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:45.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.191+0000 7f46910b5640 1 --2- 172.21.15.32:0/2287078915 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4678004800 0x7f4678004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:45.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.191+0000 7f46910b5640 1 --2- 172.21.15.32:0/2287078915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46780051c0 0x7f467800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:45.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.191+0000 7f46910b5640 1 -- 172.21.15.32:0/2287078915 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f467800c030 con 0x7f46780051c0 2024-04-24T22:59:45.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.191+0000 7f46910b5640 1 -- 172.21.15.32:0/2287078915 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f467800c1e0 con 0x7f4678004800 2024-04-24T22:59:45.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.191+0000 7f46910b5640 1 -- 172.21.15.32:0/2287078915 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f467800c3e0 con 0x7f46780039b0 2024-04-24T22:59:45.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.191+0000 7f46928b8640 1 --2- 172.21.15.32:0/2287078915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46780051c0 0x7f467800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:45.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.191+0000 7f46920b7640 1 --2- 172.21.15.32:0/2287078915 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46780039b0 0x7f4678003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:45.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.191+0000 7f46918b6640 1 --2- 172.21.15.32:0/2287078915 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4678004800 0x7f4678004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:45.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.191+0000 7f46928b8640 1 -- 172.21.15.32:0/2287078915 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46780039b0 msgr2=0x7f4678003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:45.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.191+0000 7f46928b8640 1 --2- 172.21.15.32:0/2287078915 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46780039b0 0x7f4678003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:45.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.191+0000 7f46928b8640 1 -- 172.21.15.32:0/2287078915 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4678004800 msgr2=0x7f4678004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:45.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.191+0000 7f46928b8640 1 --2- 172.21.15.32:0/2287078915 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4678004800 0x7f4678004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:45.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.191+0000 7f46928b8640 1 -- 172.21.15.32:0/2287078915 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4678003520 con 0x7f46780051c0 2024-04-24T22:59:45.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.191+0000 7f46920b7640 1 --2- 172.21.15.32:0/2287078915 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46780039b0 0x7f4678003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:45.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.192+0000 7f46928b8640 1 -- 172.21.15.32:0/2287078915 reap_dead start 2024-04-24T22:59:45.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.192+0000 7f46928b8640 1 --2- 172.21.15.32:0/2287078915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46780051c0 0x7f467800b9c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4684009fc0 tx=0x7f4678005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd5b331b0786f9f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:45.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.192+0000 7f46910b5640 1 -- 172.21.15.32:0/2287078915 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4684035020 con 0x7f46780051c0 2024-04-24T22:59:45.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.192+0000 7f46910b5640 1 -- 172.21.15.32:0/2287078915 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4684003bd0 con 0x7f46780051c0 2024-04-24T22:59:45.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.192+0000 7f46910b5640 1 -- 172.21.15.32:0/2287078915 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4684009c10 con 0x7f46780051c0 2024-04-24T22:59:45.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.192+0000 7f469431b640 1 -- 172.21.15.32:0/2287078915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46780051c0 msgr2=0x7f467800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:45.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.192+0000 7f469431b640 1 --2- 172.21.15.32:0/2287078915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46780051c0 0x7f467800b9c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4684009fc0 tx=0x7f4678005830 comp rx=0 tx=0).stop 2024-04-24T22:59:45.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.193+0000 7f469431b640 1 -- 172.21.15.32:0/2287078915 shutdown_connections 2024-04-24T22:59:45.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.193+0000 7f469431b640 1 --2- 172.21.15.32:0/2287078915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46780051c0 0x7f467800b9c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:45.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.193+0000 7f469431b640 1 -- 172.21.15.32:0/2287078915 >> 172.21.15.32:0/2287078915 conn(0x7f468c0b2e60 msgr2=0x7f468c0b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:45.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.193+0000 7f469431b640 1 -- 172.21.15.32:0/2287078915 shutdown_connections 2024-04-24T22:59:45.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.193+0000 7f469431b640 1 -- 172.21.15.32:0/2287078915 wait complete. 2024-04-24T22:59:45.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.194+0000 7f469431b640 1 Processor -- start 2024-04-24T22:59:45.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.194+0000 7f469431b640 1 -- start start 2024-04-24T22:59:45.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.194+0000 7f469431b640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46780051c0 0x7f468c15b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:45.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.195+0000 7f469431b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f468c15b660 0x7f468c15fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:45.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.195+0000 7f469431b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f468c0cdf20 0x7f468c15d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:45.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.195+0000 7f469431b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f468c0b18b0 con 0x7f468c0cdf20 2024-04-24T22:59:45.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.195+0000 7f469431b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f468c0b1c20 con 0x7f468c15b660 2024-04-24T22:59:45.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.195+0000 7f469431b640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f468c0b3ba0 con 0x7f46780051c0 2024-04-24T22:59:45.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.195+0000 7f46928b8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f468c0cdf20 0x7f468c15d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:45.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.195+0000 7f46928b8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f468c0cdf20 0x7f468c15d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57596/0 (socket says 172.21.15.32:57596) 2024-04-24T22:59:45.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.195+0000 7f46928b8640 1 -- 172.21.15.32:0/1976565594 learned_addr learned my addr 172.21.15.32:0/1976565594 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:45.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.195+0000 7f46918b6640 1 --2- 172.21.15.32:0/1976565594 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f468c15b660 0x7f468c15fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:45.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.195+0000 7f46920b7640 1 --2- 172.21.15.32:0/1976565594 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46780051c0 0x7f468c15b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:45.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.195+0000 7f46928b8640 1 -- 172.21.15.32:0/1976565594 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46780051c0 msgr2=0x7f468c15b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:45.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.195+0000 7f46928b8640 1 --2- 172.21.15.32:0/1976565594 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46780051c0 0x7f468c15b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:45.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.195+0000 7f46928b8640 1 -- 172.21.15.32:0/1976565594 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f468c15b660 msgr2=0x7f468c15fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:45.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.195+0000 7f46928b8640 1 --2- 172.21.15.32:0/1976565594 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f468c15b660 0x7f468c15fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:45.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.195+0000 7f46928b8640 1 -- 172.21.15.32:0/1976565594 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f468c15edc0 con 0x7f468c0cdf20 2024-04-24T22:59:45.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.195+0000 7f46928b8640 1 --2- 172.21.15.32:0/1976565594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f468c0cdf20 0x7f468c15d2a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f468400e230 tx=0x7f4684009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:45.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.195+0000 7f4682ffd640 1 -- 172.21.15.32:0/1976565594 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4684040070 con 0x7f468c0cdf20 2024-04-24T22:59:45.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.195+0000 7f4682ffd640 1 -- 172.21.15.32:0/1976565594 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f468403c680 con 0x7f468c0cdf20 2024-04-24T22:59:45.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.195+0000 7f469431b640 1 -- 172.21.15.32:0/1976565594 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f468c15f0a0 con 0x7f468c0cdf20 2024-04-24T22:59:45.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.195+0000 7f4682ffd640 1 -- 172.21.15.32:0/1976565594 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4684035030 con 0x7f468c0cdf20 2024-04-24T22:59:45.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.195+0000 7f469431b640 1 -- 172.21.15.32:0/1976565594 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f468c15f660 con 0x7f468c0cdf20 2024-04-24T22:59:45.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.196+0000 7f469431b640 1 -- 172.21.15.32:0/1976565594 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f468c068430 con 0x7f468c0cdf20 2024-04-24T22:59:45.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.197+0000 7f4682ffd640 1 -- 172.21.15.32:0/1976565594 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4684043070 con 0x7f468c0cdf20 2024-04-24T22:59:45.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.197+0000 7f4682ffd640 1 --2- 172.21.15.32:0/1976565594 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f467006be30 0x7f467006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:45.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.197+0000 7f4682ffd640 1 -- 172.21.15.32:0/1976565594 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f46840abfd0 con 0x7f468c0cdf20 2024-04-24T22:59:45.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.197+0000 7f46920b7640 1 --2- 172.21.15.32:0/1976565594 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f467006be30 0x7f467006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:45.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.198+0000 7f46920b7640 1 --2- 172.21.15.32:0/1976565594 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f467006be30 0x7f467006e2f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4688001200 tx=0x7f4688007040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:45.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.201+0000 7f4682ffd640 1 -- 172.21.15.32:0/1976565594 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f468407a3c0 con 0x7f468c0cdf20 2024-04-24T22:59:45.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.386+0000 7f469431b640 1 -- 172.21.15.32:0/1976565594 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f468c15f830 con 0x7f468c0cdf20 2024-04-24T22:59:45.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.386+0000 7f4682ffd640 1 -- 172.21.15.32:0/1976565594 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f468407e6c0 con 0x7f468c0cdf20 2024-04-24T22:59:45.388 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T22:59:45.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.388+0000 7f469431b640 1 -- 172.21.15.32:0/1976565594 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f467006be30 msgr2=0x7f467006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:45.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.388+0000 7f469431b640 1 --2- 172.21.15.32:0/1976565594 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f467006be30 0x7f467006e2f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4688001200 tx=0x7f4688007040 comp rx=0 tx=0).stop 2024-04-24T22:59:45.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.388+0000 7f469431b640 1 -- 172.21.15.32:0/1976565594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f468c0cdf20 msgr2=0x7f468c15d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:45.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.388+0000 7f469431b640 1 --2- 172.21.15.32:0/1976565594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f468c0cdf20 0x7f468c15d2a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f468400e230 tx=0x7f4684009750 comp rx=0 tx=0).stop 2024-04-24T22:59:45.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.388+0000 7f469431b640 1 -- 172.21.15.32:0/1976565594 shutdown_connections 2024-04-24T22:59:45.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.388+0000 7f469431b640 1 --2- 172.21.15.32:0/1976565594 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f467006be30 0x7f467006e2f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:45.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.388+0000 7f469431b640 1 --2- 172.21.15.32:0/1976565594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f468c0cdf20 0x7f468c15d2a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:45.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.388+0000 7f469431b640 1 --2- 172.21.15.32:0/1976565594 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f468c15b660 0x7f468c15fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:45.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.388+0000 7f469431b640 1 --2- 172.21.15.32:0/1976565594 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46780051c0 0x7f468c15b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:45.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.388+0000 7f469431b640 1 -- 172.21.15.32:0/1976565594 >> 172.21.15.32:0/1976565594 conn(0x7f468c0b2e60 msgr2=0x7f468c06d570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:45.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.388+0000 7f469431b640 1 -- 172.21.15.32:0/1976565594 shutdown_connections 2024-04-24T22:59:45.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.388+0000 7f469431b640 1 -- 172.21.15.32:0/1976565594 wait complete. 2024-04-24T22:59:45.401 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T22:59:45.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.503+0000 7f05b135a640 1 Processor -- start 2024-04-24T22:59:45.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.503+0000 7f05b135a640 1 -- start start 2024-04-24T22:59:45.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.504+0000 7f05b135a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f05ac074210 0x7f05ac06c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:45.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.504+0000 7f05b135a640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f05ac0b0b80 0x7f05ac0745f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:45.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.504+0000 7f05b135a640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05ac0b0770 con 0x7f05ac0ae140 2024-04-24T22:59:45.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.504+0000 7f05b135a640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f05ac0b1c20 con 0x7f05ac0b0b80 2024-04-24T22:59:45.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.504+0000 7f05b135a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f05ac074b30 con 0x7f05ac074210 2024-04-24T22:59:45.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.504+0000 7f05ab7fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f05ac0ae140 0x7f05ac0b20b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52096/0 (socket says 172.21.15.32:52096) 2024-04-24T22:59:45.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.504+0000 7f05ab7fe640 1 -- 172.21.15.32:0/3040840652 learned_addr learned my addr 172.21.15.32:0/3040840652 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:45.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.504+0000 7f05a3fff640 1 --2- 172.21.15.32:0/3040840652 >> v2:172.21.15.121:3300/0 conn(0x7f05ac0b0b80 0x7f05ac0745f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:45.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.504+0000 7f05aaffd640 1 --2- 172.21.15.32:0/3040840652 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f05ac074210 0x7f05ac06c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:45.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.504+0000 7f05aa7fc640 1 -- 172.21.15.32:0/3040840652 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2372018312 0 0) 0x7f05ac0b0770 con 0x7f05ac0ae140 2024-04-24T22:59:45.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.504+0000 7f05aa7fc640 1 -- 172.21.15.32:0/3040840652 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0598003520 con 0x7f05ac0ae140 2024-04-24T22:59:45.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.504+0000 7f05aa7fc640 1 -- 172.21.15.32:0/3040840652 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3825202316 0 0) 0x7f0598003520 con 0x7f05ac0ae140 2024-04-24T22:59:45.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.504+0000 7f05aa7fc640 1 -- 172.21.15.32:0/3040840652 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f05ac074210 msgr2=0x7f05ac06c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:45.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.504+0000 7f05aa7fc640 1 --2- 172.21.15.32:0/3040840652 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f05ac074210 0x7f05ac06c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:45.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.504+0000 7f05aa7fc640 1 -- 172.21.15.32:0/3040840652 >> v2:172.21.15.121:3300/0 conn(0x7f05ac0b0b80 msgr2=0x7f05ac0745f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:45.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.504+0000 7f05aa7fc640 1 --2- 172.21.15.32:0/3040840652 >> v2:172.21.15.121:3300/0 conn(0x7f05ac0b0b80 0x7f05ac0745f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:45.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.504+0000 7f05aa7fc640 1 -- 172.21.15.32:0/3040840652 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05ac0b1010 con 0x7f05ac0ae140 2024-04-24T22:59:45.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.504+0000 7f05aa7fc640 1 -- 172.21.15.32:0/3040840652 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f05a40019f0 con 0x7f05ac0ae140 2024-04-24T22:59:45.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.504+0000 7f05aa7fc640 1 -- 172.21.15.32:0/3040840652 >> v1:172.21.15.32:6789/0 conn(0x7f05ac0ae140 legacy=0x7f05ac0b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:45.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.505+0000 7f05aa7fc640 1 --2- 172.21.15.32:0/3040840652 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f05ac074210 0x7f05980039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:45.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.505+0000 7f05aa7fc640 1 --2- 172.21.15.32:0/3040840652 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0598003f30 0x7f0598004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:45.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.505+0000 7f05aa7fc640 1 --2- 172.21.15.32:0/3040840652 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0598004d40 0x7f05980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:45.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.505+0000 7f05aa7fc640 1 -- 172.21.15.32:0/3040840652 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f05ac0b1c20 con 0x7f0598004d40 2024-04-24T22:59:45.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.505+0000 7f05aa7fc640 1 -- 172.21.15.32:0/3040840652 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f05ac074b30 con 0x7f0598003f30 2024-04-24T22:59:45.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.505+0000 7f05aa7fc640 1 -- 172.21.15.32:0/3040840652 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f059800b9c0 con 0x7f05ac074210 2024-04-24T22:59:45.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.505+0000 7f05ab7fe640 1 --2- 172.21.15.32:0/3040840652 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0598004d40 0x7f05980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:45.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.505+0000 7f05aaffd640 1 --2- 172.21.15.32:0/3040840652 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f05ac074210 0x7f05980039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:45.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.505+0000 7f05a3fff640 1 --2- 172.21.15.32:0/3040840652 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0598003f30 0x7f0598004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:45.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.505+0000 7f05ab7fe640 1 -- 172.21.15.32:0/3040840652 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f05ac074210 msgr2=0x7f05980039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:45.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.505+0000 7f05ab7fe640 1 --2- 172.21.15.32:0/3040840652 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f05ac074210 0x7f05980039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:45.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.505+0000 7f05ab7fe640 1 -- 172.21.15.32:0/3040840652 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0598003f30 msgr2=0x7f0598004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:45.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.505+0000 7f05ab7fe640 1 --2- 172.21.15.32:0/3040840652 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0598003f30 0x7f0598004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:45.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.505+0000 7f05ab7fe640 1 -- 172.21.15.32:0/3040840652 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0598003520 con 0x7f0598004d40 2024-04-24T22:59:45.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.505+0000 7f05a3fff640 1 --2- 172.21.15.32:0/3040840652 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0598003f30 0x7f0598004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:45.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.506+0000 7f05ab7fe640 1 --2- 172.21.15.32:0/3040840652 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0598004d40 0x7f05980051c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f05a400a870 tx=0x7f0598005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc7cb33d8bc7baee server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:45.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.506+0000 7f05aa7fc640 1 -- 172.21.15.32:0/3040840652 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a400a4c0 con 0x7f0598004d40 2024-04-24T22:59:45.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.506+0000 7f05aa7fc640 1 -- 172.21.15.32:0/3040840652 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05a402f070 con 0x7f0598004d40 2024-04-24T22:59:45.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.506+0000 7f05aa7fc640 1 -- 172.21.15.32:0/3040840652 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a402bcb0 con 0x7f0598004d40 2024-04-24T22:59:45.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.506+0000 7f05b135a640 1 -- 172.21.15.32:0/3040840652 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0598004d40 msgr2=0x7f05980051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:45.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.506+0000 7f05b135a640 1 --2- 172.21.15.32:0/3040840652 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0598004d40 0x7f05980051c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f05a400a870 tx=0x7f0598005830 comp rx=0 tx=0).stop 2024-04-24T22:59:45.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.506+0000 7f05ab7fe640 1 -- 172.21.15.32:0/3040840652 reap_dead start 2024-04-24T22:59:45.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.506+0000 7f05b135a640 1 -- 172.21.15.32:0/3040840652 shutdown_connections 2024-04-24T22:59:45.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.506+0000 7f05b135a640 1 -- 172.21.15.32:0/3040840652 >> 172.21.15.32:0/3040840652 conn(0x7f05ac0b2e60 msgr2=0x7f05ac0b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:45.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.506+0000 7f05b135a640 1 -- 172.21.15.32:0/3040840652 shutdown_connections 2024-04-24T22:59:45.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.506+0000 7f05b135a640 1 -- 172.21.15.32:0/3040840652 wait complete. 2024-04-24T22:59:45.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.508+0000 7f05b135a640 1 Processor -- start 2024-04-24T22:59:45.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.508+0000 7f05b135a640 1 -- start start 2024-04-24T22:59:45.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.508+0000 7f05b135a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f05ac15db60 0x7f05ac0ca5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:45.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.508+0000 7f05b135a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05ac0caae0 0x7f05ac09d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:45.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.508+0000 7f05b135a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05ac09d610 0x7f05ac1e9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:45.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.508+0000 7f05b135a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f05ac0b18b0 con 0x7f05ac0caae0 2024-04-24T22:59:45.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.508+0000 7f05b135a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f05ac0b15d0 con 0x7f05ac09d610 2024-04-24T22:59:45.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.508+0000 7f05b135a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f05ac0b3ba0 con 0x7f05ac15db60 2024-04-24T22:59:45.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.508+0000 7f05a3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05ac0caae0 0x7f05ac09d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:45.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.508+0000 7f05a3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05ac0caae0 0x7f05ac09d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57616/0 (socket says 172.21.15.32:57616) 2024-04-24T22:59:45.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.508+0000 7f05a3fff640 1 -- 172.21.15.32:0/2714709416 learned_addr learned my addr 172.21.15.32:0/2714709416 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:45.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.509+0000 7f05aaffd640 1 --2- 172.21.15.32:0/2714709416 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f05ac15db60 0x7f05ac0ca5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:45.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.509+0000 7f05ab7fe640 1 --2- 172.21.15.32:0/2714709416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05ac09d610 0x7f05ac1e9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:45.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.509+0000 7f05a3fff640 1 -- 172.21.15.32:0/2714709416 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f05ac15db60 msgr2=0x7f05ac0ca5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:45.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.509+0000 7f05a3fff640 1 --2- 172.21.15.32:0/2714709416 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f05ac15db60 0x7f05ac0ca5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:45.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.509+0000 7f05a3fff640 1 -- 172.21.15.32:0/2714709416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05ac09d610 msgr2=0x7f05ac1e9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:45.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.509+0000 7f05a3fff640 1 --2- 172.21.15.32:0/2714709416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05ac09d610 0x7f05ac1e9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:45.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.509+0000 7f05a3fff640 1 -- 172.21.15.32:0/2714709416 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05ac09de70 con 0x7f05ac0caae0 2024-04-24T22:59:45.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.509+0000 7f05ab7fe640 1 --2- 172.21.15.32:0/2714709416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05ac09d610 0x7f05ac1e9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:45.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.509+0000 7f05a3fff640 1 --2- 172.21.15.32:0/2714709416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05ac0caae0 0x7f05ac09d0d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f05940115d0 tx=0x7f05940119d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:45.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.509+0000 7f05a37fe640 1 -- 172.21.15.32:0/2714709416 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0594015870 con 0x7f05ac0caae0 2024-04-24T22:59:45.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.509+0000 7f05a37fe640 1 -- 172.21.15.32:0/2714709416 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0594015a10 con 0x7f05ac0caae0 2024-04-24T22:59:45.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.509+0000 7f05b135a640 1 -- 172.21.15.32:0/2714709416 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05ac09e150 con 0x7f05ac0caae0 2024-04-24T22:59:45.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.509+0000 7f05a37fe640 1 -- 172.21.15.32:0/2714709416 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0594015de0 con 0x7f05ac0caae0 2024-04-24T22:59:45.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.509+0000 7f05b135a640 1 -- 172.21.15.32:0/2714709416 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05ac09e710 con 0x7f05ac0caae0 2024-04-24T22:59:45.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.510+0000 7f05b135a640 1 -- 172.21.15.32:0/2714709416 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05ac068430 con 0x7f05ac0caae0 2024-04-24T22:59:45.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.511+0000 7f05a37fe640 1 -- 172.21.15.32:0/2714709416 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f059401d400 con 0x7f05ac0caae0 2024-04-24T22:59:45.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.511+0000 7f05a37fe640 1 --2- 172.21.15.32:0/2714709416 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f058c06be60 0x7f058c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:45.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.511+0000 7f05a37fe640 1 -- 172.21.15.32:0/2714709416 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f059408b6d0 con 0x7f05ac0caae0 2024-04-24T22:59:45.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.511+0000 7f05aaffd640 1 --2- 172.21.15.32:0/2714709416 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f058c06be60 0x7f058c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:45.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.512+0000 7f05aaffd640 1 --2- 172.21.15.32:0/2714709416 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f058c06be60 0x7f058c06e320 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f059c009bc0 tx=0x7f059c009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:45.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.515+0000 7f05a37fe640 1 -- 172.21.15.32:0/2714709416 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0594019030 con 0x7f05ac0caae0 2024-04-24T22:59:45.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.700+0000 7f05b135a640 1 -- 172.21.15.32:0/2714709416 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f05ac06b220 con 0x7f05ac0caae0 2024-04-24T22:59:45.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.700+0000 7f05a37fe640 1 -- 172.21.15.32:0/2714709416 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f059405a080 con 0x7f05ac0caae0 2024-04-24T22:59:45.702 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-24T22:59:45.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.702+0000 7f05b135a640 1 -- 172.21.15.32:0/2714709416 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f058c06be60 msgr2=0x7f058c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:45.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.702+0000 7f05b135a640 1 --2- 172.21.15.32:0/2714709416 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f058c06be60 0x7f058c06e320 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f059c009bc0 tx=0x7f059c009190 comp rx=0 tx=0).stop 2024-04-24T22:59:45.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.702+0000 7f05b135a640 1 -- 172.21.15.32:0/2714709416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05ac0caae0 msgr2=0x7f05ac09d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:45.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.702+0000 7f05b135a640 1 --2- 172.21.15.32:0/2714709416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05ac0caae0 0x7f05ac09d0d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f05940115d0 tx=0x7f05940119d0 comp rx=0 tx=0).stop 2024-04-24T22:59:45.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.703+0000 7f05b135a640 1 -- 172.21.15.32:0/2714709416 shutdown_connections 2024-04-24T22:59:45.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.703+0000 7f05b135a640 1 --2- 172.21.15.32:0/2714709416 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f058c06be60 0x7f058c06e320 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:45.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.703+0000 7f05b135a640 1 --2- 172.21.15.32:0/2714709416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05ac0caae0 0x7f05ac09d0d0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:45.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.703+0000 7f05b135a640 1 --2- 172.21.15.32:0/2714709416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05ac09d610 0x7f05ac1e9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:45.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.703+0000 7f05b135a640 1 --2- 172.21.15.32:0/2714709416 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f05ac15db60 0x7f05ac0ca5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:45.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.703+0000 7f05b135a640 1 -- 172.21.15.32:0/2714709416 >> 172.21.15.32:0/2714709416 conn(0x7f05ac0b2e60 msgr2=0x7f05ac06eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:45.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.703+0000 7f05b135a640 1 -- 172.21.15.32:0/2714709416 shutdown_connections 2024-04-24T22:59:45.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.703+0000 7f05b135a640 1 -- 172.21.15.32:0/2714709416 wait complete. 2024-04-24T22:59:45.715 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T22:59:45.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.816+0000 7ff06f5d5640 1 Processor -- start 2024-04-24T22:59:45.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.817+0000 7ff06f5d5640 1 -- start start 2024-04-24T22:59:45.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.817+0000 7ff06f5d5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff06815e9b0 0x7ff0680d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:45.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.817+0000 7ff06f5d5640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0680d3170 con 0x7ff0680d4210 2024-04-24T22:59:45.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.817+0000 7ff06f5d5640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0680d3450 con 0x7ff06806b820 2024-04-24T22:59:45.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.817+0000 7ff06f5d5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7ff0680d3730 con 0x7ff06815e9b0 2024-04-24T22:59:45.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.817+0000 7ff06d371640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff06815e9b0 0x7ff0680d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:45.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.817+0000 7ff06cb70640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff0680d4210 0x7ff06806b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52106/0 (socket says 172.21.15.32:52106) 2024-04-24T22:59:45.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.817+0000 7ff06cb70640 1 -- 172.21.15.32:0/3289151473 learned_addr learned my addr 172.21.15.32:0/3289151473 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:45.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.818+0000 7ff06d371640 1 -- 172.21.15.32:0/3289151473 >> v1:172.21.15.121:6789/0 conn(0x7ff06806b820 legacy=0x7ff0680b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:45.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.818+0000 7ff06d371640 1 -- 172.21.15.32:0/3289151473 >> v1:172.21.15.32:6789/0 conn(0x7ff0680d4210 legacy=0x7ff06806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:45.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.818+0000 7ff06d371640 1 -- 172.21.15.32:0/3289151473 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0680d3a10 con 0x7ff06815e9b0 2024-04-24T22:59:45.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.818+0000 7ff05ffff640 1 -- 172.21.15.32:0/3289151473 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 670976865 0 0) 0x7ff0680d3170 con 0x7ff0680d4210 2024-04-24T22:59:45.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.818+0000 7ff06d371640 1 --2- 172.21.15.32:0/3289151473 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff06815e9b0 0x7ff0680d2b90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff0640097a0 tx=0x7ff06402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8226f6458f87136 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:45.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.819+0000 7ff05ffff640 1 -- 172.21.15.32:0/3289151473 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff064037070 con 0x7ff06815e9b0 2024-04-24T22:59:45.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.819+0000 7ff05ffff640 1 -- 172.21.15.32:0/3289151473 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff06402ae50 con 0x7ff06815e9b0 2024-04-24T22:59:45.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.819+0000 7ff05ffff640 1 -- 172.21.15.32:0/3289151473 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff064032660 con 0x7ff06815e9b0 2024-04-24T22:59:45.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.819+0000 7ff06f5d5640 1 -- 172.21.15.32:0/3289151473 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff06815e9b0 msgr2=0x7ff0680d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:45.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.819+0000 7ff06f5d5640 1 --2- 172.21.15.32:0/3289151473 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff06815e9b0 0x7ff0680d2b90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff0640097a0 tx=0x7ff06402a140 comp rx=0 tx=0).stop 2024-04-24T22:59:45.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.819+0000 7ff06f5d5640 1 -- 172.21.15.32:0/3289151473 shutdown_connections 2024-04-24T22:59:45.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.819+0000 7ff06f5d5640 1 --2- 172.21.15.32:0/3289151473 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff06815e9b0 0x7ff0680d2b90 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:45.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.819+0000 7ff06f5d5640 1 -- 172.21.15.32:0/3289151473 >> 172.21.15.32:0/3289151473 conn(0x7ff0680d11d0 msgr2=0x7ff0680d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:45.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.819+0000 7ff06f5d5640 1 -- 172.21.15.32:0/3289151473 shutdown_connections 2024-04-24T22:59:45.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.819+0000 7ff06f5d5640 1 -- 172.21.15.32:0/3289151473 wait complete. 2024-04-24T22:59:45.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.820+0000 7ff06f5d5640 1 Processor -- start 2024-04-24T22:59:45.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.821+0000 7ff06f5d5640 1 -- start start 2024-04-24T22:59:45.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.821+0000 7ff06f5d5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff06806b820 0x7ff068150fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:45.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.821+0000 7ff06f5d5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff0680d4210 0x7ff068161520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:45.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.821+0000 7ff06f5d5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff06815e9b0 0x7ff068071730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:45.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.821+0000 7ff06f5d5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff0680ba0e0 con 0x7ff06815e9b0 2024-04-24T22:59:45.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.821+0000 7ff06f5d5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff0680d2400 con 0x7ff06806b820 2024-04-24T22:59:45.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.821+0000 7ff06f5d5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff0680ba260 con 0x7ff0680d4210 2024-04-24T22:59:45.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.821+0000 7ff06db72640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff06815e9b0 0x7ff068071730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:45.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.821+0000 7ff06db72640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff06815e9b0 0x7ff068071730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57636/0 (socket says 172.21.15.32:57636) 2024-04-24T22:59:45.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.821+0000 7ff06db72640 1 -- 172.21.15.32:0/1035112554 learned_addr learned my addr 172.21.15.32:0/1035112554 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:45.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.822+0000 7ff06cb70640 1 --2- 172.21.15.32:0/1035112554 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff0680d4210 0x7ff068161520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:45.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.822+0000 7ff06db72640 1 -- 172.21.15.32:0/1035112554 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff0680d4210 msgr2=0x7ff068161520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:45.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.822+0000 7ff06d371640 1 --2- 172.21.15.32:0/1035112554 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff06806b820 0x7ff068150fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:45.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.822+0000 7ff06db72640 1 --2- 172.21.15.32:0/1035112554 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff0680d4210 0x7ff068161520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:45.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.822+0000 7ff06db72640 1 -- 172.21.15.32:0/1035112554 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff06806b820 msgr2=0x7ff068150fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:45.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.822+0000 7ff06db72640 1 --2- 172.21.15.32:0/1035112554 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff06806b820 0x7ff068150fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:45.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.822+0000 7ff06db72640 1 -- 172.21.15.32:0/1035112554 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff06815eea0 con 0x7ff06815e9b0 2024-04-24T22:59:45.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.822+0000 7ff06cb70640 1 --2- 172.21.15.32:0/1035112554 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff0680d4210 0x7ff068161520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:59:45.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.822+0000 7ff06d371640 1 --2- 172.21.15.32:0/1035112554 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff06806b820 0x7ff068150fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:59:45.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.822+0000 7ff06db72640 1 --2- 172.21.15.32:0/1035112554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff06815e9b0 0x7ff068071730 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff06000c7b0 tx=0x7ff06000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:45.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.822+0000 7ff05dffb640 1 -- 172.21.15.32:0/1035112554 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff060017070 con 0x7ff06815e9b0 2024-04-24T22:59:45.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.822+0000 7ff05dffb640 1 -- 172.21.15.32:0/1035112554 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff060003da0 con 0x7ff06815e9b0 2024-04-24T22:59:45.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.822+0000 7ff06f5d5640 1 -- 172.21.15.32:0/1035112554 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff06815f180 con 0x7ff06815e9b0 2024-04-24T22:59:45.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.822+0000 7ff05dffb640 1 -- 172.21.15.32:0/1035112554 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0600052b0 con 0x7ff06815e9b0 2024-04-24T22:59:45.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.822+0000 7ff06f5d5640 1 -- 172.21.15.32:0/1035112554 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff06815f740 con 0x7ff06815e9b0 2024-04-24T22:59:45.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.823+0000 7ff06f5d5640 1 -- 172.21.15.32:0/1035112554 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0680cebc0 con 0x7ff06815e9b0 2024-04-24T22:59:45.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.824+0000 7ff05dffb640 1 -- 172.21.15.32:0/1035112554 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff06001a070 con 0x7ff06815e9b0 2024-04-24T22:59:45.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.824+0000 7ff05dffb640 1 --2- 172.21.15.32:0/1035112554 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff04c06be30 0x7ff04c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:45.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.824+0000 7ff05dffb640 1 -- 172.21.15.32:0/1035112554 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff06008a460 con 0x7ff06815e9b0 2024-04-24T22:59:45.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.824+0000 7ff06d371640 1 --2- 172.21.15.32:0/1035112554 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff04c06be30 0x7ff04c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:45.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.825+0000 7ff06d371640 1 --2- 172.21.15.32:0/1035112554 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff04c06be30 0x7ff04c06e2f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff064034000 tx=0x7ff06402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:45.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:45.828+0000 7ff05dffb640 1 -- 172.21.15.32:0/1035112554 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff0680cebc0 con 0x7ff06815e9b0 2024-04-24T22:59:46.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.011+0000 7ff06f5d5640 1 -- 172.21.15.32:0/1035112554 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff068068430 con 0x7ff06815e9b0 2024-04-24T22:59:46.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.011+0000 7ff05dffb640 1 -- 172.21.15.32:0/1035112554 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7ff060058e10 con 0x7ff06815e9b0 2024-04-24T22:59:46.013 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-24T22:59:46.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.013+0000 7ff06f5d5640 1 -- 172.21.15.32:0/1035112554 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff04c06be30 msgr2=0x7ff04c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:46.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.013+0000 7ff06f5d5640 1 --2- 172.21.15.32:0/1035112554 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff04c06be30 0x7ff04c06e2f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff064034000 tx=0x7ff06402e000 comp rx=0 tx=0).stop 2024-04-24T22:59:46.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.013+0000 7ff06f5d5640 1 -- 172.21.15.32:0/1035112554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff06815e9b0 msgr2=0x7ff068071730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:46.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.013+0000 7ff06f5d5640 1 --2- 172.21.15.32:0/1035112554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff06815e9b0 0x7ff068071730 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff06000c7b0 tx=0x7ff06000cc70 comp rx=0 tx=0).stop 2024-04-24T22:59:46.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.013+0000 7ff06f5d5640 1 -- 172.21.15.32:0/1035112554 shutdown_connections 2024-04-24T22:59:46.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.013+0000 7ff06f5d5640 1 --2- 172.21.15.32:0/1035112554 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff04c06be30 0x7ff04c06e2f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:46.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.013+0000 7ff06f5d5640 1 --2- 172.21.15.32:0/1035112554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff06815e9b0 0x7ff068071730 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:46.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.013+0000 7ff06f5d5640 1 --2- 172.21.15.32:0/1035112554 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff0680d4210 0x7ff068161520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:46.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.013+0000 7ff06f5d5640 1 --2- 172.21.15.32:0/1035112554 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff06806b820 0x7ff068150fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:46.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.013+0000 7ff06f5d5640 1 -- 172.21.15.32:0/1035112554 >> 172.21.15.32:0/1035112554 conn(0x7ff0680d11d0 msgr2=0x7ff0680ca290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:46.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.013+0000 7ff06f5d5640 1 -- 172.21.15.32:0/1035112554 shutdown_connections 2024-04-24T22:59:46.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.013+0000 7ff06f5d5640 1 -- 172.21.15.32:0/1035112554 wait complete. 2024-04-24T22:59:46.025 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T22:59:46.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.125+0000 7fdcf7b1b640 1 Processor -- start 2024-04-24T22:59:46.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.126+0000 7fdcf7b1b640 1 -- start start 2024-04-24T22:59:46.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.127+0000 7fdcf7b1b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdcf015b380 0x7fdcf006c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:46.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.127+0000 7fdcf7b1b640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdcf00ceb40 0x7fdcf006cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:46.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.127+0000 7fdcf7b1b640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcf00b4520 con 0x7fdcf015cb00 2024-04-24T22:59:46.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.127+0000 7fdcf7b1b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fdcf00cef70 con 0x7fdcf015b380 2024-04-24T22:59:46.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.127+0000 7fdcf7b1b640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fdcf00b4800 con 0x7fdcf00ceb40 2024-04-24T22:59:46.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.127+0000 7fdcf58b7640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdcf015b380 0x7fdcf006c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:46.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.127+0000 7fdcf58b7640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdcf015b380 0x7fdcf006c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57644/0 (socket says 172.21.15.32:57644) 2024-04-24T22:59:46.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.127+0000 7fdcf58b7640 1 -- 172.21.15.32:0/2034910398 learned_addr learned my addr 172.21.15.32:0/2034910398 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:46.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.127+0000 7fdcf60b8640 1 --2- 172.21.15.32:0/2034910398 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdcf00ceb40 0x7fdcf006cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:46.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.127+0000 7fdcf58b7640 1 -- 172.21.15.32:0/2034910398 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdcf00ceb40 msgr2=0x7fdcf006cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:46.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.127+0000 7fdcf58b7640 1 --2- 172.21.15.32:0/2034910398 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdcf00ceb40 0x7fdcf006cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:46.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.127+0000 7fdcf58b7640 1 -- 172.21.15.32:0/2034910398 >> v1:172.21.15.121:6789/0 conn(0x7fdcf015cb00 legacy=0x7fdcf00ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:46.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.127+0000 7fdcf58b7640 1 -- 172.21.15.32:0/2034910398 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcf00b4ae0 con 0x7fdcf015b380 2024-04-24T22:59:46.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.128+0000 7fdcf58b7640 1 --2- 172.21.15.32:0/2034910398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcf015b380 0x7fdcf006c5f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fdce0009870 tx=0x7fdce002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c1d79aa55caabe server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:46.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.128+0000 7fdcf48b5640 1 -- 172.21.15.32:0/2034910398 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce0037070 con 0x7fdcf015b380 2024-04-24T22:59:46.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.128+0000 7fdcf48b5640 1 -- 172.21.15.32:0/2034910398 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdce002ae10 con 0x7fdcf015b380 2024-04-24T22:59:46.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.128+0000 7fdcf48b5640 1 -- 172.21.15.32:0/2034910398 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce0032660 con 0x7fdcf015b380 2024-04-24T22:59:46.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.129+0000 7fdcf7b1b640 1 -- 172.21.15.32:0/2034910398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcf015b380 msgr2=0x7fdcf006c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:46.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.129+0000 7fdcf7b1b640 1 --2- 172.21.15.32:0/2034910398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcf015b380 0x7fdcf006c5f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fdce0009870 tx=0x7fdce002a100 comp rx=0 tx=0).stop 2024-04-24T22:59:46.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.129+0000 7fdcf7b1b640 1 -- 172.21.15.32:0/2034910398 shutdown_connections 2024-04-24T22:59:46.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.129+0000 7fdcf7b1b640 1 --2- 172.21.15.32:0/2034910398 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdcf00ceb40 0x7fdcf006cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:46.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.129+0000 7fdcf7b1b640 1 --2- 172.21.15.32:0/2034910398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcf015b380 0x7fdcf006c5f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:46.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.129+0000 7fdcf7b1b640 1 -- 172.21.15.32:0/2034910398 >> 172.21.15.32:0/2034910398 conn(0x7fdcf006b110 msgr2=0x7fdcf006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:46.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.129+0000 7fdcf7b1b640 1 -- 172.21.15.32:0/2034910398 shutdown_connections 2024-04-24T22:59:46.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.129+0000 7fdcf7b1b640 1 -- 172.21.15.32:0/2034910398 wait complete. 2024-04-24T22:59:46.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.130+0000 7fdcf7b1b640 1 Processor -- start 2024-04-24T22:59:46.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.130+0000 7fdcf7b1b640 1 -- start start 2024-04-24T22:59:46.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.131+0000 7fdcf7b1b640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdcf00ceb40 0x7fdcf015d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:46.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.131+0000 7fdcf7b1b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdcf015b380 0x7fdcf014d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:46.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.131+0000 7fdcf7b1b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcf015cb00 0x7fdcf00ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:46.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.131+0000 7fdcf7b1b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf015b760 con 0x7fdcf015cb00 2024-04-24T22:59:46.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.131+0000 7fdcf7b1b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf00b50a0 con 0x7fdcf015b380 2024-04-24T22:59:46.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.131+0000 7fdcf7b1b640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fdcf00b55c0 con 0x7fdcf00ceb40 2024-04-24T22:59:46.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.131+0000 7fdcf60b8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcf015cb00 0x7fdcf00ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:46.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.131+0000 7fdcf60b8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcf015cb00 0x7fdcf00ae8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57654/0 (socket says 172.21.15.32:57654) 2024-04-24T22:59:46.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.131+0000 7fdcf60b8640 1 -- 172.21.15.32:0/3451338563 learned_addr learned my addr 172.21.15.32:0/3451338563 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:46.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.131+0000 7fdcf50b6640 1 --2- 172.21.15.32:0/3451338563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdcf015b380 0x7fdcf014d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:46.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.131+0000 7fdcf58b7640 1 --2- 172.21.15.32:0/3451338563 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdcf00ceb40 0x7fdcf015d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:46.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.131+0000 7fdcf60b8640 1 -- 172.21.15.32:0/3451338563 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdcf00ceb40 msgr2=0x7fdcf015d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:46.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.131+0000 7fdcf60b8640 1 --2- 172.21.15.32:0/3451338563 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdcf00ceb40 0x7fdcf015d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:46.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.131+0000 7fdcf60b8640 1 -- 172.21.15.32:0/3451338563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdcf015b380 msgr2=0x7fdcf014d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:46.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.131+0000 7fdcf60b8640 1 --2- 172.21.15.32:0/3451338563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdcf015b380 0x7fdcf014d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:46.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.131+0000 7fdcf60b8640 1 -- 172.21.15.32:0/3451338563 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcf015d720 con 0x7fdcf015cb00 2024-04-24T22:59:46.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.131+0000 7fdcf60b8640 1 --2- 172.21.15.32:0/3451338563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcf015cb00 0x7fdcf00ae8b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fdce8007e70 tx=0x7fdce800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:46.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.132+0000 7fdce67fc640 1 -- 172.21.15.32:0/3451338563 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce8014620 con 0x7fdcf015cb00 2024-04-24T22:59:46.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.132+0000 7fdce67fc640 1 -- 172.21.15.32:0/3451338563 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdce8019070 con 0x7fdcf015cb00 2024-04-24T22:59:46.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.132+0000 7fdcf7b1b640 1 -- 172.21.15.32:0/3451338563 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcf015da00 con 0x7fdcf015cb00 2024-04-24T22:59:46.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.132+0000 7fdce67fc640 1 -- 172.21.15.32:0/3451338563 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce8014a20 con 0x7fdcf015cb00 2024-04-24T22:59:46.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.132+0000 7fdcf7b1b640 1 -- 172.21.15.32:0/3451338563 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcf015dfc0 con 0x7fdcf015cb00 2024-04-24T22:59:46.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.133+0000 7fdcf7b1b640 1 -- 172.21.15.32:0/3451338563 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcf0068430 con 0x7fdcf015cb00 2024-04-24T22:59:46.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.133+0000 7fdce67fc640 1 -- 172.21.15.32:0/3451338563 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdce8014bc0 con 0x7fdcf015cb00 2024-04-24T22:59:46.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.134+0000 7fdce67fc640 1 --2- 172.21.15.32:0/3451338563 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdcd406be30 0x7fdcd406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:46.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.134+0000 7fdce67fc640 1 -- 172.21.15.32:0/3451338563 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fdce808b610 con 0x7fdcf015cb00 2024-04-24T22:59:46.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.134+0000 7fdcf58b7640 1 --2- 172.21.15.32:0/3451338563 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdcd406be30 0x7fdcd406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:46.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.134+0000 7fdcf58b7640 1 --2- 172.21.15.32:0/3451338563 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdcd406be30 0x7fdcd406e2f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdce0034000 tx=0x7fdce002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:46.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.138+0000 7fdce67fc640 1 -- 172.21.15.32:0/3451338563 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdce8059970 con 0x7fdcf015cb00 2024-04-24T22:59:46.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.361+0000 7fdcf7b1b640 1 -- 172.21.15.32:0/3451338563 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdcf006c000 con 0x7fdcf015cb00 2024-04-24T22:59:46.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.362+0000 7fdce67fc640 1 -- 172.21.15.32:0/3451338563 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5178 (secure 0 0 0) 0x7fdce805dc70 con 0x7fdcf015cb00 2024-04-24T22:59:46.363 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:59:46.363 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1485884060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1485884060},{"type":"v1","addr":"172.21.15.121:6837","nonce":1485884060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/923934104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":923934104},{"type":"v1","addr":"172.21.15.32:6833","nonce":923934104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3229115835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3229115835},{"type":"v1","addr":"172.21.15.32:6835","nonce":3229115835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T22:59:03.876305+0000","modified":"2024-04-24T22:59:05.932147+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6836/2644625343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2644625343},{"type":"v1","addr":"172.21.15.121:6836","nonce":2644625343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4276]},"id":1}]} 2024-04-24T22:59:46.364 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 6 2024-04-24T22:59:46.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.364+0000 7fdcf7b1b640 1 -- 172.21.15.32:0/3451338563 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdcd406be30 msgr2=0x7fdcd406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:46.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.364+0000 7fdcf7b1b640 1 --2- 172.21.15.32:0/3451338563 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdcd406be30 0x7fdcd406e2f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdce0034000 tx=0x7fdce002e000 comp rx=0 tx=0).stop 2024-04-24T22:59:46.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.364+0000 7fdcf7b1b640 1 -- 172.21.15.32:0/3451338563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcf015cb00 msgr2=0x7fdcf00ae8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:46.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.364+0000 7fdcf7b1b640 1 --2- 172.21.15.32:0/3451338563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcf015cb00 0x7fdcf00ae8b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fdce8007e70 tx=0x7fdce800e4b0 comp rx=0 tx=0).stop 2024-04-24T22:59:46.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.364+0000 7fdcf7b1b640 1 -- 172.21.15.32:0/3451338563 shutdown_connections 2024-04-24T22:59:46.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.364+0000 7fdcf7b1b640 1 --2- 172.21.15.32:0/3451338563 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdcd406be30 0x7fdcd406e2f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:46.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.364+0000 7fdcf7b1b640 1 --2- 172.21.15.32:0/3451338563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcf015cb00 0x7fdcf00ae8b0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:46.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.364+0000 7fdcf7b1b640 1 --2- 172.21.15.32:0/3451338563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdcf015b380 0x7fdcf014d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:46.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.364+0000 7fdcf7b1b640 1 --2- 172.21.15.32:0/3451338563 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdcf00ceb40 0x7fdcf015d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:46.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.364+0000 7fdcf7b1b640 1 -- 172.21.15.32:0/3451338563 >> 172.21.15.32:0/3451338563 conn(0x7fdcf006b110 msgr2=0x7fdcf00b5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:46.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.364+0000 7fdcf7b1b640 1 -- 172.21.15.32:0/3451338563 shutdown_connections 2024-04-24T22:59:46.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.364+0000 7fdcf7b1b640 1 -- 172.21.15.32:0/3451338563 wait complete. 2024-04-24T22:59:46.376 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:59:46.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.477+0000 7f0176d63640 1 Processor -- start 2024-04-24T22:59:46.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.478+0000 7f0176d63640 1 -- start start 2024-04-24T22:59:46.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.478+0000 7f0176d63640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f017015e9b0 0x7f01700d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:46.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.478+0000 7f0176d63640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f017006b820 0x7f01700b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:46.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.478+0000 7f0176d63640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01700d3170 con 0x7f01700d4210 2024-04-24T22:59:46.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.478+0000 7f0176d63640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f01700d3450 con 0x7f017006b820 2024-04-24T22:59:46.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.478+0000 7f0176d63640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f01700d3730 con 0x7f017015e9b0 2024-04-24T22:59:46.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.479+0000 7f0175300640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f017006b820 0x7f01700b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:46.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.479+0000 7f0175300640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f017006b820 0x7f01700b78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57666/0 (socket says 172.21.15.32:57666) 2024-04-24T22:59:46.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.479+0000 7f0175300640 1 -- 172.21.15.32:0/628725129 learned_addr learned my addr 172.21.15.32:0/628725129 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:46.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.479+0000 7f0174aff640 1 --2- 172.21.15.32:0/628725129 >> v2:172.21.15.121:3300/0 conn(0x7f017015e9b0 0x7f01700d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:46.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.479+0000 7f0175300640 1 -- 172.21.15.32:0/628725129 >> v2:172.21.15.121:3300/0 conn(0x7f017015e9b0 msgr2=0x7f01700d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:46.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.479+0000 7f0175300640 1 --2- 172.21.15.32:0/628725129 >> v2:172.21.15.121:3300/0 conn(0x7f017015e9b0 0x7f01700d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:46.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.479+0000 7f0175300640 1 -- 172.21.15.32:0/628725129 >> v1:172.21.15.121:6789/0 conn(0x7f01700d4210 legacy=0x7f017006b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:46.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.479+0000 7f0175300640 1 -- 172.21.15.32:0/628725129 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01700d3a10 con 0x7f017006b820 2024-04-24T22:59:46.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.479+0000 7f0174aff640 1 --2- 172.21.15.32:0/628725129 >> v2:172.21.15.121:3300/0 conn(0x7f017015e9b0 0x7f01700d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:46.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.480+0000 7f0175300640 1 --2- 172.21.15.32:0/628725129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f017006b820 0x7f01700b78f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f017006be20 tx=0x7f016802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=663dcd11adceecad server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:46.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.480+0000 7f016f7fe640 1 -- 172.21.15.32:0/628725129 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0168036070 con 0x7f017006b820 2024-04-24T22:59:46.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.480+0000 7f016f7fe640 1 -- 172.21.15.32:0/628725129 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0168031600 con 0x7f017006b820 2024-04-24T22:59:46.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.480+0000 7f016f7fe640 1 -- 172.21.15.32:0/628725129 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01680319d0 con 0x7f017006b820 2024-04-24T22:59:46.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.480+0000 7f0176d63640 1 -- 172.21.15.32:0/628725129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f017006b820 msgr2=0x7f01700b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:46.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.480+0000 7f0176d63640 1 --2- 172.21.15.32:0/628725129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f017006b820 0x7f01700b78f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f017006be20 tx=0x7f016802a6a0 comp rx=0 tx=0).stop 2024-04-24T22:59:46.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.480+0000 7f0176d63640 1 -- 172.21.15.32:0/628725129 shutdown_connections 2024-04-24T22:59:46.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.480+0000 7f0176d63640 1 --2- 172.21.15.32:0/628725129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f017006b820 0x7f01700b78f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:46.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.480+0000 7f0176d63640 1 --2- 172.21.15.32:0/628725129 >> v2:172.21.15.121:3300/0 conn(0x7f017015e9b0 0x7f01700d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:46.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.480+0000 7f0176d63640 1 -- 172.21.15.32:0/628725129 >> 172.21.15.32:0/628725129 conn(0x7f01700d11d0 msgr2=0x7f01700d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:46.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.480+0000 7f0176d63640 1 -- 172.21.15.32:0/628725129 shutdown_connections 2024-04-24T22:59:46.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.480+0000 7f0176d63640 1 -- 172.21.15.32:0/628725129 wait complete. 2024-04-24T22:59:46.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.482+0000 7f0176d63640 1 Processor -- start 2024-04-24T22:59:46.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.482+0000 7f0176d63640 1 -- start start 2024-04-24T22:59:46.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.482+0000 7f0176d63640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f017006b820 0x7f01700ca320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:46.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.482+0000 7f0176d63640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01700d4210 0x7f01701e4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:46.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.483+0000 7f0176d63640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f017015e9b0 0x7f01701e49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:46.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.483+0000 7f0176d63640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f01700ba0e0 con 0x7f017006b820 2024-04-24T22:59:46.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.483+0000 7f0176d63640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f01700d2400 con 0x7f017015e9b0 2024-04-24T22:59:46.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.483+0000 7f0176d63640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f01700ba260 con 0x7f01700d4210 2024-04-24T22:59:46.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.483+0000 7f0174aff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f017006b820 0x7f01700ca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:46.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.483+0000 7f0174aff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f017006b820 0x7f01700ca320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57676/0 (socket says 172.21.15.32:57676) 2024-04-24T22:59:46.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.483+0000 7f0174aff640 1 -- 172.21.15.32:0/1310391087 learned_addr learned my addr 172.21.15.32:0/1310391087 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:46.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.483+0000 7f016ffff640 1 --2- 172.21.15.32:0/1310391087 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01700d4210 0x7f01701e4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:46.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.483+0000 7f0175300640 1 --2- 172.21.15.32:0/1310391087 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f017015e9b0 0x7f01701e49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:46.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.483+0000 7f0174aff640 1 -- 172.21.15.32:0/1310391087 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01700d4210 msgr2=0x7f01701e4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:46.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.483+0000 7f0174aff640 1 --2- 172.21.15.32:0/1310391087 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01700d4210 0x7f01701e4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:46.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.483+0000 7f0174aff640 1 -- 172.21.15.32:0/1310391087 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f017015e9b0 msgr2=0x7f01701e49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:46.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.483+0000 7f0174aff640 1 --2- 172.21.15.32:0/1310391087 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f017015e9b0 0x7f01701e49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:46.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.483+0000 7f0174aff640 1 -- 172.21.15.32:0/1310391087 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01701572c0 con 0x7f017006b820 2024-04-24T22:59:46.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.483+0000 7f0174aff640 1 --2- 172.21.15.32:0/1310391087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f017006b820 0x7f01700ca320 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f016400d7d0 tx=0x7f016400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:46.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.483+0000 7f016d7fa640 1 -- 172.21.15.32:0/1310391087 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0164004030 con 0x7f017006b820 2024-04-24T22:59:46.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.483+0000 7f016d7fa640 1 -- 172.21.15.32:0/1310391087 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01640041d0 con 0x7f017006b820 2024-04-24T22:59:46.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.483+0000 7f0176d63640 1 -- 172.21.15.32:0/1310391087 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0170157ac0 con 0x7f017006b820 2024-04-24T22:59:46.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.483+0000 7f016d7fa640 1 -- 172.21.15.32:0/1310391087 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01640096a0 con 0x7f017006b820 2024-04-24T22:59:46.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.483+0000 7f0176d63640 1 -- 172.21.15.32:0/1310391087 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0170158080 con 0x7f017006b820 2024-04-24T22:59:46.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.484+0000 7f0176d63640 1 -- 172.21.15.32:0/1310391087 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0170068430 con 0x7f017006b820 2024-04-24T22:59:46.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.485+0000 7f016d7fa640 1 -- 172.21.15.32:0/1310391087 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0164009840 con 0x7f017006b820 2024-04-24T22:59:46.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.485+0000 7f016d7fa640 1 --2- 172.21.15.32:0/1310391087 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f015406be30 0x7f015406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:46.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.486+0000 7f016d7fa640 1 -- 172.21.15.32:0/1310391087 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f016408af80 con 0x7f017006b820 2024-04-24T22:59:46.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.486+0000 7f016ffff640 1 --2- 172.21.15.32:0/1310391087 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f015406be30 0x7f015406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:46.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.486+0000 7f016ffff640 1 --2- 172.21.15.32:0/1310391087 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f015406be30 0x7f015406e2f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f01701e42d0 tx=0x7f015c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:46.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.490+0000 7f016d7fa640 1 -- 172.21.15.32:0/1310391087 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0164059370 con 0x7f017006b820 2024-04-24T22:59:46.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.678+0000 7f0176d63640 1 -- 172.21.15.32:0/1310391087 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f017015b630 con 0x7f017006b820 2024-04-24T22:59:46.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.679+0000 7f016d7fa640 1 -- 172.21.15.32:0/1310391087 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16621 (secure 0 0 0) 0x7f016405d670 con 0x7f017006b820 2024-04-24T22:59:46.681 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:59:46.681 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T22:59:05.517554+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:59:02.016639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T22:59:02.846087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:59:46.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.682+0000 7f0176d63640 1 -- 172.21.15.32:0/1310391087 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f015406be30 msgr2=0x7f015406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:46.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.682+0000 7f0176d63640 1 --2- 172.21.15.32:0/1310391087 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f015406be30 0x7f015406e2f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f01701e42d0 tx=0x7f015c009040 comp rx=0 tx=0).stop 2024-04-24T22:59:46.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.682+0000 7f0176d63640 1 -- 172.21.15.32:0/1310391087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f017006b820 msgr2=0x7f01700ca320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:46.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.682+0000 7f0176d63640 1 --2- 172.21.15.32:0/1310391087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f017006b820 0x7f01700ca320 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f016400d7d0 tx=0x7f016400dc90 comp rx=0 tx=0).stop 2024-04-24T22:59:46.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.682+0000 7f0176d63640 1 -- 172.21.15.32:0/1310391087 shutdown_connections 2024-04-24T22:59:46.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.682+0000 7f0176d63640 1 --2- 172.21.15.32:0/1310391087 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f015406be30 0x7f015406e2f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:46.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.682+0000 7f0176d63640 1 --2- 172.21.15.32:0/1310391087 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f017015e9b0 0x7f01701e49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:46.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.682+0000 7f0176d63640 1 --2- 172.21.15.32:0/1310391087 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01700d4210 0x7f01701e4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:46.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.682+0000 7f0176d63640 1 --2- 172.21.15.32:0/1310391087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f017006b820 0x7f01700ca320 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:46.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.682+0000 7f0176d63640 1 -- 172.21.15.32:0/1310391087 >> 172.21.15.32:0/1310391087 conn(0x7f01700d11d0 msgr2=0x7f01700b7aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:46.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.682+0000 7f0176d63640 1 -- 172.21.15.32:0/1310391087 shutdown_connections 2024-04-24T22:59:46.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.682+0000 7f0176d63640 1 -- 172.21.15.32:0/1310391087 wait complete. 2024-04-24T22:59:46.694 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-24T22:59:46.695 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T22:59:46.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.794+0000 7f89056d2640 1 Processor -- start 2024-04-24T22:59:46.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.795+0000 7f89056d2640 1 -- start start 2024-04-24T22:59:46.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.795+0000 7f89056d2640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f89000b5560 0x7f890015e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:46.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.795+0000 7f89056d2640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f89000b6050 0x7f890015ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:46.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.795+0000 7f89056d2640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89000cd910 con 0x7f89000cd3c0 2024-04-24T22:59:46.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.795+0000 7f89056d2640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f89000cdbf0 con 0x7f89000b6050 2024-04-24T22:59:46.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.795+0000 7f89056d2640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f89000cded0 con 0x7f89000b5560 2024-04-24T22:59:46.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.796+0000 7f88ff7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f89000b6050 0x7f890015ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:46.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.796+0000 7f88fe7fc640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f89000cd3c0 0x7f89000b5940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52114/0 (socket says 172.21.15.32:52114) 2024-04-24T22:59:46.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.796+0000 7f88fe7fc640 1 -- 172.21.15.32:0/1251163285 learned_addr learned my addr 172.21.15.32:0/1251163285 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:46.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.796+0000 7f88feffd640 1 --2- 172.21.15.32:0/1251163285 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f89000b5560 0x7f890015e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:46.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.796+0000 7f88ff7fe640 1 -- 172.21.15.32:0/1251163285 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f89000b5560 msgr2=0x7f890015e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:46.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.796+0000 7f88ff7fe640 1 --2- 172.21.15.32:0/1251163285 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f89000b5560 0x7f890015e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:46.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.796+0000 7f88ff7fe640 1 -- 172.21.15.32:0/1251163285 >> v1:172.21.15.32:6789/0 conn(0x7f89000cd3c0 legacy=0x7f89000b5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:46.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.796+0000 7f88ff7fe640 1 -- 172.21.15.32:0/1251163285 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89000ce1b0 con 0x7f89000b6050 2024-04-24T22:59:46.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.796+0000 7f88fdffb640 1 -- 172.21.15.32:0/1251163285 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 516430744 0 0) 0x7f89000cd910 con 0x7f89000cd3c0 2024-04-24T22:59:46.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.797+0000 7f88ff7fe640 1 --2- 172.21.15.32:0/1251163285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89000b6050 0x7f890015ec00 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f88f800bfa0 tx=0x7f88f802c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74c7cd5bffd930a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:46.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.797+0000 7f88fdffb640 1 -- 172.21.15.32:0/1251163285 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f8039070 con 0x7f89000b6050 2024-04-24T22:59:46.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.797+0000 7f88fdffb640 1 -- 172.21.15.32:0/1251163285 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88f8009540 con 0x7f89000b6050 2024-04-24T22:59:46.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.797+0000 7f88fdffb640 1 -- 172.21.15.32:0/1251163285 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f8009910 con 0x7f89000b6050 2024-04-24T22:59:46.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.797+0000 7f89056d2640 1 -- 172.21.15.32:0/1251163285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89000b6050 msgr2=0x7f890015ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:46.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.797+0000 7f89056d2640 1 --2- 172.21.15.32:0/1251163285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89000b6050 0x7f890015ec00 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f88f800bfa0 tx=0x7f88f802c860 comp rx=0 tx=0).stop 2024-04-24T22:59:46.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.797+0000 7f89056d2640 1 -- 172.21.15.32:0/1251163285 shutdown_connections 2024-04-24T22:59:46.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.797+0000 7f89056d2640 1 --2- 172.21.15.32:0/1251163285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89000b6050 0x7f890015ec00 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:46.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.797+0000 7f89056d2640 1 --2- 172.21.15.32:0/1251163285 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f89000b5560 0x7f890015e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:46.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.797+0000 7f89056d2640 1 -- 172.21.15.32:0/1251163285 >> 172.21.15.32:0/1251163285 conn(0x7f890006b110 msgr2=0x7f890006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:46.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.797+0000 7f89056d2640 1 -- 172.21.15.32:0/1251163285 shutdown_connections 2024-04-24T22:59:46.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.797+0000 7f89056d2640 1 -- 172.21.15.32:0/1251163285 wait complete. 2024-04-24T22:59:46.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.799+0000 7f89056d2640 1 Processor -- start 2024-04-24T22:59:46.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.799+0000 7f89056d2640 1 -- start start 2024-04-24T22:59:46.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.799+0000 7f89056d2640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f89000b5560 0x7f8900072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:46.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.799+0000 7f89056d2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89000b6050 0x7f89000aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:46.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.799+0000 7f89056d2640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89000cd3c0 0x7f8900070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:46.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.799+0000 7f89056d2640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f890006c770 con 0x7f89000b6050 2024-04-24T22:59:46.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.799+0000 7f89056d2640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f890006c5f0 con 0x7f89000cd3c0 2024-04-24T22:59:46.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.799+0000 7f89056d2640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f890006c8f0 con 0x7f89000b5560 2024-04-24T22:59:46.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.800+0000 7f88fe7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89000b6050 0x7f89000aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:46.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.800+0000 7f88fe7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89000b6050 0x7f89000aeb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57688/0 (socket says 172.21.15.32:57688) 2024-04-24T22:59:46.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.800+0000 7f88fe7fc640 1 -- 172.21.15.32:0/1748492399 learned_addr learned my addr 172.21.15.32:0/1748492399 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:46.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.800+0000 7f88feffd640 1 --2- 172.21.15.32:0/1748492399 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f89000b5560 0x7f8900072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:46.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.800+0000 7f88ff7fe640 1 --2- 172.21.15.32:0/1748492399 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89000cd3c0 0x7f8900070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:46.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.800+0000 7f88fe7fc640 1 -- 172.21.15.32:0/1748492399 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f89000b5560 msgr2=0x7f8900072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:46.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.800+0000 7f88fe7fc640 1 --2- 172.21.15.32:0/1748492399 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f89000b5560 0x7f8900072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:46.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.800+0000 7f88fe7fc640 1 -- 172.21.15.32:0/1748492399 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89000cd3c0 msgr2=0x7f8900070630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:46.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.800+0000 7f88fe7fc640 1 --2- 172.21.15.32:0/1748492399 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89000cd3c0 0x7f8900070630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:46.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.800+0000 7f88fe7fc640 1 -- 172.21.15.32:0/1748492399 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8900156570 con 0x7f89000b6050 2024-04-24T22:59:46.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.800+0000 7f88ff7fe640 1 --2- 172.21.15.32:0/1748492399 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89000cd3c0 0x7f8900070630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:59:46.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.800+0000 7f88fe7fc640 1 --2- 172.21.15.32:0/1748492399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89000b6050 0x7f89000aeb20 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f88f000c460 tx=0x7f88f000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:46.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.800+0000 7f88e77fe640 1 -- 172.21.15.32:0/1748492399 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f0018070 con 0x7f89000b6050 2024-04-24T22:59:46.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.800+0000 7f88e77fe640 1 -- 172.21.15.32:0/1748492399 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88f0007500 con 0x7f89000b6050 2024-04-24T22:59:46.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.800+0000 7f89056d2640 1 -- 172.21.15.32:0/1748492399 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8900156850 con 0x7f89000b6050 2024-04-24T22:59:46.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.800+0000 7f88e77fe640 1 -- 172.21.15.32:0/1748492399 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f0004a20 con 0x7f89000b6050 2024-04-24T22:59:46.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.800+0000 7f89056d2640 1 -- 172.21.15.32:0/1748492399 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89001570f0 con 0x7f89000b6050 2024-04-24T22:59:46.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.802+0000 7f89056d2640 1 -- 172.21.15.32:0/1748492399 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89001572f0 con 0x7f89000b6050 2024-04-24T22:59:46.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.802+0000 7f88e77fe640 1 -- 172.21.15.32:0/1748492399 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f88f001b070 con 0x7f89000b6050 2024-04-24T22:59:46.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.802+0000 7f88e77fe640 1 --2- 172.21.15.32:0/1748492399 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f88ec06be30 0x7f88ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:46.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.802+0000 7f88e77fe640 1 -- 172.21.15.32:0/1748492399 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f88f008a110 con 0x7f89000b6050 2024-04-24T22:59:46.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.802+0000 7f88feffd640 1 --2- 172.21.15.32:0/1748492399 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f88ec06be30 0x7f88ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:46.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.803+0000 7f88feffd640 1 --2- 172.21.15.32:0/1748492399 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f88ec06be30 0x7f88ec06e2f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f88f4009ad0 tx=0x7f88f4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:46.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:46.807+0000 7f88e77fe640 1 -- 172.21.15.32:0/1748492399 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f89001572f0 con 0x7f89000b6050 2024-04-24T22:59:47.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.060+0000 7f89056d2640 1 -- 172.21.15.32:0/1748492399 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f890006d5f0 con 0x7f89000b6050 2024-04-24T22:59:47.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.061+0000 7f88e77fe640 1 -- 172.21.15.32:0/1748492399 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5178 (secure 0 0 0) 0x7f88f0058ac0 con 0x7f89000b6050 2024-04-24T22:59:47.062 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:59:47.063 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1485884060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1485884060},{"type":"v1","addr":"172.21.15.121:6837","nonce":1485884060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/923934104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":923934104},{"type":"v1","addr":"172.21.15.32:6833","nonce":923934104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3229115835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3229115835},{"type":"v1","addr":"172.21.15.32:6835","nonce":3229115835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T22:59:03.876305+0000","modified":"2024-04-24T22:59:05.932147+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6836/2644625343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2644625343},{"type":"v1","addr":"172.21.15.121:6836","nonce":2644625343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4276]},"id":1}]} 2024-04-24T22:59:47.063 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 6 2024-04-24T22:59:47.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.063+0000 7f89056d2640 1 -- 172.21.15.32:0/1748492399 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f88ec06be30 msgr2=0x7f88ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:47.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.063+0000 7f89056d2640 1 --2- 172.21.15.32:0/1748492399 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f88ec06be30 0x7f88ec06e2f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f88f4009ad0 tx=0x7f88f4008040 comp rx=0 tx=0).stop 2024-04-24T22:59:47.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.063+0000 7f89056d2640 1 -- 172.21.15.32:0/1748492399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89000b6050 msgr2=0x7f89000aeb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:47.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.063+0000 7f89056d2640 1 --2- 172.21.15.32:0/1748492399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89000b6050 0x7f89000aeb20 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f88f000c460 tx=0x7f88f000c920 comp rx=0 tx=0).stop 2024-04-24T22:59:47.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.063+0000 7f89056d2640 1 -- 172.21.15.32:0/1748492399 shutdown_connections 2024-04-24T22:59:47.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.063+0000 7f89056d2640 1 --2- 172.21.15.32:0/1748492399 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f88ec06be30 0x7f88ec06e2f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:47.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.063+0000 7f89056d2640 1 --2- 172.21.15.32:0/1748492399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89000b6050 0x7f89000aeb20 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:47.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.063+0000 7f89056d2640 1 --2- 172.21.15.32:0/1748492399 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89000cd3c0 0x7f8900070630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:47.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.063+0000 7f89056d2640 1 --2- 172.21.15.32:0/1748492399 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f89000b5560 0x7f8900072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:47.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.063+0000 7f89056d2640 1 -- 172.21.15.32:0/1748492399 >> 172.21.15.32:0/1748492399 conn(0x7f890006b110 msgr2=0x7f89000aff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:47.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.063+0000 7f89056d2640 1 -- 172.21.15.32:0/1748492399 shutdown_connections 2024-04-24T22:59:47.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.063+0000 7f89056d2640 1 -- 172.21.15.32:0/1748492399 wait complete. 2024-04-24T22:59:47.076 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T22:59:47.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.175+0000 7f00c7683640 1 Processor -- start 2024-04-24T22:59:47.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.175+0000 7f00c7683640 1 -- start start 2024-04-24T22:59:47.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.176+0000 7f00c7683640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f00c006b820 0x7f00c00d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:47.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.176+0000 7f00c7683640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f00c00d4210 0x7f00c00b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:47.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.176+0000 7f00c7683640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00c00d3170 con 0x7f00c015e9b0 2024-04-24T22:59:47.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.176+0000 7f00c7683640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f00c00d3450 con 0x7f00c00d4210 2024-04-24T22:59:47.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.176+0000 7f00c7683640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f00c00d3730 con 0x7f00c006b820 2024-04-24T22:59:47.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.176+0000 7f00c5c20640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f00c00d4210 0x7f00c00b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:47.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.176+0000 7f00c541f640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f00c015e9b0 0x7f00c006b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52126/0 (socket says 172.21.15.32:52126) 2024-04-24T22:59:47.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.176+0000 7f00c541f640 1 -- 172.21.15.32:0/1171607057 learned_addr learned my addr 172.21.15.32:0/1171607057 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:47.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.176+0000 7f00c4c1e640 1 --2- 172.21.15.32:0/1171607057 >> v2:172.21.15.121:3300/0 conn(0x7f00c006b820 0x7f00c00d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:47.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.176+0000 7f00c5c20640 1 -- 172.21.15.32:0/1171607057 >> v2:172.21.15.121:3300/0 conn(0x7f00c006b820 msgr2=0x7f00c00d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:47.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.176+0000 7f00c5c20640 1 --2- 172.21.15.32:0/1171607057 >> v2:172.21.15.121:3300/0 conn(0x7f00c006b820 0x7f00c00d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:47.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.176+0000 7f00c5c20640 1 -- 172.21.15.32:0/1171607057 >> v1:172.21.15.32:6789/0 conn(0x7f00c015e9b0 legacy=0x7f00c006b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:47.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.176+0000 7f00b7fff640 1 -- 172.21.15.32:0/1171607057 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1233202898 0 0) 0x7f00c00d3170 con 0x7f00c015e9b0 2024-04-24T22:59:47.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.176+0000 7f00c5c20640 1 -- 172.21.15.32:0/1171607057 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00c00d3a10 con 0x7f00c00d4210 2024-04-24T22:59:47.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.177+0000 7f00c5c20640 1 --2- 172.21.15.32:0/1171607057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00c00d4210 0x7f00c00b78f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f00c00730f0 tx=0x7f00b802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b735cc427d15dbd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:47.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.177+0000 7f00b7fff640 1 -- 172.21.15.32:0/1171607057 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b8007740 con 0x7f00c00d4210 2024-04-24T22:59:47.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.177+0000 7f00b7fff640 1 -- 172.21.15.32:0/1171607057 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00b80078e0 con 0x7f00c00d4210 2024-04-24T22:59:47.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.177+0000 7f00b7fff640 1 -- 172.21.15.32:0/1171607057 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b8007cb0 con 0x7f00c00d4210 2024-04-24T22:59:47.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.178+0000 7f00c7683640 1 -- 172.21.15.32:0/1171607057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00c00d4210 msgr2=0x7f00c00b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:47.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.178+0000 7f00c7683640 1 --2- 172.21.15.32:0/1171607057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00c00d4210 0x7f00c00b78f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f00c00730f0 tx=0x7f00b802a5f0 comp rx=0 tx=0).stop 2024-04-24T22:59:47.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.178+0000 7f00c7683640 1 -- 172.21.15.32:0/1171607057 shutdown_connections 2024-04-24T22:59:47.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.178+0000 7f00c7683640 1 --2- 172.21.15.32:0/1171607057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00c00d4210 0x7f00c00b78f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:47.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.178+0000 7f00c7683640 1 --2- 172.21.15.32:0/1171607057 >> v2:172.21.15.121:3300/0 conn(0x7f00c006b820 0x7f00c00d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:47.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.178+0000 7f00c7683640 1 -- 172.21.15.32:0/1171607057 >> 172.21.15.32:0/1171607057 conn(0x7f00c00d11d0 msgr2=0x7f00c00d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:47.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.178+0000 7f00c7683640 1 -- 172.21.15.32:0/1171607057 shutdown_connections 2024-04-24T22:59:47.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.178+0000 7f00c7683640 1 -- 172.21.15.32:0/1171607057 wait complete. 2024-04-24T22:59:47.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.179+0000 7f00c7683640 1 Processor -- start 2024-04-24T22:59:47.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.179+0000 7f00c7683640 1 -- start start 2024-04-24T22:59:47.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.180+0000 7f00c7683640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00c006b820 0x7f00c00ca220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:47.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.180+0000 7f00c7683640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f00c00d4210 0x7f00c015cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:47.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.180+0000 7f00c7683640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f00c015e9b0 0x7f00c015d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:47.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.180+0000 7f00c7683640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f00c00ba0e0 con 0x7f00c006b820 2024-04-24T22:59:47.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.180+0000 7f00c7683640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f00c00d2400 con 0x7f00c00d4210 2024-04-24T22:59:47.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.180+0000 7f00c7683640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f00c00ba260 con 0x7f00c015e9b0 2024-04-24T22:59:47.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.180+0000 7f00c541f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00c006b820 0x7f00c00ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:47.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.180+0000 7f00c541f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00c006b820 0x7f00c00ca220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57704/0 (socket says 172.21.15.32:57704) 2024-04-24T22:59:47.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.180+0000 7f00c541f640 1 -- 172.21.15.32:0/74849120 learned_addr learned my addr 172.21.15.32:0/74849120 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:47.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.180+0000 7f00c4c1e640 1 --2- 172.21.15.32:0/74849120 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f00c00d4210 0x7f00c015cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:47.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.180+0000 7f00c5c20640 1 --2- 172.21.15.32:0/74849120 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f00c015e9b0 0x7f00c015d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:47.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.180+0000 7f00c541f640 1 -- 172.21.15.32:0/74849120 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f00c015e9b0 msgr2=0x7f00c015d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:47.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.180+0000 7f00c541f640 1 --2- 172.21.15.32:0/74849120 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f00c015e9b0 0x7f00c015d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:47.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.180+0000 7f00c541f640 1 -- 172.21.15.32:0/74849120 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f00c00d4210 msgr2=0x7f00c015cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:47.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.180+0000 7f00c541f640 1 --2- 172.21.15.32:0/74849120 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f00c00d4210 0x7f00c015cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:47.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.180+0000 7f00c541f640 1 -- 172.21.15.32:0/74849120 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00c014b640 con 0x7f00c006b820 2024-04-24T22:59:47.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.180+0000 7f00c5c20640 1 --2- 172.21.15.32:0/74849120 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f00c015e9b0 0x7f00c015d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:47.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.180+0000 7f00c541f640 1 --2- 172.21.15.32:0/74849120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00c006b820 0x7f00c00ca220 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f00bc00c430 tx=0x7f00bc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:47.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.181+0000 7f00b5ffb640 1 -- 172.21.15.32:0/74849120 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00bc0074e0 con 0x7f00c006b820 2024-04-24T22:59:47.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.181+0000 7f00b5ffb640 1 -- 172.21.15.32:0/74849120 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00bc0047e0 con 0x7f00c006b820 2024-04-24T22:59:47.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.181+0000 7f00b5ffb640 1 -- 172.21.15.32:0/74849120 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00bc004bd0 con 0x7f00c006b820 2024-04-24T22:59:47.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.181+0000 7f00c7683640 1 -- 172.21.15.32:0/74849120 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00c0159250 con 0x7f00c006b820 2024-04-24T22:59:47.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.181+0000 7f00c7683640 1 -- 172.21.15.32:0/74849120 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00c01579f0 con 0x7f00c006b820 2024-04-24T22:59:47.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.182+0000 7f00c7683640 1 -- 172.21.15.32:0/74849120 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00c0157500 con 0x7f00c006b820 2024-04-24T22:59:47.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.182+0000 7f00b5ffb640 1 -- 172.21.15.32:0/74849120 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f00bc011070 con 0x7f00c006b820 2024-04-24T22:59:47.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.182+0000 7f00b5ffb640 1 --2- 172.21.15.32:0/74849120 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f00a406be30 0x7f00a406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:47.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.183+0000 7f00b5ffb640 1 -- 172.21.15.32:0/74849120 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f00bc08a9f0 con 0x7f00c006b820 2024-04-24T22:59:47.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.183+0000 7f00c4c1e640 1 --2- 172.21.15.32:0/74849120 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f00a406be30 0x7f00a406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:47.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.183+0000 7f00c4c1e640 1 --2- 172.21.15.32:0/74849120 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f00a406be30 0x7f00a406e2f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f00c01532e0 tx=0x7f00b0009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:47.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.187+0000 7f00b5ffb640 1 -- 172.21.15.32:0/74849120 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f00bc0593a0 con 0x7f00c006b820 2024-04-24T22:59:47.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.407+0000 7f00c7683640 1 -- 172.21.15.32:0/74849120 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f00c015ddc0 con 0x7f00c006b820 2024-04-24T22:59:47.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.407+0000 7f00b5ffb640 1 -- 172.21.15.32:0/74849120 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5178 (secure 0 0 0) 0x7f00bc058ca0 con 0x7f00c006b820 2024-04-24T22:59:47.409 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:59:47.409 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1485884060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1485884060},{"type":"v1","addr":"172.21.15.121:6837","nonce":1485884060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/923934104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":923934104},{"type":"v1","addr":"172.21.15.32:6833","nonce":923934104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3229115835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3229115835},{"type":"v1","addr":"172.21.15.32:6835","nonce":3229115835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T22:59:03.876305+0000","modified":"2024-04-24T22:59:05.932147+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6836/2644625343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2644625343},{"type":"v1","addr":"172.21.15.121:6836","nonce":2644625343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4276]},"id":1}]} 2024-04-24T22:59:47.409 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 6 2024-04-24T22:59:47.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.409+0000 7f00c7683640 1 -- 172.21.15.32:0/74849120 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f00a406be30 msgr2=0x7f00a406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:47.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.409+0000 7f00c7683640 1 --2- 172.21.15.32:0/74849120 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f00a406be30 0x7f00a406e2f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f00c01532e0 tx=0x7f00b0009190 comp rx=0 tx=0).stop 2024-04-24T22:59:47.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.409+0000 7f00c7683640 1 -- 172.21.15.32:0/74849120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00c006b820 msgr2=0x7f00c00ca220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:47.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.409+0000 7f00c7683640 1 --2- 172.21.15.32:0/74849120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00c006b820 0x7f00c00ca220 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f00bc00c430 tx=0x7f00bc00c8f0 comp rx=0 tx=0).stop 2024-04-24T22:59:47.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.409+0000 7f00c7683640 1 -- 172.21.15.32:0/74849120 shutdown_connections 2024-04-24T22:59:47.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.409+0000 7f00c7683640 1 --2- 172.21.15.32:0/74849120 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f00a406be30 0x7f00a406e2f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:47.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.409+0000 7f00c7683640 1 --2- 172.21.15.32:0/74849120 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f00c015e9b0 0x7f00c015d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:47.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.409+0000 7f00c7683640 1 --2- 172.21.15.32:0/74849120 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f00c00d4210 0x7f00c015cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:47.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.409+0000 7f00c7683640 1 --2- 172.21.15.32:0/74849120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00c006b820 0x7f00c00ca220 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:47.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.409+0000 7f00c7683640 1 -- 172.21.15.32:0/74849120 >> 172.21.15.32:0/74849120 conn(0x7f00c00d11d0 msgr2=0x7f00c006b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:47.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.409+0000 7f00c7683640 1 -- 172.21.15.32:0/74849120 shutdown_connections 2024-04-24T22:59:47.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.409+0000 7f00c7683640 1 -- 172.21.15.32:0/74849120 wait complete. 2024-04-24T22:59:47.421 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:59:47.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.520+0000 7fd319fd5640 1 Processor -- start 2024-04-24T22:59:47.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.521+0000 7fd319fd5640 1 -- start start 2024-04-24T22:59:47.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.521+0000 7fd319fd5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd31406b820 0x7fd3140d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:47.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.521+0000 7fd319fd5640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3140d3170 con 0x7fd3140d4210 2024-04-24T22:59:47.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.521+0000 7fd319fd5640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3140d3450 con 0x7fd31415e9b0 2024-04-24T22:59:47.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.521+0000 7fd319fd5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fd3140d3730 con 0x7fd31406b820 2024-04-24T22:59:47.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.521+0000 7fd312ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd31406b820 0x7fd3140d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:47.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.522+0000 7fd312ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd31406b820 0x7fd3140d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57706/0 (socket says 172.21.15.32:57706) 2024-04-24T22:59:47.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.522+0000 7fd313fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd3140d4210 0x7fd3140b78f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52128/0 (socket says 172.21.15.32:52128) 2024-04-24T22:59:47.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.522+0000 7fd312ffd640 1 -- 172.21.15.32:0/1235463351 learned_addr learned my addr 172.21.15.32:0/1235463351 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:47.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.522+0000 7fd312ffd640 1 -- 172.21.15.32:0/1235463351 >> v1:172.21.15.121:6789/0 conn(0x7fd31415e9b0 legacy=0x7fd31406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:47.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.522+0000 7fd312ffd640 1 -- 172.21.15.32:0/1235463351 >> v1:172.21.15.32:6789/0 conn(0x7fd3140d4210 legacy=0x7fd3140b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:47.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.522+0000 7fd312ffd640 1 -- 172.21.15.32:0/1235463351 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3140d3a10 con 0x7fd31406b820 2024-04-24T22:59:47.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.523+0000 7fd312ffd640 1 --2- 172.21.15.32:0/1235463351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd31406b820 0x7fd3140d2b90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd308009870 tx=0x7fd30802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e7507ef10e8c7f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:47.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.523+0000 7fd3127fc640 1 -- 172.21.15.32:0/1235463351 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd308037070 con 0x7fd31406b820 2024-04-24T22:59:47.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.523+0000 7fd3127fc640 1 -- 172.21.15.32:0/1235463351 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd308032440 con 0x7fd31406b820 2024-04-24T22:59:47.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.523+0000 7fd3127fc640 1 -- 172.21.15.32:0/1235463351 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd308032810 con 0x7fd31406b820 2024-04-24T22:59:47.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.523+0000 7fd319fd5640 1 -- 172.21.15.32:0/1235463351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd31406b820 msgr2=0x7fd3140d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:47.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.523+0000 7fd319fd5640 1 --2- 172.21.15.32:0/1235463351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd31406b820 0x7fd3140d2b90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd308009870 tx=0x7fd30802a270 comp rx=0 tx=0).stop 2024-04-24T22:59:47.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.523+0000 7fd319fd5640 1 -- 172.21.15.32:0/1235463351 shutdown_connections 2024-04-24T22:59:47.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.523+0000 7fd319fd5640 1 --2- 172.21.15.32:0/1235463351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd31406b820 0x7fd3140d2b90 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:47.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.523+0000 7fd319fd5640 1 -- 172.21.15.32:0/1235463351 >> 172.21.15.32:0/1235463351 conn(0x7fd3140d11d0 msgr2=0x7fd3140d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:47.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.523+0000 7fd319fd5640 1 -- 172.21.15.32:0/1235463351 shutdown_connections 2024-04-24T22:59:47.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.523+0000 7fd319fd5640 1 -- 172.21.15.32:0/1235463351 wait complete. 2024-04-24T22:59:47.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.524+0000 7fd319fd5640 1 Processor -- start 2024-04-24T22:59:47.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.525+0000 7fd319fd5640 1 -- start start 2024-04-24T22:59:47.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.525+0000 7fd319fd5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd31406b820 0x7fd3140c78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:47.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.525+0000 7fd319fd5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3140d4210 0x7fd3140b8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:47.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.525+0000 7fd319fd5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd31415e9b0 0x7fd3140b92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:47.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.525+0000 7fd319fd5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd3140ba0e0 con 0x7fd3140d4210 2024-04-24T22:59:47.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.525+0000 7fd319fd5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd3140d2400 con 0x7fd31406b820 2024-04-24T22:59:47.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.525+0000 7fd319fd5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd3140ba260 con 0x7fd31415e9b0 2024-04-24T22:59:47.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.526+0000 7fd312ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3140d4210 0x7fd3140b8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:47.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.526+0000 7fd312ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3140d4210 0x7fd3140b8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57720/0 (socket says 172.21.15.32:57720) 2024-04-24T22:59:47.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.526+0000 7fd312ffd640 1 -- 172.21.15.32:0/2369261109 learned_addr learned my addr 172.21.15.32:0/2369261109 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:47.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.526+0000 7fd313fff640 1 --2- 172.21.15.32:0/2369261109 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd31415e9b0 0x7fd3140b92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:47.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.526+0000 7fd3137fe640 1 --2- 172.21.15.32:0/2369261109 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd31406b820 0x7fd3140c78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:47.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.526+0000 7fd312ffd640 1 -- 172.21.15.32:0/2369261109 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd31415e9b0 msgr2=0x7fd3140b92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:47.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.526+0000 7fd312ffd640 1 --2- 172.21.15.32:0/2369261109 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd31415e9b0 0x7fd3140b92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:47.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.526+0000 7fd312ffd640 1 -- 172.21.15.32:0/2369261109 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd31406b820 msgr2=0x7fd3140c78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:47.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.526+0000 7fd312ffd640 1 --2- 172.21.15.32:0/2369261109 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd31406b820 0x7fd3140c78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:47.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.526+0000 7fd312ffd640 1 -- 172.21.15.32:0/2369261109 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3140c8b10 con 0x7fd3140d4210 2024-04-24T22:59:47.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.526+0000 7fd312ffd640 1 --2- 172.21.15.32:0/2369261109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3140d4210 0x7fd3140b8d60 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd308000c00 tx=0x7fd3080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:47.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.526+0000 7fd2fbfff640 1 -- 172.21.15.32:0/2369261109 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd308037040 con 0x7fd3140d4210 2024-04-24T22:59:47.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.526+0000 7fd2fbfff640 1 -- 172.21.15.32:0/2369261109 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd308042070 con 0x7fd3140d4210 2024-04-24T22:59:47.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.526+0000 7fd319fd5640 1 -- 172.21.15.32:0/2369261109 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3140c8df0 con 0x7fd3140d4210 2024-04-24T22:59:47.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.526+0000 7fd2fbfff640 1 -- 172.21.15.32:0/2369261109 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd308030bd0 con 0x7fd3140d4210 2024-04-24T22:59:47.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.526+0000 7fd319fd5640 1 -- 172.21.15.32:0/2369261109 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3140c93b0 con 0x7fd3140d4210 2024-04-24T22:59:47.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.527+0000 7fd319fd5640 1 -- 172.21.15.32:0/2369261109 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd314068430 con 0x7fd3140d4210 2024-04-24T22:59:47.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.528+0000 7fd2fbfff640 1 -- 172.21.15.32:0/2369261109 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd308047460 con 0x7fd3140d4210 2024-04-24T22:59:47.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.528+0000 7fd2fbfff640 1 --2- 172.21.15.32:0/2369261109 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd30006be30 0x7fd30006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:47.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.528+0000 7fd2fbfff640 1 -- 172.21.15.32:0/2369261109 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd3080abb00 con 0x7fd3140d4210 2024-04-24T22:59:47.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.528+0000 7fd3137fe640 1 --2- 172.21.15.32:0/2369261109 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd30006be30 0x7fd30006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:47.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.529+0000 7fd3137fe640 1 --2- 172.21.15.32:0/2369261109 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd30006be30 0x7fd30006e2f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd304007fd0 tx=0x7fd304009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:47.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.532+0000 7fd2fbfff640 1 -- 172.21.15.32:0/2369261109 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd308045360 con 0x7fd3140d4210 2024-04-24T22:59:47.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.716+0000 7fd319fd5640 1 -- 172.21.15.32:0/2369261109 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd31406b290 con 0x7fd3140d4210 2024-04-24T22:59:47.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.717+0000 7fd2fbfff640 1 -- 172.21.15.32:0/2369261109 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16621 (secure 0 0 0) 0x7fd30807a430 con 0x7fd3140d4210 2024-04-24T22:59:47.719 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:59:47.719 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T22:59:05.517554+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:59:02.016639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T22:59:02.846087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:59:47.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.719+0000 7fd319fd5640 1 -- 172.21.15.32:0/2369261109 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd30006be30 msgr2=0x7fd30006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:47.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.719+0000 7fd319fd5640 1 --2- 172.21.15.32:0/2369261109 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd30006be30 0x7fd30006e2f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd304007fd0 tx=0x7fd304009040 comp rx=0 tx=0).stop 2024-04-24T22:59:47.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.719+0000 7fd319fd5640 1 -- 172.21.15.32:0/2369261109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3140d4210 msgr2=0x7fd3140b8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:47.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.719+0000 7fd319fd5640 1 --2- 172.21.15.32:0/2369261109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3140d4210 0x7fd3140b8d60 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd308000c00 tx=0x7fd3080046c0 comp rx=0 tx=0).stop 2024-04-24T22:59:47.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.719+0000 7fd319fd5640 1 -- 172.21.15.32:0/2369261109 shutdown_connections 2024-04-24T22:59:47.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.719+0000 7fd319fd5640 1 --2- 172.21.15.32:0/2369261109 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd30006be30 0x7fd30006e2f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:47.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.719+0000 7fd319fd5640 1 --2- 172.21.15.32:0/2369261109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3140d4210 0x7fd3140b8d60 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:47.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.719+0000 7fd319fd5640 1 --2- 172.21.15.32:0/2369261109 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd31415e9b0 0x7fd3140b92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:47.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.719+0000 7fd319fd5640 1 --2- 172.21.15.32:0/2369261109 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd31406b820 0x7fd3140c78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:47.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.719+0000 7fd319fd5640 1 -- 172.21.15.32:0/2369261109 >> 172.21.15.32:0/2369261109 conn(0x7fd3140d11d0 msgr2=0x7fd3140d15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:47.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.719+0000 7fd319fd5640 1 -- 172.21.15.32:0/2369261109 shutdown_connections 2024-04-24T22:59:47.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.719+0000 7fd319fd5640 1 -- 172.21.15.32:0/2369261109 wait complete. 2024-04-24T22:59:47.732 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-24T22:59:47.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.832+0000 7f60b8bb7640 1 Processor -- start 2024-04-24T22:59:47.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.832+0000 7f60b8bb7640 1 -- start start 2024-04-24T22:59:47.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.833+0000 7f60b8bb7640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f60b415e9b0 0x7f60b40d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:47.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.833+0000 7f60b8bb7640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f60b40d4210 0x7f60b406b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:47.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.833+0000 7f60b8bb7640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60b40d3170 con 0x7f60b40ba0e0 2024-04-24T22:59:47.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.833+0000 7f60b8bb7640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f60b40b8900 con 0x7f60b40d4210 2024-04-24T22:59:47.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.833+0000 7f60b8bb7640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f60b40d3450 con 0x7f60b415e9b0 2024-04-24T22:59:47.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.834+0000 7f60b1d9b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f60b40d4210 0x7f60b406b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:47.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.834+0000 7f60b1d9b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f60b40d4210 0x7f60b406b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57724/0 (socket says 172.21.15.32:57724) 2024-04-24T22:59:47.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.834+0000 7f60b1d9b640 1 -- 172.21.15.32:0/987104126 learned_addr learned my addr 172.21.15.32:0/987104126 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:47.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.834+0000 7f60b259c640 1 --2- 172.21.15.32:0/987104126 >> v2:172.21.15.121:3300/0 conn(0x7f60b415e9b0 0x7f60b40d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:47.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.834+0000 7f60b1d9b640 1 -- 172.21.15.32:0/987104126 >> v2:172.21.15.121:3300/0 conn(0x7f60b415e9b0 msgr2=0x7f60b40d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:47.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.834+0000 7f60b1d9b640 1 --2- 172.21.15.32:0/987104126 >> v2:172.21.15.121:3300/0 conn(0x7f60b415e9b0 0x7f60b40d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:47.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.834+0000 7f60b1d9b640 1 -- 172.21.15.32:0/987104126 >> v1:172.21.15.121:6789/0 conn(0x7f60b40ba0e0 legacy=0x7f60b4071730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:47.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.834+0000 7f60b1d9b640 1 -- 172.21.15.32:0/987104126 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60b40d3730 con 0x7f60b40d4210 2024-04-24T22:59:47.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.835+0000 7f60b1d9b640 1 --2- 172.21.15.32:0/987104126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60b40d4210 0x7f60b406b110 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f60b406be20 tx=0x7f60a802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3104cf082155a5e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:47.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.835+0000 7f60b159a640 1 -- 172.21.15.32:0/987104126 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a8037070 con 0x7f60b40d4210 2024-04-24T22:59:47.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.835+0000 7f60b159a640 1 -- 172.21.15.32:0/987104126 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60a8032440 con 0x7f60b40d4210 2024-04-24T22:59:47.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.835+0000 7f60b159a640 1 -- 172.21.15.32:0/987104126 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a8032810 con 0x7f60b40d4210 2024-04-24T22:59:47.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.835+0000 7f60b8bb7640 1 -- 172.21.15.32:0/987104126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60b40d4210 msgr2=0x7f60b406b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:47.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.835+0000 7f60b8bb7640 1 --2- 172.21.15.32:0/987104126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60b40d4210 0x7f60b406b110 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f60b406be20 tx=0x7f60a802a1a0 comp rx=0 tx=0).stop 2024-04-24T22:59:47.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.835+0000 7f60b8bb7640 1 -- 172.21.15.32:0/987104126 shutdown_connections 2024-04-24T22:59:47.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.835+0000 7f60b8bb7640 1 --2- 172.21.15.32:0/987104126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60b40d4210 0x7f60b406b110 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:47.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.835+0000 7f60b8bb7640 1 --2- 172.21.15.32:0/987104126 >> v2:172.21.15.121:3300/0 conn(0x7f60b415e9b0 0x7f60b40d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:47.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.835+0000 7f60b8bb7640 1 -- 172.21.15.32:0/987104126 >> 172.21.15.32:0/987104126 conn(0x7f60b40d11d0 msgr2=0x7f60b40d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:47.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.835+0000 7f60b8bb7640 1 -- 172.21.15.32:0/987104126 shutdown_connections 2024-04-24T22:59:47.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.835+0000 7f60b8bb7640 1 -- 172.21.15.32:0/987104126 wait complete. 2024-04-24T22:59:47.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.837+0000 7f60b8bb7640 1 Processor -- start 2024-04-24T22:59:47.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.837+0000 7f60b8bb7640 1 -- start start 2024-04-24T22:59:47.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.838+0000 7f60b8bb7640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f60b40ba0e0 0x7f60b40719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:47.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.838+0000 7f60b8bb7640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f60b40d4210 0x7f60b414c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:47.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.838+0000 7f60b8bb7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60b415e9b0 0x7f60b414cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:47.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.838+0000 7f60b8bb7640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f60b40d2400 con 0x7f60b415e9b0 2024-04-24T22:59:47.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.838+0000 7f60b8bb7640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f60b40d3cf0 con 0x7f60b40d4210 2024-04-24T22:59:47.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.838+0000 7f60b8bb7640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f60b406b650 con 0x7f60b40ba0e0 2024-04-24T22:59:47.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.838+0000 7f60b2d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60b415e9b0 0x7f60b414cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:47.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.838+0000 7f60b2d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60b415e9b0 0x7f60b414cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57740/0 (socket says 172.21.15.32:57740) 2024-04-24T22:59:47.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.838+0000 7f60b2d9d640 1 -- 172.21.15.32:0/4030911475 learned_addr learned my addr 172.21.15.32:0/4030911475 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:47.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.838+0000 7f60b1d9b640 1 --2- 172.21.15.32:0/4030911475 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f60b40d4210 0x7f60b414c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:47.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.838+0000 7f60b259c640 1 --2- 172.21.15.32:0/4030911475 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f60b40ba0e0 0x7f60b40719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:47.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.838+0000 7f60b2d9d640 1 -- 172.21.15.32:0/4030911475 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f60b40ba0e0 msgr2=0x7f60b40719a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:47.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.838+0000 7f60b2d9d640 1 --2- 172.21.15.32:0/4030911475 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f60b40ba0e0 0x7f60b40719a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:47.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.838+0000 7f60b2d9d640 1 -- 172.21.15.32:0/4030911475 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f60b40d4210 msgr2=0x7f60b414c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:47.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.838+0000 7f60b2d9d640 1 --2- 172.21.15.32:0/4030911475 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f60b40d4210 0x7f60b414c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:47.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.838+0000 7f60b2d9d640 1 -- 172.21.15.32:0/4030911475 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60b4161280 con 0x7f60b415e9b0 2024-04-24T22:59:47.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.838+0000 7f60b2d9d640 1 --2- 172.21.15.32:0/4030911475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60b415e9b0 0x7f60b414cc80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f60ac00c7b0 tx=0x7f60ac00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:47.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.838+0000 7f60a2ffd640 1 -- 172.21.15.32:0/4030911475 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ac004b60 con 0x7f60b415e9b0 2024-04-24T22:59:47.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.838+0000 7f60a2ffd640 1 -- 172.21.15.32:0/4030911475 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60ac004d00 con 0x7f60b415e9b0 2024-04-24T22:59:47.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.838+0000 7f60a2ffd640 1 -- 172.21.15.32:0/4030911475 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ac00aa90 con 0x7f60b415e9b0 2024-04-24T22:59:47.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.838+0000 7f60b8bb7640 1 -- 172.21.15.32:0/4030911475 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60b4161520 con 0x7f60b415e9b0 2024-04-24T22:59:47.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.839+0000 7f60b8bb7640 1 -- 172.21.15.32:0/4030911475 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60b4161ae0 con 0x7f60b415e9b0 2024-04-24T22:59:47.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.839+0000 7f60b8bb7640 1 -- 172.21.15.32:0/4030911475 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60b4068430 con 0x7f60b415e9b0 2024-04-24T22:59:47.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.840+0000 7f60a2ffd640 1 -- 172.21.15.32:0/4030911475 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f60ac00ac30 con 0x7f60b415e9b0 2024-04-24T22:59:47.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.840+0000 7f60a2ffd640 1 --2- 172.21.15.32:0/4030911475 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f609406be30 0x7f609406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:47.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.841+0000 7f60a2ffd640 1 -- 172.21.15.32:0/4030911475 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f60ac08af20 con 0x7f60b415e9b0 2024-04-24T22:59:47.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.841+0000 7f60b259c640 1 --2- 172.21.15.32:0/4030911475 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f609406be30 0x7f609406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:47.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.841+0000 7f60b259c640 1 --2- 172.21.15.32:0/4030911475 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f609406be30 0x7f609406e2f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f60a40079a0 tx=0x7f60a4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:47.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:47.845+0000 7f60a2ffd640 1 -- 172.21.15.32:0/4030911475 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f60ac020070 con 0x7f60b415e9b0 2024-04-24T22:59:48.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.067+0000 7f60b8bb7640 1 -- 172.21.15.32:0/4030911475 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f60b4161f70 con 0x7f60b415e9b0 2024-04-24T22:59:48.070 INFO:tasks.ceph.mon.a.smithi032.stderr:2024-04-24T22:59:48.068+0000 7f3b809c6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T22:59:48.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.069+0000 7f60a2ffd640 1 -- 172.21.15.32:0/4030911475 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v7) ==== 161+0+0 (secure 0 0 0) 0x7f60ac059310 con 0x7f60b415e9b0 2024-04-24T22:59:48.071 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-24T22:59:48.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.072+0000 7f60b8bb7640 1 -- 172.21.15.32:0/4030911475 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f609406be30 msgr2=0x7f609406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:48.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.072+0000 7f60b8bb7640 1 --2- 172.21.15.32:0/4030911475 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f609406be30 0x7f609406e2f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f60a40079a0 tx=0x7f60a4008040 comp rx=0 tx=0).stop 2024-04-24T22:59:48.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.072+0000 7f60b8bb7640 1 -- 172.21.15.32:0/4030911475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60b415e9b0 msgr2=0x7f60b414cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:48.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.072+0000 7f60b8bb7640 1 --2- 172.21.15.32:0/4030911475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60b415e9b0 0x7f60b414cc80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f60ac00c7b0 tx=0x7f60ac00cc70 comp rx=0 tx=0).stop 2024-04-24T22:59:48.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.073+0000 7f60b8bb7640 1 -- 172.21.15.32:0/4030911475 shutdown_connections 2024-04-24T22:59:48.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.073+0000 7f60b8bb7640 1 --2- 172.21.15.32:0/4030911475 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f609406be30 0x7f609406e2f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:48.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.073+0000 7f60b8bb7640 1 --2- 172.21.15.32:0/4030911475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60b415e9b0 0x7f60b414cc80 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:48.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.073+0000 7f60b8bb7640 1 --2- 172.21.15.32:0/4030911475 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f60b40d4210 0x7f60b414c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:48.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.073+0000 7f60b8bb7640 1 --2- 172.21.15.32:0/4030911475 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f60b40ba0e0 0x7f60b40719a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:48.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.073+0000 7f60b8bb7640 1 -- 172.21.15.32:0/4030911475 >> 172.21.15.32:0/4030911475 conn(0x7f60b40d11d0 msgr2=0x7f60b415be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:48.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.073+0000 7f60b8bb7640 1 -- 172.21.15.32:0/4030911475 shutdown_connections 2024-04-24T22:59:48.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.073+0000 7f60b8bb7640 1 -- 172.21.15.32:0/4030911475 wait complete. 2024-04-24T22:59:48.080 INFO:tasks.ceph.mds.d.smithi121.stderr: -7533> 2024-04-24T22:59:08.492+0000 7f349f396640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T22:59:48.085 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-24T22:59:48.113 INFO:tasks.ceph.mds.d.smithi121.stdout:starting mds.d at 2024-04-24T22:59:48.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.187+0000 7f2f6b3b7640 1 Processor -- start 2024-04-24T22:59:48.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.187+0000 7f2f6b3b7640 1 -- start start 2024-04-24T22:59:48.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.188+0000 7f2f6b3b7640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2f64162320 0x7f2f640ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:48.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.188+0000 7f2f6b3b7640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2f640b5af0 0x7f2f64162700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:48.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.188+0000 7f2f6b3b7640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f640cd270 con 0x7f2f64162c40 2024-04-24T22:59:48.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.188+0000 7f2f6b3b7640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f2f640b5660 con 0x7f2f640b5af0 2024-04-24T22:59:48.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.188+0000 7f2f6b3b7640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f2f64071610 con 0x7f2f64162320 2024-04-24T22:59:48.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.188+0000 7f2f68952640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2f640b5af0 0x7f2f64162700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:48.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.188+0000 7f2f68952640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2f640b5af0 0x7f2f64162700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57746/0 (socket says 172.21.15.32:57746) 2024-04-24T22:59:48.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.188+0000 7f2f68952640 1 -- 172.21.15.32:0/831911757 learned_addr learned my addr 172.21.15.32:0/831911757 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:48.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.188+0000 7f2f69153640 1 --2- 172.21.15.32:0/831911757 >> v2:172.21.15.121:3300/0 conn(0x7f2f64162320 0x7f2f640ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:48.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.189+0000 7f2f68952640 1 -- 172.21.15.32:0/831911757 >> v2:172.21.15.121:3300/0 conn(0x7f2f64162320 msgr2=0x7f2f640ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:48.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.189+0000 7f2f68952640 1 --2- 172.21.15.32:0/831911757 >> v2:172.21.15.121:3300/0 conn(0x7f2f64162320 0x7f2f640ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:48.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.189+0000 7f2f68952640 1 -- 172.21.15.32:0/831911757 >> v1:172.21.15.121:6789/0 conn(0x7f2f64162c40 legacy=0x7f2f640ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:48.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.189+0000 7f2f68952640 1 -- 172.21.15.32:0/831911757 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f640718f0 con 0x7f2f640b5af0 2024-04-24T22:59:48.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.189+0000 7f2f68952640 1 --2- 172.21.15.32:0/831911757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f640b5af0 0x7f2f64162700 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2f6406c5b0 tx=0x7f2f5802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1893de6e4591324e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:48.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.189+0000 7f2f63fff640 1 -- 172.21.15.32:0/831911757 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f58037070 con 0x7f2f640b5af0 2024-04-24T22:59:48.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.190+0000 7f2f63fff640 1 -- 172.21.15.32:0/831911757 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f58032440 con 0x7f2f640b5af0 2024-04-24T22:59:48.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.190+0000 7f2f63fff640 1 -- 172.21.15.32:0/831911757 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f58032810 con 0x7f2f640b5af0 2024-04-24T22:59:48.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.190+0000 7f2f6b3b7640 1 -- 172.21.15.32:0/831911757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f640b5af0 msgr2=0x7f2f64162700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:48.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.190+0000 7f2f6b3b7640 1 --2- 172.21.15.32:0/831911757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f640b5af0 0x7f2f64162700 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2f6406c5b0 tx=0x7f2f5802a1e0 comp rx=0 tx=0).stop 2024-04-24T22:59:48.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.190+0000 7f2f6b3b7640 1 -- 172.21.15.32:0/831911757 shutdown_connections 2024-04-24T22:59:48.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.190+0000 7f2f6b3b7640 1 --2- 172.21.15.32:0/831911757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f640b5af0 0x7f2f64162700 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:48.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.190+0000 7f2f6b3b7640 1 --2- 172.21.15.32:0/831911757 >> v2:172.21.15.121:3300/0 conn(0x7f2f64162320 0x7f2f640ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:48.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.190+0000 7f2f6b3b7640 1 -- 172.21.15.32:0/831911757 >> 172.21.15.32:0/831911757 conn(0x7f2f640d0ae0 msgr2=0x7f2f640d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:48.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.190+0000 7f2f6b3b7640 1 -- 172.21.15.32:0/831911757 shutdown_connections 2024-04-24T22:59:48.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.190+0000 7f2f6b3b7640 1 -- 172.21.15.32:0/831911757 wait complete. 2024-04-24T22:59:48.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.191+0000 7f2f6b3b7640 1 Processor -- start 2024-04-24T22:59:48.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.191+0000 7f2f6b3b7640 1 -- start start 2024-04-24T22:59:48.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.192+0000 7f2f6b3b7640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2f640b5af0 0x7f2f6414ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:48.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.192+0000 7f2f6b3b7640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2f64162320 0x7f2f640cef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:48.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.192+0000 7f2f6b3b7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f64162c40 0x7f2f64147970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:48.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.192+0000 7f2f6b3b7640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2f64163070 con 0x7f2f64162c40 2024-04-24T22:59:48.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.192+0000 7f2f6b3b7640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2f64067950 con 0x7f2f640b5af0 2024-04-24T22:59:48.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.192+0000 7f2f6b3b7640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f2f64071bd0 con 0x7f2f64162320 2024-04-24T22:59:48.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.192+0000 7f2f69954640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f64162c40 0x7f2f64147970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:48.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.192+0000 7f2f69954640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f64162c40 0x7f2f64147970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57754/0 (socket says 172.21.15.32:57754) 2024-04-24T22:59:48.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.192+0000 7f2f69954640 1 -- 172.21.15.32:0/31839869 learned_addr learned my addr 172.21.15.32:0/31839869 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:48.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.192+0000 7f2f68952640 1 --2- 172.21.15.32:0/31839869 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2f64162320 0x7f2f640cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:48.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.192+0000 7f2f69153640 1 --2- 172.21.15.32:0/31839869 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2f640b5af0 0x7f2f6414ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:48.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.192+0000 7f2f69954640 1 -- 172.21.15.32:0/31839869 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2f64162320 msgr2=0x7f2f640cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:48.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.192+0000 7f2f69954640 1 --2- 172.21.15.32:0/31839869 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2f64162320 0x7f2f640cef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:48.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.192+0000 7f2f69954640 1 -- 172.21.15.32:0/31839869 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2f640b5af0 msgr2=0x7f2f6414ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:48.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.192+0000 7f2f69954640 1 --2- 172.21.15.32:0/31839869 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2f640b5af0 0x7f2f6414ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:48.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.193+0000 7f2f69954640 1 -- 172.21.15.32:0/31839869 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f640cf4a0 con 0x7f2f64162c40 2024-04-24T22:59:48.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.193+0000 7f2f69954640 1 --2- 172.21.15.32:0/31839869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f64162c40 0x7f2f64147970 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2f5400c760 tx=0x7f2f5400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:48.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.193+0000 7f2f61ffb640 1 -- 172.21.15.32:0/31839869 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f54003b90 con 0x7f2f64162c40 2024-04-24T22:59:48.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.193+0000 7f2f61ffb640 1 -- 172.21.15.32:0/31839869 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f54003d30 con 0x7f2f64162c40 2024-04-24T22:59:48.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.193+0000 7f2f6b3b7640 1 -- 172.21.15.32:0/31839869 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f640cf770 con 0x7f2f64162c40 2024-04-24T22:59:48.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.193+0000 7f2f61ffb640 1 -- 172.21.15.32:0/31839869 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5400aa90 con 0x7f2f64162c40 2024-04-24T22:59:48.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.193+0000 7f2f6b3b7640 1 -- 172.21.15.32:0/31839869 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f640cfd30 con 0x7f2f64162c40 2024-04-24T22:59:48.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.194+0000 7f2f6b3b7640 1 -- 172.21.15.32:0/31839869 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f64068430 con 0x7f2f64162c40 2024-04-24T22:59:48.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.195+0000 7f2f61ffb640 1 -- 172.21.15.32:0/31839869 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2f5400ac30 con 0x7f2f64162c40 2024-04-24T22:59:48.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.195+0000 7f2f61ffb640 1 --2- 172.21.15.32:0/31839869 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2f5006be30 0x7f2f5006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:48.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.195+0000 7f2f61ffb640 1 -- 172.21.15.32:0/31839869 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5795+0+0 (secure 0 0 0) 0x7f2f5408ab00 con 0x7f2f64162c40 2024-04-24T22:59:48.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.195+0000 7f2f69153640 1 --2- 172.21.15.32:0/31839869 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2f5006be30 0x7f2f5006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:48.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.196+0000 7f2f69153640 1 --2- 172.21.15.32:0/31839869 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2f5006be30 0x7f2f5006e2f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c009b40 tx=0x7f2f5c009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:48.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.199+0000 7f2f61ffb640 1 -- 172.21.15.32:0/31839869 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2f54058e90 con 0x7f2f64162c40 2024-04-24T22:59:48.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:48.419+0000 7f2f6b3b7640 1 -- 172.21.15.32:0/31839869 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f2f64067730 con 0x7f2f64162c40 2024-04-24T22:59:49.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.071+0000 7f2f61ffb640 1 -- 172.21.15.32:0/31839869 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v8) ==== 104+0+0 (secure 0 0 0) 0x7f2f5405d190 con 0x7f2f64162c40 2024-04-24T22:59:49.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.073+0000 7f2f6b3b7640 1 -- 172.21.15.32:0/31839869 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2f5006be30 msgr2=0x7f2f5006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:49.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.073+0000 7f2f6b3b7640 1 --2- 172.21.15.32:0/31839869 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2f5006be30 0x7f2f5006e2f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c009b40 tx=0x7f2f5c009190 comp rx=0 tx=0).stop 2024-04-24T22:59:49.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.073+0000 7f2f6b3b7640 1 -- 172.21.15.32:0/31839869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f64162c40 msgr2=0x7f2f64147970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:49.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.073+0000 7f2f6b3b7640 1 --2- 172.21.15.32:0/31839869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f64162c40 0x7f2f64147970 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2f5400c760 tx=0x7f2f5400cc20 comp rx=0 tx=0).stop 2024-04-24T22:59:49.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.073+0000 7f2f6b3b7640 1 -- 172.21.15.32:0/31839869 shutdown_connections 2024-04-24T22:59:49.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.073+0000 7f2f6b3b7640 1 --2- 172.21.15.32:0/31839869 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2f5006be30 0x7f2f5006e2f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:49.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.073+0000 7f2f6b3b7640 1 --2- 172.21.15.32:0/31839869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f64162c40 0x7f2f64147970 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:49.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.073+0000 7f2f6b3b7640 1 --2- 172.21.15.32:0/31839869 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2f64162320 0x7f2f640cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:49.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.073+0000 7f2f6b3b7640 1 --2- 172.21.15.32:0/31839869 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2f640b5af0 0x7f2f6414ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:49.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.073+0000 7f2f6b3b7640 1 -- 172.21.15.32:0/31839869 >> 172.21.15.32:0/31839869 conn(0x7f2f640d0ae0 msgr2=0x7f2f6415f260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:49.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.073+0000 7f2f6b3b7640 1 -- 172.21.15.32:0/31839869 shutdown_connections 2024-04-24T22:59:49.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.073+0000 7f2f6b3b7640 1 -- 172.21.15.32:0/31839869 wait complete. 2024-04-24T22:59:49.087 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-24T22:59:49.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.195+0000 7f897ce61640 1 Processor -- start 2024-04-24T22:59:49.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.195+0000 7f897ce61640 1 -- start start 2024-04-24T22:59:49.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.196+0000 7f897ce61640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f897815b960 0x7f89780cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:49.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.196+0000 7f897ce61640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f89780b58e0 0x7f89780b5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:49.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.196+0000 7f897ce61640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f897815bde0 con 0x7f897815b580 2024-04-24T22:59:49.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.196+0000 7f897ce61640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f89780b6270 con 0x7f89780b58e0 2024-04-24T22:59:49.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.196+0000 7f897ce61640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f897815c9f0 con 0x7f897815b960 2024-04-24T22:59:49.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.196+0000 7f8976d9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f89780b58e0 0x7f89780b5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:49.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.196+0000 7f8976d9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f89780b58e0 0x7f89780b5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57760/0 (socket says 172.21.15.32:57760) 2024-04-24T22:59:49.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.196+0000 7f8976d9d640 1 -- 172.21.15.32:0/36102935 learned_addr learned my addr 172.21.15.32:0/36102935 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:49.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.197+0000 7f896ffff640 1 --2- 172.21.15.32:0/36102935 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f897815b960 0x7f89780cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:49.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.197+0000 7f8976d9d640 1 -- 172.21.15.32:0/36102935 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f897815b960 msgr2=0x7f89780cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:49.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.197+0000 7f8976d9d640 1 --2- 172.21.15.32:0/36102935 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f897815b960 0x7f89780cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:49.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.197+0000 7f8976d9d640 1 -- 172.21.15.32:0/36102935 >> v1:172.21.15.121:6789/0 conn(0x7f897815b580 legacy=0x7f897806cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:49.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.197+0000 7f8976d9d640 1 -- 172.21.15.32:0/36102935 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f897815c0c0 con 0x7f89780b58e0 2024-04-24T22:59:49.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.197+0000 7f8976d9d640 1 --2- 172.21.15.32:0/36102935 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89780b58e0 0x7f89780b5d30 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8970009e50 tx=0x7f897002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60ebbe2ca6776a06 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:49.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.198+0000 7f8975d9b640 1 -- 172.21.15.32:0/36102935 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8970036070 con 0x7f89780b58e0 2024-04-24T22:59:49.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.198+0000 7f8975d9b640 1 -- 172.21.15.32:0/36102935 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8970031600 con 0x7f89780b58e0 2024-04-24T22:59:49.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.198+0000 7f8975d9b640 1 -- 172.21.15.32:0/36102935 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89700319d0 con 0x7f89780b58e0 2024-04-24T22:59:49.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.198+0000 7f897ce61640 1 -- 172.21.15.32:0/36102935 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89780b58e0 msgr2=0x7f89780b5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:49.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.198+0000 7f897ce61640 1 --2- 172.21.15.32:0/36102935 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89780b58e0 0x7f89780b5d30 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8970009e50 tx=0x7f897002a710 comp rx=0 tx=0).stop 2024-04-24T22:59:49.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.198+0000 7f897ce61640 1 -- 172.21.15.32:0/36102935 shutdown_connections 2024-04-24T22:59:49.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.198+0000 7f897ce61640 1 --2- 172.21.15.32:0/36102935 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89780b58e0 0x7f89780b5d30 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:49.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.198+0000 7f897ce61640 1 --2- 172.21.15.32:0/36102935 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f897815b960 0x7f89780cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:49.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.198+0000 7f897ce61640 1 -- 172.21.15.32:0/36102935 >> 172.21.15.32:0/36102935 conn(0x7f89780b1f10 msgr2=0x7f89780b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:49.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.198+0000 7f897ce61640 1 -- 172.21.15.32:0/36102935 shutdown_connections 2024-04-24T22:59:49.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.198+0000 7f897ce61640 1 -- 172.21.15.32:0/36102935 wait complete. 2024-04-24T22:59:49.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.199+0000 7f897ce61640 1 Processor -- start 2024-04-24T22:59:49.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.199+0000 7f897ce61640 1 -- start start 2024-04-24T22:59:49.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.200+0000 7f897ce61640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89780b58e0 0x7f8978144cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:49.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.200+0000 7f897ce61640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f897815b580 0x7f8978145500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:49.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.200+0000 7f897ce61640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f897815b960 0x7f8978071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:49.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.200+0000 7f897ce61640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f897815c680 con 0x7f89780b58e0 2024-04-24T22:59:49.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.200+0000 7f897ce61640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f89780ce3c0 con 0x7f897815b580 2024-04-24T22:59:49.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.200+0000 7f897ce61640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f897806c7a0 con 0x7f897815b960 2024-04-24T22:59:49.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.200+0000 7f897659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89780b58e0 0x7f8978144cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:49.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.200+0000 7f897659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89780b58e0 0x7f8978144cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57776/0 (socket says 172.21.15.32:57776) 2024-04-24T22:59:49.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.200+0000 7f897659c640 1 -- 172.21.15.32:0/2693775213 learned_addr learned my addr 172.21.15.32:0/2693775213 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:49.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.200+0000 7f8976d9d640 1 --2- 172.21.15.32:0/2693775213 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f897815b960 0x7f8978071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:49.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.200+0000 7f896ffff640 1 --2- 172.21.15.32:0/2693775213 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f897815b580 0x7f8978145500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:49.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.201+0000 7f897659c640 1 -- 172.21.15.32:0/2693775213 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f897815b960 msgr2=0x7f8978071940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:49.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.201+0000 7f897659c640 1 --2- 172.21.15.32:0/2693775213 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f897815b960 0x7f8978071940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:49.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.201+0000 7f897659c640 1 -- 172.21.15.32:0/2693775213 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f897815b580 msgr2=0x7f8978145500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:49.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.201+0000 7f897659c640 1 --2- 172.21.15.32:0/2693775213 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f897815b580 0x7f8978145500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:49.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.201+0000 7f897659c640 1 -- 172.21.15.32:0/2693775213 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8978145a40 con 0x7f89780b58e0 2024-04-24T22:59:49.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.201+0000 7f896ffff640 1 --2- 172.21.15.32:0/2693775213 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f897815b580 0x7f8978145500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:49.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.201+0000 7f897659c640 1 --2- 172.21.15.32:0/2693775213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89780b58e0 0x7f8978144cf0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f896800c400 tx=0x7f896800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:49.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.201+0000 7f896effd640 1 -- 172.21.15.32:0/2693775213 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89680052d0 con 0x7f89780b58e0 2024-04-24T22:59:49.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.201+0000 7f896effd640 1 -- 172.21.15.32:0/2693775213 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8968005490 con 0x7f89780b58e0 2024-04-24T22:59:49.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.201+0000 7f897ce61640 1 -- 172.21.15.32:0/2693775213 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89781575d0 con 0x7f89780b58e0 2024-04-24T22:59:49.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.201+0000 7f896effd640 1 -- 172.21.15.32:0/2693775213 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8968007260 con 0x7f89780b58e0 2024-04-24T22:59:49.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.201+0000 7f897ce61640 1 -- 172.21.15.32:0/2693775213 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8978157b90 con 0x7f89780b58e0 2024-04-24T22:59:49.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.202+0000 7f897ce61640 1 -- 172.21.15.32:0/2693775213 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8978068430 con 0x7f89780b58e0 2024-04-24T22:59:49.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.203+0000 7f896effd640 1 -- 172.21.15.32:0/2693775213 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8968007400 con 0x7f89780b58e0 2024-04-24T22:59:49.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.203+0000 7f896effd640 1 --2- 172.21.15.32:0/2693775213 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f895806be30 0x7f895806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:49.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.203+0000 7f896effd640 1 -- 172.21.15.32:0/2693775213 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5795+0+0 (secure 0 0 0) 0x7f896808a340 con 0x7f89780b58e0 2024-04-24T22:59:49.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.203+0000 7f896ffff640 1 --2- 172.21.15.32:0/2693775213 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f895806be30 0x7f895806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:49.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.204+0000 7f896ffff640 1 --2- 172.21.15.32:0/2693775213 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f895806be30 0x7f895806e2f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8960006fd0 tx=0x7f8960008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:49.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.208+0000 7f896effd640 1 -- 172.21.15.32:0/2693775213 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8968058640 con 0x7f89780b58e0 2024-04-24T22:59:49.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:49.394+0000 7f897ce61640 1 -- 172.21.15.32:0/2693775213 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f89780b2960 con 0x7f89780b58e0 2024-04-24T22:59:50.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.075+0000 7f896effd640 1 -- 172.21.15.32:0/2693775213 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v23) ==== 181+0+0 (secure 0 0 0) 0x7f896805c940 con 0x7f89780b58e0 2024-04-24T22:59:50.077 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2024-04-24T22:59:50.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.079+0000 7f897ce61640 1 -- 172.21.15.32:0/2693775213 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f895806be30 msgr2=0x7f895806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:50.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.079+0000 7f897ce61640 1 --2- 172.21.15.32:0/2693775213 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f895806be30 0x7f895806e2f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8960006fd0 tx=0x7f8960008040 comp rx=0 tx=0).stop 2024-04-24T22:59:50.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.079+0000 7f897ce61640 1 -- 172.21.15.32:0/2693775213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89780b58e0 msgr2=0x7f8978144cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:50.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.079+0000 7f897ce61640 1 --2- 172.21.15.32:0/2693775213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89780b58e0 0x7f8978144cf0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f896800c400 tx=0x7f896800c8c0 comp rx=0 tx=0).stop 2024-04-24T22:59:50.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.079+0000 7f897ce61640 1 -- 172.21.15.32:0/2693775213 shutdown_connections 2024-04-24T22:59:50.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.079+0000 7f897ce61640 1 --2- 172.21.15.32:0/2693775213 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f895806be30 0x7f895806e2f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:50.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.079+0000 7f897ce61640 1 --2- 172.21.15.32:0/2693775213 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f897815b960 0x7f8978071940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:50.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.079+0000 7f897ce61640 1 --2- 172.21.15.32:0/2693775213 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f897815b580 0x7f8978145500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:50.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.079+0000 7f897ce61640 1 --2- 172.21.15.32:0/2693775213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89780b58e0 0x7f8978144cf0 secure :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f896800c400 tx=0x7f896800c8c0 comp rx=0 tx=0).stop 2024-04-24T22:59:50.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.079+0000 7f897ce61640 1 -- 172.21.15.32:0/2693775213 >> 172.21.15.32:0/2693775213 conn(0x7f89780b1f10 msgr2=0x7f89780ce570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:50.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.079+0000 7f897ce61640 1 -- 172.21.15.32:0/2693775213 shutdown_connections 2024-04-24T22:59:50.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.079+0000 7f897ce61640 1 -- 172.21.15.32:0/2693775213 wait complete. 2024-04-24T22:59:50.093 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-24T22:59:50.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.201+0000 7fe4189b7640 1 Processor -- start 2024-04-24T22:59:50.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.201+0000 7fe4189b7640 1 -- start start 2024-04-24T22:59:50.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.202+0000 7fe4189b7640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe414067980 0x7fe4140afa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:50.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.202+0000 7fe4189b7640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe414071620 con 0x7fe4140aff70 2024-04-24T22:59:50.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.202+0000 7fe4189b7640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4140af270 con 0x7fe414161c30 2024-04-24T22:59:50.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.202+0000 7fe4189b7640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fe4140af5a0 con 0x7fe414067980 2024-04-24T22:59:50.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.202+0000 7fe40bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe414067980 0x7fe4140afa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:50.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.202+0000 7fe40bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe414067980 0x7fe4140afa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50720/0 (socket says 172.21.15.32:50720) 2024-04-24T22:59:50.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.202+0000 7fe40bfff640 1 -- 172.21.15.32:0/3391799962 learned_addr learned my addr 172.21.15.32:0/3391799962 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:50.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.202+0000 7fe40bfff640 1 -- 172.21.15.32:0/3391799962 >> v1:172.21.15.121:6789/0 conn(0x7fe414161c30 legacy=0x7fe41406cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:50.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.202+0000 7fe40bfff640 1 -- 172.21.15.32:0/3391799962 >> v1:172.21.15.32:6789/0 conn(0x7fe4140aff70 legacy=0x7fe4140b4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:50.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.202+0000 7fe40bfff640 1 -- 172.21.15.32:0/3391799962 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe41406d4d0 con 0x7fe414067980 2024-04-24T22:59:50.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.203+0000 7fe40bfff640 1 --2- 172.21.15.32:0/3391799962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe414067980 0x7fe4140afa30 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe41406ca30 tx=0x7fe3fc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ec28cec525c1fd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:50.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.203+0000 7fe411d9b640 1 -- 172.21.15.32:0/3391799962 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc037070 con 0x7fe414067980 2024-04-24T22:59:50.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.203+0000 7fe411d9b640 1 -- 172.21.15.32:0/3391799962 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3fc032440 con 0x7fe414067980 2024-04-24T22:59:50.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.203+0000 7fe411d9b640 1 -- 172.21.15.32:0/3391799962 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc032810 con 0x7fe414067980 2024-04-24T22:59:50.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.204+0000 7fe4189b7640 1 -- 172.21.15.32:0/3391799962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe414067980 msgr2=0x7fe4140afa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:50.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.204+0000 7fe4189b7640 1 --2- 172.21.15.32:0/3391799962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe414067980 0x7fe4140afa30 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe41406ca30 tx=0x7fe3fc02a270 comp rx=0 tx=0).stop 2024-04-24T22:59:50.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.204+0000 7fe4189b7640 1 -- 172.21.15.32:0/3391799962 shutdown_connections 2024-04-24T22:59:50.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.204+0000 7fe4189b7640 1 --2- 172.21.15.32:0/3391799962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe414067980 0x7fe4140afa30 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:50.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.204+0000 7fe4189b7640 1 -- 172.21.15.32:0/3391799962 >> 172.21.15.32:0/3391799962 conn(0x7fe4140d0a60 msgr2=0x7fe4140d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:50.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.204+0000 7fe4189b7640 1 -- 172.21.15.32:0/3391799962 shutdown_connections 2024-04-24T22:59:50.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.204+0000 7fe4189b7640 1 -- 172.21.15.32:0/3391799962 wait complete. 2024-04-24T22:59:50.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.205+0000 7fe4189b7640 1 Processor -- start 2024-04-24T22:59:50.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.205+0000 7fe4189b7640 1 -- start start 2024-04-24T22:59:50.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.206+0000 7fe4189b7640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe414067980 0x7fe4140c5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:50.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.206+0000 7fe4189b7640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe4140aff70 0x7fe414145b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:50.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.206+0000 7fe4189b7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe414161c30 0x7fe4141e92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:50.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.206+0000 7fe4189b7640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe4140b03c0 con 0x7fe414161c30 2024-04-24T22:59:50.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.206+0000 7fe4189b7640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe414071ba0 con 0x7fe414067980 2024-04-24T22:59:50.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.206+0000 7fe4189b7640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe414162010 con 0x7fe4140aff70 2024-04-24T22:59:50.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.206+0000 7fe412d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe414161c30 0x7fe4141e92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:50.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.206+0000 7fe412d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe414161c30 0x7fe4141e92b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50730/0 (socket says 172.21.15.32:50730) 2024-04-24T22:59:50.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.206+0000 7fe412d9d640 1 -- 172.21.15.32:0/1662220260 learned_addr learned my addr 172.21.15.32:0/1662220260 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:50.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.206+0000 7fe40bfff640 1 --2- 172.21.15.32:0/1662220260 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe4140aff70 0x7fe414145b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:50.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.206+0000 7fe41259c640 1 --2- 172.21.15.32:0/1662220260 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe414067980 0x7fe4140c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:50.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.206+0000 7fe412d9d640 1 -- 172.21.15.32:0/1662220260 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe4140aff70 msgr2=0x7fe414145b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:50.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.206+0000 7fe412d9d640 1 --2- 172.21.15.32:0/1662220260 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe4140aff70 0x7fe414145b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:50.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.206+0000 7fe412d9d640 1 -- 172.21.15.32:0/1662220260 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe414067980 msgr2=0x7fe4140c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:50.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.206+0000 7fe412d9d640 1 --2- 172.21.15.32:0/1662220260 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe414067980 0x7fe4140c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:50.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.206+0000 7fe412d9d640 1 -- 172.21.15.32:0/1662220260 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4141612a0 con 0x7fe414161c30 2024-04-24T22:59:50.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.206+0000 7fe40bfff640 1 --2- 172.21.15.32:0/1662220260 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe4140aff70 0x7fe414145b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:59:50.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.206+0000 7fe41259c640 1 --2- 172.21.15.32:0/1662220260 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe414067980 0x7fe4140c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:59:50.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.206+0000 7fe412d9d640 1 --2- 172.21.15.32:0/1662220260 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe414161c30 0x7fe4141e92b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe40c00c7d0 tx=0x7fe40c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:50.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.207+0000 7fe40affd640 1 -- 172.21.15.32:0/1662220260 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40c003b70 con 0x7fe414161c30 2024-04-24T22:59:50.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.207+0000 7fe40affd640 1 -- 172.21.15.32:0/1662220260 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe40c003d30 con 0x7fe414161c30 2024-04-24T22:59:50.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.207+0000 7fe4189b7640 1 -- 172.21.15.32:0/1662220260 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe41415e810 con 0x7fe414161c30 2024-04-24T22:59:50.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.207+0000 7fe40affd640 1 -- 172.21.15.32:0/1662220260 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40c00aa90 con 0x7fe414161c30 2024-04-24T22:59:50.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.207+0000 7fe4189b7640 1 -- 172.21.15.32:0/1662220260 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4141e97f0 con 0x7fe414161c30 2024-04-24T22:59:50.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.208+0000 7fe4189b7640 1 -- 172.21.15.32:0/1662220260 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe414068430 con 0x7fe414161c30 2024-04-24T22:59:50.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.208+0000 7fe40affd640 1 -- 172.21.15.32:0/1662220260 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe40c00ac30 con 0x7fe414161c30 2024-04-24T22:59:50.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.209+0000 7fe40affd640 1 --2- 172.21.15.32:0/1662220260 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe3f406be60 0x7fe3f406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:50.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.209+0000 7fe40affd640 1 -- 172.21.15.32:0/1662220260 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5357+0+0 (secure 0 0 0) 0x7fe40c08acb0 con 0x7fe414161c30 2024-04-24T22:59:50.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.209+0000 7fe41259c640 1 --2- 172.21.15.32:0/1662220260 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe3f406be60 0x7fe3f406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:50.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.210+0000 7fe41259c640 1 --2- 172.21.15.32:0/1662220260 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe3f406be60 0x7fe3f406e320 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe40400a830 tx=0x7fe404008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:50.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.213+0000 7fe40affd640 1 -- 172.21.15.32:0/1662220260 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe40c0597c0 con 0x7fe414161c30 2024-04-24T22:59:50.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:50.403+0000 7fe4189b7640 1 -- 172.21.15.32:0/1662220260 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe4140d18d0 con 0x7fe414161c30 2024-04-24T22:59:51.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.077+0000 7fe40affd640 1 -- 172.21.15.32:0/1662220260 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v24) ==== 169+0+0 (secure 0 0 0) 0x7fe40c0590c0 con 0x7fe414161c30 2024-04-24T22:59:51.078 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2024-04-24T22:59:51.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.080+0000 7fe4189b7640 1 -- 172.21.15.32:0/1662220260 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe3f406be60 msgr2=0x7fe3f406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:51.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.080+0000 7fe4189b7640 1 --2- 172.21.15.32:0/1662220260 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe3f406be60 0x7fe3f406e320 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe40400a830 tx=0x7fe404008040 comp rx=0 tx=0).stop 2024-04-24T22:59:51.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.080+0000 7fe4189b7640 1 -- 172.21.15.32:0/1662220260 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe414161c30 msgr2=0x7fe4141e92b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:51.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.080+0000 7fe4189b7640 1 --2- 172.21.15.32:0/1662220260 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe414161c30 0x7fe4141e92b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe40c00c7d0 tx=0x7fe40c00cc90 comp rx=0 tx=0).stop 2024-04-24T22:59:51.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.080+0000 7fe4189b7640 1 -- 172.21.15.32:0/1662220260 shutdown_connections 2024-04-24T22:59:51.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.080+0000 7fe4189b7640 1 --2- 172.21.15.32:0/1662220260 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe3f406be60 0x7fe3f406e320 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:51.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.080+0000 7fe4189b7640 1 --2- 172.21.15.32:0/1662220260 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe414161c30 0x7fe4141e92b0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:51.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.080+0000 7fe4189b7640 1 --2- 172.21.15.32:0/1662220260 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe4140aff70 0x7fe414145b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:51.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.080+0000 7fe4189b7640 1 --2- 172.21.15.32:0/1662220260 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe414067980 0x7fe4140c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:51.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.080+0000 7fe4189b7640 1 -- 172.21.15.32:0/1662220260 >> 172.21.15.32:0/1662220260 conn(0x7fe4140d0a60 msgr2=0x7fe41415c640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:51.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.082+0000 7fe4189b7640 1 -- 172.21.15.32:0/1662220260 shutdown_connections 2024-04-24T22:59:51.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.082+0000 7fe4189b7640 1 -- 172.21.15.32:0/1662220260 wait complete. 2024-04-24T22:59:51.098 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-24T22:59:51.098 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-24T22:59:51.098 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-24T22:59:51.098 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T22:59:51.099 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-24T22:59:51.099 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-24T22:59:51.099 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-24T22:59:51.099 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T22:59:51.099 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-24T22:59:51.099 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-24T22:59:51.099 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-24T22:59:51.100 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T22:59:51.100 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-24T22:59:51.100 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-24T22:59:51.100 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-24T22:59:51.100 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T22:59:51.109 INFO:tasks.ceph.mds.a:Stopped 2024-04-24T22:59:51.109 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-24T22:59:51.110 INFO:tasks.ceph.mds.c:Stopped 2024-04-24T22:59:51.110 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-24T22:59:51.111 INFO:tasks.ceph.mds.b:Stopped 2024-04-24T22:59:51.111 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-24T22:59:51.112 INFO:tasks.ceph.mds.d:Stopped 2024-04-24T22:59:51.112 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-24T22:59:51.153 INFO:tasks.ceph.mds.a:Started 2024-04-24T22:59:51.154 DEBUG:teuthology.parallel:result is None 2024-04-24T22:59:51.157 INFO:tasks.ceph.mds.b:Started 2024-04-24T22:59:51.157 INFO:tasks.ceph.mds.c:Started 2024-04-24T22:59:51.158 INFO:tasks.ceph.mds.d:Started 2024-04-24T22:59:51.158 DEBUG:teuthology.parallel:result is None 2024-04-24T22:59:51.158 DEBUG:teuthology.parallel:result is None 2024-04-24T22:59:51.159 DEBUG:teuthology.parallel:result is None 2024-04-24T22:59:51.159 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-24T22:59:51.235 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2024-04-24T22:59:51.235 INFO:tasks.ceph.mds.b.smithi121.stdout:starting mds.b at 2024-04-24T22:59:51.235 INFO:tasks.ceph.mds.d.smithi121.stdout:starting mds.d at 2024-04-24T22:59:51.236 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2024-04-24T22:59:51.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.271+0000 7f7f2b35a640 1 Processor -- start 2024-04-24T22:59:51.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.271+0000 7f7f2b35a640 1 -- start start 2024-04-24T22:59:51.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.272+0000 7f7f2b35a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7f2415e9b0 0x7f7f240d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:51.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.272+0000 7f7f2b35a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7f240d4210 0x7f7f2406b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:51.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.272+0000 7f7f2b35a640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f240d3170 con 0x7f7f240ba0e0 2024-04-24T22:59:51.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.272+0000 7f7f2b35a640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f7f240b8900 con 0x7f7f240d4210 2024-04-24T22:59:51.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.272+0000 7f7f2b35a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f7f240d3450 con 0x7f7f2415e9b0 2024-04-24T22:59:51.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.272+0000 7f7f288f5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7f240d4210 0x7f7f2406b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:51.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.272+0000 7f7f298f7640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7f240ba0e0 0x7f7f24071730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53854/0 (socket says 172.21.15.32:53854) 2024-04-24T22:59:51.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.272+0000 7f7f298f7640 1 -- 172.21.15.32:0/1067644689 learned_addr learned my addr 172.21.15.32:0/1067644689 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:51.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.273+0000 7f7f290f6640 1 --2- 172.21.15.32:0/1067644689 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7f2415e9b0 0x7f7f240d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:51.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.273+0000 7f7f288f5640 1 -- 172.21.15.32:0/1067644689 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7f2415e9b0 msgr2=0x7f7f240d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:51.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.273+0000 7f7f288f5640 1 --2- 172.21.15.32:0/1067644689 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7f2415e9b0 0x7f7f240d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:51.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.273+0000 7f7f288f5640 1 -- 172.21.15.32:0/1067644689 >> v1:172.21.15.32:6789/0 conn(0x7f7f240ba0e0 legacy=0x7f7f24071730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:51.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.273+0000 7f7f288f5640 1 -- 172.21.15.32:0/1067644689 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f240d3730 con 0x7f7f240d4210 2024-04-24T22:59:51.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.273+0000 7f7f1bfff640 1 -- 172.21.15.32:0/1067644689 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 272102821 0 0) 0x7f7f240d3170 con 0x7f7f240ba0e0 2024-04-24T22:59:51.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.274+0000 7f7f288f5640 1 --2- 172.21.15.32:0/1067644689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f240d4210 0x7f7f2406b110 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7f140097a0 tx=0x7f7f1402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=565b0b0c0cf9b4a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:51.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.274+0000 7f7f1bfff640 1 -- 172.21.15.32:0/1067644689 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f14037070 con 0x7f7f240d4210 2024-04-24T22:59:51.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.274+0000 7f7f1bfff640 1 -- 172.21.15.32:0/1067644689 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f14032440 con 0x7f7f240d4210 2024-04-24T22:59:51.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.274+0000 7f7f1bfff640 1 -- 172.21.15.32:0/1067644689 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f14032810 con 0x7f7f240d4210 2024-04-24T22:59:51.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.274+0000 7f7f2b35a640 1 -- 172.21.15.32:0/1067644689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f240d4210 msgr2=0x7f7f2406b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:51.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.274+0000 7f7f2b35a640 1 --2- 172.21.15.32:0/1067644689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f240d4210 0x7f7f2406b110 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7f140097a0 tx=0x7f7f1402a180 comp rx=0 tx=0).stop 2024-04-24T22:59:51.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.274+0000 7f7f2b35a640 1 -- 172.21.15.32:0/1067644689 shutdown_connections 2024-04-24T22:59:51.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.274+0000 7f7f2b35a640 1 --2- 172.21.15.32:0/1067644689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f240d4210 0x7f7f2406b110 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:51.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.274+0000 7f7f2b35a640 1 --2- 172.21.15.32:0/1067644689 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7f2415e9b0 0x7f7f240d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:51.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.274+0000 7f7f2b35a640 1 -- 172.21.15.32:0/1067644689 >> 172.21.15.32:0/1067644689 conn(0x7f7f240d11d0 msgr2=0x7f7f240d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:51.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.274+0000 7f7f2b35a640 1 -- 172.21.15.32:0/1067644689 shutdown_connections 2024-04-24T22:59:51.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.274+0000 7f7f2b35a640 1 -- 172.21.15.32:0/1067644689 wait complete. 2024-04-24T22:59:51.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.276+0000 7f7f2b35a640 1 Processor -- start 2024-04-24T22:59:51.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.276+0000 7f7f2b35a640 1 -- start start 2024-04-24T22:59:51.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.276+0000 7f7f2b35a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f240ba0e0 0x7f7f2415c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:51.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.276+0000 7f7f2b35a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f240d4210 0x7f7f2414f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:51.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.276+0000 7f7f2b35a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7f2415e9b0 0x7f7f241e7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:51.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.276+0000 7f7f2b35a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7f240d2400 con 0x7f7f240ba0e0 2024-04-24T22:59:51.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.276+0000 7f7f2b35a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7f240d3cf0 con 0x7f7f240d4210 2024-04-24T22:59:51.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.276+0000 7f7f2b35a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f7f2406b650 con 0x7f7f2415e9b0 2024-04-24T22:59:51.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.277+0000 7f7f290f6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f240ba0e0 0x7f7f2415c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:51.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.277+0000 7f7f290f6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f240ba0e0 0x7f7f2415c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50790/0 (socket says 172.21.15.32:50790) 2024-04-24T22:59:51.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.277+0000 7f7f290f6640 1 -- 172.21.15.32:0/1954215500 learned_addr learned my addr 172.21.15.32:0/1954215500 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:51.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.277+0000 7f7f298f7640 1 --2- 172.21.15.32:0/1954215500 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7f2415e9b0 0x7f7f241e7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:51.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.277+0000 7f7f288f5640 1 --2- 172.21.15.32:0/1954215500 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f240d4210 0x7f7f2414f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:51.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.277+0000 7f7f290f6640 1 -- 172.21.15.32:0/1954215500 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7f2415e9b0 msgr2=0x7f7f241e7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:51.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.277+0000 7f7f290f6640 1 --2- 172.21.15.32:0/1954215500 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7f2415e9b0 0x7f7f241e7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:51.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.277+0000 7f7f290f6640 1 -- 172.21.15.32:0/1954215500 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f240d4210 msgr2=0x7f7f2414f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:51.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.277+0000 7f7f290f6640 1 --2- 172.21.15.32:0/1954215500 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f240d4210 0x7f7f2414f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:51.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.277+0000 7f7f290f6640 1 -- 172.21.15.32:0/1954215500 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f240cb820 con 0x7f7f240ba0e0 2024-04-24T22:59:51.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.277+0000 7f7f290f6640 1 --2- 172.21.15.32:0/1954215500 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f240ba0e0 0x7f7f2415c060 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7f2000ebe0 tx=0x7f7f2000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:51.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.277+0000 7f7f19ffb640 1 -- 172.21.15.32:0/1954215500 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f20017070 con 0x7f7f240ba0e0 2024-04-24T22:59:51.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.277+0000 7f7f19ffb640 1 -- 172.21.15.32:0/1954215500 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f2000ecd0 con 0x7f7f240ba0e0 2024-04-24T22:59:51.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.277+0000 7f7f2b35a640 1 -- 172.21.15.32:0/1954215500 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f241e8290 con 0x7f7f240ba0e0 2024-04-24T22:59:51.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.277+0000 7f7f19ffb640 1 -- 172.21.15.32:0/1954215500 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f2000ee80 con 0x7f7f240ba0e0 2024-04-24T22:59:51.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.278+0000 7f7f2b35a640 1 -- 172.21.15.32:0/1954215500 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f241e8850 con 0x7f7f240ba0e0 2024-04-24T22:59:51.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.278+0000 7f7f2b35a640 1 -- 172.21.15.32:0/1954215500 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f24068430 con 0x7f7f240ba0e0 2024-04-24T22:59:51.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.279+0000 7f7f19ffb640 1 -- 172.21.15.32:0/1954215500 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7f2001a070 con 0x7f7f240ba0e0 2024-04-24T22:59:51.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.279+0000 7f7f19ffb640 1 --2- 172.21.15.32:0/1954215500 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7f0806be30 0x7f7f0806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:51.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.280+0000 7f7f19ffb640 1 -- 172.21.15.32:0/1954215500 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 4943+0+0 (secure 0 0 0) 0x7f7f2008a980 con 0x7f7f240ba0e0 2024-04-24T22:59:51.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.280+0000 7f7f288f5640 1 --2- 172.21.15.32:0/1954215500 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7f0806be30 0x7f7f0806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:51.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.280+0000 7f7f288f5640 1 --2- 172.21.15.32:0/1954215500 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7f0806be30 0x7f7f0806e2f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7f14034000 tx=0x7f7f1402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:51.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.284+0000 7f7f19ffb640 1 -- 172.21.15.32:0/1954215500 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7f20058fe0 con 0x7f7f240ba0e0 2024-04-24T22:59:51.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:51.484+0000 7f7f2b35a640 1 -- 172.21.15.32:0/1954215500 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f7f240cb0b0 con 0x7f7f240ba0e0 2024-04-24T22:59:52.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.090+0000 7f7f19ffb640 1 -- 172.21.15.32:0/1954215500 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v26) ==== 97+0+0 (secure 0 0 0) 0x7f7f2005d2e0 con 0x7f7f240ba0e0 2024-04-24T22:59:52.091 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blocklist entries 2024-04-24T22:59:52.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.092+0000 7f7f2b35a640 1 -- 172.21.15.32:0/1954215500 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7f0806be30 msgr2=0x7f7f0806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:52.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.092+0000 7f7f2b35a640 1 --2- 172.21.15.32:0/1954215500 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7f0806be30 0x7f7f0806e2f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7f14034000 tx=0x7f7f1402e000 comp rx=0 tx=0).stop 2024-04-24T22:59:52.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.092+0000 7f7f2b35a640 1 -- 172.21.15.32:0/1954215500 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f240ba0e0 msgr2=0x7f7f2415c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:52.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.092+0000 7f7f2b35a640 1 --2- 172.21.15.32:0/1954215500 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f240ba0e0 0x7f7f2415c060 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7f2000ebe0 tx=0x7f7f2000c520 comp rx=0 tx=0).stop 2024-04-24T22:59:52.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.092+0000 7f7f2b35a640 1 -- 172.21.15.32:0/1954215500 shutdown_connections 2024-04-24T22:59:52.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.092+0000 7f7f2b35a640 1 --2- 172.21.15.32:0/1954215500 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7f0806be30 0x7f7f0806e2f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:52.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.092+0000 7f7f2b35a640 1 --2- 172.21.15.32:0/1954215500 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7f2415e9b0 0x7f7f241e7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:52.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.092+0000 7f7f2b35a640 1 --2- 172.21.15.32:0/1954215500 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f240d4210 0x7f7f2414f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:52.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.092+0000 7f7f2b35a640 1 --2- 172.21.15.32:0/1954215500 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f240ba0e0 0x7f7f2415c060 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:52.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.092+0000 7f7f2b35a640 1 -- 172.21.15.32:0/1954215500 >> 172.21.15.32:0/1954215500 conn(0x7f7f240d11d0 msgr2=0x7f7f241e9a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:52.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.092+0000 7f7f2b35a640 1 -- 172.21.15.32:0/1954215500 shutdown_connections 2024-04-24T22:59:52.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.092+0000 7f7f2b35a640 1 -- 172.21.15.32:0/1954215500 wait complete. 2024-04-24T22:59:52.106 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-24T22:59:52.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.206+0000 7f332a35a640 1 Processor -- start 2024-04-24T22:59:52.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.206+0000 7f332a35a640 1 -- start start 2024-04-24T22:59:52.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.207+0000 7f332a35a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f332415e9b0 0x7f33240d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:52.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.207+0000 7f332a35a640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33240d3170 con 0x7f33240d4210 2024-04-24T22:59:52.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.207+0000 7f332a35a640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33240d3450 con 0x7f332406b820 2024-04-24T22:59:52.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.207+0000 7f332a35a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f33240d3730 con 0x7f332415e9b0 2024-04-24T22:59:52.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.207+0000 7f33237fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f33240d4210 0x7f332406b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53862/0 (socket says 172.21.15.32:53862) 2024-04-24T22:59:52.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.207+0000 7f33237fe640 1 -- 172.21.15.32:0/922697276 learned_addr learned my addr 172.21.15.32:0/922697276 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:52.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.208+0000 7f3322ffd640 1 -- 172.21.15.32:0/922697276 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4020484305 0 0) 0x7f33240d3170 con 0x7f33240d4210 2024-04-24T22:59:52.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.208+0000 7f3323fff640 1 --2- 172.21.15.32:0/922697276 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f332415e9b0 0x7f33240d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:52.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.208+0000 7f3322ffd640 1 -- 172.21.15.32:0/922697276 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3310003520 con 0x7f33240d4210 2024-04-24T22:59:52.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.208+0000 7f3322ffd640 1 -- 172.21.15.32:0/922697276 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2468102051 0 0) 0x7f3310003520 con 0x7f33240d4210 2024-04-24T22:59:52.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.208+0000 7f3322ffd640 1 -- 172.21.15.32:0/922697276 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f332415e9b0 msgr2=0x7f33240d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:52.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.208+0000 7f3322ffd640 1 --2- 172.21.15.32:0/922697276 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f332415e9b0 0x7f33240d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:52.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.208+0000 7f3322ffd640 1 -- 172.21.15.32:0/922697276 >> v1:172.21.15.121:6789/0 conn(0x7f332406b820 legacy=0x7f33240b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:52.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.208+0000 7f3322ffd640 1 -- 172.21.15.32:0/922697276 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33240d3a10 con 0x7f33240d4210 2024-04-24T22:59:52.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.208+0000 7f3322ffd640 1 -- 172.21.15.32:0/922697276 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f330c001670 con 0x7f33240d4210 2024-04-24T22:59:52.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.208+0000 7f3322ffd640 1 -- 172.21.15.32:0/922697276 >> v1:172.21.15.32:6789/0 conn(0x7f33240d4210 legacy=0x7f332406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:52.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.208+0000 7f3322ffd640 1 --2- 172.21.15.32:0/922697276 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33100039e0 0x7f3310003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:52.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.208+0000 7f3322ffd640 1 --2- 172.21.15.32:0/922697276 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f332415e9b0 0x7f3310004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:52.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.208+0000 7f3322ffd640 1 --2- 172.21.15.32:0/922697276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3310004d40 0x7f33100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:52.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.208+0000 7f3322ffd640 1 -- 172.21.15.32:0/922697276 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f33240d3730 con 0x7f3310004d40 2024-04-24T22:59:52.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.208+0000 7f3322ffd640 1 -- 172.21.15.32:0/922697276 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f331000b9c0 con 0x7f33100039e0 2024-04-24T22:59:52.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.208+0000 7f3322ffd640 1 -- 172.21.15.32:0/922697276 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f331000bbc0 con 0x7f332415e9b0 2024-04-24T22:59:52.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.208+0000 7f33288f7640 1 --2- 172.21.15.32:0/922697276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3310004d40 0x7f33100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:52.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.208+0000 7f3323fff640 1 --2- 172.21.15.32:0/922697276 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33100039e0 0x7f3310003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:52.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.208+0000 7f33237fe640 1 --2- 172.21.15.32:0/922697276 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f332415e9b0 0x7f3310004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:52.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.208+0000 7f33288f7640 1 -- 172.21.15.32:0/922697276 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f332415e9b0 msgr2=0x7f3310004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:52.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.208+0000 7f33288f7640 1 --2- 172.21.15.32:0/922697276 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f332415e9b0 0x7f3310004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:52.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.208+0000 7f33288f7640 1 -- 172.21.15.32:0/922697276 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33100039e0 msgr2=0x7f3310003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:52.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.208+0000 7f33288f7640 1 --2- 172.21.15.32:0/922697276 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33100039e0 0x7f3310003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:52.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.208+0000 7f33288f7640 1 -- 172.21.15.32:0/922697276 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3310003520 con 0x7f3310004d40 2024-04-24T22:59:52.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.209+0000 7f33288f7640 1 --2- 172.21.15.32:0/922697276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3310004d40 0x7f33100051c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f332406bdf0 tx=0x7f3310005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c056c0df7dc9333d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:52.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.209+0000 7f3322ffd640 1 -- 172.21.15.32:0/922697276 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331c037070 con 0x7f3310004d40 2024-04-24T22:59:52.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.209+0000 7f3322ffd640 1 -- 172.21.15.32:0/922697276 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f331c032440 con 0x7f3310004d40 2024-04-24T22:59:52.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.209+0000 7f3322ffd640 1 -- 172.21.15.32:0/922697276 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331c001b30 con 0x7f3310004d40 2024-04-24T22:59:52.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.210+0000 7f332a35a640 1 -- 172.21.15.32:0/922697276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3310004d40 msgr2=0x7f33100051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:52.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.210+0000 7f332a35a640 1 --2- 172.21.15.32:0/922697276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3310004d40 0x7f33100051c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f332406bdf0 tx=0x7f3310005830 comp rx=0 tx=0).stop 2024-04-24T22:59:52.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.210+0000 7f33288f7640 1 -- 172.21.15.32:0/922697276 reap_dead start 2024-04-24T22:59:52.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.210+0000 7f332a35a640 1 -- 172.21.15.32:0/922697276 shutdown_connections 2024-04-24T22:59:52.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.210+0000 7f332a35a640 1 -- 172.21.15.32:0/922697276 >> 172.21.15.32:0/922697276 conn(0x7f33240d11d0 msgr2=0x7f33240d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:52.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.210+0000 7f332a35a640 1 -- 172.21.15.32:0/922697276 shutdown_connections 2024-04-24T22:59:52.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.210+0000 7f332a35a640 1 -- 172.21.15.32:0/922697276 wait complete. 2024-04-24T22:59:52.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.211+0000 7f332a35a640 1 Processor -- start 2024-04-24T22:59:52.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.211+0000 7f332a35a640 1 -- start start 2024-04-24T22:59:52.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.212+0000 7f332a35a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3310004d40 0x7f33240ce380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:52.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.212+0000 7f332a35a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3324158b60 0x7f332414da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:52.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.212+0000 7f332a35a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33240cc9b0 0x7f33240cce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:52.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.212+0000 7f332a35a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f33240ba0e0 con 0x7f3324158b60 2024-04-24T22:59:52.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.212+0000 7f332a35a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f33240d2400 con 0x7f33240cc9b0 2024-04-24T22:59:52.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.212+0000 7f332a35a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f33240ba260 con 0x7f3310004d40 2024-04-24T22:59:52.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.212+0000 7f33237fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3324158b60 0x7f332414da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:52.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.212+0000 7f33237fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3324158b60 0x7f332414da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50804/0 (socket says 172.21.15.32:50804) 2024-04-24T22:59:52.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.212+0000 7f33237fe640 1 -- 172.21.15.32:0/19861770 learned_addr learned my addr 172.21.15.32:0/19861770 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:52.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.212+0000 7f33288f7640 1 --2- 172.21.15.32:0/19861770 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33240cc9b0 0x7f33240cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:52.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.212+0000 7f3323fff640 1 --2- 172.21.15.32:0/19861770 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3310004d40 0x7f33240ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:52.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.212+0000 7f33237fe640 1 -- 172.21.15.32:0/19861770 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3310004d40 msgr2=0x7f33240ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:52.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.212+0000 7f33237fe640 1 --2- 172.21.15.32:0/19861770 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3310004d40 0x7f33240ce380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:52.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.212+0000 7f33237fe640 1 -- 172.21.15.32:0/19861770 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33240cc9b0 msgr2=0x7f33240cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:52.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.212+0000 7f33237fe640 1 --2- 172.21.15.32:0/19861770 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33240cc9b0 0x7f33240cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:52.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.212+0000 7f33237fe640 1 -- 172.21.15.32:0/19861770 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33240cb950 con 0x7f3324158b60 2024-04-24T22:59:52.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.213+0000 7f33237fe640 1 --2- 172.21.15.32:0/19861770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3324158b60 0x7f332414da50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f330c007340 tx=0x7f330c007740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:52.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.213+0000 7f3320ff9640 1 -- 172.21.15.32:0/19861770 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330c00f8e0 con 0x7f3324158b60 2024-04-24T22:59:52.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.213+0000 7f3320ff9640 1 -- 172.21.15.32:0/19861770 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f330c018070 con 0x7f3324158b60 2024-04-24T22:59:52.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.213+0000 7f332a35a640 1 -- 172.21.15.32:0/19861770 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33240ce8c0 con 0x7f3324158b60 2024-04-24T22:59:52.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.213+0000 7f3320ff9640 1 -- 172.21.15.32:0/19861770 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330c00fcc0 con 0x7f3324158b60 2024-04-24T22:59:52.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.213+0000 7f332a35a640 1 -- 172.21.15.32:0/19861770 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33240cbed0 con 0x7f3324158b60 2024-04-24T22:59:52.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.214+0000 7f332a35a640 1 -- 172.21.15.32:0/19861770 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3324068430 con 0x7f3324158b60 2024-04-24T22:59:52.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.214+0000 7f3320ff9640 1 -- 172.21.15.32:0/19861770 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f330c01d460 con 0x7f3324158b60 2024-04-24T22:59:52.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.215+0000 7f3320ff9640 1 --2- 172.21.15.32:0/19861770 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f330406be60 0x7f330406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:52.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.215+0000 7f3320ff9640 1 -- 172.21.15.32:0/19861770 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f330c08a9b0 con 0x7f3324158b60 2024-04-24T22:59:52.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.215+0000 7f3323fff640 1 --2- 172.21.15.32:0/19861770 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f330406be60 0x7f330406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:52.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.215+0000 7f3323fff640 1 --2- 172.21.15.32:0/19861770 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f330406be60 0x7f330406e320 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3314009bc0 tx=0x7f3314009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:52.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.219+0000 7f3320ff9640 1 -- 172.21.15.32:0/19861770 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f330c00fe60 con 0x7f3324158b60 2024-04-24T22:59:52.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.470+0000 7f332a35a640 1 -- 172.21.15.32:0/19861770 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f3324072210 con 0x7f3324158b60 2024-04-24T22:59:52.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.471+0000 7f3320ff9640 1 -- 172.21.15.32:0/19861770 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) ==== 80+0+6645 (secure 0 0 0) 0x7f330c059740 con 0x7f3324158b60 2024-04-24T22:59:52.472 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:59:52.472 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T22:59:52.472 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2024-04-24T22:59:52.472 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T22:59:52.472 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2024-04-24T22:59:52.473 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmXRdxAxAA/EYEn1C38XhG7DC+0X4y7A==", 2024-04-24T22:59:52.473 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T22:59:52.473 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T22:59:52.473 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-24T22:59:52.473 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-24T22:59:52.473 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T22:59:52.473 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T22:59:52.473 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T22:59:52.473 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T22:59:52.473 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2024-04-24T22:59:52.473 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmak7bDhAA0he+Jp/gkkrfBjYKSObjcw==", 2024-04-24T22:59:52.473 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T22:59:52.473 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T22:59:52.473 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-24T22:59:52.474 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-24T22:59:52.474 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T22:59:52.474 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T22:59:52.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T22:59:52.474 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T22:59:52.474 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.c", 2024-04-24T22:59:52.474 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmZXFdChAA8TwwkA5EIXHBzPsQSFvXcQ==", 2024-04-24T22:59:52.474 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T22:59:52.474 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T22:59:52.474 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-24T22:59:52.474 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-24T22:59:52.474 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T22:59:52.474 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T22:59:52.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T22:59:52.475 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T22:59:52.475 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.d", 2024-04-24T22:59:52.475 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmLbGuFRAAe91fZlulziWdWiDdLBLWNA==", 2024-04-24T22:59:52.475 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T22:59:52.475 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T22:59:52.475 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-24T22:59:52.475 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-24T22:59:52.475 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T22:59:52.475 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T22:59:52.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T22:59:52.475 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T22:59:52.475 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2024-04-24T22:59:52.475 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCfjilmhTwoOBAAig98xa3a2qnv4rriUO9VCQ==", 2024-04-24T22:59:52.475 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T22:59:52.475 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T22:59:52.476 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T22:59:52.476 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T22:59:52.476 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T22:59:52.476 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T22:59:52.476 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T22:59:52.476 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2024-04-24T22:59:52.476 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCgjilmwRaAAxAAUqDOyR9HV6DEnmbyn3WAjA==", 2024-04-24T22:59:52.476 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T22:59:52.476 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T22:59:52.476 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T22:59:52.476 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T22:59:52.476 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T22:59:52.476 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T22:59:52.476 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T22:59:52.477 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2024-04-24T22:59:52.477 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCgjilmDl2qDBAAF1bhTOaSOc612wmFRd8+XA==", 2024-04-24T22:59:52.477 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T22:59:52.477 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T22:59:52.477 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T22:59:52.477 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T22:59:52.477 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T22:59:52.477 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T22:59:52.477 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T22:59:52.477 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2024-04-24T22:59:52.477 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCgjilmAxHwFRAA9a0v/YwngRxq1vmMRsmEkQ==", 2024-04-24T22:59:52.477 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T22:59:52.477 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T22:59:52.477 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T22:59:52.478 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T22:59:52.478 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T22:59:52.478 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T22:59:52.478 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T22:59:52.478 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2024-04-24T22:59:52.478 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCjjilmDPmJIxAASZlEo1ZJHOhIAFGukIj1tQ==", 2024-04-24T22:59:52.478 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T22:59:52.478 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T22:59:52.478 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T22:59:52.478 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T22:59:52.478 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T22:59:52.478 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T22:59:52.478 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T22:59:52.478 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2024-04-24T22:59:52.479 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCjjilmgcpuKhAAedSq0ubNB30s3bz0fY38hw==", 2024-04-24T22:59:52.479 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T22:59:52.479 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T22:59:52.479 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T22:59:52.479 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T22:59:52.479 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T22:59:52.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T22:59:52.479 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T22:59:52.479 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2024-04-24T22:59:52.479 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCjjilmFv2VMxAAM+mUc4I0sjawy2uwuUEKYg==", 2024-04-24T22:59:52.479 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T22:59:52.479 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T22:59:52.479 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T22:59:52.479 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T22:59:52.479 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T22:59:52.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T22:59:52.480 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T22:59:52.480 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2024-04-24T22:59:52.480 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCkjilmLvEbARAAnaOvQ5Ee0clN/A9SVoKuBQ==", 2024-04-24T22:59:52.480 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T22:59:52.480 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T22:59:52.480 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T22:59:52.480 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T22:59:52.480 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T22:59:52.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T22:59:52.480 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T22:59:52.480 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2024-04-24T22:59:52.480 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilm5oeiGRAAdLXwwqrYhV3HHo7Dw/2ZqQ==", 2024-04-24T22:59:52.480 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T22:59:52.480 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T22:59:52.481 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-24T22:59:52.481 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-24T22:59:52.481 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-24T22:59:52.481 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T22:59:52.481 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T22:59:52.481 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T22:59:52.481 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2024-04-24T22:59:52.481 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmyWBDIhAARtVP4Unk3HHlQYZ2QZ3GzQ==", 2024-04-24T22:59:52.481 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T22:59:52.481 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T22:59:52.481 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-24T22:59:52.481 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-24T22:59:52.481 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-24T22:59:52.481 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T22:59:52.481 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T22:59:52.482 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T22:59:52.482 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.2", 2024-04-24T22:59:52.482 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilm4QElHxAAXgE316aRCosxX7XER5gtKQ==", 2024-04-24T22:59:52.482 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T22:59:52.482 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T22:59:52.482 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-24T22:59:52.482 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-24T22:59:52.482 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-24T22:59:52.482 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T22:59:52.482 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T22:59:52.482 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T22:59:52.482 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.3", 2024-04-24T22:59:52.482 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmMZihJxAAoWaAAFY3vVfZnC4/GoT0+g==", 2024-04-24T22:59:52.482 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T22:59:52.483 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T22:59:52.483 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-24T22:59:52.483 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-24T22:59:52.483 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-24T22:59:52.483 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T22:59:52.483 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T22:59:52.483 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T22:59:52.483 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2024-04-24T22:59:52.483 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCdjilmdVXeIxAAwGGaAfshx4pHcoc5+3kRww==", 2024-04-24T22:59:52.483 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T22:59:52.483 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2024-04-24T22:59:52.483 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-24T22:59:52.483 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-24T22:59:52.483 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T22:59:52.483 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T22:59:52.484 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T22:59:52.484 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T22:59:52.484 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.bootstrap-mds", 2024-04-24T22:59:52.484 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCmjilmv1yJORAAMhGaVdADBhoJizRF5PWHQA==", 2024-04-24T22:59:52.484 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T22:59:52.484 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile bootstrap-mds" 2024-04-24T22:59:52.484 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T22:59:52.484 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T22:59:52.484 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T22:59:52.484 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.bootstrap-mgr", 2024-04-24T22:59:52.484 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCmjilmcmWJORAAvVxCi/+9HcvH88sUraw71Q==", 2024-04-24T22:59:52.484 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T22:59:52.484 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-24T22:59:52.484 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T22:59:52.484 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T22:59:52.485 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T22:59:52.485 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.bootstrap-osd", 2024-04-24T22:59:52.485 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCmjilmjWyJORAADL0MpgS9b5+9OrCpmKhr4g==", 2024-04-24T22:59:52.485 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T22:59:52.485 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile bootstrap-osd" 2024-04-24T22:59:52.485 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T22:59:52.485 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T22:59:52.485 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T22:59:52.485 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.bootstrap-rbd", 2024-04-24T22:59:52.485 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCmjilmv3OJORAAp7LyS1U6muSpn2hWj8rerQ==", 2024-04-24T22:59:52.485 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T22:59:52.485 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-24T22:59:52.485 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T22:59:52.485 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T22:59:52.485 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T22:59:52.486 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-24T22:59:52.486 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCmjilmFXuJORAA+rl4XEFvrKzx61l1H0bKHA==", 2024-04-24T22:59:52.486 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T22:59:52.486 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-24T22:59:52.486 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T22:59:52.486 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T22:59:52.486 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T22:59:52.486 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.bootstrap-rgw", 2024-04-24T22:59:52.486 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCmjilmBISJORAAHm2I0Deo59ZDTAV4cD0l8A==", 2024-04-24T22:59:52.486 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T22:59:52.486 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-24T22:59:52.486 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T22:59:52.486 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T22:59:52.486 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T22:59:52.486 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2024-04-24T22:59:52.487 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmn3FJABAAGnvNAoNLzK20cQF6beihmw==", 2024-04-24T22:59:52.487 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T22:59:52.487 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2024-04-24T22:59:52.487 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2024-04-24T22:59:52.487 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T22:59:52.487 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T22:59:52.487 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T22:59:52.487 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T22:59:52.487 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2024-04-24T22:59:52.487 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCdjilmMqs2OBAAOe3bLpnCq+HK2EbeXnwY8Q==", 2024-04-24T22:59:52.487 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T22:59:52.487 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2024-04-24T22:59:52.487 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2024-04-24T22:59:52.487 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T22:59:52.487 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T22:59:52.488 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T22:59:52.488 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T22:59:52.488 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T22:59:52.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.473+0000 7f332a35a640 1 -- 172.21.15.32:0/19861770 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f330406be60 msgr2=0x7f330406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:52.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.473+0000 7f332a35a640 1 --2- 172.21.15.32:0/19861770 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f330406be60 0x7f330406e320 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3314009bc0 tx=0x7f3314009190 comp rx=0 tx=0).stop 2024-04-24T22:59:52.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.473+0000 7f332a35a640 1 -- 172.21.15.32:0/19861770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3324158b60 msgr2=0x7f332414da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:52.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.473+0000 7f332a35a640 1 --2- 172.21.15.32:0/19861770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3324158b60 0x7f332414da50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f330c007340 tx=0x7f330c007740 comp rx=0 tx=0).stop 2024-04-24T22:59:52.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.473+0000 7f332a35a640 1 -- 172.21.15.32:0/19861770 shutdown_connections 2024-04-24T22:59:52.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.473+0000 7f332a35a640 1 --2- 172.21.15.32:0/19861770 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f330406be60 0x7f330406e320 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:52.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.473+0000 7f332a35a640 1 --2- 172.21.15.32:0/19861770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3324158b60 0x7f332414da50 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:52.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.473+0000 7f332a35a640 1 --2- 172.21.15.32:0/19861770 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33240cc9b0 0x7f33240cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:52.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.473+0000 7f332a35a640 1 --2- 172.21.15.32:0/19861770 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3310004d40 0x7f33240ce380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:52.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.473+0000 7f332a35a640 1 -- 172.21.15.32:0/19861770 >> 172.21.15.32:0/19861770 conn(0x7f33240d11d0 msgr2=0x7f332415c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:52.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.473+0000 7f332a35a640 1 -- 172.21.15.32:0/19861770 shutdown_connections 2024-04-24T22:59:52.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.473+0000 7f332a35a640 1 -- 172.21.15.32:0/19861770 wait complete. 2024-04-24T22:59:52.490 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-24T22:59:52.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.590+0000 7f19a1b09640 1 Processor -- start 2024-04-24T22:59:52.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.591+0000 7f19a1b09640 1 -- start start 2024-04-24T22:59:52.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.591+0000 7f19a1b09640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f199c15e9b0 0x7f199c0d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:52.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.591+0000 7f19a1b09640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f199c0d3170 con 0x7f199c0d4210 2024-04-24T22:59:52.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.591+0000 7f19a1b09640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f199c0d3450 con 0x7f199c06b820 2024-04-24T22:59:52.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.591+0000 7f19a1b09640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f199c0d3730 con 0x7f199c15e9b0 2024-04-24T22:59:52.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.591+0000 7f199b7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f199c15e9b0 0x7f199c0d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:52.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.591+0000 7f199affd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f199c0d4210 0x7f199c06b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53872/0 (socket says 172.21.15.32:53872) 2024-04-24T22:59:52.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.591+0000 7f199affd640 1 -- 172.21.15.32:0/2084700908 learned_addr learned my addr 172.21.15.32:0/2084700908 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:52.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.592+0000 7f199b7fe640 1 -- 172.21.15.32:0/2084700908 >> v1:172.21.15.121:6789/0 conn(0x7f199c06b820 legacy=0x7f199c0b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:52.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.592+0000 7f199b7fe640 1 -- 172.21.15.32:0/2084700908 >> v1:172.21.15.32:6789/0 conn(0x7f199c0d4210 legacy=0x7f199c06b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:52.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.592+0000 7f199b7fe640 1 -- 172.21.15.32:0/2084700908 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f199c0d3a10 con 0x7f199c15e9b0 2024-04-24T22:59:52.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.592+0000 7f199a7fc640 1 -- 172.21.15.32:0/2084700908 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3523096381 0 0) 0x7f199c0d3170 con 0x7f199c0d4210 2024-04-24T22:59:52.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.593+0000 7f199b7fe640 1 --2- 172.21.15.32:0/2084700908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f199c15e9b0 0x7f199c0d2b90 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f198c0097a0 tx=0x7f198c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2aa87da68d2d91e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:52.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.593+0000 7f199a7fc640 1 -- 172.21.15.32:0/2084700908 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198c037070 con 0x7f199c15e9b0 2024-04-24T22:59:52.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.593+0000 7f199a7fc640 1 -- 172.21.15.32:0/2084700908 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f198c02ae50 con 0x7f199c15e9b0 2024-04-24T22:59:52.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.593+0000 7f199a7fc640 1 -- 172.21.15.32:0/2084700908 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198c032660 con 0x7f199c15e9b0 2024-04-24T22:59:52.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.593+0000 7f19a1b09640 1 -- 172.21.15.32:0/2084700908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f199c15e9b0 msgr2=0x7f199c0d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:52.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.593+0000 7f19a1b09640 1 --2- 172.21.15.32:0/2084700908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f199c15e9b0 0x7f199c0d2b90 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f198c0097a0 tx=0x7f198c02a140 comp rx=0 tx=0).stop 2024-04-24T22:59:52.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.593+0000 7f19a1b09640 1 -- 172.21.15.32:0/2084700908 shutdown_connections 2024-04-24T22:59:52.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.593+0000 7f19a1b09640 1 --2- 172.21.15.32:0/2084700908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f199c15e9b0 0x7f199c0d2b90 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:52.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.593+0000 7f19a1b09640 1 -- 172.21.15.32:0/2084700908 >> 172.21.15.32:0/2084700908 conn(0x7f199c0d11d0 msgr2=0x7f199c0d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:52.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.593+0000 7f19a1b09640 1 -- 172.21.15.32:0/2084700908 shutdown_connections 2024-04-24T22:59:52.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.593+0000 7f19a1b09640 1 -- 172.21.15.32:0/2084700908 wait complete. 2024-04-24T22:59:52.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.594+0000 7f19a1b09640 1 Processor -- start 2024-04-24T22:59:52.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.595+0000 7f19a1b09640 1 -- start start 2024-04-24T22:59:52.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.595+0000 7f19a1b09640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f199c06b820 0x7f199c0cb820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:52.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.595+0000 7f19a1b09640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f199c0d4210 0x7f199c153f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:52.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.595+0000 7f19a1b09640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f199c15e9b0 0x7f199c15d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:52.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.595+0000 7f19a1b09640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f199c0ba0e0 con 0x7f199c06b820 2024-04-24T22:59:52.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.595+0000 7f19a1b09640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f199c0d2400 con 0x7f199c0d4210 2024-04-24T22:59:52.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.595+0000 7f19a1b09640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f199c0ba260 con 0x7f199c15e9b0 2024-04-24T22:59:52.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.595+0000 7f199b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f199c06b820 0x7f199c0cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:52.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.595+0000 7f199b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f199c06b820 0x7f199c0cb820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50824/0 (socket says 172.21.15.32:50824) 2024-04-24T22:59:52.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.595+0000 7f199b7fe640 1 -- 172.21.15.32:0/2105352319 learned_addr learned my addr 172.21.15.32:0/2105352319 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:52.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.596+0000 7f199affd640 1 --2- 172.21.15.32:0/2105352319 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f199c0d4210 0x7f199c153f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:52.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.596+0000 7f199bfff640 1 --2- 172.21.15.32:0/2105352319 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f199c15e9b0 0x7f199c15d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:52.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.596+0000 7f199b7fe640 1 -- 172.21.15.32:0/2105352319 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f199c15e9b0 msgr2=0x7f199c15d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:52.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.596+0000 7f199b7fe640 1 --2- 172.21.15.32:0/2105352319 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f199c15e9b0 0x7f199c15d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:52.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.596+0000 7f199b7fe640 1 -- 172.21.15.32:0/2105352319 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f199c0d4210 msgr2=0x7f199c153f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:52.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.596+0000 7f199b7fe640 1 --2- 172.21.15.32:0/2105352319 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f199c0d4210 0x7f199c153f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:52.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.596+0000 7f199b7fe640 1 -- 172.21.15.32:0/2105352319 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f199c1544d0 con 0x7f199c06b820 2024-04-24T22:59:52.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.596+0000 7f199affd640 1 --2- 172.21.15.32:0/2105352319 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f199c0d4210 0x7f199c153f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:52.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.596+0000 7f199b7fe640 1 --2- 172.21.15.32:0/2105352319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f199c06b820 0x7f199c0cb820 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f198c0098c0 tx=0x7f198c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:52.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.596+0000 7f1983fff640 1 -- 172.21.15.32:0/2105352319 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198c042070 con 0x7f199c06b820 2024-04-24T22:59:52.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.596+0000 7f1983fff640 1 -- 172.21.15.32:0/2105352319 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f198c03e670 con 0x7f199c06b820 2024-04-24T22:59:52.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.596+0000 7f19a1b09640 1 -- 172.21.15.32:0/2105352319 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f199c1f7c40 con 0x7f199c06b820 2024-04-24T22:59:52.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.596+0000 7f1983fff640 1 -- 172.21.15.32:0/2105352319 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198c037040 con 0x7f199c06b820 2024-04-24T22:59:52.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.596+0000 7f19a1b09640 1 -- 172.21.15.32:0/2105352319 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f199c1f8200 con 0x7f199c06b820 2024-04-24T22:59:52.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.597+0000 7f19a1b09640 1 -- 172.21.15.32:0/2105352319 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f199c068430 con 0x7f199c06b820 2024-04-24T22:59:52.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.598+0000 7f1983fff640 1 -- 172.21.15.32:0/2105352319 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f198c045070 con 0x7f199c06b820 2024-04-24T22:59:52.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.598+0000 7f1983fff640 1 --2- 172.21.15.32:0/2105352319 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f198806be60 0x7f198806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:52.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.598+0000 7f1983fff640 1 -- 172.21.15.32:0/2105352319 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f198c0ab510 con 0x7f199c06b820 2024-04-24T22:59:52.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.598+0000 7f199affd640 1 --2- 172.21.15.32:0/2105352319 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f198806be60 0x7f198806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:52.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.599+0000 7f199affd640 1 --2- 172.21.15.32:0/2105352319 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f198806be60 0x7f198806e320 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1990000f60 tx=0x7f1990007c10 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:52.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.602+0000 7f1983fff640 1 -- 172.21.15.32:0/2105352319 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f198c07a330 con 0x7f199c06b820 2024-04-24T22:59:52.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.860+0000 7f19a1b09640 1 -- 172.21.15.32:0/2105352319 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f199c1f88c0 con 0x7f199c06b820 2024-04-24T22:59:52.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.862+0000 7f1983fff640 1 -- 172.21.15.32:0/2105352319 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7f198c0ab070 con 0x7f199c06b820 2024-04-24T22:59:52.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.864+0000 7f19a1b09640 1 -- 172.21.15.32:0/2105352319 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f198806be60 msgr2=0x7f198806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:52.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.864+0000 7f19a1b09640 1 --2- 172.21.15.32:0/2105352319 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f198806be60 0x7f198806e320 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1990000f60 tx=0x7f1990007c10 comp rx=0 tx=0).stop 2024-04-24T22:59:52.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.864+0000 7f19a1b09640 1 -- 172.21.15.32:0/2105352319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f199c06b820 msgr2=0x7f199c0cb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:52.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.864+0000 7f19a1b09640 1 --2- 172.21.15.32:0/2105352319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f199c06b820 0x7f199c0cb820 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f198c0098c0 tx=0x7f198c0046c0 comp rx=0 tx=0).stop 2024-04-24T22:59:52.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.864+0000 7f19a1b09640 1 -- 172.21.15.32:0/2105352319 shutdown_connections 2024-04-24T22:59:52.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.864+0000 7f19a1b09640 1 --2- 172.21.15.32:0/2105352319 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f198806be60 0x7f198806e320 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:52.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.864+0000 7f19a1b09640 1 --2- 172.21.15.32:0/2105352319 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f199c15e9b0 0x7f199c15d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:52.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.864+0000 7f19a1b09640 1 --2- 172.21.15.32:0/2105352319 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f199c0d4210 0x7f199c153f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:52.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.864+0000 7f19a1b09640 1 --2- 172.21.15.32:0/2105352319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f199c06b820 0x7f199c0cb820 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:52.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.864+0000 7f19a1b09640 1 -- 172.21.15.32:0/2105352319 >> 172.21.15.32:0/2105352319 conn(0x7f199c0d11d0 msgr2=0x7f199c0ca290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:52.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.865+0000 7f19a1b09640 1 -- 172.21.15.32:0/2105352319 shutdown_connections 2024-04-24T22:59:52.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.865+0000 7f19a1b09640 1 -- 172.21.15.32:0/2105352319 wait complete. 2024-04-24T22:59:52.877 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-24T22:59:52.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.978+0000 7f844bcee640 1 Processor -- start 2024-04-24T22:59:52.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.978+0000 7f844bcee640 1 -- start start 2024-04-24T22:59:52.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.978+0000 7f844bcee640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8444158dd0 0x7f844406b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:52.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.979+0000 7f844bcee640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f844406dc60 0x7f84440b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:52.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.979+0000 7f844bcee640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f844406bd10 con 0x7f844406b860 2024-04-24T22:59:52.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.979+0000 7f844bcee640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f844415ef10 con 0x7f844406dc60 2024-04-24T22:59:52.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.979+0000 7f844bcee640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f844415f1f0 con 0x7f8444158dd0 2024-04-24T22:59:52.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.979+0000 7f844a28b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f844406dc60 0x7f84440b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:52.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.979+0000 7f844a28b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f844406dc60 0x7f84440b5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50830/0 (socket says 172.21.15.32:50830) 2024-04-24T22:59:52.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.979+0000 7f844a28b640 1 -- 172.21.15.32:0/3979016486 learned_addr learned my addr 172.21.15.32:0/3979016486 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:52.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.979+0000 7f8449a8a640 1 --2- 172.21.15.32:0/3979016486 >> v2:172.21.15.121:3300/0 conn(0x7f8444158dd0 0x7f844406b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:52.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.979+0000 7f844a28b640 1 -- 172.21.15.32:0/3979016486 >> v2:172.21.15.121:3300/0 conn(0x7f8444158dd0 msgr2=0x7f844406b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:52.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.979+0000 7f844a28b640 1 --2- 172.21.15.32:0/3979016486 >> v2:172.21.15.121:3300/0 conn(0x7f8444158dd0 0x7f844406b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:52.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.979+0000 7f844a28b640 1 -- 172.21.15.32:0/3979016486 >> v1:172.21.15.121:6789/0 conn(0x7f844406b860 legacy=0x7f844406d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:52.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.979+0000 7f844a28b640 1 -- 172.21.15.32:0/3979016486 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f844415f4d0 con 0x7f844406dc60 2024-04-24T22:59:52.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.979+0000 7f8449a8a640 1 --2- 172.21.15.32:0/3979016486 >> v2:172.21.15.121:3300/0 conn(0x7f8444158dd0 0x7f844406b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:59:52.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.980+0000 7f844a28b640 1 --2- 172.21.15.32:0/3979016486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f844406dc60 0x7f84440b5d00 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f844406f3a0 tx=0x7f843c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2585375503d3275b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:52.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.980+0000 7f8448a88640 1 -- 172.21.15.32:0/3979016486 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843c036070 con 0x7f844406dc60 2024-04-24T22:59:52.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.980+0000 7f8448a88640 1 -- 172.21.15.32:0/3979016486 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f843c031600 con 0x7f844406dc60 2024-04-24T22:59:52.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.980+0000 7f8448a88640 1 -- 172.21.15.32:0/3979016486 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843c0319d0 con 0x7f844406dc60 2024-04-24T22:59:52.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.980+0000 7f844bcee640 1 -- 172.21.15.32:0/3979016486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f844406dc60 msgr2=0x7f84440b5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:52.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.980+0000 7f844bcee640 1 --2- 172.21.15.32:0/3979016486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f844406dc60 0x7f84440b5d00 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f844406f3a0 tx=0x7f843c02a6f0 comp rx=0 tx=0).stop 2024-04-24T22:59:52.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.980+0000 7f844bcee640 1 -- 172.21.15.32:0/3979016486 shutdown_connections 2024-04-24T22:59:52.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.980+0000 7f844bcee640 1 --2- 172.21.15.32:0/3979016486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f844406dc60 0x7f84440b5d00 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:52.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.980+0000 7f844bcee640 1 --2- 172.21.15.32:0/3979016486 >> v2:172.21.15.121:3300/0 conn(0x7f8444158dd0 0x7f844406b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:52.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.980+0000 7f844bcee640 1 -- 172.21.15.32:0/3979016486 >> 172.21.15.32:0/3979016486 conn(0x7f844406e800 msgr2=0x7f844406ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:52.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.981+0000 7f844bcee640 1 -- 172.21.15.32:0/3979016486 shutdown_connections 2024-04-24T22:59:52.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.981+0000 7f844bcee640 1 -- 172.21.15.32:0/3979016486 wait complete. 2024-04-24T22:59:52.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.982+0000 7f844bcee640 1 Processor -- start 2024-04-24T22:59:52.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.982+0000 7f844bcee640 1 -- start start 2024-04-24T22:59:52.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.982+0000 7f844bcee640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f844406b860 0x7f84441508d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:52.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.982+0000 7f844bcee640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f844406dc60 0x7f84441577b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:52.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.982+0000 7f844bcee640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8444158dd0 0x7f84441511d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:52.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.982+0000 7f844bcee640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f844406bfe0 con 0x7f844406dc60 2024-04-24T22:59:52.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.982+0000 7f844bcee640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f844415fcd0 con 0x7f8444158dd0 2024-04-24T22:59:52.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.982+0000 7f844bcee640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f84441591b0 con 0x7f844406b860 2024-04-24T22:59:52.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.983+0000 7f8449289640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f844406dc60 0x7f84441577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:52.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.983+0000 7f8449289640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f844406dc60 0x7f84441577b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50846/0 (socket says 172.21.15.32:50846) 2024-04-24T22:59:52.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.983+0000 7f8449289640 1 -- 172.21.15.32:0/1313659510 learned_addr learned my addr 172.21.15.32:0/1313659510 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:52.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.983+0000 7f8449a8a640 1 --2- 172.21.15.32:0/1313659510 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f844406b860 0x7f84441508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:52.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.983+0000 7f844a28b640 1 --2- 172.21.15.32:0/1313659510 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8444158dd0 0x7f84441511d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:52.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.983+0000 7f8449289640 1 -- 172.21.15.32:0/1313659510 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f844406b860 msgr2=0x7f84441508d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:52.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.983+0000 7f8449289640 1 --2- 172.21.15.32:0/1313659510 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f844406b860 0x7f84441508d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:52.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.983+0000 7f8449289640 1 -- 172.21.15.32:0/1313659510 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8444158dd0 msgr2=0x7f84441511d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:52.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.983+0000 7f8449289640 1 --2- 172.21.15.32:0/1313659510 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8444158dd0 0x7f84441511d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:52.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.983+0000 7f8449289640 1 -- 172.21.15.32:0/1313659510 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f844414fb10 con 0x7f844406dc60 2024-04-24T22:59:52.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.983+0000 7f8449289640 1 --2- 172.21.15.32:0/1313659510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f844406dc60 0x7f84441577b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f843400b420 tx=0x7f843400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:52.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.983+0000 7f843a7fc640 1 -- 172.21.15.32:0/1313659510 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8434011830 con 0x7f844406dc60 2024-04-24T22:59:52.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.983+0000 7f843a7fc640 1 -- 172.21.15.32:0/1313659510 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8434016070 con 0x7f844406dc60 2024-04-24T22:59:52.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.983+0000 7f844bcee640 1 -- 172.21.15.32:0/1313659510 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f844414fdf0 con 0x7f844406dc60 2024-04-24T22:59:52.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.983+0000 7f843a7fc640 1 -- 172.21.15.32:0/1313659510 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8434011c10 con 0x7f844406dc60 2024-04-24T22:59:52.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.983+0000 7f844bcee640 1 -- 172.21.15.32:0/1313659510 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84441503b0 con 0x7f844406dc60 2024-04-24T22:59:52.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.984+0000 7f844bcee640 1 -- 172.21.15.32:0/1313659510 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8444068430 con 0x7f844406dc60 2024-04-24T22:59:52.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.985+0000 7f843a7fc640 1 -- 172.21.15.32:0/1313659510 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f843401b460 con 0x7f844406dc60 2024-04-24T22:59:52.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.985+0000 7f843a7fc640 1 --2- 172.21.15.32:0/1313659510 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f842806be30 0x7f842806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:52.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.985+0000 7f843a7fc640 1 -- 172.21.15.32:0/1313659510 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f843408a680 con 0x7f844406dc60 2024-04-24T22:59:52.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.985+0000 7f8449a8a640 1 --2- 172.21.15.32:0/1313659510 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f842806be30 0x7f842806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:52.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.986+0000 7f8449a8a640 1 --2- 172.21.15.32:0/1313659510 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f842806be30 0x7f842806e2f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f844000ab60 tx=0x7f8440008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:52.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:52.989+0000 7f843a7fc640 1 -- 172.21.15.32:0/1313659510 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8434058dc0 con 0x7f844406dc60 2024-04-24T22:59:53.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.249+0000 7f844bcee640 1 -- 172.21.15.32:0/1313659510 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f8444067510 con 0x7f844406dc60 2024-04-24T22:59:53.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.251+0000 7f843a7fc640 1 -- 172.21.15.32:0/1313659510 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f843405d0c0 con 0x7f844406dc60 2024-04-24T22:59:53.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.253+0000 7f844bcee640 1 -- 172.21.15.32:0/1313659510 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f842806be30 msgr2=0x7f842806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:53.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.253+0000 7f844bcee640 1 --2- 172.21.15.32:0/1313659510 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f842806be30 0x7f842806e2f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f844000ab60 tx=0x7f8440008040 comp rx=0 tx=0).stop 2024-04-24T22:59:53.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.253+0000 7f844bcee640 1 -- 172.21.15.32:0/1313659510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f844406dc60 msgr2=0x7f84441577b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:53.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.253+0000 7f844bcee640 1 --2- 172.21.15.32:0/1313659510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f844406dc60 0x7f84441577b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f843400b420 tx=0x7f843400b8e0 comp rx=0 tx=0).stop 2024-04-24T22:59:53.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.253+0000 7f844bcee640 1 -- 172.21.15.32:0/1313659510 shutdown_connections 2024-04-24T22:59:53.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.253+0000 7f844bcee640 1 --2- 172.21.15.32:0/1313659510 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f842806be30 0x7f842806e2f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:53.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.253+0000 7f844bcee640 1 --2- 172.21.15.32:0/1313659510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f844406dc60 0x7f84441577b0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:53.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.253+0000 7f844bcee640 1 --2- 172.21.15.32:0/1313659510 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8444158dd0 0x7f84441511d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:53.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.253+0000 7f844bcee640 1 --2- 172.21.15.32:0/1313659510 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f844406b860 0x7f84441508d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:53.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.253+0000 7f844bcee640 1 -- 172.21.15.32:0/1313659510 >> 172.21.15.32:0/1313659510 conn(0x7f844406e800 msgr2=0x7f8444160860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:53.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.253+0000 7f844bcee640 1 -- 172.21.15.32:0/1313659510 shutdown_connections 2024-04-24T22:59:53.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.253+0000 7f844bcee640 1 -- 172.21.15.32:0/1313659510 wait complete. 2024-04-24T22:59:53.265 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-24T22:59:53.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.368+0000 7fd2b09c8640 1 Processor -- start 2024-04-24T22:59:53.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.368+0000 7fd2b09c8640 1 -- start start 2024-04-24T22:59:53.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.369+0000 7fd2b09c8640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd2ac06cd00 0x7fd2ac0ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:53.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.369+0000 7fd2b09c8640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd2ac15cb00 0x7fd2ac0ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:53.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.369+0000 7fd2b09c8640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2ac0b4520 con 0x7fd2ac15b380 2024-04-24T22:59:53.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.369+0000 7fd2b09c8640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fd2ac0b4800 con 0x7fd2ac15cb00 2024-04-24T22:59:53.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.369+0000 7fd2b09c8640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd2ac0b4ae0 con 0x7fd2ac06cd00 2024-04-24T22:59:53.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.369+0000 7fd2aad9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd2ac15cb00 0x7fd2ac0ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:53.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.369+0000 7fd2aad9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd2ac15cb00 0x7fd2ac0ce970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50852/0 (socket says 172.21.15.32:50852) 2024-04-24T22:59:53.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.369+0000 7fd2aad9d640 1 -- 172.21.15.32:0/4266060147 learned_addr learned my addr 172.21.15.32:0/4266060147 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:53.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.369+0000 7fd2a3fff640 1 --2- 172.21.15.32:0/4266060147 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd2ac06cd00 0x7fd2ac0ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:53.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.369+0000 7fd2aad9d640 1 -- 172.21.15.32:0/4266060147 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd2ac06cd00 msgr2=0x7fd2ac0ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:53.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.369+0000 7fd2aad9d640 1 --2- 172.21.15.32:0/4266060147 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd2ac06cd00 0x7fd2ac0ce430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:53.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.369+0000 7fd2aad9d640 1 -- 172.21.15.32:0/4266060147 >> v1:172.21.15.121:6789/0 conn(0x7fd2ac15b380 legacy=0x7fd2ac06c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:53.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.369+0000 7fd2aad9d640 1 -- 172.21.15.32:0/4266060147 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2ac0b4dc0 con 0x7fd2ac15cb00 2024-04-24T22:59:53.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.370+0000 7fd2aad9d640 1 --2- 172.21.15.32:0/4266060147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2ac15cb00 0x7fd2ac0ce970 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4009e50 tx=0x7fd2a402a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15519346e7f280c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:53.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.370+0000 7fd2a9d9b640 1 -- 172.21.15.32:0/4266060147 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a4036070 con 0x7fd2ac15cb00 2024-04-24T22:59:53.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.370+0000 7fd2a9d9b640 1 -- 172.21.15.32:0/4266060147 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2a4031600 con 0x7fd2ac15cb00 2024-04-24T22:59:53.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.370+0000 7fd2a9d9b640 1 -- 172.21.15.32:0/4266060147 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a40319d0 con 0x7fd2ac15cb00 2024-04-24T22:59:53.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.371+0000 7fd2b09c8640 1 -- 172.21.15.32:0/4266060147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2ac15cb00 msgr2=0x7fd2ac0ce970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:53.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.371+0000 7fd2b09c8640 1 --2- 172.21.15.32:0/4266060147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2ac15cb00 0x7fd2ac0ce970 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4009e50 tx=0x7fd2a402a630 comp rx=0 tx=0).stop 2024-04-24T22:59:53.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.371+0000 7fd2b09c8640 1 -- 172.21.15.32:0/4266060147 shutdown_connections 2024-04-24T22:59:53.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.371+0000 7fd2b09c8640 1 --2- 172.21.15.32:0/4266060147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2ac15cb00 0x7fd2ac0ce970 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:53.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.371+0000 7fd2b09c8640 1 --2- 172.21.15.32:0/4266060147 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd2ac06cd00 0x7fd2ac0ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:53.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.371+0000 7fd2b09c8640 1 -- 172.21.15.32:0/4266060147 >> 172.21.15.32:0/4266060147 conn(0x7fd2ac06b110 msgr2=0x7fd2ac06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:53.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.371+0000 7fd2b09c8640 1 -- 172.21.15.32:0/4266060147 shutdown_connections 2024-04-24T22:59:53.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.371+0000 7fd2b09c8640 1 -- 172.21.15.32:0/4266060147 wait complete. 2024-04-24T22:59:53.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.372+0000 7fd2b09c8640 1 Processor -- start 2024-04-24T22:59:53.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.372+0000 7fd2b09c8640 1 -- start start 2024-04-24T22:59:53.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.373+0000 7fd2b09c8640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd2ac06cd00 0x7fd2ac1565b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:53.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.373+0000 7fd2b09c8640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd2ac15b380 0x7fd2ac13fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:53.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.373+0000 7fd2b09c8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2ac15cb00 0x7fd2ac1502b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:53.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.373+0000 7fd2b09c8640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ac15b760 con 0x7fd2ac15cb00 2024-04-24T22:59:53.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.373+0000 7fd2b09c8640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ac0ceeb0 con 0x7fd2ac06cd00 2024-04-24T22:59:53.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.373+0000 7fd2b09c8640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd2ac0b55c0 con 0x7fd2ac15b380 2024-04-24T22:59:53.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.373+0000 7fd2aad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2ac15cb00 0x7fd2ac1502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:53.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.373+0000 7fd2aad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2ac15cb00 0x7fd2ac1502b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50868/0 (socket says 172.21.15.32:50868) 2024-04-24T22:59:53.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.373+0000 7fd2aad9d640 1 -- 172.21.15.32:0/1835893498 learned_addr learned my addr 172.21.15.32:0/1835893498 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:53.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.373+0000 7fd2aa59c640 1 --2- 172.21.15.32:0/1835893498 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd2ac06cd00 0x7fd2ac1565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:53.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.373+0000 7fd2a3fff640 1 --2- 172.21.15.32:0/1835893498 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd2ac15b380 0x7fd2ac13fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:53.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.373+0000 7fd2aad9d640 1 -- 172.21.15.32:0/1835893498 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd2ac15b380 msgr2=0x7fd2ac13fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:53.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.373+0000 7fd2aad9d640 1 --2- 172.21.15.32:0/1835893498 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd2ac15b380 0x7fd2ac13fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:53.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.373+0000 7fd2aad9d640 1 -- 172.21.15.32:0/1835893498 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd2ac06cd00 msgr2=0x7fd2ac1565b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:53.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.373+0000 7fd2aad9d640 1 --2- 172.21.15.32:0/1835893498 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd2ac06cd00 0x7fd2ac1565b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:53.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.373+0000 7fd2aad9d640 1 -- 172.21.15.32:0/1835893498 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2ac14f160 con 0x7fd2ac15cb00 2024-04-24T22:59:53.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.373+0000 7fd2aad9d640 1 --2- 172.21.15.32:0/1835893498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2ac15cb00 0x7fd2ac1502b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4009f70 tx=0x7fd2a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:53.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.373+0000 7fd2a2ffd640 1 -- 172.21.15.32:0/1835893498 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a4036050 con 0x7fd2ac15cb00 2024-04-24T22:59:53.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.373+0000 7fd2a2ffd640 1 -- 172.21.15.32:0/1835893498 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2a4042070 con 0x7fd2ac15cb00 2024-04-24T22:59:53.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.374+0000 7fd2b09c8640 1 -- 172.21.15.32:0/1835893498 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2ac14f440 con 0x7fd2ac15cb00 2024-04-24T22:59:53.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.374+0000 7fd2a2ffd640 1 -- 172.21.15.32:0/1835893498 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a4009510 con 0x7fd2ac15cb00 2024-04-24T22:59:53.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.374+0000 7fd2b09c8640 1 -- 172.21.15.32:0/1835893498 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2ac14fa00 con 0x7fd2ac15cb00 2024-04-24T22:59:53.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.374+0000 7fd2b09c8640 1 -- 172.21.15.32:0/1835893498 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2ac068430 con 0x7fd2ac15cb00 2024-04-24T22:59:53.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.375+0000 7fd2a2ffd640 1 -- 172.21.15.32:0/1835893498 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd2a4007bd0 con 0x7fd2ac15cb00 2024-04-24T22:59:53.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.375+0000 7fd2a2ffd640 1 --2- 172.21.15.32:0/1835893498 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd28c06be60 0x7fd28c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:53.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.375+0000 7fd2a2ffd640 1 -- 172.21.15.32:0/1835893498 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7fd2a40aba40 con 0x7fd2ac15cb00 2024-04-24T22:59:53.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.376+0000 7fd2aa59c640 1 --2- 172.21.15.32:0/1835893498 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd28c06be60 0x7fd28c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:53.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.376+0000 7fd2aa59c640 1 --2- 172.21.15.32:0/1835893498 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd28c06be60 0x7fd28c06e320 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd29c004ea0 tx=0x7fd29c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:53.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.379+0000 7fd2a2ffd640 1 -- 172.21.15.32:0/1835893498 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd2a4045360 con 0x7fd2ac15cb00 2024-04-24T22:59:53.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.630+0000 7fd2b09c8640 1 -- 172.21.15.32:0/1835893498 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7fd2ac0afd60 con 0x7fd2ac15cb00 2024-04-24T22:59:53.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.635+0000 7fd2a2ffd640 1 -- 172.21.15.32:0/1835893498 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) ==== 90+0+0 (secure 0 0 0) 0x7fd2a40abe20 con 0x7fd2ac15cb00 2024-04-24T22:59:53.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.637+0000 7fd2b09c8640 1 -- 172.21.15.32:0/1835893498 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd28c06be60 msgr2=0x7fd28c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:53.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.637+0000 7fd2b09c8640 1 --2- 172.21.15.32:0/1835893498 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd28c06be60 0x7fd28c06e320 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd29c004ea0 tx=0x7fd29c009040 comp rx=0 tx=0).stop 2024-04-24T22:59:53.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.637+0000 7fd2b09c8640 1 -- 172.21.15.32:0/1835893498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2ac15cb00 msgr2=0x7fd2ac1502b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:53.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.637+0000 7fd2b09c8640 1 --2- 172.21.15.32:0/1835893498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2ac15cb00 0x7fd2ac1502b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4009f70 tx=0x7fd2a4004a40 comp rx=0 tx=0).stop 2024-04-24T22:59:53.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.637+0000 7fd2b09c8640 1 -- 172.21.15.32:0/1835893498 shutdown_connections 2024-04-24T22:59:53.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.637+0000 7fd2b09c8640 1 --2- 172.21.15.32:0/1835893498 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd28c06be60 0x7fd28c06e320 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:53.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.637+0000 7fd2b09c8640 1 --2- 172.21.15.32:0/1835893498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2ac15cb00 0x7fd2ac1502b0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:53.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.637+0000 7fd2b09c8640 1 --2- 172.21.15.32:0/1835893498 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd2ac15b380 0x7fd2ac13fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:53.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.637+0000 7fd2b09c8640 1 --2- 172.21.15.32:0/1835893498 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd2ac06cd00 0x7fd2ac1565b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:53.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.637+0000 7fd2b09c8640 1 -- 172.21.15.32:0/1835893498 >> 172.21.15.32:0/1835893498 conn(0x7fd2ac06b110 msgr2=0x7fd2ac06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:53.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.637+0000 7fd2b09c8640 1 -- 172.21.15.32:0/1835893498 shutdown_connections 2024-04-24T22:59:53.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.637+0000 7fd2b09c8640 1 -- 172.21.15.32:0/1835893498 wait complete. 2024-04-24T22:59:53.649 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-24T22:59:53.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.749+0000 7f30b4b29640 1 Processor -- start 2024-04-24T22:59:53.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.749+0000 7f30b4b29640 1 -- start start 2024-04-24T22:59:53.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.750+0000 7f30b4b29640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f30b0158dd0 0x7f30b006b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:53.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.750+0000 7f30b4b29640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f30b006dc60 0x7f30b00b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:53.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.750+0000 7f30b4b29640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30b006bd10 con 0x7f30b006b860 2024-04-24T22:59:53.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.750+0000 7f30b4b29640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f30b015ef10 con 0x7f30b0158dd0 2024-04-24T22:59:53.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.750+0000 7f30b4b29640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f30b015f1f0 con 0x7f30b006dc60 2024-04-24T22:59:53.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.750+0000 7f30ae59c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f30b0158dd0 0x7f30b006b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:53.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.750+0000 7f30add9b640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f30b006b860 0x7f30b006d550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53886/0 (socket says 172.21.15.32:53886) 2024-04-24T22:59:53.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.750+0000 7f30add9b640 1 -- 172.21.15.32:0/2116059063 learned_addr learned my addr 172.21.15.32:0/2116059063 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:53.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.751+0000 7f30ae59c640 1 -- 172.21.15.32:0/2116059063 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f30b006dc60 msgr2=0x7f30b00b5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:53.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.751+0000 7f30ae59c640 1 --2- 172.21.15.32:0/2116059063 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f30b006dc60 0x7f30b00b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:53.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.751+0000 7f30ae59c640 1 -- 172.21.15.32:0/2116059063 >> v1:172.21.15.32:6789/0 conn(0x7f30b006b860 legacy=0x7f30b006d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:53.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.751+0000 7f30ae59c640 1 -- 172.21.15.32:0/2116059063 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30b015f4d0 con 0x7f30b0158dd0 2024-04-24T22:59:53.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.751+0000 7f30ad59a640 1 -- 172.21.15.32:0/2116059063 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 901522683 0 0) 0x7f30b006bd10 con 0x7f30b006b860 2024-04-24T22:59:53.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.751+0000 7f30ae59c640 1 --2- 172.21.15.32:0/2116059063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30b0158dd0 0x7f30b006b320 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f30a80097a0 tx=0x7f30a802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a9940179c1b3910 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:53.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.752+0000 7f30ad59a640 1 -- 172.21.15.32:0/2116059063 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a8037070 con 0x7f30b0158dd0 2024-04-24T22:59:53.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.752+0000 7f30ad59a640 1 -- 172.21.15.32:0/2116059063 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30a8032440 con 0x7f30b0158dd0 2024-04-24T22:59:53.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.752+0000 7f30ad59a640 1 -- 172.21.15.32:0/2116059063 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a8032810 con 0x7f30b0158dd0 2024-04-24T22:59:53.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.752+0000 7f30b4b29640 1 -- 172.21.15.32:0/2116059063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30b0158dd0 msgr2=0x7f30b006b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:53.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.752+0000 7f30b4b29640 1 --2- 172.21.15.32:0/2116059063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30b0158dd0 0x7f30b006b320 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f30a80097a0 tx=0x7f30a802a180 comp rx=0 tx=0).stop 2024-04-24T22:59:53.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.752+0000 7f30b4b29640 1 -- 172.21.15.32:0/2116059063 shutdown_connections 2024-04-24T22:59:53.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.752+0000 7f30b4b29640 1 --2- 172.21.15.32:0/2116059063 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f30b006dc60 0x7f30b00b5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:53.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.752+0000 7f30b4b29640 1 --2- 172.21.15.32:0/2116059063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30b0158dd0 0x7f30b006b320 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:53.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.752+0000 7f30b4b29640 1 -- 172.21.15.32:0/2116059063 >> 172.21.15.32:0/2116059063 conn(0x7f30b006e800 msgr2=0x7f30b006ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:53.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.752+0000 7f30b4b29640 1 -- 172.21.15.32:0/2116059063 shutdown_connections 2024-04-24T22:59:53.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.752+0000 7f30b4b29640 1 -- 172.21.15.32:0/2116059063 wait complete. 2024-04-24T22:59:53.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.754+0000 7f30b4b29640 1 Processor -- start 2024-04-24T22:59:53.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.754+0000 7f30b4b29640 1 -- start start 2024-04-24T22:59:53.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.754+0000 7f30b4b29640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30b006b860 0x7f30b014e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:53.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.754+0000 7f30b4b29640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f30b006dc60 0x7f30b01f4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:53.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.754+0000 7f30b4b29640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f30b0158dd0 0x7f30b0162350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:53.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.754+0000 7f30b4b29640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f30b006bfe0 con 0x7f30b006b860 2024-04-24T22:59:53.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.754+0000 7f30b4b29640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f30b015fcd0 con 0x7f30b0158dd0 2024-04-24T22:59:53.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.754+0000 7f30b4b29640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f30b01591b0 con 0x7f30b006dc60 2024-04-24T22:59:53.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.754+0000 7f30ae59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30b006b860 0x7f30b014e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:53.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.754+0000 7f30ae59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30b006b860 0x7f30b014e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50886/0 (socket says 172.21.15.32:50886) 2024-04-24T22:59:53.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.754+0000 7f30ae59c640 1 -- 172.21.15.32:0/3324741020 learned_addr learned my addr 172.21.15.32:0/3324741020 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:53.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.755+0000 7f30ae59c640 1 -- 172.21.15.32:0/3324741020 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f30b006dc60 msgr2=0x7f30b01f4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:53.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.755+0000 7f30ae59c640 1 --2- 172.21.15.32:0/3324741020 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f30b006dc60 0x7f30b01f4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:53.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.755+0000 7f30ae59c640 1 -- 172.21.15.32:0/3324741020 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f30b0158dd0 msgr2=0x7f30b0162350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:53.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.755+0000 7f30ae59c640 1 --2- 172.21.15.32:0/3324741020 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f30b0158dd0 0x7f30b0162350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:53.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.755+0000 7f30ae59c640 1 -- 172.21.15.32:0/3324741020 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30b015daa0 con 0x7f30b006b860 2024-04-24T22:59:53.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.755+0000 7f30ae59c640 1 --2- 172.21.15.32:0/3324741020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30b006b860 0x7f30b014e070 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f30a80097a0 tx=0x7f30a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:53.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.755+0000 7f30a6ffd640 1 -- 172.21.15.32:0/3324741020 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a8041070 con 0x7f30b006b860 2024-04-24T22:59:53.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.755+0000 7f30a6ffd640 1 -- 172.21.15.32:0/3324741020 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30a803c870 con 0x7f30b006b860 2024-04-24T22:59:53.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.755+0000 7f30b4b29640 1 -- 172.21.15.32:0/3324741020 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30b015d7c0 con 0x7f30b006b860 2024-04-24T22:59:53.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.755+0000 7f30a6ffd640 1 -- 172.21.15.32:0/3324741020 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a8037040 con 0x7f30b006b860 2024-04-24T22:59:53.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.755+0000 7f30b4b29640 1 -- 172.21.15.32:0/3324741020 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30b015a9f0 con 0x7f30b006b860 2024-04-24T22:59:53.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.756+0000 7f30b4b29640 1 -- 172.21.15.32:0/3324741020 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30b0068430 con 0x7f30b006b860 2024-04-24T22:59:53.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.757+0000 7f30a6ffd640 1 -- 172.21.15.32:0/3324741020 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f30a8044070 con 0x7f30b006b860 2024-04-24T22:59:53.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.757+0000 7f30a6ffd640 1 --2- 172.21.15.32:0/3324741020 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f309006be30 0x7f309006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:53.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.757+0000 7f30a6ffd640 1 -- 172.21.15.32:0/3324741020 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f30a80ab8a0 con 0x7f30b006b860 2024-04-24T22:59:53.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.757+0000 7f30add9b640 1 --2- 172.21.15.32:0/3324741020 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f309006be30 0x7f309006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:53.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.758+0000 7f30add9b640 1 --2- 172.21.15.32:0/3324741020 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f309006be30 0x7f309006e2f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3098007fd0 tx=0x7f3098007920 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:53.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:53.761+0000 7f30a6ffd640 1 -- 172.21.15.32:0/3324741020 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f30a807a750 con 0x7f30b006b860 2024-04-24T22:59:54.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.014+0000 7f30b4b29640 1 -- 172.21.15.32:0/3324741020 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f30b01f54a0 con 0x7f30b006b860 2024-04-24T22:59:54.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.017+0000 7f30a6ffd640 1 -- 172.21.15.32:0/3324741020 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7f30a807a050 con 0x7f30b006b860 2024-04-24T22:59:54.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.018+0000 7f30b4b29640 1 -- 172.21.15.32:0/3324741020 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f309006be30 msgr2=0x7f309006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:54.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.018+0000 7f30b4b29640 1 --2- 172.21.15.32:0/3324741020 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f309006be30 0x7f309006e2f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3098007fd0 tx=0x7f3098007920 comp rx=0 tx=0).stop 2024-04-24T22:59:54.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.018+0000 7f30b4b29640 1 -- 172.21.15.32:0/3324741020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30b006b860 msgr2=0x7f30b014e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:54.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.018+0000 7f30b4b29640 1 --2- 172.21.15.32:0/3324741020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30b006b860 0x7f30b014e070 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f30a80097a0 tx=0x7f30a80046c0 comp rx=0 tx=0).stop 2024-04-24T22:59:54.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.018+0000 7f30b4b29640 1 -- 172.21.15.32:0/3324741020 shutdown_connections 2024-04-24T22:59:54.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.018+0000 7f30b4b29640 1 --2- 172.21.15.32:0/3324741020 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f309006be30 0x7f309006e2f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:54.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.019+0000 7f30b4b29640 1 --2- 172.21.15.32:0/3324741020 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f30b0158dd0 0x7f30b0162350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:54.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.019+0000 7f30b4b29640 1 --2- 172.21.15.32:0/3324741020 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f30b006dc60 0x7f30b01f4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:54.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.019+0000 7f30b4b29640 1 --2- 172.21.15.32:0/3324741020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30b006b860 0x7f30b014e070 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:54.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.019+0000 7f30b4b29640 1 -- 172.21.15.32:0/3324741020 >> 172.21.15.32:0/3324741020 conn(0x7f30b006e800 msgr2=0x7f30b00b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:54.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.019+0000 7f30b4b29640 1 -- 172.21.15.32:0/3324741020 shutdown_connections 2024-04-24T22:59:54.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.019+0000 7f30b4b29640 1 -- 172.21.15.32:0/3324741020 wait complete. 2024-04-24T22:59:54.031 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-24T22:59:54.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.132+0000 7fb57f25d640 1 Processor -- start 2024-04-24T22:59:54.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.132+0000 7fb57f25d640 1 -- start start 2024-04-24T22:59:54.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.133+0000 7fb57f25d640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb57815b480 0x7fb5780b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:54.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.133+0000 7fb57f25d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb57806cd00 0x7fb5780cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:54.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.133+0000 7fb57f25d640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5780b46c0 con 0x7fb57815fd60 2024-04-24T22:59:54.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.133+0000 7fb57f25d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fb5780b4ff0 con 0x7fb57806cd00 2024-04-24T22:59:54.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.133+0000 7fb57f25d640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fb5780b49a0 con 0x7fb57815b480 2024-04-24T22:59:54.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.133+0000 7fb57d7fa640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb57806cd00 0x7fb5780cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:54.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.133+0000 7fb57d7fa640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb57806cd00 0x7fb5780cdf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50896/0 (socket says 172.21.15.32:50896) 2024-04-24T22:59:54.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.133+0000 7fb57d7fa640 1 -- 172.21.15.32:0/1252071462 learned_addr learned my addr 172.21.15.32:0/1252071462 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:54.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.133+0000 7fb57d7fa640 1 -- 172.21.15.32:0/1252071462 >> v2:172.21.15.121:3300/0 conn(0x7fb57815b480 msgr2=0x7fb5780b5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:54.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.133+0000 7fb57cff9640 1 --2- 172.21.15.32:0/1252071462 >> v2:172.21.15.121:3300/0 conn(0x7fb57815b480 0x7fb5780b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:54.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.133+0000 7fb57d7fa640 1 --2- 172.21.15.32:0/1252071462 >> v2:172.21.15.121:3300/0 conn(0x7fb57815b480 0x7fb5780b5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:54.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.133+0000 7fb57d7fa640 1 -- 172.21.15.32:0/1252071462 >> v1:172.21.15.121:6789/0 conn(0x7fb57815fd60 legacy=0x7fb57806c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:54.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.133+0000 7fb57d7fa640 1 -- 172.21.15.32:0/1252071462 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5780b4c80 con 0x7fb57806cd00 2024-04-24T22:59:54.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.133+0000 7fb57cff9640 1 --2- 172.21.15.32:0/1252071462 >> v2:172.21.15.121:3300/0 conn(0x7fb57815b480 0x7fb5780b5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:59:54.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.134+0000 7fb57d7fa640 1 --2- 172.21.15.32:0/1252071462 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb57806cd00 0x7fb5780cdf80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb57806bf90 tx=0x7fb57002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=126619c790b4deb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:54.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.134+0000 7fb5777fe640 1 -- 172.21.15.32:0/1252071462 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb570036070 con 0x7fb57806cd00 2024-04-24T22:59:54.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.134+0000 7fb5777fe640 1 -- 172.21.15.32:0/1252071462 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb570031600 con 0x7fb57806cd00 2024-04-24T22:59:54.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.134+0000 7fb5777fe640 1 -- 172.21.15.32:0/1252071462 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5700319d0 con 0x7fb57806cd00 2024-04-24T22:59:54.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.134+0000 7fb57f25d640 1 -- 172.21.15.32:0/1252071462 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb57806cd00 msgr2=0x7fb5780cdf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:54.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.134+0000 7fb57f25d640 1 --2- 172.21.15.32:0/1252071462 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb57806cd00 0x7fb5780cdf80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb57806bf90 tx=0x7fb57002a6f0 comp rx=0 tx=0).stop 2024-04-24T22:59:54.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.134+0000 7fb57f25d640 1 -- 172.21.15.32:0/1252071462 shutdown_connections 2024-04-24T22:59:54.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.134+0000 7fb57f25d640 1 --2- 172.21.15.32:0/1252071462 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb57806cd00 0x7fb5780cdf80 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:54.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.134+0000 7fb57f25d640 1 --2- 172.21.15.32:0/1252071462 >> v2:172.21.15.121:3300/0 conn(0x7fb57815b480 0x7fb5780b5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:54.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.134+0000 7fb57f25d640 1 -- 172.21.15.32:0/1252071462 >> 172.21.15.32:0/1252071462 conn(0x7fb57806b110 msgr2=0x7fb57806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:54.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.135+0000 7fb57f25d640 1 -- 172.21.15.32:0/1252071462 shutdown_connections 2024-04-24T22:59:54.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.135+0000 7fb57f25d640 1 -- 172.21.15.32:0/1252071462 wait complete. 2024-04-24T22:59:54.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.136+0000 7fb57f25d640 1 Processor -- start 2024-04-24T22:59:54.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.136+0000 7fb57f25d640 1 -- start start 2024-04-24T22:59:54.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.137+0000 7fb57f25d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb57806cd00 0x7fb578159a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:54.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.137+0000 7fb57f25d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb57815b480 0x7fb5780b0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:54.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.137+0000 7fb57f25d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb57815fd60 0x7fb57814a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:54.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.137+0000 7fb57f25d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb57815b860 con 0x7fb57806cd00 2024-04-24T22:59:54.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.137+0000 7fb57f25d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb5780b5480 con 0x7fb57815fd60 2024-04-24T22:59:54.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.137+0000 7fb57f25d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fb5780aa990 con 0x7fb57815b480 2024-04-24T22:59:54.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.137+0000 7fb57cff9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb57806cd00 0x7fb578159a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:54.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.137+0000 7fb57cff9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb57806cd00 0x7fb578159a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50902/0 (socket says 172.21.15.32:50902) 2024-04-24T22:59:54.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.137+0000 7fb57cff9640 1 -- 172.21.15.32:0/1407178193 learned_addr learned my addr 172.21.15.32:0/1407178193 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:54.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.137+0000 7fb57d7fa640 1 --2- 172.21.15.32:0/1407178193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb57815fd60 0x7fb57814a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:54.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.137+0000 7fb577fff640 1 --2- 172.21.15.32:0/1407178193 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb57815b480 0x7fb5780b0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:54.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.137+0000 7fb57cff9640 1 -- 172.21.15.32:0/1407178193 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb57815b480 msgr2=0x7fb5780b0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:54.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.137+0000 7fb57cff9640 1 --2- 172.21.15.32:0/1407178193 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb57815b480 0x7fb5780b0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:54.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.137+0000 7fb57cff9640 1 -- 172.21.15.32:0/1407178193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb57815fd60 msgr2=0x7fb57814a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:54.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.137+0000 7fb57cff9640 1 --2- 172.21.15.32:0/1407178193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb57815fd60 0x7fb57814a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:54.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.137+0000 7fb57cff9640 1 -- 172.21.15.32:0/1407178193 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5780aedd0 con 0x7fb57806cd00 2024-04-24T22:59:54.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.137+0000 7fb577fff640 1 --2- 172.21.15.32:0/1407178193 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb57815b480 0x7fb5780b0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:54.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.137+0000 7fb57cff9640 1 --2- 172.21.15.32:0/1407178193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb57806cd00 0x7fb578159a40 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb56800c7f0 tx=0x7fb56800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:54.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.137+0000 7fb5757fa640 1 -- 172.21.15.32:0/1407178193 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb568015070 con 0x7fb57806cd00 2024-04-24T22:59:54.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.137+0000 7fb5757fa640 1 -- 172.21.15.32:0/1407178193 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb568007a30 con 0x7fb57806cd00 2024-04-24T22:59:54.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.137+0000 7fb57f25d640 1 -- 172.21.15.32:0/1407178193 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5780af0b0 con 0x7fb57806cd00 2024-04-24T22:59:54.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.137+0000 7fb5757fa640 1 -- 172.21.15.32:0/1407178193 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb568007e00 con 0x7fb57806cd00 2024-04-24T22:59:54.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.137+0000 7fb57f25d640 1 -- 172.21.15.32:0/1407178193 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5780ae9f0 con 0x7fb57806cd00 2024-04-24T22:59:54.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.138+0000 7fb57f25d640 1 -- 172.21.15.32:0/1407178193 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb578068430 con 0x7fb57806cd00 2024-04-24T22:59:54.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.139+0000 7fb5757fa640 1 -- 172.21.15.32:0/1407178193 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb568018070 con 0x7fb57806cd00 2024-04-24T22:59:54.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.139+0000 7fb5757fa640 1 --2- 172.21.15.32:0/1407178193 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb56406be30 0x7fb56406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:54.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.139+0000 7fb5757fa640 1 -- 172.21.15.32:0/1407178193 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7fb56808aa50 con 0x7fb57806cd00 2024-04-24T22:59:54.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.140+0000 7fb577fff640 1 --2- 172.21.15.32:0/1407178193 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb56406be30 0x7fb56406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:54.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.140+0000 7fb577fff640 1 --2- 172.21.15.32:0/1407178193 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb56406be30 0x7fb56406e2f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb56c006fd0 tx=0x7fb56c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:54.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.144+0000 7fb5757fa640 1 -- 172.21.15.32:0/1407178193 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb568059100 con 0x7fb57806cd00 2024-04-24T22:59:54.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.402+0000 7fb57f25d640 1 -- 172.21.15.32:0/1407178193 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7fb57806c000 con 0x7fb57806cd00 2024-04-24T22:59:54.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.404+0000 7fb5757fa640 1 -- 172.21.15.32:0/1407178193 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) ==== 97+0+0 (secure 0 0 0) 0x7fb56805d400 con 0x7fb57806cd00 2024-04-24T22:59:54.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.406+0000 7fb57f25d640 1 -- 172.21.15.32:0/1407178193 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb56406be30 msgr2=0x7fb56406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:54.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.406+0000 7fb57f25d640 1 --2- 172.21.15.32:0/1407178193 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb56406be30 0x7fb56406e2f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb56c006fd0 tx=0x7fb56c008040 comp rx=0 tx=0).stop 2024-04-24T22:59:54.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.406+0000 7fb57f25d640 1 -- 172.21.15.32:0/1407178193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb57806cd00 msgr2=0x7fb578159a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:54.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.406+0000 7fb57f25d640 1 --2- 172.21.15.32:0/1407178193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb57806cd00 0x7fb578159a40 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb56800c7f0 tx=0x7fb56800cbf0 comp rx=0 tx=0).stop 2024-04-24T22:59:54.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.406+0000 7fb57f25d640 1 -- 172.21.15.32:0/1407178193 shutdown_connections 2024-04-24T22:59:54.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.406+0000 7fb57f25d640 1 --2- 172.21.15.32:0/1407178193 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb56406be30 0x7fb56406e2f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:54.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.406+0000 7fb57f25d640 1 --2- 172.21.15.32:0/1407178193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb57815fd60 0x7fb57814a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:54.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.406+0000 7fb57f25d640 1 --2- 172.21.15.32:0/1407178193 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb57815b480 0x7fb5780b0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:54.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.406+0000 7fb57f25d640 1 --2- 172.21.15.32:0/1407178193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb57806cd00 0x7fb578159a40 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:54.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.406+0000 7fb57f25d640 1 -- 172.21.15.32:0/1407178193 >> 172.21.15.32:0/1407178193 conn(0x7fb57806b110 msgr2=0x7fb5780ce810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:54.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.406+0000 7fb57f25d640 1 -- 172.21.15.32:0/1407178193 shutdown_connections 2024-04-24T22:59:54.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.406+0000 7fb57f25d640 1 -- 172.21.15.32:0/1407178193 wait complete. 2024-04-24T22:59:54.418 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-24T22:59:54.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.518+0000 7f46a922c640 1 Processor -- start 2024-04-24T22:59:54.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.518+0000 7f46a922c640 1 -- start start 2024-04-24T22:59:54.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.519+0000 7f46a922c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f46a4156bb0 0x7f46a415dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:54.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.519+0000 7f46a922c640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f46a406d400 0x7f46a406d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:54.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.519+0000 7f46a922c640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46a415ccf0 con 0x7f46a4072c80 2024-04-24T22:59:54.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.519+0000 7f46a922c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f46a415cfb0 con 0x7f46a4156bb0 2024-04-24T22:59:54.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.519+0000 7f46a922c640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f46a415d290 con 0x7f46a406d400 2024-04-24T22:59:54.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.519+0000 7f46a2d9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f46a4156bb0 0x7f46a415dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:54.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.519+0000 7f46a359e640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f46a4072c80 0x7f46a40b5840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53898/0 (socket says 172.21.15.32:53898) 2024-04-24T22:59:54.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.519+0000 7f46a359e640 1 -- 172.21.15.32:0/3770223721 learned_addr learned my addr 172.21.15.32:0/3770223721 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:54.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.519+0000 7f46a259c640 1 --2- 172.21.15.32:0/3770223721 >> v2:172.21.15.121:3300/0 conn(0x7f46a406d400 0x7f46a406d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:54.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.520+0000 7f46a2d9d640 1 -- 172.21.15.32:0/3770223721 >> v2:172.21.15.121:3300/0 conn(0x7f46a406d400 msgr2=0x7f46a406d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:54.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.520+0000 7f46a2d9d640 1 --2- 172.21.15.32:0/3770223721 >> v2:172.21.15.121:3300/0 conn(0x7f46a406d400 0x7f46a406d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:54.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.520+0000 7f46a2d9d640 1 -- 172.21.15.32:0/3770223721 >> v1:172.21.15.32:6789/0 conn(0x7f46a4072c80 legacy=0x7f46a40b5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:54.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.520+0000 7f46a2d9d640 1 -- 172.21.15.32:0/3770223721 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46a415d570 con 0x7f46a4156bb0 2024-04-24T22:59:54.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.520+0000 7f46a1d9b640 1 -- 172.21.15.32:0/3770223721 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3752205426 0 0) 0x7f46a415ccf0 con 0x7f46a4072c80 2024-04-24T22:59:54.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.520+0000 7f46a259c640 1 --2- 172.21.15.32:0/3770223721 >> v2:172.21.15.121:3300/0 conn(0x7f46a406d400 0x7f46a406d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:54.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.520+0000 7f46a2d9d640 1 --2- 172.21.15.32:0/3770223721 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a4156bb0 0x7f46a415dbc0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f46a40b6b80 tx=0x7f469402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7913c98b1fdc6944 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:54.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.520+0000 7f46a1d9b640 1 -- 172.21.15.32:0/3770223721 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4694007740 con 0x7f46a4156bb0 2024-04-24T22:59:54.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.520+0000 7f46a1d9b640 1 -- 172.21.15.32:0/3770223721 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46940078e0 con 0x7f46a4156bb0 2024-04-24T22:59:54.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.520+0000 7f46a1d9b640 1 -- 172.21.15.32:0/3770223721 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4694007cb0 con 0x7f46a4156bb0 2024-04-24T22:59:54.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.521+0000 7f46a922c640 1 -- 172.21.15.32:0/3770223721 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a4156bb0 msgr2=0x7f46a415dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:54.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.521+0000 7f46a922c640 1 --2- 172.21.15.32:0/3770223721 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a4156bb0 0x7f46a415dbc0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f46a40b6b80 tx=0x7f469402a680 comp rx=0 tx=0).stop 2024-04-24T22:59:54.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.521+0000 7f46a922c640 1 -- 172.21.15.32:0/3770223721 shutdown_connections 2024-04-24T22:59:54.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.521+0000 7f46a922c640 1 --2- 172.21.15.32:0/3770223721 >> v2:172.21.15.121:3300/0 conn(0x7f46a406d400 0x7f46a406d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:54.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.521+0000 7f46a922c640 1 --2- 172.21.15.32:0/3770223721 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a4156bb0 0x7f46a415dbc0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:54.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.521+0000 7f46a922c640 1 -- 172.21.15.32:0/3770223721 >> 172.21.15.32:0/3770223721 conn(0x7f46a406e800 msgr2=0x7f46a406ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:54.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.521+0000 7f46a922c640 1 -- 172.21.15.32:0/3770223721 shutdown_connections 2024-04-24T22:59:54.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.521+0000 7f46a922c640 1 -- 172.21.15.32:0/3770223721 wait complete. 2024-04-24T22:59:54.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.522+0000 7f46a922c640 1 Processor -- start 2024-04-24T22:59:54.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.522+0000 7f46a922c640 1 -- start start 2024-04-24T22:59:54.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.523+0000 7f46a922c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46a406d400 0x7f46a4146650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:54.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.523+0000 7f46a922c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46a4072c80 0x7f46a40ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:54.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.523+0000 7f46a922c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a4156bb0 0x7f46a4154b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:54.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.523+0000 7f46a922c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f46a4156f90 con 0x7f46a4156bb0 2024-04-24T22:59:54.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.523+0000 7f46a922c640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f46a415e100 con 0x7f46a4072c80 2024-04-24T22:59:54.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.523+0000 7f46a922c640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f46a4157110 con 0x7f46a406d400 2024-04-24T22:59:54.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.523+0000 7f46a359e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a4156bb0 0x7f46a4154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:54.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.523+0000 7f46a359e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a4156bb0 0x7f46a4154b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50922/0 (socket says 172.21.15.32:50922) 2024-04-24T22:59:54.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.523+0000 7f46a359e640 1 -- 172.21.15.32:0/2548517846 learned_addr learned my addr 172.21.15.32:0/2548517846 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:54.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.524+0000 7f46a2d9d640 1 --2- 172.21.15.32:0/2548517846 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46a406d400 0x7f46a4146650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:54.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.524+0000 7f46a359e640 1 -- 172.21.15.32:0/2548517846 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46a406d400 msgr2=0x7f46a4146650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:54.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.524+0000 7f46a359e640 1 --2- 172.21.15.32:0/2548517846 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46a406d400 0x7f46a4146650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:54.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.524+0000 7f46a359e640 1 -- 172.21.15.32:0/2548517846 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46a4072c80 msgr2=0x7f46a40ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:54.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.524+0000 7f46a359e640 1 --2- 172.21.15.32:0/2548517846 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46a4072c80 0x7f46a40ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:54.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.524+0000 7f46a359e640 1 -- 172.21.15.32:0/2548517846 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46a4146b90 con 0x7f46a4156bb0 2024-04-24T22:59:54.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.524+0000 7f46a359e640 1 --2- 172.21.15.32:0/2548517846 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a4156bb0 0x7f46a4154b10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f469c00c480 tx=0x7f469c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:54.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.524+0000 7f468b7fe640 1 -- 172.21.15.32:0/2548517846 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469c018070 con 0x7f46a4156bb0 2024-04-24T22:59:54.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.524+0000 7f468b7fe640 1 -- 172.21.15.32:0/2548517846 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f469c007500 con 0x7f46a4156bb0 2024-04-24T22:59:54.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.524+0000 7f468b7fe640 1 -- 172.21.15.32:0/2548517846 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469c004a20 con 0x7f46a4156bb0 2024-04-24T22:59:54.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.524+0000 7f46a922c640 1 -- 172.21.15.32:0/2548517846 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46a4143d60 con 0x7f46a4156bb0 2024-04-24T22:59:54.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.524+0000 7f46a922c640 1 -- 172.21.15.32:0/2548517846 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46a4144020 con 0x7f46a4156bb0 2024-04-24T22:59:54.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.525+0000 7f46a922c640 1 -- 172.21.15.32:0/2548517846 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46a4068430 con 0x7f46a4156bb0 2024-04-24T22:59:54.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.525+0000 7f468b7fe640 1 -- 172.21.15.32:0/2548517846 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f469c004bc0 con 0x7f46a4156bb0 2024-04-24T22:59:54.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.526+0000 7f468b7fe640 1 --2- 172.21.15.32:0/2548517846 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f469006be30 0x7f469006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:54.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.526+0000 7f468b7fe640 1 -- 172.21.15.32:0/2548517846 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f469c089d50 con 0x7f46a4156bb0 2024-04-24T22:59:54.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.526+0000 7f46a2d9d640 1 --2- 172.21.15.32:0/2548517846 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f469006be30 0x7f469006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:54.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.527+0000 7f46a2d9d640 1 --2- 172.21.15.32:0/2548517846 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f469006be30 0x7f469006e2f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4694034000 tx=0x7f469402f000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:54.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.530+0000 7f468b7fe640 1 -- 172.21.15.32:0/2548517846 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f469c08d030 con 0x7f46a4156bb0 2024-04-24T22:59:54.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.781+0000 7f46a922c640 1 -- 172.21.15.32:0/2548517846 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f46a40727a0 con 0x7f46a4156bb0 2024-04-24T22:59:54.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.783+0000 7f468b7fe640 1 -- 172.21.15.32:0/2548517846 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) ==== 90+0+0 (secure 0 0 0) 0x7f469c058ae0 con 0x7f46a4156bb0 2024-04-24T22:59:54.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.785+0000 7f46a922c640 1 -- 172.21.15.32:0/2548517846 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f469006be30 msgr2=0x7f469006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:54.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.785+0000 7f46a922c640 1 --2- 172.21.15.32:0/2548517846 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f469006be30 0x7f469006e2f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4694034000 tx=0x7f469402f000 comp rx=0 tx=0).stop 2024-04-24T22:59:54.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.785+0000 7f46a922c640 1 -- 172.21.15.32:0/2548517846 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a4156bb0 msgr2=0x7f46a4154b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:54.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.785+0000 7f46a922c640 1 --2- 172.21.15.32:0/2548517846 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a4156bb0 0x7f46a4154b10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f469c00c480 tx=0x7f469c00c940 comp rx=0 tx=0).stop 2024-04-24T22:59:54.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.785+0000 7f46a922c640 1 -- 172.21.15.32:0/2548517846 shutdown_connections 2024-04-24T22:59:54.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.785+0000 7f46a922c640 1 --2- 172.21.15.32:0/2548517846 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f469006be30 0x7f469006e2f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:54.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.785+0000 7f46a922c640 1 --2- 172.21.15.32:0/2548517846 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a4156bb0 0x7f46a4154b10 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:54.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.785+0000 7f46a922c640 1 --2- 172.21.15.32:0/2548517846 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46a4072c80 0x7f46a40ce7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:54.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.785+0000 7f46a922c640 1 --2- 172.21.15.32:0/2548517846 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46a406d400 0x7f46a4146650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:54.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.785+0000 7f46a922c640 1 -- 172.21.15.32:0/2548517846 >> 172.21.15.32:0/2548517846 conn(0x7f46a406e800 msgr2=0x7f46a40b4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:54.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.785+0000 7f46a922c640 1 -- 172.21.15.32:0/2548517846 shutdown_connections 2024-04-24T22:59:54.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.785+0000 7f46a922c640 1 -- 172.21.15.32:0/2548517846 wait complete. 2024-04-24T22:59:54.797 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:59:54.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.896+0000 7f0a80ff9640 1 Processor -- start 2024-04-24T22:59:54.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.896+0000 7f0a80ff9640 1 -- start start 2024-04-24T22:59:54.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.897+0000 7f0a80ff9640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0a7c15e9b0 0x7f0a7c0d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:54.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.897+0000 7f0a80ff9640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a7c0d3170 con 0x7f0a7c06b820 2024-04-24T22:59:54.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.897+0000 7f0a80ff9640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a7c0d3450 con 0x7f0a7c0d4210 2024-04-24T22:59:54.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.897+0000 7f0a80ff9640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f0a7c0d3730 con 0x7f0a7c15e9b0 2024-04-24T22:59:54.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.897+0000 7f0a7ad9d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0a7c06b820 0x7f0a7c0b78f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53912/0 (socket says 172.21.15.32:53912) 2024-04-24T22:59:54.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.898+0000 7f0a7ad9d640 1 -- 172.21.15.32:0/1741158295 learned_addr learned my addr 172.21.15.32:0/1741158295 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:54.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.898+0000 7f0a7a59c640 1 --2- 172.21.15.32:0/1741158295 >> v2:172.21.15.121:3300/0 conn(0x7f0a7c15e9b0 0x7f0a7c0d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:54.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.898+0000 7f0a7959a640 1 -- 172.21.15.32:0/1741158295 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1005989671 0 0) 0x7f0a7c0d3170 con 0x7f0a7c06b820 2024-04-24T22:59:54.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.898+0000 7f0a7959a640 1 -- 172.21.15.32:0/1741158295 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0a68003520 con 0x7f0a7c06b820 2024-04-24T22:59:54.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.898+0000 7f0a7959a640 1 -- 172.21.15.32:0/1741158295 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 608548513 0 0) 0x7f0a68003520 con 0x7f0a7c06b820 2024-04-24T22:59:54.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.898+0000 7f0a7959a640 1 -- 172.21.15.32:0/1741158295 >> v2:172.21.15.121:3300/0 conn(0x7f0a7c15e9b0 msgr2=0x7f0a7c0d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:54.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.898+0000 7f0a7959a640 1 --2- 172.21.15.32:0/1741158295 >> v2:172.21.15.121:3300/0 conn(0x7f0a7c15e9b0 0x7f0a7c0d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:54.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.898+0000 7f0a7959a640 1 -- 172.21.15.32:0/1741158295 >> v1:172.21.15.121:6789/0 conn(0x7f0a7c0d4210 legacy=0x7f0a7c06b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:54.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.898+0000 7f0a7959a640 1 -- 172.21.15.32:0/1741158295 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a7c0d3a10 con 0x7f0a7c06b820 2024-04-24T22:59:54.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.898+0000 7f0a7959a640 1 -- 172.21.15.32:0/1741158295 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f0a6c0019f0 con 0x7f0a7c06b820 2024-04-24T22:59:54.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.898+0000 7f0a7959a640 1 -- 172.21.15.32:0/1741158295 >> v1:172.21.15.32:6789/0 conn(0x7f0a7c06b820 legacy=0x7f0a7c0b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:54.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.898+0000 7f0a7959a640 1 --2- 172.21.15.32:0/1741158295 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0a680039b0 0x7f0a68003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:54.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.898+0000 7f0a7959a640 1 --2- 172.21.15.32:0/1741158295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a68004800 0x7f0a68004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:54.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.898+0000 7f0a7959a640 1 --2- 172.21.15.32:0/1741158295 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0a680051c0 0x7f0a6800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:54.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.898+0000 7f0a7959a640 1 -- 172.21.15.32:0/1741158295 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7c0d3730 con 0x7f0a68004800 2024-04-24T22:59:54.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.898+0000 7f0a7959a640 1 -- 172.21.15.32:0/1741158295 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0a6800c160 con 0x7f0a680051c0 2024-04-24T22:59:54.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.898+0000 7f0a7959a640 1 -- 172.21.15.32:0/1741158295 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f0a6800c360 con 0x7f0a680039b0 2024-04-24T22:59:54.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.898+0000 7f0a79d9b640 1 --2- 172.21.15.32:0/1741158295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a68004800 0x7f0a68004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:54.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.898+0000 7f0a7a59c640 1 --2- 172.21.15.32:0/1741158295 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0a680039b0 0x7f0a68003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:54.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.898+0000 7f0a7ad9d640 1 --2- 172.21.15.32:0/1741158295 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0a680051c0 0x7f0a6800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:54.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.899+0000 7f0a79d9b640 1 -- 172.21.15.32:0/1741158295 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0a680039b0 msgr2=0x7f0a68003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:54.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.899+0000 7f0a79d9b640 1 --2- 172.21.15.32:0/1741158295 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0a680039b0 0x7f0a68003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:54.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.899+0000 7f0a79d9b640 1 -- 172.21.15.32:0/1741158295 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0a680051c0 msgr2=0x7f0a6800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:54.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.899+0000 7f0a79d9b640 1 --2- 172.21.15.32:0/1741158295 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0a680051c0 0x7f0a6800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:54.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.899+0000 7f0a7ad9d640 1 -- 172.21.15.32:0/1741158295 reap_dead start 2024-04-24T22:59:54.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.899+0000 7f0a79d9b640 1 -- 172.21.15.32:0/1741158295 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a68003520 con 0x7f0a68004800 2024-04-24T22:59:54.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.899+0000 7f0a79d9b640 1 --2- 172.21.15.32:0/1741158295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a68004800 0x7f0a68004c80 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0a64001750 tx=0x7f0a68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dffe18bdc62995d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:54.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.899+0000 7f0a7959a640 1 -- 172.21.15.32:0/1741158295 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a64001370 con 0x7f0a68004800 2024-04-24T22:59:54.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.899+0000 7f0a7959a640 1 -- 172.21.15.32:0/1741158295 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a640052d0 con 0x7f0a68004800 2024-04-24T22:59:54.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.899+0000 7f0a7959a640 1 -- 172.21.15.32:0/1741158295 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6402abc0 con 0x7f0a68004800 2024-04-24T22:59:54.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.900+0000 7f0a80ff9640 1 -- 172.21.15.32:0/1741158295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a68004800 msgr2=0x7f0a68004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:54.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.900+0000 7f0a80ff9640 1 --2- 172.21.15.32:0/1741158295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a68004800 0x7f0a68004c80 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0a64001750 tx=0x7f0a68005830 comp rx=0 tx=0).stop 2024-04-24T22:59:54.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.900+0000 7f0a80ff9640 1 -- 172.21.15.32:0/1741158295 shutdown_connections 2024-04-24T22:59:54.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.900+0000 7f0a80ff9640 1 --2- 172.21.15.32:0/1741158295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a68004800 0x7f0a68004c80 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:54.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.900+0000 7f0a80ff9640 1 -- 172.21.15.32:0/1741158295 >> 172.21.15.32:0/1741158295 conn(0x7f0a7c0d11d0 msgr2=0x7f0a7c0d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:54.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.900+0000 7f0a80ff9640 1 -- 172.21.15.32:0/1741158295 shutdown_connections 2024-04-24T22:59:54.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.900+0000 7f0a80ff9640 1 -- 172.21.15.32:0/1741158295 wait complete. 2024-04-24T22:59:54.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.901+0000 7f0a80ff9640 1 Processor -- start 2024-04-24T22:59:54.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.901+0000 7f0a80ff9640 1 -- start start 2024-04-24T22:59:54.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.902+0000 7f0a80ff9640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0a68004800 0x7f0a7c1f4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:54.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.902+0000 7f0a80ff9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a7c0c97e0 0x7f0a7c1561e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:54.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.902+0000 7f0a80ff9640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0a7c0c9c60 0x7f0a7c156720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:54.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.902+0000 7f0a80ff9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7c0ba0e0 con 0x7f0a7c0c97e0 2024-04-24T22:59:54.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.902+0000 7f0a80ff9640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7c0d2400 con 0x7f0a68004800 2024-04-24T22:59:54.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.902+0000 7f0a80ff9640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f0a7c0ba260 con 0x7f0a7c0c9c60 2024-04-24T22:59:54.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.902+0000 7f0a79d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a7c0c97e0 0x7f0a7c1561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:54.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.902+0000 7f0a79d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a7c0c97e0 0x7f0a7c1561e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50936/0 (socket says 172.21.15.32:50936) 2024-04-24T22:59:54.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.902+0000 7f0a79d9b640 1 -- 172.21.15.32:0/2570092243 learned_addr learned my addr 172.21.15.32:0/2570092243 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:54.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.902+0000 7f0a7a59c640 1 --2- 172.21.15.32:0/2570092243 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0a68004800 0x7f0a7c1f4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:54.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.902+0000 7f0a7ad9d640 1 --2- 172.21.15.32:0/2570092243 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0a7c0c9c60 0x7f0a7c156720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:54.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.902+0000 7f0a79d9b640 1 -- 172.21.15.32:0/2570092243 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0a7c0c9c60 msgr2=0x7f0a7c156720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:54.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.902+0000 7f0a79d9b640 1 --2- 172.21.15.32:0/2570092243 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0a7c0c9c60 0x7f0a7c156720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:54.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.903+0000 7f0a79d9b640 1 -- 172.21.15.32:0/2570092243 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0a68004800 msgr2=0x7f0a7c1f4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:54.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.903+0000 7f0a79d9b640 1 --2- 172.21.15.32:0/2570092243 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0a68004800 0x7f0a7c1f4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:54.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.903+0000 7f0a79d9b640 1 -- 172.21.15.32:0/2570092243 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a7c1f42d0 con 0x7f0a7c0c97e0 2024-04-24T22:59:54.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.903+0000 7f0a7a59c640 1 --2- 172.21.15.32:0/2570092243 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0a68004800 0x7f0a7c1f4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:54.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.903+0000 7f0a79d9b640 1 --2- 172.21.15.32:0/2570092243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a7c0c97e0 0x7f0a7c1561e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0a64001870 tx=0x7f0a64001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:54.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.903+0000 7f0a72ffd640 1 -- 172.21.15.32:0/2570092243 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6403f030 con 0x7f0a7c0c97e0 2024-04-24T22:59:54.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.903+0000 7f0a72ffd640 1 -- 172.21.15.32:0/2570092243 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a64005480 con 0x7f0a7c0c97e0 2024-04-24T22:59:54.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.903+0000 7f0a80ff9640 1 -- 172.21.15.32:0/2570092243 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a7c0ca1a0 con 0x7f0a7c0c97e0 2024-04-24T22:59:54.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.903+0000 7f0a72ffd640 1 -- 172.21.15.32:0/2570092243 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a64037040 con 0x7f0a7c0c97e0 2024-04-24T22:59:54.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.903+0000 7f0a80ff9640 1 -- 172.21.15.32:0/2570092243 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a7c0ca480 con 0x7f0a7c0c97e0 2024-04-24T22:59:54.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.904+0000 7f0a80ff9640 1 -- 172.21.15.32:0/2570092243 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a7c068430 con 0x7f0a7c0c97e0 2024-04-24T22:59:54.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.904+0000 7f0a72ffd640 1 -- 172.21.15.32:0/2570092243 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0a64042070 con 0x7f0a7c0c97e0 2024-04-24T22:59:54.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.905+0000 7f0a72ffd640 1 --2- 172.21.15.32:0/2570092243 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0a5c06be60 0x7f0a5c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:54.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.905+0000 7f0a72ffd640 1 -- 172.21.15.32:0/2570092243 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f0a640a9e60 con 0x7f0a7c0c97e0 2024-04-24T22:59:54.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.905+0000 7f0a7a59c640 1 --2- 172.21.15.32:0/2570092243 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0a5c06be60 0x7f0a5c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:54.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.905+0000 7f0a7a59c640 1 --2- 172.21.15.32:0/2570092243 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0a5c06be60 0x7f0a5c06e320 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c0d2d40 tx=0x7f0a74008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:54.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:54.909+0000 7f0a72ffd640 1 -- 172.21.15.32:0/2570092243 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0a6404a020 con 0x7f0a7c0c97e0 2024-04-24T22:59:55.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.096+0000 7f0a80ff9640 1 -- 172.21.15.32:0/2570092243 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0a7c15ba80 con 0x7f0a7c0c97e0 2024-04-24T22:59:55.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.097+0000 7f0a72ffd640 1 -- 172.21.15.32:0/2570092243 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+12903 (secure 0 0 0) 0x7f0a64078ad0 con 0x7f0a7c0c97e0 2024-04-24T22:59:55.099 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:59:55.099 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":28,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T22:59:52.095996+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:59:55.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.099+0000 7f0a80ff9640 1 -- 172.21.15.32:0/2570092243 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0a5c06be60 msgr2=0x7f0a5c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:55.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.099+0000 7f0a80ff9640 1 --2- 172.21.15.32:0/2570092243 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0a5c06be60 0x7f0a5c06e320 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c0d2d40 tx=0x7f0a74008040 comp rx=0 tx=0).stop 2024-04-24T22:59:55.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.099+0000 7f0a80ff9640 1 -- 172.21.15.32:0/2570092243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a7c0c97e0 msgr2=0x7f0a7c1561e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:55.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.099+0000 7f0a80ff9640 1 --2- 172.21.15.32:0/2570092243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a7c0c97e0 0x7f0a7c1561e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0a64001870 tx=0x7f0a64001120 comp rx=0 tx=0).stop 2024-04-24T22:59:55.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.100+0000 7f0a80ff9640 1 -- 172.21.15.32:0/2570092243 shutdown_connections 2024-04-24T22:59:55.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.100+0000 7f0a80ff9640 1 --2- 172.21.15.32:0/2570092243 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0a5c06be60 0x7f0a5c06e320 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:55.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.100+0000 7f0a80ff9640 1 --2- 172.21.15.32:0/2570092243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a7c0c97e0 0x7f0a7c1561e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:55.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.100+0000 7f0a80ff9640 1 --2- 172.21.15.32:0/2570092243 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0a7c0c9c60 0x7f0a7c156720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:55.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.100+0000 7f0a80ff9640 1 --2- 172.21.15.32:0/2570092243 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0a68004800 0x7f0a7c1f4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:55.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.100+0000 7f0a80ff9640 1 -- 172.21.15.32:0/2570092243 >> 172.21.15.32:0/2570092243 conn(0x7f0a7c0d11d0 msgr2=0x7f0a7c15c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:55.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.100+0000 7f0a80ff9640 1 -- 172.21.15.32:0/2570092243 shutdown_connections 2024-04-24T22:59:55.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.100+0000 7f0a80ff9640 1 -- 172.21.15.32:0/2570092243 wait complete. 2024-04-24T22:59:55.112 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T22:59:55.113 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T22:59:55.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.215+0000 7f8a3ab14640 1 Processor -- start 2024-04-24T22:59:55.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.215+0000 7f8a3ab14640 1 -- start start 2024-04-24T22:59:55.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.216+0000 7f8a3ab14640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8a3415cc40 0x7f8a3415d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:55.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.216+0000 7f8a3ab14640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8a340d05b0 0x7f8a3415d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:55.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.216+0000 7f8a3ab14640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a340b4ee0 con 0x7f8a340cec30 2024-04-24T22:59:55.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.216+0000 7f8a3ab14640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f8a3406cca0 con 0x7f8a3415cc40 2024-04-24T22:59:55.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.216+0000 7f8a3ab14640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f8a340ce5f0 con 0x7f8a340d05b0 2024-04-24T22:59:55.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.216+0000 7f8a388b0640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8a3415cc40 0x7f8a3415d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:55.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.216+0000 7f8a388b0640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8a3415cc40 0x7f8a3415d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50940/0 (socket says 172.21.15.32:50940) 2024-04-24T22:59:55.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.216+0000 7f8a388b0640 1 -- 172.21.15.32:0/2513298737 learned_addr learned my addr 172.21.15.32:0/2513298737 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:55.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.216+0000 7f8a33fff640 1 --2- 172.21.15.32:0/2513298737 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8a340d05b0 0x7f8a3415d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:55.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.216+0000 7f8a388b0640 1 -- 172.21.15.32:0/2513298737 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8a340d05b0 msgr2=0x7f8a3415d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:55.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.216+0000 7f8a388b0640 1 --2- 172.21.15.32:0/2513298737 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8a340d05b0 0x7f8a3415d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:55.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.216+0000 7f8a388b0640 1 -- 172.21.15.32:0/2513298737 >> v1:172.21.15.121:6789/0 conn(0x7f8a340cec30 legacy=0x7f8a340cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:55.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.216+0000 7f8a388b0640 1 -- 172.21.15.32:0/2513298737 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a340b51c0 con 0x7f8a3415cc40 2024-04-24T22:59:55.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.217+0000 7f8a388b0640 1 --2- 172.21.15.32:0/2513298737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a3415cc40 0x7f8a3415d020 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c009e50 tx=0x7f8a2c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf5318caf3b870d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:55.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.217+0000 7f8a337fe640 1 -- 172.21.15.32:0/2513298737 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c036070 con 0x7f8a3415cc40 2024-04-24T22:59:55.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.217+0000 7f8a337fe640 1 -- 172.21.15.32:0/2513298737 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a2c031600 con 0x7f8a3415cc40 2024-04-24T22:59:55.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.217+0000 7f8a337fe640 1 -- 172.21.15.32:0/2513298737 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c0319d0 con 0x7f8a3415cc40 2024-04-24T22:59:55.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.218+0000 7f8a3ab14640 1 -- 172.21.15.32:0/2513298737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a3415cc40 msgr2=0x7f8a3415d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:55.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.218+0000 7f8a3ab14640 1 --2- 172.21.15.32:0/2513298737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a3415cc40 0x7f8a3415d020 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c009e50 tx=0x7f8a2c02a710 comp rx=0 tx=0).stop 2024-04-24T22:59:55.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.218+0000 7f8a3ab14640 1 -- 172.21.15.32:0/2513298737 shutdown_connections 2024-04-24T22:59:55.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.218+0000 7f8a3ab14640 1 --2- 172.21.15.32:0/2513298737 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8a340d05b0 0x7f8a3415d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:55.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.218+0000 7f8a3ab14640 1 --2- 172.21.15.32:0/2513298737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a3415cc40 0x7f8a3415d020 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:55.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.218+0000 7f8a3ab14640 1 -- 172.21.15.32:0/2513298737 >> 172.21.15.32:0/2513298737 conn(0x7f8a340d0ae0 msgr2=0x7f8a340d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:55.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.218+0000 7f8a3ab14640 1 -- 172.21.15.32:0/2513298737 shutdown_connections 2024-04-24T22:59:55.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.218+0000 7f8a3ab14640 1 -- 172.21.15.32:0/2513298737 wait complete. 2024-04-24T22:59:55.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.219+0000 7f8a3ab14640 1 Processor -- start 2024-04-24T22:59:55.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.219+0000 7f8a3ab14640 1 -- start start 2024-04-24T22:59:55.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.220+0000 7f8a3ab14640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a340cec30 0x7f8a3414fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:55.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.220+0000 7f8a3ab14640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8a340d05b0 0x7f8a341e59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:55.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.220+0000 7f8a3ab14640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8a3415cc40 0x7f8a341413e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:55.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.220+0000 7f8a3ab14640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8a340b5780 con 0x7f8a340cec30 2024-04-24T22:59:55.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.220+0000 7f8a3ab14640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3415daa0 con 0x7f8a340d05b0 2024-04-24T22:59:55.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.220+0000 7f8a3ab14640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f8a340b5a60 con 0x7f8a3415cc40 2024-04-24T22:59:55.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.220+0000 7f8a388b0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a340cec30 0x7f8a3414fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:55.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.220+0000 7f8a388b0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a340cec30 0x7f8a3414fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50946/0 (socket says 172.21.15.32:50946) 2024-04-24T22:59:55.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.220+0000 7f8a388b0640 1 -- 172.21.15.32:0/3551976237 learned_addr learned my addr 172.21.15.32:0/3551976237 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:55.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.220+0000 7f8a33fff640 1 --2- 172.21.15.32:0/3551976237 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8a340d05b0 0x7f8a341e59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:55.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.220+0000 7f8a390b1640 1 --2- 172.21.15.32:0/3551976237 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8a3415cc40 0x7f8a341413e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:55.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.221+0000 7f8a388b0640 1 -- 172.21.15.32:0/3551976237 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8a3415cc40 msgr2=0x7f8a341413e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:55.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.221+0000 7f8a388b0640 1 --2- 172.21.15.32:0/3551976237 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8a3415cc40 0x7f8a341413e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:55.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.221+0000 7f8a388b0640 1 -- 172.21.15.32:0/3551976237 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8a340d05b0 msgr2=0x7f8a341e59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:55.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.221+0000 7f8a388b0640 1 --2- 172.21.15.32:0/3551976237 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8a340d05b0 0x7f8a341e59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:55.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.221+0000 7f8a388b0640 1 -- 172.21.15.32:0/3551976237 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a341e6200 con 0x7f8a340cec30 2024-04-24T22:59:55.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.221+0000 7f8a388b0640 1 --2- 172.21.15.32:0/3551976237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a340cec30 0x7f8a3414fa80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c034040 tx=0x7f8a2c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:55.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.221+0000 7f8a317fa640 1 -- 172.21.15.32:0/3551976237 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c036050 con 0x7f8a340cec30 2024-04-24T22:59:55.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.221+0000 7f8a317fa640 1 -- 172.21.15.32:0/3551976237 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a2c042070 con 0x7f8a340cec30 2024-04-24T22:59:55.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.221+0000 7f8a3ab14640 1 -- 172.21.15.32:0/3551976237 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a341507a0 con 0x7f8a340cec30 2024-04-24T22:59:55.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.221+0000 7f8a317fa640 1 -- 172.21.15.32:0/3551976237 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c009510 con 0x7f8a340cec30 2024-04-24T22:59:55.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.221+0000 7f8a3ab14640 1 -- 172.21.15.32:0/3551976237 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a341e5f10 con 0x7f8a340cec30 2024-04-24T22:59:55.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.222+0000 7f8a3ab14640 1 -- 172.21.15.32:0/3551976237 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a34068430 con 0x7f8a340cec30 2024-04-24T22:59:55.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.222+0000 7f8a317fa640 1 -- 172.21.15.32:0/3551976237 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8a2c007c50 con 0x7f8a340cec30 2024-04-24T22:59:55.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.223+0000 7f8a317fa640 1 --2- 172.21.15.32:0/3551976237 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8a2006be30 0x7f8a2006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:55.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.223+0000 7f8a317fa640 1 -- 172.21.15.32:0/3551976237 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f8a2c0ab850 con 0x7f8a340cec30 2024-04-24T22:59:55.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.223+0000 7f8a33fff640 1 --2- 172.21.15.32:0/3551976237 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8a2006be30 0x7f8a2006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:55.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.223+0000 7f8a33fff640 1 --2- 172.21.15.32:0/3551976237 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8a2006be30 0x7f8a2006e2f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8a3414ffc0 tx=0x7f8a28008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:55.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.227+0000 7f8a317fa640 1 -- 172.21.15.32:0/3551976237 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8a2c079f00 con 0x7f8a340cec30 2024-04-24T22:59:55.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.409+0000 7f8a3ab14640 1 -- 172.21.15.32:0/3551976237 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8a34072f20 con 0x7f8a340cec30 2024-04-24T22:59:55.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.409+0000 7f8a317fa640 1 -- 172.21.15.32:0/3551976237 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7f8a2c07e200 con 0x7f8a340cec30 2024-04-24T22:59:55.411 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T22:59:55.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.411+0000 7f8a3ab14640 1 -- 172.21.15.32:0/3551976237 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8a2006be30 msgr2=0x7f8a2006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:55.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.411+0000 7f8a3ab14640 1 --2- 172.21.15.32:0/3551976237 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8a2006be30 0x7f8a2006e2f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8a3414ffc0 tx=0x7f8a28008040 comp rx=0 tx=0).stop 2024-04-24T22:59:55.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.411+0000 7f8a3ab14640 1 -- 172.21.15.32:0/3551976237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a340cec30 msgr2=0x7f8a3414fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:55.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.411+0000 7f8a3ab14640 1 --2- 172.21.15.32:0/3551976237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a340cec30 0x7f8a3414fa80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c034040 tx=0x7f8a2c004a40 comp rx=0 tx=0).stop 2024-04-24T22:59:55.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.411+0000 7f8a3ab14640 1 -- 172.21.15.32:0/3551976237 shutdown_connections 2024-04-24T22:59:55.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.411+0000 7f8a3ab14640 1 --2- 172.21.15.32:0/3551976237 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8a2006be30 0x7f8a2006e2f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:55.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.411+0000 7f8a3ab14640 1 --2- 172.21.15.32:0/3551976237 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8a3415cc40 0x7f8a341413e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:55.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.411+0000 7f8a3ab14640 1 --2- 172.21.15.32:0/3551976237 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8a340d05b0 0x7f8a341e59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:55.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.411+0000 7f8a3ab14640 1 --2- 172.21.15.32:0/3551976237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a340cec30 0x7f8a3414fa80 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:55.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.411+0000 7f8a3ab14640 1 -- 172.21.15.32:0/3551976237 >> 172.21.15.32:0/3551976237 conn(0x7f8a340d0ae0 msgr2=0x7f8a340ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:55.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.411+0000 7f8a3ab14640 1 -- 172.21.15.32:0/3551976237 shutdown_connections 2024-04-24T22:59:55.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.411+0000 7f8a3ab14640 1 -- 172.21.15.32:0/3551976237 wait complete. 2024-04-24T22:59:55.424 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-24T22:59:55.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.523+0000 7fa5d2bfb640 1 Processor -- start 2024-04-24T22:59:55.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.523+0000 7fa5d2bfb640 1 -- start start 2024-04-24T22:59:55.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.524+0000 7fa5d2bfb640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa5cc158dd0 0x7fa5cc06b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:55.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.524+0000 7fa5d2bfb640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5cc06bd10 con 0x7fa5cc06dc60 2024-04-24T22:59:55.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.524+0000 7fa5d2bfb640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5cc15ef10 con 0x7fa5cc06b860 2024-04-24T22:59:55.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.524+0000 7fa5d2bfb640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fa5cc15f1f0 con 0x7fa5cc158dd0 2024-04-24T22:59:55.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.524+0000 7fa5d1198640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa5cc06dc60 0x7fa5cc0b5d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53928/0 (socket says 172.21.15.32:53928) 2024-04-24T22:59:55.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.524+0000 7fa5d1198640 1 -- 172.21.15.32:0/2283753903 learned_addr learned my addr 172.21.15.32:0/2283753903 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:55.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.524+0000 7fa5d0997640 1 --2- 172.21.15.32:0/2283753903 >> v2:172.21.15.121:3300/0 conn(0x7fa5cc158dd0 0x7fa5cc06b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:55.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.524+0000 7fa5cb7fe640 1 -- 172.21.15.32:0/2283753903 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 889025554 0 0) 0x7fa5cc06bd10 con 0x7fa5cc06dc60 2024-04-24T22:59:55.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.524+0000 7fa5cb7fe640 1 -- 172.21.15.32:0/2283753903 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa5b4003520 con 0x7fa5cc06dc60 2024-04-24T22:59:55.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.525+0000 7fa5cb7fe640 1 -- 172.21.15.32:0/2283753903 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 977479927 0 0) 0x7fa5b4003520 con 0x7fa5cc06dc60 2024-04-24T22:59:55.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.525+0000 7fa5cb7fe640 1 -- 172.21.15.32:0/2283753903 >> v2:172.21.15.121:3300/0 conn(0x7fa5cc158dd0 msgr2=0x7fa5cc06b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:55.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.525+0000 7fa5cb7fe640 1 --2- 172.21.15.32:0/2283753903 >> v2:172.21.15.121:3300/0 conn(0x7fa5cc158dd0 0x7fa5cc06b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:55.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.525+0000 7fa5cb7fe640 1 -- 172.21.15.32:0/2283753903 >> v1:172.21.15.121:6789/0 conn(0x7fa5cc06b860 legacy=0x7fa5cc06d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:55.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.525+0000 7fa5cb7fe640 1 -- 172.21.15.32:0/2283753903 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5cc15f4d0 con 0x7fa5cc06dc60 2024-04-24T22:59:55.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.525+0000 7fa5cb7fe640 1 -- 172.21.15.32:0/2283753903 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7fa5bc001670 con 0x7fa5cc06dc60 2024-04-24T22:59:55.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.525+0000 7fa5cb7fe640 1 -- 172.21.15.32:0/2283753903 >> v1:172.21.15.32:6789/0 conn(0x7fa5cc06dc60 legacy=0x7fa5cc0b5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:55.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.525+0000 7fa5cb7fe640 1 --2- 172.21.15.32:0/2283753903 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5b40039b0 0x7fa5b4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:55.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.525+0000 7fa5cb7fe640 1 --2- 172.21.15.32:0/2283753903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5b4004800 0x7fa5b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:55.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.525+0000 7fa5cb7fe640 1 --2- 172.21.15.32:0/2283753903 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa5b40051c0 0x7fa5b400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:55.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.525+0000 7fa5cb7fe640 1 -- 172.21.15.32:0/2283753903 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa5cc15f1f0 con 0x7fa5b4004800 2024-04-24T22:59:55.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.525+0000 7fa5cb7fe640 1 -- 172.21.15.32:0/2283753903 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b400c060 con 0x7fa5b40039b0 2024-04-24T22:59:55.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.525+0000 7fa5cb7fe640 1 -- 172.21.15.32:0/2283753903 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa5b400c260 con 0x7fa5b40051c0 2024-04-24T22:59:55.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.525+0000 7fa5cbfff640 1 --2- 172.21.15.32:0/2283753903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5b4004800 0x7fa5b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:55.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.525+0000 7fa5d0997640 1 --2- 172.21.15.32:0/2283753903 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5b40039b0 0x7fa5b4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:55.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.525+0000 7fa5d1198640 1 --2- 172.21.15.32:0/2283753903 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa5b40051c0 0x7fa5b400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:55.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.525+0000 7fa5cbfff640 1 -- 172.21.15.32:0/2283753903 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa5b40051c0 msgr2=0x7fa5b400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:55.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.525+0000 7fa5cbfff640 1 --2- 172.21.15.32:0/2283753903 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa5b40051c0 0x7fa5b400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:55.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.525+0000 7fa5cbfff640 1 -- 172.21.15.32:0/2283753903 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5b40039b0 msgr2=0x7fa5b4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:55.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.525+0000 7fa5cbfff640 1 --2- 172.21.15.32:0/2283753903 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5b40039b0 0x7fa5b4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:55.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.525+0000 7fa5d1198640 1 -- 172.21.15.32:0/2283753903 reap_dead start 2024-04-24T22:59:55.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.525+0000 7fa5cbfff640 1 -- 172.21.15.32:0/2283753903 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5b4003520 con 0x7fa5b4004800 2024-04-24T22:59:55.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.526+0000 7fa5cbfff640 1 --2- 172.21.15.32:0/2283753903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5b4004800 0x7fa5b4004c80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0001750 tx=0x7fa5b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdc3087027b4d1dc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:55.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.526+0000 7fa5cb7fe640 1 -- 172.21.15.32:0/2283753903 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c0001370 con 0x7fa5b4004800 2024-04-24T22:59:55.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.526+0000 7fa5cb7fe640 1 -- 172.21.15.32:0/2283753903 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5c00052d0 con 0x7fa5b4004800 2024-04-24T22:59:55.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.526+0000 7fa5cb7fe640 1 -- 172.21.15.32:0/2283753903 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c002abc0 con 0x7fa5b4004800 2024-04-24T22:59:55.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.527+0000 7fa5d2bfb640 1 -- 172.21.15.32:0/2283753903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5b4004800 msgr2=0x7fa5b4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:55.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.527+0000 7fa5d2bfb640 1 --2- 172.21.15.32:0/2283753903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5b4004800 0x7fa5b4004c80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0001750 tx=0x7fa5b4005830 comp rx=0 tx=0).stop 2024-04-24T22:59:55.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.527+0000 7fa5d2bfb640 1 -- 172.21.15.32:0/2283753903 shutdown_connections 2024-04-24T22:59:55.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.527+0000 7fa5d2bfb640 1 --2- 172.21.15.32:0/2283753903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5b4004800 0x7fa5b4004c80 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:55.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.527+0000 7fa5d2bfb640 1 -- 172.21.15.32:0/2283753903 >> 172.21.15.32:0/2283753903 conn(0x7fa5cc06e800 msgr2=0x7fa5cc06ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:55.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.527+0000 7fa5d2bfb640 1 -- 172.21.15.32:0/2283753903 shutdown_connections 2024-04-24T22:59:55.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.527+0000 7fa5d2bfb640 1 -- 172.21.15.32:0/2283753903 wait complete. 2024-04-24T22:59:55.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.528+0000 7fa5d2bfb640 1 Processor -- start 2024-04-24T22:59:55.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.528+0000 7fa5d2bfb640 1 -- start start 2024-04-24T22:59:55.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.529+0000 7fa5d2bfb640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa5b4004800 0x7fa5cc1501c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:55.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.529+0000 7fa5d2bfb640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5cc14da60 0x7fa5cc14dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:55.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.529+0000 7fa5d2bfb640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5cc0b65e0 0x7fa5cc0b6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:55.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.529+0000 7fa5d2bfb640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa5cc06bfe0 con 0x7fa5cc0b65e0 2024-04-24T22:59:55.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.529+0000 7fa5d2bfb640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa5cc15fcd0 con 0x7fa5cc14da60 2024-04-24T22:59:55.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.529+0000 7fa5d2bfb640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa5cc1591b0 con 0x7fa5b4004800 2024-04-24T22:59:55.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.529+0000 7fa5d1198640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5cc0b65e0 0x7fa5cc0b6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:55.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.529+0000 7fa5d1198640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5cc0b65e0 0x7fa5cc0b6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50956/0 (socket says 172.21.15.32:50956) 2024-04-24T22:59:55.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.529+0000 7fa5d1198640 1 -- 172.21.15.32:0/3485829387 learned_addr learned my addr 172.21.15.32:0/3485829387 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:55.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.529+0000 7fa5cbfff640 1 --2- 172.21.15.32:0/3485829387 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5cc14da60 0x7fa5cc14dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:55.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.529+0000 7fa5d0997640 1 --2- 172.21.15.32:0/3485829387 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa5b4004800 0x7fa5cc1501c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:55.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.529+0000 7fa5d1198640 1 -- 172.21.15.32:0/3485829387 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa5b4004800 msgr2=0x7fa5cc1501c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:55.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.529+0000 7fa5d1198640 1 --2- 172.21.15.32:0/3485829387 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa5b4004800 0x7fa5cc1501c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:55.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.529+0000 7fa5d1198640 1 -- 172.21.15.32:0/3485829387 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5cc14da60 msgr2=0x7fa5cc14dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:55.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.529+0000 7fa5d1198640 1 --2- 172.21.15.32:0/3485829387 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5cc14da60 0x7fa5cc14dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:55.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.529+0000 7fa5d1198640 1 -- 172.21.15.32:0/3485829387 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5cc14e520 con 0x7fa5cc0b65e0 2024-04-24T22:59:55.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.529+0000 7fa5cbfff640 1 --2- 172.21.15.32:0/3485829387 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5cc14da60 0x7fa5cc14dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:55.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.529+0000 7fa5d0997640 1 --2- 172.21.15.32:0/3485829387 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa5b4004800 0x7fa5cc1501c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:55.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.529+0000 7fa5d1198640 1 --2- 172.21.15.32:0/3485829387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5cc0b65e0 0x7fa5cc0b6a60 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc162320 tx=0x7fa5bc007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:55.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.529+0000 7fa5c97fa640 1 -- 172.21.15.32:0/3485829387 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc01a070 con 0x7fa5cc0b65e0 2024-04-24T22:59:55.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.530+0000 7fa5c97fa640 1 -- 172.21.15.32:0/3485829387 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5bc00f680 con 0x7fa5cc0b65e0 2024-04-24T22:59:55.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.530+0000 7fa5d2bfb640 1 -- 172.21.15.32:0/3485829387 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5cc14e800 con 0x7fa5cc0b65e0 2024-04-24T22:59:55.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.530+0000 7fa5c97fa640 1 -- 172.21.15.32:0/3485829387 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc00fa90 con 0x7fa5cc0b65e0 2024-04-24T22:59:55.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.530+0000 7fa5d2bfb640 1 -- 172.21.15.32:0/3485829387 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5cc14edc0 con 0x7fa5cc0b65e0 2024-04-24T22:59:55.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.530+0000 7fa5d2bfb640 1 -- 172.21.15.32:0/3485829387 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5cc0b6fa0 con 0x7fa5cc0b65e0 2024-04-24T22:59:55.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.531+0000 7fa5c97fa640 1 -- 172.21.15.32:0/3485829387 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa5bc01d070 con 0x7fa5cc0b65e0 2024-04-24T22:59:55.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.531+0000 7fa5c97fa640 1 --2- 172.21.15.32:0/3485829387 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa5b806be30 0x7fa5b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:55.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.531+0000 7fa5c97fa640 1 -- 172.21.15.32:0/3485829387 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7fa5bc08a180 con 0x7fa5cc0b65e0 2024-04-24T22:59:55.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.532+0000 7fa5d0997640 1 --2- 172.21.15.32:0/3485829387 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa5b806be30 0x7fa5b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:55.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.532+0000 7fa5d0997640 1 --2- 172.21.15.32:0/3485829387 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa5b806be30 0x7fa5b806e2f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc0740e0 tx=0x7fa5c400b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:55.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.536+0000 7fa5c97fa640 1 -- 172.21.15.32:0/3485829387 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa5bc058f10 con 0x7fa5cc0b65e0 2024-04-24T22:59:55.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.721+0000 7fa5d2bfb640 1 -- 172.21.15.32:0/3485829387 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fa5cc06c790 con 0x7fa5cc0b65e0 2024-04-24T22:59:55.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.721+0000 7fa5c97fa640 1 -- 172.21.15.32:0/3485829387 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v28) ==== 84+0+65 (secure 0 0 0) 0x7fa5bc088e20 con 0x7fa5cc0b65e0 2024-04-24T22:59:55.723 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:59:55.723 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2024-04-24T22:59:55.723 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T22:59:55.723 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2024-04-24T22:59:55.723 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": ".mgr" 2024-04-24T22:59:55.724 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T22:59:55.724 INFO:teuthology.orchestra.run.smithi032.stdout:] 2024-04-24T22:59:55.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.723+0000 7fa5d2bfb640 1 -- 172.21.15.32:0/3485829387 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa5b806be30 msgr2=0x7fa5b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:55.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.723+0000 7fa5d2bfb640 1 --2- 172.21.15.32:0/3485829387 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa5b806be30 0x7fa5b806e2f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc0740e0 tx=0x7fa5c400b040 comp rx=0 tx=0).stop 2024-04-24T22:59:55.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.723+0000 7fa5d2bfb640 1 -- 172.21.15.32:0/3485829387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5cc0b65e0 msgr2=0x7fa5cc0b6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:55.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.723+0000 7fa5d2bfb640 1 --2- 172.21.15.32:0/3485829387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5cc0b65e0 0x7fa5cc0b6a60 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc162320 tx=0x7fa5bc007430 comp rx=0 tx=0).stop 2024-04-24T22:59:55.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.724+0000 7fa5d2bfb640 1 -- 172.21.15.32:0/3485829387 shutdown_connections 2024-04-24T22:59:55.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.724+0000 7fa5d2bfb640 1 --2- 172.21.15.32:0/3485829387 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa5b806be30 0x7fa5b806e2f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:55.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.724+0000 7fa5d2bfb640 1 --2- 172.21.15.32:0/3485829387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5cc0b65e0 0x7fa5cc0b6a60 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:55.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.724+0000 7fa5d2bfb640 1 --2- 172.21.15.32:0/3485829387 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5cc14da60 0x7fa5cc14dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:55.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.724+0000 7fa5d2bfb640 1 --2- 172.21.15.32:0/3485829387 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa5b4004800 0x7fa5cc1501c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:55.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.724+0000 7fa5d2bfb640 1 -- 172.21.15.32:0/3485829387 >> 172.21.15.32:0/3485829387 conn(0x7fa5cc06e800 msgr2=0x7fa5cc15fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:55.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.724+0000 7fa5d2bfb640 1 -- 172.21.15.32:0/3485829387 shutdown_connections 2024-04-24T22:59:55.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.724+0000 7fa5d2bfb640 1 -- 172.21.15.32:0/3485829387 wait complete. 2024-04-24T22:59:55.735 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-24T22:59:55.736 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-24T22:59:55.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.839+0000 7fabbe334640 1 Processor -- start 2024-04-24T22:59:55.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.839+0000 7fabbe334640 1 -- start start 2024-04-24T22:59:55.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.840+0000 7fabbe334640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fabb8161c30 0x7fabb8067980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:55.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.840+0000 7fabbe334640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabb8071620 con 0x7fabb806d230 2024-04-24T22:59:55.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.840+0000 7fabbe334640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabb80af270 con 0x7fabb806cdc0 2024-04-24T22:59:55.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.840+0000 7fabbe334640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fabb80af5a0 con 0x7fabb8161c30 2024-04-24T22:59:55.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.840+0000 7fabb7fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fabb8161c30 0x7fabb8067980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:55.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.840+0000 7fabbc8d1640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fabb806d230 0x7fabb80b4b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53934/0 (socket says 172.21.15.32:53934) 2024-04-24T22:59:55.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.840+0000 7fabbc8d1640 1 -- 172.21.15.32:0/1652516432 learned_addr learned my addr 172.21.15.32:0/1652516432 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:55.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.840+0000 7fabb7fff640 1 -- 172.21.15.32:0/1652516432 >> v1:172.21.15.121:6789/0 conn(0x7fabb806cdc0 legacy=0x7fabb80afa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:55.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.840+0000 7fabb7fff640 1 -- 172.21.15.32:0/1652516432 >> v1:172.21.15.32:6789/0 conn(0x7fabb806d230 legacy=0x7fabb80b4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:55.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.840+0000 7fabb7fff640 1 -- 172.21.15.32:0/1652516432 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabb80718c0 con 0x7fabb8161c30 2024-04-24T22:59:55.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.840+0000 7fabb6ffd640 1 -- 172.21.15.32:0/1652516432 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4240279030 0 0) 0x7fabb8071620 con 0x7fabb806d230 2024-04-24T22:59:55.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.841+0000 7fabb7fff640 1 --2- 172.21.15.32:0/1652516432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabb8161c30 0x7fabb8067980 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fabb80af720 tx=0x7faba802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cc9ec1fa6340707 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:55.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.841+0000 7fabb6ffd640 1 -- 172.21.15.32:0/1652516432 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba8037070 con 0x7fabb8161c30 2024-04-24T22:59:55.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.841+0000 7fabb6ffd640 1 -- 172.21.15.32:0/1652516432 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faba8032440 con 0x7fabb8161c30 2024-04-24T22:59:55.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.841+0000 7fabb6ffd640 1 -- 172.21.15.32:0/1652516432 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba8032810 con 0x7fabb8161c30 2024-04-24T22:59:55.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.842+0000 7fabbe334640 1 -- 172.21.15.32:0/1652516432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabb8161c30 msgr2=0x7fabb8067980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:55.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.842+0000 7fabbe334640 1 --2- 172.21.15.32:0/1652516432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabb8161c30 0x7fabb8067980 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fabb80af720 tx=0x7faba802a210 comp rx=0 tx=0).stop 2024-04-24T22:59:55.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.842+0000 7fabbe334640 1 -- 172.21.15.32:0/1652516432 shutdown_connections 2024-04-24T22:59:55.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.842+0000 7fabbe334640 1 --2- 172.21.15.32:0/1652516432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabb8161c30 0x7fabb8067980 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:55.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.842+0000 7fabbe334640 1 -- 172.21.15.32:0/1652516432 >> 172.21.15.32:0/1652516432 conn(0x7fabb80d0a60 msgr2=0x7fabb80d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:55.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.842+0000 7fabbe334640 1 -- 172.21.15.32:0/1652516432 shutdown_connections 2024-04-24T22:59:55.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.842+0000 7fabbe334640 1 -- 172.21.15.32:0/1652516432 wait complete. 2024-04-24T22:59:55.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.843+0000 7fabbe334640 1 Processor -- start 2024-04-24T22:59:55.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.843+0000 7fabbe334640 1 -- start start 2024-04-24T22:59:55.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.844+0000 7fabbe334640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabb806cdc0 0x7fabb81526a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:55.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.844+0000 7fabbe334640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabb806d230 0x7fabb8160180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:55.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.844+0000 7fabbe334640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fabb8161c30 0x7fabb81606c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:55.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.844+0000 7fabbe334640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fabb8070d60 con 0x7fabb806cdc0 2024-04-24T22:59:55.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.844+0000 7fabbe334640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fabb8162010 con 0x7fabb806d230 2024-04-24T22:59:55.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.844+0000 7fabbe334640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fabb8070ee0 con 0x7fabb8161c30 2024-04-24T22:59:55.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.844+0000 7fabb7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabb806cdc0 0x7fabb81526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:55.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.844+0000 7fabb7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabb806cdc0 0x7fabb81526a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50976/0 (socket says 172.21.15.32:50976) 2024-04-24T22:59:55.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.844+0000 7fabb7fff640 1 -- 172.21.15.32:0/1026268476 learned_addr learned my addr 172.21.15.32:0/1026268476 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:55.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.844+0000 7fabb77fe640 1 --2- 172.21.15.32:0/1026268476 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabb806d230 0x7fabb8160180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:55.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.844+0000 7fabb7fff640 1 -- 172.21.15.32:0/1026268476 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fabb8161c30 msgr2=0x7fabb81606c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:55.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.844+0000 7fabbc8d1640 1 --2- 172.21.15.32:0/1026268476 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fabb8161c30 0x7fabb81606c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:55.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.844+0000 7fabb7fff640 1 --2- 172.21.15.32:0/1026268476 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fabb8161c30 0x7fabb81606c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:55.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.844+0000 7fabb7fff640 1 -- 172.21.15.32:0/1026268476 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabb806d230 msgr2=0x7fabb8160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:55.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.844+0000 7fabb7fff640 1 --2- 172.21.15.32:0/1026268476 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabb806d230 0x7fabb8160180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:55.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.844+0000 7fabb7fff640 1 -- 172.21.15.32:0/1026268476 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabb815f220 con 0x7fabb806cdc0 2024-04-24T22:59:55.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.844+0000 7fabb77fe640 1 --2- 172.21.15.32:0/1026268476 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabb806d230 0x7fabb8160180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:59:55.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.844+0000 7fabbc8d1640 1 --2- 172.21.15.32:0/1026268476 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fabb8161c30 0x7fabb81606c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:59:55.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.844+0000 7fabb7fff640 1 --2- 172.21.15.32:0/1026268476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabb806cdc0 0x7fabb81526a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7faba8000c00 tx=0x7faba80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:55.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.844+0000 7fabb4ff9640 1 -- 172.21.15.32:0/1026268476 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba8037040 con 0x7fabb806cdc0 2024-04-24T22:59:55.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.844+0000 7fabb4ff9640 1 -- 172.21.15.32:0/1026268476 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faba8042070 con 0x7fabb806cdc0 2024-04-24T22:59:55.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.845+0000 7fabbe334640 1 -- 172.21.15.32:0/1026268476 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabb8157030 con 0x7fabb806cdc0 2024-04-24T22:59:55.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.845+0000 7fabb4ff9640 1 -- 172.21.15.32:0/1026268476 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba8030bd0 con 0x7fabb806cdc0 2024-04-24T22:59:55.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.845+0000 7fabbe334640 1 -- 172.21.15.32:0/1026268476 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabb81575f0 con 0x7fabb806cdc0 2024-04-24T22:59:55.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.845+0000 7fabbe334640 1 -- 172.21.15.32:0/1026268476 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabb8068430 con 0x7fabb806cdc0 2024-04-24T22:59:55.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.846+0000 7fabb4ff9640 1 -- 172.21.15.32:0/1026268476 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faba8047460 con 0x7fabb806cdc0 2024-04-24T22:59:55.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.846+0000 7fabb4ff9640 1 --2- 172.21.15.32:0/1026268476 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7faba406be30 0x7faba406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:55.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.846+0000 7fabb4ff9640 1 -- 172.21.15.32:0/1026268476 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7faba80ab840 con 0x7fabb806cdc0 2024-04-24T22:59:55.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.847+0000 7fabb77fe640 1 --2- 172.21.15.32:0/1026268476 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7faba406be30 0x7faba406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:55.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.847+0000 7fabb77fe640 1 --2- 172.21.15.32:0/1026268476 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7faba406be30 0x7faba406e2f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fabac000f60 tx=0x7fabac009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:55.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:55.850+0000 7fabb4ff9640 1 -- 172.21.15.32:0/1026268476 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7faba8032e20 con 0x7fabb806cdc0 2024-04-24T22:59:56.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.037+0000 7fabbe334640 1 -- 172.21.15.32:0/1026268476 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fabb80d18d0 con 0x7fabb806cdc0 2024-04-24T22:59:56.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.097+0000 7fabb4ff9640 1 -- 172.21.15.32:0/1026268476 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v29) ==== 138+0+0 (secure 0 0 0) 0x7faba8045360 con 0x7fabb806cdc0 2024-04-24T22:59:56.099 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2024-04-24T22:59:56.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.100+0000 7fabbe334640 1 -- 172.21.15.32:0/1026268476 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7faba406be30 msgr2=0x7faba406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:56.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.100+0000 7fabbe334640 1 --2- 172.21.15.32:0/1026268476 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7faba406be30 0x7faba406e2f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fabac000f60 tx=0x7fabac009040 comp rx=0 tx=0).stop 2024-04-24T22:59:56.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.100+0000 7fabbe334640 1 -- 172.21.15.32:0/1026268476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabb806cdc0 msgr2=0x7fabb81526a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:56.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.100+0000 7fabbe334640 1 --2- 172.21.15.32:0/1026268476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabb806cdc0 0x7fabb81526a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7faba8000c00 tx=0x7faba80046c0 comp rx=0 tx=0).stop 2024-04-24T22:59:56.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.100+0000 7fabbe334640 1 -- 172.21.15.32:0/1026268476 shutdown_connections 2024-04-24T22:59:56.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.100+0000 7fabbe334640 1 --2- 172.21.15.32:0/1026268476 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7faba406be30 0x7faba406e2f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:56.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.100+0000 7fabbe334640 1 --2- 172.21.15.32:0/1026268476 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fabb8161c30 0x7fabb81606c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:56.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.100+0000 7fabbe334640 1 --2- 172.21.15.32:0/1026268476 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabb806d230 0x7fabb8160180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:56.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.100+0000 7fabbe334640 1 --2- 172.21.15.32:0/1026268476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabb806cdc0 0x7fabb81526a0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:56.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.100+0000 7fabbe334640 1 -- 172.21.15.32:0/1026268476 >> 172.21.15.32:0/1026268476 conn(0x7fabb80d0a60 msgr2=0x7fabb81622f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:56.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.101+0000 7fabbe334640 1 -- 172.21.15.32:0/1026268476 shutdown_connections 2024-04-24T22:59:56.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.101+0000 7fabbe334640 1 -- 172.21.15.32:0/1026268476 wait complete. 2024-04-24T22:59:56.115 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-24T22:59:56.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.220+0000 7f91f0615640 1 Processor -- start 2024-04-24T22:59:56.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.220+0000 7f91f0615640 1 -- start start 2024-04-24T22:59:56.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.221+0000 7f91f0615640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f91e8158dd0 0x7f91e806b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:56.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.221+0000 7f91f0615640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91e815ef10 con 0x7f91e806b860 2024-04-24T22:59:56.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.221+0000 7f91f0615640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91e815f1f0 con 0x7f91e806dc60 2024-04-24T22:59:56.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.221+0000 7f91f0615640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f91e815f4d0 con 0x7f91e8158dd0 2024-04-24T22:59:56.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.221+0000 7f91edbb0640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f91e806b860 0x7f91e806d550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53936/0 (socket says 172.21.15.32:53936) 2024-04-24T22:59:56.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.221+0000 7f91edbb0640 1 -- 172.21.15.32:0/81402259 learned_addr learned my addr 172.21.15.32:0/81402259 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:56.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.222+0000 7f91ed3af640 1 -- 172.21.15.32:0/81402259 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4072661964 0 0) 0x7f91e815ef10 con 0x7f91e806b860 2024-04-24T22:59:56.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.222+0000 7f91ee3b1640 1 --2- 172.21.15.32:0/81402259 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f91e8158dd0 0x7f91e806b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:56.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.222+0000 7f91ed3af640 1 -- 172.21.15.32:0/81402259 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f91d4003520 con 0x7f91e806b860 2024-04-24T22:59:56.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.222+0000 7f91ed3af640 1 -- 172.21.15.32:0/81402259 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 863771868 0 0) 0x7f91d4003520 con 0x7f91e806b860 2024-04-24T22:59:56.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.222+0000 7f91ed3af640 1 -- 172.21.15.32:0/81402259 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f91e8158dd0 msgr2=0x7f91e806b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:56.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.222+0000 7f91ed3af640 1 --2- 172.21.15.32:0/81402259 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f91e8158dd0 0x7f91e806b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:56.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.222+0000 7f91ed3af640 1 -- 172.21.15.32:0/81402259 >> v1:172.21.15.121:6789/0 conn(0x7f91e806dc60 legacy=0x7f91e80b5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:56.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.222+0000 7f91ed3af640 1 -- 172.21.15.32:0/81402259 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91e815f7b0 con 0x7f91e806b860 2024-04-24T22:59:56.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.222+0000 7f91ed3af640 1 -- 172.21.15.32:0/81402259 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f91d8001670 con 0x7f91e806b860 2024-04-24T22:59:56.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.222+0000 7f91ed3af640 1 -- 172.21.15.32:0/81402259 >> v1:172.21.15.32:6789/0 conn(0x7f91e806b860 legacy=0x7f91e806d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:56.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.222+0000 7f91ed3af640 1 --2- 172.21.15.32:0/81402259 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f91e8158dd0 0x7f91d40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:56.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.222+0000 7f91ed3af640 1 --2- 172.21.15.32:0/81402259 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91d4003f60 0x7f91d4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:56.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.222+0000 7f91ed3af640 1 --2- 172.21.15.32:0/81402259 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91d4004d40 0x7f91d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:56.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.222+0000 7f91ed3af640 1 -- 172.21.15.32:0/81402259 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f91e815f4d0 con 0x7f91d4004d40 2024-04-24T22:59:56.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.222+0000 7f91ed3af640 1 -- 172.21.15.32:0/81402259 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f91d400baa0 con 0x7f91d4003f60 2024-04-24T22:59:56.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.222+0000 7f91ed3af640 1 -- 172.21.15.32:0/81402259 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f91d400bca0 con 0x7f91e8158dd0 2024-04-24T22:59:56.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.222+0000 7f91eebb2640 1 --2- 172.21.15.32:0/81402259 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91d4004d40 0x7f91d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:56.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.222+0000 7f91ee3b1640 1 --2- 172.21.15.32:0/81402259 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f91e8158dd0 0x7f91d40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:56.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.222+0000 7f91edbb0640 1 --2- 172.21.15.32:0/81402259 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91d4003f60 0x7f91d4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:56.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.223+0000 7f91eebb2640 1 -- 172.21.15.32:0/81402259 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f91e8158dd0 msgr2=0x7f91d40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:56.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.223+0000 7f91eebb2640 1 --2- 172.21.15.32:0/81402259 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f91e8158dd0 0x7f91d40039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:56.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.223+0000 7f91eebb2640 1 -- 172.21.15.32:0/81402259 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91d4003f60 msgr2=0x7f91d4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:56.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.223+0000 7f91eebb2640 1 --2- 172.21.15.32:0/81402259 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91d4003f60 0x7f91d4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:56.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.223+0000 7f91eebb2640 1 -- 172.21.15.32:0/81402259 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91d4003520 con 0x7f91d4004d40 2024-04-24T22:59:56.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.223+0000 7f91eebb2640 1 --2- 172.21.15.32:0/81402259 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91d4004d40 0x7f91d40051c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f91e0001e50 tx=0x7f91d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc6f63721868f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:56.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.223+0000 7f91ed3af640 1 -- 172.21.15.32:0/81402259 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e0001a70 con 0x7f91d4004d40 2024-04-24T22:59:56.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.223+0000 7f91ed3af640 1 -- 172.21.15.32:0/81402259 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91e0003970 con 0x7f91d4004d40 2024-04-24T22:59:56.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.223+0000 7f91ed3af640 1 -- 172.21.15.32:0/81402259 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e0003d60 con 0x7f91d4004d40 2024-04-24T22:59:56.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.224+0000 7f91f0615640 1 -- 172.21.15.32:0/81402259 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91d4004d40 msgr2=0x7f91d40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:56.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.224+0000 7f91f0615640 1 --2- 172.21.15.32:0/81402259 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91d4004d40 0x7f91d40051c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f91e0001e50 tx=0x7f91d4005830 comp rx=0 tx=0).stop 2024-04-24T22:59:56.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.224+0000 7f91eebb2640 1 -- 172.21.15.32:0/81402259 reap_dead start 2024-04-24T22:59:56.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.224+0000 7f91f0615640 1 -- 172.21.15.32:0/81402259 shutdown_connections 2024-04-24T22:59:56.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.224+0000 7f91f0615640 1 -- 172.21.15.32:0/81402259 >> 172.21.15.32:0/81402259 conn(0x7f91e806e800 msgr2=0x7f91e806ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:56.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.224+0000 7f91f0615640 1 -- 172.21.15.32:0/81402259 shutdown_connections 2024-04-24T22:59:56.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.224+0000 7f91f0615640 1 -- 172.21.15.32:0/81402259 wait complete. 2024-04-24T22:59:56.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.225+0000 7f91f0615640 1 Processor -- start 2024-04-24T22:59:56.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.225+0000 7f91f0615640 1 -- start start 2024-04-24T22:59:56.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.226+0000 7f91f0615640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f91d4004d40 0x7f91e815b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:56.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.226+0000 7f91f0615640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91e815e0d0 0x7f91e81f7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:56.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.226+0000 7f91f0615640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91e80b45b0 0x7f91e80b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:56.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.226+0000 7f91f0615640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f91e806bfe0 con 0x7f91e815e0d0 2024-04-24T22:59:56.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.226+0000 7f91f0615640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f91e815fcd0 con 0x7f91e80b45b0 2024-04-24T22:59:56.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.226+0000 7f91f0615640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f91e81591b0 con 0x7f91d4004d40 2024-04-24T22:59:56.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.226+0000 7f91edbb0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91e815e0d0 0x7f91e81f7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:56.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.226+0000 7f91edbb0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91e815e0d0 0x7f91e81f7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50994/0 (socket says 172.21.15.32:50994) 2024-04-24T22:59:56.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.226+0000 7f91edbb0640 1 -- 172.21.15.32:0/3351414604 learned_addr learned my addr 172.21.15.32:0/3351414604 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:56.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.226+0000 7f91eebb2640 1 --2- 172.21.15.32:0/3351414604 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91e80b45b0 0x7f91e80b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:56.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.226+0000 7f91ee3b1640 1 --2- 172.21.15.32:0/3351414604 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f91d4004d40 0x7f91e815b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:56.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.226+0000 7f91edbb0640 1 -- 172.21.15.32:0/3351414604 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f91d4004d40 msgr2=0x7f91e815b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:56.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.226+0000 7f91edbb0640 1 --2- 172.21.15.32:0/3351414604 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f91d4004d40 0x7f91e815b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:56.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.226+0000 7f91edbb0640 1 -- 172.21.15.32:0/3351414604 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91e80b45b0 msgr2=0x7f91e80b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:56.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.226+0000 7f91edbb0640 1 --2- 172.21.15.32:0/3351414604 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91e80b45b0 0x7f91e80b4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:56.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.226+0000 7f91edbb0640 1 -- 172.21.15.32:0/3351414604 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91e815bf30 con 0x7f91e815e0d0 2024-04-24T22:59:56.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.227+0000 7f91edbb0640 1 --2- 172.21.15.32:0/3351414604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91e815e0d0 0x7f91e81f7610 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f91d80072c0 tx=0x7f91d80076c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:56.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.227+0000 7f91deffd640 1 -- 172.21.15.32:0/3351414604 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d800f960 con 0x7f91e815e0d0 2024-04-24T22:59:56.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.227+0000 7f91deffd640 1 -- 172.21.15.32:0/3351414604 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91d8018070 con 0x7f91e815e0d0 2024-04-24T22:59:56.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.227+0000 7f91f0615640 1 -- 172.21.15.32:0/3351414604 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91e815c200 con 0x7f91e815e0d0 2024-04-24T22:59:56.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.227+0000 7f91deffd640 1 -- 172.21.15.32:0/3351414604 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d800fd60 con 0x7f91e815e0d0 2024-04-24T22:59:56.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.227+0000 7f91f0615640 1 -- 172.21.15.32:0/3351414604 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91e815c7c0 con 0x7f91e815e0d0 2024-04-24T22:59:56.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.228+0000 7f91f0615640 1 -- 172.21.15.32:0/3351414604 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91e8068430 con 0x7f91e815e0d0 2024-04-24T22:59:56.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.228+0000 7f91deffd640 1 -- 172.21.15.32:0/3351414604 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f91d801d460 con 0x7f91e815e0d0 2024-04-24T22:59:56.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.229+0000 7f91deffd640 1 --2- 172.21.15.32:0/3351414604 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f91cc06be30 0x7f91cc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:56.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.229+0000 7f91deffd640 1 -- 172.21.15.32:0/3351414604 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5227+0+0 (secure 0 0 0) 0x7f91d808ac70 con 0x7f91e815e0d0 2024-04-24T22:59:56.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.229+0000 7f91ee3b1640 1 --2- 172.21.15.32:0/3351414604 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f91cc06be30 0x7f91cc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:56.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.230+0000 7f91ee3b1640 1 --2- 172.21.15.32:0/3351414604 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f91cc06be30 0x7f91cc06e2f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f91e4009bc0 tx=0x7f91e4009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:56.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.234+0000 7f91deffd640 1 -- 172.21.15.32:0/3351414604 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f91d80597e0 con 0x7f91e815e0d0 2024-04-24T22:59:56.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:56.416+0000 7f91f0615640 1 -- 172.21.15.32:0/3351414604 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f91e80b6c00 con 0x7f91e815e0d0 2024-04-24T22:59:57.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.102+0000 7f91deffd640 1 -- 172.21.15.32:0/3351414604 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v30) ==== 170+0+0 (secure 0 0 0) 0x7f91d80590e0 con 0x7f91e815e0d0 2024-04-24T22:59:57.104 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2024-04-24T22:59:57.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.106+0000 7f91f0615640 1 -- 172.21.15.32:0/3351414604 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f91cc06be30 msgr2=0x7f91cc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:57.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.106+0000 7f91f0615640 1 --2- 172.21.15.32:0/3351414604 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f91cc06be30 0x7f91cc06e2f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f91e4009bc0 tx=0x7f91e4009190 comp rx=0 tx=0).stop 2024-04-24T22:59:57.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.106+0000 7f91f0615640 1 -- 172.21.15.32:0/3351414604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91e815e0d0 msgr2=0x7f91e81f7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:57.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.106+0000 7f91f0615640 1 --2- 172.21.15.32:0/3351414604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91e815e0d0 0x7f91e81f7610 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f91d80072c0 tx=0x7f91d80076c0 comp rx=0 tx=0).stop 2024-04-24T22:59:57.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.108+0000 7f91f0615640 1 -- 172.21.15.32:0/3351414604 shutdown_connections 2024-04-24T22:59:57.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.108+0000 7f91f0615640 1 --2- 172.21.15.32:0/3351414604 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f91cc06be30 0x7f91cc06e2f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:57.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.108+0000 7f91f0615640 1 --2- 172.21.15.32:0/3351414604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91e815e0d0 0x7f91e81f7610 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:57.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.108+0000 7f91f0615640 1 --2- 172.21.15.32:0/3351414604 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91e80b45b0 0x7f91e80b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:57.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.108+0000 7f91f0615640 1 --2- 172.21.15.32:0/3351414604 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f91d4004d40 0x7f91e815b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:57.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.108+0000 7f91f0615640 1 -- 172.21.15.32:0/3351414604 >> 172.21.15.32:0/3351414604 conn(0x7f91e806e800 msgr2=0x7f91e8157f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:57.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.110+0000 7f91f0615640 1 -- 172.21.15.32:0/3351414604 shutdown_connections 2024-04-24T22:59:57.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.112+0000 7f91f0615640 1 -- 172.21.15.32:0/3351414604 wait complete. 2024-04-24T22:59:57.127 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-24T22:59:57.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.229+0000 7f20fb31b640 1 Processor -- start 2024-04-24T22:59:57.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.230+0000 7f20fb31b640 1 -- start start 2024-04-24T22:59:57.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.230+0000 7f20fb31b640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20f40b4320 0x7f20f40b6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:57.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.230+0000 7f20fb31b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20f40cd910 con 0x7f20f415edd0 2024-04-24T22:59:57.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.230+0000 7f20fb31b640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20f40cdbf0 con 0x7f20f40b5560 2024-04-24T22:59:57.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.230+0000 7f20fb31b640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f20f40cded0 con 0x7f20f40b4320 2024-04-24T22:59:57.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.231+0000 7f20f88b6640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f20f415edd0 0x7f20f40b5940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53942/0 (socket says 172.21.15.32:53942) 2024-04-24T22:59:57.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.231+0000 7f20f88b6640 1 -- 172.21.15.32:0/1615428647 learned_addr learned my addr 172.21.15.32:0/1615428647 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:57.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.231+0000 7f20f98b8640 1 --2- 172.21.15.32:0/1615428647 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20f40b4320 0x7f20f40b6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:57.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.231+0000 7f20ebfff640 1 -- 172.21.15.32:0/1615428647 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2771653570 0 0) 0x7f20f40cd910 con 0x7f20f415edd0 2024-04-24T22:59:57.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.231+0000 7f20ebfff640 1 -- 172.21.15.32:0/1615428647 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f20e0003520 con 0x7f20f415edd0 2024-04-24T22:59:57.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.231+0000 7f20ebfff640 1 -- 172.21.15.32:0/1615428647 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2493644359 0 0) 0x7f20f40cdbf0 con 0x7f20f40b5560 2024-04-24T22:59:57.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.231+0000 7f20ebfff640 1 -- 172.21.15.32:0/1615428647 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f20f40cd910 con 0x7f20f40b5560 2024-04-24T22:59:57.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.231+0000 7f20ebfff640 1 -- 172.21.15.32:0/1615428647 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1890131024 0 0) 0x7f20e0003520 con 0x7f20f415edd0 2024-04-24T22:59:57.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.231+0000 7f20ebfff640 1 -- 172.21.15.32:0/1615428647 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20f40b4320 msgr2=0x7f20f40b6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:57.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.231+0000 7f20ebfff640 1 --2- 172.21.15.32:0/1615428647 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20f40b4320 0x7f20f40b6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:57.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.231+0000 7f20ebfff640 1 -- 172.21.15.32:0/1615428647 >> v1:172.21.15.121:6789/0 conn(0x7f20f40b5560 legacy=0x7f20f415e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:57.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.231+0000 7f20ebfff640 1 -- 172.21.15.32:0/1615428647 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20f40ce1b0 con 0x7f20f415edd0 2024-04-24T22:59:57.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.231+0000 7f20ebfff640 1 -- 172.21.15.32:0/1615428647 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f20e4001690 con 0x7f20f415edd0 2024-04-24T22:59:57.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.231+0000 7f20f98b8640 1 --2- 172.21.15.32:0/1615428647 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20f40b4320 0x7f20f40b6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:59:57.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.231+0000 7f20ebfff640 1 -- 172.21.15.32:0/1615428647 >> v1:172.21.15.32:6789/0 conn(0x7f20f415edd0 legacy=0x7f20f40b5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:57.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.231+0000 7f20ebfff640 1 --2- 172.21.15.32:0/1615428647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20e0003b70 0x7f20e0003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:57.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.231+0000 7f20ebfff640 1 --2- 172.21.15.32:0/1615428647 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20f40b4320 0x7f20e0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:57.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.231+0000 7f20ebfff640 1 --2- 172.21.15.32:0/1615428647 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20e0004d40 0x7f20e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:57.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.231+0000 7f20ebfff640 1 -- 172.21.15.32:0/1615428647 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f20f40cded0 con 0x7f20e0004d40 2024-04-24T22:59:57.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.231+0000 7f20ebfff640 1 -- 172.21.15.32:0/1615428647 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f20e000baa0 con 0x7f20e0003b70 2024-04-24T22:59:57.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.231+0000 7f20ebfff640 1 -- 172.21.15.32:0/1615428647 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f20e000bca0 con 0x7f20f40b4320 2024-04-24T22:59:57.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.231+0000 7f20f98b8640 1 --2- 172.21.15.32:0/1615428647 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20e0004d40 0x7f20e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:57.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.231+0000 7f20f90b7640 1 --2- 172.21.15.32:0/1615428647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20e0003b70 0x7f20e0003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:57.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.232+0000 7f20f88b6640 1 --2- 172.21.15.32:0/1615428647 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20f40b4320 0x7f20e0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:57.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.232+0000 7f20f98b8640 1 -- 172.21.15.32:0/1615428647 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20f40b4320 msgr2=0x7f20e0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:57.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.232+0000 7f20f98b8640 1 --2- 172.21.15.32:0/1615428647 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20f40b4320 0x7f20e0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:57.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.232+0000 7f20f98b8640 1 -- 172.21.15.32:0/1615428647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20e0003b70 msgr2=0x7f20e0003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:57.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.232+0000 7f20f98b8640 1 --2- 172.21.15.32:0/1615428647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20e0003b70 0x7f20e0003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:57.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.232+0000 7f20f98b8640 1 -- 172.21.15.32:0/1615428647 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20e0003520 con 0x7f20e0004d40 2024-04-24T22:59:57.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.232+0000 7f20f90b7640 1 --2- 172.21.15.32:0/1615428647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20e0003b70 0x7f20e0003f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:59:57.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.233+0000 7f20f98b8640 1 --2- 172.21.15.32:0/1615428647 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20e0004d40 0x7f20e00051c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f20ec00f4c0 tx=0x7f20e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eef16ff61e190657 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:57.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.233+0000 7f20ebfff640 1 -- 172.21.15.32:0/1615428647 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ec033070 con 0x7f20e0004d40 2024-04-24T22:59:57.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.233+0000 7f20ebfff640 1 -- 172.21.15.32:0/1615428647 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20ec00a2d0 con 0x7f20e0004d40 2024-04-24T22:59:57.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.233+0000 7f20ebfff640 1 -- 172.21.15.32:0/1615428647 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ec00f140 con 0x7f20e0004d40 2024-04-24T22:59:57.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.233+0000 7f20fb31b640 1 -- 172.21.15.32:0/1615428647 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20e0004d40 msgr2=0x7f20e00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:57.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.233+0000 7f20fb31b640 1 --2- 172.21.15.32:0/1615428647 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20e0004d40 0x7f20e00051c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f20ec00f4c0 tx=0x7f20e0005830 comp rx=0 tx=0).stop 2024-04-24T22:59:57.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.233+0000 7f20f98b8640 1 -- 172.21.15.32:0/1615428647 reap_dead start 2024-04-24T22:59:57.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.233+0000 7f20fb31b640 1 -- 172.21.15.32:0/1615428647 shutdown_connections 2024-04-24T22:59:57.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.233+0000 7f20fb31b640 1 -- 172.21.15.32:0/1615428647 >> 172.21.15.32:0/1615428647 conn(0x7f20f406b110 msgr2=0x7f20f406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:57.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.233+0000 7f20fb31b640 1 -- 172.21.15.32:0/1615428647 shutdown_connections 2024-04-24T22:59:57.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.233+0000 7f20fb31b640 1 -- 172.21.15.32:0/1615428647 wait complete. 2024-04-24T22:59:57.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.235+0000 7f20fb31b640 1 Processor -- start 2024-04-24T22:59:57.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.235+0000 7f20fb31b640 1 -- start start 2024-04-24T22:59:57.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.235+0000 7f20fb31b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20f4151350 0x7f20f4072d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:57.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.235+0000 7f20fb31b640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20f4151730 0x7f20f40aa4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:57.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.235+0000 7f20fb31b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20f4071e40 0x7f20f41577a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:57.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.235+0000 7f20fb31b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f20f40cd540 con 0x7f20f4071e40 2024-04-24T22:59:57.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.235+0000 7f20fb31b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f20f40cd3c0 con 0x7f20f4151350 2024-04-24T22:59:57.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.235+0000 7f20fb31b640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f20f40cd6c0 con 0x7f20f4151730 2024-04-24T22:59:57.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.235+0000 7f20f98b8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20f4071e40 0x7f20f41577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:57.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.235+0000 7f20f98b8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20f4071e40 0x7f20f41577a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51012/0 (socket says 172.21.15.32:51012) 2024-04-24T22:59:57.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.235+0000 7f20f98b8640 1 -- 172.21.15.32:0/1168051318 learned_addr learned my addr 172.21.15.32:0/1168051318 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:57.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.235+0000 7f20f90b7640 1 --2- 172.21.15.32:0/1168051318 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20f4151350 0x7f20f4072d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:57.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.235+0000 7f20f88b6640 1 --2- 172.21.15.32:0/1168051318 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20f4151730 0x7f20f40aa4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:57.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.236+0000 7f20f98b8640 1 -- 172.21.15.32:0/1168051318 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20f4151730 msgr2=0x7f20f40aa4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:57.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.236+0000 7f20f98b8640 1 --2- 172.21.15.32:0/1168051318 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20f4151730 0x7f20f40aa4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:57.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.236+0000 7f20f98b8640 1 -- 172.21.15.32:0/1168051318 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20f4151350 msgr2=0x7f20f4072d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:57.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.236+0000 7f20f98b8640 1 --2- 172.21.15.32:0/1168051318 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20f4151350 0x7f20f4072d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:57.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.236+0000 7f20f98b8640 1 -- 172.21.15.32:0/1168051318 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20f40aa9e0 con 0x7f20f4071e40 2024-04-24T22:59:57.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.236+0000 7f20f98b8640 1 --2- 172.21.15.32:0/1168051318 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20f4071e40 0x7f20f41577a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f20ec00f5b0 tx=0x7f20ec00a860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:57.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.236+0000 7f20e9ffb640 1 -- 172.21.15.32:0/1168051318 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ec033070 con 0x7f20f4071e40 2024-04-24T22:59:57.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.236+0000 7f20e9ffb640 1 -- 172.21.15.32:0/1168051318 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20ec00ab60 con 0x7f20f4071e40 2024-04-24T22:59:57.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.236+0000 7f20e9ffb640 1 -- 172.21.15.32:0/1168051318 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f00014f0 con 0x7f20f4071e40 2024-04-24T22:59:57.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.236+0000 7f20fb31b640 1 -- 172.21.15.32:0/1168051318 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20f4070da0 con 0x7f20f4071e40 2024-04-24T22:59:57.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.236+0000 7f20fb31b640 1 -- 172.21.15.32:0/1168051318 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20f4071360 con 0x7f20f4071e40 2024-04-24T22:59:57.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.237+0000 7f20fb31b640 1 -- 172.21.15.32:0/1168051318 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20f4068430 con 0x7f20f4071e40 2024-04-24T22:59:57.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.238+0000 7f20e9ffb640 1 -- 172.21.15.32:0/1168051318 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f20ec036070 con 0x7f20f4071e40 2024-04-24T22:59:57.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.238+0000 7f20e9ffb640 1 --2- 172.21.15.32:0/1168051318 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f20d806be30 0x7f20d806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:57.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.239+0000 7f20e9ffb640 1 -- 172.21.15.32:0/1168051318 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5609+0+0 (secure 0 0 0) 0x7f20ec0ab1b0 con 0x7f20f4071e40 2024-04-24T22:59:57.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.239+0000 7f20f90b7640 1 --2- 172.21.15.32:0/1168051318 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f20d806be30 0x7f20d806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:57.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.239+0000 7f20f90b7640 1 --2- 172.21.15.32:0/1168051318 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f20d806be30 0x7f20d806e2f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f20f0000c00 tx=0x7f20f0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:57.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.243+0000 7f20e9ffb640 1 -- 172.21.15.32:0/1168051318 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f20ec045060 con 0x7f20f4071e40 2024-04-24T22:59:57.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:57.467+0000 7f20fb31b640 1 -- 172.21.15.32:0/1168051318 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f20f40afd60 con 0x7f20f4071e40 2024-04-24T22:59:58.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.104+0000 7f20e9ffb640 1 -- 172.21.15.32:0/1168051318 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v11) ==== 172+0+0 (secure 0 0 0) 0x7f20f40aa9e0 con 0x7f20f4071e40 2024-04-24T22:59:58.105 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 4 and data pool 5 2024-04-24T22:59:58.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.108+0000 7f20fb31b640 1 -- 172.21.15.32:0/1168051318 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f20d806be30 msgr2=0x7f20d806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:58.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.108+0000 7f20fb31b640 1 --2- 172.21.15.32:0/1168051318 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f20d806be30 0x7f20d806e2f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f20f0000c00 tx=0x7f20f0009040 comp rx=0 tx=0).stop 2024-04-24T22:59:58.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.108+0000 7f20fb31b640 1 -- 172.21.15.32:0/1168051318 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20f4071e40 msgr2=0x7f20f41577a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:58.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.108+0000 7f20fb31b640 1 --2- 172.21.15.32:0/1168051318 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20f4071e40 0x7f20f41577a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f20ec00f5b0 tx=0x7f20ec00a860 comp rx=0 tx=0).stop 2024-04-24T22:59:58.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.109+0000 7f20fb31b640 1 -- 172.21.15.32:0/1168051318 shutdown_connections 2024-04-24T22:59:58.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.109+0000 7f20fb31b640 1 --2- 172.21.15.32:0/1168051318 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f20d806be30 0x7f20d806e2f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:58.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.109+0000 7f20fb31b640 1 --2- 172.21.15.32:0/1168051318 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20f4071e40 0x7f20f41577a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:58.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.109+0000 7f20fb31b640 1 --2- 172.21.15.32:0/1168051318 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20f4151730 0x7f20f40aa4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:58.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.109+0000 7f20fb31b640 1 --2- 172.21.15.32:0/1168051318 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20f4151350 0x7f20f4072d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:58.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.109+0000 7f20fb31b640 1 -- 172.21.15.32:0/1168051318 >> 172.21.15.32:0/1168051318 conn(0x7f20f406b110 msgr2=0x7f20f415a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:58.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.110+0000 7f20fb31b640 1 -- 172.21.15.32:0/1168051318 shutdown_connections 2024-04-24T22:59:58.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.111+0000 7f20fb31b640 1 -- 172.21.15.32:0/1168051318 wait complete. 2024-04-24T22:59:58.139 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:59:58.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.242+0000 7f475c170640 1 Processor -- start 2024-04-24T22:59:58.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.242+0000 7f475c170640 1 -- start start 2024-04-24T22:59:58.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.243+0000 7f475c170640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f475406c5f0 0x7f475406c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:58.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.243+0000 7f475c170640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47540b6430 con 0x7f47540af890 2024-04-24T22:59:58.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.243+0000 7f475c170640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4754156f80 con 0x7f47540b44d0 2024-04-24T22:59:58.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.243+0000 7f475c170640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f47540b6150 con 0x7f475406c5f0 2024-04-24T22:59:58.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.243+0000 7f475a70d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f47540af890 0x7f475415cac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53956/0 (socket says 172.21.15.32:53956) 2024-04-24T22:59:58.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.243+0000 7f475a70d640 1 -- 172.21.15.32:0/302187071 learned_addr learned my addr 172.21.15.32:0/302187071 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:58.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.243+0000 7f4759f0c640 1 --2- 172.21.15.32:0/302187071 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f475406c5f0 0x7f475406c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:58.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.243+0000 7f4758f0a640 1 -- 172.21.15.32:0/302187071 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1015789297 0 0) 0x7f47540b6430 con 0x7f47540af890 2024-04-24T22:59:58.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.243+0000 7f4758f0a640 1 -- 172.21.15.32:0/302187071 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4740003520 con 0x7f47540af890 2024-04-24T22:59:58.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.244+0000 7f4758f0a640 1 -- 172.21.15.32:0/302187071 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 741069157 0 0) 0x7f4754156f80 con 0x7f47540b44d0 2024-04-24T22:59:58.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.244+0000 7f4758f0a640 1 -- 172.21.15.32:0/302187071 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f47540b6430 con 0x7f47540b44d0 2024-04-24T22:59:58.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.244+0000 7f4758f0a640 1 -- 172.21.15.32:0/302187071 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1883569592 0 0) 0x7f4740003520 con 0x7f47540af890 2024-04-24T22:59:58.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.244+0000 7f4759f0c640 1 -- 172.21.15.32:0/302187071 >> v1:172.21.15.121:6789/0 conn(0x7f47540b44d0 legacy=0x7f47540b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:58.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.244+0000 7f4759f0c640 1 -- 172.21.15.32:0/302187071 >> v1:172.21.15.32:6789/0 conn(0x7f47540af890 legacy=0x7f475415cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:58.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.244+0000 7f4759f0c640 1 -- 172.21.15.32:0/302187071 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47540b39f0 con 0x7f475406c5f0 2024-04-24T22:59:58.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.244+0000 7f4759f0c640 1 --2- 172.21.15.32:0/302187071 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f475406c5f0 0x7f475406c9d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f474c0097d0 tx=0x7f474c02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80791af79bd0c15e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:58.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.245+0000 7f4758f0a640 1 -- 172.21.15.32:0/302187071 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474c037070 con 0x7f475406c5f0 2024-04-24T22:59:58.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.245+0000 7f4758f0a640 1 -- 172.21.15.32:0/302187071 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f474c02ab90 con 0x7f475406c5f0 2024-04-24T22:59:58.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.245+0000 7f4758f0a640 1 -- 172.21.15.32:0/302187071 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47500019f0 con 0x7f475406c5f0 2024-04-24T22:59:58.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.245+0000 7f475c170640 1 -- 172.21.15.32:0/302187071 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f475406c5f0 msgr2=0x7f475406c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:58.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.245+0000 7f475c170640 1 --2- 172.21.15.32:0/302187071 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f475406c5f0 0x7f475406c9d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f474c0097d0 tx=0x7f474c02a100 comp rx=0 tx=0).stop 2024-04-24T22:59:58.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.245+0000 7f475c170640 1 -- 172.21.15.32:0/302187071 shutdown_connections 2024-04-24T22:59:58.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.245+0000 7f475c170640 1 --2- 172.21.15.32:0/302187071 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f475406c5f0 0x7f475406c9d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:58.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.245+0000 7f475c170640 1 -- 172.21.15.32:0/302187071 >> 172.21.15.32:0/302187071 conn(0x7f475406b110 msgr2=0x7f475406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:58.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.245+0000 7f475c170640 1 -- 172.21.15.32:0/302187071 shutdown_connections 2024-04-24T22:59:58.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.245+0000 7f475c170640 1 -- 172.21.15.32:0/302187071 wait complete. 2024-04-24T22:59:58.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.246+0000 7f475c170640 1 Processor -- start 2024-04-24T22:59:58.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.246+0000 7f475c170640 1 -- start start 2024-04-24T22:59:58.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.247+0000 7f475c170640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f475406c5f0 0x7f4754159e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:58.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.247+0000 7f475c170640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47540af890 0x7f4754154ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:58.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.247+0000 7f475c170640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47540b44d0 0x7f4754143890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:58.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.247+0000 7f475c170640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f475406cf10 con 0x7f47540af890 2024-04-24T22:59:58.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.247+0000 7f475c170640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f47540b3fb0 con 0x7f47540b44d0 2024-04-24T22:59:58.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.247+0000 7f475c170640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f475415d1d0 con 0x7f475406c5f0 2024-04-24T22:59:58.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.247+0000 7f475970b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47540af890 0x7f4754154ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:58.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.247+0000 7f475970b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47540af890 0x7f4754154ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51026/0 (socket says 172.21.15.32:51026) 2024-04-24T22:59:58.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.247+0000 7f475970b640 1 -- 172.21.15.32:0/2986125535 learned_addr learned my addr 172.21.15.32:0/2986125535 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:58.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.247+0000 7f4759f0c640 1 --2- 172.21.15.32:0/2986125535 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f475406c5f0 0x7f4754159e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:58.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.247+0000 7f475a70d640 1 --2- 172.21.15.32:0/2986125535 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47540b44d0 0x7f4754143890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:58.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.247+0000 7f475970b640 1 -- 172.21.15.32:0/2986125535 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f475406c5f0 msgr2=0x7f4754159e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:58.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.247+0000 7f475970b640 1 --2- 172.21.15.32:0/2986125535 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f475406c5f0 0x7f4754159e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:58.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.247+0000 7f475970b640 1 -- 172.21.15.32:0/2986125535 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47540b44d0 msgr2=0x7f4754143890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:58.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.247+0000 7f475970b640 1 --2- 172.21.15.32:0/2986125535 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47540b44d0 0x7f4754143890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:58.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.247+0000 7f475970b640 1 -- 172.21.15.32:0/2986125535 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f475415c530 con 0x7f47540af890 2024-04-24T22:59:58.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.247+0000 7f4759f0c640 1 --2- 172.21.15.32:0/2986125535 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f475406c5f0 0x7f4754159e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:58.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.248+0000 7f475970b640 1 --2- 172.21.15.32:0/2986125535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47540af890 0x7f4754154ea0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f474400c460 tx=0x7f474400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:58.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.248+0000 7f474a7fc640 1 -- 172.21.15.32:0/2986125535 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4744017070 con 0x7f47540af890 2024-04-24T22:59:58.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.248+0000 7f474a7fc640 1 -- 172.21.15.32:0/2986125535 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47440052f0 con 0x7f47540af890 2024-04-24T22:59:58.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.248+0000 7f474a7fc640 1 -- 172.21.15.32:0/2986125535 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4744003830 con 0x7f47540af890 2024-04-24T22:59:58.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.248+0000 7f475c170640 1 -- 172.21.15.32:0/2986125535 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f475415bf70 con 0x7f47540af890 2024-04-24T22:59:58.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.248+0000 7f475c170640 1 -- 172.21.15.32:0/2986125535 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4754157290 con 0x7f47540af890 2024-04-24T22:59:58.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.249+0000 7f475c170640 1 -- 172.21.15.32:0/2986125535 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4754068430 con 0x7f47540af890 2024-04-24T22:59:58.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.250+0000 7f474a7fc640 1 -- 172.21.15.32:0/2986125535 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f474401a070 con 0x7f47540af890 2024-04-24T22:59:58.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.250+0000 7f474a7fc640 1 --2- 172.21.15.32:0/2986125535 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f473806be30 0x7f473806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:58.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.250+0000 7f474a7fc640 1 -- 172.21.15.32:0/2986125535 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f474408a1e0 con 0x7f47540af890 2024-04-24T22:59:58.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.250+0000 7f4759f0c640 1 --2- 172.21.15.32:0/2986125535 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f473806be30 0x7f473806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:58.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.251+0000 7f4759f0c640 1 --2- 172.21.15.32:0/2986125535 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f473806be30 0x7f473806e2f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f474c034000 tx=0x7f474c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:58.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.254+0000 7f474a7fc640 1 -- 172.21.15.32:0/2986125535 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4744058ce0 con 0x7f47540af890 2024-04-24T22:59:58.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.438+0000 7f475c170640 1 -- 172.21.15.32:0/2986125535 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4754067430 con 0x7f47540af890 2024-04-24T22:59:58.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.440+0000 7f474a7fc640 1 -- 172.21.15.32:0/2986125535 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) ==== 74+0+16634 (secure 0 0 0) 0x7f47440585e0 con 0x7f47540af890 2024-04-24T22:59:58.441 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:59:58.441 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":31,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T22:59:57.468463+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:59:56.068812+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-24T22:59:56.447402+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:59:58.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.441+0000 7f475c170640 1 -- 172.21.15.32:0/2986125535 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f473806be30 msgr2=0x7f473806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:58.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.441+0000 7f475c170640 1 --2- 172.21.15.32:0/2986125535 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f473806be30 0x7f473806e2f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f474c034000 tx=0x7f474c02e000 comp rx=0 tx=0).stop 2024-04-24T22:59:58.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.441+0000 7f475c170640 1 -- 172.21.15.32:0/2986125535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47540af890 msgr2=0x7f4754154ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:58.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.441+0000 7f475c170640 1 --2- 172.21.15.32:0/2986125535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47540af890 0x7f4754154ea0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f474400c460 tx=0x7f474400c920 comp rx=0 tx=0).stop 2024-04-24T22:59:58.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.442+0000 7f475c170640 1 -- 172.21.15.32:0/2986125535 shutdown_connections 2024-04-24T22:59:58.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.442+0000 7f475c170640 1 --2- 172.21.15.32:0/2986125535 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f473806be30 0x7f473806e2f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:58.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.442+0000 7f475c170640 1 --2- 172.21.15.32:0/2986125535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47540af890 0x7f4754154ea0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:58.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.442+0000 7f475c170640 1 --2- 172.21.15.32:0/2986125535 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47540b44d0 0x7f4754143890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:58.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.442+0000 7f475c170640 1 --2- 172.21.15.32:0/2986125535 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f475406c5f0 0x7f4754159e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:58.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.442+0000 7f475c170640 1 -- 172.21.15.32:0/2986125535 >> 172.21.15.32:0/2986125535 conn(0x7f475406b110 msgr2=0x7f47540b4930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:58.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.442+0000 7f475c170640 1 -- 172.21.15.32:0/2986125535 shutdown_connections 2024-04-24T22:59:58.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.442+0000 7f475c170640 1 -- 172.21.15.32:0/2986125535 wait complete. 2024-04-24T22:59:58.454 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:59:58.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.553+0000 7fbe6b97e640 1 Processor -- start 2024-04-24T22:59:58.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.554+0000 7fbe6b97e640 1 -- start start 2024-04-24T22:59:58.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.554+0000 7fbe6b97e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbe6415e9b0 0x7fbe640d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:58.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.554+0000 7fbe6b97e640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fbe6406b820 0x7fbe640b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:58.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.554+0000 7fbe6b97e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe640d3170 con 0x7fbe640d4210 2024-04-24T22:59:58.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.554+0000 7fbe6b97e640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fbe640d3450 con 0x7fbe6406b820 2024-04-24T22:59:58.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.554+0000 7fbe6b97e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fbe640d3730 con 0x7fbe6415e9b0 2024-04-24T22:59:58.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.555+0000 7fbe68f19640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fbe640d4210 0x7fbe6406b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53964/0 (socket says 172.21.15.32:53964) 2024-04-24T22:59:58.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.555+0000 7fbe68f19640 1 -- 172.21.15.32:0/2462730494 learned_addr learned my addr 172.21.15.32:0/2462730494 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:58.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.555+0000 7fbe69f1b640 1 --2- 172.21.15.32:0/2462730494 >> v2:172.21.15.121:3300/0 conn(0x7fbe6406b820 0x7fbe640b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:58.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.555+0000 7fbe6971a640 1 --2- 172.21.15.32:0/2462730494 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbe6415e9b0 0x7fbe640d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:58.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.555+0000 7fbe63fff640 1 -- 172.21.15.32:0/2462730494 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3357278431 0 0) 0x7fbe640d3170 con 0x7fbe640d4210 2024-04-24T22:59:58.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.555+0000 7fbe63fff640 1 -- 172.21.15.32:0/2462730494 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe4c003520 con 0x7fbe640d4210 2024-04-24T22:59:58.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.555+0000 7fbe69f1b640 1 -- 172.21.15.32:0/2462730494 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbe6415e9b0 msgr2=0x7fbe640d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:58.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.555+0000 7fbe69f1b640 1 --2- 172.21.15.32:0/2462730494 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbe6415e9b0 0x7fbe640d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:58.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.555+0000 7fbe69f1b640 1 -- 172.21.15.32:0/2462730494 >> v1:172.21.15.32:6789/0 conn(0x7fbe640d4210 legacy=0x7fbe6406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:58.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.555+0000 7fbe69f1b640 1 -- 172.21.15.32:0/2462730494 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe640d3a10 con 0x7fbe6406b820 2024-04-24T22:59:58.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.556+0000 7fbe69f1b640 1 --2- 172.21.15.32:0/2462730494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe6406b820 0x7fbe640b78f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbe54009d90 tx=0x7fbe5402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8350b1ab40402eab server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:58.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.556+0000 7fbe63fff640 1 -- 172.21.15.32:0/2462730494 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe54007740 con 0x7fbe6406b820 2024-04-24T22:59:58.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.556+0000 7fbe63fff640 1 -- 172.21.15.32:0/2462730494 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe540078e0 con 0x7fbe6406b820 2024-04-24T22:59:58.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.556+0000 7fbe63fff640 1 -- 172.21.15.32:0/2462730494 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe54007cb0 con 0x7fbe6406b820 2024-04-24T22:59:58.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.556+0000 7fbe6b97e640 1 -- 172.21.15.32:0/2462730494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe6406b820 msgr2=0x7fbe640b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:58.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.556+0000 7fbe6b97e640 1 --2- 172.21.15.32:0/2462730494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe6406b820 0x7fbe640b78f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbe54009d90 tx=0x7fbe5402a660 comp rx=0 tx=0).stop 2024-04-24T22:59:58.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.556+0000 7fbe6b97e640 1 -- 172.21.15.32:0/2462730494 shutdown_connections 2024-04-24T22:59:58.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.556+0000 7fbe6b97e640 1 --2- 172.21.15.32:0/2462730494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe6406b820 0x7fbe640b78f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:58.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.556+0000 7fbe6b97e640 1 --2- 172.21.15.32:0/2462730494 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbe6415e9b0 0x7fbe640d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:58.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.556+0000 7fbe6b97e640 1 -- 172.21.15.32:0/2462730494 >> 172.21.15.32:0/2462730494 conn(0x7fbe640d11d0 msgr2=0x7fbe640d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:58.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.557+0000 7fbe6b97e640 1 -- 172.21.15.32:0/2462730494 shutdown_connections 2024-04-24T22:59:58.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.557+0000 7fbe6b97e640 1 -- 172.21.15.32:0/2462730494 wait complete. 2024-04-24T22:59:58.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.558+0000 7fbe6b97e640 1 Processor -- start 2024-04-24T22:59:58.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.558+0000 7fbe6b97e640 1 -- start start 2024-04-24T22:59:58.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.559+0000 7fbe6b97e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe6406b820 0x7fbe64152cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:58.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.559+0000 7fbe6b97e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe640d4210 0x7fbe6415fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:58.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.559+0000 7fbe6b97e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbe6415e9b0 0x7fbe640cc020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:58.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.559+0000 7fbe6b97e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbe640ba0e0 con 0x7fbe640d4210 2024-04-24T22:59:58.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.559+0000 7fbe6b97e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fbe640d2400 con 0x7fbe6406b820 2024-04-24T22:59:58.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.559+0000 7fbe6b97e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fbe640ba260 con 0x7fbe6415e9b0 2024-04-24T22:59:58.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.559+0000 7fbe68f19640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe640d4210 0x7fbe6415fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:58.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.559+0000 7fbe68f19640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe640d4210 0x7fbe6415fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51032/0 (socket says 172.21.15.32:51032) 2024-04-24T22:59:58.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.559+0000 7fbe68f19640 1 -- 172.21.15.32:0/186283049 learned_addr learned my addr 172.21.15.32:0/186283049 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:58.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.559+0000 7fbe6971a640 1 --2- 172.21.15.32:0/186283049 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe6406b820 0x7fbe64152cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:58.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.559+0000 7fbe69f1b640 1 --2- 172.21.15.32:0/186283049 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbe6415e9b0 0x7fbe640cc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:58.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.559+0000 7fbe68f19640 1 -- 172.21.15.32:0/186283049 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbe6415e9b0 msgr2=0x7fbe640cc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:58.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.559+0000 7fbe68f19640 1 --2- 172.21.15.32:0/186283049 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbe6415e9b0 0x7fbe640cc020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:58.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.559+0000 7fbe68f19640 1 -- 172.21.15.32:0/186283049 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe6406b820 msgr2=0x7fbe64152cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:58.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.559+0000 7fbe68f19640 1 --2- 172.21.15.32:0/186283049 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe6406b820 0x7fbe64152cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:58.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.559+0000 7fbe68f19640 1 -- 172.21.15.32:0/186283049 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe64160750 con 0x7fbe640d4210 2024-04-24T22:59:58.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.559+0000 7fbe69f1b640 1 --2- 172.21.15.32:0/186283049 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbe6415e9b0 0x7fbe640cc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:58.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.559+0000 7fbe68f19640 1 --2- 172.21.15.32:0/186283049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe640d4210 0x7fbe6415fde0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fbe5800c460 tx=0x7fbe5800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:58.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.559+0000 7fbe61ffb640 1 -- 172.21.15.32:0/186283049 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe58017070 con 0x7fbe640d4210 2024-04-24T22:59:58.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.559+0000 7fbe61ffb640 1 -- 172.21.15.32:0/186283049 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe58005ce0 con 0x7fbe640d4210 2024-04-24T22:59:58.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.559+0000 7fbe6b97e640 1 -- 172.21.15.32:0/186283049 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe64160a30 con 0x7fbe640d4210 2024-04-24T22:59:58.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.559+0000 7fbe61ffb640 1 -- 172.21.15.32:0/186283049 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe58007290 con 0x7fbe640d4210 2024-04-24T22:59:58.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.560+0000 7fbe6b97e640 1 -- 172.21.15.32:0/186283049 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe640cb820 con 0x7fbe640d4210 2024-04-24T22:59:58.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.561+0000 7fbe6b97e640 1 -- 172.21.15.32:0/186283049 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe64068430 con 0x7fbe640d4210 2024-04-24T22:59:58.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.561+0000 7fbe61ffb640 1 -- 172.21.15.32:0/186283049 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbe5801a070 con 0x7fbe640d4210 2024-04-24T22:59:58.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.561+0000 7fbe61ffb640 1 --2- 172.21.15.32:0/186283049 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fbe5006be30 0x7fbe5006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:58.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.562+0000 7fbe61ffb640 1 -- 172.21.15.32:0/186283049 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7fbe5808a130 con 0x7fbe640d4210 2024-04-24T22:59:58.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.562+0000 7fbe6971a640 1 --2- 172.21.15.32:0/186283049 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fbe5006be30 0x7fbe5006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:58.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.562+0000 7fbe6971a640 1 --2- 172.21.15.32:0/186283049 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fbe5006be30 0x7fbe5006e2f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c009aa0 tx=0x7fbe5c009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:58.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.566+0000 7fbe61ffb640 1 -- 172.21.15.32:0/186283049 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbe58058490 con 0x7fbe640d4210 2024-04-24T22:59:58.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.749+0000 7fbe6b97e640 1 -- 172.21.15.32:0/186283049 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbe640d2720 con 0x7fbe640d4210 2024-04-24T22:59:58.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.750+0000 7fbe61ffb640 1 -- 172.21.15.32:0/186283049 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) ==== 74+0+16634 (secure 0 0 0) 0x7fbe5805c790 con 0x7fbe640d4210 2024-04-24T22:59:58.752 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T22:59:58.752 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":31,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T22:59:57.468463+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:59:56.068812+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-24T22:59:56.447402+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:59:58.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.753+0000 7fbe6b97e640 1 -- 172.21.15.32:0/186283049 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fbe5006be30 msgr2=0x7fbe5006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:58.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.753+0000 7fbe6b97e640 1 --2- 172.21.15.32:0/186283049 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fbe5006be30 0x7fbe5006e2f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c009aa0 tx=0x7fbe5c009190 comp rx=0 tx=0).stop 2024-04-24T22:59:58.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.753+0000 7fbe6b97e640 1 -- 172.21.15.32:0/186283049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe640d4210 msgr2=0x7fbe6415fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:58.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.753+0000 7fbe6b97e640 1 --2- 172.21.15.32:0/186283049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe640d4210 0x7fbe6415fde0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fbe5800c460 tx=0x7fbe5800c920 comp rx=0 tx=0).stop 2024-04-24T22:59:58.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.753+0000 7fbe6b97e640 1 -- 172.21.15.32:0/186283049 shutdown_connections 2024-04-24T22:59:58.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.753+0000 7fbe6b97e640 1 --2- 172.21.15.32:0/186283049 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fbe5006be30 0x7fbe5006e2f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:58.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.753+0000 7fbe6b97e640 1 --2- 172.21.15.32:0/186283049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe640d4210 0x7fbe6415fde0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:58.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.753+0000 7fbe6b97e640 1 --2- 172.21.15.32:0/186283049 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbe6415e9b0 0x7fbe640cc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:58.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.753+0000 7fbe6b97e640 1 --2- 172.21.15.32:0/186283049 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe6406b820 0x7fbe64152cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:58.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.753+0000 7fbe6b97e640 1 -- 172.21.15.32:0/186283049 >> 172.21.15.32:0/186283049 conn(0x7fbe640d11d0 msgr2=0x7fbe640b7db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:58.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.753+0000 7fbe6b97e640 1 -- 172.21.15.32:0/186283049 shutdown_connections 2024-04-24T22:59:58.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.753+0000 7fbe6b97e640 1 -- 172.21.15.32:0/186283049 wait complete. 2024-04-24T22:59:58.765 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-24T22:59:58.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.867+0000 7fd74369a640 1 Processor -- start 2024-04-24T22:59:58.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.867+0000 7fd74369a640 1 -- start start 2024-04-24T22:59:58.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.868+0000 7fd74369a640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd73c15e9b0 0x7fd73c0d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:58.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.868+0000 7fd74369a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd73c06b820 0x7fd73c0b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:58.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.868+0000 7fd74369a640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd73c0d3170 con 0x7fd73c0d4210 2024-04-24T22:59:58.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.868+0000 7fd74369a640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fd73c0d3450 con 0x7fd73c06b820 2024-04-24T22:59:58.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.868+0000 7fd74369a640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fd73c0d3730 con 0x7fd73c15e9b0 2024-04-24T22:59:58.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.868+0000 7fd740c35640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd73c0d4210 0x7fd73c06b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53970/0 (socket says 172.21.15.32:53970) 2024-04-24T22:59:58.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.868+0000 7fd741c37640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd73c06b820 0x7fd73c0b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:58.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.868+0000 7fd740c35640 1 -- 172.21.15.32:0/333877773 learned_addr learned my addr 172.21.15.32:0/333877773 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T22:59:58.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.869+0000 7fd741436640 1 --2- 172.21.15.32:0/333877773 >> v2:172.21.15.121:3300/0 conn(0x7fd73c15e9b0 0x7fd73c0d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:58.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.869+0000 7fd73bfff640 1 -- 172.21.15.32:0/333877773 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 322571661 0 0) 0x7fd73c0d3170 con 0x7fd73c0d4210 2024-04-24T22:59:58.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.869+0000 7fd73bfff640 1 -- 172.21.15.32:0/333877773 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd724003520 con 0x7fd73c0d4210 2024-04-24T22:59:58.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.869+0000 7fd741c37640 1 -- 172.21.15.32:0/333877773 >> v2:172.21.15.121:3300/0 conn(0x7fd73c15e9b0 msgr2=0x7fd73c0d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:58.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.869+0000 7fd741c37640 1 --2- 172.21.15.32:0/333877773 >> v2:172.21.15.121:3300/0 conn(0x7fd73c15e9b0 0x7fd73c0d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:58.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.869+0000 7fd741c37640 1 -- 172.21.15.32:0/333877773 >> v1:172.21.15.32:6789/0 conn(0x7fd73c0d4210 legacy=0x7fd73c06b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:58.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.869+0000 7fd741c37640 1 -- 172.21.15.32:0/333877773 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd73c0d3a10 con 0x7fd73c06b820 2024-04-24T22:59:58.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.869+0000 7fd741c37640 1 --2- 172.21.15.32:0/333877773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd73c06b820 0x7fd73c0b78f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd73c06be20 tx=0x7fd72c02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28dd3115000fa2f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:58.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.870+0000 7fd73bfff640 1 -- 172.21.15.32:0/333877773 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72c007740 con 0x7fd73c06b820 2024-04-24T22:59:58.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.870+0000 7fd73bfff640 1 -- 172.21.15.32:0/333877773 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd72c0078e0 con 0x7fd73c06b820 2024-04-24T22:59:58.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.870+0000 7fd73bfff640 1 -- 172.21.15.32:0/333877773 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72c007cb0 con 0x7fd73c06b820 2024-04-24T22:59:58.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.870+0000 7fd74369a640 1 -- 172.21.15.32:0/333877773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd73c06b820 msgr2=0x7fd73c0b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:58.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.870+0000 7fd74369a640 1 --2- 172.21.15.32:0/333877773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd73c06b820 0x7fd73c0b78f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd73c06be20 tx=0x7fd72c02a5f0 comp rx=0 tx=0).stop 2024-04-24T22:59:58.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.870+0000 7fd74369a640 1 -- 172.21.15.32:0/333877773 shutdown_connections 2024-04-24T22:59:58.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.870+0000 7fd74369a640 1 --2- 172.21.15.32:0/333877773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd73c06b820 0x7fd73c0b78f0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:58.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.870+0000 7fd74369a640 1 --2- 172.21.15.32:0/333877773 >> v2:172.21.15.121:3300/0 conn(0x7fd73c15e9b0 0x7fd73c0d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:58.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.870+0000 7fd74369a640 1 -- 172.21.15.32:0/333877773 >> 172.21.15.32:0/333877773 conn(0x7fd73c0d11d0 msgr2=0x7fd73c0d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:58.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.870+0000 7fd74369a640 1 -- 172.21.15.32:0/333877773 shutdown_connections 2024-04-24T22:59:58.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.870+0000 7fd74369a640 1 -- 172.21.15.32:0/333877773 wait complete. 2024-04-24T22:59:58.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.871+0000 7fd74369a640 1 Processor -- start 2024-04-24T22:59:58.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.871+0000 7fd74369a640 1 -- start start 2024-04-24T22:59:58.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.872+0000 7fd74369a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd73c06b820 0x7fd73c155f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:58.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.872+0000 7fd74369a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd73c0d4210 0x7fd73c153ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:58.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.872+0000 7fd74369a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd73c15e9b0 0x7fd73c15cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:58.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.872+0000 7fd74369a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c0ba0e0 con 0x7fd73c0d4210 2024-04-24T22:59:58.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.872+0000 7fd74369a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c0d2400 con 0x7fd73c15e9b0 2024-04-24T22:59:58.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.872+0000 7fd74369a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd73c0ba260 con 0x7fd73c06b820 2024-04-24T22:59:58.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.872+0000 7fd740c35640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd73c0d4210 0x7fd73c153ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:58.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.872+0000 7fd740c35640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd73c0d4210 0x7fd73c153ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51056/0 (socket says 172.21.15.32:51056) 2024-04-24T22:59:58.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.872+0000 7fd740c35640 1 -- 172.21.15.32:0/3571159347 learned_addr learned my addr 172.21.15.32:0/3571159347 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:58.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.872+0000 7fd741c37640 1 --2- 172.21.15.32:0/3571159347 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd73c15e9b0 0x7fd73c15cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:58.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.872+0000 7fd741436640 1 --2- 172.21.15.32:0/3571159347 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd73c06b820 0x7fd73c155f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:58.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.873+0000 7fd740c35640 1 -- 172.21.15.32:0/3571159347 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd73c06b820 msgr2=0x7fd73c155f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:58.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.873+0000 7fd740c35640 1 --2- 172.21.15.32:0/3571159347 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd73c06b820 0x7fd73c155f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:58.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.873+0000 7fd740c35640 1 -- 172.21.15.32:0/3571159347 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd73c15e9b0 msgr2=0x7fd73c15cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:58.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.873+0000 7fd740c35640 1 --2- 172.21.15.32:0/3571159347 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd73c15e9b0 0x7fd73c15cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:58.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.873+0000 7fd740c35640 1 -- 172.21.15.32:0/3571159347 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd73c154810 con 0x7fd73c0d4210 2024-04-24T22:59:58.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.873+0000 7fd740c35640 1 --2- 172.21.15.32:0/3571159347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd73c0d4210 0x7fd73c153ce0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd73000c430 tx=0x7fd73000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:58.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.873+0000 7fd739ffb640 1 -- 172.21.15.32:0/3571159347 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd730017070 con 0x7fd73c0d4210 2024-04-24T22:59:58.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.873+0000 7fd739ffb640 1 -- 172.21.15.32:0/3571159347 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd730005ce0 con 0x7fd73c0d4210 2024-04-24T22:59:58.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.873+0000 7fd739ffb640 1 -- 172.21.15.32:0/3571159347 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7300072f0 con 0x7fd73c0d4210 2024-04-24T22:59:58.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.873+0000 7fd74369a640 1 -- 172.21.15.32:0/3571159347 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd73c154af0 con 0x7fd73c0d4210 2024-04-24T22:59:58.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.873+0000 7fd74369a640 1 -- 172.21.15.32:0/3571159347 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd73c1564d0 con 0x7fd73c0d4210 2024-04-24T22:59:58.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.874+0000 7fd74369a640 1 -- 172.21.15.32:0/3571159347 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd73c068430 con 0x7fd73c0d4210 2024-04-24T22:59:58.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.875+0000 7fd739ffb640 1 -- 172.21.15.32:0/3571159347 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd73001a070 con 0x7fd73c0d4210 2024-04-24T22:59:58.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.875+0000 7fd739ffb640 1 --2- 172.21.15.32:0/3571159347 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd72806be30 0x7fd72806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:58.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.875+0000 7fd739ffb640 1 -- 172.21.15.32:0/3571159347 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7fd73008a220 con 0x7fd73c0d4210 2024-04-24T22:59:58.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.875+0000 7fd741436640 1 --2- 172.21.15.32:0/3571159347 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd72806be30 0x7fd72806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:58.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.876+0000 7fd741436640 1 --2- 172.21.15.32:0/3571159347 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd72806be30 0x7fd72806e2f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd73c157be0 tx=0x7fd734009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:58.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:58.880+0000 7fd739ffb640 1 -- 172.21.15.32:0/3571159347 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd730058610 con 0x7fd73c0d4210 2024-04-24T22:59:59.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.105+0000 7fd74369a640 1 -- 172.21.15.32:0/3571159347 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fd73c1e9da0 con 0x7fd73c0d4210 2024-04-24T22:59:59.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.116+0000 7fd739ffb640 1 -- 172.21.15.32:0/3571159347 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v12) ==== 118+0+0 (secure 0 0 0) 0x7fd73005c910 con 0x7fd73c0d4210 2024-04-24T22:59:59.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.118+0000 7fd74369a640 1 -- 172.21.15.32:0/3571159347 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd72806be30 msgr2=0x7fd72806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:59.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.118+0000 7fd74369a640 1 --2- 172.21.15.32:0/3571159347 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd72806be30 0x7fd72806e2f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd73c157be0 tx=0x7fd734009190 comp rx=0 tx=0).stop 2024-04-24T22:59:59.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.118+0000 7fd74369a640 1 -- 172.21.15.32:0/3571159347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd73c0d4210 msgr2=0x7fd73c153ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:59.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.118+0000 7fd74369a640 1 --2- 172.21.15.32:0/3571159347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd73c0d4210 0x7fd73c153ce0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd73000c430 tx=0x7fd73000c8f0 comp rx=0 tx=0).stop 2024-04-24T22:59:59.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.118+0000 7fd74369a640 1 -- 172.21.15.32:0/3571159347 shutdown_connections 2024-04-24T22:59:59.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.118+0000 7fd74369a640 1 --2- 172.21.15.32:0/3571159347 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd72806be30 0x7fd72806e2f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:59.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.118+0000 7fd74369a640 1 --2- 172.21.15.32:0/3571159347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd73c0d4210 0x7fd73c153ce0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:59.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.118+0000 7fd74369a640 1 --2- 172.21.15.32:0/3571159347 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd73c15e9b0 0x7fd73c15cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:59.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.118+0000 7fd74369a640 1 --2- 172.21.15.32:0/3571159347 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd73c06b820 0x7fd73c155f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:59.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.118+0000 7fd74369a640 1 -- 172.21.15.32:0/3571159347 >> 172.21.15.32:0/3571159347 conn(0x7fd73c0d11d0 msgr2=0x7fd73c0b7b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:59.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.119+0000 7fd74369a640 1 -- 172.21.15.32:0/3571159347 shutdown_connections 2024-04-24T22:59:59.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.119+0000 7fd74369a640 1 -- 172.21.15.32:0/3571159347 wait complete. 2024-04-24T22:59:59.131 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-24T22:59:59.131 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-24T22:59:59.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.233+0000 7f20eeea5640 1 Processor -- start 2024-04-24T22:59:59.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.233+0000 7f20eeea5640 1 -- start start 2024-04-24T22:59:59.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.234+0000 7f20eeea5640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f20e815b380 0x7f20e806c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:59.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.234+0000 7f20eeea5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f20e815cb00 0x7f20e806cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:59.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.234+0000 7f20eeea5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20e80ce430 0x7f20e80ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:59.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.234+0000 7f20eeea5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f20e80b4520 con 0x7f20e815cb00 2024-04-24T22:59:59.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.234+0000 7f20eeea5640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f20e80b4800 con 0x7f20e815b380 2024-04-24T22:59:59.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.234+0000 7f20eeea5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f20e80b4ae0 con 0x7f20e80ce430 2024-04-24T22:59:59.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.234+0000 7f20dffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f20e815cb00 0x7f20e806cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:59.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.234+0000 7f20dffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f20e815cb00 0x7f20e806cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51066/0 (socket says 172.21.15.32:51066) 2024-04-24T22:59:59.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.234+0000 7f20dffff640 1 -- 172.21.15.32:0/421708838 learned_addr learned my addr 172.21.15.32:0/421708838 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:59.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.234+0000 7f20ed442640 1 --2- 172.21.15.32:0/421708838 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20e80ce430 0x7f20e80ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:59.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.234+0000 7f20ecc41640 1 --2- 172.21.15.32:0/421708838 >> v2:172.21.15.121:3300/0 conn(0x7f20e815b380 0x7f20e806c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:59.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.234+0000 7f20dffff640 1 -- 172.21.15.32:0/421708838 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20e80ce430 msgr2=0x7f20e80ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:59.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.234+0000 7f20dffff640 1 --2- 172.21.15.32:0/421708838 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20e80ce430 0x7f20e80ce860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:59.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.234+0000 7f20dffff640 1 -- 172.21.15.32:0/421708838 >> v2:172.21.15.121:3300/0 conn(0x7f20e815b380 msgr2=0x7f20e806c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:59:59.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.234+0000 7f20dffff640 1 --2- 172.21.15.32:0/421708838 >> v2:172.21.15.121:3300/0 conn(0x7f20e815b380 0x7f20e806c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:59.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.234+0000 7f20dffff640 1 -- 172.21.15.32:0/421708838 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20e80b4dc0 con 0x7f20e815cb00 2024-04-24T22:59:59.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.234+0000 7f20ecc41640 1 --2- 172.21.15.32:0/421708838 >> v2:172.21.15.121:3300/0 conn(0x7f20e815b380 0x7f20e806c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:59:59.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.235+0000 7f20dffff640 1 --2- 172.21.15.32:0/421708838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20e815cb00 0x7f20e806cb30 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f20e806eed0 tx=0x7f20d802a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d33a8306000d5b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:59.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.235+0000 7f20df7fe640 1 -- 172.21.15.32:0/421708838 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d8037070 con 0x7f20e815cb00 2024-04-24T22:59:59.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.235+0000 7f20df7fe640 1 -- 172.21.15.32:0/421708838 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20d802ae00 con 0x7f20e815cb00 2024-04-24T22:59:59.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.235+0000 7f20df7fe640 1 -- 172.21.15.32:0/421708838 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d8032660 con 0x7f20e815cb00 2024-04-24T22:59:59.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.236+0000 7f20eeea5640 1 -- 172.21.15.32:0/421708838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20e815cb00 msgr2=0x7f20e806cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:59.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.236+0000 7f20eeea5640 1 --2- 172.21.15.32:0/421708838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20e815cb00 0x7f20e806cb30 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f20e806eed0 tx=0x7f20d802a0b0 comp rx=0 tx=0).stop 2024-04-24T22:59:59.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.236+0000 7f20eeea5640 1 -- 172.21.15.32:0/421708838 shutdown_connections 2024-04-24T22:59:59.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.236+0000 7f20eeea5640 1 --2- 172.21.15.32:0/421708838 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20e80ce430 0x7f20e80ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:59.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.236+0000 7f20eeea5640 1 --2- 172.21.15.32:0/421708838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20e815cb00 0x7f20e806cb30 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:59.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.236+0000 7f20eeea5640 1 --2- 172.21.15.32:0/421708838 >> v2:172.21.15.121:3300/0 conn(0x7f20e815b380 0x7f20e806c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:59.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.236+0000 7f20eeea5640 1 -- 172.21.15.32:0/421708838 >> 172.21.15.32:0/421708838 conn(0x7f20e806b110 msgr2=0x7f20e806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:59:59.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.236+0000 7f20eeea5640 1 -- 172.21.15.32:0/421708838 shutdown_connections 2024-04-24T22:59:59.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.236+0000 7f20eeea5640 1 -- 172.21.15.32:0/421708838 wait complete. 2024-04-24T22:59:59.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.237+0000 7f20eeea5640 1 Processor -- start 2024-04-24T22:59:59.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.237+0000 7f20eeea5640 1 -- start start 2024-04-24T22:59:59.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.238+0000 7f20eeea5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20e80ce430 0x7f20e8145cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:59.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.238+0000 7f20eeea5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20e815b380 0x7f20e8156090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:59.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.238+0000 7f20eeea5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20e815cb00 0x7f20e81565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:59.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.238+0000 7f20eeea5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f20e80ceda0 con 0x7f20e80ce430 2024-04-24T22:59:59.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.238+0000 7f20eeea5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f20e815b760 con 0x7f20e815b380 2024-04-24T22:59:59.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.238+0000 7f20eeea5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f20e80cef20 con 0x7f20e815cb00 2024-04-24T22:59:59.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.238+0000 7f20ecc41640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20e80ce430 0x7f20e8145cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:59.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.238+0000 7f20ecc41640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20e80ce430 0x7f20e8145cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51070/0 (socket says 172.21.15.32:51070) 2024-04-24T22:59:59.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.238+0000 7f20ecc41640 1 -- 172.21.15.32:0/3503526072 learned_addr learned my addr 172.21.15.32:0/3503526072 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T22:59:59.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.238+0000 7f20ed442640 1 --2- 172.21.15.32:0/3503526072 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20e815cb00 0x7f20e81565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:59.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.238+0000 7f20dffff640 1 --2- 172.21.15.32:0/3503526072 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20e815b380 0x7f20e8156090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:59.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.238+0000 7f20ecc41640 1 -- 172.21.15.32:0/3503526072 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20e815cb00 msgr2=0x7f20e81565d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:59.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.238+0000 7f20ecc41640 1 --2- 172.21.15.32:0/3503526072 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20e815cb00 0x7f20e81565d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:59.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.238+0000 7f20ecc41640 1 -- 172.21.15.32:0/3503526072 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20e815b380 msgr2=0x7f20e8156090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:59:59.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.238+0000 7f20ecc41640 1 --2- 172.21.15.32:0/3503526072 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20e815b380 0x7f20e8156090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:59:59.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.238+0000 7f20ecc41640 1 -- 172.21.15.32:0/3503526072 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20e8158a10 con 0x7f20e80ce430 2024-04-24T22:59:59.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.238+0000 7f20ed442640 1 --2- 172.21.15.32:0/3503526072 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20e815cb00 0x7f20e81565d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:59:59.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.239+0000 7f20ecc41640 1 --2- 172.21.15.32:0/3503526072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20e80ce430 0x7f20e8145cc0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f20e400c7f0 tx=0x7f20e400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:59.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.239+0000 7f20dd7fa640 1 -- 172.21.15.32:0/3503526072 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e4015070 con 0x7f20e80ce430 2024-04-24T22:59:59.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.239+0000 7f20dd7fa640 1 -- 172.21.15.32:0/3503526072 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20e4007820 con 0x7f20e80ce430 2024-04-24T22:59:59.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.239+0000 7f20dd7fa640 1 -- 172.21.15.32:0/3503526072 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e4007bd0 con 0x7f20e80ce430 2024-04-24T22:59:59.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.239+0000 7f20eeea5640 1 -- 172.21.15.32:0/3503526072 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20e8153c50 con 0x7f20e80ce430 2024-04-24T22:59:59.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.239+0000 7f20eeea5640 1 -- 172.21.15.32:0/3503526072 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20e8158cb0 con 0x7f20e80ce430 2024-04-24T22:59:59.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.240+0000 7f20eeea5640 1 -- 172.21.15.32:0/3503526072 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20e8068430 con 0x7f20e80ce430 2024-04-24T22:59:59.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.241+0000 7f20dd7fa640 1 -- 172.21.15.32:0/3503526072 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f20e4018070 con 0x7f20e80ce430 2024-04-24T22:59:59.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.241+0000 7f20dd7fa640 1 --2- 172.21.15.32:0/3503526072 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f20d406be30 0x7f20d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:59:59.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.241+0000 7f20dd7fa640 1 -- 172.21.15.32:0/3503526072 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f20e408ae90 con 0x7f20e80ce430 2024-04-24T22:59:59.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.241+0000 7f20dffff640 1 --2- 172.21.15.32:0/3503526072 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f20d406be30 0x7f20d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:59:59.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.242+0000 7f20dffff640 1 --2- 172.21.15.32:0/3503526072 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f20d406be30 0x7f20d406e2f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f20d8034000 tx=0x7f20d802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:59:59.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.245+0000 7f20dd7fa640 1 -- 172.21.15.32:0/3503526072 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f20e4059840 con 0x7f20e80ce430 2024-04-24T22:59:59.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T22:59:59.470+0000 7f20eeea5640 1 -- 172.21.15.32:0/3503526072 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f20e80afd60 con 0x7f20e80ce430 2024-04-24T23:00:00.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.118+0000 7f20dd7fa640 1 -- 172.21.15.32:0/3503526072 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v13) ==== 122+0+0 (secure 0 0 0) 0x7f20e4059140 con 0x7f20e80ce430 2024-04-24T23:00:00.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.120+0000 7f20eeea5640 1 -- 172.21.15.32:0/3503526072 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f20d406be30 msgr2=0x7f20d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:00.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.120+0000 7f20eeea5640 1 --2- 172.21.15.32:0/3503526072 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f20d406be30 0x7f20d406e2f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f20d8034000 tx=0x7f20d802e000 comp rx=0 tx=0).stop 2024-04-24T23:00:00.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.120+0000 7f20eeea5640 1 -- 172.21.15.32:0/3503526072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20e80ce430 msgr2=0x7f20e8145cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:00.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.120+0000 7f20eeea5640 1 --2- 172.21.15.32:0/3503526072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20e80ce430 0x7f20e8145cc0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f20e400c7f0 tx=0x7f20e400ccb0 comp rx=0 tx=0).stop 2024-04-24T23:00:00.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.120+0000 7f20eeea5640 1 -- 172.21.15.32:0/3503526072 shutdown_connections 2024-04-24T23:00:00.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.120+0000 7f20eeea5640 1 --2- 172.21.15.32:0/3503526072 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f20d406be30 0x7f20d406e2f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:00.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.120+0000 7f20eeea5640 1 --2- 172.21.15.32:0/3503526072 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20e815cb00 0x7f20e81565d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:00.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.120+0000 7f20eeea5640 1 --2- 172.21.15.32:0/3503526072 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20e815b380 0x7f20e8156090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:00.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.120+0000 7f20eeea5640 1 --2- 172.21.15.32:0/3503526072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20e80ce430 0x7f20e8145cc0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:00.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.120+0000 7f20eeea5640 1 -- 172.21.15.32:0/3503526072 >> 172.21.15.32:0/3503526072 conn(0x7f20e806b110 msgr2=0x7f20e80b3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:00.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.120+0000 7f20eeea5640 1 -- 172.21.15.32:0/3503526072 shutdown_connections 2024-04-24T23:00:00.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.120+0000 7f20eeea5640 1 -- 172.21.15.32:0/3503526072 wait complete. 2024-04-24T23:00:00.135 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-24T23:00:00.135 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:00:00.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.242+0000 7f31f5d70640 1 Processor -- start 2024-04-24T23:00:00.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.242+0000 7f31f5d70640 1 -- start start 2024-04-24T23:00:00.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.243+0000 7f31f5d70640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f31f006c5f0 0x7f31f006c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:00.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.243+0000 7f31f5d70640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31f00b44d0 0x7f31f00b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:00.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.243+0000 7f31f5d70640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31f00b6430 con 0x7f31f00af890 2024-04-24T23:00:00.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.243+0000 7f31f5d70640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f31f0156f80 con 0x7f31f006c5f0 2024-04-24T23:00:00.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.243+0000 7f31f5d70640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f31f00b6150 con 0x7f31f00b44d0 2024-04-24T23:00:00.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.243+0000 7f31ef7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f31f006c5f0 0x7f31f006c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:00.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.243+0000 7f31ef7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f31f006c5f0 0x7f31f006c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59990/0 (socket says 172.21.15.32:59990) 2024-04-24T23:00:00.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.243+0000 7f31ef7fe640 1 -- 172.21.15.32:0/1966240906 learned_addr learned my addr 172.21.15.32:0/1966240906 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:00.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.243+0000 7f31ef7fe640 1 -- 172.21.15.32:0/1966240906 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31f00b44d0 msgr2=0x7f31f00b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:00.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.243+0000 7f31ef7fe640 1 --2- 172.21.15.32:0/1966240906 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31f00b44d0 0x7f31f00b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:00.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.243+0000 7f31ef7fe640 1 -- 172.21.15.32:0/1966240906 >> v1:172.21.15.121:6789/0 conn(0x7f31f00af890 legacy=0x7f31f00cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:00.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.243+0000 7f31ef7fe640 1 -- 172.21.15.32:0/1966240906 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31f00b39f0 con 0x7f31f006c5f0 2024-04-24T23:00:00.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.244+0000 7f31ef7fe640 1 --2- 172.21.15.32:0/1966240906 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31f006c5f0 0x7f31f006c9d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f31e0009870 tx=0x7f31e002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=421e358a9222a3f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:00.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.244+0000 7f31ee7fc640 1 -- 172.21.15.32:0/1966240906 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e0037070 con 0x7f31f006c5f0 2024-04-24T23:00:00.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.244+0000 7f31ee7fc640 1 -- 172.21.15.32:0/1966240906 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31e0032440 con 0x7f31f006c5f0 2024-04-24T23:00:00.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.244+0000 7f31ee7fc640 1 -- 172.21.15.32:0/1966240906 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e0032810 con 0x7f31f006c5f0 2024-04-24T23:00:00.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.245+0000 7f31f5d70640 1 -- 172.21.15.32:0/1966240906 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31f006c5f0 msgr2=0x7f31f006c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:00.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.245+0000 7f31f5d70640 1 --2- 172.21.15.32:0/1966240906 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31f006c5f0 0x7f31f006c9d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f31e0009870 tx=0x7f31e002a180 comp rx=0 tx=0).stop 2024-04-24T23:00:00.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.245+0000 7f31f5d70640 1 -- 172.21.15.32:0/1966240906 shutdown_connections 2024-04-24T23:00:00.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.245+0000 7f31f5d70640 1 --2- 172.21.15.32:0/1966240906 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31f00b44d0 0x7f31f00b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:00.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.245+0000 7f31f5d70640 1 --2- 172.21.15.32:0/1966240906 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31f006c5f0 0x7f31f006c9d0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:00.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.245+0000 7f31f5d70640 1 -- 172.21.15.32:0/1966240906 >> 172.21.15.32:0/1966240906 conn(0x7f31f006b110 msgr2=0x7f31f006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:00.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.245+0000 7f31f5d70640 1 -- 172.21.15.32:0/1966240906 shutdown_connections 2024-04-24T23:00:00.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.245+0000 7f31f5d70640 1 -- 172.21.15.32:0/1966240906 wait complete. 2024-04-24T23:00:00.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.246+0000 7f31f5d70640 1 Processor -- start 2024-04-24T23:00:00.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.246+0000 7f31f5d70640 1 -- start start 2024-04-24T23:00:00.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.247+0000 7f31f5d70640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31f006c5f0 0x7f31f0150750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:00.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.247+0000 7f31f5d70640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31f00af890 0x7f31f00ce760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:00.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.247+0000 7f31f5d70640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31f00b44d0 0x7f31f01f72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:00.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.247+0000 7f31f5d70640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f31f006cf10 con 0x7f31f00b44d0 2024-04-24T23:00:00.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.247+0000 7f31f5d70640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f31f00b3fb0 con 0x7f31f006c5f0 2024-04-24T23:00:00.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.247+0000 7f31f5d70640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f31f00b5dd0 con 0x7f31f00af890 2024-04-24T23:00:00.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.247+0000 7f31effff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31f00b44d0 0x7f31f01f72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:00.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.247+0000 7f31effff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31f00b44d0 0x7f31f01f72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59994/0 (socket says 172.21.15.32:59994) 2024-04-24T23:00:00.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.247+0000 7f31effff640 1 -- 172.21.15.32:0/1097119828 learned_addr learned my addr 172.21.15.32:0/1097119828 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:00.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.247+0000 7f31ef7fe640 1 --2- 172.21.15.32:0/1097119828 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31f006c5f0 0x7f31f0150750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:00.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.247+0000 7f31eeffd640 1 --2- 172.21.15.32:0/1097119828 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31f00af890 0x7f31f00ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:00.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.248+0000 7f31effff640 1 -- 172.21.15.32:0/1097119828 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31f00af890 msgr2=0x7f31f00ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:00.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.248+0000 7f31effff640 1 --2- 172.21.15.32:0/1097119828 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31f00af890 0x7f31f00ce760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:00.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.248+0000 7f31effff640 1 -- 172.21.15.32:0/1097119828 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31f006c5f0 msgr2=0x7f31f0150750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:00.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.248+0000 7f31effff640 1 --2- 172.21.15.32:0/1097119828 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31f006c5f0 0x7f31f0150750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:00.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.248+0000 7f31effff640 1 -- 172.21.15.32:0/1097119828 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31f0150d30 con 0x7f31f00b44d0 2024-04-24T23:00:00.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.248+0000 7f31effff640 1 --2- 172.21.15.32:0/1097119828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31f00b44d0 0x7f31f01f72f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f31e800c780 tx=0x7f31e800cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:00.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.248+0000 7f31d7fff640 1 -- 172.21.15.32:0/1097119828 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e8004b60 con 0x7f31f00b44d0 2024-04-24T23:00:00.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.248+0000 7f31d7fff640 1 -- 172.21.15.32:0/1097119828 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31e8004d20 con 0x7f31f00b44d0 2024-04-24T23:00:00.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.248+0000 7f31d7fff640 1 -- 172.21.15.32:0/1097119828 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e8005270 con 0x7f31f00b44d0 2024-04-24T23:00:00.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.248+0000 7f31f5d70640 1 -- 172.21.15.32:0/1097119828 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31f0151010 con 0x7f31f00b44d0 2024-04-24T23:00:00.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.248+0000 7f31f5d70640 1 -- 172.21.15.32:0/1097119828 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31f01518b0 con 0x7f31f00b44d0 2024-04-24T23:00:00.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.249+0000 7f31f5d70640 1 -- 172.21.15.32:0/1097119828 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31f0151ab0 con 0x7f31f00b44d0 2024-04-24T23:00:00.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.249+0000 7f31d7fff640 1 -- 172.21.15.32:0/1097119828 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f31e8005410 con 0x7f31f00b44d0 2024-04-24T23:00:00.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.250+0000 7f31d7fff640 1 --2- 172.21.15.32:0/1097119828 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f31dc06be30 0x7f31dc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:00.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.250+0000 7f31d7fff640 1 -- 172.21.15.32:0/1097119828 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f31e808b070 con 0x7f31f00b44d0 2024-04-24T23:00:00.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.250+0000 7f31ef7fe640 1 --2- 172.21.15.32:0/1097119828 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f31dc06be30 0x7f31dc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:00.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.251+0000 7f31ef7fe640 1 --2- 172.21.15.32:0/1097119828 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f31dc06be30 0x7f31dc06e2f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f31e0034000 tx=0x7f31e002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:00.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.254+0000 7f31d7fff640 1 -- 172.21.15.32:0/1097119828 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f31f0151ab0 con 0x7f31f00b44d0 2024-04-24T23:00:00.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.473+0000 7f31f5d70640 1 -- 172.21.15.32:0/1097119828 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f31f00b5670 con 0x7f31f00b44d0 2024-04-24T23:00:00.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.473+0000 7f31d7fff640 1 -- 172.21.15.32:0/1097119828 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+5181 (secure 0 0 0) 0x7f31e8020070 con 0x7f31f00b44d0 2024-04-24T23:00:00.475 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:00:00.475 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1796819396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":1796819396},{"type":"v1","addr":"172.21.15.32:6835","nonce":1796819396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/298202585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":298202585},{"type":"v1","addr":"172.21.15.121:6836","nonce":298202585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4939,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/4056636026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":4056636026},{"type":"v1","addr":"172.21.15.32:6834","nonce":4056636026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T22:59:57.468765+0000","modified":"2024-04-24T22:59:59.471269+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5112},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5112":{"gid":5112,"name":"b","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6837/515107694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":515107694},{"type":"v1","addr":"172.21.15.121:6837","nonce":515107694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5112,"qdb_cluster":[5112]},"id":2}]} 2024-04-24T23:00:00.475 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 13 2024-04-24T23:00:00.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.475+0000 7f31f5d70640 1 -- 172.21.15.32:0/1097119828 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f31dc06be30 msgr2=0x7f31dc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:00.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.475+0000 7f31f5d70640 1 --2- 172.21.15.32:0/1097119828 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f31dc06be30 0x7f31dc06e2f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f31e0034000 tx=0x7f31e002e000 comp rx=0 tx=0).stop 2024-04-24T23:00:00.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.475+0000 7f31f5d70640 1 -- 172.21.15.32:0/1097119828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31f00b44d0 msgr2=0x7f31f01f72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:00.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.475+0000 7f31f5d70640 1 --2- 172.21.15.32:0/1097119828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31f00b44d0 0x7f31f01f72f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f31e800c780 tx=0x7f31e800cb80 comp rx=0 tx=0).stop 2024-04-24T23:00:00.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.475+0000 7f31f5d70640 1 -- 172.21.15.32:0/1097119828 shutdown_connections 2024-04-24T23:00:00.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.475+0000 7f31f5d70640 1 --2- 172.21.15.32:0/1097119828 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f31dc06be30 0x7f31dc06e2f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:00.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.475+0000 7f31f5d70640 1 --2- 172.21.15.32:0/1097119828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31f00b44d0 0x7f31f01f72f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:00.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.475+0000 7f31f5d70640 1 --2- 172.21.15.32:0/1097119828 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31f00af890 0x7f31f00ce760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:00.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.475+0000 7f31f5d70640 1 --2- 172.21.15.32:0/1097119828 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31f006c5f0 0x7f31f0150750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:00.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.475+0000 7f31f5d70640 1 -- 172.21.15.32:0/1097119828 >> 172.21.15.32:0/1097119828 conn(0x7f31f006b110 msgr2=0x7f31f0072870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:00.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.475+0000 7f31f5d70640 1 -- 172.21.15.32:0/1097119828 shutdown_connections 2024-04-24T23:00:00.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.475+0000 7f31f5d70640 1 -- 172.21.15.32:0/1097119828 wait complete. 2024-04-24T23:00:00.488 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:00:00.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.588+0000 7ff361676640 1 Processor -- start 2024-04-24T23:00:00.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.588+0000 7ff361676640 1 -- start start 2024-04-24T23:00:00.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.589+0000 7ff361676640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff35c15b480 0x7ff35c0b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:00.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.589+0000 7ff361676640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff35c06cd00 0x7ff35c0cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:00.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.589+0000 7ff361676640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff35c0b46c0 con 0x7ff35c15fd60 2024-04-24T23:00:00.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.589+0000 7ff361676640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7ff35c0b4ff0 con 0x7ff35c15b480 2024-04-24T23:00:00.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.589+0000 7ff361676640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff35c0b49a0 con 0x7ff35c06cd00 2024-04-24T23:00:00.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.589+0000 7ff35b7fe640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff35c06cd00 0x7ff35c0cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:00.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.589+0000 7ff35affd640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff35c15b480 0x7ff35c0b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:00.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.589+0000 7ff353fff640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7ff35c15fd60 0x7ff35c06c5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:50370/0 (socket says 172.21.15.32:50370) 2024-04-24T23:00:00.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.589+0000 7ff353fff640 1 -- 172.21.15.32:0/3437644280 learned_addr learned my addr 172.21.15.32:0/3437644280 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:00:00.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.589+0000 7ff35b7fe640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff35c06cd00 0x7ff35c0cdf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.32:45006/0 (socket says 172.21.15.32:45006) 2024-04-24T23:00:00.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.589+0000 7ff35b7fe640 1 -- 172.21.15.32:0/3437644280 >> v2:172.21.15.121:3300/0 conn(0x7ff35c15b480 msgr2=0x7ff35c0b5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:00.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.589+0000 7ff35b7fe640 1 --2- 172.21.15.32:0/3437644280 >> v2:172.21.15.121:3300/0 conn(0x7ff35c15b480 0x7ff35c0b5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:00.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.589+0000 7ff35b7fe640 1 -- 172.21.15.32:0/3437644280 >> v1:172.21.15.121:6789/0 conn(0x7ff35c15fd60 legacy=0x7ff35c06c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:00.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.589+0000 7ff35b7fe640 1 -- 172.21.15.32:0/3437644280 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff35c0b4c80 con 0x7ff35c06cd00 2024-04-24T23:00:00.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.589+0000 7ff35affd640 1 --2- 172.21.15.32:0/3437644280 >> v2:172.21.15.121:3300/0 conn(0x7ff35c15b480 0x7ff35c0b5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:00:00.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.590+0000 7ff35b7fe640 1 --2- 172.21.15.32:0/3437644280 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff35c06cd00 0x7ff35c0cdf80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff354009d90 tx=0x7ff35402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8249eb56fccddd56 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:00.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.590+0000 7ff35a7fc640 1 -- 172.21.15.32:0/3437644280 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff354007740 con 0x7ff35c06cd00 2024-04-24T23:00:00.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.590+0000 7ff35a7fc640 1 -- 172.21.15.32:0/3437644280 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3540078e0 con 0x7ff35c06cd00 2024-04-24T23:00:00.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.590+0000 7ff35a7fc640 1 -- 172.21.15.32:0/3437644280 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff354007cb0 con 0x7ff35c06cd00 2024-04-24T23:00:00.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.591+0000 7ff361676640 1 -- 172.21.15.32:0/3437644280 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff35c06cd00 msgr2=0x7ff35c0cdf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:00.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.591+0000 7ff361676640 1 --2- 172.21.15.32:0/3437644280 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff35c06cd00 0x7ff35c0cdf80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff354009d90 tx=0x7ff35402a660 comp rx=0 tx=0).stop 2024-04-24T23:00:00.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.591+0000 7ff361676640 1 -- 172.21.15.32:0/3437644280 shutdown_connections 2024-04-24T23:00:00.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.591+0000 7ff361676640 1 --2- 172.21.15.32:0/3437644280 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff35c06cd00 0x7ff35c0cdf80 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:00.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.591+0000 7ff361676640 1 --2- 172.21.15.32:0/3437644280 >> v2:172.21.15.121:3300/0 conn(0x7ff35c15b480 0x7ff35c0b5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:00.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.591+0000 7ff361676640 1 -- 172.21.15.32:0/3437644280 >> 172.21.15.32:0/3437644280 conn(0x7ff35c06b110 msgr2=0x7ff35c06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:00.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.591+0000 7ff361676640 1 -- 172.21.15.32:0/3437644280 shutdown_connections 2024-04-24T23:00:00.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.591+0000 7ff361676640 1 -- 172.21.15.32:0/3437644280 wait complete. 2024-04-24T23:00:00.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.592+0000 7ff361676640 1 Processor -- start 2024-04-24T23:00:00.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.592+0000 7ff361676640 1 -- start start 2024-04-24T23:00:00.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.593+0000 7ff361676640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff35c06cd00 0x7ff35c146c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:00.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.593+0000 7ff361676640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff35c15b480 0x7ff35c159b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:00.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.593+0000 7ff361676640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff35c15fd60 0x7ff35c15a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:00.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.593+0000 7ff361676640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff35c15b860 con 0x7ff35c15fd60 2024-04-24T23:00:00.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.593+0000 7ff361676640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff35c0b5480 con 0x7ff35c06cd00 2024-04-24T23:00:00.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.593+0000 7ff361676640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff35c1e5dc0 con 0x7ff35c15b480 2024-04-24T23:00:00.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.593+0000 7ff35b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff35c15fd60 0x7ff35c15a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:00.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.593+0000 7ff35b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff35c15fd60 0x7ff35c15a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60010/0 (socket says 172.21.15.32:60010) 2024-04-24T23:00:00.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.593+0000 7ff35b7fe640 1 -- 172.21.15.32:0/579470474 learned_addr learned my addr 172.21.15.32:0/579470474 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:00.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.593+0000 7ff35affd640 1 --2- 172.21.15.32:0/579470474 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff35c06cd00 0x7ff35c146c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:00.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.593+0000 7ff353fff640 1 --2- 172.21.15.32:0/579470474 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff35c15b480 0x7ff35c159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:00.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.593+0000 7ff35b7fe640 1 -- 172.21.15.32:0/579470474 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff35c15b480 msgr2=0x7ff35c159b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:00.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.593+0000 7ff35b7fe640 1 --2- 172.21.15.32:0/579470474 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff35c15b480 0x7ff35c159b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:00.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.593+0000 7ff35b7fe640 1 -- 172.21.15.32:0/579470474 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff35c06cd00 msgr2=0x7ff35c146c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:00.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.593+0000 7ff35b7fe640 1 --2- 172.21.15.32:0/579470474 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff35c06cd00 0x7ff35c146c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:00.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.593+0000 7ff35b7fe640 1 -- 172.21.15.32:0/579470474 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff35c1e60a0 con 0x7ff35c15fd60 2024-04-24T23:00:00.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.594+0000 7ff35b7fe640 1 --2- 172.21.15.32:0/579470474 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff35c15fd60 0x7ff35c15a0b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff354009eb0 tx=0x7ff354004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:00.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.594+0000 7ff3537fe640 1 -- 172.21.15.32:0/579470474 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff354040070 con 0x7ff35c15fd60 2024-04-24T23:00:00.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.594+0000 7ff3537fe640 1 -- 172.21.15.32:0/579470474 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff354031bc0 con 0x7ff35c15fd60 2024-04-24T23:00:00.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.594+0000 7ff361676640 1 -- 172.21.15.32:0/579470474 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff35c1e6380 con 0x7ff35c15fd60 2024-04-24T23:00:00.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.594+0000 7ff3537fe640 1 -- 172.21.15.32:0/579470474 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff354009450 con 0x7ff35c15fd60 2024-04-24T23:00:00.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.594+0000 7ff361676640 1 -- 172.21.15.32:0/579470474 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff35c147180 con 0x7ff35c15fd60 2024-04-24T23:00:00.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.595+0000 7ff361676640 1 -- 172.21.15.32:0/579470474 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff35c068430 con 0x7ff35c15fd60 2024-04-24T23:00:00.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.595+0000 7ff3537fe640 1 -- 172.21.15.32:0/579470474 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff354039040 con 0x7ff35c15fd60 2024-04-24T23:00:00.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.596+0000 7ff3537fe640 1 --2- 172.21.15.32:0/579470474 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff33c06be30 0x7ff33c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:00.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.596+0000 7ff3537fe640 1 -- 172.21.15.32:0/579470474 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7ff3540ac620 con 0x7ff35c15fd60 2024-04-24T23:00:00.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.596+0000 7ff35affd640 1 --2- 172.21.15.32:0/579470474 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff33c06be30 0x7ff33c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:00.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.596+0000 7ff35affd640 1 --2- 172.21.15.32:0/579470474 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff33c06be30 0x7ff33c06e2f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff34c009560 tx=0x7ff34c009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:00.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.600+0000 7ff3537fe640 1 -- 172.21.15.32:0/579470474 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff35407a900 con 0x7ff35c15fd60 2024-04-24T23:00:00.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.784+0000 7ff361676640 1 -- 172.21.15.32:0/579470474 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff35c0afae0 con 0x7ff35c15fd60 2024-04-24T23:00:00.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.785+0000 7ff3537fe640 1 -- 172.21.15.32:0/579470474 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16621 (secure 0 0 0) 0x7ff35407ec00 con 0x7ff35c15fd60 2024-04-24T23:00:00.787 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:00:00.787 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":32,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T22:59:59.112998+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:59:56.068812+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-24T22:59:56.447402+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:00:00.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.787+0000 7ff361676640 1 -- 172.21.15.32:0/579470474 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff33c06be30 msgr2=0x7ff33c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:00.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.787+0000 7ff361676640 1 --2- 172.21.15.32:0/579470474 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff33c06be30 0x7ff33c06e2f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff34c009560 tx=0x7ff34c009190 comp rx=0 tx=0).stop 2024-04-24T23:00:00.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.787+0000 7ff361676640 1 -- 172.21.15.32:0/579470474 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff35c15fd60 msgr2=0x7ff35c15a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:00.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.787+0000 7ff361676640 1 --2- 172.21.15.32:0/579470474 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff35c15fd60 0x7ff35c15a0b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff354009eb0 tx=0x7ff354004a40 comp rx=0 tx=0).stop 2024-04-24T23:00:00.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.787+0000 7ff361676640 1 -- 172.21.15.32:0/579470474 shutdown_connections 2024-04-24T23:00:00.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.787+0000 7ff361676640 1 --2- 172.21.15.32:0/579470474 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff33c06be30 0x7ff33c06e2f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:00.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.787+0000 7ff361676640 1 --2- 172.21.15.32:0/579470474 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff35c15fd60 0x7ff35c15a0b0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:00.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.787+0000 7ff361676640 1 --2- 172.21.15.32:0/579470474 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff35c15b480 0x7ff35c159b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:00.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.787+0000 7ff361676640 1 --2- 172.21.15.32:0/579470474 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff35c06cd00 0x7ff35c146c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:00.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.787+0000 7ff361676640 1 -- 172.21.15.32:0/579470474 >> 172.21.15.32:0/579470474 conn(0x7ff35c06b110 msgr2=0x7ff35c06c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:00.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.787+0000 7ff361676640 1 -- 172.21.15.32:0/579470474 shutdown_connections 2024-04-24T23:00:00.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.787+0000 7ff361676640 1 -- 172.21.15.32:0/579470474 wait complete. 2024-04-24T23:00:00.800 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-24T23:00:00.800 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:00:00.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.899+0000 7fe428412640 1 Processor -- start 2024-04-24T23:00:00.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.899+0000 7fe428412640 1 -- start start 2024-04-24T23:00:00.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.900+0000 7fe428412640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe42015e9b0 0x7fe4200d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:00.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.900+0000 7fe428412640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe42006b820 0x7fe4200b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:00.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.900+0000 7fe428412640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4200d3170 con 0x7fe4200d4210 2024-04-24T23:00:00.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.900+0000 7fe428412640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fe4200d3450 con 0x7fe42006b820 2024-04-24T23:00:00.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.900+0000 7fe428412640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fe4200d3730 con 0x7fe42015e9b0 2024-04-24T23:00:00.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.900+0000 7fe4269af640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe42006b820 0x7fe4200b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:00.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.900+0000 7fe4269af640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe42006b820 0x7fe4200b78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60022/0 (socket says 172.21.15.32:60022) 2024-04-24T23:00:00.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.900+0000 7fe4269af640 1 -- 172.21.15.32:0/3850832409 learned_addr learned my addr 172.21.15.32:0/3850832409 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:00.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.901+0000 7fe4261ae640 1 --2- 172.21.15.32:0/3850832409 >> v2:172.21.15.121:3300/0 conn(0x7fe42015e9b0 0x7fe4200d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:00.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.901+0000 7fe4269af640 1 -- 172.21.15.32:0/3850832409 >> v2:172.21.15.121:3300/0 conn(0x7fe42015e9b0 msgr2=0x7fe4200d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:00.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.901+0000 7fe4269af640 1 --2- 172.21.15.32:0/3850832409 >> v2:172.21.15.121:3300/0 conn(0x7fe42015e9b0 0x7fe4200d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:00.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.901+0000 7fe4269af640 1 -- 172.21.15.32:0/3850832409 >> v1:172.21.15.121:6789/0 conn(0x7fe4200d4210 legacy=0x7fe42006b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:00.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.901+0000 7fe4269af640 1 -- 172.21.15.32:0/3850832409 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4200d3a10 con 0x7fe42006b820 2024-04-24T23:00:00.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.901+0000 7fe4261ae640 1 --2- 172.21.15.32:0/3850832409 >> v2:172.21.15.121:3300/0 conn(0x7fe42015e9b0 0x7fe4200d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:00:00.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.901+0000 7fe4269af640 1 --2- 172.21.15.32:0/3850832409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe42006b820 0x7fe4200b78f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe42006be20 tx=0x7fe41c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78aa3e487f16de79 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:00.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.902+0000 7fe4251ac640 1 -- 172.21.15.32:0/3850832409 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41c037070 con 0x7fe42006b820 2024-04-24T23:00:00.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.902+0000 7fe4251ac640 1 -- 172.21.15.32:0/3850832409 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe41c032440 con 0x7fe42006b820 2024-04-24T23:00:00.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.902+0000 7fe4251ac640 1 -- 172.21.15.32:0/3850832409 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41c032810 con 0x7fe42006b820 2024-04-24T23:00:00.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.902+0000 7fe428412640 1 -- 172.21.15.32:0/3850832409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe42006b820 msgr2=0x7fe4200b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:00.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.902+0000 7fe428412640 1 --2- 172.21.15.32:0/3850832409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe42006b820 0x7fe4200b78f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe42006be20 tx=0x7fe41c02a1e0 comp rx=0 tx=0).stop 2024-04-24T23:00:00.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.902+0000 7fe428412640 1 -- 172.21.15.32:0/3850832409 shutdown_connections 2024-04-24T23:00:00.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.902+0000 7fe428412640 1 --2- 172.21.15.32:0/3850832409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe42006b820 0x7fe4200b78f0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:00.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.902+0000 7fe428412640 1 --2- 172.21.15.32:0/3850832409 >> v2:172.21.15.121:3300/0 conn(0x7fe42015e9b0 0x7fe4200d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:00.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.902+0000 7fe428412640 1 -- 172.21.15.32:0/3850832409 >> 172.21.15.32:0/3850832409 conn(0x7fe4200d11d0 msgr2=0x7fe4200d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:00.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.902+0000 7fe428412640 1 -- 172.21.15.32:0/3850832409 shutdown_connections 2024-04-24T23:00:00.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.902+0000 7fe428412640 1 -- 172.21.15.32:0/3850832409 wait complete. 2024-04-24T23:00:00.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.903+0000 7fe428412640 1 Processor -- start 2024-04-24T23:00:00.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.903+0000 7fe428412640 1 -- start start 2024-04-24T23:00:00.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.904+0000 7fe428412640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe42006b820 0x7fe420156120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:00.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.904+0000 7fe428412640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4200d4210 0x7fe4200cd460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:00.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.904+0000 7fe428412640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe42015e9b0 0x7fe4200cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:00.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.904+0000 7fe428412640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe4200ba0e0 con 0x7fe4200d4210 2024-04-24T23:00:00.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.904+0000 7fe428412640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe4200d2400 con 0x7fe42006b820 2024-04-24T23:00:00.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.904+0000 7fe428412640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe4200ba260 con 0x7fe42015e9b0 2024-04-24T23:00:00.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.904+0000 7fe4259ad640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4200d4210 0x7fe4200cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:00.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.904+0000 7fe4259ad640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4200d4210 0x7fe4200cd460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60024/0 (socket says 172.21.15.32:60024) 2024-04-24T23:00:00.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.904+0000 7fe4259ad640 1 -- 172.21.15.32:0/1835465028 learned_addr learned my addr 172.21.15.32:0/1835465028 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:00.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.904+0000 7fe4261ae640 1 --2- 172.21.15.32:0/1835465028 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe42006b820 0x7fe420156120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:00.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.904+0000 7fe4269af640 1 --2- 172.21.15.32:0/1835465028 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe42015e9b0 0x7fe4200cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:00.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.905+0000 7fe4259ad640 1 -- 172.21.15.32:0/1835465028 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe42015e9b0 msgr2=0x7fe4200cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:00.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.905+0000 7fe4259ad640 1 --2- 172.21.15.32:0/1835465028 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe42015e9b0 0x7fe4200cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:00.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.905+0000 7fe4259ad640 1 -- 172.21.15.32:0/1835465028 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe42006b820 msgr2=0x7fe420156120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:00.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.905+0000 7fe4259ad640 1 --2- 172.21.15.32:0/1835465028 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe42006b820 0x7fe420156120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:00.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.905+0000 7fe4259ad640 1 -- 172.21.15.32:0/1835465028 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe420156660 con 0x7fe4200d4210 2024-04-24T23:00:00.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.905+0000 7fe4259ad640 1 --2- 172.21.15.32:0/1835465028 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4200d4210 0x7fe4200cd460 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fe41000c400 tx=0x7fe41000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:00.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.905+0000 7fe416ffd640 1 -- 172.21.15.32:0/1835465028 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4100052d0 con 0x7fe4200d4210 2024-04-24T23:00:00.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.905+0000 7fe416ffd640 1 -- 172.21.15.32:0/1835465028 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe410005490 con 0x7fe4200d4210 2024-04-24T23:00:00.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.905+0000 7fe416ffd640 1 -- 172.21.15.32:0/1835465028 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe410007260 con 0x7fe4200d4210 2024-04-24T23:00:00.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.905+0000 7fe428412640 1 -- 172.21.15.32:0/1835465028 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe420154610 con 0x7fe4200d4210 2024-04-24T23:00:00.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.905+0000 7fe428412640 1 -- 172.21.15.32:0/1835465028 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe420154bd0 con 0x7fe4200d4210 2024-04-24T23:00:00.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.906+0000 7fe428412640 1 -- 172.21.15.32:0/1835465028 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe420068430 con 0x7fe4200d4210 2024-04-24T23:00:00.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.906+0000 7fe416ffd640 1 -- 172.21.15.32:0/1835465028 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe410007400 con 0x7fe4200d4210 2024-04-24T23:00:00.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.907+0000 7fe416ffd640 1 --2- 172.21.15.32:0/1835465028 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe40406be30 0x7fe40406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:00.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.907+0000 7fe416ffd640 1 -- 172.21.15.32:0/1835465028 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fe41008a2f0 con 0x7fe4200d4210 2024-04-24T23:00:00.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.907+0000 7fe4261ae640 1 --2- 172.21.15.32:0/1835465028 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe40406be30 0x7fe40406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:00.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.908+0000 7fe4261ae640 1 --2- 172.21.15.32:0/1835465028 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe40406be30 0x7fe40406e2f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe418009fd0 tx=0x7fe4180097f0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:00.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:00.911+0000 7fe416ffd640 1 -- 172.21.15.32:0/1835465028 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe410058650 con 0x7fe4200d4210 2024-04-24T23:00:01.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.089+0000 7fe428412640 1 -- 172.21.15.32:0/1835465028 --> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe420156890 con 0x7fe40406be30 2024-04-24T23:00:01.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.096+0000 7fe416ffd640 1 -- 172.21.15.32:0/1835465028 <== mgr.4108 v2:172.21.15.121:6832/1848157943 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+242215 (secure 0 0 0) 0x7fe420156890 con 0x7fe40406be30 2024-04-24T23:00:01.100 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:00:01.101 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-24T23:00:01.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.098+0000 7fe428412640 1 -- 172.21.15.32:0/1835465028 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe40406be30 msgr2=0x7fe40406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:01.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.098+0000 7fe428412640 1 --2- 172.21.15.32:0/1835465028 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe40406be30 0x7fe40406e2f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe418009fd0 tx=0x7fe4180097f0 comp rx=0 tx=0).stop 2024-04-24T23:00:01.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.098+0000 7fe428412640 1 -- 172.21.15.32:0/1835465028 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4200d4210 msgr2=0x7fe4200cd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:01.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.098+0000 7fe428412640 1 --2- 172.21.15.32:0/1835465028 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4200d4210 0x7fe4200cd460 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fe41000c400 tx=0x7fe41000c8c0 comp rx=0 tx=0).stop 2024-04-24T23:00:01.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.099+0000 7fe428412640 1 -- 172.21.15.32:0/1835465028 shutdown_connections 2024-04-24T23:00:01.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.099+0000 7fe428412640 1 --2- 172.21.15.32:0/1835465028 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe40406be30 0x7fe40406e2f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:01.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.099+0000 7fe428412640 1 --2- 172.21.15.32:0/1835465028 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4200d4210 0x7fe4200cd460 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:01.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.099+0000 7fe428412640 1 --2- 172.21.15.32:0/1835465028 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe42015e9b0 0x7fe4200cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:01.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.099+0000 7fe428412640 1 --2- 172.21.15.32:0/1835465028 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe42006b820 0x7fe420156120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:01.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.099+0000 7fe428412640 1 -- 172.21.15.32:0/1835465028 >> 172.21.15.32:0/1835465028 conn(0x7fe4200d11d0 msgr2=0x7fe420071730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:01.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.099+0000 7fe428412640 1 -- 172.21.15.32:0/1835465028 shutdown_connections 2024-04-24T23:00:01.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.099+0000 7fe428412640 1 -- 172.21.15.32:0/1835465028 wait complete. 2024-04-24T23:00:01.111 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":57,"stamp":"2024-04-24T22:59:59.140237+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2079,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560420,"statfs":{"total":1600000000,"available":1597870721,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000742"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.108588+0000","last_change":"2024-04-24T22:59:57.108588+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.108588+0000","last_undegraded":"2024-04-24T22:59:57.108588+0000","last_fullsized":"2024-04-24T22:59:57.108588+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.108603+0000","last_change":"2024-04-24T22:59:57.108603+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.108603+0000","last_undegraded":"2024-04-24T22:59:57.108603+0000","last_fullsized":"2024-04-24T22:59:57.108603+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107901+0000","last_change":"2024-04-24T22:59:57.107901+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107901+0000","last_undegraded":"2024-04-24T22:59:57.107901+0000","last_fullsized":"2024-04-24T22:59:57.107901+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.112595+0000","last_change":"2024-04-24T22:59:57.112595+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.112595+0000","last_undegraded":"2024-04-24T22:59:57.112595+0000","last_fullsized":"2024-04-24T22:59:57.112595+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.108208+0000","last_change":"2024-04-24T22:59:57.108208+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.108208+0000","last_undegraded":"2024-04-24T22:59:57.108208+0000","last_fullsized":"2024-04-24T22:59:57.108208+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.109478+0000","last_change":"2024-04-24T22:59:57.109478+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.109478+0000","last_undegraded":"2024-04-24T22:59:57.109478+0000","last_fullsized":"2024-04-24T22:59:57.109478+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.109077+0000","last_change":"2024-04-24T22:59:57.109077+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.109077+0000","last_undegraded":"2024-04-24T22:59:57.109077+0000","last_fullsized":"2024-04-24T22:59:57.109077+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.110299+0000","last_change":"2024-04-24T22:59:57.110299+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.110299+0000","last_undegraded":"2024-04-24T22:59:57.110299+0000","last_fullsized":"2024-04-24T22:59:57.110299+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.108010+0000","last_change":"2024-04-24T22:59:57.108010+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.108010+0000","last_undegraded":"2024-04-24T22:59:57.108010+0000","last_fullsized":"2024-04-24T22:59:57.108010+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.115008+0000","last_change":"2024-04-24T22:59:57.115008+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.115008+0000","last_undegraded":"2024-04-24T22:59:57.115008+0000","last_fullsized":"2024-04-24T22:59:57.115008+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.109250+0000","last_change":"2024-04-24T22:59:57.109250+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.109250+0000","last_undegraded":"2024-04-24T22:59:57.109250+0000","last_fullsized":"2024-04-24T22:59:57.109250+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107645+0000","last_change":"2024-04-24T22:59:57.107645+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107645+0000","last_undegraded":"2024-04-24T22:59:57.107645+0000","last_fullsized":"2024-04-24T22:59:57.107645+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.109256+0000","last_change":"2024-04-24T22:59:57.109256+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.109256+0000","last_undegraded":"2024-04-24T22:59:57.109256+0000","last_fullsized":"2024-04-24T22:59:57.109256+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.111959+0000","last_change":"2024-04-24T22:59:57.111959+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.111959+0000","last_undegraded":"2024-04-24T22:59:57.111959+0000","last_fullsized":"2024-04-24T22:59:57.111959+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.111012+0000","last_change":"2024-04-24T22:59:57.111012+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.111012+0000","last_undegraded":"2024-04-24T22:59:57.111012+0000","last_fullsized":"2024-04-24T22:59:57.111012+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.112179+0000","last_change":"2024-04-24T22:59:57.112179+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.112179+0000","last_undegraded":"2024-04-24T22:59:57.112179+0000","last_fullsized":"2024-04-24T22:59:57.112179+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.109896+0000","last_change":"2024-04-24T22:59:57.109896+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.109896+0000","last_undegraded":"2024-04-24T22:59:57.109896+0000","last_fullsized":"2024-04-24T22:59:57.109896+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.111728+0000","last_change":"2024-04-24T22:59:57.111728+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.111728+0000","last_undegraded":"2024-04-24T22:59:57.111728+0000","last_fullsized":"2024-04-24T22:59:57.111728+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.109489+0000","last_change":"2024-04-24T22:59:57.109489+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.109489+0000","last_undegraded":"2024-04-24T22:59:57.109489+0000","last_fullsized":"2024-04-24T22:59:57.109489+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.108661+0000","last_change":"2024-04-24T22:59:57.108661+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.108661+0000","last_undegraded":"2024-04-24T22:59:57.108661+0000","last_fullsized":"2024-04-24T22:59:57.108661+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.109342+0000","last_change":"2024-04-24T22:59:57.109342+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.109342+0000","last_undegraded":"2024-04-24T22:59:57.109342+0000","last_fullsized":"2024-04-24T22:59:57.109342+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.108792+0000","last_change":"2024-04-24T22:59:57.108792+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.108792+0000","last_undegraded":"2024-04-24T22:59:57.108792+0000","last_fullsized":"2024-04-24T22:59:57.108792+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.108711+0000","last_change":"2024-04-24T22:59:57.108711+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.108711+0000","last_undegraded":"2024-04-24T22:59:57.108711+0000","last_fullsized":"2024-04-24T22:59:57.108711+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107354+0000","last_change":"2024-04-24T22:59:57.107354+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107354+0000","last_undegraded":"2024-04-24T22:59:57.107354+0000","last_fullsized":"2024-04-24T22:59:57.107354+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.114694+0000","last_change":"2024-04-24T22:59:57.114694+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.114694+0000","last_undegraded":"2024-04-24T22:59:57.114694+0000","last_fullsized":"2024-04-24T22:59:57.114694+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.110685+0000","last_change":"2024-04-24T22:59:57.110685+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.110685+0000","last_undegraded":"2024-04-24T22:59:57.110685+0000","last_fullsized":"2024-04-24T22:59:57.110685+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107448+0000","last_change":"2024-04-24T22:59:57.107448+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107448+0000","last_undegraded":"2024-04-24T22:59:57.107448+0000","last_fullsized":"2024-04-24T22:59:57.107448+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.109942+0000","last_change":"2024-04-24T22:59:57.109942+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.109942+0000","last_undegraded":"2024-04-24T22:59:57.109942+0000","last_fullsized":"2024-04-24T22:59:57.109942+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.108904+0000","last_change":"2024-04-24T22:59:57.108904+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.108904+0000","last_undegraded":"2024-04-24T22:59:57.108904+0000","last_fullsized":"2024-04-24T22:59:57.108904+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.110722+0000","last_change":"2024-04-24T22:59:57.110722+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.110722+0000","last_undegraded":"2024-04-24T22:59:57.110722+0000","last_fullsized":"2024-04-24T22:59:57.110722+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.108284+0000","last_change":"2024-04-24T22:59:57.108284+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.108284+0000","last_undegraded":"2024-04-24T22:59:57.108284+0000","last_fullsized":"2024-04-24T22:59:57.108284+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.109574+0000","last_change":"2024-04-24T22:59:57.109574+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.109574+0000","last_undegraded":"2024-04-24T22:59:57.109574+0000","last_fullsized":"2024-04-24T22:59:57.109574+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.108880+0000","last_change":"2024-04-24T22:59:57.108880+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.108880+0000","last_undegraded":"2024-04-24T22:59:57.108880+0000","last_fullsized":"2024-04-24T22:59:57.108880+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.115474+0000","last_change":"2024-04-24T22:59:57.114774+0000","last_active":"2024-04-24T22:59:57.115474+0000","last_peered":"2024-04-24T22:59:57.115474+0000","last_clean":"2024-04-24T22:59:57.115474+0000","last_became_active":"2024-04-24T22:59:57.114031+0000","last_became_peered":"2024-04-24T22:59:57.114031+0000","last_unstale":"2024-04-24T22:59:57.115474+0000","last_undegraded":"2024-04-24T22:59:57.115474+0000","last_fullsized":"2024-04-24T22:59:57.115474+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:02:17.915932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.108182+0000","last_change":"2024-04-24T22:59:57.108182+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.108182+0000","last_undegraded":"2024-04-24T22:59:57.108182+0000","last_fullsized":"2024-04-24T22:59:57.108182+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.118962+0000","last_change":"2024-04-24T22:59:57.118830+0000","last_active":"2024-04-24T22:59:57.118962+0000","last_peered":"2024-04-24T22:59:57.118962+0000","last_clean":"2024-04-24T22:59:57.118962+0000","last_became_active":"2024-04-24T22:59:57.118380+0000","last_became_peered":"2024-04-24T22:59:57.118380+0000","last_unstale":"2024-04-24T22:59:57.118962+0000","last_undegraded":"2024-04-24T22:59:57.118962+0000","last_fullsized":"2024-04-24T22:59:57.118962+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:34:34.824377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107978+0000","last_change":"2024-04-24T22:59:57.107978+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107978+0000","last_undegraded":"2024-04-24T22:59:57.107978+0000","last_fullsized":"2024-04-24T22:59:57.107978+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.113528+0000","last_change":"2024-04-24T22:59:57.113067+0000","last_active":"2024-04-24T22:59:57.113528+0000","last_peered":"2024-04-24T22:59:57.113528+0000","last_clean":"2024-04-24T22:59:57.113528+0000","last_became_active":"2024-04-24T22:59:57.112774+0000","last_became_peered":"2024-04-24T22:59:57.112774+0000","last_unstale":"2024-04-24T22:59:57.113528+0000","last_undegraded":"2024-04-24T22:59:57.113528+0000","last_fullsized":"2024-04-24T22:59:57.113528+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:09.897525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.114355+0000","last_change":"2024-04-24T22:59:57.114355+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.114355+0000","last_undegraded":"2024-04-24T22:59:57.114355+0000","last_fullsized":"2024-04-24T22:59:57.114355+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.118693+0000","last_change":"2024-04-24T22:59:57.118605+0000","last_active":"2024-04-24T22:59:57.118693+0000","last_peered":"2024-04-24T22:59:57.118693+0000","last_clean":"2024-04-24T22:59:57.118693+0000","last_became_active":"2024-04-24T22:59:57.118271+0000","last_became_peered":"2024-04-24T22:59:57.118271+0000","last_unstale":"2024-04-24T22:59:57.118693+0000","last_undegraded":"2024-04-24T22:59:57.118693+0000","last_fullsized":"2024-04-24T22:59:57.118693+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:04:12.275266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107238+0000","last_change":"2024-04-24T22:59:57.107238+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107238+0000","last_undegraded":"2024-04-24T22:59:57.107238+0000","last_fullsized":"2024-04-24T22:59:57.107238+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.109905+0000","last_change":"2024-04-24T22:59:57.109569+0000","last_active":"2024-04-24T22:59:57.109905+0000","last_peered":"2024-04-24T22:59:57.109905+0000","last_clean":"2024-04-24T22:59:57.109905+0000","last_became_active":"2024-04-24T22:59:57.109350+0000","last_became_peered":"2024-04-24T22:59:57.109350+0000","last_unstale":"2024-04-24T22:59:57.109905+0000","last_undegraded":"2024-04-24T22:59:57.109905+0000","last_fullsized":"2024-04-24T22:59:57.109905+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:54.689993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107517+0000","last_change":"2024-04-24T22:59:57.107517+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107517+0000","last_undegraded":"2024-04-24T22:59:57.107517+0000","last_fullsized":"2024-04-24T22:59:57.107517+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.113519+0000","last_change":"2024-04-24T22:59:57.113397+0000","last_active":"2024-04-24T22:59:57.113519+0000","last_peered":"2024-04-24T22:59:57.113519+0000","last_clean":"2024-04-24T22:59:57.113519+0000","last_became_active":"2024-04-24T22:59:57.113167+0000","last_became_peered":"2024-04-24T22:59:57.113167+0000","last_unstale":"2024-04-24T22:59:57.113519+0000","last_undegraded":"2024-04-24T22:59:57.113519+0000","last_fullsized":"2024-04-24T22:59:57.113519+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:29:04.934394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107218+0000","last_change":"2024-04-24T22:59:57.107218+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107218+0000","last_undegraded":"2024-04-24T22:59:57.107218+0000","last_fullsized":"2024-04-24T22:59:57.107218+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.116661+0000","last_change":"2024-04-24T22:59:57.116514+0000","last_active":"2024-04-24T22:59:57.116661+0000","last_peered":"2024-04-24T22:59:57.116661+0000","last_clean":"2024-04-24T22:59:57.116661+0000","last_became_active":"2024-04-24T22:59:57.116259+0000","last_became_peered":"2024-04-24T22:59:57.116259+0000","last_unstale":"2024-04-24T22:59:57.116661+0000","last_undegraded":"2024-04-24T22:59:57.116661+0000","last_fullsized":"2024-04-24T22:59:57.116661+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:43:58.559759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.112898+0000","last_change":"2024-04-24T22:59:57.112898+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.112898+0000","last_undegraded":"2024-04-24T22:59:57.112898+0000","last_fullsized":"2024-04-24T22:59:57.112898+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.117357+0000","last_change":"2024-04-24T22:59:57.117209+0000","last_active":"2024-04-24T22:59:57.117357+0000","last_peered":"2024-04-24T22:59:57.117357+0000","last_clean":"2024-04-24T22:59:57.117357+0000","last_became_active":"2024-04-24T22:59:57.116968+0000","last_became_peered":"2024-04-24T22:59:57.116968+0000","last_unstale":"2024-04-24T22:59:57.117357+0000","last_undegraded":"2024-04-24T22:59:57.117357+0000","last_fullsized":"2024-04-24T22:59:57.117357+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:44:17.871555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.109086+0000","last_change":"2024-04-24T22:59:57.109086+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.109086+0000","last_undegraded":"2024-04-24T22:59:57.109086+0000","last_fullsized":"2024-04-24T22:59:57.109086+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.106993+0000","last_change":"2024-04-24T22:59:57.106612+0000","last_active":"2024-04-24T22:59:57.106993+0000","last_peered":"2024-04-24T22:59:57.106993+0000","last_clean":"2024-04-24T22:59:57.106993+0000","last_became_active":"2024-04-24T22:59:57.106350+0000","last_became_peered":"2024-04-24T22:59:57.106350+0000","last_unstale":"2024-04-24T22:59:57.106993+0000","last_undegraded":"2024-04-24T22:59:57.106993+0000","last_fullsized":"2024-04-24T22:59:57.106993+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:45:14.576793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.112099+0000","last_change":"2024-04-24T22:59:57.112099+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.112099+0000","last_undegraded":"2024-04-24T22:59:57.112099+0000","last_fullsized":"2024-04-24T22:59:57.112099+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.115220+0000","last_change":"2024-04-24T22:59:57.115098+0000","last_active":"2024-04-24T22:59:57.115220+0000","last_peered":"2024-04-24T22:59:57.115220+0000","last_clean":"2024-04-24T22:59:57.115220+0000","last_became_active":"2024-04-24T22:59:57.114866+0000","last_became_peered":"2024-04-24T22:59:57.114866+0000","last_unstale":"2024-04-24T22:59:57.115220+0000","last_undegraded":"2024-04-24T22:59:57.115220+0000","last_fullsized":"2024-04-24T22:59:57.115220+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:27:09.165254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.111004+0000","last_change":"2024-04-24T22:59:57.111004+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.111004+0000","last_undegraded":"2024-04-24T22:59:57.111004+0000","last_fullsized":"2024-04-24T22:59:57.111004+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.115493+0000","last_change":"2024-04-24T22:59:57.115370+0000","last_active":"2024-04-24T22:59:57.115493+0000","last_peered":"2024-04-24T22:59:57.115493+0000","last_clean":"2024-04-24T22:59:57.115493+0000","last_became_active":"2024-04-24T22:59:57.115147+0000","last_became_peered":"2024-04-24T22:59:57.115147+0000","last_unstale":"2024-04-24T22:59:57.115493+0000","last_undegraded":"2024-04-24T22:59:57.115493+0000","last_fullsized":"2024-04-24T22:59:57.115493+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:23:03.727015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.108512+0000","last_change":"2024-04-24T22:59:57.108512+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.108512+0000","last_undegraded":"2024-04-24T22:59:57.108512+0000","last_fullsized":"2024-04-24T22:59:57.108512+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.118330+0000","last_change":"2024-04-24T22:59:57.118202+0000","last_active":"2024-04-24T22:59:57.118330+0000","last_peered":"2024-04-24T22:59:57.118330+0000","last_clean":"2024-04-24T22:59:57.118330+0000","last_became_active":"2024-04-24T22:59:57.117946+0000","last_became_peered":"2024-04-24T22:59:57.117946+0000","last_unstale":"2024-04-24T22:59:57.118330+0000","last_undegraded":"2024-04-24T22:59:57.118330+0000","last_fullsized":"2024-04-24T22:59:57.118330+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:37:52.953926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107500+0000","last_change":"2024-04-24T22:59:57.107500+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107500+0000","last_undegraded":"2024-04-24T22:59:57.107500+0000","last_fullsized":"2024-04-24T22:59:57.107500+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.112009+0000","last_change":"2024-04-24T22:59:57.111899+0000","last_active":"2024-04-24T22:59:57.112009+0000","last_peered":"2024-04-24T22:59:57.112009+0000","last_clean":"2024-04-24T22:59:57.112009+0000","last_became_active":"2024-04-24T22:59:57.111632+0000","last_became_peered":"2024-04-24T22:59:57.111632+0000","last_unstale":"2024-04-24T22:59:57.112009+0000","last_undegraded":"2024-04-24T22:59:57.112009+0000","last_fullsized":"2024-04-24T22:59:57.112009+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:01.204587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.114010+0000","last_change":"2024-04-24T22:59:57.114010+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.114010+0000","last_undegraded":"2024-04-24T22:59:57.114010+0000","last_fullsized":"2024-04-24T22:59:57.114010+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.116958+0000","last_change":"2024-04-24T22:59:57.116842+0000","last_active":"2024-04-24T22:59:57.116958+0000","last_peered":"2024-04-24T22:59:57.116958+0000","last_clean":"2024-04-24T22:59:57.116958+0000","last_became_active":"2024-04-24T22:59:57.116577+0000","last_became_peered":"2024-04-24T22:59:57.116577+0000","last_unstale":"2024-04-24T22:59:57.116958+0000","last_undegraded":"2024-04-24T22:59:57.116958+0000","last_fullsized":"2024-04-24T22:59:57.116958+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:29.595256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107852+0000","last_change":"2024-04-24T22:59:57.107852+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107852+0000","last_undegraded":"2024-04-24T22:59:57.107852+0000","last_fullsized":"2024-04-24T22:59:57.107852+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.115153+0000","last_change":"2024-04-24T22:59:57.113832+0000","last_active":"2024-04-24T22:59:57.115153+0000","last_peered":"2024-04-24T22:59:57.115153+0000","last_clean":"2024-04-24T22:59:57.115153+0000","last_became_active":"2024-04-24T22:59:57.113610+0000","last_became_peered":"2024-04-24T22:59:57.113610+0000","last_unstale":"2024-04-24T22:59:57.115153+0000","last_undegraded":"2024-04-24T22:59:57.115153+0000","last_fullsized":"2024-04-24T22:59:57.115153+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:49:50.026489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.108615+0000","last_change":"2024-04-24T22:59:57.108615+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.108615+0000","last_undegraded":"2024-04-24T22:59:57.108615+0000","last_fullsized":"2024-04-24T22:59:57.108615+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.116358+0000","last_change":"2024-04-24T22:59:57.116182+0000","last_active":"2024-04-24T22:59:57.116358+0000","last_peered":"2024-04-24T22:59:57.116358+0000","last_clean":"2024-04-24T22:59:57.116358+0000","last_became_active":"2024-04-24T22:59:57.115952+0000","last_became_peered":"2024-04-24T22:59:57.115952+0000","last_unstale":"2024-04-24T22:59:57.116358+0000","last_undegraded":"2024-04-24T22:59:57.116358+0000","last_fullsized":"2024-04-24T22:59:57.116358+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:20:34.273911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107248+0000","last_change":"2024-04-24T22:59:57.107248+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107248+0000","last_undegraded":"2024-04-24T22:59:57.107248+0000","last_fullsized":"2024-04-24T22:59:57.107248+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.119286+0000","last_change":"2024-04-24T22:59:57.119183+0000","last_active":"2024-04-24T22:59:57.119286+0000","last_peered":"2024-04-24T22:59:57.119286+0000","last_clean":"2024-04-24T22:59:57.119286+0000","last_became_active":"2024-04-24T22:59:57.118911+0000","last_became_peered":"2024-04-24T22:59:57.118911+0000","last_unstale":"2024-04-24T22:59:57.119286+0000","last_undegraded":"2024-04-24T22:59:57.119286+0000","last_fullsized":"2024-04-24T22:59:57.119286+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:28:06.461401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.106893+0000","last_change":"2024-04-24T22:59:57.106893+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.106893+0000","last_undegraded":"2024-04-24T22:59:57.106893+0000","last_fullsized":"2024-04-24T22:59:57.106893+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.115503+0000","last_change":"2024-04-24T22:59:57.115384+0000","last_active":"2024-04-24T22:59:57.115503+0000","last_peered":"2024-04-24T22:59:57.115503+0000","last_clean":"2024-04-24T22:59:57.115503+0000","last_became_active":"2024-04-24T22:59:57.115129+0000","last_became_peered":"2024-04-24T22:59:57.115129+0000","last_unstale":"2024-04-24T22:59:57.115503+0000","last_undegraded":"2024-04-24T22:59:57.115503+0000","last_fullsized":"2024-04-24T22:59:57.115503+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:13:57.006308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107117+0000","last_change":"2024-04-24T22:59:57.107117+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107117+0000","last_undegraded":"2024-04-24T22:59:57.107117+0000","last_fullsized":"2024-04-24T22:59:57.107117+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.114295+0000","last_change":"2024-04-24T22:59:57.113034+0000","last_active":"2024-04-24T22:59:57.114295+0000","last_peered":"2024-04-24T22:59:57.114295+0000","last_clean":"2024-04-24T22:59:57.114295+0000","last_became_active":"2024-04-24T22:59:57.111752+0000","last_became_peered":"2024-04-24T22:59:57.111752+0000","last_unstale":"2024-04-24T22:59:57.114295+0000","last_undegraded":"2024-04-24T22:59:57.114295+0000","last_fullsized":"2024-04-24T22:59:57.114295+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:11:29.081634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.109090+0000","last_change":"2024-04-24T22:59:57.109090+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.109090+0000","last_undegraded":"2024-04-24T22:59:57.109090+0000","last_fullsized":"2024-04-24T22:59:57.109090+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.117410+0000","last_change":"2024-04-24T22:59:57.117253+0000","last_active":"2024-04-24T22:59:57.117410+0000","last_peered":"2024-04-24T22:59:57.117410+0000","last_clean":"2024-04-24T22:59:57.117410+0000","last_became_active":"2024-04-24T22:59:57.116974+0000","last_became_peered":"2024-04-24T22:59:57.116974+0000","last_unstale":"2024-04-24T22:59:57.117410+0000","last_undegraded":"2024-04-24T22:59:57.117410+0000","last_fullsized":"2024-04-24T22:59:57.117410+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:24.934926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107613+0000","last_change":"2024-04-24T22:59:57.107613+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107613+0000","last_undegraded":"2024-04-24T22:59:57.107613+0000","last_fullsized":"2024-04-24T22:59:57.107613+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.112446+0000","last_change":"2024-04-24T22:59:57.111772+0000","last_active":"2024-04-24T22:59:57.112446+0000","last_peered":"2024-04-24T22:59:57.112446+0000","last_clean":"2024-04-24T22:59:57.112446+0000","last_became_active":"2024-04-24T22:59:57.111452+0000","last_became_peered":"2024-04-24T22:59:57.111452+0000","last_unstale":"2024-04-24T22:59:57.112446+0000","last_undegraded":"2024-04-24T22:59:57.112446+0000","last_fullsized":"2024-04-24T22:59:57.112446+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:27.552548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107006+0000","last_change":"2024-04-24T22:59:57.107006+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107006+0000","last_undegraded":"2024-04-24T22:59:57.107006+0000","last_fullsized":"2024-04-24T22:59:57.107006+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.111880+0000","last_change":"2024-04-24T22:59:57.111232+0000","last_active":"2024-04-24T22:59:57.111880+0000","last_peered":"2024-04-24T22:59:57.111880+0000","last_clean":"2024-04-24T22:59:57.111880+0000","last_became_active":"2024-04-24T22:59:57.111010+0000","last_became_peered":"2024-04-24T22:59:57.111010+0000","last_unstale":"2024-04-24T22:59:57.111880+0000","last_undegraded":"2024-04-24T22:59:57.111880+0000","last_fullsized":"2024-04-24T22:59:57.111880+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:02:29.250634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107089+0000","last_change":"2024-04-24T22:59:57.107089+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107089+0000","last_undegraded":"2024-04-24T22:59:57.107089+0000","last_fullsized":"2024-04-24T22:59:57.107089+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.111016+0000","last_change":"2024-04-24T22:59:57.110898+0000","last_active":"2024-04-24T22:59:57.111016+0000","last_peered":"2024-04-24T22:59:57.111016+0000","last_clean":"2024-04-24T22:59:57.111016+0000","last_became_active":"2024-04-24T22:59:57.110665+0000","last_became_peered":"2024-04-24T22:59:57.110665+0000","last_unstale":"2024-04-24T22:59:57.111016+0000","last_undegraded":"2024-04-24T22:59:57.111016+0000","last_fullsized":"2024-04-24T22:59:57.111016+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:55.518131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.106526+0000","last_change":"2024-04-24T22:59:57.106526+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.106526+0000","last_undegraded":"2024-04-24T22:59:57.106526+0000","last_fullsized":"2024-04-24T22:59:57.106526+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.114739+0000","last_change":"2024-04-24T22:59:57.114417+0000","last_active":"2024-04-24T22:59:57.114739+0000","last_peered":"2024-04-24T22:59:57.114739+0000","last_clean":"2024-04-24T22:59:57.114739+0000","last_became_active":"2024-04-24T22:59:57.112771+0000","last_became_peered":"2024-04-24T22:59:57.112771+0000","last_unstale":"2024-04-24T22:59:57.114739+0000","last_undegraded":"2024-04-24T22:59:57.114739+0000","last_fullsized":"2024-04-24T22:59:57.114739+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:11:37.810064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.110252+0000","last_change":"2024-04-24T22:59:57.110252+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.110252+0000","last_undegraded":"2024-04-24T22:59:57.110252+0000","last_fullsized":"2024-04-24T22:59:57.110252+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.115885+0000","last_change":"2024-04-24T22:59:57.115765+0000","last_active":"2024-04-24T22:59:57.115885+0000","last_peered":"2024-04-24T22:59:57.115885+0000","last_clean":"2024-04-24T22:59:57.115885+0000","last_became_active":"2024-04-24T22:59:57.115056+0000","last_became_peered":"2024-04-24T22:59:57.115056+0000","last_unstale":"2024-04-24T22:59:57.115885+0000","last_undegraded":"2024-04-24T22:59:57.115885+0000","last_fullsized":"2024-04-24T22:59:57.115885+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:18:11.299169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.106865+0000","last_change":"2024-04-24T22:59:57.106865+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.106865+0000","last_undegraded":"2024-04-24T22:59:57.106865+0000","last_fullsized":"2024-04-24T22:59:57.106865+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.113678+0000","last_change":"2024-04-24T22:59:57.113564+0000","last_active":"2024-04-24T22:59:57.113678+0000","last_peered":"2024-04-24T22:59:57.113678+0000","last_clean":"2024-04-24T22:59:57.113678+0000","last_became_active":"2024-04-24T22:59:57.113292+0000","last_became_peered":"2024-04-24T22:59:57.113292+0000","last_unstale":"2024-04-24T22:59:57.113678+0000","last_undegraded":"2024-04-24T22:59:57.113678+0000","last_fullsized":"2024-04-24T22:59:57.113678+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:05:41.614373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.109161+0000","last_change":"2024-04-24T22:59:57.109161+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.109161+0000","last_undegraded":"2024-04-24T22:59:57.109161+0000","last_fullsized":"2024-04-24T22:59:57.109161+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.112241+0000","last_change":"2024-04-24T22:59:57.111930+0000","last_active":"2024-04-24T22:59:57.112241+0000","last_peered":"2024-04-24T22:59:57.112241+0000","last_clean":"2024-04-24T22:59:57.112241+0000","last_became_active":"2024-04-24T22:59:57.111632+0000","last_became_peered":"2024-04-24T22:59:57.111632+0000","last_unstale":"2024-04-24T22:59:57.112241+0000","last_undegraded":"2024-04-24T22:59:57.112241+0000","last_fullsized":"2024-04-24T22:59:57.112241+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:57.770833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.110730+0000","last_change":"2024-04-24T22:59:57.110730+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.110730+0000","last_undegraded":"2024-04-24T22:59:57.110730+0000","last_fullsized":"2024-04-24T22:59:57.110730+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.115239+0000","last_change":"2024-04-24T22:59:57.115123+0000","last_active":"2024-04-24T22:59:57.115239+0000","last_peered":"2024-04-24T22:59:57.115239+0000","last_clean":"2024-04-24T22:59:57.115239+0000","last_became_active":"2024-04-24T22:59:57.114845+0000","last_became_peered":"2024-04-24T22:59:57.114845+0000","last_unstale":"2024-04-24T22:59:57.115239+0000","last_undegraded":"2024-04-24T22:59:57.115239+0000","last_fullsized":"2024-04-24T22:59:57.115239+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:31.884038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.111780+0000","last_change":"2024-04-24T22:59:57.111780+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.111780+0000","last_undegraded":"2024-04-24T22:59:57.111780+0000","last_fullsized":"2024-04-24T22:59:57.111780+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.112631+0000","last_change":"2024-04-24T22:59:57.112511+0000","last_active":"2024-04-24T22:59:57.112631+0000","last_peered":"2024-04-24T22:59:57.112631+0000","last_clean":"2024-04-24T22:59:57.112631+0000","last_became_active":"2024-04-24T22:59:57.112210+0000","last_became_peered":"2024-04-24T22:59:57.112210+0000","last_unstale":"2024-04-24T22:59:57.112631+0000","last_undegraded":"2024-04-24T22:59:57.112631+0000","last_fullsized":"2024-04-24T22:59:57.112631+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:04.204693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.112288+0000","last_change":"2024-04-24T22:59:57.112288+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.112288+0000","last_undegraded":"2024-04-24T22:59:57.112288+0000","last_fullsized":"2024-04-24T22:59:57.112288+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.111354+0000","last_change":"2024-04-24T22:59:57.111232+0000","last_active":"2024-04-24T22:59:57.111354+0000","last_peered":"2024-04-24T22:59:57.111354+0000","last_clean":"2024-04-24T22:59:57.111354+0000","last_became_active":"2024-04-24T22:59:57.110997+0000","last_became_peered":"2024-04-24T22:59:57.110997+0000","last_unstale":"2024-04-24T22:59:57.111354+0000","last_undegraded":"2024-04-24T22:59:57.111354+0000","last_fullsized":"2024-04-24T22:59:57.111354+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:56:07.604519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107025+0000","last_change":"2024-04-24T22:59:57.107025+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107025+0000","last_undegraded":"2024-04-24T22:59:57.107025+0000","last_fullsized":"2024-04-24T22:59:57.107025+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.118210+0000","last_change":"2024-04-24T22:59:57.118085+0000","last_active":"2024-04-24T22:59:57.118210+0000","last_peered":"2024-04-24T22:59:57.118210+0000","last_clean":"2024-04-24T22:59:57.118210+0000","last_became_active":"2024-04-24T22:59:57.116287+0000","last_became_peered":"2024-04-24T22:59:57.116287+0000","last_unstale":"2024-04-24T22:59:57.118210+0000","last_undegraded":"2024-04-24T22:59:57.118210+0000","last_fullsized":"2024-04-24T22:59:57.118210+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:11:38.330427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.110417+0000","last_change":"2024-04-24T22:59:57.110417+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.110417+0000","last_undegraded":"2024-04-24T22:59:57.110417+0000","last_fullsized":"2024-04-24T22:59:57.110417+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.112334+0000","last_change":"2024-04-24T22:59:57.110803+0000","last_active":"2024-04-24T22:59:57.112334+0000","last_peered":"2024-04-24T22:59:57.112334+0000","last_clean":"2024-04-24T22:59:57.112334+0000","last_became_active":"2024-04-24T22:59:57.110572+0000","last_became_peered":"2024-04-24T22:59:57.110572+0000","last_unstale":"2024-04-24T22:59:57.112334+0000","last_undegraded":"2024-04-24T22:59:57.112334+0000","last_fullsized":"2024-04-24T22:59:57.112334+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:06:59.730529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":186,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.105292+0000","last_change":"2024-04-24T22:58:55.138810+0000","last_active":"2024-04-24T22:59:57.105292+0000","last_peered":"2024-04-24T22:59:57.105292+0000","last_clean":"2024-04-24T22:59:57.105292+0000","last_became_active":"2024-04-24T22:58:55.138627+0000","last_became_peered":"2024-04-24T22:58:55.138627+0000","last_unstale":"2024-04-24T22:59:57.105292+0000","last_undegraded":"2024-04-24T22:59:57.105292+0000","last_fullsized":"2024-04-24T22:59:57.105292+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_clean_scrub_stamp":"2024-04-24T22:58:54.127235+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:24.010002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":116,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195196,"statfs":{"total":200000000,"available":199880705,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":116,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195196,"statfs":{"total":200000000,"available":199880705,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":693,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194619,"statfs":{"total":200000000,"available":199290337,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":116,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195196,"statfs":{"total":200000000,"available":199880705,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":116,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195196,"statfs":{"total":200000000,"available":199880705,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":116,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195196,"statfs":{"total":200000000,"available":199880705,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":687,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194625,"statfs":{"total":200000000,"available":199296154,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":116,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195196,"statfs":{"total":200000000,"available":199880705,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-24T23:00:01.115 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:00:01.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.250+0000 7fa85c829640 1 Processor -- start 2024-04-24T23:00:01.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.250+0000 7fa85c829640 1 -- start start 2024-04-24T23:00:01.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.251+0000 7fa85c829640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa85806c5f0 0x7fa85806c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:01.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.251+0000 7fa85c829640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa8580af890 0x7fa85815cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:01.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.251+0000 7fa85c829640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8580b6430 con 0x7fa8580b44d0 2024-04-24T23:00:01.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.251+0000 7fa85c829640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fa858156f80 con 0x7fa8580af890 2024-04-24T23:00:01.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.251+0000 7fa85c829640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa8580b6150 con 0x7fa85806c5f0 2024-04-24T23:00:01.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.252+0000 7fa856d9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa8580af890 0x7fa85815cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:01.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.252+0000 7fa856d9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa8580af890 0x7fa85815cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60034/0 (socket says 172.21.15.32:60034) 2024-04-24T23:00:01.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.252+0000 7fa856d9d640 1 -- 172.21.15.32:0/1204514075 learned_addr learned my addr 172.21.15.32:0/1204514075 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:01.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.252+0000 7fa856d9d640 1 -- 172.21.15.32:0/1204514075 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa85806c5f0 msgr2=0x7fa85806c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:01.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.252+0000 7fa85659c640 1 --2- 172.21.15.32:0/1204514075 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa85806c5f0 0x7fa85806c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:01.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.252+0000 7fa856d9d640 1 --2- 172.21.15.32:0/1204514075 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa85806c5f0 0x7fa85806c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:01.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.252+0000 7fa856d9d640 1 -- 172.21.15.32:0/1204514075 >> v1:172.21.15.121:6789/0 conn(0x7fa8580b44d0 legacy=0x7fa8580b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:01.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.252+0000 7fa856d9d640 1 -- 172.21.15.32:0/1204514075 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8580b39f0 con 0x7fa8580af890 2024-04-24T23:00:01.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.252+0000 7fa85659c640 1 --2- 172.21.15.32:0/1204514075 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa85806c5f0 0x7fa85806c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:00:01.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.253+0000 7fa856d9d640 1 --2- 172.21.15.32:0/1204514075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8580af890 0x7fa85815cac0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa850009e50 tx=0x7fa85002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb6ec1f15b4571c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:01.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.253+0000 7fa85559a640 1 -- 172.21.15.32:0/1204514075 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa850036070 con 0x7fa8580af890 2024-04-24T23:00:01.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.253+0000 7fa85559a640 1 -- 172.21.15.32:0/1204514075 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa850031600 con 0x7fa8580af890 2024-04-24T23:00:01.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.253+0000 7fa85559a640 1 -- 172.21.15.32:0/1204514075 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8500319d0 con 0x7fa8580af890 2024-04-24T23:00:01.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.253+0000 7fa85c829640 1 -- 172.21.15.32:0/1204514075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8580af890 msgr2=0x7fa85815cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:01.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.253+0000 7fa85c829640 1 --2- 172.21.15.32:0/1204514075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8580af890 0x7fa85815cac0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa850009e50 tx=0x7fa85002a680 comp rx=0 tx=0).stop 2024-04-24T23:00:01.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.253+0000 7fa85c829640 1 -- 172.21.15.32:0/1204514075 shutdown_connections 2024-04-24T23:00:01.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.253+0000 7fa85c829640 1 --2- 172.21.15.32:0/1204514075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8580af890 0x7fa85815cac0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:01.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.253+0000 7fa85c829640 1 --2- 172.21.15.32:0/1204514075 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa85806c5f0 0x7fa85806c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:01.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.253+0000 7fa85c829640 1 -- 172.21.15.32:0/1204514075 >> 172.21.15.32:0/1204514075 conn(0x7fa85806b110 msgr2=0x7fa85806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:01.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.253+0000 7fa85c829640 1 -- 172.21.15.32:0/1204514075 shutdown_connections 2024-04-24T23:00:01.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.254+0000 7fa85c829640 1 -- 172.21.15.32:0/1204514075 wait complete. 2024-04-24T23:00:01.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.255+0000 7fa85c829640 1 Processor -- start 2024-04-24T23:00:01.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.255+0000 7fa85c829640 1 -- start start 2024-04-24T23:00:01.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.255+0000 7fa85c829640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa85806c5f0 0x7fa8580ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:01.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.256+0000 7fa85c829640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa8580af890 0x7fa8580b1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:01.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.256+0000 7fa85c829640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa8580b44d0 0x7fa85814acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:01.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.256+0000 7fa85c829640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa85806cf10 con 0x7fa85806c5f0 2024-04-24T23:00:01.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.256+0000 7fa85c829640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa8580b3fb0 con 0x7fa8580af890 2024-04-24T23:00:01.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.256+0000 7fa85c829640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa85815d000 con 0x7fa8580b44d0 2024-04-24T23:00:01.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.256+0000 7fa85659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa85806c5f0 0x7fa8580ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:01.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.256+0000 7fa85659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa85806c5f0 0x7fa8580ccf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60042/0 (socket says 172.21.15.32:60042) 2024-04-24T23:00:01.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.256+0000 7fa85659c640 1 -- 172.21.15.32:0/647218925 learned_addr learned my addr 172.21.15.32:0/647218925 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:01.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.256+0000 7fa856d9d640 1 --2- 172.21.15.32:0/647218925 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa8580b44d0 0x7fa85814acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:01.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.256+0000 7fa855d9b640 1 --2- 172.21.15.32:0/647218925 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa8580af890 0x7fa8580b1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:01.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.256+0000 7fa85659c640 1 -- 172.21.15.32:0/647218925 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa8580b44d0 msgr2=0x7fa85814acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:01.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.256+0000 7fa85659c640 1 --2- 172.21.15.32:0/647218925 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa8580b44d0 0x7fa85814acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:01.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.256+0000 7fa85659c640 1 -- 172.21.15.32:0/647218925 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa8580af890 msgr2=0x7fa8580b1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:01.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.256+0000 7fa85659c640 1 --2- 172.21.15.32:0/647218925 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa8580af890 0x7fa8580b1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:01.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.256+0000 7fa85659c640 1 -- 172.21.15.32:0/647218925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa858153890 con 0x7fa85806c5f0 2024-04-24T23:00:01.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.256+0000 7fa856d9d640 1 --2- 172.21.15.32:0/647218925 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa8580b44d0 0x7fa85814acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:00:01.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.256+0000 7fa855d9b640 1 --2- 172.21.15.32:0/647218925 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa8580af890 0x7fa8580b1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:00:01.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.256+0000 7fa85659c640 1 --2- 172.21.15.32:0/647218925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa85806c5f0 0x7fa8580ccf60 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa84800c820 tx=0x7fa84800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:01.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.256+0000 7fa846ffd640 1 -- 172.21.15.32:0/647218925 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa848015070 con 0x7fa85806c5f0 2024-04-24T23:00:01.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.256+0000 7fa846ffd640 1 -- 172.21.15.32:0/647218925 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa848007a30 con 0x7fa85806c5f0 2024-04-24T23:00:01.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.256+0000 7fa85c829640 1 -- 172.21.15.32:0/647218925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa858154610 con 0x7fa85806c5f0 2024-04-24T23:00:01.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.256+0000 7fa846ffd640 1 -- 172.21.15.32:0/647218925 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa848007e00 con 0x7fa85806c5f0 2024-04-24T23:00:01.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.256+0000 7fa85c829640 1 -- 172.21.15.32:0/647218925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa858153e10 con 0x7fa85806c5f0 2024-04-24T23:00:01.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.257+0000 7fa85c829640 1 -- 172.21.15.32:0/647218925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa858068430 con 0x7fa85806c5f0 2024-04-24T23:00:01.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.258+0000 7fa846ffd640 1 -- 172.21.15.32:0/647218925 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa848018070 con 0x7fa85806c5f0 2024-04-24T23:00:01.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.258+0000 7fa846ffd640 1 --2- 172.21.15.32:0/647218925 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa83806be30 0x7fa83806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:01.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.258+0000 7fa846ffd640 1 -- 172.21.15.32:0/647218925 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fa84808add0 con 0x7fa85806c5f0 2024-04-24T23:00:01.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.258+0000 7fa855d9b640 1 --2- 172.21.15.32:0/647218925 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa83806be30 0x7fa83806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:01.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.259+0000 7fa855d9b640 1 --2- 172.21.15.32:0/647218925 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa83806be30 0x7fa83806e2f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa8580cef40 tx=0x7fa84c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:01.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.262+0000 7fa846ffd640 1 -- 172.21.15.32:0/647218925 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa848059780 con 0x7fa85806c5f0 2024-04-24T23:00:01.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.441+0000 7fa85c829640 1 -- 172.21.15.32:0/647218925 --> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa8580b5670 con 0x7fa83806be30 2024-04-24T23:00:01.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.447+0000 7fa846ffd640 1 -- 172.21.15.32:0/647218925 <== mgr.4108 v2:172.21.15.121:6832/1848157943 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+242215 (secure 0 0 0) 0x7fa8580b5670 con 0x7fa83806be30 2024-04-24T23:00:01.451 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:00:01.453 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-24T23:00:01.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.450+0000 7fa85c829640 1 -- 172.21.15.32:0/647218925 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa83806be30 msgr2=0x7fa83806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:01.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.450+0000 7fa85c829640 1 --2- 172.21.15.32:0/647218925 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa83806be30 0x7fa83806e2f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa8580cef40 tx=0x7fa84c009040 comp rx=0 tx=0).stop 2024-04-24T23:00:01.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.450+0000 7fa85c829640 1 -- 172.21.15.32:0/647218925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa85806c5f0 msgr2=0x7fa8580ccf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:01.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.450+0000 7fa85c829640 1 --2- 172.21.15.32:0/647218925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa85806c5f0 0x7fa8580ccf60 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa84800c820 tx=0x7fa84800cc20 comp rx=0 tx=0).stop 2024-04-24T23:00:01.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.450+0000 7fa85c829640 1 -- 172.21.15.32:0/647218925 shutdown_connections 2024-04-24T23:00:01.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.450+0000 7fa85c829640 1 --2- 172.21.15.32:0/647218925 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa83806be30 0x7fa83806e2f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:01.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.450+0000 7fa85c829640 1 --2- 172.21.15.32:0/647218925 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa8580b44d0 0x7fa85814acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:01.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.450+0000 7fa85c829640 1 --2- 172.21.15.32:0/647218925 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa8580af890 0x7fa8580b1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:01.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.450+0000 7fa85c829640 1 --2- 172.21.15.32:0/647218925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa85806c5f0 0x7fa8580ccf60 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:01.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.450+0000 7fa85c829640 1 -- 172.21.15.32:0/647218925 >> 172.21.15.32:0/647218925 conn(0x7fa85806b110 msgr2=0x7fa858155e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:01.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.450+0000 7fa85c829640 1 -- 172.21.15.32:0/647218925 shutdown_connections 2024-04-24T23:00:01.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.450+0000 7fa85c829640 1 -- 172.21.15.32:0/647218925 wait complete. 2024-04-24T23:00:01.463 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":58,"stamp":"2024-04-24T23:00:01.140748+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2079,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560420,"statfs":{"total":1600000000,"available":1597870721,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.041342"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.108588+0000","last_change":"2024-04-24T22:59:57.108588+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.108588+0000","last_undegraded":"2024-04-24T22:59:57.108588+0000","last_fullsized":"2024-04-24T22:59:57.108588+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.108603+0000","last_change":"2024-04-24T22:59:57.108603+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.108603+0000","last_undegraded":"2024-04-24T22:59:57.108603+0000","last_fullsized":"2024-04-24T22:59:57.108603+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107901+0000","last_change":"2024-04-24T22:59:57.107901+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107901+0000","last_undegraded":"2024-04-24T22:59:57.107901+0000","last_fullsized":"2024-04-24T22:59:57.107901+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.112595+0000","last_change":"2024-04-24T22:59:57.112595+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.112595+0000","last_undegraded":"2024-04-24T22:59:57.112595+0000","last_fullsized":"2024-04-24T22:59:57.112595+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.108208+0000","last_change":"2024-04-24T22:59:57.108208+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.108208+0000","last_undegraded":"2024-04-24T22:59:57.108208+0000","last_fullsized":"2024-04-24T22:59:57.108208+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.109478+0000","last_change":"2024-04-24T22:59:57.109478+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.109478+0000","last_undegraded":"2024-04-24T22:59:57.109478+0000","last_fullsized":"2024-04-24T22:59:57.109478+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.109077+0000","last_change":"2024-04-24T22:59:57.109077+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.109077+0000","last_undegraded":"2024-04-24T22:59:57.109077+0000","last_fullsized":"2024-04-24T22:59:57.109077+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.110299+0000","last_change":"2024-04-24T22:59:57.110299+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.110299+0000","last_undegraded":"2024-04-24T22:59:57.110299+0000","last_fullsized":"2024-04-24T22:59:57.110299+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.108010+0000","last_change":"2024-04-24T22:59:57.108010+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.108010+0000","last_undegraded":"2024-04-24T22:59:57.108010+0000","last_fullsized":"2024-04-24T22:59:57.108010+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.115008+0000","last_change":"2024-04-24T22:59:57.115008+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.115008+0000","last_undegraded":"2024-04-24T22:59:57.115008+0000","last_fullsized":"2024-04-24T22:59:57.115008+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.109250+0000","last_change":"2024-04-24T22:59:57.109250+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.109250+0000","last_undegraded":"2024-04-24T22:59:57.109250+0000","last_fullsized":"2024-04-24T22:59:57.109250+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107645+0000","last_change":"2024-04-24T22:59:57.107645+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107645+0000","last_undegraded":"2024-04-24T22:59:57.107645+0000","last_fullsized":"2024-04-24T22:59:57.107645+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.109256+0000","last_change":"2024-04-24T22:59:57.109256+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.109256+0000","last_undegraded":"2024-04-24T22:59:57.109256+0000","last_fullsized":"2024-04-24T22:59:57.109256+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.111959+0000","last_change":"2024-04-24T22:59:57.111959+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.111959+0000","last_undegraded":"2024-04-24T22:59:57.111959+0000","last_fullsized":"2024-04-24T22:59:57.111959+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.111012+0000","last_change":"2024-04-24T22:59:57.111012+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.111012+0000","last_undegraded":"2024-04-24T22:59:57.111012+0000","last_fullsized":"2024-04-24T22:59:57.111012+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.112179+0000","last_change":"2024-04-24T22:59:57.112179+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.112179+0000","last_undegraded":"2024-04-24T22:59:57.112179+0000","last_fullsized":"2024-04-24T22:59:57.112179+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.109896+0000","last_change":"2024-04-24T22:59:57.109896+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.109896+0000","last_undegraded":"2024-04-24T22:59:57.109896+0000","last_fullsized":"2024-04-24T22:59:57.109896+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.111728+0000","last_change":"2024-04-24T22:59:57.111728+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.111728+0000","last_undegraded":"2024-04-24T22:59:57.111728+0000","last_fullsized":"2024-04-24T22:59:57.111728+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.109489+0000","last_change":"2024-04-24T22:59:57.109489+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.109489+0000","last_undegraded":"2024-04-24T22:59:57.109489+0000","last_fullsized":"2024-04-24T22:59:57.109489+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.108661+0000","last_change":"2024-04-24T22:59:57.108661+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.108661+0000","last_undegraded":"2024-04-24T22:59:57.108661+0000","last_fullsized":"2024-04-24T22:59:57.108661+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.109342+0000","last_change":"2024-04-24T22:59:57.109342+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.109342+0000","last_undegraded":"2024-04-24T22:59:57.109342+0000","last_fullsized":"2024-04-24T22:59:57.109342+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.108792+0000","last_change":"2024-04-24T22:59:57.108792+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.108792+0000","last_undegraded":"2024-04-24T22:59:57.108792+0000","last_fullsized":"2024-04-24T22:59:57.108792+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.108711+0000","last_change":"2024-04-24T22:59:57.108711+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.108711+0000","last_undegraded":"2024-04-24T22:59:57.108711+0000","last_fullsized":"2024-04-24T22:59:57.108711+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107354+0000","last_change":"2024-04-24T22:59:57.107354+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107354+0000","last_undegraded":"2024-04-24T22:59:57.107354+0000","last_fullsized":"2024-04-24T22:59:57.107354+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.114694+0000","last_change":"2024-04-24T22:59:57.114694+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.114694+0000","last_undegraded":"2024-04-24T22:59:57.114694+0000","last_fullsized":"2024-04-24T22:59:57.114694+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.110685+0000","last_change":"2024-04-24T22:59:57.110685+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.110685+0000","last_undegraded":"2024-04-24T22:59:57.110685+0000","last_fullsized":"2024-04-24T22:59:57.110685+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107448+0000","last_change":"2024-04-24T22:59:57.107448+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107448+0000","last_undegraded":"2024-04-24T22:59:57.107448+0000","last_fullsized":"2024-04-24T22:59:57.107448+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.109942+0000","last_change":"2024-04-24T22:59:57.109942+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.109942+0000","last_undegraded":"2024-04-24T22:59:57.109942+0000","last_fullsized":"2024-04-24T22:59:57.109942+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.108904+0000","last_change":"2024-04-24T22:59:57.108904+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.108904+0000","last_undegraded":"2024-04-24T22:59:57.108904+0000","last_fullsized":"2024-04-24T22:59:57.108904+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.110722+0000","last_change":"2024-04-24T22:59:57.110722+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.110722+0000","last_undegraded":"2024-04-24T22:59:57.110722+0000","last_fullsized":"2024-04-24T22:59:57.110722+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.108284+0000","last_change":"2024-04-24T22:59:57.108284+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.108284+0000","last_undegraded":"2024-04-24T22:59:57.108284+0000","last_fullsized":"2024-04-24T22:59:57.108284+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.109574+0000","last_change":"2024-04-24T22:59:57.109574+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.109574+0000","last_undegraded":"2024-04-24T22:59:57.109574+0000","last_fullsized":"2024-04-24T22:59:57.109574+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.108880+0000","last_change":"2024-04-24T22:59:57.108880+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.108880+0000","last_undegraded":"2024-04-24T22:59:57.108880+0000","last_fullsized":"2024-04-24T22:59:57.108880+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.115474+0000","last_change":"2024-04-24T22:59:57.114774+0000","last_active":"2024-04-24T22:59:57.115474+0000","last_peered":"2024-04-24T22:59:57.115474+0000","last_clean":"2024-04-24T22:59:57.115474+0000","last_became_active":"2024-04-24T22:59:57.114031+0000","last_became_peered":"2024-04-24T22:59:57.114031+0000","last_unstale":"2024-04-24T22:59:57.115474+0000","last_undegraded":"2024-04-24T22:59:57.115474+0000","last_fullsized":"2024-04-24T22:59:57.115474+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:02:17.915932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.108182+0000","last_change":"2024-04-24T22:59:57.108182+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.108182+0000","last_undegraded":"2024-04-24T22:59:57.108182+0000","last_fullsized":"2024-04-24T22:59:57.108182+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.118962+0000","last_change":"2024-04-24T22:59:57.118830+0000","last_active":"2024-04-24T22:59:57.118962+0000","last_peered":"2024-04-24T22:59:57.118962+0000","last_clean":"2024-04-24T22:59:57.118962+0000","last_became_active":"2024-04-24T22:59:57.118380+0000","last_became_peered":"2024-04-24T22:59:57.118380+0000","last_unstale":"2024-04-24T22:59:57.118962+0000","last_undegraded":"2024-04-24T22:59:57.118962+0000","last_fullsized":"2024-04-24T22:59:57.118962+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:34:34.824377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107978+0000","last_change":"2024-04-24T22:59:57.107978+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107978+0000","last_undegraded":"2024-04-24T22:59:57.107978+0000","last_fullsized":"2024-04-24T22:59:57.107978+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.113528+0000","last_change":"2024-04-24T22:59:57.113067+0000","last_active":"2024-04-24T22:59:57.113528+0000","last_peered":"2024-04-24T22:59:57.113528+0000","last_clean":"2024-04-24T22:59:57.113528+0000","last_became_active":"2024-04-24T22:59:57.112774+0000","last_became_peered":"2024-04-24T22:59:57.112774+0000","last_unstale":"2024-04-24T22:59:57.113528+0000","last_undegraded":"2024-04-24T22:59:57.113528+0000","last_fullsized":"2024-04-24T22:59:57.113528+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:09.897525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.114355+0000","last_change":"2024-04-24T22:59:57.114355+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.114355+0000","last_undegraded":"2024-04-24T22:59:57.114355+0000","last_fullsized":"2024-04-24T22:59:57.114355+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.118693+0000","last_change":"2024-04-24T22:59:57.118605+0000","last_active":"2024-04-24T22:59:57.118693+0000","last_peered":"2024-04-24T22:59:57.118693+0000","last_clean":"2024-04-24T22:59:57.118693+0000","last_became_active":"2024-04-24T22:59:57.118271+0000","last_became_peered":"2024-04-24T22:59:57.118271+0000","last_unstale":"2024-04-24T22:59:57.118693+0000","last_undegraded":"2024-04-24T22:59:57.118693+0000","last_fullsized":"2024-04-24T22:59:57.118693+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:04:12.275266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107238+0000","last_change":"2024-04-24T22:59:57.107238+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107238+0000","last_undegraded":"2024-04-24T22:59:57.107238+0000","last_fullsized":"2024-04-24T22:59:57.107238+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.109905+0000","last_change":"2024-04-24T22:59:57.109569+0000","last_active":"2024-04-24T22:59:57.109905+0000","last_peered":"2024-04-24T22:59:57.109905+0000","last_clean":"2024-04-24T22:59:57.109905+0000","last_became_active":"2024-04-24T22:59:57.109350+0000","last_became_peered":"2024-04-24T22:59:57.109350+0000","last_unstale":"2024-04-24T22:59:57.109905+0000","last_undegraded":"2024-04-24T22:59:57.109905+0000","last_fullsized":"2024-04-24T22:59:57.109905+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:54.689993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107517+0000","last_change":"2024-04-24T22:59:57.107517+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107517+0000","last_undegraded":"2024-04-24T22:59:57.107517+0000","last_fullsized":"2024-04-24T22:59:57.107517+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.113519+0000","last_change":"2024-04-24T22:59:57.113397+0000","last_active":"2024-04-24T22:59:57.113519+0000","last_peered":"2024-04-24T22:59:57.113519+0000","last_clean":"2024-04-24T22:59:57.113519+0000","last_became_active":"2024-04-24T22:59:57.113167+0000","last_became_peered":"2024-04-24T22:59:57.113167+0000","last_unstale":"2024-04-24T22:59:57.113519+0000","last_undegraded":"2024-04-24T22:59:57.113519+0000","last_fullsized":"2024-04-24T22:59:57.113519+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:29:04.934394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107218+0000","last_change":"2024-04-24T22:59:57.107218+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107218+0000","last_undegraded":"2024-04-24T22:59:57.107218+0000","last_fullsized":"2024-04-24T22:59:57.107218+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.116661+0000","last_change":"2024-04-24T22:59:57.116514+0000","last_active":"2024-04-24T22:59:57.116661+0000","last_peered":"2024-04-24T22:59:57.116661+0000","last_clean":"2024-04-24T22:59:57.116661+0000","last_became_active":"2024-04-24T22:59:57.116259+0000","last_became_peered":"2024-04-24T22:59:57.116259+0000","last_unstale":"2024-04-24T22:59:57.116661+0000","last_undegraded":"2024-04-24T22:59:57.116661+0000","last_fullsized":"2024-04-24T22:59:57.116661+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:43:58.559759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.112898+0000","last_change":"2024-04-24T22:59:57.112898+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.112898+0000","last_undegraded":"2024-04-24T22:59:57.112898+0000","last_fullsized":"2024-04-24T22:59:57.112898+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.117357+0000","last_change":"2024-04-24T22:59:57.117209+0000","last_active":"2024-04-24T22:59:57.117357+0000","last_peered":"2024-04-24T22:59:57.117357+0000","last_clean":"2024-04-24T22:59:57.117357+0000","last_became_active":"2024-04-24T22:59:57.116968+0000","last_became_peered":"2024-04-24T22:59:57.116968+0000","last_unstale":"2024-04-24T22:59:57.117357+0000","last_undegraded":"2024-04-24T22:59:57.117357+0000","last_fullsized":"2024-04-24T22:59:57.117357+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:44:17.871555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.109086+0000","last_change":"2024-04-24T22:59:57.109086+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.109086+0000","last_undegraded":"2024-04-24T22:59:57.109086+0000","last_fullsized":"2024-04-24T22:59:57.109086+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.106993+0000","last_change":"2024-04-24T22:59:57.106612+0000","last_active":"2024-04-24T22:59:57.106993+0000","last_peered":"2024-04-24T22:59:57.106993+0000","last_clean":"2024-04-24T22:59:57.106993+0000","last_became_active":"2024-04-24T22:59:57.106350+0000","last_became_peered":"2024-04-24T22:59:57.106350+0000","last_unstale":"2024-04-24T22:59:57.106993+0000","last_undegraded":"2024-04-24T22:59:57.106993+0000","last_fullsized":"2024-04-24T22:59:57.106993+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:45:14.576793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.112099+0000","last_change":"2024-04-24T22:59:57.112099+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.112099+0000","last_undegraded":"2024-04-24T22:59:57.112099+0000","last_fullsized":"2024-04-24T22:59:57.112099+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.115220+0000","last_change":"2024-04-24T22:59:57.115098+0000","last_active":"2024-04-24T22:59:57.115220+0000","last_peered":"2024-04-24T22:59:57.115220+0000","last_clean":"2024-04-24T22:59:57.115220+0000","last_became_active":"2024-04-24T22:59:57.114866+0000","last_became_peered":"2024-04-24T22:59:57.114866+0000","last_unstale":"2024-04-24T22:59:57.115220+0000","last_undegraded":"2024-04-24T22:59:57.115220+0000","last_fullsized":"2024-04-24T22:59:57.115220+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:27:09.165254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.111004+0000","last_change":"2024-04-24T22:59:57.111004+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.111004+0000","last_undegraded":"2024-04-24T22:59:57.111004+0000","last_fullsized":"2024-04-24T22:59:57.111004+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.115493+0000","last_change":"2024-04-24T22:59:57.115370+0000","last_active":"2024-04-24T22:59:57.115493+0000","last_peered":"2024-04-24T22:59:57.115493+0000","last_clean":"2024-04-24T22:59:57.115493+0000","last_became_active":"2024-04-24T22:59:57.115147+0000","last_became_peered":"2024-04-24T22:59:57.115147+0000","last_unstale":"2024-04-24T22:59:57.115493+0000","last_undegraded":"2024-04-24T22:59:57.115493+0000","last_fullsized":"2024-04-24T22:59:57.115493+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:23:03.727015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.108512+0000","last_change":"2024-04-24T22:59:57.108512+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.108512+0000","last_undegraded":"2024-04-24T22:59:57.108512+0000","last_fullsized":"2024-04-24T22:59:57.108512+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.118330+0000","last_change":"2024-04-24T22:59:57.118202+0000","last_active":"2024-04-24T22:59:57.118330+0000","last_peered":"2024-04-24T22:59:57.118330+0000","last_clean":"2024-04-24T22:59:57.118330+0000","last_became_active":"2024-04-24T22:59:57.117946+0000","last_became_peered":"2024-04-24T22:59:57.117946+0000","last_unstale":"2024-04-24T22:59:57.118330+0000","last_undegraded":"2024-04-24T22:59:57.118330+0000","last_fullsized":"2024-04-24T22:59:57.118330+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:37:52.953926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107500+0000","last_change":"2024-04-24T22:59:57.107500+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107500+0000","last_undegraded":"2024-04-24T22:59:57.107500+0000","last_fullsized":"2024-04-24T22:59:57.107500+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.112009+0000","last_change":"2024-04-24T22:59:57.111899+0000","last_active":"2024-04-24T22:59:57.112009+0000","last_peered":"2024-04-24T22:59:57.112009+0000","last_clean":"2024-04-24T22:59:57.112009+0000","last_became_active":"2024-04-24T22:59:57.111632+0000","last_became_peered":"2024-04-24T22:59:57.111632+0000","last_unstale":"2024-04-24T22:59:57.112009+0000","last_undegraded":"2024-04-24T22:59:57.112009+0000","last_fullsized":"2024-04-24T22:59:57.112009+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:01.204587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.114010+0000","last_change":"2024-04-24T22:59:57.114010+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.114010+0000","last_undegraded":"2024-04-24T22:59:57.114010+0000","last_fullsized":"2024-04-24T22:59:57.114010+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.116958+0000","last_change":"2024-04-24T22:59:57.116842+0000","last_active":"2024-04-24T22:59:57.116958+0000","last_peered":"2024-04-24T22:59:57.116958+0000","last_clean":"2024-04-24T22:59:57.116958+0000","last_became_active":"2024-04-24T22:59:57.116577+0000","last_became_peered":"2024-04-24T22:59:57.116577+0000","last_unstale":"2024-04-24T22:59:57.116958+0000","last_undegraded":"2024-04-24T22:59:57.116958+0000","last_fullsized":"2024-04-24T22:59:57.116958+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:29.595256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107852+0000","last_change":"2024-04-24T22:59:57.107852+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107852+0000","last_undegraded":"2024-04-24T22:59:57.107852+0000","last_fullsized":"2024-04-24T22:59:57.107852+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.115153+0000","last_change":"2024-04-24T22:59:57.113832+0000","last_active":"2024-04-24T22:59:57.115153+0000","last_peered":"2024-04-24T22:59:57.115153+0000","last_clean":"2024-04-24T22:59:57.115153+0000","last_became_active":"2024-04-24T22:59:57.113610+0000","last_became_peered":"2024-04-24T22:59:57.113610+0000","last_unstale":"2024-04-24T22:59:57.115153+0000","last_undegraded":"2024-04-24T22:59:57.115153+0000","last_fullsized":"2024-04-24T22:59:57.115153+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:49:50.026489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.108615+0000","last_change":"2024-04-24T22:59:57.108615+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.108615+0000","last_undegraded":"2024-04-24T22:59:57.108615+0000","last_fullsized":"2024-04-24T22:59:57.108615+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.116358+0000","last_change":"2024-04-24T22:59:57.116182+0000","last_active":"2024-04-24T22:59:57.116358+0000","last_peered":"2024-04-24T22:59:57.116358+0000","last_clean":"2024-04-24T22:59:57.116358+0000","last_became_active":"2024-04-24T22:59:57.115952+0000","last_became_peered":"2024-04-24T22:59:57.115952+0000","last_unstale":"2024-04-24T22:59:57.116358+0000","last_undegraded":"2024-04-24T22:59:57.116358+0000","last_fullsized":"2024-04-24T22:59:57.116358+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:20:34.273911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107248+0000","last_change":"2024-04-24T22:59:57.107248+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107248+0000","last_undegraded":"2024-04-24T22:59:57.107248+0000","last_fullsized":"2024-04-24T22:59:57.107248+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.119286+0000","last_change":"2024-04-24T22:59:57.119183+0000","last_active":"2024-04-24T22:59:57.119286+0000","last_peered":"2024-04-24T22:59:57.119286+0000","last_clean":"2024-04-24T22:59:57.119286+0000","last_became_active":"2024-04-24T22:59:57.118911+0000","last_became_peered":"2024-04-24T22:59:57.118911+0000","last_unstale":"2024-04-24T22:59:57.119286+0000","last_undegraded":"2024-04-24T22:59:57.119286+0000","last_fullsized":"2024-04-24T22:59:57.119286+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:28:06.461401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.106893+0000","last_change":"2024-04-24T22:59:57.106893+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.106893+0000","last_undegraded":"2024-04-24T22:59:57.106893+0000","last_fullsized":"2024-04-24T22:59:57.106893+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.115503+0000","last_change":"2024-04-24T22:59:57.115384+0000","last_active":"2024-04-24T22:59:57.115503+0000","last_peered":"2024-04-24T22:59:57.115503+0000","last_clean":"2024-04-24T22:59:57.115503+0000","last_became_active":"2024-04-24T22:59:57.115129+0000","last_became_peered":"2024-04-24T22:59:57.115129+0000","last_unstale":"2024-04-24T22:59:57.115503+0000","last_undegraded":"2024-04-24T22:59:57.115503+0000","last_fullsized":"2024-04-24T22:59:57.115503+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:13:57.006308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107117+0000","last_change":"2024-04-24T22:59:57.107117+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107117+0000","last_undegraded":"2024-04-24T22:59:57.107117+0000","last_fullsized":"2024-04-24T22:59:57.107117+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.114295+0000","last_change":"2024-04-24T22:59:57.113034+0000","last_active":"2024-04-24T22:59:57.114295+0000","last_peered":"2024-04-24T22:59:57.114295+0000","last_clean":"2024-04-24T22:59:57.114295+0000","last_became_active":"2024-04-24T22:59:57.111752+0000","last_became_peered":"2024-04-24T22:59:57.111752+0000","last_unstale":"2024-04-24T22:59:57.114295+0000","last_undegraded":"2024-04-24T22:59:57.114295+0000","last_fullsized":"2024-04-24T22:59:57.114295+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:11:29.081634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.109090+0000","last_change":"2024-04-24T22:59:57.109090+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.109090+0000","last_undegraded":"2024-04-24T22:59:57.109090+0000","last_fullsized":"2024-04-24T22:59:57.109090+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.117410+0000","last_change":"2024-04-24T22:59:57.117253+0000","last_active":"2024-04-24T22:59:57.117410+0000","last_peered":"2024-04-24T22:59:57.117410+0000","last_clean":"2024-04-24T22:59:57.117410+0000","last_became_active":"2024-04-24T22:59:57.116974+0000","last_became_peered":"2024-04-24T22:59:57.116974+0000","last_unstale":"2024-04-24T22:59:57.117410+0000","last_undegraded":"2024-04-24T22:59:57.117410+0000","last_fullsized":"2024-04-24T22:59:57.117410+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:24.934926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107613+0000","last_change":"2024-04-24T22:59:57.107613+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107613+0000","last_undegraded":"2024-04-24T22:59:57.107613+0000","last_fullsized":"2024-04-24T22:59:57.107613+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.112446+0000","last_change":"2024-04-24T22:59:57.111772+0000","last_active":"2024-04-24T22:59:57.112446+0000","last_peered":"2024-04-24T22:59:57.112446+0000","last_clean":"2024-04-24T22:59:57.112446+0000","last_became_active":"2024-04-24T22:59:57.111452+0000","last_became_peered":"2024-04-24T22:59:57.111452+0000","last_unstale":"2024-04-24T22:59:57.112446+0000","last_undegraded":"2024-04-24T22:59:57.112446+0000","last_fullsized":"2024-04-24T22:59:57.112446+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:27.552548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107006+0000","last_change":"2024-04-24T22:59:57.107006+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107006+0000","last_undegraded":"2024-04-24T22:59:57.107006+0000","last_fullsized":"2024-04-24T22:59:57.107006+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.111880+0000","last_change":"2024-04-24T22:59:57.111232+0000","last_active":"2024-04-24T22:59:57.111880+0000","last_peered":"2024-04-24T22:59:57.111880+0000","last_clean":"2024-04-24T22:59:57.111880+0000","last_became_active":"2024-04-24T22:59:57.111010+0000","last_became_peered":"2024-04-24T22:59:57.111010+0000","last_unstale":"2024-04-24T22:59:57.111880+0000","last_undegraded":"2024-04-24T22:59:57.111880+0000","last_fullsized":"2024-04-24T22:59:57.111880+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:02:29.250634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107089+0000","last_change":"2024-04-24T22:59:57.107089+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107089+0000","last_undegraded":"2024-04-24T22:59:57.107089+0000","last_fullsized":"2024-04-24T22:59:57.107089+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.111016+0000","last_change":"2024-04-24T22:59:57.110898+0000","last_active":"2024-04-24T22:59:57.111016+0000","last_peered":"2024-04-24T22:59:57.111016+0000","last_clean":"2024-04-24T22:59:57.111016+0000","last_became_active":"2024-04-24T22:59:57.110665+0000","last_became_peered":"2024-04-24T22:59:57.110665+0000","last_unstale":"2024-04-24T22:59:57.111016+0000","last_undegraded":"2024-04-24T22:59:57.111016+0000","last_fullsized":"2024-04-24T22:59:57.111016+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:55.518131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.106526+0000","last_change":"2024-04-24T22:59:57.106526+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.106526+0000","last_undegraded":"2024-04-24T22:59:57.106526+0000","last_fullsized":"2024-04-24T22:59:57.106526+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.114739+0000","last_change":"2024-04-24T22:59:57.114417+0000","last_active":"2024-04-24T22:59:57.114739+0000","last_peered":"2024-04-24T22:59:57.114739+0000","last_clean":"2024-04-24T22:59:57.114739+0000","last_became_active":"2024-04-24T22:59:57.112771+0000","last_became_peered":"2024-04-24T22:59:57.112771+0000","last_unstale":"2024-04-24T22:59:57.114739+0000","last_undegraded":"2024-04-24T22:59:57.114739+0000","last_fullsized":"2024-04-24T22:59:57.114739+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:11:37.810064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.110252+0000","last_change":"2024-04-24T22:59:57.110252+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.110252+0000","last_undegraded":"2024-04-24T22:59:57.110252+0000","last_fullsized":"2024-04-24T22:59:57.110252+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.115885+0000","last_change":"2024-04-24T22:59:57.115765+0000","last_active":"2024-04-24T22:59:57.115885+0000","last_peered":"2024-04-24T22:59:57.115885+0000","last_clean":"2024-04-24T22:59:57.115885+0000","last_became_active":"2024-04-24T22:59:57.115056+0000","last_became_peered":"2024-04-24T22:59:57.115056+0000","last_unstale":"2024-04-24T22:59:57.115885+0000","last_undegraded":"2024-04-24T22:59:57.115885+0000","last_fullsized":"2024-04-24T22:59:57.115885+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:18:11.299169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.106865+0000","last_change":"2024-04-24T22:59:57.106865+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.106865+0000","last_undegraded":"2024-04-24T22:59:57.106865+0000","last_fullsized":"2024-04-24T22:59:57.106865+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.113678+0000","last_change":"2024-04-24T22:59:57.113564+0000","last_active":"2024-04-24T22:59:57.113678+0000","last_peered":"2024-04-24T22:59:57.113678+0000","last_clean":"2024-04-24T22:59:57.113678+0000","last_became_active":"2024-04-24T22:59:57.113292+0000","last_became_peered":"2024-04-24T22:59:57.113292+0000","last_unstale":"2024-04-24T22:59:57.113678+0000","last_undegraded":"2024-04-24T22:59:57.113678+0000","last_fullsized":"2024-04-24T22:59:57.113678+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:05:41.614373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.109161+0000","last_change":"2024-04-24T22:59:57.109161+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.109161+0000","last_undegraded":"2024-04-24T22:59:57.109161+0000","last_fullsized":"2024-04-24T22:59:57.109161+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.112241+0000","last_change":"2024-04-24T22:59:57.111930+0000","last_active":"2024-04-24T22:59:57.112241+0000","last_peered":"2024-04-24T22:59:57.112241+0000","last_clean":"2024-04-24T22:59:57.112241+0000","last_became_active":"2024-04-24T22:59:57.111632+0000","last_became_peered":"2024-04-24T22:59:57.111632+0000","last_unstale":"2024-04-24T22:59:57.112241+0000","last_undegraded":"2024-04-24T22:59:57.112241+0000","last_fullsized":"2024-04-24T22:59:57.112241+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:57.770833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.110730+0000","last_change":"2024-04-24T22:59:57.110730+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.110730+0000","last_undegraded":"2024-04-24T22:59:57.110730+0000","last_fullsized":"2024-04-24T22:59:57.110730+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.115239+0000","last_change":"2024-04-24T22:59:57.115123+0000","last_active":"2024-04-24T22:59:57.115239+0000","last_peered":"2024-04-24T22:59:57.115239+0000","last_clean":"2024-04-24T22:59:57.115239+0000","last_became_active":"2024-04-24T22:59:57.114845+0000","last_became_peered":"2024-04-24T22:59:57.114845+0000","last_unstale":"2024-04-24T22:59:57.115239+0000","last_undegraded":"2024-04-24T22:59:57.115239+0000","last_fullsized":"2024-04-24T22:59:57.115239+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:31.884038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.111780+0000","last_change":"2024-04-24T22:59:57.111780+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.111780+0000","last_undegraded":"2024-04-24T22:59:57.111780+0000","last_fullsized":"2024-04-24T22:59:57.111780+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.112631+0000","last_change":"2024-04-24T22:59:57.112511+0000","last_active":"2024-04-24T22:59:57.112631+0000","last_peered":"2024-04-24T22:59:57.112631+0000","last_clean":"2024-04-24T22:59:57.112631+0000","last_became_active":"2024-04-24T22:59:57.112210+0000","last_became_peered":"2024-04-24T22:59:57.112210+0000","last_unstale":"2024-04-24T22:59:57.112631+0000","last_undegraded":"2024-04-24T22:59:57.112631+0000","last_fullsized":"2024-04-24T22:59:57.112631+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:04.204693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.112288+0000","last_change":"2024-04-24T22:59:57.112288+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.112288+0000","last_undegraded":"2024-04-24T22:59:57.112288+0000","last_fullsized":"2024-04-24T22:59:57.112288+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.111354+0000","last_change":"2024-04-24T22:59:57.111232+0000","last_active":"2024-04-24T22:59:57.111354+0000","last_peered":"2024-04-24T22:59:57.111354+0000","last_clean":"2024-04-24T22:59:57.111354+0000","last_became_active":"2024-04-24T22:59:57.110997+0000","last_became_peered":"2024-04-24T22:59:57.110997+0000","last_unstale":"2024-04-24T22:59:57.111354+0000","last_undegraded":"2024-04-24T22:59:57.111354+0000","last_fullsized":"2024-04-24T22:59:57.111354+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:56:07.604519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107025+0000","last_change":"2024-04-24T22:59:57.107025+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107025+0000","last_undegraded":"2024-04-24T22:59:57.107025+0000","last_fullsized":"2024-04-24T22:59:57.107025+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.118210+0000","last_change":"2024-04-24T22:59:57.118085+0000","last_active":"2024-04-24T22:59:57.118210+0000","last_peered":"2024-04-24T22:59:57.118210+0000","last_clean":"2024-04-24T22:59:57.118210+0000","last_became_active":"2024-04-24T22:59:57.116287+0000","last_became_peered":"2024-04-24T22:59:57.116287+0000","last_unstale":"2024-04-24T22:59:57.118210+0000","last_undegraded":"2024-04-24T22:59:57.118210+0000","last_fullsized":"2024-04-24T22:59:57.118210+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:11:38.330427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.110417+0000","last_change":"2024-04-24T22:59:57.110417+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.110417+0000","last_undegraded":"2024-04-24T22:59:57.110417+0000","last_fullsized":"2024-04-24T22:59:57.110417+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.112334+0000","last_change":"2024-04-24T22:59:57.110803+0000","last_active":"2024-04-24T22:59:57.112334+0000","last_peered":"2024-04-24T22:59:57.112334+0000","last_clean":"2024-04-24T22:59:57.112334+0000","last_became_active":"2024-04-24T22:59:57.110572+0000","last_became_peered":"2024-04-24T22:59:57.110572+0000","last_unstale":"2024-04-24T22:59:57.112334+0000","last_undegraded":"2024-04-24T22:59:57.112334+0000","last_fullsized":"2024-04-24T22:59:57.112334+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:06:59.730529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":186,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.105292+0000","last_change":"2024-04-24T22:58:55.138810+0000","last_active":"2024-04-24T22:59:57.105292+0000","last_peered":"2024-04-24T22:59:57.105292+0000","last_clean":"2024-04-24T22:59:57.105292+0000","last_became_active":"2024-04-24T22:58:55.138627+0000","last_became_peered":"2024-04-24T22:58:55.138627+0000","last_unstale":"2024-04-24T22:59:57.105292+0000","last_undegraded":"2024-04-24T22:59:57.105292+0000","last_fullsized":"2024-04-24T22:59:57.105292+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_clean_scrub_stamp":"2024-04-24T22:58:54.127235+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:24.010002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":116,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195196,"statfs":{"total":200000000,"available":199880705,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":116,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195196,"statfs":{"total":200000000,"available":199880705,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":693,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194619,"statfs":{"total":200000000,"available":199290337,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":116,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195196,"statfs":{"total":200000000,"available":199880705,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":116,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195196,"statfs":{"total":200000000,"available":199880705,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":116,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195196,"statfs":{"total":200000000,"available":199880705,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":687,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194625,"statfs":{"total":200000000,"available":199296154,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":116,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195196,"statfs":{"total":200000000,"available":199880705,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-24T23:00:01.467 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:00:01.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.603+0000 7fed3e012640 1 Processor -- start 2024-04-24T23:00:01.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.603+0000 7fed3e012640 1 -- start start 2024-04-24T23:00:01.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.604+0000 7fed3e012640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fed3806b820 0x7fed380d40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:01.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.604+0000 7fed3e012640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed380d3010 con 0x7fed3815e9e0 2024-04-24T23:00:01.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.604+0000 7fed3e012640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed380d32f0 con 0x7fed380ba390 2024-04-24T23:00:01.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.604+0000 7fed3e012640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fed380d2db0 con 0x7fed3806b820 2024-04-24T23:00:01.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.604+0000 7fed377fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fed3815e9e0 0x7fed380d26a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39238/0 (socket says 172.21.15.32:39238) 2024-04-24T23:00:01.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.604+0000 7fed377fe640 1 -- 172.21.15.32:0/4153761244 learned_addr learned my addr 172.21.15.32:0/4153761244 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:00:01.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.605+0000 7fed37fff640 1 --2- 172.21.15.32:0/4153761244 >> v2:172.21.15.121:3300/0 conn(0x7fed3806b820 0x7fed380d40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:01.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.605+0000 7fed367fc640 1 -- 172.21.15.32:0/4153761244 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 504727154 0 0) 0x7fed380d3010 con 0x7fed3815e9e0 2024-04-24T23:00:01.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.605+0000 7fed367fc640 1 -- 172.21.15.32:0/4153761244 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed24003520 con 0x7fed3815e9e0 2024-04-24T23:00:01.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.605+0000 7fed367fc640 1 -- 172.21.15.32:0/4153761244 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 799613898 0 0) 0x7fed24003520 con 0x7fed3815e9e0 2024-04-24T23:00:01.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.605+0000 7fed367fc640 1 -- 172.21.15.32:0/4153761244 >> v2:172.21.15.121:3300/0 conn(0x7fed3806b820 msgr2=0x7fed380d40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:01.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.605+0000 7fed367fc640 1 --2- 172.21.15.32:0/4153761244 >> v2:172.21.15.121:3300/0 conn(0x7fed3806b820 0x7fed380d40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:01.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.605+0000 7fed367fc640 1 -- 172.21.15.32:0/4153761244 >> v1:172.21.15.121:6789/0 conn(0x7fed380ba390 legacy=0x7fed3806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:01.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.605+0000 7fed367fc640 1 -- 172.21.15.32:0/4153761244 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed380d35d0 con 0x7fed3815e9e0 2024-04-24T23:00:01.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.605+0000 7fed367fc640 1 -- 172.21.15.32:0/4153761244 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7fed28001670 con 0x7fed3815e9e0 2024-04-24T23:00:01.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.605+0000 7fed37fff640 1 --2- 172.21.15.32:0/4153761244 >> v2:172.21.15.121:3300/0 conn(0x7fed3806b820 0x7fed380d40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:00:01.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.605+0000 7fed367fc640 1 -- 172.21.15.32:0/4153761244 >> v1:172.21.15.32:6789/0 conn(0x7fed3815e9e0 legacy=0x7fed380d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:01.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.605+0000 7fed367fc640 1 --2- 172.21.15.32:0/4153761244 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fed240039b0 0x7fed24003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:01.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.605+0000 7fed367fc640 1 --2- 172.21.15.32:0/4153761244 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed24004800 0x7fed24004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:01.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.605+0000 7fed367fc640 1 --2- 172.21.15.32:0/4153761244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed240051c0 0x7fed2400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:01.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.605+0000 7fed367fc640 1 -- 172.21.15.32:0/4153761244 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fed380d2db0 con 0x7fed240051c0 2024-04-24T23:00:01.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.605+0000 7fed367fc640 1 -- 172.21.15.32:0/4153761244 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fed2400c060 con 0x7fed24004800 2024-04-24T23:00:01.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.605+0000 7fed367fc640 1 -- 172.21.15.32:0/4153761244 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fed2400c260 con 0x7fed240039b0 2024-04-24T23:00:01.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.605+0000 7fed37fff640 1 --2- 172.21.15.32:0/4153761244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed240051c0 0x7fed2400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:01.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.605+0000 7fed36ffd640 1 --2- 172.21.15.32:0/4153761244 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed24004800 0x7fed24004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:01.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.605+0000 7fed377fe640 1 --2- 172.21.15.32:0/4153761244 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fed240039b0 0x7fed24003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:01.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.605+0000 7fed37fff640 1 -- 172.21.15.32:0/4153761244 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fed240039b0 msgr2=0x7fed24003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:01.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.605+0000 7fed37fff640 1 --2- 172.21.15.32:0/4153761244 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fed240039b0 0x7fed24003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:01.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.605+0000 7fed37fff640 1 -- 172.21.15.32:0/4153761244 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed24004800 msgr2=0x7fed24004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:01.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.605+0000 7fed37fff640 1 --2- 172.21.15.32:0/4153761244 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed24004800 0x7fed24004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:01.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.605+0000 7fed37fff640 1 -- 172.21.15.32:0/4153761244 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed24003520 con 0x7fed240051c0 2024-04-24T23:00:01.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.605+0000 7fed377fe640 1 --2- 172.21.15.32:0/4153761244 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fed240039b0 0x7fed24003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:00:01.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.606+0000 7fed37fff640 1 -- 172.21.15.32:0/4153761244 reap_dead start 2024-04-24T23:00:01.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.606+0000 7fed37fff640 1 --2- 172.21.15.32:0/4153761244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed240051c0 0x7fed2400b9c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fed38073060 tx=0x7fed24005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd1874b4acd1034 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:01.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.606+0000 7fed367fc640 1 -- 172.21.15.32:0/4153761244 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed30009de0 con 0x7fed240051c0 2024-04-24T23:00:01.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.606+0000 7fed367fc640 1 -- 172.21.15.32:0/4153761244 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed30003b90 con 0x7fed240051c0 2024-04-24T23:00:01.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.606+0000 7fed367fc640 1 -- 172.21.15.32:0/4153761244 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed30004020 con 0x7fed240051c0 2024-04-24T23:00:01.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.607+0000 7fed3e012640 1 -- 172.21.15.32:0/4153761244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed240051c0 msgr2=0x7fed2400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:01.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.607+0000 7fed3e012640 1 --2- 172.21.15.32:0/4153761244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed240051c0 0x7fed2400b9c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fed38073060 tx=0x7fed24005830 comp rx=0 tx=0).stop 2024-04-24T23:00:01.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.607+0000 7fed3e012640 1 -- 172.21.15.32:0/4153761244 shutdown_connections 2024-04-24T23:00:01.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.607+0000 7fed3e012640 1 --2- 172.21.15.32:0/4153761244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed240051c0 0x7fed2400b9c0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:01.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.607+0000 7fed3e012640 1 -- 172.21.15.32:0/4153761244 >> 172.21.15.32:0/4153761244 conn(0x7fed380d12e0 msgr2=0x7fed380d16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:01.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.607+0000 7fed3e012640 1 -- 172.21.15.32:0/4153761244 shutdown_connections 2024-04-24T23:00:01.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.607+0000 7fed3e012640 1 -- 172.21.15.32:0/4153761244 wait complete. 2024-04-24T23:00:01.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.608+0000 7fed3e012640 1 Processor -- start 2024-04-24T23:00:01.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.608+0000 7fed3e012640 1 -- start start 2024-04-24T23:00:01.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.609+0000 7fed3e012640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed240051c0 0x7fed3814fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:01.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.609+0000 7fed3e012640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed381502e0 0x7fed381519a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:01.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.609+0000 7fed3e012640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fed38151ee0 0x7fed38152370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:01.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.609+0000 7fed3e012640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fed380d40b0 con 0x7fed240051c0 2024-04-24T23:00:01.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.609+0000 7fed3e012640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fed380d3b90 con 0x7fed381502e0 2024-04-24T23:00:01.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.609+0000 7fed3e012640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fed380d4230 con 0x7fed38151ee0 2024-04-24T23:00:01.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.609+0000 7fed377fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed240051c0 0x7fed3814fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:01.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.609+0000 7fed377fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed240051c0 0x7fed3814fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60058/0 (socket says 172.21.15.32:60058) 2024-04-24T23:00:01.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.609+0000 7fed377fe640 1 -- 172.21.15.32:0/668596636 learned_addr learned my addr 172.21.15.32:0/668596636 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:01.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.609+0000 7fed37fff640 1 --2- 172.21.15.32:0/668596636 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fed38151ee0 0x7fed38152370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:01.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.609+0000 7fed36ffd640 1 --2- 172.21.15.32:0/668596636 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed381502e0 0x7fed381519a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:01.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.609+0000 7fed377fe640 1 -- 172.21.15.32:0/668596636 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fed38151ee0 msgr2=0x7fed38152370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:01.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.609+0000 7fed377fe640 1 --2- 172.21.15.32:0/668596636 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fed38151ee0 0x7fed38152370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:01.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.609+0000 7fed377fe640 1 -- 172.21.15.32:0/668596636 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed381502e0 msgr2=0x7fed381519a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:01.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.609+0000 7fed377fe640 1 --2- 172.21.15.32:0/668596636 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed381502e0 0x7fed381519a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:01.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.609+0000 7fed377fe640 1 -- 172.21.15.32:0/668596636 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed381528b0 con 0x7fed240051c0 2024-04-24T23:00:01.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.609+0000 7fed37fff640 1 --2- 172.21.15.32:0/668596636 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fed38151ee0 0x7fed38152370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:00:01.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.609+0000 7fed36ffd640 1 --2- 172.21.15.32:0/668596636 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed381502e0 0x7fed381519a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:00:01.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.610+0000 7fed377fe640 1 --2- 172.21.15.32:0/668596636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed240051c0 0x7fed3814fda0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fed2800e700 tx=0x7fed2800ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:01.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.610+0000 7fed2ffff640 1 -- 172.21.15.32:0/668596636 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed2800c660 con 0x7fed240051c0 2024-04-24T23:00:01.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.610+0000 7fed2ffff640 1 -- 172.21.15.32:0/668596636 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed28018070 con 0x7fed240051c0 2024-04-24T23:00:01.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.610+0000 7fed3e012640 1 -- 172.21.15.32:0/668596636 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed380cca30 con 0x7fed240051c0 2024-04-24T23:00:01.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.610+0000 7fed2ffff640 1 -- 172.21.15.32:0/668596636 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed2800ca00 con 0x7fed240051c0 2024-04-24T23:00:01.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.610+0000 7fed3e012640 1 -- 172.21.15.32:0/668596636 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed380ccfd0 con 0x7fed240051c0 2024-04-24T23:00:01.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.611+0000 7fed3e012640 1 -- 172.21.15.32:0/668596636 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed38068430 con 0x7fed240051c0 2024-04-24T23:00:01.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.611+0000 7fed2ffff640 1 -- 172.21.15.32:0/668596636 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fed2800cba0 con 0x7fed240051c0 2024-04-24T23:00:01.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.612+0000 7fed2ffff640 1 --2- 172.21.15.32:0/668596636 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fed1806be30 0x7fed1806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:01.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.612+0000 7fed2ffff640 1 -- 172.21.15.32:0/668596636 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fed2808abc0 con 0x7fed240051c0 2024-04-24T23:00:01.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.612+0000 7fed36ffd640 1 --2- 172.21.15.32:0/668596636 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fed1806be30 0x7fed1806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:01.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.612+0000 7fed36ffd640 1 --2- 172.21.15.32:0/668596636 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fed1806be30 0x7fed1806e2f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fed38151790 tx=0x7fed20009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:01.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.616+0000 7fed2ffff640 1 -- 172.21.15.32:0/668596636 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fed28059600 con 0x7fed240051c0 2024-04-24T23:00:01.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.795+0000 7fed3e012640 1 -- 172.21.15.32:0/668596636 --> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fed38156080 con 0x7fed1806be30 2024-04-24T23:00:01.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.801+0000 7fed2ffff640 1 -- 172.21.15.32:0/668596636 <== mgr.4108 v2:172.21.15.121:6832/1848157943 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+242215 (secure 0 0 0) 0x7fed38156080 con 0x7fed1806be30 2024-04-24T23:00:01.805 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:00:01.806 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-24T23:00:01.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.804+0000 7fed3e012640 1 -- 172.21.15.32:0/668596636 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fed1806be30 msgr2=0x7fed1806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:01.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.804+0000 7fed3e012640 1 --2- 172.21.15.32:0/668596636 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fed1806be30 0x7fed1806e2f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fed38151790 tx=0x7fed20009040 comp rx=0 tx=0).stop 2024-04-24T23:00:01.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.804+0000 7fed3e012640 1 -- 172.21.15.32:0/668596636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed240051c0 msgr2=0x7fed3814fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:01.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.804+0000 7fed3e012640 1 --2- 172.21.15.32:0/668596636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed240051c0 0x7fed3814fda0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fed2800e700 tx=0x7fed2800ebc0 comp rx=0 tx=0).stop 2024-04-24T23:00:01.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.804+0000 7fed3e012640 1 -- 172.21.15.32:0/668596636 shutdown_connections 2024-04-24T23:00:01.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.804+0000 7fed3e012640 1 --2- 172.21.15.32:0/668596636 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fed1806be30 0x7fed1806e2f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:01.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.804+0000 7fed3e012640 1 --2- 172.21.15.32:0/668596636 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fed38151ee0 0x7fed38152370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:01.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.804+0000 7fed3e012640 1 --2- 172.21.15.32:0/668596636 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed381502e0 0x7fed381519a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:01.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.804+0000 7fed3e012640 1 --2- 172.21.15.32:0/668596636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed240051c0 0x7fed3814fda0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:01.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.804+0000 7fed3e012640 1 -- 172.21.15.32:0/668596636 >> 172.21.15.32:0/668596636 conn(0x7fed380d12e0 msgr2=0x7fed3815bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:01.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.805+0000 7fed3e012640 1 -- 172.21.15.32:0/668596636 shutdown_connections 2024-04-24T23:00:01.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.805+0000 7fed3e012640 1 -- 172.21.15.32:0/668596636 wait complete. 2024-04-24T23:00:01.817 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":58,"stamp":"2024-04-24T23:00:01.140748+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2079,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560420,"statfs":{"total":1600000000,"available":1597870721,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.041342"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.108588+0000","last_change":"2024-04-24T22:59:57.108588+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.108588+0000","last_undegraded":"2024-04-24T22:59:57.108588+0000","last_fullsized":"2024-04-24T22:59:57.108588+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.108603+0000","last_change":"2024-04-24T22:59:57.108603+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.108603+0000","last_undegraded":"2024-04-24T22:59:57.108603+0000","last_fullsized":"2024-04-24T22:59:57.108603+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107901+0000","last_change":"2024-04-24T22:59:57.107901+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107901+0000","last_undegraded":"2024-04-24T22:59:57.107901+0000","last_fullsized":"2024-04-24T22:59:57.107901+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.112595+0000","last_change":"2024-04-24T22:59:57.112595+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.112595+0000","last_undegraded":"2024-04-24T22:59:57.112595+0000","last_fullsized":"2024-04-24T22:59:57.112595+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.108208+0000","last_change":"2024-04-24T22:59:57.108208+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.108208+0000","last_undegraded":"2024-04-24T22:59:57.108208+0000","last_fullsized":"2024-04-24T22:59:57.108208+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.109478+0000","last_change":"2024-04-24T22:59:57.109478+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.109478+0000","last_undegraded":"2024-04-24T22:59:57.109478+0000","last_fullsized":"2024-04-24T22:59:57.109478+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.109077+0000","last_change":"2024-04-24T22:59:57.109077+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.109077+0000","last_undegraded":"2024-04-24T22:59:57.109077+0000","last_fullsized":"2024-04-24T22:59:57.109077+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.110299+0000","last_change":"2024-04-24T22:59:57.110299+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.110299+0000","last_undegraded":"2024-04-24T22:59:57.110299+0000","last_fullsized":"2024-04-24T22:59:57.110299+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.108010+0000","last_change":"2024-04-24T22:59:57.108010+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.108010+0000","last_undegraded":"2024-04-24T22:59:57.108010+0000","last_fullsized":"2024-04-24T22:59:57.108010+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.115008+0000","last_change":"2024-04-24T22:59:57.115008+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.115008+0000","last_undegraded":"2024-04-24T22:59:57.115008+0000","last_fullsized":"2024-04-24T22:59:57.115008+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.109250+0000","last_change":"2024-04-24T22:59:57.109250+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.109250+0000","last_undegraded":"2024-04-24T22:59:57.109250+0000","last_fullsized":"2024-04-24T22:59:57.109250+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107645+0000","last_change":"2024-04-24T22:59:57.107645+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107645+0000","last_undegraded":"2024-04-24T22:59:57.107645+0000","last_fullsized":"2024-04-24T22:59:57.107645+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.109256+0000","last_change":"2024-04-24T22:59:57.109256+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.109256+0000","last_undegraded":"2024-04-24T22:59:57.109256+0000","last_fullsized":"2024-04-24T22:59:57.109256+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.111959+0000","last_change":"2024-04-24T22:59:57.111959+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.111959+0000","last_undegraded":"2024-04-24T22:59:57.111959+0000","last_fullsized":"2024-04-24T22:59:57.111959+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.111012+0000","last_change":"2024-04-24T22:59:57.111012+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.111012+0000","last_undegraded":"2024-04-24T22:59:57.111012+0000","last_fullsized":"2024-04-24T22:59:57.111012+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.112179+0000","last_change":"2024-04-24T22:59:57.112179+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.112179+0000","last_undegraded":"2024-04-24T22:59:57.112179+0000","last_fullsized":"2024-04-24T22:59:57.112179+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.109896+0000","last_change":"2024-04-24T22:59:57.109896+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.109896+0000","last_undegraded":"2024-04-24T22:59:57.109896+0000","last_fullsized":"2024-04-24T22:59:57.109896+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.111728+0000","last_change":"2024-04-24T22:59:57.111728+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.111728+0000","last_undegraded":"2024-04-24T22:59:57.111728+0000","last_fullsized":"2024-04-24T22:59:57.111728+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.109489+0000","last_change":"2024-04-24T22:59:57.109489+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.109489+0000","last_undegraded":"2024-04-24T22:59:57.109489+0000","last_fullsized":"2024-04-24T22:59:57.109489+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.108661+0000","last_change":"2024-04-24T22:59:57.108661+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.108661+0000","last_undegraded":"2024-04-24T22:59:57.108661+0000","last_fullsized":"2024-04-24T22:59:57.108661+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.109342+0000","last_change":"2024-04-24T22:59:57.109342+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.109342+0000","last_undegraded":"2024-04-24T22:59:57.109342+0000","last_fullsized":"2024-04-24T22:59:57.109342+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.108792+0000","last_change":"2024-04-24T22:59:57.108792+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.108792+0000","last_undegraded":"2024-04-24T22:59:57.108792+0000","last_fullsized":"2024-04-24T22:59:57.108792+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.108711+0000","last_change":"2024-04-24T22:59:57.108711+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.108711+0000","last_undegraded":"2024-04-24T22:59:57.108711+0000","last_fullsized":"2024-04-24T22:59:57.108711+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107354+0000","last_change":"2024-04-24T22:59:57.107354+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107354+0000","last_undegraded":"2024-04-24T22:59:57.107354+0000","last_fullsized":"2024-04-24T22:59:57.107354+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.114694+0000","last_change":"2024-04-24T22:59:57.114694+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.114694+0000","last_undegraded":"2024-04-24T22:59:57.114694+0000","last_fullsized":"2024-04-24T22:59:57.114694+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.110685+0000","last_change":"2024-04-24T22:59:57.110685+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.110685+0000","last_undegraded":"2024-04-24T22:59:57.110685+0000","last_fullsized":"2024-04-24T22:59:57.110685+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107448+0000","last_change":"2024-04-24T22:59:57.107448+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107448+0000","last_undegraded":"2024-04-24T22:59:57.107448+0000","last_fullsized":"2024-04-24T22:59:57.107448+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.109942+0000","last_change":"2024-04-24T22:59:57.109942+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.109942+0000","last_undegraded":"2024-04-24T22:59:57.109942+0000","last_fullsized":"2024-04-24T22:59:57.109942+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.108904+0000","last_change":"2024-04-24T22:59:57.108904+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.108904+0000","last_undegraded":"2024-04-24T22:59:57.108904+0000","last_fullsized":"2024-04-24T22:59:57.108904+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.110722+0000","last_change":"2024-04-24T22:59:57.110722+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.110722+0000","last_undegraded":"2024-04-24T22:59:57.110722+0000","last_fullsized":"2024-04-24T22:59:57.110722+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.108284+0000","last_change":"2024-04-24T22:59:57.108284+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.108284+0000","last_undegraded":"2024-04-24T22:59:57.108284+0000","last_fullsized":"2024-04-24T22:59:57.108284+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.109574+0000","last_change":"2024-04-24T22:59:57.109574+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.109574+0000","last_undegraded":"2024-04-24T22:59:57.109574+0000","last_fullsized":"2024-04-24T22:59:57.109574+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.108880+0000","last_change":"2024-04-24T22:59:57.108880+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.108880+0000","last_undegraded":"2024-04-24T22:59:57.108880+0000","last_fullsized":"2024-04-24T22:59:57.108880+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.115474+0000","last_change":"2024-04-24T22:59:57.114774+0000","last_active":"2024-04-24T22:59:57.115474+0000","last_peered":"2024-04-24T22:59:57.115474+0000","last_clean":"2024-04-24T22:59:57.115474+0000","last_became_active":"2024-04-24T22:59:57.114031+0000","last_became_peered":"2024-04-24T22:59:57.114031+0000","last_unstale":"2024-04-24T22:59:57.115474+0000","last_undegraded":"2024-04-24T22:59:57.115474+0000","last_fullsized":"2024-04-24T22:59:57.115474+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:02:17.915932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.108182+0000","last_change":"2024-04-24T22:59:57.108182+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.108182+0000","last_undegraded":"2024-04-24T22:59:57.108182+0000","last_fullsized":"2024-04-24T22:59:57.108182+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.118962+0000","last_change":"2024-04-24T22:59:57.118830+0000","last_active":"2024-04-24T22:59:57.118962+0000","last_peered":"2024-04-24T22:59:57.118962+0000","last_clean":"2024-04-24T22:59:57.118962+0000","last_became_active":"2024-04-24T22:59:57.118380+0000","last_became_peered":"2024-04-24T22:59:57.118380+0000","last_unstale":"2024-04-24T22:59:57.118962+0000","last_undegraded":"2024-04-24T22:59:57.118962+0000","last_fullsized":"2024-04-24T22:59:57.118962+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:34:34.824377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107978+0000","last_change":"2024-04-24T22:59:57.107978+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107978+0000","last_undegraded":"2024-04-24T22:59:57.107978+0000","last_fullsized":"2024-04-24T22:59:57.107978+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.113528+0000","last_change":"2024-04-24T22:59:57.113067+0000","last_active":"2024-04-24T22:59:57.113528+0000","last_peered":"2024-04-24T22:59:57.113528+0000","last_clean":"2024-04-24T22:59:57.113528+0000","last_became_active":"2024-04-24T22:59:57.112774+0000","last_became_peered":"2024-04-24T22:59:57.112774+0000","last_unstale":"2024-04-24T22:59:57.113528+0000","last_undegraded":"2024-04-24T22:59:57.113528+0000","last_fullsized":"2024-04-24T22:59:57.113528+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:09.897525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.114355+0000","last_change":"2024-04-24T22:59:57.114355+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.114355+0000","last_undegraded":"2024-04-24T22:59:57.114355+0000","last_fullsized":"2024-04-24T22:59:57.114355+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.118693+0000","last_change":"2024-04-24T22:59:57.118605+0000","last_active":"2024-04-24T22:59:57.118693+0000","last_peered":"2024-04-24T22:59:57.118693+0000","last_clean":"2024-04-24T22:59:57.118693+0000","last_became_active":"2024-04-24T22:59:57.118271+0000","last_became_peered":"2024-04-24T22:59:57.118271+0000","last_unstale":"2024-04-24T22:59:57.118693+0000","last_undegraded":"2024-04-24T22:59:57.118693+0000","last_fullsized":"2024-04-24T22:59:57.118693+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:04:12.275266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107238+0000","last_change":"2024-04-24T22:59:57.107238+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107238+0000","last_undegraded":"2024-04-24T22:59:57.107238+0000","last_fullsized":"2024-04-24T22:59:57.107238+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.109905+0000","last_change":"2024-04-24T22:59:57.109569+0000","last_active":"2024-04-24T22:59:57.109905+0000","last_peered":"2024-04-24T22:59:57.109905+0000","last_clean":"2024-04-24T22:59:57.109905+0000","last_became_active":"2024-04-24T22:59:57.109350+0000","last_became_peered":"2024-04-24T22:59:57.109350+0000","last_unstale":"2024-04-24T22:59:57.109905+0000","last_undegraded":"2024-04-24T22:59:57.109905+0000","last_fullsized":"2024-04-24T22:59:57.109905+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:54.689993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107517+0000","last_change":"2024-04-24T22:59:57.107517+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107517+0000","last_undegraded":"2024-04-24T22:59:57.107517+0000","last_fullsized":"2024-04-24T22:59:57.107517+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.113519+0000","last_change":"2024-04-24T22:59:57.113397+0000","last_active":"2024-04-24T22:59:57.113519+0000","last_peered":"2024-04-24T22:59:57.113519+0000","last_clean":"2024-04-24T22:59:57.113519+0000","last_became_active":"2024-04-24T22:59:57.113167+0000","last_became_peered":"2024-04-24T22:59:57.113167+0000","last_unstale":"2024-04-24T22:59:57.113519+0000","last_undegraded":"2024-04-24T22:59:57.113519+0000","last_fullsized":"2024-04-24T22:59:57.113519+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:29:04.934394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107218+0000","last_change":"2024-04-24T22:59:57.107218+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107218+0000","last_undegraded":"2024-04-24T22:59:57.107218+0000","last_fullsized":"2024-04-24T22:59:57.107218+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.116661+0000","last_change":"2024-04-24T22:59:57.116514+0000","last_active":"2024-04-24T22:59:57.116661+0000","last_peered":"2024-04-24T22:59:57.116661+0000","last_clean":"2024-04-24T22:59:57.116661+0000","last_became_active":"2024-04-24T22:59:57.116259+0000","last_became_peered":"2024-04-24T22:59:57.116259+0000","last_unstale":"2024-04-24T22:59:57.116661+0000","last_undegraded":"2024-04-24T22:59:57.116661+0000","last_fullsized":"2024-04-24T22:59:57.116661+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:43:58.559759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.112898+0000","last_change":"2024-04-24T22:59:57.112898+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.112898+0000","last_undegraded":"2024-04-24T22:59:57.112898+0000","last_fullsized":"2024-04-24T22:59:57.112898+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.117357+0000","last_change":"2024-04-24T22:59:57.117209+0000","last_active":"2024-04-24T22:59:57.117357+0000","last_peered":"2024-04-24T22:59:57.117357+0000","last_clean":"2024-04-24T22:59:57.117357+0000","last_became_active":"2024-04-24T22:59:57.116968+0000","last_became_peered":"2024-04-24T22:59:57.116968+0000","last_unstale":"2024-04-24T22:59:57.117357+0000","last_undegraded":"2024-04-24T22:59:57.117357+0000","last_fullsized":"2024-04-24T22:59:57.117357+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:44:17.871555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.109086+0000","last_change":"2024-04-24T22:59:57.109086+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.109086+0000","last_undegraded":"2024-04-24T22:59:57.109086+0000","last_fullsized":"2024-04-24T22:59:57.109086+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.106993+0000","last_change":"2024-04-24T22:59:57.106612+0000","last_active":"2024-04-24T22:59:57.106993+0000","last_peered":"2024-04-24T22:59:57.106993+0000","last_clean":"2024-04-24T22:59:57.106993+0000","last_became_active":"2024-04-24T22:59:57.106350+0000","last_became_peered":"2024-04-24T22:59:57.106350+0000","last_unstale":"2024-04-24T22:59:57.106993+0000","last_undegraded":"2024-04-24T22:59:57.106993+0000","last_fullsized":"2024-04-24T22:59:57.106993+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:45:14.576793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.112099+0000","last_change":"2024-04-24T22:59:57.112099+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.112099+0000","last_undegraded":"2024-04-24T22:59:57.112099+0000","last_fullsized":"2024-04-24T22:59:57.112099+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.115220+0000","last_change":"2024-04-24T22:59:57.115098+0000","last_active":"2024-04-24T22:59:57.115220+0000","last_peered":"2024-04-24T22:59:57.115220+0000","last_clean":"2024-04-24T22:59:57.115220+0000","last_became_active":"2024-04-24T22:59:57.114866+0000","last_became_peered":"2024-04-24T22:59:57.114866+0000","last_unstale":"2024-04-24T22:59:57.115220+0000","last_undegraded":"2024-04-24T22:59:57.115220+0000","last_fullsized":"2024-04-24T22:59:57.115220+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:27:09.165254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.111004+0000","last_change":"2024-04-24T22:59:57.111004+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.111004+0000","last_undegraded":"2024-04-24T22:59:57.111004+0000","last_fullsized":"2024-04-24T22:59:57.111004+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.115493+0000","last_change":"2024-04-24T22:59:57.115370+0000","last_active":"2024-04-24T22:59:57.115493+0000","last_peered":"2024-04-24T22:59:57.115493+0000","last_clean":"2024-04-24T22:59:57.115493+0000","last_became_active":"2024-04-24T22:59:57.115147+0000","last_became_peered":"2024-04-24T22:59:57.115147+0000","last_unstale":"2024-04-24T22:59:57.115493+0000","last_undegraded":"2024-04-24T22:59:57.115493+0000","last_fullsized":"2024-04-24T22:59:57.115493+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:23:03.727015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.108512+0000","last_change":"2024-04-24T22:59:57.108512+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.108512+0000","last_undegraded":"2024-04-24T22:59:57.108512+0000","last_fullsized":"2024-04-24T22:59:57.108512+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.118330+0000","last_change":"2024-04-24T22:59:57.118202+0000","last_active":"2024-04-24T22:59:57.118330+0000","last_peered":"2024-04-24T22:59:57.118330+0000","last_clean":"2024-04-24T22:59:57.118330+0000","last_became_active":"2024-04-24T22:59:57.117946+0000","last_became_peered":"2024-04-24T22:59:57.117946+0000","last_unstale":"2024-04-24T22:59:57.118330+0000","last_undegraded":"2024-04-24T22:59:57.118330+0000","last_fullsized":"2024-04-24T22:59:57.118330+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:37:52.953926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107500+0000","last_change":"2024-04-24T22:59:57.107500+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107500+0000","last_undegraded":"2024-04-24T22:59:57.107500+0000","last_fullsized":"2024-04-24T22:59:57.107500+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.112009+0000","last_change":"2024-04-24T22:59:57.111899+0000","last_active":"2024-04-24T22:59:57.112009+0000","last_peered":"2024-04-24T22:59:57.112009+0000","last_clean":"2024-04-24T22:59:57.112009+0000","last_became_active":"2024-04-24T22:59:57.111632+0000","last_became_peered":"2024-04-24T22:59:57.111632+0000","last_unstale":"2024-04-24T22:59:57.112009+0000","last_undegraded":"2024-04-24T22:59:57.112009+0000","last_fullsized":"2024-04-24T22:59:57.112009+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:01.204587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.114010+0000","last_change":"2024-04-24T22:59:57.114010+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.114010+0000","last_undegraded":"2024-04-24T22:59:57.114010+0000","last_fullsized":"2024-04-24T22:59:57.114010+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.116958+0000","last_change":"2024-04-24T22:59:57.116842+0000","last_active":"2024-04-24T22:59:57.116958+0000","last_peered":"2024-04-24T22:59:57.116958+0000","last_clean":"2024-04-24T22:59:57.116958+0000","last_became_active":"2024-04-24T22:59:57.116577+0000","last_became_peered":"2024-04-24T22:59:57.116577+0000","last_unstale":"2024-04-24T22:59:57.116958+0000","last_undegraded":"2024-04-24T22:59:57.116958+0000","last_fullsized":"2024-04-24T22:59:57.116958+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:29.595256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107852+0000","last_change":"2024-04-24T22:59:57.107852+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107852+0000","last_undegraded":"2024-04-24T22:59:57.107852+0000","last_fullsized":"2024-04-24T22:59:57.107852+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.115153+0000","last_change":"2024-04-24T22:59:57.113832+0000","last_active":"2024-04-24T22:59:57.115153+0000","last_peered":"2024-04-24T22:59:57.115153+0000","last_clean":"2024-04-24T22:59:57.115153+0000","last_became_active":"2024-04-24T22:59:57.113610+0000","last_became_peered":"2024-04-24T22:59:57.113610+0000","last_unstale":"2024-04-24T22:59:57.115153+0000","last_undegraded":"2024-04-24T22:59:57.115153+0000","last_fullsized":"2024-04-24T22:59:57.115153+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:49:50.026489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.108615+0000","last_change":"2024-04-24T22:59:57.108615+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.108615+0000","last_undegraded":"2024-04-24T22:59:57.108615+0000","last_fullsized":"2024-04-24T22:59:57.108615+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.116358+0000","last_change":"2024-04-24T22:59:57.116182+0000","last_active":"2024-04-24T22:59:57.116358+0000","last_peered":"2024-04-24T22:59:57.116358+0000","last_clean":"2024-04-24T22:59:57.116358+0000","last_became_active":"2024-04-24T22:59:57.115952+0000","last_became_peered":"2024-04-24T22:59:57.115952+0000","last_unstale":"2024-04-24T22:59:57.116358+0000","last_undegraded":"2024-04-24T22:59:57.116358+0000","last_fullsized":"2024-04-24T22:59:57.116358+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:20:34.273911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107248+0000","last_change":"2024-04-24T22:59:57.107248+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107248+0000","last_undegraded":"2024-04-24T22:59:57.107248+0000","last_fullsized":"2024-04-24T22:59:57.107248+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.119286+0000","last_change":"2024-04-24T22:59:57.119183+0000","last_active":"2024-04-24T22:59:57.119286+0000","last_peered":"2024-04-24T22:59:57.119286+0000","last_clean":"2024-04-24T22:59:57.119286+0000","last_became_active":"2024-04-24T22:59:57.118911+0000","last_became_peered":"2024-04-24T22:59:57.118911+0000","last_unstale":"2024-04-24T22:59:57.119286+0000","last_undegraded":"2024-04-24T22:59:57.119286+0000","last_fullsized":"2024-04-24T22:59:57.119286+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:28:06.461401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.106893+0000","last_change":"2024-04-24T22:59:57.106893+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.106893+0000","last_undegraded":"2024-04-24T22:59:57.106893+0000","last_fullsized":"2024-04-24T22:59:57.106893+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.115503+0000","last_change":"2024-04-24T22:59:57.115384+0000","last_active":"2024-04-24T22:59:57.115503+0000","last_peered":"2024-04-24T22:59:57.115503+0000","last_clean":"2024-04-24T22:59:57.115503+0000","last_became_active":"2024-04-24T22:59:57.115129+0000","last_became_peered":"2024-04-24T22:59:57.115129+0000","last_unstale":"2024-04-24T22:59:57.115503+0000","last_undegraded":"2024-04-24T22:59:57.115503+0000","last_fullsized":"2024-04-24T22:59:57.115503+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:13:57.006308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107117+0000","last_change":"2024-04-24T22:59:57.107117+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107117+0000","last_undegraded":"2024-04-24T22:59:57.107117+0000","last_fullsized":"2024-04-24T22:59:57.107117+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.114295+0000","last_change":"2024-04-24T22:59:57.113034+0000","last_active":"2024-04-24T22:59:57.114295+0000","last_peered":"2024-04-24T22:59:57.114295+0000","last_clean":"2024-04-24T22:59:57.114295+0000","last_became_active":"2024-04-24T22:59:57.111752+0000","last_became_peered":"2024-04-24T22:59:57.111752+0000","last_unstale":"2024-04-24T22:59:57.114295+0000","last_undegraded":"2024-04-24T22:59:57.114295+0000","last_fullsized":"2024-04-24T22:59:57.114295+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:11:29.081634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.109090+0000","last_change":"2024-04-24T22:59:57.109090+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.109090+0000","last_undegraded":"2024-04-24T22:59:57.109090+0000","last_fullsized":"2024-04-24T22:59:57.109090+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.117410+0000","last_change":"2024-04-24T22:59:57.117253+0000","last_active":"2024-04-24T22:59:57.117410+0000","last_peered":"2024-04-24T22:59:57.117410+0000","last_clean":"2024-04-24T22:59:57.117410+0000","last_became_active":"2024-04-24T22:59:57.116974+0000","last_became_peered":"2024-04-24T22:59:57.116974+0000","last_unstale":"2024-04-24T22:59:57.117410+0000","last_undegraded":"2024-04-24T22:59:57.117410+0000","last_fullsized":"2024-04-24T22:59:57.117410+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:24.934926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107613+0000","last_change":"2024-04-24T22:59:57.107613+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107613+0000","last_undegraded":"2024-04-24T22:59:57.107613+0000","last_fullsized":"2024-04-24T22:59:57.107613+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.112446+0000","last_change":"2024-04-24T22:59:57.111772+0000","last_active":"2024-04-24T22:59:57.112446+0000","last_peered":"2024-04-24T22:59:57.112446+0000","last_clean":"2024-04-24T22:59:57.112446+0000","last_became_active":"2024-04-24T22:59:57.111452+0000","last_became_peered":"2024-04-24T22:59:57.111452+0000","last_unstale":"2024-04-24T22:59:57.112446+0000","last_undegraded":"2024-04-24T22:59:57.112446+0000","last_fullsized":"2024-04-24T22:59:57.112446+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:27.552548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107006+0000","last_change":"2024-04-24T22:59:57.107006+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107006+0000","last_undegraded":"2024-04-24T22:59:57.107006+0000","last_fullsized":"2024-04-24T22:59:57.107006+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.111880+0000","last_change":"2024-04-24T22:59:57.111232+0000","last_active":"2024-04-24T22:59:57.111880+0000","last_peered":"2024-04-24T22:59:57.111880+0000","last_clean":"2024-04-24T22:59:57.111880+0000","last_became_active":"2024-04-24T22:59:57.111010+0000","last_became_peered":"2024-04-24T22:59:57.111010+0000","last_unstale":"2024-04-24T22:59:57.111880+0000","last_undegraded":"2024-04-24T22:59:57.111880+0000","last_fullsized":"2024-04-24T22:59:57.111880+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:02:29.250634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107089+0000","last_change":"2024-04-24T22:59:57.107089+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107089+0000","last_undegraded":"2024-04-24T22:59:57.107089+0000","last_fullsized":"2024-04-24T22:59:57.107089+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.111016+0000","last_change":"2024-04-24T22:59:57.110898+0000","last_active":"2024-04-24T22:59:57.111016+0000","last_peered":"2024-04-24T22:59:57.111016+0000","last_clean":"2024-04-24T22:59:57.111016+0000","last_became_active":"2024-04-24T22:59:57.110665+0000","last_became_peered":"2024-04-24T22:59:57.110665+0000","last_unstale":"2024-04-24T22:59:57.111016+0000","last_undegraded":"2024-04-24T22:59:57.111016+0000","last_fullsized":"2024-04-24T22:59:57.111016+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:55.518131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.106526+0000","last_change":"2024-04-24T22:59:57.106526+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.106526+0000","last_undegraded":"2024-04-24T22:59:57.106526+0000","last_fullsized":"2024-04-24T22:59:57.106526+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.114739+0000","last_change":"2024-04-24T22:59:57.114417+0000","last_active":"2024-04-24T22:59:57.114739+0000","last_peered":"2024-04-24T22:59:57.114739+0000","last_clean":"2024-04-24T22:59:57.114739+0000","last_became_active":"2024-04-24T22:59:57.112771+0000","last_became_peered":"2024-04-24T22:59:57.112771+0000","last_unstale":"2024-04-24T22:59:57.114739+0000","last_undegraded":"2024-04-24T22:59:57.114739+0000","last_fullsized":"2024-04-24T22:59:57.114739+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:11:37.810064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.110252+0000","last_change":"2024-04-24T22:59:57.110252+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.110252+0000","last_undegraded":"2024-04-24T22:59:57.110252+0000","last_fullsized":"2024-04-24T22:59:57.110252+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.115885+0000","last_change":"2024-04-24T22:59:57.115765+0000","last_active":"2024-04-24T22:59:57.115885+0000","last_peered":"2024-04-24T22:59:57.115885+0000","last_clean":"2024-04-24T22:59:57.115885+0000","last_became_active":"2024-04-24T22:59:57.115056+0000","last_became_peered":"2024-04-24T22:59:57.115056+0000","last_unstale":"2024-04-24T22:59:57.115885+0000","last_undegraded":"2024-04-24T22:59:57.115885+0000","last_fullsized":"2024-04-24T22:59:57.115885+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:18:11.299169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.106865+0000","last_change":"2024-04-24T22:59:57.106865+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.106865+0000","last_undegraded":"2024-04-24T22:59:57.106865+0000","last_fullsized":"2024-04-24T22:59:57.106865+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.113678+0000","last_change":"2024-04-24T22:59:57.113564+0000","last_active":"2024-04-24T22:59:57.113678+0000","last_peered":"2024-04-24T22:59:57.113678+0000","last_clean":"2024-04-24T22:59:57.113678+0000","last_became_active":"2024-04-24T22:59:57.113292+0000","last_became_peered":"2024-04-24T22:59:57.113292+0000","last_unstale":"2024-04-24T22:59:57.113678+0000","last_undegraded":"2024-04-24T22:59:57.113678+0000","last_fullsized":"2024-04-24T22:59:57.113678+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:05:41.614373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.109161+0000","last_change":"2024-04-24T22:59:57.109161+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.109161+0000","last_undegraded":"2024-04-24T22:59:57.109161+0000","last_fullsized":"2024-04-24T22:59:57.109161+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.112241+0000","last_change":"2024-04-24T22:59:57.111930+0000","last_active":"2024-04-24T22:59:57.112241+0000","last_peered":"2024-04-24T22:59:57.112241+0000","last_clean":"2024-04-24T22:59:57.112241+0000","last_became_active":"2024-04-24T22:59:57.111632+0000","last_became_peered":"2024-04-24T22:59:57.111632+0000","last_unstale":"2024-04-24T22:59:57.112241+0000","last_undegraded":"2024-04-24T22:59:57.112241+0000","last_fullsized":"2024-04-24T22:59:57.112241+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:57.770833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.110730+0000","last_change":"2024-04-24T22:59:57.110730+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.110730+0000","last_undegraded":"2024-04-24T22:59:57.110730+0000","last_fullsized":"2024-04-24T22:59:57.110730+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.115239+0000","last_change":"2024-04-24T22:59:57.115123+0000","last_active":"2024-04-24T22:59:57.115239+0000","last_peered":"2024-04-24T22:59:57.115239+0000","last_clean":"2024-04-24T22:59:57.115239+0000","last_became_active":"2024-04-24T22:59:57.114845+0000","last_became_peered":"2024-04-24T22:59:57.114845+0000","last_unstale":"2024-04-24T22:59:57.115239+0000","last_undegraded":"2024-04-24T22:59:57.115239+0000","last_fullsized":"2024-04-24T22:59:57.115239+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:31.884038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.111780+0000","last_change":"2024-04-24T22:59:57.111780+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.111780+0000","last_undegraded":"2024-04-24T22:59:57.111780+0000","last_fullsized":"2024-04-24T22:59:57.111780+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.112631+0000","last_change":"2024-04-24T22:59:57.112511+0000","last_active":"2024-04-24T22:59:57.112631+0000","last_peered":"2024-04-24T22:59:57.112631+0000","last_clean":"2024-04-24T22:59:57.112631+0000","last_became_active":"2024-04-24T22:59:57.112210+0000","last_became_peered":"2024-04-24T22:59:57.112210+0000","last_unstale":"2024-04-24T22:59:57.112631+0000","last_undegraded":"2024-04-24T22:59:57.112631+0000","last_fullsized":"2024-04-24T22:59:57.112631+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:04.204693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.112288+0000","last_change":"2024-04-24T22:59:57.112288+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.112288+0000","last_undegraded":"2024-04-24T22:59:57.112288+0000","last_fullsized":"2024-04-24T22:59:57.112288+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.111354+0000","last_change":"2024-04-24T22:59:57.111232+0000","last_active":"2024-04-24T22:59:57.111354+0000","last_peered":"2024-04-24T22:59:57.111354+0000","last_clean":"2024-04-24T22:59:57.111354+0000","last_became_active":"2024-04-24T22:59:57.110997+0000","last_became_peered":"2024-04-24T22:59:57.110997+0000","last_unstale":"2024-04-24T22:59:57.111354+0000","last_undegraded":"2024-04-24T22:59:57.111354+0000","last_fullsized":"2024-04-24T22:59:57.111354+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:56:07.604519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107025+0000","last_change":"2024-04-24T22:59:57.107025+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107025+0000","last_undegraded":"2024-04-24T22:59:57.107025+0000","last_fullsized":"2024-04-24T22:59:57.107025+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.118210+0000","last_change":"2024-04-24T22:59:57.118085+0000","last_active":"2024-04-24T22:59:57.118210+0000","last_peered":"2024-04-24T22:59:57.118210+0000","last_clean":"2024-04-24T22:59:57.118210+0000","last_became_active":"2024-04-24T22:59:57.116287+0000","last_became_peered":"2024-04-24T22:59:57.116287+0000","last_unstale":"2024-04-24T22:59:57.118210+0000","last_undegraded":"2024-04-24T22:59:57.118210+0000","last_fullsized":"2024-04-24T22:59:57.118210+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:11:38.330427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.110417+0000","last_change":"2024-04-24T22:59:57.110417+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.110417+0000","last_undegraded":"2024-04-24T22:59:57.110417+0000","last_fullsized":"2024-04-24T22:59:57.110417+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.112334+0000","last_change":"2024-04-24T22:59:57.110803+0000","last_active":"2024-04-24T22:59:57.112334+0000","last_peered":"2024-04-24T22:59:57.112334+0000","last_clean":"2024-04-24T22:59:57.112334+0000","last_became_active":"2024-04-24T22:59:57.110572+0000","last_became_peered":"2024-04-24T22:59:57.110572+0000","last_unstale":"2024-04-24T22:59:57.112334+0000","last_undegraded":"2024-04-24T22:59:57.112334+0000","last_fullsized":"2024-04-24T22:59:57.112334+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:06:59.730529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":186,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.105292+0000","last_change":"2024-04-24T22:58:55.138810+0000","last_active":"2024-04-24T22:59:57.105292+0000","last_peered":"2024-04-24T22:59:57.105292+0000","last_clean":"2024-04-24T22:59:57.105292+0000","last_became_active":"2024-04-24T22:58:55.138627+0000","last_became_peered":"2024-04-24T22:58:55.138627+0000","last_unstale":"2024-04-24T22:59:57.105292+0000","last_undegraded":"2024-04-24T22:59:57.105292+0000","last_fullsized":"2024-04-24T22:59:57.105292+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_clean_scrub_stamp":"2024-04-24T22:58:54.127235+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:24.010002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":116,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195196,"statfs":{"total":200000000,"available":199880705,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":116,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195196,"statfs":{"total":200000000,"available":199880705,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":693,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194619,"statfs":{"total":200000000,"available":199290337,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":116,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195196,"statfs":{"total":200000000,"available":199880705,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":116,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195196,"statfs":{"total":200000000,"available":199880705,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":116,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195196,"statfs":{"total":200000000,"available":199880705,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":687,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194625,"statfs":{"total":200000000,"available":199296154,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":116,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195196,"statfs":{"total":200000000,"available":199880705,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-24T23:00:01.820 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3f is not active+clean 2024-04-24T23:00:01.821 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.108588+0000', 'last_change': '2024-04-24T22:59:57.108588+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.108588+0000', 'last_undegraded': '2024-04-24T22:59:57.108588+0000', 'last_fullsized': '2024-04-24T22:59:57.108588+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-24T23:00:01.821 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3e is not active+clean 2024-04-24T23:00:01.821 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.108603+0000', 'last_change': '2024-04-24T22:59:57.108603+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.108603+0000', 'last_undegraded': '2024-04-24T22:59:57.108603+0000', 'last_fullsized': '2024-04-24T22:59:57.108603+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-24T23:00:01.821 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3d is not active+clean 2024-04-24T23:00:01.821 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.107901+0000', 'last_change': '2024-04-24T22:59:57.107901+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.107901+0000', 'last_undegraded': '2024-04-24T22:59:57.107901+0000', 'last_fullsized': '2024-04-24T22:59:57.107901+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-24T23:00:01.821 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3c is not active+clean 2024-04-24T23:00:01.821 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.112595+0000', 'last_change': '2024-04-24T22:59:57.112595+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.112595+0000', 'last_undegraded': '2024-04-24T22:59:57.112595+0000', 'last_fullsized': '2024-04-24T22:59:57.112595+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-24T23:00:01.821 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3b is not active+clean 2024-04-24T23:00:01.821 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.108208+0000', 'last_change': '2024-04-24T22:59:57.108208+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.108208+0000', 'last_undegraded': '2024-04-24T22:59:57.108208+0000', 'last_fullsized': '2024-04-24T22:59:57.108208+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-24T23:00:01.822 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3a is not active+clean 2024-04-24T23:00:01.822 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.109478+0000', 'last_change': '2024-04-24T22:59:57.109478+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.109478+0000', 'last_undegraded': '2024-04-24T22:59:57.109478+0000', 'last_fullsized': '2024-04-24T22:59:57.109478+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-24T23:00:01.822 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.39 is not active+clean 2024-04-24T23:00:01.822 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.109077+0000', 'last_change': '2024-04-24T22:59:57.109077+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.109077+0000', 'last_undegraded': '2024-04-24T22:59:57.109077+0000', 'last_fullsized': '2024-04-24T22:59:57.109077+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-24T23:00:01.822 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.38 is not active+clean 2024-04-24T23:00:01.822 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.110299+0000', 'last_change': '2024-04-24T22:59:57.110299+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.110299+0000', 'last_undegraded': '2024-04-24T22:59:57.110299+0000', 'last_fullsized': '2024-04-24T22:59:57.110299+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-24T23:00:01.822 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.37 is not active+clean 2024-04-24T23:00:01.822 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.108010+0000', 'last_change': '2024-04-24T22:59:57.108010+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.108010+0000', 'last_undegraded': '2024-04-24T22:59:57.108010+0000', 'last_fullsized': '2024-04-24T22:59:57.108010+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-24T23:00:01.822 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.36 is not active+clean 2024-04-24T23:00:01.822 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.115008+0000', 'last_change': '2024-04-24T22:59:57.115008+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.115008+0000', 'last_undegraded': '2024-04-24T22:59:57.115008+0000', 'last_fullsized': '2024-04-24T22:59:57.115008+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-24T23:00:01.822 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.35 is not active+clean 2024-04-24T23:00:01.822 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.109250+0000', 'last_change': '2024-04-24T22:59:57.109250+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.109250+0000', 'last_undegraded': '2024-04-24T22:59:57.109250+0000', 'last_fullsized': '2024-04-24T22:59:57.109250+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-24T23:00:01.823 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.34 is not active+clean 2024-04-24T23:00:01.823 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.107645+0000', 'last_change': '2024-04-24T22:59:57.107645+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.107645+0000', 'last_undegraded': '2024-04-24T22:59:57.107645+0000', 'last_fullsized': '2024-04-24T22:59:57.107645+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-24T23:00:01.823 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.33 is not active+clean 2024-04-24T23:00:01.823 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.109256+0000', 'last_change': '2024-04-24T22:59:57.109256+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.109256+0000', 'last_undegraded': '2024-04-24T22:59:57.109256+0000', 'last_fullsized': '2024-04-24T22:59:57.109256+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-24T23:00:01.823 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.32 is not active+clean 2024-04-24T23:00:01.823 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.111959+0000', 'last_change': '2024-04-24T22:59:57.111959+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.111959+0000', 'last_undegraded': '2024-04-24T22:59:57.111959+0000', 'last_fullsized': '2024-04-24T22:59:57.111959+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-24T23:00:01.823 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.31 is not active+clean 2024-04-24T23:00:01.823 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.111012+0000', 'last_change': '2024-04-24T22:59:57.111012+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.111012+0000', 'last_undegraded': '2024-04-24T22:59:57.111012+0000', 'last_fullsized': '2024-04-24T22:59:57.111012+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-24T23:00:01.823 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.30 is not active+clean 2024-04-24T23:00:01.823 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.112179+0000', 'last_change': '2024-04-24T22:59:57.112179+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.112179+0000', 'last_undegraded': '2024-04-24T22:59:57.112179+0000', 'last_fullsized': '2024-04-24T22:59:57.112179+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-24T23:00:01.823 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2f is not active+clean 2024-04-24T23:00:01.824 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.109896+0000', 'last_change': '2024-04-24T22:59:57.109896+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.109896+0000', 'last_undegraded': '2024-04-24T22:59:57.109896+0000', 'last_fullsized': '2024-04-24T22:59:57.109896+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-24T23:00:01.824 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2e is not active+clean 2024-04-24T23:00:01.824 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.111728+0000', 'last_change': '2024-04-24T22:59:57.111728+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.111728+0000', 'last_undegraded': '2024-04-24T22:59:57.111728+0000', 'last_fullsized': '2024-04-24T22:59:57.111728+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-24T23:00:01.824 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2d is not active+clean 2024-04-24T23:00:01.824 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.109489+0000', 'last_change': '2024-04-24T22:59:57.109489+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.109489+0000', 'last_undegraded': '2024-04-24T22:59:57.109489+0000', 'last_fullsized': '2024-04-24T22:59:57.109489+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-24T23:00:01.824 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2c is not active+clean 2024-04-24T23:00:01.824 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.108661+0000', 'last_change': '2024-04-24T22:59:57.108661+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.108661+0000', 'last_undegraded': '2024-04-24T22:59:57.108661+0000', 'last_fullsized': '2024-04-24T22:59:57.108661+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-24T23:00:01.824 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2b is not active+clean 2024-04-24T23:00:01.824 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.109342+0000', 'last_change': '2024-04-24T22:59:57.109342+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.109342+0000', 'last_undegraded': '2024-04-24T22:59:57.109342+0000', 'last_fullsized': '2024-04-24T22:59:57.109342+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-24T23:00:01.824 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2a is not active+clean 2024-04-24T23:00:01.824 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.108792+0000', 'last_change': '2024-04-24T22:59:57.108792+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.108792+0000', 'last_undegraded': '2024-04-24T22:59:57.108792+0000', 'last_fullsized': '2024-04-24T22:59:57.108792+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-24T23:00:01.825 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.29 is not active+clean 2024-04-24T23:00:01.825 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.108711+0000', 'last_change': '2024-04-24T22:59:57.108711+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.108711+0000', 'last_undegraded': '2024-04-24T22:59:57.108711+0000', 'last_fullsized': '2024-04-24T22:59:57.108711+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-24T23:00:01.825 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.28 is not active+clean 2024-04-24T23:00:01.825 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.107354+0000', 'last_change': '2024-04-24T22:59:57.107354+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.107354+0000', 'last_undegraded': '2024-04-24T22:59:57.107354+0000', 'last_fullsized': '2024-04-24T22:59:57.107354+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-24T23:00:01.825 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.27 is not active+clean 2024-04-24T23:00:01.825 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.114694+0000', 'last_change': '2024-04-24T22:59:57.114694+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.114694+0000', 'last_undegraded': '2024-04-24T22:59:57.114694+0000', 'last_fullsized': '2024-04-24T22:59:57.114694+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-24T23:00:01.825 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.26 is not active+clean 2024-04-24T23:00:01.825 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.110685+0000', 'last_change': '2024-04-24T22:59:57.110685+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.110685+0000', 'last_undegraded': '2024-04-24T22:59:57.110685+0000', 'last_fullsized': '2024-04-24T22:59:57.110685+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-24T23:00:01.825 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.25 is not active+clean 2024-04-24T23:00:01.825 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.107448+0000', 'last_change': '2024-04-24T22:59:57.107448+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.107448+0000', 'last_undegraded': '2024-04-24T22:59:57.107448+0000', 'last_fullsized': '2024-04-24T22:59:57.107448+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-24T23:00:01.825 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.24 is not active+clean 2024-04-24T23:00:01.826 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.109942+0000', 'last_change': '2024-04-24T22:59:57.109942+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.109942+0000', 'last_undegraded': '2024-04-24T22:59:57.109942+0000', 'last_fullsized': '2024-04-24T22:59:57.109942+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-24T23:00:01.826 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.23 is not active+clean 2024-04-24T23:00:01.826 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.108904+0000', 'last_change': '2024-04-24T22:59:57.108904+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.108904+0000', 'last_undegraded': '2024-04-24T22:59:57.108904+0000', 'last_fullsized': '2024-04-24T22:59:57.108904+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-24T23:00:01.826 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.22 is not active+clean 2024-04-24T23:00:01.826 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.110722+0000', 'last_change': '2024-04-24T22:59:57.110722+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.110722+0000', 'last_undegraded': '2024-04-24T22:59:57.110722+0000', 'last_fullsized': '2024-04-24T22:59:57.110722+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-24T23:00:01.826 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.21 is not active+clean 2024-04-24T23:00:01.826 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.108284+0000', 'last_change': '2024-04-24T22:59:57.108284+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.108284+0000', 'last_undegraded': '2024-04-24T22:59:57.108284+0000', 'last_fullsized': '2024-04-24T22:59:57.108284+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-24T23:00:01.826 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.20 is not active+clean 2024-04-24T23:00:01.826 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.109574+0000', 'last_change': '2024-04-24T22:59:57.109574+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.109574+0000', 'last_undegraded': '2024-04-24T22:59:57.109574+0000', 'last_fullsized': '2024-04-24T22:59:57.109574+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-24T23:00:01.826 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1e is not active+clean 2024-04-24T23:00:01.826 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.108880+0000', 'last_change': '2024-04-24T22:59:57.108880+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.108880+0000', 'last_undegraded': '2024-04-24T22:59:57.108880+0000', 'last_fullsized': '2024-04-24T22:59:57.108880+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-24T23:00:01.827 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1f is not active+clean 2024-04-24T23:00:01.827 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.108182+0000', 'last_change': '2024-04-24T22:59:57.108182+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.108182+0000', 'last_undegraded': '2024-04-24T22:59:57.108182+0000', 'last_fullsized': '2024-04-24T22:59:57.108182+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-24T23:00:01.827 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1c is not active+clean 2024-04-24T23:00:01.827 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.107978+0000', 'last_change': '2024-04-24T22:59:57.107978+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.107978+0000', 'last_undegraded': '2024-04-24T22:59:57.107978+0000', 'last_fullsized': '2024-04-24T22:59:57.107978+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-24T23:00:01.827 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1d is not active+clean 2024-04-24T23:00:01.827 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.114355+0000', 'last_change': '2024-04-24T22:59:57.114355+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.114355+0000', 'last_undegraded': '2024-04-24T22:59:57.114355+0000', 'last_fullsized': '2024-04-24T22:59:57.114355+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-24T23:00:01.827 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1a is not active+clean 2024-04-24T23:00:01.827 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.107238+0000', 'last_change': '2024-04-24T22:59:57.107238+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.107238+0000', 'last_undegraded': '2024-04-24T22:59:57.107238+0000', 'last_fullsized': '2024-04-24T22:59:57.107238+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-24T23:00:01.827 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1b is not active+clean 2024-04-24T23:00:01.827 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.107517+0000', 'last_change': '2024-04-24T22:59:57.107517+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.107517+0000', 'last_undegraded': '2024-04-24T22:59:57.107517+0000', 'last_fullsized': '2024-04-24T22:59:57.107517+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-24T23:00:01.827 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.18 is not active+clean 2024-04-24T23:00:01.828 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.107218+0000', 'last_change': '2024-04-24T22:59:57.107218+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.107218+0000', 'last_undegraded': '2024-04-24T22:59:57.107218+0000', 'last_fullsized': '2024-04-24T22:59:57.107218+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-24T23:00:01.828 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.19 is not active+clean 2024-04-24T23:00:01.828 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.112898+0000', 'last_change': '2024-04-24T22:59:57.112898+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.112898+0000', 'last_undegraded': '2024-04-24T22:59:57.112898+0000', 'last_fullsized': '2024-04-24T22:59:57.112898+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-24T23:00:01.828 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.16 is not active+clean 2024-04-24T23:00:01.828 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.109086+0000', 'last_change': '2024-04-24T22:59:57.109086+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.109086+0000', 'last_undegraded': '2024-04-24T22:59:57.109086+0000', 'last_fullsized': '2024-04-24T22:59:57.109086+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-24T23:00:01.828 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.17 is not active+clean 2024-04-24T23:00:01.828 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.112099+0000', 'last_change': '2024-04-24T22:59:57.112099+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.112099+0000', 'last_undegraded': '2024-04-24T22:59:57.112099+0000', 'last_fullsized': '2024-04-24T22:59:57.112099+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-24T23:00:01.828 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.14 is not active+clean 2024-04-24T23:00:01.828 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.111004+0000', 'last_change': '2024-04-24T22:59:57.111004+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.111004+0000', 'last_undegraded': '2024-04-24T22:59:57.111004+0000', 'last_fullsized': '2024-04-24T22:59:57.111004+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-24T23:00:01.828 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.15 is not active+clean 2024-04-24T23:00:01.828 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.108512+0000', 'last_change': '2024-04-24T22:59:57.108512+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.108512+0000', 'last_undegraded': '2024-04-24T22:59:57.108512+0000', 'last_fullsized': '2024-04-24T22:59:57.108512+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-24T23:00:01.829 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.12 is not active+clean 2024-04-24T23:00:01.829 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.107500+0000', 'last_change': '2024-04-24T22:59:57.107500+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.107500+0000', 'last_undegraded': '2024-04-24T22:59:57.107500+0000', 'last_fullsized': '2024-04-24T22:59:57.107500+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-24T23:00:01.829 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.13 is not active+clean 2024-04-24T23:00:01.829 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.114010+0000', 'last_change': '2024-04-24T22:59:57.114010+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.114010+0000', 'last_undegraded': '2024-04-24T22:59:57.114010+0000', 'last_fullsized': '2024-04-24T22:59:57.114010+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-24T23:00:01.829 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.10 is not active+clean 2024-04-24T23:00:01.829 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.107852+0000', 'last_change': '2024-04-24T22:59:57.107852+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.107852+0000', 'last_undegraded': '2024-04-24T22:59:57.107852+0000', 'last_fullsized': '2024-04-24T22:59:57.107852+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-24T23:00:01.829 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.11 is not active+clean 2024-04-24T23:00:01.829 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.108615+0000', 'last_change': '2024-04-24T22:59:57.108615+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.108615+0000', 'last_undegraded': '2024-04-24T22:59:57.108615+0000', 'last_fullsized': '2024-04-24T22:59:57.108615+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-24T23:00:01.829 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.e is not active+clean 2024-04-24T23:00:01.829 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.107248+0000', 'last_change': '2024-04-24T22:59:57.107248+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.107248+0000', 'last_undegraded': '2024-04-24T22:59:57.107248+0000', 'last_fullsized': '2024-04-24T22:59:57.107248+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-24T23:00:01.829 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.f is not active+clean 2024-04-24T23:00:01.830 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.106893+0000', 'last_change': '2024-04-24T22:59:57.106893+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.106893+0000', 'last_undegraded': '2024-04-24T22:59:57.106893+0000', 'last_fullsized': '2024-04-24T22:59:57.106893+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-24T23:00:01.830 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.c is not active+clean 2024-04-24T23:00:01.830 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.107117+0000', 'last_change': '2024-04-24T22:59:57.107117+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.107117+0000', 'last_undegraded': '2024-04-24T22:59:57.107117+0000', 'last_fullsized': '2024-04-24T22:59:57.107117+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-24T23:00:01.830 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.d is not active+clean 2024-04-24T23:00:01.830 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.109090+0000', 'last_change': '2024-04-24T22:59:57.109090+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.109090+0000', 'last_undegraded': '2024-04-24T22:59:57.109090+0000', 'last_fullsized': '2024-04-24T22:59:57.109090+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-24T23:00:01.830 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.a is not active+clean 2024-04-24T23:00:01.830 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.107613+0000', 'last_change': '2024-04-24T22:59:57.107613+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.107613+0000', 'last_undegraded': '2024-04-24T22:59:57.107613+0000', 'last_fullsized': '2024-04-24T22:59:57.107613+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-24T23:00:01.830 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.b is not active+clean 2024-04-24T23:00:01.830 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.107006+0000', 'last_change': '2024-04-24T22:59:57.107006+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.107006+0000', 'last_undegraded': '2024-04-24T22:59:57.107006+0000', 'last_fullsized': '2024-04-24T22:59:57.107006+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-24T23:00:01.831 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.8 is not active+clean 2024-04-24T23:00:01.831 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.107089+0000', 'last_change': '2024-04-24T22:59:57.107089+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.107089+0000', 'last_undegraded': '2024-04-24T22:59:57.107089+0000', 'last_fullsized': '2024-04-24T22:59:57.107089+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-24T23:00:01.831 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.9 is not active+clean 2024-04-24T23:00:01.831 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.106526+0000', 'last_change': '2024-04-24T22:59:57.106526+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.106526+0000', 'last_undegraded': '2024-04-24T22:59:57.106526+0000', 'last_fullsized': '2024-04-24T22:59:57.106526+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-24T23:00:01.831 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.6 is not active+clean 2024-04-24T23:00:01.831 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.110252+0000', 'last_change': '2024-04-24T22:59:57.110252+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.110252+0000', 'last_undegraded': '2024-04-24T22:59:57.110252+0000', 'last_fullsized': '2024-04-24T22:59:57.110252+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-24T23:00:01.831 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.7 is not active+clean 2024-04-24T23:00:01.831 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.106865+0000', 'last_change': '2024-04-24T22:59:57.106865+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.106865+0000', 'last_undegraded': '2024-04-24T22:59:57.106865+0000', 'last_fullsized': '2024-04-24T22:59:57.106865+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-24T23:00:01.831 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.5 is not active+clean 2024-04-24T23:00:01.831 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.109161+0000', 'last_change': '2024-04-24T22:59:57.109161+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.109161+0000', 'last_undegraded': '2024-04-24T22:59:57.109161+0000', 'last_fullsized': '2024-04-24T22:59:57.109161+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-24T23:00:01.832 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2 is not active+clean 2024-04-24T23:00:01.832 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.110730+0000', 'last_change': '2024-04-24T22:59:57.110730+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.110730+0000', 'last_undegraded': '2024-04-24T22:59:57.110730+0000', 'last_fullsized': '2024-04-24T22:59:57.110730+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-24T23:00:01.832 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3 is not active+clean 2024-04-24T23:00:01.832 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.111780+0000', 'last_change': '2024-04-24T22:59:57.111780+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.111780+0000', 'last_undegraded': '2024-04-24T22:59:57.111780+0000', 'last_fullsized': '2024-04-24T22:59:57.111780+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-24T23:00:01.832 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.0 is not active+clean 2024-04-24T23:00:01.832 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.112288+0000', 'last_change': '2024-04-24T22:59:57.112288+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.112288+0000', 'last_undegraded': '2024-04-24T22:59:57.112288+0000', 'last_fullsized': '2024-04-24T22:59:57.112288+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-24T23:00:01.832 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1 is not active+clean 2024-04-24T23:00:01.832 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.107025+0000', 'last_change': '2024-04-24T22:59:57.107025+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.107025+0000', 'last_undegraded': '2024-04-24T22:59:57.107025+0000', 'last_fullsized': '2024-04-24T22:59:57.107025+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-24T23:00:01.832 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.4 is not active+clean 2024-04-24T23:00:01.832 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-04-24T22:59:57.110417+0000', 'last_change': '2024-04-24T22:59:57.110417+0000', 'last_active': '2024-04-24T22:59:57.098836+0000', 'last_peered': '2024-04-24T22:59:57.098836+0000', 'last_clean': '2024-04-24T22:59:57.098836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T22:59:57.110417+0000', 'last_undegraded': '2024-04-24T22:59:57.110417+0000', 'last_fullsized': '2024-04-24T22:59:57.110417+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'last_clean_scrub_stamp': '2024-04-24T22:59:57.098836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-24T23:00:01.833 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-24T23:00:01.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.955+0000 7ff44b314640 1 Processor -- start 2024-04-24T23:00:01.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.956+0000 7ff44b314640 1 -- start start 2024-04-24T23:00:01.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.956+0000 7ff44b314640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff44406b820 0x7ff4440d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:01.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.956+0000 7ff44b314640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff4440d4210 0x7ff4440b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:01.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.956+0000 7ff44b314640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4440d3170 con 0x7ff44415e9b0 2024-04-24T23:00:01.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.956+0000 7ff44b314640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7ff4440d3450 con 0x7ff4440d4210 2024-04-24T23:00:01.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.956+0000 7ff44b314640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff4440d3730 con 0x7ff44406b820 2024-04-24T23:00:01.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.956+0000 7ff4498b1640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff4440d4210 0x7ff4440b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:01.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.956+0000 7ff4490b0640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff44415e9b0 0x7ff44406b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39254/0 (socket says 172.21.15.32:39254) 2024-04-24T23:00:01.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.957+0000 7ff4490b0640 1 -- 172.21.15.32:0/382375284 learned_addr learned my addr 172.21.15.32:0/382375284 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:00:01.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.957+0000 7ff4488af640 1 --2- 172.21.15.32:0/382375284 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff44406b820 0x7ff4440d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:01.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.957+0000 7ff4498b1640 1 -- 172.21.15.32:0/382375284 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff44406b820 msgr2=0x7ff4440d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:01.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.957+0000 7ff4498b1640 1 --2- 172.21.15.32:0/382375284 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff44406b820 0x7ff4440d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:01.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.957+0000 7ff4498b1640 1 -- 172.21.15.32:0/382375284 >> v1:172.21.15.32:6789/0 conn(0x7ff44415e9b0 legacy=0x7ff44406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:01.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.957+0000 7ff443fff640 1 -- 172.21.15.32:0/382375284 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3601077548 0 0) 0x7ff4440d3170 con 0x7ff44415e9b0 2024-04-24T23:00:01.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.957+0000 7ff4498b1640 1 -- 172.21.15.32:0/382375284 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4440d3a10 con 0x7ff4440d4210 2024-04-24T23:00:01.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.957+0000 7ff4488af640 1 --2- 172.21.15.32:0/382375284 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff44406b820 0x7ff4440d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:00:01.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.957+0000 7ff4498b1640 1 --2- 172.21.15.32:0/382375284 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4440d4210 0x7ff4440b78f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff434009d90 tx=0x7ff43402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8135becbda2d9967 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:01.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.958+0000 7ff443fff640 1 -- 172.21.15.32:0/382375284 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff434007740 con 0x7ff4440d4210 2024-04-24T23:00:01.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.958+0000 7ff443fff640 1 -- 172.21.15.32:0/382375284 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4340078e0 con 0x7ff4440d4210 2024-04-24T23:00:01.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.958+0000 7ff443fff640 1 -- 172.21.15.32:0/382375284 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff434007cb0 con 0x7ff4440d4210 2024-04-24T23:00:01.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.958+0000 7ff44b314640 1 -- 172.21.15.32:0/382375284 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4440d4210 msgr2=0x7ff4440b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:01.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.958+0000 7ff44b314640 1 --2- 172.21.15.32:0/382375284 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4440d4210 0x7ff4440b78f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff434009d90 tx=0x7ff43402a660 comp rx=0 tx=0).stop 2024-04-24T23:00:01.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.958+0000 7ff44b314640 1 -- 172.21.15.32:0/382375284 shutdown_connections 2024-04-24T23:00:01.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.958+0000 7ff44b314640 1 --2- 172.21.15.32:0/382375284 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4440d4210 0x7ff4440b78f0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:01.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.958+0000 7ff44b314640 1 --2- 172.21.15.32:0/382375284 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff44406b820 0x7ff4440d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:01.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.958+0000 7ff44b314640 1 -- 172.21.15.32:0/382375284 >> 172.21.15.32:0/382375284 conn(0x7ff4440d11d0 msgr2=0x7ff4440d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:01.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.958+0000 7ff44b314640 1 -- 172.21.15.32:0/382375284 shutdown_connections 2024-04-24T23:00:01.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.958+0000 7ff44b314640 1 -- 172.21.15.32:0/382375284 wait complete. 2024-04-24T23:00:01.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.959+0000 7ff44b314640 1 Processor -- start 2024-04-24T23:00:01.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.960+0000 7ff44b314640 1 -- start start 2024-04-24T23:00:01.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.960+0000 7ff44b314640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff44406b820 0x7ff44415bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:01.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.960+0000 7ff44b314640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4440d4210 0x7ff44415c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:01.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.960+0000 7ff44b314640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff44415e9b0 0x7ff4441f79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:01.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.960+0000 7ff44b314640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff4440ba0e0 con 0x7ff4440d4210 2024-04-24T23:00:01.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.960+0000 7ff44b314640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff4440d2400 con 0x7ff44415e9b0 2024-04-24T23:00:01.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.960+0000 7ff44b314640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff4440ba260 con 0x7ff44406b820 2024-04-24T23:00:01.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.960+0000 7ff4488af640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4440d4210 0x7ff44415c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:01.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.960+0000 7ff4488af640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4440d4210 0x7ff44415c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60072/0 (socket says 172.21.15.32:60072) 2024-04-24T23:00:01.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.960+0000 7ff4488af640 1 -- 172.21.15.32:0/3958602955 learned_addr learned my addr 172.21.15.32:0/3958602955 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:01.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.961+0000 7ff4490b0640 1 --2- 172.21.15.32:0/3958602955 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff44406b820 0x7ff44415bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:01.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.961+0000 7ff4498b1640 1 --2- 172.21.15.32:0/3958602955 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff44415e9b0 0x7ff4441f79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:01.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.961+0000 7ff4488af640 1 -- 172.21.15.32:0/3958602955 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff44406b820 msgr2=0x7ff44415bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:01.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.961+0000 7ff4488af640 1 --2- 172.21.15.32:0/3958602955 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff44406b820 0x7ff44415bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:01.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.961+0000 7ff4488af640 1 -- 172.21.15.32:0/3958602955 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff44415e9b0 msgr2=0x7ff4441f79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:01.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.961+0000 7ff4488af640 1 --2- 172.21.15.32:0/3958602955 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff44415e9b0 0x7ff4441f79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:01.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.961+0000 7ff4488af640 1 -- 172.21.15.32:0/3958602955 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff444160a40 con 0x7ff4440d4210 2024-04-24T23:00:01.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.961+0000 7ff4490b0640 1 --2- 172.21.15.32:0/3958602955 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff44406b820 0x7ff44415bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:00:01.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.961+0000 7ff4488af640 1 --2- 172.21.15.32:0/3958602955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4440d4210 0x7ff44415c2a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff43800c7f0 tx=0x7ff43800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:01.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.961+0000 7ff441ffb640 1 -- 172.21.15.32:0/3958602955 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff438015070 con 0x7ff4440d4210 2024-04-24T23:00:01.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.961+0000 7ff441ffb640 1 -- 172.21.15.32:0/3958602955 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff438007a30 con 0x7ff4440d4210 2024-04-24T23:00:01.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.961+0000 7ff441ffb640 1 -- 172.21.15.32:0/3958602955 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff438007e00 con 0x7ff4440d4210 2024-04-24T23:00:01.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.961+0000 7ff44b314640 1 -- 172.21.15.32:0/3958602955 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff444160d20 con 0x7ff4440d4210 2024-04-24T23:00:01.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.961+0000 7ff44b314640 1 -- 172.21.15.32:0/3958602955 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff44414a5d0 con 0x7ff4440d4210 2024-04-24T23:00:01.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.962+0000 7ff44b314640 1 -- 172.21.15.32:0/3958602955 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4441616c0 con 0x7ff4440d4210 2024-04-24T23:00:01.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.963+0000 7ff441ffb640 1 -- 172.21.15.32:0/3958602955 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff438018070 con 0x7ff4440d4210 2024-04-24T23:00:01.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.963+0000 7ff441ffb640 1 --2- 172.21.15.32:0/3958602955 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff43006be30 0x7ff43006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:01.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.963+0000 7ff441ffb640 1 -- 172.21.15.32:0/3958602955 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7ff43808ae50 con 0x7ff4440d4210 2024-04-24T23:00:01.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.963+0000 7ff4490b0640 1 --2- 172.21.15.32:0/3958602955 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff43006be30 0x7ff43006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:01.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.964+0000 7ff4490b0640 1 --2- 172.21.15.32:0/3958602955 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff43006be30 0x7ff43006e2f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff43c000f60 tx=0x7ff43c007ca0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:01.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:01.968+0000 7ff441ffb640 1 -- 172.21.15.32:0/3958602955 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff4441616c0 con 0x7ff4440d4210 2024-04-24T23:00:02.090 INFO:tasks.ceph.mds.b.smithi121.stderr:2024-04-24T23:00:02.089+0000 7f605de34640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T23:00:02.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.221+0000 7ff44b314640 1 -- 172.21.15.32:0/3958602955 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7ff444068430 con 0x7ff4440d4210 2024-04-24T23:00:02.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.222+0000 7ff441ffb640 1 -- 172.21.15.32:0/3958602955 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1575 (secure 0 0 0) 0x7ff438059800 con 0x7ff4440d4210 2024-04-24T23:00:02.223 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:00:02.223 INFO:teuthology.orchestra.run.smithi032.stdout:{"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":75,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":32,"num_osds":8,"num_up_osds":8,"osd_up_since":1713999532,"num_in_osds":8,"osd_in_since":1713999529,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":2129279,"bytes_avail":1597870721,"bytes_total":1600000000,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":13,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"b","status":"up:active","gid":5112}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-04-24T22:59:53.139154+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-24T23:00:02.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.224+0000 7ff44b314640 1 -- 172.21.15.32:0/3958602955 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff43006be30 msgr2=0x7ff43006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:02.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.224+0000 7ff44b314640 1 --2- 172.21.15.32:0/3958602955 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff43006be30 0x7ff43006e2f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff43c000f60 tx=0x7ff43c007ca0 comp rx=0 tx=0).stop 2024-04-24T23:00:02.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.224+0000 7ff44b314640 1 -- 172.21.15.32:0/3958602955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4440d4210 msgr2=0x7ff44415c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:02.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.224+0000 7ff44b314640 1 --2- 172.21.15.32:0/3958602955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4440d4210 0x7ff44415c2a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff43800c7f0 tx=0x7ff43800cbf0 comp rx=0 tx=0).stop 2024-04-24T23:00:02.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.224+0000 7ff44b314640 1 -- 172.21.15.32:0/3958602955 shutdown_connections 2024-04-24T23:00:02.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.224+0000 7ff44b314640 1 --2- 172.21.15.32:0/3958602955 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff43006be30 0x7ff43006e2f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:02.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.224+0000 7ff44b314640 1 --2- 172.21.15.32:0/3958602955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4440d4210 0x7ff44415c2a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:02.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.224+0000 7ff44b314640 1 --2- 172.21.15.32:0/3958602955 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff44415e9b0 0x7ff4441f79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:02.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.224+0000 7ff44b314640 1 --2- 172.21.15.32:0/3958602955 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff44406b820 0x7ff44415bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:02.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.224+0000 7ff44b314640 1 -- 172.21.15.32:0/3958602955 >> 172.21.15.32:0/3958602955 conn(0x7ff4440d11d0 msgr2=0x7ff44406b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:02.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.224+0000 7ff44b314640 1 -- 172.21.15.32:0/3958602955 shutdown_connections 2024-04-24T23:00:02.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.224+0000 7ff44b314640 1 -- 172.21.15.32:0/3958602955 wait complete. 2024-04-24T23:00:02.236 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-24T23:00:02.237 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:00:02.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.335+0000 7fe338e3f640 1 Processor -- start 2024-04-24T23:00:02.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.335+0000 7fe338e3f640 1 -- start start 2024-04-24T23:00:02.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.336+0000 7fe338e3f640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe334074210 0x7fe33406c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:02.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.336+0000 7fe338e3f640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe3340b0b80 0x7fe3340745f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:02.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.336+0000 7fe338e3f640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3340b0770 con 0x7fe3340ae140 2024-04-24T23:00:02.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.336+0000 7fe338e3f640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fe3340b1c20 con 0x7fe334074210 2024-04-24T23:00:02.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.336+0000 7fe338e3f640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fe334074b30 con 0x7fe3340b0b80 2024-04-24T23:00:02.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.336+0000 7fe33259c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe334074210 0x7fe33406c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:02.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.336+0000 7fe332d9d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe3340ae140 0x7fe3340b20b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39268/0 (socket says 172.21.15.32:39268) 2024-04-24T23:00:02.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.336+0000 7fe332d9d640 1 -- 172.21.15.32:0/4235651739 learned_addr learned my addr 172.21.15.32:0/4235651739 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:00:02.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.336+0000 7fe331d9b640 1 --2- 172.21.15.32:0/4235651739 >> v2:172.21.15.121:3300/0 conn(0x7fe3340b0b80 0x7fe3340745f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:02.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.336+0000 7fe33259c640 1 -- 172.21.15.32:0/4235651739 >> v2:172.21.15.121:3300/0 conn(0x7fe3340b0b80 msgr2=0x7fe3340745f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:02.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.336+0000 7fe33259c640 1 --2- 172.21.15.32:0/4235651739 >> v2:172.21.15.121:3300/0 conn(0x7fe3340b0b80 0x7fe3340745f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:02.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.336+0000 7fe33259c640 1 -- 172.21.15.32:0/4235651739 >> v1:172.21.15.32:6789/0 conn(0x7fe3340ae140 legacy=0x7fe3340b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:02.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.336+0000 7fe33159a640 1 -- 172.21.15.32:0/4235651739 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 441472001 0 0) 0x7fe3340b0770 con 0x7fe3340ae140 2024-04-24T23:00:02.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.337+0000 7fe33259c640 1 -- 172.21.15.32:0/4235651739 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3340b1010 con 0x7fe334074210 2024-04-24T23:00:02.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.337+0000 7fe331d9b640 1 --2- 172.21.15.32:0/4235651739 >> v2:172.21.15.121:3300/0 conn(0x7fe3340b0b80 0x7fe3340745f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:00:02.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.337+0000 7fe33259c640 1 --2- 172.21.15.32:0/4235651739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe334074210 0x7fe33406c480 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe3340722d0 tx=0x7fe32c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9221450f4421d81 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:02.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.337+0000 7fe33159a640 1 -- 172.21.15.32:0/4235651739 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32c037070 con 0x7fe334074210 2024-04-24T23:00:02.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.337+0000 7fe33159a640 1 -- 172.21.15.32:0/4235651739 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe32c032440 con 0x7fe334074210 2024-04-24T23:00:02.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.337+0000 7fe33159a640 1 -- 172.21.15.32:0/4235651739 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32c032810 con 0x7fe334074210 2024-04-24T23:00:02.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.338+0000 7fe338e3f640 1 -- 172.21.15.32:0/4235651739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe334074210 msgr2=0x7fe33406c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:02.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.338+0000 7fe338e3f640 1 --2- 172.21.15.32:0/4235651739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe334074210 0x7fe33406c480 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe3340722d0 tx=0x7fe32c02a1a0 comp rx=0 tx=0).stop 2024-04-24T23:00:02.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.338+0000 7fe338e3f640 1 -- 172.21.15.32:0/4235651739 shutdown_connections 2024-04-24T23:00:02.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.338+0000 7fe338e3f640 1 --2- 172.21.15.32:0/4235651739 >> v2:172.21.15.121:3300/0 conn(0x7fe3340b0b80 0x7fe3340745f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:02.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.338+0000 7fe338e3f640 1 --2- 172.21.15.32:0/4235651739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe334074210 0x7fe33406c480 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:02.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.338+0000 7fe338e3f640 1 -- 172.21.15.32:0/4235651739 >> 172.21.15.32:0/4235651739 conn(0x7fe3340b2e60 msgr2=0x7fe3340b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:02.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.338+0000 7fe338e3f640 1 -- 172.21.15.32:0/4235651739 shutdown_connections 2024-04-24T23:00:02.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.338+0000 7fe338e3f640 1 -- 172.21.15.32:0/4235651739 wait complete. 2024-04-24T23:00:02.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.339+0000 7fe338e3f640 1 Processor -- start 2024-04-24T23:00:02.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.339+0000 7fe338e3f640 1 -- start start 2024-04-24T23:00:02.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.340+0000 7fe338e3f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe334074210 0x7fe33406ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:02.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.340+0000 7fe338e3f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3340ae140 0x7fe3341f5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:02.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.340+0000 7fe338e3f640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3341f5c70 0x7fe3340c7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:02.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.340+0000 7fe338e3f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe3340b18b0 con 0x7fe3340ae140 2024-04-24T23:00:02.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.340+0000 7fe338e3f640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe3340b15d0 con 0x7fe334074210 2024-04-24T23:00:02.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.340+0000 7fe338e3f640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe3340b3ba0 con 0x7fe3341f5c70 2024-04-24T23:00:02.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.340+0000 7fe331d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3340ae140 0x7fe3341f5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:02.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.340+0000 7fe331d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3340ae140 0x7fe3341f5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60086/0 (socket says 172.21.15.32:60086) 2024-04-24T23:00:02.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.340+0000 7fe331d9b640 1 -- 172.21.15.32:0/3621155264 learned_addr learned my addr 172.21.15.32:0/3621155264 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:02.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.340+0000 7fe331d9b640 1 -- 172.21.15.32:0/3621155264 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3341f5c70 msgr2=0x7fe3340c7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:02.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.340+0000 7fe331d9b640 1 --2- 172.21.15.32:0/3621155264 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3341f5c70 0x7fe3340c7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:02.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.340+0000 7fe331d9b640 1 -- 172.21.15.32:0/3621155264 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe334074210 msgr2=0x7fe33406ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:02.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.340+0000 7fe33259c640 1 --2- 172.21.15.32:0/3621155264 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe334074210 0x7fe33406ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:02.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.340+0000 7fe331d9b640 1 --2- 172.21.15.32:0/3621155264 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe334074210 0x7fe33406ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:02.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.340+0000 7fe331d9b640 1 -- 172.21.15.32:0/3621155264 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3340c65a0 con 0x7fe3340ae140 2024-04-24T23:00:02.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.341+0000 7fe331d9b640 1 --2- 172.21.15.32:0/3621155264 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3340ae140 0x7fe3341f5730 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe31c00c820 tx=0x7fe31c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:02.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.341+0000 7fe32affd640 1 -- 172.21.15.32:0/3621155264 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31c015070 con 0x7fe3340ae140 2024-04-24T23:00:02.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.341+0000 7fe32affd640 1 -- 172.21.15.32:0/3621155264 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe31c007a30 con 0x7fe3340ae140 2024-04-24T23:00:02.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.341+0000 7fe338e3f640 1 -- 172.21.15.32:0/3621155264 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3340c6880 con 0x7fe3340ae140 2024-04-24T23:00:02.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.341+0000 7fe32affd640 1 -- 172.21.15.32:0/3621155264 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31c007e00 con 0x7fe3340ae140 2024-04-24T23:00:02.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.341+0000 7fe338e3f640 1 -- 172.21.15.32:0/3621155264 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3340c6e40 con 0x7fe3340ae140 2024-04-24T23:00:02.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.342+0000 7fe338e3f640 1 -- 172.21.15.32:0/3621155264 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3340c7040 con 0x7fe3340ae140 2024-04-24T23:00:02.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.342+0000 7fe32affd640 1 -- 172.21.15.32:0/3621155264 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe31c018070 con 0x7fe3340ae140 2024-04-24T23:00:02.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.343+0000 7fe32affd640 1 --2- 172.21.15.32:0/3621155264 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe31406be30 0x7fe31406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:02.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.343+0000 7fe32affd640 1 -- 172.21.15.32:0/3621155264 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fe31c08ae50 con 0x7fe3340ae140 2024-04-24T23:00:02.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.343+0000 7fe33259c640 1 --2- 172.21.15.32:0/3621155264 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe31406be30 0x7fe31406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:02.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.343+0000 7fe33259c640 1 --2- 172.21.15.32:0/3621155264 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe31406be30 0x7fe31406e2f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe32c034000 tx=0x7fe32c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:02.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.347+0000 7fe32affd640 1 -- 172.21.15.32:0/3621155264 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe3340c7040 con 0x7fe3340ae140 2024-04-24T23:00:02.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.530+0000 7fe338e3f640 1 -- 172.21.15.32:0/3621155264 --> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe33406b220 con 0x7fe31406be30 2024-04-24T23:00:02.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.536+0000 7fe32affd640 1 -- 172.21.15.32:0/3621155264 <== mgr.4108 v2:172.21.15.121:6832/1848157943 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+242215 (secure 0 0 0) 0x7fe33406b220 con 0x7fe31406be30 2024-04-24T23:00:02.540 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:00:02.541 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-24T23:00:02.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.539+0000 7fe338e3f640 1 -- 172.21.15.32:0/3621155264 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe31406be30 msgr2=0x7fe31406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:02.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.539+0000 7fe338e3f640 1 --2- 172.21.15.32:0/3621155264 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe31406be30 0x7fe31406e2f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe32c034000 tx=0x7fe32c02e000 comp rx=0 tx=0).stop 2024-04-24T23:00:02.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.539+0000 7fe338e3f640 1 -- 172.21.15.32:0/3621155264 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3340ae140 msgr2=0x7fe3341f5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:02.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.539+0000 7fe338e3f640 1 --2- 172.21.15.32:0/3621155264 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3340ae140 0x7fe3341f5730 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe31c00c820 tx=0x7fe31c00cc20 comp rx=0 tx=0).stop 2024-04-24T23:00:02.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.539+0000 7fe338e3f640 1 -- 172.21.15.32:0/3621155264 shutdown_connections 2024-04-24T23:00:02.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.539+0000 7fe338e3f640 1 --2- 172.21.15.32:0/3621155264 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe31406be30 0x7fe31406e2f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:02.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.539+0000 7fe338e3f640 1 --2- 172.21.15.32:0/3621155264 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3340ae140 0x7fe3341f5730 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:02.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.539+0000 7fe338e3f640 1 --2- 172.21.15.32:0/3621155264 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3341f5c70 0x7fe3340c7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:02.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.539+0000 7fe338e3f640 1 --2- 172.21.15.32:0/3621155264 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe334074210 0x7fe33406ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:02.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.539+0000 7fe338e3f640 1 -- 172.21.15.32:0/3621155264 >> 172.21.15.32:0/3621155264 conn(0x7fe3340b2e60 msgr2=0x7fe3340af670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:02.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.539+0000 7fe338e3f640 1 -- 172.21.15.32:0/3621155264 shutdown_connections 2024-04-24T23:00:02.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:02.539+0000 7fe338e3f640 1 -- 172.21.15.32:0/3621155264 wait complete. 2024-04-24T23:00:02.551 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":58,"stamp":"2024-04-24T23:00:01.140748+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2079,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560420,"statfs":{"total":1600000000,"available":1597870721,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.041342"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.108588+0000","last_change":"2024-04-24T22:59:57.108588+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.108588+0000","last_undegraded":"2024-04-24T22:59:57.108588+0000","last_fullsized":"2024-04-24T22:59:57.108588+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.108603+0000","last_change":"2024-04-24T22:59:57.108603+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.108603+0000","last_undegraded":"2024-04-24T22:59:57.108603+0000","last_fullsized":"2024-04-24T22:59:57.108603+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107901+0000","last_change":"2024-04-24T22:59:57.107901+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107901+0000","last_undegraded":"2024-04-24T22:59:57.107901+0000","last_fullsized":"2024-04-24T22:59:57.107901+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.112595+0000","last_change":"2024-04-24T22:59:57.112595+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.112595+0000","last_undegraded":"2024-04-24T22:59:57.112595+0000","last_fullsized":"2024-04-24T22:59:57.112595+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.108208+0000","last_change":"2024-04-24T22:59:57.108208+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.108208+0000","last_undegraded":"2024-04-24T22:59:57.108208+0000","last_fullsized":"2024-04-24T22:59:57.108208+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.109478+0000","last_change":"2024-04-24T22:59:57.109478+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.109478+0000","last_undegraded":"2024-04-24T22:59:57.109478+0000","last_fullsized":"2024-04-24T22:59:57.109478+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.109077+0000","last_change":"2024-04-24T22:59:57.109077+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.109077+0000","last_undegraded":"2024-04-24T22:59:57.109077+0000","last_fullsized":"2024-04-24T22:59:57.109077+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.110299+0000","last_change":"2024-04-24T22:59:57.110299+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.110299+0000","last_undegraded":"2024-04-24T22:59:57.110299+0000","last_fullsized":"2024-04-24T22:59:57.110299+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.108010+0000","last_change":"2024-04-24T22:59:57.108010+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.108010+0000","last_undegraded":"2024-04-24T22:59:57.108010+0000","last_fullsized":"2024-04-24T22:59:57.108010+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.115008+0000","last_change":"2024-04-24T22:59:57.115008+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.115008+0000","last_undegraded":"2024-04-24T22:59:57.115008+0000","last_fullsized":"2024-04-24T22:59:57.115008+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.109250+0000","last_change":"2024-04-24T22:59:57.109250+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.109250+0000","last_undegraded":"2024-04-24T22:59:57.109250+0000","last_fullsized":"2024-04-24T22:59:57.109250+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107645+0000","last_change":"2024-04-24T22:59:57.107645+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107645+0000","last_undegraded":"2024-04-24T22:59:57.107645+0000","last_fullsized":"2024-04-24T22:59:57.107645+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.109256+0000","last_change":"2024-04-24T22:59:57.109256+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.109256+0000","last_undegraded":"2024-04-24T22:59:57.109256+0000","last_fullsized":"2024-04-24T22:59:57.109256+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.111959+0000","last_change":"2024-04-24T22:59:57.111959+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.111959+0000","last_undegraded":"2024-04-24T22:59:57.111959+0000","last_fullsized":"2024-04-24T22:59:57.111959+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.111012+0000","last_change":"2024-04-24T22:59:57.111012+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.111012+0000","last_undegraded":"2024-04-24T22:59:57.111012+0000","last_fullsized":"2024-04-24T22:59:57.111012+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.112179+0000","last_change":"2024-04-24T22:59:57.112179+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.112179+0000","last_undegraded":"2024-04-24T22:59:57.112179+0000","last_fullsized":"2024-04-24T22:59:57.112179+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.109896+0000","last_change":"2024-04-24T22:59:57.109896+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.109896+0000","last_undegraded":"2024-04-24T22:59:57.109896+0000","last_fullsized":"2024-04-24T22:59:57.109896+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.111728+0000","last_change":"2024-04-24T22:59:57.111728+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.111728+0000","last_undegraded":"2024-04-24T22:59:57.111728+0000","last_fullsized":"2024-04-24T22:59:57.111728+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.109489+0000","last_change":"2024-04-24T22:59:57.109489+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.109489+0000","last_undegraded":"2024-04-24T22:59:57.109489+0000","last_fullsized":"2024-04-24T22:59:57.109489+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.108661+0000","last_change":"2024-04-24T22:59:57.108661+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.108661+0000","last_undegraded":"2024-04-24T22:59:57.108661+0000","last_fullsized":"2024-04-24T22:59:57.108661+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.109342+0000","last_change":"2024-04-24T22:59:57.109342+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.109342+0000","last_undegraded":"2024-04-24T22:59:57.109342+0000","last_fullsized":"2024-04-24T22:59:57.109342+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.108792+0000","last_change":"2024-04-24T22:59:57.108792+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.108792+0000","last_undegraded":"2024-04-24T22:59:57.108792+0000","last_fullsized":"2024-04-24T22:59:57.108792+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.108711+0000","last_change":"2024-04-24T22:59:57.108711+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.108711+0000","last_undegraded":"2024-04-24T22:59:57.108711+0000","last_fullsized":"2024-04-24T22:59:57.108711+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107354+0000","last_change":"2024-04-24T22:59:57.107354+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107354+0000","last_undegraded":"2024-04-24T22:59:57.107354+0000","last_fullsized":"2024-04-24T22:59:57.107354+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.114694+0000","last_change":"2024-04-24T22:59:57.114694+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.114694+0000","last_undegraded":"2024-04-24T22:59:57.114694+0000","last_fullsized":"2024-04-24T22:59:57.114694+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.110685+0000","last_change":"2024-04-24T22:59:57.110685+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.110685+0000","last_undegraded":"2024-04-24T22:59:57.110685+0000","last_fullsized":"2024-04-24T22:59:57.110685+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107448+0000","last_change":"2024-04-24T22:59:57.107448+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107448+0000","last_undegraded":"2024-04-24T22:59:57.107448+0000","last_fullsized":"2024-04-24T22:59:57.107448+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.109942+0000","last_change":"2024-04-24T22:59:57.109942+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.109942+0000","last_undegraded":"2024-04-24T22:59:57.109942+0000","last_fullsized":"2024-04-24T22:59:57.109942+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.108904+0000","last_change":"2024-04-24T22:59:57.108904+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.108904+0000","last_undegraded":"2024-04-24T22:59:57.108904+0000","last_fullsized":"2024-04-24T22:59:57.108904+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.110722+0000","last_change":"2024-04-24T22:59:57.110722+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.110722+0000","last_undegraded":"2024-04-24T22:59:57.110722+0000","last_fullsized":"2024-04-24T22:59:57.110722+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.108284+0000","last_change":"2024-04-24T22:59:57.108284+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.108284+0000","last_undegraded":"2024-04-24T22:59:57.108284+0000","last_fullsized":"2024-04-24T22:59:57.108284+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.109574+0000","last_change":"2024-04-24T22:59:57.109574+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.109574+0000","last_undegraded":"2024-04-24T22:59:57.109574+0000","last_fullsized":"2024-04-24T22:59:57.109574+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.108880+0000","last_change":"2024-04-24T22:59:57.108880+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.108880+0000","last_undegraded":"2024-04-24T22:59:57.108880+0000","last_fullsized":"2024-04-24T22:59:57.108880+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.115474+0000","last_change":"2024-04-24T22:59:57.114774+0000","last_active":"2024-04-24T22:59:57.115474+0000","last_peered":"2024-04-24T22:59:57.115474+0000","last_clean":"2024-04-24T22:59:57.115474+0000","last_became_active":"2024-04-24T22:59:57.114031+0000","last_became_peered":"2024-04-24T22:59:57.114031+0000","last_unstale":"2024-04-24T22:59:57.115474+0000","last_undegraded":"2024-04-24T22:59:57.115474+0000","last_fullsized":"2024-04-24T22:59:57.115474+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:02:17.915932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.108182+0000","last_change":"2024-04-24T22:59:57.108182+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.108182+0000","last_undegraded":"2024-04-24T22:59:57.108182+0000","last_fullsized":"2024-04-24T22:59:57.108182+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.118962+0000","last_change":"2024-04-24T22:59:57.118830+0000","last_active":"2024-04-24T22:59:57.118962+0000","last_peered":"2024-04-24T22:59:57.118962+0000","last_clean":"2024-04-24T22:59:57.118962+0000","last_became_active":"2024-04-24T22:59:57.118380+0000","last_became_peered":"2024-04-24T22:59:57.118380+0000","last_unstale":"2024-04-24T22:59:57.118962+0000","last_undegraded":"2024-04-24T22:59:57.118962+0000","last_fullsized":"2024-04-24T22:59:57.118962+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:34:34.824377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107978+0000","last_change":"2024-04-24T22:59:57.107978+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107978+0000","last_undegraded":"2024-04-24T22:59:57.107978+0000","last_fullsized":"2024-04-24T22:59:57.107978+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.113528+0000","last_change":"2024-04-24T22:59:57.113067+0000","last_active":"2024-04-24T22:59:57.113528+0000","last_peered":"2024-04-24T22:59:57.113528+0000","last_clean":"2024-04-24T22:59:57.113528+0000","last_became_active":"2024-04-24T22:59:57.112774+0000","last_became_peered":"2024-04-24T22:59:57.112774+0000","last_unstale":"2024-04-24T22:59:57.113528+0000","last_undegraded":"2024-04-24T22:59:57.113528+0000","last_fullsized":"2024-04-24T22:59:57.113528+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:09.897525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.114355+0000","last_change":"2024-04-24T22:59:57.114355+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.114355+0000","last_undegraded":"2024-04-24T22:59:57.114355+0000","last_fullsized":"2024-04-24T22:59:57.114355+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.118693+0000","last_change":"2024-04-24T22:59:57.118605+0000","last_active":"2024-04-24T22:59:57.118693+0000","last_peered":"2024-04-24T22:59:57.118693+0000","last_clean":"2024-04-24T22:59:57.118693+0000","last_became_active":"2024-04-24T22:59:57.118271+0000","last_became_peered":"2024-04-24T22:59:57.118271+0000","last_unstale":"2024-04-24T22:59:57.118693+0000","last_undegraded":"2024-04-24T22:59:57.118693+0000","last_fullsized":"2024-04-24T22:59:57.118693+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:04:12.275266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107238+0000","last_change":"2024-04-24T22:59:57.107238+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107238+0000","last_undegraded":"2024-04-24T22:59:57.107238+0000","last_fullsized":"2024-04-24T22:59:57.107238+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.109905+0000","last_change":"2024-04-24T22:59:57.109569+0000","last_active":"2024-04-24T22:59:57.109905+0000","last_peered":"2024-04-24T22:59:57.109905+0000","last_clean":"2024-04-24T22:59:57.109905+0000","last_became_active":"2024-04-24T22:59:57.109350+0000","last_became_peered":"2024-04-24T22:59:57.109350+0000","last_unstale":"2024-04-24T22:59:57.109905+0000","last_undegraded":"2024-04-24T22:59:57.109905+0000","last_fullsized":"2024-04-24T22:59:57.109905+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:54.689993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107517+0000","last_change":"2024-04-24T22:59:57.107517+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107517+0000","last_undegraded":"2024-04-24T22:59:57.107517+0000","last_fullsized":"2024-04-24T22:59:57.107517+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.113519+0000","last_change":"2024-04-24T22:59:57.113397+0000","last_active":"2024-04-24T22:59:57.113519+0000","last_peered":"2024-04-24T22:59:57.113519+0000","last_clean":"2024-04-24T22:59:57.113519+0000","last_became_active":"2024-04-24T22:59:57.113167+0000","last_became_peered":"2024-04-24T22:59:57.113167+0000","last_unstale":"2024-04-24T22:59:57.113519+0000","last_undegraded":"2024-04-24T22:59:57.113519+0000","last_fullsized":"2024-04-24T22:59:57.113519+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:29:04.934394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107218+0000","last_change":"2024-04-24T22:59:57.107218+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107218+0000","last_undegraded":"2024-04-24T22:59:57.107218+0000","last_fullsized":"2024-04-24T22:59:57.107218+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.116661+0000","last_change":"2024-04-24T22:59:57.116514+0000","last_active":"2024-04-24T22:59:57.116661+0000","last_peered":"2024-04-24T22:59:57.116661+0000","last_clean":"2024-04-24T22:59:57.116661+0000","last_became_active":"2024-04-24T22:59:57.116259+0000","last_became_peered":"2024-04-24T22:59:57.116259+0000","last_unstale":"2024-04-24T22:59:57.116661+0000","last_undegraded":"2024-04-24T22:59:57.116661+0000","last_fullsized":"2024-04-24T22:59:57.116661+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:43:58.559759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.112898+0000","last_change":"2024-04-24T22:59:57.112898+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.112898+0000","last_undegraded":"2024-04-24T22:59:57.112898+0000","last_fullsized":"2024-04-24T22:59:57.112898+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.117357+0000","last_change":"2024-04-24T22:59:57.117209+0000","last_active":"2024-04-24T22:59:57.117357+0000","last_peered":"2024-04-24T22:59:57.117357+0000","last_clean":"2024-04-24T22:59:57.117357+0000","last_became_active":"2024-04-24T22:59:57.116968+0000","last_became_peered":"2024-04-24T22:59:57.116968+0000","last_unstale":"2024-04-24T22:59:57.117357+0000","last_undegraded":"2024-04-24T22:59:57.117357+0000","last_fullsized":"2024-04-24T22:59:57.117357+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:44:17.871555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.109086+0000","last_change":"2024-04-24T22:59:57.109086+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.109086+0000","last_undegraded":"2024-04-24T22:59:57.109086+0000","last_fullsized":"2024-04-24T22:59:57.109086+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.106993+0000","last_change":"2024-04-24T22:59:57.106612+0000","last_active":"2024-04-24T22:59:57.106993+0000","last_peered":"2024-04-24T22:59:57.106993+0000","last_clean":"2024-04-24T22:59:57.106993+0000","last_became_active":"2024-04-24T22:59:57.106350+0000","last_became_peered":"2024-04-24T22:59:57.106350+0000","last_unstale":"2024-04-24T22:59:57.106993+0000","last_undegraded":"2024-04-24T22:59:57.106993+0000","last_fullsized":"2024-04-24T22:59:57.106993+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:45:14.576793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.112099+0000","last_change":"2024-04-24T22:59:57.112099+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.112099+0000","last_undegraded":"2024-04-24T22:59:57.112099+0000","last_fullsized":"2024-04-24T22:59:57.112099+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.115220+0000","last_change":"2024-04-24T22:59:57.115098+0000","last_active":"2024-04-24T22:59:57.115220+0000","last_peered":"2024-04-24T22:59:57.115220+0000","last_clean":"2024-04-24T22:59:57.115220+0000","last_became_active":"2024-04-24T22:59:57.114866+0000","last_became_peered":"2024-04-24T22:59:57.114866+0000","last_unstale":"2024-04-24T22:59:57.115220+0000","last_undegraded":"2024-04-24T22:59:57.115220+0000","last_fullsized":"2024-04-24T22:59:57.115220+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:27:09.165254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.111004+0000","last_change":"2024-04-24T22:59:57.111004+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.111004+0000","last_undegraded":"2024-04-24T22:59:57.111004+0000","last_fullsized":"2024-04-24T22:59:57.111004+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.115493+0000","last_change":"2024-04-24T22:59:57.115370+0000","last_active":"2024-04-24T22:59:57.115493+0000","last_peered":"2024-04-24T22:59:57.115493+0000","last_clean":"2024-04-24T22:59:57.115493+0000","last_became_active":"2024-04-24T22:59:57.115147+0000","last_became_peered":"2024-04-24T22:59:57.115147+0000","last_unstale":"2024-04-24T22:59:57.115493+0000","last_undegraded":"2024-04-24T22:59:57.115493+0000","last_fullsized":"2024-04-24T22:59:57.115493+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:23:03.727015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.108512+0000","last_change":"2024-04-24T22:59:57.108512+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.108512+0000","last_undegraded":"2024-04-24T22:59:57.108512+0000","last_fullsized":"2024-04-24T22:59:57.108512+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.118330+0000","last_change":"2024-04-24T22:59:57.118202+0000","last_active":"2024-04-24T22:59:57.118330+0000","last_peered":"2024-04-24T22:59:57.118330+0000","last_clean":"2024-04-24T22:59:57.118330+0000","last_became_active":"2024-04-24T22:59:57.117946+0000","last_became_peered":"2024-04-24T22:59:57.117946+0000","last_unstale":"2024-04-24T22:59:57.118330+0000","last_undegraded":"2024-04-24T22:59:57.118330+0000","last_fullsized":"2024-04-24T22:59:57.118330+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:37:52.953926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107500+0000","last_change":"2024-04-24T22:59:57.107500+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107500+0000","last_undegraded":"2024-04-24T22:59:57.107500+0000","last_fullsized":"2024-04-24T22:59:57.107500+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.112009+0000","last_change":"2024-04-24T22:59:57.111899+0000","last_active":"2024-04-24T22:59:57.112009+0000","last_peered":"2024-04-24T22:59:57.112009+0000","last_clean":"2024-04-24T22:59:57.112009+0000","last_became_active":"2024-04-24T22:59:57.111632+0000","last_became_peered":"2024-04-24T22:59:57.111632+0000","last_unstale":"2024-04-24T22:59:57.112009+0000","last_undegraded":"2024-04-24T22:59:57.112009+0000","last_fullsized":"2024-04-24T22:59:57.112009+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:01.204587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.114010+0000","last_change":"2024-04-24T22:59:57.114010+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.114010+0000","last_undegraded":"2024-04-24T22:59:57.114010+0000","last_fullsized":"2024-04-24T22:59:57.114010+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.116958+0000","last_change":"2024-04-24T22:59:57.116842+0000","last_active":"2024-04-24T22:59:57.116958+0000","last_peered":"2024-04-24T22:59:57.116958+0000","last_clean":"2024-04-24T22:59:57.116958+0000","last_became_active":"2024-04-24T22:59:57.116577+0000","last_became_peered":"2024-04-24T22:59:57.116577+0000","last_unstale":"2024-04-24T22:59:57.116958+0000","last_undegraded":"2024-04-24T22:59:57.116958+0000","last_fullsized":"2024-04-24T22:59:57.116958+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:29.595256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107852+0000","last_change":"2024-04-24T22:59:57.107852+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107852+0000","last_undegraded":"2024-04-24T22:59:57.107852+0000","last_fullsized":"2024-04-24T22:59:57.107852+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.115153+0000","last_change":"2024-04-24T22:59:57.113832+0000","last_active":"2024-04-24T22:59:57.115153+0000","last_peered":"2024-04-24T22:59:57.115153+0000","last_clean":"2024-04-24T22:59:57.115153+0000","last_became_active":"2024-04-24T22:59:57.113610+0000","last_became_peered":"2024-04-24T22:59:57.113610+0000","last_unstale":"2024-04-24T22:59:57.115153+0000","last_undegraded":"2024-04-24T22:59:57.115153+0000","last_fullsized":"2024-04-24T22:59:57.115153+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:49:50.026489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.108615+0000","last_change":"2024-04-24T22:59:57.108615+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.108615+0000","last_undegraded":"2024-04-24T22:59:57.108615+0000","last_fullsized":"2024-04-24T22:59:57.108615+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.116358+0000","last_change":"2024-04-24T22:59:57.116182+0000","last_active":"2024-04-24T22:59:57.116358+0000","last_peered":"2024-04-24T22:59:57.116358+0000","last_clean":"2024-04-24T22:59:57.116358+0000","last_became_active":"2024-04-24T22:59:57.115952+0000","last_became_peered":"2024-04-24T22:59:57.115952+0000","last_unstale":"2024-04-24T22:59:57.116358+0000","last_undegraded":"2024-04-24T22:59:57.116358+0000","last_fullsized":"2024-04-24T22:59:57.116358+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:20:34.273911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107248+0000","last_change":"2024-04-24T22:59:57.107248+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107248+0000","last_undegraded":"2024-04-24T22:59:57.107248+0000","last_fullsized":"2024-04-24T22:59:57.107248+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.119286+0000","last_change":"2024-04-24T22:59:57.119183+0000","last_active":"2024-04-24T22:59:57.119286+0000","last_peered":"2024-04-24T22:59:57.119286+0000","last_clean":"2024-04-24T22:59:57.119286+0000","last_became_active":"2024-04-24T22:59:57.118911+0000","last_became_peered":"2024-04-24T22:59:57.118911+0000","last_unstale":"2024-04-24T22:59:57.119286+0000","last_undegraded":"2024-04-24T22:59:57.119286+0000","last_fullsized":"2024-04-24T22:59:57.119286+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:28:06.461401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.106893+0000","last_change":"2024-04-24T22:59:57.106893+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.106893+0000","last_undegraded":"2024-04-24T22:59:57.106893+0000","last_fullsized":"2024-04-24T22:59:57.106893+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.115503+0000","last_change":"2024-04-24T22:59:57.115384+0000","last_active":"2024-04-24T22:59:57.115503+0000","last_peered":"2024-04-24T22:59:57.115503+0000","last_clean":"2024-04-24T22:59:57.115503+0000","last_became_active":"2024-04-24T22:59:57.115129+0000","last_became_peered":"2024-04-24T22:59:57.115129+0000","last_unstale":"2024-04-24T22:59:57.115503+0000","last_undegraded":"2024-04-24T22:59:57.115503+0000","last_fullsized":"2024-04-24T22:59:57.115503+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:13:57.006308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107117+0000","last_change":"2024-04-24T22:59:57.107117+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107117+0000","last_undegraded":"2024-04-24T22:59:57.107117+0000","last_fullsized":"2024-04-24T22:59:57.107117+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.114295+0000","last_change":"2024-04-24T22:59:57.113034+0000","last_active":"2024-04-24T22:59:57.114295+0000","last_peered":"2024-04-24T22:59:57.114295+0000","last_clean":"2024-04-24T22:59:57.114295+0000","last_became_active":"2024-04-24T22:59:57.111752+0000","last_became_peered":"2024-04-24T22:59:57.111752+0000","last_unstale":"2024-04-24T22:59:57.114295+0000","last_undegraded":"2024-04-24T22:59:57.114295+0000","last_fullsized":"2024-04-24T22:59:57.114295+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:11:29.081634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.109090+0000","last_change":"2024-04-24T22:59:57.109090+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.109090+0000","last_undegraded":"2024-04-24T22:59:57.109090+0000","last_fullsized":"2024-04-24T22:59:57.109090+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.117410+0000","last_change":"2024-04-24T22:59:57.117253+0000","last_active":"2024-04-24T22:59:57.117410+0000","last_peered":"2024-04-24T22:59:57.117410+0000","last_clean":"2024-04-24T22:59:57.117410+0000","last_became_active":"2024-04-24T22:59:57.116974+0000","last_became_peered":"2024-04-24T22:59:57.116974+0000","last_unstale":"2024-04-24T22:59:57.117410+0000","last_undegraded":"2024-04-24T22:59:57.117410+0000","last_fullsized":"2024-04-24T22:59:57.117410+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:24.934926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107613+0000","last_change":"2024-04-24T22:59:57.107613+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107613+0000","last_undegraded":"2024-04-24T22:59:57.107613+0000","last_fullsized":"2024-04-24T22:59:57.107613+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.112446+0000","last_change":"2024-04-24T22:59:57.111772+0000","last_active":"2024-04-24T22:59:57.112446+0000","last_peered":"2024-04-24T22:59:57.112446+0000","last_clean":"2024-04-24T22:59:57.112446+0000","last_became_active":"2024-04-24T22:59:57.111452+0000","last_became_peered":"2024-04-24T22:59:57.111452+0000","last_unstale":"2024-04-24T22:59:57.112446+0000","last_undegraded":"2024-04-24T22:59:57.112446+0000","last_fullsized":"2024-04-24T22:59:57.112446+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:27.552548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107006+0000","last_change":"2024-04-24T22:59:57.107006+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107006+0000","last_undegraded":"2024-04-24T22:59:57.107006+0000","last_fullsized":"2024-04-24T22:59:57.107006+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.111880+0000","last_change":"2024-04-24T22:59:57.111232+0000","last_active":"2024-04-24T22:59:57.111880+0000","last_peered":"2024-04-24T22:59:57.111880+0000","last_clean":"2024-04-24T22:59:57.111880+0000","last_became_active":"2024-04-24T22:59:57.111010+0000","last_became_peered":"2024-04-24T22:59:57.111010+0000","last_unstale":"2024-04-24T22:59:57.111880+0000","last_undegraded":"2024-04-24T22:59:57.111880+0000","last_fullsized":"2024-04-24T22:59:57.111880+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:02:29.250634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107089+0000","last_change":"2024-04-24T22:59:57.107089+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107089+0000","last_undegraded":"2024-04-24T22:59:57.107089+0000","last_fullsized":"2024-04-24T22:59:57.107089+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.111016+0000","last_change":"2024-04-24T22:59:57.110898+0000","last_active":"2024-04-24T22:59:57.111016+0000","last_peered":"2024-04-24T22:59:57.111016+0000","last_clean":"2024-04-24T22:59:57.111016+0000","last_became_active":"2024-04-24T22:59:57.110665+0000","last_became_peered":"2024-04-24T22:59:57.110665+0000","last_unstale":"2024-04-24T22:59:57.111016+0000","last_undegraded":"2024-04-24T22:59:57.111016+0000","last_fullsized":"2024-04-24T22:59:57.111016+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:55.518131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.106526+0000","last_change":"2024-04-24T22:59:57.106526+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.106526+0000","last_undegraded":"2024-04-24T22:59:57.106526+0000","last_fullsized":"2024-04-24T22:59:57.106526+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.114739+0000","last_change":"2024-04-24T22:59:57.114417+0000","last_active":"2024-04-24T22:59:57.114739+0000","last_peered":"2024-04-24T22:59:57.114739+0000","last_clean":"2024-04-24T22:59:57.114739+0000","last_became_active":"2024-04-24T22:59:57.112771+0000","last_became_peered":"2024-04-24T22:59:57.112771+0000","last_unstale":"2024-04-24T22:59:57.114739+0000","last_undegraded":"2024-04-24T22:59:57.114739+0000","last_fullsized":"2024-04-24T22:59:57.114739+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:11:37.810064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.110252+0000","last_change":"2024-04-24T22:59:57.110252+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.110252+0000","last_undegraded":"2024-04-24T22:59:57.110252+0000","last_fullsized":"2024-04-24T22:59:57.110252+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.115885+0000","last_change":"2024-04-24T22:59:57.115765+0000","last_active":"2024-04-24T22:59:57.115885+0000","last_peered":"2024-04-24T22:59:57.115885+0000","last_clean":"2024-04-24T22:59:57.115885+0000","last_became_active":"2024-04-24T22:59:57.115056+0000","last_became_peered":"2024-04-24T22:59:57.115056+0000","last_unstale":"2024-04-24T22:59:57.115885+0000","last_undegraded":"2024-04-24T22:59:57.115885+0000","last_fullsized":"2024-04-24T22:59:57.115885+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:18:11.299169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.106865+0000","last_change":"2024-04-24T22:59:57.106865+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.106865+0000","last_undegraded":"2024-04-24T22:59:57.106865+0000","last_fullsized":"2024-04-24T22:59:57.106865+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.113678+0000","last_change":"2024-04-24T22:59:57.113564+0000","last_active":"2024-04-24T22:59:57.113678+0000","last_peered":"2024-04-24T22:59:57.113678+0000","last_clean":"2024-04-24T22:59:57.113678+0000","last_became_active":"2024-04-24T22:59:57.113292+0000","last_became_peered":"2024-04-24T22:59:57.113292+0000","last_unstale":"2024-04-24T22:59:57.113678+0000","last_undegraded":"2024-04-24T22:59:57.113678+0000","last_fullsized":"2024-04-24T22:59:57.113678+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:05:41.614373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.109161+0000","last_change":"2024-04-24T22:59:57.109161+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.109161+0000","last_undegraded":"2024-04-24T22:59:57.109161+0000","last_fullsized":"2024-04-24T22:59:57.109161+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.112241+0000","last_change":"2024-04-24T22:59:57.111930+0000","last_active":"2024-04-24T22:59:57.112241+0000","last_peered":"2024-04-24T22:59:57.112241+0000","last_clean":"2024-04-24T22:59:57.112241+0000","last_became_active":"2024-04-24T22:59:57.111632+0000","last_became_peered":"2024-04-24T22:59:57.111632+0000","last_unstale":"2024-04-24T22:59:57.112241+0000","last_undegraded":"2024-04-24T22:59:57.112241+0000","last_fullsized":"2024-04-24T22:59:57.112241+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:57.770833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.110730+0000","last_change":"2024-04-24T22:59:57.110730+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.110730+0000","last_undegraded":"2024-04-24T22:59:57.110730+0000","last_fullsized":"2024-04-24T22:59:57.110730+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.115239+0000","last_change":"2024-04-24T22:59:57.115123+0000","last_active":"2024-04-24T22:59:57.115239+0000","last_peered":"2024-04-24T22:59:57.115239+0000","last_clean":"2024-04-24T22:59:57.115239+0000","last_became_active":"2024-04-24T22:59:57.114845+0000","last_became_peered":"2024-04-24T22:59:57.114845+0000","last_unstale":"2024-04-24T22:59:57.115239+0000","last_undegraded":"2024-04-24T22:59:57.115239+0000","last_fullsized":"2024-04-24T22:59:57.115239+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:31.884038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.111780+0000","last_change":"2024-04-24T22:59:57.111780+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.111780+0000","last_undegraded":"2024-04-24T22:59:57.111780+0000","last_fullsized":"2024-04-24T22:59:57.111780+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.112631+0000","last_change":"2024-04-24T22:59:57.112511+0000","last_active":"2024-04-24T22:59:57.112631+0000","last_peered":"2024-04-24T22:59:57.112631+0000","last_clean":"2024-04-24T22:59:57.112631+0000","last_became_active":"2024-04-24T22:59:57.112210+0000","last_became_peered":"2024-04-24T22:59:57.112210+0000","last_unstale":"2024-04-24T22:59:57.112631+0000","last_undegraded":"2024-04-24T22:59:57.112631+0000","last_fullsized":"2024-04-24T22:59:57.112631+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:04.204693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.112288+0000","last_change":"2024-04-24T22:59:57.112288+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.112288+0000","last_undegraded":"2024-04-24T22:59:57.112288+0000","last_fullsized":"2024-04-24T22:59:57.112288+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.111354+0000","last_change":"2024-04-24T22:59:57.111232+0000","last_active":"2024-04-24T22:59:57.111354+0000","last_peered":"2024-04-24T22:59:57.111354+0000","last_clean":"2024-04-24T22:59:57.111354+0000","last_became_active":"2024-04-24T22:59:57.110997+0000","last_became_peered":"2024-04-24T22:59:57.110997+0000","last_unstale":"2024-04-24T22:59:57.111354+0000","last_undegraded":"2024-04-24T22:59:57.111354+0000","last_fullsized":"2024-04-24T22:59:57.111354+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:56:07.604519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.107025+0000","last_change":"2024-04-24T22:59:57.107025+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.107025+0000","last_undegraded":"2024-04-24T22:59:57.107025+0000","last_fullsized":"2024-04-24T22:59:57.107025+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.118210+0000","last_change":"2024-04-24T22:59:57.118085+0000","last_active":"2024-04-24T22:59:57.118210+0000","last_peered":"2024-04-24T22:59:57.118210+0000","last_clean":"2024-04-24T22:59:57.118210+0000","last_became_active":"2024-04-24T22:59:57.116287+0000","last_became_peered":"2024-04-24T22:59:57.116287+0000","last_unstale":"2024-04-24T22:59:57.118210+0000","last_undegraded":"2024-04-24T22:59:57.118210+0000","last_fullsized":"2024-04-24T22:59:57.118210+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:11:38.330427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T22:59:57.110417+0000","last_change":"2024-04-24T22:59:57.110417+0000","last_active":"2024-04-24T22:59:57.098836+0000","last_peered":"2024-04-24T22:59:57.098836+0000","last_clean":"2024-04-24T22:59:57.098836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T22:59:57.110417+0000","last_undegraded":"2024-04-24T22:59:57.110417+0000","last_fullsized":"2024-04-24T22:59:57.110417+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.112334+0000","last_change":"2024-04-24T22:59:57.110803+0000","last_active":"2024-04-24T22:59:57.112334+0000","last_peered":"2024-04-24T22:59:57.112334+0000","last_clean":"2024-04-24T22:59:57.112334+0000","last_became_active":"2024-04-24T22:59:57.110572+0000","last_became_peered":"2024-04-24T22:59:57.110572+0000","last_unstale":"2024-04-24T22:59:57.112334+0000","last_undegraded":"2024-04-24T22:59:57.112334+0000","last_fullsized":"2024-04-24T22:59:57.112334+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:06:59.730529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":186,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T22:59:57.105292+0000","last_change":"2024-04-24T22:58:55.138810+0000","last_active":"2024-04-24T22:59:57.105292+0000","last_peered":"2024-04-24T22:59:57.105292+0000","last_clean":"2024-04-24T22:59:57.105292+0000","last_became_active":"2024-04-24T22:58:55.138627+0000","last_became_peered":"2024-04-24T22:58:55.138627+0000","last_unstale":"2024-04-24T22:59:57.105292+0000","last_undegraded":"2024-04-24T22:59:57.105292+0000","last_fullsized":"2024-04-24T22:59:57.105292+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_clean_scrub_stamp":"2024-04-24T22:58:54.127235+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:24.010002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":116,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195196,"statfs":{"total":200000000,"available":199880705,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":116,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195196,"statfs":{"total":200000000,"available":199880705,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":693,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194619,"statfs":{"total":200000000,"available":199290337,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":116,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195196,"statfs":{"total":200000000,"available":199880705,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":116,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195196,"statfs":{"total":200000000,"available":199880705,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":116,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195196,"statfs":{"total":200000000,"available":199880705,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":687,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194625,"statfs":{"total":200000000,"available":199296154,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":116,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195196,"statfs":{"total":200000000,"available":199880705,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-24T23:00:05.556 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:00:05.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.667+0000 7fc1065b0640 1 Processor -- start 2024-04-24T23:00:05.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.667+0000 7fc1065b0640 1 -- start start 2024-04-24T23:00:05.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.668+0000 7fc1065b0640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc1000b5af0 0x7fc1000ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:05.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.668+0000 7fc1065b0640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc100162e10 0x7fc10015ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:05.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.668+0000 7fc1065b0640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1000cd270 con 0x7fc100162320 2024-04-24T23:00:05.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.668+0000 7fc1065b0640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fc1000b5660 con 0x7fc1000b5af0 2024-04-24T23:00:05.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.668+0000 7fc1065b0640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc100071610 con 0x7fc100162e10 2024-04-24T23:00:05.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.668+0000 7fc0ff7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc1000b5af0 0x7fc1000ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:05.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.668+0000 7fc0fffff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc100162320 0x7fc100162700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39272/0 (socket says 172.21.15.32:39272) 2024-04-24T23:00:05.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.668+0000 7fc0fffff640 1 -- 172.21.15.32:0/1128042237 learned_addr learned my addr 172.21.15.32:0/1128042237 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:00:05.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.668+0000 7fc104b4d640 1 --2- 172.21.15.32:0/1128042237 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc100162e10 0x7fc10015ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:05.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.668+0000 7fc0ff7fe640 1 -- 172.21.15.32:0/1128042237 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc100162e10 msgr2=0x7fc10015ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:05.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.668+0000 7fc0ff7fe640 1 --2- 172.21.15.32:0/1128042237 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc100162e10 0x7fc10015ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:05.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.668+0000 7fc0ff7fe640 1 -- 172.21.15.32:0/1128042237 >> v1:172.21.15.32:6789/0 conn(0x7fc100162320 legacy=0x7fc100162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:05.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.669+0000 7fc0feffd640 1 -- 172.21.15.32:0/1128042237 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 804276216 0 0) 0x7fc1000cd270 con 0x7fc100162320 2024-04-24T23:00:05.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.669+0000 7fc0ff7fe640 1 -- 172.21.15.32:0/1128042237 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1000718f0 con 0x7fc1000b5af0 2024-04-24T23:00:05.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.669+0000 7fc104b4d640 1 --2- 172.21.15.32:0/1128042237 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc100162e10 0x7fc10015ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:00:05.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.669+0000 7fc0ff7fe640 1 --2- 172.21.15.32:0/1128042237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1000b5af0 0x7fc1000ccc90 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc0f00097a0 tx=0x7fc0f002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c4f743b57726a64 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:05.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.669+0000 7fc0feffd640 1 -- 172.21.15.32:0/1128042237 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f0037070 con 0x7fc1000b5af0 2024-04-24T23:00:05.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.669+0000 7fc0feffd640 1 -- 172.21.15.32:0/1128042237 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0f002ae50 con 0x7fc1000b5af0 2024-04-24T23:00:05.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.669+0000 7fc0feffd640 1 -- 172.21.15.32:0/1128042237 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f0032660 con 0x7fc1000b5af0 2024-04-24T23:00:05.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.670+0000 7fc1065b0640 1 -- 172.21.15.32:0/1128042237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1000b5af0 msgr2=0x7fc1000ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:05.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.670+0000 7fc1065b0640 1 --2- 172.21.15.32:0/1128042237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1000b5af0 0x7fc1000ccc90 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc0f00097a0 tx=0x7fc0f002a100 comp rx=0 tx=0).stop 2024-04-24T23:00:05.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.670+0000 7fc1065b0640 1 -- 172.21.15.32:0/1128042237 shutdown_connections 2024-04-24T23:00:05.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.670+0000 7fc1065b0640 1 --2- 172.21.15.32:0/1128042237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1000b5af0 0x7fc1000ccc90 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:05.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.670+0000 7fc1065b0640 1 --2- 172.21.15.32:0/1128042237 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc100162e10 0x7fc10015ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:05.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.670+0000 7fc1065b0640 1 -- 172.21.15.32:0/1128042237 >> 172.21.15.32:0/1128042237 conn(0x7fc1000d0ae0 msgr2=0x7fc1000d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:05.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.670+0000 7fc1065b0640 1 -- 172.21.15.32:0/1128042237 shutdown_connections 2024-04-24T23:00:05.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.670+0000 7fc1065b0640 1 -- 172.21.15.32:0/1128042237 wait complete. 2024-04-24T23:00:05.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.671+0000 7fc1065b0640 1 Processor -- start 2024-04-24T23:00:05.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.671+0000 7fc1065b0640 1 -- start start 2024-04-24T23:00:05.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.672+0000 7fc1065b0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1000b5af0 0x7fc10014a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:05.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.672+0000 7fc1065b0640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc100162320 0x7fc1000cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:05.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.672+0000 7fc1065b0640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc100162e10 0x7fc1000ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:05.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.672+0000 7fc1065b0640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc100071bd0 con 0x7fc1000b5af0 2024-04-24T23:00:05.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.672+0000 7fc1065b0640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc100067950 con 0x7fc100162e10 2024-04-24T23:00:05.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.672+0000 7fc1065b0640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc100071d50 con 0x7fc100162320 2024-04-24T23:00:05.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.672+0000 7fc0fffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1000b5af0 0x7fc10014a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:05.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.672+0000 7fc0fffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1000b5af0 0x7fc10014a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60114/0 (socket says 172.21.15.32:60114) 2024-04-24T23:00:05.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.672+0000 7fc0fffff640 1 -- 172.21.15.32:0/2919539509 learned_addr learned my addr 172.21.15.32:0/2919539509 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:05.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.672+0000 7fc104b4d640 1 --2- 172.21.15.32:0/2919539509 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc100162e10 0x7fc1000ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:05.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.672+0000 7fc0ff7fe640 1 --2- 172.21.15.32:0/2919539509 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc100162320 0x7fc1000cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:05.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.673+0000 7fc0fffff640 1 -- 172.21.15.32:0/2919539509 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc100162320 msgr2=0x7fc1000cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:05.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.673+0000 7fc0fffff640 1 --2- 172.21.15.32:0/2919539509 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc100162320 0x7fc1000cfd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:05.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.673+0000 7fc0fffff640 1 -- 172.21.15.32:0/2919539509 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc100162e10 msgr2=0x7fc1000ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:05.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.673+0000 7fc0fffff640 1 --2- 172.21.15.32:0/2919539509 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc100162e10 0x7fc1000ce2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:05.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.673+0000 7fc0fffff640 1 -- 172.21.15.32:0/2919539509 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc10014aae0 con 0x7fc1000b5af0 2024-04-24T23:00:05.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.673+0000 7fc0fffff640 1 --2- 172.21.15.32:0/2919539509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1000b5af0 0x7fc10014a5a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc0f400c500 tx=0x7fc0f400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:05.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.673+0000 7fc0fcff9640 1 -- 172.21.15.32:0/2919539509 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4018070 con 0x7fc1000b5af0 2024-04-24T23:00:05.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.673+0000 7fc0fcff9640 1 -- 172.21.15.32:0/2919539509 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0f4007580 con 0x7fc1000b5af0 2024-04-24T23:00:05.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.673+0000 7fc1065b0640 1 -- 172.21.15.32:0/2919539509 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1000d0290 con 0x7fc1000b5af0 2024-04-24T23:00:05.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.673+0000 7fc0fcff9640 1 -- 172.21.15.32:0/2919539509 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4004a20 con 0x7fc1000b5af0 2024-04-24T23:00:05.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.673+0000 7fc1065b0640 1 -- 172.21.15.32:0/2919539509 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1000cef90 con 0x7fc1000b5af0 2024-04-24T23:00:05.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.674+0000 7fc1065b0640 1 -- 172.21.15.32:0/2919539509 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc100068430 con 0x7fc1000b5af0 2024-04-24T23:00:05.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.675+0000 7fc0fcff9640 1 -- 172.21.15.32:0/2919539509 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc0f401b070 con 0x7fc1000b5af0 2024-04-24T23:00:05.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.675+0000 7fc0fcff9640 1 --2- 172.21.15.32:0/2919539509 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc0ec06be30 0x7fc0ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:05.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.675+0000 7fc0fcff9640 1 -- 172.21.15.32:0/2919539509 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fc0f408a130 con 0x7fc1000b5af0 2024-04-24T23:00:05.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.675+0000 7fc0ff7fe640 1 --2- 172.21.15.32:0/2919539509 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc0ec06be30 0x7fc0ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:05.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.676+0000 7fc0ff7fe640 1 --2- 172.21.15.32:0/2919539509 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc0ec06be30 0x7fc0ec06e2f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc0f0034000 tx=0x7fc0f002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:05.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.679+0000 7fc0fcff9640 1 -- 172.21.15.32:0/2919539509 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc10014aae0 con 0x7fc1000b5af0 2024-04-24T23:00:05.728 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:00:05.731 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:00:05.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.861+0000 7fc1065b0640 1 -- 172.21.15.32:0/2919539509 --> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc100067730 con 0x7fc0ec06be30 2024-04-24T23:00:05.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.867+0000 7fc0fcff9640 1 -- 172.21.15.32:0/2919539509 <== mgr.4108 v2:172.21.15.121:6832/1848157943 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247491 (secure 0 0 0) 0x7fc100067730 con 0x7fc0ec06be30 2024-04-24T23:00:05.872 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:00:05.873 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-24T23:00:05.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.870+0000 7fc1065b0640 1 -- 172.21.15.32:0/2919539509 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc0ec06be30 msgr2=0x7fc0ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:05.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.870+0000 7fc1065b0640 1 --2- 172.21.15.32:0/2919539509 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc0ec06be30 0x7fc0ec06e2f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc0f0034000 tx=0x7fc0f002e000 comp rx=0 tx=0).stop 2024-04-24T23:00:05.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.870+0000 7fc1065b0640 1 -- 172.21.15.32:0/2919539509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1000b5af0 msgr2=0x7fc10014a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:05.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.870+0000 7fc1065b0640 1 --2- 172.21.15.32:0/2919539509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1000b5af0 0x7fc10014a5a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc0f400c500 tx=0x7fc0f400c9c0 comp rx=0 tx=0).stop 2024-04-24T23:00:05.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.870+0000 7fc1065b0640 1 -- 172.21.15.32:0/2919539509 shutdown_connections 2024-04-24T23:00:05.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.870+0000 7fc1065b0640 1 --2- 172.21.15.32:0/2919539509 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc0ec06be30 0x7fc0ec06e2f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:05.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.870+0000 7fc1065b0640 1 --2- 172.21.15.32:0/2919539509 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc100162e10 0x7fc1000ce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:05.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.870+0000 7fc1065b0640 1 --2- 172.21.15.32:0/2919539509 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc100162320 0x7fc1000cfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:05.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.870+0000 7fc1065b0640 1 --2- 172.21.15.32:0/2919539509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1000b5af0 0x7fc10014a5a0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:05.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.870+0000 7fc1065b0640 1 -- 172.21.15.32:0/2919539509 >> 172.21.15.32:0/2919539509 conn(0x7fc1000d0ae0 msgr2=0x7fc1000b3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:05.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.870+0000 7fc1065b0640 1 -- 172.21.15.32:0/2919539509 shutdown_connections 2024-04-24T23:00:05.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:05.870+0000 7fc1065b0640 1 -- 172.21.15.32:0/2919539509 wait complete. 2024-04-24T23:00:05.883 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":60,"stamp":"2024-04-24T23:00:05.141552+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2240,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560259,"statfs":{"total":1600000000,"available":1597705284,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001807"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.118646+0000","last_change":"2024-04-24T22:59:58.121663+0000","last_active":"2024-04-24T22:59:59.118646+0000","last_peered":"2024-04-24T22:59:59.118646+0000","last_clean":"2024-04-24T22:59:59.118646+0000","last_became_active":"2024-04-24T22:59:58.121438+0000","last_became_peered":"2024-04-24T22:59:58.121438+0000","last_unstale":"2024-04-24T22:59:59.118646+0000","last_undegraded":"2024-04-24T22:59:59.118646+0000","last_fullsized":"2024-04-24T22:59:59.118646+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:00:44.995974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.118845+0000","last_change":"2024-04-24T22:59:58.121011+0000","last_active":"2024-04-24T22:59:59.118845+0000","last_peered":"2024-04-24T22:59:59.118845+0000","last_clean":"2024-04-24T22:59:59.118845+0000","last_became_active":"2024-04-24T22:59:58.120442+0000","last_became_peered":"2024-04-24T22:59:58.120442+0000","last_unstale":"2024-04-24T22:59:59.118845+0000","last_undegraded":"2024-04-24T22:59:59.118845+0000","last_fullsized":"2024-04-24T22:59:59.118845+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:08.378052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.909067+0000","last_change":"2024-04-24T22:59:58.111062+0000","last_active":"2024-04-24T22:59:59.909067+0000","last_peered":"2024-04-24T22:59:59.909067+0000","last_clean":"2024-04-24T22:59:59.909067+0000","last_became_active":"2024-04-24T22:59:58.110804+0000","last_became_peered":"2024-04-24T22:59:58.110804+0000","last_unstale":"2024-04-24T22:59:59.909067+0000","last_undegraded":"2024-04-24T22:59:59.909067+0000","last_fullsized":"2024-04-24T22:59:59.909067+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:16:36.702722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.909037+0000","last_change":"2024-04-24T22:59:58.125911+0000","last_active":"2024-04-24T22:59:59.909037+0000","last_peered":"2024-04-24T22:59:59.909037+0000","last_clean":"2024-04-24T22:59:59.909037+0000","last_became_active":"2024-04-24T22:59:58.122013+0000","last_became_peered":"2024-04-24T22:59:58.122013+0000","last_unstale":"2024-04-24T22:59:59.909037+0000","last_undegraded":"2024-04-24T22:59:59.909037+0000","last_fullsized":"2024-04-24T22:59:59.909037+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:38:19.186035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.118637+0000","last_change":"2024-04-24T22:59:58.114062+0000","last_active":"2024-04-24T22:59:59.118637+0000","last_peered":"2024-04-24T22:59:59.118637+0000","last_clean":"2024-04-24T22:59:59.118637+0000","last_became_active":"2024-04-24T22:59:58.113828+0000","last_became_peered":"2024-04-24T22:59:58.113828+0000","last_unstale":"2024-04-24T22:59:59.118637+0000","last_undegraded":"2024-04-24T22:59:59.118637+0000","last_fullsized":"2024-04-24T22:59:59.118637+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:44.404908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.908474+0000","last_change":"2024-04-24T22:59:58.116747+0000","last_active":"2024-04-24T22:59:59.908474+0000","last_peered":"2024-04-24T22:59:59.908474+0000","last_clean":"2024-04-24T22:59:59.908474+0000","last_became_active":"2024-04-24T22:59:58.116503+0000","last_became_peered":"2024-04-24T22:59:58.116503+0000","last_unstale":"2024-04-24T22:59:59.908474+0000","last_undegraded":"2024-04-24T22:59:59.908474+0000","last_fullsized":"2024-04-24T22:59:59.908474+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:09:07.136604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.118762+0000","last_change":"2024-04-24T22:59:58.114041+0000","last_active":"2024-04-24T22:59:59.118762+0000","last_peered":"2024-04-24T22:59:59.118762+0000","last_clean":"2024-04-24T22:59:59.118762+0000","last_became_active":"2024-04-24T22:59:58.113794+0000","last_became_peered":"2024-04-24T22:59:58.113794+0000","last_unstale":"2024-04-24T22:59:59.118762+0000","last_undegraded":"2024-04-24T22:59:59.118762+0000","last_fullsized":"2024-04-24T22:59:59.118762+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:16:08.327209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.118479+0000","last_change":"2024-04-24T22:59:58.118003+0000","last_active":"2024-04-24T22:59:59.118479+0000","last_peered":"2024-04-24T22:59:59.118479+0000","last_clean":"2024-04-24T22:59:59.118479+0000","last_became_active":"2024-04-24T22:59:58.117524+0000","last_became_peered":"2024-04-24T22:59:58.117524+0000","last_unstale":"2024-04-24T22:59:59.118479+0000","last_undegraded":"2024-04-24T22:59:59.118479+0000","last_fullsized":"2024-04-24T22:59:59.118479+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:47:28.326943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.908390+0000","last_change":"2024-04-24T22:59:58.124012+0000","last_active":"2024-04-24T22:59:59.908390+0000","last_peered":"2024-04-24T22:59:59.908390+0000","last_clean":"2024-04-24T22:59:59.908390+0000","last_became_active":"2024-04-24T22:59:58.123759+0000","last_became_peered":"2024-04-24T22:59:58.123759+0000","last_unstale":"2024-04-24T22:59:59.908390+0000","last_undegraded":"2024-04-24T22:59:59.908390+0000","last_fullsized":"2024-04-24T22:59:59.908390+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:08:20.358472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.908748+0000","last_change":"2024-04-24T22:59:58.125301+0000","last_active":"2024-04-24T22:59:59.908748+0000","last_peered":"2024-04-24T22:59:59.908748+0000","last_clean":"2024-04-24T22:59:59.908748+0000","last_became_active":"2024-04-24T22:59:58.125065+0000","last_became_peered":"2024-04-24T22:59:58.125065+0000","last_unstale":"2024-04-24T22:59:59.908748+0000","last_undegraded":"2024-04-24T22:59:59.908748+0000","last_fullsized":"2024-04-24T22:59:59.908748+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:17.654168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.908773+0000","last_change":"2024-04-24T22:59:58.127131+0000","last_active":"2024-04-24T22:59:59.908773+0000","last_peered":"2024-04-24T22:59:59.908773+0000","last_clean":"2024-04-24T22:59:59.908773+0000","last_became_active":"2024-04-24T22:59:58.126885+0000","last_became_peered":"2024-04-24T22:59:58.126885+0000","last_unstale":"2024-04-24T22:59:59.908773+0000","last_undegraded":"2024-04-24T22:59:59.908773+0000","last_fullsized":"2024-04-24T22:59:59.908773+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:19.958817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.908453+0000","last_change":"2024-04-24T22:59:58.117223+0000","last_active":"2024-04-24T22:59:59.908453+0000","last_peered":"2024-04-24T22:59:59.908453+0000","last_clean":"2024-04-24T22:59:59.908453+0000","last_became_active":"2024-04-24T22:59:58.116993+0000","last_became_peered":"2024-04-24T22:59:58.116993+0000","last_unstale":"2024-04-24T22:59:59.908453+0000","last_undegraded":"2024-04-24T22:59:59.908453+0000","last_fullsized":"2024-04-24T22:59:59.908453+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:22:07.032420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.119464+0000","last_change":"2024-04-24T22:59:58.122965+0000","last_active":"2024-04-24T22:59:59.119464+0000","last_peered":"2024-04-24T22:59:59.119464+0000","last_clean":"2024-04-24T22:59:59.119464+0000","last_became_active":"2024-04-24T22:59:58.122695+0000","last_became_peered":"2024-04-24T22:59:58.122695+0000","last_unstale":"2024-04-24T22:59:59.119464+0000","last_undegraded":"2024-04-24T22:59:59.119464+0000","last_fullsized":"2024-04-24T22:59:59.119464+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:12:39.035762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.909383+0000","last_change":"2024-04-24T22:59:58.126204+0000","last_active":"2024-04-24T22:59:59.909383+0000","last_peered":"2024-04-24T22:59:59.909383+0000","last_clean":"2024-04-24T22:59:59.909383+0000","last_became_active":"2024-04-24T22:59:58.122319+0000","last_became_peered":"2024-04-24T22:59:58.122319+0000","last_unstale":"2024-04-24T22:59:59.909383+0000","last_undegraded":"2024-04-24T22:59:59.909383+0000","last_fullsized":"2024-04-24T22:59:59.909383+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:21.340332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.909219+0000","last_change":"2024-04-24T22:59:58.122359+0000","last_active":"2024-04-24T22:59:59.909219+0000","last_peered":"2024-04-24T22:59:59.909219+0000","last_clean":"2024-04-24T22:59:59.909219+0000","last_became_active":"2024-04-24T22:59:58.121446+0000","last_became_peered":"2024-04-24T22:59:58.121446+0000","last_unstale":"2024-04-24T22:59:59.909219+0000","last_undegraded":"2024-04-24T22:59:59.909219+0000","last_fullsized":"2024-04-24T22:59:59.909219+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:20:08.283582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.908589+0000","last_change":"2024-04-24T22:59:58.123582+0000","last_active":"2024-04-24T22:59:59.908589+0000","last_peered":"2024-04-24T22:59:59.908589+0000","last_clean":"2024-04-24T22:59:59.908589+0000","last_became_active":"2024-04-24T22:59:58.123355+0000","last_became_peered":"2024-04-24T22:59:58.123355+0000","last_unstale":"2024-04-24T22:59:59.908589+0000","last_undegraded":"2024-04-24T22:59:59.908589+0000","last_fullsized":"2024-04-24T22:59:59.908589+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:04:47.817244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.908949+0000","last_change":"2024-04-24T22:59:58.115156+0000","last_active":"2024-04-24T22:59:59.908949+0000","last_peered":"2024-04-24T22:59:59.908949+0000","last_clean":"2024-04-24T22:59:59.908949+0000","last_became_active":"2024-04-24T22:59:58.113942+0000","last_became_peered":"2024-04-24T22:59:58.113942+0000","last_unstale":"2024-04-24T22:59:59.908949+0000","last_undegraded":"2024-04-24T22:59:59.908949+0000","last_fullsized":"2024-04-24T22:59:59.908949+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:07.583452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.908539+0000","last_change":"2024-04-24T22:59:58.115966+0000","last_active":"2024-04-24T22:59:59.908539+0000","last_peered":"2024-04-24T22:59:59.908539+0000","last_clean":"2024-04-24T22:59:59.908539+0000","last_became_active":"2024-04-24T22:59:58.115708+0000","last_became_peered":"2024-04-24T22:59:58.115708+0000","last_unstale":"2024-04-24T22:59:59.908539+0000","last_undegraded":"2024-04-24T22:59:59.908539+0000","last_fullsized":"2024-04-24T22:59:59.908539+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:13.890683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.908173+0000","last_change":"2024-04-24T22:59:58.116899+0000","last_active":"2024-04-24T22:59:59.908173+0000","last_peered":"2024-04-24T22:59:59.908173+0000","last_clean":"2024-04-24T22:59:59.908173+0000","last_became_active":"2024-04-24T22:59:58.116198+0000","last_became_peered":"2024-04-24T22:59:58.116198+0000","last_unstale":"2024-04-24T22:59:59.908173+0000","last_undegraded":"2024-04-24T22:59:59.908173+0000","last_fullsized":"2024-04-24T22:59:59.908173+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:29:07.482999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.119088+0000","last_change":"2024-04-24T22:59:58.123800+0000","last_active":"2024-04-24T22:59:59.119088+0000","last_peered":"2024-04-24T22:59:59.119088+0000","last_clean":"2024-04-24T22:59:59.119088+0000","last_became_active":"2024-04-24T22:59:58.123563+0000","last_became_peered":"2024-04-24T22:59:58.123563+0000","last_unstale":"2024-04-24T22:59:59.119088+0000","last_undegraded":"2024-04-24T22:59:59.119088+0000","last_fullsized":"2024-04-24T22:59:59.119088+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:46:01.697064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.908822+0000","last_change":"2024-04-24T22:59:58.120442+0000","last_active":"2024-04-24T22:59:59.908822+0000","last_peered":"2024-04-24T22:59:59.908822+0000","last_clean":"2024-04-24T22:59:59.908822+0000","last_became_active":"2024-04-24T22:59:58.120107+0000","last_became_peered":"2024-04-24T22:59:58.120107+0000","last_unstale":"2024-04-24T22:59:59.908822+0000","last_undegraded":"2024-04-24T22:59:59.908822+0000","last_fullsized":"2024-04-24T22:59:59.908822+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:33:21.125181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.118897+0000","last_change":"2024-04-24T22:59:58.120261+0000","last_active":"2024-04-24T22:59:59.118897+0000","last_peered":"2024-04-24T22:59:59.118897+0000","last_clean":"2024-04-24T22:59:59.118897+0000","last_became_active":"2024-04-24T22:59:58.119890+0000","last_became_peered":"2024-04-24T22:59:58.119890+0000","last_unstale":"2024-04-24T22:59:59.118897+0000","last_undegraded":"2024-04-24T22:59:59.118897+0000","last_fullsized":"2024-04-24T22:59:59.118897+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:12:19.837249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.119665+0000","last_change":"2024-04-24T22:59:58.114407+0000","last_active":"2024-04-24T22:59:59.119665+0000","last_peered":"2024-04-24T22:59:59.119665+0000","last_clean":"2024-04-24T22:59:59.119665+0000","last_became_active":"2024-04-24T22:59:58.114112+0000","last_became_peered":"2024-04-24T22:59:58.114112+0000","last_unstale":"2024-04-24T22:59:59.119665+0000","last_undegraded":"2024-04-24T22:59:59.119665+0000","last_fullsized":"2024-04-24T22:59:59.119665+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:17:00.303114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.118636+0000","last_change":"2024-04-24T22:59:58.127474+0000","last_active":"2024-04-24T22:59:59.118636+0000","last_peered":"2024-04-24T22:59:59.118636+0000","last_clean":"2024-04-24T22:59:59.118636+0000","last_became_active":"2024-04-24T22:59:58.127207+0000","last_became_peered":"2024-04-24T22:59:58.127207+0000","last_unstale":"2024-04-24T22:59:59.118636+0000","last_undegraded":"2024-04-24T22:59:59.118636+0000","last_fullsized":"2024-04-24T22:59:59.118636+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:52:48.401498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.908941+0000","last_change":"2024-04-24T22:59:58.124285+0000","last_active":"2024-04-24T22:59:59.908941+0000","last_peered":"2024-04-24T22:59:59.908941+0000","last_clean":"2024-04-24T22:59:59.908941+0000","last_became_active":"2024-04-24T22:59:58.123277+0000","last_became_peered":"2024-04-24T22:59:58.123277+0000","last_unstale":"2024-04-24T22:59:59.908941+0000","last_undegraded":"2024-04-24T22:59:59.908941+0000","last_fullsized":"2024-04-24T22:59:59.908941+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:47.656301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.909200+0000","last_change":"2024-04-24T22:59:58.129436+0000","last_active":"2024-04-24T22:59:59.909200+0000","last_peered":"2024-04-24T22:59:59.909200+0000","last_clean":"2024-04-24T22:59:59.909200+0000","last_became_active":"2024-04-24T22:59:58.128780+0000","last_became_peered":"2024-04-24T22:59:58.128780+0000","last_unstale":"2024-04-24T22:59:59.909200+0000","last_undegraded":"2024-04-24T22:59:59.909200+0000","last_fullsized":"2024-04-24T22:59:59.909200+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:13.118025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.118998+0000","last_change":"2024-04-24T22:59:58.121218+0000","last_active":"2024-04-24T22:59:59.118998+0000","last_peered":"2024-04-24T22:59:59.118998+0000","last_clean":"2024-04-24T22:59:59.118998+0000","last_became_active":"2024-04-24T22:59:58.120763+0000","last_became_peered":"2024-04-24T22:59:58.120763+0000","last_unstale":"2024-04-24T22:59:59.118998+0000","last_undegraded":"2024-04-24T22:59:59.118998+0000","last_fullsized":"2024-04-24T22:59:59.118998+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:02:51.498210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.118652+0000","last_change":"2024-04-24T22:59:58.117752+0000","last_active":"2024-04-24T22:59:59.118652+0000","last_peered":"2024-04-24T22:59:59.118652+0000","last_clean":"2024-04-24T22:59:59.118652+0000","last_became_active":"2024-04-24T22:59:58.116622+0000","last_became_peered":"2024-04-24T22:59:58.116622+0000","last_unstale":"2024-04-24T22:59:59.118652+0000","last_undegraded":"2024-04-24T22:59:59.118652+0000","last_fullsized":"2024-04-24T22:59:59.118652+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:01.888604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.908165+0000","last_change":"2024-04-24T22:59:58.127943+0000","last_active":"2024-04-24T22:59:59.908165+0000","last_peered":"2024-04-24T22:59:59.908165+0000","last_clean":"2024-04-24T22:59:59.908165+0000","last_became_active":"2024-04-24T22:59:58.127684+0000","last_became_peered":"2024-04-24T22:59:58.127684+0000","last_unstale":"2024-04-24T22:59:59.908165+0000","last_undegraded":"2024-04-24T22:59:59.908165+0000","last_fullsized":"2024-04-24T22:59:59.908165+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:36:06.687989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.909366+0000","last_change":"2024-04-24T22:59:58.122606+0000","last_active":"2024-04-24T22:59:59.909366+0000","last_peered":"2024-04-24T22:59:59.909366+0000","last_clean":"2024-04-24T22:59:59.909366+0000","last_became_active":"2024-04-24T22:59:58.121835+0000","last_became_peered":"2024-04-24T22:59:58.121835+0000","last_unstale":"2024-04-24T22:59:59.909366+0000","last_undegraded":"2024-04-24T22:59:59.909366+0000","last_fullsized":"2024-04-24T22:59:59.909366+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:10:52.967375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.118811+0000","last_change":"2024-04-24T22:59:58.116326+0000","last_active":"2024-04-24T22:59:59.118811+0000","last_peered":"2024-04-24T22:59:59.118811+0000","last_clean":"2024-04-24T22:59:59.118811+0000","last_became_active":"2024-04-24T22:59:58.116080+0000","last_became_peered":"2024-04-24T22:59:58.116080+0000","last_unstale":"2024-04-24T22:59:59.118811+0000","last_undegraded":"2024-04-24T22:59:59.118811+0000","last_fullsized":"2024-04-24T22:59:59.118811+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:55:43.591448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.909119+0000","last_change":"2024-04-24T22:59:58.115523+0000","last_active":"2024-04-24T22:59:59.909119+0000","last_peered":"2024-04-24T22:59:59.909119+0000","last_clean":"2024-04-24T22:59:59.909119+0000","last_became_active":"2024-04-24T22:59:58.114293+0000","last_became_peered":"2024-04-24T22:59:58.114293+0000","last_unstale":"2024-04-24T22:59:59.909119+0000","last_undegraded":"2024-04-24T22:59:59.909119+0000","last_fullsized":"2024-04-24T22:59:59.909119+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:54.553184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.119598+0000","last_change":"2024-04-24T22:59:58.114075+0000","last_active":"2024-04-24T22:59:59.119598+0000","last_peered":"2024-04-24T22:59:59.119598+0000","last_clean":"2024-04-24T22:59:59.119598+0000","last_became_active":"2024-04-24T22:59:58.113838+0000","last_became_peered":"2024-04-24T22:59:58.113838+0000","last_unstale":"2024-04-24T22:59:59.119598+0000","last_undegraded":"2024-04-24T22:59:59.119598+0000","last_fullsized":"2024-04-24T22:59:59.119598+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:32:55.064050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"31'4","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.908457+0000","last_change":"2024-04-24T22:59:57.114774+0000","last_active":"2024-04-24T22:59:59.908457+0000","last_peered":"2024-04-24T22:59:59.908457+0000","last_clean":"2024-04-24T22:59:59.908457+0000","last_became_active":"2024-04-24T22:59:57.114031+0000","last_became_peered":"2024-04-24T22:59:57.114031+0000","last_unstale":"2024-04-24T22:59:59.908457+0000","last_undegraded":"2024-04-24T22:59:59.908457+0000","last_fullsized":"2024-04-24T22:59:59.908457+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:26.715230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.118798+0000","last_change":"2024-04-24T22:59:58.111037+0000","last_active":"2024-04-24T22:59:59.118798+0000","last_peered":"2024-04-24T22:59:59.118798+0000","last_clean":"2024-04-24T22:59:59.118798+0000","last_became_active":"2024-04-24T22:59:58.110585+0000","last_became_peered":"2024-04-24T22:59:58.110585+0000","last_unstale":"2024-04-24T22:59:59.118798+0000","last_undegraded":"2024-04-24T22:59:59.118798+0000","last_fullsized":"2024-04-24T22:59:59.118798+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:20:52.820159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"31'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.909476+0000","last_change":"2024-04-24T22:59:57.118830+0000","last_active":"2024-04-24T22:59:59.909476+0000","last_peered":"2024-04-24T22:59:59.909476+0000","last_clean":"2024-04-24T22:59:59.909476+0000","last_became_active":"2024-04-24T22:59:57.118380+0000","last_became_peered":"2024-04-24T22:59:57.118380+0000","last_unstale":"2024-04-24T22:59:59.909476+0000","last_undegraded":"2024-04-24T22:59:59.909476+0000","last_fullsized":"2024-04-24T22:59:59.909476+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:32:14.813304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.118974+0000","last_change":"2024-04-24T22:59:58.128989+0000","last_active":"2024-04-24T22:59:59.118974+0000","last_peered":"2024-04-24T22:59:59.118974+0000","last_clean":"2024-04-24T22:59:59.118974+0000","last_became_active":"2024-04-24T22:59:58.128729+0000","last_became_peered":"2024-04-24T22:59:58.128729+0000","last_unstale":"2024-04-24T22:59:59.118974+0000","last_undegraded":"2024-04-24T22:59:59.118974+0000","last_fullsized":"2024-04-24T22:59:59.118974+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:48.701557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.908361+0000","last_change":"2024-04-24T22:59:57.113067+0000","last_active":"2024-04-24T22:59:59.908361+0000","last_peered":"2024-04-24T22:59:59.908361+0000","last_clean":"2024-04-24T22:59:59.908361+0000","last_became_active":"2024-04-24T22:59:57.112774+0000","last_became_peered":"2024-04-24T22:59:57.112774+0000","last_unstale":"2024-04-24T22:59:59.908361+0000","last_undegraded":"2024-04-24T22:59:59.908361+0000","last_fullsized":"2024-04-24T22:59:59.908361+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:03:49.921319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.909139+0000","last_change":"2024-04-24T22:59:58.123788+0000","last_active":"2024-04-24T22:59:59.909139+0000","last_peered":"2024-04-24T22:59:59.909139+0000","last_clean":"2024-04-24T22:59:59.909139+0000","last_became_active":"2024-04-24T22:59:58.122746+0000","last_became_peered":"2024-04-24T22:59:58.122746+0000","last_unstale":"2024-04-24T22:59:59.909139+0000","last_undegraded":"2024-04-24T22:59:59.909139+0000","last_fullsized":"2024-04-24T22:59:59.909139+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:18:39.104383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.908983+0000","last_change":"2024-04-24T22:59:57.118605+0000","last_active":"2024-04-24T22:59:59.908983+0000","last_peered":"2024-04-24T22:59:59.908983+0000","last_clean":"2024-04-24T22:59:59.908983+0000","last_became_active":"2024-04-24T22:59:57.118271+0000","last_became_peered":"2024-04-24T22:59:57.118271+0000","last_unstale":"2024-04-24T22:59:59.908983+0000","last_undegraded":"2024-04-24T22:59:59.908983+0000","last_fullsized":"2024-04-24T22:59:59.908983+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:26:21.517743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.909297+0000","last_change":"2024-04-24T22:59:58.124333+0000","last_active":"2024-04-24T22:59:59.909297+0000","last_peered":"2024-04-24T22:59:59.909297+0000","last_clean":"2024-04-24T22:59:59.909297+0000","last_became_active":"2024-04-24T22:59:58.124031+0000","last_became_peered":"2024-04-24T22:59:58.124031+0000","last_unstale":"2024-04-24T22:59:59.909297+0000","last_undegraded":"2024-04-24T22:59:59.909297+0000","last_fullsized":"2024-04-24T22:59:59.909297+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:37:25.868020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"31'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.119087+0000","last_change":"2024-04-24T22:59:57.109569+0000","last_active":"2024-04-24T22:59:59.119087+0000","last_peered":"2024-04-24T22:59:59.119087+0000","last_clean":"2024-04-24T22:59:59.119087+0000","last_became_active":"2024-04-24T22:59:57.109350+0000","last_became_peered":"2024-04-24T22:59:57.109350+0000","last_unstale":"2024-04-24T22:59:59.119087+0000","last_undegraded":"2024-04-24T22:59:59.119087+0000","last_fullsized":"2024-04-24T22:59:59.119087+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:29:28.145801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.119004+0000","last_change":"2024-04-24T22:59:58.116283+0000","last_active":"2024-04-24T22:59:59.119004+0000","last_peered":"2024-04-24T22:59:59.119004+0000","last_clean":"2024-04-24T22:59:59.119004+0000","last_became_active":"2024-04-24T22:59:58.115991+0000","last_became_peered":"2024-04-24T22:59:58.115991+0000","last_unstale":"2024-04-24T22:59:59.119004+0000","last_undegraded":"2024-04-24T22:59:59.119004+0000","last_fullsized":"2024-04-24T22:59:59.119004+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:41:28.677123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"31'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.119099+0000","last_change":"2024-04-24T22:59:57.113397+0000","last_active":"2024-04-24T22:59:59.119099+0000","last_peered":"2024-04-24T22:59:59.119099+0000","last_clean":"2024-04-24T22:59:59.119099+0000","last_became_active":"2024-04-24T22:59:57.113167+0000","last_became_peered":"2024-04-24T22:59:57.113167+0000","last_unstale":"2024-04-24T22:59:59.119099+0000","last_undegraded":"2024-04-24T22:59:59.119099+0000","last_fullsized":"2024-04-24T22:59:59.119099+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:50.500557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.908563+0000","last_change":"2024-04-24T22:59:58.123222+0000","last_active":"2024-04-24T22:59:59.908563+0000","last_peered":"2024-04-24T22:59:59.908563+0000","last_clean":"2024-04-24T22:59:59.908563+0000","last_became_active":"2024-04-24T22:59:58.122971+0000","last_became_peered":"2024-04-24T22:59:58.122971+0000","last_unstale":"2024-04-24T22:59:59.908563+0000","last_undegraded":"2024-04-24T22:59:59.908563+0000","last_fullsized":"2024-04-24T22:59:59.908563+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:11:47.825294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.908503+0000","last_change":"2024-04-24T22:59:57.116514+0000","last_active":"2024-04-24T22:59:59.908503+0000","last_peered":"2024-04-24T22:59:59.908503+0000","last_clean":"2024-04-24T22:59:59.908503+0000","last_became_active":"2024-04-24T22:59:57.116259+0000","last_became_peered":"2024-04-24T22:59:57.116259+0000","last_unstale":"2024-04-24T22:59:59.908503+0000","last_undegraded":"2024-04-24T22:59:59.908503+0000","last_fullsized":"2024-04-24T22:59:59.908503+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:50:12.673297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.908890+0000","last_change":"2024-04-24T22:59:58.126697+0000","last_active":"2024-04-24T22:59:59.908890+0000","last_peered":"2024-04-24T22:59:59.908890+0000","last_clean":"2024-04-24T22:59:59.908890+0000","last_became_active":"2024-04-24T22:59:58.124205+0000","last_became_peered":"2024-04-24T22:59:58.124205+0000","last_unstale":"2024-04-24T22:59:59.908890+0000","last_undegraded":"2024-04-24T22:59:59.908890+0000","last_fullsized":"2024-04-24T22:59:59.908890+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:16.440972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.118630+0000","last_change":"2024-04-24T22:59:57.117209+0000","last_active":"2024-04-24T22:59:59.118630+0000","last_peered":"2024-04-24T22:59:59.118630+0000","last_clean":"2024-04-24T22:59:59.118630+0000","last_became_active":"2024-04-24T22:59:57.116968+0000","last_became_peered":"2024-04-24T22:59:57.116968+0000","last_unstale":"2024-04-24T22:59:59.118630+0000","last_undegraded":"2024-04-24T22:59:59.118630+0000","last_fullsized":"2024-04-24T22:59:59.118630+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:26:43.518425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.909421+0000","last_change":"2024-04-24T22:59:58.118241+0000","last_active":"2024-04-24T22:59:59.909421+0000","last_peered":"2024-04-24T22:59:59.909421+0000","last_clean":"2024-04-24T22:59:59.909421+0000","last_became_active":"2024-04-24T22:59:58.118019+0000","last_became_peered":"2024-04-24T22:59:58.118019+0000","last_unstale":"2024-04-24T22:59:59.909421+0000","last_undegraded":"2024-04-24T22:59:59.909421+0000","last_fullsized":"2024-04-24T22:59:59.909421+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:55:00.615610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.118378+0000","last_change":"2024-04-24T22:59:57.106612+0000","last_active":"2024-04-24T22:59:59.118378+0000","last_peered":"2024-04-24T22:59:59.118378+0000","last_clean":"2024-04-24T22:59:59.118378+0000","last_became_active":"2024-04-24T22:59:57.106350+0000","last_became_peered":"2024-04-24T22:59:57.106350+0000","last_unstale":"2024-04-24T22:59:59.118378+0000","last_undegraded":"2024-04-24T22:59:59.118378+0000","last_fullsized":"2024-04-24T22:59:59.118378+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:04:31.130091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.908653+0000","last_change":"2024-04-24T22:59:58.128358+0000","last_active":"2024-04-24T22:59:59.908653+0000","last_peered":"2024-04-24T22:59:59.908653+0000","last_clean":"2024-04-24T22:59:59.908653+0000","last_became_active":"2024-04-24T22:59:58.126394+0000","last_became_peered":"2024-04-24T22:59:58.126394+0000","last_unstale":"2024-04-24T22:59:59.908653+0000","last_undegraded":"2024-04-24T22:59:59.908653+0000","last_fullsized":"2024-04-24T22:59:59.908653+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:37:28.755807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.119295+0000","last_change":"2024-04-24T22:59:57.115098+0000","last_active":"2024-04-24T22:59:59.119295+0000","last_peered":"2024-04-24T22:59:59.119295+0000","last_clean":"2024-04-24T22:59:59.119295+0000","last_became_active":"2024-04-24T22:59:57.114866+0000","last_became_peered":"2024-04-24T22:59:57.114866+0000","last_unstale":"2024-04-24T22:59:59.119295+0000","last_undegraded":"2024-04-24T22:59:59.119295+0000","last_fullsized":"2024-04-24T22:59:59.119295+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:34.827308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.909558+0000","last_change":"2024-04-24T22:59:58.124471+0000","last_active":"2024-04-24T22:59:59.909558+0000","last_peered":"2024-04-24T22:59:59.909558+0000","last_clean":"2024-04-24T22:59:59.909558+0000","last_became_active":"2024-04-24T22:59:58.120440+0000","last_became_peered":"2024-04-24T22:59:58.120440+0000","last_unstale":"2024-04-24T22:59:59.909558+0000","last_undegraded":"2024-04-24T22:59:59.909558+0000","last_fullsized":"2024-04-24T22:59:59.909558+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:32:29.298793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"31'3","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.908721+0000","last_change":"2024-04-24T22:59:57.115370+0000","last_active":"2024-04-24T22:59:59.908721+0000","last_peered":"2024-04-24T22:59:59.908721+0000","last_clean":"2024-04-24T22:59:59.908721+0000","last_became_active":"2024-04-24T22:59:57.115147+0000","last_became_peered":"2024-04-24T22:59:57.115147+0000","last_unstale":"2024-04-24T22:59:59.908721+0000","last_undegraded":"2024-04-24T22:59:59.908721+0000","last_fullsized":"2024-04-24T22:59:59.908721+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:15.729432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.908349+0000","last_change":"2024-04-24T22:59:58.119929+0000","last_active":"2024-04-24T22:59:59.908349+0000","last_peered":"2024-04-24T22:59:59.908349+0000","last_clean":"2024-04-24T22:59:59.908349+0000","last_became_active":"2024-04-24T22:59:58.119679+0000","last_became_peered":"2024-04-24T22:59:58.119679+0000","last_unstale":"2024-04-24T22:59:59.908349+0000","last_undegraded":"2024-04-24T22:59:59.908349+0000","last_fullsized":"2024-04-24T22:59:59.908349+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:44:07.177684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"31'6","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.119899+0000","last_change":"2024-04-24T22:59:57.118202+0000","last_active":"2024-04-24T22:59:59.119899+0000","last_peered":"2024-04-24T22:59:59.119899+0000","last_clean":"2024-04-24T22:59:59.119899+0000","last_became_active":"2024-04-24T22:59:57.117946+0000","last_became_peered":"2024-04-24T22:59:57.117946+0000","last_unstale":"2024-04-24T22:59:59.119899+0000","last_undegraded":"2024-04-24T22:59:59.119899+0000","last_fullsized":"2024-04-24T22:59:59.119899+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:49.462155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.118935+0000","last_change":"2024-04-24T22:59:58.115793+0000","last_active":"2024-04-24T22:59:59.118935+0000","last_peered":"2024-04-24T22:59:59.118935+0000","last_clean":"2024-04-24T22:59:59.118935+0000","last_became_active":"2024-04-24T22:59:58.115096+0000","last_became_peered":"2024-04-24T22:59:58.115096+0000","last_unstale":"2024-04-24T22:59:59.118935+0000","last_undegraded":"2024-04-24T22:59:59.118935+0000","last_fullsized":"2024-04-24T22:59:59.118935+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:45:14.086849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.118742+0000","last_change":"2024-04-24T22:59:57.111899+0000","last_active":"2024-04-24T22:59:59.118742+0000","last_peered":"2024-04-24T22:59:59.118742+0000","last_clean":"2024-04-24T22:59:59.118742+0000","last_became_active":"2024-04-24T22:59:57.111632+0000","last_became_peered":"2024-04-24T22:59:57.111632+0000","last_unstale":"2024-04-24T22:59:59.118742+0000","last_undegraded":"2024-04-24T22:59:59.118742+0000","last_fullsized":"2024-04-24T22:59:59.118742+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:08:39.617815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.909244+0000","last_change":"2024-04-24T22:59:58.123530+0000","last_active":"2024-04-24T22:59:59.909244+0000","last_peered":"2024-04-24T22:59:59.909244+0000","last_clean":"2024-04-24T22:59:59.909244+0000","last_became_active":"2024-04-24T22:59:58.122383+0000","last_became_peered":"2024-04-24T22:59:58.122383+0000","last_unstale":"2024-04-24T22:59:59.909244+0000","last_undegraded":"2024-04-24T22:59:59.909244+0000","last_fullsized":"2024-04-24T22:59:59.909244+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:02:37.263958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.909271+0000","last_change":"2024-04-24T22:59:57.116842+0000","last_active":"2024-04-24T22:59:59.909271+0000","last_peered":"2024-04-24T22:59:59.909271+0000","last_clean":"2024-04-24T22:59:59.909271+0000","last_became_active":"2024-04-24T22:59:57.116577+0000","last_became_peered":"2024-04-24T22:59:57.116577+0000","last_unstale":"2024-04-24T22:59:59.909271+0000","last_undegraded":"2024-04-24T22:59:59.909271+0000","last_fullsized":"2024-04-24T22:59:59.909271+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:06:00.482124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.118937+0000","last_change":"2024-04-24T22:59:58.110821+0000","last_active":"2024-04-24T22:59:59.118937+0000","last_peered":"2024-04-24T22:59:59.118937+0000","last_clean":"2024-04-24T22:59:59.118937+0000","last_became_active":"2024-04-24T22:59:58.110221+0000","last_became_peered":"2024-04-24T22:59:58.110221+0000","last_unstale":"2024-04-24T22:59:59.118937+0000","last_undegraded":"2024-04-24T22:59:59.118937+0000","last_fullsized":"2024-04-24T22:59:59.118937+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:23:25.976273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.908531+0000","last_change":"2024-04-24T22:59:57.113832+0000","last_active":"2024-04-24T22:59:59.908531+0000","last_peered":"2024-04-24T22:59:59.908531+0000","last_clean":"2024-04-24T22:59:59.908531+0000","last_became_active":"2024-04-24T22:59:57.113610+0000","last_became_peered":"2024-04-24T22:59:57.113610+0000","last_unstale":"2024-04-24T22:59:59.908531+0000","last_undegraded":"2024-04-24T22:59:59.908531+0000","last_fullsized":"2024-04-24T22:59:59.908531+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:31:51.205193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.909270+0000","last_change":"2024-04-24T22:59:58.116121+0000","last_active":"2024-04-24T22:59:59.909270+0000","last_peered":"2024-04-24T22:59:59.909270+0000","last_clean":"2024-04-24T22:59:59.909270+0000","last_became_active":"2024-04-24T22:59:58.114650+0000","last_became_peered":"2024-04-24T22:59:58.114650+0000","last_unstale":"2024-04-24T22:59:59.909270+0000","last_undegraded":"2024-04-24T22:59:59.909270+0000","last_fullsized":"2024-04-24T22:59:59.909270+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:23:35.524317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"31'8","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.908856+0000","last_change":"2024-04-24T22:59:57.116182+0000","last_active":"2024-04-24T22:59:59.908856+0000","last_peered":"2024-04-24T22:59:59.908856+0000","last_clean":"2024-04-24T22:59:59.908856+0000","last_became_active":"2024-04-24T22:59:57.115952+0000","last_became_peered":"2024-04-24T22:59:57.115952+0000","last_unstale":"2024-04-24T22:59:59.908856+0000","last_undegraded":"2024-04-24T22:59:59.908856+0000","last_fullsized":"2024-04-24T22:59:59.908856+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:39.294177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.908575+0000","last_change":"2024-04-24T22:59:58.121582+0000","last_active":"2024-04-24T22:59:59.908575+0000","last_peered":"2024-04-24T22:59:59.908575+0000","last_clean":"2024-04-24T22:59:59.908575+0000","last_became_active":"2024-04-24T22:59:58.121294+0000","last_became_peered":"2024-04-24T22:59:58.121294+0000","last_unstale":"2024-04-24T22:59:59.908575+0000","last_undegraded":"2024-04-24T22:59:59.908575+0000","last_fullsized":"2024-04-24T22:59:59.908575+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:40:06.614089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"31'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.908896+0000","last_change":"2024-04-24T22:59:57.119183+0000","last_active":"2024-04-24T22:59:59.908896+0000","last_peered":"2024-04-24T22:59:59.908896+0000","last_clean":"2024-04-24T22:59:59.908896+0000","last_became_active":"2024-04-24T22:59:57.118911+0000","last_became_peered":"2024-04-24T22:59:57.118911+0000","last_unstale":"2024-04-24T22:59:59.908896+0000","last_undegraded":"2024-04-24T22:59:59.908896+0000","last_fullsized":"2024-04-24T22:59:59.908896+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:46:50.998138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.908550+0000","last_change":"2024-04-24T22:59:58.115684+0000","last_active":"2024-04-24T22:59:59.908550+0000","last_peered":"2024-04-24T22:59:59.908550+0000","last_clean":"2024-04-24T22:59:59.908550+0000","last_became_active":"2024-04-24T22:59:58.115109+0000","last_became_peered":"2024-04-24T22:59:58.115109+0000","last_unstale":"2024-04-24T22:59:59.908550+0000","last_undegraded":"2024-04-24T22:59:59.908550+0000","last_fullsized":"2024-04-24T22:59:59.908550+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:25:11.951176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.118853+0000","last_change":"2024-04-24T22:59:57.115384+0000","last_active":"2024-04-24T22:59:59.118853+0000","last_peered":"2024-04-24T22:59:59.118853+0000","last_clean":"2024-04-24T22:59:59.118853+0000","last_became_active":"2024-04-24T22:59:57.115129+0000","last_became_peered":"2024-04-24T22:59:57.115129+0000","last_unstale":"2024-04-24T22:59:59.118853+0000","last_undegraded":"2024-04-24T22:59:59.118853+0000","last_fullsized":"2024-04-24T22:59:59.118853+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:55.108854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.119140+0000","last_change":"2024-04-24T22:59:58.115641+0000","last_active":"2024-04-24T22:59:59.119140+0000","last_peered":"2024-04-24T22:59:59.119140+0000","last_clean":"2024-04-24T22:59:59.119140+0000","last_became_active":"2024-04-24T22:59:58.115416+0000","last_became_peered":"2024-04-24T22:59:58.115416+0000","last_unstale":"2024-04-24T22:59:59.119140+0000","last_undegraded":"2024-04-24T22:59:59.119140+0000","last_fullsized":"2024-04-24T22:59:59.119140+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:22:58.656660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"31'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.908832+0000","last_change":"2024-04-24T22:59:57.113034+0000","last_active":"2024-04-24T22:59:59.908832+0000","last_peered":"2024-04-24T22:59:59.908832+0000","last_clean":"2024-04-24T22:59:59.908832+0000","last_became_active":"2024-04-24T22:59:57.111752+0000","last_became_peered":"2024-04-24T22:59:57.111752+0000","last_unstale":"2024-04-24T22:59:59.908832+0000","last_undegraded":"2024-04-24T22:59:59.908832+0000","last_fullsized":"2024-04-24T22:59:59.908832+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:05.000326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.908588+0000","last_change":"2024-04-24T22:59:58.121062+0000","last_active":"2024-04-24T22:59:59.908588+0000","last_peered":"2024-04-24T22:59:59.908588+0000","last_clean":"2024-04-24T22:59:59.908588+0000","last_became_active":"2024-04-24T22:59:58.120803+0000","last_became_peered":"2024-04-24T22:59:58.120803+0000","last_unstale":"2024-04-24T22:59:59.908588+0000","last_undegraded":"2024-04-24T22:59:59.908588+0000","last_fullsized":"2024-04-24T22:59:59.908588+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:03.114923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.908401+0000","last_change":"2024-04-24T22:59:57.117253+0000","last_active":"2024-04-24T22:59:59.908401+0000","last_peered":"2024-04-24T22:59:59.908401+0000","last_clean":"2024-04-24T22:59:59.908401+0000","last_became_active":"2024-04-24T22:59:57.116974+0000","last_became_peered":"2024-04-24T22:59:57.116974+0000","last_unstale":"2024-04-24T22:59:59.908401+0000","last_undegraded":"2024-04-24T22:59:59.908401+0000","last_fullsized":"2024-04-24T22:59:59.908401+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:41.680283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.119761+0000","last_change":"2024-04-24T22:59:58.116160+0000","last_active":"2024-04-24T22:59:59.119761+0000","last_peered":"2024-04-24T22:59:59.119761+0000","last_clean":"2024-04-24T22:59:59.119761+0000","last_became_active":"2024-04-24T22:59:58.114650+0000","last_became_peered":"2024-04-24T22:59:58.114650+0000","last_unstale":"2024-04-24T22:59:59.119761+0000","last_undegraded":"2024-04-24T22:59:59.119761+0000","last_fullsized":"2024-04-24T22:59:59.119761+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:23.478198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.118563+0000","last_change":"2024-04-24T22:59:57.111772+0000","last_active":"2024-04-24T22:59:59.118563+0000","last_peered":"2024-04-24T22:59:59.118563+0000","last_clean":"2024-04-24T22:59:59.118563+0000","last_became_active":"2024-04-24T22:59:57.111452+0000","last_became_peered":"2024-04-24T22:59:57.111452+0000","last_unstale":"2024-04-24T22:59:59.118563+0000","last_undegraded":"2024-04-24T22:59:59.118563+0000","last_fullsized":"2024-04-24T22:59:59.118563+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:41:50.141249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.118445+0000","last_change":"2024-04-24T22:59:58.114690+0000","last_active":"2024-04-24T22:59:59.118445+0000","last_peered":"2024-04-24T22:59:59.118445+0000","last_clean":"2024-04-24T22:59:59.118445+0000","last_became_active":"2024-04-24T22:59:58.114479+0000","last_became_peered":"2024-04-24T22:59:58.114479+0000","last_unstale":"2024-04-24T22:59:59.118445+0000","last_undegraded":"2024-04-24T22:59:59.118445+0000","last_fullsized":"2024-04-24T22:59:59.118445+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:46.058724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.118734+0000","last_change":"2024-04-24T22:59:57.111232+0000","last_active":"2024-04-24T22:59:59.118734+0000","last_peered":"2024-04-24T22:59:59.118734+0000","last_clean":"2024-04-24T22:59:59.118734+0000","last_became_active":"2024-04-24T22:59:57.111010+0000","last_became_peered":"2024-04-24T22:59:57.111010+0000","last_unstale":"2024-04-24T22:59:59.118734+0000","last_undegraded":"2024-04-24T22:59:59.118734+0000","last_fullsized":"2024-04-24T22:59:59.118734+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:23:05.375749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.119108+0000","last_change":"2024-04-24T22:59:58.127636+0000","last_active":"2024-04-24T22:59:59.119108+0000","last_peered":"2024-04-24T22:59:59.119108+0000","last_clean":"2024-04-24T22:59:59.119108+0000","last_became_active":"2024-04-24T22:59:58.127339+0000","last_became_peered":"2024-04-24T22:59:58.127339+0000","last_unstale":"2024-04-24T22:59:59.119108+0000","last_undegraded":"2024-04-24T22:59:59.119108+0000","last_fullsized":"2024-04-24T22:59:59.119108+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:22.617204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.908804+0000","last_change":"2024-04-24T22:59:57.110898+0000","last_active":"2024-04-24T22:59:59.908804+0000","last_peered":"2024-04-24T22:59:59.908804+0000","last_clean":"2024-04-24T22:59:59.908804+0000","last_became_active":"2024-04-24T22:59:57.110665+0000","last_became_peered":"2024-04-24T22:59:57.110665+0000","last_unstale":"2024-04-24T22:59:59.908804+0000","last_undegraded":"2024-04-24T22:59:59.908804+0000","last_fullsized":"2024-04-24T22:59:59.908804+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:46:22.442092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.908714+0000","last_change":"2024-04-24T22:59:58.122026+0000","last_active":"2024-04-24T22:59:59.908714+0000","last_peered":"2024-04-24T22:59:59.908714+0000","last_clean":"2024-04-24T22:59:59.908714+0000","last_became_active":"2024-04-24T22:59:58.121701+0000","last_became_peered":"2024-04-24T22:59:58.121701+0000","last_unstale":"2024-04-24T22:59:59.908714+0000","last_undegraded":"2024-04-24T22:59:59.908714+0000","last_fullsized":"2024-04-24T22:59:59.908714+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:39.938598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.908934+0000","last_change":"2024-04-24T22:59:57.114417+0000","last_active":"2024-04-24T22:59:59.908934+0000","last_peered":"2024-04-24T22:59:59.908934+0000","last_clean":"2024-04-24T22:59:59.908934+0000","last_became_active":"2024-04-24T22:59:57.112771+0000","last_became_peered":"2024-04-24T22:59:57.112771+0000","last_unstale":"2024-04-24T22:59:59.908934+0000","last_undegraded":"2024-04-24T22:59:59.908934+0000","last_fullsized":"2024-04-24T22:59:59.908934+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:40.327920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.908697+0000","last_change":"2024-04-24T22:59:58.116722+0000","last_active":"2024-04-24T22:59:59.908697+0000","last_peered":"2024-04-24T22:59:59.908697+0000","last_clean":"2024-04-24T22:59:59.908697+0000","last_became_active":"2024-04-24T22:59:58.116230+0000","last_became_peered":"2024-04-24T22:59:58.116230+0000","last_unstale":"2024-04-24T22:59:59.908697+0000","last_undegraded":"2024-04-24T22:59:59.908697+0000","last_fullsized":"2024-04-24T22:59:59.908697+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:52:31.705005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.119510+0000","last_change":"2024-04-24T22:59:57.115765+0000","last_active":"2024-04-24T22:59:59.119510+0000","last_peered":"2024-04-24T22:59:59.119510+0000","last_clean":"2024-04-24T22:59:59.119510+0000","last_became_active":"2024-04-24T22:59:57.115056+0000","last_became_peered":"2024-04-24T22:59:57.115056+0000","last_unstale":"2024-04-24T22:59:59.119510+0000","last_undegraded":"2024-04-24T22:59:59.119510+0000","last_fullsized":"2024-04-24T22:59:59.119510+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:15:04.446161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.909575+0000","last_change":"2024-04-24T22:59:58.114890+0000","last_active":"2024-04-24T22:59:59.909575+0000","last_peered":"2024-04-24T22:59:59.909575+0000","last_clean":"2024-04-24T22:59:59.909575+0000","last_became_active":"2024-04-24T22:59:58.113608+0000","last_became_peered":"2024-04-24T22:59:58.113608+0000","last_unstale":"2024-04-24T22:59:59.909575+0000","last_undegraded":"2024-04-24T22:59:59.909575+0000","last_fullsized":"2024-04-24T22:59:59.909575+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:42:33.905025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"32'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.119023+0000","last_change":"2024-04-24T22:59:57.113564+0000","last_active":"2024-04-24T22:59:59.119023+0000","last_peered":"2024-04-24T22:59:59.119023+0000","last_clean":"2024-04-24T22:59:59.119023+0000","last_became_active":"2024-04-24T22:59:57.113292+0000","last_became_peered":"2024-04-24T22:59:57.113292+0000","last_unstale":"2024-04-24T22:59:59.119023+0000","last_undegraded":"2024-04-24T22:59:59.119023+0000","last_fullsized":"2024-04-24T22:59:59.119023+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:22.389410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.908387+0000","last_change":"2024-04-24T22:59:58.126092+0000","last_active":"2024-04-24T22:59:59.908387+0000","last_peered":"2024-04-24T22:59:59.908387+0000","last_clean":"2024-04-24T22:59:59.908387+0000","last_became_active":"2024-04-24T22:59:58.125856+0000","last_became_peered":"2024-04-24T22:59:58.125856+0000","last_unstale":"2024-04-24T22:59:59.908387+0000","last_undegraded":"2024-04-24T22:59:59.908387+0000","last_fullsized":"2024-04-24T22:59:59.908387+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:18:24.694224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.119195+0000","last_change":"2024-04-24T22:59:57.111930+0000","last_active":"2024-04-24T22:59:59.119195+0000","last_peered":"2024-04-24T22:59:59.119195+0000","last_clean":"2024-04-24T22:59:59.119195+0000","last_became_active":"2024-04-24T22:59:57.111632+0000","last_became_peered":"2024-04-24T22:59:57.111632+0000","last_unstale":"2024-04-24T22:59:59.119195+0000","last_undegraded":"2024-04-24T22:59:59.119195+0000","last_fullsized":"2024-04-24T22:59:59.119195+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:18:47.477657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.908166+0000","last_change":"2024-04-24T22:59:58.120060+0000","last_active":"2024-04-24T22:59:59.908166+0000","last_peered":"2024-04-24T22:59:59.908166+0000","last_clean":"2024-04-24T22:59:59.908166+0000","last_became_active":"2024-04-24T22:59:58.119799+0000","last_became_peered":"2024-04-24T22:59:58.119799+0000","last_unstale":"2024-04-24T22:59:59.908166+0000","last_undegraded":"2024-04-24T22:59:59.908166+0000","last_fullsized":"2024-04-24T22:59:59.908166+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:05:20.545015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.119339+0000","last_change":"2024-04-24T22:59:57.115123+0000","last_active":"2024-04-24T22:59:59.119339+0000","last_peered":"2024-04-24T22:59:59.119339+0000","last_clean":"2024-04-24T22:59:59.119339+0000","last_became_active":"2024-04-24T22:59:57.114845+0000","last_became_peered":"2024-04-24T22:59:57.114845+0000","last_unstale":"2024-04-24T22:59:59.119339+0000","last_undegraded":"2024-04-24T22:59:59.119339+0000","last_fullsized":"2024-04-24T22:59:59.119339+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:34:39.508853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.908862+0000","last_change":"2024-04-24T22:59:58.126690+0000","last_active":"2024-04-24T22:59:59.908862+0000","last_peered":"2024-04-24T22:59:59.908862+0000","last_clean":"2024-04-24T22:59:59.908862+0000","last_became_active":"2024-04-24T22:59:58.126060+0000","last_became_peered":"2024-04-24T22:59:58.126060+0000","last_unstale":"2024-04-24T22:59:59.908862+0000","last_undegraded":"2024-04-24T22:59:59.908862+0000","last_fullsized":"2024-04-24T22:59:59.908862+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:37:48.527826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.119740+0000","last_change":"2024-04-24T22:59:57.112511+0000","last_active":"2024-04-24T22:59:59.119740+0000","last_peered":"2024-04-24T22:59:59.119740+0000","last_clean":"2024-04-24T22:59:59.119740+0000","last_became_active":"2024-04-24T22:59:57.112210+0000","last_became_peered":"2024-04-24T22:59:57.112210+0000","last_unstale":"2024-04-24T22:59:59.119740+0000","last_undegraded":"2024-04-24T22:59:59.119740+0000","last_fullsized":"2024-04-24T22:59:59.119740+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:29:17.700262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.909138+0000","last_change":"2024-04-24T22:59:58.124788+0000","last_active":"2024-04-24T22:59:59.909138+0000","last_peered":"2024-04-24T22:59:59.909138+0000","last_clean":"2024-04-24T22:59:59.909138+0000","last_became_active":"2024-04-24T22:59:58.121200+0000","last_became_peered":"2024-04-24T22:59:58.121200+0000","last_unstale":"2024-04-24T22:59:59.909138+0000","last_undegraded":"2024-04-24T22:59:59.909138+0000","last_fullsized":"2024-04-24T22:59:59.909138+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:56:03.763221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.119189+0000","last_change":"2024-04-24T22:59:57.111232+0000","last_active":"2024-04-24T22:59:59.119189+0000","last_peered":"2024-04-24T22:59:59.119189+0000","last_clean":"2024-04-24T22:59:59.119189+0000","last_became_active":"2024-04-24T22:59:57.110997+0000","last_became_peered":"2024-04-24T22:59:57.110997+0000","last_unstale":"2024-04-24T22:59:59.119189+0000","last_undegraded":"2024-04-24T22:59:59.119189+0000","last_fullsized":"2024-04-24T22:59:59.119189+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:38:15.290589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.908615+0000","last_change":"2024-04-24T22:59:58.117188+0000","last_active":"2024-04-24T22:59:59.908615+0000","last_peered":"2024-04-24T22:59:59.908615+0000","last_clean":"2024-04-24T22:59:59.908615+0000","last_became_active":"2024-04-24T22:59:58.116959+0000","last_became_peered":"2024-04-24T22:59:58.116959+0000","last_unstale":"2024-04-24T22:59:59.908615+0000","last_undegraded":"2024-04-24T22:59:59.908615+0000","last_fullsized":"2024-04-24T22:59:59.908615+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:35:09.275951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.909614+0000","last_change":"2024-04-24T22:59:57.118085+0000","last_active":"2024-04-24T22:59:59.909614+0000","last_peered":"2024-04-24T22:59:59.909614+0000","last_clean":"2024-04-24T22:59:59.909614+0000","last_became_active":"2024-04-24T22:59:57.116287+0000","last_became_peered":"2024-04-24T22:59:57.116287+0000","last_unstale":"2024-04-24T22:59:59.909614+0000","last_undegraded":"2024-04-24T22:59:59.909614+0000","last_fullsized":"2024-04-24T22:59:59.909614+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:01:47.114454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.909449+0000","last_change":"2024-04-24T22:59:58.122143+0000","last_active":"2024-04-24T22:59:59.909449+0000","last_peered":"2024-04-24T22:59:59.909449+0000","last_clean":"2024-04-24T22:59:59.909449+0000","last_became_active":"2024-04-24T22:59:58.120947+0000","last_became_peered":"2024-04-24T22:59:58.120947+0000","last_unstale":"2024-04-24T22:59:59.909449+0000","last_undegraded":"2024-04-24T22:59:59.909449+0000","last_fullsized":"2024-04-24T22:59:59.909449+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:57.098836+0000","last_clean_scrub_stamp":"2024-04-24T22:59:57.098836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:35:41.444314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.909085+0000","last_change":"2024-04-24T22:59:57.110803+0000","last_active":"2024-04-24T22:59:59.909085+0000","last_peered":"2024-04-24T22:59:59.909085+0000","last_clean":"2024-04-24T22:59:59.909085+0000","last_became_active":"2024-04-24T22:59:57.110572+0000","last_became_peered":"2024-04-24T22:59:57.110572+0000","last_unstale":"2024-04-24T22:59:59.909085+0000","last_undegraded":"2024-04-24T22:59:59.909085+0000","last_fullsized":"2024-04-24T22:59:59.909085+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:59:56.094499+0000","last_clean_scrub_stamp":"2024-04-24T22:59:56.094499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:32:52.754986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":190,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-24T22:59:59.908065+0000","last_change":"2024-04-24T22:58:55.138810+0000","last_active":"2024-04-24T22:59:59.908065+0000","last_peered":"2024-04-24T22:59:59.908065+0000","last_clean":"2024-04-24T22:59:59.908065+0000","last_became_active":"2024-04-24T22:58:55.138627+0000","last_became_peered":"2024-04-24T22:58:55.138627+0000","last_unstale":"2024-04-24T22:59:59.908065+0000","last_undegraded":"2024-04-24T22:59:59.908065+0000","last_fullsized":"2024-04-24T22:59:59.908065+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_clean_scrub_stamp":"2024-04-24T22:58:54.127235+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:24.010002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":135,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195177,"statfs":{"total":200000000,"available":199861333,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":136,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195175,"statfs":{"total":200000000,"available":199859743,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":712,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194600,"statfs":{"total":200000000,"available":199270405,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":35,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":136,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195175,"statfs":{"total":200000000,"available":199859721,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":136,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195176,"statfs":{"total":200000000,"available":199860579,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":135,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195177,"statfs":{"total":200000000,"available":199861321,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":711,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194600,"statfs":{"total":200000000,"available":199270917,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":135,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195177,"statfs":{"total":200000000,"available":199861265,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-24T23:00:05.887 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-24T23:00:05.887 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-24T23:00:06.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.029+0000 7f55b5812640 1 Processor -- start 2024-04-24T23:00:06.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.029+0000 7f55b5812640 1 -- start start 2024-04-24T23:00:06.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.030+0000 7f55b5812640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f55b015e9b0 0x7f55b00d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:06.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.030+0000 7f55b5812640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f55b006b820 0x7f55b00b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:06.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.030+0000 7f55b5812640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55b00d3170 con 0x7f55b00d4210 2024-04-24T23:00:06.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.030+0000 7f55b5812640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f55b00d3450 con 0x7f55b006b820 2024-04-24T23:00:06.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.030+0000 7f55b5812640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f55b00d3730 con 0x7f55b015e9b0 2024-04-24T23:00:06.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.030+0000 7f55af7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f55b006b820 0x7f55b00b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:06.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.030+0000 7f55af7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f55b006b820 0x7f55b00b78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60120/0 (socket says 172.21.15.32:60120) 2024-04-24T23:00:06.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.030+0000 7f55af7fe640 1 -- 172.21.15.32:0/1085177223 learned_addr learned my addr 172.21.15.32:0/1085177223 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:06.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.030+0000 7f55aeffd640 1 --2- 172.21.15.32:0/1085177223 >> v2:172.21.15.121:3300/0 conn(0x7f55b015e9b0 0x7f55b00d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:06.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.031+0000 7f55af7fe640 1 -- 172.21.15.32:0/1085177223 >> v2:172.21.15.121:3300/0 conn(0x7f55b015e9b0 msgr2=0x7f55b00d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:06.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.031+0000 7f55af7fe640 1 --2- 172.21.15.32:0/1085177223 >> v2:172.21.15.121:3300/0 conn(0x7f55b015e9b0 0x7f55b00d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:06.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.031+0000 7f55af7fe640 1 -- 172.21.15.32:0/1085177223 >> v1:172.21.15.121:6789/0 conn(0x7f55b00d4210 legacy=0x7f55b006b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:06.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.031+0000 7f55af7fe640 1 -- 172.21.15.32:0/1085177223 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55b00d3a10 con 0x7f55b006b820 2024-04-24T23:00:06.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.031+0000 7f55af7fe640 1 --2- 172.21.15.32:0/1085177223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55b006b820 0x7f55b00b78f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f55b006be20 tx=0x7f55a002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=434e1e806e319ca2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:06.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.031+0000 7f55adffb640 1 -- 172.21.15.32:0/1085177223 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a0036070 con 0x7f55b006b820 2024-04-24T23:00:06.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.032+0000 7f55adffb640 1 -- 172.21.15.32:0/1085177223 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55a0031600 con 0x7f55b006b820 2024-04-24T23:00:06.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.032+0000 7f55adffb640 1 -- 172.21.15.32:0/1085177223 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a00319d0 con 0x7f55b006b820 2024-04-24T23:00:06.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.032+0000 7f55b5812640 1 -- 172.21.15.32:0/1085177223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55b006b820 msgr2=0x7f55b00b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:06.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.032+0000 7f55b5812640 1 --2- 172.21.15.32:0/1085177223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55b006b820 0x7f55b00b78f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f55b006be20 tx=0x7f55a002a6a0 comp rx=0 tx=0).stop 2024-04-24T23:00:06.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.032+0000 7f55b5812640 1 -- 172.21.15.32:0/1085177223 shutdown_connections 2024-04-24T23:00:06.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.032+0000 7f55b5812640 1 --2- 172.21.15.32:0/1085177223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55b006b820 0x7f55b00b78f0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:06.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.032+0000 7f55b5812640 1 --2- 172.21.15.32:0/1085177223 >> v2:172.21.15.121:3300/0 conn(0x7f55b015e9b0 0x7f55b00d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:06.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.032+0000 7f55b5812640 1 -- 172.21.15.32:0/1085177223 >> 172.21.15.32:0/1085177223 conn(0x7f55b00d11d0 msgr2=0x7f55b00d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:06.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.032+0000 7f55b5812640 1 -- 172.21.15.32:0/1085177223 shutdown_connections 2024-04-24T23:00:06.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.032+0000 7f55b5812640 1 -- 172.21.15.32:0/1085177223 wait complete. 2024-04-24T23:00:06.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.033+0000 7f55b5812640 1 Processor -- start 2024-04-24T23:00:06.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.034+0000 7f55b5812640 1 -- start start 2024-04-24T23:00:06.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.034+0000 7f55b5812640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55b006b820 0x7f55b0156120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:06.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.034+0000 7f55b5812640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55b00d4210 0x7f55b00cd460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:06.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.034+0000 7f55b5812640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55b015e9b0 0x7f55b00cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:06.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.034+0000 7f55b5812640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f55b00ba0e0 con 0x7f55b00d4210 2024-04-24T23:00:06.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.034+0000 7f55b5812640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f55b00d2400 con 0x7f55b015e9b0 2024-04-24T23:00:06.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.034+0000 7f55b5812640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f55b00ba260 con 0x7f55b006b820 2024-04-24T23:00:06.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.034+0000 7f55ae7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55b00d4210 0x7f55b00cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:06.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.034+0000 7f55ae7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55b00d4210 0x7f55b00cd460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60130/0 (socket says 172.21.15.32:60130) 2024-04-24T23:00:06.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.034+0000 7f55ae7fc640 1 -- 172.21.15.32:0/3532936833 learned_addr learned my addr 172.21.15.32:0/3532936833 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:06.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.034+0000 7f55aeffd640 1 --2- 172.21.15.32:0/3532936833 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55b006b820 0x7f55b0156120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:06.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.034+0000 7f55af7fe640 1 --2- 172.21.15.32:0/3532936833 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55b015e9b0 0x7f55b00cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:06.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.035+0000 7f55ae7fc640 1 -- 172.21.15.32:0/3532936833 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55b006b820 msgr2=0x7f55b0156120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:06.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.035+0000 7f55ae7fc640 1 --2- 172.21.15.32:0/3532936833 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55b006b820 0x7f55b0156120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:06.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.035+0000 7f55ae7fc640 1 -- 172.21.15.32:0/3532936833 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55b015e9b0 msgr2=0x7f55b00cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:06.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.035+0000 7f55ae7fc640 1 --2- 172.21.15.32:0/3532936833 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55b015e9b0 0x7f55b00cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:06.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.035+0000 7f55ae7fc640 1 -- 172.21.15.32:0/3532936833 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55b0156660 con 0x7f55b00d4210 2024-04-24T23:00:06.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.035+0000 7f55ae7fc640 1 --2- 172.21.15.32:0/3532936833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55b00d4210 0x7f55b00cd460 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f559800c400 tx=0x7f559800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:06.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.035+0000 7f55a7fff640 1 -- 172.21.15.32:0/3532936833 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55980052d0 con 0x7f55b00d4210 2024-04-24T23:00:06.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.035+0000 7f55a7fff640 1 -- 172.21.15.32:0/3532936833 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5598005490 con 0x7f55b00d4210 2024-04-24T23:00:06.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.035+0000 7f55a7fff640 1 -- 172.21.15.32:0/3532936833 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5598007260 con 0x7f55b00d4210 2024-04-24T23:00:06.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.035+0000 7f55b5812640 1 -- 172.21.15.32:0/3532936833 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55b0154610 con 0x7f55b00d4210 2024-04-24T23:00:06.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.035+0000 7f55b5812640 1 -- 172.21.15.32:0/3532936833 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55b0154bd0 con 0x7f55b00d4210 2024-04-24T23:00:06.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.036+0000 7f55b5812640 1 -- 172.21.15.32:0/3532936833 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55b0068430 con 0x7f55b00d4210 2024-04-24T23:00:06.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.037+0000 7f55a7fff640 1 -- 172.21.15.32:0/3532936833 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5598007400 con 0x7f55b00d4210 2024-04-24T23:00:06.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.037+0000 7f55a7fff640 1 --2- 172.21.15.32:0/3532936833 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f559006be30 0x7f559006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:06.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.037+0000 7f55a7fff640 1 -- 172.21.15.32:0/3532936833 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f559808a2f0 con 0x7f55b00d4210 2024-04-24T23:00:06.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.037+0000 7f55aeffd640 1 --2- 172.21.15.32:0/3532936833 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f559006be30 0x7f559006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:06.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.038+0000 7f55aeffd640 1 --2- 172.21.15.32:0/3532936833 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f559006be30 0x7f559006e2f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f55a8009aa0 tx=0x7f55a8009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:06.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.041+0000 7f55a7fff640 1 -- 172.21.15.32:0/3532936833 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5598058650 con 0x7f55b00d4210 2024-04-24T23:00:06.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.296+0000 7f55b5812640 1 -- 172.21.15.32:0/3532936833 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f55b0156c10 con 0x7f55b00d4210 2024-04-24T23:00:06.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.299+0000 7f55a7fff640 1 -- 172.21.15.32:0/3532936833 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) ==== 190+0+147 (secure 0 0 0) 0x7f559805c950 con 0x7f55b00d4210 2024-04-24T23:00:06.300 INFO:teuthology.orchestra.run.smithi032.stdout:[client.0] 2024-04-24T23:00:06.300 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQCejilm5oeiGRAAdLXwwqrYhV3HHo7Dw/2ZqQ== 2024-04-24T23:00:06.300 INFO:teuthology.orchestra.run.smithi032.stdout: caps mds = "allow" 2024-04-24T23:00:06.300 INFO:teuthology.orchestra.run.smithi032.stdout: caps mon = "allow r" 2024-04-24T23:00:06.301 INFO:teuthology.orchestra.run.smithi032.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-24T23:00:06.301 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2024-04-24T23:00:06.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.301+0000 7f55b5812640 1 -- 172.21.15.32:0/3532936833 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f559006be30 msgr2=0x7f559006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:06.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.301+0000 7f55b5812640 1 --2- 172.21.15.32:0/3532936833 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f559006be30 0x7f559006e2f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f55a8009aa0 tx=0x7f55a8009190 comp rx=0 tx=0).stop 2024-04-24T23:00:06.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.301+0000 7f55b5812640 1 -- 172.21.15.32:0/3532936833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55b00d4210 msgr2=0x7f55b00cd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:06.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.301+0000 7f55b5812640 1 --2- 172.21.15.32:0/3532936833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55b00d4210 0x7f55b00cd460 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f559800c400 tx=0x7f559800c8c0 comp rx=0 tx=0).stop 2024-04-24T23:00:06.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.301+0000 7f55b5812640 1 -- 172.21.15.32:0/3532936833 shutdown_connections 2024-04-24T23:00:06.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.301+0000 7f55b5812640 1 --2- 172.21.15.32:0/3532936833 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f559006be30 0x7f559006e2f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:06.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.301+0000 7f55b5812640 1 --2- 172.21.15.32:0/3532936833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55b00d4210 0x7f55b00cd460 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:06.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.301+0000 7f55b5812640 1 --2- 172.21.15.32:0/3532936833 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55b015e9b0 0x7f55b00cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:06.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.301+0000 7f55b5812640 1 --2- 172.21.15.32:0/3532936833 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55b006b820 0x7f55b0156120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:06.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.301+0000 7f55b5812640 1 -- 172.21.15.32:0/3532936833 >> 172.21.15.32:0/3532936833 conn(0x7f55b00d11d0 msgr2=0x7f55b0071730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:06.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.301+0000 7f55b5812640 1 -- 172.21.15.32:0/3532936833 shutdown_connections 2024-04-24T23:00:06.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.301+0000 7f55b5812640 1 -- 172.21.15.32:0/3532936833 wait complete. 2024-04-24T23:00:06.313 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:00:06.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.412+0000 7f1d9d6ee640 1 Processor -- start 2024-04-24T23:00:06.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.412+0000 7f1d9d6ee640 1 -- start start 2024-04-24T23:00:06.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.413+0000 7f1d9d6ee640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1d9806b820 0x7f1d980d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:06.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.413+0000 7f1d9d6ee640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1d980d4210 0x7f1d980b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:06.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.413+0000 7f1d9d6ee640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d980d3170 con 0x7f1d9815e9b0 2024-04-24T23:00:06.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.413+0000 7f1d9d6ee640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f1d980d3450 con 0x7f1d9806b820 2024-04-24T23:00:06.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.413+0000 7f1d9d6ee640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f1d980d3730 con 0x7f1d980d4210 2024-04-24T23:00:06.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.413+0000 7f1d96ffd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1d9815e9b0 0x7f1d9806b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39278/0 (socket says 172.21.15.32:39278) 2024-04-24T23:00:06.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.413+0000 7f1d96ffd640 1 -- 172.21.15.32:0/3612349648 learned_addr learned my addr 172.21.15.32:0/3612349648 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:00:06.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.414+0000 7f1d95ffb640 1 -- 172.21.15.32:0/3612349648 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 48988814 0 0) 0x7f1d980d3170 con 0x7f1d9815e9b0 2024-04-24T23:00:06.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.414+0000 7f1d95ffb640 1 -- 172.21.15.32:0/3612349648 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d84003520 con 0x7f1d9815e9b0 2024-04-24T23:00:06.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.414+0000 7f1d967fc640 1 --2- 172.21.15.32:0/3612349648 >> v2:172.21.15.121:3300/0 conn(0x7f1d9806b820 0x7f1d980d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:06.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.414+0000 7f1d977fe640 1 --2- 172.21.15.32:0/3612349648 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1d980d4210 0x7f1d980b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:06.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.414+0000 7f1d95ffb640 1 -- 172.21.15.32:0/3612349648 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 661056904 0 0) 0x7f1d84003520 con 0x7f1d9815e9b0 2024-04-24T23:00:06.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.414+0000 7f1d95ffb640 1 -- 172.21.15.32:0/3612349648 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1d980d4210 msgr2=0x7f1d980b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:06.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.414+0000 7f1d95ffb640 1 --2- 172.21.15.32:0/3612349648 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1d980d4210 0x7f1d980b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:06.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.414+0000 7f1d95ffb640 1 -- 172.21.15.32:0/3612349648 >> v2:172.21.15.121:3300/0 conn(0x7f1d9806b820 msgr2=0x7f1d980d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:06.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.414+0000 7f1d95ffb640 1 --2- 172.21.15.32:0/3612349648 >> v2:172.21.15.121:3300/0 conn(0x7f1d9806b820 0x7f1d980d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:06.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.414+0000 7f1d95ffb640 1 -- 172.21.15.32:0/3612349648 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d980d3a10 con 0x7f1d9815e9b0 2024-04-24T23:00:06.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.414+0000 7f1d95ffb640 1 -- 172.21.15.32:0/3612349648 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f1d90001670 con 0x7f1d9815e9b0 2024-04-24T23:00:06.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.414+0000 7f1d977fe640 1 --2- 172.21.15.32:0/3612349648 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1d980d4210 0x7f1d980b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:00:06.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.414+0000 7f1d95ffb640 1 -- 172.21.15.32:0/3612349648 >> v1:172.21.15.32:6789/0 conn(0x7f1d9815e9b0 legacy=0x7f1d9806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:06.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.414+0000 7f1d95ffb640 1 --2- 172.21.15.32:0/3612349648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d840039b0 0x7f1d84003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:06.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.414+0000 7f1d95ffb640 1 --2- 172.21.15.32:0/3612349648 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1d980d4210 0x7f1d84004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:06.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.414+0000 7f1d95ffb640 1 --2- 172.21.15.32:0/3612349648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d84004d40 0x7f1d840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:06.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.414+0000 7f1d95ffb640 1 -- 172.21.15.32:0/3612349648 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1d980d3450 con 0x7f1d840039b0 2024-04-24T23:00:06.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.414+0000 7f1d95ffb640 1 -- 172.21.15.32:0/3612349648 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1d980d3730 con 0x7f1d84004d40 2024-04-24T23:00:06.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.414+0000 7f1d95ffb640 1 -- 172.21.15.32:0/3612349648 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f1d8400b9c0 con 0x7f1d980d4210 2024-04-24T23:00:06.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.414+0000 7f1d96ffd640 1 --2- 172.21.15.32:0/3612349648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d840039b0 0x7f1d84003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:06.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.414+0000 7f1d967fc640 1 --2- 172.21.15.32:0/3612349648 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1d980d4210 0x7f1d84004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:06.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.414+0000 7f1d977fe640 1 --2- 172.21.15.32:0/3612349648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d84004d40 0x7f1d840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:06.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.414+0000 7f1d96ffd640 1 -- 172.21.15.32:0/3612349648 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1d980d4210 msgr2=0x7f1d84004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:06.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.414+0000 7f1d96ffd640 1 --2- 172.21.15.32:0/3612349648 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1d980d4210 0x7f1d84004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:06.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.414+0000 7f1d96ffd640 1 -- 172.21.15.32:0/3612349648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d84004d40 msgr2=0x7f1d840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:06.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.414+0000 7f1d96ffd640 1 --2- 172.21.15.32:0/3612349648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d84004d40 0x7f1d840051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:06.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.414+0000 7f1d96ffd640 1 -- 172.21.15.32:0/3612349648 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d84003520 con 0x7f1d840039b0 2024-04-24T23:00:06.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.415+0000 7f1d96ffd640 1 --2- 172.21.15.32:0/3612349648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d840039b0 0x7f1d84003dd0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1d9000c240 tx=0x7f1d84005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1698bca2b0e7fbe server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:06.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.415+0000 7f1d95ffb640 1 -- 172.21.15.32:0/3612349648 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9002e070 con 0x7f1d840039b0 2024-04-24T23:00:06.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.415+0000 7f1d95ffb640 1 -- 172.21.15.32:0/3612349648 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d90005af0 con 0x7f1d840039b0 2024-04-24T23:00:06.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.415+0000 7f1d95ffb640 1 -- 172.21.15.32:0/3612349648 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d90007d30 con 0x7f1d840039b0 2024-04-24T23:00:06.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.416+0000 7f1d9d6ee640 1 -- 172.21.15.32:0/3612349648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d840039b0 msgr2=0x7f1d84003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:06.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.416+0000 7f1d9d6ee640 1 --2- 172.21.15.32:0/3612349648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d840039b0 0x7f1d84003dd0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1d9000c240 tx=0x7f1d84005830 comp rx=0 tx=0).stop 2024-04-24T23:00:06.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.416+0000 7f1d977fe640 1 -- 172.21.15.32:0/3612349648 reap_dead start 2024-04-24T23:00:06.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.416+0000 7f1d9d6ee640 1 -- 172.21.15.32:0/3612349648 shutdown_connections 2024-04-24T23:00:06.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.416+0000 7f1d9d6ee640 1 -- 172.21.15.32:0/3612349648 >> 172.21.15.32:0/3612349648 conn(0x7f1d980d11d0 msgr2=0x7f1d980d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:06.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.416+0000 7f1d9d6ee640 1 -- 172.21.15.32:0/3612349648 shutdown_connections 2024-04-24T23:00:06.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.416+0000 7f1d9d6ee640 1 -- 172.21.15.32:0/3612349648 wait complete. 2024-04-24T23:00:06.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.417+0000 7f1d9d6ee640 1 Processor -- start 2024-04-24T23:00:06.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.417+0000 7f1d9d6ee640 1 -- start start 2024-04-24T23:00:06.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.418+0000 7f1d9d6ee640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d981f7680 0x7f1d980cc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:06.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.418+0000 7f1d9d6ee640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d98147bf0 0x7f1d98148070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:06.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.418+0000 7f1d9d6ee640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1d98146e30 0x7f1d981472e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:06.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.418+0000 7f1d9d6ee640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1d980ba0e0 con 0x7f1d98147bf0 2024-04-24T23:00:06.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.418+0000 7f1d9d6ee640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1d980d2400 con 0x7f1d981f7680 2024-04-24T23:00:06.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.418+0000 7f1d9d6ee640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f1d980ba260 con 0x7f1d98146e30 2024-04-24T23:00:06.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.418+0000 7f1d967fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d98147bf0 0x7f1d98148070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:06.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.418+0000 7f1d967fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d98147bf0 0x7f1d98148070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60148/0 (socket says 172.21.15.32:60148) 2024-04-24T23:00:06.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.418+0000 7f1d967fc640 1 -- 172.21.15.32:0/1268928768 learned_addr learned my addr 172.21.15.32:0/1268928768 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:06.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.418+0000 7f1d967fc640 1 -- 172.21.15.32:0/1268928768 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1d98146e30 msgr2=0x7f1d981472e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:06.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.418+0000 7f1d977fe640 1 --2- 172.21.15.32:0/1268928768 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1d98146e30 0x7f1d981472e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:06.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.418+0000 7f1d96ffd640 1 --2- 172.21.15.32:0/1268928768 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d981f7680 0x7f1d980cc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:06.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.418+0000 7f1d967fc640 1 --2- 172.21.15.32:0/1268928768 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1d98146e30 0x7f1d981472e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:06.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.418+0000 7f1d967fc640 1 -- 172.21.15.32:0/1268928768 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d981f7680 msgr2=0x7f1d980cc0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:06.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.418+0000 7f1d967fc640 1 --2- 172.21.15.32:0/1268928768 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d981f7680 0x7f1d980cc0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:06.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.418+0000 7f1d967fc640 1 -- 172.21.15.32:0/1268928768 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d98145d90 con 0x7f1d98147bf0 2024-04-24T23:00:06.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.419+0000 7f1d967fc640 1 --2- 172.21.15.32:0/1268928768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d98147bf0 0x7f1d98148070 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f1d80011a70 tx=0x7f1d80011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:06.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.419+0000 7f1d8f7fe640 1 -- 172.21.15.32:0/1268928768 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d8000fb50 con 0x7f1d98147bf0 2024-04-24T23:00:06.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.419+0000 7f1d8f7fe640 1 -- 172.21.15.32:0/1268928768 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d8000fcf0 con 0x7f1d98147bf0 2024-04-24T23:00:06.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.419+0000 7f1d9d6ee640 1 -- 172.21.15.32:0/1268928768 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d98146070 con 0x7f1d98147bf0 2024-04-24T23:00:06.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.419+0000 7f1d8f7fe640 1 -- 172.21.15.32:0/1268928768 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d8000fea0 con 0x7f1d98147bf0 2024-04-24T23:00:06.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.419+0000 7f1d9d6ee640 1 -- 172.21.15.32:0/1268928768 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d98146630 con 0x7f1d98147bf0 2024-04-24T23:00:06.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.420+0000 7f1d9d6ee640 1 -- 172.21.15.32:0/1268928768 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d98068430 con 0x7f1d98147bf0 2024-04-24T23:00:06.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.420+0000 7f1d8f7fe640 1 -- 172.21.15.32:0/1268928768 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1d80012040 con 0x7f1d98147bf0 2024-04-24T23:00:06.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.421+0000 7f1d8f7fe640 1 --2- 172.21.15.32:0/1268928768 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1d7806be60 0x7f1d7806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:06.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.421+0000 7f1d8f7fe640 1 -- 172.21.15.32:0/1268928768 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f1d8008b430 con 0x7f1d98147bf0 2024-04-24T23:00:06.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.421+0000 7f1d96ffd640 1 --2- 172.21.15.32:0/1268928768 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1d7806be60 0x7f1d7806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:06.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.421+0000 7f1d96ffd640 1 --2- 172.21.15.32:0/1268928768 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1d7806be60 0x7f1d7806e320 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1d90033000 tx=0x7f1d90009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:06.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.425+0000 7f1d8f7fe640 1 -- 172.21.15.32:0/1268928768 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1d80019030 con 0x7f1d98147bf0 2024-04-24T23:00:06.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.646+0000 7f1d9d6ee640 1 -- 172.21.15.32:0/1268928768 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d980c79a0 con 0x7f1d98147bf0 2024-04-24T23:00:06.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.647+0000 7f1d8f7fe640 1 -- 172.21.15.32:0/1268928768 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+5181 (secure 0 0 0) 0x7f1d80059e70 con 0x7f1d98147bf0 2024-04-24T23:00:06.648 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:00:06.648 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1796819396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":1796819396},{"type":"v1","addr":"172.21.15.32:6835","nonce":1796819396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/298202585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":298202585},{"type":"v1","addr":"172.21.15.121:6836","nonce":298202585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4939,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/4056636026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":4056636026},{"type":"v1","addr":"172.21.15.32:6834","nonce":4056636026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T22:59:57.468765+0000","modified":"2024-04-24T22:59:59.471269+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5112},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5112":{"gid":5112,"name":"b","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6837/515107694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":515107694},{"type":"v1","addr":"172.21.15.121:6837","nonce":515107694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5112,"qdb_cluster":[5112]},"id":2}]} 2024-04-24T23:00:06.649 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 13 2024-04-24T23:00:06.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.648+0000 7f1d9d6ee640 1 -- 172.21.15.32:0/1268928768 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1d7806be60 msgr2=0x7f1d7806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:06.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.648+0000 7f1d9d6ee640 1 --2- 172.21.15.32:0/1268928768 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1d7806be60 0x7f1d7806e320 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1d90033000 tx=0x7f1d90009030 comp rx=0 tx=0).stop 2024-04-24T23:00:06.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.648+0000 7f1d9d6ee640 1 -- 172.21.15.32:0/1268928768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d98147bf0 msgr2=0x7f1d98148070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:06.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.648+0000 7f1d9d6ee640 1 --2- 172.21.15.32:0/1268928768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d98147bf0 0x7f1d98148070 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f1d80011a70 tx=0x7f1d80011f30 comp rx=0 tx=0).stop 2024-04-24T23:00:06.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.648+0000 7f1d9d6ee640 1 -- 172.21.15.32:0/1268928768 shutdown_connections 2024-04-24T23:00:06.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.648+0000 7f1d9d6ee640 1 --2- 172.21.15.32:0/1268928768 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1d7806be60 0x7f1d7806e320 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:06.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.648+0000 7f1d9d6ee640 1 --2- 172.21.15.32:0/1268928768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d98147bf0 0x7f1d98148070 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:06.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.648+0000 7f1d9d6ee640 1 --2- 172.21.15.32:0/1268928768 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1d98146e30 0x7f1d981472e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:06.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.648+0000 7f1d9d6ee640 1 --2- 172.21.15.32:0/1268928768 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d981f7680 0x7f1d980cc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:06.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.649+0000 7f1d9d6ee640 1 -- 172.21.15.32:0/1268928768 >> 172.21.15.32:0/1268928768 conn(0x7f1d980d11d0 msgr2=0x7f1d980cefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:06.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.649+0000 7f1d9d6ee640 1 -- 172.21.15.32:0/1268928768 shutdown_connections 2024-04-24T23:00:06.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:06.649+0000 7f1d9d6ee640 1 -- 172.21.15.32:0/1268928768 wait complete. 2024-04-24T23:00:07.064 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 13, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T22:59:57.468765+0000', 'modified': '2024-04-24T22:59:59.471269+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5112}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5112': {'gid': 5112, 'name': 'b', 'rank': 0, 'incarnation': 11, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.121:6837/515107694', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6834', 'nonce': 515107694}, {'type': 'v1', 'addr': '172.21.15.121:6837', 'nonce': 515107694}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5112, 'qdb_cluster': [5112]} 2024-04-24T23:00:07.065 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-24T23:00:07.065 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-24T23:00:07.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.167+0000 7f4e8df9e640 1 Processor -- start 2024-04-24T23:00:07.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.167+0000 7f4e8df9e640 1 -- start start 2024-04-24T23:00:07.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.168+0000 7f4e8df9e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4e88147dd0 0x7f4e880cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:07.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.168+0000 7f4e8df9e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4e880b6b00 0x7f4e880b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:07.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.168+0000 7f4e8df9e640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e880ca800 con 0x7f4e880c9fd0 2024-04-24T23:00:07.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.168+0000 7f4e8df9e640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f4e880caae0 con 0x7f4e88147dd0 2024-04-24T23:00:07.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.168+0000 7f4e8df9e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f4e880cadc0 con 0x7f4e880b6b00 2024-04-24T23:00:07.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.168+0000 7f4e8cf9c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4e88147dd0 0x7f4e880cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:07.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.168+0000 7f4e8cf9c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4e88147dd0 0x7f4e880cf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60152/0 (socket says 172.21.15.32:60152) 2024-04-24T23:00:07.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.168+0000 7f4e8cf9c640 1 -- 172.21.15.32:0/1636972982 learned_addr learned my addr 172.21.15.32:0/1636972982 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:07.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.168+0000 7f4e8d79d640 1 --2- 172.21.15.32:0/1636972982 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4e880b6b00 0x7f4e880b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:07.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.168+0000 7f4e8cf9c640 1 -- 172.21.15.32:0/1636972982 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4e880b6b00 msgr2=0x7f4e880b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:07.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.168+0000 7f4e8cf9c640 1 --2- 172.21.15.32:0/1636972982 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4e880b6b00 0x7f4e880b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:07.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.168+0000 7f4e8cf9c640 1 -- 172.21.15.32:0/1636972982 >> v1:172.21.15.121:6789/0 conn(0x7f4e880c9fd0 legacy=0x7f4e880b63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:07.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.168+0000 7f4e8cf9c640 1 -- 172.21.15.32:0/1636972982 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e880cb0a0 con 0x7f4e88147dd0 2024-04-24T23:00:07.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.169+0000 7f4e8d79d640 1 --2- 172.21.15.32:0/1636972982 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4e880b6b00 0x7f4e880b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:00:07.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.169+0000 7f4e8cf9c640 1 --2- 172.21.15.32:0/1636972982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e88147dd0 0x7f4e880cf7b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f4e78009870 tx=0x7f4e7802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7dc9ba1896c7190 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:07.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.169+0000 7f4e877fe640 1 -- 172.21.15.32:0/1636972982 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e78037070 con 0x7f4e88147dd0 2024-04-24T23:00:07.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.169+0000 7f4e877fe640 1 -- 172.21.15.32:0/1636972982 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e78032440 con 0x7f4e88147dd0 2024-04-24T23:00:07.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.169+0000 7f4e877fe640 1 -- 172.21.15.32:0/1636972982 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e78032810 con 0x7f4e88147dd0 2024-04-24T23:00:07.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.170+0000 7f4e8df9e640 1 -- 172.21.15.32:0/1636972982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e88147dd0 msgr2=0x7f4e880cf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:07.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.170+0000 7f4e8df9e640 1 --2- 172.21.15.32:0/1636972982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e88147dd0 0x7f4e880cf7b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f4e78009870 tx=0x7f4e7802a1c0 comp rx=0 tx=0).stop 2024-04-24T23:00:07.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.170+0000 7f4e8df9e640 1 -- 172.21.15.32:0/1636972982 shutdown_connections 2024-04-24T23:00:07.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.170+0000 7f4e8df9e640 1 --2- 172.21.15.32:0/1636972982 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4e880b6b00 0x7f4e880b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:07.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.170+0000 7f4e8df9e640 1 --2- 172.21.15.32:0/1636972982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e88147dd0 0x7f4e880cf7b0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:07.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.170+0000 7f4e8df9e640 1 -- 172.21.15.32:0/1636972982 >> 172.21.15.32:0/1636972982 conn(0x7f4e88071730 msgr2=0x7f4e88071b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:07.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.170+0000 7f4e8df9e640 1 -- 172.21.15.32:0/1636972982 shutdown_connections 2024-04-24T23:00:07.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.170+0000 7f4e8df9e640 1 -- 172.21.15.32:0/1636972982 wait complete. 2024-04-24T23:00:07.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.171+0000 7f4e8df9e640 1 Processor -- start 2024-04-24T23:00:07.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.171+0000 7f4e8df9e640 1 -- start start 2024-04-24T23:00:07.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.172+0000 7f4e8df9e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4e880b6b00 0x7f4e8813ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:07.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.172+0000 7f4e8df9e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4e880c9fd0 0x7f4e8813e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:07.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.172+0000 7f4e8df9e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e88147dd0 0x7f4e881612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:07.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.172+0000 7f4e8df9e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4e880ca5e0 con 0x7f4e88147dd0 2024-04-24T23:00:07.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.172+0000 7f4e8df9e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4e880ca460 con 0x7f4e880b6b00 2024-04-24T23:00:07.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.172+0000 7f4e8df9e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f4e881481b0 con 0x7f4e880c9fd0 2024-04-24T23:00:07.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.172+0000 7f4e8d79d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e88147dd0 0x7f4e881612a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:07.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.172+0000 7f4e8d79d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e88147dd0 0x7f4e881612a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60166/0 (socket says 172.21.15.32:60166) 2024-04-24T23:00:07.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.172+0000 7f4e8d79d640 1 -- 172.21.15.32:0/4150272228 learned_addr learned my addr 172.21.15.32:0/4150272228 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:07.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.173+0000 7f4e8cf9c640 1 --2- 172.21.15.32:0/4150272228 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4e880b6b00 0x7f4e8813ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:07.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.173+0000 7f4e87fff640 1 --2- 172.21.15.32:0/4150272228 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4e880c9fd0 0x7f4e8813e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:07.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.173+0000 7f4e8d79d640 1 -- 172.21.15.32:0/4150272228 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4e880c9fd0 msgr2=0x7f4e8813e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:07.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.173+0000 7f4e8d79d640 1 --2- 172.21.15.32:0/4150272228 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4e880c9fd0 0x7f4e8813e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:07.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.173+0000 7f4e8d79d640 1 -- 172.21.15.32:0/4150272228 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4e880b6b00 msgr2=0x7f4e8813ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:07.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.173+0000 7f4e8d79d640 1 --2- 172.21.15.32:0/4150272228 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4e880b6b00 0x7f4e8813ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:07.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.173+0000 7f4e8d79d640 1 -- 172.21.15.32:0/4150272228 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e880cc2a0 con 0x7f4e88147dd0 2024-04-24T23:00:07.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.173+0000 7f4e8d79d640 1 --2- 172.21.15.32:0/4150272228 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e88147dd0 0x7f4e881612a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4e80007ea0 tx=0x7f4e80007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:07.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.173+0000 7f4e857fa640 1 -- 172.21.15.32:0/4150272228 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e8001a070 con 0x7f4e88147dd0 2024-04-24T23:00:07.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.173+0000 7f4e857fa640 1 -- 172.21.15.32:0/4150272228 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e80003bb0 con 0x7f4e88147dd0 2024-04-24T23:00:07.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.173+0000 7f4e8df9e640 1 -- 172.21.15.32:0/4150272228 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e880b9470 con 0x7f4e88147dd0 2024-04-24T23:00:07.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.173+0000 7f4e857fa640 1 -- 172.21.15.32:0/4150272228 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e8000de70 con 0x7f4e88147dd0 2024-04-24T23:00:07.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.173+0000 7f4e8df9e640 1 -- 172.21.15.32:0/4150272228 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e880b9750 con 0x7f4e88147dd0 2024-04-24T23:00:07.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.175+0000 7f4e857fa640 1 -- 172.21.15.32:0/4150272228 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4e8001d070 con 0x7f4e88147dd0 2024-04-24T23:00:07.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.175+0000 7f4e857fa640 1 --2- 172.21.15.32:0/4150272228 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4e7406be60 0x7f4e7406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:07.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.175+0000 7f4e857fa640 1 -- 172.21.15.32:0/4150272228 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f4e8008b020 con 0x7f4e88147dd0 2024-04-24T23:00:07.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.175+0000 7f4e8cf9c640 1 --2- 172.21.15.32:0/4150272228 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4e7406be60 0x7f4e7406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:07.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.176+0000 7f4e91fee740 1 Processor -- start 2024-04-24T23:00:07.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.176+0000 7f4e91fee740 1 -- start start 2024-04-24T23:00:07.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.176+0000 7f4e8cf9c640 1 --2- 172.21.15.32:0/4150272228 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4e7406be60 0x7f4e7406e320 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4e78034000 tx=0x7f4e7802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:07.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.177+0000 7f4e91fee740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560c87d6d60 0x5560c87896f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:07.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.177+0000 7f4e91fee740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5560c87b9e80 0x5560c87b9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:07.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.177+0000 7f4e91fee740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5560c87b9740 0x5560c878e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:07.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.177+0000 7f4e91fee740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5560c86f1f00 con 0x5560c87b9740 2024-04-24T23:00:07.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.177+0000 7f4e91fee740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5560c869a750 con 0x5560c87d6d60 2024-04-24T23:00:07.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.177+0000 7f4e91fee740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x5560c86edec0 con 0x5560c87b9e80 2024-04-24T23:00:07.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.177+0000 7f4e8d79d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5560c87b9740 0x5560c878e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:07.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.177+0000 7f4e8d79d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5560c87b9740 0x5560c878e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60178/0 (socket says 172.21.15.32:60178) 2024-04-24T23:00:07.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.177+0000 7f4e8d79d640 1 -- 172.21.15.32:0/804342267 learned_addr learned my addr 172.21.15.32:0/804342267 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:07.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.177+0000 7f4e8cf9c640 1 --2- 172.21.15.32:0/804342267 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560c87d6d60 0x5560c87896f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:07.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.177+0000 7f4e87fff640 1 --2- 172.21.15.32:0/804342267 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5560c87b9e80 0x5560c87b9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:07.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.177+0000 7f4e8d79d640 1 -- 172.21.15.32:0/804342267 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5560c87b9e80 msgr2=0x5560c87b9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:07.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.177+0000 7f4e8d79d640 1 --2- 172.21.15.32:0/804342267 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5560c87b9e80 0x5560c87b9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:07.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.177+0000 7f4e8d79d640 1 -- 172.21.15.32:0/804342267 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560c87d6d60 msgr2=0x5560c87896f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:07.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.177+0000 7f4e8d79d640 1 --2- 172.21.15.32:0/804342267 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560c87d6d60 0x5560c87896f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:07.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.177+0000 7f4e8d79d640 1 -- 172.21.15.32:0/804342267 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560c84062d0 con 0x5560c87b9740 2024-04-24T23:00:07.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.177+0000 7f4e8d79d640 1 --2- 172.21.15.32:0/804342267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5560c87b9740 0x5560c878e550 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f4e800894c0 tx=0x7f4e8005d740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7449838c3b92aba4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:07.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.177+0000 7f4e6e7fc640 1 -- 172.21.15.32:0/804342267 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e800559f0 con 0x5560c87b9740 2024-04-24T23:00:07.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.177+0000 7f4e6e7fc640 1 -- 172.21.15.32:0/804342267 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e80071180 con 0x5560c87b9740 2024-04-24T23:00:07.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.177+0000 7f4e6e7fc640 1 -- 172.21.15.32:0/804342267 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e80055b90 con 0x5560c87b9740 2024-04-24T23:00:07.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.178+0000 7f4e91fee740 1 -- 172.21.15.32:0/804342267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5560c87b9740 msgr2=0x5560c878e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:07.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.178+0000 7f4e91fee740 1 --2- 172.21.15.32:0/804342267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5560c87b9740 0x5560c878e550 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f4e800894c0 tx=0x7f4e8005d740 comp rx=0 tx=0).stop 2024-04-24T23:00:07.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.178+0000 7f4e91fee740 1 -- 172.21.15.32:0/804342267 shutdown_connections 2024-04-24T23:00:07.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.178+0000 7f4e91fee740 1 --2- 172.21.15.32:0/804342267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5560c87b9740 0x5560c878e550 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:07.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.178+0000 7f4e91fee740 1 --2- 172.21.15.32:0/804342267 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5560c87b9e80 0x5560c87b9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:07.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.178+0000 7f4e91fee740 1 --2- 172.21.15.32:0/804342267 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560c87d6d60 0x5560c87896f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:07.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.178+0000 7f4e91fee740 1 -- 172.21.15.32:0/804342267 >> 172.21.15.32:0/804342267 conn(0x5560c878b960 msgr2=0x5560c87d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:07.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.178+0000 7f4e91fee740 1 -- 172.21.15.32:0/804342267 shutdown_connections 2024-04-24T23:00:07.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.178+0000 7f4e91fee740 1 -- 172.21.15.32:0/804342267 wait complete. 2024-04-24T23:00:07.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.179+0000 7f4e91fee740 1 -- start start 2024-04-24T23:00:07.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.179+0000 7f4e91fee740 1 Processor -- start 2024-04-24T23:00:07.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.180+0000 7f4e91fee740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560c87b9e80 0x5560c87e84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:07.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.180+0000 7f4e91fee740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5560c87d6d60 0x5560c87b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:07.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.180+0000 7f4e91fee740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5560c878e470 0x5560c878e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:07.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.180+0000 7f4e91fee740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5560c878db10 con 0x5560c878e470 2024-04-24T23:00:07.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.180+0000 7f4e91fee740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5560c86adcf0 con 0x5560c87b9e80 2024-04-24T23:00:07.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.180+0000 7f4e91fee740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x5560c878cf00 con 0x5560c87d6d60 2024-04-24T23:00:07.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.180+0000 7f4e91fee740 10 client.0 _ll_register_callbacks cb 0x5560c86ad6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:00:07.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.180+0000 7f4e8d79d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5560c878e470 0x5560c878e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:07.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.180+0000 7f4e8d79d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5560c878e470 0x5560c878e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60186/0 (socket says 172.21.15.32:60186) 2024-04-24T23:00:07.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.180+0000 7f4e8d79d640 1 -- 172.21.15.32:0/500672790 learned_addr learned my addr 172.21.15.32:0/500672790 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:07.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.180+0000 7f4e8cf9c640 1 --2- 172.21.15.32:0/500672790 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560c87b9e80 0x5560c87e84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:07.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.180+0000 7f4e87fff640 1 --2- 172.21.15.32:0/500672790 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5560c87d6d60 0x5560c87b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:07.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.180+0000 7f4e8d79d640 1 -- 172.21.15.32:0/500672790 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5560c87d6d60 msgr2=0x5560c87b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:07.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.180+0000 7f4e8d79d640 1 --2- 172.21.15.32:0/500672790 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5560c87d6d60 0x5560c87b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:07.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.180+0000 7f4e8d79d640 1 -- 172.21.15.32:0/500672790 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560c87b9e80 msgr2=0x5560c87e84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:07.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.180+0000 7f4e8d79d640 1 --2- 172.21.15.32:0/500672790 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560c87b9e80 0x5560c87e84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:07.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.180+0000 7f4e8d79d640 1 -- 172.21.15.32:0/500672790 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560c84d06b0 con 0x5560c878e470 2024-04-24T23:00:07.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.180+0000 7f4e8d79d640 1 --2- 172.21.15.32:0/500672790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5560c878e470 0x5560c878e8f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4e80055790 tx=0x7f4e80017070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=646dcb4136c301f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:07.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.180+0000 7f4e677fe640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-24T23:00:07.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.180+0000 7f4e677fe640 1 -- 172.21.15.32:0/500672790 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e80088070 con 0x5560c878e470 2024-04-24T23:00:07.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.180+0000 7f4e677fe640 1 -- 172.21.15.32:0/500672790 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f4e80078300 con 0x5560c878e470 2024-04-24T23:00:07.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.180+0000 7f4e91fee740 1 -- 172.21.15.32:0/500672790 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e80071180 con 0x5560c878e470 2024-04-24T23:00:07.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.180+0000 7f4e677fe640 1 -- 172.21.15.32:0/500672790 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) ==== 24+0+0 (secure 0 0 0) 0x7f4e80071330 con 0x5560c878e470 2024-04-24T23:00:07.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.180+0000 7f4e91fee740 10 client.5325 fetch_fsmap learned FSMap version 13 2024-04-24T23:00:07.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.180+0000 7f4e91fee740 1 -- 172.21.15.32:0/500672790 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) -- 0x5560c87ba830 con 0x5560c878e470 2024-04-24T23:00:07.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.181+0000 7f4e677fe640 1 -- 172.21.15.32:0/500672790 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e80071180 con 0x5560c878e470 2024-04-24T23:00:07.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.181+0000 7f4e677fe640 1 -- 172.21.15.32:0/500672790 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 13) ==== 3494+0+0 (secure 0 0 0) 0x7f4e80061a20 con 0x5560c878e470 2024-04-24T23:00:07.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.181+0000 7f4e91fee740 10 client.5325 fetch_fsmap finished waiting for FSMap version 13 2024-04-24T23:00:07.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.181+0000 7f4e91fee740 10 client.5325 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T23:00:07.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.181+0000 7f4e91fee740 20 client.5325 populate_metadata read hostname 'smithi032' 2024-04-24T23:00:07.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.181+0000 7f4e91fee740 1 --2- 172.21.15.32:0/500672790 >> [v2:172.21.15.121:6834/515107694,v1:172.21.15.121:6837/515107694] conn(0x5560c87d1ea0 0x5560c87d2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:07.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.181+0000 7f4e91fee740 4 client.5325 mds_command: new command op to 5112 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:00:07.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.181+0000 7f4e91fee740 1 -- 172.21.15.32:0/500672790 --> [v2:172.21.15.121:6834/515107694,v1:172.21.15.121:6837/515107694] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560c86ff440 con 0x5560c87d1ea0 2024-04-24T23:00:07.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.181+0000 7f4e8cf9c640 1 --2- 172.21.15.32:0/500672790 >> [v2:172.21.15.121:6834/515107694,v1:172.21.15.121:6837/515107694] conn(0x5560c87d1ea0 0x5560c87d2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:07.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.181+0000 7f4e8cf9c640 1 --2- 172.21.15.32:0/500672790 >> [v2:172.21.15.121:6834/515107694,v1:172.21.15.121:6837/515107694] conn(0x5560c87d1ea0 0x5560c87d2280 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33783566e413dcde server_cookie=c2bed7ae738bba8b in_seq=0 out_seq=0 2024-04-24T23:00:07.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.181+0000 7f4e677fe640 10 client.5325 ms_handle_connect on v2:172.21.15.121:6834/515107694 2024-04-24T23:00:07.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.182+0000 7f4e677fe640 1 -- 172.21.15.32:0/500672790 <== mds.0 v2:172.21.15.121:6834/515107694 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5560c86ff440 con 0x5560c87d1ea0 2024-04-24T23:00:07.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.182+0000 7f4e677fe640 10 client.5325 handle_command_reply: tid=0 2024-04-24T23:00:07.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.182+0000 7f4e91fee740 1 client.5325 shutdown 2024-04-24T23:00:07.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.182+0000 7f4e91fee740 1 -- 172.21.15.32:0/500672790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5560c878e470 msgr2=0x5560c878e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:07.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.182+0000 7f4e91fee740 1 --2- 172.21.15.32:0/500672790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5560c878e470 0x5560c878e8f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4e80055790 tx=0x7f4e80017070 comp rx=0 tx=0).stop 2024-04-24T23:00:07.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.182+0000 7f4e91fee740 1 -- 172.21.15.32:0/500672790 shutdown_connections 2024-04-24T23:00:07.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.182+0000 7f4e91fee740 1 --2- 172.21.15.32:0/500672790 >> [v2:172.21.15.121:6834/515107694,v1:172.21.15.121:6837/515107694] conn(0x5560c87d1ea0 0x5560c87d2280 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:07.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.182+0000 7f4e91fee740 1 --2- 172.21.15.32:0/500672790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5560c878e470 0x5560c878e8f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:07.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.182+0000 7f4e91fee740 1 --2- 172.21.15.32:0/500672790 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5560c87d6d60 0x5560c87b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:07.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.182+0000 7f4e91fee740 1 --2- 172.21.15.32:0/500672790 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560c87b9e80 0x5560c87e84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:07.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.182+0000 7f4e91fee740 1 -- 172.21.15.32:0/500672790 >> 172.21.15.32:0/500672790 conn(0x5560c87b9740 msgr2=0x5560c87d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:07.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.182+0000 7f4e677fe640 0 client.5325 ms_handle_reset on v2:172.21.15.121:6834/515107694 2024-04-24T23:00:07.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.182+0000 7f4e91fee740 1 -- 172.21.15.32:0/500672790 shutdown_connections 2024-04-24T23:00:07.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.183+0000 7f4e91fee740 1 -- 172.21.15.32:0/500672790 wait complete. 2024-04-24T23:00:07.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.183+0000 7f4e91fee740 20 client.5325 trim_cache size 0 max 16384 2024-04-24T23:00:07.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.201+0000 7f4e91fee740 1 Processor -- start 2024-04-24T23:00:07.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.201+0000 7f4e91fee740 1 -- start start 2024-04-24T23:00:07.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.202+0000 7f4e91fee740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560c878e470 0x5560c87e3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:07.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.202+0000 7f4e91fee740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5560c87b9e80 0x5560c87e3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:07.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.202+0000 7f4e91fee740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5560c87d6d60 0x5560c87b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:07.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.202+0000 7f4e91fee740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4e881fff90 con 0x5560c87d6d60 2024-04-24T23:00:07.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.202+0000 7f4e91fee740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4e880b8690 con 0x5560c878e470 2024-04-24T23:00:07.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.202+0000 7f4e91fee740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f4e880b8ce0 con 0x5560c87b9e80 2024-04-24T23:00:07.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.202+0000 7f4e8d79d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5560c87d6d60 0x5560c87b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:07.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.202+0000 7f4e8d79d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5560c87d6d60 0x5560c87b3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60196/0 (socket says 172.21.15.32:60196) 2024-04-24T23:00:07.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.202+0000 7f4e8d79d640 1 -- 172.21.15.32:0/3968701763 learned_addr learned my addr 172.21.15.32:0/3968701763 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:07.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.202+0000 7f4e8cf9c640 1 --2- 172.21.15.32:0/3968701763 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560c878e470 0x5560c87e3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:07.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.202+0000 7f4e87fff640 1 --2- 172.21.15.32:0/3968701763 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5560c87b9e80 0x5560c87e3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:07.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.202+0000 7f4e8d79d640 1 -- 172.21.15.32:0/3968701763 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5560c87b9e80 msgr2=0x5560c87e3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:07.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.202+0000 7f4e8d79d640 1 --2- 172.21.15.32:0/3968701763 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5560c87b9e80 0x5560c87e3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:07.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.202+0000 7f4e8d79d640 1 -- 172.21.15.32:0/3968701763 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560c878e470 msgr2=0x5560c87e3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:07.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.202+0000 7f4e8d79d640 1 --2- 172.21.15.32:0/3968701763 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560c878e470 0x5560c87e3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:07.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.202+0000 7f4e8d79d640 1 -- 172.21.15.32:0/3968701763 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560c86959e0 con 0x5560c87d6d60 2024-04-24T23:00:07.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.202+0000 7f4e8cf9c640 1 --2- 172.21.15.32:0/3968701763 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560c878e470 0x5560c87e3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:00:07.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.202+0000 7f4e8d79d640 1 --2- 172.21.15.32:0/3968701763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5560c87d6d60 0x5560c87b3de0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4e8000d930 tx=0x7f4e8000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ccaab1fe8293336 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:07.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.202+0000 7f4e6dffb640 1 -- 172.21.15.32:0/3968701763 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e80071180 con 0x5560c87d6d60 2024-04-24T23:00:07.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.202+0000 7f4e6dffb640 1 -- 172.21.15.32:0/3968701763 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e80076070 con 0x5560c87d6d60 2024-04-24T23:00:07.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.202+0000 7f4e6dffb640 1 -- 172.21.15.32:0/3968701763 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e80089490 con 0x5560c87d6d60 2024-04-24T23:00:07.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.203+0000 7f4e91fee740 1 -- 172.21.15.32:0/3968701763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5560c87d6d60 msgr2=0x5560c87b3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:07.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.203+0000 7f4e91fee740 1 --2- 172.21.15.32:0/3968701763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5560c87d6d60 0x5560c87b3de0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4e8000d930 tx=0x7f4e8000b040 comp rx=0 tx=0).stop 2024-04-24T23:00:07.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.203+0000 7f4e91fee740 1 -- 172.21.15.32:0/3968701763 shutdown_connections 2024-04-24T23:00:07.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.203+0000 7f4e91fee740 1 --2- 172.21.15.32:0/3968701763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5560c87d6d60 0x5560c87b3de0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:07.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.203+0000 7f4e91fee740 1 --2- 172.21.15.32:0/3968701763 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5560c87b9e80 0x5560c87e3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:07.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.203+0000 7f4e91fee740 1 --2- 172.21.15.32:0/3968701763 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560c878e470 0x5560c87e3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:07.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.203+0000 7f4e91fee740 1 -- 172.21.15.32:0/3968701763 >> 172.21.15.32:0/3968701763 conn(0x5560c87b9740 msgr2=0x5560c878c500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:07.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.203+0000 7f4e91fee740 1 -- 172.21.15.32:0/3968701763 shutdown_connections 2024-04-24T23:00:07.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.203+0000 7f4e91fee740 1 -- 172.21.15.32:0/3968701763 wait complete. 2024-04-24T23:00:07.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.204+0000 7f4e91fee740 1 -- start start 2024-04-24T23:00:07.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.204+0000 7f4e91fee740 1 Processor -- start 2024-04-24T23:00:07.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.205+0000 7f4e91fee740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5560c878e470 0x5560c87d7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:07.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.205+0000 7f4e91fee740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5560c87b9e80 0x5560c87d7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:07.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.205+0000 7f4e91fee740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560c87d6d60 0x5560c87e2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:07.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.205+0000 7f4e91fee740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5560c87e4860 con 0x5560c87b9e80 2024-04-24T23:00:07.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.205+0000 7f4e91fee740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5560c878db10 con 0x5560c87d6d60 2024-04-24T23:00:07.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.205+0000 7f4e91fee740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x5560c87caf60 con 0x5560c878e470 2024-04-24T23:00:07.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.205+0000 7f4e91fee740 10 client.0 _ll_register_callbacks cb 0x7f4e80059410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:00:07.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.205+0000 7f4e87fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5560c87b9e80 0x5560c87d7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:07.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.205+0000 7f4e87fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5560c87b9e80 0x5560c87d7b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60212/0 (socket says 172.21.15.32:60212) 2024-04-24T23:00:07.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.205+0000 7f4e87fff640 1 -- 172.21.15.32:0/2229891506 learned_addr learned my addr 172.21.15.32:0/2229891506 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:07.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.205+0000 7f4e8d79d640 1 --2- 172.21.15.32:0/2229891506 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560c87d6d60 0x5560c87e2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:07.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.205+0000 7f4e8cf9c640 1 --2- 172.21.15.32:0/2229891506 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5560c878e470 0x5560c87d7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:07.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.205+0000 7f4e87fff640 1 -- 172.21.15.32:0/2229891506 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5560c878e470 msgr2=0x5560c87d7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:07.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.205+0000 7f4e87fff640 1 --2- 172.21.15.32:0/2229891506 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5560c878e470 0x5560c87d7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:07.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.205+0000 7f4e87fff640 1 -- 172.21.15.32:0/2229891506 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560c87d6d60 msgr2=0x5560c87e2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:07.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.205+0000 7f4e87fff640 1 --2- 172.21.15.32:0/2229891506 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560c87d6d60 0x5560c87e2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:07.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.205+0000 7f4e87fff640 1 -- 172.21.15.32:0/2229891506 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560c87d4a10 con 0x5560c87b9e80 2024-04-24T23:00:07.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.205+0000 7f4e87fff640 1 --2- 172.21.15.32:0/2229891506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5560c87b9e80 0x5560c87d7b60 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x5560c87d5300 tx=0x7f4e7c017800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37f8d52e0304e602 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:07.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.205+0000 7f4e677fe640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-24T23:00:07.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.205+0000 7f4e677fe640 1 -- 172.21.15.32:0/2229891506 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c020440 con 0x5560c87b9e80 2024-04-24T23:00:07.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.205+0000 7f4e677fe640 1 -- 172.21.15.32:0/2229891506 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f4e7c0205e0 con 0x5560c87b9e80 2024-04-24T23:00:07.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.205+0000 7f4e91fee740 1 -- 172.21.15.32:0/2229891506 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e80076070 con 0x5560c87b9e80 2024-04-24T23:00:07.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.205+0000 7f4e677fe640 1 -- 172.21.15.32:0/2229891506 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) ==== 24+0+0 (secure 0 0 0) 0x7f4e7c028050 con 0x5560c87b9e80 2024-04-24T23:00:07.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.205+0000 7f4e91fee740 10 client.5331 fetch_fsmap learned FSMap version 13 2024-04-24T23:00:07.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.205+0000 7f4e91fee740 1 -- 172.21.15.32:0/2229891506 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) -- 0x5560c87d4cf0 con 0x5560c87b9e80 2024-04-24T23:00:07.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.206+0000 7f4e677fe640 1 -- 172.21.15.32:0/2229891506 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e80076070 con 0x5560c87b9e80 2024-04-24T23:00:07.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.206+0000 7f4e677fe640 1 -- 172.21.15.32:0/2229891506 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 13) ==== 3494+0+0 (secure 0 0 0) 0x7f4e7c02b020 con 0x5560c87b9e80 2024-04-24T23:00:07.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.206+0000 7f4e91fee740 10 client.5331 fetch_fsmap finished waiting for FSMap version 13 2024-04-24T23:00:07.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.206+0000 7f4e91fee740 10 client.5331 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T23:00:07.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.206+0000 7f4e91fee740 20 client.5331 populate_metadata read hostname 'smithi032' 2024-04-24T23:00:07.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.206+0000 7f4e91fee740 1 --2- 172.21.15.32:0/2229891506 >> [v2:172.21.15.121:6834/515107694,v1:172.21.15.121:6837/515107694] conn(0x5560c87c9b90 0x5560c87c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:07.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.206+0000 7f4e91fee740 4 client.5331 mds_command: new command op to 5112 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:00:07.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.206+0000 7f4e91fee740 1 -- 172.21.15.32:0/2229891506 --> [v2:172.21.15.121:6834/515107694,v1:172.21.15.121:6837/515107694] -- command(tid 0: {"prefix": "status"}) -- 0x5560c86f9790 con 0x5560c87c9b90 2024-04-24T23:00:07.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.206+0000 7f4e8cf9c640 1 --2- 172.21.15.32:0/2229891506 >> [v2:172.21.15.121:6834/515107694,v1:172.21.15.121:6837/515107694] conn(0x5560c87c9b90 0x5560c87c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:07.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.206+0000 7f4e677fe640 10 client.5331 ms_handle_connect on v2:172.21.15.121:6834/515107694 2024-04-24T23:00:07.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.206+0000 7f4e8cf9c640 1 --2- 172.21.15.32:0/2229891506 >> [v2:172.21.15.121:6834/515107694,v1:172.21.15.121:6837/515107694] conn(0x5560c87c9b90 0x5560c87c9f70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f195688b49be2a0 server_cookie=f0ccb04dd966a9af in_seq=0 out_seq=0 2024-04-24T23:00:07.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.207+0000 7f4e677fe640 1 -- 172.21.15.32:0/2229891506 <== mds.0 v2:172.21.15.121:6834/515107694 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x5560c86f9790 con 0x5560c87c9b90 2024-04-24T23:00:07.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.207+0000 7f4e677fe640 10 client.5331 handle_command_reply: tid=0 2024-04-24T23:00:07.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.207+0000 7f4e91fee740 1 client.5331 shutdown 2024-04-24T23:00:07.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.207+0000 7f4e91fee740 1 -- 172.21.15.32:0/2229891506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5560c87b9e80 msgr2=0x5560c87d7b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:07.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.207+0000 7f4e91fee740 1 --2- 172.21.15.32:0/2229891506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5560c87b9e80 0x5560c87d7b60 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x5560c87d5300 tx=0x7f4e7c017800 comp rx=0 tx=0).stop 2024-04-24T23:00:07.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.207+0000 7f4e91fee740 1 -- 172.21.15.32:0/2229891506 shutdown_connections 2024-04-24T23:00:07.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.207+0000 7f4e91fee740 1 --2- 172.21.15.32:0/2229891506 >> [v2:172.21.15.121:6834/515107694,v1:172.21.15.121:6837/515107694] conn(0x5560c87c9b90 0x5560c87c9f70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:07.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.207+0000 7f4e91fee740 1 --2- 172.21.15.32:0/2229891506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5560c87b9e80 0x5560c87d7b60 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:07.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.207+0000 7f4e677fe640 0 client.5331 ms_handle_reset on v2:172.21.15.121:6834/515107694 2024-04-24T23:00:07.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.207+0000 7f4e91fee740 1 --2- 172.21.15.32:0/2229891506 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560c87d6d60 0x5560c87e2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:07.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.207+0000 7f4e91fee740 1 --2- 172.21.15.32:0/2229891506 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5560c878e470 0x5560c87d7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:07.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.207+0000 7f4e91fee740 1 -- 172.21.15.32:0/2229891506 >> 172.21.15.32:0/2229891506 conn(0x5560c87b9740 msgr2=0x5560c87bccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:07.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.207+0000 7f4e91fee740 1 -- 172.21.15.32:0/2229891506 shutdown_connections 2024-04-24T23:00:07.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.207+0000 7f4e91fee740 1 -- 172.21.15.32:0/2229891506 wait complete. 2024-04-24T23:00:07.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.207+0000 7f4e91fee740 20 client.5331 trim_cache size 0 max 16384 2024-04-24T23:00:07.214 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:00:07.214 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", 2024-04-24T23:00:07.214 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2024-04-24T23:00:07.215 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 5112, 2024-04-24T23:00:07.215 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-04-24T23:00:07.215 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-04-24T23:00:07.215 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-04-24T23:00:07.215 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 9.1022902119999998, 2024-04-24T23:00:07.215 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 13, 2024-04-24T23:00:07.215 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 32, 2024-04-24T23:00:07.215 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 32, 2024-04-24T23:00:07.215 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 15.971942322 2024-04-24T23:00:07.215 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:00:07.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.207+0000 7f4e8df9e640 1 -- 172.21.15.32:0/4150272228 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4e7406be60 msgr2=0x7f4e7406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:07.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.207+0000 7f4e8df9e640 1 --2- 172.21.15.32:0/4150272228 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4e7406be60 0x7f4e7406e320 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4e78034000 tx=0x7f4e7802e000 comp rx=0 tx=0).stop 2024-04-24T23:00:07.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.207+0000 7f4e8df9e640 1 -- 172.21.15.32:0/4150272228 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e88147dd0 msgr2=0x7f4e881612a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:07.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.207+0000 7f4e8df9e640 1 --2- 172.21.15.32:0/4150272228 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e88147dd0 0x7f4e881612a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4e80007ea0 tx=0x7f4e80007f90 comp rx=0 tx=0).stop 2024-04-24T23:00:07.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.208+0000 7f4e8df9e640 1 -- 172.21.15.32:0/4150272228 shutdown_connections 2024-04-24T23:00:07.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.208+0000 7f4e8df9e640 1 --2- 172.21.15.32:0/4150272228 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4e7406be60 0x7f4e7406e320 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:07.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.208+0000 7f4e8df9e640 1 --2- 172.21.15.32:0/4150272228 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e88147dd0 0x7f4e881612a0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:07.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.208+0000 7f4e8df9e640 1 --2- 172.21.15.32:0/4150272228 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4e880c9fd0 0x7f4e8813e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:07.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.208+0000 7f4e8df9e640 1 --2- 172.21.15.32:0/4150272228 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4e880b6b00 0x7f4e8813ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:07.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.208+0000 7f4e8df9e640 1 -- 172.21.15.32:0/4150272228 >> 172.21.15.32:0/4150272228 conn(0x7f4e88071730 msgr2=0x7f4e881464e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:07.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.208+0000 7f4e8df9e640 1 -- 172.21.15.32:0/4150272228 shutdown_connections 2024-04-24T23:00:07.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.208+0000 7f4e8df9e640 1 -- 172.21.15.32:0/4150272228 wait complete. 2024-04-24T23:00:07.220 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T23:00:07.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.320+0000 7f6557a2a640 1 Processor -- start 2024-04-24T23:00:07.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.320+0000 7f6557a2a640 1 -- start start 2024-04-24T23:00:07.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.321+0000 7f6557a2a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f65500ae140 0x7f65500ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:07.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.321+0000 7f6557a2a640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6550074f00 con 0x7f655015c8a0 2024-04-24T23:00:07.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.321+0000 7f6557a2a640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65500b0de0 con 0x7f6550074920 2024-04-24T23:00:07.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.321+0000 7f6557a2a640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f655006ca70 con 0x7f65500ae140 2024-04-24T23:00:07.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.321+0000 7f65557c6640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f655015c8a0 0x7f6550074210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39282/0 (socket says 172.21.15.32:39282) 2024-04-24T23:00:07.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.321+0000 7f65557c6640 1 -- 172.21.15.32:0/316979398 learned_addr learned my addr 172.21.15.32:0/316979398 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:00:07.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.321+0000 7f6555fc7640 1 --2- 172.21.15.32:0/316979398 >> v2:172.21.15.32:3300/0 conn(0x7f65500ae140 0x7f65500ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:07.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.321+0000 7f6547fff640 1 -- 172.21.15.32:0/316979398 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2948896091 0 0) 0x7f6550074f00 con 0x7f655015c8a0 2024-04-24T23:00:07.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.321+0000 7f6547fff640 1 -- 172.21.15.32:0/316979398 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f653c003520 con 0x7f655015c8a0 2024-04-24T23:00:07.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.321+0000 7f6555fc7640 1 -- 172.21.15.32:0/316979398 >> v1:172.21.15.121:6789/0 conn(0x7f6550074920 legacy=0x7f65500b1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:07.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.321+0000 7f6555fc7640 1 -- 172.21.15.32:0/316979398 >> v1:172.21.15.32:6789/0 conn(0x7f655015c8a0 legacy=0x7f6550074210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:07.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.321+0000 7f6555fc7640 1 -- 172.21.15.32:0/316979398 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65500b10c0 con 0x7f65500ae140 2024-04-24T23:00:07.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.322+0000 7f6555fc7640 1 --2- 172.21.15.32:0/316979398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65500ae140 0x7f65500ce7a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6548009d90 tx=0x7f654802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a5aef7786d4b1e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:07.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.322+0000 7f6547fff640 1 -- 172.21.15.32:0/316979398 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6548007740 con 0x7f65500ae140 2024-04-24T23:00:07.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.322+0000 7f6547fff640 1 -- 172.21.15.32:0/316979398 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65480078e0 con 0x7f65500ae140 2024-04-24T23:00:07.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.322+0000 7f6547fff640 1 -- 172.21.15.32:0/316979398 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6548007cb0 con 0x7f65500ae140 2024-04-24T23:00:07.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.323+0000 7f6557a2a640 1 -- 172.21.15.32:0/316979398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65500ae140 msgr2=0x7f65500ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:07.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.323+0000 7f6557a2a640 1 --2- 172.21.15.32:0/316979398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65500ae140 0x7f65500ce7a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6548009d90 tx=0x7f654802a660 comp rx=0 tx=0).stop 2024-04-24T23:00:07.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.323+0000 7f6557a2a640 1 -- 172.21.15.32:0/316979398 shutdown_connections 2024-04-24T23:00:07.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.323+0000 7f6557a2a640 1 --2- 172.21.15.32:0/316979398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65500ae140 0x7f65500ce7a0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:07.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.323+0000 7f6557a2a640 1 -- 172.21.15.32:0/316979398 >> 172.21.15.32:0/316979398 conn(0x7f65500b2e60 msgr2=0x7f65500b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:07.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.323+0000 7f6557a2a640 1 -- 172.21.15.32:0/316979398 shutdown_connections 2024-04-24T23:00:07.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.323+0000 7f6557a2a640 1 -- 172.21.15.32:0/316979398 wait complete. 2024-04-24T23:00:07.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.324+0000 7f6557a2a640 1 Processor -- start 2024-04-24T23:00:07.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.324+0000 7f6557a2a640 1 -- start start 2024-04-24T23:00:07.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.325+0000 7f6557a2a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6550074920 0x7f65500c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:07.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.325+0000 7f6557a2a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65500ae140 0x7f6550158f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:07.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.325+0000 7f6557a2a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f655015c8a0 0x7f6550159450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:07.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.325+0000 7f6557a2a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f655015cc80 con 0x7f65500ae140 2024-04-24T23:00:07.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.325+0000 7f6557a2a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f65500b1680 con 0x7f6550074920 2024-04-24T23:00:07.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.325+0000 7f6557a2a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f65500c52b0 con 0x7f655015c8a0 2024-04-24T23:00:07.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.325+0000 7f6554fc5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65500ae140 0x7f6550158f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:07.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.325+0000 7f6554fc5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65500ae140 0x7f6550158f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60234/0 (socket says 172.21.15.32:60234) 2024-04-24T23:00:07.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.325+0000 7f6554fc5640 1 -- 172.21.15.32:0/713197544 learned_addr learned my addr 172.21.15.32:0/713197544 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:07.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.325+0000 7f65557c6640 1 --2- 172.21.15.32:0/713197544 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6550074920 0x7f65500c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:07.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.325+0000 7f6555fc7640 1 --2- 172.21.15.32:0/713197544 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f655015c8a0 0x7f6550159450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:07.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.325+0000 7f6554fc5640 1 -- 172.21.15.32:0/713197544 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f655015c8a0 msgr2=0x7f6550159450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:07.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.325+0000 7f6554fc5640 1 --2- 172.21.15.32:0/713197544 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f655015c8a0 0x7f6550159450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:07.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.325+0000 7f6554fc5640 1 -- 172.21.15.32:0/713197544 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6550074920 msgr2=0x7f65500c83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:07.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.325+0000 7f6554fc5640 1 --2- 172.21.15.32:0/713197544 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6550074920 0x7f65500c83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:07.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.325+0000 7f6554fc5640 1 -- 172.21.15.32:0/713197544 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65500c8d30 con 0x7f65500ae140 2024-04-24T23:00:07.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.325+0000 7f6554fc5640 1 --2- 172.21.15.32:0/713197544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65500ae140 0x7f6550158f10 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f654000c400 tx=0x7f654000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:07.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.325+0000 7f6545ffb640 1 -- 172.21.15.32:0/713197544 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65400052d0 con 0x7f65500ae140 2024-04-24T23:00:07.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.325+0000 7f6545ffb640 1 -- 172.21.15.32:0/713197544 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6540005490 con 0x7f65500ae140 2024-04-24T23:00:07.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.325+0000 7f6557a2a640 1 -- 172.21.15.32:0/713197544 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65500c4980 con 0x7f65500ae140 2024-04-24T23:00:07.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.325+0000 7f6545ffb640 1 -- 172.21.15.32:0/713197544 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6540007260 con 0x7f65500ae140 2024-04-24T23:00:07.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.326+0000 7f6557a2a640 1 -- 172.21.15.32:0/713197544 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65500c4f40 con 0x7f65500ae140 2024-04-24T23:00:07.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.327+0000 7f6557a2a640 1 -- 172.21.15.32:0/713197544 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65500b6570 con 0x7f65500ae140 2024-04-24T23:00:07.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.327+0000 7f6545ffb640 1 -- 172.21.15.32:0/713197544 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6540007400 con 0x7f65500ae140 2024-04-24T23:00:07.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.327+0000 7f6545ffb640 1 --2- 172.21.15.32:0/713197544 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f653406be30 0x7f653406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:07.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.327+0000 7f6545ffb640 1 -- 172.21.15.32:0/713197544 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f654008a400 con 0x7f65500ae140 2024-04-24T23:00:07.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.328+0000 7f65557c6640 1 --2- 172.21.15.32:0/713197544 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f653406be30 0x7f653406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:07.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.328+0000 7f65557c6640 1 --2- 172.21.15.32:0/713197544 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f653406be30 0x7f653406e2f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f654c0011e0 tx=0x7f654c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:07.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.332+0000 7f6545ffb640 1 -- 172.21.15.32:0/713197544 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f65500b6570 con 0x7f65500ae140 2024-04-24T23:00:07.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.550+0000 7f6557a2a640 1 -- 172.21.15.32:0/713197544 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6550068430 con 0x7f65500ae140 2024-04-24T23:00:07.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.551+0000 7f6545ffb640 1 -- 172.21.15.32:0/713197544 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v13) ==== 53+0+73 (secure 0 0 0) 0x7f6540058da0 con 0x7f65500ae140 2024-04-24T23:00:07.552 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T23:00:07.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.553+0000 7f6557a2a640 1 -- 172.21.15.32:0/713197544 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f653406be30 msgr2=0x7f653406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:07.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.553+0000 7f6557a2a640 1 --2- 172.21.15.32:0/713197544 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f653406be30 0x7f653406e2f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f654c0011e0 tx=0x7f654c009040 comp rx=0 tx=0).stop 2024-04-24T23:00:07.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.553+0000 7f6557a2a640 1 -- 172.21.15.32:0/713197544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65500ae140 msgr2=0x7f6550158f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:07.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.553+0000 7f6557a2a640 1 --2- 172.21.15.32:0/713197544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65500ae140 0x7f6550158f10 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f654000c400 tx=0x7f654000c8c0 comp rx=0 tx=0).stop 2024-04-24T23:00:07.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.553+0000 7f6557a2a640 1 -- 172.21.15.32:0/713197544 shutdown_connections 2024-04-24T23:00:07.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.553+0000 7f6557a2a640 1 --2- 172.21.15.32:0/713197544 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f653406be30 0x7f653406e2f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:07.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.553+0000 7f6557a2a640 1 --2- 172.21.15.32:0/713197544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65500ae140 0x7f6550158f10 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:07.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.553+0000 7f6557a2a640 1 --2- 172.21.15.32:0/713197544 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f655015c8a0 0x7f6550159450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:07.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.553+0000 7f6557a2a640 1 --2- 172.21.15.32:0/713197544 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6550074920 0x7f65500c83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:07.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.553+0000 7f6557a2a640 1 -- 172.21.15.32:0/713197544 >> 172.21.15.32:0/713197544 conn(0x7f65500b2e60 msgr2=0x7f655015b730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:07.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.553+0000 7f6557a2a640 1 -- 172.21.15.32:0/713197544 shutdown_connections 2024-04-24T23:00:07.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07.553+0000 7f6557a2a640 1 -- 172.21.15.32:0/713197544 wait complete. 2024-04-24T23:00:07.565 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-24T23:00:07.565 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T23:00:07.565 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2024-04-24T23:00:07.565 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-24T23:00:07.565 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T23:00:07.565 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-24T23:00:07.565 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-24T23:00:07.565 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T23:00:07.566 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:00:07.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:00:07.566 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2024-04-24T23:00:07.582 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T23:00:07.582 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T23:00:07.582 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T23:00:07.582 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:00:07.582 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2024-04-24T23:00:07.582 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:00:07.582 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:00:07.583 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2024-04-24T23:00:07.583 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2024-04-24T23:00:07.583 INFO:teuthology.orchestra.run.smithi032.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:00:07.583 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2024-04-24T23:00:07.583 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2024-04-24T23:00:07.583 INFO:teuthology.orchestra.run.smithi032.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T23:00:07.583 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:00:07.583 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2024-04-24T23:00:07.583 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-24T23:00:07.583 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 42701sec preferred_lft 42701sec 2024-04-24T23:00:07.583 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link noprefixroute 2024-04-24T23:00:07.583 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:00:07.583 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:00:07.583 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:00:07.584 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2024-04-24T23:00:07.584 INFO:teuthology.orchestra.run.smithi032.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T23:00:07.584 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 72:d4:96:00:43:8c brd ff:ff:ff:ff:ff:ff 2024-04-24T23:00:07.584 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-24T23:00:07.584 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:00:07.584 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::f4c6:20ff:fe22:2647/64 scope link 2024-04-24T23:00:07.584 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:00:07.584 INFO:teuthology.orchestra.run.smithi032.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:00:07.584 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 72:d4:96:00:43:8c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T23:00:07.584 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::70d4:96ff:fe00:438c/64 scope link 2024-04-24T23:00:07.585 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:00:07.585 INFO:teuthology.orchestra.run.smithi032.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:00:07.585 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether e6:6b:2a:90:1a:f8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T23:00:07.585 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::e46b:2aff:fe90:1af8/64 scope link 2024-04-24T23:00:07.585 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:00:07.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:00:07.585 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T23:00:07.645 INFO:teuthology.orchestra.run.smithi032.stdout:1 2024-04-24T23:00:07.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:00:07.647 DEBUG:teuthology.orchestra.run.smithi032:> route 2024-04-24T23:00:07.703 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2024-04-24T23:00:07.703 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T23:00:07.703 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-24T23:00:07.703 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-24T23:00:07.704 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T23:00:07.704 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:00:07.704 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-24T23:00:07.704 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-24T23:00:07.704 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-24T23:00:07.704 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-24T23:00:07.704 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-24T23:00:07.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:00:07.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:00:07.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:00:07.830 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-04-24T23:00:07.845 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-24T23:00:07.855 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-24T23:00:07.855 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:00:07.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:00:07.855 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2024-04-24T23:00:07.910 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:00:07.910 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-24T23:00:07.966 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:00:07.966 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:00:07.966 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-24T23:00:08.021 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-24T23:00:08.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:00:08.021 DEBUG:teuthology.orchestra.run.smithi032:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:00:08.105 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:00:08.105 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-24T23:00:08.126 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:00:08.126 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-24T23:00:08.126 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.121:3300,172.21.15.121:3301" 2024-04-24T23:00:08.126 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: trying mount with new device syntax: 0@eb1b7a01-6f65-4534-9a12-83011862ba51.cephfs=/ 2024-04-24T23:00:08.126 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.32:3300/172.21.15.121:3300/172.21.15.121:3301" will pass to kernel 2024-04-24T23:00:08.127 INFO:teuthology.orchestra.run.smithi032.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-24T23:00:08.127 INFO:teuthology.orchestra.run.smithi032.stdout: You just mounted a file system that supports labels which does not 2024-04-24T23:00:08.127 INFO:teuthology.orchestra.run.smithi032.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-24T23:00:08.127 INFO:teuthology.orchestra.run.smithi032.stdout: applications will generate AVC messages and not be allowed access to 2024-04-24T23:00:08.127 INFO:teuthology.orchestra.run.smithi032.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-24T23:00:08.128 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-24T23:00:08.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:00:08.128 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-24T23:00:08.194 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:00:08.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:00:08.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:00:08.297 INFO:teuthology.orchestra.run.smithi032.stdout:client4853 2024-04-24T23:00:08.297 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:00:08.297 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:00:08.313 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T23:00:08.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.452+0000 7f7f29fd5640 1 Processor -- start 2024-04-24T23:00:08.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.453+0000 7f7f29fd5640 1 -- start start 2024-04-24T23:00:08.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.453+0000 7f7f29fd5640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7f2406b820 0x7f7f240d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:08.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.453+0000 7f7f29fd5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7f240d4210 0x7f7f240b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:08.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.453+0000 7f7f29fd5640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f240d3170 con 0x7f7f2415e9b0 2024-04-24T23:00:08.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.453+0000 7f7f29fd5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f7f240d3450 con 0x7f7f240d4210 2024-04-24T23:00:08.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.453+0000 7f7f29fd5640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f7f240d3730 con 0x7f7f2406b820 2024-04-24T23:00:08.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.453+0000 7f7f23fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7f240d4210 0x7f7f240b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:08.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.454+0000 7f7f23fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7f240d4210 0x7f7f240b78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60242/0 (socket says 172.21.15.32:60242) 2024-04-24T23:00:08.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.454+0000 7f7f23fff640 1 -- 172.21.15.32:0/3390850043 learned_addr learned my addr 172.21.15.32:0/3390850043 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:08.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.454+0000 7f7f22ffd640 1 --2- 172.21.15.32:0/3390850043 >> v2:172.21.15.121:3300/0 conn(0x7f7f2406b820 0x7f7f240d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:08.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.454+0000 7f7f23fff640 1 -- 172.21.15.32:0/3390850043 >> v2:172.21.15.121:3300/0 conn(0x7f7f2406b820 msgr2=0x7f7f240d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:08.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.454+0000 7f7f23fff640 1 --2- 172.21.15.32:0/3390850043 >> v2:172.21.15.121:3300/0 conn(0x7f7f2406b820 0x7f7f240d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:08.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.454+0000 7f7f23fff640 1 -- 172.21.15.32:0/3390850043 >> v1:172.21.15.121:6789/0 conn(0x7f7f2415e9b0 legacy=0x7f7f2406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:08.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.454+0000 7f7f23fff640 1 -- 172.21.15.32:0/3390850043 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f240d3a10 con 0x7f7f240d4210 2024-04-24T23:00:08.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.454+0000 7f7f23fff640 1 --2- 172.21.15.32:0/3390850043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f240d4210 0x7f7f240b78f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7f240730f0 tx=0x7f7f1c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bfb3aa6a5a2c7a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:08.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.455+0000 7f7f227fc640 1 -- 172.21.15.32:0/3390850043 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c036070 con 0x7f7f240d4210 2024-04-24T23:00:08.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.455+0000 7f7f227fc640 1 -- 172.21.15.32:0/3390850043 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f1c031600 con 0x7f7f240d4210 2024-04-24T23:00:08.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.455+0000 7f7f227fc640 1 -- 172.21.15.32:0/3390850043 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c0319d0 con 0x7f7f240d4210 2024-04-24T23:00:08.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.455+0000 7f7f29fd5640 1 -- 172.21.15.32:0/3390850043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f240d4210 msgr2=0x7f7f240b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:08.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.455+0000 7f7f29fd5640 1 --2- 172.21.15.32:0/3390850043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f240d4210 0x7f7f240b78f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7f240730f0 tx=0x7f7f1c02a6a0 comp rx=0 tx=0).stop 2024-04-24T23:00:08.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.455+0000 7f7f29fd5640 1 -- 172.21.15.32:0/3390850043 shutdown_connections 2024-04-24T23:00:08.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.455+0000 7f7f29fd5640 1 --2- 172.21.15.32:0/3390850043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f240d4210 0x7f7f240b78f0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:08.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.455+0000 7f7f29fd5640 1 --2- 172.21.15.32:0/3390850043 >> v2:172.21.15.121:3300/0 conn(0x7f7f2406b820 0x7f7f240d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:08.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.455+0000 7f7f29fd5640 1 -- 172.21.15.32:0/3390850043 >> 172.21.15.32:0/3390850043 conn(0x7f7f240d11d0 msgr2=0x7f7f240d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:08.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.455+0000 7f7f29fd5640 1 -- 172.21.15.32:0/3390850043 shutdown_connections 2024-04-24T23:00:08.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.455+0000 7f7f29fd5640 1 -- 172.21.15.32:0/3390850043 wait complete. 2024-04-24T23:00:08.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.456+0000 7f7f29fd5640 1 Processor -- start 2024-04-24T23:00:08.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.457+0000 7f7f29fd5640 1 -- start start 2024-04-24T23:00:08.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.457+0000 7f7f29fd5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f2406b820 0x7f7f241e3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:08.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.457+0000 7f7f29fd5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7f240d4210 0x7f7f240ca220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:08.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.457+0000 7f7f29fd5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f2415e9b0 0x7f7f241e44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:08.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.457+0000 7f7f29fd5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7f240ba0e0 con 0x7f7f2406b820 2024-04-24T23:00:08.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.457+0000 7f7f29fd5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7f240d2400 con 0x7f7f2415e9b0 2024-04-24T23:00:08.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.457+0000 7f7f29fd5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f7f240ba260 con 0x7f7f240d4210 2024-04-24T23:00:08.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.457+0000 7f7f237fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f2406b820 0x7f7f241e3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:08.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.457+0000 7f7f237fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f2406b820 0x7f7f241e3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60244/0 (socket says 172.21.15.32:60244) 2024-04-24T23:00:08.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.458+0000 7f7f237fe640 1 -- 172.21.15.32:0/2028080836 learned_addr learned my addr 172.21.15.32:0/2028080836 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:08.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.458+0000 7f7f22ffd640 1 --2- 172.21.15.32:0/2028080836 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7f240d4210 0x7f7f240ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:08.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.458+0000 7f7f23fff640 1 --2- 172.21.15.32:0/2028080836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f2415e9b0 0x7f7f241e44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:08.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.458+0000 7f7f237fe640 1 -- 172.21.15.32:0/2028080836 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7f240d4210 msgr2=0x7f7f240ca220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:08.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.458+0000 7f7f237fe640 1 --2- 172.21.15.32:0/2028080836 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7f240d4210 0x7f7f240ca220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:08.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.458+0000 7f7f237fe640 1 -- 172.21.15.32:0/2028080836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f2415e9b0 msgr2=0x7f7f241e44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:08.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.458+0000 7f7f237fe640 1 --2- 172.21.15.32:0/2028080836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f2415e9b0 0x7f7f241e44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:08.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.458+0000 7f7f237fe640 1 -- 172.21.15.32:0/2028080836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f24148b00 con 0x7f7f2406b820 2024-04-24T23:00:08.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.458+0000 7f7f237fe640 1 --2- 172.21.15.32:0/2028080836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f2406b820 0x7f7f241e3aa0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7f1400c3f0 tx=0x7f7f1400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:08.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.458+0000 7f7f0bfff640 1 -- 172.21.15.32:0/2028080836 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f140052d0 con 0x7f7f2406b820 2024-04-24T23:00:08.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.458+0000 7f7f0bfff640 1 -- 172.21.15.32:0/2028080836 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f14005490 con 0x7f7f2406b820 2024-04-24T23:00:08.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.458+0000 7f7f29fd5640 1 -- 172.21.15.32:0/2028080836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f24148de0 con 0x7f7f2406b820 2024-04-24T23:00:08.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.458+0000 7f7f0bfff640 1 -- 172.21.15.32:0/2028080836 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f14007260 con 0x7f7f2406b820 2024-04-24T23:00:08.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.458+0000 7f7f29fd5640 1 -- 172.21.15.32:0/2028080836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f241493a0 con 0x7f7f2406b820 2024-04-24T23:00:08.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.459+0000 7f7f29fd5640 1 -- 172.21.15.32:0/2028080836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f24068430 con 0x7f7f2406b820 2024-04-24T23:00:08.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.460+0000 7f7f0bfff640 1 -- 172.21.15.32:0/2028080836 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7f14007400 con 0x7f7f2406b820 2024-04-24T23:00:08.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.460+0000 7f7f0bfff640 1 --2- 172.21.15.32:0/2028080836 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7f1006be30 0x7f7f1006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:08.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.460+0000 7f7f0bfff640 1 -- 172.21.15.32:0/2028080836 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f7f1408a210 con 0x7f7f2406b820 2024-04-24T23:00:08.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.460+0000 7f7f22ffd640 1 --2- 172.21.15.32:0/2028080836 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7f1006be30 0x7f7f1006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:08.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.461+0000 7f7f22ffd640 1 --2- 172.21.15.32:0/2028080836 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7f1006be30 0x7f7f1006e2f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7f240b8a00 tx=0x7f7f18008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:08.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.464+0000 7f7f0bfff640 1 -- 172.21.15.32:0/2028080836 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7f14058600 con 0x7f7f2406b820 2024-04-24T23:00:08.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.686+0000 7f7f29fd5640 1 -- 172.21.15.32:0/2028080836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7f2406b240 con 0x7f7f2406b820 2024-04-24T23:00:08.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.687+0000 7f7f0bfff640 1 -- 172.21.15.32:0/2028080836 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v13) ==== 53+0+73 (secure 0 0 0) 0x7f7f1405c900 con 0x7f7f2406b820 2024-04-24T23:00:08.688 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T23:00:08.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.689+0000 7f7f29fd5640 1 -- 172.21.15.32:0/2028080836 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7f1006be30 msgr2=0x7f7f1006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:08.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.689+0000 7f7f29fd5640 1 --2- 172.21.15.32:0/2028080836 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7f1006be30 0x7f7f1006e2f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7f240b8a00 tx=0x7f7f18008040 comp rx=0 tx=0).stop 2024-04-24T23:00:08.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.689+0000 7f7f29fd5640 1 -- 172.21.15.32:0/2028080836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f2406b820 msgr2=0x7f7f241e3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:08.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.689+0000 7f7f29fd5640 1 --2- 172.21.15.32:0/2028080836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f2406b820 0x7f7f241e3aa0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7f1400c3f0 tx=0x7f7f1400c8b0 comp rx=0 tx=0).stop 2024-04-24T23:00:08.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.689+0000 7f7f29fd5640 1 -- 172.21.15.32:0/2028080836 shutdown_connections 2024-04-24T23:00:08.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.689+0000 7f7f29fd5640 1 --2- 172.21.15.32:0/2028080836 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7f1006be30 0x7f7f1006e2f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:08.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.689+0000 7f7f29fd5640 1 --2- 172.21.15.32:0/2028080836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f2415e9b0 0x7f7f241e44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:08.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.689+0000 7f7f29fd5640 1 --2- 172.21.15.32:0/2028080836 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7f240d4210 0x7f7f240ca220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:08.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.689+0000 7f7f29fd5640 1 --2- 172.21.15.32:0/2028080836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f2406b820 0x7f7f241e3aa0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:08.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.689+0000 7f7f29fd5640 1 -- 172.21.15.32:0/2028080836 >> 172.21.15.32:0/2028080836 conn(0x7f7f240d11d0 msgr2=0x7f7f240d15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:08.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.689+0000 7f7f29fd5640 1 -- 172.21.15.32:0/2028080836 shutdown_connections 2024-04-24T23:00:08.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:08.689+0000 7f7f29fd5640 1 -- 172.21.15.32:0/2028080836 wait complete. 2024-04-24T23:00:08.701 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-24T23:00:08.701 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T23:00:08.702 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi121.front.sepia.ceph.com 2024-04-24T23:00:08.702 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-24T23:00:08.702 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-24T23:00:08.702 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-24T23:00:08.702 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-24T23:00:08.702 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T23:00:08.702 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-24T23:00:08.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:00:08.702 DEBUG:teuthology.orchestra.run.smithi121:> ip addr 2024-04-24T23:00:08.721 INFO:teuthology.orchestra.run.smithi121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T23:00:08.721 INFO:teuthology.orchestra.run.smithi121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T23:00:08.721 INFO:teuthology.orchestra.run.smithi121.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T23:00:08.721 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:00:08.721 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 ::1/128 scope host 2024-04-24T23:00:08.721 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:00:08.721 INFO:teuthology.orchestra.run.smithi121.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:00:08.722 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:82 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:00:08.722 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f0 2024-04-24T23:00:08.722 INFO:teuthology.orchestra.run.smithi121.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:00:08.722 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:83 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:00:08.722 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f1 2024-04-24T23:00:08.722 INFO:teuthology.orchestra.run.smithi121.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:00:08.722 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a2 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:00:08.722 INFO:teuthology.orchestra.run.smithi121.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T23:00:08.722 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a3 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:00:08.722 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.21.15.121/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-24T23:00:08.722 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft 42722sec preferred_lft 42722sec 2024-04-24T23:00:08.722 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::93dd:8489:4369:e127/64 scope link noprefixroute 2024-04-24T23:00:08.722 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:00:08.722 INFO:teuthology.orchestra.run.smithi121.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T23:00:08.722 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether c6:c0:c7:45:ae:ba brd ff:ff:ff:ff:ff:ff 2024-04-24T23:00:08.723 INFO:teuthology.orchestra.run.smithi121.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-24T23:00:08.723 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:00:08.723 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::80fe:d2ff:fe13:d1/64 scope link 2024-04-24T23:00:08.723 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:00:08.723 INFO:teuthology.orchestra.run.smithi121.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:00:08.723 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether e2:ca:b9:e9:9e:bb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T23:00:08.723 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::e0ca:b9ff:fee9:9ebb/64 scope link 2024-04-24T23:00:08.723 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:00:08.723 INFO:teuthology.orchestra.run.smithi121.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:00:08.723 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether c6:c0:c7:45:ae:ba brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-24T23:00:08.723 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::c4c0:c7ff:fe45:aeba/64 scope link 2024-04-24T23:00:08.723 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:00:08.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:00:08.724 DEBUG:teuthology.orchestra.run.smithi121:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T23:00:08.788 INFO:teuthology.orchestra.run.smithi121.stdout:1 2024-04-24T23:00:08.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:00:08.790 DEBUG:teuthology.orchestra.run.smithi121:> route 2024-04-24T23:00:08.848 INFO:teuthology.orchestra.run.smithi121.stdout:Kernel IP routing table 2024-04-24T23:00:08.849 INFO:teuthology.orchestra.run.smithi121.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T23:00:08.849 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-24T23:00:08.849 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-24T23:00:08.849 INFO:teuthology.orchestra.run.smithi121.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T23:00:08.849 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:00:08.849 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2024-04-24T23:00:08.849 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-24T23:00:08.849 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-24T23:00:08.849 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-24T23:00:08.849 DEBUG:teuthology.orchestra.run.smithi121:> ') 2024-04-24T23:00:08.942 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T23:00:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:00:08.973 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T23:00:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:00:08.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:00:08.980 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2024-04-24T23:00:08.994 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-24T23:00:08.994 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-24T23:00:08.995 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-24T23:00:08.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:00:08.995 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set brx.0 up 2024-04-24T23:00:09.058 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:00:09.058 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-24T23:00:09.113 INFO:teuthology.orchestra.run.smithi121.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-24T23:00:09.113 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:00:09.113 DEBUG:teuthology.orchestra.run.smithi121:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-24T23:00:09.168 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-24T23:00:09.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:00:09.168 DEBUG:teuthology.orchestra.run.smithi121:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:00:09.250 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:00:09.250 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-24T23:00:09.256 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:00:09.256 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-24T23:00:09.256 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.121:3300,172.21.15.121:3301" 2024-04-24T23:00:09.257 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: trying mount with new device syntax: 1@eb1b7a01-6f65-4534-9a12-83011862ba51.cephfs=/ 2024-04-24T23:00:09.257 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.32:3300/172.21.15.121:3300/172.21.15.121:3301" will pass to kernel 2024-04-24T23:00:09.257 INFO:teuthology.orchestra.run.smithi121.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-24T23:00:09.257 INFO:teuthology.orchestra.run.smithi121.stdout: You just mounted a file system that supports labels which does not 2024-04-24T23:00:09.257 INFO:teuthology.orchestra.run.smithi121.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-24T23:00:09.257 INFO:teuthology.orchestra.run.smithi121.stdout: applications will generate AVC messages and not be allowed access to 2024-04-24T23:00:09.257 INFO:teuthology.orchestra.run.smithi121.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-24T23:00:09.259 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-24T23:00:09.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:00:09.259 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-24T23:00:09.323 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:00:09.412 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T23:00:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:00:09.413 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T23:00:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:00:09.419 INFO:teuthology.orchestra.run.smithi121.stdout:client4859 2024-04-24T23:00:09.419 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:00:09.419 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:00:09.434 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:00:09.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.544+0000 7f4af8bf9640 1 Processor -- start 2024-04-24T23:00:09.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.544+0000 7f4af8bf9640 1 -- start start 2024-04-24T23:00:09.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.545+0000 7f4af8bf9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4af415b480 0x7f4af40b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:09.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.545+0000 7f4af8bf9640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4af415fd60 0x7f4af406c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:09.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.545+0000 7f4af8bf9640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4af406cb30 0x7f4af40ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:09.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.545+0000 7f4af8bf9640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f4af40b4ff0 con 0x7f4af415b480 2024-04-24T23:00:09.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.545+0000 7f4af8bf9640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f4af40b46c0 con 0x7f4af415fd60 2024-04-24T23:00:09.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.545+0000 7f4af8bf9640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f4af40b49a0 con 0x7f4af406cb30 2024-04-24T23:00:09.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.545+0000 7f4af259c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4af415b480 0x7f4af40b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:09.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.545+0000 7f4af259c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4af415b480 0x7f4af40b5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60256/0 (socket says 172.21.15.32:60256) 2024-04-24T23:00:09.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.545+0000 7f4af259c640 1 -- 172.21.15.32:0/4292093255 learned_addr learned my addr 172.21.15.32:0/4292093255 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:09.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.545+0000 7f4af1d9b640 1 --2- 172.21.15.32:0/4292093255 >> v2:172.21.15.121:3300/0 conn(0x7f4af415fd60 0x7f4af406c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:09.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.545+0000 7f4af2d9d640 1 --2- 172.21.15.32:0/4292093255 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4af406cb30 0x7f4af40ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:09.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.546+0000 7f4af259c640 1 -- 172.21.15.32:0/4292093255 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4af406cb30 msgr2=0x7f4af40ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:09.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.546+0000 7f4af259c640 1 --2- 172.21.15.32:0/4292093255 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4af406cb30 0x7f4af40ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:09.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.546+0000 7f4af259c640 1 -- 172.21.15.32:0/4292093255 >> v2:172.21.15.121:3300/0 conn(0x7f4af415fd60 msgr2=0x7f4af406c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:09.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.546+0000 7f4af259c640 1 --2- 172.21.15.32:0/4292093255 >> v2:172.21.15.121:3300/0 conn(0x7f4af415fd60 0x7f4af406c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:09.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.546+0000 7f4af259c640 1 -- 172.21.15.32:0/4292093255 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4af40b4c80 con 0x7f4af415b480 2024-04-24T23:00:09.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.546+0000 7f4af1d9b640 1 --2- 172.21.15.32:0/4292093255 >> v2:172.21.15.121:3300/0 conn(0x7f4af415fd60 0x7f4af406c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:00:09.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.546+0000 7f4af2d9d640 1 --2- 172.21.15.32:0/4292093255 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4af406cb30 0x7f4af40ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:00:09.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.546+0000 7f4af259c640 1 --2- 172.21.15.32:0/4292093255 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4af415b480 0x7f4af40b5c60 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4af406ee10 tx=0x7f4aec02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9df1f9217a7095bd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:09.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.547+0000 7f4af159a640 1 -- 172.21.15.32:0/4292093255 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aec037070 con 0x7f4af415b480 2024-04-24T23:00:09.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.547+0000 7f4af159a640 1 -- 172.21.15.32:0/4292093255 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4aec032440 con 0x7f4af415b480 2024-04-24T23:00:09.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.547+0000 7f4af159a640 1 -- 172.21.15.32:0/4292093255 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aec032810 con 0x7f4af415b480 2024-04-24T23:00:09.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.547+0000 7f4af8bf9640 1 -- 172.21.15.32:0/4292093255 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4af415b480 msgr2=0x7f4af40b5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:09.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.547+0000 7f4af8bf9640 1 --2- 172.21.15.32:0/4292093255 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4af415b480 0x7f4af40b5c60 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4af406ee10 tx=0x7f4aec02a1e0 comp rx=0 tx=0).stop 2024-04-24T23:00:09.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.547+0000 7f4af8bf9640 1 -- 172.21.15.32:0/4292093255 shutdown_connections 2024-04-24T23:00:09.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.547+0000 7f4af8bf9640 1 --2- 172.21.15.32:0/4292093255 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4af406cb30 0x7f4af40ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:09.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.547+0000 7f4af8bf9640 1 --2- 172.21.15.32:0/4292093255 >> v2:172.21.15.121:3300/0 conn(0x7f4af415fd60 0x7f4af406c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:09.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.547+0000 7f4af8bf9640 1 --2- 172.21.15.32:0/4292093255 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4af415b480 0x7f4af40b5c60 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:09.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.547+0000 7f4af8bf9640 1 -- 172.21.15.32:0/4292093255 >> 172.21.15.32:0/4292093255 conn(0x7f4af406b110 msgr2=0x7f4af406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:09.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.547+0000 7f4af8bf9640 1 -- 172.21.15.32:0/4292093255 shutdown_connections 2024-04-24T23:00:09.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.547+0000 7f4af8bf9640 1 -- 172.21.15.32:0/4292093255 wait complete. 2024-04-24T23:00:09.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.549+0000 7f4af8bf9640 1 Processor -- start 2024-04-24T23:00:09.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.549+0000 7f4af8bf9640 1 -- start start 2024-04-24T23:00:09.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.549+0000 7f4af8bf9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4af406cb30 0x7f4af4150840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:09.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.549+0000 7f4af8bf9640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4af415b480 0x7f4af414ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:09.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.549+0000 7f4af8bf9640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4af415fd60 0x7f4af415cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:09.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.549+0000 7f4af8bf9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4af415b860 con 0x7f4af406cb30 2024-04-24T23:00:09.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.549+0000 7f4af8bf9640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4af40b5480 con 0x7f4af415fd60 2024-04-24T23:00:09.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.549+0000 7f4af8bf9640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f4af414c620 con 0x7f4af415b480 2024-04-24T23:00:09.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.549+0000 7f4af259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4af406cb30 0x7f4af4150840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:09.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.550+0000 7f4af259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4af406cb30 0x7f4af4150840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60262/0 (socket says 172.21.15.32:60262) 2024-04-24T23:00:09.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.550+0000 7f4af259c640 1 -- 172.21.15.32:0/3224068909 learned_addr learned my addr 172.21.15.32:0/3224068909 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:09.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.550+0000 7f4af1d9b640 1 --2- 172.21.15.32:0/3224068909 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4af415b480 0x7f4af414ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:09.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.550+0000 7f4af2d9d640 1 --2- 172.21.15.32:0/3224068909 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4af415fd60 0x7f4af415cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:09.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.550+0000 7f4af259c640 1 -- 172.21.15.32:0/3224068909 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4af415b480 msgr2=0x7f4af414ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:09.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.550+0000 7f4af259c640 1 --2- 172.21.15.32:0/3224068909 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4af415b480 0x7f4af414ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:09.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.550+0000 7f4af259c640 1 -- 172.21.15.32:0/3224068909 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4af415fd60 msgr2=0x7f4af415cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:09.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.550+0000 7f4af259c640 1 --2- 172.21.15.32:0/3224068909 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4af415fd60 0x7f4af415cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:09.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.550+0000 7f4af259c640 1 -- 172.21.15.32:0/3224068909 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4af4150ee0 con 0x7f4af406cb30 2024-04-24T23:00:09.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.550+0000 7f4af2d9d640 1 --2- 172.21.15.32:0/3224068909 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4af415fd60 0x7f4af415cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:00:09.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.550+0000 7f4af259c640 1 --2- 172.21.15.32:0/3224068909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4af406cb30 0x7f4af4150840 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f4aec02a220 tx=0x7f4aec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:09.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.550+0000 7f4ae2ffd640 1 -- 172.21.15.32:0/3224068909 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aec041020 con 0x7f4af406cb30 2024-04-24T23:00:09.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.550+0000 7f4ae2ffd640 1 -- 172.21.15.32:0/3224068909 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4aec0306c0 con 0x7f4af406cb30 2024-04-24T23:00:09.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.550+0000 7f4af8bf9640 1 -- 172.21.15.32:0/3224068909 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4af414e1d0 con 0x7f4af406cb30 2024-04-24T23:00:09.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.550+0000 7f4ae2ffd640 1 -- 172.21.15.32:0/3224068909 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aec037040 con 0x7f4af406cb30 2024-04-24T23:00:09.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.550+0000 7f4af8bf9640 1 -- 172.21.15.32:0/3224068909 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4af414e750 con 0x7f4af406cb30 2024-04-24T23:00:09.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.551+0000 7f4af8bf9640 1 -- 172.21.15.32:0/3224068909 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4af4068430 con 0x7f4af406cb30 2024-04-24T23:00:09.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.552+0000 7f4ae2ffd640 1 -- 172.21.15.32:0/3224068909 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4aec044020 con 0x7f4af406cb30 2024-04-24T23:00:09.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.552+0000 7f4ae2ffd640 1 --2- 172.21.15.32:0/3224068909 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4ad406be30 0x7f4ad406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:09.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.552+0000 7f4ae2ffd640 1 -- 172.21.15.32:0/3224068909 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f4aec0ab5c0 con 0x7f4af406cb30 2024-04-24T23:00:09.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.553+0000 7f4af1d9b640 1 --2- 172.21.15.32:0/3224068909 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4ad406be30 0x7f4ad406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:09.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.553+0000 7f4af1d9b640 1 --2- 172.21.15.32:0/3224068909 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4ad406be30 0x7f4ad406e2f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4af415e310 tx=0x7f4ae8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:09.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.557+0000 7f4ae2ffd640 1 -- 172.21.15.32:0/3224068909 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4aec04b030 con 0x7f4af406cb30 2024-04-24T23:00:09.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.777+0000 7f4af8bf9640 1 -- 172.21.15.32:0/3224068909 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4af40b3970 con 0x7f4af406cb30 2024-04-24T23:00:09.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.778+0000 7f4ae2ffd640 1 -- 172.21.15.32:0/3224068909 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+5181 (secure 0 0 0) 0x7f4aec07a000 con 0x7f4af406cb30 2024-04-24T23:00:09.780 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:00:09.780 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1796819396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":1796819396},{"type":"v1","addr":"172.21.15.32:6835","nonce":1796819396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/298202585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":298202585},{"type":"v1","addr":"172.21.15.121:6836","nonce":298202585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4939,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/4056636026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":4056636026},{"type":"v1","addr":"172.21.15.32:6834","nonce":4056636026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T22:59:57.468765+0000","modified":"2024-04-24T22:59:59.471269+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5112},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5112":{"gid":5112,"name":"b","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6837/515107694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":515107694},{"type":"v1","addr":"172.21.15.121:6837","nonce":515107694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5112,"qdb_cluster":[5112]},"id":2}]} 2024-04-24T23:00:09.780 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 13 2024-04-24T23:00:09.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.780+0000 7f4af8bf9640 1 -- 172.21.15.32:0/3224068909 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4ad406be30 msgr2=0x7f4ad406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:09.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.780+0000 7f4af8bf9640 1 --2- 172.21.15.32:0/3224068909 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4ad406be30 0x7f4ad406e2f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4af415e310 tx=0x7f4ae8008040 comp rx=0 tx=0).stop 2024-04-24T23:00:09.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.780+0000 7f4af8bf9640 1 -- 172.21.15.32:0/3224068909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4af406cb30 msgr2=0x7f4af4150840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:09.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.780+0000 7f4af8bf9640 1 --2- 172.21.15.32:0/3224068909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4af406cb30 0x7f4af4150840 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f4aec02a220 tx=0x7f4aec0046c0 comp rx=0 tx=0).stop 2024-04-24T23:00:09.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.780+0000 7f4af8bf9640 1 -- 172.21.15.32:0/3224068909 shutdown_connections 2024-04-24T23:00:09.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.780+0000 7f4af8bf9640 1 --2- 172.21.15.32:0/3224068909 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4ad406be30 0x7f4ad406e2f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:09.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.780+0000 7f4af8bf9640 1 --2- 172.21.15.32:0/3224068909 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4af415fd60 0x7f4af415cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:09.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.780+0000 7f4af8bf9640 1 --2- 172.21.15.32:0/3224068909 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4af415b480 0x7f4af414ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:09.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.780+0000 7f4af8bf9640 1 --2- 172.21.15.32:0/3224068909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4af406cb30 0x7f4af4150840 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:09.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.780+0000 7f4af8bf9640 1 -- 172.21.15.32:0/3224068909 >> 172.21.15.32:0/3224068909 conn(0x7f4af406b110 msgr2=0x7f4af40cd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:09.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.780+0000 7f4af8bf9640 1 -- 172.21.15.32:0/3224068909 shutdown_connections 2024-04-24T23:00:09.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:09.780+0000 7f4af8bf9640 1 -- 172.21.15.32:0/3224068909 wait complete. 2024-04-24T23:00:09.792 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2024-04-24T23:00:09.909 INFO:teuthology.orchestra.run.smithi121.stdout:{"cluster_fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","whoami":0,"id":5112,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":11.793755894,"mdsmap_epoch":13,"osdmap_epoch":32,"osdmap_epoch_barrier":32,"uptime":18.663410859999999} 2024-04-24T23:00:09.909 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", "fs_name": "cephfs", "id": 5112, "mdsmap_epoch": 13, "osdmap_epoch": 32, "osdmap_epoch_barrier": 32, "rank_uptime": 11.793755894, "state": "up:active", "uptime": 18.66341086, "want_state": "up:active", "whoami": 0 } 2024-04-24T23:00:09.909 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph df --format=json-pretty 2024-04-24T23:00:10.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.008+0000 7f6ba1a64640 1 Processor -- start 2024-04-24T23:00:10.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.008+0000 7f6ba1a64640 1 -- start start 2024-04-24T23:00:10.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.009+0000 7f6ba1a64640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6b9c15e9b0 0x7f6b9c0d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:10.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.009+0000 7f6ba1a64640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6b9c0d4210 0x7f6b9c06b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:10.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.009+0000 7f6ba1a64640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b9c0d3170 con 0x7f6b9c0ba0e0 2024-04-24T23:00:10.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.009+0000 7f6ba1a64640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f6b9c0b8900 con 0x7f6b9c15e9b0 2024-04-24T23:00:10.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.009+0000 7f6ba1a64640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f6b9c0d3450 con 0x7f6b9c0d4210 2024-04-24T23:00:10.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.009+0000 7f6b9b7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6b9c15e9b0 0x7f6b9c0d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:10.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.009+0000 7f6b9b7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6b9c15e9b0 0x7f6b9c0d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41474/0 (socket says 172.21.15.32:41474) 2024-04-24T23:00:10.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.009+0000 7f6b9b7fe640 1 -- 172.21.15.32:0/1258355733 learned_addr learned my addr 172.21.15.32:0/1258355733 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:10.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.009+0000 7f6b9affd640 1 --2- 172.21.15.32:0/1258355733 >> v2:172.21.15.121:3300/0 conn(0x7f6b9c0d4210 0x7f6b9c06b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:10.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.009+0000 7f6b9b7fe640 1 -- 172.21.15.32:0/1258355733 >> v2:172.21.15.121:3300/0 conn(0x7f6b9c0d4210 msgr2=0x7f6b9c06b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:10.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.009+0000 7f6b9b7fe640 1 --2- 172.21.15.32:0/1258355733 >> v2:172.21.15.121:3300/0 conn(0x7f6b9c0d4210 0x7f6b9c06b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:10.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.009+0000 7f6b9b7fe640 1 -- 172.21.15.32:0/1258355733 >> v1:172.21.15.121:6789/0 conn(0x7f6b9c0ba0e0 legacy=0x7f6b9c071730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:10.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.009+0000 7f6b9b7fe640 1 -- 172.21.15.32:0/1258355733 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b9c0d3730 con 0x7f6b9c15e9b0 2024-04-24T23:00:10.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.009+0000 7f6b9affd640 1 --2- 172.21.15.32:0/1258355733 >> v2:172.21.15.121:3300/0 conn(0x7f6b9c0d4210 0x7f6b9c06b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:00:10.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.010+0000 7f6b9b7fe640 1 --2- 172.21.15.32:0/1258355733 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b9c15e9b0 0x7f6b9c0d2b90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c0730f0 tx=0x7f6b8c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fff1d902be09447 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:10.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.010+0000 7f6b9a7fc640 1 -- 172.21.15.32:0/1258355733 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c036070 con 0x7f6b9c15e9b0 2024-04-24T23:00:10.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.010+0000 7f6b9a7fc640 1 -- 172.21.15.32:0/1258355733 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b8c031600 con 0x7f6b9c15e9b0 2024-04-24T23:00:10.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.010+0000 7f6b9a7fc640 1 -- 172.21.15.32:0/1258355733 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c0319d0 con 0x7f6b9c15e9b0 2024-04-24T23:00:10.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.011+0000 7f6ba1a64640 1 -- 172.21.15.32:0/1258355733 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b9c15e9b0 msgr2=0x7f6b9c0d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:10.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.011+0000 7f6ba1a64640 1 --2- 172.21.15.32:0/1258355733 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b9c15e9b0 0x7f6b9c0d2b90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c0730f0 tx=0x7f6b8c02a660 comp rx=0 tx=0).stop 2024-04-24T23:00:10.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.011+0000 7f6ba1a64640 1 -- 172.21.15.32:0/1258355733 shutdown_connections 2024-04-24T23:00:10.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.011+0000 7f6ba1a64640 1 --2- 172.21.15.32:0/1258355733 >> v2:172.21.15.121:3300/0 conn(0x7f6b9c0d4210 0x7f6b9c06b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:10.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.011+0000 7f6ba1a64640 1 --2- 172.21.15.32:0/1258355733 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b9c15e9b0 0x7f6b9c0d2b90 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:10.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.011+0000 7f6ba1a64640 1 -- 172.21.15.32:0/1258355733 >> 172.21.15.32:0/1258355733 conn(0x7f6b9c0d11d0 msgr2=0x7f6b9c0d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:10.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.011+0000 7f6ba1a64640 1 -- 172.21.15.32:0/1258355733 shutdown_connections 2024-04-24T23:00:10.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.011+0000 7f6ba1a64640 1 -- 172.21.15.32:0/1258355733 wait complete. 2024-04-24T23:00:10.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.012+0000 7f6ba1a64640 1 Processor -- start 2024-04-24T23:00:10.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.012+0000 7f6ba1a64640 1 -- start start 2024-04-24T23:00:10.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.013+0000 7f6ba1a64640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b9c0ba0e0 0x7f6b9c161440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:10.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.013+0000 7f6ba1a64640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b9c0d4210 0x7f6b9c161b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:10.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.013+0000 7f6ba1a64640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6b9c15e9b0 0x7f6b9c1f3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:10.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.013+0000 7f6ba1a64640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9c0d2400 con 0x7f6b9c0ba0e0 2024-04-24T23:00:10.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.013+0000 7f6ba1a64640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9c0d3cf0 con 0x7f6b9c0d4210 2024-04-24T23:00:10.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.013+0000 7f6ba1a64640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f6b9c06b650 con 0x7f6b9c15e9b0 2024-04-24T23:00:10.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.013+0000 7f6b9b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b9c0ba0e0 0x7f6b9c161440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:10.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.013+0000 7f6b9b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b9c0ba0e0 0x7f6b9c161440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41488/0 (socket says 172.21.15.32:41488) 2024-04-24T23:00:10.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.013+0000 7f6b9b7fe640 1 -- 172.21.15.32:0/3249956635 learned_addr learned my addr 172.21.15.32:0/3249956635 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:10.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.013+0000 7f6b9affd640 1 --2- 172.21.15.32:0/3249956635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b9c0d4210 0x7f6b9c161b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:10.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.013+0000 7f6b9bfff640 1 --2- 172.21.15.32:0/3249956635 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6b9c15e9b0 0x7f6b9c1f3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:10.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.013+0000 7f6b9b7fe640 1 -- 172.21.15.32:0/3249956635 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6b9c15e9b0 msgr2=0x7f6b9c1f3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:10.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.013+0000 7f6b9b7fe640 1 --2- 172.21.15.32:0/3249956635 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6b9c15e9b0 0x7f6b9c1f3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:10.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.013+0000 7f6b9b7fe640 1 -- 172.21.15.32:0/3249956635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b9c0d4210 msgr2=0x7f6b9c161b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:10.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.013+0000 7f6b9b7fe640 1 --2- 172.21.15.32:0/3249956635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b9c0d4210 0x7f6b9c161b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:10.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.014+0000 7f6b9b7fe640 1 -- 172.21.15.32:0/3249956635 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b9c1603a0 con 0x7f6b9c0ba0e0 2024-04-24T23:00:10.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.014+0000 7f6b9b7fe640 1 --2- 172.21.15.32:0/3249956635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b9c0ba0e0 0x7f6b9c161440 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c034040 tx=0x7f6b8c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:10.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.014+0000 7f6b83fff640 1 -- 172.21.15.32:0/3249956635 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c036050 con 0x7f6b9c0ba0e0 2024-04-24T23:00:10.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.014+0000 7f6b83fff640 1 -- 172.21.15.32:0/3249956635 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b8c042070 con 0x7f6b9c0ba0e0 2024-04-24T23:00:10.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.014+0000 7f6b83fff640 1 -- 172.21.15.32:0/3249956635 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c009510 con 0x7f6b9c0ba0e0 2024-04-24T23:00:10.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.014+0000 7f6ba1a64640 1 -- 172.21.15.32:0/3249956635 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b9c160680 con 0x7f6b9c0ba0e0 2024-04-24T23:00:10.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.014+0000 7f6ba1a64640 1 -- 172.21.15.32:0/3249956635 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b9c160c40 con 0x7f6b9c0ba0e0 2024-04-24T23:00:10.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.015+0000 7f6ba1a64640 1 -- 172.21.15.32:0/3249956635 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b9c068430 con 0x7f6b9c0ba0e0 2024-04-24T23:00:10.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.015+0000 7f6b83fff640 1 -- 172.21.15.32:0/3249956635 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6b8c007c50 con 0x7f6b9c0ba0e0 2024-04-24T23:00:10.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.016+0000 7f6b83fff640 1 --2- 172.21.15.32:0/3249956635 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6b8806be30 0x7f6b8806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:10.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.016+0000 7f6b83fff640 1 -- 172.21.15.32:0/3249956635 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f6b8c0abd70 con 0x7f6b9c0ba0e0 2024-04-24T23:00:10.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.016+0000 7f6b9affd640 1 --2- 172.21.15.32:0/3249956635 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6b8806be30 0x7f6b8806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:10.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.016+0000 7f6b9affd640 1 --2- 172.21.15.32:0/3249956635 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6b8806be30 0x7f6b8806e2f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c068d60 tx=0x7f6b90008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:10.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.020+0000 7f6b83fff640 1 -- 172.21.15.32:0/3249956635 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6b8c07a0d0 con 0x7f6b9c0ba0e0 2024-04-24T23:00:10.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.278+0000 7f6ba1a64640 1 -- 172.21.15.32:0/3249956635 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "df", "format": "json-pretty"} v 0) -- 0x7f6b9c0b82c0 con 0x7f6b9c0ba0e0 2024-04-24T23:00:10.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.278+0000 7f6b83fff640 1 -- 172.21.15.32:0/3249956635 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "df", "format": "json-pretty"}]=0 v0) ==== 75+0+1620 (secure 0 0 0) 0x7f6b8c07e3d0 con 0x7f6b9c0ba0e0 2024-04-24T23:00:10.280 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:00:10.280 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:00:10.280 INFO:teuthology.orchestra.run.smithi032.stdout: "stats": { 2024-04-24T23:00:10.280 INFO:teuthology.orchestra.run.smithi032.stdout: "total_bytes": 1600000000, 2024-04-24T23:00:10.280 INFO:teuthology.orchestra.run.smithi032.stdout: "total_avail_bytes": 1597705284, 2024-04-24T23:00:10.280 INFO:teuthology.orchestra.run.smithi032.stdout: "total_used_bytes": 2294716, 2024-04-24T23:00:10.280 INFO:teuthology.orchestra.run.smithi032.stdout: "total_used_raw_bytes": 2294716, 2024-04-24T23:00:10.280 INFO:teuthology.orchestra.run.smithi032.stdout: "total_used_raw_ratio": 0.0014341975329443812, 2024-04-24T23:00:10.280 INFO:teuthology.orchestra.run.smithi032.stdout: "num_osds": 8, 2024-04-24T23:00:10.280 INFO:teuthology.orchestra.run.smithi032.stdout: "num_per_pool_osds": 0, 2024-04-24T23:00:10.280 INFO:teuthology.orchestra.run.smithi032.stdout: "num_per_pool_omap_osds": 0 2024-04-24T23:00:10.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:10.281 INFO:teuthology.orchestra.run.smithi032.stdout: "stats_by_class": { 2024-04-24T23:00:10.281 INFO:teuthology.orchestra.run.smithi032.stdout: "hdd": { 2024-04-24T23:00:10.281 INFO:teuthology.orchestra.run.smithi032.stdout: "total_bytes": 1600000000, 2024-04-24T23:00:10.281 INFO:teuthology.orchestra.run.smithi032.stdout: "total_avail_bytes": 1597705284, 2024-04-24T23:00:10.281 INFO:teuthology.orchestra.run.smithi032.stdout: "total_used_bytes": 2294716, 2024-04-24T23:00:10.281 INFO:teuthology.orchestra.run.smithi032.stdout: "total_used_raw_bytes": 2294716, 2024-04-24T23:00:10.281 INFO:teuthology.orchestra.run.smithi032.stdout: "total_used_raw_ratio": 0.0014341975329443812 2024-04-24T23:00:10.281 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:10.281 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:10.281 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2024-04-24T23:00:10.281 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:10.281 INFO:teuthology.orchestra.run.smithi032.stdout: "name": ".mgr", 2024-04-24T23:00:10.281 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 1, 2024-04-24T23:00:10.282 INFO:teuthology.orchestra.run.smithi032.stdout: "stats": { 2024-04-24T23:00:10.282 INFO:teuthology.orchestra.run.smithi032.stdout: "stored": 590368, 2024-04-24T23:00:10.282 INFO:teuthology.orchestra.run.smithi032.stdout: "objects": 2, 2024-04-24T23:00:10.282 INFO:teuthology.orchestra.run.smithi032.stdout: "kb_used": 577, 2024-04-24T23:00:10.282 INFO:teuthology.orchestra.run.smithi032.stdout: "bytes_used": 590368, 2024-04-24T23:00:10.282 INFO:teuthology.orchestra.run.smithi032.stdout: "percent_used": 0.00052959454478695989, 2024-04-24T23:00:10.282 INFO:teuthology.orchestra.run.smithi032.stdout: "max_avail": 557082176 2024-04-24T23:00:10.282 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:10.282 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:10.282 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:10.282 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cephfs_metadata", 2024-04-24T23:00:10.283 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 4, 2024-04-24T23:00:10.283 INFO:teuthology.orchestra.run.smithi032.stdout: "stats": { 2024-04-24T23:00:10.283 INFO:teuthology.orchestra.run.smithi032.stdout: "stored": 2410, 2024-04-24T23:00:10.283 INFO:teuthology.orchestra.run.smithi032.stdout: "objects": 22, 2024-04-24T23:00:10.283 INFO:teuthology.orchestra.run.smithi032.stdout: "kb_used": 3, 2024-04-24T23:00:10.283 INFO:teuthology.orchestra.run.smithi032.stdout: "bytes_used": 2410, 2024-04-24T23:00:10.283 INFO:teuthology.orchestra.run.smithi032.stdout: "percent_used": 2.1630514766002307e-06, 2024-04-24T23:00:10.283 INFO:teuthology.orchestra.run.smithi032.stdout: "max_avail": 557082176 2024-04-24T23:00:10.283 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:10.283 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:10.283 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:10.283 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cephfs_data", 2024-04-24T23:00:10.283 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 5, 2024-04-24T23:00:10.283 INFO:teuthology.orchestra.run.smithi032.stdout: "stats": { 2024-04-24T23:00:10.284 INFO:teuthology.orchestra.run.smithi032.stdout: "stored": 0, 2024-04-24T23:00:10.284 INFO:teuthology.orchestra.run.smithi032.stdout: "objects": 0, 2024-04-24T23:00:10.284 INFO:teuthology.orchestra.run.smithi032.stdout: "kb_used": 0, 2024-04-24T23:00:10.284 INFO:teuthology.orchestra.run.smithi032.stdout: "bytes_used": 0, 2024-04-24T23:00:10.284 INFO:teuthology.orchestra.run.smithi032.stdout: "percent_used": 0, 2024-04-24T23:00:10.284 INFO:teuthology.orchestra.run.smithi032.stdout: "max_avail": 557082176 2024-04-24T23:00:10.284 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:10.284 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:10.284 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:10.285 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:00:10.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.280+0000 7f6ba1a64640 1 -- 172.21.15.32:0/3249956635 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6b8806be30 msgr2=0x7f6b8806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:10.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.280+0000 7f6ba1a64640 1 --2- 172.21.15.32:0/3249956635 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6b8806be30 0x7f6b8806e2f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c068d60 tx=0x7f6b90008040 comp rx=0 tx=0).stop 2024-04-24T23:00:10.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.280+0000 7f6ba1a64640 1 -- 172.21.15.32:0/3249956635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b9c0ba0e0 msgr2=0x7f6b9c161440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:10.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.280+0000 7f6ba1a64640 1 --2- 172.21.15.32:0/3249956635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b9c0ba0e0 0x7f6b9c161440 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c034040 tx=0x7f6b8c004a40 comp rx=0 tx=0).stop 2024-04-24T23:00:10.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.280+0000 7f6ba1a64640 1 -- 172.21.15.32:0/3249956635 shutdown_connections 2024-04-24T23:00:10.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.280+0000 7f6ba1a64640 1 --2- 172.21.15.32:0/3249956635 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6b8806be30 0x7f6b8806e2f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:10.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.280+0000 7f6ba1a64640 1 --2- 172.21.15.32:0/3249956635 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6b9c15e9b0 0x7f6b9c1f3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:10.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.280+0000 7f6ba1a64640 1 --2- 172.21.15.32:0/3249956635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b9c0d4210 0x7f6b9c161b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:10.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.280+0000 7f6ba1a64640 1 --2- 172.21.15.32:0/3249956635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b9c0ba0e0 0x7f6b9c161440 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:10.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.280+0000 7f6ba1a64640 1 -- 172.21.15.32:0/3249956635 >> 172.21.15.32:0/3249956635 conn(0x7f6b9c0d11d0 msgr2=0x7f6b9c0cefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:10.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.280+0000 7f6ba1a64640 1 -- 172.21.15.32:0/3249956635 shutdown_connections 2024-04-24T23:00:10.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.280+0000 7f6ba1a64640 1 -- 172.21.15.32:0/3249956635 wait complete. 2024-04-24T23:00:10.293 DEBUG:teuthology.orchestra.run.smithi032:> stdin-killer -- python3 -c ' 2024-04-24T23:00:10.293 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-24T23:00:10.293 DEBUG:teuthology.orchestra.run.smithi032:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2024-04-24T23:00:10.293 DEBUG:teuthology.orchestra.run.smithi032:> os.fchmod(fd, 0O666) 2024-04-24T23:00:10.293 DEBUG:teuthology.orchestra.run.smithi032:> os.fsync(fd) 2024-04-24T23:00:10.293 DEBUG:teuthology.orchestra.run.smithi032:> os.close(fd) 2024-04-24T23:00:10.293 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-24T23:00:10.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:00:10.375 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:00:10.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:00:10.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:00:10.474 INFO:teuthology.orchestra.run.smithi032.stdout:client4853 2024-04-24T23:00:10.474 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/eb1b7a01-6f65-4534-9a12-83011862ba51.client4853/osdmap') 2024-04-24T23:00:10.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:00:10.533 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 32 barrier 32 flags 0x588000 2024-04-24T23:00:10.533 INFO:teuthology.orchestra.run.smithi032.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-04-24T23:00:10.533 INFO:teuthology.orchestra.run.smithi032.stdout:pool 4 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-04-24T23:00:10.533 INFO:teuthology.orchestra.run.smithi032.stdout:pool 5 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-04-24T23:00:10.533 INFO:teuthology.orchestra.run.smithi032.stdout:osd0 (2)172.21.15.32:6800 100% (exists, up) 100% -1 2024-04-24T23:00:10.533 INFO:teuthology.orchestra.run.smithi032.stdout:osd1 (2)172.21.15.32:6808 100% (exists, up) 100% -1 2024-04-24T23:00:10.533 INFO:teuthology.orchestra.run.smithi032.stdout:osd2 (2)172.21.15.32:6816 100% (exists, up) 100% -1 2024-04-24T23:00:10.533 INFO:teuthology.orchestra.run.smithi032.stdout:osd3 (2)172.21.15.32:6818 100% (exists, up) 100% -1 2024-04-24T23:00:10.534 INFO:teuthology.orchestra.run.smithi032.stdout:osd4 (2)172.21.15.121:6800 100% (exists, up) 100% -1 2024-04-24T23:00:10.534 INFO:teuthology.orchestra.run.smithi032.stdout:osd5 (2)172.21.15.121:6808 100% (exists, up) 100% -1 2024-04-24T23:00:10.534 INFO:teuthology.orchestra.run.smithi032.stdout:osd6 (2)172.21.15.121:6823 100% (exists, up) 100% -1 2024-04-24T23:00:10.534 INFO:teuthology.orchestra.run.smithi032.stdout:osd7 (2)172.21.15.121:6816 100% (exists, up) 100% -1 2024-04-24T23:00:10.534 INFO:teuthology.orchestra.run.smithi032.stderr:1+1 records in 2024-04-24T23:00:10.534 INFO:teuthology.orchestra.run.smithi032.stderr:1+1 records out 2024-04-24T23:00:10.534 INFO:teuthology.orchestra.run.smithi032.stderr:795 bytes copied, 4.2143e-05 s, 18.9 MB/s 2024-04-24T23:00:10.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:00:10.540 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set pause 2024-04-24T23:00:10.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.640+0000 7f1a8015a640 1 Processor -- start 2024-04-24T23:00:10.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.641+0000 7f1a8015a640 1 -- start start 2024-04-24T23:00:10.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.641+0000 7f1a8015a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1a7815e9b0 0x7f1a780d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:10.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.641+0000 7f1a8015a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1a780d4210 0x7f1a7806b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:10.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.641+0000 7f1a8015a640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1a780ba0e0 0x7f1a7806b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:10.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.641+0000 7f1a8015a640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f1a780d3170 con 0x7f1a780d4210 2024-04-24T23:00:10.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.641+0000 7f1a8015a640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f1a780d3450 con 0x7f1a780ba0e0 2024-04-24T23:00:10.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.641+0000 7f1a8015a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f1a780d3730 con 0x7f1a7815e9b0 2024-04-24T23:00:10.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.641+0000 7f1a7d6f5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1a780d4210 0x7f1a7806b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:10.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.642+0000 7f1a7d6f5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1a780d4210 0x7f1a7806b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41498/0 (socket says 172.21.15.32:41498) 2024-04-24T23:00:10.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.642+0000 7f1a7d6f5640 1 -- 172.21.15.32:0/26845684 learned_addr learned my addr 172.21.15.32:0/26845684 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:10.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.642+0000 7f1a7def6640 1 --2- 172.21.15.32:0/26845684 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1a7815e9b0 0x7f1a780d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:10.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.642+0000 7f1a7e6f7640 1 --2- 172.21.15.32:0/26845684 >> v2:172.21.15.121:3300/0 conn(0x7f1a780ba0e0 0x7f1a7806b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:10.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.642+0000 7f1a7d6f5640 1 -- 172.21.15.32:0/26845684 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1a7815e9b0 msgr2=0x7f1a780d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:10.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.642+0000 7f1a7d6f5640 1 --2- 172.21.15.32:0/26845684 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1a7815e9b0 0x7f1a780d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:10.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.642+0000 7f1a7d6f5640 1 -- 172.21.15.32:0/26845684 >> v2:172.21.15.121:3300/0 conn(0x7f1a780ba0e0 msgr2=0x7f1a7806b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:10.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.642+0000 7f1a7d6f5640 1 --2- 172.21.15.32:0/26845684 >> v2:172.21.15.121:3300/0 conn(0x7f1a780ba0e0 0x7f1a7806b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:10.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.642+0000 7f1a7d6f5640 1 -- 172.21.15.32:0/26845684 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a780d3a10 con 0x7f1a780d4210 2024-04-24T23:00:10.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.643+0000 7f1a7d6f5640 1 --2- 172.21.15.32:0/26845684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a780d4210 0x7f1a7806b110 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1a780b61d0 tx=0x7f1a6802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ddef86c7b9417a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:10.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.643+0000 7f1a7cef4640 1 -- 172.21.15.32:0/26845684 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a68037070 con 0x7f1a780d4210 2024-04-24T23:00:10.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.643+0000 7f1a7cef4640 1 -- 172.21.15.32:0/26845684 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a68032440 con 0x7f1a780d4210 2024-04-24T23:00:10.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.643+0000 7f1a7cef4640 1 -- 172.21.15.32:0/26845684 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a68032810 con 0x7f1a780d4210 2024-04-24T23:00:10.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.643+0000 7f1a8015a640 1 -- 172.21.15.32:0/26845684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a780d4210 msgr2=0x7f1a7806b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:10.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.643+0000 7f1a8015a640 1 --2- 172.21.15.32:0/26845684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a780d4210 0x7f1a7806b110 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1a780b61d0 tx=0x7f1a6802a1e0 comp rx=0 tx=0).stop 2024-04-24T23:00:10.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.643+0000 7f1a8015a640 1 -- 172.21.15.32:0/26845684 shutdown_connections 2024-04-24T23:00:10.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.643+0000 7f1a8015a640 1 --2- 172.21.15.32:0/26845684 >> v2:172.21.15.121:3300/0 conn(0x7f1a780ba0e0 0x7f1a7806b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:10.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.643+0000 7f1a8015a640 1 --2- 172.21.15.32:0/26845684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a780d4210 0x7f1a7806b110 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:10.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.643+0000 7f1a8015a640 1 --2- 172.21.15.32:0/26845684 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1a7815e9b0 0x7f1a780d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:10.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.643+0000 7f1a8015a640 1 -- 172.21.15.32:0/26845684 >> 172.21.15.32:0/26845684 conn(0x7f1a780d11d0 msgr2=0x7f1a780d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:10.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.643+0000 7f1a8015a640 1 -- 172.21.15.32:0/26845684 shutdown_connections 2024-04-24T23:00:10.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.643+0000 7f1a8015a640 1 -- 172.21.15.32:0/26845684 wait complete. 2024-04-24T23:00:10.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.644+0000 7f1a8015a640 1 Processor -- start 2024-04-24T23:00:10.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.645+0000 7f1a8015a640 1 -- start start 2024-04-24T23:00:10.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.645+0000 7f1a8015a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a780ba0e0 0x7f1a780c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:10.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.645+0000 7f1a8015a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1a780d4210 0x7f1a780cd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:10.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.645+0000 7f1a8015a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a7815e9b0 0x7f1a780cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:10.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.645+0000 7f1a8015a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1a78071730 con 0x7f1a7815e9b0 2024-04-24T23:00:10.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.645+0000 7f1a8015a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1a780d2400 con 0x7f1a780ba0e0 2024-04-24T23:00:10.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.645+0000 7f1a8015a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f1a780718b0 con 0x7f1a780d4210 2024-04-24T23:00:10.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.645+0000 7f1a7e6f7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a7815e9b0 0x7f1a780cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:10.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.645+0000 7f1a7e6f7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a7815e9b0 0x7f1a780cdf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41510/0 (socket says 172.21.15.32:41510) 2024-04-24T23:00:10.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.645+0000 7f1a7e6f7640 1 -- 172.21.15.32:0/2657046913 learned_addr learned my addr 172.21.15.32:0/2657046913 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:10.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.646+0000 7f1a7d6f5640 1 --2- 172.21.15.32:0/2657046913 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1a780d4210 0x7f1a780cd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:10.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.646+0000 7f1a7def6640 1 --2- 172.21.15.32:0/2657046913 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a780ba0e0 0x7f1a780c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:10.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.646+0000 7f1a7e6f7640 1 -- 172.21.15.32:0/2657046913 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1a780d4210 msgr2=0x7f1a780cd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:10.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.646+0000 7f1a7e6f7640 1 --2- 172.21.15.32:0/2657046913 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1a780d4210 0x7f1a780cd9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:10.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.646+0000 7f1a7e6f7640 1 -- 172.21.15.32:0/2657046913 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a780ba0e0 msgr2=0x7f1a780c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:10.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.646+0000 7f1a7e6f7640 1 --2- 172.21.15.32:0/2657046913 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a780ba0e0 0x7f1a780c82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:10.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.646+0000 7f1a7e6f7640 1 -- 172.21.15.32:0/2657046913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a7815f1e0 con 0x7f1a7815e9b0 2024-04-24T23:00:10.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.646+0000 7f1a7e6f7640 1 --2- 172.21.15.32:0/2657046913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a7815e9b0 0x7f1a780cdf30 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1a70007e90 tx=0x7f1a7000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:10.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.646+0000 7f1a6e7fc640 1 -- 172.21.15.32:0/2657046913 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a70014420 con 0x7f1a7815e9b0 2024-04-24T23:00:10.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.646+0000 7f1a6e7fc640 1 -- 172.21.15.32:0/2657046913 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a70019070 con 0x7f1a7815e9b0 2024-04-24T23:00:10.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.646+0000 7f1a6e7fc640 1 -- 172.21.15.32:0/2657046913 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a70014820 con 0x7f1a7815e9b0 2024-04-24T23:00:10.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.646+0000 7f1a8015a640 1 -- 172.21.15.32:0/2657046913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a7815f4c0 con 0x7f1a7815e9b0 2024-04-24T23:00:10.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.646+0000 7f1a8015a640 1 -- 172.21.15.32:0/2657046913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a7815fa80 con 0x7f1a7815e9b0 2024-04-24T23:00:10.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.647+0000 7f1a8015a640 1 -- 172.21.15.32:0/2657046913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a7815e210 con 0x7f1a7815e9b0 2024-04-24T23:00:10.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.647+0000 7f1a6e7fc640 1 -- 172.21.15.32:0/2657046913 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1a700149c0 con 0x7f1a7815e9b0 2024-04-24T23:00:10.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.648+0000 7f1a6e7fc640 1 --2- 172.21.15.32:0/2657046913 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1a5c06be30 0x7f1a5c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:10.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.648+0000 7f1a6e7fc640 1 -- 172.21.15.32:0/2657046913 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f1a7008b6e0 con 0x7f1a7815e9b0 2024-04-24T23:00:10.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.648+0000 7f1a7def6640 1 --2- 172.21.15.32:0/2657046913 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1a5c06be30 0x7f1a5c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:10.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.649+0000 7f1a7def6640 1 --2- 172.21.15.32:0/2657046913 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1a5c06be30 0x7f1a5c06e2f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1a74006fd0 tx=0x7f1a74008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:10.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.652+0000 7f1a6e7fc640 1 -- 172.21.15.32:0/2657046913 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1a7815e210 con 0x7f1a7815e9b0 2024-04-24T23:00:10.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:10.845+0000 7f1a8015a640 1 -- 172.21.15.32:0/2657046913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd set", "key": "pause"} v 0) -- 0x7f1a78068430 con 0x7f1a7815e9b0 2024-04-24T23:00:11.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.311+0000 7f1a6e7fc640 1 -- 172.21.15.32:0/2657046913 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "pause"}]=0 pauserd,pausewr is set v33) ==== 93+0+0 (secure 0 0 0) 0x7f1a7005a080 con 0x7f1a7815e9b0 2024-04-24T23:00:11.313 INFO:teuthology.orchestra.run.smithi032.stderr:pauserd,pausewr is set 2024-04-24T23:00:11.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.314+0000 7f1a8015a640 1 -- 172.21.15.32:0/2657046913 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1a5c06be30 msgr2=0x7f1a5c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:11.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.314+0000 7f1a8015a640 1 --2- 172.21.15.32:0/2657046913 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1a5c06be30 0x7f1a5c06e2f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1a74006fd0 tx=0x7f1a74008040 comp rx=0 tx=0).stop 2024-04-24T23:00:11.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.314+0000 7f1a8015a640 1 -- 172.21.15.32:0/2657046913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a7815e9b0 msgr2=0x7f1a780cdf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:11.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.314+0000 7f1a8015a640 1 --2- 172.21.15.32:0/2657046913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a7815e9b0 0x7f1a780cdf30 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1a70007e90 tx=0x7f1a7000e4b0 comp rx=0 tx=0).stop 2024-04-24T23:00:11.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.314+0000 7f1a8015a640 1 -- 172.21.15.32:0/2657046913 shutdown_connections 2024-04-24T23:00:11.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.314+0000 7f1a8015a640 1 --2- 172.21.15.32:0/2657046913 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1a5c06be30 0x7f1a5c06e2f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:11.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.314+0000 7f1a8015a640 1 --2- 172.21.15.32:0/2657046913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a7815e9b0 0x7f1a780cdf30 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:11.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.314+0000 7f1a8015a640 1 --2- 172.21.15.32:0/2657046913 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1a780d4210 0x7f1a780cd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:11.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.314+0000 7f1a8015a640 1 --2- 172.21.15.32:0/2657046913 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a780ba0e0 0x7f1a780c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:11.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.315+0000 7f1a8015a640 1 -- 172.21.15.32:0/2657046913 >> 172.21.15.32:0/2657046913 conn(0x7f1a780d11d0 msgr2=0x7f1a78159a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:11.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.315+0000 7f1a8015a640 1 -- 172.21.15.32:0/2657046913 shutdown_connections 2024-04-24T23:00:11.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.315+0000 7f1a8015a640 1 -- 172.21.15.32:0/2657046913 wait complete. 2024-04-24T23:00:11.328 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset pause 2024-04-24T23:00:11.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.432+0000 7f2c9af29640 1 Processor -- start 2024-04-24T23:00:11.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.432+0000 7f2c9af29640 1 -- start start 2024-04-24T23:00:11.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.433+0000 7f2c9af29640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2c94156bb0 0x7f2c9415dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:11.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.433+0000 7f2c9af29640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2c9406d400 0x7f2c9406d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:11.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.433+0000 7f2c9af29640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c9415ccf0 con 0x7f2c94072c80 2024-04-24T23:00:11.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.433+0000 7f2c9af29640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f2c9415cfb0 con 0x7f2c9406d400 2024-04-24T23:00:11.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.433+0000 7f2c9af29640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f2c9415d290 con 0x7f2c94156bb0 2024-04-24T23:00:11.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.433+0000 7f2c93fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2c9406d400 0x7f2c9406d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:11.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.433+0000 7f2c994c6640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2c94072c80 0x7f2c940b5840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58052/0 (socket says 172.21.15.32:58052) 2024-04-24T23:00:11.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.433+0000 7f2c994c6640 1 -- 172.21.15.32:0/4125047754 learned_addr learned my addr 172.21.15.32:0/4125047754 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:00:11.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.433+0000 7f2c98cc5640 1 --2- 172.21.15.32:0/4125047754 >> v2:172.21.15.121:3300/0 conn(0x7f2c94156bb0 0x7f2c9415dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:11.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.433+0000 7f2c93fff640 1 -- 172.21.15.32:0/4125047754 >> v2:172.21.15.121:3300/0 conn(0x7f2c94156bb0 msgr2=0x7f2c9415dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:11.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.433+0000 7f2c93fff640 1 --2- 172.21.15.32:0/4125047754 >> v2:172.21.15.121:3300/0 conn(0x7f2c94156bb0 0x7f2c9415dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:11.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.433+0000 7f2c93fff640 1 -- 172.21.15.32:0/4125047754 >> v1:172.21.15.32:6789/0 conn(0x7f2c94072c80 legacy=0x7f2c940b5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:11.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.433+0000 7f2c937fe640 1 -- 172.21.15.32:0/4125047754 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2879697802 0 0) 0x7f2c9415ccf0 con 0x7f2c94072c80 2024-04-24T23:00:11.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.433+0000 7f2c93fff640 1 -- 172.21.15.32:0/4125047754 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c9415d570 con 0x7f2c9406d400 2024-04-24T23:00:11.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.433+0000 7f2c98cc5640 1 --2- 172.21.15.32:0/4125047754 >> v2:172.21.15.121:3300/0 conn(0x7f2c94156bb0 0x7f2c9415dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:00:11.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.434+0000 7f2c93fff640 1 --2- 172.21.15.32:0/4125047754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c9406d400 0x7f2c9406d8b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2c9406f680 tx=0x7f2c8002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f44c95244e7896c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:11.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.434+0000 7f2c937fe640 1 -- 172.21.15.32:0/4125047754 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c80037070 con 0x7f2c9406d400 2024-04-24T23:00:11.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.434+0000 7f2c937fe640 1 -- 172.21.15.32:0/4125047754 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c80032440 con 0x7f2c9406d400 2024-04-24T23:00:11.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.434+0000 7f2c937fe640 1 -- 172.21.15.32:0/4125047754 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c80032810 con 0x7f2c9406d400 2024-04-24T23:00:11.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.435+0000 7f2c9af29640 1 -- 172.21.15.32:0/4125047754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c9406d400 msgr2=0x7f2c9406d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:11.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.435+0000 7f2c9af29640 1 --2- 172.21.15.32:0/4125047754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c9406d400 0x7f2c9406d8b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2c9406f680 tx=0x7f2c8002a1a0 comp rx=0 tx=0).stop 2024-04-24T23:00:11.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.435+0000 7f2c9af29640 1 -- 172.21.15.32:0/4125047754 shutdown_connections 2024-04-24T23:00:11.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.435+0000 7f2c9af29640 1 --2- 172.21.15.32:0/4125047754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c9406d400 0x7f2c9406d8b0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:11.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.435+0000 7f2c9af29640 1 --2- 172.21.15.32:0/4125047754 >> v2:172.21.15.121:3300/0 conn(0x7f2c94156bb0 0x7f2c9415dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:11.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.435+0000 7f2c9af29640 1 -- 172.21.15.32:0/4125047754 >> 172.21.15.32:0/4125047754 conn(0x7f2c9406e800 msgr2=0x7f2c9406ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:11.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.435+0000 7f2c9af29640 1 -- 172.21.15.32:0/4125047754 shutdown_connections 2024-04-24T23:00:11.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.435+0000 7f2c9af29640 1 -- 172.21.15.32:0/4125047754 wait complete. 2024-04-24T23:00:11.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.436+0000 7f2c9af29640 1 Processor -- start 2024-04-24T23:00:11.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.436+0000 7f2c9af29640 1 -- start start 2024-04-24T23:00:11.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.437+0000 7f2c9af29640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c9406d400 0x7f2c9415b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:11.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.437+0000 7f2c9af29640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c94072c80 0x7f2c940af2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:11.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.437+0000 7f2c9af29640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c94156bb0 0x7f2c940af7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:11.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.437+0000 7f2c9af29640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2c94156f90 con 0x7f2c9406d400 2024-04-24T23:00:11.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.437+0000 7f2c9af29640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9415e100 con 0x7f2c94072c80 2024-04-24T23:00:11.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.437+0000 7f2c9af29640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f2c94157110 con 0x7f2c94156bb0 2024-04-24T23:00:11.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.437+0000 7f2c98cc5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c9406d400 0x7f2c9415b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:11.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.437+0000 7f2c98cc5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c9406d400 0x7f2c9415b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41518/0 (socket says 172.21.15.32:41518) 2024-04-24T23:00:11.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.437+0000 7f2c98cc5640 1 -- 172.21.15.32:0/4170050136 learned_addr learned my addr 172.21.15.32:0/4170050136 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:11.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.437+0000 7f2c93fff640 1 --2- 172.21.15.32:0/4170050136 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c94072c80 0x7f2c940af2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:11.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.437+0000 7f2c994c6640 1 --2- 172.21.15.32:0/4170050136 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c94156bb0 0x7f2c940af7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:11.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.437+0000 7f2c98cc5640 1 -- 172.21.15.32:0/4170050136 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c94156bb0 msgr2=0x7f2c940af7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:11.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.437+0000 7f2c98cc5640 1 --2- 172.21.15.32:0/4170050136 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c94156bb0 0x7f2c940af7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:11.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.437+0000 7f2c98cc5640 1 -- 172.21.15.32:0/4170050136 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c94072c80 msgr2=0x7f2c940af2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:11.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.437+0000 7f2c98cc5640 1 --2- 172.21.15.32:0/4170050136 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c94072c80 0x7f2c940af2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:11.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.437+0000 7f2c98cc5640 1 -- 172.21.15.32:0/4170050136 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c9406b110 con 0x7f2c9406d400 2024-04-24T23:00:11.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.437+0000 7f2c93fff640 1 --2- 172.21.15.32:0/4170050136 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c94072c80 0x7f2c940af2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:00:11.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.437+0000 7f2c994c6640 1 --2- 172.21.15.32:0/4170050136 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c94156bb0 0x7f2c940af7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:00:11.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.437+0000 7f2c98cc5640 1 --2- 172.21.15.32:0/4170050136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c9406d400 0x7f2c9415b960 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2c8800c820 tx=0x7f2c8800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:11.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.438+0000 7f2c917fa640 1 -- 172.21.15.32:0/4170050136 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c88015070 con 0x7f2c9406d400 2024-04-24T23:00:11.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.438+0000 7f2c917fa640 1 -- 172.21.15.32:0/4170050136 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c88007a30 con 0x7f2c9406d400 2024-04-24T23:00:11.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.438+0000 7f2c9af29640 1 -- 172.21.15.32:0/4170050136 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c9406b3f0 con 0x7f2c9406d400 2024-04-24T23:00:11.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.438+0000 7f2c917fa640 1 -- 172.21.15.32:0/4170050136 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c88007e00 con 0x7f2c9406d400 2024-04-24T23:00:11.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.438+0000 7f2c9af29640 1 -- 172.21.15.32:0/4170050136 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c9406b9b0 con 0x7f2c9406d400 2024-04-24T23:00:11.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.439+0000 7f2c9af29640 1 -- 172.21.15.32:0/4170050136 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c94068430 con 0x7f2c9406d400 2024-04-24T23:00:11.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.439+0000 7f2c917fa640 1 -- 172.21.15.32:0/4170050136 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2c88018070 con 0x7f2c9406d400 2024-04-24T23:00:11.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.439+0000 7f2c917fa640 1 --2- 172.21.15.32:0/4170050136 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2c7806be30 0x7f2c7806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:11.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.440+0000 7f2c917fa640 1 -- 172.21.15.32:0/4170050136 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f2c8808adc0 con 0x7f2c9406d400 2024-04-24T23:00:11.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.440+0000 7f2c917fa640 1 -- 172.21.15.32:0/4170050136 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=34+}) -- 0x7f2c78073960 con 0x7f2c9406d400 2024-04-24T23:00:11.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.440+0000 7f2c93fff640 1 --2- 172.21.15.32:0/4170050136 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2c7806be30 0x7f2c7806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:11.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.440+0000 7f2c93fff640 1 --2- 172.21.15.32:0/4170050136 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2c7806be30 0x7f2c7806e2f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2c80034000 tx=0x7f2c8002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:11.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.444+0000 7f2c917fa640 1 -- 172.21.15.32:0/4170050136 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2c88059800 con 0x7f2c9406d400 2024-04-24T23:00:11.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:11.636+0000 7f2c9af29640 1 -- 172.21.15.32:0/4170050136 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd unset", "key": "pause"} v 0) -- 0x7f2c9415f650 con 0x7f2c9406d400 2024-04-24T23:00:12.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.313+0000 7f2c917fa640 1 -- 172.21.15.32:0/4170050136 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "pause"}]=0 pauserd,pausewr is unset v34) ==== 97+0+0 (secure 0 0 0) 0x7f2c88059100 con 0x7f2c9406d400 2024-04-24T23:00:12.315 INFO:teuthology.orchestra.run.smithi032.stderr:pauserd,pausewr is unset 2024-04-24T23:00:12.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.313+0000 7f2c917fa640 1 -- 172.21.15.32:0/4170050136 <== mon.0 v2:172.21.15.32:3300/0 8 ==== osd_map(34..34 src has 1..34) ==== 288+0+0 (secure 0 0 0) 0x7f2c88089400 con 0x7f2c9406d400 2024-04-24T23:00:12.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.313+0000 7f2c917fa640 1 -- 172.21.15.32:0/4170050136 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=35}) -- 0x7f2c780742d0 con 0x7f2c9406d400 2024-04-24T23:00:12.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.317+0000 7f2c9af29640 1 -- 172.21.15.32:0/4170050136 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2c7806be30 msgr2=0x7f2c7806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:12.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.317+0000 7f2c9af29640 1 --2- 172.21.15.32:0/4170050136 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2c7806be30 0x7f2c7806e2f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2c80034000 tx=0x7f2c8002e000 comp rx=0 tx=0).stop 2024-04-24T23:00:12.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.317+0000 7f2c9af29640 1 -- 172.21.15.32:0/4170050136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c9406d400 msgr2=0x7f2c9415b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:12.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.317+0000 7f2c9af29640 1 --2- 172.21.15.32:0/4170050136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c9406d400 0x7f2c9415b960 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2c8800c820 tx=0x7f2c8800cc20 comp rx=0 tx=0).stop 2024-04-24T23:00:12.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.317+0000 7f2c9af29640 1 -- 172.21.15.32:0/4170050136 shutdown_connections 2024-04-24T23:00:12.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.317+0000 7f2c9af29640 1 --2- 172.21.15.32:0/4170050136 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2c7806be30 0x7f2c7806e2f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:12.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.317+0000 7f2c9af29640 1 --2- 172.21.15.32:0/4170050136 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c94156bb0 0x7f2c940af7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:12.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.317+0000 7f2c9af29640 1 --2- 172.21.15.32:0/4170050136 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c94072c80 0x7f2c940af2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:12.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.317+0000 7f2c9af29640 1 --2- 172.21.15.32:0/4170050136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c9406d400 0x7f2c9415b960 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:12.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.317+0000 7f2c9af29640 1 -- 172.21.15.32:0/4170050136 >> 172.21.15.32:0/4170050136 conn(0x7f2c9406e800 msgr2=0x7f2c9415e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:12.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.317+0000 7f2c9af29640 1 -- 172.21.15.32:0/4170050136 shutdown_connections 2024-04-24T23:00:12.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.317+0000 7f2c9af29640 1 -- 172.21.15.32:0/4170050136 wait complete. 2024-04-24T23:00:12.332 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:00:12.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.437+0000 7fab78de7640 1 Processor -- start 2024-04-24T23:00:12.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.437+0000 7fab78de7640 1 -- start start 2024-04-24T23:00:12.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.438+0000 7fab78de7640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fab7415e9b0 0x7fab740d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:12.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.438+0000 7fab78de7640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fab740d4210 0x7fab7406b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:12.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.438+0000 7fab78de7640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab740d3170 con 0x7fab740ba0e0 2024-04-24T23:00:12.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.438+0000 7fab78de7640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fab740b8900 con 0x7fab7415e9b0 2024-04-24T23:00:12.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.438+0000 7fab78de7640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fab740d3450 con 0x7fab740d4210 2024-04-24T23:00:12.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.438+0000 7fab7259c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fab7415e9b0 0x7fab740d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:12.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.439+0000 7fab7259c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fab7415e9b0 0x7fab740d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41524/0 (socket says 172.21.15.32:41524) 2024-04-24T23:00:12.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.439+0000 7fab7259c640 1 -- 172.21.15.32:0/1190793683 learned_addr learned my addr 172.21.15.32:0/1190793683 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:12.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.439+0000 7fab71d9b640 1 --2- 172.21.15.32:0/1190793683 >> v2:172.21.15.121:3300/0 conn(0x7fab740d4210 0x7fab7406b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:12.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.439+0000 7fab7259c640 1 -- 172.21.15.32:0/1190793683 >> v2:172.21.15.121:3300/0 conn(0x7fab740d4210 msgr2=0x7fab7406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:12.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.439+0000 7fab7259c640 1 --2- 172.21.15.32:0/1190793683 >> v2:172.21.15.121:3300/0 conn(0x7fab740d4210 0x7fab7406b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:12.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.439+0000 7fab7259c640 1 -- 172.21.15.32:0/1190793683 >> v1:172.21.15.121:6789/0 conn(0x7fab740ba0e0 legacy=0x7fab74071730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:12.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.439+0000 7fab7259c640 1 -- 172.21.15.32:0/1190793683 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab740d3730 con 0x7fab7415e9b0 2024-04-24T23:00:12.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.439+0000 7fab7259c640 1 --2- 172.21.15.32:0/1190793683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab7415e9b0 0x7fab740d2b90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fab740730f0 tx=0x7fab6c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34c00a7784dee673 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:12.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.440+0000 7fab7159a640 1 -- 172.21.15.32:0/1190793683 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6c037070 con 0x7fab7415e9b0 2024-04-24T23:00:12.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.440+0000 7fab7159a640 1 -- 172.21.15.32:0/1190793683 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab6c032440 con 0x7fab7415e9b0 2024-04-24T23:00:12.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.440+0000 7fab7159a640 1 -- 172.21.15.32:0/1190793683 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6c032810 con 0x7fab7415e9b0 2024-04-24T23:00:12.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.440+0000 7fab78de7640 1 -- 172.21.15.32:0/1190793683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab7415e9b0 msgr2=0x7fab740d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:12.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.440+0000 7fab78de7640 1 --2- 172.21.15.32:0/1190793683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab7415e9b0 0x7fab740d2b90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fab740730f0 tx=0x7fab6c02a1a0 comp rx=0 tx=0).stop 2024-04-24T23:00:12.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.440+0000 7fab78de7640 1 -- 172.21.15.32:0/1190793683 shutdown_connections 2024-04-24T23:00:12.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.440+0000 7fab78de7640 1 --2- 172.21.15.32:0/1190793683 >> v2:172.21.15.121:3300/0 conn(0x7fab740d4210 0x7fab7406b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:12.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.440+0000 7fab78de7640 1 --2- 172.21.15.32:0/1190793683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab7415e9b0 0x7fab740d2b90 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:12.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.440+0000 7fab78de7640 1 -- 172.21.15.32:0/1190793683 >> 172.21.15.32:0/1190793683 conn(0x7fab740d11d0 msgr2=0x7fab740d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:12.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.440+0000 7fab78de7640 1 -- 172.21.15.32:0/1190793683 shutdown_connections 2024-04-24T23:00:12.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.440+0000 7fab78de7640 1 -- 172.21.15.32:0/1190793683 wait complete. 2024-04-24T23:00:12.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.442+0000 7fab78de7640 1 Processor -- start 2024-04-24T23:00:12.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.442+0000 7fab78de7640 1 -- start start 2024-04-24T23:00:12.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.442+0000 7fab78de7640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fab740ba0e0 0x7fab74144d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:12.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.442+0000 7fab78de7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab740d4210 0x7fab741422e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:12.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.442+0000 7fab78de7640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab7415e9b0 0x7fab741619e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:12.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.442+0000 7fab78de7640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fab740d2400 con 0x7fab740d4210 2024-04-24T23:00:12.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.442+0000 7fab78de7640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fab740d3cf0 con 0x7fab7415e9b0 2024-04-24T23:00:12.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.442+0000 7fab78de7640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fab7406b650 con 0x7fab740ba0e0 2024-04-24T23:00:12.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.443+0000 7fab71d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab740d4210 0x7fab741422e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:12.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.443+0000 7fab71d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab740d4210 0x7fab741422e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41540/0 (socket says 172.21.15.32:41540) 2024-04-24T23:00:12.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.443+0000 7fab71d9b640 1 -- 172.21.15.32:0/1056776715 learned_addr learned my addr 172.21.15.32:0/1056776715 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:12.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.443+0000 7fab7259c640 1 --2- 172.21.15.32:0/1056776715 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fab740ba0e0 0x7fab74144d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:12.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.443+0000 7fab72d9d640 1 --2- 172.21.15.32:0/1056776715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab7415e9b0 0x7fab741619e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:12.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.443+0000 7fab71d9b640 1 -- 172.21.15.32:0/1056776715 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fab740ba0e0 msgr2=0x7fab74144d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:12.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.443+0000 7fab71d9b640 1 --2- 172.21.15.32:0/1056776715 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fab740ba0e0 0x7fab74144d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:12.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.443+0000 7fab71d9b640 1 -- 172.21.15.32:0/1056776715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab7415e9b0 msgr2=0x7fab741619e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:12.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.443+0000 7fab71d9b640 1 --2- 172.21.15.32:0/1056776715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab7415e9b0 0x7fab741619e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:12.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.443+0000 7fab71d9b640 1 -- 172.21.15.32:0/1056776715 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab74152b80 con 0x7fab740d4210 2024-04-24T23:00:12.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.443+0000 7fab71d9b640 1 --2- 172.21.15.32:0/1056776715 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab740d4210 0x7fab741422e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fab5c00d800 tx=0x7fab5c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:12.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.443+0000 7fab6affd640 1 -- 172.21.15.32:0/1056776715 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c00b660 con 0x7fab740d4210 2024-04-24T23:00:12.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.443+0000 7fab6affd640 1 -- 172.21.15.32:0/1056776715 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab5c019070 con 0x7fab740d4210 2024-04-24T23:00:12.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.443+0000 7fab78de7640 1 -- 172.21.15.32:0/1056776715 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab741452c0 con 0x7fab740d4210 2024-04-24T23:00:12.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.443+0000 7fab6affd640 1 -- 172.21.15.32:0/1056776715 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c00ba20 con 0x7fab740d4210 2024-04-24T23:00:12.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.443+0000 7fab78de7640 1 -- 172.21.15.32:0/1056776715 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab74152e00 con 0x7fab740d4210 2024-04-24T23:00:12.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.444+0000 7fab78de7640 1 -- 172.21.15.32:0/1056776715 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab74068430 con 0x7fab740d4210 2024-04-24T23:00:12.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.445+0000 7fab6affd640 1 -- 172.21.15.32:0/1056776715 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fab5c00bbc0 con 0x7fab740d4210 2024-04-24T23:00:12.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.445+0000 7fab6affd640 1 --2- 172.21.15.32:0/1056776715 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fab5406be60 0x7fab5406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:12.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.445+0000 7fab6affd640 1 -- 172.21.15.32:0/1056776715 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fab5c08b130 con 0x7fab740d4210 2024-04-24T23:00:12.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.445+0000 7fab7259c640 1 --2- 172.21.15.32:0/1056776715 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fab5406be60 0x7fab5406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:12.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.446+0000 7fab7259c640 1 --2- 172.21.15.32:0/1056776715 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fab5406be60 0x7fab5406e320 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fab6c034000 tx=0x7fab6c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:12.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.450+0000 7fab6affd640 1 -- 172.21.15.32:0/1056776715 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fab741452c0 con 0x7fab740d4210 2024-04-24T23:00:12.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.633+0000 7fab78de7640 1 -- 172.21.15.32:0/1056776715 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fab740cb0b0 con 0x7fab740d4210 2024-04-24T23:00:12.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.634+0000 7fab6affd640 1 -- 172.21.15.32:0/1056776715 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16621 (secure 0 0 0) 0x7fab74152b80 con 0x7fab740d4210 2024-04-24T23:00:12.635 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:00:12.636 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":34,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:00:12.312472+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:59:56.068812+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-24T22:59:56.447402+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:00:12.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.636+0000 7fab78de7640 1 -- 172.21.15.32:0/1056776715 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fab5406be60 msgr2=0x7fab5406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:12.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.636+0000 7fab78de7640 1 --2- 172.21.15.32:0/1056776715 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fab5406be60 0x7fab5406e320 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fab6c034000 tx=0x7fab6c02e000 comp rx=0 tx=0).stop 2024-04-24T23:00:12.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.636+0000 7fab78de7640 1 -- 172.21.15.32:0/1056776715 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab740d4210 msgr2=0x7fab741422e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:12.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.636+0000 7fab78de7640 1 --2- 172.21.15.32:0/1056776715 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab740d4210 0x7fab741422e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fab5c00d800 tx=0x7fab5c00dcc0 comp rx=0 tx=0).stop 2024-04-24T23:00:12.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.636+0000 7fab78de7640 1 -- 172.21.15.32:0/1056776715 shutdown_connections 2024-04-24T23:00:12.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.636+0000 7fab78de7640 1 --2- 172.21.15.32:0/1056776715 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fab5406be60 0x7fab5406e320 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:12.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.636+0000 7fab78de7640 1 --2- 172.21.15.32:0/1056776715 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab740d4210 0x7fab741422e0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:12.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.636+0000 7fab78de7640 1 --2- 172.21.15.32:0/1056776715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab7415e9b0 0x7fab741619e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:12.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.636+0000 7fab78de7640 1 --2- 172.21.15.32:0/1056776715 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fab740ba0e0 0x7fab74144d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:12.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.636+0000 7fab78de7640 1 -- 172.21.15.32:0/1056776715 >> 172.21.15.32:0/1056776715 conn(0x7fab740d11d0 msgr2=0x7fab740cf7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:12.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.636+0000 7fab78de7640 1 -- 172.21.15.32:0/1056776715 shutdown_connections 2024-04-24T23:00:12.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.636+0000 7fab78de7640 1 -- 172.21.15.32:0/1056776715 wait complete. 2024-04-24T23:00:12.649 DEBUG:teuthology.orchestra.run.smithi032:> stdin-killer -- python3 -c ' 2024-04-24T23:00:12.649 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-24T23:00:12.649 DEBUG:teuthology.orchestra.run.smithi032:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2024-04-24T23:00:12.649 DEBUG:teuthology.orchestra.run.smithi032:> os.fchmod(fd, 0O666) 2024-04-24T23:00:12.649 DEBUG:teuthology.orchestra.run.smithi032:> os.fsync(fd) 2024-04-24T23:00:12.650 DEBUG:teuthology.orchestra.run.smithi032:> os.close(fd) 2024-04-24T23:00:12.650 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-24T23:00:12.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:00:12.727 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:00:12.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:00:12.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:00:12.824 INFO:teuthology.orchestra.run.smithi032.stdout:client4853 2024-04-24T23:00:12.824 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/eb1b7a01-6f65-4534-9a12-83011862ba51.client4853/osdmap') 2024-04-24T23:00:12.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:00:12.883 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 32 barrier 32 flags 0x588000 2024-04-24T23:00:12.883 INFO:teuthology.orchestra.run.smithi032.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-04-24T23:00:12.884 INFO:teuthology.orchestra.run.smithi032.stdout:pool 4 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-04-24T23:00:12.884 INFO:teuthology.orchestra.run.smithi032.stdout:pool 5 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-04-24T23:00:12.884 INFO:teuthology.orchestra.run.smithi032.stdout:osd0 (2)172.21.15.32:6800 100% (exists, up) 100% -1 2024-04-24T23:00:12.884 INFO:teuthology.orchestra.run.smithi032.stdout:osd1 (2)172.21.15.32:6808 100% (exists, up) 100% -1 2024-04-24T23:00:12.885 INFO:teuthology.orchestra.run.smithi032.stdout:osd2 (2)172.21.15.32:6816 100% (exists, up) 100% -1 2024-04-24T23:00:12.885 INFO:teuthology.orchestra.run.smithi032.stdout:osd3 (2)172.21.15.32:6818 100% (exists, up) 100% -1 2024-04-24T23:00:12.885 INFO:teuthology.orchestra.run.smithi032.stdout:osd4 (2)172.21.15.121:6800 100% (exists, up) 100% -1 2024-04-24T23:00:12.885 INFO:teuthology.orchestra.run.smithi032.stdout:osd5 (2)172.21.15.121:6808 100% (exists, up) 100% -1 2024-04-24T23:00:12.885 INFO:teuthology.orchestra.run.smithi032.stdout:osd6 (2)172.21.15.121:6823 100% (exists, up) 100% -1 2024-04-24T23:00:12.885 INFO:teuthology.orchestra.run.smithi032.stdout:osd7 (2)172.21.15.121:6816 100% (exists, up) 100% -1 2024-04-24T23:00:12.886 INFO:teuthology.orchestra.run.smithi032.stderr:1+1 records in 2024-04-24T23:00:12.886 INFO:teuthology.orchestra.run.smithi032.stderr:1+1 records out 2024-04-24T23:00:12.886 INFO:teuthology.orchestra.run.smithi032.stderr:795 bytes copied, 3.4874e-05 s, 22.8 MB/s 2024-04-24T23:00:12.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:00:12.891 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:00:12.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.989+0000 7f2b591c1640 1 Processor -- start 2024-04-24T23:00:12.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.990+0000 7f2b591c1640 1 -- start start 2024-04-24T23:00:12.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.990+0000 7f2b591c1640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2b5415e9b0 0x7f2b540d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:12.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.990+0000 7f2b591c1640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2b540d4210 0x7f2b5406b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:12.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.990+0000 7f2b591c1640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b540d3170 con 0x7f2b540ba0e0 2024-04-24T23:00:12.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.990+0000 7f2b591c1640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f2b540b8900 con 0x7f2b540d4210 2024-04-24T23:00:12.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.990+0000 7f2b591c1640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f2b540d3450 con 0x7f2b5415e9b0 2024-04-24T23:00:12.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.990+0000 7f2b5259c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2b540d4210 0x7f2b5406b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:12.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.990+0000 7f2b5259c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2b540d4210 0x7f2b5406b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41542/0 (socket says 172.21.15.32:41542) 2024-04-24T23:00:12.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.990+0000 7f2b5259c640 1 -- 172.21.15.32:0/482395356 learned_addr learned my addr 172.21.15.32:0/482395356 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:12.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.991+0000 7f2b52d9d640 1 --2- 172.21.15.32:0/482395356 >> v2:172.21.15.121:3300/0 conn(0x7f2b5415e9b0 0x7f2b540d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:12.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.991+0000 7f2b5259c640 1 -- 172.21.15.32:0/482395356 >> v2:172.21.15.121:3300/0 conn(0x7f2b5415e9b0 msgr2=0x7f2b540d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:12.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.991+0000 7f2b5259c640 1 --2- 172.21.15.32:0/482395356 >> v2:172.21.15.121:3300/0 conn(0x7f2b5415e9b0 0x7f2b540d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:12.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.991+0000 7f2b5259c640 1 -- 172.21.15.32:0/482395356 >> v1:172.21.15.121:6789/0 conn(0x7f2b540ba0e0 legacy=0x7f2b54071730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:12.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.991+0000 7f2b5259c640 1 -- 172.21.15.32:0/482395356 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b540d3730 con 0x7f2b540d4210 2024-04-24T23:00:12.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.991+0000 7f2b5259c640 1 --2- 172.21.15.32:0/482395356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b540d4210 0x7f2b5406b110 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f2b5406be20 tx=0x7f2b3c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2019096b170f1e27 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:12.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.992+0000 7f2b51d9b640 1 -- 172.21.15.32:0/482395356 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3c037070 con 0x7f2b540d4210 2024-04-24T23:00:12.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.992+0000 7f2b51d9b640 1 -- 172.21.15.32:0/482395356 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b3c032440 con 0x7f2b540d4210 2024-04-24T23:00:12.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.992+0000 7f2b51d9b640 1 -- 172.21.15.32:0/482395356 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3c032810 con 0x7f2b540d4210 2024-04-24T23:00:12.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.992+0000 7f2b591c1640 1 -- 172.21.15.32:0/482395356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b540d4210 msgr2=0x7f2b5406b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:12.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.992+0000 7f2b591c1640 1 --2- 172.21.15.32:0/482395356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b540d4210 0x7f2b5406b110 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f2b5406be20 tx=0x7f2b3c02a1e0 comp rx=0 tx=0).stop 2024-04-24T23:00:12.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.992+0000 7f2b591c1640 1 -- 172.21.15.32:0/482395356 shutdown_connections 2024-04-24T23:00:12.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.992+0000 7f2b591c1640 1 --2- 172.21.15.32:0/482395356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b540d4210 0x7f2b5406b110 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:12.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.992+0000 7f2b591c1640 1 --2- 172.21.15.32:0/482395356 >> v2:172.21.15.121:3300/0 conn(0x7f2b5415e9b0 0x7f2b540d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:12.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.992+0000 7f2b591c1640 1 -- 172.21.15.32:0/482395356 >> 172.21.15.32:0/482395356 conn(0x7f2b540d11d0 msgr2=0x7f2b540d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:12.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.992+0000 7f2b591c1640 1 -- 172.21.15.32:0/482395356 shutdown_connections 2024-04-24T23:00:12.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.992+0000 7f2b591c1640 1 -- 172.21.15.32:0/482395356 wait complete. 2024-04-24T23:00:13.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.994+0000 7f2b591c1640 1 Processor -- start 2024-04-24T23:00:13.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.994+0000 7f2b591c1640 1 -- start start 2024-04-24T23:00:13.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.994+0000 7f2b591c1640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2b540ba0e0 0x7f2b541f4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:13.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.994+0000 7f2b591c1640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2b540d4210 0x7f2b540b5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:13.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.994+0000 7f2b591c1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b5415e9b0 0x7f2b541f8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:13.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.994+0000 7f2b591c1640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2b540d2400 con 0x7f2b5415e9b0 2024-04-24T23:00:13.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.994+0000 7f2b591c1640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2b540d3cf0 con 0x7f2b540ba0e0 2024-04-24T23:00:13.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.994+0000 7f2b591c1640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f2b5406b650 con 0x7f2b540d4210 2024-04-24T23:00:13.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.995+0000 7f2b5359e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b5415e9b0 0x7f2b541f8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:13.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.995+0000 7f2b5359e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b5415e9b0 0x7f2b541f8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41558/0 (socket says 172.21.15.32:41558) 2024-04-24T23:00:13.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.995+0000 7f2b5359e640 1 -- 172.21.15.32:0/3522222010 learned_addr learned my addr 172.21.15.32:0/3522222010 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:13.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.995+0000 7f2b52d9d640 1 --2- 172.21.15.32:0/3522222010 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2b540ba0e0 0x7f2b541f4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:13.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.995+0000 7f2b5259c640 1 --2- 172.21.15.32:0/3522222010 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2b540d4210 0x7f2b540b5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:13.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.995+0000 7f2b5359e640 1 -- 172.21.15.32:0/3522222010 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2b540d4210 msgr2=0x7f2b540b5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:13.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.995+0000 7f2b5359e640 1 --2- 172.21.15.32:0/3522222010 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2b540d4210 0x7f2b540b5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:13.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.995+0000 7f2b5359e640 1 -- 172.21.15.32:0/3522222010 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2b540ba0e0 msgr2=0x7f2b541f4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:13.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.995+0000 7f2b5359e640 1 --2- 172.21.15.32:0/3522222010 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2b540ba0e0 0x7f2b541f4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:13.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.995+0000 7f2b5359e640 1 -- 172.21.15.32:0/3522222010 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b541f8840 con 0x7f2b5415e9b0 2024-04-24T23:00:13.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.995+0000 7f2b52d9d640 1 --2- 172.21.15.32:0/3522222010 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2b540ba0e0 0x7f2b541f4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:00:13.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.995+0000 7f2b5359e640 1 --2- 172.21.15.32:0/3522222010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b5415e9b0 0x7f2b541f8300 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f2b4400c710 tx=0x7f2b4400cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:13.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.995+0000 7f2b4b7fe640 1 -- 172.21.15.32:0/3522222010 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b44016020 con 0x7f2b5415e9b0 2024-04-24T23:00:13.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.995+0000 7f2b4b7fe640 1 -- 172.21.15.32:0/3522222010 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b44006d50 con 0x7f2b5415e9b0 2024-04-24T23:00:13.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.995+0000 7f2b591c1640 1 -- 172.21.15.32:0/3522222010 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b541f4910 con 0x7f2b5415e9b0 2024-04-24T23:00:13.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.995+0000 7f2b4b7fe640 1 -- 172.21.15.32:0/3522222010 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b44003bb0 con 0x7f2b5415e9b0 2024-04-24T23:00:13.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.995+0000 7f2b591c1640 1 -- 172.21.15.32:0/3522222010 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b541f4ed0 con 0x7f2b5415e9b0 2024-04-24T23:00:13.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.996+0000 7f2b591c1640 1 -- 172.21.15.32:0/3522222010 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b54068430 con 0x7f2b5415e9b0 2024-04-24T23:00:13.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.997+0000 7f2b4b7fe640 1 -- 172.21.15.32:0/3522222010 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2b4400a850 con 0x7f2b5415e9b0 2024-04-24T23:00:13.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.997+0000 7f2b4b7fe640 1 --2- 172.21.15.32:0/3522222010 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2b3406be30 0x7f2b3406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:13.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.997+0000 7f2b4b7fe640 1 -- 172.21.15.32:0/3522222010 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f2b4408aac0 con 0x7f2b5415e9b0 2024-04-24T23:00:13.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.998+0000 7f2b52d9d640 1 --2- 172.21.15.32:0/3522222010 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2b3406be30 0x7f2b3406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:13.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:12.998+0000 7f2b52d9d640 1 --2- 172.21.15.32:0/3522222010 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2b3406be30 0x7f2b3406e2f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c006fd0 tx=0x7f2b4c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:13.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:13.002+0000 7f2b4b7fe640 1 -- 172.21.15.32:0/3522222010 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2b44020020 con 0x7f2b5415e9b0 2024-04-24T23:00:13.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:13.225+0000 7f2b591c1640 1 -- 172.21.15.32:0/3522222010 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2b54072390 con 0x7f2b5415e9b0 2024-04-24T23:00:13.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:13.226+0000 7f2b4b7fe640 1 -- 172.21.15.32:0/3522222010 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+5181 (secure 0 0 0) 0x7f2b541f8840 con 0x7f2b5415e9b0 2024-04-24T23:00:13.228 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:00:13.228 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1796819396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":1796819396},{"type":"v1","addr":"172.21.15.32:6835","nonce":1796819396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/298202585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":298202585},{"type":"v1","addr":"172.21.15.121:6836","nonce":298202585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4939,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/4056636026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":4056636026},{"type":"v1","addr":"172.21.15.32:6834","nonce":4056636026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T22:59:57.468765+0000","modified":"2024-04-24T22:59:59.471269+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5112},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5112":{"gid":5112,"name":"b","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6837/515107694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":515107694},{"type":"v1","addr":"172.21.15.121:6837","nonce":515107694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5112,"qdb_cluster":[5112]},"id":2}]} 2024-04-24T23:00:13.228 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 13 2024-04-24T23:00:13.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:13.228+0000 7f2b591c1640 1 -- 172.21.15.32:0/3522222010 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2b3406be30 msgr2=0x7f2b3406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:13.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:13.228+0000 7f2b591c1640 1 --2- 172.21.15.32:0/3522222010 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2b3406be30 0x7f2b3406e2f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c006fd0 tx=0x7f2b4c008040 comp rx=0 tx=0).stop 2024-04-24T23:00:13.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:13.228+0000 7f2b591c1640 1 -- 172.21.15.32:0/3522222010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b5415e9b0 msgr2=0x7f2b541f8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:13.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:13.228+0000 7f2b591c1640 1 --2- 172.21.15.32:0/3522222010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b5415e9b0 0x7f2b541f8300 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f2b4400c710 tx=0x7f2b4400cbd0 comp rx=0 tx=0).stop 2024-04-24T23:00:13.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:13.228+0000 7f2b591c1640 1 -- 172.21.15.32:0/3522222010 shutdown_connections 2024-04-24T23:00:13.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:13.228+0000 7f2b591c1640 1 --2- 172.21.15.32:0/3522222010 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2b3406be30 0x7f2b3406e2f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:13.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:13.228+0000 7f2b591c1640 1 --2- 172.21.15.32:0/3522222010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b5415e9b0 0x7f2b541f8300 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:13.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:13.228+0000 7f2b591c1640 1 --2- 172.21.15.32:0/3522222010 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2b540d4210 0x7f2b540b5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:13.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:13.228+0000 7f2b591c1640 1 --2- 172.21.15.32:0/3522222010 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2b540ba0e0 0x7f2b541f4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:13.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:13.228+0000 7f2b591c1640 1 -- 172.21.15.32:0/3522222010 >> 172.21.15.32:0/3522222010 conn(0x7f2b540d11d0 msgr2=0x7f2b540b8b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:13.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:13.228+0000 7f2b591c1640 1 -- 172.21.15.32:0/3522222010 shutdown_connections 2024-04-24T23:00:13.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:13.228+0000 7f2b591c1640 1 -- 172.21.15.32:0/3522222010 wait complete. 2024-04-24T23:00:13.241 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json osdmap barrier 34 2024-04-24T23:00:13.367 DEBUG:tasks.cephfs.filesystem:_json_asok output empty 2024-04-24T23:00:13.367 DEBUG:teuthology.orchestra.run.smithi032:> stdin-killer -- python3 -c ' 2024-04-24T23:00:13.367 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-24T23:00:13.367 DEBUG:teuthology.orchestra.run.smithi032:> fd = os.open("/home/ubuntu/cephtest/mnt.0/baz", os.O_CREAT | os.O_RDWR, 0O600) 2024-04-24T23:00:13.367 DEBUG:teuthology.orchestra.run.smithi032:> os.fchmod(fd, 0O666) 2024-04-24T23:00:13.367 DEBUG:teuthology.orchestra.run.smithi032:> os.fsync(fd) 2024-04-24T23:00:13.368 DEBUG:teuthology.orchestra.run.smithi032:> os.close(fd) 2024-04-24T23:00:13.368 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-24T23:00:13.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:00:13.448 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:00:13.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:13 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:00:13.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:00:13.546 INFO:teuthology.orchestra.run.smithi032.stdout:client4853 2024-04-24T23:00:13.547 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/eb1b7a01-6f65-4534-9a12-83011862ba51.client4853/osdmap') 2024-04-24T23:00:13.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:00:13.605 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 34 barrier 34 flags 0x588000 2024-04-24T23:00:13.605 INFO:teuthology.orchestra.run.smithi032.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-04-24T23:00:13.605 INFO:teuthology.orchestra.run.smithi032.stdout:pool 4 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-04-24T23:00:13.605 INFO:teuthology.orchestra.run.smithi032.stdout:pool 5 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-04-24T23:00:13.605 INFO:teuthology.orchestra.run.smithi032.stdout:osd0 (2)172.21.15.32:6800 100% (exists, up) 100% -1 2024-04-24T23:00:13.605 INFO:teuthology.orchestra.run.smithi032.stdout:osd1 (2)172.21.15.32:6808 100% (exists, up) 100% -1 2024-04-24T23:00:13.605 INFO:teuthology.orchestra.run.smithi032.stdout:osd2 (2)172.21.15.32:6816 100% (exists, up) 100% -1 2024-04-24T23:00:13.605 INFO:teuthology.orchestra.run.smithi032.stdout:osd3 (2)172.21.15.32:6818 100% (exists, up) 100% -1 2024-04-24T23:00:13.605 INFO:teuthology.orchestra.run.smithi032.stdout:osd4 (2)172.21.15.121:6800 100% (exists, up) 100% -1 2024-04-24T23:00:13.605 INFO:teuthology.orchestra.run.smithi032.stdout:osd5 (2)172.21.15.121:6808 100% (exists, up) 100% -1 2024-04-24T23:00:13.606 INFO:teuthology.orchestra.run.smithi032.stdout:osd6 (2)172.21.15.121:6823 100% (exists, up) 100% -1 2024-04-24T23:00:13.606 INFO:teuthology.orchestra.run.smithi032.stdout:osd7 (2)172.21.15.121:6816 100% (exists, up) 100% -1 2024-04-24T23:00:13.606 INFO:teuthology.orchestra.run.smithi032.stderr:1+1 records in 2024-04-24T23:00:13.606 INFO:teuthology.orchestra.run.smithi032.stderr:1+1 records out 2024-04-24T23:00:13.606 INFO:teuthology.orchestra.run.smithi032.stderr:795 bytes copied, 3.6059e-05 s, 22.0 MB/s 2024-04-24T23:00:13.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:00:13.612 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:00:13.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:13 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:00:13.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:00:13.668 INFO:teuthology.orchestra.run.smithi032.stdout:client4853 2024-04-24T23:00:13.669 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/eb1b7a01-6f65-4534-9a12-83011862ba51.client4853/osdmap') 2024-04-24T23:00:13.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:00:13.727 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 34 barrier 34 flags 0x588000 2024-04-24T23:00:13.727 INFO:teuthology.orchestra.run.smithi032.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-04-24T23:00:13.727 INFO:teuthology.orchestra.run.smithi032.stdout:pool 4 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-04-24T23:00:13.727 INFO:teuthology.orchestra.run.smithi032.stdout:pool 5 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-04-24T23:00:13.727 INFO:teuthology.orchestra.run.smithi032.stdout:osd0 (2)172.21.15.32:6800 100% (exists, up) 100% -1 2024-04-24T23:00:13.727 INFO:teuthology.orchestra.run.smithi032.stdout:osd1 (2)172.21.15.32:6808 100% (exists, up) 100% -1 2024-04-24T23:00:13.727 INFO:teuthology.orchestra.run.smithi032.stdout:osd2 (2)172.21.15.32:6816 100% (exists, up) 100% -1 2024-04-24T23:00:13.727 INFO:teuthology.orchestra.run.smithi032.stdout:osd3 (2)172.21.15.32:6818 100% (exists, up) 100% -1 2024-04-24T23:00:13.727 INFO:teuthology.orchestra.run.smithi032.stdout:osd4 (2)172.21.15.121:6800 100% (exists, up) 100% -1 2024-04-24T23:00:13.727 INFO:teuthology.orchestra.run.smithi032.stdout:osd5 (2)172.21.15.121:6808 100% (exists, up) 100% -1 2024-04-24T23:00:13.727 INFO:teuthology.orchestra.run.smithi032.stdout:osd6 (2)172.21.15.121:6823 100% (exists, up) 100% -1 2024-04-24T23:00:13.728 INFO:teuthology.orchestra.run.smithi032.stdout:osd7 (2)172.21.15.121:6816 100% (exists, up) 100% -1 2024-04-24T23:00:13.729 INFO:teuthology.orchestra.run.smithi032.stderr:1+1 records in 2024-04-24T23:00:13.729 INFO:teuthology.orchestra.run.smithi032.stderr:1+1 records out 2024-04-24T23:00:13.729 INFO:teuthology.orchestra.run.smithi032.stderr:795 bytes copied, 3.5445e-05 s, 22.4 MB/s 2024-04-24T23:00:13.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:00:13.735 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-24T23:00:13.735 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-24T23:00:13.735 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:00:13.772 DEBUG:teuthology.orchestra.run.smithi121:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:00:13.816 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-24T23:00:13.816 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:00:13.816 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:00:13.816 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:00:13.833 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:00:13.833 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:00:13.889 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:00:14.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.029+0000 7fc1ca438640 1 Processor -- start 2024-04-24T23:00:14.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.029+0000 7fc1ca438640 1 -- start start 2024-04-24T23:00:14.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.029+0000 7fc1ca438640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc1c415e9b0 0x7fc1c40d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:14.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.029+0000 7fc1ca438640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc1c40d4210 0x7fc1c406b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:14.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.029+0000 7fc1ca438640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1c40d3170 con 0x7fc1c40ba0e0 2024-04-24T23:00:14.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.029+0000 7fc1ca438640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fc1c40b8900 con 0x7fc1c40d4210 2024-04-24T23:00:14.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.029+0000 7fc1ca438640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fc1c40d3450 con 0x7fc1c415e9b0 2024-04-24T23:00:14.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.030+0000 7fc1c37fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc1c40d4210 0x7fc1c406b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:14.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.030+0000 7fc1c89d5640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc1c40ba0e0 0x7fc1c4071730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58068/0 (socket says 172.21.15.32:58068) 2024-04-24T23:00:14.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.030+0000 7fc1c89d5640 1 -- 172.21.15.32:0/3158313438 learned_addr learned my addr 172.21.15.32:0/3158313438 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:00:14.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.030+0000 7fc1c3fff640 1 --2- 172.21.15.32:0/3158313438 >> v2:172.21.15.121:3300/0 conn(0x7fc1c415e9b0 0x7fc1c40d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:14.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.030+0000 7fc1c37fe640 1 -- 172.21.15.32:0/3158313438 >> v2:172.21.15.121:3300/0 conn(0x7fc1c415e9b0 msgr2=0x7fc1c40d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:14.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.030+0000 7fc1c37fe640 1 --2- 172.21.15.32:0/3158313438 >> v2:172.21.15.121:3300/0 conn(0x7fc1c415e9b0 0x7fc1c40d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:14.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.030+0000 7fc1c37fe640 1 -- 172.21.15.32:0/3158313438 >> v1:172.21.15.32:6789/0 conn(0x7fc1c40ba0e0 legacy=0x7fc1c4071730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:14.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.030+0000 7fc1c2ffd640 1 -- 172.21.15.32:0/3158313438 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3515455416 0 0) 0x7fc1c40d3170 con 0x7fc1c40ba0e0 2024-04-24T23:00:14.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.030+0000 7fc1c37fe640 1 -- 172.21.15.32:0/3158313438 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1c40d3730 con 0x7fc1c40d4210 2024-04-24T23:00:14.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.030+0000 7fc1c3fff640 1 --2- 172.21.15.32:0/3158313438 >> v2:172.21.15.121:3300/0 conn(0x7fc1c415e9b0 0x7fc1c40d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:00:14.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.031+0000 7fc1c37fe640 1 --2- 172.21.15.32:0/3158313438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1c40d4210 0x7fc1c406b110 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc1c406be20 tx=0x7fc1b802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aebf712b7a42b983 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:14.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.031+0000 7fc1c2ffd640 1 -- 172.21.15.32:0/3158313438 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b8037070 con 0x7fc1c40d4210 2024-04-24T23:00:14.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.031+0000 7fc1c2ffd640 1 -- 172.21.15.32:0/3158313438 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1b8032440 con 0x7fc1c40d4210 2024-04-24T23:00:14.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.031+0000 7fc1c2ffd640 1 -- 172.21.15.32:0/3158313438 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b802ae60 con 0x7fc1c40d4210 2024-04-24T23:00:14.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.031+0000 7fc1ca438640 1 -- 172.21.15.32:0/3158313438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1c40d4210 msgr2=0x7fc1c406b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:14.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.031+0000 7fc1ca438640 1 --2- 172.21.15.32:0/3158313438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1c40d4210 0x7fc1c406b110 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc1c406be20 tx=0x7fc1b802a110 comp rx=0 tx=0).stop 2024-04-24T23:00:14.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.031+0000 7fc1ca438640 1 -- 172.21.15.32:0/3158313438 shutdown_connections 2024-04-24T23:00:14.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.031+0000 7fc1ca438640 1 --2- 172.21.15.32:0/3158313438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1c40d4210 0x7fc1c406b110 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:14.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.031+0000 7fc1ca438640 1 --2- 172.21.15.32:0/3158313438 >> v2:172.21.15.121:3300/0 conn(0x7fc1c415e9b0 0x7fc1c40d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:14.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.031+0000 7fc1ca438640 1 -- 172.21.15.32:0/3158313438 >> 172.21.15.32:0/3158313438 conn(0x7fc1c40d11d0 msgr2=0x7fc1c40d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:14.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.031+0000 7fc1ca438640 1 -- 172.21.15.32:0/3158313438 shutdown_connections 2024-04-24T23:00:14.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.031+0000 7fc1ca438640 1 -- 172.21.15.32:0/3158313438 wait complete. 2024-04-24T23:00:14.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.032+0000 7fc1ca438640 1 Processor -- start 2024-04-24T23:00:14.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.033+0000 7fc1ca438640 1 -- start start 2024-04-24T23:00:14.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.033+0000 7fc1ca438640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc1c40ba0e0 0x7fc1c41f39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:14.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.033+0000 7fc1ca438640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1c40d4210 0x7fc1c41564b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:14.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.033+0000 7fc1ca438640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1c415e9b0 0x7fc1c41569f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:14.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.033+0000 7fc1ca438640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c40d2400 con 0x7fc1c40d4210 2024-04-24T23:00:14.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.033+0000 7fc1ca438640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c40d3cf0 con 0x7fc1c415e9b0 2024-04-24T23:00:14.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.033+0000 7fc1ca438640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc1c406b650 con 0x7fc1c40ba0e0 2024-04-24T23:00:14.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.034+0000 7fc1c37fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1c40d4210 0x7fc1c41564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:14.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.034+0000 7fc1c37fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1c40d4210 0x7fc1c41564b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41578/0 (socket says 172.21.15.32:41578) 2024-04-24T23:00:14.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.034+0000 7fc1c37fe640 1 -- 172.21.15.32:0/3486774373 learned_addr learned my addr 172.21.15.32:0/3486774373 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:14.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.034+0000 7fc1c89d5640 1 --2- 172.21.15.32:0/3486774373 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1c415e9b0 0x7fc1c41569f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:14.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.034+0000 7fc1c3fff640 1 --2- 172.21.15.32:0/3486774373 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc1c40ba0e0 0x7fc1c41f39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:14.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.034+0000 7fc1c37fe640 1 -- 172.21.15.32:0/3486774373 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc1c40ba0e0 msgr2=0x7fc1c41f39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:14.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.034+0000 7fc1c37fe640 1 --2- 172.21.15.32:0/3486774373 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc1c40ba0e0 0x7fc1c41f39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:14.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.034+0000 7fc1c37fe640 1 -- 172.21.15.32:0/3486774373 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1c415e9b0 msgr2=0x7fc1c41569f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:14.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.034+0000 7fc1c37fe640 1 --2- 172.21.15.32:0/3486774373 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1c415e9b0 0x7fc1c41569f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:14.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.034+0000 7fc1c37fe640 1 -- 172.21.15.32:0/3486774373 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1c40cd5f0 con 0x7fc1c40d4210 2024-04-24T23:00:14.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.034+0000 7fc1c89d5640 1 --2- 172.21.15.32:0/3486774373 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1c415e9b0 0x7fc1c41569f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:00:14.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.034+0000 7fc1c37fe640 1 --2- 172.21.15.32:0/3486774373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1c40d4210 0x7fc1c41564b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8009890 tx=0x7fc1b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:14.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.034+0000 7fc1c0ff9640 1 -- 172.21.15.32:0/3486774373 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b8037040 con 0x7fc1c40d4210 2024-04-24T23:00:14.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.034+0000 7fc1c0ff9640 1 -- 172.21.15.32:0/3486774373 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1b8042070 con 0x7fc1c40d4210 2024-04-24T23:00:14.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.034+0000 7fc1ca438640 1 -- 172.21.15.32:0/3486774373 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1c40cd8d0 con 0x7fc1c40d4210 2024-04-24T23:00:14.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.034+0000 7fc1c0ff9640 1 -- 172.21.15.32:0/3486774373 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b8030bd0 con 0x7fc1c40d4210 2024-04-24T23:00:14.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.034+0000 7fc1ca438640 1 -- 172.21.15.32:0/3486774373 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1c40cde90 con 0x7fc1c40d4210 2024-04-24T23:00:14.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.035+0000 7fc1ca438640 1 -- 172.21.15.32:0/3486774373 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1c4068430 con 0x7fc1c40d4210 2024-04-24T23:00:14.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.036+0000 7fc1c0ff9640 1 -- 172.21.15.32:0/3486774373 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc1b8047460 con 0x7fc1c40d4210 2024-04-24T23:00:14.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.036+0000 7fc1c0ff9640 1 --2- 172.21.15.32:0/3486774373 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc1b006be30 0x7fc1b006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:14.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.036+0000 7fc1c0ff9640 1 -- 172.21.15.32:0/3486774373 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fc1b80ab950 con 0x7fc1c40d4210 2024-04-24T23:00:14.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.036+0000 7fc1c3fff640 1 --2- 172.21.15.32:0/3486774373 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc1b006be30 0x7fc1b006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:14.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.037+0000 7fc1c3fff640 1 --2- 172.21.15.32:0/3486774373 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc1b006be30 0x7fc1b006e2f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4006fd0 tx=0x7fc1b4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:14.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.040+0000 7fc1c0ff9640 1 -- 172.21.15.32:0/3486774373 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc1b8032e20 con 0x7fc1c40d4210 2024-04-24T23:00:14.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.227+0000 7fc1ca438640 1 -- 172.21.15.32:0/3486774373 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc1c4072220 con 0x7fc1c40d4210 2024-04-24T23:00:14.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.228+0000 7fc1c0ff9640 1 -- 172.21.15.32:0/3486774373 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16621 (secure 0 0 0) 0x7fc1b807a390 con 0x7fc1c40d4210 2024-04-24T23:00:14.230 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:00:14.230 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":34,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:00:12.312472+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:59:56.068812+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-24T22:59:56.447402+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:00:14.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.230+0000 7fc1ca438640 1 -- 172.21.15.32:0/3486774373 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc1b006be30 msgr2=0x7fc1b006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:14.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.230+0000 7fc1ca438640 1 --2- 172.21.15.32:0/3486774373 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc1b006be30 0x7fc1b006e2f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4006fd0 tx=0x7fc1b4008040 comp rx=0 tx=0).stop 2024-04-24T23:00:14.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.230+0000 7fc1ca438640 1 -- 172.21.15.32:0/3486774373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1c40d4210 msgr2=0x7fc1c41564b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:14.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.230+0000 7fc1ca438640 1 --2- 172.21.15.32:0/3486774373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1c40d4210 0x7fc1c41564b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8009890 tx=0x7fc1b80046c0 comp rx=0 tx=0).stop 2024-04-24T23:00:14.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.231+0000 7fc1ca438640 1 -- 172.21.15.32:0/3486774373 shutdown_connections 2024-04-24T23:00:14.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.231+0000 7fc1ca438640 1 --2- 172.21.15.32:0/3486774373 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc1b006be30 0x7fc1b006e2f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:14.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.231+0000 7fc1ca438640 1 --2- 172.21.15.32:0/3486774373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1c40d4210 0x7fc1c41564b0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:14.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.231+0000 7fc1ca438640 1 --2- 172.21.15.32:0/3486774373 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1c415e9b0 0x7fc1c41569f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:14.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.231+0000 7fc1ca438640 1 --2- 172.21.15.32:0/3486774373 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc1c40ba0e0 0x7fc1c41f39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:14.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.231+0000 7fc1ca438640 1 -- 172.21.15.32:0/3486774373 >> 172.21.15.32:0/3486774373 conn(0x7fc1c40d11d0 msgr2=0x7fc1c4155a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:14.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.231+0000 7fc1ca438640 1 -- 172.21.15.32:0/3486774373 shutdown_connections 2024-04-24T23:00:14.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.231+0000 7fc1ca438640 1 -- 172.21.15.32:0/3486774373 wait complete. 2024-04-24T23:00:14.243 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T22:59:56.068812+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '31', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T22:59:56.447402+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:00:14.243 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:00:14.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.346+0000 7f4be609a640 1 Processor -- start 2024-04-24T23:00:14.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.346+0000 7f4be609a640 1 -- start start 2024-04-24T23:00:14.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.347+0000 7f4be609a640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4be0158dd0 0x7f4be006b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:14.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.347+0000 7f4be609a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4be006b860 0x7f4be006d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:14.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.347+0000 7f4be609a640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4be00b6d10 con 0x7f4be006da90 2024-04-24T23:00:14.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.347+0000 7f4be609a640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f4be006bd10 con 0x7f4be006b860 2024-04-24T23:00:14.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.347+0000 7f4be609a640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f4be015ef10 con 0x7f4be0158dd0 2024-04-24T23:00:14.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.347+0000 7f4bdeffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4be006b860 0x7f4be006d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:14.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.347+0000 7f4bdffff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4be006da90 0x7f4be0072580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58076/0 (socket says 172.21.15.32:58076) 2024-04-24T23:00:14.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.347+0000 7f4bdffff640 1 -- 172.21.15.32:0/2356924014 learned_addr learned my addr 172.21.15.32:0/2356924014 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:00:14.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.347+0000 7f4bdf7fe640 1 --2- 172.21.15.32:0/2356924014 >> v2:172.21.15.121:3300/0 conn(0x7f4be0158dd0 0x7f4be006b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:14.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.347+0000 7f4bdeffd640 1 -- 172.21.15.32:0/2356924014 >> v2:172.21.15.121:3300/0 conn(0x7f4be0158dd0 msgr2=0x7f4be006b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:14.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.347+0000 7f4bdeffd640 1 --2- 172.21.15.32:0/2356924014 >> v2:172.21.15.121:3300/0 conn(0x7f4be0158dd0 0x7f4be006b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:14.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.347+0000 7f4bdeffd640 1 -- 172.21.15.32:0/2356924014 >> v1:172.21.15.32:6789/0 conn(0x7f4be006da90 legacy=0x7f4be0072580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:14.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.347+0000 7f4bde7fc640 1 -- 172.21.15.32:0/2356924014 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3714538505 0 0) 0x7f4be00b6d10 con 0x7f4be006da90 2024-04-24T23:00:14.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.347+0000 7f4bdeffd640 1 -- 172.21.15.32:0/2356924014 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4be015f1f0 con 0x7f4be006b860 2024-04-24T23:00:14.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.347+0000 7f4bdf7fe640 1 --2- 172.21.15.32:0/2356924014 >> v2:172.21.15.121:3300/0 conn(0x7f4be0158dd0 0x7f4be006b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:00:14.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.348+0000 7f4bdeffd640 1 --2- 172.21.15.32:0/2356924014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4be006b860 0x7f4be006d550 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f4be006f3a0 tx=0x7f4bd002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=282d62965691edde server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:14.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.348+0000 7f4bde7fc640 1 -- 172.21.15.32:0/2356924014 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd0037070 con 0x7f4be006b860 2024-04-24T23:00:14.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.348+0000 7f4bde7fc640 1 -- 172.21.15.32:0/2356924014 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bd0032440 con 0x7f4be006b860 2024-04-24T23:00:14.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.348+0000 7f4bde7fc640 1 -- 172.21.15.32:0/2356924014 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd0032810 con 0x7f4be006b860 2024-04-24T23:00:14.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.349+0000 7f4be609a640 1 -- 172.21.15.32:0/2356924014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4be006b860 msgr2=0x7f4be006d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:14.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.349+0000 7f4be609a640 1 --2- 172.21.15.32:0/2356924014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4be006b860 0x7f4be006d550 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f4be006f3a0 tx=0x7f4bd002a1a0 comp rx=0 tx=0).stop 2024-04-24T23:00:14.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.349+0000 7f4be609a640 1 -- 172.21.15.32:0/2356924014 shutdown_connections 2024-04-24T23:00:14.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.349+0000 7f4be609a640 1 --2- 172.21.15.32:0/2356924014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4be006b860 0x7f4be006d550 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:14.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.349+0000 7f4be609a640 1 --2- 172.21.15.32:0/2356924014 >> v2:172.21.15.121:3300/0 conn(0x7f4be0158dd0 0x7f4be006b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:14.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.349+0000 7f4be609a640 1 -- 172.21.15.32:0/2356924014 >> 172.21.15.32:0/2356924014 conn(0x7f4be006e800 msgr2=0x7f4be006ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:14.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.349+0000 7f4be609a640 1 -- 172.21.15.32:0/2356924014 shutdown_connections 2024-04-24T23:00:14.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.349+0000 7f4be609a640 1 -- 172.21.15.32:0/2356924014 wait complete. 2024-04-24T23:00:14.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.350+0000 7f4be609a640 1 Processor -- start 2024-04-24T23:00:14.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.350+0000 7f4be609a640 1 -- start start 2024-04-24T23:00:14.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.351+0000 7f4be609a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4be006b860 0x7f4be0149e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:14.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.351+0000 7f4be609a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4be006da90 0x7f4be00b34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:14.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.351+0000 7f4be609a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4be0158dd0 0x7f4be0160820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:14.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.351+0000 7f4be609a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4be015fcd0 con 0x7f4be0158dd0 2024-04-24T23:00:14.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.351+0000 7f4be609a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4be015f7b0 con 0x7f4be006da90 2024-04-24T23:00:14.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.351+0000 7f4be609a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f4be006ded0 con 0x7f4be006b860 2024-04-24T23:00:14.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.351+0000 7f4bdffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4be0158dd0 0x7f4be0160820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:14.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.351+0000 7f4bdffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4be0158dd0 0x7f4be0160820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41582/0 (socket says 172.21.15.32:41582) 2024-04-24T23:00:14.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.351+0000 7f4bdffff640 1 -- 172.21.15.32:0/4013091261 learned_addr learned my addr 172.21.15.32:0/4013091261 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:14.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.351+0000 7f4bdeffd640 1 --2- 172.21.15.32:0/4013091261 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4be006da90 0x7f4be00b34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:14.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.351+0000 7f4bdf7fe640 1 --2- 172.21.15.32:0/4013091261 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4be006b860 0x7f4be0149e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:14.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.351+0000 7f4bdffff640 1 -- 172.21.15.32:0/4013091261 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4be006b860 msgr2=0x7f4be0149e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:14.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.351+0000 7f4bdffff640 1 --2- 172.21.15.32:0/4013091261 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4be006b860 0x7f4be0149e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:14.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.351+0000 7f4bdffff640 1 -- 172.21.15.32:0/4013091261 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4be006da90 msgr2=0x7f4be00b34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:14.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.351+0000 7f4bdffff640 1 --2- 172.21.15.32:0/4013091261 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4be006da90 0x7f4be00b34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:14.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.351+0000 7f4bdffff640 1 -- 172.21.15.32:0/4013091261 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4be014b8e0 con 0x7f4be0158dd0 2024-04-24T23:00:14.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.351+0000 7f4bdffff640 1 --2- 172.21.15.32:0/4013091261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4be0158dd0 0x7f4be0160820 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f4bd800c7a0 tx=0x7f4bd800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:14.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.351+0000 7f4bc7fff640 1 -- 172.21.15.32:0/4013091261 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd8018070 con 0x7f4be0158dd0 2024-04-24T23:00:14.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.351+0000 7f4bc7fff640 1 -- 172.21.15.32:0/4013091261 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bd800a640 con 0x7f4be0158dd0 2024-04-24T23:00:14.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.351+0000 7f4be609a640 1 -- 172.21.15.32:0/4013091261 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4be014bbc0 con 0x7f4be0158dd0 2024-04-24T23:00:14.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.351+0000 7f4bc7fff640 1 -- 172.21.15.32:0/4013091261 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd800aab0 con 0x7f4be0158dd0 2024-04-24T23:00:14.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.352+0000 7f4be609a640 1 -- 172.21.15.32:0/4013091261 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4be014c460 con 0x7f4be0158dd0 2024-04-24T23:00:14.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.353+0000 7f4be609a640 1 -- 172.21.15.32:0/4013091261 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4be014bd90 con 0x7f4be0158dd0 2024-04-24T23:00:14.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.353+0000 7f4bc7fff640 1 -- 172.21.15.32:0/4013091261 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4bd801b070 con 0x7f4be0158dd0 2024-04-24T23:00:14.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.353+0000 7f4bc7fff640 1 --2- 172.21.15.32:0/4013091261 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4bcc06be30 0x7f4bcc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:14.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.353+0000 7f4bc7fff640 1 -- 172.21.15.32:0/4013091261 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f4bd808a410 con 0x7f4be0158dd0 2024-04-24T23:00:14.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.354+0000 7f4bdf7fe640 1 --2- 172.21.15.32:0/4013091261 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4bcc06be30 0x7f4bcc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:14.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.354+0000 7f4bdf7fe640 1 --2- 172.21.15.32:0/4013091261 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4bcc06be30 0x7f4bcc06e2f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4bd400b840 tx=0x7f4bd4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:14.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.358+0000 7f4bc7fff640 1 -- 172.21.15.32:0/4013091261 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4bd8058e50 con 0x7f4be0158dd0 2024-04-24T23:00:14.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.540+0000 7f4be609a640 1 -- 172.21.15.32:0/4013091261 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4be0068430 con 0x7f4be0158dd0 2024-04-24T23:00:14.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.540+0000 7f4bc7fff640 1 -- 172.21.15.32:0/4013091261 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) ==== 93+0+10 (secure 0 0 0) 0x7f4bd8058750 con 0x7f4be0158dd0 2024-04-24T23:00:14.542 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T23:00:14.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.542+0000 7f4be609a640 1 -- 172.21.15.32:0/4013091261 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4bcc06be30 msgr2=0x7f4bcc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:14.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.542+0000 7f4be609a640 1 --2- 172.21.15.32:0/4013091261 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4bcc06be30 0x7f4bcc06e2f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4bd400b840 tx=0x7f4bd4008040 comp rx=0 tx=0).stop 2024-04-24T23:00:14.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.542+0000 7f4be609a640 1 -- 172.21.15.32:0/4013091261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4be0158dd0 msgr2=0x7f4be0160820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:14.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.542+0000 7f4be609a640 1 --2- 172.21.15.32:0/4013091261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4be0158dd0 0x7f4be0160820 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f4bd800c7a0 tx=0x7f4bd800cc60 comp rx=0 tx=0).stop 2024-04-24T23:00:14.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.542+0000 7f4be609a640 1 -- 172.21.15.32:0/4013091261 shutdown_connections 2024-04-24T23:00:14.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.542+0000 7f4be609a640 1 --2- 172.21.15.32:0/4013091261 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4bcc06be30 0x7f4bcc06e2f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:14.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.542+0000 7f4be609a640 1 --2- 172.21.15.32:0/4013091261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4be0158dd0 0x7f4be0160820 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:14.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.542+0000 7f4be609a640 1 --2- 172.21.15.32:0/4013091261 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4be006da90 0x7f4be00b34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:14.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.542+0000 7f4be609a640 1 --2- 172.21.15.32:0/4013091261 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4be006b860 0x7f4be0149e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:14.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.542+0000 7f4be609a640 1 -- 172.21.15.32:0/4013091261 >> 172.21.15.32:0/4013091261 conn(0x7f4be006e800 msgr2=0x7f4be006c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:14.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.542+0000 7f4be609a640 1 -- 172.21.15.32:0/4013091261 shutdown_connections 2024-04-24T23:00:14.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.542+0000 7f4be609a640 1 -- 172.21.15.32:0/4013091261 wait complete. 2024-04-24T23:00:14.555 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:00:14.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.656+0000 7f7d98be9640 1 Processor -- start 2024-04-24T23:00:14.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.656+0000 7f7d98be9640 1 -- start start 2024-04-24T23:00:14.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.657+0000 7f7d98be9640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d9415e9b0 0x7f7d940d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:14.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.657+0000 7f7d98be9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7d9406b820 0x7f7d940b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:14.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.657+0000 7f7d98be9640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d940d3170 con 0x7f7d940d4210 2024-04-24T23:00:14.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.657+0000 7f7d98be9640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f7d940d3450 con 0x7f7d9406b820 2024-04-24T23:00:14.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.657+0000 7f7d98be9640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f7d940d3730 con 0x7f7d9415e9b0 2024-04-24T23:00:14.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.658+0000 7f7d92d9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7d9406b820 0x7f7d940b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:14.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.658+0000 7f7d91d9b640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7d940d4210 0x7f7d9406b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58080/0 (socket says 172.21.15.32:58080) 2024-04-24T23:00:14.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.658+0000 7f7d91d9b640 1 -- 172.21.15.32:0/3145426294 learned_addr learned my addr 172.21.15.32:0/3145426294 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:00:14.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.658+0000 7f7d9259c640 1 --2- 172.21.15.32:0/3145426294 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d9415e9b0 0x7f7d940d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:14.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.658+0000 7f7d92d9d640 1 -- 172.21.15.32:0/3145426294 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d9415e9b0 msgr2=0x7f7d940d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:14.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.658+0000 7f7d92d9d640 1 --2- 172.21.15.32:0/3145426294 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d9415e9b0 0x7f7d940d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:14.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.658+0000 7f7d92d9d640 1 -- 172.21.15.32:0/3145426294 >> v1:172.21.15.32:6789/0 conn(0x7f7d940d4210 legacy=0x7f7d9406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:14.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.658+0000 7f7d9159a640 1 -- 172.21.15.32:0/3145426294 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 965396230 0 0) 0x7f7d940d3170 con 0x7f7d940d4210 2024-04-24T23:00:14.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.658+0000 7f7d92d9d640 1 -- 172.21.15.32:0/3145426294 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d940d3a10 con 0x7f7d9406b820 2024-04-24T23:00:14.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.658+0000 7f7d9259c640 1 --2- 172.21.15.32:0/3145426294 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d9415e9b0 0x7f7d940d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:00:14.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.659+0000 7f7d92d9d640 1 --2- 172.21.15.32:0/3145426294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d9406b820 0x7f7d940b78f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f7d84009d90 tx=0x7f7d8402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bda0dc68215e348 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:14.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.659+0000 7f7d9159a640 1 -- 172.21.15.32:0/3145426294 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d84007740 con 0x7f7d9406b820 2024-04-24T23:00:14.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.659+0000 7f7d9159a640 1 -- 172.21.15.32:0/3145426294 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d840078e0 con 0x7f7d9406b820 2024-04-24T23:00:14.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.659+0000 7f7d9159a640 1 -- 172.21.15.32:0/3145426294 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d84007cb0 con 0x7f7d9406b820 2024-04-24T23:00:14.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.659+0000 7f7d98be9640 1 -- 172.21.15.32:0/3145426294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d9406b820 msgr2=0x7f7d940b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:14.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.659+0000 7f7d98be9640 1 --2- 172.21.15.32:0/3145426294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d9406b820 0x7f7d940b78f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f7d84009d90 tx=0x7f7d8402a660 comp rx=0 tx=0).stop 2024-04-24T23:00:14.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.659+0000 7f7d98be9640 1 -- 172.21.15.32:0/3145426294 shutdown_connections 2024-04-24T23:00:14.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.659+0000 7f7d98be9640 1 --2- 172.21.15.32:0/3145426294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d9406b820 0x7f7d940b78f0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:14.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.659+0000 7f7d98be9640 1 --2- 172.21.15.32:0/3145426294 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d9415e9b0 0x7f7d940d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:14.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.659+0000 7f7d98be9640 1 -- 172.21.15.32:0/3145426294 >> 172.21.15.32:0/3145426294 conn(0x7f7d940d11d0 msgr2=0x7f7d940d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:14.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.660+0000 7f7d98be9640 1 -- 172.21.15.32:0/3145426294 shutdown_connections 2024-04-24T23:00:14.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.660+0000 7f7d98be9640 1 -- 172.21.15.32:0/3145426294 wait complete. 2024-04-24T23:00:14.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.661+0000 7f7d98be9640 1 Processor -- start 2024-04-24T23:00:14.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.661+0000 7f7d98be9640 1 -- start start 2024-04-24T23:00:14.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.662+0000 7f7d98be9640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d9406b820 0x7f7d94161910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:14.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.662+0000 7f7d98be9640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d940d4210 0x7f7d9415ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:14.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.662+0000 7f7d98be9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d9415e9b0 0x7f7d9415f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:14.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.662+0000 7f7d98be9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7d940ba0e0 con 0x7f7d9415e9b0 2024-04-24T23:00:14.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.662+0000 7f7d98be9640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7d940d2400 con 0x7f7d9406b820 2024-04-24T23:00:14.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.662+0000 7f7d98be9640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f7d940ba260 con 0x7f7d940d4210 2024-04-24T23:00:14.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.662+0000 7f7d92d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d9415e9b0 0x7f7d9415f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:14.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.662+0000 7f7d92d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d9415e9b0 0x7f7d9415f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41608/0 (socket says 172.21.15.32:41608) 2024-04-24T23:00:14.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.662+0000 7f7d92d9d640 1 -- 172.21.15.32:0/491589416 learned_addr learned my addr 172.21.15.32:0/491589416 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:14.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.662+0000 7f7d9259c640 1 --2- 172.21.15.32:0/491589416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d9406b820 0x7f7d94161910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:14.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.662+0000 7f7d91d9b640 1 --2- 172.21.15.32:0/491589416 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d940d4210 0x7f7d9415ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:14.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.662+0000 7f7d92d9d640 1 -- 172.21.15.32:0/491589416 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d940d4210 msgr2=0x7f7d9415ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:14.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.662+0000 7f7d92d9d640 1 --2- 172.21.15.32:0/491589416 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d940d4210 0x7f7d9415ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:14.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.662+0000 7f7d92d9d640 1 -- 172.21.15.32:0/491589416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d9406b820 msgr2=0x7f7d94161910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:14.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.662+0000 7f7d92d9d640 1 --2- 172.21.15.32:0/491589416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d9406b820 0x7f7d94161910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:14.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.662+0000 7f7d92d9d640 1 -- 172.21.15.32:0/491589416 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d940cd9e0 con 0x7f7d9415e9b0 2024-04-24T23:00:14.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.662+0000 7f7d92d9d640 1 --2- 172.21.15.32:0/491589416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d9415e9b0 0x7f7d9415f2d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7d84009d90 tx=0x7f7d84004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:14.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.662+0000 7f7d8affd640 1 -- 172.21.15.32:0/491589416 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d84040070 con 0x7f7d9415e9b0 2024-04-24T23:00:14.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.662+0000 7f7d8affd640 1 -- 172.21.15.32:0/491589416 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d84031c40 con 0x7f7d9415e9b0 2024-04-24T23:00:14.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.662+0000 7f7d98be9640 1 -- 172.21.15.32:0/491589416 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d940cdcc0 con 0x7f7d9415e9b0 2024-04-24T23:00:14.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.662+0000 7f7d8affd640 1 -- 172.21.15.32:0/491589416 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d84009450 con 0x7f7d9415e9b0 2024-04-24T23:00:14.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.663+0000 7f7d98be9640 1 -- 172.21.15.32:0/491589416 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d940ce280 con 0x7f7d9415e9b0 2024-04-24T23:00:14.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.663+0000 7f7d98be9640 1 -- 172.21.15.32:0/491589416 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d94068430 con 0x7f7d9415e9b0 2024-04-24T23:00:14.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.664+0000 7f7d8affd640 1 -- 172.21.15.32:0/491589416 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7d84039040 con 0x7f7d9415e9b0 2024-04-24T23:00:14.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.664+0000 7f7d8affd640 1 --2- 172.21.15.32:0/491589416 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7d7406be60 0x7f7d7406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:14.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.665+0000 7f7d8affd640 1 -- 172.21.15.32:0/491589416 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f7d840abc70 con 0x7f7d9415e9b0 2024-04-24T23:00:14.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.665+0000 7f7d9259c640 1 --2- 172.21.15.32:0/491589416 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7d7406be60 0x7f7d7406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:14.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.665+0000 7f7d9259c640 1 --2- 172.21.15.32:0/491589416 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7d7406be60 0x7f7d7406e320 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c006fd0 tx=0x7f7d8c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:14.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.669+0000 7f7d8affd640 1 -- 172.21.15.32:0/491589416 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7d8407a060 con 0x7f7d9415e9b0 2024-04-24T23:00:14.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.851+0000 7f7d98be9640 1 -- 172.21.15.32:0/491589416 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7d9415dd40 con 0x7f7d9415e9b0 2024-04-24T23:00:14.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.851+0000 7f7d8affd640 1 -- 172.21.15.32:0/491589416 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) ==== 104+0+11 (secure 0 0 0) 0x7f7d8407e360 con 0x7f7d9415e9b0 2024-04-24T23:00:14.853 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-24T23:00:14.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.853+0000 7f7d98be9640 1 -- 172.21.15.32:0/491589416 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7d7406be60 msgr2=0x7f7d7406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:14.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.853+0000 7f7d98be9640 1 --2- 172.21.15.32:0/491589416 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7d7406be60 0x7f7d7406e320 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c006fd0 tx=0x7f7d8c008040 comp rx=0 tx=0).stop 2024-04-24T23:00:14.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.853+0000 7f7d98be9640 1 -- 172.21.15.32:0/491589416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d9415e9b0 msgr2=0x7f7d9415f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:14.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.853+0000 7f7d98be9640 1 --2- 172.21.15.32:0/491589416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d9415e9b0 0x7f7d9415f2d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7d84009d90 tx=0x7f7d84004a40 comp rx=0 tx=0).stop 2024-04-24T23:00:14.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.853+0000 7f7d98be9640 1 -- 172.21.15.32:0/491589416 shutdown_connections 2024-04-24T23:00:14.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.853+0000 7f7d98be9640 1 --2- 172.21.15.32:0/491589416 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7d7406be60 0x7f7d7406e320 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:14.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.853+0000 7f7d98be9640 1 --2- 172.21.15.32:0/491589416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d9415e9b0 0x7f7d9415f2d0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:14.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.853+0000 7f7d98be9640 1 --2- 172.21.15.32:0/491589416 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d940d4210 0x7f7d9415ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:14.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.853+0000 7f7d98be9640 1 --2- 172.21.15.32:0/491589416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d9406b820 0x7f7d94161910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:14.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.853+0000 7f7d98be9640 1 -- 172.21.15.32:0/491589416 >> 172.21.15.32:0/491589416 conn(0x7f7d940d11d0 msgr2=0x7f7d940b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:14.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.853+0000 7f7d98be9640 1 -- 172.21.15.32:0/491589416 shutdown_connections 2024-04-24T23:00:14.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.853+0000 7f7d98be9640 1 -- 172.21.15.32:0/491589416 wait complete. 2024-04-24T23:00:14.866 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:00:14.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.969+0000 7f6101a9a640 1 Processor -- start 2024-04-24T23:00:14.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.969+0000 7f6101a9a640 1 -- start start 2024-04-24T23:00:14.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.970+0000 7f6101a9a640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f60fc15c8a0 0x7f60fc0ae140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:14.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.970+0000 7f6101a9a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f60fc0ce7a0 0x7f60fc06c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:14.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.970+0000 7f6101a9a640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60fc074f00 con 0x7f60fc074210 2024-04-24T23:00:14.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.970+0000 7f6101a9a640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f60fc06ca70 con 0x7f60fc0ce7a0 2024-04-24T23:00:14.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.970+0000 7f6101a9a640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f60fc0b0de0 con 0x7f60fc15c8a0 2024-04-24T23:00:14.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.970+0000 7f60faffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f60fc0ce7a0 0x7f60fc06c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:14.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.970+0000 7f60faffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f60fc0ce7a0 0x7f60fc06c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41610/0 (socket says 172.21.15.32:41610) 2024-04-24T23:00:14.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.970+0000 7f60faffd640 1 -- 172.21.15.32:0/1687497921 learned_addr learned my addr 172.21.15.32:0/1687497921 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:14.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.970+0000 7f60faffd640 1 -- 172.21.15.32:0/1687497921 >> v2:172.21.15.121:3300/0 conn(0x7f60fc15c8a0 msgr2=0x7f60fc0ae140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:14.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.970+0000 7f60faffd640 1 --2- 172.21.15.32:0/1687497921 >> v2:172.21.15.121:3300/0 conn(0x7f60fc15c8a0 0x7f60fc0ae140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:14.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.970+0000 7f60faffd640 1 -- 172.21.15.32:0/1687497921 >> v1:172.21.15.121:6789/0 conn(0x7f60fc074210 legacy=0x7f60fc0b1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:14.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.970+0000 7f60faffd640 1 -- 172.21.15.32:0/1687497921 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60fc0b10c0 con 0x7f60fc0ce7a0 2024-04-24T23:00:14.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.971+0000 7f60faffd640 1 --2- 172.21.15.32:0/1687497921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60fc0ce7a0 0x7f60fc06c390 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f60fc070b50 tx=0x7f60f002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12f53309f3ff33fd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:14.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.971+0000 7f60fa7fc640 1 -- 172.21.15.32:0/1687497921 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f0037070 con 0x7f60fc0ce7a0 2024-04-24T23:00:14.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.971+0000 7f60fa7fc640 1 -- 172.21.15.32:0/1687497921 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60f0032440 con 0x7f60fc0ce7a0 2024-04-24T23:00:14.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.971+0000 7f60fa7fc640 1 -- 172.21.15.32:0/1687497921 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f0032810 con 0x7f60fc0ce7a0 2024-04-24T23:00:14.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.971+0000 7f6101a9a640 1 -- 172.21.15.32:0/1687497921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60fc0ce7a0 msgr2=0x7f60fc06c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:14.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.971+0000 7f6101a9a640 1 --2- 172.21.15.32:0/1687497921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60fc0ce7a0 0x7f60fc06c390 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f60fc070b50 tx=0x7f60f002a1a0 comp rx=0 tx=0).stop 2024-04-24T23:00:14.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.972+0000 7f6101a9a640 1 -- 172.21.15.32:0/1687497921 shutdown_connections 2024-04-24T23:00:14.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.972+0000 7f6101a9a640 1 --2- 172.21.15.32:0/1687497921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60fc0ce7a0 0x7f60fc06c390 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:14.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.972+0000 7f6101a9a640 1 --2- 172.21.15.32:0/1687497921 >> v2:172.21.15.121:3300/0 conn(0x7f60fc15c8a0 0x7f60fc0ae140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:14.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.972+0000 7f6101a9a640 1 -- 172.21.15.32:0/1687497921 >> 172.21.15.32:0/1687497921 conn(0x7f60fc0b2e60 msgr2=0x7f60fc0b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:14.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.972+0000 7f6101a9a640 1 -- 172.21.15.32:0/1687497921 shutdown_connections 2024-04-24T23:00:14.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.972+0000 7f6101a9a640 1 -- 172.21.15.32:0/1687497921 wait complete. 2024-04-24T23:00:14.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.973+0000 7f6101a9a640 1 Processor -- start 2024-04-24T23:00:14.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.973+0000 7f6101a9a640 1 -- start start 2024-04-24T23:00:14.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.974+0000 7f6101a9a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60fc074210 0x7f60fc156fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:14.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.974+0000 7f6101a9a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f60fc0ce7a0 0x7f60fc15f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:14.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.974+0000 7f6101a9a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f60fc15c8a0 0x7f60fc06ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:14.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.974+0000 7f6101a9a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc15cc80 con 0x7f60fc074210 2024-04-24T23:00:14.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.974+0000 7f6101a9a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc0b1680 con 0x7f60fc15c8a0 2024-04-24T23:00:14.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.974+0000 7f6101a9a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f60fc0c9120 con 0x7f60fc0ce7a0 2024-04-24T23:00:14.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.974+0000 7f60fb7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60fc074210 0x7f60fc156fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:14.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.974+0000 7f60fb7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60fc074210 0x7f60fc156fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41616/0 (socket says 172.21.15.32:41616) 2024-04-24T23:00:14.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.974+0000 7f60fb7fe640 1 -- 172.21.15.32:0/1599017412 learned_addr learned my addr 172.21.15.32:0/1599017412 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:14.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.974+0000 7f60faffd640 1 --2- 172.21.15.32:0/1599017412 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f60fc0ce7a0 0x7f60fc15f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:14.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.974+0000 7f60fbfff640 1 --2- 172.21.15.32:0/1599017412 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f60fc15c8a0 0x7f60fc06ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:14.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.974+0000 7f60fb7fe640 1 -- 172.21.15.32:0/1599017412 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f60fc0ce7a0 msgr2=0x7f60fc15f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:14.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.974+0000 7f60fb7fe640 1 --2- 172.21.15.32:0/1599017412 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f60fc0ce7a0 0x7f60fc15f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:14.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.974+0000 7f60fb7fe640 1 -- 172.21.15.32:0/1599017412 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f60fc15c8a0 msgr2=0x7f60fc06ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:14.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.974+0000 7f60fb7fe640 1 --2- 172.21.15.32:0/1599017412 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f60fc15c8a0 0x7f60fc06ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:14.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.974+0000 7f60fb7fe640 1 -- 172.21.15.32:0/1599017412 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60fc0cf760 con 0x7f60fc074210 2024-04-24T23:00:14.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.974+0000 7f60fbfff640 1 --2- 172.21.15.32:0/1599017412 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f60fc15c8a0 0x7f60fc06ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:00:14.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.974+0000 7f60fb7fe640 1 --2- 172.21.15.32:0/1599017412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60fc074210 0x7f60fc156fb0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f60ec00b440 tx=0x7f60ec00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:14.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.975+0000 7f60e3fff640 1 -- 172.21.15.32:0/1599017412 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec015020 con 0x7f60fc074210 2024-04-24T23:00:14.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.975+0000 7f60e3fff640 1 -- 172.21.15.32:0/1599017412 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60ec011890 con 0x7f60fc074210 2024-04-24T23:00:14.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.975+0000 7f60e3fff640 1 -- 172.21.15.32:0/1599017412 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec011c80 con 0x7f60fc074210 2024-04-24T23:00:14.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.975+0000 7f6101a9a640 1 -- 172.21.15.32:0/1599017412 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60fc0cfa40 con 0x7f60fc074210 2024-04-24T23:00:14.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.975+0000 7f6101a9a640 1 -- 172.21.15.32:0/1599017412 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60fc0d0000 con 0x7f60fc074210 2024-04-24T23:00:14.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.976+0000 7f6101a9a640 1 -- 172.21.15.32:0/1599017412 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60fc068430 con 0x7f60fc074210 2024-04-24T23:00:14.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.976+0000 7f60e3fff640 1 -- 172.21.15.32:0/1599017412 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f60ec018020 con 0x7f60fc074210 2024-04-24T23:00:14.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.977+0000 7f60e3fff640 1 --2- 172.21.15.32:0/1599017412 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f60e806be30 0x7f60e806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:14.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.977+0000 7f60e3fff640 1 -- 172.21.15.32:0/1599017412 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f60ec08af10 con 0x7f60fc074210 2024-04-24T23:00:14.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.977+0000 7f60faffd640 1 --2- 172.21.15.32:0/1599017412 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f60e806be30 0x7f60e806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:14.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.977+0000 7f60faffd640 1 --2- 172.21.15.32:0/1599017412 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f60e806be30 0x7f60e806e2f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f60f0034000 tx=0x7f60f002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:14.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:14.981+0000 7f60e3fff640 1 -- 172.21.15.32:0/1599017412 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f60ec01f020 con 0x7f60fc074210 2024-04-24T23:00:15.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.163+0000 7f6101a9a640 1 -- 172.21.15.32:0/1599017412 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f60fc0c4550 con 0x7f60fc074210 2024-04-24T23:00:15.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.163+0000 7f60e3fff640 1 -- 172.21.15.32:0/1599017412 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) ==== 100+0+11 (secure 0 0 0) 0x7f60ec00d040 con 0x7f60fc074210 2024-04-24T23:00:15.165 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-24T23:00:15.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.166+0000 7f6101a9a640 1 -- 172.21.15.32:0/1599017412 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f60e806be30 msgr2=0x7f60e806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:15.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.166+0000 7f6101a9a640 1 --2- 172.21.15.32:0/1599017412 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f60e806be30 0x7f60e806e2f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f60f0034000 tx=0x7f60f002e000 comp rx=0 tx=0).stop 2024-04-24T23:00:15.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.166+0000 7f6101a9a640 1 -- 172.21.15.32:0/1599017412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60fc074210 msgr2=0x7f60fc156fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:15.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.166+0000 7f6101a9a640 1 --2- 172.21.15.32:0/1599017412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60fc074210 0x7f60fc156fb0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f60ec00b440 tx=0x7f60ec00b900 comp rx=0 tx=0).stop 2024-04-24T23:00:15.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.166+0000 7f6101a9a640 1 -- 172.21.15.32:0/1599017412 shutdown_connections 2024-04-24T23:00:15.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.166+0000 7f6101a9a640 1 --2- 172.21.15.32:0/1599017412 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f60e806be30 0x7f60e806e2f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:15.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.166+0000 7f6101a9a640 1 --2- 172.21.15.32:0/1599017412 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f60fc15c8a0 0x7f60fc06ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:15.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.166+0000 7f6101a9a640 1 --2- 172.21.15.32:0/1599017412 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f60fc0ce7a0 0x7f60fc15f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:15.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.166+0000 7f6101a9a640 1 --2- 172.21.15.32:0/1599017412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60fc074210 0x7f60fc156fb0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:15.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.166+0000 7f6101a9a640 1 -- 172.21.15.32:0/1599017412 >> 172.21.15.32:0/1599017412 conn(0x7f60fc0b2e60 msgr2=0x7f60fc0b6120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:15.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.166+0000 7f6101a9a640 1 -- 172.21.15.32:0/1599017412 shutdown_connections 2024-04-24T23:00:15.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.166+0000 7f6101a9a640 1 -- 172.21.15.32:0/1599017412 wait complete. 2024-04-24T23:00:15.178 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T23:00:15.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.279+0000 7fbd6e23f640 1 Processor -- start 2024-04-24T23:00:15.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.279+0000 7fbd6e23f640 1 -- start start 2024-04-24T23:00:15.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.279+0000 7fbd6e23f640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fbd680b44d0 0x7fbd680b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:15.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.279+0000 7fbd6e23f640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbd680af890 0x7fbd6815cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:15.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.280+0000 7fbd6e23f640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd680b6430 con 0x7fbd6806c5f0 2024-04-24T23:00:15.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.280+0000 7fbd6e23f640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fbd68156f80 con 0x7fbd680af890 2024-04-24T23:00:15.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.280+0000 7fbd6e23f640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fbd680b6150 con 0x7fbd680b44d0 2024-04-24T23:00:15.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.280+0000 7fbd677fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fbd6806c5f0 0x7fbd6806c9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58096/0 (socket says 172.21.15.32:58096) 2024-04-24T23:00:15.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.280+0000 7fbd677fe640 1 -- 172.21.15.32:0/3791370605 learned_addr learned my addr 172.21.15.32:0/3791370605 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:00:15.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.280+0000 7fbd67fff640 1 --2- 172.21.15.32:0/3791370605 >> v2:172.21.15.32:3300/0 conn(0x7fbd680af890 0x7fbd6815cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:15.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.280+0000 7fbd667fc640 1 -- 172.21.15.32:0/3791370605 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 442603956 0 0) 0x7fbd680b6430 con 0x7fbd6806c5f0 2024-04-24T23:00:15.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.280+0000 7fbd66ffd640 1 --2- 172.21.15.32:0/3791370605 >> v2:172.21.15.121:3300/0 conn(0x7fbd680b44d0 0x7fbd680b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:15.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.280+0000 7fbd667fc640 1 -- 172.21.15.32:0/3791370605 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd50003520 con 0x7fbd6806c5f0 2024-04-24T23:00:15.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.280+0000 7fbd67fff640 1 -- 172.21.15.32:0/3791370605 >> v2:172.21.15.121:3300/0 conn(0x7fbd680b44d0 msgr2=0x7fbd680b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:15.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.280+0000 7fbd67fff640 1 --2- 172.21.15.32:0/3791370605 >> v2:172.21.15.121:3300/0 conn(0x7fbd680b44d0 0x7fbd680b5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:15.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.280+0000 7fbd67fff640 1 -- 172.21.15.32:0/3791370605 >> v1:172.21.15.32:6789/0 conn(0x7fbd6806c5f0 legacy=0x7fbd6806c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:15.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.280+0000 7fbd67fff640 1 -- 172.21.15.32:0/3791370605 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd680b39f0 con 0x7fbd680af890 2024-04-24T23:00:15.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.281+0000 7fbd67fff640 1 --2- 172.21.15.32:0/3791370605 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd680af890 0x7fbd6815cac0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fbd68073080 tx=0x7fbd6002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcc1fdab54e5eb7a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:15.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.281+0000 7fbd667fc640 1 -- 172.21.15.32:0/3791370605 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd60007740 con 0x7fbd680af890 2024-04-24T23:00:15.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.281+0000 7fbd667fc640 1 -- 172.21.15.32:0/3791370605 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd600078e0 con 0x7fbd680af890 2024-04-24T23:00:15.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.281+0000 7fbd667fc640 1 -- 172.21.15.32:0/3791370605 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd60007c90 con 0x7fbd680af890 2024-04-24T23:00:15.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.281+0000 7fbd6e23f640 1 -- 172.21.15.32:0/3791370605 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd680af890 msgr2=0x7fbd6815cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:15.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.281+0000 7fbd6e23f640 1 --2- 172.21.15.32:0/3791370605 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd680af890 0x7fbd6815cac0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fbd68073080 tx=0x7fbd6002a680 comp rx=0 tx=0).stop 2024-04-24T23:00:15.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.281+0000 7fbd6e23f640 1 -- 172.21.15.32:0/3791370605 shutdown_connections 2024-04-24T23:00:15.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.281+0000 7fbd6e23f640 1 --2- 172.21.15.32:0/3791370605 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd680af890 0x7fbd6815cac0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:15.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.281+0000 7fbd6e23f640 1 --2- 172.21.15.32:0/3791370605 >> v2:172.21.15.121:3300/0 conn(0x7fbd680b44d0 0x7fbd680b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:15.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.281+0000 7fbd6e23f640 1 -- 172.21.15.32:0/3791370605 >> 172.21.15.32:0/3791370605 conn(0x7fbd6806b110 msgr2=0x7fbd6806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:15.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.281+0000 7fbd6e23f640 1 -- 172.21.15.32:0/3791370605 shutdown_connections 2024-04-24T23:00:15.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.281+0000 7fbd6e23f640 1 -- 172.21.15.32:0/3791370605 wait complete. 2024-04-24T23:00:15.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.283+0000 7fbd6e23f640 1 Processor -- start 2024-04-24T23:00:15.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.283+0000 7fbd6e23f640 1 -- start start 2024-04-24T23:00:15.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.283+0000 7fbd6e23f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd6806c5f0 0x7fbd68150f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:15.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.283+0000 7fbd6e23f640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbd680af890 0x7fbd68157470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:15.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.283+0000 7fbd6e23f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd680b44d0 0x7fbd681579b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:15.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.283+0000 7fbd6e23f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbd680b5dd0 con 0x7fbd6806c5f0 2024-04-24T23:00:15.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.283+0000 7fbd6e23f640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fbd680b3fb0 con 0x7fbd680b44d0 2024-04-24T23:00:15.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.283+0000 7fbd6e23f640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fbd6815d000 con 0x7fbd680af890 2024-04-24T23:00:15.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.284+0000 7fbd677fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd6806c5f0 0x7fbd68150f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:15.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.284+0000 7fbd677fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd6806c5f0 0x7fbd68150f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41638/0 (socket says 172.21.15.32:41638) 2024-04-24T23:00:15.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.284+0000 7fbd677fe640 1 -- 172.21.15.32:0/1040539994 learned_addr learned my addr 172.21.15.32:0/1040539994 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:15.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.284+0000 7fbd67fff640 1 --2- 172.21.15.32:0/1040539994 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd680b44d0 0x7fbd681579b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:15.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.284+0000 7fbd66ffd640 1 --2- 172.21.15.32:0/1040539994 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbd680af890 0x7fbd68157470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:15.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.284+0000 7fbd677fe640 1 -- 172.21.15.32:0/1040539994 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbd680af890 msgr2=0x7fbd68157470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:15.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.284+0000 7fbd677fe640 1 --2- 172.21.15.32:0/1040539994 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbd680af890 0x7fbd68157470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:15.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.284+0000 7fbd677fe640 1 -- 172.21.15.32:0/1040539994 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd680b44d0 msgr2=0x7fbd681579b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:15.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.284+0000 7fbd677fe640 1 --2- 172.21.15.32:0/1040539994 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd680b44d0 0x7fbd681579b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:15.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.284+0000 7fbd677fe640 1 -- 172.21.15.32:0/1040539994 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd681516b0 con 0x7fbd6806c5f0 2024-04-24T23:00:15.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.284+0000 7fbd677fe640 1 --2- 172.21.15.32:0/1040539994 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd6806c5f0 0x7fbd68150f40 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fbd5800c430 tx=0x7fbd5800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:15.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.284+0000 7fbd4ffff640 1 -- 172.21.15.32:0/1040539994 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd58017070 con 0x7fbd6806c5f0 2024-04-24T23:00:15.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.284+0000 7fbd4ffff640 1 -- 172.21.15.32:0/1040539994 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd58005ce0 con 0x7fbd6806c5f0 2024-04-24T23:00:15.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.284+0000 7fbd4ffff640 1 -- 172.21.15.32:0/1040539994 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd580072f0 con 0x7fbd6806c5f0 2024-04-24T23:00:15.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.284+0000 7fbd6e23f640 1 -- 172.21.15.32:0/1040539994 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd68151990 con 0x7fbd6806c5f0 2024-04-24T23:00:15.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.284+0000 7fbd6e23f640 1 -- 172.21.15.32:0/1040539994 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd68151f50 con 0x7fbd6806c5f0 2024-04-24T23:00:15.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.285+0000 7fbd6e23f640 1 -- 172.21.15.32:0/1040539994 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd68068430 con 0x7fbd6806c5f0 2024-04-24T23:00:15.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.286+0000 7fbd4ffff640 1 -- 172.21.15.32:0/1040539994 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbd5801a070 con 0x7fbd6806c5f0 2024-04-24T23:00:15.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.286+0000 7fbd4ffff640 1 --2- 172.21.15.32:0/1040539994 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fbd5406be30 0x7fbd5406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:15.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.286+0000 7fbd4ffff640 1 -- 172.21.15.32:0/1040539994 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fbd5808a320 con 0x7fbd6806c5f0 2024-04-24T23:00:15.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.287+0000 7fbd66ffd640 1 --2- 172.21.15.32:0/1040539994 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fbd5406be30 0x7fbd5406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:15.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.287+0000 7fbd66ffd640 1 --2- 172.21.15.32:0/1040539994 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fbd5406be30 0x7fbd5406e2f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c006fd0 tx=0x7fbd5c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:15.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.291+0000 7fbd4ffff640 1 -- 172.21.15.32:0/1040539994 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbd58058cc0 con 0x7fbd6806c5f0 2024-04-24T23:00:15.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.490+0000 7fbd6e23f640 1 -- 172.21.15.32:0/1040539994 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbd680b5670 con 0x7fbd6806c5f0 2024-04-24T23:00:15.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.491+0000 7fbd4ffff640 1 -- 172.21.15.32:0/1040539994 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v34) ==== 80+0+0 (secure 0 0 0) 0x7fbd580585c0 con 0x7fbd6806c5f0 2024-04-24T23:00:15.492 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2024-04-24T23:00:15.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.492+0000 7fbd6e23f640 1 -- 172.21.15.32:0/1040539994 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fbd5406be30 msgr2=0x7fbd5406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:15.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.492+0000 7fbd6e23f640 1 --2- 172.21.15.32:0/1040539994 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fbd5406be30 0x7fbd5406e2f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c006fd0 tx=0x7fbd5c008040 comp rx=0 tx=0).stop 2024-04-24T23:00:15.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.492+0000 7fbd6e23f640 1 -- 172.21.15.32:0/1040539994 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd6806c5f0 msgr2=0x7fbd68150f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:15.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.492+0000 7fbd6e23f640 1 --2- 172.21.15.32:0/1040539994 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd6806c5f0 0x7fbd68150f40 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fbd5800c430 tx=0x7fbd5800c8f0 comp rx=0 tx=0).stop 2024-04-24T23:00:15.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.493+0000 7fbd6e23f640 1 -- 172.21.15.32:0/1040539994 shutdown_connections 2024-04-24T23:00:15.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.493+0000 7fbd6e23f640 1 --2- 172.21.15.32:0/1040539994 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fbd5406be30 0x7fbd5406e2f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:15.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.493+0000 7fbd6e23f640 1 --2- 172.21.15.32:0/1040539994 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd680b44d0 0x7fbd681579b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:15.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.493+0000 7fbd6e23f640 1 --2- 172.21.15.32:0/1040539994 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbd680af890 0x7fbd68157470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:15.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.493+0000 7fbd6e23f640 1 --2- 172.21.15.32:0/1040539994 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd6806c5f0 0x7fbd68150f40 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:15.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.493+0000 7fbd6e23f640 1 -- 172.21.15.32:0/1040539994 >> 172.21.15.32:0/1040539994 conn(0x7fbd6806b110 msgr2=0x7fbd6806c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:15.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.493+0000 7fbd6e23f640 1 -- 172.21.15.32:0/1040539994 shutdown_connections 2024-04-24T23:00:15.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.493+0000 7fbd6e23f640 1 -- 172.21.15.32:0/1040539994 wait complete. 2024-04-24T23:00:15.505 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-24T23:00:15.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:00:15.505 DEBUG:teuthology.orchestra.run.smithi032:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-24T23:00:15.558 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2024-04-24T23:00:15.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:00:15.559 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-24T23:00:15.579 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:00:15.579 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:00:15.580 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:00:15.598 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:00:15.598 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:00:15.655 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:00:15.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.754+0000 7f46a5c12640 1 Processor -- start 2024-04-24T23:00:15.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.754+0000 7f46a5c12640 1 -- start start 2024-04-24T23:00:15.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.755+0000 7f46a5c12640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f46a00b44d0 0x7f46a00b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:15.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.755+0000 7f46a5c12640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46a00af890 0x7f46a015cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:15.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.755+0000 7f46a5c12640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46a00b6430 con 0x7f46a006c5f0 2024-04-24T23:00:15.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.755+0000 7f46a5c12640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f46a0156f80 con 0x7f46a00b44d0 2024-04-24T23:00:15.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.755+0000 7f46a5c12640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f46a00b6150 con 0x7f46a00af890 2024-04-24T23:00:15.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.756+0000 7f469ffff640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46a00af890 0x7f46a015cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:15.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.756+0000 7f469effd640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f46a00b44d0 0x7f46a00b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:15.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.756+0000 7f469ffff640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46a00af890 0x7f46a015cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.32:53484/0 (socket says 172.21.15.32:53484) 2024-04-24T23:00:15.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.756+0000 7f469ffff640 1 -- 172.21.15.32:0/4289204813 learned_addr learned my addr 172.21.15.32:0/4289204813 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:15.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.756+0000 7f469effd640 1 -- 172.21.15.32:0/4289204813 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46a00af890 msgr2=0x7f46a015cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:15.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.756+0000 7f469effd640 1 --2- 172.21.15.32:0/4289204813 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46a00af890 0x7f46a015cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:15.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.756+0000 7f469effd640 1 -- 172.21.15.32:0/4289204813 >> v1:172.21.15.121:6789/0 conn(0x7f46a006c5f0 legacy=0x7f46a006c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:15.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.756+0000 7f469e7fc640 1 -- 172.21.15.32:0/4289204813 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1075273359 0 0) 0x7f46a00b6430 con 0x7f46a006c5f0 2024-04-24T23:00:15.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.756+0000 7f469effd640 1 -- 172.21.15.32:0/4289204813 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46a00b39f0 con 0x7f46a00b44d0 2024-04-24T23:00:15.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.756+0000 7f469ffff640 1 --2- 172.21.15.32:0/4289204813 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46a00af890 0x7f46a015cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:00:15.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.757+0000 7f469effd640 1 --2- 172.21.15.32:0/4289204813 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46a00b44d0 0x7f46a00b5890 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f46940097a0 tx=0x7f469402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93d9810b89ea9306 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:15.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.757+0000 7f469e7fc640 1 -- 172.21.15.32:0/4289204813 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4694037070 con 0x7f46a00b44d0 2024-04-24T23:00:15.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.757+0000 7f469e7fc640 1 -- 172.21.15.32:0/4289204813 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f469402ae50 con 0x7f46a00b44d0 2024-04-24T23:00:15.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.757+0000 7f469e7fc640 1 -- 172.21.15.32:0/4289204813 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4694032660 con 0x7f46a00b44d0 2024-04-24T23:00:15.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.757+0000 7f46a5c12640 1 -- 172.21.15.32:0/4289204813 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46a00b44d0 msgr2=0x7f46a00b5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:15.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.757+0000 7f46a5c12640 1 --2- 172.21.15.32:0/4289204813 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46a00b44d0 0x7f46a00b5890 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f46940097a0 tx=0x7f469402a100 comp rx=0 tx=0).stop 2024-04-24T23:00:15.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.757+0000 7f46a5c12640 1 -- 172.21.15.32:0/4289204813 shutdown_connections 2024-04-24T23:00:15.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.757+0000 7f46a5c12640 1 --2- 172.21.15.32:0/4289204813 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46a00af890 0x7f46a015cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:15.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.757+0000 7f46a5c12640 1 --2- 172.21.15.32:0/4289204813 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46a00b44d0 0x7f46a00b5890 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:15.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.757+0000 7f46a5c12640 1 -- 172.21.15.32:0/4289204813 >> 172.21.15.32:0/4289204813 conn(0x7f46a006b110 msgr2=0x7f46a006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:15.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.757+0000 7f46a5c12640 1 -- 172.21.15.32:0/4289204813 shutdown_connections 2024-04-24T23:00:15.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.757+0000 7f46a5c12640 1 -- 172.21.15.32:0/4289204813 wait complete. 2024-04-24T23:00:15.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.759+0000 7f46a5c12640 1 Processor -- start 2024-04-24T23:00:15.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.759+0000 7f46a5c12640 1 -- start start 2024-04-24T23:00:15.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.759+0000 7f46a5c12640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46a006c5f0 0x7f46a0151470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:15.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.759+0000 7f46a5c12640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a00af890 0x7f46a0159da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:15.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.759+0000 7f46a5c12640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46a00b44d0 0x7f46a00ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:15.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.759+0000 7f46a5c12640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f46a00b5dd0 con 0x7f46a00af890 2024-04-24T23:00:15.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.759+0000 7f46a5c12640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f46a00b3fb0 con 0x7f46a006c5f0 2024-04-24T23:00:15.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.759+0000 7f46a5c12640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f46a015d000 con 0x7f46a00b44d0 2024-04-24T23:00:15.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.760+0000 7f469effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a00af890 0x7f46a0159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:15.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.760+0000 7f469effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a00af890 0x7f46a0159da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41646/0 (socket says 172.21.15.32:41646) 2024-04-24T23:00:15.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.760+0000 7f469effd640 1 -- 172.21.15.32:0/1283480308 learned_addr learned my addr 172.21.15.32:0/1283480308 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:15.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.760+0000 7f469f7fe640 1 --2- 172.21.15.32:0/1283480308 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46a006c5f0 0x7f46a0151470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:15.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.760+0000 7f469ffff640 1 --2- 172.21.15.32:0/1283480308 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46a00b44d0 0x7f46a00ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:15.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.760+0000 7f469effd640 1 -- 172.21.15.32:0/1283480308 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46a00b44d0 msgr2=0x7f46a00ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:15.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.760+0000 7f469effd640 1 --2- 172.21.15.32:0/1283480308 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46a00b44d0 0x7f46a00ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:15.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.760+0000 7f469effd640 1 -- 172.21.15.32:0/1283480308 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46a006c5f0 msgr2=0x7f46a0151470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:15.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.760+0000 7f469effd640 1 --2- 172.21.15.32:0/1283480308 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46a006c5f0 0x7f46a0151470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:15.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.760+0000 7f469effd640 1 -- 172.21.15.32:0/1283480308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46a015de00 con 0x7f46a00af890 2024-04-24T23:00:15.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.760+0000 7f469effd640 1 --2- 172.21.15.32:0/1283480308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a00af890 0x7f46a0159da0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f46940097a0 tx=0x7f46940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:15.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.760+0000 7f4687fff640 1 -- 172.21.15.32:0/1283480308 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4694037040 con 0x7f46a00af890 2024-04-24T23:00:15.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.760+0000 7f4687fff640 1 -- 172.21.15.32:0/1283480308 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4694004030 con 0x7f46a00af890 2024-04-24T23:00:15.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.760+0000 7f46a5c12640 1 -- 172.21.15.32:0/1283480308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46a015e0a0 con 0x7f46a00af890 2024-04-24T23:00:15.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.760+0000 7f4687fff640 1 -- 172.21.15.32:0/1283480308 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4694004420 con 0x7f46a00af890 2024-04-24T23:00:15.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.760+0000 7f46a5c12640 1 -- 172.21.15.32:0/1283480308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46a015e660 con 0x7f46a00af890 2024-04-24T23:00:15.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.761+0000 7f46a5c12640 1 -- 172.21.15.32:0/1283480308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46a0068430 con 0x7f46a00af890 2024-04-24T23:00:15.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.762+0000 7f4687fff640 1 -- 172.21.15.32:0/1283480308 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4694003610 con 0x7f46a00af890 2024-04-24T23:00:15.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.762+0000 7f4687fff640 1 --2- 172.21.15.32:0/1283480308 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f468c06be30 0x7f468c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:15.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.763+0000 7f4687fff640 1 -- 172.21.15.32:0/1283480308 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f46940ab9b0 con 0x7f46a00af890 2024-04-24T23:00:15.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.763+0000 7f469f7fe640 1 --2- 172.21.15.32:0/1283480308 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f468c06be30 0x7f468c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:15.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.763+0000 7f469f7fe640 1 --2- 172.21.15.32:0/1283480308 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f468c06be30 0x7f468c06e2f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4690000f60 tx=0x7f4690009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:15.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.767+0000 7f4687fff640 1 -- 172.21.15.32:0/1283480308 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4694079da0 con 0x7f46a00af890 2024-04-24T23:00:15.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.949+0000 7f46a5c12640 1 -- 172.21.15.32:0/1283480308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f46a00b5670 con 0x7f46a00af890 2024-04-24T23:00:15.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.950+0000 7f4687fff640 1 -- 172.21.15.32:0/1283480308 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16621 (secure 0 0 0) 0x7f469407e0a0 con 0x7f46a00af890 2024-04-24T23:00:15.952 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:00:15.952 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":34,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:00:12.312472+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:59:56.068812+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-24T22:59:56.447402+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:00:15.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.952+0000 7f46a5c12640 1 -- 172.21.15.32:0/1283480308 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f468c06be30 msgr2=0x7f468c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:15.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.952+0000 7f46a5c12640 1 --2- 172.21.15.32:0/1283480308 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f468c06be30 0x7f468c06e2f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4690000f60 tx=0x7f4690009040 comp rx=0 tx=0).stop 2024-04-24T23:00:15.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.952+0000 7f46a5c12640 1 -- 172.21.15.32:0/1283480308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a00af890 msgr2=0x7f46a0159da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:15.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.952+0000 7f46a5c12640 1 --2- 172.21.15.32:0/1283480308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a00af890 0x7f46a0159da0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f46940097a0 tx=0x7f46940046c0 comp rx=0 tx=0).stop 2024-04-24T23:00:15.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.952+0000 7f46a5c12640 1 -- 172.21.15.32:0/1283480308 shutdown_connections 2024-04-24T23:00:15.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.952+0000 7f46a5c12640 1 --2- 172.21.15.32:0/1283480308 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f468c06be30 0x7f468c06e2f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:15.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.952+0000 7f46a5c12640 1 --2- 172.21.15.32:0/1283480308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a00af890 0x7f46a0159da0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:15.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.952+0000 7f46a5c12640 1 --2- 172.21.15.32:0/1283480308 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46a00b44d0 0x7f46a00ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:15.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.952+0000 7f46a5c12640 1 --2- 172.21.15.32:0/1283480308 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46a006c5f0 0x7f46a0151470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:15.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.952+0000 7f46a5c12640 1 -- 172.21.15.32:0/1283480308 >> 172.21.15.32:0/1283480308 conn(0x7f46a006b110 msgr2=0x7f46a00d0190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:15.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.952+0000 7f46a5c12640 1 -- 172.21.15.32:0/1283480308 shutdown_connections 2024-04-24T23:00:15.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:15.952+0000 7f46a5c12640 1 -- 172.21.15.32:0/1283480308 wait complete. 2024-04-24T23:00:15.965 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T22:59:56.068812+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '31', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T22:59:56.447402+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:00:15.965 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:00:16.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.067+0000 7fd48da9a640 1 Processor -- start 2024-04-24T23:00:16.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.067+0000 7fd48da9a640 1 -- start start 2024-04-24T23:00:16.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.068+0000 7fd48da9a640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd48806b820 0x7fd4880d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:16.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.068+0000 7fd48da9a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd4880d4210 0x7fd4880b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:16.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.068+0000 7fd48da9a640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4880d3170 con 0x7fd48815e9b0 2024-04-24T23:00:16.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.068+0000 7fd48da9a640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fd4880d3450 con 0x7fd4880d4210 2024-04-24T23:00:16.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.068+0000 7fd48da9a640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fd4880d3730 con 0x7fd48806b820 2024-04-24T23:00:16.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.068+0000 7fd487fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd4880d4210 0x7fd4880b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:16.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.068+0000 7fd4877fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd48815e9b0 0x7fd48806b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58112/0 (socket says 172.21.15.32:58112) 2024-04-24T23:00:16.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.068+0000 7fd4877fe640 1 -- 172.21.15.32:0/1095531976 learned_addr learned my addr 172.21.15.32:0/1095531976 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:00:16.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.069+0000 7fd486ffd640 1 --2- 172.21.15.32:0/1095531976 >> v2:172.21.15.121:3300/0 conn(0x7fd48806b820 0x7fd4880d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:16.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.069+0000 7fd487fff640 1 -- 172.21.15.32:0/1095531976 >> v2:172.21.15.121:3300/0 conn(0x7fd48806b820 msgr2=0x7fd4880d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:16.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.069+0000 7fd487fff640 1 --2- 172.21.15.32:0/1095531976 >> v2:172.21.15.121:3300/0 conn(0x7fd48806b820 0x7fd4880d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:16.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.069+0000 7fd487fff640 1 -- 172.21.15.32:0/1095531976 >> v1:172.21.15.32:6789/0 conn(0x7fd48815e9b0 legacy=0x7fd48806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:16.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.069+0000 7fd4867fc640 1 -- 172.21.15.32:0/1095531976 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 780991395 0 0) 0x7fd4880d3170 con 0x7fd48815e9b0 2024-04-24T23:00:16.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.069+0000 7fd487fff640 1 -- 172.21.15.32:0/1095531976 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4880d3a10 con 0x7fd4880d4210 2024-04-24T23:00:16.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.069+0000 7fd487fff640 1 --2- 172.21.15.32:0/1095531976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4880d4210 0x7fd4880b78f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd4880730f0 tx=0x7fd48002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85a1a08355ddc37b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:16.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.070+0000 7fd4867fc640 1 -- 172.21.15.32:0/1095531976 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd480007740 con 0x7fd4880d4210 2024-04-24T23:00:16.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.070+0000 7fd4867fc640 1 -- 172.21.15.32:0/1095531976 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4800078e0 con 0x7fd4880d4210 2024-04-24T23:00:16.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.070+0000 7fd4867fc640 1 -- 172.21.15.32:0/1095531976 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd480007cb0 con 0x7fd4880d4210 2024-04-24T23:00:16.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.070+0000 7fd48da9a640 1 -- 172.21.15.32:0/1095531976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4880d4210 msgr2=0x7fd4880b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:16.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.070+0000 7fd48da9a640 1 --2- 172.21.15.32:0/1095531976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4880d4210 0x7fd4880b78f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd4880730f0 tx=0x7fd48002a5f0 comp rx=0 tx=0).stop 2024-04-24T23:00:16.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.070+0000 7fd48da9a640 1 -- 172.21.15.32:0/1095531976 shutdown_connections 2024-04-24T23:00:16.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.070+0000 7fd48da9a640 1 --2- 172.21.15.32:0/1095531976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4880d4210 0x7fd4880b78f0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:16.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.070+0000 7fd48da9a640 1 --2- 172.21.15.32:0/1095531976 >> v2:172.21.15.121:3300/0 conn(0x7fd48806b820 0x7fd4880d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:16.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.070+0000 7fd48da9a640 1 -- 172.21.15.32:0/1095531976 >> 172.21.15.32:0/1095531976 conn(0x7fd4880d11d0 msgr2=0x7fd4880d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:16.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.070+0000 7fd48da9a640 1 -- 172.21.15.32:0/1095531976 shutdown_connections 2024-04-24T23:00:16.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.070+0000 7fd48da9a640 1 -- 172.21.15.32:0/1095531976 wait complete. 2024-04-24T23:00:16.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.071+0000 7fd48da9a640 1 Processor -- start 2024-04-24T23:00:16.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.071+0000 7fd48da9a640 1 -- start start 2024-04-24T23:00:16.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.072+0000 7fd48da9a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd48806b820 0x7fd4881f7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:16.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.072+0000 7fd48da9a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4880d4210 0x7fd48815ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:16.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.072+0000 7fd48da9a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd48815e9b0 0x7fd48815b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:16.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.072+0000 7fd48da9a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd4880ba0e0 con 0x7fd48815e9b0 2024-04-24T23:00:16.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.072+0000 7fd48da9a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd4880d2400 con 0x7fd4880d4210 2024-04-24T23:00:16.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.072+0000 7fd48da9a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd4880ba260 con 0x7fd48806b820 2024-04-24T23:00:16.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.072+0000 7fd487fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd48815e9b0 0x7fd48815b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:16.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.072+0000 7fd487fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd48815e9b0 0x7fd48815b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41670/0 (socket says 172.21.15.32:41670) 2024-04-24T23:00:16.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.072+0000 7fd487fff640 1 -- 172.21.15.32:0/4080204993 learned_addr learned my addr 172.21.15.32:0/4080204993 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:16.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.073+0000 7fd486ffd640 1 --2- 172.21.15.32:0/4080204993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4880d4210 0x7fd48815ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:16.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.073+0000 7fd4877fe640 1 --2- 172.21.15.32:0/4080204993 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd48806b820 0x7fd4881f7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:16.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.073+0000 7fd487fff640 1 -- 172.21.15.32:0/4080204993 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd48806b820 msgr2=0x7fd4881f7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:16.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.073+0000 7fd487fff640 1 --2- 172.21.15.32:0/4080204993 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd48806b820 0x7fd4881f7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:16.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.073+0000 7fd487fff640 1 -- 172.21.15.32:0/4080204993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4880d4210 msgr2=0x7fd48815ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:16.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.073+0000 7fd487fff640 1 --2- 172.21.15.32:0/4080204993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4880d4210 0x7fd48815ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:16.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.073+0000 7fd487fff640 1 -- 172.21.15.32:0/4080204993 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4881e78d0 con 0x7fd48815e9b0 2024-04-24T23:00:16.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.073+0000 7fd487fff640 1 --2- 172.21.15.32:0/4080204993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd48815e9b0 0x7fd48815b290 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd480009e80 tx=0x7fd480004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:16.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.073+0000 7fd46ffff640 1 -- 172.21.15.32:0/4080204993 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd480040070 con 0x7fd48815e9b0 2024-04-24T23:00:16.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.073+0000 7fd46ffff640 1 -- 172.21.15.32:0/4080204993 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd480031c40 con 0x7fd48815e9b0 2024-04-24T23:00:16.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.073+0000 7fd48da9a640 1 -- 172.21.15.32:0/4080204993 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4881e7bb0 con 0x7fd48815e9b0 2024-04-24T23:00:16.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.073+0000 7fd46ffff640 1 -- 172.21.15.32:0/4080204993 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd480009450 con 0x7fd48815e9b0 2024-04-24T23:00:16.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.073+0000 7fd48da9a640 1 -- 172.21.15.32:0/4080204993 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4881e8170 con 0x7fd48815e9b0 2024-04-24T23:00:16.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.074+0000 7fd48da9a640 1 -- 172.21.15.32:0/4080204993 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd488068430 con 0x7fd48815e9b0 2024-04-24T23:00:16.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.074+0000 7fd46ffff640 1 -- 172.21.15.32:0/4080204993 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd480039040 con 0x7fd48815e9b0 2024-04-24T23:00:16.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.075+0000 7fd46ffff640 1 --2- 172.21.15.32:0/4080204993 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd47406be60 0x7fd47406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:16.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.075+0000 7fd46ffff640 1 -- 172.21.15.32:0/4080204993 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fd4800ac050 con 0x7fd48815e9b0 2024-04-24T23:00:16.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.075+0000 7fd4877fe640 1 --2- 172.21.15.32:0/4080204993 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd47406be60 0x7fd47406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:16.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.076+0000 7fd4877fe640 1 --2- 172.21.15.32:0/4080204993 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd47406be60 0x7fd47406e320 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd478000f60 tx=0x7fd478009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:16.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.079+0000 7fd46ffff640 1 -- 172.21.15.32:0/4080204993 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd48007aa90 con 0x7fd48815e9b0 2024-04-24T23:00:16.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.261+0000 7fd48da9a640 1 -- 172.21.15.32:0/4080204993 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd48806b5e0 con 0x7fd48815e9b0 2024-04-24T23:00:16.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.261+0000 7fd46ffff640 1 -- 172.21.15.32:0/4080204993 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) ==== 93+0+10 (secure 0 0 0) 0x7fd48007a390 con 0x7fd48815e9b0 2024-04-24T23:00:16.263 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T23:00:16.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.263+0000 7fd48da9a640 1 -- 172.21.15.32:0/4080204993 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd47406be60 msgr2=0x7fd47406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:16.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.263+0000 7fd48da9a640 1 --2- 172.21.15.32:0/4080204993 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd47406be60 0x7fd47406e320 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd478000f60 tx=0x7fd478009040 comp rx=0 tx=0).stop 2024-04-24T23:00:16.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.263+0000 7fd48da9a640 1 -- 172.21.15.32:0/4080204993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd48815e9b0 msgr2=0x7fd48815b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:16.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.263+0000 7fd48da9a640 1 --2- 172.21.15.32:0/4080204993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd48815e9b0 0x7fd48815b290 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd480009e80 tx=0x7fd480004a40 comp rx=0 tx=0).stop 2024-04-24T23:00:16.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.263+0000 7fd48da9a640 1 -- 172.21.15.32:0/4080204993 shutdown_connections 2024-04-24T23:00:16.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.263+0000 7fd48da9a640 1 --2- 172.21.15.32:0/4080204993 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd47406be60 0x7fd47406e320 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:16.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.263+0000 7fd48da9a640 1 --2- 172.21.15.32:0/4080204993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd48815e9b0 0x7fd48815b290 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:16.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.263+0000 7fd48da9a640 1 --2- 172.21.15.32:0/4080204993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4880d4210 0x7fd48815ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:16.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.263+0000 7fd48da9a640 1 --2- 172.21.15.32:0/4080204993 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd48806b820 0x7fd4881f7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:16.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.263+0000 7fd48da9a640 1 -- 172.21.15.32:0/4080204993 >> 172.21.15.32:0/4080204993 conn(0x7fd4880d11d0 msgr2=0x7fd4880b78f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:16.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.263+0000 7fd48da9a640 1 -- 172.21.15.32:0/4080204993 shutdown_connections 2024-04-24T23:00:16.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.263+0000 7fd48da9a640 1 -- 172.21.15.32:0/4080204993 wait complete. 2024-04-24T23:00:16.275 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:00:16.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.377+0000 7fe31f361640 1 Processor -- start 2024-04-24T23:00:16.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.377+0000 7fe31f361640 1 -- start start 2024-04-24T23:00:16.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.378+0000 7fe31f361640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe318074210 0x7fe31806c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:16.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.378+0000 7fe31f361640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3180b0770 con 0x7fe3180ae140 2024-04-24T23:00:16.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.378+0000 7fe31f361640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3180b1010 con 0x7fe3180b0b80 2024-04-24T23:00:16.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.378+0000 7fe31f361640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe3180b1c20 con 0x7fe318074210 2024-04-24T23:00:16.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.378+0000 7fe31d8fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe3180ae140 0x7fe3180b20b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58128/0 (socket says 172.21.15.32:58128) 2024-04-24T23:00:16.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.378+0000 7fe31d8fe640 1 -- 172.21.15.32:0/1760728413 learned_addr learned my addr 172.21.15.32:0/1760728413 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:00:16.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.378+0000 7fe317fff640 1 -- 172.21.15.32:0/1760728413 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4229099835 0 0) 0x7fe3180b0770 con 0x7fe3180ae140 2024-04-24T23:00:16.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.378+0000 7fe31d0fd640 1 --2- 172.21.15.32:0/1760728413 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe318074210 0x7fe31806c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:16.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.378+0000 7fe317fff640 1 -- 172.21.15.32:0/1760728413 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe300003520 con 0x7fe3180ae140 2024-04-24T23:00:16.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.379+0000 7fe317fff640 1 -- 172.21.15.32:0/1760728413 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1424735326 0 0) 0x7fe300003520 con 0x7fe3180ae140 2024-04-24T23:00:16.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.379+0000 7fe317fff640 1 -- 172.21.15.32:0/1760728413 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe318074210 msgr2=0x7fe31806c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:16.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.379+0000 7fe317fff640 1 --2- 172.21.15.32:0/1760728413 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe318074210 0x7fe31806c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:16.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.379+0000 7fe317fff640 1 -- 172.21.15.32:0/1760728413 >> v1:172.21.15.121:6789/0 conn(0x7fe3180b0b80 legacy=0x7fe3180745f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:16.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.379+0000 7fe317fff640 1 -- 172.21.15.32:0/1760728413 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3180b12f0 con 0x7fe3180ae140 2024-04-24T23:00:16.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.379+0000 7fe317fff640 1 -- 172.21.15.32:0/1760728413 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7fe3080019f0 con 0x7fe3180ae140 2024-04-24T23:00:16.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.379+0000 7fe317fff640 1 -- 172.21.15.32:0/1760728413 >> v1:172.21.15.32:6789/0 conn(0x7fe3180ae140 legacy=0x7fe3180b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:16.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.379+0000 7fe317fff640 1 --2- 172.21.15.32:0/1760728413 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe318074210 0x7fe3000039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:16.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.379+0000 7fe317fff640 1 --2- 172.21.15.32:0/1760728413 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe300003f20 0x7fe300004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:16.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.379+0000 7fe317fff640 1 --2- 172.21.15.32:0/1760728413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe300004d40 0x7fe3000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:16.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.379+0000 7fe317fff640 1 -- 172.21.15.32:0/1760728413 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe3180b1c20 con 0x7fe300004d40 2024-04-24T23:00:16.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.379+0000 7fe317fff640 1 -- 172.21.15.32:0/1760728413 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe30000ba20 con 0x7fe300003f20 2024-04-24T23:00:16.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.379+0000 7fe317fff640 1 -- 172.21.15.32:0/1760728413 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe30000bc20 con 0x7fe318074210 2024-04-24T23:00:16.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.379+0000 7fe31d8fe640 1 --2- 172.21.15.32:0/1760728413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe300004d40 0x7fe3000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:16.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.379+0000 7fe31d0fd640 1 --2- 172.21.15.32:0/1760728413 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe318074210 0x7fe3000039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:16.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.379+0000 7fe31c8fc640 1 --2- 172.21.15.32:0/1760728413 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe300003f20 0x7fe300004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:16.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.379+0000 7fe31d8fe640 1 -- 172.21.15.32:0/1760728413 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe318074210 msgr2=0x7fe3000039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:16.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.379+0000 7fe31d8fe640 1 --2- 172.21.15.32:0/1760728413 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe318074210 0x7fe3000039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:16.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.379+0000 7fe31d8fe640 1 -- 172.21.15.32:0/1760728413 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe300003f20 msgr2=0x7fe300004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:16.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.379+0000 7fe31d8fe640 1 --2- 172.21.15.32:0/1760728413 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe300003f20 0x7fe300004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:16.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.379+0000 7fe31d8fe640 1 -- 172.21.15.32:0/1760728413 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe300003520 con 0x7fe300004d40 2024-04-24T23:00:16.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.380+0000 7fe31d8fe640 1 --2- 172.21.15.32:0/1760728413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe300004d40 0x7fe3000051c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe30800a880 tx=0x7fe300005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=808a69d3fc2ca46 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:16.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.380+0000 7fe317fff640 1 -- 172.21.15.32:0/1760728413 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe30800a4a0 con 0x7fe300004d40 2024-04-24T23:00:16.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.380+0000 7fe317fff640 1 -- 172.21.15.32:0/1760728413 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe30802f070 con 0x7fe300004d40 2024-04-24T23:00:16.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.380+0000 7fe317fff640 1 -- 172.21.15.32:0/1760728413 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe30802bd20 con 0x7fe300004d40 2024-04-24T23:00:16.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.381+0000 7fe31f361640 1 -- 172.21.15.32:0/1760728413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe300004d40 msgr2=0x7fe3000051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:16.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.381+0000 7fe31f361640 1 --2- 172.21.15.32:0/1760728413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe300004d40 0x7fe3000051c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe30800a880 tx=0x7fe300005830 comp rx=0 tx=0).stop 2024-04-24T23:00:16.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.381+0000 7fe31d8fe640 1 -- 172.21.15.32:0/1760728413 reap_dead start 2024-04-24T23:00:16.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.381+0000 7fe31f361640 1 -- 172.21.15.32:0/1760728413 shutdown_connections 2024-04-24T23:00:16.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.381+0000 7fe31f361640 1 -- 172.21.15.32:0/1760728413 >> 172.21.15.32:0/1760728413 conn(0x7fe3180b2e60 msgr2=0x7fe3180b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:16.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.381+0000 7fe31f361640 1 -- 172.21.15.32:0/1760728413 shutdown_connections 2024-04-24T23:00:16.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.381+0000 7fe31f361640 1 -- 172.21.15.32:0/1760728413 wait complete. 2024-04-24T23:00:16.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.382+0000 7fe31f361640 1 Processor -- start 2024-04-24T23:00:16.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.382+0000 7fe31f361640 1 -- start start 2024-04-24T23:00:16.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.383+0000 7fe31f361640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe300004d40 0x7fe3180c6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:16.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.383+0000 7fe31f361640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe31815e2c0 0x7fe3180c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:16.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.383+0000 7fe31f361640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3180c7400 0x7fe3180c7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:16.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.383+0000 7fe31f361640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe3180b3ba0 con 0x7fe31815e2c0 2024-04-24T23:00:16.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.383+0000 7fe31f361640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe3180b18b0 con 0x7fe3180c7400 2024-04-24T23:00:16.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.383+0000 7fe31f361640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe3180b3d20 con 0x7fe300004d40 2024-04-24T23:00:16.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.383+0000 7fe31c8fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe31815e2c0 0x7fe3180c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:16.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.383+0000 7fe31c8fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe31815e2c0 0x7fe3180c9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41684/0 (socket says 172.21.15.32:41684) 2024-04-24T23:00:16.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.383+0000 7fe31c8fc640 1 -- 172.21.15.32:0/626214590 learned_addr learned my addr 172.21.15.32:0/626214590 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:16.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.383+0000 7fe31d8fe640 1 --2- 172.21.15.32:0/626214590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3180c7400 0x7fe3180c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:16.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.383+0000 7fe31d0fd640 1 --2- 172.21.15.32:0/626214590 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe300004d40 0x7fe3180c6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:16.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.383+0000 7fe31c8fc640 1 -- 172.21.15.32:0/626214590 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe300004d40 msgr2=0x7fe3180c6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:16.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.383+0000 7fe31c8fc640 1 --2- 172.21.15.32:0/626214590 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe300004d40 0x7fe3180c6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:16.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.383+0000 7fe31c8fc640 1 -- 172.21.15.32:0/626214590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3180c7400 msgr2=0x7fe3180c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:16.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.383+0000 7fe31c8fc640 1 --2- 172.21.15.32:0/626214590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3180c7400 0x7fe3180c7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:16.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.383+0000 7fe31c8fc640 1 -- 172.21.15.32:0/626214590 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3180ca5c0 con 0x7fe31815e2c0 2024-04-24T23:00:16.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.383+0000 7fe31d8fe640 1 --2- 172.21.15.32:0/626214590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3180c7400 0x7fe3180c7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:00:16.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.383+0000 7fe31c8fc640 1 --2- 172.21.15.32:0/626214590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe31815e2c0 0x7fe3180c9e20 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fe30c00f7c0 tx=0x7fe30c00fc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:16.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.383+0000 7fe315ffb640 1 -- 172.21.15.32:0/626214590 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe30c0052e0 con 0x7fe31815e2c0 2024-04-24T23:00:16.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.383+0000 7fe315ffb640 1 -- 172.21.15.32:0/626214590 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe30c018070 con 0x7fe31815e2c0 2024-04-24T23:00:16.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.383+0000 7fe31f361640 1 -- 172.21.15.32:0/626214590 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3180ca8a0 con 0x7fe31815e2c0 2024-04-24T23:00:16.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.383+0000 7fe315ffb640 1 -- 172.21.15.32:0/626214590 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe30c00d910 con 0x7fe31815e2c0 2024-04-24T23:00:16.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.384+0000 7fe31f361640 1 -- 172.21.15.32:0/626214590 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3180cae60 con 0x7fe31815e2c0 2024-04-24T23:00:16.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.385+0000 7fe31f361640 1 -- 172.21.15.32:0/626214590 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3180b56b0 con 0x7fe31815e2c0 2024-04-24T23:00:16.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.385+0000 7fe315ffb640 1 -- 172.21.15.32:0/626214590 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe30c00dab0 con 0x7fe31815e2c0 2024-04-24T23:00:16.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.385+0000 7fe315ffb640 1 --2- 172.21.15.32:0/626214590 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe30406be60 0x7fe30406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:16.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.385+0000 7fe315ffb640 1 -- 172.21.15.32:0/626214590 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fe30c08ad60 con 0x7fe31815e2c0 2024-04-24T23:00:16.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.386+0000 7fe31d0fd640 1 --2- 172.21.15.32:0/626214590 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe30406be60 0x7fe30406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:16.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.386+0000 7fe31d0fd640 1 --2- 172.21.15.32:0/626214590 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe30406be60 0x7fe30406e320 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe310009ae0 tx=0x7fe310008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:16.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.390+0000 7fe315ffb640 1 -- 172.21.15.32:0/626214590 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe3180b56b0 con 0x7fe31815e2c0 2024-04-24T23:00:16.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.572+0000 7fe31f361640 1 -- 172.21.15.32:0/626214590 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe318068430 con 0x7fe31815e2c0 2024-04-24T23:00:16.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.573+0000 7fe315ffb640 1 -- 172.21.15.32:0/626214590 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) ==== 104+0+11 (secure 0 0 0) 0x7fe30c01a310 con 0x7fe31815e2c0 2024-04-24T23:00:16.574 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-24T23:00:16.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.574+0000 7fe31f361640 1 -- 172.21.15.32:0/626214590 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe30406be60 msgr2=0x7fe30406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:16.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.574+0000 7fe31f361640 1 --2- 172.21.15.32:0/626214590 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe30406be60 0x7fe30406e320 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe310009ae0 tx=0x7fe310008040 comp rx=0 tx=0).stop 2024-04-24T23:00:16.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.574+0000 7fe31f361640 1 -- 172.21.15.32:0/626214590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe31815e2c0 msgr2=0x7fe3180c9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:16.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.574+0000 7fe31f361640 1 --2- 172.21.15.32:0/626214590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe31815e2c0 0x7fe3180c9e20 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fe30c00f7c0 tx=0x7fe30c00fc80 comp rx=0 tx=0).stop 2024-04-24T23:00:16.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.575+0000 7fe31f361640 1 -- 172.21.15.32:0/626214590 shutdown_connections 2024-04-24T23:00:16.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.575+0000 7fe31f361640 1 --2- 172.21.15.32:0/626214590 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe30406be60 0x7fe30406e320 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:16.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.575+0000 7fe31f361640 1 --2- 172.21.15.32:0/626214590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe31815e2c0 0x7fe3180c9e20 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:16.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.575+0000 7fe31f361640 1 --2- 172.21.15.32:0/626214590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3180c7400 0x7fe3180c7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:16.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.575+0000 7fe31f361640 1 --2- 172.21.15.32:0/626214590 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe300004d40 0x7fe3180c6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:16.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.575+0000 7fe31f361640 1 -- 172.21.15.32:0/626214590 >> 172.21.15.32:0/626214590 conn(0x7fe3180b2e60 msgr2=0x7fe3180b1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:16.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.575+0000 7fe31f361640 1 -- 172.21.15.32:0/626214590 shutdown_connections 2024-04-24T23:00:16.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.575+0000 7fe31f361640 1 -- 172.21.15.32:0/626214590 wait complete. 2024-04-24T23:00:16.587 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:00:16.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.689+0000 7fef7d729640 1 Processor -- start 2024-04-24T23:00:16.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.689+0000 7fef7d729640 1 -- start start 2024-04-24T23:00:16.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.690+0000 7fef7d729640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef7815edd0 0x7fef780b5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:16.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.690+0000 7fef7d729640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef780cd910 con 0x7fef780b5560 2024-04-24T23:00:16.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.690+0000 7fef7d729640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef780cdbf0 con 0x7fef780cd3c0 2024-04-24T23:00:16.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.690+0000 7fef7d729640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fef780cded0 con 0x7fef7815edd0 2024-04-24T23:00:16.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.690+0000 7fef76ffd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fef780b5560 0x7fef7815e6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58138/0 (socket says 172.21.15.32:58138) 2024-04-24T23:00:16.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.690+0000 7fef76ffd640 1 -- 172.21.15.32:0/2044608938 learned_addr learned my addr 172.21.15.32:0/2044608938 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:00:16.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.690+0000 7fef767fc640 1 --2- 172.21.15.32:0/2044608938 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef7815edd0 0x7fef780b5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:16.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.690+0000 7fef75ffb640 1 -- 172.21.15.32:0/2044608938 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 559462362 0 0) 0x7fef780cd910 con 0x7fef780b5560 2024-04-24T23:00:16.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.690+0000 7fef75ffb640 1 -- 172.21.15.32:0/2044608938 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fef60003520 con 0x7fef780b5560 2024-04-24T23:00:16.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.690+0000 7fef75ffb640 1 -- 172.21.15.32:0/2044608938 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4207724029 0 0) 0x7fef780cdbf0 con 0x7fef780cd3c0 2024-04-24T23:00:16.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.690+0000 7fef75ffb640 1 -- 172.21.15.32:0/2044608938 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fef780cd910 con 0x7fef780cd3c0 2024-04-24T23:00:16.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.690+0000 7fef767fc640 1 -- 172.21.15.32:0/2044608938 >> v1:172.21.15.121:6789/0 conn(0x7fef780cd3c0 legacy=0x7fef780b5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:16.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.690+0000 7fef75ffb640 1 -- 172.21.15.32:0/2044608938 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3186921669 0 0) 0x7fef60003520 con 0x7fef780b5560 2024-04-24T23:00:16.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.690+0000 7fef767fc640 1 -- 172.21.15.32:0/2044608938 >> v1:172.21.15.32:6789/0 conn(0x7fef780b5560 legacy=0x7fef7815e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:16.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.690+0000 7fef767fc640 1 -- 172.21.15.32:0/2044608938 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef780ce1b0 con 0x7fef7815edd0 2024-04-24T23:00:16.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.691+0000 7fef767fc640 1 --2- 172.21.15.32:0/2044608938 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef7815edd0 0x7fef780b5940 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fef6c00a970 tx=0x7fef6c02b550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5231831f986a6083 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:16.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.691+0000 7fef75ffb640 1 -- 172.21.15.32:0/2044608938 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef70001670 con 0x7fef7815edd0 2024-04-24T23:00:16.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.691+0000 7fef75ffb640 1 -- 172.21.15.32:0/2044608938 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef6c030510 con 0x7fef7815edd0 2024-04-24T23:00:16.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.691+0000 7fef75ffb640 1 -- 172.21.15.32:0/2044608938 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6c00b040 con 0x7fef7815edd0 2024-04-24T23:00:16.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.692+0000 7fef7d729640 1 -- 172.21.15.32:0/2044608938 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef7815edd0 msgr2=0x7fef780b5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:16.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.692+0000 7fef7d729640 1 --2- 172.21.15.32:0/2044608938 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef7815edd0 0x7fef780b5940 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fef6c00a970 tx=0x7fef6c02b550 comp rx=0 tx=0).stop 2024-04-24T23:00:16.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.692+0000 7fef7d729640 1 -- 172.21.15.32:0/2044608938 shutdown_connections 2024-04-24T23:00:16.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.692+0000 7fef7d729640 1 --2- 172.21.15.32:0/2044608938 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef7815edd0 0x7fef780b5940 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:16.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.692+0000 7fef7d729640 1 -- 172.21.15.32:0/2044608938 >> 172.21.15.32:0/2044608938 conn(0x7fef7806b110 msgr2=0x7fef7806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:16.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.692+0000 7fef7d729640 1 -- 172.21.15.32:0/2044608938 shutdown_connections 2024-04-24T23:00:16.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.692+0000 7fef7d729640 1 -- 172.21.15.32:0/2044608938 wait complete. 2024-04-24T23:00:16.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.693+0000 7fef7d729640 1 Processor -- start 2024-04-24T23:00:16.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.693+0000 7fef7d729640 1 -- start start 2024-04-24T23:00:16.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.694+0000 7fef7d729640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef780b5560 0x7fef7814a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:16.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.694+0000 7fef7d729640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef780cd3c0 0x7fef7815e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:16.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.694+0000 7fef7d729640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef7815edd0 0x7fef7815e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:16.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.694+0000 7fef7d729640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fef780b44a0 con 0x7fef7815edd0 2024-04-24T23:00:16.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.694+0000 7fef7d729640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fef780b4320 con 0x7fef780cd3c0 2024-04-24T23:00:16.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.694+0000 7fef7d729640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fef7806c5f0 con 0x7fef780b5560 2024-04-24T23:00:16.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.694+0000 7fef777fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef7815edd0 0x7fef7815e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:16.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.694+0000 7fef777fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef7815edd0 0x7fef7815e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41698/0 (socket says 172.21.15.32:41698) 2024-04-24T23:00:16.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.694+0000 7fef777fe640 1 -- 172.21.15.32:0/3922814367 learned_addr learned my addr 172.21.15.32:0/3922814367 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:16.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.694+0000 7fef767fc640 1 --2- 172.21.15.32:0/3922814367 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef780cd3c0 0x7fef7815e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:16.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.694+0000 7fef76ffd640 1 --2- 172.21.15.32:0/3922814367 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef780b5560 0x7fef7814a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:16.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.695+0000 7fef777fe640 1 -- 172.21.15.32:0/3922814367 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef780b5560 msgr2=0x7fef7814a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:16.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.695+0000 7fef777fe640 1 --2- 172.21.15.32:0/3922814367 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef780b5560 0x7fef7814a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:16.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.695+0000 7fef777fe640 1 -- 172.21.15.32:0/3922814367 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef780cd3c0 msgr2=0x7fef7815e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:16.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.695+0000 7fef777fe640 1 --2- 172.21.15.32:0/3922814367 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef780cd3c0 0x7fef7815e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:16.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.695+0000 7fef777fe640 1 -- 172.21.15.32:0/3922814367 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef7815ce90 con 0x7fef7815edd0 2024-04-24T23:00:16.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.695+0000 7fef76ffd640 1 --2- 172.21.15.32:0/3922814367 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef780b5560 0x7fef7814a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:00:16.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.695+0000 7fef777fe640 1 --2- 172.21.15.32:0/3922814367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef7815edd0 0x7fef7815e7c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fef6800c7c0 tx=0x7fef6800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:16.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.695+0000 7fef5f7fe640 1 -- 172.21.15.32:0/3922814367 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef68003b70 con 0x7fef7815edd0 2024-04-24T23:00:16.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.695+0000 7fef5f7fe640 1 -- 172.21.15.32:0/3922814367 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef68003d10 con 0x7fef7815edd0 2024-04-24T23:00:16.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.695+0000 7fef7d729640 1 -- 172.21.15.32:0/3922814367 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef7815d170 con 0x7fef7815edd0 2024-04-24T23:00:16.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.695+0000 7fef5f7fe640 1 -- 172.21.15.32:0/3922814367 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6800aa50 con 0x7fef7815edd0 2024-04-24T23:00:16.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.695+0000 7fef7d729640 1 -- 172.21.15.32:0/3922814367 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef7815d730 con 0x7fef7815edd0 2024-04-24T23:00:16.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.696+0000 7fef7d729640 1 -- 172.21.15.32:0/3922814367 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef78068430 con 0x7fef7815edd0 2024-04-24T23:00:16.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.697+0000 7fef5f7fe640 1 -- 172.21.15.32:0/3922814367 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fef6800abf0 con 0x7fef7815edd0 2024-04-24T23:00:16.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.697+0000 7fef5f7fe640 1 --2- 172.21.15.32:0/3922814367 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fef6406be30 0x7fef6406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:16.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.697+0000 7fef5f7fe640 1 -- 172.21.15.32:0/3922814367 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fef6808ad80 con 0x7fef7815edd0 2024-04-24T23:00:16.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.698+0000 7fef76ffd640 1 --2- 172.21.15.32:0/3922814367 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fef6406be30 0x7fef6406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:16.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.698+0000 7fef76ffd640 1 --2- 172.21.15.32:0/3922814367 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fef6406be30 0x7fef6406e2f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fef700017e0 tx=0x7fef70009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:16.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.702+0000 7fef5f7fe640 1 -- 172.21.15.32:0/3922814367 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fef680597c0 con 0x7fef7815edd0 2024-04-24T23:00:16.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.882+0000 7fef7d729640 1 -- 172.21.15.32:0/3922814367 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fef780afd60 con 0x7fef7815edd0 2024-04-24T23:00:16.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.883+0000 7fef5f7fe640 1 -- 172.21.15.32:0/3922814367 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) ==== 100+0+11 (secure 0 0 0) 0x7fef680590c0 con 0x7fef7815edd0 2024-04-24T23:00:16.884 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-24T23:00:16.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.884+0000 7fef7d729640 1 -- 172.21.15.32:0/3922814367 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fef6406be30 msgr2=0x7fef6406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:16.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.884+0000 7fef7d729640 1 --2- 172.21.15.32:0/3922814367 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fef6406be30 0x7fef6406e2f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fef700017e0 tx=0x7fef70009040 comp rx=0 tx=0).stop 2024-04-24T23:00:16.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.884+0000 7fef7d729640 1 -- 172.21.15.32:0/3922814367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef7815edd0 msgr2=0x7fef7815e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:16.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.884+0000 7fef7d729640 1 --2- 172.21.15.32:0/3922814367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef7815edd0 0x7fef7815e7c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fef6800c7c0 tx=0x7fef6800cc80 comp rx=0 tx=0).stop 2024-04-24T23:00:16.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.884+0000 7fef7d729640 1 -- 172.21.15.32:0/3922814367 shutdown_connections 2024-04-24T23:00:16.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.884+0000 7fef7d729640 1 --2- 172.21.15.32:0/3922814367 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fef6406be30 0x7fef6406e2f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:16.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.884+0000 7fef7d729640 1 --2- 172.21.15.32:0/3922814367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef7815edd0 0x7fef7815e7c0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:16.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.884+0000 7fef7d729640 1 --2- 172.21.15.32:0/3922814367 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef780cd3c0 0x7fef7815e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:16.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.884+0000 7fef7d729640 1 --2- 172.21.15.32:0/3922814367 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef780b5560 0x7fef7814a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:16.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.884+0000 7fef7d729640 1 -- 172.21.15.32:0/3922814367 >> 172.21.15.32:0/3922814367 conn(0x7fef7806b110 msgr2=0x7fef7806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:16.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.885+0000 7fef7d729640 1 -- 172.21.15.32:0/3922814367 shutdown_connections 2024-04-24T23:00:16.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.885+0000 7fef7d729640 1 -- 172.21.15.32:0/3922814367 wait complete. 2024-04-24T23:00:16.897 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T23:00:16.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.997+0000 7f940e272640 1 Processor -- start 2024-04-24T23:00:16.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.997+0000 7f940e272640 1 -- start start 2024-04-24T23:00:17.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.998+0000 7f940e272640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f94080b44d0 0x7f94080b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:17.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.998+0000 7f940e272640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94080b6430 con 0x7f94080af890 2024-04-24T23:00:17.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.998+0000 7f940e272640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9408156f80 con 0x7f940806c5f0 2024-04-24T23:00:17.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.998+0000 7f940e272640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f94080b6150 con 0x7f94080b44d0 2024-04-24T23:00:17.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.998+0000 7f940c80f640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f94080af890 0x7f940815cac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58152/0 (socket says 172.21.15.32:58152) 2024-04-24T23:00:17.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.998+0000 7f940c80f640 1 -- 172.21.15.32:0/3494863274 learned_addr learned my addr 172.21.15.32:0/3494863274 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:00:17.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.998+0000 7f94077fe640 1 --2- 172.21.15.32:0/3494863274 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f94080b44d0 0x7f94080b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:17.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.998+0000 7f9406ffd640 1 -- 172.21.15.32:0/3494863274 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2806321160 0 0) 0x7f94080b6430 con 0x7f94080af890 2024-04-24T23:00:17.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.998+0000 7f9406ffd640 1 -- 172.21.15.32:0/3494863274 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f93f0003520 con 0x7f94080af890 2024-04-24T23:00:17.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.999+0000 7f9406ffd640 1 -- 172.21.15.32:0/3494863274 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3884292705 0 0) 0x7f93f0003520 con 0x7f94080af890 2024-04-24T23:00:17.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.999+0000 7f9406ffd640 1 -- 172.21.15.32:0/3494863274 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f94080b44d0 msgr2=0x7f94080b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:17.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.999+0000 7f9406ffd640 1 --2- 172.21.15.32:0/3494863274 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f94080b44d0 0x7f94080b5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:17.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.999+0000 7f9406ffd640 1 -- 172.21.15.32:0/3494863274 >> v1:172.21.15.121:6789/0 conn(0x7f940806c5f0 legacy=0x7f940806c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:17.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.999+0000 7f9406ffd640 1 -- 172.21.15.32:0/3494863274 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94080b39f0 con 0x7f94080af890 2024-04-24T23:00:17.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.999+0000 7f9406ffd640 1 -- 172.21.15.32:0/3494863274 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f94000019f0 con 0x7f94080af890 2024-04-24T23:00:17.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.999+0000 7f9406ffd640 1 -- 172.21.15.32:0/3494863274 >> v1:172.21.15.32:6789/0 conn(0x7f94080af890 legacy=0x7f940815cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:17.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.999+0000 7f9406ffd640 1 --2- 172.21.15.32:0/3494863274 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93f00039e0 0x7f93f0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:17.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.999+0000 7f9406ffd640 1 --2- 172.21.15.32:0/3494863274 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93f0004390 0x7f93f0004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:17.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.999+0000 7f9406ffd640 1 --2- 172.21.15.32:0/3494863274 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f94080b44d0 0x7f93f0005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:17.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.999+0000 7f9406ffd640 1 -- 172.21.15.32:0/3494863274 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f94080b6150 con 0x7f93f0004390 2024-04-24T23:00:17.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.999+0000 7f9406ffd640 1 -- 172.21.15.32:0/3494863274 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f93f000ba70 con 0x7f93f00039e0 2024-04-24T23:00:17.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.999+0000 7f9406ffd640 1 -- 172.21.15.32:0/3494863274 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f93f000bc70 con 0x7f94080b44d0 2024-04-24T23:00:17.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.999+0000 7f94077fe640 1 --2- 172.21.15.32:0/3494863274 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93f0004390 0x7f93f0004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:17.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.999+0000 7f940c80f640 1 --2- 172.21.15.32:0/3494863274 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f94080b44d0 0x7f93f0005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:17.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.999+0000 7f9407fff640 1 --2- 172.21.15.32:0/3494863274 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93f00039e0 0x7f93f0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:17.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.999+0000 7f94077fe640 1 -- 172.21.15.32:0/3494863274 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f94080b44d0 msgr2=0x7f93f0005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:17.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.999+0000 7f94077fe640 1 --2- 172.21.15.32:0/3494863274 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f94080b44d0 0x7f93f0005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:17.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.999+0000 7f94077fe640 1 -- 172.21.15.32:0/3494863274 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93f00039e0 msgr2=0x7f93f0003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:17.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.999+0000 7f94077fe640 1 --2- 172.21.15.32:0/3494863274 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93f00039e0 0x7f93f0003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:17.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.999+0000 7f94077fe640 1 -- 172.21.15.32:0/3494863274 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93f0003520 con 0x7f93f0004390 2024-04-24T23:00:17.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.999+0000 7f940c80f640 1 --2- 172.21.15.32:0/3494863274 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f94080b44d0 0x7f93f0005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:00:17.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:16.999+0000 7f9407fff640 1 --2- 172.21.15.32:0/3494863274 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93f00039e0 0x7f93f0003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:00:17.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.000+0000 7f94077fe640 1 --2- 172.21.15.32:0/3494863274 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93f0004390 0x7f93f0004860 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f93f8009ac0 tx=0x7f93f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3db3c71554d84f4e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:17.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.000+0000 7f9406ffd640 1 -- 172.21.15.32:0/3494863274 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f8036070 con 0x7f93f0004390 2024-04-24T23:00:17.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.000+0000 7f9406ffd640 1 -- 172.21.15.32:0/3494863274 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93f8031400 con 0x7f93f0004390 2024-04-24T23:00:17.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.000+0000 7f9406ffd640 1 -- 172.21.15.32:0/3494863274 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f8009690 con 0x7f93f0004390 2024-04-24T23:00:17.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.000+0000 7f940e272640 1 -- 172.21.15.32:0/3494863274 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93f0004390 msgr2=0x7f93f0004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:17.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.000+0000 7f940e272640 1 --2- 172.21.15.32:0/3494863274 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93f0004390 0x7f93f0004860 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f93f8009ac0 tx=0x7f93f0005830 comp rx=0 tx=0).stop 2024-04-24T23:00:17.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.000+0000 7f940c80f640 1 -- 172.21.15.32:0/3494863274 reap_dead start 2024-04-24T23:00:17.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.001+0000 7f940e272640 1 -- 172.21.15.32:0/3494863274 shutdown_connections 2024-04-24T23:00:17.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.001+0000 7f940e272640 1 -- 172.21.15.32:0/3494863274 >> 172.21.15.32:0/3494863274 conn(0x7f940806b110 msgr2=0x7f940806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:17.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.001+0000 7f940e272640 1 -- 172.21.15.32:0/3494863274 shutdown_connections 2024-04-24T23:00:17.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.001+0000 7f940e272640 1 -- 172.21.15.32:0/3494863274 wait complete. 2024-04-24T23:00:17.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.002+0000 7f940e272640 1 Processor -- start 2024-04-24T23:00:17.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.002+0000 7f940e272640 1 -- start start 2024-04-24T23:00:17.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.003+0000 7f940e272640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93f0004390 0x7f94081599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:17.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.003+0000 7f940e272640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9408157d80 0x7f9408158200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:17.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.003+0000 7f940e272640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9408158740 0x7f9408158bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:17.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.003+0000 7f940e272640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f94080b5890 con 0x7f93f0004390 2024-04-24T23:00:17.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.003+0000 7f940e272640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f94080b3fb0 con 0x7f9408158740 2024-04-24T23:00:17.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.003+0000 7f940e272640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f94080b5a10 con 0x7f9408157d80 2024-04-24T23:00:17.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.003+0000 7f9407fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93f0004390 0x7f94081599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:17.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.003+0000 7f9407fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93f0004390 0x7f94081599c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41716/0 (socket says 172.21.15.32:41716) 2024-04-24T23:00:17.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.003+0000 7f9407fff640 1 -- 172.21.15.32:0/1384540184 learned_addr learned my addr 172.21.15.32:0/1384540184 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:17.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.003+0000 7f94077fe640 1 --2- 172.21.15.32:0/1384540184 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9408157d80 0x7f9408158200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:17.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.003+0000 7f940c80f640 1 --2- 172.21.15.32:0/1384540184 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9408158740 0x7f9408158bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:17.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.003+0000 7f9407fff640 1 -- 172.21.15.32:0/1384540184 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9408157d80 msgr2=0x7f9408158200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:17.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.003+0000 7f9407fff640 1 --2- 172.21.15.32:0/1384540184 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9408157d80 0x7f9408158200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:17.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.003+0000 7f9407fff640 1 -- 172.21.15.32:0/1384540184 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9408158740 msgr2=0x7f9408158bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:17.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.003+0000 7f9407fff640 1 --2- 172.21.15.32:0/1384540184 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9408158740 0x7f9408158bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:17.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.003+0000 7f9407fff640 1 -- 172.21.15.32:0/1384540184 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f940815f1e0 con 0x7f93f0004390 2024-04-24T23:00:17.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.003+0000 7f9407fff640 1 --2- 172.21.15.32:0/1384540184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93f0004390 0x7f94081599c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f93fc00e760 tx=0x7f93fc00ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:17.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.003+0000 7f9404ff9640 1 -- 172.21.15.32:0/1384540184 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93fc00ee70 con 0x7f93f0004390 2024-04-24T23:00:17.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.003+0000 7f9404ff9640 1 -- 172.21.15.32:0/1384540184 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93fc0052e0 con 0x7f93f0004390 2024-04-24T23:00:17.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.003+0000 7f940e272640 1 -- 172.21.15.32:0/1384540184 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f940815f4c0 con 0x7f93f0004390 2024-04-24T23:00:17.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.004+0000 7f9404ff9640 1 -- 172.21.15.32:0/1384540184 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93fc007020 con 0x7f93f0004390 2024-04-24T23:00:17.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.004+0000 7f940e272640 1 -- 172.21.15.32:0/1384540184 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9408157580 con 0x7f93f0004390 2024-04-24T23:00:17.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.005+0000 7f940e272640 1 -- 172.21.15.32:0/1384540184 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9408068430 con 0x7f93f0004390 2024-04-24T23:00:17.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.005+0000 7f9404ff9640 1 -- 172.21.15.32:0/1384540184 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f93fc01a020 con 0x7f93f0004390 2024-04-24T23:00:17.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.005+0000 7f9404ff9640 1 --2- 172.21.15.32:0/1384540184 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f93f406be60 0x7f93f406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:17.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.005+0000 7f9404ff9640 1 -- 172.21.15.32:0/1384540184 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f93fc08a560 con 0x7f93f0004390 2024-04-24T23:00:17.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.006+0000 7f94077fe640 1 --2- 172.21.15.32:0/1384540184 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f93f406be60 0x7f93f406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:17.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.006+0000 7f94077fe640 1 --2- 172.21.15.32:0/1384540184 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f93f406be60 0x7f93f406e320 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f93f8033000 tx=0x7f93f8008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:17.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.010+0000 7f9404ff9640 1 -- 172.21.15.32:0/1384540184 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f93fc0588c0 con 0x7f93f0004390 2024-04-24T23:00:17.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.205+0000 7f940e272640 1 -- 172.21.15.32:0/1384540184 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f94080b5670 con 0x7f93f0004390 2024-04-24T23:00:17.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.205+0000 7f9404ff9640 1 -- 172.21.15.32:0/1384540184 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v34) ==== 80+0+0 (secure 0 0 0) 0x7f93fc05cbc0 con 0x7f93f0004390 2024-04-24T23:00:17.207 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2024-04-24T23:00:17.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.207+0000 7f940e272640 1 -- 172.21.15.32:0/1384540184 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f93f406be60 msgr2=0x7f93f406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:17.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.207+0000 7f940e272640 1 --2- 172.21.15.32:0/1384540184 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f93f406be60 0x7f93f406e320 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f93f8033000 tx=0x7f93f8008030 comp rx=0 tx=0).stop 2024-04-24T23:00:17.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.207+0000 7f940e272640 1 -- 172.21.15.32:0/1384540184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93f0004390 msgr2=0x7f94081599c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:17.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.207+0000 7f940e272640 1 --2- 172.21.15.32:0/1384540184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93f0004390 0x7f94081599c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f93fc00e760 tx=0x7f93fc00ec20 comp rx=0 tx=0).stop 2024-04-24T23:00:17.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.207+0000 7f940e272640 1 -- 172.21.15.32:0/1384540184 shutdown_connections 2024-04-24T23:00:17.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.207+0000 7f940e272640 1 --2- 172.21.15.32:0/1384540184 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f93f406be60 0x7f93f406e320 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:17.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.207+0000 7f940e272640 1 --2- 172.21.15.32:0/1384540184 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9408158740 0x7f9408158bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:17.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.207+0000 7f940e272640 1 --2- 172.21.15.32:0/1384540184 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9408157d80 0x7f9408158200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:17.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.207+0000 7f940e272640 1 --2- 172.21.15.32:0/1384540184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93f0004390 0x7f94081599c0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:17.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.207+0000 7f940e272640 1 -- 172.21.15.32:0/1384540184 >> 172.21.15.32:0/1384540184 conn(0x7f940806b110 msgr2=0x7f940806c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:17.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.208+0000 7f940e272640 1 -- 172.21.15.32:0/1384540184 shutdown_connections 2024-04-24T23:00:17.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.208+0000 7f940e272640 1 -- 172.21.15.32:0/1384540184 wait complete. 2024-04-24T23:00:17.220 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-24T23:00:17.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:00:17.220 DEBUG:teuthology.orchestra.run.smithi121:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-24T23:00:17.252 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2024-04-24T23:00:17.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:00:17.252 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-24T23:00:17.310 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:00:17.311 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:00:17.311 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:00:17.327 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=413 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T23:00:17.327 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:00:17.327 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:00:17.327 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:00:17.370 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3271 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T23:00:17.370 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:00:17.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.469+0000 7f822ce9e640 1 Processor -- start 2024-04-24T23:00:17.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.469+0000 7f822ce9e640 1 -- start start 2024-04-24T23:00:17.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.470+0000 7f822ce9e640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f822815e9b0 0x7f82280d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:17.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.470+0000 7f822ce9e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f822806b820 0x7f82280b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:17.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.470+0000 7f822ce9e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82280d3170 con 0x7f82280d4210 2024-04-24T23:00:17.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.470+0000 7f822ce9e640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f82280d3450 con 0x7f822806b820 2024-04-24T23:00:17.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.470+0000 7f822ce9e640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f82280d3730 con 0x7f822815e9b0 2024-04-24T23:00:17.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.470+0000 7f8226d9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f822806b820 0x7f82280b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:17.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.470+0000 7f8225d9b640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f82280d4210 0x7f822806b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58162/0 (socket says 172.21.15.32:58162) 2024-04-24T23:00:17.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.470+0000 7f8225d9b640 1 -- 172.21.15.32:0/2806761341 learned_addr learned my addr 172.21.15.32:0/2806761341 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:00:17.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.471+0000 7f822659c640 1 --2- 172.21.15.32:0/2806761341 >> v2:172.21.15.121:3300/0 conn(0x7f822815e9b0 0x7f82280d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:17.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.471+0000 7f8226d9d640 1 -- 172.21.15.32:0/2806761341 >> v2:172.21.15.121:3300/0 conn(0x7f822815e9b0 msgr2=0x7f82280d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:17.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.471+0000 7f8226d9d640 1 --2- 172.21.15.32:0/2806761341 >> v2:172.21.15.121:3300/0 conn(0x7f822815e9b0 0x7f82280d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:17.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.471+0000 7f8226d9d640 1 -- 172.21.15.32:0/2806761341 >> v1:172.21.15.32:6789/0 conn(0x7f82280d4210 legacy=0x7f822806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:17.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.471+0000 7f8226d9d640 1 -- 172.21.15.32:0/2806761341 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82280d3a10 con 0x7f822806b820 2024-04-24T23:00:17.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.471+0000 7f822559a640 1 -- 172.21.15.32:0/2806761341 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 464783175 0 0) 0x7f82280d3170 con 0x7f82280d4210 2024-04-24T23:00:17.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.471+0000 7f8226d9d640 1 --2- 172.21.15.32:0/2806761341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f822806b820 0x7f82280b78f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f822806be20 tx=0x7f821802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83ce00ff04111974 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:17.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.472+0000 7f822559a640 1 -- 172.21.15.32:0/2806761341 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8218007740 con 0x7f822806b820 2024-04-24T23:00:17.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.472+0000 7f822559a640 1 -- 172.21.15.32:0/2806761341 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82180078e0 con 0x7f822806b820 2024-04-24T23:00:17.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.472+0000 7f822559a640 1 -- 172.21.15.32:0/2806761341 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8218007cb0 con 0x7f822806b820 2024-04-24T23:00:17.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.472+0000 7f822ce9e640 1 -- 172.21.15.32:0/2806761341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f822806b820 msgr2=0x7f82280b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:17.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.472+0000 7f822ce9e640 1 --2- 172.21.15.32:0/2806761341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f822806b820 0x7f82280b78f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f822806be20 tx=0x7f821802a5f0 comp rx=0 tx=0).stop 2024-04-24T23:00:17.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.472+0000 7f822ce9e640 1 -- 172.21.15.32:0/2806761341 shutdown_connections 2024-04-24T23:00:17.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.472+0000 7f822ce9e640 1 --2- 172.21.15.32:0/2806761341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f822806b820 0x7f82280b78f0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:17.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.472+0000 7f822ce9e640 1 --2- 172.21.15.32:0/2806761341 >> v2:172.21.15.121:3300/0 conn(0x7f822815e9b0 0x7f82280d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:17.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.472+0000 7f822ce9e640 1 -- 172.21.15.32:0/2806761341 >> 172.21.15.32:0/2806761341 conn(0x7f82280d11d0 msgr2=0x7f82280d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:17.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.472+0000 7f822ce9e640 1 -- 172.21.15.32:0/2806761341 shutdown_connections 2024-04-24T23:00:17.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.472+0000 7f822ce9e640 1 -- 172.21.15.32:0/2806761341 wait complete. 2024-04-24T23:00:17.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.474+0000 7f822ce9e640 1 Processor -- start 2024-04-24T23:00:17.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.474+0000 7f822ce9e640 1 -- start start 2024-04-24T23:00:17.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.474+0000 7f822ce9e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f822806b820 0x7f822814dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:17.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.474+0000 7f822ce9e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82280d4210 0x7f82280cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:17.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.474+0000 7f822ce9e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f822815e9b0 0x7f82280ce530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:17.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.474+0000 7f822ce9e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f82280ba0e0 con 0x7f822815e9b0 2024-04-24T23:00:17.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.474+0000 7f822ce9e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f82280d2400 con 0x7f82280d4210 2024-04-24T23:00:17.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.474+0000 7f822ce9e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f82280ba260 con 0x7f822806b820 2024-04-24T23:00:17.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.474+0000 7f8226d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f822815e9b0 0x7f82280ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:17.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.474+0000 7f8226d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f822815e9b0 0x7f82280ce530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41736/0 (socket says 172.21.15.32:41736) 2024-04-24T23:00:17.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.474+0000 7f8226d9d640 1 -- 172.21.15.32:0/1845559989 learned_addr learned my addr 172.21.15.32:0/1845559989 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:17.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.475+0000 7f8225d9b640 1 --2- 172.21.15.32:0/1845559989 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82280d4210 0x7f82280cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:17.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.475+0000 7f822659c640 1 --2- 172.21.15.32:0/1845559989 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f822806b820 0x7f822814dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:17.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.475+0000 7f8226d9d640 1 -- 172.21.15.32:0/1845559989 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f822806b820 msgr2=0x7f822814dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:17.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.475+0000 7f8226d9d640 1 --2- 172.21.15.32:0/1845559989 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f822806b820 0x7f822814dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:17.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.475+0000 7f8226d9d640 1 -- 172.21.15.32:0/1845559989 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82280d4210 msgr2=0x7f82280cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:17.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.475+0000 7f8226d9d640 1 --2- 172.21.15.32:0/1845559989 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82280d4210 0x7f82280cdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:17.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.475+0000 7f8226d9d640 1 -- 172.21.15.32:0/1845559989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82280cb940 con 0x7f822815e9b0 2024-04-24T23:00:17.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.475+0000 7f8225d9b640 1 --2- 172.21.15.32:0/1845559989 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82280d4210 0x7f82280cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:00:17.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.475+0000 7f8226d9d640 1 --2- 172.21.15.32:0/1845559989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f822815e9b0 0x7f82280ce530 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f821802abe0 tx=0x7f8218004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:17.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.475+0000 7f821effd640 1 -- 172.21.15.32:0/1845559989 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8218040070 con 0x7f822815e9b0 2024-04-24T23:00:17.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.475+0000 7f821effd640 1 -- 172.21.15.32:0/1845559989 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8218031bc0 con 0x7f822815e9b0 2024-04-24T23:00:17.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.475+0000 7f822ce9e640 1 -- 172.21.15.32:0/1845559989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82280cbc20 con 0x7f822815e9b0 2024-04-24T23:00:17.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.475+0000 7f821effd640 1 -- 172.21.15.32:0/1845559989 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8218045460 con 0x7f822815e9b0 2024-04-24T23:00:17.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.475+0000 7f822ce9e640 1 -- 172.21.15.32:0/1845559989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8228158320 con 0x7f822815e9b0 2024-04-24T23:00:17.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.476+0000 7f822ce9e640 1 -- 172.21.15.32:0/1845559989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8228068430 con 0x7f822815e9b0 2024-04-24T23:00:17.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.477+0000 7f821effd640 1 -- 172.21.15.32:0/1845559989 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8218039040 con 0x7f822815e9b0 2024-04-24T23:00:17.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.477+0000 7f821effd640 1 --2- 172.21.15.32:0/1845559989 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f820806be30 0x7f820806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:17.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.477+0000 7f821effd640 1 -- 172.21.15.32:0/1845559989 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f82180ac800 con 0x7f822815e9b0 2024-04-24T23:00:17.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.477+0000 7f822659c640 1 --2- 172.21.15.32:0/1845559989 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f820806be30 0x7f820806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:17.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.478+0000 7f822659c640 1 --2- 172.21.15.32:0/1845559989 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f820806be30 0x7f820806e2f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8220006fd0 tx=0x7f8220008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:17.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.481+0000 7f821effd640 1 -- 172.21.15.32:0/1845559989 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f821807b1b0 con 0x7f822815e9b0 2024-04-24T23:00:17.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.701+0000 7f822ce9e640 1 -- 172.21.15.32:0/1845559989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8228074df0 con 0x7f822815e9b0 2024-04-24T23:00:17.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.701+0000 7f821effd640 1 -- 172.21.15.32:0/1845559989 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+5181 (secure 0 0 0) 0x7f821807aab0 con 0x7f822815e9b0 2024-04-24T23:00:17.703 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:00:17.703 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1796819396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":1796819396},{"type":"v1","addr":"172.21.15.32:6835","nonce":1796819396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/298202585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":298202585},{"type":"v1","addr":"172.21.15.121:6836","nonce":298202585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4939,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/4056636026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":4056636026},{"type":"v1","addr":"172.21.15.32:6834","nonce":4056636026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T22:59:57.468765+0000","modified":"2024-04-24T22:59:59.471269+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5112},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5112":{"gid":5112,"name":"b","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6837/515107694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":515107694},{"type":"v1","addr":"172.21.15.121:6837","nonce":515107694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5112,"qdb_cluster":[5112]},"id":2}]} 2024-04-24T23:00:17.703 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 13 2024-04-24T23:00:17.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.703+0000 7f822ce9e640 1 -- 172.21.15.32:0/1845559989 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f820806be30 msgr2=0x7f820806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:17.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.703+0000 7f822ce9e640 1 --2- 172.21.15.32:0/1845559989 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f820806be30 0x7f820806e2f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8220006fd0 tx=0x7f8220008040 comp rx=0 tx=0).stop 2024-04-24T23:00:17.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.703+0000 7f822ce9e640 1 -- 172.21.15.32:0/1845559989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f822815e9b0 msgr2=0x7f82280ce530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:17.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.703+0000 7f822ce9e640 1 --2- 172.21.15.32:0/1845559989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f822815e9b0 0x7f82280ce530 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f821802abe0 tx=0x7f8218004a40 comp rx=0 tx=0).stop 2024-04-24T23:00:17.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.703+0000 7f822ce9e640 1 -- 172.21.15.32:0/1845559989 shutdown_connections 2024-04-24T23:00:17.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.703+0000 7f822ce9e640 1 --2- 172.21.15.32:0/1845559989 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f820806be30 0x7f820806e2f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:17.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.703+0000 7f822ce9e640 1 --2- 172.21.15.32:0/1845559989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f822815e9b0 0x7f82280ce530 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:17.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.703+0000 7f822ce9e640 1 --2- 172.21.15.32:0/1845559989 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82280d4210 0x7f82280cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:17.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.703+0000 7f822ce9e640 1 --2- 172.21.15.32:0/1845559989 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f822806b820 0x7f822814dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:17.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.703+0000 7f822ce9e640 1 -- 172.21.15.32:0/1845559989 >> 172.21.15.32:0/1845559989 conn(0x7f82280d11d0 msgr2=0x7f8228072060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:17.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.703+0000 7f822ce9e640 1 -- 172.21.15.32:0/1845559989 shutdown_connections 2024-04-24T23:00:17.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.703+0000 7f822ce9e640 1 -- 172.21.15.32:0/1845559989 wait complete. 2024-04-24T23:00:17.716 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:00:17.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.814+0000 7f19944d4640 1 Processor -- start 2024-04-24T23:00:17.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.814+0000 7f19944d4640 1 -- start start 2024-04-24T23:00:17.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.815+0000 7f19944d4640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f198c15e9b0 0x7f198c0d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:17.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.815+0000 7f19944d4640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f198c06b820 0x7f198c0b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:17.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.815+0000 7f19944d4640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f198c0d3170 con 0x7f198c0d4210 2024-04-24T23:00:17.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.815+0000 7f19944d4640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f198c0d3450 con 0x7f198c15e9b0 2024-04-24T23:00:17.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.815+0000 7f19944d4640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f198c0d3730 con 0x7f198c06b820 2024-04-24T23:00:17.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.815+0000 7f1992270640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f198c15e9b0 0x7f198c0d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:17.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.815+0000 7f1991a6f640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f198c0d4210 0x7f198c06b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58172/0 (socket says 172.21.15.32:58172) 2024-04-24T23:00:17.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.815+0000 7f1991a6f640 1 -- 172.21.15.32:0/3911842122 learned_addr learned my addr 172.21.15.32:0/3911842122 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:00:17.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.816+0000 7f1992a71640 1 --2- 172.21.15.32:0/3911842122 >> v2:172.21.15.121:3300/0 conn(0x7f198c06b820 0x7f198c0b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:17.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.816+0000 7f1992270640 1 -- 172.21.15.32:0/3911842122 >> v2:172.21.15.121:3300/0 conn(0x7f198c06b820 msgr2=0x7f198c0b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:17.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.816+0000 7f1992270640 1 --2- 172.21.15.32:0/3911842122 >> v2:172.21.15.121:3300/0 conn(0x7f198c06b820 0x7f198c0b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:17.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.816+0000 7f1992270640 1 -- 172.21.15.32:0/3911842122 >> v1:172.21.15.32:6789/0 conn(0x7f198c0d4210 legacy=0x7f198c06b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:17.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.816+0000 7f1992270640 1 -- 172.21.15.32:0/3911842122 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f198c0d3a10 con 0x7f198c15e9b0 2024-04-24T23:00:17.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.816+0000 7f199126e640 1 -- 172.21.15.32:0/3911842122 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4222415323 0 0) 0x7f198c0d3170 con 0x7f198c0d4210 2024-04-24T23:00:17.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.817+0000 7f1992270640 1 --2- 172.21.15.32:0/3911842122 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f198c15e9b0 0x7f198c0d2b90 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f198c1622f0 tx=0x7f198402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd380129dacba36c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:17.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.817+0000 7f199126e640 1 -- 172.21.15.32:0/3911842122 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1984037070 con 0x7f198c15e9b0 2024-04-24T23:00:17.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.817+0000 7f199126e640 1 -- 172.21.15.32:0/3911842122 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1984032440 con 0x7f198c15e9b0 2024-04-24T23:00:17.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.817+0000 7f199126e640 1 -- 172.21.15.32:0/3911842122 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198402ae60 con 0x7f198c15e9b0 2024-04-24T23:00:17.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.817+0000 7f19944d4640 1 -- 172.21.15.32:0/3911842122 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f198c15e9b0 msgr2=0x7f198c0d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:17.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.817+0000 7f19944d4640 1 --2- 172.21.15.32:0/3911842122 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f198c15e9b0 0x7f198c0d2b90 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f198c1622f0 tx=0x7f198402a110 comp rx=0 tx=0).stop 2024-04-24T23:00:17.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.817+0000 7f19944d4640 1 -- 172.21.15.32:0/3911842122 shutdown_connections 2024-04-24T23:00:17.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.817+0000 7f19944d4640 1 --2- 172.21.15.32:0/3911842122 >> v2:172.21.15.121:3300/0 conn(0x7f198c06b820 0x7f198c0b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:17.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.817+0000 7f19944d4640 1 --2- 172.21.15.32:0/3911842122 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f198c15e9b0 0x7f198c0d2b90 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:17.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.817+0000 7f19944d4640 1 -- 172.21.15.32:0/3911842122 >> 172.21.15.32:0/3911842122 conn(0x7f198c0d11d0 msgr2=0x7f198c0d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:17.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.817+0000 7f19944d4640 1 -- 172.21.15.32:0/3911842122 shutdown_connections 2024-04-24T23:00:17.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.817+0000 7f19944d4640 1 -- 172.21.15.32:0/3911842122 wait complete. 2024-04-24T23:00:17.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.818+0000 7f19944d4640 1 Processor -- start 2024-04-24T23:00:17.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.819+0000 7f19944d4640 1 -- start start 2024-04-24T23:00:17.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.819+0000 7f19944d4640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f198c06b820 0x7f198c159430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:17.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.819+0000 7f19944d4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f198c0d4210 0x7f198c152a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:17.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.819+0000 7f19944d4640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f198c15e9b0 0x7f198c15b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:17.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.819+0000 7f19944d4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f198c0ba0e0 con 0x7f198c0d4210 2024-04-24T23:00:17.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.819+0000 7f19944d4640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f198c0d2400 con 0x7f198c15e9b0 2024-04-24T23:00:17.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.819+0000 7f19944d4640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f198c0ba260 con 0x7f198c06b820 2024-04-24T23:00:17.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.819+0000 7f1991a6f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f198c0d4210 0x7f198c152a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:17.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.819+0000 7f1991a6f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f198c0d4210 0x7f198c152a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41744/0 (socket says 172.21.15.32:41744) 2024-04-24T23:00:17.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.819+0000 7f1991a6f640 1 -- 172.21.15.32:0/4085906782 learned_addr learned my addr 172.21.15.32:0/4085906782 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:17.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.820+0000 7f1992a71640 1 --2- 172.21.15.32:0/4085906782 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f198c15e9b0 0x7f198c15b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:17.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.820+0000 7f1992270640 1 --2- 172.21.15.32:0/4085906782 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f198c06b820 0x7f198c159430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:17.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.820+0000 7f1991a6f640 1 -- 172.21.15.32:0/4085906782 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f198c06b820 msgr2=0x7f198c159430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:17.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.820+0000 7f1991a6f640 1 --2- 172.21.15.32:0/4085906782 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f198c06b820 0x7f198c159430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:17.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.820+0000 7f1991a6f640 1 -- 172.21.15.32:0/4085906782 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f198c15e9b0 msgr2=0x7f198c15b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:17.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.820+0000 7f1991a6f640 1 --2- 172.21.15.32:0/4085906782 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f198c15e9b0 0x7f198c15b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:17.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.820+0000 7f1991a6f640 1 -- 172.21.15.32:0/4085906782 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f198c15a410 con 0x7f198c0d4210 2024-04-24T23:00:17.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.820+0000 7f1992a71640 1 --2- 172.21.15.32:0/4085906782 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f198c15e9b0 0x7f198c15b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:00:17.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.820+0000 7f1991a6f640 1 --2- 172.21.15.32:0/4085906782 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f198c0d4210 0x7f198c152a30 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f198000c500 tx=0x7f198000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:17.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.820+0000 7f198affd640 1 -- 172.21.15.32:0/4085906782 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1980018070 con 0x7f198c0d4210 2024-04-24T23:00:17.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.820+0000 7f198affd640 1 -- 172.21.15.32:0/4085906782 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1980007580 con 0x7f198c0d4210 2024-04-24T23:00:17.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.820+0000 7f19944d4640 1 -- 172.21.15.32:0/4085906782 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f198c15bc80 con 0x7f198c0d4210 2024-04-24T23:00:17.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.820+0000 7f198affd640 1 -- 172.21.15.32:0/4085906782 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1980004a20 con 0x7f198c0d4210 2024-04-24T23:00:17.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.820+0000 7f19944d4640 1 -- 172.21.15.32:0/4085906782 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f198c15a980 con 0x7f198c0d4210 2024-04-24T23:00:17.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.821+0000 7f19944d4640 1 -- 172.21.15.32:0/4085906782 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f198c068430 con 0x7f198c0d4210 2024-04-24T23:00:17.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.821+0000 7f198affd640 1 -- 172.21.15.32:0/4085906782 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f198001b070 con 0x7f198c0d4210 2024-04-24T23:00:17.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.822+0000 7f198affd640 1 --2- 172.21.15.32:0/4085906782 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f197806be30 0x7f197806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:17.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.822+0000 7f198affd640 1 -- 172.21.15.32:0/4085906782 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f198008a140 con 0x7f198c0d4210 2024-04-24T23:00:17.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.822+0000 7f1992270640 1 --2- 172.21.15.32:0/4085906782 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f197806be30 0x7f197806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:17.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.823+0000 7f1992270640 1 --2- 172.21.15.32:0/4085906782 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f197806be30 0x7f197806e2f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1984034000 tx=0x7f198402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:17.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:17.826+0000 7f198affd640 1 -- 172.21.15.32:0/4085906782 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1980058b80 con 0x7f198c0d4210 2024-04-24T23:00:18.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.009+0000 7f19944d4640 1 -- 172.21.15.32:0/4085906782 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f198c0b8f80 con 0x7f198c0d4210 2024-04-24T23:00:18.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.010+0000 7f198affd640 1 -- 172.21.15.32:0/4085906782 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16621 (secure 0 0 0) 0x7f1980058480 con 0x7f198c0d4210 2024-04-24T23:00:18.012 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:00:18.012 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":34,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:00:12.312472+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:59:56.068812+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-24T22:59:56.447402+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:00:18.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.012+0000 7f19944d4640 1 -- 172.21.15.32:0/4085906782 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f197806be30 msgr2=0x7f197806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:18.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.012+0000 7f19944d4640 1 --2- 172.21.15.32:0/4085906782 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f197806be30 0x7f197806e2f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1984034000 tx=0x7f198402e000 comp rx=0 tx=0).stop 2024-04-24T23:00:18.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.012+0000 7f19944d4640 1 -- 172.21.15.32:0/4085906782 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f198c0d4210 msgr2=0x7f198c152a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:18.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.012+0000 7f19944d4640 1 --2- 172.21.15.32:0/4085906782 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f198c0d4210 0x7f198c152a30 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f198000c500 tx=0x7f198000c9c0 comp rx=0 tx=0).stop 2024-04-24T23:00:18.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.012+0000 7f19944d4640 1 -- 172.21.15.32:0/4085906782 shutdown_connections 2024-04-24T23:00:18.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.012+0000 7f19944d4640 1 --2- 172.21.15.32:0/4085906782 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f197806be30 0x7f197806e2f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:18.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.012+0000 7f19944d4640 1 --2- 172.21.15.32:0/4085906782 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f198c0d4210 0x7f198c152a30 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:18.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.012+0000 7f19944d4640 1 --2- 172.21.15.32:0/4085906782 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f198c15e9b0 0x7f198c15b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:18.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.012+0000 7f19944d4640 1 --2- 172.21.15.32:0/4085906782 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f198c06b820 0x7f198c159430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:18.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.012+0000 7f19944d4640 1 -- 172.21.15.32:0/4085906782 >> 172.21.15.32:0/4085906782 conn(0x7f198c0d11d0 msgr2=0x7f198c0d46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:18.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.012+0000 7f19944d4640 1 -- 172.21.15.32:0/4085906782 shutdown_connections 2024-04-24T23:00:18.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.012+0000 7f19944d4640 1 -- 172.21.15.32:0/4085906782 wait complete. 2024-04-24T23:00:18.024 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T22:59:56.068812+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '31', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T22:59:56.447402+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:00:18.025 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:00:18.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.127+0000 7fb3b7a88640 1 Processor -- start 2024-04-24T23:00:18.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.127+0000 7fb3b7a88640 1 -- start start 2024-04-24T23:00:18.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.128+0000 7fb3b7a88640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb3b00afbb0 0x7fb3b00b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:18.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.128+0000 7fb3b7a88640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb3b006b820 0x7fb3b015b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:18.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.128+0000 7fb3b7a88640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3b00b4460 con 0x7fb3b00aff90 2024-04-24T23:00:18.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.128+0000 7fb3b7a88640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fb3b00b4700 con 0x7fb3b006b820 2024-04-24T23:00:18.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.128+0000 7fb3b7a88640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fb3b00b49e0 con 0x7fb3b00afbb0 2024-04-24T23:00:18.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.128+0000 7fb3b6025640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb3b006b820 0x7fb3b015b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:18.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.128+0000 7fb3b6025640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb3b006b820 0x7fb3b015b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41752/0 (socket says 172.21.15.32:41752) 2024-04-24T23:00:18.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.128+0000 7fb3b6025640 1 -- 172.21.15.32:0/2720928821 learned_addr learned my addr 172.21.15.32:0/2720928821 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:18.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.128+0000 7fb3b5824640 1 --2- 172.21.15.32:0/2720928821 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb3b00afbb0 0x7fb3b00b54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:18.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.128+0000 7fb3b6025640 1 -- 172.21.15.32:0/2720928821 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb3b00afbb0 msgr2=0x7fb3b00b54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:18.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.128+0000 7fb3b6025640 1 --2- 172.21.15.32:0/2720928821 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb3b00afbb0 0x7fb3b00b54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:18.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.128+0000 7fb3b6025640 1 -- 172.21.15.32:0/2720928821 >> v1:172.21.15.121:6789/0 conn(0x7fb3b00aff90 legacy=0x7fb3b006b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:18.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.128+0000 7fb3b6025640 1 -- 172.21.15.32:0/2720928821 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3b00b4cc0 con 0x7fb3b006b820 2024-04-24T23:00:18.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.129+0000 7fb3b6025640 1 --2- 172.21.15.32:0/2720928821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3b006b820 0x7fb3b015b130 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8009e50 tx=0x7fb3a802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e61d56e307f504cb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:18.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.129+0000 7fb3b4822640 1 -- 172.21.15.32:0/2720928821 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a8036070 con 0x7fb3b006b820 2024-04-24T23:00:18.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.129+0000 7fb3b4822640 1 -- 172.21.15.32:0/2720928821 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3a8031600 con 0x7fb3b006b820 2024-04-24T23:00:18.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.129+0000 7fb3b4822640 1 -- 172.21.15.32:0/2720928821 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a80319d0 con 0x7fb3b006b820 2024-04-24T23:00:18.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.130+0000 7fb3b7a88640 1 -- 172.21.15.32:0/2720928821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3b006b820 msgr2=0x7fb3b015b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:18.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.130+0000 7fb3b7a88640 1 --2- 172.21.15.32:0/2720928821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3b006b820 0x7fb3b015b130 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8009e50 tx=0x7fb3a802a710 comp rx=0 tx=0).stop 2024-04-24T23:00:18.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.130+0000 7fb3b7a88640 1 -- 172.21.15.32:0/2720928821 shutdown_connections 2024-04-24T23:00:18.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.130+0000 7fb3b7a88640 1 --2- 172.21.15.32:0/2720928821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3b006b820 0x7fb3b015b130 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:18.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.130+0000 7fb3b7a88640 1 --2- 172.21.15.32:0/2720928821 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb3b00afbb0 0x7fb3b00b54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:18.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.130+0000 7fb3b7a88640 1 -- 172.21.15.32:0/2720928821 >> 172.21.15.32:0/2720928821 conn(0x7fb3b00d0730 msgr2=0x7fb3b00d0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:18.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.130+0000 7fb3b7a88640 1 -- 172.21.15.32:0/2720928821 shutdown_connections 2024-04-24T23:00:18.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.130+0000 7fb3b7a88640 1 -- 172.21.15.32:0/2720928821 wait complete. 2024-04-24T23:00:18.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.131+0000 7fb3b7a88640 1 Processor -- start 2024-04-24T23:00:18.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.131+0000 7fb3b7a88640 1 -- start start 2024-04-24T23:00:18.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.132+0000 7fb3b7a88640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3b006b820 0x7fb3b01f3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:18.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.132+0000 7fb3b7a88640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb3b00afbb0 0x7fb3b014efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:18.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.132+0000 7fb3b7a88640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3b00aff90 0x7fb3b015eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:18.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.132+0000 7fb3b7a88640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b0069260 con 0x7fb3b006b820 2024-04-24T23:00:18.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.132+0000 7fb3b7a88640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b00690e0 con 0x7fb3b00aff90 2024-04-24T23:00:18.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.132+0000 7fb3b7a88640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fb3b00cf790 con 0x7fb3b00afbb0 2024-04-24T23:00:18.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.132+0000 7fb3b5824640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3b006b820 0x7fb3b01f3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:18.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.132+0000 7fb3b5824640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3b006b820 0x7fb3b01f3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41756/0 (socket says 172.21.15.32:41756) 2024-04-24T23:00:18.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.132+0000 7fb3b5824640 1 -- 172.21.15.32:0/2421023807 learned_addr learned my addr 172.21.15.32:0/2421023807 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:18.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.132+0000 7fb3b6025640 1 --2- 172.21.15.32:0/2421023807 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3b00aff90 0x7fb3b015eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:18.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.132+0000 7fb3b5023640 1 --2- 172.21.15.32:0/2421023807 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb3b00afbb0 0x7fb3b014efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:18.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.132+0000 7fb3b5824640 1 -- 172.21.15.32:0/2421023807 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb3b00afbb0 msgr2=0x7fb3b014efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:18.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.132+0000 7fb3b5824640 1 --2- 172.21.15.32:0/2421023807 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb3b00afbb0 0x7fb3b014efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:18.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.132+0000 7fb3b5824640 1 -- 172.21.15.32:0/2421023807 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3b00aff90 msgr2=0x7fb3b015eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:18.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.132+0000 7fb3b5824640 1 --2- 172.21.15.32:0/2421023807 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3b00aff90 0x7fb3b015eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:18.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.132+0000 7fb3b5824640 1 -- 172.21.15.32:0/2421023807 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3b00ad4c0 con 0x7fb3b006b820 2024-04-24T23:00:18.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.132+0000 7fb3b5824640 1 --2- 172.21.15.32:0/2421023807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3b006b820 0x7fb3b01f3120 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac00d800 tx=0x7fb3ac00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:18.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.132+0000 7fb3a67fc640 1 -- 172.21.15.32:0/2421023807 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac00b6c0 con 0x7fb3b006b820 2024-04-24T23:00:18.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.133+0000 7fb3a67fc640 1 -- 172.21.15.32:0/2421023807 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3ac019070 con 0x7fb3b006b820 2024-04-24T23:00:18.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.133+0000 7fb3b7a88640 1 -- 172.21.15.32:0/2421023807 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3b00ad1e0 con 0x7fb3b006b820 2024-04-24T23:00:18.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.133+0000 7fb3a67fc640 1 -- 172.21.15.32:0/2421023807 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac00ba80 con 0x7fb3b006b820 2024-04-24T23:00:18.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.133+0000 7fb3b7a88640 1 -- 172.21.15.32:0/2421023807 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3b01f2920 con 0x7fb3b006b820 2024-04-24T23:00:18.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.134+0000 7fb3b7a88640 1 -- 172.21.15.32:0/2421023807 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3b00670d0 con 0x7fb3b006b820 2024-04-24T23:00:18.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.134+0000 7fb3a67fc640 1 -- 172.21.15.32:0/2421023807 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb3ac00bc20 con 0x7fb3b006b820 2024-04-24T23:00:18.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.134+0000 7fb3a67fc640 1 --2- 172.21.15.32:0/2421023807 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb39406be30 0x7fb39406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:18.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.134+0000 7fb3a67fc640 1 -- 172.21.15.32:0/2421023807 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fb3ac08b110 con 0x7fb3b006b820 2024-04-24T23:00:18.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.135+0000 7fb3b5023640 1 --2- 172.21.15.32:0/2421023807 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb39406be30 0x7fb39406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:18.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.135+0000 7fb3b5023640 1 --2- 172.21.15.32:0/2421023807 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb39406be30 0x7fb39406e2f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0158cb0 tx=0x7fb3a0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:18.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.139+0000 7fb3a67fc640 1 -- 172.21.15.32:0/2421023807 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb3b00670d0 con 0x7fb3b006b820 2024-04-24T23:00:18.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.320+0000 7fb3b7a88640 1 -- 172.21.15.32:0/2421023807 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb3b00b1cd0 con 0x7fb3b006b820 2024-04-24T23:00:18.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.320+0000 7fb3a67fc640 1 -- 172.21.15.32:0/2421023807 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) ==== 93+0+10 (secure 0 0 0) 0x7fb3b00b1cd0 con 0x7fb3b006b820 2024-04-24T23:00:18.322 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T23:00:18.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.322+0000 7fb3b7a88640 1 -- 172.21.15.32:0/2421023807 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb39406be30 msgr2=0x7fb39406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:18.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.322+0000 7fb3b7a88640 1 --2- 172.21.15.32:0/2421023807 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb39406be30 0x7fb39406e2f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0158cb0 tx=0x7fb3a0009040 comp rx=0 tx=0).stop 2024-04-24T23:00:18.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.322+0000 7fb3b7a88640 1 -- 172.21.15.32:0/2421023807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3b006b820 msgr2=0x7fb3b01f3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:18.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.322+0000 7fb3b7a88640 1 --2- 172.21.15.32:0/2421023807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3b006b820 0x7fb3b01f3120 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac00d800 tx=0x7fb3ac00dcc0 comp rx=0 tx=0).stop 2024-04-24T23:00:18.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.322+0000 7fb3b7a88640 1 -- 172.21.15.32:0/2421023807 shutdown_connections 2024-04-24T23:00:18.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.322+0000 7fb3b7a88640 1 --2- 172.21.15.32:0/2421023807 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb39406be30 0x7fb39406e2f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:18.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.322+0000 7fb3b7a88640 1 --2- 172.21.15.32:0/2421023807 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3b00aff90 0x7fb3b015eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:18.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.322+0000 7fb3b7a88640 1 --2- 172.21.15.32:0/2421023807 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb3b00afbb0 0x7fb3b014efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:18.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.322+0000 7fb3b7a88640 1 --2- 172.21.15.32:0/2421023807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3b006b820 0x7fb3b01f3120 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:18.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.322+0000 7fb3b7a88640 1 -- 172.21.15.32:0/2421023807 >> 172.21.15.32:0/2421023807 conn(0x7fb3b00d0730 msgr2=0x7fb3b00cfec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:18.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.322+0000 7fb3b7a88640 1 -- 172.21.15.32:0/2421023807 shutdown_connections 2024-04-24T23:00:18.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.322+0000 7fb3b7a88640 1 -- 172.21.15.32:0/2421023807 wait complete. 2024-04-24T23:00:18.335 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:00:18.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.437+0000 7f9688cfc640 1 Processor -- start 2024-04-24T23:00:18.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.437+0000 7f9688cfc640 1 -- start start 2024-04-24T23:00:18.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.437+0000 7f9688cfc640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f968415e9b0 0x7f96840d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:18.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.437+0000 7f9688cfc640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96840d3170 con 0x7f968406b820 2024-04-24T23:00:18.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.437+0000 7f9688cfc640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96840d3450 con 0x7f96840d4210 2024-04-24T23:00:18.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.437+0000 7f9688cfc640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f96840d3730 con 0x7f968415e9b0 2024-04-24T23:00:18.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.438+0000 7f9682d9d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f968406b820 0x7f96840b78f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58176/0 (socket says 172.21.15.32:58176) 2024-04-24T23:00:18.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.438+0000 7f9682d9d640 1 -- 172.21.15.32:0/4112203115 learned_addr learned my addr 172.21.15.32:0/4112203115 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:00:18.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.438+0000 7f968259c640 1 --2- 172.21.15.32:0/4112203115 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f968415e9b0 0x7f96840d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:18.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.438+0000 7f9681d9b640 1 -- 172.21.15.32:0/4112203115 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 256877224 0 0) 0x7f96840d3170 con 0x7f968406b820 2024-04-24T23:00:18.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.438+0000 7f9681d9b640 1 -- 172.21.15.32:0/4112203115 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9670003520 con 0x7f968406b820 2024-04-24T23:00:18.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.438+0000 7f9681d9b640 1 -- 172.21.15.32:0/4112203115 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3906170198 0 0) 0x7f96840d3450 con 0x7f96840d4210 2024-04-24T23:00:18.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.438+0000 7f9681d9b640 1 -- 172.21.15.32:0/4112203115 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f96840d3170 con 0x7f96840d4210 2024-04-24T23:00:18.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.438+0000 7f9681d9b640 1 -- 172.21.15.32:0/4112203115 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 410687335 0 0) 0x7f9670003520 con 0x7f968406b820 2024-04-24T23:00:18.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.438+0000 7f9681d9b640 1 -- 172.21.15.32:0/4112203115 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f968415e9b0 msgr2=0x7f96840d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:18.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.438+0000 7f9681d9b640 1 --2- 172.21.15.32:0/4112203115 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f968415e9b0 0x7f96840d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:18.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.438+0000 7f9681d9b640 1 -- 172.21.15.32:0/4112203115 >> v1:172.21.15.121:6789/0 conn(0x7f96840d4210 legacy=0x7f968406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:18.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.438+0000 7f9681d9b640 1 -- 172.21.15.32:0/4112203115 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96840d3a10 con 0x7f968406b820 2024-04-24T23:00:18.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.438+0000 7f9681d9b640 1 -- 172.21.15.32:0/4112203115 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f967c0019f0 con 0x7f968406b820 2024-04-24T23:00:18.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.438+0000 7f9681d9b640 1 -- 172.21.15.32:0/4112203115 >> v1:172.21.15.32:6789/0 conn(0x7f968406b820 legacy=0x7f96840b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:18.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.438+0000 7f9681d9b640 1 --2- 172.21.15.32:0/4112203115 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f968415e9b0 0x7f9670003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:18.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.438+0000 7f9681d9b640 1 --2- 172.21.15.32:0/4112203115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96700040d0 0x7f9670004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:18.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.438+0000 7f9681d9b640 1 --2- 172.21.15.32:0/4112203115 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9670004d40 0x7f96700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:18.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.438+0000 7f9681d9b640 1 -- 172.21.15.32:0/4112203115 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f96840d3730 con 0x7f96700040d0 2024-04-24T23:00:18.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.438+0000 7f9681d9b640 1 -- 172.21.15.32:0/4112203115 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f967000baa0 con 0x7f9670004d40 2024-04-24T23:00:18.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.438+0000 7f9681d9b640 1 -- 172.21.15.32:0/4112203115 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f967000bca0 con 0x7f968415e9b0 2024-04-24T23:00:18.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.438+0000 7f967bfff640 1 --2- 172.21.15.32:0/4112203115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96700040d0 0x7f9670004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:18.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.438+0000 7f968259c640 1 --2- 172.21.15.32:0/4112203115 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f968415e9b0 0x7f9670003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:18.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.438+0000 7f9682d9d640 1 --2- 172.21.15.32:0/4112203115 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9670004d40 0x7f96700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:18.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.439+0000 7f967bfff640 1 -- 172.21.15.32:0/4112203115 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f968415e9b0 msgr2=0x7f9670003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:18.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.439+0000 7f967bfff640 1 --2- 172.21.15.32:0/4112203115 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f968415e9b0 0x7f9670003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:18.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.439+0000 7f967bfff640 1 -- 172.21.15.32:0/4112203115 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9670004d40 msgr2=0x7f96700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:18.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.439+0000 7f967bfff640 1 --2- 172.21.15.32:0/4112203115 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9670004d40 0x7f96700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:18.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.439+0000 7f967bfff640 1 -- 172.21.15.32:0/4112203115 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9670003520 con 0x7f96700040d0 2024-04-24T23:00:18.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.439+0000 7f967bfff640 1 --2- 172.21.15.32:0/4112203115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96700040d0 0x7f9670004800 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f966c0018d0 tx=0x7f9670005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d1e253817c6c35 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:18.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.439+0000 7f9681d9b640 1 -- 172.21.15.32:0/4112203115 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f966c0014f0 con 0x7f96700040d0 2024-04-24T23:00:18.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.439+0000 7f9681d9b640 1 -- 172.21.15.32:0/4112203115 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f966c0052d0 con 0x7f96700040d0 2024-04-24T23:00:18.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.439+0000 7f9681d9b640 1 -- 172.21.15.32:0/4112203115 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f966c02aca0 con 0x7f96700040d0 2024-04-24T23:00:18.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.440+0000 7f9688cfc640 1 -- 172.21.15.32:0/4112203115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96700040d0 msgr2=0x7f9670004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:18.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.440+0000 7f9688cfc640 1 --2- 172.21.15.32:0/4112203115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96700040d0 0x7f9670004800 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f966c0018d0 tx=0x7f9670005830 comp rx=0 tx=0).stop 2024-04-24T23:00:18.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.440+0000 7f9682d9d640 1 -- 172.21.15.32:0/4112203115 reap_dead start 2024-04-24T23:00:18.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.440+0000 7f9688cfc640 1 -- 172.21.15.32:0/4112203115 shutdown_connections 2024-04-24T23:00:18.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.440+0000 7f9688cfc640 1 -- 172.21.15.32:0/4112203115 >> 172.21.15.32:0/4112203115 conn(0x7f96840d11d0 msgr2=0x7f96840d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:18.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.440+0000 7f9688cfc640 1 -- 172.21.15.32:0/4112203115 shutdown_connections 2024-04-24T23:00:18.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.440+0000 7f9688cfc640 1 -- 172.21.15.32:0/4112203115 wait complete. 2024-04-24T23:00:18.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.441+0000 7f9688cfc640 1 Processor -- start 2024-04-24T23:00:18.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.441+0000 7f9688cfc640 1 -- start start 2024-04-24T23:00:18.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.442+0000 7f9688cfc640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96700040d0 0x7f96840ab430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:18.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.442+0000 7f9688cfc640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f96840ab970 0x7f96841f4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:18.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.442+0000 7f9688cfc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96841f4570 0x7f96841f4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:18.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.442+0000 7f9688cfc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f96840ba0e0 con 0x7f96841f4570 2024-04-24T23:00:18.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.442+0000 7f9688cfc640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f96840d2400 con 0x7f96700040d0 2024-04-24T23:00:18.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.442+0000 7f9688cfc640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f96840ba260 con 0x7f96840ab970 2024-04-24T23:00:18.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.442+0000 7f9682d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96841f4570 0x7f96841f4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:18.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.442+0000 7f9682d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96841f4570 0x7f96841f4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41772/0 (socket says 172.21.15.32:41772) 2024-04-24T23:00:18.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.442+0000 7f9682d9d640 1 -- 172.21.15.32:0/1232076682 learned_addr learned my addr 172.21.15.32:0/1232076682 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:18.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.442+0000 7f967bfff640 1 --2- 172.21.15.32:0/1232076682 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f96840ab970 0x7f96841f4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:18.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.442+0000 7f968259c640 1 --2- 172.21.15.32:0/1232076682 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96700040d0 0x7f96840ab430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:18.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.442+0000 7f9682d9d640 1 -- 172.21.15.32:0/1232076682 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f96840ab970 msgr2=0x7f96841f4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:18.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.442+0000 7f9682d9d640 1 --2- 172.21.15.32:0/1232076682 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f96840ab970 0x7f96841f4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:18.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.442+0000 7f9682d9d640 1 -- 172.21.15.32:0/1232076682 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96700040d0 msgr2=0x7f96840ab430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:18.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.442+0000 7f9682d9d640 1 --2- 172.21.15.32:0/1232076682 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96700040d0 0x7f96840ab430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:18.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.442+0000 7f9682d9d640 1 -- 172.21.15.32:0/1232076682 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96840abef0 con 0x7f96841f4570 2024-04-24T23:00:18.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.442+0000 7f967bfff640 1 --2- 172.21.15.32:0/1232076682 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f96840ab970 0x7f96841f4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:00:18.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.442+0000 7f968259c640 1 --2- 172.21.15.32:0/1232076682 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96700040d0 0x7f96840ab430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:00:18.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.443+0000 7f9682d9d640 1 --2- 172.21.15.32:0/1232076682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96841f4570 0x7f96841f4a20 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f96841622f0 tx=0x7f967c00e160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:18.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.443+0000 7f967affd640 1 -- 172.21.15.32:0/1232076682 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967c00eb70 con 0x7f96841f4570 2024-04-24T23:00:18.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.443+0000 7f967affd640 1 -- 172.21.15.32:0/1232076682 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f967c018070 con 0x7f96841f4570 2024-04-24T23:00:18.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.443+0000 7f9688cfc640 1 -- 172.21.15.32:0/1232076682 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96840ac1d0 con 0x7f96841f4570 2024-04-24T23:00:18.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.443+0000 7f967affd640 1 -- 172.21.15.32:0/1232076682 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967c013420 con 0x7f96841f4570 2024-04-24T23:00:18.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.443+0000 7f9688cfc640 1 -- 172.21.15.32:0/1232076682 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96840ac790 con 0x7f96841f4570 2024-04-24T23:00:18.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.444+0000 7f9688cfc640 1 -- 172.21.15.32:0/1232076682 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9684068430 con 0x7f96841f4570 2024-04-24T23:00:18.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.444+0000 7f967affd640 1 -- 172.21.15.32:0/1232076682 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f967c0135c0 con 0x7f96841f4570 2024-04-24T23:00:18.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.445+0000 7f967affd640 1 --2- 172.21.15.32:0/1232076682 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f966406be60 0x7f966406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:18.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.445+0000 7f967affd640 1 -- 172.21.15.32:0/1232076682 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f967c08b140 con 0x7f96841f4570 2024-04-24T23:00:18.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.445+0000 7f968259c640 1 --2- 172.21.15.32:0/1232076682 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f966406be60 0x7f966406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:18.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.445+0000 7f968259c640 1 --2- 172.21.15.32:0/1232076682 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f966406be60 0x7f966406e320 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f96840cdb40 tx=0x7f9674008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:18.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.449+0000 7f967affd640 1 -- 172.21.15.32:0/1232076682 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f967c059af0 con 0x7f96841f4570 2024-04-24T23:00:18.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.632+0000 7f9688cfc640 1 -- 172.21.15.32:0/1232076682 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f96840724b0 con 0x7f96841f4570 2024-04-24T23:00:18.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.633+0000 7f967affd640 1 -- 172.21.15.32:0/1232076682 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) ==== 104+0+11 (secure 0 0 0) 0x7f967c0593f0 con 0x7f96841f4570 2024-04-24T23:00:18.635 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-24T23:00:18.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.635+0000 7f9688cfc640 1 -- 172.21.15.32:0/1232076682 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f966406be60 msgr2=0x7f966406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:18.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.635+0000 7f9688cfc640 1 --2- 172.21.15.32:0/1232076682 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f966406be60 0x7f966406e320 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f96840cdb40 tx=0x7f9674008040 comp rx=0 tx=0).stop 2024-04-24T23:00:18.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.635+0000 7f9688cfc640 1 -- 172.21.15.32:0/1232076682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96841f4570 msgr2=0x7f96841f4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:18.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.635+0000 7f9688cfc640 1 --2- 172.21.15.32:0/1232076682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96841f4570 0x7f96841f4a20 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f96841622f0 tx=0x7f967c00e160 comp rx=0 tx=0).stop 2024-04-24T23:00:18.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.635+0000 7f9688cfc640 1 -- 172.21.15.32:0/1232076682 shutdown_connections 2024-04-24T23:00:18.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.635+0000 7f9688cfc640 1 --2- 172.21.15.32:0/1232076682 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f966406be60 0x7f966406e320 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:18.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.635+0000 7f9688cfc640 1 --2- 172.21.15.32:0/1232076682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96841f4570 0x7f96841f4a20 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:18.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.635+0000 7f9688cfc640 1 --2- 172.21.15.32:0/1232076682 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f96840ab970 0x7f96841f4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:18.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.635+0000 7f9688cfc640 1 --2- 172.21.15.32:0/1232076682 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96700040d0 0x7f96840ab430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:18.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.635+0000 7f9688cfc640 1 -- 172.21.15.32:0/1232076682 >> 172.21.15.32:0/1232076682 conn(0x7f96840d11d0 msgr2=0x7f9684160da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:18.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.635+0000 7f9688cfc640 1 -- 172.21.15.32:0/1232076682 shutdown_connections 2024-04-24T23:00:18.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.635+0000 7f9688cfc640 1 -- 172.21.15.32:0/1232076682 wait complete. 2024-04-24T23:00:18.647 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:00:18.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.749+0000 7eff6d57d640 1 Processor -- start 2024-04-24T23:00:18.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.749+0000 7eff6d57d640 1 -- start start 2024-04-24T23:00:18.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.750+0000 7eff6d57d640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7eff6815cc40 0x7eff6815d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:18.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.750+0000 7eff6d57d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7eff680d05b0 0x7eff6815d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:18.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.750+0000 7eff6d57d640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff680b4ee0 con 0x7eff680cec30 2024-04-24T23:00:18.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.750+0000 7eff6d57d640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7eff6806cca0 con 0x7eff6815cc40 2024-04-24T23:00:18.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.750+0000 7eff6d57d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7eff680ce5f0 con 0x7eff680d05b0 2024-04-24T23:00:18.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.750+0000 7eff667fc640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7eff680d05b0 0x7eff6815d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:18.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.750+0000 7eff66ffd640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7eff6815cc40 0x7eff6815d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:18.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.750+0000 7eff667fc640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7eff680d05b0 0x7eff6815d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.32:53658/0 (socket says 172.21.15.32:53658) 2024-04-24T23:00:18.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.750+0000 7eff667fc640 1 -- 172.21.15.32:0/2349107110 learned_addr learned my addr 172.21.15.32:0/2349107110 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:18.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.750+0000 7eff677fe640 1 --1- 172.21.15.32:0/2349107110 >> v1:172.21.15.121:6789/0 conn(0x7eff680cec30 0x7eff680cf0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:43668/0 (socket says 172.21.15.32:43668) 2024-04-24T23:00:18.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.750+0000 7eff667fc640 1 -- 172.21.15.32:0/2349107110 >> v2:172.21.15.121:3300/0 conn(0x7eff6815cc40 msgr2=0x7eff6815d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:18.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.750+0000 7eff667fc640 1 --2- 172.21.15.32:0/2349107110 >> v2:172.21.15.121:3300/0 conn(0x7eff6815cc40 0x7eff6815d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:18.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.750+0000 7eff667fc640 1 -- 172.21.15.32:0/2349107110 >> v1:172.21.15.121:6789/0 conn(0x7eff680cec30 legacy=0x7eff680cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:18.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.750+0000 7eff667fc640 1 -- 172.21.15.32:0/2349107110 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff680b51c0 con 0x7eff680d05b0 2024-04-24T23:00:18.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.750+0000 7eff66ffd640 1 --2- 172.21.15.32:0/2349107110 >> v2:172.21.15.121:3300/0 conn(0x7eff6815cc40 0x7eff6815d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:00:18.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.751+0000 7eff667fc640 1 --2- 172.21.15.32:0/2349107110 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7eff680d05b0 0x7eff6815d560 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7eff5c009870 tx=0x7eff5c02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f7d3117ea5143f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:18.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.751+0000 7eff65ffb640 1 -- 172.21.15.32:0/2349107110 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5c037070 con 0x7eff680d05b0 2024-04-24T23:00:18.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.751+0000 7eff65ffb640 1 -- 172.21.15.32:0/2349107110 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff5c032440 con 0x7eff680d05b0 2024-04-24T23:00:18.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.751+0000 7eff65ffb640 1 -- 172.21.15.32:0/2349107110 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5c032810 con 0x7eff680d05b0 2024-04-24T23:00:18.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.752+0000 7eff6d57d640 1 -- 172.21.15.32:0/2349107110 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7eff680d05b0 msgr2=0x7eff6815d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:18.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.752+0000 7eff6d57d640 1 --2- 172.21.15.32:0/2349107110 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7eff680d05b0 0x7eff6815d560 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7eff5c009870 tx=0x7eff5c02a270 comp rx=0 tx=0).stop 2024-04-24T23:00:18.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.752+0000 7eff6d57d640 1 -- 172.21.15.32:0/2349107110 shutdown_connections 2024-04-24T23:00:18.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.752+0000 7eff6d57d640 1 --2- 172.21.15.32:0/2349107110 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7eff680d05b0 0x7eff6815d560 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:18.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.752+0000 7eff6d57d640 1 --2- 172.21.15.32:0/2349107110 >> v2:172.21.15.121:3300/0 conn(0x7eff6815cc40 0x7eff6815d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:18.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.752+0000 7eff6d57d640 1 -- 172.21.15.32:0/2349107110 >> 172.21.15.32:0/2349107110 conn(0x7eff680d0ae0 msgr2=0x7eff680d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:18.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.752+0000 7eff6d57d640 1 -- 172.21.15.32:0/2349107110 shutdown_connections 2024-04-24T23:00:18.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.752+0000 7eff6d57d640 1 -- 172.21.15.32:0/2349107110 wait complete. 2024-04-24T23:00:18.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.753+0000 7eff6d57d640 1 Processor -- start 2024-04-24T23:00:18.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.753+0000 7eff6d57d640 1 -- start start 2024-04-24T23:00:18.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.754+0000 7eff6d57d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff680cec30 0x7eff6815a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:18.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.754+0000 7eff6d57d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7eff680d05b0 0x7eff68142520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:18.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.754+0000 7eff6d57d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff6815cc40 0x7eff681f59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:18.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.754+0000 7eff6d57d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7eff680b5780 con 0x7eff6815cc40 2024-04-24T23:00:18.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.754+0000 7eff6d57d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7eff6815daa0 con 0x7eff680cec30 2024-04-24T23:00:18.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.754+0000 7eff6d57d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7eff680b5a60 con 0x7eff680d05b0 2024-04-24T23:00:18.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.754+0000 7eff677fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff6815cc40 0x7eff681f59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:18.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.754+0000 7eff677fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff6815cc40 0x7eff681f59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41782/0 (socket says 172.21.15.32:41782) 2024-04-24T23:00:18.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.754+0000 7eff677fe640 1 -- 172.21.15.32:0/2259823550 learned_addr learned my addr 172.21.15.32:0/2259823550 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:18.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.754+0000 7eff66ffd640 1 --2- 172.21.15.32:0/2259823550 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff680cec30 0x7eff6815a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:18.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.754+0000 7eff667fc640 1 --2- 172.21.15.32:0/2259823550 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7eff680d05b0 0x7eff68142520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:18.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.754+0000 7eff677fe640 1 -- 172.21.15.32:0/2259823550 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7eff680d05b0 msgr2=0x7eff68142520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:18.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.754+0000 7eff677fe640 1 --2- 172.21.15.32:0/2259823550 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7eff680d05b0 0x7eff68142520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:18.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.754+0000 7eff677fe640 1 -- 172.21.15.32:0/2259823550 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff680cec30 msgr2=0x7eff6815a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:18.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.754+0000 7eff677fe640 1 --2- 172.21.15.32:0/2259823550 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff680cec30 0x7eff6815a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:18.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.754+0000 7eff677fe640 1 -- 172.21.15.32:0/2259823550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff68142a60 con 0x7eff6815cc40 2024-04-24T23:00:18.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.755+0000 7eff677fe640 1 --2- 172.21.15.32:0/2259823550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff6815cc40 0x7eff681f59f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7eff6000c730 tx=0x7eff6000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:18.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.755+0000 7eff4f7fe640 1 -- 172.21.15.32:0/2259823550 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff60017070 con 0x7eff6815cc40 2024-04-24T23:00:18.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.755+0000 7eff4f7fe640 1 -- 172.21.15.32:0/2259823550 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff6000ce40 con 0x7eff6815cc40 2024-04-24T23:00:18.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.755+0000 7eff6d57d640 1 -- 172.21.15.32:0/2259823550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff681f63d0 con 0x7eff6815cc40 2024-04-24T23:00:18.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.755+0000 7eff4f7fe640 1 -- 172.21.15.32:0/2259823550 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6000a840 con 0x7eff6815cc40 2024-04-24T23:00:18.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.755+0000 7eff6d57d640 1 -- 172.21.15.32:0/2259823550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff68150df0 con 0x7eff6815cc40 2024-04-24T23:00:18.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.756+0000 7eff6d57d640 1 -- 172.21.15.32:0/2259823550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff681f68f0 con 0x7eff6815cc40 2024-04-24T23:00:18.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.756+0000 7eff4f7fe640 1 -- 172.21.15.32:0/2259823550 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7eff6001a070 con 0x7eff6815cc40 2024-04-24T23:00:18.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.757+0000 7eff4f7fe640 1 --2- 172.21.15.32:0/2259823550 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7eff5406be30 0x7eff5406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:18.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.757+0000 7eff4f7fe640 1 -- 172.21.15.32:0/2259823550 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7eff6008a220 con 0x7eff6815cc40 2024-04-24T23:00:18.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.757+0000 7eff66ffd640 1 --2- 172.21.15.32:0/2259823550 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7eff5406be30 0x7eff5406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:18.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.757+0000 7eff66ffd640 1 --2- 172.21.15.32:0/2259823550 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7eff5406be30 0x7eff5406e2f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7eff6815cc10 tx=0x7eff58009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:18.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.761+0000 7eff4f7fe640 1 -- 172.21.15.32:0/2259823550 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7eff68142a60 con 0x7eff6815cc40 2024-04-24T23:00:18.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.942+0000 7eff6d57d640 1 -- 172.21.15.32:0/2259823550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7eff68068430 con 0x7eff6815cc40 2024-04-24T23:00:18.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.943+0000 7eff4f7fe640 1 -- 172.21.15.32:0/2259823550 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) ==== 100+0+11 (secure 0 0 0) 0x7eff60058610 con 0x7eff6815cc40 2024-04-24T23:00:18.945 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-24T23:00:18.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.945+0000 7eff6d57d640 1 -- 172.21.15.32:0/2259823550 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7eff5406be30 msgr2=0x7eff5406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:18.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.945+0000 7eff6d57d640 1 --2- 172.21.15.32:0/2259823550 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7eff5406be30 0x7eff5406e2f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7eff6815cc10 tx=0x7eff58009190 comp rx=0 tx=0).stop 2024-04-24T23:00:18.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.945+0000 7eff6d57d640 1 -- 172.21.15.32:0/2259823550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff6815cc40 msgr2=0x7eff681f59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:18.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.945+0000 7eff6d57d640 1 --2- 172.21.15.32:0/2259823550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff6815cc40 0x7eff681f59f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7eff6000c730 tx=0x7eff6000cbf0 comp rx=0 tx=0).stop 2024-04-24T23:00:18.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.945+0000 7eff6d57d640 1 -- 172.21.15.32:0/2259823550 shutdown_connections 2024-04-24T23:00:18.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.945+0000 7eff6d57d640 1 --2- 172.21.15.32:0/2259823550 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7eff5406be30 0x7eff5406e2f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:18.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.945+0000 7eff6d57d640 1 --2- 172.21.15.32:0/2259823550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff6815cc40 0x7eff681f59f0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:18.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.945+0000 7eff6d57d640 1 --2- 172.21.15.32:0/2259823550 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7eff680d05b0 0x7eff68142520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:18.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.945+0000 7eff6d57d640 1 --2- 172.21.15.32:0/2259823550 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff680cec30 0x7eff6815a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:18.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.945+0000 7eff6d57d640 1 -- 172.21.15.32:0/2259823550 >> 172.21.15.32:0/2259823550 conn(0x7eff680d0ae0 msgr2=0x7eff680d0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:18.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.945+0000 7eff6d57d640 1 -- 172.21.15.32:0/2259823550 shutdown_connections 2024-04-24T23:00:18.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:18.945+0000 7eff6d57d640 1 -- 172.21.15.32:0/2259823550 wait complete. 2024-04-24T23:00:18.957 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:00:19.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.056+0000 7fefdac64640 1 Processor -- start 2024-04-24T23:00:19.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.056+0000 7fefdac64640 1 -- start start 2024-04-24T23:00:19.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.057+0000 7fefdac64640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fefd4162320 0x7fefd40ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:19.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.057+0000 7fefdac64640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fefd4162e10 0x7fefd415ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:19.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.057+0000 7fefdac64640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefd40cd270 con 0x7fefd40b5af0 2024-04-24T23:00:19.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.057+0000 7fefdac64640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fefd40b5660 con 0x7fefd4162e10 2024-04-24T23:00:19.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.057+0000 7fefdac64640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fefd4071610 con 0x7fefd4162320 2024-04-24T23:00:19.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.057+0000 7fefd3fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fefd40b5af0 0x7fefd4162700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58182/0 (socket says 172.21.15.32:58182) 2024-04-24T23:00:19.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.057+0000 7fefd3fff640 1 -- 172.21.15.32:0/4131465837 learned_addr learned my addr 172.21.15.32:0/4131465837 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:00:19.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.057+0000 7fefd8a00640 1 --2- 172.21.15.32:0/4131465837 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fefd4162320 0x7fefd40ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:19.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.057+0000 7fefd9201640 1 --2- 172.21.15.32:0/4131465837 >> v2:172.21.15.121:3300/0 conn(0x7fefd4162e10 0x7fefd415ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:19.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.058+0000 7fefd37fe640 1 -- 172.21.15.32:0/4131465837 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 121290891 0 0) 0x7fefd40cd270 con 0x7fefd40b5af0 2024-04-24T23:00:19.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.058+0000 7fefd37fe640 1 -- 172.21.15.32:0/4131465837 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fefbc003520 con 0x7fefd40b5af0 2024-04-24T23:00:19.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.058+0000 7fefd8a00640 1 -- 172.21.15.32:0/4131465837 >> v2:172.21.15.121:3300/0 conn(0x7fefd4162e10 msgr2=0x7fefd415ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:19.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.058+0000 7fefd8a00640 1 --2- 172.21.15.32:0/4131465837 >> v2:172.21.15.121:3300/0 conn(0x7fefd4162e10 0x7fefd415ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:19.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.058+0000 7fefd8a00640 1 -- 172.21.15.32:0/4131465837 >> v1:172.21.15.32:6789/0 conn(0x7fefd40b5af0 legacy=0x7fefd4162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:19.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.058+0000 7fefd8a00640 1 -- 172.21.15.32:0/4131465837 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefd40718f0 con 0x7fefd4162320 2024-04-24T23:00:19.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.059+0000 7fefd8a00640 1 --2- 172.21.15.32:0/4131465837 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fefd4162320 0x7fefd40ccc90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fefc40097a0 tx=0x7fefc402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e15b6d386d54f37b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:19.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.059+0000 7fefd37fe640 1 -- 172.21.15.32:0/4131465837 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc4037070 con 0x7fefd4162320 2024-04-24T23:00:19.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.059+0000 7fefd37fe640 1 -- 172.21.15.32:0/4131465837 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefc402ae50 con 0x7fefd4162320 2024-04-24T23:00:19.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.059+0000 7fefd37fe640 1 -- 172.21.15.32:0/4131465837 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc4032660 con 0x7fefd4162320 2024-04-24T23:00:19.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.059+0000 7fefdac64640 1 -- 172.21.15.32:0/4131465837 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fefd4162320 msgr2=0x7fefd40ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:19.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.059+0000 7fefdac64640 1 --2- 172.21.15.32:0/4131465837 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fefd4162320 0x7fefd40ccc90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fefc40097a0 tx=0x7fefc402a100 comp rx=0 tx=0).stop 2024-04-24T23:00:19.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.059+0000 7fefdac64640 1 -- 172.21.15.32:0/4131465837 shutdown_connections 2024-04-24T23:00:19.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.059+0000 7fefdac64640 1 --2- 172.21.15.32:0/4131465837 >> v2:172.21.15.121:3300/0 conn(0x7fefd4162e10 0x7fefd415ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:19.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.059+0000 7fefdac64640 1 --2- 172.21.15.32:0/4131465837 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fefd4162320 0x7fefd40ccc90 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:19.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.059+0000 7fefdac64640 1 -- 172.21.15.32:0/4131465837 >> 172.21.15.32:0/4131465837 conn(0x7fefd40d0ae0 msgr2=0x7fefd40d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:19.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.059+0000 7fefdac64640 1 -- 172.21.15.32:0/4131465837 shutdown_connections 2024-04-24T23:00:19.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.059+0000 7fefdac64640 1 -- 172.21.15.32:0/4131465837 wait complete. 2024-04-24T23:00:19.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.061+0000 7fefdac64640 1 Processor -- start 2024-04-24T23:00:19.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.061+0000 7fefdac64640 1 -- start start 2024-04-24T23:00:19.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.061+0000 7fefdac64640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefd40b5af0 0x7fefd40cdf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:19.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.061+0000 7fefdac64640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fefd4162320 0x7fefd4147c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:19.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.061+0000 7fefdac64640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fefd4162e10 0x7fefd414a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:19.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.062+0000 7fefdac64640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fefd4071bd0 con 0x7fefd40b5af0 2024-04-24T23:00:19.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.062+0000 7fefdac64640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fefd4067950 con 0x7fefd4162320 2024-04-24T23:00:19.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.062+0000 7fefdac64640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fefd4071d50 con 0x7fefd4162e10 2024-04-24T23:00:19.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.062+0000 7fefd8a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefd40b5af0 0x7fefd40cdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:19.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.062+0000 7fefd8a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefd40b5af0 0x7fefd40cdf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41786/0 (socket says 172.21.15.32:41786) 2024-04-24T23:00:19.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.062+0000 7fefd8a00640 1 -- 172.21.15.32:0/3957691496 learned_addr learned my addr 172.21.15.32:0/3957691496 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:19.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.062+0000 7fefd3fff640 1 --2- 172.21.15.32:0/3957691496 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fefd4162320 0x7fefd4147c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:19.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.062+0000 7fefd9201640 1 --2- 172.21.15.32:0/3957691496 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fefd4162e10 0x7fefd414a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:19.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.062+0000 7fefd8a00640 1 -- 172.21.15.32:0/3957691496 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fefd4162e10 msgr2=0x7fefd414a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:19.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.062+0000 7fefd8a00640 1 --2- 172.21.15.32:0/3957691496 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fefd4162e10 0x7fefd414a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:19.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.062+0000 7fefd8a00640 1 -- 172.21.15.32:0/3957691496 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fefd4162320 msgr2=0x7fefd4147c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:19.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.062+0000 7fefd8a00640 1 --2- 172.21.15.32:0/3957691496 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fefd4162320 0x7fefd4147c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:19.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.062+0000 7fefd8a00640 1 -- 172.21.15.32:0/3957691496 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefd40ce780 con 0x7fefd40b5af0 2024-04-24T23:00:19.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.062+0000 7fefd8a00640 1 --2- 172.21.15.32:0/3957691496 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefd40b5af0 0x7fefd40cdf00 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fefc40098c0 tx=0x7fefc40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:19.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.062+0000 7fefd17fa640 1 -- 172.21.15.32:0/3957691496 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc4042070 con 0x7fefd40b5af0 2024-04-24T23:00:19.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.062+0000 7fefd17fa640 1 -- 172.21.15.32:0/3957691496 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefc403e460 con 0x7fefd40b5af0 2024-04-24T23:00:19.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.062+0000 7fefdac64640 1 -- 172.21.15.32:0/3957691496 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefd40cea60 con 0x7fefd40b5af0 2024-04-24T23:00:19.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.062+0000 7fefd17fa640 1 -- 172.21.15.32:0/3957691496 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc4037040 con 0x7fefd40b5af0 2024-04-24T23:00:19.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.062+0000 7fefdac64640 1 -- 172.21.15.32:0/3957691496 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefd40ce440 con 0x7fefd40b5af0 2024-04-24T23:00:19.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.063+0000 7fefdac64640 1 -- 172.21.15.32:0/3957691496 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefd4068430 con 0x7fefd40b5af0 2024-04-24T23:00:19.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.064+0000 7fefd17fa640 1 -- 172.21.15.32:0/3957691496 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fefc4045070 con 0x7fefd40b5af0 2024-04-24T23:00:19.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.064+0000 7fefd17fa640 1 --2- 172.21.15.32:0/3957691496 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fefc006be30 0x7fefc006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:19.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.064+0000 7fefd17fa640 1 -- 172.21.15.32:0/3957691496 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fefc40ab230 con 0x7fefd40b5af0 2024-04-24T23:00:19.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.064+0000 7fefd3fff640 1 --2- 172.21.15.32:0/3957691496 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fefc006be30 0x7fefc006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:19.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.065+0000 7fefd3fff640 1 --2- 172.21.15.32:0/3957691496 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fefc006be30 0x7fefc006e2f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fefd40af550 tx=0x7fefc8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:19.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.068+0000 7fefd17fa640 1 -- 172.21.15.32:0/3957691496 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fefc4079d90 con 0x7fefd40b5af0 2024-04-24T23:00:19.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.287+0000 7fefdac64640 1 -- 172.21.15.32:0/3957691496 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fefd4067730 con 0x7fefd40b5af0 2024-04-24T23:00:19.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.287+0000 7fefd17fa640 1 -- 172.21.15.32:0/3957691496 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+5181 (secure 0 0 0) 0x7fefc4079690 con 0x7fefd40b5af0 2024-04-24T23:00:19.289 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:00:19.289 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1796819396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":1796819396},{"type":"v1","addr":"172.21.15.32:6835","nonce":1796819396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/298202585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":298202585},{"type":"v1","addr":"172.21.15.121:6836","nonce":298202585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4939,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/4056636026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":4056636026},{"type":"v1","addr":"172.21.15.32:6834","nonce":4056636026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T22:59:57.468765+0000","modified":"2024-04-24T22:59:59.471269+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5112},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5112":{"gid":5112,"name":"b","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6837/515107694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":515107694},{"type":"v1","addr":"172.21.15.121:6837","nonce":515107694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5112,"qdb_cluster":[5112]},"id":2}]} 2024-04-24T23:00:19.289 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 13 2024-04-24T23:00:19.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.289+0000 7fefdac64640 1 -- 172.21.15.32:0/3957691496 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fefc006be30 msgr2=0x7fefc006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:19.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.289+0000 7fefdac64640 1 --2- 172.21.15.32:0/3957691496 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fefc006be30 0x7fefc006e2f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fefd40af550 tx=0x7fefc8009040 comp rx=0 tx=0).stop 2024-04-24T23:00:19.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.289+0000 7fefdac64640 1 -- 172.21.15.32:0/3957691496 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefd40b5af0 msgr2=0x7fefd40cdf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:19.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.289+0000 7fefdac64640 1 --2- 172.21.15.32:0/3957691496 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefd40b5af0 0x7fefd40cdf00 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fefc40098c0 tx=0x7fefc40046c0 comp rx=0 tx=0).stop 2024-04-24T23:00:19.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.289+0000 7fefdac64640 1 -- 172.21.15.32:0/3957691496 shutdown_connections 2024-04-24T23:00:19.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.289+0000 7fefdac64640 1 --2- 172.21.15.32:0/3957691496 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fefc006be30 0x7fefc006e2f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:19.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.289+0000 7fefdac64640 1 --2- 172.21.15.32:0/3957691496 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fefd4162e10 0x7fefd414a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:19.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.289+0000 7fefdac64640 1 --2- 172.21.15.32:0/3957691496 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fefd4162320 0x7fefd4147c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:19.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.289+0000 7fefdac64640 1 --2- 172.21.15.32:0/3957691496 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefd40b5af0 0x7fefd40cdf00 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:19.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.289+0000 7fefdac64640 1 -- 172.21.15.32:0/3957691496 >> 172.21.15.32:0/3957691496 conn(0x7fefd40d0ae0 msgr2=0x7fefd40b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:19.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.289+0000 7fefdac64640 1 -- 172.21.15.32:0/3957691496 shutdown_connections 2024-04-24T23:00:19.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.289+0000 7fefdac64640 1 -- 172.21.15.32:0/3957691496 wait complete. 2024-04-24T23:00:19.304 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:00:19.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.404+0000 7fa1df59e640 1 Processor -- start 2024-04-24T23:00:19.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.404+0000 7fa1df59e640 1 -- start start 2024-04-24T23:00:19.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.405+0000 7fa1df59e640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa1e015c640 0x7fa1e0074210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:19.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.405+0000 7fa1df59e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa1e0074750 0x7fa1e00cf2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:19.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.405+0000 7fa1df59e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1e00ae2b0 con 0x7fa1e00cf810 2024-04-24T23:00:19.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.405+0000 7fa1df59e640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fa1e006b110 con 0x7fa1e0074750 2024-04-24T23:00:19.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.405+0000 7fa1df59e640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fa1e00cff40 con 0x7fa1e015c640 2024-04-24T23:00:19.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.405+0000 7fa1d7fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa1e0074750 0x7fa1e00cf2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:19.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.405+0000 7fa1ded9d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa1e00cf810 0x7fa1e006d2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58196/0 (socket says 172.21.15.32:58196) 2024-04-24T23:00:19.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.405+0000 7fa1ded9d640 1 -- 172.21.15.32:0/2432053830 learned_addr learned my addr 172.21.15.32:0/2432053830 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:00:19.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.405+0000 7fa1de59c640 1 --2- 172.21.15.32:0/2432053830 >> v2:172.21.15.121:3300/0 conn(0x7fa1e015c640 0x7fa1e0074210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:19.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.405+0000 7fa1d7fff640 1 -- 172.21.15.32:0/2432053830 >> v2:172.21.15.121:3300/0 conn(0x7fa1e015c640 msgr2=0x7fa1e0074210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:19.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.405+0000 7fa1d7fff640 1 --2- 172.21.15.32:0/2432053830 >> v2:172.21.15.121:3300/0 conn(0x7fa1e015c640 0x7fa1e0074210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:19.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.405+0000 7fa1d7fff640 1 -- 172.21.15.32:0/2432053830 >> v1:172.21.15.32:6789/0 conn(0x7fa1e00cf810 legacy=0x7fa1e006d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:19.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.405+0000 7fa1d7fff640 1 -- 172.21.15.32:0/2432053830 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1e00d0220 con 0x7fa1e0074750 2024-04-24T23:00:19.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.406+0000 7fa1d7fff640 1 --2- 172.21.15.32:0/2432053830 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1e0074750 0x7fa1e00cf2d0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0067f60 tx=0x7fa1c802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76b8168a04c0e3e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:19.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.406+0000 7fa1ddd9b640 1 -- 172.21.15.32:0/2432053830 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8037070 con 0x7fa1e0074750 2024-04-24T23:00:19.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.406+0000 7fa1ddd9b640 1 -- 172.21.15.32:0/2432053830 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1c8032440 con 0x7fa1e0074750 2024-04-24T23:00:19.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.406+0000 7fa1ddd9b640 1 -- 172.21.15.32:0/2432053830 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8032810 con 0x7fa1e0074750 2024-04-24T23:00:19.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.407+0000 7fa1df59e640 1 -- 172.21.15.32:0/2432053830 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1e0074750 msgr2=0x7fa1e00cf2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:19.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.407+0000 7fa1df59e640 1 --2- 172.21.15.32:0/2432053830 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1e0074750 0x7fa1e00cf2d0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0067f60 tx=0x7fa1c802a1a0 comp rx=0 tx=0).stop 2024-04-24T23:00:19.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.407+0000 7fa1df59e640 1 -- 172.21.15.32:0/2432053830 shutdown_connections 2024-04-24T23:00:19.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.407+0000 7fa1df59e640 1 --2- 172.21.15.32:0/2432053830 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1e0074750 0x7fa1e00cf2d0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:19.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.407+0000 7fa1df59e640 1 --2- 172.21.15.32:0/2432053830 >> v2:172.21.15.121:3300/0 conn(0x7fa1e015c640 0x7fa1e0074210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:19.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.407+0000 7fa1df59e640 1 -- 172.21.15.32:0/2432053830 >> 172.21.15.32:0/2432053830 conn(0x7fa1e00d10f0 msgr2=0x7fa1e00d1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:19.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.407+0000 7fa1df59e640 1 -- 172.21.15.32:0/2432053830 shutdown_connections 2024-04-24T23:00:19.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.407+0000 7fa1df59e640 1 -- 172.21.15.32:0/2432053830 wait complete. 2024-04-24T23:00:19.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.408+0000 7fa1df59e640 1 Processor -- start 2024-04-24T23:00:19.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.408+0000 7fa1df59e640 1 -- start start 2024-04-24T23:00:19.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.409+0000 7fa1df59e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1e0074750 0x7fa1e014f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:19.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.409+0000 7fa1df59e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa1e00cf810 0x7fa1e014c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:19.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.409+0000 7fa1df59e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1e015c640 0x7fa1e015a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:19.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.409+0000 7fa1df59e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e00cfc40 con 0x7fa1e0074750 2024-04-24T23:00:19.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.409+0000 7fa1df59e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e006b360 con 0x7fa1e015c640 2024-04-24T23:00:19.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.409+0000 7fa1df59e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa1e015ca20 con 0x7fa1e00cf810 2024-04-24T23:00:19.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.409+0000 7fa1de59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1e0074750 0x7fa1e014f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:19.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.409+0000 7fa1de59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1e0074750 0x7fa1e014f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41800/0 (socket says 172.21.15.32:41800) 2024-04-24T23:00:19.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.409+0000 7fa1de59c640 1 -- 172.21.15.32:0/2977155002 learned_addr learned my addr 172.21.15.32:0/2977155002 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:19.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.409+0000 7fa1d7fff640 1 --2- 172.21.15.32:0/2977155002 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa1e00cf810 0x7fa1e014c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:19.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.409+0000 7fa1ded9d640 1 --2- 172.21.15.32:0/2977155002 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1e015c640 0x7fa1e015a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:19.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.409+0000 7fa1de59c640 1 -- 172.21.15.32:0/2977155002 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa1e00cf810 msgr2=0x7fa1e014c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:19.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.409+0000 7fa1de59c640 1 --2- 172.21.15.32:0/2977155002 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa1e00cf810 0x7fa1e014c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:19.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.409+0000 7fa1de59c640 1 -- 172.21.15.32:0/2977155002 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1e015c640 msgr2=0x7fa1e015a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:19.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.409+0000 7fa1de59c640 1 --2- 172.21.15.32:0/2977155002 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1e015c640 0x7fa1e015a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:19.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.409+0000 7fa1de59c640 1 -- 172.21.15.32:0/2977155002 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1e014f8f0 con 0x7fa1e0074750 2024-04-24T23:00:19.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.410+0000 7fa1de59c640 1 --2- 172.21.15.32:0/2977155002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1e0074750 0x7fa1e014f3b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa1d000d5f0 tx=0x7fa1d000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:19.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.410+0000 7fa1d6ffd640 1 -- 172.21.15.32:0/2977155002 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d00125f0 con 0x7fa1e0074750 2024-04-24T23:00:19.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.410+0000 7fa1d6ffd640 1 -- 172.21.15.32:0/2977155002 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1d0019070 con 0x7fa1e0074750 2024-04-24T23:00:19.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.410+0000 7fa1d6ffd640 1 -- 172.21.15.32:0/2977155002 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d00129d0 con 0x7fa1e0074750 2024-04-24T23:00:19.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.410+0000 7fa1df59e640 1 -- 172.21.15.32:0/2977155002 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1e014c630 con 0x7fa1e0074750 2024-04-24T23:00:19.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.410+0000 7fa1df59e640 1 -- 172.21.15.32:0/2977155002 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1e00c53f0 con 0x7fa1e0074750 2024-04-24T23:00:19.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.411+0000 7fa1df59e640 1 -- 172.21.15.32:0/2977155002 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1e00682b0 con 0x7fa1e0074750 2024-04-24T23:00:19.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.411+0000 7fa1d6ffd640 1 -- 172.21.15.32:0/2977155002 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa1d0012b70 con 0x7fa1e0074750 2024-04-24T23:00:19.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.412+0000 7fa1d6ffd640 1 --2- 172.21.15.32:0/2977155002 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa1c006be30 0x7fa1c006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:19.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.412+0000 7fa1d6ffd640 1 -- 172.21.15.32:0/2977155002 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fa1d008b0f0 con 0x7fa1e0074750 2024-04-24T23:00:19.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.412+0000 7fa1d7fff640 1 --2- 172.21.15.32:0/2977155002 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa1c006be30 0x7fa1c006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:19.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.412+0000 7fa1d7fff640 1 --2- 172.21.15.32:0/2977155002 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa1c006be30 0x7fa1c006e2f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8034000 tx=0x7fa1c802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:19.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.416+0000 7fa1d6ffd640 1 -- 172.21.15.32:0/2977155002 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa1e014c630 con 0x7fa1e0074750 2024-04-24T23:00:19.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.603+0000 7fa1df59e640 1 -- 172.21.15.32:0/2977155002 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa1e00c5e70 con 0x7fa1e0074750 2024-04-24T23:00:19.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.604+0000 7fa1d6ffd640 1 -- 172.21.15.32:0/2977155002 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16621 (secure 0 0 0) 0x7fa1e014f8f0 con 0x7fa1e0074750 2024-04-24T23:00:19.605 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:00:19.606 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":34,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:00:12.312472+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:59:56.068812+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-24T22:59:56.447402+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:00:19.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.606+0000 7fa1df59e640 1 -- 172.21.15.32:0/2977155002 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa1c006be30 msgr2=0x7fa1c006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:19.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.606+0000 7fa1df59e640 1 --2- 172.21.15.32:0/2977155002 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa1c006be30 0x7fa1c006e2f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8034000 tx=0x7fa1c802e000 comp rx=0 tx=0).stop 2024-04-24T23:00:19.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.606+0000 7fa1df59e640 1 -- 172.21.15.32:0/2977155002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1e0074750 msgr2=0x7fa1e014f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:19.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.606+0000 7fa1df59e640 1 --2- 172.21.15.32:0/2977155002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1e0074750 0x7fa1e014f3b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa1d000d5f0 tx=0x7fa1d000dab0 comp rx=0 tx=0).stop 2024-04-24T23:00:19.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.606+0000 7fa1df59e640 1 -- 172.21.15.32:0/2977155002 shutdown_connections 2024-04-24T23:00:19.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.606+0000 7fa1df59e640 1 --2- 172.21.15.32:0/2977155002 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa1c006be30 0x7fa1c006e2f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:19.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.606+0000 7fa1df59e640 1 --2- 172.21.15.32:0/2977155002 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1e015c640 0x7fa1e015a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:19.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.606+0000 7fa1df59e640 1 --2- 172.21.15.32:0/2977155002 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa1e00cf810 0x7fa1e014c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:19.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.606+0000 7fa1df59e640 1 --2- 172.21.15.32:0/2977155002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1e0074750 0x7fa1e014f3b0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:19.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.606+0000 7fa1df59e640 1 -- 172.21.15.32:0/2977155002 >> 172.21.15.32:0/2977155002 conn(0x7fa1e00d10f0 msgr2=0x7fa1e006db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:19.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.606+0000 7fa1df59e640 1 -- 172.21.15.32:0/2977155002 shutdown_connections 2024-04-24T23:00:19.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.606+0000 7fa1df59e640 1 -- 172.21.15.32:0/2977155002 wait complete. 2024-04-24T23:00:19.619 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-24T23:00:19.619 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:00:19.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.718+0000 7f22efed2640 1 Processor -- start 2024-04-24T23:00:19.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.718+0000 7f22efed2640 1 -- start start 2024-04-24T23:00:19.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.719+0000 7f22efed2640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f22e815e9b0 0x7f22e80d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:19.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.719+0000 7f22efed2640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f22e806b820 0x7f22e80b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:19.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.719+0000 7f22efed2640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22e80d3170 con 0x7f22e80d4210 2024-04-24T23:00:19.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.719+0000 7f22efed2640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f22e80d3450 con 0x7f22e815e9b0 2024-04-24T23:00:19.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.719+0000 7f22efed2640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f22e80d3730 con 0x7f22e806b820 2024-04-24T23:00:19.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.719+0000 7f22edc6e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f22e815e9b0 0x7f22e80d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:19.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.719+0000 7f22edc6e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f22e815e9b0 0x7f22e80d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41808/0 (socket says 172.21.15.32:41808) 2024-04-24T23:00:19.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.719+0000 7f22edc6e640 1 -- 172.21.15.32:0/3418717725 learned_addr learned my addr 172.21.15.32:0/3418717725 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:19.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.720+0000 7f22ee46f640 1 --2- 172.21.15.32:0/3418717725 >> v2:172.21.15.121:3300/0 conn(0x7f22e806b820 0x7f22e80b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:19.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.720+0000 7f22edc6e640 1 -- 172.21.15.32:0/3418717725 >> v2:172.21.15.121:3300/0 conn(0x7f22e806b820 msgr2=0x7f22e80b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:19.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.720+0000 7f22edc6e640 1 --2- 172.21.15.32:0/3418717725 >> v2:172.21.15.121:3300/0 conn(0x7f22e806b820 0x7f22e80b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:19.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.720+0000 7f22edc6e640 1 -- 172.21.15.32:0/3418717725 >> v1:172.21.15.121:6789/0 conn(0x7f22e80d4210 legacy=0x7f22e806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:19.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.720+0000 7f22edc6e640 1 -- 172.21.15.32:0/3418717725 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22e80d3a10 con 0x7f22e815e9b0 2024-04-24T23:00:19.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.720+0000 7f22edc6e640 1 --2- 172.21.15.32:0/3418717725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22e815e9b0 0x7f22e80d2b90 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f22e81622f0 tx=0x7f22e002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9203e1373160a037 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:19.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.720+0000 7f22ecc6c640 1 -- 172.21.15.32:0/3418717725 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e0037070 con 0x7f22e815e9b0 2024-04-24T23:00:19.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.721+0000 7f22ecc6c640 1 -- 172.21.15.32:0/3418717725 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22e0032440 con 0x7f22e815e9b0 2024-04-24T23:00:19.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.721+0000 7f22ecc6c640 1 -- 172.21.15.32:0/3418717725 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e0032810 con 0x7f22e815e9b0 2024-04-24T23:00:19.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.721+0000 7f22efed2640 1 -- 172.21.15.32:0/3418717725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22e815e9b0 msgr2=0x7f22e80d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:19.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.721+0000 7f22efed2640 1 --2- 172.21.15.32:0/3418717725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22e815e9b0 0x7f22e80d2b90 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f22e81622f0 tx=0x7f22e002a1a0 comp rx=0 tx=0).stop 2024-04-24T23:00:19.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.721+0000 7f22efed2640 1 -- 172.21.15.32:0/3418717725 shutdown_connections 2024-04-24T23:00:19.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.721+0000 7f22efed2640 1 --2- 172.21.15.32:0/3418717725 >> v2:172.21.15.121:3300/0 conn(0x7f22e806b820 0x7f22e80b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:19.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.721+0000 7f22efed2640 1 --2- 172.21.15.32:0/3418717725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22e815e9b0 0x7f22e80d2b90 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:19.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.721+0000 7f22efed2640 1 -- 172.21.15.32:0/3418717725 >> 172.21.15.32:0/3418717725 conn(0x7f22e80d11d0 msgr2=0x7f22e80d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:19.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.721+0000 7f22efed2640 1 -- 172.21.15.32:0/3418717725 shutdown_connections 2024-04-24T23:00:19.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.721+0000 7f22efed2640 1 -- 172.21.15.32:0/3418717725 wait complete. 2024-04-24T23:00:19.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.722+0000 7f22efed2640 1 Processor -- start 2024-04-24T23:00:19.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.722+0000 7f22efed2640 1 -- start start 2024-04-24T23:00:19.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.723+0000 7f22efed2640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22e806b820 0x7f22e8157620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:19.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.723+0000 7f22efed2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22e80d4210 0x7f22e81f7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:19.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.723+0000 7f22efed2640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22e815e9b0 0x7f22e81f4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:19.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.723+0000 7f22efed2640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f22e80ba0e0 con 0x7f22e80d4210 2024-04-24T23:00:19.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.723+0000 7f22efed2640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f22e80d2400 con 0x7f22e815e9b0 2024-04-24T23:00:19.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.723+0000 7f22efed2640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f22e80ba260 con 0x7f22e806b820 2024-04-24T23:00:19.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.723+0000 7f22ed46d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22e80d4210 0x7f22e81f7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:19.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.723+0000 7f22ed46d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22e80d4210 0x7f22e81f7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41818/0 (socket says 172.21.15.32:41818) 2024-04-24T23:00:19.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.723+0000 7f22ed46d640 1 -- 172.21.15.32:0/2153519906 learned_addr learned my addr 172.21.15.32:0/2153519906 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:19.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.723+0000 7f22ee46f640 1 --2- 172.21.15.32:0/2153519906 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22e815e9b0 0x7f22e81f4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:19.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.723+0000 7f22edc6e640 1 --2- 172.21.15.32:0/2153519906 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22e806b820 0x7f22e8157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:19.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.723+0000 7f22ed46d640 1 -- 172.21.15.32:0/2153519906 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22e806b820 msgr2=0x7f22e8157620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:19.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.723+0000 7f22ed46d640 1 --2- 172.21.15.32:0/2153519906 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22e806b820 0x7f22e8157620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:19.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.723+0000 7f22ed46d640 1 -- 172.21.15.32:0/2153519906 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22e815e9b0 msgr2=0x7f22e81f4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:19.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.723+0000 7f22ed46d640 1 --2- 172.21.15.32:0/2153519906 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22e815e9b0 0x7f22e81f4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:19.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.724+0000 7f22ed46d640 1 -- 172.21.15.32:0/2153519906 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22e81f82b0 con 0x7f22e80d4210 2024-04-24T23:00:19.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.724+0000 7f22ed46d640 1 --2- 172.21.15.32:0/2153519906 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22e80d4210 0x7f22e81f7a80 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f22dc00c430 tx=0x7f22dc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:19.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.724+0000 7f22e67fc640 1 -- 172.21.15.32:0/2153519906 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22dc011790 con 0x7f22e80d4210 2024-04-24T23:00:19.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.724+0000 7f22e67fc640 1 -- 172.21.15.32:0/2153519906 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22dc017070 con 0x7f22e80d4210 2024-04-24T23:00:19.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.724+0000 7f22efed2640 1 -- 172.21.15.32:0/2153519906 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22e81f8590 con 0x7f22e80d4210 2024-04-24T23:00:19.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.724+0000 7f22e67fc640 1 -- 172.21.15.32:0/2153519906 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22dc011bb0 con 0x7f22e80d4210 2024-04-24T23:00:19.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.724+0000 7f22efed2640 1 -- 172.21.15.32:0/2153519906 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22e80cd270 con 0x7f22e80d4210 2024-04-24T23:00:19.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.725+0000 7f22efed2640 1 -- 172.21.15.32:0/2153519906 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22e8068430 con 0x7f22e80d4210 2024-04-24T23:00:19.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.725+0000 7f22e67fc640 1 -- 172.21.15.32:0/2153519906 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f22dc01c460 con 0x7f22e80d4210 2024-04-24T23:00:19.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.726+0000 7f22e67fc640 1 --2- 172.21.15.32:0/2153519906 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f22d406be30 0x7f22d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:19.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.726+0000 7f22e67fc640 1 -- 172.21.15.32:0/2153519906 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f22dc08ab80 con 0x7f22e80d4210 2024-04-24T23:00:19.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.726+0000 7f22edc6e640 1 --2- 172.21.15.32:0/2153519906 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f22d406be30 0x7f22d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:19.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.726+0000 7f22edc6e640 1 --2- 172.21.15.32:0/2153519906 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f22d406be30 0x7f22d406e2f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f22e0034000 tx=0x7f22e002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:19.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.730+0000 7f22e67fc640 1 -- 172.21.15.32:0/2153519906 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f22dc058ee0 con 0x7f22e80d4210 2024-04-24T23:00:19.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.950+0000 7f22efed2640 1 -- 172.21.15.32:0/2153519906 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f22e80d5530 con 0x7f22e80d4210 2024-04-24T23:00:19.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.951+0000 7f22e67fc640 1 -- 172.21.15.32:0/2153519906 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+5181 (secure 0 0 0) 0x7f22dc05d1e0 con 0x7f22e80d4210 2024-04-24T23:00:19.953 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:00:19.953 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1796819396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":1796819396},{"type":"v1","addr":"172.21.15.32:6835","nonce":1796819396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/298202585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":298202585},{"type":"v1","addr":"172.21.15.121:6836","nonce":298202585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4939,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/4056636026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":4056636026},{"type":"v1","addr":"172.21.15.32:6834","nonce":4056636026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T22:59:57.468765+0000","modified":"2024-04-24T22:59:59.471269+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5112},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5112":{"gid":5112,"name":"b","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6837/515107694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":515107694},{"type":"v1","addr":"172.21.15.121:6837","nonce":515107694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5112,"qdb_cluster":[5112]},"id":2}]} 2024-04-24T23:00:19.953 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 13 2024-04-24T23:00:19.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.953+0000 7f22efed2640 1 -- 172.21.15.32:0/2153519906 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f22d406be30 msgr2=0x7f22d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:19.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.953+0000 7f22efed2640 1 --2- 172.21.15.32:0/2153519906 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f22d406be30 0x7f22d406e2f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f22e0034000 tx=0x7f22e002e000 comp rx=0 tx=0).stop 2024-04-24T23:00:19.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.953+0000 7f22efed2640 1 -- 172.21.15.32:0/2153519906 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22e80d4210 msgr2=0x7f22e81f7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:19.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.953+0000 7f22efed2640 1 --2- 172.21.15.32:0/2153519906 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22e80d4210 0x7f22e81f7a80 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f22dc00c430 tx=0x7f22dc00c8f0 comp rx=0 tx=0).stop 2024-04-24T23:00:19.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.953+0000 7f22efed2640 1 -- 172.21.15.32:0/2153519906 shutdown_connections 2024-04-24T23:00:19.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.953+0000 7f22efed2640 1 --2- 172.21.15.32:0/2153519906 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f22d406be30 0x7f22d406e2f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:19.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.953+0000 7f22efed2640 1 --2- 172.21.15.32:0/2153519906 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22e80d4210 0x7f22e81f7a80 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:19.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.953+0000 7f22efed2640 1 --2- 172.21.15.32:0/2153519906 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22e815e9b0 0x7f22e81f4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:19.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.953+0000 7f22efed2640 1 --2- 172.21.15.32:0/2153519906 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22e806b820 0x7f22e8157620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:19.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.953+0000 7f22efed2640 1 -- 172.21.15.32:0/2153519906 >> 172.21.15.32:0/2153519906 conn(0x7f22e80d11d0 msgr2=0x7f22e80c9d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:19.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.953+0000 7f22efed2640 1 -- 172.21.15.32:0/2153519906 shutdown_connections 2024-04-24T23:00:19.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:19.953+0000 7f22efed2640 1 -- 172.21.15.32:0/2153519906 wait complete. 2024-04-24T23:00:19.965 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:00:20.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.065+0000 7f829b1b7640 1 Processor -- start 2024-04-24T23:00:20.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.065+0000 7f829b1b7640 1 -- start start 2024-04-24T23:00:20.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.065+0000 7f829b1b7640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f829415e9b0 0x7f82940d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:20.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.065+0000 7f829b1b7640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f829406b820 0x7f82940b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:20.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.065+0000 7f829b1b7640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82940d3170 con 0x7f82940d4210 2024-04-24T23:00:20.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.065+0000 7f829b1b7640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f82940d3450 con 0x7f829415e9b0 2024-04-24T23:00:20.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.065+0000 7f829b1b7640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f82940d3730 con 0x7f829406b820 2024-04-24T23:00:20.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.066+0000 7f8293fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f82940d4210 0x7f829406b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33426/0 (socket says 172.21.15.32:33426) 2024-04-24T23:00:20.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.066+0000 7f8293fff640 1 -- 172.21.15.32:0/3547559325 learned_addr learned my addr 172.21.15.32:0/3547559325 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:00:20.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.066+0000 7f8298f53640 1 --2- 172.21.15.32:0/3547559325 >> v2:172.21.15.121:3300/0 conn(0x7f829415e9b0 0x7f82940d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:20.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.066+0000 7f8299754640 1 --2- 172.21.15.32:0/3547559325 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f829406b820 0x7f82940b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:20.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.066+0000 7f82937fe640 1 -- 172.21.15.32:0/3547559325 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2628210527 0 0) 0x7f82940d3170 con 0x7f82940d4210 2024-04-24T23:00:20.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.066+0000 7f82937fe640 1 -- 172.21.15.32:0/3547559325 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f827c003520 con 0x7f82940d4210 2024-04-24T23:00:20.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.066+0000 7f8298f53640 1 -- 172.21.15.32:0/3547559325 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f829406b820 msgr2=0x7f82940b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:20.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.066+0000 7f8298f53640 1 --2- 172.21.15.32:0/3547559325 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f829406b820 0x7f82940b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:20.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.066+0000 7f8298f53640 1 -- 172.21.15.32:0/3547559325 >> v1:172.21.15.32:6789/0 conn(0x7f82940d4210 legacy=0x7f829406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:20.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.066+0000 7f8298f53640 1 -- 172.21.15.32:0/3547559325 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82940d3a10 con 0x7f829415e9b0 2024-04-24T23:00:20.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.067+0000 7f8298f53640 1 --2- 172.21.15.32:0/3547559325 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f829415e9b0 0x7f82940d2b90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f82840097a0 tx=0x7f828402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50eef085aa2730b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:20.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.067+0000 7f82937fe640 1 -- 172.21.15.32:0/3547559325 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8284037070 con 0x7f829415e9b0 2024-04-24T23:00:20.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.067+0000 7f82937fe640 1 -- 172.21.15.32:0/3547559325 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8284032440 con 0x7f829415e9b0 2024-04-24T23:00:20.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.067+0000 7f82937fe640 1 -- 172.21.15.32:0/3547559325 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82840327f0 con 0x7f829415e9b0 2024-04-24T23:00:20.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.067+0000 7f829b1b7640 1 -- 172.21.15.32:0/3547559325 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f829415e9b0 msgr2=0x7f82940d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:20.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.067+0000 7f829b1b7640 1 --2- 172.21.15.32:0/3547559325 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f829415e9b0 0x7f82940d2b90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f82840097a0 tx=0x7f828402a180 comp rx=0 tx=0).stop 2024-04-24T23:00:20.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.067+0000 7f829b1b7640 1 -- 172.21.15.32:0/3547559325 shutdown_connections 2024-04-24T23:00:20.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.067+0000 7f829b1b7640 1 --2- 172.21.15.32:0/3547559325 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f829406b820 0x7f82940b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:20.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.067+0000 7f829b1b7640 1 --2- 172.21.15.32:0/3547559325 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f829415e9b0 0x7f82940d2b90 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:20.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.068+0000 7f829b1b7640 1 -- 172.21.15.32:0/3547559325 >> 172.21.15.32:0/3547559325 conn(0x7f82940d11d0 msgr2=0x7f82940d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:20.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.068+0000 7f829b1b7640 1 -- 172.21.15.32:0/3547559325 shutdown_connections 2024-04-24T23:00:20.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.068+0000 7f829b1b7640 1 -- 172.21.15.32:0/3547559325 wait complete. 2024-04-24T23:00:20.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.069+0000 7f829b1b7640 1 Processor -- start 2024-04-24T23:00:20.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.069+0000 7f829b1b7640 1 -- start start 2024-04-24T23:00:20.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.069+0000 7f829b1b7640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f829406b820 0x7f82940cdac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:20.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.070+0000 7f829b1b7640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82940d4210 0x7f82941f4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:20.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.070+0000 7f829b1b7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f829415e9b0 0x7f82940cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:20.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.070+0000 7f829b1b7640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f82940ba0e0 con 0x7f829415e9b0 2024-04-24T23:00:20.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.070+0000 7f829b1b7640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f82940d2400 con 0x7f829406b820 2024-04-24T23:00:20.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.070+0000 7f829b1b7640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f82940ba260 con 0x7f82940d4210 2024-04-24T23:00:20.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.070+0000 7f8299754640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f829415e9b0 0x7f82940cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:20.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.070+0000 7f8299754640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f829415e9b0 0x7f82940cd1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48040/0 (socket says 172.21.15.32:48040) 2024-04-24T23:00:20.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.070+0000 7f8299754640 1 -- 172.21.15.32:0/1154762254 learned_addr learned my addr 172.21.15.32:0/1154762254 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:20.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.070+0000 7f8298f53640 1 --2- 172.21.15.32:0/1154762254 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f829406b820 0x7f82940cdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:20.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.070+0000 7f8293fff640 1 --2- 172.21.15.32:0/1154762254 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82940d4210 0x7f82941f4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:20.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.070+0000 7f8299754640 1 -- 172.21.15.32:0/1154762254 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82940d4210 msgr2=0x7f82941f4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:20.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.070+0000 7f8299754640 1 --2- 172.21.15.32:0/1154762254 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82940d4210 0x7f82941f4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:20.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.070+0000 7f8299754640 1 -- 172.21.15.32:0/1154762254 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f829406b820 msgr2=0x7f82940cdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:20.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.070+0000 7f8299754640 1 --2- 172.21.15.32:0/1154762254 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f829406b820 0x7f82940cdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:20.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.070+0000 7f8299754640 1 -- 172.21.15.32:0/1154762254 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82940cd6f0 con 0x7f829415e9b0 2024-04-24T23:00:20.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.070+0000 7f8299754640 1 --2- 172.21.15.32:0/1154762254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f829415e9b0 0x7f82940cd1b0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f828c009cb0 tx=0x7f828c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:20.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.070+0000 7f82917fa640 1 -- 172.21.15.32:0/1154762254 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f828c018020 con 0x7f829415e9b0 2024-04-24T23:00:20.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.070+0000 7f82917fa640 1 -- 172.21.15.32:0/1154762254 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f828c014480 con 0x7f829415e9b0 2024-04-24T23:00:20.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.070+0000 7f829b1b7640 1 -- 172.21.15.32:0/1154762254 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82941f8450 con 0x7f829415e9b0 2024-04-24T23:00:20.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.070+0000 7f82917fa640 1 -- 172.21.15.32:0/1154762254 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f828c014870 con 0x7f829415e9b0 2024-04-24T23:00:20.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.070+0000 7f829b1b7640 1 -- 172.21.15.32:0/1154762254 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82941f7970 con 0x7f829415e9b0 2024-04-24T23:00:20.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.072+0000 7f829b1b7640 1 -- 172.21.15.32:0/1154762254 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8294068430 con 0x7f829415e9b0 2024-04-24T23:00:20.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.072+0000 7f82917fa640 1 -- 172.21.15.32:0/1154762254 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f828c005d10 con 0x7f829415e9b0 2024-04-24T23:00:20.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.072+0000 7f82917fa640 1 --2- 172.21.15.32:0/1154762254 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f828006be30 0x7f828006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:20.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.072+0000 7f82917fa640 1 -- 172.21.15.32:0/1154762254 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f828c08b990 con 0x7f829415e9b0 2024-04-24T23:00:20.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.072+0000 7f8298f53640 1 --2- 172.21.15.32:0/1154762254 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f828006be30 0x7f828006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:20.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.073+0000 7f8298f53640 1 --2- 172.21.15.32:0/1154762254 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f828006be30 0x7f828006e2f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8284034000 tx=0x7f828402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:20.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.077+0000 7f82917fa640 1 -- 172.21.15.32:0/1154762254 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f828c022030 con 0x7f829415e9b0 2024-04-24T23:00:20.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.300+0000 7f829b1b7640 1 -- 172.21.15.32:0/1154762254 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82940ca7d0 con 0x7f829415e9b0 2024-04-24T23:00:20.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.301+0000 7f82917fa640 1 -- 172.21.15.32:0/1154762254 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+5181 (secure 0 0 0) 0x7f828c010040 con 0x7f829415e9b0 2024-04-24T23:00:20.302 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:00:20.302 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1796819396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":1796819396},{"type":"v1","addr":"172.21.15.32:6835","nonce":1796819396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/298202585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":298202585},{"type":"v1","addr":"172.21.15.121:6836","nonce":298202585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4939,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/4056636026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":4056636026},{"type":"v1","addr":"172.21.15.32:6834","nonce":4056636026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T22:59:57.468765+0000","modified":"2024-04-24T22:59:59.471269+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5112},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5112":{"gid":5112,"name":"b","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6837/515107694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":515107694},{"type":"v1","addr":"172.21.15.121:6837","nonce":515107694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5112,"qdb_cluster":[5112]},"id":2}]} 2024-04-24T23:00:20.303 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 13 2024-04-24T23:00:20.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.303+0000 7f829b1b7640 1 -- 172.21.15.32:0/1154762254 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f828006be30 msgr2=0x7f828006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:20.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.303+0000 7f829b1b7640 1 --2- 172.21.15.32:0/1154762254 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f828006be30 0x7f828006e2f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8284034000 tx=0x7f828402e000 comp rx=0 tx=0).stop 2024-04-24T23:00:20.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.303+0000 7f829b1b7640 1 -- 172.21.15.32:0/1154762254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f829415e9b0 msgr2=0x7f82940cd1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:20.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.303+0000 7f829b1b7640 1 --2- 172.21.15.32:0/1154762254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f829415e9b0 0x7f82940cd1b0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f828c009cb0 tx=0x7f828c00c490 comp rx=0 tx=0).stop 2024-04-24T23:00:20.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.303+0000 7f829b1b7640 1 -- 172.21.15.32:0/1154762254 shutdown_connections 2024-04-24T23:00:20.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.303+0000 7f829b1b7640 1 --2- 172.21.15.32:0/1154762254 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f828006be30 0x7f828006e2f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:20.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.303+0000 7f829b1b7640 1 --2- 172.21.15.32:0/1154762254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f829415e9b0 0x7f82940cd1b0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:20.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.303+0000 7f829b1b7640 1 --2- 172.21.15.32:0/1154762254 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82940d4210 0x7f82941f4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:20.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.303+0000 7f829b1b7640 1 --2- 172.21.15.32:0/1154762254 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f829406b820 0x7f82940cdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:20.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.303+0000 7f829b1b7640 1 -- 172.21.15.32:0/1154762254 >> 172.21.15.32:0/1154762254 conn(0x7f82940d11d0 msgr2=0x7f829415c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:20.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.303+0000 7f829b1b7640 1 -- 172.21.15.32:0/1154762254 shutdown_connections 2024-04-24T23:00:20.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.303+0000 7f829b1b7640 1 -- 172.21.15.32:0/1154762254 wait complete. 2024-04-24T23:00:20.315 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:00:20.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.415+0000 7fe2f939e640 1 Processor -- start 2024-04-24T23:00:20.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.415+0000 7fe2f939e640 1 -- start start 2024-04-24T23:00:20.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.416+0000 7fe2f939e640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe2f415c2e0 0x7fe2f40d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:20.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.416+0000 7fe2f939e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe2f40b3ee0 0x7fe2f40b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:20.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.416+0000 7fe2f939e640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2f415f670 con 0x7fe2f415bf00 2024-04-24T23:00:20.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.416+0000 7fe2f939e640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fe2f40b4870 con 0x7fe2f40b3ee0 2024-04-24T23:00:20.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.416+0000 7fe2f939e640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fe2f40d2140 con 0x7fe2f415c2e0 2024-04-24T23:00:20.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.416+0000 7fe2f8b9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe2f40b3ee0 0x7fe2f40b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:20.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.416+0000 7fe2f8b9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe2f40b3ee0 0x7fe2f40b4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48052/0 (socket says 172.21.15.32:48052) 2024-04-24T23:00:20.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.416+0000 7fe2f8b9d640 1 -- 172.21.15.32:0/2855897631 learned_addr learned my addr 172.21.15.32:0/2855897631 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:20.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.416+0000 7fe2f37fe640 1 --2- 172.21.15.32:0/2855897631 >> v2:172.21.15.121:3300/0 conn(0x7fe2f415c2e0 0x7fe2f40d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:20.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.416+0000 7fe2f8b9d640 1 -- 172.21.15.32:0/2855897631 >> v2:172.21.15.121:3300/0 conn(0x7fe2f415c2e0 msgr2=0x7fe2f40d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:20.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.416+0000 7fe2f8b9d640 1 --2- 172.21.15.32:0/2855897631 >> v2:172.21.15.121:3300/0 conn(0x7fe2f415c2e0 0x7fe2f40d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:20.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.416+0000 7fe2f8b9d640 1 -- 172.21.15.32:0/2855897631 >> v1:172.21.15.121:6789/0 conn(0x7fe2f415bf00 legacy=0x7fe2f415ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:20.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.416+0000 7fe2f8b9d640 1 -- 172.21.15.32:0/2855897631 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2f415f950 con 0x7fe2f40b3ee0 2024-04-24T23:00:20.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.417+0000 7fe2f8b9d640 1 --2- 172.21.15.32:0/2855897631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2f40b3ee0 0x7fe2f40b4330 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe2f40b4b40 tx=0x7fe2ec02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=908bdff85c0d4725 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:20.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.417+0000 7fe2f2ffd640 1 -- 172.21.15.32:0/2855897631 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec036070 con 0x7fe2f40b3ee0 2024-04-24T23:00:20.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.417+0000 7fe2f2ffd640 1 -- 172.21.15.32:0/2855897631 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2ec031600 con 0x7fe2f40b3ee0 2024-04-24T23:00:20.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.417+0000 7fe2f2ffd640 1 -- 172.21.15.32:0/2855897631 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec0319d0 con 0x7fe2f40b3ee0 2024-04-24T23:00:20.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.418+0000 7fe2f939e640 1 -- 172.21.15.32:0/2855897631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2f40b3ee0 msgr2=0x7fe2f40b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:20.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.418+0000 7fe2f939e640 1 --2- 172.21.15.32:0/2855897631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2f40b3ee0 0x7fe2f40b4330 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe2f40b4b40 tx=0x7fe2ec02a6f0 comp rx=0 tx=0).stop 2024-04-24T23:00:20.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.418+0000 7fe2f939e640 1 -- 172.21.15.32:0/2855897631 shutdown_connections 2024-04-24T23:00:20.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.418+0000 7fe2f939e640 1 --2- 172.21.15.32:0/2855897631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2f40b3ee0 0x7fe2f40b4330 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:20.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.418+0000 7fe2f939e640 1 --2- 172.21.15.32:0/2855897631 >> v2:172.21.15.121:3300/0 conn(0x7fe2f415c2e0 0x7fe2f40d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:20.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.418+0000 7fe2f939e640 1 -- 172.21.15.32:0/2855897631 >> 172.21.15.32:0/2855897631 conn(0x7fe2f40743f0 msgr2=0x7fe2f406d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:20.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.418+0000 7fe2f939e640 1 -- 172.21.15.32:0/2855897631 shutdown_connections 2024-04-24T23:00:20.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.418+0000 7fe2f939e640 1 -- 172.21.15.32:0/2855897631 wait complete. 2024-04-24T23:00:20.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.419+0000 7fe2f939e640 1 Processor -- start 2024-04-24T23:00:20.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.419+0000 7fe2f939e640 1 -- start start 2024-04-24T23:00:20.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.420+0000 7fe2f939e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2f40b3ee0 0x7fe2f413bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:20.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.420+0000 7fe2f939e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2f415bf00 0x7fe2f40d1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:20.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.420+0000 7fe2f939e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe2f415c2e0 0x7fe2f4162930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:20.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.420+0000 7fe2f939e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f415f2c0 con 0x7fe2f40b3ee0 2024-04-24T23:00:20.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.420+0000 7fe2f939e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f40d2670 con 0x7fe2f415bf00 2024-04-24T23:00:20.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.420+0000 7fe2f939e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe2f40d13e0 con 0x7fe2f415c2e0 2024-04-24T23:00:20.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.420+0000 7fe2f3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2f40b3ee0 0x7fe2f413bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:20.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.420+0000 7fe2f3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2f40b3ee0 0x7fe2f413bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48060/0 (socket says 172.21.15.32:48060) 2024-04-24T23:00:20.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.420+0000 7fe2f3fff640 1 -- 172.21.15.32:0/2874513938 learned_addr learned my addr 172.21.15.32:0/2874513938 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:20.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.420+0000 7fe2f37fe640 1 --2- 172.21.15.32:0/2874513938 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2f415bf00 0x7fe2f40d1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:20.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.420+0000 7fe2f8b9d640 1 --2- 172.21.15.32:0/2874513938 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe2f415c2e0 0x7fe2f4162930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:20.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.420+0000 7fe2f3fff640 1 -- 172.21.15.32:0/2874513938 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe2f415c2e0 msgr2=0x7fe2f4162930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:20.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.420+0000 7fe2f3fff640 1 --2- 172.21.15.32:0/2874513938 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe2f415c2e0 0x7fe2f4162930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:20.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.420+0000 7fe2f3fff640 1 -- 172.21.15.32:0/2874513938 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2f415bf00 msgr2=0x7fe2f40d1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:20.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.420+0000 7fe2f3fff640 1 --2- 172.21.15.32:0/2874513938 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2f415bf00 0x7fe2f40d1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:20.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.420+0000 7fe2f3fff640 1 -- 172.21.15.32:0/2874513938 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2f40d07d0 con 0x7fe2f40b3ee0 2024-04-24T23:00:20.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.420+0000 7fe2f3fff640 1 --2- 172.21.15.32:0/2874513938 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2f40b3ee0 0x7fe2f413bb30 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc00c430 tx=0x7fe2dc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:20.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.421+0000 7fe2f0ff9640 1 -- 172.21.15.32:0/2874513938 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc011790 con 0x7fe2f40b3ee0 2024-04-24T23:00:20.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.421+0000 7fe2f0ff9640 1 -- 172.21.15.32:0/2874513938 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2dc017070 con 0x7fe2f40b3ee0 2024-04-24T23:00:20.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.421+0000 7fe2f0ff9640 1 -- 172.21.15.32:0/2874513938 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc011bb0 con 0x7fe2f40b3ee0 2024-04-24T23:00:20.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.421+0000 7fe2f939e640 1 -- 172.21.15.32:0/2874513938 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2f40d0ab0 con 0x7fe2f40b3ee0 2024-04-24T23:00:20.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.421+0000 7fe2f939e640 1 -- 172.21.15.32:0/2874513938 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2f40d1070 con 0x7fe2f40b3ee0 2024-04-24T23:00:20.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.422+0000 7fe2f939e640 1 -- 172.21.15.32:0/2874513938 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2f415f0e0 con 0x7fe2f40b3ee0 2024-04-24T23:00:20.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.422+0000 7fe2f0ff9640 1 -- 172.21.15.32:0/2874513938 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe2dc01c460 con 0x7fe2f40b3ee0 2024-04-24T23:00:20.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.423+0000 7fe2f0ff9640 1 --2- 172.21.15.32:0/2874513938 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe2d406be60 0x7fe2d406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:20.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.423+0000 7fe2f0ff9640 1 -- 172.21.15.32:0/2874513938 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fe2dc08abf0 con 0x7fe2f40b3ee0 2024-04-24T23:00:20.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.423+0000 7fe2f37fe640 1 --2- 172.21.15.32:0/2874513938 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe2d406be60 0x7fe2d406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:20.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.423+0000 7fe2f37fe640 1 --2- 172.21.15.32:0/2874513938 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe2d406be60 0x7fe2d406e320 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe2f41615e0 tx=0x7fe2e4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:20.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.427+0000 7fe2f0ff9640 1 -- 172.21.15.32:0/2874513938 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe2f415f0e0 con 0x7fe2f40b3ee0 2024-04-24T23:00:20.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.609+0000 7fe2f939e640 1 -- 172.21.15.32:0/2874513938 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe2f40682b0 con 0x7fe2f40b3ee0 2024-04-24T23:00:20.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.610+0000 7fe2f0ff9640 1 -- 172.21.15.32:0/2874513938 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16621 (secure 0 0 0) 0x7fe2dc0595a0 con 0x7fe2f40b3ee0 2024-04-24T23:00:20.612 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:00:20.612 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":34,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:00:12.312472+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:59:56.068812+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-24T22:59:56.447402+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:00:20.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.612+0000 7fe2f939e640 1 -- 172.21.15.32:0/2874513938 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe2d406be60 msgr2=0x7fe2d406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:20.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.612+0000 7fe2f939e640 1 --2- 172.21.15.32:0/2874513938 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe2d406be60 0x7fe2d406e320 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe2f41615e0 tx=0x7fe2e4008040 comp rx=0 tx=0).stop 2024-04-24T23:00:20.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.612+0000 7fe2f939e640 1 -- 172.21.15.32:0/2874513938 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2f40b3ee0 msgr2=0x7fe2f413bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:20.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.612+0000 7fe2f939e640 1 --2- 172.21.15.32:0/2874513938 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2f40b3ee0 0x7fe2f413bb30 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc00c430 tx=0x7fe2dc00c8f0 comp rx=0 tx=0).stop 2024-04-24T23:00:20.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.612+0000 7fe2f939e640 1 -- 172.21.15.32:0/2874513938 shutdown_connections 2024-04-24T23:00:20.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.612+0000 7fe2f939e640 1 --2- 172.21.15.32:0/2874513938 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe2d406be60 0x7fe2d406e320 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:20.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.612+0000 7fe2f939e640 1 --2- 172.21.15.32:0/2874513938 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe2f415c2e0 0x7fe2f4162930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:20.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.612+0000 7fe2f939e640 1 --2- 172.21.15.32:0/2874513938 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2f415bf00 0x7fe2f40d1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:20.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.612+0000 7fe2f939e640 1 --2- 172.21.15.32:0/2874513938 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2f40b3ee0 0x7fe2f413bb30 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:20.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.612+0000 7fe2f939e640 1 -- 172.21.15.32:0/2874513938 >> 172.21.15.32:0/2874513938 conn(0x7fe2f40743f0 msgr2=0x7fe2f40b2810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:20.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.612+0000 7fe2f939e640 1 -- 172.21.15.32:0/2874513938 shutdown_connections 2024-04-24T23:00:20.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.612+0000 7fe2f939e640 1 -- 172.21.15.32:0/2874513938 wait complete. 2024-04-24T23:00:20.625 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-24T23:00:20.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.725+0000 7f8e1f093640 1 Processor -- start 2024-04-24T23:00:20.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.725+0000 7f8e1f093640 1 -- start start 2024-04-24T23:00:20.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.725+0000 7f8e1f093640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8e180b5560 0x7f8e1815e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:20.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.726+0000 7f8e1f093640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8e180cd3c0 0x7f8e1815ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:20.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.726+0000 7f8e1f093640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8e180b5940 0x7f8e180b5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:20.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.726+0000 7f8e1f093640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f8e180cd910 con 0x7f8e180b5940 2024-04-24T23:00:20.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.726+0000 7f8e1f093640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f8e180cdbf0 con 0x7f8e180cd3c0 2024-04-24T23:00:20.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.726+0000 7f8e1f093640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f8e180cded0 con 0x7f8e180b5560 2024-04-24T23:00:20.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.726+0000 7f8e1d630640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8e180b5940 0x7f8e180b5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:20.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.726+0000 7f8e1d630640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8e180b5940 0x7f8e180b5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48070/0 (socket says 172.21.15.32:48070) 2024-04-24T23:00:20.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.726+0000 7f8e1d630640 1 -- 172.21.15.32:0/1144169137 learned_addr learned my addr 172.21.15.32:0/1144169137 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:20.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.726+0000 7f8e17fff640 1 --2- 172.21.15.32:0/1144169137 >> v2:172.21.15.121:3300/0 conn(0x7f8e180cd3c0 0x7f8e1815ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:20.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.726+0000 7f8e1ce2f640 1 --2- 172.21.15.32:0/1144169137 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8e180b5560 0x7f8e1815e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:20.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.726+0000 7f8e1d630640 1 -- 172.21.15.32:0/1144169137 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8e180b5560 msgr2=0x7f8e1815e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:20.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.726+0000 7f8e1d630640 1 --2- 172.21.15.32:0/1144169137 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8e180b5560 0x7f8e1815e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:20.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.726+0000 7f8e1d630640 1 -- 172.21.15.32:0/1144169137 >> v2:172.21.15.121:3300/0 conn(0x7f8e180cd3c0 msgr2=0x7f8e1815ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:20.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.726+0000 7f8e1d630640 1 --2- 172.21.15.32:0/1144169137 >> v2:172.21.15.121:3300/0 conn(0x7f8e180cd3c0 0x7f8e1815ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:20.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.726+0000 7f8e1d630640 1 -- 172.21.15.32:0/1144169137 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e180ce1b0 con 0x7f8e180b5940 2024-04-24T23:00:20.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.727+0000 7f8e1d630640 1 --2- 172.21.15.32:0/1144169137 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e180b5940 0x7f8e180b5d90 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f8e1806d830 tx=0x7f8e1002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c19f88c29b27f10 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:20.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.727+0000 7f8e177fe640 1 -- 172.21.15.32:0/1144169137 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e10036070 con 0x7f8e180b5940 2024-04-24T23:00:20.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.727+0000 7f8e177fe640 1 -- 172.21.15.32:0/1144169137 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e10031600 con 0x7f8e180b5940 2024-04-24T23:00:20.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.727+0000 7f8e177fe640 1 -- 172.21.15.32:0/1144169137 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e100319d0 con 0x7f8e180b5940 2024-04-24T23:00:20.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.727+0000 7f8e1f093640 1 -- 172.21.15.32:0/1144169137 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e180b5940 msgr2=0x7f8e180b5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:20.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.727+0000 7f8e1f093640 1 --2- 172.21.15.32:0/1144169137 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e180b5940 0x7f8e180b5d90 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f8e1806d830 tx=0x7f8e1002a6a0 comp rx=0 tx=0).stop 2024-04-24T23:00:20.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.727+0000 7f8e1f093640 1 -- 172.21.15.32:0/1144169137 shutdown_connections 2024-04-24T23:00:20.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.727+0000 7f8e1f093640 1 --2- 172.21.15.32:0/1144169137 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e180b5940 0x7f8e180b5d90 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:20.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.727+0000 7f8e1f093640 1 --2- 172.21.15.32:0/1144169137 >> v2:172.21.15.121:3300/0 conn(0x7f8e180cd3c0 0x7f8e1815ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:20.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.727+0000 7f8e1f093640 1 --2- 172.21.15.32:0/1144169137 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8e180b5560 0x7f8e1815e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:20.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.727+0000 7f8e1f093640 1 -- 172.21.15.32:0/1144169137 >> 172.21.15.32:0/1144169137 conn(0x7f8e1806b110 msgr2=0x7f8e1806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:20.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.728+0000 7f8e1f093640 1 -- 172.21.15.32:0/1144169137 shutdown_connections 2024-04-24T23:00:20.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.728+0000 7f8e1f093640 1 -- 172.21.15.32:0/1144169137 wait complete. 2024-04-24T23:00:20.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.729+0000 7f8e1f093640 1 Processor -- start 2024-04-24T23:00:20.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.729+0000 7f8e1f093640 1 -- start start 2024-04-24T23:00:20.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.730+0000 7f8e1f093640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e180b5560 0x7f8e1814af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:20.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.730+0000 7f8e1f093640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e180b5940 0x7f8e180ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:20.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.730+0000 7f8e1f093640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8e180cd3c0 0x7f8e180aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:20.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.730+0000 7f8e1f093640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8e180b6450 con 0x7f8e180b5940 2024-04-24T23:00:20.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.730+0000 7f8e1f093640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8e180b62d0 con 0x7f8e180b5560 2024-04-24T23:00:20.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.730+0000 7f8e1f093640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f8e1806c5f0 con 0x7f8e180cd3c0 2024-04-24T23:00:20.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.730+0000 7f8e17fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e180b5940 0x7f8e180ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:20.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.730+0000 7f8e17fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e180b5940 0x7f8e180ae8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48084/0 (socket says 172.21.15.32:48084) 2024-04-24T23:00:20.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.730+0000 7f8e17fff640 1 -- 172.21.15.32:0/1257628545 learned_addr learned my addr 172.21.15.32:0/1257628545 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:20.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.730+0000 7f8e1ce2f640 1 --2- 172.21.15.32:0/1257628545 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e180b5560 0x7f8e1814af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:20.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.730+0000 7f8e1d630640 1 --2- 172.21.15.32:0/1257628545 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8e180cd3c0 0x7f8e180aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:20.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.730+0000 7f8e17fff640 1 -- 172.21.15.32:0/1257628545 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8e180cd3c0 msgr2=0x7f8e180aedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:20.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.730+0000 7f8e17fff640 1 --2- 172.21.15.32:0/1257628545 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8e180cd3c0 0x7f8e180aedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:20.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.730+0000 7f8e17fff640 1 -- 172.21.15.32:0/1257628545 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e180b5560 msgr2=0x7f8e1814af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:20.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.730+0000 7f8e17fff640 1 --2- 172.21.15.32:0/1257628545 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e180b5560 0x7f8e1814af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:20.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.730+0000 7f8e17fff640 1 -- 172.21.15.32:0/1257628545 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e18071190 con 0x7f8e180b5940 2024-04-24T23:00:20.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.730+0000 7f8e1d630640 1 --2- 172.21.15.32:0/1257628545 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8e180cd3c0 0x7f8e180aedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:00:20.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.730+0000 7f8e17fff640 1 --2- 172.21.15.32:0/1257628545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e180b5940 0x7f8e180ae8b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8e0400d650 tx=0x7f8e0400db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:20.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.730+0000 7f8e157fa640 1 -- 172.21.15.32:0/1257628545 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e04015070 con 0x7f8e180b5940 2024-04-24T23:00:20.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.730+0000 7f8e157fa640 1 -- 172.21.15.32:0/1257628545 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e04007900 con 0x7f8e180b5940 2024-04-24T23:00:20.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.730+0000 7f8e1f093640 1 -- 172.21.15.32:0/1257628545 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e180703f0 con 0x7f8e180b5940 2024-04-24T23:00:20.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.730+0000 7f8e157fa640 1 -- 172.21.15.32:0/1257628545 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0400b040 con 0x7f8e180b5940 2024-04-24T23:00:20.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.730+0000 7f8e1f093640 1 -- 172.21.15.32:0/1257628545 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e18070990 con 0x7f8e180b5940 2024-04-24T23:00:20.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.731+0000 7f8e1f093640 1 -- 172.21.15.32:0/1257628545 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e18068430 con 0x7f8e180b5940 2024-04-24T23:00:20.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.732+0000 7f8e157fa640 1 -- 172.21.15.32:0/1257628545 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8e04018070 con 0x7f8e180b5940 2024-04-24T23:00:20.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.732+0000 7f8e157fa640 1 --2- 172.21.15.32:0/1257628545 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8dfc06be30 0x7f8dfc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:20.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.732+0000 7f8e157fa640 1 -- 172.21.15.32:0/1257628545 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f8e0408ad00 con 0x7f8e180b5940 2024-04-24T23:00:20.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.732+0000 7f8e1ce2f640 1 --2- 172.21.15.32:0/1257628545 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8dfc06be30 0x7f8dfc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:20.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.733+0000 7f8e1ce2f640 1 --2- 172.21.15.32:0/1257628545 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8dfc06be30 0x7f8dfc06e2f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c006fd0 tx=0x7f8e0c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:20.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.736+0000 7f8e157fa640 1 -- 172.21.15.32:0/1257628545 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8e04059060 con 0x7f8e180b5940 2024-04-24T23:00:20.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.955+0000 7f8e1f093640 1 -- 172.21.15.32:0/1257628545 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f8e1806d5f0 con 0x7f8e180b5940 2024-04-24T23:00:20.958 INFO:tasks.ceph.mon.a.smithi032.stderr:2024-04-24T23:00:20.956+0000 7f3b809c6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T23:00:20.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.957+0000 7f8e157fa640 1 -- 172.21.15.32:0/1257628545 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v14) ==== 161+0+0 (secure 0 0 0) 0x7f8e0405d360 con 0x7f8e180b5940 2024-04-24T23:00:20.960 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-24T23:00:20.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.960+0000 7f8e1f093640 1 -- 172.21.15.32:0/1257628545 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8dfc06be30 msgr2=0x7f8dfc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:20.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.960+0000 7f8e1f093640 1 --2- 172.21.15.32:0/1257628545 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8dfc06be30 0x7f8dfc06e2f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c006fd0 tx=0x7f8e0c008040 comp rx=0 tx=0).stop 2024-04-24T23:00:20.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.960+0000 7f8e1f093640 1 -- 172.21.15.32:0/1257628545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e180b5940 msgr2=0x7f8e180ae8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:20.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.960+0000 7f8e1f093640 1 --2- 172.21.15.32:0/1257628545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e180b5940 0x7f8e180ae8b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8e0400d650 tx=0x7f8e0400db10 comp rx=0 tx=0).stop 2024-04-24T23:00:20.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.960+0000 7f8e1f093640 1 -- 172.21.15.32:0/1257628545 shutdown_connections 2024-04-24T23:00:20.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.960+0000 7f8e1f093640 1 --2- 172.21.15.32:0/1257628545 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8dfc06be30 0x7f8dfc06e2f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:20.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.960+0000 7f8e1f093640 1 --2- 172.21.15.32:0/1257628545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e180b5940 0x7f8e180ae8b0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:20.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.960+0000 7f8e1f093640 1 --2- 172.21.15.32:0/1257628545 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8e180cd3c0 0x7f8e180aedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:20.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.960+0000 7f8e1f093640 1 --2- 172.21.15.32:0/1257628545 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e180b5560 0x7f8e1814af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:20.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.960+0000 7f8e1f093640 1 -- 172.21.15.32:0/1257628545 >> 172.21.15.32:0/1257628545 conn(0x7f8e1806b110 msgr2=0x7f8e1814b910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:20.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.961+0000 7f8e1f093640 1 -- 172.21.15.32:0/1257628545 shutdown_connections 2024-04-24T23:00:20.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:20.961+0000 7f8e1f093640 1 -- 172.21.15.32:0/1257628545 wait complete. 2024-04-24T23:00:20.966 INFO:tasks.ceph.mds.b.smithi121.stderr: -5114> 2024-04-24T23:00:02.089+0000 7f605de34640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T23:00:20.974 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-24T23:00:20.997 INFO:tasks.ceph.mds.b.smithi121.stdout:starting mds.b at 2024-04-24T23:00:21.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.074+0000 7fec5077e640 1 Processor -- start 2024-04-24T23:00:21.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.074+0000 7fec5077e640 1 -- start start 2024-04-24T23:00:21.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.075+0000 7fec5077e640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fec4815e9b0 0x7fec480d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:21.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.075+0000 7fec5077e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fec480d4210 0x7fec4806b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:21.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.075+0000 7fec5077e640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec480d3170 con 0x7fec480ba0e0 2024-04-24T23:00:21.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.075+0000 7fec5077e640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fec480b8900 con 0x7fec480d4210 2024-04-24T23:00:21.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.075+0000 7fec5077e640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fec480d3450 con 0x7fec4815e9b0 2024-04-24T23:00:21.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.075+0000 7fec4dd19640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fec480d4210 0x7fec4806b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:21.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.075+0000 7fec4dd19640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fec480d4210 0x7fec4806b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48094/0 (socket says 172.21.15.32:48094) 2024-04-24T23:00:21.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.075+0000 7fec4dd19640 1 -- 172.21.15.32:0/4105098039 learned_addr learned my addr 172.21.15.32:0/4105098039 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:21.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.076+0000 7fec4e51a640 1 --2- 172.21.15.32:0/4105098039 >> v2:172.21.15.121:3300/0 conn(0x7fec4815e9b0 0x7fec480d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:21.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.076+0000 7fec4dd19640 1 -- 172.21.15.32:0/4105098039 >> v2:172.21.15.121:3300/0 conn(0x7fec4815e9b0 msgr2=0x7fec480d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:21.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.076+0000 7fec4dd19640 1 --2- 172.21.15.32:0/4105098039 >> v2:172.21.15.121:3300/0 conn(0x7fec4815e9b0 0x7fec480d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:21.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.076+0000 7fec4dd19640 1 -- 172.21.15.32:0/4105098039 >> v1:172.21.15.121:6789/0 conn(0x7fec480ba0e0 legacy=0x7fec48071730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:21.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.076+0000 7fec4dd19640 1 -- 172.21.15.32:0/4105098039 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec480d3730 con 0x7fec480d4210 2024-04-24T23:00:21.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.076+0000 7fec4e51a640 1 --2- 172.21.15.32:0/4105098039 >> v2:172.21.15.121:3300/0 conn(0x7fec4815e9b0 0x7fec480d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:00:21.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.076+0000 7fec4dd19640 1 --2- 172.21.15.32:0/4105098039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec480d4210 0x7fec4806b110 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fec4806be20 tx=0x7fec3802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df942785f110748f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:21.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.076+0000 7fec4d518640 1 -- 172.21.15.32:0/4105098039 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec38037070 con 0x7fec480d4210 2024-04-24T23:00:21.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.077+0000 7fec4d518640 1 -- 172.21.15.32:0/4105098039 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec38032440 con 0x7fec480d4210 2024-04-24T23:00:21.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.077+0000 7fec4d518640 1 -- 172.21.15.32:0/4105098039 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec38032810 con 0x7fec480d4210 2024-04-24T23:00:21.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.077+0000 7fec5077e640 1 -- 172.21.15.32:0/4105098039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec480d4210 msgr2=0x7fec4806b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:21.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.077+0000 7fec5077e640 1 --2- 172.21.15.32:0/4105098039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec480d4210 0x7fec4806b110 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fec4806be20 tx=0x7fec3802a1a0 comp rx=0 tx=0).stop 2024-04-24T23:00:21.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.077+0000 7fec5077e640 1 -- 172.21.15.32:0/4105098039 shutdown_connections 2024-04-24T23:00:21.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.077+0000 7fec5077e640 1 --2- 172.21.15.32:0/4105098039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec480d4210 0x7fec4806b110 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:21.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.077+0000 7fec5077e640 1 --2- 172.21.15.32:0/4105098039 >> v2:172.21.15.121:3300/0 conn(0x7fec4815e9b0 0x7fec480d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:21.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.077+0000 7fec5077e640 1 -- 172.21.15.32:0/4105098039 >> 172.21.15.32:0/4105098039 conn(0x7fec480d11d0 msgr2=0x7fec480d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:21.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.077+0000 7fec5077e640 1 -- 172.21.15.32:0/4105098039 shutdown_connections 2024-04-24T23:00:21.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.077+0000 7fec5077e640 1 -- 172.21.15.32:0/4105098039 wait complete. 2024-04-24T23:00:21.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.078+0000 7fec5077e640 1 Processor -- start 2024-04-24T23:00:21.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.078+0000 7fec5077e640 1 -- start start 2024-04-24T23:00:21.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.079+0000 7fec5077e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec480ba0e0 0x7fec480ca3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:21.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.079+0000 7fec5077e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec480d4210 0x7fec480ce030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:21.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.079+0000 7fec5077e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec4815e9b0 0x7fec4815d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:21.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.079+0000 7fec5077e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fec480d2400 con 0x7fec4815e9b0 2024-04-24T23:00:21.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.079+0000 7fec5077e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fec480d3cf0 con 0x7fec480d4210 2024-04-24T23:00:21.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.079+0000 7fec5077e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fec4806b650 con 0x7fec480ba0e0 2024-04-24T23:00:21.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.079+0000 7fec4ed1b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec4815e9b0 0x7fec4815d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:21.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.079+0000 7fec4ed1b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec4815e9b0 0x7fec4815d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48110/0 (socket says 172.21.15.32:48110) 2024-04-24T23:00:21.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.079+0000 7fec4ed1b640 1 -- 172.21.15.32:0/2911883685 learned_addr learned my addr 172.21.15.32:0/2911883685 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:21.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.079+0000 7fec4e51a640 1 --2- 172.21.15.32:0/2911883685 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec480ba0e0 0x7fec480ca3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:21.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.079+0000 7fec4dd19640 1 --2- 172.21.15.32:0/2911883685 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec480d4210 0x7fec480ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:21.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.080+0000 7fec4ed1b640 1 -- 172.21.15.32:0/2911883685 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec480ba0e0 msgr2=0x7fec480ca3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:21.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.080+0000 7fec4ed1b640 1 --2- 172.21.15.32:0/2911883685 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec480ba0e0 0x7fec480ca3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:21.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.080+0000 7fec4ed1b640 1 -- 172.21.15.32:0/2911883685 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec480d4210 msgr2=0x7fec480ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:21.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.080+0000 7fec4ed1b640 1 --2- 172.21.15.32:0/2911883685 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec480d4210 0x7fec480ce030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:21.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.080+0000 7fec4ed1b640 1 -- 172.21.15.32:0/2911883685 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec481f7b60 con 0x7fec4815e9b0 2024-04-24T23:00:21.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.080+0000 7fec4dd19640 1 --2- 172.21.15.32:0/2911883685 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec480d4210 0x7fec480ce030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:00:21.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.080+0000 7fec4ed1b640 1 --2- 172.21.15.32:0/2911883685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec4815e9b0 0x7fec4815d320 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fec4000c780 tx=0x7fec4000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:21.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.080+0000 7fec3effd640 1 -- 172.21.15.32:0/2911883685 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec40004b60 con 0x7fec4815e9b0 2024-04-24T23:00:21.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.080+0000 7fec3effd640 1 -- 172.21.15.32:0/2911883685 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec40004d00 con 0x7fec4815e9b0 2024-04-24T23:00:21.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.080+0000 7fec5077e640 1 -- 172.21.15.32:0/2911883685 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec481f7e40 con 0x7fec4815e9b0 2024-04-24T23:00:21.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.080+0000 7fec3effd640 1 -- 172.21.15.32:0/2911883685 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4000aa30 con 0x7fec4815e9b0 2024-04-24T23:00:21.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.080+0000 7fec5077e640 1 -- 172.21.15.32:0/2911883685 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec481f8920 con 0x7fec4815e9b0 2024-04-24T23:00:21.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.081+0000 7fec5077e640 1 -- 172.21.15.32:0/2911883685 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec48068430 con 0x7fec4815e9b0 2024-04-24T23:00:21.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.081+0000 7fec3effd640 1 -- 172.21.15.32:0/2911883685 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fec4000abd0 con 0x7fec4815e9b0 2024-04-24T23:00:21.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.082+0000 7fec3effd640 1 --2- 172.21.15.32:0/2911883685 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fec2c06be30 0x7fec2c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:21.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.082+0000 7fec3effd640 1 -- 172.21.15.32:0/2911883685 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7fec4008b0c0 con 0x7fec4815e9b0 2024-04-24T23:00:21.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.082+0000 7fec4e51a640 1 --2- 172.21.15.32:0/2911883685 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fec2c06be30 0x7fec2c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:21.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.083+0000 7fec4e51a640 1 --2- 172.21.15.32:0/2911883685 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fec2c06be30 0x7fec2c06e2f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fec4400a950 tx=0x7fec44008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:21.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.086+0000 7fec3effd640 1 -- 172.21.15.32:0/2911883685 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fec40020070 con 0x7fec4815e9b0 2024-04-24T23:00:21.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.313+0000 7fec5077e640 1 -- 172.21.15.32:0/2911883685 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fec480caca0 con 0x7fec4815e9b0 2024-04-24T23:00:21.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.961+0000 7fec3effd640 1 -- 172.21.15.32:0/2911883685 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v15) ==== 104+0+0 (secure 0 0 0) 0x7fec40059aa0 con 0x7fec4815e9b0 2024-04-24T23:00:21.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.964+0000 7fec5077e640 1 -- 172.21.15.32:0/2911883685 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fec2c06be30 msgr2=0x7fec2c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:21.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.964+0000 7fec5077e640 1 --2- 172.21.15.32:0/2911883685 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fec2c06be30 0x7fec2c06e2f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fec4400a950 tx=0x7fec44008040 comp rx=0 tx=0).stop 2024-04-24T23:00:21.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.964+0000 7fec5077e640 1 -- 172.21.15.32:0/2911883685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec4815e9b0 msgr2=0x7fec4815d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:21.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.964+0000 7fec5077e640 1 --2- 172.21.15.32:0/2911883685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec4815e9b0 0x7fec4815d320 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fec4000c780 tx=0x7fec4000cc40 comp rx=0 tx=0).stop 2024-04-24T23:00:21.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.964+0000 7fec5077e640 1 -- 172.21.15.32:0/2911883685 shutdown_connections 2024-04-24T23:00:21.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.964+0000 7fec5077e640 1 --2- 172.21.15.32:0/2911883685 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fec2c06be30 0x7fec2c06e2f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:21.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.964+0000 7fec5077e640 1 --2- 172.21.15.32:0/2911883685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec4815e9b0 0x7fec4815d320 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:21.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.964+0000 7fec5077e640 1 --2- 172.21.15.32:0/2911883685 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec480d4210 0x7fec480ce030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:21.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.964+0000 7fec5077e640 1 --2- 172.21.15.32:0/2911883685 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec480ba0e0 0x7fec480ca3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:21.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.964+0000 7fec5077e640 1 -- 172.21.15.32:0/2911883685 >> 172.21.15.32:0/2911883685 conn(0x7fec480d11d0 msgr2=0x7fec48071730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:21.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.964+0000 7fec5077e640 1 -- 172.21.15.32:0/2911883685 shutdown_connections 2024-04-24T23:00:21.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:21.964+0000 7fec5077e640 1 -- 172.21.15.32:0/2911883685 wait complete. 2024-04-24T23:00:21.979 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-24T23:00:22.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.085+0000 7fd1f153b640 1 Processor -- start 2024-04-24T23:00:22.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.085+0000 7fd1f153b640 1 -- start start 2024-04-24T23:00:22.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.086+0000 7fd1f153b640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd1ec067980 0x7fd1ec0afa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:22.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.086+0000 7fd1f153b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd1ec0aff70 0x7fd1ec0b4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:22.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.086+0000 7fd1f153b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1ec071620 con 0x7fd1ec161c30 2024-04-24T23:00:22.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.086+0000 7fd1f153b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fd1ec0af5a0 con 0x7fd1ec0aff70 2024-04-24T23:00:22.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.086+0000 7fd1f153b640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fd1ec0af270 con 0x7fd1ec067980 2024-04-24T23:00:22.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.086+0000 7fd1eb7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd1ec0aff70 0x7fd1ec0b4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:22.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.086+0000 7fd1eaffd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd1ec161c30 0x7fd1ec06cdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33428/0 (socket says 172.21.15.32:33428) 2024-04-24T23:00:22.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.086+0000 7fd1eaffd640 1 -- 172.21.15.32:0/3853609850 learned_addr learned my addr 172.21.15.32:0/3853609850 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:00:22.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.086+0000 7fd1ea7fc640 1 --2- 172.21.15.32:0/3853609850 >> v2:172.21.15.121:3300/0 conn(0x7fd1ec067980 0x7fd1ec0afa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:22.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.086+0000 7fd1eb7fe640 1 -- 172.21.15.32:0/3853609850 >> v2:172.21.15.121:3300/0 conn(0x7fd1ec067980 msgr2=0x7fd1ec0afa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:22.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.086+0000 7fd1eb7fe640 1 --2- 172.21.15.32:0/3853609850 >> v2:172.21.15.121:3300/0 conn(0x7fd1ec067980 0x7fd1ec0afa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:22.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.086+0000 7fd1eb7fe640 1 -- 172.21.15.32:0/3853609850 >> v1:172.21.15.32:6789/0 conn(0x7fd1ec161c30 legacy=0x7fd1ec06cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:22.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.086+0000 7fd1eb7fe640 1 -- 172.21.15.32:0/3853609850 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1ec06d4d0 con 0x7fd1ec0aff70 2024-04-24T23:00:22.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.086+0000 7fd1e9ffb640 1 -- 172.21.15.32:0/3853609850 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1372244582 0 0) 0x7fd1ec071620 con 0x7fd1ec161c30 2024-04-24T23:00:22.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.086+0000 7fd1ea7fc640 1 --2- 172.21.15.32:0/3853609850 >> v2:172.21.15.121:3300/0 conn(0x7fd1ec067980 0x7fd1ec0afa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:00:22.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.087+0000 7fd1eb7fe640 1 --2- 172.21.15.32:0/3853609850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1ec0aff70 0x7fd1ec0b4b20 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec0af720 tx=0x7fd1dc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24bfb34238254012 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:22.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.087+0000 7fd1e9ffb640 1 -- 172.21.15.32:0/3853609850 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc007740 con 0x7fd1ec0aff70 2024-04-24T23:00:22.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.087+0000 7fd1e9ffb640 1 -- 172.21.15.32:0/3853609850 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1dc0078e0 con 0x7fd1ec0aff70 2024-04-24T23:00:22.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.087+0000 7fd1e9ffb640 1 -- 172.21.15.32:0/3853609850 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc007cb0 con 0x7fd1ec0aff70 2024-04-24T23:00:22.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.088+0000 7fd1f153b640 1 -- 172.21.15.32:0/3853609850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1ec0aff70 msgr2=0x7fd1ec0b4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:22.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.088+0000 7fd1f153b640 1 --2- 172.21.15.32:0/3853609850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1ec0aff70 0x7fd1ec0b4b20 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec0af720 tx=0x7fd1dc02a660 comp rx=0 tx=0).stop 2024-04-24T23:00:22.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.088+0000 7fd1f153b640 1 -- 172.21.15.32:0/3853609850 shutdown_connections 2024-04-24T23:00:22.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.088+0000 7fd1f153b640 1 --2- 172.21.15.32:0/3853609850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1ec0aff70 0x7fd1ec0b4b20 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:22.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.088+0000 7fd1f153b640 1 --2- 172.21.15.32:0/3853609850 >> v2:172.21.15.121:3300/0 conn(0x7fd1ec067980 0x7fd1ec0afa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:22.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.088+0000 7fd1f153b640 1 -- 172.21.15.32:0/3853609850 >> 172.21.15.32:0/3853609850 conn(0x7fd1ec0d0a60 msgr2=0x7fd1ec0d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:22.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.088+0000 7fd1f153b640 1 -- 172.21.15.32:0/3853609850 shutdown_connections 2024-04-24T23:00:22.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.088+0000 7fd1f153b640 1 -- 172.21.15.32:0/3853609850 wait complete. 2024-04-24T23:00:22.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.089+0000 7fd1f153b640 1 Processor -- start 2024-04-24T23:00:22.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.089+0000 7fd1f153b640 1 -- start start 2024-04-24T23:00:22.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.090+0000 7fd1f153b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1ec067980 0x7fd1ec14f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:22.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.090+0000 7fd1f153b640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd1ec0aff70 0x7fd1ec0c5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:22.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.090+0000 7fd1f153b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1ec161c30 0x7fd1ec0c8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:22.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.090+0000 7fd1f153b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ec0b03c0 con 0x7fd1ec067980 2024-04-24T23:00:22.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.090+0000 7fd1f153b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ec071ba0 con 0x7fd1ec161c30 2024-04-24T23:00:22.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.090+0000 7fd1f153b640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd1ec162010 con 0x7fd1ec0aff70 2024-04-24T23:00:22.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.090+0000 7fd1eaffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1ec067980 0x7fd1ec14f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:22.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.090+0000 7fd1eaffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1ec067980 0x7fd1ec14f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48118/0 (socket says 172.21.15.32:48118) 2024-04-24T23:00:22.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.090+0000 7fd1eaffd640 1 -- 172.21.15.32:0/2371468633 learned_addr learned my addr 172.21.15.32:0/2371468633 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:22.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.090+0000 7fd1ea7fc640 1 --2- 172.21.15.32:0/2371468633 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd1ec0aff70 0x7fd1ec0c5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:22.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.090+0000 7fd1eb7fe640 1 --2- 172.21.15.32:0/2371468633 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1ec161c30 0x7fd1ec0c8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:22.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.090+0000 7fd1eaffd640 1 -- 172.21.15.32:0/2371468633 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd1ec0aff70 msgr2=0x7fd1ec0c5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:22.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.090+0000 7fd1eaffd640 1 --2- 172.21.15.32:0/2371468633 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd1ec0aff70 0x7fd1ec0c5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:22.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.090+0000 7fd1eaffd640 1 -- 172.21.15.32:0/2371468633 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1ec161c30 msgr2=0x7fd1ec0c8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:22.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.090+0000 7fd1eaffd640 1 --2- 172.21.15.32:0/2371468633 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1ec161c30 0x7fd1ec0c8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:22.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.090+0000 7fd1eaffd640 1 -- 172.21.15.32:0/2371468633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1ec14f980 con 0x7fd1ec067980 2024-04-24T23:00:22.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.090+0000 7fd1ea7fc640 1 --2- 172.21.15.32:0/2371468633 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd1ec0aff70 0x7fd1ec0c5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:00:22.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.090+0000 7fd1eaffd640 1 --2- 172.21.15.32:0/2371468633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1ec067980 0x7fd1ec14f440 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fd1e400c460 tx=0x7fd1e400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:22.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.091+0000 7fd1e37fe640 1 -- 172.21.15.32:0/2371468633 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e4018070 con 0x7fd1ec067980 2024-04-24T23:00:22.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.091+0000 7fd1e37fe640 1 -- 172.21.15.32:0/2371468633 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1e4007500 con 0x7fd1ec067980 2024-04-24T23:00:22.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.091+0000 7fd1f153b640 1 -- 172.21.15.32:0/2371468633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1ec0c64f0 con 0x7fd1ec067980 2024-04-24T23:00:22.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.091+0000 7fd1e37fe640 1 -- 172.21.15.32:0/2371468633 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e4004a20 con 0x7fd1ec067980 2024-04-24T23:00:22.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.091+0000 7fd1f153b640 1 -- 172.21.15.32:0/2371468633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1ec0c6ab0 con 0x7fd1ec067980 2024-04-24T23:00:22.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.092+0000 7fd1f153b640 1 -- 172.21.15.32:0/2371468633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1ec068430 con 0x7fd1ec067980 2024-04-24T23:00:22.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.092+0000 7fd1e37fe640 1 -- 172.21.15.32:0/2371468633 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd1e401b070 con 0x7fd1ec067980 2024-04-24T23:00:22.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.092+0000 7fd1e37fe640 1 --2- 172.21.15.32:0/2371468633 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd1cc06be30 0x7fd1cc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:22.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.093+0000 7fd1e37fe640 1 -- 172.21.15.32:0/2371468633 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7fd1e408a0e0 con 0x7fd1ec067980 2024-04-24T23:00:22.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.093+0000 7fd1ea7fc640 1 --2- 172.21.15.32:0/2371468633 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd1cc06be30 0x7fd1cc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:22.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.093+0000 7fd1ea7fc640 1 --2- 172.21.15.32:0/2371468633 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd1cc06be30 0x7fd1cc06e2f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec0b5560 tx=0x7fd1d4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:22.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.097+0000 7fd1e37fe640 1 -- 172.21.15.32:0/2371468633 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd1e4058ac0 con 0x7fd1ec067980 2024-04-24T23:00:22.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.283+0000 7fd1f153b640 1 -- 172.21.15.32:0/2371468633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd1ec0d18d0 con 0x7fd1ec067980 2024-04-24T23:00:22.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.963+0000 7fd1e37fe640 1 -- 172.21.15.32:0/2371468633 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v36) ==== 181+0+0 (secure 0 0 0) 0x7fd1e40583c0 con 0x7fd1ec067980 2024-04-24T23:00:22.966 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2024-04-24T23:00:22.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.966+0000 7fd1f153b640 1 -- 172.21.15.32:0/2371468633 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd1cc06be30 msgr2=0x7fd1cc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:22.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.966+0000 7fd1f153b640 1 --2- 172.21.15.32:0/2371468633 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd1cc06be30 0x7fd1cc06e2f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec0b5560 tx=0x7fd1d4008040 comp rx=0 tx=0).stop 2024-04-24T23:00:22.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.967+0000 7fd1f153b640 1 -- 172.21.15.32:0/2371468633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1ec067980 msgr2=0x7fd1ec14f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:22.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.967+0000 7fd1f153b640 1 --2- 172.21.15.32:0/2371468633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1ec067980 0x7fd1ec14f440 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fd1e400c460 tx=0x7fd1e400c920 comp rx=0 tx=0).stop 2024-04-24T23:00:22.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.967+0000 7fd1f153b640 1 -- 172.21.15.32:0/2371468633 shutdown_connections 2024-04-24T23:00:22.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.967+0000 7fd1f153b640 1 --2- 172.21.15.32:0/2371468633 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd1cc06be30 0x7fd1cc06e2f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:22.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.967+0000 7fd1f153b640 1 --2- 172.21.15.32:0/2371468633 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1ec161c30 0x7fd1ec0c8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:22.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.967+0000 7fd1f153b640 1 --2- 172.21.15.32:0/2371468633 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd1ec0aff70 0x7fd1ec0c5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:22.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.967+0000 7fd1f153b640 1 --2- 172.21.15.32:0/2371468633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1ec067980 0x7fd1ec14f440 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:22.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.967+0000 7fd1f153b640 1 -- 172.21.15.32:0/2371468633 >> 172.21.15.32:0/2371468633 conn(0x7fd1ec0d0a60 msgr2=0x7fd1ec0b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:22.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.967+0000 7fd1f153b640 1 -- 172.21.15.32:0/2371468633 shutdown_connections 2024-04-24T23:00:22.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:22.967+0000 7fd1f153b640 1 -- 172.21.15.32:0/2371468633 wait complete. 2024-04-24T23:00:22.984 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-24T23:00:23.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.091+0000 7fe1cf109640 1 Processor -- start 2024-04-24T23:00:23.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.091+0000 7fe1cf109640 1 -- start start 2024-04-24T23:00:23.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.091+0000 7fe1cf109640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe1c815b580 0x7fe1c80cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:23.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.091+0000 7fe1cf109640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe1c80b58e0 0x7fe1c80b5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:23.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.091+0000 7fe1cf109640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1c815bde0 con 0x7fe1c815b960 2024-04-24T23:00:23.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.091+0000 7fe1cf109640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fe1c815c9f0 con 0x7fe1c80b58e0 2024-04-24T23:00:23.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.091+0000 7fe1cf109640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fe1c80ce3c0 con 0x7fe1c815b580 2024-04-24T23:00:23.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.092+0000 7fe1cd6a6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe1c80b58e0 0x7fe1c80b5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:23.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.092+0000 7fe1cd6a6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe1c80b58e0 0x7fe1c80b5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48124/0 (socket says 172.21.15.32:48124) 2024-04-24T23:00:23.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.092+0000 7fe1c7fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe1c815b960 0x7fe1c806cb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33440/0 (socket says 172.21.15.32:33440) 2024-04-24T23:00:23.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.092+0000 7fe1cd6a6640 1 -- 172.21.15.32:0/333404222 learned_addr learned my addr 172.21.15.32:0/333404222 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:23.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.092+0000 7fe1ccea5640 1 --2- 172.21.15.32:0/333404222 >> v2:172.21.15.121:3300/0 conn(0x7fe1c815b580 0x7fe1c80cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:23.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.092+0000 7fe1cd6a6640 1 -- 172.21.15.32:0/333404222 >> v2:172.21.15.121:3300/0 conn(0x7fe1c815b580 msgr2=0x7fe1c80cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:23.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.092+0000 7fe1cd6a6640 1 --2- 172.21.15.32:0/333404222 >> v2:172.21.15.121:3300/0 conn(0x7fe1c815b580 0x7fe1c80cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:23.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.092+0000 7fe1cd6a6640 1 -- 172.21.15.32:0/333404222 >> v1:172.21.15.32:6789/0 conn(0x7fe1c815b960 legacy=0x7fe1c806cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:23.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.092+0000 7fe1cd6a6640 1 -- 172.21.15.32:0/333404222 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1c815c0c0 con 0x7fe1c80b58e0 2024-04-24T23:00:23.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.092+0000 7fe1c77fe640 1 -- 172.21.15.32:0/333404222 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 108056215 0 0) 0x7fe1c815bde0 con 0x7fe1c815b960 2024-04-24T23:00:23.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.093+0000 7fe1cd6a6640 1 --2- 172.21.15.32:0/333404222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1c80b58e0 0x7fe1c80b5d20 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fe1c806e500 tx=0x7fe1c002c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e4aa30a330243bb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:23.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.093+0000 7fe1c77fe640 1 -- 172.21.15.32:0/333404222 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c000b2e0 con 0x7fe1c80b58e0 2024-04-24T23:00:23.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.093+0000 7fe1c77fe640 1 -- 172.21.15.32:0/333404222 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1c0037070 con 0x7fe1c80b58e0 2024-04-24T23:00:23.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.093+0000 7fe1c77fe640 1 -- 172.21.15.32:0/333404222 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c000b480 con 0x7fe1c80b58e0 2024-04-24T23:00:23.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.093+0000 7fe1cf109640 1 -- 172.21.15.32:0/333404222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1c80b58e0 msgr2=0x7fe1c80b5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:23.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.093+0000 7fe1cf109640 1 --2- 172.21.15.32:0/333404222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1c80b58e0 0x7fe1c80b5d20 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fe1c806e500 tx=0x7fe1c002c940 comp rx=0 tx=0).stop 2024-04-24T23:00:23.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.093+0000 7fe1cf109640 1 -- 172.21.15.32:0/333404222 shutdown_connections 2024-04-24T23:00:23.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.093+0000 7fe1cf109640 1 --2- 172.21.15.32:0/333404222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1c80b58e0 0x7fe1c80b5d20 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:23.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.093+0000 7fe1cf109640 1 --2- 172.21.15.32:0/333404222 >> v2:172.21.15.121:3300/0 conn(0x7fe1c815b580 0x7fe1c80cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:23.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.093+0000 7fe1cf109640 1 -- 172.21.15.32:0/333404222 >> 172.21.15.32:0/333404222 conn(0x7fe1c80b1f10 msgr2=0x7fe1c80b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:23.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.093+0000 7fe1cf109640 1 -- 172.21.15.32:0/333404222 shutdown_connections 2024-04-24T23:00:23.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.093+0000 7fe1cf109640 1 -- 172.21.15.32:0/333404222 wait complete. 2024-04-24T23:00:23.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.095+0000 7fe1cf109640 1 Processor -- start 2024-04-24T23:00:23.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.095+0000 7fe1cf109640 1 -- start start 2024-04-24T23:00:23.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.095+0000 7fe1cf109640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe1c80b58e0 0x7fe1c81448a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:23.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.095+0000 7fe1cf109640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1c815b580 0x7fe1c80b3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:23.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.096+0000 7fe1cf109640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1c815b960 0x7fe1c8158200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:23.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.096+0000 7fe1cf109640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c815c680 con 0x7fe1c815b580 2024-04-24T23:00:23.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.096+0000 7fe1cf109640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c80b6260 con 0x7fe1c815b960 2024-04-24T23:00:23.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.096+0000 7fe1cf109640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe1c806c7a0 con 0x7fe1c80b58e0 2024-04-24T23:00:23.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.096+0000 7fe1c7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1c815b580 0x7fe1c80b3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:23.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.096+0000 7fe1c7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1c815b580 0x7fe1c80b3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48134/0 (socket says 172.21.15.32:48134) 2024-04-24T23:00:23.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.096+0000 7fe1c7fff640 1 -- 172.21.15.32:0/1782173206 learned_addr learned my addr 172.21.15.32:0/1782173206 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:23.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.096+0000 7fe1ccea5640 1 --2- 172.21.15.32:0/1782173206 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe1c80b58e0 0x7fe1c81448a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:23.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.096+0000 7fe1cd6a6640 1 --2- 172.21.15.32:0/1782173206 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1c815b960 0x7fe1c8158200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:23.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.096+0000 7fe1c7fff640 1 -- 172.21.15.32:0/1782173206 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe1c80b58e0 msgr2=0x7fe1c81448a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:23.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.096+0000 7fe1c7fff640 1 --2- 172.21.15.32:0/1782173206 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe1c80b58e0 0x7fe1c81448a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:23.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.096+0000 7fe1c7fff640 1 -- 172.21.15.32:0/1782173206 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1c815b960 msgr2=0x7fe1c8158200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:23.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.096+0000 7fe1c7fff640 1 --2- 172.21.15.32:0/1782173206 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1c815b960 0x7fe1c8158200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:23.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.096+0000 7fe1c7fff640 1 -- 172.21.15.32:0/1782173206 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1c80b3da0 con 0x7fe1c815b580 2024-04-24T23:00:23.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.096+0000 7fe1ccea5640 1 --2- 172.21.15.32:0/1782173206 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe1c80b58e0 0x7fe1c81448a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:00:23.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.096+0000 7fe1cd6a6640 1 --2- 172.21.15.32:0/1782173206 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1c815b960 0x7fe1c8158200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:00:23.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.096+0000 7fe1c7fff640 1 --2- 172.21.15.32:0/1782173206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1c815b580 0x7fe1c80b3860 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc00c3c0 tx=0x7fe1bc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:23.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.096+0000 7fe1c57fa640 1 -- 172.21.15.32:0/1782173206 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc0074e0 con 0x7fe1c815b580 2024-04-24T23:00:23.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.096+0000 7fe1c57fa640 1 -- 172.21.15.32:0/1782173206 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1bc0047e0 con 0x7fe1c815b580 2024-04-24T23:00:23.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.096+0000 7fe1cf109640 1 -- 172.21.15.32:0/1782173206 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1c8144fc0 con 0x7fe1c815b580 2024-04-24T23:00:23.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.096+0000 7fe1c57fa640 1 -- 172.21.15.32:0/1782173206 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc004bd0 con 0x7fe1c815b580 2024-04-24T23:00:23.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.096+0000 7fe1cf109640 1 -- 172.21.15.32:0/1782173206 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1c8145580 con 0x7fe1c815b580 2024-04-24T23:00:23.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.097+0000 7fe1cf109640 1 -- 172.21.15.32:0/1782173206 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1c8068430 con 0x7fe1c815b580 2024-04-24T23:00:23.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.098+0000 7fe1c57fa640 1 -- 172.21.15.32:0/1782173206 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe1bc011070 con 0x7fe1c815b580 2024-04-24T23:00:23.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.098+0000 7fe1c57fa640 1 --2- 172.21.15.32:0/1782173206 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe1b406be30 0x7fe1b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:23.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.098+0000 7fe1c57fa640 1 -- 172.21.15.32:0/1782173206 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5357+0+0 (secure 0 0 0) 0x7fe1bc08a900 con 0x7fe1c815b580 2024-04-24T23:00:23.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.098+0000 7fe1ccea5640 1 --2- 172.21.15.32:0/1782173206 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe1b406be30 0x7fe1b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:23.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.099+0000 7fe1ccea5640 1 --2- 172.21.15.32:0/1782173206 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe1b406be30 0x7fe1b406e2f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8009ac0 tx=0x7fe1b8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:23.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.102+0000 7fe1c57fa640 1 -- 172.21.15.32:0/1782173206 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe1bc00e280 con 0x7fe1c815b580 2024-04-24T23:00:23.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.286+0000 7fe1cf109640 1 -- 172.21.15.32:0/1782173206 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe1c80af9c0 con 0x7fe1c815b580 2024-04-24T23:00:23.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.967+0000 7fe1c57fa640 1 -- 172.21.15.32:0/1782173206 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v37) ==== 169+0+0 (secure 0 0 0) 0x7fe1bc059410 con 0x7fe1c815b580 2024-04-24T23:00:23.970 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2024-04-24T23:00:23.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.971+0000 7fe1cf109640 1 -- 172.21.15.32:0/1782173206 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe1b406be30 msgr2=0x7fe1b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:23.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.971+0000 7fe1cf109640 1 --2- 172.21.15.32:0/1782173206 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe1b406be30 0x7fe1b406e2f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8009ac0 tx=0x7fe1b8008040 comp rx=0 tx=0).stop 2024-04-24T23:00:23.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.971+0000 7fe1cf109640 1 -- 172.21.15.32:0/1782173206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1c815b580 msgr2=0x7fe1c80b3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:23.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.971+0000 7fe1cf109640 1 --2- 172.21.15.32:0/1782173206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1c815b580 0x7fe1c80b3860 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc00c3c0 tx=0x7fe1bc00c880 comp rx=0 tx=0).stop 2024-04-24T23:00:23.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.972+0000 7fe1cf109640 1 -- 172.21.15.32:0/1782173206 shutdown_connections 2024-04-24T23:00:23.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.972+0000 7fe1cf109640 1 --2- 172.21.15.32:0/1782173206 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe1b406be30 0x7fe1b406e2f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:23.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.972+0000 7fe1cf109640 1 --2- 172.21.15.32:0/1782173206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1c815b580 0x7fe1c80b3860 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:23.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.972+0000 7fe1cf109640 1 --2- 172.21.15.32:0/1782173206 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1c815b960 0x7fe1c8158200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:23.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.972+0000 7fe1cf109640 1 --2- 172.21.15.32:0/1782173206 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe1c80b58e0 0x7fe1c81448a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:23.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.972+0000 7fe1cf109640 1 -- 172.21.15.32:0/1782173206 >> 172.21.15.32:0/1782173206 conn(0x7fe1c80b1f10 msgr2=0x7fe1c815cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:23.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.972+0000 7fe1cf109640 1 -- 172.21.15.32:0/1782173206 shutdown_connections 2024-04-24T23:00:23.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:23.972+0000 7fe1cf109640 1 -- 172.21.15.32:0/1782173206 wait complete. 2024-04-24T23:00:23.988 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-24T23:00:24.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.091+0000 7efc9967b640 1 Processor -- start 2024-04-24T23:00:24.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.091+0000 7efc9967b640 1 -- start start 2024-04-24T23:00:24.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.092+0000 7efc9967b640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7efc9406cd00 0x7efc94167130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:24.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.092+0000 7efc9967b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc9415c990 con 0x7efc94072cf0 2024-04-24T23:00:24.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.092+0000 7efc9967b640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc94071c50 con 0x7efc9415c5b0 2024-04-24T23:00:24.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.092+0000 7efc9967b640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7efc940b4740 con 0x7efc9406cd00 2024-04-24T23:00:24.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.092+0000 7efc927fc640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7efc94072cf0 0x7efc9406c5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33444/0 (socket says 172.21.15.32:33444) 2024-04-24T23:00:24.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.092+0000 7efc927fc640 1 -- 172.21.15.32:0/3575296813 learned_addr learned my addr 172.21.15.32:0/3575296813 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:00:24.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.092+0000 7efc91ffb640 1 -- 172.21.15.32:0/3575296813 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2742221397 0 0) 0x7efc7c001110 con 0x7efc94072cf0 2024-04-24T23:00:24.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.092+0000 7efc937fe640 1 --2- 172.21.15.32:0/3575296813 >> v2:172.21.15.121:3300/0 conn(0x7efc9406cd00 0x7efc94167130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:24.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.092+0000 7efc91ffb640 1 -- 172.21.15.32:0/3575296813 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efc80003520 con 0x7efc94072cf0 2024-04-24T23:00:24.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.093+0000 7efc91ffb640 1 -- 172.21.15.32:0/3575296813 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1142754073 0 0) 0x7efc80003520 con 0x7efc94072cf0 2024-04-24T23:00:24.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.093+0000 7efc91ffb640 1 -- 172.21.15.32:0/3575296813 >> v2:172.21.15.121:3300/0 conn(0x7efc9406cd00 msgr2=0x7efc94167130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:24.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.093+0000 7efc91ffb640 1 --2- 172.21.15.32:0/3575296813 >> v2:172.21.15.121:3300/0 conn(0x7efc9406cd00 0x7efc94167130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:24.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.093+0000 7efc91ffb640 1 -- 172.21.15.32:0/3575296813 >> v1:172.21.15.121:6789/0 conn(0x7efc9415c5b0 legacy=0x7efc940b4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:24.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.093+0000 7efc91ffb640 1 -- 172.21.15.32:0/3575296813 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc94071f30 con 0x7efc94072cf0 2024-04-24T23:00:24.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.093+0000 7efc91ffb640 1 -- 172.21.15.32:0/3575296813 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7efc7c001840 con 0x7efc94072cf0 2024-04-24T23:00:24.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.093+0000 7efc91ffb640 1 -- 172.21.15.32:0/3575296813 >> v1:172.21.15.32:6789/0 conn(0x7efc94072cf0 legacy=0x7efc9406c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:24.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.093+0000 7efc91ffb640 1 --2- 172.21.15.32:0/3575296813 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efc800039b0 0x7efc80003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:24.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.093+0000 7efc91ffb640 1 --2- 172.21.15.32:0/3575296813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc80004800 0x7efc80004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:24.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.093+0000 7efc91ffb640 1 --2- 172.21.15.32:0/3575296813 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc800051c0 0x7efc8000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:24.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.093+0000 7efc91ffb640 1 -- 172.21.15.32:0/3575296813 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7efc8000c130 con 0x7efc80004800 2024-04-24T23:00:24.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.093+0000 7efc91ffb640 1 -- 172.21.15.32:0/3575296813 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7efc8000c2e0 con 0x7efc800051c0 2024-04-24T23:00:24.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.093+0000 7efc91ffb640 1 -- 172.21.15.32:0/3575296813 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7efc8000c4e0 con 0x7efc800039b0 2024-04-24T23:00:24.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.093+0000 7efc927fc640 1 --2- 172.21.15.32:0/3575296813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc80004800 0x7efc80004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:24.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.093+0000 7efc92ffd640 1 --2- 172.21.15.32:0/3575296813 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efc800039b0 0x7efc80003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:24.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.093+0000 7efc937fe640 1 --2- 172.21.15.32:0/3575296813 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc800051c0 0x7efc8000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:24.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.093+0000 7efc927fc640 1 -- 172.21.15.32:0/3575296813 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efc800039b0 msgr2=0x7efc80003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:24.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.093+0000 7efc927fc640 1 --2- 172.21.15.32:0/3575296813 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efc800039b0 0x7efc80003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:24.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.093+0000 7efc927fc640 1 -- 172.21.15.32:0/3575296813 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc800051c0 msgr2=0x7efc8000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:24.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.093+0000 7efc927fc640 1 --2- 172.21.15.32:0/3575296813 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc800051c0 0x7efc8000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:24.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.093+0000 7efc937fe640 1 -- 172.21.15.32:0/3575296813 reap_dead start 2024-04-24T23:00:24.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.093+0000 7efc927fc640 1 -- 172.21.15.32:0/3575296813 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc80003520 con 0x7efc80004800 2024-04-24T23:00:24.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.094+0000 7efc927fc640 1 --2- 172.21.15.32:0/3575296813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc80004800 0x7efc80004c80 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7efc7c00c4c0 tx=0x7efc80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b002542811c7c21 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:24.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.094+0000 7efc91ffb640 1 -- 172.21.15.32:0/3575296813 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7c00c0e0 con 0x7efc80004800 2024-04-24T23:00:24.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.094+0000 7efc91ffb640 1 -- 172.21.15.32:0/3575296813 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc7c034070 con 0x7efc80004800 2024-04-24T23:00:24.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.094+0000 7efc91ffb640 1 -- 172.21.15.32:0/3575296813 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7c005d70 con 0x7efc80004800 2024-04-24T23:00:24.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.094+0000 7efc9967b640 1 -- 172.21.15.32:0/3575296813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc80004800 msgr2=0x7efc80004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:24.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.094+0000 7efc9967b640 1 --2- 172.21.15.32:0/3575296813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc80004800 0x7efc80004c80 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7efc7c00c4c0 tx=0x7efc80005830 comp rx=0 tx=0).stop 2024-04-24T23:00:24.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.095+0000 7efc9967b640 1 -- 172.21.15.32:0/3575296813 shutdown_connections 2024-04-24T23:00:24.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.095+0000 7efc9967b640 1 --2- 172.21.15.32:0/3575296813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc80004800 0x7efc80004c80 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:24.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.095+0000 7efc9967b640 1 -- 172.21.15.32:0/3575296813 >> 172.21.15.32:0/3575296813 conn(0x7efc9406b110 msgr2=0x7efc9406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:24.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.095+0000 7efc9967b640 1 -- 172.21.15.32:0/3575296813 shutdown_connections 2024-04-24T23:00:24.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.095+0000 7efc9967b640 1 -- 172.21.15.32:0/3575296813 wait complete. 2024-04-24T23:00:24.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.096+0000 7efc9967b640 1 Processor -- start 2024-04-24T23:00:24.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.096+0000 7efc9967b640 1 -- start start 2024-04-24T23:00:24.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.097+0000 7efc9967b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc80004800 0x7efc940aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:24.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.097+0000 7efc9967b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc94152990 0x7efc94152df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:24.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.097+0000 7efc9967b640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efc940b37e0 0x7efc940b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:24.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.097+0000 7efc9967b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7efc940724f0 con 0x7efc94152990 2024-04-24T23:00:24.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.097+0000 7efc9967b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7efc94072860 con 0x7efc80004800 2024-04-24T23:00:24.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.097+0000 7efc9967b640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7efc94071770 con 0x7efc940b37e0 2024-04-24T23:00:24.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.097+0000 7efc927fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc94152990 0x7efc94152df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:24.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.097+0000 7efc927fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc94152990 0x7efc94152df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48148/0 (socket says 172.21.15.32:48148) 2024-04-24T23:00:24.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.097+0000 7efc927fc640 1 -- 172.21.15.32:0/2807446555 learned_addr learned my addr 172.21.15.32:0/2807446555 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:24.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.097+0000 7efc92ffd640 1 --2- 172.21.15.32:0/2807446555 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc80004800 0x7efc940aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:24.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.097+0000 7efc937fe640 1 --2- 172.21.15.32:0/2807446555 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efc940b37e0 0x7efc940b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:24.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.097+0000 7efc927fc640 1 -- 172.21.15.32:0/2807446555 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efc940b37e0 msgr2=0x7efc940b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:24.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.097+0000 7efc927fc640 1 --2- 172.21.15.32:0/2807446555 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efc940b37e0 0x7efc940b3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:24.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.097+0000 7efc927fc640 1 -- 172.21.15.32:0/2807446555 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc80004800 msgr2=0x7efc940aeb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:24.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.097+0000 7efc927fc640 1 --2- 172.21.15.32:0/2807446555 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc80004800 0x7efc940aeb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:24.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.097+0000 7efc927fc640 1 -- 172.21.15.32:0/2807446555 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc941f88d0 con 0x7efc94152990 2024-04-24T23:00:24.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.097+0000 7efc927fc640 1 --2- 172.21.15.32:0/2807446555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc94152990 0x7efc94152df0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7efc7c00c4c0 tx=0x7efc94072100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:24.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.097+0000 7efc8b7fe640 1 -- 172.21.15.32:0/2807446555 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7c001840 con 0x7efc94152990 2024-04-24T23:00:24.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.097+0000 7efc8b7fe640 1 -- 172.21.15.32:0/2807446555 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc7c03f030 con 0x7efc94152990 2024-04-24T23:00:24.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.098+0000 7efc9967b640 1 -- 172.21.15.32:0/2807446555 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc941f8bb0 con 0x7efc94152990 2024-04-24T23:00:24.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.098+0000 7efc8b7fe640 1 -- 172.21.15.32:0/2807446555 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7c007270 con 0x7efc94152990 2024-04-24T23:00:24.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.098+0000 7efc9967b640 1 -- 172.21.15.32:0/2807446555 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc94152190 con 0x7efc94152990 2024-04-24T23:00:24.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.099+0000 7efc9967b640 1 -- 172.21.15.32:0/2807446555 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc94068430 con 0x7efc94152990 2024-04-24T23:00:24.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.099+0000 7efc8b7fe640 1 -- 172.21.15.32:0/2807446555 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efc7c007410 con 0x7efc94152990 2024-04-24T23:00:24.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.099+0000 7efc8b7fe640 1 --2- 172.21.15.32:0/2807446555 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7efc7406be30 0x7efc7406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:24.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.099+0000 7efc8b7fe640 1 -- 172.21.15.32:0/2807446555 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7efc7c0ab220 con 0x7efc94152990 2024-04-24T23:00:24.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.100+0000 7efc92ffd640 1 --2- 172.21.15.32:0/2807446555 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7efc7406be30 0x7efc7406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:24.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.100+0000 7efc92ffd640 1 --2- 172.21.15.32:0/2807446555 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7efc7406be30 0x7efc7406e2f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efc8c000c00 tx=0x7efc8c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:24.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.104+0000 7efc8b7fe640 1 -- 172.21.15.32:0/2807446555 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7efc7c049030 con 0x7efc94152990 2024-04-24T23:00:24.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.285+0000 7efc9967b640 1 -- 172.21.15.32:0/2807446555 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7efc940b1af0 con 0x7efc94152990 2024-04-24T23:00:24.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.288+0000 7efc8b7fe640 1 -- 172.21.15.32:0/2807446555 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v3) ==== 104+0+0 (secure 0 0 0) 0x7efc9415c990 con 0x7efc94152990 2024-04-24T23:00:24.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.290+0000 7efc9967b640 1 -- 172.21.15.32:0/2807446555 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7efc7406be30 msgr2=0x7efc7406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:24.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.290+0000 7efc9967b640 1 --2- 172.21.15.32:0/2807446555 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7efc7406be30 0x7efc7406e2f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efc8c000c00 tx=0x7efc8c009040 comp rx=0 tx=0).stop 2024-04-24T23:00:24.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.290+0000 7efc9967b640 1 -- 172.21.15.32:0/2807446555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc94152990 msgr2=0x7efc94152df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:24.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.290+0000 7efc9967b640 1 --2- 172.21.15.32:0/2807446555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc94152990 0x7efc94152df0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7efc7c00c4c0 tx=0x7efc94072100 comp rx=0 tx=0).stop 2024-04-24T23:00:24.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.290+0000 7efc9967b640 1 -- 172.21.15.32:0/2807446555 shutdown_connections 2024-04-24T23:00:24.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.290+0000 7efc9967b640 1 --2- 172.21.15.32:0/2807446555 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7efc7406be30 0x7efc7406e2f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:24.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.290+0000 7efc9967b640 1 --2- 172.21.15.32:0/2807446555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc94152990 0x7efc94152df0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:24.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.290+0000 7efc9967b640 1 --2- 172.21.15.32:0/2807446555 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efc940b37e0 0x7efc940b3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:24.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.290+0000 7efc9967b640 1 --2- 172.21.15.32:0/2807446555 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc80004800 0x7efc940aeb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:24.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.290+0000 7efc9967b640 1 -- 172.21.15.32:0/2807446555 >> 172.21.15.32:0/2807446555 conn(0x7efc9406b110 msgr2=0x7efc94159480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:24.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.291+0000 7efc9967b640 1 -- 172.21.15.32:0/2807446555 shutdown_connections 2024-04-24T23:00:24.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.291+0000 7efc9967b640 1 -- 172.21.15.32:0/2807446555 wait complete. 2024-04-24T23:00:24.306 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_barrier' 2024-04-24T23:00:24.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.404+0000 7fb7c6114640 1 Processor -- start 2024-04-24T23:00:24.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.404+0000 7fb7c6114640 1 -- start start 2024-04-24T23:00:24.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.405+0000 7fb7c6114640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb7c0162320 0x7fb7c00ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:24.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.405+0000 7fb7c6114640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb7c0162e10 0x7fb7c015ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:24.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.405+0000 7fb7c6114640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7c00cd270 con 0x7fb7c00b5af0 2024-04-24T23:00:24.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.405+0000 7fb7c6114640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fb7c00b5660 con 0x7fb7c0162320 2024-04-24T23:00:24.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.405+0000 7fb7c6114640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fb7c0071610 con 0x7fb7c0162e10 2024-04-24T23:00:24.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.405+0000 7fb7bf7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb7c0162320 0x7fb7c00ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:24.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.405+0000 7fb7beffd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb7c00b5af0 0x7fb7c0162700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33458/0 (socket says 172.21.15.32:33458) 2024-04-24T23:00:24.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.405+0000 7fb7beffd640 1 -- 172.21.15.32:0/1434803765 learned_addr learned my addr 172.21.15.32:0/1434803765 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:00:24.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.405+0000 7fb7bf7fe640 1 -- 172.21.15.32:0/1434803765 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb7c0162e10 msgr2=0x7fb7c015ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:24.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.405+0000 7fb7bf7fe640 1 --2- 172.21.15.32:0/1434803765 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb7c0162e10 0x7fb7c015ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:24.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.406+0000 7fb7bf7fe640 1 -- 172.21.15.32:0/1434803765 >> v1:172.21.15.32:6789/0 conn(0x7fb7c00b5af0 legacy=0x7fb7c0162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:24.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.406+0000 7fb7be7fc640 1 -- 172.21.15.32:0/1434803765 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2244014372 0 0) 0x7fb7c00cd270 con 0x7fb7c00b5af0 2024-04-24T23:00:24.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.406+0000 7fb7bf7fe640 1 -- 172.21.15.32:0/1434803765 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7c00718f0 con 0x7fb7c0162320 2024-04-24T23:00:24.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.406+0000 7fb7bf7fe640 1 --2- 172.21.15.32:0/1434803765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7c0162320 0x7fb7c00ccc90 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb7b00097a0 tx=0x7fb7b002a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1250135645420fc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:24.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.406+0000 7fb7be7fc640 1 -- 172.21.15.32:0/1434803765 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b0037070 con 0x7fb7c0162320 2024-04-24T23:00:24.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.406+0000 7fb7be7fc640 1 -- 172.21.15.32:0/1434803765 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7b002adb0 con 0x7fb7c0162320 2024-04-24T23:00:24.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.406+0000 7fb7be7fc640 1 -- 172.21.15.32:0/1434803765 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b0032660 con 0x7fb7c0162320 2024-04-24T23:00:24.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.407+0000 7fb7c6114640 1 -- 172.21.15.32:0/1434803765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7c0162320 msgr2=0x7fb7c00ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:24.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.407+0000 7fb7c6114640 1 --2- 172.21.15.32:0/1434803765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7c0162320 0x7fb7c00ccc90 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb7b00097a0 tx=0x7fb7b002a060 comp rx=0 tx=0).stop 2024-04-24T23:00:24.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.407+0000 7fb7c6114640 1 -- 172.21.15.32:0/1434803765 shutdown_connections 2024-04-24T23:00:24.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.407+0000 7fb7c6114640 1 --2- 172.21.15.32:0/1434803765 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb7c0162e10 0x7fb7c015ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:24.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.407+0000 7fb7c6114640 1 --2- 172.21.15.32:0/1434803765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7c0162320 0x7fb7c00ccc90 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:24.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.407+0000 7fb7c6114640 1 -- 172.21.15.32:0/1434803765 >> 172.21.15.32:0/1434803765 conn(0x7fb7c00d0ae0 msgr2=0x7fb7c00d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:24.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.407+0000 7fb7c6114640 1 -- 172.21.15.32:0/1434803765 shutdown_connections 2024-04-24T23:00:24.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.407+0000 7fb7c6114640 1 -- 172.21.15.32:0/1434803765 wait complete. 2024-04-24T23:00:24.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.408+0000 7fb7c6114640 1 Processor -- start 2024-04-24T23:00:24.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.408+0000 7fb7c6114640 1 -- start start 2024-04-24T23:00:24.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.409+0000 7fb7c6114640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb7c00b5af0 0x7fb7c014b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:24.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.409+0000 7fb7c6114640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7c0162320 0x7fb7c014bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:24.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.409+0000 7fb7c6114640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7c0162e10 0x7fb7c01609b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:24.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.409+0000 7fb7c6114640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c0071bd0 con 0x7fb7c0162e10 2024-04-24T23:00:24.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.409+0000 7fb7c6114640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c0067950 con 0x7fb7c0162320 2024-04-24T23:00:24.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.409+0000 7fb7c6114640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fb7c0071d50 con 0x7fb7c00b5af0 2024-04-24T23:00:24.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.409+0000 7fb7bffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7c0162e10 0x7fb7c01609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:24.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.409+0000 7fb7bffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7c0162e10 0x7fb7c01609b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48178/0 (socket says 172.21.15.32:48178) 2024-04-24T23:00:24.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.409+0000 7fb7bffff640 1 -- 172.21.15.32:0/3241256226 learned_addr learned my addr 172.21.15.32:0/3241256226 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:24.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.409+0000 7fb7bf7fe640 1 --2- 172.21.15.32:0/3241256226 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb7c00b5af0 0x7fb7c014b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:24.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.409+0000 7fb7beffd640 1 --2- 172.21.15.32:0/3241256226 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7c0162320 0x7fb7c014bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:24.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.410+0000 7fb7bffff640 1 -- 172.21.15.32:0/3241256226 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb7c00b5af0 msgr2=0x7fb7c014b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:24.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.410+0000 7fb7bffff640 1 --2- 172.21.15.32:0/3241256226 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb7c00b5af0 0x7fb7c014b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:24.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.410+0000 7fb7bffff640 1 -- 172.21.15.32:0/3241256226 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7c0162320 msgr2=0x7fb7c014bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:24.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.410+0000 7fb7bffff640 1 --2- 172.21.15.32:0/3241256226 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7c0162320 0x7fb7c014bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:24.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.410+0000 7fb7bffff640 1 -- 172.21.15.32:0/3241256226 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7c014baf0 con 0x7fb7c0162e10 2024-04-24T23:00:24.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.410+0000 7fb7beffd640 1 --2- 172.21.15.32:0/3241256226 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7c0162320 0x7fb7c014bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:00:24.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.410+0000 7fb7bffff640 1 --2- 172.21.15.32:0/3241256226 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7c0162e10 0x7fb7c01609b0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb7b800b9a0 tx=0x7fb7b800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:24.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.410+0000 7fb7a7fff640 1 -- 172.21.15.32:0/3241256226 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b8019070 con 0x7fb7c0162e10 2024-04-24T23:00:24.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.410+0000 7fb7a7fff640 1 -- 172.21.15.32:0/3241256226 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7b800cbc0 con 0x7fb7c0162e10 2024-04-24T23:00:24.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.410+0000 7fb7c6114640 1 -- 172.21.15.32:0/3241256226 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7c01e9fc0 con 0x7fb7c0162e10 2024-04-24T23:00:24.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.410+0000 7fb7a7fff640 1 -- 172.21.15.32:0/3241256226 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b8014440 con 0x7fb7c0162e10 2024-04-24T23:00:24.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.410+0000 7fb7c6114640 1 -- 172.21.15.32:0/3241256226 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7c00aed10 con 0x7fb7c0162e10 2024-04-24T23:00:24.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.411+0000 7fb7c6114640 1 -- 172.21.15.32:0/3241256226 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7c0068430 con 0x7fb7c0162e10 2024-04-24T23:00:24.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.411+0000 7fb7a7fff640 1 -- 172.21.15.32:0/3241256226 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb7b801c070 con 0x7fb7c0162e10 2024-04-24T23:00:24.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.412+0000 7fb7a7fff640 1 --2- 172.21.15.32:0/3241256226 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb7ac06be30 0x7fb7ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:24.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.412+0000 7fb7a7fff640 1 -- 172.21.15.32:0/3241256226 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7fb7b808a950 con 0x7fb7c0162e10 2024-04-24T23:00:24.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.412+0000 7fb7bf7fe640 1 --2- 172.21.15.32:0/3241256226 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb7ac06be30 0x7fb7ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:24.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.413+0000 7fb7bf7fe640 1 --2- 172.21.15.32:0/3241256226 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb7ac06be30 0x7fb7ac06e2f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0034000 tx=0x7fb7b002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:24.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.417+0000 7fb7a7fff640 1 -- 172.21.15.32:0/3241256226 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb7c014baf0 con 0x7fb7c0162e10 2024-04-24T23:00:24.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.671+0000 7fb7c6114640 1 -- 172.21.15.32:0/3241256226 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_barrier"]} v 0) -- 0x7fb7c0067730 con 0x7fb7c0162e10 2024-04-24T23:00:24.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.969+0000 7fb7a7fff640 1 -- 172.21.15.32:0/3241256226 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_barrier"]}]=0 v91) ==== 130+0+0 (secure 0 0 0) 0x7fb7b8059560 con 0x7fb7c0162e10 2024-04-24T23:00:24.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.972+0000 7fb7c6114640 1 -- 172.21.15.32:0/3241256226 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb7ac06be30 msgr2=0x7fb7ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:24.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.972+0000 7fb7c6114640 1 --2- 172.21.15.32:0/3241256226 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb7ac06be30 0x7fb7ac06e2f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0034000 tx=0x7fb7b002e000 comp rx=0 tx=0).stop 2024-04-24T23:00:24.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.972+0000 7fb7c6114640 1 -- 172.21.15.32:0/3241256226 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7c0162e10 msgr2=0x7fb7c01609b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:24.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.972+0000 7fb7c6114640 1 --2- 172.21.15.32:0/3241256226 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7c0162e10 0x7fb7c01609b0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb7b800b9a0 tx=0x7fb7b800be60 comp rx=0 tx=0).stop 2024-04-24T23:00:24.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.972+0000 7fb7c6114640 1 -- 172.21.15.32:0/3241256226 shutdown_connections 2024-04-24T23:00:24.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.972+0000 7fb7c6114640 1 --2- 172.21.15.32:0/3241256226 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb7ac06be30 0x7fb7ac06e2f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:24.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.972+0000 7fb7c6114640 1 --2- 172.21.15.32:0/3241256226 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7c0162e10 0x7fb7c01609b0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:24.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.972+0000 7fb7c6114640 1 --2- 172.21.15.32:0/3241256226 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7c0162320 0x7fb7c014bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:24.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.972+0000 7fb7c6114640 1 --2- 172.21.15.32:0/3241256226 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb7c00b5af0 0x7fb7c014b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:24.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.972+0000 7fb7c6114640 1 -- 172.21.15.32:0/3241256226 >> 172.21.15.32:0/3241256226 conn(0x7fb7c00d0ae0 msgr2=0x7fb7c00b3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:24.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.972+0000 7fb7c6114640 1 -- 172.21.15.32:0/3241256226 shutdown_connections 2024-04-24T23:00:24.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:24.972+0000 7fb7c6114640 1 -- 172.21.15.32:0/3241256226 wait complete. 2024-04-24T23:00:24.986 INFO:tasks.cephfs_test_runner:That when an OSD epoch barrier is set on an MDS, subsequently ... ok 2024-04-24T23:00:24.986 INFO:tasks.cephfs_test_runner:Starting test: test_full_different_file (tasks.cephfs.test_full.TestClusterFull) 2024-04-24T23:00:24.987 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:00:25.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.085+0000 7fb8302ae640 1 Processor -- start 2024-04-24T23:00:25.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.085+0000 7fb8302ae640 1 -- start start 2024-04-24T23:00:25.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.086+0000 7fb8302ae640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb82815cc40 0x7fb82815d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:25.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.086+0000 7fb8302ae640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8280d05b0 0x7fb82815d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:25.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.086+0000 7fb8302ae640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8280b4ee0 con 0x7fb8280cec30 2024-04-24T23:00:25.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.086+0000 7fb8302ae640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fb82806cca0 con 0x7fb82815cc40 2024-04-24T23:00:25.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.086+0000 7fb8302ae640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fb8280ce5f0 con 0x7fb8280d05b0 2024-04-24T23:00:25.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.087+0000 7fb82e04a640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb82815cc40 0x7fb82815d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:25.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.087+0000 7fb82d849640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8280d05b0 0x7fb82815d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:25.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.087+0000 7fb82e04a640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb82815cc40 0x7fb82815d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.32:35134/0 (socket says 172.21.15.32:35134) 2024-04-24T23:00:25.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.087+0000 7fb82e04a640 1 -- 172.21.15.32:0/4044866800 learned_addr learned my addr 172.21.15.32:0/4044866800 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:25.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.087+0000 7fb82e04a640 1 -- 172.21.15.32:0/4044866800 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8280d05b0 msgr2=0x7fb82815d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:25.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.087+0000 7fb82e04a640 1 --2- 172.21.15.32:0/4044866800 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8280d05b0 0x7fb82815d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:25.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.087+0000 7fb82e04a640 1 -- 172.21.15.32:0/4044866800 >> v1:172.21.15.121:6789/0 conn(0x7fb8280cec30 legacy=0x7fb8280cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:25.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.087+0000 7fb82e04a640 1 -- 172.21.15.32:0/4044866800 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8280b51c0 con 0x7fb82815cc40 2024-04-24T23:00:25.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.087+0000 7fb82d849640 1 --2- 172.21.15.32:0/4044866800 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8280d05b0 0x7fb82815d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:00:25.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.088+0000 7fb82e04a640 1 --2- 172.21.15.32:0/4044866800 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb82815cc40 0x7fb82815d020 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb820009870 tx=0x7fb82002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c1a1f0bbd71e503 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:25.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.088+0000 7fb82d048640 1 -- 172.21.15.32:0/4044866800 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb820037070 con 0x7fb82815cc40 2024-04-24T23:00:25.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.088+0000 7fb82d048640 1 -- 172.21.15.32:0/4044866800 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb820032440 con 0x7fb82815cc40 2024-04-24T23:00:25.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.088+0000 7fb82d048640 1 -- 172.21.15.32:0/4044866800 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb820032810 con 0x7fb82815cc40 2024-04-24T23:00:25.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.089+0000 7fb8302ae640 1 -- 172.21.15.32:0/4044866800 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb82815cc40 msgr2=0x7fb82815d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:25.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.089+0000 7fb8302ae640 1 --2- 172.21.15.32:0/4044866800 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb82815cc40 0x7fb82815d020 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb820009870 tx=0x7fb82002a270 comp rx=0 tx=0).stop 2024-04-24T23:00:25.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.089+0000 7fb8302ae640 1 -- 172.21.15.32:0/4044866800 shutdown_connections 2024-04-24T23:00:25.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.089+0000 7fb8302ae640 1 --2- 172.21.15.32:0/4044866800 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8280d05b0 0x7fb82815d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:25.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.089+0000 7fb8302ae640 1 --2- 172.21.15.32:0/4044866800 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb82815cc40 0x7fb82815d020 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:25.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.089+0000 7fb8302ae640 1 -- 172.21.15.32:0/4044866800 >> 172.21.15.32:0/4044866800 conn(0x7fb8280d0ae0 msgr2=0x7fb8280d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:25.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.089+0000 7fb8302ae640 1 -- 172.21.15.32:0/4044866800 shutdown_connections 2024-04-24T23:00:25.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.089+0000 7fb8302ae640 1 -- 172.21.15.32:0/4044866800 wait complete. 2024-04-24T23:00:25.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.090+0000 7fb8302ae640 1 Processor -- start 2024-04-24T23:00:25.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.090+0000 7fb8302ae640 1 -- start start 2024-04-24T23:00:25.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.091+0000 7fb8302ae640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8280cec30 0x7fb82813ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:25.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.091+0000 7fb8302ae640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8280d05b0 0x7fb82815e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:25.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.091+0000 7fb8302ae640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb82815cc40 0x7fb82815e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:25.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.091+0000 7fb8302ae640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb8280b5780 con 0x7fb82815cc40 2024-04-24T23:00:25.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.091+0000 7fb8302ae640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb82815daa0 con 0x7fb8280cec30 2024-04-24T23:00:25.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.091+0000 7fb8302ae640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fb8280b5a60 con 0x7fb8280d05b0 2024-04-24T23:00:25.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.091+0000 7fb82e84b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb82815cc40 0x7fb82815e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:25.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.091+0000 7fb82e84b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb82815cc40 0x7fb82815e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48194/0 (socket says 172.21.15.32:48194) 2024-04-24T23:00:25.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.091+0000 7fb82e84b640 1 -- 172.21.15.32:0/822866482 learned_addr learned my addr 172.21.15.32:0/822866482 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:25.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.091+0000 7fb82d849640 1 --2- 172.21.15.32:0/822866482 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8280d05b0 0x7fb82815e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:25.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.091+0000 7fb82e04a640 1 --2- 172.21.15.32:0/822866482 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8280cec30 0x7fb82813ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:25.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.091+0000 7fb82e84b640 1 -- 172.21.15.32:0/822866482 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8280d05b0 msgr2=0x7fb82815e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:25.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.091+0000 7fb82e84b640 1 --2- 172.21.15.32:0/822866482 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8280d05b0 0x7fb82815e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:25.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.091+0000 7fb82e84b640 1 -- 172.21.15.32:0/822866482 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8280cec30 msgr2=0x7fb82813ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:25.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.091+0000 7fb82e84b640 1 --2- 172.21.15.32:0/822866482 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8280cec30 0x7fb82813ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:25.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.091+0000 7fb82e84b640 1 -- 172.21.15.32:0/822866482 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb82813b370 con 0x7fb82815cc40 2024-04-24T23:00:25.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.091+0000 7fb82e04a640 1 --2- 172.21.15.32:0/822866482 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8280cec30 0x7fb82813ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:00:25.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.091+0000 7fb82e84b640 1 --2- 172.21.15.32:0/822866482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb82815cc40 0x7fb82815e970 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb81800c6b0 tx=0x7fb81800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:25.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.091+0000 7fb826ffd640 1 -- 172.21.15.32:0/822866482 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb818003b70 con 0x7fb82815cc40 2024-04-24T23:00:25.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.092+0000 7fb826ffd640 1 -- 172.21.15.32:0/822866482 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb818003d30 con 0x7fb82815cc40 2024-04-24T23:00:25.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.092+0000 7fb8302ae640 1 -- 172.21.15.32:0/822866482 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb82813e5d0 con 0x7fb82815cc40 2024-04-24T23:00:25.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.092+0000 7fb826ffd640 1 -- 172.21.15.32:0/822866482 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb818005310 con 0x7fb82815cc40 2024-04-24T23:00:25.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.092+0000 7fb8302ae640 1 -- 172.21.15.32:0/822866482 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8281f61a0 con 0x7fb82815cc40 2024-04-24T23:00:25.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.093+0000 7fb8302ae640 1 -- 172.21.15.32:0/822866482 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb828068430 con 0x7fb82815cc40 2024-04-24T23:00:25.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.093+0000 7fb826ffd640 1 -- 172.21.15.32:0/822866482 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb8180054b0 con 0x7fb82815cc40 2024-04-24T23:00:25.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.094+0000 7fb826ffd640 1 --2- 172.21.15.32:0/822866482 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb81406be30 0x7fb81406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:25.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.094+0000 7fb826ffd640 1 -- 172.21.15.32:0/822866482 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7fb81808a8f0 con 0x7fb82815cc40 2024-04-24T23:00:25.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.094+0000 7fb82e04a640 1 --2- 172.21.15.32:0/822866482 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb81406be30 0x7fb81406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:25.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.096+0000 7fb82e04a640 1 --2- 172.21.15.32:0/822866482 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb81406be30 0x7fb81406e2f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb820034000 tx=0x7fb82002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:25.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.099+0000 7fb826ffd640 1 -- 172.21.15.32:0/822866482 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb818058ed0 con 0x7fb82815cc40 2024-04-24T23:00:25.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.286+0000 7fb8302ae640 1 -- 172.21.15.32:0/822866482 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb828067c20 con 0x7fb82815cc40 2024-04-24T23:00:25.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.287+0000 7fb826ffd640 1 -- 172.21.15.32:0/822866482 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+13032 (secure 0 0 0) 0x7fb81805d1d0 con 0x7fb82815cc40 2024-04-24T23:00:25.289 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:00:25.289 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":37,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:00:23.965453+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.121:6837/515107694":"2024-04-25T23:00:20.957956+0000","172.21.15.121:6834/515107694":"2024-04-25T23:00:20.957956+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:00:25.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.290+0000 7fb8302ae640 1 -- 172.21.15.32:0/822866482 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb81406be30 msgr2=0x7fb81406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:25.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.290+0000 7fb8302ae640 1 --2- 172.21.15.32:0/822866482 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb81406be30 0x7fb81406e2f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb820034000 tx=0x7fb82002e000 comp rx=0 tx=0).stop 2024-04-24T23:00:25.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.290+0000 7fb8302ae640 1 -- 172.21.15.32:0/822866482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb82815cc40 msgr2=0x7fb82815e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:25.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.290+0000 7fb8302ae640 1 --2- 172.21.15.32:0/822866482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb82815cc40 0x7fb82815e970 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb81800c6b0 tx=0x7fb81800cb70 comp rx=0 tx=0).stop 2024-04-24T23:00:25.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.290+0000 7fb8302ae640 1 -- 172.21.15.32:0/822866482 shutdown_connections 2024-04-24T23:00:25.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.290+0000 7fb8302ae640 1 --2- 172.21.15.32:0/822866482 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb81406be30 0x7fb81406e2f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:25.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.290+0000 7fb8302ae640 1 --2- 172.21.15.32:0/822866482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb82815cc40 0x7fb82815e970 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:25.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.290+0000 7fb8302ae640 1 --2- 172.21.15.32:0/822866482 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8280d05b0 0x7fb82815e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:25.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.290+0000 7fb8302ae640 1 --2- 172.21.15.32:0/822866482 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8280cec30 0x7fb82813ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:25.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.290+0000 7fb8302ae640 1 -- 172.21.15.32:0/822866482 >> 172.21.15.32:0/822866482 conn(0x7fb8280d0ae0 msgr2=0x7fb8280ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:25.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.290+0000 7fb8302ae640 1 -- 172.21.15.32:0/822866482 shutdown_connections 2024-04-24T23:00:25.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.290+0000 7fb8302ae640 1 -- 172.21.15.32:0/822866482 wait complete. 2024-04-24T23:00:25.302 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:00:25.303 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:00:25.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.405+0000 7f0c8e2ba640 1 Processor -- start 2024-04-24T23:00:25.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.405+0000 7f0c8e2ba640 1 -- start start 2024-04-24T23:00:25.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.406+0000 7f0c8e2ba640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0c8815e9e0 0x7f0c880d40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:25.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.406+0000 7f0c8e2ba640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c880d3010 con 0x7f0c880ba390 2024-04-24T23:00:25.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.406+0000 7f0c8e2ba640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c880d32f0 con 0x7f0c8806b110 2024-04-24T23:00:25.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.406+0000 7f0c8e2ba640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f0c880d2db0 con 0x7f0c8815e9e0 2024-04-24T23:00:25.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.406+0000 7f0c877fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0c880ba390 0x7f0c880d26a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33468/0 (socket says 172.21.15.32:33468) 2024-04-24T23:00:25.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.406+0000 7f0c877fe640 1 -- 172.21.15.32:0/1398704740 learned_addr learned my addr 172.21.15.32:0/1398704740 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:00:25.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.407+0000 7f0c87fff640 1 --2- 172.21.15.32:0/1398704740 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0c8815e9e0 0x7f0c880d40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:25.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.407+0000 7f0c86ffd640 1 -- 172.21.15.32:0/1398704740 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 697691245 0 0) 0x7f0c880d3010 con 0x7f0c880ba390 2024-04-24T23:00:25.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.407+0000 7f0c86ffd640 1 -- 172.21.15.32:0/1398704740 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0c74003520 con 0x7f0c880ba390 2024-04-24T23:00:25.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.407+0000 7f0c86ffd640 1 -- 172.21.15.32:0/1398704740 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3411272470 0 0) 0x7f0c74003520 con 0x7f0c880ba390 2024-04-24T23:00:25.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.407+0000 7f0c86ffd640 1 -- 172.21.15.32:0/1398704740 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0c8815e9e0 msgr2=0x7f0c880d40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:25.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.407+0000 7f0c86ffd640 1 --2- 172.21.15.32:0/1398704740 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0c8815e9e0 0x7f0c880d40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:25.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.407+0000 7f0c86ffd640 1 -- 172.21.15.32:0/1398704740 >> v1:172.21.15.121:6789/0 conn(0x7f0c8806b110 legacy=0x7f0c880b7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:25.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.407+0000 7f0c86ffd640 1 -- 172.21.15.32:0/1398704740 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c880d35d0 con 0x7f0c880ba390 2024-04-24T23:00:25.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.407+0000 7f0c86ffd640 1 -- 172.21.15.32:0/1398704740 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f0c70001670 con 0x7f0c880ba390 2024-04-24T23:00:25.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.407+0000 7f0c87fff640 1 --2- 172.21.15.32:0/1398704740 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0c8815e9e0 0x7f0c880d40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:00:25.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.407+0000 7f0c86ffd640 1 -- 172.21.15.32:0/1398704740 >> v1:172.21.15.32:6789/0 conn(0x7f0c880ba390 legacy=0x7f0c880d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:25.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.407+0000 7f0c86ffd640 1 --2- 172.21.15.32:0/1398704740 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0c740039e0 0x7f0c74003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:25.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.407+0000 7f0c86ffd640 1 --2- 172.21.15.32:0/1398704740 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0c8815e9e0 0x7f0c74004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:25.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.407+0000 7f0c86ffd640 1 --2- 172.21.15.32:0/1398704740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c74004d40 0x7f0c740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:25.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.407+0000 7f0c86ffd640 1 -- 172.21.15.32:0/1398704740 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0c880d2db0 con 0x7f0c74004d40 2024-04-24T23:00:25.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.407+0000 7f0c86ffd640 1 -- 172.21.15.32:0/1398704740 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0c7400b9c0 con 0x7f0c740039e0 2024-04-24T23:00:25.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.407+0000 7f0c86ffd640 1 -- 172.21.15.32:0/1398704740 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f0c7400bb70 con 0x7f0c8815e9e0 2024-04-24T23:00:25.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.407+0000 7f0c8c857640 1 --2- 172.21.15.32:0/1398704740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c74004d40 0x7f0c740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:25.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.407+0000 7f0c87fff640 1 --2- 172.21.15.32:0/1398704740 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0c740039e0 0x7f0c74003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:25.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.407+0000 7f0c877fe640 1 --2- 172.21.15.32:0/1398704740 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0c8815e9e0 0x7f0c74004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:25.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.407+0000 7f0c8c857640 1 -- 172.21.15.32:0/1398704740 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0c8815e9e0 msgr2=0x7f0c74004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:25.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.407+0000 7f0c8c857640 1 --2- 172.21.15.32:0/1398704740 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0c8815e9e0 0x7f0c74004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:25.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.407+0000 7f0c8c857640 1 -- 172.21.15.32:0/1398704740 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0c740039e0 msgr2=0x7f0c74003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:25.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.407+0000 7f0c8c857640 1 --2- 172.21.15.32:0/1398704740 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0c740039e0 0x7f0c74003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:25.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.408+0000 7f0c8c857640 1 -- 172.21.15.32:0/1398704740 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c74003520 con 0x7f0c74004d40 2024-04-24T23:00:25.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.408+0000 7f0c8c857640 1 --2- 172.21.15.32:0/1398704740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c74004d40 0x7f0c740051c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f0c80001f60 tx=0x7f0c74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=867f905e27c22aad server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:25.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.408+0000 7f0c86ffd640 1 -- 172.21.15.32:0/1398704740 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c80001b30 con 0x7f0c74004d40 2024-04-24T23:00:25.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.408+0000 7f0c86ffd640 1 -- 172.21.15.32:0/1398704740 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c80031620 con 0x7f0c74004d40 2024-04-24T23:00:25.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.408+0000 7f0c86ffd640 1 -- 172.21.15.32:0/1398704740 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c80031a10 con 0x7f0c74004d40 2024-04-24T23:00:25.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.409+0000 7f0c8e2ba640 1 -- 172.21.15.32:0/1398704740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c74004d40 msgr2=0x7f0c740051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:25.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.409+0000 7f0c8e2ba640 1 --2- 172.21.15.32:0/1398704740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c74004d40 0x7f0c740051c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f0c80001f60 tx=0x7f0c74005830 comp rx=0 tx=0).stop 2024-04-24T23:00:25.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.409+0000 7f0c8c857640 1 -- 172.21.15.32:0/1398704740 reap_dead start 2024-04-24T23:00:25.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.409+0000 7f0c8e2ba640 1 -- 172.21.15.32:0/1398704740 shutdown_connections 2024-04-24T23:00:25.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.409+0000 7f0c8e2ba640 1 -- 172.21.15.32:0/1398704740 >> 172.21.15.32:0/1398704740 conn(0x7f0c880d12e0 msgr2=0x7f0c880d16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:25.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.409+0000 7f0c8e2ba640 1 -- 172.21.15.32:0/1398704740 shutdown_connections 2024-04-24T23:00:25.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.409+0000 7f0c8e2ba640 1 -- 172.21.15.32:0/1398704740 wait complete. 2024-04-24T23:00:25.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.410+0000 7f0c8e2ba640 1 Processor -- start 2024-04-24T23:00:25.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.410+0000 7f0c8e2ba640 1 -- start start 2024-04-24T23:00:25.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.411+0000 7f0c8e2ba640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0c8815c5a0 0x7f0c88160410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:25.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.411+0000 7f0c8e2ba640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0c8814f8b0 0x7f0c88146510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:25.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.411+0000 7f0c8e2ba640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c8815c980 0x7f0c88146a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:25.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.411+0000 7f0c8e2ba640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0c880d40b0 con 0x7f0c8815c980 2024-04-24T23:00:25.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.411+0000 7f0c8e2ba640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0c880d3b90 con 0x7f0c8815c5a0 2024-04-24T23:00:25.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.411+0000 7f0c8e2ba640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f0c880d4230 con 0x7f0c8814f8b0 2024-04-24T23:00:25.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.411+0000 7f0c8c857640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c8815c980 0x7f0c88146a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:25.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.411+0000 7f0c8c857640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c8815c980 0x7f0c88146a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48204/0 (socket says 172.21.15.32:48204) 2024-04-24T23:00:25.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.411+0000 7f0c8c857640 1 -- 172.21.15.32:0/630496611 learned_addr learned my addr 172.21.15.32:0/630496611 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:25.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.411+0000 7f0c87fff640 1 --2- 172.21.15.32:0/630496611 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0c8815c5a0 0x7f0c88160410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:25.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.411+0000 7f0c877fe640 1 --2- 172.21.15.32:0/630496611 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0c8814f8b0 0x7f0c88146510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:25.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.411+0000 7f0c8c857640 1 -- 172.21.15.32:0/630496611 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0c8814f8b0 msgr2=0x7f0c88146510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:25.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.411+0000 7f0c8c857640 1 --2- 172.21.15.32:0/630496611 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0c8814f8b0 0x7f0c88146510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:25.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.411+0000 7f0c8c857640 1 -- 172.21.15.32:0/630496611 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0c8815c5a0 msgr2=0x7f0c88160410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:25.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.411+0000 7f0c8c857640 1 --2- 172.21.15.32:0/630496611 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0c8815c5a0 0x7f0c88160410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:25.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.411+0000 7f0c8c857640 1 -- 172.21.15.32:0/630496611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c88160ba0 con 0x7f0c8815c980 2024-04-24T23:00:25.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.411+0000 7f0c87fff640 1 --2- 172.21.15.32:0/630496611 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0c8815c5a0 0x7f0c88160410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:00:25.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.412+0000 7f0c8c857640 1 --2- 172.21.15.32:0/630496611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c8815c980 0x7f0c88146a50 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0c80002050 tx=0x7f0c800015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:25.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.412+0000 7f0c84ff9640 1 -- 172.21.15.32:0/630496611 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c80043070 con 0x7f0c8815c980 2024-04-24T23:00:25.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.412+0000 7f0c84ff9640 1 -- 172.21.15.32:0/630496611 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c80031be0 con 0x7f0c8815c980 2024-04-24T23:00:25.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.412+0000 7f0c8e2ba640 1 -- 172.21.15.32:0/630496611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c88160e80 con 0x7f0c8815c980 2024-04-24T23:00:25.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.412+0000 7f0c84ff9640 1 -- 172.21.15.32:0/630496611 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c80001b30 con 0x7f0c8815c980 2024-04-24T23:00:25.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.412+0000 7f0c8e2ba640 1 -- 172.21.15.32:0/630496611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c88161440 con 0x7f0c8815c980 2024-04-24T23:00:25.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.413+0000 7f0c8e2ba640 1 -- 172.21.15.32:0/630496611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c8806b5c0 con 0x7f0c8815c980 2024-04-24T23:00:25.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.413+0000 7f0c84ff9640 1 -- 172.21.15.32:0/630496611 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0c80046070 con 0x7f0c8815c980 2024-04-24T23:00:25.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.414+0000 7f0c84ff9640 1 --2- 172.21.15.32:0/630496611 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0c6806be30 0x7f0c6806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:25.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.414+0000 7f0c84ff9640 1 -- 172.21.15.32:0/630496611 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f0c800aabf0 con 0x7f0c8815c980 2024-04-24T23:00:25.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.414+0000 7f0c87fff640 1 --2- 172.21.15.32:0/630496611 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0c6806be30 0x7f0c6806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:25.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.415+0000 7f0c87fff640 1 --2- 172.21.15.32:0/630496611 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0c6806be30 0x7f0c6806e2f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0c78009ab0 tx=0x7f0c78008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:25.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.418+0000 7f0c84ff9640 1 -- 172.21.15.32:0/630496611 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0c8806b5c0 con 0x7f0c8815c980 2024-04-24T23:00:25.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.601+0000 7f0c8e2ba640 1 -- 172.21.15.32:0/630496611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0c88068430 con 0x7f0c8815c980 2024-04-24T23:00:25.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.602+0000 7f0c84ff9640 1 -- 172.21.15.32:0/630496611 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) ==== 93+0+10 (secure 0 0 0) 0x7f0c80079140 con 0x7f0c8815c980 2024-04-24T23:00:25.603 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T23:00:25.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.603+0000 7f0c8e2ba640 1 -- 172.21.15.32:0/630496611 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0c6806be30 msgr2=0x7f0c6806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:25.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.603+0000 7f0c8e2ba640 1 --2- 172.21.15.32:0/630496611 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0c6806be30 0x7f0c6806e2f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0c78009ab0 tx=0x7f0c78008040 comp rx=0 tx=0).stop 2024-04-24T23:00:25.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.603+0000 7f0c8e2ba640 1 -- 172.21.15.32:0/630496611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c8815c980 msgr2=0x7f0c88146a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:25.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.603+0000 7f0c8e2ba640 1 --2- 172.21.15.32:0/630496611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c8815c980 0x7f0c88146a50 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0c80002050 tx=0x7f0c800015b0 comp rx=0 tx=0).stop 2024-04-24T23:00:25.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.604+0000 7f0c8e2ba640 1 -- 172.21.15.32:0/630496611 shutdown_connections 2024-04-24T23:00:25.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.604+0000 7f0c8e2ba640 1 --2- 172.21.15.32:0/630496611 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0c6806be30 0x7f0c6806e2f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:25.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.604+0000 7f0c8e2ba640 1 --2- 172.21.15.32:0/630496611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c8815c980 0x7f0c88146a50 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:25.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.604+0000 7f0c8e2ba640 1 --2- 172.21.15.32:0/630496611 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0c8814f8b0 0x7f0c88146510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:25.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.604+0000 7f0c8e2ba640 1 --2- 172.21.15.32:0/630496611 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0c8815c5a0 0x7f0c88160410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:25.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.604+0000 7f0c8e2ba640 1 -- 172.21.15.32:0/630496611 >> 172.21.15.32:0/630496611 conn(0x7f0c880d12e0 msgr2=0x7f0c880d16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:25.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.604+0000 7f0c8e2ba640 1 -- 172.21.15.32:0/630496611 shutdown_connections 2024-04-24T23:00:25.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.604+0000 7f0c8e2ba640 1 -- 172.21.15.32:0/630496611 wait complete. 2024-04-24T23:00:25.616 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_different_file' 2024-04-24T23:00:25.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.716+0000 7fb5f719e640 1 Processor -- start 2024-04-24T23:00:25.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.716+0000 7fb5f719e640 1 -- start start 2024-04-24T23:00:25.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.716+0000 7fb5f719e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb5f0158db0 0x7fb5f015fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:25.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.717+0000 7fb5f719e640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb5f015f5b0 0x7fb5f00d05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:25.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.717+0000 7fb5f719e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb5f00d0b00 0x7fb5f006ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:25.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.717+0000 7fb5f719e640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fb5f006e620 con 0x7fb5f00d0b00 2024-04-24T23:00:25.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.717+0000 7fb5f719e640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fb5f0070b70 con 0x7fb5f015f5b0 2024-04-24T23:00:25.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.717+0000 7fb5f719e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fb5f0070e50 con 0x7fb5f0158db0 2024-04-24T23:00:25.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.717+0000 7fb5f699d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb5f00d0b00 0x7fb5f006ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:25.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.717+0000 7fb5f699d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb5f00d0b00 0x7fb5f006ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48216/0 (socket says 172.21.15.32:48216) 2024-04-24T23:00:25.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.717+0000 7fb5f699d640 1 -- 172.21.15.32:0/3893355075 learned_addr learned my addr 172.21.15.32:0/3893355075 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:25.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.717+0000 7fb5f599b640 1 --2- 172.21.15.32:0/3893355075 >> v2:172.21.15.121:3300/0 conn(0x7fb5f015f5b0 0x7fb5f00d05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:25.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.717+0000 7fb5f619c640 1 --2- 172.21.15.32:0/3893355075 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb5f0158db0 0x7fb5f015fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:25.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.717+0000 7fb5f699d640 1 -- 172.21.15.32:0/3893355075 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb5f0158db0 msgr2=0x7fb5f015fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:25.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.717+0000 7fb5f699d640 1 --2- 172.21.15.32:0/3893355075 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb5f0158db0 0x7fb5f015fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:25.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.717+0000 7fb5f699d640 1 -- 172.21.15.32:0/3893355075 >> v2:172.21.15.121:3300/0 conn(0x7fb5f015f5b0 msgr2=0x7fb5f00d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:25.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.717+0000 7fb5f699d640 1 --2- 172.21.15.32:0/3893355075 >> v2:172.21.15.121:3300/0 conn(0x7fb5f015f5b0 0x7fb5f00d05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:25.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.717+0000 7fb5f699d640 1 -- 172.21.15.32:0/3893355075 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5f0072a20 con 0x7fb5f00d0b00 2024-04-24T23:00:25.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.718+0000 7fb5f699d640 1 --2- 172.21.15.32:0/3893355075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5f00d0b00 0x7fb5f006ca30 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fb5f006c000 tx=0x7fb5e802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4676bcb620e4fd7c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:25.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.718+0000 7fb5f519a640 1 -- 172.21.15.32:0/3893355075 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e8036070 con 0x7fb5f00d0b00 2024-04-24T23:00:25.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.718+0000 7fb5f519a640 1 -- 172.21.15.32:0/3893355075 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5e8031600 con 0x7fb5f00d0b00 2024-04-24T23:00:25.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.718+0000 7fb5f519a640 1 -- 172.21.15.32:0/3893355075 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e80319d0 con 0x7fb5f00d0b00 2024-04-24T23:00:25.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.718+0000 7fb5f719e640 1 -- 172.21.15.32:0/3893355075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5f00d0b00 msgr2=0x7fb5f006ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:25.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.718+0000 7fb5f719e640 1 --2- 172.21.15.32:0/3893355075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5f00d0b00 0x7fb5f006ca30 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fb5f006c000 tx=0x7fb5e802a6a0 comp rx=0 tx=0).stop 2024-04-24T23:00:25.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.718+0000 7fb5f719e640 1 -- 172.21.15.32:0/3893355075 shutdown_connections 2024-04-24T23:00:25.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.718+0000 7fb5f719e640 1 --2- 172.21.15.32:0/3893355075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5f00d0b00 0x7fb5f006ca30 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:25.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.718+0000 7fb5f719e640 1 --2- 172.21.15.32:0/3893355075 >> v2:172.21.15.121:3300/0 conn(0x7fb5f015f5b0 0x7fb5f00d05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:25.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.718+0000 7fb5f719e640 1 --2- 172.21.15.32:0/3893355075 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb5f0158db0 0x7fb5f015fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:25.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.718+0000 7fb5f719e640 1 -- 172.21.15.32:0/3893355075 >> 172.21.15.32:0/3893355075 conn(0x7fb5f006b110 msgr2=0x7fb5f006b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:25.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.719+0000 7fb5f719e640 1 -- 172.21.15.32:0/3893355075 shutdown_connections 2024-04-24T23:00:25.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.719+0000 7fb5f719e640 1 -- 172.21.15.32:0/3893355075 wait complete. 2024-04-24T23:00:25.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.720+0000 7fb5f719e640 1 Processor -- start 2024-04-24T23:00:25.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.720+0000 7fb5f719e640 1 -- start start 2024-04-24T23:00:25.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.721+0000 7fb5f719e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb5f00d0b00 0x7fb5f01f3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:25.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.721+0000 7fb5f719e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb5f0158db0 0x7fb5f00b52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:25.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.721+0000 7fb5f719e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5f015f5b0 0x7fb5f00b57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:25.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.721+0000 7fb5f719e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f006e400 con 0x7fb5f015f5b0 2024-04-24T23:00:25.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.721+0000 7fb5f719e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f006e280 con 0x7fb5f0158db0 2024-04-24T23:00:25.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.721+0000 7fb5f719e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fb5f0071130 con 0x7fb5f00d0b00 2024-04-24T23:00:25.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.721+0000 7fb5f699d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5f015f5b0 0x7fb5f00b57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:25.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.721+0000 7fb5f699d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5f015f5b0 0x7fb5f00b57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48228/0 (socket says 172.21.15.32:48228) 2024-04-24T23:00:25.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.721+0000 7fb5f699d640 1 -- 172.21.15.32:0/1665067768 learned_addr learned my addr 172.21.15.32:0/1665067768 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:25.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.721+0000 7fb5f599b640 1 --2- 172.21.15.32:0/1665067768 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb5f0158db0 0x7fb5f00b52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:25.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.721+0000 7fb5f619c640 1 --2- 172.21.15.32:0/1665067768 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb5f00d0b00 0x7fb5f01f3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:25.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.721+0000 7fb5f699d640 1 -- 172.21.15.32:0/1665067768 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb5f00d0b00 msgr2=0x7fb5f01f3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:25.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.721+0000 7fb5f699d640 1 --2- 172.21.15.32:0/1665067768 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb5f00d0b00 0x7fb5f01f3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:25.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.721+0000 7fb5f699d640 1 -- 172.21.15.32:0/1665067768 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb5f0158db0 msgr2=0x7fb5f00b52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:25.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.721+0000 7fb5f699d640 1 --2- 172.21.15.32:0/1665067768 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb5f0158db0 0x7fb5f00b52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:25.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.721+0000 7fb5f699d640 1 -- 172.21.15.32:0/1665067768 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5f013ec70 con 0x7fb5f015f5b0 2024-04-24T23:00:25.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.721+0000 7fb5f699d640 1 --2- 172.21.15.32:0/1665067768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5f015f5b0 0x7fb5f00b57e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8034040 tx=0x7fb5e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:25.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.721+0000 7fb5e6ffd640 1 -- 172.21.15.32:0/1665067768 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e8041020 con 0x7fb5f015f5b0 2024-04-24T23:00:25.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.721+0000 7fb5e6ffd640 1 -- 172.21.15.32:0/1665067768 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5e80077a0 con 0x7fb5f015f5b0 2024-04-24T23:00:25.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.721+0000 7fb5f719e640 1 -- 172.21.15.32:0/1665067768 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5f013ef50 con 0x7fb5f015f5b0 2024-04-24T23:00:25.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.721+0000 7fb5e6ffd640 1 -- 172.21.15.32:0/1665067768 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e8036040 con 0x7fb5f015f5b0 2024-04-24T23:00:25.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.721+0000 7fb5f719e640 1 -- 172.21.15.32:0/1665067768 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5f00b2900 con 0x7fb5f015f5b0 2024-04-24T23:00:25.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.722+0000 7fb5f719e640 1 -- 172.21.15.32:0/1665067768 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5f00682b0 con 0x7fb5f015f5b0 2024-04-24T23:00:25.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.723+0000 7fb5e6ffd640 1 -- 172.21.15.32:0/1665067768 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb5e8044020 con 0x7fb5f015f5b0 2024-04-24T23:00:25.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.723+0000 7fb5e6ffd640 1 --2- 172.21.15.32:0/1665067768 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb5d406be30 0x7fb5d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:25.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.723+0000 7fb5e6ffd640 1 -- 172.21.15.32:0/1665067768 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7fb5e80ab880 con 0x7fb5f015f5b0 2024-04-24T23:00:25.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.723+0000 7fb5f619c640 1 --2- 172.21.15.32:0/1665067768 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb5d406be30 0x7fb5d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:25.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.724+0000 7fb5f619c640 1 --2- 172.21.15.32:0/1665067768 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb5d406be30 0x7fb5d406e2f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec006fd0 tx=0x7fb5ec008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:25.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.727+0000 7fb5e6ffd640 1 -- 172.21.15.32:0/1665067768 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb5e804b030 con 0x7fb5f015f5b0 2024-04-24T23:00:25.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:25.981+0000 7fb5f719e640 1 -- 172.21.15.32:0/1665067768 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]} v 0) -- 0x7fb5f01f3da0 con 0x7fb5f015f5b0 2024-04-24T23:00:26.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:26.975+0000 7fb5e6ffd640 1 -- 172.21.15.32:0/1665067768 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]}]=0 v93) ==== 145+0+0 (secure 0 0 0) 0x7fb5e8079db0 con 0x7fb5f015f5b0 2024-04-24T23:00:26.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:26.978+0000 7fb5f719e640 1 -- 172.21.15.32:0/1665067768 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb5d406be30 msgr2=0x7fb5d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:26.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:26.978+0000 7fb5f719e640 1 --2- 172.21.15.32:0/1665067768 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb5d406be30 0x7fb5d406e2f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec006fd0 tx=0x7fb5ec008040 comp rx=0 tx=0).stop 2024-04-24T23:00:26.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:26.978+0000 7fb5f719e640 1 -- 172.21.15.32:0/1665067768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5f015f5b0 msgr2=0x7fb5f00b57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:26.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:26.978+0000 7fb5f719e640 1 --2- 172.21.15.32:0/1665067768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5f015f5b0 0x7fb5f00b57e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8034040 tx=0x7fb5e8004a40 comp rx=0 tx=0).stop 2024-04-24T23:00:26.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:26.978+0000 7fb5f719e640 1 -- 172.21.15.32:0/1665067768 shutdown_connections 2024-04-24T23:00:26.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:26.978+0000 7fb5f719e640 1 --2- 172.21.15.32:0/1665067768 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb5d406be30 0x7fb5d406e2f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:26.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:26.978+0000 7fb5f719e640 1 --2- 172.21.15.32:0/1665067768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5f015f5b0 0x7fb5f00b57e0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:26.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:26.978+0000 7fb5f719e640 1 --2- 172.21.15.32:0/1665067768 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb5f0158db0 0x7fb5f00b52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:26.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:26.978+0000 7fb5f719e640 1 --2- 172.21.15.32:0/1665067768 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb5f00d0b00 0x7fb5f01f3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:26.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:26.978+0000 7fb5f719e640 1 -- 172.21.15.32:0/1665067768 >> 172.21.15.32:0/1665067768 conn(0x7fb5f006b110 msgr2=0x7fb5f00b1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:26.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:26.979+0000 7fb5f719e640 1 -- 172.21.15.32:0/1665067768 shutdown_connections 2024-04-24T23:00:26.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:26.979+0000 7fb5f719e640 1 -- 172.21.15.32:0/1665067768 wait complete. 2024-04-24T23:00:26.994 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2024-04-24T23:00:27.113 INFO:teuthology.orchestra.run.smithi032.stdout:{"osd_objectstore":"memstore"} 2024-04-24T23:00:27.114 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2024-04-24T23:00:27.114 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-24T23:00:27.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.217+0000 7f6f41510640 1 Processor -- start 2024-04-24T23:00:27.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.217+0000 7f6f41510640 1 -- start start 2024-04-24T23:00:27.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.218+0000 7f6f41510640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6f3c15e9e0 0x7f6f3c0d40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:27.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.218+0000 7f6f41510640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6f3c0ba390 0x7f6f3c0d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:27.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.218+0000 7f6f41510640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f3c06b110 0x7f6f3c06b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:27.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.218+0000 7f6f41510640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f6f3c0d3010 con 0x7f6f3c15e9e0 2024-04-24T23:00:27.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.218+0000 7f6f41510640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f6f3c0d32f0 con 0x7f6f3c0ba390 2024-04-24T23:00:27.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.218+0000 7f6f41510640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f6f3c0d35d0 con 0x7f6f3c06b110 2024-04-24T23:00:27.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.218+0000 7f6f3affd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6f3c15e9e0 0x7f6f3c0d40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:27.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.218+0000 7f6f3affd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6f3c15e9e0 0x7f6f3c0d40b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48230/0 (socket says 172.21.15.32:48230) 2024-04-24T23:00:27.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.218+0000 7f6f3affd640 1 -- 172.21.15.32:0/3692760505 learned_addr learned my addr 172.21.15.32:0/3692760505 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:27.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.218+0000 7f6f3affd640 1 -- 172.21.15.32:0/3692760505 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f3c06b110 msgr2=0x7f6f3c06b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:27.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.218+0000 7f6f3b7fe640 1 --2- 172.21.15.32:0/3692760505 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f3c06b110 0x7f6f3c06b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:27.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.218+0000 7f6f3a7fc640 1 --2- 172.21.15.32:0/3692760505 >> v2:172.21.15.121:3300/0 conn(0x7f6f3c0ba390 0x7f6f3c0d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:27.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.219+0000 7f6f3affd640 1 --2- 172.21.15.32:0/3692760505 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f3c06b110 0x7f6f3c06b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:27.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.219+0000 7f6f3affd640 1 -- 172.21.15.32:0/3692760505 >> v2:172.21.15.121:3300/0 conn(0x7f6f3c0ba390 msgr2=0x7f6f3c0d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:27.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.219+0000 7f6f3affd640 1 --2- 172.21.15.32:0/3692760505 >> v2:172.21.15.121:3300/0 conn(0x7f6f3c0ba390 0x7f6f3c0d26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:27.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.219+0000 7f6f3affd640 1 -- 172.21.15.32:0/3692760505 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f3c0d38b0 con 0x7f6f3c15e9e0 2024-04-24T23:00:27.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.219+0000 7f6f3affd640 1 --2- 172.21.15.32:0/3692760505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f3c15e9e0 0x7f6f3c0d40b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c0b71d0 tx=0x7f6f2c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=812d3b6ffc7fce6c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:27.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.219+0000 7f6f39ffb640 1 -- 172.21.15.32:0/3692760505 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c036070 con 0x7f6f3c15e9e0 2024-04-24T23:00:27.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.219+0000 7f6f39ffb640 1 -- 172.21.15.32:0/3692760505 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f2c031600 con 0x7f6f3c15e9e0 2024-04-24T23:00:27.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.219+0000 7f6f39ffb640 1 -- 172.21.15.32:0/3692760505 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c0319d0 con 0x7f6f3c15e9e0 2024-04-24T23:00:27.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.220+0000 7f6f41510640 1 -- 172.21.15.32:0/3692760505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f3c15e9e0 msgr2=0x7f6f3c0d40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:27.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.220+0000 7f6f41510640 1 --2- 172.21.15.32:0/3692760505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f3c15e9e0 0x7f6f3c0d40b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c0b71d0 tx=0x7f6f2c02a6a0 comp rx=0 tx=0).stop 2024-04-24T23:00:27.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.220+0000 7f6f41510640 1 -- 172.21.15.32:0/3692760505 shutdown_connections 2024-04-24T23:00:27.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.220+0000 7f6f41510640 1 --2- 172.21.15.32:0/3692760505 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f3c06b110 0x7f6f3c06b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:27.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.220+0000 7f6f41510640 1 --2- 172.21.15.32:0/3692760505 >> v2:172.21.15.121:3300/0 conn(0x7f6f3c0ba390 0x7f6f3c0d26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:27.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.220+0000 7f6f41510640 1 --2- 172.21.15.32:0/3692760505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f3c15e9e0 0x7f6f3c0d40b0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:27.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.220+0000 7f6f41510640 1 -- 172.21.15.32:0/3692760505 >> 172.21.15.32:0/3692760505 conn(0x7f6f3c0d12e0 msgr2=0x7f6f3c0d16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:27.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.220+0000 7f6f41510640 1 -- 172.21.15.32:0/3692760505 shutdown_connections 2024-04-24T23:00:27.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.220+0000 7f6f41510640 1 -- 172.21.15.32:0/3692760505 wait complete. 2024-04-24T23:00:27.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.221+0000 7f6f41510640 1 Processor -- start 2024-04-24T23:00:27.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.221+0000 7f6f41510640 1 -- start start 2024-04-24T23:00:27.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.222+0000 7f6f41510640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f3c06b110 0x7f6f3c1618d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:27.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.222+0000 7f6f41510640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f3c0ba390 0x7f6f3c14d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:27.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.222+0000 7f6f41510640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f3c15e9e0 0x7f6f3c0cce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:27.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.222+0000 7f6f41510640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3c0d2d60 con 0x7f6f3c0ba390 2024-04-24T23:00:27.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.222+0000 7f6f41510640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3c0d2be0 con 0x7f6f3c06b110 2024-04-24T23:00:27.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.222+0000 7f6f41510640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f6f3c071730 con 0x7f6f3c15e9e0 2024-04-24T23:00:27.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.222+0000 7f6f3a7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f3c0ba390 0x7f6f3c14d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:27.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.222+0000 7f6f3a7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f3c0ba390 0x7f6f3c14d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48238/0 (socket says 172.21.15.32:48238) 2024-04-24T23:00:27.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.222+0000 7f6f3a7fc640 1 -- 172.21.15.32:0/3989938870 learned_addr learned my addr 172.21.15.32:0/3989938870 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:27.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.222+0000 7f6f3affd640 1 --2- 172.21.15.32:0/3989938870 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f3c06b110 0x7f6f3c1618d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:27.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.222+0000 7f6f3b7fe640 1 --2- 172.21.15.32:0/3989938870 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f3c15e9e0 0x7f6f3c0cce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:27.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.222+0000 7f6f3a7fc640 1 -- 172.21.15.32:0/3989938870 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f3c15e9e0 msgr2=0x7f6f3c0cce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:27.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.222+0000 7f6f3a7fc640 1 --2- 172.21.15.32:0/3989938870 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f3c15e9e0 0x7f6f3c0cce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:27.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.222+0000 7f6f3a7fc640 1 -- 172.21.15.32:0/3989938870 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f3c06b110 msgr2=0x7f6f3c1618d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:27.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.222+0000 7f6f3a7fc640 1 --2- 172.21.15.32:0/3989938870 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f3c06b110 0x7f6f3c1618d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:27.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.222+0000 7f6f3a7fc640 1 -- 172.21.15.32:0/3989938870 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f3c0cba80 con 0x7f6f3c0ba390 2024-04-24T23:00:27.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.223+0000 7f6f3a7fc640 1 --2- 172.21.15.32:0/3989938870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f3c0ba390 0x7f6f3c14d310 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f6f2400c7f0 tx=0x7f6f2400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:27.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.223+0000 7f6f337fe640 1 -- 172.21.15.32:0/3989938870 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f24015070 con 0x7f6f3c0ba390 2024-04-24T23:00:27.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.223+0000 7f6f337fe640 1 -- 172.21.15.32:0/3989938870 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f24004240 con 0x7f6f3c0ba390 2024-04-24T23:00:27.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.223+0000 7f6f337fe640 1 -- 172.21.15.32:0/3989938870 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f24007620 con 0x7f6f3c0ba390 2024-04-24T23:00:27.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.223+0000 7f6f41510640 1 -- 172.21.15.32:0/3989938870 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f3c0cbd60 con 0x7f6f3c0ba390 2024-04-24T23:00:27.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.223+0000 7f6f41510640 1 -- 172.21.15.32:0/3989938870 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f3c0cc320 con 0x7f6f3c0ba390 2024-04-24T23:00:27.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.224+0000 7f6f41510640 1 -- 172.21.15.32:0/3989938870 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f3c068430 con 0x7f6f3c0ba390 2024-04-24T23:00:27.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.224+0000 7f6f337fe640 1 -- 172.21.15.32:0/3989938870 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6f240079d0 con 0x7f6f3c0ba390 2024-04-24T23:00:27.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.225+0000 7f6f337fe640 1 --2- 172.21.15.32:0/3989938870 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6f1c06be30 0x7f6f1c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:27.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.225+0000 7f6f337fe640 1 -- 172.21.15.32:0/3989938870 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f6f2408af50 con 0x7f6f3c0ba390 2024-04-24T23:00:27.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.225+0000 7f6f3affd640 1 --2- 172.21.15.32:0/3989938870 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6f1c06be30 0x7f6f1c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:27.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.225+0000 7f6f3affd640 1 --2- 172.21.15.32:0/3989938870 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6f1c06be30 0x7f6f1c06e2f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c033000 tx=0x7f6f2c009000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:27.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.229+0000 7f6f337fe640 1 -- 172.21.15.32:0/3989938870 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6f240595b0 con 0x7f6f3c0ba390 2024-04-24T23:00:27.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.412+0000 7f6f41510640 1 -- 172.21.15.32:0/3989938870 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f6f3c0cfab0 con 0x7f6f3c0ba390 2024-04-24T23:00:27.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.415+0000 7f6f337fe640 1 -- 172.21.15.32:0/3989938870 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v4)=0 v4) ==== 122+0+0 (secure 0 0 0) 0x7f6f2405d8b0 con 0x7f6f3c0ba390 2024-04-24T23:00:27.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.417+0000 7f6f41510640 1 -- 172.21.15.32:0/3989938870 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6f1c06be30 msgr2=0x7f6f1c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:27.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.417+0000 7f6f41510640 1 --2- 172.21.15.32:0/3989938870 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6f1c06be30 0x7f6f1c06e2f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c033000 tx=0x7f6f2c009000 comp rx=0 tx=0).stop 2024-04-24T23:00:27.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.417+0000 7f6f41510640 1 -- 172.21.15.32:0/3989938870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f3c0ba390 msgr2=0x7f6f3c14d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:27.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.417+0000 7f6f41510640 1 --2- 172.21.15.32:0/3989938870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f3c0ba390 0x7f6f3c14d310 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f6f2400c7f0 tx=0x7f6f2400ccb0 comp rx=0 tx=0).stop 2024-04-24T23:00:27.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.417+0000 7f6f41510640 1 -- 172.21.15.32:0/3989938870 shutdown_connections 2024-04-24T23:00:27.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.417+0000 7f6f41510640 1 --2- 172.21.15.32:0/3989938870 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6f1c06be30 0x7f6f1c06e2f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:27.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.417+0000 7f6f41510640 1 --2- 172.21.15.32:0/3989938870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f3c0ba390 0x7f6f3c14d310 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:27.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.417+0000 7f6f41510640 1 --2- 172.21.15.32:0/3989938870 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f3c15e9e0 0x7f6f3c0cce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:27.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.417+0000 7f6f41510640 1 --2- 172.21.15.32:0/3989938870 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f3c06b110 0x7f6f3c1618d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:27.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.417+0000 7f6f41510640 1 -- 172.21.15.32:0/3989938870 >> 172.21.15.32:0/3989938870 conn(0x7f6f3c0d12e0 msgr2=0x7f6f3c15d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:27.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.417+0000 7f6f41510640 1 -- 172.21.15.32:0/3989938870 shutdown_connections 2024-04-24T23:00:27.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.417+0000 7f6f41510640 1 -- 172.21.15.32:0/3989938870 wait complete. 2024-04-24T23:00:27.429 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-24T23:00:27.429 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:00:27.462 DEBUG:teuthology.orchestra.run.smithi121:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:00:27.505 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-24T23:00:27.505 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:00:27.506 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:00:27.521 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=413 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T23:00:27.521 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:00:27.522 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:00:27.563 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3271 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T23:00:27.563 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:00:27.563 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:00:27.578 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=413 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T23:00:27.578 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:00:27.579 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:00:27.621 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3271 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T23:00:27.621 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-24T23:00:27.621 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:00:27.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.721+0000 7fd0139d5640 1 Processor -- start 2024-04-24T23:00:27.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.721+0000 7fd0139d5640 1 -- start start 2024-04-24T23:00:27.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.722+0000 7fd0139d5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd00c15e9b0 0x7fd00c0d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:27.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.722+0000 7fd0139d5640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd00c0d3170 con 0x7fd00c0d4210 2024-04-24T23:00:27.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.722+0000 7fd0139d5640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd00c0d3450 con 0x7fd00c06b820 2024-04-24T23:00:27.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.722+0000 7fd0139d5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fd00c0d3730 con 0x7fd00c15e9b0 2024-04-24T23:00:27.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.722+0000 7fd011771640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd00c15e9b0 0x7fd00c0d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:27.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.722+0000 7fd010f70640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd00c0d4210 0x7fd00c06b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33476/0 (socket says 172.21.15.32:33476) 2024-04-24T23:00:27.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.722+0000 7fd010f70640 1 -- 172.21.15.32:0/2377866955 learned_addr learned my addr 172.21.15.32:0/2377866955 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:00:27.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.723+0000 7fd011771640 1 -- 172.21.15.32:0/2377866955 >> v1:172.21.15.121:6789/0 conn(0x7fd00c06b820 legacy=0x7fd00c0b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:27.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.723+0000 7fd011771640 1 -- 172.21.15.32:0/2377866955 >> v1:172.21.15.32:6789/0 conn(0x7fd00c0d4210 legacy=0x7fd00c06b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:27.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.723+0000 7fd011771640 1 -- 172.21.15.32:0/2377866955 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd00c0d3a10 con 0x7fd00c15e9b0 2024-04-24T23:00:27.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.723+0000 7fd00bfff640 1 -- 172.21.15.32:0/2377866955 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3847162825 0 0) 0x7fd00c0d3170 con 0x7fd00c0d4210 2024-04-24T23:00:27.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.723+0000 7fd011771640 1 --2- 172.21.15.32:0/2377866955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd00c15e9b0 0x7fd00c0d2b90 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fcffc009d90 tx=0x7fcffc02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fa77f915cb2b6ae server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:27.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.724+0000 7fd00bfff640 1 -- 172.21.15.32:0/2377866955 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcffc031600 con 0x7fd00c15e9b0 2024-04-24T23:00:27.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.724+0000 7fd00bfff640 1 -- 172.21.15.32:0/2377866955 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcffc0317a0 con 0x7fd00c15e9b0 2024-04-24T23:00:27.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.724+0000 7fd00bfff640 1 -- 172.21.15.32:0/2377866955 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcffc031b70 con 0x7fd00c15e9b0 2024-04-24T23:00:27.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.724+0000 7fd0139d5640 1 -- 172.21.15.32:0/2377866955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd00c15e9b0 msgr2=0x7fd00c0d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:27.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.724+0000 7fd0139d5640 1 --2- 172.21.15.32:0/2377866955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd00c15e9b0 0x7fd00c0d2b90 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fcffc009d90 tx=0x7fcffc02a610 comp rx=0 tx=0).stop 2024-04-24T23:00:27.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.724+0000 7fd0139d5640 1 -- 172.21.15.32:0/2377866955 shutdown_connections 2024-04-24T23:00:27.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.724+0000 7fd0139d5640 1 --2- 172.21.15.32:0/2377866955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd00c15e9b0 0x7fd00c0d2b90 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:27.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.724+0000 7fd0139d5640 1 -- 172.21.15.32:0/2377866955 >> 172.21.15.32:0/2377866955 conn(0x7fd00c0d11d0 msgr2=0x7fd00c0d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:27.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.724+0000 7fd0139d5640 1 -- 172.21.15.32:0/2377866955 shutdown_connections 2024-04-24T23:00:27.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.724+0000 7fd0139d5640 1 -- 172.21.15.32:0/2377866955 wait complete. 2024-04-24T23:00:27.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.725+0000 7fd0139d5640 1 Processor -- start 2024-04-24T23:00:27.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.725+0000 7fd0139d5640 1 -- start start 2024-04-24T23:00:27.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.726+0000 7fd0139d5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd00c06b820 0x7fd00c0c7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:27.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.726+0000 7fd0139d5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd00c0d4210 0x7fd00c0cd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:27.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.726+0000 7fd0139d5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd00c15e9b0 0x7fd00c148340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:27.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.726+0000 7fd0139d5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd00c0ba0e0 con 0x7fd00c15e9b0 2024-04-24T23:00:27.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.726+0000 7fd0139d5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd00c0d2400 con 0x7fd00c0d4210 2024-04-24T23:00:27.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.726+0000 7fd0139d5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd00c0ba260 con 0x7fd00c06b820 2024-04-24T23:00:27.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.726+0000 7fd011f72640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd00c15e9b0 0x7fd00c148340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:27.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.726+0000 7fd011f72640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd00c15e9b0 0x7fd00c148340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48248/0 (socket says 172.21.15.32:48248) 2024-04-24T23:00:27.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.726+0000 7fd011f72640 1 -- 172.21.15.32:0/3441200194 learned_addr learned my addr 172.21.15.32:0/3441200194 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:27.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.727+0000 7fd010f70640 1 --2- 172.21.15.32:0/3441200194 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd00c0d4210 0x7fd00c0cd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:27.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.727+0000 7fd011771640 1 --2- 172.21.15.32:0/3441200194 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd00c06b820 0x7fd00c0c7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:27.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.727+0000 7fd011f72640 1 -- 172.21.15.32:0/3441200194 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd00c06b820 msgr2=0x7fd00c0c7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:27.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.727+0000 7fd011f72640 1 --2- 172.21.15.32:0/3441200194 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd00c06b820 0x7fd00c0c7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:27.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.727+0000 7fd011f72640 1 -- 172.21.15.32:0/3441200194 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd00c0d4210 msgr2=0x7fd00c0cd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:27.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.727+0000 7fd011f72640 1 --2- 172.21.15.32:0/3441200194 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd00c0d4210 0x7fd00c0cd9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:27.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.727+0000 7fd011f72640 1 -- 172.21.15.32:0/3441200194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd00c0cc930 con 0x7fd00c15e9b0 2024-04-24T23:00:27.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.727+0000 7fd010f70640 1 --2- 172.21.15.32:0/3441200194 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd00c0d4210 0x7fd00c0cd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:00:27.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.727+0000 7fd011f72640 1 --2- 172.21.15.32:0/3441200194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd00c15e9b0 0x7fd00c148340 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fd00400c440 tx=0x7fd00400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:27.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.727+0000 7fd009ffb640 1 -- 172.21.15.32:0/3441200194 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd004011790 con 0x7fd00c15e9b0 2024-04-24T23:00:27.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.727+0000 7fd009ffb640 1 -- 172.21.15.32:0/3441200194 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd004017070 con 0x7fd00c15e9b0 2024-04-24T23:00:27.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.727+0000 7fd0139d5640 1 -- 172.21.15.32:0/3441200194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd00c0ccc10 con 0x7fd00c15e9b0 2024-04-24T23:00:27.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.727+0000 7fd009ffb640 1 -- 172.21.15.32:0/3441200194 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd004011b70 con 0x7fd00c15e9b0 2024-04-24T23:00:27.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.727+0000 7fd0139d5640 1 -- 172.21.15.32:0/3441200194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd00c0cd1d0 con 0x7fd00c15e9b0 2024-04-24T23:00:27.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.728+0000 7fd0139d5640 1 -- 172.21.15.32:0/3441200194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd00c068430 con 0x7fd00c15e9b0 2024-04-24T23:00:27.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.728+0000 7fd009ffb640 1 -- 172.21.15.32:0/3441200194 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd004011d10 con 0x7fd00c15e9b0 2024-04-24T23:00:27.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.729+0000 7fd009ffb640 1 --2- 172.21.15.32:0/3441200194 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcff806be30 0x7fcff806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:27.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.729+0000 7fd009ffb640 1 -- 172.21.15.32:0/3441200194 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7fd00408a940 con 0x7fd00c15e9b0 2024-04-24T23:00:27.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.729+0000 7fd011771640 1 --2- 172.21.15.32:0/3441200194 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcff806be30 0x7fcff806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:27.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.730+0000 7fd011771640 1 --2- 172.21.15.32:0/3441200194 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcff806be30 0x7fcff806e2f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcffc034000 tx=0x7fcffc02f000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:27.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.733+0000 7fd009ffb640 1 -- 172.21.15.32:0/3441200194 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd0040595e0 con 0x7fd00c15e9b0 2024-04-24T23:00:27.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.957+0000 7fd0139d5640 1 -- 172.21.15.32:0/3441200194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd00c15bea0 con 0x7fd00c15e9b0 2024-04-24T23:00:27.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.958+0000 7fd009ffb640 1 -- 172.21.15.32:0/3441200194 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+3869 (secure 0 0 0) 0x7fd004058ee0 con 0x7fd00c15e9b0 2024-04-24T23:00:27.959 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:00:27.960 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":15,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1796819396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":1796819396},{"type":"v1","addr":"172.21.15.32:6835","nonce":1796819396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/298202585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":298202585},{"type":"v1","addr":"172.21.15.121:6836","nonce":298202585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4939,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/4056636026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":4056636026},{"type":"v1","addr":"172.21.15.32:6834","nonce":4056636026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5233,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/778021204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":778021204},{"type":"v1","addr":"172.21.15.121:6837","nonce":778021204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":15}],"filesystems":[]} 2024-04-24T23:00:27.960 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 15 2024-04-24T23:00:27.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.960+0000 7fd0139d5640 1 -- 172.21.15.32:0/3441200194 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcff806be30 msgr2=0x7fcff806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:27.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.960+0000 7fd0139d5640 1 --2- 172.21.15.32:0/3441200194 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcff806be30 0x7fcff806e2f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcffc034000 tx=0x7fcffc02f000 comp rx=0 tx=0).stop 2024-04-24T23:00:27.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.960+0000 7fd0139d5640 1 -- 172.21.15.32:0/3441200194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd00c15e9b0 msgr2=0x7fd00c148340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:27.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.960+0000 7fd0139d5640 1 --2- 172.21.15.32:0/3441200194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd00c15e9b0 0x7fd00c148340 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fd00400c440 tx=0x7fd00400c900 comp rx=0 tx=0).stop 2024-04-24T23:00:27.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.960+0000 7fd0139d5640 1 -- 172.21.15.32:0/3441200194 shutdown_connections 2024-04-24T23:00:27.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.960+0000 7fd0139d5640 1 --2- 172.21.15.32:0/3441200194 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcff806be30 0x7fcff806e2f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:27.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.960+0000 7fd0139d5640 1 --2- 172.21.15.32:0/3441200194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd00c15e9b0 0x7fd00c148340 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:27.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.960+0000 7fd0139d5640 1 --2- 172.21.15.32:0/3441200194 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd00c0d4210 0x7fd00c0cd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:27.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.960+0000 7fd0139d5640 1 --2- 172.21.15.32:0/3441200194 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd00c06b820 0x7fd00c0c7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:27.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.960+0000 7fd0139d5640 1 -- 172.21.15.32:0/3441200194 >> 172.21.15.32:0/3441200194 conn(0x7fd00c0d11d0 msgr2=0x7fd00c15c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:27.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.960+0000 7fd0139d5640 1 -- 172.21.15.32:0/3441200194 shutdown_connections 2024-04-24T23:00:27.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:27.960+0000 7fd0139d5640 1 -- 172.21.15.32:0/3441200194 wait complete. 2024-04-24T23:00:27.972 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-24T23:00:27.972 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-24T23:00:27.972 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-24T23:00:27.973 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:00:27.973 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-24T23:00:27.973 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-24T23:00:27.973 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-24T23:00:27.973 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:00:27.973 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-24T23:00:27.973 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-24T23:00:27.974 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-24T23:00:27.974 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:00:27.974 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-24T23:00:27.974 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-24T23:00:27.974 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-24T23:00:27.974 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:00:27.982 INFO:tasks.ceph.mds.a:Stopped 2024-04-24T23:00:27.982 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-24T23:00:27.983 INFO:tasks.ceph.mds.c:Stopped 2024-04-24T23:00:27.983 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-24T23:00:27.987 INFO:tasks.ceph.mds.b:Stopped 2024-04-24T23:00:27.987 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-24T23:00:27.988 INFO:tasks.ceph.mds.d:Stopped 2024-04-24T23:00:27.988 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-24T23:00:28.025 INFO:tasks.ceph.mds.a:Started 2024-04-24T23:00:28.025 DEBUG:teuthology.parallel:result is None 2024-04-24T23:00:28.026 INFO:tasks.ceph.mds.c:Started 2024-04-24T23:00:28.026 DEBUG:teuthology.parallel:result is None 2024-04-24T23:00:28.029 INFO:tasks.ceph.mds.b:Started 2024-04-24T23:00:28.030 DEBUG:teuthology.parallel:result is None 2024-04-24T23:00:28.030 INFO:tasks.ceph.mds.d:Started 2024-04-24T23:00:28.030 DEBUG:teuthology.parallel:result is None 2024-04-24T23:00:28.031 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-24T23:00:28.101 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2024-04-24T23:00:28.102 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2024-04-24T23:00:28.102 INFO:tasks.ceph.mds.c.smithi032.stderr:2024-04-24T23:00:28.101+0000 7fe545c9b640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-04-24T23:00:28.107 INFO:tasks.ceph.mds.d.smithi121.stdout:starting mds.d at 2024-04-24T23:00:28.108 INFO:tasks.ceph.mds.b.smithi121.stdout:starting mds.b at 2024-04-24T23:00:28.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.135+0000 7fef831c1640 1 Processor -- start 2024-04-24T23:00:28.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.135+0000 7fef831c1640 1 -- start start 2024-04-24T23:00:28.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.136+0000 7fef831c1640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef7c15e9b0 0x7fef7c0d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:28.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.136+0000 7fef831c1640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fef7c0d4210 0x7fef7c06b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:28.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.136+0000 7fef831c1640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef7c0d3170 con 0x7fef7c0ba0e0 2024-04-24T23:00:28.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.136+0000 7fef7bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fef7c0d4210 0x7fef7c06b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:28.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.136+0000 7fef8175e640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fef7c0ba0e0 0x7fef7c071730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33514/0 (socket says 172.21.15.32:33514) 2024-04-24T23:00:28.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.136+0000 7fef8175e640 1 -- 172.21.15.32:0/2080385697 learned_addr learned my addr 172.21.15.32:0/2080385697 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:00:28.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.136+0000 7fef7bfff640 1 --2- 172.21.15.32:0/2080385697 >> v2:172.21.15.32:3300/0 conn(0x7fef7c0d4210 0x7fef7c06b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48292/0 (socket says 172.21.15.32:48292) 2024-04-24T23:00:28.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.136+0000 7fef831c1640 1 -- 172.21.15.32:0/2080385697 --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fef7c0b8900 con 0x7fef7c0d4210 2024-04-24T23:00:28.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.136+0000 7fef831c1640 1 -- 172.21.15.32:0/2080385697 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fef7c0d3450 con 0x7fef7c15e9b0 2024-04-24T23:00:28.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.136+0000 7fef80f5d640 1 --2- 172.21.15.32:0/2080385697 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef7c15e9b0 0x7fef7c0d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:28.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.136+0000 7fef7b7fe640 1 -- 172.21.15.32:0/2080385697 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3704116255 0 0) 0x7fef7c0d3170 con 0x7fef7c0ba0e0 2024-04-24T23:00:28.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.136+0000 7fef7b7fe640 1 -- 172.21.15.32:0/2080385697 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fef6c003520 con 0x7fef7c0ba0e0 2024-04-24T23:00:28.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.137+0000 7fef7bfff640 1 -- 172.21.15.32:0/2080385697 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef7c15e9b0 msgr2=0x7fef7c0d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:28.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.137+0000 7fef7bfff640 1 --2- 172.21.15.32:0/2080385697 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef7c15e9b0 0x7fef7c0d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:28.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.137+0000 7fef7bfff640 1 -- 172.21.15.32:0/2080385697 >> v1:172.21.15.32:6789/0 conn(0x7fef7c0ba0e0 legacy=0x7fef7c071730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:28.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.137+0000 7fef7bfff640 1 -- 172.21.15.32:0/2080385697 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef7c0d3730 con 0x7fef7c0d4210 2024-04-24T23:00:28.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.137+0000 7fef7bfff640 1 --2- 172.21.15.32:0/2080385697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef7c0d4210 0x7fef7c06b110 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fef680097a0 tx=0x7fef6802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e73ee56207b9695 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:28.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.137+0000 7fef7b7fe640 1 -- 172.21.15.32:0/2080385697 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef68037070 con 0x7fef7c0d4210 2024-04-24T23:00:28.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.137+0000 7fef7b7fe640 1 -- 172.21.15.32:0/2080385697 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef68032440 con 0x7fef7c0d4210 2024-04-24T23:00:28.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.137+0000 7fef7b7fe640 1 -- 172.21.15.32:0/2080385697 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef68032810 con 0x7fef7c0d4210 2024-04-24T23:00:28.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.138+0000 7fef831c1640 1 -- 172.21.15.32:0/2080385697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef7c0d4210 msgr2=0x7fef7c06b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:28.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.138+0000 7fef831c1640 1 --2- 172.21.15.32:0/2080385697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef7c0d4210 0x7fef7c06b110 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fef680097a0 tx=0x7fef6802a180 comp rx=0 tx=0).stop 2024-04-24T23:00:28.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.138+0000 7fef831c1640 1 -- 172.21.15.32:0/2080385697 shutdown_connections 2024-04-24T23:00:28.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.138+0000 7fef831c1640 1 --2- 172.21.15.32:0/2080385697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef7c0d4210 0x7fef7c06b110 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:28.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.138+0000 7fef831c1640 1 --2- 172.21.15.32:0/2080385697 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef7c15e9b0 0x7fef7c0d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:28.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.138+0000 7fef831c1640 1 -- 172.21.15.32:0/2080385697 >> 172.21.15.32:0/2080385697 conn(0x7fef7c0d11d0 msgr2=0x7fef7c0d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:28.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.138+0000 7fef831c1640 1 -- 172.21.15.32:0/2080385697 shutdown_connections 2024-04-24T23:00:28.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.138+0000 7fef831c1640 1 -- 172.21.15.32:0/2080385697 wait complete. 2024-04-24T23:00:28.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.139+0000 7fef831c1640 1 Processor -- start 2024-04-24T23:00:28.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.139+0000 7fef831c1640 1 -- start start 2024-04-24T23:00:28.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.140+0000 7fef831c1640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef7c0ba0e0 0x7fef7c1f7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:28.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.140+0000 7fef831c1640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef7c0d4210 0x7fef7c0ce790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:28.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.140+0000 7fef831c1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef7c15e9b0 0x7fef7c0cab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:28.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.140+0000 7fef831c1640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c0d2400 con 0x7fef7c15e9b0 2024-04-24T23:00:28.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.140+0000 7fef831c1640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c0d3cf0 con 0x7fef7c0d4210 2024-04-24T23:00:28.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.140+0000 7fef831c1640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fef7c06b650 con 0x7fef7c0ba0e0 2024-04-24T23:00:28.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.140+0000 7fef8175e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef7c15e9b0 0x7fef7c0cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:28.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.141+0000 7fef8175e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef7c15e9b0 0x7fef7c0cab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48296/0 (socket says 172.21.15.32:48296) 2024-04-24T23:00:28.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.141+0000 7fef8175e640 1 -- 172.21.15.32:0/3446298529 learned_addr learned my addr 172.21.15.32:0/3446298529 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:28.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.141+0000 7fef7bfff640 1 --2- 172.21.15.32:0/3446298529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef7c0d4210 0x7fef7c0ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:28.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.141+0000 7fef80f5d640 1 --2- 172.21.15.32:0/3446298529 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef7c0ba0e0 0x7fef7c1f7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:28.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.141+0000 7fef8175e640 1 -- 172.21.15.32:0/3446298529 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef7c0ba0e0 msgr2=0x7fef7c1f7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:28.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.141+0000 7fef8175e640 1 --2- 172.21.15.32:0/3446298529 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef7c0ba0e0 0x7fef7c1f7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:28.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.141+0000 7fef8175e640 1 -- 172.21.15.32:0/3446298529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef7c0d4210 msgr2=0x7fef7c0ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:28.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.141+0000 7fef8175e640 1 --2- 172.21.15.32:0/3446298529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef7c0d4210 0x7fef7c0ce790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:28.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.141+0000 7fef8175e640 1 -- 172.21.15.32:0/3446298529 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef7c1f7ed0 con 0x7fef7c15e9b0 2024-04-24T23:00:28.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.141+0000 7fef8175e640 1 --2- 172.21.15.32:0/3446298529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef7c15e9b0 0x7fef7c0cab40 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fef7400cb40 tx=0x7fef7400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:28.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.141+0000 7fef797fa640 1 -- 172.21.15.32:0/3446298529 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef74017070 con 0x7fef7c15e9b0 2024-04-24T23:00:28.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.141+0000 7fef797fa640 1 -- 172.21.15.32:0/3446298529 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef7400aca0 con 0x7fef7c15e9b0 2024-04-24T23:00:28.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.141+0000 7fef831c1640 1 -- 172.21.15.32:0/3446298529 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef7c1f81b0 con 0x7fef7c15e9b0 2024-04-24T23:00:28.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.142+0000 7fef797fa640 1 -- 172.21.15.32:0/3446298529 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef74012620 con 0x7fef7c15e9b0 2024-04-24T23:00:28.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.142+0000 7fef831c1640 1 -- 172.21.15.32:0/3446298529 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef7c1f8770 con 0x7fef7c15e9b0 2024-04-24T23:00:28.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.142+0000 7fef831c1640 1 -- 172.21.15.32:0/3446298529 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef7c068430 con 0x7fef7c15e9b0 2024-04-24T23:00:28.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.143+0000 7fef797fa640 1 -- 172.21.15.32:0/3446298529 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fef7401a070 con 0x7fef7c15e9b0 2024-04-24T23:00:28.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.143+0000 7fef797fa640 1 --2- 172.21.15.32:0/3446298529 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fef6006be30 0x7fef6006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:28.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.143+0000 7fef797fa640 1 -- 172.21.15.32:0/3446298529 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7fef7408a880 con 0x7fef7c15e9b0 2024-04-24T23:00:28.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.143+0000 7fef80f5d640 1 --2- 172.21.15.32:0/3446298529 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fef6006be30 0x7fef6006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:28.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.144+0000 7fef80f5d640 1 --2- 172.21.15.32:0/3446298529 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fef6006be30 0x7fef6006e2f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fef70009a30 tx=0x7fef70008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:28.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.147+0000 7fef797fa640 1 -- 172.21.15.32:0/3446298529 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fef740595c0 con 0x7fef7c15e9b0 2024-04-24T23:00:28.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.341+0000 7fef831c1640 1 -- 172.21.15.32:0/3446298529 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fef7c1556f0 con 0x7fef7c15e9b0 2024-04-24T23:00:28.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.990+0000 7fef797fa640 1 -- 172.21.15.32:0/3446298529 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v39) ==== 97+0+0 (secure 0 0 0) 0x7fef74058ec0 con 0x7fef7c15e9b0 2024-04-24T23:00:28.991 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blocklist entries 2024-04-24T23:00:28.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.992+0000 7fef831c1640 1 -- 172.21.15.32:0/3446298529 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fef6006be30 msgr2=0x7fef6006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:28.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.992+0000 7fef831c1640 1 --2- 172.21.15.32:0/3446298529 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fef6006be30 0x7fef6006e2f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fef70009a30 tx=0x7fef70008040 comp rx=0 tx=0).stop 2024-04-24T23:00:28.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.992+0000 7fef831c1640 1 -- 172.21.15.32:0/3446298529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef7c15e9b0 msgr2=0x7fef7c0cab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:28.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.992+0000 7fef831c1640 1 --2- 172.21.15.32:0/3446298529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef7c15e9b0 0x7fef7c0cab40 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fef7400cb40 tx=0x7fef7400a060 comp rx=0 tx=0).stop 2024-04-24T23:00:28.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.992+0000 7fef831c1640 1 -- 172.21.15.32:0/3446298529 shutdown_connections 2024-04-24T23:00:28.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.992+0000 7fef831c1640 1 --2- 172.21.15.32:0/3446298529 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fef6006be30 0x7fef6006e2f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:28.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.992+0000 7fef831c1640 1 --2- 172.21.15.32:0/3446298529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef7c15e9b0 0x7fef7c0cab40 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:28.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.992+0000 7fef831c1640 1 --2- 172.21.15.32:0/3446298529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef7c0d4210 0x7fef7c0ce790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:28.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.992+0000 7fef831c1640 1 --2- 172.21.15.32:0/3446298529 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef7c0ba0e0 0x7fef7c1f7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:28.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.992+0000 7fef831c1640 1 -- 172.21.15.32:0/3446298529 >> 172.21.15.32:0/3446298529 conn(0x7fef7c0d11d0 msgr2=0x7fef7c15c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:28.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.992+0000 7fef831c1640 1 -- 172.21.15.32:0/3446298529 shutdown_connections 2024-04-24T23:00:28.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:28.992+0000 7fef831c1640 1 -- 172.21.15.32:0/3446298529 wait complete. 2024-04-24T23:00:29.008 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-24T23:00:29.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.111+0000 7f9cdafa5640 1 Processor -- start 2024-04-24T23:00:29.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.112+0000 7f9cdafa5640 1 -- start start 2024-04-24T23:00:29.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.112+0000 7f9cdafa5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9cd415e9b0 0x7f9cd40d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:29.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.112+0000 7f9cdafa5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9cd40d4210 0x7f9cd406b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:29.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.112+0000 7f9cdafa5640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9cd40d3170 con 0x7f9cd40ba0e0 2024-04-24T23:00:29.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.112+0000 7f9cdafa5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f9cd40b8900 con 0x7f9cd415e9b0 2024-04-24T23:00:29.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.112+0000 7f9cdafa5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f9cd40d3450 con 0x7f9cd40d4210 2024-04-24T23:00:29.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.112+0000 7f9cd8d41640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9cd415e9b0 0x7f9cd40d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:29.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.112+0000 7f9cd8d41640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9cd415e9b0 0x7f9cd40d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48312/0 (socket says 172.21.15.32:48312) 2024-04-24T23:00:29.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.112+0000 7f9cd8d41640 1 -- 172.21.15.32:0/4231795799 learned_addr learned my addr 172.21.15.32:0/4231795799 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:29.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.113+0000 7f9cd3fff640 1 --2- 172.21.15.32:0/4231795799 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9cd40d4210 0x7f9cd406b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:29.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.113+0000 7f9cd8d41640 1 -- 172.21.15.32:0/4231795799 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9cd40d4210 msgr2=0x7f9cd406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:29.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.113+0000 7f9cd8d41640 1 --2- 172.21.15.32:0/4231795799 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9cd40d4210 0x7f9cd406b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:29.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.113+0000 7f9cd8d41640 1 -- 172.21.15.32:0/4231795799 >> v1:172.21.15.121:6789/0 conn(0x7f9cd40ba0e0 legacy=0x7f9cd4071730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:29.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.113+0000 7f9cd8d41640 1 -- 172.21.15.32:0/4231795799 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cd40d3730 con 0x7f9cd415e9b0 2024-04-24T23:00:29.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.113+0000 7f9cd8d41640 1 --2- 172.21.15.32:0/4231795799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cd415e9b0 0x7f9cd40d2b90 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4009870 tx=0x7f9cc402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1fcb848a55d11ce server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:29.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.114+0000 7f9cd37fe640 1 -- 172.21.15.32:0/4231795799 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc4037070 con 0x7f9cd415e9b0 2024-04-24T23:00:29.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.114+0000 7f9cd37fe640 1 -- 172.21.15.32:0/4231795799 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cc4032440 con 0x7f9cd415e9b0 2024-04-24T23:00:29.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.114+0000 7f9cd37fe640 1 -- 172.21.15.32:0/4231795799 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc4032810 con 0x7f9cd415e9b0 2024-04-24T23:00:29.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.114+0000 7f9cdafa5640 1 -- 172.21.15.32:0/4231795799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cd415e9b0 msgr2=0x7f9cd40d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:29.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.114+0000 7f9cdafa5640 1 --2- 172.21.15.32:0/4231795799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cd415e9b0 0x7f9cd40d2b90 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4009870 tx=0x7f9cc402a210 comp rx=0 tx=0).stop 2024-04-24T23:00:29.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.114+0000 7f9cdafa5640 1 -- 172.21.15.32:0/4231795799 shutdown_connections 2024-04-24T23:00:29.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.114+0000 7f9cdafa5640 1 --2- 172.21.15.32:0/4231795799 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9cd40d4210 0x7f9cd406b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:29.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.114+0000 7f9cdafa5640 1 --2- 172.21.15.32:0/4231795799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cd415e9b0 0x7f9cd40d2b90 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:29.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.114+0000 7f9cdafa5640 1 -- 172.21.15.32:0/4231795799 >> 172.21.15.32:0/4231795799 conn(0x7f9cd40d11d0 msgr2=0x7f9cd40d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:29.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.114+0000 7f9cdafa5640 1 -- 172.21.15.32:0/4231795799 shutdown_connections 2024-04-24T23:00:29.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.114+0000 7f9cdafa5640 1 -- 172.21.15.32:0/4231795799 wait complete. 2024-04-24T23:00:29.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.115+0000 7f9cdafa5640 1 Processor -- start 2024-04-24T23:00:29.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.116+0000 7f9cdafa5640 1 -- start start 2024-04-24T23:00:29.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.116+0000 7f9cdafa5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cd40ba0e0 0x7f9cd41f8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:29.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.116+0000 7f9cdafa5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9cd40d4210 0x7f9cd40ca500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:29.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.116+0000 7f9cdafa5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cd415e9b0 0x7f9cd415ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:29.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.116+0000 7f9cdafa5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd40d2400 con 0x7f9cd40ba0e0 2024-04-24T23:00:29.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.116+0000 7f9cdafa5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd40d3cf0 con 0x7f9cd415e9b0 2024-04-24T23:00:29.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.116+0000 7f9cdafa5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f9cd406b650 con 0x7f9cd40d4210 2024-04-24T23:00:29.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.117+0000 7f9cd8d41640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cd40ba0e0 0x7f9cd41f8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:29.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.117+0000 7f9cd8d41640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cd40ba0e0 0x7f9cd41f8390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48320/0 (socket says 172.21.15.32:48320) 2024-04-24T23:00:29.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.117+0000 7f9cd8d41640 1 -- 172.21.15.32:0/2885396866 learned_addr learned my addr 172.21.15.32:0/2885396866 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:29.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.117+0000 7f9cd3fff640 1 --2- 172.21.15.32:0/2885396866 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9cd40d4210 0x7f9cd40ca500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:29.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.117+0000 7f9cd9542640 1 --2- 172.21.15.32:0/2885396866 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cd415e9b0 0x7f9cd415ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:29.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.117+0000 7f9cd8d41640 1 -- 172.21.15.32:0/2885396866 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9cd40d4210 msgr2=0x7f9cd40ca500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:29.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.117+0000 7f9cd8d41640 1 --2- 172.21.15.32:0/2885396866 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9cd40d4210 0x7f9cd40ca500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:29.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.117+0000 7f9cd8d41640 1 -- 172.21.15.32:0/2885396866 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cd415e9b0 msgr2=0x7f9cd415ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:29.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.117+0000 7f9cd8d41640 1 --2- 172.21.15.32:0/2885396866 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cd415e9b0 0x7f9cd415ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:29.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.117+0000 7f9cd8d41640 1 -- 172.21.15.32:0/2885396866 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cd41f78f0 con 0x7f9cd40ba0e0 2024-04-24T23:00:29.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.117+0000 7f9cd3fff640 1 --2- 172.21.15.32:0/2885396866 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9cd40d4210 0x7f9cd40ca500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:00:29.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.117+0000 7f9cd9542640 1 --2- 172.21.15.32:0/2885396866 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cd415e9b0 0x7f9cd415ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:00:29.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.117+0000 7f9cd8d41640 1 --2- 172.21.15.32:0/2885396866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cd40ba0e0 0x7f9cd41f8390 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4000c00 tx=0x7f9cc40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:29.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.117+0000 7f9cd17fa640 1 -- 172.21.15.32:0/2885396866 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc4041020 con 0x7f9cd40ba0e0 2024-04-24T23:00:29.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.117+0000 7f9cd17fa640 1 -- 172.21.15.32:0/2885396866 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cc40306c0 con 0x7f9cd40ba0e0 2024-04-24T23:00:29.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.117+0000 7f9cdafa5640 1 -- 172.21.15.32:0/2885396866 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cd41f4e10 con 0x7f9cd40ba0e0 2024-04-24T23:00:29.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.117+0000 7f9cd17fa640 1 -- 172.21.15.32:0/2885396866 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc4037040 con 0x7f9cd40ba0e0 2024-04-24T23:00:29.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.117+0000 7f9cdafa5640 1 -- 172.21.15.32:0/2885396866 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cd41f7b90 con 0x7f9cd40ba0e0 2024-04-24T23:00:29.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.118+0000 7f9cdafa5640 1 -- 172.21.15.32:0/2885396866 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cd4068430 con 0x7f9cd40ba0e0 2024-04-24T23:00:29.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.119+0000 7f9cd17fa640 1 -- 172.21.15.32:0/2885396866 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9cc4044020 con 0x7f9cd40ba0e0 2024-04-24T23:00:29.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.119+0000 7f9cd17fa640 1 --2- 172.21.15.32:0/2885396866 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9cc006be30 0x7f9cc006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:29.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.119+0000 7f9cd17fa640 1 -- 172.21.15.32:0/2885396866 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4857+0+0 (secure 0 0 0) 0x7f9cc40ab050 con 0x7f9cd40ba0e0 2024-04-24T23:00:29.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.120+0000 7f9cd3fff640 1 --2- 172.21.15.32:0/2885396866 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9cc006be30 0x7f9cc006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:29.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.120+0000 7f9cd3fff640 1 --2- 172.21.15.32:0/2885396866 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9cc006be30 0x7f9cc006e2f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9cd40724c0 tx=0x7f9cc8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:29.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.124+0000 7f9cd17fa640 1 -- 172.21.15.32:0/2885396866 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9cc404b070 con 0x7f9cd40ba0e0 2024-04-24T23:00:29.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.374+0000 7f9cdafa5640 1 -- 172.21.15.32:0/2885396866 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f9cd40cf7e0 con 0x7f9cd40ba0e0 2024-04-24T23:00:29.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.375+0000 7f9cd17fa640 1 -- 172.21.15.32:0/2885396866 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) ==== 80+0+5304 (secure 0 0 0) 0x7f9cc4079cb0 con 0x7f9cd40ba0e0 2024-04-24T23:00:29.377 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:00:29.377 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:00:29.377 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2024-04-24T23:00:29.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:29.377 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2024-04-24T23:00:29.377 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmXRdxAxAA/EYEn1C38XhG7DC+0X4y7A==", 2024-04-24T23:00:29.377 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:00:29.377 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:00:29.377 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-24T23:00:29.377 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-24T23:00:29.377 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:00:29.378 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:29.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:29.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:29.378 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2024-04-24T23:00:29.378 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmak7bDhAA0he+Jp/gkkrfBjYKSObjcw==", 2024-04-24T23:00:29.378 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:00:29.378 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:00:29.378 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-24T23:00:29.378 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-24T23:00:29.378 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:00:29.378 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:29.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:29.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:29.378 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.c", 2024-04-24T23:00:29.379 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmZXFdChAA8TwwkA5EIXHBzPsQSFvXcQ==", 2024-04-24T23:00:29.379 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:00:29.379 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:00:29.379 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-24T23:00:29.379 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-24T23:00:29.379 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:00:29.379 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:29.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:29.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:29.379 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.d", 2024-04-24T23:00:29.379 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmLbGuFRAAe91fZlulziWdWiDdLBLWNA==", 2024-04-24T23:00:29.379 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:00:29.379 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:00:29.379 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-24T23:00:29.380 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-24T23:00:29.380 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:00:29.380 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:29.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:29.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:29.380 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2024-04-24T23:00:29.380 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCfjilmhTwoOBAAig98xa3a2qnv4rriUO9VCQ==", 2024-04-24T23:00:29.380 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:00:29.380 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:00:29.380 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:00:29.380 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:00:29.380 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:29.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:29.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:29.381 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2024-04-24T23:00:29.381 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCgjilmwRaAAxAAUqDOyR9HV6DEnmbyn3WAjA==", 2024-04-24T23:00:29.381 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:00:29.381 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:00:29.381 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:00:29.381 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:00:29.381 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:29.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:29.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:29.381 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2024-04-24T23:00:29.381 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCgjilmDl2qDBAAF1bhTOaSOc612wmFRd8+XA==", 2024-04-24T23:00:29.381 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:00:29.381 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:00:29.382 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:00:29.382 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:00:29.382 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:29.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:29.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:29.382 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2024-04-24T23:00:29.382 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCgjilmAxHwFRAA9a0v/YwngRxq1vmMRsmEkQ==", 2024-04-24T23:00:29.382 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:00:29.382 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:00:29.382 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:00:29.382 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:00:29.382 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:29.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:29.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:29.383 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2024-04-24T23:00:29.383 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCjjilmDPmJIxAASZlEo1ZJHOhIAFGukIj1tQ==", 2024-04-24T23:00:29.383 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:00:29.383 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:00:29.383 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:00:29.383 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:00:29.383 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:29.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:29.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:29.383 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2024-04-24T23:00:29.383 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCjjilmgcpuKhAAedSq0ubNB30s3bz0fY38hw==", 2024-04-24T23:00:29.383 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:00:29.384 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:00:29.384 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:00:29.384 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:00:29.384 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:29.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:29.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:29.384 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2024-04-24T23:00:29.384 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCjjilmFv2VMxAAM+mUc4I0sjawy2uwuUEKYg==", 2024-04-24T23:00:29.384 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:00:29.384 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:00:29.384 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:00:29.384 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:00:29.384 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:29.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:29.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:29.384 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2024-04-24T23:00:29.385 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCkjilmLvEbARAAnaOvQ5Ee0clN/A9SVoKuBQ==", 2024-04-24T23:00:29.385 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:00:29.385 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:00:29.385 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:00:29.385 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:00:29.385 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:29.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:29.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:29.385 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2024-04-24T23:00:29.385 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilm5oeiGRAAdLXwwqrYhV3HHo7Dw/2ZqQ==", 2024-04-24T23:00:29.385 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:00:29.385 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:00:29.385 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2024-04-24T23:00:29.385 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-24T23:00:29.386 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:29.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:29.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:29.386 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2024-04-24T23:00:29.386 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmyWBDIhAARtVP4Unk3HHlQYZ2QZ3GzQ==", 2024-04-24T23:00:29.386 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:00:29.386 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:00:29.386 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-24T23:00:29.386 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-24T23:00:29.386 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-24T23:00:29.386 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:29.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:29.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:29.387 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.2", 2024-04-24T23:00:29.387 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilm4QElHxAAXgE316aRCosxX7XER5gtKQ==", 2024-04-24T23:00:29.387 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:00:29.387 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:00:29.387 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-24T23:00:29.387 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-24T23:00:29.387 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-24T23:00:29.387 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:29.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:29.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:29.387 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.3", 2024-04-24T23:00:29.387 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmMZihJxAAoWaAAFY3vVfZnC4/GoT0+g==", 2024-04-24T23:00:29.387 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:00:29.387 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:00:29.388 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-24T23:00:29.388 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-24T23:00:29.388 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-24T23:00:29.388 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:29.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:29.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:29.388 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2024-04-24T23:00:29.388 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCdjilmdVXeIxAAwGGaAfshx4pHcoc5+3kRww==", 2024-04-24T23:00:29.388 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:00:29.388 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2024-04-24T23:00:29.388 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-24T23:00:29.388 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-24T23:00:29.388 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:00:29.388 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:29.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:29.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:29.389 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2024-04-24T23:00:29.389 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmn3FJABAAGnvNAoNLzK20cQF6beihmw==", 2024-04-24T23:00:29.389 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:00:29.389 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2024-04-24T23:00:29.389 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2024-04-24T23:00:29.389 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:00:29.389 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:29.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:29.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:29.389 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2024-04-24T23:00:29.389 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCdjilmMqs2OBAAOe3bLpnCq+HK2EbeXnwY8Q==", 2024-04-24T23:00:29.390 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:00:29.390 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2024-04-24T23:00:29.390 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2024-04-24T23:00:29.390 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:00:29.390 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:29.390 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:29.390 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:29.390 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:00:29.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.377+0000 7f9cdafa5640 1 -- 172.21.15.32:0/2885396866 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9cc006be30 msgr2=0x7f9cc006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:29.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.377+0000 7f9cdafa5640 1 --2- 172.21.15.32:0/2885396866 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9cc006be30 0x7f9cc006e2f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9cd40724c0 tx=0x7f9cc8008040 comp rx=0 tx=0).stop 2024-04-24T23:00:29.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.377+0000 7f9cdafa5640 1 -- 172.21.15.32:0/2885396866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cd40ba0e0 msgr2=0x7f9cd41f8390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:29.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.377+0000 7f9cdafa5640 1 --2- 172.21.15.32:0/2885396866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cd40ba0e0 0x7f9cd41f8390 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4000c00 tx=0x7f9cc40046c0 comp rx=0 tx=0).stop 2024-04-24T23:00:29.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.377+0000 7f9cdafa5640 1 -- 172.21.15.32:0/2885396866 shutdown_connections 2024-04-24T23:00:29.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.377+0000 7f9cdafa5640 1 --2- 172.21.15.32:0/2885396866 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9cc006be30 0x7f9cc006e2f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:29.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.377+0000 7f9cdafa5640 1 --2- 172.21.15.32:0/2885396866 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cd415e9b0 0x7f9cd415ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:29.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.377+0000 7f9cdafa5640 1 --2- 172.21.15.32:0/2885396866 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9cd40d4210 0x7f9cd40ca500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:29.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.377+0000 7f9cdafa5640 1 --2- 172.21.15.32:0/2885396866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cd40ba0e0 0x7f9cd41f8390 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:29.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.377+0000 7f9cdafa5640 1 -- 172.21.15.32:0/2885396866 >> 172.21.15.32:0/2885396866 conn(0x7f9cd40d11d0 msgr2=0x7f9cd40d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:29.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.377+0000 7f9cdafa5640 1 -- 172.21.15.32:0/2885396866 shutdown_connections 2024-04-24T23:00:29.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.377+0000 7f9cdafa5640 1 -- 172.21.15.32:0/2885396866 wait complete. 2024-04-24T23:00:29.393 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:00:29.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.492+0000 7f83bf132640 1 Processor -- start 2024-04-24T23:00:29.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.492+0000 7f83bf132640 1 -- start start 2024-04-24T23:00:29.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.493+0000 7f83bf132640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f83b815b380 0x7f83b806c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:29.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.493+0000 7f83bf132640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f83b815cb00 0x7f83b806cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:29.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.493+0000 7f83bf132640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f83b80ce430 0x7f83b80ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:29.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.493+0000 7f83bf132640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f83b80b4520 con 0x7f83b815cb00 2024-04-24T23:00:29.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.493+0000 7f83bf132640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f83b80b4800 con 0x7f83b80ce430 2024-04-24T23:00:29.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.493+0000 7f83bf132640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f83b80b4ae0 con 0x7f83b815b380 2024-04-24T23:00:29.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.493+0000 7f83b7fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f83b815cb00 0x7f83b806cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:29.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.493+0000 7f83b7fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f83b815cb00 0x7f83b806cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48328/0 (socket says 172.21.15.32:48328) 2024-04-24T23:00:29.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.493+0000 7f83b7fff640 1 -- 172.21.15.32:0/307367642 learned_addr learned my addr 172.21.15.32:0/307367642 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:29.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.494+0000 7f83bcece640 1 --2- 172.21.15.32:0/307367642 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f83b815b380 0x7f83b806c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:29.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.494+0000 7f83bd6cf640 1 --2- 172.21.15.32:0/307367642 >> v2:172.21.15.121:3300/0 conn(0x7f83b80ce430 0x7f83b80ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:29.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.494+0000 7f83b7fff640 1 -- 172.21.15.32:0/307367642 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f83b815b380 msgr2=0x7f83b806c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:29.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.494+0000 7f83b7fff640 1 --2- 172.21.15.32:0/307367642 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f83b815b380 0x7f83b806c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:29.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.494+0000 7f83b7fff640 1 -- 172.21.15.32:0/307367642 >> v2:172.21.15.121:3300/0 conn(0x7f83b80ce430 msgr2=0x7f83b80ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:29.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.494+0000 7f83b7fff640 1 --2- 172.21.15.32:0/307367642 >> v2:172.21.15.121:3300/0 conn(0x7f83b80ce430 0x7f83b80ce880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:29.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.494+0000 7f83b7fff640 1 -- 172.21.15.32:0/307367642 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83b80b4dc0 con 0x7f83b815cb00 2024-04-24T23:00:29.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.494+0000 7f83b7fff640 1 --2- 172.21.15.32:0/307367642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83b815cb00 0x7f83b806cb30 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f83b8070130 tx=0x7f83a002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f6cc8c1c02fbd27 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:29.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.494+0000 7f83b77fe640 1 -- 172.21.15.32:0/307367642 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a0037070 con 0x7f83b815cb00 2024-04-24T23:00:29.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.495+0000 7f83b77fe640 1 -- 172.21.15.32:0/307367642 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83a0032440 con 0x7f83b815cb00 2024-04-24T23:00:29.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.495+0000 7f83b77fe640 1 -- 172.21.15.32:0/307367642 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a0032810 con 0x7f83b815cb00 2024-04-24T23:00:29.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.495+0000 7f83bf132640 1 -- 172.21.15.32:0/307367642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83b815cb00 msgr2=0x7f83b806cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:29.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.495+0000 7f83bf132640 1 --2- 172.21.15.32:0/307367642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83b815cb00 0x7f83b806cb30 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f83b8070130 tx=0x7f83a002a140 comp rx=0 tx=0).stop 2024-04-24T23:00:29.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.495+0000 7f83bf132640 1 -- 172.21.15.32:0/307367642 shutdown_connections 2024-04-24T23:00:29.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.495+0000 7f83bf132640 1 --2- 172.21.15.32:0/307367642 >> v2:172.21.15.121:3300/0 conn(0x7f83b80ce430 0x7f83b80ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:29.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.495+0000 7f83bf132640 1 --2- 172.21.15.32:0/307367642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83b815cb00 0x7f83b806cb30 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:29.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.495+0000 7f83bf132640 1 --2- 172.21.15.32:0/307367642 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f83b815b380 0x7f83b806c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:29.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.495+0000 7f83bf132640 1 -- 172.21.15.32:0/307367642 >> 172.21.15.32:0/307367642 conn(0x7f83b806b110 msgr2=0x7f83b806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:29.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.495+0000 7f83bf132640 1 -- 172.21.15.32:0/307367642 shutdown_connections 2024-04-24T23:00:29.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.495+0000 7f83bf132640 1 -- 172.21.15.32:0/307367642 wait complete. 2024-04-24T23:00:29.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.496+0000 7f83bf132640 1 Processor -- start 2024-04-24T23:00:29.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.496+0000 7f83bf132640 1 -- start start 2024-04-24T23:00:29.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.497+0000 7f83bf132640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83b80ce430 0x7f83b8154be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:29.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.497+0000 7f83bf132640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f83b815b380 0x7f83b8142db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:29.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.497+0000 7f83bf132640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83b815cb00 0x7f83b81552f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:29.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.497+0000 7f83bf132640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f83b80cedc0 con 0x7f83b815cb00 2024-04-24T23:00:29.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.497+0000 7f83bf132640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f83b815b760 con 0x7f83b80ce430 2024-04-24T23:00:29.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.497+0000 7f83bf132640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f83b80cef40 con 0x7f83b815b380 2024-04-24T23:00:29.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.497+0000 7f83bd6cf640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83b815cb00 0x7f83b81552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:29.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.497+0000 7f83bd6cf640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83b815cb00 0x7f83b81552f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48332/0 (socket says 172.21.15.32:48332) 2024-04-24T23:00:29.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.497+0000 7f83bd6cf640 1 -- 172.21.15.32:0/3553069655 learned_addr learned my addr 172.21.15.32:0/3553069655 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:29.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.498+0000 7f83bd6cf640 1 -- 172.21.15.32:0/3553069655 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f83b815b380 msgr2=0x7f83b8142db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:29.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.498+0000 7f83b7fff640 1 --2- 172.21.15.32:0/3553069655 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f83b815b380 0x7f83b8142db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:29.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.498+0000 7f83bcece640 1 --2- 172.21.15.32:0/3553069655 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83b80ce430 0x7f83b8154be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:29.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.498+0000 7f83bd6cf640 1 --2- 172.21.15.32:0/3553069655 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f83b815b380 0x7f83b8142db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:29.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.498+0000 7f83bd6cf640 1 -- 172.21.15.32:0/3553069655 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83b80ce430 msgr2=0x7f83b8154be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:29.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.498+0000 7f83bd6cf640 1 --2- 172.21.15.32:0/3553069655 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83b80ce430 0x7f83b8154be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:29.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.498+0000 7f83bd6cf640 1 -- 172.21.15.32:0/3553069655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83b81432f0 con 0x7f83b815cb00 2024-04-24T23:00:29.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.498+0000 7f83b7fff640 1 --2- 172.21.15.32:0/3553069655 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f83b815b380 0x7f83b8142db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:00:29.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.498+0000 7f83bcece640 1 --2- 172.21.15.32:0/3553069655 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83b80ce430 0x7f83b8154be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:00:29.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.498+0000 7f83bd6cf640 1 --2- 172.21.15.32:0/3553069655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83b815cb00 0x7f83b81552f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f83a800c550 tx=0x7f83a800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:29.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.498+0000 7f83b57fa640 1 -- 172.21.15.32:0/3553069655 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a8015070 con 0x7f83b815cb00 2024-04-24T23:00:29.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.498+0000 7f83b57fa640 1 -- 172.21.15.32:0/3553069655 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83a8004220 con 0x7f83b815cb00 2024-04-24T23:00:29.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.498+0000 7f83bf132640 1 -- 172.21.15.32:0/3553069655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83b8153f70 con 0x7f83b815cb00 2024-04-24T23:00:29.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.498+0000 7f83b57fa640 1 -- 172.21.15.32:0/3553069655 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a8003610 con 0x7f83b815cb00 2024-04-24T23:00:29.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.498+0000 7f83bf132640 1 -- 172.21.15.32:0/3553069655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83b8154530 con 0x7f83b815cb00 2024-04-24T23:00:29.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.499+0000 7f83bf132640 1 -- 172.21.15.32:0/3553069655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83b8068430 con 0x7f83b815cb00 2024-04-24T23:00:29.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.500+0000 7f83b57fa640 1 -- 172.21.15.32:0/3553069655 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f83a80037b0 con 0x7f83b815cb00 2024-04-24T23:00:29.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.500+0000 7f83b57fa640 1 --2- 172.21.15.32:0/3553069655 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f839806be60 0x7f839806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:29.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.500+0000 7f83b57fa640 1 -- 172.21.15.32:0/3553069655 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4857+0+0 (secure 0 0 0) 0x7f83a808a830 con 0x7f83b815cb00 2024-04-24T23:00:29.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.500+0000 7f83bcece640 1 --2- 172.21.15.32:0/3553069655 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f839806be60 0x7f839806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:29.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.501+0000 7f83bcece640 1 --2- 172.21.15.32:0/3553069655 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f839806be60 0x7f839806e320 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f83b0007ed0 tx=0x7f83b0007420 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:29.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.504+0000 7f83b57fa640 1 -- 172.21.15.32:0/3553069655 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f83a8059530 con 0x7f83b815cb00 2024-04-24T23:00:29.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.688+0000 7f83bf132640 1 -- 172.21.15.32:0/3553069655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f83b80afd60 con 0x7f83b815cb00 2024-04-24T23:00:29.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.689+0000 7f83b57fa640 1 -- 172.21.15.32:0/3553069655 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+12903 (secure 0 0 0) 0x7f83a8058e30 con 0x7f83b815cb00 2024-04-24T23:00:29.691 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:00:29.691 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":41,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:00:28.997257+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:00:29.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.691+0000 7f83bf132640 1 -- 172.21.15.32:0/3553069655 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f839806be60 msgr2=0x7f839806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:29.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.691+0000 7f83bf132640 1 --2- 172.21.15.32:0/3553069655 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f839806be60 0x7f839806e320 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f83b0007ed0 tx=0x7f83b0007420 comp rx=0 tx=0).stop 2024-04-24T23:00:29.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.691+0000 7f83bf132640 1 -- 172.21.15.32:0/3553069655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83b815cb00 msgr2=0x7f83b81552f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:29.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.691+0000 7f83bf132640 1 --2- 172.21.15.32:0/3553069655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83b815cb00 0x7f83b81552f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f83a800c550 tx=0x7f83a800ca10 comp rx=0 tx=0).stop 2024-04-24T23:00:29.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.691+0000 7f83bf132640 1 -- 172.21.15.32:0/3553069655 shutdown_connections 2024-04-24T23:00:29.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.691+0000 7f83bf132640 1 --2- 172.21.15.32:0/3553069655 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f839806be60 0x7f839806e320 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:29.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.691+0000 7f83bf132640 1 --2- 172.21.15.32:0/3553069655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83b815cb00 0x7f83b81552f0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:29.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.691+0000 7f83bf132640 1 --2- 172.21.15.32:0/3553069655 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f83b815b380 0x7f83b8142db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:29.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.691+0000 7f83bf132640 1 --2- 172.21.15.32:0/3553069655 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83b80ce430 0x7f83b8154be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:29.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.691+0000 7f83bf132640 1 -- 172.21.15.32:0/3553069655 >> 172.21.15.32:0/3553069655 conn(0x7f83b806b110 msgr2=0x7f83b80b56a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:29.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.691+0000 7f83bf132640 1 -- 172.21.15.32:0/3553069655 shutdown_connections 2024-04-24T23:00:29.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.691+0000 7f83bf132640 1 -- 172.21.15.32:0/3553069655 wait complete. 2024-04-24T23:00:29.704 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:00:29.704 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:00:29.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.806+0000 7f3af01ed640 1 Processor -- start 2024-04-24T23:00:29.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.806+0000 7f3af01ed640 1 -- start start 2024-04-24T23:00:29.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.807+0000 7f3af01ed640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3ae8162320 0x7f3ae80ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:29.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.807+0000 7f3af01ed640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3ae8162e10 0x7f3ae815ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:29.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.807+0000 7f3af01ed640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ae80cd270 con 0x7f3ae80b5af0 2024-04-24T23:00:29.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.807+0000 7f3af01ed640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f3ae80b5660 con 0x7f3ae8162e10 2024-04-24T23:00:29.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.807+0000 7f3af01ed640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f3ae8071610 con 0x7f3ae8162320 2024-04-24T23:00:29.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.807+0000 7f3aed788640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3ae80b5af0 0x7f3ae8162700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38404/0 (socket says 172.21.15.32:38404) 2024-04-24T23:00:29.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.807+0000 7f3aed788640 1 -- 172.21.15.32:0/4062216571 learned_addr learned my addr 172.21.15.32:0/4062216571 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:00:29.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.808+0000 7f3aecf87640 1 -- 172.21.15.32:0/4062216571 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3916661981 0 0) 0x7f3ae80cd270 con 0x7f3ae80b5af0 2024-04-24T23:00:29.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.808+0000 7f3aedf89640 1 --2- 172.21.15.32:0/4062216571 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3ae8162320 0x7f3ae80ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:29.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.808+0000 7f3aee78a640 1 --2- 172.21.15.32:0/4062216571 >> v2:172.21.15.121:3300/0 conn(0x7f3ae8162e10 0x7f3ae815ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:29.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.808+0000 7f3aecf87640 1 -- 172.21.15.32:0/4062216571 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ad0003520 con 0x7f3ae80b5af0 2024-04-24T23:00:29.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.808+0000 7f3aecf87640 1 -- 172.21.15.32:0/4062216571 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2053610176 0 0) 0x7f3ad0003520 con 0x7f3ae80b5af0 2024-04-24T23:00:29.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.808+0000 7f3aecf87640 1 -- 172.21.15.32:0/4062216571 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3ae8162320 msgr2=0x7f3ae80ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:29.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.808+0000 7f3aecf87640 1 --2- 172.21.15.32:0/4062216571 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3ae8162320 0x7f3ae80ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:29.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.808+0000 7f3aecf87640 1 -- 172.21.15.32:0/4062216571 >> v2:172.21.15.121:3300/0 conn(0x7f3ae8162e10 msgr2=0x7f3ae815ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:29.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.808+0000 7f3aecf87640 1 --2- 172.21.15.32:0/4062216571 >> v2:172.21.15.121:3300/0 conn(0x7f3ae8162e10 0x7f3ae815ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:29.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.808+0000 7f3aecf87640 1 -- 172.21.15.32:0/4062216571 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ae80718f0 con 0x7f3ae80b5af0 2024-04-24T23:00:29.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.808+0000 7f3aecf87640 1 -- 172.21.15.32:0/4062216571 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f3adc001670 con 0x7f3ae80b5af0 2024-04-24T23:00:29.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.808+0000 7f3aecf87640 1 -- 172.21.15.32:0/4062216571 >> v1:172.21.15.32:6789/0 conn(0x7f3ae80b5af0 legacy=0x7f3ae8162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:29.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.808+0000 7f3aecf87640 1 --2- 172.21.15.32:0/4062216571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ad00039b0 0x7f3ad0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:29.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.808+0000 7f3aecf87640 1 --2- 172.21.15.32:0/4062216571 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ad0004800 0x7f3ad0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:29.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.808+0000 7f3aecf87640 1 --2- 172.21.15.32:0/4062216571 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3ae8162320 0x7f3ad00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:29.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.808+0000 7f3aecf87640 1 -- 172.21.15.32:0/4062216571 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae80b5660 con 0x7f3ad00039b0 2024-04-24T23:00:29.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.808+0000 7f3aecf87640 1 -- 172.21.15.32:0/4062216571 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae8071610 con 0x7f3ad0004800 2024-04-24T23:00:29.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.808+0000 7f3aecf87640 1 -- 172.21.15.32:0/4062216571 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f3ad000b9c0 con 0x7f3ae8162320 2024-04-24T23:00:29.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.808+0000 7f3aedf89640 1 --2- 172.21.15.32:0/4062216571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ad00039b0 0x7f3ad0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:29.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.808+0000 7f3aed788640 1 --2- 172.21.15.32:0/4062216571 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ad0004800 0x7f3ad0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:29.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.808+0000 7f3aee78a640 1 --2- 172.21.15.32:0/4062216571 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3ae8162320 0x7f3ad00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:29.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.808+0000 7f3aedf89640 1 -- 172.21.15.32:0/4062216571 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3ae8162320 msgr2=0x7f3ad00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:29.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.808+0000 7f3aedf89640 1 --2- 172.21.15.32:0/4062216571 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3ae8162320 0x7f3ad00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:29.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.808+0000 7f3aedf89640 1 -- 172.21.15.32:0/4062216571 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ad0004800 msgr2=0x7f3ad0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:29.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.808+0000 7f3aedf89640 1 --2- 172.21.15.32:0/4062216571 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ad0004800 0x7f3ad0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:29.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.808+0000 7f3aedf89640 1 -- 172.21.15.32:0/4062216571 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ad0003520 con 0x7f3ad00039b0 2024-04-24T23:00:29.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.809+0000 7f3aedf89640 1 --2- 172.21.15.32:0/4062216571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ad00039b0 0x7f3ad0003dd0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0009af0 tx=0x7f3ad0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70e3102d52ebb1b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:29.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.809+0000 7f3aecf87640 1 -- 172.21.15.32:0/4062216571 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae0036070 con 0x7f3ad00039b0 2024-04-24T23:00:29.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.809+0000 7f3aecf87640 1 -- 172.21.15.32:0/4062216571 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ae0031400 con 0x7f3ad00039b0 2024-04-24T23:00:29.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.809+0000 7f3aecf87640 1 -- 172.21.15.32:0/4062216571 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae0009690 con 0x7f3ad00039b0 2024-04-24T23:00:29.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.810+0000 7f3af01ed640 1 -- 172.21.15.32:0/4062216571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ad00039b0 msgr2=0x7f3ad0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:29.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.810+0000 7f3af01ed640 1 --2- 172.21.15.32:0/4062216571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ad00039b0 0x7f3ad0003dd0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0009af0 tx=0x7f3ad0005830 comp rx=0 tx=0).stop 2024-04-24T23:00:29.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.810+0000 7f3aee78a640 1 -- 172.21.15.32:0/4062216571 reap_dead start 2024-04-24T23:00:29.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.810+0000 7f3af01ed640 1 -- 172.21.15.32:0/4062216571 shutdown_connections 2024-04-24T23:00:29.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.810+0000 7f3af01ed640 1 -- 172.21.15.32:0/4062216571 >> 172.21.15.32:0/4062216571 conn(0x7f3ae80d0ae0 msgr2=0x7f3ae80d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:29.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.810+0000 7f3af01ed640 1 -- 172.21.15.32:0/4062216571 shutdown_connections 2024-04-24T23:00:29.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.810+0000 7f3af01ed640 1 -- 172.21.15.32:0/4062216571 wait complete. 2024-04-24T23:00:29.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.811+0000 7f3af01ed640 1 Processor -- start 2024-04-24T23:00:29.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.811+0000 7f3af01ed640 1 -- start start 2024-04-24T23:00:29.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.812+0000 7f3af01ed640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ad00039b0 0x7f3ae80b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:29.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.812+0000 7f3af01ed640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3ae806e000 0x7f3ae80cf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:29.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.812+0000 7f3af01ed640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ae806e970 0x7f3ae81612f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:29.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.812+0000 7f3af01ed640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae8071bd0 con 0x7f3ae806e970 2024-04-24T23:00:29.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.812+0000 7f3af01ed640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae8067950 con 0x7f3ad00039b0 2024-04-24T23:00:29.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.812+0000 7f3af01ed640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f3ae8071d50 con 0x7f3ae806e000 2024-04-24T23:00:29.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.812+0000 7f3aee78a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ae806e970 0x7f3ae81612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:29.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.812+0000 7f3aee78a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ae806e970 0x7f3ae81612f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46116/0 (socket says 172.21.15.32:46116) 2024-04-24T23:00:29.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.812+0000 7f3aee78a640 1 -- 172.21.15.32:0/1861380662 learned_addr learned my addr 172.21.15.32:0/1861380662 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:29.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.812+0000 7f3aedf89640 1 --2- 172.21.15.32:0/1861380662 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ad00039b0 0x7f3ae80b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:29.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.812+0000 7f3aed788640 1 --2- 172.21.15.32:0/1861380662 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3ae806e000 0x7f3ae80cf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:29.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.812+0000 7f3aee78a640 1 -- 172.21.15.32:0/1861380662 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3ae806e000 msgr2=0x7f3ae80cf9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:29.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.812+0000 7f3aee78a640 1 --2- 172.21.15.32:0/1861380662 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3ae806e000 0x7f3ae80cf9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:29.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.812+0000 7f3aee78a640 1 -- 172.21.15.32:0/1861380662 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ad00039b0 msgr2=0x7f3ae80b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:29.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.812+0000 7f3aee78a640 1 --2- 172.21.15.32:0/1861380662 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ad00039b0 0x7f3ae80b0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:29.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.812+0000 7f3aee78a640 1 -- 172.21.15.32:0/1861380662 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ae806cf60 con 0x7f3ae806e970 2024-04-24T23:00:29.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.812+0000 7f3aee78a640 1 --2- 172.21.15.32:0/1861380662 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ae806e970 0x7f3ae81612f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f3ae81631f0 tx=0x7f3ad800f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:29.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.813+0000 7f3ae67fc640 1 -- 172.21.15.32:0/1861380662 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad800fc80 con 0x7f3ae806e970 2024-04-24T23:00:29.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.813+0000 7f3ae67fc640 1 -- 172.21.15.32:0/1861380662 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ad801d070 con 0x7f3ae806e970 2024-04-24T23:00:29.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.813+0000 7f3af01ed640 1 -- 172.21.15.32:0/1861380662 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ae806d240 con 0x7f3ae806e970 2024-04-24T23:00:29.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.813+0000 7f3ae67fc640 1 -- 172.21.15.32:0/1861380662 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad8015430 con 0x7f3ae806e970 2024-04-24T23:00:29.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.813+0000 7f3af01ed640 1 -- 172.21.15.32:0/1861380662 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ae806d800 con 0x7f3ae806e970 2024-04-24T23:00:29.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.814+0000 7f3af01ed640 1 -- 172.21.15.32:0/1861380662 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ae8072a30 con 0x7f3ae806e970 2024-04-24T23:00:29.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.814+0000 7f3ae67fc640 1 -- 172.21.15.32:0/1861380662 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3ad80155d0 con 0x7f3ae806e970 2024-04-24T23:00:29.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.815+0000 7f3ae67fc640 1 --2- 172.21.15.32:0/1861380662 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3ad406be60 0x7f3ad406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:29.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.815+0000 7f3ae67fc640 1 -- 172.21.15.32:0/1861380662 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4857+0+0 (secure 0 0 0) 0x7f3ad808b2e0 con 0x7f3ae806e970 2024-04-24T23:00:29.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.815+0000 7f3aedf89640 1 --2- 172.21.15.32:0/1861380662 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3ad406be60 0x7f3ad406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:29.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.815+0000 7f3aedf89640 1 --2- 172.21.15.32:0/1861380662 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3ad406be60 0x7f3ad406e320 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0033000 tx=0x7f3ae0008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:29.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:29.819+0000 7f3ae67fc640 1 -- 172.21.15.32:0/1861380662 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3ae8072a30 con 0x7f3ae806e970 2024-04-24T23:00:30.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.006+0000 7f3af01ed640 1 -- 172.21.15.32:0/1861380662 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3ae8068430 con 0x7f3ae806e970 2024-04-24T23:00:30.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.007+0000 7f3ae67fc640 1 -- 172.21.15.32:0/1861380662 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) ==== 93+0+10 (secure 0 0 0) 0x7f3ad8010040 con 0x7f3ae806e970 2024-04-24T23:00:30.008 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T23:00:30.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.009+0000 7f3af01ed640 1 -- 172.21.15.32:0/1861380662 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3ad406be60 msgr2=0x7f3ad406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:30.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.009+0000 7f3af01ed640 1 --2- 172.21.15.32:0/1861380662 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3ad406be60 0x7f3ad406e320 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0033000 tx=0x7f3ae0008030 comp rx=0 tx=0).stop 2024-04-24T23:00:30.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.009+0000 7f3af01ed640 1 -- 172.21.15.32:0/1861380662 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ae806e970 msgr2=0x7f3ae81612f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:30.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.009+0000 7f3af01ed640 1 --2- 172.21.15.32:0/1861380662 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ae806e970 0x7f3ae81612f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f3ae81631f0 tx=0x7f3ad800f1b0 comp rx=0 tx=0).stop 2024-04-24T23:00:30.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.009+0000 7f3af01ed640 1 -- 172.21.15.32:0/1861380662 shutdown_connections 2024-04-24T23:00:30.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.009+0000 7f3af01ed640 1 --2- 172.21.15.32:0/1861380662 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3ad406be60 0x7f3ad406e320 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:30.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.009+0000 7f3af01ed640 1 --2- 172.21.15.32:0/1861380662 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ae806e970 0x7f3ae81612f0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:30.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.009+0000 7f3af01ed640 1 --2- 172.21.15.32:0/1861380662 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3ae806e000 0x7f3ae80cf9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:30.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.009+0000 7f3af01ed640 1 --2- 172.21.15.32:0/1861380662 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ad00039b0 0x7f3ae80b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:30.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.009+0000 7f3af01ed640 1 -- 172.21.15.32:0/1861380662 >> 172.21.15.32:0/1861380662 conn(0x7f3ae80d0ae0 msgr2=0x7f3ae80ae8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:30.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.009+0000 7f3af01ed640 1 -- 172.21.15.32:0/1861380662 shutdown_connections 2024-04-24T23:00:30.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.009+0000 7f3af01ed640 1 -- 172.21.15.32:0/1861380662 wait complete. 2024-04-24T23:00:30.021 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-24T23:00:30.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.121+0000 7f4dc1109640 1 Processor -- start 2024-04-24T23:00:30.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.121+0000 7f4dc1109640 1 -- start start 2024-04-24T23:00:30.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.121+0000 7f4dc1109640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4dbc06b820 0x7f4dbc0d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:30.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.121+0000 7f4dc1109640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4dbc0d4210 0x7f4dbc0b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:30.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.121+0000 7f4dc1109640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4dbc0d3170 con 0x7f4dbc15e9b0 2024-04-24T23:00:30.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.121+0000 7f4dc1109640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f4dbc0d3450 con 0x7f4dbc0d4210 2024-04-24T23:00:30.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.121+0000 7f4dc1109640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f4dbc0d3730 con 0x7f4dbc06b820 2024-04-24T23:00:30.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.122+0000 7f4dbb59e640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4dbc0d4210 0x7f4dbc0b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:30.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.122+0000 7f4db3fff640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4dbc06b820 0x7f4dbc0d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:30.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.122+0000 7f4dbad9d640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f4dbc15e9b0 0x7f4dbc06b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:44970/0 (socket says 172.21.15.32:44970) 2024-04-24T23:00:30.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.122+0000 7f4dbad9d640 1 -- 172.21.15.32:0/2316843192 learned_addr learned my addr 172.21.15.32:0/2316843192 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:00:30.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.122+0000 7f4db3fff640 1 -- 172.21.15.32:0/2316843192 >> v2:172.21.15.121:3300/0 conn(0x7f4dbc0d4210 msgr2=0x7f4dbc0b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:30.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.122+0000 7f4db3fff640 1 --2- 172.21.15.32:0/2316843192 >> v2:172.21.15.121:3300/0 conn(0x7f4dbc0d4210 0x7f4dbc0b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:30.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.122+0000 7f4db3fff640 1 -- 172.21.15.32:0/2316843192 >> v1:172.21.15.121:6789/0 conn(0x7f4dbc15e9b0 legacy=0x7f4dbc06b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:30.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.122+0000 7f4dba59c640 1 -- 172.21.15.32:0/2316843192 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2404768181 0 0) 0x7f4dbc0d3170 con 0x7f4dbc15e9b0 2024-04-24T23:00:30.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.122+0000 7f4db3fff640 1 -- 172.21.15.32:0/2316843192 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dbc0d3a10 con 0x7f4dbc06b820 2024-04-24T23:00:30.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.122+0000 7f4dbb59e640 1 --2- 172.21.15.32:0/2316843192 >> v2:172.21.15.121:3300/0 conn(0x7f4dbc0d4210 0x7f4dbc0b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:00:30.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.123+0000 7f4db3fff640 1 --2- 172.21.15.32:0/2316843192 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4dbc06b820 0x7f4dbc0d2b90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4da40097a0 tx=0x7f4da402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21f1615154933a87 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:30.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.123+0000 7f4dba59c640 1 -- 172.21.15.32:0/2316843192 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da4037070 con 0x7f4dbc06b820 2024-04-24T23:00:30.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.123+0000 7f4dba59c640 1 -- 172.21.15.32:0/2316843192 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4da4032440 con 0x7f4dbc06b820 2024-04-24T23:00:30.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.123+0000 7f4dba59c640 1 -- 172.21.15.32:0/2316843192 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da4032810 con 0x7f4dbc06b820 2024-04-24T23:00:30.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.123+0000 7f4dc1109640 1 -- 172.21.15.32:0/2316843192 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4dbc06b820 msgr2=0x7f4dbc0d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:30.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.123+0000 7f4dc1109640 1 --2- 172.21.15.32:0/2316843192 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4dbc06b820 0x7f4dbc0d2b90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4da40097a0 tx=0x7f4da402a180 comp rx=0 tx=0).stop 2024-04-24T23:00:30.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.124+0000 7f4dc1109640 1 -- 172.21.15.32:0/2316843192 shutdown_connections 2024-04-24T23:00:30.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.124+0000 7f4dc1109640 1 --2- 172.21.15.32:0/2316843192 >> v2:172.21.15.121:3300/0 conn(0x7f4dbc0d4210 0x7f4dbc0b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:30.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.124+0000 7f4dc1109640 1 --2- 172.21.15.32:0/2316843192 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4dbc06b820 0x7f4dbc0d2b90 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:30.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.124+0000 7f4dc1109640 1 -- 172.21.15.32:0/2316843192 >> 172.21.15.32:0/2316843192 conn(0x7f4dbc0d11d0 msgr2=0x7f4dbc0d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:30.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.124+0000 7f4dc1109640 1 -- 172.21.15.32:0/2316843192 shutdown_connections 2024-04-24T23:00:30.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.124+0000 7f4dc1109640 1 -- 172.21.15.32:0/2316843192 wait complete. 2024-04-24T23:00:30.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.125+0000 7f4dc1109640 1 Processor -- start 2024-04-24T23:00:30.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.125+0000 7f4dc1109640 1 -- start start 2024-04-24T23:00:30.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.126+0000 7f4dc1109640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4dbc06b820 0x7f4dbc1f3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:30.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.126+0000 7f4dc1109640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4dbc0d4210 0x7f4dbc144870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:30.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.126+0000 7f4dc1109640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4dbc15e9b0 0x7f4dbc0cefd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:30.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.126+0000 7f4dc1109640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4dbc0ba0e0 con 0x7f4dbc15e9b0 2024-04-24T23:00:30.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.126+0000 7f4dc1109640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4dbc0d2400 con 0x7f4dbc06b820 2024-04-24T23:00:30.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.126+0000 7f4dc1109640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f4dbc0ba260 con 0x7f4dbc0d4210 2024-04-24T23:00:30.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.126+0000 7f4dbb59e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4dbc15e9b0 0x7f4dbc0cefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:30.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.126+0000 7f4dbb59e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4dbc15e9b0 0x7f4dbc0cefd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46124/0 (socket says 172.21.15.32:46124) 2024-04-24T23:00:30.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.126+0000 7f4dbb59e640 1 -- 172.21.15.32:0/2742963632 learned_addr learned my addr 172.21.15.32:0/2742963632 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:30.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.126+0000 7f4dbad9d640 1 --2- 172.21.15.32:0/2742963632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4dbc06b820 0x7f4dbc1f3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:30.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.126+0000 7f4db3fff640 1 --2- 172.21.15.32:0/2742963632 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4dbc0d4210 0x7f4dbc144870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:30.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.126+0000 7f4dbb59e640 1 -- 172.21.15.32:0/2742963632 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4dbc0d4210 msgr2=0x7f4dbc144870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:30.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.126+0000 7f4dbb59e640 1 --2- 172.21.15.32:0/2742963632 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4dbc0d4210 0x7f4dbc144870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:30.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.126+0000 7f4dbb59e640 1 -- 172.21.15.32:0/2742963632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4dbc06b820 msgr2=0x7f4dbc1f3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:30.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.126+0000 7f4dbb59e640 1 --2- 172.21.15.32:0/2742963632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4dbc06b820 0x7f4dbc1f3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:30.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.126+0000 7f4dbb59e640 1 -- 172.21.15.32:0/2742963632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dbc145330 con 0x7f4dbc15e9b0 2024-04-24T23:00:30.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.126+0000 7f4dbb59e640 1 --2- 172.21.15.32:0/2742963632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4dbc15e9b0 0x7f4dbc0cefd0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f4db400efc0 tx=0x7f4db400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:30.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.127+0000 7f4db37fe640 1 -- 172.21.15.32:0/2742963632 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db4018030 con 0x7f4dbc15e9b0 2024-04-24T23:00:30.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.127+0000 7f4db37fe640 1 -- 172.21.15.32:0/2742963632 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4db400cdf0 con 0x7f4dbc15e9b0 2024-04-24T23:00:30.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.127+0000 7f4dc1109640 1 -- 172.21.15.32:0/2742963632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dbc145610 con 0x7f4dbc15e9b0 2024-04-24T23:00:30.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.127+0000 7f4db37fe640 1 -- 172.21.15.32:0/2742963632 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db4003bb0 con 0x7f4dbc15e9b0 2024-04-24T23:00:30.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.127+0000 7f4dc1109640 1 -- 172.21.15.32:0/2742963632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dbc145bd0 con 0x7f4dbc15e9b0 2024-04-24T23:00:30.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.128+0000 7f4dc1109640 1 -- 172.21.15.32:0/2742963632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dbc068430 con 0x7f4dbc15e9b0 2024-04-24T23:00:30.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.128+0000 7f4db37fe640 1 -- 172.21.15.32:0/2742963632 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4db401b070 con 0x7f4dbc15e9b0 2024-04-24T23:00:30.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.129+0000 7f4db37fe640 1 --2- 172.21.15.32:0/2742963632 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4d9c06be30 0x7f4d9c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:30.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.129+0000 7f4db37fe640 1 -- 172.21.15.32:0/2742963632 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4857+0+0 (secure 0 0 0) 0x7f4db408b470 con 0x7f4dbc15e9b0 2024-04-24T23:00:30.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.129+0000 7f4dbad9d640 1 --2- 172.21.15.32:0/2742963632 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4d9c06be30 0x7f4d9c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:30.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.130+0000 7f4dbad9d640 1 --2- 172.21.15.32:0/2742963632 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4d9c06be30 0x7f4d9c06e2f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4dac000f60 tx=0x7f4dac007d50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:30.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.133+0000 7f4db37fe640 1 -- 172.21.15.32:0/2742963632 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4db4010040 con 0x7f4dbc15e9b0 2024-04-24T23:00:30.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.321+0000 7f4dc1109640 1 -- 172.21.15.32:0/2742963632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f4dbc0723c0 con 0x7f4dbc15e9b0 2024-04-24T23:00:30.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.321+0000 7f4db37fe640 1 -- 172.21.15.32:0/2742963632 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v41) ==== 84+0+65 (secure 0 0 0) 0x7f4db4059aa0 con 0x7f4dbc15e9b0 2024-04-24T23:00:30.323 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:00:30.323 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2024-04-24T23:00:30.323 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:30.323 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2024-04-24T23:00:30.323 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": ".mgr" 2024-04-24T23:00:30.323 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:30.323 INFO:teuthology.orchestra.run.smithi032.stdout:] 2024-04-24T23:00:30.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.323+0000 7f4dc1109640 1 -- 172.21.15.32:0/2742963632 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4d9c06be30 msgr2=0x7f4d9c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:30.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.323+0000 7f4dc1109640 1 --2- 172.21.15.32:0/2742963632 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4d9c06be30 0x7f4d9c06e2f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4dac000f60 tx=0x7f4dac007d50 comp rx=0 tx=0).stop 2024-04-24T23:00:30.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.323+0000 7f4dc1109640 1 -- 172.21.15.32:0/2742963632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4dbc15e9b0 msgr2=0x7f4dbc0cefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:30.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.323+0000 7f4dc1109640 1 --2- 172.21.15.32:0/2742963632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4dbc15e9b0 0x7f4dbc0cefd0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f4db400efc0 tx=0x7f4db400c490 comp rx=0 tx=0).stop 2024-04-24T23:00:30.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.323+0000 7f4dc1109640 1 -- 172.21.15.32:0/2742963632 shutdown_connections 2024-04-24T23:00:30.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.323+0000 7f4dc1109640 1 --2- 172.21.15.32:0/2742963632 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4d9c06be30 0x7f4d9c06e2f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:30.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.323+0000 7f4dc1109640 1 --2- 172.21.15.32:0/2742963632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4dbc15e9b0 0x7f4dbc0cefd0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:30.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.323+0000 7f4dc1109640 1 --2- 172.21.15.32:0/2742963632 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4dbc0d4210 0x7f4dbc144870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:30.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.323+0000 7f4dc1109640 1 --2- 172.21.15.32:0/2742963632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4dbc06b820 0x7f4dbc1f3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:30.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.323+0000 7f4dc1109640 1 -- 172.21.15.32:0/2742963632 >> 172.21.15.32:0/2742963632 conn(0x7f4dbc0d11d0 msgr2=0x7f4dbc0d15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:30.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.324+0000 7f4dc1109640 1 -- 172.21.15.32:0/2742963632 shutdown_connections 2024-04-24T23:00:30.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.324+0000 7f4dc1109640 1 -- 172.21.15.32:0/2742963632 wait complete. 2024-04-24T23:00:30.336 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-24T23:00:30.336 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-24T23:00:30.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.439+0000 7ff99c893640 1 Processor -- start 2024-04-24T23:00:30.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.439+0000 7ff99c893640 1 -- start start 2024-04-24T23:00:30.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.439+0000 7ff99c893640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff99815c420 0x7ff9980af700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:30.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.439+0000 7ff99c893640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff9980b5f30 0x7ff9980b36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:30.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.439+0000 7ff99c893640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9980afc40 con 0x7ff9980b5390 2024-04-24T23:00:30.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.439+0000 7ff99c893640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7ff998160140 con 0x7ff9980b5f30 2024-04-24T23:00:30.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.439+0000 7ff99c893640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff998160fa0 con 0x7ff99815c420 2024-04-24T23:00:30.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.440+0000 7ff996d9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff9980b5f30 0x7ff9980b36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:30.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.440+0000 7ff996d9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff9980b5f30 0x7ff9980b36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46130/0 (socket says 172.21.15.32:46130) 2024-04-24T23:00:30.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.440+0000 7ff996d9d640 1 -- 172.21.15.32:0/1726679363 learned_addr learned my addr 172.21.15.32:0/1726679363 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:30.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.440+0000 7ff99659c640 1 --2- 172.21.15.32:0/1726679363 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff99815c420 0x7ff9980af700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:30.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.440+0000 7ff996d9d640 1 -- 172.21.15.32:0/1726679363 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff99815c420 msgr2=0x7ff9980af700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:30.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.440+0000 7ff996d9d640 1 --2- 172.21.15.32:0/1726679363 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff99815c420 0x7ff9980af700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:30.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.440+0000 7ff996d9d640 1 -- 172.21.15.32:0/1726679363 >> v1:172.21.15.121:6789/0 conn(0x7ff9980b5390 legacy=0x7ff9980b5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:30.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.440+0000 7ff996d9d640 1 -- 172.21.15.32:0/1726679363 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff998160390 con 0x7ff9980b5f30 2024-04-24T23:00:30.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.441+0000 7ff996d9d640 1 --2- 172.21.15.32:0/1726679363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9980b5f30 0x7ff9980b36d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff990009e50 tx=0x7ff99002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a109cf3e91f8d1a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:30.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.441+0000 7ff995d9b640 1 -- 172.21.15.32:0/1726679363 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff990036070 con 0x7ff9980b5f30 2024-04-24T23:00:30.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.441+0000 7ff995d9b640 1 -- 172.21.15.32:0/1726679363 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff990031600 con 0x7ff9980b5f30 2024-04-24T23:00:30.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.441+0000 7ff995d9b640 1 -- 172.21.15.32:0/1726679363 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9900319d0 con 0x7ff9980b5f30 2024-04-24T23:00:30.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.441+0000 7ff99c893640 1 -- 172.21.15.32:0/1726679363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9980b5f30 msgr2=0x7ff9980b36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:30.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.441+0000 7ff99c893640 1 --2- 172.21.15.32:0/1726679363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9980b5f30 0x7ff9980b36d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff990009e50 tx=0x7ff99002a6c0 comp rx=0 tx=0).stop 2024-04-24T23:00:30.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.441+0000 7ff99c893640 1 -- 172.21.15.32:0/1726679363 shutdown_connections 2024-04-24T23:00:30.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.441+0000 7ff99c893640 1 --2- 172.21.15.32:0/1726679363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9980b5f30 0x7ff9980b36d0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:30.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.441+0000 7ff99c893640 1 --2- 172.21.15.32:0/1726679363 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff99815c420 0x7ff9980af700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:30.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.441+0000 7ff99c893640 1 -- 172.21.15.32:0/1726679363 >> 172.21.15.32:0/1726679363 conn(0x7ff9980b1f10 msgr2=0x7ff9980b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:30.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.441+0000 7ff99c893640 1 -- 172.21.15.32:0/1726679363 shutdown_connections 2024-04-24T23:00:30.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.441+0000 7ff99c893640 1 -- 172.21.15.32:0/1726679363 wait complete. 2024-04-24T23:00:30.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.443+0000 7ff99c893640 1 Processor -- start 2024-04-24T23:00:30.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.443+0000 7ff99c893640 1 -- start start 2024-04-24T23:00:30.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.443+0000 7ff99c893640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9980b5390 0x7ff9981f4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:30.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.443+0000 7ff99c893640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9980b5f30 0x7ff99815c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:30.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.443+0000 7ff99c893640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff99815c420 0x7ff9980ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:30.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.443+0000 7ff99c893640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff998160c30 con 0x7ff9980b5390 2024-04-24T23:00:30.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.443+0000 7ff99c893640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff998160990 con 0x7ff99815c420 2024-04-24T23:00:30.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.443+0000 7ff99c893640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff9980717b0 con 0x7ff9980b5f30 2024-04-24T23:00:30.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.443+0000 7ff99659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9980b5390 0x7ff9981f4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:30.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.444+0000 7ff99659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9980b5390 0x7ff9981f4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46144/0 (socket says 172.21.15.32:46144) 2024-04-24T23:00:30.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.444+0000 7ff99659c640 1 -- 172.21.15.32:0/3450322976 learned_addr learned my addr 172.21.15.32:0/3450322976 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:30.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.444+0000 7ff996d9d640 1 --2- 172.21.15.32:0/3450322976 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff99815c420 0x7ff9980ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:30.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.444+0000 7ff98ffff640 1 --2- 172.21.15.32:0/3450322976 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9980b5f30 0x7ff99815c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:30.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.444+0000 7ff99659c640 1 -- 172.21.15.32:0/3450322976 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9980b5f30 msgr2=0x7ff99815c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:30.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.444+0000 7ff99659c640 1 --2- 172.21.15.32:0/3450322976 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9980b5f30 0x7ff99815c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:30.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.444+0000 7ff99659c640 1 -- 172.21.15.32:0/3450322976 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff99815c420 msgr2=0x7ff9980ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:30.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.444+0000 7ff99659c640 1 --2- 172.21.15.32:0/3450322976 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff99815c420 0x7ff9980ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:30.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.444+0000 7ff99659c640 1 -- 172.21.15.32:0/3450322976 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff99815d190 con 0x7ff9980b5390 2024-04-24T23:00:30.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.444+0000 7ff996d9d640 1 --2- 172.21.15.32:0/3450322976 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff99815c420 0x7ff9980ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:00:30.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.444+0000 7ff99659c640 1 --2- 172.21.15.32:0/3450322976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9980b5390 0x7ff9981f4640 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ff98800c5e0 tx=0x7ff98800caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:30.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.444+0000 7ff98effd640 1 -- 172.21.15.32:0/3450322976 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff988015070 con 0x7ff9980b5390 2024-04-24T23:00:30.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.444+0000 7ff98effd640 1 -- 172.21.15.32:0/3450322976 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff988004260 con 0x7ff9980b5390 2024-04-24T23:00:30.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.444+0000 7ff99c893640 1 -- 172.21.15.32:0/3450322976 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff99815d470 con 0x7ff9980b5390 2024-04-24T23:00:30.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.444+0000 7ff98effd640 1 -- 172.21.15.32:0/3450322976 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff988003610 con 0x7ff9980b5390 2024-04-24T23:00:30.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.444+0000 7ff99c893640 1 -- 172.21.15.32:0/3450322976 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff99815da30 con 0x7ff9980b5390 2024-04-24T23:00:30.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.445+0000 7ff99c893640 1 -- 172.21.15.32:0/3450322976 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9980691a0 con 0x7ff9980b5390 2024-04-24T23:00:30.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.446+0000 7ff98effd640 1 -- 172.21.15.32:0/3450322976 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff9880037b0 con 0x7ff9980b5390 2024-04-24T23:00:30.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.446+0000 7ff98effd640 1 --2- 172.21.15.32:0/3450322976 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff97806be30 0x7ff97806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:30.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.446+0000 7ff98effd640 1 -- 172.21.15.32:0/3450322976 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4857+0+0 (secure 0 0 0) 0x7ff98808a950 con 0x7ff9980b5390 2024-04-24T23:00:30.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.446+0000 7ff98ffff640 1 --2- 172.21.15.32:0/3450322976 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff97806be30 0x7ff97806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:30.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.447+0000 7ff98ffff640 1 --2- 172.21.15.32:0/3450322976 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff97806be30 0x7ff97806e2f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff998159840 tx=0x7ff980009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:30.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.451+0000 7ff98effd640 1 -- 172.21.15.32:0/3450322976 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff9980691a0 con 0x7ff9980b5390 2024-04-24T23:00:30.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:30.634+0000 7ff99c893640 1 -- 172.21.15.32:0/3450322976 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7ff9980b32d0 con 0x7ff9980b5390 2024-04-24T23:00:31.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.001+0000 7ff98effd640 1 -- 172.21.15.32:0/3450322976 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v42) ==== 138+0+0 (secure 0 0 0) 0x7ff9980b32d0 con 0x7ff9980b5390 2024-04-24T23:00:31.003 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2024-04-24T23:00:31.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.004+0000 7ff99c893640 1 -- 172.21.15.32:0/3450322976 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff97806be30 msgr2=0x7ff97806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:31.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.004+0000 7ff99c893640 1 --2- 172.21.15.32:0/3450322976 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff97806be30 0x7ff97806e2f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff998159840 tx=0x7ff980009040 comp rx=0 tx=0).stop 2024-04-24T23:00:31.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.004+0000 7ff99c893640 1 -- 172.21.15.32:0/3450322976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9980b5390 msgr2=0x7ff9981f4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:31.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.004+0000 7ff99c893640 1 --2- 172.21.15.32:0/3450322976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9980b5390 0x7ff9981f4640 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ff98800c5e0 tx=0x7ff98800caa0 comp rx=0 tx=0).stop 2024-04-24T23:00:31.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.004+0000 7ff99c893640 1 -- 172.21.15.32:0/3450322976 shutdown_connections 2024-04-24T23:00:31.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.004+0000 7ff99c893640 1 --2- 172.21.15.32:0/3450322976 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff97806be30 0x7ff97806e2f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:31.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.004+0000 7ff99c893640 1 --2- 172.21.15.32:0/3450322976 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff99815c420 0x7ff9980ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:31.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.004+0000 7ff99c893640 1 --2- 172.21.15.32:0/3450322976 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9980b5f30 0x7ff99815c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:31.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.004+0000 7ff99c893640 1 --2- 172.21.15.32:0/3450322976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9980b5390 0x7ff9981f4640 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:31.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.004+0000 7ff99c893640 1 -- 172.21.15.32:0/3450322976 >> 172.21.15.32:0/3450322976 conn(0x7ff9980b1f10 msgr2=0x7ff998159a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:31.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.006+0000 7ff99c893640 1 -- 172.21.15.32:0/3450322976 shutdown_connections 2024-04-24T23:00:31.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.006+0000 7ff99c893640 1 -- 172.21.15.32:0/3450322976 wait complete. 2024-04-24T23:00:31.019 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-24T23:00:31.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.127+0000 7f8aac472640 1 Processor -- start 2024-04-24T23:00:31.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.127+0000 7f8aac472640 1 -- start start 2024-04-24T23:00:31.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.128+0000 7f8aac472640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8aa415cc40 0x7f8aa415d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:31.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.128+0000 7f8aac472640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8aa40b4ee0 con 0x7f8aa40d05b0 2024-04-24T23:00:31.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.128+0000 7f8aac472640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8aa40b51c0 con 0x7f8aa415d560 2024-04-24T23:00:31.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.128+0000 7f8aac472640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f8aa406cca0 con 0x7f8aa415cc40 2024-04-24T23:00:31.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.128+0000 7f8aaa20e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8aa415cc40 0x7f8aa415d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:31.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.128+0000 7f8aa9a0d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8aa40d05b0 0x7f8aa40cec30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38416/0 (socket says 172.21.15.32:38416) 2024-04-24T23:00:31.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.128+0000 7f8aa9a0d640 1 -- 172.21.15.32:0/3191582154 learned_addr learned my addr 172.21.15.32:0/3191582154 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:00:31.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.128+0000 7f8aaa20e640 1 -- 172.21.15.32:0/3191582154 >> v1:172.21.15.121:6789/0 conn(0x7f8aa415d560 legacy=0x7f8aa40cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:31.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.128+0000 7f8aaa20e640 1 -- 172.21.15.32:0/3191582154 >> v1:172.21.15.32:6789/0 conn(0x7f8aa40d05b0 legacy=0x7f8aa40cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:31.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.128+0000 7f8aa920c640 1 -- 172.21.15.32:0/3191582154 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 710481155 0 0) 0x7f8aa40b4ee0 con 0x7f8aa40d05b0 2024-04-24T23:00:31.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.128+0000 7f8aaa20e640 1 -- 172.21.15.32:0/3191582154 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aa40b54a0 con 0x7f8aa415cc40 2024-04-24T23:00:31.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.129+0000 7f8aaa20e640 1 --2- 172.21.15.32:0/3191582154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8aa415cc40 0x7f8aa415d020 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8aa00097a0 tx=0x7f8aa002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14539d6329b57e1e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:31.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.129+0000 7f8aa920c640 1 -- 172.21.15.32:0/3191582154 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa0037070 con 0x7f8aa415cc40 2024-04-24T23:00:31.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.129+0000 7f8aa920c640 1 -- 172.21.15.32:0/3191582154 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8aa0032440 con 0x7f8aa415cc40 2024-04-24T23:00:31.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.129+0000 7f8aa920c640 1 -- 172.21.15.32:0/3191582154 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa0032810 con 0x7f8aa415cc40 2024-04-24T23:00:31.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.130+0000 7f8aac472640 1 -- 172.21.15.32:0/3191582154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8aa415cc40 msgr2=0x7f8aa415d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:31.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.130+0000 7f8aac472640 1 --2- 172.21.15.32:0/3191582154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8aa415cc40 0x7f8aa415d020 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8aa00097a0 tx=0x7f8aa002a1e0 comp rx=0 tx=0).stop 2024-04-24T23:00:31.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.130+0000 7f8aac472640 1 -- 172.21.15.32:0/3191582154 shutdown_connections 2024-04-24T23:00:31.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.130+0000 7f8aac472640 1 --2- 172.21.15.32:0/3191582154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8aa415cc40 0x7f8aa415d020 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:31.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.130+0000 7f8aac472640 1 -- 172.21.15.32:0/3191582154 >> 172.21.15.32:0/3191582154 conn(0x7f8aa40d0ae0 msgr2=0x7f8aa40d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:31.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.130+0000 7f8aac472640 1 -- 172.21.15.32:0/3191582154 shutdown_connections 2024-04-24T23:00:31.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.130+0000 7f8aac472640 1 -- 172.21.15.32:0/3191582154 wait complete. 2024-04-24T23:00:31.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.131+0000 7f8aac472640 1 Processor -- start 2024-04-24T23:00:31.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.131+0000 7f8aac472640 1 -- start start 2024-04-24T23:00:31.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.132+0000 7f8aac472640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8aa40d05b0 0x7f8aa4144880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:31.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.132+0000 7f8aac472640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8aa415cc40 0x7f8aa40715c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:31.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.132+0000 7f8aac472640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8aa415d560 0x7f8aa4141480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:31.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.132+0000 7f8aac472640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa40b5a60 con 0x7f8aa415cc40 2024-04-24T23:00:31.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.132+0000 7f8aac472640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa40ce5f0 con 0x7f8aa415d560 2024-04-24T23:00:31.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.132+0000 7f8aac472640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f8aa415d9b0 con 0x7f8aa40d05b0 2024-04-24T23:00:31.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.132+0000 7f8aa9a0d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8aa415cc40 0x7f8aa40715c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:31.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.132+0000 7f8aa9a0d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8aa415cc40 0x7f8aa40715c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46162/0 (socket says 172.21.15.32:46162) 2024-04-24T23:00:31.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.132+0000 7f8aa9a0d640 1 -- 172.21.15.32:0/1541515925 learned_addr learned my addr 172.21.15.32:0/1541515925 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:31.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.132+0000 7f8aaa20e640 1 --2- 172.21.15.32:0/1541515925 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8aa40d05b0 0x7f8aa4144880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:31.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.132+0000 7f8aaaa0f640 1 --2- 172.21.15.32:0/1541515925 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8aa415d560 0x7f8aa4141480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:31.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.132+0000 7f8aa9a0d640 1 -- 172.21.15.32:0/1541515925 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8aa40d05b0 msgr2=0x7f8aa4144880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:31.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.132+0000 7f8aa9a0d640 1 --2- 172.21.15.32:0/1541515925 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8aa40d05b0 0x7f8aa4144880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:31.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.132+0000 7f8aa9a0d640 1 -- 172.21.15.32:0/1541515925 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8aa415d560 msgr2=0x7f8aa4141480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:31.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.132+0000 7f8aa9a0d640 1 --2- 172.21.15.32:0/1541515925 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8aa415d560 0x7f8aa4141480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:31.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.132+0000 7f8aa9a0d640 1 -- 172.21.15.32:0/1541515925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aa4071d00 con 0x7f8aa415cc40 2024-04-24T23:00:31.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.132+0000 7f8aa9a0d640 1 --2- 172.21.15.32:0/1541515925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8aa415cc40 0x7f8aa40715c0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f8a9400c410 tx=0x7f8a9400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:31.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.132+0000 7f8a9affd640 1 -- 172.21.15.32:0/1541515925 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a940074e0 con 0x7f8aa415cc40 2024-04-24T23:00:31.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.132+0000 7f8a9affd640 1 -- 172.21.15.32:0/1541515925 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a940047e0 con 0x7f8aa415cc40 2024-04-24T23:00:31.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.132+0000 7f8aac472640 1 -- 172.21.15.32:0/1541515925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8aa4071fe0 con 0x7f8aa415cc40 2024-04-24T23:00:31.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.132+0000 7f8a9affd640 1 -- 172.21.15.32:0/1541515925 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a94004bd0 con 0x7f8aa415cc40 2024-04-24T23:00:31.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.132+0000 7f8aac472640 1 -- 172.21.15.32:0/1541515925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8aa40725a0 con 0x7f8aa415cc40 2024-04-24T23:00:31.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.133+0000 7f8aac472640 1 -- 172.21.15.32:0/1541515925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8aa415e2c0 con 0x7f8aa415cc40 2024-04-24T23:00:31.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.134+0000 7f8a9affd640 1 -- 172.21.15.32:0/1541515925 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8a94011070 con 0x7f8aa415cc40 2024-04-24T23:00:31.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.134+0000 7f8a9affd640 1 --2- 172.21.15.32:0/1541515925 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8a8806be30 0x7f8a8806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:31.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.135+0000 7f8a9affd640 1 -- 172.21.15.32:0/1541515925 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5227+0+0 (secure 0 0 0) 0x7f8a9408a7c0 con 0x7f8aa415cc40 2024-04-24T23:00:31.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.135+0000 7f8aaa20e640 1 --2- 172.21.15.32:0/1541515925 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8a8806be30 0x7f8a8806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:31.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.135+0000 7f8aaa20e640 1 --2- 172.21.15.32:0/1541515925 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8a8806be30 0x7f8a8806e2f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0034000 tx=0x7f8aa002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:31.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.139+0000 7f8a9affd640 1 -- 172.21.15.32:0/1541515925 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8aa415e2c0 con 0x7f8aa415cc40 2024-04-24T23:00:31.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:31.326+0000 7f8aac472640 1 -- 172.21.15.32:0/1541515925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f8aa4068430 con 0x7f8aa415cc40 2024-04-24T23:00:32.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.006+0000 7f8a9affd640 1 -- 172.21.15.32:0/1541515925 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v43) ==== 170+0+0 (secure 0 0 0) 0x7f8a940593e0 con 0x7f8aa415cc40 2024-04-24T23:00:32.007 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2024-04-24T23:00:32.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.016+0000 7f8aac472640 1 -- 172.21.15.32:0/1541515925 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8a8806be30 msgr2=0x7f8a8806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:32.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.016+0000 7f8aac472640 1 --2- 172.21.15.32:0/1541515925 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8a8806be30 0x7f8a8806e2f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0034000 tx=0x7f8aa002e000 comp rx=0 tx=0).stop 2024-04-24T23:00:32.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.016+0000 7f8aac472640 1 -- 172.21.15.32:0/1541515925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8aa415cc40 msgr2=0x7f8aa40715c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:32.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.016+0000 7f8aac472640 1 --2- 172.21.15.32:0/1541515925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8aa415cc40 0x7f8aa40715c0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f8a9400c410 tx=0x7f8a9400c8d0 comp rx=0 tx=0).stop 2024-04-24T23:00:32.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.016+0000 7f8aac472640 1 -- 172.21.15.32:0/1541515925 shutdown_connections 2024-04-24T23:00:32.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.016+0000 7f8aac472640 1 --2- 172.21.15.32:0/1541515925 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8a8806be30 0x7f8a8806e2f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:32.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.016+0000 7f8aac472640 1 --2- 172.21.15.32:0/1541515925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8aa415cc40 0x7f8aa40715c0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:32.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.016+0000 7f8aac472640 1 --2- 172.21.15.32:0/1541515925 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8aa415d560 0x7f8aa4141480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:32.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.016+0000 7f8aac472640 1 --2- 172.21.15.32:0/1541515925 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8aa40d05b0 0x7f8aa4144880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:32.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.016+0000 7f8aac472640 1 -- 172.21.15.32:0/1541515925 >> 172.21.15.32:0/1541515925 conn(0x7f8aa40d0ae0 msgr2=0x7f8aa415dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:32.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.017+0000 7f8aac472640 1 -- 172.21.15.32:0/1541515925 shutdown_connections 2024-04-24T23:00:32.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.019+0000 7f8aac472640 1 -- 172.21.15.32:0/1541515925 wait complete. 2024-04-24T23:00:32.034 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-24T23:00:32.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.141+0000 7f7391272640 1 Processor -- start 2024-04-24T23:00:32.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.141+0000 7f7391272640 1 -- start start 2024-04-24T23:00:32.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.142+0000 7f7391272640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f738c15edd0 0x7f738c0b5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:32.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.142+0000 7f7391272640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f738c0cd910 con 0x7f738c0b5560 2024-04-24T23:00:32.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.142+0000 7f7391272640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f738c0cdbf0 con 0x7f738c0cd3c0 2024-04-24T23:00:32.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.142+0000 7f7391272640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f738c0cded0 con 0x7f738c15edd0 2024-04-24T23:00:32.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.142+0000 7f738affd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f738c0b5560 0x7f738c15e6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38418/0 (socket says 172.21.15.32:38418) 2024-04-24T23:00:32.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.142+0000 7f738affd640 1 -- 172.21.15.32:0/2953653588 learned_addr learned my addr 172.21.15.32:0/2953653588 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:00:32.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.142+0000 7f738a7fc640 1 --2- 172.21.15.32:0/2953653588 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f738c15edd0 0x7f738c0b5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:32.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.142+0000 7f7389ffb640 1 -- 172.21.15.32:0/2953653588 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 249892134 0 0) 0x7f738c0cd910 con 0x7f738c0b5560 2024-04-24T23:00:32.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.142+0000 7f7389ffb640 1 -- 172.21.15.32:0/2953653588 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7374003440 con 0x7f738c0b5560 2024-04-24T23:00:32.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.142+0000 7f7389ffb640 1 -- 172.21.15.32:0/2953653588 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3551576754 0 0) 0x7f738c0cdbf0 con 0x7f738c0cd3c0 2024-04-24T23:00:32.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.142+0000 7f7389ffb640 1 -- 172.21.15.32:0/2953653588 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f738c0cd910 con 0x7f738c0cd3c0 2024-04-24T23:00:32.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.142+0000 7f7389ffb640 1 -- 172.21.15.32:0/2953653588 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1477693723 0 0) 0x7f7374003440 con 0x7f738c0b5560 2024-04-24T23:00:32.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.142+0000 7f738a7fc640 1 -- 172.21.15.32:0/2953653588 >> v1:172.21.15.121:6789/0 conn(0x7f738c0cd3c0 legacy=0x7f738c0b5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:32.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.142+0000 7f738a7fc640 1 -- 172.21.15.32:0/2953653588 >> v1:172.21.15.32:6789/0 conn(0x7f738c0b5560 legacy=0x7f738c15e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:32.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.142+0000 7f738a7fc640 1 -- 172.21.15.32:0/2953653588 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f738c0ce1b0 con 0x7f738c15edd0 2024-04-24T23:00:32.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.143+0000 7f738a7fc640 1 --2- 172.21.15.32:0/2953653588 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f738c15edd0 0x7f738c0b5940 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f73800097d0 tx=0x7f738002a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf434fd7bb536dd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:32.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.143+0000 7f7389ffb640 1 -- 172.21.15.32:0/2953653588 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7380037070 con 0x7f738c15edd0 2024-04-24T23:00:32.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.143+0000 7f7389ffb640 1 -- 172.21.15.32:0/2953653588 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f738002ac30 con 0x7f738c15edd0 2024-04-24T23:00:32.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.143+0000 7f7389ffb640 1 -- 172.21.15.32:0/2953653588 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f737c001670 con 0x7f738c15edd0 2024-04-24T23:00:32.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.144+0000 7f7391272640 1 -- 172.21.15.32:0/2953653588 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f738c15edd0 msgr2=0x7f738c0b5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:32.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.144+0000 7f7391272640 1 --2- 172.21.15.32:0/2953653588 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f738c15edd0 0x7f738c0b5940 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f73800097d0 tx=0x7f738002a1a0 comp rx=0 tx=0).stop 2024-04-24T23:00:32.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.144+0000 7f7391272640 1 -- 172.21.15.32:0/2953653588 shutdown_connections 2024-04-24T23:00:32.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.144+0000 7f7391272640 1 --2- 172.21.15.32:0/2953653588 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f738c15edd0 0x7f738c0b5940 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:32.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.144+0000 7f7391272640 1 -- 172.21.15.32:0/2953653588 >> 172.21.15.32:0/2953653588 conn(0x7f738c06b110 msgr2=0x7f738c06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:32.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.144+0000 7f7391272640 1 -- 172.21.15.32:0/2953653588 shutdown_connections 2024-04-24T23:00:32.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.144+0000 7f7391272640 1 -- 172.21.15.32:0/2953653588 wait complete. 2024-04-24T23:00:32.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.145+0000 7f7391272640 1 Processor -- start 2024-04-24T23:00:32.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.145+0000 7f7391272640 1 -- start start 2024-04-24T23:00:32.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.146+0000 7f7391272640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f738c0b5560 0x7f738c152db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:32.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.146+0000 7f7391272640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f738c0cd3c0 0x7f738c1efed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:32.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.146+0000 7f7391272640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f738c15edd0 0x7f738c157170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:32.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.146+0000 7f7391272640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f738c0b44a0 con 0x7f738c0cd3c0 2024-04-24T23:00:32.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.146+0000 7f7391272640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f738c0b4320 con 0x7f738c0b5560 2024-04-24T23:00:32.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.146+0000 7f7391272640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f738c06c5f0 con 0x7f738c15edd0 2024-04-24T23:00:32.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.146+0000 7f738a7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f738c0cd3c0 0x7f738c1efed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:32.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.146+0000 7f738a7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f738c0cd3c0 0x7f738c1efed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46164/0 (socket says 172.21.15.32:46164) 2024-04-24T23:00:32.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.146+0000 7f738a7fc640 1 -- 172.21.15.32:0/4130836789 learned_addr learned my addr 172.21.15.32:0/4130836789 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:32.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.146+0000 7f738b7fe640 1 --2- 172.21.15.32:0/4130836789 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f738c15edd0 0x7f738c157170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:32.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.146+0000 7f738affd640 1 --2- 172.21.15.32:0/4130836789 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f738c0b5560 0x7f738c152db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:32.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.146+0000 7f738a7fc640 1 -- 172.21.15.32:0/4130836789 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f738c15edd0 msgr2=0x7f738c157170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:32.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.146+0000 7f738a7fc640 1 --2- 172.21.15.32:0/4130836789 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f738c15edd0 0x7f738c157170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:32.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.146+0000 7f738a7fc640 1 -- 172.21.15.32:0/4130836789 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f738c0b5560 msgr2=0x7f738c152db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:32.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.146+0000 7f738a7fc640 1 --2- 172.21.15.32:0/4130836789 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f738c0b5560 0x7f738c152db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:32.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.146+0000 7f738a7fc640 1 -- 172.21.15.32:0/4130836789 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f738c0704c0 con 0x7f738c0cd3c0 2024-04-24T23:00:32.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.146+0000 7f738b7fe640 1 --2- 172.21.15.32:0/4130836789 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f738c15edd0 0x7f738c157170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:00:32.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.147+0000 7f738a7fc640 1 --2- 172.21.15.32:0/4130836789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f738c0cd3c0 0x7f738c1efed0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f73800097d0 tx=0x7f73800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:32.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.147+0000 7f73737fe640 1 -- 172.21.15.32:0/4130836789 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7380037050 con 0x7f738c0cd3c0 2024-04-24T23:00:32.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.147+0000 7f73737fe640 1 -- 172.21.15.32:0/4130836789 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7380038040 con 0x7f738c0cd3c0 2024-04-24T23:00:32.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.147+0000 7f7391272640 1 -- 172.21.15.32:0/4130836789 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f738c0707a0 con 0x7f738c0cd3c0 2024-04-24T23:00:32.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.147+0000 7f73737fe640 1 -- 172.21.15.32:0/4130836789 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7380006e10 con 0x7f738c0cd3c0 2024-04-24T23:00:32.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.147+0000 7f7391272640 1 -- 172.21.15.32:0/4130836789 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f738c070d60 con 0x7f738c0cd3c0 2024-04-24T23:00:32.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.148+0000 7f7391272640 1 -- 172.21.15.32:0/4130836789 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f738c068430 con 0x7f738c0cd3c0 2024-04-24T23:00:32.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.148+0000 7f73737fe640 1 -- 172.21.15.32:0/4130836789 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7380045020 con 0x7f738c0cd3c0 2024-04-24T23:00:32.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.149+0000 7f73737fe640 1 --2- 172.21.15.32:0/4130836789 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f737806be30 0x7f737806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:32.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.149+0000 7f73737fe640 1 -- 172.21.15.32:0/4130836789 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5609+0+0 (secure 0 0 0) 0x7f73800aade0 con 0x7f738c0cd3c0 2024-04-24T23:00:32.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.149+0000 7f738affd640 1 --2- 172.21.15.32:0/4130836789 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f737806be30 0x7f737806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:32.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.149+0000 7f738affd640 1 --2- 172.21.15.32:0/4130836789 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f737806be30 0x7f737806e2f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f737c0011e0 tx=0x7f737c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:32.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.153+0000 7f73737fe640 1 -- 172.21.15.32:0/4130836789 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f738004c020 con 0x7f738c0cd3c0 2024-04-24T23:00:32.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:32.374+0000 7f7391272640 1 -- 172.21.15.32:0/4130836789 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f738c0afd60 con 0x7f738c0cd3c0 2024-04-24T23:00:33.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.008+0000 7f73737fe640 1 -- 172.21.15.32:0/4130836789 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v18) ==== 172+0+0 (secure 0 0 0) 0x7f7380079880 con 0x7f738c0cd3c0 2024-04-24T23:00:33.010 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 6 and data pool 7 2024-04-24T23:00:33.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.012+0000 7f7391272640 1 -- 172.21.15.32:0/4130836789 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f737806be30 msgr2=0x7f737806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:33.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.012+0000 7f7391272640 1 --2- 172.21.15.32:0/4130836789 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f737806be30 0x7f737806e2f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f737c0011e0 tx=0x7f737c009040 comp rx=0 tx=0).stop 2024-04-24T23:00:33.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.012+0000 7f7391272640 1 -- 172.21.15.32:0/4130836789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f738c0cd3c0 msgr2=0x7f738c1efed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:33.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.012+0000 7f7391272640 1 --2- 172.21.15.32:0/4130836789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f738c0cd3c0 0x7f738c1efed0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f73800097d0 tx=0x7f73800046c0 comp rx=0 tx=0).stop 2024-04-24T23:00:33.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.013+0000 7f7391272640 1 -- 172.21.15.32:0/4130836789 shutdown_connections 2024-04-24T23:00:33.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.013+0000 7f7391272640 1 --2- 172.21.15.32:0/4130836789 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f737806be30 0x7f737806e2f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:33.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.013+0000 7f7391272640 1 --2- 172.21.15.32:0/4130836789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f738c0cd3c0 0x7f738c1efed0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:33.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.013+0000 7f7391272640 1 --2- 172.21.15.32:0/4130836789 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f738c15edd0 0x7f738c157170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:33.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.013+0000 7f7391272640 1 --2- 172.21.15.32:0/4130836789 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f738c0b5560 0x7f738c152db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:33.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.013+0000 7f7391272640 1 -- 172.21.15.32:0/4130836789 >> 172.21.15.32:0/4130836789 conn(0x7f738c06b110 msgr2=0x7f738c153e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:33.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.014+0000 7f7391272640 1 -- 172.21.15.32:0/4130836789 shutdown_connections 2024-04-24T23:00:33.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.014+0000 7f7391272640 1 -- 172.21.15.32:0/4130836789 wait complete. 2024-04-24T23:00:33.045 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:00:33.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.149+0000 7f1727b09640 1 Processor -- start 2024-04-24T23:00:33.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.149+0000 7f1727b09640 1 -- start start 2024-04-24T23:00:33.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.150+0000 7f1727b09640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f172015e9b0 0x7f17200d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:33.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.150+0000 7f1727b09640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f172006b820 0x7f17200b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:33.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.150+0000 7f1727b09640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17200d3170 con 0x7f17200d4210 2024-04-24T23:00:33.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.150+0000 7f1727b09640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f17200d3450 con 0x7f172006b820 2024-04-24T23:00:33.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.150+0000 7f1727b09640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f17200d3730 con 0x7f172015e9b0 2024-04-24T23:00:33.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.150+0000 7f17250a4640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f17200d4210 0x7f172006b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38430/0 (socket says 172.21.15.32:38430) 2024-04-24T23:00:33.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.150+0000 7f17250a4640 1 -- 172.21.15.32:0/1829979749 learned_addr learned my addr 172.21.15.32:0/1829979749 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:00:33.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.151+0000 7f17258a5640 1 --2- 172.21.15.32:0/1829979749 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f172015e9b0 0x7f17200d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:33.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.151+0000 7f17260a6640 1 --2- 172.21.15.32:0/1829979749 >> v2:172.21.15.121:3300/0 conn(0x7f172006b820 0x7f17200b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:33.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.151+0000 7f17248a3640 1 -- 172.21.15.32:0/1829979749 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3099131609 0 0) 0x7f17200d3170 con 0x7f17200d4210 2024-04-24T23:00:33.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.151+0000 7f17248a3640 1 -- 172.21.15.32:0/1829979749 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f170c003520 con 0x7f17200d4210 2024-04-24T23:00:33.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.151+0000 7f17248a3640 1 -- 172.21.15.32:0/1829979749 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1879389596 0 0) 0x7f170c003520 con 0x7f17200d4210 2024-04-24T23:00:33.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.151+0000 7f17248a3640 1 -- 172.21.15.32:0/1829979749 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f172015e9b0 msgr2=0x7f17200d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:33.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.151+0000 7f17248a3640 1 --2- 172.21.15.32:0/1829979749 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f172015e9b0 0x7f17200d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:33.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.151+0000 7f17248a3640 1 -- 172.21.15.32:0/1829979749 >> v2:172.21.15.121:3300/0 conn(0x7f172006b820 msgr2=0x7f17200b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:33.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.151+0000 7f17248a3640 1 --2- 172.21.15.32:0/1829979749 >> v2:172.21.15.121:3300/0 conn(0x7f172006b820 0x7f17200b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:33.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.151+0000 7f17248a3640 1 -- 172.21.15.32:0/1829979749 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17200d3a10 con 0x7f17200d4210 2024-04-24T23:00:33.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.151+0000 7f17248a3640 1 -- 172.21.15.32:0/1829979749 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f1710001670 con 0x7f17200d4210 2024-04-24T23:00:33.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.151+0000 7f17248a3640 1 -- 172.21.15.32:0/1829979749 >> v1:172.21.15.32:6789/0 conn(0x7f17200d4210 legacy=0x7f172006b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:33.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.151+0000 7f17248a3640 1 --2- 172.21.15.32:0/1829979749 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f172015e9b0 0x7f170c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:33.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.151+0000 7f17248a3640 1 --2- 172.21.15.32:0/1829979749 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f170c003f30 0x7f170c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:33.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.151+0000 7f17248a3640 1 --2- 172.21.15.32:0/1829979749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f170c004d40 0x7f170c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:33.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.151+0000 7f17248a3640 1 -- 172.21.15.32:0/1829979749 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f17200d3450 con 0x7f170c004d40 2024-04-24T23:00:33.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.151+0000 7f17248a3640 1 -- 172.21.15.32:0/1829979749 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f17200d3730 con 0x7f170c003f30 2024-04-24T23:00:33.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.151+0000 7f17248a3640 1 -- 172.21.15.32:0/1829979749 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f170c00b9c0 con 0x7f172015e9b0 2024-04-24T23:00:33.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.151+0000 7f17260a6640 1 --2- 172.21.15.32:0/1829979749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f170c004d40 0x7f170c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:33.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.151+0000 7f17258a5640 1 --2- 172.21.15.32:0/1829979749 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f172015e9b0 0x7f170c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:33.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.151+0000 7f17250a4640 1 --2- 172.21.15.32:0/1829979749 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f170c003f30 0x7f170c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:33.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.151+0000 7f17260a6640 1 -- 172.21.15.32:0/1829979749 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f172015e9b0 msgr2=0x7f170c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:33.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.151+0000 7f17260a6640 1 --2- 172.21.15.32:0/1829979749 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f172015e9b0 0x7f170c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:33.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.151+0000 7f17260a6640 1 -- 172.21.15.32:0/1829979749 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f170c003f30 msgr2=0x7f170c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:33.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.151+0000 7f17260a6640 1 --2- 172.21.15.32:0/1829979749 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f170c003f30 0x7f170c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:33.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.151+0000 7f17260a6640 1 -- 172.21.15.32:0/1829979749 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f170c003520 con 0x7f170c004d40 2024-04-24T23:00:33.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.151+0000 7f17250a4640 1 --2- 172.21.15.32:0/1829979749 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f170c003f30 0x7f170c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:00:33.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.152+0000 7f17260a6640 1 --2- 172.21.15.32:0/1829979749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f170c004d40 0x7f170c0051c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f172006bdf0 tx=0x7f170c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e38b781224c5a6be server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:33.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.152+0000 7f17248a3640 1 -- 172.21.15.32:0/1829979749 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1718009c10 con 0x7f170c004d40 2024-04-24T23:00:33.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.152+0000 7f17248a3640 1 -- 172.21.15.32:0/1829979749 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1718036070 con 0x7f170c004d40 2024-04-24T23:00:33.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.152+0000 7f17248a3640 1 -- 172.21.15.32:0/1829979749 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1718031670 con 0x7f170c004d40 2024-04-24T23:00:33.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.153+0000 7f1727b09640 1 -- 172.21.15.32:0/1829979749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f170c004d40 msgr2=0x7f170c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:33.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.153+0000 7f1727b09640 1 --2- 172.21.15.32:0/1829979749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f170c004d40 0x7f170c0051c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f172006bdf0 tx=0x7f170c005830 comp rx=0 tx=0).stop 2024-04-24T23:00:33.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.153+0000 7f17260a6640 1 -- 172.21.15.32:0/1829979749 reap_dead start 2024-04-24T23:00:33.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.153+0000 7f1727b09640 1 -- 172.21.15.32:0/1829979749 shutdown_connections 2024-04-24T23:00:33.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.153+0000 7f1727b09640 1 -- 172.21.15.32:0/1829979749 >> 172.21.15.32:0/1829979749 conn(0x7f17200d11d0 msgr2=0x7f17200d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:33.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.153+0000 7f1727b09640 1 -- 172.21.15.32:0/1829979749 shutdown_connections 2024-04-24T23:00:33.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.153+0000 7f1727b09640 1 -- 172.21.15.32:0/1829979749 wait complete. 2024-04-24T23:00:33.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.154+0000 7f1727b09640 1 Processor -- start 2024-04-24T23:00:33.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.154+0000 7f1727b09640 1 -- start start 2024-04-24T23:00:33.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.155+0000 7f1727b09640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f170c004d40 0x7f17201f17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:33.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.155+0000 7f1727b09640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1720159dc0 0x7f172015a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:33.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.155+0000 7f1727b09640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f172015ff30 0x7f17201603b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:33.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.155+0000 7f1727b09640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f17200ba0e0 con 0x7f172015ff30 2024-04-24T23:00:33.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.155+0000 7f1727b09640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f17200d2400 con 0x7f1720159dc0 2024-04-24T23:00:33.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.155+0000 7f1727b09640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f17200ba260 con 0x7f170c004d40 2024-04-24T23:00:33.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.155+0000 7f17260a6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f172015ff30 0x7f17201603b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:33.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.155+0000 7f17260a6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f172015ff30 0x7f17201603b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46188/0 (socket says 172.21.15.32:46188) 2024-04-24T23:00:33.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.155+0000 7f17260a6640 1 -- 172.21.15.32:0/1211811665 learned_addr learned my addr 172.21.15.32:0/1211811665 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:33.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.155+0000 7f17258a5640 1 --2- 172.21.15.32:0/1211811665 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f170c004d40 0x7f17201f17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:33.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.155+0000 7f17250a4640 1 --2- 172.21.15.32:0/1211811665 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1720159dc0 0x7f172015a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:33.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.155+0000 7f17260a6640 1 -- 172.21.15.32:0/1211811665 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f170c004d40 msgr2=0x7f17201f17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:33.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.155+0000 7f17260a6640 1 --2- 172.21.15.32:0/1211811665 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f170c004d40 0x7f17201f17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:33.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.155+0000 7f17260a6640 1 -- 172.21.15.32:0/1211811665 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1720159dc0 msgr2=0x7f172015a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:33.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.155+0000 7f17260a6640 1 --2- 172.21.15.32:0/1211811665 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1720159dc0 0x7f172015a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:33.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.155+0000 7f17260a6640 1 -- 172.21.15.32:0/1211811665 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1720160c60 con 0x7f172015ff30 2024-04-24T23:00:33.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.155+0000 7f17250a4640 1 --2- 172.21.15.32:0/1211811665 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1720159dc0 0x7f172015a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:00:33.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.155+0000 7f17260a6640 1 --2- 172.21.15.32:0/1211811665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f172015ff30 0x7f17201603b0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f1718033040 tx=0x7f1718009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:33.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.155+0000 7f17167fc640 1 -- 172.21.15.32:0/1211811665 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1718009c10 con 0x7f172015ff30 2024-04-24T23:00:33.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.156+0000 7f17167fc640 1 -- 172.21.15.32:0/1211811665 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1718031dd0 con 0x7f172015ff30 2024-04-24T23:00:33.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.156+0000 7f1727b09640 1 -- 172.21.15.32:0/1211811665 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1720160f40 con 0x7f172015ff30 2024-04-24T23:00:33.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.156+0000 7f17167fc640 1 -- 172.21.15.32:0/1211811665 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1718003bd0 con 0x7f172015ff30 2024-04-24T23:00:33.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.156+0000 7f1727b09640 1 -- 172.21.15.32:0/1211811665 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17201608f0 con 0x7f172015ff30 2024-04-24T23:00:33.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.157+0000 7f1727b09640 1 -- 172.21.15.32:0/1211811665 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1720068430 con 0x7f172015ff30 2024-04-24T23:00:33.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.157+0000 7f17167fc640 1 -- 172.21.15.32:0/1211811665 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1718005d10 con 0x7f172015ff30 2024-04-24T23:00:33.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.157+0000 7f17167fc640 1 --2- 172.21.15.32:0/1211811665 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f170406be60 0x7f170406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:33.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.157+0000 7f17167fc640 1 -- 172.21.15.32:0/1211811665 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f17180ac060 con 0x7f172015ff30 2024-04-24T23:00:33.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.158+0000 7f17258a5640 1 --2- 172.21.15.32:0/1211811665 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f170406be60 0x7f170406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:33.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.158+0000 7f17258a5640 1 --2- 172.21.15.32:0/1211811665 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f170406be60 0x7f170406e320 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f17200cffc0 tx=0x7f171c009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:33.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.162+0000 7f17167fc640 1 -- 172.21.15.32:0/1211811665 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1718049060 con 0x7f172015ff30 2024-04-24T23:00:33.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.351+0000 7f1727b09640 1 -- 172.21.15.32:0/1211811665 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f17200cfd10 con 0x7f172015ff30 2024-04-24T23:00:33.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.352+0000 7f17167fc640 1 -- 172.21.15.32:0/1211811665 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) ==== 74+0+16634 (secure 0 0 0) 0x7f171807aaa0 con 0x7f172015ff30 2024-04-24T23:00:33.354 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:00:33.354 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":44,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:00:32.375873+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:00:30.665592+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-24T23:00:31.357756+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:00:33.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.354+0000 7f1727b09640 1 -- 172.21.15.32:0/1211811665 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f170406be60 msgr2=0x7f170406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:33.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.354+0000 7f1727b09640 1 --2- 172.21.15.32:0/1211811665 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f170406be60 0x7f170406e320 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f17200cffc0 tx=0x7f171c009190 comp rx=0 tx=0).stop 2024-04-24T23:00:33.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.354+0000 7f1727b09640 1 -- 172.21.15.32:0/1211811665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f172015ff30 msgr2=0x7f17201603b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:33.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.354+0000 7f1727b09640 1 --2- 172.21.15.32:0/1211811665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f172015ff30 0x7f17201603b0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f1718033040 tx=0x7f1718009750 comp rx=0 tx=0).stop 2024-04-24T23:00:33.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.354+0000 7f1727b09640 1 -- 172.21.15.32:0/1211811665 shutdown_connections 2024-04-24T23:00:33.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.354+0000 7f1727b09640 1 --2- 172.21.15.32:0/1211811665 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f170406be60 0x7f170406e320 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:33.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.354+0000 7f1727b09640 1 --2- 172.21.15.32:0/1211811665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f172015ff30 0x7f17201603b0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:33.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.354+0000 7f1727b09640 1 --2- 172.21.15.32:0/1211811665 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1720159dc0 0x7f172015a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:33.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.354+0000 7f1727b09640 1 --2- 172.21.15.32:0/1211811665 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f170c004d40 0x7f17201f17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:33.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.354+0000 7f1727b09640 1 -- 172.21.15.32:0/1211811665 >> 172.21.15.32:0/1211811665 conn(0x7f17200d11d0 msgr2=0x7f17200cefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:33.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.354+0000 7f1727b09640 1 -- 172.21.15.32:0/1211811665 shutdown_connections 2024-04-24T23:00:33.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.354+0000 7f1727b09640 1 -- 172.21.15.32:0/1211811665 wait complete. 2024-04-24T23:00:33.367 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:00:33.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.466+0000 7f5b164c3640 1 Processor -- start 2024-04-24T23:00:33.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.466+0000 7f5b164c3640 1 -- start start 2024-04-24T23:00:33.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.467+0000 7f5b164c3640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5b100b44d0 0x7f5b100b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:33.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.467+0000 7f5b164c3640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5b100af890 0x7f5b1015cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:33.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.467+0000 7f5b164c3640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b100b6430 con 0x7f5b1006c5f0 2024-04-24T23:00:33.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.467+0000 7f5b164c3640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f5b10156f80 con 0x7f5b100af890 2024-04-24T23:00:33.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.467+0000 7f5b164c3640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f5b100b6150 con 0x7f5b100b44d0 2024-04-24T23:00:33.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.467+0000 7f5b0ffff640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f5b1006c5f0 0x7f5b1006c9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:45000/0 (socket says 172.21.15.32:45000) 2024-04-24T23:00:33.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.467+0000 7f5b14a60640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5b100af890 0x7f5b1015cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:33.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.467+0000 7f5b0f7fe640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5b100b44d0 0x7f5b100b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:33.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.467+0000 7f5b0ffff640 1 -- 172.21.15.32:0/1703323140 learned_addr learned my addr 172.21.15.32:0/1703323140 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:00:33.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.467+0000 7f5b0f7fe640 1 -- 172.21.15.32:0/1703323140 >> v2:172.21.15.121:3300/0 conn(0x7f5b100af890 msgr2=0x7f5b1015cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:33.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.467+0000 7f5b0f7fe640 1 --2- 172.21.15.32:0/1703323140 >> v2:172.21.15.121:3300/0 conn(0x7f5b100af890 0x7f5b1015cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:33.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.467+0000 7f5b0f7fe640 1 -- 172.21.15.32:0/1703323140 >> v1:172.21.15.121:6789/0 conn(0x7f5b1006c5f0 legacy=0x7f5b1006c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:33.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.467+0000 7f5b0f7fe640 1 -- 172.21.15.32:0/1703323140 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b100b39f0 con 0x7f5b100b44d0 2024-04-24T23:00:33.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.467+0000 7f5b0effd640 1 -- 172.21.15.32:0/1703323140 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3100632061 0 0) 0x7f5b100b6430 con 0x7f5b1006c5f0 2024-04-24T23:00:33.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.467+0000 7f5b14a60640 1 --2- 172.21.15.32:0/1703323140 >> v2:172.21.15.121:3300/0 conn(0x7f5b100af890 0x7f5b1015cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:00:33.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.468+0000 7f5b0f7fe640 1 --2- 172.21.15.32:0/1703323140 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5b100b44d0 0x7f5b100b5890 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5b000097a0 tx=0x7f5b0002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c551f7577b46a0af server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:33.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.468+0000 7f5b0effd640 1 -- 172.21.15.32:0/1703323140 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b00037070 con 0x7f5b100b44d0 2024-04-24T23:00:33.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.468+0000 7f5b0effd640 1 -- 172.21.15.32:0/1703323140 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b0002ae50 con 0x7f5b100b44d0 2024-04-24T23:00:33.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.468+0000 7f5b0effd640 1 -- 172.21.15.32:0/1703323140 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b00032660 con 0x7f5b100b44d0 2024-04-24T23:00:33.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.469+0000 7f5b164c3640 1 -- 172.21.15.32:0/1703323140 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5b100b44d0 msgr2=0x7f5b100b5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:33.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.469+0000 7f5b164c3640 1 --2- 172.21.15.32:0/1703323140 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5b100b44d0 0x7f5b100b5890 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5b000097a0 tx=0x7f5b0002a100 comp rx=0 tx=0).stop 2024-04-24T23:00:33.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.469+0000 7f5b164c3640 1 -- 172.21.15.32:0/1703323140 shutdown_connections 2024-04-24T23:00:33.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.469+0000 7f5b164c3640 1 --2- 172.21.15.32:0/1703323140 >> v2:172.21.15.121:3300/0 conn(0x7f5b100af890 0x7f5b1015cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:33.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.469+0000 7f5b164c3640 1 --2- 172.21.15.32:0/1703323140 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5b100b44d0 0x7f5b100b5890 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:33.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.469+0000 7f5b164c3640 1 -- 172.21.15.32:0/1703323140 >> 172.21.15.32:0/1703323140 conn(0x7f5b1006b110 msgr2=0x7f5b1006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:33.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.469+0000 7f5b164c3640 1 -- 172.21.15.32:0/1703323140 shutdown_connections 2024-04-24T23:00:33.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.469+0000 7f5b164c3640 1 -- 172.21.15.32:0/1703323140 wait complete. 2024-04-24T23:00:33.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.470+0000 7f5b164c3640 1 Processor -- start 2024-04-24T23:00:33.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.470+0000 7f5b164c3640 1 -- start start 2024-04-24T23:00:33.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.471+0000 7f5b164c3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b1006c5f0 0x7f5b1014c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:33.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.471+0000 7f5b164c3640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5b100af890 0x7f5b10159000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:33.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.471+0000 7f5b164c3640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b100b44d0 0x7f5b10159540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:33.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.471+0000 7f5b164c3640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5b100b5dd0 con 0x7f5b1006c5f0 2024-04-24T23:00:33.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.471+0000 7f5b164c3640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5b100b3fb0 con 0x7f5b100b44d0 2024-04-24T23:00:33.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.471+0000 7f5b164c3640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f5b1015d000 con 0x7f5b100af890 2024-04-24T23:00:33.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.471+0000 7f5b0ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b1006c5f0 0x7f5b1014c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:33.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.471+0000 7f5b0ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b1006c5f0 0x7f5b1014c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46200/0 (socket says 172.21.15.32:46200) 2024-04-24T23:00:33.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.471+0000 7f5b0ffff640 1 -- 172.21.15.32:0/3678034836 learned_addr learned my addr 172.21.15.32:0/3678034836 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:33.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.471+0000 7f5b0f7fe640 1 --2- 172.21.15.32:0/3678034836 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5b100af890 0x7f5b10159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:33.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.471+0000 7f5b14a60640 1 --2- 172.21.15.32:0/3678034836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b100b44d0 0x7f5b10159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:33.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.471+0000 7f5b0ffff640 1 -- 172.21.15.32:0/3678034836 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5b100af890 msgr2=0x7f5b10159000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:33.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.471+0000 7f5b0ffff640 1 --2- 172.21.15.32:0/3678034836 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5b100af890 0x7f5b10159000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:33.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.471+0000 7f5b0ffff640 1 -- 172.21.15.32:0/3678034836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b100b44d0 msgr2=0x7f5b10159540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:33.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.471+0000 7f5b0ffff640 1 --2- 172.21.15.32:0/3678034836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b100b44d0 0x7f5b10159540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:33.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.471+0000 7f5b0ffff640 1 -- 172.21.15.32:0/3678034836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b1015b3f0 con 0x7f5b1006c5f0 2024-04-24T23:00:33.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.472+0000 7f5b0f7fe640 1 --2- 172.21.15.32:0/3678034836 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5b100af890 0x7f5b10159000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:00:33.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.472+0000 7f5b14a60640 1 --2- 172.21.15.32:0/3678034836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b100b44d0 0x7f5b10159540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:00:33.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.472+0000 7f5b0ffff640 1 --2- 172.21.15.32:0/3678034836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b1006c5f0 0x7f5b1014c160 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f5b0400c4d0 tx=0x7f5b0400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:33.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.472+0000 7f5b0cff9640 1 -- 172.21.15.32:0/3678034836 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b04018070 con 0x7f5b1006c5f0 2024-04-24T23:00:33.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.472+0000 7f5b0cff9640 1 -- 172.21.15.32:0/3678034836 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b04007500 con 0x7f5b1006c5f0 2024-04-24T23:00:33.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.472+0000 7f5b164c3640 1 -- 172.21.15.32:0/3678034836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b1015b6d0 con 0x7f5b1006c5f0 2024-04-24T23:00:33.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.472+0000 7f5b0cff9640 1 -- 172.21.15.32:0/3678034836 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b04004a20 con 0x7f5b1006c5f0 2024-04-24T23:00:33.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.472+0000 7f5b164c3640 1 -- 172.21.15.32:0/3678034836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b1015bc90 con 0x7f5b1006c5f0 2024-04-24T23:00:33.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.473+0000 7f5b164c3640 1 -- 172.21.15.32:0/3678034836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b10068430 con 0x7f5b1006c5f0 2024-04-24T23:00:33.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.473+0000 7f5b0cff9640 1 -- 172.21.15.32:0/3678034836 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5b04004bc0 con 0x7f5b1006c5f0 2024-04-24T23:00:33.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.474+0000 7f5b0cff9640 1 --2- 172.21.15.32:0/3678034836 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5afc06be30 0x7f5afc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:33.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.474+0000 7f5b0cff9640 1 -- 172.21.15.32:0/3678034836 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f5b0408a1b0 con 0x7f5b1006c5f0 2024-04-24T23:00:33.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.474+0000 7f5b0f7fe640 1 --2- 172.21.15.32:0/3678034836 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5afc06be30 0x7f5afc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:33.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.474+0000 7f5b0f7fe640 1 --2- 172.21.15.32:0/3678034836 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5afc06be30 0x7f5afc06e2f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5b00034000 tx=0x7f5b0002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:33.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.478+0000 7f5b0cff9640 1 -- 172.21.15.32:0/3678034836 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5b04058ae0 con 0x7f5b1006c5f0 2024-04-24T23:00:33.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.661+0000 7f5b164c3640 1 -- 172.21.15.32:0/3678034836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5b100cc690 con 0x7f5b1006c5f0 2024-04-24T23:00:33.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.662+0000 7f5b0cff9640 1 -- 172.21.15.32:0/3678034836 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) ==== 74+0+16634 (secure 0 0 0) 0x7f5b040583e0 con 0x7f5b1006c5f0 2024-04-24T23:00:33.664 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:00:33.664 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":44,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:00:32.375873+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:00:30.665592+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-24T23:00:31.357756+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:00:33.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.664+0000 7f5b164c3640 1 -- 172.21.15.32:0/3678034836 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5afc06be30 msgr2=0x7f5afc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:33.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.664+0000 7f5b164c3640 1 --2- 172.21.15.32:0/3678034836 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5afc06be30 0x7f5afc06e2f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5b00034000 tx=0x7f5b0002e000 comp rx=0 tx=0).stop 2024-04-24T23:00:33.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.664+0000 7f5b164c3640 1 -- 172.21.15.32:0/3678034836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b1006c5f0 msgr2=0x7f5b1014c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:33.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.664+0000 7f5b164c3640 1 --2- 172.21.15.32:0/3678034836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b1006c5f0 0x7f5b1014c160 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f5b0400c4d0 tx=0x7f5b0400c990 comp rx=0 tx=0).stop 2024-04-24T23:00:33.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.664+0000 7f5b164c3640 1 -- 172.21.15.32:0/3678034836 shutdown_connections 2024-04-24T23:00:33.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.664+0000 7f5b164c3640 1 --2- 172.21.15.32:0/3678034836 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5afc06be30 0x7f5afc06e2f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:33.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.664+0000 7f5b164c3640 1 --2- 172.21.15.32:0/3678034836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b100b44d0 0x7f5b10159540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:33.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.664+0000 7f5b164c3640 1 --2- 172.21.15.32:0/3678034836 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5b100af890 0x7f5b10159000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:33.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.664+0000 7f5b164c3640 1 --2- 172.21.15.32:0/3678034836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b1006c5f0 0x7f5b1014c160 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:33.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.664+0000 7f5b164c3640 1 -- 172.21.15.32:0/3678034836 >> 172.21.15.32:0/3678034836 conn(0x7f5b1006b110 msgr2=0x7f5b100723e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:33.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.664+0000 7f5b164c3640 1 -- 172.21.15.32:0/3678034836 shutdown_connections 2024-04-24T23:00:33.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.664+0000 7f5b164c3640 1 -- 172.21.15.32:0/3678034836 wait complete. 2024-04-24T23:00:33.677 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-24T23:00:33.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.779+0000 7fef9ab61640 1 Processor -- start 2024-04-24T23:00:33.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.779+0000 7fef9ab61640 1 -- start start 2024-04-24T23:00:33.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.780+0000 7fef9ab61640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fef940b5af0 0x7fef940ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:33.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.780+0000 7fef9ab61640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef94162e10 0x7fef9415ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:33.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.780+0000 7fef9ab61640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef940cd270 con 0x7fef94162320 2024-04-24T23:00:33.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.780+0000 7fef9ab61640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fef940b5660 con 0x7fef940b5af0 2024-04-24T23:00:33.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.780+0000 7fef9ab61640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fef94071610 con 0x7fef94162e10 2024-04-24T23:00:33.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.780+0000 7fef990fe640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef94162e10 0x7fef9415ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:33.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.780+0000 7fef93fff640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fef940b5af0 0x7fef940ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:33.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.780+0000 7fef988fd640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fef94162320 0x7fef94162700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:45016/0 (socket says 172.21.15.32:45016) 2024-04-24T23:00:33.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.780+0000 7fef988fd640 1 -- 172.21.15.32:0/278734226 learned_addr learned my addr 172.21.15.32:0/278734226 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:00:33.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.781+0000 7fef937fe640 1 -- 172.21.15.32:0/278734226 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2844148212 0 0) 0x7fef940cd270 con 0x7fef94162320 2024-04-24T23:00:33.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.781+0000 7fef990fe640 1 -- 172.21.15.32:0/278734226 >> v2:172.21.15.121:3300/0 conn(0x7fef940b5af0 msgr2=0x7fef940ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:33.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.781+0000 7fef990fe640 1 --2- 172.21.15.32:0/278734226 >> v2:172.21.15.121:3300/0 conn(0x7fef940b5af0 0x7fef940ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:33.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.781+0000 7fef990fe640 1 -- 172.21.15.32:0/278734226 >> v1:172.21.15.121:6789/0 conn(0x7fef94162320 legacy=0x7fef94162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:33.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.781+0000 7fef990fe640 1 -- 172.21.15.32:0/278734226 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef940718f0 con 0x7fef94162e10 2024-04-24T23:00:33.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.782+0000 7fef990fe640 1 --2- 172.21.15.32:0/278734226 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef94162e10 0x7fef9415ed30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fef840097a0 tx=0x7fef8402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b32aa0339a4f6e5b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:33.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.782+0000 7fef937fe640 1 -- 172.21.15.32:0/278734226 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef84037070 con 0x7fef94162e10 2024-04-24T23:00:33.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.782+0000 7fef937fe640 1 -- 172.21.15.32:0/278734226 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef8402ae50 con 0x7fef94162e10 2024-04-24T23:00:33.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.782+0000 7fef937fe640 1 -- 172.21.15.32:0/278734226 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef84032660 con 0x7fef94162e10 2024-04-24T23:00:33.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.782+0000 7fef9ab61640 1 -- 172.21.15.32:0/278734226 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef94162e10 msgr2=0x7fef9415ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:33.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.782+0000 7fef9ab61640 1 --2- 172.21.15.32:0/278734226 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef94162e10 0x7fef9415ed30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fef840097a0 tx=0x7fef8402a100 comp rx=0 tx=0).stop 2024-04-24T23:00:33.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.782+0000 7fef9ab61640 1 -- 172.21.15.32:0/278734226 shutdown_connections 2024-04-24T23:00:33.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.782+0000 7fef9ab61640 1 --2- 172.21.15.32:0/278734226 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef94162e10 0x7fef9415ed30 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:33.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.782+0000 7fef9ab61640 1 --2- 172.21.15.32:0/278734226 >> v2:172.21.15.121:3300/0 conn(0x7fef940b5af0 0x7fef940ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:33.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.782+0000 7fef9ab61640 1 -- 172.21.15.32:0/278734226 >> 172.21.15.32:0/278734226 conn(0x7fef940d0ae0 msgr2=0x7fef940d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:33.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.782+0000 7fef9ab61640 1 -- 172.21.15.32:0/278734226 shutdown_connections 2024-04-24T23:00:33.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.782+0000 7fef9ab61640 1 -- 172.21.15.32:0/278734226 wait complete. 2024-04-24T23:00:33.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.783+0000 7fef9ab61640 1 Processor -- start 2024-04-24T23:00:33.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.783+0000 7fef9ab61640 1 -- start start 2024-04-24T23:00:33.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.784+0000 7fef9ab61640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef940b5af0 0x7fef941f4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:33.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.784+0000 7fef9ab61640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef94162320 0x7fef941558f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:33.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.784+0000 7fef9ab61640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef94162e10 0x7fef941f4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:33.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.784+0000 7fef9ab61640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fef94071bd0 con 0x7fef94162e10 2024-04-24T23:00:33.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.784+0000 7fef9ab61640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fef94067950 con 0x7fef94162320 2024-04-24T23:00:33.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.784+0000 7fef9ab61640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fef94071d50 con 0x7fef940b5af0 2024-04-24T23:00:33.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.784+0000 7fef990fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef94162e10 0x7fef941f4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:33.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.784+0000 7fef990fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef94162e10 0x7fef941f4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46212/0 (socket says 172.21.15.32:46212) 2024-04-24T23:00:33.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.784+0000 7fef990fe640 1 -- 172.21.15.32:0/284117598 learned_addr learned my addr 172.21.15.32:0/284117598 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:33.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.785+0000 7fef988fd640 1 --2- 172.21.15.32:0/284117598 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef940b5af0 0x7fef941f4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:33.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.785+0000 7fef93fff640 1 --2- 172.21.15.32:0/284117598 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef94162320 0x7fef941558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:33.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.785+0000 7fef990fe640 1 -- 172.21.15.32:0/284117598 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef940b5af0 msgr2=0x7fef941f4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:33.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.785+0000 7fef990fe640 1 --2- 172.21.15.32:0/284117598 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef940b5af0 0x7fef941f4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:33.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.785+0000 7fef990fe640 1 -- 172.21.15.32:0/284117598 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef94162320 msgr2=0x7fef941558f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:33.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.785+0000 7fef990fe640 1 --2- 172.21.15.32:0/284117598 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef94162320 0x7fef941558f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:33.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.785+0000 7fef990fe640 1 -- 172.21.15.32:0/284117598 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef94156160 con 0x7fef94162e10 2024-04-24T23:00:33.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.785+0000 7fef988fd640 1 --2- 172.21.15.32:0/284117598 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef940b5af0 0x7fef941f4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:00:33.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.785+0000 7fef990fe640 1 --2- 172.21.15.32:0/284117598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef94162e10 0x7fef941f4ca0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fef8402a140 tx=0x7fef840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:33.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.785+0000 7fef917fa640 1 -- 172.21.15.32:0/284117598 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef84041020 con 0x7fef94162e10 2024-04-24T23:00:33.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.785+0000 7fef917fa640 1 -- 172.21.15.32:0/284117598 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef8403c670 con 0x7fef94162e10 2024-04-24T23:00:33.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.785+0000 7fef917fa640 1 -- 172.21.15.32:0/284117598 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef84037040 con 0x7fef94162e10 2024-04-24T23:00:33.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.785+0000 7fef9ab61640 1 -- 172.21.15.32:0/284117598 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef94156440 con 0x7fef94162e10 2024-04-24T23:00:33.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.785+0000 7fef9ab61640 1 -- 172.21.15.32:0/284117598 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef94156a00 con 0x7fef94162e10 2024-04-24T23:00:33.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.786+0000 7fef9ab61640 1 -- 172.21.15.32:0/284117598 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef94068430 con 0x7fef94162e10 2024-04-24T23:00:33.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.787+0000 7fef917fa640 1 -- 172.21.15.32:0/284117598 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fef84044020 con 0x7fef94162e10 2024-04-24T23:00:33.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.787+0000 7fef917fa640 1 --2- 172.21.15.32:0/284117598 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fef7406be30 0x7fef7406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:33.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.787+0000 7fef917fa640 1 -- 172.21.15.32:0/284117598 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7fef840ab620 con 0x7fef94162e10 2024-04-24T23:00:33.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.787+0000 7fef988fd640 1 --2- 172.21.15.32:0/284117598 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fef7406be30 0x7fef7406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:33.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.788+0000 7fef988fd640 1 --2- 172.21.15.32:0/284117598 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fef7406be30 0x7fef7406e2f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fef8c0012e0 tx=0x7fef8c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:33.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:33.791+0000 7fef917fa640 1 -- 172.21.15.32:0/284117598 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fef8404b030 con 0x7fef94162e10 2024-04-24T23:00:34.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.017+0000 7fef9ab61640 1 -- 172.21.15.32:0/284117598 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fef94067730 con 0x7fef94162e10 2024-04-24T23:00:34.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.030+0000 7fef917fa640 1 -- 172.21.15.32:0/284117598 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v19) ==== 118+0+0 (secure 0 0 0) 0x7fef84079fd0 con 0x7fef94162e10 2024-04-24T23:00:34.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.033+0000 7fef9ab61640 1 -- 172.21.15.32:0/284117598 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fef7406be30 msgr2=0x7fef7406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:34.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.033+0000 7fef9ab61640 1 --2- 172.21.15.32:0/284117598 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fef7406be30 0x7fef7406e2f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fef8c0012e0 tx=0x7fef8c009040 comp rx=0 tx=0).stop 2024-04-24T23:00:34.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.033+0000 7fef9ab61640 1 -- 172.21.15.32:0/284117598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef94162e10 msgr2=0x7fef941f4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:34.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.033+0000 7fef9ab61640 1 --2- 172.21.15.32:0/284117598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef94162e10 0x7fef941f4ca0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fef8402a140 tx=0x7fef840046c0 comp rx=0 tx=0).stop 2024-04-24T23:00:34.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.033+0000 7fef9ab61640 1 -- 172.21.15.32:0/284117598 shutdown_connections 2024-04-24T23:00:34.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.033+0000 7fef9ab61640 1 --2- 172.21.15.32:0/284117598 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fef7406be30 0x7fef7406e2f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:34.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.033+0000 7fef9ab61640 1 --2- 172.21.15.32:0/284117598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef94162e10 0x7fef941f4ca0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:34.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.033+0000 7fef9ab61640 1 --2- 172.21.15.32:0/284117598 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef94162320 0x7fef941558f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:34.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.033+0000 7fef9ab61640 1 --2- 172.21.15.32:0/284117598 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef940b5af0 0x7fef941f4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:34.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.033+0000 7fef9ab61640 1 -- 172.21.15.32:0/284117598 >> 172.21.15.32:0/284117598 conn(0x7fef940d0ae0 msgr2=0x7fef940b3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:34.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.033+0000 7fef9ab61640 1 -- 172.21.15.32:0/284117598 shutdown_connections 2024-04-24T23:00:34.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.033+0000 7fef9ab61640 1 -- 172.21.15.32:0/284117598 wait complete. 2024-04-24T23:00:34.045 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-24T23:00:34.045 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-24T23:00:34.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.147+0000 7f8e7228c640 1 Processor -- start 2024-04-24T23:00:34.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.147+0000 7f8e7228c640 1 -- start start 2024-04-24T23:00:34.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.148+0000 7f8e7228c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8e6c158dd0 0x7f8e6c06b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:34.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.148+0000 7f8e7228c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8e6c06dc60 0x7f8e6c0b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:34.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.148+0000 7f8e7228c640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e6c15ef10 con 0x7f8e6c06b860 2024-04-24T23:00:34.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.148+0000 7f8e7228c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f8e6c15f1f0 con 0x7f8e6c06dc60 2024-04-24T23:00:34.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.148+0000 7f8e7228c640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f8e6c15f4d0 con 0x7f8e6c158dd0 2024-04-24T23:00:34.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.148+0000 7f8e70829640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8e6c06dc60 0x7f8e6c0b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:34.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.148+0000 7f8e70829640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8e6c06dc60 0x7f8e6c0b5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46222/0 (socket says 172.21.15.32:46222) 2024-04-24T23:00:34.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.148+0000 7f8e70829640 1 -- 172.21.15.32:0/4109949426 learned_addr learned my addr 172.21.15.32:0/4109949426 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:34.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.148+0000 7f8e70829640 1 -- 172.21.15.32:0/4109949426 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8e6c158dd0 msgr2=0x7f8e6c06b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:34.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.148+0000 7f8e70829640 1 --2- 172.21.15.32:0/4109949426 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8e6c158dd0 0x7f8e6c06b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:34.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.149+0000 7f8e70829640 1 -- 172.21.15.32:0/4109949426 >> v1:172.21.15.121:6789/0 conn(0x7f8e6c06b860 legacy=0x7f8e6c06d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:34.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.149+0000 7f8e70829640 1 -- 172.21.15.32:0/4109949426 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e6c15f7b0 con 0x7f8e6c06dc60 2024-04-24T23:00:34.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.149+0000 7f8e70829640 1 --2- 172.21.15.32:0/4109949426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e6c06dc60 0x7f8e6c0b5d00 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f8e64009e50 tx=0x7f8e6402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5151f93ef43ea339 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:34.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.149+0000 7f8e6affd640 1 -- 172.21.15.32:0/4109949426 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e64036070 con 0x7f8e6c06dc60 2024-04-24T23:00:34.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.149+0000 7f8e6affd640 1 -- 172.21.15.32:0/4109949426 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e64031600 con 0x7f8e6c06dc60 2024-04-24T23:00:34.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.149+0000 7f8e6affd640 1 -- 172.21.15.32:0/4109949426 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e640319d0 con 0x7f8e6c06dc60 2024-04-24T23:00:34.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.150+0000 7f8e7228c640 1 -- 172.21.15.32:0/4109949426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e6c06dc60 msgr2=0x7f8e6c0b5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:34.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.150+0000 7f8e7228c640 1 --2- 172.21.15.32:0/4109949426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e6c06dc60 0x7f8e6c0b5d00 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f8e64009e50 tx=0x7f8e6402a6c0 comp rx=0 tx=0).stop 2024-04-24T23:00:34.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.150+0000 7f8e7228c640 1 -- 172.21.15.32:0/4109949426 shutdown_connections 2024-04-24T23:00:34.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.150+0000 7f8e7228c640 1 --2- 172.21.15.32:0/4109949426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e6c06dc60 0x7f8e6c0b5d00 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:34.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.150+0000 7f8e7228c640 1 --2- 172.21.15.32:0/4109949426 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8e6c158dd0 0x7f8e6c06b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:34.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.150+0000 7f8e7228c640 1 -- 172.21.15.32:0/4109949426 >> 172.21.15.32:0/4109949426 conn(0x7f8e6c06e800 msgr2=0x7f8e6c06ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:34.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.150+0000 7f8e7228c640 1 -- 172.21.15.32:0/4109949426 shutdown_connections 2024-04-24T23:00:34.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.150+0000 7f8e7228c640 1 -- 172.21.15.32:0/4109949426 wait complete. 2024-04-24T23:00:34.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.151+0000 7f8e7228c640 1 Processor -- start 2024-04-24T23:00:34.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.151+0000 7f8e7228c640 1 -- start start 2024-04-24T23:00:34.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.152+0000 7f8e7228c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8e6c06b860 0x7f8e6c1f8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:34.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.152+0000 7f8e7228c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e6c06dc60 0x7f8e6c1f4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:34.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.152+0000 7f8e7228c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e6c158dd0 0x7f8e6c151b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:34.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.152+0000 7f8e7228c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6c06bfe0 con 0x7f8e6c06dc60 2024-04-24T23:00:34.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.152+0000 7f8e7228c640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6c15fcd0 con 0x7f8e6c158dd0 2024-04-24T23:00:34.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.152+0000 7f8e7228c640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f8e6c1591b0 con 0x7f8e6c06b860 2024-04-24T23:00:34.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.152+0000 7f8e6b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e6c06dc60 0x7f8e6c1f4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:34.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.152+0000 7f8e6b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e6c06dc60 0x7f8e6c1f4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46236/0 (socket says 172.21.15.32:46236) 2024-04-24T23:00:34.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.152+0000 7f8e6b7fe640 1 -- 172.21.15.32:0/462998736 learned_addr learned my addr 172.21.15.32:0/462998736 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:34.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.152+0000 7f8e6bfff640 1 --2- 172.21.15.32:0/462998736 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8e6c06b860 0x7f8e6c1f8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:34.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.153+0000 7f8e70829640 1 --2- 172.21.15.32:0/462998736 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e6c158dd0 0x7f8e6c151b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:34.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.153+0000 7f8e6b7fe640 1 -- 172.21.15.32:0/462998736 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8e6c06b860 msgr2=0x7f8e6c1f8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:34.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.153+0000 7f8e6b7fe640 1 --2- 172.21.15.32:0/462998736 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8e6c06b860 0x7f8e6c1f8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:34.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.153+0000 7f8e6b7fe640 1 -- 172.21.15.32:0/462998736 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e6c158dd0 msgr2=0x7f8e6c151b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:34.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.153+0000 7f8e6b7fe640 1 --2- 172.21.15.32:0/462998736 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e6c158dd0 0x7f8e6c151b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:34.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.153+0000 7f8e6b7fe640 1 -- 172.21.15.32:0/462998736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e6c1f48e0 con 0x7f8e6c06dc60 2024-04-24T23:00:34.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.153+0000 7f8e6bfff640 1 --2- 172.21.15.32:0/462998736 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8e6c06b860 0x7f8e6c1f8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:00:34.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.153+0000 7f8e70829640 1 --2- 172.21.15.32:0/462998736 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e6c158dd0 0x7f8e6c151b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:00:34.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.153+0000 7f8e6b7fe640 1 --2- 172.21.15.32:0/462998736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e6c06dc60 0x7f8e6c1f4080 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f8e6000c410 tx=0x7f8e6000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:34.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.153+0000 7f8e68ff9640 1 -- 172.21.15.32:0/462998736 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e60017070 con 0x7f8e6c06dc60 2024-04-24T23:00:34.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.153+0000 7f8e68ff9640 1 -- 172.21.15.32:0/462998736 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e600035f0 con 0x7f8e6c06dc60 2024-04-24T23:00:34.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.153+0000 7f8e7228c640 1 -- 172.21.15.32:0/462998736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e6c1f4bc0 con 0x7f8e6c06dc60 2024-04-24T23:00:34.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.153+0000 7f8e68ff9640 1 -- 172.21.15.32:0/462998736 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e600039e0 con 0x7f8e6c06dc60 2024-04-24T23:00:34.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.153+0000 7f8e7228c640 1 -- 172.21.15.32:0/462998736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e6c1f45c0 con 0x7f8e6c06dc60 2024-04-24T23:00:34.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.154+0000 7f8e7228c640 1 -- 172.21.15.32:0/462998736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e6c068430 con 0x7f8e6c06dc60 2024-04-24T23:00:34.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.155+0000 7f8e68ff9640 1 -- 172.21.15.32:0/462998736 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8e6001a070 con 0x7f8e6c06dc60 2024-04-24T23:00:34.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.155+0000 7f8e68ff9640 1 --2- 172.21.15.32:0/462998736 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8e5806be30 0x7f8e5806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:34.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.155+0000 7f8e68ff9640 1 -- 172.21.15.32:0/462998736 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f8e6008a330 con 0x7f8e6c06dc60 2024-04-24T23:00:34.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.155+0000 7f8e6bfff640 1 --2- 172.21.15.32:0/462998736 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8e5806be30 0x7f8e5806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:34.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.156+0000 7f8e6bfff640 1 --2- 172.21.15.32:0/462998736 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8e5806be30 0x7f8e5806e2f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c00a870 tx=0x7f8e5c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:34.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.159+0000 7f8e68ff9640 1 -- 172.21.15.32:0/462998736 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8e60058ce0 con 0x7f8e6c06dc60 2024-04-24T23:00:34.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:34.380+0000 7f8e7228c640 1 -- 172.21.15.32:0/462998736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f8e6c074e60 con 0x7f8e6c06dc60 2024-04-24T23:00:34.997 INFO:tasks.ceph.mds.c.smithi032.stderr:2024-04-24T23:00:34.996+0000 7fe53fc8f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T23:00:35.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.037+0000 7f8e68ff9640 1 -- 172.21.15.32:0/462998736 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v20) ==== 122+0+0 (secure 0 0 0) 0x7f8e600585e0 con 0x7f8e6c06dc60 2024-04-24T23:00:35.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.040+0000 7f8e7228c640 1 -- 172.21.15.32:0/462998736 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8e5806be30 msgr2=0x7f8e5806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:35.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.040+0000 7f8e7228c640 1 --2- 172.21.15.32:0/462998736 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8e5806be30 0x7f8e5806e2f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c00a870 tx=0x7f8e5c008040 comp rx=0 tx=0).stop 2024-04-24T23:00:35.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.040+0000 7f8e7228c640 1 -- 172.21.15.32:0/462998736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e6c06dc60 msgr2=0x7f8e6c1f4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:35.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.040+0000 7f8e7228c640 1 --2- 172.21.15.32:0/462998736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e6c06dc60 0x7f8e6c1f4080 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f8e6000c410 tx=0x7f8e6000c8d0 comp rx=0 tx=0).stop 2024-04-24T23:00:35.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.040+0000 7f8e7228c640 1 -- 172.21.15.32:0/462998736 shutdown_connections 2024-04-24T23:00:35.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.040+0000 7f8e7228c640 1 --2- 172.21.15.32:0/462998736 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8e5806be30 0x7f8e5806e2f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:35.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.040+0000 7f8e7228c640 1 --2- 172.21.15.32:0/462998736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e6c06dc60 0x7f8e6c1f4080 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:35.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.040+0000 7f8e7228c640 1 --2- 172.21.15.32:0/462998736 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e6c158dd0 0x7f8e6c151b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:35.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.040+0000 7f8e7228c640 1 --2- 172.21.15.32:0/462998736 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8e6c06b860 0x7f8e6c1f8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:35.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.040+0000 7f8e7228c640 1 -- 172.21.15.32:0/462998736 >> 172.21.15.32:0/462998736 conn(0x7f8e6c06e800 msgr2=0x7f8e6c1619a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:35.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.040+0000 7f8e7228c640 1 -- 172.21.15.32:0/462998736 shutdown_connections 2024-04-24T23:00:35.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.040+0000 7f8e7228c640 1 -- 172.21.15.32:0/462998736 wait complete. 2024-04-24T23:00:35.054 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-24T23:00:35.055 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:00:35.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.159+0000 7f8911bc1640 1 Processor -- start 2024-04-24T23:00:35.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.159+0000 7f8911bc1640 1 -- start start 2024-04-24T23:00:35.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.159+0000 7f8911bc1640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f890c15b480 0x7f890c0b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:35.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.159+0000 7f8911bc1640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f890c06cd00 0x7f890c0cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:35.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.159+0000 7f8911bc1640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f890c0b46c0 con 0x7f890c15fd60 2024-04-24T23:00:35.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.159+0000 7f8911bc1640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f890c0b4ff0 con 0x7f890c06cd00 2024-04-24T23:00:35.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.159+0000 7f8911bc1640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f890c0b49a0 con 0x7f890c15b480 2024-04-24T23:00:35.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.160+0000 7f890affd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f890c15fd60 0x7f890c06c5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38446/0 (socket says 172.21.15.32:38446) 2024-04-24T23:00:35.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.160+0000 7f890affd640 1 -- 172.21.15.32:0/1152211859 learned_addr learned my addr 172.21.15.32:0/1152211859 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:00:35.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.160+0000 7f890b7fe640 1 --2- 172.21.15.32:0/1152211859 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f890c15b480 0x7f890c0b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:35.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.160+0000 7f890a7fc640 1 -- 172.21.15.32:0/1152211859 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1444875633 0 0) 0x7f890c0b46c0 con 0x7f890c15fd60 2024-04-24T23:00:35.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.160+0000 7f890bfff640 1 --2- 172.21.15.32:0/1152211859 >> v2:172.21.15.121:3300/0 conn(0x7f890c06cd00 0x7f890c0cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:35.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.160+0000 7f890a7fc640 1 -- 172.21.15.32:0/1152211859 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f88f4003520 con 0x7f890c15fd60 2024-04-24T23:00:35.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.160+0000 7f890a7fc640 1 -- 172.21.15.32:0/1152211859 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1590781967 0 0) 0x7f88f4003520 con 0x7f890c15fd60 2024-04-24T23:00:35.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.160+0000 7f890a7fc640 1 -- 172.21.15.32:0/1152211859 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f890c15b480 msgr2=0x7f890c0b5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:35.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.160+0000 7f890a7fc640 1 --2- 172.21.15.32:0/1152211859 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f890c15b480 0x7f890c0b5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:35.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.160+0000 7f890a7fc640 1 -- 172.21.15.32:0/1152211859 >> v2:172.21.15.121:3300/0 conn(0x7f890c06cd00 msgr2=0x7f890c0cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:35.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.160+0000 7f890a7fc640 1 --2- 172.21.15.32:0/1152211859 >> v2:172.21.15.121:3300/0 conn(0x7f890c06cd00 0x7f890c0cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:35.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.160+0000 7f890a7fc640 1 -- 172.21.15.32:0/1152211859 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f890c0b4c80 con 0x7f890c15fd60 2024-04-24T23:00:35.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.160+0000 7f890a7fc640 1 -- 172.21.15.32:0/1152211859 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f8900001670 con 0x7f890c15fd60 2024-04-24T23:00:35.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.160+0000 7f890a7fc640 1 -- 172.21.15.32:0/1152211859 >> v1:172.21.15.32:6789/0 conn(0x7f890c15fd60 legacy=0x7f890c06c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:35.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.160+0000 7f890a7fc640 1 --2- 172.21.15.32:0/1152211859 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f88f40039b0 0x7f88f4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:35.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.160+0000 7f890a7fc640 1 --2- 172.21.15.32:0/1152211859 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88f4004800 0x7f88f4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:35.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.160+0000 7f890a7fc640 1 --2- 172.21.15.32:0/1152211859 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f890c15b480 0x7f88f40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:35.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.160+0000 7f890a7fc640 1 -- 172.21.15.32:0/1152211859 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f890c0b4ff0 con 0x7f88f4004800 2024-04-24T23:00:35.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.160+0000 7f890a7fc640 1 -- 172.21.15.32:0/1152211859 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f890c0b49a0 con 0x7f88f40039b0 2024-04-24T23:00:35.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.160+0000 7f890a7fc640 1 -- 172.21.15.32:0/1152211859 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f88f400b9c0 con 0x7f890c15b480 2024-04-24T23:00:35.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.160+0000 7f890affd640 1 --2- 172.21.15.32:0/1152211859 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88f4004800 0x7f88f4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:35.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.160+0000 7f890b7fe640 1 --2- 172.21.15.32:0/1152211859 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f88f40039b0 0x7f88f4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:35.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.160+0000 7f890bfff640 1 --2- 172.21.15.32:0/1152211859 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f890c15b480 0x7f88f40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:35.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.161+0000 7f890affd640 1 -- 172.21.15.32:0/1152211859 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f890c15b480 msgr2=0x7f88f40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:35.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.161+0000 7f890affd640 1 --2- 172.21.15.32:0/1152211859 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f890c15b480 0x7f88f40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:35.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.161+0000 7f890affd640 1 -- 172.21.15.32:0/1152211859 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f88f40039b0 msgr2=0x7f88f4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:35.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.161+0000 7f890affd640 1 --2- 172.21.15.32:0/1152211859 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f88f40039b0 0x7f88f4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:35.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.161+0000 7f890affd640 1 -- 172.21.15.32:0/1152211859 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88f4003520 con 0x7f88f4004800 2024-04-24T23:00:35.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.161+0000 7f890bfff640 1 --2- 172.21.15.32:0/1152211859 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f890c15b480 0x7f88f40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:00:35.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.161+0000 7f890affd640 1 --2- 172.21.15.32:0/1152211859 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88f4004800 0x7f88f4004c80 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f890000c160 tx=0x7f88f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db924f5256e59710 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:35.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.161+0000 7f890a7fc640 1 -- 172.21.15.32:0/1152211859 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8900007c80 con 0x7f88f4004800 2024-04-24T23:00:35.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.161+0000 7f890a7fc640 1 -- 172.21.15.32:0/1152211859 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8900005af0 con 0x7f88f4004800 2024-04-24T23:00:35.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.162+0000 7f890a7fc640 1 -- 172.21.15.32:0/1152211859 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8900007070 con 0x7f88f4004800 2024-04-24T23:00:35.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.162+0000 7f8911bc1640 1 -- 172.21.15.32:0/1152211859 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88f4004800 msgr2=0x7f88f4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:35.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.162+0000 7f8911bc1640 1 --2- 172.21.15.32:0/1152211859 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88f4004800 0x7f88f4004c80 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f890000c160 tx=0x7f88f4005830 comp rx=0 tx=0).stop 2024-04-24T23:00:35.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.162+0000 7f890bfff640 1 -- 172.21.15.32:0/1152211859 reap_dead start 2024-04-24T23:00:35.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.162+0000 7f8911bc1640 1 -- 172.21.15.32:0/1152211859 shutdown_connections 2024-04-24T23:00:35.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.162+0000 7f8911bc1640 1 -- 172.21.15.32:0/1152211859 >> 172.21.15.32:0/1152211859 conn(0x7f890c06b110 msgr2=0x7f890c06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:35.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.162+0000 7f8911bc1640 1 -- 172.21.15.32:0/1152211859 shutdown_connections 2024-04-24T23:00:35.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.162+0000 7f8911bc1640 1 -- 172.21.15.32:0/1152211859 wait complete. 2024-04-24T23:00:35.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.163+0000 7f8911bc1640 1 Processor -- start 2024-04-24T23:00:35.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.163+0000 7f8911bc1640 1 -- start start 2024-04-24T23:00:35.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.164+0000 7f8911bc1640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f88f4004800 0x7f890c14d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:35.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.164+0000 7f8911bc1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f890c14ab70 0x7f890c15bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:35.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.164+0000 7f8911bc1640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f890c0ae8b0 0x7f890c0aed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:35.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.164+0000 7f8911bc1640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f890c15b860 con 0x7f890c14ab70 2024-04-24T23:00:35.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.164+0000 7f8911bc1640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f890c0b5c60 con 0x7f890c0ae8b0 2024-04-24T23:00:35.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.164+0000 7f8911bc1640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f890c15c660 con 0x7f88f4004800 2024-04-24T23:00:35.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.164+0000 7f890affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f890c14ab70 0x7f890c15bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:35.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.164+0000 7f890affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f890c14ab70 0x7f890c15bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46264/0 (socket says 172.21.15.32:46264) 2024-04-24T23:00:35.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.164+0000 7f890affd640 1 -- 172.21.15.32:0/2531038370 learned_addr learned my addr 172.21.15.32:0/2531038370 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:35.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.164+0000 7f890b7fe640 1 --2- 172.21.15.32:0/2531038370 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f88f4004800 0x7f890c14d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:35.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.164+0000 7f890bfff640 1 --2- 172.21.15.32:0/2531038370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f890c0ae8b0 0x7f890c0aed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:35.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.164+0000 7f890affd640 1 -- 172.21.15.32:0/2531038370 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f88f4004800 msgr2=0x7f890c14d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:35.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.165+0000 7f890affd640 1 --2- 172.21.15.32:0/2531038370 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f88f4004800 0x7f890c14d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:35.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.165+0000 7f890affd640 1 -- 172.21.15.32:0/2531038370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f890c0ae8b0 msgr2=0x7f890c0aed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:35.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.165+0000 7f890affd640 1 --2- 172.21.15.32:0/2531038370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f890c0ae8b0 0x7f890c0aed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:35.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.165+0000 7f890affd640 1 -- 172.21.15.32:0/2531038370 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f890c15c900 con 0x7f890c14ab70 2024-04-24T23:00:35.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.165+0000 7f890affd640 1 --2- 172.21.15.32:0/2531038370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f890c14ab70 0x7f890c15bb70 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f8900000c00 tx=0x7f890c0b4f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:35.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.165+0000 7f88f3fff640 1 -- 172.21.15.32:0/2531038370 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8900041030 con 0x7f890c14ab70 2024-04-24T23:00:35.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.165+0000 7f88f3fff640 1 -- 172.21.15.32:0/2531038370 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8900038040 con 0x7f890c14ab70 2024-04-24T23:00:35.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.165+0000 7f8911bc1640 1 -- 172.21.15.32:0/2531038370 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f890c15cbe0 con 0x7f890c14ab70 2024-04-24T23:00:35.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.165+0000 7f88f3fff640 1 -- 172.21.15.32:0/2531038370 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8900001650 con 0x7f890c14ab70 2024-04-24T23:00:35.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.165+0000 7f8911bc1640 1 -- 172.21.15.32:0/2531038370 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f890c15d1a0 con 0x7f890c14ab70 2024-04-24T23:00:35.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.166+0000 7f8911bc1640 1 -- 172.21.15.32:0/2531038370 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f890c068430 con 0x7f890c14ab70 2024-04-24T23:00:35.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.166+0000 7f88f3fff640 1 -- 172.21.15.32:0/2531038370 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8900044070 con 0x7f890c14ab70 2024-04-24T23:00:35.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.167+0000 7f88f3fff640 1 --2- 172.21.15.32:0/2531038370 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f88f806be30 0x7f88f806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:35.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.167+0000 7f88f3fff640 1 -- 172.21.15.32:0/2531038370 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f89000aa970 con 0x7f890c14ab70 2024-04-24T23:00:35.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.167+0000 7f890b7fe640 1 --2- 172.21.15.32:0/2531038370 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f88f806be30 0x7f88f806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:35.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.168+0000 7f890b7fe640 1 --2- 172.21.15.32:0/2531038370 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f88f806be30 0x7f88f806e2f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f88fc0098a0 tx=0x7f88fc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:35.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.171+0000 7f88f3fff640 1 -- 172.21.15.32:0/2531038370 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f89000793b0 con 0x7f890c14ab70 2024-04-24T23:00:35.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.394+0000 7f8911bc1640 1 -- 172.21.15.32:0/2531038370 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f890c06c000 con 0x7f890c14ab70 2024-04-24T23:00:35.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.395+0000 7f88f3fff640 1 -- 172.21.15.32:0/2531038370 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5187 (secure 0 0 0) 0x7f8900078cb0 con 0x7f890c14ab70 2024-04-24T23:00:35.396 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:00:35.396 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":20,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4994,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/1076304225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1076304225},{"type":"v1","addr":"172.21.15.32:6834","nonce":1076304225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":16},{"gid":5000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1388367206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1388367206},{"type":"v1","addr":"172.21.15.121:6835","nonce":1388367206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":5299,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2065945523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2065945523},{"type":"v1","addr":"172.21.15.121:6837","nonce":2065945523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:00:32.376194+0000","modified":"2024-04-24T23:00:34.381949+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"c","rank":0,"incarnation":18,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/3274703471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":3274703471},{"type":"v1","addr":"172.21.15.32:6835","nonce":3274703471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5631,"qdb_cluster":[5631]},"id":3}]} 2024-04-24T23:00:35.397 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 20 2024-04-24T23:00:35.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.397+0000 7f8911bc1640 1 -- 172.21.15.32:0/2531038370 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f88f806be30 msgr2=0x7f88f806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:35.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.397+0000 7f8911bc1640 1 --2- 172.21.15.32:0/2531038370 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f88f806be30 0x7f88f806e2f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f88fc0098a0 tx=0x7f88fc008040 comp rx=0 tx=0).stop 2024-04-24T23:00:35.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.397+0000 7f8911bc1640 1 -- 172.21.15.32:0/2531038370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f890c14ab70 msgr2=0x7f890c15bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:35.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.397+0000 7f8911bc1640 1 --2- 172.21.15.32:0/2531038370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f890c14ab70 0x7f890c15bb70 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f8900000c00 tx=0x7f890c0b4f60 comp rx=0 tx=0).stop 2024-04-24T23:00:35.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.397+0000 7f8911bc1640 1 -- 172.21.15.32:0/2531038370 shutdown_connections 2024-04-24T23:00:35.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.397+0000 7f8911bc1640 1 --2- 172.21.15.32:0/2531038370 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f88f806be30 0x7f88f806e2f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:35.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.397+0000 7f8911bc1640 1 --2- 172.21.15.32:0/2531038370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f890c14ab70 0x7f890c15bb70 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:35.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.397+0000 7f8911bc1640 1 --2- 172.21.15.32:0/2531038370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f890c0ae8b0 0x7f890c0aed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:35.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.397+0000 7f8911bc1640 1 --2- 172.21.15.32:0/2531038370 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f88f4004800 0x7f890c14d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:35.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.397+0000 7f8911bc1640 1 -- 172.21.15.32:0/2531038370 >> 172.21.15.32:0/2531038370 conn(0x7f890c06b110 msgr2=0x7f890c0aff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:35.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.397+0000 7f8911bc1640 1 -- 172.21.15.32:0/2531038370 shutdown_connections 2024-04-24T23:00:35.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.397+0000 7f8911bc1640 1 -- 172.21.15.32:0/2531038370 wait complete. 2024-04-24T23:00:35.410 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:00:35.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.509+0000 7fac8ad09640 1 Processor -- start 2024-04-24T23:00:35.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.509+0000 7fac8ad09640 1 -- start start 2024-04-24T23:00:35.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.510+0000 7fac8ad09640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fac8415b480 0x7fac840b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:35.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.510+0000 7fac8ad09640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fac8415fd60 0x7fac8406c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:35.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.510+0000 7fac8ad09640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac840cef90 con 0x7fac8406cb30 2024-04-24T23:00:35.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.510+0000 7fac8ad09640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fac840b4ff0 con 0x7fac8415b480 2024-04-24T23:00:35.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.510+0000 7fac8ad09640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fac840b46c0 con 0x7fac8415fd60 2024-04-24T23:00:35.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.510+0000 7fac88aa5640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fac8415b480 0x7fac840b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:35.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.510+0000 7fac83fff640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fac8415fd60 0x7fac8406c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:35.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.510+0000 7fac892a6640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fac8406cb30 0x7fac840ccc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:45032/0 (socket says 172.21.15.32:45032) 2024-04-24T23:00:35.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.510+0000 7fac892a6640 1 -- 172.21.15.32:0/3109545703 learned_addr learned my addr 172.21.15.32:0/3109545703 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:00:35.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.511+0000 7fac83fff640 1 -- 172.21.15.32:0/3109545703 >> v2:172.21.15.121:3300/0 conn(0x7fac8415b480 msgr2=0x7fac840b5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:35.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.511+0000 7fac83fff640 1 --2- 172.21.15.32:0/3109545703 >> v2:172.21.15.121:3300/0 conn(0x7fac8415b480 0x7fac840b5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:35.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.511+0000 7fac83fff640 1 -- 172.21.15.32:0/3109545703 >> v1:172.21.15.121:6789/0 conn(0x7fac8406cb30 legacy=0x7fac840ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:35.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.511+0000 7fac83fff640 1 -- 172.21.15.32:0/3109545703 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac840b49a0 con 0x7fac8415fd60 2024-04-24T23:00:35.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.512+0000 7fac83fff640 1 --2- 172.21.15.32:0/3109545703 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fac8415fd60 0x7fac8406c5f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fac780097a0 tx=0x7fac7802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a915b7bcfd6abd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:35.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.512+0000 7fac837fe640 1 -- 172.21.15.32:0/3109545703 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac78037070 con 0x7fac8415fd60 2024-04-24T23:00:35.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.512+0000 7fac837fe640 1 -- 172.21.15.32:0/3109545703 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac78032440 con 0x7fac8415fd60 2024-04-24T23:00:35.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.512+0000 7fac837fe640 1 -- 172.21.15.32:0/3109545703 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac78032810 con 0x7fac8415fd60 2024-04-24T23:00:35.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.512+0000 7fac8ad09640 1 -- 172.21.15.32:0/3109545703 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fac8415fd60 msgr2=0x7fac8406c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:35.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.512+0000 7fac8ad09640 1 --2- 172.21.15.32:0/3109545703 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fac8415fd60 0x7fac8406c5f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fac780097a0 tx=0x7fac7802a180 comp rx=0 tx=0).stop 2024-04-24T23:00:35.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.512+0000 7fac8ad09640 1 -- 172.21.15.32:0/3109545703 shutdown_connections 2024-04-24T23:00:35.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.512+0000 7fac8ad09640 1 --2- 172.21.15.32:0/3109545703 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fac8415fd60 0x7fac8406c5f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:35.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.512+0000 7fac8ad09640 1 --2- 172.21.15.32:0/3109545703 >> v2:172.21.15.121:3300/0 conn(0x7fac8415b480 0x7fac840b5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:35.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.512+0000 7fac8ad09640 1 -- 172.21.15.32:0/3109545703 >> 172.21.15.32:0/3109545703 conn(0x7fac8406b110 msgr2=0x7fac8406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:35.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.512+0000 7fac8ad09640 1 -- 172.21.15.32:0/3109545703 shutdown_connections 2024-04-24T23:00:35.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.512+0000 7fac8ad09640 1 -- 172.21.15.32:0/3109545703 wait complete. 2024-04-24T23:00:35.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.513+0000 7fac8ad09640 1 Processor -- start 2024-04-24T23:00:35.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.513+0000 7fac8ad09640 1 -- start start 2024-04-24T23:00:35.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.514+0000 7fac8ad09640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac8406cb30 0x7fac84141100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:35.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.514+0000 7fac8ad09640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac8415b480 0x7fac84142780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:35.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.514+0000 7fac8ad09640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fac8415fd60 0x7fac841445f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:35.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.514+0000 7fac8ad09640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fac840b5480 con 0x7fac8415b480 2024-04-24T23:00:35.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.514+0000 7fac8ad09640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fac840b61a0 con 0x7fac8406cb30 2024-04-24T23:00:35.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.514+0000 7fac8ad09640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fac8415b860 con 0x7fac8415fd60 2024-04-24T23:00:35.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.514+0000 7fac83fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac8415b480 0x7fac84142780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:35.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.514+0000 7fac83fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac8415b480 0x7fac84142780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46270/0 (socket says 172.21.15.32:46270) 2024-04-24T23:00:35.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.514+0000 7fac83fff640 1 -- 172.21.15.32:0/466050314 learned_addr learned my addr 172.21.15.32:0/466050314 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:35.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.515+0000 7fac892a6640 1 --2- 172.21.15.32:0/466050314 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fac8415fd60 0x7fac841445f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:35.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.515+0000 7fac88aa5640 1 --2- 172.21.15.32:0/466050314 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac8406cb30 0x7fac84141100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:35.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.515+0000 7fac83fff640 1 -- 172.21.15.32:0/466050314 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fac8415fd60 msgr2=0x7fac841445f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:35.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.515+0000 7fac83fff640 1 --2- 172.21.15.32:0/466050314 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fac8415fd60 0x7fac841445f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:35.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.515+0000 7fac83fff640 1 -- 172.21.15.32:0/466050314 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac8406cb30 msgr2=0x7fac84141100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:35.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.515+0000 7fac83fff640 1 --2- 172.21.15.32:0/466050314 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac8406cb30 0x7fac84141100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:35.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.515+0000 7fac83fff640 1 -- 172.21.15.32:0/466050314 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac841416e0 con 0x7fac8415b480 2024-04-24T23:00:35.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.515+0000 7fac892a6640 1 --2- 172.21.15.32:0/466050314 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fac8415fd60 0x7fac841445f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:00:35.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.515+0000 7fac88aa5640 1 --2- 172.21.15.32:0/466050314 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac8406cb30 0x7fac84141100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:00:35.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.515+0000 7fac83fff640 1 --2- 172.21.15.32:0/466050314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac8415b480 0x7fac84142780 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fac78035040 tx=0x7fac780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:35.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.515+0000 7fac817fa640 1 -- 172.21.15.32:0/466050314 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac78037040 con 0x7fac8415b480 2024-04-24T23:00:35.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.515+0000 7fac817fa640 1 -- 172.21.15.32:0/466050314 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac78030500 con 0x7fac8415b480 2024-04-24T23:00:35.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.515+0000 7fac8ad09640 1 -- 172.21.15.32:0/466050314 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac841419c0 con 0x7fac8415b480 2024-04-24T23:00:35.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.515+0000 7fac817fa640 1 -- 172.21.15.32:0/466050314 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac78030910 con 0x7fac8415b480 2024-04-24T23:00:35.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.515+0000 7fac8ad09640 1 -- 172.21.15.32:0/466050314 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac84141f80 con 0x7fac8415b480 2024-04-24T23:00:35.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.516+0000 7fac8ad09640 1 -- 172.21.15.32:0/466050314 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac84068430 con 0x7fac8415b480 2024-04-24T23:00:35.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.517+0000 7fac817fa640 1 -- 172.21.15.32:0/466050314 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fac78044020 con 0x7fac8415b480 2024-04-24T23:00:35.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.517+0000 7fac817fa640 1 --2- 172.21.15.32:0/466050314 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fac7006be30 0x7fac7006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:35.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.517+0000 7fac817fa640 1 -- 172.21.15.32:0/466050314 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fac780ab150 con 0x7fac8415b480 2024-04-24T23:00:35.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.517+0000 7fac88aa5640 1 --2- 172.21.15.32:0/466050314 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fac7006be30 0x7fac7006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:35.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.518+0000 7fac88aa5640 1 --2- 172.21.15.32:0/466050314 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fac7006be30 0x7fac7006e2f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fac7c007520 tx=0x7fac7c00b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:35.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.521+0000 7fac817fa640 1 -- 172.21.15.32:0/466050314 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fac78079b00 con 0x7fac8415b480 2024-04-24T23:00:35.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.706+0000 7fac8ad09640 1 -- 172.21.15.32:0/466050314 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fac8406c000 con 0x7fac8415b480 2024-04-24T23:00:35.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.707+0000 7fac817fa640 1 -- 172.21.15.32:0/466050314 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+16621 (secure 0 0 0) 0x7fac78079400 con 0x7fac8415b480 2024-04-24T23:00:35.709 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:00:35.709 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":45,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:00:34.008750+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:00:30.665592+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-24T23:00:31.357756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:00:35.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.709+0000 7fac8ad09640 1 -- 172.21.15.32:0/466050314 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fac7006be30 msgr2=0x7fac7006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:35.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.709+0000 7fac8ad09640 1 --2- 172.21.15.32:0/466050314 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fac7006be30 0x7fac7006e2f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fac7c007520 tx=0x7fac7c00b040 comp rx=0 tx=0).stop 2024-04-24T23:00:35.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.709+0000 7fac8ad09640 1 -- 172.21.15.32:0/466050314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac8415b480 msgr2=0x7fac84142780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:35.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.709+0000 7fac8ad09640 1 --2- 172.21.15.32:0/466050314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac8415b480 0x7fac84142780 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fac78035040 tx=0x7fac780046c0 comp rx=0 tx=0).stop 2024-04-24T23:00:35.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.709+0000 7fac8ad09640 1 -- 172.21.15.32:0/466050314 shutdown_connections 2024-04-24T23:00:35.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.709+0000 7fac8ad09640 1 --2- 172.21.15.32:0/466050314 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fac7006be30 0x7fac7006e2f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:35.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.709+0000 7fac8ad09640 1 --2- 172.21.15.32:0/466050314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac8415b480 0x7fac84142780 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:35.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.709+0000 7fac8ad09640 1 --2- 172.21.15.32:0/466050314 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fac8415fd60 0x7fac841445f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:35.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.709+0000 7fac8ad09640 1 --2- 172.21.15.32:0/466050314 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac8406cb30 0x7fac84141100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:35.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.710+0000 7fac8ad09640 1 -- 172.21.15.32:0/466050314 >> 172.21.15.32:0/466050314 conn(0x7fac8406b110 msgr2=0x7fac840d0890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:35.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.710+0000 7fac8ad09640 1 -- 172.21.15.32:0/466050314 shutdown_connections 2024-04-24T23:00:35.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.710+0000 7fac8ad09640 1 -- 172.21.15.32:0/466050314 wait complete. 2024-04-24T23:00:35.722 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-24T23:00:35.722 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:00:35.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.822+0000 7f9a14a37640 1 Processor -- start 2024-04-24T23:00:35.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.822+0000 7f9a14a37640 1 -- start start 2024-04-24T23:00:35.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.823+0000 7f9a14a37640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9a10156bb0 0x7f9a1015dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:35.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.823+0000 7f9a14a37640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9a10073390 0x7f9a1006d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:35.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.823+0000 7f9a14a37640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a1015ccf0 con 0x7f9a1006d400 2024-04-24T23:00:35.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.823+0000 7f9a14a37640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f9a1015cfb0 con 0x7f9a10073390 2024-04-24T23:00:35.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.823+0000 7f9a14a37640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f9a1015d290 con 0x7f9a10156bb0 2024-04-24T23:00:35.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.823+0000 7f9a0ed9d640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9a10073390 0x7f9a1006d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:35.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.823+0000 7f9a0e59c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9a10156bb0 0x7f9a1015dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:35.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.823+0000 7f9a0e59c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9a10156bb0 0x7f9a1015dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.32:50004/0 (socket says 172.21.15.32:50004) 2024-04-24T23:00:35.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.823+0000 7f9a0ed9d640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9a10073390 0x7f9a1006d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.32:42914/0 (socket says 172.21.15.32:42914) 2024-04-24T23:00:35.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.823+0000 7f9a0e59c640 1 -- 172.21.15.32:0/2708341007 learned_addr learned my addr 172.21.15.32:0/2708341007 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:35.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.823+0000 7f9a0dd9b640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f9a1006d400 0x7f9a10072c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:45038/0 (socket says 172.21.15.32:45038) 2024-04-24T23:00:35.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.823+0000 7f9a0e59c640 1 -- 172.21.15.32:0/2708341007 >> v2:172.21.15.121:3300/0 conn(0x7f9a10073390 msgr2=0x7f9a1006d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:35.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.823+0000 7f9a0e59c640 1 --2- 172.21.15.32:0/2708341007 >> v2:172.21.15.121:3300/0 conn(0x7f9a10073390 0x7f9a1006d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:35.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.823+0000 7f9a0e59c640 1 -- 172.21.15.32:0/2708341007 >> v1:172.21.15.121:6789/0 conn(0x7f9a1006d400 legacy=0x7f9a10072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:35.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.823+0000 7f9a0e59c640 1 -- 172.21.15.32:0/2708341007 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a1015d570 con 0x7f9a10156bb0 2024-04-24T23:00:35.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.824+0000 7f9a0e59c640 1 --2- 172.21.15.32:0/2708341007 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9a10156bb0 0x7f9a1015dbc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9a00009e50 tx=0x7f9a0002a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d10ac540f017dce server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:35.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.824+0000 7f9a0d59a640 1 -- 172.21.15.32:0/2708341007 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a00036070 con 0x7f9a10156bb0 2024-04-24T23:00:35.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.824+0000 7f9a0d59a640 1 -- 172.21.15.32:0/2708341007 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a00031600 con 0x7f9a10156bb0 2024-04-24T23:00:35.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.824+0000 7f9a0d59a640 1 -- 172.21.15.32:0/2708341007 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a000319d0 con 0x7f9a10156bb0 2024-04-24T23:00:35.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.825+0000 7f9a14a37640 1 -- 172.21.15.32:0/2708341007 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9a10156bb0 msgr2=0x7f9a1015dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:35.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.825+0000 7f9a14a37640 1 --2- 172.21.15.32:0/2708341007 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9a10156bb0 0x7f9a1015dbc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9a00009e50 tx=0x7f9a0002a710 comp rx=0 tx=0).stop 2024-04-24T23:00:35.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.825+0000 7f9a14a37640 1 -- 172.21.15.32:0/2708341007 shutdown_connections 2024-04-24T23:00:35.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.825+0000 7f9a14a37640 1 --2- 172.21.15.32:0/2708341007 >> v2:172.21.15.121:3300/0 conn(0x7f9a10073390 0x7f9a1006d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:35.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.825+0000 7f9a14a37640 1 --2- 172.21.15.32:0/2708341007 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9a10156bb0 0x7f9a1015dbc0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:35.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.825+0000 7f9a14a37640 1 -- 172.21.15.32:0/2708341007 >> 172.21.15.32:0/2708341007 conn(0x7f9a1006e800 msgr2=0x7f9a1006ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:35.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.825+0000 7f9a14a37640 1 -- 172.21.15.32:0/2708341007 shutdown_connections 2024-04-24T23:00:35.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.825+0000 7f9a14a37640 1 -- 172.21.15.32:0/2708341007 wait complete. 2024-04-24T23:00:35.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.826+0000 7f9a14a37640 1 Processor -- start 2024-04-24T23:00:35.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.826+0000 7f9a14a37640 1 -- start start 2024-04-24T23:00:35.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.827+0000 7f9a14a37640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a1006d400 0x7f9a100b0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:35.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.827+0000 7f9a14a37640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a10073390 0x7f9a101f5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:35.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.827+0000 7f9a14a37640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9a10156bb0 0x7f9a101f6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:35.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.827+0000 7f9a14a37640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9a10156f90 con 0x7f9a1006d400 2024-04-24T23:00:35.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.827+0000 7f9a14a37640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1015e100 con 0x7f9a10073390 2024-04-24T23:00:35.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.827+0000 7f9a14a37640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f9a10157110 con 0x7f9a10156bb0 2024-04-24T23:00:35.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.827+0000 7f9a0e59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a1006d400 0x7f9a100b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:35.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.827+0000 7f9a0e59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a1006d400 0x7f9a100b0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46274/0 (socket says 172.21.15.32:46274) 2024-04-24T23:00:35.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.827+0000 7f9a0e59c640 1 -- 172.21.15.32:0/3644376601 learned_addr learned my addr 172.21.15.32:0/3644376601 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:35.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.827+0000 7f9a0dd9b640 1 --2- 172.21.15.32:0/3644376601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a10073390 0x7f9a101f5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:35.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.827+0000 7f9a0ed9d640 1 --2- 172.21.15.32:0/3644376601 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9a10156bb0 0x7f9a101f6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:35.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.828+0000 7f9a0e59c640 1 -- 172.21.15.32:0/3644376601 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9a10156bb0 msgr2=0x7f9a101f6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:35.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.828+0000 7f9a0e59c640 1 --2- 172.21.15.32:0/3644376601 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9a10156bb0 0x7f9a101f6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:35.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.828+0000 7f9a0e59c640 1 -- 172.21.15.32:0/3644376601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a10073390 msgr2=0x7f9a101f5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:35.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.828+0000 7f9a0e59c640 1 --2- 172.21.15.32:0/3644376601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a10073390 0x7f9a101f5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:35.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.828+0000 7f9a0e59c640 1 -- 172.21.15.32:0/3644376601 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a10159980 con 0x7f9a1006d400 2024-04-24T23:00:35.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.828+0000 7f9a0dd9b640 1 --2- 172.21.15.32:0/3644376601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a10073390 0x7f9a101f5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:00:35.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.828+0000 7f9a0e59c640 1 --2- 172.21.15.32:0/3644376601 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a1006d400 0x7f9a100b0580 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f9a00034040 tx=0x7f9a00004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:35.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.828+0000 7f99feffd640 1 -- 172.21.15.32:0/3644376601 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a00036050 con 0x7f9a1006d400 2024-04-24T23:00:35.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.828+0000 7f99feffd640 1 -- 172.21.15.32:0/3644376601 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a00042070 con 0x7f9a1006d400 2024-04-24T23:00:35.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.828+0000 7f99feffd640 1 -- 172.21.15.32:0/3644376601 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a00009510 con 0x7f9a1006d400 2024-04-24T23:00:35.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.828+0000 7f9a14a37640 1 -- 172.21.15.32:0/3644376601 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a10159c60 con 0x7f9a1006d400 2024-04-24T23:00:35.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.828+0000 7f9a14a37640 1 -- 172.21.15.32:0/3644376601 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a1015a220 con 0x7f9a1006d400 2024-04-24T23:00:35.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.829+0000 7f9a14a37640 1 -- 172.21.15.32:0/3644376601 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a10068430 con 0x7f9a1006d400 2024-04-24T23:00:35.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.829+0000 7f99feffd640 1 -- 172.21.15.32:0/3644376601 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9a00007c70 con 0x7f9a1006d400 2024-04-24T23:00:35.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.830+0000 7f99feffd640 1 --2- 172.21.15.32:0/3644376601 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f99f006be30 0x7f99f006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:35.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.830+0000 7f99feffd640 1 -- 172.21.15.32:0/3644376601 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f9a000abb70 con 0x7f9a1006d400 2024-04-24T23:00:35.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.830+0000 7f9a0dd9b640 1 --2- 172.21.15.32:0/3644376601 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f99f006be30 0x7f99f006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:35.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.830+0000 7f9a0dd9b640 1 --2- 172.21.15.32:0/3644376601 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f99f006be30 0x7f99f006e2f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9a04007fd0 tx=0x7f9a04007d60 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:35.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:35.834+0000 7f99feffd640 1 -- 172.21.15.32:0/3644376601 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9a0007a520 con 0x7f9a1006d400 2024-04-24T23:00:35.956 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:00:35.959 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:00:36.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.014+0000 7f9a14a37640 1 -- 172.21.15.32:0/3644376601 --> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9a100727a0 con 0x7f99f006be30 2024-04-24T23:00:36.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.021+0000 7f99feffd640 1 -- 172.21.15.32:0/3644376601 <== mgr.4108 v2:172.21.15.121:6832/1848157943 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+242215 (secure 0 0 0) 0x7f9a100727a0 con 0x7f99f006be30 2024-04-24T23:00:36.024 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:00:36.026 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-24T23:00:36.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.023+0000 7f9a14a37640 1 -- 172.21.15.32:0/3644376601 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f99f006be30 msgr2=0x7f99f006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:36.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.023+0000 7f9a14a37640 1 --2- 172.21.15.32:0/3644376601 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f99f006be30 0x7f99f006e2f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9a04007fd0 tx=0x7f9a04007d60 comp rx=0 tx=0).stop 2024-04-24T23:00:36.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.023+0000 7f9a14a37640 1 -- 172.21.15.32:0/3644376601 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a1006d400 msgr2=0x7f9a100b0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:36.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.023+0000 7f9a14a37640 1 --2- 172.21.15.32:0/3644376601 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a1006d400 0x7f9a100b0580 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f9a00034040 tx=0x7f9a00004a40 comp rx=0 tx=0).stop 2024-04-24T23:00:36.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.023+0000 7f9a14a37640 1 -- 172.21.15.32:0/3644376601 shutdown_connections 2024-04-24T23:00:36.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.023+0000 7f9a14a37640 1 --2- 172.21.15.32:0/3644376601 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f99f006be30 0x7f99f006e2f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:36.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.023+0000 7f9a14a37640 1 --2- 172.21.15.32:0/3644376601 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9a10156bb0 0x7f9a101f6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:36.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.023+0000 7f9a14a37640 1 --2- 172.21.15.32:0/3644376601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a10073390 0x7f9a101f5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:36.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.023+0000 7f9a14a37640 1 --2- 172.21.15.32:0/3644376601 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a1006d400 0x7f9a100b0580 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:36.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.023+0000 7f9a14a37640 1 -- 172.21.15.32:0/3644376601 >> 172.21.15.32:0/3644376601 conn(0x7f9a1006e800 msgr2=0x7f9a100d25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:36.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.023+0000 7f9a14a37640 1 -- 172.21.15.32:0/3644376601 shutdown_connections 2024-04-24T23:00:36.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.023+0000 7f9a14a37640 1 -- 172.21.15.32:0/3644376601 wait complete. 2024-04-24T23:00:36.036 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":88,"stamp":"2024-04-24T23:00:35.147698+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":4,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2683,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1559816,"statfs":{"total":1600000000,"available":1597252210,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.143686"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.015384+0000","last_change":"2024-04-24T23:00:32.015384+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.015384+0000","last_undegraded":"2024-04-24T23:00:32.015384+0000","last_fullsized":"2024-04-24T23:00:32.015384+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.012040+0000","last_change":"2024-04-24T23:00:32.012040+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.012040+0000","last_undegraded":"2024-04-24T23:00:32.012040+0000","last_fullsized":"2024-04-24T23:00:32.012040+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.013243+0000","last_change":"2024-04-24T23:00:32.013243+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.013243+0000","last_undegraded":"2024-04-24T23:00:32.013243+0000","last_fullsized":"2024-04-24T23:00:32.013243+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.021880+0000","last_change":"2024-04-24T23:00:32.021880+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.021880+0000","last_undegraded":"2024-04-24T23:00:32.021880+0000","last_fullsized":"2024-04-24T23:00:32.021880+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.013633+0000","last_change":"2024-04-24T23:00:32.013633+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.013633+0000","last_undegraded":"2024-04-24T23:00:32.013633+0000","last_fullsized":"2024-04-24T23:00:32.013633+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.010238+0000","last_change":"2024-04-24T23:00:32.010238+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.010238+0000","last_undegraded":"2024-04-24T23:00:32.010238+0000","last_fullsized":"2024-04-24T23:00:32.010238+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.019822+0000","last_change":"2024-04-24T23:00:32.019822+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.019822+0000","last_undegraded":"2024-04-24T23:00:32.019822+0000","last_fullsized":"2024-04-24T23:00:32.019822+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.022253+0000","last_change":"2024-04-24T23:00:32.022253+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.022253+0000","last_undegraded":"2024-04-24T23:00:32.022253+0000","last_fullsized":"2024-04-24T23:00:32.022253+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.009087+0000","last_change":"2024-04-24T23:00:32.009087+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.009087+0000","last_undegraded":"2024-04-24T23:00:32.009087+0000","last_fullsized":"2024-04-24T23:00:32.009087+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.014404+0000","last_change":"2024-04-24T23:00:32.014404+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.014404+0000","last_undegraded":"2024-04-24T23:00:32.014404+0000","last_fullsized":"2024-04-24T23:00:32.014404+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.022474+0000","last_change":"2024-04-24T23:00:32.022474+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.022474+0000","last_undegraded":"2024-04-24T23:00:32.022474+0000","last_fullsized":"2024-04-24T23:00:32.022474+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.011480+0000","last_change":"2024-04-24T23:00:32.011480+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.011480+0000","last_undegraded":"2024-04-24T23:00:32.011480+0000","last_fullsized":"2024-04-24T23:00:32.011480+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.010000+0000","last_change":"2024-04-24T23:00:32.010000+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.010000+0000","last_undegraded":"2024-04-24T23:00:32.010000+0000","last_fullsized":"2024-04-24T23:00:32.010000+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.021494+0000","last_change":"2024-04-24T23:00:32.021494+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.021494+0000","last_undegraded":"2024-04-24T23:00:32.021494+0000","last_fullsized":"2024-04-24T23:00:32.021494+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.021499+0000","last_change":"2024-04-24T23:00:32.021499+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.021499+0000","last_undegraded":"2024-04-24T23:00:32.021499+0000","last_fullsized":"2024-04-24T23:00:32.021499+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.015039+0000","last_change":"2024-04-24T23:00:32.015039+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.015039+0000","last_undegraded":"2024-04-24T23:00:32.015039+0000","last_fullsized":"2024-04-24T23:00:32.015039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.011162+0000","last_change":"2024-04-24T23:00:32.011162+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.011162+0000","last_undegraded":"2024-04-24T23:00:32.011162+0000","last_fullsized":"2024-04-24T23:00:32.011162+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.012933+0000","last_change":"2024-04-24T23:00:32.012933+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.012933+0000","last_undegraded":"2024-04-24T23:00:32.012933+0000","last_fullsized":"2024-04-24T23:00:32.012933+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.011970+0000","last_change":"2024-04-24T23:00:32.011970+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.011970+0000","last_undegraded":"2024-04-24T23:00:32.011970+0000","last_fullsized":"2024-04-24T23:00:32.011970+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.011779+0000","last_change":"2024-04-24T23:00:32.011779+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.011779+0000","last_undegraded":"2024-04-24T23:00:32.011779+0000","last_fullsized":"2024-04-24T23:00:32.011779+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.021919+0000","last_change":"2024-04-24T23:00:32.021919+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.021919+0000","last_undegraded":"2024-04-24T23:00:32.021919+0000","last_fullsized":"2024-04-24T23:00:32.021919+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.013643+0000","last_change":"2024-04-24T23:00:32.013643+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.013643+0000","last_undegraded":"2024-04-24T23:00:32.013643+0000","last_fullsized":"2024-04-24T23:00:32.013643+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.020052+0000","last_change":"2024-04-24T23:00:32.020052+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.020052+0000","last_undegraded":"2024-04-24T23:00:32.020052+0000","last_fullsized":"2024-04-24T23:00:32.020052+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.010783+0000","last_change":"2024-04-24T23:00:32.010783+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.010783+0000","last_undegraded":"2024-04-24T23:00:32.010783+0000","last_fullsized":"2024-04-24T23:00:32.010783+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.013276+0000","last_change":"2024-04-24T23:00:32.013276+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.013276+0000","last_undegraded":"2024-04-24T23:00:32.013276+0000","last_fullsized":"2024-04-24T23:00:32.013276+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.020899+0000","last_change":"2024-04-24T23:00:32.020899+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.020899+0000","last_undegraded":"2024-04-24T23:00:32.020899+0000","last_fullsized":"2024-04-24T23:00:32.020899+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.019540+0000","last_change":"2024-04-24T23:00:32.019540+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.019540+0000","last_undegraded":"2024-04-24T23:00:32.019540+0000","last_fullsized":"2024-04-24T23:00:32.019540+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.019756+0000","last_change":"2024-04-24T23:00:32.019756+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.019756+0000","last_undegraded":"2024-04-24T23:00:32.019756+0000","last_fullsized":"2024-04-24T23:00:32.019756+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.012654+0000","last_change":"2024-04-24T23:00:32.012654+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.012654+0000","last_undegraded":"2024-04-24T23:00:32.012654+0000","last_fullsized":"2024-04-24T23:00:32.012654+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.009579+0000","last_change":"2024-04-24T23:00:32.009579+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.009579+0000","last_undegraded":"2024-04-24T23:00:32.009579+0000","last_fullsized":"2024-04-24T23:00:32.009579+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.009836+0000","last_change":"2024-04-24T23:00:32.009836+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.009836+0000","last_undegraded":"2024-04-24T23:00:32.009836+0000","last_fullsized":"2024-04-24T23:00:32.009836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.020843+0000","last_change":"2024-04-24T23:00:32.020843+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.020843+0000","last_undegraded":"2024-04-24T23:00:32.020843+0000","last_fullsized":"2024-04-24T23:00:32.020843+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.015012+0000","last_change":"2024-04-24T23:00:32.015012+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.015012+0000","last_undegraded":"2024-04-24T23:00:32.015012+0000","last_fullsized":"2024-04-24T23:00:32.015012+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.014172+0000","last_change":"2024-04-24T23:00:32.013964+0000","last_active":"2024-04-24T23:00:32.014172+0000","last_peered":"2024-04-24T23:00:32.014172+0000","last_clean":"2024-04-24T23:00:32.014172+0000","last_became_active":"2024-04-24T23:00:32.013661+0000","last_became_peered":"2024-04-24T23:00:32.013661+0000","last_unstale":"2024-04-24T23:00:32.014172+0000","last_undegraded":"2024-04-24T23:00:32.014172+0000","last_fullsized":"2024-04-24T23:00:32.014172+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:34.956602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.012160+0000","last_change":"2024-04-24T23:00:32.012160+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.012160+0000","last_undegraded":"2024-04-24T23:00:32.012160+0000","last_fullsized":"2024-04-24T23:00:32.012160+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.012887+0000","last_change":"2024-04-24T23:00:32.012770+0000","last_active":"2024-04-24T23:00:32.012887+0000","last_peered":"2024-04-24T23:00:32.012887+0000","last_clean":"2024-04-24T23:00:32.012887+0000","last_became_active":"2024-04-24T23:00:32.012092+0000","last_became_peered":"2024-04-24T23:00:32.012092+0000","last_unstale":"2024-04-24T23:00:32.012887+0000","last_undegraded":"2024-04-24T23:00:32.012887+0000","last_fullsized":"2024-04-24T23:00:32.012887+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:03.280807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.009497+0000","last_change":"2024-04-24T23:00:32.009497+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.009497+0000","last_undegraded":"2024-04-24T23:00:32.009497+0000","last_fullsized":"2024-04-24T23:00:32.009497+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.016002+0000","last_change":"2024-04-24T23:00:32.015863+0000","last_active":"2024-04-24T23:00:32.016002+0000","last_peered":"2024-04-24T23:00:32.016002+0000","last_clean":"2024-04-24T23:00:32.016002+0000","last_became_active":"2024-04-24T23:00:32.015616+0000","last_became_peered":"2024-04-24T23:00:32.015616+0000","last_unstale":"2024-04-24T23:00:32.016002+0000","last_undegraded":"2024-04-24T23:00:32.016002+0000","last_fullsized":"2024-04-24T23:00:32.016002+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:29:18.370977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.020793+0000","last_change":"2024-04-24T23:00:32.020793+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.020793+0000","last_undegraded":"2024-04-24T23:00:32.020793+0000","last_fullsized":"2024-04-24T23:00:32.020793+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.010577+0000","last_change":"2024-04-24T23:00:32.010390+0000","last_active":"2024-04-24T23:00:32.010577+0000","last_peered":"2024-04-24T23:00:32.010577+0000","last_clean":"2024-04-24T23:00:32.010577+0000","last_became_active":"2024-04-24T23:00:32.010156+0000","last_became_peered":"2024-04-24T23:00:32.010156+0000","last_unstale":"2024-04-24T23:00:32.010577+0000","last_undegraded":"2024-04-24T23:00:32.010577+0000","last_fullsized":"2024-04-24T23:00:32.010577+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:38:29.668465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.019446+0000","last_change":"2024-04-24T23:00:32.019446+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.019446+0000","last_undegraded":"2024-04-24T23:00:32.019446+0000","last_fullsized":"2024-04-24T23:00:32.019446+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.012416+0000","last_change":"2024-04-24T23:00:32.012144+0000","last_active":"2024-04-24T23:00:32.012416+0000","last_peered":"2024-04-24T23:00:32.012416+0000","last_clean":"2024-04-24T23:00:32.012416+0000","last_became_active":"2024-04-24T23:00:32.011876+0000","last_became_peered":"2024-04-24T23:00:32.011876+0000","last_unstale":"2024-04-24T23:00:32.012416+0000","last_undegraded":"2024-04-24T23:00:32.012416+0000","last_fullsized":"2024-04-24T23:00:32.012416+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:25:17.401466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.012251+0000","last_change":"2024-04-24T23:00:32.012251+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.012251+0000","last_undegraded":"2024-04-24T23:00:32.012251+0000","last_fullsized":"2024-04-24T23:00:32.012251+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.015221+0000","last_change":"2024-04-24T23:00:32.015060+0000","last_active":"2024-04-24T23:00:32.015221+0000","last_peered":"2024-04-24T23:00:32.015221+0000","last_clean":"2024-04-24T23:00:32.015221+0000","last_became_active":"2024-04-24T23:00:32.013636+0000","last_became_peered":"2024-04-24T23:00:32.013636+0000","last_unstale":"2024-04-24T23:00:32.015221+0000","last_undegraded":"2024-04-24T23:00:32.015221+0000","last_fullsized":"2024-04-24T23:00:32.015221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:57:18.083931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.014094+0000","last_change":"2024-04-24T23:00:32.014094+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.014094+0000","last_undegraded":"2024-04-24T23:00:32.014094+0000","last_fullsized":"2024-04-24T23:00:32.014094+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.019078+0000","last_change":"2024-04-24T23:00:32.018961+0000","last_active":"2024-04-24T23:00:32.019078+0000","last_peered":"2024-04-24T23:00:32.019078+0000","last_clean":"2024-04-24T23:00:32.019078+0000","last_became_active":"2024-04-24T23:00:32.018718+0000","last_became_peered":"2024-04-24T23:00:32.018718+0000","last_unstale":"2024-04-24T23:00:32.019078+0000","last_undegraded":"2024-04-24T23:00:32.019078+0000","last_fullsized":"2024-04-24T23:00:32.019078+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:42:02.507492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.021039+0000","last_change":"2024-04-24T23:00:32.021039+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.021039+0000","last_undegraded":"2024-04-24T23:00:32.021039+0000","last_fullsized":"2024-04-24T23:00:32.021039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.016759+0000","last_change":"2024-04-24T23:00:32.016634+0000","last_active":"2024-04-24T23:00:32.016759+0000","last_peered":"2024-04-24T23:00:32.016759+0000","last_clean":"2024-04-24T23:00:32.016759+0000","last_became_active":"2024-04-24T23:00:32.015425+0000","last_became_peered":"2024-04-24T23:00:32.015425+0000","last_unstale":"2024-04-24T23:00:32.016759+0000","last_undegraded":"2024-04-24T23:00:32.016759+0000","last_fullsized":"2024-04-24T23:00:32.016759+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:32:23.611062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.012408+0000","last_change":"2024-04-24T23:00:32.012408+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.012408+0000","last_undegraded":"2024-04-24T23:00:32.012408+0000","last_fullsized":"2024-04-24T23:00:32.012408+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.017952+0000","last_change":"2024-04-24T23:00:32.017822+0000","last_active":"2024-04-24T23:00:32.017952+0000","last_peered":"2024-04-24T23:00:32.017952+0000","last_clean":"2024-04-24T23:00:32.017952+0000","last_became_active":"2024-04-24T23:00:32.017548+0000","last_became_peered":"2024-04-24T23:00:32.017548+0000","last_unstale":"2024-04-24T23:00:32.017952+0000","last_undegraded":"2024-04-24T23:00:32.017952+0000","last_fullsized":"2024-04-24T23:00:32.017952+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:18:08.921860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.015793+0000","last_change":"2024-04-24T23:00:32.015793+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.015793+0000","last_undegraded":"2024-04-24T23:00:32.015793+0000","last_fullsized":"2024-04-24T23:00:32.015793+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.016524+0000","last_change":"2024-04-24T23:00:32.016337+0000","last_active":"2024-04-24T23:00:32.016524+0000","last_peered":"2024-04-24T23:00:32.016524+0000","last_clean":"2024-04-24T23:00:32.016524+0000","last_became_active":"2024-04-24T23:00:32.015833+0000","last_became_peered":"2024-04-24T23:00:32.015833+0000","last_unstale":"2024-04-24T23:00:32.016524+0000","last_undegraded":"2024-04-24T23:00:32.016524+0000","last_fullsized":"2024-04-24T23:00:32.016524+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:00:47.017154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.020618+0000","last_change":"2024-04-24T23:00:32.020618+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.020618+0000","last_undegraded":"2024-04-24T23:00:32.020618+0000","last_fullsized":"2024-04-24T23:00:32.020618+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.017321+0000","last_change":"2024-04-24T23:00:32.017203+0000","last_active":"2024-04-24T23:00:32.017321+0000","last_peered":"2024-04-24T23:00:32.017321+0000","last_clean":"2024-04-24T23:00:32.017321+0000","last_became_active":"2024-04-24T23:00:32.016743+0000","last_became_peered":"2024-04-24T23:00:32.016743+0000","last_unstale":"2024-04-24T23:00:32.017321+0000","last_undegraded":"2024-04-24T23:00:32.017321+0000","last_fullsized":"2024-04-24T23:00:32.017321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:30:49.169292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.019128+0000","last_change":"2024-04-24T23:00:32.019128+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.019128+0000","last_undegraded":"2024-04-24T23:00:32.019128+0000","last_fullsized":"2024-04-24T23:00:32.019128+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.014710+0000","last_change":"2024-04-24T23:00:32.014044+0000","last_active":"2024-04-24T23:00:32.014710+0000","last_peered":"2024-04-24T23:00:32.014710+0000","last_clean":"2024-04-24T23:00:32.014710+0000","last_became_active":"2024-04-24T23:00:32.013781+0000","last_became_peered":"2024-04-24T23:00:32.013781+0000","last_unstale":"2024-04-24T23:00:32.014710+0000","last_undegraded":"2024-04-24T23:00:32.014710+0000","last_fullsized":"2024-04-24T23:00:32.014710+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:24:27.447216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.015479+0000","last_change":"2024-04-24T23:00:32.015479+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.015479+0000","last_undegraded":"2024-04-24T23:00:32.015479+0000","last_fullsized":"2024-04-24T23:00:32.015479+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.019354+0000","last_change":"2024-04-24T23:00:32.019200+0000","last_active":"2024-04-24T23:00:32.019354+0000","last_peered":"2024-04-24T23:00:32.019354+0000","last_clean":"2024-04-24T23:00:32.019354+0000","last_became_active":"2024-04-24T23:00:32.018935+0000","last_became_peered":"2024-04-24T23:00:32.018935+0000","last_unstale":"2024-04-24T23:00:32.019354+0000","last_undegraded":"2024-04-24T23:00:32.019354+0000","last_fullsized":"2024-04-24T23:00:32.019354+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:13.644308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.014218+0000","last_change":"2024-04-24T23:00:32.014218+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.014218+0000","last_undegraded":"2024-04-24T23:00:32.014218+0000","last_fullsized":"2024-04-24T23:00:32.014218+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.018213+0000","last_change":"2024-04-24T23:00:32.018091+0000","last_active":"2024-04-24T23:00:32.018213+0000","last_peered":"2024-04-24T23:00:32.018213+0000","last_clean":"2024-04-24T23:00:32.018213+0000","last_became_active":"2024-04-24T23:00:32.017828+0000","last_became_peered":"2024-04-24T23:00:32.017828+0000","last_unstale":"2024-04-24T23:00:32.018213+0000","last_undegraded":"2024-04-24T23:00:32.018213+0000","last_fullsized":"2024-04-24T23:00:32.018213+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:29:21.830443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.014085+0000","last_change":"2024-04-24T23:00:32.014085+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.014085+0000","last_undegraded":"2024-04-24T23:00:32.014085+0000","last_fullsized":"2024-04-24T23:00:32.014085+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.015462+0000","last_change":"2024-04-24T23:00:32.015309+0000","last_active":"2024-04-24T23:00:32.015462+0000","last_peered":"2024-04-24T23:00:32.015462+0000","last_clean":"2024-04-24T23:00:32.015462+0000","last_became_active":"2024-04-24T23:00:32.014784+0000","last_became_peered":"2024-04-24T23:00:32.014784+0000","last_unstale":"2024-04-24T23:00:32.015462+0000","last_undegraded":"2024-04-24T23:00:32.015462+0000","last_fullsized":"2024-04-24T23:00:32.015462+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:44:08.784625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.010204+0000","last_change":"2024-04-24T23:00:32.010204+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.010204+0000","last_undegraded":"2024-04-24T23:00:32.010204+0000","last_fullsized":"2024-04-24T23:00:32.010204+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.018350+0000","last_change":"2024-04-24T23:00:32.018235+0000","last_active":"2024-04-24T23:00:32.018350+0000","last_peered":"2024-04-24T23:00:32.018350+0000","last_clean":"2024-04-24T23:00:32.018350+0000","last_became_active":"2024-04-24T23:00:32.017933+0000","last_became_peered":"2024-04-24T23:00:32.017933+0000","last_unstale":"2024-04-24T23:00:32.018350+0000","last_undegraded":"2024-04-24T23:00:32.018350+0000","last_fullsized":"2024-04-24T23:00:32.018350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:30:30.768770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.013648+0000","last_change":"2024-04-24T23:00:32.013648+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.013648+0000","last_undegraded":"2024-04-24T23:00:32.013648+0000","last_fullsized":"2024-04-24T23:00:32.013648+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.015776+0000","last_change":"2024-04-24T23:00:32.015665+0000","last_active":"2024-04-24T23:00:32.015776+0000","last_peered":"2024-04-24T23:00:32.015776+0000","last_clean":"2024-04-24T23:00:32.015776+0000","last_became_active":"2024-04-24T23:00:32.015414+0000","last_became_peered":"2024-04-24T23:00:32.015414+0000","last_unstale":"2024-04-24T23:00:32.015776+0000","last_undegraded":"2024-04-24T23:00:32.015776+0000","last_fullsized":"2024-04-24T23:00:32.015776+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:34:48.498696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.014550+0000","last_change":"2024-04-24T23:00:32.014550+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.014550+0000","last_undegraded":"2024-04-24T23:00:32.014550+0000","last_fullsized":"2024-04-24T23:00:32.014550+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.016899+0000","last_change":"2024-04-24T23:00:32.016781+0000","last_active":"2024-04-24T23:00:32.016899+0000","last_peered":"2024-04-24T23:00:32.016899+0000","last_clean":"2024-04-24T23:00:32.016899+0000","last_became_active":"2024-04-24T23:00:32.016515+0000","last_became_peered":"2024-04-24T23:00:32.016515+0000","last_unstale":"2024-04-24T23:00:32.016899+0000","last_undegraded":"2024-04-24T23:00:32.016899+0000","last_fullsized":"2024-04-24T23:00:32.016899+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:57:27.039125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.009520+0000","last_change":"2024-04-24T23:00:32.009520+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.009520+0000","last_undegraded":"2024-04-24T23:00:32.009520+0000","last_fullsized":"2024-04-24T23:00:32.009520+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.019478+0000","last_change":"2024-04-24T23:00:32.019334+0000","last_active":"2024-04-24T23:00:32.019478+0000","last_peered":"2024-04-24T23:00:32.019478+0000","last_clean":"2024-04-24T23:00:32.019478+0000","last_became_active":"2024-04-24T23:00:32.019103+0000","last_became_peered":"2024-04-24T23:00:32.019103+0000","last_unstale":"2024-04-24T23:00:32.019478+0000","last_undegraded":"2024-04-24T23:00:32.019478+0000","last_fullsized":"2024-04-24T23:00:32.019478+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:08:14.771602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.015184+0000","last_change":"2024-04-24T23:00:32.015184+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.015184+0000","last_undegraded":"2024-04-24T23:00:32.015184+0000","last_fullsized":"2024-04-24T23:00:32.015184+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.015147+0000","last_change":"2024-04-24T23:00:32.015023+0000","last_active":"2024-04-24T23:00:32.015147+0000","last_peered":"2024-04-24T23:00:32.015147+0000","last_clean":"2024-04-24T23:00:32.015147+0000","last_became_active":"2024-04-24T23:00:32.014455+0000","last_became_peered":"2024-04-24T23:00:32.014455+0000","last_unstale":"2024-04-24T23:00:32.015147+0000","last_undegraded":"2024-04-24T23:00:32.015147+0000","last_fullsized":"2024-04-24T23:00:32.015147+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:16.368500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.011378+0000","last_change":"2024-04-24T23:00:32.011378+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.011378+0000","last_undegraded":"2024-04-24T23:00:32.011378+0000","last_fullsized":"2024-04-24T23:00:32.011378+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.019875+0000","last_change":"2024-04-24T23:00:32.019708+0000","last_active":"2024-04-24T23:00:32.019875+0000","last_peered":"2024-04-24T23:00:32.019875+0000","last_clean":"2024-04-24T23:00:32.019875+0000","last_became_active":"2024-04-24T23:00:32.019473+0000","last_became_peered":"2024-04-24T23:00:32.019473+0000","last_unstale":"2024-04-24T23:00:32.019875+0000","last_undegraded":"2024-04-24T23:00:32.019875+0000","last_fullsized":"2024-04-24T23:00:32.019875+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:57:15.174813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.009609+0000","last_change":"2024-04-24T23:00:32.009609+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.009609+0000","last_undegraded":"2024-04-24T23:00:32.009609+0000","last_fullsized":"2024-04-24T23:00:32.009609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.016029+0000","last_change":"2024-04-24T23:00:32.015397+0000","last_active":"2024-04-24T23:00:32.016029+0000","last_peered":"2024-04-24T23:00:32.016029+0000","last_clean":"2024-04-24T23:00:32.016029+0000","last_became_active":"2024-04-24T23:00:32.015154+0000","last_became_peered":"2024-04-24T23:00:32.015154+0000","last_unstale":"2024-04-24T23:00:32.016029+0000","last_undegraded":"2024-04-24T23:00:32.016029+0000","last_fullsized":"2024-04-24T23:00:32.016029+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:04:05.572144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.020075+0000","last_change":"2024-04-24T23:00:32.020075+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.020075+0000","last_undegraded":"2024-04-24T23:00:32.020075+0000","last_fullsized":"2024-04-24T23:00:32.020075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.018003+0000","last_change":"2024-04-24T23:00:32.017383+0000","last_active":"2024-04-24T23:00:32.018003+0000","last_peered":"2024-04-24T23:00:32.018003+0000","last_clean":"2024-04-24T23:00:32.018003+0000","last_became_active":"2024-04-24T23:00:32.016962+0000","last_became_peered":"2024-04-24T23:00:32.016962+0000","last_unstale":"2024-04-24T23:00:32.018003+0000","last_undegraded":"2024-04-24T23:00:32.018003+0000","last_fullsized":"2024-04-24T23:00:32.018003+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:44:38.000055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.020501+0000","last_change":"2024-04-24T23:00:32.020501+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.020501+0000","last_undegraded":"2024-04-24T23:00:32.020501+0000","last_fullsized":"2024-04-24T23:00:32.020501+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.012229+0000","last_change":"2024-04-24T23:00:32.011951+0000","last_active":"2024-04-24T23:00:32.012229+0000","last_peered":"2024-04-24T23:00:32.012229+0000","last_clean":"2024-04-24T23:00:32.012229+0000","last_became_active":"2024-04-24T23:00:32.011483+0000","last_became_peered":"2024-04-24T23:00:32.011483+0000","last_unstale":"2024-04-24T23:00:32.012229+0000","last_undegraded":"2024-04-24T23:00:32.012229+0000","last_fullsized":"2024-04-24T23:00:32.012229+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:05.541366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.019204+0000","last_change":"2024-04-24T23:00:32.019204+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.019204+0000","last_undegraded":"2024-04-24T23:00:32.019204+0000","last_fullsized":"2024-04-24T23:00:32.019204+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.017120+0000","last_change":"2024-04-24T23:00:32.016280+0000","last_active":"2024-04-24T23:00:32.017120+0000","last_peered":"2024-04-24T23:00:32.017120+0000","last_clean":"2024-04-24T23:00:32.017120+0000","last_became_active":"2024-04-24T23:00:32.016057+0000","last_became_peered":"2024-04-24T23:00:32.016057+0000","last_unstale":"2024-04-24T23:00:32.017120+0000","last_undegraded":"2024-04-24T23:00:32.017120+0000","last_fullsized":"2024-04-24T23:00:32.017120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:25:31.020426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.013723+0000","last_change":"2024-04-24T23:00:32.013723+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.013723+0000","last_undegraded":"2024-04-24T23:00:32.013723+0000","last_fullsized":"2024-04-24T23:00:32.013723+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.016054+0000","last_change":"2024-04-24T23:00:32.015930+0000","last_active":"2024-04-24T23:00:32.016054+0000","last_peered":"2024-04-24T23:00:32.016054+0000","last_clean":"2024-04-24T23:00:32.016054+0000","last_became_active":"2024-04-24T23:00:32.015671+0000","last_became_peered":"2024-04-24T23:00:32.015671+0000","last_unstale":"2024-04-24T23:00:32.016054+0000","last_undegraded":"2024-04-24T23:00:32.016054+0000","last_fullsized":"2024-04-24T23:00:32.016054+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:47.216626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.020647+0000","last_change":"2024-04-24T23:00:32.020647+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.020647+0000","last_undegraded":"2024-04-24T23:00:32.020647+0000","last_fullsized":"2024-04-24T23:00:32.020647+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.020210+0000","last_change":"2024-04-24T23:00:32.020060+0000","last_active":"2024-04-24T23:00:32.020210+0000","last_peered":"2024-04-24T23:00:32.020210+0000","last_clean":"2024-04-24T23:00:32.020210+0000","last_became_active":"2024-04-24T23:00:32.019783+0000","last_became_peered":"2024-04-24T23:00:32.019783+0000","last_unstale":"2024-04-24T23:00:32.020210+0000","last_undegraded":"2024-04-24T23:00:32.020210+0000","last_fullsized":"2024-04-24T23:00:32.020210+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:24:44.692764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.011948+0000","last_change":"2024-04-24T23:00:32.011948+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.011948+0000","last_undegraded":"2024-04-24T23:00:32.011948+0000","last_fullsized":"2024-04-24T23:00:32.011948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.020335+0000","last_change":"2024-04-24T23:00:32.020217+0000","last_active":"2024-04-24T23:00:32.020335+0000","last_peered":"2024-04-24T23:00:32.020335+0000","last_clean":"2024-04-24T23:00:32.020335+0000","last_became_active":"2024-04-24T23:00:32.019728+0000","last_became_peered":"2024-04-24T23:00:32.019728+0000","last_unstale":"2024-04-24T23:00:32.020335+0000","last_undegraded":"2024-04-24T23:00:32.020335+0000","last_fullsized":"2024-04-24T23:00:32.020335+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:33:53.601267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.018940+0000","last_change":"2024-04-24T23:00:32.018940+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.018940+0000","last_undegraded":"2024-04-24T23:00:32.018940+0000","last_fullsized":"2024-04-24T23:00:32.018940+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.018841+0000","last_change":"2024-04-24T23:00:32.018725+0000","last_active":"2024-04-24T23:00:32.018841+0000","last_peered":"2024-04-24T23:00:32.018841+0000","last_clean":"2024-04-24T23:00:32.018841+0000","last_became_active":"2024-04-24T23:00:32.018472+0000","last_became_peered":"2024-04-24T23:00:32.018472+0000","last_unstale":"2024-04-24T23:00:32.018841+0000","last_undegraded":"2024-04-24T23:00:32.018841+0000","last_fullsized":"2024-04-24T23:00:32.018841+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:58:07.274166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.011506+0000","last_change":"2024-04-24T23:00:32.011506+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.011506+0000","last_undegraded":"2024-04-24T23:00:32.011506+0000","last_fullsized":"2024-04-24T23:00:32.011506+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.012098+0000","last_change":"2024-04-24T23:00:32.011954+0000","last_active":"2024-04-24T23:00:32.012098+0000","last_peered":"2024-04-24T23:00:32.012098+0000","last_clean":"2024-04-24T23:00:32.012098+0000","last_became_active":"2024-04-24T23:00:32.011189+0000","last_became_peered":"2024-04-24T23:00:32.011189+0000","last_unstale":"2024-04-24T23:00:32.012098+0000","last_undegraded":"2024-04-24T23:00:32.012098+0000","last_fullsized":"2024-04-24T23:00:32.012098+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:10:23.000844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.009799+0000","last_change":"2024-04-24T23:00:32.009799+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.009799+0000","last_undegraded":"2024-04-24T23:00:32.009799+0000","last_fullsized":"2024-04-24T23:00:32.009799+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.020146+0000","last_change":"2024-04-24T23:00:32.019965+0000","last_active":"2024-04-24T23:00:32.020146+0000","last_peered":"2024-04-24T23:00:32.020146+0000","last_clean":"2024-04-24T23:00:32.020146+0000","last_became_active":"2024-04-24T23:00:32.019691+0000","last_became_peered":"2024-04-24T23:00:32.019691+0000","last_unstale":"2024-04-24T23:00:32.020146+0000","last_undegraded":"2024-04-24T23:00:32.020146+0000","last_fullsized":"2024-04-24T23:00:32.020146+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:32:54.766031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.019001+0000","last_change":"2024-04-24T23:00:32.019001+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.019001+0000","last_undegraded":"2024-04-24T23:00:32.019001+0000","last_fullsized":"2024-04-24T23:00:32.019001+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.016843+0000","last_change":"2024-04-24T23:00:32.016499+0000","last_active":"2024-04-24T23:00:32.016843+0000","last_peered":"2024-04-24T23:00:32.016843+0000","last_clean":"2024-04-24T23:00:32.016843+0000","last_became_active":"2024-04-24T23:00:32.016247+0000","last_became_peered":"2024-04-24T23:00:32.016247+0000","last_unstale":"2024-04-24T23:00:32.016843+0000","last_undegraded":"2024-04-24T23:00:32.016843+0000","last_fullsized":"2024-04-24T23:00:32.016843+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:07:25.745306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":209,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.011684+0000","last_change":"2024-04-24T22:58:55.138810+0000","last_active":"2024-04-24T23:00:32.011684+0000","last_peered":"2024-04-24T23:00:32.011684+0000","last_clean":"2024-04-24T23:00:32.011684+0000","last_became_active":"2024-04-24T22:58:55.138627+0000","last_became_peered":"2024-04-24T22:58:55.138627+0000","last_unstale":"2024-04-24T23:00:32.011684+0000","last_undegraded":"2024-04-24T23:00:32.011684+0000","last_fullsized":"2024-04-24T23:00:32.011684+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_clean_scrub_stamp":"2024-04-24T22:58:54.127235+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:24.010002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":189,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195122,"statfs":{"total":200000000,"available":199805687,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":189,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195122,"statfs":{"total":200000000,"available":199805687,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":766,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194546,"statfs":{"total":200000000,"available":199215319,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":195,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195116,"statfs":{"total":200000000,"available":199799412,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":189,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195122,"statfs":{"total":200000000,"available":199805687,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":189,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195122,"statfs":{"total":200000000,"available":199805687,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":772,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194540,"statfs":{"total":200000000,"available":199209044,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":189,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195122,"statfs":{"total":200000000,"available":199805687,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-24T23:00:36.041 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:00:36.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.175+0000 7f01f0d90640 1 Processor -- start 2024-04-24T23:00:36.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.175+0000 7f01f0d90640 1 -- start start 2024-04-24T23:00:36.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.176+0000 7f01f0d90640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01ec06b820 0x7f01ec0d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:36.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.176+0000 7f01f0d90640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01ec0d3170 con 0x7f01ec0d4210 2024-04-24T23:00:36.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.176+0000 7f01f0d90640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01ec0d3450 con 0x7f01ec15e9b0 2024-04-24T23:00:36.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.176+0000 7f01f0d90640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f01ec0d3730 con 0x7f01ec06b820 2024-04-24T23:00:36.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.176+0000 7f01ead9d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f01ec0d4210 0x7f01ec0b78f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38460/0 (socket says 172.21.15.32:38460) 2024-04-24T23:00:36.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.176+0000 7f01ead9d640 1 -- 172.21.15.32:0/1656276730 learned_addr learned my addr 172.21.15.32:0/1656276730 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:00:36.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.177+0000 7f01e9d9b640 1 --2- 172.21.15.32:0/1656276730 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01ec06b820 0x7f01ec0d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:36.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.177+0000 7f01e959a640 1 -- 172.21.15.32:0/1656276730 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2373816842 0 0) 0x7f01ec0d3170 con 0x7f01ec0d4210 2024-04-24T23:00:36.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.177+0000 7f01e959a640 1 -- 172.21.15.32:0/1656276730 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f01d4003520 con 0x7f01ec0d4210 2024-04-24T23:00:36.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.177+0000 7f01e959a640 1 -- 172.21.15.32:0/1656276730 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4088769756 0 0) 0x7f01d4003520 con 0x7f01ec0d4210 2024-04-24T23:00:36.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.177+0000 7f01e959a640 1 -- 172.21.15.32:0/1656276730 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01ec06b820 msgr2=0x7f01ec0d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:36.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.177+0000 7f01e959a640 1 --2- 172.21.15.32:0/1656276730 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01ec06b820 0x7f01ec0d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:36.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.177+0000 7f01e959a640 1 -- 172.21.15.32:0/1656276730 >> v1:172.21.15.121:6789/0 conn(0x7f01ec15e9b0 legacy=0x7f01ec06b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:36.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.177+0000 7f01e959a640 1 -- 172.21.15.32:0/1656276730 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01ec0d3a10 con 0x7f01ec0d4210 2024-04-24T23:00:36.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.177+0000 7f01e959a640 1 -- 172.21.15.32:0/1656276730 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f01dc0019f0 con 0x7f01ec0d4210 2024-04-24T23:00:36.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.177+0000 7f01e959a640 1 -- 172.21.15.32:0/1656276730 >> v1:172.21.15.32:6789/0 conn(0x7f01ec0d4210 legacy=0x7f01ec0b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:36.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.177+0000 7f01e959a640 1 --2- 172.21.15.32:0/1656276730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01d40039e0 0x7f01d4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:36.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.177+0000 7f01e959a640 1 --2- 172.21.15.32:0/1656276730 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01ec06b820 0x7f01d4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:36.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.177+0000 7f01e959a640 1 --2- 172.21.15.32:0/1656276730 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01d4004d40 0x7f01d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:36.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.177+0000 7f01e959a640 1 -- 172.21.15.32:0/1656276730 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f01ec0d3730 con 0x7f01d40039e0 2024-04-24T23:00:36.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.177+0000 7f01e959a640 1 -- 172.21.15.32:0/1656276730 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f01d400baa0 con 0x7f01d4004d40 2024-04-24T23:00:36.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.177+0000 7f01e959a640 1 -- 172.21.15.32:0/1656276730 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f01d400bca0 con 0x7f01ec06b820 2024-04-24T23:00:36.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.177+0000 7f01ea59c640 1 --2- 172.21.15.32:0/1656276730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01d40039e0 0x7f01d4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:36.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.177+0000 7f01e9d9b640 1 --2- 172.21.15.32:0/1656276730 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01ec06b820 0x7f01d4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:36.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.177+0000 7f01ead9d640 1 --2- 172.21.15.32:0/1656276730 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01d4004d40 0x7f01d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:36.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.177+0000 7f01ea59c640 1 -- 172.21.15.32:0/1656276730 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01ec06b820 msgr2=0x7f01d4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:36.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.177+0000 7f01ea59c640 1 --2- 172.21.15.32:0/1656276730 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01ec06b820 0x7f01d4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:36.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.177+0000 7f01ea59c640 1 -- 172.21.15.32:0/1656276730 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01d4004d40 msgr2=0x7f01d40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:36.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.177+0000 7f01ea59c640 1 --2- 172.21.15.32:0/1656276730 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01d4004d40 0x7f01d40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:36.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.177+0000 7f01ea59c640 1 -- 172.21.15.32:0/1656276730 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01d4003520 con 0x7f01d40039e0 2024-04-24T23:00:36.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.178+0000 7f01ea59c640 1 --2- 172.21.15.32:0/1656276730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01d40039e0 0x7f01d4003e00 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f01e4001750 tx=0x7f01d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=136f01eaec122484 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:36.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.178+0000 7f01e959a640 1 -- 172.21.15.32:0/1656276730 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e4001370 con 0x7f01d40039e0 2024-04-24T23:00:36.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.178+0000 7f01e959a640 1 -- 172.21.15.32:0/1656276730 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01e40052d0 con 0x7f01d40039e0 2024-04-24T23:00:36.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.178+0000 7f01e959a640 1 -- 172.21.15.32:0/1656276730 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e402abe0 con 0x7f01d40039e0 2024-04-24T23:00:36.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.179+0000 7f01f0d90640 1 -- 172.21.15.32:0/1656276730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01d40039e0 msgr2=0x7f01d4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:36.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.179+0000 7f01f0d90640 1 --2- 172.21.15.32:0/1656276730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01d40039e0 0x7f01d4003e00 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f01e4001750 tx=0x7f01d4005830 comp rx=0 tx=0).stop 2024-04-24T23:00:36.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.179+0000 7f01ead9d640 1 -- 172.21.15.32:0/1656276730 reap_dead start 2024-04-24T23:00:36.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.179+0000 7f01f0d90640 1 -- 172.21.15.32:0/1656276730 shutdown_connections 2024-04-24T23:00:36.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.179+0000 7f01f0d90640 1 -- 172.21.15.32:0/1656276730 >> 172.21.15.32:0/1656276730 conn(0x7f01ec0d11d0 msgr2=0x7f01ec0d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:36.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.179+0000 7f01f0d90640 1 -- 172.21.15.32:0/1656276730 shutdown_connections 2024-04-24T23:00:36.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.179+0000 7f01f0d90640 1 -- 172.21.15.32:0/1656276730 wait complete. 2024-04-24T23:00:36.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.180+0000 7f01f0d90640 1 Processor -- start 2024-04-24T23:00:36.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.180+0000 7f01f0d90640 1 -- start start 2024-04-24T23:00:36.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.181+0000 7f01f0d90640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01d40039e0 0x7f01ec14bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:36.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.181+0000 7f01f0d90640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01ec158600 0x7f01ec1f77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:36.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.181+0000 7f01f0d90640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01ec1f4b10 0x7f01ec0c9120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:36.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.181+0000 7f01f0d90640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f01ec0ba0e0 con 0x7f01ec158600 2024-04-24T23:00:36.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.181+0000 7f01f0d90640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f01ec0d2400 con 0x7f01ec1f4b10 2024-04-24T23:00:36.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.181+0000 7f01f0d90640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f01ec0ba260 con 0x7f01d40039e0 2024-04-24T23:00:36.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.181+0000 7f01e9d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01ec158600 0x7f01ec1f77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:36.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.181+0000 7f01e9d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01ec158600 0x7f01ec1f77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46292/0 (socket says 172.21.15.32:46292) 2024-04-24T23:00:36.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.181+0000 7f01e9d9b640 1 -- 172.21.15.32:0/2368224875 learned_addr learned my addr 172.21.15.32:0/2368224875 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:36.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.181+0000 7f01ead9d640 1 --2- 172.21.15.32:0/2368224875 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01ec1f4b10 0x7f01ec0c9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:36.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.181+0000 7f01ea59c640 1 --2- 172.21.15.32:0/2368224875 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01d40039e0 0x7f01ec14bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:36.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.181+0000 7f01e9d9b640 1 -- 172.21.15.32:0/2368224875 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01d40039e0 msgr2=0x7f01ec14bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:36.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.181+0000 7f01e9d9b640 1 --2- 172.21.15.32:0/2368224875 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01d40039e0 0x7f01ec14bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:36.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.181+0000 7f01e9d9b640 1 -- 172.21.15.32:0/2368224875 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01ec1f4b10 msgr2=0x7f01ec0c9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:36.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.181+0000 7f01e9d9b640 1 --2- 172.21.15.32:0/2368224875 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01ec1f4b10 0x7f01ec0c9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:36.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.181+0000 7f01e9d9b640 1 -- 172.21.15.32:0/2368224875 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01ec1f80f0 con 0x7f01ec158600 2024-04-24T23:00:36.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.181+0000 7f01ea59c640 1 --2- 172.21.15.32:0/2368224875 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01d40039e0 0x7f01ec14bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:00:36.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.182+0000 7f01e9d9b640 1 --2- 172.21.15.32:0/2368224875 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01ec158600 0x7f01ec1f77f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f01e00119d0 tx=0x7f01e0011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:36.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.182+0000 7f01daffd640 1 -- 172.21.15.32:0/2368224875 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e000faf0 con 0x7f01ec158600 2024-04-24T23:00:36.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.182+0000 7f01daffd640 1 -- 172.21.15.32:0/2368224875 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01e000fc90 con 0x7f01ec158600 2024-04-24T23:00:36.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.182+0000 7f01f0d90640 1 -- 172.21.15.32:0/2368224875 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01ec1f83d0 con 0x7f01ec158600 2024-04-24T23:00:36.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.182+0000 7f01daffd640 1 -- 172.21.15.32:0/2368224875 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e0014440 con 0x7f01ec158600 2024-04-24T23:00:36.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.182+0000 7f01f0d90640 1 -- 172.21.15.32:0/2368224875 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01ec1f4310 con 0x7f01ec158600 2024-04-24T23:00:36.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.183+0000 7f01f0d90640 1 -- 172.21.15.32:0/2368224875 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01ec068430 con 0x7f01ec158600 2024-04-24T23:00:36.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.183+0000 7f01daffd640 1 -- 172.21.15.32:0/2368224875 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f01e0012040 con 0x7f01ec158600 2024-04-24T23:00:36.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.184+0000 7f01daffd640 1 --2- 172.21.15.32:0/2368224875 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f01cc06be60 0x7f01cc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:36.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.184+0000 7f01daffd640 1 -- 172.21.15.32:0/2368224875 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f01e008b3e0 con 0x7f01ec158600 2024-04-24T23:00:36.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.184+0000 7f01ea59c640 1 --2- 172.21.15.32:0/2368224875 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f01cc06be60 0x7f01cc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:36.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.185+0000 7f01ea59c640 1 --2- 172.21.15.32:0/2368224875 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f01cc06be60 0x7f01cc06e320 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f01e4032000 tx=0x7f01e402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:36.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.188+0000 7f01daffd640 1 -- 172.21.15.32:0/2368224875 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f01e0019030 con 0x7f01ec158600 2024-04-24T23:00:36.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.365+0000 7f01f0d90640 1 -- 172.21.15.32:0/2368224875 --> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f01ec0b7fa0 con 0x7f01cc06be60 2024-04-24T23:00:36.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.372+0000 7f01daffd640 1 -- 172.21.15.32:0/2368224875 <== mgr.4108 v2:172.21.15.121:6832/1848157943 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+242215 (secure 0 0 0) 0x7f01ec0b7fa0 con 0x7f01cc06be60 2024-04-24T23:00:36.375 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:00:36.376 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-24T23:00:36.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.374+0000 7f01f0d90640 1 -- 172.21.15.32:0/2368224875 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f01cc06be60 msgr2=0x7f01cc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:36.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.374+0000 7f01f0d90640 1 --2- 172.21.15.32:0/2368224875 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f01cc06be60 0x7f01cc06e320 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f01e4032000 tx=0x7f01e402e000 comp rx=0 tx=0).stop 2024-04-24T23:00:36.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.374+0000 7f01f0d90640 1 -- 172.21.15.32:0/2368224875 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01ec158600 msgr2=0x7f01ec1f77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:36.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.374+0000 7f01f0d90640 1 --2- 172.21.15.32:0/2368224875 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01ec158600 0x7f01ec1f77f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f01e00119d0 tx=0x7f01e0011e90 comp rx=0 tx=0).stop 2024-04-24T23:00:36.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.374+0000 7f01f0d90640 1 -- 172.21.15.32:0/2368224875 shutdown_connections 2024-04-24T23:00:36.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.374+0000 7f01f0d90640 1 --2- 172.21.15.32:0/2368224875 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f01cc06be60 0x7f01cc06e320 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:36.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.374+0000 7f01f0d90640 1 --2- 172.21.15.32:0/2368224875 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01ec158600 0x7f01ec1f77f0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:36.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.374+0000 7f01f0d90640 1 --2- 172.21.15.32:0/2368224875 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01ec1f4b10 0x7f01ec0c9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:36.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.374+0000 7f01f0d90640 1 --2- 172.21.15.32:0/2368224875 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01d40039e0 0x7f01ec14bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:36.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.374+0000 7f01f0d90640 1 -- 172.21.15.32:0/2368224875 >> 172.21.15.32:0/2368224875 conn(0x7f01ec0d11d0 msgr2=0x7f01ec15d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:36.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.374+0000 7f01f0d90640 1 -- 172.21.15.32:0/2368224875 shutdown_connections 2024-04-24T23:00:36.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.375+0000 7f01f0d90640 1 -- 172.21.15.32:0/2368224875 wait complete. 2024-04-24T23:00:36.387 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":88,"stamp":"2024-04-24T23:00:35.147698+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":4,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2683,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1559816,"statfs":{"total":1600000000,"available":1597252210,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.143686"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.015384+0000","last_change":"2024-04-24T23:00:32.015384+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.015384+0000","last_undegraded":"2024-04-24T23:00:32.015384+0000","last_fullsized":"2024-04-24T23:00:32.015384+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.012040+0000","last_change":"2024-04-24T23:00:32.012040+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.012040+0000","last_undegraded":"2024-04-24T23:00:32.012040+0000","last_fullsized":"2024-04-24T23:00:32.012040+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.013243+0000","last_change":"2024-04-24T23:00:32.013243+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.013243+0000","last_undegraded":"2024-04-24T23:00:32.013243+0000","last_fullsized":"2024-04-24T23:00:32.013243+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.021880+0000","last_change":"2024-04-24T23:00:32.021880+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.021880+0000","last_undegraded":"2024-04-24T23:00:32.021880+0000","last_fullsized":"2024-04-24T23:00:32.021880+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.013633+0000","last_change":"2024-04-24T23:00:32.013633+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.013633+0000","last_undegraded":"2024-04-24T23:00:32.013633+0000","last_fullsized":"2024-04-24T23:00:32.013633+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.010238+0000","last_change":"2024-04-24T23:00:32.010238+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.010238+0000","last_undegraded":"2024-04-24T23:00:32.010238+0000","last_fullsized":"2024-04-24T23:00:32.010238+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.019822+0000","last_change":"2024-04-24T23:00:32.019822+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.019822+0000","last_undegraded":"2024-04-24T23:00:32.019822+0000","last_fullsized":"2024-04-24T23:00:32.019822+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.022253+0000","last_change":"2024-04-24T23:00:32.022253+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.022253+0000","last_undegraded":"2024-04-24T23:00:32.022253+0000","last_fullsized":"2024-04-24T23:00:32.022253+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.009087+0000","last_change":"2024-04-24T23:00:32.009087+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.009087+0000","last_undegraded":"2024-04-24T23:00:32.009087+0000","last_fullsized":"2024-04-24T23:00:32.009087+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.014404+0000","last_change":"2024-04-24T23:00:32.014404+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.014404+0000","last_undegraded":"2024-04-24T23:00:32.014404+0000","last_fullsized":"2024-04-24T23:00:32.014404+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.022474+0000","last_change":"2024-04-24T23:00:32.022474+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.022474+0000","last_undegraded":"2024-04-24T23:00:32.022474+0000","last_fullsized":"2024-04-24T23:00:32.022474+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.011480+0000","last_change":"2024-04-24T23:00:32.011480+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.011480+0000","last_undegraded":"2024-04-24T23:00:32.011480+0000","last_fullsized":"2024-04-24T23:00:32.011480+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.010000+0000","last_change":"2024-04-24T23:00:32.010000+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.010000+0000","last_undegraded":"2024-04-24T23:00:32.010000+0000","last_fullsized":"2024-04-24T23:00:32.010000+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.021494+0000","last_change":"2024-04-24T23:00:32.021494+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.021494+0000","last_undegraded":"2024-04-24T23:00:32.021494+0000","last_fullsized":"2024-04-24T23:00:32.021494+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.021499+0000","last_change":"2024-04-24T23:00:32.021499+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.021499+0000","last_undegraded":"2024-04-24T23:00:32.021499+0000","last_fullsized":"2024-04-24T23:00:32.021499+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.015039+0000","last_change":"2024-04-24T23:00:32.015039+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.015039+0000","last_undegraded":"2024-04-24T23:00:32.015039+0000","last_fullsized":"2024-04-24T23:00:32.015039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.011162+0000","last_change":"2024-04-24T23:00:32.011162+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.011162+0000","last_undegraded":"2024-04-24T23:00:32.011162+0000","last_fullsized":"2024-04-24T23:00:32.011162+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.012933+0000","last_change":"2024-04-24T23:00:32.012933+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.012933+0000","last_undegraded":"2024-04-24T23:00:32.012933+0000","last_fullsized":"2024-04-24T23:00:32.012933+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.011970+0000","last_change":"2024-04-24T23:00:32.011970+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.011970+0000","last_undegraded":"2024-04-24T23:00:32.011970+0000","last_fullsized":"2024-04-24T23:00:32.011970+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.011779+0000","last_change":"2024-04-24T23:00:32.011779+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.011779+0000","last_undegraded":"2024-04-24T23:00:32.011779+0000","last_fullsized":"2024-04-24T23:00:32.011779+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.021919+0000","last_change":"2024-04-24T23:00:32.021919+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.021919+0000","last_undegraded":"2024-04-24T23:00:32.021919+0000","last_fullsized":"2024-04-24T23:00:32.021919+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.013643+0000","last_change":"2024-04-24T23:00:32.013643+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.013643+0000","last_undegraded":"2024-04-24T23:00:32.013643+0000","last_fullsized":"2024-04-24T23:00:32.013643+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.020052+0000","last_change":"2024-04-24T23:00:32.020052+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.020052+0000","last_undegraded":"2024-04-24T23:00:32.020052+0000","last_fullsized":"2024-04-24T23:00:32.020052+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.010783+0000","last_change":"2024-04-24T23:00:32.010783+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.010783+0000","last_undegraded":"2024-04-24T23:00:32.010783+0000","last_fullsized":"2024-04-24T23:00:32.010783+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.013276+0000","last_change":"2024-04-24T23:00:32.013276+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.013276+0000","last_undegraded":"2024-04-24T23:00:32.013276+0000","last_fullsized":"2024-04-24T23:00:32.013276+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.020899+0000","last_change":"2024-04-24T23:00:32.020899+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.020899+0000","last_undegraded":"2024-04-24T23:00:32.020899+0000","last_fullsized":"2024-04-24T23:00:32.020899+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.019540+0000","last_change":"2024-04-24T23:00:32.019540+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.019540+0000","last_undegraded":"2024-04-24T23:00:32.019540+0000","last_fullsized":"2024-04-24T23:00:32.019540+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.019756+0000","last_change":"2024-04-24T23:00:32.019756+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.019756+0000","last_undegraded":"2024-04-24T23:00:32.019756+0000","last_fullsized":"2024-04-24T23:00:32.019756+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.012654+0000","last_change":"2024-04-24T23:00:32.012654+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.012654+0000","last_undegraded":"2024-04-24T23:00:32.012654+0000","last_fullsized":"2024-04-24T23:00:32.012654+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.009579+0000","last_change":"2024-04-24T23:00:32.009579+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.009579+0000","last_undegraded":"2024-04-24T23:00:32.009579+0000","last_fullsized":"2024-04-24T23:00:32.009579+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.009836+0000","last_change":"2024-04-24T23:00:32.009836+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.009836+0000","last_undegraded":"2024-04-24T23:00:32.009836+0000","last_fullsized":"2024-04-24T23:00:32.009836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.020843+0000","last_change":"2024-04-24T23:00:32.020843+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.020843+0000","last_undegraded":"2024-04-24T23:00:32.020843+0000","last_fullsized":"2024-04-24T23:00:32.020843+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.015012+0000","last_change":"2024-04-24T23:00:32.015012+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.015012+0000","last_undegraded":"2024-04-24T23:00:32.015012+0000","last_fullsized":"2024-04-24T23:00:32.015012+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.014172+0000","last_change":"2024-04-24T23:00:32.013964+0000","last_active":"2024-04-24T23:00:32.014172+0000","last_peered":"2024-04-24T23:00:32.014172+0000","last_clean":"2024-04-24T23:00:32.014172+0000","last_became_active":"2024-04-24T23:00:32.013661+0000","last_became_peered":"2024-04-24T23:00:32.013661+0000","last_unstale":"2024-04-24T23:00:32.014172+0000","last_undegraded":"2024-04-24T23:00:32.014172+0000","last_fullsized":"2024-04-24T23:00:32.014172+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:34.956602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.012160+0000","last_change":"2024-04-24T23:00:32.012160+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.012160+0000","last_undegraded":"2024-04-24T23:00:32.012160+0000","last_fullsized":"2024-04-24T23:00:32.012160+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.012887+0000","last_change":"2024-04-24T23:00:32.012770+0000","last_active":"2024-04-24T23:00:32.012887+0000","last_peered":"2024-04-24T23:00:32.012887+0000","last_clean":"2024-04-24T23:00:32.012887+0000","last_became_active":"2024-04-24T23:00:32.012092+0000","last_became_peered":"2024-04-24T23:00:32.012092+0000","last_unstale":"2024-04-24T23:00:32.012887+0000","last_undegraded":"2024-04-24T23:00:32.012887+0000","last_fullsized":"2024-04-24T23:00:32.012887+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:03.280807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.009497+0000","last_change":"2024-04-24T23:00:32.009497+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.009497+0000","last_undegraded":"2024-04-24T23:00:32.009497+0000","last_fullsized":"2024-04-24T23:00:32.009497+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.016002+0000","last_change":"2024-04-24T23:00:32.015863+0000","last_active":"2024-04-24T23:00:32.016002+0000","last_peered":"2024-04-24T23:00:32.016002+0000","last_clean":"2024-04-24T23:00:32.016002+0000","last_became_active":"2024-04-24T23:00:32.015616+0000","last_became_peered":"2024-04-24T23:00:32.015616+0000","last_unstale":"2024-04-24T23:00:32.016002+0000","last_undegraded":"2024-04-24T23:00:32.016002+0000","last_fullsized":"2024-04-24T23:00:32.016002+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:29:18.370977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.020793+0000","last_change":"2024-04-24T23:00:32.020793+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.020793+0000","last_undegraded":"2024-04-24T23:00:32.020793+0000","last_fullsized":"2024-04-24T23:00:32.020793+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.010577+0000","last_change":"2024-04-24T23:00:32.010390+0000","last_active":"2024-04-24T23:00:32.010577+0000","last_peered":"2024-04-24T23:00:32.010577+0000","last_clean":"2024-04-24T23:00:32.010577+0000","last_became_active":"2024-04-24T23:00:32.010156+0000","last_became_peered":"2024-04-24T23:00:32.010156+0000","last_unstale":"2024-04-24T23:00:32.010577+0000","last_undegraded":"2024-04-24T23:00:32.010577+0000","last_fullsized":"2024-04-24T23:00:32.010577+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:38:29.668465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.019446+0000","last_change":"2024-04-24T23:00:32.019446+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.019446+0000","last_undegraded":"2024-04-24T23:00:32.019446+0000","last_fullsized":"2024-04-24T23:00:32.019446+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.012416+0000","last_change":"2024-04-24T23:00:32.012144+0000","last_active":"2024-04-24T23:00:32.012416+0000","last_peered":"2024-04-24T23:00:32.012416+0000","last_clean":"2024-04-24T23:00:32.012416+0000","last_became_active":"2024-04-24T23:00:32.011876+0000","last_became_peered":"2024-04-24T23:00:32.011876+0000","last_unstale":"2024-04-24T23:00:32.012416+0000","last_undegraded":"2024-04-24T23:00:32.012416+0000","last_fullsized":"2024-04-24T23:00:32.012416+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:25:17.401466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.012251+0000","last_change":"2024-04-24T23:00:32.012251+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.012251+0000","last_undegraded":"2024-04-24T23:00:32.012251+0000","last_fullsized":"2024-04-24T23:00:32.012251+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.015221+0000","last_change":"2024-04-24T23:00:32.015060+0000","last_active":"2024-04-24T23:00:32.015221+0000","last_peered":"2024-04-24T23:00:32.015221+0000","last_clean":"2024-04-24T23:00:32.015221+0000","last_became_active":"2024-04-24T23:00:32.013636+0000","last_became_peered":"2024-04-24T23:00:32.013636+0000","last_unstale":"2024-04-24T23:00:32.015221+0000","last_undegraded":"2024-04-24T23:00:32.015221+0000","last_fullsized":"2024-04-24T23:00:32.015221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:57:18.083931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.014094+0000","last_change":"2024-04-24T23:00:32.014094+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.014094+0000","last_undegraded":"2024-04-24T23:00:32.014094+0000","last_fullsized":"2024-04-24T23:00:32.014094+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.019078+0000","last_change":"2024-04-24T23:00:32.018961+0000","last_active":"2024-04-24T23:00:32.019078+0000","last_peered":"2024-04-24T23:00:32.019078+0000","last_clean":"2024-04-24T23:00:32.019078+0000","last_became_active":"2024-04-24T23:00:32.018718+0000","last_became_peered":"2024-04-24T23:00:32.018718+0000","last_unstale":"2024-04-24T23:00:32.019078+0000","last_undegraded":"2024-04-24T23:00:32.019078+0000","last_fullsized":"2024-04-24T23:00:32.019078+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:42:02.507492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.021039+0000","last_change":"2024-04-24T23:00:32.021039+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.021039+0000","last_undegraded":"2024-04-24T23:00:32.021039+0000","last_fullsized":"2024-04-24T23:00:32.021039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.016759+0000","last_change":"2024-04-24T23:00:32.016634+0000","last_active":"2024-04-24T23:00:32.016759+0000","last_peered":"2024-04-24T23:00:32.016759+0000","last_clean":"2024-04-24T23:00:32.016759+0000","last_became_active":"2024-04-24T23:00:32.015425+0000","last_became_peered":"2024-04-24T23:00:32.015425+0000","last_unstale":"2024-04-24T23:00:32.016759+0000","last_undegraded":"2024-04-24T23:00:32.016759+0000","last_fullsized":"2024-04-24T23:00:32.016759+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:32:23.611062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.012408+0000","last_change":"2024-04-24T23:00:32.012408+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.012408+0000","last_undegraded":"2024-04-24T23:00:32.012408+0000","last_fullsized":"2024-04-24T23:00:32.012408+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.017952+0000","last_change":"2024-04-24T23:00:32.017822+0000","last_active":"2024-04-24T23:00:32.017952+0000","last_peered":"2024-04-24T23:00:32.017952+0000","last_clean":"2024-04-24T23:00:32.017952+0000","last_became_active":"2024-04-24T23:00:32.017548+0000","last_became_peered":"2024-04-24T23:00:32.017548+0000","last_unstale":"2024-04-24T23:00:32.017952+0000","last_undegraded":"2024-04-24T23:00:32.017952+0000","last_fullsized":"2024-04-24T23:00:32.017952+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:18:08.921860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.015793+0000","last_change":"2024-04-24T23:00:32.015793+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.015793+0000","last_undegraded":"2024-04-24T23:00:32.015793+0000","last_fullsized":"2024-04-24T23:00:32.015793+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.016524+0000","last_change":"2024-04-24T23:00:32.016337+0000","last_active":"2024-04-24T23:00:32.016524+0000","last_peered":"2024-04-24T23:00:32.016524+0000","last_clean":"2024-04-24T23:00:32.016524+0000","last_became_active":"2024-04-24T23:00:32.015833+0000","last_became_peered":"2024-04-24T23:00:32.015833+0000","last_unstale":"2024-04-24T23:00:32.016524+0000","last_undegraded":"2024-04-24T23:00:32.016524+0000","last_fullsized":"2024-04-24T23:00:32.016524+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:00:47.017154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.020618+0000","last_change":"2024-04-24T23:00:32.020618+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.020618+0000","last_undegraded":"2024-04-24T23:00:32.020618+0000","last_fullsized":"2024-04-24T23:00:32.020618+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.017321+0000","last_change":"2024-04-24T23:00:32.017203+0000","last_active":"2024-04-24T23:00:32.017321+0000","last_peered":"2024-04-24T23:00:32.017321+0000","last_clean":"2024-04-24T23:00:32.017321+0000","last_became_active":"2024-04-24T23:00:32.016743+0000","last_became_peered":"2024-04-24T23:00:32.016743+0000","last_unstale":"2024-04-24T23:00:32.017321+0000","last_undegraded":"2024-04-24T23:00:32.017321+0000","last_fullsized":"2024-04-24T23:00:32.017321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:30:49.169292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.019128+0000","last_change":"2024-04-24T23:00:32.019128+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.019128+0000","last_undegraded":"2024-04-24T23:00:32.019128+0000","last_fullsized":"2024-04-24T23:00:32.019128+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.014710+0000","last_change":"2024-04-24T23:00:32.014044+0000","last_active":"2024-04-24T23:00:32.014710+0000","last_peered":"2024-04-24T23:00:32.014710+0000","last_clean":"2024-04-24T23:00:32.014710+0000","last_became_active":"2024-04-24T23:00:32.013781+0000","last_became_peered":"2024-04-24T23:00:32.013781+0000","last_unstale":"2024-04-24T23:00:32.014710+0000","last_undegraded":"2024-04-24T23:00:32.014710+0000","last_fullsized":"2024-04-24T23:00:32.014710+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:24:27.447216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.015479+0000","last_change":"2024-04-24T23:00:32.015479+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.015479+0000","last_undegraded":"2024-04-24T23:00:32.015479+0000","last_fullsized":"2024-04-24T23:00:32.015479+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.019354+0000","last_change":"2024-04-24T23:00:32.019200+0000","last_active":"2024-04-24T23:00:32.019354+0000","last_peered":"2024-04-24T23:00:32.019354+0000","last_clean":"2024-04-24T23:00:32.019354+0000","last_became_active":"2024-04-24T23:00:32.018935+0000","last_became_peered":"2024-04-24T23:00:32.018935+0000","last_unstale":"2024-04-24T23:00:32.019354+0000","last_undegraded":"2024-04-24T23:00:32.019354+0000","last_fullsized":"2024-04-24T23:00:32.019354+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:13.644308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.014218+0000","last_change":"2024-04-24T23:00:32.014218+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.014218+0000","last_undegraded":"2024-04-24T23:00:32.014218+0000","last_fullsized":"2024-04-24T23:00:32.014218+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.018213+0000","last_change":"2024-04-24T23:00:32.018091+0000","last_active":"2024-04-24T23:00:32.018213+0000","last_peered":"2024-04-24T23:00:32.018213+0000","last_clean":"2024-04-24T23:00:32.018213+0000","last_became_active":"2024-04-24T23:00:32.017828+0000","last_became_peered":"2024-04-24T23:00:32.017828+0000","last_unstale":"2024-04-24T23:00:32.018213+0000","last_undegraded":"2024-04-24T23:00:32.018213+0000","last_fullsized":"2024-04-24T23:00:32.018213+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:29:21.830443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.014085+0000","last_change":"2024-04-24T23:00:32.014085+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.014085+0000","last_undegraded":"2024-04-24T23:00:32.014085+0000","last_fullsized":"2024-04-24T23:00:32.014085+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.015462+0000","last_change":"2024-04-24T23:00:32.015309+0000","last_active":"2024-04-24T23:00:32.015462+0000","last_peered":"2024-04-24T23:00:32.015462+0000","last_clean":"2024-04-24T23:00:32.015462+0000","last_became_active":"2024-04-24T23:00:32.014784+0000","last_became_peered":"2024-04-24T23:00:32.014784+0000","last_unstale":"2024-04-24T23:00:32.015462+0000","last_undegraded":"2024-04-24T23:00:32.015462+0000","last_fullsized":"2024-04-24T23:00:32.015462+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:44:08.784625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.010204+0000","last_change":"2024-04-24T23:00:32.010204+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.010204+0000","last_undegraded":"2024-04-24T23:00:32.010204+0000","last_fullsized":"2024-04-24T23:00:32.010204+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.018350+0000","last_change":"2024-04-24T23:00:32.018235+0000","last_active":"2024-04-24T23:00:32.018350+0000","last_peered":"2024-04-24T23:00:32.018350+0000","last_clean":"2024-04-24T23:00:32.018350+0000","last_became_active":"2024-04-24T23:00:32.017933+0000","last_became_peered":"2024-04-24T23:00:32.017933+0000","last_unstale":"2024-04-24T23:00:32.018350+0000","last_undegraded":"2024-04-24T23:00:32.018350+0000","last_fullsized":"2024-04-24T23:00:32.018350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:30:30.768770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.013648+0000","last_change":"2024-04-24T23:00:32.013648+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.013648+0000","last_undegraded":"2024-04-24T23:00:32.013648+0000","last_fullsized":"2024-04-24T23:00:32.013648+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.015776+0000","last_change":"2024-04-24T23:00:32.015665+0000","last_active":"2024-04-24T23:00:32.015776+0000","last_peered":"2024-04-24T23:00:32.015776+0000","last_clean":"2024-04-24T23:00:32.015776+0000","last_became_active":"2024-04-24T23:00:32.015414+0000","last_became_peered":"2024-04-24T23:00:32.015414+0000","last_unstale":"2024-04-24T23:00:32.015776+0000","last_undegraded":"2024-04-24T23:00:32.015776+0000","last_fullsized":"2024-04-24T23:00:32.015776+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:34:48.498696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.014550+0000","last_change":"2024-04-24T23:00:32.014550+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.014550+0000","last_undegraded":"2024-04-24T23:00:32.014550+0000","last_fullsized":"2024-04-24T23:00:32.014550+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.016899+0000","last_change":"2024-04-24T23:00:32.016781+0000","last_active":"2024-04-24T23:00:32.016899+0000","last_peered":"2024-04-24T23:00:32.016899+0000","last_clean":"2024-04-24T23:00:32.016899+0000","last_became_active":"2024-04-24T23:00:32.016515+0000","last_became_peered":"2024-04-24T23:00:32.016515+0000","last_unstale":"2024-04-24T23:00:32.016899+0000","last_undegraded":"2024-04-24T23:00:32.016899+0000","last_fullsized":"2024-04-24T23:00:32.016899+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:57:27.039125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.009520+0000","last_change":"2024-04-24T23:00:32.009520+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.009520+0000","last_undegraded":"2024-04-24T23:00:32.009520+0000","last_fullsized":"2024-04-24T23:00:32.009520+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.019478+0000","last_change":"2024-04-24T23:00:32.019334+0000","last_active":"2024-04-24T23:00:32.019478+0000","last_peered":"2024-04-24T23:00:32.019478+0000","last_clean":"2024-04-24T23:00:32.019478+0000","last_became_active":"2024-04-24T23:00:32.019103+0000","last_became_peered":"2024-04-24T23:00:32.019103+0000","last_unstale":"2024-04-24T23:00:32.019478+0000","last_undegraded":"2024-04-24T23:00:32.019478+0000","last_fullsized":"2024-04-24T23:00:32.019478+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:08:14.771602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.015184+0000","last_change":"2024-04-24T23:00:32.015184+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.015184+0000","last_undegraded":"2024-04-24T23:00:32.015184+0000","last_fullsized":"2024-04-24T23:00:32.015184+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.015147+0000","last_change":"2024-04-24T23:00:32.015023+0000","last_active":"2024-04-24T23:00:32.015147+0000","last_peered":"2024-04-24T23:00:32.015147+0000","last_clean":"2024-04-24T23:00:32.015147+0000","last_became_active":"2024-04-24T23:00:32.014455+0000","last_became_peered":"2024-04-24T23:00:32.014455+0000","last_unstale":"2024-04-24T23:00:32.015147+0000","last_undegraded":"2024-04-24T23:00:32.015147+0000","last_fullsized":"2024-04-24T23:00:32.015147+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:16.368500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.011378+0000","last_change":"2024-04-24T23:00:32.011378+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.011378+0000","last_undegraded":"2024-04-24T23:00:32.011378+0000","last_fullsized":"2024-04-24T23:00:32.011378+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.019875+0000","last_change":"2024-04-24T23:00:32.019708+0000","last_active":"2024-04-24T23:00:32.019875+0000","last_peered":"2024-04-24T23:00:32.019875+0000","last_clean":"2024-04-24T23:00:32.019875+0000","last_became_active":"2024-04-24T23:00:32.019473+0000","last_became_peered":"2024-04-24T23:00:32.019473+0000","last_unstale":"2024-04-24T23:00:32.019875+0000","last_undegraded":"2024-04-24T23:00:32.019875+0000","last_fullsized":"2024-04-24T23:00:32.019875+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:57:15.174813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.009609+0000","last_change":"2024-04-24T23:00:32.009609+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.009609+0000","last_undegraded":"2024-04-24T23:00:32.009609+0000","last_fullsized":"2024-04-24T23:00:32.009609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.016029+0000","last_change":"2024-04-24T23:00:32.015397+0000","last_active":"2024-04-24T23:00:32.016029+0000","last_peered":"2024-04-24T23:00:32.016029+0000","last_clean":"2024-04-24T23:00:32.016029+0000","last_became_active":"2024-04-24T23:00:32.015154+0000","last_became_peered":"2024-04-24T23:00:32.015154+0000","last_unstale":"2024-04-24T23:00:32.016029+0000","last_undegraded":"2024-04-24T23:00:32.016029+0000","last_fullsized":"2024-04-24T23:00:32.016029+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:04:05.572144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.020075+0000","last_change":"2024-04-24T23:00:32.020075+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.020075+0000","last_undegraded":"2024-04-24T23:00:32.020075+0000","last_fullsized":"2024-04-24T23:00:32.020075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.018003+0000","last_change":"2024-04-24T23:00:32.017383+0000","last_active":"2024-04-24T23:00:32.018003+0000","last_peered":"2024-04-24T23:00:32.018003+0000","last_clean":"2024-04-24T23:00:32.018003+0000","last_became_active":"2024-04-24T23:00:32.016962+0000","last_became_peered":"2024-04-24T23:00:32.016962+0000","last_unstale":"2024-04-24T23:00:32.018003+0000","last_undegraded":"2024-04-24T23:00:32.018003+0000","last_fullsized":"2024-04-24T23:00:32.018003+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:44:38.000055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.020501+0000","last_change":"2024-04-24T23:00:32.020501+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.020501+0000","last_undegraded":"2024-04-24T23:00:32.020501+0000","last_fullsized":"2024-04-24T23:00:32.020501+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.012229+0000","last_change":"2024-04-24T23:00:32.011951+0000","last_active":"2024-04-24T23:00:32.012229+0000","last_peered":"2024-04-24T23:00:32.012229+0000","last_clean":"2024-04-24T23:00:32.012229+0000","last_became_active":"2024-04-24T23:00:32.011483+0000","last_became_peered":"2024-04-24T23:00:32.011483+0000","last_unstale":"2024-04-24T23:00:32.012229+0000","last_undegraded":"2024-04-24T23:00:32.012229+0000","last_fullsized":"2024-04-24T23:00:32.012229+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:05.541366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.019204+0000","last_change":"2024-04-24T23:00:32.019204+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.019204+0000","last_undegraded":"2024-04-24T23:00:32.019204+0000","last_fullsized":"2024-04-24T23:00:32.019204+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.017120+0000","last_change":"2024-04-24T23:00:32.016280+0000","last_active":"2024-04-24T23:00:32.017120+0000","last_peered":"2024-04-24T23:00:32.017120+0000","last_clean":"2024-04-24T23:00:32.017120+0000","last_became_active":"2024-04-24T23:00:32.016057+0000","last_became_peered":"2024-04-24T23:00:32.016057+0000","last_unstale":"2024-04-24T23:00:32.017120+0000","last_undegraded":"2024-04-24T23:00:32.017120+0000","last_fullsized":"2024-04-24T23:00:32.017120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:25:31.020426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.013723+0000","last_change":"2024-04-24T23:00:32.013723+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.013723+0000","last_undegraded":"2024-04-24T23:00:32.013723+0000","last_fullsized":"2024-04-24T23:00:32.013723+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.016054+0000","last_change":"2024-04-24T23:00:32.015930+0000","last_active":"2024-04-24T23:00:32.016054+0000","last_peered":"2024-04-24T23:00:32.016054+0000","last_clean":"2024-04-24T23:00:32.016054+0000","last_became_active":"2024-04-24T23:00:32.015671+0000","last_became_peered":"2024-04-24T23:00:32.015671+0000","last_unstale":"2024-04-24T23:00:32.016054+0000","last_undegraded":"2024-04-24T23:00:32.016054+0000","last_fullsized":"2024-04-24T23:00:32.016054+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:47.216626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.020647+0000","last_change":"2024-04-24T23:00:32.020647+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.020647+0000","last_undegraded":"2024-04-24T23:00:32.020647+0000","last_fullsized":"2024-04-24T23:00:32.020647+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.020210+0000","last_change":"2024-04-24T23:00:32.020060+0000","last_active":"2024-04-24T23:00:32.020210+0000","last_peered":"2024-04-24T23:00:32.020210+0000","last_clean":"2024-04-24T23:00:32.020210+0000","last_became_active":"2024-04-24T23:00:32.019783+0000","last_became_peered":"2024-04-24T23:00:32.019783+0000","last_unstale":"2024-04-24T23:00:32.020210+0000","last_undegraded":"2024-04-24T23:00:32.020210+0000","last_fullsized":"2024-04-24T23:00:32.020210+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:24:44.692764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.011948+0000","last_change":"2024-04-24T23:00:32.011948+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.011948+0000","last_undegraded":"2024-04-24T23:00:32.011948+0000","last_fullsized":"2024-04-24T23:00:32.011948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.020335+0000","last_change":"2024-04-24T23:00:32.020217+0000","last_active":"2024-04-24T23:00:32.020335+0000","last_peered":"2024-04-24T23:00:32.020335+0000","last_clean":"2024-04-24T23:00:32.020335+0000","last_became_active":"2024-04-24T23:00:32.019728+0000","last_became_peered":"2024-04-24T23:00:32.019728+0000","last_unstale":"2024-04-24T23:00:32.020335+0000","last_undegraded":"2024-04-24T23:00:32.020335+0000","last_fullsized":"2024-04-24T23:00:32.020335+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:33:53.601267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.018940+0000","last_change":"2024-04-24T23:00:32.018940+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.018940+0000","last_undegraded":"2024-04-24T23:00:32.018940+0000","last_fullsized":"2024-04-24T23:00:32.018940+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.018841+0000","last_change":"2024-04-24T23:00:32.018725+0000","last_active":"2024-04-24T23:00:32.018841+0000","last_peered":"2024-04-24T23:00:32.018841+0000","last_clean":"2024-04-24T23:00:32.018841+0000","last_became_active":"2024-04-24T23:00:32.018472+0000","last_became_peered":"2024-04-24T23:00:32.018472+0000","last_unstale":"2024-04-24T23:00:32.018841+0000","last_undegraded":"2024-04-24T23:00:32.018841+0000","last_fullsized":"2024-04-24T23:00:32.018841+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:58:07.274166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.011506+0000","last_change":"2024-04-24T23:00:32.011506+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.011506+0000","last_undegraded":"2024-04-24T23:00:32.011506+0000","last_fullsized":"2024-04-24T23:00:32.011506+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.012098+0000","last_change":"2024-04-24T23:00:32.011954+0000","last_active":"2024-04-24T23:00:32.012098+0000","last_peered":"2024-04-24T23:00:32.012098+0000","last_clean":"2024-04-24T23:00:32.012098+0000","last_became_active":"2024-04-24T23:00:32.011189+0000","last_became_peered":"2024-04-24T23:00:32.011189+0000","last_unstale":"2024-04-24T23:00:32.012098+0000","last_undegraded":"2024-04-24T23:00:32.012098+0000","last_fullsized":"2024-04-24T23:00:32.012098+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:10:23.000844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.009799+0000","last_change":"2024-04-24T23:00:32.009799+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.009799+0000","last_undegraded":"2024-04-24T23:00:32.009799+0000","last_fullsized":"2024-04-24T23:00:32.009799+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.020146+0000","last_change":"2024-04-24T23:00:32.019965+0000","last_active":"2024-04-24T23:00:32.020146+0000","last_peered":"2024-04-24T23:00:32.020146+0000","last_clean":"2024-04-24T23:00:32.020146+0000","last_became_active":"2024-04-24T23:00:32.019691+0000","last_became_peered":"2024-04-24T23:00:32.019691+0000","last_unstale":"2024-04-24T23:00:32.020146+0000","last_undegraded":"2024-04-24T23:00:32.020146+0000","last_fullsized":"2024-04-24T23:00:32.020146+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:32:54.766031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.019001+0000","last_change":"2024-04-24T23:00:32.019001+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.019001+0000","last_undegraded":"2024-04-24T23:00:32.019001+0000","last_fullsized":"2024-04-24T23:00:32.019001+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.016843+0000","last_change":"2024-04-24T23:00:32.016499+0000","last_active":"2024-04-24T23:00:32.016843+0000","last_peered":"2024-04-24T23:00:32.016843+0000","last_clean":"2024-04-24T23:00:32.016843+0000","last_became_active":"2024-04-24T23:00:32.016247+0000","last_became_peered":"2024-04-24T23:00:32.016247+0000","last_unstale":"2024-04-24T23:00:32.016843+0000","last_undegraded":"2024-04-24T23:00:32.016843+0000","last_fullsized":"2024-04-24T23:00:32.016843+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:07:25.745306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":209,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.011684+0000","last_change":"2024-04-24T22:58:55.138810+0000","last_active":"2024-04-24T23:00:32.011684+0000","last_peered":"2024-04-24T23:00:32.011684+0000","last_clean":"2024-04-24T23:00:32.011684+0000","last_became_active":"2024-04-24T22:58:55.138627+0000","last_became_peered":"2024-04-24T22:58:55.138627+0000","last_unstale":"2024-04-24T23:00:32.011684+0000","last_undegraded":"2024-04-24T23:00:32.011684+0000","last_fullsized":"2024-04-24T23:00:32.011684+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_clean_scrub_stamp":"2024-04-24T22:58:54.127235+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:24.010002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":189,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195122,"statfs":{"total":200000000,"available":199805687,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":189,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195122,"statfs":{"total":200000000,"available":199805687,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":766,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194546,"statfs":{"total":200000000,"available":199215319,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":195,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195116,"statfs":{"total":200000000,"available":199799412,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":189,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195122,"statfs":{"total":200000000,"available":199805687,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":189,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195122,"statfs":{"total":200000000,"available":199805687,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":772,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194540,"statfs":{"total":200000000,"available":199209044,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":189,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195122,"statfs":{"total":200000000,"available":199805687,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-24T23:00:36.390 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:00:36.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.526+0000 7ff0d321c640 1 Processor -- start 2024-04-24T23:00:36.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.527+0000 7ff0d321c640 1 -- start start 2024-04-24T23:00:36.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.527+0000 7ff0d321c640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff0cc06b820 0x7ff0cc0d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:36.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.527+0000 7ff0d321c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff0cc0d4210 0x7ff0cc0b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:36.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.527+0000 7ff0d321c640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0cc0d3170 con 0x7ff0cc15e9b0 2024-04-24T23:00:36.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.527+0000 7ff0d321c640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7ff0cc0d3450 con 0x7ff0cc06b820 2024-04-24T23:00:36.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.527+0000 7ff0d321c640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff0cc0d3730 con 0x7ff0cc0d4210 2024-04-24T23:00:36.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.527+0000 7ff0d0fb8640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff0cc15e9b0 0x7ff0cc06b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38476/0 (socket says 172.21.15.32:38476) 2024-04-24T23:00:36.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.527+0000 7ff0d0fb8640 1 -- 172.21.15.32:0/3563631901 learned_addr learned my addr 172.21.15.32:0/3563631901 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:00:36.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.528+0000 7ff0cbfff640 1 --2- 172.21.15.32:0/3563631901 >> v2:172.21.15.121:3300/0 conn(0x7ff0cc06b820 0x7ff0cc0d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:36.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.528+0000 7ff0d17b9640 1 --2- 172.21.15.32:0/3563631901 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff0cc0d4210 0x7ff0cc0b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:36.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.528+0000 7ff0cb7fe640 1 -- 172.21.15.32:0/3563631901 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2779482588 0 0) 0x7ff0cc0d3170 con 0x7ff0cc15e9b0 2024-04-24T23:00:36.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.528+0000 7ff0cb7fe640 1 -- 172.21.15.32:0/3563631901 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff0b8003520 con 0x7ff0cc15e9b0 2024-04-24T23:00:36.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.528+0000 7ff0cb7fe640 1 -- 172.21.15.32:0/3563631901 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1180786147 0 0) 0x7ff0b8003520 con 0x7ff0cc15e9b0 2024-04-24T23:00:36.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.528+0000 7ff0cb7fe640 1 -- 172.21.15.32:0/3563631901 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff0cc0d4210 msgr2=0x7ff0cc0b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:36.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.528+0000 7ff0cb7fe640 1 --2- 172.21.15.32:0/3563631901 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff0cc0d4210 0x7ff0cc0b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:36.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.528+0000 7ff0cb7fe640 1 -- 172.21.15.32:0/3563631901 >> v2:172.21.15.121:3300/0 conn(0x7ff0cc06b820 msgr2=0x7ff0cc0d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:36.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.528+0000 7ff0cb7fe640 1 --2- 172.21.15.32:0/3563631901 >> v2:172.21.15.121:3300/0 conn(0x7ff0cc06b820 0x7ff0cc0d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:36.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.528+0000 7ff0cb7fe640 1 -- 172.21.15.32:0/3563631901 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0cc0d3a10 con 0x7ff0cc15e9b0 2024-04-24T23:00:36.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.528+0000 7ff0cb7fe640 1 -- 172.21.15.32:0/3563631901 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7ff0c40019f0 con 0x7ff0cc15e9b0 2024-04-24T23:00:36.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.528+0000 7ff0cb7fe640 1 -- 172.21.15.32:0/3563631901 >> v1:172.21.15.32:6789/0 conn(0x7ff0cc15e9b0 legacy=0x7ff0cc06b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:36.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.528+0000 7ff0cb7fe640 1 --2- 172.21.15.32:0/3563631901 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff0cc0d4210 0x7ff0b80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:36.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.528+0000 7ff0cb7fe640 1 --2- 172.21.15.32:0/3563631901 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0b8003f30 0x7ff0b8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:36.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.528+0000 7ff0cb7fe640 1 --2- 172.21.15.32:0/3563631901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0b8004d40 0x7ff0b80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:36.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.528+0000 7ff0cb7fe640 1 -- 172.21.15.32:0/3563631901 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff0cc0d3450 con 0x7ff0b8004d40 2024-04-24T23:00:36.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.528+0000 7ff0cb7fe640 1 -- 172.21.15.32:0/3563631901 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff0cc0d3730 con 0x7ff0b8003f30 2024-04-24T23:00:36.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.528+0000 7ff0cb7fe640 1 -- 172.21.15.32:0/3563631901 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff0b800b9c0 con 0x7ff0cc0d4210 2024-04-24T23:00:36.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.528+0000 7ff0d17b9640 1 --2- 172.21.15.32:0/3563631901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0b8004d40 0x7ff0b80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:36.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.528+0000 7ff0cbfff640 1 --2- 172.21.15.32:0/3563631901 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0b8003f30 0x7ff0b8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:36.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.528+0000 7ff0d0fb8640 1 --2- 172.21.15.32:0/3563631901 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff0cc0d4210 0x7ff0b80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:36.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.529+0000 7ff0d17b9640 1 -- 172.21.15.32:0/3563631901 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff0cc0d4210 msgr2=0x7ff0b80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:36.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.529+0000 7ff0d17b9640 1 --2- 172.21.15.32:0/3563631901 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff0cc0d4210 0x7ff0b80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:36.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.529+0000 7ff0d17b9640 1 -- 172.21.15.32:0/3563631901 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0b8003f30 msgr2=0x7ff0b8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:36.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.529+0000 7ff0d17b9640 1 --2- 172.21.15.32:0/3563631901 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0b8003f30 0x7ff0b8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:36.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.529+0000 7ff0d17b9640 1 -- 172.21.15.32:0/3563631901 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0b8003520 con 0x7ff0b8004d40 2024-04-24T23:00:36.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.529+0000 7ff0d17b9640 1 --2- 172.21.15.32:0/3563631901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0b8004d40 0x7ff0b80051c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4009a90 tx=0x7ff0b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c679aca9a28a0192 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:36.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.529+0000 7ff0cb7fe640 1 -- 172.21.15.32:0/3563631901 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b4036070 con 0x7ff0b8004d40 2024-04-24T23:00:36.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.529+0000 7ff0cb7fe640 1 -- 172.21.15.32:0/3563631901 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0b4031400 con 0x7ff0b8004d40 2024-04-24T23:00:36.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.529+0000 7ff0cb7fe640 1 -- 172.21.15.32:0/3563631901 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b4009690 con 0x7ff0b8004d40 2024-04-24T23:00:36.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.530+0000 7ff0d321c640 1 -- 172.21.15.32:0/3563631901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0b8004d40 msgr2=0x7ff0b80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:36.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.530+0000 7ff0d321c640 1 --2- 172.21.15.32:0/3563631901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0b8004d40 0x7ff0b80051c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4009a90 tx=0x7ff0b8005830 comp rx=0 tx=0).stop 2024-04-24T23:00:36.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.530+0000 7ff0d17b9640 1 -- 172.21.15.32:0/3563631901 reap_dead start 2024-04-24T23:00:36.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.530+0000 7ff0d321c640 1 -- 172.21.15.32:0/3563631901 shutdown_connections 2024-04-24T23:00:36.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.530+0000 7ff0d321c640 1 -- 172.21.15.32:0/3563631901 >> 172.21.15.32:0/3563631901 conn(0x7ff0cc0d11d0 msgr2=0x7ff0cc0d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:36.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.530+0000 7ff0d321c640 1 -- 172.21.15.32:0/3563631901 shutdown_connections 2024-04-24T23:00:36.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.530+0000 7ff0d321c640 1 -- 172.21.15.32:0/3563631901 wait complete. 2024-04-24T23:00:36.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.531+0000 7ff0d321c640 1 Processor -- start 2024-04-24T23:00:36.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.531+0000 7ff0d321c640 1 -- start start 2024-04-24T23:00:36.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.532+0000 7ff0d321c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff0cc149bb0 0x7ff0cc1f46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:36.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.532+0000 7ff0d321c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0cc0b8d50 0x7ff0cc0b91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:36.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.532+0000 7ff0d321c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0cc15f6d0 0x7ff0cc15fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:36.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.532+0000 7ff0d321c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff0cc0ba0e0 con 0x7ff0cc15f6d0 2024-04-24T23:00:36.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.532+0000 7ff0d321c640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff0cc0d2400 con 0x7ff0cc0b8d50 2024-04-24T23:00:36.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.532+0000 7ff0d321c640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff0cc0ba260 con 0x7ff0cc149bb0 2024-04-24T23:00:36.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.532+0000 7ff0d17b9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0cc15f6d0 0x7ff0cc15fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:36.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.532+0000 7ff0d17b9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0cc15f6d0 0x7ff0cc15fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46312/0 (socket says 172.21.15.32:46312) 2024-04-24T23:00:36.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.532+0000 7ff0d17b9640 1 -- 172.21.15.32:0/1491988270 learned_addr learned my addr 172.21.15.32:0/1491988270 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:36.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.532+0000 7ff0cbfff640 1 --2- 172.21.15.32:0/1491988270 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0cc0b8d50 0x7ff0cc0b91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:36.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.532+0000 7ff0d0fb8640 1 --2- 172.21.15.32:0/1491988270 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff0cc149bb0 0x7ff0cc1f46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:36.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.532+0000 7ff0d17b9640 1 -- 172.21.15.32:0/1491988270 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff0cc149bb0 msgr2=0x7ff0cc1f46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:36.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.532+0000 7ff0d17b9640 1 --2- 172.21.15.32:0/1491988270 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff0cc149bb0 0x7ff0cc1f46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:36.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.532+0000 7ff0d17b9640 1 -- 172.21.15.32:0/1491988270 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0cc0b8d50 msgr2=0x7ff0cc0b91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:36.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.532+0000 7ff0d17b9640 1 --2- 172.21.15.32:0/1491988270 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0cc0b8d50 0x7ff0cc0b91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:36.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.532+0000 7ff0d17b9640 1 -- 172.21.15.32:0/1491988270 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0cc0cc980 con 0x7ff0cc15f6d0 2024-04-24T23:00:36.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.532+0000 7ff0cbfff640 1 --2- 172.21.15.32:0/1491988270 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0cc0b8d50 0x7ff0cc0b91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:00:36.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.533+0000 7ff0d17b9640 1 --2- 172.21.15.32:0/1491988270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0cc15f6d0 0x7ff0cc15fb50 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4034040 tx=0x7ff0b4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:36.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.533+0000 7ff0c97fa640 1 -- 172.21.15.32:0/1491988270 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b4040020 con 0x7ff0cc15f6d0 2024-04-24T23:00:36.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.533+0000 7ff0c97fa640 1 -- 172.21.15.32:0/1491988270 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0b4037040 con 0x7ff0cc15f6d0 2024-04-24T23:00:36.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.533+0000 7ff0d321c640 1 -- 172.21.15.32:0/1491988270 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0cc0ccc60 con 0x7ff0cc15f6d0 2024-04-24T23:00:36.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.533+0000 7ff0c97fa640 1 -- 172.21.15.32:0/1491988270 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b4036070 con 0x7ff0cc15f6d0 2024-04-24T23:00:36.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.533+0000 7ff0d321c640 1 -- 172.21.15.32:0/1491988270 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0cc0cd220 con 0x7ff0cc15f6d0 2024-04-24T23:00:36.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.534+0000 7ff0d321c640 1 -- 172.21.15.32:0/1491988270 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0cc0b9710 con 0x7ff0cc15f6d0 2024-04-24T23:00:36.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.534+0000 7ff0c97fa640 1 -- 172.21.15.32:0/1491988270 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff0b4006030 con 0x7ff0cc15f6d0 2024-04-24T23:00:36.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.535+0000 7ff0c97fa640 1 --2- 172.21.15.32:0/1491988270 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff0ac06be30 0x7ff0ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:36.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.535+0000 7ff0c97fa640 1 -- 172.21.15.32:0/1491988270 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7ff0b40aaa10 con 0x7ff0cc15f6d0 2024-04-24T23:00:36.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.535+0000 7ff0d0fb8640 1 --2- 172.21.15.32:0/1491988270 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff0ac06be30 0x7ff0ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:36.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.536+0000 7ff0d0fb8640 1 --2- 172.21.15.32:0/1491988270 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff0ac06be30 0x7ff0ac06e2f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4004bc0 tx=0x7ff0c4006040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:36.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.539+0000 7ff0c97fa640 1 -- 172.21.15.32:0/1491988270 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff0b404a020 con 0x7ff0cc15f6d0 2024-04-24T23:00:36.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.717+0000 7ff0d321c640 1 -- 172.21.15.32:0/1491988270 --> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff0cc0ca690 con 0x7ff0ac06be30 2024-04-24T23:00:36.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.723+0000 7ff0c97fa640 1 -- 172.21.15.32:0/1491988270 <== mgr.4108 v2:172.21.15.121:6832/1848157943 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+242215 (secure 0 0 0) 0x7ff0cc0ca690 con 0x7ff0ac06be30 2024-04-24T23:00:36.727 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:00:36.728 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-24T23:00:36.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.726+0000 7ff0d321c640 1 -- 172.21.15.32:0/1491988270 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff0ac06be30 msgr2=0x7ff0ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:36.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.726+0000 7ff0d321c640 1 --2- 172.21.15.32:0/1491988270 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff0ac06be30 0x7ff0ac06e2f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4004bc0 tx=0x7ff0c4006040 comp rx=0 tx=0).stop 2024-04-24T23:00:36.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.726+0000 7ff0d321c640 1 -- 172.21.15.32:0/1491988270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0cc15f6d0 msgr2=0x7ff0cc15fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:36.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.726+0000 7ff0d321c640 1 --2- 172.21.15.32:0/1491988270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0cc15f6d0 0x7ff0cc15fb50 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4034040 tx=0x7ff0b4004730 comp rx=0 tx=0).stop 2024-04-24T23:00:36.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.726+0000 7ff0d321c640 1 -- 172.21.15.32:0/1491988270 shutdown_connections 2024-04-24T23:00:36.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.726+0000 7ff0d321c640 1 --2- 172.21.15.32:0/1491988270 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff0ac06be30 0x7ff0ac06e2f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:36.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.726+0000 7ff0d321c640 1 --2- 172.21.15.32:0/1491988270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0cc15f6d0 0x7ff0cc15fb50 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:36.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.726+0000 7ff0d321c640 1 --2- 172.21.15.32:0/1491988270 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0cc0b8d50 0x7ff0cc0b91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:36.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.726+0000 7ff0d321c640 1 --2- 172.21.15.32:0/1491988270 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff0cc149bb0 0x7ff0cc1f46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:36.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.726+0000 7ff0d321c640 1 -- 172.21.15.32:0/1491988270 >> 172.21.15.32:0/1491988270 conn(0x7ff0cc0d11d0 msgr2=0x7ff0cc06b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:36.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.726+0000 7ff0d321c640 1 -- 172.21.15.32:0/1491988270 shutdown_connections 2024-04-24T23:00:36.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.726+0000 7ff0d321c640 1 -- 172.21.15.32:0/1491988270 wait complete. 2024-04-24T23:00:36.739 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":88,"stamp":"2024-04-24T23:00:35.147698+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":4,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2683,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1559816,"statfs":{"total":1600000000,"available":1597252210,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.143686"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.015384+0000","last_change":"2024-04-24T23:00:32.015384+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.015384+0000","last_undegraded":"2024-04-24T23:00:32.015384+0000","last_fullsized":"2024-04-24T23:00:32.015384+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.012040+0000","last_change":"2024-04-24T23:00:32.012040+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.012040+0000","last_undegraded":"2024-04-24T23:00:32.012040+0000","last_fullsized":"2024-04-24T23:00:32.012040+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.013243+0000","last_change":"2024-04-24T23:00:32.013243+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.013243+0000","last_undegraded":"2024-04-24T23:00:32.013243+0000","last_fullsized":"2024-04-24T23:00:32.013243+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.021880+0000","last_change":"2024-04-24T23:00:32.021880+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.021880+0000","last_undegraded":"2024-04-24T23:00:32.021880+0000","last_fullsized":"2024-04-24T23:00:32.021880+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.013633+0000","last_change":"2024-04-24T23:00:32.013633+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.013633+0000","last_undegraded":"2024-04-24T23:00:32.013633+0000","last_fullsized":"2024-04-24T23:00:32.013633+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.010238+0000","last_change":"2024-04-24T23:00:32.010238+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.010238+0000","last_undegraded":"2024-04-24T23:00:32.010238+0000","last_fullsized":"2024-04-24T23:00:32.010238+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.019822+0000","last_change":"2024-04-24T23:00:32.019822+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.019822+0000","last_undegraded":"2024-04-24T23:00:32.019822+0000","last_fullsized":"2024-04-24T23:00:32.019822+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.022253+0000","last_change":"2024-04-24T23:00:32.022253+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.022253+0000","last_undegraded":"2024-04-24T23:00:32.022253+0000","last_fullsized":"2024-04-24T23:00:32.022253+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.009087+0000","last_change":"2024-04-24T23:00:32.009087+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.009087+0000","last_undegraded":"2024-04-24T23:00:32.009087+0000","last_fullsized":"2024-04-24T23:00:32.009087+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.014404+0000","last_change":"2024-04-24T23:00:32.014404+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.014404+0000","last_undegraded":"2024-04-24T23:00:32.014404+0000","last_fullsized":"2024-04-24T23:00:32.014404+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.022474+0000","last_change":"2024-04-24T23:00:32.022474+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.022474+0000","last_undegraded":"2024-04-24T23:00:32.022474+0000","last_fullsized":"2024-04-24T23:00:32.022474+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.011480+0000","last_change":"2024-04-24T23:00:32.011480+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.011480+0000","last_undegraded":"2024-04-24T23:00:32.011480+0000","last_fullsized":"2024-04-24T23:00:32.011480+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.010000+0000","last_change":"2024-04-24T23:00:32.010000+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.010000+0000","last_undegraded":"2024-04-24T23:00:32.010000+0000","last_fullsized":"2024-04-24T23:00:32.010000+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.021494+0000","last_change":"2024-04-24T23:00:32.021494+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.021494+0000","last_undegraded":"2024-04-24T23:00:32.021494+0000","last_fullsized":"2024-04-24T23:00:32.021494+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.021499+0000","last_change":"2024-04-24T23:00:32.021499+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.021499+0000","last_undegraded":"2024-04-24T23:00:32.021499+0000","last_fullsized":"2024-04-24T23:00:32.021499+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.015039+0000","last_change":"2024-04-24T23:00:32.015039+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.015039+0000","last_undegraded":"2024-04-24T23:00:32.015039+0000","last_fullsized":"2024-04-24T23:00:32.015039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.011162+0000","last_change":"2024-04-24T23:00:32.011162+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.011162+0000","last_undegraded":"2024-04-24T23:00:32.011162+0000","last_fullsized":"2024-04-24T23:00:32.011162+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.012933+0000","last_change":"2024-04-24T23:00:32.012933+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.012933+0000","last_undegraded":"2024-04-24T23:00:32.012933+0000","last_fullsized":"2024-04-24T23:00:32.012933+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.011970+0000","last_change":"2024-04-24T23:00:32.011970+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.011970+0000","last_undegraded":"2024-04-24T23:00:32.011970+0000","last_fullsized":"2024-04-24T23:00:32.011970+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.011779+0000","last_change":"2024-04-24T23:00:32.011779+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.011779+0000","last_undegraded":"2024-04-24T23:00:32.011779+0000","last_fullsized":"2024-04-24T23:00:32.011779+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.021919+0000","last_change":"2024-04-24T23:00:32.021919+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.021919+0000","last_undegraded":"2024-04-24T23:00:32.021919+0000","last_fullsized":"2024-04-24T23:00:32.021919+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.013643+0000","last_change":"2024-04-24T23:00:32.013643+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.013643+0000","last_undegraded":"2024-04-24T23:00:32.013643+0000","last_fullsized":"2024-04-24T23:00:32.013643+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.020052+0000","last_change":"2024-04-24T23:00:32.020052+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.020052+0000","last_undegraded":"2024-04-24T23:00:32.020052+0000","last_fullsized":"2024-04-24T23:00:32.020052+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.010783+0000","last_change":"2024-04-24T23:00:32.010783+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.010783+0000","last_undegraded":"2024-04-24T23:00:32.010783+0000","last_fullsized":"2024-04-24T23:00:32.010783+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.013276+0000","last_change":"2024-04-24T23:00:32.013276+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.013276+0000","last_undegraded":"2024-04-24T23:00:32.013276+0000","last_fullsized":"2024-04-24T23:00:32.013276+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.020899+0000","last_change":"2024-04-24T23:00:32.020899+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.020899+0000","last_undegraded":"2024-04-24T23:00:32.020899+0000","last_fullsized":"2024-04-24T23:00:32.020899+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.019540+0000","last_change":"2024-04-24T23:00:32.019540+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.019540+0000","last_undegraded":"2024-04-24T23:00:32.019540+0000","last_fullsized":"2024-04-24T23:00:32.019540+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.019756+0000","last_change":"2024-04-24T23:00:32.019756+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.019756+0000","last_undegraded":"2024-04-24T23:00:32.019756+0000","last_fullsized":"2024-04-24T23:00:32.019756+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.012654+0000","last_change":"2024-04-24T23:00:32.012654+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.012654+0000","last_undegraded":"2024-04-24T23:00:32.012654+0000","last_fullsized":"2024-04-24T23:00:32.012654+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.009579+0000","last_change":"2024-04-24T23:00:32.009579+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.009579+0000","last_undegraded":"2024-04-24T23:00:32.009579+0000","last_fullsized":"2024-04-24T23:00:32.009579+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.009836+0000","last_change":"2024-04-24T23:00:32.009836+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.009836+0000","last_undegraded":"2024-04-24T23:00:32.009836+0000","last_fullsized":"2024-04-24T23:00:32.009836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.020843+0000","last_change":"2024-04-24T23:00:32.020843+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.020843+0000","last_undegraded":"2024-04-24T23:00:32.020843+0000","last_fullsized":"2024-04-24T23:00:32.020843+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.015012+0000","last_change":"2024-04-24T23:00:32.015012+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.015012+0000","last_undegraded":"2024-04-24T23:00:32.015012+0000","last_fullsized":"2024-04-24T23:00:32.015012+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.014172+0000","last_change":"2024-04-24T23:00:32.013964+0000","last_active":"2024-04-24T23:00:32.014172+0000","last_peered":"2024-04-24T23:00:32.014172+0000","last_clean":"2024-04-24T23:00:32.014172+0000","last_became_active":"2024-04-24T23:00:32.013661+0000","last_became_peered":"2024-04-24T23:00:32.013661+0000","last_unstale":"2024-04-24T23:00:32.014172+0000","last_undegraded":"2024-04-24T23:00:32.014172+0000","last_fullsized":"2024-04-24T23:00:32.014172+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:34.956602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.012160+0000","last_change":"2024-04-24T23:00:32.012160+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.012160+0000","last_undegraded":"2024-04-24T23:00:32.012160+0000","last_fullsized":"2024-04-24T23:00:32.012160+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.012887+0000","last_change":"2024-04-24T23:00:32.012770+0000","last_active":"2024-04-24T23:00:32.012887+0000","last_peered":"2024-04-24T23:00:32.012887+0000","last_clean":"2024-04-24T23:00:32.012887+0000","last_became_active":"2024-04-24T23:00:32.012092+0000","last_became_peered":"2024-04-24T23:00:32.012092+0000","last_unstale":"2024-04-24T23:00:32.012887+0000","last_undegraded":"2024-04-24T23:00:32.012887+0000","last_fullsized":"2024-04-24T23:00:32.012887+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:03.280807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.009497+0000","last_change":"2024-04-24T23:00:32.009497+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.009497+0000","last_undegraded":"2024-04-24T23:00:32.009497+0000","last_fullsized":"2024-04-24T23:00:32.009497+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.016002+0000","last_change":"2024-04-24T23:00:32.015863+0000","last_active":"2024-04-24T23:00:32.016002+0000","last_peered":"2024-04-24T23:00:32.016002+0000","last_clean":"2024-04-24T23:00:32.016002+0000","last_became_active":"2024-04-24T23:00:32.015616+0000","last_became_peered":"2024-04-24T23:00:32.015616+0000","last_unstale":"2024-04-24T23:00:32.016002+0000","last_undegraded":"2024-04-24T23:00:32.016002+0000","last_fullsized":"2024-04-24T23:00:32.016002+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:29:18.370977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.020793+0000","last_change":"2024-04-24T23:00:32.020793+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.020793+0000","last_undegraded":"2024-04-24T23:00:32.020793+0000","last_fullsized":"2024-04-24T23:00:32.020793+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.010577+0000","last_change":"2024-04-24T23:00:32.010390+0000","last_active":"2024-04-24T23:00:32.010577+0000","last_peered":"2024-04-24T23:00:32.010577+0000","last_clean":"2024-04-24T23:00:32.010577+0000","last_became_active":"2024-04-24T23:00:32.010156+0000","last_became_peered":"2024-04-24T23:00:32.010156+0000","last_unstale":"2024-04-24T23:00:32.010577+0000","last_undegraded":"2024-04-24T23:00:32.010577+0000","last_fullsized":"2024-04-24T23:00:32.010577+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:38:29.668465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.019446+0000","last_change":"2024-04-24T23:00:32.019446+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.019446+0000","last_undegraded":"2024-04-24T23:00:32.019446+0000","last_fullsized":"2024-04-24T23:00:32.019446+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.012416+0000","last_change":"2024-04-24T23:00:32.012144+0000","last_active":"2024-04-24T23:00:32.012416+0000","last_peered":"2024-04-24T23:00:32.012416+0000","last_clean":"2024-04-24T23:00:32.012416+0000","last_became_active":"2024-04-24T23:00:32.011876+0000","last_became_peered":"2024-04-24T23:00:32.011876+0000","last_unstale":"2024-04-24T23:00:32.012416+0000","last_undegraded":"2024-04-24T23:00:32.012416+0000","last_fullsized":"2024-04-24T23:00:32.012416+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:25:17.401466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.012251+0000","last_change":"2024-04-24T23:00:32.012251+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.012251+0000","last_undegraded":"2024-04-24T23:00:32.012251+0000","last_fullsized":"2024-04-24T23:00:32.012251+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.015221+0000","last_change":"2024-04-24T23:00:32.015060+0000","last_active":"2024-04-24T23:00:32.015221+0000","last_peered":"2024-04-24T23:00:32.015221+0000","last_clean":"2024-04-24T23:00:32.015221+0000","last_became_active":"2024-04-24T23:00:32.013636+0000","last_became_peered":"2024-04-24T23:00:32.013636+0000","last_unstale":"2024-04-24T23:00:32.015221+0000","last_undegraded":"2024-04-24T23:00:32.015221+0000","last_fullsized":"2024-04-24T23:00:32.015221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:57:18.083931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.014094+0000","last_change":"2024-04-24T23:00:32.014094+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.014094+0000","last_undegraded":"2024-04-24T23:00:32.014094+0000","last_fullsized":"2024-04-24T23:00:32.014094+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.019078+0000","last_change":"2024-04-24T23:00:32.018961+0000","last_active":"2024-04-24T23:00:32.019078+0000","last_peered":"2024-04-24T23:00:32.019078+0000","last_clean":"2024-04-24T23:00:32.019078+0000","last_became_active":"2024-04-24T23:00:32.018718+0000","last_became_peered":"2024-04-24T23:00:32.018718+0000","last_unstale":"2024-04-24T23:00:32.019078+0000","last_undegraded":"2024-04-24T23:00:32.019078+0000","last_fullsized":"2024-04-24T23:00:32.019078+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:42:02.507492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.021039+0000","last_change":"2024-04-24T23:00:32.021039+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.021039+0000","last_undegraded":"2024-04-24T23:00:32.021039+0000","last_fullsized":"2024-04-24T23:00:32.021039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.016759+0000","last_change":"2024-04-24T23:00:32.016634+0000","last_active":"2024-04-24T23:00:32.016759+0000","last_peered":"2024-04-24T23:00:32.016759+0000","last_clean":"2024-04-24T23:00:32.016759+0000","last_became_active":"2024-04-24T23:00:32.015425+0000","last_became_peered":"2024-04-24T23:00:32.015425+0000","last_unstale":"2024-04-24T23:00:32.016759+0000","last_undegraded":"2024-04-24T23:00:32.016759+0000","last_fullsized":"2024-04-24T23:00:32.016759+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:32:23.611062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.012408+0000","last_change":"2024-04-24T23:00:32.012408+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.012408+0000","last_undegraded":"2024-04-24T23:00:32.012408+0000","last_fullsized":"2024-04-24T23:00:32.012408+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.017952+0000","last_change":"2024-04-24T23:00:32.017822+0000","last_active":"2024-04-24T23:00:32.017952+0000","last_peered":"2024-04-24T23:00:32.017952+0000","last_clean":"2024-04-24T23:00:32.017952+0000","last_became_active":"2024-04-24T23:00:32.017548+0000","last_became_peered":"2024-04-24T23:00:32.017548+0000","last_unstale":"2024-04-24T23:00:32.017952+0000","last_undegraded":"2024-04-24T23:00:32.017952+0000","last_fullsized":"2024-04-24T23:00:32.017952+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:18:08.921860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.015793+0000","last_change":"2024-04-24T23:00:32.015793+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.015793+0000","last_undegraded":"2024-04-24T23:00:32.015793+0000","last_fullsized":"2024-04-24T23:00:32.015793+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.016524+0000","last_change":"2024-04-24T23:00:32.016337+0000","last_active":"2024-04-24T23:00:32.016524+0000","last_peered":"2024-04-24T23:00:32.016524+0000","last_clean":"2024-04-24T23:00:32.016524+0000","last_became_active":"2024-04-24T23:00:32.015833+0000","last_became_peered":"2024-04-24T23:00:32.015833+0000","last_unstale":"2024-04-24T23:00:32.016524+0000","last_undegraded":"2024-04-24T23:00:32.016524+0000","last_fullsized":"2024-04-24T23:00:32.016524+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:00:47.017154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.020618+0000","last_change":"2024-04-24T23:00:32.020618+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.020618+0000","last_undegraded":"2024-04-24T23:00:32.020618+0000","last_fullsized":"2024-04-24T23:00:32.020618+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.017321+0000","last_change":"2024-04-24T23:00:32.017203+0000","last_active":"2024-04-24T23:00:32.017321+0000","last_peered":"2024-04-24T23:00:32.017321+0000","last_clean":"2024-04-24T23:00:32.017321+0000","last_became_active":"2024-04-24T23:00:32.016743+0000","last_became_peered":"2024-04-24T23:00:32.016743+0000","last_unstale":"2024-04-24T23:00:32.017321+0000","last_undegraded":"2024-04-24T23:00:32.017321+0000","last_fullsized":"2024-04-24T23:00:32.017321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:30:49.169292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.019128+0000","last_change":"2024-04-24T23:00:32.019128+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.019128+0000","last_undegraded":"2024-04-24T23:00:32.019128+0000","last_fullsized":"2024-04-24T23:00:32.019128+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.014710+0000","last_change":"2024-04-24T23:00:32.014044+0000","last_active":"2024-04-24T23:00:32.014710+0000","last_peered":"2024-04-24T23:00:32.014710+0000","last_clean":"2024-04-24T23:00:32.014710+0000","last_became_active":"2024-04-24T23:00:32.013781+0000","last_became_peered":"2024-04-24T23:00:32.013781+0000","last_unstale":"2024-04-24T23:00:32.014710+0000","last_undegraded":"2024-04-24T23:00:32.014710+0000","last_fullsized":"2024-04-24T23:00:32.014710+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:24:27.447216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.015479+0000","last_change":"2024-04-24T23:00:32.015479+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.015479+0000","last_undegraded":"2024-04-24T23:00:32.015479+0000","last_fullsized":"2024-04-24T23:00:32.015479+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.019354+0000","last_change":"2024-04-24T23:00:32.019200+0000","last_active":"2024-04-24T23:00:32.019354+0000","last_peered":"2024-04-24T23:00:32.019354+0000","last_clean":"2024-04-24T23:00:32.019354+0000","last_became_active":"2024-04-24T23:00:32.018935+0000","last_became_peered":"2024-04-24T23:00:32.018935+0000","last_unstale":"2024-04-24T23:00:32.019354+0000","last_undegraded":"2024-04-24T23:00:32.019354+0000","last_fullsized":"2024-04-24T23:00:32.019354+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:13.644308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.014218+0000","last_change":"2024-04-24T23:00:32.014218+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.014218+0000","last_undegraded":"2024-04-24T23:00:32.014218+0000","last_fullsized":"2024-04-24T23:00:32.014218+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.018213+0000","last_change":"2024-04-24T23:00:32.018091+0000","last_active":"2024-04-24T23:00:32.018213+0000","last_peered":"2024-04-24T23:00:32.018213+0000","last_clean":"2024-04-24T23:00:32.018213+0000","last_became_active":"2024-04-24T23:00:32.017828+0000","last_became_peered":"2024-04-24T23:00:32.017828+0000","last_unstale":"2024-04-24T23:00:32.018213+0000","last_undegraded":"2024-04-24T23:00:32.018213+0000","last_fullsized":"2024-04-24T23:00:32.018213+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:29:21.830443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.014085+0000","last_change":"2024-04-24T23:00:32.014085+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.014085+0000","last_undegraded":"2024-04-24T23:00:32.014085+0000","last_fullsized":"2024-04-24T23:00:32.014085+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.015462+0000","last_change":"2024-04-24T23:00:32.015309+0000","last_active":"2024-04-24T23:00:32.015462+0000","last_peered":"2024-04-24T23:00:32.015462+0000","last_clean":"2024-04-24T23:00:32.015462+0000","last_became_active":"2024-04-24T23:00:32.014784+0000","last_became_peered":"2024-04-24T23:00:32.014784+0000","last_unstale":"2024-04-24T23:00:32.015462+0000","last_undegraded":"2024-04-24T23:00:32.015462+0000","last_fullsized":"2024-04-24T23:00:32.015462+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:44:08.784625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.010204+0000","last_change":"2024-04-24T23:00:32.010204+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.010204+0000","last_undegraded":"2024-04-24T23:00:32.010204+0000","last_fullsized":"2024-04-24T23:00:32.010204+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.018350+0000","last_change":"2024-04-24T23:00:32.018235+0000","last_active":"2024-04-24T23:00:32.018350+0000","last_peered":"2024-04-24T23:00:32.018350+0000","last_clean":"2024-04-24T23:00:32.018350+0000","last_became_active":"2024-04-24T23:00:32.017933+0000","last_became_peered":"2024-04-24T23:00:32.017933+0000","last_unstale":"2024-04-24T23:00:32.018350+0000","last_undegraded":"2024-04-24T23:00:32.018350+0000","last_fullsized":"2024-04-24T23:00:32.018350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:30:30.768770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.013648+0000","last_change":"2024-04-24T23:00:32.013648+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.013648+0000","last_undegraded":"2024-04-24T23:00:32.013648+0000","last_fullsized":"2024-04-24T23:00:32.013648+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.015776+0000","last_change":"2024-04-24T23:00:32.015665+0000","last_active":"2024-04-24T23:00:32.015776+0000","last_peered":"2024-04-24T23:00:32.015776+0000","last_clean":"2024-04-24T23:00:32.015776+0000","last_became_active":"2024-04-24T23:00:32.015414+0000","last_became_peered":"2024-04-24T23:00:32.015414+0000","last_unstale":"2024-04-24T23:00:32.015776+0000","last_undegraded":"2024-04-24T23:00:32.015776+0000","last_fullsized":"2024-04-24T23:00:32.015776+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:34:48.498696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.014550+0000","last_change":"2024-04-24T23:00:32.014550+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.014550+0000","last_undegraded":"2024-04-24T23:00:32.014550+0000","last_fullsized":"2024-04-24T23:00:32.014550+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.016899+0000","last_change":"2024-04-24T23:00:32.016781+0000","last_active":"2024-04-24T23:00:32.016899+0000","last_peered":"2024-04-24T23:00:32.016899+0000","last_clean":"2024-04-24T23:00:32.016899+0000","last_became_active":"2024-04-24T23:00:32.016515+0000","last_became_peered":"2024-04-24T23:00:32.016515+0000","last_unstale":"2024-04-24T23:00:32.016899+0000","last_undegraded":"2024-04-24T23:00:32.016899+0000","last_fullsized":"2024-04-24T23:00:32.016899+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:57:27.039125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.009520+0000","last_change":"2024-04-24T23:00:32.009520+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.009520+0000","last_undegraded":"2024-04-24T23:00:32.009520+0000","last_fullsized":"2024-04-24T23:00:32.009520+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.019478+0000","last_change":"2024-04-24T23:00:32.019334+0000","last_active":"2024-04-24T23:00:32.019478+0000","last_peered":"2024-04-24T23:00:32.019478+0000","last_clean":"2024-04-24T23:00:32.019478+0000","last_became_active":"2024-04-24T23:00:32.019103+0000","last_became_peered":"2024-04-24T23:00:32.019103+0000","last_unstale":"2024-04-24T23:00:32.019478+0000","last_undegraded":"2024-04-24T23:00:32.019478+0000","last_fullsized":"2024-04-24T23:00:32.019478+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:08:14.771602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.015184+0000","last_change":"2024-04-24T23:00:32.015184+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.015184+0000","last_undegraded":"2024-04-24T23:00:32.015184+0000","last_fullsized":"2024-04-24T23:00:32.015184+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.015147+0000","last_change":"2024-04-24T23:00:32.015023+0000","last_active":"2024-04-24T23:00:32.015147+0000","last_peered":"2024-04-24T23:00:32.015147+0000","last_clean":"2024-04-24T23:00:32.015147+0000","last_became_active":"2024-04-24T23:00:32.014455+0000","last_became_peered":"2024-04-24T23:00:32.014455+0000","last_unstale":"2024-04-24T23:00:32.015147+0000","last_undegraded":"2024-04-24T23:00:32.015147+0000","last_fullsized":"2024-04-24T23:00:32.015147+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:16.368500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.011378+0000","last_change":"2024-04-24T23:00:32.011378+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.011378+0000","last_undegraded":"2024-04-24T23:00:32.011378+0000","last_fullsized":"2024-04-24T23:00:32.011378+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.019875+0000","last_change":"2024-04-24T23:00:32.019708+0000","last_active":"2024-04-24T23:00:32.019875+0000","last_peered":"2024-04-24T23:00:32.019875+0000","last_clean":"2024-04-24T23:00:32.019875+0000","last_became_active":"2024-04-24T23:00:32.019473+0000","last_became_peered":"2024-04-24T23:00:32.019473+0000","last_unstale":"2024-04-24T23:00:32.019875+0000","last_undegraded":"2024-04-24T23:00:32.019875+0000","last_fullsized":"2024-04-24T23:00:32.019875+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:57:15.174813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.009609+0000","last_change":"2024-04-24T23:00:32.009609+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.009609+0000","last_undegraded":"2024-04-24T23:00:32.009609+0000","last_fullsized":"2024-04-24T23:00:32.009609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.016029+0000","last_change":"2024-04-24T23:00:32.015397+0000","last_active":"2024-04-24T23:00:32.016029+0000","last_peered":"2024-04-24T23:00:32.016029+0000","last_clean":"2024-04-24T23:00:32.016029+0000","last_became_active":"2024-04-24T23:00:32.015154+0000","last_became_peered":"2024-04-24T23:00:32.015154+0000","last_unstale":"2024-04-24T23:00:32.016029+0000","last_undegraded":"2024-04-24T23:00:32.016029+0000","last_fullsized":"2024-04-24T23:00:32.016029+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:04:05.572144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.020075+0000","last_change":"2024-04-24T23:00:32.020075+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.020075+0000","last_undegraded":"2024-04-24T23:00:32.020075+0000","last_fullsized":"2024-04-24T23:00:32.020075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.018003+0000","last_change":"2024-04-24T23:00:32.017383+0000","last_active":"2024-04-24T23:00:32.018003+0000","last_peered":"2024-04-24T23:00:32.018003+0000","last_clean":"2024-04-24T23:00:32.018003+0000","last_became_active":"2024-04-24T23:00:32.016962+0000","last_became_peered":"2024-04-24T23:00:32.016962+0000","last_unstale":"2024-04-24T23:00:32.018003+0000","last_undegraded":"2024-04-24T23:00:32.018003+0000","last_fullsized":"2024-04-24T23:00:32.018003+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:44:38.000055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.020501+0000","last_change":"2024-04-24T23:00:32.020501+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.020501+0000","last_undegraded":"2024-04-24T23:00:32.020501+0000","last_fullsized":"2024-04-24T23:00:32.020501+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.012229+0000","last_change":"2024-04-24T23:00:32.011951+0000","last_active":"2024-04-24T23:00:32.012229+0000","last_peered":"2024-04-24T23:00:32.012229+0000","last_clean":"2024-04-24T23:00:32.012229+0000","last_became_active":"2024-04-24T23:00:32.011483+0000","last_became_peered":"2024-04-24T23:00:32.011483+0000","last_unstale":"2024-04-24T23:00:32.012229+0000","last_undegraded":"2024-04-24T23:00:32.012229+0000","last_fullsized":"2024-04-24T23:00:32.012229+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:05.541366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.019204+0000","last_change":"2024-04-24T23:00:32.019204+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.019204+0000","last_undegraded":"2024-04-24T23:00:32.019204+0000","last_fullsized":"2024-04-24T23:00:32.019204+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.017120+0000","last_change":"2024-04-24T23:00:32.016280+0000","last_active":"2024-04-24T23:00:32.017120+0000","last_peered":"2024-04-24T23:00:32.017120+0000","last_clean":"2024-04-24T23:00:32.017120+0000","last_became_active":"2024-04-24T23:00:32.016057+0000","last_became_peered":"2024-04-24T23:00:32.016057+0000","last_unstale":"2024-04-24T23:00:32.017120+0000","last_undegraded":"2024-04-24T23:00:32.017120+0000","last_fullsized":"2024-04-24T23:00:32.017120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:25:31.020426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.013723+0000","last_change":"2024-04-24T23:00:32.013723+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.013723+0000","last_undegraded":"2024-04-24T23:00:32.013723+0000","last_fullsized":"2024-04-24T23:00:32.013723+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.016054+0000","last_change":"2024-04-24T23:00:32.015930+0000","last_active":"2024-04-24T23:00:32.016054+0000","last_peered":"2024-04-24T23:00:32.016054+0000","last_clean":"2024-04-24T23:00:32.016054+0000","last_became_active":"2024-04-24T23:00:32.015671+0000","last_became_peered":"2024-04-24T23:00:32.015671+0000","last_unstale":"2024-04-24T23:00:32.016054+0000","last_undegraded":"2024-04-24T23:00:32.016054+0000","last_fullsized":"2024-04-24T23:00:32.016054+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:47.216626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.020647+0000","last_change":"2024-04-24T23:00:32.020647+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.020647+0000","last_undegraded":"2024-04-24T23:00:32.020647+0000","last_fullsized":"2024-04-24T23:00:32.020647+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.020210+0000","last_change":"2024-04-24T23:00:32.020060+0000","last_active":"2024-04-24T23:00:32.020210+0000","last_peered":"2024-04-24T23:00:32.020210+0000","last_clean":"2024-04-24T23:00:32.020210+0000","last_became_active":"2024-04-24T23:00:32.019783+0000","last_became_peered":"2024-04-24T23:00:32.019783+0000","last_unstale":"2024-04-24T23:00:32.020210+0000","last_undegraded":"2024-04-24T23:00:32.020210+0000","last_fullsized":"2024-04-24T23:00:32.020210+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:24:44.692764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.011948+0000","last_change":"2024-04-24T23:00:32.011948+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.011948+0000","last_undegraded":"2024-04-24T23:00:32.011948+0000","last_fullsized":"2024-04-24T23:00:32.011948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.020335+0000","last_change":"2024-04-24T23:00:32.020217+0000","last_active":"2024-04-24T23:00:32.020335+0000","last_peered":"2024-04-24T23:00:32.020335+0000","last_clean":"2024-04-24T23:00:32.020335+0000","last_became_active":"2024-04-24T23:00:32.019728+0000","last_became_peered":"2024-04-24T23:00:32.019728+0000","last_unstale":"2024-04-24T23:00:32.020335+0000","last_undegraded":"2024-04-24T23:00:32.020335+0000","last_fullsized":"2024-04-24T23:00:32.020335+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:33:53.601267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.018940+0000","last_change":"2024-04-24T23:00:32.018940+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.018940+0000","last_undegraded":"2024-04-24T23:00:32.018940+0000","last_fullsized":"2024-04-24T23:00:32.018940+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.018841+0000","last_change":"2024-04-24T23:00:32.018725+0000","last_active":"2024-04-24T23:00:32.018841+0000","last_peered":"2024-04-24T23:00:32.018841+0000","last_clean":"2024-04-24T23:00:32.018841+0000","last_became_active":"2024-04-24T23:00:32.018472+0000","last_became_peered":"2024-04-24T23:00:32.018472+0000","last_unstale":"2024-04-24T23:00:32.018841+0000","last_undegraded":"2024-04-24T23:00:32.018841+0000","last_fullsized":"2024-04-24T23:00:32.018841+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:58:07.274166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.011506+0000","last_change":"2024-04-24T23:00:32.011506+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.011506+0000","last_undegraded":"2024-04-24T23:00:32.011506+0000","last_fullsized":"2024-04-24T23:00:32.011506+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.012098+0000","last_change":"2024-04-24T23:00:32.011954+0000","last_active":"2024-04-24T23:00:32.012098+0000","last_peered":"2024-04-24T23:00:32.012098+0000","last_clean":"2024-04-24T23:00:32.012098+0000","last_became_active":"2024-04-24T23:00:32.011189+0000","last_became_peered":"2024-04-24T23:00:32.011189+0000","last_unstale":"2024-04-24T23:00:32.012098+0000","last_undegraded":"2024-04-24T23:00:32.012098+0000","last_fullsized":"2024-04-24T23:00:32.012098+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:10:23.000844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.009799+0000","last_change":"2024-04-24T23:00:32.009799+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.009799+0000","last_undegraded":"2024-04-24T23:00:32.009799+0000","last_fullsized":"2024-04-24T23:00:32.009799+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.020146+0000","last_change":"2024-04-24T23:00:32.019965+0000","last_active":"2024-04-24T23:00:32.020146+0000","last_peered":"2024-04-24T23:00:32.020146+0000","last_clean":"2024-04-24T23:00:32.020146+0000","last_became_active":"2024-04-24T23:00:32.019691+0000","last_became_peered":"2024-04-24T23:00:32.019691+0000","last_unstale":"2024-04-24T23:00:32.020146+0000","last_undegraded":"2024-04-24T23:00:32.020146+0000","last_fullsized":"2024-04-24T23:00:32.020146+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:32:54.766031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T23:00:32.019001+0000","last_change":"2024-04-24T23:00:32.019001+0000","last_active":"2024-04-24T23:00:32.002613+0000","last_peered":"2024-04-24T23:00:32.002613+0000","last_clean":"2024-04-24T23:00:32.002613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:00:32.019001+0000","last_undegraded":"2024-04-24T23:00:32.019001+0000","last_fullsized":"2024-04-24T23:00:32.019001+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.016843+0000","last_change":"2024-04-24T23:00:32.016499+0000","last_active":"2024-04-24T23:00:32.016843+0000","last_peered":"2024-04-24T23:00:32.016843+0000","last_clean":"2024-04-24T23:00:32.016843+0000","last_became_active":"2024-04-24T23:00:32.016247+0000","last_became_peered":"2024-04-24T23:00:32.016247+0000","last_unstale":"2024-04-24T23:00:32.016843+0000","last_undegraded":"2024-04-24T23:00:32.016843+0000","last_fullsized":"2024-04-24T23:00:32.016843+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:07:25.745306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":209,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T23:00:32.011684+0000","last_change":"2024-04-24T22:58:55.138810+0000","last_active":"2024-04-24T23:00:32.011684+0000","last_peered":"2024-04-24T23:00:32.011684+0000","last_clean":"2024-04-24T23:00:32.011684+0000","last_became_active":"2024-04-24T22:58:55.138627+0000","last_became_peered":"2024-04-24T22:58:55.138627+0000","last_unstale":"2024-04-24T23:00:32.011684+0000","last_undegraded":"2024-04-24T23:00:32.011684+0000","last_fullsized":"2024-04-24T23:00:32.011684+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_clean_scrub_stamp":"2024-04-24T22:58:54.127235+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:24.010002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":189,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195122,"statfs":{"total":200000000,"available":199805687,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":189,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195122,"statfs":{"total":200000000,"available":199805687,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":766,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194546,"statfs":{"total":200000000,"available":199215319,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":195,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195116,"statfs":{"total":200000000,"available":199799412,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":189,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195122,"statfs":{"total":200000000,"available":199805687,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":189,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195122,"statfs":{"total":200000000,"available":199805687,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":772,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194540,"statfs":{"total":200000000,"available":199209044,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":189,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195122,"statfs":{"total":200000000,"available":199805687,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-24T23:00:36.742 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3f is not active+clean 2024-04-24T23:00:36.742 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.015384+0000', 'last_change': '2024-04-24T23:00:32.015384+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.015384+0000', 'last_undegraded': '2024-04-24T23:00:32.015384+0000', 'last_fullsized': '2024-04-24T23:00:32.015384+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-24T23:00:36.743 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3e is not active+clean 2024-04-24T23:00:36.743 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.012040+0000', 'last_change': '2024-04-24T23:00:32.012040+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.012040+0000', 'last_undegraded': '2024-04-24T23:00:32.012040+0000', 'last_fullsized': '2024-04-24T23:00:32.012040+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-24T23:00:36.743 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3d is not active+clean 2024-04-24T23:00:36.743 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.013243+0000', 'last_change': '2024-04-24T23:00:32.013243+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.013243+0000', 'last_undegraded': '2024-04-24T23:00:32.013243+0000', 'last_fullsized': '2024-04-24T23:00:32.013243+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-24T23:00:36.743 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3c is not active+clean 2024-04-24T23:00:36.743 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.021880+0000', 'last_change': '2024-04-24T23:00:32.021880+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.021880+0000', 'last_undegraded': '2024-04-24T23:00:32.021880+0000', 'last_fullsized': '2024-04-24T23:00:32.021880+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-24T23:00:36.743 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3b is not active+clean 2024-04-24T23:00:36.743 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.013633+0000', 'last_change': '2024-04-24T23:00:32.013633+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.013633+0000', 'last_undegraded': '2024-04-24T23:00:32.013633+0000', 'last_fullsized': '2024-04-24T23:00:32.013633+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-24T23:00:36.743 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3a is not active+clean 2024-04-24T23:00:36.743 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.010238+0000', 'last_change': '2024-04-24T23:00:32.010238+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.010238+0000', 'last_undegraded': '2024-04-24T23:00:32.010238+0000', 'last_fullsized': '2024-04-24T23:00:32.010238+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-24T23:00:36.743 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.39 is not active+clean 2024-04-24T23:00:36.743 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.019822+0000', 'last_change': '2024-04-24T23:00:32.019822+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.019822+0000', 'last_undegraded': '2024-04-24T23:00:32.019822+0000', 'last_fullsized': '2024-04-24T23:00:32.019822+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-24T23:00:36.744 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.38 is not active+clean 2024-04-24T23:00:36.744 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.022253+0000', 'last_change': '2024-04-24T23:00:32.022253+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.022253+0000', 'last_undegraded': '2024-04-24T23:00:32.022253+0000', 'last_fullsized': '2024-04-24T23:00:32.022253+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-24T23:00:36.744 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.37 is not active+clean 2024-04-24T23:00:36.744 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.009087+0000', 'last_change': '2024-04-24T23:00:32.009087+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.009087+0000', 'last_undegraded': '2024-04-24T23:00:32.009087+0000', 'last_fullsized': '2024-04-24T23:00:32.009087+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-24T23:00:36.744 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.36 is not active+clean 2024-04-24T23:00:36.744 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.014404+0000', 'last_change': '2024-04-24T23:00:32.014404+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.014404+0000', 'last_undegraded': '2024-04-24T23:00:32.014404+0000', 'last_fullsized': '2024-04-24T23:00:32.014404+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-24T23:00:36.744 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.35 is not active+clean 2024-04-24T23:00:36.744 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.022474+0000', 'last_change': '2024-04-24T23:00:32.022474+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.022474+0000', 'last_undegraded': '2024-04-24T23:00:32.022474+0000', 'last_fullsized': '2024-04-24T23:00:32.022474+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-24T23:00:36.744 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.34 is not active+clean 2024-04-24T23:00:36.744 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.011480+0000', 'last_change': '2024-04-24T23:00:32.011480+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.011480+0000', 'last_undegraded': '2024-04-24T23:00:32.011480+0000', 'last_fullsized': '2024-04-24T23:00:32.011480+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-24T23:00:36.744 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.33 is not active+clean 2024-04-24T23:00:36.745 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.010000+0000', 'last_change': '2024-04-24T23:00:32.010000+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.010000+0000', 'last_undegraded': '2024-04-24T23:00:32.010000+0000', 'last_fullsized': '2024-04-24T23:00:32.010000+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-24T23:00:36.745 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.32 is not active+clean 2024-04-24T23:00:36.745 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.021494+0000', 'last_change': '2024-04-24T23:00:32.021494+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.021494+0000', 'last_undegraded': '2024-04-24T23:00:32.021494+0000', 'last_fullsized': '2024-04-24T23:00:32.021494+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-24T23:00:36.745 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.31 is not active+clean 2024-04-24T23:00:36.745 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.021499+0000', 'last_change': '2024-04-24T23:00:32.021499+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.021499+0000', 'last_undegraded': '2024-04-24T23:00:32.021499+0000', 'last_fullsized': '2024-04-24T23:00:32.021499+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-24T23:00:36.745 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.30 is not active+clean 2024-04-24T23:00:36.745 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.015039+0000', 'last_change': '2024-04-24T23:00:32.015039+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.015039+0000', 'last_undegraded': '2024-04-24T23:00:32.015039+0000', 'last_fullsized': '2024-04-24T23:00:32.015039+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-24T23:00:36.745 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2f is not active+clean 2024-04-24T23:00:36.745 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.011162+0000', 'last_change': '2024-04-24T23:00:32.011162+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.011162+0000', 'last_undegraded': '2024-04-24T23:00:32.011162+0000', 'last_fullsized': '2024-04-24T23:00:32.011162+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-24T23:00:36.746 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2e is not active+clean 2024-04-24T23:00:36.746 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.012933+0000', 'last_change': '2024-04-24T23:00:32.012933+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.012933+0000', 'last_undegraded': '2024-04-24T23:00:32.012933+0000', 'last_fullsized': '2024-04-24T23:00:32.012933+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-24T23:00:36.746 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2d is not active+clean 2024-04-24T23:00:36.746 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.011970+0000', 'last_change': '2024-04-24T23:00:32.011970+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.011970+0000', 'last_undegraded': '2024-04-24T23:00:32.011970+0000', 'last_fullsized': '2024-04-24T23:00:32.011970+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-24T23:00:36.746 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2c is not active+clean 2024-04-24T23:00:36.746 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.011779+0000', 'last_change': '2024-04-24T23:00:32.011779+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.011779+0000', 'last_undegraded': '2024-04-24T23:00:32.011779+0000', 'last_fullsized': '2024-04-24T23:00:32.011779+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-24T23:00:36.746 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2b is not active+clean 2024-04-24T23:00:36.746 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.021919+0000', 'last_change': '2024-04-24T23:00:32.021919+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.021919+0000', 'last_undegraded': '2024-04-24T23:00:32.021919+0000', 'last_fullsized': '2024-04-24T23:00:32.021919+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-24T23:00:36.747 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2a is not active+clean 2024-04-24T23:00:36.747 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.013643+0000', 'last_change': '2024-04-24T23:00:32.013643+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.013643+0000', 'last_undegraded': '2024-04-24T23:00:32.013643+0000', 'last_fullsized': '2024-04-24T23:00:32.013643+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-24T23:00:36.747 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.29 is not active+clean 2024-04-24T23:00:36.747 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.020052+0000', 'last_change': '2024-04-24T23:00:32.020052+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.020052+0000', 'last_undegraded': '2024-04-24T23:00:32.020052+0000', 'last_fullsized': '2024-04-24T23:00:32.020052+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-24T23:00:36.747 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.28 is not active+clean 2024-04-24T23:00:36.747 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.010783+0000', 'last_change': '2024-04-24T23:00:32.010783+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.010783+0000', 'last_undegraded': '2024-04-24T23:00:32.010783+0000', 'last_fullsized': '2024-04-24T23:00:32.010783+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-24T23:00:36.747 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.27 is not active+clean 2024-04-24T23:00:36.747 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.013276+0000', 'last_change': '2024-04-24T23:00:32.013276+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.013276+0000', 'last_undegraded': '2024-04-24T23:00:32.013276+0000', 'last_fullsized': '2024-04-24T23:00:32.013276+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-24T23:00:36.747 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.26 is not active+clean 2024-04-24T23:00:36.747 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.020899+0000', 'last_change': '2024-04-24T23:00:32.020899+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.020899+0000', 'last_undegraded': '2024-04-24T23:00:32.020899+0000', 'last_fullsized': '2024-04-24T23:00:32.020899+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-24T23:00:36.747 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.25 is not active+clean 2024-04-24T23:00:36.748 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.019540+0000', 'last_change': '2024-04-24T23:00:32.019540+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.019540+0000', 'last_undegraded': '2024-04-24T23:00:32.019540+0000', 'last_fullsized': '2024-04-24T23:00:32.019540+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-24T23:00:36.748 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.24 is not active+clean 2024-04-24T23:00:36.748 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.019756+0000', 'last_change': '2024-04-24T23:00:32.019756+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.019756+0000', 'last_undegraded': '2024-04-24T23:00:32.019756+0000', 'last_fullsized': '2024-04-24T23:00:32.019756+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-24T23:00:36.748 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.23 is not active+clean 2024-04-24T23:00:36.748 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.012654+0000', 'last_change': '2024-04-24T23:00:32.012654+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.012654+0000', 'last_undegraded': '2024-04-24T23:00:32.012654+0000', 'last_fullsized': '2024-04-24T23:00:32.012654+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-24T23:00:36.749 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.22 is not active+clean 2024-04-24T23:00:36.749 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.009579+0000', 'last_change': '2024-04-24T23:00:32.009579+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.009579+0000', 'last_undegraded': '2024-04-24T23:00:32.009579+0000', 'last_fullsized': '2024-04-24T23:00:32.009579+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-24T23:00:36.749 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.21 is not active+clean 2024-04-24T23:00:36.749 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.009836+0000', 'last_change': '2024-04-24T23:00:32.009836+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.009836+0000', 'last_undegraded': '2024-04-24T23:00:32.009836+0000', 'last_fullsized': '2024-04-24T23:00:32.009836+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-24T23:00:36.749 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.20 is not active+clean 2024-04-24T23:00:36.749 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.020843+0000', 'last_change': '2024-04-24T23:00:32.020843+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.020843+0000', 'last_undegraded': '2024-04-24T23:00:32.020843+0000', 'last_fullsized': '2024-04-24T23:00:32.020843+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-24T23:00:36.749 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1e is not active+clean 2024-04-24T23:00:36.749 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.015012+0000', 'last_change': '2024-04-24T23:00:32.015012+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.015012+0000', 'last_undegraded': '2024-04-24T23:00:32.015012+0000', 'last_fullsized': '2024-04-24T23:00:32.015012+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-24T23:00:36.749 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1f is not active+clean 2024-04-24T23:00:36.749 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.012160+0000', 'last_change': '2024-04-24T23:00:32.012160+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.012160+0000', 'last_undegraded': '2024-04-24T23:00:32.012160+0000', 'last_fullsized': '2024-04-24T23:00:32.012160+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-24T23:00:36.749 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1c is not active+clean 2024-04-24T23:00:36.749 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.009497+0000', 'last_change': '2024-04-24T23:00:32.009497+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.009497+0000', 'last_undegraded': '2024-04-24T23:00:32.009497+0000', 'last_fullsized': '2024-04-24T23:00:32.009497+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-24T23:00:36.750 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1d is not active+clean 2024-04-24T23:00:36.750 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.020793+0000', 'last_change': '2024-04-24T23:00:32.020793+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.020793+0000', 'last_undegraded': '2024-04-24T23:00:32.020793+0000', 'last_fullsized': '2024-04-24T23:00:32.020793+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-24T23:00:36.750 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1a is not active+clean 2024-04-24T23:00:36.750 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.019446+0000', 'last_change': '2024-04-24T23:00:32.019446+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.019446+0000', 'last_undegraded': '2024-04-24T23:00:32.019446+0000', 'last_fullsized': '2024-04-24T23:00:32.019446+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-24T23:00:36.750 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1b is not active+clean 2024-04-24T23:00:36.750 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.012251+0000', 'last_change': '2024-04-24T23:00:32.012251+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.012251+0000', 'last_undegraded': '2024-04-24T23:00:32.012251+0000', 'last_fullsized': '2024-04-24T23:00:32.012251+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-24T23:00:36.750 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.18 is not active+clean 2024-04-24T23:00:36.750 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.014094+0000', 'last_change': '2024-04-24T23:00:32.014094+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.014094+0000', 'last_undegraded': '2024-04-24T23:00:32.014094+0000', 'last_fullsized': '2024-04-24T23:00:32.014094+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-24T23:00:36.750 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.19 is not active+clean 2024-04-24T23:00:36.750 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.021039+0000', 'last_change': '2024-04-24T23:00:32.021039+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.021039+0000', 'last_undegraded': '2024-04-24T23:00:32.021039+0000', 'last_fullsized': '2024-04-24T23:00:32.021039+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-24T23:00:36.750 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.16 is not active+clean 2024-04-24T23:00:36.751 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.012408+0000', 'last_change': '2024-04-24T23:00:32.012408+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.012408+0000', 'last_undegraded': '2024-04-24T23:00:32.012408+0000', 'last_fullsized': '2024-04-24T23:00:32.012408+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-24T23:00:36.751 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.17 is not active+clean 2024-04-24T23:00:36.751 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.015793+0000', 'last_change': '2024-04-24T23:00:32.015793+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.015793+0000', 'last_undegraded': '2024-04-24T23:00:32.015793+0000', 'last_fullsized': '2024-04-24T23:00:32.015793+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-24T23:00:36.751 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.14 is not active+clean 2024-04-24T23:00:36.751 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.020618+0000', 'last_change': '2024-04-24T23:00:32.020618+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.020618+0000', 'last_undegraded': '2024-04-24T23:00:32.020618+0000', 'last_fullsized': '2024-04-24T23:00:32.020618+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-24T23:00:36.751 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.15 is not active+clean 2024-04-24T23:00:36.751 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.019128+0000', 'last_change': '2024-04-24T23:00:32.019128+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.019128+0000', 'last_undegraded': '2024-04-24T23:00:32.019128+0000', 'last_fullsized': '2024-04-24T23:00:32.019128+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-24T23:00:36.751 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.12 is not active+clean 2024-04-24T23:00:36.751 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.015479+0000', 'last_change': '2024-04-24T23:00:32.015479+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.015479+0000', 'last_undegraded': '2024-04-24T23:00:32.015479+0000', 'last_fullsized': '2024-04-24T23:00:32.015479+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-24T23:00:36.751 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.13 is not active+clean 2024-04-24T23:00:36.751 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.014218+0000', 'last_change': '2024-04-24T23:00:32.014218+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.014218+0000', 'last_undegraded': '2024-04-24T23:00:32.014218+0000', 'last_fullsized': '2024-04-24T23:00:32.014218+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-24T23:00:36.751 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.10 is not active+clean 2024-04-24T23:00:36.752 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.014085+0000', 'last_change': '2024-04-24T23:00:32.014085+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.014085+0000', 'last_undegraded': '2024-04-24T23:00:32.014085+0000', 'last_fullsized': '2024-04-24T23:00:32.014085+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-24T23:00:36.752 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.11 is not active+clean 2024-04-24T23:00:36.752 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.010204+0000', 'last_change': '2024-04-24T23:00:32.010204+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.010204+0000', 'last_undegraded': '2024-04-24T23:00:32.010204+0000', 'last_fullsized': '2024-04-24T23:00:32.010204+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-24T23:00:36.752 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.e is not active+clean 2024-04-24T23:00:36.752 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.013648+0000', 'last_change': '2024-04-24T23:00:32.013648+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.013648+0000', 'last_undegraded': '2024-04-24T23:00:32.013648+0000', 'last_fullsized': '2024-04-24T23:00:32.013648+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-24T23:00:36.752 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.f is not active+clean 2024-04-24T23:00:36.752 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.014550+0000', 'last_change': '2024-04-24T23:00:32.014550+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.014550+0000', 'last_undegraded': '2024-04-24T23:00:32.014550+0000', 'last_fullsized': '2024-04-24T23:00:32.014550+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-24T23:00:36.752 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.c is not active+clean 2024-04-24T23:00:36.752 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.009520+0000', 'last_change': '2024-04-24T23:00:32.009520+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.009520+0000', 'last_undegraded': '2024-04-24T23:00:32.009520+0000', 'last_fullsized': '2024-04-24T23:00:32.009520+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-24T23:00:36.752 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.d is not active+clean 2024-04-24T23:00:36.752 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.015184+0000', 'last_change': '2024-04-24T23:00:32.015184+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.015184+0000', 'last_undegraded': '2024-04-24T23:00:32.015184+0000', 'last_fullsized': '2024-04-24T23:00:32.015184+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-24T23:00:36.753 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.a is not active+clean 2024-04-24T23:00:36.753 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.011378+0000', 'last_change': '2024-04-24T23:00:32.011378+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.011378+0000', 'last_undegraded': '2024-04-24T23:00:32.011378+0000', 'last_fullsized': '2024-04-24T23:00:32.011378+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-24T23:00:36.753 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.b is not active+clean 2024-04-24T23:00:36.753 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.009609+0000', 'last_change': '2024-04-24T23:00:32.009609+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.009609+0000', 'last_undegraded': '2024-04-24T23:00:32.009609+0000', 'last_fullsized': '2024-04-24T23:00:32.009609+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-24T23:00:36.753 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.8 is not active+clean 2024-04-24T23:00:36.753 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.020075+0000', 'last_change': '2024-04-24T23:00:32.020075+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.020075+0000', 'last_undegraded': '2024-04-24T23:00:32.020075+0000', 'last_fullsized': '2024-04-24T23:00:32.020075+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-24T23:00:36.753 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.9 is not active+clean 2024-04-24T23:00:36.753 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.020501+0000', 'last_change': '2024-04-24T23:00:32.020501+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.020501+0000', 'last_undegraded': '2024-04-24T23:00:32.020501+0000', 'last_fullsized': '2024-04-24T23:00:32.020501+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-24T23:00:36.753 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.7 is not active+clean 2024-04-24T23:00:36.753 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.019204+0000', 'last_change': '2024-04-24T23:00:32.019204+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.019204+0000', 'last_undegraded': '2024-04-24T23:00:32.019204+0000', 'last_fullsized': '2024-04-24T23:00:32.019204+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-24T23:00:36.753 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.4 is not active+clean 2024-04-24T23:00:36.753 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.013723+0000', 'last_change': '2024-04-24T23:00:32.013723+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.013723+0000', 'last_undegraded': '2024-04-24T23:00:32.013723+0000', 'last_fullsized': '2024-04-24T23:00:32.013723+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-24T23:00:36.754 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.5 is not active+clean 2024-04-24T23:00:36.754 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.020647+0000', 'last_change': '2024-04-24T23:00:32.020647+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.020647+0000', 'last_undegraded': '2024-04-24T23:00:32.020647+0000', 'last_fullsized': '2024-04-24T23:00:32.020647+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-24T23:00:36.754 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2 is not active+clean 2024-04-24T23:00:36.754 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.011948+0000', 'last_change': '2024-04-24T23:00:32.011948+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.011948+0000', 'last_undegraded': '2024-04-24T23:00:32.011948+0000', 'last_fullsized': '2024-04-24T23:00:32.011948+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-24T23:00:36.754 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3 is not active+clean 2024-04-24T23:00:36.754 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.018940+0000', 'last_change': '2024-04-24T23:00:32.018940+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.018940+0000', 'last_undegraded': '2024-04-24T23:00:32.018940+0000', 'last_fullsized': '2024-04-24T23:00:32.018940+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-24T23:00:36.754 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.0 is not active+clean 2024-04-24T23:00:36.754 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.011506+0000', 'last_change': '2024-04-24T23:00:32.011506+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.011506+0000', 'last_undegraded': '2024-04-24T23:00:32.011506+0000', 'last_fullsized': '2024-04-24T23:00:32.011506+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-24T23:00:36.754 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1 is not active+clean 2024-04-24T23:00:36.754 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.009799+0000', 'last_change': '2024-04-24T23:00:32.009799+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.009799+0000', 'last_undegraded': '2024-04-24T23:00:32.009799+0000', 'last_fullsized': '2024-04-24T23:00:32.009799+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-24T23:00:36.754 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.6 is not active+clean 2024-04-24T23:00:36.755 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:00:32.019001+0000', 'last_change': '2024-04-24T23:00:32.019001+0000', 'last_active': '2024-04-24T23:00:32.002613+0000', 'last_peered': '2024-04-24T23:00:32.002613+0000', 'last_clean': '2024-04-24T23:00:32.002613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:00:32.019001+0000', 'last_undegraded': '2024-04-24T23:00:32.019001+0000', 'last_fullsized': '2024-04-24T23:00:32.019001+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'last_clean_scrub_stamp': '2024-04-24T23:00:32.002613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-24T23:00:36.755 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-24T23:00:36.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.878+0000 7f76d062a640 1 Processor -- start 2024-04-24T23:00:36.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.878+0000 7f76d062a640 1 -- start start 2024-04-24T23:00:36.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.879+0000 7f76d062a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f76c815c5b0 0x7f76c80b4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:36.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.879+0000 7f76d062a640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f76c8072cf0 0x7f76c806c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:36.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.879+0000 7f76d062a640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76c815c990 con 0x7f76c8071770 2024-04-24T23:00:36.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.879+0000 7f76d062a640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f76c8072860 con 0x7f76c815c5b0 2024-04-24T23:00:36.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.879+0000 7f76d062a640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f76c8071c50 con 0x7f76c8072cf0 2024-04-24T23:00:36.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.879+0000 7f76ce3c6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f76c815c5b0 0x7f76c80b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:36.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.879+0000 7f76ce3c6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f76c815c5b0 0x7f76c80b4030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46322/0 (socket says 172.21.15.32:46322) 2024-04-24T23:00:36.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.879+0000 7f76ce3c6640 1 -- 172.21.15.32:0/2806916033 learned_addr learned my addr 172.21.15.32:0/2806916033 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:36.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.880+0000 7f76cdbc5640 1 --2- 172.21.15.32:0/2806916033 >> v2:172.21.15.121:3300/0 conn(0x7f76c8072cf0 0x7f76c806c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:36.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.880+0000 7f76ce3c6640 1 -- 172.21.15.32:0/2806916033 >> v2:172.21.15.121:3300/0 conn(0x7f76c8072cf0 msgr2=0x7f76c806c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:36.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.880+0000 7f76ce3c6640 1 --2- 172.21.15.32:0/2806916033 >> v2:172.21.15.121:3300/0 conn(0x7f76c8072cf0 0x7f76c806c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:36.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.880+0000 7f76ce3c6640 1 -- 172.21.15.32:0/2806916033 >> v1:172.21.15.121:6789/0 conn(0x7f76c8071770 legacy=0x7f76c80cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:36.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.880+0000 7f76ce3c6640 1 -- 172.21.15.32:0/2806916033 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76c8071f30 con 0x7f76c815c5b0 2024-04-24T23:00:36.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.880+0000 7f76cdbc5640 1 --2- 172.21.15.32:0/2806916033 >> v2:172.21.15.121:3300/0 conn(0x7f76c8072cf0 0x7f76c806c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:00:36.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.880+0000 7f76ce3c6640 1 --2- 172.21.15.32:0/2806916033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76c815c5b0 0x7f76c80b4030 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f76c8160050 tx=0x7f76c002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dea89d2da52f6e2e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:36.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.881+0000 7f76cd3c4640 1 -- 172.21.15.32:0/2806916033 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c0037070 con 0x7f76c815c5b0 2024-04-24T23:00:36.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.881+0000 7f76cd3c4640 1 -- 172.21.15.32:0/2806916033 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76c0032440 con 0x7f76c815c5b0 2024-04-24T23:00:36.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.881+0000 7f76cd3c4640 1 -- 172.21.15.32:0/2806916033 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c0032810 con 0x7f76c815c5b0 2024-04-24T23:00:36.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.881+0000 7f76d062a640 1 -- 172.21.15.32:0/2806916033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76c815c5b0 msgr2=0x7f76c80b4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:36.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.881+0000 7f76d062a640 1 --2- 172.21.15.32:0/2806916033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76c815c5b0 0x7f76c80b4030 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f76c8160050 tx=0x7f76c002a1e0 comp rx=0 tx=0).stop 2024-04-24T23:00:36.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.881+0000 7f76d062a640 1 -- 172.21.15.32:0/2806916033 shutdown_connections 2024-04-24T23:00:36.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.881+0000 7f76d062a640 1 --2- 172.21.15.32:0/2806916033 >> v2:172.21.15.121:3300/0 conn(0x7f76c8072cf0 0x7f76c806c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:36.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.881+0000 7f76d062a640 1 --2- 172.21.15.32:0/2806916033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76c815c5b0 0x7f76c80b4030 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:36.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.881+0000 7f76d062a640 1 -- 172.21.15.32:0/2806916033 >> 172.21.15.32:0/2806916033 conn(0x7f76c806b110 msgr2=0x7f76c806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:36.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.881+0000 7f76d062a640 1 -- 172.21.15.32:0/2806916033 shutdown_connections 2024-04-24T23:00:36.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.881+0000 7f76d062a640 1 -- 172.21.15.32:0/2806916033 wait complete. 2024-04-24T23:00:36.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.882+0000 7f76d062a640 1 Processor -- start 2024-04-24T23:00:36.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.882+0000 7f76d062a640 1 -- start start 2024-04-24T23:00:36.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.883+0000 7f76d062a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f76c8071770 0x7f76c8146360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:36.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.883+0000 7f76d062a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76c8072cf0 0x7f76c80ae830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:36.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.883+0000 7f76d062a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76c815c5b0 0x7f76c80aed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:36.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.883+0000 7f76d062a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f76c80b4570 con 0x7f76c8072cf0 2024-04-24T23:00:36.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.883+0000 7f76d062a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f76c80724f0 con 0x7f76c815c5b0 2024-04-24T23:00:36.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.883+0000 7f76d062a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f76c80b46f0 con 0x7f76c8071770 2024-04-24T23:00:36.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.883+0000 7f76cdbc5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76c8072cf0 0x7f76c80ae830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:36.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.883+0000 7f76cdbc5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76c8072cf0 0x7f76c80ae830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46332/0 (socket says 172.21.15.32:46332) 2024-04-24T23:00:36.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.883+0000 7f76cdbc5640 1 -- 172.21.15.32:0/2480965722 learned_addr learned my addr 172.21.15.32:0/2480965722 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:36.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.883+0000 7f76cebc7640 1 --2- 172.21.15.32:0/2480965722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76c815c5b0 0x7f76c80aed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:36.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.883+0000 7f76ce3c6640 1 --2- 172.21.15.32:0/2480965722 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f76c8071770 0x7f76c8146360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:36.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.884+0000 7f76cdbc5640 1 -- 172.21.15.32:0/2480965722 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f76c8071770 msgr2=0x7f76c8146360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:36.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.884+0000 7f76cdbc5640 1 --2- 172.21.15.32:0/2480965722 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f76c8071770 0x7f76c8146360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:36.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.884+0000 7f76cdbc5640 1 -- 172.21.15.32:0/2480965722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76c815c5b0 msgr2=0x7f76c80aed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:36.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.884+0000 7f76cdbc5640 1 --2- 172.21.15.32:0/2480965722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76c815c5b0 0x7f76c80aed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:36.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.884+0000 7f76cdbc5640 1 -- 172.21.15.32:0/2480965722 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76c80af2b0 con 0x7f76c8072cf0 2024-04-24T23:00:36.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.884+0000 7f76ce3c6640 1 --2- 172.21.15.32:0/2480965722 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f76c8071770 0x7f76c8146360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:00:36.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.884+0000 7f76cebc7640 1 --2- 172.21.15.32:0/2480965722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76c815c5b0 0x7f76c80aed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:00:36.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.884+0000 7f76cdbc5640 1 --2- 172.21.15.32:0/2480965722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76c8072cf0 0x7f76c80ae830 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f76b800e7e0 tx=0x7f76b800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:36.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.884+0000 7f76beffd640 1 -- 172.21.15.32:0/2480965722 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b800f040 con 0x7f76c8072cf0 2024-04-24T23:00:36.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.884+0000 7f76beffd640 1 -- 172.21.15.32:0/2480965722 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76b8004030 con 0x7f76c8072cf0 2024-04-24T23:00:36.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.884+0000 7f76d062a640 1 -- 172.21.15.32:0/2480965722 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76c80aa4a0 con 0x7f76c8072cf0 2024-04-24T23:00:36.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.884+0000 7f76beffd640 1 -- 172.21.15.32:0/2480965722 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b8004400 con 0x7f76c8072cf0 2024-04-24T23:00:36.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.884+0000 7f76d062a640 1 -- 172.21.15.32:0/2480965722 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76c80aad40 con 0x7f76c8072cf0 2024-04-24T23:00:36.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.885+0000 7f76d062a640 1 -- 172.21.15.32:0/2480965722 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76c8068430 con 0x7f76c8072cf0 2024-04-24T23:00:36.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.885+0000 7f76beffd640 1 -- 172.21.15.32:0/2480965722 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f76b800c6d0 con 0x7f76c8072cf0 2024-04-24T23:00:36.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.886+0000 7f76beffd640 1 --2- 172.21.15.32:0/2480965722 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f76ac06be30 0x7f76ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:36.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.886+0000 7f76beffd640 1 -- 172.21.15.32:0/2480965722 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f76b808ac90 con 0x7f76c8072cf0 2024-04-24T23:00:36.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.886+0000 7f76ce3c6640 1 --2- 172.21.15.32:0/2480965722 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f76ac06be30 0x7f76ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:36.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.887+0000 7f76ce3c6640 1 --2- 172.21.15.32:0/2480965722 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f76ac06be30 0x7f76ac06e2f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f76c0034000 tx=0x7f76c002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:36.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:36.890+0000 7f76beffd640 1 -- 172.21.15.32:0/2480965722 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f76c80af2b0 con 0x7f76c8072cf0 2024-04-24T23:00:37.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.142+0000 7f76d062a640 1 -- 172.21.15.32:0/2480965722 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f76c80b53c0 con 0x7f76c8072cf0 2024-04-24T23:00:37.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.143+0000 7f76beffd640 1 -- 172.21.15.32:0/2480965722 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1447 (secure 0 0 0) 0x7f76b8017020 con 0x7f76c8072cf0 2024-04-24T23:00:37.145 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:00:37.146 INFO:teuthology.orchestra.run.smithi032.stdout:{"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":110,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":8,"num_up_osds":8,"osd_up_since":1713999532,"num_in_osds":8,"osd_in_since":1713999529,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":2747790,"bytes_avail":1597252210,"bytes_total":1600000000,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":20,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"c","status":"up:active","gid":5631}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-04-24T23:00:21.144822+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-24T23:00:37.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.145+0000 7f76d062a640 1 -- 172.21.15.32:0/2480965722 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f76ac06be30 msgr2=0x7f76ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:37.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.145+0000 7f76d062a640 1 --2- 172.21.15.32:0/2480965722 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f76ac06be30 0x7f76ac06e2f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f76c0034000 tx=0x7f76c002e000 comp rx=0 tx=0).stop 2024-04-24T23:00:37.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.145+0000 7f76d062a640 1 -- 172.21.15.32:0/2480965722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76c8072cf0 msgr2=0x7f76c80ae830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:37.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.145+0000 7f76d062a640 1 --2- 172.21.15.32:0/2480965722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76c8072cf0 0x7f76c80ae830 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f76b800e7e0 tx=0x7f76b800eca0 comp rx=0 tx=0).stop 2024-04-24T23:00:37.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.145+0000 7f76d062a640 1 -- 172.21.15.32:0/2480965722 shutdown_connections 2024-04-24T23:00:37.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.145+0000 7f76d062a640 1 --2- 172.21.15.32:0/2480965722 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f76ac06be30 0x7f76ac06e2f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:37.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.145+0000 7f76d062a640 1 --2- 172.21.15.32:0/2480965722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76c8072cf0 0x7f76c80ae830 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:37.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.145+0000 7f76d062a640 1 --2- 172.21.15.32:0/2480965722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76c815c5b0 0x7f76c80aed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:37.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.145+0000 7f76d062a640 1 --2- 172.21.15.32:0/2480965722 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f76c8071770 0x7f76c8146360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:37.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.145+0000 7f76d062a640 1 -- 172.21.15.32:0/2480965722 >> 172.21.15.32:0/2480965722 conn(0x7f76c806b110 msgr2=0x7f76c806c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:37.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.145+0000 7f76d062a640 1 -- 172.21.15.32:0/2480965722 shutdown_connections 2024-04-24T23:00:37.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.145+0000 7f76d062a640 1 -- 172.21.15.32:0/2480965722 wait complete. 2024-04-24T23:00:37.157 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-24T23:00:37.157 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:00:37.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.256+0000 7fc69d984640 1 Processor -- start 2024-04-24T23:00:37.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.256+0000 7fc69d984640 1 -- start start 2024-04-24T23:00:37.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.257+0000 7fc69d984640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc69806c5f0 0x7fc69806c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:37.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.257+0000 7fc69d984640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc6980b44d0 0x7fc6980b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:37.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.257+0000 7fc69d984640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc6980af890 0x7fc6980cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:37.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.257+0000 7fc69d984640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fc6980b6430 con 0x7fc6980b44d0 2024-04-24T23:00:37.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.257+0000 7fc69d984640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fc698156f80 con 0x7fc6980af890 2024-04-24T23:00:37.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.257+0000 7fc69d984640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc6980b6150 con 0x7fc69806c5f0 2024-04-24T23:00:37.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.257+0000 7fc6967fc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc6980b44d0 0x7fc6980b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:37.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.257+0000 7fc6967fc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc6980b44d0 0x7fc6980b5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46338/0 (socket says 172.21.15.32:46338) 2024-04-24T23:00:37.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.257+0000 7fc6967fc640 1 -- 172.21.15.32:0/3053651530 learned_addr learned my addr 172.21.15.32:0/3053651530 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:37.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.257+0000 7fc6967fc640 1 -- 172.21.15.32:0/3053651530 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc69806c5f0 msgr2=0x7fc69806c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:37.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.257+0000 7fc6967fc640 1 --2- 172.21.15.32:0/3053651530 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc69806c5f0 0x7fc69806c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:37.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.257+0000 7fc6967fc640 1 -- 172.21.15.32:0/3053651530 >> v2:172.21.15.121:3300/0 conn(0x7fc6980af890 msgr2=0x7fc6980cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:37.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.257+0000 7fc6977fe640 1 --2- 172.21.15.32:0/3053651530 >> v2:172.21.15.121:3300/0 conn(0x7fc6980af890 0x7fc6980cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:37.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.257+0000 7fc6967fc640 1 --2- 172.21.15.32:0/3053651530 >> v2:172.21.15.121:3300/0 conn(0x7fc6980af890 0x7fc6980cef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:37.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.257+0000 7fc6967fc640 1 -- 172.21.15.32:0/3053651530 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6980b39f0 con 0x7fc6980b44d0 2024-04-24T23:00:37.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.258+0000 7fc6967fc640 1 --2- 172.21.15.32:0/3053651530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6980b44d0 0x7fc6980b5890 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fc6980aff30 tx=0x7fc68002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffcb8a8498bbf1f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:37.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.258+0000 7fc695ffb640 1 -- 172.21.15.32:0/3053651530 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc680037070 con 0x7fc6980b44d0 2024-04-24T23:00:37.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.258+0000 7fc695ffb640 1 -- 172.21.15.32:0/3053651530 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc680032440 con 0x7fc6980b44d0 2024-04-24T23:00:37.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.258+0000 7fc695ffb640 1 -- 172.21.15.32:0/3053651530 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc680032810 con 0x7fc6980b44d0 2024-04-24T23:00:37.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.259+0000 7fc69d984640 1 -- 172.21.15.32:0/3053651530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6980b44d0 msgr2=0x7fc6980b5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:37.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.259+0000 7fc69d984640 1 --2- 172.21.15.32:0/3053651530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6980b44d0 0x7fc6980b5890 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fc6980aff30 tx=0x7fc68002a140 comp rx=0 tx=0).stop 2024-04-24T23:00:37.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.259+0000 7fc69d984640 1 -- 172.21.15.32:0/3053651530 shutdown_connections 2024-04-24T23:00:37.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.259+0000 7fc69d984640 1 --2- 172.21.15.32:0/3053651530 >> v2:172.21.15.121:3300/0 conn(0x7fc6980af890 0x7fc6980cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:37.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.259+0000 7fc69d984640 1 --2- 172.21.15.32:0/3053651530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6980b44d0 0x7fc6980b5890 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:37.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.259+0000 7fc69d984640 1 --2- 172.21.15.32:0/3053651530 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc69806c5f0 0x7fc69806c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:37.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.259+0000 7fc69d984640 1 -- 172.21.15.32:0/3053651530 >> 172.21.15.32:0/3053651530 conn(0x7fc69806b110 msgr2=0x7fc69806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:37.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.259+0000 7fc69d984640 1 -- 172.21.15.32:0/3053651530 shutdown_connections 2024-04-24T23:00:37.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.259+0000 7fc69d984640 1 -- 172.21.15.32:0/3053651530 wait complete. 2024-04-24T23:00:37.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.260+0000 7fc69d984640 1 Processor -- start 2024-04-24T23:00:37.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.260+0000 7fc69d984640 1 -- start start 2024-04-24T23:00:37.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.261+0000 7fc69d984640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc69806c5f0 0x7fc698145d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:37.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.261+0000 7fc69d984640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6980af890 0x7fc69815e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:37.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.261+0000 7fc69d984640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc6980b44d0 0x7fc698151390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:37.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.261+0000 7fc69d984640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc69806cf10 con 0x7fc6980af890 2024-04-24T23:00:37.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.261+0000 7fc69d984640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc6980b3fb0 con 0x7fc69806c5f0 2024-04-24T23:00:37.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.261+0000 7fc69d984640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc6980b5dd0 con 0x7fc6980b44d0 2024-04-24T23:00:37.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.261+0000 7fc6967fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6980af890 0x7fc69815e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:37.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.261+0000 7fc6967fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6980af890 0x7fc69815e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46344/0 (socket says 172.21.15.32:46344) 2024-04-24T23:00:37.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.261+0000 7fc6967fc640 1 -- 172.21.15.32:0/1472734688 learned_addr learned my addr 172.21.15.32:0/1472734688 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:37.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.261+0000 7fc6977fe640 1 --2- 172.21.15.32:0/1472734688 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc6980b44d0 0x7fc698151390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:37.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.261+0000 7fc696ffd640 1 --2- 172.21.15.32:0/1472734688 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc69806c5f0 0x7fc698145d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:37.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.261+0000 7fc6967fc640 1 -- 172.21.15.32:0/1472734688 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc6980b44d0 msgr2=0x7fc698151390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:37.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.261+0000 7fc6967fc640 1 --2- 172.21.15.32:0/1472734688 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc6980b44d0 0x7fc698151390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:37.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.261+0000 7fc6967fc640 1 -- 172.21.15.32:0/1472734688 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc69806c5f0 msgr2=0x7fc698145d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:37.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.261+0000 7fc6967fc640 1 --2- 172.21.15.32:0/1472734688 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc69806c5f0 0x7fc698145d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:37.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.261+0000 7fc6967fc640 1 -- 172.21.15.32:0/1472734688 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc69815f130 con 0x7fc6980af890 2024-04-24T23:00:37.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.261+0000 7fc6967fc640 1 --2- 172.21.15.32:0/1472734688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6980af890 0x7fc69815e670 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fc680000c00 tx=0x7fc6800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:37.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.261+0000 7fc68ffff640 1 -- 172.21.15.32:0/1472734688 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc680041020 con 0x7fc6980af890 2024-04-24T23:00:37.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.261+0000 7fc68ffff640 1 -- 172.21.15.32:0/1472734688 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc680030970 con 0x7fc6980af890 2024-04-24T23:00:37.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.261+0000 7fc69d984640 1 -- 172.21.15.32:0/1472734688 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc69815f410 con 0x7fc6980af890 2024-04-24T23:00:37.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.261+0000 7fc68ffff640 1 -- 172.21.15.32:0/1472734688 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc680037040 con 0x7fc6980af890 2024-04-24T23:00:37.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.262+0000 7fc69d984640 1 -- 172.21.15.32:0/1472734688 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc69815f9d0 con 0x7fc6980af890 2024-04-24T23:00:37.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.262+0000 7fc69d984640 1 -- 172.21.15.32:0/1472734688 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc698068430 con 0x7fc6980af890 2024-04-24T23:00:37.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.263+0000 7fc68ffff640 1 -- 172.21.15.32:0/1472734688 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc680044020 con 0x7fc6980af890 2024-04-24T23:00:37.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.263+0000 7fc68ffff640 1 --2- 172.21.15.32:0/1472734688 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc67806be30 0x7fc67806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:37.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.263+0000 7fc68ffff640 1 -- 172.21.15.32:0/1472734688 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fc6800ab890 con 0x7fc6980af890 2024-04-24T23:00:37.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.264+0000 7fc696ffd640 1 --2- 172.21.15.32:0/1472734688 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc67806be30 0x7fc67806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:37.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.264+0000 7fc696ffd640 1 --2- 172.21.15.32:0/1472734688 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc67806be30 0x7fc67806e2f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc69000a770 tx=0x7fc690008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:37.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.268+0000 7fc68ffff640 1 -- 172.21.15.32:0/1472734688 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc68004b030 con 0x7fc6980af890 2024-04-24T23:00:37.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.445+0000 7fc69d984640 1 -- 172.21.15.32:0/1472734688 --> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc6980b5670 con 0x7fc67806be30 2024-04-24T23:00:37.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.451+0000 7fc68ffff640 1 -- 172.21.15.32:0/1472734688 <== mgr.4108 v2:172.21.15.121:6832/1848157943 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247491 (secure 0 0 0) 0x7fc6980b5670 con 0x7fc67806be30 2024-04-24T23:00:37.456 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:00:37.457 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-24T23:00:37.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.454+0000 7fc69d984640 1 -- 172.21.15.32:0/1472734688 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc67806be30 msgr2=0x7fc67806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:37.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.454+0000 7fc69d984640 1 --2- 172.21.15.32:0/1472734688 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc67806be30 0x7fc67806e2f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc69000a770 tx=0x7fc690008040 comp rx=0 tx=0).stop 2024-04-24T23:00:37.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.454+0000 7fc69d984640 1 -- 172.21.15.32:0/1472734688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6980af890 msgr2=0x7fc69815e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:37.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.454+0000 7fc69d984640 1 --2- 172.21.15.32:0/1472734688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6980af890 0x7fc69815e670 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fc680000c00 tx=0x7fc6800046c0 comp rx=0 tx=0).stop 2024-04-24T23:00:37.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.454+0000 7fc69d984640 1 -- 172.21.15.32:0/1472734688 shutdown_connections 2024-04-24T23:00:37.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.454+0000 7fc69d984640 1 --2- 172.21.15.32:0/1472734688 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc67806be30 0x7fc67806e2f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:37.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.454+0000 7fc69d984640 1 --2- 172.21.15.32:0/1472734688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6980af890 0x7fc69815e670 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:37.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.454+0000 7fc69d984640 1 --2- 172.21.15.32:0/1472734688 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc6980b44d0 0x7fc698151390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:37.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.454+0000 7fc69d984640 1 --2- 172.21.15.32:0/1472734688 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc69806c5f0 0x7fc698145d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:37.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.454+0000 7fc69d984640 1 -- 172.21.15.32:0/1472734688 >> 172.21.15.32:0/1472734688 conn(0x7fc69806b110 msgr2=0x7fc69815c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:37.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.454+0000 7fc69d984640 1 -- 172.21.15.32:0/1472734688 shutdown_connections 2024-04-24T23:00:37.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:37.454+0000 7fc69d984640 1 -- 172.21.15.32:0/1472734688 wait complete. 2024-04-24T23:00:37.467 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":89,"stamp":"2024-04-24T23:00:37.148193+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":186,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2827,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1559672,"statfs":{"total":1600000000,"available":1597105140,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001228"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.817034+0000","last_change":"2024-04-24T23:00:33.023527+0000","last_active":"2024-04-24T23:00:35.817034+0000","last_peered":"2024-04-24T23:00:35.817034+0000","last_clean":"2024-04-24T23:00:35.817034+0000","last_became_active":"2024-04-24T23:00:33.023243+0000","last_became_peered":"2024-04-24T23:00:33.023243+0000","last_unstale":"2024-04-24T23:00:35.817034+0000","last_undegraded":"2024-04-24T23:00:35.817034+0000","last_fullsized":"2024-04-24T23:00:35.817034+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:22:32.712700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.816666+0000","last_change":"2024-04-24T23:00:33.021299+0000","last_active":"2024-04-24T23:00:35.816666+0000","last_peered":"2024-04-24T23:00:35.816666+0000","last_clean":"2024-04-24T23:00:35.816666+0000","last_became_active":"2024-04-24T23:00:33.021010+0000","last_became_peered":"2024-04-24T23:00:33.021010+0000","last_unstale":"2024-04-24T23:00:35.816666+0000","last_undegraded":"2024-04-24T23:00:35.816666+0000","last_fullsized":"2024-04-24T23:00:35.816666+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:15:35.010392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.111511+0000","last_change":"2024-04-24T23:00:33.025927+0000","last_active":"2024-04-24T23:00:35.111511+0000","last_peered":"2024-04-24T23:00:35.111511+0000","last_clean":"2024-04-24T23:00:35.111511+0000","last_became_active":"2024-04-24T23:00:33.025699+0000","last_became_peered":"2024-04-24T23:00:33.025699+0000","last_unstale":"2024-04-24T23:00:35.111511+0000","last_undegraded":"2024-04-24T23:00:35.111511+0000","last_fullsized":"2024-04-24T23:00:35.111511+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:07.115951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.086601+0000","last_change":"2024-04-24T23:00:33.029803+0000","last_active":"2024-04-24T23:00:34.086601+0000","last_peered":"2024-04-24T23:00:34.086601+0000","last_clean":"2024-04-24T23:00:34.086601+0000","last_became_active":"2024-04-24T23:00:33.029528+0000","last_became_peered":"2024-04-24T23:00:33.029528+0000","last_unstale":"2024-04-24T23:00:34.086601+0000","last_undegraded":"2024-04-24T23:00:34.086601+0000","last_fullsized":"2024-04-24T23:00:34.086601+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:38:46.179892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.817120+0000","last_change":"2024-04-24T23:00:33.022364+0000","last_active":"2024-04-24T23:00:35.817120+0000","last_peered":"2024-04-24T23:00:35.817120+0000","last_clean":"2024-04-24T23:00:35.817120+0000","last_became_active":"2024-04-24T23:00:33.021999+0000","last_became_peered":"2024-04-24T23:00:33.021999+0000","last_unstale":"2024-04-24T23:00:35.817120+0000","last_undegraded":"2024-04-24T23:00:35.817120+0000","last_fullsized":"2024-04-24T23:00:35.817120+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:20:05.537797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.014252+0000","last_change":"2024-04-24T23:00:33.021960+0000","last_active":"2024-04-24T23:00:34.014252+0000","last_peered":"2024-04-24T23:00:34.014252+0000","last_clean":"2024-04-24T23:00:34.014252+0000","last_became_active":"2024-04-24T23:00:33.021507+0000","last_became_peered":"2024-04-24T23:00:33.021507+0000","last_unstale":"2024-04-24T23:00:34.014252+0000","last_undegraded":"2024-04-24T23:00:34.014252+0000","last_fullsized":"2024-04-24T23:00:34.014252+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:26:00.374805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.010873+0000","last_change":"2024-04-24T23:00:33.024126+0000","last_active":"2024-04-24T23:00:34.010873+0000","last_peered":"2024-04-24T23:00:34.010873+0000","last_clean":"2024-04-24T23:00:34.010873+0000","last_became_active":"2024-04-24T23:00:33.023881+0000","last_became_peered":"2024-04-24T23:00:33.023881+0000","last_unstale":"2024-04-24T23:00:34.010873+0000","last_undegraded":"2024-04-24T23:00:34.010873+0000","last_fullsized":"2024-04-24T23:00:34.010873+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:51:10.777829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.086491+0000","last_change":"2024-04-24T23:00:33.023498+0000","last_active":"2024-04-24T23:00:34.086491+0000","last_peered":"2024-04-24T23:00:34.086491+0000","last_clean":"2024-04-24T23:00:34.086491+0000","last_became_active":"2024-04-24T23:00:33.022718+0000","last_became_peered":"2024-04-24T23:00:33.022718+0000","last_unstale":"2024-04-24T23:00:34.086491+0000","last_undegraded":"2024-04-24T23:00:34.086491+0000","last_fullsized":"2024-04-24T23:00:34.086491+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:44:47.688330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.014235+0000","last_change":"2024-04-24T23:00:33.029934+0000","last_active":"2024-04-24T23:00:34.014235+0000","last_peered":"2024-04-24T23:00:34.014235+0000","last_clean":"2024-04-24T23:00:34.014235+0000","last_became_active":"2024-04-24T23:00:33.029609+0000","last_became_peered":"2024-04-24T23:00:33.029609+0000","last_unstale":"2024-04-24T23:00:34.014235+0000","last_undegraded":"2024-04-24T23:00:34.014235+0000","last_fullsized":"2024-04-24T23:00:34.014235+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:05.664812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.014309+0000","last_change":"2024-04-24T23:00:33.026698+0000","last_active":"2024-04-24T23:00:34.014309+0000","last_peered":"2024-04-24T23:00:34.014309+0000","last_clean":"2024-04-24T23:00:34.014309+0000","last_became_active":"2024-04-24T23:00:33.025699+0000","last_became_peered":"2024-04-24T23:00:33.025699+0000","last_unstale":"2024-04-24T23:00:34.014309+0000","last_undegraded":"2024-04-24T23:00:34.014309+0000","last_fullsized":"2024-04-24T23:00:34.014309+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:01:08.342156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.086658+0000","last_change":"2024-04-24T23:00:33.020762+0000","last_active":"2024-04-24T23:00:34.086658+0000","last_peered":"2024-04-24T23:00:34.086658+0000","last_clean":"2024-04-24T23:00:34.086658+0000","last_became_active":"2024-04-24T23:00:33.020263+0000","last_became_peered":"2024-04-24T23:00:33.020263+0000","last_unstale":"2024-04-24T23:00:34.086658+0000","last_undegraded":"2024-04-24T23:00:34.086658+0000","last_fullsized":"2024-04-24T23:00:34.086658+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:19:52.368532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.816831+0000","last_change":"2024-04-24T23:00:33.027989+0000","last_active":"2024-04-24T23:00:35.816831+0000","last_peered":"2024-04-24T23:00:35.816831+0000","last_clean":"2024-04-24T23:00:35.816831+0000","last_became_active":"2024-04-24T23:00:33.025402+0000","last_became_peered":"2024-04-24T23:00:33.025402+0000","last_unstale":"2024-04-24T23:00:35.816831+0000","last_undegraded":"2024-04-24T23:00:35.816831+0000","last_fullsized":"2024-04-24T23:00:35.816831+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:10:21.505047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.014578+0000","last_change":"2024-04-24T23:00:33.021111+0000","last_active":"2024-04-24T23:00:34.014578+0000","last_peered":"2024-04-24T23:00:34.014578+0000","last_clean":"2024-04-24T23:00:34.014578+0000","last_became_active":"2024-04-24T23:00:33.020822+0000","last_became_peered":"2024-04-24T23:00:33.020822+0000","last_unstale":"2024-04-24T23:00:34.014578+0000","last_undegraded":"2024-04-24T23:00:34.014578+0000","last_fullsized":"2024-04-24T23:00:34.014578+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:39:32.044154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.086862+0000","last_change":"2024-04-24T23:00:33.032538+0000","last_active":"2024-04-24T23:00:34.086862+0000","last_peered":"2024-04-24T23:00:34.086862+0000","last_clean":"2024-04-24T23:00:34.086862+0000","last_became_active":"2024-04-24T23:00:33.032060+0000","last_became_peered":"2024-04-24T23:00:33.032060+0000","last_unstale":"2024-04-24T23:00:34.086862+0000","last_undegraded":"2024-04-24T23:00:34.086862+0000","last_fullsized":"2024-04-24T23:00:34.086862+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:42:05.030592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.086547+0000","last_change":"2024-04-24T23:00:33.023972+0000","last_active":"2024-04-24T23:00:34.086547+0000","last_peered":"2024-04-24T23:00:34.086547+0000","last_clean":"2024-04-24T23:00:34.086547+0000","last_became_active":"2024-04-24T23:00:33.023742+0000","last_became_peered":"2024-04-24T23:00:33.023742+0000","last_unstale":"2024-04-24T23:00:34.086547+0000","last_undegraded":"2024-04-24T23:00:34.086547+0000","last_fullsized":"2024-04-24T23:00:34.086547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:16:14.948047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.817277+0000","last_change":"2024-04-24T23:00:33.022971+0000","last_active":"2024-04-24T23:00:35.817277+0000","last_peered":"2024-04-24T23:00:35.817277+0000","last_clean":"2024-04-24T23:00:35.817277+0000","last_became_active":"2024-04-24T23:00:33.022639+0000","last_became_peered":"2024-04-24T23:00:33.022639+0000","last_unstale":"2024-04-24T23:00:35.817277+0000","last_undegraded":"2024-04-24T23:00:35.817277+0000","last_fullsized":"2024-04-24T23:00:35.817277+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:58:17.058198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.816982+0000","last_change":"2024-04-24T23:00:33.028457+0000","last_active":"2024-04-24T23:00:35.816982+0000","last_peered":"2024-04-24T23:00:35.816982+0000","last_clean":"2024-04-24T23:00:35.816982+0000","last_became_active":"2024-04-24T23:00:33.027401+0000","last_became_peered":"2024-04-24T23:00:33.027401+0000","last_unstale":"2024-04-24T23:00:35.816982+0000","last_undegraded":"2024-04-24T23:00:35.816982+0000","last_fullsized":"2024-04-24T23:00:35.816982+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:33:50.382484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.111661+0000","last_change":"2024-04-24T23:00:33.027850+0000","last_active":"2024-04-24T23:00:35.111661+0000","last_peered":"2024-04-24T23:00:35.111661+0000","last_clean":"2024-04-24T23:00:35.111661+0000","last_became_active":"2024-04-24T23:00:33.027238+0000","last_became_peered":"2024-04-24T23:00:33.027238+0000","last_unstale":"2024-04-24T23:00:35.111661+0000","last_undegraded":"2024-04-24T23:00:35.111661+0000","last_fullsized":"2024-04-24T23:00:35.111661+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:49:59.031348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.817057+0000","last_change":"2024-04-24T23:00:33.020153+0000","last_active":"2024-04-24T23:00:35.817057+0000","last_peered":"2024-04-24T23:00:35.817057+0000","last_clean":"2024-04-24T23:00:35.817057+0000","last_became_active":"2024-04-24T23:00:33.019897+0000","last_became_peered":"2024-04-24T23:00:33.019897+0000","last_unstale":"2024-04-24T23:00:35.817057+0000","last_undegraded":"2024-04-24T23:00:35.817057+0000","last_fullsized":"2024-04-24T23:00:35.817057+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:33:56.003472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.014798+0000","last_change":"2024-04-24T23:00:33.026517+0000","last_active":"2024-04-24T23:00:34.014798+0000","last_peered":"2024-04-24T23:00:34.014798+0000","last_clean":"2024-04-24T23:00:34.014798+0000","last_became_active":"2024-04-24T23:00:33.025323+0000","last_became_peered":"2024-04-24T23:00:33.025323+0000","last_unstale":"2024-04-24T23:00:34.014798+0000","last_undegraded":"2024-04-24T23:00:34.014798+0000","last_fullsized":"2024-04-24T23:00:34.014798+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:12.264162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.087078+0000","last_change":"2024-04-24T23:00:33.029989+0000","last_active":"2024-04-24T23:00:34.087078+0000","last_peered":"2024-04-24T23:00:34.087078+0000","last_clean":"2024-04-24T23:00:34.087078+0000","last_became_active":"2024-04-24T23:00:33.029261+0000","last_became_peered":"2024-04-24T23:00:33.029261+0000","last_unstale":"2024-04-24T23:00:34.087078+0000","last_undegraded":"2024-04-24T23:00:34.087078+0000","last_fullsized":"2024-04-24T23:00:34.087078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:22:39.629270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.111977+0000","last_change":"2024-04-24T23:00:33.018131+0000","last_active":"2024-04-24T23:00:35.111977+0000","last_peered":"2024-04-24T23:00:35.111977+0000","last_clean":"2024-04-24T23:00:35.111977+0000","last_became_active":"2024-04-24T23:00:33.017878+0000","last_became_peered":"2024-04-24T23:00:33.017878+0000","last_unstale":"2024-04-24T23:00:35.111977+0000","last_undegraded":"2024-04-24T23:00:35.111977+0000","last_fullsized":"2024-04-24T23:00:35.111977+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:11:31.967754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.011441+0000","last_change":"2024-04-24T23:00:33.025102+0000","last_active":"2024-04-24T23:00:34.011441+0000","last_peered":"2024-04-24T23:00:34.011441+0000","last_clean":"2024-04-24T23:00:34.011441+0000","last_became_active":"2024-04-24T23:00:33.022808+0000","last_became_peered":"2024-04-24T23:00:33.022808+0000","last_unstale":"2024-04-24T23:00:34.011441+0000","last_undegraded":"2024-04-24T23:00:34.011441+0000","last_fullsized":"2024-04-24T23:00:34.011441+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:17:31.920100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.816640+0000","last_change":"2024-04-24T23:00:33.022669+0000","last_active":"2024-04-24T23:00:35.816640+0000","last_peered":"2024-04-24T23:00:35.816640+0000","last_clean":"2024-04-24T23:00:35.816640+0000","last_became_active":"2024-04-24T23:00:33.022385+0000","last_became_peered":"2024-04-24T23:00:33.022385+0000","last_unstale":"2024-04-24T23:00:35.816640+0000","last_undegraded":"2024-04-24T23:00:35.816640+0000","last_fullsized":"2024-04-24T23:00:35.816640+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:48:20.573793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.817207+0000","last_change":"2024-04-24T23:00:33.025403+0000","last_active":"2024-04-24T23:00:35.817207+0000","last_peered":"2024-04-24T23:00:35.817207+0000","last_clean":"2024-04-24T23:00:35.817207+0000","last_became_active":"2024-04-24T23:00:33.025072+0000","last_became_peered":"2024-04-24T23:00:33.025072+0000","last_unstale":"2024-04-24T23:00:35.817207+0000","last_undegraded":"2024-04-24T23:00:35.817207+0000","last_fullsized":"2024-04-24T23:00:35.817207+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:02.831738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.086814+0000","last_change":"2024-04-24T23:00:33.020508+0000","last_active":"2024-04-24T23:00:34.086814+0000","last_peered":"2024-04-24T23:00:34.086814+0000","last_clean":"2024-04-24T23:00:34.086814+0000","last_became_active":"2024-04-24T23:00:33.019950+0000","last_became_peered":"2024-04-24T23:00:33.019950+0000","last_unstale":"2024-04-24T23:00:34.086814+0000","last_undegraded":"2024-04-24T23:00:34.086814+0000","last_fullsized":"2024-04-24T23:00:34.086814+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:30:38.408344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.011035+0000","last_change":"2024-04-24T23:00:33.018305+0000","last_active":"2024-04-24T23:00:34.011035+0000","last_peered":"2024-04-24T23:00:34.011035+0000","last_clean":"2024-04-24T23:00:34.011035+0000","last_became_active":"2024-04-24T23:00:33.017968+0000","last_became_peered":"2024-04-24T23:00:33.017968+0000","last_unstale":"2024-04-24T23:00:34.011035+0000","last_undegraded":"2024-04-24T23:00:34.011035+0000","last_fullsized":"2024-04-24T23:00:34.011035+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:35:37.311179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.011127+0000","last_change":"2024-04-24T23:00:33.023042+0000","last_active":"2024-04-24T23:00:34.011127+0000","last_peered":"2024-04-24T23:00:34.011127+0000","last_clean":"2024-04-24T23:00:34.011127+0000","last_became_active":"2024-04-24T23:00:33.022480+0000","last_became_peered":"2024-04-24T23:00:33.022480+0000","last_unstale":"2024-04-24T23:00:34.011127+0000","last_undegraded":"2024-04-24T23:00:34.011127+0000","last_fullsized":"2024-04-24T23:00:34.011127+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:54:49.637246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.816806+0000","last_change":"2024-04-24T23:00:33.031873+0000","last_active":"2024-04-24T23:00:35.816806+0000","last_peered":"2024-04-24T23:00:35.816806+0000","last_clean":"2024-04-24T23:00:35.816806+0000","last_became_active":"2024-04-24T23:00:33.031636+0000","last_became_peered":"2024-04-24T23:00:33.031636+0000","last_unstale":"2024-04-24T23:00:35.816806+0000","last_undegraded":"2024-04-24T23:00:35.816806+0000","last_fullsized":"2024-04-24T23:00:35.816806+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:09.669278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.816981+0000","last_change":"2024-04-24T23:00:33.020565+0000","last_active":"2024-04-24T23:00:35.816981+0000","last_peered":"2024-04-24T23:00:35.816981+0000","last_clean":"2024-04-24T23:00:35.816981+0000","last_became_active":"2024-04-24T23:00:33.019812+0000","last_became_peered":"2024-04-24T23:00:33.019812+0000","last_unstale":"2024-04-24T23:00:35.816981+0000","last_undegraded":"2024-04-24T23:00:35.816981+0000","last_fullsized":"2024-04-24T23:00:35.816981+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:27:57.748785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.014405+0000","last_change":"2024-04-24T23:00:33.023994+0000","last_active":"2024-04-24T23:00:34.014405+0000","last_peered":"2024-04-24T23:00:34.014405+0000","last_clean":"2024-04-24T23:00:34.014405+0000","last_became_active":"2024-04-24T23:00:33.023755+0000","last_became_peered":"2024-04-24T23:00:33.023755+0000","last_unstale":"2024-04-24T23:00:34.014405+0000","last_undegraded":"2024-04-24T23:00:34.014405+0000","last_fullsized":"2024-04-24T23:00:34.014405+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:57:59.769840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.086585+0000","last_change":"2024-04-24T23:00:33.020190+0000","last_active":"2024-04-24T23:00:34.086585+0000","last_peered":"2024-04-24T23:00:34.086585+0000","last_clean":"2024-04-24T23:00:34.086585+0000","last_became_active":"2024-04-24T23:00:33.019894+0000","last_became_peered":"2024-04-24T23:00:33.019894+0000","last_unstale":"2024-04-24T23:00:34.086585+0000","last_undegraded":"2024-04-24T23:00:34.086585+0000","last_fullsized":"2024-04-24T23:00:34.086585+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:49:22.102434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.111670+0000","last_change":"2024-04-24T23:00:33.029760+0000","last_active":"2024-04-24T23:00:35.111670+0000","last_peered":"2024-04-24T23:00:35.111670+0000","last_clean":"2024-04-24T23:00:35.111670+0000","last_became_active":"2024-04-24T23:00:33.029171+0000","last_became_peered":"2024-04-24T23:00:33.029171+0000","last_unstale":"2024-04-24T23:00:35.111670+0000","last_undegraded":"2024-04-24T23:00:35.111670+0000","last_fullsized":"2024-04-24T23:00:35.111670+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:50:38.830879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"44'4","reported_seq":18,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.014821+0000","last_change":"2024-04-24T23:00:32.013964+0000","last_active":"2024-04-24T23:00:34.014821+0000","last_peered":"2024-04-24T23:00:34.014821+0000","last_clean":"2024-04-24T23:00:34.014821+0000","last_became_active":"2024-04-24T23:00:32.013661+0000","last_became_peered":"2024-04-24T23:00:32.013661+0000","last_unstale":"2024-04-24T23:00:34.014821+0000","last_undegraded":"2024-04-24T23:00:34.014821+0000","last_fullsized":"2024-04-24T23:00:34.014821+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:34:35.486621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.817554+0000","last_change":"2024-04-24T23:00:33.021778+0000","last_active":"2024-04-24T23:00:35.817554+0000","last_peered":"2024-04-24T23:00:35.817554+0000","last_clean":"2024-04-24T23:00:35.817554+0000","last_became_active":"2024-04-24T23:00:33.021385+0000","last_became_peered":"2024-04-24T23:00:33.021385+0000","last_unstale":"2024-04-24T23:00:35.817554+0000","last_undegraded":"2024-04-24T23:00:35.817554+0000","last_fullsized":"2024-04-24T23:00:35.817554+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:40:30.680600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.011199+0000","last_change":"2024-04-24T23:00:32.012770+0000","last_active":"2024-04-24T23:00:34.011199+0000","last_peered":"2024-04-24T23:00:34.011199+0000","last_clean":"2024-04-24T23:00:34.011199+0000","last_became_active":"2024-04-24T23:00:32.012092+0000","last_became_peered":"2024-04-24T23:00:32.012092+0000","last_unstale":"2024-04-24T23:00:34.011199+0000","last_undegraded":"2024-04-24T23:00:34.011199+0000","last_fullsized":"2024-04-24T23:00:34.011199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:49:58.754592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.817113+0000","last_change":"2024-04-24T23:00:33.021178+0000","last_active":"2024-04-24T23:00:35.817113+0000","last_peered":"2024-04-24T23:00:35.817113+0000","last_clean":"2024-04-24T23:00:35.817113+0000","last_became_active":"2024-04-24T23:00:33.020678+0000","last_became_peered":"2024-04-24T23:00:33.020678+0000","last_unstale":"2024-04-24T23:00:35.817113+0000","last_undegraded":"2024-04-24T23:00:35.817113+0000","last_fullsized":"2024-04-24T23:00:35.817113+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:58.669126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.014327+0000","last_change":"2024-04-24T23:00:32.015863+0000","last_active":"2024-04-24T23:00:34.014327+0000","last_peered":"2024-04-24T23:00:34.014327+0000","last_clean":"2024-04-24T23:00:34.014327+0000","last_became_active":"2024-04-24T23:00:32.015616+0000","last_became_peered":"2024-04-24T23:00:32.015616+0000","last_unstale":"2024-04-24T23:00:34.014327+0000","last_undegraded":"2024-04-24T23:00:34.014327+0000","last_fullsized":"2024-04-24T23:00:34.014327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:36:07.320735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.086716+0000","last_change":"2024-04-24T23:00:33.021120+0000","last_active":"2024-04-24T23:00:34.086716+0000","last_peered":"2024-04-24T23:00:34.086716+0000","last_clean":"2024-04-24T23:00:34.086716+0000","last_became_active":"2024-04-24T23:00:33.020671+0000","last_became_peered":"2024-04-24T23:00:33.020671+0000","last_unstale":"2024-04-24T23:00:34.086716+0000","last_undegraded":"2024-04-24T23:00:34.086716+0000","last_fullsized":"2024-04-24T23:00:34.086716+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:54:07.317918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.010827+0000","last_change":"2024-04-24T23:00:32.010390+0000","last_active":"2024-04-24T23:00:34.010827+0000","last_peered":"2024-04-24T23:00:34.010827+0000","last_clean":"2024-04-24T23:00:34.010827+0000","last_became_active":"2024-04-24T23:00:32.010156+0000","last_became_peered":"2024-04-24T23:00:32.010156+0000","last_unstale":"2024-04-24T23:00:34.010827+0000","last_undegraded":"2024-04-24T23:00:34.010827+0000","last_fullsized":"2024-04-24T23:00:34.010827+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:53.937003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.011256+0000","last_change":"2024-04-24T23:00:33.027125+0000","last_active":"2024-04-24T23:00:34.011256+0000","last_peered":"2024-04-24T23:00:34.011256+0000","last_clean":"2024-04-24T23:00:34.011256+0000","last_became_active":"2024-04-24T23:00:33.026356+0000","last_became_peered":"2024-04-24T23:00:33.026356+0000","last_unstale":"2024-04-24T23:00:34.011256+0000","last_undegraded":"2024-04-24T23:00:34.011256+0000","last_fullsized":"2024-04-24T23:00:34.011256+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:02:37.996954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.014648+0000","last_change":"2024-04-24T23:00:32.012144+0000","last_active":"2024-04-24T23:00:34.014648+0000","last_peered":"2024-04-24T23:00:34.014648+0000","last_clean":"2024-04-24T23:00:34.014648+0000","last_became_active":"2024-04-24T23:00:32.011876+0000","last_became_peered":"2024-04-24T23:00:32.011876+0000","last_unstale":"2024-04-24T23:00:34.014648+0000","last_undegraded":"2024-04-24T23:00:34.014648+0000","last_fullsized":"2024-04-24T23:00:34.014648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:19:27.134959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.014032+0000","last_change":"2024-04-24T23:00:33.021801+0000","last_active":"2024-04-24T23:00:34.014032+0000","last_peered":"2024-04-24T23:00:34.014032+0000","last_clean":"2024-04-24T23:00:34.014032+0000","last_became_active":"2024-04-24T23:00:33.021575+0000","last_became_peered":"2024-04-24T23:00:33.021575+0000","last_unstale":"2024-04-24T23:00:34.014032+0000","last_undegraded":"2024-04-24T23:00:34.014032+0000","last_fullsized":"2024-04-24T23:00:34.014032+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:30:09.050918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.011510+0000","last_change":"2024-04-24T23:00:32.015060+0000","last_active":"2024-04-24T23:00:34.011510+0000","last_peered":"2024-04-24T23:00:34.011510+0000","last_clean":"2024-04-24T23:00:34.011510+0000","last_became_active":"2024-04-24T23:00:32.013636+0000","last_became_peered":"2024-04-24T23:00:32.013636+0000","last_unstale":"2024-04-24T23:00:34.011510+0000","last_undegraded":"2024-04-24T23:00:34.011510+0000","last_fullsized":"2024-04-24T23:00:34.011510+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:43:05.640033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.014477+0000","last_change":"2024-04-24T23:00:33.027352+0000","last_active":"2024-04-24T23:00:34.014477+0000","last_peered":"2024-04-24T23:00:34.014477+0000","last_clean":"2024-04-24T23:00:34.014477+0000","last_became_active":"2024-04-24T23:00:33.027121+0000","last_became_peered":"2024-04-24T23:00:33.027121+0000","last_unstale":"2024-04-24T23:00:34.014477+0000","last_undegraded":"2024-04-24T23:00:34.014477+0000","last_fullsized":"2024-04-24T23:00:34.014477+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:07:27.945918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.816900+0000","last_change":"2024-04-24T23:00:32.018961+0000","last_active":"2024-04-24T23:00:35.816900+0000","last_peered":"2024-04-24T23:00:35.816900+0000","last_clean":"2024-04-24T23:00:35.816900+0000","last_became_active":"2024-04-24T23:00:32.018718+0000","last_became_peered":"2024-04-24T23:00:32.018718+0000","last_unstale":"2024-04-24T23:00:35.816900+0000","last_undegraded":"2024-04-24T23:00:35.816900+0000","last_fullsized":"2024-04-24T23:00:35.816900+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:05:42.477328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.010885+0000","last_change":"2024-04-24T23:00:33.012456+0000","last_active":"2024-04-24T23:00:34.010885+0000","last_peered":"2024-04-24T23:00:34.010885+0000","last_clean":"2024-04-24T23:00:34.010885+0000","last_became_active":"2024-04-24T23:00:33.012207+0000","last_became_peered":"2024-04-24T23:00:33.012207+0000","last_unstale":"2024-04-24T23:00:34.010885+0000","last_undegraded":"2024-04-24T23:00:34.010885+0000","last_fullsized":"2024-04-24T23:00:34.010885+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:28.579298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.014895+0000","last_change":"2024-04-24T23:00:32.016634+0000","last_active":"2024-04-24T23:00:34.014895+0000","last_peered":"2024-04-24T23:00:34.014895+0000","last_clean":"2024-04-24T23:00:34.014895+0000","last_became_active":"2024-04-24T23:00:32.015425+0000","last_became_peered":"2024-04-24T23:00:32.015425+0000","last_unstale":"2024-04-24T23:00:34.014895+0000","last_undegraded":"2024-04-24T23:00:34.014895+0000","last_fullsized":"2024-04-24T23:00:34.014895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:21:01.877515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.111606+0000","last_change":"2024-04-24T23:00:33.017687+0000","last_active":"2024-04-24T23:00:35.111606+0000","last_peered":"2024-04-24T23:00:35.111606+0000","last_clean":"2024-04-24T23:00:35.111606+0000","last_became_active":"2024-04-24T23:00:33.017433+0000","last_became_peered":"2024-04-24T23:00:33.017433+0000","last_unstale":"2024-04-24T23:00:35.111606+0000","last_undegraded":"2024-04-24T23:00:35.111606+0000","last_fullsized":"2024-04-24T23:00:35.111606+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:32:36.803741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.111814+0000","last_change":"2024-04-24T23:00:32.017822+0000","last_active":"2024-04-24T23:00:35.111814+0000","last_peered":"2024-04-24T23:00:35.111814+0000","last_clean":"2024-04-24T23:00:35.111814+0000","last_became_active":"2024-04-24T23:00:32.017548+0000","last_became_peered":"2024-04-24T23:00:32.017548+0000","last_unstale":"2024-04-24T23:00:35.111814+0000","last_undegraded":"2024-04-24T23:00:35.111814+0000","last_fullsized":"2024-04-24T23:00:35.111814+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:38:13.659009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.111557+0000","last_change":"2024-04-24T23:00:33.025375+0000","last_active":"2024-04-24T23:00:35.111557+0000","last_peered":"2024-04-24T23:00:35.111557+0000","last_clean":"2024-04-24T23:00:35.111557+0000","last_became_active":"2024-04-24T23:00:33.024368+0000","last_became_peered":"2024-04-24T23:00:33.024368+0000","last_unstale":"2024-04-24T23:00:35.111557+0000","last_undegraded":"2024-04-24T23:00:35.111557+0000","last_fullsized":"2024-04-24T23:00:35.111557+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:33:19.427462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.014409+0000","last_change":"2024-04-24T23:00:32.016337+0000","last_active":"2024-04-24T23:00:34.014409+0000","last_peered":"2024-04-24T23:00:34.014409+0000","last_clean":"2024-04-24T23:00:34.014409+0000","last_became_active":"2024-04-24T23:00:32.015833+0000","last_became_peered":"2024-04-24T23:00:32.015833+0000","last_unstale":"2024-04-24T23:00:34.014409+0000","last_undegraded":"2024-04-24T23:00:34.014409+0000","last_fullsized":"2024-04-24T23:00:34.014409+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:23:46.588298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.086366+0000","last_change":"2024-04-24T23:00:33.032317+0000","last_active":"2024-04-24T23:00:34.086366+0000","last_peered":"2024-04-24T23:00:34.086366+0000","last_clean":"2024-04-24T23:00:34.086366+0000","last_became_active":"2024-04-24T23:00:33.031714+0000","last_became_peered":"2024-04-24T23:00:33.031714+0000","last_unstale":"2024-04-24T23:00:34.086366+0000","last_undegraded":"2024-04-24T23:00:34.086366+0000","last_fullsized":"2024-04-24T23:00:34.086366+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:09:43.053763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"44'3","reported_seq":17,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.816962+0000","last_change":"2024-04-24T23:00:32.017203+0000","last_active":"2024-04-24T23:00:35.816962+0000","last_peered":"2024-04-24T23:00:35.816962+0000","last_clean":"2024-04-24T23:00:35.816962+0000","last_became_active":"2024-04-24T23:00:32.016743+0000","last_became_peered":"2024-04-24T23:00:32.016743+0000","last_unstale":"2024-04-24T23:00:35.816962+0000","last_undegraded":"2024-04-24T23:00:35.816962+0000","last_fullsized":"2024-04-24T23:00:35.816962+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:28.587565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.011341+0000","last_change":"2024-04-24T23:00:33.026608+0000","last_active":"2024-04-24T23:00:34.011341+0000","last_peered":"2024-04-24T23:00:34.011341+0000","last_clean":"2024-04-24T23:00:34.011341+0000","last_became_active":"2024-04-24T23:00:33.025380+0000","last_became_peered":"2024-04-24T23:00:33.025380+0000","last_unstale":"2024-04-24T23:00:34.011341+0000","last_undegraded":"2024-04-24T23:00:34.011341+0000","last_fullsized":"2024-04-24T23:00:34.011341+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:41:28.669943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"44'6","reported_seq":20,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.014745+0000","last_change":"2024-04-24T23:00:32.014044+0000","last_active":"2024-04-24T23:00:34.014745+0000","last_peered":"2024-04-24T23:00:34.014745+0000","last_clean":"2024-04-24T23:00:34.014745+0000","last_became_active":"2024-04-24T23:00:32.013781+0000","last_became_peered":"2024-04-24T23:00:32.013781+0000","last_unstale":"2024-04-24T23:00:34.014745+0000","last_undegraded":"2024-04-24T23:00:34.014745+0000","last_fullsized":"2024-04-24T23:00:34.014745+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:31:21.219632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.111671+0000","last_change":"2024-04-24T23:00:33.026903+0000","last_active":"2024-04-24T23:00:35.111671+0000","last_peered":"2024-04-24T23:00:35.111671+0000","last_clean":"2024-04-24T23:00:35.111671+0000","last_became_active":"2024-04-24T23:00:33.026648+0000","last_became_peered":"2024-04-24T23:00:33.026648+0000","last_unstale":"2024-04-24T23:00:35.111671+0000","last_undegraded":"2024-04-24T23:00:35.111671+0000","last_fullsized":"2024-04-24T23:00:35.111671+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:49.955693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.112010+0000","last_change":"2024-04-24T23:00:32.019200+0000","last_active":"2024-04-24T23:00:35.112010+0000","last_peered":"2024-04-24T23:00:35.112010+0000","last_clean":"2024-04-24T23:00:35.112010+0000","last_became_active":"2024-04-24T23:00:32.018935+0000","last_became_peered":"2024-04-24T23:00:32.018935+0000","last_unstale":"2024-04-24T23:00:35.112010+0000","last_undegraded":"2024-04-24T23:00:35.112010+0000","last_fullsized":"2024-04-24T23:00:35.112010+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:18:59.543394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.111817+0000","last_change":"2024-04-24T23:00:33.022700+0000","last_active":"2024-04-24T23:00:35.111817+0000","last_peered":"2024-04-24T23:00:35.111817+0000","last_clean":"2024-04-24T23:00:35.111817+0000","last_became_active":"2024-04-24T23:00:33.022475+0000","last_became_peered":"2024-04-24T23:00:33.022475+0000","last_unstale":"2024-04-24T23:00:35.111817+0000","last_undegraded":"2024-04-24T23:00:35.111817+0000","last_fullsized":"2024-04-24T23:00:35.111817+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:19:49.215022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.817457+0000","last_change":"2024-04-24T23:00:32.018091+0000","last_active":"2024-04-24T23:00:35.817457+0000","last_peered":"2024-04-24T23:00:35.817457+0000","last_clean":"2024-04-24T23:00:35.817457+0000","last_became_active":"2024-04-24T23:00:32.017828+0000","last_became_peered":"2024-04-24T23:00:32.017828+0000","last_unstale":"2024-04-24T23:00:35.817457+0000","last_undegraded":"2024-04-24T23:00:35.817457+0000","last_fullsized":"2024-04-24T23:00:35.817457+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:49:51.828270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.111692+0000","last_change":"2024-04-24T23:00:33.022022+0000","last_active":"2024-04-24T23:00:35.111692+0000","last_peered":"2024-04-24T23:00:35.111692+0000","last_clean":"2024-04-24T23:00:35.111692+0000","last_became_active":"2024-04-24T23:00:33.021782+0000","last_became_peered":"2024-04-24T23:00:33.021782+0000","last_unstale":"2024-04-24T23:00:35.111692+0000","last_undegraded":"2024-04-24T23:00:35.111692+0000","last_fullsized":"2024-04-24T23:00:35.111692+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:20:57.145395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.014765+0000","last_change":"2024-04-24T23:00:32.015309+0000","last_active":"2024-04-24T23:00:34.014765+0000","last_peered":"2024-04-24T23:00:34.014765+0000","last_clean":"2024-04-24T23:00:34.014765+0000","last_became_active":"2024-04-24T23:00:32.014784+0000","last_became_peered":"2024-04-24T23:00:32.014784+0000","last_unstale":"2024-04-24T23:00:34.014765+0000","last_undegraded":"2024-04-24T23:00:34.014765+0000","last_fullsized":"2024-04-24T23:00:34.014765+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:05:50.378062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.817181+0000","last_change":"2024-04-24T23:00:33.028237+0000","last_active":"2024-04-24T23:00:35.817181+0000","last_peered":"2024-04-24T23:00:35.817181+0000","last_clean":"2024-04-24T23:00:35.817181+0000","last_became_active":"2024-04-24T23:00:33.027083+0000","last_became_peered":"2024-04-24T23:00:33.027083+0000","last_unstale":"2024-04-24T23:00:35.817181+0000","last_undegraded":"2024-04-24T23:00:35.817181+0000","last_fullsized":"2024-04-24T23:00:35.817181+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:09:29.327093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"44'8","reported_seq":22,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.014481+0000","last_change":"2024-04-24T23:00:32.018235+0000","last_active":"2024-04-24T23:00:34.014481+0000","last_peered":"2024-04-24T23:00:34.014481+0000","last_clean":"2024-04-24T23:00:34.014481+0000","last_became_active":"2024-04-24T23:00:32.017933+0000","last_became_peered":"2024-04-24T23:00:32.017933+0000","last_unstale":"2024-04-24T23:00:34.014481+0000","last_undegraded":"2024-04-24T23:00:34.014481+0000","last_fullsized":"2024-04-24T23:00:34.014481+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:16.141997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.111948+0000","last_change":"2024-04-24T23:00:33.017444+0000","last_active":"2024-04-24T23:00:35.111948+0000","last_peered":"2024-04-24T23:00:35.111948+0000","last_clean":"2024-04-24T23:00:35.111948+0000","last_became_active":"2024-04-24T23:00:33.017112+0000","last_became_peered":"2024-04-24T23:00:33.017112+0000","last_unstale":"2024-04-24T23:00:35.111948+0000","last_undegraded":"2024-04-24T23:00:35.111948+0000","last_fullsized":"2024-04-24T23:00:35.111948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:39:24.740803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.086739+0000","last_change":"2024-04-24T23:00:32.015665+0000","last_active":"2024-04-24T23:00:34.086739+0000","last_peered":"2024-04-24T23:00:34.086739+0000","last_clean":"2024-04-24T23:00:34.086739+0000","last_became_active":"2024-04-24T23:00:32.015414+0000","last_became_peered":"2024-04-24T23:00:32.015414+0000","last_unstale":"2024-04-24T23:00:34.086739+0000","last_undegraded":"2024-04-24T23:00:34.086739+0000","last_fullsized":"2024-04-24T23:00:34.086739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:28:23.691908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.111818+0000","last_change":"2024-04-24T23:00:33.029957+0000","last_active":"2024-04-24T23:00:35.111818+0000","last_peered":"2024-04-24T23:00:35.111818+0000","last_clean":"2024-04-24T23:00:35.111818+0000","last_became_active":"2024-04-24T23:00:33.029515+0000","last_became_peered":"2024-04-24T23:00:33.029515+0000","last_unstale":"2024-04-24T23:00:35.111818+0000","last_undegraded":"2024-04-24T23:00:35.111818+0000","last_fullsized":"2024-04-24T23:00:35.111818+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:02:21.930006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.111690+0000","last_change":"2024-04-24T23:00:32.016781+0000","last_active":"2024-04-24T23:00:35.111690+0000","last_peered":"2024-04-24T23:00:35.111690+0000","last_clean":"2024-04-24T23:00:35.111690+0000","last_became_active":"2024-04-24T23:00:32.016515+0000","last_became_peered":"2024-04-24T23:00:32.016515+0000","last_unstale":"2024-04-24T23:00:35.111690+0000","last_undegraded":"2024-04-24T23:00:35.111690+0000","last_fullsized":"2024-04-24T23:00:35.111690+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:03:55.033966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.817317+0000","last_change":"2024-04-24T23:00:33.027722+0000","last_active":"2024-04-24T23:00:35.817317+0000","last_peered":"2024-04-24T23:00:35.817317+0000","last_clean":"2024-04-24T23:00:35.817317+0000","last_became_active":"2024-04-24T23:00:33.025042+0000","last_became_peered":"2024-04-24T23:00:33.025042+0000","last_unstale":"2024-04-24T23:00:35.817317+0000","last_undegraded":"2024-04-24T23:00:35.817317+0000","last_fullsized":"2024-04-24T23:00:35.817317+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:20.171730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.111861+0000","last_change":"2024-04-24T23:00:32.019334+0000","last_active":"2024-04-24T23:00:35.111861+0000","last_peered":"2024-04-24T23:00:35.111861+0000","last_clean":"2024-04-24T23:00:35.111861+0000","last_became_active":"2024-04-24T23:00:32.019103+0000","last_became_peered":"2024-04-24T23:00:32.019103+0000","last_unstale":"2024-04-24T23:00:35.111861+0000","last_undegraded":"2024-04-24T23:00:35.111861+0000","last_fullsized":"2024-04-24T23:00:35.111861+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:50:21.381560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.111769+0000","last_change":"2024-04-24T23:00:33.021125+0000","last_active":"2024-04-24T23:00:35.111769+0000","last_peered":"2024-04-24T23:00:35.111769+0000","last_clean":"2024-04-24T23:00:35.111769+0000","last_became_active":"2024-04-24T23:00:33.020882+0000","last_became_peered":"2024-04-24T23:00:33.020882+0000","last_unstale":"2024-04-24T23:00:35.111769+0000","last_undegraded":"2024-04-24T23:00:35.111769+0000","last_fullsized":"2024-04-24T23:00:35.111769+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:10:05.572925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.014867+0000","last_change":"2024-04-24T23:00:32.015023+0000","last_active":"2024-04-24T23:00:34.014867+0000","last_peered":"2024-04-24T23:00:34.014867+0000","last_clean":"2024-04-24T23:00:34.014867+0000","last_became_active":"2024-04-24T23:00:32.014455+0000","last_became_peered":"2024-04-24T23:00:32.014455+0000","last_unstale":"2024-04-24T23:00:34.014867+0000","last_undegraded":"2024-04-24T23:00:34.014867+0000","last_fullsized":"2024-04-24T23:00:34.014867+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:12:05.738094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.014641+0000","last_change":"2024-04-24T23:00:33.028691+0000","last_active":"2024-04-24T23:00:34.014641+0000","last_peered":"2024-04-24T23:00:34.014641+0000","last_clean":"2024-04-24T23:00:34.014641+0000","last_became_active":"2024-04-24T23:00:33.028122+0000","last_became_peered":"2024-04-24T23:00:33.028122+0000","last_unstale":"2024-04-24T23:00:34.014641+0000","last_undegraded":"2024-04-24T23:00:34.014641+0000","last_fullsized":"2024-04-24T23:00:34.014641+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:05:35.652787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.014850+0000","last_change":"2024-04-24T23:00:32.019708+0000","last_active":"2024-04-24T23:00:34.014850+0000","last_peered":"2024-04-24T23:00:34.014850+0000","last_clean":"2024-04-24T23:00:34.014850+0000","last_became_active":"2024-04-24T23:00:32.019473+0000","last_became_peered":"2024-04-24T23:00:32.019473+0000","last_unstale":"2024-04-24T23:00:34.014850+0000","last_undegraded":"2024-04-24T23:00:34.014850+0000","last_fullsized":"2024-04-24T23:00:34.014850+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:30.220564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.014729+0000","last_change":"2024-04-24T23:00:33.018734+0000","last_active":"2024-04-24T23:00:34.014729+0000","last_peered":"2024-04-24T23:00:34.014729+0000","last_clean":"2024-04-24T23:00:34.014729+0000","last_became_active":"2024-04-24T23:00:33.018493+0000","last_became_peered":"2024-04-24T23:00:33.018493+0000","last_unstale":"2024-04-24T23:00:34.014729+0000","last_undegraded":"2024-04-24T23:00:34.014729+0000","last_fullsized":"2024-04-24T23:00:34.014729+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:04:52.434732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.817258+0000","last_change":"2024-04-24T23:00:32.015397+0000","last_active":"2024-04-24T23:00:35.817258+0000","last_peered":"2024-04-24T23:00:35.817258+0000","last_clean":"2024-04-24T23:00:35.817258+0000","last_became_active":"2024-04-24T23:00:32.015154+0000","last_became_peered":"2024-04-24T23:00:32.015154+0000","last_unstale":"2024-04-24T23:00:35.817258+0000","last_undegraded":"2024-04-24T23:00:35.817258+0000","last_fullsized":"2024-04-24T23:00:35.817258+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:09:22.441674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.086914+0000","last_change":"2024-04-24T23:00:33.023752+0000","last_active":"2024-04-24T23:00:34.086914+0000","last_peered":"2024-04-24T23:00:34.086914+0000","last_clean":"2024-04-24T23:00:34.086914+0000","last_became_active":"2024-04-24T23:00:33.023405+0000","last_became_peered":"2024-04-24T23:00:33.023405+0000","last_unstale":"2024-04-24T23:00:34.086914+0000","last_undegraded":"2024-04-24T23:00:34.086914+0000","last_fullsized":"2024-04-24T23:00:34.086914+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:04:55.906206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.015009+0000","last_change":"2024-04-24T23:00:32.017383+0000","last_active":"2024-04-24T23:00:34.015009+0000","last_peered":"2024-04-24T23:00:34.015009+0000","last_clean":"2024-04-24T23:00:34.015009+0000","last_became_active":"2024-04-24T23:00:32.016962+0000","last_became_peered":"2024-04-24T23:00:32.016962+0000","last_unstale":"2024-04-24T23:00:34.015009+0000","last_undegraded":"2024-04-24T23:00:34.015009+0000","last_fullsized":"2024-04-24T23:00:34.015009+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:47:07.143610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.086870+0000","last_change":"2024-04-24T23:00:33.025709+0000","last_active":"2024-04-24T23:00:34.086870+0000","last_peered":"2024-04-24T23:00:34.086870+0000","last_clean":"2024-04-24T23:00:34.086870+0000","last_became_active":"2024-04-24T23:00:33.025441+0000","last_became_peered":"2024-04-24T23:00:33.025441+0000","last_unstale":"2024-04-24T23:00:34.086870+0000","last_undegraded":"2024-04-24T23:00:34.086870+0000","last_fullsized":"2024-04-24T23:00:34.086870+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:54:46.938217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.010911+0000","last_change":"2024-04-24T23:00:32.011951+0000","last_active":"2024-04-24T23:00:34.010911+0000","last_peered":"2024-04-24T23:00:34.010911+0000","last_clean":"2024-04-24T23:00:34.010911+0000","last_became_active":"2024-04-24T23:00:32.011483+0000","last_became_peered":"2024-04-24T23:00:32.011483+0000","last_unstale":"2024-04-24T23:00:34.010911+0000","last_undegraded":"2024-04-24T23:00:34.010911+0000","last_fullsized":"2024-04-24T23:00:34.010911+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:07.472930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.011175+0000","last_change":"2024-04-24T23:00:33.012962+0000","last_active":"2024-04-24T23:00:34.011175+0000","last_peered":"2024-04-24T23:00:34.011175+0000","last_clean":"2024-04-24T23:00:34.011175+0000","last_became_active":"2024-04-24T23:00:33.012675+0000","last_became_peered":"2024-04-24T23:00:33.012675+0000","last_unstale":"2024-04-24T23:00:34.011175+0000","last_undegraded":"2024-04-24T23:00:34.011175+0000","last_fullsized":"2024-04-24T23:00:34.011175+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:40:06.285416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"45'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.086870+0000","last_change":"2024-04-24T23:00:32.016280+0000","last_active":"2024-04-24T23:00:34.086870+0000","last_peered":"2024-04-24T23:00:34.086870+0000","last_clean":"2024-04-24T23:00:34.086870+0000","last_became_active":"2024-04-24T23:00:32.016057+0000","last_became_peered":"2024-04-24T23:00:32.016057+0000","last_unstale":"2024-04-24T23:00:34.086870+0000","last_undegraded":"2024-04-24T23:00:34.086870+0000","last_fullsized":"2024-04-24T23:00:34.086870+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:27.231505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.014640+0000","last_change":"2024-04-24T23:00:33.025946+0000","last_active":"2024-04-24T23:00:34.014640+0000","last_peered":"2024-04-24T23:00:34.014640+0000","last_clean":"2024-04-24T23:00:34.014640+0000","last_became_active":"2024-04-24T23:00:33.024916+0000","last_became_peered":"2024-04-24T23:00:33.024916+0000","last_unstale":"2024-04-24T23:00:34.014640+0000","last_undegraded":"2024-04-24T23:00:34.014640+0000","last_fullsized":"2024-04-24T23:00:34.014640+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:41:24.055196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.817152+0000","last_change":"2024-04-24T23:00:32.015930+0000","last_active":"2024-04-24T23:00:35.817152+0000","last_peered":"2024-04-24T23:00:35.817152+0000","last_clean":"2024-04-24T23:00:35.817152+0000","last_became_active":"2024-04-24T23:00:32.015671+0000","last_became_peered":"2024-04-24T23:00:32.015671+0000","last_unstale":"2024-04-24T23:00:35.817152+0000","last_undegraded":"2024-04-24T23:00:35.817152+0000","last_fullsized":"2024-04-24T23:00:35.817152+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:45.511844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.011043+0000","last_change":"2024-04-24T23:00:33.028161+0000","last_active":"2024-04-24T23:00:34.011043+0000","last_peered":"2024-04-24T23:00:34.011043+0000","last_clean":"2024-04-24T23:00:34.011043+0000","last_became_active":"2024-04-24T23:00:33.027100+0000","last_became_peered":"2024-04-24T23:00:33.027100+0000","last_unstale":"2024-04-24T23:00:34.011043+0000","last_undegraded":"2024-04-24T23:00:34.011043+0000","last_fullsized":"2024-04-24T23:00:34.011043+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:45:20.571133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.112072+0000","last_change":"2024-04-24T23:00:32.020060+0000","last_active":"2024-04-24T23:00:35.112072+0000","last_peered":"2024-04-24T23:00:35.112072+0000","last_clean":"2024-04-24T23:00:35.112072+0000","last_became_active":"2024-04-24T23:00:32.019783+0000","last_became_peered":"2024-04-24T23:00:32.019783+0000","last_unstale":"2024-04-24T23:00:35.112072+0000","last_undegraded":"2024-04-24T23:00:35.112072+0000","last_fullsized":"2024-04-24T23:00:35.112072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:28:38.519563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.014254+0000","last_change":"2024-04-24T23:00:33.024647+0000","last_active":"2024-04-24T23:00:34.014254+0000","last_peered":"2024-04-24T23:00:34.014254+0000","last_clean":"2024-04-24T23:00:34.014254+0000","last_became_active":"2024-04-24T23:00:33.024326+0000","last_became_peered":"2024-04-24T23:00:33.024326+0000","last_unstale":"2024-04-24T23:00:34.014254+0000","last_undegraded":"2024-04-24T23:00:34.014254+0000","last_fullsized":"2024-04-24T23:00:34.014254+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:04.949857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.817365+0000","last_change":"2024-04-24T23:00:32.020217+0000","last_active":"2024-04-24T23:00:35.817365+0000","last_peered":"2024-04-24T23:00:35.817365+0000","last_clean":"2024-04-24T23:00:35.817365+0000","last_became_active":"2024-04-24T23:00:32.019728+0000","last_became_peered":"2024-04-24T23:00:32.019728+0000","last_unstale":"2024-04-24T23:00:35.817365+0000","last_undegraded":"2024-04-24T23:00:35.817365+0000","last_fullsized":"2024-04-24T23:00:35.817365+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:27:05.085107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.087010+0000","last_change":"2024-04-24T23:00:33.030169+0000","last_active":"2024-04-24T23:00:34.087010+0000","last_peered":"2024-04-24T23:00:34.087010+0000","last_clean":"2024-04-24T23:00:34.087010+0000","last_became_active":"2024-04-24T23:00:33.029597+0000","last_became_peered":"2024-04-24T23:00:33.029597+0000","last_unstale":"2024-04-24T23:00:34.087010+0000","last_undegraded":"2024-04-24T23:00:34.087010+0000","last_fullsized":"2024-04-24T23:00:34.087010+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:07:05.050164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.011354+0000","last_change":"2024-04-24T23:00:32.018725+0000","last_active":"2024-04-24T23:00:34.011354+0000","last_peered":"2024-04-24T23:00:34.011354+0000","last_clean":"2024-04-24T23:00:34.011354+0000","last_became_active":"2024-04-24T23:00:32.018472+0000","last_became_peered":"2024-04-24T23:00:32.018472+0000","last_unstale":"2024-04-24T23:00:34.011354+0000","last_undegraded":"2024-04-24T23:00:34.011354+0000","last_fullsized":"2024-04-24T23:00:34.011354+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:16:36.870866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.111406+0000","last_change":"2024-04-24T23:00:33.023847+0000","last_active":"2024-04-24T23:00:35.111406+0000","last_peered":"2024-04-24T23:00:35.111406+0000","last_clean":"2024-04-24T23:00:35.111406+0000","last_became_active":"2024-04-24T23:00:33.023599+0000","last_became_peered":"2024-04-24T23:00:33.023599+0000","last_unstale":"2024-04-24T23:00:35.111406+0000","last_undegraded":"2024-04-24T23:00:35.111406+0000","last_fullsized":"2024-04-24T23:00:35.111406+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:46.077303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.014933+0000","last_change":"2024-04-24T23:00:32.011954+0000","last_active":"2024-04-24T23:00:34.014933+0000","last_peered":"2024-04-24T23:00:34.014933+0000","last_clean":"2024-04-24T23:00:34.014933+0000","last_became_active":"2024-04-24T23:00:32.011189+0000","last_became_peered":"2024-04-24T23:00:32.011189+0000","last_unstale":"2024-04-24T23:00:34.014933+0000","last_undegraded":"2024-04-24T23:00:34.014933+0000","last_fullsized":"2024-04-24T23:00:34.014933+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:05:28.926498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.816849+0000","last_change":"2024-04-24T23:00:33.025596+0000","last_active":"2024-04-24T23:00:35.816849+0000","last_peered":"2024-04-24T23:00:35.816849+0000","last_clean":"2024-04-24T23:00:35.816849+0000","last_became_active":"2024-04-24T23:00:33.025348+0000","last_became_peered":"2024-04-24T23:00:33.025348+0000","last_unstale":"2024-04-24T23:00:35.816849+0000","last_undegraded":"2024-04-24T23:00:35.816849+0000","last_fullsized":"2024-04-24T23:00:35.816849+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:44:07.066426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.111892+0000","last_change":"2024-04-24T23:00:32.019965+0000","last_active":"2024-04-24T23:00:35.111892+0000","last_peered":"2024-04-24T23:00:35.111892+0000","last_clean":"2024-04-24T23:00:35.111892+0000","last_became_active":"2024-04-24T23:00:32.019691+0000","last_became_peered":"2024-04-24T23:00:32.019691+0000","last_unstale":"2024-04-24T23:00:35.111892+0000","last_undegraded":"2024-04-24T23:00:35.111892+0000","last_fullsized":"2024-04-24T23:00:35.111892+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:05:21.719262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.086660+0000","last_change":"2024-04-24T23:00:33.023228+0000","last_active":"2024-04-24T23:00:34.086660+0000","last_peered":"2024-04-24T23:00:34.086660+0000","last_clean":"2024-04-24T23:00:34.086660+0000","last_became_active":"2024-04-24T23:00:33.022394+0000","last_became_peered":"2024-04-24T23:00:33.022394+0000","last_unstale":"2024-04-24T23:00:34.086660+0000","last_undegraded":"2024-04-24T23:00:34.086660+0000","last_fullsized":"2024-04-24T23:00:34.086660+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:22:50.530894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.011260+0000","last_change":"2024-04-24T23:00:32.016499+0000","last_active":"2024-04-24T23:00:34.011260+0000","last_peered":"2024-04-24T23:00:34.011260+0000","last_clean":"2024-04-24T23:00:34.011260+0000","last_became_active":"2024-04-24T23:00:32.016247+0000","last_became_peered":"2024-04-24T23:00:32.016247+0000","last_unstale":"2024-04-24T23:00:34.011260+0000","last_undegraded":"2024-04-24T23:00:34.011260+0000","last_fullsized":"2024-04-24T23:00:34.011260+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:44:04.110059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":213,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.111524+0000","last_change":"2024-04-24T22:58:55.138810+0000","last_active":"2024-04-24T23:00:35.111524+0000","last_peered":"2024-04-24T23:00:35.111524+0000","last_clean":"2024-04-24T23:00:35.111524+0000","last_became_active":"2024-04-24T22:58:55.138627+0000","last_became_peered":"2024-04-24T22:58:55.138627+0000","last_unstale":"2024-04-24T23:00:35.111524+0000","last_undegraded":"2024-04-24T23:00:35.111524+0000","last_fullsized":"2024-04-24T23:00:35.111524+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_clean_scrub_stamp":"2024-04-24T22:58:54.127235+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:24.010002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":208,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195103,"statfs":{"total":200000000,"available":199786303,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":209,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195103,"statfs":{"total":200000000,"available":199785721,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":33,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":785,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194527,"statfs":{"total":200000000,"available":199195947,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":210,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195102,"statfs":{"total":200000000,"available":199784633,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":210,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195102,"statfs":{"total":200000000,"available":199784725,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":14,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":208,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195103,"statfs":{"total":200000000,"available":199786247,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":785,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194527,"statfs":{"total":200000000,"available":199195899,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":209,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195103,"statfs":{"total":200000000,"available":199785665,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-24T23:00:40.474 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:00:40.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.587+0000 7fa4aa909640 1 Processor -- start 2024-04-24T23:00:40.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.587+0000 7fa4aa909640 1 -- start start 2024-04-24T23:00:40.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.588+0000 7fa4aa909640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa4a415fd60 0x7fa4a406c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:40.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.588+0000 7fa4aa909640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4a40b46c0 con 0x7fa4a406cb30 2024-04-24T23:00:40.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.588+0000 7fa4aa909640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4a40b49a0 con 0x7fa4a415b480 2024-04-24T23:00:40.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.588+0000 7fa4aa909640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa4a40b4ff0 con 0x7fa4a415fd60 2024-04-24T23:00:40.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.588+0000 7fa4a8ea6640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa4a406cb30 0x7fa4a40cdf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55086/0 (socket says 172.21.15.32:55086) 2024-04-24T23:00:40.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.588+0000 7fa4a8ea6640 1 -- 172.21.15.32:0/235695886 learned_addr learned my addr 172.21.15.32:0/235695886 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:00:40.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.588+0000 7fa4a37fe640 1 --2- 172.21.15.32:0/235695886 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa4a415fd60 0x7fa4a406c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:40.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.589+0000 7fa4a2ffd640 1 -- 172.21.15.32:0/235695886 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2780866096 0 0) 0x7fa4a40b46c0 con 0x7fa4a406cb30 2024-04-24T23:00:40.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.589+0000 7fa4a2ffd640 1 -- 172.21.15.32:0/235695886 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa490003520 con 0x7fa4a406cb30 2024-04-24T23:00:40.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.589+0000 7fa4a2ffd640 1 -- 172.21.15.32:0/235695886 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1200781276 0 0) 0x7fa4a40b49a0 con 0x7fa4a415b480 2024-04-24T23:00:40.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.589+0000 7fa4a2ffd640 1 -- 172.21.15.32:0/235695886 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa4a40b46c0 con 0x7fa4a415b480 2024-04-24T23:00:40.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.589+0000 7fa4a37fe640 1 -- 172.21.15.32:0/235695886 >> v1:172.21.15.121:6789/0 conn(0x7fa4a415b480 legacy=0x7fa4a40b5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:40.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.589+0000 7fa4a37fe640 1 -- 172.21.15.32:0/235695886 >> v1:172.21.15.32:6789/0 conn(0x7fa4a406cb30 legacy=0x7fa4a40cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:40.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.589+0000 7fa4a37fe640 1 -- 172.21.15.32:0/235695886 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4a40b4c80 con 0x7fa4a415fd60 2024-04-24T23:00:40.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.589+0000 7fa4a37fe640 1 --2- 172.21.15.32:0/235695886 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa4a415fd60 0x7fa4a406c5f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa4940097d0 tx=0x7fa49402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50423e3498fc3199 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:40.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.590+0000 7fa4a2ffd640 1 -- 172.21.15.32:0/235695886 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa494037070 con 0x7fa4a415fd60 2024-04-24T23:00:40.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.590+0000 7fa4a2ffd640 1 -- 172.21.15.32:0/235695886 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa494032440 con 0x7fa4a415fd60 2024-04-24T23:00:40.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.590+0000 7fa4a2ffd640 1 -- 172.21.15.32:0/235695886 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa494032810 con 0x7fa4a415fd60 2024-04-24T23:00:40.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.590+0000 7fa4aa909640 1 -- 172.21.15.32:0/235695886 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa4a415fd60 msgr2=0x7fa4a406c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:40.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.590+0000 7fa4aa909640 1 --2- 172.21.15.32:0/235695886 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa4a415fd60 0x7fa4a406c5f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa4940097d0 tx=0x7fa49402a210 comp rx=0 tx=0).stop 2024-04-24T23:00:40.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.590+0000 7fa4aa909640 1 -- 172.21.15.32:0/235695886 shutdown_connections 2024-04-24T23:00:40.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.590+0000 7fa4aa909640 1 --2- 172.21.15.32:0/235695886 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa4a415fd60 0x7fa4a406c5f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:40.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.590+0000 7fa4aa909640 1 -- 172.21.15.32:0/235695886 >> 172.21.15.32:0/235695886 conn(0x7fa4a406b110 msgr2=0x7fa4a406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:40.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.590+0000 7fa4aa909640 1 -- 172.21.15.32:0/235695886 shutdown_connections 2024-04-24T23:00:40.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.590+0000 7fa4aa909640 1 -- 172.21.15.32:0/235695886 wait complete. 2024-04-24T23:00:40.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.591+0000 7fa4aa909640 1 Processor -- start 2024-04-24T23:00:40.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.592+0000 7fa4aa909640 1 -- start start 2024-04-24T23:00:40.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.592+0000 7fa4aa909640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa4a406cb30 0x7fa4a41f5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:40.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.592+0000 7fa4aa909640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4a415b480 0x7fa4a4148ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:40.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.592+0000 7fa4aa909640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4a415fd60 0x7fa4a415df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:40.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.592+0000 7fa4aa909640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a415b860 con 0x7fa4a415b480 2024-04-24T23:00:40.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.592+0000 7fa4aa909640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a40ce690 con 0x7fa4a415fd60 2024-04-24T23:00:40.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.592+0000 7fa4aa909640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa4a4156fe0 con 0x7fa4a406cb30 2024-04-24T23:00:40.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.592+0000 7fa4a37fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4a415b480 0x7fa4a4148ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:40.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.592+0000 7fa4a37fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4a415b480 0x7fa4a4148ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53394/0 (socket says 172.21.15.32:53394) 2024-04-24T23:00:40.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.592+0000 7fa4a37fe640 1 -- 172.21.15.32:0/2272477155 learned_addr learned my addr 172.21.15.32:0/2272477155 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:40.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.593+0000 7fa4a3fff640 1 --2- 172.21.15.32:0/2272477155 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa4a406cb30 0x7fa4a41f5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:40.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.593+0000 7fa4a8ea6640 1 --2- 172.21.15.32:0/2272477155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4a415fd60 0x7fa4a415df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:40.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.593+0000 7fa4a37fe640 1 -- 172.21.15.32:0/2272477155 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa4a406cb30 msgr2=0x7fa4a41f5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:40.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.593+0000 7fa4a37fe640 1 --2- 172.21.15.32:0/2272477155 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa4a406cb30 0x7fa4a41f5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:40.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.593+0000 7fa4a37fe640 1 -- 172.21.15.32:0/2272477155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4a415fd60 msgr2=0x7fa4a415df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:40.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.593+0000 7fa4a37fe640 1 --2- 172.21.15.32:0/2272477155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4a415fd60 0x7fa4a415df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:40.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.593+0000 7fa4a37fe640 1 -- 172.21.15.32:0/2272477155 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4a41554b0 con 0x7fa4a415b480 2024-04-24T23:00:40.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.593+0000 7fa4a37fe640 1 --2- 172.21.15.32:0/2272477155 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4a415b480 0x7fa4a4148ca0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fa494035040 tx=0x7fa4940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:40.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.593+0000 7fa4a0ff9640 1 -- 172.21.15.32:0/2272477155 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa494041020 con 0x7fa4a415b480 2024-04-24T23:00:40.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.593+0000 7fa4a0ff9640 1 -- 172.21.15.32:0/2272477155 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa49403d720 con 0x7fa4a415b480 2024-04-24T23:00:40.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.593+0000 7fa4aa909640 1 -- 172.21.15.32:0/2272477155 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4a4155790 con 0x7fa4a415b480 2024-04-24T23:00:40.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.593+0000 7fa4a0ff9640 1 -- 172.21.15.32:0/2272477155 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa494037040 con 0x7fa4a415b480 2024-04-24T23:00:40.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.593+0000 7fa4aa909640 1 -- 172.21.15.32:0/2272477155 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4a4155d50 con 0x7fa4a415b480 2024-04-24T23:00:40.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.594+0000 7fa4aa909640 1 -- 172.21.15.32:0/2272477155 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4a4068430 con 0x7fa4a415b480 2024-04-24T23:00:40.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.595+0000 7fa4a0ff9640 1 -- 172.21.15.32:0/2272477155 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa494044020 con 0x7fa4a415b480 2024-04-24T23:00:40.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.595+0000 7fa4a0ff9640 1 --2- 172.21.15.32:0/2272477155 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa48406be30 0x7fa48406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:40.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.596+0000 7fa4a0ff9640 1 -- 172.21.15.32:0/2272477155 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fa4940ab4d0 con 0x7fa4a415b480 2024-04-24T23:00:40.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.596+0000 7fa4a3fff640 1 --2- 172.21.15.32:0/2272477155 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa48406be30 0x7fa48406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:40.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.596+0000 7fa4a3fff640 1 --2- 172.21.15.32:0/2272477155 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa48406be30 0x7fa48406e2f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa48c0011e0 tx=0x7fa48c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:40.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.600+0000 7fa4a0ff9640 1 -- 172.21.15.32:0/2272477155 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa49404b030 con 0x7fa4a415b480 2024-04-24T23:00:40.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.778+0000 7fa4aa909640 1 -- 172.21.15.32:0/2272477155 --> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa4a40b1df0 con 0x7fa48406be30 2024-04-24T23:00:40.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.785+0000 7fa4a0ff9640 1 -- 172.21.15.32:0/2272477155 <== mgr.4108 v2:172.21.15.121:6832/1848157943 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247491 (secure 0 0 0) 0x7fa4a40b1df0 con 0x7fa48406be30 2024-04-24T23:00:40.789 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:00:40.791 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-24T23:00:40.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.788+0000 7fa4aa909640 1 -- 172.21.15.32:0/2272477155 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa48406be30 msgr2=0x7fa48406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:40.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.788+0000 7fa4aa909640 1 --2- 172.21.15.32:0/2272477155 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa48406be30 0x7fa48406e2f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa48c0011e0 tx=0x7fa48c009040 comp rx=0 tx=0).stop 2024-04-24T23:00:40.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.788+0000 7fa4aa909640 1 -- 172.21.15.32:0/2272477155 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4a415b480 msgr2=0x7fa4a4148ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:40.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.788+0000 7fa4aa909640 1 --2- 172.21.15.32:0/2272477155 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4a415b480 0x7fa4a4148ca0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fa494035040 tx=0x7fa4940046c0 comp rx=0 tx=0).stop 2024-04-24T23:00:40.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.788+0000 7fa4aa909640 1 -- 172.21.15.32:0/2272477155 shutdown_connections 2024-04-24T23:00:40.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.788+0000 7fa4aa909640 1 --2- 172.21.15.32:0/2272477155 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa48406be30 0x7fa48406e2f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:40.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.788+0000 7fa4aa909640 1 --2- 172.21.15.32:0/2272477155 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4a415b480 0x7fa4a4148ca0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:40.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.788+0000 7fa4aa909640 1 --2- 172.21.15.32:0/2272477155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4a415fd60 0x7fa4a415df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:40.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.788+0000 7fa4aa909640 1 --2- 172.21.15.32:0/2272477155 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa4a406cb30 0x7fa4a41f5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:40.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.788+0000 7fa4aa909640 1 -- 172.21.15.32:0/2272477155 >> 172.21.15.32:0/2272477155 conn(0x7fa4a406b110 msgr2=0x7fa4a40b5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:40.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.788+0000 7fa4aa909640 1 -- 172.21.15.32:0/2272477155 shutdown_connections 2024-04-24T23:00:40.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.788+0000 7fa4aa909640 1 -- 172.21.15.32:0/2272477155 wait complete. 2024-04-24T23:00:40.800 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":90,"stamp":"2024-04-24T23:00:39.148804+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":186,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2827,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1559672,"statfs":{"total":1600000000,"available":1597105140,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.140969"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.817034+0000","last_change":"2024-04-24T23:00:33.023527+0000","last_active":"2024-04-24T23:00:35.817034+0000","last_peered":"2024-04-24T23:00:35.817034+0000","last_clean":"2024-04-24T23:00:35.817034+0000","last_became_active":"2024-04-24T23:00:33.023243+0000","last_became_peered":"2024-04-24T23:00:33.023243+0000","last_unstale":"2024-04-24T23:00:35.817034+0000","last_undegraded":"2024-04-24T23:00:35.817034+0000","last_fullsized":"2024-04-24T23:00:35.817034+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:22:32.712700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.816666+0000","last_change":"2024-04-24T23:00:33.021299+0000","last_active":"2024-04-24T23:00:35.816666+0000","last_peered":"2024-04-24T23:00:35.816666+0000","last_clean":"2024-04-24T23:00:35.816666+0000","last_became_active":"2024-04-24T23:00:33.021010+0000","last_became_peered":"2024-04-24T23:00:33.021010+0000","last_unstale":"2024-04-24T23:00:35.816666+0000","last_undegraded":"2024-04-24T23:00:35.816666+0000","last_fullsized":"2024-04-24T23:00:35.816666+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:15:35.010392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.111511+0000","last_change":"2024-04-24T23:00:33.025927+0000","last_active":"2024-04-24T23:00:35.111511+0000","last_peered":"2024-04-24T23:00:35.111511+0000","last_clean":"2024-04-24T23:00:35.111511+0000","last_became_active":"2024-04-24T23:00:33.025699+0000","last_became_peered":"2024-04-24T23:00:33.025699+0000","last_unstale":"2024-04-24T23:00:35.111511+0000","last_undegraded":"2024-04-24T23:00:35.111511+0000","last_fullsized":"2024-04-24T23:00:35.111511+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:07.115951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.086601+0000","last_change":"2024-04-24T23:00:33.029803+0000","last_active":"2024-04-24T23:00:34.086601+0000","last_peered":"2024-04-24T23:00:34.086601+0000","last_clean":"2024-04-24T23:00:34.086601+0000","last_became_active":"2024-04-24T23:00:33.029528+0000","last_became_peered":"2024-04-24T23:00:33.029528+0000","last_unstale":"2024-04-24T23:00:34.086601+0000","last_undegraded":"2024-04-24T23:00:34.086601+0000","last_fullsized":"2024-04-24T23:00:34.086601+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:38:46.179892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.817120+0000","last_change":"2024-04-24T23:00:33.022364+0000","last_active":"2024-04-24T23:00:35.817120+0000","last_peered":"2024-04-24T23:00:35.817120+0000","last_clean":"2024-04-24T23:00:35.817120+0000","last_became_active":"2024-04-24T23:00:33.021999+0000","last_became_peered":"2024-04-24T23:00:33.021999+0000","last_unstale":"2024-04-24T23:00:35.817120+0000","last_undegraded":"2024-04-24T23:00:35.817120+0000","last_fullsized":"2024-04-24T23:00:35.817120+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:20:05.537797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.014252+0000","last_change":"2024-04-24T23:00:33.021960+0000","last_active":"2024-04-24T23:00:34.014252+0000","last_peered":"2024-04-24T23:00:34.014252+0000","last_clean":"2024-04-24T23:00:34.014252+0000","last_became_active":"2024-04-24T23:00:33.021507+0000","last_became_peered":"2024-04-24T23:00:33.021507+0000","last_unstale":"2024-04-24T23:00:34.014252+0000","last_undegraded":"2024-04-24T23:00:34.014252+0000","last_fullsized":"2024-04-24T23:00:34.014252+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:26:00.374805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.010873+0000","last_change":"2024-04-24T23:00:33.024126+0000","last_active":"2024-04-24T23:00:34.010873+0000","last_peered":"2024-04-24T23:00:34.010873+0000","last_clean":"2024-04-24T23:00:34.010873+0000","last_became_active":"2024-04-24T23:00:33.023881+0000","last_became_peered":"2024-04-24T23:00:33.023881+0000","last_unstale":"2024-04-24T23:00:34.010873+0000","last_undegraded":"2024-04-24T23:00:34.010873+0000","last_fullsized":"2024-04-24T23:00:34.010873+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:51:10.777829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.086491+0000","last_change":"2024-04-24T23:00:33.023498+0000","last_active":"2024-04-24T23:00:34.086491+0000","last_peered":"2024-04-24T23:00:34.086491+0000","last_clean":"2024-04-24T23:00:34.086491+0000","last_became_active":"2024-04-24T23:00:33.022718+0000","last_became_peered":"2024-04-24T23:00:33.022718+0000","last_unstale":"2024-04-24T23:00:34.086491+0000","last_undegraded":"2024-04-24T23:00:34.086491+0000","last_fullsized":"2024-04-24T23:00:34.086491+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:44:47.688330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.014235+0000","last_change":"2024-04-24T23:00:33.029934+0000","last_active":"2024-04-24T23:00:34.014235+0000","last_peered":"2024-04-24T23:00:34.014235+0000","last_clean":"2024-04-24T23:00:34.014235+0000","last_became_active":"2024-04-24T23:00:33.029609+0000","last_became_peered":"2024-04-24T23:00:33.029609+0000","last_unstale":"2024-04-24T23:00:34.014235+0000","last_undegraded":"2024-04-24T23:00:34.014235+0000","last_fullsized":"2024-04-24T23:00:34.014235+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:05.664812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.014309+0000","last_change":"2024-04-24T23:00:33.026698+0000","last_active":"2024-04-24T23:00:34.014309+0000","last_peered":"2024-04-24T23:00:34.014309+0000","last_clean":"2024-04-24T23:00:34.014309+0000","last_became_active":"2024-04-24T23:00:33.025699+0000","last_became_peered":"2024-04-24T23:00:33.025699+0000","last_unstale":"2024-04-24T23:00:34.014309+0000","last_undegraded":"2024-04-24T23:00:34.014309+0000","last_fullsized":"2024-04-24T23:00:34.014309+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:01:08.342156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.086658+0000","last_change":"2024-04-24T23:00:33.020762+0000","last_active":"2024-04-24T23:00:34.086658+0000","last_peered":"2024-04-24T23:00:34.086658+0000","last_clean":"2024-04-24T23:00:34.086658+0000","last_became_active":"2024-04-24T23:00:33.020263+0000","last_became_peered":"2024-04-24T23:00:33.020263+0000","last_unstale":"2024-04-24T23:00:34.086658+0000","last_undegraded":"2024-04-24T23:00:34.086658+0000","last_fullsized":"2024-04-24T23:00:34.086658+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:19:52.368532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.816831+0000","last_change":"2024-04-24T23:00:33.027989+0000","last_active":"2024-04-24T23:00:35.816831+0000","last_peered":"2024-04-24T23:00:35.816831+0000","last_clean":"2024-04-24T23:00:35.816831+0000","last_became_active":"2024-04-24T23:00:33.025402+0000","last_became_peered":"2024-04-24T23:00:33.025402+0000","last_unstale":"2024-04-24T23:00:35.816831+0000","last_undegraded":"2024-04-24T23:00:35.816831+0000","last_fullsized":"2024-04-24T23:00:35.816831+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:10:21.505047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.014578+0000","last_change":"2024-04-24T23:00:33.021111+0000","last_active":"2024-04-24T23:00:34.014578+0000","last_peered":"2024-04-24T23:00:34.014578+0000","last_clean":"2024-04-24T23:00:34.014578+0000","last_became_active":"2024-04-24T23:00:33.020822+0000","last_became_peered":"2024-04-24T23:00:33.020822+0000","last_unstale":"2024-04-24T23:00:34.014578+0000","last_undegraded":"2024-04-24T23:00:34.014578+0000","last_fullsized":"2024-04-24T23:00:34.014578+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:39:32.044154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.086862+0000","last_change":"2024-04-24T23:00:33.032538+0000","last_active":"2024-04-24T23:00:34.086862+0000","last_peered":"2024-04-24T23:00:34.086862+0000","last_clean":"2024-04-24T23:00:34.086862+0000","last_became_active":"2024-04-24T23:00:33.032060+0000","last_became_peered":"2024-04-24T23:00:33.032060+0000","last_unstale":"2024-04-24T23:00:34.086862+0000","last_undegraded":"2024-04-24T23:00:34.086862+0000","last_fullsized":"2024-04-24T23:00:34.086862+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:42:05.030592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.086547+0000","last_change":"2024-04-24T23:00:33.023972+0000","last_active":"2024-04-24T23:00:34.086547+0000","last_peered":"2024-04-24T23:00:34.086547+0000","last_clean":"2024-04-24T23:00:34.086547+0000","last_became_active":"2024-04-24T23:00:33.023742+0000","last_became_peered":"2024-04-24T23:00:33.023742+0000","last_unstale":"2024-04-24T23:00:34.086547+0000","last_undegraded":"2024-04-24T23:00:34.086547+0000","last_fullsized":"2024-04-24T23:00:34.086547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:16:14.948047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.817277+0000","last_change":"2024-04-24T23:00:33.022971+0000","last_active":"2024-04-24T23:00:35.817277+0000","last_peered":"2024-04-24T23:00:35.817277+0000","last_clean":"2024-04-24T23:00:35.817277+0000","last_became_active":"2024-04-24T23:00:33.022639+0000","last_became_peered":"2024-04-24T23:00:33.022639+0000","last_unstale":"2024-04-24T23:00:35.817277+0000","last_undegraded":"2024-04-24T23:00:35.817277+0000","last_fullsized":"2024-04-24T23:00:35.817277+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:58:17.058198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.816982+0000","last_change":"2024-04-24T23:00:33.028457+0000","last_active":"2024-04-24T23:00:35.816982+0000","last_peered":"2024-04-24T23:00:35.816982+0000","last_clean":"2024-04-24T23:00:35.816982+0000","last_became_active":"2024-04-24T23:00:33.027401+0000","last_became_peered":"2024-04-24T23:00:33.027401+0000","last_unstale":"2024-04-24T23:00:35.816982+0000","last_undegraded":"2024-04-24T23:00:35.816982+0000","last_fullsized":"2024-04-24T23:00:35.816982+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:33:50.382484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.111661+0000","last_change":"2024-04-24T23:00:33.027850+0000","last_active":"2024-04-24T23:00:35.111661+0000","last_peered":"2024-04-24T23:00:35.111661+0000","last_clean":"2024-04-24T23:00:35.111661+0000","last_became_active":"2024-04-24T23:00:33.027238+0000","last_became_peered":"2024-04-24T23:00:33.027238+0000","last_unstale":"2024-04-24T23:00:35.111661+0000","last_undegraded":"2024-04-24T23:00:35.111661+0000","last_fullsized":"2024-04-24T23:00:35.111661+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:49:59.031348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.817057+0000","last_change":"2024-04-24T23:00:33.020153+0000","last_active":"2024-04-24T23:00:35.817057+0000","last_peered":"2024-04-24T23:00:35.817057+0000","last_clean":"2024-04-24T23:00:35.817057+0000","last_became_active":"2024-04-24T23:00:33.019897+0000","last_became_peered":"2024-04-24T23:00:33.019897+0000","last_unstale":"2024-04-24T23:00:35.817057+0000","last_undegraded":"2024-04-24T23:00:35.817057+0000","last_fullsized":"2024-04-24T23:00:35.817057+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:33:56.003472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.014798+0000","last_change":"2024-04-24T23:00:33.026517+0000","last_active":"2024-04-24T23:00:34.014798+0000","last_peered":"2024-04-24T23:00:34.014798+0000","last_clean":"2024-04-24T23:00:34.014798+0000","last_became_active":"2024-04-24T23:00:33.025323+0000","last_became_peered":"2024-04-24T23:00:33.025323+0000","last_unstale":"2024-04-24T23:00:34.014798+0000","last_undegraded":"2024-04-24T23:00:34.014798+0000","last_fullsized":"2024-04-24T23:00:34.014798+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:12.264162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.087078+0000","last_change":"2024-04-24T23:00:33.029989+0000","last_active":"2024-04-24T23:00:34.087078+0000","last_peered":"2024-04-24T23:00:34.087078+0000","last_clean":"2024-04-24T23:00:34.087078+0000","last_became_active":"2024-04-24T23:00:33.029261+0000","last_became_peered":"2024-04-24T23:00:33.029261+0000","last_unstale":"2024-04-24T23:00:34.087078+0000","last_undegraded":"2024-04-24T23:00:34.087078+0000","last_fullsized":"2024-04-24T23:00:34.087078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:22:39.629270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.111977+0000","last_change":"2024-04-24T23:00:33.018131+0000","last_active":"2024-04-24T23:00:35.111977+0000","last_peered":"2024-04-24T23:00:35.111977+0000","last_clean":"2024-04-24T23:00:35.111977+0000","last_became_active":"2024-04-24T23:00:33.017878+0000","last_became_peered":"2024-04-24T23:00:33.017878+0000","last_unstale":"2024-04-24T23:00:35.111977+0000","last_undegraded":"2024-04-24T23:00:35.111977+0000","last_fullsized":"2024-04-24T23:00:35.111977+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:11:31.967754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.011441+0000","last_change":"2024-04-24T23:00:33.025102+0000","last_active":"2024-04-24T23:00:34.011441+0000","last_peered":"2024-04-24T23:00:34.011441+0000","last_clean":"2024-04-24T23:00:34.011441+0000","last_became_active":"2024-04-24T23:00:33.022808+0000","last_became_peered":"2024-04-24T23:00:33.022808+0000","last_unstale":"2024-04-24T23:00:34.011441+0000","last_undegraded":"2024-04-24T23:00:34.011441+0000","last_fullsized":"2024-04-24T23:00:34.011441+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:17:31.920100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.816640+0000","last_change":"2024-04-24T23:00:33.022669+0000","last_active":"2024-04-24T23:00:35.816640+0000","last_peered":"2024-04-24T23:00:35.816640+0000","last_clean":"2024-04-24T23:00:35.816640+0000","last_became_active":"2024-04-24T23:00:33.022385+0000","last_became_peered":"2024-04-24T23:00:33.022385+0000","last_unstale":"2024-04-24T23:00:35.816640+0000","last_undegraded":"2024-04-24T23:00:35.816640+0000","last_fullsized":"2024-04-24T23:00:35.816640+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:48:20.573793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.817207+0000","last_change":"2024-04-24T23:00:33.025403+0000","last_active":"2024-04-24T23:00:35.817207+0000","last_peered":"2024-04-24T23:00:35.817207+0000","last_clean":"2024-04-24T23:00:35.817207+0000","last_became_active":"2024-04-24T23:00:33.025072+0000","last_became_peered":"2024-04-24T23:00:33.025072+0000","last_unstale":"2024-04-24T23:00:35.817207+0000","last_undegraded":"2024-04-24T23:00:35.817207+0000","last_fullsized":"2024-04-24T23:00:35.817207+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:02.831738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.086814+0000","last_change":"2024-04-24T23:00:33.020508+0000","last_active":"2024-04-24T23:00:34.086814+0000","last_peered":"2024-04-24T23:00:34.086814+0000","last_clean":"2024-04-24T23:00:34.086814+0000","last_became_active":"2024-04-24T23:00:33.019950+0000","last_became_peered":"2024-04-24T23:00:33.019950+0000","last_unstale":"2024-04-24T23:00:34.086814+0000","last_undegraded":"2024-04-24T23:00:34.086814+0000","last_fullsized":"2024-04-24T23:00:34.086814+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:30:38.408344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.011035+0000","last_change":"2024-04-24T23:00:33.018305+0000","last_active":"2024-04-24T23:00:34.011035+0000","last_peered":"2024-04-24T23:00:34.011035+0000","last_clean":"2024-04-24T23:00:34.011035+0000","last_became_active":"2024-04-24T23:00:33.017968+0000","last_became_peered":"2024-04-24T23:00:33.017968+0000","last_unstale":"2024-04-24T23:00:34.011035+0000","last_undegraded":"2024-04-24T23:00:34.011035+0000","last_fullsized":"2024-04-24T23:00:34.011035+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:35:37.311179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.011127+0000","last_change":"2024-04-24T23:00:33.023042+0000","last_active":"2024-04-24T23:00:34.011127+0000","last_peered":"2024-04-24T23:00:34.011127+0000","last_clean":"2024-04-24T23:00:34.011127+0000","last_became_active":"2024-04-24T23:00:33.022480+0000","last_became_peered":"2024-04-24T23:00:33.022480+0000","last_unstale":"2024-04-24T23:00:34.011127+0000","last_undegraded":"2024-04-24T23:00:34.011127+0000","last_fullsized":"2024-04-24T23:00:34.011127+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:54:49.637246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.816806+0000","last_change":"2024-04-24T23:00:33.031873+0000","last_active":"2024-04-24T23:00:35.816806+0000","last_peered":"2024-04-24T23:00:35.816806+0000","last_clean":"2024-04-24T23:00:35.816806+0000","last_became_active":"2024-04-24T23:00:33.031636+0000","last_became_peered":"2024-04-24T23:00:33.031636+0000","last_unstale":"2024-04-24T23:00:35.816806+0000","last_undegraded":"2024-04-24T23:00:35.816806+0000","last_fullsized":"2024-04-24T23:00:35.816806+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:09.669278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.816981+0000","last_change":"2024-04-24T23:00:33.020565+0000","last_active":"2024-04-24T23:00:35.816981+0000","last_peered":"2024-04-24T23:00:35.816981+0000","last_clean":"2024-04-24T23:00:35.816981+0000","last_became_active":"2024-04-24T23:00:33.019812+0000","last_became_peered":"2024-04-24T23:00:33.019812+0000","last_unstale":"2024-04-24T23:00:35.816981+0000","last_undegraded":"2024-04-24T23:00:35.816981+0000","last_fullsized":"2024-04-24T23:00:35.816981+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:27:57.748785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.014405+0000","last_change":"2024-04-24T23:00:33.023994+0000","last_active":"2024-04-24T23:00:34.014405+0000","last_peered":"2024-04-24T23:00:34.014405+0000","last_clean":"2024-04-24T23:00:34.014405+0000","last_became_active":"2024-04-24T23:00:33.023755+0000","last_became_peered":"2024-04-24T23:00:33.023755+0000","last_unstale":"2024-04-24T23:00:34.014405+0000","last_undegraded":"2024-04-24T23:00:34.014405+0000","last_fullsized":"2024-04-24T23:00:34.014405+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:57:59.769840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.086585+0000","last_change":"2024-04-24T23:00:33.020190+0000","last_active":"2024-04-24T23:00:34.086585+0000","last_peered":"2024-04-24T23:00:34.086585+0000","last_clean":"2024-04-24T23:00:34.086585+0000","last_became_active":"2024-04-24T23:00:33.019894+0000","last_became_peered":"2024-04-24T23:00:33.019894+0000","last_unstale":"2024-04-24T23:00:34.086585+0000","last_undegraded":"2024-04-24T23:00:34.086585+0000","last_fullsized":"2024-04-24T23:00:34.086585+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:49:22.102434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.111670+0000","last_change":"2024-04-24T23:00:33.029760+0000","last_active":"2024-04-24T23:00:35.111670+0000","last_peered":"2024-04-24T23:00:35.111670+0000","last_clean":"2024-04-24T23:00:35.111670+0000","last_became_active":"2024-04-24T23:00:33.029171+0000","last_became_peered":"2024-04-24T23:00:33.029171+0000","last_unstale":"2024-04-24T23:00:35.111670+0000","last_undegraded":"2024-04-24T23:00:35.111670+0000","last_fullsized":"2024-04-24T23:00:35.111670+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:50:38.830879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"44'4","reported_seq":18,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.014821+0000","last_change":"2024-04-24T23:00:32.013964+0000","last_active":"2024-04-24T23:00:34.014821+0000","last_peered":"2024-04-24T23:00:34.014821+0000","last_clean":"2024-04-24T23:00:34.014821+0000","last_became_active":"2024-04-24T23:00:32.013661+0000","last_became_peered":"2024-04-24T23:00:32.013661+0000","last_unstale":"2024-04-24T23:00:34.014821+0000","last_undegraded":"2024-04-24T23:00:34.014821+0000","last_fullsized":"2024-04-24T23:00:34.014821+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:34:35.486621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.817554+0000","last_change":"2024-04-24T23:00:33.021778+0000","last_active":"2024-04-24T23:00:35.817554+0000","last_peered":"2024-04-24T23:00:35.817554+0000","last_clean":"2024-04-24T23:00:35.817554+0000","last_became_active":"2024-04-24T23:00:33.021385+0000","last_became_peered":"2024-04-24T23:00:33.021385+0000","last_unstale":"2024-04-24T23:00:35.817554+0000","last_undegraded":"2024-04-24T23:00:35.817554+0000","last_fullsized":"2024-04-24T23:00:35.817554+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:40:30.680600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.011199+0000","last_change":"2024-04-24T23:00:32.012770+0000","last_active":"2024-04-24T23:00:34.011199+0000","last_peered":"2024-04-24T23:00:34.011199+0000","last_clean":"2024-04-24T23:00:34.011199+0000","last_became_active":"2024-04-24T23:00:32.012092+0000","last_became_peered":"2024-04-24T23:00:32.012092+0000","last_unstale":"2024-04-24T23:00:34.011199+0000","last_undegraded":"2024-04-24T23:00:34.011199+0000","last_fullsized":"2024-04-24T23:00:34.011199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:49:58.754592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.817113+0000","last_change":"2024-04-24T23:00:33.021178+0000","last_active":"2024-04-24T23:00:35.817113+0000","last_peered":"2024-04-24T23:00:35.817113+0000","last_clean":"2024-04-24T23:00:35.817113+0000","last_became_active":"2024-04-24T23:00:33.020678+0000","last_became_peered":"2024-04-24T23:00:33.020678+0000","last_unstale":"2024-04-24T23:00:35.817113+0000","last_undegraded":"2024-04-24T23:00:35.817113+0000","last_fullsized":"2024-04-24T23:00:35.817113+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:58.669126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.014327+0000","last_change":"2024-04-24T23:00:32.015863+0000","last_active":"2024-04-24T23:00:34.014327+0000","last_peered":"2024-04-24T23:00:34.014327+0000","last_clean":"2024-04-24T23:00:34.014327+0000","last_became_active":"2024-04-24T23:00:32.015616+0000","last_became_peered":"2024-04-24T23:00:32.015616+0000","last_unstale":"2024-04-24T23:00:34.014327+0000","last_undegraded":"2024-04-24T23:00:34.014327+0000","last_fullsized":"2024-04-24T23:00:34.014327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:36:07.320735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.086716+0000","last_change":"2024-04-24T23:00:33.021120+0000","last_active":"2024-04-24T23:00:34.086716+0000","last_peered":"2024-04-24T23:00:34.086716+0000","last_clean":"2024-04-24T23:00:34.086716+0000","last_became_active":"2024-04-24T23:00:33.020671+0000","last_became_peered":"2024-04-24T23:00:33.020671+0000","last_unstale":"2024-04-24T23:00:34.086716+0000","last_undegraded":"2024-04-24T23:00:34.086716+0000","last_fullsized":"2024-04-24T23:00:34.086716+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:54:07.317918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.010827+0000","last_change":"2024-04-24T23:00:32.010390+0000","last_active":"2024-04-24T23:00:34.010827+0000","last_peered":"2024-04-24T23:00:34.010827+0000","last_clean":"2024-04-24T23:00:34.010827+0000","last_became_active":"2024-04-24T23:00:32.010156+0000","last_became_peered":"2024-04-24T23:00:32.010156+0000","last_unstale":"2024-04-24T23:00:34.010827+0000","last_undegraded":"2024-04-24T23:00:34.010827+0000","last_fullsized":"2024-04-24T23:00:34.010827+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:53.937003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.011256+0000","last_change":"2024-04-24T23:00:33.027125+0000","last_active":"2024-04-24T23:00:34.011256+0000","last_peered":"2024-04-24T23:00:34.011256+0000","last_clean":"2024-04-24T23:00:34.011256+0000","last_became_active":"2024-04-24T23:00:33.026356+0000","last_became_peered":"2024-04-24T23:00:33.026356+0000","last_unstale":"2024-04-24T23:00:34.011256+0000","last_undegraded":"2024-04-24T23:00:34.011256+0000","last_fullsized":"2024-04-24T23:00:34.011256+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:02:37.996954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.014648+0000","last_change":"2024-04-24T23:00:32.012144+0000","last_active":"2024-04-24T23:00:34.014648+0000","last_peered":"2024-04-24T23:00:34.014648+0000","last_clean":"2024-04-24T23:00:34.014648+0000","last_became_active":"2024-04-24T23:00:32.011876+0000","last_became_peered":"2024-04-24T23:00:32.011876+0000","last_unstale":"2024-04-24T23:00:34.014648+0000","last_undegraded":"2024-04-24T23:00:34.014648+0000","last_fullsized":"2024-04-24T23:00:34.014648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:19:27.134959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.014032+0000","last_change":"2024-04-24T23:00:33.021801+0000","last_active":"2024-04-24T23:00:34.014032+0000","last_peered":"2024-04-24T23:00:34.014032+0000","last_clean":"2024-04-24T23:00:34.014032+0000","last_became_active":"2024-04-24T23:00:33.021575+0000","last_became_peered":"2024-04-24T23:00:33.021575+0000","last_unstale":"2024-04-24T23:00:34.014032+0000","last_undegraded":"2024-04-24T23:00:34.014032+0000","last_fullsized":"2024-04-24T23:00:34.014032+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:30:09.050918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.011510+0000","last_change":"2024-04-24T23:00:32.015060+0000","last_active":"2024-04-24T23:00:34.011510+0000","last_peered":"2024-04-24T23:00:34.011510+0000","last_clean":"2024-04-24T23:00:34.011510+0000","last_became_active":"2024-04-24T23:00:32.013636+0000","last_became_peered":"2024-04-24T23:00:32.013636+0000","last_unstale":"2024-04-24T23:00:34.011510+0000","last_undegraded":"2024-04-24T23:00:34.011510+0000","last_fullsized":"2024-04-24T23:00:34.011510+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:43:05.640033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.014477+0000","last_change":"2024-04-24T23:00:33.027352+0000","last_active":"2024-04-24T23:00:34.014477+0000","last_peered":"2024-04-24T23:00:34.014477+0000","last_clean":"2024-04-24T23:00:34.014477+0000","last_became_active":"2024-04-24T23:00:33.027121+0000","last_became_peered":"2024-04-24T23:00:33.027121+0000","last_unstale":"2024-04-24T23:00:34.014477+0000","last_undegraded":"2024-04-24T23:00:34.014477+0000","last_fullsized":"2024-04-24T23:00:34.014477+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:07:27.945918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.816900+0000","last_change":"2024-04-24T23:00:32.018961+0000","last_active":"2024-04-24T23:00:35.816900+0000","last_peered":"2024-04-24T23:00:35.816900+0000","last_clean":"2024-04-24T23:00:35.816900+0000","last_became_active":"2024-04-24T23:00:32.018718+0000","last_became_peered":"2024-04-24T23:00:32.018718+0000","last_unstale":"2024-04-24T23:00:35.816900+0000","last_undegraded":"2024-04-24T23:00:35.816900+0000","last_fullsized":"2024-04-24T23:00:35.816900+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:05:42.477328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.010885+0000","last_change":"2024-04-24T23:00:33.012456+0000","last_active":"2024-04-24T23:00:34.010885+0000","last_peered":"2024-04-24T23:00:34.010885+0000","last_clean":"2024-04-24T23:00:34.010885+0000","last_became_active":"2024-04-24T23:00:33.012207+0000","last_became_peered":"2024-04-24T23:00:33.012207+0000","last_unstale":"2024-04-24T23:00:34.010885+0000","last_undegraded":"2024-04-24T23:00:34.010885+0000","last_fullsized":"2024-04-24T23:00:34.010885+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:28.579298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.014895+0000","last_change":"2024-04-24T23:00:32.016634+0000","last_active":"2024-04-24T23:00:34.014895+0000","last_peered":"2024-04-24T23:00:34.014895+0000","last_clean":"2024-04-24T23:00:34.014895+0000","last_became_active":"2024-04-24T23:00:32.015425+0000","last_became_peered":"2024-04-24T23:00:32.015425+0000","last_unstale":"2024-04-24T23:00:34.014895+0000","last_undegraded":"2024-04-24T23:00:34.014895+0000","last_fullsized":"2024-04-24T23:00:34.014895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:21:01.877515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.111606+0000","last_change":"2024-04-24T23:00:33.017687+0000","last_active":"2024-04-24T23:00:35.111606+0000","last_peered":"2024-04-24T23:00:35.111606+0000","last_clean":"2024-04-24T23:00:35.111606+0000","last_became_active":"2024-04-24T23:00:33.017433+0000","last_became_peered":"2024-04-24T23:00:33.017433+0000","last_unstale":"2024-04-24T23:00:35.111606+0000","last_undegraded":"2024-04-24T23:00:35.111606+0000","last_fullsized":"2024-04-24T23:00:35.111606+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:32:36.803741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.111814+0000","last_change":"2024-04-24T23:00:32.017822+0000","last_active":"2024-04-24T23:00:35.111814+0000","last_peered":"2024-04-24T23:00:35.111814+0000","last_clean":"2024-04-24T23:00:35.111814+0000","last_became_active":"2024-04-24T23:00:32.017548+0000","last_became_peered":"2024-04-24T23:00:32.017548+0000","last_unstale":"2024-04-24T23:00:35.111814+0000","last_undegraded":"2024-04-24T23:00:35.111814+0000","last_fullsized":"2024-04-24T23:00:35.111814+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:38:13.659009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.111557+0000","last_change":"2024-04-24T23:00:33.025375+0000","last_active":"2024-04-24T23:00:35.111557+0000","last_peered":"2024-04-24T23:00:35.111557+0000","last_clean":"2024-04-24T23:00:35.111557+0000","last_became_active":"2024-04-24T23:00:33.024368+0000","last_became_peered":"2024-04-24T23:00:33.024368+0000","last_unstale":"2024-04-24T23:00:35.111557+0000","last_undegraded":"2024-04-24T23:00:35.111557+0000","last_fullsized":"2024-04-24T23:00:35.111557+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:33:19.427462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.014409+0000","last_change":"2024-04-24T23:00:32.016337+0000","last_active":"2024-04-24T23:00:34.014409+0000","last_peered":"2024-04-24T23:00:34.014409+0000","last_clean":"2024-04-24T23:00:34.014409+0000","last_became_active":"2024-04-24T23:00:32.015833+0000","last_became_peered":"2024-04-24T23:00:32.015833+0000","last_unstale":"2024-04-24T23:00:34.014409+0000","last_undegraded":"2024-04-24T23:00:34.014409+0000","last_fullsized":"2024-04-24T23:00:34.014409+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:23:46.588298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.086366+0000","last_change":"2024-04-24T23:00:33.032317+0000","last_active":"2024-04-24T23:00:34.086366+0000","last_peered":"2024-04-24T23:00:34.086366+0000","last_clean":"2024-04-24T23:00:34.086366+0000","last_became_active":"2024-04-24T23:00:33.031714+0000","last_became_peered":"2024-04-24T23:00:33.031714+0000","last_unstale":"2024-04-24T23:00:34.086366+0000","last_undegraded":"2024-04-24T23:00:34.086366+0000","last_fullsized":"2024-04-24T23:00:34.086366+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:09:43.053763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"44'3","reported_seq":17,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.816962+0000","last_change":"2024-04-24T23:00:32.017203+0000","last_active":"2024-04-24T23:00:35.816962+0000","last_peered":"2024-04-24T23:00:35.816962+0000","last_clean":"2024-04-24T23:00:35.816962+0000","last_became_active":"2024-04-24T23:00:32.016743+0000","last_became_peered":"2024-04-24T23:00:32.016743+0000","last_unstale":"2024-04-24T23:00:35.816962+0000","last_undegraded":"2024-04-24T23:00:35.816962+0000","last_fullsized":"2024-04-24T23:00:35.816962+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:28.587565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.011341+0000","last_change":"2024-04-24T23:00:33.026608+0000","last_active":"2024-04-24T23:00:34.011341+0000","last_peered":"2024-04-24T23:00:34.011341+0000","last_clean":"2024-04-24T23:00:34.011341+0000","last_became_active":"2024-04-24T23:00:33.025380+0000","last_became_peered":"2024-04-24T23:00:33.025380+0000","last_unstale":"2024-04-24T23:00:34.011341+0000","last_undegraded":"2024-04-24T23:00:34.011341+0000","last_fullsized":"2024-04-24T23:00:34.011341+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:41:28.669943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"44'6","reported_seq":20,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.014745+0000","last_change":"2024-04-24T23:00:32.014044+0000","last_active":"2024-04-24T23:00:34.014745+0000","last_peered":"2024-04-24T23:00:34.014745+0000","last_clean":"2024-04-24T23:00:34.014745+0000","last_became_active":"2024-04-24T23:00:32.013781+0000","last_became_peered":"2024-04-24T23:00:32.013781+0000","last_unstale":"2024-04-24T23:00:34.014745+0000","last_undegraded":"2024-04-24T23:00:34.014745+0000","last_fullsized":"2024-04-24T23:00:34.014745+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:31:21.219632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.111671+0000","last_change":"2024-04-24T23:00:33.026903+0000","last_active":"2024-04-24T23:00:35.111671+0000","last_peered":"2024-04-24T23:00:35.111671+0000","last_clean":"2024-04-24T23:00:35.111671+0000","last_became_active":"2024-04-24T23:00:33.026648+0000","last_became_peered":"2024-04-24T23:00:33.026648+0000","last_unstale":"2024-04-24T23:00:35.111671+0000","last_undegraded":"2024-04-24T23:00:35.111671+0000","last_fullsized":"2024-04-24T23:00:35.111671+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:49.955693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.112010+0000","last_change":"2024-04-24T23:00:32.019200+0000","last_active":"2024-04-24T23:00:35.112010+0000","last_peered":"2024-04-24T23:00:35.112010+0000","last_clean":"2024-04-24T23:00:35.112010+0000","last_became_active":"2024-04-24T23:00:32.018935+0000","last_became_peered":"2024-04-24T23:00:32.018935+0000","last_unstale":"2024-04-24T23:00:35.112010+0000","last_undegraded":"2024-04-24T23:00:35.112010+0000","last_fullsized":"2024-04-24T23:00:35.112010+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:18:59.543394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.111817+0000","last_change":"2024-04-24T23:00:33.022700+0000","last_active":"2024-04-24T23:00:35.111817+0000","last_peered":"2024-04-24T23:00:35.111817+0000","last_clean":"2024-04-24T23:00:35.111817+0000","last_became_active":"2024-04-24T23:00:33.022475+0000","last_became_peered":"2024-04-24T23:00:33.022475+0000","last_unstale":"2024-04-24T23:00:35.111817+0000","last_undegraded":"2024-04-24T23:00:35.111817+0000","last_fullsized":"2024-04-24T23:00:35.111817+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:19:49.215022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.817457+0000","last_change":"2024-04-24T23:00:32.018091+0000","last_active":"2024-04-24T23:00:35.817457+0000","last_peered":"2024-04-24T23:00:35.817457+0000","last_clean":"2024-04-24T23:00:35.817457+0000","last_became_active":"2024-04-24T23:00:32.017828+0000","last_became_peered":"2024-04-24T23:00:32.017828+0000","last_unstale":"2024-04-24T23:00:35.817457+0000","last_undegraded":"2024-04-24T23:00:35.817457+0000","last_fullsized":"2024-04-24T23:00:35.817457+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:49:51.828270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.111692+0000","last_change":"2024-04-24T23:00:33.022022+0000","last_active":"2024-04-24T23:00:35.111692+0000","last_peered":"2024-04-24T23:00:35.111692+0000","last_clean":"2024-04-24T23:00:35.111692+0000","last_became_active":"2024-04-24T23:00:33.021782+0000","last_became_peered":"2024-04-24T23:00:33.021782+0000","last_unstale":"2024-04-24T23:00:35.111692+0000","last_undegraded":"2024-04-24T23:00:35.111692+0000","last_fullsized":"2024-04-24T23:00:35.111692+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:20:57.145395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.014765+0000","last_change":"2024-04-24T23:00:32.015309+0000","last_active":"2024-04-24T23:00:34.014765+0000","last_peered":"2024-04-24T23:00:34.014765+0000","last_clean":"2024-04-24T23:00:34.014765+0000","last_became_active":"2024-04-24T23:00:32.014784+0000","last_became_peered":"2024-04-24T23:00:32.014784+0000","last_unstale":"2024-04-24T23:00:34.014765+0000","last_undegraded":"2024-04-24T23:00:34.014765+0000","last_fullsized":"2024-04-24T23:00:34.014765+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:05:50.378062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.817181+0000","last_change":"2024-04-24T23:00:33.028237+0000","last_active":"2024-04-24T23:00:35.817181+0000","last_peered":"2024-04-24T23:00:35.817181+0000","last_clean":"2024-04-24T23:00:35.817181+0000","last_became_active":"2024-04-24T23:00:33.027083+0000","last_became_peered":"2024-04-24T23:00:33.027083+0000","last_unstale":"2024-04-24T23:00:35.817181+0000","last_undegraded":"2024-04-24T23:00:35.817181+0000","last_fullsized":"2024-04-24T23:00:35.817181+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:09:29.327093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"44'8","reported_seq":22,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.014481+0000","last_change":"2024-04-24T23:00:32.018235+0000","last_active":"2024-04-24T23:00:34.014481+0000","last_peered":"2024-04-24T23:00:34.014481+0000","last_clean":"2024-04-24T23:00:34.014481+0000","last_became_active":"2024-04-24T23:00:32.017933+0000","last_became_peered":"2024-04-24T23:00:32.017933+0000","last_unstale":"2024-04-24T23:00:34.014481+0000","last_undegraded":"2024-04-24T23:00:34.014481+0000","last_fullsized":"2024-04-24T23:00:34.014481+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:16.141997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.111948+0000","last_change":"2024-04-24T23:00:33.017444+0000","last_active":"2024-04-24T23:00:35.111948+0000","last_peered":"2024-04-24T23:00:35.111948+0000","last_clean":"2024-04-24T23:00:35.111948+0000","last_became_active":"2024-04-24T23:00:33.017112+0000","last_became_peered":"2024-04-24T23:00:33.017112+0000","last_unstale":"2024-04-24T23:00:35.111948+0000","last_undegraded":"2024-04-24T23:00:35.111948+0000","last_fullsized":"2024-04-24T23:00:35.111948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:39:24.740803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.086739+0000","last_change":"2024-04-24T23:00:32.015665+0000","last_active":"2024-04-24T23:00:34.086739+0000","last_peered":"2024-04-24T23:00:34.086739+0000","last_clean":"2024-04-24T23:00:34.086739+0000","last_became_active":"2024-04-24T23:00:32.015414+0000","last_became_peered":"2024-04-24T23:00:32.015414+0000","last_unstale":"2024-04-24T23:00:34.086739+0000","last_undegraded":"2024-04-24T23:00:34.086739+0000","last_fullsized":"2024-04-24T23:00:34.086739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:28:23.691908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.111818+0000","last_change":"2024-04-24T23:00:33.029957+0000","last_active":"2024-04-24T23:00:35.111818+0000","last_peered":"2024-04-24T23:00:35.111818+0000","last_clean":"2024-04-24T23:00:35.111818+0000","last_became_active":"2024-04-24T23:00:33.029515+0000","last_became_peered":"2024-04-24T23:00:33.029515+0000","last_unstale":"2024-04-24T23:00:35.111818+0000","last_undegraded":"2024-04-24T23:00:35.111818+0000","last_fullsized":"2024-04-24T23:00:35.111818+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:02:21.930006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.111690+0000","last_change":"2024-04-24T23:00:32.016781+0000","last_active":"2024-04-24T23:00:35.111690+0000","last_peered":"2024-04-24T23:00:35.111690+0000","last_clean":"2024-04-24T23:00:35.111690+0000","last_became_active":"2024-04-24T23:00:32.016515+0000","last_became_peered":"2024-04-24T23:00:32.016515+0000","last_unstale":"2024-04-24T23:00:35.111690+0000","last_undegraded":"2024-04-24T23:00:35.111690+0000","last_fullsized":"2024-04-24T23:00:35.111690+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:03:55.033966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.817317+0000","last_change":"2024-04-24T23:00:33.027722+0000","last_active":"2024-04-24T23:00:35.817317+0000","last_peered":"2024-04-24T23:00:35.817317+0000","last_clean":"2024-04-24T23:00:35.817317+0000","last_became_active":"2024-04-24T23:00:33.025042+0000","last_became_peered":"2024-04-24T23:00:33.025042+0000","last_unstale":"2024-04-24T23:00:35.817317+0000","last_undegraded":"2024-04-24T23:00:35.817317+0000","last_fullsized":"2024-04-24T23:00:35.817317+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:20.171730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.111861+0000","last_change":"2024-04-24T23:00:32.019334+0000","last_active":"2024-04-24T23:00:35.111861+0000","last_peered":"2024-04-24T23:00:35.111861+0000","last_clean":"2024-04-24T23:00:35.111861+0000","last_became_active":"2024-04-24T23:00:32.019103+0000","last_became_peered":"2024-04-24T23:00:32.019103+0000","last_unstale":"2024-04-24T23:00:35.111861+0000","last_undegraded":"2024-04-24T23:00:35.111861+0000","last_fullsized":"2024-04-24T23:00:35.111861+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:50:21.381560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.111769+0000","last_change":"2024-04-24T23:00:33.021125+0000","last_active":"2024-04-24T23:00:35.111769+0000","last_peered":"2024-04-24T23:00:35.111769+0000","last_clean":"2024-04-24T23:00:35.111769+0000","last_became_active":"2024-04-24T23:00:33.020882+0000","last_became_peered":"2024-04-24T23:00:33.020882+0000","last_unstale":"2024-04-24T23:00:35.111769+0000","last_undegraded":"2024-04-24T23:00:35.111769+0000","last_fullsized":"2024-04-24T23:00:35.111769+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:10:05.572925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.014867+0000","last_change":"2024-04-24T23:00:32.015023+0000","last_active":"2024-04-24T23:00:34.014867+0000","last_peered":"2024-04-24T23:00:34.014867+0000","last_clean":"2024-04-24T23:00:34.014867+0000","last_became_active":"2024-04-24T23:00:32.014455+0000","last_became_peered":"2024-04-24T23:00:32.014455+0000","last_unstale":"2024-04-24T23:00:34.014867+0000","last_undegraded":"2024-04-24T23:00:34.014867+0000","last_fullsized":"2024-04-24T23:00:34.014867+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:12:05.738094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.014641+0000","last_change":"2024-04-24T23:00:33.028691+0000","last_active":"2024-04-24T23:00:34.014641+0000","last_peered":"2024-04-24T23:00:34.014641+0000","last_clean":"2024-04-24T23:00:34.014641+0000","last_became_active":"2024-04-24T23:00:33.028122+0000","last_became_peered":"2024-04-24T23:00:33.028122+0000","last_unstale":"2024-04-24T23:00:34.014641+0000","last_undegraded":"2024-04-24T23:00:34.014641+0000","last_fullsized":"2024-04-24T23:00:34.014641+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:05:35.652787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.014850+0000","last_change":"2024-04-24T23:00:32.019708+0000","last_active":"2024-04-24T23:00:34.014850+0000","last_peered":"2024-04-24T23:00:34.014850+0000","last_clean":"2024-04-24T23:00:34.014850+0000","last_became_active":"2024-04-24T23:00:32.019473+0000","last_became_peered":"2024-04-24T23:00:32.019473+0000","last_unstale":"2024-04-24T23:00:34.014850+0000","last_undegraded":"2024-04-24T23:00:34.014850+0000","last_fullsized":"2024-04-24T23:00:34.014850+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:30.220564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.014729+0000","last_change":"2024-04-24T23:00:33.018734+0000","last_active":"2024-04-24T23:00:34.014729+0000","last_peered":"2024-04-24T23:00:34.014729+0000","last_clean":"2024-04-24T23:00:34.014729+0000","last_became_active":"2024-04-24T23:00:33.018493+0000","last_became_peered":"2024-04-24T23:00:33.018493+0000","last_unstale":"2024-04-24T23:00:34.014729+0000","last_undegraded":"2024-04-24T23:00:34.014729+0000","last_fullsized":"2024-04-24T23:00:34.014729+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:04:52.434732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.817258+0000","last_change":"2024-04-24T23:00:32.015397+0000","last_active":"2024-04-24T23:00:35.817258+0000","last_peered":"2024-04-24T23:00:35.817258+0000","last_clean":"2024-04-24T23:00:35.817258+0000","last_became_active":"2024-04-24T23:00:32.015154+0000","last_became_peered":"2024-04-24T23:00:32.015154+0000","last_unstale":"2024-04-24T23:00:35.817258+0000","last_undegraded":"2024-04-24T23:00:35.817258+0000","last_fullsized":"2024-04-24T23:00:35.817258+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:09:22.441674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.086914+0000","last_change":"2024-04-24T23:00:33.023752+0000","last_active":"2024-04-24T23:00:34.086914+0000","last_peered":"2024-04-24T23:00:34.086914+0000","last_clean":"2024-04-24T23:00:34.086914+0000","last_became_active":"2024-04-24T23:00:33.023405+0000","last_became_peered":"2024-04-24T23:00:33.023405+0000","last_unstale":"2024-04-24T23:00:34.086914+0000","last_undegraded":"2024-04-24T23:00:34.086914+0000","last_fullsized":"2024-04-24T23:00:34.086914+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:04:55.906206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.015009+0000","last_change":"2024-04-24T23:00:32.017383+0000","last_active":"2024-04-24T23:00:34.015009+0000","last_peered":"2024-04-24T23:00:34.015009+0000","last_clean":"2024-04-24T23:00:34.015009+0000","last_became_active":"2024-04-24T23:00:32.016962+0000","last_became_peered":"2024-04-24T23:00:32.016962+0000","last_unstale":"2024-04-24T23:00:34.015009+0000","last_undegraded":"2024-04-24T23:00:34.015009+0000","last_fullsized":"2024-04-24T23:00:34.015009+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:47:07.143610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.086870+0000","last_change":"2024-04-24T23:00:33.025709+0000","last_active":"2024-04-24T23:00:34.086870+0000","last_peered":"2024-04-24T23:00:34.086870+0000","last_clean":"2024-04-24T23:00:34.086870+0000","last_became_active":"2024-04-24T23:00:33.025441+0000","last_became_peered":"2024-04-24T23:00:33.025441+0000","last_unstale":"2024-04-24T23:00:34.086870+0000","last_undegraded":"2024-04-24T23:00:34.086870+0000","last_fullsized":"2024-04-24T23:00:34.086870+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:54:46.938217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.010911+0000","last_change":"2024-04-24T23:00:32.011951+0000","last_active":"2024-04-24T23:00:34.010911+0000","last_peered":"2024-04-24T23:00:34.010911+0000","last_clean":"2024-04-24T23:00:34.010911+0000","last_became_active":"2024-04-24T23:00:32.011483+0000","last_became_peered":"2024-04-24T23:00:32.011483+0000","last_unstale":"2024-04-24T23:00:34.010911+0000","last_undegraded":"2024-04-24T23:00:34.010911+0000","last_fullsized":"2024-04-24T23:00:34.010911+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:07.472930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.011175+0000","last_change":"2024-04-24T23:00:33.012962+0000","last_active":"2024-04-24T23:00:34.011175+0000","last_peered":"2024-04-24T23:00:34.011175+0000","last_clean":"2024-04-24T23:00:34.011175+0000","last_became_active":"2024-04-24T23:00:33.012675+0000","last_became_peered":"2024-04-24T23:00:33.012675+0000","last_unstale":"2024-04-24T23:00:34.011175+0000","last_undegraded":"2024-04-24T23:00:34.011175+0000","last_fullsized":"2024-04-24T23:00:34.011175+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:40:06.285416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"45'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.086870+0000","last_change":"2024-04-24T23:00:32.016280+0000","last_active":"2024-04-24T23:00:34.086870+0000","last_peered":"2024-04-24T23:00:34.086870+0000","last_clean":"2024-04-24T23:00:34.086870+0000","last_became_active":"2024-04-24T23:00:32.016057+0000","last_became_peered":"2024-04-24T23:00:32.016057+0000","last_unstale":"2024-04-24T23:00:34.086870+0000","last_undegraded":"2024-04-24T23:00:34.086870+0000","last_fullsized":"2024-04-24T23:00:34.086870+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:27.231505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.014640+0000","last_change":"2024-04-24T23:00:33.025946+0000","last_active":"2024-04-24T23:00:34.014640+0000","last_peered":"2024-04-24T23:00:34.014640+0000","last_clean":"2024-04-24T23:00:34.014640+0000","last_became_active":"2024-04-24T23:00:33.024916+0000","last_became_peered":"2024-04-24T23:00:33.024916+0000","last_unstale":"2024-04-24T23:00:34.014640+0000","last_undegraded":"2024-04-24T23:00:34.014640+0000","last_fullsized":"2024-04-24T23:00:34.014640+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:41:24.055196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.817152+0000","last_change":"2024-04-24T23:00:32.015930+0000","last_active":"2024-04-24T23:00:35.817152+0000","last_peered":"2024-04-24T23:00:35.817152+0000","last_clean":"2024-04-24T23:00:35.817152+0000","last_became_active":"2024-04-24T23:00:32.015671+0000","last_became_peered":"2024-04-24T23:00:32.015671+0000","last_unstale":"2024-04-24T23:00:35.817152+0000","last_undegraded":"2024-04-24T23:00:35.817152+0000","last_fullsized":"2024-04-24T23:00:35.817152+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:45.511844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.011043+0000","last_change":"2024-04-24T23:00:33.028161+0000","last_active":"2024-04-24T23:00:34.011043+0000","last_peered":"2024-04-24T23:00:34.011043+0000","last_clean":"2024-04-24T23:00:34.011043+0000","last_became_active":"2024-04-24T23:00:33.027100+0000","last_became_peered":"2024-04-24T23:00:33.027100+0000","last_unstale":"2024-04-24T23:00:34.011043+0000","last_undegraded":"2024-04-24T23:00:34.011043+0000","last_fullsized":"2024-04-24T23:00:34.011043+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:45:20.571133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.112072+0000","last_change":"2024-04-24T23:00:32.020060+0000","last_active":"2024-04-24T23:00:35.112072+0000","last_peered":"2024-04-24T23:00:35.112072+0000","last_clean":"2024-04-24T23:00:35.112072+0000","last_became_active":"2024-04-24T23:00:32.019783+0000","last_became_peered":"2024-04-24T23:00:32.019783+0000","last_unstale":"2024-04-24T23:00:35.112072+0000","last_undegraded":"2024-04-24T23:00:35.112072+0000","last_fullsized":"2024-04-24T23:00:35.112072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:28:38.519563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.014254+0000","last_change":"2024-04-24T23:00:33.024647+0000","last_active":"2024-04-24T23:00:34.014254+0000","last_peered":"2024-04-24T23:00:34.014254+0000","last_clean":"2024-04-24T23:00:34.014254+0000","last_became_active":"2024-04-24T23:00:33.024326+0000","last_became_peered":"2024-04-24T23:00:33.024326+0000","last_unstale":"2024-04-24T23:00:34.014254+0000","last_undegraded":"2024-04-24T23:00:34.014254+0000","last_fullsized":"2024-04-24T23:00:34.014254+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:04.949857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.817365+0000","last_change":"2024-04-24T23:00:32.020217+0000","last_active":"2024-04-24T23:00:35.817365+0000","last_peered":"2024-04-24T23:00:35.817365+0000","last_clean":"2024-04-24T23:00:35.817365+0000","last_became_active":"2024-04-24T23:00:32.019728+0000","last_became_peered":"2024-04-24T23:00:32.019728+0000","last_unstale":"2024-04-24T23:00:35.817365+0000","last_undegraded":"2024-04-24T23:00:35.817365+0000","last_fullsized":"2024-04-24T23:00:35.817365+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:27:05.085107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.087010+0000","last_change":"2024-04-24T23:00:33.030169+0000","last_active":"2024-04-24T23:00:34.087010+0000","last_peered":"2024-04-24T23:00:34.087010+0000","last_clean":"2024-04-24T23:00:34.087010+0000","last_became_active":"2024-04-24T23:00:33.029597+0000","last_became_peered":"2024-04-24T23:00:33.029597+0000","last_unstale":"2024-04-24T23:00:34.087010+0000","last_undegraded":"2024-04-24T23:00:34.087010+0000","last_fullsized":"2024-04-24T23:00:34.087010+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:07:05.050164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.011354+0000","last_change":"2024-04-24T23:00:32.018725+0000","last_active":"2024-04-24T23:00:34.011354+0000","last_peered":"2024-04-24T23:00:34.011354+0000","last_clean":"2024-04-24T23:00:34.011354+0000","last_became_active":"2024-04-24T23:00:32.018472+0000","last_became_peered":"2024-04-24T23:00:32.018472+0000","last_unstale":"2024-04-24T23:00:34.011354+0000","last_undegraded":"2024-04-24T23:00:34.011354+0000","last_fullsized":"2024-04-24T23:00:34.011354+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:16:36.870866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.111406+0000","last_change":"2024-04-24T23:00:33.023847+0000","last_active":"2024-04-24T23:00:35.111406+0000","last_peered":"2024-04-24T23:00:35.111406+0000","last_clean":"2024-04-24T23:00:35.111406+0000","last_became_active":"2024-04-24T23:00:33.023599+0000","last_became_peered":"2024-04-24T23:00:33.023599+0000","last_unstale":"2024-04-24T23:00:35.111406+0000","last_undegraded":"2024-04-24T23:00:35.111406+0000","last_fullsized":"2024-04-24T23:00:35.111406+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:46.077303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.014933+0000","last_change":"2024-04-24T23:00:32.011954+0000","last_active":"2024-04-24T23:00:34.014933+0000","last_peered":"2024-04-24T23:00:34.014933+0000","last_clean":"2024-04-24T23:00:34.014933+0000","last_became_active":"2024-04-24T23:00:32.011189+0000","last_became_peered":"2024-04-24T23:00:32.011189+0000","last_unstale":"2024-04-24T23:00:34.014933+0000","last_undegraded":"2024-04-24T23:00:34.014933+0000","last_fullsized":"2024-04-24T23:00:34.014933+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:05:28.926498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.816849+0000","last_change":"2024-04-24T23:00:33.025596+0000","last_active":"2024-04-24T23:00:35.816849+0000","last_peered":"2024-04-24T23:00:35.816849+0000","last_clean":"2024-04-24T23:00:35.816849+0000","last_became_active":"2024-04-24T23:00:33.025348+0000","last_became_peered":"2024-04-24T23:00:33.025348+0000","last_unstale":"2024-04-24T23:00:35.816849+0000","last_undegraded":"2024-04-24T23:00:35.816849+0000","last_fullsized":"2024-04-24T23:00:35.816849+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:44:07.066426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.111892+0000","last_change":"2024-04-24T23:00:32.019965+0000","last_active":"2024-04-24T23:00:35.111892+0000","last_peered":"2024-04-24T23:00:35.111892+0000","last_clean":"2024-04-24T23:00:35.111892+0000","last_became_active":"2024-04-24T23:00:32.019691+0000","last_became_peered":"2024-04-24T23:00:32.019691+0000","last_unstale":"2024-04-24T23:00:35.111892+0000","last_undegraded":"2024-04-24T23:00:35.111892+0000","last_fullsized":"2024-04-24T23:00:35.111892+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:05:21.719262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.086660+0000","last_change":"2024-04-24T23:00:33.023228+0000","last_active":"2024-04-24T23:00:34.086660+0000","last_peered":"2024-04-24T23:00:34.086660+0000","last_clean":"2024-04-24T23:00:34.086660+0000","last_became_active":"2024-04-24T23:00:33.022394+0000","last_became_peered":"2024-04-24T23:00:33.022394+0000","last_unstale":"2024-04-24T23:00:34.086660+0000","last_undegraded":"2024-04-24T23:00:34.086660+0000","last_fullsized":"2024-04-24T23:00:34.086660+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:32.002613+0000","last_clean_scrub_stamp":"2024-04-24T23:00:32.002613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:22:50.530894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:34.011260+0000","last_change":"2024-04-24T23:00:32.016499+0000","last_active":"2024-04-24T23:00:34.011260+0000","last_peered":"2024-04-24T23:00:34.011260+0000","last_clean":"2024-04-24T23:00:34.011260+0000","last_became_active":"2024-04-24T23:00:32.016247+0000","last_became_peered":"2024-04-24T23:00:32.016247+0000","last_unstale":"2024-04-24T23:00:34.011260+0000","last_undegraded":"2024-04-24T23:00:34.011260+0000","last_fullsized":"2024-04-24T23:00:34.011260+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:00:31.000854+0000","last_clean_scrub_stamp":"2024-04-24T23:00:31.000854+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:44:04.110059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":213,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T23:00:35.111524+0000","last_change":"2024-04-24T22:58:55.138810+0000","last_active":"2024-04-24T23:00:35.111524+0000","last_peered":"2024-04-24T23:00:35.111524+0000","last_clean":"2024-04-24T23:00:35.111524+0000","last_became_active":"2024-04-24T22:58:55.138627+0000","last_became_peered":"2024-04-24T22:58:55.138627+0000","last_unstale":"2024-04-24T23:00:35.111524+0000","last_undegraded":"2024-04-24T23:00:35.111524+0000","last_fullsized":"2024-04-24T23:00:35.111524+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_clean_scrub_stamp":"2024-04-24T22:58:54.127235+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:24.010002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":208,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195103,"statfs":{"total":200000000,"available":199786303,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":209,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195103,"statfs":{"total":200000000,"available":199785721,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":33,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":785,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194527,"statfs":{"total":200000000,"available":199195947,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":210,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195102,"statfs":{"total":200000000,"available":199784633,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":210,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195102,"statfs":{"total":200000000,"available":199784725,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":14,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":208,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195103,"statfs":{"total":200000000,"available":199786247,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":785,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194527,"statfs":{"total":200000000,"available":199195899,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":209,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195103,"statfs":{"total":200000000,"available":199785665,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-24T23:00:40.804 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-24T23:00:40.804 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-24T23:00:40.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.946+0000 7fb4d1ab2640 1 Processor -- start 2024-04-24T23:00:40.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.946+0000 7fb4d1ab2640 1 -- start start 2024-04-24T23:00:40.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.947+0000 7fb4d1ab2640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb4cc15c5b0 0x7fb4cc0b4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:40.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.947+0000 7fb4d1ab2640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb4cc072cf0 0x7fb4cc06c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:40.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.947+0000 7fb4d1ab2640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb4cc071770 0x7fb4cc06cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:40.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.947+0000 7fb4d1ab2640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fb4cc072860 con 0x7fb4cc072cf0 2024-04-24T23:00:40.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.947+0000 7fb4d1ab2640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fb4cc15c990 con 0x7fb4cc15c5b0 2024-04-24T23:00:40.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.947+0000 7fb4d1ab2640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fb4cc071c50 con 0x7fb4cc071770 2024-04-24T23:00:40.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.947+0000 7fb4caffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb4cc072cf0 0x7fb4cc06c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:40.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.947+0000 7fb4caffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb4cc072cf0 0x7fb4cc06c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53400/0 (socket says 172.21.15.32:53400) 2024-04-24T23:00:40.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.947+0000 7fb4caffd640 1 -- 172.21.15.32:0/3414962781 learned_addr learned my addr 172.21.15.32:0/3414962781 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:40.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.947+0000 7fb4caffd640 1 -- 172.21.15.32:0/3414962781 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb4cc071770 msgr2=0x7fb4cc06cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:40.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.947+0000 7fb4caffd640 1 --2- 172.21.15.32:0/3414962781 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb4cc071770 0x7fb4cc06cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:40.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.947+0000 7fb4caffd640 1 -- 172.21.15.32:0/3414962781 >> v2:172.21.15.121:3300/0 conn(0x7fb4cc15c5b0 msgr2=0x7fb4cc0b4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:40.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.947+0000 7fb4caffd640 1 --2- 172.21.15.32:0/3414962781 >> v2:172.21.15.121:3300/0 conn(0x7fb4cc15c5b0 0x7fb4cc0b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:40.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.947+0000 7fb4caffd640 1 -- 172.21.15.32:0/3414962781 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4cc071f30 con 0x7fb4cc072cf0 2024-04-24T23:00:40.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.948+0000 7fb4caffd640 1 --2- 172.21.15.32:0/3414962781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4cc072cf0 0x7fb4cc06c5f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc067850 tx=0x7fb4c002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=120312c4477e4e6e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:40.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.948+0000 7fb4ca7fc640 1 -- 172.21.15.32:0/3414962781 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c0037070 con 0x7fb4cc072cf0 2024-04-24T23:00:40.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.948+0000 7fb4ca7fc640 1 -- 172.21.15.32:0/3414962781 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4c0032440 con 0x7fb4cc072cf0 2024-04-24T23:00:40.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.948+0000 7fb4ca7fc640 1 -- 172.21.15.32:0/3414962781 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c0032810 con 0x7fb4cc072cf0 2024-04-24T23:00:40.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.949+0000 7fb4d1ab2640 1 -- 172.21.15.32:0/3414962781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4cc072cf0 msgr2=0x7fb4cc06c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:40.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.949+0000 7fb4d1ab2640 1 --2- 172.21.15.32:0/3414962781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4cc072cf0 0x7fb4cc06c5f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc067850 tx=0x7fb4c002a140 comp rx=0 tx=0).stop 2024-04-24T23:00:40.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.949+0000 7fb4d1ab2640 1 -- 172.21.15.32:0/3414962781 shutdown_connections 2024-04-24T23:00:40.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.949+0000 7fb4d1ab2640 1 --2- 172.21.15.32:0/3414962781 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb4cc071770 0x7fb4cc06cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:40.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.949+0000 7fb4d1ab2640 1 --2- 172.21.15.32:0/3414962781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4cc072cf0 0x7fb4cc06c5f0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:40.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.949+0000 7fb4d1ab2640 1 --2- 172.21.15.32:0/3414962781 >> v2:172.21.15.121:3300/0 conn(0x7fb4cc15c5b0 0x7fb4cc0b4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:40.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.949+0000 7fb4d1ab2640 1 -- 172.21.15.32:0/3414962781 >> 172.21.15.32:0/3414962781 conn(0x7fb4cc06b110 msgr2=0x7fb4cc06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:40.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.949+0000 7fb4d1ab2640 1 -- 172.21.15.32:0/3414962781 shutdown_connections 2024-04-24T23:00:40.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.949+0000 7fb4d1ab2640 1 -- 172.21.15.32:0/3414962781 wait complete. 2024-04-24T23:00:40.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.950+0000 7fb4d1ab2640 1 Processor -- start 2024-04-24T23:00:40.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.950+0000 7fb4d1ab2640 1 -- start start 2024-04-24T23:00:40.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.951+0000 7fb4d1ab2640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4cc071770 0x7fb4cc144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:40.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.951+0000 7fb4d1ab2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4cc072cf0 0x7fb4cc155b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:40.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.951+0000 7fb4d1ab2640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb4cc15c5b0 0x7fb4cc06b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:40.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.951+0000 7fb4d1ab2640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb4cc0b4570 con 0x7fb4cc072cf0 2024-04-24T23:00:40.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.951+0000 7fb4d1ab2640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb4cc0724f0 con 0x7fb4cc071770 2024-04-24T23:00:40.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.951+0000 7fb4d1ab2640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fb4cc0b46f0 con 0x7fb4cc15c5b0 2024-04-24T23:00:40.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.951+0000 7fb4caffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4cc072cf0 0x7fb4cc155b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:40.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.951+0000 7fb4caffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4cc072cf0 0x7fb4cc155b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53414/0 (socket says 172.21.15.32:53414) 2024-04-24T23:00:40.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.951+0000 7fb4caffd640 1 -- 172.21.15.32:0/2081723377 learned_addr learned my addr 172.21.15.32:0/2081723377 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:40.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.951+0000 7fb4cbfff640 1 --2- 172.21.15.32:0/2081723377 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb4cc15c5b0 0x7fb4cc06b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:40.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.951+0000 7fb4cb7fe640 1 --2- 172.21.15.32:0/2081723377 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4cc071770 0x7fb4cc144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:40.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.951+0000 7fb4caffd640 1 -- 172.21.15.32:0/2081723377 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb4cc15c5b0 msgr2=0x7fb4cc06b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:40.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.951+0000 7fb4caffd640 1 --2- 172.21.15.32:0/2081723377 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb4cc15c5b0 0x7fb4cc06b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:40.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.951+0000 7fb4caffd640 1 -- 172.21.15.32:0/2081723377 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4cc071770 msgr2=0x7fb4cc144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:40.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.951+0000 7fb4caffd640 1 --2- 172.21.15.32:0/2081723377 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4cc071770 0x7fb4cc144cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:40.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.951+0000 7fb4caffd640 1 -- 172.21.15.32:0/2081723377 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4cc15cee0 con 0x7fb4cc072cf0 2024-04-24T23:00:40.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.951+0000 7fb4cb7fe640 1 --2- 172.21.15.32:0/2081723377 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4cc071770 0x7fb4cc144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:00:40.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.951+0000 7fb4caffd640 1 --2- 172.21.15.32:0/2081723377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4cc072cf0 0x7fb4cc155b60 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0000c00 tx=0x7fb4c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:40.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.952+0000 7fb4b3fff640 1 -- 172.21.15.32:0/2081723377 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c0041070 con 0x7fb4cc072cf0 2024-04-24T23:00:40.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.952+0000 7fb4b3fff640 1 -- 172.21.15.32:0/2081723377 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4c003c870 con 0x7fb4cc072cf0 2024-04-24T23:00:40.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.952+0000 7fb4b3fff640 1 -- 172.21.15.32:0/2081723377 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c0037040 con 0x7fb4cc072cf0 2024-04-24T23:00:40.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.952+0000 7fb4d1ab2640 1 -- 172.21.15.32:0/2081723377 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4cc15d1c0 con 0x7fb4cc072cf0 2024-04-24T23:00:40.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.952+0000 7fb4d1ab2640 1 -- 172.21.15.32:0/2081723377 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4cc15da60 con 0x7fb4cc072cf0 2024-04-24T23:00:40.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.953+0000 7fb4d1ab2640 1 -- 172.21.15.32:0/2081723377 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4cc15d390 con 0x7fb4cc072cf0 2024-04-24T23:00:40.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.953+0000 7fb4b3fff640 1 -- 172.21.15.32:0/2081723377 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb4c0044070 con 0x7fb4cc072cf0 2024-04-24T23:00:40.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.953+0000 7fb4b3fff640 1 --2- 172.21.15.32:0/2081723377 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb4b806be60 0x7fb4b806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:40.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.953+0000 7fb4b3fff640 1 -- 172.21.15.32:0/2081723377 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fb4c00abaa0 con 0x7fb4cc072cf0 2024-04-24T23:00:40.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.954+0000 7fb4cb7fe640 1 --2- 172.21.15.32:0/2081723377 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb4b806be60 0x7fb4b806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:40.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.954+0000 7fb4cb7fe640 1 --2- 172.21.15.32:0/2081723377 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb4b806be60 0x7fb4b806e320 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc14e470 tx=0x7fb4bc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:40.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:40.958+0000 7fb4b3fff640 1 -- 172.21.15.32:0/2081723377 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb4c0079fb0 con 0x7fb4cc072cf0 2024-04-24T23:00:41.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.221+0000 7fb4d1ab2640 1 -- 172.21.15.32:0/2081723377 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fb4cc068430 con 0x7fb4cc072cf0 2024-04-24T23:00:41.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.222+0000 7fb4b3fff640 1 -- 172.21.15.32:0/2081723377 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) ==== 190+0+147 (secure 0 0 0) 0x7fb4c007e2b0 con 0x7fb4cc072cf0 2024-04-24T23:00:41.224 INFO:teuthology.orchestra.run.smithi032.stdout:[client.0] 2024-04-24T23:00:41.224 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQCejilm5oeiGRAAdLXwwqrYhV3HHo7Dw/2ZqQ== 2024-04-24T23:00:41.224 INFO:teuthology.orchestra.run.smithi032.stdout: caps mds = "allow" 2024-04-24T23:00:41.224 INFO:teuthology.orchestra.run.smithi032.stdout: caps mon = "allow r" 2024-04-24T23:00:41.224 INFO:teuthology.orchestra.run.smithi032.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-24T23:00:41.225 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2024-04-24T23:00:41.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.225+0000 7fb4d1ab2640 1 -- 172.21.15.32:0/2081723377 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb4b806be60 msgr2=0x7fb4b806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:41.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.225+0000 7fb4d1ab2640 1 --2- 172.21.15.32:0/2081723377 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb4b806be60 0x7fb4b806e320 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc14e470 tx=0x7fb4bc008040 comp rx=0 tx=0).stop 2024-04-24T23:00:41.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.225+0000 7fb4d1ab2640 1 -- 172.21.15.32:0/2081723377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4cc072cf0 msgr2=0x7fb4cc155b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:41.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.225+0000 7fb4d1ab2640 1 --2- 172.21.15.32:0/2081723377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4cc072cf0 0x7fb4cc155b60 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0000c00 tx=0x7fb4c00046c0 comp rx=0 tx=0).stop 2024-04-24T23:00:41.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.225+0000 7fb4d1ab2640 1 -- 172.21.15.32:0/2081723377 shutdown_connections 2024-04-24T23:00:41.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.225+0000 7fb4d1ab2640 1 --2- 172.21.15.32:0/2081723377 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb4b806be60 0x7fb4b806e320 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:41.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.225+0000 7fb4d1ab2640 1 --2- 172.21.15.32:0/2081723377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4cc072cf0 0x7fb4cc155b60 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:41.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.225+0000 7fb4d1ab2640 1 --2- 172.21.15.32:0/2081723377 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb4cc15c5b0 0x7fb4cc06b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:41.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.225+0000 7fb4d1ab2640 1 --2- 172.21.15.32:0/2081723377 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4cc071770 0x7fb4cc144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:41.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.225+0000 7fb4d1ab2640 1 -- 172.21.15.32:0/2081723377 >> 172.21.15.32:0/2081723377 conn(0x7fb4cc06b110 msgr2=0x7fb4cc0b5ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:41.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.225+0000 7fb4d1ab2640 1 -- 172.21.15.32:0/2081723377 shutdown_connections 2024-04-24T23:00:41.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.225+0000 7fb4d1ab2640 1 -- 172.21.15.32:0/2081723377 wait complete. 2024-04-24T23:00:41.238 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:00:41.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.337+0000 7fa430427640 1 Processor -- start 2024-04-24T23:00:41.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.337+0000 7fa430427640 1 -- start start 2024-04-24T23:00:41.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.338+0000 7fa430427640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa42806cd00 0x7fa4280ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:41.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.338+0000 7fa430427640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa42815cb00 0x7fa4280ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:41.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.338+0000 7fa430427640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4280b4520 con 0x7fa42815b380 2024-04-24T23:00:41.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.338+0000 7fa430427640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fa4280b4800 con 0x7fa42806cd00 2024-04-24T23:00:41.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.338+0000 7fa430427640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa4280b4ae0 con 0x7fa42815cb00 2024-04-24T23:00:41.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.338+0000 7fa42d9c2640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa42806cd00 0x7fa4280ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:41.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.338+0000 7fa42e1c3640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa42815b380 0x7fa42806c5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55100/0 (socket says 172.21.15.32:55100) 2024-04-24T23:00:41.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.338+0000 7fa42e1c3640 1 -- 172.21.15.32:0/1555465345 learned_addr learned my addr 172.21.15.32:0/1555465345 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:00:41.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.338+0000 7fa42e9c4640 1 --2- 172.21.15.32:0/1555465345 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa42815cb00 0x7fa4280ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:41.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.338+0000 7fa42d9c2640 1 -- 172.21.15.32:0/1555465345 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa42815cb00 msgr2=0x7fa4280ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:41.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.338+0000 7fa42d9c2640 1 --2- 172.21.15.32:0/1555465345 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa42815cb00 0x7fa4280ce970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:41.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.338+0000 7fa42d9c2640 1 -- 172.21.15.32:0/1555465345 >> v1:172.21.15.32:6789/0 conn(0x7fa42815b380 legacy=0x7fa42806c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:41.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.338+0000 7fa42d9c2640 1 -- 172.21.15.32:0/1555465345 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4280b4dc0 con 0x7fa42806cd00 2024-04-24T23:00:41.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.338+0000 7fa42e9c4640 1 --2- 172.21.15.32:0/1555465345 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa42815cb00 0x7fa4280ce970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:00:41.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.339+0000 7fa42d9c2640 1 --2- 172.21.15.32:0/1555465345 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa42806cd00 0x7fa4280ce430 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa4180097a0 tx=0x7fa41802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b10b4607f2234de server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:41.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.339+0000 7fa42d1c1640 1 -- 172.21.15.32:0/1555465345 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa418037070 con 0x7fa42806cd00 2024-04-24T23:00:41.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.339+0000 7fa42d1c1640 1 -- 172.21.15.32:0/1555465345 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa41802ae50 con 0x7fa42806cd00 2024-04-24T23:00:41.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.339+0000 7fa42d1c1640 1 -- 172.21.15.32:0/1555465345 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa418032660 con 0x7fa42806cd00 2024-04-24T23:00:41.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.339+0000 7fa430427640 1 -- 172.21.15.32:0/1555465345 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa42806cd00 msgr2=0x7fa4280ce430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:41.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.339+0000 7fa430427640 1 --2- 172.21.15.32:0/1555465345 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa42806cd00 0x7fa4280ce430 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa4180097a0 tx=0x7fa41802a100 comp rx=0 tx=0).stop 2024-04-24T23:00:41.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.340+0000 7fa430427640 1 -- 172.21.15.32:0/1555465345 shutdown_connections 2024-04-24T23:00:41.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.340+0000 7fa430427640 1 --2- 172.21.15.32:0/1555465345 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa42806cd00 0x7fa4280ce430 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:41.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.340+0000 7fa430427640 1 --2- 172.21.15.32:0/1555465345 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa42815cb00 0x7fa4280ce970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:41.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.340+0000 7fa430427640 1 -- 172.21.15.32:0/1555465345 >> 172.21.15.32:0/1555465345 conn(0x7fa42806b110 msgr2=0x7fa42806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:41.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.340+0000 7fa430427640 1 -- 172.21.15.32:0/1555465345 shutdown_connections 2024-04-24T23:00:41.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.340+0000 7fa430427640 1 -- 172.21.15.32:0/1555465345 wait complete. 2024-04-24T23:00:41.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.341+0000 7fa430427640 1 Processor -- start 2024-04-24T23:00:41.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.341+0000 7fa430427640 1 -- start start 2024-04-24T23:00:41.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.341+0000 7fa430427640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa42806cd00 0x7fa428142e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:41.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.341+0000 7fa430427640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa42815b380 0x7fa4281523a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:41.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.341+0000 7fa430427640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa42815cb00 0x7fa4281439c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:41.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.341+0000 7fa430427640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa42815b760 con 0x7fa42815b380 2024-04-24T23:00:41.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.341+0000 7fa430427640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa4280ceeb0 con 0x7fa42806cd00 2024-04-24T23:00:41.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.341+0000 7fa430427640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa4280b55c0 con 0x7fa42815cb00 2024-04-24T23:00:41.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.342+0000 7fa42d9c2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa42815b380 0x7fa4281523a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:41.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.342+0000 7fa42d9c2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa42815b380 0x7fa4281523a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53420/0 (socket says 172.21.15.32:53420) 2024-04-24T23:00:41.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.342+0000 7fa42d9c2640 1 -- 172.21.15.32:0/3470076518 learned_addr learned my addr 172.21.15.32:0/3470076518 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:41.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.342+0000 7fa42e9c4640 1 --2- 172.21.15.32:0/3470076518 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa42815cb00 0x7fa4281439c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:41.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.342+0000 7fa42e1c3640 1 --2- 172.21.15.32:0/3470076518 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa42806cd00 0x7fa428142e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:41.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.342+0000 7fa42d9c2640 1 -- 172.21.15.32:0/3470076518 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa42815cb00 msgr2=0x7fa4281439c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:41.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.342+0000 7fa42d9c2640 1 --2- 172.21.15.32:0/3470076518 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa42815cb00 0x7fa4281439c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:41.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.342+0000 7fa42d9c2640 1 -- 172.21.15.32:0/3470076518 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa42806cd00 msgr2=0x7fa428142e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:41.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.342+0000 7fa42d9c2640 1 --2- 172.21.15.32:0/3470076518 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa42806cd00 0x7fa428142e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:41.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.342+0000 7fa42d9c2640 1 -- 172.21.15.32:0/3470076518 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4281515e0 con 0x7fa42815b380 2024-04-24T23:00:41.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.342+0000 7fa42e1c3640 1 --2- 172.21.15.32:0/3470076518 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa42806cd00 0x7fa428142e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:00:41.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.342+0000 7fa42d9c2640 1 --2- 172.21.15.32:0/3470076518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa42815b380 0x7fa4281523a0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fa418035040 tx=0x7fa4180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:41.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.342+0000 7fa41effd640 1 -- 172.21.15.32:0/3470076518 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa418041070 con 0x7fa42815b380 2024-04-24T23:00:41.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.342+0000 7fa41effd640 1 -- 172.21.15.32:0/3470076518 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa41803c670 con 0x7fa42815b380 2024-04-24T23:00:41.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.342+0000 7fa430427640 1 -- 172.21.15.32:0/3470076518 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4281518c0 con 0x7fa42815b380 2024-04-24T23:00:41.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.342+0000 7fa41effd640 1 -- 172.21.15.32:0/3470076518 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa418037040 con 0x7fa42815b380 2024-04-24T23:00:41.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.342+0000 7fa430427640 1 -- 172.21.15.32:0/3470076518 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa428151e80 con 0x7fa42815b380 2024-04-24T23:00:41.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.343+0000 7fa430427640 1 -- 172.21.15.32:0/3470076518 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa42815e150 con 0x7fa42815b380 2024-04-24T23:00:41.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.344+0000 7fa41effd640 1 -- 172.21.15.32:0/3470076518 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa41803a070 con 0x7fa42815b380 2024-04-24T23:00:41.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.344+0000 7fa41effd640 1 --2- 172.21.15.32:0/3470076518 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa40c06be30 0x7fa40c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:41.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.344+0000 7fa41effd640 1 -- 172.21.15.32:0/3470076518 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fa4180ab250 con 0x7fa42815b380 2024-04-24T23:00:41.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.344+0000 7fa42e1c3640 1 --2- 172.21.15.32:0/3470076518 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa40c06be30 0x7fa40c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:41.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.345+0000 7fa42e1c3640 1 --2- 172.21.15.32:0/3470076518 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa40c06be30 0x7fa40c06e2f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa424000f60 tx=0x7fa424009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:41.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.348+0000 7fa41effd640 1 -- 172.21.15.32:0/3470076518 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa42815e150 con 0x7fa42815b380 2024-04-24T23:00:41.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.567+0000 7fa430427640 1 -- 172.21.15.32:0/3470076518 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa428072650 con 0x7fa42815b380 2024-04-24T23:00:41.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.568+0000 7fa41effd640 1 -- 172.21.15.32:0/3470076518 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5187 (secure 0 0 0) 0x7fa418079d20 con 0x7fa42815b380 2024-04-24T23:00:41.570 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:00:41.570 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":20,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4994,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/1076304225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1076304225},{"type":"v1","addr":"172.21.15.32:6834","nonce":1076304225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":16},{"gid":5000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1388367206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1388367206},{"type":"v1","addr":"172.21.15.121:6835","nonce":1388367206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":5299,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2065945523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2065945523},{"type":"v1","addr":"172.21.15.121:6837","nonce":2065945523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:00:32.376194+0000","modified":"2024-04-24T23:00:34.381949+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"c","rank":0,"incarnation":18,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/3274703471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":3274703471},{"type":"v1","addr":"172.21.15.32:6835","nonce":3274703471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5631,"qdb_cluster":[5631]},"id":3}]} 2024-04-24T23:00:41.570 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 20 2024-04-24T23:00:41.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.570+0000 7fa430427640 1 -- 172.21.15.32:0/3470076518 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa40c06be30 msgr2=0x7fa40c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:41.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.570+0000 7fa430427640 1 --2- 172.21.15.32:0/3470076518 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa40c06be30 0x7fa40c06e2f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa424000f60 tx=0x7fa424009040 comp rx=0 tx=0).stop 2024-04-24T23:00:41.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.570+0000 7fa430427640 1 -- 172.21.15.32:0/3470076518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa42815b380 msgr2=0x7fa4281523a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:41.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.570+0000 7fa430427640 1 --2- 172.21.15.32:0/3470076518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa42815b380 0x7fa4281523a0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fa418035040 tx=0x7fa4180046c0 comp rx=0 tx=0).stop 2024-04-24T23:00:41.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.570+0000 7fa430427640 1 -- 172.21.15.32:0/3470076518 shutdown_connections 2024-04-24T23:00:41.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.570+0000 7fa430427640 1 --2- 172.21.15.32:0/3470076518 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa40c06be30 0x7fa40c06e2f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:41.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.570+0000 7fa430427640 1 --2- 172.21.15.32:0/3470076518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa42815b380 0x7fa4281523a0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:41.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.570+0000 7fa430427640 1 --2- 172.21.15.32:0/3470076518 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa42815cb00 0x7fa4281439c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:41.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.570+0000 7fa430427640 1 --2- 172.21.15.32:0/3470076518 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa42806cd00 0x7fa428142e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:41.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.570+0000 7fa430427640 1 -- 172.21.15.32:0/3470076518 >> 172.21.15.32:0/3470076518 conn(0x7fa42806b110 msgr2=0x7fa42815b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:41.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.570+0000 7fa430427640 1 -- 172.21.15.32:0/3470076518 shutdown_connections 2024-04-24T23:00:41.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:41.570+0000 7fa430427640 1 -- 172.21.15.32:0/3470076518 wait complete. 2024-04-24T23:00:41.985 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T23:00:32.376194+0000', 'modified': '2024-04-24T23:00:34.381949+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5631}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5631': {'gid': 5631, 'name': 'c', 'rank': 0, 'incarnation': 18, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/3274703471', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6833', 'nonce': 3274703471}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 3274703471}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5631, 'qdb_cluster': [5631]} 2024-04-24T23:00:41.985 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-24T23:00:41.986 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-24T23:00:42.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.087+0000 7f15299d5640 1 Processor -- start 2024-04-24T23:00:42.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.087+0000 7f15299d5640 1 -- start start 2024-04-24T23:00:42.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.088+0000 7f15299d5640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f152415e9b0 0x7f15240d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:42.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.088+0000 7f15299d5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f15240d4210 0x7f152406b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:42.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.088+0000 7f15299d5640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15240d3170 con 0x7f15240ba0e0 2024-04-24T23:00:42.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.088+0000 7f15299d5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f15240b8900 con 0x7f15240d4210 2024-04-24T23:00:42.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.088+0000 7f15299d5640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f15240d3450 con 0x7f152415e9b0 2024-04-24T23:00:42.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.088+0000 7f15227fc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f15240d4210 0x7f152406b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:42.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.088+0000 7f15227fc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f15240d4210 0x7f152406b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53430/0 (socket says 172.21.15.32:53430) 2024-04-24T23:00:42.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.088+0000 7f15227fc640 1 -- 172.21.15.32:0/567895138 learned_addr learned my addr 172.21.15.32:0/567895138 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:42.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.089+0000 7f1522ffd640 1 --2- 172.21.15.32:0/567895138 >> v2:172.21.15.121:3300/0 conn(0x7f152415e9b0 0x7f15240d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:42.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.089+0000 7f15227fc640 1 -- 172.21.15.32:0/567895138 >> v2:172.21.15.121:3300/0 conn(0x7f152415e9b0 msgr2=0x7f15240d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:42.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.089+0000 7f15227fc640 1 --2- 172.21.15.32:0/567895138 >> v2:172.21.15.121:3300/0 conn(0x7f152415e9b0 0x7f15240d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:42.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.089+0000 7f15227fc640 1 -- 172.21.15.32:0/567895138 >> v1:172.21.15.121:6789/0 conn(0x7f15240ba0e0 legacy=0x7f1524071730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:42.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.089+0000 7f15227fc640 1 -- 172.21.15.32:0/567895138 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15240d3730 con 0x7f15240d4210 2024-04-24T23:00:42.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.089+0000 7f1522ffd640 1 --2- 172.21.15.32:0/567895138 >> v2:172.21.15.121:3300/0 conn(0x7f152415e9b0 0x7f15240d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:00:42.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.089+0000 7f15227fc640 1 --2- 172.21.15.32:0/567895138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15240d4210 0x7f152406b110 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f152406be20 tx=0x7f150c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc9f48610818644d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:42.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.090+0000 7f1521ffb640 1 -- 172.21.15.32:0/567895138 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c037070 con 0x7f15240d4210 2024-04-24T23:00:42.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.090+0000 7f1521ffb640 1 -- 172.21.15.32:0/567895138 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f150c032440 con 0x7f15240d4210 2024-04-24T23:00:42.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.090+0000 7f1521ffb640 1 -- 172.21.15.32:0/567895138 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c032810 con 0x7f15240d4210 2024-04-24T23:00:42.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.090+0000 7f15299d5640 1 -- 172.21.15.32:0/567895138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15240d4210 msgr2=0x7f152406b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:42.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.090+0000 7f15299d5640 1 --2- 172.21.15.32:0/567895138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15240d4210 0x7f152406b110 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f152406be20 tx=0x7f150c02a1e0 comp rx=0 tx=0).stop 2024-04-24T23:00:42.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.090+0000 7f15299d5640 1 -- 172.21.15.32:0/567895138 shutdown_connections 2024-04-24T23:00:42.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.090+0000 7f15299d5640 1 --2- 172.21.15.32:0/567895138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15240d4210 0x7f152406b110 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:42.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.090+0000 7f15299d5640 1 --2- 172.21.15.32:0/567895138 >> v2:172.21.15.121:3300/0 conn(0x7f152415e9b0 0x7f15240d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:42.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.090+0000 7f15299d5640 1 -- 172.21.15.32:0/567895138 >> 172.21.15.32:0/567895138 conn(0x7f15240d11d0 msgr2=0x7f15240d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:42.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.090+0000 7f15299d5640 1 -- 172.21.15.32:0/567895138 shutdown_connections 2024-04-24T23:00:42.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.090+0000 7f15299d5640 1 -- 172.21.15.32:0/567895138 wait complete. 2024-04-24T23:00:42.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.091+0000 7f15299d5640 1 Processor -- start 2024-04-24T23:00:42.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.092+0000 7f15299d5640 1 -- start start 2024-04-24T23:00:42.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.092+0000 7f15299d5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15240ba0e0 0x7f1524161b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:42.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.092+0000 7f15299d5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f15240d4210 0x7f152415f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:42.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.092+0000 7f15299d5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f152415e9b0 0x7f15240cd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:42.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.092+0000 7f15299d5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f15240d2400 con 0x7f15240ba0e0 2024-04-24T23:00:42.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.092+0000 7f15299d5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f15240d3cf0 con 0x7f152415e9b0 2024-04-24T23:00:42.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.092+0000 7f15299d5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f152406b650 con 0x7f15240d4210 2024-04-24T23:00:42.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.092+0000 7f1522ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15240ba0e0 0x7f1524161b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:42.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.092+0000 7f1522ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15240ba0e0 0x7f1524161b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53442/0 (socket says 172.21.15.32:53442) 2024-04-24T23:00:42.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.092+0000 7f1522ffd640 1 -- 172.21.15.32:0/2599039801 learned_addr learned my addr 172.21.15.32:0/2599039801 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:42.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.093+0000 7f15227fc640 1 --2- 172.21.15.32:0/2599039801 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f15240d4210 0x7f152415f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:42.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.093+0000 7f15237fe640 1 --2- 172.21.15.32:0/2599039801 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f152415e9b0 0x7f15240cd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:42.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.093+0000 7f1522ffd640 1 -- 172.21.15.32:0/2599039801 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f15240d4210 msgr2=0x7f152415f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:42.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.093+0000 7f1522ffd640 1 --2- 172.21.15.32:0/2599039801 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f15240d4210 0x7f152415f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:42.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.093+0000 7f1522ffd640 1 -- 172.21.15.32:0/2599039801 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f152415e9b0 msgr2=0x7f15240cd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:42.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.093+0000 7f1522ffd640 1 --2- 172.21.15.32:0/2599039801 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f152415e9b0 0x7f15240cd9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:42.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.093+0000 7f1522ffd640 1 -- 172.21.15.32:0/2599039801 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1524162040 con 0x7f15240ba0e0 2024-04-24T23:00:42.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.093+0000 7f15227fc640 1 --2- 172.21.15.32:0/2599039801 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f15240d4210 0x7f152415f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:00:42.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.093+0000 7f15237fe640 1 --2- 172.21.15.32:0/2599039801 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f152415e9b0 0x7f15240cd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:00:42.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.093+0000 7f1522ffd640 1 --2- 172.21.15.32:0/2599039801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15240ba0e0 0x7f1524161b00 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f151c00d7d0 tx=0x7f151c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:42.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.093+0000 7f151bfff640 1 -- 172.21.15.32:0/2599039801 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151c004030 con 0x7f15240ba0e0 2024-04-24T23:00:42.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.093+0000 7f151bfff640 1 -- 172.21.15.32:0/2599039801 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f151c0041d0 con 0x7f15240ba0e0 2024-04-24T23:00:42.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.093+0000 7f151bfff640 1 -- 172.21.15.32:0/2599039801 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151c0096a0 con 0x7f15240ba0e0 2024-04-24T23:00:42.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.093+0000 7f15299d5640 1 -- 172.21.15.32:0/2599039801 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1524160a60 con 0x7f15240ba0e0 2024-04-24T23:00:42.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.093+0000 7f15299d5640 1 -- 172.21.15.32:0/2599039801 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1524161020 con 0x7f15240ba0e0 2024-04-24T23:00:42.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.095+0000 7f151bfff640 1 -- 172.21.15.32:0/2599039801 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f151c009840 con 0x7f15240ba0e0 2024-04-24T23:00:42.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.095+0000 7f151bfff640 1 --2- 172.21.15.32:0/2599039801 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f150406be30 0x7f150406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:42.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.095+0000 7f151bfff640 1 -- 172.21.15.32:0/2599039801 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f151c08b160 con 0x7f15240ba0e0 2024-04-24T23:00:42.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.096+0000 7f15227fc640 1 --2- 172.21.15.32:0/2599039801 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f150406be30 0x7f150406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:42.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.096+0000 7f15227fc640 1 --2- 172.21.15.32:0/2599039801 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f150406be30 0x7f150406e2f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f150c034000 tx=0x7f150c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:42.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.096+0000 7f152c7c2740 1 Processor -- start 2024-04-24T23:00:42.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.096+0000 7f152c7c2740 1 -- start start 2024-04-24T23:00:42.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.097+0000 7f152c7c2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2f753ab60 0x55b2f74ed4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:42.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.097+0000 7f152c7c2740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2f751dc80 0x55b2f751d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:42.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.097+0000 7f152c7c2740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55b2f751d540 0x55b2f74f2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:42.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.097+0000 7f152c7c2740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55b2f7455f00 con 0x55b2f751dc80 2024-04-24T23:00:42.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.097+0000 7f152c7c2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55b2f73fe750 con 0x55b2f753ab60 2024-04-24T23:00:42.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.097+0000 7f152c7c2740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x55b2f7451ec0 con 0x55b2f751d540 2024-04-24T23:00:42.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.097+0000 7f15227fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2f751dc80 0x55b2f751d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:42.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.097+0000 7f15227fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2f751dc80 0x55b2f751d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53452/0 (socket says 172.21.15.32:53452) 2024-04-24T23:00:42.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.097+0000 7f15227fc640 1 -- 172.21.15.32:0/2722826960 learned_addr learned my addr 172.21.15.32:0/2722826960 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:42.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.097+0000 7f1522ffd640 1 --2- 172.21.15.32:0/2722826960 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2f753ab60 0x55b2f74ed4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:42.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.097+0000 7f15237fe640 1 --2- 172.21.15.32:0/2722826960 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55b2f751d540 0x55b2f74f2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:42.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.097+0000 7f15227fc640 1 -- 172.21.15.32:0/2722826960 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55b2f751d540 msgr2=0x55b2f74f2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:42.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.097+0000 7f15227fc640 1 --2- 172.21.15.32:0/2722826960 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55b2f751d540 0x55b2f74f2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:42.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.097+0000 7f15227fc640 1 -- 172.21.15.32:0/2722826960 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2f753ab60 msgr2=0x55b2f74ed4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:42.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.097+0000 7f15227fc640 1 --2- 172.21.15.32:0/2722826960 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2f753ab60 0x55b2f74ed4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:42.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.097+0000 7f15227fc640 1 -- 172.21.15.32:0/2722826960 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2f716a2d0 con 0x55b2f751dc80 2024-04-24T23:00:42.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.098+0000 7f15227fc640 1 --2- 172.21.15.32:0/2722826960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2f751dc80 0x55b2f751d000 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f150c009930 tx=0x7f150c02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dc4fd8468ae832d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:42.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.098+0000 7f15197fa640 1 -- 172.21.15.32:0/2722826960 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c04b070 con 0x55b2f751dc80 2024-04-24T23:00:42.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.098+0000 7f15197fa640 1 -- 172.21.15.32:0/2722826960 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f150c03c460 con 0x55b2f751dc80 2024-04-24T23:00:42.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.098+0000 7f15197fa640 1 -- 172.21.15.32:0/2722826960 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c037040 con 0x55b2f751dc80 2024-04-24T23:00:42.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.098+0000 7f152c7c2740 1 -- 172.21.15.32:0/2722826960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2f751dc80 msgr2=0x55b2f751d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:42.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.098+0000 7f152c7c2740 1 --2- 172.21.15.32:0/2722826960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2f751dc80 0x55b2f751d000 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f150c009930 tx=0x7f150c02c000 comp rx=0 tx=0).stop 2024-04-24T23:00:42.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.099+0000 7f152c7c2740 1 -- 172.21.15.32:0/2722826960 shutdown_connections 2024-04-24T23:00:42.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.099+0000 7f152c7c2740 1 --2- 172.21.15.32:0/2722826960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2f751dc80 0x55b2f751d000 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:42.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.099+0000 7f152c7c2740 1 --2- 172.21.15.32:0/2722826960 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55b2f751d540 0x55b2f74f2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:42.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.099+0000 7f152c7c2740 1 --2- 172.21.15.32:0/2722826960 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2f753ab60 0x55b2f74ed4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:42.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.099+0000 7f152c7c2740 1 -- 172.21.15.32:0/2722826960 >> 172.21.15.32:0/2722826960 conn(0x55b2f74ef760 msgr2=0x55b2f7538000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:42.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.099+0000 7f152c7c2740 1 -- 172.21.15.32:0/2722826960 shutdown_connections 2024-04-24T23:00:42.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.099+0000 7f152c7c2740 1 -- 172.21.15.32:0/2722826960 wait complete. 2024-04-24T23:00:42.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.099+0000 7f152c7c2740 1 -- start start 2024-04-24T23:00:42.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.099+0000 7f152c7c2740 1 Processor -- start 2024-04-24T23:00:42.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.100+0000 7f152c7c2740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2f751dc80 0x55b2f750d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:42.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.100+0000 7f152c7c2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2f753ab60 0x55b2f74ec250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:42.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.100+0000 7f152c7c2740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55b2f751ccf0 0x55b2f74ec790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:42.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.100+0000 7f152c7c2740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55b2f74f1910 con 0x55b2f751dc80 2024-04-24T23:00:42.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.100+0000 7f152c7c2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55b2f7411cf0 con 0x55b2f753ab60 2024-04-24T23:00:42.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.100+0000 7f152c7c2740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x55b2f74f0d00 con 0x55b2f751ccf0 2024-04-24T23:00:42.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.100+0000 7f152c7c2740 10 client.0 _ll_register_callbacks cb 0x55b2f74116e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:00:42.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.100+0000 7f1522ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2f751dc80 0x55b2f750d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:42.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.100+0000 7f1522ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2f751dc80 0x55b2f750d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53462/0 (socket says 172.21.15.32:53462) 2024-04-24T23:00:42.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.100+0000 7f1522ffd640 1 -- 172.21.15.32:0/3991078853 learned_addr learned my addr 172.21.15.32:0/3991078853 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:42.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.100+0000 7f15227fc640 1 --2- 172.21.15.32:0/3991078853 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2f753ab60 0x55b2f74ec250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:42.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.100+0000 7f15237fe640 1 --2- 172.21.15.32:0/3991078853 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55b2f751ccf0 0x55b2f74ec790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:42.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.100+0000 7f1522ffd640 1 -- 172.21.15.32:0/3991078853 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55b2f751ccf0 msgr2=0x55b2f74ec790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:42.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.100+0000 7f1522ffd640 1 --2- 172.21.15.32:0/3991078853 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55b2f751ccf0 0x55b2f74ec790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:42.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.100+0000 7f1522ffd640 1 -- 172.21.15.32:0/3991078853 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2f753ab60 msgr2=0x55b2f74ec250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:42.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.100+0000 7f1522ffd640 1 --2- 172.21.15.32:0/3991078853 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2f753ab60 0x55b2f74ec250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:42.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.100+0000 7f1522ffd640 1 -- 172.21.15.32:0/3991078853 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2f72346b0 con 0x55b2f751dc80 2024-04-24T23:00:42.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.100+0000 7f1522ffd640 1 --2- 172.21.15.32:0/3991078853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2f751dc80 0x55b2f750d680 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f151c055860 tx=0x7f151c059400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=731530a39c05780d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:42.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.100+0000 7f15027fc640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-24T23:00:42.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.101+0000 7f15027fc640 1 -- 172.21.15.32:0/3991078853 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151c061fe0 con 0x55b2f751dc80 2024-04-24T23:00:42.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.101+0000 7f15027fc640 1 -- 172.21.15.32:0/3991078853 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f151c055a70 con 0x55b2f751dc80 2024-04-24T23:00:42.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.101+0000 7f152c7c2740 1 -- 172.21.15.32:0/3991078853 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f150c03c460 con 0x55b2f751dc80 2024-04-24T23:00:42.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.101+0000 7f15027fc640 1 -- 172.21.15.32:0/3991078853 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f151c077050 con 0x55b2f751dc80 2024-04-24T23:00:42.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.101+0000 7f152c7c2740 10 client.5775 fetch_fsmap learned FSMap version 20 2024-04-24T23:00:42.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.101+0000 7f152c7c2740 1 -- 172.21.15.32:0/3991078853 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x55b2f754c0e0 con 0x55b2f751dc80 2024-04-24T23:00:42.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.101+0000 7f15027fc640 1 -- 172.21.15.32:0/3991078853 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f150c03c460 con 0x55b2f751dc80 2024-04-24T23:00:42.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.101+0000 7f15027fc640 1 -- 172.21.15.32:0/3991078853 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7f151c013070 con 0x55b2f751dc80 2024-04-24T23:00:42.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.101+0000 7f152c7c2740 10 client.5775 fetch_fsmap finished waiting for FSMap version 20 2024-04-24T23:00:42.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.101+0000 7f152c7c2740 10 client.5775 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-24T23:00:42.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.101+0000 7f152c7c2740 20 client.5775 populate_metadata read hostname 'smithi032' 2024-04-24T23:00:42.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.101+0000 7f152c7c2740 1 --2- 172.21.15.32:0/3991078853 >> [v2:172.21.15.32:6833/3274703471,v1:172.21.15.32:6835/3274703471] conn(0x55b2f751b3c0 0x55b2f751eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:42.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.101+0000 7f152c7c2740 4 client.5775 mds_command: new command op to 5631 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:00:42.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.101+0000 7f152c7c2740 1 -- 172.21.15.32:0/3991078853 --> [v2:172.21.15.32:6833/3274703471,v1:172.21.15.32:6835/3274703471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2f7463440 con 0x55b2f751b3c0 2024-04-24T23:00:42.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.101+0000 7f15227fc640 1 --2- 172.21.15.32:0/3991078853 >> [v2:172.21.15.32:6833/3274703471,v1:172.21.15.32:6835/3274703471] conn(0x55b2f751b3c0 0x55b2f751eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:42.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.102+0000 7f15027fc640 10 client.5775 ms_handle_connect on v2:172.21.15.32:6833/3274703471 2024-04-24T23:00:42.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.102+0000 7f15227fc640 1 --2- 172.21.15.32:0/3991078853 >> [v2:172.21.15.32:6833/3274703471,v1:172.21.15.32:6835/3274703471] conn(0x55b2f751b3c0 0x55b2f751eee0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5876c9445b92e6a server_cookie=ca253b64fc2b3a09 in_seq=0 out_seq=0 2024-04-24T23:00:42.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.102+0000 7f15027fc640 1 -- 172.21.15.32:0/3991078853 <== mds.0 v2:172.21.15.32:6833/3274703471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b2f7463440 con 0x55b2f751b3c0 2024-04-24T23:00:42.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.102+0000 7f15027fc640 10 client.5775 handle_command_reply: tid=0 2024-04-24T23:00:42.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.102+0000 7f152c7c2740 1 client.5775 shutdown 2024-04-24T23:00:42.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.102+0000 7f152c7c2740 1 -- 172.21.15.32:0/3991078853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2f751dc80 msgr2=0x55b2f750d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:42.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.102+0000 7f152c7c2740 1 --2- 172.21.15.32:0/3991078853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2f751dc80 0x55b2f750d680 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f151c055860 tx=0x7f151c059400 comp rx=0 tx=0).stop 2024-04-24T23:00:42.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.102+0000 7f152c7c2740 1 -- 172.21.15.32:0/3991078853 shutdown_connections 2024-04-24T23:00:42.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.102+0000 7f152c7c2740 1 --2- 172.21.15.32:0/3991078853 >> [v2:172.21.15.32:6833/3274703471,v1:172.21.15.32:6835/3274703471] conn(0x55b2f751b3c0 0x55b2f751eee0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:42.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.102+0000 7f152c7c2740 1 --2- 172.21.15.32:0/3991078853 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55b2f751ccf0 0x55b2f74ec790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:42.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.102+0000 7f152c7c2740 1 --2- 172.21.15.32:0/3991078853 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2f753ab60 0x55b2f74ec250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:42.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.102+0000 7f152c7c2740 1 --2- 172.21.15.32:0/3991078853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2f751dc80 0x55b2f750d680 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:42.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.102+0000 7f15027fc640 0 client.5775 ms_handle_reset on v2:172.21.15.32:6833/3274703471 2024-04-24T23:00:42.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.102+0000 7f152c7c2740 1 -- 172.21.15.32:0/3991078853 >> 172.21.15.32:0/3991078853 conn(0x55b2f751d540 msgr2=0x55b2f7538840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:42.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.102+0000 7f152c7c2740 1 -- 172.21.15.32:0/3991078853 shutdown_connections 2024-04-24T23:00:42.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.102+0000 7f152c7c2740 1 -- 172.21.15.32:0/3991078853 wait complete. 2024-04-24T23:00:42.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.103+0000 7f152c7c2740 20 client.5775 trim_cache size 0 max 16384 2024-04-24T23:00:42.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.121+0000 7f152c7c2740 1 Processor -- start 2024-04-24T23:00:42.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.121+0000 7f152c7c2740 1 -- start start 2024-04-24T23:00:42.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.122+0000 7f152c7c2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2f751ccf0 0x55b2f7518780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:42.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.122+0000 7f152c7c2740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55b2f751dc80 0x55b2f752f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:42.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.122+0000 7f152c7c2740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2f753ab60 0x55b2f752f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:42.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.122+0000 7f152c7c2740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f15240ca000 con 0x55b2f753ab60 2024-04-24T23:00:42.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.122+0000 7f152c7c2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f15240c9360 con 0x55b2f751ccf0 2024-04-24T23:00:42.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.122+0000 7f152c7c2740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f15240c99b0 con 0x55b2f751dc80 2024-04-24T23:00:42.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.122+0000 7f15237fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2f753ab60 0x55b2f752f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:42.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.122+0000 7f15237fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2f753ab60 0x55b2f752f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53478/0 (socket says 172.21.15.32:53478) 2024-04-24T23:00:42.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.122+0000 7f15237fe640 1 -- 172.21.15.32:0/773164319 learned_addr learned my addr 172.21.15.32:0/773164319 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:42.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.122+0000 7f1522ffd640 1 --2- 172.21.15.32:0/773164319 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2f751ccf0 0x55b2f7518780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:42.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.122+0000 7f15227fc640 1 --2- 172.21.15.32:0/773164319 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55b2f751dc80 0x55b2f752f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:42.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.122+0000 7f15237fe640 1 -- 172.21.15.32:0/773164319 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55b2f751dc80 msgr2=0x55b2f752f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:42.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.122+0000 7f15237fe640 1 --2- 172.21.15.32:0/773164319 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55b2f751dc80 0x55b2f752f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:42.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.122+0000 7f15237fe640 1 -- 172.21.15.32:0/773164319 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2f751ccf0 msgr2=0x55b2f7518780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:42.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.122+0000 7f15237fe640 1 --2- 172.21.15.32:0/773164319 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2f751ccf0 0x55b2f7518780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:42.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.122+0000 7f15237fe640 1 -- 172.21.15.32:0/773164319 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2f74f0890 con 0x55b2f753ab60 2024-04-24T23:00:42.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.122+0000 7f1522ffd640 1 --2- 172.21.15.32:0/773164319 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2f751ccf0 0x55b2f7518780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:00:42.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.122+0000 7f15227fc640 1 --2- 172.21.15.32:0/773164319 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55b2f751dc80 0x55b2f752f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:00:42.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.122+0000 7f15237fe640 1 --2- 172.21.15.32:0/773164319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2f753ab60 0x55b2f752f920 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1514001e90 tx=0x7f1514013700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87e7fe2f0023dd89 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:42.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.122+0000 7f1518ff9640 1 -- 172.21.15.32:0/773164319 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1514006b50 con 0x55b2f753ab60 2024-04-24T23:00:42.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.122+0000 7f1518ff9640 1 -- 172.21.15.32:0/773164319 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f151401a020 con 0x55b2f753ab60 2024-04-24T23:00:42.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.122+0000 7f1518ff9640 1 -- 172.21.15.32:0/773164319 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1514003970 con 0x55b2f753ab60 2024-04-24T23:00:42.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.123+0000 7f152c7c2740 1 -- 172.21.15.32:0/773164319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2f753ab60 msgr2=0x55b2f752f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:42.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.123+0000 7f152c7c2740 1 --2- 172.21.15.32:0/773164319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2f753ab60 0x55b2f752f920 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1514001e90 tx=0x7f1514013700 comp rx=0 tx=0).stop 2024-04-24T23:00:42.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.123+0000 7f152c7c2740 1 -- 172.21.15.32:0/773164319 shutdown_connections 2024-04-24T23:00:42.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.123+0000 7f152c7c2740 1 --2- 172.21.15.32:0/773164319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2f753ab60 0x55b2f752f920 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:42.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.123+0000 7f152c7c2740 1 --2- 172.21.15.32:0/773164319 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55b2f751dc80 0x55b2f752f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:42.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.123+0000 7f152c7c2740 1 --2- 172.21.15.32:0/773164319 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2f751ccf0 0x55b2f7518780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:42.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.123+0000 7f152c7c2740 1 -- 172.21.15.32:0/773164319 >> 172.21.15.32:0/773164319 conn(0x55b2f751d540 msgr2=0x55b2f750d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:42.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.123+0000 7f152c7c2740 1 -- 172.21.15.32:0/773164319 shutdown_connections 2024-04-24T23:00:42.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.123+0000 7f152c7c2740 1 -- 172.21.15.32:0/773164319 wait complete. 2024-04-24T23:00:42.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.124+0000 7f152c7c2740 1 -- start start 2024-04-24T23:00:42.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.124+0000 7f152c7c2740 1 Processor -- start 2024-04-24T23:00:42.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.124+0000 7f152c7c2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2f751ccf0 0x55b2f7519760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:42.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.124+0000 7f152c7c2740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2f751dc80 0x55b2f74f2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:42.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.124+0000 7f152c7c2740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55b2f753ab60 0x55b2f752f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:42.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.124+0000 7f152c7c2740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55b2f751b240 con 0x55b2f751dc80 2024-04-24T23:00:42.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.124+0000 7f152c7c2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55b2f751f960 con 0x55b2f751ccf0 2024-04-24T23:00:42.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.124+0000 7f152c7c2740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x55b2f751b890 con 0x55b2f753ab60 2024-04-24T23:00:42.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.124+0000 7f152c7c2740 10 client.0 _ll_register_callbacks cb 0x7f151c059b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:00:42.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.124+0000 7f15227fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2f751dc80 0x55b2f74f2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:42.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.125+0000 7f15227fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2f751dc80 0x55b2f74f2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53486/0 (socket says 172.21.15.32:53486) 2024-04-24T23:00:42.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.125+0000 7f15227fc640 1 -- 172.21.15.32:0/3365644931 learned_addr learned my addr 172.21.15.32:0/3365644931 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:42.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.125+0000 7f15237fe640 1 --2- 172.21.15.32:0/3365644931 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55b2f753ab60 0x55b2f752f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:42.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.125+0000 7f1522ffd640 1 --2- 172.21.15.32:0/3365644931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2f751ccf0 0x55b2f7519760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:42.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.125+0000 7f15227fc640 1 -- 172.21.15.32:0/3365644931 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55b2f753ab60 msgr2=0x55b2f752f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:42.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.125+0000 7f15227fc640 1 --2- 172.21.15.32:0/3365644931 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55b2f753ab60 0x55b2f752f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:42.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.125+0000 7f15227fc640 1 -- 172.21.15.32:0/3365644931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2f751ccf0 msgr2=0x55b2f7519760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:42.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.125+0000 7f15227fc640 1 --2- 172.21.15.32:0/3365644931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2f751ccf0 0x55b2f7519760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:42.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.125+0000 7f15227fc640 1 -- 172.21.15.32:0/3365644931 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2f7535590 con 0x55b2f751dc80 2024-04-24T23:00:42.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.125+0000 7f15227fc640 1 --2- 172.21.15.32:0/3365644931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2f751dc80 0x55b2f74f2d10 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f150c049040 tx=0x7f150c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fda6c20da785185e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:42.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.125+0000 7f15027fc640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-24T23:00:42.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.125+0000 7f15027fc640 1 -- 172.21.15.32:0/3365644931 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c054020 con 0x55b2f751dc80 2024-04-24T23:00:42.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.125+0000 7f15027fc640 1 -- 172.21.15.32:0/3365644931 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f150c004200 con 0x55b2f751dc80 2024-04-24T23:00:42.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.125+0000 7f152c7c2740 1 -- 172.21.15.32:0/3365644931 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f151401a020 con 0x55b2f751dc80 2024-04-24T23:00:42.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.125+0000 7f15027fc640 1 -- 172.21.15.32:0/3365644931 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f150c05e050 con 0x55b2f751dc80 2024-04-24T23:00:42.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.125+0000 7f152c7c2740 10 client.5781 fetch_fsmap learned FSMap version 20 2024-04-24T23:00:42.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.125+0000 7f152c7c2740 1 -- 172.21.15.32:0/3365644931 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x55b2f75989f0 con 0x55b2f751dc80 2024-04-24T23:00:42.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.125+0000 7f15027fc640 1 -- 172.21.15.32:0/3365644931 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f151401a020 con 0x55b2f751dc80 2024-04-24T23:00:42.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.126+0000 7f15027fc640 1 -- 172.21.15.32:0/3365644931 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7f150c061020 con 0x55b2f751dc80 2024-04-24T23:00:42.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.126+0000 7f152c7c2740 10 client.5781 fetch_fsmap finished waiting for FSMap version 20 2024-04-24T23:00:42.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.126+0000 7f152c7c2740 10 client.5781 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-24T23:00:42.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.126+0000 7f152c7c2740 20 client.5781 populate_metadata read hostname 'smithi032' 2024-04-24T23:00:42.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.126+0000 7f152c7c2740 1 --2- 172.21.15.32:0/3365644931 >> [v2:172.21.15.32:6833/3274703471,v1:172.21.15.32:6835/3274703471] conn(0x55b2f752fb30 0x55b2f755ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:42.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.126+0000 7f152c7c2740 4 client.5781 mds_command: new command op to 5631 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:00:42.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.126+0000 7f152c7c2740 1 -- 172.21.15.32:0/3365644931 --> [v2:172.21.15.32:6833/3274703471,v1:172.21.15.32:6835/3274703471] -- command(tid 0: {"prefix": "status"}) -- 0x55b2f753a9b0 con 0x55b2f752fb30 2024-04-24T23:00:42.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.126+0000 7f1522ffd640 1 --2- 172.21.15.32:0/3365644931 >> [v2:172.21.15.32:6833/3274703471,v1:172.21.15.32:6835/3274703471] conn(0x55b2f752fb30 0x55b2f755ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:42.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.126+0000 7f15027fc640 10 client.5781 ms_handle_connect on v2:172.21.15.32:6833/3274703471 2024-04-24T23:00:42.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.126+0000 7f1522ffd640 1 --2- 172.21.15.32:0/3365644931 >> [v2:172.21.15.32:6833/3274703471,v1:172.21.15.32:6835/3274703471] conn(0x55b2f752fb30 0x55b2f755ab30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=320b642bcb8e2f5a server_cookie=aefe9398db20b5eb in_seq=0 out_seq=0 2024-04-24T23:00:42.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.126+0000 7f15027fc640 1 -- 172.21.15.32:0/3365644931 <== mds.0 v2:172.21.15.32:6833/3274703471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55b2f753a9b0 con 0x55b2f752fb30 2024-04-24T23:00:42.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.126+0000 7f15027fc640 10 client.5781 handle_command_reply: tid=0 2024-04-24T23:00:42.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.126+0000 7f152c7c2740 1 client.5781 shutdown 2024-04-24T23:00:42.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.126+0000 7f152c7c2740 1 -- 172.21.15.32:0/3365644931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2f751dc80 msgr2=0x55b2f74f2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:42.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.126+0000 7f152c7c2740 1 --2- 172.21.15.32:0/3365644931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2f751dc80 0x55b2f74f2d10 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f150c049040 tx=0x7f150c004730 comp rx=0 tx=0).stop 2024-04-24T23:00:42.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.126+0000 7f152c7c2740 1 -- 172.21.15.32:0/3365644931 shutdown_connections 2024-04-24T23:00:42.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.126+0000 7f152c7c2740 1 --2- 172.21.15.32:0/3365644931 >> [v2:172.21.15.32:6833/3274703471,v1:172.21.15.32:6835/3274703471] conn(0x55b2f752fb30 0x55b2f755ab30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:42.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.126+0000 7f152c7c2740 1 --2- 172.21.15.32:0/3365644931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2f751dc80 0x55b2f74f2d10 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:42.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.126+0000 7f15027fc640 0 client.5781 ms_handle_reset on v2:172.21.15.32:6833/3274703471 2024-04-24T23:00:42.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.126+0000 7f152c7c2740 1 --2- 172.21.15.32:0/3365644931 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55b2f753ab60 0x55b2f752f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:42.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.126+0000 7f152c7c2740 1 --2- 172.21.15.32:0/3365644931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2f751ccf0 0x55b2f7519760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:42.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.126+0000 7f152c7c2740 1 -- 172.21.15.32:0/3365644931 >> 172.21.15.32:0/3365644931 conn(0x55b2f751d540 msgr2=0x55b2f74e8690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:42.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.126+0000 7f152c7c2740 1 -- 172.21.15.32:0/3365644931 shutdown_connections 2024-04-24T23:00:42.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.126+0000 7f152c7c2740 1 -- 172.21.15.32:0/3365644931 wait complete. 2024-04-24T23:00:42.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.127+0000 7f152c7c2740 20 client.5781 trim_cache size 0 max 16384 2024-04-24T23:00:42.136 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:00:42.136 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", 2024-04-24T23:00:42.136 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2024-04-24T23:00:42.136 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 5631, 2024-04-24T23:00:42.136 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-04-24T23:00:42.136 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-04-24T23:00:42.137 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-04-24T23:00:42.137 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 9.1180369900000002, 2024-04-24T23:00:42.137 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 20, 2024-04-24T23:00:42.137 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 45, 2024-04-24T23:00:42.137 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 45, 2024-04-24T23:00:42.137 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 14.023855072 2024-04-24T23:00:42.137 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:00:42.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.127+0000 7f15299d5640 1 -- 172.21.15.32:0/2599039801 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f150406be30 msgr2=0x7f150406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:42.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.127+0000 7f15299d5640 1 --2- 172.21.15.32:0/2599039801 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f150406be30 0x7f150406e2f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f150c034000 tx=0x7f150c02e000 comp rx=0 tx=0).stop 2024-04-24T23:00:42.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.127+0000 7f15299d5640 1 -- 172.21.15.32:0/2599039801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15240ba0e0 msgr2=0x7f1524161b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:42.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.127+0000 7f15299d5640 1 --2- 172.21.15.32:0/2599039801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15240ba0e0 0x7f1524161b00 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f151c00d7d0 tx=0x7f151c00dc90 comp rx=0 tx=0).stop 2024-04-24T23:00:42.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.127+0000 7f15299d5640 1 -- 172.21.15.32:0/2599039801 shutdown_connections 2024-04-24T23:00:42.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.127+0000 7f15299d5640 1 --2- 172.21.15.32:0/2599039801 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f150406be30 0x7f150406e2f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:42.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.127+0000 7f15299d5640 1 --2- 172.21.15.32:0/2599039801 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f152415e9b0 0x7f15240cd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:42.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.127+0000 7f15299d5640 1 --2- 172.21.15.32:0/2599039801 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f15240d4210 0x7f152415f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:42.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.127+0000 7f15299d5640 1 --2- 172.21.15.32:0/2599039801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15240ba0e0 0x7f1524161b00 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:42.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.127+0000 7f15299d5640 1 -- 172.21.15.32:0/2599039801 >> 172.21.15.32:0/2599039801 conn(0x7f15240d11d0 msgr2=0x7f15240d46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:42.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.127+0000 7f15299d5640 1 -- 172.21.15.32:0/2599039801 shutdown_connections 2024-04-24T23:00:42.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.127+0000 7f15299d5640 1 -- 172.21.15.32:0/2599039801 wait complete. 2024-04-24T23:00:42.140 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T23:00:42.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.240+0000 7f7b70516640 1 Processor -- start 2024-04-24T23:00:42.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.240+0000 7f7b70516640 1 -- start start 2024-04-24T23:00:42.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.241+0000 7f7b70516640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7b6815cc40 0x7f7b6815d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:42.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.241+0000 7f7b70516640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b680b4ee0 con 0x7f7b6815d560 2024-04-24T23:00:42.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.241+0000 7f7b70516640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b680b51c0 con 0x7f7b680d05b0 2024-04-24T23:00:42.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.241+0000 7f7b70516640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f7b6806cca0 con 0x7f7b6815cc40 2024-04-24T23:00:42.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.241+0000 7f7b6eab3640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7b6815d560 0x7f7b680cf340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55104/0 (socket says 172.21.15.32:55104) 2024-04-24T23:00:42.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.241+0000 7f7b6eab3640 1 -- 172.21.15.32:0/1130648336 learned_addr learned my addr 172.21.15.32:0/1130648336 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:00:42.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.241+0000 7f7b6e2b2640 1 --2- 172.21.15.32:0/1130648336 >> v2:172.21.15.32:3300/0 conn(0x7f7b6815cc40 0x7f7b6815d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:42.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.241+0000 7f7b6e2b2640 1 -- 172.21.15.32:0/1130648336 >> v1:172.21.15.121:6789/0 conn(0x7f7b680d05b0 legacy=0x7f7b680cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:42.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.241+0000 7f7b6e2b2640 1 -- 172.21.15.32:0/1130648336 >> v1:172.21.15.32:6789/0 conn(0x7f7b6815d560 legacy=0x7f7b680cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:42.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.241+0000 7f7b6d2b0640 1 -- 172.21.15.32:0/1130648336 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 378904621 0 0) 0x7f7b680b4ee0 con 0x7f7b6815d560 2024-04-24T23:00:42.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.241+0000 7f7b6e2b2640 1 -- 172.21.15.32:0/1130648336 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b680b54a0 con 0x7f7b6815cc40 2024-04-24T23:00:42.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.242+0000 7f7b6e2b2640 1 --2- 172.21.15.32:0/1130648336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b6815cc40 0x7f7b6815d020 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f7b640097a0 tx=0x7f7b6402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c513a72a863c6c6d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:42.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.242+0000 7f7b6d2b0640 1 -- 172.21.15.32:0/1130648336 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b64037070 con 0x7f7b6815cc40 2024-04-24T23:00:42.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.242+0000 7f7b6d2b0640 1 -- 172.21.15.32:0/1130648336 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b64032440 con 0x7f7b6815cc40 2024-04-24T23:00:42.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.242+0000 7f7b6d2b0640 1 -- 172.21.15.32:0/1130648336 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b64032810 con 0x7f7b6815cc40 2024-04-24T23:00:42.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.242+0000 7f7b70516640 1 -- 172.21.15.32:0/1130648336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b6815cc40 msgr2=0x7f7b6815d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:42.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.242+0000 7f7b70516640 1 --2- 172.21.15.32:0/1130648336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b6815cc40 0x7f7b6815d020 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f7b640097a0 tx=0x7f7b6402a1e0 comp rx=0 tx=0).stop 2024-04-24T23:00:42.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.243+0000 7f7b70516640 1 -- 172.21.15.32:0/1130648336 shutdown_connections 2024-04-24T23:00:42.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.243+0000 7f7b70516640 1 --2- 172.21.15.32:0/1130648336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b6815cc40 0x7f7b6815d020 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:42.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.243+0000 7f7b70516640 1 -- 172.21.15.32:0/1130648336 >> 172.21.15.32:0/1130648336 conn(0x7f7b680d0ae0 msgr2=0x7f7b680d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:42.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.243+0000 7f7b70516640 1 -- 172.21.15.32:0/1130648336 shutdown_connections 2024-04-24T23:00:42.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.243+0000 7f7b70516640 1 -- 172.21.15.32:0/1130648336 wait complete. 2024-04-24T23:00:42.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.244+0000 7f7b70516640 1 Processor -- start 2024-04-24T23:00:42.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.244+0000 7f7b70516640 1 -- start start 2024-04-24T23:00:42.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.244+0000 7f7b70516640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7b680d05b0 0x7f7b681e52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:42.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.244+0000 7f7b70516640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b6815cc40 0x7f7b6813ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:42.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.244+0000 7f7b70516640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7b6815d560 0x7f7b681e7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:42.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.244+0000 7f7b70516640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7b680b5a60 con 0x7f7b6815cc40 2024-04-24T23:00:42.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.244+0000 7f7b70516640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7b680ce5f0 con 0x7f7b680d05b0 2024-04-24T23:00:42.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.244+0000 7f7b70516640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f7b6815d9b0 con 0x7f7b6815d560 2024-04-24T23:00:42.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.245+0000 7f7b6dab1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b6815cc40 0x7f7b6813ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:42.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.245+0000 7f7b6dab1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b6815cc40 0x7f7b6813ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53502/0 (socket says 172.21.15.32:53502) 2024-04-24T23:00:42.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.245+0000 7f7b6dab1640 1 -- 172.21.15.32:0/1506132390 learned_addr learned my addr 172.21.15.32:0/1506132390 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:42.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.245+0000 7f7b6e2b2640 1 --2- 172.21.15.32:0/1506132390 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7b680d05b0 0x7f7b681e52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:42.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.245+0000 7f7b6eab3640 1 --2- 172.21.15.32:0/1506132390 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7b6815d560 0x7f7b681e7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:42.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.245+0000 7f7b6dab1640 1 -- 172.21.15.32:0/1506132390 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7b6815d560 msgr2=0x7f7b681e7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:42.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.245+0000 7f7b6dab1640 1 --2- 172.21.15.32:0/1506132390 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7b6815d560 0x7f7b681e7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:42.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.245+0000 7f7b6dab1640 1 -- 172.21.15.32:0/1506132390 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7b680d05b0 msgr2=0x7f7b681e52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:42.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.245+0000 7f7b6dab1640 1 --2- 172.21.15.32:0/1506132390 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7b680d05b0 0x7f7b681e52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:42.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.245+0000 7f7b6dab1640 1 -- 172.21.15.32:0/1506132390 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b6813d620 con 0x7f7b6815cc40 2024-04-24T23:00:42.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.245+0000 7f7b6dab1640 1 --2- 172.21.15.32:0/1506132390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b6815cc40 0x7f7b6813ce50 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f7b5800b3f0 tx=0x7f7b5800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:42.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.245+0000 7f7b5effd640 1 -- 172.21.15.32:0/1506132390 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b58015020 con 0x7f7b6815cc40 2024-04-24T23:00:42.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.245+0000 7f7b5effd640 1 -- 172.21.15.32:0/1506132390 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b58011830 con 0x7f7b6815cc40 2024-04-24T23:00:42.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.245+0000 7f7b5effd640 1 -- 172.21.15.32:0/1506132390 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b58011c20 con 0x7f7b6815cc40 2024-04-24T23:00:42.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.245+0000 7f7b70516640 1 -- 172.21.15.32:0/1506132390 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b6813d900 con 0x7f7b6815cc40 2024-04-24T23:00:42.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.245+0000 7f7b70516640 1 -- 172.21.15.32:0/1506132390 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b6813dec0 con 0x7f7b6815cc40 2024-04-24T23:00:42.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.246+0000 7f7b70516640 1 -- 172.21.15.32:0/1506132390 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b68068430 con 0x7f7b6815cc40 2024-04-24T23:00:42.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.247+0000 7f7b5effd640 1 -- 172.21.15.32:0/1506132390 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7b58018020 con 0x7f7b6815cc40 2024-04-24T23:00:42.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.247+0000 7f7b5effd640 1 --2- 172.21.15.32:0/1506132390 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7b4c06be60 0x7f7b4c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:42.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.247+0000 7f7b5effd640 1 -- 172.21.15.32:0/1506132390 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f7b5808ab00 con 0x7f7b6815cc40 2024-04-24T23:00:42.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.247+0000 7f7b6e2b2640 1 --2- 172.21.15.32:0/1506132390 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7b4c06be60 0x7f7b4c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:42.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.248+0000 7f7b6e2b2640 1 --2- 172.21.15.32:0/1506132390 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7b4c06be60 0x7f7b4c06e320 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7b64034000 tx=0x7f7b6402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:42.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.251+0000 7f7b5effd640 1 -- 172.21.15.32:0/1506132390 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7b5801f030 con 0x7f7b6815cc40 2024-04-24T23:00:42.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.474+0000 7f7b70516640 1 -- 172.21.15.32:0/1506132390 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7b680cd2e0 con 0x7f7b6815cc40 2024-04-24T23:00:42.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.475+0000 7f7b5effd640 1 -- 172.21.15.32:0/1506132390 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v20) ==== 53+0+73 (secure 0 0 0) 0x7f7b580172c0 con 0x7f7b6815cc40 2024-04-24T23:00:42.476 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T23:00:42.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.477+0000 7f7b70516640 1 -- 172.21.15.32:0/1506132390 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7b4c06be60 msgr2=0x7f7b4c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:42.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.477+0000 7f7b70516640 1 --2- 172.21.15.32:0/1506132390 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7b4c06be60 0x7f7b4c06e320 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7b64034000 tx=0x7f7b6402e000 comp rx=0 tx=0).stop 2024-04-24T23:00:42.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.477+0000 7f7b70516640 1 -- 172.21.15.32:0/1506132390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b6815cc40 msgr2=0x7f7b6813ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:42.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.477+0000 7f7b70516640 1 --2- 172.21.15.32:0/1506132390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b6815cc40 0x7f7b6813ce50 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f7b5800b3f0 tx=0x7f7b5800b8b0 comp rx=0 tx=0).stop 2024-04-24T23:00:42.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.477+0000 7f7b70516640 1 -- 172.21.15.32:0/1506132390 shutdown_connections 2024-04-24T23:00:42.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.477+0000 7f7b70516640 1 --2- 172.21.15.32:0/1506132390 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7b4c06be60 0x7f7b4c06e320 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:42.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.477+0000 7f7b70516640 1 --2- 172.21.15.32:0/1506132390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b6815cc40 0x7f7b6813ce50 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:42.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.477+0000 7f7b70516640 1 --2- 172.21.15.32:0/1506132390 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7b6815d560 0x7f7b681e7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:42.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.477+0000 7f7b70516640 1 --2- 172.21.15.32:0/1506132390 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7b680d05b0 0x7f7b681e52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:42.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.477+0000 7f7b70516640 1 -- 172.21.15.32:0/1506132390 >> 172.21.15.32:0/1506132390 conn(0x7f7b680d0ae0 msgr2=0x7f7b680b1090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:42.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.477+0000 7f7b70516640 1 -- 172.21.15.32:0/1506132390 shutdown_connections 2024-04-24T23:00:42.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42.477+0000 7f7b70516640 1 -- 172.21.15.32:0/1506132390 wait complete. 2024-04-24T23:00:42.489 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-24T23:00:42.489 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T23:00:42.489 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2024-04-24T23:00:42.489 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-24T23:00:42.489 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T23:00:42.489 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-24T23:00:42.489 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-24T23:00:42.489 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T23:00:42.489 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:00:42.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:00:42.490 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2024-04-24T23:00:42.505 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T23:00:42.506 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T23:00:42.506 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T23:00:42.506 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:00:42.506 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2024-04-24T23:00:42.506 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:00:42.506 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:00:42.506 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2024-04-24T23:00:42.506 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2024-04-24T23:00:42.506 INFO:teuthology.orchestra.run.smithi032.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:00:42.506 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2024-04-24T23:00:42.506 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2024-04-24T23:00:42.506 INFO:teuthology.orchestra.run.smithi032.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T23:00:42.507 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:00:42.507 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2024-04-24T23:00:42.507 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-24T23:00:42.507 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 42666sec preferred_lft 42666sec 2024-04-24T23:00:42.507 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link noprefixroute 2024-04-24T23:00:42.507 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:00:42.507 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:00:42.507 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:00:42.507 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2024-04-24T23:00:42.507 INFO:teuthology.orchestra.run.smithi032.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T23:00:42.507 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 72:d4:96:00:43:8c brd ff:ff:ff:ff:ff:ff 2024-04-24T23:00:42.507 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-24T23:00:42.507 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:00:42.507 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::f4c6:20ff:fe22:2647/64 scope link 2024-04-24T23:00:42.507 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:00:42.508 INFO:teuthology.orchestra.run.smithi032.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:00:42.508 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 72:d4:96:00:43:8c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T23:00:42.508 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::70d4:96ff:fe00:438c/64 scope link 2024-04-24T23:00:42.508 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:00:42.508 INFO:teuthology.orchestra.run.smithi032.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:00:42.508 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether e6:6b:2a:90:1a:f8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T23:00:42.508 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::e46b:2aff:fe90:1af8/64 scope link 2024-04-24T23:00:42.508 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:00:42.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:00:42.508 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T23:00:42.569 INFO:teuthology.orchestra.run.smithi032.stdout:1 2024-04-24T23:00:42.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:00:42.571 DEBUG:teuthology.orchestra.run.smithi032:> route 2024-04-24T23:00:42.628 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2024-04-24T23:00:42.628 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T23:00:42.628 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-24T23:00:42.628 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-24T23:00:42.628 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T23:00:42.628 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:00:42.628 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-24T23:00:42.629 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-24T23:00:42.629 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-24T23:00:42.629 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-24T23:00:42.629 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-24T23:00:42.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:00:42.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:00:42.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:00:42.754 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-04-24T23:00:42.770 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-24T23:00:42.770 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-24T23:00:42.770 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:00:42.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:00:42.770 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2024-04-24T23:00:42.835 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:00:42.835 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-24T23:00:42.891 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:00:42.891 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:00:42.892 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-24T23:00:42.946 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-24T23:00:42.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:00:42.947 DEBUG:teuthology.orchestra.run.smithi032:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:00:43.029 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:00:43.030 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-24T23:00:43.041 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:00:43.041 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-24T23:00:43.041 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.121:3300,172.21.15.121:3301" 2024-04-24T23:00:43.041 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: trying mount with new device syntax: 0@eb1b7a01-6f65-4534-9a12-83011862ba51.cephfs=/ 2024-04-24T23:00:43.042 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.32:3300/172.21.15.121:3300/172.21.15.121:3301" will pass to kernel 2024-04-24T23:00:43.044 INFO:teuthology.orchestra.run.smithi032.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-24T23:00:43.044 INFO:teuthology.orchestra.run.smithi032.stdout: You just mounted a file system that supports labels which does not 2024-04-24T23:00:43.044 INFO:teuthology.orchestra.run.smithi032.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-24T23:00:43.045 INFO:teuthology.orchestra.run.smithi032.stdout: applications will generate AVC messages and not be allowed access to 2024-04-24T23:00:43.045 INFO:teuthology.orchestra.run.smithi032.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-24T23:00:43.045 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-24T23:00:43.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:00:43.046 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-24T23:00:43.112 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:00:43.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:00:43.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:00:43.213 INFO:teuthology.orchestra.run.smithi032.stdout:client5102 2024-04-24T23:00:43.214 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:00:43.214 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:00:43.230 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T23:00:43.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.369+0000 7f0d78e2a640 1 Processor -- start 2024-04-24T23:00:43.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.369+0000 7f0d78e2a640 1 -- start start 2024-04-24T23:00:43.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.370+0000 7f0d78e2a640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0d7415c8a0 0x7f0d740ae140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:43.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.370+0000 7f0d78e2a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0d740ce7a0 0x7f0d7406c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:43.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.370+0000 7f0d78e2a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d74074210 0x7f0d740746a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:43.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.370+0000 7f0d78e2a640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f0d7406ca70 con 0x7f0d740ce7a0 2024-04-24T23:00:43.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.370+0000 7f0d78e2a640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f0d74074f00 con 0x7f0d7415c8a0 2024-04-24T23:00:43.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.370+0000 7f0d78e2a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f0d740b0de0 con 0x7f0d74074210 2024-04-24T23:00:43.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.370+0000 7f0d6bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0d740ce7a0 0x7f0d7406c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:43.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.370+0000 7f0d6bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0d740ce7a0 0x7f0d7406c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53514/0 (socket says 172.21.15.32:53514) 2024-04-24T23:00:43.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.370+0000 7f0d6bfff640 1 -- 172.21.15.32:0/2208700010 learned_addr learned my addr 172.21.15.32:0/2208700010 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:43.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.370+0000 7f0d7259c640 1 --2- 172.21.15.32:0/2208700010 >> v2:172.21.15.121:3300/0 conn(0x7f0d7415c8a0 0x7f0d740ae140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:43.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.370+0000 7f0d6bfff640 1 -- 172.21.15.32:0/2208700010 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d74074210 msgr2=0x7f0d740746a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:43.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.370+0000 7f0d72d9d640 1 --2- 172.21.15.32:0/2208700010 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d74074210 0x7f0d740746a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:43.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.370+0000 7f0d6bfff640 1 --2- 172.21.15.32:0/2208700010 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d74074210 0x7f0d740746a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:43.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.370+0000 7f0d6bfff640 1 -- 172.21.15.32:0/2208700010 >> v2:172.21.15.121:3300/0 conn(0x7f0d7415c8a0 msgr2=0x7f0d740ae140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:43.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.370+0000 7f0d6bfff640 1 --2- 172.21.15.32:0/2208700010 >> v2:172.21.15.121:3300/0 conn(0x7f0d7415c8a0 0x7f0d740ae140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:43.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.370+0000 7f0d6bfff640 1 -- 172.21.15.32:0/2208700010 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d740b10c0 con 0x7f0d740ce7a0 2024-04-24T23:00:43.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.371+0000 7f0d6bfff640 1 --2- 172.21.15.32:0/2208700010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d740ce7a0 0x7f0d7406c390 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f0d74070b50 tx=0x7f0d5c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b2932606ec4913 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:43.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.371+0000 7f0d71d9b640 1 -- 172.21.15.32:0/2208700010 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c037070 con 0x7f0d740ce7a0 2024-04-24T23:00:43.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.371+0000 7f0d71d9b640 1 -- 172.21.15.32:0/2208700010 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d5c032440 con 0x7f0d740ce7a0 2024-04-24T23:00:43.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.371+0000 7f0d71d9b640 1 -- 172.21.15.32:0/2208700010 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c032810 con 0x7f0d740ce7a0 2024-04-24T23:00:43.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.372+0000 7f0d78e2a640 1 -- 172.21.15.32:0/2208700010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d740ce7a0 msgr2=0x7f0d7406c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:43.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.372+0000 7f0d78e2a640 1 --2- 172.21.15.32:0/2208700010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d740ce7a0 0x7f0d7406c390 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f0d74070b50 tx=0x7f0d5c02a140 comp rx=0 tx=0).stop 2024-04-24T23:00:43.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.372+0000 7f0d78e2a640 1 -- 172.21.15.32:0/2208700010 shutdown_connections 2024-04-24T23:00:43.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.372+0000 7f0d78e2a640 1 --2- 172.21.15.32:0/2208700010 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d74074210 0x7f0d740746a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:43.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.372+0000 7f0d78e2a640 1 --2- 172.21.15.32:0/2208700010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d740ce7a0 0x7f0d7406c390 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:43.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.372+0000 7f0d78e2a640 1 --2- 172.21.15.32:0/2208700010 >> v2:172.21.15.121:3300/0 conn(0x7f0d7415c8a0 0x7f0d740ae140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:43.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.372+0000 7f0d78e2a640 1 -- 172.21.15.32:0/2208700010 >> 172.21.15.32:0/2208700010 conn(0x7f0d740b2e60 msgr2=0x7f0d740b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:43.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.372+0000 7f0d78e2a640 1 -- 172.21.15.32:0/2208700010 shutdown_connections 2024-04-24T23:00:43.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.372+0000 7f0d78e2a640 1 -- 172.21.15.32:0/2208700010 wait complete. 2024-04-24T23:00:43.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.373+0000 7f0d78e2a640 1 Processor -- start 2024-04-24T23:00:43.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.373+0000 7f0d78e2a640 1 -- start start 2024-04-24T23:00:43.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.374+0000 7f0d78e2a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d74074210 0x7f0d7415a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:43.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.374+0000 7f0d78e2a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d740ce7a0 0x7f0d7409a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:43.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.374+0000 7f0d78e2a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d7415c8a0 0x7f0d74154c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:43.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.374+0000 7f0d78e2a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7415cc80 con 0x7f0d7415c8a0 2024-04-24T23:00:43.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.374+0000 7f0d78e2a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0d740b1680 con 0x7f0d740ce7a0 2024-04-24T23:00:43.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.374+0000 7f0d78e2a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f0d740cf390 con 0x7f0d74074210 2024-04-24T23:00:43.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.374+0000 7f0d72d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d7415c8a0 0x7f0d74154c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:43.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.374+0000 7f0d72d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d7415c8a0 0x7f0d74154c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53526/0 (socket says 172.21.15.32:53526) 2024-04-24T23:00:43.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.374+0000 7f0d72d9d640 1 -- 172.21.15.32:0/808666263 learned_addr learned my addr 172.21.15.32:0/808666263 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:43.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.374+0000 7f0d7259c640 1 --2- 172.21.15.32:0/808666263 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d74074210 0x7f0d7415a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:43.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.374+0000 7f0d6bfff640 1 --2- 172.21.15.32:0/808666263 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d740ce7a0 0x7f0d7409a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:43.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.374+0000 7f0d72d9d640 1 -- 172.21.15.32:0/808666263 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d74074210 msgr2=0x7f0d7415a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:43.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.374+0000 7f0d72d9d640 1 --2- 172.21.15.32:0/808666263 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d74074210 0x7f0d7415a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:43.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.374+0000 7f0d72d9d640 1 -- 172.21.15.32:0/808666263 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d740ce7a0 msgr2=0x7f0d7409a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:43.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.374+0000 7f0d72d9d640 1 --2- 172.21.15.32:0/808666263 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d740ce7a0 0x7f0d7409a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:43.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.374+0000 7f0d72d9d640 1 -- 172.21.15.32:0/808666263 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d740cf670 con 0x7f0d7415c8a0 2024-04-24T23:00:43.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.374+0000 7f0d72d9d640 1 --2- 172.21.15.32:0/808666263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d7415c8a0 0x7f0d74154c20 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c00cbb0 tx=0x7f0d6c00a4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:43.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.374+0000 7f0d6affd640 1 -- 172.21.15.32:0/808666263 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c015070 con 0x7f0d7415c8a0 2024-04-24T23:00:43.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.375+0000 7f0d6affd640 1 -- 172.21.15.32:0/808666263 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d6c00abd0 con 0x7f0d7415c8a0 2024-04-24T23:00:43.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.375+0000 7f0d78e2a640 1 -- 172.21.15.32:0/808666263 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d740cf950 con 0x7f0d7415c8a0 2024-04-24T23:00:43.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.375+0000 7f0d6affd640 1 -- 172.21.15.32:0/808666263 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c00d040 con 0x7f0d7415c8a0 2024-04-24T23:00:43.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.375+0000 7f0d78e2a640 1 -- 172.21.15.32:0/808666263 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d7409abf0 con 0x7f0d7415c8a0 2024-04-24T23:00:43.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.375+0000 7f0d78e2a640 1 -- 172.21.15.32:0/808666263 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d74068430 con 0x7f0d7415c8a0 2024-04-24T23:00:43.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.376+0000 7f0d6affd640 1 -- 172.21.15.32:0/808666263 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0d6c018070 con 0x7f0d7415c8a0 2024-04-24T23:00:43.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.376+0000 7f0d6affd640 1 --2- 172.21.15.32:0/808666263 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0d5406be30 0x7f0d5406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:43.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.377+0000 7f0d6affd640 1 -- 172.21.15.32:0/808666263 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f0d6c08aae0 con 0x7f0d7415c8a0 2024-04-24T23:00:43.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.377+0000 7f0d7259c640 1 --2- 172.21.15.32:0/808666263 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0d5406be30 0x7f0d5406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:43.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.377+0000 7f0d7259c640 1 --2- 172.21.15.32:0/808666263 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0d5406be30 0x7f0d5406e2f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0d6400a970 tx=0x7f0d64008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:43.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.381+0000 7f0d6affd640 1 -- 172.21.15.32:0/808666263 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0d6c059490 con 0x7f0d7415c8a0 2024-04-24T23:00:43.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.599+0000 7f0d78e2a640 1 -- 172.21.15.32:0/808666263 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0d740c4550 con 0x7f0d7415c8a0 2024-04-24T23:00:43.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.600+0000 7f0d6affd640 1 -- 172.21.15.32:0/808666263 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v20) ==== 53+0+73 (secure 0 0 0) 0x7f0d6c058d90 con 0x7f0d7415c8a0 2024-04-24T23:00:43.601 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T23:00:43.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.602+0000 7f0d78e2a640 1 -- 172.21.15.32:0/808666263 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0d5406be30 msgr2=0x7f0d5406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:43.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.602+0000 7f0d78e2a640 1 --2- 172.21.15.32:0/808666263 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0d5406be30 0x7f0d5406e2f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0d6400a970 tx=0x7f0d64008040 comp rx=0 tx=0).stop 2024-04-24T23:00:43.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.602+0000 7f0d78e2a640 1 -- 172.21.15.32:0/808666263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d7415c8a0 msgr2=0x7f0d74154c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:43.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.602+0000 7f0d78e2a640 1 --2- 172.21.15.32:0/808666263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d7415c8a0 0x7f0d74154c20 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c00cbb0 tx=0x7f0d6c00a4c0 comp rx=0 tx=0).stop 2024-04-24T23:00:43.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.602+0000 7f0d78e2a640 1 -- 172.21.15.32:0/808666263 shutdown_connections 2024-04-24T23:00:43.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.602+0000 7f0d78e2a640 1 --2- 172.21.15.32:0/808666263 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0d5406be30 0x7f0d5406e2f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:43.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.602+0000 7f0d78e2a640 1 --2- 172.21.15.32:0/808666263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d7415c8a0 0x7f0d74154c20 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:43.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.602+0000 7f0d78e2a640 1 --2- 172.21.15.32:0/808666263 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d740ce7a0 0x7f0d7409a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:43.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.602+0000 7f0d78e2a640 1 --2- 172.21.15.32:0/808666263 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d74074210 0x7f0d7415a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:43.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.602+0000 7f0d78e2a640 1 -- 172.21.15.32:0/808666263 >> 172.21.15.32:0/808666263 conn(0x7f0d740b2e60 msgr2=0x7f0d7415b760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:43.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.602+0000 7f0d78e2a640 1 -- 172.21.15.32:0/808666263 shutdown_connections 2024-04-24T23:00:43.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:43.602+0000 7f0d78e2a640 1 -- 172.21.15.32:0/808666263 wait complete. 2024-04-24T23:00:43.614 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-24T23:00:43.614 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T23:00:43.614 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi121.front.sepia.ceph.com 2024-04-24T23:00:43.614 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-24T23:00:43.614 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-24T23:00:43.614 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-24T23:00:43.614 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-24T23:00:43.614 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T23:00:43.614 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-24T23:00:43.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:00:43.615 DEBUG:teuthology.orchestra.run.smithi121:> ip addr 2024-04-24T23:00:43.634 INFO:teuthology.orchestra.run.smithi121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T23:00:43.634 INFO:teuthology.orchestra.run.smithi121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T23:00:43.634 INFO:teuthology.orchestra.run.smithi121.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T23:00:43.634 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:00:43.634 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 ::1/128 scope host 2024-04-24T23:00:43.634 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:00:43.634 INFO:teuthology.orchestra.run.smithi121.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:00:43.634 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:82 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:00:43.634 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f0 2024-04-24T23:00:43.635 INFO:teuthology.orchestra.run.smithi121.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:00:43.635 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:83 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:00:43.635 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f1 2024-04-24T23:00:43.635 INFO:teuthology.orchestra.run.smithi121.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:00:43.635 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a2 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:00:43.635 INFO:teuthology.orchestra.run.smithi121.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T23:00:43.635 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a3 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:00:43.635 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.21.15.121/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-24T23:00:43.635 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft 42687sec preferred_lft 42687sec 2024-04-24T23:00:43.635 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::93dd:8489:4369:e127/64 scope link noprefixroute 2024-04-24T23:00:43.635 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:00:43.635 INFO:teuthology.orchestra.run.smithi121.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T23:00:43.635 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether c6:c0:c7:45:ae:ba brd ff:ff:ff:ff:ff:ff 2024-04-24T23:00:43.636 INFO:teuthology.orchestra.run.smithi121.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-24T23:00:43.636 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:00:43.636 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::80fe:d2ff:fe13:d1/64 scope link 2024-04-24T23:00:43.636 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:00:43.636 INFO:teuthology.orchestra.run.smithi121.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:00:43.636 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether e2:ca:b9:e9:9e:bb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T23:00:43.636 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::e0ca:b9ff:fee9:9ebb/64 scope link 2024-04-24T23:00:43.636 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:00:43.636 INFO:teuthology.orchestra.run.smithi121.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:00:43.636 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether c6:c0:c7:45:ae:ba brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-24T23:00:43.636 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::c4c0:c7ff:fe45:aeba/64 scope link 2024-04-24T23:00:43.636 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:00:43.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:00:43.637 DEBUG:teuthology.orchestra.run.smithi121:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T23:00:43.700 INFO:teuthology.orchestra.run.smithi121.stdout:1 2024-04-24T23:00:43.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:00:43.702 DEBUG:teuthology.orchestra.run.smithi121:> route 2024-04-24T23:00:43.759 INFO:teuthology.orchestra.run.smithi121.stdout:Kernel IP routing table 2024-04-24T23:00:43.759 INFO:teuthology.orchestra.run.smithi121.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T23:00:43.760 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-24T23:00:43.760 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-24T23:00:43.760 INFO:teuthology.orchestra.run.smithi121.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T23:00:43.760 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:00:43.760 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2024-04-24T23:00:43.760 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-24T23:00:43.760 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-24T23:00:43.760 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-24T23:00:43.760 DEBUG:teuthology.orchestra.run.smithi121:> ') 2024-04-24T23:00:43.851 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T23:00:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:00:43.881 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T23:00:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:00:43.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:00:43.888 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2024-04-24T23:00:43.903 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-24T23:00:43.903 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-24T23:00:43.903 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-24T23:00:43.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:00:43.903 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set brx.0 up 2024-04-24T23:00:43.967 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:00:43.967 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-24T23:00:44.022 INFO:teuthology.orchestra.run.smithi121.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-24T23:00:44.022 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:00:44.022 DEBUG:teuthology.orchestra.run.smithi121:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-24T23:00:44.077 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-24T23:00:44.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:00:44.078 DEBUG:teuthology.orchestra.run.smithi121:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:00:44.158 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:00:44.158 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-24T23:00:44.165 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:00:44.166 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-24T23:00:44.166 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.121:3300,172.21.15.121:3301" 2024-04-24T23:00:44.166 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: trying mount with new device syntax: 1@eb1b7a01-6f65-4534-9a12-83011862ba51.cephfs=/ 2024-04-24T23:00:44.166 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.32:3300/172.21.15.121:3300/172.21.15.121:3301" will pass to kernel 2024-04-24T23:00:44.166 INFO:teuthology.orchestra.run.smithi121.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-24T23:00:44.166 INFO:teuthology.orchestra.run.smithi121.stdout: You just mounted a file system that supports labels which does not 2024-04-24T23:00:44.166 INFO:teuthology.orchestra.run.smithi121.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-24T23:00:44.167 INFO:teuthology.orchestra.run.smithi121.stdout: applications will generate AVC messages and not be allowed access to 2024-04-24T23:00:44.167 INFO:teuthology.orchestra.run.smithi121.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-24T23:00:44.168 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-24T23:00:44.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:00:44.168 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-24T23:00:44.232 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:00:44.320 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T23:00:44 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:00:44.322 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T23:00:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:00:44.328 INFO:teuthology.orchestra.run.smithi121.stdout:client5108 2024-04-24T23:00:44.328 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:00:44.328 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:00:44.343 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:00:44.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.450+0000 7f611bb5a640 1 Processor -- start 2024-04-24T23:00:44.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.451+0000 7f611bb5a640 1 -- start start 2024-04-24T23:00:44.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.451+0000 7f611bb5a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f611406b820 0x7f61140d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:44.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.451+0000 7f611bb5a640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f61140d4210 0x7f61140b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:44.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.451+0000 7f611bb5a640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61140d3170 con 0x7f611415e9b0 2024-04-24T23:00:44.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.451+0000 7f611bb5a640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f61140d3450 con 0x7f61140d4210 2024-04-24T23:00:44.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.451+0000 7f611bb5a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f61140d3730 con 0x7f611406b820 2024-04-24T23:00:44.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.452+0000 7f61190f5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f611406b820 0x7f61140d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:44.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.452+0000 7f611a0f7640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f61140d4210 0x7f61140b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:44.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.452+0000 7f61190f5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f611406b820 0x7f61140d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.32:37562/0 (socket says 172.21.15.32:37562) 2024-04-24T23:00:44.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.452+0000 7f61198f6640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f611415e9b0 0x7f611406b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:57118/0 (socket says 172.21.15.32:57118) 2024-04-24T23:00:44.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.452+0000 7f611a0f7640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f61140d4210 0x7f61140b78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.32:59858/0 (socket says 172.21.15.32:59858) 2024-04-24T23:00:44.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.452+0000 7f61190f5640 1 -- 172.21.15.32:0/3427702173 learned_addr learned my addr 172.21.15.32:0/3427702173 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:44.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.452+0000 7f611a0f7640 1 -- 172.21.15.32:0/3427702173 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f611406b820 msgr2=0x7f61140d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:44.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.452+0000 7f611a0f7640 1 --2- 172.21.15.32:0/3427702173 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f611406b820 0x7f61140d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:44.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.452+0000 7f611a0f7640 1 -- 172.21.15.32:0/3427702173 >> v1:172.21.15.121:6789/0 conn(0x7f611415e9b0 legacy=0x7f611406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:44.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.452+0000 7f611a0f7640 1 -- 172.21.15.32:0/3427702173 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61140d3a10 con 0x7f61140d4210 2024-04-24T23:00:44.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.452+0000 7f61188f4640 1 -- 172.21.15.32:0/3427702173 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3791924551 0 0) 0x7f61140d3170 con 0x7f611415e9b0 2024-04-24T23:00:44.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.452+0000 7f61190f5640 1 --2- 172.21.15.32:0/3427702173 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f611406b820 0x7f61140d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:00:44.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.453+0000 7f611a0f7640 1 --2- 172.21.15.32:0/3427702173 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f61140d4210 0x7f61140b78f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f610c009d90 tx=0x7f610c02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27019e969a9f8650 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:44.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.453+0000 7f61188f4640 1 -- 172.21.15.32:0/3427702173 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610c007740 con 0x7f61140d4210 2024-04-24T23:00:44.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.453+0000 7f61188f4640 1 -- 172.21.15.32:0/3427702173 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f610c0078e0 con 0x7f61140d4210 2024-04-24T23:00:44.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.453+0000 7f61188f4640 1 -- 172.21.15.32:0/3427702173 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610c007cb0 con 0x7f61140d4210 2024-04-24T23:00:44.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.454+0000 7f611bb5a640 1 -- 172.21.15.32:0/3427702173 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f61140d4210 msgr2=0x7f61140b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:44.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.454+0000 7f611bb5a640 1 --2- 172.21.15.32:0/3427702173 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f61140d4210 0x7f61140b78f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f610c009d90 tx=0x7f610c02a660 comp rx=0 tx=0).stop 2024-04-24T23:00:44.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.454+0000 7f611bb5a640 1 -- 172.21.15.32:0/3427702173 shutdown_connections 2024-04-24T23:00:44.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.454+0000 7f611bb5a640 1 --2- 172.21.15.32:0/3427702173 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f61140d4210 0x7f61140b78f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:44.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.454+0000 7f611bb5a640 1 --2- 172.21.15.32:0/3427702173 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f611406b820 0x7f61140d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:44.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.454+0000 7f611bb5a640 1 -- 172.21.15.32:0/3427702173 >> 172.21.15.32:0/3427702173 conn(0x7f61140d11d0 msgr2=0x7f61140d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:44.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.454+0000 7f611bb5a640 1 -- 172.21.15.32:0/3427702173 shutdown_connections 2024-04-24T23:00:44.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.454+0000 7f611bb5a640 1 -- 172.21.15.32:0/3427702173 wait complete. 2024-04-24T23:00:44.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.455+0000 7f611bb5a640 1 Processor -- start 2024-04-24T23:00:44.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.455+0000 7f611bb5a640 1 -- start start 2024-04-24T23:00:44.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.456+0000 7f611bb5a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f611406b820 0x7f6114153e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:44.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.456+0000 7f611bb5a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f61140d4210 0x7f6114160fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:44.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.456+0000 7f611bb5a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f611415e9b0 0x7f6114161500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:44.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.456+0000 7f611bb5a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f61140ba0e0 con 0x7f611406b820 2024-04-24T23:00:44.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.456+0000 7f611bb5a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f61140d2400 con 0x7f611415e9b0 2024-04-24T23:00:44.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.456+0000 7f611bb5a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f61140ba260 con 0x7f61140d4210 2024-04-24T23:00:44.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.456+0000 7f61198f6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f611406b820 0x7f6114153e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:44.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.456+0000 7f61198f6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f611406b820 0x7f6114153e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53538/0 (socket says 172.21.15.32:53538) 2024-04-24T23:00:44.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.456+0000 7f61198f6640 1 -- 172.21.15.32:0/1039813078 learned_addr learned my addr 172.21.15.32:0/1039813078 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:44.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.456+0000 7f611a0f7640 1 --2- 172.21.15.32:0/1039813078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f611415e9b0 0x7f6114161500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:44.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.456+0000 7f61190f5640 1 --2- 172.21.15.32:0/1039813078 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f61140d4210 0x7f6114160fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:44.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.456+0000 7f61198f6640 1 -- 172.21.15.32:0/1039813078 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f61140d4210 msgr2=0x7f6114160fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:44.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.456+0000 7f61198f6640 1 --2- 172.21.15.32:0/1039813078 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f61140d4210 0x7f6114160fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:44.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.456+0000 7f61198f6640 1 -- 172.21.15.32:0/1039813078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f611415e9b0 msgr2=0x7f6114161500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:44.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.456+0000 7f61198f6640 1 --2- 172.21.15.32:0/1039813078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f611415e9b0 0x7f6114161500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:44.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.456+0000 7f61198f6640 1 -- 172.21.15.32:0/1039813078 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61141f3ee0 con 0x7f611406b820 2024-04-24T23:00:44.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.456+0000 7f61198f6640 1 --2- 172.21.15.32:0/1039813078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f611406b820 0x7f6114153e00 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f611000c430 tx=0x7f611000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:44.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.457+0000 7f610a7fc640 1 -- 172.21.15.32:0/1039813078 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61100074e0 con 0x7f611406b820 2024-04-24T23:00:44.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.457+0000 7f610a7fc640 1 -- 172.21.15.32:0/1039813078 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61100047e0 con 0x7f611406b820 2024-04-24T23:00:44.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.457+0000 7f611bb5a640 1 -- 172.21.15.32:0/1039813078 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61140c9170 con 0x7f611406b820 2024-04-24T23:00:44.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.457+0000 7f610a7fc640 1 -- 172.21.15.32:0/1039813078 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6110004bd0 con 0x7f611406b820 2024-04-24T23:00:44.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.457+0000 7f611bb5a640 1 -- 172.21.15.32:0/1039813078 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61140c9730 con 0x7f611406b820 2024-04-24T23:00:44.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.458+0000 7f611bb5a640 1 -- 172.21.15.32:0/1039813078 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6114068430 con 0x7f611406b820 2024-04-24T23:00:44.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.458+0000 7f610a7fc640 1 -- 172.21.15.32:0/1039813078 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6110011070 con 0x7f611406b820 2024-04-24T23:00:44.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.459+0000 7f610a7fc640 1 --2- 172.21.15.32:0/1039813078 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f60f806be30 0x7f60f806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:44.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.459+0000 7f610a7fc640 1 -- 172.21.15.32:0/1039813078 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f611008aa80 con 0x7f611406b820 2024-04-24T23:00:44.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.459+0000 7f61190f5640 1 --2- 172.21.15.32:0/1039813078 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f60f806be30 0x7f60f806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:44.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.459+0000 7f61190f5640 1 --2- 172.21.15.32:0/1039813078 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f60f806be30 0x7f60f806e2f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f61140d2980 tx=0x7f6104008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:44.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.463+0000 7f610a7fc640 1 -- 172.21.15.32:0/1039813078 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6110059430 con 0x7f611406b820 2024-04-24T23:00:44.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.682+0000 7f611bb5a640 1 -- 172.21.15.32:0/1039813078 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6114072420 con 0x7f611406b820 2024-04-24T23:00:44.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.682+0000 7f610a7fc640 1 -- 172.21.15.32:0/1039813078 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5187 (secure 0 0 0) 0x7f6110058d30 con 0x7f611406b820 2024-04-24T23:00:44.684 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:00:44.684 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":20,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4994,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/1076304225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1076304225},{"type":"v1","addr":"172.21.15.32:6834","nonce":1076304225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":16},{"gid":5000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1388367206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1388367206},{"type":"v1","addr":"172.21.15.121:6835","nonce":1388367206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":5299,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2065945523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2065945523},{"type":"v1","addr":"172.21.15.121:6837","nonce":2065945523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:00:32.376194+0000","modified":"2024-04-24T23:00:34.381949+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"c","rank":0,"incarnation":18,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/3274703471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":3274703471},{"type":"v1","addr":"172.21.15.32:6835","nonce":3274703471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5631,"qdb_cluster":[5631]},"id":3}]} 2024-04-24T23:00:44.685 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 20 2024-04-24T23:00:44.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.684+0000 7f611bb5a640 1 -- 172.21.15.32:0/1039813078 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f60f806be30 msgr2=0x7f60f806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:44.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.684+0000 7f611bb5a640 1 --2- 172.21.15.32:0/1039813078 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f60f806be30 0x7f60f806e2f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f61140d2980 tx=0x7f6104008040 comp rx=0 tx=0).stop 2024-04-24T23:00:44.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.684+0000 7f611bb5a640 1 -- 172.21.15.32:0/1039813078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f611406b820 msgr2=0x7f6114153e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:44.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.684+0000 7f611bb5a640 1 --2- 172.21.15.32:0/1039813078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f611406b820 0x7f6114153e00 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f611000c430 tx=0x7f611000c8f0 comp rx=0 tx=0).stop 2024-04-24T23:00:44.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.684+0000 7f611bb5a640 1 -- 172.21.15.32:0/1039813078 shutdown_connections 2024-04-24T23:00:44.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.684+0000 7f611bb5a640 1 --2- 172.21.15.32:0/1039813078 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f60f806be30 0x7f60f806e2f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:44.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.684+0000 7f611bb5a640 1 --2- 172.21.15.32:0/1039813078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f611415e9b0 0x7f6114161500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:44.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.684+0000 7f611bb5a640 1 --2- 172.21.15.32:0/1039813078 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f61140d4210 0x7f6114160fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:44.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.684+0000 7f611bb5a640 1 --2- 172.21.15.32:0/1039813078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f611406b820 0x7f6114153e00 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:44.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.684+0000 7f611bb5a640 1 -- 172.21.15.32:0/1039813078 >> 172.21.15.32:0/1039813078 conn(0x7f61140d11d0 msgr2=0x7f61140d15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:44.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.684+0000 7f611bb5a640 1 -- 172.21.15.32:0/1039813078 shutdown_connections 2024-04-24T23:00:44.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:44.684+0000 7f611bb5a640 1 -- 172.21.15.32:0/1039813078 wait complete. 2024-04-24T23:00:44.697 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2024-04-24T23:00:44.816 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","whoami":0,"id":5631,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":11.797359618,"mdsmap_epoch":20,"osdmap_epoch":45,"osdmap_epoch_barrier":45,"uptime":16.703181221000001} 2024-04-24T23:00:44.816 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", "fs_name": "cephfs", "id": 5631, "mdsmap_epoch": 20, "osdmap_epoch": 45, "osdmap_epoch_barrier": 45, "rank_uptime": 11.797359618, "state": "up:active", "uptime": 16.703181221, "want_state": "up:active", "whoami": 0 } 2024-04-24T23:00:44.817 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-04-24T23:00:44.930 INFO:teuthology.orchestra.run.smithi032.stdout:{"osd_mon_report_interval":"5"} 2024-04-24T23:00:44.930 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-04-24T23:00:44.930 INFO:tasks.cephfs.test_full:Writing 531MB should fill this cluster 2024-04-24T23:00:44.930 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:00:44.930 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:00:44.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:00:44.946 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=265 seek=0) 2024-04-24T23:00:45.895 INFO:teuthology.orchestra.run.smithi032.stderr:265+0 records in 2024-04-24T23:00:45.895 INFO:teuthology.orchestra.run.smithi032.stderr:265+0 records out 2024-04-24T23:00:45.895 INFO:teuthology.orchestra.run.smithi032.stderr:277872640 bytes (278 MB, 265 MiB) copied, 0.890082 s, 312 MB/s 2024-04-24T23:00:45.896 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:00:45.896 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:00:45.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:00:45.912 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=292 seek=0) 2024-04-24T23:00:46.949 INFO:teuthology.orchestra.run.smithi032.stderr:292+0 records in 2024-04-24T23:00:46.949 INFO:teuthology.orchestra.run.smithi032.stderr:292+0 records out 2024-04-24T23:00:46.949 INFO:teuthology.orchestra.run.smithi032.stderr:306184192 bytes (306 MB, 292 MiB) copied, 0.979731 s, 313 MB/s 2024-04-24T23:00:46.950 INFO:tasks.cephfs.test_full:Writing file B succeeded (full status will happen soon) 2024-04-24T23:00:46.950 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-24T23:00:47.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.049+0000 7f59f7ddc640 1 Processor -- start 2024-04-24T23:00:47.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.049+0000 7f59f7ddc640 1 -- start start 2024-04-24T23:00:47.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.050+0000 7f59f7ddc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f59f015b480 0x7f59f00b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:47.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.050+0000 7f59f7ddc640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f59f015fd60 0x7f59f006c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:47.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.050+0000 7f59f7ddc640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59f00cef90 con 0x7f59f006cb30 2024-04-24T23:00:47.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.050+0000 7f59f7ddc640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f59f00b4ff0 con 0x7f59f015b480 2024-04-24T23:00:47.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.050+0000 7f59f7ddc640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f59f00b46c0 con 0x7f59f015fd60 2024-04-24T23:00:47.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.050+0000 7f59f5b78640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f59f015b480 0x7f59f00b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:47.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.050+0000 7f59f5b78640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f59f015b480 0x7f59f00b5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53548/0 (socket says 172.21.15.32:53548) 2024-04-24T23:00:47.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.050+0000 7f59f5b78640 1 -- 172.21.15.32:0/407543123 learned_addr learned my addr 172.21.15.32:0/407543123 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:47.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.050+0000 7f59f5377640 1 --2- 172.21.15.32:0/407543123 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f59f015fd60 0x7f59f006c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:47.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.050+0000 7f59f5b78640 1 -- 172.21.15.32:0/407543123 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f59f015fd60 msgr2=0x7f59f006c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:47.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.050+0000 7f59f5b78640 1 --2- 172.21.15.32:0/407543123 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f59f015fd60 0x7f59f006c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:47.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.050+0000 7f59f5b78640 1 -- 172.21.15.32:0/407543123 >> v1:172.21.15.121:6789/0 conn(0x7f59f006cb30 legacy=0x7f59f00ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:47.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.050+0000 7f59f5b78640 1 -- 172.21.15.32:0/407543123 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59f00b49a0 con 0x7f59f015b480 2024-04-24T23:00:47.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.051+0000 7f59f5b78640 1 --2- 172.21.15.32:0/407543123 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59f015b480 0x7f59f00b5c60 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f59ec009870 tx=0x7f59ec02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78137542aaa2f1c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:47.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.051+0000 7f59f4b76640 1 -- 172.21.15.32:0/407543123 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ec037070 con 0x7f59f015b480 2024-04-24T23:00:47.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.051+0000 7f59f4b76640 1 -- 172.21.15.32:0/407543123 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59ec032440 con 0x7f59f015b480 2024-04-24T23:00:47.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.051+0000 7f59f4b76640 1 -- 172.21.15.32:0/407543123 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ec032810 con 0x7f59f015b480 2024-04-24T23:00:47.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.052+0000 7f59f7ddc640 1 -- 172.21.15.32:0/407543123 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59f015b480 msgr2=0x7f59f00b5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:47.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.052+0000 7f59f7ddc640 1 --2- 172.21.15.32:0/407543123 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59f015b480 0x7f59f00b5c60 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f59ec009870 tx=0x7f59ec02a210 comp rx=0 tx=0).stop 2024-04-24T23:00:47.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.052+0000 7f59f7ddc640 1 -- 172.21.15.32:0/407543123 shutdown_connections 2024-04-24T23:00:47.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.052+0000 7f59f7ddc640 1 --2- 172.21.15.32:0/407543123 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f59f015fd60 0x7f59f006c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:47.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.052+0000 7f59f7ddc640 1 --2- 172.21.15.32:0/407543123 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59f015b480 0x7f59f00b5c60 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:47.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.052+0000 7f59f7ddc640 1 -- 172.21.15.32:0/407543123 >> 172.21.15.32:0/407543123 conn(0x7f59f006b110 msgr2=0x7f59f006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:47.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.052+0000 7f59f7ddc640 1 -- 172.21.15.32:0/407543123 shutdown_connections 2024-04-24T23:00:47.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.052+0000 7f59f7ddc640 1 -- 172.21.15.32:0/407543123 wait complete. 2024-04-24T23:00:47.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.053+0000 7f59f7ddc640 1 Processor -- start 2024-04-24T23:00:47.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.053+0000 7f59f7ddc640 1 -- start start 2024-04-24T23:00:47.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.054+0000 7f59f7ddc640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59f006cb30 0x7f59f014c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:47.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.054+0000 7f59f7ddc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59f015b480 0x7f59f01e6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:47.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.054+0000 7f59f7ddc640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f59f015fd60 0x7f59f014cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:47.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.054+0000 7f59f7ddc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f59f00b5480 con 0x7f59f015b480 2024-04-24T23:00:47.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.054+0000 7f59f7ddc640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f59f00b61a0 con 0x7f59f006cb30 2024-04-24T23:00:47.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.054+0000 7f59f7ddc640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f59f015b860 con 0x7f59f015fd60 2024-04-24T23:00:47.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.054+0000 7f59f5377640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59f015b480 0x7f59f01e6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:47.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.054+0000 7f59f5b78640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59f006cb30 0x7f59f014c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:47.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.054+0000 7f59f6379640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f59f015fd60 0x7f59f014cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:47.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.054+0000 7f59f5b78640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59f006cb30 0x7f59f014c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.32:59876/0 (socket says 172.21.15.32:59876) 2024-04-24T23:00:47.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.054+0000 7f59f5b78640 1 -- 172.21.15.32:0/540251942 learned_addr learned my addr 172.21.15.32:0/540251942 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:47.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.054+0000 7f59f5377640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59f015b480 0x7f59f01e6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53558/0 (socket says 172.21.15.32:53558) 2024-04-24T23:00:47.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.054+0000 7f59f5377640 1 -- 172.21.15.32:0/540251942 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f59f015fd60 msgr2=0x7f59f014cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:47.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.054+0000 7f59f5377640 1 --2- 172.21.15.32:0/540251942 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f59f015fd60 0x7f59f014cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:47.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.054+0000 7f59f5377640 1 -- 172.21.15.32:0/540251942 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59f006cb30 msgr2=0x7f59f014c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:47.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.054+0000 7f59f5377640 1 --2- 172.21.15.32:0/540251942 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59f006cb30 0x7f59f014c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:47.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.054+0000 7f59f5377640 1 -- 172.21.15.32:0/540251942 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59f01e4f70 con 0x7f59f015b480 2024-04-24T23:00:47.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.054+0000 7f59f5b78640 1 --2- 172.21.15.32:0/540251942 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59f006cb30 0x7f59f014c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:00:47.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.054+0000 7f59f5377640 1 --2- 172.21.15.32:0/540251942 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59f015b480 0x7f59f01e6010 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f59e000d550 tx=0x7f59e000da10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:47.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.054+0000 7f59e67fc640 1 -- 172.21.15.32:0/540251942 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e00125f0 con 0x7f59f015b480 2024-04-24T23:00:47.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.055+0000 7f59e67fc640 1 -- 172.21.15.32:0/540251942 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59e0019070 con 0x7f59f015b480 2024-04-24T23:00:47.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.055+0000 7f59f7ddc640 1 -- 172.21.15.32:0/540251942 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59f01e5250 con 0x7f59f015b480 2024-04-24T23:00:47.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.055+0000 7f59e67fc640 1 -- 172.21.15.32:0/540251942 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e00129b0 con 0x7f59f015b480 2024-04-24T23:00:47.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.055+0000 7f59f7ddc640 1 -- 172.21.15.32:0/540251942 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59f01e5810 con 0x7f59f015b480 2024-04-24T23:00:47.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.056+0000 7f59f7ddc640 1 -- 172.21.15.32:0/540251942 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59f0068430 con 0x7f59f015b480 2024-04-24T23:00:47.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.056+0000 7f59e67fc640 1 -- 172.21.15.32:0/540251942 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f59e0012b50 con 0x7f59f015b480 2024-04-24T23:00:47.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.056+0000 7f59e67fc640 1 --2- 172.21.15.32:0/540251942 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f59d406be30 0x7f59d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:47.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.057+0000 7f59e67fc640 1 -- 172.21.15.32:0/540251942 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f59e008b060 con 0x7f59f015b480 2024-04-24T23:00:47.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.057+0000 7f59f5b78640 1 --2- 172.21.15.32:0/540251942 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f59d406be30 0x7f59d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:47.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.057+0000 7f59f5b78640 1 --2- 172.21.15.32:0/540251942 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f59d406be30 0x7f59d406e2f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f59ec0090b0 tx=0x7f59ec02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:47.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.061+0000 7f59e67fc640 1 -- 172.21.15.32:0/540251942 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f59e00593c0 con 0x7f59f015b480 2024-04-24T23:00:47.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.244+0000 7f59f7ddc640 1 -- 172.21.15.32:0/540251942 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f59f00b3030 con 0x7f59f015b480 2024-04-24T23:00:47.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.245+0000 7f59e67fc640 1 -- 172.21.15.32:0/540251942 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v45) ==== 81+0+34110 (secure 0 0 0) 0x7f59e005d6c0 con 0x7f59f015b480 2024-04-24T23:00:47.247 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:00:47.247 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:00:47.247 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 45, 2024-04-24T23:00:47.248 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", 2024-04-24T23:00:47.248 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2024-04-24T22:58:46.964851+0000", 2024-04-24T23:00:47.248 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2024-04-24T23:00:34.008750+0000", 2024-04-24T23:00:47.248 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2024-04-24T22:58:52.048787+0000", 2024-04-24T23:00:47.248 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2024-04-24T22:58:49.045609+0000", 2024-04-24T23:00:47.248 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-24T23:00:47.248 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2024-04-24T23:00:47.248 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2024-04-24T23:00:47.248 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2024-04-24T23:00:47.248 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2024-04-24T23:00:47.248 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2024-04-24T23:00:47.248 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2024-04-24T23:00:47.248 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:00:47.248 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 4, 2024-04-24T23:00:47.249 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.69999998807907104, 2024-04-24T23:00:47.249 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-24T23:00:47.249 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-24T23:00:47.249 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2024-04-24T23:00:47.249 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 7, 2024-04-24T23:00:47.249 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2024-04-24T23:00:47.249 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "luminous", 2024-04-24T23:00:47.249 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "jewel", 2024-04-24T23:00:47.249 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "squid", 2024-04-24T23:00:47.249 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_crimson": false, 2024-04-24T23:00:47.249 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2024-04-24T23:00:47.249 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.249 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2024-04-24T23:00:47.249 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": ".mgr", 2024-04-24T23:00:47.250 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T22:58:53.171408+0000", 2024-04-24T23:00:47.250 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:00:47.250 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:00:47.250 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:00:47.250 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:00:47.250 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:00:47.250 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:00:47.250 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:00:47.250 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:00:47.250 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:00:47.250 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:00:47.250 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:00:47.250 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:00:47.250 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2024-04-24T23:00:47.251 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2024-04-24T23:00:47.251 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2024-04-24T23:00:47.251 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2024-04-24T23:00:47.251 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2024-04-24T23:00:47.251 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:00:47.251 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:00:47.251 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:00:47.251 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:00:47.251 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:00:47.251 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:00:47.251 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:00:47.251 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.251 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "17", 2024-04-24T23:00:47.252 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:00:47.252 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:00:47.252 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:00:47.252 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:00:47.252 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:00:47.252 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:00:47.252 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:00:47.252 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:00:47.252 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:00:47.252 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:00:47.252 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:00:47.252 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:00:47.252 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:00:47.253 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:00:47.253 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:00:47.253 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:00:47.253 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:00:47.253 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:00:47.253 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:00:47.253 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:00:47.253 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:00:47.253 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:00:47.253 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:00:47.253 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:00:47.253 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:00:47.253 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:00:47.253 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.254 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:00:47.254 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:00:47.254 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:00:47.254 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:00:47.254 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:00:47.254 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:00:47.254 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:00:47.254 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:00:47.254 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:00:47.254 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:00:47.254 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:00:47.254 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:00:47.254 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_max": 32, 2024-04-24T23:00:47.254 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2024-04-24T23:00:47.255 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.255 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:00:47.255 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": {} 2024-04-24T23:00:47.255 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.255 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:00:47.255 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:00:47.255 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 8, 2024-04-24T23:00:47.255 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 8, 2024-04-24T23:00:47.255 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.25, 2024-04-24T23:00:47.255 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-04-24T23:00:47.255 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-04-24T23:00:47.255 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:00:47.255 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:00:47.255 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:00:47.256 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:47.256 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.256 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.256 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 6, 2024-04-24T23:00:47.256 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2024-04-24T23:00:47.256 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:00:30.665592+0000", 2024-04-24T23:00:47.256 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:00:47.256 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:00:47.256 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:00:47.256 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:00:47.256 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:00:47.256 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:00:47.256 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:00:47.256 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:00:47.257 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:00:47.257 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:00:47.257 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:00:47.257 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:00:47.257 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 32, 2024-04-24T23:00:47.257 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 32, 2024-04-24T23:00:47.257 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 32, 2024-04-24T23:00:47.257 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 32, 2024-04-24T23:00:47.257 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 32, 2024-04-24T23:00:47.257 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:00:47.257 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:00:47.257 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:00:47.257 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:00:47.257 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:00:47.258 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:00:47.258 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:00:47.258 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.258 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "44", 2024-04-24T23:00:47.258 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:00:47.258 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:00:47.258 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:00:47.258 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:00:47.258 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:00:47.258 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:00:47.258 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:00:47.258 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:00:47.258 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:00:47.258 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:00:47.259 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:00:47.259 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:00:47.259 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:00:47.259 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:00:47.259 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:00:47.259 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:00:47.259 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:00:47.259 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:00:47.259 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:00:47.259 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:00:47.259 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:00:47.259 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:00:47.259 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:00:47.259 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:00:47.259 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:00:47.260 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:00:47.260 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.260 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:00:47.260 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:00:47.260 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:00:47.260 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:00:47.260 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:00:47.260 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:00:47.260 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:00:47.260 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:00:47.260 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:00:47.260 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:00:47.260 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:00:47.260 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:00:47.260 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2024-04-24T23:00:47.261 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2024-04-24T23:00:47.261 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2024-04-24T23:00:47.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.261 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:00:47.261 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:00:47.261 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2024-04-24T23:00:47.261 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:47.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.261 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:00:47.261 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:00:47.261 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.5, 2024-04-24T23:00:47.261 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.5, 2024-04-24T23:00:47.261 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:00:47.261 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.5, 2024-04-24T23:00:47.262 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.5, 2024-04-24T23:00:47.262 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:00:47.262 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:00:47.262 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:00:47.262 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:47.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.262 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 7, 2024-04-24T23:00:47.262 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2024-04-24T23:00:47.262 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:00:31.357756+0000", 2024-04-24T23:00:47.262 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:00:47.262 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:00:47.262 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:00:47.262 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:00:47.262 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:00:47.263 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:00:47.263 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:00:47.263 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:00:47.263 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:00:47.263 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:00:47.263 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:00:47.263 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:00:47.263 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 64, 2024-04-24T23:00:47.263 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 64, 2024-04-24T23:00:47.263 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 64, 2024-04-24T23:00:47.263 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 64, 2024-04-24T23:00:47.263 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 64, 2024-04-24T23:00:47.263 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:00:47.263 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:00:47.263 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:00:47.264 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:00:47.264 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:00:47.264 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:00:47.264 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:00:47.264 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.264 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "45", 2024-04-24T23:00:47.264 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:00:47.264 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:00:47.264 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:00:47.264 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:00:47.264 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:00:47.264 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:00:47.264 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:00:47.264 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:00:47.265 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:00:47.265 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:00:47.265 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:00:47.265 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:00:47.265 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:00:47.265 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:00:47.265 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:00:47.265 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:00:47.265 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:00:47.265 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:00:47.265 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:00:47.265 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:00:47.265 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:00:47.265 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:00:47.266 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:00:47.266 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:00:47.266 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:00:47.266 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:00:47.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.266 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:00:47.266 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:00:47.266 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:00:47.267 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:00:47.267 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:00:47.267 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:00:47.267 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:00:47.267 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:00:47.267 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:00:47.267 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:00:47.267 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:00:47.267 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:00:47.267 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 64, 2024-04-24T23:00:47.267 INFO:teuthology.orchestra.run.smithi032.stdout: "target_size_ratio": 0.89999997615814209 2024-04-24T23:00:47.267 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.267 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:00:47.267 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:00:47.268 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2024-04-24T23:00:47.268 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:47.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.268 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:00:47.268 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:00:47.268 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.75, 2024-04-24T23:00:47.268 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.75, 2024-04-24T23:00:47.268 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:00:47.268 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.75, 2024-04-24T23:00:47.268 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.75, 2024-04-24T23:00:47.268 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:00:47.268 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:00:47.268 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:00:47.268 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:47.268 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:47.269 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:00:47.269 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-04-24T23:00:47.269 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.269 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:00:47.269 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:00:47.269 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:00:47.269 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:00:47.269 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:00:47.269 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:00:47.269 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:00:47.269 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:00:47.269 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:00:47.269 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 43, 2024-04-24T23:00:47.269 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:00:47.269 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:00:47.270 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:00:47.270 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:47.270 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.270 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:47.270 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2024-04-24T23:00:47.270 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:00:47.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.270 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.270 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:47.270 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2024-04-24T23:00:47.270 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:00:47.270 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:47.270 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:47.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.270 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:00:47.271 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:47.271 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.271 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:47.271 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2024-04-24T23:00:47.271 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:00:47.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.271 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.271 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:47.271 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2024-04-24T23:00:47.271 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:00:47.271 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:47.271 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:47.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.271 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:00:47.271 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:47.272 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.272 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:47.272 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2024-04-24T23:00:47.272 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:00:47.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.272 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.272 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:47.272 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2024-04-24T23:00:47.272 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:00:47.272 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:47.272 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:47.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.272 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:00:47.272 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:47.272 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.273 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:47.273 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2024-04-24T23:00:47.273 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:00:47.273 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.273 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.273 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:47.273 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2024-04-24T23:00:47.273 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:00:47.273 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:47.273 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:47.273 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.273 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/1479395426", 2024-04-24T23:00:47.273 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/1479395426", 2024-04-24T23:00:47.273 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/1479395426", 2024-04-24T23:00:47.273 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/1479395426", 2024-04-24T23:00:47.274 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:00:47.274 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:00:47.274 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:00:47.274 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:47.274 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.274 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.274 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:00:47.274 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:00:47.274 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:00:47.274 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:00:47.274 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:00:47.274 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:00:47.274 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:00:47.274 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:00:47.275 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:00:47.275 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 43, 2024-04-24T23:00:47.275 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:00:47.275 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:00:47.275 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:00:47.275 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:47.275 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.275 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:47.275 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2024-04-24T23:00:47.275 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:00:47.275 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.275 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.275 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:47.275 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2024-04-24T23:00:47.275 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:00:47.276 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:47.276 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:47.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.276 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:00:47.276 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:47.276 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.276 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:47.276 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2024-04-24T23:00:47.276 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:00:47.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.276 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.276 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:47.276 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2024-04-24T23:00:47.276 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:00:47.276 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:47.276 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:47.277 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.277 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:00:47.277 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:47.277 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.277 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:47.277 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2024-04-24T23:00:47.277 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:00:47.277 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.277 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.277 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:47.277 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2024-04-24T23:00:47.277 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:00:47.277 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:47.277 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:47.277 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.278 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:00:47.278 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:47.278 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.278 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:47.278 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2024-04-24T23:00:47.278 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:00:47.278 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.278 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.278 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:47.278 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2024-04-24T23:00:47.278 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:00:47.278 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:47.278 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:47.278 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.279 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/1340633718", 2024-04-24T23:00:47.279 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/1340633718", 2024-04-24T23:00:47.279 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/1340633718", 2024-04-24T23:00:47.279 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/1340633718", 2024-04-24T23:00:47.279 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:00:47.279 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:00:47.279 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:00:47.279 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:47.279 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.279 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.279 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:00:47.279 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:00:47.279 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:00:47.279 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:00:47.279 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:00:47.280 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:00:47.280 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:00:47.280 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:00:47.280 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:00:47.280 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 43, 2024-04-24T23:00:47.280 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:00:47.280 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:00:47.280 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:00:47.280 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:47.280 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.280 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:47.280 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2024-04-24T23:00:47.280 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:00:47.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.280 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.281 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:47.281 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2024-04-24T23:00:47.281 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:00:47.281 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:47.281 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:47.281 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.281 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:00:47.281 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:47.281 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.281 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:47.281 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2024-04-24T23:00:47.281 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:00:47.281 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.281 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.281 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:47.282 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2024-04-24T23:00:47.282 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:00:47.282 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:47.282 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:47.282 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.282 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:00:47.282 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:47.282 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.282 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:47.282 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2024-04-24T23:00:47.282 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:00:47.282 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.282 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.282 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:47.283 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2024-04-24T23:00:47.283 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:00:47.283 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:47.283 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:47.283 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.283 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:00:47.283 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:47.283 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.283 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:47.283 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2024-04-24T23:00:47.283 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:00:47.283 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.283 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.283 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:47.283 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2024-04-24T23:00:47.284 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:00:47.284 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:47.284 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:47.284 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.284 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/3405899334", 2024-04-24T23:00:47.284 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6820/3405899334", 2024-04-24T23:00:47.284 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6828/3405899334", 2024-04-24T23:00:47.284 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6824/3405899334", 2024-04-24T23:00:47.284 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:00:47.284 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:00:47.284 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:00:47.284 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:47.284 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.284 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.284 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:00:47.285 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:00:47.285 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:00:47.285 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:00:47.285 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:00:47.285 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:00:47.285 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:00:47.285 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:00:47.285 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:00:47.285 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 43, 2024-04-24T23:00:47.285 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:00:47.285 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:00:47.285 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:00:47.285 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:47.285 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.285 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:47.286 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2024-04-24T23:00:47.286 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:00:47.286 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.286 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.286 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:47.286 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2024-04-24T23:00:47.286 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:00:47.286 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:47.286 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:47.286 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.286 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:00:47.286 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:47.286 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.286 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:47.287 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2024-04-24T23:00:47.287 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:00:47.287 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.287 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.287 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:47.287 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2024-04-24T23:00:47.287 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:00:47.287 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:47.287 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:47.287 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.287 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:00:47.287 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:47.287 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.287 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:47.287 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2024-04-24T23:00:47.288 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:00:47.288 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.288 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.288 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:47.288 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2024-04-24T23:00:47.288 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:00:47.288 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:47.288 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:47.288 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.288 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:00:47.288 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:47.288 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.288 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:47.288 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2024-04-24T23:00:47.288 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:00:47.289 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.289 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.289 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:47.289 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2024-04-24T23:00:47.289 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:00:47.289 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:47.289 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:47.289 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.289 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6821/3381057190", 2024-04-24T23:00:47.289 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6825/3381057190", 2024-04-24T23:00:47.289 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/3381057190", 2024-04-24T23:00:47.289 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/3381057190", 2024-04-24T23:00:47.289 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:00:47.289 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:00:47.289 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:00:47.290 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:47.290 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.290 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.290 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:00:47.290 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:00:47.290 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:00:47.290 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:00:47.290 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:00:47.290 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:00:47.290 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:00:47.290 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:00:47.290 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:00:47.290 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 43, 2024-04-24T23:00:47.290 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:00:47.291 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:00:47.291 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:00:47.291 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:47.291 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.291 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:47.291 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6800", 2024-04-24T23:00:47.291 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:00:47.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.291 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.291 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:47.291 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6801", 2024-04-24T23:00:47.291 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:00:47.291 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:47.291 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:47.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.292 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:00:47.292 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:47.292 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.292 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:47.292 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6802", 2024-04-24T23:00:47.292 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:00:47.292 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.292 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.292 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:47.292 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6803", 2024-04-24T23:00:47.292 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:00:47.292 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:47.292 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:47.292 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.292 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:00:47.293 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:47.293 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.293 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:47.293 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6806", 2024-04-24T23:00:47.293 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:00:47.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.293 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.293 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:47.293 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6807", 2024-04-24T23:00:47.293 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:00:47.293 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:47.293 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:47.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.293 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:00:47.293 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:47.293 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.294 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:47.294 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6804", 2024-04-24T23:00:47.294 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:00:47.294 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.294 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.294 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:47.294 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6805", 2024-04-24T23:00:47.294 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:00:47.294 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:47.294 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:47.294 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.294 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6801/4010553979", 2024-04-24T23:00:47.294 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6803/4010553979", 2024-04-24T23:00:47.294 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6807/4010553979", 2024-04-24T23:00:47.295 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6805/4010553979", 2024-04-24T23:00:47.295 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:00:47.295 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:00:47.295 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:00:47.295 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:47.295 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.295 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.295 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:00:47.295 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:00:47.295 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:00:47.295 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:00:47.295 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:00:47.295 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:00:47.295 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:00:47.295 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:00:47.295 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:00:47.296 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 43, 2024-04-24T23:00:47.296 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:00:47.296 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:00:47.296 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:00:47.296 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:47.296 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.296 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:47.296 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6808", 2024-04-24T23:00:47.296 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:00:47.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.296 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.296 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:47.296 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6809", 2024-04-24T23:00:47.296 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:00:47.296 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:47.297 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:47.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.297 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:00:47.297 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:47.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.297 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:47.297 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6810", 2024-04-24T23:00:47.297 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:00:47.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.297 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:47.297 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6811", 2024-04-24T23:00:47.297 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:00:47.297 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:47.297 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:47.298 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.298 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:00:47.298 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:47.298 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.298 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:47.298 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6814", 2024-04-24T23:00:47.298 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:00:47.298 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.298 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.298 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:47.298 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6815", 2024-04-24T23:00:47.298 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:00:47.298 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:47.298 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:47.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.299 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:00:47.299 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:47.299 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.299 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:47.299 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6812", 2024-04-24T23:00:47.299 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:00:47.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.299 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.299 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:47.299 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6813", 2024-04-24T23:00:47.299 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:00:47.299 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:47.299 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:47.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.300 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6809/3794084091", 2024-04-24T23:00:47.300 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6811/3794084091", 2024-04-24T23:00:47.300 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6815/3794084091", 2024-04-24T23:00:47.300 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6813/3794084091", 2024-04-24T23:00:47.300 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:00:47.300 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:00:47.300 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:00:47.300 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:47.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.300 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.300 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:00:47.300 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:00:47.300 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:00:47.300 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:00:47.300 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:00:47.301 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:00:47.301 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:00:47.301 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:00:47.301 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:00:47.301 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 43, 2024-04-24T23:00:47.301 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:00:47.301 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:00:47.301 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:00:47.301 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:47.301 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.301 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:47.301 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6823", 2024-04-24T23:00:47.301 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:00:47.301 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.302 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.302 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:47.302 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6825", 2024-04-24T23:00:47.302 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:00:47.302 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:47.302 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:47.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.302 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:00:47.302 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:47.302 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.302 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:47.302 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6826", 2024-04-24T23:00:47.302 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:00:47.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.303 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.303 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:47.303 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6827", 2024-04-24T23:00:47.303 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:00:47.303 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:47.303 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:47.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.303 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:00:47.303 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:47.303 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.303 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:47.303 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6830", 2024-04-24T23:00:47.303 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:00:47.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.303 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.303 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:47.304 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6831", 2024-04-24T23:00:47.304 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:00:47.304 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:47.304 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:47.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.304 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:00:47.304 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:47.304 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.304 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:47.304 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6828", 2024-04-24T23:00:47.304 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:00:47.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.304 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.304 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:47.305 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6829", 2024-04-24T23:00:47.305 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:00:47.305 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:47.305 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:47.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.305 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6825/1348640258", 2024-04-24T23:00:47.305 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6827/1348640258", 2024-04-24T23:00:47.305 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6831/1348640258", 2024-04-24T23:00:47.305 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6829/1348640258", 2024-04-24T23:00:47.305 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:00:47.305 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:00:47.305 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:00:47.305 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:47.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.305 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.306 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:00:47.306 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:00:47.306 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:00:47.306 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:00:47.306 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:00:47.306 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:00:47.306 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:00:47.306 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:00:47.306 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:00:47.306 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 43, 2024-04-24T23:00:47.306 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:00:47.306 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:00:47.306 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:00:47.306 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:47.306 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.307 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:47.307 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6816", 2024-04-24T23:00:47.307 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:00:47.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.307 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.307 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:47.307 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6817", 2024-04-24T23:00:47.307 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:00:47.307 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:47.307 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:47.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.307 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:00:47.307 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:47.307 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.307 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:47.308 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6818", 2024-04-24T23:00:47.308 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:00:47.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.308 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.308 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:47.308 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6819", 2024-04-24T23:00:47.308 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:00:47.308 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:47.308 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:47.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.308 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:00:47.308 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:47.308 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.308 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:47.308 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6822", 2024-04-24T23:00:47.309 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:00:47.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.309 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.309 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:47.309 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6824", 2024-04-24T23:00:47.309 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:00:47.309 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:47.309 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:47.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.309 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:00:47.309 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:47.309 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.309 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:47.309 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6820", 2024-04-24T23:00:47.309 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:00:47.310 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.310 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.310 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:47.310 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6821", 2024-04-24T23:00:47.310 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:00:47.310 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:47.310 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:47.310 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.310 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6817/3250005122", 2024-04-24T23:00:47.310 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6819/3250005122", 2024-04-24T23:00:47.310 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6824/3250005122", 2024-04-24T23:00:47.310 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6821/3250005122", 2024-04-24T23:00:47.310 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:00:47.310 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:00:47.310 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:00:47.311 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:47.311 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:47.311 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:00:47.311 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2024-04-24T23:00:47.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.311 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:00:47.311 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:00:47.311 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:00:47.311 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:00:47.311 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:00:47.311 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:00:47.311 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.156184+0000", 2024-04-24T23:00:47.311 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:00:47.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.311 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:00:47.312 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:00:47.312 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:00:47.312 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:00:47.312 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:00:47.312 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:00:47.312 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240774+0000", 2024-04-24T23:00:47.312 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:00:47.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.312 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.312 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:00:47.312 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:00:47.312 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:00:47.312 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:00:47.312 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:00:47.312 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:00:47.313 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240949+0000", 2024-04-24T23:00:47.313 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:00:47.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.313 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.313 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:00:47.313 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:00:47.313 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:00:47.313 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:00:47.313 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:00:47.313 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:00:47.313 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.171672+0000", 2024-04-24T23:00:47.313 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:00:47.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.313 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.313 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:00:47.314 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:00:47.314 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:00:47.314 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:00:47.314 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:00:47.314 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:00:47.314 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.220379+0000", 2024-04-24T23:00:47.314 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:00:47.314 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.314 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.314 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:00:47.314 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:00:47.314 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:00:47.314 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:00:47.314 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:00:47.314 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:00:47.315 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.204963+0000", 2024-04-24T23:00:47.315 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:00:47.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.315 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.315 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:00:47.315 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:00:47.315 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:00:47.315 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:00:47.315 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:00:47.315 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:00:47.315 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.239748+0000", 2024-04-24T23:00:47.315 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:00:47.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.315 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:47.316 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:00:47.316 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:00:47.316 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:00:47.316 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:00:47.316 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:00:47.316 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:00:47.316 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.154104+0000", 2024-04-24T23:00:47.316 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:00:47.316 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:47.316 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:00:47.316 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2024-04-24T23:00:47.316 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2024-04-24T23:00:47.316 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_primaries": [], 2024-04-24T23:00:47.316 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2024-04-24T23:00:47.316 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2024-04-24T23:00:47.317 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklist": {}, 2024-04-24T23:00:47.317 INFO:teuthology.orchestra.run.smithi032.stdout: "range_blocklist": {}, 2024-04-24T23:00:47.317 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2024-04-24T23:00:47.317 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2024-04-24T23:00:47.317 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2024-04-24T23:00:47.317 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2024-04-24T23:00:47.317 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2024-04-24T23:00:47.317 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2024-04-24T23:00:47.317 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2024-04-24T23:00:47.317 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:47.317 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:47.317 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2024-04-24T23:00:47.317 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2024-04-24T23:00:47.317 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2024-04-24T23:00:47.317 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2024-04-24T23:00:47.317 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {}, 2024-04-24T23:00:47.318 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode": { 2024-04-24T23:00:47.318 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_enabled": false, 2024-04-24T23:00:47.318 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_bucket_count": 0, 2024-04-24T23:00:47.318 INFO:teuthology.orchestra.run.smithi032.stdout: "degraded_stretch_mode": 0, 2024-04-24T23:00:47.318 INFO:teuthology.orchestra.run.smithi032.stdout: "recovering_stretch_mode": 0, 2024-04-24T23:00:47.318 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_bucket": 0 2024-04-24T23:00:47.318 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:47.318 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:00:47.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.247+0000 7f59f7ddc640 1 -- 172.21.15.32:0/540251942 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f59d406be30 msgr2=0x7f59d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:47.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.247+0000 7f59f7ddc640 1 --2- 172.21.15.32:0/540251942 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f59d406be30 0x7f59d406e2f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f59ec0090b0 tx=0x7f59ec02e000 comp rx=0 tx=0).stop 2024-04-24T23:00:47.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.247+0000 7f59f7ddc640 1 -- 172.21.15.32:0/540251942 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59f015b480 msgr2=0x7f59f01e6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:47.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.247+0000 7f59f7ddc640 1 --2- 172.21.15.32:0/540251942 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59f015b480 0x7f59f01e6010 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f59e000d550 tx=0x7f59e000da10 comp rx=0 tx=0).stop 2024-04-24T23:00:47.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.247+0000 7f59f7ddc640 1 -- 172.21.15.32:0/540251942 shutdown_connections 2024-04-24T23:00:47.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.247+0000 7f59f7ddc640 1 --2- 172.21.15.32:0/540251942 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f59d406be30 0x7f59d406e2f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:47.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.247+0000 7f59f7ddc640 1 --2- 172.21.15.32:0/540251942 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59f015b480 0x7f59f01e6010 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:47.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.247+0000 7f59f7ddc640 1 --2- 172.21.15.32:0/540251942 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f59f015fd60 0x7f59f014cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:47.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.247+0000 7f59f7ddc640 1 --2- 172.21.15.32:0/540251942 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59f006cb30 0x7f59f014c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:47.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.247+0000 7f59f7ddc640 1 -- 172.21.15.32:0/540251942 >> 172.21.15.32:0/540251942 conn(0x7f59f006b110 msgr2=0x7f59f00cdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:47.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.247+0000 7f59f7ddc640 1 -- 172.21.15.32:0/540251942 shutdown_connections 2024-04-24T23:00:47.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:47.247+0000 7f59f7ddc640 1 -- 172.21.15.32:0/540251942 wait complete. 2024-04-24T23:00:47.321 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-04-24T23:00:48.224 INFO:tasks.ceph.mon.a.smithi032.stderr:2024-04-24T23:00:48.222+0000 7f3b831cb640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 full osd(s) (OSD_FULL) 2024-04-24T23:00:52.323 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-24T23:00:52.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.427+0000 7fa4796da640 1 Processor -- start 2024-04-24T23:00:52.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.427+0000 7fa4796da640 1 -- start start 2024-04-24T23:00:52.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.428+0000 7fa4796da640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa47406cd00 0x7fa4740ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:52.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.428+0000 7fa4796da640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4740b4520 con 0x7fa47415cb00 2024-04-24T23:00:52.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.428+0000 7fa4796da640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4740b4800 con 0x7fa47415b380 2024-04-24T23:00:52.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.428+0000 7fa4796da640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa4740b4ae0 con 0x7fa47406cd00 2024-04-24T23:00:52.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.428+0000 7fa4737fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa47415cb00 0x7fa4740ce970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57144/0 (socket says 172.21.15.32:57144) 2024-04-24T23:00:52.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.428+0000 7fa4737fe640 1 -- 172.21.15.32:0/3765219931 learned_addr learned my addr 172.21.15.32:0/3765219931 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:00:52.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.429+0000 7fa4727fc640 1 --2- 172.21.15.32:0/3765219931 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa47406cd00 0x7fa4740ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:52.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.429+0000 7fa471ffb640 1 -- 172.21.15.32:0/3765219931 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 754844433 0 0) 0x7fa4740b4520 con 0x7fa47415cb00 2024-04-24T23:00:52.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.429+0000 7fa471ffb640 1 -- 172.21.15.32:0/3765219931 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa45c003520 con 0x7fa47415cb00 2024-04-24T23:00:52.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.429+0000 7fa471ffb640 1 -- 172.21.15.32:0/3765219931 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2989545471 0 0) 0x7fa4740b4800 con 0x7fa47415b380 2024-04-24T23:00:52.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.429+0000 7fa471ffb640 1 -- 172.21.15.32:0/3765219931 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa4740b4520 con 0x7fa47415b380 2024-04-24T23:00:52.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.429+0000 7fa4727fc640 1 -- 172.21.15.32:0/3765219931 >> v1:172.21.15.121:6789/0 conn(0x7fa47415b380 legacy=0x7fa47406c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:52.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.429+0000 7fa4727fc640 1 -- 172.21.15.32:0/3765219931 >> v1:172.21.15.32:6789/0 conn(0x7fa47415cb00 legacy=0x7fa4740ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:52.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.429+0000 7fa4727fc640 1 -- 172.21.15.32:0/3765219931 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4740b4dc0 con 0x7fa47406cd00 2024-04-24T23:00:52.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.430+0000 7fa4727fc640 1 --2- 172.21.15.32:0/3765219931 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa47406cd00 0x7fa4740ce430 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa4680097d0 tx=0x7fa46802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1a6bb97ac3383b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:52.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.430+0000 7fa471ffb640 1 -- 172.21.15.32:0/3765219931 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa468037070 con 0x7fa47406cd00 2024-04-24T23:00:52.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.430+0000 7fa471ffb640 1 -- 172.21.15.32:0/3765219931 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa468032440 con 0x7fa47406cd00 2024-04-24T23:00:52.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.430+0000 7fa471ffb640 1 -- 172.21.15.32:0/3765219931 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa468032810 con 0x7fa47406cd00 2024-04-24T23:00:52.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.430+0000 7fa4796da640 1 -- 172.21.15.32:0/3765219931 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa47406cd00 msgr2=0x7fa4740ce430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:52.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.430+0000 7fa4796da640 1 --2- 172.21.15.32:0/3765219931 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa47406cd00 0x7fa4740ce430 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa4680097d0 tx=0x7fa46802a180 comp rx=0 tx=0).stop 2024-04-24T23:00:52.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.430+0000 7fa4796da640 1 -- 172.21.15.32:0/3765219931 shutdown_connections 2024-04-24T23:00:52.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.430+0000 7fa4796da640 1 --2- 172.21.15.32:0/3765219931 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa47406cd00 0x7fa4740ce430 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:52.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.430+0000 7fa4796da640 1 -- 172.21.15.32:0/3765219931 >> 172.21.15.32:0/3765219931 conn(0x7fa47406b110 msgr2=0x7fa47406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:52.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.431+0000 7fa4796da640 1 -- 172.21.15.32:0/3765219931 shutdown_connections 2024-04-24T23:00:52.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.431+0000 7fa4796da640 1 -- 172.21.15.32:0/3765219931 wait complete. 2024-04-24T23:00:52.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.432+0000 7fa4796da640 1 Processor -- start 2024-04-24T23:00:52.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.432+0000 7fa4796da640 1 -- start start 2024-04-24T23:00:52.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.433+0000 7fa4796da640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa47406cd00 0x7fa474158970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:52.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.433+0000 7fa4796da640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa47415b380 0x7fa4740b19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:52.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.433+0000 7fa4796da640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa47415cb00 0x7fa474072170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:52.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.433+0000 7fa4796da640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa4740b55c0 con 0x7fa47406cd00 2024-04-24T23:00:52.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.433+0000 7fa4796da640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa47415b760 con 0x7fa47415cb00 2024-04-24T23:00:52.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.433+0000 7fa4796da640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa4740b5740 con 0x7fa47415b380 2024-04-24T23:00:52.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.433+0000 7fa472ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa47406cd00 0x7fa474158970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:52.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.433+0000 7fa472ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa47406cd00 0x7fa474158970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48682/0 (socket says 172.21.15.32:48682) 2024-04-24T23:00:52.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.433+0000 7fa472ffd640 1 -- 172.21.15.32:0/1785846999 learned_addr learned my addr 172.21.15.32:0/1785846999 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:52.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.433+0000 7fa4737fe640 1 --2- 172.21.15.32:0/1785846999 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa47415cb00 0x7fa474072170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:52.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.433+0000 7fa4727fc640 1 --2- 172.21.15.32:0/1785846999 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa47415b380 0x7fa4740b19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:52.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.433+0000 7fa472ffd640 1 -- 172.21.15.32:0/1785846999 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa47415b380 msgr2=0x7fa4740b19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:52.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.433+0000 7fa472ffd640 1 --2- 172.21.15.32:0/1785846999 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa47415b380 0x7fa4740b19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:52.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.433+0000 7fa472ffd640 1 -- 172.21.15.32:0/1785846999 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa47415cb00 msgr2=0x7fa474072170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:52.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.433+0000 7fa472ffd640 1 --2- 172.21.15.32:0/1785846999 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa47415cb00 0x7fa474072170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:52.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.433+0000 7fa472ffd640 1 -- 172.21.15.32:0/1785846999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4740708d0 con 0x7fa47406cd00 2024-04-24T23:00:52.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.433+0000 7fa472ffd640 1 --2- 172.21.15.32:0/1785846999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa47406cd00 0x7fa474158970 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fa46400c500 tx=0x7fa46400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:52.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.433+0000 7fa45b7fe640 1 -- 172.21.15.32:0/1785846999 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa464017070 con 0x7fa47406cd00 2024-04-24T23:00:52.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.433+0000 7fa45b7fe640 1 -- 172.21.15.32:0/1785846999 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa464005310 con 0x7fa47406cd00 2024-04-24T23:00:52.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.433+0000 7fa4796da640 1 -- 172.21.15.32:0/1785846999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa474152880 con 0x7fa47406cd00 2024-04-24T23:00:52.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.433+0000 7fa45b7fe640 1 -- 172.21.15.32:0/1785846999 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa464003830 con 0x7fa47406cd00 2024-04-24T23:00:52.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.434+0000 7fa4796da640 1 -- 172.21.15.32:0/1785846999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa474070bb0 con 0x7fa47406cd00 2024-04-24T23:00:52.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.435+0000 7fa4796da640 1 -- 172.21.15.32:0/1785846999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa474068430 con 0x7fa47406cd00 2024-04-24T23:00:52.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.435+0000 7fa45b7fe640 1 -- 172.21.15.32:0/1785846999 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa46401a070 con 0x7fa47406cd00 2024-04-24T23:00:52.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.436+0000 7fa45b7fe640 1 --2- 172.21.15.32:0/1785846999 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa46006be30 0x7fa46006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:52.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.436+0000 7fa45b7fe640 1 -- 172.21.15.32:0/1785846999 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5854+0+0 (secure 0 0 0) 0x7fa46408a180 con 0x7fa47406cd00 2024-04-24T23:00:52.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.436+0000 7fa4727fc640 1 --2- 172.21.15.32:0/1785846999 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa46006be30 0x7fa46006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:52.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.436+0000 7fa45b7fe640 1 -- 172.21.15.32:0/1785846999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=51}) -- 0x7fa460073ba0 con 0x7fa47406cd00 2024-04-24T23:00:52.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.436+0000 7fa4727fc640 1 --2- 172.21.15.32:0/1785846999 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa46006be30 0x7fa46006e2f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa468034000 tx=0x7fa46802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:52.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.440+0000 7fa45b7fe640 1 -- 172.21.15.32:0/1785846999 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa464058460 con 0x7fa47406cd00 2024-04-24T23:00:52.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.622+0000 7fa4796da640 1 -- 172.21.15.32:0/1785846999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7fa47406c000 con 0x7fa47406cd00 2024-04-24T23:00:52.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.624+0000 7fa45b7fe640 1 -- 172.21.15.32:0/1785846999 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v50) ==== 81+0+35282 (secure 0 0 0) 0x7fa46405c760 con 0x7fa47406cd00 2024-04-24T23:00:52.626 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:00:52.626 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:00:52.626 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 50, 2024-04-24T23:00:52.626 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", 2024-04-24T23:00:52.626 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2024-04-24T22:58:46.964851+0000", 2024-04-24T23:00:52.626 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2024-04-24T23:00:52.244581+0000", 2024-04-24T23:00:52.626 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2024-04-24T22:58:52.048787+0000", 2024-04-24T23:00:52.626 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2024-04-24T22:58:49.045609+0000", 2024-04-24T23:00:52.627 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-24T23:00:52.627 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2024-04-24T23:00:52.627 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2024-04-24T23:00:52.627 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2024-04-24T23:00:52.627 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2024-04-24T23:00:52.627 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2024-04-24T23:00:52.627 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2024-04-24T23:00:52.627 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:00:52.627 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 4, 2024-04-24T23:00:52.627 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.69999998807907104, 2024-04-24T23:00:52.627 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-24T23:00:52.627 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-24T23:00:52.628 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2024-04-24T23:00:52.628 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 7, 2024-04-24T23:00:52.628 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2024-04-24T23:00:52.628 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "luminous", 2024-04-24T23:00:52.628 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "luminous", 2024-04-24T23:00:52.628 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "squid", 2024-04-24T23:00:52.628 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_crimson": false, 2024-04-24T23:00:52.628 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2024-04-24T23:00:52.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.628 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2024-04-24T23:00:52.628 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": ".mgr", 2024-04-24T23:00:52.628 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T22:58:53.171408+0000", 2024-04-24T23:00:52.629 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 3, 2024-04-24T23:00:52.629 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,full", 2024-04-24T23:00:52.629 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:00:52.629 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:00:52.629 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:00:52.629 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:00:52.629 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:00:52.629 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:00:52.629 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:00:52.629 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:00:52.629 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:00:52.629 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:00:52.629 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2024-04-24T23:00:52.629 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2024-04-24T23:00:52.629 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2024-04-24T23:00:52.630 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2024-04-24T23:00:52.630 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2024-04-24T23:00:52.630 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:00:52.630 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:00:52.630 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:00:52.630 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:00:52.630 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:00:52.630 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:00:52.630 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:00:52.630 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.630 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "46", 2024-04-24T23:00:52.631 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:00:52.631 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:00:52.631 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:00:52.631 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:00:52.631 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:00:52.631 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:00:52.631 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:00:52.631 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:00:52.631 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:00:52.631 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:00:52.631 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:00:52.632 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:00:52.632 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:00:52.632 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:00:52.632 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:00:52.632 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:00:52.632 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:00:52.632 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:00:52.632 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:00:52.632 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:00:52.632 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:00:52.632 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:00:52.632 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:00:52.632 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:00:52.632 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:00:52.633 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:00:52.633 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.633 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:00:52.633 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:00:52.633 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:00:52.633 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:00:52.633 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:00:52.633 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:00:52.633 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:00:52.633 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:00:52.633 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:00:52.633 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:00:52.633 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:00:52.633 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:00:52.633 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_max": 32, 2024-04-24T23:00:52.634 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2024-04-24T23:00:52.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.634 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:00:52.634 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": {} 2024-04-24T23:00:52.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.634 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:00:52.634 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:00:52.634 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 8, 2024-04-24T23:00:52.634 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 8, 2024-04-24T23:00:52.634 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.25, 2024-04-24T23:00:52.635 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-04-24T23:00:52.635 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-04-24T23:00:52.635 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:00:52.635 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:00:52.635 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:00:52.635 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:52.635 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.635 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.635 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 6, 2024-04-24T23:00:52.635 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2024-04-24T23:00:52.635 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:00:30.665592+0000", 2024-04-24T23:00:52.635 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 3, 2024-04-24T23:00:52.635 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,full", 2024-04-24T23:00:52.635 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:00:52.636 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:00:52.636 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:00:52.636 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:00:52.636 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:00:52.636 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:00:52.636 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:00:52.636 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:00:52.636 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:00:52.636 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:00:52.636 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 32, 2024-04-24T23:00:52.636 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 32, 2024-04-24T23:00:52.636 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 32, 2024-04-24T23:00:52.636 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 32, 2024-04-24T23:00:52.637 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 32, 2024-04-24T23:00:52.637 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:00:52.637 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:00:52.637 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:00:52.637 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:00:52.637 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:00:52.637 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:00:52.637 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:00:52.637 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.637 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "46", 2024-04-24T23:00:52.637 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:00:52.637 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:00:52.637 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:00:52.637 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:00:52.637 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:00:52.638 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:00:52.638 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:00:52.638 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:00:52.638 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:00:52.638 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:00:52.638 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:00:52.638 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:00:52.638 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:00:52.638 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:00:52.638 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:00:52.638 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:00:52.638 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:00:52.638 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:00:52.638 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:00:52.639 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:00:52.639 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:00:52.639 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:00:52.639 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:00:52.639 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:00:52.639 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:00:52.639 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:00:52.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.639 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:00:52.639 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:00:52.639 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:00:52.639 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:00:52.639 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:00:52.639 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:00:52.639 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:00:52.640 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:00:52.640 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:00:52.640 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:00:52.640 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:00:52.640 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:00:52.640 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2024-04-24T23:00:52.640 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2024-04-24T23:00:52.640 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2024-04-24T23:00:52.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.640 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:00:52.640 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:00:52.640 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2024-04-24T23:00:52.640 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:52.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.640 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:00:52.641 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:00:52.641 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.5, 2024-04-24T23:00:52.641 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.5, 2024-04-24T23:00:52.641 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:00:52.641 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.5, 2024-04-24T23:00:52.641 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.5, 2024-04-24T23:00:52.641 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:00:52.641 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:00:52.641 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:00:52.641 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:52.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.641 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 7, 2024-04-24T23:00:52.641 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2024-04-24T23:00:52.642 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:00:31.357756+0000", 2024-04-24T23:00:52.642 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 3, 2024-04-24T23:00:52.642 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,full", 2024-04-24T23:00:52.642 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:00:52.642 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:00:52.642 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:00:52.642 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:00:52.642 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:00:52.642 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:00:52.642 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:00:52.642 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:00:52.642 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:00:52.642 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:00:52.642 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 64, 2024-04-24T23:00:52.643 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 64, 2024-04-24T23:00:52.643 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 64, 2024-04-24T23:00:52.643 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 64, 2024-04-24T23:00:52.643 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 64, 2024-04-24T23:00:52.643 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:00:52.643 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:00:52.643 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:00:52.643 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:00:52.643 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:00:52.643 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:00:52.643 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:00:52.644 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.644 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "46", 2024-04-24T23:00:52.644 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:00:52.644 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:00:52.644 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:00:52.644 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:00:52.644 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:00:52.644 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:00:52.644 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:00:52.644 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:00:52.644 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:00:52.644 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:00:52.644 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:00:52.644 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:00:52.645 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:00:52.645 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:00:52.645 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:00:52.645 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:00:52.645 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:00:52.645 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:00:52.645 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:00:52.645 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:00:52.645 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:00:52.645 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:00:52.645 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:00:52.645 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:00:52.645 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:00:52.645 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:00:52.646 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.646 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:00:52.646 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:00:52.646 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:00:52.646 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:00:52.646 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:00:52.646 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:00:52.646 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:00:52.646 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:00:52.646 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:00:52.646 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:00:52.646 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:00:52.646 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:00:52.646 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 64, 2024-04-24T23:00:52.647 INFO:teuthology.orchestra.run.smithi032.stdout: "target_size_ratio": 0.89999997615814209 2024-04-24T23:00:52.647 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.647 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:00:52.647 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:00:52.647 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2024-04-24T23:00:52.647 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:52.647 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.647 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:00:52.647 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:00:52.647 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.3799999952316284, 2024-04-24T23:00:52.647 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.3799999952316284, 2024-04-24T23:00:52.647 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:00:52.647 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.3799999952316284, 2024-04-24T23:00:52.647 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.3799999952316284, 2024-04-24T23:00:52.647 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:00:52.648 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:00:52.648 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:00:52.648 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:52.648 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:52.648 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:00:52.648 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-04-24T23:00:52.648 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.648 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:00:52.648 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:00:52.648 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:00:52.648 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:00:52.648 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:00:52.648 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:00:52.648 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:00:52.649 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:00:52.649 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:00:52.649 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 49, 2024-04-24T23:00:52.649 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:00:52.649 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:00:52.649 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:00:52.649 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:52.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.649 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:52.649 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2024-04-24T23:00:52.649 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:00:52.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.649 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:52.650 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2024-04-24T23:00:52.650 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:00:52.650 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:52.650 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:52.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.650 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:00:52.650 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:52.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.650 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:52.650 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2024-04-24T23:00:52.650 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:00:52.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.650 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:52.651 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2024-04-24T23:00:52.651 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:00:52.651 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:52.651 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:52.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.651 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:00:52.651 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:52.651 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.651 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:52.651 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2024-04-24T23:00:52.651 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:00:52.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.651 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.651 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:52.652 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2024-04-24T23:00:52.652 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:00:52.652 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:52.652 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:52.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.652 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:00:52.652 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:52.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.652 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:52.652 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2024-04-24T23:00:52.652 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:00:52.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.652 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:52.653 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2024-04-24T23:00:52.653 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:00:52.653 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:52.653 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:52.653 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.653 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/1479395426", 2024-04-24T23:00:52.653 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/1479395426", 2024-04-24T23:00:52.653 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/1479395426", 2024-04-24T23:00:52.653 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/1479395426", 2024-04-24T23:00:52.653 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:00:52.653 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull", 2024-04-24T23:00:52.653 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:00:52.653 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:00:52.653 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:52.654 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.654 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.654 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:00:52.654 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:00:52.654 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:00:52.654 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:00:52.654 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:00:52.654 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:00:52.654 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:00:52.654 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:00:52.654 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:00:52.654 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 49, 2024-04-24T23:00:52.654 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:00:52.654 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:00:52.655 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:00:52.655 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:52.655 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.655 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:52.655 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2024-04-24T23:00:52.655 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:00:52.655 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.655 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.655 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:52.655 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2024-04-24T23:00:52.655 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:00:52.655 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:52.655 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:52.655 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.656 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:00:52.656 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:52.656 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.656 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:52.656 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2024-04-24T23:00:52.656 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:00:52.656 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.656 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.656 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:52.656 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2024-04-24T23:00:52.656 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:00:52.656 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:52.656 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:52.656 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.656 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:00:52.657 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:52.657 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.657 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:52.657 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2024-04-24T23:00:52.657 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:00:52.657 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.657 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.657 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:52.657 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2024-04-24T23:00:52.657 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:00:52.657 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:52.657 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:52.657 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.657 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:00:52.658 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:52.658 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.658 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:52.658 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2024-04-24T23:00:52.658 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:00:52.658 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.658 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.658 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:52.658 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2024-04-24T23:00:52.658 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:00:52.658 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:52.658 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:52.658 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.658 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/1340633718", 2024-04-24T23:00:52.658 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/1340633718", 2024-04-24T23:00:52.659 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/1340633718", 2024-04-24T23:00:52.659 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/1340633718", 2024-04-24T23:00:52.659 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:00:52.659 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull", 2024-04-24T23:00:52.659 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:00:52.659 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:00:52.659 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:52.659 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.659 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.659 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:00:52.659 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:00:52.659 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:00:52.659 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:00:52.659 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:00:52.660 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:00:52.660 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:00:52.660 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:00:52.660 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:00:52.660 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 49, 2024-04-24T23:00:52.660 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:00:52.660 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:00:52.660 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:00:52.660 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:52.660 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.660 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:52.660 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2024-04-24T23:00:52.660 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:00:52.660 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.660 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.661 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:52.661 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2024-04-24T23:00:52.661 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:00:52.661 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:52.661 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:52.661 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.661 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:00:52.661 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:52.661 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.661 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:52.661 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2024-04-24T23:00:52.661 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:00:52.661 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.661 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.661 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:52.662 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2024-04-24T23:00:52.662 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:00:52.662 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:52.662 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:52.662 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.662 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:00:52.662 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:52.662 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.662 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:52.662 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2024-04-24T23:00:52.662 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:00:52.662 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.662 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.662 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:52.663 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2024-04-24T23:00:52.663 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:00:52.663 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:52.663 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:52.663 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.663 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:00:52.663 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:52.663 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.663 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:52.663 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2024-04-24T23:00:52.663 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:00:52.663 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.663 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.663 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:52.663 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2024-04-24T23:00:52.664 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:00:52.664 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:52.664 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:52.664 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.664 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/3405899334", 2024-04-24T23:00:52.664 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6820/3405899334", 2024-04-24T23:00:52.664 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6828/3405899334", 2024-04-24T23:00:52.664 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6824/3405899334", 2024-04-24T23:00:52.664 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:00:52.664 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:00:52.664 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:00:52.664 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:52.664 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.664 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.664 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:00:52.665 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:00:52.665 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:00:52.665 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:00:52.665 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:00:52.665 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:00:52.665 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:00:52.665 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:00:52.665 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:00:52.665 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 43, 2024-04-24T23:00:52.665 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:00:52.665 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:00:52.665 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:00:52.665 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:52.665 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.666 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:52.666 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2024-04-24T23:00:52.666 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:00:52.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.666 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.666 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:52.666 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2024-04-24T23:00:52.666 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:00:52.666 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:52.666 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:52.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.666 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:00:52.666 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:52.666 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.667 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:52.667 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2024-04-24T23:00:52.667 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:00:52.667 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.667 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.667 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:52.667 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2024-04-24T23:00:52.667 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:00:52.667 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:52.667 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:52.667 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.667 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:00:52.667 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:52.667 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.668 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:52.668 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2024-04-24T23:00:52.668 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:00:52.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.668 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.668 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:52.668 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2024-04-24T23:00:52.668 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:00:52.668 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:52.668 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:52.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.668 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:00:52.668 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:52.668 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.669 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:52.669 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2024-04-24T23:00:52.669 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:00:52.669 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.669 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.669 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:52.669 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2024-04-24T23:00:52.669 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:00:52.669 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:52.669 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:52.669 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.669 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6821/3381057190", 2024-04-24T23:00:52.669 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6825/3381057190", 2024-04-24T23:00:52.669 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/3381057190", 2024-04-24T23:00:52.670 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/3381057190", 2024-04-24T23:00:52.670 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:00:52.670 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull", 2024-04-24T23:00:52.670 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:00:52.670 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:00:52.670 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:52.670 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.670 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.670 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:00:52.670 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:00:52.670 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:00:52.670 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:00:52.670 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:00:52.671 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:00:52.671 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:00:52.671 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:00:52.671 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:00:52.671 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 43, 2024-04-24T23:00:52.671 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:00:52.671 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:00:52.671 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:00:52.671 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:52.671 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.671 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:52.671 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6800", 2024-04-24T23:00:52.671 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:00:52.671 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.672 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.672 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:52.672 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6801", 2024-04-24T23:00:52.672 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:00:52.672 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:52.672 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:52.672 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.672 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:00:52.672 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:52.672 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.672 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:52.672 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6802", 2024-04-24T23:00:52.672 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:00:52.673 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.673 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.673 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:52.673 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6803", 2024-04-24T23:00:52.673 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:00:52.673 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:52.673 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:52.673 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.673 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:00:52.673 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:52.673 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.673 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:52.673 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6806", 2024-04-24T23:00:52.673 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:00:52.674 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.674 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.674 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:52.674 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6807", 2024-04-24T23:00:52.674 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:00:52.674 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:52.674 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:52.674 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.674 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:00:52.674 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:52.674 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.674 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:52.674 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6804", 2024-04-24T23:00:52.674 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:00:52.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.675 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.675 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:52.675 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6805", 2024-04-24T23:00:52.675 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:00:52.675 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:52.675 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:52.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.675 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6801/4010553979", 2024-04-24T23:00:52.675 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6803/4010553979", 2024-04-24T23:00:52.676 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6807/4010553979", 2024-04-24T23:00:52.676 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6805/4010553979", 2024-04-24T23:00:52.676 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:00:52.676 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:00:52.676 INFO:teuthology.orchestra.run.smithi032.stdout: "full", 2024-04-24T23:00:52.676 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:00:52.676 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:52.676 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.676 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.676 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:00:52.676 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:00:52.676 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:00:52.676 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:00:52.676 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:00:52.676 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:00:52.677 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:00:52.677 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:00:52.677 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:00:52.677 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 43, 2024-04-24T23:00:52.677 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:00:52.677 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:00:52.677 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:00:52.677 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:52.677 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.677 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:52.677 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6808", 2024-04-24T23:00:52.677 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:00:52.677 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.677 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.678 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:52.678 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6809", 2024-04-24T23:00:52.678 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:00:52.678 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:52.678 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:52.678 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.678 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:00:52.678 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:52.678 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.678 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:52.678 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6810", 2024-04-24T23:00:52.678 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:00:52.678 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.678 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.679 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:52.679 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6811", 2024-04-24T23:00:52.679 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:00:52.679 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:52.679 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:52.679 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.679 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:00:52.679 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:52.679 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.679 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:52.679 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6814", 2024-04-24T23:00:52.679 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:00:52.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.680 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.680 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:52.680 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6815", 2024-04-24T23:00:52.680 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:00:52.680 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:52.680 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:52.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.680 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:00:52.680 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:52.680 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.680 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:52.680 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6812", 2024-04-24T23:00:52.680 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:00:52.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.681 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.681 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:52.681 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6813", 2024-04-24T23:00:52.681 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:00:52.681 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:52.681 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:52.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.681 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6809/3794084091", 2024-04-24T23:00:52.681 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6811/3794084091", 2024-04-24T23:00:52.681 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6815/3794084091", 2024-04-24T23:00:52.681 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6813/3794084091", 2024-04-24T23:00:52.681 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:00:52.682 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:00:52.682 INFO:teuthology.orchestra.run.smithi032.stdout: "full", 2024-04-24T23:00:52.682 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:00:52.682 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:52.682 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.682 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.682 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:00:52.682 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:00:52.682 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:00:52.682 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:00:52.682 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:00:52.682 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:00:52.682 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:00:52.682 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:00:52.683 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:00:52.683 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 43, 2024-04-24T23:00:52.683 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:00:52.683 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:00:52.683 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:00:52.683 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:52.683 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.683 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:52.683 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6823", 2024-04-24T23:00:52.683 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:00:52.683 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.683 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.683 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:52.683 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6825", 2024-04-24T23:00:52.684 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:00:52.684 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:52.684 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:52.684 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.684 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:00:52.684 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:52.684 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.684 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:52.684 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6826", 2024-04-24T23:00:52.684 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:00:52.684 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.684 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.684 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:52.685 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6827", 2024-04-24T23:00:52.685 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:00:52.685 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:52.685 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:52.685 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.685 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:00:52.685 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:52.685 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.685 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:52.686 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6830", 2024-04-24T23:00:52.686 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:00:52.686 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.686 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.686 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:52.686 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6831", 2024-04-24T23:00:52.686 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:00:52.686 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:52.686 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:52.686 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.686 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:00:52.686 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:52.686 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.686 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:52.687 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6828", 2024-04-24T23:00:52.687 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:00:52.687 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.687 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.687 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:52.687 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6829", 2024-04-24T23:00:52.687 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:00:52.687 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:52.687 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:52.687 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.687 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6825/1348640258", 2024-04-24T23:00:52.687 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6827/1348640258", 2024-04-24T23:00:52.687 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6831/1348640258", 2024-04-24T23:00:52.688 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6829/1348640258", 2024-04-24T23:00:52.688 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:00:52.688 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:00:52.688 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:00:52.688 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:52.688 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.688 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.688 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:00:52.688 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:00:52.688 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:00:52.688 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:00:52.688 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:00:52.688 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:00:52.688 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:00:52.689 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:00:52.689 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:00:52.689 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 49, 2024-04-24T23:00:52.689 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:00:52.689 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:00:52.689 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:00:52.689 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:52.689 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.689 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:52.689 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6816", 2024-04-24T23:00:52.689 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:00:52.689 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.689 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.689 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:52.690 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6817", 2024-04-24T23:00:52.690 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:00:52.690 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:52.690 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:52.690 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.690 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:00:52.690 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:52.690 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.690 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:52.690 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6818", 2024-04-24T23:00:52.690 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:00:52.690 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.690 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.691 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:52.691 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6819", 2024-04-24T23:00:52.691 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:00:52.691 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:52.691 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:52.691 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.691 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:00:52.691 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:52.691 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.691 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:52.691 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6822", 2024-04-24T23:00:52.691 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:00:52.691 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.692 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.692 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:52.692 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6824", 2024-04-24T23:00:52.692 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:00:52.692 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:52.692 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:52.692 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.692 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:00:52.692 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:52.692 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.692 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:52.692 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6820", 2024-04-24T23:00:52.692 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:00:52.693 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.693 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.693 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:52.693 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6821", 2024-04-24T23:00:52.693 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:00:52.693 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:52.693 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:52.693 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.693 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6817/3250005122", 2024-04-24T23:00:52.693 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6819/3250005122", 2024-04-24T23:00:52.693 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6824/3250005122", 2024-04-24T23:00:52.693 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6821/3250005122", 2024-04-24T23:00:52.693 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:00:52.693 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull", 2024-04-24T23:00:52.694 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:00:52.694 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:00:52.694 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:52.694 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:52.694 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:00:52.694 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2024-04-24T23:00:52.694 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.694 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:00:52.694 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:00:52.694 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:00:52.694 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:00:52.694 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:00:52.694 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:00:52.694 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.156184+0000", 2024-04-24T23:00:52.695 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:00:52.695 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.695 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.695 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:00:52.695 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:00:52.695 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:00:52.695 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:00:52.695 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:00:52.695 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:00:52.695 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240774+0000", 2024-04-24T23:00:52.695 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:00:52.695 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.695 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.696 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:00:52.696 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:00:52.696 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:00:52.696 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:00:52.696 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:00:52.696 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:00:52.696 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240949+0000", 2024-04-24T23:00:52.696 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:00:52.696 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.696 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.696 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:00:52.696 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:00:52.696 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:00:52.697 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:00:52.697 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:00:52.697 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:00:52.697 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.171672+0000", 2024-04-24T23:00:52.697 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:00:52.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.697 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.697 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:00:52.697 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:00:52.697 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:00:52.697 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:00:52.697 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:00:52.697 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:00:52.697 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.220379+0000", 2024-04-24T23:00:52.698 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:00:52.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.698 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.698 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:00:52.698 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:00:52.698 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:00:52.698 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:00:52.698 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:00:52.698 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:00:52.698 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.204963+0000", 2024-04-24T23:00:52.698 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:00:52.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.698 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.699 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:00:52.699 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:00:52.699 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:00:52.699 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:00:52.699 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:00:52.699 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:00:52.699 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.239748+0000", 2024-04-24T23:00:52.699 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:00:52.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.699 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.699 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:00:52.699 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:00:52.699 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:00:52.699 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:00:52.700 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:00:52.700 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:00:52.700 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.154104+0000", 2024-04-24T23:00:52.700 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:00:52.700 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:52.700 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:00:52.700 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2024-04-24T23:00:52.700 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [ 2024-04-24T23:00:52.700 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.700 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "7.6", 2024-04-24T23:00:52.700 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:00:52.700 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.700 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 5, 2024-04-24T23:00:52.700 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 7 2024-04-24T23:00:52.701 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:52.701 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:52.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.701 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.701 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "7.c", 2024-04-24T23:00:52.701 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:00:52.701 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.701 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 5, 2024-04-24T23:00:52.701 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 7 2024-04-24T23:00:52.701 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:52.701 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:52.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.701 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.701 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "7.12", 2024-04-24T23:00:52.701 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:00:52.702 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.702 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 5, 2024-04-24T23:00:52.702 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 7 2024-04-24T23:00:52.702 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:52.702 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:52.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.702 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.702 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "7.31", 2024-04-24T23:00:52.702 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:00:52.702 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.702 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 5, 2024-04-24T23:00:52.702 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 7 2024-04-24T23:00:52.702 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:52.702 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:52.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.703 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.703 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "7.35", 2024-04-24T23:00:52.703 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:00:52.703 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:52.703 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 5, 2024-04-24T23:00:52.703 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 0 2024-04-24T23:00:52.703 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:52.703 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:52.703 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:52.703 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:00:52.703 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_primaries": [], 2024-04-24T23:00:52.703 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2024-04-24T23:00:52.703 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2024-04-24T23:00:52.703 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklist": {}, 2024-04-24T23:00:52.703 INFO:teuthology.orchestra.run.smithi032.stdout: "range_blocklist": {}, 2024-04-24T23:00:52.704 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2024-04-24T23:00:52.704 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2024-04-24T23:00:52.704 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2024-04-24T23:00:52.704 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2024-04-24T23:00:52.704 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2024-04-24T23:00:52.704 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2024-04-24T23:00:52.704 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2024-04-24T23:00:52.704 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:52.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:52.704 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2024-04-24T23:00:52.704 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2024-04-24T23:00:52.704 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2024-04-24T23:00:52.704 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2024-04-24T23:00:52.704 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {}, 2024-04-24T23:00:52.704 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode": { 2024-04-24T23:00:52.705 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_enabled": false, 2024-04-24T23:00:52.705 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_bucket_count": 0, 2024-04-24T23:00:52.705 INFO:teuthology.orchestra.run.smithi032.stdout: "degraded_stretch_mode": 0, 2024-04-24T23:00:52.705 INFO:teuthology.orchestra.run.smithi032.stdout: "recovering_stretch_mode": 0, 2024-04-24T23:00:52.705 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_bucket": 0 2024-04-24T23:00:52.705 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:52.705 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:00:52.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.626+0000 7fa4796da640 1 -- 172.21.15.32:0/1785846999 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa46006be30 msgr2=0x7fa46006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:52.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.626+0000 7fa4796da640 1 --2- 172.21.15.32:0/1785846999 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa46006be30 0x7fa46006e2f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa468034000 tx=0x7fa46802e000 comp rx=0 tx=0).stop 2024-04-24T23:00:52.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.626+0000 7fa4796da640 1 -- 172.21.15.32:0/1785846999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa47406cd00 msgr2=0x7fa474158970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:52.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.626+0000 7fa4796da640 1 --2- 172.21.15.32:0/1785846999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa47406cd00 0x7fa474158970 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fa46400c500 tx=0x7fa46400c9c0 comp rx=0 tx=0).stop 2024-04-24T23:00:52.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.626+0000 7fa4796da640 1 -- 172.21.15.32:0/1785846999 shutdown_connections 2024-04-24T23:00:52.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.626+0000 7fa4796da640 1 --2- 172.21.15.32:0/1785846999 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa46006be30 0x7fa46006e2f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:52.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.626+0000 7fa4796da640 1 --2- 172.21.15.32:0/1785846999 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa47415cb00 0x7fa474072170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:52.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.626+0000 7fa4796da640 1 --2- 172.21.15.32:0/1785846999 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa47415b380 0x7fa4740b19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:52.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.626+0000 7fa4796da640 1 --2- 172.21.15.32:0/1785846999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa47406cd00 0x7fa474158970 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:52.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.626+0000 7fa4796da640 1 -- 172.21.15.32:0/1785846999 >> 172.21.15.32:0/1785846999 conn(0x7fa47406b110 msgr2=0x7fa47415cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:52.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.626+0000 7fa4796da640 1 -- 172.21.15.32:0/1785846999 shutdown_connections 2024-04-24T23:00:52.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:52.626+0000 7fa4796da640 1 -- 172.21.15.32:0/1785846999 wait complete. 2024-04-24T23:00:52.708 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-04-24T23:00:52.708 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:00:52.708 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:00:52.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:00:52.724 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=265) 2024-04-24T23:00:52.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:00:52.925 INFO:teuthology.orchestra.run.smithi032.stderr:dd: fdatasync failed for 'large_file_b': No space left on device 2024-04-24T23:00:52.925 INFO:teuthology.orchestra.run.smithi032.stderr:50+0 records in 2024-04-24T23:00:52.925 INFO:teuthology.orchestra.run.smithi032.stderr:50+0 records out 2024-04-24T23:00:52.925 INFO:teuthology.orchestra.run.smithi032.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.140586 s, 373 MB/s 2024-04-24T23:00:52.925 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-24T23:00:53.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.025+0000 7fd0d93ed640 1 Processor -- start 2024-04-24T23:00:53.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.025+0000 7fd0d93ed640 1 -- start start 2024-04-24T23:00:53.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.026+0000 7fd0d93ed640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd0d40b44d0 0x7fd0d40b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:53.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.026+0000 7fd0d93ed640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd0d40af890 0x7fd0d415cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:53.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.026+0000 7fd0d93ed640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0d40b6430 con 0x7fd0d406c5f0 2024-04-24T23:00:53.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.026+0000 7fd0d93ed640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fd0d4156f80 con 0x7fd0d40af890 2024-04-24T23:00:53.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.026+0000 7fd0d93ed640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fd0d40b6150 con 0x7fd0d40b44d0 2024-04-24T23:00:53.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.026+0000 7fd0d37fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd0d40af890 0x7fd0d415cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:53.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.026+0000 7fd0d37fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd0d40af890 0x7fd0d415cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48692/0 (socket says 172.21.15.32:48692) 2024-04-24T23:00:53.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.026+0000 7fd0d37fe640 1 -- 172.21.15.32:0/3426984145 learned_addr learned my addr 172.21.15.32:0/3426984145 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:53.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.026+0000 7fd0d27fc640 1 --2- 172.21.15.32:0/3426984145 >> v2:172.21.15.121:3300/0 conn(0x7fd0d40b44d0 0x7fd0d40b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:53.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.026+0000 7fd0d37fe640 1 -- 172.21.15.32:0/3426984145 >> v2:172.21.15.121:3300/0 conn(0x7fd0d40b44d0 msgr2=0x7fd0d40b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:53.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.026+0000 7fd0d37fe640 1 --2- 172.21.15.32:0/3426984145 >> v2:172.21.15.121:3300/0 conn(0x7fd0d40b44d0 0x7fd0d40b5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:53.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.026+0000 7fd0d37fe640 1 -- 172.21.15.32:0/3426984145 >> v1:172.21.15.121:6789/0 conn(0x7fd0d406c5f0 legacy=0x7fd0d406c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:53.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.026+0000 7fd0d37fe640 1 -- 172.21.15.32:0/3426984145 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0d40b39f0 con 0x7fd0d40af890 2024-04-24T23:00:53.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.027+0000 7fd0d37fe640 1 --2- 172.21.15.32:0/3426984145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0d40af890 0x7fd0d415cac0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fd0d4073080 tx=0x7fd0c402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44f64324738c6363 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:53.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.027+0000 7fd0d1ffb640 1 -- 172.21.15.32:0/3426984145 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c4036070 con 0x7fd0d40af890 2024-04-24T23:00:53.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.027+0000 7fd0d1ffb640 1 -- 172.21.15.32:0/3426984145 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0c4031600 con 0x7fd0d40af890 2024-04-24T23:00:53.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.027+0000 7fd0d1ffb640 1 -- 172.21.15.32:0/3426984145 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c40319d0 con 0x7fd0d40af890 2024-04-24T23:00:53.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.028+0000 7fd0d93ed640 1 -- 172.21.15.32:0/3426984145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0d40af890 msgr2=0x7fd0d415cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:53.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.028+0000 7fd0d93ed640 1 --2- 172.21.15.32:0/3426984145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0d40af890 0x7fd0d415cac0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fd0d4073080 tx=0x7fd0c402a730 comp rx=0 tx=0).stop 2024-04-24T23:00:53.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.028+0000 7fd0d93ed640 1 -- 172.21.15.32:0/3426984145 shutdown_connections 2024-04-24T23:00:53.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.028+0000 7fd0d93ed640 1 --2- 172.21.15.32:0/3426984145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0d40af890 0x7fd0d415cac0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:53.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.028+0000 7fd0d93ed640 1 --2- 172.21.15.32:0/3426984145 >> v2:172.21.15.121:3300/0 conn(0x7fd0d40b44d0 0x7fd0d40b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:53.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.028+0000 7fd0d93ed640 1 -- 172.21.15.32:0/3426984145 >> 172.21.15.32:0/3426984145 conn(0x7fd0d406b110 msgr2=0x7fd0d406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:53.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.028+0000 7fd0d93ed640 1 -- 172.21.15.32:0/3426984145 shutdown_connections 2024-04-24T23:00:53.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.028+0000 7fd0d93ed640 1 -- 172.21.15.32:0/3426984145 wait complete. 2024-04-24T23:00:53.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.029+0000 7fd0d93ed640 1 Processor -- start 2024-04-24T23:00:53.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.029+0000 7fd0d93ed640 1 -- start start 2024-04-24T23:00:53.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.030+0000 7fd0d93ed640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd0d406c5f0 0x7fd0d41590f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:53.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.030+0000 7fd0d93ed640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0d40af890 0x7fd0d414bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:53.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.030+0000 7fd0d93ed640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0d40b44d0 0x7fd0d40ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:53.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.030+0000 7fd0d93ed640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d40b5dd0 con 0x7fd0d40af890 2024-04-24T23:00:53.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.030+0000 7fd0d93ed640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d40b3fb0 con 0x7fd0d40b44d0 2024-04-24T23:00:53.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.030+0000 7fd0d93ed640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd0d415d000 con 0x7fd0d406c5f0 2024-04-24T23:00:53.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.030+0000 7fd0d27fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0d40af890 0x7fd0d414bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:53.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.030+0000 7fd0d27fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0d40af890 0x7fd0d414bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48708/0 (socket says 172.21.15.32:48708) 2024-04-24T23:00:53.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.030+0000 7fd0d27fc640 1 -- 172.21.15.32:0/1515942296 learned_addr learned my addr 172.21.15.32:0/1515942296 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:53.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.030+0000 7fd0d27fc640 1 -- 172.21.15.32:0/1515942296 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd0d406c5f0 msgr2=0x7fd0d41590f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:53.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.030+0000 7fd0d2ffd640 1 --2- 172.21.15.32:0/1515942296 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd0d406c5f0 0x7fd0d41590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:53.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.030+0000 7fd0d37fe640 1 --2- 172.21.15.32:0/1515942296 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0d40b44d0 0x7fd0d40ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:53.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.030+0000 7fd0d27fc640 1 --2- 172.21.15.32:0/1515942296 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd0d406c5f0 0x7fd0d41590f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:53.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.030+0000 7fd0d27fc640 1 -- 172.21.15.32:0/1515942296 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0d40b44d0 msgr2=0x7fd0d40ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:53.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.030+0000 7fd0d27fc640 1 --2- 172.21.15.32:0/1515942296 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0d40b44d0 0x7fd0d40ae8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:53.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.030+0000 7fd0d27fc640 1 -- 172.21.15.32:0/1515942296 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0d414c260 con 0x7fd0d40af890 2024-04-24T23:00:53.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.031+0000 7fd0d27fc640 1 --2- 172.21.15.32:0/1515942296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0d40af890 0x7fd0d414bd20 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd0c800d5a0 tx=0x7fd0c800da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:53.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.031+0000 7fd0bb7fe640 1 -- 172.21.15.32:0/1515942296 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c8019070 con 0x7fd0d40af890 2024-04-24T23:00:53.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.031+0000 7fd0bb7fe640 1 -- 172.21.15.32:0/1515942296 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0c8004240 con 0x7fd0d40af890 2024-04-24T23:00:53.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.031+0000 7fd0d93ed640 1 -- 172.21.15.32:0/1515942296 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0d4158050 con 0x7fd0d40af890 2024-04-24T23:00:53.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.031+0000 7fd0bb7fe640 1 -- 172.21.15.32:0/1515942296 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c8003610 con 0x7fd0d40af890 2024-04-24T23:00:53.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.031+0000 7fd0d93ed640 1 -- 172.21.15.32:0/1515942296 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0d4158610 con 0x7fd0d40af890 2024-04-24T23:00:53.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.032+0000 7fd0d93ed640 1 -- 172.21.15.32:0/1515942296 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0d4068430 con 0x7fd0d40af890 2024-04-24T23:00:53.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.032+0000 7fd0bb7fe640 1 -- 172.21.15.32:0/1515942296 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd0c801c070 con 0x7fd0d40af890 2024-04-24T23:00:53.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.033+0000 7fd0bb7fe640 1 --2- 172.21.15.32:0/1515942296 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd0c006be30 0x7fd0c006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:53.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.033+0000 7fd0bb7fe640 1 -- 172.21.15.32:0/1515942296 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5854+0+0 (secure 0 0 0) 0x7fd0c808a780 con 0x7fd0d40af890 2024-04-24T23:00:53.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.033+0000 7fd0bb7fe640 1 -- 172.21.15.32:0/1515942296 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=51}) -- 0x7fd0c0073b40 con 0x7fd0d40af890 2024-04-24T23:00:53.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.033+0000 7fd0d2ffd640 1 --2- 172.21.15.32:0/1515942296 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd0c006be30 0x7fd0c006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:53.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.034+0000 7fd0d2ffd640 1 --2- 172.21.15.32:0/1515942296 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd0c006be30 0x7fd0c006e2f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd0d415fd70 tx=0x7fd0cc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:53.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.037+0000 7fd0bb7fe640 1 -- 172.21.15.32:0/1515942296 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd0c8058a50 con 0x7fd0d40af890 2024-04-24T23:00:53.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.221+0000 7fd0d93ed640 1 -- 172.21.15.32:0/1515942296 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7fd0d40b5670 con 0x7fd0d40af890 2024-04-24T23:00:53.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.222+0000 7fd0bb7fe640 1 -- 172.21.15.32:0/1515942296 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v50) ==== 81+0+35282 (secure 0 0 0) 0x7fd0c805cd50 con 0x7fd0d40af890 2024-04-24T23:00:53.224 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:00:53.224 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:00:53.224 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 50, 2024-04-24T23:00:53.225 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", 2024-04-24T23:00:53.225 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2024-04-24T22:58:46.964851+0000", 2024-04-24T23:00:53.225 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2024-04-24T23:00:52.244581+0000", 2024-04-24T23:00:53.225 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2024-04-24T22:58:52.048787+0000", 2024-04-24T23:00:53.225 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2024-04-24T22:58:49.045609+0000", 2024-04-24T23:00:53.225 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-24T23:00:53.225 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2024-04-24T23:00:53.225 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2024-04-24T23:00:53.225 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2024-04-24T23:00:53.225 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2024-04-24T23:00:53.225 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2024-04-24T23:00:53.225 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2024-04-24T23:00:53.225 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:00:53.225 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 4, 2024-04-24T23:00:53.226 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.69999998807907104, 2024-04-24T23:00:53.226 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-24T23:00:53.226 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-24T23:00:53.226 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2024-04-24T23:00:53.226 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 7, 2024-04-24T23:00:53.226 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2024-04-24T23:00:53.226 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "luminous", 2024-04-24T23:00:53.226 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "luminous", 2024-04-24T23:00:53.226 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "squid", 2024-04-24T23:00:53.226 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_crimson": false, 2024-04-24T23:00:53.226 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2024-04-24T23:00:53.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.226 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2024-04-24T23:00:53.226 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": ".mgr", 2024-04-24T23:00:53.227 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T22:58:53.171408+0000", 2024-04-24T23:00:53.227 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 3, 2024-04-24T23:00:53.227 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,full", 2024-04-24T23:00:53.227 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:00:53.227 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:00:53.227 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:00:53.227 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:00:53.227 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:00:53.227 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:00:53.227 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:00:53.227 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:00:53.227 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:00:53.227 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:00:53.227 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2024-04-24T23:00:53.228 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2024-04-24T23:00:53.228 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2024-04-24T23:00:53.228 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2024-04-24T23:00:53.228 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2024-04-24T23:00:53.228 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:00:53.228 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:00:53.228 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:00:53.228 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:00:53.228 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:00:53.228 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:00:53.228 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:00:53.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.228 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "46", 2024-04-24T23:00:53.228 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:00:53.229 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:00:53.229 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:00:53.229 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:00:53.229 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:00:53.229 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:00:53.229 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:00:53.229 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:00:53.229 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:00:53.229 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:00:53.229 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:00:53.229 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:00:53.229 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:00:53.229 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:00:53.229 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:00:53.230 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:00:53.230 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:00:53.230 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:00:53.230 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:00:53.230 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:00:53.230 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:00:53.230 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:00:53.230 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:00:53.230 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:00:53.230 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:00:53.230 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:00:53.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.231 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:00:53.231 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:00:53.231 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:00:53.231 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:00:53.231 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:00:53.231 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:00:53.231 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:00:53.231 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:00:53.231 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:00:53.231 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:00:53.231 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:00:53.231 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:00:53.232 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_max": 32, 2024-04-24T23:00:53.232 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2024-04-24T23:00:53.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.232 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:00:53.232 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": {} 2024-04-24T23:00:53.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.232 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:00:53.232 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:00:53.232 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 8, 2024-04-24T23:00:53.232 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 8, 2024-04-24T23:00:53.232 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.25, 2024-04-24T23:00:53.232 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-04-24T23:00:53.232 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-04-24T23:00:53.232 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:00:53.232 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:00:53.233 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:00:53.233 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:53.233 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.233 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.233 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 6, 2024-04-24T23:00:53.233 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2024-04-24T23:00:53.233 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:00:30.665592+0000", 2024-04-24T23:00:53.233 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 3, 2024-04-24T23:00:53.233 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,full", 2024-04-24T23:00:53.233 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:00:53.233 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:00:53.233 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:00:53.234 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:00:53.234 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:00:53.234 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:00:53.234 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:00:53.234 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:00:53.234 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:00:53.234 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:00:53.234 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 32, 2024-04-24T23:00:53.234 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 32, 2024-04-24T23:00:53.234 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 32, 2024-04-24T23:00:53.234 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 32, 2024-04-24T23:00:53.234 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 32, 2024-04-24T23:00:53.234 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:00:53.235 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:00:53.235 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:00:53.235 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:00:53.235 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:00:53.235 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:00:53.235 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:00:53.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.235 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "46", 2024-04-24T23:00:53.235 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:00:53.235 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:00:53.235 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:00:53.235 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:00:53.235 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:00:53.235 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:00:53.236 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:00:53.236 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:00:53.236 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:00:53.236 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:00:53.236 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:00:53.236 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:00:53.236 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:00:53.236 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:00:53.236 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:00:53.236 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:00:53.236 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:00:53.236 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:00:53.236 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:00:53.236 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:00:53.237 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:00:53.237 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:00:53.237 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:00:53.237 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:00:53.237 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:00:53.237 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:00:53.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.237 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:00:53.237 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:00:53.237 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:00:53.237 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:00:53.237 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:00:53.237 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:00:53.237 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:00:53.238 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:00:53.238 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:00:53.238 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:00:53.238 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:00:53.238 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:00:53.238 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2024-04-24T23:00:53.238 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2024-04-24T23:00:53.238 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2024-04-24T23:00:53.238 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.238 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:00:53.238 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:00:53.238 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2024-04-24T23:00:53.238 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:53.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.239 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:00:53.239 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:00:53.239 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.5, 2024-04-24T23:00:53.239 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.5, 2024-04-24T23:00:53.239 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:00:53.239 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.5, 2024-04-24T23:00:53.239 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.5, 2024-04-24T23:00:53.239 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:00:53.239 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:00:53.239 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:00:53.239 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:53.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.239 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.240 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 7, 2024-04-24T23:00:53.240 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2024-04-24T23:00:53.240 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:00:31.357756+0000", 2024-04-24T23:00:53.240 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 3, 2024-04-24T23:00:53.240 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,full", 2024-04-24T23:00:53.240 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:00:53.240 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:00:53.240 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:00:53.240 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:00:53.240 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:00:53.240 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:00:53.240 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:00:53.240 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:00:53.241 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:00:53.241 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:00:53.241 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 64, 2024-04-24T23:00:53.241 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 64, 2024-04-24T23:00:53.241 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 64, 2024-04-24T23:00:53.241 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 64, 2024-04-24T23:00:53.241 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 64, 2024-04-24T23:00:53.241 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:00:53.241 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:00:53.241 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:00:53.241 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:00:53.241 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:00:53.241 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:00:53.242 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:00:53.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.242 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "46", 2024-04-24T23:00:53.242 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:00:53.242 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:00:53.242 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:00:53.242 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:00:53.242 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:00:53.242 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:00:53.242 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:00:53.242 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:00:53.242 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:00:53.242 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:00:53.242 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:00:53.242 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:00:53.243 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:00:53.243 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:00:53.243 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:00:53.243 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:00:53.243 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:00:53.243 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:00:53.243 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:00:53.243 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:00:53.243 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:00:53.243 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:00:53.243 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:00:53.243 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:00:53.243 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:00:53.243 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:00:53.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.244 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:00:53.244 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:00:53.244 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:00:53.244 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:00:53.244 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:00:53.244 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:00:53.244 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:00:53.244 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:00:53.244 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:00:53.244 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:00:53.244 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:00:53.244 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:00:53.244 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 64, 2024-04-24T23:00:53.244 INFO:teuthology.orchestra.run.smithi032.stdout: "target_size_ratio": 0.89999997615814209 2024-04-24T23:00:53.244 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.245 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:00:53.245 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:00:53.245 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2024-04-24T23:00:53.245 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:53.245 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.245 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:00:53.245 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:00:53.245 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.3799999952316284, 2024-04-24T23:00:53.245 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.3799999952316284, 2024-04-24T23:00:53.245 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:00:53.245 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.3799999952316284, 2024-04-24T23:00:53.245 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.3799999952316284, 2024-04-24T23:00:53.245 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:00:53.245 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:00:53.246 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:00:53.246 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:53.246 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:53.246 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:00:53.246 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-04-24T23:00:53.246 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.246 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:00:53.246 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:00:53.246 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:00:53.246 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:00:53.246 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:00:53.246 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:00:53.246 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:00:53.247 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:00:53.247 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:00:53.247 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 49, 2024-04-24T23:00:53.247 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:00:53.247 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:00:53.247 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:00:53.247 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:53.247 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.247 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:53.247 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2024-04-24T23:00:53.247 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:00:53.247 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.247 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.247 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:53.247 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2024-04-24T23:00:53.248 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:00:53.248 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:53.248 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:53.248 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.248 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:00:53.248 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:53.248 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.248 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:53.248 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2024-04-24T23:00:53.248 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:00:53.248 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.248 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.248 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:53.249 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2024-04-24T23:00:53.249 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:00:53.249 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:53.249 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:53.249 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.249 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:00:53.249 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:53.249 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.249 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:53.249 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2024-04-24T23:00:53.249 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:00:53.249 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.249 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.249 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:53.249 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2024-04-24T23:00:53.250 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:00:53.250 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:53.250 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:53.250 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.250 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:00:53.250 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:53.250 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.250 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:53.250 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2024-04-24T23:00:53.250 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:00:53.250 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.250 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.250 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:53.250 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2024-04-24T23:00:53.251 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:00:53.251 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:53.251 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:53.251 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.251 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/1479395426", 2024-04-24T23:00:53.251 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/1479395426", 2024-04-24T23:00:53.251 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/1479395426", 2024-04-24T23:00:53.251 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/1479395426", 2024-04-24T23:00:53.251 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:00:53.251 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull", 2024-04-24T23:00:53.251 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:00:53.251 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:00:53.251 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:53.252 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.252 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.252 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:00:53.252 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:00:53.252 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:00:53.252 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:00:53.252 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:00:53.252 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:00:53.252 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:00:53.252 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:00:53.252 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:00:53.252 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 49, 2024-04-24T23:00:53.252 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:00:53.252 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:00:53.252 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:00:53.253 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:53.253 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.253 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:53.253 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2024-04-24T23:00:53.253 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:00:53.253 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.253 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.253 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:53.253 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2024-04-24T23:00:53.253 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:00:53.253 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:53.253 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:53.253 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.253 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:00:53.253 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:53.254 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.254 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:53.254 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2024-04-24T23:00:53.254 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:00:53.254 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.254 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.254 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:53.254 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2024-04-24T23:00:53.254 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:00:53.254 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:53.254 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:53.254 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.254 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:00:53.255 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:53.255 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.255 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:53.255 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2024-04-24T23:00:53.255 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:00:53.255 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.255 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.255 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:53.255 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2024-04-24T23:00:53.255 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:00:53.255 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:53.255 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:53.255 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.255 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:00:53.256 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:53.256 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.256 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:53.256 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2024-04-24T23:00:53.256 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:00:53.256 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.256 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.256 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:53.256 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2024-04-24T23:00:53.256 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:00:53.256 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:53.256 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:53.256 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.257 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/1340633718", 2024-04-24T23:00:53.257 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/1340633718", 2024-04-24T23:00:53.257 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/1340633718", 2024-04-24T23:00:53.257 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/1340633718", 2024-04-24T23:00:53.257 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:00:53.257 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull", 2024-04-24T23:00:53.257 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:00:53.257 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:00:53.257 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:53.257 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.257 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.257 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:00:53.257 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:00:53.257 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:00:53.257 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:00:53.258 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:00:53.258 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:00:53.258 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:00:53.258 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:00:53.258 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:00:53.258 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 49, 2024-04-24T23:00:53.258 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:00:53.258 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:00:53.258 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:00:53.258 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:53.258 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.258 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:53.258 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2024-04-24T23:00:53.258 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:00:53.258 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.259 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.259 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:53.259 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2024-04-24T23:00:53.259 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:00:53.259 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:53.259 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:53.259 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.259 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:00:53.259 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:53.259 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.259 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:53.259 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2024-04-24T23:00:53.259 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:00:53.260 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.260 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.260 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:53.260 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2024-04-24T23:00:53.260 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:00:53.260 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:53.260 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:53.260 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.260 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:00:53.260 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:53.260 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.260 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:53.260 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2024-04-24T23:00:53.260 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:00:53.260 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.261 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.261 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:53.261 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2024-04-24T23:00:53.261 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:00:53.261 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:53.261 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:53.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.261 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:00:53.261 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:53.261 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.261 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:53.261 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2024-04-24T23:00:53.261 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:00:53.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.262 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:53.262 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2024-04-24T23:00:53.262 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:00:53.262 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:53.262 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:53.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.262 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/3405899334", 2024-04-24T23:00:53.262 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6820/3405899334", 2024-04-24T23:00:53.262 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6828/3405899334", 2024-04-24T23:00:53.262 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6824/3405899334", 2024-04-24T23:00:53.262 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:00:53.262 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:00:53.262 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:00:53.263 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:53.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.263 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.263 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:00:53.263 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:00:53.263 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:00:53.263 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:00:53.263 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:00:53.263 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:00:53.263 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:00:53.263 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:00:53.264 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:00:53.264 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 43, 2024-04-24T23:00:53.264 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:00:53.264 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:00:53.264 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:00:53.264 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:53.264 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.264 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:53.264 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2024-04-24T23:00:53.264 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:00:53.264 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.265 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.265 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:53.265 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2024-04-24T23:00:53.265 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:00:53.265 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:53.265 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:53.265 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.265 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:00:53.265 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:53.265 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.265 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:53.265 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2024-04-24T23:00:53.265 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:00:53.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.266 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.266 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:53.266 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2024-04-24T23:00:53.266 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:00:53.266 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:53.266 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:53.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.266 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:00:53.266 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:53.266 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.266 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:53.266 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2024-04-24T23:00:53.267 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:00:53.267 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.267 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.267 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:53.267 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2024-04-24T23:00:53.267 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:00:53.267 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:53.267 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:53.267 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.267 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:00:53.268 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:53.268 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.268 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:53.268 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2024-04-24T23:00:53.268 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:00:53.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.268 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.268 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:53.268 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2024-04-24T23:00:53.268 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:00:53.268 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:53.268 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:53.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.269 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6821/3381057190", 2024-04-24T23:00:53.269 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6825/3381057190", 2024-04-24T23:00:53.269 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/3381057190", 2024-04-24T23:00:53.269 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/3381057190", 2024-04-24T23:00:53.269 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:00:53.269 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull", 2024-04-24T23:00:53.269 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:00:53.269 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:00:53.269 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:53.269 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.269 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.269 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:00:53.269 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:00:53.269 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:00:53.269 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:00:53.270 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:00:53.270 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:00:53.270 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:00:53.270 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:00:53.270 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:00:53.270 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 43, 2024-04-24T23:00:53.270 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:00:53.270 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:00:53.270 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:00:53.270 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:53.270 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.270 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:53.270 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6800", 2024-04-24T23:00:53.270 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:00:53.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.271 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.271 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:53.271 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6801", 2024-04-24T23:00:53.271 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:00:53.271 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:53.271 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:53.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.271 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:00:53.271 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:53.271 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.271 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:53.272 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6802", 2024-04-24T23:00:53.272 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:00:53.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.272 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.272 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:53.272 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6803", 2024-04-24T23:00:53.272 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:00:53.272 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:53.273 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:53.273 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.273 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:00:53.273 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:53.273 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.273 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:53.273 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6806", 2024-04-24T23:00:53.273 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:00:53.273 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.273 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.273 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:53.273 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6807", 2024-04-24T23:00:53.273 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:00:53.274 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:53.274 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:53.274 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.274 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:00:53.274 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:53.274 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.274 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:53.274 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6804", 2024-04-24T23:00:53.278 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:00:53.278 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.279 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.279 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:53.279 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6805", 2024-04-24T23:00:53.279 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:00:53.280 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:53.280 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:53.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.280 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6801/4010553979", 2024-04-24T23:00:53.280 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6803/4010553979", 2024-04-24T23:00:53.281 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6807/4010553979", 2024-04-24T23:00:53.281 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6805/4010553979", 2024-04-24T23:00:53.281 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:00:53.281 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:00:53.281 INFO:teuthology.orchestra.run.smithi032.stdout: "full", 2024-04-24T23:00:53.281 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:00:53.281 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:53.281 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.281 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.282 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:00:53.282 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:00:53.282 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:00:53.282 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:00:53.282 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:00:53.282 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:00:53.282 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:00:53.282 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:00:53.282 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:00:53.283 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 43, 2024-04-24T23:00:53.283 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:00:53.283 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:00:53.283 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:00:53.283 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:53.283 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.283 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:53.283 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6808", 2024-04-24T23:00:53.283 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:00:53.283 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.284 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.284 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:53.284 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6809", 2024-04-24T23:00:53.284 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:00:53.284 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:53.284 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:53.284 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.284 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:00:53.285 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:53.285 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.285 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:53.285 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6810", 2024-04-24T23:00:53.285 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:00:53.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.286 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.286 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:53.286 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6811", 2024-04-24T23:00:53.286 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:00:53.286 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:53.286 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:53.286 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.287 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:00:53.287 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:53.287 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.287 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:53.287 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6814", 2024-04-24T23:00:53.287 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:00:53.287 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.287 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.287 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:53.287 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6815", 2024-04-24T23:00:53.287 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:00:53.287 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:53.287 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:53.287 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.288 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:00:53.288 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:53.288 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.288 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:53.288 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6812", 2024-04-24T23:00:53.288 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:00:53.288 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.288 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.288 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:53.288 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6813", 2024-04-24T23:00:53.289 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:00:53.289 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:53.289 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:53.289 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.289 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6809/3794084091", 2024-04-24T23:00:53.289 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6811/3794084091", 2024-04-24T23:00:53.289 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6815/3794084091", 2024-04-24T23:00:53.289 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6813/3794084091", 2024-04-24T23:00:53.289 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:00:53.289 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:00:53.289 INFO:teuthology.orchestra.run.smithi032.stdout: "full", 2024-04-24T23:00:53.289 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:00:53.289 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:53.290 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.290 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.290 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:00:53.290 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:00:53.290 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:00:53.290 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:00:53.290 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:00:53.290 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:00:53.290 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:00:53.290 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:00:53.290 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:00:53.290 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 43, 2024-04-24T23:00:53.290 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:00:53.291 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:00:53.291 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:00:53.291 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:53.291 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.291 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:53.291 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6823", 2024-04-24T23:00:53.291 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:00:53.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.291 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.291 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:53.292 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6825", 2024-04-24T23:00:53.292 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:00:53.292 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:53.292 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:53.292 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.292 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:00:53.292 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:53.292 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.292 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:53.292 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6826", 2024-04-24T23:00:53.292 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:00:53.292 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.293 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.293 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:53.293 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6827", 2024-04-24T23:00:53.293 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:00:53.293 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:53.293 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:53.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.293 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:00:53.293 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:53.293 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.293 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:53.293 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6830", 2024-04-24T23:00:53.293 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:00:53.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.293 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.294 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:53.294 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6831", 2024-04-24T23:00:53.294 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:00:53.294 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:53.294 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:53.294 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.294 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:00:53.294 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:53.294 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.294 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:53.294 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6828", 2024-04-24T23:00:53.294 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:00:53.294 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.294 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.295 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:53.295 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6829", 2024-04-24T23:00:53.295 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:00:53.295 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:53.295 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:53.295 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.295 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6825/1348640258", 2024-04-24T23:00:53.295 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6827/1348640258", 2024-04-24T23:00:53.295 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6831/1348640258", 2024-04-24T23:00:53.295 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6829/1348640258", 2024-04-24T23:00:53.295 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:00:53.295 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:00:53.295 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:00:53.295 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:53.295 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.296 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.296 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:00:53.296 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:00:53.296 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:00:53.296 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:00:53.296 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:00:53.296 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:00:53.296 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:00:53.296 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:00:53.296 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:00:53.296 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 49, 2024-04-24T23:00:53.296 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:00:53.296 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:00:53.296 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:00:53.296 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:53.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.297 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:53.297 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6816", 2024-04-24T23:00:53.297 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:00:53.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.297 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:53.297 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6817", 2024-04-24T23:00:53.297 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:00:53.297 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:53.297 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:53.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.297 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:00:53.297 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:53.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.298 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:53.298 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6818", 2024-04-24T23:00:53.298 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:00:53.298 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.298 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.298 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:53.298 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6819", 2024-04-24T23:00:53.298 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:00:53.298 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:53.298 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:53.298 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.298 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:00:53.298 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:53.298 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.298 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:53.299 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6822", 2024-04-24T23:00:53.299 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:00:53.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.299 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.299 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:53.299 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6824", 2024-04-24T23:00:53.299 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:00:53.299 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:53.299 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:53.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.299 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:00:53.299 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:53.299 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.299 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:53.299 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6820", 2024-04-24T23:00:53.300 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:00:53.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.300 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.300 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:53.300 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6821", 2024-04-24T23:00:53.300 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:00:53.300 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:53.300 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:53.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.300 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6817/3250005122", 2024-04-24T23:00:53.300 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6819/3250005122", 2024-04-24T23:00:53.300 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6824/3250005122", 2024-04-24T23:00:53.300 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6821/3250005122", 2024-04-24T23:00:53.300 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:00:53.300 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull", 2024-04-24T23:00:53.301 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:00:53.301 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:00:53.301 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:53.301 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:53.301 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:00:53.301 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2024-04-24T23:00:53.301 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.301 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:00:53.301 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:00:53.301 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:00:53.301 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:00:53.301 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:00:53.301 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:00:53.301 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.156184+0000", 2024-04-24T23:00:53.301 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:00:53.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.302 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.302 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:00:53.302 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:00:53.302 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:00:53.302 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:00:53.302 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:00:53.302 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:00:53.302 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240774+0000", 2024-04-24T23:00:53.302 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:00:53.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.302 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.302 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:00:53.302 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:00:53.302 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:00:53.303 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:00:53.303 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:00:53.303 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:00:53.303 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240949+0000", 2024-04-24T23:00:53.303 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:00:53.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.303 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.303 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:00:53.303 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:00:53.303 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:00:53.303 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:00:53.303 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:00:53.303 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:00:53.303 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.171672+0000", 2024-04-24T23:00:53.303 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:00:53.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.304 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.304 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:00:53.304 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:00:53.304 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:00:53.304 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:00:53.304 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:00:53.304 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:00:53.304 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.220379+0000", 2024-04-24T23:00:53.304 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:00:53.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.304 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.304 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:00:53.304 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:00:53.304 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:00:53.304 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:00:53.305 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:00:53.305 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:00:53.305 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.204963+0000", 2024-04-24T23:00:53.305 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:00:53.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.305 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.305 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:00:53.305 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:00:53.305 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:00:53.305 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:00:53.305 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:00:53.305 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:00:53.305 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.239748+0000", 2024-04-24T23:00:53.305 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:00:53.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.306 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.306 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:00:53.306 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:00:53.306 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:00:53.306 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:00:53.306 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:00:53.306 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:00:53.306 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.154104+0000", 2024-04-24T23:00:53.306 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:00:53.306 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:53.306 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:00:53.306 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2024-04-24T23:00:53.306 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [ 2024-04-24T23:00:53.306 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.307 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "7.6", 2024-04-24T23:00:53.307 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:00:53.307 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.307 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 5, 2024-04-24T23:00:53.307 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 7 2024-04-24T23:00:53.307 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:53.307 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:53.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.307 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.307 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "7.c", 2024-04-24T23:00:53.307 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:00:53.307 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.307 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 5, 2024-04-24T23:00:53.307 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 7 2024-04-24T23:00:53.307 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:53.308 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:53.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.308 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.308 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "7.12", 2024-04-24T23:00:53.308 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:00:53.308 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.308 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 5, 2024-04-24T23:00:53.308 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 7 2024-04-24T23:00:53.308 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:53.308 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:53.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.308 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.308 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "7.31", 2024-04-24T23:00:53.308 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:00:53.308 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.308 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 5, 2024-04-24T23:00:53.309 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 7 2024-04-24T23:00:53.309 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:53.309 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:53.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.309 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.309 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "7.35", 2024-04-24T23:00:53.309 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:00:53.309 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:53.309 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 5, 2024-04-24T23:00:53.309 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 0 2024-04-24T23:00:53.309 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:53.309 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:53.309 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:53.309 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:00:53.309 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_primaries": [], 2024-04-24T23:00:53.310 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2024-04-24T23:00:53.310 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2024-04-24T23:00:53.310 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklist": {}, 2024-04-24T23:00:53.310 INFO:teuthology.orchestra.run.smithi032.stdout: "range_blocklist": {}, 2024-04-24T23:00:53.310 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2024-04-24T23:00:53.310 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2024-04-24T23:00:53.310 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2024-04-24T23:00:53.310 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2024-04-24T23:00:53.310 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2024-04-24T23:00:53.310 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2024-04-24T23:00:53.310 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2024-04-24T23:00:53.310 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:53.310 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:53.310 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2024-04-24T23:00:53.310 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2024-04-24T23:00:53.311 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2024-04-24T23:00:53.311 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2024-04-24T23:00:53.311 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {}, 2024-04-24T23:00:53.311 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode": { 2024-04-24T23:00:53.311 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_enabled": false, 2024-04-24T23:00:53.311 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_bucket_count": 0, 2024-04-24T23:00:53.311 INFO:teuthology.orchestra.run.smithi032.stdout: "degraded_stretch_mode": 0, 2024-04-24T23:00:53.311 INFO:teuthology.orchestra.run.smithi032.stdout: "recovering_stretch_mode": 0, 2024-04-24T23:00:53.311 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_bucket": 0 2024-04-24T23:00:53.311 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:53.311 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:00:53.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.225+0000 7fd0d93ed640 1 -- 172.21.15.32:0/1515942296 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd0c006be30 msgr2=0x7fd0c006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:53.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.225+0000 7fd0d93ed640 1 --2- 172.21.15.32:0/1515942296 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd0c006be30 0x7fd0c006e2f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd0d415fd70 tx=0x7fd0cc009040 comp rx=0 tx=0).stop 2024-04-24T23:00:53.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.225+0000 7fd0d93ed640 1 -- 172.21.15.32:0/1515942296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0d40af890 msgr2=0x7fd0d414bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:53.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.225+0000 7fd0d93ed640 1 --2- 172.21.15.32:0/1515942296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0d40af890 0x7fd0d414bd20 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd0c800d5a0 tx=0x7fd0c800da60 comp rx=0 tx=0).stop 2024-04-24T23:00:53.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.225+0000 7fd0d93ed640 1 -- 172.21.15.32:0/1515942296 shutdown_connections 2024-04-24T23:00:53.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.225+0000 7fd0d93ed640 1 --2- 172.21.15.32:0/1515942296 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd0c006be30 0x7fd0c006e2f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:53.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.225+0000 7fd0d93ed640 1 --2- 172.21.15.32:0/1515942296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0d40af890 0x7fd0d414bd20 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:53.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.225+0000 7fd0d93ed640 1 --2- 172.21.15.32:0/1515942296 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0d40b44d0 0x7fd0d40ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:53.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.225+0000 7fd0d93ed640 1 --2- 172.21.15.32:0/1515942296 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd0d406c5f0 0x7fd0d41590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:53.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.225+0000 7fd0d93ed640 1 -- 172.21.15.32:0/1515942296 >> 172.21.15.32:0/1515942296 conn(0x7fd0d406b110 msgr2=0x7fd0d4155580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:53.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.225+0000 7fd0d93ed640 1 -- 172.21.15.32:0/1515942296 shutdown_connections 2024-04-24T23:00:53.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.225+0000 7fd0d93ed640 1 -- 172.21.15.32:0/1515942296 wait complete. 2024-04-24T23:00:53.314 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:00:53.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.413+0000 7f262d841640 1 Processor -- start 2024-04-24T23:00:53.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.413+0000 7f262d841640 1 -- start start 2024-04-24T23:00:53.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.414+0000 7f262d841640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f26281623d0 0x7f2628162820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:53.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.414+0000 7f262d841640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26280b1df0 con 0x7f262815e810 2024-04-24T23:00:53.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.414+0000 7f262d841640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26280b70d0 con 0x7f26280b6bf0 2024-04-24T23:00:53.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.414+0000 7f262d841640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f26280b7970 con 0x7f26281623d0 2024-04-24T23:00:53.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.414+0000 7f2626ffd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f262815e810 0x7f26280d1c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57150/0 (socket says 172.21.15.32:57150) 2024-04-24T23:00:53.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.414+0000 7f2626ffd640 1 -- 172.21.15.32:0/3282148921 learned_addr learned my addr 172.21.15.32:0/3282148921 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:00:53.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.415+0000 7f2625ffb640 1 -- 172.21.15.32:0/3282148921 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3446868340 0 0) 0x7f26280b1df0 con 0x7f262815e810 2024-04-24T23:00:53.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.415+0000 7f2625ffb640 1 -- 172.21.15.32:0/3282148921 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2614003520 con 0x7f262815e810 2024-04-24T23:00:53.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.415+0000 7f26277fe640 1 --2- 172.21.15.32:0/3282148921 >> v2:172.21.15.121:3300/0 conn(0x7f26281623d0 0x7f2628162820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:53.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.415+0000 7f2625ffb640 1 -- 172.21.15.32:0/3282148921 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 676116338 0 0) 0x7f2614003520 con 0x7f262815e810 2024-04-24T23:00:53.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.415+0000 7f2625ffb640 1 -- 172.21.15.32:0/3282148921 >> v2:172.21.15.121:3300/0 conn(0x7f26281623d0 msgr2=0x7f2628162820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:53.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.415+0000 7f2625ffb640 1 --2- 172.21.15.32:0/3282148921 >> v2:172.21.15.121:3300/0 conn(0x7f26281623d0 0x7f2628162820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:53.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.415+0000 7f2625ffb640 1 -- 172.21.15.32:0/3282148921 >> v1:172.21.15.121:6789/0 conn(0x7f26280b6bf0 legacy=0x7f262815f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:53.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.415+0000 7f2625ffb640 1 -- 172.21.15.32:0/3282148921 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26280b73b0 con 0x7f262815e810 2024-04-24T23:00:53.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.415+0000 7f2625ffb640 1 -- 172.21.15.32:0/3282148921 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f26180019f0 con 0x7f262815e810 2024-04-24T23:00:53.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.415+0000 7f2625ffb640 1 -- 172.21.15.32:0/3282148921 >> v1:172.21.15.32:6789/0 conn(0x7f262815e810 legacy=0x7f26280d1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:53.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.415+0000 7f2625ffb640 1 --2- 172.21.15.32:0/3282148921 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26140039b0 0x7f2614003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:53.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.415+0000 7f2625ffb640 1 --2- 172.21.15.32:0/3282148921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2614004800 0x7f2614004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:53.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.415+0000 7f2625ffb640 1 --2- 172.21.15.32:0/3282148921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26140051c0 0x7f261400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:53.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.415+0000 7f2625ffb640 1 -- 172.21.15.32:0/3282148921 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f26280b7970 con 0x7f26140051c0 2024-04-24T23:00:53.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.415+0000 7f2625ffb640 1 -- 172.21.15.32:0/3282148921 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2614014870 con 0x7f2614004800 2024-04-24T23:00:53.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.415+0000 7f2625ffb640 1 -- 172.21.15.32:0/3282148921 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f2614014a70 con 0x7f26140039b0 2024-04-24T23:00:53.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.415+0000 7f26277fe640 1 --2- 172.21.15.32:0/3282148921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26140051c0 0x7f261400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:53.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.415+0000 7f26267fc640 1 --2- 172.21.15.32:0/3282148921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2614004800 0x7f2614004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:53.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.415+0000 7f2626ffd640 1 --2- 172.21.15.32:0/3282148921 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26140039b0 0x7f2614003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:53.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.415+0000 7f26277fe640 1 -- 172.21.15.32:0/3282148921 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26140039b0 msgr2=0x7f2614003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:53.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.415+0000 7f26277fe640 1 --2- 172.21.15.32:0/3282148921 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26140039b0 0x7f2614003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:53.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.415+0000 7f26277fe640 1 -- 172.21.15.32:0/3282148921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2614004800 msgr2=0x7f2614004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:53.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.415+0000 7f26277fe640 1 --2- 172.21.15.32:0/3282148921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2614004800 0x7f2614004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:53.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.415+0000 7f26277fe640 1 -- 172.21.15.32:0/3282148921 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2614003520 con 0x7f26140051c0 2024-04-24T23:00:53.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.416+0000 7f26277fe640 1 -- 172.21.15.32:0/3282148921 reap_dead start 2024-04-24T23:00:53.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.416+0000 7f26277fe640 1 --2- 172.21.15.32:0/3282148921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26140051c0 0x7f261400b9c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f2620009ac0 tx=0x7f2614005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4342f564b4eea824 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:53.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.416+0000 7f2625ffb640 1 -- 172.21.15.32:0/3282148921 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2620009690 con 0x7f26140051c0 2024-04-24T23:00:53.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.416+0000 7f2625ffb640 1 -- 172.21.15.32:0/3282148921 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2620004020 con 0x7f26140051c0 2024-04-24T23:00:53.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.416+0000 7f2625ffb640 1 -- 172.21.15.32:0/3282148921 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26200043d0 con 0x7f26140051c0 2024-04-24T23:00:53.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.417+0000 7f262d841640 1 -- 172.21.15.32:0/3282148921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26140051c0 msgr2=0x7f261400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:53.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.417+0000 7f262d841640 1 --2- 172.21.15.32:0/3282148921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26140051c0 0x7f261400b9c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f2620009ac0 tx=0x7f2614005830 comp rx=0 tx=0).stop 2024-04-24T23:00:53.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.417+0000 7f262d841640 1 -- 172.21.15.32:0/3282148921 shutdown_connections 2024-04-24T23:00:53.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.417+0000 7f262d841640 1 --2- 172.21.15.32:0/3282148921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26140051c0 0x7f261400b9c0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:53.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.417+0000 7f262d841640 1 -- 172.21.15.32:0/3282148921 >> 172.21.15.32:0/3282148921 conn(0x7f26280d24c0 msgr2=0x7f26280d28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:53.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.417+0000 7f262d841640 1 -- 172.21.15.32:0/3282148921 shutdown_connections 2024-04-24T23:00:53.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.417+0000 7f262d841640 1 -- 172.21.15.32:0/3282148921 wait complete. 2024-04-24T23:00:53.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.418+0000 7f262d841640 1 Processor -- start 2024-04-24T23:00:53.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.418+0000 7f262d841640 1 -- start start 2024-04-24T23:00:53.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.419+0000 7f262d841640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26140051c0 0x7f26280d11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:53.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.419+0000 7f262d841640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f262806dfa0 0x7f262806b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:53.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.419+0000 7f262d841640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26280cfc50 0x7f26281f36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:53.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.419+0000 7f262d841640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f262815ebf0 con 0x7f262806dfa0 2024-04-24T23:00:53.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.419+0000 7f262d841640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f262815f010 con 0x7f26280cfc50 2024-04-24T23:00:53.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.419+0000 7f262d841640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f262815ed70 con 0x7f26140051c0 2024-04-24T23:00:53.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.419+0000 7f26267fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f262806dfa0 0x7f262806b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:53.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.419+0000 7f26267fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f262806dfa0 0x7f262806b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48718/0 (socket says 172.21.15.32:48718) 2024-04-24T23:00:53.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.419+0000 7f26267fc640 1 -- 172.21.15.32:0/1358556388 learned_addr learned my addr 172.21.15.32:0/1358556388 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:53.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.419+0000 7f2626ffd640 1 --2- 172.21.15.32:0/1358556388 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26140051c0 0x7f26280d11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:53.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.419+0000 7f26277fe640 1 --2- 172.21.15.32:0/1358556388 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26280cfc50 0x7f26281f36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:53.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.419+0000 7f26267fc640 1 -- 172.21.15.32:0/1358556388 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26140051c0 msgr2=0x7f26280d11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:53.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.419+0000 7f26267fc640 1 --2- 172.21.15.32:0/1358556388 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26140051c0 0x7f26280d11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:53.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.419+0000 7f26267fc640 1 -- 172.21.15.32:0/1358556388 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26280cfc50 msgr2=0x7f26281f36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:53.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.419+0000 7f26267fc640 1 --2- 172.21.15.32:0/1358556388 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26280cfc50 0x7f26281f36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:53.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.419+0000 7f26267fc640 1 -- 172.21.15.32:0/1358556388 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26280cebb0 con 0x7f262806dfa0 2024-04-24T23:00:53.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.420+0000 7f26267fc640 1 --2- 172.21.15.32:0/1358556388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f262806dfa0 0x7f262806b110 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f261000f650 tx=0x7f261000fb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:53.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.420+0000 7f261ffff640 1 -- 172.21.15.32:0/1358556388 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26100050d0 con 0x7f262806dfa0 2024-04-24T23:00:53.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.420+0000 7f261ffff640 1 -- 172.21.15.32:0/1358556388 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2610018070 con 0x7f262806dfa0 2024-04-24T23:00:53.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.420+0000 7f262d841640 1 -- 172.21.15.32:0/1358556388 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26280cee90 con 0x7f262806dfa0 2024-04-24T23:00:53.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.420+0000 7f261ffff640 1 -- 172.21.15.32:0/1358556388 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2610005490 con 0x7f262806dfa0 2024-04-24T23:00:53.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.420+0000 7f262d841640 1 -- 172.21.15.32:0/1358556388 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26280cf450 con 0x7f262806dfa0 2024-04-24T23:00:53.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.421+0000 7f262d841640 1 -- 172.21.15.32:0/1358556388 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2628068430 con 0x7f262806dfa0 2024-04-24T23:00:53.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.421+0000 7f261ffff640 1 -- 172.21.15.32:0/1358556388 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f261000d730 con 0x7f262806dfa0 2024-04-24T23:00:53.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.422+0000 7f261ffff640 1 --2- 172.21.15.32:0/1358556388 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f260806be30 0x7f260806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:53.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.422+0000 7f261ffff640 1 -- 172.21.15.32:0/1358556388 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5854+0+0 (secure 0 0 0) 0x7f261008aab0 con 0x7f262806dfa0 2024-04-24T23:00:53.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.422+0000 7f2626ffd640 1 --2- 172.21.15.32:0/1358556388 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f260806be30 0x7f260806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:53.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.422+0000 7f261ffff640 1 -- 172.21.15.32:0/1358556388 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=51}) -- 0x7f2608073b40 con 0x7f262806dfa0 2024-04-24T23:00:53.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.422+0000 7f2626ffd640 1 --2- 172.21.15.32:0/1358556388 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f260806be30 0x7f260806e2f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2618001560 tx=0x7f2618006040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:53.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.426+0000 7f261ffff640 1 -- 172.21.15.32:0/1358556388 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2610058e10 con 0x7f262806dfa0 2024-04-24T23:00:53.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.649+0000 7f262d841640 1 -- 172.21.15.32:0/1358556388 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f26280729d0 con 0x7f262806dfa0 2024-04-24T23:00:53.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.650+0000 7f261ffff640 1 -- 172.21.15.32:0/1358556388 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5187 (secure 0 0 0) 0x7f261005d110 con 0x7f262806dfa0 2024-04-24T23:00:53.652 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:00:53.652 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":20,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4994,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/1076304225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1076304225},{"type":"v1","addr":"172.21.15.32:6834","nonce":1076304225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":16},{"gid":5000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1388367206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1388367206},{"type":"v1","addr":"172.21.15.121:6835","nonce":1388367206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":5299,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2065945523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2065945523},{"type":"v1","addr":"172.21.15.121:6837","nonce":2065945523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:00:32.376194+0000","modified":"2024-04-24T23:00:34.381949+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"c","rank":0,"incarnation":18,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/3274703471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":3274703471},{"type":"v1","addr":"172.21.15.32:6835","nonce":3274703471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5631,"qdb_cluster":[5631]},"id":3}]} 2024-04-24T23:00:53.652 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 20 2024-04-24T23:00:53.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.652+0000 7f262d841640 1 -- 172.21.15.32:0/1358556388 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f260806be30 msgr2=0x7f260806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:53.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.652+0000 7f262d841640 1 --2- 172.21.15.32:0/1358556388 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f260806be30 0x7f260806e2f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2618001560 tx=0x7f2618006040 comp rx=0 tx=0).stop 2024-04-24T23:00:53.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.652+0000 7f262d841640 1 -- 172.21.15.32:0/1358556388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f262806dfa0 msgr2=0x7f262806b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:53.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.652+0000 7f262d841640 1 --2- 172.21.15.32:0/1358556388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f262806dfa0 0x7f262806b110 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f261000f650 tx=0x7f261000fb10 comp rx=0 tx=0).stop 2024-04-24T23:00:53.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.652+0000 7f262d841640 1 -- 172.21.15.32:0/1358556388 shutdown_connections 2024-04-24T23:00:53.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.652+0000 7f262d841640 1 --2- 172.21.15.32:0/1358556388 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f260806be30 0x7f260806e2f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:53.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.652+0000 7f262d841640 1 --2- 172.21.15.32:0/1358556388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f262806dfa0 0x7f262806b110 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:53.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.652+0000 7f262d841640 1 --2- 172.21.15.32:0/1358556388 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26280cfc50 0x7f26281f36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:53.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.652+0000 7f262d841640 1 --2- 172.21.15.32:0/1358556388 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26140051c0 0x7f26280d11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:53.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.652+0000 7f262d841640 1 -- 172.21.15.32:0/1358556388 >> 172.21.15.32:0/1358556388 conn(0x7f26280d24c0 msgr2=0x7f2628155e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:53.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.652+0000 7f262d841640 1 -- 172.21.15.32:0/1358556388 shutdown_connections 2024-04-24T23:00:53.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:53.652+0000 7f262d841640 1 -- 172.21.15.32:0/1358556388 wait complete. 2024-04-24T23:00:53.665 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2024-04-24T23:00:53.784 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","whoami":0,"id":5631,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":20.765824134999999,"mdsmap_epoch":20,"osdmap_epoch":50,"osdmap_epoch_barrier":50,"uptime":25.671646124999999} 2024-04-24T23:00:53.784 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", "fs_name": "cephfs", "id": 5631, "mdsmap_epoch": 20, "osdmap_epoch": 50, "osdmap_epoch_barrier": 50, "rank_uptime": 20.765824135, "state": "up:active", "uptime": 25.671646125, "want_state": "up:active", "whoami": 0 } 2024-04-24T23:00:53.785 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-24T23:00:53.785 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:00:53.785 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:00:53.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:00:53.801 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=small_file_1 bs=1M conv=fdatasync count=0 seek=0) 2024-04-24T23:00:53.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:00:53.857 INFO:teuthology.orchestra.run.smithi032.stderr:dd: failed to open 'small_file_1': Permission denied 2024-04-24T23:00:53.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:00:53.857 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2024-04-24T23:00:53.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:00:53.916 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2024-04-24T23:00:53.990 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-24T23:00:54.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.129+0000 7f02eb7ab640 1 Processor -- start 2024-04-24T23:00:54.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.129+0000 7f02eb7ab640 1 -- start start 2024-04-24T23:00:54.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.130+0000 7f02eb7ab640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f02e406cd00 0x7f02e40ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:54.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.130+0000 7f02eb7ab640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f02e415cb00 0x7f02e40ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:54.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.130+0000 7f02eb7ab640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02e40b4520 con 0x7f02e415b380 2024-04-24T23:00:54.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.130+0000 7f02eb7ab640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f02e40b4800 con 0x7f02e415cb00 2024-04-24T23:00:54.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.130+0000 7f02eb7ab640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f02e40b4ae0 con 0x7f02e406cd00 2024-04-24T23:00:54.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.130+0000 7f02e9d48640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f02e415cb00 0x7f02e40ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:54.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.130+0000 7f02e9d48640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f02e415cb00 0x7f02e40ce970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48728/0 (socket says 172.21.15.32:48728) 2024-04-24T23:00:54.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.130+0000 7f02e9d48640 1 -- 172.21.15.32:0/2914586193 learned_addr learned my addr 172.21.15.32:0/2914586193 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:54.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.130+0000 7f02e8d46640 1 --2- 172.21.15.32:0/2914586193 >> v2:172.21.15.121:3300/0 conn(0x7f02e406cd00 0x7f02e40ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:54.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.130+0000 7f02e9d48640 1 -- 172.21.15.32:0/2914586193 >> v2:172.21.15.121:3300/0 conn(0x7f02e406cd00 msgr2=0x7f02e40ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:54.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.130+0000 7f02e9d48640 1 --2- 172.21.15.32:0/2914586193 >> v2:172.21.15.121:3300/0 conn(0x7f02e406cd00 0x7f02e40ce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:54.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.130+0000 7f02e9d48640 1 -- 172.21.15.32:0/2914586193 >> v1:172.21.15.121:6789/0 conn(0x7f02e415b380 legacy=0x7f02e406c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:54.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.130+0000 7f02e9d48640 1 -- 172.21.15.32:0/2914586193 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02e40b4dc0 con 0x7f02e415cb00 2024-04-24T23:00:54.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.130+0000 7f02e8d46640 1 --2- 172.21.15.32:0/2914586193 >> v2:172.21.15.121:3300/0 conn(0x7f02e406cd00 0x7f02e40ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:00:54.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.131+0000 7f02e9d48640 1 --2- 172.21.15.32:0/2914586193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02e415cb00 0x7f02e40ce970 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f02e40b33c0 tx=0x7f02d402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f16bae1de346f57 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:54.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.131+0000 7f02e3fff640 1 -- 172.21.15.32:0/2914586193 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d4036070 con 0x7f02e415cb00 2024-04-24T23:00:54.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.131+0000 7f02e3fff640 1 -- 172.21.15.32:0/2914586193 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02d4031600 con 0x7f02e415cb00 2024-04-24T23:00:54.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.131+0000 7f02e3fff640 1 -- 172.21.15.32:0/2914586193 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d40319d0 con 0x7f02e415cb00 2024-04-24T23:00:54.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.132+0000 7f02eb7ab640 1 -- 172.21.15.32:0/2914586193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02e415cb00 msgr2=0x7f02e40ce970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:54.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.132+0000 7f02eb7ab640 1 --2- 172.21.15.32:0/2914586193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02e415cb00 0x7f02e40ce970 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f02e40b33c0 tx=0x7f02d402a660 comp rx=0 tx=0).stop 2024-04-24T23:00:54.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.132+0000 7f02eb7ab640 1 -- 172.21.15.32:0/2914586193 shutdown_connections 2024-04-24T23:00:54.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.132+0000 7f02eb7ab640 1 --2- 172.21.15.32:0/2914586193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02e415cb00 0x7f02e40ce970 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:54.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.132+0000 7f02eb7ab640 1 --2- 172.21.15.32:0/2914586193 >> v2:172.21.15.121:3300/0 conn(0x7f02e406cd00 0x7f02e40ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:54.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.132+0000 7f02eb7ab640 1 -- 172.21.15.32:0/2914586193 >> 172.21.15.32:0/2914586193 conn(0x7f02e406b110 msgr2=0x7f02e406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:54.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.132+0000 7f02eb7ab640 1 -- 172.21.15.32:0/2914586193 shutdown_connections 2024-04-24T23:00:54.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.132+0000 7f02eb7ab640 1 -- 172.21.15.32:0/2914586193 wait complete. 2024-04-24T23:00:54.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.133+0000 7f02eb7ab640 1 Processor -- start 2024-04-24T23:00:54.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.133+0000 7f02eb7ab640 1 -- start start 2024-04-24T23:00:54.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.134+0000 7f02eb7ab640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02e406cd00 0x7f02e4155790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:54.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.134+0000 7f02eb7ab640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02e415b380 0x7f02e41397d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:54.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.134+0000 7f02eb7ab640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02e415cb00 0x7f02e414cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:54.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.134+0000 7f02eb7ab640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f02e415b760 con 0x7f02e415b380 2024-04-24T23:00:54.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.134+0000 7f02eb7ab640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f02e40ceeb0 con 0x7f02e406cd00 2024-04-24T23:00:54.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.134+0000 7f02eb7ab640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f02e40b55c0 con 0x7f02e415cb00 2024-04-24T23:00:54.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.134+0000 7f02e8d46640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02e415b380 0x7f02e41397d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:54.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.134+0000 7f02e8d46640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02e415b380 0x7f02e41397d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48736/0 (socket says 172.21.15.32:48736) 2024-04-24T23:00:54.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.134+0000 7f02e8d46640 1 -- 172.21.15.32:0/247608639 learned_addr learned my addr 172.21.15.32:0/247608639 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:54.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.134+0000 7f02e9d48640 1 --2- 172.21.15.32:0/247608639 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02e415cb00 0x7f02e414cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:54.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.134+0000 7f02e9547640 1 --2- 172.21.15.32:0/247608639 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02e406cd00 0x7f02e4155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:54.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.134+0000 7f02e8d46640 1 -- 172.21.15.32:0/247608639 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02e415cb00 msgr2=0x7f02e414cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:54.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.134+0000 7f02e8d46640 1 --2- 172.21.15.32:0/247608639 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02e415cb00 0x7f02e414cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:54.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.134+0000 7f02e8d46640 1 -- 172.21.15.32:0/247608639 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02e406cd00 msgr2=0x7f02e4155790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:54.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.134+0000 7f02e8d46640 1 --2- 172.21.15.32:0/247608639 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02e406cd00 0x7f02e4155790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:54.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.134+0000 7f02e8d46640 1 -- 172.21.15.32:0/247608639 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02e4139d10 con 0x7f02e415b380 2024-04-24T23:00:54.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.134+0000 7f02e9547640 1 --2- 172.21.15.32:0/247608639 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02e406cd00 0x7f02e4155790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:00:54.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.134+0000 7f02e9d48640 1 --2- 172.21.15.32:0/247608639 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02e415cb00 0x7f02e414cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:00:54.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.134+0000 7f02e8d46640 1 --2- 172.21.15.32:0/247608639 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02e415b380 0x7f02e41397d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f02d800c7f0 tx=0x7f02d800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:54.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.135+0000 7f02e1ffb640 1 -- 172.21.15.32:0/247608639 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d8015070 con 0x7f02e415b380 2024-04-24T23:00:54.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.135+0000 7f02e1ffb640 1 -- 172.21.15.32:0/247608639 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02d80079d0 con 0x7f02e415b380 2024-04-24T23:00:54.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.135+0000 7f02e1ffb640 1 -- 172.21.15.32:0/247608639 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d8007da0 con 0x7f02e415b380 2024-04-24T23:00:54.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.135+0000 7f02eb7ab640 1 -- 172.21.15.32:0/247608639 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02e413cc90 con 0x7f02e415b380 2024-04-24T23:00:54.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.135+0000 7f02eb7ab640 1 -- 172.21.15.32:0/247608639 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02e4154cb0 con 0x7f02e415b380 2024-04-24T23:00:54.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.136+0000 7f02eb7ab640 1 -- 172.21.15.32:0/247608639 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02e4068430 con 0x7f02e415b380 2024-04-24T23:00:54.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.136+0000 7f02e1ffb640 1 -- 172.21.15.32:0/247608639 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f02d8018070 con 0x7f02e415b380 2024-04-24T23:00:54.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.137+0000 7f02e1ffb640 1 --2- 172.21.15.32:0/247608639 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f02d006be30 0x7f02d006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:54.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.137+0000 7f02e1ffb640 1 -- 172.21.15.32:0/247608639 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5854+0+0 (secure 0 0 0) 0x7f02d808aeb0 con 0x7f02e415b380 2024-04-24T23:00:54.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.137+0000 7f02e1ffb640 1 -- 172.21.15.32:0/247608639 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=51}) -- 0x7f02d0073b40 con 0x7f02e415b380 2024-04-24T23:00:54.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.137+0000 7f02e9547640 1 --2- 172.21.15.32:0/247608639 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f02d006be30 0x7f02d006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:54.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.137+0000 7f02e9547640 1 --2- 172.21.15.32:0/247608639 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f02d006be30 0x7f02d006e2f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f02dc003eb0 tx=0x7f02dc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:54.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.141+0000 7f02e1ffb640 1 -- 172.21.15.32:0/247608639 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f02e4139d10 con 0x7f02e415b380 2024-04-24T23:00:54.257 INFO:tasks.ceph.mon.a.smithi032.stderr:2024-04-24T23:00:54.255+0000 7f3b831cb640 -1 log_channel(cluster) log [ERR] : Health check update: 3 full osd(s) (OSD_FULL) 2024-04-24T23:00:54.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.260+0000 7f02e1ffb640 1 -- 172.21.15.32:0/247608639 <== mon.0 v2:172.21.15.32:3300/0 7 ==== osd_map(51..51 src has 1..51) ==== 304+0+0 (secure 0 0 0) 0x7f02d80570d0 con 0x7f02e415b380 2024-04-24T23:00:54.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.260+0000 7f02e1ffb640 1 -- 172.21.15.32:0/247608639 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=52}) -- 0x7f02d0074410 con 0x7f02e415b380 2024-04-24T23:00:54.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.329+0000 7f02eb7ab640 1 -- 172.21.15.32:0/247608639 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f02e40ce790 con 0x7f02e415b380 2024-04-24T23:00:54.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.330+0000 7f02e1ffb640 1 -- 172.21.15.32:0/247608639 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v51) ==== 81+0+35274 (secure 0 0 0) 0x7f02d8059730 con 0x7f02e415b380 2024-04-24T23:00:54.332 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:00:54.332 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:00:54.332 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 51, 2024-04-24T23:00:54.332 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", 2024-04-24T23:00:54.332 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2024-04-24T22:58:46.964851+0000", 2024-04-24T23:00:54.332 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2024-04-24T23:00:54.256662+0000", 2024-04-24T23:00:54.332 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2024-04-24T22:58:52.048787+0000", 2024-04-24T23:00:54.332 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2024-04-24T22:58:49.045609+0000", 2024-04-24T23:00:54.333 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-24T23:00:54.333 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2024-04-24T23:00:54.333 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2024-04-24T23:00:54.333 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2024-04-24T23:00:54.333 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2024-04-24T23:00:54.333 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2024-04-24T23:00:54.333 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2024-04-24T23:00:54.333 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:00:54.333 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 4, 2024-04-24T23:00:54.333 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.69999998807907104, 2024-04-24T23:00:54.333 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-24T23:00:54.333 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-24T23:00:54.333 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2024-04-24T23:00:54.334 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 7, 2024-04-24T23:00:54.334 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2024-04-24T23:00:54.334 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "luminous", 2024-04-24T23:00:54.334 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "luminous", 2024-04-24T23:00:54.334 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "squid", 2024-04-24T23:00:54.334 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_crimson": false, 2024-04-24T23:00:54.334 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2024-04-24T23:00:54.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.335 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2024-04-24T23:00:54.335 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": ".mgr", 2024-04-24T23:00:54.335 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T22:58:53.171408+0000", 2024-04-24T23:00:54.335 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 3, 2024-04-24T23:00:54.335 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,full", 2024-04-24T23:00:54.335 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:00:54.335 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:00:54.335 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:00:54.335 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:00:54.335 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:00:54.335 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:00:54.335 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:00:54.335 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:00:54.335 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:00:54.336 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:00:54.336 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2024-04-24T23:00:54.336 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2024-04-24T23:00:54.336 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2024-04-24T23:00:54.336 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2024-04-24T23:00:54.336 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2024-04-24T23:00:54.336 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:00:54.336 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:00:54.336 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:00:54.336 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:00:54.336 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:00:54.336 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:00:54.336 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:00:54.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.337 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "46", 2024-04-24T23:00:54.337 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:00:54.337 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:00:54.337 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:00:54.337 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:00:54.337 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:00:54.337 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:00:54.337 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:00:54.337 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:00:54.337 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:00:54.337 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:00:54.337 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:00:54.337 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:00:54.337 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:00:54.338 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:00:54.338 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:00:54.338 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:00:54.338 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:00:54.338 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:00:54.338 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:00:54.338 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:00:54.338 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:00:54.338 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:00:54.339 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:00:54.339 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:00:54.339 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:00:54.339 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:00:54.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.339 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:00:54.339 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:00:54.339 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:00:54.339 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:00:54.339 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:00:54.339 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:00:54.339 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:00:54.339 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:00:54.339 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:00:54.339 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:00:54.340 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:00:54.340 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:00:54.340 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_max": 32, 2024-04-24T23:00:54.340 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2024-04-24T23:00:54.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.340 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:00:54.340 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": {} 2024-04-24T23:00:54.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.341 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:00:54.341 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:00:54.341 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 8, 2024-04-24T23:00:54.341 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 8, 2024-04-24T23:00:54.341 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.25, 2024-04-24T23:00:54.341 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-04-24T23:00:54.341 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-04-24T23:00:54.341 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:00:54.341 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:00:54.341 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:00:54.341 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:54.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.341 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 6, 2024-04-24T23:00:54.341 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2024-04-24T23:00:54.342 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:00:30.665592+0000", 2024-04-24T23:00:54.342 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 3, 2024-04-24T23:00:54.342 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,full", 2024-04-24T23:00:54.342 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:00:54.342 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:00:54.342 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:00:54.342 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:00:54.342 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:00:54.342 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:00:54.342 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:00:54.342 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:00:54.342 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:00:54.343 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:00:54.343 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 32, 2024-04-24T23:00:54.343 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 32, 2024-04-24T23:00:54.343 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 32, 2024-04-24T23:00:54.343 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 32, 2024-04-24T23:00:54.343 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 32, 2024-04-24T23:00:54.343 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:00:54.343 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:00:54.343 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:00:54.343 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:00:54.343 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:00:54.343 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:00:54.343 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:00:54.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.343 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "46", 2024-04-24T23:00:54.344 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:00:54.344 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:00:54.344 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:00:54.344 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:00:54.344 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:00:54.344 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:00:54.344 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:00:54.344 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:00:54.345 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:00:54.345 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:00:54.345 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:00:54.345 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:00:54.345 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:00:54.345 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:00:54.345 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:00:54.345 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:00:54.345 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:00:54.345 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:00:54.345 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:00:54.345 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:00:54.345 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:00:54.345 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:00:54.346 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:00:54.346 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:00:54.346 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:00:54.346 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:00:54.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.346 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:00:54.346 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:00:54.346 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:00:54.346 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:00:54.346 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:00:54.346 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:00:54.346 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:00:54.346 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:00:54.346 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:00:54.346 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:00:54.347 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:00:54.347 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:00:54.347 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2024-04-24T23:00:54.347 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2024-04-24T23:00:54.347 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2024-04-24T23:00:54.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.347 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:00:54.347 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:00:54.347 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2024-04-24T23:00:54.347 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:54.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.347 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:00:54.347 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:00:54.347 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.5, 2024-04-24T23:00:54.348 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.5, 2024-04-24T23:00:54.348 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:00:54.348 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.5, 2024-04-24T23:00:54.348 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.5, 2024-04-24T23:00:54.348 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:00:54.348 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:00:54.348 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:00:54.348 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:54.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.348 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 7, 2024-04-24T23:00:54.348 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2024-04-24T23:00:54.348 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:00:31.357756+0000", 2024-04-24T23:00:54.348 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 3, 2024-04-24T23:00:54.348 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,full", 2024-04-24T23:00:54.349 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:00:54.349 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:00:54.349 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:00:54.349 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:00:54.349 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:00:54.349 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:00:54.349 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:00:54.349 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:00:54.349 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:00:54.349 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:00:54.349 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 64, 2024-04-24T23:00:54.349 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 64, 2024-04-24T23:00:54.349 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 64, 2024-04-24T23:00:54.349 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 64, 2024-04-24T23:00:54.349 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 64, 2024-04-24T23:00:54.350 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:00:54.350 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:00:54.350 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:00:54.350 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:00:54.350 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:00:54.350 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:00:54.350 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:00:54.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.351 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "46", 2024-04-24T23:00:54.351 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:00:54.351 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:00:54.351 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:00:54.351 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:00:54.351 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:00:54.351 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:00:54.351 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:00:54.351 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:00:54.351 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:00:54.351 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:00:54.351 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:00:54.351 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:00:54.352 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:00:54.352 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:00:54.352 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:00:54.352 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:00:54.352 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:00:54.352 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:00:54.352 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:00:54.352 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:00:54.352 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:00:54.352 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:00:54.352 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:00:54.352 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:00:54.352 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:00:54.352 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:00:54.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.353 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:00:54.353 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:00:54.353 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:00:54.353 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:00:54.353 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:00:54.353 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:00:54.353 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:00:54.353 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:00:54.353 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:00:54.353 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:00:54.353 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:00:54.353 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:00:54.354 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 64, 2024-04-24T23:00:54.354 INFO:teuthology.orchestra.run.smithi032.stdout: "target_size_ratio": 0.89999997615814209 2024-04-24T23:00:54.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.354 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:00:54.354 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:00:54.354 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2024-04-24T23:00:54.354 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:54.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.354 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:00:54.354 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:00:54.354 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.3799999952316284, 2024-04-24T23:00:54.354 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.3799999952316284, 2024-04-24T23:00:54.354 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:00:54.355 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.3799999952316284, 2024-04-24T23:00:54.355 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.3799999952316284, 2024-04-24T23:00:54.355 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:00:54.355 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:00:54.355 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:00:54.355 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:54.355 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:54.355 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:00:54.355 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-04-24T23:00:54.355 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.355 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:00:54.355 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:00:54.355 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:00:54.355 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:00:54.356 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:00:54.356 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:00:54.356 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:00:54.356 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:00:54.356 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:00:54.356 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 49, 2024-04-24T23:00:54.356 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:00:54.356 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:00:54.356 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:00:54.356 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:54.356 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.356 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:54.356 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2024-04-24T23:00:54.356 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:00:54.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.357 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:54.357 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2024-04-24T23:00:54.357 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:00:54.357 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:54.357 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:54.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.357 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:00:54.357 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:54.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.357 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:54.357 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2024-04-24T23:00:54.357 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:00:54.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.358 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:54.358 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2024-04-24T23:00:54.358 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:00:54.359 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:54.359 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:54.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.359 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:00:54.359 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:54.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.359 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:54.359 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2024-04-24T23:00:54.359 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:00:54.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.359 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:54.359 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2024-04-24T23:00:54.359 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:00:54.360 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:54.360 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:54.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.360 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:00:54.360 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:54.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.360 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:54.360 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2024-04-24T23:00:54.360 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:00:54.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.360 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:54.360 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2024-04-24T23:00:54.360 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:00:54.361 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:54.361 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:54.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.361 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/1479395426", 2024-04-24T23:00:54.361 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/1479395426", 2024-04-24T23:00:54.361 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/1479395426", 2024-04-24T23:00:54.361 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/1479395426", 2024-04-24T23:00:54.361 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:00:54.361 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull", 2024-04-24T23:00:54.361 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:00:54.361 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:00:54.361 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:54.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.361 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.362 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:00:54.362 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:00:54.362 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:00:54.362 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:00:54.362 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:00:54.362 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:00:54.362 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:00:54.362 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:00:54.362 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:00:54.362 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 49, 2024-04-24T23:00:54.362 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:00:54.362 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:00:54.362 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:00:54.363 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:54.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.363 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:54.363 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2024-04-24T23:00:54.363 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:00:54.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.364 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:54.364 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2024-04-24T23:00:54.364 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:00:54.364 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:54.364 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:54.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.364 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:00:54.364 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:54.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.364 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:54.365 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2024-04-24T23:00:54.365 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:00:54.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.365 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:54.365 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2024-04-24T23:00:54.365 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:00:54.365 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:54.365 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:54.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.365 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:00:54.365 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:54.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.365 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:54.366 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2024-04-24T23:00:54.366 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:00:54.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.366 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:54.366 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2024-04-24T23:00:54.366 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:00:54.366 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:54.366 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:54.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.366 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:00:54.366 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:54.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.366 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:54.367 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2024-04-24T23:00:54.367 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:00:54.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.367 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:54.367 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2024-04-24T23:00:54.367 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:00:54.367 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:54.367 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:54.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.367 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/1340633718", 2024-04-24T23:00:54.367 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/1340633718", 2024-04-24T23:00:54.367 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/1340633718", 2024-04-24T23:00:54.367 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/1340633718", 2024-04-24T23:00:54.368 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:00:54.368 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:00:54.368 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:00:54.368 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:54.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.368 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:00:54.368 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:00:54.368 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:00:54.368 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:00:54.368 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:00:54.368 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:00:54.368 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:00:54.368 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:00:54.368 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:00:54.369 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 49, 2024-04-24T23:00:54.369 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:00:54.369 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:00:54.369 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:00:54.369 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:54.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.369 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:54.369 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2024-04-24T23:00:54.369 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:00:54.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.369 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:54.370 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2024-04-24T23:00:54.370 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:00:54.370 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:54.370 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:54.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.370 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:00:54.371 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:54.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.371 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:54.371 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2024-04-24T23:00:54.371 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:00:54.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.372 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:54.372 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2024-04-24T23:00:54.372 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:00:54.372 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:54.373 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:54.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.373 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:00:54.373 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:54.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.373 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:54.374 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2024-04-24T23:00:54.374 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:00:54.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.374 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:54.374 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2024-04-24T23:00:54.374 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:00:54.374 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:54.375 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:54.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.375 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:00:54.375 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:54.375 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.375 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:54.375 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2024-04-24T23:00:54.375 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:00:54.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.375 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.375 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:54.375 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2024-04-24T23:00:54.376 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:00:54.376 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:54.376 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:54.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.376 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/3405899334", 2024-04-24T23:00:54.376 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6820/3405899334", 2024-04-24T23:00:54.376 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6828/3405899334", 2024-04-24T23:00:54.376 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6824/3405899334", 2024-04-24T23:00:54.376 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:00:54.376 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:00:54.377 INFO:teuthology.orchestra.run.smithi032.stdout: "full", 2024-04-24T23:00:54.377 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:00:54.377 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:54.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.377 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:00:54.377 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:00:54.377 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:00:54.378 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:00:54.378 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:00:54.378 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:00:54.378 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:00:54.378 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:00:54.378 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:00:54.378 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 43, 2024-04-24T23:00:54.378 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:00:54.379 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:00:54.379 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:00:54.379 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:54.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.379 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:54.379 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2024-04-24T23:00:54.379 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:00:54.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.380 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:54.380 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2024-04-24T23:00:54.380 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:00:54.380 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:54.380 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:54.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.380 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:00:54.380 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:54.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.380 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:54.380 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2024-04-24T23:00:54.380 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:00:54.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.381 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:54.381 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2024-04-24T23:00:54.381 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:00:54.381 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:54.381 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:54.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.381 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:00:54.381 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:54.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.381 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:54.381 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2024-04-24T23:00:54.381 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:00:54.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.382 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:54.382 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2024-04-24T23:00:54.382 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:00:54.382 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:54.382 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:54.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.382 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:00:54.382 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:54.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.382 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:54.382 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2024-04-24T23:00:54.382 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:00:54.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.383 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:54.383 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2024-04-24T23:00:54.383 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:00:54.383 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:54.383 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:54.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.383 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6821/3381057190", 2024-04-24T23:00:54.383 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6825/3381057190", 2024-04-24T23:00:54.383 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/3381057190", 2024-04-24T23:00:54.383 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/3381057190", 2024-04-24T23:00:54.383 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:00:54.383 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull", 2024-04-24T23:00:54.383 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:00:54.383 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:00:54.384 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:54.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.384 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:00:54.384 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:00:54.384 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:00:54.384 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:00:54.384 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:00:54.384 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:00:54.384 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:00:54.384 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:00:54.385 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:00:54.385 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 43, 2024-04-24T23:00:54.385 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:00:54.385 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:00:54.385 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:00:54.385 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:54.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.385 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:54.385 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6800", 2024-04-24T23:00:54.385 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:00:54.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.386 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:54.386 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6801", 2024-04-24T23:00:54.386 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:00:54.386 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:54.386 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:54.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.386 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:00:54.386 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:54.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.386 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:54.386 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6802", 2024-04-24T23:00:54.386 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:00:54.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.387 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:54.387 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6803", 2024-04-24T23:00:54.387 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:00:54.387 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:54.387 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:54.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.387 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:00:54.387 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:54.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.387 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:54.387 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6806", 2024-04-24T23:00:54.387 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:00:54.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.388 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:54.388 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6807", 2024-04-24T23:00:54.388 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:00:54.388 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:54.388 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:54.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.388 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:00:54.388 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:54.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.388 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:54.388 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6804", 2024-04-24T23:00:54.388 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:00:54.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.389 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:54.389 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6805", 2024-04-24T23:00:54.389 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:00:54.389 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:54.389 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:54.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.389 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6801/4010553979", 2024-04-24T23:00:54.389 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6803/4010553979", 2024-04-24T23:00:54.389 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6807/4010553979", 2024-04-24T23:00:54.389 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6805/4010553979", 2024-04-24T23:00:54.389 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:00:54.389 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:00:54.389 INFO:teuthology.orchestra.run.smithi032.stdout: "full", 2024-04-24T23:00:54.390 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:00:54.390 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:54.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.390 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:00:54.390 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:00:54.390 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:00:54.390 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:00:54.390 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:00:54.390 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:00:54.390 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:00:54.390 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:00:54.390 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:00:54.390 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 43, 2024-04-24T23:00:54.391 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:00:54.391 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:00:54.391 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:00:54.391 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:54.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.391 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:54.391 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6808", 2024-04-24T23:00:54.391 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:00:54.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.391 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:54.391 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6809", 2024-04-24T23:00:54.392 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:00:54.392 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:54.392 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:54.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.392 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:00:54.393 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:54.393 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.393 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:54.393 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6810", 2024-04-24T23:00:54.393 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:00:54.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.393 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.393 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:54.393 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6811", 2024-04-24T23:00:54.393 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:00:54.393 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:54.393 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:54.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.394 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:00:54.394 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:54.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.394 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:54.394 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6814", 2024-04-24T23:00:54.394 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:00:54.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.394 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:54.394 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6815", 2024-04-24T23:00:54.394 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:00:54.394 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:54.394 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:54.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.395 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:00:54.395 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:54.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.395 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:54.395 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6812", 2024-04-24T23:00:54.395 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:00:54.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.395 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:54.395 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6813", 2024-04-24T23:00:54.395 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:00:54.395 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:54.395 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:54.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.396 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6809/3794084091", 2024-04-24T23:00:54.396 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6811/3794084091", 2024-04-24T23:00:54.396 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6815/3794084091", 2024-04-24T23:00:54.396 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6813/3794084091", 2024-04-24T23:00:54.396 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:00:54.396 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:00:54.396 INFO:teuthology.orchestra.run.smithi032.stdout: "full", 2024-04-24T23:00:54.396 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:00:54.396 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:54.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.396 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:00:54.396 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:00:54.396 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:00:54.396 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:00:54.397 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:00:54.397 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:00:54.397 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:00:54.397 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:00:54.397 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:00:54.397 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 43, 2024-04-24T23:00:54.397 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:00:54.397 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:00:54.397 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:00:54.397 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:54.397 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.397 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:54.397 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6823", 2024-04-24T23:00:54.397 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:00:54.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.398 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.398 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:54.398 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6825", 2024-04-24T23:00:54.398 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:00:54.398 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:54.398 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:54.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.398 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:00:54.398 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:54.398 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.399 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:54.399 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6826", 2024-04-24T23:00:54.399 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:00:54.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.399 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.399 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:54.399 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6827", 2024-04-24T23:00:54.399 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:00:54.399 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:54.399 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:54.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.400 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:00:54.400 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:54.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.400 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:54.400 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6830", 2024-04-24T23:00:54.400 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:00:54.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.400 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:54.400 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6831", 2024-04-24T23:00:54.400 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:00:54.400 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:54.400 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:54.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.401 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:00:54.401 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:54.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.401 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:54.401 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6828", 2024-04-24T23:00:54.401 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:00:54.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.401 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:54.401 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6829", 2024-04-24T23:00:54.401 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:00:54.401 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:54.401 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:54.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.401 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6825/1348640258", 2024-04-24T23:00:54.402 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6827/1348640258", 2024-04-24T23:00:54.402 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6831/1348640258", 2024-04-24T23:00:54.402 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6829/1348640258", 2024-04-24T23:00:54.402 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:00:54.402 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:00:54.402 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:00:54.402 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:54.402 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.402 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.402 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:00:54.403 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:00:54.403 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:00:54.403 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:00:54.403 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:00:54.403 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:00:54.403 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:00:54.403 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:00:54.403 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:00:54.404 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 49, 2024-04-24T23:00:54.404 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:00:54.404 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:00:54.404 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:00:54.404 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:54.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.404 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:54.404 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6816", 2024-04-24T23:00:54.404 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:00:54.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.404 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:54.404 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6817", 2024-04-24T23:00:54.405 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:00:54.405 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:54.405 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:54.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.405 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:00:54.405 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:54.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.405 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:54.405 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6818", 2024-04-24T23:00:54.405 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:00:54.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.405 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:54.405 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6819", 2024-04-24T23:00:54.405 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:00:54.406 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:54.406 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:54.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.406 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:00:54.406 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:54.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.406 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:54.406 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6822", 2024-04-24T23:00:54.406 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:00:54.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.406 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:54.406 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6824", 2024-04-24T23:00:54.407 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:00:54.407 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:54.407 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:54.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.407 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:00:54.407 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:54.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.407 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:54.407 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6820", 2024-04-24T23:00:54.407 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:00:54.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.407 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:54.407 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6821", 2024-04-24T23:00:54.407 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:00:54.408 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:54.408 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:54.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.408 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6817/3250005122", 2024-04-24T23:00:54.408 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6819/3250005122", 2024-04-24T23:00:54.408 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6824/3250005122", 2024-04-24T23:00:54.408 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6821/3250005122", 2024-04-24T23:00:54.408 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:00:54.408 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull", 2024-04-24T23:00:54.408 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:00:54.408 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:00:54.409 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:54.409 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:54.409 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:00:54.409 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2024-04-24T23:00:54.409 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.409 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:00:54.409 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:00:54.409 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:00:54.409 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:00:54.409 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:00:54.410 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:00:54.410 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.156184+0000", 2024-04-24T23:00:54.410 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:00:54.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.410 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.410 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:00:54.410 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:00:54.410 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:00:54.410 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:00:54.410 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:00:54.410 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:00:54.410 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240774+0000", 2024-04-24T23:00:54.410 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:00:54.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.411 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.411 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:00:54.411 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:00:54.411 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:00:54.411 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:00:54.411 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:00:54.411 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:00:54.411 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240949+0000", 2024-04-24T23:00:54.411 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:00:54.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.411 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.411 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:00:54.411 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:00:54.412 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:00:54.412 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:00:54.412 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:00:54.412 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:00:54.412 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.171672+0000", 2024-04-24T23:00:54.412 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:00:54.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.412 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.412 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:00:54.412 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:00:54.412 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:00:54.413 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:00:54.413 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:00:54.413 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:00:54.413 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.220379+0000", 2024-04-24T23:00:54.413 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:00:54.413 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.413 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.413 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:00:54.413 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:00:54.413 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:00:54.413 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:00:54.413 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:00:54.413 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:00:54.414 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.204963+0000", 2024-04-24T23:00:54.414 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:00:54.414 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.414 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.414 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:00:54.414 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:00:54.414 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:00:54.414 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:00:54.414 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:00:54.414 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:00:54.414 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.239748+0000", 2024-04-24T23:00:54.414 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:00:54.414 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.414 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.415 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:00:54.415 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:00:54.415 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:00:54.415 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:00:54.415 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:00:54.415 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:00:54.415 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.154104+0000", 2024-04-24T23:00:54.415 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:00:54.415 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:54.416 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:00:54.416 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2024-04-24T23:00:54.416 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [ 2024-04-24T23:00:54.416 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.416 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "7.6", 2024-04-24T23:00:54.416 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:00:54.416 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.416 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 5, 2024-04-24T23:00:54.416 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 7 2024-04-24T23:00:54.417 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:54.417 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:54.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.417 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "7.c", 2024-04-24T23:00:54.417 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:00:54.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.417 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 5, 2024-04-24T23:00:54.417 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 7 2024-04-24T23:00:54.418 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:54.418 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:54.418 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.418 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.418 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "7.12", 2024-04-24T23:00:54.418 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:00:54.418 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.418 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 5, 2024-04-24T23:00:54.418 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 7 2024-04-24T23:00:54.418 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:54.419 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:54.419 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.419 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.419 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "7.31", 2024-04-24T23:00:54.419 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:00:54.419 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.419 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 5, 2024-04-24T23:00:54.419 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 7 2024-04-24T23:00:54.419 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:54.419 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:54.419 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.419 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.419 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "7.35", 2024-04-24T23:00:54.419 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:00:54.419 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:54.420 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 5, 2024-04-24T23:00:54.420 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 0 2024-04-24T23:00:54.420 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:54.420 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:54.420 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:54.420 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:00:54.420 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_primaries": [], 2024-04-24T23:00:54.420 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2024-04-24T23:00:54.420 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2024-04-24T23:00:54.420 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklist": {}, 2024-04-24T23:00:54.420 INFO:teuthology.orchestra.run.smithi032.stdout: "range_blocklist": {}, 2024-04-24T23:00:54.420 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2024-04-24T23:00:54.420 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2024-04-24T23:00:54.420 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2024-04-24T23:00:54.420 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2024-04-24T23:00:54.421 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2024-04-24T23:00:54.421 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2024-04-24T23:00:54.421 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2024-04-24T23:00:54.421 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:54.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:54.421 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2024-04-24T23:00:54.421 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2024-04-24T23:00:54.421 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2024-04-24T23:00:54.421 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2024-04-24T23:00:54.422 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {}, 2024-04-24T23:00:54.422 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode": { 2024-04-24T23:00:54.422 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_enabled": false, 2024-04-24T23:00:54.422 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_bucket_count": 0, 2024-04-24T23:00:54.422 INFO:teuthology.orchestra.run.smithi032.stdout: "degraded_stretch_mode": 0, 2024-04-24T23:00:54.422 INFO:teuthology.orchestra.run.smithi032.stdout: "recovering_stretch_mode": 0, 2024-04-24T23:00:54.422 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_bucket": 0 2024-04-24T23:00:54.422 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:54.422 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:00:54.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.332+0000 7f02eb7ab640 1 -- 172.21.15.32:0/247608639 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f02d006be30 msgr2=0x7f02d006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:54.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.332+0000 7f02eb7ab640 1 --2- 172.21.15.32:0/247608639 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f02d006be30 0x7f02d006e2f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f02dc003eb0 tx=0x7f02dc009040 comp rx=0 tx=0).stop 2024-04-24T23:00:54.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.332+0000 7f02eb7ab640 1 -- 172.21.15.32:0/247608639 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02e415b380 msgr2=0x7f02e41397d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:54.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.332+0000 7f02eb7ab640 1 --2- 172.21.15.32:0/247608639 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02e415b380 0x7f02e41397d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f02d800c7f0 tx=0x7f02d800cbf0 comp rx=0 tx=0).stop 2024-04-24T23:00:54.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.332+0000 7f02eb7ab640 1 -- 172.21.15.32:0/247608639 shutdown_connections 2024-04-24T23:00:54.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.332+0000 7f02eb7ab640 1 --2- 172.21.15.32:0/247608639 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f02d006be30 0x7f02d006e2f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:54.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.332+0000 7f02eb7ab640 1 --2- 172.21.15.32:0/247608639 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02e415b380 0x7f02e41397d0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:54.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.332+0000 7f02eb7ab640 1 --2- 172.21.15.32:0/247608639 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02e415cb00 0x7f02e414cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:54.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.332+0000 7f02eb7ab640 1 --2- 172.21.15.32:0/247608639 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02e406cd00 0x7f02e4155790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:54.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.332+0000 7f02eb7ab640 1 -- 172.21.15.32:0/247608639 >> 172.21.15.32:0/247608639 conn(0x7f02e406b110 msgr2=0x7f02e40b3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:54.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.332+0000 7f02eb7ab640 1 -- 172.21.15.32:0/247608639 shutdown_connections 2024-04-24T23:00:54.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:54.332+0000 7f02eb7ab640 1 -- 172.21.15.32:0/247608639 wait complete. 2024-04-24T23:00:54.426 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-04-24T23:00:59.279 INFO:tasks.ceph.mon.a.smithi032.stderr:2024-04-24T23:00:59.278+0000 7f3b831cb640 -1 log_channel(cluster) log [ERR] : Health check update: 1 full osd(s) (OSD_FULL) 2024-04-24T23:00:59.427 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-24T23:00:59.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.539+0000 7f347be1c640 1 Processor -- start 2024-04-24T23:00:59.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.540+0000 7f347be1c640 1 -- start start 2024-04-24T23:00:59.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.540+0000 7f347be1c640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f34740d4210 0x7f34740d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:59.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.540+0000 7f347be1c640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34740d3170 con 0x7f347415e9b0 2024-04-24T23:00:59.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.540+0000 7f347be1c640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34740d3450 con 0x7f347406b820 2024-04-24T23:00:59.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.540+0000 7f347be1c640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f34740d3730 con 0x7f34740d4210 2024-04-24T23:00:59.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.541+0000 7f3479bb8640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f347415e9b0 0x7f347406b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57156/0 (socket says 172.21.15.32:57156) 2024-04-24T23:00:59.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.541+0000 7f3479bb8640 1 -- 172.21.15.32:0/1608519353 learned_addr learned my addr 172.21.15.32:0/1608519353 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:00:59.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.541+0000 7f3478bb6640 1 -- 172.21.15.32:0/1608519353 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1521018372 0 0) 0x7f34740d3170 con 0x7f347415e9b0 2024-04-24T23:00:59.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.541+0000 7f347a3b9640 1 --2- 172.21.15.32:0/1608519353 >> v2:172.21.15.121:3300/0 conn(0x7f34740d4210 0x7f34740d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:59.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.541+0000 7f3478bb6640 1 -- 172.21.15.32:0/1608519353 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3460003520 con 0x7f347415e9b0 2024-04-24T23:00:59.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.541+0000 7f3478bb6640 1 -- 172.21.15.32:0/1608519353 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2560088459 0 0) 0x7f3460003520 con 0x7f347415e9b0 2024-04-24T23:00:59.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.541+0000 7f3478bb6640 1 -- 172.21.15.32:0/1608519353 >> v2:172.21.15.121:3300/0 conn(0x7f34740d4210 msgr2=0x7f34740d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:59.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.541+0000 7f3478bb6640 1 --2- 172.21.15.32:0/1608519353 >> v2:172.21.15.121:3300/0 conn(0x7f34740d4210 0x7f34740d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:59.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.541+0000 7f3478bb6640 1 -- 172.21.15.32:0/1608519353 >> v1:172.21.15.121:6789/0 conn(0x7f347406b820 legacy=0x7f34740b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:59.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.541+0000 7f3478bb6640 1 -- 172.21.15.32:0/1608519353 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34740d3a10 con 0x7f347415e9b0 2024-04-24T23:00:59.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.541+0000 7f3478bb6640 1 -- 172.21.15.32:0/1608519353 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f3470001670 con 0x7f347415e9b0 2024-04-24T23:00:59.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.541+0000 7f3478bb6640 1 -- 172.21.15.32:0/1608519353 >> v1:172.21.15.32:6789/0 conn(0x7f347415e9b0 legacy=0x7f347406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:59.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.541+0000 7f3478bb6640 1 --2- 172.21.15.32:0/1608519353 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f34600039b0 0x7f3460003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:59.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.541+0000 7f3478bb6640 1 --2- 172.21.15.32:0/1608519353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3460004800 0x7f3460004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:59.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.541+0000 7f3478bb6640 1 --2- 172.21.15.32:0/1608519353 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f34600051c0 0x7f346000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:59.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.541+0000 7f3478bb6640 1 -- 172.21.15.32:0/1608519353 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f34740d3730 con 0x7f3460004800 2024-04-24T23:00:59.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.541+0000 7f3478bb6640 1 -- 172.21.15.32:0/1608519353 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f346000c160 con 0x7f34600039b0 2024-04-24T23:00:59.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.541+0000 7f3478bb6640 1 -- 172.21.15.32:0/1608519353 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f346000c360 con 0x7f34600051c0 2024-04-24T23:00:59.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.541+0000 7f34793b7640 1 --2- 172.21.15.32:0/1608519353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3460004800 0x7f3460004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:59.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.541+0000 7f3479bb8640 1 --2- 172.21.15.32:0/1608519353 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f34600039b0 0x7f3460003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:59.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.541+0000 7f347a3b9640 1 --2- 172.21.15.32:0/1608519353 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f34600051c0 0x7f346000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:59.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.542+0000 7f34793b7640 1 -- 172.21.15.32:0/1608519353 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f34600051c0 msgr2=0x7f346000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:59.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.542+0000 7f34793b7640 1 --2- 172.21.15.32:0/1608519353 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f34600051c0 0x7f346000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:59.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.542+0000 7f34793b7640 1 -- 172.21.15.32:0/1608519353 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f34600039b0 msgr2=0x7f3460003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:00:59.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.542+0000 7f34793b7640 1 --2- 172.21.15.32:0/1608519353 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f34600039b0 0x7f3460003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:59.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.542+0000 7f34793b7640 1 -- 172.21.15.32:0/1608519353 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3460003520 con 0x7f3460004800 2024-04-24T23:00:59.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.542+0000 7f347a3b9640 1 -- 172.21.15.32:0/1608519353 reap_dead start 2024-04-24T23:00:59.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.542+0000 7f3479bb8640 1 --2- 172.21.15.32:0/1608519353 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f34600039b0 0x7f3460003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:00:59.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.542+0000 7f34793b7640 1 --2- 172.21.15.32:0/1608519353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3460004800 0x7f3460004c80 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f3464001750 tx=0x7f3460005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de25b5e9030044b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:59.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.543+0000 7f3478bb6640 1 -- 172.21.15.32:0/1608519353 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3464001370 con 0x7f3460004800 2024-04-24T23:00:59.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.543+0000 7f3478bb6640 1 -- 172.21.15.32:0/1608519353 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34640052d0 con 0x7f3460004800 2024-04-24T23:00:59.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.543+0000 7f3478bb6640 1 -- 172.21.15.32:0/1608519353 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346402abc0 con 0x7f3460004800 2024-04-24T23:00:59.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.543+0000 7f347be1c640 1 -- 172.21.15.32:0/1608519353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3460004800 msgr2=0x7f3460004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:59.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.543+0000 7f347be1c640 1 --2- 172.21.15.32:0/1608519353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3460004800 0x7f3460004c80 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f3464001750 tx=0x7f3460005830 comp rx=0 tx=0).stop 2024-04-24T23:00:59.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.543+0000 7f347be1c640 1 -- 172.21.15.32:0/1608519353 shutdown_connections 2024-04-24T23:00:59.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.543+0000 7f347be1c640 1 --2- 172.21.15.32:0/1608519353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3460004800 0x7f3460004c80 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:59.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.543+0000 7f347be1c640 1 -- 172.21.15.32:0/1608519353 >> 172.21.15.32:0/1608519353 conn(0x7f34740d11d0 msgr2=0x7f34740d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:59.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.543+0000 7f347be1c640 1 -- 172.21.15.32:0/1608519353 shutdown_connections 2024-04-24T23:00:59.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.543+0000 7f347be1c640 1 -- 172.21.15.32:0/1608519353 wait complete. 2024-04-24T23:00:59.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.545+0000 7f347be1c640 1 Processor -- start 2024-04-24T23:00:59.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.545+0000 7f347be1c640 1 -- start start 2024-04-24T23:00:59.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.545+0000 7f347be1c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3460004800 0x7f3474155120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:59.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.545+0000 7f347be1c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f34740ce610 0x7f347415ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:59.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.545+0000 7f347be1c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f34740c9970 0x7f34740c9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:59.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.545+0000 7f347be1c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f34740ba0e0 con 0x7f3460004800 2024-04-24T23:00:59.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.545+0000 7f347be1c640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f34740d2400 con 0x7f34740c9970 2024-04-24T23:00:59.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.545+0000 7f347be1c640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f34740ba260 con 0x7f34740ce610 2024-04-24T23:00:59.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.545+0000 7f3479bb8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3460004800 0x7f3474155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:59.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.545+0000 7f3479bb8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3460004800 0x7f3474155120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48742/0 (socket says 172.21.15.32:48742) 2024-04-24T23:00:59.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.545+0000 7f3479bb8640 1 -- 172.21.15.32:0/863367630 learned_addr learned my addr 172.21.15.32:0/863367630 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:00:59.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.546+0000 7f347a3b9640 1 --2- 172.21.15.32:0/863367630 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f34740c9970 0x7f34740c9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:59.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.546+0000 7f34793b7640 1 --2- 172.21.15.32:0/863367630 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f34740ce610 0x7f347415ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:59.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.546+0000 7f3479bb8640 1 -- 172.21.15.32:0/863367630 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f34740ce610 msgr2=0x7f347415ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:59.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.546+0000 7f3479bb8640 1 --2- 172.21.15.32:0/863367630 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f34740ce610 0x7f347415ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:59.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.546+0000 7f3479bb8640 1 -- 172.21.15.32:0/863367630 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f34740c9970 msgr2=0x7f34740c9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:59.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.546+0000 7f3479bb8640 1 --2- 172.21.15.32:0/863367630 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f34740c9970 0x7f34740c9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:59.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.546+0000 7f3479bb8640 1 -- 172.21.15.32:0/863367630 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f347415f5c0 con 0x7f3460004800 2024-04-24T23:00:59.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.546+0000 7f347a3b9640 1 --2- 172.21.15.32:0/863367630 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f34740c9970 0x7f34740c9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:00:59.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.546+0000 7f3479bb8640 1 --2- 172.21.15.32:0/863367630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3460004800 0x7f3474155120 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f34700017e0 tx=0x7f3470007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:59.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.546+0000 7f346a7fc640 1 -- 172.21.15.32:0/863367630 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3470007cf0 con 0x7f3460004800 2024-04-24T23:00:59.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.546+0000 7f346a7fc640 1 -- 172.21.15.32:0/863367630 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3470018070 con 0x7f3460004800 2024-04-24T23:00:59.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.546+0000 7f347be1c640 1 -- 172.21.15.32:0/863367630 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f347415f8a0 con 0x7f3460004800 2024-04-24T23:00:59.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.546+0000 7f346a7fc640 1 -- 172.21.15.32:0/863367630 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f347000fc20 con 0x7f3460004800 2024-04-24T23:00:59.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.546+0000 7f347be1c640 1 -- 172.21.15.32:0/863367630 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f347415fe60 con 0x7f3460004800 2024-04-24T23:00:59.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.547+0000 7f347be1c640 1 -- 172.21.15.32:0/863367630 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34740689c0 con 0x7f3460004800 2024-04-24T23:00:59.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.548+0000 7f346a7fc640 1 -- 172.21.15.32:0/863367630 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f347001d460 con 0x7f3460004800 2024-04-24T23:00:59.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.548+0000 7f346a7fc640 1 --2- 172.21.15.32:0/863367630 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f345806be30 0x7f345806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:00:59.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.548+0000 7f346a7fc640 1 -- 172.21.15.32:0/863367630 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5854+0+0 (secure 0 0 0) 0x7f347008af60 con 0x7f3460004800 2024-04-24T23:00:59.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.549+0000 7f34793b7640 1 --2- 172.21.15.32:0/863367630 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f345806be30 0x7f345806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:00:59.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.549+0000 7f346a7fc640 1 -- 172.21.15.32:0/863367630 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=56}) -- 0x7f3458073ba0 con 0x7f3460004800 2024-04-24T23:00:59.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.549+0000 7f34793b7640 1 --2- 172.21.15.32:0/863367630 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f345806be30 0x7f345806e2f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3464032000 tx=0x7f346402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:00:59.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.553+0000 7f346a7fc640 1 -- 172.21.15.32:0/863367630 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f34740689c0 con 0x7f3460004800 2024-04-24T23:00:59.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.735+0000 7f347be1c640 1 -- 172.21.15.32:0/863367630 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f3474068430 con 0x7f3460004800 2024-04-24T23:00:59.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.736+0000 7f346a7fc640 1 -- 172.21.15.32:0/863367630 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v55) ==== 81+0+35130 (secure 0 0 0) 0x7f3470059880 con 0x7f3460004800 2024-04-24T23:00:59.738 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:00:59.738 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:00:59.739 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 55, 2024-04-24T23:00:59.739 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", 2024-04-24T23:00:59.739 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2024-04-24T22:58:46.964851+0000", 2024-04-24T23:00:59.739 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2024-04-24T23:00:59.279247+0000", 2024-04-24T23:00:59.739 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2024-04-24T22:58:52.048787+0000", 2024-04-24T23:00:59.739 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2024-04-24T22:58:49.045609+0000", 2024-04-24T23:00:59.739 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-24T23:00:59.739 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2024-04-24T23:00:59.739 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2024-04-24T23:00:59.739 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2024-04-24T23:00:59.739 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2024-04-24T23:00:59.740 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2024-04-24T23:00:59.740 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2024-04-24T23:00:59.740 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:00:59.740 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 4, 2024-04-24T23:00:59.740 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.69999998807907104, 2024-04-24T23:00:59.740 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-24T23:00:59.740 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-24T23:00:59.740 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2024-04-24T23:00:59.740 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 7, 2024-04-24T23:00:59.740 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2024-04-24T23:00:59.740 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "luminous", 2024-04-24T23:00:59.740 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "luminous", 2024-04-24T23:00:59.740 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "squid", 2024-04-24T23:00:59.740 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_crimson": false, 2024-04-24T23:00:59.740 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2024-04-24T23:00:59.741 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.741 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2024-04-24T23:00:59.741 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": ".mgr", 2024-04-24T23:00:59.741 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T22:58:53.171408+0000", 2024-04-24T23:00:59.741 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 3, 2024-04-24T23:00:59.741 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,full", 2024-04-24T23:00:59.741 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:00:59.741 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:00:59.741 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:00:59.741 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:00:59.741 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:00:59.741 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:00:59.741 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:00:59.741 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:00:59.742 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:00:59.742 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:00:59.742 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2024-04-24T23:00:59.742 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2024-04-24T23:00:59.742 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2024-04-24T23:00:59.742 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2024-04-24T23:00:59.742 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2024-04-24T23:00:59.742 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:00:59.742 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:00:59.742 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:00:59.742 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:00:59.742 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:00:59.742 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:00:59.742 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:00:59.743 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.743 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "46", 2024-04-24T23:00:59.743 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:00:59.743 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:00:59.743 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:00:59.743 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:00:59.743 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:00:59.743 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:00:59.743 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:00:59.743 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:00:59.744 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:00:59.744 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:00:59.744 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:00:59.744 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:00:59.744 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:00:59.744 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:00:59.744 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:00:59.744 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:00:59.744 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:00:59.744 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:00:59.744 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:00:59.744 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:00:59.744 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:00:59.744 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:00:59.745 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:00:59.745 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:00:59.745 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:00:59.745 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:00:59.745 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.745 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:00:59.745 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:00:59.745 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:00:59.745 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:00:59.745 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:00:59.745 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:00:59.745 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:00:59.745 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:00:59.746 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:00:59.746 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:00:59.746 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:00:59.746 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:00:59.746 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_max": 32, 2024-04-24T23:00:59.746 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2024-04-24T23:00:59.746 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.746 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:00:59.746 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": {} 2024-04-24T23:00:59.746 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.746 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:00:59.746 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:00:59.746 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 8, 2024-04-24T23:00:59.746 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 8, 2024-04-24T23:00:59.747 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.25, 2024-04-24T23:00:59.747 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-04-24T23:00:59.747 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-04-24T23:00:59.747 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:00:59.747 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:00:59.747 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:00:59.747 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:59.747 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.747 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.747 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 6, 2024-04-24T23:00:59.747 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2024-04-24T23:00:59.747 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:00:30.665592+0000", 2024-04-24T23:00:59.747 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 3, 2024-04-24T23:00:59.747 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,full", 2024-04-24T23:00:59.748 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:00:59.748 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:00:59.748 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:00:59.748 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:00:59.748 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:00:59.748 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:00:59.748 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:00:59.748 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:00:59.748 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:00:59.748 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:00:59.748 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 32, 2024-04-24T23:00:59.748 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 32, 2024-04-24T23:00:59.748 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 32, 2024-04-24T23:00:59.748 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 32, 2024-04-24T23:00:59.748 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 32, 2024-04-24T23:00:59.749 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:00:59.749 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:00:59.749 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:00:59.749 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:00:59.749 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:00:59.749 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:00:59.749 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:00:59.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.749 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "46", 2024-04-24T23:00:59.749 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:00:59.749 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:00:59.749 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:00:59.749 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:00:59.749 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:00:59.750 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:00:59.750 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:00:59.750 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:00:59.750 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:00:59.750 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:00:59.750 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:00:59.750 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:00:59.750 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:00:59.750 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:00:59.750 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:00:59.750 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:00:59.750 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:00:59.750 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:00:59.751 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:00:59.751 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:00:59.751 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:00:59.751 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:00:59.751 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:00:59.751 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:00:59.751 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:00:59.751 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:00:59.751 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.751 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:00:59.751 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:00:59.751 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:00:59.751 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:00:59.751 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:00:59.752 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:00:59.752 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:00:59.752 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:00:59.752 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:00:59.752 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:00:59.752 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:00:59.752 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:00:59.752 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2024-04-24T23:00:59.752 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2024-04-24T23:00:59.752 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2024-04-24T23:00:59.752 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.752 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:00:59.752 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:00:59.752 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2024-04-24T23:00:59.753 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:59.753 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.753 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:00:59.753 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:00:59.753 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.5, 2024-04-24T23:00:59.753 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.5, 2024-04-24T23:00:59.753 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:00:59.753 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.5, 2024-04-24T23:00:59.753 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.5, 2024-04-24T23:00:59.753 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:00:59.753 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:00:59.753 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:00:59.753 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:59.753 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.753 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.754 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 7, 2024-04-24T23:00:59.754 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2024-04-24T23:00:59.754 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:00:31.357756+0000", 2024-04-24T23:00:59.754 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 3, 2024-04-24T23:00:59.754 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,full", 2024-04-24T23:00:59.754 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:00:59.754 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:00:59.754 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:00:59.754 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:00:59.754 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:00:59.754 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:00:59.754 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:00:59.755 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:00:59.755 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:00:59.755 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:00:59.755 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 64, 2024-04-24T23:00:59.755 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 64, 2024-04-24T23:00:59.755 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 64, 2024-04-24T23:00:59.755 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 64, 2024-04-24T23:00:59.755 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 64, 2024-04-24T23:00:59.755 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:00:59.755 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:00:59.755 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:00:59.755 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:00:59.755 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:00:59.755 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:00:59.756 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:00:59.756 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.756 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "46", 2024-04-24T23:00:59.756 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:00:59.756 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:00:59.756 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:00:59.756 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:00:59.756 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:00:59.756 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:00:59.756 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:00:59.756 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:00:59.756 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:00:59.757 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:00:59.757 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:00:59.757 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:00:59.757 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:00:59.757 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:00:59.757 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:00:59.757 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:00:59.757 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:00:59.757 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:00:59.757 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:00:59.757 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:00:59.757 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:00:59.757 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:00:59.757 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:00:59.757 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:00:59.758 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:00:59.758 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:00:59.758 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.758 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:00:59.758 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:00:59.758 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:00:59.758 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:00:59.758 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:00:59.758 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:00:59.758 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:00:59.758 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:00:59.758 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:00:59.758 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:00:59.758 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:00:59.759 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:00:59.759 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 64, 2024-04-24T23:00:59.759 INFO:teuthology.orchestra.run.smithi032.stdout: "target_size_ratio": 0.89999997615814209 2024-04-24T23:00:59.759 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.759 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:00:59.759 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:00:59.759 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2024-04-24T23:00:59.759 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:59.759 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.759 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:00:59.759 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:00:59.759 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.3799999952316284, 2024-04-24T23:00:59.759 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.3799999952316284, 2024-04-24T23:00:59.760 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:00:59.760 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.3799999952316284, 2024-04-24T23:00:59.760 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.3799999952316284, 2024-04-24T23:00:59.760 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:00:59.760 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:00:59.760 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:00:59.760 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:59.760 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:59.760 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:00:59.760 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-04-24T23:00:59.760 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.760 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:00:59.760 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:00:59.760 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:00:59.760 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:00:59.761 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:00:59.761 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:00:59.761 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:00:59.761 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:00:59.761 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:00:59.761 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 49, 2024-04-24T23:00:59.761 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:00:59.761 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:00:59.761 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:00:59.761 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:59.761 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.761 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:59.761 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2024-04-24T23:00:59.762 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:00:59.762 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.762 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.762 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:59.762 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2024-04-24T23:00:59.762 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:00:59.762 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:59.762 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:59.762 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.762 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:00:59.762 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:59.762 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.762 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:59.762 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2024-04-24T23:00:59.763 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:00:59.763 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.763 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.763 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:59.763 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2024-04-24T23:00:59.763 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:00:59.763 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:59.763 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:59.763 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.763 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:00:59.763 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:59.763 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.763 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:59.763 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2024-04-24T23:00:59.764 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:00:59.764 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.764 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.764 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:59.764 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2024-04-24T23:00:59.764 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:00:59.764 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:59.764 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:59.764 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.764 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:00:59.764 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:59.764 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.764 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:59.764 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2024-04-24T23:00:59.764 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:00:59.765 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.765 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.765 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:59.765 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2024-04-24T23:00:59.765 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:00:59.765 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:59.765 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:59.765 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.765 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/1479395426", 2024-04-24T23:00:59.765 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/1479395426", 2024-04-24T23:00:59.765 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/1479395426", 2024-04-24T23:00:59.765 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/1479395426", 2024-04-24T23:00:59.765 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:00:59.765 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:00:59.766 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:00:59.766 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:59.766 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.766 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.766 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:00:59.766 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:00:59.766 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:00:59.766 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:00:59.766 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:00:59.766 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:00:59.766 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:00:59.766 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:00:59.766 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:00:59.766 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 49, 2024-04-24T23:00:59.767 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:00:59.767 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:00:59.767 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:00:59.767 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:59.767 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.767 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:59.767 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2024-04-24T23:00:59.767 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:00:59.767 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.767 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.767 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:59.767 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2024-04-24T23:00:59.767 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:00:59.767 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:59.768 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:59.768 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.768 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:00:59.768 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:59.768 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.768 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:59.768 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2024-04-24T23:00:59.768 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:00:59.768 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.768 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.768 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:59.768 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2024-04-24T23:00:59.769 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:00:59.769 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:59.769 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:59.769 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.769 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:00:59.769 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:59.769 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.769 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:59.769 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2024-04-24T23:00:59.769 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:00:59.769 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.769 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.769 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:59.769 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2024-04-24T23:00:59.769 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:00:59.770 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:59.770 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:59.770 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.770 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:00:59.770 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:59.770 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.770 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:59.770 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2024-04-24T23:00:59.770 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:00:59.770 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.770 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.770 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:59.770 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2024-04-24T23:00:59.771 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:00:59.771 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:59.771 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:59.771 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.771 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/1340633718", 2024-04-24T23:00:59.771 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/1340633718", 2024-04-24T23:00:59.771 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/1340633718", 2024-04-24T23:00:59.771 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/1340633718", 2024-04-24T23:00:59.771 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:00:59.771 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:00:59.771 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:00:59.771 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:59.771 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.771 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.772 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:00:59.772 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:00:59.772 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:00:59.772 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:00:59.772 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:00:59.772 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:00:59.772 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:00:59.772 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:00:59.772 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:00:59.772 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 49, 2024-04-24T23:00:59.772 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:00:59.772 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:00:59.772 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:00:59.772 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:59.772 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.773 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:59.773 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2024-04-24T23:00:59.773 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:00:59.773 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.773 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.773 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:59.773 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2024-04-24T23:00:59.773 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:00:59.773 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:59.773 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:59.773 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.773 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:00:59.773 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:59.773 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.774 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:59.774 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2024-04-24T23:00:59.774 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:00:59.774 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.774 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.774 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:59.774 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2024-04-24T23:00:59.774 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:00:59.774 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:59.774 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:59.774 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.774 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:00:59.774 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:59.774 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.775 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:59.775 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2024-04-24T23:00:59.775 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:00:59.775 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.775 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.775 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:59.775 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2024-04-24T23:00:59.775 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:00:59.775 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:59.775 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:59.775 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.775 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:00:59.775 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:59.775 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.775 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:59.776 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2024-04-24T23:00:59.776 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:00:59.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.776 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.776 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:59.776 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2024-04-24T23:00:59.776 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:00:59.776 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:59.776 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:59.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.776 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/3405899334", 2024-04-24T23:00:59.776 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6820/3405899334", 2024-04-24T23:00:59.776 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6828/3405899334", 2024-04-24T23:00:59.776 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6824/3405899334", 2024-04-24T23:00:59.776 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:00:59.777 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:00:59.777 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:00:59.777 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:59.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.777 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.777 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:00:59.777 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:00:59.777 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:00:59.777 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:00:59.777 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:00:59.777 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:00:59.777 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:00:59.777 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:00:59.777 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:00:59.777 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 43, 2024-04-24T23:00:59.778 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:00:59.778 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:00:59.778 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:00:59.778 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:59.778 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.778 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:59.778 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2024-04-24T23:00:59.778 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:00:59.778 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.778 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.778 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:59.778 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2024-04-24T23:00:59.778 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:00:59.778 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:59.779 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:59.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.779 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:00:59.779 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:59.779 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.779 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:59.779 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2024-04-24T23:00:59.779 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:00:59.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.779 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.779 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:59.779 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2024-04-24T23:00:59.779 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:00:59.780 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:59.780 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:59.780 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.780 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:00:59.780 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:59.780 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.780 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:59.780 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2024-04-24T23:00:59.780 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:00:59.780 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.780 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.780 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:59.780 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2024-04-24T23:00:59.781 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:00:59.781 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:59.781 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:59.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.781 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:00:59.781 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:59.781 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.781 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:59.781 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2024-04-24T23:00:59.781 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:00:59.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.781 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.781 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:59.782 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2024-04-24T23:00:59.782 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:00:59.782 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:59.782 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:59.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.782 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6821/3381057190", 2024-04-24T23:00:59.782 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6825/3381057190", 2024-04-24T23:00:59.782 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/3381057190", 2024-04-24T23:00:59.782 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/3381057190", 2024-04-24T23:00:59.782 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:00:59.782 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:00:59.782 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:00:59.782 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:59.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.783 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.783 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:00:59.783 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:00:59.783 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:00:59.783 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:00:59.783 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:00:59.783 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:00:59.783 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:00:59.783 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:00:59.783 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:00:59.783 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 43, 2024-04-24T23:00:59.784 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:00:59.784 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:00:59.784 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:00:59.784 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:59.784 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.784 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:59.784 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6800", 2024-04-24T23:00:59.784 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:00:59.784 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.784 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.784 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:59.784 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6801", 2024-04-24T23:00:59.784 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:00:59.784 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:59.785 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:59.785 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.785 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:00:59.785 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:59.785 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.785 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:59.785 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6802", 2024-04-24T23:00:59.785 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:00:59.785 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.785 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.785 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:59.785 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6803", 2024-04-24T23:00:59.785 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:00:59.785 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:59.786 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:59.786 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.786 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:00:59.786 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:59.786 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.786 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:59.786 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6806", 2024-04-24T23:00:59.786 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:00:59.786 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.786 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.786 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:59.786 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6807", 2024-04-24T23:00:59.786 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:00:59.786 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:59.787 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:59.787 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.787 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:00:59.787 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:59.787 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.787 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:59.787 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6804", 2024-04-24T23:00:59.787 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:00:59.787 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.787 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.787 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:59.787 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6805", 2024-04-24T23:00:59.787 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:00:59.787 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:59.788 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:59.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.788 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6801/4010553979", 2024-04-24T23:00:59.788 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6803/4010553979", 2024-04-24T23:00:59.788 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6807/4010553979", 2024-04-24T23:00:59.788 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6805/4010553979", 2024-04-24T23:00:59.788 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:00:59.788 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:00:59.788 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:00:59.788 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:59.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.788 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.788 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:00:59.788 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:00:59.789 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:00:59.789 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:00:59.789 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:00:59.789 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:00:59.789 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:00:59.789 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:00:59.789 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:00:59.789 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 43, 2024-04-24T23:00:59.789 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:00:59.789 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:00:59.789 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:00:59.789 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:59.789 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.789 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:59.790 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6808", 2024-04-24T23:00:59.790 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:00:59.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.790 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.790 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:59.790 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6809", 2024-04-24T23:00:59.790 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:00:59.790 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:59.790 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:59.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.790 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:00:59.790 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:59.790 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.790 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:59.791 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6810", 2024-04-24T23:00:59.791 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:00:59.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.791 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.791 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:59.791 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6811", 2024-04-24T23:00:59.791 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:00:59.791 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:59.791 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:59.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.791 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:00:59.791 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:59.791 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.791 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:59.791 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6814", 2024-04-24T23:00:59.792 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:00:59.792 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.792 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.792 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:59.792 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6815", 2024-04-24T23:00:59.792 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:00:59.792 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:59.792 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:59.792 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.792 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:00:59.792 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:59.792 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.792 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:59.792 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6812", 2024-04-24T23:00:59.793 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:00:59.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.793 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.793 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:59.793 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6813", 2024-04-24T23:00:59.793 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:00:59.793 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:59.793 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:59.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.793 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6809/3794084091", 2024-04-24T23:00:59.793 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6811/3794084091", 2024-04-24T23:00:59.793 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6815/3794084091", 2024-04-24T23:00:59.793 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6813/3794084091", 2024-04-24T23:00:59.794 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:00:59.794 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:00:59.794 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:00:59.794 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:59.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.794 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.794 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:00:59.794 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:00:59.794 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:00:59.794 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:00:59.794 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:00:59.794 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:00:59.794 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:00:59.795 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:00:59.795 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:00:59.795 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 43, 2024-04-24T23:00:59.795 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:00:59.795 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:00:59.795 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:00:59.795 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:59.795 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.795 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:59.795 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6823", 2024-04-24T23:00:59.795 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:00:59.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.795 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.795 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:59.795 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6825", 2024-04-24T23:00:59.796 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:00:59.796 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:59.796 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:59.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.796 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:00:59.796 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:59.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.796 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:59.796 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6826", 2024-04-24T23:00:59.796 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:00:59.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.796 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:59.796 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6827", 2024-04-24T23:00:59.797 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:00:59.797 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:59.797 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:59.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.797 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:00:59.797 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:59.797 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.797 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:59.797 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6830", 2024-04-24T23:00:59.797 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:00:59.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.798 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:59.798 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6831", 2024-04-24T23:00:59.798 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:00:59.798 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:59.798 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:59.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.798 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:00:59.798 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:59.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.798 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:59.798 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6828", 2024-04-24T23:00:59.798 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:00:59.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.799 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:59.799 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6829", 2024-04-24T23:00:59.799 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:00:59.799 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:59.799 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:59.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.799 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6825/1348640258", 2024-04-24T23:00:59.799 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6827/1348640258", 2024-04-24T23:00:59.799 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6831/1348640258", 2024-04-24T23:00:59.799 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6829/1348640258", 2024-04-24T23:00:59.799 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:00:59.799 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:00:59.799 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:00:59.799 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:59.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.800 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:00:59.800 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:00:59.800 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:00:59.800 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:00:59.800 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:00:59.800 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:00:59.800 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:00:59.800 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:00:59.800 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:00:59.800 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 49, 2024-04-24T23:00:59.800 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:00:59.800 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:00:59.800 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:00:59.801 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:59.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.801 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:59.801 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6816", 2024-04-24T23:00:59.801 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:00:59.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.801 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:59.801 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6817", 2024-04-24T23:00:59.801 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:00:59.801 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:59.801 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:59.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.801 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:00:59.801 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:59.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.802 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:59.802 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6818", 2024-04-24T23:00:59.802 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:00:59.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.802 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:59.802 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6819", 2024-04-24T23:00:59.802 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:00:59.802 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:59.802 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:59.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.802 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:00:59.802 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:59.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.803 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:59.803 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6822", 2024-04-24T23:00:59.803 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:00:59.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.803 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:59.803 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6824", 2024-04-24T23:00:59.803 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:00:59.803 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:59.803 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:59.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.803 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:00:59.803 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:00:59.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.804 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:00:59.804 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6820", 2024-04-24T23:00:59.804 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:00:59.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.804 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:00:59.804 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6821", 2024-04-24T23:00:59.804 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:00:59.804 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:59.804 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:59.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.804 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6817/3250005122", 2024-04-24T23:00:59.804 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6819/3250005122", 2024-04-24T23:00:59.805 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6824/3250005122", 2024-04-24T23:00:59.805 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6821/3250005122", 2024-04-24T23:00:59.805 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:00:59.805 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:00:59.805 INFO:teuthology.orchestra.run.smithi032.stdout: "full", 2024-04-24T23:00:59.805 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:00:59.805 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:59.805 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:59.805 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:00:59.805 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2024-04-24T23:00:59.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.805 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:00:59.805 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:00:59.805 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:00:59.806 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:00:59.806 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:00:59.806 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:00:59.806 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.156184+0000", 2024-04-24T23:00:59.806 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:00:59.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.806 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:00:59.806 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:00:59.806 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:00:59.806 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:00:59.806 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:00:59.806 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:00:59.807 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240774+0000", 2024-04-24T23:00:59.807 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:00:59.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.807 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:00:59.807 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:00:59.807 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:00:59.807 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:00:59.807 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:00:59.807 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:00:59.807 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240949+0000", 2024-04-24T23:00:59.807 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:00:59.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.807 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:00:59.808 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:00:59.808 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:00:59.808 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:00:59.808 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:00:59.808 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:00:59.808 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.171672+0000", 2024-04-24T23:00:59.808 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:00:59.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.808 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:00:59.808 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:00:59.808 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:00:59.809 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:00:59.809 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:00:59.809 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:00:59.809 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.220379+0000", 2024-04-24T23:00:59.809 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:00:59.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.809 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:00:59.809 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:00:59.809 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:00:59.809 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:00:59.809 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:00:59.809 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:00:59.809 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.204963+0000", 2024-04-24T23:00:59.810 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:00:59.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.810 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:00:59.810 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:00:59.810 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:00:59.810 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:00:59.810 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:00:59.810 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:00:59.810 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.239748+0000", 2024-04-24T23:00:59.810 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:00:59.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.811 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:00:59.811 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:00:59.811 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:00:59.811 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:00:59.811 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:00:59.811 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:00:59.811 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.154104+0000", 2024-04-24T23:00:59.811 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:00:59.811 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:59.811 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:00:59.811 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2024-04-24T23:00:59.811 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [ 2024-04-24T23:00:59.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.813 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "7.6", 2024-04-24T23:00:59.813 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:00:59.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.813 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 5, 2024-04-24T23:00:59.813 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 7 2024-04-24T23:00:59.813 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:59.813 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:59.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.814 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "7.c", 2024-04-24T23:00:59.814 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:00:59.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.814 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 5, 2024-04-24T23:00:59.814 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 7 2024-04-24T23:00:59.814 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:59.814 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:59.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.814 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "7.12", 2024-04-24T23:00:59.814 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:00:59.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.814 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 5, 2024-04-24T23:00:59.814 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 7 2024-04-24T23:00:59.815 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:59.815 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:59.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.815 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "7.31", 2024-04-24T23:00:59.815 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:00:59.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.815 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 5, 2024-04-24T23:00:59.815 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 7 2024-04-24T23:00:59.815 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:59.815 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:59.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.815 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "7.35", 2024-04-24T23:00:59.815 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:00:59.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:00:59.816 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 5, 2024-04-24T23:00:59.816 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 0 2024-04-24T23:00:59.816 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:59.816 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:00:59.816 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:59.816 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:00:59.816 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_primaries": [], 2024-04-24T23:00:59.816 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2024-04-24T23:00:59.816 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2024-04-24T23:00:59.816 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklist": {}, 2024-04-24T23:00:59.816 INFO:teuthology.orchestra.run.smithi032.stdout: "range_blocklist": {}, 2024-04-24T23:00:59.816 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2024-04-24T23:00:59.816 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2024-04-24T23:00:59.816 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2024-04-24T23:00:59.817 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2024-04-24T23:00:59.817 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2024-04-24T23:00:59.817 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2024-04-24T23:00:59.817 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2024-04-24T23:00:59.817 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:59.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:00:59.817 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2024-04-24T23:00:59.817 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2024-04-24T23:00:59.817 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2024-04-24T23:00:59.817 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2024-04-24T23:00:59.817 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {}, 2024-04-24T23:00:59.817 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode": { 2024-04-24T23:00:59.817 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_enabled": false, 2024-04-24T23:00:59.817 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_bucket_count": 0, 2024-04-24T23:00:59.817 INFO:teuthology.orchestra.run.smithi032.stdout: "degraded_stretch_mode": 0, 2024-04-24T23:00:59.818 INFO:teuthology.orchestra.run.smithi032.stdout: "recovering_stretch_mode": 0, 2024-04-24T23:00:59.818 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_bucket": 0 2024-04-24T23:00:59.818 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:00:59.818 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:00:59.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.738+0000 7f347be1c640 1 -- 172.21.15.32:0/863367630 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f345806be30 msgr2=0x7f345806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:59.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.738+0000 7f347be1c640 1 --2- 172.21.15.32:0/863367630 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f345806be30 0x7f345806e2f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3464032000 tx=0x7f346402e000 comp rx=0 tx=0).stop 2024-04-24T23:00:59.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.738+0000 7f347be1c640 1 -- 172.21.15.32:0/863367630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3460004800 msgr2=0x7f3474155120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:00:59.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.738+0000 7f347be1c640 1 --2- 172.21.15.32:0/863367630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3460004800 0x7f3474155120 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f34700017e0 tx=0x7f3470007720 comp rx=0 tx=0).stop 2024-04-24T23:00:59.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.738+0000 7f347be1c640 1 -- 172.21.15.32:0/863367630 shutdown_connections 2024-04-24T23:00:59.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.738+0000 7f347be1c640 1 --2- 172.21.15.32:0/863367630 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f345806be30 0x7f345806e2f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:59.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.738+0000 7f347be1c640 1 --2- 172.21.15.32:0/863367630 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f34740c9970 0x7f34740c9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:59.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.738+0000 7f347be1c640 1 --2- 172.21.15.32:0/863367630 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f34740ce610 0x7f347415ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:59.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.738+0000 7f347be1c640 1 --2- 172.21.15.32:0/863367630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3460004800 0x7f3474155120 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:00:59.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.738+0000 7f347be1c640 1 -- 172.21.15.32:0/863367630 >> 172.21.15.32:0/863367630 conn(0x7f34740d11d0 msgr2=0x7f347415cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:00:59.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.738+0000 7f347be1c640 1 -- 172.21.15.32:0/863367630 shutdown_connections 2024-04-24T23:00:59.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:00:59.738+0000 7f347be1c640 1 -- 172.21.15.32:0/863367630 wait complete. 2024-04-24T23:00:59.821 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-04-24T23:01:04.822 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-24T23:01:04.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.934+0000 7f3743989640 1 Processor -- start 2024-04-24T23:01:04.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.934+0000 7f3743989640 1 -- start start 2024-04-24T23:01:04.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.935+0000 7f3743989640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f373c0b44d0 0x7f373c0b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:04.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.935+0000 7f3743989640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f373c0b6430 con 0x7f373c0af890 2024-04-24T23:01:04.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.935+0000 7f3743989640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f373c156f80 con 0x7f373c06c5f0 2024-04-24T23:01:04.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.935+0000 7f3743989640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f373c0b6150 con 0x7f373c0b44d0 2024-04-24T23:01:04.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.935+0000 7f3741f26640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f373c0af890 0x7f373c15cac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56752/0 (socket says 172.21.15.32:56752) 2024-04-24T23:01:04.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.935+0000 7f3741f26640 1 -- 172.21.15.32:0/906985320 learned_addr learned my addr 172.21.15.32:0/906985320 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:01:04.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.935+0000 7f3740f24640 1 --2- 172.21.15.32:0/906985320 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f373c0b44d0 0x7f373c0b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:04.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.936+0000 7f373bfff640 1 -- 172.21.15.32:0/906985320 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3657688036 0 0) 0x7f373c0b6430 con 0x7f373c0af890 2024-04-24T23:01:04.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.936+0000 7f373bfff640 1 -- 172.21.15.32:0/906985320 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3724003520 con 0x7f373c0af890 2024-04-24T23:01:04.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.936+0000 7f373bfff640 1 -- 172.21.15.32:0/906985320 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 582621009 0 0) 0x7f3724003520 con 0x7f373c0af890 2024-04-24T23:01:04.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.936+0000 7f373bfff640 1 -- 172.21.15.32:0/906985320 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f373c0b44d0 msgr2=0x7f373c0b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:04.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.936+0000 7f373bfff640 1 --2- 172.21.15.32:0/906985320 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f373c0b44d0 0x7f373c0b5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:04.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.936+0000 7f373bfff640 1 -- 172.21.15.32:0/906985320 >> v1:172.21.15.121:6789/0 conn(0x7f373c06c5f0 legacy=0x7f373c06c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:04.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.936+0000 7f373bfff640 1 -- 172.21.15.32:0/906985320 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f373c0b39f0 con 0x7f373c0af890 2024-04-24T23:01:04.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.936+0000 7f373bfff640 1 -- 172.21.15.32:0/906985320 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f372c0019f0 con 0x7f373c0af890 2024-04-24T23:01:04.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.936+0000 7f373bfff640 1 -- 172.21.15.32:0/906985320 >> v1:172.21.15.32:6789/0 conn(0x7f373c0af890 legacy=0x7f373c15cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:04.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.936+0000 7f373bfff640 1 --2- 172.21.15.32:0/906985320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37240039e0 0x7f3724003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:04.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.936+0000 7f373bfff640 1 --2- 172.21.15.32:0/906985320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3724004800 0x7f3724004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:04.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.936+0000 7f373bfff640 1 --2- 172.21.15.32:0/906985320 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f373c0b44d0 0x7f3724005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:04.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.936+0000 7f373bfff640 1 -- 172.21.15.32:0/906985320 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f373c0b6150 con 0x7f37240039e0 2024-04-24T23:01:04.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.936+0000 7f373bfff640 1 -- 172.21.15.32:0/906985320 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f372400baf0 con 0x7f3724004800 2024-04-24T23:01:04.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.936+0000 7f373bfff640 1 -- 172.21.15.32:0/906985320 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f372400bcf0 con 0x7f373c0b44d0 2024-04-24T23:01:04.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.936+0000 7f3741725640 1 --2- 172.21.15.32:0/906985320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37240039e0 0x7f3724003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:04.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.936+0000 7f3740f24640 1 --2- 172.21.15.32:0/906985320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3724004800 0x7f3724004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:04.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.936+0000 7f3741f26640 1 --2- 172.21.15.32:0/906985320 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f373c0b44d0 0x7f3724005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:04.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.936+0000 7f3741725640 1 -- 172.21.15.32:0/906985320 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f373c0b44d0 msgr2=0x7f3724005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:04.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.936+0000 7f3741725640 1 --2- 172.21.15.32:0/906985320 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f373c0b44d0 0x7f3724005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:04.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.936+0000 7f3741725640 1 -- 172.21.15.32:0/906985320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3724004800 msgr2=0x7f3724004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:04.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.936+0000 7f3741725640 1 --2- 172.21.15.32:0/906985320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3724004800 0x7f3724004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:04.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.936+0000 7f3741725640 1 -- 172.21.15.32:0/906985320 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3724003520 con 0x7f37240039e0 2024-04-24T23:01:04.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.936+0000 7f3741f26640 1 --2- 172.21.15.32:0/906985320 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f373c0b44d0 0x7f3724005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:01:04.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.937+0000 7f3741725640 1 --2- 172.21.15.32:0/906985320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37240039e0 0x7f3724003e00 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f3734001720 tx=0x7f3724005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df00f414f330c24e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:04.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.937+0000 7f373bfff640 1 -- 172.21.15.32:0/906985320 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3734001370 con 0x7f37240039e0 2024-04-24T23:01:04.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.937+0000 7f373bfff640 1 -- 172.21.15.32:0/906985320 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37340052d0 con 0x7f37240039e0 2024-04-24T23:01:04.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.937+0000 7f373bfff640 1 -- 172.21.15.32:0/906985320 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373402abc0 con 0x7f37240039e0 2024-04-24T23:01:04.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.938+0000 7f3743989640 1 -- 172.21.15.32:0/906985320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37240039e0 msgr2=0x7f3724003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:04.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.938+0000 7f3743989640 1 --2- 172.21.15.32:0/906985320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37240039e0 0x7f3724003e00 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f3734001720 tx=0x7f3724005830 comp rx=0 tx=0).stop 2024-04-24T23:01:04.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.938+0000 7f3741f26640 1 -- 172.21.15.32:0/906985320 reap_dead start 2024-04-24T23:01:04.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.938+0000 7f3743989640 1 -- 172.21.15.32:0/906985320 shutdown_connections 2024-04-24T23:01:04.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.938+0000 7f3743989640 1 -- 172.21.15.32:0/906985320 >> 172.21.15.32:0/906985320 conn(0x7f373c06b110 msgr2=0x7f373c06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:04.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.938+0000 7f3743989640 1 -- 172.21.15.32:0/906985320 shutdown_connections 2024-04-24T23:01:04.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.938+0000 7f3743989640 1 -- 172.21.15.32:0/906985320 wait complete. 2024-04-24T23:01:04.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.939+0000 7f3743989640 1 Processor -- start 2024-04-24T23:01:04.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.939+0000 7f3743989640 1 -- start start 2024-04-24T23:01:04.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.940+0000 7f3743989640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37240039e0 0x7f373c155400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:04.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.940+0000 7f3743989640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f373c155960 0x7f373c1f2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:04.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.940+0000 7f3743989640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f373c1f63a0 0x7f373c1e6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:04.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.940+0000 7f3743989640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f373c0b5890 con 0x7f373c1f63a0 2024-04-24T23:01:04.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.940+0000 7f3743989640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f373c0b3fb0 con 0x7f37240039e0 2024-04-24T23:01:04.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.940+0000 7f3743989640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f373c0b5a10 con 0x7f373c155960 2024-04-24T23:01:04.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.940+0000 7f3741f26640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f373c1f63a0 0x7f373c1e6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:04.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.940+0000 7f3741f26640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f373c1f63a0 0x7f373c1e6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50122/0 (socket says 172.21.15.32:50122) 2024-04-24T23:01:04.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.940+0000 7f3741f26640 1 -- 172.21.15.32:0/3030385874 learned_addr learned my addr 172.21.15.32:0/3030385874 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:04.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.940+0000 7f3740f24640 1 --2- 172.21.15.32:0/3030385874 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f373c155960 0x7f373c1f2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:04.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.940+0000 7f3741725640 1 --2- 172.21.15.32:0/3030385874 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37240039e0 0x7f373c155400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:04.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.940+0000 7f3741f26640 1 -- 172.21.15.32:0/3030385874 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f373c155960 msgr2=0x7f373c1f2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:04.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.940+0000 7f3741f26640 1 --2- 172.21.15.32:0/3030385874 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f373c155960 0x7f373c1f2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:04.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.940+0000 7f3741f26640 1 -- 172.21.15.32:0/3030385874 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37240039e0 msgr2=0x7f373c155400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:04.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.940+0000 7f3741f26640 1 --2- 172.21.15.32:0/3030385874 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37240039e0 0x7f373c155400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:04.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.940+0000 7f3741f26640 1 -- 172.21.15.32:0/3030385874 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f373c072030 con 0x7f373c1f63a0 2024-04-24T23:01:04.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.940+0000 7f3741f26640 1 --2- 172.21.15.32:0/3030385874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f373c1f63a0 0x7f373c1e6c60 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f373c073960 tx=0x7f372c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:04.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.941+0000 7f3739ffb640 1 -- 172.21.15.32:0/3030385874 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372c00ebb0 con 0x7f373c1f63a0 2024-04-24T23:01:04.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.941+0000 7f3739ffb640 1 -- 172.21.15.32:0/3030385874 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f372c00ed50 con 0x7f373c1f63a0 2024-04-24T23:01:04.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.941+0000 7f3743989640 1 -- 172.21.15.32:0/3030385874 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f373c072310 con 0x7f373c1f63a0 2024-04-24T23:01:04.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.941+0000 7f3739ffb640 1 -- 172.21.15.32:0/3030385874 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372c014650 con 0x7f373c1f63a0 2024-04-24T23:01:04.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.941+0000 7f3743989640 1 -- 172.21.15.32:0/3030385874 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f373c1f6820 con 0x7f373c1f63a0 2024-04-24T23:01:04.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.942+0000 7f3743989640 1 -- 172.21.15.32:0/3030385874 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f373c068430 con 0x7f373c1f63a0 2024-04-24T23:01:04.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.942+0000 7f3739ffb640 1 -- 172.21.15.32:0/3030385874 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f372c0147f0 con 0x7f373c1f63a0 2024-04-24T23:01:04.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.943+0000 7f3739ffb640 1 --2- 172.21.15.32:0/3030385874 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f372806be60 0x7f372806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:04.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.943+0000 7f3739ffb640 1 -- 172.21.15.32:0/3030385874 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5854+0+0 (secure 0 0 0) 0x7f372c08aa90 con 0x7f373c1f63a0 2024-04-24T23:01:04.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.943+0000 7f3741725640 1 --2- 172.21.15.32:0/3030385874 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f372806be60 0x7f372806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:04.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.943+0000 7f3741725640 1 --2- 172.21.15.32:0/3030385874 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f372806be60 0x7f372806e320 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3734032000 tx=0x7f373402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:04.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:04.947+0000 7f3739ffb640 1 -- 172.21.15.32:0/3030385874 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f372c058df0 con 0x7f373c1f63a0 2024-04-24T23:01:05.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.133+0000 7f3743989640 1 -- 172.21.15.32:0/3030385874 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f373c0b5670 con 0x7f373c1f63a0 2024-04-24T23:01:05.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.134+0000 7f3739ffb640 1 -- 172.21.15.32:0/3030385874 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v56) ==== 81+0+35091 (secure 0 0 0) 0x7f372c05d0f0 con 0x7f373c1f63a0 2024-04-24T23:01:05.136 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:01:05.136 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:01:05.136 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 56, 2024-04-24T23:01:05.137 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", 2024-04-24T23:01:05.137 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2024-04-24T22:58:46.964851+0000", 2024-04-24T23:01:05.137 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2024-04-24T23:01:00.283572+0000", 2024-04-24T23:01:05.137 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2024-04-24T22:58:52.048787+0000", 2024-04-24T23:01:05.137 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2024-04-24T22:58:49.045609+0000", 2024-04-24T23:01:05.137 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-24T23:01:05.137 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2024-04-24T23:01:05.137 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2024-04-24T23:01:05.137 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2024-04-24T23:01:05.137 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2024-04-24T23:01:05.137 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2024-04-24T23:01:05.137 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2024-04-24T23:01:05.138 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:01:05.138 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 4, 2024-04-24T23:01:05.138 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.69999998807907104, 2024-04-24T23:01:05.138 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-24T23:01:05.138 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-24T23:01:05.138 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2024-04-24T23:01:05.138 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 7, 2024-04-24T23:01:05.138 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2024-04-24T23:01:05.138 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "luminous", 2024-04-24T23:01:05.138 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "luminous", 2024-04-24T23:01:05.138 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "squid", 2024-04-24T23:01:05.138 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_crimson": false, 2024-04-24T23:01:05.138 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2024-04-24T23:01:05.138 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.139 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2024-04-24T23:01:05.139 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": ".mgr", 2024-04-24T23:01:05.139 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T22:58:53.171408+0000", 2024-04-24T23:01:05.139 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:01:05.139 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:01:05.139 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:01:05.139 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:01:05.139 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:01:05.139 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:01:05.139 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:01:05.139 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:01:05.139 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:01:05.139 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:01:05.139 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:01:05.139 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:01:05.140 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2024-04-24T23:01:05.140 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2024-04-24T23:01:05.140 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2024-04-24T23:01:05.140 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2024-04-24T23:01:05.140 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2024-04-24T23:01:05.140 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:01:05.140 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:01:05.140 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:01:05.140 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:01:05.140 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:01:05.140 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:01:05.140 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:01:05.140 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.140 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "56", 2024-04-24T23:01:05.140 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:01:05.141 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:01:05.141 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:01:05.141 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:01:05.141 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:01:05.141 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:01:05.141 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:01:05.141 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:01:05.141 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:01:05.141 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:01:05.141 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:01:05.141 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:01:05.141 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:01:05.141 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:01:05.141 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:01:05.142 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:01:05.142 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:01:05.142 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:01:05.142 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:01:05.142 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:01:05.142 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:01:05.142 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:01:05.142 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:01:05.142 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:01:05.142 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:01:05.142 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:01:05.142 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.142 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:01:05.142 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:01:05.143 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:01:05.143 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:01:05.143 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:01:05.143 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:01:05.143 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:01:05.143 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:01:05.143 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:01:05.143 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:01:05.143 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:01:05.145 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:01:05.145 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_max": 32, 2024-04-24T23:01:05.145 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2024-04-24T23:01:05.145 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.145 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:01:05.145 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": {} 2024-04-24T23:01:05.145 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.145 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:01:05.146 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:01:05.146 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 8, 2024-04-24T23:01:05.146 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 8, 2024-04-24T23:01:05.146 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.25, 2024-04-24T23:01:05.146 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-04-24T23:01:05.146 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-04-24T23:01:05.146 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:01:05.146 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:01:05.146 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:01:05.146 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.146 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.146 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.146 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 6, 2024-04-24T23:01:05.146 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2024-04-24T23:01:05.146 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:00:30.665592+0000", 2024-04-24T23:01:05.147 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:01:05.147 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:01:05.147 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:01:05.147 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:01:05.147 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:01:05.147 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:01:05.147 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:01:05.147 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:01:05.147 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:01:05.148 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:01:05.148 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:01:05.148 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:01:05.148 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 32, 2024-04-24T23:01:05.148 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 32, 2024-04-24T23:01:05.148 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 32, 2024-04-24T23:01:05.148 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 32, 2024-04-24T23:01:05.148 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 32, 2024-04-24T23:01:05.149 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:01:05.149 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:01:05.149 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:01:05.149 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:01:05.149 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:01:05.149 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:01:05.149 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:01:05.149 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.149 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "56", 2024-04-24T23:01:05.149 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:01:05.149 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:01:05.149 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:01:05.149 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:01:05.149 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:01:05.149 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:01:05.150 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:01:05.150 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:01:05.150 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:01:05.150 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:01:05.150 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:01:05.150 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:01:05.150 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:01:05.150 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:01:05.150 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:01:05.150 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:01:05.150 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:01:05.150 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:01:05.150 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:01:05.150 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:01:05.151 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:01:05.151 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:01:05.151 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:01:05.151 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:01:05.151 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:01:05.151 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:01:05.151 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.151 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:01:05.151 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:01:05.151 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:01:05.151 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:01:05.151 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:01:05.151 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:01:05.151 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:01:05.151 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:01:05.152 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:01:05.152 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:01:05.152 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:01:05.152 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:01:05.152 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2024-04-24T23:01:05.152 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2024-04-24T23:01:05.152 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2024-04-24T23:01:05.152 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.152 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:01:05.152 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:01:05.152 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2024-04-24T23:01:05.152 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.152 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.152 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:01:05.152 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:01:05.153 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.5, 2024-04-24T23:01:05.153 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.5, 2024-04-24T23:01:05.153 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:01:05.153 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.5, 2024-04-24T23:01:05.153 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.5, 2024-04-24T23:01:05.153 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:01:05.153 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:01:05.153 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:01:05.153 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.153 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.153 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.153 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 7, 2024-04-24T23:01:05.153 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2024-04-24T23:01:05.153 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:00:31.357756+0000", 2024-04-24T23:01:05.154 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:01:05.154 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:01:05.154 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:01:05.154 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:01:05.154 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:01:05.154 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:01:05.154 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:01:05.154 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:01:05.154 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:01:05.154 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:01:05.154 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:01:05.154 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:01:05.154 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 64, 2024-04-24T23:01:05.154 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 64, 2024-04-24T23:01:05.155 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 64, 2024-04-24T23:01:05.155 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 64, 2024-04-24T23:01:05.155 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 64, 2024-04-24T23:01:05.155 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:01:05.155 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:01:05.155 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:01:05.155 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:01:05.155 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:01:05.155 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:01:05.155 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:01:05.155 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.155 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "56", 2024-04-24T23:01:05.155 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:01:05.156 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:01:05.156 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:01:05.156 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:01:05.156 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:01:05.156 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:01:05.156 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:01:05.156 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:01:05.156 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:01:05.156 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:01:05.156 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:01:05.156 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:01:05.156 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:01:05.156 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:01:05.156 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:01:05.157 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:01:05.157 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:01:05.157 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:01:05.157 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:01:05.157 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:01:05.157 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:01:05.157 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:01:05.157 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:01:05.157 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:01:05.157 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:01:05.157 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:01:05.157 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.157 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:01:05.157 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:01:05.158 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:01:05.158 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:01:05.158 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:01:05.158 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:01:05.158 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:01:05.158 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:01:05.158 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:01:05.158 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:01:05.158 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:01:05.158 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:01:05.158 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 64, 2024-04-24T23:01:05.158 INFO:teuthology.orchestra.run.smithi032.stdout: "target_size_ratio": 0.89999997615814209 2024-04-24T23:01:05.158 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.158 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:01:05.159 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:01:05.159 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2024-04-24T23:01:05.159 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.159 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.159 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:01:05.159 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:01:05.159 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.3799999952316284, 2024-04-24T23:01:05.159 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.3799999952316284, 2024-04-24T23:01:05.159 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:01:05.159 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.3799999952316284, 2024-04-24T23:01:05.159 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.3799999952316284, 2024-04-24T23:01:05.159 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:01:05.159 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:01:05.159 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:01:05.160 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.160 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.160 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:01:05.160 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-04-24T23:01:05.160 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.160 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:01:05.160 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:01:05.160 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:01:05.160 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:01:05.160 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:01:05.160 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:01:05.161 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:01:05.161 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:01:05.161 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:01:05.161 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 49, 2024-04-24T23:01:05.161 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:01:05.161 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:01:05.161 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:01:05.161 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.161 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.161 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.161 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2024-04-24T23:01:05.161 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:01:05.161 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.161 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.161 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.162 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2024-04-24T23:01:05.162 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:01:05.162 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.162 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.162 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.162 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:01:05.162 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.162 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.162 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.162 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2024-04-24T23:01:05.162 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:01:05.162 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.162 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.163 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.163 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2024-04-24T23:01:05.163 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:01:05.163 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.163 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.163 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.163 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:01:05.163 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.163 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.163 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.163 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2024-04-24T23:01:05.163 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:01:05.163 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.163 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.163 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.164 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2024-04-24T23:01:05.164 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:01:05.164 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.164 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.164 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.164 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:01:05.164 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.164 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.164 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.164 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2024-04-24T23:01:05.164 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:01:05.164 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.164 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.164 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.164 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2024-04-24T23:01:05.164 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:01:05.165 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.165 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.165 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.165 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/1479395426", 2024-04-24T23:01:05.165 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/1479395426", 2024-04-24T23:01:05.165 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/1479395426", 2024-04-24T23:01:05.165 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/1479395426", 2024-04-24T23:01:05.165 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:01:05.165 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:01:05.165 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:01:05.165 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.165 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.165 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.166 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:01:05.166 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:01:05.166 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:01:05.166 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:01:05.166 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:01:05.166 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:01:05.166 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:01:05.166 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:01:05.166 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:01:05.166 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 49, 2024-04-24T23:01:05.166 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:01:05.166 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:01:05.166 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:01:05.166 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.166 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.167 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.167 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2024-04-24T23:01:05.167 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:01:05.167 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.167 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.167 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.167 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2024-04-24T23:01:05.167 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:01:05.167 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.167 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.167 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.167 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:01:05.167 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.167 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.167 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.168 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2024-04-24T23:01:05.168 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:01:05.168 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.168 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.168 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.168 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2024-04-24T23:01:05.168 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:01:05.168 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.168 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.168 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.168 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:01:05.168 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.168 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.168 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.169 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2024-04-24T23:01:05.169 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:01:05.169 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.169 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.169 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.169 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2024-04-24T23:01:05.169 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:01:05.169 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.169 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.169 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.169 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:01:05.169 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.169 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.169 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.169 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2024-04-24T23:01:05.170 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:01:05.170 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.170 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.170 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.170 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2024-04-24T23:01:05.170 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:01:05.170 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.170 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.170 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.170 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/1340633718", 2024-04-24T23:01:05.170 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/1340633718", 2024-04-24T23:01:05.170 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/1340633718", 2024-04-24T23:01:05.170 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/1340633718", 2024-04-24T23:01:05.170 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:01:05.170 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:01:05.171 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:01:05.171 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.171 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.171 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.171 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:01:05.171 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:01:05.171 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:01:05.171 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:01:05.171 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:01:05.171 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:01:05.171 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:01:05.171 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:01:05.171 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:01:05.171 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 49, 2024-04-24T23:01:05.172 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:01:05.172 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:01:05.172 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:01:05.172 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.172 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.172 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.172 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2024-04-24T23:01:05.172 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:01:05.172 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.172 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.172 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.172 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2024-04-24T23:01:05.173 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:01:05.173 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.173 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.173 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.173 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:01:05.173 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.173 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.173 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.173 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2024-04-24T23:01:05.173 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:01:05.173 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.173 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.174 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.174 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2024-04-24T23:01:05.174 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:01:05.174 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.174 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.174 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:01:05.174 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.174 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.174 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.174 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2024-04-24T23:01:05.174 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:01:05.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.174 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.175 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.175 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2024-04-24T23:01:05.175 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:01:05.175 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.175 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.175 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.175 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:01:05.175 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.175 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.175 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.175 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2024-04-24T23:01:05.175 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:01:05.175 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.175 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.176 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.176 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2024-04-24T23:01:05.176 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:01:05.176 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.176 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.176 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/3405899334", 2024-04-24T23:01:05.176 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6820/3405899334", 2024-04-24T23:01:05.176 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6828/3405899334", 2024-04-24T23:01:05.176 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6824/3405899334", 2024-04-24T23:01:05.176 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:01:05.176 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:01:05.176 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:01:05.176 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.176 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.177 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:01:05.177 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:01:05.177 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:01:05.177 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:01:05.177 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:01:05.177 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:01:05.177 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:01:05.177 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:01:05.177 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:01:05.177 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 43, 2024-04-24T23:01:05.177 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:01:05.177 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:01:05.177 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:01:05.177 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.178 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.178 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2024-04-24T23:01:05.178 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:01:05.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.178 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.178 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2024-04-24T23:01:05.178 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:01:05.179 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.179 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.179 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:01:05.179 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.179 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.179 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.179 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2024-04-24T23:01:05.179 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:01:05.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.179 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.179 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.179 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2024-04-24T23:01:05.179 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:01:05.179 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.180 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.180 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:01:05.180 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.180 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.180 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.180 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2024-04-24T23:01:05.180 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:01:05.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.180 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.180 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.180 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2024-04-24T23:01:05.180 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:01:05.180 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.181 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.181 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.181 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:01:05.181 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.181 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.181 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.181 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2024-04-24T23:01:05.181 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:01:05.181 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.181 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.181 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.181 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2024-04-24T23:01:05.181 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:01:05.181 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.181 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.182 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.182 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6821/3381057190", 2024-04-24T23:01:05.182 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6825/3381057190", 2024-04-24T23:01:05.182 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/3381057190", 2024-04-24T23:01:05.182 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/3381057190", 2024-04-24T23:01:05.182 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:01:05.182 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:01:05.182 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:01:05.182 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.182 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.182 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.182 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:01:05.182 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:01:05.182 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:01:05.182 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:01:05.183 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:01:05.183 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:01:05.183 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:01:05.183 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:01:05.183 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:01:05.183 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 43, 2024-04-24T23:01:05.183 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:01:05.183 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:01:05.183 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:01:05.183 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.183 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.183 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.183 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6800", 2024-04-24T23:01:05.183 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:01:05.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.184 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.184 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.184 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6801", 2024-04-24T23:01:05.184 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:01:05.184 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.184 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.184 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:01:05.184 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.184 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.184 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.184 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6802", 2024-04-24T23:01:05.184 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:01:05.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.185 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.185 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.185 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6803", 2024-04-24T23:01:05.185 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:01:05.185 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.185 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.185 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.185 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:01:05.185 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.185 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.185 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.185 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6806", 2024-04-24T23:01:05.185 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:01:05.185 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.185 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.186 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.186 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6807", 2024-04-24T23:01:05.186 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:01:05.186 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.186 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.186 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.186 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:01:05.186 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.186 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.186 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.186 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6804", 2024-04-24T23:01:05.186 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:01:05.186 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.186 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.187 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.187 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6805", 2024-04-24T23:01:05.187 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:01:05.187 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.187 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.187 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6801/4010553979", 2024-04-24T23:01:05.187 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6803/4010553979", 2024-04-24T23:01:05.187 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6807/4010553979", 2024-04-24T23:01:05.187 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6805/4010553979", 2024-04-24T23:01:05.187 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:01:05.187 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:01:05.187 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:01:05.187 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.187 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.188 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:01:05.188 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:01:05.188 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:01:05.188 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:01:05.188 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:01:05.188 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:01:05.188 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:01:05.188 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:01:05.188 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:01:05.188 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 43, 2024-04-24T23:01:05.188 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:01:05.188 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:01:05.188 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:01:05.188 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.188 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.189 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.189 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6808", 2024-04-24T23:01:05.189 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:01:05.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.189 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.189 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.189 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6809", 2024-04-24T23:01:05.189 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:01:05.189 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.189 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.189 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:01:05.189 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.189 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.189 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.189 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6810", 2024-04-24T23:01:05.190 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:01:05.190 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.190 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.190 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.190 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6811", 2024-04-24T23:01:05.190 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:01:05.190 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.190 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.190 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.190 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:01:05.190 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.190 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.190 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.191 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6814", 2024-04-24T23:01:05.191 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:01:05.191 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.191 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.191 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.191 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6815", 2024-04-24T23:01:05.191 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:01:05.191 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.191 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.191 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.191 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:01:05.191 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.191 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.191 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.191 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6812", 2024-04-24T23:01:05.192 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:01:05.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.192 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.192 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6813", 2024-04-24T23:01:05.192 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:01:05.192 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.192 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.192 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6809/3794084091", 2024-04-24T23:01:05.192 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6811/3794084091", 2024-04-24T23:01:05.192 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6815/3794084091", 2024-04-24T23:01:05.192 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6813/3794084091", 2024-04-24T23:01:05.192 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:01:05.192 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:01:05.193 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:01:05.193 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.193 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:01:05.193 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:01:05.193 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:01:05.193 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:01:05.193 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:01:05.193 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:01:05.193 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:01:05.193 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:01:05.193 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:01:05.193 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 43, 2024-04-24T23:01:05.193 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:01:05.194 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:01:05.194 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:01:05.194 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.194 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.194 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.194 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6823", 2024-04-24T23:01:05.194 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:01:05.194 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.194 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.194 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.194 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6825", 2024-04-24T23:01:05.194 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:01:05.194 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.194 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.194 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.195 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:01:05.195 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.195 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.195 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.195 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6826", 2024-04-24T23:01:05.195 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:01:05.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.195 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.195 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.195 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6827", 2024-04-24T23:01:05.195 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:01:05.195 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.195 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.195 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:01:05.196 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.196 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.196 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6830", 2024-04-24T23:01:05.196 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:01:05.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.196 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.196 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6831", 2024-04-24T23:01:05.196 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:01:05.196 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.196 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.196 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:01:05.196 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.197 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.197 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.197 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6828", 2024-04-24T23:01:05.197 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:01:05.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.197 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.197 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.197 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6829", 2024-04-24T23:01:05.197 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:01:05.197 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.197 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.197 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6825/1348640258", 2024-04-24T23:01:05.197 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6827/1348640258", 2024-04-24T23:01:05.197 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6831/1348640258", 2024-04-24T23:01:05.198 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6829/1348640258", 2024-04-24T23:01:05.198 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:01:05.198 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:01:05.198 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:01:05.198 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.198 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.198 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:01:05.198 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:01:05.198 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:01:05.198 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:01:05.198 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:01:05.198 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:01:05.198 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:01:05.198 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:01:05.199 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:01:05.199 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 49, 2024-04-24T23:01:05.199 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:01:05.199 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:01:05.199 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:01:05.199 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.199 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.199 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.199 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6816", 2024-04-24T23:01:05.199 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:01:05.199 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.199 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.199 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.199 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6817", 2024-04-24T23:01:05.199 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:01:05.200 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.200 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.200 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.200 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:01:05.200 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.200 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.200 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.200 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6818", 2024-04-24T23:01:05.200 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:01:05.200 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.200 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.200 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.200 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6819", 2024-04-24T23:01:05.200 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:01:05.201 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.201 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.201 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:01:05.201 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.201 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.201 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6822", 2024-04-24T23:01:05.201 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:01:05.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.201 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.201 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6824", 2024-04-24T23:01:05.201 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:01:05.201 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.202 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.202 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:01:05.202 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.202 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.202 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6820", 2024-04-24T23:01:05.202 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:01:05.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.202 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.202 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6821", 2024-04-24T23:01:05.202 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:01:05.202 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.203 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.203 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.203 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6817/3250005122", 2024-04-24T23:01:05.203 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6819/3250005122", 2024-04-24T23:01:05.203 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6824/3250005122", 2024-04-24T23:01:05.203 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6821/3250005122", 2024-04-24T23:01:05.203 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:01:05.203 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:01:05.203 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:01:05.203 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.203 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.203 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:01:05.203 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2024-04-24T23:01:05.203 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.203 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:01:05.204 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:01:05.204 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:01:05.204 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:01:05.204 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:01:05.204 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:01:05.204 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.156184+0000", 2024-04-24T23:01:05.204 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:01:05.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.204 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.204 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:01:05.204 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:01:05.204 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:01:05.204 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:01:05.204 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:01:05.204 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:01:05.205 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240774+0000", 2024-04-24T23:01:05.205 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:01:05.205 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.205 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:01:05.205 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:01:05.205 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:01:05.205 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:01:05.205 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:01:05.205 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:01:05.205 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240949+0000", 2024-04-24T23:01:05.205 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:01:05.205 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.205 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:01:05.206 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:01:05.206 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:01:05.206 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:01:05.206 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:01:05.206 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:01:05.206 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.171672+0000", 2024-04-24T23:01:05.206 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:01:05.206 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.206 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.206 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:01:05.206 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:01:05.206 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:01:05.206 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:01:05.206 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:01:05.207 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:01:05.207 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.220379+0000", 2024-04-24T23:01:05.207 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:01:05.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.207 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.207 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:01:05.207 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:01:05.207 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:01:05.207 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:01:05.207 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:01:05.207 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:01:05.207 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.204963+0000", 2024-04-24T23:01:05.207 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:01:05.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.207 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.208 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:01:05.208 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:01:05.208 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:01:05.208 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:01:05.208 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:01:05.208 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:01:05.208 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.239748+0000", 2024-04-24T23:01:05.208 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:01:05.208 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.208 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.208 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:01:05.208 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:01:05.208 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:01:05.208 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:01:05.208 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:01:05.209 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:01:05.209 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.154104+0000", 2024-04-24T23:01:05.209 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:01:05.209 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.209 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:01:05.209 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2024-04-24T23:01:05.209 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [ 2024-04-24T23:01:05.209 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.209 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "7.6", 2024-04-24T23:01:05.209 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:01:05.209 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.209 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 5, 2024-04-24T23:01:05.209 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 7 2024-04-24T23:01:05.209 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.210 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.210 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.210 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "7.c", 2024-04-24T23:01:05.210 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:01:05.210 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.210 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 5, 2024-04-24T23:01:05.210 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 7 2024-04-24T23:01:05.210 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.210 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.210 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.210 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "7.12", 2024-04-24T23:01:05.210 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:01:05.210 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.211 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 5, 2024-04-24T23:01:05.211 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 7 2024-04-24T23:01:05.211 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.211 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.211 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.211 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "7.31", 2024-04-24T23:01:05.211 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:01:05.211 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.211 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 5, 2024-04-24T23:01:05.211 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 7 2024-04-24T23:01:05.211 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.211 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.211 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.211 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "7.35", 2024-04-24T23:01:05.212 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:01:05.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.212 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 5, 2024-04-24T23:01:05.212 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 0 2024-04-24T23:01:05.212 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.212 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.212 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.212 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:01:05.212 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_primaries": [], 2024-04-24T23:01:05.212 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2024-04-24T23:01:05.212 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2024-04-24T23:01:05.212 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklist": {}, 2024-04-24T23:01:05.212 INFO:teuthology.orchestra.run.smithi032.stdout: "range_blocklist": {}, 2024-04-24T23:01:05.212 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2024-04-24T23:01:05.213 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2024-04-24T23:01:05.213 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2024-04-24T23:01:05.213 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2024-04-24T23:01:05.213 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2024-04-24T23:01:05.213 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2024-04-24T23:01:05.213 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2024-04-24T23:01:05.213 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.213 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2024-04-24T23:01:05.213 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2024-04-24T23:01:05.213 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2024-04-24T23:01:05.213 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2024-04-24T23:01:05.213 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {}, 2024-04-24T23:01:05.213 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode": { 2024-04-24T23:01:05.213 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_enabled": false, 2024-04-24T23:01:05.214 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_bucket_count": 0, 2024-04-24T23:01:05.214 INFO:teuthology.orchestra.run.smithi032.stdout: "degraded_stretch_mode": 0, 2024-04-24T23:01:05.214 INFO:teuthology.orchestra.run.smithi032.stdout: "recovering_stretch_mode": 0, 2024-04-24T23:01:05.214 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_bucket": 0 2024-04-24T23:01:05.214 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.214 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:01:05.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.136+0000 7f3743989640 1 -- 172.21.15.32:0/3030385874 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f372806be60 msgr2=0x7f372806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:05.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.136+0000 7f3743989640 1 --2- 172.21.15.32:0/3030385874 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f372806be60 0x7f372806e320 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3734032000 tx=0x7f373402e000 comp rx=0 tx=0).stop 2024-04-24T23:01:05.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.136+0000 7f3743989640 1 -- 172.21.15.32:0/3030385874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f373c1f63a0 msgr2=0x7f373c1e6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:05.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.136+0000 7f3743989640 1 --2- 172.21.15.32:0/3030385874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f373c1f63a0 0x7f373c1e6c60 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f373c073960 tx=0x7f372c00e530 comp rx=0 tx=0).stop 2024-04-24T23:01:05.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.136+0000 7f3743989640 1 -- 172.21.15.32:0/3030385874 shutdown_connections 2024-04-24T23:01:05.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.136+0000 7f3743989640 1 --2- 172.21.15.32:0/3030385874 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f372806be60 0x7f372806e320 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:05.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.136+0000 7f3743989640 1 --2- 172.21.15.32:0/3030385874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f373c1f63a0 0x7f373c1e6c60 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:05.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.136+0000 7f3743989640 1 --2- 172.21.15.32:0/3030385874 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f373c155960 0x7f373c1f2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:05.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.136+0000 7f3743989640 1 --2- 172.21.15.32:0/3030385874 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37240039e0 0x7f373c155400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:05.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.136+0000 7f3743989640 1 -- 172.21.15.32:0/3030385874 >> 172.21.15.32:0/3030385874 conn(0x7f373c06b110 msgr2=0x7f373c15c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:05.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.137+0000 7f3743989640 1 -- 172.21.15.32:0/3030385874 shutdown_connections 2024-04-24T23:01:05.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.137+0000 7f3743989640 1 -- 172.21.15.32:0/3030385874 wait complete. 2024-04-24T23:01:05.217 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2024-04-24T23:01:05.218 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-24T23:01:05.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.318+0000 7f05b4b29640 1 Processor -- start 2024-04-24T23:01:05.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.318+0000 7f05b4b29640 1 -- start start 2024-04-24T23:01:05.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.319+0000 7f05b4b29640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f05b015fd60 0x7f05b006c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:05.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.319+0000 7f05b4b29640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f05b006cb30 0x7f05b00cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:05.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.319+0000 7f05b4b29640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05b00b46c0 con 0x7f05b015b480 2024-04-24T23:01:05.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.319+0000 7f05b4b29640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f05b00b4ff0 con 0x7f05b006cb30 2024-04-24T23:01:05.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.319+0000 7f05b4b29640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f05b00b49a0 con 0x7f05b015fd60 2024-04-24T23:01:05.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.319+0000 7f05ae59c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f05b015b480 0x7f05b00b5c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56756/0 (socket says 172.21.15.32:56756) 2024-04-24T23:01:05.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.319+0000 7f05ae59c640 1 -- 172.21.15.32:0/782649060 learned_addr learned my addr 172.21.15.32:0/782649060 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:01:05.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.319+0000 7f05add9b640 1 --2- 172.21.15.32:0/782649060 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f05b015fd60 0x7f05b006c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:05.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.319+0000 7f05aed9d640 1 --2- 172.21.15.32:0/782649060 >> v2:172.21.15.121:3300/0 conn(0x7f05b006cb30 0x7f05b00cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:05.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.319+0000 7f05ad59a640 1 -- 172.21.15.32:0/782649060 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3492724242 0 0) 0x7f05b00b46c0 con 0x7f05b015b480 2024-04-24T23:01:05.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.319+0000 7f05ad59a640 1 -- 172.21.15.32:0/782649060 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0598003520 con 0x7f05b015b480 2024-04-24T23:01:05.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.319+0000 7f05add9b640 1 -- 172.21.15.32:0/782649060 >> v2:172.21.15.121:3300/0 conn(0x7f05b006cb30 msgr2=0x7f05b00cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:05.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.319+0000 7f05add9b640 1 --2- 172.21.15.32:0/782649060 >> v2:172.21.15.121:3300/0 conn(0x7f05b006cb30 0x7f05b00cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:05.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.319+0000 7f05add9b640 1 -- 172.21.15.32:0/782649060 >> v1:172.21.15.32:6789/0 conn(0x7f05b015b480 legacy=0x7f05b00b5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:05.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.319+0000 7f05add9b640 1 -- 172.21.15.32:0/782649060 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05b00b4c80 con 0x7f05b015fd60 2024-04-24T23:01:05.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.319+0000 7f05aed9d640 1 --2- 172.21.15.32:0/782649060 >> v2:172.21.15.121:3300/0 conn(0x7f05b006cb30 0x7f05b00cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:01:05.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.320+0000 7f05add9b640 1 --2- 172.21.15.32:0/782649060 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f05b015fd60 0x7f05b006c5f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f05a40097a0 tx=0x7f05a402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aef69bd8450b96be server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:05.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.320+0000 7f05ad59a640 1 -- 172.21.15.32:0/782649060 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a4037070 con 0x7f05b015fd60 2024-04-24T23:01:05.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.320+0000 7f05ad59a640 1 -- 172.21.15.32:0/782649060 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05a4032440 con 0x7f05b015fd60 2024-04-24T23:01:05.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.320+0000 7f05ad59a640 1 -- 172.21.15.32:0/782649060 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a4032810 con 0x7f05b015fd60 2024-04-24T23:01:05.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.321+0000 7f05b4b29640 1 -- 172.21.15.32:0/782649060 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f05b015fd60 msgr2=0x7f05b006c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:05.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.321+0000 7f05b4b29640 1 --2- 172.21.15.32:0/782649060 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f05b015fd60 0x7f05b006c5f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f05a40097a0 tx=0x7f05a402a180 comp rx=0 tx=0).stop 2024-04-24T23:01:05.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.321+0000 7f05b4b29640 1 -- 172.21.15.32:0/782649060 shutdown_connections 2024-04-24T23:01:05.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.321+0000 7f05b4b29640 1 --2- 172.21.15.32:0/782649060 >> v2:172.21.15.121:3300/0 conn(0x7f05b006cb30 0x7f05b00cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:05.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.321+0000 7f05b4b29640 1 --2- 172.21.15.32:0/782649060 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f05b015fd60 0x7f05b006c5f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:05.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.321+0000 7f05b4b29640 1 -- 172.21.15.32:0/782649060 >> 172.21.15.32:0/782649060 conn(0x7f05b006b110 msgr2=0x7f05b006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:05.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.321+0000 7f05b4b29640 1 -- 172.21.15.32:0/782649060 shutdown_connections 2024-04-24T23:01:05.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.321+0000 7f05b4b29640 1 -- 172.21.15.32:0/782649060 wait complete. 2024-04-24T23:01:05.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.322+0000 7f05b4b29640 1 Processor -- start 2024-04-24T23:01:05.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.322+0000 7f05b4b29640 1 -- start start 2024-04-24T23:01:05.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.323+0000 7f05b4b29640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05b006cb30 0x7f05b015e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:05.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.323+0000 7f05b4b29640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05b015b480 0x7f05b015bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:05.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.323+0000 7f05b4b29640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f05b015fd60 0x7f05b0159560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:05.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.323+0000 7f05b4b29640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f05b015b860 con 0x7f05b006cb30 2024-04-24T23:01:05.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.323+0000 7f05b4b29640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f05b00ce4c0 con 0x7f05b015b480 2024-04-24T23:01:05.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.323+0000 7f05b4b29640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f05b00726e0 con 0x7f05b015fd60 2024-04-24T23:01:05.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.323+0000 7f05ae59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05b006cb30 0x7f05b015e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:05.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.323+0000 7f05ae59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05b006cb30 0x7f05b015e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50130/0 (socket says 172.21.15.32:50130) 2024-04-24T23:01:05.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.323+0000 7f05ae59c640 1 -- 172.21.15.32:0/1220770112 learned_addr learned my addr 172.21.15.32:0/1220770112 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:05.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.323+0000 7f05aed9d640 1 --2- 172.21.15.32:0/1220770112 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f05b015fd60 0x7f05b0159560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:05.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.323+0000 7f05add9b640 1 --2- 172.21.15.32:0/1220770112 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05b015b480 0x7f05b015bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:05.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.323+0000 7f05ae59c640 1 -- 172.21.15.32:0/1220770112 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f05b015fd60 msgr2=0x7f05b0159560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:05.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.323+0000 7f05ae59c640 1 --2- 172.21.15.32:0/1220770112 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f05b015fd60 0x7f05b0159560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:05.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.323+0000 7f05ae59c640 1 -- 172.21.15.32:0/1220770112 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05b015b480 msgr2=0x7f05b015bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:05.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.323+0000 7f05ae59c640 1 --2- 172.21.15.32:0/1220770112 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05b015b480 0x7f05b015bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:05.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.323+0000 7f05ae59c640 1 -- 172.21.15.32:0/1220770112 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05b01f4e60 con 0x7f05b006cb30 2024-04-24T23:01:05.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.323+0000 7f05add9b640 1 --2- 172.21.15.32:0/1220770112 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05b015b480 0x7f05b015bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:01:05.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.324+0000 7f05ae59c640 1 --2- 172.21.15.32:0/1220770112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05b006cb30 0x7f05b015e550 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f05a000c430 tx=0x7f05a000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:05.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.324+0000 7f059effd640 1 -- 172.21.15.32:0/1220770112 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a0017070 con 0x7f05b006cb30 2024-04-24T23:01:05.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.324+0000 7f05b4b29640 1 -- 172.21.15.32:0/1220770112 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05b01f5140 con 0x7f05b006cb30 2024-04-24T23:01:05.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.324+0000 7f059effd640 1 -- 172.21.15.32:0/1220770112 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05a0005ce0 con 0x7f05b006cb30 2024-04-24T23:01:05.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.324+0000 7f059effd640 1 -- 172.21.15.32:0/1220770112 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a00072f0 con 0x7f05b006cb30 2024-04-24T23:01:05.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.324+0000 7f05b4b29640 1 -- 172.21.15.32:0/1220770112 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05b0159040 con 0x7f05b006cb30 2024-04-24T23:01:05.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.325+0000 7f05b4b29640 1 -- 172.21.15.32:0/1220770112 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05b0068430 con 0x7f05b006cb30 2024-04-24T23:01:05.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.325+0000 7f059effd640 1 -- 172.21.15.32:0/1220770112 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f05a001a070 con 0x7f05b006cb30 2024-04-24T23:01:05.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.326+0000 7f059effd640 1 --2- 172.21.15.32:0/1220770112 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f059006be30 0x7f059006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:05.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.326+0000 7f059effd640 1 -- 172.21.15.32:0/1220770112 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5854+0+0 (secure 0 0 0) 0x7f05a008a470 con 0x7f05b006cb30 2024-04-24T23:01:05.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.326+0000 7f05add9b640 1 --2- 172.21.15.32:0/1220770112 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f059006be30 0x7f059006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:05.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.326+0000 7f05add9b640 1 --2- 172.21.15.32:0/1220770112 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f059006be30 0x7f059006e2f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f05a4034000 tx=0x7f05a402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:05.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.330+0000 7f059effd640 1 -- 172.21.15.32:0/1220770112 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f05a0058d10 con 0x7f05b006cb30 2024-04-24T23:01:05.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.512+0000 7f05b4b29640 1 -- 172.21.15.32:0/1220770112 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f05b006c000 con 0x7f05b006cb30 2024-04-24T23:01:05.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.514+0000 7f059effd640 1 -- 172.21.15.32:0/1220770112 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v56) ==== 81+0+35091 (secure 0 0 0) 0x7f05a0058610 con 0x7f05b006cb30 2024-04-24T23:01:05.516 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:01:05.516 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:01:05.516 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 56, 2024-04-24T23:01:05.516 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", 2024-04-24T23:01:05.516 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2024-04-24T22:58:46.964851+0000", 2024-04-24T23:01:05.516 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2024-04-24T23:01:00.283572+0000", 2024-04-24T23:01:05.516 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2024-04-24T22:58:52.048787+0000", 2024-04-24T23:01:05.516 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2024-04-24T22:58:49.045609+0000", 2024-04-24T23:01:05.516 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-24T23:01:05.517 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2024-04-24T23:01:05.517 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2024-04-24T23:01:05.517 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2024-04-24T23:01:05.517 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2024-04-24T23:01:05.517 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2024-04-24T23:01:05.517 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2024-04-24T23:01:05.517 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:01:05.517 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 4, 2024-04-24T23:01:05.517 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.69999998807907104, 2024-04-24T23:01:05.517 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-24T23:01:05.517 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-24T23:01:05.517 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2024-04-24T23:01:05.517 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 7, 2024-04-24T23:01:05.517 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2024-04-24T23:01:05.518 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "luminous", 2024-04-24T23:01:05.518 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "luminous", 2024-04-24T23:01:05.518 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "squid", 2024-04-24T23:01:05.518 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_crimson": false, 2024-04-24T23:01:05.518 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2024-04-24T23:01:05.518 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.518 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2024-04-24T23:01:05.518 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": ".mgr", 2024-04-24T23:01:05.518 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T22:58:53.171408+0000", 2024-04-24T23:01:05.518 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:01:05.518 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:01:05.519 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:01:05.519 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:01:05.519 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:01:05.519 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:01:05.519 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:01:05.519 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:01:05.519 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:01:05.519 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:01:05.519 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:01:05.519 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:01:05.519 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2024-04-24T23:01:05.519 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2024-04-24T23:01:05.519 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2024-04-24T23:01:05.519 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2024-04-24T23:01:05.520 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2024-04-24T23:01:05.520 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:01:05.520 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:01:05.520 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:01:05.520 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:01:05.520 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:01:05.520 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:01:05.520 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:01:05.520 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.520 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "56", 2024-04-24T23:01:05.520 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:01:05.520 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:01:05.520 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:01:05.521 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:01:05.521 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:01:05.521 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:01:05.521 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:01:05.521 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:01:05.521 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:01:05.521 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:01:05.521 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:01:05.521 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:01:05.521 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:01:05.521 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:01:05.521 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:01:05.522 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:01:05.522 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:01:05.522 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:01:05.522 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:01:05.522 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:01:05.522 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:01:05.522 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:01:05.522 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:01:05.522 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:01:05.522 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:01:05.522 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:01:05.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.522 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:01:05.523 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:01:05.523 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:01:05.523 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:01:05.523 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:01:05.523 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:01:05.523 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:01:05.523 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:01:05.523 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:01:05.523 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:01:05.523 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:01:05.523 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:01:05.523 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_max": 32, 2024-04-24T23:01:05.523 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2024-04-24T23:01:05.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.524 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:01:05.524 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": {} 2024-04-24T23:01:05.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.524 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:01:05.524 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:01:05.524 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 8, 2024-04-24T23:01:05.524 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 8, 2024-04-24T23:01:05.524 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.25, 2024-04-24T23:01:05.524 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-04-24T23:01:05.524 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-04-24T23:01:05.524 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:01:05.524 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:01:05.525 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:01:05.525 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.525 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.525 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.525 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 6, 2024-04-24T23:01:05.525 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2024-04-24T23:01:05.525 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:00:30.665592+0000", 2024-04-24T23:01:05.525 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:01:05.525 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:01:05.525 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:01:05.525 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:01:05.525 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:01:05.525 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:01:05.525 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:01:05.526 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:01:05.526 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:01:05.526 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:01:05.526 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:01:05.526 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:01:05.526 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 32, 2024-04-24T23:01:05.526 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 32, 2024-04-24T23:01:05.526 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 32, 2024-04-24T23:01:05.526 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 32, 2024-04-24T23:01:05.526 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 32, 2024-04-24T23:01:05.526 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:01:05.527 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:01:05.527 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:01:05.527 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:01:05.527 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:01:05.527 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:01:05.527 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:01:05.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.527 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "56", 2024-04-24T23:01:05.527 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:01:05.527 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:01:05.527 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:01:05.527 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:01:05.528 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:01:05.528 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:01:05.528 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:01:05.528 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:01:05.528 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:01:05.528 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:01:05.528 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:01:05.528 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:01:05.528 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:01:05.528 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:01:05.529 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:01:05.529 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:01:05.529 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:01:05.529 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:01:05.529 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:01:05.529 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:01:05.529 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:01:05.529 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:01:05.529 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:01:05.529 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:01:05.529 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:01:05.529 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:01:05.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.530 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:01:05.530 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:01:05.530 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:01:05.530 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:01:05.530 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:01:05.530 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:01:05.530 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:01:05.530 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:01:05.530 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:01:05.530 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:01:05.530 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:01:05.530 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:01:05.531 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2024-04-24T23:01:05.531 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2024-04-24T23:01:05.531 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2024-04-24T23:01:05.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.531 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:01:05.531 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:01:05.531 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2024-04-24T23:01:05.531 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.531 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:01:05.531 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:01:05.531 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.5, 2024-04-24T23:01:05.531 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.5, 2024-04-24T23:01:05.531 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:01:05.532 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.5, 2024-04-24T23:01:05.532 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.5, 2024-04-24T23:01:05.532 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:01:05.532 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:01:05.532 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:01:05.532 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.532 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.532 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.532 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 7, 2024-04-24T23:01:05.532 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2024-04-24T23:01:05.532 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:00:31.357756+0000", 2024-04-24T23:01:05.532 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:01:05.533 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:01:05.533 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:01:05.533 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:01:05.533 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:01:05.533 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:01:05.533 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:01:05.533 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:01:05.533 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:01:05.533 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:01:05.533 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:01:05.533 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:01:05.533 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 64, 2024-04-24T23:01:05.533 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 64, 2024-04-24T23:01:05.533 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 64, 2024-04-24T23:01:05.534 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 64, 2024-04-24T23:01:05.534 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 64, 2024-04-24T23:01:05.534 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:01:05.534 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:01:05.534 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:01:05.534 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:01:05.534 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:01:05.534 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:01:05.534 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:01:05.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.535 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "56", 2024-04-24T23:01:05.535 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:01:05.535 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:01:05.535 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:01:05.535 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:01:05.535 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:01:05.535 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:01:05.536 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:01:05.536 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:01:05.536 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:01:05.536 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:01:05.536 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:01:05.536 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:01:05.536 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:01:05.536 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:01:05.536 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:01:05.536 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:01:05.536 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:01:05.536 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:01:05.537 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:01:05.537 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:01:05.537 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:01:05.537 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:01:05.537 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:01:05.537 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:01:05.537 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:01:05.537 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:01:05.537 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.537 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:01:05.537 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:01:05.537 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:01:05.537 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:01:05.538 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:01:05.538 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:01:05.538 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:01:05.538 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:01:05.538 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:01:05.538 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:01:05.538 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:01:05.538 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:01:05.538 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 64, 2024-04-24T23:01:05.538 INFO:teuthology.orchestra.run.smithi032.stdout: "target_size_ratio": 0.89999997615814209 2024-04-24T23:01:05.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.539 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:01:05.539 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:01:05.539 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2024-04-24T23:01:05.539 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.539 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:01:05.539 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:01:05.539 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.3799999952316284, 2024-04-24T23:01:05.539 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.3799999952316284, 2024-04-24T23:01:05.539 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:01:05.540 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.3799999952316284, 2024-04-24T23:01:05.540 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.3799999952316284, 2024-04-24T23:01:05.540 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:01:05.540 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:01:05.540 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:01:05.540 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.540 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.540 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:01:05.540 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-04-24T23:01:05.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.540 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:01:05.540 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:01:05.540 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:01:05.540 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:01:05.541 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:01:05.541 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:01:05.541 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:01:05.541 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:01:05.541 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:01:05.541 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 49, 2024-04-24T23:01:05.541 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:01:05.541 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:01:05.541 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:01:05.541 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.542 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.542 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2024-04-24T23:01:05.542 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:01:05.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.542 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.542 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.542 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2024-04-24T23:01:05.543 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:01:05.543 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.543 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.543 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:01:05.543 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.543 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.543 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2024-04-24T23:01:05.543 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:01:05.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.544 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.544 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2024-04-24T23:01:05.544 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:01:05.544 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.544 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.544 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:01:05.544 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.544 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.544 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2024-04-24T23:01:05.544 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:01:05.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.545 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.545 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2024-04-24T23:01:05.545 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:01:05.545 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.545 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.545 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:01:05.545 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.545 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.545 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2024-04-24T23:01:05.545 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:01:05.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.546 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.546 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2024-04-24T23:01:05.546 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:01:05.546 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.546 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.546 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/1479395426", 2024-04-24T23:01:05.546 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/1479395426", 2024-04-24T23:01:05.546 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/1479395426", 2024-04-24T23:01:05.546 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/1479395426", 2024-04-24T23:01:05.546 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:01:05.546 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:01:05.546 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:01:05.546 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.547 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:01:05.547 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:01:05.547 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:01:05.547 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:01:05.547 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:01:05.547 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:01:05.547 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:01:05.547 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:01:05.547 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:01:05.547 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 49, 2024-04-24T23:01:05.547 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:01:05.547 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:01:05.547 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:01:05.548 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.548 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.548 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2024-04-24T23:01:05.548 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:01:05.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.548 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.548 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2024-04-24T23:01:05.548 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:01:05.548 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.548 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.549 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:01:05.549 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.549 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.549 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2024-04-24T23:01:05.549 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:01:05.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.549 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.549 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2024-04-24T23:01:05.549 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:01:05.549 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.549 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.550 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:01:05.550 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.550 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.550 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2024-04-24T23:01:05.550 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:01:05.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.550 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.550 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2024-04-24T23:01:05.550 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:01:05.550 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.550 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.551 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:01:05.551 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.551 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.551 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2024-04-24T23:01:05.551 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:01:05.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.551 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.551 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2024-04-24T23:01:05.551 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:01:05.551 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.551 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.551 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/1340633718", 2024-04-24T23:01:05.552 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/1340633718", 2024-04-24T23:01:05.552 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/1340633718", 2024-04-24T23:01:05.552 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/1340633718", 2024-04-24T23:01:05.552 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:01:05.552 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:01:05.552 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:01:05.552 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.552 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:01:05.552 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:01:05.552 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:01:05.552 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:01:05.552 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:01:05.552 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:01:05.553 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:01:05.553 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:01:05.553 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:01:05.553 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 49, 2024-04-24T23:01:05.553 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:01:05.553 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:01:05.553 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:01:05.553 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.553 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.553 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2024-04-24T23:01:05.553 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:01:05.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.554 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.554 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2024-04-24T23:01:05.554 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:01:05.554 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.554 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.554 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:01:05.554 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.554 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.554 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2024-04-24T23:01:05.554 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:01:05.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.555 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.555 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2024-04-24T23:01:05.555 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:01:05.555 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.555 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.555 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:01:05.555 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.555 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.555 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2024-04-24T23:01:05.555 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:01:05.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.557 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.557 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2024-04-24T23:01:05.558 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:01:05.558 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.558 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.558 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:01:05.558 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.558 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.558 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2024-04-24T23:01:05.558 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:01:05.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.558 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.558 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2024-04-24T23:01:05.559 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:01:05.559 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.559 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.559 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/3405899334", 2024-04-24T23:01:05.559 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6820/3405899334", 2024-04-24T23:01:05.559 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6828/3405899334", 2024-04-24T23:01:05.559 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6824/3405899334", 2024-04-24T23:01:05.559 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:01:05.559 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:01:05.559 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:01:05.559 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.560 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:01:05.560 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:01:05.560 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:01:05.560 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:01:05.560 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:01:05.560 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:01:05.560 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:01:05.560 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:01:05.560 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:01:05.560 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 43, 2024-04-24T23:01:05.560 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:01:05.560 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:01:05.560 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:01:05.560 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.561 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.561 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2024-04-24T23:01:05.561 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:01:05.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.561 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.561 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2024-04-24T23:01:05.561 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:01:05.561 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.561 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.561 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:01:05.561 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.561 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.562 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2024-04-24T23:01:05.562 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:01:05.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.562 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.562 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2024-04-24T23:01:05.562 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:01:05.562 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.562 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.562 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:01:05.562 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.562 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.563 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2024-04-24T23:01:05.563 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:01:05.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.563 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.563 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2024-04-24T23:01:05.563 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:01:05.563 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.563 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.563 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:01:05.563 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.563 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.563 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2024-04-24T23:01:05.564 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:01:05.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.564 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.564 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2024-04-24T23:01:05.564 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:01:05.564 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.564 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.564 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6821/3381057190", 2024-04-24T23:01:05.564 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6825/3381057190", 2024-04-24T23:01:05.564 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/3381057190", 2024-04-24T23:01:05.564 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/3381057190", 2024-04-24T23:01:05.564 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:01:05.564 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:01:05.565 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:01:05.565 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.565 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:01:05.565 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:01:05.565 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:01:05.565 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:01:05.565 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:01:05.565 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:01:05.565 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:01:05.565 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:01:05.565 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:01:05.565 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 43, 2024-04-24T23:01:05.565 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:01:05.566 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:01:05.566 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:01:05.566 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.566 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.566 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6800", 2024-04-24T23:01:05.566 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:01:05.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.566 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.566 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6801", 2024-04-24T23:01:05.566 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:01:05.566 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.566 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.567 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:01:05.567 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.567 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.567 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6802", 2024-04-24T23:01:05.567 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:01:05.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.567 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.567 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6803", 2024-04-24T23:01:05.567 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:01:05.567 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.567 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.568 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:01:05.568 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.568 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.568 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6806", 2024-04-24T23:01:05.568 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:01:05.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.568 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.568 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6807", 2024-04-24T23:01:05.568 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:01:05.568 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.568 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.568 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:01:05.569 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.569 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.569 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6804", 2024-04-24T23:01:05.569 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:01:05.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.569 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.569 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6805", 2024-04-24T23:01:05.569 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:01:05.569 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.569 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.569 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6801/4010553979", 2024-04-24T23:01:05.569 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6803/4010553979", 2024-04-24T23:01:05.570 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6807/4010553979", 2024-04-24T23:01:05.570 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6805/4010553979", 2024-04-24T23:01:05.570 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:01:05.570 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:01:05.570 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:01:05.570 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.570 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:01:05.570 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:01:05.570 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:01:05.570 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:01:05.570 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:01:05.570 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:01:05.570 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:01:05.571 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:01:05.571 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:01:05.571 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 43, 2024-04-24T23:01:05.571 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:01:05.571 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:01:05.571 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:01:05.571 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.571 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.571 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6808", 2024-04-24T23:01:05.571 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:01:05.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.571 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.571 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6809", 2024-04-24T23:01:05.572 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:01:05.572 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.572 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.572 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:01:05.572 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.572 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.572 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6810", 2024-04-24T23:01:05.572 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:01:05.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.572 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.573 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6811", 2024-04-24T23:01:05.573 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:01:05.573 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.573 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.573 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:01:05.573 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.573 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.573 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6814", 2024-04-24T23:01:05.573 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:01:05.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.573 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.574 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6815", 2024-04-24T23:01:05.574 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:01:05.574 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.574 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.574 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:01:05.574 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.574 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.574 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6812", 2024-04-24T23:01:05.574 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:01:05.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.574 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.575 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6813", 2024-04-24T23:01:05.575 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:01:05.575 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.575 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.575 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6809/3794084091", 2024-04-24T23:01:05.575 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6811/3794084091", 2024-04-24T23:01:05.575 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6815/3794084091", 2024-04-24T23:01:05.575 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6813/3794084091", 2024-04-24T23:01:05.575 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:01:05.575 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:01:05.575 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:01:05.575 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.576 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:01:05.576 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:01:05.576 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:01:05.576 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:01:05.576 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:01:05.576 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:01:05.576 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:01:05.576 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:01:05.576 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:01:05.576 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 43, 2024-04-24T23:01:05.576 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:01:05.576 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:01:05.576 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:01:05.576 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.577 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.577 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6823", 2024-04-24T23:01:05.577 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:01:05.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.577 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.577 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6825", 2024-04-24T23:01:05.577 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:01:05.577 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.577 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.577 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:01:05.577 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.577 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.578 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6826", 2024-04-24T23:01:05.578 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:01:05.578 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.578 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.578 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.578 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6827", 2024-04-24T23:01:05.578 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:01:05.578 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.578 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.578 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.578 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:01:05.578 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.578 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.579 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.579 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6830", 2024-04-24T23:01:05.579 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:01:05.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.579 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.579 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6831", 2024-04-24T23:01:05.579 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:01:05.579 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.579 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.579 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:01:05.579 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.580 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.580 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6828", 2024-04-24T23:01:05.580 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:01:05.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.580 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.580 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6829", 2024-04-24T23:01:05.580 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:01:05.580 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.580 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.580 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6825/1348640258", 2024-04-24T23:01:05.580 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6827/1348640258", 2024-04-24T23:01:05.581 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6831/1348640258", 2024-04-24T23:01:05.581 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6829/1348640258", 2024-04-24T23:01:05.581 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:01:05.581 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:01:05.581 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:01:05.581 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.581 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:01:05.581 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:01:05.581 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:01:05.581 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:01:05.581 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:01:05.581 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:01:05.582 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:01:05.582 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:01:05.582 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:01:05.582 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 49, 2024-04-24T23:01:05.582 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:01:05.582 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:01:05.582 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:01:05.582 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.582 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.582 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.582 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6816", 2024-04-24T23:01:05.582 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:01:05.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.582 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.583 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.583 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6817", 2024-04-24T23:01:05.583 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:01:05.583 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.583 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.583 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:01:05.583 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.583 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.583 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6818", 2024-04-24T23:01:05.583 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:01:05.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.583 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.584 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6819", 2024-04-24T23:01:05.584 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:01:05.584 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.584 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.584 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:01:05.584 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.584 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.584 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6822", 2024-04-24T23:01:05.584 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:01:05.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.584 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.584 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6824", 2024-04-24T23:01:05.585 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:01:05.585 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.585 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.585 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:01:05.585 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:01:05.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.585 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:01:05.585 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6820", 2024-04-24T23:01:05.585 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:01:05.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.585 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:01:05.585 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6821", 2024-04-24T23:01:05.585 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:01:05.586 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.586 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.586 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6817/3250005122", 2024-04-24T23:01:05.586 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6819/3250005122", 2024-04-24T23:01:05.586 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6824/3250005122", 2024-04-24T23:01:05.586 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6821/3250005122", 2024-04-24T23:01:05.586 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:01:05.586 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:01:05.586 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:01:05.586 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.586 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.586 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:01:05.586 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2024-04-24T23:01:05.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.587 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:01:05.587 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:01:05.587 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:01:05.587 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:01:05.587 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:01:05.587 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:01:05.587 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.156184+0000", 2024-04-24T23:01:05.587 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:01:05.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.587 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:01:05.587 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:01:05.587 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:01:05.587 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:01:05.588 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:01:05.588 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:01:05.588 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240774+0000", 2024-04-24T23:01:05.588 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:01:05.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.588 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:01:05.588 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:01:05.588 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:01:05.588 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:01:05.588 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:01:05.588 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:01:05.588 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240949+0000", 2024-04-24T23:01:05.588 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:01:05.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.589 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:01:05.589 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:01:05.589 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:01:05.589 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:01:05.589 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:01:05.589 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:01:05.589 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.171672+0000", 2024-04-24T23:01:05.589 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:01:05.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.589 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:01:05.589 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:01:05.590 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:01:05.590 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:01:05.590 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:01:05.590 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:01:05.590 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.220379+0000", 2024-04-24T23:01:05.590 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:01:05.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.591 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:01:05.591 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:01:05.591 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:01:05.591 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:01:05.591 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:01:05.591 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:01:05.591 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.204963+0000", 2024-04-24T23:01:05.591 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:01:05.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.592 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:01:05.592 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:01:05.592 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:01:05.592 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:01:05.592 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:01:05.592 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:01:05.592 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.239748+0000", 2024-04-24T23:01:05.592 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:01:05.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.592 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.592 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:01:05.592 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:01:05.592 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:01:05.593 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:01:05.593 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:01:05.593 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:01:05.593 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.154104+0000", 2024-04-24T23:01:05.593 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:01:05.593 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.593 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:01:05.593 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2024-04-24T23:01:05.593 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [ 2024-04-24T23:01:05.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.593 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "7.6", 2024-04-24T23:01:05.593 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:01:05.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.593 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 5, 2024-04-24T23:01:05.593 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 7 2024-04-24T23:01:05.594 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.594 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.594 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "7.c", 2024-04-24T23:01:05.594 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:01:05.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.594 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 5, 2024-04-24T23:01:05.594 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 7 2024-04-24T23:01:05.594 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.594 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.595 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "7.12", 2024-04-24T23:01:05.595 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:01:05.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.595 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 5, 2024-04-24T23:01:05.595 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 7 2024-04-24T23:01:05.595 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.595 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.595 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "7.31", 2024-04-24T23:01:05.595 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:01:05.596 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.596 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 5, 2024-04-24T23:01:05.596 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 7 2024-04-24T23:01:05.596 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.596 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.596 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.596 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "7.35", 2024-04-24T23:01:05.596 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:01:05.596 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:05.596 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 5, 2024-04-24T23:01:05.596 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 0 2024-04-24T23:01:05.596 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.596 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:05.597 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.597 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:01:05.597 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_primaries": [], 2024-04-24T23:01:05.597 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2024-04-24T23:01:05.597 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2024-04-24T23:01:05.597 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklist": {}, 2024-04-24T23:01:05.597 INFO:teuthology.orchestra.run.smithi032.stdout: "range_blocklist": {}, 2024-04-24T23:01:05.597 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2024-04-24T23:01:05.597 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2024-04-24T23:01:05.597 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2024-04-24T23:01:05.597 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2024-04-24T23:01:05.597 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2024-04-24T23:01:05.597 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2024-04-24T23:01:05.597 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2024-04-24T23:01:05.597 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:05.598 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2024-04-24T23:01:05.598 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2024-04-24T23:01:05.598 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2024-04-24T23:01:05.598 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2024-04-24T23:01:05.598 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {}, 2024-04-24T23:01:05.598 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode": { 2024-04-24T23:01:05.598 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_enabled": false, 2024-04-24T23:01:05.598 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_bucket_count": 0, 2024-04-24T23:01:05.598 INFO:teuthology.orchestra.run.smithi032.stdout: "degraded_stretch_mode": 0, 2024-04-24T23:01:05.598 INFO:teuthology.orchestra.run.smithi032.stdout: "recovering_stretch_mode": 0, 2024-04-24T23:01:05.598 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_bucket": 0 2024-04-24T23:01:05.598 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:05.598 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:01:05.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.516+0000 7f05b4b29640 1 -- 172.21.15.32:0/1220770112 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f059006be30 msgr2=0x7f059006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:05.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.516+0000 7f05b4b29640 1 --2- 172.21.15.32:0/1220770112 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f059006be30 0x7f059006e2f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f05a4034000 tx=0x7f05a402e000 comp rx=0 tx=0).stop 2024-04-24T23:01:05.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.516+0000 7f05b4b29640 1 -- 172.21.15.32:0/1220770112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05b006cb30 msgr2=0x7f05b015e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:05.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.516+0000 7f05b4b29640 1 --2- 172.21.15.32:0/1220770112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05b006cb30 0x7f05b015e550 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f05a000c430 tx=0x7f05a000c8f0 comp rx=0 tx=0).stop 2024-04-24T23:01:05.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.516+0000 7f05b4b29640 1 -- 172.21.15.32:0/1220770112 shutdown_connections 2024-04-24T23:01:05.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.516+0000 7f05b4b29640 1 --2- 172.21.15.32:0/1220770112 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f059006be30 0x7f059006e2f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:05.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.516+0000 7f05b4b29640 1 --2- 172.21.15.32:0/1220770112 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f05b015fd60 0x7f05b0159560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:05.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.516+0000 7f05b4b29640 1 --2- 172.21.15.32:0/1220770112 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05b015b480 0x7f05b015bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:05.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.516+0000 7f05b4b29640 1 --2- 172.21.15.32:0/1220770112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05b006cb30 0x7f05b015e550 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:05.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.516+0000 7f05b4b29640 1 -- 172.21.15.32:0/1220770112 >> 172.21.15.32:0/1220770112 conn(0x7f05b006b110 msgr2=0x7f05b00d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:05.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.516+0000 7f05b4b29640 1 -- 172.21.15.32:0/1220770112 shutdown_connections 2024-04-24T23:01:05.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.516+0000 7f05b4b29640 1 -- 172.21.15.32:0/1220770112 wait complete. 2024-04-24T23:01:05.602 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:01:05.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.701+0000 7fad3f89a640 1 Processor -- start 2024-04-24T23:01:05.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.701+0000 7fad3f89a640 1 -- start start 2024-04-24T23:01:05.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.702+0000 7fad3f89a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fad3815e9b0 0x7fad380d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:05.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.702+0000 7fad3f89a640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fad380d4210 0x7fad3806b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:05.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.702+0000 7fad3f89a640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad380d3170 con 0x7fad380ba0e0 2024-04-24T23:01:05.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.702+0000 7fad3f89a640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fad380b8900 con 0x7fad380d4210 2024-04-24T23:01:05.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.702+0000 7fad3f89a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fad380d3450 con 0x7fad3815e9b0 2024-04-24T23:01:05.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.703+0000 7fad3d636640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fad3815e9b0 0x7fad380d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:05.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.703+0000 7fad3ce35640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fad380d4210 0x7fad3806b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:05.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.703+0000 7fad3d636640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fad3815e9b0 0x7fad380d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.32:47946/0 (socket says 172.21.15.32:47946) 2024-04-24T23:01:05.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.703+0000 7fad3d636640 1 -- 172.21.15.32:0/3405603852 learned_addr learned my addr 172.21.15.32:0/3405603852 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:05.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.703+0000 7fad3d636640 1 -- 172.21.15.32:0/3405603852 >> v2:172.21.15.121:3300/0 conn(0x7fad380d4210 msgr2=0x7fad3806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:05.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.703+0000 7fad3d636640 1 --2- 172.21.15.32:0/3405603852 >> v2:172.21.15.121:3300/0 conn(0x7fad380d4210 0x7fad3806b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:05.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.703+0000 7fad3d636640 1 -- 172.21.15.32:0/3405603852 >> v1:172.21.15.121:6789/0 conn(0x7fad380ba0e0 legacy=0x7fad38071730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:05.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.703+0000 7fad3d636640 1 -- 172.21.15.32:0/3405603852 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad380d3730 con 0x7fad3815e9b0 2024-04-24T23:01:05.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.704+0000 7fad3d636640 1 --2- 172.21.15.32:0/3405603852 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fad3815e9b0 0x7fad380d2b90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fad30009870 tx=0x7fad3002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8a1036c034eafea server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:05.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.704+0000 7fad37fff640 1 -- 172.21.15.32:0/3405603852 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad30037070 con 0x7fad3815e9b0 2024-04-24T23:01:05.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.704+0000 7fad37fff640 1 -- 172.21.15.32:0/3405603852 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad30032440 con 0x7fad3815e9b0 2024-04-24T23:01:05.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.704+0000 7fad37fff640 1 -- 172.21.15.32:0/3405603852 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad30032810 con 0x7fad3815e9b0 2024-04-24T23:01:05.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.704+0000 7fad3f89a640 1 -- 172.21.15.32:0/3405603852 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fad3815e9b0 msgr2=0x7fad380d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:05.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.704+0000 7fad3f89a640 1 --2- 172.21.15.32:0/3405603852 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fad3815e9b0 0x7fad380d2b90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fad30009870 tx=0x7fad3002a270 comp rx=0 tx=0).stop 2024-04-24T23:01:05.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.704+0000 7fad3f89a640 1 -- 172.21.15.32:0/3405603852 shutdown_connections 2024-04-24T23:01:05.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.704+0000 7fad3f89a640 1 --2- 172.21.15.32:0/3405603852 >> v2:172.21.15.121:3300/0 conn(0x7fad380d4210 0x7fad3806b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:05.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.704+0000 7fad3f89a640 1 --2- 172.21.15.32:0/3405603852 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fad3815e9b0 0x7fad380d2b90 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:05.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.704+0000 7fad3f89a640 1 -- 172.21.15.32:0/3405603852 >> 172.21.15.32:0/3405603852 conn(0x7fad380d11d0 msgr2=0x7fad380d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:05.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.705+0000 7fad3f89a640 1 -- 172.21.15.32:0/3405603852 shutdown_connections 2024-04-24T23:01:05.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.705+0000 7fad3f89a640 1 -- 172.21.15.32:0/3405603852 wait complete. 2024-04-24T23:01:05.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.706+0000 7fad3f89a640 1 Processor -- start 2024-04-24T23:01:05.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.706+0000 7fad3f89a640 1 -- start start 2024-04-24T23:01:05.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.706+0000 7fad3f89a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fad380ba0e0 0x7fad381f7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:05.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.706+0000 7fad3f89a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fad380d4210 0x7fad381f80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:05.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.706+0000 7fad3f89a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad3815e9b0 0x7fad38160920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:05.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.706+0000 7fad3f89a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fad380d2400 con 0x7fad3815e9b0 2024-04-24T23:01:05.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.706+0000 7fad3f89a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fad380d3cf0 con 0x7fad380ba0e0 2024-04-24T23:01:05.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.706+0000 7fad3f89a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fad3806b650 con 0x7fad380d4210 2024-04-24T23:01:05.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.707+0000 7fad3de37640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad3815e9b0 0x7fad38160920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:05.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.707+0000 7fad3de37640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad3815e9b0 0x7fad38160920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50144/0 (socket says 172.21.15.32:50144) 2024-04-24T23:01:05.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.707+0000 7fad3de37640 1 -- 172.21.15.32:0/1796767290 learned_addr learned my addr 172.21.15.32:0/1796767290 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:05.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.707+0000 7fad3d636640 1 --2- 172.21.15.32:0/1796767290 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fad380ba0e0 0x7fad381f7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:05.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.707+0000 7fad3ce35640 1 --2- 172.21.15.32:0/1796767290 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fad380d4210 0x7fad381f80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:05.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.707+0000 7fad3de37640 1 -- 172.21.15.32:0/1796767290 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fad380d4210 msgr2=0x7fad381f80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:05.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.707+0000 7fad3de37640 1 --2- 172.21.15.32:0/1796767290 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fad380d4210 0x7fad381f80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:05.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.707+0000 7fad3de37640 1 -- 172.21.15.32:0/1796767290 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fad380ba0e0 msgr2=0x7fad381f7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:05.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.707+0000 7fad3de37640 1 --2- 172.21.15.32:0/1796767290 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fad380ba0e0 0x7fad381f7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:05.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.707+0000 7fad3de37640 1 -- 172.21.15.32:0/1796767290 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad381463a0 con 0x7fad3815e9b0 2024-04-24T23:01:05.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.707+0000 7fad3de37640 1 --2- 172.21.15.32:0/1796767290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad3815e9b0 0x7fad38160920 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fad2800c7a0 tx=0x7fad2800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:05.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.707+0000 7fad35ffb640 1 -- 172.21.15.32:0/1796767290 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad28003b70 con 0x7fad3815e9b0 2024-04-24T23:01:05.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.707+0000 7fad35ffb640 1 -- 172.21.15.32:0/1796767290 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad28003d30 con 0x7fad3815e9b0 2024-04-24T23:01:05.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.707+0000 7fad35ffb640 1 -- 172.21.15.32:0/1796767290 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad2800aa90 con 0x7fad3815e9b0 2024-04-24T23:01:05.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.707+0000 7fad3f89a640 1 -- 172.21.15.32:0/1796767290 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad38161650 con 0x7fad3815e9b0 2024-04-24T23:01:05.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.707+0000 7fad3f89a640 1 -- 172.21.15.32:0/1796767290 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad38146940 con 0x7fad3815e9b0 2024-04-24T23:01:05.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.708+0000 7fad3f89a640 1 -- 172.21.15.32:0/1796767290 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad38146b40 con 0x7fad3815e9b0 2024-04-24T23:01:05.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.709+0000 7fad35ffb640 1 -- 172.21.15.32:0/1796767290 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fad2800ac30 con 0x7fad3815e9b0 2024-04-24T23:01:05.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.709+0000 7fad35ffb640 1 --2- 172.21.15.32:0/1796767290 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fad2406be30 0x7fad2406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:05.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.709+0000 7fad35ffb640 1 -- 172.21.15.32:0/1796767290 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5854+0+0 (secure 0 0 0) 0x7fad2808ad00 con 0x7fad3815e9b0 2024-04-24T23:01:05.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.709+0000 7fad3d636640 1 --2- 172.21.15.32:0/1796767290 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fad2406be30 0x7fad2406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:05.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.710+0000 7fad3d636640 1 --2- 172.21.15.32:0/1796767290 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fad2406be30 0x7fad2406e2f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fad30034000 tx=0x7fad3002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:05.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.713+0000 7fad35ffb640 1 -- 172.21.15.32:0/1796767290 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fad38146b40 con 0x7fad3815e9b0 2024-04-24T23:01:05.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.940+0000 7fad3f89a640 1 -- 172.21.15.32:0/1796767290 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fad38068430 con 0x7fad3815e9b0 2024-04-24T23:01:05.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.941+0000 7fad35ffb640 1 -- 172.21.15.32:0/1796767290 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5187 (secure 0 0 0) 0x7fad28058fd0 con 0x7fad3815e9b0 2024-04-24T23:01:05.943 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:01:05.943 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":20,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4994,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/1076304225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1076304225},{"type":"v1","addr":"172.21.15.32:6834","nonce":1076304225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":16},{"gid":5000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1388367206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1388367206},{"type":"v1","addr":"172.21.15.121:6835","nonce":1388367206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":5299,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2065945523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2065945523},{"type":"v1","addr":"172.21.15.121:6837","nonce":2065945523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:00:32.376194+0000","modified":"2024-04-24T23:00:34.381949+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"c","rank":0,"incarnation":18,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/3274703471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":3274703471},{"type":"v1","addr":"172.21.15.32:6835","nonce":3274703471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5631,"qdb_cluster":[5631]},"id":3}]} 2024-04-24T23:01:05.943 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 20 2024-04-24T23:01:05.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.943+0000 7fad3f89a640 1 -- 172.21.15.32:0/1796767290 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fad2406be30 msgr2=0x7fad2406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:05.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.943+0000 7fad3f89a640 1 --2- 172.21.15.32:0/1796767290 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fad2406be30 0x7fad2406e2f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fad30034000 tx=0x7fad3002e000 comp rx=0 tx=0).stop 2024-04-24T23:01:05.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.943+0000 7fad3f89a640 1 -- 172.21.15.32:0/1796767290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad3815e9b0 msgr2=0x7fad38160920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:05.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.943+0000 7fad3f89a640 1 --2- 172.21.15.32:0/1796767290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad3815e9b0 0x7fad38160920 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fad2800c7a0 tx=0x7fad2800cc60 comp rx=0 tx=0).stop 2024-04-24T23:01:05.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.943+0000 7fad3f89a640 1 -- 172.21.15.32:0/1796767290 shutdown_connections 2024-04-24T23:01:05.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.943+0000 7fad3f89a640 1 --2- 172.21.15.32:0/1796767290 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fad2406be30 0x7fad2406e2f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:05.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.943+0000 7fad3f89a640 1 --2- 172.21.15.32:0/1796767290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad3815e9b0 0x7fad38160920 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:05.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.943+0000 7fad3f89a640 1 --2- 172.21.15.32:0/1796767290 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fad380d4210 0x7fad381f80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:05.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.943+0000 7fad3f89a640 1 --2- 172.21.15.32:0/1796767290 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fad380ba0e0 0x7fad381f7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:05.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.943+0000 7fad3f89a640 1 -- 172.21.15.32:0/1796767290 >> 172.21.15.32:0/1796767290 conn(0x7fad380d11d0 msgr2=0x7fad380ca290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:05.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.943+0000 7fad3f89a640 1 -- 172.21.15.32:0/1796767290 shutdown_connections 2024-04-24T23:01:05.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:05.943+0000 7fad3f89a640 1 -- 172.21.15.32:0/1796767290 wait complete. 2024-04-24T23:01:05.956 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2024-04-24T23:01:06.070 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:01:06.072 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:01:06.076 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","whoami":0,"id":5631,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":33.057533489999997,"mdsmap_epoch":20,"osdmap_epoch":56,"osdmap_epoch_barrier":50,"uptime":37.963354940000002} 2024-04-24T23:01:06.076 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", "fs_name": "cephfs", "id": 5631, "mdsmap_epoch": 20, "osdmap_epoch": 56, "osdmap_epoch_barrier": 50, "rank_uptime": 33.05753349, "state": "up:active", "uptime": 37.96335494, "want_state": "up:active", "whoami": 0 } 2024-04-24T23:01:06.077 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-24T23:01:06.077 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:01:06.077 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:01:06.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:01:06.133 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2024-04-24T23:01:06.374 INFO:teuthology.orchestra.run.smithi032.stderr:50+0 records in 2024-04-24T23:01:06.374 INFO:teuthology.orchestra.run.smithi032.stderr:50+0 records out 2024-04-24T23:01:06.374 INFO:teuthology.orchestra.run.smithi032.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.18442 s, 284 MB/s 2024-04-24T23:01:06.375 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-24T23:01:06.375 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:01:06.412 DEBUG:teuthology.orchestra.run.smithi121:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:01:06.457 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-24T23:01:06.457 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:01:06.457 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:01:06.458 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:01:06.473 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:01:06.473 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:01:06.529 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:01:06.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.668+0000 7f6ce9bc4640 1 Processor -- start 2024-04-24T23:01:06.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.668+0000 7f6ce9bc4640 1 -- start start 2024-04-24T23:01:06.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.669+0000 7f6ce9bc4640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6ce406c5f0 0x7f6ce406c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:06.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.669+0000 7f6ce9bc4640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ce40af890 0x7f6ce415cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:06.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.669+0000 7f6ce9bc4640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ce40b6430 con 0x7f6ce40b44d0 2024-04-24T23:01:06.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.669+0000 7f6ce9bc4640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f6ce4156f80 con 0x7f6ce406c5f0 2024-04-24T23:01:06.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.669+0000 7f6ce9bc4640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f6ce40b6150 con 0x7f6ce40af890 2024-04-24T23:01:06.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.669+0000 7f6ce37fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6ce406c5f0 0x7f6ce406c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:06.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.669+0000 7f6ce37fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6ce406c5f0 0x7f6ce406c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50146/0 (socket says 172.21.15.32:50146) 2024-04-24T23:01:06.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.669+0000 7f6ce37fe640 1 -- 172.21.15.32:0/645174979 learned_addr learned my addr 172.21.15.32:0/645174979 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:06.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.669+0000 7f6ce3fff640 1 --2- 172.21.15.32:0/645174979 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ce40af890 0x7f6ce415cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:06.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.669+0000 7f6ce37fe640 1 -- 172.21.15.32:0/645174979 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ce40af890 msgr2=0x7f6ce415cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:06.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.669+0000 7f6ce37fe640 1 --2- 172.21.15.32:0/645174979 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ce40af890 0x7f6ce415cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:06.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.669+0000 7f6ce37fe640 1 -- 172.21.15.32:0/645174979 >> v1:172.21.15.121:6789/0 conn(0x7f6ce40b44d0 legacy=0x7f6ce40b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:06.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.669+0000 7f6ce37fe640 1 -- 172.21.15.32:0/645174979 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ce40b39f0 con 0x7f6ce406c5f0 2024-04-24T23:01:06.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.670+0000 7f6ce37fe640 1 --2- 172.21.15.32:0/645174979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ce406c5f0 0x7f6ce406c9d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc009870 tx=0x7f6ccc02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6408eb097adb7c1a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:06.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.670+0000 7f6ce27fc640 1 -- 172.21.15.32:0/645174979 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc037070 con 0x7f6ce406c5f0 2024-04-24T23:01:06.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.670+0000 7f6ce27fc640 1 -- 172.21.15.32:0/645174979 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ccc032440 con 0x7f6ce406c5f0 2024-04-24T23:01:06.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.670+0000 7f6ce27fc640 1 -- 172.21.15.32:0/645174979 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc032810 con 0x7f6ce406c5f0 2024-04-24T23:01:06.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.671+0000 7f6ce9bc4640 1 -- 172.21.15.32:0/645174979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ce406c5f0 msgr2=0x7f6ce406c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:06.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.671+0000 7f6ce9bc4640 1 --2- 172.21.15.32:0/645174979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ce406c5f0 0x7f6ce406c9d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc009870 tx=0x7f6ccc02a1c0 comp rx=0 tx=0).stop 2024-04-24T23:01:06.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.671+0000 7f6ce9bc4640 1 -- 172.21.15.32:0/645174979 shutdown_connections 2024-04-24T23:01:06.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.671+0000 7f6ce9bc4640 1 --2- 172.21.15.32:0/645174979 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ce40af890 0x7f6ce415cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:06.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.671+0000 7f6ce9bc4640 1 --2- 172.21.15.32:0/645174979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ce406c5f0 0x7f6ce406c9d0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:06.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.671+0000 7f6ce9bc4640 1 -- 172.21.15.32:0/645174979 >> 172.21.15.32:0/645174979 conn(0x7f6ce406b110 msgr2=0x7f6ce406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:06.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.671+0000 7f6ce9bc4640 1 -- 172.21.15.32:0/645174979 shutdown_connections 2024-04-24T23:01:06.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.671+0000 7f6ce9bc4640 1 -- 172.21.15.32:0/645174979 wait complete. 2024-04-24T23:01:06.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.672+0000 7f6ce9bc4640 1 Processor -- start 2024-04-24T23:01:06.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.672+0000 7f6ce9bc4640 1 -- start start 2024-04-24T23:01:06.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.673+0000 7f6ce9bc4640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ce406c5f0 0x7f6ce40ce990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:06.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.673+0000 7f6ce9bc4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ce40af890 0x7f6ce4144b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:06.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.673+0000 7f6ce9bc4640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ce40b44d0 0x7f6ce415f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:06.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.673+0000 7f6ce9bc4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce406cf10 con 0x7f6ce40af890 2024-04-24T23:01:06.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.673+0000 7f6ce9bc4640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce40b3fb0 con 0x7f6ce406c5f0 2024-04-24T23:01:06.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.673+0000 7f6ce9bc4640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f6ce415d000 con 0x7f6ce40b44d0 2024-04-24T23:01:06.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.673+0000 7f6ce2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ce40af890 0x7f6ce4144b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:06.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.673+0000 7f6ce2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ce40af890 0x7f6ce4144b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50160/0 (socket says 172.21.15.32:50160) 2024-04-24T23:01:06.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.673+0000 7f6ce2ffd640 1 -- 172.21.15.32:0/169696088 learned_addr learned my addr 172.21.15.32:0/169696088 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:06.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.673+0000 7f6ce37fe640 1 --2- 172.21.15.32:0/169696088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ce406c5f0 0x7f6ce40ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:06.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.673+0000 7f6ce3fff640 1 --2- 172.21.15.32:0/169696088 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ce40b44d0 0x7f6ce415f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:06.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.673+0000 7f6ce2ffd640 1 -- 172.21.15.32:0/169696088 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ce40b44d0 msgr2=0x7f6ce415f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:06.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.673+0000 7f6ce2ffd640 1 --2- 172.21.15.32:0/169696088 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ce40b44d0 0x7f6ce415f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:06.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.673+0000 7f6ce2ffd640 1 -- 172.21.15.32:0/169696088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ce406c5f0 msgr2=0x7f6ce40ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:06.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.673+0000 7f6ce2ffd640 1 --2- 172.21.15.32:0/169696088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ce406c5f0 0x7f6ce40ce990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:06.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.673+0000 7f6ce2ffd640 1 -- 172.21.15.32:0/169696088 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ce40aa840 con 0x7f6ce40af890 2024-04-24T23:01:06.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.673+0000 7f6ce2ffd640 1 --2- 172.21.15.32:0/169696088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ce40af890 0x7f6ce4144b30 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f6cd400c3f0 tx=0x7f6cd400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:06.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.673+0000 7f6cdbfff640 1 -- 172.21.15.32:0/169696088 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd4017070 con 0x7f6ce40af890 2024-04-24T23:01:06.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.673+0000 7f6cdbfff640 1 -- 172.21.15.32:0/169696088 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cd40052f0 con 0x7f6ce40af890 2024-04-24T23:01:06.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.673+0000 7f6cdbfff640 1 -- 172.21.15.32:0/169696088 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd4003850 con 0x7f6ce40af890 2024-04-24T23:01:06.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.673+0000 7f6ce9bc4640 1 -- 172.21.15.32:0/169696088 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ce40ccfb0 con 0x7f6ce40af890 2024-04-24T23:01:06.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.674+0000 7f6ce9bc4640 1 -- 172.21.15.32:0/169696088 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ce40cd570 con 0x7f6ce40af890 2024-04-24T23:01:06.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.674+0000 7f6ce9bc4640 1 -- 172.21.15.32:0/169696088 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ce4068430 con 0x7f6ce40af890 2024-04-24T23:01:06.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.675+0000 7f6cdbfff640 1 -- 172.21.15.32:0/169696088 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6cd401a070 con 0x7f6ce40af890 2024-04-24T23:01:06.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.675+0000 7f6cdbfff640 1 --2- 172.21.15.32:0/169696088 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6cc406be30 0x7f6cc406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:06.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.675+0000 7f6cdbfff640 1 -- 172.21.15.32:0/169696088 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5854+0+0 (secure 0 0 0) 0x7f6cd408a0a0 con 0x7f6ce40af890 2024-04-24T23:01:06.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.676+0000 7f6ce37fe640 1 --2- 172.21.15.32:0/169696088 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6cc406be30 0x7f6cc406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:06.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.676+0000 7f6ce37fe640 1 --2- 172.21.15.32:0/169696088 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6cc406be30 0x7f6cc406e2f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc034000 tx=0x7f6ccc02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:06.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.680+0000 7f6cdbfff640 1 -- 172.21.15.32:0/169696088 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6cd4058400 con 0x7f6ce40af890 2024-04-24T23:01:06.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.863+0000 7f6ce9bc4640 1 -- 172.21.15.32:0/169696088 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6ce415d7a0 con 0x7f6ce40af890 2024-04-24T23:01:06.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.864+0000 7f6cdbfff640 1 -- 172.21.15.32:0/169696088 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) ==== 74+0+16912 (secure 0 0 0) 0x7f6cd405c700 con 0x7f6ce40af890 2024-04-24T23:01:06.866 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:01:06.866 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":56,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:01:00.283572+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:00:30.665592+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-24T23:00:31.357756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.6","mappings":[{"from":5,"to":7}]},{"pgid":"7.c","mappings":[{"from":5,"to":7}]},{"pgid":"7.12","mappings":[{"from":5,"to":7}]},{"pgid":"7.31","mappings":[{"from":5,"to":7}]},{"pgid":"7.35","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:01:06.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.867+0000 7f6ce9bc4640 1 -- 172.21.15.32:0/169696088 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6cc406be30 msgr2=0x7f6cc406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:06.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.867+0000 7f6ce9bc4640 1 --2- 172.21.15.32:0/169696088 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6cc406be30 0x7f6cc406e2f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc034000 tx=0x7f6ccc02e000 comp rx=0 tx=0).stop 2024-04-24T23:01:06.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.867+0000 7f6ce9bc4640 1 -- 172.21.15.32:0/169696088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ce40af890 msgr2=0x7f6ce4144b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:06.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.867+0000 7f6ce9bc4640 1 --2- 172.21.15.32:0/169696088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ce40af890 0x7f6ce4144b30 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f6cd400c3f0 tx=0x7f6cd400c8b0 comp rx=0 tx=0).stop 2024-04-24T23:01:06.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.867+0000 7f6ce9bc4640 1 -- 172.21.15.32:0/169696088 shutdown_connections 2024-04-24T23:01:06.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.867+0000 7f6ce9bc4640 1 --2- 172.21.15.32:0/169696088 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6cc406be30 0x7f6cc406e2f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:06.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.867+0000 7f6ce9bc4640 1 --2- 172.21.15.32:0/169696088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ce40af890 0x7f6ce4144b30 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:06.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.867+0000 7f6ce9bc4640 1 --2- 172.21.15.32:0/169696088 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ce40b44d0 0x7f6ce415f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:06.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.867+0000 7f6ce9bc4640 1 --2- 172.21.15.32:0/169696088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ce406c5f0 0x7f6ce40ce990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:06.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.867+0000 7f6ce9bc4640 1 -- 172.21.15.32:0/169696088 >> 172.21.15.32:0/169696088 conn(0x7f6ce406b110 msgr2=0x7f6ce40b13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:06.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.867+0000 7f6ce9bc4640 1 -- 172.21.15.32:0/169696088 shutdown_connections 2024-04-24T23:01:06.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.867+0000 7f6ce9bc4640 1 -- 172.21.15.32:0/169696088 wait complete. 2024-04-24T23:01:06.880 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:00:30.665592+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:00:31.357756+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:01:06.880 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:01:06.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.982+0000 7fa2bd4d4640 1 Processor -- start 2024-04-24T23:01:06.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.982+0000 7fa2bd4d4640 1 -- start start 2024-04-24T23:01:06.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.983+0000 7fa2bd4d4640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2b815e9b0 0x7fa2b80d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:06.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.983+0000 7fa2bd4d4640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa2b806b820 0x7fa2b80b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:06.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.983+0000 7fa2bd4d4640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2b80d3170 con 0x7fa2b80d4210 2024-04-24T23:01:06.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.983+0000 7fa2bd4d4640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fa2b80d3450 con 0x7fa2b806b820 2024-04-24T23:01:06.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.983+0000 7fa2bd4d4640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa2b80d3730 con 0x7fa2b815e9b0 2024-04-24T23:01:06.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.983+0000 7fa2b67fc640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa2b80d4210 0x7fa2b806b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56772/0 (socket says 172.21.15.32:56772) 2024-04-24T23:01:06.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.983+0000 7fa2b67fc640 1 -- 172.21.15.32:0/3551933629 learned_addr learned my addr 172.21.15.32:0/3551933629 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:01:06.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.983+0000 7fa2b6ffd640 1 --2- 172.21.15.32:0/3551933629 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2b815e9b0 0x7fa2b80d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:06.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.983+0000 7fa2b77fe640 1 --2- 172.21.15.32:0/3551933629 >> v2:172.21.15.121:3300/0 conn(0x7fa2b806b820 0x7fa2b80b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:06.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.983+0000 7fa2b5ffb640 1 -- 172.21.15.32:0/3551933629 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1908070139 0 0) 0x7fa2b80d3170 con 0x7fa2b80d4210 2024-04-24T23:01:06.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.983+0000 7fa2b5ffb640 1 -- 172.21.15.32:0/3551933629 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa2a0003520 con 0x7fa2b80d4210 2024-04-24T23:01:06.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.983+0000 7fa2b6ffd640 1 -- 172.21.15.32:0/3551933629 >> v2:172.21.15.121:3300/0 conn(0x7fa2b806b820 msgr2=0x7fa2b80b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:06.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.983+0000 7fa2b6ffd640 1 --2- 172.21.15.32:0/3551933629 >> v2:172.21.15.121:3300/0 conn(0x7fa2b806b820 0x7fa2b80b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:06.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.983+0000 7fa2b6ffd640 1 -- 172.21.15.32:0/3551933629 >> v1:172.21.15.32:6789/0 conn(0x7fa2b80d4210 legacy=0x7fa2b806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:06.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.983+0000 7fa2b6ffd640 1 -- 172.21.15.32:0/3551933629 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2b80d3a10 con 0x7fa2b815e9b0 2024-04-24T23:01:06.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.984+0000 7fa2b6ffd640 1 --2- 172.21.15.32:0/3551933629 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2b815e9b0 0x7fa2b80d2b90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa2b00097a0 tx=0x7fa2b002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e06e884a6fb4fbbc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:06.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.984+0000 7fa2b5ffb640 1 -- 172.21.15.32:0/3551933629 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b0037070 con 0x7fa2b815e9b0 2024-04-24T23:01:06.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.984+0000 7fa2b5ffb640 1 -- 172.21.15.32:0/3551933629 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2b0032440 con 0x7fa2b815e9b0 2024-04-24T23:01:06.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.984+0000 7fa2b5ffb640 1 -- 172.21.15.32:0/3551933629 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b00327f0 con 0x7fa2b815e9b0 2024-04-24T23:01:06.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.985+0000 7fa2bd4d4640 1 -- 172.21.15.32:0/3551933629 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2b815e9b0 msgr2=0x7fa2b80d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:06.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.985+0000 7fa2bd4d4640 1 --2- 172.21.15.32:0/3551933629 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2b815e9b0 0x7fa2b80d2b90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa2b00097a0 tx=0x7fa2b002a180 comp rx=0 tx=0).stop 2024-04-24T23:01:06.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.985+0000 7fa2bd4d4640 1 -- 172.21.15.32:0/3551933629 shutdown_connections 2024-04-24T23:01:06.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.985+0000 7fa2bd4d4640 1 --2- 172.21.15.32:0/3551933629 >> v2:172.21.15.121:3300/0 conn(0x7fa2b806b820 0x7fa2b80b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:06.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.985+0000 7fa2bd4d4640 1 --2- 172.21.15.32:0/3551933629 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2b815e9b0 0x7fa2b80d2b90 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:06.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.985+0000 7fa2bd4d4640 1 -- 172.21.15.32:0/3551933629 >> 172.21.15.32:0/3551933629 conn(0x7fa2b80d11d0 msgr2=0x7fa2b80d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:06.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.985+0000 7fa2bd4d4640 1 -- 172.21.15.32:0/3551933629 shutdown_connections 2024-04-24T23:01:06.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.985+0000 7fa2bd4d4640 1 -- 172.21.15.32:0/3551933629 wait complete. 2024-04-24T23:01:06.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.986+0000 7fa2bd4d4640 1 Processor -- start 2024-04-24T23:01:06.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.986+0000 7fa2bd4d4640 1 -- start start 2024-04-24T23:01:06.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.987+0000 7fa2bd4d4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2b806b820 0x7fa2b8152550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:06.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.987+0000 7fa2bd4d4640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2b80d4210 0x7fa2b80cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:06.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.987+0000 7fa2bd4d4640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2b815e9b0 0x7fa2b81f48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:06.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.987+0000 7fa2bd4d4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b80ba0e0 con 0x7fa2b806b820 2024-04-24T23:01:06.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.987+0000 7fa2bd4d4640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b80d2400 con 0x7fa2b815e9b0 2024-04-24T23:01:06.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.987+0000 7fa2bd4d4640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa2b80ba260 con 0x7fa2b80d4210 2024-04-24T23:01:06.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.987+0000 7fa2b6ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2b806b820 0x7fa2b8152550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:06.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.987+0000 7fa2b6ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2b806b820 0x7fa2b8152550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50174/0 (socket says 172.21.15.32:50174) 2024-04-24T23:01:06.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.987+0000 7fa2b6ffd640 1 -- 172.21.15.32:0/1764885358 learned_addr learned my addr 172.21.15.32:0/1764885358 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:06.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.987+0000 7fa2b77fe640 1 --2- 172.21.15.32:0/1764885358 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2b815e9b0 0x7fa2b81f48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:06.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.987+0000 7fa2b67fc640 1 --2- 172.21.15.32:0/1764885358 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2b80d4210 0x7fa2b80cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:06.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.987+0000 7fa2b6ffd640 1 -- 172.21.15.32:0/1764885358 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2b80d4210 msgr2=0x7fa2b80cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:06.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.987+0000 7fa2b6ffd640 1 --2- 172.21.15.32:0/1764885358 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2b80d4210 0x7fa2b80cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:06.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.987+0000 7fa2b6ffd640 1 -- 172.21.15.32:0/1764885358 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2b815e9b0 msgr2=0x7fa2b81f48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:06.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.987+0000 7fa2b6ffd640 1 --2- 172.21.15.32:0/1764885358 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2b815e9b0 0x7fa2b81f48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:06.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.987+0000 7fa2b6ffd640 1 -- 172.21.15.32:0/1764885358 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2b80cd310 con 0x7fa2b806b820 2024-04-24T23:01:06.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.988+0000 7fa2b6ffd640 1 --2- 172.21.15.32:0/1764885358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2b806b820 0x7fa2b8152550 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fa2b002a1c0 tx=0x7fa2b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:06.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.988+0000 7fa29f7fe640 1 -- 172.21.15.32:0/1764885358 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b0041020 con 0x7fa2b806b820 2024-04-24T23:01:06.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.988+0000 7fa29f7fe640 1 -- 172.21.15.32:0/1764885358 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2b003d620 con 0x7fa2b806b820 2024-04-24T23:01:06.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.988+0000 7fa2bd4d4640 1 -- 172.21.15.32:0/1764885358 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2b80cd5f0 con 0x7fa2b806b820 2024-04-24T23:01:06.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.988+0000 7fa29f7fe640 1 -- 172.21.15.32:0/1764885358 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b0037040 con 0x7fa2b806b820 2024-04-24T23:01:06.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.988+0000 7fa2bd4d4640 1 -- 172.21.15.32:0/1764885358 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2b815f840 con 0x7fa2b806b820 2024-04-24T23:01:06.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.989+0000 7fa2bd4d4640 1 -- 172.21.15.32:0/1764885358 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2b8068430 con 0x7fa2b806b820 2024-04-24T23:01:06.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.989+0000 7fa29f7fe640 1 -- 172.21.15.32:0/1764885358 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa2b0044020 con 0x7fa2b806b820 2024-04-24T23:01:06.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.990+0000 7fa29f7fe640 1 --2- 172.21.15.32:0/1764885358 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa2a406be60 0x7fa2a406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:06.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.990+0000 7fa29f7fe640 1 -- 172.21.15.32:0/1764885358 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5854+0+0 (secure 0 0 0) 0x7fa2b00ab520 con 0x7fa2b806b820 2024-04-24T23:01:06.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.990+0000 7fa2b67fc640 1 --2- 172.21.15.32:0/1764885358 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa2a406be60 0x7fa2a406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:06.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.990+0000 7fa2b67fc640 1 --2- 172.21.15.32:0/1764885358 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa2a406be60 0x7fa2a406e320 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa2b815b110 tx=0x7fa2ac009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:06.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:06.994+0000 7fa29f7fe640 1 -- 172.21.15.32:0/1764885358 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa2b004b030 con 0x7fa2b806b820 2024-04-24T23:01:07.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.179+0000 7fa2bd4d4640 1 -- 172.21.15.32:0/1764885358 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa2b80d5450 con 0x7fa2b806b820 2024-04-24T23:01:07.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.179+0000 7fa29f7fe640 1 -- 172.21.15.32:0/1764885358 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v56) ==== 93+0+10 (secure 0 0 0) 0x7fa2b0079800 con 0x7fa2b806b820 2024-04-24T23:01:07.181 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T23:01:07.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.181+0000 7fa2bd4d4640 1 -- 172.21.15.32:0/1764885358 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa2a406be60 msgr2=0x7fa2a406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:07.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.181+0000 7fa2bd4d4640 1 --2- 172.21.15.32:0/1764885358 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa2a406be60 0x7fa2a406e320 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa2b815b110 tx=0x7fa2ac009040 comp rx=0 tx=0).stop 2024-04-24T23:01:07.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.181+0000 7fa2bd4d4640 1 -- 172.21.15.32:0/1764885358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2b806b820 msgr2=0x7fa2b8152550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:07.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.181+0000 7fa2bd4d4640 1 --2- 172.21.15.32:0/1764885358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2b806b820 0x7fa2b8152550 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fa2b002a1c0 tx=0x7fa2b00046c0 comp rx=0 tx=0).stop 2024-04-24T23:01:07.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.181+0000 7fa2bd4d4640 1 -- 172.21.15.32:0/1764885358 shutdown_connections 2024-04-24T23:01:07.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.181+0000 7fa2bd4d4640 1 --2- 172.21.15.32:0/1764885358 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa2a406be60 0x7fa2a406e320 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:07.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.181+0000 7fa2bd4d4640 1 --2- 172.21.15.32:0/1764885358 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2b815e9b0 0x7fa2b81f48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:07.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.181+0000 7fa2bd4d4640 1 --2- 172.21.15.32:0/1764885358 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2b80d4210 0x7fa2b80cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:07.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.181+0000 7fa2bd4d4640 1 --2- 172.21.15.32:0/1764885358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2b806b820 0x7fa2b8152550 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:07.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.181+0000 7fa2bd4d4640 1 -- 172.21.15.32:0/1764885358 >> 172.21.15.32:0/1764885358 conn(0x7fa2b80d11d0 msgr2=0x7fa2b80b92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:07.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.181+0000 7fa2bd4d4640 1 -- 172.21.15.32:0/1764885358 shutdown_connections 2024-04-24T23:01:07.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.181+0000 7fa2bd4d4640 1 -- 172.21.15.32:0/1764885358 wait complete. 2024-04-24T23:01:07.194 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:01:07.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.296+0000 7f1e987ce640 1 Processor -- start 2024-04-24T23:01:07.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.296+0000 7f1e987ce640 1 -- start start 2024-04-24T23:01:07.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.297+0000 7f1e987ce640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1e90162e10 0x7f1e900ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:07.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.297+0000 7f1e987ce640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e900cd270 con 0x7f1e900b5af0 2024-04-24T23:01:07.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.297+0000 7f1e987ce640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e90071610 con 0x7f1e90162320 2024-04-24T23:01:07.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.297+0000 7f1e987ce640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f1e900b5660 con 0x7f1e90162e10 2024-04-24T23:01:07.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.297+0000 7f1e96d6b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1e90162e10 0x7f1e900ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:07.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.297+0000 7f1e95d69640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1e900b5af0 0x7f1e9015ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56774/0 (socket says 172.21.15.32:56774) 2024-04-24T23:01:07.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.297+0000 7f1e95d69640 1 -- 172.21.15.32:0/2343295546 learned_addr learned my addr 172.21.15.32:0/2343295546 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:01:07.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.297+0000 7f1e96d6b640 1 -- 172.21.15.32:0/2343295546 >> v1:172.21.15.121:6789/0 conn(0x7f1e90162320 legacy=0x7f1e90162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:07.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.297+0000 7f1e96d6b640 1 -- 172.21.15.32:0/2343295546 >> v1:172.21.15.32:6789/0 conn(0x7f1e900b5af0 legacy=0x7f1e9015ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:07.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.297+0000 7f1e96d6b640 1 -- 172.21.15.32:0/2343295546 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e900718f0 con 0x7f1e90162e10 2024-04-24T23:01:07.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.298+0000 7f1e96d6b640 1 --2- 172.21.15.32:0/2343295546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e90162e10 0x7f1e900ccc90 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f1e88009d90 tx=0x7f1e8802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=121e137920a8e036 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:07.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.298+0000 7f1e95568640 1 -- 172.21.15.32:0/2343295546 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e88007740 con 0x7f1e90162e10 2024-04-24T23:01:07.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.298+0000 7f1e95568640 1 -- 172.21.15.32:0/2343295546 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e880078e0 con 0x7f1e90162e10 2024-04-24T23:01:07.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.298+0000 7f1e95568640 1 -- 172.21.15.32:0/2343295546 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e88007cb0 con 0x7f1e90162e10 2024-04-24T23:01:07.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.299+0000 7f1e987ce640 1 -- 172.21.15.32:0/2343295546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e90162e10 msgr2=0x7f1e900ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:07.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.299+0000 7f1e987ce640 1 --2- 172.21.15.32:0/2343295546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e90162e10 0x7f1e900ccc90 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f1e88009d90 tx=0x7f1e8802a660 comp rx=0 tx=0).stop 2024-04-24T23:01:07.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.299+0000 7f1e987ce640 1 -- 172.21.15.32:0/2343295546 shutdown_connections 2024-04-24T23:01:07.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.299+0000 7f1e987ce640 1 --2- 172.21.15.32:0/2343295546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e90162e10 0x7f1e900ccc90 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:07.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.299+0000 7f1e987ce640 1 -- 172.21.15.32:0/2343295546 >> 172.21.15.32:0/2343295546 conn(0x7f1e900d0ae0 msgr2=0x7f1e900d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:07.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.299+0000 7f1e987ce640 1 -- 172.21.15.32:0/2343295546 shutdown_connections 2024-04-24T23:01:07.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.299+0000 7f1e987ce640 1 -- 172.21.15.32:0/2343295546 wait complete. 2024-04-24T23:01:07.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.300+0000 7f1e987ce640 1 Processor -- start 2024-04-24T23:01:07.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.300+0000 7f1e987ce640 1 -- start start 2024-04-24T23:01:07.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.301+0000 7f1e987ce640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1e900b5af0 0x7f1e900ce070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:07.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.301+0000 7f1e987ce640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e90162320 0x7f1e90152790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:07.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.301+0000 7f1e987ce640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e90162e10 0x7f1e900cf730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:07.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.301+0000 7f1e987ce640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1e90071bd0 con 0x7f1e90162320 2024-04-24T23:01:07.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.301+0000 7f1e987ce640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1e90067950 con 0x7f1e90162e10 2024-04-24T23:01:07.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.301+0000 7f1e987ce640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f1e90071d50 con 0x7f1e900b5af0 2024-04-24T23:01:07.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.301+0000 7f1e95d69640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e90162320 0x7f1e90152790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:07.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.301+0000 7f1e95d69640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e90162320 0x7f1e90152790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50182/0 (socket says 172.21.15.32:50182) 2024-04-24T23:01:07.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.301+0000 7f1e95d69640 1 -- 172.21.15.32:0/1598327097 learned_addr learned my addr 172.21.15.32:0/1598327097 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:07.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.301+0000 7f1e9656a640 1 --2- 172.21.15.32:0/1598327097 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1e900b5af0 0x7f1e900ce070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:07.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.301+0000 7f1e96d6b640 1 --2- 172.21.15.32:0/1598327097 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e90162e10 0x7f1e900cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:07.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.301+0000 7f1e95d69640 1 -- 172.21.15.32:0/1598327097 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1e900b5af0 msgr2=0x7f1e900ce070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:07.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.301+0000 7f1e95d69640 1 --2- 172.21.15.32:0/1598327097 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1e900b5af0 0x7f1e900ce070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:07.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.301+0000 7f1e95d69640 1 -- 172.21.15.32:0/1598327097 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e90162e10 msgr2=0x7f1e900cf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:07.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.301+0000 7f1e95d69640 1 --2- 172.21.15.32:0/1598327097 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e90162e10 0x7f1e900cf730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:07.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.301+0000 7f1e95d69640 1 -- 172.21.15.32:0/1598327097 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e900d0260 con 0x7f1e90162320 2024-04-24T23:01:07.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.302+0000 7f1e95d69640 1 --2- 172.21.15.32:0/1598327097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e90162320 0x7f1e90152790 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f1e8000c4a0 tx=0x7f1e8000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:07.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.303+0000 7f1e86ffd640 1 -- 172.21.15.32:0/1598327097 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e80017070 con 0x7f1e90162320 2024-04-24T23:01:07.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.303+0000 7f1e86ffd640 1 -- 172.21.15.32:0/1598327097 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e80005ce0 con 0x7f1e90162320 2024-04-24T23:01:07.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.303+0000 7f1e987ce640 1 -- 172.21.15.32:0/1598327097 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e900cff80 con 0x7f1e90162320 2024-04-24T23:01:07.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.303+0000 7f1e86ffd640 1 -- 172.21.15.32:0/1598327097 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e80007270 con 0x7f1e90162320 2024-04-24T23:01:07.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.303+0000 7f1e987ce640 1 -- 172.21.15.32:0/1598327097 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e900cd870 con 0x7f1e90162320 2024-04-24T23:01:07.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.304+0000 7f1e987ce640 1 -- 172.21.15.32:0/1598327097 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e90068430 con 0x7f1e90162320 2024-04-24T23:01:07.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.304+0000 7f1e86ffd640 1 -- 172.21.15.32:0/1598327097 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1e8001a070 con 0x7f1e90162320 2024-04-24T23:01:07.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.305+0000 7f1e86ffd640 1 --2- 172.21.15.32:0/1598327097 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1e7406be30 0x7f1e7406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:07.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.305+0000 7f1e86ffd640 1 -- 172.21.15.32:0/1598327097 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5854+0+0 (secure 0 0 0) 0x7f1e8008a150 con 0x7f1e90162320 2024-04-24T23:01:07.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.305+0000 7f1e9656a640 1 --2- 172.21.15.32:0/1598327097 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1e7406be30 0x7f1e7406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:07.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.305+0000 7f1e9656a640 1 --2- 172.21.15.32:0/1598327097 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1e7406be30 0x7f1e7406e2f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1e9015bc90 tx=0x7f1e8c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:07.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.309+0000 7f1e86ffd640 1 -- 172.21.15.32:0/1598327097 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1e80058420 con 0x7f1e90162320 2024-04-24T23:01:07.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.491+0000 7f1e987ce640 1 -- 172.21.15.32:0/1598327097 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1e90067730 con 0x7f1e90162320 2024-04-24T23:01:07.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.491+0000 7f1e86ffd640 1 -- 172.21.15.32:0/1598327097 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v56) ==== 104+0+11 (secure 0 0 0) 0x7f1e8005c720 con 0x7f1e90162320 2024-04-24T23:01:07.493 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-24T23:01:07.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.493+0000 7f1e987ce640 1 -- 172.21.15.32:0/1598327097 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1e7406be30 msgr2=0x7f1e7406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:07.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.493+0000 7f1e987ce640 1 --2- 172.21.15.32:0/1598327097 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1e7406be30 0x7f1e7406e2f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1e9015bc90 tx=0x7f1e8c009040 comp rx=0 tx=0).stop 2024-04-24T23:01:07.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.493+0000 7f1e987ce640 1 -- 172.21.15.32:0/1598327097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e90162320 msgr2=0x7f1e90152790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:07.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.493+0000 7f1e987ce640 1 --2- 172.21.15.32:0/1598327097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e90162320 0x7f1e90152790 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f1e8000c4a0 tx=0x7f1e8000c960 comp rx=0 tx=0).stop 2024-04-24T23:01:07.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.493+0000 7f1e987ce640 1 -- 172.21.15.32:0/1598327097 shutdown_connections 2024-04-24T23:01:07.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.493+0000 7f1e987ce640 1 --2- 172.21.15.32:0/1598327097 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1e7406be30 0x7f1e7406e2f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:07.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.493+0000 7f1e987ce640 1 --2- 172.21.15.32:0/1598327097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e90162320 0x7f1e90152790 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:07.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.493+0000 7f1e987ce640 1 --2- 172.21.15.32:0/1598327097 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e90162e10 0x7f1e900cf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:07.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.493+0000 7f1e987ce640 1 --2- 172.21.15.32:0/1598327097 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1e900b5af0 0x7f1e900ce070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:07.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.493+0000 7f1e987ce640 1 -- 172.21.15.32:0/1598327097 >> 172.21.15.32:0/1598327097 conn(0x7f1e900d0ae0 msgr2=0x7f1e900b3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:07.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.493+0000 7f1e987ce640 1 -- 172.21.15.32:0/1598327097 shutdown_connections 2024-04-24T23:01:07.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.493+0000 7f1e987ce640 1 -- 172.21.15.32:0/1598327097 wait complete. 2024-04-24T23:01:07.505 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:01:07.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.607+0000 7f1d5f8da640 1 Processor -- start 2024-04-24T23:01:07.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.608+0000 7f1d5f8da640 1 -- start start 2024-04-24T23:01:07.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.608+0000 7f1d5f8da640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1d58074210 0x7f1d5806c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:07.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.608+0000 7f1d5f8da640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1d580b0b80 0x7f1d580745f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:07.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.608+0000 7f1d5f8da640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d580b0770 con 0x7f1d580ae140 2024-04-24T23:01:07.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.608+0000 7f1d5f8da640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f1d580b1c20 con 0x7f1d58074210 2024-04-24T23:01:07.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.608+0000 7f1d5f8da640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f1d58074b30 con 0x7f1d580b0b80 2024-04-24T23:01:07.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.609+0000 7f1d5d676640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1d58074210 0x7f1d5806c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:07.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.609+0000 7f1d5de77640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1d580ae140 0x7f1d580b20b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56778/0 (socket says 172.21.15.32:56778) 2024-04-24T23:01:07.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.609+0000 7f1d5de77640 1 -- 172.21.15.32:0/469583650 learned_addr learned my addr 172.21.15.32:0/469583650 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:01:07.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.609+0000 7f1d5ce75640 1 --2- 172.21.15.32:0/469583650 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1d580b0b80 0x7f1d580745f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:07.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.609+0000 7f1d5d676640 1 -- 172.21.15.32:0/469583650 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1d580b0b80 msgr2=0x7f1d580745f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:07.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.609+0000 7f1d5d676640 1 --2- 172.21.15.32:0/469583650 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1d580b0b80 0x7f1d580745f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:07.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.609+0000 7f1d5d676640 1 -- 172.21.15.32:0/469583650 >> v1:172.21.15.32:6789/0 conn(0x7f1d580ae140 legacy=0x7f1d580b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:07.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.609+0000 7f1d5d676640 1 -- 172.21.15.32:0/469583650 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d580b1010 con 0x7f1d58074210 2024-04-24T23:01:07.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.609+0000 7f1d5ce75640 1 --2- 172.21.15.32:0/469583650 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1d580b0b80 0x7f1d580745f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:01:07.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.610+0000 7f1d5d676640 1 --2- 172.21.15.32:0/469583650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d58074210 0x7f1d5806c480 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f1d500097a0 tx=0x7f1d5002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8acb10a8883b641e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:07.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.610+0000 7f1d57fff640 1 -- 172.21.15.32:0/469583650 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d50037070 con 0x7f1d58074210 2024-04-24T23:01:07.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.610+0000 7f1d57fff640 1 -- 172.21.15.32:0/469583650 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d50032440 con 0x7f1d58074210 2024-04-24T23:01:07.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.610+0000 7f1d57fff640 1 -- 172.21.15.32:0/469583650 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d50032810 con 0x7f1d58074210 2024-04-24T23:01:07.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.610+0000 7f1d5f8da640 1 -- 172.21.15.32:0/469583650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d58074210 msgr2=0x7f1d5806c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:07.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.610+0000 7f1d5f8da640 1 --2- 172.21.15.32:0/469583650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d58074210 0x7f1d5806c480 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f1d500097a0 tx=0x7f1d5002a180 comp rx=0 tx=0).stop 2024-04-24T23:01:07.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.610+0000 7f1d5f8da640 1 -- 172.21.15.32:0/469583650 shutdown_connections 2024-04-24T23:01:07.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.610+0000 7f1d5f8da640 1 --2- 172.21.15.32:0/469583650 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1d580b0b80 0x7f1d580745f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:07.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.610+0000 7f1d5f8da640 1 --2- 172.21.15.32:0/469583650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d58074210 0x7f1d5806c480 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:07.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.610+0000 7f1d5f8da640 1 -- 172.21.15.32:0/469583650 >> 172.21.15.32:0/469583650 conn(0x7f1d580b2e60 msgr2=0x7f1d580b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:07.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.610+0000 7f1d5f8da640 1 -- 172.21.15.32:0/469583650 shutdown_connections 2024-04-24T23:01:07.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.610+0000 7f1d5f8da640 1 -- 172.21.15.32:0/469583650 wait complete. 2024-04-24T23:01:07.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.612+0000 7f1d5f8da640 1 Processor -- start 2024-04-24T23:01:07.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.612+0000 7f1d5f8da640 1 -- start start 2024-04-24T23:01:07.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.612+0000 7f1d5f8da640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d58074210 0x7f1d581f42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:07.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.612+0000 7f1d5f8da640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1d580ae140 0x7f1d580c6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:07.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.612+0000 7f1d5f8da640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d5806d570 0x7f1d5806da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:07.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.612+0000 7f1d5f8da640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1d580b18b0 con 0x7f1d5806d570 2024-04-24T23:01:07.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.612+0000 7f1d5f8da640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1d580b15d0 con 0x7f1d58074210 2024-04-24T23:01:07.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.612+0000 7f1d5f8da640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f1d580b3ba0 con 0x7f1d580ae140 2024-04-24T23:01:07.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.612+0000 7f1d5de77640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d5806d570 0x7f1d5806da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:07.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.612+0000 7f1d5de77640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d5806d570 0x7f1d5806da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50196/0 (socket says 172.21.15.32:50196) 2024-04-24T23:01:07.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.612+0000 7f1d5de77640 1 -- 172.21.15.32:0/1287881371 learned_addr learned my addr 172.21.15.32:0/1287881371 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:07.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.612+0000 7f1d5ce75640 1 --2- 172.21.15.32:0/1287881371 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1d580ae140 0x7f1d580c6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:07.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.613+0000 7f1d5d676640 1 --2- 172.21.15.32:0/1287881371 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d58074210 0x7f1d581f42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:07.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.613+0000 7f1d5de77640 1 -- 172.21.15.32:0/1287881371 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1d580ae140 msgr2=0x7f1d580c6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:07.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.613+0000 7f1d5de77640 1 --2- 172.21.15.32:0/1287881371 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1d580ae140 0x7f1d580c6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:07.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.613+0000 7f1d5de77640 1 -- 172.21.15.32:0/1287881371 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d58074210 msgr2=0x7f1d581f42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:07.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.613+0000 7f1d5de77640 1 --2- 172.21.15.32:0/1287881371 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d58074210 0x7f1d581f42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:07.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.613+0000 7f1d5de77640 1 -- 172.21.15.32:0/1287881371 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d580cbd00 con 0x7f1d5806d570 2024-04-24T23:01:07.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.613+0000 7f1d5de77640 1 --2- 172.21.15.32:0/1287881371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d5806d570 0x7f1d5806da00 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f1d4800cbc0 tx=0x7f1d4800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:07.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.613+0000 7f1d55ffb640 1 -- 172.21.15.32:0/1287881371 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d48017070 con 0x7f1d5806d570 2024-04-24T23:01:07.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.613+0000 7f1d55ffb640 1 -- 172.21.15.32:0/1287881371 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d4800ad40 con 0x7f1d5806d570 2024-04-24T23:01:07.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.613+0000 7f1d55ffb640 1 -- 172.21.15.32:0/1287881371 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d48012600 con 0x7f1d5806d570 2024-04-24T23:01:07.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.613+0000 7f1d5f8da640 1 -- 172.21.15.32:0/1287881371 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d580cbfe0 con 0x7f1d5806d570 2024-04-24T23:01:07.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.613+0000 7f1d5f8da640 1 -- 172.21.15.32:0/1287881371 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d580cc5a0 con 0x7f1d5806d570 2024-04-24T23:01:07.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.614+0000 7f1d5f8da640 1 -- 172.21.15.32:0/1287881371 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d58068430 con 0x7f1d5806d570 2024-04-24T23:01:07.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.615+0000 7f1d55ffb640 1 -- 172.21.15.32:0/1287881371 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1d4801a070 con 0x7f1d5806d570 2024-04-24T23:01:07.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.615+0000 7f1d55ffb640 1 --2- 172.21.15.32:0/1287881371 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1d4406be30 0x7f1d4406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:07.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.615+0000 7f1d55ffb640 1 -- 172.21.15.32:0/1287881371 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5854+0+0 (secure 0 0 0) 0x7f1d4808ab60 con 0x7f1d5806d570 2024-04-24T23:01:07.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.615+0000 7f1d5d676640 1 --2- 172.21.15.32:0/1287881371 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1d4406be30 0x7f1d4406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:07.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.616+0000 7f1d5d676640 1 --2- 172.21.15.32:0/1287881371 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1d4406be30 0x7f1d4406e2f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1d50034000 tx=0x7f1d5002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:07.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.620+0000 7f1d55ffb640 1 -- 172.21.15.32:0/1287881371 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1d48058e30 con 0x7f1d5806d570 2024-04-24T23:01:07.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.805+0000 7f1d5f8da640 1 -- 172.21.15.32:0/1287881371 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1d5806b220 con 0x7f1d5806d570 2024-04-24T23:01:07.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.806+0000 7f1d55ffb640 1 -- 172.21.15.32:0/1287881371 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v56) ==== 100+0+11 (secure 0 0 0) 0x7f1d4805d130 con 0x7f1d5806d570 2024-04-24T23:01:07.807 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-24T23:01:07.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.808+0000 7f1d5f8da640 1 -- 172.21.15.32:0/1287881371 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1d4406be30 msgr2=0x7f1d4406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:07.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.808+0000 7f1d5f8da640 1 --2- 172.21.15.32:0/1287881371 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1d4406be30 0x7f1d4406e2f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1d50034000 tx=0x7f1d5002e000 comp rx=0 tx=0).stop 2024-04-24T23:01:07.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.808+0000 7f1d5f8da640 1 -- 172.21.15.32:0/1287881371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d5806d570 msgr2=0x7f1d5806da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:07.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.808+0000 7f1d5f8da640 1 --2- 172.21.15.32:0/1287881371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d5806d570 0x7f1d5806da00 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f1d4800cbc0 tx=0x7f1d4800a430 comp rx=0 tx=0).stop 2024-04-24T23:01:07.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.808+0000 7f1d5f8da640 1 -- 172.21.15.32:0/1287881371 shutdown_connections 2024-04-24T23:01:07.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.808+0000 7f1d5f8da640 1 --2- 172.21.15.32:0/1287881371 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1d4406be30 0x7f1d4406e2f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:07.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.808+0000 7f1d5f8da640 1 --2- 172.21.15.32:0/1287881371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d5806d570 0x7f1d5806da00 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:07.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.808+0000 7f1d5f8da640 1 --2- 172.21.15.32:0/1287881371 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1d580ae140 0x7f1d580c6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:07.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.808+0000 7f1d5f8da640 1 --2- 172.21.15.32:0/1287881371 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d58074210 0x7f1d581f42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:07.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.808+0000 7f1d5f8da640 1 -- 172.21.15.32:0/1287881371 >> 172.21.15.32:0/1287881371 conn(0x7f1d580b2e60 msgr2=0x7f1d5806e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:07.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.808+0000 7f1d5f8da640 1 -- 172.21.15.32:0/1287881371 shutdown_connections 2024-04-24T23:01:07.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.808+0000 7f1d5f8da640 1 -- 172.21.15.32:0/1287881371 wait complete. 2024-04-24T23:01:07.820 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T23:01:07.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.920+0000 7fb6141a9640 1 Processor -- start 2024-04-24T23:01:07.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.920+0000 7fb6141a9640 1 -- start start 2024-04-24T23:01:07.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.921+0000 7fb6141a9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb60c15b480 0x7fb60c0b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:07.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.921+0000 7fb6141a9640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb60c06cd00 0x7fb60c0cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:07.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.921+0000 7fb6141a9640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb60c0b46c0 con 0x7fb60c15fd60 2024-04-24T23:01:07.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.921+0000 7fb6141a9640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fb60c0b4ff0 con 0x7fb60c15b480 2024-04-24T23:01:07.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.921+0000 7fb6141a9640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fb60c0b49a0 con 0x7fb60c06cd00 2024-04-24T23:01:07.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.921+0000 7fb611f45640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb60c15b480 0x7fb60c0b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:07.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.921+0000 7fb611f45640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb60c15b480 0x7fb60c0b5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50204/0 (socket says 172.21.15.32:50204) 2024-04-24T23:01:07.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.921+0000 7fb611f45640 1 -- 172.21.15.32:0/637275441 learned_addr learned my addr 172.21.15.32:0/637275441 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:07.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.921+0000 7fb612746640 1 --2- 172.21.15.32:0/637275441 >> v2:172.21.15.121:3300/0 conn(0x7fb60c06cd00 0x7fb60c0cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:07.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.921+0000 7fb611f45640 1 -- 172.21.15.32:0/637275441 >> v2:172.21.15.121:3300/0 conn(0x7fb60c06cd00 msgr2=0x7fb60c0cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:07.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.921+0000 7fb611f45640 1 --2- 172.21.15.32:0/637275441 >> v2:172.21.15.121:3300/0 conn(0x7fb60c06cd00 0x7fb60c0cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:07.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.921+0000 7fb611f45640 1 -- 172.21.15.32:0/637275441 >> v1:172.21.15.121:6789/0 conn(0x7fb60c15fd60 legacy=0x7fb60c06c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:07.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.921+0000 7fb611f45640 1 -- 172.21.15.32:0/637275441 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb60c0b4c80 con 0x7fb60c15b480 2024-04-24T23:01:07.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.922+0000 7fb611f45640 1 --2- 172.21.15.32:0/637275441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb60c15b480 0x7fb60c0b5c60 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fb60c06bf30 tx=0x7fb60802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8ae04504e85efee server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:07.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.922+0000 7fb610f43640 1 -- 172.21.15.32:0/637275441 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb608036070 con 0x7fb60c15b480 2024-04-24T23:01:07.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.922+0000 7fb610f43640 1 -- 172.21.15.32:0/637275441 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb608031600 con 0x7fb60c15b480 2024-04-24T23:01:07.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.922+0000 7fb610f43640 1 -- 172.21.15.32:0/637275441 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6080319d0 con 0x7fb60c15b480 2024-04-24T23:01:07.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.923+0000 7fb6141a9640 1 -- 172.21.15.32:0/637275441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb60c15b480 msgr2=0x7fb60c0b5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:07.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.923+0000 7fb6141a9640 1 --2- 172.21.15.32:0/637275441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb60c15b480 0x7fb60c0b5c60 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fb60c06bf30 tx=0x7fb60802a6f0 comp rx=0 tx=0).stop 2024-04-24T23:01:07.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.923+0000 7fb6141a9640 1 -- 172.21.15.32:0/637275441 shutdown_connections 2024-04-24T23:01:07.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.923+0000 7fb6141a9640 1 --2- 172.21.15.32:0/637275441 >> v2:172.21.15.121:3300/0 conn(0x7fb60c06cd00 0x7fb60c0cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:07.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.923+0000 7fb6141a9640 1 --2- 172.21.15.32:0/637275441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb60c15b480 0x7fb60c0b5c60 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:07.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.923+0000 7fb6141a9640 1 -- 172.21.15.32:0/637275441 >> 172.21.15.32:0/637275441 conn(0x7fb60c06b110 msgr2=0x7fb60c06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:07.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.923+0000 7fb6141a9640 1 -- 172.21.15.32:0/637275441 shutdown_connections 2024-04-24T23:01:07.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.923+0000 7fb6141a9640 1 -- 172.21.15.32:0/637275441 wait complete. 2024-04-24T23:01:07.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.924+0000 7fb6141a9640 1 Processor -- start 2024-04-24T23:01:07.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.924+0000 7fb6141a9640 1 -- start start 2024-04-24T23:01:07.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.925+0000 7fb6141a9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb60c06cd00 0x7fb60c14d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:07.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.925+0000 7fb6141a9640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb60c15b480 0x7fb60c1579d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:07.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.925+0000 7fb6141a9640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb60c15fd60 0x7fb60c14ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:07.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.925+0000 7fb6141a9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb60c15b860 con 0x7fb60c06cd00 2024-04-24T23:01:07.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.925+0000 7fb6141a9640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb60c0b5480 con 0x7fb60c15fd60 2024-04-24T23:01:07.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.925+0000 7fb6141a9640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fb60c150a30 con 0x7fb60c15b480 2024-04-24T23:01:07.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.925+0000 7fb611f45640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb60c06cd00 0x7fb60c14d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:07.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.925+0000 7fb611f45640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb60c06cd00 0x7fb60c14d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50216/0 (socket says 172.21.15.32:50216) 2024-04-24T23:01:07.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.925+0000 7fb611f45640 1 -- 172.21.15.32:0/886000747 learned_addr learned my addr 172.21.15.32:0/886000747 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:07.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.925+0000 7fb612746640 1 --2- 172.21.15.32:0/886000747 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb60c15fd60 0x7fb60c14ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:07.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.925+0000 7fb611744640 1 --2- 172.21.15.32:0/886000747 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb60c15b480 0x7fb60c1579d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:07.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.925+0000 7fb611f45640 1 -- 172.21.15.32:0/886000747 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb60c15b480 msgr2=0x7fb60c1579d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:07.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.925+0000 7fb611f45640 1 --2- 172.21.15.32:0/886000747 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb60c15b480 0x7fb60c1579d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:07.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.925+0000 7fb611f45640 1 -- 172.21.15.32:0/886000747 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb60c15fd60 msgr2=0x7fb60c14ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:07.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.925+0000 7fb611f45640 1 --2- 172.21.15.32:0/886000747 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb60c15fd60 0x7fb60c14ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:07.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.925+0000 7fb611f45640 1 -- 172.21.15.32:0/886000747 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb60c150d10 con 0x7fb60c06cd00 2024-04-24T23:01:07.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.925+0000 7fb611f45640 1 --2- 172.21.15.32:0/886000747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb60c06cd00 0x7fb60c14d070 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fb608034040 tx=0x7fb608004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:07.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.926+0000 7fb6027fc640 1 -- 172.21.15.32:0/886000747 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb608041020 con 0x7fb60c06cd00 2024-04-24T23:01:07.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.926+0000 7fb6027fc640 1 -- 172.21.15.32:0/886000747 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb608007800 con 0x7fb60c06cd00 2024-04-24T23:01:07.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.926+0000 7fb6141a9640 1 -- 172.21.15.32:0/886000747 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb60c151750 con 0x7fb60c06cd00 2024-04-24T23:01:07.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.926+0000 7fb6027fc640 1 -- 172.21.15.32:0/886000747 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb608036040 con 0x7fb60c06cd00 2024-04-24T23:01:07.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.926+0000 7fb6141a9640 1 -- 172.21.15.32:0/886000747 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb60c14d5b0 con 0x7fb60c06cd00 2024-04-24T23:01:07.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.927+0000 7fb6141a9640 1 -- 172.21.15.32:0/886000747 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb60c068430 con 0x7fb60c06cd00 2024-04-24T23:01:07.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.927+0000 7fb6027fc640 1 -- 172.21.15.32:0/886000747 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb608044020 con 0x7fb60c06cd00 2024-04-24T23:01:07.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.927+0000 7fb6027fc640 1 --2- 172.21.15.32:0/886000747 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb5f006be30 0x7fb5f006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:07.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.927+0000 7fb6027fc640 1 -- 172.21.15.32:0/886000747 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5854+0+0 (secure 0 0 0) 0x7fb6080abc80 con 0x7fb60c06cd00 2024-04-24T23:01:07.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.928+0000 7fb611744640 1 --2- 172.21.15.32:0/886000747 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb5f006be30 0x7fb5f006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:07.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.928+0000 7fb611744640 1 --2- 172.21.15.32:0/886000747 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb5f006be30 0x7fb5f006e2f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc006fd0 tx=0x7fb5fc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:07.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:07.932+0000 7fb6027fc640 1 -- 172.21.15.32:0/886000747 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb60804b030 con 0x7fb60c06cd00 2024-04-24T23:01:08.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.126+0000 7fb6141a9640 1 -- 172.21.15.32:0/886000747 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb60c06c000 con 0x7fb60c06cd00 2024-04-24T23:01:08.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.127+0000 7fb6027fc640 1 -- 172.21.15.32:0/886000747 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v56) ==== 80+0+0 (secure 0 0 0) 0x7fb60807a5a0 con 0x7fb60c06cd00 2024-04-24T23:01:08.128 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2024-04-24T23:01:08.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.129+0000 7fb6141a9640 1 -- 172.21.15.32:0/886000747 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb5f006be30 msgr2=0x7fb5f006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:08.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.129+0000 7fb6141a9640 1 --2- 172.21.15.32:0/886000747 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb5f006be30 0x7fb5f006e2f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc006fd0 tx=0x7fb5fc008040 comp rx=0 tx=0).stop 2024-04-24T23:01:08.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.129+0000 7fb6141a9640 1 -- 172.21.15.32:0/886000747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb60c06cd00 msgr2=0x7fb60c14d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:08.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.129+0000 7fb6141a9640 1 --2- 172.21.15.32:0/886000747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb60c06cd00 0x7fb60c14d070 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fb608034040 tx=0x7fb608004a40 comp rx=0 tx=0).stop 2024-04-24T23:01:08.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.129+0000 7fb6141a9640 1 -- 172.21.15.32:0/886000747 shutdown_connections 2024-04-24T23:01:08.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.129+0000 7fb6141a9640 1 --2- 172.21.15.32:0/886000747 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb5f006be30 0x7fb5f006e2f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:08.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.129+0000 7fb6141a9640 1 --2- 172.21.15.32:0/886000747 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb60c15fd60 0x7fb60c14ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:08.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.129+0000 7fb6141a9640 1 --2- 172.21.15.32:0/886000747 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb60c15b480 0x7fb60c1579d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:08.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.129+0000 7fb6141a9640 1 --2- 172.21.15.32:0/886000747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb60c06cd00 0x7fb60c14d070 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:08.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.129+0000 7fb6141a9640 1 -- 172.21.15.32:0/886000747 >> 172.21.15.32:0/886000747 conn(0x7fb60c06b110 msgr2=0x7fb60c06c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:08.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.129+0000 7fb6141a9640 1 -- 172.21.15.32:0/886000747 shutdown_connections 2024-04-24T23:01:08.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.129+0000 7fb6141a9640 1 -- 172.21.15.32:0/886000747 wait complete. 2024-04-24T23:01:08.141 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-24T23:01:08.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:01:08.141 DEBUG:teuthology.orchestra.run.smithi032:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-24T23:01:08.182 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2024-04-24T23:01:08.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:01:08.182 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-24T23:01:08.238 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:01:08.238 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:01:08.238 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:01:08.256 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:01:08.256 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:01:08.313 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:01:08.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.412+0000 7ffb97f42640 1 Processor -- start 2024-04-24T23:01:08.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.413+0000 7ffb97f42640 1 -- start start 2024-04-24T23:01:08.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.413+0000 7ffb97f42640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffb9015e9b0 0x7ffb900d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:08.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.413+0000 7ffb97f42640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ffb9006b820 0x7ffb900b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:08.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.413+0000 7ffb97f42640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb900d3170 con 0x7ffb900d4210 2024-04-24T23:01:08.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.413+0000 7ffb97f42640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7ffb900d3450 con 0x7ffb9015e9b0 2024-04-24T23:01:08.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.413+0000 7ffb97f42640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7ffb900d3730 con 0x7ffb9006b820 2024-04-24T23:01:08.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.413+0000 7ffb95cde640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffb9015e9b0 0x7ffb900d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:08.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.413+0000 7ffb95cde640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffb9015e9b0 0x7ffb900d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50232/0 (socket says 172.21.15.32:50232) 2024-04-24T23:01:08.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.413+0000 7ffb95cde640 1 -- 172.21.15.32:0/700828379 learned_addr learned my addr 172.21.15.32:0/700828379 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:08.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.414+0000 7ffb964df640 1 --2- 172.21.15.32:0/700828379 >> v2:172.21.15.121:3300/0 conn(0x7ffb9006b820 0x7ffb900b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:08.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.414+0000 7ffb95cde640 1 -- 172.21.15.32:0/700828379 >> v2:172.21.15.121:3300/0 conn(0x7ffb9006b820 msgr2=0x7ffb900b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:08.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.414+0000 7ffb95cde640 1 --2- 172.21.15.32:0/700828379 >> v2:172.21.15.121:3300/0 conn(0x7ffb9006b820 0x7ffb900b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:08.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.414+0000 7ffb95cde640 1 -- 172.21.15.32:0/700828379 >> v1:172.21.15.121:6789/0 conn(0x7ffb900d4210 legacy=0x7ffb9006b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:08.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.414+0000 7ffb95cde640 1 -- 172.21.15.32:0/700828379 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb900d3a10 con 0x7ffb9015e9b0 2024-04-24T23:01:08.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.414+0000 7ffb95cde640 1 --2- 172.21.15.32:0/700828379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb9015e9b0 0x7ffb900d2b90 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ffb901622f0 tx=0x7ffb8002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1d02823eab8c9fd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:08.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.415+0000 7ffb94cdc640 1 -- 172.21.15.32:0/700828379 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb80036070 con 0x7ffb9015e9b0 2024-04-24T23:01:08.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.415+0000 7ffb94cdc640 1 -- 172.21.15.32:0/700828379 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb80031600 con 0x7ffb9015e9b0 2024-04-24T23:01:08.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.415+0000 7ffb94cdc640 1 -- 172.21.15.32:0/700828379 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb800319d0 con 0x7ffb9015e9b0 2024-04-24T23:01:08.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.415+0000 7ffb97f42640 1 -- 172.21.15.32:0/700828379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb9015e9b0 msgr2=0x7ffb900d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:08.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.415+0000 7ffb97f42640 1 --2- 172.21.15.32:0/700828379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb9015e9b0 0x7ffb900d2b90 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ffb901622f0 tx=0x7ffb8002a660 comp rx=0 tx=0).stop 2024-04-24T23:01:08.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.415+0000 7ffb97f42640 1 -- 172.21.15.32:0/700828379 shutdown_connections 2024-04-24T23:01:08.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.415+0000 7ffb97f42640 1 --2- 172.21.15.32:0/700828379 >> v2:172.21.15.121:3300/0 conn(0x7ffb9006b820 0x7ffb900b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:08.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.415+0000 7ffb97f42640 1 --2- 172.21.15.32:0/700828379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb9015e9b0 0x7ffb900d2b90 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:08.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.415+0000 7ffb97f42640 1 -- 172.21.15.32:0/700828379 >> 172.21.15.32:0/700828379 conn(0x7ffb900d11d0 msgr2=0x7ffb900d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:08.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.415+0000 7ffb97f42640 1 -- 172.21.15.32:0/700828379 shutdown_connections 2024-04-24T23:01:08.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.415+0000 7ffb97f42640 1 -- 172.21.15.32:0/700828379 wait complete. 2024-04-24T23:01:08.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.416+0000 7ffb97f42640 1 Processor -- start 2024-04-24T23:01:08.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.416+0000 7ffb97f42640 1 -- start start 2024-04-24T23:01:08.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.417+0000 7ffb97f42640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb9006b820 0x7ffb90157620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:08.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.417+0000 7ffb97f42640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb900d4210 0x7ffb901f7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:08.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.417+0000 7ffb97f42640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb9015e9b0 0x7ffb901f4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:08.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.417+0000 7ffb97f42640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ffb900ba0e0 con 0x7ffb9015e9b0 2024-04-24T23:01:08.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.417+0000 7ffb97f42640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ffb900d2400 con 0x7ffb900d4210 2024-04-24T23:01:08.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.417+0000 7ffb97f42640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ffb900ba260 con 0x7ffb9006b820 2024-04-24T23:01:08.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.417+0000 7ffb964df640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb9015e9b0 0x7ffb901f4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:08.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.417+0000 7ffb964df640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb9015e9b0 0x7ffb901f4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50234/0 (socket says 172.21.15.32:50234) 2024-04-24T23:01:08.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.417+0000 7ffb964df640 1 -- 172.21.15.32:0/1206165624 learned_addr learned my addr 172.21.15.32:0/1206165624 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:08.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.417+0000 7ffb954dd640 1 --2- 172.21.15.32:0/1206165624 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb900d4210 0x7ffb901f7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:08.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.417+0000 7ffb95cde640 1 --2- 172.21.15.32:0/1206165624 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb9006b820 0x7ffb90157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:08.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.418+0000 7ffb964df640 1 -- 172.21.15.32:0/1206165624 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb9006b820 msgr2=0x7ffb90157620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:08.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.418+0000 7ffb964df640 1 --2- 172.21.15.32:0/1206165624 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb9006b820 0x7ffb90157620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:08.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.418+0000 7ffb964df640 1 -- 172.21.15.32:0/1206165624 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb900d4210 msgr2=0x7ffb901f7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:08.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.418+0000 7ffb964df640 1 --2- 172.21.15.32:0/1206165624 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb900d4210 0x7ffb901f7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:08.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.418+0000 7ffb964df640 1 -- 172.21.15.32:0/1206165624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb901f82b0 con 0x7ffb9015e9b0 2024-04-24T23:01:08.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.418+0000 7ffb954dd640 1 --2- 172.21.15.32:0/1206165624 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb900d4210 0x7ffb901f7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:01:08.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.418+0000 7ffb964df640 1 --2- 172.21.15.32:0/1206165624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb9015e9b0 0x7ffb901f4440 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ffb8800d800 tx=0x7ffb8800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:08.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.418+0000 7ffb8e7fc640 1 -- 172.21.15.32:0/1206165624 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8800b6c0 con 0x7ffb9015e9b0 2024-04-24T23:01:08.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.418+0000 7ffb8e7fc640 1 -- 172.21.15.32:0/1206165624 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb88019070 con 0x7ffb9015e9b0 2024-04-24T23:01:08.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.418+0000 7ffb97f42640 1 -- 172.21.15.32:0/1206165624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb901f8590 con 0x7ffb9015e9b0 2024-04-24T23:01:08.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.418+0000 7ffb8e7fc640 1 -- 172.21.15.32:0/1206165624 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8800bac0 con 0x7ffb9015e9b0 2024-04-24T23:01:08.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.418+0000 7ffb97f42640 1 -- 172.21.15.32:0/1206165624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb900cd270 con 0x7ffb9015e9b0 2024-04-24T23:01:08.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.419+0000 7ffb97f42640 1 -- 172.21.15.32:0/1206165624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb90068430 con 0x7ffb9015e9b0 2024-04-24T23:01:08.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.419+0000 7ffb8e7fc640 1 -- 172.21.15.32:0/1206165624 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffb8800bc60 con 0x7ffb9015e9b0 2024-04-24T23:01:08.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.420+0000 7ffb8e7fc640 1 --2- 172.21.15.32:0/1206165624 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ffb7c06be30 0x7ffb7c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:08.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.420+0000 7ffb8e7fc640 1 -- 172.21.15.32:0/1206165624 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5854+0+0 (secure 0 0 0) 0x7ffb8808b2d0 con 0x7ffb9015e9b0 2024-04-24T23:01:08.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.420+0000 7ffb95cde640 1 --2- 172.21.15.32:0/1206165624 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ffb7c06be30 0x7ffb7c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:08.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.421+0000 7ffb95cde640 1 --2- 172.21.15.32:0/1206165624 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ffb7c06be30 0x7ffb7c06e2f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ffb80033000 tx=0x7ffb80009000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:08.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.424+0000 7ffb8e7fc640 1 -- 172.21.15.32:0/1206165624 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ffb88059bf0 con 0x7ffb9015e9b0 2024-04-24T23:01:08.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.607+0000 7ffb97f42640 1 -- 172.21.15.32:0/1206165624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffb900d5530 con 0x7ffb9015e9b0 2024-04-24T23:01:08.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.608+0000 7ffb8e7fc640 1 -- 172.21.15.32:0/1206165624 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) ==== 74+0+16912 (secure 0 0 0) 0x7ffb880594f0 con 0x7ffb9015e9b0 2024-04-24T23:01:08.609 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:01:08.610 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":56,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:01:00.283572+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:00:30.665592+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-24T23:00:31.357756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.6","mappings":[{"from":5,"to":7}]},{"pgid":"7.c","mappings":[{"from":5,"to":7}]},{"pgid":"7.12","mappings":[{"from":5,"to":7}]},{"pgid":"7.31","mappings":[{"from":5,"to":7}]},{"pgid":"7.35","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:01:08.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.610+0000 7ffb97f42640 1 -- 172.21.15.32:0/1206165624 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ffb7c06be30 msgr2=0x7ffb7c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:08.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.610+0000 7ffb97f42640 1 --2- 172.21.15.32:0/1206165624 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ffb7c06be30 0x7ffb7c06e2f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ffb80033000 tx=0x7ffb80009000 comp rx=0 tx=0).stop 2024-04-24T23:01:08.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.610+0000 7ffb97f42640 1 -- 172.21.15.32:0/1206165624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb9015e9b0 msgr2=0x7ffb901f4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:08.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.610+0000 7ffb97f42640 1 --2- 172.21.15.32:0/1206165624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb9015e9b0 0x7ffb901f4440 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ffb8800d800 tx=0x7ffb8800dcc0 comp rx=0 tx=0).stop 2024-04-24T23:01:08.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.610+0000 7ffb97f42640 1 -- 172.21.15.32:0/1206165624 shutdown_connections 2024-04-24T23:01:08.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.610+0000 7ffb97f42640 1 --2- 172.21.15.32:0/1206165624 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ffb7c06be30 0x7ffb7c06e2f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:08.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.610+0000 7ffb97f42640 1 --2- 172.21.15.32:0/1206165624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb9015e9b0 0x7ffb901f4440 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:08.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.610+0000 7ffb97f42640 1 --2- 172.21.15.32:0/1206165624 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb900d4210 0x7ffb901f7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:08.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.610+0000 7ffb97f42640 1 --2- 172.21.15.32:0/1206165624 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb9006b820 0x7ffb90157620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:08.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.610+0000 7ffb97f42640 1 -- 172.21.15.32:0/1206165624 >> 172.21.15.32:0/1206165624 conn(0x7ffb900d11d0 msgr2=0x7ffb900c9d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:08.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.610+0000 7ffb97f42640 1 -- 172.21.15.32:0/1206165624 shutdown_connections 2024-04-24T23:01:08.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.610+0000 7ffb97f42640 1 -- 172.21.15.32:0/1206165624 wait complete. 2024-04-24T23:01:08.622 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:00:30.665592+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:00:31.357756+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:01:08.623 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:01:08.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.724+0000 7f46bfe2a640 1 Processor -- start 2024-04-24T23:01:08.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.724+0000 7f46bfe2a640 1 -- start start 2024-04-24T23:01:08.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.725+0000 7f46bfe2a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f46b815e9b0 0x7f46b80d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:08.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.725+0000 7f46bfe2a640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f46b80d4210 0x7f46b806b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:08.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.725+0000 7f46bfe2a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46b80ba0e0 0x7f46b806b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:08.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.725+0000 7f46bfe2a640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f46b80d3170 con 0x7f46b815e9b0 2024-04-24T23:01:08.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.725+0000 7f46bfe2a640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f46b80d3450 con 0x7f46b80d4210 2024-04-24T23:01:08.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.725+0000 7f46bfe2a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f46b80d3730 con 0x7f46b80ba0e0 2024-04-24T23:01:08.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.725+0000 7f46bdbc6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f46b815e9b0 0x7f46b80d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:08.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.725+0000 7f46bdbc6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f46b815e9b0 0x7f46b80d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50238/0 (socket says 172.21.15.32:50238) 2024-04-24T23:01:08.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.725+0000 7f46bdbc6640 1 -- 172.21.15.32:0/3229572188 learned_addr learned my addr 172.21.15.32:0/3229572188 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:08.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.726+0000 7f46bd3c5640 1 --2- 172.21.15.32:0/3229572188 >> v2:172.21.15.121:3300/0 conn(0x7f46b80d4210 0x7f46b806b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:08.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.726+0000 7f46be3c7640 1 --2- 172.21.15.32:0/3229572188 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46b80ba0e0 0x7f46b806b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:08.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.726+0000 7f46bdbc6640 1 -- 172.21.15.32:0/3229572188 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46b80ba0e0 msgr2=0x7f46b806b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:08.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.726+0000 7f46bdbc6640 1 --2- 172.21.15.32:0/3229572188 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46b80ba0e0 0x7f46b806b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:08.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.726+0000 7f46bdbc6640 1 -- 172.21.15.32:0/3229572188 >> v2:172.21.15.121:3300/0 conn(0x7f46b80d4210 msgr2=0x7f46b806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:08.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.726+0000 7f46bdbc6640 1 --2- 172.21.15.32:0/3229572188 >> v2:172.21.15.121:3300/0 conn(0x7f46b80d4210 0x7f46b806b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:08.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.726+0000 7f46bdbc6640 1 -- 172.21.15.32:0/3229572188 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46b80d3a10 con 0x7f46b815e9b0 2024-04-24T23:01:08.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.726+0000 7f46bd3c5640 1 --2- 172.21.15.32:0/3229572188 >> v2:172.21.15.121:3300/0 conn(0x7f46b80d4210 0x7f46b806b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:01:08.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.726+0000 7f46bdbc6640 1 --2- 172.21.15.32:0/3229572188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46b815e9b0 0x7f46b80d2b90 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f46b80730f0 tx=0x7f46a802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cbe4bc16fc7b2a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:08.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.727+0000 7f46bcbc4640 1 -- 172.21.15.32:0/3229572188 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a8036070 con 0x7f46b815e9b0 2024-04-24T23:01:08.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.727+0000 7f46bcbc4640 1 -- 172.21.15.32:0/3229572188 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46a8031600 con 0x7f46b815e9b0 2024-04-24T23:01:08.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.727+0000 7f46bcbc4640 1 -- 172.21.15.32:0/3229572188 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a80319d0 con 0x7f46b815e9b0 2024-04-24T23:01:08.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.727+0000 7f46bfe2a640 1 -- 172.21.15.32:0/3229572188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46b815e9b0 msgr2=0x7f46b80d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:08.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.727+0000 7f46bfe2a640 1 --2- 172.21.15.32:0/3229572188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46b815e9b0 0x7f46b80d2b90 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f46b80730f0 tx=0x7f46a802a6a0 comp rx=0 tx=0).stop 2024-04-24T23:01:08.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.727+0000 7f46bfe2a640 1 -- 172.21.15.32:0/3229572188 shutdown_connections 2024-04-24T23:01:08.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.727+0000 7f46bfe2a640 1 --2- 172.21.15.32:0/3229572188 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46b80ba0e0 0x7f46b806b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:08.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.727+0000 7f46bfe2a640 1 --2- 172.21.15.32:0/3229572188 >> v2:172.21.15.121:3300/0 conn(0x7f46b80d4210 0x7f46b806b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:08.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.727+0000 7f46bfe2a640 1 --2- 172.21.15.32:0/3229572188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46b815e9b0 0x7f46b80d2b90 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:08.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.727+0000 7f46bfe2a640 1 -- 172.21.15.32:0/3229572188 >> 172.21.15.32:0/3229572188 conn(0x7f46b80d11d0 msgr2=0x7f46b80d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:08.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.727+0000 7f46bfe2a640 1 -- 172.21.15.32:0/3229572188 shutdown_connections 2024-04-24T23:01:08.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.727+0000 7f46bfe2a640 1 -- 172.21.15.32:0/3229572188 wait complete. 2024-04-24T23:01:08.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.728+0000 7f46bfe2a640 1 Processor -- start 2024-04-24T23:01:08.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.728+0000 7f46bfe2a640 1 -- start start 2024-04-24T23:01:08.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.729+0000 7f46bfe2a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46b80ba0e0 0x7f46b81f4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:08.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.729+0000 7f46bfe2a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46b80d4210 0x7f46b8157960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:08.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.729+0000 7f46bfe2a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46b815e9b0 0x7f46b8158470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:08.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.729+0000 7f46bfe2a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f46b8071730 con 0x7f46b80d4210 2024-04-24T23:01:08.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.729+0000 7f46bfe2a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f46b80d2400 con 0x7f46b815e9b0 2024-04-24T23:01:08.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.729+0000 7f46bfe2a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f46b80718b0 con 0x7f46b80ba0e0 2024-04-24T23:01:08.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.729+0000 7f46bd3c5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46b80d4210 0x7f46b8157960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:08.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.729+0000 7f46bd3c5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46b80d4210 0x7f46b8157960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50246/0 (socket says 172.21.15.32:50246) 2024-04-24T23:01:08.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.729+0000 7f46bd3c5640 1 -- 172.21.15.32:0/4151643757 learned_addr learned my addr 172.21.15.32:0/4151643757 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:08.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.730+0000 7f46bdbc6640 1 --2- 172.21.15.32:0/4151643757 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46b80ba0e0 0x7f46b81f4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:08.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.730+0000 7f46be3c7640 1 --2- 172.21.15.32:0/4151643757 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46b815e9b0 0x7f46b8158470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:08.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.730+0000 7f46bd3c5640 1 -- 172.21.15.32:0/4151643757 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46b80ba0e0 msgr2=0x7f46b81f4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:08.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.730+0000 7f46bd3c5640 1 --2- 172.21.15.32:0/4151643757 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46b80ba0e0 0x7f46b81f4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:08.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.730+0000 7f46bd3c5640 1 -- 172.21.15.32:0/4151643757 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46b815e9b0 msgr2=0x7f46b8158470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:08.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.730+0000 7f46bd3c5640 1 --2- 172.21.15.32:0/4151643757 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46b815e9b0 0x7f46b8158470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:08.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.730+0000 7f46bd3c5640 1 -- 172.21.15.32:0/4151643757 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46b81f5030 con 0x7f46b80d4210 2024-04-24T23:01:08.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.730+0000 7f46bdbc6640 1 --2- 172.21.15.32:0/4151643757 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46b80ba0e0 0x7f46b81f4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:01:08.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.730+0000 7f46be3c7640 1 --2- 172.21.15.32:0/4151643757 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46b815e9b0 0x7f46b8158470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:01:08.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.730+0000 7f46bd3c5640 1 --2- 172.21.15.32:0/4151643757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46b80d4210 0x7f46b8157960 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f46ac00e7e0 tx=0x7f46ac00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:08.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.730+0000 7f46b67fc640 1 -- 172.21.15.32:0/4151643757 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac00f040 con 0x7f46b80d4210 2024-04-24T23:01:08.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.730+0000 7f46b67fc640 1 -- 172.21.15.32:0/4151643757 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46ac004030 con 0x7f46b80d4210 2024-04-24T23:01:08.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.730+0000 7f46bfe2a640 1 -- 172.21.15.32:0/4151643757 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46b8157ea0 con 0x7f46b80d4210 2024-04-24T23:01:08.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.730+0000 7f46b67fc640 1 -- 172.21.15.32:0/4151643757 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac004400 con 0x7f46b80d4210 2024-04-24T23:01:08.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.730+0000 7f46bfe2a640 1 -- 172.21.15.32:0/4151643757 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46b815ac40 con 0x7f46b80d4210 2024-04-24T23:01:08.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.731+0000 7f46bfe2a640 1 -- 172.21.15.32:0/4151643757 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46b8162080 con 0x7f46b80d4210 2024-04-24T23:01:08.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.732+0000 7f46b67fc640 1 -- 172.21.15.32:0/4151643757 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f46ac00c6d0 con 0x7f46b80d4210 2024-04-24T23:01:08.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.732+0000 7f46b67fc640 1 --2- 172.21.15.32:0/4151643757 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f46a406be30 0x7f46a406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:08.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.732+0000 7f46b67fc640 1 -- 172.21.15.32:0/4151643757 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5854+0+0 (secure 0 0 0) 0x7f46ac08ac40 con 0x7f46b80d4210 2024-04-24T23:01:08.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.732+0000 7f46bdbc6640 1 --2- 172.21.15.32:0/4151643757 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f46a406be30 0x7f46a406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:08.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.733+0000 7f46bdbc6640 1 --2- 172.21.15.32:0/4151643757 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f46a406be30 0x7f46a406e2f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f46a8033000 tx=0x7f46a8009000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:08.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.736+0000 7f46b67fc640 1 -- 172.21.15.32:0/4151643757 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f46b8162080 con 0x7f46b80d4210 2024-04-24T23:01:08.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.918+0000 7f46bfe2a640 1 -- 172.21.15.32:0/4151643757 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f46b8068430 con 0x7f46b80d4210 2024-04-24T23:01:08.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.918+0000 7f46b67fc640 1 -- 172.21.15.32:0/4151643757 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v56) ==== 93+0+10 (secure 0 0 0) 0x7f46b81f5030 con 0x7f46b80d4210 2024-04-24T23:01:08.919 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T23:01:08.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.920+0000 7f46bfe2a640 1 -- 172.21.15.32:0/4151643757 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f46a406be30 msgr2=0x7f46a406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:08.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.920+0000 7f46bfe2a640 1 --2- 172.21.15.32:0/4151643757 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f46a406be30 0x7f46a406e2f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f46a8033000 tx=0x7f46a8009000 comp rx=0 tx=0).stop 2024-04-24T23:01:08.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.920+0000 7f46bfe2a640 1 -- 172.21.15.32:0/4151643757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46b80d4210 msgr2=0x7f46b8157960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:08.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.920+0000 7f46bfe2a640 1 --2- 172.21.15.32:0/4151643757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46b80d4210 0x7f46b8157960 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f46ac00e7e0 tx=0x7f46ac00eca0 comp rx=0 tx=0).stop 2024-04-24T23:01:08.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.920+0000 7f46bfe2a640 1 -- 172.21.15.32:0/4151643757 shutdown_connections 2024-04-24T23:01:08.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.920+0000 7f46bfe2a640 1 --2- 172.21.15.32:0/4151643757 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f46a406be30 0x7f46a406e2f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:08.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.920+0000 7f46bfe2a640 1 --2- 172.21.15.32:0/4151643757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46b80d4210 0x7f46b8157960 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:08.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.920+0000 7f46bfe2a640 1 --2- 172.21.15.32:0/4151643757 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46b815e9b0 0x7f46b8158470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:08.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.920+0000 7f46bfe2a640 1 --2- 172.21.15.32:0/4151643757 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46b80ba0e0 0x7f46b81f4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:08.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.920+0000 7f46bfe2a640 1 -- 172.21.15.32:0/4151643757 >> 172.21.15.32:0/4151643757 conn(0x7f46b80d11d0 msgr2=0x7f46b815bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:08.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.920+0000 7f46bfe2a640 1 -- 172.21.15.32:0/4151643757 shutdown_connections 2024-04-24T23:01:08.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:08.920+0000 7f46bfe2a640 1 -- 172.21.15.32:0/4151643757 wait complete. 2024-04-24T23:01:08.932 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:01:09.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.035+0000 7f34641d5640 1 Processor -- start 2024-04-24T23:01:09.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.035+0000 7f34641d5640 1 -- start start 2024-04-24T23:01:09.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.036+0000 7f34641d5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f345c162e10 0x7f345c0ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:09.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.036+0000 7f34641d5640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f345c0cd270 con 0x7f345c0b5af0 2024-04-24T23:01:09.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.036+0000 7f34641d5640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f345c071610 con 0x7f345c162320 2024-04-24T23:01:09.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.036+0000 7f34641d5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f345c0b5660 con 0x7f345c162e10 2024-04-24T23:01:09.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.036+0000 7f3461770640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f345c0b5af0 0x7f345c15ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56780/0 (socket says 172.21.15.32:56780) 2024-04-24T23:01:09.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.036+0000 7f3461770640 1 -- 172.21.15.32:0/1179163025 learned_addr learned my addr 172.21.15.32:0/1179163025 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:01:09.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.036+0000 7f3462772640 1 --2- 172.21.15.32:0/1179163025 >> v2:172.21.15.32:3300/0 conn(0x7f345c162e10 0x7f345c0ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:09.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.037+0000 7f3462772640 1 -- 172.21.15.32:0/1179163025 >> v1:172.21.15.121:6789/0 conn(0x7f345c162320 legacy=0x7f345c162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:09.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.037+0000 7f3462772640 1 -- 172.21.15.32:0/1179163025 >> v1:172.21.15.32:6789/0 conn(0x7f345c0b5af0 legacy=0x7f345c15ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:09.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.037+0000 7f3460f6f640 1 -- 172.21.15.32:0/1179163025 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3336194075 0 0) 0x7f345c0cd270 con 0x7f345c0b5af0 2024-04-24T23:01:09.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.037+0000 7f3462772640 1 -- 172.21.15.32:0/1179163025 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f345c0718f0 con 0x7f345c162e10 2024-04-24T23:01:09.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.037+0000 7f3462772640 1 --2- 172.21.15.32:0/1179163025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f345c162e10 0x7f345c0ccc90 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f344c009d90 tx=0x7f344c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19351c3876cb6d57 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:09.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.037+0000 7f3460f6f640 1 -- 172.21.15.32:0/1179163025 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344c007740 con 0x7f345c162e10 2024-04-24T23:01:09.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.037+0000 7f3460f6f640 1 -- 172.21.15.32:0/1179163025 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f344c0078e0 con 0x7f345c162e10 2024-04-24T23:01:09.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.037+0000 7f3460f6f640 1 -- 172.21.15.32:0/1179163025 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344c007cb0 con 0x7f345c162e10 2024-04-24T23:01:09.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.038+0000 7f34641d5640 1 -- 172.21.15.32:0/1179163025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f345c162e10 msgr2=0x7f345c0ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:09.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.038+0000 7f34641d5640 1 --2- 172.21.15.32:0/1179163025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f345c162e10 0x7f345c0ccc90 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f344c009d90 tx=0x7f344c02a660 comp rx=0 tx=0).stop 2024-04-24T23:01:09.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.038+0000 7f34641d5640 1 -- 172.21.15.32:0/1179163025 shutdown_connections 2024-04-24T23:01:09.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.038+0000 7f34641d5640 1 --2- 172.21.15.32:0/1179163025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f345c162e10 0x7f345c0ccc90 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:09.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.038+0000 7f34641d5640 1 -- 172.21.15.32:0/1179163025 >> 172.21.15.32:0/1179163025 conn(0x7f345c0d0ae0 msgr2=0x7f345c0d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:09.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.038+0000 7f34641d5640 1 -- 172.21.15.32:0/1179163025 shutdown_connections 2024-04-24T23:01:09.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.038+0000 7f34641d5640 1 -- 172.21.15.32:0/1179163025 wait complete. 2024-04-24T23:01:09.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.039+0000 7f34641d5640 1 Processor -- start 2024-04-24T23:01:09.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.039+0000 7f34641d5640 1 -- start start 2024-04-24T23:01:09.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.040+0000 7f34641d5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f345c0b5af0 0x7f345c0cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:09.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.040+0000 7f34641d5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f345c162320 0x7f345c160830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:09.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.040+0000 7f34641d5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f345c162e10 0x7f345c160d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:09.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.040+0000 7f34641d5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f345c071bd0 con 0x7f345c0b5af0 2024-04-24T23:01:09.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.040+0000 7f34641d5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f345c067950 con 0x7f345c162320 2024-04-24T23:01:09.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.040+0000 7f34641d5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f345c071d50 con 0x7f345c162e10 2024-04-24T23:01:09.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.040+0000 7f3461f71640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f345c0b5af0 0x7f345c0cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:09.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.040+0000 7f3461f71640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f345c0b5af0 0x7f345c0cfca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50266/0 (socket says 172.21.15.32:50266) 2024-04-24T23:01:09.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.040+0000 7f3461f71640 1 -- 172.21.15.32:0/4062959798 learned_addr learned my addr 172.21.15.32:0/4062959798 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:09.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.040+0000 7f3461f71640 1 -- 172.21.15.32:0/4062959798 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f345c162e10 msgr2=0x7f345c160d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:09.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.040+0000 7f3462772640 1 --2- 172.21.15.32:0/4062959798 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f345c162e10 0x7f345c160d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:09.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.040+0000 7f3461770640 1 --2- 172.21.15.32:0/4062959798 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f345c162320 0x7f345c160830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:09.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.040+0000 7f3461f71640 1 --2- 172.21.15.32:0/4062959798 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f345c162e10 0x7f345c160d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:09.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.040+0000 7f3461f71640 1 -- 172.21.15.32:0/4062959798 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f345c162320 msgr2=0x7f345c160830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:09.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.040+0000 7f3461f71640 1 --2- 172.21.15.32:0/4062959798 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f345c162320 0x7f345c160830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:09.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.040+0000 7f3461f71640 1 -- 172.21.15.32:0/4062959798 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f345c1e9b50 con 0x7f345c0b5af0 2024-04-24T23:01:09.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.041+0000 7f3461f71640 1 --2- 172.21.15.32:0/4062959798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f345c0b5af0 0x7f345c0cfca0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f345400c450 tx=0x7f345400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:09.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.041+0000 7f345a7fc640 1 -- 172.21.15.32:0/4062959798 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34540052d0 con 0x7f345c0b5af0 2024-04-24T23:01:09.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.041+0000 7f345a7fc640 1 -- 172.21.15.32:0/4062959798 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3454005470 con 0x7f345c0b5af0 2024-04-24T23:01:09.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.041+0000 7f34641d5640 1 -- 172.21.15.32:0/4062959798 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f345c1e95c0 con 0x7f345c0b5af0 2024-04-24T23:01:09.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.041+0000 7f345a7fc640 1 -- 172.21.15.32:0/4062959798 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3454012870 con 0x7f345c0b5af0 2024-04-24T23:01:09.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.041+0000 7f34641d5640 1 -- 172.21.15.32:0/4062959798 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f345c1e8800 con 0x7f345c0b5af0 2024-04-24T23:01:09.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.042+0000 7f34641d5640 1 -- 172.21.15.32:0/4062959798 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f345c068430 con 0x7f345c0b5af0 2024-04-24T23:01:09.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.042+0000 7f345a7fc640 1 -- 172.21.15.32:0/4062959798 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3454012a10 con 0x7f345c0b5af0 2024-04-24T23:01:09.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.043+0000 7f345a7fc640 1 --2- 172.21.15.32:0/4062959798 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f344806be30 0x7f344806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:09.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.043+0000 7f345a7fc640 1 -- 172.21.15.32:0/4062959798 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5854+0+0 (secure 0 0 0) 0x7f345408aa70 con 0x7f345c0b5af0 2024-04-24T23:01:09.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.043+0000 7f3461770640 1 --2- 172.21.15.32:0/4062959798 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f344806be30 0x7f344806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:09.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.044+0000 7f3461770640 1 --2- 172.21.15.32:0/4062959798 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f344806be30 0x7f344806e2f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3450007fd0 tx=0x7f3450009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:09.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.047+0000 7f345a7fc640 1 -- 172.21.15.32:0/4062959798 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f345c1e9b50 con 0x7f345c0b5af0 2024-04-24T23:01:09.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.230+0000 7f34641d5640 1 -- 172.21.15.32:0/4062959798 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f345c0d0850 con 0x7f345c0b5af0 2024-04-24T23:01:09.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.230+0000 7f345a7fc640 1 -- 172.21.15.32:0/4062959798 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v56) ==== 104+0+11 (secure 0 0 0) 0x7f3454058dd0 con 0x7f345c0b5af0 2024-04-24T23:01:09.232 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-24T23:01:09.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.232+0000 7f34641d5640 1 -- 172.21.15.32:0/4062959798 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f344806be30 msgr2=0x7f344806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:09.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.232+0000 7f34641d5640 1 --2- 172.21.15.32:0/4062959798 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f344806be30 0x7f344806e2f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3450007fd0 tx=0x7f3450009040 comp rx=0 tx=0).stop 2024-04-24T23:01:09.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.232+0000 7f34641d5640 1 -- 172.21.15.32:0/4062959798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f345c0b5af0 msgr2=0x7f345c0cfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:09.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.232+0000 7f34641d5640 1 --2- 172.21.15.32:0/4062959798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f345c0b5af0 0x7f345c0cfca0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f345400c450 tx=0x7f345400c910 comp rx=0 tx=0).stop 2024-04-24T23:01:09.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.232+0000 7f34641d5640 1 -- 172.21.15.32:0/4062959798 shutdown_connections 2024-04-24T23:01:09.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.232+0000 7f34641d5640 1 --2- 172.21.15.32:0/4062959798 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f344806be30 0x7f344806e2f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:09.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.232+0000 7f34641d5640 1 --2- 172.21.15.32:0/4062959798 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f345c162e10 0x7f345c160d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:09.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.232+0000 7f34641d5640 1 --2- 172.21.15.32:0/4062959798 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f345c162320 0x7f345c160830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:09.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.232+0000 7f34641d5640 1 --2- 172.21.15.32:0/4062959798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f345c0b5af0 0x7f345c0cfca0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:09.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.232+0000 7f34641d5640 1 -- 172.21.15.32:0/4062959798 >> 172.21.15.32:0/4062959798 conn(0x7f345c0d0ae0 msgr2=0x7f345c162700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:09.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.232+0000 7f34641d5640 1 -- 172.21.15.32:0/4062959798 shutdown_connections 2024-04-24T23:01:09.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.232+0000 7f34641d5640 1 -- 172.21.15.32:0/4062959798 wait complete. 2024-04-24T23:01:09.244 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:01:09.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.346+0000 7ff18b553640 1 Processor -- start 2024-04-24T23:01:09.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.347+0000 7ff18b553640 1 -- start start 2024-04-24T23:01:09.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.347+0000 7ff18b553640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff18415e9b0 0x7ff1840d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:09.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.347+0000 7ff18b553640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff18406b820 0x7ff1840b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:09.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.347+0000 7ff18b553640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1840d3170 con 0x7ff1840d4210 2024-04-24T23:01:09.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.347+0000 7ff18b553640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7ff1840d3450 con 0x7ff18415e9b0 2024-04-24T23:01:09.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.347+0000 7ff18b553640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff1840d3730 con 0x7ff18406b820 2024-04-24T23:01:09.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.348+0000 7ff189af0640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff18406b820 0x7ff1840b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:09.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.348+0000 7ff188aee640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7ff1840d4210 0x7ff18406b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:58588/0 (socket says 172.21.15.32:58588) 2024-04-24T23:01:09.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.348+0000 7ff1892ef640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff18415e9b0 0x7ff1840d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:09.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.348+0000 7ff188aee640 1 -- 172.21.15.32:0/1709473834 learned_addr learned my addr 172.21.15.32:0/1709473834 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:01:09.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.348+0000 7ff189af0640 1 -- 172.21.15.32:0/1709473834 >> v2:172.21.15.121:3300/0 conn(0x7ff18415e9b0 msgr2=0x7ff1840d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:09.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.348+0000 7ff189af0640 1 --2- 172.21.15.32:0/1709473834 >> v2:172.21.15.121:3300/0 conn(0x7ff18415e9b0 0x7ff1840d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:09.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.348+0000 7ff189af0640 1 -- 172.21.15.32:0/1709473834 >> v1:172.21.15.121:6789/0 conn(0x7ff1840d4210 legacy=0x7ff18406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:09.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.348+0000 7ff189af0640 1 -- 172.21.15.32:0/1709473834 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1840d3a10 con 0x7ff18406b820 2024-04-24T23:01:09.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.349+0000 7ff189af0640 1 --2- 172.21.15.32:0/1709473834 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff18406b820 0x7ff1840b78f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff174009d90 tx=0x7ff17402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea0daf78a8c42b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:09.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.349+0000 7ff183fff640 1 -- 172.21.15.32:0/1709473834 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff174007740 con 0x7ff18406b820 2024-04-24T23:01:09.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.349+0000 7ff183fff640 1 -- 172.21.15.32:0/1709473834 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1740078e0 con 0x7ff18406b820 2024-04-24T23:01:09.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.349+0000 7ff183fff640 1 -- 172.21.15.32:0/1709473834 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff174007cb0 con 0x7ff18406b820 2024-04-24T23:01:09.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.349+0000 7ff18b553640 1 -- 172.21.15.32:0/1709473834 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff18406b820 msgr2=0x7ff1840b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:09.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.349+0000 7ff18b553640 1 --2- 172.21.15.32:0/1709473834 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff18406b820 0x7ff1840b78f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff174009d90 tx=0x7ff17402a660 comp rx=0 tx=0).stop 2024-04-24T23:01:09.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.349+0000 7ff18b553640 1 -- 172.21.15.32:0/1709473834 shutdown_connections 2024-04-24T23:01:09.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.349+0000 7ff18b553640 1 --2- 172.21.15.32:0/1709473834 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff18406b820 0x7ff1840b78f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:09.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.349+0000 7ff18b553640 1 --2- 172.21.15.32:0/1709473834 >> v2:172.21.15.121:3300/0 conn(0x7ff18415e9b0 0x7ff1840d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:09.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.349+0000 7ff18b553640 1 -- 172.21.15.32:0/1709473834 >> 172.21.15.32:0/1709473834 conn(0x7ff1840d11d0 msgr2=0x7ff1840d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:09.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.350+0000 7ff18b553640 1 -- 172.21.15.32:0/1709473834 shutdown_connections 2024-04-24T23:01:09.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.350+0000 7ff18b553640 1 -- 172.21.15.32:0/1709473834 wait complete. 2024-04-24T23:01:09.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.351+0000 7ff18b553640 1 Processor -- start 2024-04-24T23:01:09.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.351+0000 7ff18b553640 1 -- start start 2024-04-24T23:01:09.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.351+0000 7ff18b553640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff18406b820 0x7ff1840cb820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:09.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.351+0000 7ff18b553640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff1840d4210 0x7ff1841548a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:09.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.351+0000 7ff18b553640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff18415e9b0 0x7ff1840cc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:09.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.351+0000 7ff18b553640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff1840ba0e0 con 0x7ff18415e9b0 2024-04-24T23:01:09.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.351+0000 7ff18b553640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff1840d2400 con 0x7ff1840d4210 2024-04-24T23:01:09.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.351+0000 7ff18b553640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff1840ba260 con 0x7ff18406b820 2024-04-24T23:01:09.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.352+0000 7ff189af0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff18415e9b0 0x7ff1840cc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:09.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.352+0000 7ff189af0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff18415e9b0 0x7ff1840cc3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50276/0 (socket says 172.21.15.32:50276) 2024-04-24T23:01:09.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.352+0000 7ff189af0640 1 -- 172.21.15.32:0/1266836716 learned_addr learned my addr 172.21.15.32:0/1266836716 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:09.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.352+0000 7ff1892ef640 1 --2- 172.21.15.32:0/1266836716 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff18406b820 0x7ff1840cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:09.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.352+0000 7ff188aee640 1 --2- 172.21.15.32:0/1266836716 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff1840d4210 0x7ff1841548a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:09.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.352+0000 7ff189af0640 1 -- 172.21.15.32:0/1266836716 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff18406b820 msgr2=0x7ff1840cb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:09.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.352+0000 7ff189af0640 1 --2- 172.21.15.32:0/1266836716 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff18406b820 0x7ff1840cb820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:09.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.352+0000 7ff189af0640 1 -- 172.21.15.32:0/1266836716 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff1840d4210 msgr2=0x7ff1841548a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:09.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.352+0000 7ff189af0640 1 --2- 172.21.15.32:0/1266836716 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff1840d4210 0x7ff1841548a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:09.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.352+0000 7ff189af0640 1 -- 172.21.15.32:0/1266836716 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1840cbe00 con 0x7ff18415e9b0 2024-04-24T23:01:09.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.352+0000 7ff1892ef640 1 --2- 172.21.15.32:0/1266836716 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff18406b820 0x7ff1840cb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:01:09.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.352+0000 7ff189af0640 1 --2- 172.21.15.32:0/1266836716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff18415e9b0 0x7ff1840cc3a0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7ff174009eb0 tx=0x7ff174004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:09.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.352+0000 7ff181ffb640 1 -- 172.21.15.32:0/1266836716 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff174040070 con 0x7ff18415e9b0 2024-04-24T23:01:09.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.352+0000 7ff181ffb640 1 -- 172.21.15.32:0/1266836716 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff174031c60 con 0x7ff18415e9b0 2024-04-24T23:01:09.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.352+0000 7ff181ffb640 1 -- 172.21.15.32:0/1266836716 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff174009450 con 0x7ff18415e9b0 2024-04-24T23:01:09.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.352+0000 7ff18b553640 1 -- 172.21.15.32:0/1266836716 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1841f5630 con 0x7ff18415e9b0 2024-04-24T23:01:09.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.353+0000 7ff18b553640 1 -- 172.21.15.32:0/1266836716 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1841f5bf0 con 0x7ff18415e9b0 2024-04-24T23:01:09.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.353+0000 7ff18b553640 1 -- 172.21.15.32:0/1266836716 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff184068430 con 0x7ff18415e9b0 2024-04-24T23:01:09.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.354+0000 7ff181ffb640 1 -- 172.21.15.32:0/1266836716 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff174039040 con 0x7ff18415e9b0 2024-04-24T23:01:09.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.354+0000 7ff181ffb640 1 --2- 172.21.15.32:0/1266836716 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff17006be30 0x7ff17006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:09.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.354+0000 7ff181ffb640 1 -- 172.21.15.32:0/1266836716 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5854+0+0 (secure 0 0 0) 0x7ff1740ac1f0 con 0x7ff18415e9b0 2024-04-24T23:01:09.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.355+0000 7ff1892ef640 1 --2- 172.21.15.32:0/1266836716 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff17006be30 0x7ff17006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:09.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.355+0000 7ff1892ef640 1 --2- 172.21.15.32:0/1266836716 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff17006be30 0x7ff17006e2f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff17c009e90 tx=0x7ff17c009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:09.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.359+0000 7ff181ffb640 1 -- 172.21.15.32:0/1266836716 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff17407a550 con 0x7ff18415e9b0 2024-04-24T23:01:09.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.539+0000 7ff18b553640 1 -- 172.21.15.32:0/1266836716 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff184068cc0 con 0x7ff18415e9b0 2024-04-24T23:01:09.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.539+0000 7ff181ffb640 1 -- 172.21.15.32:0/1266836716 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v56) ==== 100+0+11 (secure 0 0 0) 0x7ff17407e850 con 0x7ff18415e9b0 2024-04-24T23:01:09.541 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-24T23:01:09.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.541+0000 7ff18b553640 1 -- 172.21.15.32:0/1266836716 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff17006be30 msgr2=0x7ff17006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:09.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.541+0000 7ff18b553640 1 --2- 172.21.15.32:0/1266836716 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff17006be30 0x7ff17006e2f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff17c009e90 tx=0x7ff17c009190 comp rx=0 tx=0).stop 2024-04-24T23:01:09.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.541+0000 7ff18b553640 1 -- 172.21.15.32:0/1266836716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff18415e9b0 msgr2=0x7ff1840cc3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:09.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.541+0000 7ff18b553640 1 --2- 172.21.15.32:0/1266836716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff18415e9b0 0x7ff1840cc3a0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7ff174009eb0 tx=0x7ff174004a40 comp rx=0 tx=0).stop 2024-04-24T23:01:09.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.541+0000 7ff18b553640 1 -- 172.21.15.32:0/1266836716 shutdown_connections 2024-04-24T23:01:09.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.541+0000 7ff18b553640 1 --2- 172.21.15.32:0/1266836716 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff17006be30 0x7ff17006e2f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:09.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.541+0000 7ff18b553640 1 --2- 172.21.15.32:0/1266836716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff18415e9b0 0x7ff1840cc3a0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:09.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.541+0000 7ff18b553640 1 --2- 172.21.15.32:0/1266836716 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff1840d4210 0x7ff1841548a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:09.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.541+0000 7ff18b553640 1 --2- 172.21.15.32:0/1266836716 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff18406b820 0x7ff1840cb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:09.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.541+0000 7ff18b553640 1 -- 172.21.15.32:0/1266836716 >> 172.21.15.32:0/1266836716 conn(0x7ff1840d11d0 msgr2=0x7ff184072040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:09.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.541+0000 7ff18b553640 1 -- 172.21.15.32:0/1266836716 shutdown_connections 2024-04-24T23:01:09.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.541+0000 7ff18b553640 1 -- 172.21.15.32:0/1266836716 wait complete. 2024-04-24T23:01:09.553 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T23:01:09.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.655+0000 7fe5025a5640 1 Processor -- start 2024-04-24T23:01:09.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.655+0000 7fe5025a5640 1 -- start start 2024-04-24T23:01:09.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.656+0000 7fe5025a5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe4fc06c5f0 0x7fe4fc06c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:09.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.656+0000 7fe5025a5640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe4fc0b44d0 0x7fe4fc0b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:09.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.656+0000 7fe5025a5640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4fc0b6430 con 0x7fe4fc0af890 2024-04-24T23:01:09.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.656+0000 7fe5025a5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fe4fc156f80 con 0x7fe4fc06c5f0 2024-04-24T23:01:09.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.656+0000 7fe5025a5640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fe4fc0b6150 con 0x7fe4fc0b44d0 2024-04-24T23:01:09.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.656+0000 7fe4fbfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe4fc06c5f0 0x7fe4fc06c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:09.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.656+0000 7fe4fbfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe4fc06c5f0 0x7fe4fc06c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50288/0 (socket says 172.21.15.32:50288) 2024-04-24T23:01:09.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.656+0000 7fe4fbfff640 1 -- 172.21.15.32:0/2414120351 learned_addr learned my addr 172.21.15.32:0/2414120351 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:09.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.656+0000 7fe4fbfff640 1 -- 172.21.15.32:0/2414120351 >> v2:172.21.15.121:3300/0 conn(0x7fe4fc0b44d0 msgr2=0x7fe4fc0b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:09.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.656+0000 7fe4fbfff640 1 --2- 172.21.15.32:0/2414120351 >> v2:172.21.15.121:3300/0 conn(0x7fe4fc0b44d0 0x7fe4fc0b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:09.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.656+0000 7fe4fbfff640 1 -- 172.21.15.32:0/2414120351 >> v1:172.21.15.121:6789/0 conn(0x7fe4fc0af890 legacy=0x7fe4fc0cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:09.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.656+0000 7fe4fbfff640 1 -- 172.21.15.32:0/2414120351 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4fc0b39f0 con 0x7fe4fc06c5f0 2024-04-24T23:01:09.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.657+0000 7fe4fbfff640 1 --2- 172.21.15.32:0/2414120351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4fc06c5f0 0x7fe4fc06c9d0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc073080 tx=0x7fe4ec02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b076113f6d09f57 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:09.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.657+0000 7fe4faffd640 1 -- 172.21.15.32:0/2414120351 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec037070 con 0x7fe4fc06c5f0 2024-04-24T23:01:09.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.657+0000 7fe4faffd640 1 -- 172.21.15.32:0/2414120351 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4ec032440 con 0x7fe4fc06c5f0 2024-04-24T23:01:09.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.657+0000 7fe4faffd640 1 -- 172.21.15.32:0/2414120351 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec032810 con 0x7fe4fc06c5f0 2024-04-24T23:01:09.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.658+0000 7fe5025a5640 1 -- 172.21.15.32:0/2414120351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4fc06c5f0 msgr2=0x7fe4fc06c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:09.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.658+0000 7fe5025a5640 1 --2- 172.21.15.32:0/2414120351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4fc06c5f0 0x7fe4fc06c9d0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc073080 tx=0x7fe4ec02a250 comp rx=0 tx=0).stop 2024-04-24T23:01:09.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.658+0000 7fe5025a5640 1 -- 172.21.15.32:0/2414120351 shutdown_connections 2024-04-24T23:01:09.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.658+0000 7fe5025a5640 1 --2- 172.21.15.32:0/2414120351 >> v2:172.21.15.121:3300/0 conn(0x7fe4fc0b44d0 0x7fe4fc0b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:09.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.658+0000 7fe5025a5640 1 --2- 172.21.15.32:0/2414120351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4fc06c5f0 0x7fe4fc06c9d0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:09.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.658+0000 7fe5025a5640 1 -- 172.21.15.32:0/2414120351 >> 172.21.15.32:0/2414120351 conn(0x7fe4fc06b110 msgr2=0x7fe4fc06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:09.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.658+0000 7fe5025a5640 1 -- 172.21.15.32:0/2414120351 shutdown_connections 2024-04-24T23:01:09.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.658+0000 7fe5025a5640 1 -- 172.21.15.32:0/2414120351 wait complete. 2024-04-24T23:01:09.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.659+0000 7fe5025a5640 1 Processor -- start 2024-04-24T23:01:09.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.659+0000 7fe5025a5640 1 -- start start 2024-04-24T23:01:09.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.660+0000 7fe5025a5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4fc06c5f0 0x7fe4fc159040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:09.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.660+0000 7fe5025a5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe4fc0af890 0x7fe4fc143810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:09.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.660+0000 7fe5025a5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe4fc0b44d0 0x7fe4fc1403f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:09.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.660+0000 7fe5025a5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe4fc06cf10 con 0x7fe4fc06c5f0 2024-04-24T23:01:09.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.660+0000 7fe5025a5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe4fc0b3fb0 con 0x7fe4fc0af890 2024-04-24T23:01:09.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.660+0000 7fe5025a5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe4fc0b5dd0 con 0x7fe4fc0b44d0 2024-04-24T23:01:09.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.660+0000 7fe4fbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4fc06c5f0 0x7fe4fc159040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:09.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.660+0000 7fe4fbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4fc06c5f0 0x7fe4fc159040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50304/0 (socket says 172.21.15.32:50304) 2024-04-24T23:01:09.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.660+0000 7fe4fbfff640 1 -- 172.21.15.32:0/742394654 learned_addr learned my addr 172.21.15.32:0/742394654 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:09.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.660+0000 7fe4fb7fe640 1 --2- 172.21.15.32:0/742394654 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe4fc0af890 0x7fe4fc143810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:09.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.660+0000 7fe500b42640 1 --2- 172.21.15.32:0/742394654 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe4fc0b44d0 0x7fe4fc1403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:09.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.660+0000 7fe4fbfff640 1 -- 172.21.15.32:0/742394654 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe4fc0b44d0 msgr2=0x7fe4fc1403f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:09.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.660+0000 7fe4fbfff640 1 --2- 172.21.15.32:0/742394654 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe4fc0b44d0 0x7fe4fc1403f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:09.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.660+0000 7fe4fbfff640 1 -- 172.21.15.32:0/742394654 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe4fc0af890 msgr2=0x7fe4fc143810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:09.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.660+0000 7fe4fbfff640 1 --2- 172.21.15.32:0/742394654 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe4fc0af890 0x7fe4fc143810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:09.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.660+0000 7fe4fbfff640 1 -- 172.21.15.32:0/742394654 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4fc159750 con 0x7fe4fc06c5f0 2024-04-24T23:01:09.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.661+0000 7fe4fbfff640 1 --2- 172.21.15.32:0/742394654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4fc06c5f0 0x7fe4fc159040 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec000c00 tx=0x7fe4ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:09.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.661+0000 7fe4f8ff9640 1 -- 172.21.15.32:0/742394654 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec041070 con 0x7fe4fc06c5f0 2024-04-24T23:01:09.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.661+0000 7fe4f8ff9640 1 -- 172.21.15.32:0/742394654 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4ec03c870 con 0x7fe4fc06c5f0 2024-04-24T23:01:09.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.661+0000 7fe5025a5640 1 -- 172.21.15.32:0/742394654 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4fc159a30 con 0x7fe4fc06c5f0 2024-04-24T23:01:09.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.661+0000 7fe4f8ff9640 1 -- 172.21.15.32:0/742394654 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec037040 con 0x7fe4fc06c5f0 2024-04-24T23:01:09.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.661+0000 7fe5025a5640 1 -- 172.21.15.32:0/742394654 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4fc159ff0 con 0x7fe4fc06c5f0 2024-04-24T23:01:09.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.662+0000 7fe5025a5640 1 -- 172.21.15.32:0/742394654 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4fc068430 con 0x7fe4fc06c5f0 2024-04-24T23:01:09.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.662+0000 7fe4f8ff9640 1 -- 172.21.15.32:0/742394654 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe4ec044070 con 0x7fe4fc06c5f0 2024-04-24T23:01:09.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.663+0000 7fe4f8ff9640 1 --2- 172.21.15.32:0/742394654 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe4dc06be60 0x7fe4dc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:09.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.663+0000 7fe4f8ff9640 1 -- 172.21.15.32:0/742394654 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5854+0+0 (secure 0 0 0) 0x7fe4ec0abb60 con 0x7fe4fc06c5f0 2024-04-24T23:01:09.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.663+0000 7fe4fb7fe640 1 --2- 172.21.15.32:0/742394654 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe4dc06be60 0x7fe4dc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:09.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.663+0000 7fe4fb7fe640 1 --2- 172.21.15.32:0/742394654 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe4dc06be60 0x7fe4dc06e320 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc072180 tx=0x7fe4e4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:09.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.667+0000 7fe4f8ff9640 1 -- 172.21.15.32:0/742394654 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe4ec07a630 con 0x7fe4fc06c5f0 2024-04-24T23:01:09.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.859+0000 7fe5025a5640 1 -- 172.21.15.32:0/742394654 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe4fc0b5670 con 0x7fe4fc06c5f0 2024-04-24T23:01:09.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.860+0000 7fe4f8ff9640 1 -- 172.21.15.32:0/742394654 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v56) ==== 80+0+0 (secure 0 0 0) 0x7fe4ec079f30 con 0x7fe4fc06c5f0 2024-04-24T23:01:09.864 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2024-04-24T23:01:09.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.862+0000 7fe5025a5640 1 -- 172.21.15.32:0/742394654 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe4dc06be60 msgr2=0x7fe4dc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:09.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.862+0000 7fe5025a5640 1 --2- 172.21.15.32:0/742394654 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe4dc06be60 0x7fe4dc06e320 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc072180 tx=0x7fe4e4008040 comp rx=0 tx=0).stop 2024-04-24T23:01:09.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.862+0000 7fe5025a5640 1 -- 172.21.15.32:0/742394654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4fc06c5f0 msgr2=0x7fe4fc159040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:09.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.862+0000 7fe5025a5640 1 --2- 172.21.15.32:0/742394654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4fc06c5f0 0x7fe4fc159040 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec000c00 tx=0x7fe4ec0046c0 comp rx=0 tx=0).stop 2024-04-24T23:01:09.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.862+0000 7fe5025a5640 1 -- 172.21.15.32:0/742394654 shutdown_connections 2024-04-24T23:01:09.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.862+0000 7fe5025a5640 1 --2- 172.21.15.32:0/742394654 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe4dc06be60 0x7fe4dc06e320 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:09.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.862+0000 7fe5025a5640 1 --2- 172.21.15.32:0/742394654 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe4fc0b44d0 0x7fe4fc1403f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:09.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.862+0000 7fe5025a5640 1 --2- 172.21.15.32:0/742394654 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe4fc0af890 0x7fe4fc143810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:09.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.862+0000 7fe5025a5640 1 --2- 172.21.15.32:0/742394654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4fc06c5f0 0x7fe4fc159040 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:09.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.862+0000 7fe5025a5640 1 -- 172.21.15.32:0/742394654 >> 172.21.15.32:0/742394654 conn(0x7fe4fc06b110 msgr2=0x7fe4fc071750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:09.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.862+0000 7fe5025a5640 1 -- 172.21.15.32:0/742394654 shutdown_connections 2024-04-24T23:01:09.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:09.862+0000 7fe5025a5640 1 -- 172.21.15.32:0/742394654 wait complete. 2024-04-24T23:01:09.874 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-24T23:01:09.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:01:09.875 DEBUG:teuthology.orchestra.run.smithi121:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-24T23:01:09.907 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2024-04-24T23:01:09.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:01:09.907 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-24T23:01:09.965 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:01:09.965 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:01:09.966 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:01:09.982 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=413 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T23:01:09.982 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:01:09.982 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:01:09.982 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:01:10.023 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3271 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T23:01:10.023 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:01:10.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.122+0000 7f67a4076640 1 Processor -- start 2024-04-24T23:01:10.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.122+0000 7f67a4076640 1 -- start start 2024-04-24T23:01:10.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.123+0000 7f67a4076640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f679c0b5560 0x7f679c15e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:10.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.123+0000 7f67a4076640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f679c0cd3c0 0x7f679c15ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:10.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.123+0000 7f67a4076640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f679c0b5940 0x7f679c0b5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:10.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.123+0000 7f67a4076640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f679c0cd910 con 0x7f679c0b5940 2024-04-24T23:01:10.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.123+0000 7f67a4076640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f679c0cdbf0 con 0x7f679c0cd3c0 2024-04-24T23:01:10.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.123+0000 7f67a4076640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f679c0cded0 con 0x7f679c0b5560 2024-04-24T23:01:10.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.123+0000 7f67a2613640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f679c0b5940 0x7f679c0b5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:10.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.123+0000 7f67a2613640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f679c0b5940 0x7f679c0b5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43598/0 (socket says 172.21.15.32:43598) 2024-04-24T23:01:10.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.123+0000 7f67a2613640 1 -- 172.21.15.32:0/2018722607 learned_addr learned my addr 172.21.15.32:0/2018722607 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:10.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.123+0000 7f67a1e12640 1 --2- 172.21.15.32:0/2018722607 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f679c0b5560 0x7f679c15e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:10.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.123+0000 7f67a1611640 1 --2- 172.21.15.32:0/2018722607 >> v2:172.21.15.121:3300/0 conn(0x7f679c0cd3c0 0x7f679c15ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:10.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.123+0000 7f67a2613640 1 -- 172.21.15.32:0/2018722607 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f679c0b5560 msgr2=0x7f679c15e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:10.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.123+0000 7f67a2613640 1 --2- 172.21.15.32:0/2018722607 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f679c0b5560 0x7f679c15e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:10.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.123+0000 7f67a2613640 1 -- 172.21.15.32:0/2018722607 >> v2:172.21.15.121:3300/0 conn(0x7f679c0cd3c0 msgr2=0x7f679c15ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:10.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.123+0000 7f67a2613640 1 --2- 172.21.15.32:0/2018722607 >> v2:172.21.15.121:3300/0 conn(0x7f679c0cd3c0 0x7f679c15ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:10.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.123+0000 7f67a2613640 1 -- 172.21.15.32:0/2018722607 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f679c0ce1b0 con 0x7f679c0b5940 2024-04-24T23:01:10.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.124+0000 7f67a2613640 1 --2- 172.21.15.32:0/2018722607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f679c0b5940 0x7f679c0b5d90 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f679c06d830 tx=0x7f678c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23030ea42f60ce9e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:10.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.124+0000 7f67a0e10640 1 -- 172.21.15.32:0/2018722607 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678c036070 con 0x7f679c0b5940 2024-04-24T23:01:10.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.124+0000 7f67a0e10640 1 -- 172.21.15.32:0/2018722607 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f678c031600 con 0x7f679c0b5940 2024-04-24T23:01:10.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.124+0000 7f67a0e10640 1 -- 172.21.15.32:0/2018722607 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678c0319d0 con 0x7f679c0b5940 2024-04-24T23:01:10.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.125+0000 7f67a4076640 1 -- 172.21.15.32:0/2018722607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f679c0b5940 msgr2=0x7f679c0b5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:10.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.125+0000 7f67a4076640 1 --2- 172.21.15.32:0/2018722607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f679c0b5940 0x7f679c0b5d90 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f679c06d830 tx=0x7f678c02a6a0 comp rx=0 tx=0).stop 2024-04-24T23:01:10.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.125+0000 7f67a4076640 1 -- 172.21.15.32:0/2018722607 shutdown_connections 2024-04-24T23:01:10.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.125+0000 7f67a4076640 1 --2- 172.21.15.32:0/2018722607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f679c0b5940 0x7f679c0b5d90 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:10.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.125+0000 7f67a4076640 1 --2- 172.21.15.32:0/2018722607 >> v2:172.21.15.121:3300/0 conn(0x7f679c0cd3c0 0x7f679c15ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:10.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.125+0000 7f67a4076640 1 --2- 172.21.15.32:0/2018722607 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f679c0b5560 0x7f679c15e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:10.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.125+0000 7f67a4076640 1 -- 172.21.15.32:0/2018722607 >> 172.21.15.32:0/2018722607 conn(0x7f679c06b110 msgr2=0x7f679c06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:10.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.125+0000 7f67a4076640 1 -- 172.21.15.32:0/2018722607 shutdown_connections 2024-04-24T23:01:10.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.125+0000 7f67a4076640 1 -- 172.21.15.32:0/2018722607 wait complete. 2024-04-24T23:01:10.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.126+0000 7f67a4076640 1 Processor -- start 2024-04-24T23:01:10.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.126+0000 7f67a4076640 1 -- start start 2024-04-24T23:01:10.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.127+0000 7f67a4076640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f679c0b5560 0x7f679c142310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:10.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.127+0000 7f67a4076640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f679c0b5940 0x7f679c152590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:10.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.127+0000 7f67a4076640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f679c0cd3c0 0x7f679c0ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:10.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.127+0000 7f67a4076640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f679c0b6450 con 0x7f679c0b5940 2024-04-24T23:01:10.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.127+0000 7f67a4076640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f679c0b62d0 con 0x7f679c0b5560 2024-04-24T23:01:10.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.127+0000 7f67a4076640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f679c0b4420 con 0x7f679c0cd3c0 2024-04-24T23:01:10.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.127+0000 7f67a1611640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f679c0b5940 0x7f679c152590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:10.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.127+0000 7f67a1611640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f679c0b5940 0x7f679c152590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43606/0 (socket says 172.21.15.32:43606) 2024-04-24T23:01:10.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.127+0000 7f67a1611640 1 -- 172.21.15.32:0/705618497 learned_addr learned my addr 172.21.15.32:0/705618497 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:10.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.127+0000 7f67a2613640 1 --2- 172.21.15.32:0/705618497 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f679c0cd3c0 0x7f679c0ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:10.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.127+0000 7f67a1e12640 1 --2- 172.21.15.32:0/705618497 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f679c0b5560 0x7f679c142310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:10.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.127+0000 7f67a1611640 1 -- 172.21.15.32:0/705618497 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f679c0cd3c0 msgr2=0x7f679c0ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:10.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.127+0000 7f67a1611640 1 --2- 172.21.15.32:0/705618497 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f679c0cd3c0 0x7f679c0ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:10.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.127+0000 7f67a1611640 1 -- 172.21.15.32:0/705618497 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f679c0b5560 msgr2=0x7f679c142310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:10.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.127+0000 7f67a1611640 1 --2- 172.21.15.32:0/705618497 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f679c0b5560 0x7f679c142310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:10.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.127+0000 7f67a1611640 1 -- 172.21.15.32:0/705618497 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f679c145a40 con 0x7f679c0b5940 2024-04-24T23:01:10.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.127+0000 7f67a1611640 1 --2- 172.21.15.32:0/705618497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f679c0b5940 0x7f679c152590 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f679000e8a0 tx=0x7f679000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:10.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.128+0000 7f679a7fc640 1 -- 172.21.15.32:0/705618497 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6790017070 con 0x7f679c0b5940 2024-04-24T23:01:10.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.128+0000 7f679a7fc640 1 -- 172.21.15.32:0/705618497 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f679000c940 con 0x7f679c0b5940 2024-04-24T23:01:10.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.128+0000 7f67a4076640 1 -- 172.21.15.32:0/705618497 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f679c1517d0 con 0x7f679c0b5940 2024-04-24T23:01:10.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.128+0000 7f679a7fc640 1 -- 172.21.15.32:0/705618497 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f679000cd30 con 0x7f679c0b5940 2024-04-24T23:01:10.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.128+0000 7f67a4076640 1 -- 172.21.15.32:0/705618497 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f679c151d90 con 0x7f679c0b5940 2024-04-24T23:01:10.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.129+0000 7f67a4076640 1 -- 172.21.15.32:0/705618497 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f679c068430 con 0x7f679c0b5940 2024-04-24T23:01:10.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.129+0000 7f679a7fc640 1 -- 172.21.15.32:0/705618497 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f679001a070 con 0x7f679c0b5940 2024-04-24T23:01:10.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.130+0000 7f679a7fc640 1 --2- 172.21.15.32:0/705618497 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f678806be30 0x7f678806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:10.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.130+0000 7f679a7fc640 1 -- 172.21.15.32:0/705618497 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5854+0+0 (secure 0 0 0) 0x7f679008aca0 con 0x7f679c0b5940 2024-04-24T23:01:10.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.130+0000 7f67a1e12640 1 --2- 172.21.15.32:0/705618497 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f678806be30 0x7f678806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:10.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.130+0000 7f67a1e12640 1 --2- 172.21.15.32:0/705618497 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f678806be30 0x7f678806e2f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6794009aa0 tx=0x7f6794009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:10.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.134+0000 7f679a7fc640 1 -- 172.21.15.32:0/705618497 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6790058ee0 con 0x7f679c0b5940 2024-04-24T23:01:10.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.354+0000 7f67a4076640 1 -- 172.21.15.32:0/705618497 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f679c06d5f0 con 0x7f679c0b5940 2024-04-24T23:01:10.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.355+0000 7f679a7fc640 1 -- 172.21.15.32:0/705618497 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5187 (secure 0 0 0) 0x7f679005d1e0 con 0x7f679c0b5940 2024-04-24T23:01:10.357 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:01:10.357 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":20,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4994,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/1076304225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1076304225},{"type":"v1","addr":"172.21.15.32:6834","nonce":1076304225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":16},{"gid":5000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1388367206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1388367206},{"type":"v1","addr":"172.21.15.121:6835","nonce":1388367206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":5299,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2065945523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2065945523},{"type":"v1","addr":"172.21.15.121:6837","nonce":2065945523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:00:32.376194+0000","modified":"2024-04-24T23:00:34.381949+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"c","rank":0,"incarnation":18,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/3274703471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":3274703471},{"type":"v1","addr":"172.21.15.32:6835","nonce":3274703471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5631,"qdb_cluster":[5631]},"id":3}]} 2024-04-24T23:01:10.357 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 20 2024-04-24T23:01:10.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.357+0000 7f67a4076640 1 -- 172.21.15.32:0/705618497 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f678806be30 msgr2=0x7f678806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:10.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.357+0000 7f67a4076640 1 --2- 172.21.15.32:0/705618497 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f678806be30 0x7f678806e2f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6794009aa0 tx=0x7f6794009190 comp rx=0 tx=0).stop 2024-04-24T23:01:10.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.357+0000 7f67a4076640 1 -- 172.21.15.32:0/705618497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f679c0b5940 msgr2=0x7f679c152590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:10.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.357+0000 7f67a4076640 1 --2- 172.21.15.32:0/705618497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f679c0b5940 0x7f679c152590 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f679000e8a0 tx=0x7f679000ed60 comp rx=0 tx=0).stop 2024-04-24T23:01:10.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.357+0000 7f67a4076640 1 -- 172.21.15.32:0/705618497 shutdown_connections 2024-04-24T23:01:10.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.357+0000 7f67a4076640 1 --2- 172.21.15.32:0/705618497 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f678806be30 0x7f678806e2f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:10.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.357+0000 7f67a4076640 1 --2- 172.21.15.32:0/705618497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f679c0b5940 0x7f679c152590 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:10.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.357+0000 7f67a4076640 1 --2- 172.21.15.32:0/705618497 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f679c0cd3c0 0x7f679c0ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:10.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.357+0000 7f67a4076640 1 --2- 172.21.15.32:0/705618497 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f679c0b5560 0x7f679c142310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:10.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.357+0000 7f67a4076640 1 -- 172.21.15.32:0/705618497 >> 172.21.15.32:0/705618497 conn(0x7f679c06b110 msgr2=0x7f679c0b0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:10.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.357+0000 7f67a4076640 1 -- 172.21.15.32:0/705618497 shutdown_connections 2024-04-24T23:01:10.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.357+0000 7f67a4076640 1 -- 172.21.15.32:0/705618497 wait complete. 2024-04-24T23:01:10.370 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:01:10.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.469+0000 7f3310ebd640 1 Processor -- start 2024-04-24T23:01:10.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.470+0000 7f3310ebd640 1 -- start start 2024-04-24T23:01:10.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.470+0000 7f3310ebd640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f330c06b820 0x7f330c0d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:10.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.470+0000 7f3310ebd640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f330c0d4210 0x7f330c0b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:10.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.470+0000 7f3310ebd640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f330c0d3170 con 0x7f330c15e9b0 2024-04-24T23:01:10.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.470+0000 7f3310ebd640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f330c0d3450 con 0x7f330c0d4210 2024-04-24T23:01:10.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.470+0000 7f3310ebd640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f330c0d3730 con 0x7f330c06b820 2024-04-24T23:01:10.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.470+0000 7f330a59c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f330c15e9b0 0x7f330c06b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47744/0 (socket says 172.21.15.32:47744) 2024-04-24T23:01:10.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.471+0000 7f330a59c640 1 -- 172.21.15.32:0/3569630568 learned_addr learned my addr 172.21.15.32:0/3569630568 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:01:10.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.471+0000 7f330ad9d640 1 --2- 172.21.15.32:0/3569630568 >> v2:172.21.15.121:3300/0 conn(0x7f330c0d4210 0x7f330c0b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:10.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.471+0000 7f3309d9b640 1 -- 172.21.15.32:0/3569630568 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1179195073 0 0) 0x7f330c0d3170 con 0x7f330c15e9b0 2024-04-24T23:01:10.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.471+0000 7f3303fff640 1 --2- 172.21.15.32:0/3569630568 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f330c06b820 0x7f330c0d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:10.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.471+0000 7f3309d9b640 1 -- 172.21.15.32:0/3569630568 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32f8003520 con 0x7f330c15e9b0 2024-04-24T23:01:10.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.471+0000 7f3309d9b640 1 -- 172.21.15.32:0/3569630568 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3738327676 0 0) 0x7f32f8003520 con 0x7f330c15e9b0 2024-04-24T23:01:10.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.471+0000 7f3309d9b640 1 -- 172.21.15.32:0/3569630568 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f330c06b820 msgr2=0x7f330c0d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:10.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.471+0000 7f3309d9b640 1 --2- 172.21.15.32:0/3569630568 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f330c06b820 0x7f330c0d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:10.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.471+0000 7f3309d9b640 1 -- 172.21.15.32:0/3569630568 >> v2:172.21.15.121:3300/0 conn(0x7f330c0d4210 msgr2=0x7f330c0b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:10.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.471+0000 7f3309d9b640 1 --2- 172.21.15.32:0/3569630568 >> v2:172.21.15.121:3300/0 conn(0x7f330c0d4210 0x7f330c0b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:10.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.471+0000 7f3309d9b640 1 -- 172.21.15.32:0/3569630568 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f330c0d3a10 con 0x7f330c15e9b0 2024-04-24T23:01:10.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.471+0000 7f3309d9b640 1 -- 172.21.15.32:0/3569630568 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f32fc001670 con 0x7f330c15e9b0 2024-04-24T23:01:10.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.471+0000 7f3309d9b640 1 -- 172.21.15.32:0/3569630568 >> v1:172.21.15.32:6789/0 conn(0x7f330c15e9b0 legacy=0x7f330c06b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:10.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.471+0000 7f3309d9b640 1 --2- 172.21.15.32:0/3569630568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32f80039b0 0x7f32f8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:10.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.471+0000 7f3309d9b640 1 --2- 172.21.15.32:0/3569630568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32f8004800 0x7f32f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:10.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.471+0000 7f3309d9b640 1 --2- 172.21.15.32:0/3569630568 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f330c06b820 0x7f32f80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:10.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.471+0000 7f3309d9b640 1 -- 172.21.15.32:0/3569630568 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f330c0d3450 con 0x7f32f80039b0 2024-04-24T23:01:10.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.471+0000 7f3309d9b640 1 -- 172.21.15.32:0/3569630568 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f330c0d3730 con 0x7f32f8004800 2024-04-24T23:01:10.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.471+0000 7f3309d9b640 1 -- 172.21.15.32:0/3569630568 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f32f800b9c0 con 0x7f330c06b820 2024-04-24T23:01:10.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.471+0000 7f330a59c640 1 --2- 172.21.15.32:0/3569630568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32f80039b0 0x7f32f8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:10.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.471+0000 7f3303fff640 1 --2- 172.21.15.32:0/3569630568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32f8004800 0x7f32f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:10.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.471+0000 7f330ad9d640 1 --2- 172.21.15.32:0/3569630568 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f330c06b820 0x7f32f80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:10.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.472+0000 7f330a59c640 1 -- 172.21.15.32:0/3569630568 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f330c06b820 msgr2=0x7f32f80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:10.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.472+0000 7f330a59c640 1 --2- 172.21.15.32:0/3569630568 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f330c06b820 0x7f32f80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:10.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.472+0000 7f330a59c640 1 -- 172.21.15.32:0/3569630568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32f8004800 msgr2=0x7f32f8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:10.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.472+0000 7f330a59c640 1 --2- 172.21.15.32:0/3569630568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32f8004800 0x7f32f8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:10.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.472+0000 7f330a59c640 1 -- 172.21.15.32:0/3569630568 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32f8003520 con 0x7f32f80039b0 2024-04-24T23:01:10.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.472+0000 7f3303fff640 1 --2- 172.21.15.32:0/3569630568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32f8004800 0x7f32f8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:01:10.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.472+0000 7f330a59c640 1 --2- 172.21.15.32:0/3569630568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32f80039b0 0x7f32f8003dd0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f32fc00c270 tx=0x7f32f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1973e5f2aa2c2d01 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:10.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.472+0000 7f3309d9b640 1 -- 172.21.15.32:0/3569630568 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32fc007d30 con 0x7f32f80039b0 2024-04-24T23:01:10.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.472+0000 7f3309d9b640 1 -- 172.21.15.32:0/3569630568 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32fc005ad0 con 0x7f32f80039b0 2024-04-24T23:01:10.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.472+0000 7f3309d9b640 1 -- 172.21.15.32:0/3569630568 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32fc007070 con 0x7f32f80039b0 2024-04-24T23:01:10.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.473+0000 7f3310ebd640 1 -- 172.21.15.32:0/3569630568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32f80039b0 msgr2=0x7f32f8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:10.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.473+0000 7f3310ebd640 1 --2- 172.21.15.32:0/3569630568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32f80039b0 0x7f32f8003dd0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f32fc00c270 tx=0x7f32f8005830 comp rx=0 tx=0).stop 2024-04-24T23:01:10.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.473+0000 7f330ad9d640 1 -- 172.21.15.32:0/3569630568 reap_dead start 2024-04-24T23:01:10.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.473+0000 7f3310ebd640 1 -- 172.21.15.32:0/3569630568 shutdown_connections 2024-04-24T23:01:10.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.473+0000 7f3310ebd640 1 -- 172.21.15.32:0/3569630568 >> 172.21.15.32:0/3569630568 conn(0x7f330c0d11d0 msgr2=0x7f330c0d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:10.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.473+0000 7f3310ebd640 1 -- 172.21.15.32:0/3569630568 shutdown_connections 2024-04-24T23:01:10.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.473+0000 7f3310ebd640 1 -- 172.21.15.32:0/3569630568 wait complete. 2024-04-24T23:01:10.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.474+0000 7f3310ebd640 1 Processor -- start 2024-04-24T23:01:10.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.474+0000 7f3310ebd640 1 -- start start 2024-04-24T23:01:10.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.475+0000 7f3310ebd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32f80039b0 0x7f330c158370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:10.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.475+0000 7f3310ebd640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f330c0c90f0 0x7f330c0c9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:10.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.475+0000 7f3310ebd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f330c15f700 0x7f330c15fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:10.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.475+0000 7f3310ebd640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f330c0ba0e0 con 0x7f330c15f700 2024-04-24T23:01:10.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.475+0000 7f3310ebd640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f330c0d2400 con 0x7f32f80039b0 2024-04-24T23:01:10.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.475+0000 7f3310ebd640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f330c0ba260 con 0x7f330c0c90f0 2024-04-24T23:01:10.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.475+0000 7f330ad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f330c15f700 0x7f330c15fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:10.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.475+0000 7f330ad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f330c15f700 0x7f330c15fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43612/0 (socket says 172.21.15.32:43612) 2024-04-24T23:01:10.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.475+0000 7f330ad9d640 1 -- 172.21.15.32:0/2078163085 learned_addr learned my addr 172.21.15.32:0/2078163085 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:10.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.475+0000 7f330ad9d640 1 -- 172.21.15.32:0/2078163085 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f330c0c90f0 msgr2=0x7f330c0c9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:10.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.475+0000 7f330ad9d640 1 --2- 172.21.15.32:0/2078163085 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f330c0c90f0 0x7f330c0c9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:10.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.475+0000 7f330ad9d640 1 -- 172.21.15.32:0/2078163085 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32f80039b0 msgr2=0x7f330c158370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:10.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.475+0000 7f330ad9d640 1 --2- 172.21.15.32:0/2078163085 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32f80039b0 0x7f330c158370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:10.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.475+0000 7f330ad9d640 1 -- 172.21.15.32:0/2078163085 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f330c1f7780 con 0x7f330c15f700 2024-04-24T23:01:10.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.476+0000 7f330ad9d640 1 --2- 172.21.15.32:0/2078163085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f330c15f700 0x7f330c15fbb0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f3304007d10 tx=0x7f33040115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:10.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.476+0000 7f3302ffd640 1 -- 172.21.15.32:0/2078163085 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3304011de0 con 0x7f330c15f700 2024-04-24T23:01:10.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.476+0000 7f3302ffd640 1 -- 172.21.15.32:0/2078163085 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f330401b070 con 0x7f330c15f700 2024-04-24T23:01:10.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.476+0000 7f3310ebd640 1 -- 172.21.15.32:0/2078163085 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f330c1600f0 con 0x7f330c15f700 2024-04-24T23:01:10.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.476+0000 7f3302ffd640 1 -- 172.21.15.32:0/2078163085 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3304003990 con 0x7f330c15f700 2024-04-24T23:01:10.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.476+0000 7f3310ebd640 1 -- 172.21.15.32:0/2078163085 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f330c1f7d00 con 0x7f330c15f700 2024-04-24T23:01:10.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.477+0000 7f3310ebd640 1 -- 172.21.15.32:0/2078163085 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f330c068430 con 0x7f330c15f700 2024-04-24T23:01:10.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.477+0000 7f3302ffd640 1 -- 172.21.15.32:0/2078163085 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3304005d10 con 0x7f330c15f700 2024-04-24T23:01:10.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.478+0000 7f3302ffd640 1 --2- 172.21.15.32:0/2078163085 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f32ec06be60 0x7f32ec06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:10.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.478+0000 7f3302ffd640 1 -- 172.21.15.32:0/2078163085 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5854+0+0 (secure 0 0 0) 0x7f330408b230 con 0x7f330c15f700 2024-04-24T23:01:10.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.478+0000 7f330a59c640 1 --2- 172.21.15.32:0/2078163085 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f32ec06be60 0x7f32ec06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:10.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.478+0000 7f330a59c640 1 --2- 172.21.15.32:0/2078163085 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f32ec06be60 0x7f32ec06e320 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f32fc033000 tx=0x7f32fc007000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:10.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.482+0000 7f3302ffd640 1 -- 172.21.15.32:0/2078163085 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3304014350 con 0x7f330c15f700 2024-04-24T23:01:10.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.665+0000 7f3310ebd640 1 -- 172.21.15.32:0/2078163085 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f330c15dc80 con 0x7f330c15f700 2024-04-24T23:01:10.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.666+0000 7f3302ffd640 1 -- 172.21.15.32:0/2078163085 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) ==== 74+0+16912 (secure 0 0 0) 0x7f3304059b30 con 0x7f330c15f700 2024-04-24T23:01:10.668 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:01:10.668 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":56,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:01:00.283572+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:00:30.665592+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-24T23:00:31.357756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.6","mappings":[{"from":5,"to":7}]},{"pgid":"7.c","mappings":[{"from":5,"to":7}]},{"pgid":"7.12","mappings":[{"from":5,"to":7}]},{"pgid":"7.31","mappings":[{"from":5,"to":7}]},{"pgid":"7.35","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:01:10.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.668+0000 7f3310ebd640 1 -- 172.21.15.32:0/2078163085 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f32ec06be60 msgr2=0x7f32ec06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:10.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.668+0000 7f3310ebd640 1 --2- 172.21.15.32:0/2078163085 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f32ec06be60 0x7f32ec06e320 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f32fc033000 tx=0x7f32fc007000 comp rx=0 tx=0).stop 2024-04-24T23:01:10.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.668+0000 7f3310ebd640 1 -- 172.21.15.32:0/2078163085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f330c15f700 msgr2=0x7f330c15fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:10.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.668+0000 7f3310ebd640 1 --2- 172.21.15.32:0/2078163085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f330c15f700 0x7f330c15fbb0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f3304007d10 tx=0x7f33040115a0 comp rx=0 tx=0).stop 2024-04-24T23:01:10.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.668+0000 7f3310ebd640 1 -- 172.21.15.32:0/2078163085 shutdown_connections 2024-04-24T23:01:10.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.668+0000 7f3310ebd640 1 --2- 172.21.15.32:0/2078163085 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f32ec06be60 0x7f32ec06e320 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:10.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.668+0000 7f3310ebd640 1 --2- 172.21.15.32:0/2078163085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f330c15f700 0x7f330c15fbb0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:10.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.668+0000 7f3310ebd640 1 --2- 172.21.15.32:0/2078163085 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f330c0c90f0 0x7f330c0c9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:10.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.668+0000 7f3310ebd640 1 --2- 172.21.15.32:0/2078163085 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32f80039b0 0x7f330c158370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:10.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.668+0000 7f3310ebd640 1 -- 172.21.15.32:0/2078163085 >> 172.21.15.32:0/2078163085 conn(0x7f330c0d11d0 msgr2=0x7f330c15d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:10.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.668+0000 7f3310ebd640 1 -- 172.21.15.32:0/2078163085 shutdown_connections 2024-04-24T23:01:10.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.668+0000 7f3310ebd640 1 -- 172.21.15.32:0/2078163085 wait complete. 2024-04-24T23:01:10.681 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:00:30.665592+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:00:31.357756+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:01:10.681 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:01:10.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.783+0000 7ff16f3fb640 1 Processor -- start 2024-04-24T23:01:10.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.783+0000 7ff16f3fb640 1 -- start start 2024-04-24T23:01:10.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.784+0000 7ff16f3fb640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff16815e9b0 0x7ff1680d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:10.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.784+0000 7ff16f3fb640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff1680d4210 0x7ff16806b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:10.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.784+0000 7ff16f3fb640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff1680ba0e0 0x7ff16806b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:10.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.784+0000 7ff16f3fb640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7ff1680d3170 con 0x7ff1680ba0e0 2024-04-24T23:01:10.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.784+0000 7ff16f3fb640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7ff1680d3450 con 0x7ff1680d4210 2024-04-24T23:01:10.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.784+0000 7ff16f3fb640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff1680d3730 con 0x7ff16815e9b0 2024-04-24T23:01:10.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.784+0000 7ff16d998640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff1680ba0e0 0x7ff16806b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:10.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.784+0000 7ff16d998640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff1680ba0e0 0x7ff16806b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43618/0 (socket says 172.21.15.32:43618) 2024-04-24T23:01:10.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.784+0000 7ff16d998640 1 -- 172.21.15.32:0/1481089795 learned_addr learned my addr 172.21.15.32:0/1481089795 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:10.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.785+0000 7ff16d197640 1 --2- 172.21.15.32:0/1481089795 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff16815e9b0 0x7ff1680d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:10.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.785+0000 7ff16c996640 1 --2- 172.21.15.32:0/1481089795 >> v2:172.21.15.121:3300/0 conn(0x7ff1680d4210 0x7ff16806b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:10.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.785+0000 7ff16d998640 1 -- 172.21.15.32:0/1481089795 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff16815e9b0 msgr2=0x7ff1680d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:10.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.785+0000 7ff16d998640 1 --2- 172.21.15.32:0/1481089795 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff16815e9b0 0x7ff1680d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:10.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.785+0000 7ff16d998640 1 -- 172.21.15.32:0/1481089795 >> v2:172.21.15.121:3300/0 conn(0x7ff1680d4210 msgr2=0x7ff16806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:10.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.785+0000 7ff16d998640 1 --2- 172.21.15.32:0/1481089795 >> v2:172.21.15.121:3300/0 conn(0x7ff1680d4210 0x7ff16806b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:10.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.785+0000 7ff16d998640 1 -- 172.21.15.32:0/1481089795 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1680d3a10 con 0x7ff1680ba0e0 2024-04-24T23:01:10.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.785+0000 7ff16d197640 1 --2- 172.21.15.32:0/1481089795 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff16815e9b0 0x7ff1680d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:01:10.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.785+0000 7ff16c996640 1 --2- 172.21.15.32:0/1481089795 >> v2:172.21.15.121:3300/0 conn(0x7ff1680d4210 0x7ff16806b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:01:10.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.785+0000 7ff16d998640 1 --2- 172.21.15.32:0/1481089795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1680ba0e0 0x7ff16806b650 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7ff16806bc40 tx=0x7ff15802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca5f9d906d47dd58 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:10.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.786+0000 7ff167fff640 1 -- 172.21.15.32:0/1481089795 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff158036070 con 0x7ff1680ba0e0 2024-04-24T23:01:10.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.786+0000 7ff167fff640 1 -- 172.21.15.32:0/1481089795 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff158031600 con 0x7ff1680ba0e0 2024-04-24T23:01:10.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.786+0000 7ff167fff640 1 -- 172.21.15.32:0/1481089795 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1580319d0 con 0x7ff1680ba0e0 2024-04-24T23:01:10.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.786+0000 7ff16f3fb640 1 -- 172.21.15.32:0/1481089795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1680ba0e0 msgr2=0x7ff16806b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:10.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.786+0000 7ff16f3fb640 1 --2- 172.21.15.32:0/1481089795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1680ba0e0 0x7ff16806b650 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7ff16806bc40 tx=0x7ff15802a6a0 comp rx=0 tx=0).stop 2024-04-24T23:01:10.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.786+0000 7ff16f3fb640 1 -- 172.21.15.32:0/1481089795 shutdown_connections 2024-04-24T23:01:10.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.786+0000 7ff16f3fb640 1 --2- 172.21.15.32:0/1481089795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1680ba0e0 0x7ff16806b650 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:10.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.786+0000 7ff16f3fb640 1 --2- 172.21.15.32:0/1481089795 >> v2:172.21.15.121:3300/0 conn(0x7ff1680d4210 0x7ff16806b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:10.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.786+0000 7ff16f3fb640 1 --2- 172.21.15.32:0/1481089795 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff16815e9b0 0x7ff1680d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:10.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.786+0000 7ff16f3fb640 1 -- 172.21.15.32:0/1481089795 >> 172.21.15.32:0/1481089795 conn(0x7ff1680d11d0 msgr2=0x7ff1680d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:10.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.786+0000 7ff16f3fb640 1 -- 172.21.15.32:0/1481089795 shutdown_connections 2024-04-24T23:01:10.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.786+0000 7ff16f3fb640 1 -- 172.21.15.32:0/1481089795 wait complete. 2024-04-24T23:01:10.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.787+0000 7ff16f3fb640 1 Processor -- start 2024-04-24T23:01:10.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.787+0000 7ff16f3fb640 1 -- start start 2024-04-24T23:01:10.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.788+0000 7ff16f3fb640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff1680ba0e0 0x7ff168159d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:10.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.788+0000 7ff16f3fb640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff1680d4210 0x7ff1680ca600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:10.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.788+0000 7ff16f3fb640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff16815e9b0 0x7ff1680cd080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:10.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.788+0000 7ff16f3fb640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff168071730 con 0x7ff16815e9b0 2024-04-24T23:01:10.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.788+0000 7ff16f3fb640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff1680d2400 con 0x7ff1680d4210 2024-04-24T23:01:10.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.788+0000 7ff16f3fb640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff1680718b0 con 0x7ff1680ba0e0 2024-04-24T23:01:10.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.788+0000 7ff16d998640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff16815e9b0 0x7ff1680cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:10.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.788+0000 7ff16d998640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff16815e9b0 0x7ff1680cd080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43624/0 (socket says 172.21.15.32:43624) 2024-04-24T23:01:10.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.788+0000 7ff16d998640 1 -- 172.21.15.32:0/2565593334 learned_addr learned my addr 172.21.15.32:0/2565593334 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:10.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.789+0000 7ff16d197640 1 --2- 172.21.15.32:0/2565593334 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff1680ba0e0 0x7ff168159d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:10.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.789+0000 7ff16d998640 1 -- 172.21.15.32:0/2565593334 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff1680ba0e0 msgr2=0x7ff168159d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:10.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.789+0000 7ff16d998640 1 --2- 172.21.15.32:0/2565593334 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff1680ba0e0 0x7ff168159d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:10.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.789+0000 7ff16d998640 1 -- 172.21.15.32:0/2565593334 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff1680d4210 msgr2=0x7ff1680ca600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:10.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.789+0000 7ff16d998640 1 --2- 172.21.15.32:0/2565593334 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff1680d4210 0x7ff1680ca600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:10.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.789+0000 7ff16d998640 1 -- 172.21.15.32:0/2565593334 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1680cdd70 con 0x7ff16815e9b0 2024-04-24T23:01:10.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.789+0000 7ff16d998640 1 --2- 172.21.15.32:0/2565593334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff16815e9b0 0x7ff1680cd080 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7ff15802ac90 tx=0x7ff158004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:10.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.789+0000 7ff165ffb640 1 -- 172.21.15.32:0/2565593334 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15803f020 con 0x7ff16815e9b0 2024-04-24T23:01:10.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.789+0000 7ff165ffb640 1 -- 172.21.15.32:0/2565593334 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff158037040 con 0x7ff16815e9b0 2024-04-24T23:01:10.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.789+0000 7ff16f3fb640 1 -- 172.21.15.32:0/2565593334 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1680c9e20 con 0x7ff16815e9b0 2024-04-24T23:01:10.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.789+0000 7ff165ffb640 1 -- 172.21.15.32:0/2565593334 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff158036070 con 0x7ff16815e9b0 2024-04-24T23:01:10.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.789+0000 7ff16f3fb640 1 -- 172.21.15.32:0/2565593334 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff168150690 con 0x7ff16815e9b0 2024-04-24T23:01:10.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.790+0000 7ff16f3fb640 1 -- 172.21.15.32:0/2565593334 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff168068d40 con 0x7ff16815e9b0 2024-04-24T23:01:10.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.791+0000 7ff165ffb640 1 -- 172.21.15.32:0/2565593334 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff158042020 con 0x7ff16815e9b0 2024-04-24T23:01:10.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.791+0000 7ff165ffb640 1 --2- 172.21.15.32:0/2565593334 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff15406be30 0x7ff15406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:10.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.791+0000 7ff165ffb640 1 -- 172.21.15.32:0/2565593334 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5854+0+0 (secure 0 0 0) 0x7ff1580ab450 con 0x7ff16815e9b0 2024-04-24T23:01:10.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.791+0000 7ff16d197640 1 --2- 172.21.15.32:0/2565593334 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff15406be30 0x7ff15406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:10.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.792+0000 7ff16d197640 1 --2- 172.21.15.32:0/2565593334 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff15406be30 0x7ff15406e2f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff16000b840 tx=0x7ff160008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:10.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.795+0000 7ff165ffb640 1 -- 172.21.15.32:0/2565593334 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff168068d40 con 0x7ff16815e9b0 2024-04-24T23:01:10.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.976+0000 7ff16f3fb640 1 -- 172.21.15.32:0/2565593334 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff168068430 con 0x7ff16815e9b0 2024-04-24T23:01:10.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.976+0000 7ff165ffb640 1 -- 172.21.15.32:0/2565593334 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v56) ==== 93+0+10 (secure 0 0 0) 0x7ff158049030 con 0x7ff16815e9b0 2024-04-24T23:01:10.978 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T23:01:10.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.978+0000 7ff16f3fb640 1 -- 172.21.15.32:0/2565593334 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff15406be30 msgr2=0x7ff15406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:10.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.978+0000 7ff16f3fb640 1 --2- 172.21.15.32:0/2565593334 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff15406be30 0x7ff15406e2f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff16000b840 tx=0x7ff160008040 comp rx=0 tx=0).stop 2024-04-24T23:01:10.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.978+0000 7ff16f3fb640 1 -- 172.21.15.32:0/2565593334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff16815e9b0 msgr2=0x7ff1680cd080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:10.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.978+0000 7ff16f3fb640 1 --2- 172.21.15.32:0/2565593334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff16815e9b0 0x7ff1680cd080 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7ff15802ac90 tx=0x7ff158004a40 comp rx=0 tx=0).stop 2024-04-24T23:01:10.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.978+0000 7ff16f3fb640 1 -- 172.21.15.32:0/2565593334 shutdown_connections 2024-04-24T23:01:10.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.978+0000 7ff16f3fb640 1 --2- 172.21.15.32:0/2565593334 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff15406be30 0x7ff15406e2f0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:10.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.978+0000 7ff16f3fb640 1 --2- 172.21.15.32:0/2565593334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff16815e9b0 0x7ff1680cd080 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:10.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.978+0000 7ff16f3fb640 1 --2- 172.21.15.32:0/2565593334 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff1680d4210 0x7ff1680ca600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:10.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.978+0000 7ff16f3fb640 1 --2- 172.21.15.32:0/2565593334 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff1680ba0e0 0x7ff168159d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:10.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.978+0000 7ff16f3fb640 1 -- 172.21.15.32:0/2565593334 >> 172.21.15.32:0/2565593334 conn(0x7ff1680d11d0 msgr2=0x7ff1680cefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:10.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.978+0000 7ff16f3fb640 1 -- 172.21.15.32:0/2565593334 shutdown_connections 2024-04-24T23:01:10.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:10.978+0000 7ff16f3fb640 1 -- 172.21.15.32:0/2565593334 wait complete. 2024-04-24T23:01:10.991 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:01:11.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.093+0000 7fa2e0790640 1 Processor -- start 2024-04-24T23:01:11.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.093+0000 7fa2e0790640 1 -- start start 2024-04-24T23:01:11.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.093+0000 7fa2e0790640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2d815b380 0x7fa2d806c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:11.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.094+0000 7fa2e0790640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2d80b4520 con 0x7fa2d80ceb40 2024-04-24T23:01:11.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.094+0000 7fa2e0790640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2d80b4800 con 0x7fa2d815cb00 2024-04-24T23:01:11.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.094+0000 7fa2e0790640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa2d80cef90 con 0x7fa2d815b380 2024-04-24T23:01:11.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.094+0000 7fa2ded2d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa2d80ceb40 0x7fa2d80b55c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47754/0 (socket says 172.21.15.32:47754) 2024-04-24T23:01:11.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.094+0000 7fa2ded2d640 1 -- 172.21.15.32:0/3143524832 learned_addr learned my addr 172.21.15.32:0/3143524832 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:01:11.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.094+0000 7fa2de52c640 1 --2- 172.21.15.32:0/3143524832 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2d815b380 0x7fa2d806c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:11.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.094+0000 7fa2dd52a640 1 -- 172.21.15.32:0/3143524832 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2243155104 0 0) 0x7fa2d80b4520 con 0x7fa2d80ceb40 2024-04-24T23:01:11.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.094+0000 7fa2dd52a640 1 -- 172.21.15.32:0/3143524832 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa2c4003520 con 0x7fa2d80ceb40 2024-04-24T23:01:11.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.094+0000 7fa2dd52a640 1 -- 172.21.15.32:0/3143524832 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4248845301 0 0) 0x7fa2c4003520 con 0x7fa2d80ceb40 2024-04-24T23:01:11.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.094+0000 7fa2dd52a640 1 -- 172.21.15.32:0/3143524832 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2d815b380 msgr2=0x7fa2d806c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:11.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.094+0000 7fa2dd52a640 1 --2- 172.21.15.32:0/3143524832 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2d815b380 0x7fa2d806c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:11.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.094+0000 7fa2dd52a640 1 -- 172.21.15.32:0/3143524832 >> v1:172.21.15.121:6789/0 conn(0x7fa2d815cb00 legacy=0x7fa2d80ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:11.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.094+0000 7fa2dd52a640 1 -- 172.21.15.32:0/3143524832 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2d80b4ae0 con 0x7fa2d80ceb40 2024-04-24T23:01:11.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.094+0000 7fa2dd52a640 1 -- 172.21.15.32:0/3143524832 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7fa2d40019f0 con 0x7fa2d80ceb40 2024-04-24T23:01:11.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.094+0000 7fa2dd52a640 1 -- 172.21.15.32:0/3143524832 >> v1:172.21.15.32:6789/0 conn(0x7fa2d80ceb40 legacy=0x7fa2d80b55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:11.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.094+0000 7fa2dd52a640 1 --2- 172.21.15.32:0/3143524832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2c40039e0 0x7fa2c4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:11.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.094+0000 7fa2dd52a640 1 --2- 172.21.15.32:0/3143524832 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2c4004800 0x7fa2c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:11.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.094+0000 7fa2dd52a640 1 --2- 172.21.15.32:0/3143524832 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2d815b380 0x7fa2c4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:11.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.094+0000 7fa2dd52a640 1 -- 172.21.15.32:0/3143524832 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d80cef90 con 0x7fa2c40039e0 2024-04-24T23:01:11.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.094+0000 7fa2dd52a640 1 -- 172.21.15.32:0/3143524832 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c400baf0 con 0x7fa2c4004800 2024-04-24T23:01:11.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.094+0000 7fa2dd52a640 1 -- 172.21.15.32:0/3143524832 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa2c400bcf0 con 0x7fa2d815b380 2024-04-24T23:01:11.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.094+0000 7fa2de52c640 1 --2- 172.21.15.32:0/3143524832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2c40039e0 0x7fa2c4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:11.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.095+0000 7fa2ded2d640 1 --2- 172.21.15.32:0/3143524832 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2d815b380 0x7fa2c4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:11.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.095+0000 7fa2ddd2b640 1 --2- 172.21.15.32:0/3143524832 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2c4004800 0x7fa2c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:11.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.095+0000 7fa2de52c640 1 -- 172.21.15.32:0/3143524832 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2d815b380 msgr2=0x7fa2c4005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:11.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.095+0000 7fa2de52c640 1 --2- 172.21.15.32:0/3143524832 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2d815b380 0x7fa2c4005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:11.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.095+0000 7fa2de52c640 1 -- 172.21.15.32:0/3143524832 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2c4004800 msgr2=0x7fa2c4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:11.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.095+0000 7fa2de52c640 1 --2- 172.21.15.32:0/3143524832 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2c4004800 0x7fa2c4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:11.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.095+0000 7fa2de52c640 1 -- 172.21.15.32:0/3143524832 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2c4003520 con 0x7fa2c40039e0 2024-04-24T23:01:11.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.095+0000 7fa2de52c640 1 --2- 172.21.15.32:0/3143524832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2c40039e0 0x7fa2c4003e00 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0009ac0 tx=0x7fa2c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b182047757e13450 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:11.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.096+0000 7fa2dd52a640 1 -- 172.21.15.32:0/3143524832 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0036070 con 0x7fa2c40039e0 2024-04-24T23:01:11.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.096+0000 7fa2dd52a640 1 -- 172.21.15.32:0/3143524832 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2d0031400 con 0x7fa2c40039e0 2024-04-24T23:01:11.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.096+0000 7fa2dd52a640 1 -- 172.21.15.32:0/3143524832 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0009690 con 0x7fa2c40039e0 2024-04-24T23:01:11.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.096+0000 7fa2e0790640 1 -- 172.21.15.32:0/3143524832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2c40039e0 msgr2=0x7fa2c4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:11.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.096+0000 7fa2e0790640 1 --2- 172.21.15.32:0/3143524832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2c40039e0 0x7fa2c4003e00 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0009ac0 tx=0x7fa2c4005830 comp rx=0 tx=0).stop 2024-04-24T23:01:11.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.096+0000 7fa2ded2d640 1 -- 172.21.15.32:0/3143524832 reap_dead start 2024-04-24T23:01:11.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.096+0000 7fa2e0790640 1 -- 172.21.15.32:0/3143524832 shutdown_connections 2024-04-24T23:01:11.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.096+0000 7fa2e0790640 1 -- 172.21.15.32:0/3143524832 >> 172.21.15.32:0/3143524832 conn(0x7fa2d806b110 msgr2=0x7fa2d806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:11.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.096+0000 7fa2e0790640 1 -- 172.21.15.32:0/3143524832 shutdown_connections 2024-04-24T23:01:11.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.096+0000 7fa2e0790640 1 -- 172.21.15.32:0/3143524832 wait complete. 2024-04-24T23:01:11.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.097+0000 7fa2e0790640 1 Processor -- start 2024-04-24T23:01:11.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.097+0000 7fa2e0790640 1 -- start start 2024-04-24T23:01:11.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.098+0000 7fa2e0790640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2c40039e0 0x7fa2d80711c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:11.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.098+0000 7fa2e0790640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2d81454f0 0x7fa2d8070680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:11.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.098+0000 7fa2e0790640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2d8144bd0 0x7fa2d80ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:11.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.098+0000 7fa2e0790640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d815b760 con 0x7fa2d81454f0 2024-04-24T23:01:11.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.098+0000 7fa2e0790640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d80b50a0 con 0x7fa2d8144bd0 2024-04-24T23:01:11.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.098+0000 7fa2e0790640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa2d80b5cd0 con 0x7fa2c40039e0 2024-04-24T23:01:11.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.098+0000 7fa2ddd2b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2d81454f0 0x7fa2d8070680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:11.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.098+0000 7fa2ddd2b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2d81454f0 0x7fa2d8070680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43640/0 (socket says 172.21.15.32:43640) 2024-04-24T23:01:11.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.098+0000 7fa2ddd2b640 1 -- 172.21.15.32:0/380894589 learned_addr learned my addr 172.21.15.32:0/380894589 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:11.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.098+0000 7fa2ded2d640 1 --2- 172.21.15.32:0/380894589 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2d8144bd0 0x7fa2d80ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:11.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.098+0000 7fa2de52c640 1 --2- 172.21.15.32:0/380894589 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2c40039e0 0x7fa2d80711c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:11.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.099+0000 7fa2ddd2b640 1 -- 172.21.15.32:0/380894589 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2c40039e0 msgr2=0x7fa2d80711c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:11.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.099+0000 7fa2ddd2b640 1 --2- 172.21.15.32:0/380894589 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2c40039e0 0x7fa2d80711c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:11.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.099+0000 7fa2ddd2b640 1 -- 172.21.15.32:0/380894589 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2d8144bd0 msgr2=0x7fa2d80ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:11.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.099+0000 7fa2ddd2b640 1 --2- 172.21.15.32:0/380894589 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2d8144bd0 0x7fa2d80ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:11.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.099+0000 7fa2ddd2b640 1 -- 172.21.15.32:0/380894589 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2d8071810 con 0x7fa2d81454f0 2024-04-24T23:01:11.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.099+0000 7fa2ded2d640 1 --2- 172.21.15.32:0/380894589 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2d8144bd0 0x7fa2d80ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:01:11.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.099+0000 7fa2ddd2b640 1 --2- 172.21.15.32:0/380894589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2d81454f0 0x7fa2d8070680 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fa2c800f5f0 tx=0x7fa2c800fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:11.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.099+0000 7fa2ceffd640 1 -- 172.21.15.32:0/380894589 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c80052e0 con 0x7fa2d81454f0 2024-04-24T23:01:11.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.099+0000 7fa2ceffd640 1 -- 172.21.15.32:0/380894589 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2c8018070 con 0x7fa2d81454f0 2024-04-24T23:01:11.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.099+0000 7fa2e0790640 1 -- 172.21.15.32:0/380894589 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2d8071af0 con 0x7fa2d81454f0 2024-04-24T23:01:11.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.099+0000 7fa2ceffd640 1 -- 172.21.15.32:0/380894589 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c8003810 con 0x7fa2d81454f0 2024-04-24T23:01:11.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.099+0000 7fa2e0790640 1 -- 172.21.15.32:0/380894589 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2d8145050 con 0x7fa2d81454f0 2024-04-24T23:01:11.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.100+0000 7fa2e0790640 1 -- 172.21.15.32:0/380894589 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2d8068430 con 0x7fa2d81454f0 2024-04-24T23:01:11.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.100+0000 7fa2ceffd640 1 -- 172.21.15.32:0/380894589 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa2c80039b0 con 0x7fa2d81454f0 2024-04-24T23:01:11.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.101+0000 7fa2ceffd640 1 --2- 172.21.15.32:0/380894589 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa2bc06be60 0x7fa2bc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:11.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.101+0000 7fa2ceffd640 1 -- 172.21.15.32:0/380894589 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5854+0+0 (secure 0 0 0) 0x7fa2c808ad00 con 0x7fa2d81454f0 2024-04-24T23:01:11.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.101+0000 7fa2de52c640 1 --2- 172.21.15.32:0/380894589 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa2bc06be60 0x7fa2bc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:11.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.102+0000 7fa2de52c640 1 --2- 172.21.15.32:0/380894589 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa2bc06be60 0x7fa2bc06e320 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0033000 tx=0x7fa2d0008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:11.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.105+0000 7fa2ceffd640 1 -- 172.21.15.32:0/380894589 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa2c8059620 con 0x7fa2d81454f0 2024-04-24T23:01:11.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.286+0000 7fa2e0790640 1 -- 172.21.15.32:0/380894589 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa2d815ece0 con 0x7fa2d81454f0 2024-04-24T23:01:11.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.286+0000 7fa2ceffd640 1 -- 172.21.15.32:0/380894589 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v56) ==== 104+0+11 (secure 0 0 0) 0x7fa2c8058f20 con 0x7fa2d81454f0 2024-04-24T23:01:11.288 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-24T23:01:11.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.288+0000 7fa2e0790640 1 -- 172.21.15.32:0/380894589 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa2bc06be60 msgr2=0x7fa2bc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:11.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.288+0000 7fa2e0790640 1 --2- 172.21.15.32:0/380894589 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa2bc06be60 0x7fa2bc06e320 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0033000 tx=0x7fa2d0008030 comp rx=0 tx=0).stop 2024-04-24T23:01:11.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.288+0000 7fa2e0790640 1 -- 172.21.15.32:0/380894589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2d81454f0 msgr2=0x7fa2d8070680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:11.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.288+0000 7fa2e0790640 1 --2- 172.21.15.32:0/380894589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2d81454f0 0x7fa2d8070680 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fa2c800f5f0 tx=0x7fa2c800fab0 comp rx=0 tx=0).stop 2024-04-24T23:01:11.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.288+0000 7fa2e0790640 1 -- 172.21.15.32:0/380894589 shutdown_connections 2024-04-24T23:01:11.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.288+0000 7fa2e0790640 1 --2- 172.21.15.32:0/380894589 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa2bc06be60 0x7fa2bc06e320 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:11.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.288+0000 7fa2e0790640 1 --2- 172.21.15.32:0/380894589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2d81454f0 0x7fa2d8070680 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:11.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.288+0000 7fa2e0790640 1 --2- 172.21.15.32:0/380894589 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2d8144bd0 0x7fa2d80ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:11.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.288+0000 7fa2e0790640 1 --2- 172.21.15.32:0/380894589 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2c40039e0 0x7fa2d80711c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:11.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.288+0000 7fa2e0790640 1 -- 172.21.15.32:0/380894589 >> 172.21.15.32:0/380894589 conn(0x7fa2d806b110 msgr2=0x7fa2d806c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:11.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.288+0000 7fa2e0790640 1 -- 172.21.15.32:0/380894589 shutdown_connections 2024-04-24T23:01:11.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.288+0000 7fa2e0790640 1 -- 172.21.15.32:0/380894589 wait complete. 2024-04-24T23:01:11.300 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:01:11.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.403+0000 7f71f6e2c640 1 Processor -- start 2024-04-24T23:01:11.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.403+0000 7f71f6e2c640 1 -- start start 2024-04-24T23:01:11.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.404+0000 7f71f6e2c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71f006cd00 0x7f71f00ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:11.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.404+0000 7f71f6e2c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f71f015cb00 0x7f71f00ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:11.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.404+0000 7f71f6e2c640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71f00b4520 con 0x7f71f015b380 2024-04-24T23:01:11.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.404+0000 7f71f6e2c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f71f00b4800 con 0x7f71f015cb00 2024-04-24T23:01:11.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.404+0000 7f71f6e2c640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f71f00b4ae0 con 0x7f71f006cd00 2024-04-24T23:01:11.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.404+0000 7f71f53c9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f71f015cb00 0x7f71f00ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:11.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.404+0000 7f71f53c9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f71f015cb00 0x7f71f00ce970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43642/0 (socket says 172.21.15.32:43642) 2024-04-24T23:01:11.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.404+0000 7f71f53c9640 1 -- 172.21.15.32:0/782327003 learned_addr learned my addr 172.21.15.32:0/782327003 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:11.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.404+0000 7f71e7fff640 1 --2- 172.21.15.32:0/782327003 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71f006cd00 0x7f71f00ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:11.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.404+0000 7f71f53c9640 1 -- 172.21.15.32:0/782327003 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71f006cd00 msgr2=0x7f71f00ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:11.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.404+0000 7f71f53c9640 1 --2- 172.21.15.32:0/782327003 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71f006cd00 0x7f71f00ce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:11.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.404+0000 7f71f53c9640 1 -- 172.21.15.32:0/782327003 >> v1:172.21.15.121:6789/0 conn(0x7f71f015b380 legacy=0x7f71f006c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:11.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.404+0000 7f71f53c9640 1 -- 172.21.15.32:0/782327003 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71f00b4dc0 con 0x7f71f015cb00 2024-04-24T23:01:11.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.404+0000 7f71e7fff640 1 --2- 172.21.15.32:0/782327003 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71f006cd00 0x7f71f00ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:01:11.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.405+0000 7f71f53c9640 1 --2- 172.21.15.32:0/782327003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71f015cb00 0x7f71f00ce970 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f71e800d150 tx=0x7f71e802c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ed57a3627409f21 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:11.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.405+0000 7f71e77fe640 1 -- 172.21.15.32:0/782327003 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e8039070 con 0x7f71f015cb00 2024-04-24T23:01:11.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.405+0000 7f71e77fe640 1 -- 172.21.15.32:0/782327003 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71e8009600 con 0x7f71f015cb00 2024-04-24T23:01:11.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.405+0000 7f71e77fe640 1 -- 172.21.15.32:0/782327003 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e80099d0 con 0x7f71f015cb00 2024-04-24T23:01:11.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.406+0000 7f71f6e2c640 1 -- 172.21.15.32:0/782327003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71f015cb00 msgr2=0x7f71f00ce970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:11.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.406+0000 7f71f6e2c640 1 --2- 172.21.15.32:0/782327003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71f015cb00 0x7f71f00ce970 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f71e800d150 tx=0x7f71e802c890 comp rx=0 tx=0).stop 2024-04-24T23:01:11.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.406+0000 7f71f6e2c640 1 -- 172.21.15.32:0/782327003 shutdown_connections 2024-04-24T23:01:11.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.406+0000 7f71f6e2c640 1 --2- 172.21.15.32:0/782327003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71f015cb00 0x7f71f00ce970 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:11.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.406+0000 7f71f6e2c640 1 --2- 172.21.15.32:0/782327003 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71f006cd00 0x7f71f00ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:11.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.406+0000 7f71f6e2c640 1 -- 172.21.15.32:0/782327003 >> 172.21.15.32:0/782327003 conn(0x7f71f006b110 msgr2=0x7f71f006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:11.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.406+0000 7f71f6e2c640 1 -- 172.21.15.32:0/782327003 shutdown_connections 2024-04-24T23:01:11.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.406+0000 7f71f6e2c640 1 -- 172.21.15.32:0/782327003 wait complete. 2024-04-24T23:01:11.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.407+0000 7f71f6e2c640 1 Processor -- start 2024-04-24T23:01:11.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.407+0000 7f71f6e2c640 1 -- start start 2024-04-24T23:01:11.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.408+0000 7f71f6e2c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71f006cd00 0x7f71f0150d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:11.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.408+0000 7f71f6e2c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71f015b380 0x7f71f013f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:11.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.408+0000 7f71f6e2c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71f015cb00 0x7f71f0072140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:11.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.408+0000 7f71f6e2c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f71f015b760 con 0x7f71f015b380 2024-04-24T23:01:11.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.408+0000 7f71f6e2c640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f71f00ceeb0 con 0x7f71f006cd00 2024-04-24T23:01:11.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.408+0000 7f71f6e2c640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f71f00b55c0 con 0x7f71f015cb00 2024-04-24T23:01:11.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.408+0000 7f71e7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71f015b380 0x7f71f013f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:11.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.408+0000 7f71e7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71f015b380 0x7f71f013f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43644/0 (socket says 172.21.15.32:43644) 2024-04-24T23:01:11.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.408+0000 7f71e7fff640 1 -- 172.21.15.32:0/2975688887 learned_addr learned my addr 172.21.15.32:0/2975688887 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:11.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.408+0000 7f71f53c9640 1 --2- 172.21.15.32:0/2975688887 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71f015cb00 0x7f71f0072140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:11.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.408+0000 7f71f4bc8640 1 --2- 172.21.15.32:0/2975688887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71f006cd00 0x7f71f0150d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:11.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.408+0000 7f71e7fff640 1 -- 172.21.15.32:0/2975688887 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71f015cb00 msgr2=0x7f71f0072140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:11.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.408+0000 7f71e7fff640 1 --2- 172.21.15.32:0/2975688887 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71f015cb00 0x7f71f0072140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:11.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.408+0000 7f71e7fff640 1 -- 172.21.15.32:0/2975688887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71f006cd00 msgr2=0x7f71f0150d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:11.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.408+0000 7f71e7fff640 1 --2- 172.21.15.32:0/2975688887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71f006cd00 0x7f71f0150d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:11.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.408+0000 7f71e7fff640 1 -- 172.21.15.32:0/2975688887 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71f0141ed0 con 0x7f71f015b380 2024-04-24T23:01:11.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.408+0000 7f71f4bc8640 1 --2- 172.21.15.32:0/2975688887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71f006cd00 0x7f71f0150d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:01:11.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.408+0000 7f71e7fff640 1 --2- 172.21.15.32:0/2975688887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71f015b380 0x7f71f013f600 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f71e000e730 tx=0x7f71e000ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:11.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.409+0000 7f71e57fa640 1 -- 172.21.15.32:0/2975688887 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e000f040 con 0x7f71f015b380 2024-04-24T23:01:11.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.409+0000 7f71e57fa640 1 -- 172.21.15.32:0/2975688887 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71e0004230 con 0x7f71f015b380 2024-04-24T23:01:11.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.409+0000 7f71f6e2c640 1 -- 172.21.15.32:0/2975688887 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71f01421b0 con 0x7f71f015b380 2024-04-24T23:01:11.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.409+0000 7f71e57fa640 1 -- 172.21.15.32:0/2975688887 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e000ee40 con 0x7f71f015b380 2024-04-24T23:01:11.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.409+0000 7f71f6e2c640 1 -- 172.21.15.32:0/2975688887 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71f0142770 con 0x7f71f015b380 2024-04-24T23:01:11.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.410+0000 7f71f6e2c640 1 -- 172.21.15.32:0/2975688887 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71f0068430 con 0x7f71f015b380 2024-04-24T23:01:11.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.410+0000 7f71e57fa640 1 -- 172.21.15.32:0/2975688887 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f71e000c6d0 con 0x7f71f015b380 2024-04-24T23:01:11.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.411+0000 7f71e57fa640 1 --2- 172.21.15.32:0/2975688887 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f71dc06be30 0x7f71dc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:11.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.411+0000 7f71e57fa640 1 -- 172.21.15.32:0/2975688887 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5854+0+0 (secure 0 0 0) 0x7f71e008ab80 con 0x7f71f015b380 2024-04-24T23:01:11.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.411+0000 7f71f4bc8640 1 --2- 172.21.15.32:0/2975688887 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f71dc06be30 0x7f71dc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:11.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.411+0000 7f71f4bc8640 1 --2- 172.21.15.32:0/2975688887 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f71dc06be30 0x7f71dc06e2f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f71f006be00 tx=0x7f71ec009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:11.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.415+0000 7f71e57fa640 1 -- 172.21.15.32:0/2975688887 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f71e0017020 con 0x7f71f015b380 2024-04-24T23:01:11.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.595+0000 7f71f6e2c640 1 -- 172.21.15.32:0/2975688887 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f71f00afd60 con 0x7f71f015b380 2024-04-24T23:01:11.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.595+0000 7f71e57fa640 1 -- 172.21.15.32:0/2975688887 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v56) ==== 100+0+11 (secure 0 0 0) 0x7f71e0058e40 con 0x7f71f015b380 2024-04-24T23:01:11.597 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-24T23:01:11.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.597+0000 7f71f6e2c640 1 -- 172.21.15.32:0/2975688887 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f71dc06be30 msgr2=0x7f71dc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:11.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.597+0000 7f71f6e2c640 1 --2- 172.21.15.32:0/2975688887 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f71dc06be30 0x7f71dc06e2f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f71f006be00 tx=0x7f71ec009040 comp rx=0 tx=0).stop 2024-04-24T23:01:11.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.597+0000 7f71f6e2c640 1 -- 172.21.15.32:0/2975688887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71f015b380 msgr2=0x7f71f013f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:11.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.597+0000 7f71f6e2c640 1 --2- 172.21.15.32:0/2975688887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71f015b380 0x7f71f013f600 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f71e000e730 tx=0x7f71e000ebf0 comp rx=0 tx=0).stop 2024-04-24T23:01:11.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.597+0000 7f71f6e2c640 1 -- 172.21.15.32:0/2975688887 shutdown_connections 2024-04-24T23:01:11.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.597+0000 7f71f6e2c640 1 --2- 172.21.15.32:0/2975688887 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f71dc06be30 0x7f71dc06e2f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:11.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.597+0000 7f71f6e2c640 1 --2- 172.21.15.32:0/2975688887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71f015b380 0x7f71f013f600 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:11.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.597+0000 7f71f6e2c640 1 --2- 172.21.15.32:0/2975688887 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71f015cb00 0x7f71f0072140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:11.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.597+0000 7f71f6e2c640 1 --2- 172.21.15.32:0/2975688887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71f006cd00 0x7f71f0150d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:11.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.597+0000 7f71f6e2c640 1 -- 172.21.15.32:0/2975688887 >> 172.21.15.32:0/2975688887 conn(0x7f71f006b110 msgr2=0x7f71f015ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:11.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.597+0000 7f71f6e2c640 1 -- 172.21.15.32:0/2975688887 shutdown_connections 2024-04-24T23:01:11.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.597+0000 7f71f6e2c640 1 -- 172.21.15.32:0/2975688887 wait complete. 2024-04-24T23:01:11.610 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:01:11.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.709+0000 7f2365e15640 1 Processor -- start 2024-04-24T23:01:11.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.709+0000 7f2365e15640 1 -- start start 2024-04-24T23:01:11.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.709+0000 7f2365e15640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f236015e9b0 0x7f23600d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:11.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.709+0000 7f2365e15640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f23600d4210 0x7f236006b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:11.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.709+0000 7f2365e15640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23600d3170 con 0x7f23600ba0e0 2024-04-24T23:01:11.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.709+0000 7f2365e15640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f23600b8900 con 0x7f23600d4210 2024-04-24T23:01:11.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.709+0000 7f2365e15640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f23600d3450 con 0x7f236015e9b0 2024-04-24T23:01:11.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.710+0000 7f235effd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f23600d4210 0x7f236006b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:11.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.710+0000 7f235ffff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f23600ba0e0 0x7f2360071730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47758/0 (socket says 172.21.15.32:47758) 2024-04-24T23:01:11.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.710+0000 7f235ffff640 1 -- 172.21.15.32:0/312727759 learned_addr learned my addr 172.21.15.32:0/312727759 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:01:11.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.710+0000 7f235f7fe640 1 --2- 172.21.15.32:0/312727759 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f236015e9b0 0x7f23600d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:11.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.710+0000 7f235effd640 1 -- 172.21.15.32:0/312727759 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f236015e9b0 msgr2=0x7f23600d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:11.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.710+0000 7f235effd640 1 --2- 172.21.15.32:0/312727759 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f236015e9b0 0x7f23600d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:11.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.710+0000 7f235effd640 1 -- 172.21.15.32:0/312727759 >> v1:172.21.15.32:6789/0 conn(0x7f23600ba0e0 legacy=0x7f2360071730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:11.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.710+0000 7f235effd640 1 -- 172.21.15.32:0/312727759 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23600d3730 con 0x7f23600d4210 2024-04-24T23:01:11.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.710+0000 7f235f7fe640 1 --2- 172.21.15.32:0/312727759 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f236015e9b0 0x7f23600d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:01:11.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.711+0000 7f235effd640 1 --2- 172.21.15.32:0/312727759 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23600d4210 0x7f236006b110 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f23540097a0 tx=0x7f235402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=948dae277114f1d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:11.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.711+0000 7f235e7fc640 1 -- 172.21.15.32:0/312727759 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2354037070 con 0x7f23600d4210 2024-04-24T23:01:11.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.711+0000 7f235e7fc640 1 -- 172.21.15.32:0/312727759 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2354032440 con 0x7f23600d4210 2024-04-24T23:01:11.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.711+0000 7f235e7fc640 1 -- 172.21.15.32:0/312727759 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2354032810 con 0x7f23600d4210 2024-04-24T23:01:11.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.711+0000 7f2365e15640 1 -- 172.21.15.32:0/312727759 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23600d4210 msgr2=0x7f236006b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:11.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.711+0000 7f2365e15640 1 --2- 172.21.15.32:0/312727759 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23600d4210 0x7f236006b110 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f23540097a0 tx=0x7f235402a180 comp rx=0 tx=0).stop 2024-04-24T23:01:11.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.711+0000 7f2365e15640 1 -- 172.21.15.32:0/312727759 shutdown_connections 2024-04-24T23:01:11.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.711+0000 7f2365e15640 1 --2- 172.21.15.32:0/312727759 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23600d4210 0x7f236006b110 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:11.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.711+0000 7f2365e15640 1 --2- 172.21.15.32:0/312727759 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f236015e9b0 0x7f23600d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:11.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.711+0000 7f2365e15640 1 -- 172.21.15.32:0/312727759 >> 172.21.15.32:0/312727759 conn(0x7f23600d11d0 msgr2=0x7f23600d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:11.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.711+0000 7f2365e15640 1 -- 172.21.15.32:0/312727759 shutdown_connections 2024-04-24T23:01:11.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.711+0000 7f2365e15640 1 -- 172.21.15.32:0/312727759 wait complete. 2024-04-24T23:01:11.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.712+0000 7f2365e15640 1 Processor -- start 2024-04-24T23:01:11.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.713+0000 7f2365e15640 1 -- start start 2024-04-24T23:01:11.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.713+0000 7f2365e15640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23600ba0e0 0x7f2360152550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:11.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.713+0000 7f2365e15640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23600d4210 0x7f23601577a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:11.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.713+0000 7f2365e15640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f236015e9b0 0x7f23600cb820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:11.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.713+0000 7f2365e15640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f23600d2400 con 0x7f23600ba0e0 2024-04-24T23:01:11.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.713+0000 7f2365e15640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f23600d3cf0 con 0x7f23600d4210 2024-04-24T23:01:11.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.713+0000 7f2365e15640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f236006b650 con 0x7f236015e9b0 2024-04-24T23:01:11.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.713+0000 7f235f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23600ba0e0 0x7f2360152550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:11.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.713+0000 7f235f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23600ba0e0 0x7f2360152550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43652/0 (socket says 172.21.15.32:43652) 2024-04-24T23:01:11.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.713+0000 7f235f7fe640 1 -- 172.21.15.32:0/2835750214 learned_addr learned my addr 172.21.15.32:0/2835750214 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:11.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.714+0000 7f235effd640 1 --2- 172.21.15.32:0/2835750214 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23600d4210 0x7f23601577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:11.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.714+0000 7f235ffff640 1 --2- 172.21.15.32:0/2835750214 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f236015e9b0 0x7f23600cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:11.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.714+0000 7f235f7fe640 1 -- 172.21.15.32:0/2835750214 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f236015e9b0 msgr2=0x7f23600cb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:11.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.714+0000 7f235f7fe640 1 --2- 172.21.15.32:0/2835750214 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f236015e9b0 0x7f23600cb820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:11.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.714+0000 7f235f7fe640 1 -- 172.21.15.32:0/2835750214 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23600d4210 msgr2=0x7f23601577a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:11.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.714+0000 7f235f7fe640 1 --2- 172.21.15.32:0/2835750214 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23600d4210 0x7f23601577a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:11.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.714+0000 7f235f7fe640 1 -- 172.21.15.32:0/2835750214 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2360153020 con 0x7f23600ba0e0 2024-04-24T23:01:11.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.714+0000 7f235f7fe640 1 --2- 172.21.15.32:0/2835750214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23600ba0e0 0x7f2360152550 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f235000c7f0 tx=0x7f235000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:11.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.714+0000 7f2347fff640 1 -- 172.21.15.32:0/2835750214 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2350015070 con 0x7f23600ba0e0 2024-04-24T23:01:11.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.714+0000 7f2347fff640 1 -- 172.21.15.32:0/2835750214 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2350007820 con 0x7f23600ba0e0 2024-04-24T23:01:11.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.714+0000 7f2365e15640 1 -- 172.21.15.32:0/2835750214 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2360156700 con 0x7f23600ba0e0 2024-04-24T23:01:11.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.714+0000 7f2347fff640 1 -- 172.21.15.32:0/2835750214 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2350007bd0 con 0x7f23600ba0e0 2024-04-24T23:01:11.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.714+0000 7f2365e15640 1 -- 172.21.15.32:0/2835750214 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2360156cc0 con 0x7f23600ba0e0 2024-04-24T23:01:11.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.715+0000 7f2365e15640 1 -- 172.21.15.32:0/2835750214 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2360068430 con 0x7f23600ba0e0 2024-04-24T23:01:11.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.716+0000 7f2347fff640 1 -- 172.21.15.32:0/2835750214 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2350018070 con 0x7f23600ba0e0 2024-04-24T23:01:11.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.716+0000 7f2347fff640 1 --2- 172.21.15.32:0/2835750214 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f234c06be30 0x7f234c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:11.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.716+0000 7f2347fff640 1 -- 172.21.15.32:0/2835750214 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5854+0+0 (secure 0 0 0) 0x7f235008afd0 con 0x7f23600ba0e0 2024-04-24T23:01:11.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.716+0000 7f235effd640 1 --2- 172.21.15.32:0/2835750214 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f234c06be30 0x7f234c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:11.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.717+0000 7f235effd640 1 --2- 172.21.15.32:0/2835750214 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f234c06be30 0x7f234c06e2f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2354034000 tx=0x7f235402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:11.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.720+0000 7f2347fff640 1 -- 172.21.15.32:0/2835750214 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2360153020 con 0x7f23600ba0e0 2024-04-24T23:01:11.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.940+0000 7f2365e15640 1 -- 172.21.15.32:0/2835750214 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2360152a90 con 0x7f23600ba0e0 2024-04-24T23:01:11.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.941+0000 7f2347fff640 1 -- 172.21.15.32:0/2835750214 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5187 (secure 0 0 0) 0x7f23500598f0 con 0x7f23600ba0e0 2024-04-24T23:01:11.943 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:01:11.943 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":20,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4994,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/1076304225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1076304225},{"type":"v1","addr":"172.21.15.32:6834","nonce":1076304225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":16},{"gid":5000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1388367206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1388367206},{"type":"v1","addr":"172.21.15.121:6835","nonce":1388367206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":5299,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2065945523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2065945523},{"type":"v1","addr":"172.21.15.121:6837","nonce":2065945523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:00:32.376194+0000","modified":"2024-04-24T23:00:34.381949+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"c","rank":0,"incarnation":18,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/3274703471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":3274703471},{"type":"v1","addr":"172.21.15.32:6835","nonce":3274703471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5631,"qdb_cluster":[5631]},"id":3}]} 2024-04-24T23:01:11.943 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 20 2024-04-24T23:01:11.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.943+0000 7f2365e15640 1 -- 172.21.15.32:0/2835750214 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f234c06be30 msgr2=0x7f234c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:11.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.943+0000 7f2365e15640 1 --2- 172.21.15.32:0/2835750214 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f234c06be30 0x7f234c06e2f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2354034000 tx=0x7f235402e000 comp rx=0 tx=0).stop 2024-04-24T23:01:11.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.943+0000 7f2365e15640 1 -- 172.21.15.32:0/2835750214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23600ba0e0 msgr2=0x7f2360152550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:11.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.943+0000 7f2365e15640 1 --2- 172.21.15.32:0/2835750214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23600ba0e0 0x7f2360152550 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f235000c7f0 tx=0x7f235000ccb0 comp rx=0 tx=0).stop 2024-04-24T23:01:11.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.943+0000 7f2365e15640 1 -- 172.21.15.32:0/2835750214 shutdown_connections 2024-04-24T23:01:11.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.943+0000 7f2365e15640 1 --2- 172.21.15.32:0/2835750214 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f234c06be30 0x7f234c06e2f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:11.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.943+0000 7f2365e15640 1 --2- 172.21.15.32:0/2835750214 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f236015e9b0 0x7f23600cb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:11.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.943+0000 7f2365e15640 1 --2- 172.21.15.32:0/2835750214 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23600d4210 0x7f23601577a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:11.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.943+0000 7f2365e15640 1 --2- 172.21.15.32:0/2835750214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23600ba0e0 0x7f2360152550 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:11.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.943+0000 7f2365e15640 1 -- 172.21.15.32:0/2835750214 >> 172.21.15.32:0/2835750214 conn(0x7f23600d11d0 msgr2=0x7f23600b9390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:11.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.943+0000 7f2365e15640 1 -- 172.21.15.32:0/2835750214 shutdown_connections 2024-04-24T23:01:11.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:11.943+0000 7f2365e15640 1 -- 172.21.15.32:0/2835750214 wait complete. 2024-04-24T23:01:11.955 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:01:12.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.055+0000 7fc9e7638640 1 Processor -- start 2024-04-24T23:01:12.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.056+0000 7fc9e7638640 1 -- start start 2024-04-24T23:01:12.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.056+0000 7fc9e7638640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9e00d4210 0x7fc9e00d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:12.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.056+0000 7fc9e7638640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9e00d3170 con 0x7fc9e015e9b0 2024-04-24T23:01:12.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.056+0000 7fc9e7638640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9e00d3450 con 0x7fc9e006b820 2024-04-24T23:01:12.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.056+0000 7fc9e7638640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc9e00d3730 con 0x7fc9e00d4210 2024-04-24T23:01:12.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.056+0000 7fc9e53d4640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc9e015e9b0 0x7fc9e006b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47764/0 (socket says 172.21.15.32:47764) 2024-04-24T23:01:12.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.056+0000 7fc9e53d4640 1 -- 172.21.15.32:0/1505071786 learned_addr learned my addr 172.21.15.32:0/1505071786 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:01:12.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.057+0000 7fc9dffff640 1 -- 172.21.15.32:0/1505071786 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3109064668 0 0) 0x7fc9e00d3170 con 0x7fc9e015e9b0 2024-04-24T23:01:12.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.057+0000 7fc9dffff640 1 -- 172.21.15.32:0/1505071786 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc9c8003520 con 0x7fc9e015e9b0 2024-04-24T23:01:12.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.057+0000 7fc9e5bd5640 1 --2- 172.21.15.32:0/1505071786 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9e00d4210 0x7fc9e00d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:12.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.057+0000 7fc9dffff640 1 -- 172.21.15.32:0/1505071786 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4144212111 0 0) 0x7fc9c8003520 con 0x7fc9e015e9b0 2024-04-24T23:01:12.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.057+0000 7fc9dffff640 1 -- 172.21.15.32:0/1505071786 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9e00d4210 msgr2=0x7fc9e00d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:12.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.057+0000 7fc9dffff640 1 --2- 172.21.15.32:0/1505071786 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9e00d4210 0x7fc9e00d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:12.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.057+0000 7fc9dffff640 1 -- 172.21.15.32:0/1505071786 >> v1:172.21.15.121:6789/0 conn(0x7fc9e006b820 legacy=0x7fc9e00b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:12.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.057+0000 7fc9dffff640 1 -- 172.21.15.32:0/1505071786 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9e00d3a10 con 0x7fc9e015e9b0 2024-04-24T23:01:12.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.057+0000 7fc9dffff640 1 -- 172.21.15.32:0/1505071786 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7fc9d8001670 con 0x7fc9e015e9b0 2024-04-24T23:01:12.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.057+0000 7fc9dffff640 1 -- 172.21.15.32:0/1505071786 >> v1:172.21.15.32:6789/0 conn(0x7fc9e015e9b0 legacy=0x7fc9e006b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:12.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.057+0000 7fc9dffff640 1 --2- 172.21.15.32:0/1505071786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9c80039e0 0x7fc9c8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:12.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.057+0000 7fc9dffff640 1 --2- 172.21.15.32:0/1505071786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9c8004800 0x7fc9c8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:12.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.057+0000 7fc9dffff640 1 --2- 172.21.15.32:0/1505071786 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9e00d4210 0x7fc9c8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:12.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.057+0000 7fc9dffff640 1 -- 172.21.15.32:0/1505071786 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e00d3730 con 0x7fc9c80039e0 2024-04-24T23:01:12.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.057+0000 7fc9dffff640 1 -- 172.21.15.32:0/1505071786 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c800bb20 con 0x7fc9c8004800 2024-04-24T23:01:12.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.057+0000 7fc9dffff640 1 -- 172.21.15.32:0/1505071786 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc9c800bd20 con 0x7fc9e00d4210 2024-04-24T23:01:12.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.057+0000 7fc9e53d4640 1 --2- 172.21.15.32:0/1505071786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9c80039e0 0x7fc9c8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:12.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.057+0000 7fc9e4bd3640 1 --2- 172.21.15.32:0/1505071786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9c8004800 0x7fc9c8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:12.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.057+0000 7fc9e5bd5640 1 --2- 172.21.15.32:0/1505071786 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9e00d4210 0x7fc9c8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:12.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.057+0000 7fc9e53d4640 1 -- 172.21.15.32:0/1505071786 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9e00d4210 msgr2=0x7fc9c8005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:12.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.057+0000 7fc9e53d4640 1 --2- 172.21.15.32:0/1505071786 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9e00d4210 0x7fc9c8005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:12.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.057+0000 7fc9e53d4640 1 -- 172.21.15.32:0/1505071786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9c8004800 msgr2=0x7fc9c8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:12.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.057+0000 7fc9e53d4640 1 --2- 172.21.15.32:0/1505071786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9c8004800 0x7fc9c8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:12.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.057+0000 7fc9e53d4640 1 -- 172.21.15.32:0/1505071786 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9c8003520 con 0x7fc9c80039e0 2024-04-24T23:01:12.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.058+0000 7fc9e53d4640 1 --2- 172.21.15.32:0/1505071786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9c80039e0 0x7fc9c8003e00 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fc9d800c240 tx=0x7fc9c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd916af275c6755c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:12.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.058+0000 7fc9dffff640 1 -- 172.21.15.32:0/1505071786 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d8007cf0 con 0x7fc9c80039e0 2024-04-24T23:01:12.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.058+0000 7fc9dffff640 1 -- 172.21.15.32:0/1505071786 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9d80058d0 con 0x7fc9c80039e0 2024-04-24T23:01:12.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.058+0000 7fc9dffff640 1 -- 172.21.15.32:0/1505071786 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d8005ca0 con 0x7fc9c80039e0 2024-04-24T23:01:12.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.059+0000 7fc9e7638640 1 -- 172.21.15.32:0/1505071786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9c80039e0 msgr2=0x7fc9c8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:12.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.059+0000 7fc9e7638640 1 --2- 172.21.15.32:0/1505071786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9c80039e0 0x7fc9c8003e00 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fc9d800c240 tx=0x7fc9c8005830 comp rx=0 tx=0).stop 2024-04-24T23:01:12.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.059+0000 7fc9e5bd5640 1 -- 172.21.15.32:0/1505071786 reap_dead start 2024-04-24T23:01:12.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.059+0000 7fc9e7638640 1 -- 172.21.15.32:0/1505071786 shutdown_connections 2024-04-24T23:01:12.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.059+0000 7fc9e7638640 1 -- 172.21.15.32:0/1505071786 >> 172.21.15.32:0/1505071786 conn(0x7fc9e00d11d0 msgr2=0x7fc9e00d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:12.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.059+0000 7fc9e7638640 1 -- 172.21.15.32:0/1505071786 shutdown_connections 2024-04-24T23:01:12.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.059+0000 7fc9e7638640 1 -- 172.21.15.32:0/1505071786 wait complete. 2024-04-24T23:01:12.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.060+0000 7fc9e7638640 1 Processor -- start 2024-04-24T23:01:12.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.060+0000 7fc9e7638640 1 -- start start 2024-04-24T23:01:12.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.061+0000 7fc9e7638640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9c80039e0 0x7fc9e015c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:12.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.061+0000 7fc9e7638640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9e0156520 0x7fc9e01569a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:12.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.061+0000 7fc9e7638640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9e0155390 0x7fc9e01557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:12.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.061+0000 7fc9e7638640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e00ba0e0 con 0x7fc9e0156520 2024-04-24T23:01:12.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.061+0000 7fc9e7638640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e00d2400 con 0x7fc9e0155390 2024-04-24T23:01:12.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.061+0000 7fc9e7638640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc9e00ba260 con 0x7fc9c80039e0 2024-04-24T23:01:12.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.061+0000 7fc9e4bd3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9e0156520 0x7fc9e01569a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:12.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.061+0000 7fc9e4bd3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9e0156520 0x7fc9e01569a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43678/0 (socket says 172.21.15.32:43678) 2024-04-24T23:01:12.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.061+0000 7fc9e4bd3640 1 -- 172.21.15.32:0/544499142 learned_addr learned my addr 172.21.15.32:0/544499142 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:12.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.061+0000 7fc9e53d4640 1 --2- 172.21.15.32:0/544499142 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9c80039e0 0x7fc9e015c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:12.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.061+0000 7fc9e5bd5640 1 --2- 172.21.15.32:0/544499142 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9e0155390 0x7fc9e01557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:12.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.061+0000 7fc9e4bd3640 1 -- 172.21.15.32:0/544499142 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9c80039e0 msgr2=0x7fc9e015c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:12.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.061+0000 7fc9e4bd3640 1 --2- 172.21.15.32:0/544499142 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9c80039e0 0x7fc9e015c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:12.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.061+0000 7fc9e4bd3640 1 -- 172.21.15.32:0/544499142 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9e0155390 msgr2=0x7fc9e01557f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:12.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.061+0000 7fc9e4bd3640 1 --2- 172.21.15.32:0/544499142 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9e0155390 0x7fc9e01557f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:12.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.061+0000 7fc9e4bd3640 1 -- 172.21.15.32:0/544499142 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9e0155d30 con 0x7fc9e0156520 2024-04-24T23:01:12.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.061+0000 7fc9e4bd3640 1 --2- 172.21.15.32:0/544499142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9e0156520 0x7fc9e01569a0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fc9d400f5f0 tx=0x7fc9d400fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:12.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.062+0000 7fc9ddffb640 1 -- 172.21.15.32:0/544499142 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d40052e0 con 0x7fc9e0156520 2024-04-24T23:01:12.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.062+0000 7fc9ddffb640 1 -- 172.21.15.32:0/544499142 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9d4018070 con 0x7fc9e0156520 2024-04-24T23:01:12.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.062+0000 7fc9e7638640 1 -- 172.21.15.32:0/544499142 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9e0156fa0 con 0x7fc9e0156520 2024-04-24T23:01:12.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.062+0000 7fc9ddffb640 1 -- 172.21.15.32:0/544499142 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d4003810 con 0x7fc9e0156520 2024-04-24T23:01:12.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.062+0000 7fc9e7638640 1 -- 172.21.15.32:0/544499142 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9e0157560 con 0x7fc9e0156520 2024-04-24T23:01:12.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.063+0000 7fc9e7638640 1 -- 172.21.15.32:0/544499142 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9e0068430 con 0x7fc9e0156520 2024-04-24T23:01:12.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.063+0000 7fc9ddffb640 1 -- 172.21.15.32:0/544499142 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc9d40039b0 con 0x7fc9e0156520 2024-04-24T23:01:12.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.064+0000 7fc9ddffb640 1 --2- 172.21.15.32:0/544499142 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc9cc06be30 0x7fc9cc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:12.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.064+0000 7fc9ddffb640 1 -- 172.21.15.32:0/544499142 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5854+0+0 (secure 0 0 0) 0x7fc9d408ac80 con 0x7fc9e0156520 2024-04-24T23:01:12.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.064+0000 7fc9e53d4640 1 --2- 172.21.15.32:0/544499142 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc9cc06be30 0x7fc9cc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:12.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.064+0000 7fc9e53d4640 1 --2- 172.21.15.32:0/544499142 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc9cc06be30 0x7fc9cc06e2f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8033000 tx=0x7fc9d8009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:12.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.068+0000 7fc9ddffb640 1 -- 172.21.15.32:0/544499142 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc9e0155d30 con 0x7fc9e0156520 2024-04-24T23:01:12.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.254+0000 7fc9e7638640 1 -- 172.21.15.32:0/544499142 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc9e00cac30 con 0x7fc9e0156520 2024-04-24T23:01:12.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.255+0000 7fc9ddffb640 1 -- 172.21.15.32:0/544499142 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) ==== 74+0+16912 (secure 0 0 0) 0x7fc9d4059660 con 0x7fc9e0156520 2024-04-24T23:01:12.257 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:01:12.257 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":56,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:01:00.283572+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:00:30.665592+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-24T23:00:31.357756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.6","mappings":[{"from":5,"to":7}]},{"pgid":"7.c","mappings":[{"from":5,"to":7}]},{"pgid":"7.12","mappings":[{"from":5,"to":7}]},{"pgid":"7.31","mappings":[{"from":5,"to":7}]},{"pgid":"7.35","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:01:12.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.257+0000 7fc9e7638640 1 -- 172.21.15.32:0/544499142 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc9cc06be30 msgr2=0x7fc9cc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:12.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.257+0000 7fc9e7638640 1 --2- 172.21.15.32:0/544499142 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc9cc06be30 0x7fc9cc06e2f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8033000 tx=0x7fc9d8009030 comp rx=0 tx=0).stop 2024-04-24T23:01:12.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.257+0000 7fc9e7638640 1 -- 172.21.15.32:0/544499142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9e0156520 msgr2=0x7fc9e01569a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:12.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.257+0000 7fc9e7638640 1 --2- 172.21.15.32:0/544499142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9e0156520 0x7fc9e01569a0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fc9d400f5f0 tx=0x7fc9d400fab0 comp rx=0 tx=0).stop 2024-04-24T23:01:12.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.257+0000 7fc9e7638640 1 -- 172.21.15.32:0/544499142 shutdown_connections 2024-04-24T23:01:12.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.257+0000 7fc9e7638640 1 --2- 172.21.15.32:0/544499142 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc9cc06be30 0x7fc9cc06e2f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:12.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.257+0000 7fc9e7638640 1 --2- 172.21.15.32:0/544499142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9e0156520 0x7fc9e01569a0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:12.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.257+0000 7fc9e7638640 1 --2- 172.21.15.32:0/544499142 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9e0155390 0x7fc9e01557f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:12.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.257+0000 7fc9e7638640 1 --2- 172.21.15.32:0/544499142 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9c80039e0 0x7fc9e015c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:12.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.258+0000 7fc9e7638640 1 -- 172.21.15.32:0/544499142 >> 172.21.15.32:0/544499142 conn(0x7fc9e00d11d0 msgr2=0x7fc9e00b91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:12.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.258+0000 7fc9e7638640 1 -- 172.21.15.32:0/544499142 shutdown_connections 2024-04-24T23:01:12.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.258+0000 7fc9e7638640 1 -- 172.21.15.32:0/544499142 wait complete. 2024-04-24T23:01:12.270 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-24T23:01:12.270 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:01:12.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.369+0000 7f99bd8df640 1 Processor -- start 2024-04-24T23:01:12.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.369+0000 7f99bd8df640 1 -- start start 2024-04-24T23:01:12.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.370+0000 7f99bd8df640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99b806c5f0 0x7f99b806c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:12.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.370+0000 7f99bd8df640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f99b80b44d0 0x7f99b80b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:12.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.370+0000 7f99bd8df640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99b80b6430 con 0x7f99b80af890 2024-04-24T23:01:12.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.370+0000 7f99bd8df640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f99b8156f80 con 0x7f99b80b44d0 2024-04-24T23:01:12.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.370+0000 7f99bd8df640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f99b80b6150 con 0x7f99b806c5f0 2024-04-24T23:01:12.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.370+0000 7f99affff640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f99b80b44d0 0x7f99b80b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:12.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.370+0000 7f99b77fe640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f99b80af890 0x7f99b80cef70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:58864/0 (socket says 172.21.15.32:58864) 2024-04-24T23:01:12.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.370+0000 7f99b6ffd640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99b806c5f0 0x7f99b806c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:12.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.370+0000 7f99b77fe640 1 -- 172.21.15.32:0/3510888462 learned_addr learned my addr 172.21.15.32:0/3510888462 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:01:12.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.370+0000 7f99b6ffd640 1 -- 172.21.15.32:0/3510888462 >> v2:172.21.15.121:3300/0 conn(0x7f99b80b44d0 msgr2=0x7f99b80b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:12.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.370+0000 7f99b6ffd640 1 --2- 172.21.15.32:0/3510888462 >> v2:172.21.15.121:3300/0 conn(0x7f99b80b44d0 0x7f99b80b5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:12.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.370+0000 7f99b6ffd640 1 -- 172.21.15.32:0/3510888462 >> v1:172.21.15.121:6789/0 conn(0x7f99b80af890 legacy=0x7f99b80cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:12.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.370+0000 7f99b6ffd640 1 -- 172.21.15.32:0/3510888462 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99b80b39f0 con 0x7f99b806c5f0 2024-04-24T23:01:12.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.370+0000 7f99affff640 1 --2- 172.21.15.32:0/3510888462 >> v2:172.21.15.121:3300/0 conn(0x7f99b80b44d0 0x7f99b80b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:01:12.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.371+0000 7f99b6ffd640 1 --2- 172.21.15.32:0/3510888462 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99b806c5f0 0x7f99b806c9d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f99a80097a0 tx=0x7f99a802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5415cc7275ed0d72 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:12.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.371+0000 7f99b67fc640 1 -- 172.21.15.32:0/3510888462 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a8037070 con 0x7f99b806c5f0 2024-04-24T23:01:12.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.371+0000 7f99b67fc640 1 -- 172.21.15.32:0/3510888462 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99a802ae50 con 0x7f99b806c5f0 2024-04-24T23:01:12.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.371+0000 7f99b67fc640 1 -- 172.21.15.32:0/3510888462 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a8032660 con 0x7f99b806c5f0 2024-04-24T23:01:12.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.372+0000 7f99bd8df640 1 -- 172.21.15.32:0/3510888462 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99b806c5f0 msgr2=0x7f99b806c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:12.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.372+0000 7f99bd8df640 1 --2- 172.21.15.32:0/3510888462 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99b806c5f0 0x7f99b806c9d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f99a80097a0 tx=0x7f99a802a100 comp rx=0 tx=0).stop 2024-04-24T23:01:12.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.372+0000 7f99bd8df640 1 -- 172.21.15.32:0/3510888462 shutdown_connections 2024-04-24T23:01:12.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.372+0000 7f99bd8df640 1 --2- 172.21.15.32:0/3510888462 >> v2:172.21.15.121:3300/0 conn(0x7f99b80b44d0 0x7f99b80b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:12.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.372+0000 7f99bd8df640 1 --2- 172.21.15.32:0/3510888462 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99b806c5f0 0x7f99b806c9d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:12.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.372+0000 7f99bd8df640 1 -- 172.21.15.32:0/3510888462 >> 172.21.15.32:0/3510888462 conn(0x7f99b806b110 msgr2=0x7f99b806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:12.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.372+0000 7f99bd8df640 1 -- 172.21.15.32:0/3510888462 shutdown_connections 2024-04-24T23:01:12.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.372+0000 7f99bd8df640 1 -- 172.21.15.32:0/3510888462 wait complete. 2024-04-24T23:01:12.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.373+0000 7f99bd8df640 1 Processor -- start 2024-04-24T23:01:12.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.373+0000 7f99bd8df640 1 -- start start 2024-04-24T23:01:12.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.374+0000 7f99bd8df640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99b806c5f0 0x7f99b8071ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:12.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.374+0000 7f99bd8df640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99b80af890 0x7f99b8072220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:12.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.374+0000 7f99bd8df640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99b80b44d0 0x7f99b8152e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:12.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.374+0000 7f99bd8df640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f99b806cf10 con 0x7f99b80af890 2024-04-24T23:01:12.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.374+0000 7f99bd8df640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f99b80b3fb0 con 0x7f99b806c5f0 2024-04-24T23:01:12.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.374+0000 7f99bd8df640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f99b80b5dd0 con 0x7f99b80b44d0 2024-04-24T23:01:12.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.374+0000 7f99affff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99b80af890 0x7f99b8072220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:12.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.374+0000 7f99affff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99b80af890 0x7f99b8072220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43684/0 (socket says 172.21.15.32:43684) 2024-04-24T23:01:12.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.374+0000 7f99affff640 1 -- 172.21.15.32:0/2691910922 learned_addr learned my addr 172.21.15.32:0/2691910922 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:12.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.374+0000 7f99b77fe640 1 --2- 172.21.15.32:0/2691910922 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99b80b44d0 0x7f99b8152e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:12.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.374+0000 7f99b6ffd640 1 --2- 172.21.15.32:0/2691910922 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99b806c5f0 0x7f99b8071ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:12.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.374+0000 7f99affff640 1 -- 172.21.15.32:0/2691910922 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99b80b44d0 msgr2=0x7f99b8152e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:12.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.374+0000 7f99affff640 1 --2- 172.21.15.32:0/2691910922 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99b80b44d0 0x7f99b8152e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:12.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.374+0000 7f99affff640 1 -- 172.21.15.32:0/2691910922 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99b806c5f0 msgr2=0x7f99b8071ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:12.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.374+0000 7f99affff640 1 --2- 172.21.15.32:0/2691910922 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99b806c5f0 0x7f99b8071ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:12.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.374+0000 7f99affff640 1 -- 172.21.15.32:0/2691910922 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99b81f2960 con 0x7f99b80af890 2024-04-24T23:01:12.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.374+0000 7f99b6ffd640 1 --2- 172.21.15.32:0/2691910922 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99b806c5f0 0x7f99b8071ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:01:12.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.375+0000 7f99affff640 1 --2- 172.21.15.32:0/2691910922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99b80af890 0x7f99b8072220 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f99a000ebe0 tx=0x7f99a000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:12.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.375+0000 7f99b4ff9640 1 -- 172.21.15.32:0/2691910922 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a0017070 con 0x7f99b80af890 2024-04-24T23:01:12.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.375+0000 7f99bd8df640 1 -- 172.21.15.32:0/2691910922 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99b81f2c40 con 0x7f99b80af890 2024-04-24T23:01:12.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.375+0000 7f99b4ff9640 1 -- 172.21.15.32:0/2691910922 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99a000cc30 con 0x7f99b80af890 2024-04-24T23:01:12.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.375+0000 7f99b4ff9640 1 -- 172.21.15.32:0/2691910922 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a0013470 con 0x7f99b80af890 2024-04-24T23:01:12.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.375+0000 7f99bd8df640 1 -- 172.21.15.32:0/2691910922 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99b81f3200 con 0x7f99b80af890 2024-04-24T23:01:12.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.376+0000 7f99bd8df640 1 -- 172.21.15.32:0/2691910922 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99b8159fa0 con 0x7f99b80af890 2024-04-24T23:01:12.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.376+0000 7f99b4ff9640 1 -- 172.21.15.32:0/2691910922 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f99a001a070 con 0x7f99b80af890 2024-04-24T23:01:12.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.377+0000 7f99b4ff9640 1 --2- 172.21.15.32:0/2691910922 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f999806be30 0x7f999806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:12.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.377+0000 7f99b4ff9640 1 -- 172.21.15.32:0/2691910922 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5854+0+0 (secure 0 0 0) 0x7f99a008af70 con 0x7f99b80af890 2024-04-24T23:01:12.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.377+0000 7f99b6ffd640 1 --2- 172.21.15.32:0/2691910922 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f999806be30 0x7f999806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:12.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.377+0000 7f99b6ffd640 1 --2- 172.21.15.32:0/2691910922 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f999806be30 0x7f999806e2f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f99a8034000 tx=0x7f99a802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:12.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.381+0000 7f99b4ff9640 1 -- 172.21.15.32:0/2691910922 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f99a0059780 con 0x7f99b80af890 2024-04-24T23:01:12.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.600+0000 7f99bd8df640 1 -- 172.21.15.32:0/2691910922 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f99b8068430 con 0x7f99b80af890 2024-04-24T23:01:12.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.600+0000 7f99b4ff9640 1 -- 172.21.15.32:0/2691910922 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5187 (secure 0 0 0) 0x7f99a0059080 con 0x7f99b80af890 2024-04-24T23:01:12.603 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:01:12.603 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":20,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4994,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/1076304225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1076304225},{"type":"v1","addr":"172.21.15.32:6834","nonce":1076304225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":16},{"gid":5000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1388367206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1388367206},{"type":"v1","addr":"172.21.15.121:6835","nonce":1388367206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":5299,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2065945523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2065945523},{"type":"v1","addr":"172.21.15.121:6837","nonce":2065945523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:00:32.376194+0000","modified":"2024-04-24T23:00:34.381949+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"c","rank":0,"incarnation":18,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/3274703471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":3274703471},{"type":"v1","addr":"172.21.15.32:6835","nonce":3274703471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5631,"qdb_cluster":[5631]},"id":3}]} 2024-04-24T23:01:12.603 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 20 2024-04-24T23:01:12.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.602+0000 7f99bd8df640 1 -- 172.21.15.32:0/2691910922 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f999806be30 msgr2=0x7f999806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:12.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.602+0000 7f99bd8df640 1 --2- 172.21.15.32:0/2691910922 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f999806be30 0x7f999806e2f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f99a8034000 tx=0x7f99a802e000 comp rx=0 tx=0).stop 2024-04-24T23:01:12.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.602+0000 7f99bd8df640 1 -- 172.21.15.32:0/2691910922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99b80af890 msgr2=0x7f99b8072220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:12.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.602+0000 7f99bd8df640 1 --2- 172.21.15.32:0/2691910922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99b80af890 0x7f99b8072220 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f99a000ebe0 tx=0x7f99a000c520 comp rx=0 tx=0).stop 2024-04-24T23:01:12.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.602+0000 7f99bd8df640 1 -- 172.21.15.32:0/2691910922 shutdown_connections 2024-04-24T23:01:12.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.602+0000 7f99bd8df640 1 --2- 172.21.15.32:0/2691910922 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f999806be30 0x7f999806e2f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:12.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.602+0000 7f99bd8df640 1 --2- 172.21.15.32:0/2691910922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99b80af890 0x7f99b8072220 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:12.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.602+0000 7f99bd8df640 1 --2- 172.21.15.32:0/2691910922 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99b80b44d0 0x7f99b8152e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:12.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.602+0000 7f99bd8df640 1 --2- 172.21.15.32:0/2691910922 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99b806c5f0 0x7f99b8071ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:12.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.602+0000 7f99bd8df640 1 -- 172.21.15.32:0/2691910922 >> 172.21.15.32:0/2691910922 conn(0x7f99b806b110 msgr2=0x7f99b80b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:12.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.602+0000 7f99bd8df640 1 -- 172.21.15.32:0/2691910922 shutdown_connections 2024-04-24T23:01:12.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.602+0000 7f99bd8df640 1 -- 172.21.15.32:0/2691910922 wait complete. 2024-04-24T23:01:12.616 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:01:12.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.713+0000 7fb75eb70640 1 Processor -- start 2024-04-24T23:01:12.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.713+0000 7fb75eb70640 1 -- start start 2024-04-24T23:01:12.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.714+0000 7fb75eb70640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb75815b4f0 0x7fb75815f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:12.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.714+0000 7fb75eb70640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb7580679b0 0x7fb75815f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:12.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.714+0000 7fb75eb70640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb75806e610 con 0x7fb75815fc10 2024-04-24T23:01:12.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.714+0000 7fb75eb70640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fb75806d850 con 0x7fb7580679b0 2024-04-24T23:01:12.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.714+0000 7fb75eb70640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fb75806db30 con 0x7fb75815b4f0 2024-04-24T23:01:12.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.714+0000 7fb757fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb7580679b0 0x7fb75815f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:12.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.714+0000 7fb75d10d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb75815fc10 0x7fb75815cd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47772/0 (socket says 172.21.15.32:47772) 2024-04-24T23:01:12.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.714+0000 7fb75d10d640 1 -- 172.21.15.32:0/3429263852 learned_addr learned my addr 172.21.15.32:0/3429263852 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:01:12.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.714+0000 7fb75c90c640 1 --2- 172.21.15.32:0/3429263852 >> v2:172.21.15.121:3300/0 conn(0x7fb75815b4f0 0x7fb75815f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:12.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.714+0000 7fb757fff640 1 -- 172.21.15.32:0/3429263852 >> v2:172.21.15.121:3300/0 conn(0x7fb75815b4f0 msgr2=0x7fb75815f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:12.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.714+0000 7fb757fff640 1 --2- 172.21.15.32:0/3429263852 >> v2:172.21.15.121:3300/0 conn(0x7fb75815b4f0 0x7fb75815f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:12.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.714+0000 7fb757fff640 1 -- 172.21.15.32:0/3429263852 >> v1:172.21.15.32:6789/0 conn(0x7fb75815fc10 legacy=0x7fb75815cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:12.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.714+0000 7fb7577fe640 1 -- 172.21.15.32:0/3429263852 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 439618792 0 0) 0x7fb75806e610 con 0x7fb75815fc10 2024-04-24T23:01:12.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.714+0000 7fb757fff640 1 -- 172.21.15.32:0/3429263852 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb75806de10 con 0x7fb7580679b0 2024-04-24T23:01:12.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.714+0000 7fb75c90c640 1 --2- 172.21.15.32:0/3429263852 >> v2:172.21.15.121:3300/0 conn(0x7fb75815b4f0 0x7fb75815f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:01:12.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.715+0000 7fb757fff640 1 --2- 172.21.15.32:0/3429263852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7580679b0 0x7fb75815f6d0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fb7580705f0 tx=0x7fb74c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83d0d38ff6a22a61 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:12.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.715+0000 7fb7577fe640 1 -- 172.21.15.32:0/3429263852 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74c037070 con 0x7fb7580679b0 2024-04-24T23:01:12.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.715+0000 7fb7577fe640 1 -- 172.21.15.32:0/3429263852 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb74c032440 con 0x7fb7580679b0 2024-04-24T23:01:12.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.715+0000 7fb7577fe640 1 -- 172.21.15.32:0/3429263852 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74c032810 con 0x7fb7580679b0 2024-04-24T23:01:12.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.716+0000 7fb75eb70640 1 -- 172.21.15.32:0/3429263852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7580679b0 msgr2=0x7fb75815f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:12.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.716+0000 7fb75eb70640 1 --2- 172.21.15.32:0/3429263852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7580679b0 0x7fb75815f6d0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fb7580705f0 tx=0x7fb74c02a1a0 comp rx=0 tx=0).stop 2024-04-24T23:01:12.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.716+0000 7fb75eb70640 1 -- 172.21.15.32:0/3429263852 shutdown_connections 2024-04-24T23:01:12.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.716+0000 7fb75eb70640 1 --2- 172.21.15.32:0/3429263852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7580679b0 0x7fb75815f6d0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:12.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.716+0000 7fb75eb70640 1 --2- 172.21.15.32:0/3429263852 >> v2:172.21.15.121:3300/0 conn(0x7fb75815b4f0 0x7fb75815f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:12.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.716+0000 7fb75eb70640 1 -- 172.21.15.32:0/3429263852 >> 172.21.15.32:0/3429263852 conn(0x7fb7580d0ae0 msgr2=0x7fb7580d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:12.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.716+0000 7fb75eb70640 1 -- 172.21.15.32:0/3429263852 shutdown_connections 2024-04-24T23:01:12.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.716+0000 7fb75eb70640 1 -- 172.21.15.32:0/3429263852 wait complete. 2024-04-24T23:01:12.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.717+0000 7fb75eb70640 1 Processor -- start 2024-04-24T23:01:12.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.717+0000 7fb75eb70640 1 -- start start 2024-04-24T23:01:12.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.718+0000 7fb75eb70640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb7580679b0 0x7fb7580af420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:12.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.718+0000 7fb75eb70640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb75815b4f0 0x7fb75814e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:12.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.718+0000 7fb75eb70640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb75815fc10 0x7fb758159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:12.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.718+0000 7fb75eb70640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb75815ca60 con 0x7fb75815b4f0 2024-04-24T23:01:12.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.718+0000 7fb75eb70640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb75815c8e0 con 0x7fb75815fc10 2024-04-24T23:01:12.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.718+0000 7fb75eb70640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fb7580d04d0 con 0x7fb7580679b0 2024-04-24T23:01:12.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.718+0000 7fb757fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb75815b4f0 0x7fb75814e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:12.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.718+0000 7fb757fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb75815b4f0 0x7fb75814e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43706/0 (socket says 172.21.15.32:43706) 2024-04-24T23:01:12.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.718+0000 7fb757fff640 1 -- 172.21.15.32:0/3206200583 learned_addr learned my addr 172.21.15.32:0/3206200583 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:12.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.718+0000 7fb75c90c640 1 --2- 172.21.15.32:0/3206200583 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb7580679b0 0x7fb7580af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:12.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.718+0000 7fb75d10d640 1 --2- 172.21.15.32:0/3206200583 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb75815fc10 0x7fb758159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:12.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.718+0000 7fb757fff640 1 -- 172.21.15.32:0/3206200583 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb7580679b0 msgr2=0x7fb7580af420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:12.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.718+0000 7fb757fff640 1 --2- 172.21.15.32:0/3206200583 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb7580679b0 0x7fb7580af420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:12.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.718+0000 7fb757fff640 1 -- 172.21.15.32:0/3206200583 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb75815fc10 msgr2=0x7fb758159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:12.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.718+0000 7fb757fff640 1 --2- 172.21.15.32:0/3206200583 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb75815fc10 0x7fb758159c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:12.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.718+0000 7fb757fff640 1 -- 172.21.15.32:0/3206200583 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb75815a1a0 con 0x7fb75815b4f0 2024-04-24T23:01:12.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.718+0000 7fb75d10d640 1 --2- 172.21.15.32:0/3206200583 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb75815fc10 0x7fb758159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:01:12.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.719+0000 7fb757fff640 1 --2- 172.21.15.32:0/3206200583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb75815b4f0 0x7fb75814e750 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fb74c000c00 tx=0x7fb74c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:12.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.719+0000 7fb7557fa640 1 -- 172.21.15.32:0/3206200583 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74c037050 con 0x7fb75815b4f0 2024-04-24T23:01:12.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.719+0000 7fb7557fa640 1 -- 172.21.15.32:0/3206200583 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb74c038040 con 0x7fb75815b4f0 2024-04-24T23:01:12.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.719+0000 7fb75eb70640 1 -- 172.21.15.32:0/3206200583 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb75815a460 con 0x7fb75815b4f0 2024-04-24T23:01:12.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.719+0000 7fb7557fa640 1 -- 172.21.15.32:0/3206200583 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74c004450 con 0x7fb75815b4f0 2024-04-24T23:01:12.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.719+0000 7fb75eb70640 1 -- 172.21.15.32:0/3206200583 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb75814ee30 con 0x7fb75815b4f0 2024-04-24T23:01:12.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.720+0000 7fb75eb70640 1 -- 172.21.15.32:0/3206200583 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb758068430 con 0x7fb75815b4f0 2024-04-24T23:01:12.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.720+0000 7fb7557fa640 1 -- 172.21.15.32:0/3206200583 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb74c030980 con 0x7fb75815b4f0 2024-04-24T23:01:12.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.721+0000 7fb7557fa640 1 --2- 172.21.15.32:0/3206200583 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb74406be30 0x7fb74406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:12.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.721+0000 7fb7557fa640 1 -- 172.21.15.32:0/3206200583 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5854+0+0 (secure 0 0 0) 0x7fb74c0abac0 con 0x7fb75815b4f0 2024-04-24T23:01:12.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.721+0000 7fb75c90c640 1 --2- 172.21.15.32:0/3206200583 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb74406be30 0x7fb74406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:12.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.722+0000 7fb75c90c640 1 --2- 172.21.15.32:0/3206200583 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb74406be30 0x7fb74406e2f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb75000d620 tx=0x7fb750007810 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:12.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.725+0000 7fb7557fa640 1 -- 172.21.15.32:0/3206200583 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb74c07a590 con 0x7fb75815b4f0 2024-04-24T23:01:12.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.945+0000 7fb75eb70640 1 -- 172.21.15.32:0/3206200583 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb7580d1600 con 0x7fb75815b4f0 2024-04-24T23:01:12.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.946+0000 7fb7557fa640 1 -- 172.21.15.32:0/3206200583 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5187 (secure 0 0 0) 0x7fb74c079e90 con 0x7fb75815b4f0 2024-04-24T23:01:12.949 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:01:12.950 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":20,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4994,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/1076304225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1076304225},{"type":"v1","addr":"172.21.15.32:6834","nonce":1076304225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":16},{"gid":5000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1388367206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1388367206},{"type":"v1","addr":"172.21.15.121:6835","nonce":1388367206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":5299,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2065945523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2065945523},{"type":"v1","addr":"172.21.15.121:6837","nonce":2065945523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:00:32.376194+0000","modified":"2024-04-24T23:00:34.381949+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"c","rank":0,"incarnation":18,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/3274703471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":3274703471},{"type":"v1","addr":"172.21.15.32:6835","nonce":3274703471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5631,"qdb_cluster":[5631]},"id":3}]} 2024-04-24T23:01:12.951 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 20 2024-04-24T23:01:12.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.948+0000 7fb75eb70640 1 -- 172.21.15.32:0/3206200583 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb74406be30 msgr2=0x7fb74406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:12.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.948+0000 7fb75eb70640 1 --2- 172.21.15.32:0/3206200583 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb74406be30 0x7fb74406e2f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb75000d620 tx=0x7fb750007810 comp rx=0 tx=0).stop 2024-04-24T23:01:12.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.948+0000 7fb75eb70640 1 -- 172.21.15.32:0/3206200583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb75815b4f0 msgr2=0x7fb75814e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:12.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.948+0000 7fb75eb70640 1 --2- 172.21.15.32:0/3206200583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb75815b4f0 0x7fb75814e750 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fb74c000c00 tx=0x7fb74c0046c0 comp rx=0 tx=0).stop 2024-04-24T23:01:12.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.948+0000 7fb75eb70640 1 -- 172.21.15.32:0/3206200583 shutdown_connections 2024-04-24T23:01:12.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.948+0000 7fb75eb70640 1 --2- 172.21.15.32:0/3206200583 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb74406be30 0x7fb74406e2f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:12.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.948+0000 7fb75eb70640 1 --2- 172.21.15.32:0/3206200583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb75815b4f0 0x7fb75814e750 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:12.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.948+0000 7fb75eb70640 1 --2- 172.21.15.32:0/3206200583 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb75815fc10 0x7fb758159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:12.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.948+0000 7fb75eb70640 1 --2- 172.21.15.32:0/3206200583 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb7580679b0 0x7fb7580af420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:12.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.948+0000 7fb75eb70640 1 -- 172.21.15.32:0/3206200583 >> 172.21.15.32:0/3206200583 conn(0x7fb7580d0ae0 msgr2=0x7fb7581564a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:12.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.948+0000 7fb75eb70640 1 -- 172.21.15.32:0/3206200583 shutdown_connections 2024-04-24T23:01:12.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:12.948+0000 7fb75eb70640 1 -- 172.21.15.32:0/3206200583 wait complete. 2024-04-24T23:01:12.962 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:01:13.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.062+0000 7f434b645640 1 Processor -- start 2024-04-24T23:01:13.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.062+0000 7f434b645640 1 -- start start 2024-04-24T23:01:13.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.063+0000 7f434b645640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f43440b0b80 0x7f434406c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:13.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.063+0000 7f434b645640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43440b0770 con 0x7f43440ae140 2024-04-24T23:01:13.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.063+0000 7f434b645640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43440b1010 con 0x7f4344074210 2024-04-24T23:01:13.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.063+0000 7f434b645640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f43440b1c20 con 0x7f43440b0b80 2024-04-24T23:01:13.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.063+0000 7f4349be2640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f43440ae140 0x7f43440b20b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47780/0 (socket says 172.21.15.32:47780) 2024-04-24T23:01:13.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.063+0000 7f4349be2640 1 -- 172.21.15.32:0/2893064370 learned_addr learned my addr 172.21.15.32:0/2893064370 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:01:13.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.063+0000 7f4348be0640 1 --2- 172.21.15.32:0/2893064370 >> v2:172.21.15.121:3300/0 conn(0x7f43440b0b80 0x7f434406c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:13.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.063+0000 7f433bfff640 1 -- 172.21.15.32:0/2893064370 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2838138186 0 0) 0x7f43440b0770 con 0x7f43440ae140 2024-04-24T23:01:13.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.063+0000 7f433bfff640 1 -- 172.21.15.32:0/2893064370 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4330003520 con 0x7f43440ae140 2024-04-24T23:01:13.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.064+0000 7f433bfff640 1 -- 172.21.15.32:0/2893064370 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3068300090 0 0) 0x7f43440b1010 con 0x7f4344074210 2024-04-24T23:01:13.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.064+0000 7f433bfff640 1 -- 172.21.15.32:0/2893064370 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f43440b0770 con 0x7f4344074210 2024-04-24T23:01:13.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.064+0000 7f433bfff640 1 -- 172.21.15.32:0/2893064370 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3086227858 0 0) 0x7f4330003520 con 0x7f43440ae140 2024-04-24T23:01:13.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.064+0000 7f4348be0640 1 -- 172.21.15.32:0/2893064370 >> v1:172.21.15.121:6789/0 conn(0x7f4344074210 legacy=0x7f43440745f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:13.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.064+0000 7f4348be0640 1 -- 172.21.15.32:0/2893064370 >> v1:172.21.15.32:6789/0 conn(0x7f43440ae140 legacy=0x7f43440b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:13.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.064+0000 7f4348be0640 1 -- 172.21.15.32:0/2893064370 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43440b12f0 con 0x7f43440b0b80 2024-04-24T23:01:13.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.064+0000 7f4348be0640 1 --2- 172.21.15.32:0/2893064370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43440b0b80 0x7f434406c480 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f43340097a0 tx=0x7f433402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1da9e981059ae25e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:13.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.065+0000 7f433bfff640 1 -- 172.21.15.32:0/2893064370 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4334037070 con 0x7f43440b0b80 2024-04-24T23:01:13.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.065+0000 7f433bfff640 1 -- 172.21.15.32:0/2893064370 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f433402ac50 con 0x7f43440b0b80 2024-04-24T23:01:13.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.065+0000 7f433bfff640 1 -- 172.21.15.32:0/2893064370 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43400019f0 con 0x7f43440b0b80 2024-04-24T23:01:13.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.065+0000 7f434b645640 1 -- 172.21.15.32:0/2893064370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43440b0b80 msgr2=0x7f434406c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:13.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.065+0000 7f434b645640 1 --2- 172.21.15.32:0/2893064370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43440b0b80 0x7f434406c480 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f43340097a0 tx=0x7f433402a1c0 comp rx=0 tx=0).stop 2024-04-24T23:01:13.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.065+0000 7f434b645640 1 -- 172.21.15.32:0/2893064370 shutdown_connections 2024-04-24T23:01:13.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.065+0000 7f434b645640 1 --2- 172.21.15.32:0/2893064370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43440b0b80 0x7f434406c480 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:13.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.065+0000 7f434b645640 1 -- 172.21.15.32:0/2893064370 >> 172.21.15.32:0/2893064370 conn(0x7f43440b2e60 msgr2=0x7f43440b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:13.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.065+0000 7f434b645640 1 -- 172.21.15.32:0/2893064370 shutdown_connections 2024-04-24T23:01:13.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.065+0000 7f434b645640 1 -- 172.21.15.32:0/2893064370 wait complete. 2024-04-24T23:01:13.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.067+0000 7f434b645640 1 Processor -- start 2024-04-24T23:01:13.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.067+0000 7f434b645640 1 -- start start 2024-04-24T23:01:13.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.067+0000 7f434b645640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4344074210 0x7f43441ea220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:13.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.067+0000 7f434b645640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43440ae140 0x7f4344157dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:13.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.067+0000 7f434b645640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f43440d08b0 0x7f43440d0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:13.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.067+0000 7f434b645640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f43440b3ba0 con 0x7f43440ae140 2024-04-24T23:01:13.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.067+0000 7f434b645640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f43440b18b0 con 0x7f4344074210 2024-04-24T23:01:13.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.067+0000 7f434b645640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f43440b3d20 con 0x7f43440d08b0 2024-04-24T23:01:13.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.067+0000 7f4348be0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43440ae140 0x7f4344157dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:13.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.068+0000 7f4348be0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43440ae140 0x7f4344157dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43720/0 (socket says 172.21.15.32:43720) 2024-04-24T23:01:13.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.068+0000 7f4348be0640 1 -- 172.21.15.32:0/719698367 learned_addr learned my addr 172.21.15.32:0/719698367 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:13.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.068+0000 7f4349be2640 1 --2- 172.21.15.32:0/719698367 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f43440d08b0 0x7f43440d0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:13.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.068+0000 7f43493e1640 1 --2- 172.21.15.32:0/719698367 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4344074210 0x7f43441ea220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:13.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.068+0000 7f4348be0640 1 -- 172.21.15.32:0/719698367 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f43440d08b0 msgr2=0x7f43440d0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:13.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.068+0000 7f4348be0640 1 --2- 172.21.15.32:0/719698367 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f43440d08b0 0x7f43440d0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:13.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.068+0000 7f4348be0640 1 -- 172.21.15.32:0/719698367 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4344074210 msgr2=0x7f43441ea220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:13.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.068+0000 7f4348be0640 1 --2- 172.21.15.32:0/719698367 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4344074210 0x7f43441ea220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:13.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.068+0000 7f4348be0640 1 -- 172.21.15.32:0/719698367 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43441e9180 con 0x7f43440ae140 2024-04-24T23:01:13.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.068+0000 7f4348be0640 1 --2- 172.21.15.32:0/719698367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43440ae140 0x7f4344157dc0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f4334035040 tx=0x7f43340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:13.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.068+0000 7f4339ffb640 1 -- 172.21.15.32:0/719698367 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4334041020 con 0x7f43440ae140 2024-04-24T23:01:13.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.068+0000 7f4339ffb640 1 -- 172.21.15.32:0/719698367 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f433403d420 con 0x7f43440ae140 2024-04-24T23:01:13.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.068+0000 7f434b645640 1 -- 172.21.15.32:0/719698367 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43441e9460 con 0x7f43440ae140 2024-04-24T23:01:13.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.068+0000 7f4339ffb640 1 -- 172.21.15.32:0/719698367 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4334037040 con 0x7f43440ae140 2024-04-24T23:01:13.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.068+0000 7f434b645640 1 -- 172.21.15.32:0/719698367 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43441e9a20 con 0x7f43440ae140 2024-04-24T23:01:13.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.069+0000 7f434b645640 1 -- 172.21.15.32:0/719698367 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4344068430 con 0x7f43440ae140 2024-04-24T23:01:13.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.070+0000 7f4339ffb640 1 -- 172.21.15.32:0/719698367 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4334044020 con 0x7f43440ae140 2024-04-24T23:01:13.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.070+0000 7f4339ffb640 1 --2- 172.21.15.32:0/719698367 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f432806be30 0x7f432806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:13.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.070+0000 7f4339ffb640 1 -- 172.21.15.32:0/719698367 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5854+0+0 (secure 0 0 0) 0x7f43340ab270 con 0x7f43440ae140 2024-04-24T23:01:13.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.070+0000 7f43493e1640 1 --2- 172.21.15.32:0/719698367 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f432806be30 0x7f432806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:13.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.071+0000 7f43493e1640 1 --2- 172.21.15.32:0/719698367 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f432806be30 0x7f432806e2f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f433c0011e0 tx=0x7f433c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:13.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.074+0000 7f4339ffb640 1 -- 172.21.15.32:0/719698367 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f433404b030 con 0x7f43440ae140 2024-04-24T23:01:13.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.259+0000 7f434b645640 1 -- 172.21.15.32:0/719698367 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f434406b220 con 0x7f43440ae140 2024-04-24T23:01:13.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.261+0000 7f4339ffb640 1 -- 172.21.15.32:0/719698367 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) ==== 74+0+16912 (secure 0 0 0) 0x7f4334079b80 con 0x7f43440ae140 2024-04-24T23:01:13.263 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:01:13.263 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":56,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:01:00.283572+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:00:30.665592+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-24T23:00:31.357756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.6","mappings":[{"from":5,"to":7}]},{"pgid":"7.c","mappings":[{"from":5,"to":7}]},{"pgid":"7.12","mappings":[{"from":5,"to":7}]},{"pgid":"7.31","mappings":[{"from":5,"to":7}]},{"pgid":"7.35","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:01:13.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.263+0000 7f434b645640 1 -- 172.21.15.32:0/719698367 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f432806be30 msgr2=0x7f432806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:13.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.263+0000 7f434b645640 1 --2- 172.21.15.32:0/719698367 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f432806be30 0x7f432806e2f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f433c0011e0 tx=0x7f433c009040 comp rx=0 tx=0).stop 2024-04-24T23:01:13.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.263+0000 7f434b645640 1 -- 172.21.15.32:0/719698367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43440ae140 msgr2=0x7f4344157dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:13.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.263+0000 7f434b645640 1 --2- 172.21.15.32:0/719698367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43440ae140 0x7f4344157dc0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f4334035040 tx=0x7f43340046c0 comp rx=0 tx=0).stop 2024-04-24T23:01:13.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.263+0000 7f434b645640 1 -- 172.21.15.32:0/719698367 shutdown_connections 2024-04-24T23:01:13.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.263+0000 7f434b645640 1 --2- 172.21.15.32:0/719698367 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f432806be30 0x7f432806e2f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:13.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.263+0000 7f434b645640 1 --2- 172.21.15.32:0/719698367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43440ae140 0x7f4344157dc0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:13.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.263+0000 7f434b645640 1 --2- 172.21.15.32:0/719698367 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f43440d08b0 0x7f43440d0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:13.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.263+0000 7f434b645640 1 --2- 172.21.15.32:0/719698367 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4344074210 0x7f43441ea220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:13.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.263+0000 7f434b645640 1 -- 172.21.15.32:0/719698367 >> 172.21.15.32:0/719698367 conn(0x7f43440b2e60 msgr2=0x7f43440b69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:13.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.263+0000 7f434b645640 1 -- 172.21.15.32:0/719698367 shutdown_connections 2024-04-24T23:01:13.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.263+0000 7f434b645640 1 -- 172.21.15.32:0/719698367 wait complete. 2024-04-24T23:01:13.276 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-24T23:01:13.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.376+0000 7fcb33415640 1 Processor -- start 2024-04-24T23:01:13.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.376+0000 7fcb33415640 1 -- start start 2024-04-24T23:01:13.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.377+0000 7fcb33415640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcb2c06cd00 0x7fcb2c0ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:13.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.377+0000 7fcb33415640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb2c0b4520 con 0x7fcb2c15b380 2024-04-24T23:01:13.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.377+0000 7fcb33415640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb2c0b4800 con 0x7fcb2c15cb00 2024-04-24T23:01:13.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.377+0000 7fcb33415640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fcb2c0b4ae0 con 0x7fcb2c06cd00 2024-04-24T23:01:13.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.377+0000 7fcb311b1640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcb2c15b380 0x7fcb2c06c5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47788/0 (socket says 172.21.15.32:47788) 2024-04-24T23:01:13.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.377+0000 7fcb311b1640 1 -- 172.21.15.32:0/1688641941 learned_addr learned my addr 172.21.15.32:0/1688641941 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:01:13.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.377+0000 7fcb309b0640 1 --2- 172.21.15.32:0/1688641941 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcb2c06cd00 0x7fcb2c0ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:13.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.377+0000 7fcb23fff640 1 -- 172.21.15.32:0/1688641941 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3836863406 0 0) 0x7fcb2c0b4520 con 0x7fcb2c15b380 2024-04-24T23:01:13.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.377+0000 7fcb23fff640 1 -- 172.21.15.32:0/1688641941 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcb18003520 con 0x7fcb2c15b380 2024-04-24T23:01:13.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.377+0000 7fcb23fff640 1 -- 172.21.15.32:0/1688641941 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3092673422 0 0) 0x7fcb18003520 con 0x7fcb2c15b380 2024-04-24T23:01:13.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.377+0000 7fcb23fff640 1 -- 172.21.15.32:0/1688641941 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcb2c06cd00 msgr2=0x7fcb2c0ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:13.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.377+0000 7fcb23fff640 1 --2- 172.21.15.32:0/1688641941 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcb2c06cd00 0x7fcb2c0ce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:13.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.377+0000 7fcb23fff640 1 -- 172.21.15.32:0/1688641941 >> v1:172.21.15.121:6789/0 conn(0x7fcb2c15cb00 legacy=0x7fcb2c0ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:13.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.377+0000 7fcb23fff640 1 -- 172.21.15.32:0/1688641941 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb2c0b4dc0 con 0x7fcb2c15b380 2024-04-24T23:01:13.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.377+0000 7fcb23fff640 1 -- 172.21.15.32:0/1688641941 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7fcb28001670 con 0x7fcb2c15b380 2024-04-24T23:01:13.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.377+0000 7fcb309b0640 1 --2- 172.21.15.32:0/1688641941 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcb2c06cd00 0x7fcb2c0ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:01:13.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.377+0000 7fcb23fff640 1 -- 172.21.15.32:0/1688641941 >> v1:172.21.15.32:6789/0 conn(0x7fcb2c15b380 legacy=0x7fcb2c06c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:13.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.377+0000 7fcb23fff640 1 --2- 172.21.15.32:0/1688641941 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcb2c06cd00 0x7fcb180039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:13.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.377+0000 7fcb23fff640 1 --2- 172.21.15.32:0/1688641941 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcb18003f20 0x7fcb18004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:13.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.378+0000 7fcb23fff640 1 --2- 172.21.15.32:0/1688641941 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb18004d40 0x7fcb180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:13.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.378+0000 7fcb23fff640 1 -- 172.21.15.32:0/1688641941 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2c0b4ae0 con 0x7fcb18004d40 2024-04-24T23:01:13.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.378+0000 7fcb23fff640 1 -- 172.21.15.32:0/1688641941 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1800bad0 con 0x7fcb18003f20 2024-04-24T23:01:13.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.378+0000 7fcb23fff640 1 -- 172.21.15.32:0/1688641941 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fcb1800bcd0 con 0x7fcb2c06cd00 2024-04-24T23:01:13.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.378+0000 7fcb319b2640 1 --2- 172.21.15.32:0/1688641941 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb18004d40 0x7fcb180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:13.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.378+0000 7fcb311b1640 1 --2- 172.21.15.32:0/1688641941 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcb2c06cd00 0x7fcb180039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:13.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.378+0000 7fcb309b0640 1 --2- 172.21.15.32:0/1688641941 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcb18003f20 0x7fcb18004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:13.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.378+0000 7fcb319b2640 1 -- 172.21.15.32:0/1688641941 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcb2c06cd00 msgr2=0x7fcb180039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:13.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.378+0000 7fcb319b2640 1 --2- 172.21.15.32:0/1688641941 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcb2c06cd00 0x7fcb180039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:13.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.378+0000 7fcb319b2640 1 -- 172.21.15.32:0/1688641941 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcb18003f20 msgr2=0x7fcb18004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:13.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.378+0000 7fcb319b2640 1 --2- 172.21.15.32:0/1688641941 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcb18003f20 0x7fcb18004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:13.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.378+0000 7fcb319b2640 1 -- 172.21.15.32:0/1688641941 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb18003520 con 0x7fcb18004d40 2024-04-24T23:01:13.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.379+0000 7fcb319b2640 1 --2- 172.21.15.32:0/1688641941 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb18004d40 0x7fcb180051c0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fcb24001af0 tx=0x7fcb18005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=619ce20cff07ab21 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:13.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.379+0000 7fcb23fff640 1 -- 172.21.15.32:0/1688641941 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb24001b30 con 0x7fcb18004d40 2024-04-24T23:01:13.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.379+0000 7fcb23fff640 1 -- 172.21.15.32:0/1688641941 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb24033620 con 0x7fcb18004d40 2024-04-24T23:01:13.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.379+0000 7fcb23fff640 1 -- 172.21.15.32:0/1688641941 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb24033a10 con 0x7fcb18004d40 2024-04-24T23:01:13.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.379+0000 7fcb33415640 1 -- 172.21.15.32:0/1688641941 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb18004d40 msgr2=0x7fcb180051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:13.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.379+0000 7fcb33415640 1 --2- 172.21.15.32:0/1688641941 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb18004d40 0x7fcb180051c0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fcb24001af0 tx=0x7fcb18005830 comp rx=0 tx=0).stop 2024-04-24T23:01:13.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.379+0000 7fcb319b2640 1 -- 172.21.15.32:0/1688641941 reap_dead start 2024-04-24T23:01:13.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.379+0000 7fcb33415640 1 -- 172.21.15.32:0/1688641941 shutdown_connections 2024-04-24T23:01:13.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.379+0000 7fcb33415640 1 -- 172.21.15.32:0/1688641941 >> 172.21.15.32:0/1688641941 conn(0x7fcb2c06b110 msgr2=0x7fcb2c06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:13.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.379+0000 7fcb33415640 1 -- 172.21.15.32:0/1688641941 shutdown_connections 2024-04-24T23:01:13.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.380+0000 7fcb33415640 1 -- 172.21.15.32:0/1688641941 wait complete. 2024-04-24T23:01:13.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.381+0000 7fcb33415640 1 Processor -- start 2024-04-24T23:01:13.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.381+0000 7fcb33415640 1 -- start start 2024-04-24T23:01:13.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.381+0000 7fcb33415640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcb18004d40 0x7fcb2c14aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:13.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.381+0000 7fcb33415640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcb2c071120 0x7fcb2c0715a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:13.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.381+0000 7fcb33415640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb2c0b33e0 0x7fcb2c0b3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:13.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.381+0000 7fcb33415640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2c0ce430 con 0x7fcb2c0b33e0 2024-04-24T23:01:13.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.381+0000 7fcb33415640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2c15b760 con 0x7fcb2c071120 2024-04-24T23:01:13.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.381+0000 7fcb33415640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fcb2c0ce5b0 con 0x7fcb18004d40 2024-04-24T23:01:13.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.381+0000 7fcb319b2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb2c0b33e0 0x7fcb2c0b3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:13.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.382+0000 7fcb319b2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb2c0b33e0 0x7fcb2c0b3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43738/0 (socket says 172.21.15.32:43738) 2024-04-24T23:01:13.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.382+0000 7fcb319b2640 1 -- 172.21.15.32:0/2481465867 learned_addr learned my addr 172.21.15.32:0/2481465867 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:13.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.382+0000 7fcb309b0640 1 --2- 172.21.15.32:0/2481465867 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcb2c071120 0x7fcb2c0715a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:13.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.382+0000 7fcb311b1640 1 --2- 172.21.15.32:0/2481465867 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcb18004d40 0x7fcb2c14aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:13.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.382+0000 7fcb319b2640 1 -- 172.21.15.32:0/2481465867 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcb18004d40 msgr2=0x7fcb2c14aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:13.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.382+0000 7fcb319b2640 1 --2- 172.21.15.32:0/2481465867 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcb18004d40 0x7fcb2c14aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:13.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.382+0000 7fcb319b2640 1 -- 172.21.15.32:0/2481465867 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcb2c071120 msgr2=0x7fcb2c0715a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:13.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.382+0000 7fcb319b2640 1 --2- 172.21.15.32:0/2481465867 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcb2c071120 0x7fcb2c0715a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:13.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.382+0000 7fcb319b2640 1 -- 172.21.15.32:0/2481465867 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb2c159740 con 0x7fcb2c0b33e0 2024-04-24T23:01:13.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.382+0000 7fcb319b2640 1 --2- 172.21.15.32:0/2481465867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb2c0b33e0 0x7fcb2c0b3860 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fcb24002020 tx=0x7fcb240015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:13.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.382+0000 7fcb21ffb640 1 -- 172.21.15.32:0/2481465867 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb24042070 con 0x7fcb2c0b33e0 2024-04-24T23:01:13.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.382+0000 7fcb21ffb640 1 -- 172.21.15.32:0/2481465867 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb240337d0 con 0x7fcb2c0b33e0 2024-04-24T23:01:13.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.382+0000 7fcb33415640 1 -- 172.21.15.32:0/2481465867 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb2c159a20 con 0x7fcb2c0b33e0 2024-04-24T23:01:13.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.382+0000 7fcb21ffb640 1 -- 172.21.15.32:0/2481465867 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb24001b30 con 0x7fcb2c0b33e0 2024-04-24T23:01:13.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.382+0000 7fcb33415640 1 -- 172.21.15.32:0/2481465867 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb2c159fe0 con 0x7fcb2c0b33e0 2024-04-24T23:01:13.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.383+0000 7fcb33415640 1 -- 172.21.15.32:0/2481465867 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb2c15a1e0 con 0x7fcb2c0b33e0 2024-04-24T23:01:13.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.384+0000 7fcb21ffb640 1 -- 172.21.15.32:0/2481465867 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcb24045070 con 0x7fcb2c0b33e0 2024-04-24T23:01:13.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.384+0000 7fcb21ffb640 1 --2- 172.21.15.32:0/2481465867 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcb1006be60 0x7fcb1006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:13.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.384+0000 7fcb21ffb640 1 -- 172.21.15.32:0/2481465867 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5854+0+0 (secure 0 0 0) 0x7fcb240aada0 con 0x7fcb2c0b33e0 2024-04-24T23:01:13.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.385+0000 7fcb311b1640 1 --2- 172.21.15.32:0/2481465867 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcb1006be60 0x7fcb1006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:13.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.385+0000 7fcb311b1640 1 --2- 172.21.15.32:0/2481465867 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcb1006be60 0x7fcb1006e320 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fcb28007cb0 tx=0x7fcb280078f0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:13.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.389+0000 7fcb21ffb640 1 -- 172.21.15.32:0/2481465867 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcb24078f40 con 0x7fcb2c0b33e0 2024-04-24T23:01:13.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.617+0000 7fcb33415640 1 -- 172.21.15.32:0/2481465867 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fcb2c068430 con 0x7fcb2c0b33e0 2024-04-24T23:01:13.620 INFO:tasks.ceph.mon.a.smithi032.stderr:2024-04-24T23:01:13.618+0000 7f3b809c6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T23:01:13.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.619+0000 7fcb21ffb640 1 -- 172.21.15.32:0/2481465867 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v21) ==== 161+0+0 (secure 0 0 0) 0x7fcb2407d240 con 0x7fcb2c0b33e0 2024-04-24T23:01:13.622 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-24T23:01:13.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.622+0000 7fcb33415640 1 -- 172.21.15.32:0/2481465867 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcb1006be60 msgr2=0x7fcb1006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:13.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.622+0000 7fcb33415640 1 --2- 172.21.15.32:0/2481465867 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcb1006be60 0x7fcb1006e320 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fcb28007cb0 tx=0x7fcb280078f0 comp rx=0 tx=0).stop 2024-04-24T23:01:13.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.622+0000 7fcb33415640 1 -- 172.21.15.32:0/2481465867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb2c0b33e0 msgr2=0x7fcb2c0b3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:13.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.622+0000 7fcb33415640 1 --2- 172.21.15.32:0/2481465867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb2c0b33e0 0x7fcb2c0b3860 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fcb24002020 tx=0x7fcb240015b0 comp rx=0 tx=0).stop 2024-04-24T23:01:13.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.623+0000 7fcb33415640 1 -- 172.21.15.32:0/2481465867 shutdown_connections 2024-04-24T23:01:13.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.623+0000 7fcb33415640 1 --2- 172.21.15.32:0/2481465867 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcb1006be60 0x7fcb1006e320 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:13.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.623+0000 7fcb33415640 1 --2- 172.21.15.32:0/2481465867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb2c0b33e0 0x7fcb2c0b3860 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:13.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.623+0000 7fcb33415640 1 --2- 172.21.15.32:0/2481465867 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcb2c071120 0x7fcb2c0715a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:13.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.623+0000 7fcb33415640 1 --2- 172.21.15.32:0/2481465867 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcb18004d40 0x7fcb2c14aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:13.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.623+0000 7fcb33415640 1 -- 172.21.15.32:0/2481465867 >> 172.21.15.32:0/2481465867 conn(0x7fcb2c06b110 msgr2=0x7fcb2c0cd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:13.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.623+0000 7fcb33415640 1 -- 172.21.15.32:0/2481465867 shutdown_connections 2024-04-24T23:01:13.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.623+0000 7fcb33415640 1 -- 172.21.15.32:0/2481465867 wait complete. 2024-04-24T23:01:13.637 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-24T23:01:13.662 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2024-04-24T23:01:13.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.739+0000 7f3aa82ae640 1 Processor -- start 2024-04-24T23:01:13.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.739+0000 7f3aa82ae640 1 -- start start 2024-04-24T23:01:13.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.739+0000 7f3aa82ae640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3aa015b480 0x7f3aa00b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:13.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.740+0000 7f3aa82ae640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3aa00b46c0 con 0x7f3aa006cd00 2024-04-24T23:01:13.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.740+0000 7f3aa82ae640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3aa00b49a0 con 0x7f3aa015fd60 2024-04-24T23:01:13.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.740+0000 7f3aa82ae640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f3aa00b4ff0 con 0x7f3aa015b480 2024-04-24T23:01:13.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.740+0000 7f3aa684b640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3aa006cd00 0x7f3aa00cdf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47800/0 (socket says 172.21.15.32:47800) 2024-04-24T23:01:13.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.740+0000 7f3aa684b640 1 -- 172.21.15.32:0/3767363307 learned_addr learned my addr 172.21.15.32:0/3767363307 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:01:13.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.740+0000 7f3aa604a640 1 --2- 172.21.15.32:0/3767363307 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3aa015b480 0x7f3aa00b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:13.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.740+0000 7f3aa5048640 1 -- 172.21.15.32:0/3767363307 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 377088741 0 0) 0x7f3aa00b46c0 con 0x7f3aa006cd00 2024-04-24T23:01:13.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.740+0000 7f3aa5048640 1 -- 172.21.15.32:0/3767363307 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3a88003520 con 0x7f3aa006cd00 2024-04-24T23:01:13.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.740+0000 7f3aa5048640 1 -- 172.21.15.32:0/3767363307 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1313455182 0 0) 0x7f3aa00b49a0 con 0x7f3aa015fd60 2024-04-24T23:01:13.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.740+0000 7f3aa5048640 1 -- 172.21.15.32:0/3767363307 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3aa00b46c0 con 0x7f3aa015fd60 2024-04-24T23:01:13.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.740+0000 7f3aa5048640 1 -- 172.21.15.32:0/3767363307 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1200008046 0 0) 0x7f3a88003520 con 0x7f3aa006cd00 2024-04-24T23:01:13.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.740+0000 7f3aa604a640 1 -- 172.21.15.32:0/3767363307 >> v1:172.21.15.121:6789/0 conn(0x7f3aa015fd60 legacy=0x7f3aa006c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:13.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.740+0000 7f3aa604a640 1 -- 172.21.15.32:0/3767363307 >> v1:172.21.15.32:6789/0 conn(0x7f3aa006cd00 legacy=0x7f3aa00cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:13.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.740+0000 7f3aa604a640 1 -- 172.21.15.32:0/3767363307 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aa00b4c80 con 0x7f3aa015b480 2024-04-24T23:01:13.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.741+0000 7f3aa604a640 1 --2- 172.21.15.32:0/3767363307 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3aa015b480 0x7f3aa00b5c60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3a980097d0 tx=0x7f3a9802a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b50f090687f9b64a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:13.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.741+0000 7f3aa5048640 1 -- 172.21.15.32:0/3767363307 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a98037070 con 0x7f3aa015b480 2024-04-24T23:01:13.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.741+0000 7f3aa5048640 1 -- 172.21.15.32:0/3767363307 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a9802ac30 con 0x7f3aa015b480 2024-04-24T23:01:13.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.741+0000 7f3aa5048640 1 -- 172.21.15.32:0/3767363307 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a900019f0 con 0x7f3aa015b480 2024-04-24T23:01:13.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.742+0000 7f3aa82ae640 1 -- 172.21.15.32:0/3767363307 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3aa015b480 msgr2=0x7f3aa00b5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:13.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.742+0000 7f3aa82ae640 1 --2- 172.21.15.32:0/3767363307 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3aa015b480 0x7f3aa00b5c60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3a980097d0 tx=0x7f3a9802a1a0 comp rx=0 tx=0).stop 2024-04-24T23:01:13.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.742+0000 7f3aa82ae640 1 -- 172.21.15.32:0/3767363307 shutdown_connections 2024-04-24T23:01:13.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.742+0000 7f3aa82ae640 1 --2- 172.21.15.32:0/3767363307 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3aa015b480 0x7f3aa00b5c60 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:13.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.742+0000 7f3aa82ae640 1 -- 172.21.15.32:0/3767363307 >> 172.21.15.32:0/3767363307 conn(0x7f3aa006b110 msgr2=0x7f3aa006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:13.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.742+0000 7f3aa82ae640 1 -- 172.21.15.32:0/3767363307 shutdown_connections 2024-04-24T23:01:13.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.742+0000 7f3aa82ae640 1 -- 172.21.15.32:0/3767363307 wait complete. 2024-04-24T23:01:13.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.743+0000 7f3aa82ae640 1 Processor -- start 2024-04-24T23:01:13.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.743+0000 7f3aa82ae640 1 -- start start 2024-04-24T23:01:13.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.744+0000 7f3aa82ae640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3aa006cd00 0x7f3aa0140d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:13.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.744+0000 7f3aa82ae640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3aa015b480 0x7f3aa015dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:13.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.744+0000 7f3aa82ae640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3aa015fd60 0x7f3aa015e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:13.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.744+0000 7f3aa82ae640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa015b860 con 0x7f3aa015fd60 2024-04-24T23:01:13.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.744+0000 7f3aa82ae640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa00b5480 con 0x7f3aa015b480 2024-04-24T23:01:13.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.744+0000 7f3aa82ae640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f3aa01416e0 con 0x7f3aa006cd00 2024-04-24T23:01:13.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.744+0000 7f3aa684b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3aa015fd60 0x7f3aa015e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:13.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.744+0000 7f3aa684b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3aa015fd60 0x7f3aa015e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43756/0 (socket says 172.21.15.32:43756) 2024-04-24T23:01:13.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.744+0000 7f3aa684b640 1 -- 172.21.15.32:0/166217483 learned_addr learned my addr 172.21.15.32:0/166217483 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:13.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.744+0000 7f3aa5849640 1 --2- 172.21.15.32:0/166217483 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3aa015b480 0x7f3aa015dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:13.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.744+0000 7f3aa604a640 1 --2- 172.21.15.32:0/166217483 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3aa006cd00 0x7f3aa0140d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:13.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.744+0000 7f3aa684b640 1 -- 172.21.15.32:0/166217483 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3aa006cd00 msgr2=0x7f3aa0140d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:13.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.744+0000 7f3aa684b640 1 --2- 172.21.15.32:0/166217483 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3aa006cd00 0x7f3aa0140d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:13.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.744+0000 7f3aa684b640 1 -- 172.21.15.32:0/166217483 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3aa015b480 msgr2=0x7f3aa015dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:13.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.744+0000 7f3aa684b640 1 --2- 172.21.15.32:0/166217483 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3aa015b480 0x7f3aa015dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:13.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.744+0000 7f3aa684b640 1 -- 172.21.15.32:0/166217483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aa014efc0 con 0x7f3aa015fd60 2024-04-24T23:01:13.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.745+0000 7f3aa684b640 1 --2- 172.21.15.32:0/166217483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3aa015fd60 0x7f3aa015e280 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f3a9000cc70 tx=0x7f3a9000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:13.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.745+0000 7f3a9effd640 1 -- 172.21.15.32:0/166217483 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a90012400 con 0x7f3aa015fd60 2024-04-24T23:01:13.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.745+0000 7f3a9effd640 1 -- 172.21.15.32:0/166217483 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a900125a0 con 0x7f3aa015fd60 2024-04-24T23:01:13.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.745+0000 7f3aa82ae640 1 -- 172.21.15.32:0/166217483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3aa014f230 con 0x7f3aa015fd60 2024-04-24T23:01:13.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.745+0000 7f3a9effd640 1 -- 172.21.15.32:0/166217483 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a90012990 con 0x7f3aa015fd60 2024-04-24T23:01:13.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.745+0000 7f3aa82ae640 1 -- 172.21.15.32:0/166217483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3aa014f7f0 con 0x7f3aa015fd60 2024-04-24T23:01:13.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.746+0000 7f3aa82ae640 1 -- 172.21.15.32:0/166217483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3aa0068430 con 0x7f3aa015fd60 2024-04-24T23:01:13.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.746+0000 7f3a9effd640 1 -- 172.21.15.32:0/166217483 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3a90012b30 con 0x7f3aa015fd60 2024-04-24T23:01:13.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.747+0000 7f3a9effd640 1 --2- 172.21.15.32:0/166217483 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3a8c06be30 0x7f3a8c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:13.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.747+0000 7f3a9effd640 1 -- 172.21.15.32:0/166217483 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5940+0+0 (secure 0 0 0) 0x7f3a9008b6d0 con 0x7f3aa015fd60 2024-04-24T23:01:13.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.747+0000 7f3aa604a640 1 --2- 172.21.15.32:0/166217483 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3a8c06be30 0x7f3a8c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:13.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.747+0000 7f3aa604a640 1 --2- 172.21.15.32:0/166217483 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3a8c06be30 0x7f3a8c06e2f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3a98034000 tx=0x7f3a9802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:13.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.751+0000 7f3a9effd640 1 -- 172.21.15.32:0/166217483 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3a900182b0 con 0x7f3aa015fd60 2024-04-24T23:01:13.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:13.982+0000 7f3aa82ae640 1 -- 172.21.15.32:0/166217483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f3aa006c000 con 0x7f3aa015fd60 2024-04-24T23:01:14.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.621+0000 7f3a9effd640 1 -- 172.21.15.32:0/166217483 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v22) ==== 104+0+0 (secure 0 0 0) 0x7f3a90059f90 con 0x7f3aa015fd60 2024-04-24T23:01:14.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.624+0000 7f3aa82ae640 1 -- 172.21.15.32:0/166217483 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3a8c06be30 msgr2=0x7f3a8c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:14.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.624+0000 7f3aa82ae640 1 --2- 172.21.15.32:0/166217483 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3a8c06be30 0x7f3a8c06e2f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3a98034000 tx=0x7f3a9802e000 comp rx=0 tx=0).stop 2024-04-24T23:01:14.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.624+0000 7f3aa82ae640 1 -- 172.21.15.32:0/166217483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3aa015fd60 msgr2=0x7f3aa015e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:14.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.624+0000 7f3aa82ae640 1 --2- 172.21.15.32:0/166217483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3aa015fd60 0x7f3aa015e280 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f3a9000cc70 tx=0x7f3a9000a430 comp rx=0 tx=0).stop 2024-04-24T23:01:14.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.624+0000 7f3aa82ae640 1 -- 172.21.15.32:0/166217483 shutdown_connections 2024-04-24T23:01:14.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.624+0000 7f3aa82ae640 1 --2- 172.21.15.32:0/166217483 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3a8c06be30 0x7f3a8c06e2f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:14.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.624+0000 7f3aa82ae640 1 --2- 172.21.15.32:0/166217483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3aa015fd60 0x7f3aa015e280 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:14.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.624+0000 7f3aa82ae640 1 --2- 172.21.15.32:0/166217483 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3aa015b480 0x7f3aa015dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:14.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.624+0000 7f3aa82ae640 1 --2- 172.21.15.32:0/166217483 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3aa006cd00 0x7f3aa0140d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:14.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.624+0000 7f3aa82ae640 1 -- 172.21.15.32:0/166217483 >> 172.21.15.32:0/166217483 conn(0x7f3aa006b110 msgr2=0x7f3aa00b2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:14.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.624+0000 7f3aa82ae640 1 -- 172.21.15.32:0/166217483 shutdown_connections 2024-04-24T23:01:14.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.624+0000 7f3aa82ae640 1 -- 172.21.15.32:0/166217483 wait complete. 2024-04-24T23:01:14.641 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-24T23:01:14.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.796+0000 7f38a0fe7640 1 Processor -- start 2024-04-24T23:01:14.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.796+0000 7f38a0fe7640 1 -- start start 2024-04-24T23:01:14.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.796+0000 7f38a0fe7640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f389c15b960 0x7f389c0cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:14.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.796+0000 7f38a0fe7640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f389c15bde0 con 0x7f389c0b58e0 2024-04-24T23:01:14.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.796+0000 7f38a0fe7640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f389c15c0c0 con 0x7f389c15b580 2024-04-24T23:01:14.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.796+0000 7f38a0fe7640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f389c15c9f0 con 0x7f389c15b960 2024-04-24T23:01:14.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.797+0000 7f389ad9d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f389c0b58e0 0x7f389c15f340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47804/0 (socket says 172.21.15.32:47804) 2024-04-24T23:01:14.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.797+0000 7f389ad9d640 1 -- 172.21.15.32:0/75032267 learned_addr learned my addr 172.21.15.32:0/75032267 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:01:14.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.797+0000 7f3893fff640 1 --2- 172.21.15.32:0/75032267 >> v2:172.21.15.121:3300/0 conn(0x7f389c15b960 0x7f389c0cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:14.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.797+0000 7f3899d9b640 1 -- 172.21.15.32:0/75032267 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3262555186 0 0) 0x7f389c15bde0 con 0x7f389c0b58e0 2024-04-24T23:01:14.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.797+0000 7f3899d9b640 1 -- 172.21.15.32:0/75032267 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3888003520 con 0x7f389c0b58e0 2024-04-24T23:01:14.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.797+0000 7f3899d9b640 1 -- 172.21.15.32:0/75032267 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1158854983 0 0) 0x7f3888003520 con 0x7f389c0b58e0 2024-04-24T23:01:14.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.797+0000 7f3899d9b640 1 -- 172.21.15.32:0/75032267 >> v2:172.21.15.121:3300/0 conn(0x7f389c15b960 msgr2=0x7f389c0cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:14.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.797+0000 7f3899d9b640 1 --2- 172.21.15.32:0/75032267 >> v2:172.21.15.121:3300/0 conn(0x7f389c15b960 0x7f389c0cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:14.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.797+0000 7f3899d9b640 1 -- 172.21.15.32:0/75032267 >> v1:172.21.15.121:6789/0 conn(0x7f389c15b580 legacy=0x7f389c06cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:14.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.797+0000 7f3899d9b640 1 -- 172.21.15.32:0/75032267 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f389c15c3a0 con 0x7f389c0b58e0 2024-04-24T23:01:14.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.797+0000 7f3899d9b640 1 -- 172.21.15.32:0/75032267 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f38940019f0 con 0x7f389c0b58e0 2024-04-24T23:01:14.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.797+0000 7f3899d9b640 1 -- 172.21.15.32:0/75032267 >> v1:172.21.15.32:6789/0 conn(0x7f389c0b58e0 legacy=0x7f389c15f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:14.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.797+0000 7f3899d9b640 1 --2- 172.21.15.32:0/75032267 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f38880039b0 0x7f3888003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:14.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.797+0000 7f3899d9b640 1 --2- 172.21.15.32:0/75032267 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3888004800 0x7f3888004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:14.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.797+0000 7f3899d9b640 1 --2- 172.21.15.32:0/75032267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38880051c0 0x7f388800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:14.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.797+0000 7f3899d9b640 1 -- 172.21.15.32:0/75032267 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f389c15c9f0 con 0x7f38880051c0 2024-04-24T23:01:14.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.797+0000 7f3899d9b640 1 -- 172.21.15.32:0/75032267 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f388800c060 con 0x7f38880039b0 2024-04-24T23:01:14.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.797+0000 7f3899d9b640 1 -- 172.21.15.32:0/75032267 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f388800c260 con 0x7f3888004800 2024-04-24T23:01:14.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.797+0000 7f389a59c640 1 --2- 172.21.15.32:0/75032267 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f38880039b0 0x7f3888003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:14.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.797+0000 7f389ad9d640 1 --2- 172.21.15.32:0/75032267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38880051c0 0x7f388800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:14.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.797+0000 7f3893fff640 1 --2- 172.21.15.32:0/75032267 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3888004800 0x7f3888004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:14.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.798+0000 7f389ad9d640 1 -- 172.21.15.32:0/75032267 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3888004800 msgr2=0x7f3888004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:14.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.798+0000 7f389ad9d640 1 --2- 172.21.15.32:0/75032267 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3888004800 0x7f3888004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:14.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.798+0000 7f389ad9d640 1 -- 172.21.15.32:0/75032267 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f38880039b0 msgr2=0x7f3888003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:14.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.798+0000 7f389ad9d640 1 --2- 172.21.15.32:0/75032267 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f38880039b0 0x7f3888003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:14.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.798+0000 7f389ad9d640 1 -- 172.21.15.32:0/75032267 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3888003520 con 0x7f38880051c0 2024-04-24T23:01:14.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.798+0000 7f389ad9d640 1 -- 172.21.15.32:0/75032267 reap_dead start 2024-04-24T23:01:14.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.798+0000 7f389ad9d640 1 --2- 172.21.15.32:0/75032267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38880051c0 0x7f388800b9c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f389400a840 tx=0x7f3888005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=876e183afc3b8884 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:14.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.799+0000 7f3899d9b640 1 -- 172.21.15.32:0/75032267 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389400a460 con 0x7f38880051c0 2024-04-24T23:01:14.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.799+0000 7f3899d9b640 1 -- 172.21.15.32:0/75032267 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f389402f070 con 0x7f38880051c0 2024-04-24T23:01:14.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.799+0000 7f3899d9b640 1 -- 172.21.15.32:0/75032267 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389402bcb0 con 0x7f38880051c0 2024-04-24T23:01:14.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.799+0000 7f38a0fe7640 1 -- 172.21.15.32:0/75032267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38880051c0 msgr2=0x7f388800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:14.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.799+0000 7f38a0fe7640 1 --2- 172.21.15.32:0/75032267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38880051c0 0x7f388800b9c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f389400a840 tx=0x7f3888005830 comp rx=0 tx=0).stop 2024-04-24T23:01:14.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.799+0000 7f38a0fe7640 1 -- 172.21.15.32:0/75032267 shutdown_connections 2024-04-24T23:01:14.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.799+0000 7f38a0fe7640 1 --2- 172.21.15.32:0/75032267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38880051c0 0x7f388800b9c0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:14.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.799+0000 7f38a0fe7640 1 -- 172.21.15.32:0/75032267 >> 172.21.15.32:0/75032267 conn(0x7f389c0b1f10 msgr2=0x7f389c0b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:14.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.799+0000 7f38a0fe7640 1 -- 172.21.15.32:0/75032267 shutdown_connections 2024-04-24T23:01:14.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.799+0000 7f38a0fe7640 1 -- 172.21.15.32:0/75032267 wait complete. 2024-04-24T23:01:14.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.801+0000 7f38a0fe7640 1 Processor -- start 2024-04-24T23:01:14.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.801+0000 7f38a0fe7640 1 -- start start 2024-04-24T23:01:14.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.801+0000 7f38a0fe7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38880051c0 0x7f389c0b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:14.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.801+0000 7f38a0fe7640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f389c0726f0 0x7f389c072b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:14.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.801+0000 7f38a0fe7640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f389c144a00 0x7f389c144e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:14.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.801+0000 7f38a0fe7640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f389c06c920 con 0x7f38880051c0 2024-04-24T23:01:14.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.801+0000 7f38a0fe7640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f389c06c7a0 con 0x7f389c0726f0 2024-04-24T23:01:14.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.801+0000 7f38a0fe7640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f389c0cde80 con 0x7f389c144a00 2024-04-24T23:01:14.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.801+0000 7f389a59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38880051c0 0x7f389c0b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:14.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.801+0000 7f389a59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38880051c0 0x7f389c0b14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43780/0 (socket says 172.21.15.32:43780) 2024-04-24T23:01:14.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.801+0000 7f389a59c640 1 -- 172.21.15.32:0/1509861929 learned_addr learned my addr 172.21.15.32:0/1509861929 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:14.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.802+0000 7f389a59c640 1 -- 172.21.15.32:0/1509861929 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f389c144a00 msgr2=0x7f389c144e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:14.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.802+0000 7f389a59c640 1 --2- 172.21.15.32:0/1509861929 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f389c144a00 0x7f389c144e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:14.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.802+0000 7f389a59c640 1 -- 172.21.15.32:0/1509861929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f389c0726f0 msgr2=0x7f389c072b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:14.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.802+0000 7f389a59c640 1 --2- 172.21.15.32:0/1509861929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f389c0726f0 0x7f389c072b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:14.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.802+0000 7f389a59c640 1 -- 172.21.15.32:0/1509861929 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f389c073370 con 0x7f38880051c0 2024-04-24T23:01:14.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.802+0000 7f389a59c640 1 --2- 172.21.15.32:0/1509861929 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38880051c0 0x7f389c0b14a0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f388c00f620 tx=0x7f388c00fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:14.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.802+0000 7f3892ffd640 1 -- 172.21.15.32:0/1509861929 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388c0052e0 con 0x7f38880051c0 2024-04-24T23:01:14.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.802+0000 7f3892ffd640 1 -- 172.21.15.32:0/1509861929 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f388c018070 con 0x7f38880051c0 2024-04-24T23:01:14.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.802+0000 7f38a0fe7640 1 -- 172.21.15.32:0/1509861929 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f389c142820 con 0x7f38880051c0 2024-04-24T23:01:14.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.802+0000 7f3892ffd640 1 -- 172.21.15.32:0/1509861929 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388c003810 con 0x7f38880051c0 2024-04-24T23:01:14.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.802+0000 7f38a0fe7640 1 -- 172.21.15.32:0/1509861929 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f389c142de0 con 0x7f38880051c0 2024-04-24T23:01:14.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.803+0000 7f38a0fe7640 1 -- 172.21.15.32:0/1509861929 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f389c068430 con 0x7f38880051c0 2024-04-24T23:01:14.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.804+0000 7f3892ffd640 1 -- 172.21.15.32:0/1509861929 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f388c0039b0 con 0x7f38880051c0 2024-04-24T23:01:14.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.804+0000 7f3892ffd640 1 --2- 172.21.15.32:0/1509861929 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f387c06be30 0x7f387c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:14.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.804+0000 7f3892ffd640 1 -- 172.21.15.32:0/1509861929 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5940+0+0 (secure 0 0 0) 0x7f388c08ade0 con 0x7f38880051c0 2024-04-24T23:01:14.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.804+0000 7f3893fff640 1 --2- 172.21.15.32:0/1509861929 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f387c06be30 0x7f387c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:14.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.805+0000 7f3893fff640 1 --2- 172.21.15.32:0/1509861929 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f387c06be30 0x7f387c06e2f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f389c0b1ec0 tx=0x7f3884009320 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:14.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.809+0000 7f3892ffd640 1 -- 172.21.15.32:0/1509861929 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f388c059610 con 0x7f38880051c0 2024-04-24T23:01:14.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:14.992+0000 7f38a0fe7640 1 -- 172.21.15.32:0/1509861929 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f389c15ff00 con 0x7f38880051c0 2024-04-24T23:01:15.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.626+0000 7f3892ffd640 1 -- 172.21.15.32:0/1509861929 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v58) ==== 181+0+0 (secure 0 0 0) 0x7f388c058f10 con 0x7f38880051c0 2024-04-24T23:01:15.628 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2024-04-24T23:01:15.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.629+0000 7f38a0fe7640 1 -- 172.21.15.32:0/1509861929 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f387c06be30 msgr2=0x7f387c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:15.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.629+0000 7f38a0fe7640 1 --2- 172.21.15.32:0/1509861929 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f387c06be30 0x7f387c06e2f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f389c0b1ec0 tx=0x7f3884009320 comp rx=0 tx=0).stop 2024-04-24T23:01:15.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.629+0000 7f38a0fe7640 1 -- 172.21.15.32:0/1509861929 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38880051c0 msgr2=0x7f389c0b14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:15.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.629+0000 7f38a0fe7640 1 --2- 172.21.15.32:0/1509861929 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38880051c0 0x7f389c0b14a0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f388c00f620 tx=0x7f388c00fae0 comp rx=0 tx=0).stop 2024-04-24T23:01:15.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.629+0000 7f38a0fe7640 1 -- 172.21.15.32:0/1509861929 shutdown_connections 2024-04-24T23:01:15.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.629+0000 7f38a0fe7640 1 --2- 172.21.15.32:0/1509861929 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f387c06be30 0x7f387c06e2f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:15.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.629+0000 7f38a0fe7640 1 --2- 172.21.15.32:0/1509861929 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f389c144a00 0x7f389c144e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:15.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.629+0000 7f38a0fe7640 1 --2- 172.21.15.32:0/1509861929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f389c0726f0 0x7f389c072b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:15.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.629+0000 7f38a0fe7640 1 --2- 172.21.15.32:0/1509861929 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38880051c0 0x7f389c0b14a0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:15.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.629+0000 7f38a0fe7640 1 -- 172.21.15.32:0/1509861929 >> 172.21.15.32:0/1509861929 conn(0x7f389c0b1f10 msgr2=0x7f389c156540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:15.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.629+0000 7f38a0fe7640 1 -- 172.21.15.32:0/1509861929 shutdown_connections 2024-04-24T23:01:15.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.629+0000 7f38a0fe7640 1 -- 172.21.15.32:0/1509861929 wait complete. 2024-04-24T23:01:15.645 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-24T23:01:15.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.755+0000 7f6bd0b90640 1 Processor -- start 2024-04-24T23:01:15.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.755+0000 7f6bd0b90640 1 -- start start 2024-04-24T23:01:15.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.756+0000 7f6bd0b90640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6bcc0b5390 0x7f6bcc0b5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:15.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.756+0000 7f6bd0b90640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6bcc0b5d60 0x7f6bcc0b36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:15.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.756+0000 7f6bd0b90640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6bcc160390 con 0x7f6bcc15c420 2024-04-24T23:01:15.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.756+0000 7f6bd0b90640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f6bcc160140 con 0x7f6bcc0b5d60 2024-04-24T23:01:15.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.756+0000 7f6bd0b90640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f6bcc160fa0 con 0x7f6bcc0b5390 2024-04-24T23:01:15.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.756+0000 7f6bcad9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6bcc0b5d60 0x7f6bcc0b36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:15.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.756+0000 7f6bcad9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6bcc0b5d60 0x7f6bcc0b36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43790/0 (socket says 172.21.15.32:43790) 2024-04-24T23:01:15.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.756+0000 7f6bcad9d640 1 -- 172.21.15.32:0/564324058 learned_addr learned my addr 172.21.15.32:0/564324058 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:15.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.756+0000 7f6bc9d9b640 1 --2- 172.21.15.32:0/564324058 >> v2:172.21.15.121:3300/0 conn(0x7f6bcc0b5390 0x7f6bcc0b5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:15.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.756+0000 7f6bcad9d640 1 -- 172.21.15.32:0/564324058 >> v2:172.21.15.121:3300/0 conn(0x7f6bcc0b5390 msgr2=0x7f6bcc0b5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:15.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.756+0000 7f6bcad9d640 1 --2- 172.21.15.32:0/564324058 >> v2:172.21.15.121:3300/0 conn(0x7f6bcc0b5390 0x7f6bcc0b5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:15.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.756+0000 7f6bcad9d640 1 -- 172.21.15.32:0/564324058 >> v1:172.21.15.121:6789/0 conn(0x7f6bcc15c420 legacy=0x7f6bcc0af700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:15.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.756+0000 7f6bcad9d640 1 -- 172.21.15.32:0/564324058 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bcc160670 con 0x7f6bcc0b5d60 2024-04-24T23:01:15.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.756+0000 7f6bc9d9b640 1 --2- 172.21.15.32:0/564324058 >> v2:172.21.15.121:3300/0 conn(0x7f6bcc0b5390 0x7f6bcc0b5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:01:15.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.757+0000 7f6bcad9d640 1 --2- 172.21.15.32:0/564324058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bcc0b5d60 0x7f6bcc0b36d0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4009e50 tx=0x7f6bc402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=866e70c0dfa16e94 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:15.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.757+0000 7f6bc959a640 1 -- 172.21.15.32:0/564324058 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc4036070 con 0x7f6bcc0b5d60 2024-04-24T23:01:15.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.757+0000 7f6bc959a640 1 -- 172.21.15.32:0/564324058 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bc4031600 con 0x7f6bcc0b5d60 2024-04-24T23:01:15.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.757+0000 7f6bc959a640 1 -- 172.21.15.32:0/564324058 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc402ae60 con 0x7f6bcc0b5d60 2024-04-24T23:01:15.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.758+0000 7f6bd0b90640 1 -- 172.21.15.32:0/564324058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bcc0b5d60 msgr2=0x7f6bcc0b36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:15.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.758+0000 7f6bd0b90640 1 --2- 172.21.15.32:0/564324058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bcc0b5d60 0x7f6bcc0b36d0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4009e50 tx=0x7f6bc402a680 comp rx=0 tx=0).stop 2024-04-24T23:01:15.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.758+0000 7f6bd0b90640 1 -- 172.21.15.32:0/564324058 shutdown_connections 2024-04-24T23:01:15.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.758+0000 7f6bd0b90640 1 --2- 172.21.15.32:0/564324058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bcc0b5d60 0x7f6bcc0b36d0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:15.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.758+0000 7f6bd0b90640 1 --2- 172.21.15.32:0/564324058 >> v2:172.21.15.121:3300/0 conn(0x7f6bcc0b5390 0x7f6bcc0b5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:15.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.758+0000 7f6bd0b90640 1 -- 172.21.15.32:0/564324058 >> 172.21.15.32:0/564324058 conn(0x7f6bcc0b1f10 msgr2=0x7f6bcc0b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:15.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.758+0000 7f6bd0b90640 1 -- 172.21.15.32:0/564324058 shutdown_connections 2024-04-24T23:01:15.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.758+0000 7f6bd0b90640 1 -- 172.21.15.32:0/564324058 wait complete. 2024-04-24T23:01:15.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.759+0000 7f6bd0b90640 1 Processor -- start 2024-04-24T23:01:15.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.759+0000 7f6bd0b90640 1 -- start start 2024-04-24T23:01:15.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.760+0000 7f6bd0b90640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6bcc0b5390 0x7f6bcc1e6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:15.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.760+0000 7f6bd0b90640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6bcc0b5d60 0x7f6bcc155320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:15.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.760+0000 7f6bd0b90640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bcc15c420 0x7f6bcc15aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:15.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.760+0000 7f6bd0b90640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6bcc0717b0 con 0x7f6bcc15c420 2024-04-24T23:01:15.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.760+0000 7f6bd0b90640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6bcc160c30 con 0x7f6bcc0b5390 2024-04-24T23:01:15.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.760+0000 7f6bd0b90640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f6bcc071930 con 0x7f6bcc0b5d60 2024-04-24T23:01:15.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.760+0000 7f6bcad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bcc15c420 0x7f6bcc15aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:15.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.760+0000 7f6bcad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bcc15c420 0x7f6bcc15aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43798/0 (socket says 172.21.15.32:43798) 2024-04-24T23:01:15.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.760+0000 7f6bcad9d640 1 -- 172.21.15.32:0/1861977254 learned_addr learned my addr 172.21.15.32:0/1861977254 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:15.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.760+0000 7f6bcad9d640 1 -- 172.21.15.32:0/1861977254 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6bcc0b5d60 msgr2=0x7f6bcc155320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:15.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.760+0000 7f6bcad9d640 1 --2- 172.21.15.32:0/1861977254 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6bcc0b5d60 0x7f6bcc155320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:15.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.760+0000 7f6bcad9d640 1 -- 172.21.15.32:0/1861977254 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6bcc0b5390 msgr2=0x7f6bcc1e6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:15.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.760+0000 7f6bcad9d640 1 --2- 172.21.15.32:0/1861977254 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6bcc0b5390 0x7f6bcc1e6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:15.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.760+0000 7f6bcad9d640 1 -- 172.21.15.32:0/1861977254 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bcc1e70d0 con 0x7f6bcc15c420 2024-04-24T23:01:15.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.760+0000 7f6bcad9d640 1 --2- 172.21.15.32:0/1861977254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bcc15c420 0x7f6bcc15aef0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4009e50 tx=0x7f6bc4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:15.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.761+0000 7f6bbaffd640 1 -- 172.21.15.32:0/1861977254 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc4036050 con 0x7f6bcc15c420 2024-04-24T23:01:15.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.761+0000 7f6bbaffd640 1 -- 172.21.15.32:0/1861977254 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bc4042070 con 0x7f6bcc15c420 2024-04-24T23:01:15.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.761+0000 7f6bd0b90640 1 -- 172.21.15.32:0/1861977254 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bcc1e73b0 con 0x7f6bcc15c420 2024-04-24T23:01:15.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.761+0000 7f6bbaffd640 1 -- 172.21.15.32:0/1861977254 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc4009510 con 0x7f6bcc15c420 2024-04-24T23:01:15.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.761+0000 7f6bd0b90640 1 -- 172.21.15.32:0/1861977254 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bcc1e7970 con 0x7f6bcc15c420 2024-04-24T23:01:15.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.762+0000 7f6bd0b90640 1 -- 172.21.15.32:0/1861977254 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bcc0b5790 con 0x7f6bcc15c420 2024-04-24T23:01:15.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.762+0000 7f6bbaffd640 1 -- 172.21.15.32:0/1861977254 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6bc4007bf0 con 0x7f6bcc15c420 2024-04-24T23:01:15.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.763+0000 7f6bbaffd640 1 --2- 172.21.15.32:0/1861977254 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6bac06be30 0x7f6bac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:15.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.763+0000 7f6bbaffd640 1 -- 172.21.15.32:0/1861977254 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5502+0+0 (secure 0 0 0) 0x7f6bc40abb20 con 0x7f6bcc15c420 2024-04-24T23:01:15.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.763+0000 7f6bca59c640 1 --2- 172.21.15.32:0/1861977254 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6bac06be30 0x7f6bac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:15.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.763+0000 7f6bca59c640 1 --2- 172.21.15.32:0/1861977254 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6bac06be30 0x7f6bac06e2f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc006fd0 tx=0x7f6bbc006bc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:15.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.767+0000 7f6bbaffd640 1 -- 172.21.15.32:0/1861977254 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6bcc0b5790 con 0x7f6bcc15c420 2024-04-24T23:01:15.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:15.951+0000 7f6bd0b90640 1 -- 172.21.15.32:0/1861977254 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6bcc068430 con 0x7f6bcc15c420 2024-04-24T23:01:16.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.627+0000 7f6bbaffd640 1 -- 172.21.15.32:0/1861977254 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v59) ==== 169+0+0 (secure 0 0 0) 0x7f6bc407a5a0 con 0x7f6bcc15c420 2024-04-24T23:01:16.630 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2024-04-24T23:01:16.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.631+0000 7f6bd0b90640 1 -- 172.21.15.32:0/1861977254 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6bac06be30 msgr2=0x7f6bac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:16.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.631+0000 7f6bd0b90640 1 --2- 172.21.15.32:0/1861977254 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6bac06be30 0x7f6bac06e2f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc006fd0 tx=0x7f6bbc006bc0 comp rx=0 tx=0).stop 2024-04-24T23:01:16.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.631+0000 7f6bd0b90640 1 -- 172.21.15.32:0/1861977254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bcc15c420 msgr2=0x7f6bcc15aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:16.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.631+0000 7f6bd0b90640 1 --2- 172.21.15.32:0/1861977254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bcc15c420 0x7f6bcc15aef0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4009e50 tx=0x7f6bc4004a40 comp rx=0 tx=0).stop 2024-04-24T23:01:16.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.631+0000 7f6bd0b90640 1 -- 172.21.15.32:0/1861977254 shutdown_connections 2024-04-24T23:01:16.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.631+0000 7f6bd0b90640 1 --2- 172.21.15.32:0/1861977254 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6bac06be30 0x7f6bac06e2f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:16.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.631+0000 7f6bd0b90640 1 --2- 172.21.15.32:0/1861977254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bcc15c420 0x7f6bcc15aef0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:16.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.631+0000 7f6bd0b90640 1 --2- 172.21.15.32:0/1861977254 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6bcc0b5d60 0x7f6bcc155320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:16.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.631+0000 7f6bd0b90640 1 --2- 172.21.15.32:0/1861977254 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6bcc0b5390 0x7f6bcc1e6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:16.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.631+0000 7f6bd0b90640 1 -- 172.21.15.32:0/1861977254 >> 172.21.15.32:0/1861977254 conn(0x7f6bcc0b1f10 msgr2=0x7f6bcc0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:16.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.631+0000 7f6bd0b90640 1 -- 172.21.15.32:0/1861977254 shutdown_connections 2024-04-24T23:01:16.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.631+0000 7f6bd0b90640 1 -- 172.21.15.32:0/1861977254 wait complete. 2024-04-24T23:01:16.648 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-24T23:01:16.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.755+0000 7f2c78514640 1 Processor -- start 2024-04-24T23:01:16.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.755+0000 7f2c78514640 1 -- start start 2024-04-24T23:01:16.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.756+0000 7f2c78514640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c70072cf0 0x7f2c7006c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:16.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.756+0000 7f2c78514640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2c70071770 0x7f2c7006cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:16.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.756+0000 7f2c78514640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c7015c990 con 0x7f2c7015c5b0 2024-04-24T23:01:16.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.756+0000 7f2c78514640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f2c700b4740 con 0x7f2c70071770 2024-04-24T23:01:16.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.756+0000 7f2c78514640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f2c70072860 con 0x7f2c70072cf0 2024-04-24T23:01:16.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.756+0000 7f2c76ab1640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2c70071770 0x7f2c7006cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:16.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.756+0000 7f2c762b0640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2c7015c5b0 0x7f2c700b4030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47806/0 (socket says 172.21.15.32:47806) 2024-04-24T23:01:16.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.756+0000 7f2c762b0640 1 -- 172.21.15.32:0/3366968543 learned_addr learned my addr 172.21.15.32:0/3366968543 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:01:16.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.756+0000 7f2c75aaf640 1 --2- 172.21.15.32:0/3366968543 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c70072cf0 0x7f2c7006c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:16.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.756+0000 7f2c76ab1640 1 -- 172.21.15.32:0/3366968543 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c70072cf0 msgr2=0x7f2c7006c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:16.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.756+0000 7f2c76ab1640 1 --2- 172.21.15.32:0/3366968543 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c70072cf0 0x7f2c7006c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:16.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.756+0000 7f2c76ab1640 1 -- 172.21.15.32:0/3366968543 >> v1:172.21.15.32:6789/0 conn(0x7f2c7015c5b0 legacy=0x7f2c700b4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:16.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.756+0000 7f2c76ab1640 1 -- 172.21.15.32:0/3366968543 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c70071c50 con 0x7f2c70071770 2024-04-24T23:01:16.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.757+0000 7f2c76ab1640 1 --2- 172.21.15.32:0/3366968543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c70071770 0x7f2c7006cb30 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f2c60009d90 tx=0x7f2c6002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a6664fc07488ec3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:16.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.757+0000 7f2c752ae640 1 -- 172.21.15.32:0/3366968543 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c60031600 con 0x7f2c70071770 2024-04-24T23:01:16.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.757+0000 7f2c752ae640 1 -- 172.21.15.32:0/3366968543 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c600317a0 con 0x7f2c70071770 2024-04-24T23:01:16.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.757+0000 7f2c752ae640 1 -- 172.21.15.32:0/3366968543 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c60031b70 con 0x7f2c70071770 2024-04-24T23:01:16.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.757+0000 7f2c78514640 1 -- 172.21.15.32:0/3366968543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c70071770 msgr2=0x7f2c7006cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:16.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.757+0000 7f2c78514640 1 --2- 172.21.15.32:0/3366968543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c70071770 0x7f2c7006cb30 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f2c60009d90 tx=0x7f2c6002a5d0 comp rx=0 tx=0).stop 2024-04-24T23:01:16.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.757+0000 7f2c78514640 1 -- 172.21.15.32:0/3366968543 shutdown_connections 2024-04-24T23:01:16.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.757+0000 7f2c78514640 1 --2- 172.21.15.32:0/3366968543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c70071770 0x7f2c7006cb30 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:16.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.757+0000 7f2c78514640 1 --2- 172.21.15.32:0/3366968543 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c70072cf0 0x7f2c7006c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:16.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.757+0000 7f2c78514640 1 -- 172.21.15.32:0/3366968543 >> 172.21.15.32:0/3366968543 conn(0x7f2c7006b110 msgr2=0x7f2c7006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:16.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.757+0000 7f2c78514640 1 -- 172.21.15.32:0/3366968543 shutdown_connections 2024-04-24T23:01:16.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.757+0000 7f2c78514640 1 -- 172.21.15.32:0/3366968543 wait complete. 2024-04-24T23:01:16.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.759+0000 7f2c78514640 1 Processor -- start 2024-04-24T23:01:16.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.759+0000 7f2c78514640 1 -- start start 2024-04-24T23:01:16.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.759+0000 7f2c78514640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c70071770 0x7f2c700cd740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:16.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.759+0000 7f2c78514640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c70072cf0 0x7f2c70146ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:16.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.759+0000 7f2c78514640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c7015c5b0 0x7f2c700ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:16.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.759+0000 7f2c78514640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2c700724f0 con 0x7f2c70072cf0 2024-04-24T23:01:16.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.759+0000 7f2c78514640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2c70072210 con 0x7f2c7015c5b0 2024-04-24T23:01:16.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.759+0000 7f2c78514640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f2c700b4b40 con 0x7f2c70071770 2024-04-24T23:01:16.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.759+0000 7f2c75aaf640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c70072cf0 0x7f2c70146ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:16.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.759+0000 7f2c75aaf640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c70072cf0 0x7f2c70146ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43828/0 (socket says 172.21.15.32:43828) 2024-04-24T23:01:16.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.759+0000 7f2c75aaf640 1 -- 172.21.15.32:0/1426726791 learned_addr learned my addr 172.21.15.32:0/1426726791 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:16.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.759+0000 7f2c762b0640 1 --2- 172.21.15.32:0/1426726791 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c70071770 0x7f2c700cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:16.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.759+0000 7f2c76ab1640 1 --2- 172.21.15.32:0/1426726791 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c7015c5b0 0x7f2c700ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:16.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.759+0000 7f2c75aaf640 1 -- 172.21.15.32:0/1426726791 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c70071770 msgr2=0x7f2c700cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:16.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.759+0000 7f2c75aaf640 1 --2- 172.21.15.32:0/1426726791 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c70071770 0x7f2c700cd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:16.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.759+0000 7f2c75aaf640 1 -- 172.21.15.32:0/1426726791 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c7015c5b0 msgr2=0x7f2c700ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:16.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.759+0000 7f2c75aaf640 1 --2- 172.21.15.32:0/1426726791 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c7015c5b0 0x7f2c700ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:16.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.759+0000 7f2c75aaf640 1 -- 172.21.15.32:0/1426726791 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c7014ecf0 con 0x7f2c70072cf0 2024-04-24T23:01:16.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.759+0000 7f2c762b0640 1 --2- 172.21.15.32:0/1426726791 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c70071770 0x7f2c700cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:01:16.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.761+0000 7f2c75aaf640 1 --2- 172.21.15.32:0/1426726791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c70072cf0 0x7f2c70146ac0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f2c6400d800 tx=0x7f2c6400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:16.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.761+0000 7f2c6effd640 1 -- 172.21.15.32:0/1426726791 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c6400b6c0 con 0x7f2c70072cf0 2024-04-24T23:01:16.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.761+0000 7f2c6effd640 1 -- 172.21.15.32:0/1426726791 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c64019070 con 0x7f2c70072cf0 2024-04-24T23:01:16.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.761+0000 7f2c6effd640 1 -- 172.21.15.32:0/1426726791 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c6400ba80 con 0x7f2c70072cf0 2024-04-24T23:01:16.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.761+0000 7f2c78514640 1 -- 172.21.15.32:0/1426726791 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c7014efd0 con 0x7f2c70072cf0 2024-04-24T23:01:16.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.761+0000 7f2c78514640 1 -- 172.21.15.32:0/1426726791 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c7014f590 con 0x7f2c70072cf0 2024-04-24T23:01:16.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.762+0000 7f2c78514640 1 -- 172.21.15.32:0/1426726791 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c70068430 con 0x7f2c70072cf0 2024-04-24T23:01:16.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.762+0000 7f2c6effd640 1 -- 172.21.15.32:0/1426726791 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2c6400bc20 con 0x7f2c70072cf0 2024-04-24T23:01:16.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.763+0000 7f2c6effd640 1 --2- 172.21.15.32:0/1426726791 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2c5c06be30 0x7f2c5c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:16.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.763+0000 7f2c6effd640 1 -- 172.21.15.32:0/1426726791 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 4943+0+0 (secure 0 0 0) 0x7f2c6408b030 con 0x7f2c70072cf0 2024-04-24T23:01:16.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.763+0000 7f2c762b0640 1 --2- 172.21.15.32:0/1426726791 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2c5c06be30 0x7f2c5c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:16.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.764+0000 7f2c762b0640 1 --2- 172.21.15.32:0/1426726791 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2c5c06be30 0x7f2c5c06e2f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2c68000f60 tx=0x7f2c68009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:16.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.767+0000 7f2c6effd640 1 -- 172.21.15.32:0/1426726791 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2c64059ce0 con 0x7f2c70072cf0 2024-04-24T23:01:16.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.948+0000 7f2c78514640 1 -- 172.21.15.32:0/1426726791 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f2c700b5e70 con 0x7f2c70072cf0 2024-04-24T23:01:16.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.952+0000 7f2c6effd640 1 -- 172.21.15.32:0/1426726791 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v5) ==== 104+0+0 (secure 0 0 0) 0x7f2c640595e0 con 0x7f2c70072cf0 2024-04-24T23:01:16.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.954+0000 7f2c78514640 1 -- 172.21.15.32:0/1426726791 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2c5c06be30 msgr2=0x7f2c5c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:16.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.954+0000 7f2c78514640 1 --2- 172.21.15.32:0/1426726791 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2c5c06be30 0x7f2c5c06e2f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2c68000f60 tx=0x7f2c68009040 comp rx=0 tx=0).stop 2024-04-24T23:01:16.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.954+0000 7f2c78514640 1 -- 172.21.15.32:0/1426726791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c70072cf0 msgr2=0x7f2c70146ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:16.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.954+0000 7f2c78514640 1 --2- 172.21.15.32:0/1426726791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c70072cf0 0x7f2c70146ac0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f2c6400d800 tx=0x7f2c6400dcc0 comp rx=0 tx=0).stop 2024-04-24T23:01:16.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.954+0000 7f2c78514640 1 -- 172.21.15.32:0/1426726791 shutdown_connections 2024-04-24T23:01:16.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.954+0000 7f2c78514640 1 --2- 172.21.15.32:0/1426726791 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2c5c06be30 0x7f2c5c06e2f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:16.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.954+0000 7f2c78514640 1 --2- 172.21.15.32:0/1426726791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c70072cf0 0x7f2c70146ac0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:16.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.954+0000 7f2c78514640 1 --2- 172.21.15.32:0/1426726791 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c7015c5b0 0x7f2c700ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:16.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.954+0000 7f2c78514640 1 --2- 172.21.15.32:0/1426726791 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c70071770 0x7f2c700cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:16.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.954+0000 7f2c78514640 1 -- 172.21.15.32:0/1426726791 >> 172.21.15.32:0/1426726791 conn(0x7f2c7006b110 msgr2=0x7f2c700b53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:16.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.954+0000 7f2c78514640 1 -- 172.21.15.32:0/1426726791 shutdown_connections 2024-04-24T23:01:16.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:16.954+0000 7f2c78514640 1 -- 172.21.15.32:0/1426726791 wait complete. 2024-04-24T23:01:16.966 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_different_file' 2024-04-24T23:01:17.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.066+0000 7f52acd48640 1 Processor -- start 2024-04-24T23:01:17.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.066+0000 7f52acd48640 1 -- start start 2024-04-24T23:01:17.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.067+0000 7f52acd48640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f52a8073390 0x7f52a806d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:17.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.067+0000 7f52acd48640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52a815ccf0 con 0x7f52a806d400 2024-04-24T23:01:17.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.067+0000 7f52acd48640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52a815cfb0 con 0x7f52a8156bb0 2024-04-24T23:01:17.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.067+0000 7f52acd48640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f52a815d290 con 0x7f52a8073390 2024-04-24T23:01:17.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.067+0000 7f52a6d9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f52a8073390 0x7f52a806d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:17.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.067+0000 7f52a5d9b640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f52a806d400 0x7f52a8072c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47818/0 (socket says 172.21.15.32:47818) 2024-04-24T23:01:17.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.067+0000 7f52a5d9b640 1 -- 172.21.15.32:0/2622759249 learned_addr learned my addr 172.21.15.32:0/2622759249 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:01:17.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.067+0000 7f52a6d9d640 1 -- 172.21.15.32:0/2622759249 >> v1:172.21.15.121:6789/0 conn(0x7f52a8156bb0 legacy=0x7f52a815dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:17.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.067+0000 7f52a6d9d640 1 -- 172.21.15.32:0/2622759249 >> v1:172.21.15.32:6789/0 conn(0x7f52a806d400 legacy=0x7f52a8072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:17.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.067+0000 7f52a6d9d640 1 -- 172.21.15.32:0/2622759249 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52a815d570 con 0x7f52a8073390 2024-04-24T23:01:17.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.067+0000 7f52a559a640 1 -- 172.21.15.32:0/2622759249 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 349523325 0 0) 0x7f52a815ccf0 con 0x7f52a806d400 2024-04-24T23:01:17.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.068+0000 7f52a6d9d640 1 --2- 172.21.15.32:0/2622759249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52a8073390 0x7f52a806d8b0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f52a00097a0 tx=0x7f52a002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=370e92ebdf65f7fe server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:17.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.068+0000 7f52a559a640 1 -- 172.21.15.32:0/2622759249 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a0037070 con 0x7f52a8073390 2024-04-24T23:01:17.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.068+0000 7f52a559a640 1 -- 172.21.15.32:0/2622759249 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52a0032440 con 0x7f52a8073390 2024-04-24T23:01:17.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.068+0000 7f52a559a640 1 -- 172.21.15.32:0/2622759249 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a0032810 con 0x7f52a8073390 2024-04-24T23:01:17.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.068+0000 7f52acd48640 1 -- 172.21.15.32:0/2622759249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52a8073390 msgr2=0x7f52a806d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:17.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.068+0000 7f52acd48640 1 --2- 172.21.15.32:0/2622759249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52a8073390 0x7f52a806d8b0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f52a00097a0 tx=0x7f52a002a1a0 comp rx=0 tx=0).stop 2024-04-24T23:01:17.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.068+0000 7f52acd48640 1 -- 172.21.15.32:0/2622759249 shutdown_connections 2024-04-24T23:01:17.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.068+0000 7f52acd48640 1 --2- 172.21.15.32:0/2622759249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52a8073390 0x7f52a806d8b0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:17.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.068+0000 7f52acd48640 1 -- 172.21.15.32:0/2622759249 >> 172.21.15.32:0/2622759249 conn(0x7f52a806e800 msgr2=0x7f52a806ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:17.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.069+0000 7f52acd48640 1 -- 172.21.15.32:0/2622759249 shutdown_connections 2024-04-24T23:01:17.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.069+0000 7f52acd48640 1 -- 172.21.15.32:0/2622759249 wait complete. 2024-04-24T23:01:17.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.070+0000 7f52acd48640 1 Processor -- start 2024-04-24T23:01:17.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.070+0000 7f52acd48640 1 -- start start 2024-04-24T23:01:17.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.071+0000 7f52acd48640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f52a806d400 0x7f52a8145a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:17.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.071+0000 7f52acd48640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52a8073390 0x7f52a8145f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:17.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.071+0000 7f52acd48640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52a8156bb0 0x7f52a806c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:17.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.071+0000 7f52acd48640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f52a8157110 con 0x7f52a8156bb0 2024-04-24T23:01:17.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.071+0000 7f52acd48640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f52a8156f90 con 0x7f52a8073390 2024-04-24T23:01:17.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.071+0000 7f52acd48640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f52a80b5840 con 0x7f52a806d400 2024-04-24T23:01:17.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.071+0000 7f52a6d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52a8156bb0 0x7f52a806c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:17.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.071+0000 7f52a6d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52a8156bb0 0x7f52a806c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43838/0 (socket says 172.21.15.32:43838) 2024-04-24T23:01:17.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.071+0000 7f52a6d9d640 1 -- 172.21.15.32:0/3870138926 learned_addr learned my addr 172.21.15.32:0/3870138926 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:17.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.071+0000 7f52a659c640 1 --2- 172.21.15.32:0/3870138926 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f52a806d400 0x7f52a8145a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:17.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.071+0000 7f52a5d9b640 1 --2- 172.21.15.32:0/3870138926 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52a8073390 0x7f52a8145f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:17.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.071+0000 7f52a6d9d640 1 -- 172.21.15.32:0/3870138926 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f52a806d400 msgr2=0x7f52a8145a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:17.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.071+0000 7f52a6d9d640 1 --2- 172.21.15.32:0/3870138926 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f52a806d400 0x7f52a8145a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:17.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.071+0000 7f52a6d9d640 1 -- 172.21.15.32:0/3870138926 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52a8073390 msgr2=0x7f52a8145f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:17.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.071+0000 7f52a6d9d640 1 --2- 172.21.15.32:0/3870138926 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52a8073390 0x7f52a8145f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:17.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.071+0000 7f52a6d9d640 1 -- 172.21.15.32:0/3870138926 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52a806ce90 con 0x7f52a8156bb0 2024-04-24T23:01:17.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.071+0000 7f52a5d9b640 1 --2- 172.21.15.32:0/3870138926 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52a8073390 0x7f52a8145f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:01:17.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.071+0000 7f52a6d9d640 1 --2- 172.21.15.32:0/3870138926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52a8156bb0 0x7f52a806c1d0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f52a00097a0 tx=0x7f52a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:17.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.071+0000 7f529effd640 1 -- 172.21.15.32:0/3870138926 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a0037040 con 0x7f52a8156bb0 2024-04-24T23:01:17.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.071+0000 7f529effd640 1 -- 172.21.15.32:0/3870138926 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52a0042070 con 0x7f52a8156bb0 2024-04-24T23:01:17.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.071+0000 7f52acd48640 1 -- 172.21.15.32:0/3870138926 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52a814cde0 con 0x7f52a8156bb0 2024-04-24T23:01:17.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.071+0000 7f529effd640 1 -- 172.21.15.32:0/3870138926 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a0030b30 con 0x7f52a8156bb0 2024-04-24T23:01:17.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.071+0000 7f52acd48640 1 -- 172.21.15.32:0/3870138926 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52a814d3a0 con 0x7f52a8156bb0 2024-04-24T23:01:17.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.072+0000 7f52acd48640 1 -- 172.21.15.32:0/3870138926 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52a8068430 con 0x7f52a8156bb0 2024-04-24T23:01:17.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.073+0000 7f529effd640 1 -- 172.21.15.32:0/3870138926 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f52a0030cd0 con 0x7f52a8156bb0 2024-04-24T23:01:17.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.073+0000 7f529effd640 1 --2- 172.21.15.32:0/3870138926 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f528806be30 0x7f528806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:17.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.073+0000 7f529effd640 1 -- 172.21.15.32:0/3870138926 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 4943+0+0 (secure 0 0 0) 0x7f52a00ab690 con 0x7f52a8156bb0 2024-04-24T23:01:17.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.074+0000 7f52a659c640 1 --2- 172.21.15.32:0/3870138926 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f528806be30 0x7f528806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:17.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.074+0000 7f52a659c640 1 --2- 172.21.15.32:0/3870138926 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f528806be30 0x7f528806e2f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f52a814fde0 tx=0x7f5298009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:17.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.078+0000 7f529effd640 1 -- 172.21.15.32:0/3870138926 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f52a0079d80 con 0x7f52a8156bb0 2024-04-24T23:01:17.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.338+0000 7f52acd48640 1 -- 172.21.15.32:0/3870138926 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]} v 0) -- 0x7f52a815c4a0 con 0x7f52a8156bb0 2024-04-24T23:01:17.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.635+0000 7f529effd640 1 -- 172.21.15.32:0/3870138926 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]}]=0 v140) ==== 142+0+0 (secure 0 0 0) 0x7f52a007e080 con 0x7f52a8156bb0 2024-04-24T23:01:17.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.637+0000 7f52acd48640 1 -- 172.21.15.32:0/3870138926 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f528806be30 msgr2=0x7f528806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:17.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.637+0000 7f52acd48640 1 --2- 172.21.15.32:0/3870138926 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f528806be30 0x7f528806e2f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f52a814fde0 tx=0x7f5298009040 comp rx=0 tx=0).stop 2024-04-24T23:01:17.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.637+0000 7f52acd48640 1 -- 172.21.15.32:0/3870138926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52a8156bb0 msgr2=0x7f52a806c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:17.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.637+0000 7f52acd48640 1 --2- 172.21.15.32:0/3870138926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52a8156bb0 0x7f52a806c1d0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f52a00097a0 tx=0x7f52a00046c0 comp rx=0 tx=0).stop 2024-04-24T23:01:17.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.637+0000 7f52acd48640 1 -- 172.21.15.32:0/3870138926 shutdown_connections 2024-04-24T23:01:17.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.637+0000 7f52acd48640 1 --2- 172.21.15.32:0/3870138926 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f528806be30 0x7f528806e2f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:17.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.637+0000 7f52acd48640 1 --2- 172.21.15.32:0/3870138926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52a8156bb0 0x7f52a806c1d0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:17.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.637+0000 7f52acd48640 1 --2- 172.21.15.32:0/3870138926 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52a8073390 0x7f52a8145f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:17.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.637+0000 7f52acd48640 1 --2- 172.21.15.32:0/3870138926 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f52a806d400 0x7f52a8145a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:17.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.637+0000 7f52acd48640 1 -- 172.21.15.32:0/3870138926 >> 172.21.15.32:0/3870138926 conn(0x7f52a806e800 msgr2=0x7f52a80d2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:17.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.637+0000 7f52acd48640 1 -- 172.21.15.32:0/3870138926 shutdown_connections 2024-04-24T23:01:17.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.637+0000 7f52acd48640 1 -- 172.21.15.32:0/3870138926 wait complete. 2024-04-24T23:01:17.651 INFO:tasks.cephfs_test_runner:test_full_different_file (tasks.cephfs.test_full.TestClusterFull) ... ok 2024-04-24T23:01:17.651 INFO:tasks.cephfs_test_runner:Starting test: test_full_fclose (tasks.cephfs.test_full.TestClusterFull) 2024-04-24T23:01:17.652 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:01:17.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.752+0000 7fa66615a640 1 Processor -- start 2024-04-24T23:01:17.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.752+0000 7fa66615a640 1 -- start start 2024-04-24T23:01:17.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.753+0000 7fa66615a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa66015b380 0x7fa66006c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:17.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.753+0000 7fa66615a640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6600cef50 con 0x7fa6600ceb40 2024-04-24T23:01:17.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.753+0000 7fa66615a640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6600b4520 con 0x7fa66015cb00 2024-04-24T23:01:17.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.753+0000 7fa66615a640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fa6600b4800 con 0x7fa66015b380 2024-04-24T23:01:17.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.753+0000 7fa65f7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa66015b380 0x7fa66006c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:17.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.753+0000 7fa65ffff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa6600ceb40 0x7fa6600b55c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47824/0 (socket says 172.21.15.32:47824) 2024-04-24T23:01:17.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.753+0000 7fa65ffff640 1 -- 172.21.15.32:0/3345545773 learned_addr learned my addr 172.21.15.32:0/3345545773 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:01:17.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.753+0000 7fa65f7fe640 1 -- 172.21.15.32:0/3345545773 >> v1:172.21.15.121:6789/0 conn(0x7fa66015cb00 legacy=0x7fa6600ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:17.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.754+0000 7fa65f7fe640 1 -- 172.21.15.32:0/3345545773 >> v1:172.21.15.32:6789/0 conn(0x7fa6600ceb40 legacy=0x7fa6600b55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:17.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.754+0000 7fa65e7fc640 1 -- 172.21.15.32:0/3345545773 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3082081990 0 0) 0x7fa658001490 con 0x7fa6600ceb40 2024-04-24T23:01:17.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.754+0000 7fa65f7fe640 1 -- 172.21.15.32:0/3345545773 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6600b4ae0 con 0x7fa66015b380 2024-04-24T23:01:17.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.754+0000 7fa65f7fe640 1 --2- 172.21.15.32:0/3345545773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa66015b380 0x7fa66006c5f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fa6500097a0 tx=0x7fa65002a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8222cbe96c92438 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:17.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.754+0000 7fa65e7fc640 1 -- 172.21.15.32:0/3345545773 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa650037070 con 0x7fa66015b380 2024-04-24T23:01:17.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.754+0000 7fa65e7fc640 1 -- 172.21.15.32:0/3345545773 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa65002adc0 con 0x7fa66015b380 2024-04-24T23:01:17.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.754+0000 7fa65e7fc640 1 -- 172.21.15.32:0/3345545773 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa650032660 con 0x7fa66015b380 2024-04-24T23:01:17.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.755+0000 7fa66615a640 1 -- 172.21.15.32:0/3345545773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa66015b380 msgr2=0x7fa66006c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:17.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.755+0000 7fa66615a640 1 --2- 172.21.15.32:0/3345545773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa66015b380 0x7fa66006c5f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fa6500097a0 tx=0x7fa65002a0b0 comp rx=0 tx=0).stop 2024-04-24T23:01:17.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.755+0000 7fa66615a640 1 -- 172.21.15.32:0/3345545773 shutdown_connections 2024-04-24T23:01:17.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.755+0000 7fa66615a640 1 --2- 172.21.15.32:0/3345545773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa66015b380 0x7fa66006c5f0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:17.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.755+0000 7fa66615a640 1 -- 172.21.15.32:0/3345545773 >> 172.21.15.32:0/3345545773 conn(0x7fa66006b110 msgr2=0x7fa66006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:17.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.755+0000 7fa66615a640 1 -- 172.21.15.32:0/3345545773 shutdown_connections 2024-04-24T23:01:17.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.755+0000 7fa66615a640 1 -- 172.21.15.32:0/3345545773 wait complete. 2024-04-24T23:01:17.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.756+0000 7fa66615a640 1 Processor -- start 2024-04-24T23:01:17.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.756+0000 7fa66615a640 1 -- start start 2024-04-24T23:01:17.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.757+0000 7fa66615a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6600ceb40 0x7fa660155ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:17.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.757+0000 7fa66615a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa66015b380 0x7fa660156180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:17.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.757+0000 7fa66615a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa66015cb00 0x7fa660150e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:17.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.757+0000 7fa66615a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa66015b760 con 0x7fa66015cb00 2024-04-24T23:01:17.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.757+0000 7fa66615a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa6600b50a0 con 0x7fa6600ceb40 2024-04-24T23:01:17.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.757+0000 7fa66615a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa66006cb30 con 0x7fa66015b380 2024-04-24T23:01:17.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.757+0000 7fa65ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa66015cb00 0x7fa660150e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:17.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.757+0000 7fa65ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa66015cb00 0x7fa660150e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43858/0 (socket says 172.21.15.32:43858) 2024-04-24T23:01:17.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.757+0000 7fa65ffff640 1 -- 172.21.15.32:0/1510701514 learned_addr learned my addr 172.21.15.32:0/1510701514 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:17.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.757+0000 7fa65f7fe640 1 --2- 172.21.15.32:0/1510701514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6600ceb40 0x7fa660155ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:17.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.757+0000 7fa65effd640 1 --2- 172.21.15.32:0/1510701514 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa66015b380 0x7fa660156180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:17.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.757+0000 7fa65ffff640 1 -- 172.21.15.32:0/1510701514 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa66015b380 msgr2=0x7fa660156180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:17.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.757+0000 7fa65ffff640 1 --2- 172.21.15.32:0/1510701514 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa66015b380 0x7fa660156180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:17.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.757+0000 7fa65ffff640 1 -- 172.21.15.32:0/1510701514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6600ceb40 msgr2=0x7fa660155ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:17.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.757+0000 7fa65ffff640 1 --2- 172.21.15.32:0/1510701514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6600ceb40 0x7fa660155ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:17.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.757+0000 7fa65ffff640 1 -- 172.21.15.32:0/1510701514 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6601529e0 con 0x7fa66015cb00 2024-04-24T23:01:17.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.757+0000 7fa65f7fe640 1 --2- 172.21.15.32:0/1510701514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6600ceb40 0x7fa660155ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:01:17.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.757+0000 7fa65effd640 1 --2- 172.21.15.32:0/1510701514 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa66015b380 0x7fa660156180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:01:17.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.758+0000 7fa65ffff640 1 --2- 172.21.15.32:0/1510701514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa66015cb00 0x7fa660150e90 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fa65800c7f0 tx=0x7fa65800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:17.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.758+0000 7fa647fff640 1 -- 172.21.15.32:0/1510701514 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa658018070 con 0x7fa66015cb00 2024-04-24T23:01:17.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.758+0000 7fa647fff640 1 -- 172.21.15.32:0/1510701514 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa65800a5e0 con 0x7fa66015cb00 2024-04-24T23:01:17.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.758+0000 7fa66615a640 1 -- 172.21.15.32:0/1510701514 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6601568f0 con 0x7fa66015cb00 2024-04-24T23:01:17.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.758+0000 7fa647fff640 1 -- 172.21.15.32:0/1510701514 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65800a9d0 con 0x7fa66015cb00 2024-04-24T23:01:17.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.758+0000 7fa66615a640 1 -- 172.21.15.32:0/1510701514 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6601513d0 con 0x7fa66015cb00 2024-04-24T23:01:17.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.759+0000 7fa66615a640 1 -- 172.21.15.32:0/1510701514 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa660068430 con 0x7fa66015cb00 2024-04-24T23:01:17.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.759+0000 7fa647fff640 1 -- 172.21.15.32:0/1510701514 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa65801b070 con 0x7fa66015cb00 2024-04-24T23:01:17.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.760+0000 7fa647fff640 1 --2- 172.21.15.32:0/1510701514 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa64c06be30 0x7fa64c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:17.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.760+0000 7fa647fff640 1 -- 172.21.15.32:0/1510701514 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 4943+0+0 (secure 0 0 0) 0x7fa65808a110 con 0x7fa66015cb00 2024-04-24T23:01:17.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.760+0000 7fa65f7fe640 1 --2- 172.21.15.32:0/1510701514 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa64c06be30 0x7fa64c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:17.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.761+0000 7fa65f7fe640 1 --2- 172.21.15.32:0/1510701514 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa64c06be30 0x7fa64c06e2f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa650034000 tx=0x7fa65002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:17.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.764+0000 7fa647fff640 1 -- 172.21.15.32:0/1510701514 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa6600cef50 con 0x7fa66015cb00 2024-04-24T23:01:17.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.949+0000 7fa66615a640 1 -- 172.21.15.32:0/1510701514 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa6600afd60 con 0x7fa66015cb00 2024-04-24T23:01:17.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.950+0000 7fa647fff640 1 -- 172.21.15.32:0/1510701514 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+13032 (secure 0 0 0) 0x7fa658058dc0 con 0x7fa66015cb00 2024-04-24T23:01:17.951 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:01:17.951 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":59,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:01:16.627936+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6835/3274703471":"2024-04-25T23:01:13.619892+0000","172.21.15.32:6833/3274703471":"2024-04-25T23:01:13.619892+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:01:17.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.951+0000 7fa66615a640 1 -- 172.21.15.32:0/1510701514 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa64c06be30 msgr2=0x7fa64c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:17.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.951+0000 7fa66615a640 1 --2- 172.21.15.32:0/1510701514 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa64c06be30 0x7fa64c06e2f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa650034000 tx=0x7fa65002e000 comp rx=0 tx=0).stop 2024-04-24T23:01:17.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.951+0000 7fa66615a640 1 -- 172.21.15.32:0/1510701514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa66015cb00 msgr2=0x7fa660150e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:17.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.951+0000 7fa66615a640 1 --2- 172.21.15.32:0/1510701514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa66015cb00 0x7fa660150e90 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fa65800c7f0 tx=0x7fa65800ccb0 comp rx=0 tx=0).stop 2024-04-24T23:01:17.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.952+0000 7fa66615a640 1 -- 172.21.15.32:0/1510701514 shutdown_connections 2024-04-24T23:01:17.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.952+0000 7fa66615a640 1 --2- 172.21.15.32:0/1510701514 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa64c06be30 0x7fa64c06e2f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:17.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.952+0000 7fa66615a640 1 --2- 172.21.15.32:0/1510701514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa66015cb00 0x7fa660150e90 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:17.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.952+0000 7fa66615a640 1 --2- 172.21.15.32:0/1510701514 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa66015b380 0x7fa660156180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:17.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.952+0000 7fa66615a640 1 --2- 172.21.15.32:0/1510701514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6600ceb40 0x7fa660155ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:17.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.952+0000 7fa66615a640 1 -- 172.21.15.32:0/1510701514 >> 172.21.15.32:0/1510701514 conn(0x7fa66006b110 msgr2=0x7fa66015d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:17.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.952+0000 7fa66615a640 1 -- 172.21.15.32:0/1510701514 shutdown_connections 2024-04-24T23:01:17.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:17.952+0000 7fa66615a640 1 -- 172.21.15.32:0/1510701514 wait complete. 2024-04-24T23:01:17.964 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:01:17.965 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:01:18.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.068+0000 7f1fb0134640 1 Processor -- start 2024-04-24T23:01:18.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.068+0000 7f1fb0134640 1 -- start start 2024-04-24T23:01:18.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.069+0000 7f1fb0134640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1fa8158dd0 0x7f1fa806b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:18.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.069+0000 7f1fb0134640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1fa806dc60 0x7f1fa80b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:18.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.069+0000 7f1fb0134640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fa806bd10 con 0x7f1fa806b860 2024-04-24T23:01:18.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.069+0000 7f1fb0134640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f1fa815ef10 con 0x7f1fa806dc60 2024-04-24T23:01:18.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.069+0000 7f1fb0134640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f1fa815f1f0 con 0x7f1fa8158dd0 2024-04-24T23:01:18.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.069+0000 7f1fae6d1640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1fa806dc60 0x7f1fa80b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:18.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.069+0000 7f1fae6d1640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1fa806dc60 0x7f1fa80b5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43862/0 (socket says 172.21.15.32:43862) 2024-04-24T23:01:18.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.069+0000 7f1fae6d1640 1 -- 172.21.15.32:0/3587375635 learned_addr learned my addr 172.21.15.32:0/3587375635 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:18.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.069+0000 7f1faded0640 1 --2- 172.21.15.32:0/3587375635 >> v2:172.21.15.121:3300/0 conn(0x7f1fa8158dd0 0x7f1fa806b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:18.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.069+0000 7f1fae6d1640 1 -- 172.21.15.32:0/3587375635 >> v2:172.21.15.121:3300/0 conn(0x7f1fa8158dd0 msgr2=0x7f1fa806b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:18.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.069+0000 7f1fae6d1640 1 --2- 172.21.15.32:0/3587375635 >> v2:172.21.15.121:3300/0 conn(0x7f1fa8158dd0 0x7f1fa806b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:18.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.069+0000 7f1fae6d1640 1 -- 172.21.15.32:0/3587375635 >> v1:172.21.15.121:6789/0 conn(0x7f1fa806b860 legacy=0x7f1fa806d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:18.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.069+0000 7f1fae6d1640 1 -- 172.21.15.32:0/3587375635 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fa815f4d0 con 0x7f1fa806dc60 2024-04-24T23:01:18.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.069+0000 7f1faded0640 1 --2- 172.21.15.32:0/3587375635 >> v2:172.21.15.121:3300/0 conn(0x7f1fa8158dd0 0x7f1fa806b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:01:18.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.070+0000 7f1fae6d1640 1 --2- 172.21.15.32:0/3587375635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fa806dc60 0x7f1fa80b5d00 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f1fa806f3a0 tx=0x7f1f9802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33e471cf6646cd02 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:18.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.070+0000 7f1facece640 1 -- 172.21.15.32:0/3587375635 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f98037070 con 0x7f1fa806dc60 2024-04-24T23:01:18.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.070+0000 7f1facece640 1 -- 172.21.15.32:0/3587375635 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f98032440 con 0x7f1fa806dc60 2024-04-24T23:01:18.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.070+0000 7f1facece640 1 -- 172.21.15.32:0/3587375635 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f98032810 con 0x7f1fa806dc60 2024-04-24T23:01:18.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.070+0000 7f1fb0134640 1 -- 172.21.15.32:0/3587375635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fa806dc60 msgr2=0x7f1fa80b5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:18.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.070+0000 7f1fb0134640 1 --2- 172.21.15.32:0/3587375635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fa806dc60 0x7f1fa80b5d00 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f1fa806f3a0 tx=0x7f1f9802a230 comp rx=0 tx=0).stop 2024-04-24T23:01:18.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.070+0000 7f1fb0134640 1 -- 172.21.15.32:0/3587375635 shutdown_connections 2024-04-24T23:01:18.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.071+0000 7f1fb0134640 1 --2- 172.21.15.32:0/3587375635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fa806dc60 0x7f1fa80b5d00 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:18.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.071+0000 7f1fb0134640 1 --2- 172.21.15.32:0/3587375635 >> v2:172.21.15.121:3300/0 conn(0x7f1fa8158dd0 0x7f1fa806b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:18.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.071+0000 7f1fb0134640 1 -- 172.21.15.32:0/3587375635 >> 172.21.15.32:0/3587375635 conn(0x7f1fa806e800 msgr2=0x7f1fa806ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:18.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.071+0000 7f1fb0134640 1 -- 172.21.15.32:0/3587375635 shutdown_connections 2024-04-24T23:01:18.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.071+0000 7f1fb0134640 1 -- 172.21.15.32:0/3587375635 wait complete. 2024-04-24T23:01:18.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.072+0000 7f1fb0134640 1 Processor -- start 2024-04-24T23:01:18.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.072+0000 7f1fb0134640 1 -- start start 2024-04-24T23:01:18.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.072+0000 7f1fb0134640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fa806b860 0x7f1fa80b5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:18.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.072+0000 7f1fb0134640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1fa806dc60 0x7f1fa8159fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:18.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.072+0000 7f1fb0134640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fa8158dd0 0x7f1fa80b3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:18.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.072+0000 7f1fb0134640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa806bfe0 con 0x7f1fa8158dd0 2024-04-24T23:01:18.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.072+0000 7f1fb0134640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa815fcd0 con 0x7f1fa806b860 2024-04-24T23:01:18.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.072+0000 7f1fb0134640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f1fa81591b0 con 0x7f1fa806dc60 2024-04-24T23:01:18.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.073+0000 7f1fae6d1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fa8158dd0 0x7f1fa80b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:18.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.073+0000 7f1fae6d1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fa8158dd0 0x7f1fa80b3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43878/0 (socket says 172.21.15.32:43878) 2024-04-24T23:01:18.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.073+0000 7f1fae6d1640 1 -- 172.21.15.32:0/492954155 learned_addr learned my addr 172.21.15.32:0/492954155 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:18.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.073+0000 7f1faded0640 1 --2- 172.21.15.32:0/492954155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fa806b860 0x7f1fa80b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:18.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.073+0000 7f1fad6cf640 1 --2- 172.21.15.32:0/492954155 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1fa806dc60 0x7f1fa8159fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:18.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.073+0000 7f1fae6d1640 1 -- 172.21.15.32:0/492954155 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1fa806dc60 msgr2=0x7f1fa8159fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:18.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.073+0000 7f1fae6d1640 1 --2- 172.21.15.32:0/492954155 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1fa806dc60 0x7f1fa8159fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:18.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.073+0000 7f1fae6d1640 1 -- 172.21.15.32:0/492954155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fa806b860 msgr2=0x7f1fa80b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:18.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.073+0000 7f1fae6d1640 1 --2- 172.21.15.32:0/492954155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fa806b860 0x7f1fa80b5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:18.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.073+0000 7f1fae6d1640 1 -- 172.21.15.32:0/492954155 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fa80b3fc0 con 0x7f1fa8158dd0 2024-04-24T23:01:18.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.073+0000 7f1fae6d1640 1 --2- 172.21.15.32:0/492954155 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fa8158dd0 0x7f1fa80b3390 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f1f980090b0 tx=0x7f1f980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:18.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.073+0000 7f1fa67fc640 1 -- 172.21.15.32:0/492954155 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f98041020 con 0x7f1fa8158dd0 2024-04-24T23:01:18.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.073+0000 7f1fa67fc640 1 -- 172.21.15.32:0/492954155 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f98030990 con 0x7f1fa8158dd0 2024-04-24T23:01:18.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.073+0000 7f1fb0134640 1 -- 172.21.15.32:0/492954155 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fa80b42a0 con 0x7f1fa8158dd0 2024-04-24T23:01:18.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.073+0000 7f1fa67fc640 1 -- 172.21.15.32:0/492954155 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f98037040 con 0x7f1fa8158dd0 2024-04-24T23:01:18.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.073+0000 7f1fb0134640 1 -- 172.21.15.32:0/492954155 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fa80b4b40 con 0x7f1fa8158dd0 2024-04-24T23:01:18.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.074+0000 7f1fb0134640 1 -- 172.21.15.32:0/492954155 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fa80b4470 con 0x7f1fa8158dd0 2024-04-24T23:01:18.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.075+0000 7f1fa67fc640 1 -- 172.21.15.32:0/492954155 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1f98044020 con 0x7f1fa8158dd0 2024-04-24T23:01:18.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.075+0000 7f1fa67fc640 1 --2- 172.21.15.32:0/492954155 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1f9406be30 0x7f1f9406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:18.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.076+0000 7f1fa67fc640 1 -- 172.21.15.32:0/492954155 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 4943+0+0 (secure 0 0 0) 0x7f1f980ab3e0 con 0x7f1fa8158dd0 2024-04-24T23:01:18.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.076+0000 7f1faded0640 1 --2- 172.21.15.32:0/492954155 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1f9406be30 0x7f1f9406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:18.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.076+0000 7f1faded0640 1 --2- 172.21.15.32:0/492954155 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1f9406be30 0x7f1f9406e2f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1fa000d620 tx=0x7f1fa0007810 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:18.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.080+0000 7f1fa67fc640 1 -- 172.21.15.32:0/492954155 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1f9804b030 con 0x7f1fa8158dd0 2024-04-24T23:01:18.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.262+0000 7f1fb0134640 1 -- 172.21.15.32:0/492954155 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1fa8068430 con 0x7f1fa8158dd0 2024-04-24T23:01:18.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.262+0000 7f1fa67fc640 1 -- 172.21.15.32:0/492954155 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) ==== 93+0+10 (secure 0 0 0) 0x7f1f98079a40 con 0x7f1fa8158dd0 2024-04-24T23:01:18.264 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T23:01:18.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.264+0000 7f1fb0134640 1 -- 172.21.15.32:0/492954155 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1f9406be30 msgr2=0x7f1f9406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:18.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.264+0000 7f1fb0134640 1 --2- 172.21.15.32:0/492954155 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1f9406be30 0x7f1f9406e2f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1fa000d620 tx=0x7f1fa0007810 comp rx=0 tx=0).stop 2024-04-24T23:01:18.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.264+0000 7f1fb0134640 1 -- 172.21.15.32:0/492954155 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fa8158dd0 msgr2=0x7f1fa80b3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:18.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.264+0000 7f1fb0134640 1 --2- 172.21.15.32:0/492954155 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fa8158dd0 0x7f1fa80b3390 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f1f980090b0 tx=0x7f1f980046c0 comp rx=0 tx=0).stop 2024-04-24T23:01:18.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.264+0000 7f1fb0134640 1 -- 172.21.15.32:0/492954155 shutdown_connections 2024-04-24T23:01:18.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.264+0000 7f1fb0134640 1 --2- 172.21.15.32:0/492954155 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1f9406be30 0x7f1f9406e2f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:18.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.264+0000 7f1fb0134640 1 --2- 172.21.15.32:0/492954155 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fa8158dd0 0x7f1fa80b3390 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:18.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.264+0000 7f1fb0134640 1 --2- 172.21.15.32:0/492954155 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1fa806dc60 0x7f1fa8159fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:18.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.264+0000 7f1fb0134640 1 --2- 172.21.15.32:0/492954155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fa806b860 0x7f1fa80b5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:18.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.264+0000 7f1fb0134640 1 -- 172.21.15.32:0/492954155 >> 172.21.15.32:0/492954155 conn(0x7f1fa806e800 msgr2=0x7f1fa81526b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:18.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.264+0000 7f1fb0134640 1 -- 172.21.15.32:0/492954155 shutdown_connections 2024-04-24T23:01:18.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.264+0000 7f1fb0134640 1 -- 172.21.15.32:0/492954155 wait complete. 2024-04-24T23:01:18.277 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fclose' 2024-04-24T23:01:18.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.376+0000 7ff3129b7640 1 Processor -- start 2024-04-24T23:01:18.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.376+0000 7ff3129b7640 1 -- start start 2024-04-24T23:01:18.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.377+0000 7ff3129b7640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff30c06c5f0 0x7ff30c06c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:18.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.377+0000 7ff3129b7640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff30c0af890 0x7ff30c15cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:18.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.377+0000 7ff3129b7640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff30c0b6430 con 0x7ff30c0b44d0 2024-04-24T23:01:18.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.377+0000 7ff3129b7640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7ff30c156f80 con 0x7ff30c06c5f0 2024-04-24T23:01:18.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.377+0000 7ff3129b7640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff30c0b6150 con 0x7ff30c0af890 2024-04-24T23:01:18.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.377+0000 7ff30b7fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff30c0b44d0 0x7ff30c0b5890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47838/0 (socket says 172.21.15.32:47838) 2024-04-24T23:01:18.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.377+0000 7ff30b7fe640 1 -- 172.21.15.32:0/332450643 learned_addr learned my addr 172.21.15.32:0/332450643 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:01:18.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.377+0000 7ff310f54640 1 --2- 172.21.15.32:0/332450643 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff30c0af890 0x7ff30c15cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:18.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.377+0000 7ff30affd640 1 -- 172.21.15.32:0/332450643 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 433947978 0 0) 0x7ff30c0b6430 con 0x7ff30c0b44d0 2024-04-24T23:01:18.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.377+0000 7ff30affd640 1 -- 172.21.15.32:0/332450643 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff2f4003520 con 0x7ff30c0b44d0 2024-04-24T23:01:18.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.377+0000 7ff30bfff640 1 --2- 172.21.15.32:0/332450643 >> v2:172.21.15.121:3300/0 conn(0x7ff30c06c5f0 0x7ff30c06c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:18.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.377+0000 7ff30affd640 1 -- 172.21.15.32:0/332450643 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3758833081 0 0) 0x7ff2f4003520 con 0x7ff30c0b44d0 2024-04-24T23:01:18.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.378+0000 7ff30affd640 1 -- 172.21.15.32:0/332450643 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff30c0af890 msgr2=0x7ff30c15cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:18.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.378+0000 7ff30affd640 1 --2- 172.21.15.32:0/332450643 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff30c0af890 0x7ff30c15cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:18.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.378+0000 7ff30affd640 1 -- 172.21.15.32:0/332450643 >> v2:172.21.15.121:3300/0 conn(0x7ff30c06c5f0 msgr2=0x7ff30c06c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:18.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.378+0000 7ff30affd640 1 --2- 172.21.15.32:0/332450643 >> v2:172.21.15.121:3300/0 conn(0x7ff30c06c5f0 0x7ff30c06c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:18.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.378+0000 7ff30affd640 1 -- 172.21.15.32:0/332450643 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff30c0b39f0 con 0x7ff30c0b44d0 2024-04-24T23:01:18.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.378+0000 7ff30affd640 1 -- 172.21.15.32:0/332450643 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7ff2fc001690 con 0x7ff30c0b44d0 2024-04-24T23:01:18.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.378+0000 7ff30affd640 1 -- 172.21.15.32:0/332450643 >> v1:172.21.15.32:6789/0 conn(0x7ff30c0b44d0 legacy=0x7ff30c0b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:18.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.378+0000 7ff30affd640 1 --2- 172.21.15.32:0/332450643 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff30c0af890 0x7ff2f40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:18.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.378+0000 7ff30affd640 1 --2- 172.21.15.32:0/332450643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff2f4003f30 0x7ff2f4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:18.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.378+0000 7ff30affd640 1 --2- 172.21.15.32:0/332450643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2f4004d40 0x7ff2f40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:18.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.378+0000 7ff30affd640 1 -- 172.21.15.32:0/332450643 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c156f80 con 0x7ff2f4004d40 2024-04-24T23:01:18.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.378+0000 7ff30affd640 1 -- 172.21.15.32:0/332450643 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c0b6150 con 0x7ff2f4003f30 2024-04-24T23:01:18.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.378+0000 7ff30affd640 1 -- 172.21.15.32:0/332450643 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff2f400ba10 con 0x7ff30c0af890 2024-04-24T23:01:18.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.378+0000 7ff310f54640 1 --2- 172.21.15.32:0/332450643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2f4004d40 0x7ff2f40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:18.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.378+0000 7ff30bfff640 1 --2- 172.21.15.32:0/332450643 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff30c0af890 0x7ff2f40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:18.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.378+0000 7ff30b7fe640 1 --2- 172.21.15.32:0/332450643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff2f4003f30 0x7ff2f4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:18.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.378+0000 7ff310f54640 1 -- 172.21.15.32:0/332450643 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff30c0af890 msgr2=0x7ff2f40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:18.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.378+0000 7ff310f54640 1 --2- 172.21.15.32:0/332450643 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff30c0af890 0x7ff2f40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:18.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.378+0000 7ff310f54640 1 -- 172.21.15.32:0/332450643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff2f4003f30 msgr2=0x7ff2f4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:18.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.378+0000 7ff310f54640 1 --2- 172.21.15.32:0/332450643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff2f4003f30 0x7ff2f4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:18.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.378+0000 7ff310f54640 1 -- 172.21.15.32:0/332450643 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2f4003520 con 0x7ff2f4004d40 2024-04-24T23:01:18.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.379+0000 7ff310f54640 1 --2- 172.21.15.32:0/332450643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2f4004d40 0x7ff2f40051c0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7ff30400e170 tx=0x7ff2f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=376f06395647f548 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:18.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.379+0000 7ff30affd640 1 -- 172.21.15.32:0/332450643 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff304034020 con 0x7ff2f4004d40 2024-04-24T23:01:18.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.379+0000 7ff30affd640 1 -- 172.21.15.32:0/332450643 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3040305f0 con 0x7ff2f4004d40 2024-04-24T23:01:18.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.379+0000 7ff30affd640 1 -- 172.21.15.32:0/332450643 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff304009c10 con 0x7ff2f4004d40 2024-04-24T23:01:18.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.379+0000 7ff3129b7640 1 -- 172.21.15.32:0/332450643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2f4004d40 msgr2=0x7ff2f40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:18.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.379+0000 7ff3129b7640 1 --2- 172.21.15.32:0/332450643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2f4004d40 0x7ff2f40051c0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7ff30400e170 tx=0x7ff2f4005830 comp rx=0 tx=0).stop 2024-04-24T23:01:18.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.379+0000 7ff310f54640 1 -- 172.21.15.32:0/332450643 reap_dead start 2024-04-24T23:01:18.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.379+0000 7ff3129b7640 1 -- 172.21.15.32:0/332450643 shutdown_connections 2024-04-24T23:01:18.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.379+0000 7ff3129b7640 1 -- 172.21.15.32:0/332450643 >> 172.21.15.32:0/332450643 conn(0x7ff30c06b110 msgr2=0x7ff30c06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:18.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.379+0000 7ff3129b7640 1 -- 172.21.15.32:0/332450643 shutdown_connections 2024-04-24T23:01:18.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.379+0000 7ff3129b7640 1 -- 172.21.15.32:0/332450643 wait complete. 2024-04-24T23:01:18.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.381+0000 7ff3129b7640 1 Processor -- start 2024-04-24T23:01:18.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.381+0000 7ff3129b7640 1 -- start start 2024-04-24T23:01:18.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.381+0000 7ff3129b7640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff30c159c60 0x7ff30c1f8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:18.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.381+0000 7ff3129b7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff30c15e880 0x7ff30c157ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:18.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.382+0000 7ff3129b7640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff30c0ae8b0 0x7ff30c0aed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:18.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.382+0000 7ff3129b7640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c06cf10 con 0x7ff30c15e880 2024-04-24T23:01:18.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.382+0000 7ff3129b7640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c0b3fb0 con 0x7ff30c0ae8b0 2024-04-24T23:01:18.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.382+0000 7ff3129b7640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff30c15cac0 con 0x7ff30c159c60 2024-04-24T23:01:18.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.382+0000 7ff30b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff30c15e880 0x7ff30c157ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:18.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.382+0000 7ff30b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff30c15e880 0x7ff30c157ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43890/0 (socket says 172.21.15.32:43890) 2024-04-24T23:01:18.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.382+0000 7ff30b7fe640 1 -- 172.21.15.32:0/944751843 learned_addr learned my addr 172.21.15.32:0/944751843 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:18.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.382+0000 7ff30bfff640 1 --2- 172.21.15.32:0/944751843 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff30c159c60 0x7ff30c1f8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:18.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.382+0000 7ff310f54640 1 --2- 172.21.15.32:0/944751843 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff30c0ae8b0 0x7ff30c0aed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:18.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.382+0000 7ff30b7fe640 1 -- 172.21.15.32:0/944751843 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff30c159c60 msgr2=0x7ff30c1f8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:18.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.382+0000 7ff30b7fe640 1 --2- 172.21.15.32:0/944751843 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff30c159c60 0x7ff30c1f8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:18.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.382+0000 7ff30b7fe640 1 -- 172.21.15.32:0/944751843 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff30c0ae8b0 msgr2=0x7ff30c0aed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:18.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.382+0000 7ff30b7fe640 1 --2- 172.21.15.32:0/944751843 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff30c0ae8b0 0x7ff30c0aed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:18.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.382+0000 7ff30b7fe640 1 -- 172.21.15.32:0/944751843 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff30c15ee20 con 0x7ff30c15e880 2024-04-24T23:01:18.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.382+0000 7ff310f54640 1 --2- 172.21.15.32:0/944751843 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff30c0ae8b0 0x7ff30c0aed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:01:18.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.382+0000 7ff30bfff640 1 --2- 172.21.15.32:0/944751843 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff30c159c60 0x7ff30c1f8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:01:18.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.382+0000 7ff30b7fe640 1 --2- 172.21.15.32:0/944751843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff30c15e880 0x7ff30c157ec0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc0072c0 tx=0x7ff2fc007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:18.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.382+0000 7ff308ff9640 1 -- 172.21.15.32:0/944751843 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc00f960 con 0x7ff30c15e880 2024-04-24T23:01:18.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.382+0000 7ff308ff9640 1 -- 172.21.15.32:0/944751843 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2fc018070 con 0x7ff30c15e880 2024-04-24T23:01:18.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.382+0000 7ff3129b7640 1 -- 172.21.15.32:0/944751843 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff30c15f100 con 0x7ff30c15e880 2024-04-24T23:01:18.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.382+0000 7ff308ff9640 1 -- 172.21.15.32:0/944751843 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc00fd80 con 0x7ff30c15e880 2024-04-24T23:01:18.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.382+0000 7ff3129b7640 1 -- 172.21.15.32:0/944751843 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff30c15f6c0 con 0x7ff30c15e880 2024-04-24T23:01:18.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.383+0000 7ff3129b7640 1 -- 172.21.15.32:0/944751843 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff30c0af270 con 0x7ff30c15e880 2024-04-24T23:01:18.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.384+0000 7ff308ff9640 1 -- 172.21.15.32:0/944751843 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff2fc01d460 con 0x7ff30c15e880 2024-04-24T23:01:18.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.384+0000 7ff308ff9640 1 --2- 172.21.15.32:0/944751843 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff2f806be30 0x7ff2f806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:18.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.384+0000 7ff308ff9640 1 -- 172.21.15.32:0/944751843 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 4943+0+0 (secure 0 0 0) 0x7ff2fc08ac50 con 0x7ff30c15e880 2024-04-24T23:01:18.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.384+0000 7ff30bfff640 1 --2- 172.21.15.32:0/944751843 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff2f806be30 0x7ff2f806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:18.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.385+0000 7ff30bfff640 1 --2- 172.21.15.32:0/944751843 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff2f806be30 0x7ff2f806e2f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff300009b30 tx=0x7ff300008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:18.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.389+0000 7ff308ff9640 1 -- 172.21.15.32:0/944751843 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff2fc090020 con 0x7ff30c15e880 2024-04-24T23:01:18.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:18.643+0000 7ff3129b7640 1 -- 172.21.15.32:0/944751843 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]} v 0) -- 0x7ff30c068430 con 0x7ff30c15e880 2024-04-24T23:01:19.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.640+0000 7ff308ff9640 1 -- 172.21.15.32:0/944751843 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]}]=0 v142) ==== 137+0+0 (secure 0 0 0) 0x7ff2fc059900 con 0x7ff30c15e880 2024-04-24T23:01:19.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.644+0000 7ff3129b7640 1 -- 172.21.15.32:0/944751843 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff2f806be30 msgr2=0x7ff2f806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:19.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.644+0000 7ff3129b7640 1 --2- 172.21.15.32:0/944751843 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff2f806be30 0x7ff2f806e2f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff300009b30 tx=0x7ff300008040 comp rx=0 tx=0).stop 2024-04-24T23:01:19.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.644+0000 7ff3129b7640 1 -- 172.21.15.32:0/944751843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff30c15e880 msgr2=0x7ff30c157ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:19.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.644+0000 7ff3129b7640 1 --2- 172.21.15.32:0/944751843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff30c15e880 0x7ff30c157ec0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc0072c0 tx=0x7ff2fc007780 comp rx=0 tx=0).stop 2024-04-24T23:01:19.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.644+0000 7ff3129b7640 1 -- 172.21.15.32:0/944751843 shutdown_connections 2024-04-24T23:01:19.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.644+0000 7ff3129b7640 1 --2- 172.21.15.32:0/944751843 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff2f806be30 0x7ff2f806e2f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:19.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.644+0000 7ff3129b7640 1 --2- 172.21.15.32:0/944751843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff30c15e880 0x7ff30c157ec0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:19.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.644+0000 7ff3129b7640 1 --2- 172.21.15.32:0/944751843 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff30c0ae8b0 0x7ff30c0aed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:19.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.644+0000 7ff3129b7640 1 --2- 172.21.15.32:0/944751843 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff30c159c60 0x7ff30c1f8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:19.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.644+0000 7ff3129b7640 1 -- 172.21.15.32:0/944751843 >> 172.21.15.32:0/944751843 conn(0x7ff30c06b110 msgr2=0x7ff30c06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:19.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.644+0000 7ff3129b7640 1 -- 172.21.15.32:0/944751843 shutdown_connections 2024-04-24T23:01:19.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.644+0000 7ff3129b7640 1 -- 172.21.15.32:0/944751843 wait complete. 2024-04-24T23:01:19.660 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2024-04-24T23:01:19.780 INFO:teuthology.orchestra.run.smithi032.stdout:{"osd_objectstore":"memstore"} 2024-04-24T23:01:19.780 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2024-04-24T23:01:19.781 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-24T23:01:19.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.883+0000 7f1984088640 1 Processor -- start 2024-04-24T23:01:19.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.883+0000 7f1984088640 1 -- start start 2024-04-24T23:01:19.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.884+0000 7f1984088640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f197c06cd00 0x7f197c0ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:19.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.884+0000 7f1984088640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f197c15cb00 0x7f197c0ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:19.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.884+0000 7f1984088640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f197c0b4520 con 0x7f197c15b380 2024-04-24T23:01:19.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.884+0000 7f1984088640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f197c0b4800 con 0x7f197c15cb00 2024-04-24T23:01:19.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.884+0000 7f1984088640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f197c0b4ae0 con 0x7f197c06cd00 2024-04-24T23:01:19.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.884+0000 7f1982625640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f197c15cb00 0x7f197c0ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:19.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.884+0000 7f1982625640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f197c15cb00 0x7f197c0ce970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41548/0 (socket says 172.21.15.32:41548) 2024-04-24T23:01:19.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.884+0000 7f1982625640 1 -- 172.21.15.32:0/375824495 learned_addr learned my addr 172.21.15.32:0/375824495 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:19.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.884+0000 7f1981623640 1 --2- 172.21.15.32:0/375824495 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f197c06cd00 0x7f197c0ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:19.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.884+0000 7f1982625640 1 -- 172.21.15.32:0/375824495 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f197c06cd00 msgr2=0x7f197c0ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:19.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.884+0000 7f1982625640 1 --2- 172.21.15.32:0/375824495 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f197c06cd00 0x7f197c0ce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:19.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.884+0000 7f1982625640 1 -- 172.21.15.32:0/375824495 >> v1:172.21.15.121:6789/0 conn(0x7f197c15b380 legacy=0x7f197c06c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:19.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.884+0000 7f1982625640 1 -- 172.21.15.32:0/375824495 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f197c0b4dc0 con 0x7f197c15cb00 2024-04-24T23:01:19.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.884+0000 7f1981623640 1 --2- 172.21.15.32:0/375824495 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f197c06cd00 0x7f197c0ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:01:19.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.885+0000 7f1982625640 1 --2- 172.21.15.32:0/375824495 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f197c15cb00 0x7f197c0ce970 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f1974009e50 tx=0x7f197402a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eec21d0a590cf52 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:19.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.885+0000 7f1980e22640 1 -- 172.21.15.32:0/375824495 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1974036070 con 0x7f197c15cb00 2024-04-24T23:01:19.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.885+0000 7f1980e22640 1 -- 172.21.15.32:0/375824495 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1974031600 con 0x7f197c15cb00 2024-04-24T23:01:19.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.885+0000 7f1980e22640 1 -- 172.21.15.32:0/375824495 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19740319d0 con 0x7f197c15cb00 2024-04-24T23:01:19.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.886+0000 7f1984088640 1 -- 172.21.15.32:0/375824495 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f197c15cb00 msgr2=0x7f197c0ce970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:19.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.886+0000 7f1984088640 1 --2- 172.21.15.32:0/375824495 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f197c15cb00 0x7f197c0ce970 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f1974009e50 tx=0x7f197402a630 comp rx=0 tx=0).stop 2024-04-24T23:01:19.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.886+0000 7f1984088640 1 -- 172.21.15.32:0/375824495 shutdown_connections 2024-04-24T23:01:19.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.886+0000 7f1984088640 1 --2- 172.21.15.32:0/375824495 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f197c15cb00 0x7f197c0ce970 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:19.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.886+0000 7f1984088640 1 --2- 172.21.15.32:0/375824495 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f197c06cd00 0x7f197c0ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:19.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.886+0000 7f1984088640 1 -- 172.21.15.32:0/375824495 >> 172.21.15.32:0/375824495 conn(0x7f197c06b110 msgr2=0x7f197c06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:19.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.886+0000 7f1984088640 1 -- 172.21.15.32:0/375824495 shutdown_connections 2024-04-24T23:01:19.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.886+0000 7f1984088640 1 -- 172.21.15.32:0/375824495 wait complete. 2024-04-24T23:01:19.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.887+0000 7f1984088640 1 Processor -- start 2024-04-24T23:01:19.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.887+0000 7f1984088640 1 -- start start 2024-04-24T23:01:19.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.888+0000 7f1984088640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f197c06cd00 0x7f197c1f5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:19.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.888+0000 7f1984088640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f197c15b380 0x7f197c13bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:19.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.888+0000 7f1984088640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f197c15cb00 0x7f197c1599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:19.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.888+0000 7f1984088640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f197c15b760 con 0x7f197c15cb00 2024-04-24T23:01:19.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.888+0000 7f1984088640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f197c0ceeb0 con 0x7f197c15b380 2024-04-24T23:01:19.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.888+0000 7f1984088640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f197c0b55c0 con 0x7f197c06cd00 2024-04-24T23:01:19.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.888+0000 7f1982625640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f197c15cb00 0x7f197c1599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:19.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.888+0000 7f1982625640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f197c15cb00 0x7f197c1599c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41558/0 (socket says 172.21.15.32:41558) 2024-04-24T23:01:19.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.888+0000 7f1982625640 1 -- 172.21.15.32:0/1884750487 learned_addr learned my addr 172.21.15.32:0/1884750487 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:19.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.888+0000 7f1981623640 1 --2- 172.21.15.32:0/1884750487 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f197c15b380 0x7f197c13bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:19.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.888+0000 7f1981e24640 1 --2- 172.21.15.32:0/1884750487 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f197c06cd00 0x7f197c1f5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:19.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.888+0000 7f1982625640 1 -- 172.21.15.32:0/1884750487 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f197c06cd00 msgr2=0x7f197c1f5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:19.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.888+0000 7f1982625640 1 --2- 172.21.15.32:0/1884750487 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f197c06cd00 0x7f197c1f5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:19.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.888+0000 7f1982625640 1 -- 172.21.15.32:0/1884750487 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f197c15b380 msgr2=0x7f197c13bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:19.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.888+0000 7f1982625640 1 --2- 172.21.15.32:0/1884750487 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f197c15b380 0x7f197c13bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:19.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.888+0000 7f1982625640 1 -- 172.21.15.32:0/1884750487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f197c13ca70 con 0x7f197c15cb00 2024-04-24T23:01:19.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.888+0000 7f1981623640 1 --2- 172.21.15.32:0/1884750487 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f197c15b380 0x7f197c13bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:01:19.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.888+0000 7f1981e24640 1 --2- 172.21.15.32:0/1884750487 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f197c06cd00 0x7f197c1f5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:01:19.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.888+0000 7f1982625640 1 --2- 172.21.15.32:0/1884750487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f197c15cb00 0x7f197c1599c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f1974009e50 tx=0x7f1974004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:19.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.888+0000 7f19727fc640 1 -- 172.21.15.32:0/1884750487 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1974036050 con 0x7f197c15cb00 2024-04-24T23:01:19.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.889+0000 7f19727fc640 1 -- 172.21.15.32:0/1884750487 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1974042070 con 0x7f197c15cb00 2024-04-24T23:01:19.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.889+0000 7f19727fc640 1 -- 172.21.15.32:0/1884750487 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1974009510 con 0x7f197c15cb00 2024-04-24T23:01:19.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.889+0000 7f1984088640 1 -- 172.21.15.32:0/1884750487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f197c13c4b0 con 0x7f197c15cb00 2024-04-24T23:01:19.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.889+0000 7f1984088640 1 -- 172.21.15.32:0/1884750487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f197c139b70 con 0x7f197c15cb00 2024-04-24T23:01:19.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.889+0000 7f1984088640 1 -- 172.21.15.32:0/1884750487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f197c139ea0 con 0x7f197c15cb00 2024-04-24T23:01:19.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.890+0000 7f19727fc640 1 -- 172.21.15.32:0/1884750487 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1974007bd0 con 0x7f197c15cb00 2024-04-24T23:01:19.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.890+0000 7f19727fc640 1 --2- 172.21.15.32:0/1884750487 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f196006be60 0x7f196006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:19.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.891+0000 7f19727fc640 1 -- 172.21.15.32:0/1884750487 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 4943+0+0 (secure 0 0 0) 0x7f19740abb10 con 0x7f197c15cb00 2024-04-24T23:01:19.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.891+0000 7f1981e24640 1 --2- 172.21.15.32:0/1884750487 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f196006be60 0x7f196006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:19.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.891+0000 7f1981e24640 1 --2- 172.21.15.32:0/1884750487 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f196006be60 0x7f196006e320 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1978003eb0 tx=0x7f1978009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:19.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:19.895+0000 7f19727fc640 1 -- 172.21.15.32:0/1884750487 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1974045360 con 0x7f197c15cb00 2024-04-24T23:01:20.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.082+0000 7f1984088640 1 -- 172.21.15.32:0/1884750487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f197c068430 con 0x7f197c15cb00 2024-04-24T23:01:20.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.085+0000 7f19727fc640 1 -- 172.21.15.32:0/1884750487 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v6)=0 v6) ==== 122+0+0 (secure 0 0 0) 0x7f197407a7c0 con 0x7f197c15cb00 2024-04-24T23:01:20.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.087+0000 7f1984088640 1 -- 172.21.15.32:0/1884750487 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f196006be60 msgr2=0x7f196006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:20.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.087+0000 7f1984088640 1 --2- 172.21.15.32:0/1884750487 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f196006be60 0x7f196006e320 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1978003eb0 tx=0x7f1978009040 comp rx=0 tx=0).stop 2024-04-24T23:01:20.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.087+0000 7f1984088640 1 -- 172.21.15.32:0/1884750487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f197c15cb00 msgr2=0x7f197c1599c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:20.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.087+0000 7f1984088640 1 --2- 172.21.15.32:0/1884750487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f197c15cb00 0x7f197c1599c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f1974009e50 tx=0x7f1974004a40 comp rx=0 tx=0).stop 2024-04-24T23:01:20.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.087+0000 7f1984088640 1 -- 172.21.15.32:0/1884750487 shutdown_connections 2024-04-24T23:01:20.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.087+0000 7f1984088640 1 --2- 172.21.15.32:0/1884750487 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f196006be60 0x7f196006e320 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:20.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.087+0000 7f1984088640 1 --2- 172.21.15.32:0/1884750487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f197c15cb00 0x7f197c1599c0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:20.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.087+0000 7f1984088640 1 --2- 172.21.15.32:0/1884750487 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f197c15b380 0x7f197c13bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:20.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.087+0000 7f1984088640 1 --2- 172.21.15.32:0/1884750487 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f197c06cd00 0x7f197c1f5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:20.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.087+0000 7f1984088640 1 -- 172.21.15.32:0/1884750487 >> 172.21.15.32:0/1884750487 conn(0x7f197c06b110 msgr2=0x7f197c06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:20.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.088+0000 7f1984088640 1 -- 172.21.15.32:0/1884750487 shutdown_connections 2024-04-24T23:01:20.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.088+0000 7f1984088640 1 -- 172.21.15.32:0/1884750487 wait complete. 2024-04-24T23:01:20.100 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-24T23:01:20.100 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:01:20.139 DEBUG:teuthology.orchestra.run.smithi121:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:01:20.188 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-24T23:01:20.188 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:01:20.188 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:01:20.204 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=413 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T23:01:20.204 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:01:20.204 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:01:20.221 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3271 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T23:01:20.221 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:01:20.221 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:01:20.260 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=413 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T23:01:20.260 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:01:20.260 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:01:20.278 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3271 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T23:01:20.278 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-24T23:01:20.278 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:01:20.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.399+0000 7f2173514640 1 Processor -- start 2024-04-24T23:01:20.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.399+0000 7f2173514640 1 -- start start 2024-04-24T23:01:20.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.400+0000 7f2173514640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f216c162320 0x7f216c0ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:20.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.400+0000 7f2173514640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f216c0cd270 con 0x7f216c162e10 2024-04-24T23:01:20.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.400+0000 7f2173514640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f216c071610 con 0x7f216c0b5af0 2024-04-24T23:01:20.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.400+0000 7f2173514640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f216c0b5660 con 0x7f216c162320 2024-04-24T23:01:20.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.400+0000 7f21712b0640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f216c162320 0x7f216c0ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:20.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.400+0000 7f2171ab1640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f216c162e10 0x7f216c15ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42976/0 (socket says 172.21.15.32:42976) 2024-04-24T23:01:20.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.400+0000 7f2171ab1640 1 -- 172.21.15.32:0/1366242253 learned_addr learned my addr 172.21.15.32:0/1366242253 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:01:20.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.400+0000 7f21712b0640 1 -- 172.21.15.32:0/1366242253 >> v1:172.21.15.121:6789/0 conn(0x7f216c0b5af0 legacy=0x7f216c162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:20.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.400+0000 7f21712b0640 1 -- 172.21.15.32:0/1366242253 >> v1:172.21.15.32:6789/0 conn(0x7f216c162e10 legacy=0x7f216c15ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:20.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.400+0000 7f21712b0640 1 -- 172.21.15.32:0/1366242253 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f216c0718f0 con 0x7f216c162320 2024-04-24T23:01:20.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.400+0000 7f2163fff640 1 -- 172.21.15.32:0/1366242253 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3035597155 0 0) 0x7f216c0cd270 con 0x7f216c162e10 2024-04-24T23:01:20.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.401+0000 7f21712b0640 1 --2- 172.21.15.32:0/1366242253 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f216c162320 0x7f216c0ccc90 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f216c06c5b0 tx=0x7f216802a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e68a294ad714cda5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:20.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.401+0000 7f2163fff640 1 -- 172.21.15.32:0/1366242253 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2168037070 con 0x7f216c162320 2024-04-24T23:01:20.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.401+0000 7f2163fff640 1 -- 172.21.15.32:0/1366242253 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f216802ae40 con 0x7f216c162320 2024-04-24T23:01:20.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.401+0000 7f2163fff640 1 -- 172.21.15.32:0/1366242253 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2168032660 con 0x7f216c162320 2024-04-24T23:01:20.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.402+0000 7f2173514640 1 -- 172.21.15.32:0/1366242253 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f216c162320 msgr2=0x7f216c0ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:20.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.402+0000 7f2173514640 1 --2- 172.21.15.32:0/1366242253 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f216c162320 0x7f216c0ccc90 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f216c06c5b0 tx=0x7f216802a130 comp rx=0 tx=0).stop 2024-04-24T23:01:20.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.402+0000 7f2173514640 1 -- 172.21.15.32:0/1366242253 shutdown_connections 2024-04-24T23:01:20.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.402+0000 7f2173514640 1 --2- 172.21.15.32:0/1366242253 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f216c162320 0x7f216c0ccc90 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:20.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.402+0000 7f2173514640 1 -- 172.21.15.32:0/1366242253 >> 172.21.15.32:0/1366242253 conn(0x7f216c0d0ae0 msgr2=0x7f216c0d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:20.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.402+0000 7f2173514640 1 -- 172.21.15.32:0/1366242253 shutdown_connections 2024-04-24T23:01:20.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.402+0000 7f2173514640 1 -- 172.21.15.32:0/1366242253 wait complete. 2024-04-24T23:01:20.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.403+0000 7f2173514640 1 Processor -- start 2024-04-24T23:01:20.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.403+0000 7f2173514640 1 -- start start 2024-04-24T23:01:20.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.404+0000 7f2173514640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f216c0b5af0 0x7f216c1493f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:20.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.404+0000 7f2173514640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f216c162320 0x7f216c06ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:20.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.404+0000 7f2173514640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f216c162e10 0x7f216c161690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:20.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.404+0000 7f2173514640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f216c071bd0 con 0x7f216c0b5af0 2024-04-24T23:01:20.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.404+0000 7f2173514640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f216c067950 con 0x7f216c162320 2024-04-24T23:01:20.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.404+0000 7f2173514640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f216c071d50 con 0x7f216c162e10 2024-04-24T23:01:20.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.404+0000 7f21712b0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f216c0b5af0 0x7f216c1493f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:20.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.404+0000 7f21712b0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f216c0b5af0 0x7f216c1493f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41576/0 (socket says 172.21.15.32:41576) 2024-04-24T23:01:20.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.404+0000 7f21712b0640 1 -- 172.21.15.32:0/3237797908 learned_addr learned my addr 172.21.15.32:0/3237797908 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:20.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.404+0000 7f2170aaf640 1 --2- 172.21.15.32:0/3237797908 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f216c162320 0x7f216c06ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:20.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.404+0000 7f2171ab1640 1 --2- 172.21.15.32:0/3237797908 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f216c162e10 0x7f216c161690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:20.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.404+0000 7f21712b0640 1 -- 172.21.15.32:0/3237797908 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f216c162e10 msgr2=0x7f216c161690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:20.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.404+0000 7f21712b0640 1 --2- 172.21.15.32:0/3237797908 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f216c162e10 0x7f216c161690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:20.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.404+0000 7f21712b0640 1 -- 172.21.15.32:0/3237797908 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f216c162320 msgr2=0x7f216c06ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:20.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.404+0000 7f21712b0640 1 --2- 172.21.15.32:0/3237797908 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f216c162320 0x7f216c06ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:20.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.404+0000 7f21712b0640 1 -- 172.21.15.32:0/3237797908 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f216c1f8050 con 0x7f216c0b5af0 2024-04-24T23:01:20.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.404+0000 7f21712b0640 1 --2- 172.21.15.32:0/3237797908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f216c0b5af0 0x7f216c1493f0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f2168000c00 tx=0x7f21680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:20.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.404+0000 7f2161ffb640 1 -- 172.21.15.32:0/3237797908 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2168042070 con 0x7f216c0b5af0 2024-04-24T23:01:20.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.404+0000 7f2161ffb640 1 -- 172.21.15.32:0/3237797908 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f216803e670 con 0x7f216c0b5af0 2024-04-24T23:01:20.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.404+0000 7f2173514640 1 -- 172.21.15.32:0/3237797908 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f216c1f82f0 con 0x7f216c0b5af0 2024-04-24T23:01:20.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.404+0000 7f2161ffb640 1 -- 172.21.15.32:0/3237797908 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2168037040 con 0x7f216c0b5af0 2024-04-24T23:01:20.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.405+0000 7f2173514640 1 -- 172.21.15.32:0/3237797908 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f216c1f8b90 con 0x7f216c0b5af0 2024-04-24T23:01:20.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.406+0000 7f2173514640 1 -- 172.21.15.32:0/3237797908 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f216c068430 con 0x7f216c0b5af0 2024-04-24T23:01:20.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.406+0000 7f2161ffb640 1 -- 172.21.15.32:0/3237797908 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2168045070 con 0x7f216c0b5af0 2024-04-24T23:01:20.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.407+0000 7f2161ffb640 1 --2- 172.21.15.32:0/3237797908 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f215006be30 0x7f215006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:20.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.407+0000 7f2161ffb640 1 -- 172.21.15.32:0/3237797908 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 4943+0+0 (secure 0 0 0) 0x7f21680ab390 con 0x7f216c0b5af0 2024-04-24T23:01:20.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.407+0000 7f2170aaf640 1 --2- 172.21.15.32:0/3237797908 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f215006be30 0x7f215006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:20.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.407+0000 7f2170aaf640 1 --2- 172.21.15.32:0/3237797908 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f215006be30 0x7f215006e2f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f216c155c80 tx=0x7f215c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:20.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.411+0000 7f2161ffb640 1 -- 172.21.15.32:0/3237797908 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2168079950 con 0x7f216c0b5af0 2024-04-24T23:01:20.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.636+0000 7f2173514640 1 -- 172.21.15.32:0/3237797908 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f216c067730 con 0x7f216c0b5af0 2024-04-24T23:01:20.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.636+0000 7f2161ffb640 1 -- 172.21.15.32:0/3237797908 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+3875 (secure 0 0 0) 0x7f216807dc50 con 0x7f216c0b5af0 2024-04-24T23:01:20.638 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:01:20.638 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":22,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4994,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/1076304225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1076304225},{"type":"v1","addr":"172.21.15.32:6834","nonce":1076304225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":16},{"gid":5000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1388367206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1388367206},{"type":"v1","addr":"172.21.15.121:6835","nonce":1388367206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":5299,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2065945523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2065945523},{"type":"v1","addr":"172.21.15.121:6837","nonce":2065945523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":5994,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/2687218500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":2687218500},{"type":"v1","addr":"172.21.15.32:6835","nonce":2687218500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[]} 2024-04-24T23:01:20.638 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 22 2024-04-24T23:01:20.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.638+0000 7f2173514640 1 -- 172.21.15.32:0/3237797908 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f215006be30 msgr2=0x7f215006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:20.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.638+0000 7f2173514640 1 --2- 172.21.15.32:0/3237797908 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f215006be30 0x7f215006e2f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f216c155c80 tx=0x7f215c009040 comp rx=0 tx=0).stop 2024-04-24T23:01:20.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.638+0000 7f2173514640 1 -- 172.21.15.32:0/3237797908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f216c0b5af0 msgr2=0x7f216c1493f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:20.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.638+0000 7f2173514640 1 --2- 172.21.15.32:0/3237797908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f216c0b5af0 0x7f216c1493f0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f2168000c00 tx=0x7f21680046c0 comp rx=0 tx=0).stop 2024-04-24T23:01:20.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.638+0000 7f2173514640 1 -- 172.21.15.32:0/3237797908 shutdown_connections 2024-04-24T23:01:20.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.638+0000 7f2173514640 1 --2- 172.21.15.32:0/3237797908 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f215006be30 0x7f215006e2f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:20.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.638+0000 7f2173514640 1 --2- 172.21.15.32:0/3237797908 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f216c162e10 0x7f216c161690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:20.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.638+0000 7f2173514640 1 --2- 172.21.15.32:0/3237797908 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f216c162320 0x7f216c06ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:20.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.638+0000 7f2173514640 1 --2- 172.21.15.32:0/3237797908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f216c0b5af0 0x7f216c1493f0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:20.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.638+0000 7f2173514640 1 -- 172.21.15.32:0/3237797908 >> 172.21.15.32:0/3237797908 conn(0x7f216c0d0ae0 msgr2=0x7f216c0b3ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:20.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.639+0000 7f2173514640 1 -- 172.21.15.32:0/3237797908 shutdown_connections 2024-04-24T23:01:20.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.639+0000 7f2173514640 1 -- 172.21.15.32:0/3237797908 wait complete. 2024-04-24T23:01:20.652 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-24T23:01:20.652 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-24T23:01:20.652 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-24T23:01:20.652 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:01:20.653 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-24T23:01:20.653 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-24T23:01:20.653 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-24T23:01:20.653 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:01:20.653 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-24T23:01:20.653 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-24T23:01:20.654 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-24T23:01:20.654 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:01:20.654 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-24T23:01:20.654 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-24T23:01:20.654 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-24T23:01:20.654 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:01:20.661 INFO:tasks.ceph.mds.a:Stopped 2024-04-24T23:01:20.662 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-24T23:01:20.662 INFO:tasks.ceph.mds.c:Stopped 2024-04-24T23:01:20.663 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-24T23:01:20.665 INFO:tasks.ceph.mds.d:Stopped 2024-04-24T23:01:20.665 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-24T23:01:20.666 INFO:tasks.ceph.mds.b:Stopped 2024-04-24T23:01:20.666 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-24T23:01:20.704 INFO:tasks.ceph.mds.a:Started 2024-04-24T23:01:20.705 DEBUG:teuthology.parallel:result is None 2024-04-24T23:01:20.705 INFO:tasks.ceph.mds.c:Started 2024-04-24T23:01:20.705 DEBUG:teuthology.parallel:result is None 2024-04-24T23:01:20.707 INFO:tasks.ceph.mds.d:Started 2024-04-24T23:01:20.708 DEBUG:teuthology.parallel:result is None 2024-04-24T23:01:20.708 INFO:tasks.ceph.mds.b:Started 2024-04-24T23:01:20.708 DEBUG:teuthology.parallel:result is None 2024-04-24T23:01:20.709 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-24T23:01:20.780 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2024-04-24T23:01:20.780 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2024-04-24T23:01:20.788 INFO:tasks.ceph.mds.d.smithi121.stdout:starting mds.d at 2024-04-24T23:01:20.788 INFO:tasks.ceph.mds.b.smithi121.stdout:starting mds.b at 2024-04-24T23:01:20.788 INFO:tasks.ceph.mds.b.smithi121.stderr:2024-04-24T23:01:20.787+0000 7fefe9066640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-04-24T23:01:20.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.813+0000 7f296a922640 1 Processor -- start 2024-04-24T23:01:20.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.814+0000 7f296a922640 1 -- start start 2024-04-24T23:01:20.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.814+0000 7f296a922640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f296415e9b0 0x7f29640d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:20.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.814+0000 7f296a922640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f29640d4210 0x7f296406b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:20.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.814+0000 7f296a922640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29640d3170 con 0x7f29640ba0e0 2024-04-24T23:01:20.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.814+0000 7f296a922640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f29640b8900 con 0x7f29640d4210 2024-04-24T23:01:20.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.814+0000 7f296a922640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f29640d3450 con 0x7f296415e9b0 2024-04-24T23:01:20.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.814+0000 7f29637fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f29640d4210 0x7f296406b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:20.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.815+0000 7f29637fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f29640d4210 0x7f296406b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41618/0 (socket says 172.21.15.32:41618) 2024-04-24T23:01:20.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.815+0000 7f29637fe640 1 -- 172.21.15.32:0/2809356399 learned_addr learned my addr 172.21.15.32:0/2809356399 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:20.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.815+0000 7f2963fff640 1 --2- 172.21.15.32:0/2809356399 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f296415e9b0 0x7f29640d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:20.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.815+0000 7f29637fe640 1 -- 172.21.15.32:0/2809356399 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f296415e9b0 msgr2=0x7f29640d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:20.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.815+0000 7f29637fe640 1 --2- 172.21.15.32:0/2809356399 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f296415e9b0 0x7f29640d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:20.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.815+0000 7f29637fe640 1 -- 172.21.15.32:0/2809356399 >> v1:172.21.15.121:6789/0 conn(0x7f29640ba0e0 legacy=0x7f2964071730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:20.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.815+0000 7f29637fe640 1 -- 172.21.15.32:0/2809356399 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29640d3730 con 0x7f29640d4210 2024-04-24T23:01:20.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.815+0000 7f29637fe640 1 --2- 172.21.15.32:0/2809356399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29640d4210 0x7f296406b110 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f2958009870 tx=0x7f295802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4707fcc51ea92f67 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:20.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.816+0000 7f2962ffd640 1 -- 172.21.15.32:0/2809356399 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2958037070 con 0x7f29640d4210 2024-04-24T23:01:20.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.816+0000 7f2962ffd640 1 -- 172.21.15.32:0/2809356399 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2958032440 con 0x7f29640d4210 2024-04-24T23:01:20.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.816+0000 7f2962ffd640 1 -- 172.21.15.32:0/2809356399 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2958032810 con 0x7f29640d4210 2024-04-24T23:01:20.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.816+0000 7f296a922640 1 -- 172.21.15.32:0/2809356399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29640d4210 msgr2=0x7f296406b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:20.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.816+0000 7f296a922640 1 --2- 172.21.15.32:0/2809356399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29640d4210 0x7f296406b110 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f2958009870 tx=0x7f295802a270 comp rx=0 tx=0).stop 2024-04-24T23:01:20.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.816+0000 7f296a922640 1 -- 172.21.15.32:0/2809356399 shutdown_connections 2024-04-24T23:01:20.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.816+0000 7f296a922640 1 --2- 172.21.15.32:0/2809356399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29640d4210 0x7f296406b110 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:20.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.816+0000 7f296a922640 1 --2- 172.21.15.32:0/2809356399 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f296415e9b0 0x7f29640d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:20.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.816+0000 7f296a922640 1 -- 172.21.15.32:0/2809356399 >> 172.21.15.32:0/2809356399 conn(0x7f29640d11d0 msgr2=0x7f29640d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:20.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.816+0000 7f296a922640 1 -- 172.21.15.32:0/2809356399 shutdown_connections 2024-04-24T23:01:20.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.816+0000 7f296a922640 1 -- 172.21.15.32:0/2809356399 wait complete. 2024-04-24T23:01:20.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.817+0000 7f296a922640 1 Processor -- start 2024-04-24T23:01:20.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.818+0000 7f296a922640 1 -- start start 2024-04-24T23:01:20.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.818+0000 7f296a922640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29640ba0e0 0x7f29640cc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:20.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.818+0000 7f296a922640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f29640d4210 0x7f2964155be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:20.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.818+0000 7f296a922640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f296415e9b0 0x7f29640cd620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:20.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.818+0000 7f296a922640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f29640d2400 con 0x7f29640ba0e0 2024-04-24T23:01:20.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.818+0000 7f296a922640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f29640d3cf0 con 0x7f29640d4210 2024-04-24T23:01:20.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.818+0000 7f296a922640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f296406b650 con 0x7f296415e9b0 2024-04-24T23:01:20.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.818+0000 7f2963fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29640ba0e0 0x7f29640cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:20.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.818+0000 7f2963fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29640ba0e0 0x7f29640cc230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41626/0 (socket says 172.21.15.32:41626) 2024-04-24T23:01:20.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.818+0000 7f2963fff640 1 -- 172.21.15.32:0/869358024 learned_addr learned my addr 172.21.15.32:0/869358024 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:20.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.819+0000 7f29637fe640 1 --2- 172.21.15.32:0/869358024 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f29640d4210 0x7f2964155be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:20.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.819+0000 7f2968ebf640 1 --2- 172.21.15.32:0/869358024 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f296415e9b0 0x7f29640cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:20.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.819+0000 7f2963fff640 1 -- 172.21.15.32:0/869358024 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f296415e9b0 msgr2=0x7f29640cd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:20.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.819+0000 7f2963fff640 1 --2- 172.21.15.32:0/869358024 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f296415e9b0 0x7f29640cd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:20.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.819+0000 7f2963fff640 1 -- 172.21.15.32:0/869358024 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f29640d4210 msgr2=0x7f2964155be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:20.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.819+0000 7f2963fff640 1 --2- 172.21.15.32:0/869358024 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f29640d4210 0x7f2964155be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:20.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.819+0000 7f2963fff640 1 -- 172.21.15.32:0/869358024 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29640cc8c0 con 0x7f29640ba0e0 2024-04-24T23:01:20.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.819+0000 7f2968ebf640 1 --2- 172.21.15.32:0/869358024 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f296415e9b0 0x7f29640cd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:01:20.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.819+0000 7f2963fff640 1 --2- 172.21.15.32:0/869358024 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29640ba0e0 0x7f29640cc230 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f295400e870 tx=0x7f295400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:20.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.819+0000 7f2960ff9640 1 -- 172.21.15.32:0/869358024 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2954017070 con 0x7f29640ba0e0 2024-04-24T23:01:20.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.819+0000 7f2960ff9640 1 -- 172.21.15.32:0/869358024 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f295400c940 con 0x7f29640ba0e0 2024-04-24T23:01:20.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.819+0000 7f296a922640 1 -- 172.21.15.32:0/869358024 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29640ccba0 con 0x7f29640ba0e0 2024-04-24T23:01:20.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.819+0000 7f2960ff9640 1 -- 172.21.15.32:0/869358024 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f295400cd30 con 0x7f29640ba0e0 2024-04-24T23:01:20.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.819+0000 7f296a922640 1 -- 172.21.15.32:0/869358024 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29640c96e0 con 0x7f29640ba0e0 2024-04-24T23:01:20.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.820+0000 7f296a922640 1 -- 172.21.15.32:0/869358024 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2964068430 con 0x7f29640ba0e0 2024-04-24T23:01:20.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.821+0000 7f2960ff9640 1 -- 172.21.15.32:0/869358024 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f295401a070 con 0x7f29640ba0e0 2024-04-24T23:01:20.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.822+0000 7f2960ff9640 1 --2- 172.21.15.32:0/869358024 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f295006be30 0x7f295006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:20.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.822+0000 7f2960ff9640 1 -- 172.21.15.32:0/869358024 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 4943+0+0 (secure 0 0 0) 0x7f295408b1c0 con 0x7f29640ba0e0 2024-04-24T23:01:20.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.822+0000 7f29637fe640 1 --2- 172.21.15.32:0/869358024 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f295006be30 0x7f295006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:20.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.823+0000 7f29637fe640 1 --2- 172.21.15.32:0/869358024 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f295006be30 0x7f295006e2f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2958034000 tx=0x7f295802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:20.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:20.826+0000 7f2960ff9640 1 -- 172.21.15.32:0/869358024 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2954059ee0 con 0x7f29640ba0e0 2024-04-24T23:01:21.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.021+0000 7f296a922640 1 -- 172.21.15.32:0/869358024 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f29640b8c40 con 0x7f29640ba0e0 2024-04-24T23:01:21.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.651+0000 7f2960ff9640 1 -- 172.21.15.32:0/869358024 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v61) ==== 97+0+0 (secure 0 0 0) 0x7f29540597e0 con 0x7f29640ba0e0 2024-04-24T23:01:21.652 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blocklist entries 2024-04-24T23:01:21.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.654+0000 7f296a922640 1 -- 172.21.15.32:0/869358024 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f295006be30 msgr2=0x7f295006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:21.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.654+0000 7f296a922640 1 --2- 172.21.15.32:0/869358024 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f295006be30 0x7f295006e2f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2958034000 tx=0x7f295802e000 comp rx=0 tx=0).stop 2024-04-24T23:01:21.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.654+0000 7f296a922640 1 -- 172.21.15.32:0/869358024 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29640ba0e0 msgr2=0x7f29640cc230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:21.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.654+0000 7f296a922640 1 --2- 172.21.15.32:0/869358024 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29640ba0e0 0x7f29640cc230 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f295400e870 tx=0x7f295400ed30 comp rx=0 tx=0).stop 2024-04-24T23:01:21.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.654+0000 7f296a922640 1 -- 172.21.15.32:0/869358024 shutdown_connections 2024-04-24T23:01:21.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.654+0000 7f296a922640 1 --2- 172.21.15.32:0/869358024 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f295006be30 0x7f295006e2f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:21.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.654+0000 7f296a922640 1 --2- 172.21.15.32:0/869358024 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f296415e9b0 0x7f29640cd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:21.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.654+0000 7f296a922640 1 --2- 172.21.15.32:0/869358024 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f29640d4210 0x7f2964155be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:21.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.654+0000 7f296a922640 1 --2- 172.21.15.32:0/869358024 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29640ba0e0 0x7f29640cc230 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:21.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.654+0000 7f296a922640 1 -- 172.21.15.32:0/869358024 >> 172.21.15.32:0/869358024 conn(0x7f29640d11d0 msgr2=0x7f29640b8e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:21.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.654+0000 7f296a922640 1 -- 172.21.15.32:0/869358024 shutdown_connections 2024-04-24T23:01:21.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.654+0000 7f296a922640 1 -- 172.21.15.32:0/869358024 wait complete. 2024-04-24T23:01:21.669 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-24T23:01:21.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.773+0000 7f5fd484d640 1 Processor -- start 2024-04-24T23:01:21.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.773+0000 7f5fd484d640 1 -- start start 2024-04-24T23:01:21.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.774+0000 7f5fd484d640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5fd00b0b80 0x7f5fd006c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:21.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.774+0000 7f5fd484d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fd00b0770 con 0x7f5fd0074210 2024-04-24T23:01:21.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.774+0000 7f5fd484d640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fd00b1010 con 0x7f5fd00ae140 2024-04-24T23:01:21.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.774+0000 7f5fd484d640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f5fd00b1c20 con 0x7f5fd00b0b80 2024-04-24T23:01:21.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.774+0000 7f5fce59c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5fd0074210 0x7f5fd00745f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43004/0 (socket says 172.21.15.32:43004) 2024-04-24T23:01:21.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.774+0000 7f5fce59c640 1 -- 172.21.15.32:0/1677152476 learned_addr learned my addr 172.21.15.32:0/1677152476 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:01:21.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.774+0000 7f5fcdd9b640 1 --2- 172.21.15.32:0/1677152476 >> v2:172.21.15.121:3300/0 conn(0x7f5fd00b0b80 0x7f5fd006c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:21.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.775+0000 7f5fcd59a640 1 -- 172.21.15.32:0/1677152476 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3554108691 0 0) 0x7f5fd00b0770 con 0x7f5fd0074210 2024-04-24T23:01:21.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.775+0000 7f5fcd59a640 1 -- 172.21.15.32:0/1677152476 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5fb8003520 con 0x7f5fd0074210 2024-04-24T23:01:21.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.775+0000 7f5fcd59a640 1 -- 172.21.15.32:0/1677152476 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2843112864 0 0) 0x7f5fb8003520 con 0x7f5fd0074210 2024-04-24T23:01:21.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.775+0000 7f5fcdd9b640 1 -- 172.21.15.32:0/1677152476 >> v1:172.21.15.121:6789/0 conn(0x7f5fd00ae140 legacy=0x7f5fd00b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:21.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.775+0000 7f5fcdd9b640 1 -- 172.21.15.32:0/1677152476 >> v1:172.21.15.32:6789/0 conn(0x7f5fd0074210 legacy=0x7f5fd00745f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:21.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.775+0000 7f5fcdd9b640 1 -- 172.21.15.32:0/1677152476 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fd00b12f0 con 0x7f5fd00b0b80 2024-04-24T23:01:21.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.776+0000 7f5fcdd9b640 1 --2- 172.21.15.32:0/1677152476 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5fd00b0b80 0x7f5fd006c480 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5fc40097a0 tx=0x7f5fc402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ed6a4ec2072ce47 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:21.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.776+0000 7f5fcd59a640 1 -- 172.21.15.32:0/1677152476 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc4037070 con 0x7f5fd00b0b80 2024-04-24T23:01:21.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.776+0000 7f5fcd59a640 1 -- 172.21.15.32:0/1677152476 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fc402ac50 con 0x7f5fd00b0b80 2024-04-24T23:01:21.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.776+0000 7f5fcd59a640 1 -- 172.21.15.32:0/1677152476 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc8001670 con 0x7f5fd00b0b80 2024-04-24T23:01:21.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.776+0000 7f5fd484d640 1 -- 172.21.15.32:0/1677152476 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5fd00b0b80 msgr2=0x7f5fd006c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:21.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.776+0000 7f5fd484d640 1 --2- 172.21.15.32:0/1677152476 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5fd00b0b80 0x7f5fd006c480 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5fc40097a0 tx=0x7f5fc402a1c0 comp rx=0 tx=0).stop 2024-04-24T23:01:21.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.776+0000 7f5fd484d640 1 -- 172.21.15.32:0/1677152476 shutdown_connections 2024-04-24T23:01:21.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.776+0000 7f5fd484d640 1 --2- 172.21.15.32:0/1677152476 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5fd00b0b80 0x7f5fd006c480 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:21.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.776+0000 7f5fd484d640 1 -- 172.21.15.32:0/1677152476 >> 172.21.15.32:0/1677152476 conn(0x7f5fd00b2e60 msgr2=0x7f5fd00b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:21.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.776+0000 7f5fd484d640 1 -- 172.21.15.32:0/1677152476 shutdown_connections 2024-04-24T23:01:21.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.776+0000 7f5fd484d640 1 -- 172.21.15.32:0/1677152476 wait complete. 2024-04-24T23:01:21.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.778+0000 7f5fd484d640 1 Processor -- start 2024-04-24T23:01:21.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.778+0000 7f5fd484d640 1 -- start start 2024-04-24T23:01:21.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.778+0000 7f5fd484d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5fd0074210 0x7f5fd015b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:21.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.778+0000 7f5fd484d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fd00ae140 0x7f5fd00ce090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:21.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.778+0000 7f5fd484d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5fd00cd590 0x7f5fd00c8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:21.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.778+0000 7f5fd484d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd00b3ba0 con 0x7f5fd00ae140 2024-04-24T23:01:21.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.778+0000 7f5fd484d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd00b18b0 con 0x7f5fd00cd590 2024-04-24T23:01:21.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.778+0000 7f5fd484d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f5fd00b3d20 con 0x7f5fd0074210 2024-04-24T23:01:21.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.779+0000 7f5fcdd9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fd00ae140 0x7f5fd00ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:21.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.779+0000 7f5fcdd9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fd00ae140 0x7f5fd00ce090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41642/0 (socket says 172.21.15.32:41642) 2024-04-24T23:01:21.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.779+0000 7f5fcdd9b640 1 -- 172.21.15.32:0/1570587434 learned_addr learned my addr 172.21.15.32:0/1570587434 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:21.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.779+0000 7f5fce59c640 1 --2- 172.21.15.32:0/1570587434 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5fd0074210 0x7f5fd015b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:21.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.779+0000 7f5fced9d640 1 --2- 172.21.15.32:0/1570587434 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5fd00cd590 0x7f5fd00c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:21.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.779+0000 7f5fcdd9b640 1 -- 172.21.15.32:0/1570587434 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5fd0074210 msgr2=0x7f5fd015b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:21.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.779+0000 7f5fcdd9b640 1 --2- 172.21.15.32:0/1570587434 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5fd0074210 0x7f5fd015b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:21.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.779+0000 7f5fcdd9b640 1 -- 172.21.15.32:0/1570587434 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5fd00cd590 msgr2=0x7f5fd00c8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:21.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.779+0000 7f5fcdd9b640 1 --2- 172.21.15.32:0/1570587434 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5fd00cd590 0x7f5fd00c8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:21.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.779+0000 7f5fcdd9b640 1 -- 172.21.15.32:0/1570587434 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fd00cc4f0 con 0x7f5fd00ae140 2024-04-24T23:01:21.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.779+0000 7f5fced9d640 1 --2- 172.21.15.32:0/1570587434 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5fd00cd590 0x7f5fd00c8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:01:21.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.779+0000 7f5fcdd9b640 1 --2- 172.21.15.32:0/1570587434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fd00ae140 0x7f5fd00ce090 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4035040 tx=0x7f5fc40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:21.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.779+0000 7f5fbeffd640 1 -- 172.21.15.32:0/1570587434 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc4041070 con 0x7f5fd00ae140 2024-04-24T23:01:21.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.779+0000 7f5fbeffd640 1 -- 172.21.15.32:0/1570587434 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fc403c460 con 0x7f5fd00ae140 2024-04-24T23:01:21.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.779+0000 7f5fd484d640 1 -- 172.21.15.32:0/1570587434 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fd00cc7d0 con 0x7f5fd00ae140 2024-04-24T23:01:21.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.779+0000 7f5fbeffd640 1 -- 172.21.15.32:0/1570587434 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc4037040 con 0x7f5fd00ae140 2024-04-24T23:01:21.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.779+0000 7f5fd484d640 1 -- 172.21.15.32:0/1570587434 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fd00ccd90 con 0x7f5fd00ae140 2024-04-24T23:01:21.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.780+0000 7f5fd484d640 1 -- 172.21.15.32:0/1570587434 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fd0068430 con 0x7f5fd00ae140 2024-04-24T23:01:21.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.781+0000 7f5fbeffd640 1 -- 172.21.15.32:0/1570587434 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5fc4044070 con 0x7f5fd00ae140 2024-04-24T23:01:21.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.781+0000 7f5fbeffd640 1 --2- 172.21.15.32:0/1570587434 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5fb006be60 0x7f5fb006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:21.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.782+0000 7f5fbeffd640 1 -- 172.21.15.32:0/1570587434 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 4857+0+0 (secure 0 0 0) 0x7f5fc40ab250 con 0x7f5fd00ae140 2024-04-24T23:01:21.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.782+0000 7f5fce59c640 1 --2- 172.21.15.32:0/1570587434 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5fb006be60 0x7f5fb006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:21.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.782+0000 7f5fce59c640 1 --2- 172.21.15.32:0/1570587434 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5fb006be60 0x7f5fb006e320 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8001eb0 tx=0x7f5fc8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:21.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:21.786+0000 7f5fbeffd640 1 -- 172.21.15.32:0/1570587434 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5fc4079e40 con 0x7f5fd00ae140 2024-04-24T23:01:22.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.041+0000 7f5fd484d640 1 -- 172.21.15.32:0/1570587434 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f5fd006b220 con 0x7f5fd00ae140 2024-04-24T23:01:22.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.042+0000 7f5fbeffd640 1 -- 172.21.15.32:0/1570587434 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) ==== 80+0+5304 (secure 0 0 0) 0x7f5fc4079740 con 0x7f5fd00ae140 2024-04-24T23:01:22.043 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:01:22.043 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:01:22.043 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2024-04-24T23:01:22.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:22.044 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2024-04-24T23:01:22.044 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmXRdxAxAA/EYEn1C38XhG7DC+0X4y7A==", 2024-04-24T23:01:22.044 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:01:22.044 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:01:22.044 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-24T23:01:22.044 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-24T23:01:22.044 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:01:22.044 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:22.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:22.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:22.044 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2024-04-24T23:01:22.044 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmak7bDhAA0he+Jp/gkkrfBjYKSObjcw==", 2024-04-24T23:01:22.044 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:01:22.044 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:01:22.045 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-24T23:01:22.045 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-24T23:01:22.045 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:01:22.045 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:22.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:22.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:22.045 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.c", 2024-04-24T23:01:22.045 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmZXFdChAA8TwwkA5EIXHBzPsQSFvXcQ==", 2024-04-24T23:01:22.045 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:01:22.045 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:01:22.045 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-24T23:01:22.045 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-24T23:01:22.045 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:01:22.046 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:22.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:22.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:22.046 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.d", 2024-04-24T23:01:22.046 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmLbGuFRAAe91fZlulziWdWiDdLBLWNA==", 2024-04-24T23:01:22.046 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:01:22.046 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:01:22.046 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-24T23:01:22.046 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-24T23:01:22.046 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:01:22.046 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:22.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:22.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:22.047 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2024-04-24T23:01:22.047 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCfjilmhTwoOBAAig98xa3a2qnv4rriUO9VCQ==", 2024-04-24T23:01:22.047 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:01:22.047 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:01:22.047 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:01:22.047 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:01:22.047 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:22.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:22.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:22.047 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2024-04-24T23:01:22.047 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCgjilmwRaAAxAAUqDOyR9HV6DEnmbyn3WAjA==", 2024-04-24T23:01:22.047 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:01:22.047 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:01:22.047 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:01:22.048 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:01:22.048 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:22.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:22.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:22.048 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2024-04-24T23:01:22.048 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCgjilmDl2qDBAAF1bhTOaSOc612wmFRd8+XA==", 2024-04-24T23:01:22.048 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:01:22.048 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:01:22.048 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:01:22.048 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:01:22.048 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:22.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:22.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:22.048 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2024-04-24T23:01:22.049 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCgjilmAxHwFRAA9a0v/YwngRxq1vmMRsmEkQ==", 2024-04-24T23:01:22.049 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:01:22.049 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:01:22.049 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:01:22.049 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:01:22.049 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:22.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:22.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:22.049 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2024-04-24T23:01:22.049 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCjjilmDPmJIxAASZlEo1ZJHOhIAFGukIj1tQ==", 2024-04-24T23:01:22.049 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:01:22.049 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:01:22.049 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:01:22.050 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:01:22.050 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:22.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:22.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:22.050 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2024-04-24T23:01:22.050 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCjjilmgcpuKhAAedSq0ubNB30s3bz0fY38hw==", 2024-04-24T23:01:22.050 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:01:22.050 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:01:22.050 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:01:22.050 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:01:22.050 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:22.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:22.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:22.050 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2024-04-24T23:01:22.050 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCjjilmFv2VMxAAM+mUc4I0sjawy2uwuUEKYg==", 2024-04-24T23:01:22.051 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:01:22.051 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:01:22.051 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:01:22.051 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:01:22.051 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:22.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:22.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:22.051 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2024-04-24T23:01:22.051 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCkjilmLvEbARAAnaOvQ5Ee0clN/A9SVoKuBQ==", 2024-04-24T23:01:22.051 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:01:22.051 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:01:22.051 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:01:22.051 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:01:22.051 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:22.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:22.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:22.052 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2024-04-24T23:01:22.052 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilm5oeiGRAAdLXwwqrYhV3HHo7Dw/2ZqQ==", 2024-04-24T23:01:22.052 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:01:22.052 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:01:22.052 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2024-04-24T23:01:22.052 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-24T23:01:22.052 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:22.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:22.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:22.053 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2024-04-24T23:01:22.053 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmyWBDIhAARtVP4Unk3HHlQYZ2QZ3GzQ==", 2024-04-24T23:01:22.053 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:01:22.053 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:01:22.053 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-24T23:01:22.053 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-24T23:01:22.053 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-24T23:01:22.053 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:22.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:22.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:22.053 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.2", 2024-04-24T23:01:22.053 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilm4QElHxAAXgE316aRCosxX7XER5gtKQ==", 2024-04-24T23:01:22.053 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:01:22.053 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:01:22.054 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-24T23:01:22.054 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-24T23:01:22.054 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-24T23:01:22.054 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:22.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:22.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:22.054 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.3", 2024-04-24T23:01:22.054 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmMZihJxAAoWaAAFY3vVfZnC4/GoT0+g==", 2024-04-24T23:01:22.054 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:01:22.054 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:01:22.054 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-24T23:01:22.054 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-24T23:01:22.054 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-24T23:01:22.054 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:22.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:22.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:22.055 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2024-04-24T23:01:22.055 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCdjilmdVXeIxAAwGGaAfshx4pHcoc5+3kRww==", 2024-04-24T23:01:22.055 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:01:22.055 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2024-04-24T23:01:22.055 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-24T23:01:22.055 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-24T23:01:22.055 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:01:22.055 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:22.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:22.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:22.055 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2024-04-24T23:01:22.055 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmn3FJABAAGnvNAoNLzK20cQF6beihmw==", 2024-04-24T23:01:22.055 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:01:22.056 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2024-04-24T23:01:22.056 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2024-04-24T23:01:22.056 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:01:22.056 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:22.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:01:22.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:22.056 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2024-04-24T23:01:22.056 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCdjilmMqs2OBAAOe3bLpnCq+HK2EbeXnwY8Q==", 2024-04-24T23:01:22.056 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:01:22.056 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2024-04-24T23:01:22.056 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2024-04-24T23:01:22.056 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:01:22.056 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:22.056 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:22.057 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:01:22.057 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:01:22.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.043+0000 7f5fd484d640 1 -- 172.21.15.32:0/1570587434 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5fb006be60 msgr2=0x7f5fb006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:22.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.043+0000 7f5fd484d640 1 --2- 172.21.15.32:0/1570587434 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5fb006be60 0x7f5fb006e320 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8001eb0 tx=0x7f5fc8009040 comp rx=0 tx=0).stop 2024-04-24T23:01:22.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.043+0000 7f5fd484d640 1 -- 172.21.15.32:0/1570587434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fd00ae140 msgr2=0x7f5fd00ce090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:22.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.043+0000 7f5fd484d640 1 --2- 172.21.15.32:0/1570587434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fd00ae140 0x7f5fd00ce090 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4035040 tx=0x7f5fc40046c0 comp rx=0 tx=0).stop 2024-04-24T23:01:22.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.043+0000 7f5fd484d640 1 -- 172.21.15.32:0/1570587434 shutdown_connections 2024-04-24T23:01:22.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.043+0000 7f5fd484d640 1 --2- 172.21.15.32:0/1570587434 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5fb006be60 0x7f5fb006e320 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:22.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.043+0000 7f5fd484d640 1 --2- 172.21.15.32:0/1570587434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fd00ae140 0x7f5fd00ce090 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:22.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.043+0000 7f5fd484d640 1 --2- 172.21.15.32:0/1570587434 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5fd00cd590 0x7f5fd00c8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:22.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.043+0000 7f5fd484d640 1 --2- 172.21.15.32:0/1570587434 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5fd0074210 0x7f5fd015b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:22.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.043+0000 7f5fd484d640 1 -- 172.21.15.32:0/1570587434 >> 172.21.15.32:0/1570587434 conn(0x7f5fd00b2e60 msgr2=0x7f5fd006da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:22.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.044+0000 7f5fd484d640 1 -- 172.21.15.32:0/1570587434 shutdown_connections 2024-04-24T23:01:22.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.044+0000 7f5fd484d640 1 -- 172.21.15.32:0/1570587434 wait complete. 2024-04-24T23:01:22.059 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:01:22.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.159+0000 7ff6e42d4640 1 Processor -- start 2024-04-24T23:01:22.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.159+0000 7ff6e42d4640 1 -- start start 2024-04-24T23:01:22.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.160+0000 7ff6e42d4640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff6dc0af890 0x7ff6dc1670f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:22.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.160+0000 7ff6e42d4640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6dc0b6430 con 0x7ff6dc06c5f0 2024-04-24T23:01:22.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.160+0000 7ff6e42d4640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6dc156f80 con 0x7ff6dc0b44d0 2024-04-24T23:01:22.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.160+0000 7ff6e42d4640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7ff6dc0b6150 con 0x7ff6dc0af890 2024-04-24T23:01:22.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.160+0000 7ff6e2070640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff6dc06c5f0 0x7ff6dc06c9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43008/0 (socket says 172.21.15.32:43008) 2024-04-24T23:01:22.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.160+0000 7ff6e2070640 1 -- 172.21.15.32:0/567926236 learned_addr learned my addr 172.21.15.32:0/567926236 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:01:22.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.161+0000 7ff6e2871640 1 --2- 172.21.15.32:0/567926236 >> v2:172.21.15.121:3300/0 conn(0x7ff6dc0af890 0x7ff6dc1670f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:22.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.161+0000 7ff6e106e640 1 -- 172.21.15.32:0/567926236 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 764893348 0 0) 0x7ff6dc0b6430 con 0x7ff6dc06c5f0 2024-04-24T23:01:22.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.161+0000 7ff6e106e640 1 -- 172.21.15.32:0/567926236 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff6c4003520 con 0x7ff6dc06c5f0 2024-04-24T23:01:22.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.161+0000 7ff6e106e640 1 -- 172.21.15.32:0/567926236 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3610880198 0 0) 0x7ff6c4003520 con 0x7ff6dc06c5f0 2024-04-24T23:01:22.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.161+0000 7ff6e106e640 1 -- 172.21.15.32:0/567926236 >> v2:172.21.15.121:3300/0 conn(0x7ff6dc0af890 msgr2=0x7ff6dc1670f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:22.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.161+0000 7ff6e106e640 1 --2- 172.21.15.32:0/567926236 >> v2:172.21.15.121:3300/0 conn(0x7ff6dc0af890 0x7ff6dc1670f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:22.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.161+0000 7ff6e106e640 1 -- 172.21.15.32:0/567926236 >> v1:172.21.15.121:6789/0 conn(0x7ff6dc0b44d0 legacy=0x7ff6dc0b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:22.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.161+0000 7ff6e106e640 1 -- 172.21.15.32:0/567926236 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6dc0b39f0 con 0x7ff6dc06c5f0 2024-04-24T23:01:22.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.161+0000 7ff6e106e640 1 -- 172.21.15.32:0/567926236 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7ff6d4001670 con 0x7ff6dc06c5f0 2024-04-24T23:01:22.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.161+0000 7ff6e106e640 1 -- 172.21.15.32:0/567926236 >> v1:172.21.15.32:6789/0 conn(0x7ff6dc06c5f0 legacy=0x7ff6dc06c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:22.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.161+0000 7ff6e106e640 1 --2- 172.21.15.32:0/567926236 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6c40039b0 0x7ff6c4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:22.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.161+0000 7ff6e106e640 1 --2- 172.21.15.32:0/567926236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6c40043b0 0x7ff6c4004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:22.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.161+0000 7ff6e106e640 1 --2- 172.21.15.32:0/567926236 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6c4005080 0x7ff6c400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:22.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.161+0000 7ff6e106e640 1 -- 172.21.15.32:0/567926236 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff6dc0b6150 con 0x7ff6c40043b0 2024-04-24T23:01:22.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.161+0000 7ff6e106e640 1 -- 172.21.15.32:0/567926236 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c400bfe0 con 0x7ff6c4005080 2024-04-24T23:01:22.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.161+0000 7ff6e106e640 1 -- 172.21.15.32:0/567926236 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff6c400c1e0 con 0x7ff6c40039b0 2024-04-24T23:01:22.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.161+0000 7ff6e186f640 1 --2- 172.21.15.32:0/567926236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6c40043b0 0x7ff6c4004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:22.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.161+0000 7ff6e2070640 1 --2- 172.21.15.32:0/567926236 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6c40039b0 0x7ff6c4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:22.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.161+0000 7ff6e2871640 1 --2- 172.21.15.32:0/567926236 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6c4005080 0x7ff6c400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:22.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.161+0000 7ff6e186f640 1 -- 172.21.15.32:0/567926236 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6c40039b0 msgr2=0x7ff6c4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:22.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.161+0000 7ff6e186f640 1 --2- 172.21.15.32:0/567926236 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6c40039b0 0x7ff6c4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:22.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.161+0000 7ff6e186f640 1 -- 172.21.15.32:0/567926236 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6c4005080 msgr2=0x7ff6c400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:22.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.161+0000 7ff6e186f640 1 --2- 172.21.15.32:0/567926236 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6c4005080 0x7ff6c400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:22.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.161+0000 7ff6e186f640 1 -- 172.21.15.32:0/567926236 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6c4003520 con 0x7ff6c40043b0 2024-04-24T23:01:22.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.161+0000 7ff6e2871640 1 -- 172.21.15.32:0/567926236 reap_dead start 2024-04-24T23:01:22.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.161+0000 7ff6e2070640 1 --2- 172.21.15.32:0/567926236 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6c40039b0 0x7ff6c4003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:01:22.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.162+0000 7ff6e186f640 1 --2- 172.21.15.32:0/567926236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6c40043b0 0x7ff6c4004880 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc0b33c0 tx=0x7ff6c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9feeeb7efd947f4e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:22.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.162+0000 7ff6e106e640 1 -- 172.21.15.32:0/567926236 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d0037070 con 0x7ff6c40043b0 2024-04-24T23:01:22.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.162+0000 7ff6e106e640 1 -- 172.21.15.32:0/567926236 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6d002ab90 con 0x7ff6c40043b0 2024-04-24T23:01:22.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.162+0000 7ff6e106e640 1 -- 172.21.15.32:0/567926236 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d0001430 con 0x7ff6c40043b0 2024-04-24T23:01:22.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.163+0000 7ff6e42d4640 1 -- 172.21.15.32:0/567926236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6c40043b0 msgr2=0x7ff6c4004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:22.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.163+0000 7ff6e42d4640 1 --2- 172.21.15.32:0/567926236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6c40043b0 0x7ff6c4004880 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc0b33c0 tx=0x7ff6c4005830 comp rx=0 tx=0).stop 2024-04-24T23:01:22.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.163+0000 7ff6e42d4640 1 -- 172.21.15.32:0/567926236 shutdown_connections 2024-04-24T23:01:22.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.163+0000 7ff6e42d4640 1 --2- 172.21.15.32:0/567926236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6c40043b0 0x7ff6c4004880 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:22.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.163+0000 7ff6e42d4640 1 -- 172.21.15.32:0/567926236 >> 172.21.15.32:0/567926236 conn(0x7ff6dc06b110 msgr2=0x7ff6dc06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:22.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.163+0000 7ff6e42d4640 1 -- 172.21.15.32:0/567926236 shutdown_connections 2024-04-24T23:01:22.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.163+0000 7ff6e42d4640 1 -- 172.21.15.32:0/567926236 wait complete. 2024-04-24T23:01:22.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.164+0000 7ff6e42d4640 1 Processor -- start 2024-04-24T23:01:22.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.164+0000 7ff6e42d4640 1 -- start start 2024-04-24T23:01:22.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.165+0000 7ff6e42d4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6c40043b0 0x7ff6dc0ce910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:22.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.165+0000 7ff6e42d4640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6dc159d10 0x7ff6dc1527e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:22.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.165+0000 7ff6e42d4640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6dc0b0fb0 0x7ff6dc0b1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:22.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.165+0000 7ff6e42d4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff6dc06c9d0 con 0x7ff6c40043b0 2024-04-24T23:01:22.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.165+0000 7ff6e42d4640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff6dc0b3fb0 con 0x7ff6dc0b0fb0 2024-04-24T23:01:22.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.165+0000 7ff6e42d4640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff6dc06cb50 con 0x7ff6dc159d10 2024-04-24T23:01:22.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.165+0000 7ff6e2070640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6c40043b0 0x7ff6dc0ce910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:22.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.165+0000 7ff6e2070640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6c40043b0 0x7ff6dc0ce910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41662/0 (socket says 172.21.15.32:41662) 2024-04-24T23:01:22.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.165+0000 7ff6e2070640 1 -- 172.21.15.32:0/2055824692 learned_addr learned my addr 172.21.15.32:0/2055824692 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:22.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.165+0000 7ff6e2871640 1 --2- 172.21.15.32:0/2055824692 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6dc0b0fb0 0x7ff6dc0b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:22.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.165+0000 7ff6e186f640 1 --2- 172.21.15.32:0/2055824692 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6dc159d10 0x7ff6dc1527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:22.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.165+0000 7ff6e2070640 1 -- 172.21.15.32:0/2055824692 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6dc159d10 msgr2=0x7ff6dc1527e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:22.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.165+0000 7ff6e2070640 1 --2- 172.21.15.32:0/2055824692 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6dc159d10 0x7ff6dc1527e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:22.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.165+0000 7ff6e2070640 1 -- 172.21.15.32:0/2055824692 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6dc0b0fb0 msgr2=0x7ff6dc0b1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:22.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.165+0000 7ff6e2070640 1 --2- 172.21.15.32:0/2055824692 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6dc0b0fb0 0x7ff6dc0b1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:22.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.165+0000 7ff6e2070640 1 -- 172.21.15.32:0/2055824692 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6dc153030 con 0x7ff6c40043b0 2024-04-24T23:01:22.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.165+0000 7ff6e2070640 1 --2- 172.21.15.32:0/2055824692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6c40043b0 0x7ff6dc0ce910 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7ff6d400e730 tx=0x7ff6d400ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:22.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.165+0000 7ff6daffd640 1 -- 172.21.15.32:0/2055824692 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d400c660 con 0x7ff6c40043b0 2024-04-24T23:01:22.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.165+0000 7ff6daffd640 1 -- 172.21.15.32:0/2055824692 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6d4018070 con 0x7ff6c40043b0 2024-04-24T23:01:22.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.165+0000 7ff6e42d4640 1 -- 172.21.15.32:0/2055824692 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6dc153310 con 0x7ff6c40043b0 2024-04-24T23:01:22.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.166+0000 7ff6daffd640 1 -- 172.21.15.32:0/2055824692 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d400ca00 con 0x7ff6c40043b0 2024-04-24T23:01:22.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.166+0000 7ff6e42d4640 1 -- 172.21.15.32:0/2055824692 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6dc1538d0 con 0x7ff6c40043b0 2024-04-24T23:01:22.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.167+0000 7ff6e42d4640 1 -- 172.21.15.32:0/2055824692 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6dc068430 con 0x7ff6c40043b0 2024-04-24T23:01:22.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.167+0000 7ff6daffd640 1 -- 172.21.15.32:0/2055824692 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff6d400cba0 con 0x7ff6c40043b0 2024-04-24T23:01:22.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.167+0000 7ff6daffd640 1 --2- 172.21.15.32:0/2055824692 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff6c806be60 0x7ff6c806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:22.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.168+0000 7ff6daffd640 1 -- 172.21.15.32:0/2055824692 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 4857+0+0 (secure 0 0 0) 0x7ff6d408a580 con 0x7ff6c40043b0 2024-04-24T23:01:22.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.168+0000 7ff6e186f640 1 --2- 172.21.15.32:0/2055824692 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff6c806be60 0x7ff6c806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:22.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.168+0000 7ff6e186f640 1 --2- 172.21.15.32:0/2055824692 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff6c806be60 0x7ff6c806e320 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0034000 tx=0x7ff6d0009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:22.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.172+0000 7ff6daffd640 1 -- 172.21.15.32:0/2055824692 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff6d4059280 con 0x7ff6c40043b0 2024-04-24T23:01:22.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.355+0000 7ff6e42d4640 1 -- 172.21.15.32:0/2055824692 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff6dc0b5670 con 0x7ff6c40043b0 2024-04-24T23:01:22.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.356+0000 7ff6daffd640 1 -- 172.21.15.32:0/2055824692 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) ==== 74+0+12903 (secure 0 0 0) 0x7ff6d4058b80 con 0x7ff6c40043b0 2024-04-24T23:01:22.358 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:01:22.358 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":63,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:01:21.657781+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:01:22.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.358+0000 7ff6e42d4640 1 -- 172.21.15.32:0/2055824692 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff6c806be60 msgr2=0x7ff6c806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:22.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.358+0000 7ff6e42d4640 1 --2- 172.21.15.32:0/2055824692 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff6c806be60 0x7ff6c806e320 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0034000 tx=0x7ff6d0009030 comp rx=0 tx=0).stop 2024-04-24T23:01:22.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.358+0000 7ff6e42d4640 1 -- 172.21.15.32:0/2055824692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6c40043b0 msgr2=0x7ff6dc0ce910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:22.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.358+0000 7ff6e42d4640 1 --2- 172.21.15.32:0/2055824692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6c40043b0 0x7ff6dc0ce910 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7ff6d400e730 tx=0x7ff6d400ebf0 comp rx=0 tx=0).stop 2024-04-24T23:01:22.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.358+0000 7ff6e42d4640 1 -- 172.21.15.32:0/2055824692 shutdown_connections 2024-04-24T23:01:22.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.358+0000 7ff6e42d4640 1 --2- 172.21.15.32:0/2055824692 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff6c806be60 0x7ff6c806e320 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:22.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.358+0000 7ff6e42d4640 1 --2- 172.21.15.32:0/2055824692 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6dc0b0fb0 0x7ff6dc0b1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:22.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.358+0000 7ff6e42d4640 1 --2- 172.21.15.32:0/2055824692 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6dc159d10 0x7ff6dc1527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:22.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.358+0000 7ff6e42d4640 1 --2- 172.21.15.32:0/2055824692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6c40043b0 0x7ff6dc0ce910 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:22.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.358+0000 7ff6e42d4640 1 -- 172.21.15.32:0/2055824692 >> 172.21.15.32:0/2055824692 conn(0x7ff6dc06b110 msgr2=0x7ff6dc155580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:22.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.358+0000 7ff6e42d4640 1 -- 172.21.15.32:0/2055824692 shutdown_connections 2024-04-24T23:01:22.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.358+0000 7ff6e42d4640 1 -- 172.21.15.32:0/2055824692 wait complete. 2024-04-24T23:01:22.371 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:01:22.371 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:01:22.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.474+0000 7f9cddbe3640 1 Processor -- start 2024-04-24T23:01:22.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.474+0000 7f9cddbe3640 1 -- start start 2024-04-24T23:01:22.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.475+0000 7f9cddbe3640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9cd80b44d0 0x7f9cd80b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:22.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.475+0000 7f9cddbe3640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9cd80af890 0x7f9cd815cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:22.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.475+0000 7f9cddbe3640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9cd80b6430 con 0x7f9cd806c5f0 2024-04-24T23:01:22.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.475+0000 7f9cddbe3640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f9cd8156f80 con 0x7f9cd80b44d0 2024-04-24T23:01:22.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.475+0000 7f9cddbe3640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f9cd80b6150 con 0x7f9cd80af890 2024-04-24T23:01:22.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.475+0000 7f9cd6ffd640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9cd80b44d0 0x7f9cd80b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:22.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.475+0000 7f9cd7fff640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9cd80af890 0x7f9cd815cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:22.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.475+0000 7f9cd77fe640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f9cd806c5f0 0x7f9cd806c9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:52764/0 (socket says 172.21.15.32:52764) 2024-04-24T23:01:22.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.475+0000 7f9cd77fe640 1 -- 172.21.15.32:0/3142584097 learned_addr learned my addr 172.21.15.32:0/3142584097 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:01:22.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.476+0000 7f9cd67fc640 1 -- 172.21.15.32:0/3142584097 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2048790083 0 0) 0x7f9cd80b6430 con 0x7f9cd806c5f0 2024-04-24T23:01:22.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.476+0000 7f9cd67fc640 1 -- 172.21.15.32:0/3142584097 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9cc0003520 con 0x7f9cd806c5f0 2024-04-24T23:01:22.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.476+0000 7f9cd6ffd640 1 -- 172.21.15.32:0/3142584097 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9cd80af890 msgr2=0x7f9cd815cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:22.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.476+0000 7f9cd6ffd640 1 --2- 172.21.15.32:0/3142584097 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9cd80af890 0x7f9cd815cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:22.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.476+0000 7f9cd6ffd640 1 -- 172.21.15.32:0/3142584097 >> v1:172.21.15.121:6789/0 conn(0x7f9cd806c5f0 legacy=0x7f9cd806c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:22.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.476+0000 7f9cd6ffd640 1 -- 172.21.15.32:0/3142584097 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cd80b39f0 con 0x7f9cd80b44d0 2024-04-24T23:01:22.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.476+0000 7f9cd6ffd640 1 --2- 172.21.15.32:0/3142584097 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cd80b44d0 0x7f9cd80b5890 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc0097a0 tx=0x7f9ccc02a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e246288ccf0b5859 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:22.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.477+0000 7f9cd67fc640 1 -- 172.21.15.32:0/3142584097 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc037070 con 0x7f9cd80b44d0 2024-04-24T23:01:22.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.477+0000 7f9cd67fc640 1 -- 172.21.15.32:0/3142584097 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ccc02ae50 con 0x7f9cd80b44d0 2024-04-24T23:01:22.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.477+0000 7f9cd67fc640 1 -- 172.21.15.32:0/3142584097 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc032660 con 0x7f9cd80b44d0 2024-04-24T23:01:22.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.477+0000 7f9cddbe3640 1 -- 172.21.15.32:0/3142584097 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cd80b44d0 msgr2=0x7f9cd80b5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:22.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.477+0000 7f9cddbe3640 1 --2- 172.21.15.32:0/3142584097 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cd80b44d0 0x7f9cd80b5890 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc0097a0 tx=0x7f9ccc02a100 comp rx=0 tx=0).stop 2024-04-24T23:01:22.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.477+0000 7f9cddbe3640 1 -- 172.21.15.32:0/3142584097 shutdown_connections 2024-04-24T23:01:22.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.477+0000 7f9cddbe3640 1 --2- 172.21.15.32:0/3142584097 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9cd80af890 0x7f9cd815cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:22.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.477+0000 7f9cddbe3640 1 --2- 172.21.15.32:0/3142584097 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cd80b44d0 0x7f9cd80b5890 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:22.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.477+0000 7f9cddbe3640 1 -- 172.21.15.32:0/3142584097 >> 172.21.15.32:0/3142584097 conn(0x7f9cd806b110 msgr2=0x7f9cd806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:22.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.477+0000 7f9cddbe3640 1 -- 172.21.15.32:0/3142584097 shutdown_connections 2024-04-24T23:01:22.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.477+0000 7f9cddbe3640 1 -- 172.21.15.32:0/3142584097 wait complete. 2024-04-24T23:01:22.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.478+0000 7f9cddbe3640 1 Processor -- start 2024-04-24T23:01:22.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.479+0000 7f9cddbe3640 1 -- start start 2024-04-24T23:01:22.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.479+0000 7f9cddbe3640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9cd806c5f0 0x7f9cd814a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:22.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.479+0000 7f9cddbe3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cd80af890 0x7f9cd815ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:22.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.479+0000 7f9cddbe3640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cd80b44d0 0x7f9cd815f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:22.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.479+0000 7f9cddbe3640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd80b5dd0 con 0x7f9cd80af890 2024-04-24T23:01:22.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.479+0000 7f9cddbe3640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd80b3fb0 con 0x7f9cd80b44d0 2024-04-24T23:01:22.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.479+0000 7f9cddbe3640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f9cd815d000 con 0x7f9cd806c5f0 2024-04-24T23:01:22.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.479+0000 7f9cd6ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cd80af890 0x7f9cd815ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:22.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.479+0000 7f9cd6ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cd80af890 0x7f9cd815ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41678/0 (socket says 172.21.15.32:41678) 2024-04-24T23:01:22.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.479+0000 7f9cd6ffd640 1 -- 172.21.15.32:0/2300832647 learned_addr learned my addr 172.21.15.32:0/2300832647 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:22.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.480+0000 7f9cd7fff640 1 --2- 172.21.15.32:0/2300832647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cd80b44d0 0x7f9cd815f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:22.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.480+0000 7f9cd77fe640 1 --2- 172.21.15.32:0/2300832647 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9cd806c5f0 0x7f9cd814a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:22.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.480+0000 7f9cd6ffd640 1 -- 172.21.15.32:0/2300832647 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9cd806c5f0 msgr2=0x7f9cd814a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:22.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.480+0000 7f9cd6ffd640 1 --2- 172.21.15.32:0/2300832647 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9cd806c5f0 0x7f9cd814a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:22.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.480+0000 7f9cd6ffd640 1 -- 172.21.15.32:0/2300832647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cd80b44d0 msgr2=0x7f9cd815f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:22.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.480+0000 7f9cd6ffd640 1 --2- 172.21.15.32:0/2300832647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cd80b44d0 0x7f9cd815f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:22.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.480+0000 7f9cd6ffd640 1 -- 172.21.15.32:0/2300832647 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cd814ba80 con 0x7f9cd80af890 2024-04-24T23:01:22.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.480+0000 7f9cd6ffd640 1 --2- 172.21.15.32:0/2300832647 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cd80af890 0x7f9cd815ecb0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc035040 tx=0x7f9ccc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:22.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.480+0000 7f9cbffff640 1 -- 172.21.15.32:0/2300832647 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc041070 con 0x7f9cd80af890 2024-04-24T23:01:22.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.480+0000 7f9cbffff640 1 -- 172.21.15.32:0/2300832647 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ccc03c460 con 0x7f9cd80af890 2024-04-24T23:01:22.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.480+0000 7f9cddbe3640 1 -- 172.21.15.32:0/2300832647 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cd814bd60 con 0x7f9cd80af890 2024-04-24T23:01:22.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.480+0000 7f9cbffff640 1 -- 172.21.15.32:0/2300832647 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc037040 con 0x7f9cd80af890 2024-04-24T23:01:22.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.480+0000 7f9cddbe3640 1 -- 172.21.15.32:0/2300832647 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cd8150880 con 0x7f9cd80af890 2024-04-24T23:01:22.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.481+0000 7f9cddbe3640 1 -- 172.21.15.32:0/2300832647 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cd8068430 con 0x7f9cd80af890 2024-04-24T23:01:22.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.482+0000 7f9cbffff640 1 -- 172.21.15.32:0/2300832647 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9ccc03a070 con 0x7f9cd80af890 2024-04-24T23:01:22.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.482+0000 7f9cbffff640 1 --2- 172.21.15.32:0/2300832647 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9cc406be60 0x7f9cc406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:22.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.482+0000 7f9cbffff640 1 -- 172.21.15.32:0/2300832647 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 4857+0+0 (secure 0 0 0) 0x7f9ccc0aac70 con 0x7f9cd80af890 2024-04-24T23:01:22.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.482+0000 7f9cd77fe640 1 --2- 172.21.15.32:0/2300832647 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9cc406be60 0x7f9cc406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:22.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.483+0000 7f9cd77fe640 1 --2- 172.21.15.32:0/2300832647 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9cc406be60 0x7f9cc406e320 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8001200 tx=0x7f9cc8000bc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:22.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.486+0000 7f9cbffff640 1 -- 172.21.15.32:0/2300832647 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9ccc079320 con 0x7f9cd80af890 2024-04-24T23:01:22.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.669+0000 7f9cddbe3640 1 -- 172.21.15.32:0/2300832647 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9cd80b5670 con 0x7f9cd80af890 2024-04-24T23:01:22.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.670+0000 7f9cbffff640 1 -- 172.21.15.32:0/2300832647 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v63) ==== 93+0+10 (secure 0 0 0) 0x7f9ccc07d620 con 0x7f9cd80af890 2024-04-24T23:01:22.672 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T23:01:22.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.672+0000 7f9cddbe3640 1 -- 172.21.15.32:0/2300832647 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9cc406be60 msgr2=0x7f9cc406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:22.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.672+0000 7f9cddbe3640 1 --2- 172.21.15.32:0/2300832647 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9cc406be60 0x7f9cc406e320 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8001200 tx=0x7f9cc8000bc0 comp rx=0 tx=0).stop 2024-04-24T23:01:22.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.672+0000 7f9cddbe3640 1 -- 172.21.15.32:0/2300832647 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cd80af890 msgr2=0x7f9cd815ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:22.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.672+0000 7f9cddbe3640 1 --2- 172.21.15.32:0/2300832647 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cd80af890 0x7f9cd815ecb0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc035040 tx=0x7f9ccc0046c0 comp rx=0 tx=0).stop 2024-04-24T23:01:22.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.672+0000 7f9cddbe3640 1 -- 172.21.15.32:0/2300832647 shutdown_connections 2024-04-24T23:01:22.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.672+0000 7f9cddbe3640 1 --2- 172.21.15.32:0/2300832647 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9cc406be60 0x7f9cc406e320 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:22.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.672+0000 7f9cddbe3640 1 --2- 172.21.15.32:0/2300832647 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cd80af890 0x7f9cd815ecb0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:22.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.672+0000 7f9cddbe3640 1 --2- 172.21.15.32:0/2300832647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cd80b44d0 0x7f9cd815f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:22.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.672+0000 7f9cddbe3640 1 --2- 172.21.15.32:0/2300832647 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9cd806c5f0 0x7f9cd814a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:22.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.672+0000 7f9cddbe3640 1 -- 172.21.15.32:0/2300832647 >> 172.21.15.32:0/2300832647 conn(0x7f9cd806b110 msgr2=0x7f9cd80d0190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:22.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.672+0000 7f9cddbe3640 1 -- 172.21.15.32:0/2300832647 shutdown_connections 2024-04-24T23:01:22.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.672+0000 7f9cddbe3640 1 -- 172.21.15.32:0/2300832647 wait complete. 2024-04-24T23:01:22.685 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-24T23:01:22.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.784+0000 7f160ac2a640 1 Processor -- start 2024-04-24T23:01:22.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.784+0000 7f160ac2a640 1 -- start start 2024-04-24T23:01:22.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.785+0000 7f160ac2a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f160415e9b0 0x7f16040d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:22.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.785+0000 7f160ac2a640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f160406b820 0x7f16040b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:22.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.785+0000 7f160ac2a640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16040d3170 con 0x7f16040d4210 2024-04-24T23:01:22.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.785+0000 7f160ac2a640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f16040d3450 con 0x7f160415e9b0 2024-04-24T23:01:22.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.785+0000 7f160ac2a640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f16040d3730 con 0x7f160406b820 2024-04-24T23:01:22.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.785+0000 7f16089c6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f160415e9b0 0x7f16040d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:22.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.785+0000 7f16089c6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f160415e9b0 0x7f16040d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41686/0 (socket says 172.21.15.32:41686) 2024-04-24T23:01:22.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.785+0000 7f16089c6640 1 -- 172.21.15.32:0/3265726449 learned_addr learned my addr 172.21.15.32:0/3265726449 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:22.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.786+0000 7f16091c7640 1 --2- 172.21.15.32:0/3265726449 >> v2:172.21.15.121:3300/0 conn(0x7f160406b820 0x7f16040b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:22.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.786+0000 7f16089c6640 1 -- 172.21.15.32:0/3265726449 >> v2:172.21.15.121:3300/0 conn(0x7f160406b820 msgr2=0x7f16040b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:22.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.786+0000 7f16089c6640 1 --2- 172.21.15.32:0/3265726449 >> v2:172.21.15.121:3300/0 conn(0x7f160406b820 0x7f16040b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:22.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.786+0000 7f16089c6640 1 -- 172.21.15.32:0/3265726449 >> v1:172.21.15.121:6789/0 conn(0x7f16040d4210 legacy=0x7f160406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:22.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.786+0000 7f16089c6640 1 -- 172.21.15.32:0/3265726449 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16040d3a10 con 0x7f160415e9b0 2024-04-24T23:01:22.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.786+0000 7f16091c7640 1 --2- 172.21.15.32:0/3265726449 >> v2:172.21.15.121:3300/0 conn(0x7f160406b820 0x7f16040b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:01:22.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.786+0000 7f16089c6640 1 --2- 172.21.15.32:0/3265726449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f160415e9b0 0x7f16040d2b90 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f16041622f0 tx=0x7f15fc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9240636fb9816ff0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:22.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.786+0000 7f16037fe640 1 -- 172.21.15.32:0/3265726449 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc036070 con 0x7f160415e9b0 2024-04-24T23:01:22.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.787+0000 7f16037fe640 1 -- 172.21.15.32:0/3265726449 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15fc031600 con 0x7f160415e9b0 2024-04-24T23:01:22.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.787+0000 7f16037fe640 1 -- 172.21.15.32:0/3265726449 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc0319d0 con 0x7f160415e9b0 2024-04-24T23:01:22.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.787+0000 7f160ac2a640 1 -- 172.21.15.32:0/3265726449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f160415e9b0 msgr2=0x7f16040d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:22.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.787+0000 7f160ac2a640 1 --2- 172.21.15.32:0/3265726449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f160415e9b0 0x7f16040d2b90 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f16041622f0 tx=0x7f15fc02a660 comp rx=0 tx=0).stop 2024-04-24T23:01:22.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.787+0000 7f160ac2a640 1 -- 172.21.15.32:0/3265726449 shutdown_connections 2024-04-24T23:01:22.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.787+0000 7f160ac2a640 1 --2- 172.21.15.32:0/3265726449 >> v2:172.21.15.121:3300/0 conn(0x7f160406b820 0x7f16040b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:22.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.787+0000 7f160ac2a640 1 --2- 172.21.15.32:0/3265726449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f160415e9b0 0x7f16040d2b90 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:22.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.787+0000 7f160ac2a640 1 -- 172.21.15.32:0/3265726449 >> 172.21.15.32:0/3265726449 conn(0x7f16040d11d0 msgr2=0x7f16040d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:22.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.787+0000 7f160ac2a640 1 -- 172.21.15.32:0/3265726449 shutdown_connections 2024-04-24T23:01:22.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.787+0000 7f160ac2a640 1 -- 172.21.15.32:0/3265726449 wait complete. 2024-04-24T23:01:22.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.788+0000 7f160ac2a640 1 Processor -- start 2024-04-24T23:01:22.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.788+0000 7f160ac2a640 1 -- start start 2024-04-24T23:01:22.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.789+0000 7f160ac2a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f160406b820 0x7f16041564b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:22.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.789+0000 7f160ac2a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16040d4210 0x7f16041605f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:22.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.789+0000 7f160ac2a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f160415e9b0 0x7f16041f6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:22.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.789+0000 7f160ac2a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f16040ba0e0 con 0x7f16040d4210 2024-04-24T23:01:22.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.789+0000 7f160ac2a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f16040d2400 con 0x7f160415e9b0 2024-04-24T23:01:22.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.789+0000 7f160ac2a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f16040ba260 con 0x7f160406b820 2024-04-24T23:01:22.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.789+0000 7f1603fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16040d4210 0x7f16041605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:22.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.789+0000 7f1603fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16040d4210 0x7f16041605f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41698/0 (socket says 172.21.15.32:41698) 2024-04-24T23:01:22.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.789+0000 7f1603fff640 1 -- 172.21.15.32:0/3452727632 learned_addr learned my addr 172.21.15.32:0/3452727632 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:22.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.789+0000 7f16091c7640 1 --2- 172.21.15.32:0/3452727632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f160415e9b0 0x7f16041f6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:22.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.789+0000 7f16089c6640 1 --2- 172.21.15.32:0/3452727632 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f160406b820 0x7f16041564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:22.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.790+0000 7f1603fff640 1 -- 172.21.15.32:0/3452727632 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f160406b820 msgr2=0x7f16041564b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:22.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.790+0000 7f1603fff640 1 --2- 172.21.15.32:0/3452727632 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f160406b820 0x7f16041564b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:22.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.790+0000 7f1603fff640 1 -- 172.21.15.32:0/3452727632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f160415e9b0 msgr2=0x7f16041f6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:22.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.790+0000 7f1603fff640 1 --2- 172.21.15.32:0/3452727632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f160415e9b0 0x7f16041f6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:22.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.790+0000 7f1603fff640 1 -- 172.21.15.32:0/3452727632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1604161ab0 con 0x7f16040d4210 2024-04-24T23:01:22.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.790+0000 7f1603fff640 1 --2- 172.21.15.32:0/3452727632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16040d4210 0x7f16041605f0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f15f800c430 tx=0x7f15f800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:22.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.790+0000 7f16017fa640 1 -- 172.21.15.32:0/3452727632 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f8011790 con 0x7f16040d4210 2024-04-24T23:01:22.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.790+0000 7f16017fa640 1 -- 172.21.15.32:0/3452727632 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15f8017070 con 0x7f16040d4210 2024-04-24T23:01:22.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.790+0000 7f160ac2a640 1 -- 172.21.15.32:0/3452727632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16040cd490 con 0x7f16040d4210 2024-04-24T23:01:22.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.790+0000 7f16017fa640 1 -- 172.21.15.32:0/3452727632 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f8011bb0 con 0x7f16040d4210 2024-04-24T23:01:22.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.790+0000 7f160ac2a640 1 -- 172.21.15.32:0/3452727632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16040cda50 con 0x7f16040d4210 2024-04-24T23:01:22.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.791+0000 7f160ac2a640 1 -- 172.21.15.32:0/3452727632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16040cd2a0 con 0x7f16040d4210 2024-04-24T23:01:22.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.792+0000 7f16017fa640 1 -- 172.21.15.32:0/3452727632 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f15f801c460 con 0x7f16040d4210 2024-04-24T23:01:22.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.792+0000 7f16017fa640 1 --2- 172.21.15.32:0/3452727632 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f15f006be30 0x7f15f006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:22.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.792+0000 7f16017fa640 1 -- 172.21.15.32:0/3452727632 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 4857+0+0 (secure 0 0 0) 0x7f15f808a820 con 0x7f16040d4210 2024-04-24T23:01:22.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.792+0000 7f16089c6640 1 --2- 172.21.15.32:0/3452727632 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f15f006be30 0x7f15f006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:22.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.793+0000 7f16089c6640 1 --2- 172.21.15.32:0/3452727632 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f15f006be30 0x7f15f006e2f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f15fc033000 tx=0x7f15fc009000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:22.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.797+0000 7f16017fa640 1 -- 172.21.15.32:0/3452727632 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f15f8058f60 con 0x7f16040d4210 2024-04-24T23:01:22.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.982+0000 7f160ac2a640 1 -- 172.21.15.32:0/3452727632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f1604068430 con 0x7f16040d4210 2024-04-24T23:01:22.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.982+0000 7f16017fa640 1 -- 172.21.15.32:0/3452727632 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v63) ==== 84+0+65 (secure 0 0 0) 0x7f15f805d260 con 0x7f16040d4210 2024-04-24T23:01:22.984 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:01:22.984 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2024-04-24T23:01:22.984 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:01:22.984 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2024-04-24T23:01:22.984 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": ".mgr" 2024-04-24T23:01:22.985 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:01:22.985 INFO:teuthology.orchestra.run.smithi032.stdout:] 2024-04-24T23:01:22.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.984+0000 7f160ac2a640 1 -- 172.21.15.32:0/3452727632 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f15f006be30 msgr2=0x7f15f006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:22.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.984+0000 7f160ac2a640 1 --2- 172.21.15.32:0/3452727632 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f15f006be30 0x7f15f006e2f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f15fc033000 tx=0x7f15fc009000 comp rx=0 tx=0).stop 2024-04-24T23:01:22.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.984+0000 7f160ac2a640 1 -- 172.21.15.32:0/3452727632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16040d4210 msgr2=0x7f16041605f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:22.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.984+0000 7f160ac2a640 1 --2- 172.21.15.32:0/3452727632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16040d4210 0x7f16041605f0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f15f800c430 tx=0x7f15f800c8f0 comp rx=0 tx=0).stop 2024-04-24T23:01:22.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.984+0000 7f160ac2a640 1 -- 172.21.15.32:0/3452727632 shutdown_connections 2024-04-24T23:01:22.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.984+0000 7f160ac2a640 1 --2- 172.21.15.32:0/3452727632 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f15f006be30 0x7f15f006e2f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:22.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.984+0000 7f160ac2a640 1 --2- 172.21.15.32:0/3452727632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16040d4210 0x7f16041605f0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:22.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.984+0000 7f160ac2a640 1 --2- 172.21.15.32:0/3452727632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f160415e9b0 0x7f16041f6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:22.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.984+0000 7f160ac2a640 1 --2- 172.21.15.32:0/3452727632 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f160406b820 0x7f16041564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:22.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.984+0000 7f160ac2a640 1 -- 172.21.15.32:0/3452727632 >> 172.21.15.32:0/3452727632 conn(0x7f16040d11d0 msgr2=0x7f160415d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:22.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.984+0000 7f160ac2a640 1 -- 172.21.15.32:0/3452727632 shutdown_connections 2024-04-24T23:01:22.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:22.984+0000 7f160ac2a640 1 -- 172.21.15.32:0/3452727632 wait complete. 2024-04-24T23:01:22.997 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-24T23:01:22.998 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-24T23:01:23.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.101+0000 7fcf7687d640 1 Processor -- start 2024-04-24T23:01:23.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.101+0000 7fcf7687d640 1 -- start start 2024-04-24T23:01:23.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.102+0000 7fcf7687d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcf7015c420 0x7fcf700af700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:23.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.102+0000 7fcf7687d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf700afc40 con 0x7fcf700b5390 2024-04-24T23:01:23.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.102+0000 7fcf7687d640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf70160390 con 0x7fcf700b5f30 2024-04-24T23:01:23.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.102+0000 7fcf7687d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fcf70160140 con 0x7fcf7015c420 2024-04-24T23:01:23.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.102+0000 7fcf6f7fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcf700b5390 0x7fcf700b5820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43014/0 (socket says 172.21.15.32:43014) 2024-04-24T23:01:23.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.102+0000 7fcf6f7fe640 1 -- 172.21.15.32:0/4283865529 learned_addr learned my addr 172.21.15.32:0/4283865529 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:01:23.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.102+0000 7fcf6ffff640 1 --2- 172.21.15.32:0/4283865529 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcf7015c420 0x7fcf700af700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:23.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.103+0000 7fcf6effd640 1 -- 172.21.15.32:0/4283865529 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2078235147 0 0) 0x7fcf64001110 con 0x7fcf700b5390 2024-04-24T23:01:23.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.103+0000 7fcf6effd640 1 -- 172.21.15.32:0/4283865529 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf58003520 con 0x7fcf700b5390 2024-04-24T23:01:23.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.103+0000 7fcf6effd640 1 -- 172.21.15.32:0/4283865529 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2158069778 0 0) 0x7fcf70160390 con 0x7fcf700b5f30 2024-04-24T23:01:23.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.103+0000 7fcf6effd640 1 -- 172.21.15.32:0/4283865529 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf64001110 con 0x7fcf700b5f30 2024-04-24T23:01:23.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.103+0000 7fcf6ffff640 1 -- 172.21.15.32:0/4283865529 >> v1:172.21.15.121:6789/0 conn(0x7fcf700b5f30 legacy=0x7fcf700b36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:23.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.103+0000 7fcf6effd640 1 -- 172.21.15.32:0/4283865529 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2132875860 0 0) 0x7fcf58003520 con 0x7fcf700b5390 2024-04-24T23:01:23.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.103+0000 7fcf6ffff640 1 -- 172.21.15.32:0/4283865529 >> v1:172.21.15.32:6789/0 conn(0x7fcf700b5390 legacy=0x7fcf700b5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:23.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.103+0000 7fcf6ffff640 1 -- 172.21.15.32:0/4283865529 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf70160670 con 0x7fcf7015c420 2024-04-24T23:01:23.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.104+0000 7fcf6ffff640 1 --2- 172.21.15.32:0/4283865529 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcf7015c420 0x7fcf700af700 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcf600097d0 tx=0x7fcf6002a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24b2b6018ecc09e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:23.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.104+0000 7fcf6effd640 1 -- 172.21.15.32:0/4283865529 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf60037070 con 0x7fcf7015c420 2024-04-24T23:01:23.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.104+0000 7fcf6effd640 1 -- 172.21.15.32:0/4283865529 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf6002ac30 con 0x7fcf7015c420 2024-04-24T23:01:23.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.104+0000 7fcf6effd640 1 -- 172.21.15.32:0/4283865529 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf64001840 con 0x7fcf7015c420 2024-04-24T23:01:23.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.104+0000 7fcf7687d640 1 -- 172.21.15.32:0/4283865529 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcf7015c420 msgr2=0x7fcf700af700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:23.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.104+0000 7fcf7687d640 1 --2- 172.21.15.32:0/4283865529 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcf7015c420 0x7fcf700af700 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcf600097d0 tx=0x7fcf6002a1a0 comp rx=0 tx=0).stop 2024-04-24T23:01:23.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.104+0000 7fcf7687d640 1 -- 172.21.15.32:0/4283865529 shutdown_connections 2024-04-24T23:01:23.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.104+0000 7fcf7687d640 1 --2- 172.21.15.32:0/4283865529 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcf7015c420 0x7fcf700af700 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:23.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.104+0000 7fcf7687d640 1 -- 172.21.15.32:0/4283865529 >> 172.21.15.32:0/4283865529 conn(0x7fcf700b1f10 msgr2=0x7fcf700b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:23.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.104+0000 7fcf7687d640 1 -- 172.21.15.32:0/4283865529 shutdown_connections 2024-04-24T23:01:23.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.104+0000 7fcf7687d640 1 -- 172.21.15.32:0/4283865529 wait complete. 2024-04-24T23:01:23.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.106+0000 7fcf7687d640 1 Processor -- start 2024-04-24T23:01:23.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.106+0000 7fcf7687d640 1 -- start start 2024-04-24T23:01:23.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.106+0000 7fcf7687d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcf700b5390 0x7fcf701425f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:23.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.106+0000 7fcf7687d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcf700b5f30 0x7fcf7015d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:23.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.106+0000 7fcf7687d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf7015c420 0x7fcf700ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:23.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.106+0000 7fcf7687d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcf70160c30 con 0x7fcf7015c420 2024-04-24T23:01:23.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.106+0000 7fcf7687d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fcf70160fa0 con 0x7fcf700b5390 2024-04-24T23:01:23.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.106+0000 7fcf7687d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fcf700717b0 con 0x7fcf700b5f30 2024-04-24T23:01:23.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.107+0000 7fcf74e1a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf7015c420 0x7fcf700ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:23.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.107+0000 7fcf74e1a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf7015c420 0x7fcf700ae8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41712/0 (socket says 172.21.15.32:41712) 2024-04-24T23:01:23.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.107+0000 7fcf74e1a640 1 -- 172.21.15.32:0/4179350273 learned_addr learned my addr 172.21.15.32:0/4179350273 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:23.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.107+0000 7fcf6f7fe640 1 --2- 172.21.15.32:0/4179350273 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcf700b5f30 0x7fcf7015d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:23.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.107+0000 7fcf6ffff640 1 --2- 172.21.15.32:0/4179350273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcf700b5390 0x7fcf701425f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:23.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.107+0000 7fcf74e1a640 1 -- 172.21.15.32:0/4179350273 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcf700b5f30 msgr2=0x7fcf7015d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:23.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.107+0000 7fcf74e1a640 1 --2- 172.21.15.32:0/4179350273 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcf700b5f30 0x7fcf7015d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:23.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.107+0000 7fcf74e1a640 1 -- 172.21.15.32:0/4179350273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcf700b5390 msgr2=0x7fcf701425f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:23.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.107+0000 7fcf74e1a640 1 --2- 172.21.15.32:0/4179350273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcf700b5390 0x7fcf701425f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:23.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.107+0000 7fcf74e1a640 1 -- 172.21.15.32:0/4179350273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf701f4990 con 0x7fcf7015c420 2024-04-24T23:01:23.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.107+0000 7fcf6ffff640 1 --2- 172.21.15.32:0/4179350273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcf700b5390 0x7fcf701425f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:01:23.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.107+0000 7fcf74e1a640 1 --2- 172.21.15.32:0/4179350273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf7015c420 0x7fcf700ae8b0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fcf6800c700 tx=0x7fcf6800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:23.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.107+0000 7fcf6cff9640 1 -- 172.21.15.32:0/4179350273 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf68003b70 con 0x7fcf7015c420 2024-04-24T23:01:23.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.107+0000 7fcf6cff9640 1 -- 172.21.15.32:0/4179350273 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf68003d10 con 0x7fcf7015c420 2024-04-24T23:01:23.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.107+0000 7fcf7687d640 1 -- 172.21.15.32:0/4179350273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf7015e3b0 con 0x7fcf7015c420 2024-04-24T23:01:23.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.107+0000 7fcf6cff9640 1 -- 172.21.15.32:0/4179350273 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf680052d0 con 0x7fcf7015c420 2024-04-24T23:01:23.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.107+0000 7fcf7687d640 1 -- 172.21.15.32:0/4179350273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf7015e970 con 0x7fcf7015c420 2024-04-24T23:01:23.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.108+0000 7fcf7687d640 1 -- 172.21.15.32:0/4179350273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf70068430 con 0x7fcf7015c420 2024-04-24T23:01:23.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.109+0000 7fcf6cff9640 1 -- 172.21.15.32:0/4179350273 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcf68005470 con 0x7fcf7015c420 2024-04-24T23:01:23.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.109+0000 7fcf6cff9640 1 --2- 172.21.15.32:0/4179350273 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcf5c06be30 0x7fcf5c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:23.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.109+0000 7fcf6cff9640 1 -- 172.21.15.32:0/4179350273 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 4857+0+0 (secure 0 0 0) 0x7fcf6808a910 con 0x7fcf7015c420 2024-04-24T23:01:23.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.109+0000 7fcf6ffff640 1 --2- 172.21.15.32:0/4179350273 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcf5c06be30 0x7fcf5c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:23.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.110+0000 7fcf6ffff640 1 --2- 172.21.15.32:0/4179350273 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcf5c06be30 0x7fcf5c06e2f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fcf60034000 tx=0x7fcf6002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:23.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.113+0000 7fcf6cff9640 1 -- 172.21.15.32:0/4179350273 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcf68059680 con 0x7fcf7015c420 2024-04-24T23:01:23.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.297+0000 7fcf7687d640 1 -- 172.21.15.32:0/4179350273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fcf70067510 con 0x7fcf7015c420 2024-04-24T23:01:23.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.656+0000 7fcf6cff9640 1 -- 172.21.15.32:0/4179350273 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v64) ==== 138+0+0 (secure 0 0 0) 0x7fcf68058f80 con 0x7fcf7015c420 2024-04-24T23:01:23.658 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2024-04-24T23:01:23.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.659+0000 7fcf7687d640 1 -- 172.21.15.32:0/4179350273 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcf5c06be30 msgr2=0x7fcf5c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:23.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.659+0000 7fcf7687d640 1 --2- 172.21.15.32:0/4179350273 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcf5c06be30 0x7fcf5c06e2f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fcf60034000 tx=0x7fcf6002e000 comp rx=0 tx=0).stop 2024-04-24T23:01:23.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.659+0000 7fcf7687d640 1 -- 172.21.15.32:0/4179350273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf7015c420 msgr2=0x7fcf700ae8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:23.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.659+0000 7fcf7687d640 1 --2- 172.21.15.32:0/4179350273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf7015c420 0x7fcf700ae8b0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fcf6800c700 tx=0x7fcf6800cbc0 comp rx=0 tx=0).stop 2024-04-24T23:01:23.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.659+0000 7fcf7687d640 1 -- 172.21.15.32:0/4179350273 shutdown_connections 2024-04-24T23:01:23.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.659+0000 7fcf7687d640 1 --2- 172.21.15.32:0/4179350273 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcf5c06be30 0x7fcf5c06e2f0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:23.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.659+0000 7fcf7687d640 1 --2- 172.21.15.32:0/4179350273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf7015c420 0x7fcf700ae8b0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:23.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.659+0000 7fcf7687d640 1 --2- 172.21.15.32:0/4179350273 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcf700b5f30 0x7fcf7015d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:23.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.659+0000 7fcf7687d640 1 --2- 172.21.15.32:0/4179350273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcf700b5390 0x7fcf701425f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:23.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.659+0000 7fcf7687d640 1 -- 172.21.15.32:0/4179350273 >> 172.21.15.32:0/4179350273 conn(0x7fcf700b1f10 msgr2=0x7fcf7014d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:23.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.659+0000 7fcf7687d640 1 -- 172.21.15.32:0/4179350273 shutdown_connections 2024-04-24T23:01:23.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.659+0000 7fcf7687d640 1 -- 172.21.15.32:0/4179350273 wait complete. 2024-04-24T23:01:23.674 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-24T23:01:23.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.782+0000 7fc4df73b640 1 Processor -- start 2024-04-24T23:01:23.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.782+0000 7fc4df73b640 1 -- start start 2024-04-24T23:01:23.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.783+0000 7fc4df73b640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc4d8162e10 0x7fc4d80ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:23.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.783+0000 7fc4df73b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4d80cd270 con 0x7fc4d80b5af0 2024-04-24T23:01:23.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.783+0000 7fc4df73b640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4d8071610 con 0x7fc4d8162320 2024-04-24T23:01:23.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.783+0000 7fc4df73b640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc4d80b5660 con 0x7fc4d8162e10 2024-04-24T23:01:23.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.783+0000 7fc4dccd6640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc4d80b5af0 0x7fc4d815ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43028/0 (socket says 172.21.15.32:43028) 2024-04-24T23:01:23.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.783+0000 7fc4dccd6640 1 -- 172.21.15.32:0/2426145967 learned_addr learned my addr 172.21.15.32:0/2426145967 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:01:23.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.783+0000 7fc4ddcd8640 1 --2- 172.21.15.32:0/2426145967 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc4d8162e10 0x7fc4d80ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:23.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.783+0000 7fc4d7fff640 1 -- 172.21.15.32:0/2426145967 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1381698713 0 0) 0x7fc4d80cd270 con 0x7fc4d80b5af0 2024-04-24T23:01:23.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.783+0000 7fc4d7fff640 1 -- 172.21.15.32:0/2426145967 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc4c0003520 con 0x7fc4d80b5af0 2024-04-24T23:01:23.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.784+0000 7fc4d7fff640 1 -- 172.21.15.32:0/2426145967 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3702680952 0 0) 0x7fc4d8071610 con 0x7fc4d8162320 2024-04-24T23:01:23.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.784+0000 7fc4d7fff640 1 -- 172.21.15.32:0/2426145967 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc4d80cd270 con 0x7fc4d8162320 2024-04-24T23:01:23.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.784+0000 7fc4d7fff640 1 -- 172.21.15.32:0/2426145967 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3709483247 0 0) 0x7fc4c0003520 con 0x7fc4d80b5af0 2024-04-24T23:01:23.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.784+0000 7fc4d7fff640 1 -- 172.21.15.32:0/2426145967 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc4d8162e10 msgr2=0x7fc4d80ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:23.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.784+0000 7fc4d7fff640 1 --2- 172.21.15.32:0/2426145967 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc4d8162e10 0x7fc4d80ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:23.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.784+0000 7fc4d7fff640 1 -- 172.21.15.32:0/2426145967 >> v1:172.21.15.121:6789/0 conn(0x7fc4d8162320 legacy=0x7fc4d8162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:23.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.784+0000 7fc4d7fff640 1 -- 172.21.15.32:0/2426145967 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4d80718f0 con 0x7fc4d80b5af0 2024-04-24T23:01:23.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.784+0000 7fc4d7fff640 1 -- 172.21.15.32:0/2426145967 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7fc4cc001670 con 0x7fc4d80b5af0 2024-04-24T23:01:23.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.784+0000 7fc4d7fff640 1 -- 172.21.15.32:0/2426145967 >> v1:172.21.15.32:6789/0 conn(0x7fc4d80b5af0 legacy=0x7fc4d815ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:23.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.784+0000 7fc4d7fff640 1 --2- 172.21.15.32:0/2426145967 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4c0003b70 0x7fc4c0003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:23.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.784+0000 7fc4d7fff640 1 --2- 172.21.15.32:0/2426145967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4c0004800 0x7fc4c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:23.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.784+0000 7fc4d7fff640 1 --2- 172.21.15.32:0/2426145967 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc4d8162e10 0x7fc4c00051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:23.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.784+0000 7fc4d7fff640 1 -- 172.21.15.32:0/2426145967 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d80b5660 con 0x7fc4c0004800 2024-04-24T23:01:23.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.784+0000 7fc4d7fff640 1 -- 172.21.15.32:0/2426145967 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c000bad0 con 0x7fc4c0003b70 2024-04-24T23:01:23.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.784+0000 7fc4d7fff640 1 -- 172.21.15.32:0/2426145967 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc4c000bcd0 con 0x7fc4d8162e10 2024-04-24T23:01:23.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.784+0000 7fc4dccd6640 1 --2- 172.21.15.32:0/2426145967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4c0004800 0x7fc4c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:23.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.784+0000 7fc4dd4d7640 1 --2- 172.21.15.32:0/2426145967 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4c0003b70 0x7fc4c0003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:23.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.784+0000 7fc4ddcd8640 1 --2- 172.21.15.32:0/2426145967 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc4d8162e10 0x7fc4c00051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:23.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.784+0000 7fc4dccd6640 1 -- 172.21.15.32:0/2426145967 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc4d8162e10 msgr2=0x7fc4c00051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:23.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.784+0000 7fc4dccd6640 1 --2- 172.21.15.32:0/2426145967 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc4d8162e10 0x7fc4c00051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:23.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.784+0000 7fc4dccd6640 1 -- 172.21.15.32:0/2426145967 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4c0003b70 msgr2=0x7fc4c0003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:23.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.784+0000 7fc4dccd6640 1 --2- 172.21.15.32:0/2426145967 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4c0003b70 0x7fc4c0003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:23.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.784+0000 7fc4dccd6640 1 -- 172.21.15.32:0/2426145967 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c0003520 con 0x7fc4c0004800 2024-04-24T23:01:23.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.785+0000 7fc4dccd6640 1 --2- 172.21.15.32:0/2426145967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4c0004800 0x7fc4c0004c80 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc00c1c0 tx=0x7fc4c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4799316496900497 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:23.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.785+0000 7fc4d7fff640 1 -- 172.21.15.32:0/2426145967 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc007cc0 con 0x7fc4c0004800 2024-04-24T23:01:23.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.785+0000 7fc4d7fff640 1 -- 172.21.15.32:0/2426145967 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4cc005ad0 con 0x7fc4c0004800 2024-04-24T23:01:23.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.785+0000 7fc4d7fff640 1 -- 172.21.15.32:0/2426145967 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc007070 con 0x7fc4c0004800 2024-04-24T23:01:23.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.785+0000 7fc4df73b640 1 -- 172.21.15.32:0/2426145967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4c0004800 msgr2=0x7fc4c0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:23.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.785+0000 7fc4df73b640 1 --2- 172.21.15.32:0/2426145967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4c0004800 0x7fc4c0004c80 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc00c1c0 tx=0x7fc4c0005830 comp rx=0 tx=0).stop 2024-04-24T23:01:23.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.785+0000 7fc4ddcd8640 1 -- 172.21.15.32:0/2426145967 reap_dead start 2024-04-24T23:01:23.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.785+0000 7fc4df73b640 1 -- 172.21.15.32:0/2426145967 shutdown_connections 2024-04-24T23:01:23.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.785+0000 7fc4df73b640 1 -- 172.21.15.32:0/2426145967 >> 172.21.15.32:0/2426145967 conn(0x7fc4d80d0ae0 msgr2=0x7fc4d80d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:23.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.786+0000 7fc4df73b640 1 -- 172.21.15.32:0/2426145967 shutdown_connections 2024-04-24T23:01:23.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.786+0000 7fc4df73b640 1 -- 172.21.15.32:0/2426145967 wait complete. 2024-04-24T23:01:23.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.787+0000 7fc4df73b640 1 Processor -- start 2024-04-24T23:01:23.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.787+0000 7fc4df73b640 1 -- start start 2024-04-24T23:01:23.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.787+0000 7fc4df73b640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc4c0004800 0x7fc4d814e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:23.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.787+0000 7fc4df73b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4d8157450 0x7fc4d80ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:23.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.787+0000 7fc4df73b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4d81578d0 0x7fc4d80cf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:23.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.787+0000 7fc4df73b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d8071bd0 con 0x7fc4d81578d0 2024-04-24T23:01:23.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.787+0000 7fc4df73b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d8067950 con 0x7fc4d8157450 2024-04-24T23:01:23.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.787+0000 7fc4df73b640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc4d8071d50 con 0x7fc4c0004800 2024-04-24T23:01:23.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.788+0000 7fc4ddcd8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4d81578d0 0x7fc4d80cf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:23.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.788+0000 7fc4ddcd8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4d81578d0 0x7fc4d80cf0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41720/0 (socket says 172.21.15.32:41720) 2024-04-24T23:01:23.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.788+0000 7fc4ddcd8640 1 -- 172.21.15.32:0/1236960905 learned_addr learned my addr 172.21.15.32:0/1236960905 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:23.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.788+0000 7fc4dccd6640 1 --2- 172.21.15.32:0/1236960905 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4d8157450 0x7fc4d80ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:23.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.788+0000 7fc4dd4d7640 1 --2- 172.21.15.32:0/1236960905 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc4c0004800 0x7fc4d814e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:23.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.788+0000 7fc4ddcd8640 1 -- 172.21.15.32:0/1236960905 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc4c0004800 msgr2=0x7fc4d814e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:23.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.788+0000 7fc4ddcd8640 1 --2- 172.21.15.32:0/1236960905 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc4c0004800 0x7fc4d814e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:23.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.788+0000 7fc4ddcd8640 1 -- 172.21.15.32:0/1236960905 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4d8157450 msgr2=0x7fc4d80ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:23.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.788+0000 7fc4ddcd8640 1 --2- 172.21.15.32:0/1236960905 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4d8157450 0x7fc4d80ceb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:23.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.788+0000 7fc4ddcd8640 1 -- 172.21.15.32:0/1236960905 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4d81558d0 con 0x7fc4d81578d0 2024-04-24T23:01:23.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.788+0000 7fc4ddcd8640 1 --2- 172.21.15.32:0/1236960905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4d81578d0 0x7fc4d80cf0a0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8009e90 tx=0x7fc4c80115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:23.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.788+0000 7fc4d5ffb640 1 -- 172.21.15.32:0/1236960905 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c8011e20 con 0x7fc4d81578d0 2024-04-24T23:01:23.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.788+0000 7fc4d5ffb640 1 -- 172.21.15.32:0/1236960905 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4c801b070 con 0x7fc4d81578d0 2024-04-24T23:01:23.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.788+0000 7fc4df73b640 1 -- 172.21.15.32:0/1236960905 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4d8155bb0 con 0x7fc4d81578d0 2024-04-24T23:01:23.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.788+0000 7fc4d5ffb640 1 -- 172.21.15.32:0/1236960905 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c8003990 con 0x7fc4d81578d0 2024-04-24T23:01:23.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.788+0000 7fc4df73b640 1 -- 172.21.15.32:0/1236960905 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4d8156170 con 0x7fc4d81578d0 2024-04-24T23:01:23.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.790+0000 7fc4df73b640 1 -- 172.21.15.32:0/1236960905 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4d80b4660 con 0x7fc4d81578d0 2024-04-24T23:01:23.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.790+0000 7fc4d5ffb640 1 -- 172.21.15.32:0/1236960905 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc4c8005d10 con 0x7fc4d81578d0 2024-04-24T23:01:23.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.790+0000 7fc4d5ffb640 1 --2- 172.21.15.32:0/1236960905 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc4c406be30 0x7fc4c406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:23.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.791+0000 7fc4d5ffb640 1 -- 172.21.15.32:0/1236960905 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5227+0+0 (secure 0 0 0) 0x7fc4c808af60 con 0x7fc4d81578d0 2024-04-24T23:01:23.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.791+0000 7fc4dd4d7640 1 --2- 172.21.15.32:0/1236960905 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc4c406be30 0x7fc4c406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:23.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.791+0000 7fc4dd4d7640 1 --2- 172.21.15.32:0/1236960905 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc4c406be30 0x7fc4c406e2f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0005510 tx=0x7fc4d0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:23.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.795+0000 7fc4d5ffb640 1 -- 172.21.15.32:0/1236960905 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc4d80b4660 con 0x7fc4d81578d0 2024-04-24T23:01:23.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:23.979+0000 7fc4df73b640 1 -- 172.21.15.32:0/1236960905 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fc4d8068430 con 0x7fc4d81578d0 2024-04-24T23:01:24.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.659+0000 7fc4d5ffb640 1 -- 172.21.15.32:0/1236960905 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v65) ==== 170+0+0 (secure 0 0 0) 0x7fc4c8014350 con 0x7fc4d81578d0 2024-04-24T23:01:24.661 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2024-04-24T23:01:24.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.663+0000 7fc4df73b640 1 -- 172.21.15.32:0/1236960905 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc4c406be30 msgr2=0x7fc4c406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:24.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.663+0000 7fc4df73b640 1 --2- 172.21.15.32:0/1236960905 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc4c406be30 0x7fc4c406e2f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0005510 tx=0x7fc4d0009040 comp rx=0 tx=0).stop 2024-04-24T23:01:24.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.663+0000 7fc4df73b640 1 -- 172.21.15.32:0/1236960905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4d81578d0 msgr2=0x7fc4d80cf0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:24.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.663+0000 7fc4df73b640 1 --2- 172.21.15.32:0/1236960905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4d81578d0 0x7fc4d80cf0a0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8009e90 tx=0x7fc4c80115a0 comp rx=0 tx=0).stop 2024-04-24T23:01:24.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.663+0000 7fc4df73b640 1 -- 172.21.15.32:0/1236960905 shutdown_connections 2024-04-24T23:01:24.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.663+0000 7fc4df73b640 1 --2- 172.21.15.32:0/1236960905 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc4c406be30 0x7fc4c406e2f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:24.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.663+0000 7fc4df73b640 1 --2- 172.21.15.32:0/1236960905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4d81578d0 0x7fc4d80cf0a0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:24.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.663+0000 7fc4df73b640 1 --2- 172.21.15.32:0/1236960905 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4d8157450 0x7fc4d80ceb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:24.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.663+0000 7fc4df73b640 1 --2- 172.21.15.32:0/1236960905 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc4c0004800 0x7fc4d814e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:24.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.663+0000 7fc4df73b640 1 -- 172.21.15.32:0/1236960905 >> 172.21.15.32:0/1236960905 conn(0x7fc4d80d0ae0 msgr2=0x7fc4d80b4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:24.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.663+0000 7fc4df73b640 1 -- 172.21.15.32:0/1236960905 shutdown_connections 2024-04-24T23:01:24.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.663+0000 7fc4df73b640 1 -- 172.21.15.32:0/1236960905 wait complete. 2024-04-24T23:01:24.683 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-24T23:01:24.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.793+0000 7f7327672640 1 Processor -- start 2024-04-24T23:01:24.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.793+0000 7f7327672640 1 -- start start 2024-04-24T23:01:24.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.794+0000 7f7327672640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f73200af890 0x7f73201670f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:24.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.794+0000 7f7327672640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73200b6430 con 0x7f73200b44d0 2024-04-24T23:01:24.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.794+0000 7f7327672640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7320156f80 con 0x7f732006c5f0 2024-04-24T23:01:24.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.794+0000 7f7327672640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f73200b6150 con 0x7f73200af890 2024-04-24T23:01:24.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.794+0000 7f7324c0d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f73200b44d0 0x7f73200b5890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43044/0 (socket says 172.21.15.32:43044) 2024-04-24T23:01:24.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.794+0000 7f7324c0d640 1 -- 172.21.15.32:0/3014477893 learned_addr learned my addr 172.21.15.32:0/3014477893 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:01:24.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.794+0000 7f7325c0f640 1 --2- 172.21.15.32:0/3014477893 >> v2:172.21.15.32:3300/0 conn(0x7f73200af890 0x7f73201670f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:24.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.794+0000 7f7325c0f640 1 -- 172.21.15.32:0/3014477893 >> v1:172.21.15.121:6789/0 conn(0x7f732006c5f0 legacy=0x7f732006c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:24.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.794+0000 7f7325c0f640 1 -- 172.21.15.32:0/3014477893 >> v1:172.21.15.32:6789/0 conn(0x7f73200b44d0 legacy=0x7f73200b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:24.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.794+0000 7f7325c0f640 1 -- 172.21.15.32:0/3014477893 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73200b39f0 con 0x7f73200af890 2024-04-24T23:01:24.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.795+0000 7f7325c0f640 1 --2- 172.21.15.32:0/3014477893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73200af890 0x7f73201670f0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f73180097a0 tx=0x7f731802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc56304f4ba617fb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:24.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.795+0000 7f731ffff640 1 -- 172.21.15.32:0/3014477893 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7318037070 con 0x7f73200af890 2024-04-24T23:01:24.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.795+0000 7f731ffff640 1 -- 172.21.15.32:0/3014477893 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7318032440 con 0x7f73200af890 2024-04-24T23:01:24.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.795+0000 7f731ffff640 1 -- 172.21.15.32:0/3014477893 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7318032810 con 0x7f73200af890 2024-04-24T23:01:24.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.796+0000 7f7327672640 1 -- 172.21.15.32:0/3014477893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73200af890 msgr2=0x7f73201670f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:24.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.796+0000 7f7327672640 1 --2- 172.21.15.32:0/3014477893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73200af890 0x7f73201670f0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f73180097a0 tx=0x7f731802a1e0 comp rx=0 tx=0).stop 2024-04-24T23:01:24.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.796+0000 7f7327672640 1 -- 172.21.15.32:0/3014477893 shutdown_connections 2024-04-24T23:01:24.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.796+0000 7f7327672640 1 --2- 172.21.15.32:0/3014477893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73200af890 0x7f73201670f0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:24.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.796+0000 7f7327672640 1 -- 172.21.15.32:0/3014477893 >> 172.21.15.32:0/3014477893 conn(0x7f732006b110 msgr2=0x7f732006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:24.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.796+0000 7f7327672640 1 -- 172.21.15.32:0/3014477893 shutdown_connections 2024-04-24T23:01:24.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.796+0000 7f7327672640 1 -- 172.21.15.32:0/3014477893 wait complete. 2024-04-24T23:01:24.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.797+0000 7f7327672640 1 Processor -- start 2024-04-24T23:01:24.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.797+0000 7f7327672640 1 -- start start 2024-04-24T23:01:24.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.798+0000 7f7327672640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f732006c5f0 0x7f732014c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:24.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.798+0000 7f7327672640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73200af890 0x7f7320153ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:24.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.798+0000 7f7327672640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f73200b44d0 0x7f73201543e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:24.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.798+0000 7f7327672640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7320167630 con 0x7f73200af890 2024-04-24T23:01:24.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.798+0000 7f7327672640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f73200b3fb0 con 0x7f732006c5f0 2024-04-24T23:01:24.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.798+0000 7f7327672640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f73201677b0 con 0x7f73200b44d0 2024-04-24T23:01:24.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.798+0000 7f7324c0d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73200af890 0x7f7320153ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:24.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.798+0000 7f7324c0d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73200af890 0x7f7320153ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41752/0 (socket says 172.21.15.32:41752) 2024-04-24T23:01:24.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.798+0000 7f7324c0d640 1 -- 172.21.15.32:0/2496659544 learned_addr learned my addr 172.21.15.32:0/2496659544 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:24.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.798+0000 7f732540e640 1 --2- 172.21.15.32:0/2496659544 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f732006c5f0 0x7f732014c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:24.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.798+0000 7f7325c0f640 1 --2- 172.21.15.32:0/2496659544 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f73200b44d0 0x7f73201543e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:24.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.798+0000 7f7324c0d640 1 -- 172.21.15.32:0/2496659544 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f73200b44d0 msgr2=0x7f73201543e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:24.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.798+0000 7f7324c0d640 1 --2- 172.21.15.32:0/2496659544 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f73200b44d0 0x7f73201543e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:24.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.798+0000 7f7324c0d640 1 -- 172.21.15.32:0/2496659544 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f732006c5f0 msgr2=0x7f732014c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:24.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.798+0000 7f7324c0d640 1 --2- 172.21.15.32:0/2496659544 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f732006c5f0 0x7f732014c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:24.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.798+0000 7f7324c0d640 1 -- 172.21.15.32:0/2496659544 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f732015fbd0 con 0x7f73200af890 2024-04-24T23:01:24.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.798+0000 7f7325c0f640 1 --2- 172.21.15.32:0/2496659544 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f73200b44d0 0x7f73201543e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:01:24.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.798+0000 7f7324c0d640 1 --2- 172.21.15.32:0/2496659544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73200af890 0x7f7320153ea0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f731400c3f0 tx=0x7f731400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:24.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.799+0000 7f731dffb640 1 -- 172.21.15.32:0/2496659544 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7314017070 con 0x7f73200af890 2024-04-24T23:01:24.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.799+0000 7f731dffb640 1 -- 172.21.15.32:0/2496659544 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7314005ce0 con 0x7f73200af890 2024-04-24T23:01:24.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.799+0000 7f7327672640 1 -- 172.21.15.32:0/2496659544 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f732015e670 con 0x7f73200af890 2024-04-24T23:01:24.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.799+0000 7f731dffb640 1 -- 172.21.15.32:0/2496659544 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73140072f0 con 0x7f73200af890 2024-04-24T23:01:24.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.799+0000 7f7327672640 1 -- 172.21.15.32:0/2496659544 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f732015ec30 con 0x7f73200af890 2024-04-24T23:01:24.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.800+0000 7f7327672640 1 -- 172.21.15.32:0/2496659544 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7320068430 con 0x7f73200af890 2024-04-24T23:01:24.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.800+0000 7f731dffb640 1 -- 172.21.15.32:0/2496659544 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f731401a070 con 0x7f73200af890 2024-04-24T23:01:24.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.801+0000 7f731dffb640 1 --2- 172.21.15.32:0/2496659544 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f730c06be30 0x7f730c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:24.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.801+0000 7f731dffb640 1 -- 172.21.15.32:0/2496659544 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5609+0+0 (secure 0 0 0) 0x7f731408a040 con 0x7f73200af890 2024-04-24T23:01:24.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.801+0000 7f732540e640 1 --2- 172.21.15.32:0/2496659544 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f730c06be30 0x7f730c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:24.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.802+0000 7f732540e640 1 --2- 172.21.15.32:0/2496659544 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f730c06be30 0x7f730c06e2f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f73201f5f60 tx=0x7f7310003d90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:24.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:24.805+0000 7f731dffb640 1 -- 172.21.15.32:0/2496659544 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7314058400 con 0x7f73200af890 2024-04-24T23:01:25.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.026+0000 7f7327672640 1 -- 172.21.15.32:0/2496659544 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f7320071050 con 0x7f73200af890 2024-04-24T23:01:25.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.683+0000 7f731dffb640 1 -- 172.21.15.32:0/2496659544 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v25) ==== 172+0+0 (secure 0 0 0) 0x7f731405c700 con 0x7f73200af890 2024-04-24T23:01:25.687 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 8 and data pool 9 2024-04-24T23:01:25.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.688+0000 7f7327672640 1 -- 172.21.15.32:0/2496659544 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f730c06be30 msgr2=0x7f730c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:25.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.688+0000 7f7327672640 1 --2- 172.21.15.32:0/2496659544 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f730c06be30 0x7f730c06e2f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f73201f5f60 tx=0x7f7310003d90 comp rx=0 tx=0).stop 2024-04-24T23:01:25.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.688+0000 7f7327672640 1 -- 172.21.15.32:0/2496659544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73200af890 msgr2=0x7f7320153ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:25.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.688+0000 7f7327672640 1 --2- 172.21.15.32:0/2496659544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73200af890 0x7f7320153ea0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f731400c3f0 tx=0x7f731400c7f0 comp rx=0 tx=0).stop 2024-04-24T23:01:25.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.688+0000 7f7327672640 1 -- 172.21.15.32:0/2496659544 shutdown_connections 2024-04-24T23:01:25.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.688+0000 7f7327672640 1 --2- 172.21.15.32:0/2496659544 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f730c06be30 0x7f730c06e2f0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:25.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.688+0000 7f7327672640 1 --2- 172.21.15.32:0/2496659544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73200af890 0x7f7320153ea0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:25.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.688+0000 7f7327672640 1 --2- 172.21.15.32:0/2496659544 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f73200b44d0 0x7f73201543e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:25.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.688+0000 7f7327672640 1 --2- 172.21.15.32:0/2496659544 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f732006c5f0 0x7f732014c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:25.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.688+0000 7f7327672640 1 -- 172.21.15.32:0/2496659544 >> 172.21.15.32:0/2496659544 conn(0x7f732006b110 msgr2=0x7f73200cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:25.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.689+0000 7f7327672640 1 -- 172.21.15.32:0/2496659544 shutdown_connections 2024-04-24T23:01:25.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.689+0000 7f7327672640 1 -- 172.21.15.32:0/2496659544 wait complete. 2024-04-24T23:01:25.707 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:01:25.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.811+0000 7f0da65b0640 1 Processor -- start 2024-04-24T23:01:25.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.811+0000 7f0da65b0640 1 -- start start 2024-04-24T23:01:25.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.812+0000 7f0da65b0640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0da006ba30 0x7f0da006d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:25.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.812+0000 7f0da65b0640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0da006da90 0x7f0da00b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:25.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.812+0000 7f0da65b0640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0da015ef10 con 0x7f0da0158dd0 2024-04-24T23:01:25.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.812+0000 7f0da65b0640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f0da015f1f0 con 0x7f0da006da90 2024-04-24T23:01:25.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.812+0000 7f0da65b0640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f0da015f4d0 con 0x7f0da006ba30 2024-04-24T23:01:25.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.812+0000 7f0da4b4d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0da006da90 0x7f0da00b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:25.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.812+0000 7f0d9ffff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0da0158dd0 0x7f0da006b320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43056/0 (socket says 172.21.15.32:43056) 2024-04-24T23:01:25.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.812+0000 7f0d9ffff640 1 -- 172.21.15.32:0/296260755 learned_addr learned my addr 172.21.15.32:0/296260755 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:01:25.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.813+0000 7f0da4b4d640 1 -- 172.21.15.32:0/296260755 >> v2:172.21.15.121:3300/0 conn(0x7f0da006ba30 msgr2=0x7f0da006d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:25.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.813+0000 7f0da4b4d640 1 --2- 172.21.15.32:0/296260755 >> v2:172.21.15.121:3300/0 conn(0x7f0da006ba30 0x7f0da006d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:25.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.813+0000 7f0da4b4d640 1 -- 172.21.15.32:0/296260755 >> v1:172.21.15.32:6789/0 conn(0x7f0da0158dd0 legacy=0x7f0da006b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:25.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.813+0000 7f0da4b4d640 1 -- 172.21.15.32:0/296260755 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0da015f7b0 con 0x7f0da006da90 2024-04-24T23:01:25.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.813+0000 7f0da4b4d640 1 --2- 172.21.15.32:0/296260755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0da006da90 0x7f0da00b5d00 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f0da0070460 tx=0x7f0d9802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38eead0f178d7108 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:25.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.813+0000 7f0d9effd640 1 -- 172.21.15.32:0/296260755 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d98007740 con 0x7f0da006da90 2024-04-24T23:01:25.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.813+0000 7f0d9effd640 1 -- 172.21.15.32:0/296260755 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d980078e0 con 0x7f0da006da90 2024-04-24T23:01:25.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.814+0000 7f0d9effd640 1 -- 172.21.15.32:0/296260755 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d98007cb0 con 0x7f0da006da90 2024-04-24T23:01:25.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.814+0000 7f0da65b0640 1 -- 172.21.15.32:0/296260755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0da006da90 msgr2=0x7f0da00b5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:25.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.814+0000 7f0da65b0640 1 --2- 172.21.15.32:0/296260755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0da006da90 0x7f0da00b5d00 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f0da0070460 tx=0x7f0d9802a680 comp rx=0 tx=0).stop 2024-04-24T23:01:25.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.814+0000 7f0da65b0640 1 -- 172.21.15.32:0/296260755 shutdown_connections 2024-04-24T23:01:25.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.814+0000 7f0da65b0640 1 --2- 172.21.15.32:0/296260755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0da006da90 0x7f0da00b5d00 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:25.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.814+0000 7f0da65b0640 1 --2- 172.21.15.32:0/296260755 >> v2:172.21.15.121:3300/0 conn(0x7f0da006ba30 0x7f0da006d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:25.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.814+0000 7f0da65b0640 1 -- 172.21.15.32:0/296260755 >> 172.21.15.32:0/296260755 conn(0x7f0da006e800 msgr2=0x7f0da006ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:25.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.814+0000 7f0da65b0640 1 -- 172.21.15.32:0/296260755 shutdown_connections 2024-04-24T23:01:25.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.814+0000 7f0da65b0640 1 -- 172.21.15.32:0/296260755 wait complete. 2024-04-24T23:01:25.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.815+0000 7f0da65b0640 1 Processor -- start 2024-04-24T23:01:25.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.815+0000 7f0da65b0640 1 -- start start 2024-04-24T23:01:25.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.816+0000 7f0da65b0640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0da006ba30 0x7f0da01f84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:25.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.816+0000 7f0da65b0640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0da006da90 0x7f0da00b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:25.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.816+0000 7f0da65b0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0da0158dd0 0x7f0da00b2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:25.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.816+0000 7f0da65b0640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0da006bfe0 con 0x7f0da0158dd0 2024-04-24T23:01:25.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.816+0000 7f0da65b0640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0da006ded0 con 0x7f0da006ba30 2024-04-24T23:01:25.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.816+0000 7f0da65b0640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f0da01591b0 con 0x7f0da006da90 2024-04-24T23:01:25.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.816+0000 7f0da4b4d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0da0158dd0 0x7f0da00b2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:25.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.816+0000 7f0da4b4d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0da0158dd0 0x7f0da00b2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41772/0 (socket says 172.21.15.32:41772) 2024-04-24T23:01:25.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.816+0000 7f0da4b4d640 1 -- 172.21.15.32:0/3159687804 learned_addr learned my addr 172.21.15.32:0/3159687804 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:25.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.816+0000 7f0d9ffff640 1 --2- 172.21.15.32:0/3159687804 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0da006ba30 0x7f0da01f84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:25.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.816+0000 7f0d9f7fe640 1 --2- 172.21.15.32:0/3159687804 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0da006da90 0x7f0da00b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:25.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.817+0000 7f0da4b4d640 1 -- 172.21.15.32:0/3159687804 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0da006da90 msgr2=0x7f0da00b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:25.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.817+0000 7f0da4b4d640 1 --2- 172.21.15.32:0/3159687804 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0da006da90 0x7f0da00b4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:25.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.817+0000 7f0da4b4d640 1 -- 172.21.15.32:0/3159687804 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0da006ba30 msgr2=0x7f0da01f84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:25.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.817+0000 7f0da4b4d640 1 --2- 172.21.15.32:0/3159687804 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0da006ba30 0x7f0da01f84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:25.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.817+0000 7f0da4b4d640 1 -- 172.21.15.32:0/3159687804 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0da01f5850 con 0x7f0da0158dd0 2024-04-24T23:01:25.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.817+0000 7f0da4b4d640 1 --2- 172.21.15.32:0/3159687804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0da0158dd0 0x7f0da00b2af0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f0d9802ac70 tx=0x7f0d98004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:25.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.817+0000 7f0d9cff9640 1 -- 172.21.15.32:0/3159687804 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d98040070 con 0x7f0da0158dd0 2024-04-24T23:01:25.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.817+0000 7f0d9cff9640 1 -- 172.21.15.32:0/3159687804 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d98031bc0 con 0x7f0da0158dd0 2024-04-24T23:01:25.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.817+0000 7f0d9cff9640 1 -- 172.21.15.32:0/3159687804 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d98045460 con 0x7f0da0158dd0 2024-04-24T23:01:25.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.817+0000 7f0da65b0640 1 -- 172.21.15.32:0/3159687804 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0da01f5b30 con 0x7f0da0158dd0 2024-04-24T23:01:25.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.817+0000 7f0da65b0640 1 -- 172.21.15.32:0/3159687804 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0da01f60f0 con 0x7f0da0158dd0 2024-04-24T23:01:25.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.818+0000 7f0da65b0640 1 -- 172.21.15.32:0/3159687804 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0da01f62f0 con 0x7f0da0158dd0 2024-04-24T23:01:25.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.819+0000 7f0d9cff9640 1 -- 172.21.15.32:0/3159687804 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0d98039040 con 0x7f0da0158dd0 2024-04-24T23:01:25.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.819+0000 7f0d9cff9640 1 --2- 172.21.15.32:0/3159687804 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0d8c06be30 0x7f0d8c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:25.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.819+0000 7f0d9cff9640 1 -- 172.21.15.32:0/3159687804 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5709+0+0 (secure 0 0 0) 0x7f0d980ac6d0 con 0x7f0da0158dd0 2024-04-24T23:01:25.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.819+0000 7f0d9ffff640 1 --2- 172.21.15.32:0/3159687804 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0d8c06be30 0x7f0d8c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:25.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.820+0000 7f0d9ffff640 1 --2- 172.21.15.32:0/3159687804 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0d8c06be30 0x7f0d8c06e2f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0d94007fd0 tx=0x7f0d94007e80 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:25.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:25.823+0000 7f0d9cff9640 1 -- 172.21.15.32:0/3159687804 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0da01f62f0 con 0x7f0da0158dd0 2024-04-24T23:01:26.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.007+0000 7f0da65b0640 1 -- 172.21.15.32:0/3159687804 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0da0068430 con 0x7f0da0158dd0 2024-04-24T23:01:26.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.008+0000 7f0d9cff9640 1 -- 172.21.15.32:0/3159687804 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) ==== 74+0+16690 (secure 0 0 0) 0x7f0d9807b110 con 0x7f0da0158dd0 2024-04-24T23:01:26.009 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:01:26.010 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":66,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:01:25.027996+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:01:23.328944+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-24T23:01:24.010365+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:01:26.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.010+0000 7f0da65b0640 1 -- 172.21.15.32:0/3159687804 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0d8c06be30 msgr2=0x7f0d8c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:26.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.010+0000 7f0da65b0640 1 --2- 172.21.15.32:0/3159687804 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0d8c06be30 0x7f0d8c06e2f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0d94007fd0 tx=0x7f0d94007e80 comp rx=0 tx=0).stop 2024-04-24T23:01:26.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.010+0000 7f0da65b0640 1 -- 172.21.15.32:0/3159687804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0da0158dd0 msgr2=0x7f0da00b2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:26.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.010+0000 7f0da65b0640 1 --2- 172.21.15.32:0/3159687804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0da0158dd0 0x7f0da00b2af0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f0d9802ac70 tx=0x7f0d98004a40 comp rx=0 tx=0).stop 2024-04-24T23:01:26.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.010+0000 7f0da65b0640 1 -- 172.21.15.32:0/3159687804 shutdown_connections 2024-04-24T23:01:26.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.010+0000 7f0da65b0640 1 --2- 172.21.15.32:0/3159687804 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0d8c06be30 0x7f0d8c06e2f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:26.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.010+0000 7f0da65b0640 1 --2- 172.21.15.32:0/3159687804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0da0158dd0 0x7f0da00b2af0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:26.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.010+0000 7f0da65b0640 1 --2- 172.21.15.32:0/3159687804 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0da006da90 0x7f0da00b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:26.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.010+0000 7f0da65b0640 1 --2- 172.21.15.32:0/3159687804 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0da006ba30 0x7f0da01f84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:26.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.010+0000 7f0da65b0640 1 -- 172.21.15.32:0/3159687804 >> 172.21.15.32:0/3159687804 conn(0x7f0da006e800 msgr2=0x7f0da00b6600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:26.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.010+0000 7f0da65b0640 1 -- 172.21.15.32:0/3159687804 shutdown_connections 2024-04-24T23:01:26.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.010+0000 7f0da65b0640 1 -- 172.21.15.32:0/3159687804 wait complete. 2024-04-24T23:01:26.023 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:01:26.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.123+0000 7f14a6ec1640 1 Processor -- start 2024-04-24T23:01:26.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.123+0000 7f14a6ec1640 1 -- start start 2024-04-24T23:01:26.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.124+0000 7f14a6ec1640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14a015e9b0 0x7f14a00d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:26.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.124+0000 7f14a6ec1640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f14a00d4210 0x7f14a006b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:26.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.124+0000 7f14a6ec1640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f14a00ba0e0 0x7f14a006b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:26.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.124+0000 7f14a6ec1640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f14a00d3170 con 0x7f14a00d4210 2024-04-24T23:01:26.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.124+0000 7f14a6ec1640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f14a00d3450 con 0x7f14a00ba0e0 2024-04-24T23:01:26.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.124+0000 7f14a6ec1640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f14a00d3730 con 0x7f14a015e9b0 2024-04-24T23:01:26.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.124+0000 7f149ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f14a00d4210 0x7f14a006b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:26.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.124+0000 7f149ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f14a00d4210 0x7f14a006b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41784/0 (socket says 172.21.15.32:41784) 2024-04-24T23:01:26.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.124+0000 7f149ffff640 1 -- 172.21.15.32:0/2814820838 learned_addr learned my addr 172.21.15.32:0/2814820838 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:26.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.125+0000 7f14a545e640 1 --2- 172.21.15.32:0/2814820838 >> v2:172.21.15.121:3300/0 conn(0x7f14a00ba0e0 0x7f14a006b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:26.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.125+0000 7f14a4c5d640 1 --2- 172.21.15.32:0/2814820838 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14a015e9b0 0x7f14a00d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:26.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.125+0000 7f149ffff640 1 -- 172.21.15.32:0/2814820838 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14a015e9b0 msgr2=0x7f14a00d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:26.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.125+0000 7f149ffff640 1 --2- 172.21.15.32:0/2814820838 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14a015e9b0 0x7f14a00d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:26.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.125+0000 7f149ffff640 1 -- 172.21.15.32:0/2814820838 >> v2:172.21.15.121:3300/0 conn(0x7f14a00ba0e0 msgr2=0x7f14a006b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:26.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.125+0000 7f149ffff640 1 --2- 172.21.15.32:0/2814820838 >> v2:172.21.15.121:3300/0 conn(0x7f14a00ba0e0 0x7f14a006b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:26.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.125+0000 7f149ffff640 1 -- 172.21.15.32:0/2814820838 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14a00d3a10 con 0x7f14a00d4210 2024-04-24T23:01:26.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.125+0000 7f149ffff640 1 --2- 172.21.15.32:0/2814820838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14a00d4210 0x7f14a006b110 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f14a00b61d0 tx=0x7f149402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b6a989618e693c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:26.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.126+0000 7f149f7fe640 1 -- 172.21.15.32:0/2814820838 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1494037070 con 0x7f14a00d4210 2024-04-24T23:01:26.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.126+0000 7f149f7fe640 1 -- 172.21.15.32:0/2814820838 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1494032440 con 0x7f14a00d4210 2024-04-24T23:01:26.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.126+0000 7f149f7fe640 1 -- 172.21.15.32:0/2814820838 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1494032810 con 0x7f14a00d4210 2024-04-24T23:01:26.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.126+0000 7f14a6ec1640 1 -- 172.21.15.32:0/2814820838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14a00d4210 msgr2=0x7f14a006b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:26.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.126+0000 7f14a6ec1640 1 --2- 172.21.15.32:0/2814820838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14a00d4210 0x7f14a006b110 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f14a00b61d0 tx=0x7f149402a1e0 comp rx=0 tx=0).stop 2024-04-24T23:01:26.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.126+0000 7f14a6ec1640 1 -- 172.21.15.32:0/2814820838 shutdown_connections 2024-04-24T23:01:26.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.126+0000 7f14a6ec1640 1 --2- 172.21.15.32:0/2814820838 >> v2:172.21.15.121:3300/0 conn(0x7f14a00ba0e0 0x7f14a006b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:26.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.126+0000 7f14a6ec1640 1 --2- 172.21.15.32:0/2814820838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14a00d4210 0x7f14a006b110 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:26.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.126+0000 7f14a6ec1640 1 --2- 172.21.15.32:0/2814820838 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14a015e9b0 0x7f14a00d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:26.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.126+0000 7f14a6ec1640 1 -- 172.21.15.32:0/2814820838 >> 172.21.15.32:0/2814820838 conn(0x7f14a00d11d0 msgr2=0x7f14a00d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:26.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.126+0000 7f14a6ec1640 1 -- 172.21.15.32:0/2814820838 shutdown_connections 2024-04-24T23:01:26.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.126+0000 7f14a6ec1640 1 -- 172.21.15.32:0/2814820838 wait complete. 2024-04-24T23:01:26.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.127+0000 7f14a6ec1640 1 Processor -- start 2024-04-24T23:01:26.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.127+0000 7f14a6ec1640 1 -- start start 2024-04-24T23:01:26.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.128+0000 7f14a6ec1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14a00ba0e0 0x7f14a00c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:26.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.128+0000 7f14a6ec1640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14a00d4210 0x7f14a00cd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:26.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.128+0000 7f14a6ec1640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14a015e9b0 0x7f14a00cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:26.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.128+0000 7f14a6ec1640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f14a0071730 con 0x7f14a00ba0e0 2024-04-24T23:01:26.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.128+0000 7f14a6ec1640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f14a00d2400 con 0x7f14a015e9b0 2024-04-24T23:01:26.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.128+0000 7f14a6ec1640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f14a00718b0 con 0x7f14a00d4210 2024-04-24T23:01:26.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.128+0000 7f14a4c5d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14a00ba0e0 0x7f14a00c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:26.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.128+0000 7f14a4c5d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14a00ba0e0 0x7f14a00c82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41790/0 (socket says 172.21.15.32:41790) 2024-04-24T23:01:26.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.128+0000 7f14a4c5d640 1 -- 172.21.15.32:0/420610746 learned_addr learned my addr 172.21.15.32:0/420610746 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:26.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.129+0000 7f14a545e640 1 --2- 172.21.15.32:0/420610746 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14a015e9b0 0x7f14a00cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:26.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.129+0000 7f149ffff640 1 --2- 172.21.15.32:0/420610746 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14a00d4210 0x7f14a00cd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:26.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.129+0000 7f14a4c5d640 1 -- 172.21.15.32:0/420610746 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14a00d4210 msgr2=0x7f14a00cd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:26.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.129+0000 7f14a4c5d640 1 --2- 172.21.15.32:0/420610746 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14a00d4210 0x7f14a00cd9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:26.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.129+0000 7f14a4c5d640 1 -- 172.21.15.32:0/420610746 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14a015e9b0 msgr2=0x7f14a00cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:26.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.129+0000 7f14a4c5d640 1 --2- 172.21.15.32:0/420610746 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14a015e9b0 0x7f14a00cdf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:26.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.129+0000 7f14a4c5d640 1 -- 172.21.15.32:0/420610746 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14a015f1e0 con 0x7f14a00ba0e0 2024-04-24T23:01:26.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.129+0000 7f149ffff640 1 --2- 172.21.15.32:0/420610746 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14a00d4210 0x7f14a00cd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:01:26.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.129+0000 7f14a4c5d640 1 --2- 172.21.15.32:0/420610746 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14a00ba0e0 0x7f14a00c82f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f1498007e70 tx=0x7f149800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:26.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.129+0000 7f149d7fa640 1 -- 172.21.15.32:0/420610746 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1498014620 con 0x7f14a00ba0e0 2024-04-24T23:01:26.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.129+0000 7f149d7fa640 1 -- 172.21.15.32:0/420610746 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1498019070 con 0x7f14a00ba0e0 2024-04-24T23:01:26.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.129+0000 7f14a6ec1640 1 -- 172.21.15.32:0/420610746 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14a015f4c0 con 0x7f14a00ba0e0 2024-04-24T23:01:26.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.129+0000 7f149d7fa640 1 -- 172.21.15.32:0/420610746 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1498014a00 con 0x7f14a00ba0e0 2024-04-24T23:01:26.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.129+0000 7f14a6ec1640 1 -- 172.21.15.32:0/420610746 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14a015fa80 con 0x7f14a00ba0e0 2024-04-24T23:01:26.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.130+0000 7f14a6ec1640 1 -- 172.21.15.32:0/420610746 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14a015e210 con 0x7f14a00ba0e0 2024-04-24T23:01:26.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.130+0000 7f149d7fa640 1 -- 172.21.15.32:0/420610746 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1498014ba0 con 0x7f14a00ba0e0 2024-04-24T23:01:26.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.131+0000 7f149d7fa640 1 --2- 172.21.15.32:0/420610746 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f148c06be30 0x7f148c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:26.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.131+0000 7f149d7fa640 1 -- 172.21.15.32:0/420610746 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5709+0+0 (secure 0 0 0) 0x7f149808b680 con 0x7f14a00ba0e0 2024-04-24T23:01:26.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.131+0000 7f149ffff640 1 --2- 172.21.15.32:0/420610746 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f148c06be30 0x7f148c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:26.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.132+0000 7f149ffff640 1 --2- 172.21.15.32:0/420610746 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f148c06be30 0x7f148c06e2f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1494034000 tx=0x7f149402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:26.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.136+0000 7f149d7fa640 1 -- 172.21.15.32:0/420610746 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f14a015e210 con 0x7f14a00ba0e0 2024-04-24T23:01:26.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.321+0000 7f14a6ec1640 1 -- 172.21.15.32:0/420610746 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f14a0068430 con 0x7f14a00ba0e0 2024-04-24T23:01:26.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.322+0000 7f149d7fa640 1 -- 172.21.15.32:0/420610746 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) ==== 74+0+16690 (secure 0 0 0) 0x7f149805a0c0 con 0x7f14a00ba0e0 2024-04-24T23:01:26.324 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:01:26.324 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":66,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:01:25.027996+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:01:23.328944+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-24T23:01:24.010365+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:01:26.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.324+0000 7f14a6ec1640 1 -- 172.21.15.32:0/420610746 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f148c06be30 msgr2=0x7f148c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:26.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.324+0000 7f14a6ec1640 1 --2- 172.21.15.32:0/420610746 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f148c06be30 0x7f148c06e2f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1494034000 tx=0x7f149402e000 comp rx=0 tx=0).stop 2024-04-24T23:01:26.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.324+0000 7f14a6ec1640 1 -- 172.21.15.32:0/420610746 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14a00ba0e0 msgr2=0x7f14a00c82f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:26.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.324+0000 7f14a6ec1640 1 --2- 172.21.15.32:0/420610746 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14a00ba0e0 0x7f14a00c82f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f1498007e70 tx=0x7f149800e4b0 comp rx=0 tx=0).stop 2024-04-24T23:01:26.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.324+0000 7f14a6ec1640 1 -- 172.21.15.32:0/420610746 shutdown_connections 2024-04-24T23:01:26.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.324+0000 7f14a6ec1640 1 --2- 172.21.15.32:0/420610746 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f148c06be30 0x7f148c06e2f0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:26.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.324+0000 7f14a6ec1640 1 --2- 172.21.15.32:0/420610746 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14a015e9b0 0x7f14a00cdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:26.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.324+0000 7f14a6ec1640 1 --2- 172.21.15.32:0/420610746 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14a00d4210 0x7f14a00cd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:26.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.324+0000 7f14a6ec1640 1 --2- 172.21.15.32:0/420610746 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14a00ba0e0 0x7f14a00c82f0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:26.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.324+0000 7f14a6ec1640 1 -- 172.21.15.32:0/420610746 >> 172.21.15.32:0/420610746 conn(0x7f14a00d11d0 msgr2=0x7f14a0159a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:26.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.324+0000 7f14a6ec1640 1 -- 172.21.15.32:0/420610746 shutdown_connections 2024-04-24T23:01:26.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.324+0000 7f14a6ec1640 1 -- 172.21.15.32:0/420610746 wait complete. 2024-04-24T23:01:26.337 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-24T23:01:26.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.439+0000 7f5182110640 1 Processor -- start 2024-04-24T23:01:26.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.439+0000 7f5182110640 1 -- start start 2024-04-24T23:01:26.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.439+0000 7f5182110640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f517c06b820 0x7f517c0d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:26.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.439+0000 7f5182110640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f517c0d3170 con 0x7f517c15e9b0 2024-04-24T23:01:26.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.439+0000 7f5182110640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f517c0d3450 con 0x7f517c0d4210 2024-04-24T23:01:26.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.439+0000 7f5182110640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f517c0d3730 con 0x7f517c06b820 2024-04-24T23:01:26.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.440+0000 7f517affd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f517c06b820 0x7f517c0d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:26.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.440+0000 7f517b7fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f517c15e9b0 0x7f517c06b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43060/0 (socket says 172.21.15.32:43060) 2024-04-24T23:01:26.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.440+0000 7f517b7fe640 1 -- 172.21.15.32:0/3612161022 learned_addr learned my addr 172.21.15.32:0/3612161022 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:01:26.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.440+0000 7f517affd640 1 -- 172.21.15.32:0/3612161022 >> v1:172.21.15.121:6789/0 conn(0x7f517c0d4210 legacy=0x7f517c0b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:26.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.440+0000 7f517affd640 1 -- 172.21.15.32:0/3612161022 >> v1:172.21.15.32:6789/0 conn(0x7f517c15e9b0 legacy=0x7f517c06b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:26.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.440+0000 7f517affd640 1 -- 172.21.15.32:0/3612161022 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f517c0d3a10 con 0x7f517c06b820 2024-04-24T23:01:26.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.441+0000 7f517affd640 1 --2- 172.21.15.32:0/3612161022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f517c06b820 0x7f517c0d2b90 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f51700097a0 tx=0x7f517002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2716cf97f8fe070 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:26.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.441+0000 7f517a7fc640 1 -- 172.21.15.32:0/3612161022 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5170037070 con 0x7f517c06b820 2024-04-24T23:01:26.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.441+0000 7f517a7fc640 1 -- 172.21.15.32:0/3612161022 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5170032440 con 0x7f517c06b820 2024-04-24T23:01:26.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.441+0000 7f517a7fc640 1 -- 172.21.15.32:0/3612161022 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5170032810 con 0x7f517c06b820 2024-04-24T23:01:26.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.441+0000 7f5182110640 1 -- 172.21.15.32:0/3612161022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f517c06b820 msgr2=0x7f517c0d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:26.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.441+0000 7f5182110640 1 --2- 172.21.15.32:0/3612161022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f517c06b820 0x7f517c0d2b90 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f51700097a0 tx=0x7f517002a1a0 comp rx=0 tx=0).stop 2024-04-24T23:01:26.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.441+0000 7f5182110640 1 -- 172.21.15.32:0/3612161022 shutdown_connections 2024-04-24T23:01:26.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.441+0000 7f5182110640 1 --2- 172.21.15.32:0/3612161022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f517c06b820 0x7f517c0d2b90 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:26.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.441+0000 7f5182110640 1 -- 172.21.15.32:0/3612161022 >> 172.21.15.32:0/3612161022 conn(0x7f517c0d11d0 msgr2=0x7f517c0d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:26.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.441+0000 7f5182110640 1 -- 172.21.15.32:0/3612161022 shutdown_connections 2024-04-24T23:01:26.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.441+0000 7f5182110640 1 -- 172.21.15.32:0/3612161022 wait complete. 2024-04-24T23:01:26.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.442+0000 7f5182110640 1 Processor -- start 2024-04-24T23:01:26.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.443+0000 7f5182110640 1 -- start start 2024-04-24T23:01:26.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.443+0000 7f5182110640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f517c06b820 0x7f517c0cb820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:26.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.443+0000 7f5182110640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f517c0d4210 0x7f517c158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:26.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.443+0000 7f5182110640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f517c15e9b0 0x7f517c14d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:26.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.443+0000 7f5182110640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f517c0ba0e0 con 0x7f517c06b820 2024-04-24T23:01:26.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.443+0000 7f5182110640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f517c0d2400 con 0x7f517c0d4210 2024-04-24T23:01:26.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.443+0000 7f5182110640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f517c0ba260 con 0x7f517c15e9b0 2024-04-24T23:01:26.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.443+0000 7f517b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f517c06b820 0x7f517c0cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:26.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.444+0000 7f517b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f517c06b820 0x7f517c0cb820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41806/0 (socket says 172.21.15.32:41806) 2024-04-24T23:01:26.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.444+0000 7f517b7fe640 1 -- 172.21.15.32:0/3396963819 learned_addr learned my addr 172.21.15.32:0/3396963819 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:26.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.444+0000 7f517affd640 1 --2- 172.21.15.32:0/3396963819 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f517c0d4210 0x7f517c158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:26.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.444+0000 7f517bfff640 1 --2- 172.21.15.32:0/3396963819 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f517c15e9b0 0x7f517c14d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:26.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.444+0000 7f517b7fe640 1 -- 172.21.15.32:0/3396963819 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f517c15e9b0 msgr2=0x7f517c14d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:26.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.444+0000 7f517b7fe640 1 --2- 172.21.15.32:0/3396963819 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f517c15e9b0 0x7f517c14d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:26.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.444+0000 7f517b7fe640 1 -- 172.21.15.32:0/3396963819 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f517c0d4210 msgr2=0x7f517c158400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:26.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.444+0000 7f517b7fe640 1 --2- 172.21.15.32:0/3396963819 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f517c0d4210 0x7f517c158400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:26.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.444+0000 7f517b7fe640 1 -- 172.21.15.32:0/3396963819 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f517c0cc120 con 0x7f517c06b820 2024-04-24T23:01:26.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.444+0000 7f517b7fe640 1 --2- 172.21.15.32:0/3396963819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f517c06b820 0x7f517c0cb820 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f516c00c4b0 tx=0x7f516c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:26.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.444+0000 7f5163fff640 1 -- 172.21.15.32:0/3396963819 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516c017070 con 0x7f517c06b820 2024-04-24T23:01:26.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.444+0000 7f5163fff640 1 -- 172.21.15.32:0/3396963819 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f516c005ce0 con 0x7f517c06b820 2024-04-24T23:01:26.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.444+0000 7f5182110640 1 -- 172.21.15.32:0/3396963819 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f517c0cc400 con 0x7f517c06b820 2024-04-24T23:01:26.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.444+0000 7f5163fff640 1 -- 172.21.15.32:0/3396963819 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516c007270 con 0x7f517c06b820 2024-04-24T23:01:26.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.444+0000 7f5182110640 1 -- 172.21.15.32:0/3396963819 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f517c0ccca0 con 0x7f517c06b820 2024-04-24T23:01:26.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.445+0000 7f5182110640 1 -- 172.21.15.32:0/3396963819 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f517c068430 con 0x7f517c06b820 2024-04-24T23:01:26.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.446+0000 7f5163fff640 1 -- 172.21.15.32:0/3396963819 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f516c01a070 con 0x7f517c06b820 2024-04-24T23:01:26.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.446+0000 7f5163fff640 1 --2- 172.21.15.32:0/3396963819 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f516806be30 0x7f516806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:26.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.446+0000 7f5163fff640 1 -- 172.21.15.32:0/3396963819 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5709+0+0 (secure 0 0 0) 0x7f516c08a030 con 0x7f517c06b820 2024-04-24T23:01:26.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.446+0000 7f517affd640 1 --2- 172.21.15.32:0/3396963819 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f516806be30 0x7f516806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:26.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.447+0000 7f517affd640 1 --2- 172.21.15.32:0/3396963819 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f516806be30 0x7f516806e2f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5170034000 tx=0x7f517002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:26.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.450+0000 7f5163fff640 1 -- 172.21.15.32:0/3396963819 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f516c058420 con 0x7f517c06b820 2024-04-24T23:01:26.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.672+0000 7f5182110640 1 -- 172.21.15.32:0/3396963819 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f517c0cc5d0 con 0x7f517c06b820 2024-04-24T23:01:26.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.694+0000 7f5163fff640 1 -- 172.21.15.32:0/3396963819 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v26) ==== 118+0+0 (secure 0 0 0) 0x7f516c05c720 con 0x7f517c06b820 2024-04-24T23:01:26.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.696+0000 7f5182110640 1 -- 172.21.15.32:0/3396963819 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f516806be30 msgr2=0x7f516806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:26.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.696+0000 7f5182110640 1 --2- 172.21.15.32:0/3396963819 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f516806be30 0x7f516806e2f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5170034000 tx=0x7f517002e000 comp rx=0 tx=0).stop 2024-04-24T23:01:26.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.696+0000 7f5182110640 1 -- 172.21.15.32:0/3396963819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f517c06b820 msgr2=0x7f517c0cb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:26.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.696+0000 7f5182110640 1 --2- 172.21.15.32:0/3396963819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f517c06b820 0x7f517c0cb820 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f516c00c4b0 tx=0x7f516c00c970 comp rx=0 tx=0).stop 2024-04-24T23:01:26.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.696+0000 7f5182110640 1 -- 172.21.15.32:0/3396963819 shutdown_connections 2024-04-24T23:01:26.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.696+0000 7f5182110640 1 --2- 172.21.15.32:0/3396963819 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f516806be30 0x7f516806e2f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:26.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.696+0000 7f5182110640 1 --2- 172.21.15.32:0/3396963819 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f517c15e9b0 0x7f517c14d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:26.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.696+0000 7f5182110640 1 --2- 172.21.15.32:0/3396963819 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f517c0d4210 0x7f517c158400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:26.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.696+0000 7f5182110640 1 --2- 172.21.15.32:0/3396963819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f517c06b820 0x7f517c0cb820 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:26.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.696+0000 7f5182110640 1 -- 172.21.15.32:0/3396963819 >> 172.21.15.32:0/3396963819 conn(0x7f517c0d11d0 msgr2=0x7f517c0b8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:26.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.696+0000 7f5182110640 1 -- 172.21.15.32:0/3396963819 shutdown_connections 2024-04-24T23:01:26.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.696+0000 7f5182110640 1 -- 172.21.15.32:0/3396963819 wait complete. 2024-04-24T23:01:26.708 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-24T23:01:26.709 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-24T23:01:26.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.811+0000 7f98fe01c640 1 Processor -- start 2024-04-24T23:01:26.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.811+0000 7f98fe01c640 1 -- start start 2024-04-24T23:01:26.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.812+0000 7f98fe01c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f98f815e9b0 0x7f98f80d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:26.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.812+0000 7f98fe01c640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f98f80d4210 0x7f98f806b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:26.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.812+0000 7f98fe01c640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98f80d3170 con 0x7f98f80ba0e0 2024-04-24T23:01:26.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.812+0000 7f98fe01c640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f98f80b8900 con 0x7f98f80d4210 2024-04-24T23:01:26.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.812+0000 7f98fe01c640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f98f80d3450 con 0x7f98f815e9b0 2024-04-24T23:01:26.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.812+0000 7f98f7fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f98f80ba0e0 0x7f98f8071730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43076/0 (socket says 172.21.15.32:43076) 2024-04-24T23:01:26.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.812+0000 7f98f7fff640 1 -- 172.21.15.32:0/299901524 learned_addr learned my addr 172.21.15.32:0/299901524 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:01:26.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.813+0000 7f98f77fe640 1 --2- 172.21.15.32:0/299901524 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f98f815e9b0 0x7f98f80d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:26.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.813+0000 7f98f6ffd640 1 --2- 172.21.15.32:0/299901524 >> v2:172.21.15.121:3300/0 conn(0x7f98f80d4210 0x7f98f806b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:26.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.813+0000 7f98f67fc640 1 -- 172.21.15.32:0/299901524 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4100303470 0 0) 0x7f98f80d3170 con 0x7f98f80ba0e0 2024-04-24T23:01:26.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.813+0000 7f98f67fc640 1 -- 172.21.15.32:0/299901524 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f98e0003520 con 0x7f98f80ba0e0 2024-04-24T23:01:26.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.813+0000 7f98f67fc640 1 -- 172.21.15.32:0/299901524 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3487521570 0 0) 0x7f98e0003520 con 0x7f98f80ba0e0 2024-04-24T23:01:26.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.813+0000 7f98f67fc640 1 -- 172.21.15.32:0/299901524 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f98f815e9b0 msgr2=0x7f98f80d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:26.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.813+0000 7f98f67fc640 1 --2- 172.21.15.32:0/299901524 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f98f815e9b0 0x7f98f80d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:26.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.813+0000 7f98f67fc640 1 -- 172.21.15.32:0/299901524 >> v2:172.21.15.121:3300/0 conn(0x7f98f80d4210 msgr2=0x7f98f806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:26.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.813+0000 7f98f67fc640 1 --2- 172.21.15.32:0/299901524 >> v2:172.21.15.121:3300/0 conn(0x7f98f80d4210 0x7f98f806b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:26.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.813+0000 7f98f67fc640 1 -- 172.21.15.32:0/299901524 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98f80d3730 con 0x7f98f80ba0e0 2024-04-24T23:01:26.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.813+0000 7f98f67fc640 1 -- 172.21.15.32:0/299901524 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f98f00019f0 con 0x7f98f80ba0e0 2024-04-24T23:01:26.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.813+0000 7f98f67fc640 1 -- 172.21.15.32:0/299901524 >> v1:172.21.15.32:6789/0 conn(0x7f98f80ba0e0 legacy=0x7f98f8071730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:26.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.813+0000 7f98f67fc640 1 --2- 172.21.15.32:0/299901524 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98e00039b0 0x7f98e0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:26.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.813+0000 7f98f67fc640 1 --2- 172.21.15.32:0/299901524 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f98f815e9b0 0x7f98e0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:26.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.813+0000 7f98f67fc640 1 --2- 172.21.15.32:0/299901524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98e0004d40 0x7f98e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:26.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.813+0000 7f98f67fc640 1 -- 172.21.15.32:0/299901524 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f98f80b8900 con 0x7f98e0004d40 2024-04-24T23:01:26.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.813+0000 7f98f67fc640 1 -- 172.21.15.32:0/299901524 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f98f80d3450 con 0x7f98e00039b0 2024-04-24T23:01:26.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.813+0000 7f98f67fc640 1 -- 172.21.15.32:0/299901524 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f98e000b9c0 con 0x7f98f815e9b0 2024-04-24T23:01:26.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.813+0000 7f98f7fff640 1 --2- 172.21.15.32:0/299901524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98e0004d40 0x7f98e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:26.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.813+0000 7f98f6ffd640 1 --2- 172.21.15.32:0/299901524 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f98f815e9b0 0x7f98e0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:26.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.813+0000 7f98f77fe640 1 --2- 172.21.15.32:0/299901524 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98e00039b0 0x7f98e0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:26.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.813+0000 7f98f7fff640 1 -- 172.21.15.32:0/299901524 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f98f815e9b0 msgr2=0x7f98e0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:26.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.813+0000 7f98f7fff640 1 --2- 172.21.15.32:0/299901524 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f98f815e9b0 0x7f98e0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:26.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.813+0000 7f98f7fff640 1 -- 172.21.15.32:0/299901524 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98e00039b0 msgr2=0x7f98e0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:26.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.813+0000 7f98f7fff640 1 --2- 172.21.15.32:0/299901524 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98e00039b0 0x7f98e0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:26.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.814+0000 7f98f7fff640 1 -- 172.21.15.32:0/299901524 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98e0003520 con 0x7f98e0004d40 2024-04-24T23:01:26.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.814+0000 7f98f7fff640 1 --2- 172.21.15.32:0/299901524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98e0004d40 0x7f98e00051c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f98f000a880 tx=0x7f98e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5289cfe0617189e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:26.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.814+0000 7f98f67fc640 1 -- 172.21.15.32:0/299901524 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f000a4a0 con 0x7f98e0004d40 2024-04-24T23:01:26.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.814+0000 7f98f67fc640 1 -- 172.21.15.32:0/299901524 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98f002f070 con 0x7f98e0004d40 2024-04-24T23:01:26.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.814+0000 7f98f67fc640 1 -- 172.21.15.32:0/299901524 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f002bd00 con 0x7f98e0004d40 2024-04-24T23:01:26.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.815+0000 7f98fe01c640 1 -- 172.21.15.32:0/299901524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98e0004d40 msgr2=0x7f98e00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:26.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.815+0000 7f98fe01c640 1 --2- 172.21.15.32:0/299901524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98e0004d40 0x7f98e00051c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f98f000a880 tx=0x7f98e0005830 comp rx=0 tx=0).stop 2024-04-24T23:01:26.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.815+0000 7f98f7fff640 1 -- 172.21.15.32:0/299901524 reap_dead start 2024-04-24T23:01:26.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.815+0000 7f98fe01c640 1 -- 172.21.15.32:0/299901524 shutdown_connections 2024-04-24T23:01:26.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.815+0000 7f98fe01c640 1 -- 172.21.15.32:0/299901524 >> 172.21.15.32:0/299901524 conn(0x7f98f80d11d0 msgr2=0x7f98f80d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:26.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.815+0000 7f98fe01c640 1 -- 172.21.15.32:0/299901524 shutdown_connections 2024-04-24T23:01:26.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.815+0000 7f98fe01c640 1 -- 172.21.15.32:0/299901524 wait complete. 2024-04-24T23:01:26.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.816+0000 7f98fe01c640 1 Processor -- start 2024-04-24T23:01:26.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.816+0000 7f98fe01c640 1 -- start start 2024-04-24T23:01:26.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.817+0000 7f98fe01c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98f814f2b0 0x7f98f80ce0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:26.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.817+0000 7f98fe01c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98f814f690 0x7f98f80cb820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:26.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.817+0000 7f98fe01c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f98f80cbd60 0x7f98f80cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:26.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.817+0000 7f98fe01c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f98f80d2400 con 0x7f98f814f690 2024-04-24T23:01:26.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.817+0000 7f98fe01c640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f98f80d3cf0 con 0x7f98f814f2b0 2024-04-24T23:01:26.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.817+0000 7f98fe01c640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f98f80d2b90 con 0x7f98f80cbd60 2024-04-24T23:01:26.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.817+0000 7f98f6ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98f814f690 0x7f98f80cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:26.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.817+0000 7f98f6ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98f814f690 0x7f98f80cb820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41836/0 (socket says 172.21.15.32:41836) 2024-04-24T23:01:26.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.817+0000 7f98f6ffd640 1 -- 172.21.15.32:0/2914442290 learned_addr learned my addr 172.21.15.32:0/2914442290 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:26.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.817+0000 7f98f77fe640 1 --2- 172.21.15.32:0/2914442290 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98f814f2b0 0x7f98f80ce0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:26.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.817+0000 7f98f7fff640 1 --2- 172.21.15.32:0/2914442290 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f98f80cbd60 0x7f98f80cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:26.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.817+0000 7f98f6ffd640 1 -- 172.21.15.32:0/2914442290 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f98f80cbd60 msgr2=0x7f98f80cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:26.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.817+0000 7f98f6ffd640 1 --2- 172.21.15.32:0/2914442290 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f98f80cbd60 0x7f98f80cc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:26.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.817+0000 7f98f6ffd640 1 -- 172.21.15.32:0/2914442290 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98f814f2b0 msgr2=0x7f98f80ce0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:26.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.817+0000 7f98f6ffd640 1 --2- 172.21.15.32:0/2914442290 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98f814f2b0 0x7f98f80ce0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:26.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.817+0000 7f98f6ffd640 1 -- 172.21.15.32:0/2914442290 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98f80ce630 con 0x7f98f814f690 2024-04-24T23:01:26.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.817+0000 7f98f7fff640 1 --2- 172.21.15.32:0/2914442290 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f98f80cbd60 0x7f98f80cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:01:26.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.817+0000 7f98f77fe640 1 --2- 172.21.15.32:0/2914442290 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98f814f2b0 0x7f98f80ce0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:01:26.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.818+0000 7f98f6ffd640 1 --2- 172.21.15.32:0/2914442290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98f814f690 0x7f98f80cb820 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f98ec011ad0 tx=0x7f98ec011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:26.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.818+0000 7f98dffff640 1 -- 172.21.15.32:0/2914442290 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ec00fb50 con 0x7f98f814f690 2024-04-24T23:01:26.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.818+0000 7f98dffff640 1 -- 172.21.15.32:0/2914442290 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98ec00fcf0 con 0x7f98f814f690 2024-04-24T23:01:26.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.818+0000 7f98fe01c640 1 -- 172.21.15.32:0/2914442290 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98f8157a10 con 0x7f98f814f690 2024-04-24T23:01:26.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.818+0000 7f98dffff640 1 -- 172.21.15.32:0/2914442290 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ec00fea0 con 0x7f98f814f690 2024-04-24T23:01:26.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.818+0000 7f98fe01c640 1 -- 172.21.15.32:0/2914442290 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98f80c8280 con 0x7f98f814f690 2024-04-24T23:01:26.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.819+0000 7f98fe01c640 1 -- 172.21.15.32:0/2914442290 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98f8068430 con 0x7f98f814f690 2024-04-24T23:01:26.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.819+0000 7f98dffff640 1 -- 172.21.15.32:0/2914442290 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f98ec012040 con 0x7f98f814f690 2024-04-24T23:01:26.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.820+0000 7f98dffff640 1 --2- 172.21.15.32:0/2914442290 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f98e406be60 0x7f98e406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:26.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.820+0000 7f98dffff640 1 -- 172.21.15.32:0/2914442290 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7f98ec08b490 con 0x7f98f814f690 2024-04-24T23:01:26.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.820+0000 7f98f77fe640 1 --2- 172.21.15.32:0/2914442290 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f98e406be60 0x7f98e406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:26.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.820+0000 7f98f77fe640 1 --2- 172.21.15.32:0/2914442290 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f98e406be60 0x7f98e406e320 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f98e8009a40 tx=0x7f98e8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:26.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:26.824+0000 7f98dffff640 1 -- 172.21.15.32:0/2914442290 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f98ec019030 con 0x7f98f814f690 2024-04-24T23:01:27.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.047+0000 7f98fe01c640 1 -- 172.21.15.32:0/2914442290 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f98f815db00 con 0x7f98f814f690 2024-04-24T23:01:27.658 INFO:tasks.ceph.mds.d.smithi121.stderr:2024-04-24T23:01:27.657+0000 7f8035f8a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T23:01:27.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.699+0000 7f98dffff640 1 -- 172.21.15.32:0/2914442290 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v27) ==== 122+0+0 (secure 0 0 0) 0x7f98ec059ed0 con 0x7f98f814f690 2024-04-24T23:01:27.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.702+0000 7f98fe01c640 1 -- 172.21.15.32:0/2914442290 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f98e406be60 msgr2=0x7f98e406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:27.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.702+0000 7f98fe01c640 1 --2- 172.21.15.32:0/2914442290 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f98e406be60 0x7f98e406e320 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f98e8009a40 tx=0x7f98e8008040 comp rx=0 tx=0).stop 2024-04-24T23:01:27.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.702+0000 7f98fe01c640 1 -- 172.21.15.32:0/2914442290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98f814f690 msgr2=0x7f98f80cb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:27.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.702+0000 7f98fe01c640 1 --2- 172.21.15.32:0/2914442290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98f814f690 0x7f98f80cb820 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f98ec011ad0 tx=0x7f98ec011f90 comp rx=0 tx=0).stop 2024-04-24T23:01:27.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.702+0000 7f98fe01c640 1 -- 172.21.15.32:0/2914442290 shutdown_connections 2024-04-24T23:01:27.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.702+0000 7f98fe01c640 1 --2- 172.21.15.32:0/2914442290 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f98e406be60 0x7f98e406e320 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:27.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.702+0000 7f98fe01c640 1 --2- 172.21.15.32:0/2914442290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98f814f690 0x7f98f80cb820 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:27.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.702+0000 7f98fe01c640 1 --2- 172.21.15.32:0/2914442290 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f98f80cbd60 0x7f98f80cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:27.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.702+0000 7f98fe01c640 1 --2- 172.21.15.32:0/2914442290 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98f814f2b0 0x7f98f80ce0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:27.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.702+0000 7f98fe01c640 1 -- 172.21.15.32:0/2914442290 >> 172.21.15.32:0/2914442290 conn(0x7f98f80d11d0 msgr2=0x7f98f80cefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:27.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.702+0000 7f98fe01c640 1 -- 172.21.15.32:0/2914442290 shutdown_connections 2024-04-24T23:01:27.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.702+0000 7f98fe01c640 1 -- 172.21.15.32:0/2914442290 wait complete. 2024-04-24T23:01:27.717 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-24T23:01:27.717 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:01:27.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.823+0000 7f90f64ee640 1 Processor -- start 2024-04-24T23:01:27.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.823+0000 7f90f64ee640 1 -- start start 2024-04-24T23:01:27.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.824+0000 7f90f64ee640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f90f006b820 0x7f90f00d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:27.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.824+0000 7f90f64ee640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f90f00d4210 0x7f90f00b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:27.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.824+0000 7f90f64ee640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90f00d3170 con 0x7f90f015e9b0 2024-04-24T23:01:27.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.824+0000 7f90f64ee640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f90f00d3450 con 0x7f90f006b820 2024-04-24T23:01:27.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.824+0000 7f90f64ee640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f90f00d3730 con 0x7f90f00d4210 2024-04-24T23:01:27.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.824+0000 7f90ef7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f90f006b820 0x7f90f00d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:27.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.824+0000 7f90effff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f90f015e9b0 0x7f90f006b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43082/0 (socket says 172.21.15.32:43082) 2024-04-24T23:01:27.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.824+0000 7f90ef7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f90f006b820 0x7f90f00d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41838/0 (socket says 172.21.15.32:41838) 2024-04-24T23:01:27.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.824+0000 7f90effff640 1 -- 172.21.15.32:0/3352148879 learned_addr learned my addr 172.21.15.32:0/3352148879 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:01:27.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.825+0000 7f90f4a8b640 1 --2- 172.21.15.32:0/3352148879 >> v2:172.21.15.121:3300/0 conn(0x7f90f00d4210 0x7f90f00b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:27.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.825+0000 7f90ef7fe640 1 -- 172.21.15.32:0/3352148879 >> v2:172.21.15.121:3300/0 conn(0x7f90f00d4210 msgr2=0x7f90f00b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:27.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.825+0000 7f90ef7fe640 1 --2- 172.21.15.32:0/3352148879 >> v2:172.21.15.121:3300/0 conn(0x7f90f00d4210 0x7f90f00b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:27.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.825+0000 7f90ef7fe640 1 -- 172.21.15.32:0/3352148879 >> v1:172.21.15.32:6789/0 conn(0x7f90f015e9b0 legacy=0x7f90f006b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:27.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.825+0000 7f90eeffd640 1 -- 172.21.15.32:0/3352148879 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3003166546 0 0) 0x7f90f00d3170 con 0x7f90f015e9b0 2024-04-24T23:01:27.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.825+0000 7f90ef7fe640 1 -- 172.21.15.32:0/3352148879 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90f00d3a10 con 0x7f90f006b820 2024-04-24T23:01:27.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.825+0000 7f90ef7fe640 1 --2- 172.21.15.32:0/3352148879 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90f006b820 0x7f90f00d2b90 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f90f01622f0 tx=0x7f90e002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e463904877a08add server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:27.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.826+0000 7f90eeffd640 1 -- 172.21.15.32:0/3352148879 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e0037070 con 0x7f90f006b820 2024-04-24T23:01:27.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.826+0000 7f90eeffd640 1 -- 172.21.15.32:0/3352148879 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90e0032440 con 0x7f90f006b820 2024-04-24T23:01:27.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.826+0000 7f90eeffd640 1 -- 172.21.15.32:0/3352148879 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e002ae60 con 0x7f90f006b820 2024-04-24T23:01:27.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.826+0000 7f90f64ee640 1 -- 172.21.15.32:0/3352148879 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90f006b820 msgr2=0x7f90f00d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:27.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.826+0000 7f90f64ee640 1 --2- 172.21.15.32:0/3352148879 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90f006b820 0x7f90f00d2b90 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f90f01622f0 tx=0x7f90e002a110 comp rx=0 tx=0).stop 2024-04-24T23:01:27.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.826+0000 7f90f64ee640 1 -- 172.21.15.32:0/3352148879 shutdown_connections 2024-04-24T23:01:27.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.826+0000 7f90f64ee640 1 --2- 172.21.15.32:0/3352148879 >> v2:172.21.15.121:3300/0 conn(0x7f90f00d4210 0x7f90f00b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:27.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.826+0000 7f90f64ee640 1 --2- 172.21.15.32:0/3352148879 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90f006b820 0x7f90f00d2b90 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:27.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.826+0000 7f90f64ee640 1 -- 172.21.15.32:0/3352148879 >> 172.21.15.32:0/3352148879 conn(0x7f90f00d11d0 msgr2=0x7f90f00d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:27.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.826+0000 7f90f64ee640 1 -- 172.21.15.32:0/3352148879 shutdown_connections 2024-04-24T23:01:27.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.826+0000 7f90f64ee640 1 -- 172.21.15.32:0/3352148879 wait complete. 2024-04-24T23:01:27.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.827+0000 7f90f64ee640 1 Processor -- start 2024-04-24T23:01:27.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.828+0000 7f90f64ee640 1 -- start start 2024-04-24T23:01:27.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.828+0000 7f90f64ee640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f90f006b820 0x7f90f0158ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:27.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.828+0000 7f90f64ee640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90f00d4210 0x7f90f00ce450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:27.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.828+0000 7f90f64ee640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90f015e9b0 0x7f90f0152cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:27.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.828+0000 7f90f64ee640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f90f00ba0e0 con 0x7f90f015e9b0 2024-04-24T23:01:27.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.828+0000 7f90f64ee640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f90f00d2400 con 0x7f90f00d4210 2024-04-24T23:01:27.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.828+0000 7f90f64ee640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f90f00ba260 con 0x7f90f006b820 2024-04-24T23:01:27.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.828+0000 7f90f4a8b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90f015e9b0 0x7f90f0152cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:27.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.828+0000 7f90f4a8b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90f015e9b0 0x7f90f0152cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41854/0 (socket says 172.21.15.32:41854) 2024-04-24T23:01:27.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.829+0000 7f90f4a8b640 1 -- 172.21.15.32:0/3592373571 learned_addr learned my addr 172.21.15.32:0/3592373571 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:27.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.829+0000 7f90ef7fe640 1 --2- 172.21.15.32:0/3592373571 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90f00d4210 0x7f90f00ce450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:27.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.829+0000 7f90effff640 1 --2- 172.21.15.32:0/3592373571 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f90f006b820 0x7f90f0158ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:27.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.829+0000 7f90f4a8b640 1 -- 172.21.15.32:0/3592373571 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f90f006b820 msgr2=0x7f90f0158ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:27.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.829+0000 7f90f4a8b640 1 --2- 172.21.15.32:0/3592373571 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f90f006b820 0x7f90f0158ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:27.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.829+0000 7f90f4a8b640 1 -- 172.21.15.32:0/3592373571 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90f00d4210 msgr2=0x7f90f00ce450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:27.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.829+0000 7f90f4a8b640 1 --2- 172.21.15.32:0/3592373571 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90f00d4210 0x7f90f00ce450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:27.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.829+0000 7f90f4a8b640 1 -- 172.21.15.32:0/3592373571 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90f00cc9a0 con 0x7f90f015e9b0 2024-04-24T23:01:27.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.829+0000 7f90f4a8b640 1 --2- 172.21.15.32:0/3592373571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90f015e9b0 0x7f90f0152cd0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f90e8007e40 tx=0x7f90e800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:27.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.829+0000 7f90ecff9640 1 -- 172.21.15.32:0/3592373571 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e8014620 con 0x7f90f015e9b0 2024-04-24T23:01:27.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.829+0000 7f90ecff9640 1 -- 172.21.15.32:0/3592373571 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90e8019070 con 0x7f90f015e9b0 2024-04-24T23:01:27.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.829+0000 7f90f64ee640 1 -- 172.21.15.32:0/3592373571 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90f00ccc80 con 0x7f90f015e9b0 2024-04-24T23:01:27.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.829+0000 7f90ecff9640 1 -- 172.21.15.32:0/3592373571 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e8014a20 con 0x7f90f015e9b0 2024-04-24T23:01:27.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.829+0000 7f90f64ee640 1 -- 172.21.15.32:0/3592373571 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90f00cd240 con 0x7f90f015e9b0 2024-04-24T23:01:27.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.830+0000 7f90f64ee640 1 -- 172.21.15.32:0/3592373571 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90f0068430 con 0x7f90f015e9b0 2024-04-24T23:01:27.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.831+0000 7f90ecff9640 1 -- 172.21.15.32:0/3592373571 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f90e8014bc0 con 0x7f90f015e9b0 2024-04-24T23:01:27.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.831+0000 7f90ecff9640 1 --2- 172.21.15.32:0/3592373571 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f90d006be30 0x7f90d006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:27.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.831+0000 7f90ecff9640 1 -- 172.21.15.32:0/3592373571 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7f90e808b630 con 0x7f90f015e9b0 2024-04-24T23:01:27.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.831+0000 7f90effff640 1 --2- 172.21.15.32:0/3592373571 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f90d006be30 0x7f90d006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:27.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.832+0000 7f90effff640 1 --2- 172.21.15.32:0/3592373571 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f90d006be30 0x7f90d006e2f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f90d8000f60 tx=0x7f90d8007c00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:27.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:27.835+0000 7f90ecff9640 1 -- 172.21.15.32:0/3592373571 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f90e805a050 con 0x7f90f015e9b0 2024-04-24T23:01:28.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.055+0000 7f90f64ee640 1 -- 172.21.15.32:0/3592373571 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f90f00722c0 con 0x7f90f015e9b0 2024-04-24T23:01:28.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.055+0000 7f90ecff9640 1 -- 172.21.15.32:0/3592373571 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5187 (secure 0 0 0) 0x7f90e8059950 con 0x7f90f015e9b0 2024-04-24T23:01:28.057 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:01:28.057 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":27,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3085809520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3085809520},{"type":"v1","addr":"172.21.15.121:6837","nonce":3085809520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":6078,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/1405465715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1405465715},{"type":"v1","addr":"172.21.15.32:6834","nonce":1405465715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":6081,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1160416671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":1160416671},{"type":"v1","addr":"172.21.15.32:6835","nonce":1160416671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:01:25.028304+0000","modified":"2024-04-24T23:01:27.048925+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6090},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6090":{"gid":6090,"name":"d","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6836/2650840833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2650840833},{"type":"v1","addr":"172.21.15.121:6836","nonce":2650840833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6090,"qdb_cluster":[6090]},"id":4}]} 2024-04-24T23:01:28.057 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 27 2024-04-24T23:01:28.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.057+0000 7f90f64ee640 1 -- 172.21.15.32:0/3592373571 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f90d006be30 msgr2=0x7f90d006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:28.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.057+0000 7f90f64ee640 1 --2- 172.21.15.32:0/3592373571 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f90d006be30 0x7f90d006e2f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f90d8000f60 tx=0x7f90d8007c00 comp rx=0 tx=0).stop 2024-04-24T23:01:28.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.057+0000 7f90f64ee640 1 -- 172.21.15.32:0/3592373571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90f015e9b0 msgr2=0x7f90f0152cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:28.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.057+0000 7f90f64ee640 1 --2- 172.21.15.32:0/3592373571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90f015e9b0 0x7f90f0152cd0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f90e8007e40 tx=0x7f90e800e4b0 comp rx=0 tx=0).stop 2024-04-24T23:01:28.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.057+0000 7f90f64ee640 1 -- 172.21.15.32:0/3592373571 shutdown_connections 2024-04-24T23:01:28.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.057+0000 7f90f64ee640 1 --2- 172.21.15.32:0/3592373571 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f90d006be30 0x7f90d006e2f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:28.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.057+0000 7f90f64ee640 1 --2- 172.21.15.32:0/3592373571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90f015e9b0 0x7f90f0152cd0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:28.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.057+0000 7f90f64ee640 1 --2- 172.21.15.32:0/3592373571 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90f00d4210 0x7f90f00ce450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:28.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.057+0000 7f90f64ee640 1 --2- 172.21.15.32:0/3592373571 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f90f006b820 0x7f90f0158ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:28.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.057+0000 7f90f64ee640 1 -- 172.21.15.32:0/3592373571 >> 172.21.15.32:0/3592373571 conn(0x7f90f00d11d0 msgr2=0x7f90f015d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:28.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.057+0000 7f90f64ee640 1 -- 172.21.15.32:0/3592373571 shutdown_connections 2024-04-24T23:01:28.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.058+0000 7f90f64ee640 1 -- 172.21.15.32:0/3592373571 wait complete. 2024-04-24T23:01:28.070 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:01:28.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.169+0000 7fee75d1d640 1 Processor -- start 2024-04-24T23:01:28.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.169+0000 7fee75d1d640 1 -- start start 2024-04-24T23:01:28.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.170+0000 7fee75d1d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fee70162320 0x7fee700ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:28.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.170+0000 7fee75d1d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fee700b5af0 0x7fee70162700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:28.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.170+0000 7fee75d1d640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fee70162c40 0x7fee700ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:28.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.170+0000 7fee75d1d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fee700b5660 con 0x7fee700b5af0 2024-04-24T23:01:28.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.170+0000 7fee75d1d640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fee700cd270 con 0x7fee70162c40 2024-04-24T23:01:28.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.170+0000 7fee75d1d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fee70071610 con 0x7fee70162320 2024-04-24T23:01:28.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.170+0000 7fee6effd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fee700b5af0 0x7fee70162700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:28.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.170+0000 7fee6effd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fee700b5af0 0x7fee70162700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41856/0 (socket says 172.21.15.32:41856) 2024-04-24T23:01:28.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.170+0000 7fee6effd640 1 -- 172.21.15.32:0/841673660 learned_addr learned my addr 172.21.15.32:0/841673660 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:28.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.170+0000 7fee6f7fe640 1 --2- 172.21.15.32:0/841673660 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fee70162320 0x7fee700ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:28.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.170+0000 7fee6ffff640 1 --2- 172.21.15.32:0/841673660 >> v2:172.21.15.121:3300/0 conn(0x7fee70162c40 0x7fee700ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:28.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.171+0000 7fee6effd640 1 -- 172.21.15.32:0/841673660 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fee70162320 msgr2=0x7fee700ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:28.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.171+0000 7fee6effd640 1 --2- 172.21.15.32:0/841673660 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fee70162320 0x7fee700ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:28.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.171+0000 7fee6effd640 1 -- 172.21.15.32:0/841673660 >> v2:172.21.15.121:3300/0 conn(0x7fee70162c40 msgr2=0x7fee700ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:28.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.171+0000 7fee6effd640 1 --2- 172.21.15.32:0/841673660 >> v2:172.21.15.121:3300/0 conn(0x7fee70162c40 0x7fee700ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:28.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.171+0000 7fee6effd640 1 -- 172.21.15.32:0/841673660 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee700718f0 con 0x7fee700b5af0 2024-04-24T23:01:28.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.171+0000 7fee6effd640 1 --2- 172.21.15.32:0/841673660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee700b5af0 0x7fee70162700 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fee700b3ac0 tx=0x7fee6402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cfff989feb07159 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:28.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.171+0000 7fee6e7fc640 1 -- 172.21.15.32:0/841673660 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee64037070 con 0x7fee700b5af0 2024-04-24T23:01:28.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.171+0000 7fee6e7fc640 1 -- 172.21.15.32:0/841673660 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee64032440 con 0x7fee700b5af0 2024-04-24T23:01:28.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.171+0000 7fee6e7fc640 1 -- 172.21.15.32:0/841673660 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee64032810 con 0x7fee700b5af0 2024-04-24T23:01:28.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.172+0000 7fee75d1d640 1 -- 172.21.15.32:0/841673660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee700b5af0 msgr2=0x7fee70162700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:28.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.172+0000 7fee75d1d640 1 --2- 172.21.15.32:0/841673660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee700b5af0 0x7fee70162700 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fee700b3ac0 tx=0x7fee6402a140 comp rx=0 tx=0).stop 2024-04-24T23:01:28.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.172+0000 7fee75d1d640 1 -- 172.21.15.32:0/841673660 shutdown_connections 2024-04-24T23:01:28.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.172+0000 7fee75d1d640 1 --2- 172.21.15.32:0/841673660 >> v2:172.21.15.121:3300/0 conn(0x7fee70162c40 0x7fee700ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:28.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.172+0000 7fee75d1d640 1 --2- 172.21.15.32:0/841673660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee700b5af0 0x7fee70162700 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:28.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.172+0000 7fee75d1d640 1 --2- 172.21.15.32:0/841673660 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fee70162320 0x7fee700ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:28.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.172+0000 7fee75d1d640 1 -- 172.21.15.32:0/841673660 >> 172.21.15.32:0/841673660 conn(0x7fee700d0ae0 msgr2=0x7fee700d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:28.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.172+0000 7fee75d1d640 1 -- 172.21.15.32:0/841673660 shutdown_connections 2024-04-24T23:01:28.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.172+0000 7fee75d1d640 1 -- 172.21.15.32:0/841673660 wait complete. 2024-04-24T23:01:28.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.173+0000 7fee75d1d640 1 Processor -- start 2024-04-24T23:01:28.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.173+0000 7fee75d1d640 1 -- start start 2024-04-24T23:01:28.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.174+0000 7fee75d1d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee700b5af0 0x7fee7015b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:28.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.174+0000 7fee75d1d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fee70162320 0x7fee70160eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:28.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.174+0000 7fee75d1d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee70162c40 0x7fee701613f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:28.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.174+0000 7fee75d1d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fee70067950 con 0x7fee70162c40 2024-04-24T23:01:28.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.174+0000 7fee75d1d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fee7015c630 con 0x7fee700b5af0 2024-04-24T23:01:28.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.174+0000 7fee75d1d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fee70071bd0 con 0x7fee70162320 2024-04-24T23:01:28.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.174+0000 7fee6ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee70162c40 0x7fee701613f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:28.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.174+0000 7fee6ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee70162c40 0x7fee701613f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41866/0 (socket says 172.21.15.32:41866) 2024-04-24T23:01:28.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.174+0000 7fee6ffff640 1 -- 172.21.15.32:0/2097991052 learned_addr learned my addr 172.21.15.32:0/2097991052 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:28.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.175+0000 7fee6effd640 1 --2- 172.21.15.32:0/2097991052 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fee70162320 0x7fee70160eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:28.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.175+0000 7fee6f7fe640 1 --2- 172.21.15.32:0/2097991052 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee700b5af0 0x7fee7015b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:28.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.175+0000 7fee6ffff640 1 -- 172.21.15.32:0/2097991052 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fee70162320 msgr2=0x7fee70160eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:28.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.175+0000 7fee6ffff640 1 --2- 172.21.15.32:0/2097991052 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fee70162320 0x7fee70160eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:28.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.175+0000 7fee6ffff640 1 -- 172.21.15.32:0/2097991052 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee700b5af0 msgr2=0x7fee7015b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:28.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.175+0000 7fee6ffff640 1 --2- 172.21.15.32:0/2097991052 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee700b5af0 0x7fee7015b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:28.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.175+0000 7fee6ffff640 1 -- 172.21.15.32:0/2097991052 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee701e8350 con 0x7fee70162c40 2024-04-24T23:01:28.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.175+0000 7fee6f7fe640 1 --2- 172.21.15.32:0/2097991052 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee700b5af0 0x7fee7015b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:01:28.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.175+0000 7fee6ffff640 1 --2- 172.21.15.32:0/2097991052 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee70162c40 0x7fee701613f0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fee6800efc0 tx=0x7fee6800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:28.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.175+0000 7fee57fff640 1 -- 172.21.15.32:0/2097991052 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee68018020 con 0x7fee70162c40 2024-04-24T23:01:28.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.175+0000 7fee57fff640 1 -- 172.21.15.32:0/2097991052 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee68014480 con 0x7fee70162c40 2024-04-24T23:01:28.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.175+0000 7fee75d1d640 1 -- 172.21.15.32:0/2097991052 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee701e8630 con 0x7fee70162c40 2024-04-24T23:01:28.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.175+0000 7fee57fff640 1 -- 172.21.15.32:0/2097991052 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee68014870 con 0x7fee70162c40 2024-04-24T23:01:28.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.175+0000 7fee75d1d640 1 -- 172.21.15.32:0/2097991052 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee701e8bf0 con 0x7fee70162c40 2024-04-24T23:01:28.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.176+0000 7fee75d1d640 1 -- 172.21.15.32:0/2097991052 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee70068430 con 0x7fee70162c40 2024-04-24T23:01:28.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.177+0000 7fee57fff640 1 -- 172.21.15.32:0/2097991052 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fee68005d10 con 0x7fee70162c40 2024-04-24T23:01:28.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.177+0000 7fee57fff640 1 --2- 172.21.15.32:0/2097991052 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fee5c06be30 0x7fee5c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:28.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.177+0000 7fee57fff640 1 -- 172.21.15.32:0/2097991052 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7fee6808b9f0 con 0x7fee70162c40 2024-04-24T23:01:28.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.177+0000 7fee6f7fe640 1 --2- 172.21.15.32:0/2097991052 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fee5c06be30 0x7fee5c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:28.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.178+0000 7fee6f7fe640 1 --2- 172.21.15.32:0/2097991052 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fee5c06be30 0x7fee5c06e2f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fee60009a40 tx=0x7fee60008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:28.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.181+0000 7fee57fff640 1 -- 172.21.15.32:0/2097991052 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fee68022070 con 0x7fee70162c40 2024-04-24T23:01:28.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.366+0000 7fee75d1d640 1 -- 172.21.15.32:0/2097991052 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fee70067730 con 0x7fee70162c40 2024-04-24T23:01:28.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.367+0000 7fee57fff640 1 -- 172.21.15.32:0/2097991052 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) ==== 74+0+16677 (secure 0 0 0) 0x7fee68010040 con 0x7fee70162c40 2024-04-24T23:01:28.369 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:01:28.369 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":67,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:01:26.675300+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:01:23.328944+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-24T23:01:24.010365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:01:28.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.370+0000 7fee75d1d640 1 -- 172.21.15.32:0/2097991052 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fee5c06be30 msgr2=0x7fee5c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:28.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.370+0000 7fee75d1d640 1 --2- 172.21.15.32:0/2097991052 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fee5c06be30 0x7fee5c06e2f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fee60009a40 tx=0x7fee60008040 comp rx=0 tx=0).stop 2024-04-24T23:01:28.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.370+0000 7fee75d1d640 1 -- 172.21.15.32:0/2097991052 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee70162c40 msgr2=0x7fee701613f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:28.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.370+0000 7fee75d1d640 1 --2- 172.21.15.32:0/2097991052 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee70162c40 0x7fee701613f0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fee6800efc0 tx=0x7fee6800c490 comp rx=0 tx=0).stop 2024-04-24T23:01:28.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.370+0000 7fee75d1d640 1 -- 172.21.15.32:0/2097991052 shutdown_connections 2024-04-24T23:01:28.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.370+0000 7fee75d1d640 1 --2- 172.21.15.32:0/2097991052 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fee5c06be30 0x7fee5c06e2f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:28.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.370+0000 7fee75d1d640 1 --2- 172.21.15.32:0/2097991052 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee70162c40 0x7fee701613f0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:28.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.370+0000 7fee75d1d640 1 --2- 172.21.15.32:0/2097991052 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fee70162320 0x7fee70160eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:28.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.370+0000 7fee75d1d640 1 --2- 172.21.15.32:0/2097991052 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee700b5af0 0x7fee7015b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:28.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.370+0000 7fee75d1d640 1 -- 172.21.15.32:0/2097991052 >> 172.21.15.32:0/2097991052 conn(0x7fee700d0ae0 msgr2=0x7fee700b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:28.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.370+0000 7fee75d1d640 1 -- 172.21.15.32:0/2097991052 shutdown_connections 2024-04-24T23:01:28.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.370+0000 7fee75d1d640 1 -- 172.21.15.32:0/2097991052 wait complete. 2024-04-24T23:01:28.382 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-24T23:01:28.383 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:01:28.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.482+0000 7f6663528640 1 Processor -- start 2024-04-24T23:01:28.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.482+0000 7f6663528640 1 -- start start 2024-04-24T23:01:28.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.483+0000 7f6663528640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f665c06cd00 0x7f665c0ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:28.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.483+0000 7f6663528640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f665c15cb00 0x7f665c0ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:28.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.483+0000 7f6663528640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f665c0b4520 con 0x7f665c15b380 2024-04-24T23:01:28.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.483+0000 7f6663528640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f665c0b4800 con 0x7f665c06cd00 2024-04-24T23:01:28.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.483+0000 7f6663528640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f665c0b4ae0 con 0x7f665c15cb00 2024-04-24T23:01:28.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.483+0000 7f6660ac3640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f665c06cd00 0x7f665c0ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:28.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.483+0000 7f6660ac3640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f665c06cd00 0x7f665c0ce430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.32:52296/0 (socket says 172.21.15.32:52296) 2024-04-24T23:01:28.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.483+0000 7f6661ac5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f665c15cb00 0x7f665c0ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:28.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.483+0000 7f6660ac3640 1 -- 172.21.15.32:0/2768230056 learned_addr learned my addr 172.21.15.32:0/2768230056 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:28.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.483+0000 7f6660ac3640 1 -- 172.21.15.32:0/2768230056 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f665c15cb00 msgr2=0x7f665c0ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:28.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.483+0000 7f6660ac3640 1 --2- 172.21.15.32:0/2768230056 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f665c15cb00 0x7f665c0ce970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:28.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.483+0000 7f6660ac3640 1 -- 172.21.15.32:0/2768230056 >> v1:172.21.15.121:6789/0 conn(0x7f665c15b380 legacy=0x7f665c06c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:28.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.483+0000 7f6660ac3640 1 -- 172.21.15.32:0/2768230056 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f665c0b4dc0 con 0x7f665c06cd00 2024-04-24T23:01:28.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.483+0000 7f665bfff640 1 -- 172.21.15.32:0/2768230056 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2409869812 0 0) 0x7f665c0b4520 con 0x7f665c15b380 2024-04-24T23:01:28.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.483+0000 7f6661ac5640 1 --2- 172.21.15.32:0/2768230056 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f665c15cb00 0x7f665c0ce970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:01:28.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.484+0000 7f6660ac3640 1 --2- 172.21.15.32:0/2768230056 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f665c06cd00 0x7f665c0ce430 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6650009870 tx=0x7f665002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ba259b790d7338a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:28.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.484+0000 7f665bfff640 1 -- 172.21.15.32:0/2768230056 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6650037070 con 0x7f665c06cd00 2024-04-24T23:01:28.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.484+0000 7f665bfff640 1 -- 172.21.15.32:0/2768230056 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6650032440 con 0x7f665c06cd00 2024-04-24T23:01:28.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.484+0000 7f665bfff640 1 -- 172.21.15.32:0/2768230056 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6650032810 con 0x7f665c06cd00 2024-04-24T23:01:28.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.485+0000 7f6663528640 1 -- 172.21.15.32:0/2768230056 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f665c06cd00 msgr2=0x7f665c0ce430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:28.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.485+0000 7f6663528640 1 --2- 172.21.15.32:0/2768230056 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f665c06cd00 0x7f665c0ce430 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6650009870 tx=0x7f665002a1c0 comp rx=0 tx=0).stop 2024-04-24T23:01:28.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.485+0000 7f6663528640 1 -- 172.21.15.32:0/2768230056 shutdown_connections 2024-04-24T23:01:28.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.485+0000 7f6663528640 1 --2- 172.21.15.32:0/2768230056 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f665c15cb00 0x7f665c0ce970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:28.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.485+0000 7f6663528640 1 --2- 172.21.15.32:0/2768230056 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f665c06cd00 0x7f665c0ce430 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:28.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.485+0000 7f6663528640 1 -- 172.21.15.32:0/2768230056 >> 172.21.15.32:0/2768230056 conn(0x7f665c06b110 msgr2=0x7f665c06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:28.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.485+0000 7f6663528640 1 -- 172.21.15.32:0/2768230056 shutdown_connections 2024-04-24T23:01:28.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.485+0000 7f6663528640 1 -- 172.21.15.32:0/2768230056 wait complete. 2024-04-24T23:01:28.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.486+0000 7f6663528640 1 Processor -- start 2024-04-24T23:01:28.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.486+0000 7f6663528640 1 -- start start 2024-04-24T23:01:28.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.487+0000 7f6663528640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f665c06cd00 0x7f665c15a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:28.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.487+0000 7f6663528640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f665c15b380 0x7f665c1496a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:28.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.487+0000 7f6663528640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f665c15cb00 0x7f665c149be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:28.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.487+0000 7f6663528640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f665c15b760 con 0x7f665c06cd00 2024-04-24T23:01:28.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.487+0000 7f6663528640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f665c0ceeb0 con 0x7f665c15cb00 2024-04-24T23:01:28.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.487+0000 7f6663528640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f665c0b55c0 con 0x7f665c15b380 2024-04-24T23:01:28.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.487+0000 7f66612c4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f665c06cd00 0x7f665c15a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:28.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.487+0000 7f66612c4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f665c06cd00 0x7f665c15a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41872/0 (socket says 172.21.15.32:41872) 2024-04-24T23:01:28.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.487+0000 7f66612c4640 1 -- 172.21.15.32:0/3903699336 learned_addr learned my addr 172.21.15.32:0/3903699336 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:28.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.487+0000 7f6661ac5640 1 --2- 172.21.15.32:0/3903699336 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f665c15cb00 0x7f665c149be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:28.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.487+0000 7f6660ac3640 1 --2- 172.21.15.32:0/3903699336 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f665c15b380 0x7f665c1496a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:28.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.487+0000 7f66612c4640 1 -- 172.21.15.32:0/3903699336 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f665c15b380 msgr2=0x7f665c1496a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:28.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.487+0000 7f66612c4640 1 --2- 172.21.15.32:0/3903699336 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f665c15b380 0x7f665c1496a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:28.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.487+0000 7f66612c4640 1 -- 172.21.15.32:0/3903699336 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f665c15cb00 msgr2=0x7f665c149be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:28.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.487+0000 7f66612c4640 1 --2- 172.21.15.32:0/3903699336 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f665c15cb00 0x7f665c149be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:28.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.487+0000 7f66612c4640 1 -- 172.21.15.32:0/3903699336 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f665c072620 con 0x7f665c06cd00 2024-04-24T23:01:28.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.487+0000 7f6661ac5640 1 --2- 172.21.15.32:0/3903699336 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f665c15cb00 0x7f665c149be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:01:28.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.487+0000 7f6660ac3640 1 --2- 172.21.15.32:0/3903699336 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f665c15b380 0x7f665c1496a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:01:28.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.487+0000 7f66612c4640 1 --2- 172.21.15.32:0/3903699336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f665c06cd00 0x7f665c15a530 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f665400c4b0 tx=0x7f665400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:28.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.487+0000 7f6659ffb640 1 -- 172.21.15.32:0/3903699336 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6654017070 con 0x7f665c06cd00 2024-04-24T23:01:28.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.487+0000 7f6659ffb640 1 -- 172.21.15.32:0/3903699336 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6654005480 con 0x7f665c06cd00 2024-04-24T23:01:28.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.487+0000 7f6659ffb640 1 -- 172.21.15.32:0/3903699336 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6654003830 con 0x7f665c06cd00 2024-04-24T23:01:28.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.487+0000 7f6663528640 1 -- 172.21.15.32:0/3903699336 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f665c14a240 con 0x7f665c06cd00 2024-04-24T23:01:28.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.488+0000 7f6663528640 1 -- 172.21.15.32:0/3903699336 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f665c071e20 con 0x7f665c06cd00 2024-04-24T23:01:28.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.489+0000 7f6663528640 1 -- 172.21.15.32:0/3903699336 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f665c068430 con 0x7f665c06cd00 2024-04-24T23:01:28.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.489+0000 7f6659ffb640 1 -- 172.21.15.32:0/3903699336 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f665401a070 con 0x7f665c06cd00 2024-04-24T23:01:28.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.490+0000 7f6659ffb640 1 --2- 172.21.15.32:0/3903699336 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f664806be30 0x7f664806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:28.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.490+0000 7f6659ffb640 1 -- 172.21.15.32:0/3903699336 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7f665408a1c0 con 0x7f665c06cd00 2024-04-24T23:01:28.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.490+0000 7f6660ac3640 1 --2- 172.21.15.32:0/3903699336 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f664806be30 0x7f664806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:28.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.490+0000 7f6660ac3640 1 --2- 172.21.15.32:0/3903699336 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f664806be30 0x7f664806e2f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6650034000 tx=0x7f665002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:28.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.494+0000 7f6659ffb640 1 -- 172.21.15.32:0/3903699336 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6654058c00 con 0x7f665c06cd00 2024-04-24T23:01:28.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.673+0000 7f6663528640 1 -- 172.21.15.32:0/3903699336 --> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f665c06c000 con 0x7f664806be30 2024-04-24T23:01:28.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.680+0000 7f6659ffb640 1 -- 172.21.15.32:0/3903699336 <== mgr.4108 v2:172.21.15.121:6832/1848157943 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247489 (secure 0 0 0) 0x7f665c06c000 con 0x7f664806be30 2024-04-24T23:01:28.684 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:01:28.686 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-24T23:01:28.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.683+0000 7f6663528640 1 -- 172.21.15.32:0/3903699336 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f664806be30 msgr2=0x7f664806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:28.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.683+0000 7f6663528640 1 --2- 172.21.15.32:0/3903699336 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f664806be30 0x7f664806e2f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6650034000 tx=0x7f665002e000 comp rx=0 tx=0).stop 2024-04-24T23:01:28.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.683+0000 7f6663528640 1 -- 172.21.15.32:0/3903699336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f665c06cd00 msgr2=0x7f665c15a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:28.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.683+0000 7f6663528640 1 --2- 172.21.15.32:0/3903699336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f665c06cd00 0x7f665c15a530 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f665400c4b0 tx=0x7f665400c970 comp rx=0 tx=0).stop 2024-04-24T23:01:28.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.683+0000 7f6663528640 1 -- 172.21.15.32:0/3903699336 shutdown_connections 2024-04-24T23:01:28.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.683+0000 7f6663528640 1 --2- 172.21.15.32:0/3903699336 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f664806be30 0x7f664806e2f0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:28.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.683+0000 7f6663528640 1 --2- 172.21.15.32:0/3903699336 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f665c15cb00 0x7f665c149be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:28.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.683+0000 7f6663528640 1 --2- 172.21.15.32:0/3903699336 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f665c15b380 0x7f665c1496a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:28.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.683+0000 7f6663528640 1 --2- 172.21.15.32:0/3903699336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f665c06cd00 0x7f665c15a530 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:28.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.683+0000 7f6663528640 1 -- 172.21.15.32:0/3903699336 >> 172.21.15.32:0/3903699336 conn(0x7f665c06b110 msgr2=0x7f665c0cf1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:28.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.683+0000 7f6663528640 1 -- 172.21.15.32:0/3903699336 shutdown_connections 2024-04-24T23:01:28.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.683+0000 7f6663528640 1 -- 172.21.15.32:0/3903699336 wait complete. 2024-04-24T23:01:28.696 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":136,"stamp":"2024-04-24T23:01:27.159414+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":141,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":3836,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1558663,"statfs":{"total":1600000000,"available":1596071340,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000806"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.796748+0000","last_change":"2024-04-24T23:01:25.691368+0000","last_active":"2024-04-24T23:01:26.796748+0000","last_peered":"2024-04-24T23:01:26.796748+0000","last_clean":"2024-04-24T23:01:26.796748+0000","last_became_active":"2024-04-24T23:01:25.687857+0000","last_became_peered":"2024-04-24T23:01:25.687857+0000","last_unstale":"2024-04-24T23:01:26.796748+0000","last_undegraded":"2024-04-24T23:01:26.796748+0000","last_fullsized":"2024-04-24T23:01:26.796748+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:58:39.507832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.796217+0000","last_change":"2024-04-24T23:01:25.694501+0000","last_active":"2024-04-24T23:01:26.796217+0000","last_peered":"2024-04-24T23:01:26.796217+0000","last_clean":"2024-04-24T23:01:26.796217+0000","last_became_active":"2024-04-24T23:01:25.694272+0000","last_became_peered":"2024-04-24T23:01:25.694272+0000","last_unstale":"2024-04-24T23:01:26.796217+0000","last_undegraded":"2024-04-24T23:01:26.796217+0000","last_fullsized":"2024-04-24T23:01:26.796217+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:53:32.039806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795450+0000","last_change":"2024-04-24T23:01:25.685044+0000","last_active":"2024-04-24T23:01:26.795450+0000","last_peered":"2024-04-24T23:01:26.795450+0000","last_clean":"2024-04-24T23:01:26.795450+0000","last_became_active":"2024-04-24T23:01:25.684733+0000","last_became_peered":"2024-04-24T23:01:25.684733+0000","last_unstale":"2024-04-24T23:01:26.795450+0000","last_undegraded":"2024-04-24T23:01:26.795450+0000","last_fullsized":"2024-04-24T23:01:26.795450+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:15:14.681765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.677486+0000","last_change":"2024-04-24T23:01:25.680115+0000","last_active":"2024-04-24T23:01:26.677486+0000","last_peered":"2024-04-24T23:01:26.677486+0000","last_clean":"2024-04-24T23:01:26.677486+0000","last_became_active":"2024-04-24T23:01:25.679838+0000","last_became_peered":"2024-04-24T23:01:25.679838+0000","last_unstale":"2024-04-24T23:01:26.677486+0000","last_undegraded":"2024-04-24T23:01:26.677486+0000","last_fullsized":"2024-04-24T23:01:26.677486+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:37:48.507918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795953+0000","last_change":"2024-04-24T23:01:25.684189+0000","last_active":"2024-04-24T23:01:26.795953+0000","last_peered":"2024-04-24T23:01:26.795953+0000","last_clean":"2024-04-24T23:01:26.795953+0000","last_became_active":"2024-04-24T23:01:25.683490+0000","last_became_peered":"2024-04-24T23:01:25.683490+0000","last_unstale":"2024-04-24T23:01:26.795953+0000","last_undegraded":"2024-04-24T23:01:26.795953+0000","last_fullsized":"2024-04-24T23:01:26.795953+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:05.975380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.796834+0000","last_change":"2024-04-24T23:01:25.691587+0000","last_active":"2024-04-24T23:01:26.796834+0000","last_peered":"2024-04-24T23:01:26.796834+0000","last_clean":"2024-04-24T23:01:26.796834+0000","last_became_active":"2024-04-24T23:01:25.688792+0000","last_became_peered":"2024-04-24T23:01:25.688792+0000","last_unstale":"2024-04-24T23:01:26.796834+0000","last_undegraded":"2024-04-24T23:01:26.796834+0000","last_fullsized":"2024-04-24T23:01:26.796834+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:16:30.812523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.677496+0000","last_change":"2024-04-24T23:01:25.679660+0000","last_active":"2024-04-24T23:01:26.677496+0000","last_peered":"2024-04-24T23:01:26.677496+0000","last_clean":"2024-04-24T23:01:26.677496+0000","last_became_active":"2024-04-24T23:01:25.679354+0000","last_became_peered":"2024-04-24T23:01:25.679354+0000","last_unstale":"2024-04-24T23:01:26.677496+0000","last_undegraded":"2024-04-24T23:01:26.677496+0000","last_fullsized":"2024-04-24T23:01:26.677496+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:10.435460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795418+0000","last_change":"2024-04-24T23:01:25.689661+0000","last_active":"2024-04-24T23:01:26.795418+0000","last_peered":"2024-04-24T23:01:26.795418+0000","last_clean":"2024-04-24T23:01:26.795418+0000","last_became_active":"2024-04-24T23:01:25.689215+0000","last_became_peered":"2024-04-24T23:01:25.689215+0000","last_unstale":"2024-04-24T23:01:26.795418+0000","last_undegraded":"2024-04-24T23:01:26.795418+0000","last_fullsized":"2024-04-24T23:01:26.795418+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:25.526121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.680521+0000","last_change":"2024-04-24T23:01:25.694295+0000","last_active":"2024-04-24T23:01:26.680521+0000","last_peered":"2024-04-24T23:01:26.680521+0000","last_clean":"2024-04-24T23:01:26.680521+0000","last_became_active":"2024-04-24T23:01:25.693633+0000","last_became_peered":"2024-04-24T23:01:25.693633+0000","last_unstale":"2024-04-24T23:01:26.680521+0000","last_undegraded":"2024-04-24T23:01:26.680521+0000","last_fullsized":"2024-04-24T23:01:26.680521+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:02:18.861058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.796108+0000","last_change":"2024-04-24T23:01:25.688030+0000","last_active":"2024-04-24T23:01:26.796108+0000","last_peered":"2024-04-24T23:01:26.796108+0000","last_clean":"2024-04-24T23:01:26.796108+0000","last_became_active":"2024-04-24T23:01:25.687772+0000","last_became_peered":"2024-04-24T23:01:25.687772+0000","last_unstale":"2024-04-24T23:01:26.796108+0000","last_undegraded":"2024-04-24T23:01:26.796108+0000","last_fullsized":"2024-04-24T23:01:26.796108+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:57:34.794353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.680819+0000","last_change":"2024-04-24T23:01:25.697315+0000","last_active":"2024-04-24T23:01:26.680819+0000","last_peered":"2024-04-24T23:01:26.680819+0000","last_clean":"2024-04-24T23:01:26.680819+0000","last_became_active":"2024-04-24T23:01:25.697073+0000","last_became_peered":"2024-04-24T23:01:25.697073+0000","last_unstale":"2024-04-24T23:01:26.680819+0000","last_undegraded":"2024-04-24T23:01:26.680819+0000","last_fullsized":"2024-04-24T23:01:26.680819+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:25:29.199057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.680418+0000","last_change":"2024-04-24T23:01:25.687587+0000","last_active":"2024-04-24T23:01:26.680418+0000","last_peered":"2024-04-24T23:01:26.680418+0000","last_clean":"2024-04-24T23:01:26.680418+0000","last_became_active":"2024-04-24T23:01:25.687321+0000","last_became_peered":"2024-04-24T23:01:25.687321+0000","last_unstale":"2024-04-24T23:01:26.680418+0000","last_undegraded":"2024-04-24T23:01:26.680418+0000","last_fullsized":"2024-04-24T23:01:26.680418+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:30.462115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795845+0000","last_change":"2024-04-24T23:01:25.689404+0000","last_active":"2024-04-24T23:01:26.795845+0000","last_peered":"2024-04-24T23:01:26.795845+0000","last_clean":"2024-04-24T23:01:26.795845+0000","last_became_active":"2024-04-24T23:01:25.687976+0000","last_became_peered":"2024-04-24T23:01:25.687976+0000","last_unstale":"2024-04-24T23:01:26.795845+0000","last_undegraded":"2024-04-24T23:01:26.795845+0000","last_fullsized":"2024-04-24T23:01:26.795845+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:11.552562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795303+0000","last_change":"2024-04-24T23:01:25.686179+0000","last_active":"2024-04-24T23:01:26.795303+0000","last_peered":"2024-04-24T23:01:26.795303+0000","last_clean":"2024-04-24T23:01:26.795303+0000","last_became_active":"2024-04-24T23:01:25.685699+0000","last_became_peered":"2024-04-24T23:01:25.685699+0000","last_unstale":"2024-04-24T23:01:26.795303+0000","last_undegraded":"2024-04-24T23:01:26.795303+0000","last_fullsized":"2024-04-24T23:01:26.795303+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:58.285055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795224+0000","last_change":"2024-04-24T23:01:25.686857+0000","last_active":"2024-04-24T23:01:26.795224+0000","last_peered":"2024-04-24T23:01:26.795224+0000","last_clean":"2024-04-24T23:01:26.795224+0000","last_became_active":"2024-04-24T23:01:25.686568+0000","last_became_peered":"2024-04-24T23:01:25.686568+0000","last_unstale":"2024-04-24T23:01:26.795224+0000","last_undegraded":"2024-04-24T23:01:26.795224+0000","last_fullsized":"2024-04-24T23:01:26.795224+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:27:46.233436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.796916+0000","last_change":"2024-04-24T23:01:25.681029+0000","last_active":"2024-04-24T23:01:26.796916+0000","last_peered":"2024-04-24T23:01:26.796916+0000","last_clean":"2024-04-24T23:01:26.796916+0000","last_became_active":"2024-04-24T23:01:25.680752+0000","last_became_peered":"2024-04-24T23:01:25.680752+0000","last_unstale":"2024-04-24T23:01:26.796916+0000","last_undegraded":"2024-04-24T23:01:26.796916+0000","last_fullsized":"2024-04-24T23:01:26.796916+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:14:28.139891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795398+0000","last_change":"2024-04-24T23:01:25.684469+0000","last_active":"2024-04-24T23:01:26.795398+0000","last_peered":"2024-04-24T23:01:26.795398+0000","last_clean":"2024-04-24T23:01:26.795398+0000","last_became_active":"2024-04-24T23:01:25.684190+0000","last_became_peered":"2024-04-24T23:01:25.684190+0000","last_unstale":"2024-04-24T23:01:26.795398+0000","last_undegraded":"2024-04-24T23:01:26.795398+0000","last_fullsized":"2024-04-24T23:01:26.795398+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:08:04.009629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795589+0000","last_change":"2024-04-24T23:01:25.688192+0000","last_active":"2024-04-24T23:01:26.795589+0000","last_peered":"2024-04-24T23:01:26.795589+0000","last_clean":"2024-04-24T23:01:26.795589+0000","last_became_active":"2024-04-24T23:01:25.686705+0000","last_became_peered":"2024-04-24T23:01:25.686705+0000","last_unstale":"2024-04-24T23:01:26.795589+0000","last_undegraded":"2024-04-24T23:01:26.795589+0000","last_fullsized":"2024-04-24T23:01:26.795589+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:17:48.971011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795341+0000","last_change":"2024-04-24T23:01:25.687816+0000","last_active":"2024-04-24T23:01:26.795341+0000","last_peered":"2024-04-24T23:01:26.795341+0000","last_clean":"2024-04-24T23:01:26.795341+0000","last_became_active":"2024-04-24T23:01:25.687580+0000","last_became_peered":"2024-04-24T23:01:25.687580+0000","last_unstale":"2024-04-24T23:01:26.795341+0000","last_undegraded":"2024-04-24T23:01:26.795341+0000","last_fullsized":"2024-04-24T23:01:26.795341+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:49.976810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.680826+0000","last_change":"2024-04-24T23:01:25.694317+0000","last_active":"2024-04-24T23:01:26.680826+0000","last_peered":"2024-04-24T23:01:26.680826+0000","last_clean":"2024-04-24T23:01:26.680826+0000","last_became_active":"2024-04-24T23:01:25.694095+0000","last_became_peered":"2024-04-24T23:01:25.694095+0000","last_unstale":"2024-04-24T23:01:26.680826+0000","last_undegraded":"2024-04-24T23:01:26.680826+0000","last_fullsized":"2024-04-24T23:01:26.680826+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:39:17.395666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795043+0000","last_change":"2024-04-24T23:01:25.683853+0000","last_active":"2024-04-24T23:01:26.795043+0000","last_peered":"2024-04-24T23:01:26.795043+0000","last_clean":"2024-04-24T23:01:26.795043+0000","last_became_active":"2024-04-24T23:01:25.683268+0000","last_became_peered":"2024-04-24T23:01:25.683268+0000","last_unstale":"2024-04-24T23:01:26.795043+0000","last_undegraded":"2024-04-24T23:01:26.795043+0000","last_fullsized":"2024-04-24T23:01:26.795043+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:16:33.422056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.680846+0000","last_change":"2024-04-24T23:01:25.695665+0000","last_active":"2024-04-24T23:01:26.680846+0000","last_peered":"2024-04-24T23:01:26.680846+0000","last_clean":"2024-04-24T23:01:26.680846+0000","last_became_active":"2024-04-24T23:01:25.695050+0000","last_became_peered":"2024-04-24T23:01:25.695050+0000","last_unstale":"2024-04-24T23:01:26.680846+0000","last_undegraded":"2024-04-24T23:01:26.680846+0000","last_fullsized":"2024-04-24T23:01:26.680846+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:20:19.138677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.677396+0000","last_change":"2024-04-24T23:01:25.686702+0000","last_active":"2024-04-24T23:01:26.677396+0000","last_peered":"2024-04-24T23:01:26.677396+0000","last_clean":"2024-04-24T23:01:26.677396+0000","last_became_active":"2024-04-24T23:01:25.686451+0000","last_became_peered":"2024-04-24T23:01:25.686451+0000","last_unstale":"2024-04-24T23:01:26.677396+0000","last_undegraded":"2024-04-24T23:01:26.677396+0000","last_fullsized":"2024-04-24T23:01:26.677396+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:36.030977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.680380+0000","last_change":"2024-04-24T23:01:25.694438+0000","last_active":"2024-04-24T23:01:26.680380+0000","last_peered":"2024-04-24T23:01:26.680380+0000","last_clean":"2024-04-24T23:01:26.680380+0000","last_became_active":"2024-04-24T23:01:25.693892+0000","last_became_peered":"2024-04-24T23:01:25.693892+0000","last_unstale":"2024-04-24T23:01:26.680380+0000","last_undegraded":"2024-04-24T23:01:26.680380+0000","last_fullsized":"2024-04-24T23:01:26.680380+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:16:06.768138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795823+0000","last_change":"2024-04-24T23:01:25.690611+0000","last_active":"2024-04-24T23:01:26.795823+0000","last_peered":"2024-04-24T23:01:26.795823+0000","last_clean":"2024-04-24T23:01:26.795823+0000","last_became_active":"2024-04-24T23:01:25.690228+0000","last_became_peered":"2024-04-24T23:01:25.690228+0000","last_unstale":"2024-04-24T23:01:26.795823+0000","last_undegraded":"2024-04-24T23:01:26.795823+0000","last_fullsized":"2024-04-24T23:01:26.795823+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:30:01.210701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795188+0000","last_change":"2024-04-24T23:01:25.683526+0000","last_active":"2024-04-24T23:01:26.795188+0000","last_peered":"2024-04-24T23:01:26.795188+0000","last_clean":"2024-04-24T23:01:26.795188+0000","last_became_active":"2024-04-24T23:01:25.682995+0000","last_became_peered":"2024-04-24T23:01:25.682995+0000","last_unstale":"2024-04-24T23:01:26.795188+0000","last_undegraded":"2024-04-24T23:01:26.795188+0000","last_fullsized":"2024-04-24T23:01:26.795188+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:36.893814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.796471+0000","last_change":"2024-04-24T23:01:25.692849+0000","last_active":"2024-04-24T23:01:26.796471+0000","last_peered":"2024-04-24T23:01:26.796471+0000","last_clean":"2024-04-24T23:01:26.796471+0000","last_became_active":"2024-04-24T23:01:25.692318+0000","last_became_peered":"2024-04-24T23:01:25.692318+0000","last_unstale":"2024-04-24T23:01:26.796471+0000","last_undegraded":"2024-04-24T23:01:26.796471+0000","last_fullsized":"2024-04-24T23:01:26.796471+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:27:22.189961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795576+0000","last_change":"2024-04-24T23:01:25.694348+0000","last_active":"2024-04-24T23:01:26.795576+0000","last_peered":"2024-04-24T23:01:26.795576+0000","last_clean":"2024-04-24T23:01:26.795576+0000","last_became_active":"2024-04-24T23:01:25.694081+0000","last_became_peered":"2024-04-24T23:01:25.694081+0000","last_unstale":"2024-04-24T23:01:26.795576+0000","last_undegraded":"2024-04-24T23:01:26.795576+0000","last_fullsized":"2024-04-24T23:01:26.795576+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:00:27.519286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795376+0000","last_change":"2024-04-24T23:01:25.691847+0000","last_active":"2024-04-24T23:01:26.795376+0000","last_peered":"2024-04-24T23:01:26.795376+0000","last_clean":"2024-04-24T23:01:26.795376+0000","last_became_active":"2024-04-24T23:01:25.689754+0000","last_became_peered":"2024-04-24T23:01:25.689754+0000","last_unstale":"2024-04-24T23:01:26.795376+0000","last_undegraded":"2024-04-24T23:01:26.795376+0000","last_fullsized":"2024-04-24T23:01:26.795376+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:37.207955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.677600+0000","last_change":"2024-04-24T23:01:25.684143+0000","last_active":"2024-04-24T23:01:26.677600+0000","last_peered":"2024-04-24T23:01:26.677600+0000","last_clean":"2024-04-24T23:01:26.677600+0000","last_became_active":"2024-04-24T23:01:25.683879+0000","last_became_peered":"2024-04-24T23:01:25.683879+0000","last_unstale":"2024-04-24T23:01:26.677600+0000","last_undegraded":"2024-04-24T23:01:26.677600+0000","last_fullsized":"2024-04-24T23:01:26.677600+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:04:52.760103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795500+0000","last_change":"2024-04-24T23:01:25.689471+0000","last_active":"2024-04-24T23:01:26.795500+0000","last_peered":"2024-04-24T23:01:26.795500+0000","last_clean":"2024-04-24T23:01:26.795500+0000","last_became_active":"2024-04-24T23:01:25.689059+0000","last_became_peered":"2024-04-24T23:01:25.689059+0000","last_unstale":"2024-04-24T23:01:26.795500+0000","last_undegraded":"2024-04-24T23:01:26.795500+0000","last_fullsized":"2024-04-24T23:01:26.795500+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:32:55.533168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.681018+0000","last_change":"2024-04-24T23:01:25.695467+0000","last_active":"2024-04-24T23:01:26.681018+0000","last_peered":"2024-04-24T23:01:26.681018+0000","last_clean":"2024-04-24T23:01:26.681018+0000","last_became_active":"2024-04-24T23:01:25.694710+0000","last_became_peered":"2024-04-24T23:01:25.694710+0000","last_unstale":"2024-04-24T23:01:26.681018+0000","last_undegraded":"2024-04-24T23:01:26.681018+0000","last_fullsized":"2024-04-24T23:01:26.681018+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:16:14.961098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795252+0000","last_change":"2024-04-24T23:01:25.691425+0000","last_active":"2024-04-24T23:01:26.795252+0000","last_peered":"2024-04-24T23:01:26.795252+0000","last_clean":"2024-04-24T23:01:26.795252+0000","last_became_active":"2024-04-24T23:01:25.687581+0000","last_became_peered":"2024-04-24T23:01:25.687581+0000","last_unstale":"2024-04-24T23:01:26.795252+0000","last_undegraded":"2024-04-24T23:01:26.795252+0000","last_fullsized":"2024-04-24T23:01:26.795252+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:32:29.563928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"66'4","reported_seq":18,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.680888+0000","last_change":"2024-04-24T23:01:24.675437+0000","last_active":"2024-04-24T23:01:26.680888+0000","last_peered":"2024-04-24T23:01:26.680888+0000","last_clean":"2024-04-24T23:01:26.680888+0000","last_became_active":"2024-04-24T23:01:24.675202+0000","last_became_peered":"2024-04-24T23:01:24.675202+0000","last_unstale":"2024-04-24T23:01:26.680888+0000","last_undegraded":"2024-04-24T23:01:26.680888+0000","last_fullsized":"2024-04-24T23:01:26.680888+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:11:40.898914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.796569+0000","last_change":"2024-04-24T23:01:25.684935+0000","last_active":"2024-04-24T23:01:26.796569+0000","last_peered":"2024-04-24T23:01:26.796569+0000","last_clean":"2024-04-24T23:01:26.796569+0000","last_became_active":"2024-04-24T23:01:25.684677+0000","last_became_peered":"2024-04-24T23:01:25.684677+0000","last_unstale":"2024-04-24T23:01:26.796569+0000","last_undegraded":"2024-04-24T23:01:26.796569+0000","last_fullsized":"2024-04-24T23:01:26.796569+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:57:35.708359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795453+0000","last_change":"2024-04-24T23:01:24.668594+0000","last_active":"2024-04-24T23:01:26.795453+0000","last_peered":"2024-04-24T23:01:26.795453+0000","last_clean":"2024-04-24T23:01:26.795453+0000","last_became_active":"2024-04-24T23:01:24.668204+0000","last_became_peered":"2024-04-24T23:01:24.668204+0000","last_unstale":"2024-04-24T23:01:26.795453+0000","last_undegraded":"2024-04-24T23:01:26.795453+0000","last_fullsized":"2024-04-24T23:01:26.795453+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:15.955906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.796290+0000","last_change":"2024-04-24T23:01:25.686989+0000","last_active":"2024-04-24T23:01:26.796290+0000","last_peered":"2024-04-24T23:01:26.796290+0000","last_clean":"2024-04-24T23:01:26.796290+0000","last_became_active":"2024-04-24T23:01:25.686769+0000","last_became_peered":"2024-04-24T23:01:25.686769+0000","last_unstale":"2024-04-24T23:01:26.796290+0000","last_undegraded":"2024-04-24T23:01:26.796290+0000","last_fullsized":"2024-04-24T23:01:26.796290+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:01:04.547945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.796451+0000","last_change":"2024-04-24T23:01:24.672026+0000","last_active":"2024-04-24T23:01:26.796451+0000","last_peered":"2024-04-24T23:01:26.796451+0000","last_clean":"2024-04-24T23:01:26.796451+0000","last_became_active":"2024-04-24T23:01:24.671743+0000","last_became_peered":"2024-04-24T23:01:24.671743+0000","last_unstale":"2024-04-24T23:01:26.796451+0000","last_undegraded":"2024-04-24T23:01:26.796451+0000","last_fullsized":"2024-04-24T23:01:26.796451+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:43:53.956277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.796336+0000","last_change":"2024-04-24T23:01:25.687012+0000","last_active":"2024-04-24T23:01:26.796336+0000","last_peered":"2024-04-24T23:01:26.796336+0000","last_clean":"2024-04-24T23:01:26.796336+0000","last_became_active":"2024-04-24T23:01:25.686734+0000","last_became_peered":"2024-04-24T23:01:25.686734+0000","last_unstale":"2024-04-24T23:01:26.796336+0000","last_undegraded":"2024-04-24T23:01:26.796336+0000","last_fullsized":"2024-04-24T23:01:26.796336+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:30:27.317521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.677620+0000","last_change":"2024-04-24T23:01:24.670512+0000","last_active":"2024-04-24T23:01:26.677620+0000","last_peered":"2024-04-24T23:01:26.677620+0000","last_clean":"2024-04-24T23:01:26.677620+0000","last_became_active":"2024-04-24T23:01:24.670278+0000","last_became_peered":"2024-04-24T23:01:24.670278+0000","last_unstale":"2024-04-24T23:01:26.677620+0000","last_undegraded":"2024-04-24T23:01:26.677620+0000","last_fullsized":"2024-04-24T23:01:26.677620+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:50:23.727996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795732+0000","last_change":"2024-04-24T23:01:25.686247+0000","last_active":"2024-04-24T23:01:26.795732+0000","last_peered":"2024-04-24T23:01:26.795732+0000","last_clean":"2024-04-24T23:01:26.795732+0000","last_became_active":"2024-04-24T23:01:25.685961+0000","last_became_peered":"2024-04-24T23:01:25.685961+0000","last_unstale":"2024-04-24T23:01:26.795732+0000","last_undegraded":"2024-04-24T23:01:26.795732+0000","last_fullsized":"2024-04-24T23:01:26.795732+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:52:11.921979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795154+0000","last_change":"2024-04-24T23:01:24.676505+0000","last_active":"2024-04-24T23:01:26.795154+0000","last_peered":"2024-04-24T23:01:26.795154+0000","last_clean":"2024-04-24T23:01:26.795154+0000","last_became_active":"2024-04-24T23:01:24.676128+0000","last_became_peered":"2024-04-24T23:01:24.676128+0000","last_unstale":"2024-04-24T23:01:26.795154+0000","last_undegraded":"2024-04-24T23:01:26.795154+0000","last_fullsized":"2024-04-24T23:01:26.795154+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:35:43.063596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.681233+0000","last_change":"2024-04-24T23:01:25.695268+0000","last_active":"2024-04-24T23:01:26.681233+0000","last_peered":"2024-04-24T23:01:26.681233+0000","last_clean":"2024-04-24T23:01:26.681233+0000","last_became_active":"2024-04-24T23:01:25.694386+0000","last_became_peered":"2024-04-24T23:01:25.694386+0000","last_unstale":"2024-04-24T23:01:26.681233+0000","last_undegraded":"2024-04-24T23:01:26.681233+0000","last_fullsized":"2024-04-24T23:01:26.681233+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:40:43.553477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.796309+0000","last_change":"2024-04-24T23:01:24.678336+0000","last_active":"2024-04-24T23:01:26.796309+0000","last_peered":"2024-04-24T23:01:26.796309+0000","last_clean":"2024-04-24T23:01:26.796309+0000","last_became_active":"2024-04-24T23:01:24.677580+0000","last_became_peered":"2024-04-24T23:01:24.677580+0000","last_unstale":"2024-04-24T23:01:26.796309+0000","last_undegraded":"2024-04-24T23:01:26.796309+0000","last_fullsized":"2024-04-24T23:01:26.796309+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:03:13.884937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795321+0000","last_change":"2024-04-24T23:01:25.687136+0000","last_active":"2024-04-24T23:01:26.795321+0000","last_peered":"2024-04-24T23:01:26.795321+0000","last_clean":"2024-04-24T23:01:26.795321+0000","last_became_active":"2024-04-24T23:01:25.684693+0000","last_became_peered":"2024-04-24T23:01:25.684693+0000","last_unstale":"2024-04-24T23:01:26.795321+0000","last_undegraded":"2024-04-24T23:01:26.795321+0000","last_fullsized":"2024-04-24T23:01:26.795321+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:03:25.489635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.680784+0000","last_change":"2024-04-24T23:01:24.676646+0000","last_active":"2024-04-24T23:01:26.680784+0000","last_peered":"2024-04-24T23:01:26.680784+0000","last_clean":"2024-04-24T23:01:26.680784+0000","last_became_active":"2024-04-24T23:01:24.676390+0000","last_became_peered":"2024-04-24T23:01:24.676390+0000","last_unstale":"2024-04-24T23:01:26.680784+0000","last_undegraded":"2024-04-24T23:01:26.680784+0000","last_fullsized":"2024-04-24T23:01:26.680784+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:39.274912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795580+0000","last_change":"2024-04-24T23:01:25.681512+0000","last_active":"2024-04-24T23:01:26.795580+0000","last_peered":"2024-04-24T23:01:26.795580+0000","last_clean":"2024-04-24T23:01:26.795580+0000","last_became_active":"2024-04-24T23:01:25.681146+0000","last_became_peered":"2024-04-24T23:01:25.681146+0000","last_unstale":"2024-04-24T23:01:26.795580+0000","last_undegraded":"2024-04-24T23:01:26.795580+0000","last_fullsized":"2024-04-24T23:01:26.795580+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:48:44.254638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795389+0000","last_change":"2024-04-24T23:01:24.674059+0000","last_active":"2024-04-24T23:01:26.795389+0000","last_peered":"2024-04-24T23:01:26.795389+0000","last_clean":"2024-04-24T23:01:26.795389+0000","last_became_active":"2024-04-24T23:01:24.673804+0000","last_became_peered":"2024-04-24T23:01:24.673804+0000","last_unstale":"2024-04-24T23:01:26.795389+0000","last_undegraded":"2024-04-24T23:01:26.795389+0000","last_fullsized":"2024-04-24T23:01:26.795389+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:30:36.248416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795584+0000","last_change":"2024-04-24T23:01:25.688965+0000","last_active":"2024-04-24T23:01:26.795584+0000","last_peered":"2024-04-24T23:01:26.795584+0000","last_clean":"2024-04-24T23:01:26.795584+0000","last_became_active":"2024-04-24T23:01:25.688653+0000","last_became_peered":"2024-04-24T23:01:25.688653+0000","last_unstale":"2024-04-24T23:01:26.795584+0000","last_undegraded":"2024-04-24T23:01:26.795584+0000","last_fullsized":"2024-04-24T23:01:26.795584+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:26:49.445951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.680569+0000","last_change":"2024-04-24T23:01:24.674041+0000","last_active":"2024-04-24T23:01:26.680569+0000","last_peered":"2024-04-24T23:01:26.680569+0000","last_clean":"2024-04-24T23:01:26.680569+0000","last_became_active":"2024-04-24T23:01:24.673656+0000","last_became_peered":"2024-04-24T23:01:24.673656+0000","last_unstale":"2024-04-24T23:01:26.680569+0000","last_undegraded":"2024-04-24T23:01:26.680569+0000","last_fullsized":"2024-04-24T23:01:26.680569+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:58:40.427542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.794971+0000","last_change":"2024-04-24T23:01:25.687177+0000","last_active":"2024-04-24T23:01:26.794971+0000","last_peered":"2024-04-24T23:01:26.794971+0000","last_clean":"2024-04-24T23:01:26.794971+0000","last_became_active":"2024-04-24T23:01:25.686782+0000","last_became_peered":"2024-04-24T23:01:25.686782+0000","last_unstale":"2024-04-24T23:01:26.794971+0000","last_undegraded":"2024-04-24T23:01:26.794971+0000","last_fullsized":"2024-04-24T23:01:26.794971+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:19.870378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795780+0000","last_change":"2024-04-24T23:01:24.676788+0000","last_active":"2024-04-24T23:01:26.795780+0000","last_peered":"2024-04-24T23:01:26.795780+0000","last_clean":"2024-04-24T23:01:26.795780+0000","last_became_active":"2024-04-24T23:01:24.676589+0000","last_became_peered":"2024-04-24T23:01:24.676589+0000","last_unstale":"2024-04-24T23:01:26.795780+0000","last_undegraded":"2024-04-24T23:01:26.795780+0000","last_fullsized":"2024-04-24T23:01:26.795780+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:01:42.996449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795523+0000","last_change":"2024-04-24T23:01:25.687994+0000","last_active":"2024-04-24T23:01:26.795523+0000","last_peered":"2024-04-24T23:01:26.795523+0000","last_clean":"2024-04-24T23:01:26.795523+0000","last_became_active":"2024-04-24T23:01:25.687761+0000","last_became_peered":"2024-04-24T23:01:25.687761+0000","last_unstale":"2024-04-24T23:01:26.795523+0000","last_undegraded":"2024-04-24T23:01:26.795523+0000","last_fullsized":"2024-04-24T23:01:26.795523+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:01:06.838635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"66'3","reported_seq":17,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.677735+0000","last_change":"2024-04-24T23:01:24.670733+0000","last_active":"2024-04-24T23:01:26.677735+0000","last_peered":"2024-04-24T23:01:26.677735+0000","last_clean":"2024-04-24T23:01:26.677735+0000","last_became_active":"2024-04-24T23:01:24.670466+0000","last_became_peered":"2024-04-24T23:01:24.670466+0000","last_unstale":"2024-04-24T23:01:26.677735+0000","last_undegraded":"2024-04-24T23:01:26.677735+0000","last_fullsized":"2024-04-24T23:01:26.677735+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:56.197002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.677636+0000","last_change":"2024-04-24T23:01:25.689104+0000","last_active":"2024-04-24T23:01:26.677636+0000","last_peered":"2024-04-24T23:01:26.677636+0000","last_clean":"2024-04-24T23:01:26.677636+0000","last_became_active":"2024-04-24T23:01:25.688826+0000","last_became_peered":"2024-04-24T23:01:25.688826+0000","last_unstale":"2024-04-24T23:01:26.677636+0000","last_undegraded":"2024-04-24T23:01:26.677636+0000","last_fullsized":"2024-04-24T23:01:26.677636+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:14:57.421224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"66'6","reported_seq":20,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795165+0000","last_change":"2024-04-24T23:01:24.675095+0000","last_active":"2024-04-24T23:01:26.795165+0000","last_peered":"2024-04-24T23:01:26.795165+0000","last_clean":"2024-04-24T23:01:26.795165+0000","last_became_active":"2024-04-24T23:01:24.674835+0000","last_became_peered":"2024-04-24T23:01:24.674835+0000","last_unstale":"2024-04-24T23:01:26.795165+0000","last_undegraded":"2024-04-24T23:01:26.795165+0000","last_fullsized":"2024-04-24T23:01:26.795165+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:05:20.246031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.680477+0000","last_change":"2024-04-24T23:01:25.685986+0000","last_active":"2024-04-24T23:01:26.680477+0000","last_peered":"2024-04-24T23:01:26.680477+0000","last_clean":"2024-04-24T23:01:26.680477+0000","last_became_active":"2024-04-24T23:01:25.685668+0000","last_became_peered":"2024-04-24T23:01:25.685668+0000","last_unstale":"2024-04-24T23:01:26.680477+0000","last_undegraded":"2024-04-24T23:01:26.680477+0000","last_fullsized":"2024-04-24T23:01:26.680477+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:21.278605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.796755+0000","last_change":"2024-04-24T23:01:24.674942+0000","last_active":"2024-04-24T23:01:26.796755+0000","last_peered":"2024-04-24T23:01:26.796755+0000","last_clean":"2024-04-24T23:01:26.796755+0000","last_became_active":"2024-04-24T23:01:24.673969+0000","last_became_peered":"2024-04-24T23:01:24.673969+0000","last_unstale":"2024-04-24T23:01:26.796755+0000","last_undegraded":"2024-04-24T23:01:26.796755+0000","last_fullsized":"2024-04-24T23:01:26.796755+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:59.650806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795466+0000","last_change":"2024-04-24T23:01:25.684946+0000","last_active":"2024-04-24T23:01:26.795466+0000","last_peered":"2024-04-24T23:01:26.795466+0000","last_clean":"2024-04-24T23:01:26.795466+0000","last_became_active":"2024-04-24T23:01:25.683407+0000","last_became_peered":"2024-04-24T23:01:25.683407+0000","last_unstale":"2024-04-24T23:01:26.795466+0000","last_undegraded":"2024-04-24T23:01:26.795466+0000","last_fullsized":"2024-04-24T23:01:26.795466+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:20:19.277989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.677668+0000","last_change":"2024-04-24T23:01:24.669720+0000","last_active":"2024-04-24T23:01:26.677668+0000","last_peered":"2024-04-24T23:01:26.677668+0000","last_clean":"2024-04-24T23:01:26.677668+0000","last_became_active":"2024-04-24T23:01:24.669486+0000","last_became_peered":"2024-04-24T23:01:24.669486+0000","last_unstale":"2024-04-24T23:01:26.677668+0000","last_undegraded":"2024-04-24T23:01:26.677668+0000","last_fullsized":"2024-04-24T23:01:26.677668+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:35:01.170901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.680392+0000","last_change":"2024-04-24T23:01:25.687447+0000","last_active":"2024-04-24T23:01:26.680392+0000","last_peered":"2024-04-24T23:01:26.680392+0000","last_clean":"2024-04-24T23:01:26.680392+0000","last_became_active":"2024-04-24T23:01:25.687205+0000","last_became_peered":"2024-04-24T23:01:25.687205+0000","last_unstale":"2024-04-24T23:01:26.680392+0000","last_undegraded":"2024-04-24T23:01:26.680392+0000","last_fullsized":"2024-04-24T23:01:26.680392+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:40:09.564255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.677732+0000","last_change":"2024-04-24T23:01:24.671197+0000","last_active":"2024-04-24T23:01:26.677732+0000","last_peered":"2024-04-24T23:01:26.677732+0000","last_clean":"2024-04-24T23:01:26.677732+0000","last_became_active":"2024-04-24T23:01:24.670708+0000","last_became_peered":"2024-04-24T23:01:24.670708+0000","last_unstale":"2024-04-24T23:01:26.677732+0000","last_undegraded":"2024-04-24T23:01:26.677732+0000","last_fullsized":"2024-04-24T23:01:26.677732+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:08:34.350673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.677637+0000","last_change":"2024-04-24T23:01:25.687844+0000","last_active":"2024-04-24T23:01:26.677637+0000","last_peered":"2024-04-24T23:01:26.677637+0000","last_clean":"2024-04-24T23:01:26.677637+0000","last_became_active":"2024-04-24T23:01:25.687594+0000","last_became_peered":"2024-04-24T23:01:25.687594+0000","last_unstale":"2024-04-24T23:01:26.677637+0000","last_undegraded":"2024-04-24T23:01:26.677637+0000","last_fullsized":"2024-04-24T23:01:26.677637+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:58.803035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"66'8","reported_seq":22,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.677799+0000","last_change":"2024-04-24T23:01:24.668906+0000","last_active":"2024-04-24T23:01:26.677799+0000","last_peered":"2024-04-24T23:01:26.677799+0000","last_clean":"2024-04-24T23:01:26.677799+0000","last_became_active":"2024-04-24T23:01:24.668668+0000","last_became_peered":"2024-04-24T23:01:24.668668+0000","last_unstale":"2024-04-24T23:01:26.677799+0000","last_undegraded":"2024-04-24T23:01:26.677799+0000","last_fullsized":"2024-04-24T23:01:26.677799+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:35.586026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795589+0000","last_change":"2024-04-24T23:01:25.688984+0000","last_active":"2024-04-24T23:01:26.795589+0000","last_peered":"2024-04-24T23:01:26.795589+0000","last_clean":"2024-04-24T23:01:26.795589+0000","last_became_active":"2024-04-24T23:01:25.688729+0000","last_became_peered":"2024-04-24T23:01:25.688729+0000","last_unstale":"2024-04-24T23:01:26.795589+0000","last_undegraded":"2024-04-24T23:01:26.795589+0000","last_fullsized":"2024-04-24T23:01:26.795589+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:06:17.364816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795560+0000","last_change":"2024-04-24T23:01:24.670205+0000","last_active":"2024-04-24T23:01:26.795560+0000","last_peered":"2024-04-24T23:01:26.795560+0000","last_clean":"2024-04-24T23:01:26.795560+0000","last_became_active":"2024-04-24T23:01:24.669391+0000","last_became_peered":"2024-04-24T23:01:24.669391+0000","last_unstale":"2024-04-24T23:01:26.795560+0000","last_undegraded":"2024-04-24T23:01:26.795560+0000","last_fullsized":"2024-04-24T23:01:26.795560+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:13.191807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795452+0000","last_change":"2024-04-24T23:01:25.685965+0000","last_active":"2024-04-24T23:01:26.795452+0000","last_peered":"2024-04-24T23:01:26.795452+0000","last_clean":"2024-04-24T23:01:26.795452+0000","last_became_active":"2024-04-24T23:01:25.685385+0000","last_became_peered":"2024-04-24T23:01:25.685385+0000","last_unstale":"2024-04-24T23:01:26.795452+0000","last_undegraded":"2024-04-24T23:01:26.795452+0000","last_fullsized":"2024-04-24T23:01:26.795452+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:09:29.661588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.796862+0000","last_change":"2024-04-24T23:01:24.674710+0000","last_active":"2024-04-24T23:01:26.796862+0000","last_peered":"2024-04-24T23:01:26.796862+0000","last_clean":"2024-04-24T23:01:26.796862+0000","last_became_active":"2024-04-24T23:01:24.673627+0000","last_became_peered":"2024-04-24T23:01:24.673627+0000","last_unstale":"2024-04-24T23:01:26.796862+0000","last_undegraded":"2024-04-24T23:01:26.796862+0000","last_fullsized":"2024-04-24T23:01:26.796862+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:29:11.495801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.796640+0000","last_change":"2024-04-24T23:01:25.692604+0000","last_active":"2024-04-24T23:01:26.796640+0000","last_peered":"2024-04-24T23:01:26.796640+0000","last_clean":"2024-04-24T23:01:26.796640+0000","last_became_active":"2024-04-24T23:01:25.691592+0000","last_became_peered":"2024-04-24T23:01:25.691592+0000","last_unstale":"2024-04-24T23:01:26.796640+0000","last_undegraded":"2024-04-24T23:01:26.796640+0000","last_fullsized":"2024-04-24T23:01:26.796640+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:30:15.571691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795074+0000","last_change":"2024-04-24T23:01:24.671347+0000","last_active":"2024-04-24T23:01:26.795074+0000","last_peered":"2024-04-24T23:01:26.795074+0000","last_clean":"2024-04-24T23:01:26.795074+0000","last_became_active":"2024-04-24T23:01:24.671004+0000","last_became_peered":"2024-04-24T23:01:24.671004+0000","last_unstale":"2024-04-24T23:01:26.795074+0000","last_undegraded":"2024-04-24T23:01:26.795074+0000","last_fullsized":"2024-04-24T23:01:26.795074+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:18.314318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.677552+0000","last_change":"2024-04-24T23:01:25.681253+0000","last_active":"2024-04-24T23:01:26.677552+0000","last_peered":"2024-04-24T23:01:26.677552+0000","last_clean":"2024-04-24T23:01:26.677552+0000","last_became_active":"2024-04-24T23:01:25.680999+0000","last_became_peered":"2024-04-24T23:01:25.680999+0000","last_unstale":"2024-04-24T23:01:26.677552+0000","last_undegraded":"2024-04-24T23:01:26.677552+0000","last_fullsized":"2024-04-24T23:01:26.677552+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:22:32.016539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795889+0000","last_change":"2024-04-24T23:01:24.674321+0000","last_active":"2024-04-24T23:01:26.795889+0000","last_peered":"2024-04-24T23:01:26.795889+0000","last_clean":"2024-04-24T23:01:26.795889+0000","last_became_active":"2024-04-24T23:01:24.672742+0000","last_became_peered":"2024-04-24T23:01:24.672742+0000","last_unstale":"2024-04-24T23:01:26.795889+0000","last_undegraded":"2024-04-24T23:01:26.795889+0000","last_fullsized":"2024-04-24T23:01:26.795889+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:45:50.092244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.680738+0000","last_change":"2024-04-24T23:01:25.694131+0000","last_active":"2024-04-24T23:01:26.680738+0000","last_peered":"2024-04-24T23:01:26.680738+0000","last_clean":"2024-04-24T23:01:26.680738+0000","last_became_active":"2024-04-24T23:01:25.693263+0000","last_became_peered":"2024-04-24T23:01:25.693263+0000","last_unstale":"2024-04-24T23:01:26.680738+0000","last_undegraded":"2024-04-24T23:01:26.680738+0000","last_fullsized":"2024-04-24T23:01:26.680738+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:46:26.253107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.681013+0000","last_change":"2024-04-24T23:01:24.675932+0000","last_active":"2024-04-24T23:01:26.681013+0000","last_peered":"2024-04-24T23:01:26.681013+0000","last_clean":"2024-04-24T23:01:26.681013+0000","last_became_active":"2024-04-24T23:01:24.675703+0000","last_became_peered":"2024-04-24T23:01:24.675703+0000","last_unstale":"2024-04-24T23:01:26.681013+0000","last_undegraded":"2024-04-24T23:01:26.681013+0000","last_fullsized":"2024-04-24T23:01:26.681013+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:13:13.489036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795988+0000","last_change":"2024-04-24T23:01:25.689467+0000","last_active":"2024-04-24T23:01:26.795988+0000","last_peered":"2024-04-24T23:01:26.795988+0000","last_clean":"2024-04-24T23:01:26.795988+0000","last_became_active":"2024-04-24T23:01:25.688872+0000","last_became_peered":"2024-04-24T23:01:25.688872+0000","last_unstale":"2024-04-24T23:01:26.795988+0000","last_undegraded":"2024-04-24T23:01:26.795988+0000","last_fullsized":"2024-04-24T23:01:26.795988+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:48:57.787836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795953+0000","last_change":"2024-04-24T23:01:24.677214+0000","last_active":"2024-04-24T23:01:26.795953+0000","last_peered":"2024-04-24T23:01:26.795953+0000","last_clean":"2024-04-24T23:01:26.795953+0000","last_became_active":"2024-04-24T23:01:24.676980+0000","last_became_peered":"2024-04-24T23:01:24.676980+0000","last_unstale":"2024-04-24T23:01:26.795953+0000","last_undegraded":"2024-04-24T23:01:26.795953+0000","last_fullsized":"2024-04-24T23:01:26.795953+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:56:19.290947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.796643+0000","last_change":"2024-04-24T23:01:25.691570+0000","last_active":"2024-04-24T23:01:26.796643+0000","last_peered":"2024-04-24T23:01:26.796643+0000","last_clean":"2024-04-24T23:01:26.796643+0000","last_became_active":"2024-04-24T23:01:25.690694+0000","last_became_peered":"2024-04-24T23:01:25.690694+0000","last_unstale":"2024-04-24T23:01:26.796643+0000","last_undegraded":"2024-04-24T23:01:26.796643+0000","last_fullsized":"2024-04-24T23:01:26.796643+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:55.037551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.680657+0000","last_change":"2024-04-24T23:01:24.676113+0000","last_active":"2024-04-24T23:01:26.680657+0000","last_peered":"2024-04-24T23:01:26.680657+0000","last_clean":"2024-04-24T23:01:26.680657+0000","last_became_active":"2024-04-24T23:01:24.675810+0000","last_became_peered":"2024-04-24T23:01:24.675810+0000","last_unstale":"2024-04-24T23:01:26.680657+0000","last_undegraded":"2024-04-24T23:01:26.680657+0000","last_fullsized":"2024-04-24T23:01:26.680657+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:15:15.452325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795734+0000","last_change":"2024-04-24T23:01:25.686958+0000","last_active":"2024-04-24T23:01:26.795734+0000","last_peered":"2024-04-24T23:01:26.795734+0000","last_clean":"2024-04-24T23:01:26.795734+0000","last_became_active":"2024-04-24T23:01:25.686354+0000","last_became_peered":"2024-04-24T23:01:25.686354+0000","last_unstale":"2024-04-24T23:01:26.795734+0000","last_undegraded":"2024-04-24T23:01:26.795734+0000","last_fullsized":"2024-04-24T23:01:26.795734+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:34:14.547983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.796932+0000","last_change":"2024-04-24T23:01:24.675846+0000","last_active":"2024-04-24T23:01:26.796932+0000","last_peered":"2024-04-24T23:01:26.796932+0000","last_clean":"2024-04-24T23:01:26.796932+0000","last_became_active":"2024-04-24T23:01:24.675551+0000","last_became_peered":"2024-04-24T23:01:24.675551+0000","last_unstale":"2024-04-24T23:01:26.796932+0000","last_undegraded":"2024-04-24T23:01:26.796932+0000","last_fullsized":"2024-04-24T23:01:26.796932+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:50.040159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.794974+0000","last_change":"2024-04-24T23:01:25.680176+0000","last_active":"2024-04-24T23:01:26.794974+0000","last_peered":"2024-04-24T23:01:26.794974+0000","last_clean":"2024-04-24T23:01:26.794974+0000","last_became_active":"2024-04-24T23:01:25.679731+0000","last_became_peered":"2024-04-24T23:01:25.679731+0000","last_unstale":"2024-04-24T23:01:26.794974+0000","last_undegraded":"2024-04-24T23:01:26.794974+0000","last_fullsized":"2024-04-24T23:01:26.794974+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:20.438750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.796418+0000","last_change":"2024-04-24T23:01:24.678561+0000","last_active":"2024-04-24T23:01:26.796418+0000","last_peered":"2024-04-24T23:01:26.796418+0000","last_clean":"2024-04-24T23:01:26.796418+0000","last_became_active":"2024-04-24T23:01:24.677898+0000","last_became_peered":"2024-04-24T23:01:24.677898+0000","last_unstale":"2024-04-24T23:01:26.796418+0000","last_undegraded":"2024-04-24T23:01:26.796418+0000","last_fullsized":"2024-04-24T23:01:26.796418+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:08:45.956355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795981+0000","last_change":"2024-04-24T23:01:25.688840+0000","last_active":"2024-04-24T23:01:26.795981+0000","last_peered":"2024-04-24T23:01:26.795981+0000","last_clean":"2024-04-24T23:01:26.795981+0000","last_became_active":"2024-04-24T23:01:25.688608+0000","last_became_peered":"2024-04-24T23:01:25.688608+0000","last_unstale":"2024-04-24T23:01:26.795981+0000","last_undegraded":"2024-04-24T23:01:26.795981+0000","last_fullsized":"2024-04-24T23:01:26.795981+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:23.511995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.680827+0000","last_change":"2024-04-24T23:01:24.674057+0000","last_active":"2024-04-24T23:01:26.680827+0000","last_peered":"2024-04-24T23:01:26.680827+0000","last_clean":"2024-04-24T23:01:26.680827+0000","last_became_active":"2024-04-24T23:01:24.673808+0000","last_became_peered":"2024-04-24T23:01:24.673808+0000","last_unstale":"2024-04-24T23:01:26.680827+0000","last_undegraded":"2024-04-24T23:01:26.680827+0000","last_fullsized":"2024-04-24T23:01:26.680827+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:02:36.252012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795680+0000","last_change":"2024-04-24T23:01:25.693803+0000","last_active":"2024-04-24T23:01:26.795680+0000","last_peered":"2024-04-24T23:01:26.795680+0000","last_clean":"2024-04-24T23:01:26.795680+0000","last_became_active":"2024-04-24T23:01:25.692116+0000","last_became_peered":"2024-04-24T23:01:25.692116+0000","last_unstale":"2024-04-24T23:01:26.795680+0000","last_undegraded":"2024-04-24T23:01:26.795680+0000","last_fullsized":"2024-04-24T23:01:26.795680+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:24:02.265824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795698+0000","last_change":"2024-04-24T23:01:24.670600+0000","last_active":"2024-04-24T23:01:26.795698+0000","last_peered":"2024-04-24T23:01:26.795698+0000","last_clean":"2024-04-24T23:01:26.795698+0000","last_became_active":"2024-04-24T23:01:24.669627+0000","last_became_peered":"2024-04-24T23:01:24.669627+0000","last_unstale":"2024-04-24T23:01:26.795698+0000","last_undegraded":"2024-04-24T23:01:26.795698+0000","last_fullsized":"2024-04-24T23:01:26.795698+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:47:52.135891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795557+0000","last_change":"2024-04-24T23:01:25.695872+0000","last_active":"2024-04-24T23:01:26.795557+0000","last_peered":"2024-04-24T23:01:26.795557+0000","last_clean":"2024-04-24T23:01:26.795557+0000","last_became_active":"2024-04-24T23:01:25.695275+0000","last_became_peered":"2024-04-24T23:01:25.695275+0000","last_unstale":"2024-04-24T23:01:26.795557+0000","last_undegraded":"2024-04-24T23:01:26.795557+0000","last_fullsized":"2024-04-24T23:01:26.795557+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:30.122415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795594+0000","last_change":"2024-04-24T23:01:24.668280+0000","last_active":"2024-04-24T23:01:26.795594+0000","last_peered":"2024-04-24T23:01:26.795594+0000","last_clean":"2024-04-24T23:01:26.795594+0000","last_became_active":"2024-04-24T23:01:24.668049+0000","last_became_peered":"2024-04-24T23:01:24.668049+0000","last_unstale":"2024-04-24T23:01:26.795594+0000","last_undegraded":"2024-04-24T23:01:26.795594+0000","last_fullsized":"2024-04-24T23:01:26.795594+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:19.190128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795324+0000","last_change":"2024-04-24T23:01:25.692475+0000","last_active":"2024-04-24T23:01:26.795324+0000","last_peered":"2024-04-24T23:01:26.795324+0000","last_clean":"2024-04-24T23:01:26.795324+0000","last_became_active":"2024-04-24T23:01:25.692213+0000","last_became_peered":"2024-04-24T23:01:25.692213+0000","last_unstale":"2024-04-24T23:01:26.795324+0000","last_undegraded":"2024-04-24T23:01:26.795324+0000","last_fullsized":"2024-04-24T23:01:26.795324+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:26:03.587432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795258+0000","last_change":"2024-04-24T23:01:24.675164+0000","last_active":"2024-04-24T23:01:26.795258+0000","last_peered":"2024-04-24T23:01:26.795258+0000","last_clean":"2024-04-24T23:01:26.795258+0000","last_became_active":"2024-04-24T23:01:24.674910+0000","last_became_peered":"2024-04-24T23:01:24.674910+0000","last_unstale":"2024-04-24T23:01:26.795258+0000","last_undegraded":"2024-04-24T23:01:26.795258+0000","last_fullsized":"2024-04-24T23:01:26.795258+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:14:14.969110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795833+0000","last_change":"2024-04-24T23:01:25.694095+0000","last_active":"2024-04-24T23:01:26.795833+0000","last_peered":"2024-04-24T23:01:26.795833+0000","last_clean":"2024-04-24T23:01:26.795833+0000","last_became_active":"2024-04-24T23:01:25.693376+0000","last_became_peered":"2024-04-24T23:01:25.693376+0000","last_unstale":"2024-04-24T23:01:26.795833+0000","last_undegraded":"2024-04-24T23:01:26.795833+0000","last_fullsized":"2024-04-24T23:01:26.795833+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:03:31.570479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.796665+0000","last_change":"2024-04-24T23:01:24.678150+0000","last_active":"2024-04-24T23:01:26.796665+0000","last_peered":"2024-04-24T23:01:26.796665+0000","last_clean":"2024-04-24T23:01:26.796665+0000","last_became_active":"2024-04-24T23:01:24.677094+0000","last_became_peered":"2024-04-24T23:01:24.677094+0000","last_unstale":"2024-04-24T23:01:26.796665+0000","last_undegraded":"2024-04-24T23:01:26.796665+0000","last_fullsized":"2024-04-24T23:01:26.796665+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:19.392137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.796204+0000","last_change":"2024-04-24T23:01:25.685502+0000","last_active":"2024-04-24T23:01:26.796204+0000","last_peered":"2024-04-24T23:01:26.796204+0000","last_clean":"2024-04-24T23:01:26.796204+0000","last_became_active":"2024-04-24T23:01:25.685262+0000","last_became_peered":"2024-04-24T23:01:25.685262+0000","last_unstale":"2024-04-24T23:01:26.796204+0000","last_undegraded":"2024-04-24T23:01:26.796204+0000","last_fullsized":"2024-04-24T23:01:26.796204+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:30:40.588733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.796077+0000","last_change":"2024-04-24T23:01:24.668762+0000","last_active":"2024-04-24T23:01:26.796077+0000","last_peered":"2024-04-24T23:01:26.796077+0000","last_clean":"2024-04-24T23:01:26.796077+0000","last_became_active":"2024-04-24T23:01:24.668498+0000","last_became_peered":"2024-04-24T23:01:24.668498+0000","last_unstale":"2024-04-24T23:01:26.796077+0000","last_undegraded":"2024-04-24T23:01:26.796077+0000","last_fullsized":"2024-04-24T23:01:26.796077+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:30:33.504533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.796430+0000","last_change":"2024-04-24T23:01:25.685651+0000","last_active":"2024-04-24T23:01:26.796430+0000","last_peered":"2024-04-24T23:01:26.796430+0000","last_clean":"2024-04-24T23:01:26.796430+0000","last_became_active":"2024-04-24T23:01:25.685372+0000","last_became_peered":"2024-04-24T23:01:25.685372+0000","last_unstale":"2024-04-24T23:01:26.796430+0000","last_undegraded":"2024-04-24T23:01:26.796430+0000","last_fullsized":"2024-04-24T23:01:26.796430+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:19:05.047211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795480+0000","last_change":"2024-04-24T23:01:24.675175+0000","last_active":"2024-04-24T23:01:26.795480+0000","last_peered":"2024-04-24T23:01:26.795480+0000","last_clean":"2024-04-24T23:01:26.795480+0000","last_became_active":"2024-04-24T23:01:24.674880+0000","last_became_peered":"2024-04-24T23:01:24.674880+0000","last_unstale":"2024-04-24T23:01:26.795480+0000","last_undegraded":"2024-04-24T23:01:26.795480+0000","last_fullsized":"2024-04-24T23:01:26.795480+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:17:35.597607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":257,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.680852+0000","last_change":"2024-04-24T22:58:55.138810+0000","last_active":"2024-04-24T23:01:26.680852+0000","last_peered":"2024-04-24T23:01:26.680852+0000","last_clean":"2024-04-24T23:01:26.680852+0000","last_became_active":"2024-04-24T22:58:55.138627+0000","last_became_peered":"2024-04-24T22:58:55.138627+0000","last_unstale":"2024-04-24T23:01:26.680852+0000","last_undegraded":"2024-04-24T23:01:26.680852+0000","last_fullsized":"2024-04-24T23:01:26.680852+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_clean_scrub_stamp":"2024-04-24T22:58:54.127235+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:06:31.042429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":16,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":331,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194980,"statfs":{"total":200000000,"available":199660170,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":31,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":331,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194980,"statfs":{"total":200000000,"available":199660240,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":914,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194398,"statfs":{"total":200000000,"available":199063560,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":332,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194980,"statfs":{"total":200000000,"available":199659646,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":338,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194974,"statfs":{"total":200000000,"available":199653872,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":333,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194978,"statfs":{"total":200000000,"available":199658490,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":915,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194397,"statfs":{"total":200000000,"available":199063012,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":339,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194972,"statfs":{"total":200000000,"available":199652350,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-24T23:01:28.701 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:01:28.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.836+0000 7f2c8e612640 1 Processor -- start 2024-04-24T23:01:28.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.836+0000 7f2c8e612640 1 -- start start 2024-04-24T23:01:28.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.837+0000 7f2c8e612640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2c880af890 0x7f2c881670f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:28.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.837+0000 7f2c8e612640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c880b6430 con 0x7f2c880b44d0 2024-04-24T23:01:28.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.837+0000 7f2c8e612640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c88156f80 con 0x7f2c8806c5f0 2024-04-24T23:01:28.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.837+0000 7f2c8e612640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f2c880b6150 con 0x7f2c880af890 2024-04-24T23:01:28.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.837+0000 7f2c8cbaf640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2c880af890 0x7f2c881670f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:28.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.837+0000 7f2c877fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2c880b44d0 0x7f2c880b5890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43088/0 (socket says 172.21.15.32:43088) 2024-04-24T23:01:28.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.837+0000 7f2c877fe640 1 -- 172.21.15.32:0/3620793540 learned_addr learned my addr 172.21.15.32:0/3620793540 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:01:28.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.837+0000 7f2c86ffd640 1 -- 172.21.15.32:0/3620793540 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2964033469 0 0) 0x7f2c880b6430 con 0x7f2c880b44d0 2024-04-24T23:01:28.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.837+0000 7f2c86ffd640 1 -- 172.21.15.32:0/3620793540 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2c74003520 con 0x7f2c880b44d0 2024-04-24T23:01:28.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.837+0000 7f2c8cbaf640 1 -- 172.21.15.32:0/3620793540 >> v1:172.21.15.121:6789/0 conn(0x7f2c8806c5f0 legacy=0x7f2c8806c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:28.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.837+0000 7f2c8cbaf640 1 -- 172.21.15.32:0/3620793540 >> v1:172.21.15.32:6789/0 conn(0x7f2c880b44d0 legacy=0x7f2c880b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:28.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.837+0000 7f2c8cbaf640 1 -- 172.21.15.32:0/3620793540 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c880b39f0 con 0x7f2c880af890 2024-04-24T23:01:28.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.838+0000 7f2c8cbaf640 1 --2- 172.21.15.32:0/3620793540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c880af890 0x7f2c881670f0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f2c80009d90 tx=0x7f2c8002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=513704202104eb27 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:28.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.838+0000 7f2c86ffd640 1 -- 172.21.15.32:0/3620793540 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c80007740 con 0x7f2c880af890 2024-04-24T23:01:28.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.838+0000 7f2c86ffd640 1 -- 172.21.15.32:0/3620793540 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c800078e0 con 0x7f2c880af890 2024-04-24T23:01:28.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.838+0000 7f2c86ffd640 1 -- 172.21.15.32:0/3620793540 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c80007c90 con 0x7f2c880af890 2024-04-24T23:01:28.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.839+0000 7f2c8e612640 1 -- 172.21.15.32:0/3620793540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c880af890 msgr2=0x7f2c881670f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:28.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.839+0000 7f2c8e612640 1 --2- 172.21.15.32:0/3620793540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c880af890 0x7f2c881670f0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f2c80009d90 tx=0x7f2c8002a6a0 comp rx=0 tx=0).stop 2024-04-24T23:01:28.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.839+0000 7f2c8e612640 1 -- 172.21.15.32:0/3620793540 shutdown_connections 2024-04-24T23:01:28.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.839+0000 7f2c8e612640 1 --2- 172.21.15.32:0/3620793540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c880af890 0x7f2c881670f0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:28.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.839+0000 7f2c8e612640 1 -- 172.21.15.32:0/3620793540 >> 172.21.15.32:0/3620793540 conn(0x7f2c8806b110 msgr2=0x7f2c8806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:28.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.839+0000 7f2c8e612640 1 -- 172.21.15.32:0/3620793540 shutdown_connections 2024-04-24T23:01:28.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.839+0000 7f2c8e612640 1 -- 172.21.15.32:0/3620793540 wait complete. 2024-04-24T23:01:28.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.840+0000 7f2c8e612640 1 Processor -- start 2024-04-24T23:01:28.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.840+0000 7f2c8e612640 1 -- start start 2024-04-24T23:01:28.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.841+0000 7f2c8e612640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c8806c5f0 0x7f2c88150a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:28.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.841+0000 7f2c8e612640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c880af890 0x7f2c8813ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:28.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.841+0000 7f2c8e612640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c880b44d0 0x7f2c8813cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:28.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.841+0000 7f2c8e612640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2c88167630 con 0x7f2c8806c5f0 2024-04-24T23:01:28.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.841+0000 7f2c8e612640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2c880b3fb0 con 0x7f2c880af890 2024-04-24T23:01:28.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.841+0000 7f2c8e612640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f2c881677b0 con 0x7f2c880b44d0 2024-04-24T23:01:28.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.841+0000 7f2c87fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c8806c5f0 0x7f2c88150a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:28.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.841+0000 7f2c87fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c8806c5f0 0x7f2c88150a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41884/0 (socket says 172.21.15.32:41884) 2024-04-24T23:01:28.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.841+0000 7f2c87fff640 1 -- 172.21.15.32:0/271826261 learned_addr learned my addr 172.21.15.32:0/271826261 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:28.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.841+0000 7f2c87fff640 1 -- 172.21.15.32:0/271826261 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c880b44d0 msgr2=0x7f2c8813cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:28.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.841+0000 7f2c87fff640 1 --2- 172.21.15.32:0/271826261 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c880b44d0 0x7f2c8813cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:28.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.841+0000 7f2c877fe640 1 --2- 172.21.15.32:0/271826261 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c880af890 0x7f2c8813ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:28.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.841+0000 7f2c87fff640 1 -- 172.21.15.32:0/271826261 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c880af890 msgr2=0x7f2c8813ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:28.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.841+0000 7f2c87fff640 1 --2- 172.21.15.32:0/271826261 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c880af890 0x7f2c8813ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:28.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.841+0000 7f2c87fff640 1 -- 172.21.15.32:0/271826261 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c8813b730 con 0x7f2c8806c5f0 2024-04-24T23:01:28.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.841+0000 7f2c877fe640 1 --2- 172.21.15.32:0/271826261 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c880af890 0x7f2c8813ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:01:28.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.841+0000 7f2c87fff640 1 --2- 172.21.15.32:0/271826261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c8806c5f0 0x7f2c88150a70 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f2c7000c430 tx=0x7f2c7000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:28.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.842+0000 7f2c84ff9640 1 -- 172.21.15.32:0/271826261 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c70011790 con 0x7f2c8806c5f0 2024-04-24T23:01:28.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.842+0000 7f2c84ff9640 1 -- 172.21.15.32:0/271826261 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c70017070 con 0x7f2c8806c5f0 2024-04-24T23:01:28.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.842+0000 7f2c8e612640 1 -- 172.21.15.32:0/271826261 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c8813ba10 con 0x7f2c8806c5f0 2024-04-24T23:01:28.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.842+0000 7f2c84ff9640 1 -- 172.21.15.32:0/271826261 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c70011bb0 con 0x7f2c8806c5f0 2024-04-24T23:01:28.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.842+0000 7f2c8e612640 1 -- 172.21.15.32:0/271826261 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c8813bfd0 con 0x7f2c8806c5f0 2024-04-24T23:01:28.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.843+0000 7f2c8e612640 1 -- 172.21.15.32:0/271826261 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c88068430 con 0x7f2c8806c5f0 2024-04-24T23:01:28.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.843+0000 7f2c84ff9640 1 -- 172.21.15.32:0/271826261 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2c7001c460 con 0x7f2c8806c5f0 2024-04-24T23:01:28.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.843+0000 7f2c84ff9640 1 --2- 172.21.15.32:0/271826261 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2c6806be30 0x7f2c6806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:28.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.844+0000 7f2c84ff9640 1 -- 172.21.15.32:0/271826261 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7f2c7008ad30 con 0x7f2c8806c5f0 2024-04-24T23:01:28.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.844+0000 7f2c877fe640 1 --2- 172.21.15.32:0/271826261 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2c6806be30 0x7f2c6806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:28.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.844+0000 7f2c877fe640 1 --2- 172.21.15.32:0/271826261 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2c6806be30 0x7f2c6806e2f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2c88141380 tx=0x7f2c78009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:28.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:28.848+0000 7f2c84ff9640 1 -- 172.21.15.32:0/271826261 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2c700596d0 con 0x7f2c8806c5f0 2024-04-24T23:01:29.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.027+0000 7f2c8e612640 1 -- 172.21.15.32:0/271826261 --> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2c88155d40 con 0x7f2c6806be30 2024-04-24T23:01:29.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.034+0000 7f2c84ff9640 1 -- 172.21.15.32:0/271826261 <== mgr.4108 v2:172.21.15.121:6832/1848157943 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247489 (secure 0 0 0) 0x7f2c88155d40 con 0x7f2c6806be30 2024-04-24T23:01:29.038 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:01:29.039 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-24T23:01:29.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.037+0000 7f2c8e612640 1 -- 172.21.15.32:0/271826261 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2c6806be30 msgr2=0x7f2c6806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:29.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.037+0000 7f2c8e612640 1 --2- 172.21.15.32:0/271826261 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2c6806be30 0x7f2c6806e2f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2c88141380 tx=0x7f2c78009040 comp rx=0 tx=0).stop 2024-04-24T23:01:29.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.037+0000 7f2c8e612640 1 -- 172.21.15.32:0/271826261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c8806c5f0 msgr2=0x7f2c88150a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:29.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.037+0000 7f2c8e612640 1 --2- 172.21.15.32:0/271826261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c8806c5f0 0x7f2c88150a70 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f2c7000c430 tx=0x7f2c7000c8f0 comp rx=0 tx=0).stop 2024-04-24T23:01:29.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.037+0000 7f2c8e612640 1 -- 172.21.15.32:0/271826261 shutdown_connections 2024-04-24T23:01:29.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.037+0000 7f2c8e612640 1 --2- 172.21.15.32:0/271826261 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2c6806be30 0x7f2c6806e2f0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:29.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.037+0000 7f2c8e612640 1 --2- 172.21.15.32:0/271826261 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c880b44d0 0x7f2c8813cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:29.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.037+0000 7f2c8e612640 1 --2- 172.21.15.32:0/271826261 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c880af890 0x7f2c8813ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:29.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.037+0000 7f2c8e612640 1 --2- 172.21.15.32:0/271826261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c8806c5f0 0x7f2c88150a70 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:29.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.037+0000 7f2c8e612640 1 -- 172.21.15.32:0/271826261 >> 172.21.15.32:0/271826261 conn(0x7f2c8806b110 msgr2=0x7f2c8806c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:29.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.037+0000 7f2c8e612640 1 -- 172.21.15.32:0/271826261 shutdown_connections 2024-04-24T23:01:29.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.037+0000 7f2c8e612640 1 -- 172.21.15.32:0/271826261 wait complete. 2024-04-24T23:01:29.049 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":136,"stamp":"2024-04-24T23:01:27.159414+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":141,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":3836,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1558663,"statfs":{"total":1600000000,"available":1596071340,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000806"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.796748+0000","last_change":"2024-04-24T23:01:25.691368+0000","last_active":"2024-04-24T23:01:26.796748+0000","last_peered":"2024-04-24T23:01:26.796748+0000","last_clean":"2024-04-24T23:01:26.796748+0000","last_became_active":"2024-04-24T23:01:25.687857+0000","last_became_peered":"2024-04-24T23:01:25.687857+0000","last_unstale":"2024-04-24T23:01:26.796748+0000","last_undegraded":"2024-04-24T23:01:26.796748+0000","last_fullsized":"2024-04-24T23:01:26.796748+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:58:39.507832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.796217+0000","last_change":"2024-04-24T23:01:25.694501+0000","last_active":"2024-04-24T23:01:26.796217+0000","last_peered":"2024-04-24T23:01:26.796217+0000","last_clean":"2024-04-24T23:01:26.796217+0000","last_became_active":"2024-04-24T23:01:25.694272+0000","last_became_peered":"2024-04-24T23:01:25.694272+0000","last_unstale":"2024-04-24T23:01:26.796217+0000","last_undegraded":"2024-04-24T23:01:26.796217+0000","last_fullsized":"2024-04-24T23:01:26.796217+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:53:32.039806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795450+0000","last_change":"2024-04-24T23:01:25.685044+0000","last_active":"2024-04-24T23:01:26.795450+0000","last_peered":"2024-04-24T23:01:26.795450+0000","last_clean":"2024-04-24T23:01:26.795450+0000","last_became_active":"2024-04-24T23:01:25.684733+0000","last_became_peered":"2024-04-24T23:01:25.684733+0000","last_unstale":"2024-04-24T23:01:26.795450+0000","last_undegraded":"2024-04-24T23:01:26.795450+0000","last_fullsized":"2024-04-24T23:01:26.795450+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:15:14.681765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.677486+0000","last_change":"2024-04-24T23:01:25.680115+0000","last_active":"2024-04-24T23:01:26.677486+0000","last_peered":"2024-04-24T23:01:26.677486+0000","last_clean":"2024-04-24T23:01:26.677486+0000","last_became_active":"2024-04-24T23:01:25.679838+0000","last_became_peered":"2024-04-24T23:01:25.679838+0000","last_unstale":"2024-04-24T23:01:26.677486+0000","last_undegraded":"2024-04-24T23:01:26.677486+0000","last_fullsized":"2024-04-24T23:01:26.677486+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:37:48.507918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795953+0000","last_change":"2024-04-24T23:01:25.684189+0000","last_active":"2024-04-24T23:01:26.795953+0000","last_peered":"2024-04-24T23:01:26.795953+0000","last_clean":"2024-04-24T23:01:26.795953+0000","last_became_active":"2024-04-24T23:01:25.683490+0000","last_became_peered":"2024-04-24T23:01:25.683490+0000","last_unstale":"2024-04-24T23:01:26.795953+0000","last_undegraded":"2024-04-24T23:01:26.795953+0000","last_fullsized":"2024-04-24T23:01:26.795953+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:05.975380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.796834+0000","last_change":"2024-04-24T23:01:25.691587+0000","last_active":"2024-04-24T23:01:26.796834+0000","last_peered":"2024-04-24T23:01:26.796834+0000","last_clean":"2024-04-24T23:01:26.796834+0000","last_became_active":"2024-04-24T23:01:25.688792+0000","last_became_peered":"2024-04-24T23:01:25.688792+0000","last_unstale":"2024-04-24T23:01:26.796834+0000","last_undegraded":"2024-04-24T23:01:26.796834+0000","last_fullsized":"2024-04-24T23:01:26.796834+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:16:30.812523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.677496+0000","last_change":"2024-04-24T23:01:25.679660+0000","last_active":"2024-04-24T23:01:26.677496+0000","last_peered":"2024-04-24T23:01:26.677496+0000","last_clean":"2024-04-24T23:01:26.677496+0000","last_became_active":"2024-04-24T23:01:25.679354+0000","last_became_peered":"2024-04-24T23:01:25.679354+0000","last_unstale":"2024-04-24T23:01:26.677496+0000","last_undegraded":"2024-04-24T23:01:26.677496+0000","last_fullsized":"2024-04-24T23:01:26.677496+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:10.435460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795418+0000","last_change":"2024-04-24T23:01:25.689661+0000","last_active":"2024-04-24T23:01:26.795418+0000","last_peered":"2024-04-24T23:01:26.795418+0000","last_clean":"2024-04-24T23:01:26.795418+0000","last_became_active":"2024-04-24T23:01:25.689215+0000","last_became_peered":"2024-04-24T23:01:25.689215+0000","last_unstale":"2024-04-24T23:01:26.795418+0000","last_undegraded":"2024-04-24T23:01:26.795418+0000","last_fullsized":"2024-04-24T23:01:26.795418+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:25.526121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.680521+0000","last_change":"2024-04-24T23:01:25.694295+0000","last_active":"2024-04-24T23:01:26.680521+0000","last_peered":"2024-04-24T23:01:26.680521+0000","last_clean":"2024-04-24T23:01:26.680521+0000","last_became_active":"2024-04-24T23:01:25.693633+0000","last_became_peered":"2024-04-24T23:01:25.693633+0000","last_unstale":"2024-04-24T23:01:26.680521+0000","last_undegraded":"2024-04-24T23:01:26.680521+0000","last_fullsized":"2024-04-24T23:01:26.680521+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:02:18.861058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.796108+0000","last_change":"2024-04-24T23:01:25.688030+0000","last_active":"2024-04-24T23:01:26.796108+0000","last_peered":"2024-04-24T23:01:26.796108+0000","last_clean":"2024-04-24T23:01:26.796108+0000","last_became_active":"2024-04-24T23:01:25.687772+0000","last_became_peered":"2024-04-24T23:01:25.687772+0000","last_unstale":"2024-04-24T23:01:26.796108+0000","last_undegraded":"2024-04-24T23:01:26.796108+0000","last_fullsized":"2024-04-24T23:01:26.796108+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:57:34.794353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.680819+0000","last_change":"2024-04-24T23:01:25.697315+0000","last_active":"2024-04-24T23:01:26.680819+0000","last_peered":"2024-04-24T23:01:26.680819+0000","last_clean":"2024-04-24T23:01:26.680819+0000","last_became_active":"2024-04-24T23:01:25.697073+0000","last_became_peered":"2024-04-24T23:01:25.697073+0000","last_unstale":"2024-04-24T23:01:26.680819+0000","last_undegraded":"2024-04-24T23:01:26.680819+0000","last_fullsized":"2024-04-24T23:01:26.680819+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:25:29.199057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.680418+0000","last_change":"2024-04-24T23:01:25.687587+0000","last_active":"2024-04-24T23:01:26.680418+0000","last_peered":"2024-04-24T23:01:26.680418+0000","last_clean":"2024-04-24T23:01:26.680418+0000","last_became_active":"2024-04-24T23:01:25.687321+0000","last_became_peered":"2024-04-24T23:01:25.687321+0000","last_unstale":"2024-04-24T23:01:26.680418+0000","last_undegraded":"2024-04-24T23:01:26.680418+0000","last_fullsized":"2024-04-24T23:01:26.680418+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:30.462115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795845+0000","last_change":"2024-04-24T23:01:25.689404+0000","last_active":"2024-04-24T23:01:26.795845+0000","last_peered":"2024-04-24T23:01:26.795845+0000","last_clean":"2024-04-24T23:01:26.795845+0000","last_became_active":"2024-04-24T23:01:25.687976+0000","last_became_peered":"2024-04-24T23:01:25.687976+0000","last_unstale":"2024-04-24T23:01:26.795845+0000","last_undegraded":"2024-04-24T23:01:26.795845+0000","last_fullsized":"2024-04-24T23:01:26.795845+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:11.552562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795303+0000","last_change":"2024-04-24T23:01:25.686179+0000","last_active":"2024-04-24T23:01:26.795303+0000","last_peered":"2024-04-24T23:01:26.795303+0000","last_clean":"2024-04-24T23:01:26.795303+0000","last_became_active":"2024-04-24T23:01:25.685699+0000","last_became_peered":"2024-04-24T23:01:25.685699+0000","last_unstale":"2024-04-24T23:01:26.795303+0000","last_undegraded":"2024-04-24T23:01:26.795303+0000","last_fullsized":"2024-04-24T23:01:26.795303+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:58.285055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795224+0000","last_change":"2024-04-24T23:01:25.686857+0000","last_active":"2024-04-24T23:01:26.795224+0000","last_peered":"2024-04-24T23:01:26.795224+0000","last_clean":"2024-04-24T23:01:26.795224+0000","last_became_active":"2024-04-24T23:01:25.686568+0000","last_became_peered":"2024-04-24T23:01:25.686568+0000","last_unstale":"2024-04-24T23:01:26.795224+0000","last_undegraded":"2024-04-24T23:01:26.795224+0000","last_fullsized":"2024-04-24T23:01:26.795224+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:27:46.233436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.796916+0000","last_change":"2024-04-24T23:01:25.681029+0000","last_active":"2024-04-24T23:01:26.796916+0000","last_peered":"2024-04-24T23:01:26.796916+0000","last_clean":"2024-04-24T23:01:26.796916+0000","last_became_active":"2024-04-24T23:01:25.680752+0000","last_became_peered":"2024-04-24T23:01:25.680752+0000","last_unstale":"2024-04-24T23:01:26.796916+0000","last_undegraded":"2024-04-24T23:01:26.796916+0000","last_fullsized":"2024-04-24T23:01:26.796916+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:14:28.139891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795398+0000","last_change":"2024-04-24T23:01:25.684469+0000","last_active":"2024-04-24T23:01:26.795398+0000","last_peered":"2024-04-24T23:01:26.795398+0000","last_clean":"2024-04-24T23:01:26.795398+0000","last_became_active":"2024-04-24T23:01:25.684190+0000","last_became_peered":"2024-04-24T23:01:25.684190+0000","last_unstale":"2024-04-24T23:01:26.795398+0000","last_undegraded":"2024-04-24T23:01:26.795398+0000","last_fullsized":"2024-04-24T23:01:26.795398+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:08:04.009629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795589+0000","last_change":"2024-04-24T23:01:25.688192+0000","last_active":"2024-04-24T23:01:26.795589+0000","last_peered":"2024-04-24T23:01:26.795589+0000","last_clean":"2024-04-24T23:01:26.795589+0000","last_became_active":"2024-04-24T23:01:25.686705+0000","last_became_peered":"2024-04-24T23:01:25.686705+0000","last_unstale":"2024-04-24T23:01:26.795589+0000","last_undegraded":"2024-04-24T23:01:26.795589+0000","last_fullsized":"2024-04-24T23:01:26.795589+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:17:48.971011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795341+0000","last_change":"2024-04-24T23:01:25.687816+0000","last_active":"2024-04-24T23:01:26.795341+0000","last_peered":"2024-04-24T23:01:26.795341+0000","last_clean":"2024-04-24T23:01:26.795341+0000","last_became_active":"2024-04-24T23:01:25.687580+0000","last_became_peered":"2024-04-24T23:01:25.687580+0000","last_unstale":"2024-04-24T23:01:26.795341+0000","last_undegraded":"2024-04-24T23:01:26.795341+0000","last_fullsized":"2024-04-24T23:01:26.795341+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:49.976810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.680826+0000","last_change":"2024-04-24T23:01:25.694317+0000","last_active":"2024-04-24T23:01:26.680826+0000","last_peered":"2024-04-24T23:01:26.680826+0000","last_clean":"2024-04-24T23:01:26.680826+0000","last_became_active":"2024-04-24T23:01:25.694095+0000","last_became_peered":"2024-04-24T23:01:25.694095+0000","last_unstale":"2024-04-24T23:01:26.680826+0000","last_undegraded":"2024-04-24T23:01:26.680826+0000","last_fullsized":"2024-04-24T23:01:26.680826+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:39:17.395666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795043+0000","last_change":"2024-04-24T23:01:25.683853+0000","last_active":"2024-04-24T23:01:26.795043+0000","last_peered":"2024-04-24T23:01:26.795043+0000","last_clean":"2024-04-24T23:01:26.795043+0000","last_became_active":"2024-04-24T23:01:25.683268+0000","last_became_peered":"2024-04-24T23:01:25.683268+0000","last_unstale":"2024-04-24T23:01:26.795043+0000","last_undegraded":"2024-04-24T23:01:26.795043+0000","last_fullsized":"2024-04-24T23:01:26.795043+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:16:33.422056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.680846+0000","last_change":"2024-04-24T23:01:25.695665+0000","last_active":"2024-04-24T23:01:26.680846+0000","last_peered":"2024-04-24T23:01:26.680846+0000","last_clean":"2024-04-24T23:01:26.680846+0000","last_became_active":"2024-04-24T23:01:25.695050+0000","last_became_peered":"2024-04-24T23:01:25.695050+0000","last_unstale":"2024-04-24T23:01:26.680846+0000","last_undegraded":"2024-04-24T23:01:26.680846+0000","last_fullsized":"2024-04-24T23:01:26.680846+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:20:19.138677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.677396+0000","last_change":"2024-04-24T23:01:25.686702+0000","last_active":"2024-04-24T23:01:26.677396+0000","last_peered":"2024-04-24T23:01:26.677396+0000","last_clean":"2024-04-24T23:01:26.677396+0000","last_became_active":"2024-04-24T23:01:25.686451+0000","last_became_peered":"2024-04-24T23:01:25.686451+0000","last_unstale":"2024-04-24T23:01:26.677396+0000","last_undegraded":"2024-04-24T23:01:26.677396+0000","last_fullsized":"2024-04-24T23:01:26.677396+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:36.030977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.680380+0000","last_change":"2024-04-24T23:01:25.694438+0000","last_active":"2024-04-24T23:01:26.680380+0000","last_peered":"2024-04-24T23:01:26.680380+0000","last_clean":"2024-04-24T23:01:26.680380+0000","last_became_active":"2024-04-24T23:01:25.693892+0000","last_became_peered":"2024-04-24T23:01:25.693892+0000","last_unstale":"2024-04-24T23:01:26.680380+0000","last_undegraded":"2024-04-24T23:01:26.680380+0000","last_fullsized":"2024-04-24T23:01:26.680380+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:16:06.768138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795823+0000","last_change":"2024-04-24T23:01:25.690611+0000","last_active":"2024-04-24T23:01:26.795823+0000","last_peered":"2024-04-24T23:01:26.795823+0000","last_clean":"2024-04-24T23:01:26.795823+0000","last_became_active":"2024-04-24T23:01:25.690228+0000","last_became_peered":"2024-04-24T23:01:25.690228+0000","last_unstale":"2024-04-24T23:01:26.795823+0000","last_undegraded":"2024-04-24T23:01:26.795823+0000","last_fullsized":"2024-04-24T23:01:26.795823+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:30:01.210701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795188+0000","last_change":"2024-04-24T23:01:25.683526+0000","last_active":"2024-04-24T23:01:26.795188+0000","last_peered":"2024-04-24T23:01:26.795188+0000","last_clean":"2024-04-24T23:01:26.795188+0000","last_became_active":"2024-04-24T23:01:25.682995+0000","last_became_peered":"2024-04-24T23:01:25.682995+0000","last_unstale":"2024-04-24T23:01:26.795188+0000","last_undegraded":"2024-04-24T23:01:26.795188+0000","last_fullsized":"2024-04-24T23:01:26.795188+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:36.893814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.796471+0000","last_change":"2024-04-24T23:01:25.692849+0000","last_active":"2024-04-24T23:01:26.796471+0000","last_peered":"2024-04-24T23:01:26.796471+0000","last_clean":"2024-04-24T23:01:26.796471+0000","last_became_active":"2024-04-24T23:01:25.692318+0000","last_became_peered":"2024-04-24T23:01:25.692318+0000","last_unstale":"2024-04-24T23:01:26.796471+0000","last_undegraded":"2024-04-24T23:01:26.796471+0000","last_fullsized":"2024-04-24T23:01:26.796471+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:27:22.189961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795576+0000","last_change":"2024-04-24T23:01:25.694348+0000","last_active":"2024-04-24T23:01:26.795576+0000","last_peered":"2024-04-24T23:01:26.795576+0000","last_clean":"2024-04-24T23:01:26.795576+0000","last_became_active":"2024-04-24T23:01:25.694081+0000","last_became_peered":"2024-04-24T23:01:25.694081+0000","last_unstale":"2024-04-24T23:01:26.795576+0000","last_undegraded":"2024-04-24T23:01:26.795576+0000","last_fullsized":"2024-04-24T23:01:26.795576+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:00:27.519286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795376+0000","last_change":"2024-04-24T23:01:25.691847+0000","last_active":"2024-04-24T23:01:26.795376+0000","last_peered":"2024-04-24T23:01:26.795376+0000","last_clean":"2024-04-24T23:01:26.795376+0000","last_became_active":"2024-04-24T23:01:25.689754+0000","last_became_peered":"2024-04-24T23:01:25.689754+0000","last_unstale":"2024-04-24T23:01:26.795376+0000","last_undegraded":"2024-04-24T23:01:26.795376+0000","last_fullsized":"2024-04-24T23:01:26.795376+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:37.207955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.677600+0000","last_change":"2024-04-24T23:01:25.684143+0000","last_active":"2024-04-24T23:01:26.677600+0000","last_peered":"2024-04-24T23:01:26.677600+0000","last_clean":"2024-04-24T23:01:26.677600+0000","last_became_active":"2024-04-24T23:01:25.683879+0000","last_became_peered":"2024-04-24T23:01:25.683879+0000","last_unstale":"2024-04-24T23:01:26.677600+0000","last_undegraded":"2024-04-24T23:01:26.677600+0000","last_fullsized":"2024-04-24T23:01:26.677600+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:04:52.760103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795500+0000","last_change":"2024-04-24T23:01:25.689471+0000","last_active":"2024-04-24T23:01:26.795500+0000","last_peered":"2024-04-24T23:01:26.795500+0000","last_clean":"2024-04-24T23:01:26.795500+0000","last_became_active":"2024-04-24T23:01:25.689059+0000","last_became_peered":"2024-04-24T23:01:25.689059+0000","last_unstale":"2024-04-24T23:01:26.795500+0000","last_undegraded":"2024-04-24T23:01:26.795500+0000","last_fullsized":"2024-04-24T23:01:26.795500+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:32:55.533168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.681018+0000","last_change":"2024-04-24T23:01:25.695467+0000","last_active":"2024-04-24T23:01:26.681018+0000","last_peered":"2024-04-24T23:01:26.681018+0000","last_clean":"2024-04-24T23:01:26.681018+0000","last_became_active":"2024-04-24T23:01:25.694710+0000","last_became_peered":"2024-04-24T23:01:25.694710+0000","last_unstale":"2024-04-24T23:01:26.681018+0000","last_undegraded":"2024-04-24T23:01:26.681018+0000","last_fullsized":"2024-04-24T23:01:26.681018+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:16:14.961098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795252+0000","last_change":"2024-04-24T23:01:25.691425+0000","last_active":"2024-04-24T23:01:26.795252+0000","last_peered":"2024-04-24T23:01:26.795252+0000","last_clean":"2024-04-24T23:01:26.795252+0000","last_became_active":"2024-04-24T23:01:25.687581+0000","last_became_peered":"2024-04-24T23:01:25.687581+0000","last_unstale":"2024-04-24T23:01:26.795252+0000","last_undegraded":"2024-04-24T23:01:26.795252+0000","last_fullsized":"2024-04-24T23:01:26.795252+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:32:29.563928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"66'4","reported_seq":18,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.680888+0000","last_change":"2024-04-24T23:01:24.675437+0000","last_active":"2024-04-24T23:01:26.680888+0000","last_peered":"2024-04-24T23:01:26.680888+0000","last_clean":"2024-04-24T23:01:26.680888+0000","last_became_active":"2024-04-24T23:01:24.675202+0000","last_became_peered":"2024-04-24T23:01:24.675202+0000","last_unstale":"2024-04-24T23:01:26.680888+0000","last_undegraded":"2024-04-24T23:01:26.680888+0000","last_fullsized":"2024-04-24T23:01:26.680888+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:11:40.898914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.796569+0000","last_change":"2024-04-24T23:01:25.684935+0000","last_active":"2024-04-24T23:01:26.796569+0000","last_peered":"2024-04-24T23:01:26.796569+0000","last_clean":"2024-04-24T23:01:26.796569+0000","last_became_active":"2024-04-24T23:01:25.684677+0000","last_became_peered":"2024-04-24T23:01:25.684677+0000","last_unstale":"2024-04-24T23:01:26.796569+0000","last_undegraded":"2024-04-24T23:01:26.796569+0000","last_fullsized":"2024-04-24T23:01:26.796569+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:57:35.708359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795453+0000","last_change":"2024-04-24T23:01:24.668594+0000","last_active":"2024-04-24T23:01:26.795453+0000","last_peered":"2024-04-24T23:01:26.795453+0000","last_clean":"2024-04-24T23:01:26.795453+0000","last_became_active":"2024-04-24T23:01:24.668204+0000","last_became_peered":"2024-04-24T23:01:24.668204+0000","last_unstale":"2024-04-24T23:01:26.795453+0000","last_undegraded":"2024-04-24T23:01:26.795453+0000","last_fullsized":"2024-04-24T23:01:26.795453+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:15.955906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.796290+0000","last_change":"2024-04-24T23:01:25.686989+0000","last_active":"2024-04-24T23:01:26.796290+0000","last_peered":"2024-04-24T23:01:26.796290+0000","last_clean":"2024-04-24T23:01:26.796290+0000","last_became_active":"2024-04-24T23:01:25.686769+0000","last_became_peered":"2024-04-24T23:01:25.686769+0000","last_unstale":"2024-04-24T23:01:26.796290+0000","last_undegraded":"2024-04-24T23:01:26.796290+0000","last_fullsized":"2024-04-24T23:01:26.796290+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:01:04.547945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.796451+0000","last_change":"2024-04-24T23:01:24.672026+0000","last_active":"2024-04-24T23:01:26.796451+0000","last_peered":"2024-04-24T23:01:26.796451+0000","last_clean":"2024-04-24T23:01:26.796451+0000","last_became_active":"2024-04-24T23:01:24.671743+0000","last_became_peered":"2024-04-24T23:01:24.671743+0000","last_unstale":"2024-04-24T23:01:26.796451+0000","last_undegraded":"2024-04-24T23:01:26.796451+0000","last_fullsized":"2024-04-24T23:01:26.796451+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:43:53.956277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.796336+0000","last_change":"2024-04-24T23:01:25.687012+0000","last_active":"2024-04-24T23:01:26.796336+0000","last_peered":"2024-04-24T23:01:26.796336+0000","last_clean":"2024-04-24T23:01:26.796336+0000","last_became_active":"2024-04-24T23:01:25.686734+0000","last_became_peered":"2024-04-24T23:01:25.686734+0000","last_unstale":"2024-04-24T23:01:26.796336+0000","last_undegraded":"2024-04-24T23:01:26.796336+0000","last_fullsized":"2024-04-24T23:01:26.796336+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:30:27.317521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.677620+0000","last_change":"2024-04-24T23:01:24.670512+0000","last_active":"2024-04-24T23:01:26.677620+0000","last_peered":"2024-04-24T23:01:26.677620+0000","last_clean":"2024-04-24T23:01:26.677620+0000","last_became_active":"2024-04-24T23:01:24.670278+0000","last_became_peered":"2024-04-24T23:01:24.670278+0000","last_unstale":"2024-04-24T23:01:26.677620+0000","last_undegraded":"2024-04-24T23:01:26.677620+0000","last_fullsized":"2024-04-24T23:01:26.677620+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:50:23.727996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795732+0000","last_change":"2024-04-24T23:01:25.686247+0000","last_active":"2024-04-24T23:01:26.795732+0000","last_peered":"2024-04-24T23:01:26.795732+0000","last_clean":"2024-04-24T23:01:26.795732+0000","last_became_active":"2024-04-24T23:01:25.685961+0000","last_became_peered":"2024-04-24T23:01:25.685961+0000","last_unstale":"2024-04-24T23:01:26.795732+0000","last_undegraded":"2024-04-24T23:01:26.795732+0000","last_fullsized":"2024-04-24T23:01:26.795732+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:52:11.921979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795154+0000","last_change":"2024-04-24T23:01:24.676505+0000","last_active":"2024-04-24T23:01:26.795154+0000","last_peered":"2024-04-24T23:01:26.795154+0000","last_clean":"2024-04-24T23:01:26.795154+0000","last_became_active":"2024-04-24T23:01:24.676128+0000","last_became_peered":"2024-04-24T23:01:24.676128+0000","last_unstale":"2024-04-24T23:01:26.795154+0000","last_undegraded":"2024-04-24T23:01:26.795154+0000","last_fullsized":"2024-04-24T23:01:26.795154+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:35:43.063596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.681233+0000","last_change":"2024-04-24T23:01:25.695268+0000","last_active":"2024-04-24T23:01:26.681233+0000","last_peered":"2024-04-24T23:01:26.681233+0000","last_clean":"2024-04-24T23:01:26.681233+0000","last_became_active":"2024-04-24T23:01:25.694386+0000","last_became_peered":"2024-04-24T23:01:25.694386+0000","last_unstale":"2024-04-24T23:01:26.681233+0000","last_undegraded":"2024-04-24T23:01:26.681233+0000","last_fullsized":"2024-04-24T23:01:26.681233+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:40:43.553477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.796309+0000","last_change":"2024-04-24T23:01:24.678336+0000","last_active":"2024-04-24T23:01:26.796309+0000","last_peered":"2024-04-24T23:01:26.796309+0000","last_clean":"2024-04-24T23:01:26.796309+0000","last_became_active":"2024-04-24T23:01:24.677580+0000","last_became_peered":"2024-04-24T23:01:24.677580+0000","last_unstale":"2024-04-24T23:01:26.796309+0000","last_undegraded":"2024-04-24T23:01:26.796309+0000","last_fullsized":"2024-04-24T23:01:26.796309+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:03:13.884937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795321+0000","last_change":"2024-04-24T23:01:25.687136+0000","last_active":"2024-04-24T23:01:26.795321+0000","last_peered":"2024-04-24T23:01:26.795321+0000","last_clean":"2024-04-24T23:01:26.795321+0000","last_became_active":"2024-04-24T23:01:25.684693+0000","last_became_peered":"2024-04-24T23:01:25.684693+0000","last_unstale":"2024-04-24T23:01:26.795321+0000","last_undegraded":"2024-04-24T23:01:26.795321+0000","last_fullsized":"2024-04-24T23:01:26.795321+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:03:25.489635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.680784+0000","last_change":"2024-04-24T23:01:24.676646+0000","last_active":"2024-04-24T23:01:26.680784+0000","last_peered":"2024-04-24T23:01:26.680784+0000","last_clean":"2024-04-24T23:01:26.680784+0000","last_became_active":"2024-04-24T23:01:24.676390+0000","last_became_peered":"2024-04-24T23:01:24.676390+0000","last_unstale":"2024-04-24T23:01:26.680784+0000","last_undegraded":"2024-04-24T23:01:26.680784+0000","last_fullsized":"2024-04-24T23:01:26.680784+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:39.274912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795580+0000","last_change":"2024-04-24T23:01:25.681512+0000","last_active":"2024-04-24T23:01:26.795580+0000","last_peered":"2024-04-24T23:01:26.795580+0000","last_clean":"2024-04-24T23:01:26.795580+0000","last_became_active":"2024-04-24T23:01:25.681146+0000","last_became_peered":"2024-04-24T23:01:25.681146+0000","last_unstale":"2024-04-24T23:01:26.795580+0000","last_undegraded":"2024-04-24T23:01:26.795580+0000","last_fullsized":"2024-04-24T23:01:26.795580+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:48:44.254638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795389+0000","last_change":"2024-04-24T23:01:24.674059+0000","last_active":"2024-04-24T23:01:26.795389+0000","last_peered":"2024-04-24T23:01:26.795389+0000","last_clean":"2024-04-24T23:01:26.795389+0000","last_became_active":"2024-04-24T23:01:24.673804+0000","last_became_peered":"2024-04-24T23:01:24.673804+0000","last_unstale":"2024-04-24T23:01:26.795389+0000","last_undegraded":"2024-04-24T23:01:26.795389+0000","last_fullsized":"2024-04-24T23:01:26.795389+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:30:36.248416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795584+0000","last_change":"2024-04-24T23:01:25.688965+0000","last_active":"2024-04-24T23:01:26.795584+0000","last_peered":"2024-04-24T23:01:26.795584+0000","last_clean":"2024-04-24T23:01:26.795584+0000","last_became_active":"2024-04-24T23:01:25.688653+0000","last_became_peered":"2024-04-24T23:01:25.688653+0000","last_unstale":"2024-04-24T23:01:26.795584+0000","last_undegraded":"2024-04-24T23:01:26.795584+0000","last_fullsized":"2024-04-24T23:01:26.795584+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:26:49.445951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.680569+0000","last_change":"2024-04-24T23:01:24.674041+0000","last_active":"2024-04-24T23:01:26.680569+0000","last_peered":"2024-04-24T23:01:26.680569+0000","last_clean":"2024-04-24T23:01:26.680569+0000","last_became_active":"2024-04-24T23:01:24.673656+0000","last_became_peered":"2024-04-24T23:01:24.673656+0000","last_unstale":"2024-04-24T23:01:26.680569+0000","last_undegraded":"2024-04-24T23:01:26.680569+0000","last_fullsized":"2024-04-24T23:01:26.680569+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:58:40.427542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.794971+0000","last_change":"2024-04-24T23:01:25.687177+0000","last_active":"2024-04-24T23:01:26.794971+0000","last_peered":"2024-04-24T23:01:26.794971+0000","last_clean":"2024-04-24T23:01:26.794971+0000","last_became_active":"2024-04-24T23:01:25.686782+0000","last_became_peered":"2024-04-24T23:01:25.686782+0000","last_unstale":"2024-04-24T23:01:26.794971+0000","last_undegraded":"2024-04-24T23:01:26.794971+0000","last_fullsized":"2024-04-24T23:01:26.794971+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:19.870378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795780+0000","last_change":"2024-04-24T23:01:24.676788+0000","last_active":"2024-04-24T23:01:26.795780+0000","last_peered":"2024-04-24T23:01:26.795780+0000","last_clean":"2024-04-24T23:01:26.795780+0000","last_became_active":"2024-04-24T23:01:24.676589+0000","last_became_peered":"2024-04-24T23:01:24.676589+0000","last_unstale":"2024-04-24T23:01:26.795780+0000","last_undegraded":"2024-04-24T23:01:26.795780+0000","last_fullsized":"2024-04-24T23:01:26.795780+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:01:42.996449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795523+0000","last_change":"2024-04-24T23:01:25.687994+0000","last_active":"2024-04-24T23:01:26.795523+0000","last_peered":"2024-04-24T23:01:26.795523+0000","last_clean":"2024-04-24T23:01:26.795523+0000","last_became_active":"2024-04-24T23:01:25.687761+0000","last_became_peered":"2024-04-24T23:01:25.687761+0000","last_unstale":"2024-04-24T23:01:26.795523+0000","last_undegraded":"2024-04-24T23:01:26.795523+0000","last_fullsized":"2024-04-24T23:01:26.795523+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:01:06.838635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"66'3","reported_seq":17,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.677735+0000","last_change":"2024-04-24T23:01:24.670733+0000","last_active":"2024-04-24T23:01:26.677735+0000","last_peered":"2024-04-24T23:01:26.677735+0000","last_clean":"2024-04-24T23:01:26.677735+0000","last_became_active":"2024-04-24T23:01:24.670466+0000","last_became_peered":"2024-04-24T23:01:24.670466+0000","last_unstale":"2024-04-24T23:01:26.677735+0000","last_undegraded":"2024-04-24T23:01:26.677735+0000","last_fullsized":"2024-04-24T23:01:26.677735+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:56.197002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.677636+0000","last_change":"2024-04-24T23:01:25.689104+0000","last_active":"2024-04-24T23:01:26.677636+0000","last_peered":"2024-04-24T23:01:26.677636+0000","last_clean":"2024-04-24T23:01:26.677636+0000","last_became_active":"2024-04-24T23:01:25.688826+0000","last_became_peered":"2024-04-24T23:01:25.688826+0000","last_unstale":"2024-04-24T23:01:26.677636+0000","last_undegraded":"2024-04-24T23:01:26.677636+0000","last_fullsized":"2024-04-24T23:01:26.677636+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:14:57.421224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"66'6","reported_seq":20,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795165+0000","last_change":"2024-04-24T23:01:24.675095+0000","last_active":"2024-04-24T23:01:26.795165+0000","last_peered":"2024-04-24T23:01:26.795165+0000","last_clean":"2024-04-24T23:01:26.795165+0000","last_became_active":"2024-04-24T23:01:24.674835+0000","last_became_peered":"2024-04-24T23:01:24.674835+0000","last_unstale":"2024-04-24T23:01:26.795165+0000","last_undegraded":"2024-04-24T23:01:26.795165+0000","last_fullsized":"2024-04-24T23:01:26.795165+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:05:20.246031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.680477+0000","last_change":"2024-04-24T23:01:25.685986+0000","last_active":"2024-04-24T23:01:26.680477+0000","last_peered":"2024-04-24T23:01:26.680477+0000","last_clean":"2024-04-24T23:01:26.680477+0000","last_became_active":"2024-04-24T23:01:25.685668+0000","last_became_peered":"2024-04-24T23:01:25.685668+0000","last_unstale":"2024-04-24T23:01:26.680477+0000","last_undegraded":"2024-04-24T23:01:26.680477+0000","last_fullsized":"2024-04-24T23:01:26.680477+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:21.278605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.796755+0000","last_change":"2024-04-24T23:01:24.674942+0000","last_active":"2024-04-24T23:01:26.796755+0000","last_peered":"2024-04-24T23:01:26.796755+0000","last_clean":"2024-04-24T23:01:26.796755+0000","last_became_active":"2024-04-24T23:01:24.673969+0000","last_became_peered":"2024-04-24T23:01:24.673969+0000","last_unstale":"2024-04-24T23:01:26.796755+0000","last_undegraded":"2024-04-24T23:01:26.796755+0000","last_fullsized":"2024-04-24T23:01:26.796755+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:59.650806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795466+0000","last_change":"2024-04-24T23:01:25.684946+0000","last_active":"2024-04-24T23:01:26.795466+0000","last_peered":"2024-04-24T23:01:26.795466+0000","last_clean":"2024-04-24T23:01:26.795466+0000","last_became_active":"2024-04-24T23:01:25.683407+0000","last_became_peered":"2024-04-24T23:01:25.683407+0000","last_unstale":"2024-04-24T23:01:26.795466+0000","last_undegraded":"2024-04-24T23:01:26.795466+0000","last_fullsized":"2024-04-24T23:01:26.795466+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:20:19.277989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.677668+0000","last_change":"2024-04-24T23:01:24.669720+0000","last_active":"2024-04-24T23:01:26.677668+0000","last_peered":"2024-04-24T23:01:26.677668+0000","last_clean":"2024-04-24T23:01:26.677668+0000","last_became_active":"2024-04-24T23:01:24.669486+0000","last_became_peered":"2024-04-24T23:01:24.669486+0000","last_unstale":"2024-04-24T23:01:26.677668+0000","last_undegraded":"2024-04-24T23:01:26.677668+0000","last_fullsized":"2024-04-24T23:01:26.677668+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:35:01.170901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.680392+0000","last_change":"2024-04-24T23:01:25.687447+0000","last_active":"2024-04-24T23:01:26.680392+0000","last_peered":"2024-04-24T23:01:26.680392+0000","last_clean":"2024-04-24T23:01:26.680392+0000","last_became_active":"2024-04-24T23:01:25.687205+0000","last_became_peered":"2024-04-24T23:01:25.687205+0000","last_unstale":"2024-04-24T23:01:26.680392+0000","last_undegraded":"2024-04-24T23:01:26.680392+0000","last_fullsized":"2024-04-24T23:01:26.680392+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:40:09.564255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.677732+0000","last_change":"2024-04-24T23:01:24.671197+0000","last_active":"2024-04-24T23:01:26.677732+0000","last_peered":"2024-04-24T23:01:26.677732+0000","last_clean":"2024-04-24T23:01:26.677732+0000","last_became_active":"2024-04-24T23:01:24.670708+0000","last_became_peered":"2024-04-24T23:01:24.670708+0000","last_unstale":"2024-04-24T23:01:26.677732+0000","last_undegraded":"2024-04-24T23:01:26.677732+0000","last_fullsized":"2024-04-24T23:01:26.677732+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:08:34.350673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.677637+0000","last_change":"2024-04-24T23:01:25.687844+0000","last_active":"2024-04-24T23:01:26.677637+0000","last_peered":"2024-04-24T23:01:26.677637+0000","last_clean":"2024-04-24T23:01:26.677637+0000","last_became_active":"2024-04-24T23:01:25.687594+0000","last_became_peered":"2024-04-24T23:01:25.687594+0000","last_unstale":"2024-04-24T23:01:26.677637+0000","last_undegraded":"2024-04-24T23:01:26.677637+0000","last_fullsized":"2024-04-24T23:01:26.677637+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:58.803035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"66'8","reported_seq":22,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.677799+0000","last_change":"2024-04-24T23:01:24.668906+0000","last_active":"2024-04-24T23:01:26.677799+0000","last_peered":"2024-04-24T23:01:26.677799+0000","last_clean":"2024-04-24T23:01:26.677799+0000","last_became_active":"2024-04-24T23:01:24.668668+0000","last_became_peered":"2024-04-24T23:01:24.668668+0000","last_unstale":"2024-04-24T23:01:26.677799+0000","last_undegraded":"2024-04-24T23:01:26.677799+0000","last_fullsized":"2024-04-24T23:01:26.677799+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:35.586026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795589+0000","last_change":"2024-04-24T23:01:25.688984+0000","last_active":"2024-04-24T23:01:26.795589+0000","last_peered":"2024-04-24T23:01:26.795589+0000","last_clean":"2024-04-24T23:01:26.795589+0000","last_became_active":"2024-04-24T23:01:25.688729+0000","last_became_peered":"2024-04-24T23:01:25.688729+0000","last_unstale":"2024-04-24T23:01:26.795589+0000","last_undegraded":"2024-04-24T23:01:26.795589+0000","last_fullsized":"2024-04-24T23:01:26.795589+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:06:17.364816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795560+0000","last_change":"2024-04-24T23:01:24.670205+0000","last_active":"2024-04-24T23:01:26.795560+0000","last_peered":"2024-04-24T23:01:26.795560+0000","last_clean":"2024-04-24T23:01:26.795560+0000","last_became_active":"2024-04-24T23:01:24.669391+0000","last_became_peered":"2024-04-24T23:01:24.669391+0000","last_unstale":"2024-04-24T23:01:26.795560+0000","last_undegraded":"2024-04-24T23:01:26.795560+0000","last_fullsized":"2024-04-24T23:01:26.795560+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:13.191807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795452+0000","last_change":"2024-04-24T23:01:25.685965+0000","last_active":"2024-04-24T23:01:26.795452+0000","last_peered":"2024-04-24T23:01:26.795452+0000","last_clean":"2024-04-24T23:01:26.795452+0000","last_became_active":"2024-04-24T23:01:25.685385+0000","last_became_peered":"2024-04-24T23:01:25.685385+0000","last_unstale":"2024-04-24T23:01:26.795452+0000","last_undegraded":"2024-04-24T23:01:26.795452+0000","last_fullsized":"2024-04-24T23:01:26.795452+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:09:29.661588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.796862+0000","last_change":"2024-04-24T23:01:24.674710+0000","last_active":"2024-04-24T23:01:26.796862+0000","last_peered":"2024-04-24T23:01:26.796862+0000","last_clean":"2024-04-24T23:01:26.796862+0000","last_became_active":"2024-04-24T23:01:24.673627+0000","last_became_peered":"2024-04-24T23:01:24.673627+0000","last_unstale":"2024-04-24T23:01:26.796862+0000","last_undegraded":"2024-04-24T23:01:26.796862+0000","last_fullsized":"2024-04-24T23:01:26.796862+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:29:11.495801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.796640+0000","last_change":"2024-04-24T23:01:25.692604+0000","last_active":"2024-04-24T23:01:26.796640+0000","last_peered":"2024-04-24T23:01:26.796640+0000","last_clean":"2024-04-24T23:01:26.796640+0000","last_became_active":"2024-04-24T23:01:25.691592+0000","last_became_peered":"2024-04-24T23:01:25.691592+0000","last_unstale":"2024-04-24T23:01:26.796640+0000","last_undegraded":"2024-04-24T23:01:26.796640+0000","last_fullsized":"2024-04-24T23:01:26.796640+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:30:15.571691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795074+0000","last_change":"2024-04-24T23:01:24.671347+0000","last_active":"2024-04-24T23:01:26.795074+0000","last_peered":"2024-04-24T23:01:26.795074+0000","last_clean":"2024-04-24T23:01:26.795074+0000","last_became_active":"2024-04-24T23:01:24.671004+0000","last_became_peered":"2024-04-24T23:01:24.671004+0000","last_unstale":"2024-04-24T23:01:26.795074+0000","last_undegraded":"2024-04-24T23:01:26.795074+0000","last_fullsized":"2024-04-24T23:01:26.795074+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:18.314318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.677552+0000","last_change":"2024-04-24T23:01:25.681253+0000","last_active":"2024-04-24T23:01:26.677552+0000","last_peered":"2024-04-24T23:01:26.677552+0000","last_clean":"2024-04-24T23:01:26.677552+0000","last_became_active":"2024-04-24T23:01:25.680999+0000","last_became_peered":"2024-04-24T23:01:25.680999+0000","last_unstale":"2024-04-24T23:01:26.677552+0000","last_undegraded":"2024-04-24T23:01:26.677552+0000","last_fullsized":"2024-04-24T23:01:26.677552+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:22:32.016539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795889+0000","last_change":"2024-04-24T23:01:24.674321+0000","last_active":"2024-04-24T23:01:26.795889+0000","last_peered":"2024-04-24T23:01:26.795889+0000","last_clean":"2024-04-24T23:01:26.795889+0000","last_became_active":"2024-04-24T23:01:24.672742+0000","last_became_peered":"2024-04-24T23:01:24.672742+0000","last_unstale":"2024-04-24T23:01:26.795889+0000","last_undegraded":"2024-04-24T23:01:26.795889+0000","last_fullsized":"2024-04-24T23:01:26.795889+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:45:50.092244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.680738+0000","last_change":"2024-04-24T23:01:25.694131+0000","last_active":"2024-04-24T23:01:26.680738+0000","last_peered":"2024-04-24T23:01:26.680738+0000","last_clean":"2024-04-24T23:01:26.680738+0000","last_became_active":"2024-04-24T23:01:25.693263+0000","last_became_peered":"2024-04-24T23:01:25.693263+0000","last_unstale":"2024-04-24T23:01:26.680738+0000","last_undegraded":"2024-04-24T23:01:26.680738+0000","last_fullsized":"2024-04-24T23:01:26.680738+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:46:26.253107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.681013+0000","last_change":"2024-04-24T23:01:24.675932+0000","last_active":"2024-04-24T23:01:26.681013+0000","last_peered":"2024-04-24T23:01:26.681013+0000","last_clean":"2024-04-24T23:01:26.681013+0000","last_became_active":"2024-04-24T23:01:24.675703+0000","last_became_peered":"2024-04-24T23:01:24.675703+0000","last_unstale":"2024-04-24T23:01:26.681013+0000","last_undegraded":"2024-04-24T23:01:26.681013+0000","last_fullsized":"2024-04-24T23:01:26.681013+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:13:13.489036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795988+0000","last_change":"2024-04-24T23:01:25.689467+0000","last_active":"2024-04-24T23:01:26.795988+0000","last_peered":"2024-04-24T23:01:26.795988+0000","last_clean":"2024-04-24T23:01:26.795988+0000","last_became_active":"2024-04-24T23:01:25.688872+0000","last_became_peered":"2024-04-24T23:01:25.688872+0000","last_unstale":"2024-04-24T23:01:26.795988+0000","last_undegraded":"2024-04-24T23:01:26.795988+0000","last_fullsized":"2024-04-24T23:01:26.795988+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:48:57.787836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795953+0000","last_change":"2024-04-24T23:01:24.677214+0000","last_active":"2024-04-24T23:01:26.795953+0000","last_peered":"2024-04-24T23:01:26.795953+0000","last_clean":"2024-04-24T23:01:26.795953+0000","last_became_active":"2024-04-24T23:01:24.676980+0000","last_became_peered":"2024-04-24T23:01:24.676980+0000","last_unstale":"2024-04-24T23:01:26.795953+0000","last_undegraded":"2024-04-24T23:01:26.795953+0000","last_fullsized":"2024-04-24T23:01:26.795953+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:56:19.290947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.796643+0000","last_change":"2024-04-24T23:01:25.691570+0000","last_active":"2024-04-24T23:01:26.796643+0000","last_peered":"2024-04-24T23:01:26.796643+0000","last_clean":"2024-04-24T23:01:26.796643+0000","last_became_active":"2024-04-24T23:01:25.690694+0000","last_became_peered":"2024-04-24T23:01:25.690694+0000","last_unstale":"2024-04-24T23:01:26.796643+0000","last_undegraded":"2024-04-24T23:01:26.796643+0000","last_fullsized":"2024-04-24T23:01:26.796643+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:55.037551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.680657+0000","last_change":"2024-04-24T23:01:24.676113+0000","last_active":"2024-04-24T23:01:26.680657+0000","last_peered":"2024-04-24T23:01:26.680657+0000","last_clean":"2024-04-24T23:01:26.680657+0000","last_became_active":"2024-04-24T23:01:24.675810+0000","last_became_peered":"2024-04-24T23:01:24.675810+0000","last_unstale":"2024-04-24T23:01:26.680657+0000","last_undegraded":"2024-04-24T23:01:26.680657+0000","last_fullsized":"2024-04-24T23:01:26.680657+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:15:15.452325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795734+0000","last_change":"2024-04-24T23:01:25.686958+0000","last_active":"2024-04-24T23:01:26.795734+0000","last_peered":"2024-04-24T23:01:26.795734+0000","last_clean":"2024-04-24T23:01:26.795734+0000","last_became_active":"2024-04-24T23:01:25.686354+0000","last_became_peered":"2024-04-24T23:01:25.686354+0000","last_unstale":"2024-04-24T23:01:26.795734+0000","last_undegraded":"2024-04-24T23:01:26.795734+0000","last_fullsized":"2024-04-24T23:01:26.795734+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:34:14.547983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.796932+0000","last_change":"2024-04-24T23:01:24.675846+0000","last_active":"2024-04-24T23:01:26.796932+0000","last_peered":"2024-04-24T23:01:26.796932+0000","last_clean":"2024-04-24T23:01:26.796932+0000","last_became_active":"2024-04-24T23:01:24.675551+0000","last_became_peered":"2024-04-24T23:01:24.675551+0000","last_unstale":"2024-04-24T23:01:26.796932+0000","last_undegraded":"2024-04-24T23:01:26.796932+0000","last_fullsized":"2024-04-24T23:01:26.796932+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:50.040159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.794974+0000","last_change":"2024-04-24T23:01:25.680176+0000","last_active":"2024-04-24T23:01:26.794974+0000","last_peered":"2024-04-24T23:01:26.794974+0000","last_clean":"2024-04-24T23:01:26.794974+0000","last_became_active":"2024-04-24T23:01:25.679731+0000","last_became_peered":"2024-04-24T23:01:25.679731+0000","last_unstale":"2024-04-24T23:01:26.794974+0000","last_undegraded":"2024-04-24T23:01:26.794974+0000","last_fullsized":"2024-04-24T23:01:26.794974+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:20.438750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.796418+0000","last_change":"2024-04-24T23:01:24.678561+0000","last_active":"2024-04-24T23:01:26.796418+0000","last_peered":"2024-04-24T23:01:26.796418+0000","last_clean":"2024-04-24T23:01:26.796418+0000","last_became_active":"2024-04-24T23:01:24.677898+0000","last_became_peered":"2024-04-24T23:01:24.677898+0000","last_unstale":"2024-04-24T23:01:26.796418+0000","last_undegraded":"2024-04-24T23:01:26.796418+0000","last_fullsized":"2024-04-24T23:01:26.796418+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:08:45.956355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795981+0000","last_change":"2024-04-24T23:01:25.688840+0000","last_active":"2024-04-24T23:01:26.795981+0000","last_peered":"2024-04-24T23:01:26.795981+0000","last_clean":"2024-04-24T23:01:26.795981+0000","last_became_active":"2024-04-24T23:01:25.688608+0000","last_became_peered":"2024-04-24T23:01:25.688608+0000","last_unstale":"2024-04-24T23:01:26.795981+0000","last_undegraded":"2024-04-24T23:01:26.795981+0000","last_fullsized":"2024-04-24T23:01:26.795981+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:23.511995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.680827+0000","last_change":"2024-04-24T23:01:24.674057+0000","last_active":"2024-04-24T23:01:26.680827+0000","last_peered":"2024-04-24T23:01:26.680827+0000","last_clean":"2024-04-24T23:01:26.680827+0000","last_became_active":"2024-04-24T23:01:24.673808+0000","last_became_peered":"2024-04-24T23:01:24.673808+0000","last_unstale":"2024-04-24T23:01:26.680827+0000","last_undegraded":"2024-04-24T23:01:26.680827+0000","last_fullsized":"2024-04-24T23:01:26.680827+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:02:36.252012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795680+0000","last_change":"2024-04-24T23:01:25.693803+0000","last_active":"2024-04-24T23:01:26.795680+0000","last_peered":"2024-04-24T23:01:26.795680+0000","last_clean":"2024-04-24T23:01:26.795680+0000","last_became_active":"2024-04-24T23:01:25.692116+0000","last_became_peered":"2024-04-24T23:01:25.692116+0000","last_unstale":"2024-04-24T23:01:26.795680+0000","last_undegraded":"2024-04-24T23:01:26.795680+0000","last_fullsized":"2024-04-24T23:01:26.795680+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:24:02.265824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795698+0000","last_change":"2024-04-24T23:01:24.670600+0000","last_active":"2024-04-24T23:01:26.795698+0000","last_peered":"2024-04-24T23:01:26.795698+0000","last_clean":"2024-04-24T23:01:26.795698+0000","last_became_active":"2024-04-24T23:01:24.669627+0000","last_became_peered":"2024-04-24T23:01:24.669627+0000","last_unstale":"2024-04-24T23:01:26.795698+0000","last_undegraded":"2024-04-24T23:01:26.795698+0000","last_fullsized":"2024-04-24T23:01:26.795698+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:47:52.135891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795557+0000","last_change":"2024-04-24T23:01:25.695872+0000","last_active":"2024-04-24T23:01:26.795557+0000","last_peered":"2024-04-24T23:01:26.795557+0000","last_clean":"2024-04-24T23:01:26.795557+0000","last_became_active":"2024-04-24T23:01:25.695275+0000","last_became_peered":"2024-04-24T23:01:25.695275+0000","last_unstale":"2024-04-24T23:01:26.795557+0000","last_undegraded":"2024-04-24T23:01:26.795557+0000","last_fullsized":"2024-04-24T23:01:26.795557+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:30.122415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795594+0000","last_change":"2024-04-24T23:01:24.668280+0000","last_active":"2024-04-24T23:01:26.795594+0000","last_peered":"2024-04-24T23:01:26.795594+0000","last_clean":"2024-04-24T23:01:26.795594+0000","last_became_active":"2024-04-24T23:01:24.668049+0000","last_became_peered":"2024-04-24T23:01:24.668049+0000","last_unstale":"2024-04-24T23:01:26.795594+0000","last_undegraded":"2024-04-24T23:01:26.795594+0000","last_fullsized":"2024-04-24T23:01:26.795594+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:19.190128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795324+0000","last_change":"2024-04-24T23:01:25.692475+0000","last_active":"2024-04-24T23:01:26.795324+0000","last_peered":"2024-04-24T23:01:26.795324+0000","last_clean":"2024-04-24T23:01:26.795324+0000","last_became_active":"2024-04-24T23:01:25.692213+0000","last_became_peered":"2024-04-24T23:01:25.692213+0000","last_unstale":"2024-04-24T23:01:26.795324+0000","last_undegraded":"2024-04-24T23:01:26.795324+0000","last_fullsized":"2024-04-24T23:01:26.795324+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:26:03.587432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795258+0000","last_change":"2024-04-24T23:01:24.675164+0000","last_active":"2024-04-24T23:01:26.795258+0000","last_peered":"2024-04-24T23:01:26.795258+0000","last_clean":"2024-04-24T23:01:26.795258+0000","last_became_active":"2024-04-24T23:01:24.674910+0000","last_became_peered":"2024-04-24T23:01:24.674910+0000","last_unstale":"2024-04-24T23:01:26.795258+0000","last_undegraded":"2024-04-24T23:01:26.795258+0000","last_fullsized":"2024-04-24T23:01:26.795258+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:14:14.969110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795833+0000","last_change":"2024-04-24T23:01:25.694095+0000","last_active":"2024-04-24T23:01:26.795833+0000","last_peered":"2024-04-24T23:01:26.795833+0000","last_clean":"2024-04-24T23:01:26.795833+0000","last_became_active":"2024-04-24T23:01:25.693376+0000","last_became_peered":"2024-04-24T23:01:25.693376+0000","last_unstale":"2024-04-24T23:01:26.795833+0000","last_undegraded":"2024-04-24T23:01:26.795833+0000","last_fullsized":"2024-04-24T23:01:26.795833+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:03:31.570479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.796665+0000","last_change":"2024-04-24T23:01:24.678150+0000","last_active":"2024-04-24T23:01:26.796665+0000","last_peered":"2024-04-24T23:01:26.796665+0000","last_clean":"2024-04-24T23:01:26.796665+0000","last_became_active":"2024-04-24T23:01:24.677094+0000","last_became_peered":"2024-04-24T23:01:24.677094+0000","last_unstale":"2024-04-24T23:01:26.796665+0000","last_undegraded":"2024-04-24T23:01:26.796665+0000","last_fullsized":"2024-04-24T23:01:26.796665+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:19.392137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.796204+0000","last_change":"2024-04-24T23:01:25.685502+0000","last_active":"2024-04-24T23:01:26.796204+0000","last_peered":"2024-04-24T23:01:26.796204+0000","last_clean":"2024-04-24T23:01:26.796204+0000","last_became_active":"2024-04-24T23:01:25.685262+0000","last_became_peered":"2024-04-24T23:01:25.685262+0000","last_unstale":"2024-04-24T23:01:26.796204+0000","last_undegraded":"2024-04-24T23:01:26.796204+0000","last_fullsized":"2024-04-24T23:01:26.796204+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:30:40.588733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.796077+0000","last_change":"2024-04-24T23:01:24.668762+0000","last_active":"2024-04-24T23:01:26.796077+0000","last_peered":"2024-04-24T23:01:26.796077+0000","last_clean":"2024-04-24T23:01:26.796077+0000","last_became_active":"2024-04-24T23:01:24.668498+0000","last_became_peered":"2024-04-24T23:01:24.668498+0000","last_unstale":"2024-04-24T23:01:26.796077+0000","last_undegraded":"2024-04-24T23:01:26.796077+0000","last_fullsized":"2024-04-24T23:01:26.796077+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:30:33.504533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.796430+0000","last_change":"2024-04-24T23:01:25.685651+0000","last_active":"2024-04-24T23:01:26.796430+0000","last_peered":"2024-04-24T23:01:26.796430+0000","last_clean":"2024-04-24T23:01:26.796430+0000","last_became_active":"2024-04-24T23:01:25.685372+0000","last_became_peered":"2024-04-24T23:01:25.685372+0000","last_unstale":"2024-04-24T23:01:26.796430+0000","last_undegraded":"2024-04-24T23:01:26.796430+0000","last_fullsized":"2024-04-24T23:01:26.796430+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:24.657684+0000","last_clean_scrub_stamp":"2024-04-24T23:01:24.657684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:19:05.047211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.795480+0000","last_change":"2024-04-24T23:01:24.675175+0000","last_active":"2024-04-24T23:01:26.795480+0000","last_peered":"2024-04-24T23:01:26.795480+0000","last_clean":"2024-04-24T23:01:26.795480+0000","last_became_active":"2024-04-24T23:01:24.674880+0000","last_became_peered":"2024-04-24T23:01:24.674880+0000","last_unstale":"2024-04-24T23:01:26.795480+0000","last_undegraded":"2024-04-24T23:01:26.795480+0000","last_fullsized":"2024-04-24T23:01:26.795480+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:01:23.653113+0000","last_clean_scrub_stamp":"2024-04-24T23:01:23.653113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:17:35.597607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":257,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:01:26.680852+0000","last_change":"2024-04-24T22:58:55.138810+0000","last_active":"2024-04-24T23:01:26.680852+0000","last_peered":"2024-04-24T23:01:26.680852+0000","last_clean":"2024-04-24T23:01:26.680852+0000","last_became_active":"2024-04-24T22:58:55.138627+0000","last_became_peered":"2024-04-24T22:58:55.138627+0000","last_unstale":"2024-04-24T23:01:26.680852+0000","last_undegraded":"2024-04-24T23:01:26.680852+0000","last_fullsized":"2024-04-24T23:01:26.680852+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_clean_scrub_stamp":"2024-04-24T22:58:54.127235+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:06:31.042429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":16,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":331,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194980,"statfs":{"total":200000000,"available":199660170,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":31,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":331,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194980,"statfs":{"total":200000000,"available":199660240,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":914,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194398,"statfs":{"total":200000000,"available":199063560,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":332,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194980,"statfs":{"total":200000000,"available":199659646,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":338,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194974,"statfs":{"total":200000000,"available":199653872,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":333,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194978,"statfs":{"total":200000000,"available":199658490,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":915,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194397,"statfs":{"total":200000000,"available":199063012,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":339,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194972,"statfs":{"total":200000000,"available":199652350,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-24T23:01:29.053 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-24T23:01:29.053 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-24T23:01:29.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.197+0000 7fd6560da640 1 Processor -- start 2024-04-24T23:01:29.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.197+0000 7fd6560da640 1 -- start start 2024-04-24T23:01:29.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.198+0000 7fd6560da640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6500b5af0 0x7fd6500ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:29.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.198+0000 7fd6560da640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6500cd270 con 0x7fd650162e10 2024-04-24T23:01:29.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.198+0000 7fd6560da640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd650071610 con 0x7fd650162320 2024-04-24T23:01:29.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.198+0000 7fd6560da640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd6500b5660 con 0x7fd6500b5af0 2024-04-24T23:01:29.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.198+0000 7fd64ffff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd650162e10 0x7fd65015ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43092/0 (socket says 172.21.15.32:43092) 2024-04-24T23:01:29.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.198+0000 7fd64ffff640 1 -- 172.21.15.32:0/2870093940 learned_addr learned my addr 172.21.15.32:0/2870093940 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:01:29.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.198+0000 7fd64effd640 1 --2- 172.21.15.32:0/2870093940 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6500b5af0 0x7fd6500ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:29.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.198+0000 7fd64e7fc640 1 -- 172.21.15.32:0/2870093940 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3016553568 0 0) 0x7fd6500cd270 con 0x7fd650162e10 2024-04-24T23:01:29.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.198+0000 7fd64e7fc640 1 -- 172.21.15.32:0/2870093940 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd638003520 con 0x7fd650162e10 2024-04-24T23:01:29.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.198+0000 7fd64e7fc640 1 -- 172.21.15.32:0/2870093940 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2622169312 0 0) 0x7fd650071610 con 0x7fd650162320 2024-04-24T23:01:29.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.198+0000 7fd64e7fc640 1 -- 172.21.15.32:0/2870093940 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6500cd270 con 0x7fd650162320 2024-04-24T23:01:29.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.198+0000 7fd64e7fc640 1 -- 172.21.15.32:0/2870093940 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1280989188 0 0) 0x7fd638003520 con 0x7fd650162e10 2024-04-24T23:01:29.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.198+0000 7fd64e7fc640 1 -- 172.21.15.32:0/2870093940 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6500b5af0 msgr2=0x7fd6500ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:29.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.198+0000 7fd64e7fc640 1 --2- 172.21.15.32:0/2870093940 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6500b5af0 0x7fd6500ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:29.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.198+0000 7fd64e7fc640 1 -- 172.21.15.32:0/2870093940 >> v1:172.21.15.121:6789/0 conn(0x7fd650162320 legacy=0x7fd650162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:29.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.198+0000 7fd64e7fc640 1 -- 172.21.15.32:0/2870093940 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6500718f0 con 0x7fd650162e10 2024-04-24T23:01:29.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.198+0000 7fd64e7fc640 1 -- 172.21.15.32:0/2870093940 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7fd6480019f0 con 0x7fd650162e10 2024-04-24T23:01:29.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.198+0000 7fd64e7fc640 1 -- 172.21.15.32:0/2870093940 >> v1:172.21.15.32:6789/0 conn(0x7fd650162e10 legacy=0x7fd65015ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:29.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.198+0000 7fd64e7fc640 1 --2- 172.21.15.32:0/2870093940 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6500b5af0 0x7fd638003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:29.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.198+0000 7fd64e7fc640 1 --2- 172.21.15.32:0/2870093940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6380040d0 0x7fd638004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:29.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.198+0000 7fd64e7fc640 1 --2- 172.21.15.32:0/2870093940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd638004d40 0x7fd6380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:29.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.198+0000 7fd64e7fc640 1 -- 172.21.15.32:0/2870093940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd6500b5660 con 0x7fd6380040d0 2024-04-24T23:01:29.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.198+0000 7fd64e7fc640 1 -- 172.21.15.32:0/2870093940 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd63800bbd0 con 0x7fd638004d40 2024-04-24T23:01:29.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.198+0000 7fd64e7fc640 1 -- 172.21.15.32:0/2870093940 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd63800bdd0 con 0x7fd6500b5af0 2024-04-24T23:01:29.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.199+0000 7fd64effd640 1 --2- 172.21.15.32:0/2870093940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6380040d0 0x7fd638004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:29.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.199+0000 7fd64f7fe640 1 --2- 172.21.15.32:0/2870093940 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6500b5af0 0x7fd638003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:29.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.199+0000 7fd64ffff640 1 --2- 172.21.15.32:0/2870093940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd638004d40 0x7fd6380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:29.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.199+0000 7fd64effd640 1 -- 172.21.15.32:0/2870093940 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6500b5af0 msgr2=0x7fd638003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:29.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.199+0000 7fd64effd640 1 --2- 172.21.15.32:0/2870093940 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6500b5af0 0x7fd638003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:29.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.199+0000 7fd64effd640 1 -- 172.21.15.32:0/2870093940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd638004d40 msgr2=0x7fd6380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:29.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.199+0000 7fd64effd640 1 --2- 172.21.15.32:0/2870093940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd638004d40 0x7fd6380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:29.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.199+0000 7fd64effd640 1 -- 172.21.15.32:0/2870093940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd638003520 con 0x7fd6380040d0 2024-04-24T23:01:29.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.200+0000 7fd64effd640 1 --2- 172.21.15.32:0/2870093940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6380040d0 0x7fd638004800 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fd644009a90 tx=0x7fd638005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14d56fd3bdec264e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:29.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.200+0000 7fd64e7fc640 1 -- 172.21.15.32:0/2870093940 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd644036070 con 0x7fd6380040d0 2024-04-24T23:01:29.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.200+0000 7fd64e7fc640 1 -- 172.21.15.32:0/2870093940 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd644031400 con 0x7fd6380040d0 2024-04-24T23:01:29.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.200+0000 7fd64e7fc640 1 -- 172.21.15.32:0/2870093940 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd644009690 con 0x7fd6380040d0 2024-04-24T23:01:29.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.200+0000 7fd6560da640 1 -- 172.21.15.32:0/2870093940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6380040d0 msgr2=0x7fd638004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:29.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.200+0000 7fd6560da640 1 --2- 172.21.15.32:0/2870093940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6380040d0 0x7fd638004800 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fd644009a90 tx=0x7fd638005830 comp rx=0 tx=0).stop 2024-04-24T23:01:29.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.200+0000 7fd64ffff640 1 -- 172.21.15.32:0/2870093940 reap_dead start 2024-04-24T23:01:29.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.200+0000 7fd6560da640 1 -- 172.21.15.32:0/2870093940 shutdown_connections 2024-04-24T23:01:29.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.200+0000 7fd6560da640 1 -- 172.21.15.32:0/2870093940 >> 172.21.15.32:0/2870093940 conn(0x7fd6500d0ae0 msgr2=0x7fd6500d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:29.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.200+0000 7fd6560da640 1 -- 172.21.15.32:0/2870093940 shutdown_connections 2024-04-24T23:01:29.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.200+0000 7fd6560da640 1 -- 172.21.15.32:0/2870093940 wait complete. 2024-04-24T23:01:29.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.201+0000 7fd6560da640 1 Processor -- start 2024-04-24T23:01:29.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.202+0000 7fd6560da640 1 -- start start 2024-04-24T23:01:29.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.202+0000 7fd6560da640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6380040d0 0x7fd65006e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:29.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.202+0000 7fd6560da640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6501e8140 0x7fd6501e85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:29.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.202+0000 7fd6560da640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6500cd4f0 0x7fd6500cd950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:29.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.202+0000 7fd6560da640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd650071bd0 con 0x7fd6380040d0 2024-04-24T23:01:29.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.202+0000 7fd6560da640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd650067950 con 0x7fd6501e8140 2024-04-24T23:01:29.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.202+0000 7fd6560da640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd650071d50 con 0x7fd6500cd4f0 2024-04-24T23:01:29.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.202+0000 7fd64f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6380040d0 0x7fd65006e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:29.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.202+0000 7fd64f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6380040d0 0x7fd65006e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41904/0 (socket says 172.21.15.32:41904) 2024-04-24T23:01:29.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.202+0000 7fd64f7fe640 1 -- 172.21.15.32:0/2092669370 learned_addr learned my addr 172.21.15.32:0/2092669370 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:29.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.203+0000 7fd64ffff640 1 --2- 172.21.15.32:0/2092669370 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6500cd4f0 0x7fd6500cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:29.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.203+0000 7fd64effd640 1 --2- 172.21.15.32:0/2092669370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6501e8140 0x7fd6501e85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:29.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.203+0000 7fd64f7fe640 1 -- 172.21.15.32:0/2092669370 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6500cd4f0 msgr2=0x7fd6500cd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:29.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.203+0000 7fd64f7fe640 1 --2- 172.21.15.32:0/2092669370 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6500cd4f0 0x7fd6500cd950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:29.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.203+0000 7fd64f7fe640 1 -- 172.21.15.32:0/2092669370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6501e8140 msgr2=0x7fd6501e85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:29.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.203+0000 7fd64f7fe640 1 --2- 172.21.15.32:0/2092669370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6501e8140 0x7fd6501e85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:29.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.203+0000 7fd64f7fe640 1 -- 172.21.15.32:0/2092669370 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6501e8dc0 con 0x7fd6380040d0 2024-04-24T23:01:29.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.203+0000 7fd64f7fe640 1 --2- 172.21.15.32:0/2092669370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6380040d0 0x7fd65006e470 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fd64000f770 tx=0x7fd64000fc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:29.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.203+0000 7fd637fff640 1 -- 172.21.15.32:0/2092669370 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6400050d0 con 0x7fd6380040d0 2024-04-24T23:01:29.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.203+0000 7fd637fff640 1 -- 172.21.15.32:0/2092669370 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd640018070 con 0x7fd6380040d0 2024-04-24T23:01:29.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.203+0000 7fd6560da640 1 -- 172.21.15.32:0/2092669370 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6501e90a0 con 0x7fd6380040d0 2024-04-24T23:01:29.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.203+0000 7fd637fff640 1 -- 172.21.15.32:0/2092669370 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd64000d730 con 0x7fd6380040d0 2024-04-24T23:01:29.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.203+0000 7fd6560da640 1 -- 172.21.15.32:0/2092669370 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6501e9940 con 0x7fd6380040d0 2024-04-24T23:01:29.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.204+0000 7fd6560da640 1 -- 172.21.15.32:0/2092669370 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd650068430 con 0x7fd6380040d0 2024-04-24T23:01:29.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.204+0000 7fd637fff640 1 -- 172.21.15.32:0/2092669370 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd64000dae0 con 0x7fd6380040d0 2024-04-24T23:01:29.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.205+0000 7fd637fff640 1 --2- 172.21.15.32:0/2092669370 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd63c06be60 0x7fd63c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:29.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.205+0000 7fd637fff640 1 -- 172.21.15.32:0/2092669370 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7fd64008b000 con 0x7fd6380040d0 2024-04-24T23:01:29.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.205+0000 7fd64effd640 1 --2- 172.21.15.32:0/2092669370 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd63c06be60 0x7fd63c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:29.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.206+0000 7fd64effd640 1 --2- 172.21.15.32:0/2092669370 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd63c06be60 0x7fd63c06e320 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd644033000 tx=0x7fd644008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:29.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.209+0000 7fd637fff640 1 -- 172.21.15.32:0/2092669370 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd6400599b0 con 0x7fd6380040d0 2024-04-24T23:01:29.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.462+0000 7fd6560da640 1 -- 172.21.15.32:0/2092669370 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fd6500d0850 con 0x7fd6380040d0 2024-04-24T23:01:29.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.464+0000 7fd637fff640 1 -- 172.21.15.32:0/2092669370 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) ==== 190+0+147 (secure 0 0 0) 0x7fd6400592b0 con 0x7fd6380040d0 2024-04-24T23:01:29.465 INFO:teuthology.orchestra.run.smithi032.stdout:[client.0] 2024-04-24T23:01:29.465 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQCejilm5oeiGRAAdLXwwqrYhV3HHo7Dw/2ZqQ== 2024-04-24T23:01:29.465 INFO:teuthology.orchestra.run.smithi032.stdout: caps mds = "allow" 2024-04-24T23:01:29.465 INFO:teuthology.orchestra.run.smithi032.stdout: caps mon = "allow r" 2024-04-24T23:01:29.465 INFO:teuthology.orchestra.run.smithi032.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-24T23:01:29.466 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2024-04-24T23:01:29.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.466+0000 7fd6560da640 1 -- 172.21.15.32:0/2092669370 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd63c06be60 msgr2=0x7fd63c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:29.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.466+0000 7fd6560da640 1 --2- 172.21.15.32:0/2092669370 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd63c06be60 0x7fd63c06e320 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd644033000 tx=0x7fd644008030 comp rx=0 tx=0).stop 2024-04-24T23:01:29.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.466+0000 7fd6560da640 1 -- 172.21.15.32:0/2092669370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6380040d0 msgr2=0x7fd65006e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:29.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.466+0000 7fd6560da640 1 --2- 172.21.15.32:0/2092669370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6380040d0 0x7fd65006e470 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fd64000f770 tx=0x7fd64000fc30 comp rx=0 tx=0).stop 2024-04-24T23:01:29.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.466+0000 7fd6560da640 1 -- 172.21.15.32:0/2092669370 shutdown_connections 2024-04-24T23:01:29.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.466+0000 7fd6560da640 1 --2- 172.21.15.32:0/2092669370 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd63c06be60 0x7fd63c06e320 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:29.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.466+0000 7fd6560da640 1 --2- 172.21.15.32:0/2092669370 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6500cd4f0 0x7fd6500cd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:29.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.466+0000 7fd6560da640 1 --2- 172.21.15.32:0/2092669370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6501e8140 0x7fd6501e85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:29.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.466+0000 7fd6560da640 1 --2- 172.21.15.32:0/2092669370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6380040d0 0x7fd65006e470 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:29.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.466+0000 7fd6560da640 1 -- 172.21.15.32:0/2092669370 >> 172.21.15.32:0/2092669370 conn(0x7fd6500d0ae0 msgr2=0x7fd65015a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:29.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.466+0000 7fd6560da640 1 -- 172.21.15.32:0/2092669370 shutdown_connections 2024-04-24T23:01:29.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.466+0000 7fd6560da640 1 -- 172.21.15.32:0/2092669370 wait complete. 2024-04-24T23:01:29.479 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:01:29.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.578+0000 7f0b8feb3640 1 Processor -- start 2024-04-24T23:01:29.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.578+0000 7f0b8feb3640 1 -- start start 2024-04-24T23:01:29.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.579+0000 7f0b8feb3640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0b8815b480 0x7f0b880b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:29.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.579+0000 7f0b8feb3640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0b8815fd60 0x7f0b8806c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:29.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.579+0000 7f0b8feb3640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b880cef90 con 0x7f0b8806cb30 2024-04-24T23:01:29.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.579+0000 7f0b8feb3640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f0b880b4ff0 con 0x7f0b8815fd60 2024-04-24T23:01:29.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.579+0000 7f0b8feb3640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f0b880b46c0 con 0x7f0b8815b480 2024-04-24T23:01:29.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.579+0000 7f0b8e450640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0b8806cb30 0x7f0b880ccc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43094/0 (socket says 172.21.15.32:43094) 2024-04-24T23:01:29.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.579+0000 7f0b8e450640 1 -- 172.21.15.32:0/2059747443 learned_addr learned my addr 172.21.15.32:0/2059747443 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:01:29.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.579+0000 7f0b8d44e640 1 --2- 172.21.15.32:0/2059747443 >> v2:172.21.15.121:3300/0 conn(0x7f0b8815fd60 0x7f0b8806c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:29.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.579+0000 7f0b8dc4f640 1 --2- 172.21.15.32:0/2059747443 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0b8815b480 0x7f0b880b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:29.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.579+0000 7f0b8cc4d640 1 -- 172.21.15.32:0/2059747443 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2257833823 0 0) 0x7f0b880cef90 con 0x7f0b8806cb30 2024-04-24T23:01:29.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.580+0000 7f0b8cc4d640 1 -- 172.21.15.32:0/2059747443 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0b74003520 con 0x7f0b8806cb30 2024-04-24T23:01:29.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.580+0000 7f0b8dc4f640 1 -- 172.21.15.32:0/2059747443 >> v2:172.21.15.121:3300/0 conn(0x7f0b8815fd60 msgr2=0x7f0b8806c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:29.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.580+0000 7f0b8dc4f640 1 --2- 172.21.15.32:0/2059747443 >> v2:172.21.15.121:3300/0 conn(0x7f0b8815fd60 0x7f0b8806c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:29.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.580+0000 7f0b8dc4f640 1 -- 172.21.15.32:0/2059747443 >> v1:172.21.15.32:6789/0 conn(0x7f0b8806cb30 legacy=0x7f0b880ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:29.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.580+0000 7f0b8dc4f640 1 -- 172.21.15.32:0/2059747443 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b880b49a0 con 0x7f0b8815b480 2024-04-24T23:01:29.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.580+0000 7f0b8d44e640 1 --2- 172.21.15.32:0/2059747443 >> v2:172.21.15.121:3300/0 conn(0x7f0b8815fd60 0x7f0b8806c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:01:29.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.580+0000 7f0b8dc4f640 1 --2- 172.21.15.32:0/2059747443 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0b8815b480 0x7f0b880b5c60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0b800097a0 tx=0x7f0b8002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0ab08f7370a5284 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:29.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.581+0000 7f0b8cc4d640 1 -- 172.21.15.32:0/2059747443 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b80037070 con 0x7f0b8815b480 2024-04-24T23:01:29.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.581+0000 7f0b8cc4d640 1 -- 172.21.15.32:0/2059747443 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b80032440 con 0x7f0b8815b480 2024-04-24T23:01:29.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.581+0000 7f0b8cc4d640 1 -- 172.21.15.32:0/2059747443 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b80032810 con 0x7f0b8815b480 2024-04-24T23:01:29.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.581+0000 7f0b8feb3640 1 -- 172.21.15.32:0/2059747443 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0b8815b480 msgr2=0x7f0b880b5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:29.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.581+0000 7f0b8feb3640 1 --2- 172.21.15.32:0/2059747443 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0b8815b480 0x7f0b880b5c60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0b800097a0 tx=0x7f0b8002a180 comp rx=0 tx=0).stop 2024-04-24T23:01:29.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.581+0000 7f0b8feb3640 1 -- 172.21.15.32:0/2059747443 shutdown_connections 2024-04-24T23:01:29.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.581+0000 7f0b8feb3640 1 --2- 172.21.15.32:0/2059747443 >> v2:172.21.15.121:3300/0 conn(0x7f0b8815fd60 0x7f0b8806c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:29.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.581+0000 7f0b8feb3640 1 --2- 172.21.15.32:0/2059747443 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0b8815b480 0x7f0b880b5c60 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:29.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.581+0000 7f0b8feb3640 1 -- 172.21.15.32:0/2059747443 >> 172.21.15.32:0/2059747443 conn(0x7f0b8806b110 msgr2=0x7f0b8806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:29.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.581+0000 7f0b8feb3640 1 -- 172.21.15.32:0/2059747443 shutdown_connections 2024-04-24T23:01:29.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.581+0000 7f0b8feb3640 1 -- 172.21.15.32:0/2059747443 wait complete. 2024-04-24T23:01:29.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.582+0000 7f0b8feb3640 1 Processor -- start 2024-04-24T23:01:29.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.583+0000 7f0b8feb3640 1 -- start start 2024-04-24T23:01:29.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.583+0000 7f0b8feb3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b8806cb30 0x7f0b8814be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:29.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.583+0000 7f0b8feb3640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0b8815b480 0x7f0b88157610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:29.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.583+0000 7f0b8feb3640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b8815fd60 0x7f0b88157b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:29.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.583+0000 7f0b8feb3640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0b880b5480 con 0x7f0b8806cb30 2024-04-24T23:01:29.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.583+0000 7f0b8feb3640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0b880b61a0 con 0x7f0b8815fd60 2024-04-24T23:01:29.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.583+0000 7f0b8feb3640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f0b8815b860 con 0x7f0b8815b480 2024-04-24T23:01:29.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.583+0000 7f0b8dc4f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b8806cb30 0x7f0b8814be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:29.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.583+0000 7f0b8dc4f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b8806cb30 0x7f0b8814be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41916/0 (socket says 172.21.15.32:41916) 2024-04-24T23:01:29.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.583+0000 7f0b8dc4f640 1 -- 172.21.15.32:0/4239843552 learned_addr learned my addr 172.21.15.32:0/4239843552 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:29.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.584+0000 7f0b8dc4f640 1 -- 172.21.15.32:0/4239843552 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0b8815b480 msgr2=0x7f0b88157610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:29.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.584+0000 7f0b8dc4f640 1 --2- 172.21.15.32:0/4239843552 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0b8815b480 0x7f0b88157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:29.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.584+0000 7f0b8dc4f640 1 -- 172.21.15.32:0/4239843552 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b8815fd60 msgr2=0x7f0b88157b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:29.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.584+0000 7f0b8e450640 1 --2- 172.21.15.32:0/4239843552 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b8815fd60 0x7f0b88157b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:29.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.584+0000 7f0b8dc4f640 1 --2- 172.21.15.32:0/4239843552 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b8815fd60 0x7f0b88157b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:29.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.584+0000 7f0b8dc4f640 1 -- 172.21.15.32:0/4239843552 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b8815e270 con 0x7f0b8806cb30 2024-04-24T23:01:29.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.584+0000 7f0b8dc4f640 1 --2- 172.21.15.32:0/4239843552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b8806cb30 0x7f0b8814be40 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f0b80035040 tx=0x7f0b800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:29.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.584+0000 7f0b7e7fc640 1 -- 172.21.15.32:0/4239843552 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b80041030 con 0x7f0b8806cb30 2024-04-24T23:01:29.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.584+0000 7f0b7e7fc640 1 -- 172.21.15.32:0/4239843552 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b8003d4e0 con 0x7f0b8806cb30 2024-04-24T23:01:29.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.584+0000 7f0b8feb3640 1 -- 172.21.15.32:0/4239843552 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b88154280 con 0x7f0b8806cb30 2024-04-24T23:01:29.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.584+0000 7f0b7e7fc640 1 -- 172.21.15.32:0/4239843552 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b80037040 con 0x7f0b8806cb30 2024-04-24T23:01:29.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.584+0000 7f0b8feb3640 1 -- 172.21.15.32:0/4239843552 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b88154840 con 0x7f0b8806cb30 2024-04-24T23:01:29.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.585+0000 7f0b8feb3640 1 -- 172.21.15.32:0/4239843552 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b88068430 con 0x7f0b8806cb30 2024-04-24T23:01:29.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.586+0000 7f0b7e7fc640 1 -- 172.21.15.32:0/4239843552 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0b80044070 con 0x7f0b8806cb30 2024-04-24T23:01:29.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.586+0000 7f0b7e7fc640 1 --2- 172.21.15.32:0/4239843552 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0b6c06be30 0x7f0b6c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:29.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.586+0000 7f0b7e7fc640 1 -- 172.21.15.32:0/4239843552 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7f0b800ab4d0 con 0x7f0b8806cb30 2024-04-24T23:01:29.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.586+0000 7f0b8d44e640 1 --2- 172.21.15.32:0/4239843552 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0b6c06be30 0x7f0b6c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:29.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.587+0000 7f0b8d44e640 1 --2- 172.21.15.32:0/4239843552 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0b6c06be30 0x7f0b6c06e2f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0b78006e80 tx=0x7f0b78006dd0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:29.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.590+0000 7f0b7e7fc640 1 -- 172.21.15.32:0/4239843552 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0b80044e40 con 0x7f0b8806cb30 2024-04-24T23:01:29.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.814+0000 7f0b8feb3640 1 -- 172.21.15.32:0/4239843552 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b880afaf0 con 0x7f0b8806cb30 2024-04-24T23:01:29.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.814+0000 7f0b7e7fc640 1 -- 172.21.15.32:0/4239843552 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5187 (secure 0 0 0) 0x7f0b80079e90 con 0x7f0b8806cb30 2024-04-24T23:01:29.816 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:01:29.816 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":27,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3085809520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3085809520},{"type":"v1","addr":"172.21.15.121:6837","nonce":3085809520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":6078,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/1405465715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1405465715},{"type":"v1","addr":"172.21.15.32:6834","nonce":1405465715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":6081,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1160416671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":1160416671},{"type":"v1","addr":"172.21.15.32:6835","nonce":1160416671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:01:25.028304+0000","modified":"2024-04-24T23:01:27.048925+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6090},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6090":{"gid":6090,"name":"d","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6836/2650840833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2650840833},{"type":"v1","addr":"172.21.15.121:6836","nonce":2650840833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6090,"qdb_cluster":[6090]},"id":4}]} 2024-04-24T23:01:29.816 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 27 2024-04-24T23:01:29.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.816+0000 7f0b8feb3640 1 -- 172.21.15.32:0/4239843552 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0b6c06be30 msgr2=0x7f0b6c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:29.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.816+0000 7f0b8feb3640 1 --2- 172.21.15.32:0/4239843552 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0b6c06be30 0x7f0b6c06e2f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0b78006e80 tx=0x7f0b78006dd0 comp rx=0 tx=0).stop 2024-04-24T23:01:29.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.816+0000 7f0b8feb3640 1 -- 172.21.15.32:0/4239843552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b8806cb30 msgr2=0x7f0b8814be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:29.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.816+0000 7f0b8feb3640 1 --2- 172.21.15.32:0/4239843552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b8806cb30 0x7f0b8814be40 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f0b80035040 tx=0x7f0b800046c0 comp rx=0 tx=0).stop 2024-04-24T23:01:29.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.816+0000 7f0b8feb3640 1 -- 172.21.15.32:0/4239843552 shutdown_connections 2024-04-24T23:01:29.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.816+0000 7f0b8feb3640 1 --2- 172.21.15.32:0/4239843552 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0b6c06be30 0x7f0b6c06e2f0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:29.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.816+0000 7f0b8feb3640 1 --2- 172.21.15.32:0/4239843552 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b8815fd60 0x7f0b88157b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:29.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.816+0000 7f0b8feb3640 1 --2- 172.21.15.32:0/4239843552 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0b8815b480 0x7f0b88157610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:29.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.816+0000 7f0b8feb3640 1 --2- 172.21.15.32:0/4239843552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b8806cb30 0x7f0b8814be40 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:29.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.816+0000 7f0b8feb3640 1 -- 172.21.15.32:0/4239843552 >> 172.21.15.32:0/4239843552 conn(0x7f0b8806b110 msgr2=0x7f0b880af380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:29.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.816+0000 7f0b8feb3640 1 -- 172.21.15.32:0/4239843552 shutdown_connections 2024-04-24T23:01:29.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:29.816+0000 7f0b8feb3640 1 -- 172.21.15.32:0/4239843552 wait complete. 2024-04-24T23:01:30.231 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T23:01:25.028304+0000', 'modified': '2024-04-24T23:01:27.048925+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6090}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6090': {'gid': 6090, 'name': 'd', 'rank': 0, 'incarnation': 25, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.121:6836/2650840833', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6834', 'nonce': 2650840833}, {'type': 'v1', 'addr': '172.21.15.121:6836', 'nonce': 2650840833}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6090, 'qdb_cluster': [6090]} 2024-04-24T23:01:30.232 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-24T23:01:30.232 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-24T23:01:30.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.336+0000 7fec9b837640 1 Processor -- start 2024-04-24T23:01:30.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.337+0000 7fec9b837640 1 -- start start 2024-04-24T23:01:30.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.337+0000 7fec9b837640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec9415e9b0 0x7fec940d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:30.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.337+0000 7fec9b837640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fec940d4210 0x7fec9406b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:30.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.337+0000 7fec9b837640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec940d3170 con 0x7fec940ba0e0 2024-04-24T23:01:30.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.337+0000 7fec9b837640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fec940b8900 con 0x7fec940d4210 2024-04-24T23:01:30.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.337+0000 7fec9b837640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fec940d3450 con 0x7fec9415e9b0 2024-04-24T23:01:30.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.338+0000 7fec995d3640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec9415e9b0 0x7fec940d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:30.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.338+0000 7fec98dd2640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fec940d4210 0x7fec9406b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:30.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.338+0000 7fec995d3640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec9415e9b0 0x7fec940d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.32:40438/0 (socket says 172.21.15.32:40438) 2024-04-24T23:01:30.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.338+0000 7fec99dd4640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fec940ba0e0 0x7fec94071730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:49836/0 (socket says 172.21.15.32:49836) 2024-04-24T23:01:30.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.338+0000 7fec98dd2640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fec940d4210 0x7fec9406b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.32:57572/0 (socket says 172.21.15.32:57572) 2024-04-24T23:01:30.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.338+0000 7fec995d3640 1 -- 172.21.15.32:0/3993237274 learned_addr learned my addr 172.21.15.32:0/3993237274 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:30.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.338+0000 7fec995d3640 1 -- 172.21.15.32:0/3993237274 >> v2:172.21.15.121:3300/0 conn(0x7fec940d4210 msgr2=0x7fec9406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:30.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.338+0000 7fec995d3640 1 --2- 172.21.15.32:0/3993237274 >> v2:172.21.15.121:3300/0 conn(0x7fec940d4210 0x7fec9406b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:30.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.338+0000 7fec995d3640 1 -- 172.21.15.32:0/3993237274 >> v1:172.21.15.121:6789/0 conn(0x7fec940ba0e0 legacy=0x7fec94071730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:30.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.338+0000 7fec995d3640 1 -- 172.21.15.32:0/3993237274 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec940d3730 con 0x7fec9415e9b0 2024-04-24T23:01:30.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.338+0000 7fec98dd2640 1 --2- 172.21.15.32:0/3993237274 >> v2:172.21.15.121:3300/0 conn(0x7fec940d4210 0x7fec9406b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:01:30.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.339+0000 7fec995d3640 1 --2- 172.21.15.32:0/3993237274 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec9415e9b0 0x7fec940d2b90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fec90009870 tx=0x7fec9002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf911eb222ab08ca server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:30.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.339+0000 7fec8bfff640 1 -- 172.21.15.32:0/3993237274 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec90037070 con 0x7fec9415e9b0 2024-04-24T23:01:30.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.339+0000 7fec8bfff640 1 -- 172.21.15.32:0/3993237274 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec90032440 con 0x7fec9415e9b0 2024-04-24T23:01:30.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.339+0000 7fec8bfff640 1 -- 172.21.15.32:0/3993237274 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec90032810 con 0x7fec9415e9b0 2024-04-24T23:01:30.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.340+0000 7fec9b837640 1 -- 172.21.15.32:0/3993237274 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec9415e9b0 msgr2=0x7fec940d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:30.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.340+0000 7fec9b837640 1 --2- 172.21.15.32:0/3993237274 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec9415e9b0 0x7fec940d2b90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fec90009870 tx=0x7fec9002a270 comp rx=0 tx=0).stop 2024-04-24T23:01:30.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.340+0000 7fec9b837640 1 -- 172.21.15.32:0/3993237274 shutdown_connections 2024-04-24T23:01:30.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.340+0000 7fec9b837640 1 --2- 172.21.15.32:0/3993237274 >> v2:172.21.15.121:3300/0 conn(0x7fec940d4210 0x7fec9406b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:30.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.340+0000 7fec9b837640 1 --2- 172.21.15.32:0/3993237274 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec9415e9b0 0x7fec940d2b90 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:30.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.340+0000 7fec9b837640 1 -- 172.21.15.32:0/3993237274 >> 172.21.15.32:0/3993237274 conn(0x7fec940d11d0 msgr2=0x7fec940d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:30.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.340+0000 7fec9b837640 1 -- 172.21.15.32:0/3993237274 shutdown_connections 2024-04-24T23:01:30.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.340+0000 7fec9b837640 1 -- 172.21.15.32:0/3993237274 wait complete. 2024-04-24T23:01:30.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.341+0000 7fec9b837640 1 Processor -- start 2024-04-24T23:01:30.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.341+0000 7fec9b837640 1 -- start start 2024-04-24T23:01:30.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.342+0000 7fec9b837640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec940ba0e0 0x7fec941f3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:30.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.342+0000 7fec9b837640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec940d4210 0x7fec941521c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:30.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.342+0000 7fec9b837640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec9415e9b0 0x7fec940cc870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:30.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.342+0000 7fec9b837640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fec940d2400 con 0x7fec940ba0e0 2024-04-24T23:01:30.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.342+0000 7fec9b837640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fec940d3cf0 con 0x7fec9415e9b0 2024-04-24T23:01:30.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.342+0000 7fec9b837640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fec9406b650 con 0x7fec940d4210 2024-04-24T23:01:30.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.342+0000 7fec995d3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec940ba0e0 0x7fec941f3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:30.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.342+0000 7fec995d3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec940ba0e0 0x7fec941f3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57830/0 (socket says 172.21.15.32:57830) 2024-04-24T23:01:30.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.342+0000 7fec995d3640 1 -- 172.21.15.32:0/1573566339 learned_addr learned my addr 172.21.15.32:0/1573566339 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:30.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.342+0000 7fec98dd2640 1 --2- 172.21.15.32:0/1573566339 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec940d4210 0x7fec941521c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:30.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.342+0000 7fec99dd4640 1 --2- 172.21.15.32:0/1573566339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec9415e9b0 0x7fec940cc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:30.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.342+0000 7fec995d3640 1 -- 172.21.15.32:0/1573566339 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec940d4210 msgr2=0x7fec941521c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:30.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.342+0000 7fec995d3640 1 --2- 172.21.15.32:0/1573566339 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec940d4210 0x7fec941521c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:30.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.342+0000 7fec995d3640 1 -- 172.21.15.32:0/1573566339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec9415e9b0 msgr2=0x7fec940cc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:30.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.342+0000 7fec995d3640 1 --2- 172.21.15.32:0/1573566339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec9415e9b0 0x7fec940cc870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:30.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.342+0000 7fec995d3640 1 -- 172.21.15.32:0/1573566339 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec940cd2c0 con 0x7fec940ba0e0 2024-04-24T23:01:30.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.342+0000 7fec995d3640 1 --2- 172.21.15.32:0/1573566339 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec940ba0e0 0x7fec941f3c00 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fec900090b0 tx=0x7fec900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:30.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.343+0000 7fec89ffb640 1 -- 172.21.15.32:0/1573566339 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec90041030 con 0x7fec940ba0e0 2024-04-24T23:01:30.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.343+0000 7fec89ffb640 1 -- 172.21.15.32:0/1573566339 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec90030440 con 0x7fec940ba0e0 2024-04-24T23:01:30.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.343+0000 7fec9b837640 1 -- 172.21.15.32:0/1573566339 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec940cd5a0 con 0x7fec940ba0e0 2024-04-24T23:01:30.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.343+0000 7fec89ffb640 1 -- 172.21.15.32:0/1573566339 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec90037040 con 0x7fec940ba0e0 2024-04-24T23:01:30.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.343+0000 7fec9b837640 1 -- 172.21.15.32:0/1573566339 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec9415f210 con 0x7fec940ba0e0 2024-04-24T23:01:30.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.344+0000 7fec89ffb640 1 -- 172.21.15.32:0/1573566339 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fec90044070 con 0x7fec940ba0e0 2024-04-24T23:01:30.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.345+0000 7fec89ffb640 1 --2- 172.21.15.32:0/1573566339 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fec7806be30 0x7fec7806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:30.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.345+0000 7fec89ffb640 1 -- 172.21.15.32:0/1573566339 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7fec900ab110 con 0x7fec940ba0e0 2024-04-24T23:01:30.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.345+0000 7fec98dd2640 1 --2- 172.21.15.32:0/1573566339 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fec7806be30 0x7fec7806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:30.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.346+0000 7fec98dd2640 1 --2- 172.21.15.32:0/1573566339 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fec7806be30 0x7fec7806e2f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fec84009560 tx=0x7fec84009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:30.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.346+0000 7fec9e617740 1 Processor -- start 2024-04-24T23:01:30.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.346+0000 7fec9e617740 1 -- start start 2024-04-24T23:01:30.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.346+0000 7fec9e617740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55715930eb60 0x5571592c14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:30.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.346+0000 7fec9e617740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5571592f1c80 0x5571592f1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:30.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.346+0000 7fec9e617740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5571592f1540 0x5571592c6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:30.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.346+0000 7fec9e617740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x557159229f00 con 0x5571592f1540 2024-04-24T23:01:30.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.346+0000 7fec9e617740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5571591d2750 con 0x55715930eb60 2024-04-24T23:01:30.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.346+0000 7fec9e617740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x557159225ec0 con 0x5571592f1c80 2024-04-24T23:01:30.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.347+0000 7fec99dd4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5571592f1540 0x5571592c6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:30.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.347+0000 7fec99dd4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5571592f1540 0x5571592c6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57844/0 (socket says 172.21.15.32:57844) 2024-04-24T23:01:30.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.347+0000 7fec99dd4640 1 -- 172.21.15.32:0/1163507347 learned_addr learned my addr 172.21.15.32:0/1163507347 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:30.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.347+0000 7fec995d3640 1 --2- 172.21.15.32:0/1163507347 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55715930eb60 0x5571592c14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:30.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.347+0000 7fec98dd2640 1 --2- 172.21.15.32:0/1163507347 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5571592f1c80 0x5571592f1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:30.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.347+0000 7fec99dd4640 1 -- 172.21.15.32:0/1163507347 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5571592f1c80 msgr2=0x5571592f1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:30.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.347+0000 7fec99dd4640 1 --2- 172.21.15.32:0/1163507347 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5571592f1c80 0x5571592f1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:30.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.347+0000 7fec99dd4640 1 -- 172.21.15.32:0/1163507347 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55715930eb60 msgr2=0x5571592c14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:30.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.347+0000 7fec99dd4640 1 --2- 172.21.15.32:0/1163507347 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55715930eb60 0x5571592c14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:30.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.347+0000 7fec99dd4640 1 -- 172.21.15.32:0/1163507347 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557158f3e2d0 con 0x5571592f1540 2024-04-24T23:01:30.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.347+0000 7fec99dd4640 1 --2- 172.21.15.32:0/1163507347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5571592f1540 0x5571592c6350 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fec8c001b70 tx=0x7fec8c002030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ade5cd975ca4b3dc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:30.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.347+0000 7fec7effd640 1 -- 172.21.15.32:0/1163507347 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8c019070 con 0x5571592f1540 2024-04-24T23:01:30.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.347+0000 7fec7effd640 1 -- 172.21.15.32:0/1163507347 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec8c007910 con 0x5571592f1540 2024-04-24T23:01:30.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.347+0000 7fec7effd640 1 -- 172.21.15.32:0/1163507347 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8c014670 con 0x5571592f1540 2024-04-24T23:01:30.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.348+0000 7fec9e617740 1 -- 172.21.15.32:0/1163507347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5571592f1540 msgr2=0x5571592c6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:30.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.348+0000 7fec9e617740 1 --2- 172.21.15.32:0/1163507347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5571592f1540 0x5571592c6350 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fec8c001b70 tx=0x7fec8c002030 comp rx=0 tx=0).stop 2024-04-24T23:01:30.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.348+0000 7fec9e617740 1 -- 172.21.15.32:0/1163507347 shutdown_connections 2024-04-24T23:01:30.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.348+0000 7fec9e617740 1 --2- 172.21.15.32:0/1163507347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5571592f1540 0x5571592c6350 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:30.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.348+0000 7fec9e617740 1 --2- 172.21.15.32:0/1163507347 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5571592f1c80 0x5571592f1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:30.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.348+0000 7fec9e617740 1 --2- 172.21.15.32:0/1163507347 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55715930eb60 0x5571592c14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:30.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.348+0000 7fec9e617740 1 -- 172.21.15.32:0/1163507347 >> 172.21.15.32:0/1163507347 conn(0x5571592c3760 msgr2=0x55715930c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:30.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.348+0000 7fec9e617740 1 -- 172.21.15.32:0/1163507347 shutdown_connections 2024-04-24T23:01:30.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.348+0000 7fec9e617740 1 -- 172.21.15.32:0/1163507347 wait complete. 2024-04-24T23:01:30.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.348+0000 7fec9e617740 1 -- start start 2024-04-24T23:01:30.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.348+0000 7fec9e617740 1 Processor -- start 2024-04-24T23:01:30.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.349+0000 7fec9e617740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5571592f1c80 0x5571592e1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:30.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.349+0000 7fec9e617740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55715930eb60 0x5571592f47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:30.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.349+0000 7fec9e617740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5571592e1ca0 0x5571592f4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:30.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.349+0000 7fec9e617740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5571592c5910 con 0x5571592f1c80 2024-04-24T23:01:30.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.349+0000 7fec9e617740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5571591e5cf0 con 0x55715930eb60 2024-04-24T23:01:30.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.349+0000 7fec9e617740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x5571592c4d00 con 0x5571592e1ca0 2024-04-24T23:01:30.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.349+0000 7fec9e617740 10 client.0 _ll_register_callbacks cb 0x5571591e56e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:01:30.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.349+0000 7fec995d3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5571592f1c80 0x5571592e1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:30.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.349+0000 7fec995d3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5571592f1c80 0x5571592e1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57860/0 (socket says 172.21.15.32:57860) 2024-04-24T23:01:30.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.349+0000 7fec995d3640 1 -- 172.21.15.32:0/657379046 learned_addr learned my addr 172.21.15.32:0/657379046 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:30.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.350+0000 7fec98dd2640 1 --2- 172.21.15.32:0/657379046 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55715930eb60 0x5571592f47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:30.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.350+0000 7fec99dd4640 1 --2- 172.21.15.32:0/657379046 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5571592e1ca0 0x5571592f4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:30.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.350+0000 7fec995d3640 1 -- 172.21.15.32:0/657379046 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5571592e1ca0 msgr2=0x5571592f4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:30.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.350+0000 7fec995d3640 1 --2- 172.21.15.32:0/657379046 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5571592e1ca0 0x5571592f4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:30.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.350+0000 7fec995d3640 1 -- 172.21.15.32:0/657379046 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55715930eb60 msgr2=0x5571592f47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:30.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.350+0000 7fec995d3640 1 --2- 172.21.15.32:0/657379046 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55715930eb60 0x5571592f47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:30.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.350+0000 7fec995d3640 1 -- 172.21.15.32:0/657379046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571590086b0 con 0x5571592f1c80 2024-04-24T23:01:30.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.350+0000 7fec995d3640 1 --2- 172.21.15.32:0/657379046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5571592f1c80 0x5571592e1560 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fec9002a2b0 tx=0x7fec90079530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=440cf76d4e5d98e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:30.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.350+0000 7fec73fff640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-24T23:01:30.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.350+0000 7fec73fff640 1 -- 172.21.15.32:0/657379046 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec90091030 con 0x5571592f1c80 2024-04-24T23:01:30.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.350+0000 7fec73fff640 1 -- 172.21.15.32:0/657379046 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7fec90081b10 con 0x5571592f1c80 2024-04-24T23:01:30.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.350+0000 7fec9e617740 1 -- 172.21.15.32:0/657379046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec8c007910 con 0x5571592f1c80 2024-04-24T23:01:30.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.350+0000 7fec73fff640 1 -- 172.21.15.32:0/657379046 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fec900913d0 con 0x5571592f1c80 2024-04-24T23:01:30.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.350+0000 7fec9e617740 10 client.6219 fetch_fsmap learned FSMap version 27 2024-04-24T23:01:30.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.350+0000 7fec9e617740 1 -- 172.21.15.32:0/657379046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) -- 0x5571592f2330 con 0x5571592f1c80 2024-04-24T23:01:30.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.350+0000 7fec73fff640 1 -- 172.21.15.32:0/657379046 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec8c007910 con 0x5571592f1c80 2024-04-24T23:01:30.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.350+0000 7fec73fff640 1 -- 172.21.15.32:0/657379046 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 27) ==== 3494+0+0 (secure 0 0 0) 0x7fec9003a030 con 0x5571592f1c80 2024-04-24T23:01:30.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.350+0000 7fec9e617740 10 client.6219 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:01:30.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.350+0000 7fec9e617740 10 client.6219 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-24T23:01:30.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.351+0000 7fec9e617740 20 client.6219 populate_metadata read hostname 'smithi032' 2024-04-24T23:01:30.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.351+0000 7fec9e617740 1 --2- 172.21.15.32:0/657379046 >> [v2:172.21.15.121:6834/2650840833,v1:172.21.15.121:6836/2650840833] conn(0x5571592f29c0 0x5571592f2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:30.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.351+0000 7fec9e617740 4 client.6219 mds_command: new command op to 6090 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:01:30.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.351+0000 7fec9e617740 1 -- 172.21.15.32:0/657379046 --> [v2:172.21.15.121:6834/2650840833,v1:172.21.15.121:6836/2650840833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557159237440 con 0x5571592f29c0 2024-04-24T23:01:30.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.351+0000 7fec98dd2640 1 --2- 172.21.15.32:0/657379046 >> [v2:172.21.15.121:6834/2650840833,v1:172.21.15.121:6836/2650840833] conn(0x5571592f29c0 0x5571592f2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:30.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.351+0000 7fec73fff640 10 client.6219 ms_handle_connect on v2:172.21.15.121:6834/2650840833 2024-04-24T23:01:30.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.351+0000 7fec98dd2640 1 --2- 172.21.15.32:0/657379046 >> [v2:172.21.15.121:6834/2650840833,v1:172.21.15.121:6836/2650840833] conn(0x5571592f29c0 0x5571592f2dc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39cafa9992e9bb73 server_cookie=246a0a13c9cf8cc9 in_seq=0 out_seq=0 2024-04-24T23:01:30.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.352+0000 7fec73fff640 1 -- 172.21.15.32:0/657379046 <== mds.0 v2:172.21.15.121:6834/2650840833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557159237440 con 0x5571592f29c0 2024-04-24T23:01:30.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.352+0000 7fec73fff640 10 client.6219 handle_command_reply: tid=0 2024-04-24T23:01:30.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.352+0000 7fec9e617740 1 client.6219 shutdown 2024-04-24T23:01:30.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.352+0000 7fec9e617740 1 -- 172.21.15.32:0/657379046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5571592f1c80 msgr2=0x5571592e1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:30.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.352+0000 7fec9e617740 1 --2- 172.21.15.32:0/657379046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5571592f1c80 0x5571592e1560 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fec9002a2b0 tx=0x7fec90079530 comp rx=0 tx=0).stop 2024-04-24T23:01:30.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.352+0000 7fec9e617740 1 -- 172.21.15.32:0/657379046 shutdown_connections 2024-04-24T23:01:30.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.352+0000 7fec9e617740 1 --2- 172.21.15.32:0/657379046 >> [v2:172.21.15.121:6834/2650840833,v1:172.21.15.121:6836/2650840833] conn(0x5571592f29c0 0x5571592f2dc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:30.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.352+0000 7fec9e617740 1 --2- 172.21.15.32:0/657379046 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5571592e1ca0 0x5571592f4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:30.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.352+0000 7fec9e617740 1 --2- 172.21.15.32:0/657379046 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55715930eb60 0x5571592f47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:30.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.352+0000 7fec73fff640 0 client.6219 ms_handle_reset on v2:172.21.15.121:6834/2650840833 2024-04-24T23:01:30.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.352+0000 7fec9e617740 1 --2- 172.21.15.32:0/657379046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5571592f1c80 0x5571592e1560 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:30.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.352+0000 7fec9e617740 1 -- 172.21.15.32:0/657379046 >> 172.21.15.32:0/657379046 conn(0x5571592f1540 msgr2=0x5571592c0250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:30.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.353+0000 7fec9e617740 1 -- 172.21.15.32:0/657379046 shutdown_connections 2024-04-24T23:01:30.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.353+0000 7fec9e617740 1 -- 172.21.15.32:0/657379046 wait complete. 2024-04-24T23:01:30.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.353+0000 7fec9e617740 20 client.6219 trim_cache size 0 max 16384 2024-04-24T23:01:30.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.371+0000 7fec9e617740 1 Processor -- start 2024-04-24T23:01:30.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.371+0000 7fec9e617740 1 -- start start 2024-04-24T23:01:30.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.372+0000 7fec9e617740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5571592e1ca0 0x55715930b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:30.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.372+0000 7fec9e617740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571592f1c80 0x55715931b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:30.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.372+0000 7fec9e617740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55715930eb60 0x5571592f47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:30.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.372+0000 7fec9e617740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fec941617a0 con 0x5571592e1ca0 2024-04-24T23:01:30.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.372+0000 7fec9e617740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fec94160b00 con 0x5571592f1c80 2024-04-24T23:01:30.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.372+0000 7fec9e617740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fec94161150 con 0x55715930eb60 2024-04-24T23:01:30.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.372+0000 7fec995d3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5571592e1ca0 0x55715930b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:30.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.372+0000 7fec995d3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5571592e1ca0 0x55715930b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57876/0 (socket says 172.21.15.32:57876) 2024-04-24T23:01:30.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.372+0000 7fec995d3640 1 -- 172.21.15.32:0/2793962880 learned_addr learned my addr 172.21.15.32:0/2793962880 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:30.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.372+0000 7fec995d3640 1 -- 172.21.15.32:0/2793962880 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55715930eb60 msgr2=0x5571592f47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:30.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.372+0000 7fec995d3640 1 --2- 172.21.15.32:0/2793962880 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55715930eb60 0x5571592f47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:30.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.372+0000 7fec995d3640 1 -- 172.21.15.32:0/2793962880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571592f1c80 msgr2=0x55715931b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:30.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.372+0000 7fec995d3640 1 --2- 172.21.15.32:0/2793962880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571592f1c80 0x55715931b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:30.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.372+0000 7fec995d3640 1 -- 172.21.15.32:0/2793962880 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571592bc830 con 0x5571592e1ca0 2024-04-24T23:01:30.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.372+0000 7fec995d3640 1 --2- 172.21.15.32:0/2793962880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5571592e1ca0 0x55715930b3b0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fec90079400 tx=0x7fec90048000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74564fc0cbc4905e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:30.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.372+0000 7fec7e7fc640 1 -- 172.21.15.32:0/2793962880 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec90091040 con 0x5571592e1ca0 2024-04-24T23:01:30.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.372+0000 7fec7e7fc640 1 -- 172.21.15.32:0/2793962880 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec900a9580 con 0x5571592e1ca0 2024-04-24T23:01:30.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.372+0000 7fec7e7fc640 1 -- 172.21.15.32:0/2793962880 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec900911e0 con 0x5571592e1ca0 2024-04-24T23:01:30.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.373+0000 7fec9e617740 1 -- 172.21.15.32:0/2793962880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5571592e1ca0 msgr2=0x55715930b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:30.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.373+0000 7fec9e617740 1 --2- 172.21.15.32:0/2793962880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5571592e1ca0 0x55715930b3b0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fec90079400 tx=0x7fec90048000 comp rx=0 tx=0).stop 2024-04-24T23:01:30.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.373+0000 7fec9e617740 1 -- 172.21.15.32:0/2793962880 shutdown_connections 2024-04-24T23:01:30.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.373+0000 7fec9e617740 1 --2- 172.21.15.32:0/2793962880 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55715930eb60 0x5571592f47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:30.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.373+0000 7fec9e617740 1 --2- 172.21.15.32:0/2793962880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571592f1c80 0x55715931b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:30.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.373+0000 7fec9e617740 1 --2- 172.21.15.32:0/2793962880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5571592e1ca0 0x55715930b3b0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:30.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.373+0000 7fec9e617740 1 -- 172.21.15.32:0/2793962880 >> 172.21.15.32:0/2793962880 conn(0x5571592f1540 msgr2=0x55715930c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:30.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.373+0000 7fec9e617740 1 -- 172.21.15.32:0/2793962880 shutdown_connections 2024-04-24T23:01:30.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.373+0000 7fec9e617740 1 -- 172.21.15.32:0/2793962880 wait complete. 2024-04-24T23:01:30.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.373+0000 7fec9e617740 1 -- start start 2024-04-24T23:01:30.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.373+0000 7fec9e617740 1 Processor -- start 2024-04-24T23:01:30.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.374+0000 7fec9e617740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5571592e1ca0 0x557159385b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:30.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.374+0000 7fec9e617740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571592f1c80 0x5571593895d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:30.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.374+0000 7fec9e617740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55715930eb60 0x55715930df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:30.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.374+0000 7fec9e617740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5571592f38d0 con 0x5571592e1ca0 2024-04-24T23:01:30.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.374+0000 7fec9e617740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5571592c5910 con 0x5571592f1c80 2024-04-24T23:01:30.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.374+0000 7fec9e617740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x55715931fa80 con 0x55715930eb60 2024-04-24T23:01:30.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.374+0000 7fec9e617740 10 client.0 _ll_register_callbacks cb 0x7fec90079040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:01:30.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.374+0000 7fec995d3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5571592e1ca0 0x557159385b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:30.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.374+0000 7fec995d3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5571592e1ca0 0x557159385b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57886/0 (socket says 172.21.15.32:57886) 2024-04-24T23:01:30.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.374+0000 7fec995d3640 1 -- 172.21.15.32:0/3933555280 learned_addr learned my addr 172.21.15.32:0/3933555280 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:30.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.375+0000 7fec99dd4640 1 --2- 172.21.15.32:0/3933555280 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55715930eb60 0x55715930df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:30.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.375+0000 7fec98dd2640 1 --2- 172.21.15.32:0/3933555280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571592f1c80 0x5571593895d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:30.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.375+0000 7fec995d3640 1 -- 172.21.15.32:0/3933555280 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55715930eb60 msgr2=0x55715930df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:30.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.375+0000 7fec995d3640 1 --2- 172.21.15.32:0/3933555280 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55715930eb60 0x55715930df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:30.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.375+0000 7fec995d3640 1 -- 172.21.15.32:0/3933555280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571592f1c80 msgr2=0x5571593895d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:30.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.375+0000 7fec995d3640 1 --2- 172.21.15.32:0/3933555280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571592f1c80 0x5571593895d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:30.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.375+0000 7fec995d3640 1 -- 172.21.15.32:0/3933555280 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55715932fe80 con 0x5571592e1ca0 2024-04-24T23:01:30.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.375+0000 7fec98dd2640 1 --2- 172.21.15.32:0/3933555280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571592f1c80 0x5571593895d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:01:30.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.375+0000 7fec995d3640 1 --2- 172.21.15.32:0/3933555280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5571592e1ca0 0x557159385b70 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fec90078f40 tx=0x7fec900a9510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc4cac8a4a77dff0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:30.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.375+0000 7fec73fff640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-24T23:01:30.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.375+0000 7fec73fff640 1 -- 172.21.15.32:0/3933555280 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec90095070 con 0x5571592e1ca0 2024-04-24T23:01:30.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.375+0000 7fec73fff640 1 -- 172.21.15.32:0/3933555280 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7fec900788c0 con 0x5571592e1ca0 2024-04-24T23:01:30.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.375+0000 7fec9e617740 1 -- 172.21.15.32:0/3933555280 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec900a9580 con 0x5571592e1ca0 2024-04-24T23:01:30.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.375+0000 7fec73fff640 1 -- 172.21.15.32:0/3933555280 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fec90078ac0 con 0x5571592e1ca0 2024-04-24T23:01:30.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.375+0000 7fec9e617740 10 client.6225 fetch_fsmap learned FSMap version 27 2024-04-24T23:01:30.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.375+0000 7fec9e617740 1 -- 172.21.15.32:0/3933555280 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) -- 0x55715932fe80 con 0x5571592e1ca0 2024-04-24T23:01:30.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.375+0000 7fec73fff640 1 -- 172.21.15.32:0/3933555280 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec900a9580 con 0x5571592e1ca0 2024-04-24T23:01:30.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.375+0000 7fec73fff640 1 -- 172.21.15.32:0/3933555280 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 27) ==== 3494+0+0 (secure 0 0 0) 0x7fec900751c0 con 0x5571592e1ca0 2024-04-24T23:01:30.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.375+0000 7fec9e617740 10 client.6225 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:01:30.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.375+0000 7fec9e617740 10 client.6225 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-24T23:01:30.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.375+0000 7fec9e617740 20 client.6225 populate_metadata read hostname 'smithi032' 2024-04-24T23:01:30.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.375+0000 7fec9e617740 1 --2- 172.21.15.32:0/3933555280 >> [v2:172.21.15.121:6834/2650840833,v1:172.21.15.121:6836/2650840833] conn(0x55715931e0c0 0x55715931e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:30.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.376+0000 7fec9e617740 4 client.6225 mds_command: new command op to 6090 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:01:30.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.376+0000 7fec9e617740 1 -- 172.21.15.32:0/3933555280 --> [v2:172.21.15.121:6834/2650840833,v1:172.21.15.121:6836/2650840833] -- command(tid 0: {"prefix": "status"}) -- 0x557159231790 con 0x55715931e0c0 2024-04-24T23:01:30.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.376+0000 7fec98dd2640 1 --2- 172.21.15.32:0/3933555280 >> [v2:172.21.15.121:6834/2650840833,v1:172.21.15.121:6836/2650840833] conn(0x55715931e0c0 0x55715931e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:30.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.376+0000 7fec98dd2640 1 --2- 172.21.15.32:0/3933555280 >> [v2:172.21.15.121:6834/2650840833,v1:172.21.15.121:6836/2650840833] conn(0x55715931e0c0 0x55715931e4a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d8ae5249d69923f server_cookie=1cd399b59cd2efd5 in_seq=0 out_seq=0 2024-04-24T23:01:30.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.376+0000 7fec73fff640 10 client.6225 ms_handle_connect on v2:172.21.15.121:6834/2650840833 2024-04-24T23:01:30.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.376+0000 7fec73fff640 1 -- 172.21.15.32:0/3933555280 <== mds.0 v2:172.21.15.121:6834/2650840833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x557159231790 con 0x55715931e0c0 2024-04-24T23:01:30.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.376+0000 7fec73fff640 10 client.6225 handle_command_reply: tid=0 2024-04-24T23:01:30.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.376+0000 7fec9e617740 1 client.6225 shutdown 2024-04-24T23:01:30.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.376+0000 7fec9e617740 1 -- 172.21.15.32:0/3933555280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5571592e1ca0 msgr2=0x557159385b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:30.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.376+0000 7fec9e617740 1 --2- 172.21.15.32:0/3933555280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5571592e1ca0 0x557159385b70 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fec90078f40 tx=0x7fec900a9510 comp rx=0 tx=0).stop 2024-04-24T23:01:30.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.376+0000 7fec9e617740 1 -- 172.21.15.32:0/3933555280 shutdown_connections 2024-04-24T23:01:30.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.376+0000 7fec9e617740 1 --2- 172.21.15.32:0/3933555280 >> [v2:172.21.15.121:6834/2650840833,v1:172.21.15.121:6836/2650840833] conn(0x55715931e0c0 0x55715931e4a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:30.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.376+0000 7fec9e617740 1 --2- 172.21.15.32:0/3933555280 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55715930eb60 0x55715930df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:30.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.376+0000 7fec73fff640 0 client.6225 ms_handle_reset on v2:172.21.15.121:6834/2650840833 2024-04-24T23:01:30.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.376+0000 7fec9e617740 1 --2- 172.21.15.32:0/3933555280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571592f1c80 0x5571593895d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:30.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.376+0000 7fec9e617740 1 --2- 172.21.15.32:0/3933555280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5571592e1ca0 0x557159385b70 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:30.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.376+0000 7fec9e617740 1 -- 172.21.15.32:0/3933555280 >> 172.21.15.32:0/3933555280 conn(0x5571592f1540 msgr2=0x5571592f4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:30.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.377+0000 7fec9e617740 1 -- 172.21.15.32:0/3933555280 shutdown_connections 2024-04-24T23:01:30.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.377+0000 7fec9e617740 1 -- 172.21.15.32:0/3933555280 wait complete. 2024-04-24T23:01:30.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.377+0000 7fec9e617740 20 client.6225 trim_cache size 0 max 16384 2024-04-24T23:01:30.384 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:01:30.384 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", 2024-04-24T23:01:30.384 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2024-04-24T23:01:30.384 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 6090, 2024-04-24T23:01:30.384 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-04-24T23:01:30.384 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-04-24T23:01:30.384 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-04-24T23:01:30.384 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 4.6918893800000001, 2024-04-24T23:01:30.384 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 27, 2024-04-24T23:01:30.385 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 67, 2024-04-24T23:01:30.385 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 67, 2024-04-24T23:01:30.385 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 9.5880760469999995 2024-04-24T23:01:30.385 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:01:30.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.377+0000 7fec9b837640 1 -- 172.21.15.32:0/1573566339 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fec7806be30 msgr2=0x7fec7806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:30.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.377+0000 7fec9b837640 1 --2- 172.21.15.32:0/1573566339 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fec7806be30 0x7fec7806e2f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fec84009560 tx=0x7fec84009190 comp rx=0 tx=0).stop 2024-04-24T23:01:30.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.377+0000 7fec9b837640 1 -- 172.21.15.32:0/1573566339 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec940ba0e0 msgr2=0x7fec941f3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:30.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.377+0000 7fec9b837640 1 --2- 172.21.15.32:0/1573566339 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec940ba0e0 0x7fec941f3c00 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fec900090b0 tx=0x7fec900046c0 comp rx=0 tx=0).stop 2024-04-24T23:01:30.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.377+0000 7fec9b837640 1 -- 172.21.15.32:0/1573566339 shutdown_connections 2024-04-24T23:01:30.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.377+0000 7fec9b837640 1 --2- 172.21.15.32:0/1573566339 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fec7806be30 0x7fec7806e2f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:30.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.377+0000 7fec9b837640 1 --2- 172.21.15.32:0/1573566339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec9415e9b0 0x7fec940cc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:30.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.377+0000 7fec9b837640 1 --2- 172.21.15.32:0/1573566339 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec940d4210 0x7fec941521c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:30.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.377+0000 7fec9b837640 1 --2- 172.21.15.32:0/1573566339 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec940ba0e0 0x7fec941f3c00 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:30.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.377+0000 7fec9b837640 1 -- 172.21.15.32:0/1573566339 >> 172.21.15.32:0/1573566339 conn(0x7fec940d11d0 msgr2=0x7fec9415be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:30.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.377+0000 7fec9b837640 1 -- 172.21.15.32:0/1573566339 shutdown_connections 2024-04-24T23:01:30.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.377+0000 7fec9b837640 1 -- 172.21.15.32:0/1573566339 wait complete. 2024-04-24T23:01:30.389 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T23:01:30.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.490+0000 7f864294f640 1 Processor -- start 2024-04-24T23:01:30.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.490+0000 7f864294f640 1 -- start start 2024-04-24T23:01:30.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.491+0000 7f864294f640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f863c144040 0x7f863c0b4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:30.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.491+0000 7f864294f640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f863c071590 0x7f863c071a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:30.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.491+0000 7f864294f640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f863c144420 con 0x7f863c144650 2024-04-24T23:01:30.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.491+0000 7f864294f640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f863c071100 con 0x7f863c144040 2024-04-24T23:01:30.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.491+0000 7f864294f640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f863c06d750 con 0x7f863c071590 2024-04-24T23:01:30.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.491+0000 7f863b7fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f863c144650 0x7f863c0b5060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57264/0 (socket says 172.21.15.32:57264) 2024-04-24T23:01:30.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.491+0000 7f863b7fe640 1 -- 172.21.15.32:0/2064237128 learned_addr learned my addr 172.21.15.32:0/2064237128 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:01:30.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.491+0000 7f8640eec640 1 --2- 172.21.15.32:0/2064237128 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f863c071590 0x7f863c071a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:30.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.491+0000 7f863affd640 1 -- 172.21.15.32:0/2064237128 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4171777349 0 0) 0x7f863c144420 con 0x7f863c144650 2024-04-24T23:01:30.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.491+0000 7f863bfff640 1 --2- 172.21.15.32:0/2064237128 >> v2:172.21.15.121:3300/0 conn(0x7f863c144040 0x7f863c0b4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:30.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.491+0000 7f863affd640 1 -- 172.21.15.32:0/2064237128 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8624003520 con 0x7f863c144650 2024-04-24T23:01:30.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.491+0000 7f863affd640 1 -- 172.21.15.32:0/2064237128 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2628055652 0 0) 0x7f8624003520 con 0x7f863c144650 2024-04-24T23:01:30.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.491+0000 7f863affd640 1 -- 172.21.15.32:0/2064237128 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f863c071590 msgr2=0x7f863c071a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:30.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.491+0000 7f863affd640 1 --2- 172.21.15.32:0/2064237128 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f863c071590 0x7f863c071a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:30.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.491+0000 7f863affd640 1 -- 172.21.15.32:0/2064237128 >> v2:172.21.15.121:3300/0 conn(0x7f863c144040 msgr2=0x7f863c0b4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:30.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.491+0000 7f863affd640 1 --2- 172.21.15.32:0/2064237128 >> v2:172.21.15.121:3300/0 conn(0x7f863c144040 0x7f863c0b4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:30.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.491+0000 7f863affd640 1 -- 172.21.15.32:0/2064237128 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f863c06da30 con 0x7f863c144650 2024-04-24T23:01:30.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.491+0000 7f863bfff640 1 --2- 172.21.15.32:0/2064237128 >> v2:172.21.15.121:3300/0 conn(0x7f863c144040 0x7f863c0b4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:01:30.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.491+0000 7f863affd640 1 -- 172.21.15.32:0/2064237128 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f862c001670 con 0x7f863c144650 2024-04-24T23:01:30.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.492+0000 7f863affd640 1 -- 172.21.15.32:0/2064237128 >> v1:172.21.15.32:6789/0 conn(0x7f863c144650 legacy=0x7f863c0b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:30.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.492+0000 7f863affd640 1 --2- 172.21.15.32:0/2064237128 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86240039b0 0x7f8624003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:30.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.492+0000 7f863affd640 1 --2- 172.21.15.32:0/2064237128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8624004800 0x7f8624004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:30.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.492+0000 7f863affd640 1 --2- 172.21.15.32:0/2064237128 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f863c071590 0x7f86240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:30.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.492+0000 7f863affd640 1 -- 172.21.15.32:0/2064237128 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f863c071100 con 0x7f8624004800 2024-04-24T23:01:30.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.492+0000 7f863affd640 1 -- 172.21.15.32:0/2064237128 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f863c06d750 con 0x7f86240039b0 2024-04-24T23:01:30.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.492+0000 7f863affd640 1 -- 172.21.15.32:0/2064237128 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f862400b9c0 con 0x7f863c071590 2024-04-24T23:01:30.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.492+0000 7f863b7fe640 1 --2- 172.21.15.32:0/2064237128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8624004800 0x7f8624004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:30.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.492+0000 7f863bfff640 1 --2- 172.21.15.32:0/2064237128 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86240039b0 0x7f8624003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:30.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.492+0000 7f8640eec640 1 --2- 172.21.15.32:0/2064237128 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f863c071590 0x7f86240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:30.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.492+0000 7f863b7fe640 1 -- 172.21.15.32:0/2064237128 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f863c071590 msgr2=0x7f86240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:30.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.492+0000 7f863b7fe640 1 --2- 172.21.15.32:0/2064237128 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f863c071590 0x7f86240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:30.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.492+0000 7f863b7fe640 1 -- 172.21.15.32:0/2064237128 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86240039b0 msgr2=0x7f8624003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:30.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.492+0000 7f863b7fe640 1 --2- 172.21.15.32:0/2064237128 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86240039b0 0x7f8624003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:30.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.492+0000 7f863b7fe640 1 -- 172.21.15.32:0/2064237128 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8624003520 con 0x7f8624004800 2024-04-24T23:01:30.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.493+0000 7f863b7fe640 1 --2- 172.21.15.32:0/2064237128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8624004800 0x7f8624004c80 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f862c00c210 tx=0x7f8624005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eccf45bf049eab41 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:30.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.493+0000 7f863affd640 1 -- 172.21.15.32:0/2064237128 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862c007d70 con 0x7f8624004800 2024-04-24T23:01:30.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.493+0000 7f863affd640 1 -- 172.21.15.32:0/2064237128 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f862c005af0 con 0x7f8624004800 2024-04-24T23:01:30.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.493+0000 7f863affd640 1 -- 172.21.15.32:0/2064237128 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862c007070 con 0x7f8624004800 2024-04-24T23:01:30.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.493+0000 7f864294f640 1 -- 172.21.15.32:0/2064237128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8624004800 msgr2=0x7f8624004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:30.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.493+0000 7f864294f640 1 --2- 172.21.15.32:0/2064237128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8624004800 0x7f8624004c80 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f862c00c210 tx=0x7f8624005830 comp rx=0 tx=0).stop 2024-04-24T23:01:30.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.493+0000 7f8640eec640 1 -- 172.21.15.32:0/2064237128 reap_dead start 2024-04-24T23:01:30.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.493+0000 7f864294f640 1 -- 172.21.15.32:0/2064237128 shutdown_connections 2024-04-24T23:01:30.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.493+0000 7f864294f640 1 -- 172.21.15.32:0/2064237128 >> 172.21.15.32:0/2064237128 conn(0x7f863c0d0ae0 msgr2=0x7f863c0d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:30.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.493+0000 7f864294f640 1 -- 172.21.15.32:0/2064237128 shutdown_connections 2024-04-24T23:01:30.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.493+0000 7f864294f640 1 -- 172.21.15.32:0/2064237128 wait complete. 2024-04-24T23:01:30.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.495+0000 7f864294f640 1 Processor -- start 2024-04-24T23:01:30.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.495+0000 7f864294f640 1 -- start start 2024-04-24T23:01:30.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.495+0000 7f864294f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8624004800 0x7f863c168b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:30.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.495+0000 7f864294f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f863c0cea80 0x7f863c0ccf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:30.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.495+0000 7f864294f640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f863c0cd480 0x7f863c15cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:30.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.495+0000 7f864294f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f863c06f530 con 0x7f8624004800 2024-04-24T23:01:30.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.495+0000 7f864294f640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f863c06dd10 con 0x7f863c0cea80 2024-04-24T23:01:30.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.496+0000 7f864294f640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f863c0b5770 con 0x7f863c0cd480 2024-04-24T23:01:30.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.496+0000 7f863bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8624004800 0x7f863c168b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:30.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.496+0000 7f863bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8624004800 0x7f863c168b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57896/0 (socket says 172.21.15.32:57896) 2024-04-24T23:01:30.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.496+0000 7f863bfff640 1 -- 172.21.15.32:0/2029291077 learned_addr learned my addr 172.21.15.32:0/2029291077 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:30.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.496+0000 7f863b7fe640 1 --2- 172.21.15.32:0/2029291077 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f863c0cea80 0x7f863c0ccf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:30.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.496+0000 7f8640eec640 1 --2- 172.21.15.32:0/2029291077 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f863c0cd480 0x7f863c15cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:30.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.496+0000 7f863bfff640 1 -- 172.21.15.32:0/2029291077 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f863c0cd480 msgr2=0x7f863c15cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:30.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.496+0000 7f863bfff640 1 --2- 172.21.15.32:0/2029291077 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f863c0cd480 0x7f863c15cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:30.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.496+0000 7f863bfff640 1 -- 172.21.15.32:0/2029291077 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f863c0cea80 msgr2=0x7f863c0ccf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:30.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.496+0000 7f863bfff640 1 --2- 172.21.15.32:0/2029291077 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f863c0cea80 0x7f863c0ccf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:30.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.496+0000 7f863bfff640 1 -- 172.21.15.32:0/2029291077 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f863c0cd9e0 con 0x7f8624004800 2024-04-24T23:01:30.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.496+0000 7f8640eec640 1 --2- 172.21.15.32:0/2029291077 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f863c0cd480 0x7f863c15cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:01:30.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.496+0000 7f863bfff640 1 --2- 172.21.15.32:0/2029291077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8624004800 0x7f863c168b10 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f8630011cd0 tx=0x7f863000f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:30.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.496+0000 7f8638ff9640 1 -- 172.21.15.32:0/2029291077 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f863000fd20 con 0x7f8624004800 2024-04-24T23:01:30.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.496+0000 7f8638ff9640 1 -- 172.21.15.32:0/2029291077 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8630004030 con 0x7f8624004800 2024-04-24T23:01:30.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.496+0000 7f864294f640 1 -- 172.21.15.32:0/2029291077 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f863c0cdcc0 con 0x7f8624004800 2024-04-24T23:01:30.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.496+0000 7f8638ff9640 1 -- 172.21.15.32:0/2029291077 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8630004420 con 0x7f8624004800 2024-04-24T23:01:30.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.496+0000 7f864294f640 1 -- 172.21.15.32:0/2029291077 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f863c0ce280 con 0x7f8624004800 2024-04-24T23:01:30.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.497+0000 7f864294f640 1 -- 172.21.15.32:0/2029291077 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f863c068430 con 0x7f8624004800 2024-04-24T23:01:30.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.498+0000 7f8638ff9640 1 -- 172.21.15.32:0/2029291077 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8630012040 con 0x7f8624004800 2024-04-24T23:01:30.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.498+0000 7f8638ff9640 1 --2- 172.21.15.32:0/2029291077 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f862806be60 0x7f862806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:30.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.498+0000 7f8638ff9640 1 -- 172.21.15.32:0/2029291077 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7f863008b290 con 0x7f8624004800 2024-04-24T23:01:30.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.498+0000 7f863b7fe640 1 --2- 172.21.15.32:0/2029291077 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f862806be60 0x7f862806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:30.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.499+0000 7f863b7fe640 1 --2- 172.21.15.32:0/2029291077 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f862806be60 0x7f862806e320 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f862c033000 tx=0x7f862c009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:30.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.503+0000 7f8638ff9640 1 -- 172.21.15.32:0/2029291077 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8630019070 con 0x7f8624004800 2024-04-24T23:01:30.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.723+0000 7f864294f640 1 -- 172.21.15.32:0/2029291077 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f863c072a60 con 0x7f8624004800 2024-04-24T23:01:30.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.723+0000 7f8638ff9640 1 -- 172.21.15.32:0/2029291077 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+73 (secure 0 0 0) 0x7f8630059680 con 0x7f8624004800 2024-04-24T23:01:30.724 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T23:01:30.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.725+0000 7f864294f640 1 -- 172.21.15.32:0/2029291077 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f862806be60 msgr2=0x7f862806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:30.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.725+0000 7f864294f640 1 --2- 172.21.15.32:0/2029291077 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f862806be60 0x7f862806e320 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f862c033000 tx=0x7f862c009030 comp rx=0 tx=0).stop 2024-04-24T23:01:30.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.725+0000 7f864294f640 1 -- 172.21.15.32:0/2029291077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8624004800 msgr2=0x7f863c168b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:30.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.725+0000 7f864294f640 1 --2- 172.21.15.32:0/2029291077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8624004800 0x7f863c168b10 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f8630011cd0 tx=0x7f863000f610 comp rx=0 tx=0).stop 2024-04-24T23:01:30.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.725+0000 7f864294f640 1 -- 172.21.15.32:0/2029291077 shutdown_connections 2024-04-24T23:01:30.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.725+0000 7f864294f640 1 --2- 172.21.15.32:0/2029291077 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f862806be60 0x7f862806e320 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:30.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.725+0000 7f864294f640 1 --2- 172.21.15.32:0/2029291077 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f863c0cd480 0x7f863c15cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:30.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.725+0000 7f864294f640 1 --2- 172.21.15.32:0/2029291077 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f863c0cea80 0x7f863c0ccf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:30.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.725+0000 7f864294f640 1 --2- 172.21.15.32:0/2029291077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8624004800 0x7f863c168b10 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:30.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.725+0000 7f864294f640 1 -- 172.21.15.32:0/2029291077 >> 172.21.15.32:0/2029291077 conn(0x7f863c0d0ae0 msgr2=0x7f863c06cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:30.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.725+0000 7f864294f640 1 -- 172.21.15.32:0/2029291077 shutdown_connections 2024-04-24T23:01:30.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30.725+0000 7f864294f640 1 -- 172.21.15.32:0/2029291077 wait complete. 2024-04-24T23:01:30.737 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-24T23:01:30.737 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T23:01:30.737 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2024-04-24T23:01:30.737 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-24T23:01:30.737 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T23:01:30.738 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-24T23:01:30.738 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-24T23:01:30.738 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T23:01:30.738 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:01:30.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:01:30.738 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2024-04-24T23:01:30.754 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T23:01:30.754 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T23:01:30.754 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T23:01:30.754 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:01:30.754 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2024-04-24T23:01:30.754 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:01:30.754 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:01:30.754 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2024-04-24T23:01:30.754 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2024-04-24T23:01:30.754 INFO:teuthology.orchestra.run.smithi032.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:01:30.754 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2024-04-24T23:01:30.755 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2024-04-24T23:01:30.755 INFO:teuthology.orchestra.run.smithi032.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T23:01:30.755 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:01:30.755 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2024-04-24T23:01:30.755 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-24T23:01:30.755 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 42618sec preferred_lft 42618sec 2024-04-24T23:01:30.755 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link noprefixroute 2024-04-24T23:01:30.755 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:01:30.755 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:01:30.755 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:01:30.755 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2024-04-24T23:01:30.755 INFO:teuthology.orchestra.run.smithi032.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T23:01:30.755 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 72:d4:96:00:43:8c brd ff:ff:ff:ff:ff:ff 2024-04-24T23:01:30.755 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-24T23:01:30.756 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:01:30.756 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::f4c6:20ff:fe22:2647/64 scope link 2024-04-24T23:01:30.756 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:01:30.756 INFO:teuthology.orchestra.run.smithi032.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:01:30.756 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 72:d4:96:00:43:8c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T23:01:30.756 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::70d4:96ff:fe00:438c/64 scope link 2024-04-24T23:01:30.756 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:01:30.756 INFO:teuthology.orchestra.run.smithi032.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:01:30.756 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether e6:6b:2a:90:1a:f8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T23:01:30.756 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::e46b:2aff:fe90:1af8/64 scope link 2024-04-24T23:01:30.756 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:01:30.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:01:30.757 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T23:01:30.817 INFO:teuthology.orchestra.run.smithi032.stdout:1 2024-04-24T23:01:30.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:01:30.819 DEBUG:teuthology.orchestra.run.smithi032:> route 2024-04-24T23:01:30.875 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2024-04-24T23:01:30.875 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T23:01:30.875 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-24T23:01:30.875 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-24T23:01:30.875 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T23:01:30.875 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:01:30.876 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-24T23:01:30.876 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-24T23:01:30.876 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-24T23:01:30.876 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-24T23:01:30.876 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-24T23:01:30.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:01:30.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:01:31.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:01:31.001 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-04-24T23:01:31.016 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-24T23:01:31.016 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-24T23:01:31.016 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:01:31.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:01:31.016 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2024-04-24T23:01:31.081 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:01:31.081 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-24T23:01:31.136 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:01:31.137 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:01:31.137 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-24T23:01:31.191 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-24T23:01:31.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:01:31.191 DEBUG:teuthology.orchestra.run.smithi032:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:01:31.274 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:01:31.274 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-24T23:01:31.281 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:01:31.281 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-24T23:01:31.281 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.121:3300,172.21.15.121:3301" 2024-04-24T23:01:31.281 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: trying mount with new device syntax: 0@eb1b7a01-6f65-4534-9a12-83011862ba51.cephfs=/ 2024-04-24T23:01:31.281 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.32:3300/172.21.15.121:3300/172.21.15.121:3301" will pass to kernel 2024-04-24T23:01:31.282 INFO:teuthology.orchestra.run.smithi032.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-24T23:01:31.282 INFO:teuthology.orchestra.run.smithi032.stdout: You just mounted a file system that supports labels which does not 2024-04-24T23:01:31.282 INFO:teuthology.orchestra.run.smithi032.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-24T23:01:31.282 INFO:teuthology.orchestra.run.smithi032.stdout: applications will generate AVC messages and not be allowed access to 2024-04-24T23:01:31.282 INFO:teuthology.orchestra.run.smithi032.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-24T23:01:31.283 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-24T23:01:31.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:01:31.283 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-24T23:01:31.348 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:01:31.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:01:31.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:01:31.451 INFO:teuthology.orchestra.run.smithi032.stdout:client6234 2024-04-24T23:01:31.451 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:01:31.451 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:01:31.467 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T23:01:31.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.606+0000 7f93af89e640 1 Processor -- start 2024-04-24T23:01:31.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.606+0000 7f93af89e640 1 -- start start 2024-04-24T23:01:31.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.607+0000 7f93af89e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93a806cd00 0x7f93a80ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:31.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.607+0000 7f93af89e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f93a815cb00 0x7f93a80ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:31.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.607+0000 7f93af89e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93a80b4520 con 0x7f93a815b380 2024-04-24T23:01:31.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.607+0000 7f93af89e640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f93a80b4800 con 0x7f93a815cb00 2024-04-24T23:01:31.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.607+0000 7f93af89e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f93a80b4ae0 con 0x7f93a806cd00 2024-04-24T23:01:31.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.607+0000 7f93ade3b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f93a815cb00 0x7f93a80ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:31.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.607+0000 7f93ad63a640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f93a815b380 0x7f93a806c5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57272/0 (socket says 172.21.15.32:57272) 2024-04-24T23:01:31.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.607+0000 7f93ad63a640 1 -- 172.21.15.32:0/3161080785 learned_addr learned my addr 172.21.15.32:0/3161080785 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:01:31.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.607+0000 7f93ade3b640 1 -- 172.21.15.32:0/3161080785 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93a806cd00 msgr2=0x7f93a80ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:31.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.607+0000 7f93ace39640 1 --2- 172.21.15.32:0/3161080785 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93a806cd00 0x7f93a80ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:31.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.607+0000 7f939ffff640 1 -- 172.21.15.32:0/3161080785 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1094230380 0 0) 0x7f93a80b4520 con 0x7f93a815b380 2024-04-24T23:01:31.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.607+0000 7f93ade3b640 1 --2- 172.21.15.32:0/3161080785 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93a806cd00 0x7f93a80ce430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:31.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.607+0000 7f93ade3b640 1 -- 172.21.15.32:0/3161080785 >> v1:172.21.15.32:6789/0 conn(0x7f93a815b380 legacy=0x7f93a806c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:31.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.607+0000 7f93ade3b640 1 -- 172.21.15.32:0/3161080785 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93a80b4dc0 con 0x7f93a815cb00 2024-04-24T23:01:31.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.608+0000 7f93ade3b640 1 --2- 172.21.15.32:0/3161080785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93a815cb00 0x7f93a80ce970 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f93a0009d90 tx=0x7f93a002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30ac14f72079299f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:31.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.608+0000 7f939ffff640 1 -- 172.21.15.32:0/3161080785 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a0031600 con 0x7f93a815cb00 2024-04-24T23:01:31.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.608+0000 7f939ffff640 1 -- 172.21.15.32:0/3161080785 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93a00317a0 con 0x7f93a815cb00 2024-04-24T23:01:31.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.608+0000 7f939ffff640 1 -- 172.21.15.32:0/3161080785 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a0031b70 con 0x7f93a815cb00 2024-04-24T23:01:31.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.609+0000 7f93af89e640 1 -- 172.21.15.32:0/3161080785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93a815cb00 msgr2=0x7f93a80ce970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:31.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.609+0000 7f93af89e640 1 --2- 172.21.15.32:0/3161080785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93a815cb00 0x7f93a80ce970 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f93a0009d90 tx=0x7f93a002a5d0 comp rx=0 tx=0).stop 2024-04-24T23:01:31.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.609+0000 7f93af89e640 1 -- 172.21.15.32:0/3161080785 shutdown_connections 2024-04-24T23:01:31.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.609+0000 7f93af89e640 1 --2- 172.21.15.32:0/3161080785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93a815cb00 0x7f93a80ce970 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:31.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.609+0000 7f93af89e640 1 --2- 172.21.15.32:0/3161080785 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93a806cd00 0x7f93a80ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:31.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.609+0000 7f93af89e640 1 -- 172.21.15.32:0/3161080785 >> 172.21.15.32:0/3161080785 conn(0x7f93a806b110 msgr2=0x7f93a806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:31.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.609+0000 7f93af89e640 1 -- 172.21.15.32:0/3161080785 shutdown_connections 2024-04-24T23:01:31.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.609+0000 7f93af89e640 1 -- 172.21.15.32:0/3161080785 wait complete. 2024-04-24T23:01:31.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.610+0000 7f93af89e640 1 Processor -- start 2024-04-24T23:01:31.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.610+0000 7f93af89e640 1 -- start start 2024-04-24T23:01:31.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.611+0000 7f93af89e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93a806cd00 0x7f93a813fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:31.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.611+0000 7f93af89e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93a815b380 0x7f93a8070680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:31.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.611+0000 7f93af89e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93a815cb00 0x7f93a81e8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:31.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.611+0000 7f93af89e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f93a815b760 con 0x7f93a815cb00 2024-04-24T23:01:31.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.611+0000 7f93af89e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f93a80ceeb0 con 0x7f93a806cd00 2024-04-24T23:01:31.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.611+0000 7f93af89e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f93a80b55c0 con 0x7f93a815b380 2024-04-24T23:01:31.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.611+0000 7f93ade3b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93a815cb00 0x7f93a81e8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:31.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.611+0000 7f93ade3b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93a815cb00 0x7f93a81e8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57920/0 (socket says 172.21.15.32:57920) 2024-04-24T23:01:31.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.611+0000 7f93ade3b640 1 -- 172.21.15.32:0/1027356898 learned_addr learned my addr 172.21.15.32:0/1027356898 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:31.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.611+0000 7f93ad63a640 1 --2- 172.21.15.32:0/1027356898 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93a806cd00 0x7f93a813fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:31.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.611+0000 7f93ace39640 1 --2- 172.21.15.32:0/1027356898 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93a815b380 0x7f93a8070680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:31.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.611+0000 7f93ade3b640 1 -- 172.21.15.32:0/1027356898 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93a815b380 msgr2=0x7f93a8070680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:31.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.611+0000 7f93ade3b640 1 --2- 172.21.15.32:0/1027356898 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93a815b380 0x7f93a8070680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:31.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.611+0000 7f93ade3b640 1 -- 172.21.15.32:0/1027356898 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93a806cd00 msgr2=0x7f93a813fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:31.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.611+0000 7f93ade3b640 1 --2- 172.21.15.32:0/1027356898 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93a806cd00 0x7f93a813fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:31.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.611+0000 7f93ade3b640 1 -- 172.21.15.32:0/1027356898 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93a8070ff0 con 0x7f93a815cb00 2024-04-24T23:01:31.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.611+0000 7f93ace39640 1 --2- 172.21.15.32:0/1027356898 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93a815b380 0x7f93a8070680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:01:31.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.611+0000 7f93ad63a640 1 --2- 172.21.15.32:0/1027356898 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93a806cd00 0x7f93a813fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:01:31.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.611+0000 7f93ade3b640 1 --2- 172.21.15.32:0/1027356898 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93a815cb00 0x7f93a81e8020 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f93a0009eb0 tx=0x7f93a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:31.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.611+0000 7f939dffb640 1 -- 172.21.15.32:0/1027356898 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a003f070 con 0x7f93a815cb00 2024-04-24T23:01:31.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.612+0000 7f939dffb640 1 -- 172.21.15.32:0/1027356898 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93a0030be0 con 0x7f93a815cb00 2024-04-24T23:01:31.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.612+0000 7f93af89e640 1 -- 172.21.15.32:0/1027356898 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93a80712d0 con 0x7f93a815cb00 2024-04-24T23:01:31.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.612+0000 7f939dffb640 1 -- 172.21.15.32:0/1027356898 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a0009450 con 0x7f93a815cb00 2024-04-24T23:01:31.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.612+0000 7f93af89e640 1 -- 172.21.15.32:0/1027356898 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93a8140090 con 0x7f93a815cb00 2024-04-24T23:01:31.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.613+0000 7f93af89e640 1 -- 172.21.15.32:0/1027356898 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93a8156b30 con 0x7f93a815cb00 2024-04-24T23:01:31.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.613+0000 7f939dffb640 1 -- 172.21.15.32:0/1027356898 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f93a0003990 con 0x7f93a815cb00 2024-04-24T23:01:31.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.614+0000 7f939dffb640 1 --2- 172.21.15.32:0/1027356898 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f938c06be30 0x7f938c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:31.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.614+0000 7f939dffb640 1 -- 172.21.15.32:0/1027356898 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7f93a00abec0 con 0x7f93a815cb00 2024-04-24T23:01:31.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.614+0000 7f93ad63a640 1 --2- 172.21.15.32:0/1027356898 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f938c06be30 0x7f938c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:31.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.615+0000 7f93ad63a640 1 --2- 172.21.15.32:0/1027356898 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f938c06be30 0x7f938c06e2f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f93a4000f60 tx=0x7f93a4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:31.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.618+0000 7f939dffb640 1 -- 172.21.15.32:0/1027356898 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f93a8156b30 con 0x7f93a815cb00 2024-04-24T23:01:31.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.840+0000 7f93af89e640 1 -- 172.21.15.32:0/1027356898 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f93a8068430 con 0x7f93a815cb00 2024-04-24T23:01:31.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.841+0000 7f939dffb640 1 -- 172.21.15.32:0/1027356898 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+73 (secure 0 0 0) 0x7f93a007a870 con 0x7f93a815cb00 2024-04-24T23:01:31.842 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T23:01:31.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.843+0000 7f93af89e640 1 -- 172.21.15.32:0/1027356898 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f938c06be30 msgr2=0x7f938c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:31.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.843+0000 7f93af89e640 1 --2- 172.21.15.32:0/1027356898 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f938c06be30 0x7f938c06e2f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f93a4000f60 tx=0x7f93a4009040 comp rx=0 tx=0).stop 2024-04-24T23:01:31.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.843+0000 7f93af89e640 1 -- 172.21.15.32:0/1027356898 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93a815cb00 msgr2=0x7f93a81e8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:31.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.843+0000 7f93af89e640 1 --2- 172.21.15.32:0/1027356898 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93a815cb00 0x7f93a81e8020 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f93a0009eb0 tx=0x7f93a0004a40 comp rx=0 tx=0).stop 2024-04-24T23:01:31.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.843+0000 7f93af89e640 1 -- 172.21.15.32:0/1027356898 shutdown_connections 2024-04-24T23:01:31.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.843+0000 7f93af89e640 1 --2- 172.21.15.32:0/1027356898 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f938c06be30 0x7f938c06e2f0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:31.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.843+0000 7f93af89e640 1 --2- 172.21.15.32:0/1027356898 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93a815cb00 0x7f93a81e8020 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:31.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.843+0000 7f93af89e640 1 --2- 172.21.15.32:0/1027356898 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93a815b380 0x7f93a8070680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:31.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.843+0000 7f93af89e640 1 --2- 172.21.15.32:0/1027356898 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93a806cd00 0x7f93a813fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:31.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.843+0000 7f93af89e640 1 -- 172.21.15.32:0/1027356898 >> 172.21.15.32:0/1027356898 conn(0x7f93a806b110 msgr2=0x7f93a806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:31.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.843+0000 7f93af89e640 1 -- 172.21.15.32:0/1027356898 shutdown_connections 2024-04-24T23:01:31.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:31.843+0000 7f93af89e640 1 -- 172.21.15.32:0/1027356898 wait complete. 2024-04-24T23:01:31.855 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-24T23:01:31.855 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T23:01:31.855 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi121.front.sepia.ceph.com 2024-04-24T23:01:31.855 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-24T23:01:31.855 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-24T23:01:31.855 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-24T23:01:31.855 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-24T23:01:31.855 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T23:01:31.856 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-24T23:01:31.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:01:31.856 DEBUG:teuthology.orchestra.run.smithi121:> ip addr 2024-04-24T23:01:31.875 INFO:teuthology.orchestra.run.smithi121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T23:01:31.875 INFO:teuthology.orchestra.run.smithi121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T23:01:31.875 INFO:teuthology.orchestra.run.smithi121.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T23:01:31.875 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:01:31.875 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 ::1/128 scope host 2024-04-24T23:01:31.875 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:01:31.875 INFO:teuthology.orchestra.run.smithi121.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:01:31.875 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:82 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:01:31.875 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f0 2024-04-24T23:01:31.876 INFO:teuthology.orchestra.run.smithi121.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:01:31.876 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:83 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:01:31.876 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f1 2024-04-24T23:01:31.876 INFO:teuthology.orchestra.run.smithi121.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:01:31.876 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a2 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:01:31.876 INFO:teuthology.orchestra.run.smithi121.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T23:01:31.876 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a3 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:01:31.876 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.21.15.121/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-24T23:01:31.876 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft 42639sec preferred_lft 42639sec 2024-04-24T23:01:31.876 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::93dd:8489:4369:e127/64 scope link noprefixroute 2024-04-24T23:01:31.876 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:01:31.876 INFO:teuthology.orchestra.run.smithi121.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T23:01:31.876 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether c6:c0:c7:45:ae:ba brd ff:ff:ff:ff:ff:ff 2024-04-24T23:01:31.876 INFO:teuthology.orchestra.run.smithi121.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-24T23:01:31.876 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:01:31.877 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::80fe:d2ff:fe13:d1/64 scope link 2024-04-24T23:01:31.877 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:01:31.877 INFO:teuthology.orchestra.run.smithi121.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:01:31.877 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether e2:ca:b9:e9:9e:bb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T23:01:31.877 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::e0ca:b9ff:fee9:9ebb/64 scope link 2024-04-24T23:01:31.877 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:01:31.877 INFO:teuthology.orchestra.run.smithi121.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:01:31.877 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether c6:c0:c7:45:ae:ba brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-24T23:01:31.877 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::c4c0:c7ff:fe45:aeba/64 scope link 2024-04-24T23:01:31.877 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:01:31.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:01:31.878 DEBUG:teuthology.orchestra.run.smithi121:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T23:01:31.942 INFO:teuthology.orchestra.run.smithi121.stdout:1 2024-04-24T23:01:31.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:01:31.944 DEBUG:teuthology.orchestra.run.smithi121:> route 2024-04-24T23:01:32.003 INFO:teuthology.orchestra.run.smithi121.stdout:Kernel IP routing table 2024-04-24T23:01:32.003 INFO:teuthology.orchestra.run.smithi121.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T23:01:32.003 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-24T23:01:32.003 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-24T23:01:32.003 INFO:teuthology.orchestra.run.smithi121.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T23:01:32.003 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:01:32.003 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2024-04-24T23:01:32.004 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-24T23:01:32.004 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-24T23:01:32.004 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-24T23:01:32.004 DEBUG:teuthology.orchestra.run.smithi121:> ') 2024-04-24T23:01:32.096 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T23:01:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:01:32.128 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T23:01:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:01:32.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:01:32.134 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2024-04-24T23:01:32.149 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-24T23:01:32.150 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-24T23:01:32.150 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-24T23:01:32.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:01:32.150 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set brx.0 up 2024-04-24T23:01:32.216 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:01:32.216 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-24T23:01:32.271 INFO:teuthology.orchestra.run.smithi121.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-24T23:01:32.272 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:01:32.272 DEBUG:teuthology.orchestra.run.smithi121:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-24T23:01:32.327 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-24T23:01:32.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:01:32.327 DEBUG:teuthology.orchestra.run.smithi121:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:01:32.408 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:01:32.409 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-24T23:01:32.415 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:01:32.415 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-24T23:01:32.415 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.121:3300,172.21.15.121:3301" 2024-04-24T23:01:32.415 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: trying mount with new device syntax: 1@eb1b7a01-6f65-4534-9a12-83011862ba51.cephfs=/ 2024-04-24T23:01:32.415 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.32:3300/172.21.15.121:3300/172.21.15.121:3301" will pass to kernel 2024-04-24T23:01:32.416 INFO:teuthology.orchestra.run.smithi121.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-24T23:01:32.416 INFO:teuthology.orchestra.run.smithi121.stdout: You just mounted a file system that supports labels which does not 2024-04-24T23:01:32.416 INFO:teuthology.orchestra.run.smithi121.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-24T23:01:32.416 INFO:teuthology.orchestra.run.smithi121.stdout: applications will generate AVC messages and not be allowed access to 2024-04-24T23:01:32.416 INFO:teuthology.orchestra.run.smithi121.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-24T23:01:32.418 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-24T23:01:32.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:01:32.418 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-24T23:01:32.483 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:01:32.571 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T23:01:32 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:01:32.573 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T23:01:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:01:32.578 INFO:teuthology.orchestra.run.smithi121.stdout:client6246 2024-04-24T23:01:32.579 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:01:32.579 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:01:32.594 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:01:32.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.707+0000 7f2bbaf1d640 1 Processor -- start 2024-04-24T23:01:32.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.707+0000 7f2bbaf1d640 1 -- start start 2024-04-24T23:01:32.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.707+0000 7f2bbaf1d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2bb406b820 0x7f2bb40d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:32.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.707+0000 7f2bbaf1d640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2bb40d4210 0x7f2bb40b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:32.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.707+0000 7f2bbaf1d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bb40d3170 con 0x7f2bb415e9b0 2024-04-24T23:01:32.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.707+0000 7f2bbaf1d640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f2bb40d3450 con 0x7f2bb40d4210 2024-04-24T23:01:32.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.707+0000 7f2bbaf1d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f2bb40d3730 con 0x7f2bb406b820 2024-04-24T23:01:32.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.707+0000 7f2bb8cb9640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2bb415e9b0 0x7f2bb406b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57282/0 (socket says 172.21.15.32:57282) 2024-04-24T23:01:32.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.707+0000 7f2bb8cb9640 1 -- 172.21.15.32:0/431850003 learned_addr learned my addr 172.21.15.32:0/431850003 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:01:32.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.708+0000 7f2bb94ba640 1 --2- 172.21.15.32:0/431850003 >> v2:172.21.15.121:3300/0 conn(0x7f2bb40d4210 0x7f2bb40b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:32.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.708+0000 7f2bb3fff640 1 --2- 172.21.15.32:0/431850003 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2bb406b820 0x7f2bb40d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:32.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.708+0000 7f2bb37fe640 1 -- 172.21.15.32:0/431850003 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 116593685 0 0) 0x7f2bb40d3170 con 0x7f2bb415e9b0 2024-04-24T23:01:32.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.708+0000 7f2bb37fe640 1 -- 172.21.15.32:0/431850003 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2b9c003520 con 0x7f2bb415e9b0 2024-04-24T23:01:32.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.708+0000 7f2bb94ba640 1 -- 172.21.15.32:0/431850003 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2bb406b820 msgr2=0x7f2bb40d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:01:32.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.708+0000 7f2bb94ba640 1 --2- 172.21.15.32:0/431850003 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2bb406b820 0x7f2bb40d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:32.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.708+0000 7f2bb37fe640 1 -- 172.21.15.32:0/431850003 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4034365814 0 0) 0x7f2b9c003520 con 0x7f2bb415e9b0 2024-04-24T23:01:32.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.708+0000 7f2bb94ba640 1 -- 172.21.15.32:0/431850003 >> v1:172.21.15.32:6789/0 conn(0x7f2bb415e9b0 legacy=0x7f2bb406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:32.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.708+0000 7f2bb94ba640 1 -- 172.21.15.32:0/431850003 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bb40d3a10 con 0x7f2bb40d4210 2024-04-24T23:01:32.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.708+0000 7f2bb3fff640 1 --2- 172.21.15.32:0/431850003 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2bb406b820 0x7f2bb40d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:01:32.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.709+0000 7f2bb94ba640 1 --2- 172.21.15.32:0/431850003 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bb40d4210 0x7f2bb40b78f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2ba40097a0 tx=0x7f2ba402a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f2dd4e6b7338a79 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:32.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.709+0000 7f2bb37fe640 1 -- 172.21.15.32:0/431850003 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba4037070 con 0x7f2bb40d4210 2024-04-24T23:01:32.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.709+0000 7f2bb37fe640 1 -- 172.21.15.32:0/431850003 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ba402aba0 con 0x7f2bb40d4210 2024-04-24T23:01:32.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.709+0000 7f2bb37fe640 1 -- 172.21.15.32:0/431850003 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bac0019f0 con 0x7f2bb40d4210 2024-04-24T23:01:32.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.710+0000 7f2bbaf1d640 1 -- 172.21.15.32:0/431850003 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bb40d4210 msgr2=0x7f2bb40b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:32.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.710+0000 7f2bbaf1d640 1 --2- 172.21.15.32:0/431850003 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bb40d4210 0x7f2bb40b78f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2ba40097a0 tx=0x7f2ba402a110 comp rx=0 tx=0).stop 2024-04-24T23:01:32.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.710+0000 7f2bbaf1d640 1 -- 172.21.15.32:0/431850003 shutdown_connections 2024-04-24T23:01:32.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.710+0000 7f2bbaf1d640 1 --2- 172.21.15.32:0/431850003 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bb40d4210 0x7f2bb40b78f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:32.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.710+0000 7f2bbaf1d640 1 --2- 172.21.15.32:0/431850003 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2bb406b820 0x7f2bb40d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:32.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.710+0000 7f2bbaf1d640 1 -- 172.21.15.32:0/431850003 >> 172.21.15.32:0/431850003 conn(0x7f2bb40d11d0 msgr2=0x7f2bb40d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:32.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.710+0000 7f2bbaf1d640 1 -- 172.21.15.32:0/431850003 shutdown_connections 2024-04-24T23:01:32.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.710+0000 7f2bbaf1d640 1 -- 172.21.15.32:0/431850003 wait complete. 2024-04-24T23:01:32.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.711+0000 7f2bbaf1d640 1 Processor -- start 2024-04-24T23:01:32.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.711+0000 7f2bbaf1d640 1 -- start start 2024-04-24T23:01:32.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.712+0000 7f2bbaf1d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bb406b820 0x7f2bb415ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:32.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.712+0000 7f2bbaf1d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2bb40d4210 0x7f2bb414fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:32.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.712+0000 7f2bbaf1d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bb415e9b0 0x7f2bb4160c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:32.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.712+0000 7f2bbaf1d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb40ba0e0 con 0x7f2bb415e9b0 2024-04-24T23:01:32.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.712+0000 7f2bbaf1d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb40d2400 con 0x7f2bb406b820 2024-04-24T23:01:32.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.712+0000 7f2bbaf1d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f2bb40ba260 con 0x7f2bb40d4210 2024-04-24T23:01:32.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.712+0000 7f2bb94ba640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bb415e9b0 0x7f2bb4160c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:32.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.712+0000 7f2bb94ba640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bb415e9b0 0x7f2bb4160c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57932/0 (socket says 172.21.15.32:57932) 2024-04-24T23:01:32.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.712+0000 7f2bb94ba640 1 -- 172.21.15.32:0/2315559797 learned_addr learned my addr 172.21.15.32:0/2315559797 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:01:32.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.712+0000 7f2bb3fff640 1 --2- 172.21.15.32:0/2315559797 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2bb40d4210 0x7f2bb414fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:32.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.712+0000 7f2bb8cb9640 1 --2- 172.21.15.32:0/2315559797 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bb406b820 0x7f2bb415ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:32.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.712+0000 7f2bb94ba640 1 -- 172.21.15.32:0/2315559797 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2bb40d4210 msgr2=0x7f2bb414fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:32.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.712+0000 7f2bb94ba640 1 --2- 172.21.15.32:0/2315559797 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2bb40d4210 0x7f2bb414fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:32.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.712+0000 7f2bb94ba640 1 -- 172.21.15.32:0/2315559797 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bb406b820 msgr2=0x7f2bb415ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:32.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.712+0000 7f2bb94ba640 1 --2- 172.21.15.32:0/2315559797 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bb406b820 0x7f2bb415ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:32.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.712+0000 7f2bb94ba640 1 -- 172.21.15.32:0/2315559797 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bb41583d0 con 0x7f2bb415e9b0 2024-04-24T23:01:32.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.713+0000 7f2bb94ba640 1 --2- 172.21.15.32:0/2315559797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bb415e9b0 0x7f2bb4160c10 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4035040 tx=0x7f2ba40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:32.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.713+0000 7f2bb17fa640 1 -- 172.21.15.32:0/2315559797 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba4041020 con 0x7f2bb415e9b0 2024-04-24T23:01:32.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.713+0000 7f2bb17fa640 1 -- 172.21.15.32:0/2315559797 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ba403d620 con 0x7f2bb415e9b0 2024-04-24T23:01:32.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.713+0000 7f2bbaf1d640 1 -- 172.21.15.32:0/2315559797 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bb41586b0 con 0x7f2bb415e9b0 2024-04-24T23:01:32.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.713+0000 7f2bb17fa640 1 -- 172.21.15.32:0/2315559797 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba4037040 con 0x7f2bb415e9b0 2024-04-24T23:01:32.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.713+0000 7f2bbaf1d640 1 -- 172.21.15.32:0/2315559797 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bb4150270 con 0x7f2bb415e9b0 2024-04-24T23:01:32.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.714+0000 7f2bbaf1d640 1 -- 172.21.15.32:0/2315559797 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bb4068430 con 0x7f2bb415e9b0 2024-04-24T23:01:32.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.714+0000 7f2bb17fa640 1 -- 172.21.15.32:0/2315559797 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2ba4044020 con 0x7f2bb415e9b0 2024-04-24T23:01:32.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.715+0000 7f2bb17fa640 1 --2- 172.21.15.32:0/2315559797 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2ba006be60 0x7f2ba006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:01:32.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.715+0000 7f2bb17fa640 1 -- 172.21.15.32:0/2315559797 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7f2ba40ab180 con 0x7f2bb415e9b0 2024-04-24T23:01:32.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.715+0000 7f2bb8cb9640 1 --2- 172.21.15.32:0/2315559797 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2ba006be60 0x7f2ba006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:01:32.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.716+0000 7f2bb8cb9640 1 --2- 172.21.15.32:0/2315559797 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2ba006be60 0x7f2ba006e320 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2bac005c00 tx=0x7f2bac009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:01:32.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.719+0000 7f2bb17fa640 1 -- 172.21.15.32:0/2315559797 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2ba404b030 con 0x7f2bb415e9b0 2024-04-24T23:01:32.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.941+0000 7f2bbaf1d640 1 -- 172.21.15.32:0/2315559797 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2bb40d0390 con 0x7f2bb415e9b0 2024-04-24T23:01:32.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.941+0000 7f2bb17fa640 1 -- 172.21.15.32:0/2315559797 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5187 (secure 0 0 0) 0x7f2ba4079570 con 0x7f2bb415e9b0 2024-04-24T23:01:32.943 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:01:32.943 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":27,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3085809520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3085809520},{"type":"v1","addr":"172.21.15.121:6837","nonce":3085809520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":6078,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/1405465715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1405465715},{"type":"v1","addr":"172.21.15.32:6834","nonce":1405465715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":6081,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1160416671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":1160416671},{"type":"v1","addr":"172.21.15.32:6835","nonce":1160416671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:01:25.028304+0000","modified":"2024-04-24T23:01:27.048925+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6090},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6090":{"gid":6090,"name":"d","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6836/2650840833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2650840833},{"type":"v1","addr":"172.21.15.121:6836","nonce":2650840833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6090,"qdb_cluster":[6090]},"id":4}]} 2024-04-24T23:01:32.943 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 27 2024-04-24T23:01:32.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.943+0000 7f2bbaf1d640 1 -- 172.21.15.32:0/2315559797 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2ba006be60 msgr2=0x7f2ba006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:32.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.943+0000 7f2bbaf1d640 1 --2- 172.21.15.32:0/2315559797 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2ba006be60 0x7f2ba006e320 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2bac005c00 tx=0x7f2bac009040 comp rx=0 tx=0).stop 2024-04-24T23:01:32.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.943+0000 7f2bbaf1d640 1 -- 172.21.15.32:0/2315559797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bb415e9b0 msgr2=0x7f2bb4160c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:01:32.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.943+0000 7f2bbaf1d640 1 --2- 172.21.15.32:0/2315559797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bb415e9b0 0x7f2bb4160c10 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4035040 tx=0x7f2ba40046c0 comp rx=0 tx=0).stop 2024-04-24T23:01:32.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.943+0000 7f2bbaf1d640 1 -- 172.21.15.32:0/2315559797 shutdown_connections 2024-04-24T23:01:32.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.943+0000 7f2bbaf1d640 1 --2- 172.21.15.32:0/2315559797 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2ba006be60 0x7f2ba006e320 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:32.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.943+0000 7f2bbaf1d640 1 --2- 172.21.15.32:0/2315559797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bb415e9b0 0x7f2bb4160c10 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:32.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.943+0000 7f2bbaf1d640 1 --2- 172.21.15.32:0/2315559797 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2bb40d4210 0x7f2bb414fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:32.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.943+0000 7f2bbaf1d640 1 --2- 172.21.15.32:0/2315559797 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bb406b820 0x7f2bb415ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:01:32.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.943+0000 7f2bbaf1d640 1 -- 172.21.15.32:0/2315559797 >> 172.21.15.32:0/2315559797 conn(0x7f2bb40d11d0 msgr2=0x7f2bb40d15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:01:32.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.943+0000 7f2bbaf1d640 1 -- 172.21.15.32:0/2315559797 shutdown_connections 2024-04-24T23:01:32.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:01:32.944+0000 7f2bbaf1d640 1 -- 172.21.15.32:0/2315559797 wait complete. 2024-04-24T23:01:32.956 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json status 2024-04-24T23:01:33.074 INFO:teuthology.orchestra.run.smithi121.stdout:{"cluster_fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","whoami":0,"id":6090,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":7.3782353809999996,"mdsmap_epoch":27,"osdmap_epoch":67,"osdmap_epoch_barrier":67,"uptime":12.274421951000001} 2024-04-24T23:01:33.074 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", "fs_name": "cephfs", "id": 6090, "mdsmap_epoch": 27, "osdmap_epoch": 67, "osdmap_epoch_barrier": 67, "rank_uptime": 7.378235381, "state": "up:active", "uptime": 12.274421951, "want_state": "up:active", "whoami": 0 } 2024-04-24T23:01:33.075 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-04-24T23:01:33.189 INFO:teuthology.orchestra.run.smithi032.stdout:{"osd_mon_report_interval":"5"} 2024-04-24T23:01:33.189 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-04-24T23:01:33.189 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2024-04-24T23:01:33.296 INFO:teuthology.orchestra.run.smithi032.stdout:{"mon_tick_interval":"5"} 2024-04-24T23:01:33.296 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2024-04-24T23:01:33.297 INFO:tasks.cephfs.test_full:pool capacity 557082176, 531MB should be enough to fill it 2024-04-24T23:01:33.297 DEBUG:teuthology.orchestra.run.smithi032:> stdin-killer -- python3 -c ' 2024-04-24T23:01:33.297 DEBUG:teuthology.orchestra.run.smithi032:> import time 2024-04-24T23:01:33.297 DEBUG:teuthology.orchestra.run.smithi032:> import datetime 2024-04-24T23:01:33.297 DEBUG:teuthology.orchestra.run.smithi032:> import subprocess 2024-04-24T23:01:33.297 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-24T23:01:33.297 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-24T23:01:33.297 DEBUG:teuthology.orchestra.run.smithi032:> # Write some buffered data through before going full, all should be well 2024-04-24T23:01:33.297 DEBUG:teuthology.orchestra.run.smithi032:> print("writing some data through which we expect to succeed") 2024-04-24T23:01:33.297 DEBUG:teuthology.orchestra.run.smithi032:> bytes = 0 2024-04-24T23:01:33.297 DEBUG:teuthology.orchestra.run.smithi032:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2024-04-24T23:01:33.297 DEBUG:teuthology.orchestra.run.smithi032:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2024-04-24T23:01:33.298 DEBUG:teuthology.orchestra.run.smithi032:> bytes += os.write(f, b'"'"'a'"'"' * 512 * 1024) 2024-04-24T23:01:33.298 DEBUG:teuthology.orchestra.run.smithi032:> os.fsync(f) 2024-04-24T23:01:33.298 DEBUG:teuthology.orchestra.run.smithi032:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2024-04-24T23:01:33.298 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-24T23:01:33.298 DEBUG:teuthology.orchestra.run.smithi032:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2024-04-24T23:01:33.298 DEBUG:teuthology.orchestra.run.smithi032:> # writing until we see an error from fsync. As long as we'"'"'re doing 2024-04-24T23:01:33.298 DEBUG:teuthology.orchestra.run.smithi032:> # buffered IO, the error should always only appear from fsync and not 2024-04-24T23:01:33.298 DEBUG:teuthology.orchestra.run.smithi032:> # from write 2024-04-24T23:01:33.298 DEBUG:teuthology.orchestra.run.smithi032:> full = False 2024-04-24T23:01:33.298 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-24T23:01:33.298 DEBUG:teuthology.orchestra.run.smithi032:> for n in range(0, int(531 * 0.9)): 2024-04-24T23:01:33.298 DEBUG:teuthology.orchestra.run.smithi032:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2024-04-24T23:01:33.298 DEBUG:teuthology.orchestra.run.smithi032:> print("wrote {0} bytes via buffered write, may repeat".format(bytes)) 2024-04-24T23:01:33.298 DEBUG:teuthology.orchestra.run.smithi032:> print("done writing {0} bytes".format(bytes)) 2024-04-24T23:01:33.298 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-24T23:01:33.298 DEBUG:teuthology.orchestra.run.smithi032:> # OK, now we should sneak in under the full condition 2024-04-24T23:01:33.299 DEBUG:teuthology.orchestra.run.smithi032:> # due to the time it takes the OSDs to report to the 2024-04-24T23:01:33.299 DEBUG:teuthology.orchestra.run.smithi032:> # mons, and get a successful fsync on our full-making data 2024-04-24T23:01:33.299 DEBUG:teuthology.orchestra.run.smithi032:> os.fsync(f) 2024-04-24T23:01:33.299 DEBUG:teuthology.orchestra.run.smithi032:> print("successfully fsync'"'"'ed prior to getting full state reported") 2024-04-24T23:01:33.299 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-24T23:01:33.299 DEBUG:teuthology.orchestra.run.smithi032:> # buffered write, add more dirty data to the buffer 2024-04-24T23:01:33.299 DEBUG:teuthology.orchestra.run.smithi032:> print("starting buffered write") 2024-04-24T23:01:33.299 DEBUG:teuthology.orchestra.run.smithi032:> try: 2024-04-24T23:01:33.299 DEBUG:teuthology.orchestra.run.smithi032:> for n in range(0, int(531 * 0.2)): 2024-04-24T23:01:33.299 DEBUG:teuthology.orchestra.run.smithi032:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2024-04-24T23:01:33.299 DEBUG:teuthology.orchestra.run.smithi032:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2024-04-24T23:01:33.299 DEBUG:teuthology.orchestra.run.smithi032:> time.sleep(15.0) 2024-04-24T23:01:33.299 DEBUG:teuthology.orchestra.run.smithi032:> except OSError: 2024-04-24T23:01:33.299 DEBUG:teuthology.orchestra.run.smithi032:> pass; 2024-04-24T23:01:33.299 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-24T23:01:33.299 DEBUG:teuthology.orchestra.run.smithi032:> print("wrote, now waiting 30s and then doing a close we expect to fail") 2024-04-24T23:01:33.300 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-24T23:01:33.300 DEBUG:teuthology.orchestra.run.smithi032:> # Wait long enough for a background flush that should fail 2024-04-24T23:01:33.300 DEBUG:teuthology.orchestra.run.smithi032:> time.sleep(30) 2024-04-24T23:01:33.300 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-24T23:01:33.300 DEBUG:teuthology.orchestra.run.smithi032:> if False: 2024-04-24T23:01:33.300 DEBUG:teuthology.orchestra.run.smithi032:> # ...and check that the failed background flush is reflected in fclose 2024-04-24T23:01:33.300 DEBUG:teuthology.orchestra.run.smithi032:> try: 2024-04-24T23:01:33.300 DEBUG:teuthology.orchestra.run.smithi032:> os.close(f) 2024-04-24T23:01:33.300 DEBUG:teuthology.orchestra.run.smithi032:> except OSError: 2024-04-24T23:01:33.300 DEBUG:teuthology.orchestra.run.smithi032:> print("close() returned an error as expected") 2024-04-24T23:01:33.300 DEBUG:teuthology.orchestra.run.smithi032:> else: 2024-04-24T23:01:33.300 DEBUG:teuthology.orchestra.run.smithi032:> raise RuntimeError("close() failed to raise error") 2024-04-24T23:01:33.300 DEBUG:teuthology.orchestra.run.smithi032:> else: 2024-04-24T23:01:33.300 DEBUG:teuthology.orchestra.run.smithi032:> # The kernel cephfs client does not raise errors on fclose 2024-04-24T23:01:33.300 DEBUG:teuthology.orchestra.run.smithi032:> os.close(f) 2024-04-24T23:01:33.300 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-24T23:01:33.301 DEBUG:teuthology.orchestra.run.smithi032:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2024-04-24T23:01:33.301 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-24T23:01:35.868 INFO:teuthology.orchestra.run.smithi032.stdout:writing some data through which we expect to succeed 2024-04-24T23:01:35.868 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed data successfully, will now attempt to fill fs 2024-04-24T23:01:35.868 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 1572864 bytes via buffered write, may repeat 2024-04-24T23:01:35.868 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 2621440 bytes via buffered write, may repeat 2024-04-24T23:01:35.868 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 3670016 bytes via buffered write, may repeat 2024-04-24T23:01:35.868 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 4718592 bytes via buffered write, may repeat 2024-04-24T23:01:35.868 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 5767168 bytes via buffered write, may repeat 2024-04-24T23:01:35.868 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 6815744 bytes via buffered write, may repeat 2024-04-24T23:01:35.868 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 7864320 bytes via buffered write, may repeat 2024-04-24T23:01:35.868 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 8912896 bytes via buffered write, may repeat 2024-04-24T23:01:35.868 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 9961472 bytes via buffered write, may repeat 2024-04-24T23:01:35.869 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 11010048 bytes via buffered write, may repeat 2024-04-24T23:01:35.869 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 12058624 bytes via buffered write, may repeat 2024-04-24T23:01:35.869 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 13107200 bytes via buffered write, may repeat 2024-04-24T23:01:35.869 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 14155776 bytes via buffered write, may repeat 2024-04-24T23:01:35.869 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 15204352 bytes via buffered write, may repeat 2024-04-24T23:01:35.869 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 16252928 bytes via buffered write, may repeat 2024-04-24T23:01:35.869 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 17301504 bytes via buffered write, may repeat 2024-04-24T23:01:35.869 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 18350080 bytes via buffered write, may repeat 2024-04-24T23:01:35.869 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 19398656 bytes via buffered write, may repeat 2024-04-24T23:01:35.869 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 20447232 bytes via buffered write, may repeat 2024-04-24T23:01:35.869 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 21495808 bytes via buffered write, may repeat 2024-04-24T23:01:35.869 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 22544384 bytes via buffered write, may repeat 2024-04-24T23:01:35.869 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 23592960 bytes via buffered write, may repeat 2024-04-24T23:01:35.869 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 24641536 bytes via buffered write, may repeat 2024-04-24T23:01:35.869 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 25690112 bytes via buffered write, may repeat 2024-04-24T23:01:35.870 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 26738688 bytes via buffered write, may repeat 2024-04-24T23:01:35.870 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 27787264 bytes via buffered write, may repeat 2024-04-24T23:01:35.870 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 28835840 bytes via buffered write, may repeat 2024-04-24T23:01:35.870 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 29884416 bytes via buffered write, may repeat 2024-04-24T23:01:35.870 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 30932992 bytes via buffered write, may repeat 2024-04-24T23:01:35.870 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 31981568 bytes via buffered write, may repeat 2024-04-24T23:01:35.870 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 33030144 bytes via buffered write, may repeat 2024-04-24T23:01:35.870 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 34078720 bytes via buffered write, may repeat 2024-04-24T23:01:35.870 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 35127296 bytes via buffered write, may repeat 2024-04-24T23:01:35.870 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 36175872 bytes via buffered write, may repeat 2024-04-24T23:01:35.870 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 37224448 bytes via buffered write, may repeat 2024-04-24T23:01:35.870 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 38273024 bytes via buffered write, may repeat 2024-04-24T23:01:35.870 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 39321600 bytes via buffered write, may repeat 2024-04-24T23:01:35.871 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 40370176 bytes via buffered write, may repeat 2024-04-24T23:01:35.871 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 41418752 bytes via buffered write, may repeat 2024-04-24T23:01:35.871 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 42467328 bytes via buffered write, may repeat 2024-04-24T23:01:35.871 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 43515904 bytes via buffered write, may repeat 2024-04-24T23:01:35.871 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 44564480 bytes via buffered write, may repeat 2024-04-24T23:01:35.871 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 45613056 bytes via buffered write, may repeat 2024-04-24T23:01:35.871 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 46661632 bytes via buffered write, may repeat 2024-04-24T23:01:35.871 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 47710208 bytes via buffered write, may repeat 2024-04-24T23:01:35.871 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 48758784 bytes via buffered write, may repeat 2024-04-24T23:01:35.871 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 49807360 bytes via buffered write, may repeat 2024-04-24T23:01:35.871 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 50855936 bytes via buffered write, may repeat 2024-04-24T23:01:35.871 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 51904512 bytes via buffered write, may repeat 2024-04-24T23:01:35.871 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 52953088 bytes via buffered write, may repeat 2024-04-24T23:01:35.871 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 54001664 bytes via buffered write, may repeat 2024-04-24T23:01:35.871 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 55050240 bytes via buffered write, may repeat 2024-04-24T23:01:35.872 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 56098816 bytes via buffered write, may repeat 2024-04-24T23:01:35.874 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 57147392 bytes via buffered write, may repeat 2024-04-24T23:01:35.874 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 58195968 bytes via buffered write, may repeat 2024-04-24T23:01:35.874 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 59244544 bytes via buffered write, may repeat 2024-04-24T23:01:35.874 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 60293120 bytes via buffered write, may repeat 2024-04-24T23:01:35.874 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 61341696 bytes via buffered write, may repeat 2024-04-24T23:01:35.874 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 62390272 bytes via buffered write, may repeat 2024-04-24T23:01:35.874 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 63438848 bytes via buffered write, may repeat 2024-04-24T23:01:35.874 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 64487424 bytes via buffered write, may repeat 2024-04-24T23:01:35.874 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 65536000 bytes via buffered write, may repeat 2024-04-24T23:01:35.874 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 66584576 bytes via buffered write, may repeat 2024-04-24T23:01:35.874 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 67633152 bytes via buffered write, may repeat 2024-04-24T23:01:35.874 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 68681728 bytes via buffered write, may repeat 2024-04-24T23:01:35.874 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 69730304 bytes via buffered write, may repeat 2024-04-24T23:01:35.874 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 70778880 bytes via buffered write, may repeat 2024-04-24T23:01:35.875 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 71827456 bytes via buffered write, may repeat 2024-04-24T23:01:35.875 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 72876032 bytes via buffered write, may repeat 2024-04-24T23:01:35.875 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 73924608 bytes via buffered write, may repeat 2024-04-24T23:01:35.875 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 74973184 bytes via buffered write, may repeat 2024-04-24T23:01:35.875 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 76021760 bytes via buffered write, may repeat 2024-04-24T23:01:35.875 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 77070336 bytes via buffered write, may repeat 2024-04-24T23:01:35.875 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 78118912 bytes via buffered write, may repeat 2024-04-24T23:01:35.875 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 79167488 bytes via buffered write, may repeat 2024-04-24T23:01:35.875 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 80216064 bytes via buffered write, may repeat 2024-04-24T23:01:35.875 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 81264640 bytes via buffered write, may repeat 2024-04-24T23:01:35.875 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 82313216 bytes via buffered write, may repeat 2024-04-24T23:01:35.876 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 83361792 bytes via buffered write, may repeat 2024-04-24T23:01:35.876 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 84410368 bytes via buffered write, may repeat 2024-04-24T23:01:35.876 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 85458944 bytes via buffered write, may repeat 2024-04-24T23:01:35.876 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 86507520 bytes via buffered write, may repeat 2024-04-24T23:01:35.876 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 87556096 bytes via buffered write, may repeat 2024-04-24T23:01:35.876 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 88604672 bytes via buffered write, may repeat 2024-04-24T23:01:35.876 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 89653248 bytes via buffered write, may repeat 2024-04-24T23:01:35.876 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 90701824 bytes via buffered write, may repeat 2024-04-24T23:01:35.876 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 91750400 bytes via buffered write, may repeat 2024-04-24T23:01:35.876 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 92798976 bytes via buffered write, may repeat 2024-04-24T23:01:35.876 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 93847552 bytes via buffered write, may repeat 2024-04-24T23:01:35.876 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 94896128 bytes via buffered write, may repeat 2024-04-24T23:01:35.876 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 95944704 bytes via buffered write, may repeat 2024-04-24T23:01:35.876 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 96993280 bytes via buffered write, may repeat 2024-04-24T23:01:35.876 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 98041856 bytes via buffered write, may repeat 2024-04-24T23:01:35.877 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 99090432 bytes via buffered write, may repeat 2024-04-24T23:01:35.877 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 100139008 bytes via buffered write, may repeat 2024-04-24T23:01:35.877 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 101187584 bytes via buffered write, may repeat 2024-04-24T23:01:35.877 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 102236160 bytes via buffered write, may repeat 2024-04-24T23:01:35.877 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 103284736 bytes via buffered write, may repeat 2024-04-24T23:01:35.877 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 104333312 bytes via buffered write, may repeat 2024-04-24T23:01:35.877 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 105381888 bytes via buffered write, may repeat 2024-04-24T23:01:35.877 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 106430464 bytes via buffered write, may repeat 2024-04-24T23:01:35.877 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 107479040 bytes via buffered write, may repeat 2024-04-24T23:01:35.877 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 108527616 bytes via buffered write, may repeat 2024-04-24T23:01:35.877 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 109576192 bytes via buffered write, may repeat 2024-04-24T23:01:35.877 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 110624768 bytes via buffered write, may repeat 2024-04-24T23:01:35.877 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 111673344 bytes via buffered write, may repeat 2024-04-24T23:01:35.877 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 112721920 bytes via buffered write, may repeat 2024-04-24T23:01:35.877 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 113770496 bytes via buffered write, may repeat 2024-04-24T23:01:35.878 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 114819072 bytes via buffered write, may repeat 2024-04-24T23:01:35.878 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 115867648 bytes via buffered write, may repeat 2024-04-24T23:01:35.878 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 116916224 bytes via buffered write, may repeat 2024-04-24T23:01:35.878 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 117964800 bytes via buffered write, may repeat 2024-04-24T23:01:35.878 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 119013376 bytes via buffered write, may repeat 2024-04-24T23:01:35.878 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 120061952 bytes via buffered write, may repeat 2024-04-24T23:01:35.878 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 121110528 bytes via buffered write, may repeat 2024-04-24T23:01:35.878 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 122159104 bytes via buffered write, may repeat 2024-04-24T23:01:35.878 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 123207680 bytes via buffered write, may repeat 2024-04-24T23:01:35.878 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 124256256 bytes via buffered write, may repeat 2024-04-24T23:01:35.878 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 125304832 bytes via buffered write, may repeat 2024-04-24T23:01:35.879 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 126353408 bytes via buffered write, may repeat 2024-04-24T23:01:35.879 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 127401984 bytes via buffered write, may repeat 2024-04-24T23:01:35.879 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 128450560 bytes via buffered write, may repeat 2024-04-24T23:01:35.879 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 129499136 bytes via buffered write, may repeat 2024-04-24T23:01:35.879 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 130547712 bytes via buffered write, may repeat 2024-04-24T23:01:35.879 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 131596288 bytes via buffered write, may repeat 2024-04-24T23:01:35.879 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 132644864 bytes via buffered write, may repeat 2024-04-24T23:01:35.879 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 133693440 bytes via buffered write, may repeat 2024-04-24T23:01:35.879 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 134742016 bytes via buffered write, may repeat 2024-04-24T23:01:35.879 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 135790592 bytes via buffered write, may repeat 2024-04-24T23:01:35.880 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 136839168 bytes via buffered write, may repeat 2024-04-24T23:01:35.880 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 137887744 bytes via buffered write, may repeat 2024-04-24T23:01:35.880 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 138936320 bytes via buffered write, may repeat 2024-04-24T23:01:35.880 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 139984896 bytes via buffered write, may repeat 2024-04-24T23:01:35.880 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 141033472 bytes via buffered write, may repeat 2024-04-24T23:01:35.880 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 142082048 bytes via buffered write, may repeat 2024-04-24T23:01:35.881 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 143130624 bytes via buffered write, may repeat 2024-04-24T23:01:35.881 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 144179200 bytes via buffered write, may repeat 2024-04-24T23:01:35.881 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 145227776 bytes via buffered write, may repeat 2024-04-24T23:01:35.881 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 146276352 bytes via buffered write, may repeat 2024-04-24T23:01:35.881 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 147324928 bytes via buffered write, may repeat 2024-04-24T23:01:35.881 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 148373504 bytes via buffered write, may repeat 2024-04-24T23:01:35.882 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 149422080 bytes via buffered write, may repeat 2024-04-24T23:01:35.882 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 150470656 bytes via buffered write, may repeat 2024-04-24T23:01:35.882 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 151519232 bytes via buffered write, may repeat 2024-04-24T23:01:35.882 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 152567808 bytes via buffered write, may repeat 2024-04-24T23:01:35.882 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 153616384 bytes via buffered write, may repeat 2024-04-24T23:01:35.882 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 154664960 bytes via buffered write, may repeat 2024-04-24T23:01:35.883 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 155713536 bytes via buffered write, may repeat 2024-04-24T23:01:35.883 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 156762112 bytes via buffered write, may repeat 2024-04-24T23:01:35.883 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 157810688 bytes via buffered write, may repeat 2024-04-24T23:01:35.883 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 158859264 bytes via buffered write, may repeat 2024-04-24T23:01:35.883 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 159907840 bytes via buffered write, may repeat 2024-04-24T23:01:35.883 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 160956416 bytes via buffered write, may repeat 2024-04-24T23:01:35.884 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 162004992 bytes via buffered write, may repeat 2024-04-24T23:01:35.923 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 163053568 bytes via buffered write, may repeat 2024-04-24T23:01:35.923 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 164102144 bytes via buffered write, may repeat 2024-04-24T23:01:35.923 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 165150720 bytes via buffered write, may repeat 2024-04-24T23:01:35.923 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 166199296 bytes via buffered write, may repeat 2024-04-24T23:01:35.923 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 167247872 bytes via buffered write, may repeat 2024-04-24T23:01:35.923 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 168296448 bytes via buffered write, may repeat 2024-04-24T23:01:35.923 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 169345024 bytes via buffered write, may repeat 2024-04-24T23:01:35.923 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 170393600 bytes via buffered write, may repeat 2024-04-24T23:01:35.924 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 171442176 bytes via buffered write, may repeat 2024-04-24T23:01:35.924 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 172490752 bytes via buffered write, may repeat 2024-04-24T23:01:35.924 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 173539328 bytes via buffered write, may repeat 2024-04-24T23:01:35.924 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 174587904 bytes via buffered write, may repeat 2024-04-24T23:01:35.924 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 175636480 bytes via buffered write, may repeat 2024-04-24T23:01:35.924 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 176685056 bytes via buffered write, may repeat 2024-04-24T23:01:35.924 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 177733632 bytes via buffered write, may repeat 2024-04-24T23:01:35.924 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 178782208 bytes via buffered write, may repeat 2024-04-24T23:01:35.924 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 179830784 bytes via buffered write, may repeat 2024-04-24T23:01:35.924 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 180879360 bytes via buffered write, may repeat 2024-04-24T23:01:35.924 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 181927936 bytes via buffered write, may repeat 2024-04-24T23:01:35.924 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 182976512 bytes via buffered write, may repeat 2024-04-24T23:01:35.924 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 184025088 bytes via buffered write, may repeat 2024-04-24T23:01:35.924 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 185073664 bytes via buffered write, may repeat 2024-04-24T23:01:35.925 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 186122240 bytes via buffered write, may repeat 2024-04-24T23:01:35.925 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 187170816 bytes via buffered write, may repeat 2024-04-24T23:01:35.925 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 188219392 bytes via buffered write, may repeat 2024-04-24T23:01:35.925 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 189267968 bytes via buffered write, may repeat 2024-04-24T23:01:35.925 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 190316544 bytes via buffered write, may repeat 2024-04-24T23:01:35.925 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 191365120 bytes via buffered write, may repeat 2024-04-24T23:01:35.925 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 192413696 bytes via buffered write, may repeat 2024-04-24T23:01:35.925 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 193462272 bytes via buffered write, may repeat 2024-04-24T23:01:35.925 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 194510848 bytes via buffered write, may repeat 2024-04-24T23:01:35.925 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 195559424 bytes via buffered write, may repeat 2024-04-24T23:01:35.925 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 196608000 bytes via buffered write, may repeat 2024-04-24T23:01:35.925 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 197656576 bytes via buffered write, may repeat 2024-04-24T23:01:35.925 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 198705152 bytes via buffered write, may repeat 2024-04-24T23:01:35.925 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 199753728 bytes via buffered write, may repeat 2024-04-24T23:01:35.925 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 200802304 bytes via buffered write, may repeat 2024-04-24T23:01:35.926 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 201850880 bytes via buffered write, may repeat 2024-04-24T23:01:35.926 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 202899456 bytes via buffered write, may repeat 2024-04-24T23:01:35.926 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 203948032 bytes via buffered write, may repeat 2024-04-24T23:01:35.926 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 204996608 bytes via buffered write, may repeat 2024-04-24T23:01:35.926 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 206045184 bytes via buffered write, may repeat 2024-04-24T23:01:35.926 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 207093760 bytes via buffered write, may repeat 2024-04-24T23:01:35.926 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 208142336 bytes via buffered write, may repeat 2024-04-24T23:01:35.926 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 209190912 bytes via buffered write, may repeat 2024-04-24T23:01:35.926 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 210239488 bytes via buffered write, may repeat 2024-04-24T23:01:35.926 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 211288064 bytes via buffered write, may repeat 2024-04-24T23:01:35.926 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 212336640 bytes via buffered write, may repeat 2024-04-24T23:01:35.926 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 213385216 bytes via buffered write, may repeat 2024-04-24T23:01:35.926 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 214433792 bytes via buffered write, may repeat 2024-04-24T23:01:35.926 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 215482368 bytes via buffered write, may repeat 2024-04-24T23:01:35.927 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 216530944 bytes via buffered write, may repeat 2024-04-24T23:01:35.927 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 217579520 bytes via buffered write, may repeat 2024-04-24T23:01:35.927 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 218628096 bytes via buffered write, may repeat 2024-04-24T23:01:35.927 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 219676672 bytes via buffered write, may repeat 2024-04-24T23:01:35.927 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 220725248 bytes via buffered write, may repeat 2024-04-24T23:01:35.927 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 221773824 bytes via buffered write, may repeat 2024-04-24T23:01:35.927 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 222822400 bytes via buffered write, may repeat 2024-04-24T23:01:35.927 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 223870976 bytes via buffered write, may repeat 2024-04-24T23:01:35.927 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 224919552 bytes via buffered write, may repeat 2024-04-24T23:01:35.927 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 225968128 bytes via buffered write, may repeat 2024-04-24T23:01:35.927 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 227016704 bytes via buffered write, may repeat 2024-04-24T23:01:35.927 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 228065280 bytes via buffered write, may repeat 2024-04-24T23:01:35.927 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 229113856 bytes via buffered write, may repeat 2024-04-24T23:01:35.927 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 230162432 bytes via buffered write, may repeat 2024-04-24T23:01:35.927 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 231211008 bytes via buffered write, may repeat 2024-04-24T23:01:35.928 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 232259584 bytes via buffered write, may repeat 2024-04-24T23:01:35.928 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 233308160 bytes via buffered write, may repeat 2024-04-24T23:01:35.928 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 234356736 bytes via buffered write, may repeat 2024-04-24T23:01:35.928 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 235405312 bytes via buffered write, may repeat 2024-04-24T23:01:35.928 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 236453888 bytes via buffered write, may repeat 2024-04-24T23:01:35.928 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 237502464 bytes via buffered write, may repeat 2024-04-24T23:01:35.928 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 238551040 bytes via buffered write, may repeat 2024-04-24T23:01:35.928 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 239599616 bytes via buffered write, may repeat 2024-04-24T23:01:35.928 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 240648192 bytes via buffered write, may repeat 2024-04-24T23:01:35.928 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 241696768 bytes via buffered write, may repeat 2024-04-24T23:01:35.928 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 242745344 bytes via buffered write, may repeat 2024-04-24T23:01:35.928 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 243793920 bytes via buffered write, may repeat 2024-04-24T23:01:35.928 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 244842496 bytes via buffered write, may repeat 2024-04-24T23:01:35.928 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 245891072 bytes via buffered write, may repeat 2024-04-24T23:01:35.928 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 246939648 bytes via buffered write, may repeat 2024-04-24T23:01:35.929 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 247988224 bytes via buffered write, may repeat 2024-04-24T23:01:35.929 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 249036800 bytes via buffered write, may repeat 2024-04-24T23:01:35.929 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 250085376 bytes via buffered write, may repeat 2024-04-24T23:01:35.929 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 251133952 bytes via buffered write, may repeat 2024-04-24T23:01:35.929 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 252182528 bytes via buffered write, may repeat 2024-04-24T23:01:35.929 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 253231104 bytes via buffered write, may repeat 2024-04-24T23:01:35.929 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 254279680 bytes via buffered write, may repeat 2024-04-24T23:01:35.929 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 255328256 bytes via buffered write, may repeat 2024-04-24T23:01:35.929 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 256376832 bytes via buffered write, may repeat 2024-04-24T23:01:35.929 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 257425408 bytes via buffered write, may repeat 2024-04-24T23:01:35.929 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 258473984 bytes via buffered write, may repeat 2024-04-24T23:01:35.929 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 259522560 bytes via buffered write, may repeat 2024-04-24T23:01:35.929 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 260571136 bytes via buffered write, may repeat 2024-04-24T23:01:35.929 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 261619712 bytes via buffered write, may repeat 2024-04-24T23:01:35.930 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 262668288 bytes via buffered write, may repeat 2024-04-24T23:01:35.930 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 263716864 bytes via buffered write, may repeat 2024-04-24T23:01:35.930 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 264765440 bytes via buffered write, may repeat 2024-04-24T23:01:35.930 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 265814016 bytes via buffered write, may repeat 2024-04-24T23:01:35.930 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 266862592 bytes via buffered write, may repeat 2024-04-24T23:01:35.930 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 267911168 bytes via buffered write, may repeat 2024-04-24T23:01:35.930 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 268959744 bytes via buffered write, may repeat 2024-04-24T23:01:35.930 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 270008320 bytes via buffered write, may repeat 2024-04-24T23:01:35.930 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 271056896 bytes via buffered write, may repeat 2024-04-24T23:01:35.930 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 272105472 bytes via buffered write, may repeat 2024-04-24T23:01:35.930 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 273154048 bytes via buffered write, may repeat 2024-04-24T23:01:35.930 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 274202624 bytes via buffered write, may repeat 2024-04-24T23:01:35.930 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 275251200 bytes via buffered write, may repeat 2024-04-24T23:01:35.930 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 276299776 bytes via buffered write, may repeat 2024-04-24T23:01:35.930 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 277348352 bytes via buffered write, may repeat 2024-04-24T23:01:35.931 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 278396928 bytes via buffered write, may repeat 2024-04-24T23:01:35.931 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 279445504 bytes via buffered write, may repeat 2024-04-24T23:01:35.931 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 280494080 bytes via buffered write, may repeat 2024-04-24T23:01:35.931 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 281542656 bytes via buffered write, may repeat 2024-04-24T23:01:35.931 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 282591232 bytes via buffered write, may repeat 2024-04-24T23:01:35.931 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 283639808 bytes via buffered write, may repeat 2024-04-24T23:01:35.931 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 284688384 bytes via buffered write, may repeat 2024-04-24T23:01:35.931 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 285736960 bytes via buffered write, may repeat 2024-04-24T23:01:35.931 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 286785536 bytes via buffered write, may repeat 2024-04-24T23:01:35.931 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 287834112 bytes via buffered write, may repeat 2024-04-24T23:01:35.931 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 288882688 bytes via buffered write, may repeat 2024-04-24T23:01:35.931 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 289931264 bytes via buffered write, may repeat 2024-04-24T23:01:35.931 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 290979840 bytes via buffered write, may repeat 2024-04-24T23:01:35.931 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 292028416 bytes via buffered write, may repeat 2024-04-24T23:01:35.931 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 293076992 bytes via buffered write, may repeat 2024-04-24T23:01:35.932 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 294125568 bytes via buffered write, may repeat 2024-04-24T23:01:35.932 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 295174144 bytes via buffered write, may repeat 2024-04-24T23:01:35.932 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 296222720 bytes via buffered write, may repeat 2024-04-24T23:01:35.932 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 297271296 bytes via buffered write, may repeat 2024-04-24T23:01:35.932 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 298319872 bytes via buffered write, may repeat 2024-04-24T23:01:35.932 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 299368448 bytes via buffered write, may repeat 2024-04-24T23:01:35.932 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 300417024 bytes via buffered write, may repeat 2024-04-24T23:01:35.932 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 301465600 bytes via buffered write, may repeat 2024-04-24T23:01:35.932 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 302514176 bytes via buffered write, may repeat 2024-04-24T23:01:35.932 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 303562752 bytes via buffered write, may repeat 2024-04-24T23:01:35.932 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 304611328 bytes via buffered write, may repeat 2024-04-24T23:01:35.932 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 305659904 bytes via buffered write, may repeat 2024-04-24T23:01:35.932 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 306708480 bytes via buffered write, may repeat 2024-04-24T23:01:35.932 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 307757056 bytes via buffered write, may repeat 2024-04-24T23:01:35.932 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 308805632 bytes via buffered write, may repeat 2024-04-24T23:01:35.933 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 309854208 bytes via buffered write, may repeat 2024-04-24T23:01:35.933 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 310902784 bytes via buffered write, may repeat 2024-04-24T23:01:35.933 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 311951360 bytes via buffered write, may repeat 2024-04-24T23:01:35.934 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 312999936 bytes via buffered write, may repeat 2024-04-24T23:01:35.934 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 314048512 bytes via buffered write, may repeat 2024-04-24T23:01:35.934 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 315097088 bytes via buffered write, may repeat 2024-04-24T23:01:35.934 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 316145664 bytes via buffered write, may repeat 2024-04-24T23:01:35.934 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 317194240 bytes via buffered write, may repeat 2024-04-24T23:01:35.934 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 318242816 bytes via buffered write, may repeat 2024-04-24T23:01:35.934 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 319291392 bytes via buffered write, may repeat 2024-04-24T23:01:35.934 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 320339968 bytes via buffered write, may repeat 2024-04-24T23:01:35.934 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 321388544 bytes via buffered write, may repeat 2024-04-24T23:01:35.934 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 322437120 bytes via buffered write, may repeat 2024-04-24T23:01:35.934 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 323485696 bytes via buffered write, may repeat 2024-04-24T23:01:35.974 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 324534272 bytes via buffered write, may repeat 2024-04-24T23:01:35.974 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 325582848 bytes via buffered write, may repeat 2024-04-24T23:01:35.974 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 326631424 bytes via buffered write, may repeat 2024-04-24T23:01:35.974 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 327680000 bytes via buffered write, may repeat 2024-04-24T23:01:35.974 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 328728576 bytes via buffered write, may repeat 2024-04-24T23:01:35.974 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 329777152 bytes via buffered write, may repeat 2024-04-24T23:01:35.974 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 330825728 bytes via buffered write, may repeat 2024-04-24T23:01:35.974 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 331874304 bytes via buffered write, may repeat 2024-04-24T23:01:35.974 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 332922880 bytes via buffered write, may repeat 2024-04-24T23:01:35.974 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 333971456 bytes via buffered write, may repeat 2024-04-24T23:01:35.974 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 335020032 bytes via buffered write, may repeat 2024-04-24T23:01:35.975 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 336068608 bytes via buffered write, may repeat 2024-04-24T23:01:35.975 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 337117184 bytes via buffered write, may repeat 2024-04-24T23:01:35.975 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 338165760 bytes via buffered write, may repeat 2024-04-24T23:01:35.975 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 339214336 bytes via buffered write, may repeat 2024-04-24T23:01:35.975 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 340262912 bytes via buffered write, may repeat 2024-04-24T23:01:35.975 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 341311488 bytes via buffered write, may repeat 2024-04-24T23:01:35.975 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 342360064 bytes via buffered write, may repeat 2024-04-24T23:01:35.975 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 343408640 bytes via buffered write, may repeat 2024-04-24T23:01:35.975 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 344457216 bytes via buffered write, may repeat 2024-04-24T23:01:35.975 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 345505792 bytes via buffered write, may repeat 2024-04-24T23:01:35.975 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 346554368 bytes via buffered write, may repeat 2024-04-24T23:01:35.975 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 347602944 bytes via buffered write, may repeat 2024-04-24T23:01:35.976 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 348651520 bytes via buffered write, may repeat 2024-04-24T23:01:35.976 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 349700096 bytes via buffered write, may repeat 2024-04-24T23:01:35.976 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 350748672 bytes via buffered write, may repeat 2024-04-24T23:01:35.976 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 351797248 bytes via buffered write, may repeat 2024-04-24T23:01:35.976 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 352845824 bytes via buffered write, may repeat 2024-04-24T23:01:35.976 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 353894400 bytes via buffered write, may repeat 2024-04-24T23:01:35.976 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 354942976 bytes via buffered write, may repeat 2024-04-24T23:01:35.976 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 355991552 bytes via buffered write, may repeat 2024-04-24T23:01:35.976 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 357040128 bytes via buffered write, may repeat 2024-04-24T23:01:35.976 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 358088704 bytes via buffered write, may repeat 2024-04-24T23:01:35.976 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 359137280 bytes via buffered write, may repeat 2024-04-24T23:01:35.976 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 360185856 bytes via buffered write, may repeat 2024-04-24T23:01:35.977 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 361234432 bytes via buffered write, may repeat 2024-04-24T23:01:35.977 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 362283008 bytes via buffered write, may repeat 2024-04-24T23:01:35.977 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 363331584 bytes via buffered write, may repeat 2024-04-24T23:01:35.977 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 364380160 bytes via buffered write, may repeat 2024-04-24T23:01:35.977 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 365428736 bytes via buffered write, may repeat 2024-04-24T23:01:35.977 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 366477312 bytes via buffered write, may repeat 2024-04-24T23:01:35.977 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 367525888 bytes via buffered write, may repeat 2024-04-24T23:01:35.977 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 368574464 bytes via buffered write, may repeat 2024-04-24T23:01:35.977 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 369623040 bytes via buffered write, may repeat 2024-04-24T23:01:35.977 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 370671616 bytes via buffered write, may repeat 2024-04-24T23:01:35.977 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 371720192 bytes via buffered write, may repeat 2024-04-24T23:01:35.977 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 372768768 bytes via buffered write, may repeat 2024-04-24T23:01:35.978 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 373817344 bytes via buffered write, may repeat 2024-04-24T23:01:35.978 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 374865920 bytes via buffered write, may repeat 2024-04-24T23:01:35.978 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 375914496 bytes via buffered write, may repeat 2024-04-24T23:01:35.978 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 376963072 bytes via buffered write, may repeat 2024-04-24T23:01:35.978 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 378011648 bytes via buffered write, may repeat 2024-04-24T23:01:35.978 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 379060224 bytes via buffered write, may repeat 2024-04-24T23:01:35.978 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 380108800 bytes via buffered write, may repeat 2024-04-24T23:01:35.978 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 381157376 bytes via buffered write, may repeat 2024-04-24T23:01:35.978 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 382205952 bytes via buffered write, may repeat 2024-04-24T23:01:35.978 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 383254528 bytes via buffered write, may repeat 2024-04-24T23:01:35.978 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 384303104 bytes via buffered write, may repeat 2024-04-24T23:01:35.978 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 385351680 bytes via buffered write, may repeat 2024-04-24T23:01:35.978 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 386400256 bytes via buffered write, may repeat 2024-04-24T23:01:35.979 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 387448832 bytes via buffered write, may repeat 2024-04-24T23:01:35.979 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 388497408 bytes via buffered write, may repeat 2024-04-24T23:01:35.979 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 389545984 bytes via buffered write, may repeat 2024-04-24T23:01:35.979 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 390594560 bytes via buffered write, may repeat 2024-04-24T23:01:35.979 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 391643136 bytes via buffered write, may repeat 2024-04-24T23:01:35.979 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 392691712 bytes via buffered write, may repeat 2024-04-24T23:01:35.979 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 393740288 bytes via buffered write, may repeat 2024-04-24T23:01:35.979 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 394788864 bytes via buffered write, may repeat 2024-04-24T23:01:35.979 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 395837440 bytes via buffered write, may repeat 2024-04-24T23:01:35.979 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 396886016 bytes via buffered write, may repeat 2024-04-24T23:01:35.979 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 397934592 bytes via buffered write, may repeat 2024-04-24T23:01:35.979 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 398983168 bytes via buffered write, may repeat 2024-04-24T23:01:35.979 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 400031744 bytes via buffered write, may repeat 2024-04-24T23:01:35.980 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 401080320 bytes via buffered write, may repeat 2024-04-24T23:01:35.980 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 402128896 bytes via buffered write, may repeat 2024-04-24T23:01:35.980 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 403177472 bytes via buffered write, may repeat 2024-04-24T23:01:35.980 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 404226048 bytes via buffered write, may repeat 2024-04-24T23:01:35.980 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 405274624 bytes via buffered write, may repeat 2024-04-24T23:01:35.980 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 406323200 bytes via buffered write, may repeat 2024-04-24T23:01:35.980 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 407371776 bytes via buffered write, may repeat 2024-04-24T23:01:35.980 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 408420352 bytes via buffered write, may repeat 2024-04-24T23:01:35.980 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 409468928 bytes via buffered write, may repeat 2024-04-24T23:01:35.980 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 410517504 bytes via buffered write, may repeat 2024-04-24T23:01:35.980 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 411566080 bytes via buffered write, may repeat 2024-04-24T23:01:35.980 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 412614656 bytes via buffered write, may repeat 2024-04-24T23:01:35.980 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 413663232 bytes via buffered write, may repeat 2024-04-24T23:01:35.981 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 414711808 bytes via buffered write, may repeat 2024-04-24T23:01:35.981 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 415760384 bytes via buffered write, may repeat 2024-04-24T23:01:35.981 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 416808960 bytes via buffered write, may repeat 2024-04-24T23:01:35.981 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 417857536 bytes via buffered write, may repeat 2024-04-24T23:01:35.981 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 418906112 bytes via buffered write, may repeat 2024-04-24T23:01:35.981 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 419954688 bytes via buffered write, may repeat 2024-04-24T23:01:35.981 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 421003264 bytes via buffered write, may repeat 2024-04-24T23:01:35.981 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 422051840 bytes via buffered write, may repeat 2024-04-24T23:01:35.981 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 423100416 bytes via buffered write, may repeat 2024-04-24T23:01:35.981 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 424148992 bytes via buffered write, may repeat 2024-04-24T23:01:35.981 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 425197568 bytes via buffered write, may repeat 2024-04-24T23:01:35.981 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 426246144 bytes via buffered write, may repeat 2024-04-24T23:01:35.981 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 427294720 bytes via buffered write, may repeat 2024-04-24T23:01:35.981 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 428343296 bytes via buffered write, may repeat 2024-04-24T23:01:35.981 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 429391872 bytes via buffered write, may repeat 2024-04-24T23:01:35.982 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 430440448 bytes via buffered write, may repeat 2024-04-24T23:01:35.982 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 431489024 bytes via buffered write, may repeat 2024-04-24T23:01:35.982 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 432537600 bytes via buffered write, may repeat 2024-04-24T23:01:35.982 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 433586176 bytes via buffered write, may repeat 2024-04-24T23:01:35.982 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 434634752 bytes via buffered write, may repeat 2024-04-24T23:01:35.982 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 435683328 bytes via buffered write, may repeat 2024-04-24T23:01:35.982 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 436731904 bytes via buffered write, may repeat 2024-04-24T23:01:35.982 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 437780480 bytes via buffered write, may repeat 2024-04-24T23:01:35.982 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 438829056 bytes via buffered write, may repeat 2024-04-24T23:01:35.982 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 439877632 bytes via buffered write, may repeat 2024-04-24T23:01:35.982 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 440926208 bytes via buffered write, may repeat 2024-04-24T23:01:35.982 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 441974784 bytes via buffered write, may repeat 2024-04-24T23:01:35.983 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 443023360 bytes via buffered write, may repeat 2024-04-24T23:01:35.983 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 444071936 bytes via buffered write, may repeat 2024-04-24T23:01:35.983 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 445120512 bytes via buffered write, may repeat 2024-04-24T23:01:35.983 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 446169088 bytes via buffered write, may repeat 2024-04-24T23:01:35.983 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 447217664 bytes via buffered write, may repeat 2024-04-24T23:01:35.983 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 448266240 bytes via buffered write, may repeat 2024-04-24T23:01:35.983 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 449314816 bytes via buffered write, may repeat 2024-04-24T23:01:35.983 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 450363392 bytes via buffered write, may repeat 2024-04-24T23:01:35.983 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 451411968 bytes via buffered write, may repeat 2024-04-24T23:01:35.983 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 452460544 bytes via buffered write, may repeat 2024-04-24T23:01:35.983 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 453509120 bytes via buffered write, may repeat 2024-04-24T23:01:35.983 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 454557696 bytes via buffered write, may repeat 2024-04-24T23:01:35.983 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 455606272 bytes via buffered write, may repeat 2024-04-24T23:01:35.984 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 456654848 bytes via buffered write, may repeat 2024-04-24T23:01:35.984 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 457703424 bytes via buffered write, may repeat 2024-04-24T23:01:35.984 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 458752000 bytes via buffered write, may repeat 2024-04-24T23:01:35.984 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 459800576 bytes via buffered write, may repeat 2024-04-24T23:01:35.984 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 460849152 bytes via buffered write, may repeat 2024-04-24T23:01:35.984 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 461897728 bytes via buffered write, may repeat 2024-04-24T23:01:35.984 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 462946304 bytes via buffered write, may repeat 2024-04-24T23:01:35.984 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 463994880 bytes via buffered write, may repeat 2024-04-24T23:01:35.984 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 465043456 bytes via buffered write, may repeat 2024-04-24T23:01:35.984 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 466092032 bytes via buffered write, may repeat 2024-04-24T23:01:35.984 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 467140608 bytes via buffered write, may repeat 2024-04-24T23:01:35.984 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 468189184 bytes via buffered write, may repeat 2024-04-24T23:01:35.984 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 469237760 bytes via buffered write, may repeat 2024-04-24T23:01:35.984 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 470286336 bytes via buffered write, may repeat 2024-04-24T23:01:35.984 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 471334912 bytes via buffered write, may repeat 2024-04-24T23:01:35.985 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 472383488 bytes via buffered write, may repeat 2024-04-24T23:01:35.985 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 473432064 bytes via buffered write, may repeat 2024-04-24T23:01:35.985 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 474480640 bytes via buffered write, may repeat 2024-04-24T23:01:35.985 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 475529216 bytes via buffered write, may repeat 2024-04-24T23:01:35.985 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 476577792 bytes via buffered write, may repeat 2024-04-24T23:01:35.985 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 477626368 bytes via buffered write, may repeat 2024-04-24T23:01:35.985 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 478674944 bytes via buffered write, may repeat 2024-04-24T23:01:35.985 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 479723520 bytes via buffered write, may repeat 2024-04-24T23:01:35.985 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 480772096 bytes via buffered write, may repeat 2024-04-24T23:01:35.985 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 481820672 bytes via buffered write, may repeat 2024-04-24T23:01:35.985 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 482869248 bytes via buffered write, may repeat 2024-04-24T23:01:35.985 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 483917824 bytes via buffered write, may repeat 2024-04-24T23:01:35.985 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 484966400 bytes via buffered write, may repeat 2024-04-24T23:01:36.201 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:01:36.207 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:01:38.733 INFO:tasks.ceph.mon.a.smithi032.stderr:2024-04-24T23:01:38.731+0000 7f3b831cb640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 full osd(s) (OSD_FULL) 2024-04-24T23:01:43.799 INFO:tasks.ceph.mon.a.smithi032.stderr:2024-04-24T23:01:43.797+0000 7f3b831cb640 -1 log_channel(cluster) log [ERR] : Health check update: 3 full osd(s) (OSD_FULL) 2024-04-24T23:02:06.345 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:02:06.349 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:02:36.407 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:02:36.411 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:02:36.522 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 486014976 bytes via buffered write, may repeat 2024-04-24T23:02:36.523 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 487063552 bytes via buffered write, may repeat 2024-04-24T23:02:36.523 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 488112128 bytes via buffered write, may repeat 2024-04-24T23:02:36.523 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 489160704 bytes via buffered write, may repeat 2024-04-24T23:02:36.523 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 490209280 bytes via buffered write, may repeat 2024-04-24T23:02:36.523 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 491257856 bytes via buffered write, may repeat 2024-04-24T23:02:36.523 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 492306432 bytes via buffered write, may repeat 2024-04-24T23:02:36.523 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 493355008 bytes via buffered write, may repeat 2024-04-24T23:02:36.523 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 494403584 bytes via buffered write, may repeat 2024-04-24T23:02:36.523 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 495452160 bytes via buffered write, may repeat 2024-04-24T23:02:36.523 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 496500736 bytes via buffered write, may repeat 2024-04-24T23:02:36.523 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 497549312 bytes via buffered write, may repeat 2024-04-24T23:02:36.523 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 498597888 bytes via buffered write, may repeat 2024-04-24T23:02:36.524 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 499646464 bytes via buffered write, may repeat 2024-04-24T23:02:36.524 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 500695040 bytes via buffered write, may repeat 2024-04-24T23:02:36.524 INFO:teuthology.orchestra.run.smithi032.stdout:done writing 500695040 bytes 2024-04-24T23:02:36.524 INFO:teuthology.orchestra.run.smithi032.stdout:successfully fsync'ed prior to getting full state reported 2024-04-24T23:02:36.524 INFO:teuthology.orchestra.run.smithi032.stdout:starting buffered write 2024-04-24T23:02:36.524 INFO:teuthology.orchestra.run.smithi032.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-04-24T23:02:36.524 INFO:teuthology.orchestra.run.smithi032.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-04-24T23:02:36.524 INFO:teuthology.orchestra.run.smithi032.stdout:wrote, now waiting 30s and then doing a close we expect to fail 2024-04-24T23:02:36.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:02:36.537 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-24T23:02:36.537 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:02:36.628 DEBUG:teuthology.orchestra.run.smithi121:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:02:36.671 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-24T23:02:36.671 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:02:36.672 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:02:36.672 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:02:36.689 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:02:36.689 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:02:36.747 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:02:36.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.891+0000 7f621dc37640 1 Processor -- start 2024-04-24T23:02:36.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.891+0000 7f621dc37640 1 -- start start 2024-04-24T23:02:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.892+0000 7f621dc37640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f621815c5b0 0x7f62180b4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.892+0000 7f621dc37640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f621806cd00 0x7f62180b4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.892+0000 7f621dc37640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f621815c990 con 0x7f6218072cf0 2024-04-24T23:02:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.892+0000 7f621dc37640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f6218072860 con 0x7f621815c5b0 2024-04-24T23:02:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.892+0000 7f621dc37640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f6218071c50 con 0x7f621806cd00 2024-04-24T23:02:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.892+0000 7f6217fff640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f621806cd00 0x7f62180b4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.892+0000 7f62177fe640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f621815c5b0 0x7f62180b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.892+0000 7f6216ffd640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f6218072cf0 0x7f621806c5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:56580/0 (socket says 172.21.15.32:56580) 2024-04-24T23:02:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.892+0000 7f62177fe640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f621815c5b0 0x7f62180b4030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.32:45714/0 (socket says 172.21.15.32:45714) 2024-04-24T23:02:36.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.892+0000 7f6217fff640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f621806cd00 0x7f62180b4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.32:55876/0 (socket says 172.21.15.32:55876) 2024-04-24T23:02:36.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.892+0000 7f6216ffd640 1 -- 172.21.15.32:0/1201663264 learned_addr learned my addr 172.21.15.32:0/1201663264 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:02:36.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.892+0000 7f6217fff640 1 -- 172.21.15.32:0/1201663264 >> v2:172.21.15.121:3300/0 conn(0x7f621815c5b0 msgr2=0x7f62180b4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:36.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.892+0000 7f6217fff640 1 --2- 172.21.15.32:0/1201663264 >> v2:172.21.15.121:3300/0 conn(0x7f621815c5b0 0x7f62180b4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:36.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.892+0000 7f6217fff640 1 -- 172.21.15.32:0/1201663264 >> v1:172.21.15.121:6789/0 conn(0x7f6218072cf0 legacy=0x7f621806c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:36.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.893+0000 7f6217fff640 1 -- 172.21.15.32:0/1201663264 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6218071f30 con 0x7f621806cd00 2024-04-24T23:02:36.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.893+0000 7f62177fe640 1 --2- 172.21.15.32:0/1201663264 >> v2:172.21.15.121:3300/0 conn(0x7f621815c5b0 0x7f62180b4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:02:36.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.893+0000 7f6217fff640 1 --2- 172.21.15.32:0/1201663264 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f621806cd00 0x7f62180b4b40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6210009d90 tx=0x7f621002a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3df67c9b0b045a4f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:36.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.894+0000 7f62167fc640 1 -- 172.21.15.32:0/1201663264 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6210031600 con 0x7f621806cd00 2024-04-24T23:02:36.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.894+0000 7f62167fc640 1 -- 172.21.15.32:0/1201663264 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62100317a0 con 0x7f621806cd00 2024-04-24T23:02:36.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.894+0000 7f62167fc640 1 -- 172.21.15.32:0/1201663264 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6210031b70 con 0x7f621806cd00 2024-04-24T23:02:36.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.894+0000 7f621dc37640 1 -- 172.21.15.32:0/1201663264 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f621806cd00 msgr2=0x7f62180b4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:36.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.894+0000 7f621dc37640 1 --2- 172.21.15.32:0/1201663264 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f621806cd00 0x7f62180b4b40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6210009d90 tx=0x7f621002a5d0 comp rx=0 tx=0).stop 2024-04-24T23:02:36.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.894+0000 7f621dc37640 1 -- 172.21.15.32:0/1201663264 shutdown_connections 2024-04-24T23:02:36.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.894+0000 7f621dc37640 1 --2- 172.21.15.32:0/1201663264 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f621806cd00 0x7f62180b4b40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:36.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.894+0000 7f621dc37640 1 --2- 172.21.15.32:0/1201663264 >> v2:172.21.15.121:3300/0 conn(0x7f621815c5b0 0x7f62180b4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:36.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.894+0000 7f621dc37640 1 -- 172.21.15.32:0/1201663264 >> 172.21.15.32:0/1201663264 conn(0x7f621806b110 msgr2=0x7f621806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:36.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.894+0000 7f621dc37640 1 -- 172.21.15.32:0/1201663264 shutdown_connections 2024-04-24T23:02:36.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.894+0000 7f621dc37640 1 -- 172.21.15.32:0/1201663264 wait complete. 2024-04-24T23:02:36.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.895+0000 7f621dc37640 1 Processor -- start 2024-04-24T23:02:36.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.896+0000 7f621dc37640 1 -- start start 2024-04-24T23:02:36.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.896+0000 7f621dc37640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f621806cd00 0x7f62180cd770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:36.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.896+0000 7f621dc37640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6218072cf0 0x7f621813fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:36.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.896+0000 7f621dc37640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f621815c5b0 0x7f62181e91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:36.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.896+0000 7f621dc37640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f62180b4570 con 0x7f621815c5b0 2024-04-24T23:02:36.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.896+0000 7f621dc37640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f62180724f0 con 0x7f621806cd00 2024-04-24T23:02:36.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.896+0000 7f621dc37640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f62180b46f0 con 0x7f6218072cf0 2024-04-24T23:02:36.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.896+0000 7f6217fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f621815c5b0 0x7f62181e91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:36.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.897+0000 7f6217fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f621815c5b0 0x7f62181e91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53650/0 (socket says 172.21.15.32:53650) 2024-04-24T23:02:36.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.897+0000 7f6217fff640 1 -- 172.21.15.32:0/1020952423 learned_addr learned my addr 172.21.15.32:0/1020952423 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:36.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.897+0000 7f62177fe640 1 --2- 172.21.15.32:0/1020952423 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f621806cd00 0x7f62180cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:36.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.897+0000 7f6216ffd640 1 --2- 172.21.15.32:0/1020952423 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6218072cf0 0x7f621813fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:36.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.897+0000 7f6217fff640 1 -- 172.21.15.32:0/1020952423 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6218072cf0 msgr2=0x7f621813fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:36.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.897+0000 7f6217fff640 1 --2- 172.21.15.32:0/1020952423 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6218072cf0 0x7f621813fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:36.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.897+0000 7f6217fff640 1 -- 172.21.15.32:0/1020952423 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f621806cd00 msgr2=0x7f62180cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:36.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.897+0000 7f6217fff640 1 --2- 172.21.15.32:0/1020952423 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f621806cd00 0x7f62180cd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:36.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.897+0000 7f6217fff640 1 -- 172.21.15.32:0/1020952423 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f621815cec0 con 0x7f621815c5b0 2024-04-24T23:02:36.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.897+0000 7f6217fff640 1 --2- 172.21.15.32:0/1020952423 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f621815c5b0 0x7f62181e91a0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f6210009eb0 tx=0x7f6210004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:36.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.897+0000 7f61fffff640 1 -- 172.21.15.32:0/1020952423 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621003f070 con 0x7f621815c5b0 2024-04-24T23:02:36.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.897+0000 7f61fffff640 1 -- 172.21.15.32:0/1020952423 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6210030b60 con 0x7f621815c5b0 2024-04-24T23:02:36.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.897+0000 7f621dc37640 1 -- 172.21.15.32:0/1020952423 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f621815d1a0 con 0x7f621815c5b0 2024-04-24T23:02:36.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.897+0000 7f61fffff640 1 -- 172.21.15.32:0/1020952423 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6210009450 con 0x7f621815c5b0 2024-04-24T23:02:36.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.897+0000 7f621dc37640 1 -- 172.21.15.32:0/1020952423 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f621815d760 con 0x7f621815c5b0 2024-04-24T23:02:36.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.898+0000 7f621dc37640 1 -- 172.21.15.32:0/1020952423 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6218068430 con 0x7f621815c5b0 2024-04-24T23:02:36.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.899+0000 7f61fffff640 1 -- 172.21.15.32:0/1020952423 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f621003b410 con 0x7f621815c5b0 2024-04-24T23:02:36.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.899+0000 7f61fffff640 1 --2- 172.21.15.32:0/1020952423 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f620406be60 0x7f620406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:36.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.899+0000 7f61fffff640 1 -- 172.21.15.32:0/1020952423 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5767+0+0 (secure 0 0 0) 0x7f62100ac230 con 0x7f621815c5b0 2024-04-24T23:02:36.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.899+0000 7f61fffff640 1 -- 172.21.15.32:0/1020952423 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=73}) -- 0x7f6204073a50 con 0x7f621815c5b0 2024-04-24T23:02:36.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.899+0000 7f62177fe640 1 --2- 172.21.15.32:0/1020952423 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f620406be60 0x7f620406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:36.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.900+0000 7f62177fe640 1 --2- 172.21.15.32:0/1020952423 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f620406be60 0x7f620406e320 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6208009560 tx=0x7f6208009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:36.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:36.904+0000 7f61fffff640 1 -- 172.21.15.32:0/1020952423 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f621007a5e0 con 0x7f621815c5b0 2024-04-24T23:02:37.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.089+0000 7f621dc37640 1 -- 172.21.15.32:0/1020952423 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f62180b5260 con 0x7f621815c5b0 2024-04-24T23:02:37.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.090+0000 7f61fffff640 1 -- 172.21.15.32:0/1020952423 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+16823 (secure 0 0 0) 0x7f621007e8e0 con 0x7f621815c5b0 2024-04-24T23:02:37.135 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:02:37.135 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":72,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:01:52.865926+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:01:23.328944+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-24T23:01:24.010365+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["backfillfull","exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","full","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","full","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.a","mappings":[{"from":6,"to":4}]},{"pgid":"9.3a","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:02:37.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.092+0000 7f621dc37640 1 -- 172.21.15.32:0/1020952423 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f620406be60 msgr2=0x7f620406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:37.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.092+0000 7f621dc37640 1 --2- 172.21.15.32:0/1020952423 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f620406be60 0x7f620406e320 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6208009560 tx=0x7f6208009190 comp rx=0 tx=0).stop 2024-04-24T23:02:37.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.092+0000 7f621dc37640 1 -- 172.21.15.32:0/1020952423 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f621815c5b0 msgr2=0x7f62181e91a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:37.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.092+0000 7f621dc37640 1 --2- 172.21.15.32:0/1020952423 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f621815c5b0 0x7f62181e91a0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f6210009eb0 tx=0x7f6210004a40 comp rx=0 tx=0).stop 2024-04-24T23:02:37.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.092+0000 7f621dc37640 1 -- 172.21.15.32:0/1020952423 shutdown_connections 2024-04-24T23:02:37.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.092+0000 7f621dc37640 1 --2- 172.21.15.32:0/1020952423 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f620406be60 0x7f620406e320 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:37.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.092+0000 7f621dc37640 1 --2- 172.21.15.32:0/1020952423 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f621815c5b0 0x7f62181e91a0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:37.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.092+0000 7f621dc37640 1 --2- 172.21.15.32:0/1020952423 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6218072cf0 0x7f621813fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:37.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.092+0000 7f621dc37640 1 --2- 172.21.15.32:0/1020952423 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f621806cd00 0x7f62180cd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:37.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.092+0000 7f621dc37640 1 -- 172.21.15.32:0/1020952423 >> 172.21.15.32:0/1020952423 conn(0x7f621806b110 msgr2=0x7f62180afcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:37.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.092+0000 7f621dc37640 1 -- 172.21.15.32:0/1020952423 shutdown_connections 2024-04-24T23:02:37.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.092+0000 7f621dc37640 1 -- 172.21.15.32:0/1020952423 wait complete. 2024-04-24T23:02:37.138 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:01:23.328944+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:01:24.010365+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:02:37.139 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:02:37.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.241+0000 7f92e7ee1640 1 Processor -- start 2024-04-24T23:02:37.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.242+0000 7f92e7ee1640 1 -- start start 2024-04-24T23:02:37.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.242+0000 7f92e7ee1640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f92e006b820 0x7f92e00d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:37.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.242+0000 7f92e7ee1640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92e00d3170 con 0x7f92e015e9b0 2024-04-24T23:02:37.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.242+0000 7f92e7ee1640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92e00d3450 con 0x7f92e00d4210 2024-04-24T23:02:37.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.242+0000 7f92e7ee1640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f92e00d3730 con 0x7f92e006b820 2024-04-24T23:02:37.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.243+0000 7f92e547c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f92e006b820 0x7f92e00d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:37.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.243+0000 7f92e5c7d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f92e015e9b0 0x7f92e006b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48980/0 (socket says 172.21.15.32:48980) 2024-04-24T23:02:37.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.243+0000 7f92e5c7d640 1 -- 172.21.15.32:0/3358511953 learned_addr learned my addr 172.21.15.32:0/3358511953 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:02:37.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.243+0000 7f92e547c640 1 -- 172.21.15.32:0/3358511953 >> v1:172.21.15.121:6789/0 conn(0x7f92e00d4210 legacy=0x7f92e00b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:37.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.243+0000 7f92e547c640 1 -- 172.21.15.32:0/3358511953 >> v1:172.21.15.32:6789/0 conn(0x7f92e015e9b0 legacy=0x7f92e006b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:37.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.243+0000 7f92e4c7b640 1 -- 172.21.15.32:0/3358511953 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 996763399 0 0) 0x7f92e00d3170 con 0x7f92e015e9b0 2024-04-24T23:02:37.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.243+0000 7f92e547c640 1 -- 172.21.15.32:0/3358511953 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92e00d3a10 con 0x7f92e006b820 2024-04-24T23:02:37.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.244+0000 7f92e547c640 1 --2- 172.21.15.32:0/3358511953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92e006b820 0x7f92e00d2b90 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f92d00097a0 tx=0x7f92d002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f311181a027046a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:37.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.244+0000 7f92e4c7b640 1 -- 172.21.15.32:0/3358511953 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d0037070 con 0x7f92e006b820 2024-04-24T23:02:37.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.244+0000 7f92e4c7b640 1 -- 172.21.15.32:0/3358511953 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92d0032440 con 0x7f92e006b820 2024-04-24T23:02:37.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.244+0000 7f92e4c7b640 1 -- 172.21.15.32:0/3358511953 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d0032810 con 0x7f92e006b820 2024-04-24T23:02:37.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.244+0000 7f92e7ee1640 1 -- 172.21.15.32:0/3358511953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92e006b820 msgr2=0x7f92e00d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:37.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.244+0000 7f92e7ee1640 1 --2- 172.21.15.32:0/3358511953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92e006b820 0x7f92e00d2b90 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f92d00097a0 tx=0x7f92d002a1a0 comp rx=0 tx=0).stop 2024-04-24T23:02:37.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.244+0000 7f92e7ee1640 1 -- 172.21.15.32:0/3358511953 shutdown_connections 2024-04-24T23:02:37.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.244+0000 7f92e7ee1640 1 --2- 172.21.15.32:0/3358511953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92e006b820 0x7f92e00d2b90 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:37.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.244+0000 7f92e7ee1640 1 -- 172.21.15.32:0/3358511953 >> 172.21.15.32:0/3358511953 conn(0x7f92e00d11d0 msgr2=0x7f92e00d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:37.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.244+0000 7f92e7ee1640 1 -- 172.21.15.32:0/3358511953 shutdown_connections 2024-04-24T23:02:37.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.244+0000 7f92e7ee1640 1 -- 172.21.15.32:0/3358511953 wait complete. 2024-04-24T23:02:37.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.246+0000 7f92e7ee1640 1 Processor -- start 2024-04-24T23:02:37.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.246+0000 7f92e7ee1640 1 -- start start 2024-04-24T23:02:37.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.246+0000 7f92e7ee1640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f92e006b820 0x7f92e00ce410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:37.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.246+0000 7f92e7ee1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92e00d4210 0x7f92e0159a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:37.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.246+0000 7f92e7ee1640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92e015e9b0 0x7f92e0159f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:37.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.246+0000 7f92e7ee1640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f92e00ba0e0 con 0x7f92e00d4210 2024-04-24T23:02:37.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.246+0000 7f92e7ee1640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f92e00d2400 con 0x7f92e015e9b0 2024-04-24T23:02:37.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.246+0000 7f92e7ee1640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f92e00ba260 con 0x7f92e006b820 2024-04-24T23:02:37.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.246+0000 7f92e547c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92e00d4210 0x7f92e0159a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:37.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.247+0000 7f92e547c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92e00d4210 0x7f92e0159a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53660/0 (socket says 172.21.15.32:53660) 2024-04-24T23:02:37.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.247+0000 7f92e547c640 1 -- 172.21.15.32:0/3932524993 learned_addr learned my addr 172.21.15.32:0/3932524993 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:37.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.247+0000 7f92e647e640 1 --2- 172.21.15.32:0/3932524993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92e015e9b0 0x7f92e0159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:37.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.247+0000 7f92e5c7d640 1 --2- 172.21.15.32:0/3932524993 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f92e006b820 0x7f92e00ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:37.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.247+0000 7f92e547c640 1 -- 172.21.15.32:0/3932524993 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f92e006b820 msgr2=0x7f92e00ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:37.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.247+0000 7f92e547c640 1 --2- 172.21.15.32:0/3932524993 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f92e006b820 0x7f92e00ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:37.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.247+0000 7f92e547c640 1 -- 172.21.15.32:0/3932524993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92e015e9b0 msgr2=0x7f92e0159f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:37.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.247+0000 7f92e547c640 1 --2- 172.21.15.32:0/3932524993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92e015e9b0 0x7f92e0159f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:37.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.247+0000 7f92e547c640 1 -- 172.21.15.32:0/3932524993 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92e01565e0 con 0x7f92e00d4210 2024-04-24T23:02:37.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.247+0000 7f92e5c7d640 1 --2- 172.21.15.32:0/3932524993 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f92e006b820 0x7f92e00ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:02:37.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.247+0000 7f92e547c640 1 --2- 172.21.15.32:0/3932524993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92e00d4210 0x7f92e0159a30 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f92d0000c00 tx=0x7f92d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:37.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.247+0000 7f92d67fc640 1 -- 172.21.15.32:0/3932524993 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d0037040 con 0x7f92e00d4210 2024-04-24T23:02:37.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.247+0000 7f92d67fc640 1 -- 172.21.15.32:0/3932524993 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92d0042070 con 0x7f92e00d4210 2024-04-24T23:02:37.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.247+0000 7f92d67fc640 1 -- 172.21.15.32:0/3932524993 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d0030a70 con 0x7f92e00d4210 2024-04-24T23:02:37.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.247+0000 7f92e7ee1640 1 -- 172.21.15.32:0/3932524993 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92e0156880 con 0x7f92e00d4210 2024-04-24T23:02:37.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.247+0000 7f92e7ee1640 1 -- 172.21.15.32:0/3932524993 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92e0156e40 con 0x7f92e00d4210 2024-04-24T23:02:37.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.248+0000 7f92e7ee1640 1 -- 172.21.15.32:0/3932524993 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92e0068430 con 0x7f92e00d4210 2024-04-24T23:02:37.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.249+0000 7f92d67fc640 1 -- 172.21.15.32:0/3932524993 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f92d0030c10 con 0x7f92e00d4210 2024-04-24T23:02:37.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.249+0000 7f92d67fc640 1 --2- 172.21.15.32:0/3932524993 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f92c406be30 0x7f92c406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:37.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.249+0000 7f92d67fc640 1 -- 172.21.15.32:0/3932524993 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5767+0+0 (secure 0 0 0) 0x7f92d00ab8b0 con 0x7f92e00d4210 2024-04-24T23:02:37.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.249+0000 7f92d67fc640 1 -- 172.21.15.32:0/3932524993 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=73}) -- 0x7f92c4073a20 con 0x7f92e00d4210 2024-04-24T23:02:37.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.249+0000 7f92e5c7d640 1 --2- 172.21.15.32:0/3932524993 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f92c406be30 0x7f92c406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:37.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.250+0000 7f92e5c7d640 1 --2- 172.21.15.32:0/3932524993 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f92c406be30 0x7f92c406e2f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f92dc000f60 tx=0x7f92dc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:37.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.253+0000 7f92d67fc640 1 -- 172.21.15.32:0/3932524993 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f92d0045360 con 0x7f92e00d4210 2024-04-24T23:02:37.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.436+0000 7f92e7ee1640 1 -- 172.21.15.32:0/3932524993 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f92e00cfdd0 con 0x7f92e00d4210 2024-04-24T23:02:37.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.436+0000 7f92d67fc640 1 -- 172.21.15.32:0/3932524993 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) ==== 93+0+10 (secure 0 0 0) 0x7f92d007a2b0 con 0x7f92e00d4210 2024-04-24T23:02:37.438 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T23:02:37.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.438+0000 7f92e7ee1640 1 -- 172.21.15.32:0/3932524993 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f92c406be30 msgr2=0x7f92c406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:37.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.438+0000 7f92e7ee1640 1 --2- 172.21.15.32:0/3932524993 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f92c406be30 0x7f92c406e2f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f92dc000f60 tx=0x7f92dc009040 comp rx=0 tx=0).stop 2024-04-24T23:02:37.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.438+0000 7f92e7ee1640 1 -- 172.21.15.32:0/3932524993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92e00d4210 msgr2=0x7f92e0159a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:37.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.438+0000 7f92e7ee1640 1 --2- 172.21.15.32:0/3932524993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92e00d4210 0x7f92e0159a30 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f92d0000c00 tx=0x7f92d00046c0 comp rx=0 tx=0).stop 2024-04-24T23:02:37.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.438+0000 7f92e7ee1640 1 -- 172.21.15.32:0/3932524993 shutdown_connections 2024-04-24T23:02:37.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.438+0000 7f92e7ee1640 1 --2- 172.21.15.32:0/3932524993 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f92c406be30 0x7f92c406e2f0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:37.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.438+0000 7f92e7ee1640 1 --2- 172.21.15.32:0/3932524993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92e00d4210 0x7f92e0159a30 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:37.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.438+0000 7f92e7ee1640 1 --2- 172.21.15.32:0/3932524993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92e015e9b0 0x7f92e0159f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:37.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.438+0000 7f92e7ee1640 1 --2- 172.21.15.32:0/3932524993 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f92e006b820 0x7f92e00ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:37.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.438+0000 7f92e7ee1640 1 -- 172.21.15.32:0/3932524993 >> 172.21.15.32:0/3932524993 conn(0x7f92e00d11d0 msgr2=0x7f92e00d15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:37.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.438+0000 7f92e7ee1640 1 -- 172.21.15.32:0/3932524993 shutdown_connections 2024-04-24T23:02:37.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.438+0000 7f92e7ee1640 1 -- 172.21.15.32:0/3932524993 wait complete. 2024-04-24T23:02:37.450 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:02:37.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.553+0000 7f1cac8ee640 1 Processor -- start 2024-04-24T23:02:37.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.553+0000 7f1cac8ee640 1 -- start start 2024-04-24T23:02:37.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.553+0000 7f1cac8ee640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1ca815e9b0 0x7f1ca80d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:37.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.553+0000 7f1cac8ee640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1ca80d4210 0x7f1ca806b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:37.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.553+0000 7f1cac8ee640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1ca80ba0e0 0x7f1ca806b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:37.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.553+0000 7f1cac8ee640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f1ca80d3170 con 0x7f1ca80d4210 2024-04-24T23:02:37.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.553+0000 7f1cac8ee640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f1ca80d3450 con 0x7f1ca80ba0e0 2024-04-24T23:02:37.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.553+0000 7f1cac8ee640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f1ca80d3730 con 0x7f1ca815e9b0 2024-04-24T23:02:37.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.554+0000 7f1c9ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1ca80d4210 0x7f1ca806b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:37.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.554+0000 7f1c9ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1ca80d4210 0x7f1ca806b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53676/0 (socket says 172.21.15.32:53676) 2024-04-24T23:02:37.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.554+0000 7f1c9ffff640 1 -- 172.21.15.32:0/1826518661 learned_addr learned my addr 172.21.15.32:0/1826518661 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:37.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.554+0000 7f1ca659c640 1 --2- 172.21.15.32:0/1826518661 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1ca815e9b0 0x7f1ca80d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:37.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.554+0000 7f1ca6d9d640 1 --2- 172.21.15.32:0/1826518661 >> v2:172.21.15.121:3300/0 conn(0x7f1ca80ba0e0 0x7f1ca806b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:37.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.554+0000 7f1c9ffff640 1 -- 172.21.15.32:0/1826518661 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1ca815e9b0 msgr2=0x7f1ca80d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:37.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.554+0000 7f1c9ffff640 1 --2- 172.21.15.32:0/1826518661 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1ca815e9b0 0x7f1ca80d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:37.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.554+0000 7f1c9ffff640 1 -- 172.21.15.32:0/1826518661 >> v2:172.21.15.121:3300/0 conn(0x7f1ca80ba0e0 msgr2=0x7f1ca806b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:37.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.554+0000 7f1c9ffff640 1 --2- 172.21.15.32:0/1826518661 >> v2:172.21.15.121:3300/0 conn(0x7f1ca80ba0e0 0x7f1ca806b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:37.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.554+0000 7f1c9ffff640 1 -- 172.21.15.32:0/1826518661 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ca80d3a10 con 0x7f1ca80d4210 2024-04-24T23:02:37.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.555+0000 7f1c9ffff640 1 --2- 172.21.15.32:0/1826518661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ca80d4210 0x7f1ca806b110 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f1ca80b61d0 tx=0x7f1c9002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeef04196032fc80 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:37.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.555+0000 7f1ca5d9b640 1 -- 172.21.15.32:0/1826518661 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c90037070 con 0x7f1ca80d4210 2024-04-24T23:02:37.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.555+0000 7f1ca5d9b640 1 -- 172.21.15.32:0/1826518661 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c90032440 con 0x7f1ca80d4210 2024-04-24T23:02:37.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.555+0000 7f1ca5d9b640 1 -- 172.21.15.32:0/1826518661 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c90032810 con 0x7f1ca80d4210 2024-04-24T23:02:37.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.555+0000 7f1cac8ee640 1 -- 172.21.15.32:0/1826518661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ca80d4210 msgr2=0x7f1ca806b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:37.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.555+0000 7f1cac8ee640 1 --2- 172.21.15.32:0/1826518661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ca80d4210 0x7f1ca806b110 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f1ca80b61d0 tx=0x7f1c9002a1e0 comp rx=0 tx=0).stop 2024-04-24T23:02:37.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.555+0000 7f1cac8ee640 1 -- 172.21.15.32:0/1826518661 shutdown_connections 2024-04-24T23:02:37.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.555+0000 7f1cac8ee640 1 --2- 172.21.15.32:0/1826518661 >> v2:172.21.15.121:3300/0 conn(0x7f1ca80ba0e0 0x7f1ca806b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:37.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.555+0000 7f1cac8ee640 1 --2- 172.21.15.32:0/1826518661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ca80d4210 0x7f1ca806b110 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:37.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.555+0000 7f1cac8ee640 1 --2- 172.21.15.32:0/1826518661 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1ca815e9b0 0x7f1ca80d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:37.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.555+0000 7f1cac8ee640 1 -- 172.21.15.32:0/1826518661 >> 172.21.15.32:0/1826518661 conn(0x7f1ca80d11d0 msgr2=0x7f1ca80d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:37.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.555+0000 7f1cac8ee640 1 -- 172.21.15.32:0/1826518661 shutdown_connections 2024-04-24T23:02:37.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.555+0000 7f1cac8ee640 1 -- 172.21.15.32:0/1826518661 wait complete. 2024-04-24T23:02:37.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.557+0000 7f1cac8ee640 1 Processor -- start 2024-04-24T23:02:37.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.557+0000 7f1cac8ee640 1 -- start start 2024-04-24T23:02:37.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.557+0000 7f1cac8ee640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1ca80ba0e0 0x7f1ca80c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:37.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.557+0000 7f1cac8ee640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ca80d4210 0x7f1ca80cd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:37.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.557+0000 7f1cac8ee640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ca815e9b0 0x7f1ca80cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:37.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.557+0000 7f1cac8ee640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca8071730 con 0x7f1ca80d4210 2024-04-24T23:02:37.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.557+0000 7f1cac8ee640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca80d2400 con 0x7f1ca815e9b0 2024-04-24T23:02:37.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.557+0000 7f1cac8ee640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f1ca80718b0 con 0x7f1ca80ba0e0 2024-04-24T23:02:37.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.557+0000 7f1c9ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ca80d4210 0x7f1ca80cd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:37.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.558+0000 7f1c9ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ca80d4210 0x7f1ca80cd9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53686/0 (socket says 172.21.15.32:53686) 2024-04-24T23:02:37.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.558+0000 7f1c9ffff640 1 -- 172.21.15.32:0/2667829284 learned_addr learned my addr 172.21.15.32:0/2667829284 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:37.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.558+0000 7f1ca659c640 1 --2- 172.21.15.32:0/2667829284 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1ca80ba0e0 0x7f1ca80c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:37.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.558+0000 7f1ca6d9d640 1 --2- 172.21.15.32:0/2667829284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ca815e9b0 0x7f1ca80cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:37.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.558+0000 7f1c9ffff640 1 -- 172.21.15.32:0/2667829284 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1ca80ba0e0 msgr2=0x7f1ca80c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:37.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.558+0000 7f1c9ffff640 1 --2- 172.21.15.32:0/2667829284 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1ca80ba0e0 0x7f1ca80c82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:37.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.558+0000 7f1c9ffff640 1 -- 172.21.15.32:0/2667829284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ca815e9b0 msgr2=0x7f1ca80cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:37.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.558+0000 7f1c9ffff640 1 --2- 172.21.15.32:0/2667829284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ca815e9b0 0x7f1ca80cdf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:37.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.558+0000 7f1c9ffff640 1 -- 172.21.15.32:0/2667829284 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ca815f1e0 con 0x7f1ca80d4210 2024-04-24T23:02:37.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.558+0000 7f1c9ffff640 1 --2- 172.21.15.32:0/2667829284 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ca80d4210 0x7f1ca80cd9f0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f1c9002a220 tx=0x7f1c900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:37.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.558+0000 7f1c9effd640 1 -- 172.21.15.32:0/2667829284 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c90041020 con 0x7f1ca80d4210 2024-04-24T23:02:37.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.558+0000 7f1c9effd640 1 -- 172.21.15.32:0/2667829284 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c900306c0 con 0x7f1ca80d4210 2024-04-24T23:02:37.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.558+0000 7f1cac8ee640 1 -- 172.21.15.32:0/2667829284 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ca815f4c0 con 0x7f1ca80d4210 2024-04-24T23:02:37.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.558+0000 7f1c9effd640 1 -- 172.21.15.32:0/2667829284 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c90037040 con 0x7f1ca80d4210 2024-04-24T23:02:37.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.558+0000 7f1cac8ee640 1 -- 172.21.15.32:0/2667829284 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ca815fa80 con 0x7f1ca80d4210 2024-04-24T23:02:37.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.559+0000 7f1cac8ee640 1 -- 172.21.15.32:0/2667829284 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ca815e210 con 0x7f1ca80d4210 2024-04-24T23:02:37.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.560+0000 7f1c9effd640 1 -- 172.21.15.32:0/2667829284 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1c90044020 con 0x7f1ca80d4210 2024-04-24T23:02:37.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.560+0000 7f1c9effd640 1 --2- 172.21.15.32:0/2667829284 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1c8806be30 0x7f1c8806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:37.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.560+0000 7f1c9effd640 1 -- 172.21.15.32:0/2667829284 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5767+0+0 (secure 0 0 0) 0x7f1c900ab510 con 0x7f1ca80d4210 2024-04-24T23:02:37.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.561+0000 7f1ca659c640 1 --2- 172.21.15.32:0/2667829284 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1c8806be30 0x7f1c8806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:37.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.561+0000 7f1c9effd640 1 -- 172.21.15.32:0/2667829284 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=73}) -- 0x7f1c88073a80 con 0x7f1ca80d4210 2024-04-24T23:02:37.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.561+0000 7f1ca659c640 1 --2- 172.21.15.32:0/2667829284 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1c8806be30 0x7f1c8806e2f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1c980079a0 tx=0x7f1c98008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:37.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.565+0000 7f1c9effd640 1 -- 172.21.15.32:0/2667829284 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1c900a9e20 con 0x7f1ca80d4210 2024-04-24T23:02:37.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.750+0000 7f1cac8ee640 1 -- 172.21.15.32:0/2667829284 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1ca8068430 con 0x7f1ca80d4210 2024-04-24T23:02:37.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.750+0000 7f1c9effd640 1 -- 172.21.15.32:0/2667829284 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v72) ==== 104+0+11 (secure 0 0 0) 0x7f1ca815e210 con 0x7f1ca80d4210 2024-04-24T23:02:37.752 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-24T23:02:37.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.752+0000 7f1cac8ee640 1 -- 172.21.15.32:0/2667829284 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1c8806be30 msgr2=0x7f1c8806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:37.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.752+0000 7f1cac8ee640 1 --2- 172.21.15.32:0/2667829284 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1c8806be30 0x7f1c8806e2f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1c980079a0 tx=0x7f1c98008040 comp rx=0 tx=0).stop 2024-04-24T23:02:37.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.752+0000 7f1cac8ee640 1 -- 172.21.15.32:0/2667829284 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ca80d4210 msgr2=0x7f1ca80cd9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:37.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.752+0000 7f1cac8ee640 1 --2- 172.21.15.32:0/2667829284 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ca80d4210 0x7f1ca80cd9f0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f1c9002a220 tx=0x7f1c900046c0 comp rx=0 tx=0).stop 2024-04-24T23:02:37.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.752+0000 7f1cac8ee640 1 -- 172.21.15.32:0/2667829284 shutdown_connections 2024-04-24T23:02:37.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.752+0000 7f1cac8ee640 1 --2- 172.21.15.32:0/2667829284 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1c8806be30 0x7f1c8806e2f0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:37.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.752+0000 7f1cac8ee640 1 --2- 172.21.15.32:0/2667829284 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ca80d4210 0x7f1ca80cd9f0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:37.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.752+0000 7f1cac8ee640 1 --2- 172.21.15.32:0/2667829284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ca815e9b0 0x7f1ca80cdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:37.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.752+0000 7f1cac8ee640 1 --2- 172.21.15.32:0/2667829284 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1ca80ba0e0 0x7f1ca80c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:37.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.752+0000 7f1cac8ee640 1 -- 172.21.15.32:0/2667829284 >> 172.21.15.32:0/2667829284 conn(0x7f1ca80d11d0 msgr2=0x7f1ca8159a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:37.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.752+0000 7f1cac8ee640 1 -- 172.21.15.32:0/2667829284 shutdown_connections 2024-04-24T23:02:37.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.752+0000 7f1cac8ee640 1 -- 172.21.15.32:0/2667829284 wait complete. 2024-04-24T23:02:37.764 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:02:37.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.867+0000 7f6611768640 1 Processor -- start 2024-04-24T23:02:37.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.867+0000 7f6611768640 1 -- start start 2024-04-24T23:02:37.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.868+0000 7f6611768640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f660c15e9b0 0x7f660c0d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:37.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.868+0000 7f6611768640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f660c0d4210 0x7f660c06b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:37.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.868+0000 7f6611768640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f660c0ba0e0 0x7f660c06b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:37.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.868+0000 7f6611768640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f660c0d3170 con 0x7f660c0d4210 2024-04-24T23:02:37.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.868+0000 7f6611768640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f660c0d3450 con 0x7f660c15e9b0 2024-04-24T23:02:37.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.868+0000 7f6611768640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f660c0d3730 con 0x7f660c0ba0e0 2024-04-24T23:02:37.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.868+0000 7f660a7fc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f660c0d4210 0x7f660c06b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:37.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.868+0000 7f660a7fc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f660c0d4210 0x7f660c06b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53690/0 (socket says 172.21.15.32:53690) 2024-04-24T23:02:37.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.868+0000 7f660a7fc640 1 -- 172.21.15.32:0/26491915 learned_addr learned my addr 172.21.15.32:0/26491915 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:37.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.869+0000 7f660b7fe640 1 --2- 172.21.15.32:0/26491915 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f660c0ba0e0 0x7f660c06b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:37.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.869+0000 7f660affd640 1 --2- 172.21.15.32:0/26491915 >> v2:172.21.15.121:3300/0 conn(0x7f660c15e9b0 0x7f660c0d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:37.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.869+0000 7f660a7fc640 1 -- 172.21.15.32:0/26491915 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f660c0ba0e0 msgr2=0x7f660c06b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:37.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.869+0000 7f660a7fc640 1 --2- 172.21.15.32:0/26491915 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f660c0ba0e0 0x7f660c06b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:37.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.869+0000 7f660a7fc640 1 -- 172.21.15.32:0/26491915 >> v2:172.21.15.121:3300/0 conn(0x7f660c15e9b0 msgr2=0x7f660c0d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:37.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.869+0000 7f660a7fc640 1 --2- 172.21.15.32:0/26491915 >> v2:172.21.15.121:3300/0 conn(0x7f660c15e9b0 0x7f660c0d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:37.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.869+0000 7f660a7fc640 1 -- 172.21.15.32:0/26491915 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f660c0d3a10 con 0x7f660c0d4210 2024-04-24T23:02:37.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.869+0000 7f660a7fc640 1 --2- 172.21.15.32:0/26491915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f660c0d4210 0x7f660c06b110 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f660c06be20 tx=0x7f660002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dfc3a967ce5bc7a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:37.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.870+0000 7f6609ffb640 1 -- 172.21.15.32:0/26491915 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6600037070 con 0x7f660c0d4210 2024-04-24T23:02:37.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.870+0000 7f6609ffb640 1 -- 172.21.15.32:0/26491915 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6600032440 con 0x7f660c0d4210 2024-04-24T23:02:37.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.870+0000 7f6609ffb640 1 -- 172.21.15.32:0/26491915 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6600032810 con 0x7f660c0d4210 2024-04-24T23:02:37.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.870+0000 7f6611768640 1 -- 172.21.15.32:0/26491915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f660c0d4210 msgr2=0x7f660c06b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:37.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.870+0000 7f6611768640 1 --2- 172.21.15.32:0/26491915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f660c0d4210 0x7f660c06b110 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f660c06be20 tx=0x7f660002a1e0 comp rx=0 tx=0).stop 2024-04-24T23:02:37.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.870+0000 7f6611768640 1 -- 172.21.15.32:0/26491915 shutdown_connections 2024-04-24T23:02:37.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.870+0000 7f6611768640 1 --2- 172.21.15.32:0/26491915 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f660c0ba0e0 0x7f660c06b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:37.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.870+0000 7f6611768640 1 --2- 172.21.15.32:0/26491915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f660c0d4210 0x7f660c06b110 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:37.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.870+0000 7f6611768640 1 --2- 172.21.15.32:0/26491915 >> v2:172.21.15.121:3300/0 conn(0x7f660c15e9b0 0x7f660c0d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:37.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.870+0000 7f6611768640 1 -- 172.21.15.32:0/26491915 >> 172.21.15.32:0/26491915 conn(0x7f660c0d11d0 msgr2=0x7f660c0d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:37.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.870+0000 7f6611768640 1 -- 172.21.15.32:0/26491915 shutdown_connections 2024-04-24T23:02:37.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.870+0000 7f6611768640 1 -- 172.21.15.32:0/26491915 wait complete. 2024-04-24T23:02:37.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.872+0000 7f6611768640 1 Processor -- start 2024-04-24T23:02:37.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.872+0000 7f6611768640 1 -- start start 2024-04-24T23:02:37.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.872+0000 7f6611768640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f660c0ba0e0 0x7f660c159a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:37.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.872+0000 7f6611768640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f660c0d4210 0x7f660c1f4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:37.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.872+0000 7f6611768640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f660c15e9b0 0x7f660c1f1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:37.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.872+0000 7f6611768640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f660c071730 con 0x7f660c0ba0e0 2024-04-24T23:02:37.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.872+0000 7f6611768640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f660c0d2400 con 0x7f660c0d4210 2024-04-24T23:02:37.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.872+0000 7f6611768640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f660c0718b0 con 0x7f660c15e9b0 2024-04-24T23:02:37.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.872+0000 7f660affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f660c0ba0e0 0x7f660c159a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:37.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.872+0000 7f660affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f660c0ba0e0 0x7f660c159a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53698/0 (socket says 172.21.15.32:53698) 2024-04-24T23:02:37.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.872+0000 7f660affd640 1 -- 172.21.15.32:0/3034577390 learned_addr learned my addr 172.21.15.32:0/3034577390 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:37.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.873+0000 7f660affd640 1 -- 172.21.15.32:0/3034577390 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f660c15e9b0 msgr2=0x7f660c1f1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:37.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.873+0000 7f660affd640 1 --2- 172.21.15.32:0/3034577390 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f660c15e9b0 0x7f660c1f1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:37.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.873+0000 7f660affd640 1 -- 172.21.15.32:0/3034577390 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f660c0d4210 msgr2=0x7f660c1f4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:37.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.873+0000 7f660affd640 1 --2- 172.21.15.32:0/3034577390 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f660c0d4210 0x7f660c1f4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:37.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.873+0000 7f660affd640 1 -- 172.21.15.32:0/3034577390 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f660c15a0c0 con 0x7f660c0ba0e0 2024-04-24T23:02:37.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.873+0000 7f660affd640 1 --2- 172.21.15.32:0/3034577390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f660c0ba0e0 0x7f660c159a30 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f65fc00d5f0 tx=0x7f65fc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:37.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.873+0000 7f65f37fe640 1 -- 172.21.15.32:0/3034577390 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65fc0125f0 con 0x7f660c0ba0e0 2024-04-24T23:02:37.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.873+0000 7f65f37fe640 1 -- 172.21.15.32:0/3034577390 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65fc019070 con 0x7f660c0ba0e0 2024-04-24T23:02:37.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.873+0000 7f6611768640 1 -- 172.21.15.32:0/3034577390 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f660c15a3a0 con 0x7f660c0ba0e0 2024-04-24T23:02:37.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.873+0000 7f65f37fe640 1 -- 172.21.15.32:0/3034577390 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65fc0129d0 con 0x7f660c0ba0e0 2024-04-24T23:02:37.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.873+0000 7f6611768640 1 -- 172.21.15.32:0/3034577390 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f660c15a960 con 0x7f660c0ba0e0 2024-04-24T23:02:37.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.874+0000 7f6611768640 1 -- 172.21.15.32:0/3034577390 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f660c068430 con 0x7f660c0ba0e0 2024-04-24T23:02:37.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.874+0000 7f65f37fe640 1 -- 172.21.15.32:0/3034577390 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f65fc012b70 con 0x7f660c0ba0e0 2024-04-24T23:02:37.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.875+0000 7f65f37fe640 1 --2- 172.21.15.32:0/3034577390 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f65f806be30 0x7f65f806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:37.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.875+0000 7f65f37fe640 1 -- 172.21.15.32:0/3034577390 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5767+0+0 (secure 0 0 0) 0x7f65fc08b330 con 0x7f660c0ba0e0 2024-04-24T23:02:37.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.875+0000 7f65f37fe640 1 -- 172.21.15.32:0/3034577390 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=73}) -- 0x7f65f8073a20 con 0x7f660c0ba0e0 2024-04-24T23:02:37.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.875+0000 7f660a7fc640 1 --2- 172.21.15.32:0/3034577390 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f65f806be30 0x7f65f806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:37.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.876+0000 7f660a7fc640 1 --2- 172.21.15.32:0/3034577390 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f65f806be30 0x7f65f806e2f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6600034000 tx=0x7f660002c000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:37.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:37.879+0000 7f65f37fe640 1 -- 172.21.15.32:0/3034577390 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f65fc059c20 con 0x7f660c0ba0e0 2024-04-24T23:02:38.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.062+0000 7f6611768640 1 -- 172.21.15.32:0/3034577390 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f660c15d200 con 0x7f660c0ba0e0 2024-04-24T23:02:38.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.063+0000 7f65f37fe640 1 -- 172.21.15.32:0/3034577390 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v72) ==== 100+0+11 (secure 0 0 0) 0x7f65fc059520 con 0x7f660c0ba0e0 2024-04-24T23:02:38.065 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-24T23:02:38.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.065+0000 7f6611768640 1 -- 172.21.15.32:0/3034577390 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f65f806be30 msgr2=0x7f65f806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:38.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.065+0000 7f6611768640 1 --2- 172.21.15.32:0/3034577390 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f65f806be30 0x7f65f806e2f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6600034000 tx=0x7f660002c000 comp rx=0 tx=0).stop 2024-04-24T23:02:38.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.065+0000 7f6611768640 1 -- 172.21.15.32:0/3034577390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f660c0ba0e0 msgr2=0x7f660c159a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:38.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.065+0000 7f6611768640 1 --2- 172.21.15.32:0/3034577390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f660c0ba0e0 0x7f660c159a30 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f65fc00d5f0 tx=0x7f65fc00dab0 comp rx=0 tx=0).stop 2024-04-24T23:02:38.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.065+0000 7f6611768640 1 -- 172.21.15.32:0/3034577390 shutdown_connections 2024-04-24T23:02:38.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.065+0000 7f6611768640 1 --2- 172.21.15.32:0/3034577390 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f65f806be30 0x7f65f806e2f0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:38.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.065+0000 7f6611768640 1 --2- 172.21.15.32:0/3034577390 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f660c15e9b0 0x7f660c1f1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:38.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.065+0000 7f6611768640 1 --2- 172.21.15.32:0/3034577390 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f660c0d4210 0x7f660c1f4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:38.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.065+0000 7f6611768640 1 --2- 172.21.15.32:0/3034577390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f660c0ba0e0 0x7f660c159a30 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:38.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.065+0000 7f6611768640 1 -- 172.21.15.32:0/3034577390 >> 172.21.15.32:0/3034577390 conn(0x7f660c0d11d0 msgr2=0x7f660c15b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:38.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.065+0000 7f6611768640 1 -- 172.21.15.32:0/3034577390 shutdown_connections 2024-04-24T23:02:38.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.065+0000 7f6611768640 1 -- 172.21.15.32:0/3034577390 wait complete. 2024-04-24T23:02:38.077 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T23:02:38.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.178+0000 7fb797a38640 1 Processor -- start 2024-04-24T23:02:38.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.179+0000 7fb797a38640 1 -- start start 2024-04-24T23:02:38.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.179+0000 7fb797a38640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb79006c5f0 0x7fb79006c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:38.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.179+0000 7fb797a38640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb7900af890 0x7fb79015cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:38.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.179+0000 7fb797a38640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7900b6430 con 0x7fb7900b44d0 2024-04-24T23:02:38.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.179+0000 7fb797a38640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fb790156f80 con 0x7fb79006c5f0 2024-04-24T23:02:38.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.179+0000 7fb797a38640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fb7900b6150 con 0x7fb7900af890 2024-04-24T23:02:38.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.179+0000 7fb7957d4640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb79006c5f0 0x7fb79006c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:38.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.179+0000 7fb794fd3640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb7900b44d0 0x7fb7900b5890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48994/0 (socket says 172.21.15.32:48994) 2024-04-24T23:02:38.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.179+0000 7fb794fd3640 1 -- 172.21.15.32:0/1185529082 learned_addr learned my addr 172.21.15.32:0/1185529082 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:02:38.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.180+0000 7fb795fd5640 1 --2- 172.21.15.32:0/1185529082 >> v2:172.21.15.121:3300/0 conn(0x7fb7900af890 0x7fb79015cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:38.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.180+0000 7fb7957d4640 1 -- 172.21.15.32:0/1185529082 >> v2:172.21.15.121:3300/0 conn(0x7fb7900af890 msgr2=0x7fb79015cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:38.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.180+0000 7fb7957d4640 1 --2- 172.21.15.32:0/1185529082 >> v2:172.21.15.121:3300/0 conn(0x7fb7900af890 0x7fb79015cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:38.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.180+0000 7fb7957d4640 1 -- 172.21.15.32:0/1185529082 >> v1:172.21.15.32:6789/0 conn(0x7fb7900b44d0 legacy=0x7fb7900b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:38.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.180+0000 7fb7957d4640 1 -- 172.21.15.32:0/1185529082 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7900b39f0 con 0x7fb79006c5f0 2024-04-24T23:02:38.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.180+0000 7fb7957d4640 1 --2- 172.21.15.32:0/1185529082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb79006c5f0 0x7fb79006c9d0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fb790073960 tx=0x7fb78802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=402a84a1ce2a4399 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:38.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.181+0000 7fb78ffff640 1 -- 172.21.15.32:0/1185529082 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb788007740 con 0x7fb79006c5f0 2024-04-24T23:02:38.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.181+0000 7fb78ffff640 1 -- 172.21.15.32:0/1185529082 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7880078e0 con 0x7fb79006c5f0 2024-04-24T23:02:38.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.181+0000 7fb78ffff640 1 -- 172.21.15.32:0/1185529082 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb788007cb0 con 0x7fb79006c5f0 2024-04-24T23:02:38.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.181+0000 7fb797a38640 1 -- 172.21.15.32:0/1185529082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb79006c5f0 msgr2=0x7fb79006c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:38.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.181+0000 7fb797a38640 1 --2- 172.21.15.32:0/1185529082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb79006c5f0 0x7fb79006c9d0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fb790073960 tx=0x7fb78802a680 comp rx=0 tx=0).stop 2024-04-24T23:02:38.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.181+0000 7fb797a38640 1 -- 172.21.15.32:0/1185529082 shutdown_connections 2024-04-24T23:02:38.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.181+0000 7fb797a38640 1 --2- 172.21.15.32:0/1185529082 >> v2:172.21.15.121:3300/0 conn(0x7fb7900af890 0x7fb79015cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:38.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.181+0000 7fb797a38640 1 --2- 172.21.15.32:0/1185529082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb79006c5f0 0x7fb79006c9d0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:38.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.181+0000 7fb797a38640 1 -- 172.21.15.32:0/1185529082 >> 172.21.15.32:0/1185529082 conn(0x7fb79006b110 msgr2=0x7fb79006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:38.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.181+0000 7fb797a38640 1 -- 172.21.15.32:0/1185529082 shutdown_connections 2024-04-24T23:02:38.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.181+0000 7fb797a38640 1 -- 172.21.15.32:0/1185529082 wait complete. 2024-04-24T23:02:38.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.182+0000 7fb797a38640 1 Processor -- start 2024-04-24T23:02:38.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.182+0000 7fb797a38640 1 -- start start 2024-04-24T23:02:38.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.183+0000 7fb797a38640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb79006c5f0 0x7fb7900ccd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:38.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.183+0000 7fb797a38640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb7900af890 0x7fb7901e4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:38.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.183+0000 7fb797a38640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7900b44d0 0x7fb7901e5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:38.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.183+0000 7fb797a38640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb79006cf10 con 0x7fb79006c5f0 2024-04-24T23:02:38.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.183+0000 7fb797a38640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb7900b3fb0 con 0x7fb7900b44d0 2024-04-24T23:02:38.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.183+0000 7fb797a38640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fb79015d000 con 0x7fb7900af890 2024-04-24T23:02:38.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.183+0000 7fb7957d4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb79006c5f0 0x7fb7900ccd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:38.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.183+0000 7fb7957d4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb79006c5f0 0x7fb7900ccd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53726/0 (socket says 172.21.15.32:53726) 2024-04-24T23:02:38.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.183+0000 7fb7957d4640 1 -- 172.21.15.32:0/854091740 learned_addr learned my addr 172.21.15.32:0/854091740 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:38.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.183+0000 7fb7957d4640 1 -- 172.21.15.32:0/854091740 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb7900af890 msgr2=0x7fb7901e4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:38.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.183+0000 7fb7957d4640 1 --2- 172.21.15.32:0/854091740 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb7900af890 0x7fb7901e4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:38.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.183+0000 7fb795fd5640 1 --2- 172.21.15.32:0/854091740 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7900b44d0 0x7fb7901e5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:38.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.184+0000 7fb7957d4640 1 -- 172.21.15.32:0/854091740 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7900b44d0 msgr2=0x7fb7901e5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:38.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.184+0000 7fb7957d4640 1 --2- 172.21.15.32:0/854091740 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7900b44d0 0x7fb7901e5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:38.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.184+0000 7fb7957d4640 1 -- 172.21.15.32:0/854091740 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7900cd820 con 0x7fb79006c5f0 2024-04-24T23:02:38.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.184+0000 7fb7957d4640 1 --2- 172.21.15.32:0/854091740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb79006c5f0 0x7fb7900ccd60 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fb788009e80 tx=0x7fb788004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:38.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.184+0000 7fb78dffb640 1 -- 172.21.15.32:0/854091740 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb788040070 con 0x7fb79006c5f0 2024-04-24T23:02:38.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.184+0000 7fb78dffb640 1 -- 172.21.15.32:0/854091740 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb788031c40 con 0x7fb79006c5f0 2024-04-24T23:02:38.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.184+0000 7fb797a38640 1 -- 172.21.15.32:0/854091740 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7900cdb00 con 0x7fb79006c5f0 2024-04-24T23:02:38.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.184+0000 7fb78dffb640 1 -- 172.21.15.32:0/854091740 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78803c420 con 0x7fb79006c5f0 2024-04-24T23:02:38.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.184+0000 7fb797a38640 1 -- 172.21.15.32:0/854091740 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7900ce0c0 con 0x7fb79006c5f0 2024-04-24T23:02:38.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.185+0000 7fb797a38640 1 -- 172.21.15.32:0/854091740 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7900ce710 con 0x7fb79006c5f0 2024-04-24T23:02:38.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.186+0000 7fb78dffb640 1 -- 172.21.15.32:0/854091740 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb788039040 con 0x7fb79006c5f0 2024-04-24T23:02:38.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.186+0000 7fb78dffb640 1 --2- 172.21.15.32:0/854091740 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb77c06be30 0x7fb77c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:38.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.186+0000 7fb78dffb640 1 -- 172.21.15.32:0/854091740 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5767+0+0 (secure 0 0 0) 0x7fb7880ac570 con 0x7fb79006c5f0 2024-04-24T23:02:38.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.186+0000 7fb78dffb640 1 -- 172.21.15.32:0/854091740 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=73}) -- 0x7fb77c073a20 con 0x7fb79006c5f0 2024-04-24T23:02:38.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.186+0000 7fb794fd3640 1 --2- 172.21.15.32:0/854091740 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb77c06be30 0x7fb77c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:38.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.187+0000 7fb794fd3640 1 --2- 172.21.15.32:0/854091740 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb77c06be30 0x7fb77c06e2f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb784000f60 tx=0x7fb784007a80 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:38.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.190+0000 7fb78dffb640 1 -- 172.21.15.32:0/854091740 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb78807aee0 con 0x7fb79006c5f0 2024-04-24T23:02:38.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.385+0000 7fb797a38640 1 -- 172.21.15.32:0/854091740 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb790068430 con 0x7fb79006c5f0 2024-04-24T23:02:38.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.385+0000 7fb78dffb640 1 -- 172.21.15.32:0/854091740 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v72) ==== 80+0+0 (secure 0 0 0) 0x7fb78807a7e0 con 0x7fb79006c5f0 2024-04-24T23:02:38.387 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2024-04-24T23:02:38.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.387+0000 7fb797a38640 1 -- 172.21.15.32:0/854091740 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb77c06be30 msgr2=0x7fb77c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:38.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.387+0000 7fb797a38640 1 --2- 172.21.15.32:0/854091740 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb77c06be30 0x7fb77c06e2f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb784000f60 tx=0x7fb784007a80 comp rx=0 tx=0).stop 2024-04-24T23:02:38.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.387+0000 7fb797a38640 1 -- 172.21.15.32:0/854091740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb79006c5f0 msgr2=0x7fb7900ccd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:38.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.387+0000 7fb797a38640 1 --2- 172.21.15.32:0/854091740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb79006c5f0 0x7fb7900ccd60 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fb788009e80 tx=0x7fb788004a40 comp rx=0 tx=0).stop 2024-04-24T23:02:38.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.387+0000 7fb797a38640 1 -- 172.21.15.32:0/854091740 shutdown_connections 2024-04-24T23:02:38.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.387+0000 7fb797a38640 1 --2- 172.21.15.32:0/854091740 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb77c06be30 0x7fb77c06e2f0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:38.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.387+0000 7fb797a38640 1 --2- 172.21.15.32:0/854091740 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7900b44d0 0x7fb7901e5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:38.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.387+0000 7fb797a38640 1 --2- 172.21.15.32:0/854091740 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb7900af890 0x7fb7901e4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:38.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.387+0000 7fb797a38640 1 --2- 172.21.15.32:0/854091740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb79006c5f0 0x7fb7900ccd60 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:38.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.387+0000 7fb797a38640 1 -- 172.21.15.32:0/854091740 >> 172.21.15.32:0/854091740 conn(0x7fb79006b110 msgr2=0x7fb7900d01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:38.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.387+0000 7fb797a38640 1 -- 172.21.15.32:0/854091740 shutdown_connections 2024-04-24T23:02:38.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.387+0000 7fb797a38640 1 -- 172.21.15.32:0/854091740 wait complete. 2024-04-24T23:02:38.399 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-24T23:02:38.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:02:38.399 DEBUG:teuthology.orchestra.run.smithi032:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-24T23:02:38.429 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2024-04-24T23:02:38.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:02:38.429 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-24T23:02:38.485 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:02:38.485 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:02:38.485 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:02:38.504 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:02:38.504 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:02:38.562 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:02:38.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.661+0000 7f67b051d640 1 Processor -- start 2024-04-24T23:02:38.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.661+0000 7f67b051d640 1 -- start start 2024-04-24T23:02:38.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.662+0000 7f67b051d640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f67a806cd00 0x7f67a80ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:38.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.662+0000 7f67b051d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f67a815cb00 0x7f67a80ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:38.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.662+0000 7f67b051d640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67a80b4800 con 0x7f67a815b380 2024-04-24T23:02:38.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.662+0000 7f67b051d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f67a80b4ae0 con 0x7f67a815cb00 2024-04-24T23:02:38.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.662+0000 7f67b051d640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f67a80b4dc0 con 0x7f67a806cd00 2024-04-24T23:02:38.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.662+0000 7f67aeaba640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f67a815cb00 0x7f67a80ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:38.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.662+0000 7f67aeaba640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f67a815cb00 0x7f67a80ce970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53738/0 (socket says 172.21.15.32:53738) 2024-04-24T23:02:38.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.662+0000 7f67aeaba640 1 -- 172.21.15.32:0/842154129 learned_addr learned my addr 172.21.15.32:0/842154129 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:38.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.662+0000 7f67adab8640 1 --2- 172.21.15.32:0/842154129 >> v2:172.21.15.121:3300/0 conn(0x7f67a806cd00 0x7f67a80ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:38.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.663+0000 7f67aeaba640 1 -- 172.21.15.32:0/842154129 >> v2:172.21.15.121:3300/0 conn(0x7f67a806cd00 msgr2=0x7f67a80ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:38.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.663+0000 7f67aeaba640 1 --2- 172.21.15.32:0/842154129 >> v2:172.21.15.121:3300/0 conn(0x7f67a806cd00 0x7f67a80ce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:38.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.663+0000 7f67aeaba640 1 -- 172.21.15.32:0/842154129 >> v1:172.21.15.121:6789/0 conn(0x7f67a815b380 legacy=0x7f67a806c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:38.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.663+0000 7f67aeaba640 1 -- 172.21.15.32:0/842154129 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67a80b50a0 con 0x7f67a815cb00 2024-04-24T23:02:38.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.663+0000 7f67aeaba640 1 --2- 172.21.15.32:0/842154129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67a815cb00 0x7f67a80ce970 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f67a80b33c0 tx=0x7f67a002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ca6c87b00cf405f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:38.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.663+0000 7f67ad2b7640 1 -- 172.21.15.32:0/842154129 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a0037070 con 0x7f67a815cb00 2024-04-24T23:02:38.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.663+0000 7f67ad2b7640 1 -- 172.21.15.32:0/842154129 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67a0032440 con 0x7f67a815cb00 2024-04-24T23:02:38.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.663+0000 7f67ad2b7640 1 -- 172.21.15.32:0/842154129 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a0032810 con 0x7f67a815cb00 2024-04-24T23:02:38.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.664+0000 7f67b051d640 1 -- 172.21.15.32:0/842154129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67a815cb00 msgr2=0x7f67a80ce970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:38.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.664+0000 7f67b051d640 1 --2- 172.21.15.32:0/842154129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67a815cb00 0x7f67a80ce970 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f67a80b33c0 tx=0x7f67a002a1a0 comp rx=0 tx=0).stop 2024-04-24T23:02:38.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.664+0000 7f67b051d640 1 -- 172.21.15.32:0/842154129 shutdown_connections 2024-04-24T23:02:38.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.664+0000 7f67b051d640 1 --2- 172.21.15.32:0/842154129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67a815cb00 0x7f67a80ce970 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:38.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.664+0000 7f67b051d640 1 --2- 172.21.15.32:0/842154129 >> v2:172.21.15.121:3300/0 conn(0x7f67a806cd00 0x7f67a80ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:38.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.664+0000 7f67b051d640 1 -- 172.21.15.32:0/842154129 >> 172.21.15.32:0/842154129 conn(0x7f67a806b110 msgr2=0x7f67a806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:38.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.664+0000 7f67b051d640 1 -- 172.21.15.32:0/842154129 shutdown_connections 2024-04-24T23:02:38.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.664+0000 7f67b051d640 1 -- 172.21.15.32:0/842154129 wait complete. 2024-04-24T23:02:38.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.665+0000 7f67b051d640 1 Processor -- start 2024-04-24T23:02:38.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.665+0000 7f67b051d640 1 -- start start 2024-04-24T23:02:38.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.666+0000 7f67b051d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f67a806cd00 0x7f67a8144880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:38.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.666+0000 7f67b051d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67a815b380 0x7f67a8144ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:38.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.666+0000 7f67b051d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67a815cb00 0x7f67a814d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:38.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.666+0000 7f67b051d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f67a80b55c0 con 0x7f67a815cb00 2024-04-24T23:02:38.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.666+0000 7f67b051d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f67a815b760 con 0x7f67a815b380 2024-04-24T23:02:38.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.666+0000 7f67b051d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f67a80b5740 con 0x7f67a806cd00 2024-04-24T23:02:38.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.666+0000 7f67aeaba640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67a815cb00 0x7f67a814d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:38.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.666+0000 7f67aeaba640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67a815cb00 0x7f67a814d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53754/0 (socket says 172.21.15.32:53754) 2024-04-24T23:02:38.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.666+0000 7f67aeaba640 1 -- 172.21.15.32:0/2402487447 learned_addr learned my addr 172.21.15.32:0/2402487447 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:38.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.666+0000 7f67aeaba640 1 -- 172.21.15.32:0/2402487447 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f67a806cd00 msgr2=0x7f67a8144880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:38.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.666+0000 7f67aeaba640 1 --2- 172.21.15.32:0/2402487447 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f67a806cd00 0x7f67a8144880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:38.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.666+0000 7f67adab8640 1 --2- 172.21.15.32:0/2402487447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67a815b380 0x7f67a8144ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:38.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.666+0000 7f67aeaba640 1 -- 172.21.15.32:0/2402487447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67a815b380 msgr2=0x7f67a8144ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:38.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.666+0000 7f67aeaba640 1 --2- 172.21.15.32:0/2402487447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67a815b380 0x7f67a8144ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:38.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.666+0000 7f67aeaba640 1 -- 172.21.15.32:0/2402487447 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67a8151cf0 con 0x7f67a815cb00 2024-04-24T23:02:38.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.667+0000 7f67aeaba640 1 --2- 172.21.15.32:0/2402487447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67a815cb00 0x7f67a814d6b0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f67a00090b0 tx=0x7f67a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:38.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.667+0000 7f67a6ffd640 1 -- 172.21.15.32:0/2402487447 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a0041020 con 0x7f67a815cb00 2024-04-24T23:02:38.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.667+0000 7f67a6ffd640 1 -- 172.21.15.32:0/2402487447 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67a00306c0 con 0x7f67a815cb00 2024-04-24T23:02:38.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.667+0000 7f67b051d640 1 -- 172.21.15.32:0/2402487447 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67a8150830 con 0x7f67a815cb00 2024-04-24T23:02:38.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.667+0000 7f67a6ffd640 1 -- 172.21.15.32:0/2402487447 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a0037040 con 0x7f67a815cb00 2024-04-24T23:02:38.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.667+0000 7f67b051d640 1 -- 172.21.15.32:0/2402487447 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67a8152270 con 0x7f67a815cb00 2024-04-24T23:02:38.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.668+0000 7f67b051d640 1 -- 172.21.15.32:0/2402487447 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67a8068430 con 0x7f67a815cb00 2024-04-24T23:02:38.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.668+0000 7f67a6ffd640 1 -- 172.21.15.32:0/2402487447 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f67a0044020 con 0x7f67a815cb00 2024-04-24T23:02:38.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.669+0000 7f67a6ffd640 1 --2- 172.21.15.32:0/2402487447 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f679406be60 0x7f679406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:38.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.669+0000 7f67a6ffd640 1 -- 172.21.15.32:0/2402487447 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5767+0+0 (secure 0 0 0) 0x7f67a00ab660 con 0x7f67a815cb00 2024-04-24T23:02:38.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.669+0000 7f67a6ffd640 1 -- 172.21.15.32:0/2402487447 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=73}) -- 0x7f6794073a50 con 0x7f67a815cb00 2024-04-24T23:02:38.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.669+0000 7f67ae2b9640 1 --2- 172.21.15.32:0/2402487447 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f679406be60 0x7f679406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:38.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.670+0000 7f67ae2b9640 1 --2- 172.21.15.32:0/2402487447 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f679406be60 0x7f679406e320 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f67980043c0 tx=0x7f6798007d30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:38.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.673+0000 7f67a6ffd640 1 -- 172.21.15.32:0/2402487447 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f67a004b030 con 0x7f67a815cb00 2024-04-24T23:02:38.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.920+0000 7f67b051d640 1 -- 172.21.15.32:0/2402487447 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f67a80afd60 con 0x7f67a815cb00 2024-04-24T23:02:38.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.921+0000 7f67a6ffd640 1 -- 172.21.15.32:0/2402487447 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+16823 (secure 0 0 0) 0x7f67a0079fd0 con 0x7f67a815cb00 2024-04-24T23:02:38.922 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:02:38.923 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":72,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:01:52.865926+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:01:23.328944+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-24T23:01:24.010365+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["backfillfull","exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","full","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","full","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.a","mappings":[{"from":6,"to":4}]},{"pgid":"9.3a","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:02:38.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.923+0000 7f67b051d640 1 -- 172.21.15.32:0/2402487447 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f679406be60 msgr2=0x7f679406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:38.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.923+0000 7f67b051d640 1 --2- 172.21.15.32:0/2402487447 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f679406be60 0x7f679406e320 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f67980043c0 tx=0x7f6798007d30 comp rx=0 tx=0).stop 2024-04-24T23:02:38.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.923+0000 7f67b051d640 1 -- 172.21.15.32:0/2402487447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67a815cb00 msgr2=0x7f67a814d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:38.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.923+0000 7f67b051d640 1 --2- 172.21.15.32:0/2402487447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67a815cb00 0x7f67a814d6b0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f67a00090b0 tx=0x7f67a00046c0 comp rx=0 tx=0).stop 2024-04-24T23:02:38.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.923+0000 7f67b051d640 1 -- 172.21.15.32:0/2402487447 shutdown_connections 2024-04-24T23:02:38.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.923+0000 7f67b051d640 1 --2- 172.21.15.32:0/2402487447 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f679406be60 0x7f679406e320 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:38.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.923+0000 7f67b051d640 1 --2- 172.21.15.32:0/2402487447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67a815cb00 0x7f67a814d6b0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:38.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.923+0000 7f67b051d640 1 --2- 172.21.15.32:0/2402487447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67a815b380 0x7f67a8144ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:38.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.923+0000 7f67b051d640 1 --2- 172.21.15.32:0/2402487447 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f67a806cd00 0x7f67a8144880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:38.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.923+0000 7f67b051d640 1 -- 172.21.15.32:0/2402487447 >> 172.21.15.32:0/2402487447 conn(0x7f67a806b110 msgr2=0x7f67a815bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:38.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.923+0000 7f67b051d640 1 -- 172.21.15.32:0/2402487447 shutdown_connections 2024-04-24T23:02:38.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:38.923+0000 7f67b051d640 1 -- 172.21.15.32:0/2402487447 wait complete. 2024-04-24T23:02:38.936 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:01:23.328944+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:01:24.010365+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:02:38.936 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:02:39.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.039+0000 7f9a4611b640 1 Processor -- start 2024-04-24T23:02:39.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.039+0000 7f9a4611b640 1 -- start start 2024-04-24T23:02:39.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.040+0000 7f9a4611b640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9a4006c5f0 0x7f9a4006c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:39.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.040+0000 7f9a4611b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9a400af890 0x7f9a4015cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:39.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.040+0000 7f9a4611b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a400b6430 con 0x7f9a400b44d0 2024-04-24T23:02:39.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.040+0000 7f9a4611b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f9a40156f80 con 0x7f9a400af890 2024-04-24T23:02:39.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.040+0000 7f9a4611b640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f9a400b6150 con 0x7f9a4006c5f0 2024-04-24T23:02:39.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.040+0000 7f9a3ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9a400af890 0x7f9a4015cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:39.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.040+0000 7f9a3effd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9a400b44d0 0x7f9a400b5890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49010/0 (socket says 172.21.15.32:49010) 2024-04-24T23:02:39.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.040+0000 7f9a3effd640 1 -- 172.21.15.32:0/3598753146 learned_addr learned my addr 172.21.15.32:0/3598753146 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:02:39.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.040+0000 7f9a3f7fe640 1 --2- 172.21.15.32:0/3598753146 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9a4006c5f0 0x7f9a4006c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:39.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.040+0000 7f9a3ffff640 1 -- 172.21.15.32:0/3598753146 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9a4006c5f0 msgr2=0x7f9a4006c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:39.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.040+0000 7f9a3ffff640 1 --2- 172.21.15.32:0/3598753146 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9a4006c5f0 0x7f9a4006c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:39.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.040+0000 7f9a3ffff640 1 -- 172.21.15.32:0/3598753146 >> v1:172.21.15.32:6789/0 conn(0x7f9a400b44d0 legacy=0x7f9a400b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:39.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.040+0000 7f9a3ffff640 1 -- 172.21.15.32:0/3598753146 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a400b39f0 con 0x7f9a400af890 2024-04-24T23:02:39.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.040+0000 7f9a3e7fc640 1 -- 172.21.15.32:0/3598753146 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1652047278 0 0) 0x7f9a400b6430 con 0x7f9a400b44d0 2024-04-24T23:02:39.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.040+0000 7f9a3f7fe640 1 --2- 172.21.15.32:0/3598753146 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9a4006c5f0 0x7f9a4006c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:02:39.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.041+0000 7f9a3ffff640 1 --2- 172.21.15.32:0/3598753146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a400af890 0x7f9a4015cac0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f9a38009d90 tx=0x7f9a3802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b65a2e18dce653 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:39.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.041+0000 7f9a3e7fc640 1 -- 172.21.15.32:0/3598753146 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a38031600 con 0x7f9a400af890 2024-04-24T23:02:39.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.041+0000 7f9a3e7fc640 1 -- 172.21.15.32:0/3598753146 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a380317a0 con 0x7f9a400af890 2024-04-24T23:02:39.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.041+0000 7f9a3e7fc640 1 -- 172.21.15.32:0/3598753146 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a38031b70 con 0x7f9a400af890 2024-04-24T23:02:39.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.042+0000 7f9a4611b640 1 -- 172.21.15.32:0/3598753146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a400af890 msgr2=0x7f9a4015cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:39.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.042+0000 7f9a4611b640 1 --2- 172.21.15.32:0/3598753146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a400af890 0x7f9a4015cac0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f9a38009d90 tx=0x7f9a3802a5d0 comp rx=0 tx=0).stop 2024-04-24T23:02:39.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.042+0000 7f9a4611b640 1 -- 172.21.15.32:0/3598753146 shutdown_connections 2024-04-24T23:02:39.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.042+0000 7f9a4611b640 1 --2- 172.21.15.32:0/3598753146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a400af890 0x7f9a4015cac0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:39.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.042+0000 7f9a4611b640 1 --2- 172.21.15.32:0/3598753146 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9a4006c5f0 0x7f9a4006c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:39.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.042+0000 7f9a4611b640 1 -- 172.21.15.32:0/3598753146 >> 172.21.15.32:0/3598753146 conn(0x7f9a4006b110 msgr2=0x7f9a4006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:39.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.042+0000 7f9a4611b640 1 -- 172.21.15.32:0/3598753146 shutdown_connections 2024-04-24T23:02:39.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.042+0000 7f9a4611b640 1 -- 172.21.15.32:0/3598753146 wait complete. 2024-04-24T23:02:39.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.043+0000 7f9a4611b640 1 Processor -- start 2024-04-24T23:02:39.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.043+0000 7f9a4611b640 1 -- start start 2024-04-24T23:02:39.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.044+0000 7f9a4611b640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9a4006c5f0 0x7f9a40142b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:39.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.044+0000 7f9a4611b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a400af890 0x7f9a400b0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:39.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.044+0000 7f9a4611b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a400b44d0 0x7f9a400b1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:39.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.044+0000 7f9a4611b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4006cf10 con 0x7f9a400af890 2024-04-24T23:02:39.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.044+0000 7f9a4611b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9a400b3fb0 con 0x7f9a400b44d0 2024-04-24T23:02:39.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.044+0000 7f9a4611b640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f9a4015d000 con 0x7f9a4006c5f0 2024-04-24T23:02:39.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.044+0000 7f9a3effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a400af890 0x7f9a400b0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:39.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.044+0000 7f9a3effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a400af890 0x7f9a400b0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53784/0 (socket says 172.21.15.32:53784) 2024-04-24T23:02:39.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.044+0000 7f9a3effd640 1 -- 172.21.15.32:0/1825549047 learned_addr learned my addr 172.21.15.32:0/1825549047 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:39.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.044+0000 7f9a3ffff640 1 --2- 172.21.15.32:0/1825549047 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a400b44d0 0x7f9a400b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:39.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.044+0000 7f9a3f7fe640 1 --2- 172.21.15.32:0/1825549047 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9a4006c5f0 0x7f9a40142b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:39.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.044+0000 7f9a3effd640 1 -- 172.21.15.32:0/1825549047 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9a4006c5f0 msgr2=0x7f9a40142b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:39.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.044+0000 7f9a3effd640 1 --2- 172.21.15.32:0/1825549047 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9a4006c5f0 0x7f9a40142b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:39.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.044+0000 7f9a3effd640 1 -- 172.21.15.32:0/1825549047 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a400b44d0 msgr2=0x7f9a400b1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:39.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.044+0000 7f9a3effd640 1 --2- 172.21.15.32:0/1825549047 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a400b44d0 0x7f9a400b1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:39.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.044+0000 7f9a3effd640 1 -- 172.21.15.32:0/1825549047 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a40158070 con 0x7f9a400af890 2024-04-24T23:02:39.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.044+0000 7f9a3effd640 1 --2- 172.21.15.32:0/1825549047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a400af890 0x7f9a400b0ed0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f9a3400c430 tx=0x7f9a3400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:39.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.045+0000 7f9a27fff640 1 -- 172.21.15.32:0/1825549047 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a340074e0 con 0x7f9a400af890 2024-04-24T23:02:39.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.045+0000 7f9a27fff640 1 -- 172.21.15.32:0/1825549047 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a340047e0 con 0x7f9a400af890 2024-04-24T23:02:39.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.045+0000 7f9a4611b640 1 -- 172.21.15.32:0/1825549047 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a40150990 con 0x7f9a400af890 2024-04-24T23:02:39.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.045+0000 7f9a27fff640 1 -- 172.21.15.32:0/1825549047 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a34004bd0 con 0x7f9a400af890 2024-04-24T23:02:39.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.045+0000 7f9a4611b640 1 -- 172.21.15.32:0/1825549047 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a40151230 con 0x7f9a400af890 2024-04-24T23:02:39.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.046+0000 7f9a4611b640 1 -- 172.21.15.32:0/1825549047 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a40150d80 con 0x7f9a400af890 2024-04-24T23:02:39.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.046+0000 7f9a27fff640 1 -- 172.21.15.32:0/1825549047 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9a34011070 con 0x7f9a400af890 2024-04-24T23:02:39.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.047+0000 7f9a27fff640 1 --2- 172.21.15.32:0/1825549047 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9a2c06be30 0x7f9a2c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:39.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.047+0000 7f9a27fff640 1 -- 172.21.15.32:0/1825549047 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5767+0+0 (secure 0 0 0) 0x7f9a3408a8f0 con 0x7f9a400af890 2024-04-24T23:02:39.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.047+0000 7f9a3f7fe640 1 --2- 172.21.15.32:0/1825549047 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9a2c06be30 0x7f9a2c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:39.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.047+0000 7f9a27fff640 1 -- 172.21.15.32:0/1825549047 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=73}) -- 0x7f9a2c073a20 con 0x7f9a400af890 2024-04-24T23:02:39.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.047+0000 7f9a3f7fe640 1 --2- 172.21.15.32:0/1825549047 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9a2c06be30 0x7f9a2c06e2f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9a3000ab60 tx=0x7f9a30008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:39.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.051+0000 7f9a27fff640 1 -- 172.21.15.32:0/1825549047 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9a34058ca0 con 0x7f9a400af890 2024-04-24T23:02:39.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.233+0000 7f9a4611b640 1 -- 172.21.15.32:0/1825549047 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9a40068430 con 0x7f9a400af890 2024-04-24T23:02:39.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.234+0000 7f9a27fff640 1 -- 172.21.15.32:0/1825549047 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) ==== 93+0+10 (secure 0 0 0) 0x7f9a3405cfa0 con 0x7f9a400af890 2024-04-24T23:02:39.235 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T23:02:39.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.236+0000 7f9a4611b640 1 -- 172.21.15.32:0/1825549047 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9a2c06be30 msgr2=0x7f9a2c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:39.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.236+0000 7f9a4611b640 1 --2- 172.21.15.32:0/1825549047 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9a2c06be30 0x7f9a2c06e2f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9a3000ab60 tx=0x7f9a30008040 comp rx=0 tx=0).stop 2024-04-24T23:02:39.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.236+0000 7f9a4611b640 1 -- 172.21.15.32:0/1825549047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a400af890 msgr2=0x7f9a400b0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:39.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.236+0000 7f9a4611b640 1 --2- 172.21.15.32:0/1825549047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a400af890 0x7f9a400b0ed0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f9a3400c430 tx=0x7f9a3400c8f0 comp rx=0 tx=0).stop 2024-04-24T23:02:39.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.236+0000 7f9a4611b640 1 -- 172.21.15.32:0/1825549047 shutdown_connections 2024-04-24T23:02:39.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.236+0000 7f9a4611b640 1 --2- 172.21.15.32:0/1825549047 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9a2c06be30 0x7f9a2c06e2f0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:39.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.236+0000 7f9a4611b640 1 --2- 172.21.15.32:0/1825549047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a400af890 0x7f9a400b0ed0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:39.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.236+0000 7f9a4611b640 1 --2- 172.21.15.32:0/1825549047 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a400b44d0 0x7f9a400b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:39.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.236+0000 7f9a4611b640 1 --2- 172.21.15.32:0/1825549047 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9a4006c5f0 0x7f9a40142b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:39.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.236+0000 7f9a4611b640 1 -- 172.21.15.32:0/1825549047 >> 172.21.15.32:0/1825549047 conn(0x7f9a4006b110 msgr2=0x7f9a4006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:39.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.236+0000 7f9a4611b640 1 -- 172.21.15.32:0/1825549047 shutdown_connections 2024-04-24T23:02:39.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.236+0000 7f9a4611b640 1 -- 172.21.15.32:0/1825549047 wait complete. 2024-04-24T23:02:39.248 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:02:39.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.351+0000 7f5819abc640 1 Processor -- start 2024-04-24T23:02:39.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.351+0000 7f5819abc640 1 -- start start 2024-04-24T23:02:39.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.351+0000 7f5819abc640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f581415e9b0 0x7f58140d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:39.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.351+0000 7f5819abc640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58140d3170 con 0x7f58140d4210 2024-04-24T23:02:39.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.351+0000 7f5819abc640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58140d3450 con 0x7f581406b820 2024-04-24T23:02:39.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.351+0000 7f5819abc640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f58140d3730 con 0x7f581415e9b0 2024-04-24T23:02:39.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.352+0000 7f5812ffd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f58140d4210 0x7f581406b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49016/0 (socket says 172.21.15.32:49016) 2024-04-24T23:02:39.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.352+0000 7f5812ffd640 1 -- 172.21.15.32:0/1694836424 learned_addr learned my addr 172.21.15.32:0/1694836424 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:02:39.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.352+0000 7f58137fe640 1 --2- 172.21.15.32:0/1694836424 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f581415e9b0 0x7f58140d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:39.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.352+0000 7f58127fc640 1 -- 172.21.15.32:0/1694836424 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4049495289 0 0) 0x7f58140d3170 con 0x7f58140d4210 2024-04-24T23:02:39.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.352+0000 7f58127fc640 1 -- 172.21.15.32:0/1694836424 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5800003520 con 0x7f58140d4210 2024-04-24T23:02:39.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.352+0000 7f58127fc640 1 -- 172.21.15.32:0/1694836424 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 744116036 0 0) 0x7f58140d3450 con 0x7f581406b820 2024-04-24T23:02:39.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.352+0000 7f58137fe640 1 -- 172.21.15.32:0/1694836424 >> v1:172.21.15.121:6789/0 conn(0x7f581406b820 legacy=0x7f58140b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:39.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.352+0000 7f58137fe640 1 -- 172.21.15.32:0/1694836424 >> v1:172.21.15.32:6789/0 conn(0x7f58140d4210 legacy=0x7f581406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:39.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.352+0000 7f58137fe640 1 -- 172.21.15.32:0/1694836424 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58140d3a10 con 0x7f581415e9b0 2024-04-24T23:02:39.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.353+0000 7f58137fe640 1 --2- 172.21.15.32:0/1694836424 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f581415e9b0 0x7f58140d2b90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f57fc0097d0 tx=0x7f57fc02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=656e301e347af2f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:39.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.353+0000 7f58127fc640 1 -- 172.21.15.32:0/1694836424 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc037070 con 0x7f581415e9b0 2024-04-24T23:02:39.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.353+0000 7f58127fc640 1 -- 172.21.15.32:0/1694836424 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57fc032440 con 0x7f581415e9b0 2024-04-24T23:02:39.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.353+0000 7f58127fc640 1 -- 172.21.15.32:0/1694836424 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc0327f0 con 0x7f581415e9b0 2024-04-24T23:02:39.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.353+0000 7f5819abc640 1 -- 172.21.15.32:0/1694836424 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f581415e9b0 msgr2=0x7f58140d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:39.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.353+0000 7f5819abc640 1 --2- 172.21.15.32:0/1694836424 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f581415e9b0 0x7f58140d2b90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f57fc0097d0 tx=0x7f57fc02a210 comp rx=0 tx=0).stop 2024-04-24T23:02:39.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.353+0000 7f5819abc640 1 -- 172.21.15.32:0/1694836424 shutdown_connections 2024-04-24T23:02:39.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.353+0000 7f5819abc640 1 --2- 172.21.15.32:0/1694836424 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f581415e9b0 0x7f58140d2b90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:39.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.353+0000 7f5819abc640 1 -- 172.21.15.32:0/1694836424 >> 172.21.15.32:0/1694836424 conn(0x7f58140d11d0 msgr2=0x7f58140d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:39.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.353+0000 7f5819abc640 1 -- 172.21.15.32:0/1694836424 shutdown_connections 2024-04-24T23:02:39.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.353+0000 7f5819abc640 1 -- 172.21.15.32:0/1694836424 wait complete. 2024-04-24T23:02:39.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.355+0000 7f5819abc640 1 Processor -- start 2024-04-24T23:02:39.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.355+0000 7f5819abc640 1 -- start start 2024-04-24T23:02:39.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.355+0000 7f5819abc640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f581406b820 0x7f581415aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:39.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.355+0000 7f5819abc640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58140d4210 0x7f58140aba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:39.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.355+0000 7f5819abc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f581415e9b0 0x7f5814160930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:39.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.355+0000 7f5819abc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f58140ba0e0 con 0x7f581415e9b0 2024-04-24T23:02:39.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.355+0000 7f5819abc640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f58140d2400 con 0x7f58140d4210 2024-04-24T23:02:39.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.355+0000 7f5819abc640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f58140ba260 con 0x7f581406b820 2024-04-24T23:02:39.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.356+0000 7f5813fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f581415e9b0 0x7f5814160930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:39.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.356+0000 7f5813fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f581415e9b0 0x7f5814160930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53794/0 (socket says 172.21.15.32:53794) 2024-04-24T23:02:39.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.356+0000 7f5813fff640 1 -- 172.21.15.32:0/2008242294 learned_addr learned my addr 172.21.15.32:0/2008242294 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:39.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.356+0000 7f58137fe640 1 --2- 172.21.15.32:0/2008242294 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f581406b820 0x7f581415aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:39.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.356+0000 7f5812ffd640 1 --2- 172.21.15.32:0/2008242294 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58140d4210 0x7f58140aba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:39.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.356+0000 7f5813fff640 1 -- 172.21.15.32:0/2008242294 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f581406b820 msgr2=0x7f581415aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:39.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.356+0000 7f5813fff640 1 --2- 172.21.15.32:0/2008242294 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f581406b820 0x7f581415aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:39.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.356+0000 7f5813fff640 1 -- 172.21.15.32:0/2008242294 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58140d4210 msgr2=0x7f58140aba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:39.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.356+0000 7f5813fff640 1 --2- 172.21.15.32:0/2008242294 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58140d4210 0x7f58140aba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:39.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.356+0000 7f5813fff640 1 -- 172.21.15.32:0/2008242294 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f581415b350 con 0x7f581415e9b0 2024-04-24T23:02:39.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.356+0000 7f5812ffd640 1 --2- 172.21.15.32:0/2008242294 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58140d4210 0x7f58140aba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:02:39.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.356+0000 7f5813fff640 1 --2- 172.21.15.32:0/2008242294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f581415e9b0 0x7f5814160930 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f580c00c780 tx=0x7f580c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:39.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.356+0000 7f580bfff640 1 -- 172.21.15.32:0/2008242294 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580c003b70 con 0x7f581415e9b0 2024-04-24T23:02:39.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.356+0000 7f580bfff640 1 -- 172.21.15.32:0/2008242294 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f580c003d30 con 0x7f581415e9b0 2024-04-24T23:02:39.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.356+0000 7f5819abc640 1 -- 172.21.15.32:0/2008242294 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58140cd2e0 con 0x7f581415e9b0 2024-04-24T23:02:39.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.356+0000 7f580bfff640 1 -- 172.21.15.32:0/2008242294 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580c0052b0 con 0x7f581415e9b0 2024-04-24T23:02:39.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.356+0000 7f5819abc640 1 -- 172.21.15.32:0/2008242294 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58140cd8a0 con 0x7f581415e9b0 2024-04-24T23:02:39.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.357+0000 7f5819abc640 1 -- 172.21.15.32:0/2008242294 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5814068430 con 0x7f581415e9b0 2024-04-24T23:02:39.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.358+0000 7f580bfff640 1 -- 172.21.15.32:0/2008242294 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f580c005450 con 0x7f581415e9b0 2024-04-24T23:02:39.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.358+0000 7f580bfff640 1 --2- 172.21.15.32:0/2008242294 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f57f406be60 0x7f57f406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:39.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.358+0000 7f580bfff640 1 -- 172.21.15.32:0/2008242294 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5767+0+0 (secure 0 0 0) 0x7f580c08ab60 con 0x7f581415e9b0 2024-04-24T23:02:39.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.358+0000 7f580bfff640 1 -- 172.21.15.32:0/2008242294 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=73}) -- 0x7f57f4073a50 con 0x7f581415e9b0 2024-04-24T23:02:39.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.358+0000 7f58137fe640 1 --2- 172.21.15.32:0/2008242294 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f57f406be60 0x7f57f406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:39.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.359+0000 7f58137fe640 1 --2- 172.21.15.32:0/2008242294 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f57f406be60 0x7f57f406e320 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f57fc034000 tx=0x7f57fc02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:39.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.362+0000 7f580bfff640 1 -- 172.21.15.32:0/2008242294 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f580c059560 con 0x7f581415e9b0 2024-04-24T23:02:39.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.543+0000 7f5819abc640 1 -- 172.21.15.32:0/2008242294 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f581415d140 con 0x7f581415e9b0 2024-04-24T23:02:39.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.544+0000 7f580bfff640 1 -- 172.21.15.32:0/2008242294 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v72) ==== 104+0+11 (secure 0 0 0) 0x7f580c058e60 con 0x7f581415e9b0 2024-04-24T23:02:39.545 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-24T23:02:39.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.545+0000 7f5819abc640 1 -- 172.21.15.32:0/2008242294 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f57f406be60 msgr2=0x7f57f406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:39.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.545+0000 7f5819abc640 1 --2- 172.21.15.32:0/2008242294 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f57f406be60 0x7f57f406e320 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f57fc034000 tx=0x7f57fc02e000 comp rx=0 tx=0).stop 2024-04-24T23:02:39.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.545+0000 7f5819abc640 1 -- 172.21.15.32:0/2008242294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f581415e9b0 msgr2=0x7f5814160930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:39.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.545+0000 7f5819abc640 1 --2- 172.21.15.32:0/2008242294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f581415e9b0 0x7f5814160930 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f580c00c780 tx=0x7f580c00cc40 comp rx=0 tx=0).stop 2024-04-24T23:02:39.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.545+0000 7f5819abc640 1 -- 172.21.15.32:0/2008242294 shutdown_connections 2024-04-24T23:02:39.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.545+0000 7f5819abc640 1 --2- 172.21.15.32:0/2008242294 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f57f406be60 0x7f57f406e320 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:39.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.545+0000 7f5819abc640 1 --2- 172.21.15.32:0/2008242294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f581415e9b0 0x7f5814160930 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:39.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.545+0000 7f5819abc640 1 --2- 172.21.15.32:0/2008242294 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58140d4210 0x7f58140aba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:39.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.545+0000 7f5819abc640 1 --2- 172.21.15.32:0/2008242294 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f581406b820 0x7f581415aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:39.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.545+0000 7f5819abc640 1 -- 172.21.15.32:0/2008242294 >> 172.21.15.32:0/2008242294 conn(0x7f58140d11d0 msgr2=0x7f581415d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:39.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.545+0000 7f5819abc640 1 -- 172.21.15.32:0/2008242294 shutdown_connections 2024-04-24T23:02:39.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.545+0000 7f5819abc640 1 -- 172.21.15.32:0/2008242294 wait complete. 2024-04-24T23:02:39.558 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:02:39.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.660+0000 7fcbfdc12640 1 Processor -- start 2024-04-24T23:02:39.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.660+0000 7fcbfdc12640 1 -- start start 2024-04-24T23:02:39.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.660+0000 7fcbfdc12640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcbf8162320 0x7fcbf80ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:39.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.660+0000 7fcbfdc12640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fcbf80b5af0 0x7fcbf8162700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:39.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.660+0000 7fcbfdc12640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbf80cd270 con 0x7fcbf8162c40 2024-04-24T23:02:39.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.660+0000 7fcbfdc12640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fcbf80b5660 con 0x7fcbf8162320 2024-04-24T23:02:39.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.660+0000 7fcbfdc12640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fcbf8071610 con 0x7fcbf80b5af0 2024-04-24T23:02:39.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.661+0000 7fcbf77fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcbf8162320 0x7fcbf80ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:39.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.661+0000 7fcbf77fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcbf8162320 0x7fcbf80ccc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53810/0 (socket says 172.21.15.32:53810) 2024-04-24T23:02:39.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.661+0000 7fcbf77fe640 1 -- 172.21.15.32:0/1030759812 learned_addr learned my addr 172.21.15.32:0/1030759812 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:39.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.661+0000 7fcbf6ffd640 1 --2- 172.21.15.32:0/1030759812 >> v2:172.21.15.121:3300/0 conn(0x7fcbf80b5af0 0x7fcbf8162700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:39.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.661+0000 7fcbf77fe640 1 -- 172.21.15.32:0/1030759812 >> v2:172.21.15.121:3300/0 conn(0x7fcbf80b5af0 msgr2=0x7fcbf8162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:39.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.661+0000 7fcbf77fe640 1 --2- 172.21.15.32:0/1030759812 >> v2:172.21.15.121:3300/0 conn(0x7fcbf80b5af0 0x7fcbf8162700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:39.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.661+0000 7fcbf77fe640 1 -- 172.21.15.32:0/1030759812 >> v1:172.21.15.121:6789/0 conn(0x7fcbf8162c40 legacy=0x7fcbf80ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:39.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.661+0000 7fcbf77fe640 1 -- 172.21.15.32:0/1030759812 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbf80718f0 con 0x7fcbf8162320 2024-04-24T23:02:39.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.661+0000 7fcbf6ffd640 1 --2- 172.21.15.32:0/1030759812 >> v2:172.21.15.121:3300/0 conn(0x7fcbf80b5af0 0x7fcbf8162700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:02:39.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.662+0000 7fcbf77fe640 1 --2- 172.21.15.32:0/1030759812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbf8162320 0x7fcbf80ccc90 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fcbf80d1bc0 tx=0x7fcbe802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5d550b437f97ffa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:39.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.662+0000 7fcbf67fc640 1 -- 172.21.15.32:0/1030759812 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe8037070 con 0x7fcbf8162320 2024-04-24T23:02:39.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.662+0000 7fcbf67fc640 1 -- 172.21.15.32:0/1030759812 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbe8032440 con 0x7fcbf8162320 2024-04-24T23:02:39.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.662+0000 7fcbf67fc640 1 -- 172.21.15.32:0/1030759812 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe8032810 con 0x7fcbf8162320 2024-04-24T23:02:39.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.662+0000 7fcbfdc12640 1 -- 172.21.15.32:0/1030759812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbf8162320 msgr2=0x7fcbf80ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:39.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.662+0000 7fcbfdc12640 1 --2- 172.21.15.32:0/1030759812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbf8162320 0x7fcbf80ccc90 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fcbf80d1bc0 tx=0x7fcbe802a1e0 comp rx=0 tx=0).stop 2024-04-24T23:02:39.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.662+0000 7fcbfdc12640 1 -- 172.21.15.32:0/1030759812 shutdown_connections 2024-04-24T23:02:39.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.662+0000 7fcbfdc12640 1 --2- 172.21.15.32:0/1030759812 >> v2:172.21.15.121:3300/0 conn(0x7fcbf80b5af0 0x7fcbf8162700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:39.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.662+0000 7fcbfdc12640 1 --2- 172.21.15.32:0/1030759812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbf8162320 0x7fcbf80ccc90 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:39.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.662+0000 7fcbfdc12640 1 -- 172.21.15.32:0/1030759812 >> 172.21.15.32:0/1030759812 conn(0x7fcbf80d0ae0 msgr2=0x7fcbf80d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:39.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.662+0000 7fcbfdc12640 1 -- 172.21.15.32:0/1030759812 shutdown_connections 2024-04-24T23:02:39.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.662+0000 7fcbfdc12640 1 -- 172.21.15.32:0/1030759812 wait complete. 2024-04-24T23:02:39.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.664+0000 7fcbfdc12640 1 Processor -- start 2024-04-24T23:02:39.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.664+0000 7fcbfdc12640 1 -- start start 2024-04-24T23:02:39.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.664+0000 7fcbfdc12640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbf80b5af0 0x7fcbf806f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:39.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.664+0000 7fcbfdc12640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcbf8162320 0x7fcbf81e8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:39.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.664+0000 7fcbfdc12640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcbf8162c40 0x7fcbf80b0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:39.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.664+0000 7fcbfdc12640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf8163070 con 0x7fcbf80b5af0 2024-04-24T23:02:39.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.664+0000 7fcbfdc12640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf8067950 con 0x7fcbf8162c40 2024-04-24T23:02:39.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.664+0000 7fcbfdc12640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fcbf8071bd0 con 0x7fcbf8162320 2024-04-24T23:02:39.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.665+0000 7fcbf77fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbf80b5af0 0x7fcbf806f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:39.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.665+0000 7fcbf77fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbf80b5af0 0x7fcbf806f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53820/0 (socket says 172.21.15.32:53820) 2024-04-24T23:02:39.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.665+0000 7fcbf77fe640 1 -- 172.21.15.32:0/635437531 learned_addr learned my addr 172.21.15.32:0/635437531 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:39.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.665+0000 7fcbf6ffd640 1 --2- 172.21.15.32:0/635437531 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcbf8162320 0x7fcbf81e8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:39.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.665+0000 7fcbf7fff640 1 --2- 172.21.15.32:0/635437531 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcbf8162c40 0x7fcbf80b0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:39.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.665+0000 7fcbf77fe640 1 -- 172.21.15.32:0/635437531 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcbf8162320 msgr2=0x7fcbf81e8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:39.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.665+0000 7fcbf77fe640 1 --2- 172.21.15.32:0/635437531 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcbf8162320 0x7fcbf81e8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:39.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.665+0000 7fcbf77fe640 1 -- 172.21.15.32:0/635437531 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcbf8162c40 msgr2=0x7fcbf80b0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:39.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.665+0000 7fcbf77fe640 1 --2- 172.21.15.32:0/635437531 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcbf8162c40 0x7fcbf80b0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:39.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.665+0000 7fcbf77fe640 1 -- 172.21.15.32:0/635437531 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbf81e88a0 con 0x7fcbf80b5af0 2024-04-24T23:02:39.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.665+0000 7fcbf7fff640 1 --2- 172.21.15.32:0/635437531 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcbf8162c40 0x7fcbf80b0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:02:39.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.665+0000 7fcbf77fe640 1 --2- 172.21.15.32:0/635437531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbf80b5af0 0x7fcbf806f310 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fcbe802a220 tx=0x7fcbe80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:39.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.665+0000 7fcbdffff640 1 -- 172.21.15.32:0/635437531 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe8041020 con 0x7fcbf80b5af0 2024-04-24T23:02:39.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.665+0000 7fcbdffff640 1 -- 172.21.15.32:0/635437531 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbe80306c0 con 0x7fcbf80b5af0 2024-04-24T23:02:39.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.665+0000 7fcbfdc12640 1 -- 172.21.15.32:0/635437531 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbf81e8b80 con 0x7fcbf80b5af0 2024-04-24T23:02:39.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.665+0000 7fcbdffff640 1 -- 172.21.15.32:0/635437531 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe8037040 con 0x7fcbf80b5af0 2024-04-24T23:02:39.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.665+0000 7fcbfdc12640 1 -- 172.21.15.32:0/635437531 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbf8147f00 con 0x7fcbf80b5af0 2024-04-24T23:02:39.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.666+0000 7fcbfdc12640 1 -- 172.21.15.32:0/635437531 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbf8068430 con 0x7fcbf80b5af0 2024-04-24T23:02:39.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.667+0000 7fcbdffff640 1 -- 172.21.15.32:0/635437531 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcbe8044020 con 0x7fcbf80b5af0 2024-04-24T23:02:39.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.667+0000 7fcbdffff640 1 --2- 172.21.15.32:0/635437531 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcbe406be30 0x7fcbe406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:39.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.668+0000 7fcbdffff640 1 -- 172.21.15.32:0/635437531 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5767+0+0 (secure 0 0 0) 0x7fcbe80ab650 con 0x7fcbf80b5af0 2024-04-24T23:02:39.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.668+0000 7fcbf6ffd640 1 --2- 172.21.15.32:0/635437531 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcbe406be30 0x7fcbe406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:39.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.668+0000 7fcbdffff640 1 -- 172.21.15.32:0/635437531 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=73}) -- 0x7fcbe4073a20 con 0x7fcbf80b5af0 2024-04-24T23:02:39.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.668+0000 7fcbf6ffd640 1 --2- 172.21.15.32:0/635437531 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcbe406be30 0x7fcbe406e2f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fcbf806eef0 tx=0x7fcbec008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:39.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.672+0000 7fcbdffff640 1 -- 172.21.15.32:0/635437531 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcbe804b030 con 0x7fcbf80b5af0 2024-04-24T23:02:39.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.853+0000 7fcbfdc12640 1 -- 172.21.15.32:0/635437531 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcbf81e9320 con 0x7fcbf80b5af0 2024-04-24T23:02:39.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.853+0000 7fcbdffff640 1 -- 172.21.15.32:0/635437531 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v72) ==== 100+0+11 (secure 0 0 0) 0x7fcbe807a050 con 0x7fcbf80b5af0 2024-04-24T23:02:39.855 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-24T23:02:39.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.855+0000 7fcbfdc12640 1 -- 172.21.15.32:0/635437531 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcbe406be30 msgr2=0x7fcbe406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:39.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.855+0000 7fcbfdc12640 1 --2- 172.21.15.32:0/635437531 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcbe406be30 0x7fcbe406e2f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fcbf806eef0 tx=0x7fcbec008040 comp rx=0 tx=0).stop 2024-04-24T23:02:39.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.855+0000 7fcbfdc12640 1 -- 172.21.15.32:0/635437531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbf80b5af0 msgr2=0x7fcbf806f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:39.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.855+0000 7fcbfdc12640 1 --2- 172.21.15.32:0/635437531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbf80b5af0 0x7fcbf806f310 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fcbe802a220 tx=0x7fcbe80046c0 comp rx=0 tx=0).stop 2024-04-24T23:02:39.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.855+0000 7fcbfdc12640 1 -- 172.21.15.32:0/635437531 shutdown_connections 2024-04-24T23:02:39.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.855+0000 7fcbfdc12640 1 --2- 172.21.15.32:0/635437531 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcbe406be30 0x7fcbe406e2f0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:39.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.855+0000 7fcbfdc12640 1 --2- 172.21.15.32:0/635437531 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcbf8162c40 0x7fcbf80b0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:39.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.855+0000 7fcbfdc12640 1 --2- 172.21.15.32:0/635437531 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcbf8162320 0x7fcbf81e8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:39.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.855+0000 7fcbfdc12640 1 --2- 172.21.15.32:0/635437531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbf80b5af0 0x7fcbf806f310 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:39.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.855+0000 7fcbfdc12640 1 -- 172.21.15.32:0/635437531 >> 172.21.15.32:0/635437531 conn(0x7fcbf80d0ae0 msgr2=0x7fcbf80b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:39.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.855+0000 7fcbfdc12640 1 -- 172.21.15.32:0/635437531 shutdown_connections 2024-04-24T23:02:39.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.855+0000 7fcbfdc12640 1 -- 172.21.15.32:0/635437531 wait complete. 2024-04-24T23:02:39.867 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T23:02:39.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.969+0000 7fdac9ce1640 1 Processor -- start 2024-04-24T23:02:39.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.970+0000 7fdac9ce1640 1 -- start start 2024-04-24T23:02:39.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.970+0000 7fdac9ce1640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdac415e9b0 0x7fdac40d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:39.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.970+0000 7fdac9ce1640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdac40d4210 0x7fdac406b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:39.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.970+0000 7fdac9ce1640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fdac40ba0e0 0x7fdac406b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:39.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.970+0000 7fdac9ce1640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fdac40d3170 con 0x7fdac415e9b0 2024-04-24T23:02:39.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.970+0000 7fdac9ce1640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fdac40d3450 con 0x7fdac40ba0e0 2024-04-24T23:02:39.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.970+0000 7fdac9ce1640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fdac40d3730 con 0x7fdac40d4210 2024-04-24T23:02:39.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.970+0000 7fdac37fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdac415e9b0 0x7fdac40d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:39.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.970+0000 7fdac37fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdac415e9b0 0x7fdac40d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41826/0 (socket says 172.21.15.32:41826) 2024-04-24T23:02:39.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.970+0000 7fdac37fe640 1 -- 172.21.15.32:0/798128808 learned_addr learned my addr 172.21.15.32:0/798128808 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:39.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.971+0000 7fdac2ffd640 1 --2- 172.21.15.32:0/798128808 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdac40d4210 0x7fdac406b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:39.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.971+0000 7fdac3fff640 1 --2- 172.21.15.32:0/798128808 >> v2:172.21.15.121:3300/0 conn(0x7fdac40ba0e0 0x7fdac406b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:39.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.971+0000 7fdac37fe640 1 -- 172.21.15.32:0/798128808 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdac40d4210 msgr2=0x7fdac406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:39.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.971+0000 7fdac37fe640 1 --2- 172.21.15.32:0/798128808 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdac40d4210 0x7fdac406b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:39.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.971+0000 7fdac37fe640 1 -- 172.21.15.32:0/798128808 >> v2:172.21.15.121:3300/0 conn(0x7fdac40ba0e0 msgr2=0x7fdac406b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:39.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.971+0000 7fdac37fe640 1 --2- 172.21.15.32:0/798128808 >> v2:172.21.15.121:3300/0 conn(0x7fdac40ba0e0 0x7fdac406b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:39.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.971+0000 7fdac37fe640 1 -- 172.21.15.32:0/798128808 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdac40d3a10 con 0x7fdac415e9b0 2024-04-24T23:02:39.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.971+0000 7fdac37fe640 1 --2- 172.21.15.32:0/798128808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdac415e9b0 0x7fdac40d2b90 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fdac40b61d0 tx=0x7fdab402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b12ce32f2f61e5a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:39.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.972+0000 7fdac27fc640 1 -- 172.21.15.32:0/798128808 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab4037070 con 0x7fdac415e9b0 2024-04-24T23:02:39.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.972+0000 7fdac27fc640 1 -- 172.21.15.32:0/798128808 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdab4032440 con 0x7fdac415e9b0 2024-04-24T23:02:39.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.972+0000 7fdac27fc640 1 -- 172.21.15.32:0/798128808 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab4032810 con 0x7fdac415e9b0 2024-04-24T23:02:39.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.972+0000 7fdac9ce1640 1 -- 172.21.15.32:0/798128808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdac415e9b0 msgr2=0x7fdac40d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:39.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.972+0000 7fdac9ce1640 1 --2- 172.21.15.32:0/798128808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdac415e9b0 0x7fdac40d2b90 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fdac40b61d0 tx=0x7fdab402a1e0 comp rx=0 tx=0).stop 2024-04-24T23:02:39.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.972+0000 7fdac9ce1640 1 -- 172.21.15.32:0/798128808 shutdown_connections 2024-04-24T23:02:39.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.972+0000 7fdac9ce1640 1 --2- 172.21.15.32:0/798128808 >> v2:172.21.15.121:3300/0 conn(0x7fdac40ba0e0 0x7fdac406b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:39.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.972+0000 7fdac9ce1640 1 --2- 172.21.15.32:0/798128808 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdac40d4210 0x7fdac406b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:39.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.972+0000 7fdac9ce1640 1 --2- 172.21.15.32:0/798128808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdac415e9b0 0x7fdac40d2b90 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:39.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.972+0000 7fdac9ce1640 1 -- 172.21.15.32:0/798128808 >> 172.21.15.32:0/798128808 conn(0x7fdac40d11d0 msgr2=0x7fdac40d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:39.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.972+0000 7fdac9ce1640 1 -- 172.21.15.32:0/798128808 shutdown_connections 2024-04-24T23:02:39.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.972+0000 7fdac9ce1640 1 -- 172.21.15.32:0/798128808 wait complete. 2024-04-24T23:02:39.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.973+0000 7fdac9ce1640 1 Processor -- start 2024-04-24T23:02:39.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.974+0000 7fdac9ce1640 1 -- start start 2024-04-24T23:02:39.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.974+0000 7fdac9ce1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdac40ba0e0 0x7fdac415ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:39.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.974+0000 7fdac9ce1640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdac40d4210 0x7fdac40cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:39.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.974+0000 7fdac9ce1640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdac415e9b0 0x7fdac4152b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:39.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.974+0000 7fdac9ce1640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdac4071730 con 0x7fdac40ba0e0 2024-04-24T23:02:39.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.974+0000 7fdac9ce1640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdac40d2400 con 0x7fdac415e9b0 2024-04-24T23:02:39.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.974+0000 7fdac9ce1640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fdac40718b0 con 0x7fdac40d4210 2024-04-24T23:02:39.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.974+0000 7fdac37fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdac40ba0e0 0x7fdac415ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:39.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.974+0000 7fdac37fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdac40ba0e0 0x7fdac415ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41830/0 (socket says 172.21.15.32:41830) 2024-04-24T23:02:39.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.974+0000 7fdac37fe640 1 -- 172.21.15.32:0/786778947 learned_addr learned my addr 172.21.15.32:0/786778947 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:39.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.975+0000 7fdac37fe640 1 -- 172.21.15.32:0/786778947 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdac40d4210 msgr2=0x7fdac40cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:39.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.975+0000 7fdac37fe640 1 --2- 172.21.15.32:0/786778947 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdac40d4210 0x7fdac40cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:39.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.975+0000 7fdac37fe640 1 -- 172.21.15.32:0/786778947 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdac415e9b0 msgr2=0x7fdac4152b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:39.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.975+0000 7fdac3fff640 1 --2- 172.21.15.32:0/786778947 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdac415e9b0 0x7fdac4152b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:39.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.975+0000 7fdac37fe640 1 --2- 172.21.15.32:0/786778947 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdac415e9b0 0x7fdac4152b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:39.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.975+0000 7fdac37fe640 1 -- 172.21.15.32:0/786778947 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdac415f400 con 0x7fdac40ba0e0 2024-04-24T23:02:39.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.975+0000 7fdac37fe640 1 --2- 172.21.15.32:0/786778947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdac40ba0e0 0x7fdac415ed90 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fdab402a220 tx=0x7fdab40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:39.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.975+0000 7fdaabfff640 1 -- 172.21.15.32:0/786778947 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab4041020 con 0x7fdac40ba0e0 2024-04-24T23:02:39.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.975+0000 7fdaabfff640 1 -- 172.21.15.32:0/786778947 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdab40306c0 con 0x7fdac40ba0e0 2024-04-24T23:02:39.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.975+0000 7fdac9ce1640 1 -- 172.21.15.32:0/786778947 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdac415f6e0 con 0x7fdac40ba0e0 2024-04-24T23:02:39.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.975+0000 7fdaabfff640 1 -- 172.21.15.32:0/786778947 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab4037040 con 0x7fdac40ba0e0 2024-04-24T23:02:39.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.975+0000 7fdac9ce1640 1 -- 172.21.15.32:0/786778947 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdac415ff80 con 0x7fdac40ba0e0 2024-04-24T23:02:39.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.976+0000 7fdac9ce1640 1 -- 172.21.15.32:0/786778947 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdac415fad0 con 0x7fdac40ba0e0 2024-04-24T23:02:39.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.977+0000 7fdaabfff640 1 -- 172.21.15.32:0/786778947 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdab4044020 con 0x7fdac40ba0e0 2024-04-24T23:02:39.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.977+0000 7fdaabfff640 1 --2- 172.21.15.32:0/786778947 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdab006be60 0x7fdab006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:39.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.977+0000 7fdaabfff640 1 -- 172.21.15.32:0/786778947 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5767+0+0 (secure 0 0 0) 0x7fdab40ab610 con 0x7fdac40ba0e0 2024-04-24T23:02:39.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.977+0000 7fdac2ffd640 1 --2- 172.21.15.32:0/786778947 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdab006be60 0x7fdab006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:39.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.977+0000 7fdaabfff640 1 -- 172.21.15.32:0/786778947 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=73}) -- 0x7fdab0073ab0 con 0x7fdac40ba0e0 2024-04-24T23:02:39.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.978+0000 7fdac2ffd640 1 --2- 172.21.15.32:0/786778947 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdab006be60 0x7fdab006e320 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fdac40cd8e0 tx=0x7fdab8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:39.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:39.981+0000 7fdaabfff640 1 -- 172.21.15.32:0/786778947 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdab404b030 con 0x7fdac40ba0e0 2024-04-24T23:02:40.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.177+0000 7fdac9ce1640 1 -- 172.21.15.32:0/786778947 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdac4068430 con 0x7fdac40ba0e0 2024-04-24T23:02:40.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.177+0000 7fdaabfff640 1 -- 172.21.15.32:0/786778947 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v72) ==== 80+0+0 (secure 0 0 0) 0x7fdab407a010 con 0x7fdac40ba0e0 2024-04-24T23:02:40.179 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2024-04-24T23:02:40.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.179+0000 7fdac9ce1640 1 -- 172.21.15.32:0/786778947 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdab006be60 msgr2=0x7fdab006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:40.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.179+0000 7fdac9ce1640 1 --2- 172.21.15.32:0/786778947 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdab006be60 0x7fdab006e320 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fdac40cd8e0 tx=0x7fdab8008040 comp rx=0 tx=0).stop 2024-04-24T23:02:40.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.179+0000 7fdac9ce1640 1 -- 172.21.15.32:0/786778947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdac40ba0e0 msgr2=0x7fdac415ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:40.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.179+0000 7fdac9ce1640 1 --2- 172.21.15.32:0/786778947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdac40ba0e0 0x7fdac415ed90 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fdab402a220 tx=0x7fdab40046c0 comp rx=0 tx=0).stop 2024-04-24T23:02:40.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.179+0000 7fdac9ce1640 1 -- 172.21.15.32:0/786778947 shutdown_connections 2024-04-24T23:02:40.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.179+0000 7fdac9ce1640 1 --2- 172.21.15.32:0/786778947 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdab006be60 0x7fdab006e320 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:40.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.179+0000 7fdac9ce1640 1 --2- 172.21.15.32:0/786778947 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdac415e9b0 0x7fdac4152b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:40.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.179+0000 7fdac9ce1640 1 --2- 172.21.15.32:0/786778947 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdac40d4210 0x7fdac40cc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:40.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.179+0000 7fdac9ce1640 1 --2- 172.21.15.32:0/786778947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdac40ba0e0 0x7fdac415ed90 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:40.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.179+0000 7fdac9ce1640 1 -- 172.21.15.32:0/786778947 >> 172.21.15.32:0/786778947 conn(0x7fdac40d11d0 msgr2=0x7fdac415bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:40.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.179+0000 7fdac9ce1640 1 -- 172.21.15.32:0/786778947 shutdown_connections 2024-04-24T23:02:40.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.179+0000 7fdac9ce1640 1 -- 172.21.15.32:0/786778947 wait complete. 2024-04-24T23:02:40.191 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-24T23:02:40.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:02:40.191 DEBUG:teuthology.orchestra.run.smithi121:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-24T23:02:40.223 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2024-04-24T23:02:40.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:02:40.223 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-24T23:02:40.281 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:02:40.281 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:02:40.281 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:02:40.297 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=413 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T23:02:40.297 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:02:40.297 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:02:40.297 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:02:40.338 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3271 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T23:02:40.338 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:02:40.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.437+0000 7fda8bad4640 1 Processor -- start 2024-04-24T23:02:40.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.437+0000 7fda8bad4640 1 -- start start 2024-04-24T23:02:40.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.438+0000 7fda8bad4640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fda84162320 0x7fda840ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:40.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.438+0000 7fda8bad4640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fda840b5af0 0x7fda84162700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:40.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.438+0000 7fda8bad4640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda840cd270 con 0x7fda84162c40 2024-04-24T23:02:40.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.438+0000 7fda8bad4640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fda840b5660 con 0x7fda84162320 2024-04-24T23:02:40.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.438+0000 7fda8bad4640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fda84071610 con 0x7fda840b5af0 2024-04-24T23:02:40.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.438+0000 7fda89870640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fda84162320 0x7fda840ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:40.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.438+0000 7fda89870640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fda84162320 0x7fda840ccc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41844/0 (socket says 172.21.15.32:41844) 2024-04-24T23:02:40.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.438+0000 7fda89870640 1 -- 172.21.15.32:0/3331706041 learned_addr learned my addr 172.21.15.32:0/3331706041 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:40.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.439+0000 7fda8906f640 1 --2- 172.21.15.32:0/3331706041 >> v2:172.21.15.121:3300/0 conn(0x7fda840b5af0 0x7fda84162700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:40.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.439+0000 7fda89870640 1 -- 172.21.15.32:0/3331706041 >> v2:172.21.15.121:3300/0 conn(0x7fda840b5af0 msgr2=0x7fda84162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:40.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.439+0000 7fda89870640 1 --2- 172.21.15.32:0/3331706041 >> v2:172.21.15.121:3300/0 conn(0x7fda840b5af0 0x7fda84162700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:40.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.439+0000 7fda89870640 1 -- 172.21.15.32:0/3331706041 >> v1:172.21.15.121:6789/0 conn(0x7fda84162c40 legacy=0x7fda840ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:40.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.439+0000 7fda89870640 1 -- 172.21.15.32:0/3331706041 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda840718f0 con 0x7fda84162320 2024-04-24T23:02:40.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.439+0000 7fda89870640 1 --2- 172.21.15.32:0/3331706041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda84162320 0x7fda840ccc90 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fda8406c5b0 tx=0x7fda7c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23a808fb4b2bb16d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:40.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.439+0000 7fda8886e640 1 -- 172.21.15.32:0/3331706041 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda7c037070 con 0x7fda84162320 2024-04-24T23:02:40.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.439+0000 7fda8886e640 1 -- 172.21.15.32:0/3331706041 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda7c032440 con 0x7fda84162320 2024-04-24T23:02:40.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.439+0000 7fda8886e640 1 -- 172.21.15.32:0/3331706041 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda7c032810 con 0x7fda84162320 2024-04-24T23:02:40.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.440+0000 7fda8bad4640 1 -- 172.21.15.32:0/3331706041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda84162320 msgr2=0x7fda840ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:40.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.440+0000 7fda8bad4640 1 --2- 172.21.15.32:0/3331706041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda84162320 0x7fda840ccc90 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fda8406c5b0 tx=0x7fda7c02a180 comp rx=0 tx=0).stop 2024-04-24T23:02:40.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.440+0000 7fda8bad4640 1 -- 172.21.15.32:0/3331706041 shutdown_connections 2024-04-24T23:02:40.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.440+0000 7fda8bad4640 1 --2- 172.21.15.32:0/3331706041 >> v2:172.21.15.121:3300/0 conn(0x7fda840b5af0 0x7fda84162700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:40.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.440+0000 7fda8bad4640 1 --2- 172.21.15.32:0/3331706041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda84162320 0x7fda840ccc90 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:40.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.440+0000 7fda8bad4640 1 -- 172.21.15.32:0/3331706041 >> 172.21.15.32:0/3331706041 conn(0x7fda840d0ae0 msgr2=0x7fda840d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:40.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.440+0000 7fda8bad4640 1 -- 172.21.15.32:0/3331706041 shutdown_connections 2024-04-24T23:02:40.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.440+0000 7fda8bad4640 1 -- 172.21.15.32:0/3331706041 wait complete. 2024-04-24T23:02:40.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.441+0000 7fda8bad4640 1 Processor -- start 2024-04-24T23:02:40.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.441+0000 7fda8bad4640 1 -- start start 2024-04-24T23:02:40.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.442+0000 7fda8bad4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda840b5af0 0x7fda8406cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:40.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.442+0000 7fda8bad4640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda84162320 0x7fda84145420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:40.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.442+0000 7fda8bad4640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fda84162c40 0x7fda8415a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:40.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.442+0000 7fda8bad4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fda84163070 con 0x7fda840b5af0 2024-04-24T23:02:40.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.442+0000 7fda8bad4640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fda84067950 con 0x7fda84162320 2024-04-24T23:02:40.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.442+0000 7fda8bad4640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fda84071bd0 con 0x7fda84162c40 2024-04-24T23:02:40.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.442+0000 7fda89870640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda840b5af0 0x7fda8406cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:40.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.442+0000 7fda89870640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda840b5af0 0x7fda8406cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41858/0 (socket says 172.21.15.32:41858) 2024-04-24T23:02:40.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.442+0000 7fda89870640 1 -- 172.21.15.32:0/2175165494 learned_addr learned my addr 172.21.15.32:0/2175165494 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:40.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.442+0000 7fda89870640 1 -- 172.21.15.32:0/2175165494 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fda84162c40 msgr2=0x7fda8415a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:40.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.442+0000 7fda89870640 1 --2- 172.21.15.32:0/2175165494 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fda84162c40 0x7fda8415a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:40.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.442+0000 7fda89870640 1 -- 172.21.15.32:0/2175165494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda84162320 msgr2=0x7fda84145420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:40.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.442+0000 7fda89870640 1 --2- 172.21.15.32:0/2175165494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda84162320 0x7fda84145420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:40.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.442+0000 7fda89870640 1 -- 172.21.15.32:0/2175165494 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda8406d4f0 con 0x7fda840b5af0 2024-04-24T23:02:40.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.443+0000 7fda89870640 1 --2- 172.21.15.32:0/2175165494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda840b5af0 0x7fda8406cca0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fda7c000c00 tx=0x7fda7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:40.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.443+0000 7fda827fc640 1 -- 172.21.15.32:0/2175165494 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda7c037040 con 0x7fda840b5af0 2024-04-24T23:02:40.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.443+0000 7fda827fc640 1 -- 172.21.15.32:0/2175165494 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda7c042070 con 0x7fda840b5af0 2024-04-24T23:02:40.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.443+0000 7fda8bad4640 1 -- 172.21.15.32:0/2175165494 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda8406d7d0 con 0x7fda840b5af0 2024-04-24T23:02:40.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.443+0000 7fda827fc640 1 -- 172.21.15.32:0/2175165494 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda7c030bb0 con 0x7fda840b5af0 2024-04-24T23:02:40.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.443+0000 7fda8bad4640 1 -- 172.21.15.32:0/2175165494 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda8406d1e0 con 0x7fda840b5af0 2024-04-24T23:02:40.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.444+0000 7fda8bad4640 1 -- 172.21.15.32:0/2175165494 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda84068430 con 0x7fda840b5af0 2024-04-24T23:02:40.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.444+0000 7fda827fc640 1 -- 172.21.15.32:0/2175165494 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fda7c047460 con 0x7fda840b5af0 2024-04-24T23:02:40.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.445+0000 7fda827fc640 1 --2- 172.21.15.32:0/2175165494 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fda7006be60 0x7fda7006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:40.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.445+0000 7fda827fc640 1 -- 172.21.15.32:0/2175165494 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5767+0+0 (secure 0 0 0) 0x7fda7c0aba60 con 0x7fda840b5af0 2024-04-24T23:02:40.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.445+0000 7fda827fc640 1 -- 172.21.15.32:0/2175165494 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=73}) -- 0x7fda70073a50 con 0x7fda840b5af0 2024-04-24T23:02:40.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.445+0000 7fda8906f640 1 --2- 172.21.15.32:0/2175165494 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fda7006be60 0x7fda7006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:40.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.445+0000 7fda8906f640 1 --2- 172.21.15.32:0/2175165494 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fda7006be60 0x7fda7006e320 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fda78006fd0 tx=0x7fda78008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:40.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.449+0000 7fda827fc640 1 -- 172.21.15.32:0/2175165494 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fda7c07a460 con 0x7fda840b5af0 2024-04-24T23:02:40.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.672+0000 7fda8bad4640 1 -- 172.21.15.32:0/2175165494 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fda840d0850 con 0x7fda840b5af0 2024-04-24T23:02:40.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.672+0000 7fda827fc640 1 -- 172.21.15.32:0/2175165494 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5187 (secure 0 0 0) 0x7fda7c079d60 con 0x7fda840b5af0 2024-04-24T23:02:40.674 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:02:40.674 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":27,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3085809520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3085809520},{"type":"v1","addr":"172.21.15.121:6837","nonce":3085809520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":6078,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/1405465715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1405465715},{"type":"v1","addr":"172.21.15.32:6834","nonce":1405465715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":6081,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1160416671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":1160416671},{"type":"v1","addr":"172.21.15.32:6835","nonce":1160416671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:01:25.028304+0000","modified":"2024-04-24T23:01:27.048925+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6090},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6090":{"gid":6090,"name":"d","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6836/2650840833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2650840833},{"type":"v1","addr":"172.21.15.121:6836","nonce":2650840833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6090,"qdb_cluster":[6090]},"id":4}]} 2024-04-24T23:02:40.674 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 27 2024-04-24T23:02:40.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.674+0000 7fda8bad4640 1 -- 172.21.15.32:0/2175165494 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fda7006be60 msgr2=0x7fda7006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:40.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.674+0000 7fda8bad4640 1 --2- 172.21.15.32:0/2175165494 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fda7006be60 0x7fda7006e320 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fda78006fd0 tx=0x7fda78008040 comp rx=0 tx=0).stop 2024-04-24T23:02:40.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.674+0000 7fda8bad4640 1 -- 172.21.15.32:0/2175165494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda840b5af0 msgr2=0x7fda8406cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:40.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.674+0000 7fda8bad4640 1 --2- 172.21.15.32:0/2175165494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda840b5af0 0x7fda8406cca0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fda7c000c00 tx=0x7fda7c0046c0 comp rx=0 tx=0).stop 2024-04-24T23:02:40.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.674+0000 7fda8bad4640 1 -- 172.21.15.32:0/2175165494 shutdown_connections 2024-04-24T23:02:40.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.674+0000 7fda8bad4640 1 --2- 172.21.15.32:0/2175165494 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fda7006be60 0x7fda7006e320 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:40.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.674+0000 7fda8bad4640 1 --2- 172.21.15.32:0/2175165494 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fda84162c40 0x7fda8415a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:40.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.674+0000 7fda8bad4640 1 --2- 172.21.15.32:0/2175165494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda84162320 0x7fda84145420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:40.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.674+0000 7fda8bad4640 1 --2- 172.21.15.32:0/2175165494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda840b5af0 0x7fda8406cca0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:40.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.674+0000 7fda8bad4640 1 -- 172.21.15.32:0/2175165494 >> 172.21.15.32:0/2175165494 conn(0x7fda840d0ae0 msgr2=0x7fda840b3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:40.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.674+0000 7fda8bad4640 1 -- 172.21.15.32:0/2175165494 shutdown_connections 2024-04-24T23:02:40.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.674+0000 7fda8bad4640 1 -- 172.21.15.32:0/2175165494 wait complete. 2024-04-24T23:02:40.687 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:02:40.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.786+0000 7f261c6ee640 1 Processor -- start 2024-04-24T23:02:40.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.786+0000 7f261c6ee640 1 -- start start 2024-04-24T23:02:40.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.787+0000 7f261c6ee640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f261406b820 0x7f26140d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:40.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.787+0000 7f261c6ee640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26140d3170 con 0x7f26140d4210 2024-04-24T23:02:40.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.787+0000 7f261c6ee640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26140d3450 con 0x7f261415e9b0 2024-04-24T23:02:40.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.787+0000 7f261c6ee640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f26140d3730 con 0x7f261406b820 2024-04-24T23:02:40.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.788+0000 7f261ac8b640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f26140d4210 0x7f26140b78f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46154/0 (socket says 172.21.15.32:46154) 2024-04-24T23:02:40.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.788+0000 7f261ac8b640 1 -- 172.21.15.32:0/2593146659 learned_addr learned my addr 172.21.15.32:0/2593146659 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:02:40.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.788+0000 7f2619488640 1 -- 172.21.15.32:0/2593146659 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1625865021 0 0) 0x7f26140d3170 con 0x7f26140d4210 2024-04-24T23:02:40.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.788+0000 7f2619488640 1 -- 172.21.15.32:0/2593146659 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2600003520 con 0x7f26140d4210 2024-04-24T23:02:40.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.788+0000 7f2619c89640 1 --2- 172.21.15.32:0/2593146659 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f261406b820 0x7f26140d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:40.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.788+0000 7f2619488640 1 -- 172.21.15.32:0/2593146659 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 592822422 0 0) 0x7f2600003520 con 0x7f26140d4210 2024-04-24T23:02:40.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.788+0000 7f2619488640 1 -- 172.21.15.32:0/2593146659 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f261406b820 msgr2=0x7f26140d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:40.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.788+0000 7f2619488640 1 --2- 172.21.15.32:0/2593146659 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f261406b820 0x7f26140d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:40.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.788+0000 7f2619488640 1 -- 172.21.15.32:0/2593146659 >> v1:172.21.15.121:6789/0 conn(0x7f261415e9b0 legacy=0x7f261406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:40.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.788+0000 7f2619488640 1 -- 172.21.15.32:0/2593146659 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26140d3a10 con 0x7f26140d4210 2024-04-24T23:02:40.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.788+0000 7f2619488640 1 -- 172.21.15.32:0/2593146659 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f260c0019f0 con 0x7f26140d4210 2024-04-24T23:02:40.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.788+0000 7f2619488640 1 -- 172.21.15.32:0/2593146659 >> v1:172.21.15.32:6789/0 conn(0x7f26140d4210 legacy=0x7f26140b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:40.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.788+0000 7f2619488640 1 --2- 172.21.15.32:0/2593146659 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26000039e0 0x7f2600003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:40.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.788+0000 7f2619488640 1 --2- 172.21.15.32:0/2593146659 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f261406b820 0x7f2600004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:40.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.788+0000 7f2619488640 1 --2- 172.21.15.32:0/2593146659 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2600004d40 0x7f26000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:40.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.788+0000 7f2619488640 1 -- 172.21.15.32:0/2593146659 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f26140d3730 con 0x7f26000039e0 2024-04-24T23:02:40.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.788+0000 7f2619488640 1 -- 172.21.15.32:0/2593146659 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f260000bb80 con 0x7f2600004d40 2024-04-24T23:02:40.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.788+0000 7f2619488640 1 -- 172.21.15.32:0/2593146659 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f260000bd80 con 0x7f261406b820 2024-04-24T23:02:40.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.788+0000 7f261a48a640 1 --2- 172.21.15.32:0/2593146659 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26000039e0 0x7f2600003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:40.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.788+0000 7f2619c89640 1 --2- 172.21.15.32:0/2593146659 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f261406b820 0x7f2600004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:40.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.788+0000 7f261ac8b640 1 --2- 172.21.15.32:0/2593146659 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2600004d40 0x7f26000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:40.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.789+0000 7f261a48a640 1 -- 172.21.15.32:0/2593146659 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f261406b820 msgr2=0x7f2600004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:40.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.789+0000 7f261a48a640 1 --2- 172.21.15.32:0/2593146659 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f261406b820 0x7f2600004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:40.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.789+0000 7f261a48a640 1 -- 172.21.15.32:0/2593146659 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2600004d40 msgr2=0x7f26000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:40.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.789+0000 7f261a48a640 1 --2- 172.21.15.32:0/2593146659 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2600004d40 0x7f26000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:40.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.789+0000 7f261a48a640 1 -- 172.21.15.32:0/2593146659 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2600003520 con 0x7f26000039e0 2024-04-24T23:02:40.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.789+0000 7f261ac8b640 1 --2- 172.21.15.32:0/2593146659 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2600004d40 0x7f26000051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:02:40.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.789+0000 7f261a48a640 1 --2- 172.21.15.32:0/2593146659 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26000039e0 0x7f2600003e00 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f2610001750 tx=0x7f2600005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2be963ca1f87e6bb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:40.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.789+0000 7f2619488640 1 -- 172.21.15.32:0/2593146659 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2610001370 con 0x7f26000039e0 2024-04-24T23:02:40.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.789+0000 7f2619488640 1 -- 172.21.15.32:0/2593146659 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26100052d0 con 0x7f26000039e0 2024-04-24T23:02:40.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.789+0000 7f2619488640 1 -- 172.21.15.32:0/2593146659 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261002abe0 con 0x7f26000039e0 2024-04-24T23:02:40.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.790+0000 7f261c6ee640 1 -- 172.21.15.32:0/2593146659 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26000039e0 msgr2=0x7f2600003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:40.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.790+0000 7f261c6ee640 1 --2- 172.21.15.32:0/2593146659 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26000039e0 0x7f2600003e00 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f2610001750 tx=0x7f2600005830 comp rx=0 tx=0).stop 2024-04-24T23:02:40.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.790+0000 7f261ac8b640 1 -- 172.21.15.32:0/2593146659 reap_dead start 2024-04-24T23:02:40.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.790+0000 7f261c6ee640 1 -- 172.21.15.32:0/2593146659 shutdown_connections 2024-04-24T23:02:40.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.790+0000 7f261c6ee640 1 -- 172.21.15.32:0/2593146659 >> 172.21.15.32:0/2593146659 conn(0x7f26140d11d0 msgr2=0x7f26140d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:40.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.790+0000 7f261c6ee640 1 -- 172.21.15.32:0/2593146659 shutdown_connections 2024-04-24T23:02:40.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.790+0000 7f261c6ee640 1 -- 172.21.15.32:0/2593146659 wait complete. 2024-04-24T23:02:40.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.791+0000 7f261c6ee640 1 Processor -- start 2024-04-24T23:02:40.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.791+0000 7f261c6ee640 1 -- start start 2024-04-24T23:02:40.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.792+0000 7f261c6ee640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26000039e0 0x7f2614155c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:40.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.792+0000 7f261c6ee640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26140ce750 0x7f26141f3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:40.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.792+0000 7f261c6ee640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26141f7d10 0x7f2614071730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:40.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.792+0000 7f261c6ee640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f26140ba0e0 con 0x7f26141f7d10 2024-04-24T23:02:40.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.792+0000 7f261c6ee640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f26140d2400 con 0x7f26140ce750 2024-04-24T23:02:40.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.792+0000 7f261c6ee640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f26140ba260 con 0x7f26000039e0 2024-04-24T23:02:40.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.792+0000 7f261ac8b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26141f7d10 0x7f2614071730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:40.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.792+0000 7f261ac8b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26141f7d10 0x7f2614071730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41870/0 (socket says 172.21.15.32:41870) 2024-04-24T23:02:40.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.792+0000 7f261ac8b640 1 -- 172.21.15.32:0/2408885469 learned_addr learned my addr 172.21.15.32:0/2408885469 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:40.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.792+0000 7f2619c89640 1 --2- 172.21.15.32:0/2408885469 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26140ce750 0x7f26141f3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:40.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.792+0000 7f261a48a640 1 --2- 172.21.15.32:0/2408885469 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26000039e0 0x7f2614155c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:40.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.792+0000 7f261ac8b640 1 -- 172.21.15.32:0/2408885469 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26000039e0 msgr2=0x7f2614155c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:40.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.792+0000 7f261ac8b640 1 --2- 172.21.15.32:0/2408885469 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26000039e0 0x7f2614155c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:40.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.792+0000 7f261ac8b640 1 -- 172.21.15.32:0/2408885469 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26140ce750 msgr2=0x7f26141f3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:40.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.792+0000 7f261ac8b640 1 --2- 172.21.15.32:0/2408885469 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26140ce750 0x7f26141f3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:40.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.792+0000 7f261ac8b640 1 -- 172.21.15.32:0/2408885469 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26141f4760 con 0x7f26141f7d10 2024-04-24T23:02:40.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.793+0000 7f261ac8b640 1 --2- 172.21.15.32:0/2408885469 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26141f7d10 0x7f2614071730 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f26141622f0 tx=0x7f260c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:40.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.793+0000 7f260affd640 1 -- 172.21.15.32:0/2408885469 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260c00ec10 con 0x7f26141f7d10 2024-04-24T23:02:40.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.793+0000 7f260affd640 1 -- 172.21.15.32:0/2408885469 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f260c00edb0 con 0x7f26141f7d10 2024-04-24T23:02:40.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.793+0000 7f261c6ee640 1 -- 172.21.15.32:0/2408885469 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26141f4a40 con 0x7f26141f7d10 2024-04-24T23:02:40.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.793+0000 7f260affd640 1 -- 172.21.15.32:0/2408885469 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260c014670 con 0x7f26141f7d10 2024-04-24T23:02:40.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.793+0000 7f261c6ee640 1 -- 172.21.15.32:0/2408885469 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26141f7510 con 0x7f26141f7d10 2024-04-24T23:02:40.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.794+0000 7f261c6ee640 1 -- 172.21.15.32:0/2408885469 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2614068430 con 0x7f26141f7d10 2024-04-24T23:02:40.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.794+0000 7f260affd640 1 -- 172.21.15.32:0/2408885469 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f260c014810 con 0x7f26141f7d10 2024-04-24T23:02:40.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.795+0000 7f260affd640 1 --2- 172.21.15.32:0/2408885469 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f25f806be60 0x7f25f806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:40.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.795+0000 7f260affd640 1 -- 172.21.15.32:0/2408885469 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5767+0+0 (secure 0 0 0) 0x7f260c08ac70 con 0x7f26141f7d10 2024-04-24T23:02:40.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.795+0000 7f261a48a640 1 --2- 172.21.15.32:0/2408885469 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f25f806be60 0x7f25f806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:40.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.795+0000 7f260affd640 1 -- 172.21.15.32:0/2408885469 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=73}) -- 0x7f25f8073a50 con 0x7f26141f7d10 2024-04-24T23:02:40.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.795+0000 7f261a48a640 1 --2- 172.21.15.32:0/2408885469 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f25f806be60 0x7f25f806e320 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2610032000 tx=0x7f261002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:40.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.799+0000 7f260affd640 1 -- 172.21.15.32:0/2408885469 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f260c0595e0 con 0x7f26141f7d10 2024-04-24T23:02:40.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.986+0000 7f261c6ee640 1 -- 172.21.15.32:0/2408885469 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2614074df0 con 0x7f26141f7d10 2024-04-24T23:02:40.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.987+0000 7f260affd640 1 -- 172.21.15.32:0/2408885469 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+16823 (secure 0 0 0) 0x7f260c058ee0 con 0x7f26141f7d10 2024-04-24T23:02:40.988 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:02:40.988 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":72,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:01:52.865926+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:01:23.328944+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-24T23:01:24.010365+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["backfillfull","exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","full","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","full","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.a","mappings":[{"from":6,"to":4}]},{"pgid":"9.3a","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:02:40.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.989+0000 7f261c6ee640 1 -- 172.21.15.32:0/2408885469 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f25f806be60 msgr2=0x7f25f806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:40.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.989+0000 7f261c6ee640 1 --2- 172.21.15.32:0/2408885469 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f25f806be60 0x7f25f806e320 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2610032000 tx=0x7f261002e000 comp rx=0 tx=0).stop 2024-04-24T23:02:40.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.989+0000 7f261c6ee640 1 -- 172.21.15.32:0/2408885469 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26141f7d10 msgr2=0x7f2614071730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:40.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.989+0000 7f261c6ee640 1 --2- 172.21.15.32:0/2408885469 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26141f7d10 0x7f2614071730 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f26141622f0 tx=0x7f260c00e530 comp rx=0 tx=0).stop 2024-04-24T23:02:40.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.989+0000 7f261c6ee640 1 -- 172.21.15.32:0/2408885469 shutdown_connections 2024-04-24T23:02:40.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.989+0000 7f261c6ee640 1 --2- 172.21.15.32:0/2408885469 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f25f806be60 0x7f25f806e320 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:40.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.989+0000 7f261c6ee640 1 --2- 172.21.15.32:0/2408885469 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26141f7d10 0x7f2614071730 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:40.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.989+0000 7f261c6ee640 1 --2- 172.21.15.32:0/2408885469 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26140ce750 0x7f26141f3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:40.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.989+0000 7f261c6ee640 1 --2- 172.21.15.32:0/2408885469 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26000039e0 0x7f2614155c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:40.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.989+0000 7f261c6ee640 1 -- 172.21.15.32:0/2408885469 >> 172.21.15.32:0/2408885469 conn(0x7f26140d11d0 msgr2=0x7f261406b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:40.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.989+0000 7f261c6ee640 1 -- 172.21.15.32:0/2408885469 shutdown_connections 2024-04-24T23:02:40.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:40.989+0000 7f261c6ee640 1 -- 172.21.15.32:0/2408885469 wait complete. 2024-04-24T23:02:41.001 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:01:23.328944+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:01:24.010365+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:02:41.002 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:02:41.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.104+0000 7f06876a5640 1 Processor -- start 2024-04-24T23:02:41.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.105+0000 7f06876a5640 1 -- start start 2024-04-24T23:02:41.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.105+0000 7f06876a5640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f06800b5af0 0x7f06800ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:41.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.105+0000 7f06876a5640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06800cd270 con 0x7f0680162e10 2024-04-24T23:02:41.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.105+0000 7f06876a5640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0680071610 con 0x7f0680162320 2024-04-24T23:02:41.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.105+0000 7f06876a5640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f06800b5660 con 0x7f06800b5af0 2024-04-24T23:02:41.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.106+0000 7f0685c42640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0680162e10 0x7f068015ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46158/0 (socket says 172.21.15.32:46158) 2024-04-24T23:02:41.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.106+0000 7f0685c42640 1 -- 172.21.15.32:0/3815122062 learned_addr learned my addr 172.21.15.32:0/3815122062 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:02:41.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.106+0000 7f0684c40640 1 --2- 172.21.15.32:0/3815122062 >> v2:172.21.15.121:3300/0 conn(0x7f06800b5af0 0x7f06800ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:41.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.106+0000 7f0677fff640 1 -- 172.21.15.32:0/3815122062 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 967349800 0 0) 0x7f06800cd270 con 0x7f0680162e10 2024-04-24T23:02:41.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.106+0000 7f0677fff640 1 -- 172.21.15.32:0/3815122062 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f066c003520 con 0x7f0680162e10 2024-04-24T23:02:41.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.106+0000 7f0677fff640 1 -- 172.21.15.32:0/3815122062 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3832954814 0 0) 0x7f066c003520 con 0x7f0680162e10 2024-04-24T23:02:41.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.106+0000 7f0684c40640 1 -- 172.21.15.32:0/3815122062 >> v1:172.21.15.121:6789/0 conn(0x7f0680162320 legacy=0x7f0680162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:41.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.106+0000 7f0684c40640 1 -- 172.21.15.32:0/3815122062 >> v1:172.21.15.32:6789/0 conn(0x7f0680162e10 legacy=0x7f068015ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:41.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.106+0000 7f0684c40640 1 -- 172.21.15.32:0/3815122062 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06800718f0 con 0x7f06800b5af0 2024-04-24T23:02:41.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.107+0000 7f0684c40640 1 --2- 172.21.15.32:0/3815122062 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06800b5af0 0x7f06800ccc90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f06700097a0 tx=0x7f067002a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6b078c0dcd1177f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:41.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.107+0000 7f0677fff640 1 -- 172.21.15.32:0/3815122062 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0670037070 con 0x7f06800b5af0 2024-04-24T23:02:41.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.107+0000 7f0677fff640 1 -- 172.21.15.32:0/3815122062 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f067002aad0 con 0x7f06800b5af0 2024-04-24T23:02:41.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.107+0000 7f0677fff640 1 -- 172.21.15.32:0/3815122062 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06780019f0 con 0x7f06800b5af0 2024-04-24T23:02:41.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.108+0000 7f06876a5640 1 -- 172.21.15.32:0/3815122062 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06800b5af0 msgr2=0x7f06800ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:41.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.108+0000 7f06876a5640 1 --2- 172.21.15.32:0/3815122062 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06800b5af0 0x7f06800ccc90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f06700097a0 tx=0x7f067002a040 comp rx=0 tx=0).stop 2024-04-24T23:02:41.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.108+0000 7f06876a5640 1 -- 172.21.15.32:0/3815122062 shutdown_connections 2024-04-24T23:02:41.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.108+0000 7f06876a5640 1 --2- 172.21.15.32:0/3815122062 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06800b5af0 0x7f06800ccc90 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:41.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.108+0000 7f06876a5640 1 -- 172.21.15.32:0/3815122062 >> 172.21.15.32:0/3815122062 conn(0x7f06800d0ae0 msgr2=0x7f06800d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:41.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.108+0000 7f06876a5640 1 -- 172.21.15.32:0/3815122062 shutdown_connections 2024-04-24T23:02:41.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.108+0000 7f06876a5640 1 -- 172.21.15.32:0/3815122062 wait complete. 2024-04-24T23:02:41.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.109+0000 7f06876a5640 1 Processor -- start 2024-04-24T23:02:41.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.109+0000 7f06876a5640 1 -- start start 2024-04-24T23:02:41.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.110+0000 7f06876a5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06800b5af0 0x7f06801f7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:41.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.110+0000 7f06876a5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0680162320 0x7f0680158f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:41.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.110+0000 7f06876a5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0680162e10 0x7f06801594d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:41.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.110+0000 7f06876a5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0680071bd0 con 0x7f06800b5af0 2024-04-24T23:02:41.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.110+0000 7f06876a5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0680067950 con 0x7f0680162320 2024-04-24T23:02:41.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.110+0000 7f06876a5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f0680071d50 con 0x7f0680162e10 2024-04-24T23:02:41.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.110+0000 7f0685441640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06800b5af0 0x7f06801f7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:41.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.110+0000 7f0685441640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06800b5af0 0x7f06801f7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41876/0 (socket says 172.21.15.32:41876) 2024-04-24T23:02:41.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.110+0000 7f0685441640 1 -- 172.21.15.32:0/3825107283 learned_addr learned my addr 172.21.15.32:0/3825107283 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:41.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.110+0000 7f0684c40640 1 --2- 172.21.15.32:0/3825107283 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0680162320 0x7f0680158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:41.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.110+0000 7f0685c42640 1 --2- 172.21.15.32:0/3825107283 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0680162e10 0x7f06801594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:41.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.110+0000 7f0685441640 1 -- 172.21.15.32:0/3825107283 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0680162e10 msgr2=0x7f06801594d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:41.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.110+0000 7f0685441640 1 --2- 172.21.15.32:0/3825107283 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0680162e10 0x7f06801594d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:41.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.110+0000 7f0685441640 1 -- 172.21.15.32:0/3825107283 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0680162320 msgr2=0x7f0680158f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:41.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.110+0000 7f0685441640 1 --2- 172.21.15.32:0/3825107283 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0680162320 0x7f0680158f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:41.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.110+0000 7f0685441640 1 -- 172.21.15.32:0/3825107283 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06800cf520 con 0x7f06800b5af0 2024-04-24T23:02:41.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.110+0000 7f0685c42640 1 --2- 172.21.15.32:0/3825107283 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0680162e10 0x7f06801594d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:02:41.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.110+0000 7f0685441640 1 --2- 172.21.15.32:0/3825107283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06800b5af0 0x7f06801f7750 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f067c00c5a0 tx=0x7f067c00ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:41.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.110+0000 7f0675ffb640 1 -- 172.21.15.32:0/3825107283 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067c017070 con 0x7f06800b5af0 2024-04-24T23:02:41.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.111+0000 7f0675ffb640 1 -- 172.21.15.32:0/3825107283 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f067c0035f0 con 0x7f06800b5af0 2024-04-24T23:02:41.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.111+0000 7f06876a5640 1 -- 172.21.15.32:0/3825107283 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06800cf800 con 0x7f06800b5af0 2024-04-24T23:02:41.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.111+0000 7f0675ffb640 1 -- 172.21.15.32:0/3825107283 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067c0039e0 con 0x7f06800b5af0 2024-04-24T23:02:41.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.111+0000 7f06876a5640 1 -- 172.21.15.32:0/3825107283 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06800cfdc0 con 0x7f06800b5af0 2024-04-24T23:02:41.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.112+0000 7f06876a5640 1 -- 172.21.15.32:0/3825107283 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0680068430 con 0x7f06800b5af0 2024-04-24T23:02:41.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.112+0000 7f0675ffb640 1 -- 172.21.15.32:0/3825107283 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f067c01a070 con 0x7f06800b5af0 2024-04-24T23:02:41.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.113+0000 7f0675ffb640 1 --2- 172.21.15.32:0/3825107283 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f066406be30 0x7f066406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:41.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.113+0000 7f0675ffb640 1 -- 172.21.15.32:0/3825107283 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5767+0+0 (secure 0 0 0) 0x7f067c08a2b0 con 0x7f06800b5af0 2024-04-24T23:02:41.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.113+0000 7f0675ffb640 1 -- 172.21.15.32:0/3825107283 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=73}) -- 0x7f0664073a20 con 0x7f06800b5af0 2024-04-24T23:02:41.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.113+0000 7f0684c40640 1 --2- 172.21.15.32:0/3825107283 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f066406be30 0x7f066406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:41.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.114+0000 7f0684c40640 1 --2- 172.21.15.32:0/3825107283 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f066406be30 0x7f066406e2f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0670034000 tx=0x7f067002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:41.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.117+0000 7f0675ffb640 1 -- 172.21.15.32:0/3825107283 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f067c0585c0 con 0x7f06800b5af0 2024-04-24T23:02:41.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.300+0000 7f06876a5640 1 -- 172.21.15.32:0/3825107283 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0680067730 con 0x7f06800b5af0 2024-04-24T23:02:41.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.301+0000 7f0675ffb640 1 -- 172.21.15.32:0/3825107283 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) ==== 93+0+10 (secure 0 0 0) 0x7f067c05c8c0 con 0x7f06800b5af0 2024-04-24T23:02:41.302 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T23:02:41.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.303+0000 7f06876a5640 1 -- 172.21.15.32:0/3825107283 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f066406be30 msgr2=0x7f066406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:41.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.303+0000 7f06876a5640 1 --2- 172.21.15.32:0/3825107283 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f066406be30 0x7f066406e2f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0670034000 tx=0x7f067002e000 comp rx=0 tx=0).stop 2024-04-24T23:02:41.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.303+0000 7f06876a5640 1 -- 172.21.15.32:0/3825107283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06800b5af0 msgr2=0x7f06801f7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:41.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.303+0000 7f06876a5640 1 --2- 172.21.15.32:0/3825107283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06800b5af0 0x7f06801f7750 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f067c00c5a0 tx=0x7f067c00ca60 comp rx=0 tx=0).stop 2024-04-24T23:02:41.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.303+0000 7f06876a5640 1 -- 172.21.15.32:0/3825107283 shutdown_connections 2024-04-24T23:02:41.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.303+0000 7f06876a5640 1 --2- 172.21.15.32:0/3825107283 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f066406be30 0x7f066406e2f0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:41.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.303+0000 7f06876a5640 1 --2- 172.21.15.32:0/3825107283 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0680162e10 0x7f06801594d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:41.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.303+0000 7f06876a5640 1 --2- 172.21.15.32:0/3825107283 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0680162320 0x7f0680158f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:41.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.303+0000 7f06876a5640 1 --2- 172.21.15.32:0/3825107283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06800b5af0 0x7f06801f7750 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:41.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.303+0000 7f06876a5640 1 -- 172.21.15.32:0/3825107283 >> 172.21.15.32:0/3825107283 conn(0x7f06800d0ae0 msgr2=0x7f068015dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:41.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.303+0000 7f06876a5640 1 -- 172.21.15.32:0/3825107283 shutdown_connections 2024-04-24T23:02:41.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.303+0000 7f06876a5640 1 -- 172.21.15.32:0/3825107283 wait complete. 2024-04-24T23:02:41.315 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:02:41.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.418+0000 7f93dabb7640 1 Processor -- start 2024-04-24T23:02:41.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.418+0000 7f93dabb7640 1 -- start start 2024-04-24T23:02:41.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.419+0000 7f93dabb7640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93d415fd60 0x7f93d406c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:41.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.419+0000 7f93dabb7640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f93d406cb30 0x7f93d40cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:41.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.419+0000 7f93dabb7640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93d40b46c0 con 0x7f93d415b480 2024-04-24T23:02:41.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.419+0000 7f93dabb7640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f93d40b4ff0 con 0x7f93d406cb30 2024-04-24T23:02:41.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.419+0000 7f93dabb7640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f93d40b49a0 con 0x7f93d415fd60 2024-04-24T23:02:41.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.419+0000 7f93d8953640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f93d415b480 0x7f93d40b5c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46162/0 (socket says 172.21.15.32:46162) 2024-04-24T23:02:41.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.419+0000 7f93d8953640 1 -- 172.21.15.32:0/1186554420 learned_addr learned my addr 172.21.15.32:0/1186554420 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:02:41.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.419+0000 7f93d9154640 1 --2- 172.21.15.32:0/1186554420 >> v2:172.21.15.121:3300/0 conn(0x7f93d406cb30 0x7f93d40cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:41.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.419+0000 7f93d3fff640 1 --2- 172.21.15.32:0/1186554420 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93d415fd60 0x7f93d406c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:41.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.419+0000 7f93d37fe640 1 -- 172.21.15.32:0/1186554420 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2363405191 0 0) 0x7f93d40b46c0 con 0x7f93d415b480 2024-04-24T23:02:41.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.419+0000 7f93d37fe640 1 -- 172.21.15.32:0/1186554420 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f93bc003520 con 0x7f93d415b480 2024-04-24T23:02:41.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.419+0000 7f93d37fe640 1 -- 172.21.15.32:0/1186554420 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3055330 0 0) 0x7f93bc003520 con 0x7f93d415b480 2024-04-24T23:02:41.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.419+0000 7f93d37fe640 1 -- 172.21.15.32:0/1186554420 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93d415fd60 msgr2=0x7f93d406c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:41.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.419+0000 7f93d37fe640 1 --2- 172.21.15.32:0/1186554420 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93d415fd60 0x7f93d406c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:41.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.419+0000 7f93d37fe640 1 -- 172.21.15.32:0/1186554420 >> v2:172.21.15.121:3300/0 conn(0x7f93d406cb30 msgr2=0x7f93d40cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:41.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.419+0000 7f93d37fe640 1 --2- 172.21.15.32:0/1186554420 >> v2:172.21.15.121:3300/0 conn(0x7f93d406cb30 0x7f93d40cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:41.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.419+0000 7f93d37fe640 1 -- 172.21.15.32:0/1186554420 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93d40b4c80 con 0x7f93d415b480 2024-04-24T23:02:41.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.419+0000 7f93d37fe640 1 -- 172.21.15.32:0/1186554420 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f93cc0019f0 con 0x7f93d415b480 2024-04-24T23:02:41.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.419+0000 7f93d37fe640 1 -- 172.21.15.32:0/1186554420 >> v1:172.21.15.32:6789/0 conn(0x7f93d415b480 legacy=0x7f93d40b5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:41.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.419+0000 7f93d37fe640 1 --2- 172.21.15.32:0/1186554420 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93d415fd60 0x7f93bc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:41.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.419+0000 7f93d37fe640 1 --2- 172.21.15.32:0/1186554420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93bc003f30 0x7f93bc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:41.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.419+0000 7f93d37fe640 1 --2- 172.21.15.32:0/1186554420 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93bc004d40 0x7f93bc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:41.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.419+0000 7f93d37fe640 1 -- 172.21.15.32:0/1186554420 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f93d40b4ff0 con 0x7f93bc003f30 2024-04-24T23:02:41.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.419+0000 7f93d37fe640 1 -- 172.21.15.32:0/1186554420 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f93d40b49a0 con 0x7f93bc004d40 2024-04-24T23:02:41.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.419+0000 7f93d37fe640 1 -- 172.21.15.32:0/1186554420 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f93bc00dbd0 con 0x7f93d415fd60 2024-04-24T23:02:41.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.419+0000 7f93d3fff640 1 --2- 172.21.15.32:0/1186554420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93bc003f30 0x7f93bc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:41.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.419+0000 7f93d8953640 1 --2- 172.21.15.32:0/1186554420 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93d415fd60 0x7f93bc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:41.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.420+0000 7f93d9154640 1 --2- 172.21.15.32:0/1186554420 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93bc004d40 0x7f93bc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:41.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.420+0000 7f93d3fff640 1 -- 172.21.15.32:0/1186554420 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93d415fd60 msgr2=0x7f93bc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:41.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.420+0000 7f93d3fff640 1 --2- 172.21.15.32:0/1186554420 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93d415fd60 0x7f93bc0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:41.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.420+0000 7f93d3fff640 1 -- 172.21.15.32:0/1186554420 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93bc004d40 msgr2=0x7f93bc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:41.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.420+0000 7f93d3fff640 1 --2- 172.21.15.32:0/1186554420 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93bc004d40 0x7f93bc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:41.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.420+0000 7f93d3fff640 1 -- 172.21.15.32:0/1186554420 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93bc003520 con 0x7f93bc003f30 2024-04-24T23:02:41.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.420+0000 7f93d3fff640 1 --2- 172.21.15.32:0/1186554420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93bc003f30 0x7f93bc004800 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f93c8009ac0 tx=0x7f93bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b677b0a5fe00ca57 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:41.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.421+0000 7f93d37fe640 1 -- 172.21.15.32:0/1186554420 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c8036070 con 0x7f93bc003f30 2024-04-24T23:02:41.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.421+0000 7f93d37fe640 1 -- 172.21.15.32:0/1186554420 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93c8031400 con 0x7f93bc003f30 2024-04-24T23:02:41.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.421+0000 7f93d37fe640 1 -- 172.21.15.32:0/1186554420 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c8009690 con 0x7f93bc003f30 2024-04-24T23:02:41.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.421+0000 7f93dabb7640 1 -- 172.21.15.32:0/1186554420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93bc003f30 msgr2=0x7f93bc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:41.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.421+0000 7f93dabb7640 1 --2- 172.21.15.32:0/1186554420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93bc003f30 0x7f93bc004800 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f93c8009ac0 tx=0x7f93bc005830 comp rx=0 tx=0).stop 2024-04-24T23:02:41.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.421+0000 7f93d9154640 1 -- 172.21.15.32:0/1186554420 reap_dead start 2024-04-24T23:02:41.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.421+0000 7f93dabb7640 1 -- 172.21.15.32:0/1186554420 shutdown_connections 2024-04-24T23:02:41.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.421+0000 7f93dabb7640 1 -- 172.21.15.32:0/1186554420 >> 172.21.15.32:0/1186554420 conn(0x7f93d406b110 msgr2=0x7f93d406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:41.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.421+0000 7f93dabb7640 1 -- 172.21.15.32:0/1186554420 shutdown_connections 2024-04-24T23:02:41.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.421+0000 7f93dabb7640 1 -- 172.21.15.32:0/1186554420 wait complete. 2024-04-24T23:02:41.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.422+0000 7f93dabb7640 1 Processor -- start 2024-04-24T23:02:41.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.422+0000 7f93dabb7640 1 -- start start 2024-04-24T23:02:41.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.423+0000 7f93dabb7640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93bc003f30 0x7f93d413ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:41.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.423+0000 7f93dabb7640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93d40706c0 0x7f93d4070b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:41.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.423+0000 7f93dabb7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93d414bd60 0x7f93d414c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:41.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.423+0000 7f93dabb7640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f93d415b860 con 0x7f93d414bd60 2024-04-24T23:02:41.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.423+0000 7f93dabb7640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f93d406c5f0 con 0x7f93bc003f30 2024-04-24T23:02:41.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.423+0000 7f93dabb7640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f93d4071e20 con 0x7f93d40706c0 2024-04-24T23:02:41.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.423+0000 7f93d9154640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93d414bd60 0x7f93d414c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:41.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.423+0000 7f93d9154640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93d414bd60 0x7f93d414c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41892/0 (socket says 172.21.15.32:41892) 2024-04-24T23:02:41.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.423+0000 7f93d9154640 1 -- 172.21.15.32:0/2202443940 learned_addr learned my addr 172.21.15.32:0/2202443940 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:41.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.423+0000 7f93d8953640 1 --2- 172.21.15.32:0/2202443940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93bc003f30 0x7f93d413ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:41.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.423+0000 7f93d3fff640 1 --2- 172.21.15.32:0/2202443940 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93d40706c0 0x7f93d4070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:41.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.424+0000 7f93d9154640 1 -- 172.21.15.32:0/2202443940 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93d40706c0 msgr2=0x7f93d4070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:41.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.424+0000 7f93d9154640 1 --2- 172.21.15.32:0/2202443940 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93d40706c0 0x7f93d4070b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:41.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.424+0000 7f93d9154640 1 -- 172.21.15.32:0/2202443940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93bc003f30 msgr2=0x7f93d413ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:41.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.424+0000 7f93d9154640 1 --2- 172.21.15.32:0/2202443940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93bc003f30 0x7f93d413ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:41.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.424+0000 7f93d9154640 1 -- 172.21.15.32:0/2202443940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93d4071210 con 0x7f93d414bd60 2024-04-24T23:02:41.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.424+0000 7f93d9154640 1 --2- 172.21.15.32:0/2202443940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93d414bd60 0x7f93d414c1e0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f93d406e860 tx=0x7f93c4011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:41.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.424+0000 7f93d17fa640 1 -- 172.21.15.32:0/2202443940 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c4019020 con 0x7f93d414bd60 2024-04-24T23:02:41.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.424+0000 7f93d17fa640 1 -- 172.21.15.32:0/2202443940 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93c4004030 con 0x7f93d414bd60 2024-04-24T23:02:41.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.424+0000 7f93dabb7640 1 -- 172.21.15.32:0/2202443940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93d40714f0 con 0x7f93d414bd60 2024-04-24T23:02:41.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.424+0000 7f93d17fa640 1 -- 172.21.15.32:0/2202443940 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c4004400 con 0x7f93d414bd60 2024-04-24T23:02:41.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.424+0000 7f93dabb7640 1 -- 172.21.15.32:0/2202443940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93d4071ab0 con 0x7f93d414bd60 2024-04-24T23:02:41.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.425+0000 7f93dabb7640 1 -- 172.21.15.32:0/2202443940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93d4068430 con 0x7f93d414bd60 2024-04-24T23:02:41.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.425+0000 7f93d17fa640 1 -- 172.21.15.32:0/2202443940 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f93c401c020 con 0x7f93d414bd60 2024-04-24T23:02:41.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.426+0000 7f93d17fa640 1 --2- 172.21.15.32:0/2202443940 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f93c006be30 0x7f93c006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:41.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.426+0000 7f93d17fa640 1 -- 172.21.15.32:0/2202443940 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5767+0+0 (secure 0 0 0) 0x7f93c408b1c0 con 0x7f93d414bd60 2024-04-24T23:02:41.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.426+0000 7f93d17fa640 1 -- 172.21.15.32:0/2202443940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=73}) -- 0x7f93c0073a20 con 0x7f93d414bd60 2024-04-24T23:02:41.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.426+0000 7f93d8953640 1 --2- 172.21.15.32:0/2202443940 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f93c006be30 0x7f93c006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:41.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.426+0000 7f93d8953640 1 --2- 172.21.15.32:0/2202443940 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f93c006be30 0x7f93c006e2f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f93cc004d00 tx=0x7f93cc006040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:41.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.430+0000 7f93d17fa640 1 -- 172.21.15.32:0/2202443940 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f93c4023020 con 0x7f93d414bd60 2024-04-24T23:02:41.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.613+0000 7f93dabb7640 1 -- 172.21.15.32:0/2202443940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f93d40afae0 con 0x7f93d414bd60 2024-04-24T23:02:41.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.613+0000 7f93d17fa640 1 -- 172.21.15.32:0/2202443940 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v72) ==== 104+0+11 (secure 0 0 0) 0x7f93c4059ba0 con 0x7f93d414bd60 2024-04-24T23:02:41.615 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-24T23:02:41.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.615+0000 7f93dabb7640 1 -- 172.21.15.32:0/2202443940 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f93c006be30 msgr2=0x7f93c006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:41.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.615+0000 7f93dabb7640 1 --2- 172.21.15.32:0/2202443940 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f93c006be30 0x7f93c006e2f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f93cc004d00 tx=0x7f93cc006040 comp rx=0 tx=0).stop 2024-04-24T23:02:41.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.615+0000 7f93dabb7640 1 -- 172.21.15.32:0/2202443940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93d414bd60 msgr2=0x7f93d414c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:41.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.615+0000 7f93dabb7640 1 --2- 172.21.15.32:0/2202443940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93d414bd60 0x7f93d414c1e0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f93d406e860 tx=0x7f93c4011cf0 comp rx=0 tx=0).stop 2024-04-24T23:02:41.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.615+0000 7f93dabb7640 1 -- 172.21.15.32:0/2202443940 shutdown_connections 2024-04-24T23:02:41.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.615+0000 7f93dabb7640 1 --2- 172.21.15.32:0/2202443940 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f93c006be30 0x7f93c006e2f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:41.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.615+0000 7f93dabb7640 1 --2- 172.21.15.32:0/2202443940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93d414bd60 0x7f93d414c1e0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:41.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.615+0000 7f93dabb7640 1 --2- 172.21.15.32:0/2202443940 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93d40706c0 0x7f93d4070b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:41.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.615+0000 7f93dabb7640 1 --2- 172.21.15.32:0/2202443940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93bc003f30 0x7f93d413ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:41.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.615+0000 7f93dabb7640 1 -- 172.21.15.32:0/2202443940 >> 172.21.15.32:0/2202443940 conn(0x7f93d406b110 msgr2=0x7f93d40b0600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:41.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.615+0000 7f93dabb7640 1 -- 172.21.15.32:0/2202443940 shutdown_connections 2024-04-24T23:02:41.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.615+0000 7f93dabb7640 1 -- 172.21.15.32:0/2202443940 wait complete. 2024-04-24T23:02:41.627 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:02:41.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.730+0000 7fd664512640 1 Processor -- start 2024-04-24T23:02:41.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.730+0000 7fd664512640 1 -- start start 2024-04-24T23:02:41.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.731+0000 7fd664512640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd65c15b480 0x7fd65c0b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:41.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.731+0000 7fd664512640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd65c15fd60 0x7fd65c06c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:41.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.731+0000 7fd664512640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd65c0cef90 con 0x7fd65c06cb30 2024-04-24T23:02:41.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.731+0000 7fd664512640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fd65c0b4ff0 con 0x7fd65c15fd60 2024-04-24T23:02:41.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.731+0000 7fd664512640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd65c0b46c0 con 0x7fd65c15b480 2024-04-24T23:02:41.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.731+0000 7fd661aad640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd65c15fd60 0x7fd65c06c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:41.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.731+0000 7fd6622ae640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd65c15b480 0x7fd65c0b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:41.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.731+0000 7fd661aad640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd65c15fd60 0x7fd65c06c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.32:36922/0 (socket says 172.21.15.32:36922) 2024-04-24T23:02:41.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.731+0000 7fd6622ae640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd65c15b480 0x7fd65c0b5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.32:60056/0 (socket says 172.21.15.32:60056) 2024-04-24T23:02:41.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.731+0000 7fd661aad640 1 -- 172.21.15.32:0/543758307 learned_addr learned my addr 172.21.15.32:0/543758307 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:41.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.731+0000 7fd662aaf640 1 --1- 172.21.15.32:0/543758307 >> v1:172.21.15.121:6789/0 conn(0x7fd65c06cb30 0x7fd65c0ccc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:33148/0 (socket says 172.21.15.32:33148) 2024-04-24T23:02:41.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.731+0000 7fd661aad640 1 -- 172.21.15.32:0/543758307 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd65c15b480 msgr2=0x7fd65c0b5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:41.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.731+0000 7fd661aad640 1 --2- 172.21.15.32:0/543758307 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd65c15b480 0x7fd65c0b5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:41.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.731+0000 7fd661aad640 1 -- 172.21.15.32:0/543758307 >> v1:172.21.15.121:6789/0 conn(0x7fd65c06cb30 legacy=0x7fd65c0ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:41.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.732+0000 7fd6612ac640 1 -- 172.21.15.32:0/543758307 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2718746465 0 0) 0x7fd65c0cef90 con 0x7fd65c06cb30 2024-04-24T23:02:41.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.732+0000 7fd661aad640 1 -- 172.21.15.32:0/543758307 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd65c0b49a0 con 0x7fd65c15fd60 2024-04-24T23:02:41.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.732+0000 7fd6622ae640 1 --2- 172.21.15.32:0/543758307 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd65c15b480 0x7fd65c0b5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:02:41.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.732+0000 7fd661aad640 1 --2- 172.21.15.32:0/543758307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd65c15fd60 0x7fd65c06c5f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd64c009870 tx=0x7fd64c02a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f29d2b968ed3e55 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:41.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.732+0000 7fd6612ac640 1 -- 172.21.15.32:0/543758307 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd64c037070 con 0x7fd65c15fd60 2024-04-24T23:02:41.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.732+0000 7fd6612ac640 1 -- 172.21.15.32:0/543758307 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd64c032440 con 0x7fd65c15fd60 2024-04-24T23:02:41.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.733+0000 7fd6612ac640 1 -- 172.21.15.32:0/543758307 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd64c032810 con 0x7fd65c15fd60 2024-04-24T23:02:41.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.733+0000 7fd664512640 1 -- 172.21.15.32:0/543758307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd65c15fd60 msgr2=0x7fd65c06c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:41.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.733+0000 7fd664512640 1 --2- 172.21.15.32:0/543758307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd65c15fd60 0x7fd65c06c5f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd64c009870 tx=0x7fd64c02a270 comp rx=0 tx=0).stop 2024-04-24T23:02:41.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.733+0000 7fd664512640 1 -- 172.21.15.32:0/543758307 shutdown_connections 2024-04-24T23:02:41.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.733+0000 7fd664512640 1 --2- 172.21.15.32:0/543758307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd65c15fd60 0x7fd65c06c5f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:41.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.733+0000 7fd664512640 1 --2- 172.21.15.32:0/543758307 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd65c15b480 0x7fd65c0b5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:41.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.733+0000 7fd664512640 1 -- 172.21.15.32:0/543758307 >> 172.21.15.32:0/543758307 conn(0x7fd65c06b110 msgr2=0x7fd65c06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:41.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.733+0000 7fd664512640 1 -- 172.21.15.32:0/543758307 shutdown_connections 2024-04-24T23:02:41.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.733+0000 7fd664512640 1 -- 172.21.15.32:0/543758307 wait complete. 2024-04-24T23:02:41.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.734+0000 7fd664512640 1 Processor -- start 2024-04-24T23:02:41.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.734+0000 7fd664512640 1 -- start start 2024-04-24T23:02:41.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.735+0000 7fd664512640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd65c06cb30 0x7fd65c15e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:41.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.735+0000 7fd664512640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd65c15b480 0x7fd65c159d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:41.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.735+0000 7fd664512640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd65c15fd60 0x7fd65c1570c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:41.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.735+0000 7fd664512640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd65c0b5480 con 0x7fd65c15b480 2024-04-24T23:02:41.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.735+0000 7fd664512640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd65c0b61a0 con 0x7fd65c15fd60 2024-04-24T23:02:41.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.735+0000 7fd664512640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd65c15b860 con 0x7fd65c06cb30 2024-04-24T23:02:41.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.735+0000 7fd661aad640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd65c15b480 0x7fd65c159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:41.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.735+0000 7fd661aad640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd65c15b480 0x7fd65c159d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41908/0 (socket says 172.21.15.32:41908) 2024-04-24T23:02:41.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.735+0000 7fd661aad640 1 -- 172.21.15.32:0/2593363742 learned_addr learned my addr 172.21.15.32:0/2593363742 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:41.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.735+0000 7fd6622ae640 1 --2- 172.21.15.32:0/2593363742 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd65c06cb30 0x7fd65c15e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:41.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.735+0000 7fd662aaf640 1 --2- 172.21.15.32:0/2593363742 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd65c15fd60 0x7fd65c1570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:41.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.736+0000 7fd661aad640 1 -- 172.21.15.32:0/2593363742 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd65c06cb30 msgr2=0x7fd65c15e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:41.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.736+0000 7fd661aad640 1 --2- 172.21.15.32:0/2593363742 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd65c06cb30 0x7fd65c15e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:41.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.736+0000 7fd661aad640 1 -- 172.21.15.32:0/2593363742 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd65c15fd60 msgr2=0x7fd65c1570c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:41.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.736+0000 7fd661aad640 1 --2- 172.21.15.32:0/2593363742 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd65c15fd60 0x7fd65c1570c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:41.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.736+0000 7fd661aad640 1 -- 172.21.15.32:0/2593363742 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd65c15a2d0 con 0x7fd65c15b480 2024-04-24T23:02:41.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.736+0000 7fd661aad640 1 --2- 172.21.15.32:0/2593363742 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd65c15b480 0x7fd65c159d90 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fd64c035040 tx=0x7fd64c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:41.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.736+0000 7fd652ffd640 1 -- 172.21.15.32:0/2593363742 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd64c037040 con 0x7fd65c15b480 2024-04-24T23:02:41.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.736+0000 7fd652ffd640 1 -- 172.21.15.32:0/2593363742 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd64c004030 con 0x7fd65c15b480 2024-04-24T23:02:41.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.736+0000 7fd664512640 1 -- 172.21.15.32:0/2593363742 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd65c158d30 con 0x7fd65c15b480 2024-04-24T23:02:41.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.736+0000 7fd652ffd640 1 -- 172.21.15.32:0/2593363742 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd64c004420 con 0x7fd65c15b480 2024-04-24T23:02:41.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.736+0000 7fd664512640 1 -- 172.21.15.32:0/2593363742 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd65c1592b0 con 0x7fd65c15b480 2024-04-24T23:02:41.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.737+0000 7fd664512640 1 -- 172.21.15.32:0/2593363742 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd65c1405a0 con 0x7fd65c15b480 2024-04-24T23:02:41.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.737+0000 7fd652ffd640 1 -- 172.21.15.32:0/2593363742 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd64c03a070 con 0x7fd65c15b480 2024-04-24T23:02:41.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.738+0000 7fd652ffd640 1 --2- 172.21.15.32:0/2593363742 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd64006be30 0x7fd64006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:41.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.738+0000 7fd652ffd640 1 -- 172.21.15.32:0/2593363742 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5767+0+0 (secure 0 0 0) 0x7fd64c0ab480 con 0x7fd65c15b480 2024-04-24T23:02:41.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.738+0000 7fd652ffd640 1 -- 172.21.15.32:0/2593363742 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=73}) -- 0x7fd640073a20 con 0x7fd65c15b480 2024-04-24T23:02:41.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.738+0000 7fd6622ae640 1 --2- 172.21.15.32:0/2593363742 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd64006be30 0x7fd64006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:41.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.739+0000 7fd6622ae640 1 --2- 172.21.15.32:0/2593363742 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd64006be30 0x7fd64006e2f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd658009560 tx=0x7fd658008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:41.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.742+0000 7fd652ffd640 1 -- 172.21.15.32:0/2593363742 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd65c1405a0 con 0x7fd65c15b480 2024-04-24T23:02:41.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.924+0000 7fd664512640 1 -- 172.21.15.32:0/2593363742 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd65c068430 con 0x7fd65c15b480 2024-04-24T23:02:41.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.924+0000 7fd652ffd640 1 -- 172.21.15.32:0/2593363742 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v72) ==== 100+0+11 (secure 0 0 0) 0x7fd64c041020 con 0x7fd65c15b480 2024-04-24T23:02:41.926 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-24T23:02:41.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.926+0000 7fd664512640 1 -- 172.21.15.32:0/2593363742 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd64006be30 msgr2=0x7fd64006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:41.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.926+0000 7fd664512640 1 --2- 172.21.15.32:0/2593363742 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd64006be30 0x7fd64006e2f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd658009560 tx=0x7fd658008040 comp rx=0 tx=0).stop 2024-04-24T23:02:41.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.926+0000 7fd664512640 1 -- 172.21.15.32:0/2593363742 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd65c15b480 msgr2=0x7fd65c159d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:41.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.926+0000 7fd664512640 1 --2- 172.21.15.32:0/2593363742 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd65c15b480 0x7fd65c159d90 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fd64c035040 tx=0x7fd64c0046c0 comp rx=0 tx=0).stop 2024-04-24T23:02:41.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.926+0000 7fd664512640 1 -- 172.21.15.32:0/2593363742 shutdown_connections 2024-04-24T23:02:41.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.926+0000 7fd664512640 1 --2- 172.21.15.32:0/2593363742 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd64006be30 0x7fd64006e2f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:41.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.926+0000 7fd664512640 1 --2- 172.21.15.32:0/2593363742 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd65c15b480 0x7fd65c159d90 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:41.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.926+0000 7fd664512640 1 --2- 172.21.15.32:0/2593363742 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd65c15fd60 0x7fd65c1570c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:41.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.926+0000 7fd664512640 1 --2- 172.21.15.32:0/2593363742 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd65c06cb30 0x7fd65c15e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:41.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.926+0000 7fd664512640 1 -- 172.21.15.32:0/2593363742 >> 172.21.15.32:0/2593363742 conn(0x7fd65c06b110 msgr2=0x7fd65c0b3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:41.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.926+0000 7fd664512640 1 -- 172.21.15.32:0/2593363742 shutdown_connections 2024-04-24T23:02:41.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:41.926+0000 7fd664512640 1 -- 172.21.15.32:0/2593363742 wait complete. 2024-04-24T23:02:41.938 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:02:41.954 INFO:tasks.ceph.mon.a.smithi032.stderr:2024-04-24T23:02:41.953+0000 7f3b831cb640 -1 log_channel(cluster) log [ERR] : Health check update: 2 full osd(s) (OSD_FULL) 2024-04-24T23:02:42.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.040+0000 7f6313542640 1 Processor -- start 2024-04-24T23:02:42.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.040+0000 7f6313542640 1 -- start start 2024-04-24T23:02:42.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.041+0000 7f6313542640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f630c0b5af0 0x7f630c0ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:42.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.041+0000 7f6313542640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f630c0cd270 con 0x7f630c162320 2024-04-24T23:02:42.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.041+0000 7f6313542640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f630c071610 con 0x7f630c162e10 2024-04-24T23:02:42.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.041+0000 7f6313542640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f630c0b5660 con 0x7f630c0b5af0 2024-04-24T23:02:42.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.041+0000 7f6310add640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f630c0b5af0 0x7f630c0ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:42.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.041+0000 7f63112de640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f630c162320 0x7f630c162700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46174/0 (socket says 172.21.15.32:46174) 2024-04-24T23:02:42.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.041+0000 7f63112de640 1 -- 172.21.15.32:0/3122669649 learned_addr learned my addr 172.21.15.32:0/3122669649 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:02:42.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.041+0000 7f6310add640 1 -- 172.21.15.32:0/3122669649 >> v1:172.21.15.121:6789/0 conn(0x7f630c162e10 legacy=0x7f630c15ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:42.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.041+0000 7f6310add640 1 -- 172.21.15.32:0/3122669649 >> v1:172.21.15.32:6789/0 conn(0x7f630c162320 legacy=0x7f630c162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:42.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.041+0000 7f6310add640 1 -- 172.21.15.32:0/3122669649 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f630c0718f0 con 0x7f630c0b5af0 2024-04-24T23:02:42.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.042+0000 7f6310add640 1 --2- 172.21.15.32:0/3122669649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f630c0b5af0 0x7f630c0ccc90 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f62fc0097a0 tx=0x7f62fc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb1bf77f65bf9d11 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:42.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.042+0000 7f6303fff640 1 -- 172.21.15.32:0/3122669649 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62fc037070 con 0x7f630c0b5af0 2024-04-24T23:02:42.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.042+0000 7f6303fff640 1 -- 172.21.15.32:0/3122669649 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62fc02ae50 con 0x7f630c0b5af0 2024-04-24T23:02:42.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.042+0000 7f6303fff640 1 -- 172.21.15.32:0/3122669649 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62fc032660 con 0x7f630c0b5af0 2024-04-24T23:02:42.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.042+0000 7f6313542640 1 -- 172.21.15.32:0/3122669649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f630c0b5af0 msgr2=0x7f630c0ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:42.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.042+0000 7f6313542640 1 --2- 172.21.15.32:0/3122669649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f630c0b5af0 0x7f630c0ccc90 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f62fc0097a0 tx=0x7f62fc02a140 comp rx=0 tx=0).stop 2024-04-24T23:02:42.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.042+0000 7f6313542640 1 -- 172.21.15.32:0/3122669649 shutdown_connections 2024-04-24T23:02:42.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.042+0000 7f6313542640 1 --2- 172.21.15.32:0/3122669649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f630c0b5af0 0x7f630c0ccc90 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:42.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.042+0000 7f6313542640 1 -- 172.21.15.32:0/3122669649 >> 172.21.15.32:0/3122669649 conn(0x7f630c0d0ae0 msgr2=0x7f630c0d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:42.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.043+0000 7f6313542640 1 -- 172.21.15.32:0/3122669649 shutdown_connections 2024-04-24T23:02:42.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.043+0000 7f6313542640 1 -- 172.21.15.32:0/3122669649 wait complete. 2024-04-24T23:02:42.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.044+0000 7f6313542640 1 Processor -- start 2024-04-24T23:02:42.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.044+0000 7f6313542640 1 -- start start 2024-04-24T23:02:42.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.044+0000 7f6313542640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f630c0b5af0 0x7f630c152820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:42.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.044+0000 7f6313542640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f630c162320 0x7f630c15fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:42.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.044+0000 7f6313542640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f630c162e10 0x7f630c1590d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:42.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.044+0000 7f6313542640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f630c071bd0 con 0x7f630c0b5af0 2024-04-24T23:02:42.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.044+0000 7f6313542640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f630c067950 con 0x7f630c162320 2024-04-24T23:02:42.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.044+0000 7f6313542640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f630c071d50 con 0x7f630c162e10 2024-04-24T23:02:42.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.044+0000 7f63112de640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f630c0b5af0 0x7f630c152820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:42.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.045+0000 7f63112de640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f630c0b5af0 0x7f630c152820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41940/0 (socket says 172.21.15.32:41940) 2024-04-24T23:02:42.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.045+0000 7f63112de640 1 -- 172.21.15.32:0/421881194 learned_addr learned my addr 172.21.15.32:0/421881194 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:42.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.045+0000 7f63112de640 1 -- 172.21.15.32:0/421881194 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f630c162e10 msgr2=0x7f630c1590d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:42.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.045+0000 7f6310add640 1 --2- 172.21.15.32:0/421881194 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f630c162320 0x7f630c15fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:42.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.045+0000 7f63112de640 1 --2- 172.21.15.32:0/421881194 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f630c162e10 0x7f630c1590d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:42.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.045+0000 7f63112de640 1 -- 172.21.15.32:0/421881194 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f630c162320 msgr2=0x7f630c15fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:42.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.045+0000 7f63112de640 1 --2- 172.21.15.32:0/421881194 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f630c162320 0x7f630c15fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:42.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.045+0000 7f63112de640 1 -- 172.21.15.32:0/421881194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f630c158070 con 0x7f630c0b5af0 2024-04-24T23:02:42.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.045+0000 7f63112de640 1 --2- 172.21.15.32:0/421881194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f630c0b5af0 0x7f630c152820 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f630800c430 tx=0x7f630800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:42.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.045+0000 7f6301ffb640 1 -- 172.21.15.32:0/421881194 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6308017070 con 0x7f630c0b5af0 2024-04-24T23:02:42.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.045+0000 7f6301ffb640 1 -- 172.21.15.32:0/421881194 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6308005ce0 con 0x7f630c0b5af0 2024-04-24T23:02:42.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.045+0000 7f6313542640 1 -- 172.21.15.32:0/421881194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f630c158310 con 0x7f630c0b5af0 2024-04-24T23:02:42.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.045+0000 7f6301ffb640 1 -- 172.21.15.32:0/421881194 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63080072f0 con 0x7f630c0b5af0 2024-04-24T23:02:42.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.045+0000 7f6313542640 1 -- 172.21.15.32:0/421881194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f630c1588d0 con 0x7f630c0b5af0 2024-04-24T23:02:42.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.046+0000 7f6313542640 1 -- 172.21.15.32:0/421881194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f630c068430 con 0x7f630c0b5af0 2024-04-24T23:02:42.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.047+0000 7f6301ffb640 1 -- 172.21.15.32:0/421881194 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f630801a070 con 0x7f630c0b5af0 2024-04-24T23:02:42.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.047+0000 7f6301ffb640 1 --2- 172.21.15.32:0/421881194 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f62f006be30 0x7f62f006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:42.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.047+0000 7f6301ffb640 1 -- 172.21.15.32:0/421881194 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5767+0+0 (secure 0 0 0) 0x7f630808a210 con 0x7f630c0b5af0 2024-04-24T23:02:42.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.047+0000 7f6301ffb640 1 -- 172.21.15.32:0/421881194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=74}) -- 0x7f62f0073a80 con 0x7f630c0b5af0 2024-04-24T23:02:42.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.047+0000 7f6310add640 1 --2- 172.21.15.32:0/421881194 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f62f006be30 0x7f62f006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:42.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.048+0000 7f6310add640 1 --2- 172.21.15.32:0/421881194 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f62f006be30 0x7f62f006e2f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f62fc034000 tx=0x7f62fc02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:42.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.051+0000 7f6301ffb640 1 -- 172.21.15.32:0/421881194 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6308058c10 con 0x7f630c0b5af0 2024-04-24T23:02:42.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.275+0000 7f6313542640 1 -- 172.21.15.32:0/421881194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f630c068dc0 con 0x7f630c0b5af0 2024-04-24T23:02:42.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.275+0000 7f6301ffb640 1 -- 172.21.15.32:0/421881194 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5187 (secure 0 0 0) 0x7f6308058510 con 0x7f630c0b5af0 2024-04-24T23:02:42.278 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:02:42.278 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":27,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3085809520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3085809520},{"type":"v1","addr":"172.21.15.121:6837","nonce":3085809520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":6078,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/1405465715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1405465715},{"type":"v1","addr":"172.21.15.32:6834","nonce":1405465715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":6081,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1160416671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":1160416671},{"type":"v1","addr":"172.21.15.32:6835","nonce":1160416671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:01:25.028304+0000","modified":"2024-04-24T23:01:27.048925+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6090},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6090":{"gid":6090,"name":"d","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6836/2650840833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2650840833},{"type":"v1","addr":"172.21.15.121:6836","nonce":2650840833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6090,"qdb_cluster":[6090]},"id":4}]} 2024-04-24T23:02:42.279 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 27 2024-04-24T23:02:42.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.277+0000 7f6313542640 1 -- 172.21.15.32:0/421881194 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f62f006be30 msgr2=0x7f62f006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:42.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.277+0000 7f6313542640 1 --2- 172.21.15.32:0/421881194 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f62f006be30 0x7f62f006e2f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f62fc034000 tx=0x7f62fc02e000 comp rx=0 tx=0).stop 2024-04-24T23:02:42.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.277+0000 7f6313542640 1 -- 172.21.15.32:0/421881194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f630c0b5af0 msgr2=0x7f630c152820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:42.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.277+0000 7f6313542640 1 --2- 172.21.15.32:0/421881194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f630c0b5af0 0x7f630c152820 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f630800c430 tx=0x7f630800c8f0 comp rx=0 tx=0).stop 2024-04-24T23:02:42.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.277+0000 7f6313542640 1 -- 172.21.15.32:0/421881194 shutdown_connections 2024-04-24T23:02:42.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.277+0000 7f6313542640 1 --2- 172.21.15.32:0/421881194 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f62f006be30 0x7f62f006e2f0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:42.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.277+0000 7f6313542640 1 --2- 172.21.15.32:0/421881194 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f630c162e10 0x7f630c1590d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:42.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.277+0000 7f6313542640 1 --2- 172.21.15.32:0/421881194 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f630c162320 0x7f630c15fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:42.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.277+0000 7f6313542640 1 --2- 172.21.15.32:0/421881194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f630c0b5af0 0x7f630c152820 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:42.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.277+0000 7f6313542640 1 -- 172.21.15.32:0/421881194 >> 172.21.15.32:0/421881194 conn(0x7f630c0d0ae0 msgr2=0x7f630c06eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:42.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.278+0000 7f6313542640 1 -- 172.21.15.32:0/421881194 shutdown_connections 2024-04-24T23:02:42.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.278+0000 7f6313542640 1 -- 172.21.15.32:0/421881194 wait complete. 2024-04-24T23:02:42.290 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:02:42.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.437+0000 7f2eeb789640 1 Processor -- start 2024-04-24T23:02:42.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.437+0000 7f2eeb789640 1 -- start start 2024-04-24T23:02:42.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.438+0000 7f2eeb789640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2ee415b480 0x7f2ee40b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:42.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.438+0000 7f2eeb789640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2ee415fd60 0x7f2ee406c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:42.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.438+0000 7f2eeb789640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ee40cef90 con 0x7f2ee406cb30 2024-04-24T23:02:42.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.438+0000 7f2eeb789640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f2ee40b4ff0 con 0x7f2ee415fd60 2024-04-24T23:02:42.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.438+0000 7f2eeb789640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f2ee40b46c0 con 0x7f2ee415b480 2024-04-24T23:02:42.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.438+0000 7f2ee8d24640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2ee415fd60 0x7f2ee406c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:42.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.438+0000 7f2ee8d24640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2ee415fd60 0x7f2ee406c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41946/0 (socket says 172.21.15.32:41946) 2024-04-24T23:02:42.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.438+0000 7f2ee8d24640 1 -- 172.21.15.32:0/881764490 learned_addr learned my addr 172.21.15.32:0/881764490 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:42.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.438+0000 7f2ee9525640 1 --2- 172.21.15.32:0/881764490 >> v2:172.21.15.121:3300/0 conn(0x7f2ee415b480 0x7f2ee40b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:42.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.438+0000 7f2ee8d24640 1 -- 172.21.15.32:0/881764490 >> v2:172.21.15.121:3300/0 conn(0x7f2ee415b480 msgr2=0x7f2ee40b5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:42.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.438+0000 7f2ee8d24640 1 --2- 172.21.15.32:0/881764490 >> v2:172.21.15.121:3300/0 conn(0x7f2ee415b480 0x7f2ee40b5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:42.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.438+0000 7f2ee8d24640 1 -- 172.21.15.32:0/881764490 >> v1:172.21.15.121:6789/0 conn(0x7f2ee406cb30 legacy=0x7f2ee40ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:42.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.438+0000 7f2ee8d24640 1 -- 172.21.15.32:0/881764490 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee40b49a0 con 0x7f2ee415fd60 2024-04-24T23:02:42.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.439+0000 7f2ee8d24640 1 --2- 172.21.15.32:0/881764490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee415fd60 0x7f2ee406c5f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f2ee406bf90 tx=0x7f2ed402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bb2dd9a291a8ca3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:42.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.439+0000 7f2edbfff640 1 -- 172.21.15.32:0/881764490 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4037070 con 0x7f2ee415fd60 2024-04-24T23:02:42.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.439+0000 7f2edbfff640 1 -- 172.21.15.32:0/881764490 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ed4032440 con 0x7f2ee415fd60 2024-04-24T23:02:42.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.439+0000 7f2edbfff640 1 -- 172.21.15.32:0/881764490 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4032810 con 0x7f2ee415fd60 2024-04-24T23:02:42.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.440+0000 7f2eeb789640 1 -- 172.21.15.32:0/881764490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee415fd60 msgr2=0x7f2ee406c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:42.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.440+0000 7f2eeb789640 1 --2- 172.21.15.32:0/881764490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee415fd60 0x7f2ee406c5f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f2ee406bf90 tx=0x7f2ed402a270 comp rx=0 tx=0).stop 2024-04-24T23:02:42.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.440+0000 7f2eeb789640 1 -- 172.21.15.32:0/881764490 shutdown_connections 2024-04-24T23:02:42.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.440+0000 7f2eeb789640 1 --2- 172.21.15.32:0/881764490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee415fd60 0x7f2ee406c5f0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:42.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.440+0000 7f2eeb789640 1 --2- 172.21.15.32:0/881764490 >> v2:172.21.15.121:3300/0 conn(0x7f2ee415b480 0x7f2ee40b5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:42.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.440+0000 7f2eeb789640 1 -- 172.21.15.32:0/881764490 >> 172.21.15.32:0/881764490 conn(0x7f2ee406b110 msgr2=0x7f2ee406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:42.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.440+0000 7f2eeb789640 1 -- 172.21.15.32:0/881764490 shutdown_connections 2024-04-24T23:02:42.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.440+0000 7f2eeb789640 1 -- 172.21.15.32:0/881764490 wait complete. 2024-04-24T23:02:42.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.441+0000 7f2eeb789640 1 Processor -- start 2024-04-24T23:02:42.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.441+0000 7f2eeb789640 1 -- start start 2024-04-24T23:02:42.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.442+0000 7f2eeb789640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee406cb30 0x7f2ee413ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:42.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.442+0000 7f2eeb789640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2ee415b480 0x7f2ee414cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:42.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.442+0000 7f2eeb789640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2ee415fd60 0x7f2ee40706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:42.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.442+0000 7f2eeb789640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee40b5480 con 0x7f2ee406cb30 2024-04-24T23:02:42.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.442+0000 7f2eeb789640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee40b61a0 con 0x7f2ee415b480 2024-04-24T23:02:42.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.442+0000 7f2eeb789640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f2ee415b860 con 0x7f2ee415fd60 2024-04-24T23:02:42.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.442+0000 7f2ee9525640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee406cb30 0x7f2ee413ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:42.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.442+0000 7f2ee9525640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee406cb30 0x7f2ee413ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41954/0 (socket says 172.21.15.32:41954) 2024-04-24T23:02:42.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.442+0000 7f2ee9525640 1 -- 172.21.15.32:0/499309975 learned_addr learned my addr 172.21.15.32:0/499309975 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:42.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.442+0000 7f2ee8d24640 1 --2- 172.21.15.32:0/499309975 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2ee415b480 0x7f2ee414cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:42.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.442+0000 7f2ee9d26640 1 --2- 172.21.15.32:0/499309975 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2ee415fd60 0x7f2ee40706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:42.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.442+0000 7f2ee9525640 1 -- 172.21.15.32:0/499309975 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2ee415fd60 msgr2=0x7f2ee40706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:42.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.442+0000 7f2ee9525640 1 --2- 172.21.15.32:0/499309975 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2ee415fd60 0x7f2ee40706c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:42.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.442+0000 7f2ee9525640 1 -- 172.21.15.32:0/499309975 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2ee415b480 msgr2=0x7f2ee414cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:42.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.442+0000 7f2ee9525640 1 --2- 172.21.15.32:0/499309975 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2ee415b480 0x7f2ee414cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:42.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.442+0000 7f2ee9525640 1 -- 172.21.15.32:0/499309975 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee414b710 con 0x7f2ee406cb30 2024-04-24T23:02:42.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.442+0000 7f2ee9d26640 1 --2- 172.21.15.32:0/499309975 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2ee415fd60 0x7f2ee40706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:02:42.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.442+0000 7f2ee9525640 1 --2- 172.21.15.32:0/499309975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee406cb30 0x7f2ee413ad30 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f2ee000d830 tx=0x7f2ee000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:42.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.443+0000 7f2ed9ffb640 1 -- 172.21.15.32:0/499309975 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee000b6c0 con 0x7f2ee406cb30 2024-04-24T23:02:42.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.443+0000 7f2ed9ffb640 1 -- 172.21.15.32:0/499309975 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ee0019070 con 0x7f2ee406cb30 2024-04-24T23:02:42.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.443+0000 7f2eeb789640 1 -- 172.21.15.32:0/499309975 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ee414b9f0 con 0x7f2ee406cb30 2024-04-24T23:02:42.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.443+0000 7f2ed9ffb640 1 -- 172.21.15.32:0/499309975 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee000ba80 con 0x7f2ee406cb30 2024-04-24T23:02:42.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.443+0000 7f2eeb789640 1 -- 172.21.15.32:0/499309975 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ee414bfb0 con 0x7f2ee406cb30 2024-04-24T23:02:42.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.444+0000 7f2eeb789640 1 -- 172.21.15.32:0/499309975 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ee4068430 con 0x7f2ee406cb30 2024-04-24T23:02:42.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.444+0000 7f2ed9ffb640 1 -- 172.21.15.32:0/499309975 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2ee001e460 con 0x7f2ee406cb30 2024-04-24T23:02:42.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.445+0000 7f2ed9ffb640 1 --2- 172.21.15.32:0/499309975 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2ec806be30 0x7f2ec806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:42.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.445+0000 7f2ed9ffb640 1 -- 172.21.15.32:0/499309975 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5767+0+0 (secure 0 0 0) 0x7f2ee008b470 con 0x7f2ee406cb30 2024-04-24T23:02:42.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.445+0000 7f2ee8d24640 1 --2- 172.21.15.32:0/499309975 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2ec806be30 0x7f2ec806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:42.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.445+0000 7f2ed9ffb640 1 -- 172.21.15.32:0/499309975 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=74}) -- 0x7f2ec8073a20 con 0x7f2ee406cb30 2024-04-24T23:02:42.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.445+0000 7f2ee8d24640 1 --2- 172.21.15.32:0/499309975 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2ec806be30 0x7f2ec806e2f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4034000 tx=0x7f2ed402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:42.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.449+0000 7f2ed9ffb640 1 -- 172.21.15.32:0/499309975 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2ee0059e70 con 0x7f2ee406cb30 2024-04-24T23:02:42.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.634+0000 7f2eeb789640 1 -- 172.21.15.32:0/499309975 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2ee406c000 con 0x7f2ee406cb30 2024-04-24T23:02:42.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.635+0000 7f2ed9ffb640 1 -- 172.21.15.32:0/499309975 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+16816 (secure 0 0 0) 0x7f2ee0059770 con 0x7f2ee406cb30 2024-04-24T23:02:42.639 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:02:42.639 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":73,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:02:41.954066+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:01:23.328944+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-24T23:01:24.010365+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["backfillfull","exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","full","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.a","mappings":[{"from":6,"to":4}]},{"pgid":"9.3a","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:02:42.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.638+0000 7f2eeb789640 1 -- 172.21.15.32:0/499309975 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2ec806be30 msgr2=0x7f2ec806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:42.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.638+0000 7f2eeb789640 1 --2- 172.21.15.32:0/499309975 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2ec806be30 0x7f2ec806e2f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4034000 tx=0x7f2ed402e000 comp rx=0 tx=0).stop 2024-04-24T23:02:42.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.638+0000 7f2eeb789640 1 -- 172.21.15.32:0/499309975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee406cb30 msgr2=0x7f2ee413ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:42.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.638+0000 7f2eeb789640 1 --2- 172.21.15.32:0/499309975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee406cb30 0x7f2ee413ad30 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f2ee000d830 tx=0x7f2ee000dcf0 comp rx=0 tx=0).stop 2024-04-24T23:02:42.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.638+0000 7f2eeb789640 1 -- 172.21.15.32:0/499309975 shutdown_connections 2024-04-24T23:02:42.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.638+0000 7f2eeb789640 1 --2- 172.21.15.32:0/499309975 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2ec806be30 0x7f2ec806e2f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:42.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.638+0000 7f2eeb789640 1 --2- 172.21.15.32:0/499309975 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2ee415fd60 0x7f2ee40706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:42.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.638+0000 7f2eeb789640 1 --2- 172.21.15.32:0/499309975 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2ee415b480 0x7f2ee414cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:42.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.638+0000 7f2eeb789640 1 --2- 172.21.15.32:0/499309975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee406cb30 0x7f2ee413ad30 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:42.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.638+0000 7f2eeb789640 1 -- 172.21.15.32:0/499309975 >> 172.21.15.32:0/499309975 conn(0x7f2ee406b110 msgr2=0x7f2ee40d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:42.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.638+0000 7f2eeb789640 1 -- 172.21.15.32:0/499309975 shutdown_connections 2024-04-24T23:02:42.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.638+0000 7f2eeb789640 1 -- 172.21.15.32:0/499309975 wait complete. 2024-04-24T23:02:42.650 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-24T23:02:42.651 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:02:42.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.750+0000 7fd4a88d4640 1 Processor -- start 2024-04-24T23:02:42.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.750+0000 7fd4a88d4640 1 -- start start 2024-04-24T23:02:42.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.751+0000 7fd4a88d4640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd4a40cfb70 0x7fd4a40ce630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:42.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.751+0000 7fd4a88d4640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4a40afdd0 con 0x7fd4a41476d0 2024-04-24T23:02:42.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.751+0000 7fd4a88d4640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4a40af010 con 0x7fd4a406b820 2024-04-24T23:02:42.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.751+0000 7fd4a88d4640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fd4a40aedc0 con 0x7fd4a40cfb70 2024-04-24T23:02:42.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.751+0000 7fd4a259c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd4a41476d0 0x7fd4a406b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46180/0 (socket says 172.21.15.32:46180) 2024-04-24T23:02:42.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.751+0000 7fd4a259c640 1 -- 172.21.15.32:0/3177161446 learned_addr learned my addr 172.21.15.32:0/3177161446 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:02:42.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.751+0000 7fd4a2d9d640 1 --2- 172.21.15.32:0/3177161446 >> v2:172.21.15.121:3300/0 conn(0x7fd4a40cfb70 0x7fd4a40ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:42.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.751+0000 7fd4a159a640 1 -- 172.21.15.32:0/3177161446 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1826838842 0 0) 0x7fd4a40afdd0 con 0x7fd4a41476d0 2024-04-24T23:02:42.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.751+0000 7fd4a159a640 1 -- 172.21.15.32:0/3177161446 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd490003520 con 0x7fd4a41476d0 2024-04-24T23:02:42.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.752+0000 7fd4a159a640 1 -- 172.21.15.32:0/3177161446 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2470688795 0 0) 0x7fd490003520 con 0x7fd4a41476d0 2024-04-24T23:02:42.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.752+0000 7fd4a159a640 1 -- 172.21.15.32:0/3177161446 >> v2:172.21.15.121:3300/0 conn(0x7fd4a40cfb70 msgr2=0x7fd4a40ce630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:42.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.752+0000 7fd4a159a640 1 --2- 172.21.15.32:0/3177161446 >> v2:172.21.15.121:3300/0 conn(0x7fd4a40cfb70 0x7fd4a40ce630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:42.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.752+0000 7fd4a159a640 1 -- 172.21.15.32:0/3177161446 >> v1:172.21.15.121:6789/0 conn(0x7fd4a406b820 legacy=0x7fd4a406d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:42.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.752+0000 7fd4a159a640 1 -- 172.21.15.32:0/3177161446 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4a40af2f0 con 0x7fd4a41476d0 2024-04-24T23:02:42.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.752+0000 7fd4a159a640 1 -- 172.21.15.32:0/3177161446 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7fd49c001690 con 0x7fd4a41476d0 2024-04-24T23:02:42.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.752+0000 7fd4a159a640 1 -- 172.21.15.32:0/3177161446 >> v1:172.21.15.32:6789/0 conn(0x7fd4a41476d0 legacy=0x7fd4a406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:42.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.752+0000 7fd4a159a640 1 --2- 172.21.15.32:0/3177161446 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd4900039b0 0x7fd490003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:42.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.752+0000 7fd4a159a640 1 --2- 172.21.15.32:0/3177161446 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd490004800 0x7fd490004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:42.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.752+0000 7fd4a159a640 1 --2- 172.21.15.32:0/3177161446 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4900051c0 0x7fd49000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:42.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.752+0000 7fd4a159a640 1 -- 172.21.15.32:0/3177161446 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a40aedc0 con 0x7fd490004800 2024-04-24T23:02:42.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.752+0000 7fd4a159a640 1 -- 172.21.15.32:0/3177161446 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd49000c160 con 0x7fd4900051c0 2024-04-24T23:02:42.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.752+0000 7fd4a159a640 1 -- 172.21.15.32:0/3177161446 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd49000c360 con 0x7fd4900039b0 2024-04-24T23:02:42.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.752+0000 7fd4a1d9b640 1 --2- 172.21.15.32:0/3177161446 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd490004800 0x7fd490004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:42.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.752+0000 7fd4a259c640 1 --2- 172.21.15.32:0/3177161446 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd4900039b0 0x7fd490003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:42.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.752+0000 7fd4a2d9d640 1 --2- 172.21.15.32:0/3177161446 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4900051c0 0x7fd49000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:42.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.752+0000 7fd4a1d9b640 1 -- 172.21.15.32:0/3177161446 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd4900039b0 msgr2=0x7fd490003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:42.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.752+0000 7fd4a1d9b640 1 --2- 172.21.15.32:0/3177161446 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd4900039b0 0x7fd490003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:42.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.752+0000 7fd4a1d9b640 1 -- 172.21.15.32:0/3177161446 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4900051c0 msgr2=0x7fd49000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:42.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.752+0000 7fd4a1d9b640 1 --2- 172.21.15.32:0/3177161446 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4900051c0 0x7fd49000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:42.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.752+0000 7fd4a1d9b640 1 -- 172.21.15.32:0/3177161446 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd490003520 con 0x7fd490004800 2024-04-24T23:02:42.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.752+0000 7fd4a2d9d640 1 -- 172.21.15.32:0/3177161446 reap_dead start 2024-04-24T23:02:42.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.752+0000 7fd4a259c640 1 --2- 172.21.15.32:0/3177161446 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd4900039b0 0x7fd490003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:02:42.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.753+0000 7fd4a1d9b640 1 --2- 172.21.15.32:0/3177161446 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd490004800 0x7fd490004c80 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fd48c001770 tx=0x7fd490005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86e88a27fdecbd7d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:42.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.753+0000 7fd4a159a640 1 -- 172.21.15.32:0/3177161446 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48c0013c0 con 0x7fd490004800 2024-04-24T23:02:42.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.753+0000 7fd4a159a640 1 -- 172.21.15.32:0/3177161446 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd48c0052d0 con 0x7fd490004800 2024-04-24T23:02:42.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.753+0000 7fd4a159a640 1 -- 172.21.15.32:0/3177161446 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48c02a8a0 con 0x7fd490004800 2024-04-24T23:02:42.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.754+0000 7fd4a88d4640 1 -- 172.21.15.32:0/3177161446 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd490004800 msgr2=0x7fd490004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:42.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.754+0000 7fd4a88d4640 1 --2- 172.21.15.32:0/3177161446 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd490004800 0x7fd490004c80 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fd48c001770 tx=0x7fd490005830 comp rx=0 tx=0).stop 2024-04-24T23:02:42.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.754+0000 7fd4a88d4640 1 -- 172.21.15.32:0/3177161446 shutdown_connections 2024-04-24T23:02:42.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.754+0000 7fd4a88d4640 1 --2- 172.21.15.32:0/3177161446 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd490004800 0x7fd490004c80 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:42.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.754+0000 7fd4a88d4640 1 -- 172.21.15.32:0/3177161446 >> 172.21.15.32:0/3177161446 conn(0x7fd4a40d0730 msgr2=0x7fd4a40d0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:42.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.754+0000 7fd4a88d4640 1 -- 172.21.15.32:0/3177161446 shutdown_connections 2024-04-24T23:02:42.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.754+0000 7fd4a88d4640 1 -- 172.21.15.32:0/3177161446 wait complete. 2024-04-24T23:02:42.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.755+0000 7fd4a88d4640 1 Processor -- start 2024-04-24T23:02:42.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.755+0000 7fd4a88d4640 1 -- start start 2024-04-24T23:02:42.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.756+0000 7fd4a88d4640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd490004800 0x7fd4a4143340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:42.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.756+0000 7fd4a88d4640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4a41f7a40 0x7fd4a41f7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:42.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.756+0000 7fd4a88d4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4a415fa80 0x7fd4a415ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:42.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.756+0000 7fd4a88d4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a4147ab0 con 0x7fd4a415fa80 2024-04-24T23:02:42.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.756+0000 7fd4a88d4640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a40af940 con 0x7fd4a41f7a40 2024-04-24T23:02:42.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.756+0000 7fd4a88d4640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd4a4148470 con 0x7fd490004800 2024-04-24T23:02:42.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.756+0000 7fd4a2d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4a415fa80 0x7fd4a415ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:42.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.756+0000 7fd4a2d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4a415fa80 0x7fd4a415ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41970/0 (socket says 172.21.15.32:41970) 2024-04-24T23:02:42.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.756+0000 7fd4a2d9d640 1 -- 172.21.15.32:0/1677679530 learned_addr learned my addr 172.21.15.32:0/1677679530 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:42.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.756+0000 7fd4a1d9b640 1 --2- 172.21.15.32:0/1677679530 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4a41f7a40 0x7fd4a41f7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:42.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.756+0000 7fd4a259c640 1 --2- 172.21.15.32:0/1677679530 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd490004800 0x7fd4a4143340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:42.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.756+0000 7fd4a2d9d640 1 -- 172.21.15.32:0/1677679530 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd490004800 msgr2=0x7fd4a4143340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:42.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.756+0000 7fd4a2d9d640 1 --2- 172.21.15.32:0/1677679530 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd490004800 0x7fd4a4143340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:42.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.756+0000 7fd4a2d9d640 1 -- 172.21.15.32:0/1677679530 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4a41f7a40 msgr2=0x7fd4a41f7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:42.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.756+0000 7fd4a2d9d640 1 --2- 172.21.15.32:0/1677679530 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4a41f7a40 0x7fd4a41f7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:42.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.756+0000 7fd4a2d9d640 1 -- 172.21.15.32:0/1677679530 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4a4160870 con 0x7fd4a415fa80 2024-04-24T23:02:42.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.756+0000 7fd4a259c640 1 --2- 172.21.15.32:0/1677679530 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd490004800 0x7fd4a4143340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:02:42.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.757+0000 7fd4a2d9d640 1 --2- 172.21.15.32:0/1677679530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4a415fa80 0x7fd4a415ff00 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fd494009fd0 tx=0x7fd49400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:42.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.757+0000 7fd49affd640 1 -- 172.21.15.32:0/1677679530 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd494019070 con 0x7fd4a415fa80 2024-04-24T23:02:42.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.757+0000 7fd49affd640 1 -- 172.21.15.32:0/1677679530 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd494011e90 con 0x7fd4a415fa80 2024-04-24T23:02:42.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.757+0000 7fd4a88d4640 1 -- 172.21.15.32:0/1677679530 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4a40c57f0 con 0x7fd4a415fa80 2024-04-24T23:02:42.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.757+0000 7fd49affd640 1 -- 172.21.15.32:0/1677679530 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd494003bd0 con 0x7fd4a415fa80 2024-04-24T23:02:42.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.757+0000 7fd4a88d4640 1 -- 172.21.15.32:0/1677679530 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4a40c5db0 con 0x7fd4a415fa80 2024-04-24T23:02:42.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.758+0000 7fd4a88d4640 1 -- 172.21.15.32:0/1677679530 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4a4068430 con 0x7fd4a415fa80 2024-04-24T23:02:42.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.758+0000 7fd49affd640 1 -- 172.21.15.32:0/1677679530 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd49401c070 con 0x7fd4a415fa80 2024-04-24T23:02:42.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.759+0000 7fd49affd640 1 --2- 172.21.15.32:0/1677679530 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd48406be30 0x7fd48406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:42.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.759+0000 7fd49affd640 1 -- 172.21.15.32:0/1677679530 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5767+0+0 (secure 0 0 0) 0x7fd49408b8a0 con 0x7fd4a415fa80 2024-04-24T23:02:42.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.759+0000 7fd49affd640 1 -- 172.21.15.32:0/1677679530 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=74}) -- 0x7fd484073a20 con 0x7fd4a415fa80 2024-04-24T23:02:42.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.759+0000 7fd4a259c640 1 --2- 172.21.15.32:0/1677679530 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd48406be30 0x7fd48406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:42.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.759+0000 7fd4a259c640 1 --2- 172.21.15.32:0/1677679530 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd48406be30 0x7fd48406e2f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd4a40740e0 tx=0x7fd49c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:42.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.763+0000 7fd49affd640 1 -- 172.21.15.32:0/1677679530 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd49408b020 con 0x7fd4a415fa80 2024-04-24T23:02:42.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.961+0000 7fd49affd640 1 -- 172.21.15.32:0/1677679530 <== mon.0 v2:172.21.15.32:3300/0 7 ==== osd_map(74..74 src has 1..74) ==== 296+0+0 (secure 0 0 0) 0x7fd494089ea0 con 0x7fd4a415fa80 2024-04-24T23:02:42.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.961+0000 7fd49affd640 1 -- 172.21.15.32:0/1677679530 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=75}) -- 0x7fd4840742a0 con 0x7fd4a415fa80 2024-04-24T23:02:42.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.984+0000 7fd4a88d4640 1 -- 172.21.15.32:0/1677679530 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd4a40b55f0 con 0x7fd4a415fa80 2024-04-24T23:02:42.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.984+0000 7fd49affd640 1 -- 172.21.15.32:0/1677679530 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5187 (secure 0 0 0) 0x7fd494010040 con 0x7fd4a415fa80 2024-04-24T23:02:42.986 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:02:42.986 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":27,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3085809520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3085809520},{"type":"v1","addr":"172.21.15.121:6837","nonce":3085809520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":6078,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/1405465715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1405465715},{"type":"v1","addr":"172.21.15.32:6834","nonce":1405465715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":6081,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1160416671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":1160416671},{"type":"v1","addr":"172.21.15.32:6835","nonce":1160416671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:01:25.028304+0000","modified":"2024-04-24T23:01:27.048925+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6090},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6090":{"gid":6090,"name":"d","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6836/2650840833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2650840833},{"type":"v1","addr":"172.21.15.121:6836","nonce":2650840833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6090,"qdb_cluster":[6090]},"id":4}]} 2024-04-24T23:02:42.986 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 27 2024-04-24T23:02:42.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.987+0000 7fd4a88d4640 1 -- 172.21.15.32:0/1677679530 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd48406be30 msgr2=0x7fd48406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:42.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.987+0000 7fd4a88d4640 1 --2- 172.21.15.32:0/1677679530 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd48406be30 0x7fd48406e2f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd4a40740e0 tx=0x7fd49c009040 comp rx=0 tx=0).stop 2024-04-24T23:02:42.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.987+0000 7fd4a88d4640 1 -- 172.21.15.32:0/1677679530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4a415fa80 msgr2=0x7fd4a415ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:42.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.987+0000 7fd4a88d4640 1 --2- 172.21.15.32:0/1677679530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4a415fa80 0x7fd4a415ff00 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fd494009fd0 tx=0x7fd49400f580 comp rx=0 tx=0).stop 2024-04-24T23:02:42.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.987+0000 7fd4a88d4640 1 -- 172.21.15.32:0/1677679530 shutdown_connections 2024-04-24T23:02:42.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.987+0000 7fd4a88d4640 1 --2- 172.21.15.32:0/1677679530 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd48406be30 0x7fd48406e2f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:42.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.987+0000 7fd4a88d4640 1 --2- 172.21.15.32:0/1677679530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4a415fa80 0x7fd4a415ff00 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:42.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.987+0000 7fd4a88d4640 1 --2- 172.21.15.32:0/1677679530 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4a41f7a40 0x7fd4a41f7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:42.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.987+0000 7fd4a88d4640 1 --2- 172.21.15.32:0/1677679530 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd490004800 0x7fd4a4143340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:42.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.987+0000 7fd4a88d4640 1 -- 172.21.15.32:0/1677679530 >> 172.21.15.32:0/1677679530 conn(0x7fd4a40d0730 msgr2=0x7fd4a4144b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:42.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.987+0000 7fd4a88d4640 1 -- 172.21.15.32:0/1677679530 shutdown_connections 2024-04-24T23:02:42.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:42.987+0000 7fd4a88d4640 1 -- 172.21.15.32:0/1677679530 wait complete. 2024-04-24T23:02:42.999 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:02:43.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.098+0000 7f839d130640 1 Processor -- start 2024-04-24T23:02:43.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.098+0000 7f839d130640 1 -- start start 2024-04-24T23:02:43.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.099+0000 7f839d130640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f839815e9b0 0x7f83980d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:43.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.099+0000 7f839d130640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f839806b820 0x7f83980b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:43.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.099+0000 7f839d130640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83980d3170 con 0x7f83980d4210 2024-04-24T23:02:43.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.099+0000 7f839d130640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f83980d3450 con 0x7f839806b820 2024-04-24T23:02:43.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.099+0000 7f839d130640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f83980d3730 con 0x7f839815e9b0 2024-04-24T23:02:43.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.099+0000 7f839759e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f839806b820 0x7f83980b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:43.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.099+0000 7f839659c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f83980d4210 0x7f839806b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46192/0 (socket says 172.21.15.32:46192) 2024-04-24T23:02:43.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.099+0000 7f839659c640 1 -- 172.21.15.32:0/374069641 learned_addr learned my addr 172.21.15.32:0/374069641 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:02:43.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.100+0000 7f8396d9d640 1 --2- 172.21.15.32:0/374069641 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f839815e9b0 0x7f83980d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:43.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.100+0000 7f839759e640 1 -- 172.21.15.32:0/374069641 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f839815e9b0 msgr2=0x7f83980d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:43.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.100+0000 7f839759e640 1 --2- 172.21.15.32:0/374069641 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f839815e9b0 0x7f83980d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:43.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.100+0000 7f839759e640 1 -- 172.21.15.32:0/374069641 >> v1:172.21.15.32:6789/0 conn(0x7f83980d4210 legacy=0x7f839806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:43.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.100+0000 7f8395d9b640 1 -- 172.21.15.32:0/374069641 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3553332259 0 0) 0x7f83980d3170 con 0x7f83980d4210 2024-04-24T23:02:43.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.100+0000 7f839759e640 1 -- 172.21.15.32:0/374069641 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83980d3a10 con 0x7f839806b820 2024-04-24T23:02:43.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.100+0000 7f839759e640 1 --2- 172.21.15.32:0/374069641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f839806b820 0x7f83980b78f0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f8388009d90 tx=0x7f838802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb3911ded2a9e6e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:43.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.100+0000 7f8395d9b640 1 -- 172.21.15.32:0/374069641 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8388007740 con 0x7f839806b820 2024-04-24T23:02:43.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.100+0000 7f8395d9b640 1 -- 172.21.15.32:0/374069641 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83880078e0 con 0x7f839806b820 2024-04-24T23:02:43.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.100+0000 7f8395d9b640 1 -- 172.21.15.32:0/374069641 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8388007cb0 con 0x7f839806b820 2024-04-24T23:02:43.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.101+0000 7f839d130640 1 -- 172.21.15.32:0/374069641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f839806b820 msgr2=0x7f83980b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:43.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.101+0000 7f839d130640 1 --2- 172.21.15.32:0/374069641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f839806b820 0x7f83980b78f0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f8388009d90 tx=0x7f838802a660 comp rx=0 tx=0).stop 2024-04-24T23:02:43.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.101+0000 7f839d130640 1 -- 172.21.15.32:0/374069641 shutdown_connections 2024-04-24T23:02:43.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.101+0000 7f839d130640 1 --2- 172.21.15.32:0/374069641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f839806b820 0x7f83980b78f0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:43.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.101+0000 7f839d130640 1 --2- 172.21.15.32:0/374069641 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f839815e9b0 0x7f83980d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:43.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.101+0000 7f839d130640 1 -- 172.21.15.32:0/374069641 >> 172.21.15.32:0/374069641 conn(0x7f83980d11d0 msgr2=0x7f83980d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:43.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.101+0000 7f839d130640 1 -- 172.21.15.32:0/374069641 shutdown_connections 2024-04-24T23:02:43.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.101+0000 7f839d130640 1 -- 172.21.15.32:0/374069641 wait complete. 2024-04-24T23:02:43.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.102+0000 7f839d130640 1 Processor -- start 2024-04-24T23:02:43.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.103+0000 7f839d130640 1 -- start start 2024-04-24T23:02:43.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.103+0000 7f839d130640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f839806b820 0x7f83980cbc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:43.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.103+0000 7f839d130640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f83980d4210 0x7f8398143620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:43.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.103+0000 7f839d130640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f839815e9b0 0x7f8398143b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:43.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.103+0000 7f839d130640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f83980ba0e0 con 0x7f839815e9b0 2024-04-24T23:02:43.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.103+0000 7f839d130640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f83980d2400 con 0x7f839806b820 2024-04-24T23:02:43.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.103+0000 7f839d130640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f83980ba260 con 0x7f83980d4210 2024-04-24T23:02:43.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.103+0000 7f839759e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f839815e9b0 0x7f8398143b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:43.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.103+0000 7f839759e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f839815e9b0 0x7f8398143b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41992/0 (socket says 172.21.15.32:41992) 2024-04-24T23:02:43.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.103+0000 7f839759e640 1 -- 172.21.15.32:0/3515495585 learned_addr learned my addr 172.21.15.32:0/3515495585 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:43.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.104+0000 7f8396d9d640 1 --2- 172.21.15.32:0/3515495585 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f839806b820 0x7f83980cbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:43.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.104+0000 7f839659c640 1 --2- 172.21.15.32:0/3515495585 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f83980d4210 0x7f8398143620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:43.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.104+0000 7f839759e640 1 -- 172.21.15.32:0/3515495585 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f83980d4210 msgr2=0x7f8398143620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:43.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.104+0000 7f839759e640 1 --2- 172.21.15.32:0/3515495585 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f83980d4210 0x7f8398143620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:43.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.104+0000 7f839759e640 1 -- 172.21.15.32:0/3515495585 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f839806b820 msgr2=0x7f83980cbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:43.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.104+0000 7f839759e640 1 --2- 172.21.15.32:0/3515495585 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f839806b820 0x7f83980cbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:43.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.104+0000 7f839759e640 1 -- 172.21.15.32:0/3515495585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83981591a0 con 0x7f839815e9b0 2024-04-24T23:02:43.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.104+0000 7f8396d9d640 1 --2- 172.21.15.32:0/3515495585 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f839806b820 0x7f83980cbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:02:43.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.104+0000 7f839759e640 1 --2- 172.21.15.32:0/3515495585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f839815e9b0 0x7f8398143b60 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f8388009eb0 tx=0x7f8388004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:43.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.104+0000 7f837f7fe640 1 -- 172.21.15.32:0/3515495585 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8388040070 con 0x7f839815e9b0 2024-04-24T23:02:43.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.104+0000 7f837f7fe640 1 -- 172.21.15.32:0/3515495585 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8388031c90 con 0x7f839815e9b0 2024-04-24T23:02:43.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.104+0000 7f839d130640 1 -- 172.21.15.32:0/3515495585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8398156650 con 0x7f839815e9b0 2024-04-24T23:02:43.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.104+0000 7f837f7fe640 1 -- 172.21.15.32:0/3515495585 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8388045680 con 0x7f839815e9b0 2024-04-24T23:02:43.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.104+0000 7f839d130640 1 -- 172.21.15.32:0/3515495585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8398151c10 con 0x7f839815e9b0 2024-04-24T23:02:43.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.105+0000 7f839d130640 1 -- 172.21.15.32:0/3515495585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8398068430 con 0x7f839815e9b0 2024-04-24T23:02:43.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.106+0000 7f837f7fe640 1 -- 172.21.15.32:0/3515495585 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8388039040 con 0x7f839815e9b0 2024-04-24T23:02:43.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.106+0000 7f837f7fe640 1 --2- 172.21.15.32:0/3515495585 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f838406be30 0x7f838406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:43.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.106+0000 7f837f7fe640 1 -- 172.21.15.32:0/3515495585 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5767+0+0 (secure 0 0 0) 0x7f83880ac8e0 con 0x7f839815e9b0 2024-04-24T23:02:43.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.106+0000 7f837f7fe640 1 -- 172.21.15.32:0/3515495585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=75}) -- 0x7f8384073a20 con 0x7f839815e9b0 2024-04-24T23:02:43.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.106+0000 7f8396d9d640 1 --2- 172.21.15.32:0/3515495585 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f838406be30 0x7f838406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:43.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.107+0000 7f8396d9d640 1 --2- 172.21.15.32:0/3515495585 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f838406be30 0x7f838406e2f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8390006fd0 tx=0x7f8390008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:43.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.110+0000 7f837f7fe640 1 -- 172.21.15.32:0/3515495585 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f838807b430 con 0x7f839815e9b0 2024-04-24T23:02:43.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.336+0000 7f839d130640 1 -- 172.21.15.32:0/3515495585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f83980c9400 con 0x7f839815e9b0 2024-04-24T23:02:43.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.336+0000 7f837f7fe640 1 -- 172.21.15.32:0/3515495585 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5187 (secure 0 0 0) 0x7f838807ad30 con 0x7f839815e9b0 2024-04-24T23:02:43.338 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:02:43.338 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":27,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3085809520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3085809520},{"type":"v1","addr":"172.21.15.121:6837","nonce":3085809520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":6078,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/1405465715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1405465715},{"type":"v1","addr":"172.21.15.32:6834","nonce":1405465715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":6081,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1160416671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":1160416671},{"type":"v1","addr":"172.21.15.32:6835","nonce":1160416671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:01:25.028304+0000","modified":"2024-04-24T23:01:27.048925+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6090},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6090":{"gid":6090,"name":"d","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6836/2650840833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2650840833},{"type":"v1","addr":"172.21.15.121:6836","nonce":2650840833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6090,"qdb_cluster":[6090]},"id":4}]} 2024-04-24T23:02:43.339 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 27 2024-04-24T23:02:43.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.339+0000 7f839d130640 1 -- 172.21.15.32:0/3515495585 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f838406be30 msgr2=0x7f838406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:43.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.339+0000 7f839d130640 1 --2- 172.21.15.32:0/3515495585 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f838406be30 0x7f838406e2f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8390006fd0 tx=0x7f8390008040 comp rx=0 tx=0).stop 2024-04-24T23:02:43.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.339+0000 7f839d130640 1 -- 172.21.15.32:0/3515495585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f839815e9b0 msgr2=0x7f8398143b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:43.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.339+0000 7f839d130640 1 --2- 172.21.15.32:0/3515495585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f839815e9b0 0x7f8398143b60 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f8388009eb0 tx=0x7f8388004a40 comp rx=0 tx=0).stop 2024-04-24T23:02:43.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.339+0000 7f839d130640 1 -- 172.21.15.32:0/3515495585 shutdown_connections 2024-04-24T23:02:43.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.339+0000 7f839d130640 1 --2- 172.21.15.32:0/3515495585 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f838406be30 0x7f838406e2f0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:43.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.339+0000 7f839d130640 1 --2- 172.21.15.32:0/3515495585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f839815e9b0 0x7f8398143b60 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:43.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.339+0000 7f839d130640 1 --2- 172.21.15.32:0/3515495585 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f83980d4210 0x7f8398143620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:43.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.339+0000 7f839d130640 1 --2- 172.21.15.32:0/3515495585 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f839806b820 0x7f83980cbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:43.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.339+0000 7f839d130640 1 -- 172.21.15.32:0/3515495585 >> 172.21.15.32:0/3515495585 conn(0x7f83980d11d0 msgr2=0x7f8398154ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:43.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.339+0000 7f839d130640 1 -- 172.21.15.32:0/3515495585 shutdown_connections 2024-04-24T23:02:43.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.339+0000 7f839d130640 1 -- 172.21.15.32:0/3515495585 wait complete. 2024-04-24T23:02:43.351 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:02:43.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.451+0000 7f79c8888640 1 Processor -- start 2024-04-24T23:02:43.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.451+0000 7f79c8888640 1 -- start start 2024-04-24T23:02:43.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.452+0000 7f79c8888640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f79c406c5f0 0x7f79c406c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:43.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.452+0000 7f79c8888640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f79c40b44d0 0x7f79c40b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:43.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.452+0000 7f79c8888640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79c40b6430 con 0x7f79c40af890 2024-04-24T23:02:43.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.452+0000 7f79c8888640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f79c4156f80 con 0x7f79c406c5f0 2024-04-24T23:02:43.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.452+0000 7f79c8888640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f79c40b6150 con 0x7f79c40b44d0 2024-04-24T23:02:43.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.452+0000 7f79c259c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f79c406c5f0 0x7f79c406c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:43.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.452+0000 7f79c259c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f79c406c5f0 0x7f79c406c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42002/0 (socket says 172.21.15.32:42002) 2024-04-24T23:02:43.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.452+0000 7f79c259c640 1 -- 172.21.15.32:0/3479829515 learned_addr learned my addr 172.21.15.32:0/3479829515 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:43.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.452+0000 7f79c1d9b640 1 --2- 172.21.15.32:0/3479829515 >> v2:172.21.15.121:3300/0 conn(0x7f79c40b44d0 0x7f79c40b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:43.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.452+0000 7f79c259c640 1 -- 172.21.15.32:0/3479829515 >> v2:172.21.15.121:3300/0 conn(0x7f79c40b44d0 msgr2=0x7f79c40b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:43.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.452+0000 7f79c259c640 1 --2- 172.21.15.32:0/3479829515 >> v2:172.21.15.121:3300/0 conn(0x7f79c40b44d0 0x7f79c40b5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:43.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.452+0000 7f79c259c640 1 -- 172.21.15.32:0/3479829515 >> v1:172.21.15.121:6789/0 conn(0x7f79c40af890 legacy=0x7f79c40cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:43.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.452+0000 7f79c259c640 1 -- 172.21.15.32:0/3479829515 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79c40b39f0 con 0x7f79c406c5f0 2024-04-24T23:02:43.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.453+0000 7f79c259c640 1 --2- 172.21.15.32:0/3479829515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c406c5f0 0x7f79c406c9d0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f79c4073080 tx=0x7f79b402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ccdd8ac89623cbe server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:43.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.453+0000 7f79c159a640 1 -- 172.21.15.32:0/3479829515 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b4037070 con 0x7f79c406c5f0 2024-04-24T23:02:43.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.453+0000 7f79c159a640 1 -- 172.21.15.32:0/3479829515 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79b4032440 con 0x7f79c406c5f0 2024-04-24T23:02:43.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.453+0000 7f79c159a640 1 -- 172.21.15.32:0/3479829515 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b4032810 con 0x7f79c406c5f0 2024-04-24T23:02:43.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.453+0000 7f79c8888640 1 -- 172.21.15.32:0/3479829515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c406c5f0 msgr2=0x7f79c406c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:43.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.453+0000 7f79c8888640 1 --2- 172.21.15.32:0/3479829515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c406c5f0 0x7f79c406c9d0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f79c4073080 tx=0x7f79b402a230 comp rx=0 tx=0).stop 2024-04-24T23:02:43.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.454+0000 7f79c8888640 1 -- 172.21.15.32:0/3479829515 shutdown_connections 2024-04-24T23:02:43.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.454+0000 7f79c8888640 1 --2- 172.21.15.32:0/3479829515 >> v2:172.21.15.121:3300/0 conn(0x7f79c40b44d0 0x7f79c40b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:43.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.454+0000 7f79c8888640 1 --2- 172.21.15.32:0/3479829515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c406c5f0 0x7f79c406c9d0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:43.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.454+0000 7f79c8888640 1 -- 172.21.15.32:0/3479829515 >> 172.21.15.32:0/3479829515 conn(0x7f79c406b110 msgr2=0x7f79c406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:43.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.454+0000 7f79c8888640 1 -- 172.21.15.32:0/3479829515 shutdown_connections 2024-04-24T23:02:43.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.454+0000 7f79c8888640 1 -- 172.21.15.32:0/3479829515 wait complete. 2024-04-24T23:02:43.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.455+0000 7f79c8888640 1 Processor -- start 2024-04-24T23:02:43.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.455+0000 7f79c8888640 1 -- start start 2024-04-24T23:02:43.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.456+0000 7f79c8888640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f79c406c5f0 0x7f79c41582e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:43.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.456+0000 7f79c8888640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c40af890 0x7f79c41599e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:43.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.456+0000 7f79c8888640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79c40b44d0 0x7f79c40cef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:43.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.456+0000 7f79c8888640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f79c406cf10 con 0x7f79c40af890 2024-04-24T23:02:43.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.456+0000 7f79c8888640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f79c40b3fb0 con 0x7f79c40b44d0 2024-04-24T23:02:43.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.456+0000 7f79c8888640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f79c40b5dd0 con 0x7f79c406c5f0 2024-04-24T23:02:43.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.456+0000 7f79c1d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c40af890 0x7f79c41599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:43.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.456+0000 7f79c1d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c40af890 0x7f79c41599e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42008/0 (socket says 172.21.15.32:42008) 2024-04-24T23:02:43.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.456+0000 7f79c1d9b640 1 -- 172.21.15.32:0/3813235302 learned_addr learned my addr 172.21.15.32:0/3813235302 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:43.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.456+0000 7f79c259c640 1 --2- 172.21.15.32:0/3813235302 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f79c406c5f0 0x7f79c41582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:43.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.456+0000 7f79c2d9d640 1 --2- 172.21.15.32:0/3813235302 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79c40b44d0 0x7f79c40cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:43.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.456+0000 7f79c1d9b640 1 -- 172.21.15.32:0/3813235302 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f79c406c5f0 msgr2=0x7f79c41582e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:43.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.456+0000 7f79c1d9b640 1 --2- 172.21.15.32:0/3813235302 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f79c406c5f0 0x7f79c41582e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:43.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.456+0000 7f79c1d9b640 1 -- 172.21.15.32:0/3813235302 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79c40b44d0 msgr2=0x7f79c40cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:43.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.456+0000 7f79c1d9b640 1 --2- 172.21.15.32:0/3813235302 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79c40b44d0 0x7f79c40cef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:43.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.456+0000 7f79c1d9b640 1 -- 172.21.15.32:0/3813235302 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79c41588c0 con 0x7f79c40af890 2024-04-24T23:02:43.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.456+0000 7f79c2d9d640 1 --2- 172.21.15.32:0/3813235302 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79c40b44d0 0x7f79c40cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:02:43.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.456+0000 7f79c1d9b640 1 --2- 172.21.15.32:0/3813235302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c40af890 0x7f79c41599e0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f79b800d800 tx=0x7f79b800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:43.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.456+0000 7f79b2ffd640 1 -- 172.21.15.32:0/3813235302 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b8004030 con 0x7f79c40af890 2024-04-24T23:02:43.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.456+0000 7f79b2ffd640 1 -- 172.21.15.32:0/3813235302 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79b80041d0 con 0x7f79c40af890 2024-04-24T23:02:43.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.457+0000 7f79b2ffd640 1 -- 172.21.15.32:0/3813235302 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b800b6c0 con 0x7f79c40af890 2024-04-24T23:02:43.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.457+0000 7f79c8888640 1 -- 172.21.15.32:0/3813235302 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79c4158ba0 con 0x7f79c40af890 2024-04-24T23:02:43.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.457+0000 7f79c8888640 1 -- 172.21.15.32:0/3813235302 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79c4159160 con 0x7f79c40af890 2024-04-24T23:02:43.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.458+0000 7f79c8888640 1 -- 172.21.15.32:0/3813235302 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79c4068430 con 0x7f79c40af890 2024-04-24T23:02:43.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.458+0000 7f79b2ffd640 1 -- 172.21.15.32:0/3813235302 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f79b800b860 con 0x7f79c40af890 2024-04-24T23:02:43.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.458+0000 7f79b2ffd640 1 --2- 172.21.15.32:0/3813235302 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f79a406be30 0x7f79a406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:43.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.459+0000 7f79b2ffd640 1 -- 172.21.15.32:0/3813235302 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5767+0+0 (secure 0 0 0) 0x7f79b808b280 con 0x7f79c40af890 2024-04-24T23:02:43.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.459+0000 7f79b2ffd640 1 -- 172.21.15.32:0/3813235302 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=75}) -- 0x7f79a4073a20 con 0x7f79c40af890 2024-04-24T23:02:43.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.459+0000 7f79c259c640 1 --2- 172.21.15.32:0/3813235302 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f79a406be30 0x7f79a406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:43.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.459+0000 7f79c259c640 1 --2- 172.21.15.32:0/3813235302 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f79a406be30 0x7f79a406e2f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f79b4034000 tx=0x7f79b402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:43.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.463+0000 7f79b2ffd640 1 -- 172.21.15.32:0/3813235302 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f79b801c360 con 0x7f79c40af890 2024-04-24T23:02:43.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.646+0000 7f79c8888640 1 -- 172.21.15.32:0/3813235302 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f79c40b5670 con 0x7f79c40af890 2024-04-24T23:02:43.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.647+0000 7f79b2ffd640 1 -- 172.21.15.32:0/3813235302 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+16809 (secure 0 0 0) 0x7f79b8059b50 con 0x7f79c40af890 2024-04-24T23:02:43.649 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:02:43.649 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":74,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:02:42.961031+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:01:23.328944+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-24T23:01:24.010365+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["backfillfull","exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","full","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.a","mappings":[{"from":6,"to":4}]},{"pgid":"9.3a","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:02:43.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.649+0000 7f79c8888640 1 -- 172.21.15.32:0/3813235302 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f79a406be30 msgr2=0x7f79a406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:43.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.649+0000 7f79c8888640 1 --2- 172.21.15.32:0/3813235302 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f79a406be30 0x7f79a406e2f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f79b4034000 tx=0x7f79b402e000 comp rx=0 tx=0).stop 2024-04-24T23:02:43.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.649+0000 7f79c8888640 1 -- 172.21.15.32:0/3813235302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c40af890 msgr2=0x7f79c41599e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:43.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.649+0000 7f79c8888640 1 --2- 172.21.15.32:0/3813235302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c40af890 0x7f79c41599e0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f79b800d800 tx=0x7f79b800dcc0 comp rx=0 tx=0).stop 2024-04-24T23:02:43.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.649+0000 7f79c8888640 1 -- 172.21.15.32:0/3813235302 shutdown_connections 2024-04-24T23:02:43.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.649+0000 7f79c8888640 1 --2- 172.21.15.32:0/3813235302 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f79a406be30 0x7f79a406e2f0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:43.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.649+0000 7f79c8888640 1 --2- 172.21.15.32:0/3813235302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c40af890 0x7f79c41599e0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:43.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.649+0000 7f79c8888640 1 --2- 172.21.15.32:0/3813235302 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79c40b44d0 0x7f79c40cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:43.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.649+0000 7f79c8888640 1 --2- 172.21.15.32:0/3813235302 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f79c406c5f0 0x7f79c41582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:43.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.649+0000 7f79c8888640 1 -- 172.21.15.32:0/3813235302 >> 172.21.15.32:0/3813235302 conn(0x7f79c406b110 msgr2=0x7f79c40b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:43.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.649+0000 7f79c8888640 1 -- 172.21.15.32:0/3813235302 shutdown_connections 2024-04-24T23:02:43.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.649+0000 7f79c8888640 1 -- 172.21.15.32:0/3813235302 wait complete. 2024-04-24T23:02:43.662 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-24T23:02:43.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.762+0000 7f5a9a45d640 1 Processor -- start 2024-04-24T23:02:43.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.762+0000 7f5a9a45d640 1 -- start start 2024-04-24T23:02:43.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.763+0000 7f5a9a45d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5a9406b820 0x7f5a940d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:43.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.763+0000 7f5a9a45d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a940d3170 con 0x7f5a9415e9b0 2024-04-24T23:02:43.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.763+0000 7f5a9a45d640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a940d3450 con 0x7f5a940d4210 2024-04-24T23:02:43.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.763+0000 7f5a9a45d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f5a940d3730 con 0x7f5a9406b820 2024-04-24T23:02:43.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.763+0000 7f5a93fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5a9415e9b0 0x7f5a9406b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46196/0 (socket says 172.21.15.32:46196) 2024-04-24T23:02:43.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.763+0000 7f5a93fff640 1 -- 172.21.15.32:0/590339841 learned_addr learned my addr 172.21.15.32:0/590339841 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:02:43.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.764+0000 7f5a92ffd640 1 -- 172.21.15.32:0/590339841 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1557166424 0 0) 0x7f5a940d3170 con 0x7f5a9415e9b0 2024-04-24T23:02:43.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.764+0000 7f5a92ffd640 1 -- 172.21.15.32:0/590339841 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a7c003520 con 0x7f5a9415e9b0 2024-04-24T23:02:43.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.764+0000 7f5a937fe640 1 --2- 172.21.15.32:0/590339841 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5a9406b820 0x7f5a940d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:43.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.764+0000 7f5a92ffd640 1 -- 172.21.15.32:0/590339841 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3115454943 0 0) 0x7f5a7c003520 con 0x7f5a9415e9b0 2024-04-24T23:02:43.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.764+0000 7f5a92ffd640 1 -- 172.21.15.32:0/590339841 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5a9406b820 msgr2=0x7f5a940d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:43.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.764+0000 7f5a92ffd640 1 --2- 172.21.15.32:0/590339841 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5a9406b820 0x7f5a940d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:43.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.764+0000 7f5a92ffd640 1 -- 172.21.15.32:0/590339841 >> v1:172.21.15.121:6789/0 conn(0x7f5a940d4210 legacy=0x7f5a940b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:43.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.764+0000 7f5a92ffd640 1 -- 172.21.15.32:0/590339841 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a940d3a10 con 0x7f5a9415e9b0 2024-04-24T23:02:43.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.764+0000 7f5a92ffd640 1 -- 172.21.15.32:0/590339841 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f5a84001670 con 0x7f5a9415e9b0 2024-04-24T23:02:43.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.764+0000 7f5a92ffd640 1 -- 172.21.15.32:0/590339841 >> v1:172.21.15.32:6789/0 conn(0x7f5a9415e9b0 legacy=0x7f5a9406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:43.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.764+0000 7f5a92ffd640 1 --2- 172.21.15.32:0/590339841 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a7c0039e0 0x7f5a7c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:43.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.764+0000 7f5a92ffd640 1 --2- 172.21.15.32:0/590339841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a7c004800 0x7f5a7c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:43.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.764+0000 7f5a92ffd640 1 --2- 172.21.15.32:0/590339841 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5a9406b820 0x7f5a7c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:43.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.764+0000 7f5a92ffd640 1 -- 172.21.15.32:0/590339841 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5a940d3730 con 0x7f5a7c004800 2024-04-24T23:02:43.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.764+0000 7f5a92ffd640 1 -- 172.21.15.32:0/590339841 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7c00baa0 con 0x7f5a7c0039e0 2024-04-24T23:02:43.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.764+0000 7f5a92ffd640 1 -- 172.21.15.32:0/590339841 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f5a7c00bca0 con 0x7f5a9406b820 2024-04-24T23:02:43.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.764+0000 7f5a937fe640 1 --2- 172.21.15.32:0/590339841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a7c004800 0x7f5a7c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:43.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.764+0000 7f5a93fff640 1 --2- 172.21.15.32:0/590339841 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a7c0039e0 0x7f5a7c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:43.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.764+0000 7f5a989fa640 1 --2- 172.21.15.32:0/590339841 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5a9406b820 0x7f5a7c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:43.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.764+0000 7f5a937fe640 1 -- 172.21.15.32:0/590339841 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5a9406b820 msgr2=0x7f5a7c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:43.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.764+0000 7f5a937fe640 1 --2- 172.21.15.32:0/590339841 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5a9406b820 0x7f5a7c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:43.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.764+0000 7f5a937fe640 1 -- 172.21.15.32:0/590339841 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a7c0039e0 msgr2=0x7f5a7c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:43.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.764+0000 7f5a937fe640 1 --2- 172.21.15.32:0/590339841 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a7c0039e0 0x7f5a7c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:43.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.764+0000 7f5a937fe640 1 -- 172.21.15.32:0/590339841 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a7c003520 con 0x7f5a7c004800 2024-04-24T23:02:43.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.764+0000 7f5a989fa640 1 --2- 172.21.15.32:0/590339841 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5a9406b820 0x7f5a7c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:02:43.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.765+0000 7f5a937fe640 1 --2- 172.21.15.32:0/590339841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a7c004800 0x7f5a7c004c80 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f5a88009ac0 tx=0x7f5a7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a36ea99adec9cafd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:43.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.765+0000 7f5a92ffd640 1 -- 172.21.15.32:0/590339841 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a88036070 con 0x7f5a7c004800 2024-04-24T23:02:43.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.765+0000 7f5a92ffd640 1 -- 172.21.15.32:0/590339841 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a88031400 con 0x7f5a7c004800 2024-04-24T23:02:43.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.765+0000 7f5a92ffd640 1 -- 172.21.15.32:0/590339841 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a88009690 con 0x7f5a7c004800 2024-04-24T23:02:43.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.766+0000 7f5a9a45d640 1 -- 172.21.15.32:0/590339841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a7c004800 msgr2=0x7f5a7c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:43.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.766+0000 7f5a9a45d640 1 --2- 172.21.15.32:0/590339841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a7c004800 0x7f5a7c004c80 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f5a88009ac0 tx=0x7f5a7c005830 comp rx=0 tx=0).stop 2024-04-24T23:02:43.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.766+0000 7f5a989fa640 1 -- 172.21.15.32:0/590339841 reap_dead start 2024-04-24T23:02:43.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.766+0000 7f5a9a45d640 1 -- 172.21.15.32:0/590339841 shutdown_connections 2024-04-24T23:02:43.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.766+0000 7f5a9a45d640 1 -- 172.21.15.32:0/590339841 >> 172.21.15.32:0/590339841 conn(0x7f5a940d11d0 msgr2=0x7f5a940d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:43.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.766+0000 7f5a9a45d640 1 -- 172.21.15.32:0/590339841 shutdown_connections 2024-04-24T23:02:43.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.766+0000 7f5a9a45d640 1 -- 172.21.15.32:0/590339841 wait complete. 2024-04-24T23:02:43.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.767+0000 7f5a9a45d640 1 Processor -- start 2024-04-24T23:02:43.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.767+0000 7f5a9a45d640 1 -- start start 2024-04-24T23:02:43.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.768+0000 7f5a9a45d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a7c004800 0x7f5a941f1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:43.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.768+0000 7f5a9a45d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5a940cdd50 0x7f5a940ce1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:43.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.768+0000 7f5a9a45d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a941583b0 0x7f5a94158830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:43.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.768+0000 7f5a9a45d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5a940ba0e0 con 0x7f5a941583b0 2024-04-24T23:02:43.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.768+0000 7f5a9a45d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5a940d2400 con 0x7f5a7c004800 2024-04-24T23:02:43.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.768+0000 7f5a9a45d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f5a940ba260 con 0x7f5a940cdd50 2024-04-24T23:02:43.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.768+0000 7f5a989fa640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a941583b0 0x7f5a94158830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:43.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.768+0000 7f5a989fa640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a941583b0 0x7f5a94158830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42032/0 (socket says 172.21.15.32:42032) 2024-04-24T23:02:43.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.768+0000 7f5a989fa640 1 -- 172.21.15.32:0/493358082 learned_addr learned my addr 172.21.15.32:0/493358082 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:43.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.768+0000 7f5a93fff640 1 --2- 172.21.15.32:0/493358082 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a7c004800 0x7f5a941f1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:43.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.768+0000 7f5a937fe640 1 --2- 172.21.15.32:0/493358082 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5a940cdd50 0x7f5a940ce1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:43.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.768+0000 7f5a989fa640 1 -- 172.21.15.32:0/493358082 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5a940cdd50 msgr2=0x7f5a940ce1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:43.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.768+0000 7f5a989fa640 1 --2- 172.21.15.32:0/493358082 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5a940cdd50 0x7f5a940ce1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:43.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.769+0000 7f5a989fa640 1 -- 172.21.15.32:0/493358082 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a7c004800 msgr2=0x7f5a941f1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:43.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.769+0000 7f5a989fa640 1 --2- 172.21.15.32:0/493358082 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a7c004800 0x7f5a941f1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:43.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.769+0000 7f5a989fa640 1 -- 172.21.15.32:0/493358082 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a941f1e90 con 0x7f5a941583b0 2024-04-24T23:02:43.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.769+0000 7f5a989fa640 1 --2- 172.21.15.32:0/493358082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a941583b0 0x7f5a94158830 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c000f80 tx=0x7f5a8c00eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:43.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.769+0000 7f5a90ff9640 1 -- 172.21.15.32:0/493358082 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c006b50 con 0x7f5a941583b0 2024-04-24T23:02:43.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.769+0000 7f5a90ff9640 1 -- 172.21.15.32:0/493358082 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a8c006d10 con 0x7f5a941583b0 2024-04-24T23:02:43.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.769+0000 7f5a9a45d640 1 -- 172.21.15.32:0/493358082 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a941f45b0 con 0x7f5a941583b0 2024-04-24T23:02:43.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.769+0000 7f5a90ff9640 1 -- 172.21.15.32:0/493358082 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c003970 con 0x7f5a941583b0 2024-04-24T23:02:43.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.769+0000 7f5a9a45d640 1 -- 172.21.15.32:0/493358082 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a941f4b70 con 0x7f5a941583b0 2024-04-24T23:02:43.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.770+0000 7f5a9a45d640 1 -- 172.21.15.32:0/493358082 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a94068430 con 0x7f5a941583b0 2024-04-24T23:02:43.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.770+0000 7f5a90ff9640 1 -- 172.21.15.32:0/493358082 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5a8c01a020 con 0x7f5a941583b0 2024-04-24T23:02:43.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.771+0000 7f5a90ff9640 1 --2- 172.21.15.32:0/493358082 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5a8006be60 0x7f5a8006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:43.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.771+0000 7f5a90ff9640 1 -- 172.21.15.32:0/493358082 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5767+0+0 (secure 0 0 0) 0x7f5a8c08a3f0 con 0x7f5a941583b0 2024-04-24T23:02:43.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.771+0000 7f5a93fff640 1 --2- 172.21.15.32:0/493358082 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5a8006be60 0x7f5a8006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:43.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.771+0000 7f5a90ff9640 1 -- 172.21.15.32:0/493358082 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=75}) -- 0x7f5a80073a50 con 0x7f5a941583b0 2024-04-24T23:02:43.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.771+0000 7f5a93fff640 1 --2- 172.21.15.32:0/493358082 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5a8006be60 0x7f5a8006e320 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5a84000c00 tx=0x7f5a840078f0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:43.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.775+0000 7f5a90ff9640 1 -- 172.21.15.32:0/493358082 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5a8c058710 con 0x7f5a941583b0 2024-04-24T23:02:43.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.966+0000 7f5a90ff9640 1 -- 172.21.15.32:0/493358082 <== mon.0 v2:172.21.15.32:3300/0 7 ==== osd_map(75..75 src has 1..75) ==== 1476+0+0 (secure 0 0 0) 0x7f5a8c0be2d0 con 0x7f5a941583b0 2024-04-24T23:02:43.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:43.966+0000 7f5a90ff9640 1 -- 172.21.15.32:0/493358082 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=76}) -- 0x7f5a80074bd0 con 0x7f5a941583b0 2024-04-24T23:02:44.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.002+0000 7f5a9a45d640 1 -- 172.21.15.32:0/493358082 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f5a941560f0 con 0x7f5a941583b0 2024-04-24T23:02:44.004 INFO:tasks.ceph.mon.a.smithi032.stderr:2024-04-24T23:02:44.003+0000 7f3b809c6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T23:02:44.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.004+0000 7f5a90ff9640 1 -- 172.21.15.32:0/493358082 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v28) ==== 161+0+0 (secure 0 0 0) 0x7f5a8c05ca10 con 0x7f5a941583b0 2024-04-24T23:02:44.006 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-24T23:02:44.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.005+0000 7f5a90ff9640 1 -- 172.21.15.32:0/493358082 <== mon.0 v2:172.21.15.32:3300/0 9 ==== osd_map(76..76 src has 1..76) ==== 374+0+0 (secure 0 0 0) 0x7f5a8c0889f0 con 0x7f5a941583b0 2024-04-24T23:02:44.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.007+0000 7f5a9a45d640 1 -- 172.21.15.32:0/493358082 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5a8006be60 msgr2=0x7f5a8006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:44.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.007+0000 7f5a9a45d640 1 --2- 172.21.15.32:0/493358082 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5a8006be60 0x7f5a8006e320 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5a84000c00 tx=0x7f5a840078f0 comp rx=0 tx=0).stop 2024-04-24T23:02:44.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.007+0000 7f5a9a45d640 1 -- 172.21.15.32:0/493358082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a941583b0 msgr2=0x7f5a94158830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:44.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.007+0000 7f5a9a45d640 1 --2- 172.21.15.32:0/493358082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a941583b0 0x7f5a94158830 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c000f80 tx=0x7f5a8c00eb90 comp rx=0 tx=0).stop 2024-04-24T23:02:44.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.007+0000 7f5a9a45d640 1 -- 172.21.15.32:0/493358082 shutdown_connections 2024-04-24T23:02:44.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.007+0000 7f5a9a45d640 1 --2- 172.21.15.32:0/493358082 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5a8006be60 0x7f5a8006e320 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:44.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.007+0000 7f5a9a45d640 1 --2- 172.21.15.32:0/493358082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a941583b0 0x7f5a94158830 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:44.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.007+0000 7f5a9a45d640 1 --2- 172.21.15.32:0/493358082 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5a940cdd50 0x7f5a940ce1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:44.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.007+0000 7f5a9a45d640 1 --2- 172.21.15.32:0/493358082 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a7c004800 0x7f5a941f1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:44.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.007+0000 7f5a9a45d640 1 -- 172.21.15.32:0/493358082 >> 172.21.15.32:0/493358082 conn(0x7f5a940d11d0 msgr2=0x7f5a940d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:44.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.008+0000 7f5a9a45d640 1 -- 172.21.15.32:0/493358082 shutdown_connections 2024-04-24T23:02:44.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.008+0000 7f5a9a45d640 1 -- 172.21.15.32:0/493358082 wait complete. 2024-04-24T23:02:44.021 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-24T23:02:44.047 INFO:tasks.ceph.mds.d.smithi121.stdout:starting mds.d at 2024-04-24T23:02:44.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.122+0000 7f1b87a7d640 1 Processor -- start 2024-04-24T23:02:44.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.123+0000 7f1b87a7d640 1 -- start start 2024-04-24T23:02:44.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.123+0000 7f1b87a7d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1b8006ba30 0x7f1b8006d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:44.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.123+0000 7f1b87a7d640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1b8006da90 0x7f1b800b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:44.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.123+0000 7f1b87a7d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b8015ef10 con 0x7f1b80158dd0 2024-04-24T23:02:44.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.123+0000 7f1b87a7d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f1b8015f1f0 con 0x7f1b8006ba30 2024-04-24T23:02:44.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.123+0000 7f1b87a7d640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f1b8015f4d0 con 0x7f1b8006da90 2024-04-24T23:02:44.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.124+0000 7f1b85018640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1b8006ba30 0x7f1b8006d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:44.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.124+0000 7f1b85819640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1b80158dd0 0x7f1b8006b320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46208/0 (socket says 172.21.15.32:46208) 2024-04-24T23:02:44.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.124+0000 7f1b85018640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1b8006ba30 0x7f1b8006d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42048/0 (socket says 172.21.15.32:42048) 2024-04-24T23:02:44.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.124+0000 7f1b85819640 1 -- 172.21.15.32:0/24154687 learned_addr learned my addr 172.21.15.32:0/24154687 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:02:44.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.124+0000 7f1b8601a640 1 --2- 172.21.15.32:0/24154687 >> v2:172.21.15.121:3300/0 conn(0x7f1b8006da90 0x7f1b800b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:44.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.124+0000 7f1b85018640 1 -- 172.21.15.32:0/24154687 >> v2:172.21.15.121:3300/0 conn(0x7f1b8006da90 msgr2=0x7f1b800b5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:44.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.124+0000 7f1b85018640 1 --2- 172.21.15.32:0/24154687 >> v2:172.21.15.121:3300/0 conn(0x7f1b8006da90 0x7f1b800b5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:44.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.124+0000 7f1b85018640 1 -- 172.21.15.32:0/24154687 >> v1:172.21.15.32:6789/0 conn(0x7f1b80158dd0 legacy=0x7f1b8006b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:44.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.124+0000 7f1b85018640 1 -- 172.21.15.32:0/24154687 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b8015f7b0 con 0x7f1b8006ba30 2024-04-24T23:02:44.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.124+0000 7f1b8601a640 1 --2- 172.21.15.32:0/24154687 >> v2:172.21.15.121:3300/0 conn(0x7f1b8006da90 0x7f1b800b5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:02:44.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.125+0000 7f1b85018640 1 --2- 172.21.15.32:0/24154687 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b8006ba30 0x7f1b8006d550 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f1b800b2170 tx=0x7f1b7002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3b2c312cb818ff3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:44.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.125+0000 7f1b84817640 1 -- 172.21.15.32:0/24154687 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b70037070 con 0x7f1b8006ba30 2024-04-24T23:02:44.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.125+0000 7f1b84817640 1 -- 172.21.15.32:0/24154687 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b70032440 con 0x7f1b8006ba30 2024-04-24T23:02:44.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.125+0000 7f1b84817640 1 -- 172.21.15.32:0/24154687 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b70032810 con 0x7f1b8006ba30 2024-04-24T23:02:44.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.125+0000 7f1b87a7d640 1 -- 172.21.15.32:0/24154687 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b8006ba30 msgr2=0x7f1b8006d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:44.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.125+0000 7f1b87a7d640 1 --2- 172.21.15.32:0/24154687 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b8006ba30 0x7f1b8006d550 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f1b800b2170 tx=0x7f1b7002a1a0 comp rx=0 tx=0).stop 2024-04-24T23:02:44.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.125+0000 7f1b87a7d640 1 -- 172.21.15.32:0/24154687 shutdown_connections 2024-04-24T23:02:44.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.125+0000 7f1b87a7d640 1 --2- 172.21.15.32:0/24154687 >> v2:172.21.15.121:3300/0 conn(0x7f1b8006da90 0x7f1b800b5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:44.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.125+0000 7f1b87a7d640 1 --2- 172.21.15.32:0/24154687 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b8006ba30 0x7f1b8006d550 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:44.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.125+0000 7f1b87a7d640 1 -- 172.21.15.32:0/24154687 >> 172.21.15.32:0/24154687 conn(0x7f1b8006e800 msgr2=0x7f1b8006ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:44.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.125+0000 7f1b87a7d640 1 -- 172.21.15.32:0/24154687 shutdown_connections 2024-04-24T23:02:44.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.125+0000 7f1b87a7d640 1 -- 172.21.15.32:0/24154687 wait complete. 2024-04-24T23:02:44.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.127+0000 7f1b87a7d640 1 Processor -- start 2024-04-24T23:02:44.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.127+0000 7f1b87a7d640 1 -- start start 2024-04-24T23:02:44.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.127+0000 7f1b87a7d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b8006ba30 0x7f1b8014d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:44.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.127+0000 7f1b87a7d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b8006da90 0x7f1b801467b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:44.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.127+0000 7f1b87a7d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b80158dd0 0x7f1b80143510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:44.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.127+0000 7f1b87a7d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1b8006bfe0 con 0x7f1b80158dd0 2024-04-24T23:02:44.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.127+0000 7f1b87a7d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1b8006ded0 con 0x7f1b8006ba30 2024-04-24T23:02:44.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.127+0000 7f1b87a7d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f1b801591b0 con 0x7f1b8006da90 2024-04-24T23:02:44.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.127+0000 7f1b8601a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b80158dd0 0x7f1b80143510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:44.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.127+0000 7f1b8601a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b80158dd0 0x7f1b80143510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42050/0 (socket says 172.21.15.32:42050) 2024-04-24T23:02:44.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.127+0000 7f1b8601a640 1 -- 172.21.15.32:0/2738317754 learned_addr learned my addr 172.21.15.32:0/2738317754 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:44.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.128+0000 7f1b85819640 1 --2- 172.21.15.32:0/2738317754 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b8006ba30 0x7f1b8014d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:44.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.128+0000 7f1b85018640 1 --2- 172.21.15.32:0/2738317754 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b8006da90 0x7f1b801467b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:44.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.128+0000 7f1b8601a640 1 -- 172.21.15.32:0/2738317754 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b8006da90 msgr2=0x7f1b801467b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:44.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.128+0000 7f1b8601a640 1 --2- 172.21.15.32:0/2738317754 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b8006da90 0x7f1b801467b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:44.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.128+0000 7f1b8601a640 1 -- 172.21.15.32:0/2738317754 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b8006ba30 msgr2=0x7f1b8014d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:44.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.128+0000 7f1b8601a640 1 --2- 172.21.15.32:0/2738317754 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b8006ba30 0x7f1b8014d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:44.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.128+0000 7f1b8601a640 1 -- 172.21.15.32:0/2738317754 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b80146f20 con 0x7f1b80158dd0 2024-04-24T23:02:44.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.128+0000 7f1b8601a640 1 --2- 172.21.15.32:0/2738317754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b80158dd0 0x7f1b80143510 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f1b7800cef0 tx=0x7f1b7800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:44.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.128+0000 7f1b767fc640 1 -- 172.21.15.32:0/2738317754 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b7801a070 con 0x7f1b80158dd0 2024-04-24T23:02:44.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.128+0000 7f1b767fc640 1 -- 172.21.15.32:0/2738317754 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b7800d860 con 0x7f1b80158dd0 2024-04-24T23:02:44.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.128+0000 7f1b87a7d640 1 -- 172.21.15.32:0/2738317754 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b80147200 con 0x7f1b80158dd0 2024-04-24T23:02:44.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.128+0000 7f1b767fc640 1 -- 172.21.15.32:0/2738317754 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b7800dc50 con 0x7f1b80158dd0 2024-04-24T23:02:44.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.128+0000 7f1b87a7d640 1 -- 172.21.15.32:0/2738317754 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b8014fc40 con 0x7f1b80158dd0 2024-04-24T23:02:44.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.129+0000 7f1b87a7d640 1 -- 172.21.15.32:0/2738317754 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b80068430 con 0x7f1b80158dd0 2024-04-24T23:02:44.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.130+0000 7f1b767fc640 1 -- 172.21.15.32:0/2738317754 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1b7801d070 con 0x7f1b80158dd0 2024-04-24T23:02:44.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.130+0000 7f1b767fc640 1 --2- 172.21.15.32:0/2738317754 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1b6406be30 0x7f1b6406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:44.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.130+0000 7f1b767fc640 1 -- 172.21.15.32:0/2738317754 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5853+0+0 (secure 0 0 0) 0x7f1b7808b650 con 0x7f1b80158dd0 2024-04-24T23:02:44.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.130+0000 7f1b85819640 1 --2- 172.21.15.32:0/2738317754 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1b6406be30 0x7f1b6406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:44.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.131+0000 7f1b85819640 1 --2- 172.21.15.32:0/2738317754 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1b6406be30 0x7f1b6406e2f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c007fd0 tx=0x7f1b7c007e80 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:44.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.134+0000 7f1b767fc640 1 -- 172.21.15.32:0/2738317754 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1b78059a70 con 0x7f1b80158dd0 2024-04-24T23:02:44.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:44.357+0000 7f1b87a7d640 1 -- 172.21.15.32:0/2738317754 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f1b80143a50 con 0x7f1b80158dd0 2024-04-24T23:02:45.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.006+0000 7f1b767fc640 1 -- 172.21.15.32:0/2738317754 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v29) ==== 104+0+0 (secure 0 0 0) 0x7f1b7805dd70 con 0x7f1b80158dd0 2024-04-24T23:02:45.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.008+0000 7f1b87a7d640 1 -- 172.21.15.32:0/2738317754 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1b6406be30 msgr2=0x7f1b6406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:45.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.008+0000 7f1b87a7d640 1 --2- 172.21.15.32:0/2738317754 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1b6406be30 0x7f1b6406e2f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c007fd0 tx=0x7f1b7c007e80 comp rx=0 tx=0).stop 2024-04-24T23:02:45.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.008+0000 7f1b87a7d640 1 -- 172.21.15.32:0/2738317754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b80158dd0 msgr2=0x7f1b80143510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:45.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.008+0000 7f1b87a7d640 1 --2- 172.21.15.32:0/2738317754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b80158dd0 0x7f1b80143510 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f1b7800cef0 tx=0x7f1b7800a710 comp rx=0 tx=0).stop 2024-04-24T23:02:45.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.008+0000 7f1b87a7d640 1 -- 172.21.15.32:0/2738317754 shutdown_connections 2024-04-24T23:02:45.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.008+0000 7f1b87a7d640 1 --2- 172.21.15.32:0/2738317754 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1b6406be30 0x7f1b6406e2f0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:45.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.008+0000 7f1b87a7d640 1 --2- 172.21.15.32:0/2738317754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b80158dd0 0x7f1b80143510 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:45.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.008+0000 7f1b87a7d640 1 --2- 172.21.15.32:0/2738317754 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b8006da90 0x7f1b801467b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:45.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.008+0000 7f1b87a7d640 1 --2- 172.21.15.32:0/2738317754 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b8006ba30 0x7f1b8014d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:45.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.008+0000 7f1b87a7d640 1 -- 172.21.15.32:0/2738317754 >> 172.21.15.32:0/2738317754 conn(0x7f1b8006e800 msgr2=0x7f1b80160250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:45.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.008+0000 7f1b87a7d640 1 -- 172.21.15.32:0/2738317754 shutdown_connections 2024-04-24T23:02:45.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.008+0000 7f1b87a7d640 1 -- 172.21.15.32:0/2738317754 wait complete. 2024-04-24T23:02:45.022 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-24T23:02:45.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.129+0000 7fe72e8c3640 1 Processor -- start 2024-04-24T23:02:45.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.130+0000 7fe72e8c3640 1 -- start start 2024-04-24T23:02:45.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.130+0000 7fe72e8c3640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe7280b9ff0 0x7fe7280d40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:45.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.130+0000 7fe72e8c3640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe7280b6450 0x7fe7280b68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:45.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.130+0000 7fe72e8c3640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7280d3000 con 0x7fe72815e770 2024-04-24T23:02:45.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.130+0000 7fe72e8c3640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fe7280b6df0 con 0x7fe7280b6450 2024-04-24T23:02:45.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.130+0000 7fe72e8c3640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe7280d32e0 con 0x7fe7280b9ff0 2024-04-24T23:02:45.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.131+0000 7fe7277fe640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe7280b9ff0 0x7fe7280d40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:45.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.131+0000 7fe72ce60640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe7280b6450 0x7fe7280b68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:45.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.131+0000 7fe72ce60640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe7280b6450 0x7fe7280b68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.32:37020/0 (socket says 172.21.15.32:37020) 2024-04-24T23:02:45.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.131+0000 7fe7277fe640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe7280b9ff0 0x7fe7280d40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.32:60172/0 (socket says 172.21.15.32:60172) 2024-04-24T23:02:45.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.131+0000 7fe72ce60640 1 -- 172.21.15.32:0/2166342864 learned_addr learned my addr 172.21.15.32:0/2166342864 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:45.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.131+0000 7fe727fff640 1 --1- 172.21.15.32:0/2166342864 >> v1:172.21.15.121:6789/0 conn(0x7fe72815e770 0x7fe7280d2700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:33204/0 (socket says 172.21.15.32:33204) 2024-04-24T23:02:45.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.131+0000 7fe7277fe640 1 -- 172.21.15.32:0/2166342864 >> v2:172.21.15.121:3300/0 conn(0x7fe7280b6450 msgr2=0x7fe7280b68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:45.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.131+0000 7fe7277fe640 1 --2- 172.21.15.32:0/2166342864 >> v2:172.21.15.121:3300/0 conn(0x7fe7280b6450 0x7fe7280b68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:45.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.131+0000 7fe7277fe640 1 -- 172.21.15.32:0/2166342864 >> v1:172.21.15.121:6789/0 conn(0x7fe72815e770 legacy=0x7fe7280d2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:45.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.131+0000 7fe7277fe640 1 -- 172.21.15.32:0/2166342864 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7280d35c0 con 0x7fe7280b9ff0 2024-04-24T23:02:45.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.132+0000 7fe7277fe640 1 --2- 172.21.15.32:0/2166342864 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe7280b9ff0 0x7fe7280d40a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe71c0097a0 tx=0x7fe71c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15e8a4f061a71725 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:45.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.132+0000 7fe726ffd640 1 -- 172.21.15.32:0/2166342864 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71c037070 con 0x7fe7280b9ff0 2024-04-24T23:02:45.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.132+0000 7fe726ffd640 1 -- 172.21.15.32:0/2166342864 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe71c032440 con 0x7fe7280b9ff0 2024-04-24T23:02:45.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.132+0000 7fe726ffd640 1 -- 172.21.15.32:0/2166342864 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71c032810 con 0x7fe7280b9ff0 2024-04-24T23:02:45.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.133+0000 7fe72e8c3640 1 -- 172.21.15.32:0/2166342864 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe7280b9ff0 msgr2=0x7fe7280d40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:45.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.133+0000 7fe72e8c3640 1 --2- 172.21.15.32:0/2166342864 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe7280b9ff0 0x7fe7280d40a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe71c0097a0 tx=0x7fe71c02a180 comp rx=0 tx=0).stop 2024-04-24T23:02:45.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.133+0000 7fe72e8c3640 1 -- 172.21.15.32:0/2166342864 shutdown_connections 2024-04-24T23:02:45.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.133+0000 7fe72e8c3640 1 --2- 172.21.15.32:0/2166342864 >> v2:172.21.15.121:3300/0 conn(0x7fe7280b6450 0x7fe7280b68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:45.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.133+0000 7fe72e8c3640 1 --2- 172.21.15.32:0/2166342864 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe7280b9ff0 0x7fe7280d40a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:45.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.133+0000 7fe72e8c3640 1 -- 172.21.15.32:0/2166342864 >> 172.21.15.32:0/2166342864 conn(0x7fe728071910 msgr2=0x7fe728071d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:45.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.133+0000 7fe72e8c3640 1 -- 172.21.15.32:0/2166342864 shutdown_connections 2024-04-24T23:02:45.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.133+0000 7fe72e8c3640 1 -- 172.21.15.32:0/2166342864 wait complete. 2024-04-24T23:02:45.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.134+0000 7fe72e8c3640 1 Processor -- start 2024-04-24T23:02:45.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.134+0000 7fe72e8c3640 1 -- start start 2024-04-24T23:02:45.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.135+0000 7fe72e8c3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7280b6450 0x7fe7280ab620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:45.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.135+0000 7fe72e8c3640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe7280b9ff0 0x7fe7280c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:45.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.135+0000 7fe72e8c3640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe72815e770 0x7fe7280cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:45.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.135+0000 7fe72e8c3640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe72815eb50 con 0x7fe7280b6450 2024-04-24T23:02:45.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.135+0000 7fe72e8c3640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe7280d3b80 con 0x7fe7280b9ff0 2024-04-24T23:02:45.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.135+0000 7fe72e8c3640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe72815ecd0 con 0x7fe72815e770 2024-04-24T23:02:45.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.135+0000 7fe727fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7280b6450 0x7fe7280ab620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:45.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.135+0000 7fe727fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7280b6450 0x7fe7280ab620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42062/0 (socket says 172.21.15.32:42062) 2024-04-24T23:02:45.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.135+0000 7fe727fff640 1 -- 172.21.15.32:0/914768988 learned_addr learned my addr 172.21.15.32:0/914768988 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:45.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.135+0000 7fe7277fe640 1 --2- 172.21.15.32:0/914768988 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe7280b9ff0 0x7fe7280c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:45.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.135+0000 7fe72ce60640 1 --2- 172.21.15.32:0/914768988 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe72815e770 0x7fe7280cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:45.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.135+0000 7fe727fff640 1 -- 172.21.15.32:0/914768988 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe72815e770 msgr2=0x7fe7280cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:45.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.135+0000 7fe727fff640 1 --2- 172.21.15.32:0/914768988 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe72815e770 0x7fe7280cccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:45.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.135+0000 7fe727fff640 1 -- 172.21.15.32:0/914768988 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe7280b9ff0 msgr2=0x7fe7280c7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:45.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.135+0000 7fe727fff640 1 --2- 172.21.15.32:0/914768988 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe7280b9ff0 0x7fe7280c7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:45.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.135+0000 7fe727fff640 1 -- 172.21.15.32:0/914768988 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7280abb60 con 0x7fe7280b6450 2024-04-24T23:02:45.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.136+0000 7fe727fff640 1 --2- 172.21.15.32:0/914768988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7280b6450 0x7fe7280ab620 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fe71800c3c0 tx=0x7fe71800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:45.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.136+0000 7fe724ff9640 1 -- 172.21.15.32:0/914768988 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7180074e0 con 0x7fe7280b6450 2024-04-24T23:02:45.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.136+0000 7fe72e8c3640 1 -- 172.21.15.32:0/914768988 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7280c89a0 con 0x7fe7280b6450 2024-04-24T23:02:45.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.136+0000 7fe724ff9640 1 -- 172.21.15.32:0/914768988 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe718007680 con 0x7fe7280b6450 2024-04-24T23:02:45.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.136+0000 7fe724ff9640 1 -- 172.21.15.32:0/914768988 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe718004a00 con 0x7fe7280b6450 2024-04-24T23:02:45.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.136+0000 7fe72e8c3640 1 -- 172.21.15.32:0/914768988 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7280c8f60 con 0x7fe7280b6450 2024-04-24T23:02:45.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.137+0000 7fe72e8c3640 1 -- 172.21.15.32:0/914768988 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe728068430 con 0x7fe7280b6450 2024-04-24T23:02:45.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.137+0000 7fe724ff9640 1 -- 172.21.15.32:0/914768988 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe71801a020 con 0x7fe7280b6450 2024-04-24T23:02:45.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.138+0000 7fe724ff9640 1 --2- 172.21.15.32:0/914768988 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe71406be30 0x7fe71406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:45.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.138+0000 7fe724ff9640 1 -- 172.21.15.32:0/914768988 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5853+0+0 (secure 0 0 0) 0x7fe71808a580 con 0x7fe7280b6450 2024-04-24T23:02:45.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.138+0000 7fe7277fe640 1 --2- 172.21.15.32:0/914768988 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe71406be30 0x7fe71406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:45.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.138+0000 7fe7277fe640 1 --2- 172.21.15.32:0/914768988 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe71406be30 0x7fe71406e2f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe71c034000 tx=0x7fe71c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:45.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.142+0000 7fe724ff9640 1 -- 172.21.15.32:0/914768988 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe7180587d0 con 0x7fe7280b6450 2024-04-24T23:02:45.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:45.325+0000 7fe72e8c3640 1 -- 172.21.15.32:0/914768988 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe7280cedc0 con 0x7fe7280b6450 2024-04-24T23:02:46.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.010+0000 7fe724ff9640 1 -- 172.21.15.32:0/914768988 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v77) ==== 181+0+0 (secure 0 0 0) 0x7fe71805cad0 con 0x7fe7280b6450 2024-04-24T23:02:46.012 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2024-04-24T23:02:46.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.014+0000 7fe72e8c3640 1 -- 172.21.15.32:0/914768988 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe71406be30 msgr2=0x7fe71406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:46.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.014+0000 7fe72e8c3640 1 --2- 172.21.15.32:0/914768988 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe71406be30 0x7fe71406e2f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe71c034000 tx=0x7fe71c02e000 comp rx=0 tx=0).stop 2024-04-24T23:02:46.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.014+0000 7fe72e8c3640 1 -- 172.21.15.32:0/914768988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7280b6450 msgr2=0x7fe7280ab620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:46.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.014+0000 7fe72e8c3640 1 --2- 172.21.15.32:0/914768988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7280b6450 0x7fe7280ab620 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fe71800c3c0 tx=0x7fe71800c880 comp rx=0 tx=0).stop 2024-04-24T23:02:46.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.014+0000 7fe72e8c3640 1 -- 172.21.15.32:0/914768988 shutdown_connections 2024-04-24T23:02:46.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.014+0000 7fe72e8c3640 1 --2- 172.21.15.32:0/914768988 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe71406be30 0x7fe71406e2f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:46.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.014+0000 7fe72e8c3640 1 --2- 172.21.15.32:0/914768988 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe72815e770 0x7fe7280cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:46.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.014+0000 7fe72e8c3640 1 --2- 172.21.15.32:0/914768988 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe7280b9ff0 0x7fe7280c7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:46.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.014+0000 7fe72e8c3640 1 --2- 172.21.15.32:0/914768988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7280b6450 0x7fe7280ab620 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:46.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.014+0000 7fe72e8c3640 1 -- 172.21.15.32:0/914768988 >> 172.21.15.32:0/914768988 conn(0x7fe728071910 msgr2=0x7fe7280b4e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:46.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.014+0000 7fe72e8c3640 1 -- 172.21.15.32:0/914768988 shutdown_connections 2024-04-24T23:02:46.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.014+0000 7fe72e8c3640 1 -- 172.21.15.32:0/914768988 wait complete. 2024-04-24T23:02:46.028 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-24T23:02:46.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.134+0000 7fba8245d640 1 Processor -- start 2024-04-24T23:02:46.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.134+0000 7fba8245d640 1 -- start start 2024-04-24T23:02:46.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.135+0000 7fba8245d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fba7c15e770 0x7fba7c0d40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:46.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.135+0000 7fba8245d640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fba7c0b9ff0 0x7fba7c0d2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:46.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.135+0000 7fba8245d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba7c0d3000 con 0x7fba7c0b6450 2024-04-24T23:02:46.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.135+0000 7fba8245d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fba7c0d32e0 con 0x7fba7c15e770 2024-04-24T23:02:46.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.135+0000 7fba8245d640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fba7c0d35c0 con 0x7fba7c0b9ff0 2024-04-24T23:02:46.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.135+0000 7fba7bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fba7c15e770 0x7fba7c0d40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:46.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.135+0000 7fba809fa640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fba7c0b6450 0x7fba7c0d0ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46224/0 (socket says 172.21.15.32:46224) 2024-04-24T23:02:46.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.135+0000 7fba809fa640 1 -- 172.21.15.32:0/3286082250 learned_addr learned my addr 172.21.15.32:0/3286082250 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:02:46.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.135+0000 7fba7bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fba7c15e770 0x7fba7c0d40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42076/0 (socket says 172.21.15.32:42076) 2024-04-24T23:02:46.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.136+0000 7fba7bfff640 1 -- 172.21.15.32:0/3286082250 >> v2:172.21.15.121:3300/0 conn(0x7fba7c0b9ff0 msgr2=0x7fba7c0d2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:46.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.136+0000 7fba7b7fe640 1 --2- 172.21.15.32:0/3286082250 >> v2:172.21.15.121:3300/0 conn(0x7fba7c0b9ff0 0x7fba7c0d2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:46.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.136+0000 7fba7affd640 1 -- 172.21.15.32:0/3286082250 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2316542898 0 0) 0x7fba7c0d3000 con 0x7fba7c0b6450 2024-04-24T23:02:46.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.136+0000 7fba7bfff640 1 --2- 172.21.15.32:0/3286082250 >> v2:172.21.15.121:3300/0 conn(0x7fba7c0b9ff0 0x7fba7c0d2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:46.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.136+0000 7fba7bfff640 1 -- 172.21.15.32:0/3286082250 >> v1:172.21.15.32:6789/0 conn(0x7fba7c0b6450 legacy=0x7fba7c0d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:46.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.136+0000 7fba7bfff640 1 -- 172.21.15.32:0/3286082250 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba7c0d38a0 con 0x7fba7c15e770 2024-04-24T23:02:46.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.136+0000 7fba7b7fe640 1 --2- 172.21.15.32:0/3286082250 >> v2:172.21.15.121:3300/0 conn(0x7fba7c0b9ff0 0x7fba7c0d2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:02:46.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.136+0000 7fba7bfff640 1 --2- 172.21.15.32:0/3286082250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba7c15e770 0x7fba7c0d40a0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fba7c0d5670 tx=0x7fba6c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c740ce6bc2600a72 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:46.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.137+0000 7fba7affd640 1 -- 172.21.15.32:0/3286082250 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c037070 con 0x7fba7c15e770 2024-04-24T23:02:46.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.137+0000 7fba7affd640 1 -- 172.21.15.32:0/3286082250 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba6c032440 con 0x7fba7c15e770 2024-04-24T23:02:46.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.137+0000 7fba7affd640 1 -- 172.21.15.32:0/3286082250 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c032810 con 0x7fba7c15e770 2024-04-24T23:02:46.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.137+0000 7fba8245d640 1 -- 172.21.15.32:0/3286082250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba7c15e770 msgr2=0x7fba7c0d40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:46.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.137+0000 7fba8245d640 1 --2- 172.21.15.32:0/3286082250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba7c15e770 0x7fba7c0d40a0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fba7c0d5670 tx=0x7fba6c02a1a0 comp rx=0 tx=0).stop 2024-04-24T23:02:46.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.137+0000 7fba8245d640 1 -- 172.21.15.32:0/3286082250 shutdown_connections 2024-04-24T23:02:46.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.137+0000 7fba8245d640 1 --2- 172.21.15.32:0/3286082250 >> v2:172.21.15.121:3300/0 conn(0x7fba7c0b9ff0 0x7fba7c0d2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:46.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.137+0000 7fba8245d640 1 --2- 172.21.15.32:0/3286082250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba7c15e770 0x7fba7c0d40a0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:46.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.137+0000 7fba8245d640 1 -- 172.21.15.32:0/3286082250 >> 172.21.15.32:0/3286082250 conn(0x7fba7c071910 msgr2=0x7fba7c071d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:46.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.137+0000 7fba8245d640 1 -- 172.21.15.32:0/3286082250 shutdown_connections 2024-04-24T23:02:46.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.137+0000 7fba8245d640 1 -- 172.21.15.32:0/3286082250 wait complete. 2024-04-24T23:02:46.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.138+0000 7fba8245d640 1 Processor -- start 2024-04-24T23:02:46.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.139+0000 7fba8245d640 1 -- start start 2024-04-24T23:02:46.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.139+0000 7fba8245d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba7c0b6450 0x7fba7c0ce210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:46.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.139+0000 7fba8245d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fba7c0b9ff0 0x7fba7c0b3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:46.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.139+0000 7fba8245d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fba7c15e770 0x7fba7c14a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:46.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.139+0000 7fba8245d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c0d2dc0 con 0x7fba7c0b6450 2024-04-24T23:02:46.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.139+0000 7fba8245d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c0d2c40 con 0x7fba7c15e770 2024-04-24T23:02:46.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.139+0000 7fba8245d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fba7c15eb50 con 0x7fba7c0b9ff0 2024-04-24T23:02:46.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.139+0000 7fba7bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba7c0b6450 0x7fba7c0ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:46.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.139+0000 7fba7bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba7c0b6450 0x7fba7c0ce210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42082/0 (socket says 172.21.15.32:42082) 2024-04-24T23:02:46.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.140+0000 7fba7bfff640 1 -- 172.21.15.32:0/484874222 learned_addr learned my addr 172.21.15.32:0/484874222 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:46.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.140+0000 7fba809fa640 1 --2- 172.21.15.32:0/484874222 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fba7c15e770 0x7fba7c14a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:46.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.140+0000 7fba7bfff640 1 -- 172.21.15.32:0/484874222 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fba7c0b9ff0 msgr2=0x7fba7c0b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:46.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.140+0000 7fba7b7fe640 1 --2- 172.21.15.32:0/484874222 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fba7c0b9ff0 0x7fba7c0b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:46.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.140+0000 7fba7bfff640 1 --2- 172.21.15.32:0/484874222 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fba7c0b9ff0 0x7fba7c0b3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:46.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.140+0000 7fba7bfff640 1 -- 172.21.15.32:0/484874222 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fba7c15e770 msgr2=0x7fba7c14a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:46.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.140+0000 7fba7bfff640 1 --2- 172.21.15.32:0/484874222 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fba7c15e770 0x7fba7c14a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:46.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.140+0000 7fba7bfff640 1 -- 172.21.15.32:0/484874222 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba7c0ca670 con 0x7fba7c0b6450 2024-04-24T23:02:46.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.140+0000 7fba7bfff640 1 --2- 172.21.15.32:0/484874222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba7c0b6450 0x7fba7c0ce210 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fba6c000c00 tx=0x7fba6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:46.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.140+0000 7fba78ff9640 1 -- 172.21.15.32:0/484874222 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c037050 con 0x7fba7c0b6450 2024-04-24T23:02:46.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.140+0000 7fba78ff9640 1 -- 172.21.15.32:0/484874222 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba6c038040 con 0x7fba7c0b6450 2024-04-24T23:02:46.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.140+0000 7fba8245d640 1 -- 172.21.15.32:0/484874222 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba7c0ca950 con 0x7fba7c0b6450 2024-04-24T23:02:46.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.140+0000 7fba78ff9640 1 -- 172.21.15.32:0/484874222 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c004450 con 0x7fba7c0b6450 2024-04-24T23:02:46.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.140+0000 7fba8245d640 1 -- 172.21.15.32:0/484874222 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba7c0caf10 con 0x7fba7c0b6450 2024-04-24T23:02:46.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.141+0000 7fba8245d640 1 -- 172.21.15.32:0/484874222 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba7c155640 con 0x7fba7c0b6450 2024-04-24T23:02:46.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.142+0000 7fba78ff9640 1 -- 172.21.15.32:0/484874222 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fba6c030980 con 0x7fba7c0b6450 2024-04-24T23:02:46.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.142+0000 7fba78ff9640 1 --2- 172.21.15.32:0/484874222 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fba6806be30 0x7fba6806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:46.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.142+0000 7fba78ff9640 1 -- 172.21.15.32:0/484874222 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5415+0+0 (secure 0 0 0) 0x7fba6c0aba10 con 0x7fba7c0b6450 2024-04-24T23:02:46.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.142+0000 7fba7b7fe640 1 --2- 172.21.15.32:0/484874222 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fba6806be30 0x7fba6806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:46.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.143+0000 7fba7b7fe640 1 --2- 172.21.15.32:0/484874222 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fba6806be30 0x7fba6806e2f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fba7c1f7ed0 tx=0x7fba70008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:46.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.146+0000 7fba78ff9640 1 -- 172.21.15.32:0/484874222 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fba6c07a600 con 0x7fba7c0b6450 2024-04-24T23:02:46.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:46.330+0000 7fba8245d640 1 -- 172.21.15.32:0/484874222 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fba7c068430 con 0x7fba7c0b6450 2024-04-24T23:02:47.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.015+0000 7fba78ff9640 1 -- 172.21.15.32:0/484874222 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v78) ==== 169+0+0 (secure 0 0 0) 0x7fba6c079f00 con 0x7fba7c0b6450 2024-04-24T23:02:47.016 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2024-04-24T23:02:47.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.018+0000 7fba8245d640 1 -- 172.21.15.32:0/484874222 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fba6806be30 msgr2=0x7fba6806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:47.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.018+0000 7fba8245d640 1 --2- 172.21.15.32:0/484874222 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fba6806be30 0x7fba6806e2f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fba7c1f7ed0 tx=0x7fba70008040 comp rx=0 tx=0).stop 2024-04-24T23:02:47.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.018+0000 7fba8245d640 1 -- 172.21.15.32:0/484874222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba7c0b6450 msgr2=0x7fba7c0ce210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:47.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.018+0000 7fba8245d640 1 --2- 172.21.15.32:0/484874222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba7c0b6450 0x7fba7c0ce210 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fba6c000c00 tx=0x7fba6c0046c0 comp rx=0 tx=0).stop 2024-04-24T23:02:47.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.018+0000 7fba8245d640 1 -- 172.21.15.32:0/484874222 shutdown_connections 2024-04-24T23:02:47.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.018+0000 7fba8245d640 1 --2- 172.21.15.32:0/484874222 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fba6806be30 0x7fba6806e2f0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:47.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.018+0000 7fba8245d640 1 --2- 172.21.15.32:0/484874222 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fba7c15e770 0x7fba7c14a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:47.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.018+0000 7fba8245d640 1 --2- 172.21.15.32:0/484874222 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fba7c0b9ff0 0x7fba7c0b3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:47.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.018+0000 7fba8245d640 1 --2- 172.21.15.32:0/484874222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba7c0b6450 0x7fba7c0ce210 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:47.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.018+0000 7fba8245d640 1 -- 172.21.15.32:0/484874222 >> 172.21.15.32:0/484874222 conn(0x7fba7c071910 msgr2=0x7fba7c071cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:47.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.018+0000 7fba8245d640 1 -- 172.21.15.32:0/484874222 shutdown_connections 2024-04-24T23:02:47.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.018+0000 7fba8245d640 1 -- 172.21.15.32:0/484874222 wait complete. 2024-04-24T23:02:47.033 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-24T23:02:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.138+0000 7fc507329640 1 Processor -- start 2024-04-24T23:02:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.138+0000 7fc507329640 1 -- start start 2024-04-24T23:02:47.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.139+0000 7fc507329640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc500074210 0x7fc50006c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:47.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.139+0000 7fc507329640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc5000b0b80 0x7fc5000745f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.139+0000 7fc507329640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc5000ae140 0x7fc5000b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.139+0000 7fc507329640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fc5000b1c20 con 0x7fc5000ae140 2024-04-24T23:02:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.139+0000 7fc507329640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fc5000b0770 con 0x7fc5000b0b80 2024-04-24T23:02:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.139+0000 7fc507329640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc500074b30 con 0x7fc500074210 2024-04-24T23:02:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.139+0000 7fc5058c6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc5000ae140 0x7fc5000b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.139+0000 7fc5058c6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc5000ae140 0x7fc5000b20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42088/0 (socket says 172.21.15.32:42088) 2024-04-24T23:02:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.139+0000 7fc5058c6640 1 -- 172.21.15.32:0/1721988127 learned_addr learned my addr 172.21.15.32:0/1721988127 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.139+0000 7fc5050c5640 1 --2- 172.21.15.32:0/1721988127 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc500074210 0x7fc50006c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.139+0000 7fc5048c4640 1 --2- 172.21.15.32:0/1721988127 >> v2:172.21.15.121:3300/0 conn(0x7fc5000b0b80 0x7fc5000745f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.139+0000 7fc5058c6640 1 -- 172.21.15.32:0/1721988127 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc500074210 msgr2=0x7fc50006c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.139+0000 7fc5058c6640 1 --2- 172.21.15.32:0/1721988127 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc500074210 0x7fc50006c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.139+0000 7fc5058c6640 1 -- 172.21.15.32:0/1721988127 >> v2:172.21.15.121:3300/0 conn(0x7fc5000b0b80 msgr2=0x7fc5000745f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.139+0000 7fc5058c6640 1 --2- 172.21.15.32:0/1721988127 >> v2:172.21.15.121:3300/0 conn(0x7fc5000b0b80 0x7fc5000745f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:47.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.139+0000 7fc5058c6640 1 -- 172.21.15.32:0/1721988127 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5000b1010 con 0x7fc5000ae140 2024-04-24T23:02:47.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.140+0000 7fc5058c6640 1 --2- 172.21.15.32:0/1721988127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5000ae140 0x7fc5000b20b0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fc5000719a0 tx=0x7fc4f802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a2734b01f4df630 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:47.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.140+0000 7fc4f7fff640 1 -- 172.21.15.32:0/1721988127 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f8036070 con 0x7fc5000ae140 2024-04-24T23:02:47.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.140+0000 7fc4f7fff640 1 -- 172.21.15.32:0/1721988127 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4f8031600 con 0x7fc5000ae140 2024-04-24T23:02:47.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.140+0000 7fc4f7fff640 1 -- 172.21.15.32:0/1721988127 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f80319d0 con 0x7fc5000ae140 2024-04-24T23:02:47.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.141+0000 7fc507329640 1 -- 172.21.15.32:0/1721988127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5000ae140 msgr2=0x7fc5000b20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:47.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.141+0000 7fc507329640 1 --2- 172.21.15.32:0/1721988127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5000ae140 0x7fc5000b20b0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fc5000719a0 tx=0x7fc4f802a6a0 comp rx=0 tx=0).stop 2024-04-24T23:02:47.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.141+0000 7fc507329640 1 -- 172.21.15.32:0/1721988127 shutdown_connections 2024-04-24T23:02:47.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.141+0000 7fc507329640 1 --2- 172.21.15.32:0/1721988127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5000ae140 0x7fc5000b20b0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:47.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.141+0000 7fc507329640 1 --2- 172.21.15.32:0/1721988127 >> v2:172.21.15.121:3300/0 conn(0x7fc5000b0b80 0x7fc5000745f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:47.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.141+0000 7fc507329640 1 --2- 172.21.15.32:0/1721988127 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc500074210 0x7fc50006c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:47.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.141+0000 7fc507329640 1 -- 172.21.15.32:0/1721988127 >> 172.21.15.32:0/1721988127 conn(0x7fc5000b2e60 msgr2=0x7fc5000b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:47.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.141+0000 7fc507329640 1 -- 172.21.15.32:0/1721988127 shutdown_connections 2024-04-24T23:02:47.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.141+0000 7fc507329640 1 -- 172.21.15.32:0/1721988127 wait complete. 2024-04-24T23:02:47.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.142+0000 7fc507329640 1 Processor -- start 2024-04-24T23:02:47.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.142+0000 7fc507329640 1 -- start start 2024-04-24T23:02:47.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.143+0000 7fc507329640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc500074210 0x7fc50015db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:47.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.143+0000 7fc507329640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc5000ae140 0x7fc50014e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:47.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.143+0000 7fc507329640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc50014bb00 0x7fc5000c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:47.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.143+0000 7fc507329640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc5000b18b0 con 0x7fc500074210 2024-04-24T23:02:47.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.143+0000 7fc507329640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc5000b15d0 con 0x7fc50014bb00 2024-04-24T23:02:47.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.143+0000 7fc507329640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc5000b3ba0 con 0x7fc5000ae140 2024-04-24T23:02:47.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.143+0000 7fc5050c5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc500074210 0x7fc50015db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:47.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.143+0000 7fc5050c5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc500074210 0x7fc50015db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42100/0 (socket says 172.21.15.32:42100) 2024-04-24T23:02:47.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.143+0000 7fc5050c5640 1 -- 172.21.15.32:0/3784406237 learned_addr learned my addr 172.21.15.32:0/3784406237 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:47.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.143+0000 7fc5048c4640 1 --2- 172.21.15.32:0/3784406237 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc5000ae140 0x7fc50014e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:47.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.143+0000 7fc5058c6640 1 --2- 172.21.15.32:0/3784406237 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc50014bb00 0x7fc5000c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:47.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.143+0000 7fc5050c5640 1 -- 172.21.15.32:0/3784406237 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc5000ae140 msgr2=0x7fc50014e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:47.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.143+0000 7fc5050c5640 1 --2- 172.21.15.32:0/3784406237 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc5000ae140 0x7fc50014e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:47.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.143+0000 7fc5050c5640 1 -- 172.21.15.32:0/3784406237 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc50014bb00 msgr2=0x7fc5000c83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:47.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.143+0000 7fc5050c5640 1 --2- 172.21.15.32:0/3784406237 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc50014bb00 0x7fc5000c83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:47.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.143+0000 7fc5050c5640 1 -- 172.21.15.32:0/3784406237 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc50015e490 con 0x7fc500074210 2024-04-24T23:02:47.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.143+0000 7fc5050c5640 1 --2- 172.21.15.32:0/3784406237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc500074210 0x7fc50015db50 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc00e8a0 tx=0x7fc4fc00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:47.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.143+0000 7fc4f5ffb640 1 -- 172.21.15.32:0/3784406237 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc017070 con 0x7fc500074210 2024-04-24T23:02:47.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.143+0000 7fc4f5ffb640 1 -- 172.21.15.32:0/3784406237 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4fc00ee90 con 0x7fc500074210 2024-04-24T23:02:47.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.143+0000 7fc507329640 1 -- 172.21.15.32:0/3784406237 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc50014c000 con 0x7fc500074210 2024-04-24T23:02:47.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.143+0000 7fc4f5ffb640 1 -- 172.21.15.32:0/3784406237 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc00cb80 con 0x7fc500074210 2024-04-24T23:02:47.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.144+0000 7fc507329640 1 -- 172.21.15.32:0/3784406237 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc50014c5c0 con 0x7fc500074210 2024-04-24T23:02:47.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.145+0000 7fc507329640 1 -- 172.21.15.32:0/3784406237 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc500068430 con 0x7fc500074210 2024-04-24T23:02:47.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.145+0000 7fc4f5ffb640 1 -- 172.21.15.32:0/3784406237 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc4fc01a070 con 0x7fc500074210 2024-04-24T23:02:47.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.145+0000 7fc4f5ffb640 1 --2- 172.21.15.32:0/3784406237 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc4e406be30 0x7fc4e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:47.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.146+0000 7fc4f5ffb640 1 -- 172.21.15.32:0/3784406237 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 4943+0+0 (secure 0 0 0) 0x7fc4fc08a9e0 con 0x7fc500074210 2024-04-24T23:02:47.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.146+0000 7fc5048c4640 1 --2- 172.21.15.32:0/3784406237 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc4e406be30 0x7fc4e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:47.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.146+0000 7fc5048c4640 1 --2- 172.21.15.32:0/3784406237 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc4e406be30 0x7fc4e406e2f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc50014b9d0 tx=0x7fc4f0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:47.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.150+0000 7fc4f5ffb640 1 -- 172.21.15.32:0/3784406237 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc50015e490 con 0x7fc500074210 2024-04-24T23:02:47.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.335+0000 7fc507329640 1 -- 172.21.15.32:0/3784406237 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fc5000b3980 con 0x7fc500074210 2024-04-24T23:02:47.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.339+0000 7fc4f5ffb640 1 -- 172.21.15.32:0/3784406237 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v7) ==== 104+0+0 (secure 0 0 0) 0x7fc4fc059690 con 0x7fc500074210 2024-04-24T23:02:47.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.341+0000 7fc507329640 1 -- 172.21.15.32:0/3784406237 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc4e406be30 msgr2=0x7fc4e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:47.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.341+0000 7fc507329640 1 --2- 172.21.15.32:0/3784406237 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc4e406be30 0x7fc4e406e2f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc50014b9d0 tx=0x7fc4f0008040 comp rx=0 tx=0).stop 2024-04-24T23:02:47.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.341+0000 7fc507329640 1 -- 172.21.15.32:0/3784406237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc500074210 msgr2=0x7fc50015db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:47.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.341+0000 7fc507329640 1 --2- 172.21.15.32:0/3784406237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc500074210 0x7fc50015db50 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc00e8a0 tx=0x7fc4fc00ed60 comp rx=0 tx=0).stop 2024-04-24T23:02:47.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.341+0000 7fc507329640 1 -- 172.21.15.32:0/3784406237 shutdown_connections 2024-04-24T23:02:47.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.341+0000 7fc507329640 1 --2- 172.21.15.32:0/3784406237 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc4e406be30 0x7fc4e406e2f0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:47.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.341+0000 7fc507329640 1 --2- 172.21.15.32:0/3784406237 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc50014bb00 0x7fc5000c83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:47.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.341+0000 7fc507329640 1 --2- 172.21.15.32:0/3784406237 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc5000ae140 0x7fc50014e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:47.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.341+0000 7fc507329640 1 --2- 172.21.15.32:0/3784406237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc500074210 0x7fc50015db50 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:47.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.341+0000 7fc507329640 1 -- 172.21.15.32:0/3784406237 >> 172.21.15.32:0/3784406237 conn(0x7fc5000b2e60 msgr2=0x7fc5000b1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:47.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.341+0000 7fc507329640 1 -- 172.21.15.32:0/3784406237 shutdown_connections 2024-04-24T23:02:47.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.341+0000 7fc507329640 1 -- 172.21.15.32:0/3784406237 wait complete. 2024-04-24T23:02:47.353 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fclose' 2024-04-24T23:02:47.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.454+0000 7f59fb6d4640 1 Processor -- start 2024-04-24T23:02:47.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.454+0000 7f59fb6d4640 1 -- start start 2024-04-24T23:02:47.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.455+0000 7f59fb6d4640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f59f415b380 0x7f59f406c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:47.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.455+0000 7f59fb6d4640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f59f40ceb40 0x7f59f406cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:47.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.455+0000 7f59fb6d4640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59f40cef50 con 0x7f59f415cb00 2024-04-24T23:02:47.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.455+0000 7f59fb6d4640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f59f40b4520 con 0x7f59f415b380 2024-04-24T23:02:47.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.455+0000 7f59fb6d4640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f59f40b4800 con 0x7f59f40ceb40 2024-04-24T23:02:47.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.455+0000 7f59f9470640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f59f415b380 0x7f59f406c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:47.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.455+0000 7f59f9470640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f59f415b380 0x7f59f406c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42104/0 (socket says 172.21.15.32:42104) 2024-04-24T23:02:47.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.455+0000 7f59f9470640 1 -- 172.21.15.32:0/3666566525 learned_addr learned my addr 172.21.15.32:0/3666566525 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:47.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.455+0000 7f59f9470640 1 -- 172.21.15.32:0/3666566525 >> v2:172.21.15.121:3300/0 conn(0x7f59f40ceb40 msgr2=0x7f59f406cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:47.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.455+0000 7f59f9c71640 1 --2- 172.21.15.32:0/3666566525 >> v2:172.21.15.121:3300/0 conn(0x7f59f40ceb40 0x7f59f406cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:47.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.455+0000 7f59f9470640 1 --2- 172.21.15.32:0/3666566525 >> v2:172.21.15.121:3300/0 conn(0x7f59f40ceb40 0x7f59f406cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:47.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.456+0000 7f59f9470640 1 -- 172.21.15.32:0/3666566525 >> v1:172.21.15.121:6789/0 conn(0x7f59f415cb00 legacy=0x7f59f40ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:47.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.456+0000 7f59f9470640 1 -- 172.21.15.32:0/3666566525 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59f40b4ae0 con 0x7f59f415b380 2024-04-24T23:02:47.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.456+0000 7f59f9c71640 1 --2- 172.21.15.32:0/3666566525 >> v2:172.21.15.121:3300/0 conn(0x7f59f40ceb40 0x7f59f406cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:02:47.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.456+0000 7f59f9470640 1 --2- 172.21.15.32:0/3666566525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59f415b380 0x7f59f406c5f0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f59f4070490 tx=0x7f59ec02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54cc3f2d07602af server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:47.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.456+0000 7f59f3fff640 1 -- 172.21.15.32:0/3666566525 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ec037070 con 0x7f59f415b380 2024-04-24T23:02:47.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.456+0000 7f59f3fff640 1 -- 172.21.15.32:0/3666566525 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59ec02ae10 con 0x7f59f415b380 2024-04-24T23:02:47.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.456+0000 7f59f3fff640 1 -- 172.21.15.32:0/3666566525 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ec032660 con 0x7f59f415b380 2024-04-24T23:02:47.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.457+0000 7f59fb6d4640 1 -- 172.21.15.32:0/3666566525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59f415b380 msgr2=0x7f59f406c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:47.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.457+0000 7f59fb6d4640 1 --2- 172.21.15.32:0/3666566525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59f415b380 0x7f59f406c5f0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f59f4070490 tx=0x7f59ec02a100 comp rx=0 tx=0).stop 2024-04-24T23:02:47.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.457+0000 7f59fb6d4640 1 -- 172.21.15.32:0/3666566525 shutdown_connections 2024-04-24T23:02:47.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.457+0000 7f59fb6d4640 1 --2- 172.21.15.32:0/3666566525 >> v2:172.21.15.121:3300/0 conn(0x7f59f40ceb40 0x7f59f406cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:47.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.457+0000 7f59fb6d4640 1 --2- 172.21.15.32:0/3666566525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59f415b380 0x7f59f406c5f0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:47.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.457+0000 7f59fb6d4640 1 -- 172.21.15.32:0/3666566525 >> 172.21.15.32:0/3666566525 conn(0x7f59f406b110 msgr2=0x7f59f406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:47.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.457+0000 7f59fb6d4640 1 -- 172.21.15.32:0/3666566525 shutdown_connections 2024-04-24T23:02:47.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.457+0000 7f59fb6d4640 1 -- 172.21.15.32:0/3666566525 wait complete. 2024-04-24T23:02:47.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.458+0000 7f59fb6d4640 1 Processor -- start 2024-04-24T23:02:47.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.458+0000 7f59fb6d4640 1 -- start start 2024-04-24T23:02:47.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.459+0000 7f59fb6d4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59f40ceb40 0x7f59f4072070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:47.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.459+0000 7f59fb6d4640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59f415b380 0x7f59f4143640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:47.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.459+0000 7f59fb6d4640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f59f415cb00 0x7f59f41564f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:47.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.459+0000 7f59fb6d4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f59f415b760 con 0x7f59f40ceb40 2024-04-24T23:02:47.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.459+0000 7f59fb6d4640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f59f40b50a0 con 0x7f59f415b380 2024-04-24T23:02:47.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.459+0000 7f59fb6d4640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f59f40b55c0 con 0x7f59f415cb00 2024-04-24T23:02:47.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.459+0000 7f59f9470640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59f40ceb40 0x7f59f4072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:47.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.459+0000 7f59f9470640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59f40ceb40 0x7f59f4072070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42120/0 (socket says 172.21.15.32:42120) 2024-04-24T23:02:47.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.459+0000 7f59f9470640 1 -- 172.21.15.32:0/821333471 learned_addr learned my addr 172.21.15.32:0/821333471 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:47.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.459+0000 7f59f9470640 1 -- 172.21.15.32:0/821333471 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f59f415cb00 msgr2=0x7f59f41564f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:47.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.459+0000 7f59f9470640 1 --2- 172.21.15.32:0/821333471 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f59f415cb00 0x7f59f41564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:47.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.459+0000 7f59f9470640 1 -- 172.21.15.32:0/821333471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59f415b380 msgr2=0x7f59f4143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:47.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.459+0000 7f59f8c6f640 1 --2- 172.21.15.32:0/821333471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59f415b380 0x7f59f4143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:47.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.459+0000 7f59f9470640 1 --2- 172.21.15.32:0/821333471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59f415b380 0x7f59f4143640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:47.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.459+0000 7f59f9470640 1 -- 172.21.15.32:0/821333471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59f4143b80 con 0x7f59f40ceb40 2024-04-24T23:02:47.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.459+0000 7f59f8c6f640 1 --2- 172.21.15.32:0/821333471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59f415b380 0x7f59f4143640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:02:47.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.459+0000 7f59f9470640 1 --2- 172.21.15.32:0/821333471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59f40ceb40 0x7f59f4072070 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f59ec000c00 tx=0x7f59ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:47.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.460+0000 7f59f1ffb640 1 -- 172.21.15.32:0/821333471 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ec041070 con 0x7f59f40ceb40 2024-04-24T23:02:47.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.460+0000 7f59f1ffb640 1 -- 172.21.15.32:0/821333471 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59ec03c670 con 0x7f59f40ceb40 2024-04-24T23:02:47.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.460+0000 7f59fb6d4640 1 -- 172.21.15.32:0/821333471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59f40b37d0 con 0x7f59f40ceb40 2024-04-24T23:02:47.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.460+0000 7f59f1ffb640 1 -- 172.21.15.32:0/821333471 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ec037040 con 0x7f59f40ceb40 2024-04-24T23:02:47.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.460+0000 7f59fb6d4640 1 -- 172.21.15.32:0/821333471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59f40b3d90 con 0x7f59f40ceb40 2024-04-24T23:02:47.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.461+0000 7f59fb6d4640 1 -- 172.21.15.32:0/821333471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59f4068430 con 0x7f59f40ceb40 2024-04-24T23:02:47.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.461+0000 7f59f1ffb640 1 -- 172.21.15.32:0/821333471 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f59ec044070 con 0x7f59f40ceb40 2024-04-24T23:02:47.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.462+0000 7f59f1ffb640 1 --2- 172.21.15.32:0/821333471 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f59e006be30 0x7f59e006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:47.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.462+0000 7f59f1ffb640 1 -- 172.21.15.32:0/821333471 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 4943+0+0 (secure 0 0 0) 0x7f59ec0ab3e0 con 0x7f59f40ceb40 2024-04-24T23:02:47.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.462+0000 7f59f8c6f640 1 --2- 172.21.15.32:0/821333471 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f59e006be30 0x7f59e006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:47.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.463+0000 7f59f8c6f640 1 --2- 172.21.15.32:0/821333471 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f59e006be30 0x7f59e006e2f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f59f4066f50 tx=0x7f59e8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:47.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.466+0000 7f59f1ffb640 1 -- 172.21.15.32:0/821333471 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f59f40cef50 con 0x7f59f40ceb40 2024-04-24T23:02:47.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:47.729+0000 7f59fb6d4640 1 -- 172.21.15.32:0/821333471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]} v 0) -- 0x7f59f40afd60 con 0x7f59f40ceb40 2024-04-24T23:02:48.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.018+0000 7f59f1ffb640 1 -- 172.21.15.32:0/821333471 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]}]=0 v204) ==== 134+0+0 (secure 0 0 0) 0x7f59ec07a090 con 0x7f59f40ceb40 2024-04-24T23:02:48.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.021+0000 7f59fb6d4640 1 -- 172.21.15.32:0/821333471 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f59e006be30 msgr2=0x7f59e006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:48.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.021+0000 7f59fb6d4640 1 --2- 172.21.15.32:0/821333471 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f59e006be30 0x7f59e006e2f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f59f4066f50 tx=0x7f59e8008040 comp rx=0 tx=0).stop 2024-04-24T23:02:48.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.021+0000 7f59fb6d4640 1 -- 172.21.15.32:0/821333471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59f40ceb40 msgr2=0x7f59f4072070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:48.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.021+0000 7f59fb6d4640 1 --2- 172.21.15.32:0/821333471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59f40ceb40 0x7f59f4072070 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f59ec000c00 tx=0x7f59ec0046c0 comp rx=0 tx=0).stop 2024-04-24T23:02:48.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.021+0000 7f59fb6d4640 1 -- 172.21.15.32:0/821333471 shutdown_connections 2024-04-24T23:02:48.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.021+0000 7f59fb6d4640 1 --2- 172.21.15.32:0/821333471 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f59e006be30 0x7f59e006e2f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:48.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.021+0000 7f59fb6d4640 1 --2- 172.21.15.32:0/821333471 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f59f415cb00 0x7f59f41564f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:48.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.021+0000 7f59fb6d4640 1 --2- 172.21.15.32:0/821333471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59f415b380 0x7f59f4143640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:48.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.021+0000 7f59fb6d4640 1 --2- 172.21.15.32:0/821333471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59f40ceb40 0x7f59f4072070 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:48.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.021+0000 7f59fb6d4640 1 -- 172.21.15.32:0/821333471 >> 172.21.15.32:0/821333471 conn(0x7f59f406b110 msgr2=0x7f59f40cd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:48.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.021+0000 7f59fb6d4640 1 -- 172.21.15.32:0/821333471 shutdown_connections 2024-04-24T23:02:48.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.021+0000 7f59fb6d4640 1 -- 172.21.15.32:0/821333471 wait complete. 2024-04-24T23:02:48.034 INFO:tasks.cephfs_test_runner:test_full_fclose (tasks.cephfs.test_full.TestClusterFull) ... ok 2024-04-24T23:02:48.035 INFO:tasks.cephfs_test_runner:Starting test: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) That when the full flag is encountered during asynchronous 2024-04-24T23:02:48.035 INFO:tasks.cephfs_test_runner:test_full_fsync (tasks.cephfs.test_full.TestClusterFull) 2024-04-24T23:02:48.035 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:02:48.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.135+0000 7fbef2309640 1 Processor -- start 2024-04-24T23:02:48.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.135+0000 7fbef2309640 1 -- start start 2024-04-24T23:02:48.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.136+0000 7fbef2309640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fbeec15e9b0 0x7fbeec0d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:48.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.136+0000 7fbef2309640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbeec0d4210 0x7fbeec06b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:48.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.136+0000 7fbef2309640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbeec0ba0e0 0x7fbeec06b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:48.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.136+0000 7fbef2309640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fbeec0d3170 con 0x7fbeec0ba0e0 2024-04-24T23:02:48.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.136+0000 7fbef2309640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fbeec0d3450 con 0x7fbeec15e9b0 2024-04-24T23:02:48.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.136+0000 7fbef2309640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fbeec0d3730 con 0x7fbeec0d4210 2024-04-24T23:02:48.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.136+0000 7fbef08a6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbeec0ba0e0 0x7fbeec06b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:48.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.136+0000 7fbef08a6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbeec0ba0e0 0x7fbeec06b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42124/0 (socket says 172.21.15.32:42124) 2024-04-24T23:02:48.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.136+0000 7fbef08a6640 1 -- 172.21.15.32:0/2034339613 learned_addr learned my addr 172.21.15.32:0/2034339613 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:48.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.137+0000 7fbef08a6640 1 -- 172.21.15.32:0/2034339613 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbeec0d4210 msgr2=0x7fbeec06b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:48.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.137+0000 7fbef08a6640 1 --2- 172.21.15.32:0/2034339613 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbeec0d4210 0x7fbeec06b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:48.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.137+0000 7fbef08a6640 1 -- 172.21.15.32:0/2034339613 >> v2:172.21.15.121:3300/0 conn(0x7fbeec15e9b0 msgr2=0x7fbeec0d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:48.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.137+0000 7fbef08a6640 1 --2- 172.21.15.32:0/2034339613 >> v2:172.21.15.121:3300/0 conn(0x7fbeec15e9b0 0x7fbeec0d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:48.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.137+0000 7fbef08a6640 1 -- 172.21.15.32:0/2034339613 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeec0d3a10 con 0x7fbeec0ba0e0 2024-04-24T23:02:48.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.137+0000 7fbef08a6640 1 --2- 172.21.15.32:0/2034339613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbeec0ba0e0 0x7fbeec06b650 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fbeec06be20 tx=0x7fbee402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd97c5c6b82af1a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:48.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.138+0000 7fbeeaffd640 1 -- 172.21.15.32:0/2034339613 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee4036070 con 0x7fbeec0ba0e0 2024-04-24T23:02:48.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.138+0000 7fbeeaffd640 1 -- 172.21.15.32:0/2034339613 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbee4031600 con 0x7fbeec0ba0e0 2024-04-24T23:02:48.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.138+0000 7fbeeaffd640 1 -- 172.21.15.32:0/2034339613 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee40319d0 con 0x7fbeec0ba0e0 2024-04-24T23:02:48.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.138+0000 7fbef2309640 1 -- 172.21.15.32:0/2034339613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbeec0ba0e0 msgr2=0x7fbeec06b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:48.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.138+0000 7fbef2309640 1 --2- 172.21.15.32:0/2034339613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbeec0ba0e0 0x7fbeec06b650 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fbeec06be20 tx=0x7fbee402a6a0 comp rx=0 tx=0).stop 2024-04-24T23:02:48.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.138+0000 7fbef2309640 1 -- 172.21.15.32:0/2034339613 shutdown_connections 2024-04-24T23:02:48.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.138+0000 7fbef2309640 1 --2- 172.21.15.32:0/2034339613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbeec0ba0e0 0x7fbeec06b650 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:48.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.138+0000 7fbef2309640 1 --2- 172.21.15.32:0/2034339613 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbeec0d4210 0x7fbeec06b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:48.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.138+0000 7fbef2309640 1 --2- 172.21.15.32:0/2034339613 >> v2:172.21.15.121:3300/0 conn(0x7fbeec15e9b0 0x7fbeec0d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:48.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.138+0000 7fbef2309640 1 -- 172.21.15.32:0/2034339613 >> 172.21.15.32:0/2034339613 conn(0x7fbeec0d11d0 msgr2=0x7fbeec0d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:48.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.138+0000 7fbef2309640 1 -- 172.21.15.32:0/2034339613 shutdown_connections 2024-04-24T23:02:48.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.138+0000 7fbef2309640 1 -- 172.21.15.32:0/2034339613 wait complete. 2024-04-24T23:02:48.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.139+0000 7fbef2309640 1 Processor -- start 2024-04-24T23:02:48.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.140+0000 7fbef2309640 1 -- start start 2024-04-24T23:02:48.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.140+0000 7fbef2309640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbeec0ba0e0 0x7fbeec14cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:48.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.140+0000 7fbef2309640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbeec0d4210 0x7fbeec15fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:48.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.140+0000 7fbef2309640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbeec15e9b0 0x7fbeec1600c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:48.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.140+0000 7fbef2309640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbeec071730 con 0x7fbeec0d4210 2024-04-24T23:02:48.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.140+0000 7fbef2309640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fbeec0d2400 con 0x7fbeec0ba0e0 2024-04-24T23:02:48.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.140+0000 7fbef2309640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fbeec0718b0 con 0x7fbeec15e9b0 2024-04-24T23:02:48.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.140+0000 7fbeeb7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbeec0d4210 0x7fbeec15fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:48.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.140+0000 7fbeeb7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbeec0d4210 0x7fbeec15fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42134/0 (socket says 172.21.15.32:42134) 2024-04-24T23:02:48.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.140+0000 7fbeeb7fe640 1 -- 172.21.15.32:0/2399619987 learned_addr learned my addr 172.21.15.32:0/2399619987 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:48.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.141+0000 7fbef08a6640 1 --2- 172.21.15.32:0/2399619987 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbeec15e9b0 0x7fbeec1600c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:48.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.141+0000 7fbeebfff640 1 --2- 172.21.15.32:0/2399619987 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbeec0ba0e0 0x7fbeec14cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:48.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.141+0000 7fbeeb7fe640 1 -- 172.21.15.32:0/2399619987 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbeec15e9b0 msgr2=0x7fbeec1600c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:48.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.141+0000 7fbeeb7fe640 1 --2- 172.21.15.32:0/2399619987 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbeec15e9b0 0x7fbeec1600c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:48.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.141+0000 7fbeeb7fe640 1 -- 172.21.15.32:0/2399619987 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbeec0ba0e0 msgr2=0x7fbeec14cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:48.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.141+0000 7fbeeb7fe640 1 --2- 172.21.15.32:0/2399619987 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbeec0ba0e0 0x7fbeec14cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:48.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.141+0000 7fbeeb7fe640 1 -- 172.21.15.32:0/2399619987 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeec0cd260 con 0x7fbeec0d4210 2024-04-24T23:02:48.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.141+0000 7fbeeb7fe640 1 --2- 172.21.15.32:0/2399619987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbeec0d4210 0x7fbeec15fb80 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fbee000b4a0 tx=0x7fbee000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:48.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.141+0000 7fbee8ff9640 1 -- 172.21.15.32:0/2399619987 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee0014070 con 0x7fbeec0d4210 2024-04-24T23:02:48.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.141+0000 7fbee8ff9640 1 -- 172.21.15.32:0/2399619987 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbee0010610 con 0x7fbeec0d4210 2024-04-24T23:02:48.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.141+0000 7fbef2309640 1 -- 172.21.15.32:0/2399619987 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeec156550 con 0x7fbeec0d4210 2024-04-24T23:02:48.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.141+0000 7fbee8ff9640 1 -- 172.21.15.32:0/2399619987 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee0010a00 con 0x7fbeec0d4210 2024-04-24T23:02:48.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.141+0000 7fbef2309640 1 -- 172.21.15.32:0/2399619987 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeec156b10 con 0x7fbeec0d4210 2024-04-24T23:02:48.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.142+0000 7fbef2309640 1 -- 172.21.15.32:0/2399619987 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbeec068430 con 0x7fbeec0d4210 2024-04-24T23:02:48.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.142+0000 7fbee8ff9640 1 -- 172.21.15.32:0/2399619987 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbee0017070 con 0x7fbeec0d4210 2024-04-24T23:02:48.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.143+0000 7fbee8ff9640 1 --2- 172.21.15.32:0/2399619987 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fbed806be30 0x7fbed806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:48.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.143+0000 7fbee8ff9640 1 -- 172.21.15.32:0/2399619987 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 4943+0+0 (secure 0 0 0) 0x7fbee008a4d0 con 0x7fbeec0d4210 2024-04-24T23:02:48.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.143+0000 7fbeebfff640 1 --2- 172.21.15.32:0/2399619987 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fbed806be30 0x7fbed806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:48.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.144+0000 7fbeebfff640 1 --2- 172.21.15.32:0/2399619987 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fbed806be30 0x7fbed806e2f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fbedc0098e0 tx=0x7fbedc009060 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:48.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.147+0000 7fbee8ff9640 1 -- 172.21.15.32:0/2399619987 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbee0058b30 con 0x7fbeec0d4210 2024-04-24T23:02:48.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.329+0000 7fbef2309640 1 -- 172.21.15.32:0/2399619987 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbeec0d1850 con 0x7fbeec0d4210 2024-04-24T23:02:48.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.330+0000 7fbee8ff9640 1 -- 172.21.15.32:0/2399619987 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+13034 (secure 0 0 0) 0x7fbee005ce30 con 0x7fbeec0d4210 2024-04-24T23:02:48.332 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:02:48.332 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":78,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:02:47.011656+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.121:6836/2650840833":"2024-04-25T23:02:44.004104+0000","172.21.15.121:6834/2650840833":"2024-04-25T23:02:44.004104+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:02:48.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.332+0000 7fbef2309640 1 -- 172.21.15.32:0/2399619987 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fbed806be30 msgr2=0x7fbed806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:48.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.332+0000 7fbef2309640 1 --2- 172.21.15.32:0/2399619987 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fbed806be30 0x7fbed806e2f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fbedc0098e0 tx=0x7fbedc009060 comp rx=0 tx=0).stop 2024-04-24T23:02:48.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.332+0000 7fbef2309640 1 -- 172.21.15.32:0/2399619987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbeec0d4210 msgr2=0x7fbeec15fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:48.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.332+0000 7fbef2309640 1 --2- 172.21.15.32:0/2399619987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbeec0d4210 0x7fbeec15fb80 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fbee000b4a0 tx=0x7fbee000b960 comp rx=0 tx=0).stop 2024-04-24T23:02:48.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.332+0000 7fbef2309640 1 -- 172.21.15.32:0/2399619987 shutdown_connections 2024-04-24T23:02:48.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.332+0000 7fbef2309640 1 --2- 172.21.15.32:0/2399619987 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fbed806be30 0x7fbed806e2f0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:48.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.332+0000 7fbef2309640 1 --2- 172.21.15.32:0/2399619987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbeec0d4210 0x7fbeec15fb80 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:48.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.332+0000 7fbef2309640 1 --2- 172.21.15.32:0/2399619987 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbeec15e9b0 0x7fbeec1600c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:48.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.332+0000 7fbef2309640 1 --2- 172.21.15.32:0/2399619987 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbeec0ba0e0 0x7fbeec14cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:48.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.332+0000 7fbef2309640 1 -- 172.21.15.32:0/2399619987 >> 172.21.15.32:0/2399619987 conn(0x7fbeec0d11d0 msgr2=0x7fbeec0b78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:48.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.332+0000 7fbef2309640 1 -- 172.21.15.32:0/2399619987 shutdown_connections 2024-04-24T23:02:48.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.332+0000 7fbef2309640 1 -- 172.21.15.32:0/2399619987 wait complete. 2024-04-24T23:02:48.345 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:02:48.345 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:02:48.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.446+0000 7f1b4bd9d640 1 Processor -- start 2024-04-24T23:02:48.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.447+0000 7f1b4bd9d640 1 -- start start 2024-04-24T23:02:48.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.447+0000 7f1b4bd9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1b4406b820 0x7f1b440d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:48.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.447+0000 7f1b4bd9d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b440d4210 0x7f1b440b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:48.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.447+0000 7f1b4bd9d640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b440d3170 con 0x7f1b4415e9b0 2024-04-24T23:02:48.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.447+0000 7f1b4bd9d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f1b440d3450 con 0x7f1b4406b820 2024-04-24T23:02:48.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.447+0000 7f1b4bd9d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f1b440d3730 con 0x7f1b440d4210 2024-04-24T23:02:48.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.447+0000 7f1b49338640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1b4406b820 0x7f1b440d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:48.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.447+0000 7f1b49338640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1b4406b820 0x7f1b440d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42146/0 (socket says 172.21.15.32:42146) 2024-04-24T23:02:48.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.448+0000 7f1b49338640 1 -- 172.21.15.32:0/1459008567 learned_addr learned my addr 172.21.15.32:0/1459008567 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:48.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.448+0000 7f1b49338640 1 -- 172.21.15.32:0/1459008567 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b440d4210 msgr2=0x7f1b440b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:48.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.448+0000 7f1b49338640 1 --2- 172.21.15.32:0/1459008567 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b440d4210 0x7f1b440b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:48.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.448+0000 7f1b49338640 1 -- 172.21.15.32:0/1459008567 >> v1:172.21.15.121:6789/0 conn(0x7f1b4415e9b0 legacy=0x7f1b4406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:48.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.448+0000 7f1b49338640 1 -- 172.21.15.32:0/1459008567 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b440d3a10 con 0x7f1b4406b820 2024-04-24T23:02:48.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.448+0000 7f1b49338640 1 --2- 172.21.15.32:0/1459008567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b4406b820 0x7f1b440d2b90 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f1b38009870 tx=0x7f1b3802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c72ad8fc7f7eefe server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:48.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.449+0000 7f1b48b37640 1 -- 172.21.15.32:0/1459008567 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b38037070 con 0x7f1b4406b820 2024-04-24T23:02:48.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.449+0000 7f1b48b37640 1 -- 172.21.15.32:0/1459008567 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b38032440 con 0x7f1b4406b820 2024-04-24T23:02:48.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.449+0000 7f1b48b37640 1 -- 172.21.15.32:0/1459008567 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b38032810 con 0x7f1b4406b820 2024-04-24T23:02:48.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.449+0000 7f1b4bd9d640 1 -- 172.21.15.32:0/1459008567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b4406b820 msgr2=0x7f1b440d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:48.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.449+0000 7f1b4bd9d640 1 --2- 172.21.15.32:0/1459008567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b4406b820 0x7f1b440d2b90 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f1b38009870 tx=0x7f1b3802a210 comp rx=0 tx=0).stop 2024-04-24T23:02:48.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.449+0000 7f1b4bd9d640 1 -- 172.21.15.32:0/1459008567 shutdown_connections 2024-04-24T23:02:48.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.449+0000 7f1b4bd9d640 1 --2- 172.21.15.32:0/1459008567 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b440d4210 0x7f1b440b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:48.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.449+0000 7f1b4bd9d640 1 --2- 172.21.15.32:0/1459008567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b4406b820 0x7f1b440d2b90 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:48.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.449+0000 7f1b4bd9d640 1 -- 172.21.15.32:0/1459008567 >> 172.21.15.32:0/1459008567 conn(0x7f1b440d11d0 msgr2=0x7f1b440d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:48.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.449+0000 7f1b4bd9d640 1 -- 172.21.15.32:0/1459008567 shutdown_connections 2024-04-24T23:02:48.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.449+0000 7f1b4bd9d640 1 -- 172.21.15.32:0/1459008567 wait complete. 2024-04-24T23:02:48.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.450+0000 7f1b4bd9d640 1 Processor -- start 2024-04-24T23:02:48.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.451+0000 7f1b4bd9d640 1 -- start start 2024-04-24T23:02:48.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.451+0000 7f1b4bd9d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b4406b820 0x7f1b440ce630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:48.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.451+0000 7f1b4bd9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b440d4210 0x7f1b440ca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:48.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.451+0000 7f1b4bd9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b4415e9b0 0x7f1b44159ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:48.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.451+0000 7f1b4bd9d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1b440ba0e0 con 0x7f1b4415e9b0 2024-04-24T23:02:48.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.451+0000 7f1b4bd9d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1b440d2400 con 0x7f1b440d4210 2024-04-24T23:02:48.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.451+0000 7f1b4bd9d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f1b440ba260 con 0x7f1b4406b820 2024-04-24T23:02:48.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.451+0000 7f1b4a33a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b4415e9b0 0x7f1b44159ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:48.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.451+0000 7f1b4a33a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b4415e9b0 0x7f1b44159ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42154/0 (socket says 172.21.15.32:42154) 2024-04-24T23:02:48.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.451+0000 7f1b4a33a640 1 -- 172.21.15.32:0/2237012636 learned_addr learned my addr 172.21.15.32:0/2237012636 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:48.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.451+0000 7f1b49b39640 1 --2- 172.21.15.32:0/2237012636 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b4406b820 0x7f1b440ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:48.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.451+0000 7f1b49338640 1 --2- 172.21.15.32:0/2237012636 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b440d4210 0x7f1b440ca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:48.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.452+0000 7f1b4a33a640 1 -- 172.21.15.32:0/2237012636 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b4406b820 msgr2=0x7f1b440ce630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:48.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.452+0000 7f1b4a33a640 1 --2- 172.21.15.32:0/2237012636 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b4406b820 0x7f1b440ce630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:48.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.452+0000 7f1b4a33a640 1 -- 172.21.15.32:0/2237012636 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b440d4210 msgr2=0x7f1b440ca9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:48.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.452+0000 7f1b4a33a640 1 --2- 172.21.15.32:0/2237012636 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b440d4210 0x7f1b440ca9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:48.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.452+0000 7f1b4a33a640 1 -- 172.21.15.32:0/2237012636 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b440ceb70 con 0x7f1b4415e9b0 2024-04-24T23:02:48.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.452+0000 7f1b49b39640 1 --2- 172.21.15.32:0/2237012636 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b4406b820 0x7f1b440ce630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:02:48.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.452+0000 7f1b49338640 1 --2- 172.21.15.32:0/2237012636 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b440d4210 0x7f1b440ca9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:02:48.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.452+0000 7f1b4a33a640 1 --2- 172.21.15.32:0/2237012636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b4415e9b0 0x7f1b44159ac0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f1b3400b9e0 tx=0x7f1b3400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:48.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.452+0000 7f1b427fc640 1 -- 172.21.15.32:0/2237012636 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b34019070 con 0x7f1b4415e9b0 2024-04-24T23:02:48.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.452+0000 7f1b427fc640 1 -- 172.21.15.32:0/2237012636 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b3400cb20 con 0x7f1b4415e9b0 2024-04-24T23:02:48.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.452+0000 7f1b4bd9d640 1 -- 172.21.15.32:0/2237012636 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b4415f350 con 0x7f1b4415e9b0 2024-04-24T23:02:48.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.452+0000 7f1b427fc640 1 -- 172.21.15.32:0/2237012636 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b34014440 con 0x7f1b4415e9b0 2024-04-24T23:02:48.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.452+0000 7f1b4bd9d640 1 -- 172.21.15.32:0/2237012636 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b4415f910 con 0x7f1b4415e9b0 2024-04-24T23:02:48.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.453+0000 7f1b4bd9d640 1 -- 172.21.15.32:0/2237012636 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b44068430 con 0x7f1b4415e9b0 2024-04-24T23:02:48.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.453+0000 7f1b427fc640 1 -- 172.21.15.32:0/2237012636 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1b3401c070 con 0x7f1b4415e9b0 2024-04-24T23:02:48.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.454+0000 7f1b427fc640 1 --2- 172.21.15.32:0/2237012636 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1b3006be30 0x7f1b3006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:48.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.454+0000 7f1b427fc640 1 -- 172.21.15.32:0/2237012636 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 4943+0+0 (secure 0 0 0) 0x7f1b3408a890 con 0x7f1b4415e9b0 2024-04-24T23:02:48.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.454+0000 7f1b49b39640 1 --2- 172.21.15.32:0/2237012636 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1b3006be30 0x7f1b3006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:48.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.455+0000 7f1b49b39640 1 --2- 172.21.15.32:0/2237012636 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1b3006be30 0x7f1b3006e2f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c006fd0 tx=0x7f1b3c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:48.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.458+0000 7f1b427fc640 1 -- 172.21.15.32:0/2237012636 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1b34059540 con 0x7f1b4415e9b0 2024-04-24T23:02:48.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.639+0000 7f1b4bd9d640 1 -- 172.21.15.32:0/2237012636 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1b440b91e0 con 0x7f1b4415e9b0 2024-04-24T23:02:48.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.639+0000 7f1b427fc640 1 -- 172.21.15.32:0/2237012636 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) ==== 93+0+10 (secure 0 0 0) 0x7f1b34058e40 con 0x7f1b4415e9b0 2024-04-24T23:02:48.641 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T23:02:48.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.641+0000 7f1b4bd9d640 1 -- 172.21.15.32:0/2237012636 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1b3006be30 msgr2=0x7f1b3006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:48.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.641+0000 7f1b4bd9d640 1 --2- 172.21.15.32:0/2237012636 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1b3006be30 0x7f1b3006e2f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c006fd0 tx=0x7f1b3c008040 comp rx=0 tx=0).stop 2024-04-24T23:02:48.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.641+0000 7f1b4bd9d640 1 -- 172.21.15.32:0/2237012636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b4415e9b0 msgr2=0x7f1b44159ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:48.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.641+0000 7f1b4bd9d640 1 --2- 172.21.15.32:0/2237012636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b4415e9b0 0x7f1b44159ac0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f1b3400b9e0 tx=0x7f1b3400bea0 comp rx=0 tx=0).stop 2024-04-24T23:02:48.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.641+0000 7f1b4bd9d640 1 -- 172.21.15.32:0/2237012636 shutdown_connections 2024-04-24T23:02:48.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.641+0000 7f1b4bd9d640 1 --2- 172.21.15.32:0/2237012636 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1b3006be30 0x7f1b3006e2f0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:48.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.641+0000 7f1b4bd9d640 1 --2- 172.21.15.32:0/2237012636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b4415e9b0 0x7f1b44159ac0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:48.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.641+0000 7f1b4bd9d640 1 --2- 172.21.15.32:0/2237012636 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b440d4210 0x7f1b440ca9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:48.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.641+0000 7f1b4bd9d640 1 --2- 172.21.15.32:0/2237012636 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b4406b820 0x7f1b440ce630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:48.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.641+0000 7f1b4bd9d640 1 -- 172.21.15.32:0/2237012636 >> 172.21.15.32:0/2237012636 conn(0x7f1b440d11d0 msgr2=0x7f1b440d15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:48.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.641+0000 7f1b4bd9d640 1 -- 172.21.15.32:0/2237012636 shutdown_connections 2024-04-24T23:02:48.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.641+0000 7f1b4bd9d640 1 -- 172.21.15.32:0/2237012636 wait complete. 2024-04-24T23:02:48.654 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fsync' 2024-04-24T23:02:48.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.753+0000 7f7f6c8bd640 1 Processor -- start 2024-04-24T23:02:48.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.753+0000 7f7f6c8bd640 1 -- start start 2024-04-24T23:02:48.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.754+0000 7f7f6c8bd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7f680b4320 0x7f7f680b6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:48.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.754+0000 7f7f6c8bd640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f680cd910 con 0x7f7f6815edd0 2024-04-24T23:02:48.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.754+0000 7f7f6c8bd640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f680cdbf0 con 0x7f7f680b5560 2024-04-24T23:02:48.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.754+0000 7f7f6c8bd640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f7f680cded0 con 0x7f7f680b4320 2024-04-24T23:02:48.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.754+0000 7f7f66d9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7f680b4320 0x7f7f680b6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:48.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.754+0000 7f7f65d9b640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7f6815edd0 0x7f7f680b5940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46232/0 (socket says 172.21.15.32:46232) 2024-04-24T23:02:48.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.754+0000 7f7f65d9b640 1 -- 172.21.15.32:0/3863903442 learned_addr learned my addr 172.21.15.32:0/3863903442 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:02:48.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.754+0000 7f7f66d9d640 1 -- 172.21.15.32:0/3863903442 >> v1:172.21.15.121:6789/0 conn(0x7f7f680b5560 legacy=0x7f7f6815e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:48.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.754+0000 7f7f66d9d640 1 -- 172.21.15.32:0/3863903442 >> v1:172.21.15.32:6789/0 conn(0x7f7f6815edd0 legacy=0x7f7f680b5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:48.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.754+0000 7f7f6559a640 1 -- 172.21.15.32:0/3863903442 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3213475238 0 0) 0x7f7f680cd910 con 0x7f7f6815edd0 2024-04-24T23:02:48.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.754+0000 7f7f66d9d640 1 -- 172.21.15.32:0/3863903442 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f680ce1b0 con 0x7f7f680b4320 2024-04-24T23:02:48.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.755+0000 7f7f66d9d640 1 --2- 172.21.15.32:0/3863903442 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f680b4320 0x7f7f680b6050 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f7f58009d90 tx=0x7f7f5802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f17c49096750c316 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:48.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.755+0000 7f7f6559a640 1 -- 172.21.15.32:0/3863903442 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f58007740 con 0x7f7f680b4320 2024-04-24T23:02:48.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.755+0000 7f7f6559a640 1 -- 172.21.15.32:0/3863903442 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f580078e0 con 0x7f7f680b4320 2024-04-24T23:02:48.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.755+0000 7f7f6559a640 1 -- 172.21.15.32:0/3863903442 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f58007cb0 con 0x7f7f680b4320 2024-04-24T23:02:48.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.756+0000 7f7f6c8bd640 1 -- 172.21.15.32:0/3863903442 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f680b4320 msgr2=0x7f7f680b6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:48.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.756+0000 7f7f6c8bd640 1 --2- 172.21.15.32:0/3863903442 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f680b4320 0x7f7f680b6050 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f7f58009d90 tx=0x7f7f5802a6a0 comp rx=0 tx=0).stop 2024-04-24T23:02:48.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.756+0000 7f7f6c8bd640 1 -- 172.21.15.32:0/3863903442 shutdown_connections 2024-04-24T23:02:48.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.756+0000 7f7f6c8bd640 1 --2- 172.21.15.32:0/3863903442 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f680b4320 0x7f7f680b6050 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:48.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.756+0000 7f7f6c8bd640 1 -- 172.21.15.32:0/3863903442 >> 172.21.15.32:0/3863903442 conn(0x7f7f6806b110 msgr2=0x7f7f6806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:48.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.756+0000 7f7f6c8bd640 1 -- 172.21.15.32:0/3863903442 shutdown_connections 2024-04-24T23:02:48.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.756+0000 7f7f6c8bd640 1 -- 172.21.15.32:0/3863903442 wait complete. 2024-04-24T23:02:48.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.757+0000 7f7f6c8bd640 1 Processor -- start 2024-04-24T23:02:48.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.757+0000 7f7f6c8bd640 1 -- start start 2024-04-24T23:02:48.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.758+0000 7f7f6c8bd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f680b4320 0x7f7f6814ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:48.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.758+0000 7f7f6c8bd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f680b5560 0x7f7f681562a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:48.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.758+0000 7f7f6c8bd640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7f6815edd0 0x7f7f680aeb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:48.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.758+0000 7f7f6c8bd640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7f680cd540 con 0x7f7f680b4320 2024-04-24T23:02:48.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.758+0000 7f7f6c8bd640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7f680cd3c0 con 0x7f7f680b5560 2024-04-24T23:02:48.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.758+0000 7f7f6c8bd640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f7f680cd6c0 con 0x7f7f6815edd0 2024-04-24T23:02:48.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.758+0000 7f7f6659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f680b4320 0x7f7f6814ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:48.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.758+0000 7f7f6659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f680b4320 0x7f7f6814ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42170/0 (socket says 172.21.15.32:42170) 2024-04-24T23:02:48.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.758+0000 7f7f6659c640 1 -- 172.21.15.32:0/286093632 learned_addr learned my addr 172.21.15.32:0/286093632 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:48.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.758+0000 7f7f6659c640 1 -- 172.21.15.32:0/286093632 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7f6815edd0 msgr2=0x7f7f680aeb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:48.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.758+0000 7f7f6659c640 1 --2- 172.21.15.32:0/286093632 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7f6815edd0 0x7f7f680aeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:48.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.758+0000 7f7f6659c640 1 -- 172.21.15.32:0/286093632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f680b5560 msgr2=0x7f7f681562a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:48.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.758+0000 7f7f6659c640 1 --2- 172.21.15.32:0/286093632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f680b5560 0x7f7f681562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:48.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.758+0000 7f7f6659c640 1 -- 172.21.15.32:0/286093632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f6814f440 con 0x7f7f680b4320 2024-04-24T23:02:48.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.759+0000 7f7f6659c640 1 --2- 172.21.15.32:0/286093632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f680b4320 0x7f7f6814ee60 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f7f6000c460 tx=0x7f7f6000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:48.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.759+0000 7f7f5effd640 1 -- 172.21.15.32:0/286093632 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f60016020 con 0x7f7f680b4320 2024-04-24T23:02:48.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.759+0000 7f7f5effd640 1 -- 172.21.15.32:0/286093632 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f60012870 con 0x7f7f680b4320 2024-04-24T23:02:48.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.759+0000 7f7f5effd640 1 -- 172.21.15.32:0/286093632 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f60012c60 con 0x7f7f680b4320 2024-04-24T23:02:48.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.759+0000 7f7f6c8bd640 1 -- 172.21.15.32:0/286093632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f68159530 con 0x7f7f680b4320 2024-04-24T23:02:48.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.759+0000 7f7f6c8bd640 1 -- 172.21.15.32:0/286093632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f68159af0 con 0x7f7f680b4320 2024-04-24T23:02:48.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.760+0000 7f7f6c8bd640 1 -- 172.21.15.32:0/286093632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f68159cf0 con 0x7f7f680b4320 2024-04-24T23:02:48.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.760+0000 7f7f5effd640 1 -- 172.21.15.32:0/286093632 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7f60019020 con 0x7f7f680b4320 2024-04-24T23:02:48.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.761+0000 7f7f5effd640 1 --2- 172.21.15.32:0/286093632 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7f4806be30 0x7f7f4806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:48.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.761+0000 7f7f5effd640 1 -- 172.21.15.32:0/286093632 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 4943+0+0 (secure 0 0 0) 0x7f7f6008a860 con 0x7f7f680b4320 2024-04-24T23:02:48.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.761+0000 7f7f65d9b640 1 --2- 172.21.15.32:0/286093632 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7f4806be30 0x7f7f4806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:48.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.761+0000 7f7f65d9b640 1 --2- 172.21.15.32:0/286093632 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7f4806be30 0x7f7f4806e2f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7f50007fd0 tx=0x7f7f50007960 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:48.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:48.765+0000 7f7f5effd640 1 -- 172.21.15.32:0/286093632 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7f60020030 con 0x7f7f680b4320 2024-04-24T23:02:49.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.018+0000 7f7f6c8bd640 1 -- 172.21.15.32:0/286093632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]} v 0) -- 0x7f7f68068430 con 0x7f7f680b4320 2024-04-24T23:02:49.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.020+0000 7f7f5effd640 1 -- 172.21.15.32:0/286093632 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]}]=0 v205) ==== 136+0+0 (secure 0 0 0) 0x7f7f60059510 con 0x7f7f680b4320 2024-04-24T23:02:49.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.021+0000 7f7f6c8bd640 1 -- 172.21.15.32:0/286093632 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7f4806be30 msgr2=0x7f7f4806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:49.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.021+0000 7f7f6c8bd640 1 --2- 172.21.15.32:0/286093632 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7f4806be30 0x7f7f4806e2f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7f50007fd0 tx=0x7f7f50007960 comp rx=0 tx=0).stop 2024-04-24T23:02:49.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.021+0000 7f7f6c8bd640 1 -- 172.21.15.32:0/286093632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f680b4320 msgr2=0x7f7f6814ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:49.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.021+0000 7f7f6c8bd640 1 --2- 172.21.15.32:0/286093632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f680b4320 0x7f7f6814ee60 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f7f6000c460 tx=0x7f7f6000c920 comp rx=0 tx=0).stop 2024-04-24T23:02:49.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.022+0000 7f7f6c8bd640 1 -- 172.21.15.32:0/286093632 shutdown_connections 2024-04-24T23:02:49.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.022+0000 7f7f6c8bd640 1 --2- 172.21.15.32:0/286093632 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7f4806be30 0x7f7f4806e2f0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:49.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.022+0000 7f7f6c8bd640 1 --2- 172.21.15.32:0/286093632 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7f6815edd0 0x7f7f680aeb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:49.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.022+0000 7f7f6c8bd640 1 --2- 172.21.15.32:0/286093632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f680b5560 0x7f7f681562a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:49.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.022+0000 7f7f6c8bd640 1 --2- 172.21.15.32:0/286093632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f680b4320 0x7f7f6814ee60 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:49.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.022+0000 7f7f6c8bd640 1 -- 172.21.15.32:0/286093632 >> 172.21.15.32:0/286093632 conn(0x7f7f6806b110 msgr2=0x7f7f68150ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:49.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.022+0000 7f7f6c8bd640 1 -- 172.21.15.32:0/286093632 shutdown_connections 2024-04-24T23:02:49.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.022+0000 7f7f6c8bd640 1 -- 172.21.15.32:0/286093632 wait complete. 2024-04-24T23:02:49.034 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2024-04-24T23:02:49.147 INFO:teuthology.orchestra.run.smithi032.stdout:{"osd_objectstore":"memstore"} 2024-04-24T23:02:49.147 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2024-04-24T23:02:49.147 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-24T23:02:49.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.250+0000 7f966ef90640 1 Processor -- start 2024-04-24T23:02:49.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.250+0000 7f966ef90640 1 -- start start 2024-04-24T23:02:49.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.251+0000 7f966ef90640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f966806c5f0 0x7f966806c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:49.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.251+0000 7f966ef90640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f96680b44d0 0x7f96680b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:49.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.251+0000 7f966ef90640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96680b6430 con 0x7f96680af890 2024-04-24T23:02:49.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.251+0000 7f966ef90640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f9668156f80 con 0x7f96680b44d0 2024-04-24T23:02:49.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.251+0000 7f966ef90640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f96680b6150 con 0x7f966806c5f0 2024-04-24T23:02:49.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.251+0000 7f966d52d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f96680af890 0x7f96680cef70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46234/0 (socket says 172.21.15.32:46234) 2024-04-24T23:02:49.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.251+0000 7f966d52d640 1 -- 172.21.15.32:0/4272750356 learned_addr learned my addr 172.21.15.32:0/4272750356 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:02:49.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.251+0000 7f9667fff640 1 --2- 172.21.15.32:0/4272750356 >> v2:172.21.15.121:3300/0 conn(0x7f96680b44d0 0x7f96680b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:49.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.251+0000 7f966cd2c640 1 --2- 172.21.15.32:0/4272750356 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f966806c5f0 0x7f966806c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:49.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.251+0000 7f96677fe640 1 -- 172.21.15.32:0/4272750356 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3867745853 0 0) 0x7f96680b6430 con 0x7f96680af890 2024-04-24T23:02:49.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.251+0000 7f96677fe640 1 -- 172.21.15.32:0/4272750356 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9658003520 con 0x7f96680af890 2024-04-24T23:02:49.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.251+0000 7f96677fe640 1 -- 172.21.15.32:0/4272750356 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3845346032 0 0) 0x7f9658003520 con 0x7f96680af890 2024-04-24T23:02:49.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.251+0000 7f96677fe640 1 -- 172.21.15.32:0/4272750356 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f966806c5f0 msgr2=0x7f966806c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:49.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.251+0000 7f96677fe640 1 --2- 172.21.15.32:0/4272750356 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f966806c5f0 0x7f966806c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:49.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.251+0000 7f96677fe640 1 -- 172.21.15.32:0/4272750356 >> v2:172.21.15.121:3300/0 conn(0x7f96680b44d0 msgr2=0x7f96680b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:49.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.251+0000 7f96677fe640 1 --2- 172.21.15.32:0/4272750356 >> v2:172.21.15.121:3300/0 conn(0x7f96680b44d0 0x7f96680b5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:49.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.251+0000 7f96677fe640 1 -- 172.21.15.32:0/4272750356 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96680b39f0 con 0x7f96680af890 2024-04-24T23:02:49.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.251+0000 7f96677fe640 1 -- 172.21.15.32:0/4272750356 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f96600019f0 con 0x7f96680af890 2024-04-24T23:02:49.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.252+0000 7f9667fff640 1 --2- 172.21.15.32:0/4272750356 >> v2:172.21.15.121:3300/0 conn(0x7f96680b44d0 0x7f96680b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:02:49.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.252+0000 7f96677fe640 1 -- 172.21.15.32:0/4272750356 >> v1:172.21.15.32:6789/0 conn(0x7f96680af890 legacy=0x7f96680cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:49.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.252+0000 7f96677fe640 1 --2- 172.21.15.32:0/4272750356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96580039b0 0x7f9658003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:49.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.252+0000 7f96677fe640 1 --2- 172.21.15.32:0/4272750356 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f966806c5f0 0x7f9658004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:49.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.252+0000 7f96677fe640 1 --2- 172.21.15.32:0/4272750356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9658004d40 0x7f96580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:49.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.252+0000 7f96677fe640 1 -- 172.21.15.32:0/4272750356 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9668156f80 con 0x7f9658004d40 2024-04-24T23:02:49.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.252+0000 7f96677fe640 1 -- 172.21.15.32:0/4272750356 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f96680b6150 con 0x7f96580039b0 2024-04-24T23:02:49.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.252+0000 7f96677fe640 1 -- 172.21.15.32:0/4272750356 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f965800b9c0 con 0x7f966806c5f0 2024-04-24T23:02:49.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.252+0000 7f966d52d640 1 --2- 172.21.15.32:0/4272750356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9658004d40 0x7f96580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:49.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.252+0000 7f966cd2c640 1 --2- 172.21.15.32:0/4272750356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96580039b0 0x7f9658003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:49.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.252+0000 7f9667fff640 1 --2- 172.21.15.32:0/4272750356 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f966806c5f0 0x7f9658004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:49.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.252+0000 7f966d52d640 1 -- 172.21.15.32:0/4272750356 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f966806c5f0 msgr2=0x7f9658004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:49.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.252+0000 7f966d52d640 1 --2- 172.21.15.32:0/4272750356 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f966806c5f0 0x7f9658004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:49.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.252+0000 7f966d52d640 1 -- 172.21.15.32:0/4272750356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96580039b0 msgr2=0x7f9658003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:49.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.252+0000 7f966d52d640 1 --2- 172.21.15.32:0/4272750356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96580039b0 0x7f9658003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:49.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.252+0000 7f966d52d640 1 -- 172.21.15.32:0/4272750356 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9658003520 con 0x7f9658004d40 2024-04-24T23:02:49.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.253+0000 7f966d52d640 1 --2- 172.21.15.32:0/4272750356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9658004d40 0x7f96580051c0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f966000a870 tx=0x7f9658005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67fccc196b85bec8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:49.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.253+0000 7f96677fe640 1 -- 172.21.15.32:0/4272750356 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f966000a4c0 con 0x7f9658004d40 2024-04-24T23:02:49.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.253+0000 7f96677fe640 1 -- 172.21.15.32:0/4272750356 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f966002f070 con 0x7f9658004d40 2024-04-24T23:02:49.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.253+0000 7f96677fe640 1 -- 172.21.15.32:0/4272750356 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f966002bcc0 con 0x7f9658004d40 2024-04-24T23:02:49.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.253+0000 7f966ef90640 1 -- 172.21.15.32:0/4272750356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9658004d40 msgr2=0x7f96580051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:49.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.253+0000 7f966ef90640 1 --2- 172.21.15.32:0/4272750356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9658004d40 0x7f96580051c0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f966000a870 tx=0x7f9658005830 comp rx=0 tx=0).stop 2024-04-24T23:02:49.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.253+0000 7f966d52d640 1 -- 172.21.15.32:0/4272750356 reap_dead start 2024-04-24T23:02:49.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.253+0000 7f966ef90640 1 -- 172.21.15.32:0/4272750356 shutdown_connections 2024-04-24T23:02:49.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.253+0000 7f966ef90640 1 -- 172.21.15.32:0/4272750356 >> 172.21.15.32:0/4272750356 conn(0x7f966806b110 msgr2=0x7f966806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:49.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.253+0000 7f966ef90640 1 -- 172.21.15.32:0/4272750356 shutdown_connections 2024-04-24T23:02:49.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.253+0000 7f966ef90640 1 -- 172.21.15.32:0/4272750356 wait complete. 2024-04-24T23:02:49.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.255+0000 7f966ef90640 1 Processor -- start 2024-04-24T23:02:49.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.255+0000 7f966ef90640 1 -- start start 2024-04-24T23:02:49.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.256+0000 7f966ef90640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9658004d40 0x7f96680ce9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:49.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.256+0000 7f966ef90640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9668159070 0x7f9668157390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:49.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.256+0000 7f966ef90640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96681578d0 0x7f96680b4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:49.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.256+0000 7f966ef90640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f96680b5dd0 con 0x7f96681578d0 2024-04-24T23:02:49.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.256+0000 7f966ef90640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f96680b3fb0 con 0x7f9658004d40 2024-04-24T23:02:49.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.256+0000 7f966ef90640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f966806c9d0 con 0x7f9668159070 2024-04-24T23:02:49.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.256+0000 7f966d52d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96681578d0 0x7f96680b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:49.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.256+0000 7f966d52d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96681578d0 0x7f96680b4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42182/0 (socket says 172.21.15.32:42182) 2024-04-24T23:02:49.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.256+0000 7f966d52d640 1 -- 172.21.15.32:0/2871312475 learned_addr learned my addr 172.21.15.32:0/2871312475 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:49.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.256+0000 7f966cd2c640 1 --2- 172.21.15.32:0/2871312475 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9658004d40 0x7f96680ce9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:49.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.256+0000 7f9667fff640 1 --2- 172.21.15.32:0/2871312475 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9668159070 0x7f9668157390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:49.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.256+0000 7f966d52d640 1 -- 172.21.15.32:0/2871312475 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9668159070 msgr2=0x7f9668157390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:49.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.256+0000 7f966d52d640 1 --2- 172.21.15.32:0/2871312475 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9668159070 0x7f9668157390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:49.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.256+0000 7f966d52d640 1 -- 172.21.15.32:0/2871312475 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9658004d40 msgr2=0x7f96680ce9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:49.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.256+0000 7f966d52d640 1 --2- 172.21.15.32:0/2871312475 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9658004d40 0x7f96680ce9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:49.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.256+0000 7f966d52d640 1 -- 172.21.15.32:0/2871312475 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9668159550 con 0x7f96681578d0 2024-04-24T23:02:49.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.256+0000 7f9667fff640 1 --2- 172.21.15.32:0/2871312475 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9668159070 0x7f9668157390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:02:49.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.256+0000 7f966d52d640 1 --2- 172.21.15.32:0/2871312475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96681578d0 0x7f96680b4930 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f9660001cf0 tx=0x7f9660001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:49.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.256+0000 7f96657fa640 1 -- 172.21.15.32:0/2871312475 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f966000a4c0 con 0x7f96681578d0 2024-04-24T23:02:49.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.256+0000 7f96657fa640 1 -- 172.21.15.32:0/2871312475 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96600019e0 con 0x7f96681578d0 2024-04-24T23:02:49.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.256+0000 7f966ef90640 1 -- 172.21.15.32:0/2871312475 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9668159830 con 0x7f96681578d0 2024-04-24T23:02:49.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.256+0000 7f96657fa640 1 -- 172.21.15.32:0/2871312475 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9660002030 con 0x7f96681578d0 2024-04-24T23:02:49.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.256+0000 7f966ef90640 1 -- 172.21.15.32:0/2871312475 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9668159df0 con 0x7f96681578d0 2024-04-24T23:02:49.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.258+0000 7f966ef90640 1 -- 172.21.15.32:0/2871312475 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9668068430 con 0x7f96681578d0 2024-04-24T23:02:49.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.258+0000 7f96657fa640 1 -- 172.21.15.32:0/2871312475 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9660034670 con 0x7f96681578d0 2024-04-24T23:02:49.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.258+0000 7f96657fa640 1 --2- 172.21.15.32:0/2871312475 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f964c06be60 0x7f964c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:49.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.258+0000 7f96657fa640 1 -- 172.21.15.32:0/2871312475 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 4943+0+0 (secure 0 0 0) 0x7f96600aaca0 con 0x7f96681578d0 2024-04-24T23:02:49.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.259+0000 7f966cd2c640 1 --2- 172.21.15.32:0/2871312475 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f964c06be60 0x7f964c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:49.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.259+0000 7f966cd2c640 1 --2- 172.21.15.32:0/2871312475 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f964c06be60 0x7f964c06e320 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f965c009b40 tx=0x7f965c009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:49.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.262+0000 7f96657fa640 1 -- 172.21.15.32:0/2871312475 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9660046060 con 0x7f96681578d0 2024-04-24T23:02:49.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.444+0000 7f966ef90640 1 -- 172.21.15.32:0/2871312475 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f96680b5670 con 0x7f96681578d0 2024-04-24T23:02:49.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.447+0000 7f96657fa640 1 -- 172.21.15.32:0/2871312475 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v8)=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7f9660079950 con 0x7f96681578d0 2024-04-24T23:02:49.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.449+0000 7f966ef90640 1 -- 172.21.15.32:0/2871312475 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f964c06be60 msgr2=0x7f964c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:49.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.449+0000 7f966ef90640 1 --2- 172.21.15.32:0/2871312475 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f964c06be60 0x7f964c06e320 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f965c009b40 tx=0x7f965c009190 comp rx=0 tx=0).stop 2024-04-24T23:02:49.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.449+0000 7f966ef90640 1 -- 172.21.15.32:0/2871312475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96681578d0 msgr2=0x7f96680b4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:49.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.449+0000 7f966ef90640 1 --2- 172.21.15.32:0/2871312475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96681578d0 0x7f96680b4930 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f9660001cf0 tx=0x7f9660001360 comp rx=0 tx=0).stop 2024-04-24T23:02:49.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.450+0000 7f966ef90640 1 -- 172.21.15.32:0/2871312475 shutdown_connections 2024-04-24T23:02:49.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.450+0000 7f966ef90640 1 --2- 172.21.15.32:0/2871312475 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f964c06be60 0x7f964c06e320 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:49.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.450+0000 7f966ef90640 1 --2- 172.21.15.32:0/2871312475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96681578d0 0x7f96680b4930 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:49.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.450+0000 7f966ef90640 1 --2- 172.21.15.32:0/2871312475 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9668159070 0x7f9668157390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:49.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.450+0000 7f966ef90640 1 --2- 172.21.15.32:0/2871312475 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9658004d40 0x7f96680ce9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:49.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.450+0000 7f966ef90640 1 -- 172.21.15.32:0/2871312475 >> 172.21.15.32:0/2871312475 conn(0x7f966806b110 msgr2=0x7f96680b0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:49.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.450+0000 7f966ef90640 1 -- 172.21.15.32:0/2871312475 shutdown_connections 2024-04-24T23:02:49.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.450+0000 7f966ef90640 1 -- 172.21.15.32:0/2871312475 wait complete. 2024-04-24T23:02:49.462 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-24T23:02:49.462 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:02:49.504 DEBUG:teuthology.orchestra.run.smithi121:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:02:49.546 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-24T23:02:49.546 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:02:49.546 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:02:49.562 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=413 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T23:02:49.562 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:02:49.562 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:02:49.604 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3271 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T23:02:49.604 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:02:49.605 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:02:49.620 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=413 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T23:02:49.620 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:02:49.620 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:02:49.661 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3271 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T23:02:49.661 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-24T23:02:49.661 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:02:49.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.761+0000 7f7361b29640 1 Processor -- start 2024-04-24T23:02:49.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.761+0000 7f7361b29640 1 -- start start 2024-04-24T23:02:49.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.762+0000 7f7361b29640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f735c0b5560 0x7f735c15e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:49.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.762+0000 7f7361b29640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f735c0cd3c0 0x7f735c15ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:49.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.762+0000 7f7361b29640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f735c0cd910 con 0x7f735c0b5940 2024-04-24T23:02:49.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.762+0000 7f7361b29640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f735c0cdbf0 con 0x7f735c0cd3c0 2024-04-24T23:02:49.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.762+0000 7f7361b29640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f735c0cded0 con 0x7f735c0b5560 2024-04-24T23:02:49.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.762+0000 7f735bfff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f735c0b5940 0x7f735c0b5d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50488/0 (socket says 172.21.15.32:50488) 2024-04-24T23:02:49.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.762+0000 7f735bfff640 1 -- 172.21.15.32:0/2012587083 learned_addr learned my addr 172.21.15.32:0/2012587083 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:02:49.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.762+0000 7f735b7fe640 1 --2- 172.21.15.32:0/2012587083 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f735c0b5560 0x7f735c15e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:49.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.762+0000 7f735affd640 1 --2- 172.21.15.32:0/2012587083 >> v2:172.21.15.121:3300/0 conn(0x7f735c0cd3c0 0x7f735c15ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:49.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.762+0000 7f735a7fc640 1 -- 172.21.15.32:0/2012587083 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3326226444 0 0) 0x7f735c0cd910 con 0x7f735c0b5940 2024-04-24T23:02:49.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.762+0000 7f735a7fc640 1 -- 172.21.15.32:0/2012587083 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7344003520 con 0x7f735c0b5940 2024-04-24T23:02:49.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.763+0000 7f735a7fc640 1 -- 172.21.15.32:0/2012587083 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2426023709 0 0) 0x7f7344003520 con 0x7f735c0b5940 2024-04-24T23:02:49.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.763+0000 7f735a7fc640 1 -- 172.21.15.32:0/2012587083 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f735c0b5560 msgr2=0x7f735c15e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:49.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.763+0000 7f735a7fc640 1 --2- 172.21.15.32:0/2012587083 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f735c0b5560 0x7f735c15e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:49.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.763+0000 7f735a7fc640 1 -- 172.21.15.32:0/2012587083 >> v2:172.21.15.121:3300/0 conn(0x7f735c0cd3c0 msgr2=0x7f735c15ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:49.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.763+0000 7f735a7fc640 1 --2- 172.21.15.32:0/2012587083 >> v2:172.21.15.121:3300/0 conn(0x7f735c0cd3c0 0x7f735c15ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:49.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.763+0000 7f735a7fc640 1 -- 172.21.15.32:0/2012587083 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f735c0ce1b0 con 0x7f735c0b5940 2024-04-24T23:02:49.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.763+0000 7f735a7fc640 1 -- 172.21.15.32:0/2012587083 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f73540019f0 con 0x7f735c0b5940 2024-04-24T23:02:49.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.763+0000 7f735a7fc640 1 -- 172.21.15.32:0/2012587083 >> v1:172.21.15.32:6789/0 conn(0x7f735c0b5940 legacy=0x7f735c0b5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:49.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.763+0000 7f735a7fc640 1 --2- 172.21.15.32:0/2012587083 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f735c0b5560 0x7f73440039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:49.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.763+0000 7f735a7fc640 1 --2- 172.21.15.32:0/2012587083 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7344003f30 0x7f7344004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:49.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.763+0000 7f735a7fc640 1 --2- 172.21.15.32:0/2012587083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7344004d40 0x7f73440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:49.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.763+0000 7f735a7fc640 1 -- 172.21.15.32:0/2012587083 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f735c0cdbf0 con 0x7f7344004d40 2024-04-24T23:02:49.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.763+0000 7f735a7fc640 1 -- 172.21.15.32:0/2012587083 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f735c0cded0 con 0x7f7344003f30 2024-04-24T23:02:49.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.763+0000 7f735a7fc640 1 -- 172.21.15.32:0/2012587083 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f734400b9c0 con 0x7f735c0b5560 2024-04-24T23:02:49.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.763+0000 7f735bfff640 1 --2- 172.21.15.32:0/2012587083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7344004d40 0x7f73440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:49.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.763+0000 7f735b7fe640 1 --2- 172.21.15.32:0/2012587083 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f735c0b5560 0x7f73440039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:49.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.763+0000 7f735affd640 1 --2- 172.21.15.32:0/2012587083 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7344003f30 0x7f7344004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:49.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.763+0000 7f735bfff640 1 -- 172.21.15.32:0/2012587083 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f735c0b5560 msgr2=0x7f73440039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:49.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.763+0000 7f735bfff640 1 --2- 172.21.15.32:0/2012587083 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f735c0b5560 0x7f73440039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:49.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.763+0000 7f735bfff640 1 -- 172.21.15.32:0/2012587083 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7344003f30 msgr2=0x7f7344004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:49.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.763+0000 7f735bfff640 1 --2- 172.21.15.32:0/2012587083 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7344003f30 0x7f7344004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:49.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.763+0000 7f735bfff640 1 -- 172.21.15.32:0/2012587083 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7344003520 con 0x7f7344004d40 2024-04-24T23:02:49.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.764+0000 7f735bfff640 1 --2- 172.21.15.32:0/2012587083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7344004d40 0x7f73440051c0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f735400a8a0 tx=0x7f7344005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ee231c9ccdd75c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:49.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.764+0000 7f735a7fc640 1 -- 172.21.15.32:0/2012587083 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f735400a4c0 con 0x7f7344004d40 2024-04-24T23:02:49.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.764+0000 7f735a7fc640 1 -- 172.21.15.32:0/2012587083 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f735402f070 con 0x7f7344004d40 2024-04-24T23:02:49.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.764+0000 7f735a7fc640 1 -- 172.21.15.32:0/2012587083 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f735402bd50 con 0x7f7344004d40 2024-04-24T23:02:49.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.765+0000 7f7361b29640 1 -- 172.21.15.32:0/2012587083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7344004d40 msgr2=0x7f73440051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:49.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.765+0000 7f7361b29640 1 --2- 172.21.15.32:0/2012587083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7344004d40 0x7f73440051c0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f735400a8a0 tx=0x7f7344005830 comp rx=0 tx=0).stop 2024-04-24T23:02:49.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.765+0000 7f735bfff640 1 -- 172.21.15.32:0/2012587083 reap_dead start 2024-04-24T23:02:49.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.765+0000 7f7361b29640 1 -- 172.21.15.32:0/2012587083 shutdown_connections 2024-04-24T23:02:49.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.765+0000 7f7361b29640 1 -- 172.21.15.32:0/2012587083 >> 172.21.15.32:0/2012587083 conn(0x7f735c06b110 msgr2=0x7f735c06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:49.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.765+0000 7f7361b29640 1 -- 172.21.15.32:0/2012587083 shutdown_connections 2024-04-24T23:02:49.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.765+0000 7f7361b29640 1 -- 172.21.15.32:0/2012587083 wait complete. 2024-04-24T23:02:49.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.766+0000 7f7361b29640 1 Processor -- start 2024-04-24T23:02:49.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.766+0000 7f7361b29640 1 -- start start 2024-04-24T23:02:49.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.767+0000 7f7361b29640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f735c14bed0 0x7f735c150160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:49.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.767+0000 7f7361b29640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f735c158070 0x7f735c0b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:49.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.767+0000 7f7361b29640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f735c0b4610 0x7f735c15d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:49.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.767+0000 7f7361b29640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f735c15e840 con 0x7f735c0b4610 2024-04-24T23:02:49.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.767+0000 7f7361b29640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f735c15e6c0 con 0x7f735c14bed0 2024-04-24T23:02:49.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.767+0000 7f7361b29640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f735c15e9c0 con 0x7f735c158070 2024-04-24T23:02:49.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.767+0000 7f735bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f735c0b4610 0x7f735c15d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:49.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.767+0000 7f735bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f735c0b4610 0x7f735c15d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49376/0 (socket says 172.21.15.32:49376) 2024-04-24T23:02:49.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.767+0000 7f735bfff640 1 -- 172.21.15.32:0/4212712770 learned_addr learned my addr 172.21.15.32:0/4212712770 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:49.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.767+0000 7f735affd640 1 --2- 172.21.15.32:0/4212712770 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f735c158070 0x7f735c0b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:49.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.767+0000 7f735b7fe640 1 --2- 172.21.15.32:0/4212712770 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f735c14bed0 0x7f735c150160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:49.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.767+0000 7f735bfff640 1 -- 172.21.15.32:0/4212712770 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f735c158070 msgr2=0x7f735c0b40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:49.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.767+0000 7f735bfff640 1 --2- 172.21.15.32:0/4212712770 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f735c158070 0x7f735c0b40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:49.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.767+0000 7f735bfff640 1 -- 172.21.15.32:0/4212712770 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f735c14bed0 msgr2=0x7f735c150160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:49.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.767+0000 7f735bfff640 1 --2- 172.21.15.32:0/4212712770 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f735c14bed0 0x7f735c150160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:49.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.767+0000 7f735bfff640 1 -- 172.21.15.32:0/4212712770 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f735c151c40 con 0x7f735c0b4610 2024-04-24T23:02:49.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.767+0000 7f735bfff640 1 --2- 172.21.15.32:0/4212712770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f735c0b4610 0x7f735c15d260 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f735400a8a0 tx=0x7f7354001e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:49.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.768+0000 7f7343fff640 1 -- 172.21.15.32:0/4212712770 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7354045070 con 0x7f735c0b4610 2024-04-24T23:02:49.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.768+0000 7f7343fff640 1 -- 172.21.15.32:0/4212712770 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7354040480 con 0x7f735c0b4610 2024-04-24T23:02:49.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.768+0000 7f7361b29640 1 -- 172.21.15.32:0/4212712770 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f735c1508c0 con 0x7f735c0b4610 2024-04-24T23:02:49.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.768+0000 7f7343fff640 1 -- 172.21.15.32:0/4212712770 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f735400a4c0 con 0x7f735c0b4610 2024-04-24T23:02:49.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.768+0000 7f7361b29640 1 -- 172.21.15.32:0/4212712770 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f735c150e80 con 0x7f735c0b4610 2024-04-24T23:02:49.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.769+0000 7f7361b29640 1 -- 172.21.15.32:0/4212712770 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f735c068430 con 0x7f735c0b4610 2024-04-24T23:02:49.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.769+0000 7f7343fff640 1 -- 172.21.15.32:0/4212712770 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7354048070 con 0x7f735c0b4610 2024-04-24T23:02:49.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.769+0000 7f7343fff640 1 --2- 172.21.15.32:0/4212712770 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f734806be60 0x7f734806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:49.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.770+0000 7f7343fff640 1 -- 172.21.15.32:0/4212712770 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 4943+0+0 (secure 0 0 0) 0x7f73540aaaf0 con 0x7f735c0b4610 2024-04-24T23:02:49.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.770+0000 7f735b7fe640 1 --2- 172.21.15.32:0/4212712770 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f734806be60 0x7f734806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:49.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.770+0000 7f735b7fe640 1 --2- 172.21.15.32:0/4212712770 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f734806be60 0x7f734806e320 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f734c009bc0 tx=0x7f734c009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:49.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.774+0000 7f7343fff640 1 -- 172.21.15.32:0/4212712770 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f73540793c0 con 0x7f735c0b4610 2024-04-24T23:02:49.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.993+0000 7f7361b29640 1 -- 172.21.15.32:0/4212712770 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f735c068430 con 0x7f735c0b4610 2024-04-24T23:02:49.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.994+0000 7f7343fff640 1 -- 172.21.15.32:0/4212712770 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+3875 (secure 0 0 0) 0x7f735407d6c0 con 0x7f735c0b4610 2024-04-24T23:02:49.995 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:02:49.996 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":29,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3085809520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3085809520},{"type":"v1","addr":"172.21.15.121:6837","nonce":3085809520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":5444,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/2440197398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2440197398},{"type":"v1","addr":"172.21.15.121:6836","nonce":2440197398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":6078,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/1405465715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1405465715},{"type":"v1","addr":"172.21.15.32:6834","nonce":1405465715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":6081,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1160416671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":1160416671},{"type":"v1","addr":"172.21.15.32:6835","nonce":1160416671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[]} 2024-04-24T23:02:49.996 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 29 2024-04-24T23:02:49.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.995+0000 7f7361b29640 1 -- 172.21.15.32:0/4212712770 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f734806be60 msgr2=0x7f734806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:49.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.995+0000 7f7361b29640 1 --2- 172.21.15.32:0/4212712770 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f734806be60 0x7f734806e320 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f734c009bc0 tx=0x7f734c009190 comp rx=0 tx=0).stop 2024-04-24T23:02:49.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.995+0000 7f7361b29640 1 -- 172.21.15.32:0/4212712770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f735c0b4610 msgr2=0x7f735c15d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:49.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.995+0000 7f7361b29640 1 --2- 172.21.15.32:0/4212712770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f735c0b4610 0x7f735c15d260 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f735400a8a0 tx=0x7f7354001e30 comp rx=0 tx=0).stop 2024-04-24T23:02:49.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.996+0000 7f7361b29640 1 -- 172.21.15.32:0/4212712770 shutdown_connections 2024-04-24T23:02:49.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.996+0000 7f7361b29640 1 --2- 172.21.15.32:0/4212712770 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f734806be60 0x7f734806e320 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:49.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.996+0000 7f7361b29640 1 --2- 172.21.15.32:0/4212712770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f735c0b4610 0x7f735c15d260 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:49.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.996+0000 7f7361b29640 1 --2- 172.21.15.32:0/4212712770 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f735c158070 0x7f735c0b40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:49.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.996+0000 7f7361b29640 1 --2- 172.21.15.32:0/4212712770 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f735c14bed0 0x7f735c150160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:49.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.996+0000 7f7361b29640 1 -- 172.21.15.32:0/4212712770 >> 172.21.15.32:0/4212712770 conn(0x7f735c06b110 msgr2=0x7f735c1526a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:49.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.996+0000 7f7361b29640 1 -- 172.21.15.32:0/4212712770 shutdown_connections 2024-04-24T23:02:49.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:49.996+0000 7f7361b29640 1 -- 172.21.15.32:0/4212712770 wait complete. 2024-04-24T23:02:50.008 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-24T23:02:50.008 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-24T23:02:50.008 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-24T23:02:50.008 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:02:50.009 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-24T23:02:50.009 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-24T23:02:50.009 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-24T23:02:50.009 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:02:50.009 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-24T23:02:50.009 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-24T23:02:50.010 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-24T23:02:50.010 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:02:50.010 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-24T23:02:50.010 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-24T23:02:50.010 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-24T23:02:50.010 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:02:50.018 INFO:tasks.ceph.mds.a:Stopped 2024-04-24T23:02:50.018 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-24T23:02:50.019 INFO:tasks.ceph.mds.c:Stopped 2024-04-24T23:02:50.019 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-24T23:02:50.022 INFO:tasks.ceph.mds.b:Stopped 2024-04-24T23:02:50.022 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-24T23:02:50.061 INFO:tasks.ceph.mds.a:Started 2024-04-24T23:02:50.061 DEBUG:teuthology.parallel:result is None 2024-04-24T23:02:50.062 INFO:tasks.ceph.mds.c:Started 2024-04-24T23:02:50.062 DEBUG:teuthology.parallel:result is None 2024-04-24T23:02:50.064 INFO:tasks.ceph.mds.b:Started 2024-04-24T23:02:50.065 DEBUG:teuthology.parallel:result is None 2024-04-24T23:02:50.134 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2024-04-24T23:02:50.135 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2024-04-24T23:02:50.140 INFO:tasks.ceph.mds.b.smithi121.stdout:starting mds.b at 2024-04-24T23:02:55.548 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-04-24T23:02:56.022 INFO:tasks.ceph.mds.d:Stopped 2024-04-24T23:02:56.022 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-24T23:02:56.025 INFO:tasks.ceph.mds.d:Started 2024-04-24T23:02:56.025 DEBUG:teuthology.parallel:result is None 2024-04-24T23:02:56.025 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-24T23:02:56.107 INFO:tasks.ceph.mds.d.smithi121.stdout:starting mds.d at 2024-04-24T23:02:56.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.139+0000 7fd42ea12640 1 Processor -- start 2024-04-24T23:02:56.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.139+0000 7fd42ea12640 1 -- start start 2024-04-24T23:02:56.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.139+0000 7fd42ea12640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd42815e9b0 0x7fd4280d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:56.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.139+0000 7fd42ea12640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd4280d4210 0x7fd42806b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:56.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.139+0000 7fd42ea12640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4280d3170 con 0x7fd4280ba0e0 2024-04-24T23:02:56.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.139+0000 7fd42ea12640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fd4280b8900 con 0x7fd4280d4210 2024-04-24T23:02:56.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.139+0000 7fd42ea12640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fd4280d3450 con 0x7fd42815e9b0 2024-04-24T23:02:56.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.140+0000 7fd4277fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd4280d4210 0x7fd42806b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:56.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.140+0000 7fd4277fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd4280d4210 0x7fd42806b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49406/0 (socket says 172.21.15.32:49406) 2024-04-24T23:02:56.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.140+0000 7fd4277fe640 1 -- 172.21.15.32:0/4010524997 learned_addr learned my addr 172.21.15.32:0/4010524997 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:56.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.140+0000 7fd427fff640 1 --2- 172.21.15.32:0/4010524997 >> v2:172.21.15.121:3300/0 conn(0x7fd42815e9b0 0x7fd4280d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:56.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.140+0000 7fd4277fe640 1 -- 172.21.15.32:0/4010524997 >> v2:172.21.15.121:3300/0 conn(0x7fd42815e9b0 msgr2=0x7fd4280d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:56.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.140+0000 7fd4277fe640 1 --2- 172.21.15.32:0/4010524997 >> v2:172.21.15.121:3300/0 conn(0x7fd42815e9b0 0x7fd4280d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:56.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.140+0000 7fd4277fe640 1 -- 172.21.15.32:0/4010524997 >> v1:172.21.15.121:6789/0 conn(0x7fd4280ba0e0 legacy=0x7fd428071730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:56.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.140+0000 7fd4277fe640 1 -- 172.21.15.32:0/4010524997 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4280d3730 con 0x7fd4280d4210 2024-04-24T23:02:56.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.141+0000 7fd4277fe640 1 --2- 172.21.15.32:0/4010524997 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4280d4210 0x7fd42806b110 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fd42806be20 tx=0x7fd41c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9347be09c8fe6f82 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:56.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.141+0000 7fd426ffd640 1 -- 172.21.15.32:0/4010524997 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41c037070 con 0x7fd4280d4210 2024-04-24T23:02:56.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.141+0000 7fd426ffd640 1 -- 172.21.15.32:0/4010524997 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd41c032440 con 0x7fd4280d4210 2024-04-24T23:02:56.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.141+0000 7fd426ffd640 1 -- 172.21.15.32:0/4010524997 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41c032810 con 0x7fd4280d4210 2024-04-24T23:02:56.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.141+0000 7fd42ea12640 1 -- 172.21.15.32:0/4010524997 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4280d4210 msgr2=0x7fd42806b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:56.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.141+0000 7fd42ea12640 1 --2- 172.21.15.32:0/4010524997 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4280d4210 0x7fd42806b110 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fd42806be20 tx=0x7fd41c02a1a0 comp rx=0 tx=0).stop 2024-04-24T23:02:56.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.141+0000 7fd42ea12640 1 -- 172.21.15.32:0/4010524997 shutdown_connections 2024-04-24T23:02:56.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.141+0000 7fd42ea12640 1 --2- 172.21.15.32:0/4010524997 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4280d4210 0x7fd42806b110 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:56.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.142+0000 7fd42ea12640 1 --2- 172.21.15.32:0/4010524997 >> v2:172.21.15.121:3300/0 conn(0x7fd42815e9b0 0x7fd4280d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:56.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.142+0000 7fd42ea12640 1 -- 172.21.15.32:0/4010524997 >> 172.21.15.32:0/4010524997 conn(0x7fd4280d11d0 msgr2=0x7fd4280d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:56.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.142+0000 7fd42ea12640 1 -- 172.21.15.32:0/4010524997 shutdown_connections 2024-04-24T23:02:56.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.142+0000 7fd42ea12640 1 -- 172.21.15.32:0/4010524997 wait complete. 2024-04-24T23:02:56.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.143+0000 7fd42ea12640 1 Processor -- start 2024-04-24T23:02:56.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.143+0000 7fd42ea12640 1 -- start start 2024-04-24T23:02:56.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.144+0000 7fd42ea12640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4280ba0e0 0x7fd4280719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:56.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.144+0000 7fd42ea12640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd4280d4210 0x7fd42814c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:56.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.144+0000 7fd42ea12640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd42815e9b0 0x7fd42814cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:56.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.144+0000 7fd42ea12640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd4280d2400 con 0x7fd42815e9b0 2024-04-24T23:02:56.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.144+0000 7fd42ea12640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd4280d3cf0 con 0x7fd4280ba0e0 2024-04-24T23:02:56.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.144+0000 7fd42ea12640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd42806b650 con 0x7fd4280d4210 2024-04-24T23:02:56.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.144+0000 7fd42cfaf640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd42815e9b0 0x7fd42814cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:56.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.144+0000 7fd42cfaf640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd42815e9b0 0x7fd42814cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49408/0 (socket says 172.21.15.32:49408) 2024-04-24T23:02:56.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.144+0000 7fd42cfaf640 1 -- 172.21.15.32:0/4237772186 learned_addr learned my addr 172.21.15.32:0/4237772186 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:56.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.144+0000 7fd427fff640 1 --2- 172.21.15.32:0/4237772186 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4280ba0e0 0x7fd4280719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:56.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.144+0000 7fd4277fe640 1 --2- 172.21.15.32:0/4237772186 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd4280d4210 0x7fd42814c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:56.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.144+0000 7fd42cfaf640 1 -- 172.21.15.32:0/4237772186 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd4280d4210 msgr2=0x7fd42814c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:56.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.144+0000 7fd42cfaf640 1 --2- 172.21.15.32:0/4237772186 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd4280d4210 0x7fd42814c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:56.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.144+0000 7fd42cfaf640 1 -- 172.21.15.32:0/4237772186 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4280ba0e0 msgr2=0x7fd4280719a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:56.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.144+0000 7fd42cfaf640 1 --2- 172.21.15.32:0/4237772186 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4280ba0e0 0x7fd4280719a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:56.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.144+0000 7fd42cfaf640 1 -- 172.21.15.32:0/4237772186 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd428161280 con 0x7fd42815e9b0 2024-04-24T23:02:56.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.144+0000 7fd42cfaf640 1 --2- 172.21.15.32:0/4237772186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd42815e9b0 0x7fd42814cc80 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fd42000c7b0 tx=0x7fd42000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:56.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.144+0000 7fd424ff9640 1 -- 172.21.15.32:0/4237772186 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd420004b60 con 0x7fd42815e9b0 2024-04-24T23:02:56.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.144+0000 7fd424ff9640 1 -- 172.21.15.32:0/4237772186 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd420004d00 con 0x7fd42815e9b0 2024-04-24T23:02:56.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.145+0000 7fd42ea12640 1 -- 172.21.15.32:0/4237772186 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd428161520 con 0x7fd42815e9b0 2024-04-24T23:02:56.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.145+0000 7fd424ff9640 1 -- 172.21.15.32:0/4237772186 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42000aa90 con 0x7fd42815e9b0 2024-04-24T23:02:56.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.145+0000 7fd42ea12640 1 -- 172.21.15.32:0/4237772186 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd428161ae0 con 0x7fd42815e9b0 2024-04-24T23:02:56.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.146+0000 7fd42ea12640 1 -- 172.21.15.32:0/4237772186 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd428068430 con 0x7fd42815e9b0 2024-04-24T23:02:56.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.146+0000 7fd424ff9640 1 -- 172.21.15.32:0/4237772186 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd42000ac30 con 0x7fd42815e9b0 2024-04-24T23:02:56.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.147+0000 7fd424ff9640 1 --2- 172.21.15.32:0/4237772186 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd41406be30 0x7fd41406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:56.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.147+0000 7fd424ff9640 1 -- 172.21.15.32:0/4237772186 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4943+0+0 (secure 0 0 0) 0x7fd42008ac90 con 0x7fd42815e9b0 2024-04-24T23:02:56.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.147+0000 7fd427fff640 1 --2- 172.21.15.32:0/4237772186 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd41406be30 0x7fd41406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:56.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.147+0000 7fd427fff640 1 --2- 172.21.15.32:0/4237772186 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd41406be30 0x7fd41406e2f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd41800aa20 tx=0x7fd418008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:56.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.151+0000 7fd424ff9640 1 -- 172.21.15.32:0/4237772186 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd420020070 con 0x7fd42815e9b0 2024-04-24T23:02:56.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:56.347+0000 7fd42ea12640 1 -- 172.21.15.32:0/4237772186 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fd42815c8c0 con 0x7fd42815e9b0 2024-04-24T23:02:57.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.248+0000 7fd424ff9640 1 -- 172.21.15.32:0/4237772186 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v83) ==== 97+0+0 (secure 0 0 0) 0x7fd420059380 con 0x7fd42815e9b0 2024-04-24T23:02:57.250 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blocklist entries 2024-04-24T23:02:57.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.250+0000 7fd42ea12640 1 -- 172.21.15.32:0/4237772186 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd41406be30 msgr2=0x7fd41406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:57.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.250+0000 7fd42ea12640 1 --2- 172.21.15.32:0/4237772186 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd41406be30 0x7fd41406e2f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd41800aa20 tx=0x7fd418008040 comp rx=0 tx=0).stop 2024-04-24T23:02:57.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.250+0000 7fd42ea12640 1 -- 172.21.15.32:0/4237772186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd42815e9b0 msgr2=0x7fd42814cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:57.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.250+0000 7fd42ea12640 1 --2- 172.21.15.32:0/4237772186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd42815e9b0 0x7fd42814cc80 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fd42000c7b0 tx=0x7fd42000cc70 comp rx=0 tx=0).stop 2024-04-24T23:02:57.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.251+0000 7fd42ea12640 1 -- 172.21.15.32:0/4237772186 shutdown_connections 2024-04-24T23:02:57.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.251+0000 7fd42ea12640 1 --2- 172.21.15.32:0/4237772186 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd41406be30 0x7fd41406e2f0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:57.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.251+0000 7fd42ea12640 1 --2- 172.21.15.32:0/4237772186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd42815e9b0 0x7fd42814cc80 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:57.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.251+0000 7fd42ea12640 1 --2- 172.21.15.32:0/4237772186 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd4280d4210 0x7fd42814c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:57.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.251+0000 7fd42ea12640 1 --2- 172.21.15.32:0/4237772186 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4280ba0e0 0x7fd4280719a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:57.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.251+0000 7fd42ea12640 1 -- 172.21.15.32:0/4237772186 >> 172.21.15.32:0/4237772186 conn(0x7fd4280d11d0 msgr2=0x7fd42815be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:57.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.251+0000 7fd42ea12640 1 -- 172.21.15.32:0/4237772186 shutdown_connections 2024-04-24T23:02:57.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.251+0000 7fd42ea12640 1 -- 172.21.15.32:0/4237772186 wait complete. 2024-04-24T23:02:57.266 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-24T23:02:57.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.372+0000 7f01babe7640 1 Processor -- start 2024-04-24T23:02:57.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.372+0000 7f01babe7640 1 -- start start 2024-04-24T23:02:57.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.373+0000 7f01babe7640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f01b406c5f0 0x7f01b406c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:57.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.373+0000 7f01babe7640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f01b40b44d0 0x7f01b40b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:57.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.373+0000 7f01babe7640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01b40af890 0x7f01b40cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:57.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.373+0000 7f01babe7640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f01b40b6430 con 0x7f01b40b44d0 2024-04-24T23:02:57.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.373+0000 7f01babe7640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f01b4156f80 con 0x7f01b406c5f0 2024-04-24T23:02:57.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.373+0000 7f01babe7640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f01b40b6150 con 0x7f01b40af890 2024-04-24T23:02:57.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.373+0000 7f01b3fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f01b40b44d0 0x7f01b40b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:57.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.373+0000 7f01b3fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f01b40b44d0 0x7f01b40b5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49416/0 (socket says 172.21.15.32:49416) 2024-04-24T23:02:57.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.373+0000 7f01b3fff640 1 -- 172.21.15.32:0/646787453 learned_addr learned my addr 172.21.15.32:0/646787453 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:57.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.373+0000 7f01b8983640 1 --2- 172.21.15.32:0/646787453 >> v2:172.21.15.121:3300/0 conn(0x7f01b406c5f0 0x7f01b406c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:57.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.373+0000 7f01b9184640 1 --2- 172.21.15.32:0/646787453 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01b40af890 0x7f01b40cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:57.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.373+0000 7f01b3fff640 1 -- 172.21.15.32:0/646787453 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01b40af890 msgr2=0x7f01b40cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:57.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.373+0000 7f01b3fff640 1 --2- 172.21.15.32:0/646787453 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01b40af890 0x7f01b40cef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:57.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.373+0000 7f01b3fff640 1 -- 172.21.15.32:0/646787453 >> v2:172.21.15.121:3300/0 conn(0x7f01b406c5f0 msgr2=0x7f01b406c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:57.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.373+0000 7f01b3fff640 1 --2- 172.21.15.32:0/646787453 >> v2:172.21.15.121:3300/0 conn(0x7f01b406c5f0 0x7f01b406c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:57.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.373+0000 7f01b3fff640 1 -- 172.21.15.32:0/646787453 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01b40b39f0 con 0x7f01b40b44d0 2024-04-24T23:02:57.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.373+0000 7f01b8983640 1 --2- 172.21.15.32:0/646787453 >> v2:172.21.15.121:3300/0 conn(0x7f01b406c5f0 0x7f01b406c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:02:57.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.374+0000 7f01b3fff640 1 --2- 172.21.15.32:0/646787453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01b40b44d0 0x7f01b40b5890 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f01b406c1f0 tx=0x7f01a002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aa70f65312cc76d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:57.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.374+0000 7f01b37fe640 1 -- 172.21.15.32:0/646787453 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a0037070 con 0x7f01b40b44d0 2024-04-24T23:02:57.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.374+0000 7f01b37fe640 1 -- 172.21.15.32:0/646787453 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01a0032440 con 0x7f01b40b44d0 2024-04-24T23:02:57.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.374+0000 7f01b37fe640 1 -- 172.21.15.32:0/646787453 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a0032810 con 0x7f01b40b44d0 2024-04-24T23:02:57.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.375+0000 7f01babe7640 1 -- 172.21.15.32:0/646787453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01b40b44d0 msgr2=0x7f01b40b5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:57.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.375+0000 7f01babe7640 1 --2- 172.21.15.32:0/646787453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01b40b44d0 0x7f01b40b5890 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f01b406c1f0 tx=0x7f01a002a140 comp rx=0 tx=0).stop 2024-04-24T23:02:57.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.375+0000 7f01babe7640 1 -- 172.21.15.32:0/646787453 shutdown_connections 2024-04-24T23:02:57.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.375+0000 7f01babe7640 1 --2- 172.21.15.32:0/646787453 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01b40af890 0x7f01b40cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:57.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.375+0000 7f01babe7640 1 --2- 172.21.15.32:0/646787453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01b40b44d0 0x7f01b40b5890 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:57.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.375+0000 7f01babe7640 1 --2- 172.21.15.32:0/646787453 >> v2:172.21.15.121:3300/0 conn(0x7f01b406c5f0 0x7f01b406c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:57.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.375+0000 7f01babe7640 1 -- 172.21.15.32:0/646787453 >> 172.21.15.32:0/646787453 conn(0x7f01b406b110 msgr2=0x7f01b406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:57.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.375+0000 7f01babe7640 1 -- 172.21.15.32:0/646787453 shutdown_connections 2024-04-24T23:02:57.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.375+0000 7f01babe7640 1 -- 172.21.15.32:0/646787453 wait complete. 2024-04-24T23:02:57.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.376+0000 7f01babe7640 1 Processor -- start 2024-04-24T23:02:57.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.376+0000 7f01babe7640 1 -- start start 2024-04-24T23:02:57.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.377+0000 7f01babe7640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01b406c5f0 0x7f01b4141f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:57.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.377+0000 7f01babe7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01b40af890 0x7f01b41f8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:57.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.377+0000 7f01babe7640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01b40b44d0 0x7f01b40ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:57.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.377+0000 7f01babe7640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f01b406cf10 con 0x7f01b40af890 2024-04-24T23:02:57.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.377+0000 7f01babe7640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f01b40b3fb0 con 0x7f01b406c5f0 2024-04-24T23:02:57.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.377+0000 7f01babe7640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f01b40b5dd0 con 0x7f01b40b44d0 2024-04-24T23:02:57.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.377+0000 7f01b3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01b40af890 0x7f01b41f8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:57.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.377+0000 7f01b3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01b40af890 0x7f01b41f8720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49432/0 (socket says 172.21.15.32:49432) 2024-04-24T23:02:57.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.377+0000 7f01b3fff640 1 -- 172.21.15.32:0/557737346 learned_addr learned my addr 172.21.15.32:0/557737346 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:57.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.377+0000 7f01b9184640 1 --2- 172.21.15.32:0/557737346 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01b40b44d0 0x7f01b40ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:57.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.377+0000 7f01b8983640 1 --2- 172.21.15.32:0/557737346 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01b406c5f0 0x7f01b4141f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:57.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.377+0000 7f01b3fff640 1 -- 172.21.15.32:0/557737346 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01b40b44d0 msgr2=0x7f01b40ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:57.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.377+0000 7f01b3fff640 1 --2- 172.21.15.32:0/557737346 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01b40b44d0 0x7f01b40ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:57.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.377+0000 7f01b3fff640 1 -- 172.21.15.32:0/557737346 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01b406c5f0 msgr2=0x7f01b4141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:57.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.377+0000 7f01b3fff640 1 --2- 172.21.15.32:0/557737346 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01b406c5f0 0x7f01b4141f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:57.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.377+0000 7f01b3fff640 1 -- 172.21.15.32:0/557737346 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01b40cdd80 con 0x7f01b40af890 2024-04-24T23:02:57.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.377+0000 7f01b3fff640 1 --2- 172.21.15.32:0/557737346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01b40af890 0x7f01b41f8720 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f01a002a180 tx=0x7f01a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:57.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.378+0000 7f01b17fa640 1 -- 172.21.15.32:0/557737346 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a0041020 con 0x7f01b40af890 2024-04-24T23:02:57.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.378+0000 7f01b17fa640 1 -- 172.21.15.32:0/557737346 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01a00306c0 con 0x7f01b40af890 2024-04-24T23:02:57.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.378+0000 7f01babe7640 1 -- 172.21.15.32:0/557737346 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01b414d570 con 0x7f01b40af890 2024-04-24T23:02:57.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.378+0000 7f01b17fa640 1 -- 172.21.15.32:0/557737346 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a0037040 con 0x7f01b40af890 2024-04-24T23:02:57.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.378+0000 7f01babe7640 1 -- 172.21.15.32:0/557737346 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01b414db30 con 0x7f01b40af890 2024-04-24T23:02:57.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.379+0000 7f01babe7640 1 -- 172.21.15.32:0/557737346 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01b4068430 con 0x7f01b40af890 2024-04-24T23:02:57.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.379+0000 7f01b17fa640 1 -- 172.21.15.32:0/557737346 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f01a0044020 con 0x7f01b40af890 2024-04-24T23:02:57.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.379+0000 7f01b17fa640 1 --2- 172.21.15.32:0/557737346 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f019806be30 0x7f019806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:57.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.380+0000 7f01b17fa640 1 -- 172.21.15.32:0/557737346 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 4857+0+0 (secure 0 0 0) 0x7f01a00ab350 con 0x7f01b40af890 2024-04-24T23:02:57.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.380+0000 7f01b8983640 1 --2- 172.21.15.32:0/557737346 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f019806be30 0x7f019806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:57.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.380+0000 7f01b8983640 1 --2- 172.21.15.32:0/557737346 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f019806be30 0x7f019806e2f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f01a8006fd0 tx=0x7f01a8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:57.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.384+0000 7f01b17fa640 1 -- 172.21.15.32:0/557737346 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f01a004b030 con 0x7f01b40af890 2024-04-24T23:02:57.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.637+0000 7f01babe7640 1 -- 172.21.15.32:0/557737346 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f01b40cdff0 con 0x7f01b40af890 2024-04-24T23:02:57.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.638+0000 7f01b17fa640 1 -- 172.21.15.32:0/557737346 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) ==== 80+0+5304 (secure 0 0 0) 0x7f01a007a050 con 0x7f01b40af890 2024-04-24T23:02:57.641 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:02:57.641 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:02:57.641 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2024-04-24T23:02:57.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:02:57.641 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2024-04-24T23:02:57.641 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmXRdxAxAA/EYEn1C38XhG7DC+0X4y7A==", 2024-04-24T23:02:57.641 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:02:57.641 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:02:57.641 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-24T23:02:57.642 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-24T23:02:57.642 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:02:57.642 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:02:57.642 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:02:57.642 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:02:57.642 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2024-04-24T23:02:57.642 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmak7bDhAA0he+Jp/gkkrfBjYKSObjcw==", 2024-04-24T23:02:57.642 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:02:57.642 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:02:57.642 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-24T23:02:57.642 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-24T23:02:57.643 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:02:57.643 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:02:57.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:02:57.643 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:02:57.643 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.c", 2024-04-24T23:02:57.643 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmZXFdChAA8TwwkA5EIXHBzPsQSFvXcQ==", 2024-04-24T23:02:57.643 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:02:57.643 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:02:57.643 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-24T23:02:57.643 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-24T23:02:57.643 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:02:57.643 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:02:57.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:02:57.643 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:02:57.643 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.d", 2024-04-24T23:02:57.644 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmLbGuFRAAe91fZlulziWdWiDdLBLWNA==", 2024-04-24T23:02:57.644 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:02:57.644 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:02:57.644 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-24T23:02:57.644 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-24T23:02:57.644 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:02:57.644 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:02:57.644 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:02:57.644 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:02:57.644 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2024-04-24T23:02:57.644 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCfjilmhTwoOBAAig98xa3a2qnv4rriUO9VCQ==", 2024-04-24T23:02:57.644 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:02:57.645 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:02:57.645 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:02:57.645 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:02:57.645 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:02:57.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:02:57.645 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:02:57.645 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2024-04-24T23:02:57.646 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCgjilmwRaAAxAAUqDOyR9HV6DEnmbyn3WAjA==", 2024-04-24T23:02:57.646 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:02:57.646 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:02:57.646 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:02:57.646 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:02:57.647 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:02:57.647 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:02:57.647 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:02:57.647 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2024-04-24T23:02:57.647 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCgjilmDl2qDBAAF1bhTOaSOc612wmFRd8+XA==", 2024-04-24T23:02:57.647 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:02:57.647 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:02:57.647 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:02:57.647 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:02:57.647 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:02:57.647 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:02:57.647 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:02:57.647 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2024-04-24T23:02:57.647 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCgjilmAxHwFRAA9a0v/YwngRxq1vmMRsmEkQ==", 2024-04-24T23:02:57.648 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:02:57.648 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:02:57.648 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:02:57.648 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:02:57.648 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:02:57.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:02:57.648 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:02:57.648 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2024-04-24T23:02:57.648 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCjjilmDPmJIxAASZlEo1ZJHOhIAFGukIj1tQ==", 2024-04-24T23:02:57.648 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:02:57.648 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:02:57.648 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:02:57.648 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:02:57.648 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:02:57.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:02:57.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:02:57.649 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2024-04-24T23:02:57.649 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCjjilmgcpuKhAAedSq0ubNB30s3bz0fY38hw==", 2024-04-24T23:02:57.649 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:02:57.649 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:02:57.649 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:02:57.649 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:02:57.649 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:02:57.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:02:57.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:02:57.649 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2024-04-24T23:02:57.649 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCjjilmFv2VMxAAM+mUc4I0sjawy2uwuUEKYg==", 2024-04-24T23:02:57.649 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:02:57.649 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:02:57.650 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:02:57.650 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:02:57.650 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:02:57.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:02:57.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:02:57.650 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2024-04-24T23:02:57.650 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCkjilmLvEbARAAnaOvQ5Ee0clN/A9SVoKuBQ==", 2024-04-24T23:02:57.650 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:02:57.650 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:02:57.650 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:02:57.650 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:02:57.650 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:02:57.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:02:57.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:02:57.650 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2024-04-24T23:02:57.651 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilm5oeiGRAAdLXwwqrYhV3HHo7Dw/2ZqQ==", 2024-04-24T23:02:57.651 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:02:57.651 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:02:57.651 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2024-04-24T23:02:57.651 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-24T23:02:57.651 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:02:57.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:02:57.651 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:02:57.651 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2024-04-24T23:02:57.651 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmyWBDIhAARtVP4Unk3HHlQYZ2QZ3GzQ==", 2024-04-24T23:02:57.651 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:02:57.651 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:02:57.651 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-24T23:02:57.651 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-24T23:02:57.652 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-24T23:02:57.652 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:02:57.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:02:57.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:02:57.652 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.2", 2024-04-24T23:02:57.652 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilm4QElHxAAXgE316aRCosxX7XER5gtKQ==", 2024-04-24T23:02:57.652 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:02:57.652 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:02:57.652 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-24T23:02:57.652 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-24T23:02:57.652 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-24T23:02:57.652 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:02:57.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:02:57.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:02:57.652 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.3", 2024-04-24T23:02:57.652 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmMZihJxAAoWaAAFY3vVfZnC4/GoT0+g==", 2024-04-24T23:02:57.653 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:02:57.653 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:02:57.653 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-24T23:02:57.653 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-24T23:02:57.653 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-24T23:02:57.653 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:02:57.653 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:02:57.653 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:02:57.653 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2024-04-24T23:02:57.653 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCdjilmdVXeIxAAwGGaAfshx4pHcoc5+3kRww==", 2024-04-24T23:02:57.653 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:02:57.653 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2024-04-24T23:02:57.653 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-24T23:02:57.653 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-24T23:02:57.654 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:02:57.654 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:02:57.654 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:02:57.654 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:02:57.654 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2024-04-24T23:02:57.654 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmn3FJABAAGnvNAoNLzK20cQF6beihmw==", 2024-04-24T23:02:57.654 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:02:57.654 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2024-04-24T23:02:57.654 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2024-04-24T23:02:57.654 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:02:57.654 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:02:57.654 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:02:57.654 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:02:57.654 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2024-04-24T23:02:57.655 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCdjilmMqs2OBAAOe3bLpnCq+HK2EbeXnwY8Q==", 2024-04-24T23:02:57.655 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:02:57.655 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2024-04-24T23:02:57.655 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2024-04-24T23:02:57.655 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:02:57.655 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:02:57.655 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:02:57.655 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:02:57.655 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:02:57.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.640+0000 7f01babe7640 1 -- 172.21.15.32:0/557737346 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f019806be30 msgr2=0x7f019806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:57.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.640+0000 7f01babe7640 1 --2- 172.21.15.32:0/557737346 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f019806be30 0x7f019806e2f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f01a8006fd0 tx=0x7f01a8008040 comp rx=0 tx=0).stop 2024-04-24T23:02:57.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.640+0000 7f01babe7640 1 -- 172.21.15.32:0/557737346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01b40af890 msgr2=0x7f01b41f8720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:57.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.640+0000 7f01babe7640 1 --2- 172.21.15.32:0/557737346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01b40af890 0x7f01b41f8720 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f01a002a180 tx=0x7f01a00046c0 comp rx=0 tx=0).stop 2024-04-24T23:02:57.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.640+0000 7f01babe7640 1 -- 172.21.15.32:0/557737346 shutdown_connections 2024-04-24T23:02:57.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.640+0000 7f01babe7640 1 --2- 172.21.15.32:0/557737346 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f019806be30 0x7f019806e2f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:57.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.640+0000 7f01babe7640 1 --2- 172.21.15.32:0/557737346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01b40af890 0x7f01b41f8720 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:57.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.640+0000 7f01babe7640 1 --2- 172.21.15.32:0/557737346 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01b40b44d0 0x7f01b40ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:57.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.640+0000 7f01babe7640 1 --2- 172.21.15.32:0/557737346 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01b406c5f0 0x7f01b4141f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:57.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.640+0000 7f01babe7640 1 -- 172.21.15.32:0/557737346 >> 172.21.15.32:0/557737346 conn(0x7f01b406b110 msgr2=0x7f01b414fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:57.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.640+0000 7f01babe7640 1 -- 172.21.15.32:0/557737346 shutdown_connections 2024-04-24T23:02:57.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.640+0000 7f01babe7640 1 -- 172.21.15.32:0/557737346 wait complete. 2024-04-24T23:02:57.658 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:02:57.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.757+0000 7f2c2e377640 1 Processor -- start 2024-04-24T23:02:57.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.757+0000 7f2c2e377640 1 -- start start 2024-04-24T23:02:57.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.758+0000 7f2c2e377640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2c2815b380 0x7f2c2806c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:57.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.758+0000 7f2c2e377640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c2815cb00 0x7f2c2806cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:57.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.758+0000 7f2c2e377640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2c280ce430 0x7f2c280ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:57.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.758+0000 7f2c2e377640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f2c280b4520 con 0x7f2c280ce430 2024-04-24T23:02:57.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.758+0000 7f2c2e377640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f2c280b4800 con 0x7f2c2815b380 2024-04-24T23:02:57.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.758+0000 7f2c2e377640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f2c280b4ae0 con 0x7f2c2815cb00 2024-04-24T23:02:57.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.758+0000 7f2c2c914640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2c280ce430 0x7f2c280ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:57.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.758+0000 7f2c2c914640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2c280ce430 0x7f2c280ce880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49446/0 (socket says 172.21.15.32:49446) 2024-04-24T23:02:57.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.758+0000 7f2c2c914640 1 -- 172.21.15.32:0/1021308392 learned_addr learned my addr 172.21.15.32:0/1021308392 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:57.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.758+0000 7f2c27fff640 1 --2- 172.21.15.32:0/1021308392 >> v2:172.21.15.121:3300/0 conn(0x7f2c2815b380 0x7f2c2806c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:57.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.758+0000 7f2c277fe640 1 --2- 172.21.15.32:0/1021308392 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c2815cb00 0x7f2c2806cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:57.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.758+0000 7f2c2c914640 1 -- 172.21.15.32:0/1021308392 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c2815cb00 msgr2=0x7f2c2806cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:57.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.758+0000 7f2c2c914640 1 --2- 172.21.15.32:0/1021308392 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c2815cb00 0x7f2c2806cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:57.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.758+0000 7f2c2c914640 1 -- 172.21.15.32:0/1021308392 >> v2:172.21.15.121:3300/0 conn(0x7f2c2815b380 msgr2=0x7f2c2806c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:57.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.758+0000 7f2c2c914640 1 --2- 172.21.15.32:0/1021308392 >> v2:172.21.15.121:3300/0 conn(0x7f2c2815b380 0x7f2c2806c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:57.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.758+0000 7f2c2c914640 1 -- 172.21.15.32:0/1021308392 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c280b4dc0 con 0x7f2c280ce430 2024-04-24T23:02:57.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.758+0000 7f2c27fff640 1 --2- 172.21.15.32:0/1021308392 >> v2:172.21.15.121:3300/0 conn(0x7f2c2815b380 0x7f2c2806c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:02:57.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.758+0000 7f2c277fe640 1 --2- 172.21.15.32:0/1021308392 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c2815cb00 0x7f2c2806cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:02:57.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.759+0000 7f2c2c914640 1 --2- 172.21.15.32:0/1021308392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c280ce430 0x7f2c280ce880 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f2c2806eed0 tx=0x7f2c2002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7b216c0bb222068 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:57.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.759+0000 7f2c26ffd640 1 -- 172.21.15.32:0/1021308392 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c20036070 con 0x7f2c280ce430 2024-04-24T23:02:57.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.759+0000 7f2c26ffd640 1 -- 172.21.15.32:0/1021308392 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c20031600 con 0x7f2c280ce430 2024-04-24T23:02:57.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.759+0000 7f2c26ffd640 1 -- 172.21.15.32:0/1021308392 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c200319d0 con 0x7f2c280ce430 2024-04-24T23:02:57.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.759+0000 7f2c2e377640 1 -- 172.21.15.32:0/1021308392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c280ce430 msgr2=0x7f2c280ce880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:57.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.759+0000 7f2c2e377640 1 --2- 172.21.15.32:0/1021308392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c280ce430 0x7f2c280ce880 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f2c2806eed0 tx=0x7f2c2002a610 comp rx=0 tx=0).stop 2024-04-24T23:02:57.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.760+0000 7f2c2e377640 1 -- 172.21.15.32:0/1021308392 shutdown_connections 2024-04-24T23:02:57.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.760+0000 7f2c2e377640 1 --2- 172.21.15.32:0/1021308392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c280ce430 0x7f2c280ce880 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:57.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.760+0000 7f2c2e377640 1 --2- 172.21.15.32:0/1021308392 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c2815cb00 0x7f2c2806cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:57.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.760+0000 7f2c2e377640 1 --2- 172.21.15.32:0/1021308392 >> v2:172.21.15.121:3300/0 conn(0x7f2c2815b380 0x7f2c2806c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:57.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.760+0000 7f2c2e377640 1 -- 172.21.15.32:0/1021308392 >> 172.21.15.32:0/1021308392 conn(0x7f2c2806b110 msgr2=0x7f2c2806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:57.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.760+0000 7f2c2e377640 1 -- 172.21.15.32:0/1021308392 shutdown_connections 2024-04-24T23:02:57.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.760+0000 7f2c2e377640 1 -- 172.21.15.32:0/1021308392 wait complete. 2024-04-24T23:02:57.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.761+0000 7f2c2e377640 1 Processor -- start 2024-04-24T23:02:57.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.761+0000 7f2c2e377640 1 -- start start 2024-04-24T23:02:57.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.762+0000 7f2c2e377640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c280ce430 0x7f2c28140c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:57.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.762+0000 7f2c2e377640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c2815b380 0x7f2c280b3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:57.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.762+0000 7f2c2e377640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c2815cb00 0x7f2c280b3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:57.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.762+0000 7f2c2e377640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2c280cedc0 con 0x7f2c280ce430 2024-04-24T23:02:57.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.762+0000 7f2c2e377640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2815b760 con 0x7f2c2815b380 2024-04-24T23:02:57.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.762+0000 7f2c2e377640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f2c280cef40 con 0x7f2c2815cb00 2024-04-24T23:02:57.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.762+0000 7f2c27fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c280ce430 0x7f2c28140c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:57.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.762+0000 7f2c27fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c280ce430 0x7f2c28140c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49450/0 (socket says 172.21.15.32:49450) 2024-04-24T23:02:57.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.762+0000 7f2c27fff640 1 -- 172.21.15.32:0/2857947122 learned_addr learned my addr 172.21.15.32:0/2857947122 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:57.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.762+0000 7f2c2c914640 1 --2- 172.21.15.32:0/2857947122 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c2815cb00 0x7f2c280b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:57.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.762+0000 7f2c277fe640 1 --2- 172.21.15.32:0/2857947122 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c2815b380 0x7f2c280b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:57.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.762+0000 7f2c27fff640 1 -- 172.21.15.32:0/2857947122 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c2815cb00 msgr2=0x7f2c280b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:57.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.762+0000 7f2c27fff640 1 --2- 172.21.15.32:0/2857947122 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c2815cb00 0x7f2c280b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:57.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.762+0000 7f2c27fff640 1 -- 172.21.15.32:0/2857947122 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c2815b380 msgr2=0x7f2c280b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:57.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.762+0000 7f2c27fff640 1 --2- 172.21.15.32:0/2857947122 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c2815b380 0x7f2c280b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:57.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.762+0000 7f2c27fff640 1 -- 172.21.15.32:0/2857947122 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c281411b0 con 0x7f2c280ce430 2024-04-24T23:02:57.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.762+0000 7f2c27fff640 1 --2- 172.21.15.32:0/2857947122 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c280ce430 0x7f2c28140c70 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c00c7f0 tx=0x7f2c1c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:57.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.762+0000 7f2c24ff9640 1 -- 172.21.15.32:0/2857947122 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c015070 con 0x7f2c280ce430 2024-04-24T23:02:57.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.763+0000 7f2c24ff9640 1 -- 172.21.15.32:0/2857947122 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c1c007a30 con 0x7f2c280ce430 2024-04-24T23:02:57.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.763+0000 7f2c2e377640 1 -- 172.21.15.32:0/2857947122 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c28153800 con 0x7f2c280ce430 2024-04-24T23:02:57.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.763+0000 7f2c24ff9640 1 -- 172.21.15.32:0/2857947122 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c007e00 con 0x7f2c280ce430 2024-04-24T23:02:57.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.763+0000 7f2c2e377640 1 -- 172.21.15.32:0/2857947122 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c28153dc0 con 0x7f2c280ce430 2024-04-24T23:02:57.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.763+0000 7f2c2e377640 1 -- 172.21.15.32:0/2857947122 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c28068430 con 0x7f2c280ce430 2024-04-24T23:02:57.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.764+0000 7f2c24ff9640 1 -- 172.21.15.32:0/2857947122 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2c1c018070 con 0x7f2c280ce430 2024-04-24T23:02:57.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.764+0000 7f2c24ff9640 1 --2- 172.21.15.32:0/2857947122 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2c1406be30 0x7f2c1406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:57.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.764+0000 7f2c24ff9640 1 -- 172.21.15.32:0/2857947122 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 4857+0+0 (secure 0 0 0) 0x7f2c1c08a9c0 con 0x7f2c280ce430 2024-04-24T23:02:57.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.765+0000 7f2c277fe640 1 --2- 172.21.15.32:0/2857947122 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2c1406be30 0x7f2c1406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:57.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.765+0000 7f2c277fe640 1 --2- 172.21.15.32:0/2857947122 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2c1406be30 0x7f2c1406e2f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2c280af6a0 tx=0x7f2c18008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:57.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.769+0000 7f2c24ff9640 1 -- 172.21.15.32:0/2857947122 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2c1c08ce20 con 0x7f2c280ce430 2024-04-24T23:02:57.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.955+0000 7f2c2e377640 1 -- 172.21.15.32:0/2857947122 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2c2806c000 con 0x7f2c280ce430 2024-04-24T23:02:57.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.956+0000 7f2c24ff9640 1 -- 172.21.15.32:0/2857947122 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+12903 (secure 0 0 0) 0x7f2c1c059100 con 0x7f2c280ce430 2024-04-24T23:02:57.959 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:02:57.959 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":83,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:02:57.243196+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:02:57.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.958+0000 7f2c2e377640 1 -- 172.21.15.32:0/2857947122 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2c1406be30 msgr2=0x7f2c1406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:57.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.958+0000 7f2c2e377640 1 --2- 172.21.15.32:0/2857947122 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2c1406be30 0x7f2c1406e2f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2c280af6a0 tx=0x7f2c18008040 comp rx=0 tx=0).stop 2024-04-24T23:02:57.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.958+0000 7f2c2e377640 1 -- 172.21.15.32:0/2857947122 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c280ce430 msgr2=0x7f2c28140c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:57.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.958+0000 7f2c2e377640 1 --2- 172.21.15.32:0/2857947122 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c280ce430 0x7f2c28140c70 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c00c7f0 tx=0x7f2c1c00cbf0 comp rx=0 tx=0).stop 2024-04-24T23:02:57.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.958+0000 7f2c2e377640 1 -- 172.21.15.32:0/2857947122 shutdown_connections 2024-04-24T23:02:57.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.958+0000 7f2c2e377640 1 --2- 172.21.15.32:0/2857947122 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2c1406be30 0x7f2c1406e2f0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:57.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.958+0000 7f2c2e377640 1 --2- 172.21.15.32:0/2857947122 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c2815cb00 0x7f2c280b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:57.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.958+0000 7f2c2e377640 1 --2- 172.21.15.32:0/2857947122 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c2815b380 0x7f2c280b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:57.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.958+0000 7f2c2e377640 1 --2- 172.21.15.32:0/2857947122 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c280ce430 0x7f2c28140c70 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:57.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.958+0000 7f2c2e377640 1 -- 172.21.15.32:0/2857947122 >> 172.21.15.32:0/2857947122 conn(0x7f2c2806b110 msgr2=0x7f2c280b5fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:57.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.959+0000 7f2c2e377640 1 -- 172.21.15.32:0/2857947122 shutdown_connections 2024-04-24T23:02:57.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:57.959+0000 7f2c2e377640 1 -- 172.21.15.32:0/2857947122 wait complete. 2024-04-24T23:02:57.972 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:02:57.972 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:02:58.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.073+0000 7f1cf6a58640 1 Processor -- start 2024-04-24T23:02:58.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.074+0000 7f1cf6a58640 1 -- start start 2024-04-24T23:02:58.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.074+0000 7f1cf6a58640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1cf006b820 0x7f1cf00d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:58.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.074+0000 7f1cf6a58640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cf00d3170 con 0x7f1cf00d4210 2024-04-24T23:02:58.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.074+0000 7f1cf6a58640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cf00d3450 con 0x7f1cf015e9b0 2024-04-24T23:02:58.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.074+0000 7f1cf6a58640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f1cf00d3730 con 0x7f1cf006b820 2024-04-24T23:02:58.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.074+0000 7f1cef7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1cf006b820 0x7f1cf00d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:58.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.074+0000 7f1cf4ff5640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1cf00d4210 0x7f1cf00b78f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50498/0 (socket says 172.21.15.32:50498) 2024-04-24T23:02:58.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.075+0000 7f1cef7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1cf006b820 0x7f1cf00d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49454/0 (socket says 172.21.15.32:49454) 2024-04-24T23:02:58.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.075+0000 7f1cf4ff5640 1 -- 172.21.15.32:0/4057784978 learned_addr learned my addr 172.21.15.32:0/4057784978 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:02:58.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.075+0000 7f1cef7fe640 1 -- 172.21.15.32:0/4057784978 >> v1:172.21.15.121:6789/0 conn(0x7f1cf015e9b0 legacy=0x7f1cf006b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:58.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.075+0000 7f1cef7fe640 1 -- 172.21.15.32:0/4057784978 >> v1:172.21.15.32:6789/0 conn(0x7f1cf00d4210 legacy=0x7f1cf00b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:58.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.075+0000 7f1cef7fe640 1 -- 172.21.15.32:0/4057784978 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cf00d3a10 con 0x7f1cf006b820 2024-04-24T23:02:58.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.075+0000 7f1ceeffd640 1 -- 172.21.15.32:0/4057784978 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2347251819 0 0) 0x7f1cf00d3170 con 0x7f1cf00d4210 2024-04-24T23:02:58.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.076+0000 7f1cef7fe640 1 --2- 172.21.15.32:0/4057784978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cf006b820 0x7f1cf00d2b90 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f1ce00097a0 tx=0x7f1ce002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54daeed38f80a146 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:58.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.076+0000 7f1ceeffd640 1 -- 172.21.15.32:0/4057784978 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce0037070 con 0x7f1cf006b820 2024-04-24T23:02:58.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.076+0000 7f1ceeffd640 1 -- 172.21.15.32:0/4057784978 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ce0032440 con 0x7f1cf006b820 2024-04-24T23:02:58.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.076+0000 7f1ceeffd640 1 -- 172.21.15.32:0/4057784978 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce0032810 con 0x7f1cf006b820 2024-04-24T23:02:58.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.076+0000 7f1cf6a58640 1 -- 172.21.15.32:0/4057784978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cf006b820 msgr2=0x7f1cf00d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:58.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.076+0000 7f1cf6a58640 1 --2- 172.21.15.32:0/4057784978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cf006b820 0x7f1cf00d2b90 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f1ce00097a0 tx=0x7f1ce002a1a0 comp rx=0 tx=0).stop 2024-04-24T23:02:58.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.076+0000 7f1cf6a58640 1 -- 172.21.15.32:0/4057784978 shutdown_connections 2024-04-24T23:02:58.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.076+0000 7f1cf6a58640 1 --2- 172.21.15.32:0/4057784978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cf006b820 0x7f1cf00d2b90 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:58.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.076+0000 7f1cf6a58640 1 -- 172.21.15.32:0/4057784978 >> 172.21.15.32:0/4057784978 conn(0x7f1cf00d11d0 msgr2=0x7f1cf00d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:58.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.076+0000 7f1cf6a58640 1 -- 172.21.15.32:0/4057784978 shutdown_connections 2024-04-24T23:02:58.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.076+0000 7f1cf6a58640 1 -- 172.21.15.32:0/4057784978 wait complete. 2024-04-24T23:02:58.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.077+0000 7f1cf6a58640 1 Processor -- start 2024-04-24T23:02:58.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.078+0000 7f1cf6a58640 1 -- start start 2024-04-24T23:02:58.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.078+0000 7f1cf6a58640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cf006b820 0x7f1cf015a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:58.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.078+0000 7f1cf6a58640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1cf00d4210 0x7f1cf00ca620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:58.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.078+0000 7f1cf6a58640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1cf015e9b0 0x7f1cf00ccd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:58.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.078+0000 7f1cf6a58640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf00ba0e0 con 0x7f1cf006b820 2024-04-24T23:02:58.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.078+0000 7f1cf6a58640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf00d2400 con 0x7f1cf00d4210 2024-04-24T23:02:58.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.078+0000 7f1cf6a58640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f1cf00ba260 con 0x7f1cf015e9b0 2024-04-24T23:02:58.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.079+0000 7f1ceffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cf006b820 0x7f1cf015a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:58.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.079+0000 7f1ceffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cf006b820 0x7f1cf015a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49460/0 (socket says 172.21.15.32:49460) 2024-04-24T23:02:58.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.079+0000 7f1ceffff640 1 -- 172.21.15.32:0/2895780797 learned_addr learned my addr 172.21.15.32:0/2895780797 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:58.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.079+0000 7f1cf4ff5640 1 --2- 172.21.15.32:0/2895780797 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1cf015e9b0 0x7f1cf00ccd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:58.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.079+0000 7f1cef7fe640 1 --2- 172.21.15.32:0/2895780797 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1cf00d4210 0x7f1cf00ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:58.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.079+0000 7f1ceffff640 1 -- 172.21.15.32:0/2895780797 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1cf015e9b0 msgr2=0x7f1cf00ccd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:58.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.079+0000 7f1ceffff640 1 --2- 172.21.15.32:0/2895780797 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1cf015e9b0 0x7f1cf00ccd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:58.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.079+0000 7f1ceffff640 1 -- 172.21.15.32:0/2895780797 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1cf00d4210 msgr2=0x7f1cf00ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:58.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.079+0000 7f1ceffff640 1 --2- 172.21.15.32:0/2895780797 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1cf00d4210 0x7f1cf00ca620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:58.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.079+0000 7f1ceffff640 1 -- 172.21.15.32:0/2895780797 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cf00c8320 con 0x7f1cf006b820 2024-04-24T23:02:58.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.079+0000 7f1cf4ff5640 1 --2- 172.21.15.32:0/2895780797 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1cf015e9b0 0x7f1cf00ccd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:02:58.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.079+0000 7f1cef7fe640 1 --2- 172.21.15.32:0/2895780797 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1cf00d4210 0x7f1cf00ca620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:02:58.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.079+0000 7f1ceffff640 1 --2- 172.21.15.32:0/2895780797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cf006b820 0x7f1cf015a5d0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f1ce400c400 tx=0x7f1ce400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:58.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.079+0000 7f1cecff9640 1 -- 172.21.15.32:0/2895780797 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce4017070 con 0x7f1cf006b820 2024-04-24T23:02:58.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.079+0000 7f1cecff9640 1 -- 172.21.15.32:0/2895780797 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ce40052d0 con 0x7f1cf006b820 2024-04-24T23:02:58.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.079+0000 7f1cf6a58640 1 -- 172.21.15.32:0/2895780797 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cf00cd2a0 con 0x7f1cf006b820 2024-04-24T23:02:58.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.079+0000 7f1cecff9640 1 -- 172.21.15.32:0/2895780797 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce4003830 con 0x7f1cf006b820 2024-04-24T23:02:58.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.079+0000 7f1cf6a58640 1 -- 172.21.15.32:0/2895780797 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cf00c85c0 con 0x7f1cf006b820 2024-04-24T23:02:58.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.080+0000 7f1cf6a58640 1 -- 172.21.15.32:0/2895780797 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cf0068430 con 0x7f1cf006b820 2024-04-24T23:02:58.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.081+0000 7f1cecff9640 1 -- 172.21.15.32:0/2895780797 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1ce401a070 con 0x7f1cf006b820 2024-04-24T23:02:58.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.081+0000 7f1cecff9640 1 --2- 172.21.15.32:0/2895780797 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1cdc06be30 0x7f1cdc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:58.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.081+0000 7f1cecff9640 1 -- 172.21.15.32:0/2895780797 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 4857+0+0 (secure 0 0 0) 0x7f1ce4089e70 con 0x7f1cf006b820 2024-04-24T23:02:58.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.082+0000 7f1cef7fe640 1 --2- 172.21.15.32:0/2895780797 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1cdc06be30 0x7f1cdc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:58.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.082+0000 7f1cef7fe640 1 --2- 172.21.15.32:0/2895780797 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1cdc06be30 0x7f1cdc06e2f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1ce0034000 tx=0x7f1ce002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:58.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.086+0000 7f1cecff9640 1 -- 172.21.15.32:0/2895780797 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1ce4058b70 con 0x7f1cf006b820 2024-04-24T23:02:58.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.267+0000 7f1cf6a58640 1 -- 172.21.15.32:0/2895780797 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1cf0074df0 con 0x7f1cf006b820 2024-04-24T23:02:58.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.267+0000 7f1cecff9640 1 -- 172.21.15.32:0/2895780797 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7f1ce4058470 con 0x7f1cf006b820 2024-04-24T23:02:58.270 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T23:02:58.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.269+0000 7f1cf6a58640 1 -- 172.21.15.32:0/2895780797 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1cdc06be30 msgr2=0x7f1cdc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:58.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.269+0000 7f1cf6a58640 1 --2- 172.21.15.32:0/2895780797 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1cdc06be30 0x7f1cdc06e2f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1ce0034000 tx=0x7f1ce002e000 comp rx=0 tx=0).stop 2024-04-24T23:02:58.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.269+0000 7f1cf6a58640 1 -- 172.21.15.32:0/2895780797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cf006b820 msgr2=0x7f1cf015a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:58.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.269+0000 7f1cf6a58640 1 --2- 172.21.15.32:0/2895780797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cf006b820 0x7f1cf015a5d0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f1ce400c400 tx=0x7f1ce400c8c0 comp rx=0 tx=0).stop 2024-04-24T23:02:58.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.270+0000 7f1cf6a58640 1 -- 172.21.15.32:0/2895780797 shutdown_connections 2024-04-24T23:02:58.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.270+0000 7f1cf6a58640 1 --2- 172.21.15.32:0/2895780797 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1cdc06be30 0x7f1cdc06e2f0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:58.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.270+0000 7f1cf6a58640 1 --2- 172.21.15.32:0/2895780797 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1cf015e9b0 0x7f1cf00ccd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:58.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.270+0000 7f1cf6a58640 1 --2- 172.21.15.32:0/2895780797 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1cf00d4210 0x7f1cf00ca620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:58.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.270+0000 7f1cf6a58640 1 --2- 172.21.15.32:0/2895780797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cf006b820 0x7f1cf015a5d0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:58.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.270+0000 7f1cf6a58640 1 -- 172.21.15.32:0/2895780797 >> 172.21.15.32:0/2895780797 conn(0x7f1cf00d11d0 msgr2=0x7f1cf006b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:58.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.270+0000 7f1cf6a58640 1 -- 172.21.15.32:0/2895780797 shutdown_connections 2024-04-24T23:02:58.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.270+0000 7f1cf6a58640 1 -- 172.21.15.32:0/2895780797 wait complete. 2024-04-24T23:02:58.283 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-24T23:02:58.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.381+0000 7f8f29672640 1 Processor -- start 2024-04-24T23:02:58.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.381+0000 7f8f29672640 1 -- start start 2024-04-24T23:02:58.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.382+0000 7f8f29672640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8f2406b820 0x7f8f240d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:58.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.382+0000 7f8f29672640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f240d3170 con 0x7f8f2415e9b0 2024-04-24T23:02:58.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.382+0000 7f8f29672640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f240d3450 con 0x7f8f240d4210 2024-04-24T23:02:58.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.382+0000 7f8f29672640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f8f240d3730 con 0x7f8f2406b820 2024-04-24T23:02:58.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.382+0000 7f8f22ffd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8f2415e9b0 0x7f8f2406b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50502/0 (socket says 172.21.15.32:50502) 2024-04-24T23:02:58.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.382+0000 7f8f22ffd640 1 -- 172.21.15.32:0/3426686991 learned_addr learned my addr 172.21.15.32:0/3426686991 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:02:58.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.383+0000 7f8f227fc640 1 --2- 172.21.15.32:0/3426686991 >> v2:172.21.15.121:3300/0 conn(0x7f8f2406b820 0x7f8f240d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:58.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.383+0000 7f8f21ffb640 1 -- 172.21.15.32:0/3426686991 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 555388245 0 0) 0x7f8f240d3170 con 0x7f8f2415e9b0 2024-04-24T23:02:58.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.383+0000 7f8f21ffb640 1 -- 172.21.15.32:0/3426686991 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f10003520 con 0x7f8f2415e9b0 2024-04-24T23:02:58.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.383+0000 7f8f21ffb640 1 -- 172.21.15.32:0/3426686991 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2968328721 0 0) 0x7f8f240d3450 con 0x7f8f240d4210 2024-04-24T23:02:58.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.383+0000 7f8f21ffb640 1 -- 172.21.15.32:0/3426686991 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f240d3170 con 0x7f8f240d4210 2024-04-24T23:02:58.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.383+0000 7f8f21ffb640 1 -- 172.21.15.32:0/3426686991 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2273278438 0 0) 0x7f8f10003520 con 0x7f8f2415e9b0 2024-04-24T23:02:58.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.383+0000 7f8f21ffb640 1 -- 172.21.15.32:0/3426686991 >> v2:172.21.15.121:3300/0 conn(0x7f8f2406b820 msgr2=0x7f8f240d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:58.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.383+0000 7f8f21ffb640 1 --2- 172.21.15.32:0/3426686991 >> v2:172.21.15.121:3300/0 conn(0x7f8f2406b820 0x7f8f240d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:58.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.383+0000 7f8f21ffb640 1 -- 172.21.15.32:0/3426686991 >> v1:172.21.15.121:6789/0 conn(0x7f8f240d4210 legacy=0x7f8f240b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:58.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.383+0000 7f8f21ffb640 1 -- 172.21.15.32:0/3426686991 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f240d3a10 con 0x7f8f2415e9b0 2024-04-24T23:02:58.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.383+0000 7f8f21ffb640 1 -- 172.21.15.32:0/3426686991 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f8f1c001670 con 0x7f8f2415e9b0 2024-04-24T23:02:58.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.383+0000 7f8f21ffb640 1 -- 172.21.15.32:0/3426686991 >> v1:172.21.15.32:6789/0 conn(0x7f8f2415e9b0 legacy=0x7f8f2406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:58.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.383+0000 7f8f21ffb640 1 --2- 172.21.15.32:0/3426686991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f10003b90 0x7f8f10003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:58.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.383+0000 7f8f21ffb640 1 --2- 172.21.15.32:0/3426686991 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f10004800 0x7f8f10004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:58.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.383+0000 7f8f21ffb640 1 --2- 172.21.15.32:0/3426686991 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8f10005210 0x7f8f1000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:58.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.383+0000 7f8f21ffb640 1 -- 172.21.15.32:0/3426686991 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8f240d3730 con 0x7f8f10003b90 2024-04-24T23:02:58.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.383+0000 7f8f21ffb640 1 -- 172.21.15.32:0/3426686991 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1000c140 con 0x7f8f10004800 2024-04-24T23:02:58.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.383+0000 7f8f21ffb640 1 -- 172.21.15.32:0/3426686991 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f8f1000c340 con 0x7f8f10005210 2024-04-24T23:02:58.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.383+0000 7f8f22ffd640 1 --2- 172.21.15.32:0/3426686991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f10003b90 0x7f8f10003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:58.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.383+0000 7f8f237fe640 1 --2- 172.21.15.32:0/3426686991 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8f10005210 0x7f8f1000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:58.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.383+0000 7f8f227fc640 1 --2- 172.21.15.32:0/3426686991 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f10004800 0x7f8f10004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:58.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.384+0000 7f8f22ffd640 1 -- 172.21.15.32:0/3426686991 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8f10005210 msgr2=0x7f8f1000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:58.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.384+0000 7f8f22ffd640 1 --2- 172.21.15.32:0/3426686991 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8f10005210 0x7f8f1000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:58.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.384+0000 7f8f22ffd640 1 -- 172.21.15.32:0/3426686991 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f10004800 msgr2=0x7f8f10004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:58.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.384+0000 7f8f22ffd640 1 --2- 172.21.15.32:0/3426686991 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f10004800 0x7f8f10004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:58.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.384+0000 7f8f237fe640 1 -- 172.21.15.32:0/3426686991 reap_dead start 2024-04-24T23:02:58.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.384+0000 7f8f22ffd640 1 -- 172.21.15.32:0/3426686991 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f10003520 con 0x7f8f10003b90 2024-04-24T23:02:58.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.384+0000 7f8f22ffd640 1 --2- 172.21.15.32:0/3426686991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f10003b90 0x7f8f10003f90 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c007fc0 tx=0x7f8f10005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbb59861cf7b5220 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:58.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.384+0000 7f8f21ffb640 1 -- 172.21.15.32:0/3426686991 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c007be0 con 0x7f8f10003b90 2024-04-24T23:02:58.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.384+0000 7f8f21ffb640 1 -- 172.21.15.32:0/3426686991 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f1c005af0 con 0x7f8f10003b90 2024-04-24T23:02:58.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.384+0000 7f8f21ffb640 1 -- 172.21.15.32:0/3426686991 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c007070 con 0x7f8f10003b90 2024-04-24T23:02:58.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.385+0000 7f8f29672640 1 -- 172.21.15.32:0/3426686991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f10003b90 msgr2=0x7f8f10003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:58.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.385+0000 7f8f29672640 1 --2- 172.21.15.32:0/3426686991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f10003b90 0x7f8f10003f90 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c007fc0 tx=0x7f8f10005830 comp rx=0 tx=0).stop 2024-04-24T23:02:58.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.385+0000 7f8f29672640 1 -- 172.21.15.32:0/3426686991 shutdown_connections 2024-04-24T23:02:58.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.385+0000 7f8f29672640 1 --2- 172.21.15.32:0/3426686991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f10003b90 0x7f8f10003f90 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:58.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.385+0000 7f8f29672640 1 -- 172.21.15.32:0/3426686991 >> 172.21.15.32:0/3426686991 conn(0x7f8f240d11d0 msgr2=0x7f8f240d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:58.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.385+0000 7f8f29672640 1 -- 172.21.15.32:0/3426686991 shutdown_connections 2024-04-24T23:02:58.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.385+0000 7f8f29672640 1 -- 172.21.15.32:0/3426686991 wait complete. 2024-04-24T23:02:58.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.386+0000 7f8f29672640 1 Processor -- start 2024-04-24T23:02:58.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.386+0000 7f8f29672640 1 -- start start 2024-04-24T23:02:58.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.387+0000 7f8f29672640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8f10003b90 0x7f8f240c97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:58.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.387+0000 7f8f29672640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f240c8280 0x7f8f240c7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:58.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.387+0000 7f8f29672640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f24157ad0 0x7f8f24157f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:58.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.387+0000 7f8f29672640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8f240ba0e0 con 0x7f8f24157ad0 2024-04-24T23:02:58.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.387+0000 7f8f29672640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8f240d2400 con 0x7f8f240c8280 2024-04-24T23:02:58.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.387+0000 7f8f29672640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f8f240ba260 con 0x7f8f10003b90 2024-04-24T23:02:58.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.387+0000 7f8f237fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f24157ad0 0x7f8f24157f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:58.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.387+0000 7f8f237fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f24157ad0 0x7f8f24157f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49478/0 (socket says 172.21.15.32:49478) 2024-04-24T23:02:58.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.387+0000 7f8f237fe640 1 -- 172.21.15.32:0/1876176229 learned_addr learned my addr 172.21.15.32:0/1876176229 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:58.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.387+0000 7f8f237fe640 1 -- 172.21.15.32:0/1876176229 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8f10003b90 msgr2=0x7f8f240c97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:58.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.387+0000 7f8f237fe640 1 --2- 172.21.15.32:0/1876176229 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8f10003b90 0x7f8f240c97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:58.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.387+0000 7f8f237fe640 1 -- 172.21.15.32:0/1876176229 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f240c8280 msgr2=0x7f8f240c7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:58.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.387+0000 7f8f237fe640 1 --2- 172.21.15.32:0/1876176229 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f240c8280 0x7f8f240c7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:58.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.387+0000 7f8f237fe640 1 -- 172.21.15.32:0/1876176229 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f240c8710 con 0x7f8f24157ad0 2024-04-24T23:02:58.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.388+0000 7f8f237fe640 1 --2- 172.21.15.32:0/1876176229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f24157ad0 0x7f8f24157f80 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f8f2406bdd0 tx=0x7f8f1400ff70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:58.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.388+0000 7f8f1b7fe640 1 -- 172.21.15.32:0/1876176229 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f14019070 con 0x7f8f24157ad0 2024-04-24T23:02:58.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.388+0000 7f8f1b7fe640 1 -- 172.21.15.32:0/1876176229 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f14006e20 con 0x7f8f24157ad0 2024-04-24T23:02:58.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.388+0000 7f8f29672640 1 -- 172.21.15.32:0/1876176229 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f240c89f0 con 0x7f8f24157ad0 2024-04-24T23:02:58.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.388+0000 7f8f1b7fe640 1 -- 172.21.15.32:0/1876176229 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f14003bb0 con 0x7f8f24157ad0 2024-04-24T23:02:58.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.388+0000 7f8f29672640 1 -- 172.21.15.32:0/1876176229 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f240c8fb0 con 0x7f8f24157ad0 2024-04-24T23:02:58.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.389+0000 7f8f29672640 1 -- 172.21.15.32:0/1876176229 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f24068430 con 0x7f8f24157ad0 2024-04-24T23:02:58.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.389+0000 7f8f1b7fe640 1 -- 172.21.15.32:0/1876176229 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8f1401c070 con 0x7f8f24157ad0 2024-04-24T23:02:58.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.390+0000 7f8f1b7fe640 1 --2- 172.21.15.32:0/1876176229 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8f0406be30 0x7f8f0406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:58.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.390+0000 7f8f1b7fe640 1 -- 172.21.15.32:0/1876176229 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 4857+0+0 (secure 0 0 0) 0x7f8f14089e40 con 0x7f8f24157ad0 2024-04-24T23:02:58.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.390+0000 7f8f22ffd640 1 --2- 172.21.15.32:0/1876176229 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8f0406be30 0x7f8f0406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:58.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.391+0000 7f8f22ffd640 1 --2- 172.21.15.32:0/1876176229 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8f0406be30 0x7f8f0406e2f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c032000 tx=0x7f8f1c007000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:58.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.394+0000 7f8f1b7fe640 1 -- 172.21.15.32:0/1876176229 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8f14058bd0 con 0x7f8f24157ad0 2024-04-24T23:02:58.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.581+0000 7f8f29672640 1 -- 172.21.15.32:0/1876176229 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f8f2406b2b0 con 0x7f8f24157ad0 2024-04-24T23:02:58.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.581+0000 7f8f1b7fe640 1 -- 172.21.15.32:0/1876176229 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v83) ==== 84+0+65 (secure 0 0 0) 0x7f8f140584d0 con 0x7f8f24157ad0 2024-04-24T23:02:58.584 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:02:58.585 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2024-04-24T23:02:58.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:02:58.585 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2024-04-24T23:02:58.585 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": ".mgr" 2024-04-24T23:02:58.585 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:02:58.586 INFO:teuthology.orchestra.run.smithi032.stdout:] 2024-04-24T23:02:58.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.583+0000 7f8f29672640 1 -- 172.21.15.32:0/1876176229 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8f0406be30 msgr2=0x7f8f0406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:58.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.583+0000 7f8f29672640 1 --2- 172.21.15.32:0/1876176229 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8f0406be30 0x7f8f0406e2f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c032000 tx=0x7f8f1c007000 comp rx=0 tx=0).stop 2024-04-24T23:02:58.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.583+0000 7f8f29672640 1 -- 172.21.15.32:0/1876176229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f24157ad0 msgr2=0x7f8f24157f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:58.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.583+0000 7f8f29672640 1 --2- 172.21.15.32:0/1876176229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f24157ad0 0x7f8f24157f80 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f8f2406bdd0 tx=0x7f8f1400ff70 comp rx=0 tx=0).stop 2024-04-24T23:02:58.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.583+0000 7f8f29672640 1 -- 172.21.15.32:0/1876176229 shutdown_connections 2024-04-24T23:02:58.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.583+0000 7f8f29672640 1 --2- 172.21.15.32:0/1876176229 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8f0406be30 0x7f8f0406e2f0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:58.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.583+0000 7f8f29672640 1 --2- 172.21.15.32:0/1876176229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f24157ad0 0x7f8f24157f80 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:58.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.583+0000 7f8f29672640 1 --2- 172.21.15.32:0/1876176229 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f240c8280 0x7f8f240c7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:58.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.583+0000 7f8f29672640 1 --2- 172.21.15.32:0/1876176229 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8f10003b90 0x7f8f240c97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:58.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.583+0000 7f8f29672640 1 -- 172.21.15.32:0/1876176229 >> 172.21.15.32:0/1876176229 conn(0x7f8f240d11d0 msgr2=0x7f8f24071730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:58.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.583+0000 7f8f29672640 1 -- 172.21.15.32:0/1876176229 shutdown_connections 2024-04-24T23:02:58.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.583+0000 7f8f29672640 1 -- 172.21.15.32:0/1876176229 wait complete. 2024-04-24T23:02:58.597 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-24T23:02:58.597 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-24T23:02:58.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.698+0000 7f6726eb3640 1 Processor -- start 2024-04-24T23:02:58.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.698+0000 7f6726eb3640 1 -- start start 2024-04-24T23:02:58.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.699+0000 7f6726eb3640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f672015e770 0x7f67200d40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:58.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.699+0000 7f6726eb3640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f67200b9ff0 0x7f67200d2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:58.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.699+0000 7f6726eb3640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67200d3000 con 0x7f67200b6450 2024-04-24T23:02:58.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.699+0000 7f6726eb3640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f67200d32e0 con 0x7f67200b9ff0 2024-04-24T23:02:58.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.699+0000 7f6726eb3640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f67200d35c0 con 0x7f672015e770 2024-04-24T23:02:58.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.700+0000 7f6724c4f640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f672015e770 0x7f67200d40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:58.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.700+0000 7f6724c4f640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f672015e770 0x7f67200d40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.32:46896/0 (socket says 172.21.15.32:46896) 2024-04-24T23:02:58.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.700+0000 7f6724c4f640 1 -- 172.21.15.32:0/2593123053 learned_addr learned my addr 172.21.15.32:0/2593123053 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:58.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.700+0000 7f6725450640 1 --1- 172.21.15.32:0/2593123053 >> v1:172.21.15.121:6789/0 conn(0x7f67200b6450 0x7f67200d0ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:55726/0 (socket says 172.21.15.32:55726) 2024-04-24T23:02:58.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.700+0000 7f671ffff640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f67200b9ff0 0x7f67200d2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:58.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.700+0000 7f6724c4f640 1 -- 172.21.15.32:0/2593123053 >> v2:172.21.15.121:3300/0 conn(0x7f67200b9ff0 msgr2=0x7f67200d2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:02:58.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.700+0000 7f6724c4f640 1 --2- 172.21.15.32:0/2593123053 >> v2:172.21.15.121:3300/0 conn(0x7f67200b9ff0 0x7f67200d2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:58.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.700+0000 7f6724c4f640 1 -- 172.21.15.32:0/2593123053 >> v1:172.21.15.121:6789/0 conn(0x7f67200b6450 legacy=0x7f67200d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:58.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.700+0000 7f6724c4f640 1 -- 172.21.15.32:0/2593123053 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67200d38a0 con 0x7f672015e770 2024-04-24T23:02:58.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.701+0000 7f6724c4f640 1 --2- 172.21.15.32:0/2593123053 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f672015e770 0x7f67200d40a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6718009870 tx=0x7f671802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38769ebe0585da23 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:58.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.701+0000 7f671f7fe640 1 -- 172.21.15.32:0/2593123053 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6718037070 con 0x7f672015e770 2024-04-24T23:02:58.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.701+0000 7f671f7fe640 1 -- 172.21.15.32:0/2593123053 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6718032440 con 0x7f672015e770 2024-04-24T23:02:58.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.701+0000 7f671f7fe640 1 -- 172.21.15.32:0/2593123053 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6718032810 con 0x7f672015e770 2024-04-24T23:02:58.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.701+0000 7f6726eb3640 1 -- 172.21.15.32:0/2593123053 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f672015e770 msgr2=0x7f67200d40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:58.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.701+0000 7f6726eb3640 1 --2- 172.21.15.32:0/2593123053 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f672015e770 0x7f67200d40a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6718009870 tx=0x7f671802a270 comp rx=0 tx=0).stop 2024-04-24T23:02:58.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.701+0000 7f6726eb3640 1 -- 172.21.15.32:0/2593123053 shutdown_connections 2024-04-24T23:02:58.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.701+0000 7f6726eb3640 1 --2- 172.21.15.32:0/2593123053 >> v2:172.21.15.121:3300/0 conn(0x7f67200b9ff0 0x7f67200d2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:58.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.701+0000 7f6726eb3640 1 --2- 172.21.15.32:0/2593123053 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f672015e770 0x7f67200d40a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:58.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.701+0000 7f6726eb3640 1 -- 172.21.15.32:0/2593123053 >> 172.21.15.32:0/2593123053 conn(0x7f6720071910 msgr2=0x7f6720071d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:58.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.701+0000 7f6726eb3640 1 -- 172.21.15.32:0/2593123053 shutdown_connections 2024-04-24T23:02:58.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.701+0000 7f6726eb3640 1 -- 172.21.15.32:0/2593123053 wait complete. 2024-04-24T23:02:58.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.703+0000 7f6726eb3640 1 Processor -- start 2024-04-24T23:02:58.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.703+0000 7f6726eb3640 1 -- start start 2024-04-24T23:02:58.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.703+0000 7f6726eb3640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f67200b6450 0x7f6720160e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:58.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.703+0000 7f6726eb3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67200b9ff0 0x7f67200c8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:58.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.703+0000 7f6726eb3640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f672015e770 0x7f67201f4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:58.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.704+0000 7f6726eb3640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f67200d2dc0 con 0x7f67200b9ff0 2024-04-24T23:02:58.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.704+0000 7f6726eb3640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f67200d2c40 con 0x7f672015e770 2024-04-24T23:02:58.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.704+0000 7f6726eb3640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f672015eb50 con 0x7f67200b6450 2024-04-24T23:02:58.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.704+0000 7f671ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67200b9ff0 0x7f67200c8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:58.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.704+0000 7f671ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67200b9ff0 0x7f67200c8980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49490/0 (socket says 172.21.15.32:49490) 2024-04-24T23:02:58.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.704+0000 7f671ffff640 1 -- 172.21.15.32:0/4146816555 learned_addr learned my addr 172.21.15.32:0/4146816555 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:58.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.704+0000 7f6724c4f640 1 --2- 172.21.15.32:0/4146816555 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f67200b6450 0x7f6720160e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:58.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.704+0000 7f6725450640 1 --2- 172.21.15.32:0/4146816555 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f672015e770 0x7f67201f4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:58.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.704+0000 7f671ffff640 1 -- 172.21.15.32:0/4146816555 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f67200b6450 msgr2=0x7f6720160e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:58.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.704+0000 7f671ffff640 1 --2- 172.21.15.32:0/4146816555 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f67200b6450 0x7f6720160e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:58.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.704+0000 7f671ffff640 1 -- 172.21.15.32:0/4146816555 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f672015e770 msgr2=0x7f67201f4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:58.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.704+0000 7f671ffff640 1 --2- 172.21.15.32:0/4146816555 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f672015e770 0x7f67201f4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:58.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.704+0000 7f671ffff640 1 -- 172.21.15.32:0/4146816555 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67201f4e50 con 0x7f67200b9ff0 2024-04-24T23:02:58.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.704+0000 7f6725450640 1 --2- 172.21.15.32:0/4146816555 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f672015e770 0x7f67201f4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:02:58.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.704+0000 7f671ffff640 1 --2- 172.21.15.32:0/4146816555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67200b9ff0 0x7f67200c8980 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f671000efc0 tx=0x7f671000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:58.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.704+0000 7f671d7fa640 1 -- 172.21.15.32:0/4146816555 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6710018020 con 0x7f67200b9ff0 2024-04-24T23:02:58.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.704+0000 7f671d7fa640 1 -- 172.21.15.32:0/4146816555 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6710014480 con 0x7f67200b9ff0 2024-04-24T23:02:58.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.704+0000 7f6726eb3640 1 -- 172.21.15.32:0/4146816555 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67201f5130 con 0x7f67200b9ff0 2024-04-24T23:02:58.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.704+0000 7f671d7fa640 1 -- 172.21.15.32:0/4146816555 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6710014830 con 0x7f67200b9ff0 2024-04-24T23:02:58.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.704+0000 7f6726eb3640 1 -- 172.21.15.32:0/4146816555 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67200c8ec0 con 0x7f67200b9ff0 2024-04-24T23:02:58.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.706+0000 7f6726eb3640 1 -- 172.21.15.32:0/4146816555 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6720068430 con 0x7f67200b9ff0 2024-04-24T23:02:58.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.706+0000 7f671d7fa640 1 -- 172.21.15.32:0/4146816555 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6710005d10 con 0x7f67200b9ff0 2024-04-24T23:02:58.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.706+0000 7f671d7fa640 1 --2- 172.21.15.32:0/4146816555 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f670006be30 0x7f670006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:58.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.706+0000 7f671d7fa640 1 -- 172.21.15.32:0/4146816555 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 4857+0+0 (secure 0 0 0) 0x7f671008b880 con 0x7f67200b9ff0 2024-04-24T23:02:58.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.707+0000 7f6724c4f640 1 --2- 172.21.15.32:0/4146816555 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f670006be30 0x7f670006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:58.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.707+0000 7f6724c4f640 1 --2- 172.21.15.32:0/4146816555 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f670006be30 0x7f670006e2f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6718034000 tx=0x7f671802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:58.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.710+0000 7f671d7fa640 1 -- 172.21.15.32:0/4146816555 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6710022030 con 0x7f67200b9ff0 2024-04-24T23:02:58.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:58.897+0000 7f6726eb3640 1 -- 172.21.15.32:0/4146816555 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f67200c8560 con 0x7f67200b9ff0 2024-04-24T23:02:59.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.253+0000 7f671d7fa640 1 -- 172.21.15.32:0/4146816555 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v84) ==== 138+0+0 (secure 0 0 0) 0x7f6710010040 con 0x7f67200b9ff0 2024-04-24T23:02:59.256 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2024-04-24T23:02:59.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.256+0000 7f6726eb3640 1 -- 172.21.15.32:0/4146816555 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f670006be30 msgr2=0x7f670006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:59.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.256+0000 7f6726eb3640 1 --2- 172.21.15.32:0/4146816555 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f670006be30 0x7f670006e2f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6718034000 tx=0x7f671802e000 comp rx=0 tx=0).stop 2024-04-24T23:02:59.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.256+0000 7f6726eb3640 1 -- 172.21.15.32:0/4146816555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67200b9ff0 msgr2=0x7f67200c8980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:59.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.256+0000 7f6726eb3640 1 --2- 172.21.15.32:0/4146816555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67200b9ff0 0x7f67200c8980 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f671000efc0 tx=0x7f671000c490 comp rx=0 tx=0).stop 2024-04-24T23:02:59.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.256+0000 7f6726eb3640 1 -- 172.21.15.32:0/4146816555 shutdown_connections 2024-04-24T23:02:59.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.256+0000 7f6726eb3640 1 --2- 172.21.15.32:0/4146816555 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f670006be30 0x7f670006e2f0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:59.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.256+0000 7f6726eb3640 1 --2- 172.21.15.32:0/4146816555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67200b9ff0 0x7f67200c8980 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:59.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.256+0000 7f6726eb3640 1 --2- 172.21.15.32:0/4146816555 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f672015e770 0x7f67201f4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:59.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.256+0000 7f6726eb3640 1 --2- 172.21.15.32:0/4146816555 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f67200b6450 0x7f6720160e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:59.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.256+0000 7f6726eb3640 1 -- 172.21.15.32:0/4146816555 >> 172.21.15.32:0/4146816555 conn(0x7f6720071910 msgr2=0x7f67200b58f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:59.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.256+0000 7f6726eb3640 1 -- 172.21.15.32:0/4146816555 shutdown_connections 2024-04-24T23:02:59.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.256+0000 7f6726eb3640 1 -- 172.21.15.32:0/4146816555 wait complete. 2024-04-24T23:02:59.272 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-24T23:02:59.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.379+0000 7f74fb4ba640 1 Processor -- start 2024-04-24T23:02:59.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.379+0000 7f74fb4ba640 1 -- start start 2024-04-24T23:02:59.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.380+0000 7f74fb4ba640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f74f415b380 0x7f74f406c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:59.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.380+0000 7f74fb4ba640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74f40b4520 con 0x7f74f415cb00 2024-04-24T23:02:59.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.380+0000 7f74fb4ba640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74f40b4800 con 0x7f74f40ceb40 2024-04-24T23:02:59.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.380+0000 7f74fb4ba640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f74f40cef90 con 0x7f74f415b380 2024-04-24T23:02:59.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.380+0000 7f74f8a55640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f74f415cb00 0x7f74f40ce430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50504/0 (socket says 172.21.15.32:50504) 2024-04-24T23:02:59.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.380+0000 7f74f8a55640 1 -- 172.21.15.32:0/2850888929 learned_addr learned my addr 172.21.15.32:0/2850888929 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:02:59.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.380+0000 7f74f9256640 1 --2- 172.21.15.32:0/2850888929 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f74f415b380 0x7f74f406c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:59.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.380+0000 7f74f3fff640 1 -- 172.21.15.32:0/2850888929 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 543546379 0 0) 0x7f74f40b4520 con 0x7f74f415cb00 2024-04-24T23:02:59.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.380+0000 7f74f3fff640 1 -- 172.21.15.32:0/2850888929 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f74dc003520 con 0x7f74f415cb00 2024-04-24T23:02:59.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.381+0000 7f74f3fff640 1 -- 172.21.15.32:0/2850888929 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2477723927 0 0) 0x7f74f40b4800 con 0x7f74f40ceb40 2024-04-24T23:02:59.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.381+0000 7f74f3fff640 1 -- 172.21.15.32:0/2850888929 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f74f40b4520 con 0x7f74f40ceb40 2024-04-24T23:02:59.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.381+0000 7f74f9256640 1 -- 172.21.15.32:0/2850888929 >> v1:172.21.15.121:6789/0 conn(0x7f74f40ceb40 legacy=0x7f74f40b55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:59.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.381+0000 7f74f9256640 1 -- 172.21.15.32:0/2850888929 >> v1:172.21.15.32:6789/0 conn(0x7f74f415cb00 legacy=0x7f74f40ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:59.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.381+0000 7f74f9256640 1 -- 172.21.15.32:0/2850888929 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74f40b4ae0 con 0x7f74f415b380 2024-04-24T23:02:59.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.381+0000 7f74f9256640 1 --2- 172.21.15.32:0/2850888929 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f74f415b380 0x7f74f406c5f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f74ec0097d0 tx=0x7f74ec02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c365bc2f0ab7cb81 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:59.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.382+0000 7f74f3fff640 1 -- 172.21.15.32:0/2850888929 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ec037070 con 0x7f74f415b380 2024-04-24T23:02:59.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.382+0000 7f74f3fff640 1 -- 172.21.15.32:0/2850888929 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74ec032440 con 0x7f74f415b380 2024-04-24T23:02:59.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.382+0000 7f74f3fff640 1 -- 172.21.15.32:0/2850888929 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ec032810 con 0x7f74f415b380 2024-04-24T23:02:59.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.382+0000 7f74fb4ba640 1 -- 172.21.15.32:0/2850888929 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f74f415b380 msgr2=0x7f74f406c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:59.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.382+0000 7f74fb4ba640 1 --2- 172.21.15.32:0/2850888929 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f74f415b380 0x7f74f406c5f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f74ec0097d0 tx=0x7f74ec02a180 comp rx=0 tx=0).stop 2024-04-24T23:02:59.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.382+0000 7f74fb4ba640 1 -- 172.21.15.32:0/2850888929 shutdown_connections 2024-04-24T23:02:59.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.382+0000 7f74fb4ba640 1 --2- 172.21.15.32:0/2850888929 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f74f415b380 0x7f74f406c5f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:59.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.382+0000 7f74fb4ba640 1 -- 172.21.15.32:0/2850888929 >> 172.21.15.32:0/2850888929 conn(0x7f74f406b110 msgr2=0x7f74f406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:59.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.382+0000 7f74fb4ba640 1 -- 172.21.15.32:0/2850888929 shutdown_connections 2024-04-24T23:02:59.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.382+0000 7f74fb4ba640 1 -- 172.21.15.32:0/2850888929 wait complete. 2024-04-24T23:02:59.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.383+0000 7f74fb4ba640 1 Processor -- start 2024-04-24T23:02:59.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.383+0000 7f74fb4ba640 1 -- start start 2024-04-24T23:02:59.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.384+0000 7f74fb4ba640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f74f40ceb40 0x7f74f41500a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:59.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.384+0000 7f74fb4ba640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f74f415b380 0x7f74f41572c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:59.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.384+0000 7f74fb4ba640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74f415cb00 0x7f74f415cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:59.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.384+0000 7f74fb4ba640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f74f415b760 con 0x7f74f415cb00 2024-04-24T23:02:59.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.384+0000 7f74fb4ba640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f74f40b50a0 con 0x7f74f40ceb40 2024-04-24T23:02:59.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.384+0000 7f74fb4ba640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f74f406cb30 con 0x7f74f415b380 2024-04-24T23:02:59.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.384+0000 7f74f9a57640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74f415cb00 0x7f74f415cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:59.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.384+0000 7f74f9a57640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74f415cb00 0x7f74f415cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49504/0 (socket says 172.21.15.32:49504) 2024-04-24T23:02:59.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.384+0000 7f74f9a57640 1 -- 172.21.15.32:0/870199836 learned_addr learned my addr 172.21.15.32:0/870199836 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:02:59.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.384+0000 7f74f9256640 1 --2- 172.21.15.32:0/870199836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f74f40ceb40 0x7f74f41500a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:59.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.384+0000 7f74f8a55640 1 --2- 172.21.15.32:0/870199836 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f74f415b380 0x7f74f41572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:59.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.385+0000 7f74f9a57640 1 -- 172.21.15.32:0/870199836 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f74f415b380 msgr2=0x7f74f41572c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:59.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.385+0000 7f74f9a57640 1 --2- 172.21.15.32:0/870199836 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f74f415b380 0x7f74f41572c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:59.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.385+0000 7f74f9a57640 1 -- 172.21.15.32:0/870199836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f74f40ceb40 msgr2=0x7f74f41500a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:59.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.385+0000 7f74f9a57640 1 --2- 172.21.15.32:0/870199836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f74f40ceb40 0x7f74f41500a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:59.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.385+0000 7f74f9a57640 1 -- 172.21.15.32:0/870199836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74f4152b50 con 0x7f74f415cb00 2024-04-24T23:02:59.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.385+0000 7f74f9a57640 1 --2- 172.21.15.32:0/870199836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74f415cb00 0x7f74f415cfe0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f74e400c7c0 tx=0x7f74e400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:59.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.385+0000 7f74f1ffb640 1 -- 172.21.15.32:0/870199836 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e4003b70 con 0x7f74f415cb00 2024-04-24T23:02:59.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.385+0000 7f74f1ffb640 1 -- 172.21.15.32:0/870199836 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74e4003d10 con 0x7f74f415cb00 2024-04-24T23:02:59.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.385+0000 7f74fb4ba640 1 -- 172.21.15.32:0/870199836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74f4152e30 con 0x7f74f415cb00 2024-04-24T23:02:59.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.385+0000 7f74f1ffb640 1 -- 172.21.15.32:0/870199836 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e400aa50 con 0x7f74f415cb00 2024-04-24T23:02:59.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.385+0000 7f74fb4ba640 1 -- 172.21.15.32:0/870199836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74f41533f0 con 0x7f74f415cb00 2024-04-24T23:02:59.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.386+0000 7f74fb4ba640 1 -- 172.21.15.32:0/870199836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74f4068430 con 0x7f74f415cb00 2024-04-24T23:02:59.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.387+0000 7f74f1ffb640 1 -- 172.21.15.32:0/870199836 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f74e400abf0 con 0x7f74f415cb00 2024-04-24T23:02:59.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.387+0000 7f74f1ffb640 1 --2- 172.21.15.32:0/870199836 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f74e006be30 0x7f74e006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:59.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.387+0000 7f74f1ffb640 1 -- 172.21.15.32:0/870199836 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5227+0+0 (secure 0 0 0) 0x7f74e408acd0 con 0x7f74f415cb00 2024-04-24T23:02:59.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.387+0000 7f74f9256640 1 --2- 172.21.15.32:0/870199836 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f74e006be30 0x7f74e006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:59.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.388+0000 7f74f9256640 1 --2- 172.21.15.32:0/870199836 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f74e006be30 0x7f74e006e2f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f74ec034000 tx=0x7f74ec02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:59.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.392+0000 7f74f1ffb640 1 -- 172.21.15.32:0/870199836 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f74e40597c0 con 0x7f74f415cb00 2024-04-24T23:02:59.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:02:59.574+0000 7f74fb4ba640 1 -- 172.21.15.32:0/870199836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f74f406c000 con 0x7f74f415cb00 2024-04-24T23:03:00.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.256+0000 7f74f1ffb640 1 -- 172.21.15.32:0/870199836 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v85) ==== 170+0+0 (secure 0 0 0) 0x7f74e40590c0 con 0x7f74f415cb00 2024-04-24T23:03:00.259 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2024-04-24T23:03:00.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.259+0000 7f74fb4ba640 1 -- 172.21.15.32:0/870199836 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f74e006be30 msgr2=0x7f74e006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:00.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.259+0000 7f74fb4ba640 1 --2- 172.21.15.32:0/870199836 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f74e006be30 0x7f74e006e2f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f74ec034000 tx=0x7f74ec02e000 comp rx=0 tx=0).stop 2024-04-24T23:03:00.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.259+0000 7f74fb4ba640 1 -- 172.21.15.32:0/870199836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74f415cb00 msgr2=0x7f74f415cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:00.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.259+0000 7f74fb4ba640 1 --2- 172.21.15.32:0/870199836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74f415cb00 0x7f74f415cfe0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f74e400c7c0 tx=0x7f74e400cc80 comp rx=0 tx=0).stop 2024-04-24T23:03:00.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.259+0000 7f74fb4ba640 1 -- 172.21.15.32:0/870199836 shutdown_connections 2024-04-24T23:03:00.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.259+0000 7f74fb4ba640 1 --2- 172.21.15.32:0/870199836 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f74e006be30 0x7f74e006e2f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:00.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.259+0000 7f74fb4ba640 1 --2- 172.21.15.32:0/870199836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74f415cb00 0x7f74f415cfe0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:00.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.259+0000 7f74fb4ba640 1 --2- 172.21.15.32:0/870199836 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f74f415b380 0x7f74f41572c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:00.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.259+0000 7f74fb4ba640 1 --2- 172.21.15.32:0/870199836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f74f40ceb40 0x7f74f41500a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:00.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.259+0000 7f74fb4ba640 1 -- 172.21.15.32:0/870199836 >> 172.21.15.32:0/870199836 conn(0x7f74f406b110 msgr2=0x7f74f406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:00.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.259+0000 7f74fb4ba640 1 -- 172.21.15.32:0/870199836 shutdown_connections 2024-04-24T23:03:00.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.259+0000 7f74fb4ba640 1 -- 172.21.15.32:0/870199836 wait complete. 2024-04-24T23:03:00.286 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-24T23:03:00.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.392+0000 7fd3f6bb7640 1 Processor -- start 2024-04-24T23:03:00.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.393+0000 7fd3f6bb7640 1 -- start start 2024-04-24T23:03:00.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.393+0000 7fd3f6bb7640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd3f006b820 0x7fd3f00d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:00.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.393+0000 7fd3f6bb7640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3f00d3170 con 0x7fd3f015e9b0 2024-04-24T23:03:00.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.393+0000 7fd3f6bb7640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3f00d3450 con 0x7fd3f00d4210 2024-04-24T23:03:00.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.393+0000 7fd3f6bb7640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd3f00d3730 con 0x7fd3f006b820 2024-04-24T23:03:00.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.393+0000 7fd3f4953640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd3f015e9b0 0x7fd3f006b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57576/0 (socket says 172.21.15.32:57576) 2024-04-24T23:03:00.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.394+0000 7fd3f4953640 1 -- 172.21.15.32:0/3001906010 learned_addr learned my addr 172.21.15.32:0/3001906010 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:03:00.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.394+0000 7fd3effff640 1 --2- 172.21.15.32:0/3001906010 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd3f006b820 0x7fd3f00d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:00.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.394+0000 7fd3ef7fe640 1 -- 172.21.15.32:0/3001906010 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2934618576 0 0) 0x7fd3f00d3170 con 0x7fd3f015e9b0 2024-04-24T23:03:00.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.394+0000 7fd3ef7fe640 1 -- 172.21.15.32:0/3001906010 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd3dc003520 con 0x7fd3f015e9b0 2024-04-24T23:03:00.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.394+0000 7fd3ef7fe640 1 -- 172.21.15.32:0/3001906010 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3300100606 0 0) 0x7fd3f00d3450 con 0x7fd3f00d4210 2024-04-24T23:03:00.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.394+0000 7fd3ef7fe640 1 -- 172.21.15.32:0/3001906010 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd3f00d3170 con 0x7fd3f00d4210 2024-04-24T23:03:00.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.394+0000 7fd3effff640 1 -- 172.21.15.32:0/3001906010 >> v1:172.21.15.121:6789/0 conn(0x7fd3f00d4210 legacy=0x7fd3f00b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:00.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.394+0000 7fd3effff640 1 -- 172.21.15.32:0/3001906010 >> v1:172.21.15.32:6789/0 conn(0x7fd3f015e9b0 legacy=0x7fd3f006b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:00.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.394+0000 7fd3ef7fe640 1 -- 172.21.15.32:0/3001906010 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1851933755 0 0) 0x7fd3dc003520 con 0x7fd3f015e9b0 2024-04-24T23:03:00.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.394+0000 7fd3effff640 1 -- 172.21.15.32:0/3001906010 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3f00d3a10 con 0x7fd3f006b820 2024-04-24T23:03:00.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.395+0000 7fd3effff640 1 --2- 172.21.15.32:0/3001906010 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd3f006b820 0x7fd3f00d2b90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd3e00097d0 tx=0x7fd3e002a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4adac19f1451a857 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:00.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.395+0000 7fd3ef7fe640 1 -- 172.21.15.32:0/3001906010 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e0037070 con 0x7fd3f006b820 2024-04-24T23:03:00.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.395+0000 7fd3ef7fe640 1 -- 172.21.15.32:0/3001906010 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3e002ac30 con 0x7fd3f006b820 2024-04-24T23:03:00.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.395+0000 7fd3ef7fe640 1 -- 172.21.15.32:0/3001906010 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e80019f0 con 0x7fd3f006b820 2024-04-24T23:03:00.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.396+0000 7fd3f6bb7640 1 -- 172.21.15.32:0/3001906010 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd3f006b820 msgr2=0x7fd3f00d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:00.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.396+0000 7fd3f6bb7640 1 --2- 172.21.15.32:0/3001906010 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd3f006b820 0x7fd3f00d2b90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd3e00097d0 tx=0x7fd3e002a1a0 comp rx=0 tx=0).stop 2024-04-24T23:03:00.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.396+0000 7fd3f6bb7640 1 -- 172.21.15.32:0/3001906010 shutdown_connections 2024-04-24T23:03:00.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.396+0000 7fd3f6bb7640 1 --2- 172.21.15.32:0/3001906010 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd3f006b820 0x7fd3f00d2b90 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:00.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.396+0000 7fd3f6bb7640 1 -- 172.21.15.32:0/3001906010 >> 172.21.15.32:0/3001906010 conn(0x7fd3f00d11d0 msgr2=0x7fd3f00d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:00.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.396+0000 7fd3f6bb7640 1 -- 172.21.15.32:0/3001906010 shutdown_connections 2024-04-24T23:03:00.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.396+0000 7fd3f6bb7640 1 -- 172.21.15.32:0/3001906010 wait complete. 2024-04-24T23:03:00.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.397+0000 7fd3f6bb7640 1 Processor -- start 2024-04-24T23:03:00.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.397+0000 7fd3f6bb7640 1 -- start start 2024-04-24T23:03:00.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.398+0000 7fd3f6bb7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3f006b820 0x7fd3f014c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:00.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.398+0000 7fd3f6bb7640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3f00d4210 0x7fd3f01540e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:00.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.398+0000 7fd3f6bb7640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd3f015e9b0 0x7fd3f0154620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:00.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.398+0000 7fd3f6bb7640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f00ba0e0 con 0x7fd3f006b820 2024-04-24T23:03:00.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.398+0000 7fd3f6bb7640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f00d2400 con 0x7fd3f00d4210 2024-04-24T23:03:00.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.398+0000 7fd3f6bb7640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd3f00ba260 con 0x7fd3f015e9b0 2024-04-24T23:03:00.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.398+0000 7fd3f4953640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3f006b820 0x7fd3f014c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:00.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.398+0000 7fd3f4953640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3f006b820 0x7fd3f014c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38338/0 (socket says 172.21.15.32:38338) 2024-04-24T23:03:00.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.398+0000 7fd3f4953640 1 -- 172.21.15.32:0/2902591950 learned_addr learned my addr 172.21.15.32:0/2902591950 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:00.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.398+0000 7fd3effff640 1 --2- 172.21.15.32:0/2902591950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3f00d4210 0x7fd3f01540e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:00.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.398+0000 7fd3f5154640 1 --2- 172.21.15.32:0/2902591950 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd3f015e9b0 0x7fd3f0154620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:00.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.398+0000 7fd3f4953640 1 -- 172.21.15.32:0/2902591950 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd3f015e9b0 msgr2=0x7fd3f0154620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:00.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.398+0000 7fd3f4953640 1 --2- 172.21.15.32:0/2902591950 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd3f015e9b0 0x7fd3f0154620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:00.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.398+0000 7fd3f4953640 1 -- 172.21.15.32:0/2902591950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3f00d4210 msgr2=0x7fd3f01540e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:00.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.398+0000 7fd3f4953640 1 --2- 172.21.15.32:0/2902591950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3f00d4210 0x7fd3f01540e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:00.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.398+0000 7fd3f4953640 1 -- 172.21.15.32:0/2902591950 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3f015f6d0 con 0x7fd3f006b820 2024-04-24T23:03:00.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.398+0000 7fd3f5154640 1 --2- 172.21.15.32:0/2902591950 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd3f015e9b0 0x7fd3f0154620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:03:00.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.398+0000 7fd3f4953640 1 --2- 172.21.15.32:0/2902591950 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3f006b820 0x7fd3f014c840 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fd3e800cc20 tx=0x7fd3e800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:00.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.398+0000 7fd3ed7fa640 1 -- 172.21.15.32:0/2902591950 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e8017070 con 0x7fd3f006b820 2024-04-24T23:03:00.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.399+0000 7fd3ed7fa640 1 -- 172.21.15.32:0/2902591950 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3e800ce30 con 0x7fd3f006b820 2024-04-24T23:03:00.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.399+0000 7fd3f6bb7640 1 -- 172.21.15.32:0/2902591950 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3f015b700 con 0x7fd3f006b820 2024-04-24T23:03:00.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.399+0000 7fd3ed7fa640 1 -- 172.21.15.32:0/2902591950 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e8012620 con 0x7fd3f006b820 2024-04-24T23:03:00.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.399+0000 7fd3f6bb7640 1 -- 172.21.15.32:0/2902591950 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3f015bcc0 con 0x7fd3f006b820 2024-04-24T23:03:00.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.399+0000 7fd3f6bb7640 1 -- 172.21.15.32:0/2902591950 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3f0068430 con 0x7fd3f006b820 2024-04-24T23:03:00.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.400+0000 7fd3ed7fa640 1 -- 172.21.15.32:0/2902591950 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd3e801a070 con 0x7fd3f006b820 2024-04-24T23:03:00.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.400+0000 7fd3ed7fa640 1 --2- 172.21.15.32:0/2902591950 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd3d006be30 0x7fd3d006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:00.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.401+0000 7fd3ed7fa640 1 -- 172.21.15.32:0/2902591950 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5609+0+0 (secure 0 0 0) 0x7fd3e808ad20 con 0x7fd3f006b820 2024-04-24T23:03:00.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.401+0000 7fd3effff640 1 --2- 172.21.15.32:0/2902591950 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd3d006be30 0x7fd3d006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:00.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.401+0000 7fd3effff640 1 --2- 172.21.15.32:0/2902591950 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd3d006be30 0x7fd3d006e2f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd3e0034000 tx=0x7fd3e002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:00.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.406+0000 7fd3ed7fa640 1 -- 172.21.15.32:0/2902591950 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd3e8059040 con 0x7fd3f006b820 2024-04-24T23:03:00.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:00.628+0000 7fd3f6bb7640 1 -- 172.21.15.32:0/2902591950 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fd3f00718f0 con 0x7fd3f006b820 2024-04-24T23:03:01.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.287+0000 7fd3ed7fa640 1 -- 172.21.15.32:0/2902591950 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v33) ==== 174+0+0 (secure 0 0 0) 0x7fd3e805d340 con 0x7fd3f006b820 2024-04-24T23:03:01.293 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 10 and data pool 11 2024-04-24T23:03:01.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.296+0000 7fd3f6bb7640 1 -- 172.21.15.32:0/2902591950 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd3d006be30 msgr2=0x7fd3d006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:01.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.296+0000 7fd3f6bb7640 1 --2- 172.21.15.32:0/2902591950 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd3d006be30 0x7fd3d006e2f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd3e0034000 tx=0x7fd3e002e000 comp rx=0 tx=0).stop 2024-04-24T23:03:01.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.296+0000 7fd3f6bb7640 1 -- 172.21.15.32:0/2902591950 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3f006b820 msgr2=0x7fd3f014c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:01.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.296+0000 7fd3f6bb7640 1 --2- 172.21.15.32:0/2902591950 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3f006b820 0x7fd3f014c840 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fd3e800cc20 tx=0x7fd3e800a430 comp rx=0 tx=0).stop 2024-04-24T23:03:01.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.296+0000 7fd3f6bb7640 1 -- 172.21.15.32:0/2902591950 shutdown_connections 2024-04-24T23:03:01.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.297+0000 7fd3f6bb7640 1 --2- 172.21.15.32:0/2902591950 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd3d006be30 0x7fd3d006e2f0 secure :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd3e0034000 tx=0x7fd3e002e000 comp rx=0 tx=0).stop 2024-04-24T23:03:01.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.297+0000 7fd3f6bb7640 1 --2- 172.21.15.32:0/2902591950 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd3f015e9b0 0x7fd3f0154620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:01.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.297+0000 7fd3f6bb7640 1 --2- 172.21.15.32:0/2902591950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3f00d4210 0x7fd3f01540e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:01.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.297+0000 7fd3f6bb7640 1 --2- 172.21.15.32:0/2902591950 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3f006b820 0x7fd3f014c840 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:01.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.297+0000 7fd3f6bb7640 1 -- 172.21.15.32:0/2902591950 >> 172.21.15.32:0/2902591950 conn(0x7fd3f00d11d0 msgr2=0x7fd3f0155ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:01.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.297+0000 7fd3f6bb7640 1 -- 172.21.15.32:0/2902591950 shutdown_connections 2024-04-24T23:03:01.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.297+0000 7fd3f6bb7640 1 -- 172.21.15.32:0/2902591950 wait complete. 2024-04-24T23:03:01.317 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:03:01.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.421+0000 7f16672b3640 1 Processor -- start 2024-04-24T23:03:01.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.421+0000 7f16672b3640 1 -- start start 2024-04-24T23:03:01.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.422+0000 7f16672b3640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1660074210 0x7f166006c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:01.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.422+0000 7f16672b3640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f16600b0b80 0x7f16600745f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:01.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.422+0000 7f16672b3640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16600ae140 0x7f16600b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:01.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.422+0000 7f16672b3640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f16600b1c20 con 0x7f1660074210 2024-04-24T23:03:01.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.422+0000 7f16672b3640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f16600b0770 con 0x7f16600b0b80 2024-04-24T23:03:01.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.422+0000 7f16672b3640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f1660074b30 con 0x7f16600ae140 2024-04-24T23:03:01.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.422+0000 7f166504f640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1660074210 0x7f166006c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:01.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.422+0000 7f166504f640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1660074210 0x7f166006c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38342/0 (socket says 172.21.15.32:38342) 2024-04-24T23:03:01.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.422+0000 7f166504f640 1 -- 172.21.15.32:0/2725208428 learned_addr learned my addr 172.21.15.32:0/2725208428 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:01.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.422+0000 7f1665850640 1 --2- 172.21.15.32:0/2725208428 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16600ae140 0x7f16600b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:01.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.422+0000 7f166504f640 1 -- 172.21.15.32:0/2725208428 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16600ae140 msgr2=0x7f16600b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:01.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.422+0000 7f166484e640 1 --2- 172.21.15.32:0/2725208428 >> v2:172.21.15.121:3300/0 conn(0x7f16600b0b80 0x7f16600745f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:01.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.422+0000 7f166504f640 1 --2- 172.21.15.32:0/2725208428 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16600ae140 0x7f16600b20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:01.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.422+0000 7f166504f640 1 -- 172.21.15.32:0/2725208428 >> v2:172.21.15.121:3300/0 conn(0x7f16600b0b80 msgr2=0x7f16600745f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:01.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.422+0000 7f166504f640 1 --2- 172.21.15.32:0/2725208428 >> v2:172.21.15.121:3300/0 conn(0x7f16600b0b80 0x7f16600745f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:01.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.422+0000 7f166504f640 1 -- 172.21.15.32:0/2725208428 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16600b1010 con 0x7f1660074210 2024-04-24T23:03:01.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.423+0000 7f1665850640 1 --2- 172.21.15.32:0/2725208428 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16600ae140 0x7f16600b20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:03:01.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.423+0000 7f166484e640 1 --2- 172.21.15.32:0/2725208428 >> v2:172.21.15.121:3300/0 conn(0x7f16600b0b80 0x7f16600745f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:03:01.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.423+0000 7f166504f640 1 --2- 172.21.15.32:0/2725208428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1660074210 0x7f166006c480 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f16600722d0 tx=0x7f165c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=905f5f440953b54a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:01.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.423+0000 7f1657fff640 1 -- 172.21.15.32:0/2725208428 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165c037070 con 0x7f1660074210 2024-04-24T23:03:01.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.423+0000 7f1657fff640 1 -- 172.21.15.32:0/2725208428 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f165c032440 con 0x7f1660074210 2024-04-24T23:03:01.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.423+0000 7f1657fff640 1 -- 172.21.15.32:0/2725208428 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165c032810 con 0x7f1660074210 2024-04-24T23:03:01.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.424+0000 7f16672b3640 1 -- 172.21.15.32:0/2725208428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1660074210 msgr2=0x7f166006c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:01.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.424+0000 7f16672b3640 1 --2- 172.21.15.32:0/2725208428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1660074210 0x7f166006c480 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f16600722d0 tx=0x7f165c02a1e0 comp rx=0 tx=0).stop 2024-04-24T23:03:01.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.424+0000 7f16672b3640 1 -- 172.21.15.32:0/2725208428 shutdown_connections 2024-04-24T23:03:01.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.424+0000 7f16672b3640 1 --2- 172.21.15.32:0/2725208428 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16600ae140 0x7f16600b20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:01.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.424+0000 7f16672b3640 1 --2- 172.21.15.32:0/2725208428 >> v2:172.21.15.121:3300/0 conn(0x7f16600b0b80 0x7f16600745f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:01.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.424+0000 7f16672b3640 1 --2- 172.21.15.32:0/2725208428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1660074210 0x7f166006c480 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:01.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.424+0000 7f16672b3640 1 -- 172.21.15.32:0/2725208428 >> 172.21.15.32:0/2725208428 conn(0x7f16600b2e60 msgr2=0x7f16600b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:01.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.424+0000 7f16672b3640 1 -- 172.21.15.32:0/2725208428 shutdown_connections 2024-04-24T23:03:01.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.424+0000 7f16672b3640 1 -- 172.21.15.32:0/2725208428 wait complete. 2024-04-24T23:03:01.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.425+0000 7f16672b3640 1 Processor -- start 2024-04-24T23:03:01.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.425+0000 7f16672b3640 1 -- start start 2024-04-24T23:03:01.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.426+0000 7f16672b3640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1660074210 0x7f166015e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:01.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.426+0000 7f16672b3640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16600ae140 0x7f16600c7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:01.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.426+0000 7f16672b3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f166015d9b0 0x7f1660162600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:01.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.426+0000 7f16672b3640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f16600b18b0 con 0x7f166015d9b0 2024-04-24T23:03:01.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.426+0000 7f16672b3640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f16600b15d0 con 0x7f1660074210 2024-04-24T23:03:01.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.426+0000 7f16672b3640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f16600b3ba0 con 0x7f16600ae140 2024-04-24T23:03:01.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.426+0000 7f1665850640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f166015d9b0 0x7f1660162600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:01.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.426+0000 7f1665850640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f166015d9b0 0x7f1660162600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38346/0 (socket says 172.21.15.32:38346) 2024-04-24T23:03:01.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.426+0000 7f1665850640 1 -- 172.21.15.32:0/44547509 learned_addr learned my addr 172.21.15.32:0/44547509 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:01.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.426+0000 7f166484e640 1 --2- 172.21.15.32:0/44547509 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16600ae140 0x7f16600c7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:01.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.426+0000 7f166504f640 1 --2- 172.21.15.32:0/44547509 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1660074210 0x7f166015e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:01.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.426+0000 7f1665850640 1 -- 172.21.15.32:0/44547509 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16600ae140 msgr2=0x7f16600c7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:01.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.426+0000 7f1665850640 1 --2- 172.21.15.32:0/44547509 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16600ae140 0x7f16600c7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:01.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.426+0000 7f1665850640 1 -- 172.21.15.32:0/44547509 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1660074210 msgr2=0x7f166015e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:01.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.426+0000 7f1665850640 1 --2- 172.21.15.32:0/44547509 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1660074210 0x7f166015e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:01.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.426+0000 7f1665850640 1 -- 172.21.15.32:0/44547509 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16600cb9c0 con 0x7f166015d9b0 2024-04-24T23:03:01.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.426+0000 7f166504f640 1 --2- 172.21.15.32:0/44547509 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1660074210 0x7f166015e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:03:01.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.427+0000 7f1665850640 1 --2- 172.21.15.32:0/44547509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f166015d9b0 0x7f1660162600 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f165800cef0 tx=0x7f165800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:01.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.427+0000 7f1655ffb640 1 -- 172.21.15.32:0/44547509 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165801a070 con 0x7f166015d9b0 2024-04-24T23:03:01.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.427+0000 7f1655ffb640 1 -- 172.21.15.32:0/44547509 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f165800d840 con 0x7f166015d9b0 2024-04-24T23:03:01.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.427+0000 7f16672b3640 1 -- 172.21.15.32:0/44547509 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16600cbca0 con 0x7f166015d9b0 2024-04-24T23:03:01.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.427+0000 7f1655ffb640 1 -- 172.21.15.32:0/44547509 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165800dc30 con 0x7f166015d9b0 2024-04-24T23:03:01.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.427+0000 7f16672b3640 1 -- 172.21.15.32:0/44547509 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16600cc260 con 0x7f166015d9b0 2024-04-24T23:03:01.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.428+0000 7f16672b3640 1 -- 172.21.15.32:0/44547509 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1660068430 con 0x7f166015d9b0 2024-04-24T23:03:01.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.429+0000 7f1655ffb640 1 -- 172.21.15.32:0/44547509 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f165801d070 con 0x7f166015d9b0 2024-04-24T23:03:01.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.429+0000 7f1655ffb640 1 --2- 172.21.15.32:0/44547509 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f164406be30 0x7f164406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:01.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.429+0000 7f1655ffb640 1 -- 172.21.15.32:0/44547509 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5709+0+0 (secure 0 0 0) 0x7f165808b5f0 con 0x7f166015d9b0 2024-04-24T23:03:01.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.429+0000 7f166504f640 1 --2- 172.21.15.32:0/44547509 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f164406be30 0x7f164406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:01.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.430+0000 7f166504f640 1 --2- 172.21.15.32:0/44547509 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f164406be30 0x7f164406e2f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f165c034000 tx=0x7f165c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:01.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.433+0000 7f1655ffb640 1 -- 172.21.15.32:0/44547509 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1658059950 con 0x7f166015d9b0 2024-04-24T23:03:01.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.616+0000 7f16672b3640 1 -- 172.21.15.32:0/44547509 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f166006b220 con 0x7f166015d9b0 2024-04-24T23:03:01.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.617+0000 7f1655ffb640 1 -- 172.21.15.32:0/44547509 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+16633 (secure 0 0 0) 0x7f165805dc50 con 0x7f166015d9b0 2024-04-24T23:03:01.619 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:03:01.619 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":86,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:03:00.630618+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:02:58.930222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-24T23:02:59.607070+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:03:01.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.619+0000 7f16672b3640 1 -- 172.21.15.32:0/44547509 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f164406be30 msgr2=0x7f164406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:01.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.619+0000 7f16672b3640 1 --2- 172.21.15.32:0/44547509 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f164406be30 0x7f164406e2f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f165c034000 tx=0x7f165c02e000 comp rx=0 tx=0).stop 2024-04-24T23:03:01.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.619+0000 7f16672b3640 1 -- 172.21.15.32:0/44547509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f166015d9b0 msgr2=0x7f1660162600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:01.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.619+0000 7f16672b3640 1 --2- 172.21.15.32:0/44547509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f166015d9b0 0x7f1660162600 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f165800cef0 tx=0x7f165800a710 comp rx=0 tx=0).stop 2024-04-24T23:03:01.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.619+0000 7f16672b3640 1 -- 172.21.15.32:0/44547509 shutdown_connections 2024-04-24T23:03:01.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.619+0000 7f16672b3640 1 --2- 172.21.15.32:0/44547509 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f164406be30 0x7f164406e2f0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:01.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.619+0000 7f16672b3640 1 --2- 172.21.15.32:0/44547509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f166015d9b0 0x7f1660162600 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:01.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.619+0000 7f16672b3640 1 --2- 172.21.15.32:0/44547509 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16600ae140 0x7f16600c7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:01.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.619+0000 7f16672b3640 1 --2- 172.21.15.32:0/44547509 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1660074210 0x7f166015e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:01.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.619+0000 7f16672b3640 1 -- 172.21.15.32:0/44547509 >> 172.21.15.32:0/44547509 conn(0x7f16600b2e60 msgr2=0x7f166006d570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:01.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.619+0000 7f16672b3640 1 -- 172.21.15.32:0/44547509 shutdown_connections 2024-04-24T23:03:01.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.619+0000 7f16672b3640 1 -- 172.21.15.32:0/44547509 wait complete. 2024-04-24T23:03:01.631 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:03:01.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.730+0000 7f0ff3b5a640 1 Processor -- start 2024-04-24T23:03:01.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.730+0000 7f0ff3b5a640 1 -- start start 2024-04-24T23:03:01.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.731+0000 7f0ff3b5a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fec0af890 0x7f0fec1670f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:01.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.731+0000 7f0ff3b5a640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fec0b6430 con 0x7f0fec0b44d0 2024-04-24T23:03:01.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.731+0000 7f0ff3b5a640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fec156f80 con 0x7f0fec06c5f0 2024-04-24T23:03:01.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.731+0000 7f0ff3b5a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f0fec0b6150 con 0x7f0fec0af890 2024-04-24T23:03:01.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.731+0000 7f0ff10f5640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0fec0b44d0 0x7f0fec0b5890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57580/0 (socket says 172.21.15.32:57580) 2024-04-24T23:03:01.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.731+0000 7f0ff10f5640 1 -- 172.21.15.32:0/1650392898 learned_addr learned my addr 172.21.15.32:0/1650392898 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:03:01.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.731+0000 7f0ff20f7640 1 --2- 172.21.15.32:0/1650392898 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fec0af890 0x7f0fec1670f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:01.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.731+0000 7f0ff08f4640 1 -- 172.21.15.32:0/1650392898 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3603428826 0 0) 0x7f0fec0b6430 con 0x7f0fec0b44d0 2024-04-24T23:03:01.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.731+0000 7f0ff08f4640 1 -- 172.21.15.32:0/1650392898 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0fd4003520 con 0x7f0fec0b44d0 2024-04-24T23:03:01.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.732+0000 7f0ff08f4640 1 -- 172.21.15.32:0/1650392898 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2438007892 0 0) 0x7f0fec156f80 con 0x7f0fec06c5f0 2024-04-24T23:03:01.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.732+0000 7f0ff20f7640 1 -- 172.21.15.32:0/1650392898 >> v1:172.21.15.121:6789/0 conn(0x7f0fec06c5f0 legacy=0x7f0fec06c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:01.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.732+0000 7f0ff20f7640 1 -- 172.21.15.32:0/1650392898 >> v1:172.21.15.32:6789/0 conn(0x7f0fec0b44d0 legacy=0x7f0fec0b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:01.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.732+0000 7f0ff20f7640 1 -- 172.21.15.32:0/1650392898 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fec0b39f0 con 0x7f0fec0af890 2024-04-24T23:03:01.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.732+0000 7f0ff20f7640 1 --2- 172.21.15.32:0/1650392898 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fec0af890 0x7f0fec1670f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0fe40097d0 tx=0x7f0fe402a0e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5146e024c35db28 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:01.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.733+0000 7f0ff08f4640 1 -- 172.21.15.32:0/1650392898 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe4037070 con 0x7f0fec0af890 2024-04-24T23:03:01.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.733+0000 7f0ff08f4640 1 -- 172.21.15.32:0/1650392898 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fe402ab70 con 0x7f0fec0af890 2024-04-24T23:03:01.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.733+0000 7f0ff08f4640 1 -- 172.21.15.32:0/1650392898 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe0001690 con 0x7f0fec0af890 2024-04-24T23:03:01.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.733+0000 7f0ff3b5a640 1 -- 172.21.15.32:0/1650392898 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fec0af890 msgr2=0x7f0fec1670f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:01.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.733+0000 7f0ff3b5a640 1 --2- 172.21.15.32:0/1650392898 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fec0af890 0x7f0fec1670f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0fe40097d0 tx=0x7f0fe402a0e0 comp rx=0 tx=0).stop 2024-04-24T23:03:01.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.733+0000 7f0ff3b5a640 1 -- 172.21.15.32:0/1650392898 shutdown_connections 2024-04-24T23:03:01.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.733+0000 7f0ff3b5a640 1 --2- 172.21.15.32:0/1650392898 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fec0af890 0x7f0fec1670f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:01.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.733+0000 7f0ff3b5a640 1 -- 172.21.15.32:0/1650392898 >> 172.21.15.32:0/1650392898 conn(0x7f0fec06b110 msgr2=0x7f0fec06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:01.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.733+0000 7f0ff3b5a640 1 -- 172.21.15.32:0/1650392898 shutdown_connections 2024-04-24T23:03:01.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.733+0000 7f0ff3b5a640 1 -- 172.21.15.32:0/1650392898 wait complete. 2024-04-24T23:03:01.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.734+0000 7f0ff3b5a640 1 Processor -- start 2024-04-24T23:03:01.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.734+0000 7f0ff3b5a640 1 -- start start 2024-04-24T23:03:01.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.735+0000 7f0ff3b5a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fec06c5f0 0x7f0fec154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:01.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.735+0000 7f0ff3b5a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fec0af890 0x7f0fec1517b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:01.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.735+0000 7f0ff3b5a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fec0b44d0 0x7f0fec0ab300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:01.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.735+0000 7f0ff3b5a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0fec167630 con 0x7f0fec0b44d0 2024-04-24T23:03:01.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.735+0000 7f0ff3b5a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0fec0b3fb0 con 0x7f0fec0af890 2024-04-24T23:03:01.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.735+0000 7f0ff3b5a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f0fec1677b0 con 0x7f0fec06c5f0 2024-04-24T23:03:01.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.735+0000 7f0ff20f7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fec0b44d0 0x7f0fec0ab300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:01.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.735+0000 7f0ff20f7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fec0b44d0 0x7f0fec0ab300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38354/0 (socket says 172.21.15.32:38354) 2024-04-24T23:03:01.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.735+0000 7f0ff20f7640 1 -- 172.21.15.32:0/2523011199 learned_addr learned my addr 172.21.15.32:0/2523011199 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:01.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.735+0000 7f0ff18f6640 1 --2- 172.21.15.32:0/2523011199 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fec06c5f0 0x7f0fec154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:01.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.735+0000 7f0ff10f5640 1 --2- 172.21.15.32:0/2523011199 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fec0af890 0x7f0fec1517b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:01.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.735+0000 7f0ff20f7640 1 -- 172.21.15.32:0/2523011199 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fec06c5f0 msgr2=0x7f0fec154180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:01.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.735+0000 7f0ff20f7640 1 --2- 172.21.15.32:0/2523011199 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fec06c5f0 0x7f0fec154180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:01.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.735+0000 7f0ff20f7640 1 -- 172.21.15.32:0/2523011199 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fec0af890 msgr2=0x7f0fec1517b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:01.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.735+0000 7f0ff20f7640 1 --2- 172.21.15.32:0/2523011199 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fec0af890 0x7f0fec1517b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:01.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.736+0000 7f0ff20f7640 1 -- 172.21.15.32:0/2523011199 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fec0aa540 con 0x7f0fec0b44d0 2024-04-24T23:03:01.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.736+0000 7f0ff18f6640 1 --2- 172.21.15.32:0/2523011199 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fec06c5f0 0x7f0fec154180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:03:01.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.736+0000 7f0ff10f5640 1 --2- 172.21.15.32:0/2523011199 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fec0af890 0x7f0fec1517b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:03:01.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.736+0000 7f0ff20f7640 1 --2- 172.21.15.32:0/2523011199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fec0b44d0 0x7f0fec0ab300 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f0fe40097d0 tx=0x7f0fe40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:01.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.736+0000 7f0fea7fc640 1 -- 172.21.15.32:0/2523011199 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe4037040 con 0x7f0fec0b44d0 2024-04-24T23:03:01.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.736+0000 7f0fea7fc640 1 -- 172.21.15.32:0/2523011199 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fe4004030 con 0x7f0fec0b44d0 2024-04-24T23:03:01.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.736+0000 7f0ff3b5a640 1 -- 172.21.15.32:0/2523011199 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fec0aa820 con 0x7f0fec0b44d0 2024-04-24T23:03:01.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.736+0000 7f0fea7fc640 1 -- 172.21.15.32:0/2523011199 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe4006e10 con 0x7f0fec0b44d0 2024-04-24T23:03:01.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.736+0000 7f0ff3b5a640 1 -- 172.21.15.32:0/2523011199 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fec0aade0 con 0x7f0fec0b44d0 2024-04-24T23:03:01.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.737+0000 7f0ff3b5a640 1 -- 172.21.15.32:0/2523011199 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fec068430 con 0x7f0fec0b44d0 2024-04-24T23:03:01.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.738+0000 7f0fea7fc640 1 -- 172.21.15.32:0/2523011199 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0fe403a070 con 0x7f0fec0b44d0 2024-04-24T23:03:01.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.738+0000 7f0fea7fc640 1 --2- 172.21.15.32:0/2523011199 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0fd806be30 0x7f0fd806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:01.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.738+0000 7f0fea7fc640 1 -- 172.21.15.32:0/2523011199 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5709+0+0 (secure 0 0 0) 0x7f0fe40ab290 con 0x7f0fec0b44d0 2024-04-24T23:03:01.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.738+0000 7f0ff18f6640 1 --2- 172.21.15.32:0/2523011199 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0fd806be30 0x7f0fd806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:01.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.739+0000 7f0ff18f6640 1 --2- 172.21.15.32:0/2523011199 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0fd806be30 0x7f0fd806e2f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc0012e0 tx=0x7f0fdc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:01.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.742+0000 7f0fea7fc640 1 -- 172.21.15.32:0/2523011199 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0fe4041020 con 0x7f0fec0b44d0 2024-04-24T23:03:01.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.927+0000 7f0ff3b5a640 1 -- 172.21.15.32:0/2523011199 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0fec151000 con 0x7f0fec0b44d0 2024-04-24T23:03:01.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.928+0000 7f0fea7fc640 1 -- 172.21.15.32:0/2523011199 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+16633 (secure 0 0 0) 0x7f0fe4079cc0 con 0x7f0fec0b44d0 2024-04-24T23:03:01.930 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:03:01.931 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":86,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:03:00.630618+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:02:58.930222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-24T23:02:59.607070+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:03:01.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.930+0000 7f0ff3b5a640 1 -- 172.21.15.32:0/2523011199 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0fd806be30 msgr2=0x7f0fd806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:01.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.930+0000 7f0ff3b5a640 1 --2- 172.21.15.32:0/2523011199 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0fd806be30 0x7f0fd806e2f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc0012e0 tx=0x7f0fdc009040 comp rx=0 tx=0).stop 2024-04-24T23:03:01.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.930+0000 7f0ff3b5a640 1 -- 172.21.15.32:0/2523011199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fec0b44d0 msgr2=0x7f0fec0ab300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:01.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.930+0000 7f0ff3b5a640 1 --2- 172.21.15.32:0/2523011199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fec0b44d0 0x7f0fec0ab300 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f0fe40097d0 tx=0x7f0fe40046c0 comp rx=0 tx=0).stop 2024-04-24T23:03:01.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.930+0000 7f0ff3b5a640 1 -- 172.21.15.32:0/2523011199 shutdown_connections 2024-04-24T23:03:01.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.930+0000 7f0ff3b5a640 1 --2- 172.21.15.32:0/2523011199 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0fd806be30 0x7f0fd806e2f0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:01.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.930+0000 7f0ff3b5a640 1 --2- 172.21.15.32:0/2523011199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fec0b44d0 0x7f0fec0ab300 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:01.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.930+0000 7f0ff3b5a640 1 --2- 172.21.15.32:0/2523011199 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fec0af890 0x7f0fec1517b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:01.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.930+0000 7f0ff3b5a640 1 --2- 172.21.15.32:0/2523011199 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fec06c5f0 0x7f0fec154180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:01.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.930+0000 7f0ff3b5a640 1 -- 172.21.15.32:0/2523011199 >> 172.21.15.32:0/2523011199 conn(0x7f0fec06b110 msgr2=0x7f0fec155a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:01.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.930+0000 7f0ff3b5a640 1 -- 172.21.15.32:0/2523011199 shutdown_connections 2024-04-24T23:03:01.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:01.930+0000 7f0ff3b5a640 1 -- 172.21.15.32:0/2523011199 wait complete. 2024-04-24T23:03:01.943 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-24T23:03:02.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.045+0000 7f8d71c27640 1 Processor -- start 2024-04-24T23:03:02.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.045+0000 7f8d71c27640 1 -- start start 2024-04-24T23:03:02.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.046+0000 7f8d71c27640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8d6c162320 0x7f8d6c0ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:02.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.046+0000 7f8d71c27640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d6c162e10 0x7f8d6c15ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:02.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.046+0000 7f8d71c27640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d6c0cd270 con 0x7f8d6c0b5af0 2024-04-24T23:03:02.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.046+0000 7f8d71c27640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f8d6c0b5660 con 0x7f8d6c162320 2024-04-24T23:03:02.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.046+0000 7f8d71c27640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f8d6c071610 con 0x7f8d6c162e10 2024-04-24T23:03:02.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.046+0000 7f8d6b7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8d6c162320 0x7f8d6c0ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:02.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.046+0000 7f8d6affd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8d6c0b5af0 0x7f8d6c162700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57594/0 (socket says 172.21.15.32:57594) 2024-04-24T23:03:02.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.046+0000 7f8d6affd640 1 -- 172.21.15.32:0/2702004843 learned_addr learned my addr 172.21.15.32:0/2702004843 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:03:02.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.046+0000 7f8d6bfff640 1 --2- 172.21.15.32:0/2702004843 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d6c162e10 0x7f8d6c15ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:02.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.047+0000 7f8d6b7fe640 1 -- 172.21.15.32:0/2702004843 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d6c162e10 msgr2=0x7f8d6c15ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:02.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.047+0000 7f8d6b7fe640 1 --2- 172.21.15.32:0/2702004843 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d6c162e10 0x7f8d6c15ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:02.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.047+0000 7f8d6b7fe640 1 -- 172.21.15.32:0/2702004843 >> v1:172.21.15.32:6789/0 conn(0x7f8d6c0b5af0 legacy=0x7f8d6c162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:02.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.047+0000 7f8d6a7fc640 1 -- 172.21.15.32:0/2702004843 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1961435120 0 0) 0x7f8d6c0cd270 con 0x7f8d6c0b5af0 2024-04-24T23:03:02.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.047+0000 7f8d6b7fe640 1 -- 172.21.15.32:0/2702004843 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d6c0718f0 con 0x7f8d6c162320 2024-04-24T23:03:02.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.047+0000 7f8d6b7fe640 1 --2- 172.21.15.32:0/2702004843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d6c162320 0x7f8d6c0ccc90 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c0097a0 tx=0x7f8d5c02a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4014bd2622ca6750 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:02.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.047+0000 7f8d6a7fc640 1 -- 172.21.15.32:0/2702004843 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d5c037070 con 0x7f8d6c162320 2024-04-24T23:03:02.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.047+0000 7f8d6a7fc640 1 -- 172.21.15.32:0/2702004843 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d5c02adb0 con 0x7f8d6c162320 2024-04-24T23:03:02.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.047+0000 7f8d6a7fc640 1 -- 172.21.15.32:0/2702004843 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d5c032660 con 0x7f8d6c162320 2024-04-24T23:03:02.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.048+0000 7f8d71c27640 1 -- 172.21.15.32:0/2702004843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d6c162320 msgr2=0x7f8d6c0ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:02.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.048+0000 7f8d71c27640 1 --2- 172.21.15.32:0/2702004843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d6c162320 0x7f8d6c0ccc90 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c0097a0 tx=0x7f8d5c02a060 comp rx=0 tx=0).stop 2024-04-24T23:03:02.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.048+0000 7f8d71c27640 1 -- 172.21.15.32:0/2702004843 shutdown_connections 2024-04-24T23:03:02.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.048+0000 7f8d71c27640 1 --2- 172.21.15.32:0/2702004843 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d6c162e10 0x7f8d6c15ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:02.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.048+0000 7f8d71c27640 1 --2- 172.21.15.32:0/2702004843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d6c162320 0x7f8d6c0ccc90 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:02.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.048+0000 7f8d71c27640 1 -- 172.21.15.32:0/2702004843 >> 172.21.15.32:0/2702004843 conn(0x7f8d6c0d0ae0 msgr2=0x7f8d6c0d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:02.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.048+0000 7f8d71c27640 1 -- 172.21.15.32:0/2702004843 shutdown_connections 2024-04-24T23:03:02.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.048+0000 7f8d71c27640 1 -- 172.21.15.32:0/2702004843 wait complete. 2024-04-24T23:03:02.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.049+0000 7f8d71c27640 1 Processor -- start 2024-04-24T23:03:02.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.049+0000 7f8d71c27640 1 -- start start 2024-04-24T23:03:02.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.050+0000 7f8d71c27640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d6c0b5af0 0x7f8d6c1e9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:02.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.050+0000 7f8d71c27640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d6c162320 0x7f8d6c14bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:02.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.050+0000 7f8d71c27640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d6c162e10 0x7f8d6c0b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:02.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.050+0000 7f8d71c27640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8d6c071bd0 con 0x7f8d6c162e10 2024-04-24T23:03:02.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.050+0000 7f8d71c27640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8d6c067950 con 0x7f8d6c162320 2024-04-24T23:03:02.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.050+0000 7f8d71c27640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f8d6c071d50 con 0x7f8d6c0b5af0 2024-04-24T23:03:02.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.050+0000 7f8d6bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d6c162e10 0x7f8d6c0b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:02.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.050+0000 7f8d6bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d6c162e10 0x7f8d6c0b01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38376/0 (socket says 172.21.15.32:38376) 2024-04-24T23:03:02.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.050+0000 7f8d6bfff640 1 -- 172.21.15.32:0/4187681937 learned_addr learned my addr 172.21.15.32:0/4187681937 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:02.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.050+0000 7f8d6b7fe640 1 --2- 172.21.15.32:0/4187681937 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d6c0b5af0 0x7f8d6c1e9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:02.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.050+0000 7f8d6affd640 1 --2- 172.21.15.32:0/4187681937 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d6c162320 0x7f8d6c14bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:02.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.051+0000 7f8d6bfff640 1 -- 172.21.15.32:0/4187681937 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d6c0b5af0 msgr2=0x7f8d6c1e9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:02.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.051+0000 7f8d6bfff640 1 --2- 172.21.15.32:0/4187681937 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d6c0b5af0 0x7f8d6c1e9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:02.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.051+0000 7f8d6bfff640 1 -- 172.21.15.32:0/4187681937 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d6c162320 msgr2=0x7f8d6c14bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:02.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.051+0000 7f8d6bfff640 1 --2- 172.21.15.32:0/4187681937 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d6c162320 0x7f8d6c14bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:02.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.051+0000 7f8d6bfff640 1 -- 172.21.15.32:0/4187681937 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d6c0af100 con 0x7f8d6c162e10 2024-04-24T23:03:02.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.051+0000 7f8d6bfff640 1 --2- 172.21.15.32:0/4187681937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d6c162e10 0x7f8d6c0b01a0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f8d64007e60 tx=0x7f8d6400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:02.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.051+0000 7f8d53fff640 1 -- 172.21.15.32:0/4187681937 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d64014420 con 0x7f8d6c162e10 2024-04-24T23:03:02.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.051+0000 7f8d53fff640 1 -- 172.21.15.32:0/4187681937 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d64019070 con 0x7f8d6c162e10 2024-04-24T23:03:02.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.051+0000 7f8d71c27640 1 -- 172.21.15.32:0/4187681937 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d6c0af3e0 con 0x7f8d6c162e10 2024-04-24T23:03:02.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.051+0000 7f8d53fff640 1 -- 172.21.15.32:0/4187681937 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d64014820 con 0x7f8d6c162e10 2024-04-24T23:03:02.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.051+0000 7f8d71c27640 1 -- 172.21.15.32:0/4187681937 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d6c0af9a0 con 0x7f8d6c162e10 2024-04-24T23:03:02.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.052+0000 7f8d71c27640 1 -- 172.21.15.32:0/4187681937 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d6c068430 con 0x7f8d6c162e10 2024-04-24T23:03:02.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.052+0000 7f8d53fff640 1 -- 172.21.15.32:0/4187681937 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8d640149c0 con 0x7f8d6c162e10 2024-04-24T23:03:02.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.053+0000 7f8d53fff640 1 --2- 172.21.15.32:0/4187681937 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8d5806be30 0x7f8d5806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:02.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.053+0000 7f8d53fff640 1 -- 172.21.15.32:0/4187681937 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5709+0+0 (secure 0 0 0) 0x7f8d6408b5d0 con 0x7f8d6c162e10 2024-04-24T23:03:02.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.053+0000 7f8d6b7fe640 1 --2- 172.21.15.32:0/4187681937 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8d5806be30 0x7f8d5806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:02.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.054+0000 7f8d6b7fe640 1 --2- 172.21.15.32:0/4187681937 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8d5806be30 0x7f8d5806e2f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c034000 tx=0x7f8d5c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:02.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.057+0000 7f8d53fff640 1 -- 172.21.15.32:0/4187681937 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8d64059890 con 0x7f8d6c162e10 2024-04-24T23:03:02.261 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-04-24T23:03:02.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:02.279+0000 7f8d71c27640 1 -- 172.21.15.32:0/4187681937 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f8d6c0d0850 con 0x7f8d6c162e10 2024-04-24T23:03:03.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.278+0000 7f8d53fff640 1 -- 172.21.15.32:0/4187681937 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v35) ==== 118+0+0 (secure 0 0 0) 0x7f8d6405db90 con 0x7f8d6c162e10 2024-04-24T23:03:03.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.281+0000 7f8d71c27640 1 -- 172.21.15.32:0/4187681937 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8d5806be30 msgr2=0x7f8d5806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:03.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.281+0000 7f8d71c27640 1 --2- 172.21.15.32:0/4187681937 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8d5806be30 0x7f8d5806e2f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c034000 tx=0x7f8d5c02e000 comp rx=0 tx=0).stop 2024-04-24T23:03:03.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.281+0000 7f8d71c27640 1 -- 172.21.15.32:0/4187681937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d6c162e10 msgr2=0x7f8d6c0b01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:03.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.281+0000 7f8d71c27640 1 --2- 172.21.15.32:0/4187681937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d6c162e10 0x7f8d6c0b01a0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f8d64007e60 tx=0x7f8d6400e4b0 comp rx=0 tx=0).stop 2024-04-24T23:03:03.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.281+0000 7f8d71c27640 1 -- 172.21.15.32:0/4187681937 shutdown_connections 2024-04-24T23:03:03.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.281+0000 7f8d71c27640 1 --2- 172.21.15.32:0/4187681937 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8d5806be30 0x7f8d5806e2f0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:03.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.281+0000 7f8d71c27640 1 --2- 172.21.15.32:0/4187681937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d6c162e10 0x7f8d6c0b01a0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:03.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.281+0000 7f8d71c27640 1 --2- 172.21.15.32:0/4187681937 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d6c162320 0x7f8d6c14bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:03.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.281+0000 7f8d71c27640 1 --2- 172.21.15.32:0/4187681937 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d6c0b5af0 0x7f8d6c1e9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:03.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.281+0000 7f8d71c27640 1 -- 172.21.15.32:0/4187681937 >> 172.21.15.32:0/4187681937 conn(0x7f8d6c0d0ae0 msgr2=0x7f8d6c15d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:03.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.281+0000 7f8d71c27640 1 -- 172.21.15.32:0/4187681937 shutdown_connections 2024-04-24T23:03:03.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.281+0000 7f8d71c27640 1 -- 172.21.15.32:0/4187681937 wait complete. 2024-04-24T23:03:03.296 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-24T23:03:03.296 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-24T23:03:03.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.405+0000 7f45c6058640 1 Processor -- start 2024-04-24T23:03:03.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.405+0000 7f45c6058640 1 -- start start 2024-04-24T23:03:03.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.406+0000 7f45c6058640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f45c00cd3c0 0x7f45c006c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:03.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.406+0000 7f45c6058640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f45c015e6c0 0x7f45c015eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:03.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.406+0000 7f45c6058640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f45c01514b0 0x7f45c01518e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:03.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.406+0000 7f45c6058640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f45c00cd910 con 0x7f45c00cd3c0 2024-04-24T23:03:03.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.406+0000 7f45c6058640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f45c00cdbf0 con 0x7f45c015e6c0 2024-04-24T23:03:03.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.406+0000 7f45c6058640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f45c00cded0 con 0x7f45c01514b0 2024-04-24T23:03:03.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.406+0000 7f45bf7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f45c00cd3c0 0x7f45c006c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:03.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.406+0000 7f45bf7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f45c00cd3c0 0x7f45c006c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38378/0 (socket says 172.21.15.32:38378) 2024-04-24T23:03:03.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.406+0000 7f45bf7fe640 1 -- 172.21.15.32:0/2402545940 learned_addr learned my addr 172.21.15.32:0/2402545940 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:03.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.406+0000 7f45bffff640 1 --2- 172.21.15.32:0/2402545940 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f45c01514b0 0x7f45c01518e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:03.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.406+0000 7f45beffd640 1 --2- 172.21.15.32:0/2402545940 >> v2:172.21.15.121:3300/0 conn(0x7f45c015e6c0 0x7f45c015eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:03.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.406+0000 7f45bf7fe640 1 -- 172.21.15.32:0/2402545940 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f45c01514b0 msgr2=0x7f45c01518e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:03.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.406+0000 7f45bf7fe640 1 --2- 172.21.15.32:0/2402545940 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f45c01514b0 0x7f45c01518e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:03.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.406+0000 7f45bf7fe640 1 -- 172.21.15.32:0/2402545940 >> v2:172.21.15.121:3300/0 conn(0x7f45c015e6c0 msgr2=0x7f45c015eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:03.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.406+0000 7f45bf7fe640 1 --2- 172.21.15.32:0/2402545940 >> v2:172.21.15.121:3300/0 conn(0x7f45c015e6c0 0x7f45c015eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:03.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.406+0000 7f45bf7fe640 1 -- 172.21.15.32:0/2402545940 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45c00ce1b0 con 0x7f45c00cd3c0 2024-04-24T23:03:03.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.407+0000 7f45bf7fe640 1 --2- 172.21.15.32:0/2402545940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45c00cd3c0 0x7f45c006c5f0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f45c00cc8d0 tx=0x7f45b002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e35342f45157f1d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:03.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.407+0000 7f45be7fc640 1 -- 172.21.15.32:0/2402545940 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b0037070 con 0x7f45c00cd3c0 2024-04-24T23:03:03.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.407+0000 7f45be7fc640 1 -- 172.21.15.32:0/2402545940 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45b0032440 con 0x7f45c00cd3c0 2024-04-24T23:03:03.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.407+0000 7f45be7fc640 1 -- 172.21.15.32:0/2402545940 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b0032810 con 0x7f45c00cd3c0 2024-04-24T23:03:03.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.408+0000 7f45c6058640 1 -- 172.21.15.32:0/2402545940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45c00cd3c0 msgr2=0x7f45c006c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:03.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.408+0000 7f45c6058640 1 --2- 172.21.15.32:0/2402545940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45c00cd3c0 0x7f45c006c5f0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f45c00cc8d0 tx=0x7f45b002a1e0 comp rx=0 tx=0).stop 2024-04-24T23:03:03.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.408+0000 7f45c6058640 1 -- 172.21.15.32:0/2402545940 shutdown_connections 2024-04-24T23:03:03.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.408+0000 7f45c6058640 1 --2- 172.21.15.32:0/2402545940 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f45c01514b0 0x7f45c01518e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:03.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.408+0000 7f45c6058640 1 --2- 172.21.15.32:0/2402545940 >> v2:172.21.15.121:3300/0 conn(0x7f45c015e6c0 0x7f45c015eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:03.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.408+0000 7f45c6058640 1 --2- 172.21.15.32:0/2402545940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45c00cd3c0 0x7f45c006c5f0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:03.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.408+0000 7f45c6058640 1 -- 172.21.15.32:0/2402545940 >> 172.21.15.32:0/2402545940 conn(0x7f45c006b110 msgr2=0x7f45c006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:03.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.408+0000 7f45c6058640 1 -- 172.21.15.32:0/2402545940 shutdown_connections 2024-04-24T23:03:03.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.408+0000 7f45c6058640 1 -- 172.21.15.32:0/2402545940 wait complete. 2024-04-24T23:03:03.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.409+0000 7f45c6058640 1 Processor -- start 2024-04-24T23:03:03.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.409+0000 7f45c6058640 1 -- start start 2024-04-24T23:03:03.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.410+0000 7f45c6058640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45c00cd3c0 0x7f45c0158190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:03.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.410+0000 7f45c6058640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f45c01514b0 0x7f45c00721c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:03.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.410+0000 7f45c6058640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45c015e6c0 0x7f45c0072700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:03.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.410+0000 7f45c6058640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f45c00b45a0 con 0x7f45c015e6c0 2024-04-24T23:03:03.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.410+0000 7f45c6058640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f45c00b4420 con 0x7f45c00cd3c0 2024-04-24T23:03:03.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.410+0000 7f45c6058640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f45c006cb30 con 0x7f45c01514b0 2024-04-24T23:03:03.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.410+0000 7f45bffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45c015e6c0 0x7f45c0072700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:03.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.410+0000 7f45bffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45c015e6c0 0x7f45c0072700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38390/0 (socket says 172.21.15.32:38390) 2024-04-24T23:03:03.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.410+0000 7f45bffff640 1 -- 172.21.15.32:0/3277093866 learned_addr learned my addr 172.21.15.32:0/3277093866 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:03.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.410+0000 7f45bf7fe640 1 --2- 172.21.15.32:0/3277093866 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45c00cd3c0 0x7f45c0158190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:03.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.410+0000 7f45beffd640 1 --2- 172.21.15.32:0/3277093866 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f45c01514b0 0x7f45c00721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:03.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.410+0000 7f45bffff640 1 -- 172.21.15.32:0/3277093866 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f45c01514b0 msgr2=0x7f45c00721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:03.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.410+0000 7f45bffff640 1 --2- 172.21.15.32:0/3277093866 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f45c01514b0 0x7f45c00721c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:03.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.410+0000 7f45bffff640 1 -- 172.21.15.32:0/3277093866 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45c00cd3c0 msgr2=0x7f45c0158190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:03.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.410+0000 7f45bffff640 1 --2- 172.21.15.32:0/3277093866 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45c00cd3c0 0x7f45c0158190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:03.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.410+0000 7f45bffff640 1 -- 172.21.15.32:0/3277093866 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45c01542b0 con 0x7f45c015e6c0 2024-04-24T23:03:03.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.410+0000 7f45beffd640 1 --2- 172.21.15.32:0/3277093866 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f45c01514b0 0x7f45c00721c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:03:03.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.411+0000 7f45bffff640 1 --2- 172.21.15.32:0/3277093866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45c015e6c0 0x7f45c0072700 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f45b8007e90 tx=0x7f45b800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:03.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.411+0000 7f45a7fff640 1 -- 172.21.15.32:0/3277093866 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b8014420 con 0x7f45c015e6c0 2024-04-24T23:03:03.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.411+0000 7f45a7fff640 1 -- 172.21.15.32:0/3277093866 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45b8019070 con 0x7f45c015e6c0 2024-04-24T23:03:03.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.411+0000 7f45c6058640 1 -- 172.21.15.32:0/3277093866 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45c0071150 con 0x7f45c015e6c0 2024-04-24T23:03:03.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.411+0000 7f45a7fff640 1 -- 172.21.15.32:0/3277093866 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b8014800 con 0x7f45c015e6c0 2024-04-24T23:03:03.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.411+0000 7f45c6058640 1 -- 172.21.15.32:0/3277093866 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45c0154550 con 0x7f45c015e6c0 2024-04-24T23:03:03.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.412+0000 7f45c6058640 1 -- 172.21.15.32:0/3277093866 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45c0068430 con 0x7f45c015e6c0 2024-04-24T23:03:03.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.412+0000 7f45a7fff640 1 -- 172.21.15.32:0/3277093866 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f45b80149a0 con 0x7f45c015e6c0 2024-04-24T23:03:03.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.413+0000 7f45a7fff640 1 --2- 172.21.15.32:0/3277093866 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f45ac06be60 0x7f45ac06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:03.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.413+0000 7f45a7fff640 1 -- 172.21.15.32:0/3277093866 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7f45b808b510 con 0x7f45c015e6c0 2024-04-24T23:03:03.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.413+0000 7f45bf7fe640 1 --2- 172.21.15.32:0/3277093866 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f45ac06be60 0x7f45ac06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:03.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.413+0000 7f45bf7fe640 1 --2- 172.21.15.32:0/3277093866 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f45ac06be60 0x7f45ac06e320 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f45b0034000 tx=0x7f45b002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:03.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.417+0000 7f45a7fff640 1 -- 172.21.15.32:0/3277093866 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f45b8059f50 con 0x7f45c015e6c0 2024-04-24T23:03:03.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:03.638+0000 7f45c6058640 1 -- 172.21.15.32:0/3277093866 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f45c0070520 con 0x7f45c015e6c0 2024-04-24T23:03:04.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.290+0000 7f45a7fff640 1 -- 172.21.15.32:0/3277093866 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v36) ==== 122+0+0 (secure 0 0 0) 0x7f45b8059850 con 0x7f45c015e6c0 2024-04-24T23:03:04.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.293+0000 7f45c6058640 1 -- 172.21.15.32:0/3277093866 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f45ac06be60 msgr2=0x7f45ac06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:04.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.293+0000 7f45c6058640 1 --2- 172.21.15.32:0/3277093866 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f45ac06be60 0x7f45ac06e320 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f45b0034000 tx=0x7f45b002e000 comp rx=0 tx=0).stop 2024-04-24T23:03:04.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.293+0000 7f45c6058640 1 -- 172.21.15.32:0/3277093866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45c015e6c0 msgr2=0x7f45c0072700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:04.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.293+0000 7f45c6058640 1 --2- 172.21.15.32:0/3277093866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45c015e6c0 0x7f45c0072700 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f45b8007e90 tx=0x7f45b800e4b0 comp rx=0 tx=0).stop 2024-04-24T23:03:04.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.293+0000 7f45c6058640 1 -- 172.21.15.32:0/3277093866 shutdown_connections 2024-04-24T23:03:04.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.293+0000 7f45c6058640 1 --2- 172.21.15.32:0/3277093866 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f45ac06be60 0x7f45ac06e320 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:04.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.293+0000 7f45c6058640 1 --2- 172.21.15.32:0/3277093866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45c015e6c0 0x7f45c0072700 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:04.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.293+0000 7f45c6058640 1 --2- 172.21.15.32:0/3277093866 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f45c01514b0 0x7f45c00721c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:04.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.293+0000 7f45c6058640 1 --2- 172.21.15.32:0/3277093866 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45c00cd3c0 0x7f45c0158190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:04.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.293+0000 7f45c6058640 1 -- 172.21.15.32:0/3277093866 >> 172.21.15.32:0/3277093866 conn(0x7f45c006b110 msgr2=0x7f45c006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:04.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.293+0000 7f45c6058640 1 -- 172.21.15.32:0/3277093866 shutdown_connections 2024-04-24T23:03:04.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.293+0000 7f45c6058640 1 -- 172.21.15.32:0/3277093866 wait complete. 2024-04-24T23:03:04.309 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-24T23:03:04.309 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:03:04.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.418+0000 7f2c8c9ce640 1 Processor -- start 2024-04-24T23:03:04.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.418+0000 7f2c8c9ce640 1 -- start start 2024-04-24T23:03:04.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.419+0000 7f2c8c9ce640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2c8815b480 0x7f2c880b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:04.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.419+0000 7f2c8c9ce640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c8815fd60 0x7f2c8806c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:04.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.419+0000 7f2c8c9ce640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2c8806cb30 0x7f2c880ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:04.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.419+0000 7f2c8c9ce640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f2c880b4ff0 con 0x7f2c8815b480 2024-04-24T23:03:04.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.419+0000 7f2c8c9ce640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f2c880b46c0 con 0x7f2c8806cb30 2024-04-24T23:03:04.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.419+0000 7f2c8c9ce640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f2c880b49a0 con 0x7f2c8815fd60 2024-04-24T23:03:04.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.419+0000 7f2c8659c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2c8815b480 0x7f2c880b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:04.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.419+0000 7f2c8659c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2c8815b480 0x7f2c880b5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38402/0 (socket says 172.21.15.32:38402) 2024-04-24T23:03:04.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.419+0000 7f2c8659c640 1 -- 172.21.15.32:0/1037173257 learned_addr learned my addr 172.21.15.32:0/1037173257 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:04.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.419+0000 7f2c86d9d640 1 --2- 172.21.15.32:0/1037173257 >> v2:172.21.15.121:3300/0 conn(0x7f2c8806cb30 0x7f2c880ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:04.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.419+0000 7f2c7ffff640 1 --2- 172.21.15.32:0/1037173257 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c8815fd60 0x7f2c8806c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:04.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.420+0000 7f2c8659c640 1 -- 172.21.15.32:0/1037173257 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c8815fd60 msgr2=0x7f2c8806c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:04.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.420+0000 7f2c8659c640 1 --2- 172.21.15.32:0/1037173257 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c8815fd60 0x7f2c8806c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:04.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.420+0000 7f2c8659c640 1 -- 172.21.15.32:0/1037173257 >> v2:172.21.15.121:3300/0 conn(0x7f2c8806cb30 msgr2=0x7f2c880ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:04.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.420+0000 7f2c8659c640 1 --2- 172.21.15.32:0/1037173257 >> v2:172.21.15.121:3300/0 conn(0x7f2c8806cb30 0x7f2c880ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:04.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.420+0000 7f2c8659c640 1 -- 172.21.15.32:0/1037173257 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c880b4c80 con 0x7f2c8815b480 2024-04-24T23:03:04.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.420+0000 7f2c86d9d640 1 --2- 172.21.15.32:0/1037173257 >> v2:172.21.15.121:3300/0 conn(0x7f2c8806cb30 0x7f2c880ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:03:04.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.420+0000 7f2c8659c640 1 --2- 172.21.15.32:0/1037173257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c8815b480 0x7f2c880b5c60 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f2c880aff10 tx=0x7f2c7802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f31ce2ec0e7c27ba server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:04.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.421+0000 7f2c85d9b640 1 -- 172.21.15.32:0/1037173257 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c78037070 con 0x7f2c8815b480 2024-04-24T23:03:04.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.421+0000 7f2c85d9b640 1 -- 172.21.15.32:0/1037173257 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c78032440 con 0x7f2c8815b480 2024-04-24T23:03:04.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.421+0000 7f2c85d9b640 1 -- 172.21.15.32:0/1037173257 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c78032810 con 0x7f2c8815b480 2024-04-24T23:03:04.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.421+0000 7f2c8c9ce640 1 -- 172.21.15.32:0/1037173257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c8815b480 msgr2=0x7f2c880b5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:04.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.421+0000 7f2c8c9ce640 1 --2- 172.21.15.32:0/1037173257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c8815b480 0x7f2c880b5c60 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f2c880aff10 tx=0x7f2c7802a1e0 comp rx=0 tx=0).stop 2024-04-24T23:03:04.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.421+0000 7f2c8c9ce640 1 -- 172.21.15.32:0/1037173257 shutdown_connections 2024-04-24T23:03:04.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.421+0000 7f2c8c9ce640 1 --2- 172.21.15.32:0/1037173257 >> v2:172.21.15.121:3300/0 conn(0x7f2c8806cb30 0x7f2c880ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:04.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.421+0000 7f2c8c9ce640 1 --2- 172.21.15.32:0/1037173257 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c8815fd60 0x7f2c8806c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:04.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.421+0000 7f2c8c9ce640 1 --2- 172.21.15.32:0/1037173257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c8815b480 0x7f2c880b5c60 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:04.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.421+0000 7f2c8c9ce640 1 -- 172.21.15.32:0/1037173257 >> 172.21.15.32:0/1037173257 conn(0x7f2c8806b110 msgr2=0x7f2c8806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:04.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.421+0000 7f2c8c9ce640 1 -- 172.21.15.32:0/1037173257 shutdown_connections 2024-04-24T23:03:04.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.421+0000 7f2c8c9ce640 1 -- 172.21.15.32:0/1037173257 wait complete. 2024-04-24T23:03:04.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.423+0000 7f2c8c9ce640 1 Processor -- start 2024-04-24T23:03:04.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.423+0000 7f2c8c9ce640 1 -- start start 2024-04-24T23:03:04.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.423+0000 7f2c8c9ce640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c8806cb30 0x7f2c88072270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:04.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.423+0000 7f2c8c9ce640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c8815b480 0x7f2c8814ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:04.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.423+0000 7f2c8c9ce640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c8815fd60 0x7f2c8814f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:04.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.423+0000 7f2c8c9ce640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8815b860 con 0x7f2c8815fd60 2024-04-24T23:03:04.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.423+0000 7f2c8c9ce640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2c880b5480 con 0x7f2c8815b480 2024-04-24T23:03:04.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.423+0000 7f2c8c9ce640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f2c880711d0 con 0x7f2c8806cb30 2024-04-24T23:03:04.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.423+0000 7f2c86d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c8815fd60 0x7f2c8814f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:04.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.423+0000 7f2c86d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c8815fd60 0x7f2c8814f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38412/0 (socket says 172.21.15.32:38412) 2024-04-24T23:03:04.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.423+0000 7f2c86d9d640 1 -- 172.21.15.32:0/1585529588 learned_addr learned my addr 172.21.15.32:0/1585529588 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:04.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.424+0000 7f2c7ffff640 1 --2- 172.21.15.32:0/1585529588 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c8815b480 0x7f2c8814ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:04.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.424+0000 7f2c8659c640 1 --2- 172.21.15.32:0/1585529588 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c8806cb30 0x7f2c88072270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:04.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.424+0000 7f2c86d9d640 1 -- 172.21.15.32:0/1585529588 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c8806cb30 msgr2=0x7f2c88072270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:04.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.424+0000 7f2c86d9d640 1 --2- 172.21.15.32:0/1585529588 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c8806cb30 0x7f2c88072270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:04.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.424+0000 7f2c86d9d640 1 -- 172.21.15.32:0/1585529588 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c8815b480 msgr2=0x7f2c8814ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:04.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.424+0000 7f2c86d9d640 1 --2- 172.21.15.32:0/1585529588 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c8815b480 0x7f2c8814ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:04.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.424+0000 7f2c86d9d640 1 -- 172.21.15.32:0/1585529588 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c880714b0 con 0x7f2c8815fd60 2024-04-24T23:03:04.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.424+0000 7f2c8659c640 1 --2- 172.21.15.32:0/1585529588 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c8806cb30 0x7f2c88072270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:03:04.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.424+0000 7f2c86d9d640 1 --2- 172.21.15.32:0/1585529588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c8815fd60 0x7f2c8814f440 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f2c8000cf20 tx=0x7f2c8000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:04.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.424+0000 7f2c7effd640 1 -- 172.21.15.32:0/1585529588 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8001a070 con 0x7f2c8815fd60 2024-04-24T23:03:04.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.424+0000 7f2c7effd640 1 -- 172.21.15.32:0/1585529588 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c8000d840 con 0x7f2c8815fd60 2024-04-24T23:03:04.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.424+0000 7f2c8c9ce640 1 -- 172.21.15.32:0/1585529588 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c88071790 con 0x7f2c8815fd60 2024-04-24T23:03:04.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.424+0000 7f2c7effd640 1 -- 172.21.15.32:0/1585529588 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8000dc30 con 0x7f2c8815fd60 2024-04-24T23:03:04.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.424+0000 7f2c8c9ce640 1 -- 172.21.15.32:0/1585529588 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c88071d50 con 0x7f2c8815fd60 2024-04-24T23:03:04.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.425+0000 7f2c8c9ce640 1 -- 172.21.15.32:0/1585529588 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c88068430 con 0x7f2c8815fd60 2024-04-24T23:03:04.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.426+0000 7f2c7effd640 1 -- 172.21.15.32:0/1585529588 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2c8001d070 con 0x7f2c8815fd60 2024-04-24T23:03:04.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.426+0000 7f2c7effd640 1 --2- 172.21.15.32:0/1585529588 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2c6806be30 0x7f2c6806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:04.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.426+0000 7f2c7effd640 1 -- 172.21.15.32:0/1585529588 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7f2c8008b540 con 0x7f2c8815fd60 2024-04-24T23:03:04.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.427+0000 7f2c8659c640 1 --2- 172.21.15.32:0/1585529588 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2c6806be30 0x7f2c6806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:04.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.427+0000 7f2c8659c640 1 --2- 172.21.15.32:0/1585529588 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2c6806be30 0x7f2c6806e2f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f2c78034000 tx=0x7f2c7802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:04.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.430+0000 7f2c7effd640 1 -- 172.21.15.32:0/1585529588 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2c80059930 con 0x7f2c8815fd60 2024-04-24T23:03:04.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.652+0000 7f2c8c9ce640 1 -- 172.21.15.32:0/1585529588 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c880afae0 con 0x7f2c8815fd60 2024-04-24T23:03:04.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.653+0000 7f2c7effd640 1 -- 172.21.15.32:0/1585529588 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5186 (secure 0 0 0) 0x7f2c8005dc30 con 0x7f2c8815fd60 2024-04-24T23:03:04.654 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:03:04.654 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":36,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5486,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/3550206194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3550206194},{"type":"v1","addr":"172.21.15.32:6834","nonce":3550206194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5905,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1706188546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1706188546},{"type":"v1","addr":"172.21.15.121:6835","nonce":1706188546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5911,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2669443498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2669443498},{"type":"v1","addr":"172.21.15.121:6837","nonce":2669443498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:03:00.630938+0000","modified":"2024-04-24T23:03:03.639570+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6474":{"gid":6474,"name":"a","rank":0,"incarnation":33,"state":"up:active","state_seq":4,"addr":"172.21.15.32:6835/464246210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":464246210},{"type":"v1","addr":"172.21.15.32:6835","nonce":464246210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6474,"qdb_cluster":[6474]},"id":5}]} 2024-04-24T23:03:04.655 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 36 2024-04-24T23:03:04.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.655+0000 7f2c8c9ce640 1 -- 172.21.15.32:0/1585529588 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2c6806be30 msgr2=0x7f2c6806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:04.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.655+0000 7f2c8c9ce640 1 --2- 172.21.15.32:0/1585529588 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2c6806be30 0x7f2c6806e2f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f2c78034000 tx=0x7f2c7802e000 comp rx=0 tx=0).stop 2024-04-24T23:03:04.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.655+0000 7f2c8c9ce640 1 -- 172.21.15.32:0/1585529588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c8815fd60 msgr2=0x7f2c8814f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:04.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.655+0000 7f2c8c9ce640 1 --2- 172.21.15.32:0/1585529588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c8815fd60 0x7f2c8814f440 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f2c8000cf20 tx=0x7f2c8000a710 comp rx=0 tx=0).stop 2024-04-24T23:03:04.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.655+0000 7f2c8c9ce640 1 -- 172.21.15.32:0/1585529588 shutdown_connections 2024-04-24T23:03:04.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.655+0000 7f2c8c9ce640 1 --2- 172.21.15.32:0/1585529588 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2c6806be30 0x7f2c6806e2f0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:04.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.655+0000 7f2c8c9ce640 1 --2- 172.21.15.32:0/1585529588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c8815fd60 0x7f2c8814f440 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:04.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.655+0000 7f2c8c9ce640 1 --2- 172.21.15.32:0/1585529588 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c8815b480 0x7f2c8814ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:04.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.655+0000 7f2c8c9ce640 1 --2- 172.21.15.32:0/1585529588 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c8806cb30 0x7f2c88072270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:04.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.655+0000 7f2c8c9ce640 1 -- 172.21.15.32:0/1585529588 >> 172.21.15.32:0/1585529588 conn(0x7f2c8806b110 msgr2=0x7f2c880afe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:04.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.655+0000 7f2c8c9ce640 1 -- 172.21.15.32:0/1585529588 shutdown_connections 2024-04-24T23:03:04.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.655+0000 7f2c8c9ce640 1 -- 172.21.15.32:0/1585529588 wait complete. 2024-04-24T23:03:04.667 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:03:04.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.766+0000 7f2646b34640 1 Processor -- start 2024-04-24T23:03:04.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.766+0000 7f2646b34640 1 -- start start 2024-04-24T23:03:04.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.767+0000 7f2646b34640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f264015b480 0x7f26400b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:04.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.767+0000 7f2646b34640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f264015fd60 0x7f264006c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:04.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.767+0000 7f2646b34640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26400cef90 con 0x7f264006cb30 2024-04-24T23:03:04.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.767+0000 7f2646b34640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f26400b4ff0 con 0x7f264015b480 2024-04-24T23:03:04.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.767+0000 7f2646b34640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f26400b46c0 con 0x7f264015fd60 2024-04-24T23:03:04.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.767+0000 7f26448d0640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f264015b480 0x7f26400b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:04.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.767+0000 7f26450d1640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f264006cb30 0x7f26400ccc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57608/0 (socket says 172.21.15.32:57608) 2024-04-24T23:03:04.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.767+0000 7f26450d1640 1 -- 172.21.15.32:0/3600910996 learned_addr learned my addr 172.21.15.32:0/3600910996 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:03:04.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.767+0000 7f263ffff640 1 --2- 172.21.15.32:0/3600910996 >> v2:172.21.15.121:3300/0 conn(0x7f264015fd60 0x7f264006c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:04.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.767+0000 7f26448d0640 1 -- 172.21.15.32:0/3600910996 >> v2:172.21.15.121:3300/0 conn(0x7f264015fd60 msgr2=0x7f264006c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:04.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.767+0000 7f26448d0640 1 --2- 172.21.15.32:0/3600910996 >> v2:172.21.15.121:3300/0 conn(0x7f264015fd60 0x7f264006c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:04.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.767+0000 7f26448d0640 1 -- 172.21.15.32:0/3600910996 >> v1:172.21.15.32:6789/0 conn(0x7f264006cb30 legacy=0x7f26400ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:04.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.767+0000 7f26448d0640 1 -- 172.21.15.32:0/3600910996 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26400b49a0 con 0x7f264015b480 2024-04-24T23:03:04.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.767+0000 7f263f7fe640 1 -- 172.21.15.32:0/3600910996 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4272789880 0 0) 0x7f26400cef90 con 0x7f264006cb30 2024-04-24T23:03:04.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.767+0000 7f263ffff640 1 --2- 172.21.15.32:0/3600910996 >> v2:172.21.15.121:3300/0 conn(0x7f264015fd60 0x7f264006c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:03:04.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.768+0000 7f26448d0640 1 --2- 172.21.15.32:0/3600910996 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f264015b480 0x7f26400b5c60 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f264006ee10 tx=0x7f263402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66efae6f678d2c90 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:04.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.768+0000 7f263f7fe640 1 -- 172.21.15.32:0/3600910996 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2634037070 con 0x7f264015b480 2024-04-24T23:03:04.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.768+0000 7f263f7fe640 1 -- 172.21.15.32:0/3600910996 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2634032440 con 0x7f264015b480 2024-04-24T23:03:04.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.768+0000 7f263f7fe640 1 -- 172.21.15.32:0/3600910996 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2634032810 con 0x7f264015b480 2024-04-24T23:03:04.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.769+0000 7f2646b34640 1 -- 172.21.15.32:0/3600910996 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f264015b480 msgr2=0x7f26400b5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:04.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.769+0000 7f2646b34640 1 --2- 172.21.15.32:0/3600910996 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f264015b480 0x7f26400b5c60 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f264006ee10 tx=0x7f263402a1a0 comp rx=0 tx=0).stop 2024-04-24T23:03:04.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.769+0000 7f2646b34640 1 -- 172.21.15.32:0/3600910996 shutdown_connections 2024-04-24T23:03:04.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.769+0000 7f2646b34640 1 --2- 172.21.15.32:0/3600910996 >> v2:172.21.15.121:3300/0 conn(0x7f264015fd60 0x7f264006c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:04.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.769+0000 7f2646b34640 1 --2- 172.21.15.32:0/3600910996 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f264015b480 0x7f26400b5c60 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:04.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.769+0000 7f2646b34640 1 -- 172.21.15.32:0/3600910996 >> 172.21.15.32:0/3600910996 conn(0x7f264006b110 msgr2=0x7f264006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:04.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.769+0000 7f2646b34640 1 -- 172.21.15.32:0/3600910996 shutdown_connections 2024-04-24T23:03:04.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.769+0000 7f2646b34640 1 -- 172.21.15.32:0/3600910996 wait complete. 2024-04-24T23:03:04.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.770+0000 7f2646b34640 1 Processor -- start 2024-04-24T23:03:04.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.770+0000 7f2646b34640 1 -- start start 2024-04-24T23:03:04.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.771+0000 7f2646b34640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f264006cb30 0x7f2640148d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:04.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.771+0000 7f2646b34640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f264015b480 0x7f2640157250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:04.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.771+0000 7f2646b34640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f264015fd60 0x7f2640071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:04.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.771+0000 7f2646b34640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f26400b5480 con 0x7f264015b480 2024-04-24T23:03:04.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.771+0000 7f2646b34640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f26400b61a0 con 0x7f264006cb30 2024-04-24T23:03:04.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.771+0000 7f2646b34640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f264015b860 con 0x7f264015fd60 2024-04-24T23:03:04.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.771+0000 7f263ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f264015b480 0x7f2640157250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:04.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.771+0000 7f263ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f264015b480 0x7f2640157250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38428/0 (socket says 172.21.15.32:38428) 2024-04-24T23:03:04.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.771+0000 7f263ffff640 1 -- 172.21.15.32:0/2134895477 learned_addr learned my addr 172.21.15.32:0/2134895477 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:04.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.771+0000 7f263ffff640 1 -- 172.21.15.32:0/2134895477 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f264015fd60 msgr2=0x7f2640071ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:04.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.771+0000 7f26450d1640 1 --2- 172.21.15.32:0/2134895477 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f264015fd60 0x7f2640071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:04.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.771+0000 7f26448d0640 1 --2- 172.21.15.32:0/2134895477 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f264006cb30 0x7f2640148d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:04.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.771+0000 7f263ffff640 1 --2- 172.21.15.32:0/2134895477 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f264015fd60 0x7f2640071ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:04.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.771+0000 7f263ffff640 1 -- 172.21.15.32:0/2134895477 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f264006cb30 msgr2=0x7f2640148d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:04.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.771+0000 7f263ffff640 1 --2- 172.21.15.32:0/2134895477 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f264006cb30 0x7f2640148d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:04.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.771+0000 7f263ffff640 1 -- 172.21.15.32:0/2134895477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26401492c0 con 0x7f264015b480 2024-04-24T23:03:04.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.771+0000 7f263ffff640 1 --2- 172.21.15.32:0/2134895477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f264015b480 0x7f2640157250 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f262c00c7f0 tx=0x7f262c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:04.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.772+0000 7f263d7fa640 1 -- 172.21.15.32:0/2134895477 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262c015070 con 0x7f264015b480 2024-04-24T23:03:04.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.772+0000 7f263d7fa640 1 -- 172.21.15.32:0/2134895477 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f262c007a30 con 0x7f264015b480 2024-04-24T23:03:04.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.772+0000 7f2646b34640 1 -- 172.21.15.32:0/2134895477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2640158040 con 0x7f264015b480 2024-04-24T23:03:04.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.772+0000 7f263d7fa640 1 -- 172.21.15.32:0/2134895477 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262c007e00 con 0x7f264015b480 2024-04-24T23:03:04.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.772+0000 7f2646b34640 1 -- 172.21.15.32:0/2134895477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f264014a6f0 con 0x7f264015b480 2024-04-24T23:03:04.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.773+0000 7f2646b34640 1 -- 172.21.15.32:0/2134895477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2640068430 con 0x7f264015b480 2024-04-24T23:03:04.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.773+0000 7f263d7fa640 1 -- 172.21.15.32:0/2134895477 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f262c018070 con 0x7f264015b480 2024-04-24T23:03:04.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.774+0000 7f263d7fa640 1 --2- 172.21.15.32:0/2134895477 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f262406be30 0x7f262406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:04.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.774+0000 7f263d7fa640 1 -- 172.21.15.32:0/2134895477 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7f262c08ade0 con 0x7f264015b480 2024-04-24T23:03:04.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.774+0000 7f26448d0640 1 --2- 172.21.15.32:0/2134895477 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f262406be30 0x7f262406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:04.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.774+0000 7f26448d0640 1 --2- 172.21.15.32:0/2134895477 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f262406be30 0x7f262406e2f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2634034000 tx=0x7f263402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:04.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.778+0000 7f263d7fa640 1 -- 172.21.15.32:0/2134895477 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f262c059790 con 0x7f264015b480 2024-04-24T23:03:04.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.960+0000 7f2646b34640 1 -- 172.21.15.32:0/2134895477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f26400b1d60 con 0x7f264015b480 2024-04-24T23:03:04.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.962+0000 7f263d7fa640 1 -- 172.21.15.32:0/2134895477 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+16620 (secure 0 0 0) 0x7f262c059090 con 0x7f264015b480 2024-04-24T23:03:04.963 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:03:04.963 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":87,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:03:02.276485+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:02:58.930222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-24T23:02:59.607070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:03:04.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.963+0000 7f2646b34640 1 -- 172.21.15.32:0/2134895477 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f262406be30 msgr2=0x7f262406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:04.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.963+0000 7f2646b34640 1 --2- 172.21.15.32:0/2134895477 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f262406be30 0x7f262406e2f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2634034000 tx=0x7f263402e000 comp rx=0 tx=0).stop 2024-04-24T23:03:04.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.963+0000 7f2646b34640 1 -- 172.21.15.32:0/2134895477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f264015b480 msgr2=0x7f2640157250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:04.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.963+0000 7f2646b34640 1 --2- 172.21.15.32:0/2134895477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f264015b480 0x7f2640157250 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f262c00c7f0 tx=0x7f262c00cbf0 comp rx=0 tx=0).stop 2024-04-24T23:03:04.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.964+0000 7f2646b34640 1 -- 172.21.15.32:0/2134895477 shutdown_connections 2024-04-24T23:03:04.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.964+0000 7f2646b34640 1 --2- 172.21.15.32:0/2134895477 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f262406be30 0x7f262406e2f0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:04.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.964+0000 7f2646b34640 1 --2- 172.21.15.32:0/2134895477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f264015b480 0x7f2640157250 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:04.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.964+0000 7f2646b34640 1 --2- 172.21.15.32:0/2134895477 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f264015fd60 0x7f2640071ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:04.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.964+0000 7f2646b34640 1 --2- 172.21.15.32:0/2134895477 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f264006cb30 0x7f2640148d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:04.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.964+0000 7f2646b34640 1 -- 172.21.15.32:0/2134895477 >> 172.21.15.32:0/2134895477 conn(0x7f264006b110 msgr2=0x7f26400d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:04.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.964+0000 7f2646b34640 1 -- 172.21.15.32:0/2134895477 shutdown_connections 2024-04-24T23:03:04.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:04.964+0000 7f2646b34640 1 -- 172.21.15.32:0/2134895477 wait complete. 2024-04-24T23:03:04.976 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-24T23:03:04.976 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:03:05.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.075+0000 7f7db83db640 1 Processor -- start 2024-04-24T23:03:05.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.075+0000 7f7db83db640 1 -- start start 2024-04-24T23:03:05.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.076+0000 7f7db83db640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7db0072cf0 0x7f7db006c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:05.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.076+0000 7f7db83db640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7db0071770 0x7f7db006cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:05.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.076+0000 7f7db83db640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7db015c990 con 0x7f7db015c5b0 2024-04-24T23:03:05.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.076+0000 7f7db83db640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f7db00b4740 con 0x7f7db0072cf0 2024-04-24T23:03:05.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.076+0000 7f7db83db640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f7db0072860 con 0x7f7db0071770 2024-04-24T23:03:05.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.077+0000 7f7db6177640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7db015c5b0 0x7f7db00b4030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57618/0 (socket says 172.21.15.32:57618) 2024-04-24T23:03:05.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.077+0000 7f7db6177640 1 -- 172.21.15.32:0/2956146311 learned_addr learned my addr 172.21.15.32:0/2956146311 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:03:05.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.077+0000 7f7db6978640 1 --2- 172.21.15.32:0/2956146311 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7db0071770 0x7f7db006cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:05.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.077+0000 7f7db5976640 1 --2- 172.21.15.32:0/2956146311 >> v2:172.21.15.121:3300/0 conn(0x7f7db0072cf0 0x7f7db006c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:05.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.077+0000 7f7db5175640 1 -- 172.21.15.32:0/2956146311 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2567888431 0 0) 0x7f7dac001110 con 0x7f7db015c5b0 2024-04-24T23:03:05.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.077+0000 7f7db5175640 1 -- 172.21.15.32:0/2956146311 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d9c003520 con 0x7f7db015c5b0 2024-04-24T23:03:05.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.077+0000 7f7db5175640 1 -- 172.21.15.32:0/2956146311 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1140926845 0 0) 0x7f7d9c003520 con 0x7f7db015c5b0 2024-04-24T23:03:05.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.077+0000 7f7db5175640 1 -- 172.21.15.32:0/2956146311 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7db0071770 msgr2=0x7f7db006cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:05.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.077+0000 7f7db5175640 1 --2- 172.21.15.32:0/2956146311 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7db0071770 0x7f7db006cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:05.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.077+0000 7f7db5175640 1 -- 172.21.15.32:0/2956146311 >> v2:172.21.15.121:3300/0 conn(0x7f7db0072cf0 msgr2=0x7f7db006c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:05.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.077+0000 7f7db5175640 1 --2- 172.21.15.32:0/2956146311 >> v2:172.21.15.121:3300/0 conn(0x7f7db0072cf0 0x7f7db006c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:05.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.077+0000 7f7db5175640 1 -- 172.21.15.32:0/2956146311 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7db0071c50 con 0x7f7db015c5b0 2024-04-24T23:03:05.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.077+0000 7f7db5175640 1 -- 172.21.15.32:0/2956146311 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f7dac001840 con 0x7f7db015c5b0 2024-04-24T23:03:05.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.077+0000 7f7db6978640 1 --2- 172.21.15.32:0/2956146311 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7db0071770 0x7f7db006cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:03:05.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.077+0000 7f7db5976640 1 --2- 172.21.15.32:0/2956146311 >> v2:172.21.15.121:3300/0 conn(0x7f7db0072cf0 0x7f7db006c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:03:05.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.077+0000 7f7db5175640 1 -- 172.21.15.32:0/2956146311 >> v1:172.21.15.32:6789/0 conn(0x7f7db015c5b0 legacy=0x7f7db00b4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:05.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.077+0000 7f7db5175640 1 --2- 172.21.15.32:0/2956146311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d9c0039b0 0x7f7d9c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:05.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.077+0000 7f7db5175640 1 --2- 172.21.15.32:0/2956146311 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7db0071770 0x7f7d9c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:05.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.077+0000 7f7db5175640 1 --2- 172.21.15.32:0/2956146311 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d9c004d40 0x7f7d9c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:05.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.077+0000 7f7db5175640 1 -- 172.21.15.32:0/2956146311 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7db0072860 con 0x7f7d9c0039b0 2024-04-24T23:03:05.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.077+0000 7f7db5175640 1 -- 172.21.15.32:0/2956146311 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9c00b9c0 con 0x7f7d9c004d40 2024-04-24T23:03:05.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.077+0000 7f7db5175640 1 -- 172.21.15.32:0/2956146311 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f7d9c00bb40 con 0x7f7db0071770 2024-04-24T23:03:05.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.077+0000 7f7db6177640 1 --2- 172.21.15.32:0/2956146311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d9c0039b0 0x7f7d9c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:05.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.077+0000 7f7db6978640 1 --2- 172.21.15.32:0/2956146311 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d9c004d40 0x7f7d9c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:05.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.077+0000 7f7db5976640 1 --2- 172.21.15.32:0/2956146311 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7db0071770 0x7f7d9c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:05.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.078+0000 7f7db6177640 1 -- 172.21.15.32:0/2956146311 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7db0071770 msgr2=0x7f7d9c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:05.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.078+0000 7f7db6177640 1 --2- 172.21.15.32:0/2956146311 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7db0071770 0x7f7d9c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:05.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.078+0000 7f7db6177640 1 -- 172.21.15.32:0/2956146311 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d9c004d40 msgr2=0x7f7d9c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:05.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.078+0000 7f7db6177640 1 --2- 172.21.15.32:0/2956146311 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d9c004d40 0x7f7d9c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:05.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.078+0000 7f7db6177640 1 -- 172.21.15.32:0/2956146311 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d9c003520 con 0x7f7d9c0039b0 2024-04-24T23:03:05.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.078+0000 7f7db6177640 1 --2- 172.21.15.32:0/2956146311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d9c0039b0 0x7f7d9c003dd0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f7dac00c3e0 tx=0x7f7d9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76a9ca08af78c391 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:05.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.079+0000 7f7db5175640 1 -- 172.21.15.32:0/2956146311 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dac032020 con 0x7f7d9c0039b0 2024-04-24T23:03:05.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.079+0000 7f7db5175640 1 -- 172.21.15.32:0/2956146311 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dac005af0 con 0x7f7d9c0039b0 2024-04-24T23:03:05.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.079+0000 7f7db5175640 1 -- 172.21.15.32:0/2956146311 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dac00c0e0 con 0x7f7d9c0039b0 2024-04-24T23:03:05.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.079+0000 7f7db83db640 1 -- 172.21.15.32:0/2956146311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d9c0039b0 msgr2=0x7f7d9c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:05.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.079+0000 7f7db83db640 1 --2- 172.21.15.32:0/2956146311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d9c0039b0 0x7f7d9c003dd0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f7dac00c3e0 tx=0x7f7d9c005830 comp rx=0 tx=0).stop 2024-04-24T23:03:05.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.079+0000 7f7db6978640 1 -- 172.21.15.32:0/2956146311 reap_dead start 2024-04-24T23:03:05.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.079+0000 7f7db83db640 1 -- 172.21.15.32:0/2956146311 shutdown_connections 2024-04-24T23:03:05.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.079+0000 7f7db83db640 1 -- 172.21.15.32:0/2956146311 >> 172.21.15.32:0/2956146311 conn(0x7f7db006b110 msgr2=0x7f7db006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:05.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.079+0000 7f7db83db640 1 -- 172.21.15.32:0/2956146311 shutdown_connections 2024-04-24T23:03:05.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.079+0000 7f7db83db640 1 -- 172.21.15.32:0/2956146311 wait complete. 2024-04-24T23:03:05.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.080+0000 7f7db83db640 1 Processor -- start 2024-04-24T23:03:05.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.080+0000 7f7db83db640 1 -- start start 2024-04-24T23:03:05.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.081+0000 7f7db83db640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d9c0039b0 0x7f7db00cd750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:05.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.081+0000 7f7db83db640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7db00cf060 0x7f7db00cf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:05.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.081+0000 7f7db83db640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7db014e920 0x7f7db014eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:05.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.081+0000 7f7db83db640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7db00724f0 con 0x7f7d9c0039b0 2024-04-24T23:03:05.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.081+0000 7f7db83db640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7db0072210 con 0x7f7db014e920 2024-04-24T23:03:05.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.081+0000 7f7db83db640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f7db006cb30 con 0x7f7db00cf060 2024-04-24T23:03:05.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.081+0000 7f7db6177640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d9c0039b0 0x7f7db00cd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:05.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.081+0000 7f7db6177640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d9c0039b0 0x7f7db00cd750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38458/0 (socket says 172.21.15.32:38458) 2024-04-24T23:03:05.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.081+0000 7f7db6177640 1 -- 172.21.15.32:0/1003208494 learned_addr learned my addr 172.21.15.32:0/1003208494 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:05.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.081+0000 7f7db6177640 1 -- 172.21.15.32:0/1003208494 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7db00cf060 msgr2=0x7f7db00cf4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:05.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.081+0000 7f7db6177640 1 --2- 172.21.15.32:0/1003208494 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7db00cf060 0x7f7db00cf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:05.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.081+0000 7f7db6177640 1 -- 172.21.15.32:0/1003208494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7db014e920 msgr2=0x7f7db014eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:05.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.081+0000 7f7db6177640 1 --2- 172.21.15.32:0/1003208494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7db014e920 0x7f7db014eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:05.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.081+0000 7f7db6177640 1 -- 172.21.15.32:0/1003208494 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7db014f2e0 con 0x7f7d9c0039b0 2024-04-24T23:03:05.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.082+0000 7f7db6177640 1 --2- 172.21.15.32:0/1003208494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d9c0039b0 0x7f7db00cd750 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f7dac031040 tx=0x7f7db0071e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:05.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.082+0000 7f7da6ffd640 1 -- 172.21.15.32:0/1003208494 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dac032030 con 0x7f7d9c0039b0 2024-04-24T23:03:05.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.082+0000 7f7da6ffd640 1 -- 172.21.15.32:0/1003208494 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dac004cb0 con 0x7f7d9c0039b0 2024-04-24T23:03:05.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.082+0000 7f7da6ffd640 1 -- 172.21.15.32:0/1003208494 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dac004e60 con 0x7f7d9c0039b0 2024-04-24T23:03:05.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.082+0000 7f7db83db640 1 -- 172.21.15.32:0/1003208494 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7db00cdfa0 con 0x7f7d9c0039b0 2024-04-24T23:03:05.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.082+0000 7f7db83db640 1 -- 172.21.15.32:0/1003208494 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7db01526b0 con 0x7f7d9c0039b0 2024-04-24T23:03:05.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.083+0000 7f7db83db640 1 -- 172.21.15.32:0/1003208494 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7db0068430 con 0x7f7d9c0039b0 2024-04-24T23:03:05.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.083+0000 7f7da6ffd640 1 -- 172.21.15.32:0/1003208494 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7dac035020 con 0x7f7d9c0039b0 2024-04-24T23:03:05.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.084+0000 7f7da6ffd640 1 --2- 172.21.15.32:0/1003208494 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7d9406be30 0x7f7d9406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:05.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.084+0000 7f7da6ffd640 1 -- 172.21.15.32:0/1003208494 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7f7dac0ab5a0 con 0x7f7d9c0039b0 2024-04-24T23:03:05.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.084+0000 7f7db5976640 1 --2- 172.21.15.32:0/1003208494 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7d9406be30 0x7f7d9406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:05.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.085+0000 7f7db5976640 1 --2- 172.21.15.32:0/1003208494 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7d9406be30 0x7f7d9406e2f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7db014f520 tx=0x7f7da0006c30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:05.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.088+0000 7f7da6ffd640 1 -- 172.21.15.32:0/1003208494 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7dac047060 con 0x7f7d9c0039b0 2024-04-24T23:03:05.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.268+0000 7f7db83db640 1 -- 172.21.15.32:0/1003208494 --> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7db00ce4c0 con 0x7f7d9406be30 2024-04-24T23:03:05.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.274+0000 7f7da6ffd640 1 -- 172.21.15.32:0/1003208494 <== mgr.4108 v2:172.21.15.121:6832/1848157943 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247524 (secure 0 0 0) 0x7f7db00ce4c0 con 0x7f7d9406be30 2024-04-24T23:03:05.279 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:03:05.280 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-24T23:03:05.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.277+0000 7f7db83db640 1 -- 172.21.15.32:0/1003208494 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7d9406be30 msgr2=0x7f7d9406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:05.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.277+0000 7f7db83db640 1 --2- 172.21.15.32:0/1003208494 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7d9406be30 0x7f7d9406e2f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7db014f520 tx=0x7f7da0006c30 comp rx=0 tx=0).stop 2024-04-24T23:03:05.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.277+0000 7f7db83db640 1 -- 172.21.15.32:0/1003208494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d9c0039b0 msgr2=0x7f7db00cd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:05.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.277+0000 7f7db83db640 1 --2- 172.21.15.32:0/1003208494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d9c0039b0 0x7f7db00cd750 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f7dac031040 tx=0x7f7db0071e20 comp rx=0 tx=0).stop 2024-04-24T23:03:05.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.278+0000 7f7db83db640 1 -- 172.21.15.32:0/1003208494 shutdown_connections 2024-04-24T23:03:05.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.278+0000 7f7db83db640 1 --2- 172.21.15.32:0/1003208494 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7d9406be30 0x7f7d9406e2f0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:05.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.278+0000 7f7db83db640 1 --2- 172.21.15.32:0/1003208494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7db014e920 0x7f7db014eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:05.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.278+0000 7f7db83db640 1 --2- 172.21.15.32:0/1003208494 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7db00cf060 0x7f7db00cf4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:05.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.278+0000 7f7db83db640 1 --2- 172.21.15.32:0/1003208494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d9c0039b0 0x7f7db00cd750 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:05.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.278+0000 7f7db83db640 1 -- 172.21.15.32:0/1003208494 >> 172.21.15.32:0/1003208494 conn(0x7f7db006b110 msgr2=0x7f7db015ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:05.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.278+0000 7f7db83db640 1 -- 172.21.15.32:0/1003208494 shutdown_connections 2024-04-24T23:03:05.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.278+0000 7f7db83db640 1 -- 172.21.15.32:0/1003208494 wait complete. 2024-04-24T23:03:05.290 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":205,"stamp":"2024-04-24T23:03:05.182532+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":99,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":4731,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1557768,"statfs":{"total":1600000000,"available":1595154465,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.923742"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.288518+0000","last_change":"2024-04-24T23:03:01.287997+0000","last_active":"2024-04-24T23:03:01.288518+0000","last_peered":"2024-04-24T23:03:01.288518+0000","last_clean":"2024-04-24T23:03:01.288518+0000","last_became_active":"2024-04-24T23:03:01.285345+0000","last_became_peered":"2024-04-24T23:03:01.285345+0000","last_unstale":"2024-04-24T23:03:01.288518+0000","last_undegraded":"2024-04-24T23:03:01.288518+0000","last_fullsized":"2024-04-24T23:03:01.288518+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:53:18.209022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.288041+0000","last_change":"2024-04-24T23:03:01.287833+0000","last_active":"2024-04-24T23:03:01.288041+0000","last_peered":"2024-04-24T23:03:01.288041+0000","last_clean":"2024-04-24T23:03:01.288041+0000","last_became_active":"2024-04-24T23:03:01.287515+0000","last_became_peered":"2024-04-24T23:03:01.287515+0000","last_unstale":"2024-04-24T23:03:01.288041+0000","last_undegraded":"2024-04-24T23:03:01.288041+0000","last_fullsized":"2024-04-24T23:03:01.288041+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:56:52.361384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.294494+0000","last_change":"2024-04-24T23:03:01.294363+0000","last_active":"2024-04-24T23:03:01.294494+0000","last_peered":"2024-04-24T23:03:01.294494+0000","last_clean":"2024-04-24T23:03:01.294494+0000","last_became_active":"2024-04-24T23:03:01.293671+0000","last_became_peered":"2024-04-24T23:03:01.293671+0000","last_unstale":"2024-04-24T23:03:01.294494+0000","last_undegraded":"2024-04-24T23:03:01.294494+0000","last_fullsized":"2024-04-24T23:03:01.294494+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:39:46.041609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.294861+0000","last_change":"2024-04-24T23:03:01.294730+0000","last_active":"2024-04-24T23:03:01.294861+0000","last_peered":"2024-04-24T23:03:01.294861+0000","last_clean":"2024-04-24T23:03:01.294861+0000","last_became_active":"2024-04-24T23:03:01.294460+0000","last_became_peered":"2024-04-24T23:03:01.294460+0000","last_unstale":"2024-04-24T23:03:01.294861+0000","last_undegraded":"2024-04-24T23:03:01.294861+0000","last_fullsized":"2024-04-24T23:03:01.294861+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:58.260140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.289993+0000","last_change":"2024-04-24T23:03:01.289873+0000","last_active":"2024-04-24T23:03:01.289993+0000","last_peered":"2024-04-24T23:03:01.289993+0000","last_clean":"2024-04-24T23:03:01.289993+0000","last_became_active":"2024-04-24T23:03:01.289591+0000","last_became_peered":"2024-04-24T23:03:01.289591+0000","last_unstale":"2024-04-24T23:03:01.289993+0000","last_undegraded":"2024-04-24T23:03:01.289993+0000","last_fullsized":"2024-04-24T23:03:01.289993+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:23:22.139609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.291856+0000","last_change":"2024-04-24T23:03:01.291161+0000","last_active":"2024-04-24T23:03:01.291856+0000","last_peered":"2024-04-24T23:03:01.291856+0000","last_clean":"2024-04-24T23:03:01.291856+0000","last_became_active":"2024-04-24T23:03:01.290732+0000","last_became_peered":"2024-04-24T23:03:01.290732+0000","last_unstale":"2024-04-24T23:03:01.291856+0000","last_undegraded":"2024-04-24T23:03:01.291856+0000","last_fullsized":"2024-04-24T23:03:01.291856+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:07:11.209559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.286069+0000","last_change":"2024-04-24T23:03:01.285942+0000","last_active":"2024-04-24T23:03:01.286069+0000","last_peered":"2024-04-24T23:03:01.286069+0000","last_clean":"2024-04-24T23:03:01.286069+0000","last_became_active":"2024-04-24T23:03:01.285678+0000","last_became_peered":"2024-04-24T23:03:01.285678+0000","last_unstale":"2024-04-24T23:03:01.286069+0000","last_undegraded":"2024-04-24T23:03:01.286069+0000","last_fullsized":"2024-04-24T23:03:01.286069+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:08.412117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.291029+0000","last_change":"2024-04-24T23:03:01.290909+0000","last_active":"2024-04-24T23:03:01.291029+0000","last_peered":"2024-04-24T23:03:01.291029+0000","last_clean":"2024-04-24T23:03:01.291029+0000","last_became_active":"2024-04-24T23:03:01.290455+0000","last_became_peered":"2024-04-24T23:03:01.290455+0000","last_unstale":"2024-04-24T23:03:01.291029+0000","last_undegraded":"2024-04-24T23:03:01.291029+0000","last_fullsized":"2024-04-24T23:03:01.291029+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:56:27.406947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.287693+0000","last_change":"2024-04-24T23:03:01.287556+0000","last_active":"2024-04-24T23:03:01.287693+0000","last_peered":"2024-04-24T23:03:01.287693+0000","last_clean":"2024-04-24T23:03:01.287693+0000","last_became_active":"2024-04-24T23:03:01.287309+0000","last_became_peered":"2024-04-24T23:03:01.287309+0000","last_unstale":"2024-04-24T23:03:01.287693+0000","last_undegraded":"2024-04-24T23:03:01.287693+0000","last_fullsized":"2024-04-24T23:03:01.287693+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:58:02.114473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.294320+0000","last_change":"2024-04-24T23:03:01.294147+0000","last_active":"2024-04-24T23:03:01.294320+0000","last_peered":"2024-04-24T23:03:01.294320+0000","last_clean":"2024-04-24T23:03:01.294320+0000","last_became_active":"2024-04-24T23:03:01.292898+0000","last_became_peered":"2024-04-24T23:03:01.292898+0000","last_unstale":"2024-04-24T23:03:01.294320+0000","last_undegraded":"2024-04-24T23:03:01.294320+0000","last_fullsized":"2024-04-24T23:03:01.294320+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:20:57.212758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.288501+0000","last_change":"2024-04-24T23:03:01.287783+0000","last_active":"2024-04-24T23:03:01.288501+0000","last_peered":"2024-04-24T23:03:01.288501+0000","last_clean":"2024-04-24T23:03:01.288501+0000","last_became_active":"2024-04-24T23:03:01.284879+0000","last_became_peered":"2024-04-24T23:03:01.284879+0000","last_unstale":"2024-04-24T23:03:01.288501+0000","last_undegraded":"2024-04-24T23:03:01.288501+0000","last_fullsized":"2024-04-24T23:03:01.288501+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:47:33.729406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.291839+0000","last_change":"2024-04-24T23:03:01.291585+0000","last_active":"2024-04-24T23:03:01.291839+0000","last_peered":"2024-04-24T23:03:01.291839+0000","last_clean":"2024-04-24T23:03:01.291839+0000","last_became_active":"2024-04-24T23:03:01.291366+0000","last_became_peered":"2024-04-24T23:03:01.291366+0000","last_unstale":"2024-04-24T23:03:01.291839+0000","last_undegraded":"2024-04-24T23:03:01.291839+0000","last_fullsized":"2024-04-24T23:03:01.291839+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:32:52.765176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.294695+0000","last_change":"2024-04-24T23:03:01.294589+0000","last_active":"2024-04-24T23:03:01.294695+0000","last_peered":"2024-04-24T23:03:01.294695+0000","last_clean":"2024-04-24T23:03:01.294695+0000","last_became_active":"2024-04-24T23:03:01.294103+0000","last_became_peered":"2024-04-24T23:03:01.294103+0000","last_unstale":"2024-04-24T23:03:01.294695+0000","last_undegraded":"2024-04-24T23:03:01.294695+0000","last_fullsized":"2024-04-24T23:03:01.294695+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:31.145086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.295410+0000","last_change":"2024-04-24T23:03:01.295302+0000","last_active":"2024-04-24T23:03:01.295410+0000","last_peered":"2024-04-24T23:03:01.295410+0000","last_clean":"2024-04-24T23:03:01.295410+0000","last_became_active":"2024-04-24T23:03:01.294840+0000","last_became_peered":"2024-04-24T23:03:01.294840+0000","last_unstale":"2024-04-24T23:03:01.295410+0000","last_undegraded":"2024-04-24T23:03:01.295410+0000","last_fullsized":"2024-04-24T23:03:01.295410+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:28:36.529735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.284589+0000","last_change":"2024-04-24T23:03:01.284429+0000","last_active":"2024-04-24T23:03:01.284589+0000","last_peered":"2024-04-24T23:03:01.284589+0000","last_clean":"2024-04-24T23:03:01.284589+0000","last_became_active":"2024-04-24T23:03:01.284164+0000","last_became_peered":"2024-04-24T23:03:01.284164+0000","last_unstale":"2024-04-24T23:03:01.284589+0000","last_undegraded":"2024-04-24T23:03:01.284589+0000","last_fullsized":"2024-04-24T23:03:01.284589+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:17:09.127180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.290893+0000","last_change":"2024-04-24T23:03:01.290183+0000","last_active":"2024-04-24T23:03:01.290893+0000","last_peered":"2024-04-24T23:03:01.290893+0000","last_clean":"2024-04-24T23:03:01.290893+0000","last_became_active":"2024-04-24T23:03:01.289155+0000","last_became_peered":"2024-04-24T23:03:01.289155+0000","last_unstale":"2024-04-24T23:03:01.290893+0000","last_undegraded":"2024-04-24T23:03:01.290893+0000","last_fullsized":"2024-04-24T23:03:01.290893+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:42:33.723426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.285906+0000","last_change":"2024-04-24T23:03:01.285797+0000","last_active":"2024-04-24T23:03:01.285906+0000","last_peered":"2024-04-24T23:03:01.285906+0000","last_clean":"2024-04-24T23:03:01.285906+0000","last_became_active":"2024-04-24T23:03:01.285281+0000","last_became_peered":"2024-04-24T23:03:01.285281+0000","last_unstale":"2024-04-24T23:03:01.285906+0000","last_undegraded":"2024-04-24T23:03:01.285906+0000","last_fullsized":"2024-04-24T23:03:01.285906+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:25:37.338406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.290442+0000","last_change":"2024-04-24T23:03:01.290320+0000","last_active":"2024-04-24T23:03:01.290442+0000","last_peered":"2024-04-24T23:03:01.290442+0000","last_clean":"2024-04-24T23:03:01.290442+0000","last_became_active":"2024-04-24T23:03:01.289055+0000","last_became_peered":"2024-04-24T23:03:01.289055+0000","last_unstale":"2024-04-24T23:03:01.290442+0000","last_undegraded":"2024-04-24T23:03:01.290442+0000","last_fullsized":"2024-04-24T23:03:01.290442+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:03:29.101082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.288369+0000","last_change":"2024-04-24T23:03:01.286441+0000","last_active":"2024-04-24T23:03:01.288369+0000","last_peered":"2024-04-24T23:03:01.288369+0000","last_clean":"2024-04-24T23:03:01.288369+0000","last_became_active":"2024-04-24T23:03:01.285596+0000","last_became_peered":"2024-04-24T23:03:01.285596+0000","last_unstale":"2024-04-24T23:03:01.288369+0000","last_undegraded":"2024-04-24T23:03:01.288369+0000","last_fullsized":"2024-04-24T23:03:01.288369+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:05:10.160284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.287301+0000","last_change":"2024-04-24T23:03:01.287161+0000","last_active":"2024-04-24T23:03:01.287301+0000","last_peered":"2024-04-24T23:03:01.287301+0000","last_clean":"2024-04-24T23:03:01.287301+0000","last_became_active":"2024-04-24T23:03:01.286880+0000","last_became_peered":"2024-04-24T23:03:01.286880+0000","last_unstale":"2024-04-24T23:03:01.287301+0000","last_undegraded":"2024-04-24T23:03:01.287301+0000","last_fullsized":"2024-04-24T23:03:01.287301+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:18:18.763038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.288825+0000","last_change":"2024-04-24T23:03:01.288705+0000","last_active":"2024-04-24T23:03:01.288825+0000","last_peered":"2024-04-24T23:03:01.288825+0000","last_clean":"2024-04-24T23:03:01.288825+0000","last_became_active":"2024-04-24T23:03:01.288448+0000","last_became_peered":"2024-04-24T23:03:01.288448+0000","last_unstale":"2024-04-24T23:03:01.288825+0000","last_undegraded":"2024-04-24T23:03:01.288825+0000","last_fullsized":"2024-04-24T23:03:01.288825+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:30:36.497458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.287084+0000","last_change":"2024-04-24T23:03:01.286961+0000","last_active":"2024-04-24T23:03:01.287084+0000","last_peered":"2024-04-24T23:03:01.287084+0000","last_clean":"2024-04-24T23:03:01.287084+0000","last_became_active":"2024-04-24T23:03:01.286716+0000","last_became_peered":"2024-04-24T23:03:01.286716+0000","last_unstale":"2024-04-24T23:03:01.287084+0000","last_undegraded":"2024-04-24T23:03:01.287084+0000","last_fullsized":"2024-04-24T23:03:01.287084+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:32:49.520863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.298171+0000","last_change":"2024-04-24T23:03:01.298000+0000","last_active":"2024-04-24T23:03:01.298171+0000","last_peered":"2024-04-24T23:03:01.298171+0000","last_clean":"2024-04-24T23:03:01.298171+0000","last_became_active":"2024-04-24T23:03:01.294249+0000","last_became_peered":"2024-04-24T23:03:01.294249+0000","last_unstale":"2024-04-24T23:03:01.298171+0000","last_undegraded":"2024-04-24T23:03:01.298171+0000","last_fullsized":"2024-04-24T23:03:01.298171+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:26:35.747390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.288406+0000","last_change":"2024-04-24T23:03:01.286637+0000","last_active":"2024-04-24T23:03:01.288406+0000","last_peered":"2024-04-24T23:03:01.288406+0000","last_clean":"2024-04-24T23:03:01.288406+0000","last_became_active":"2024-04-24T23:03:01.286078+0000","last_became_peered":"2024-04-24T23:03:01.286078+0000","last_unstale":"2024-04-24T23:03:01.288406+0000","last_undegraded":"2024-04-24T23:03:01.288406+0000","last_fullsized":"2024-04-24T23:03:01.288406+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:19:34.997934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.296320+0000","last_change":"2024-04-24T23:03:01.296132+0000","last_active":"2024-04-24T23:03:01.296320+0000","last_peered":"2024-04-24T23:03:01.296320+0000","last_clean":"2024-04-24T23:03:01.296320+0000","last_became_active":"2024-04-24T23:03:01.294583+0000","last_became_peered":"2024-04-24T23:03:01.294583+0000","last_unstale":"2024-04-24T23:03:01.296320+0000","last_undegraded":"2024-04-24T23:03:01.296320+0000","last_fullsized":"2024-04-24T23:03:01.296320+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:58:27.227491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.297365+0000","last_change":"2024-04-24T23:03:01.297093+0000","last_active":"2024-04-24T23:03:01.297365+0000","last_peered":"2024-04-24T23:03:01.297365+0000","last_clean":"2024-04-24T23:03:01.297365+0000","last_became_active":"2024-04-24T23:03:01.296835+0000","last_became_peered":"2024-04-24T23:03:01.296835+0000","last_unstale":"2024-04-24T23:03:01.297365+0000","last_undegraded":"2024-04-24T23:03:01.297365+0000","last_fullsized":"2024-04-24T23:03:01.297365+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:20:50.261924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.289391+0000","last_change":"2024-04-24T23:03:01.289214+0000","last_active":"2024-04-24T23:03:01.289391+0000","last_peered":"2024-04-24T23:03:01.289391+0000","last_clean":"2024-04-24T23:03:01.289391+0000","last_became_active":"2024-04-24T23:03:01.288966+0000","last_became_peered":"2024-04-24T23:03:01.288966+0000","last_unstale":"2024-04-24T23:03:01.289391+0000","last_undegraded":"2024-04-24T23:03:01.289391+0000","last_fullsized":"2024-04-24T23:03:01.289391+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:33.116279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.287854+0000","last_change":"2024-04-24T23:03:01.287682+0000","last_active":"2024-04-24T23:03:01.287854+0000","last_peered":"2024-04-24T23:03:01.287854+0000","last_clean":"2024-04-24T23:03:01.287854+0000","last_became_active":"2024-04-24T23:03:01.287343+0000","last_became_peered":"2024-04-24T23:03:01.287343+0000","last_unstale":"2024-04-24T23:03:01.287854+0000","last_undegraded":"2024-04-24T23:03:01.287854+0000","last_fullsized":"2024-04-24T23:03:01.287854+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:56:48.889426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.284882+0000","last_change":"2024-04-24T23:03:01.284749+0000","last_active":"2024-04-24T23:03:01.284882+0000","last_peered":"2024-04-24T23:03:01.284882+0000","last_clean":"2024-04-24T23:03:01.284882+0000","last_became_active":"2024-04-24T23:03:01.284512+0000","last_became_peered":"2024-04-24T23:03:01.284512+0000","last_unstale":"2024-04-24T23:03:01.284882+0000","last_undegraded":"2024-04-24T23:03:01.284882+0000","last_fullsized":"2024-04-24T23:03:01.284882+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:59:35.976941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.288474+0000","last_change":"2024-04-24T23:03:01.287311+0000","last_active":"2024-04-24T23:03:01.288474+0000","last_peered":"2024-04-24T23:03:01.288474+0000","last_clean":"2024-04-24T23:03:01.288474+0000","last_became_active":"2024-04-24T23:03:01.286497+0000","last_became_peered":"2024-04-24T23:03:01.286497+0000","last_unstale":"2024-04-24T23:03:01.288474+0000","last_undegraded":"2024-04-24T23:03:01.288474+0000","last_fullsized":"2024-04-24T23:03:01.288474+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:37.134636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.292497+0000","last_change":"2024-04-24T23:03:01.292358+0000","last_active":"2024-04-24T23:03:01.292497+0000","last_peered":"2024-04-24T23:03:01.292497+0000","last_clean":"2024-04-24T23:03:01.292497+0000","last_became_active":"2024-04-24T23:03:01.292123+0000","last_became_peered":"2024-04-24T23:03:01.292123+0000","last_unstale":"2024-04-24T23:03:01.292497+0000","last_undegraded":"2024-04-24T23:03:01.292497+0000","last_fullsized":"2024-04-24T23:03:01.292497+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:34:14.584173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.291792+0000","last_change":"2024-04-24T23:03:01.291660+0000","last_active":"2024-04-24T23:03:01.291792+0000","last_peered":"2024-04-24T23:03:01.291792+0000","last_clean":"2024-04-24T23:03:01.291792+0000","last_became_active":"2024-04-24T23:03:01.291396+0000","last_became_peered":"2024-04-24T23:03:01.291396+0000","last_unstale":"2024-04-24T23:03:01.291792+0000","last_undegraded":"2024-04-24T23:03:01.291792+0000","last_fullsized":"2024-04-24T23:03:01.291792+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:08:31.720241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.289555+0000","last_change":"2024-04-24T23:03:01.289384+0000","last_active":"2024-04-24T23:03:01.289555+0000","last_peered":"2024-04-24T23:03:01.289555+0000","last_clean":"2024-04-24T23:03:01.289555+0000","last_became_active":"2024-04-24T23:03:01.289061+0000","last_became_peered":"2024-04-24T23:03:01.289061+0000","last_unstale":"2024-04-24T23:03:01.289555+0000","last_undegraded":"2024-04-24T23:03:01.289555+0000","last_fullsized":"2024-04-24T23:03:01.289555+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:15.537113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"86'4","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.307358+0000","last_change":"2024-04-24T23:03:00.273081+0000","last_active":"2024-04-24T23:03:01.307358+0000","last_peered":"2024-04-24T23:03:01.307358+0000","last_clean":"2024-04-24T23:03:01.307358+0000","last_became_active":"2024-04-24T23:03:00.272403+0000","last_became_peered":"2024-04-24T23:03:00.272403+0000","last_unstale":"2024-04-24T23:03:01.307358+0000","last_undegraded":"2024-04-24T23:03:01.307358+0000","last_fullsized":"2024-04-24T23:03:01.307358+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:02:45.041959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.288562+0000","last_change":"2024-04-24T23:03:01.288421+0000","last_active":"2024-04-24T23:03:01.288562+0000","last_peered":"2024-04-24T23:03:01.288562+0000","last_clean":"2024-04-24T23:03:01.288562+0000","last_became_active":"2024-04-24T23:03:01.287681+0000","last_became_peered":"2024-04-24T23:03:01.287681+0000","last_unstale":"2024-04-24T23:03:01.288562+0000","last_undegraded":"2024-04-24T23:03:01.288562+0000","last_fullsized":"2024-04-24T23:03:01.288562+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:53:37.318204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.305530+0000","last_change":"2024-04-24T23:03:00.270732+0000","last_active":"2024-04-24T23:03:01.305530+0000","last_peered":"2024-04-24T23:03:01.305530+0000","last_clean":"2024-04-24T23:03:01.305530+0000","last_became_active":"2024-04-24T23:03:00.270403+0000","last_became_peered":"2024-04-24T23:03:00.270403+0000","last_unstale":"2024-04-24T23:03:01.305530+0000","last_undegraded":"2024-04-24T23:03:01.305530+0000","last_fullsized":"2024-04-24T23:03:01.305530+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:16.585968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.291834+0000","last_change":"2024-04-24T23:03:01.290966+0000","last_active":"2024-04-24T23:03:01.291834+0000","last_peered":"2024-04-24T23:03:01.291834+0000","last_clean":"2024-04-24T23:03:01.291834+0000","last_became_active":"2024-04-24T23:03:01.290354+0000","last_became_peered":"2024-04-24T23:03:01.290354+0000","last_unstale":"2024-04-24T23:03:01.291834+0000","last_undegraded":"2024-04-24T23:03:01.291834+0000","last_fullsized":"2024-04-24T23:03:01.291834+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:34:27.208184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.305962+0000","last_change":"2024-04-24T23:03:00.268939+0000","last_active":"2024-04-24T23:03:01.305962+0000","last_peered":"2024-04-24T23:03:01.305962+0000","last_clean":"2024-04-24T23:03:01.305962+0000","last_became_active":"2024-04-24T23:03:00.268698+0000","last_became_peered":"2024-04-24T23:03:00.268698+0000","last_unstale":"2024-04-24T23:03:01.305962+0000","last_undegraded":"2024-04-24T23:03:01.305962+0000","last_fullsized":"2024-04-24T23:03:01.305962+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:58:20.761137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.295966+0000","last_change":"2024-04-24T23:03:01.295826+0000","last_active":"2024-04-24T23:03:01.295966+0000","last_peered":"2024-04-24T23:03:01.295966+0000","last_clean":"2024-04-24T23:03:01.295966+0000","last_became_active":"2024-04-24T23:03:01.293764+0000","last_became_peered":"2024-04-24T23:03:01.293764+0000","last_unstale":"2024-04-24T23:03:01.295966+0000","last_undegraded":"2024-04-24T23:03:01.295966+0000","last_fullsized":"2024-04-24T23:03:01.295966+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:30:58.068795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.304856+0000","last_change":"2024-04-24T23:03:00.267222+0000","last_active":"2024-04-24T23:03:01.304856+0000","last_peered":"2024-04-24T23:03:01.304856+0000","last_clean":"2024-04-24T23:03:01.304856+0000","last_became_active":"2024-04-24T23:03:00.266899+0000","last_became_peered":"2024-04-24T23:03:00.266899+0000","last_unstale":"2024-04-24T23:03:01.304856+0000","last_undegraded":"2024-04-24T23:03:01.304856+0000","last_fullsized":"2024-04-24T23:03:01.304856+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:12:37.789738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.290924+0000","last_change":"2024-04-24T23:03:01.290685+0000","last_active":"2024-04-24T23:03:01.290924+0000","last_peered":"2024-04-24T23:03:01.290924+0000","last_clean":"2024-04-24T23:03:01.290924+0000","last_became_active":"2024-04-24T23:03:01.290127+0000","last_became_peered":"2024-04-24T23:03:01.290127+0000","last_unstale":"2024-04-24T23:03:01.290924+0000","last_undegraded":"2024-04-24T23:03:01.290924+0000","last_fullsized":"2024-04-24T23:03:01.290924+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:32:12.500767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.306194+0000","last_change":"2024-04-24T23:03:00.273984+0000","last_active":"2024-04-24T23:03:01.306194+0000","last_peered":"2024-04-24T23:03:01.306194+0000","last_clean":"2024-04-24T23:03:01.306194+0000","last_became_active":"2024-04-24T23:03:00.273768+0000","last_became_peered":"2024-04-24T23:03:00.273768+0000","last_unstale":"2024-04-24T23:03:01.306194+0000","last_undegraded":"2024-04-24T23:03:01.306194+0000","last_fullsized":"2024-04-24T23:03:01.306194+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:25:11.779263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.286419+0000","last_change":"2024-04-24T23:03:01.286217+0000","last_active":"2024-04-24T23:03:01.286419+0000","last_peered":"2024-04-24T23:03:01.286419+0000","last_clean":"2024-04-24T23:03:01.286419+0000","last_became_active":"2024-04-24T23:03:01.285959+0000","last_became_peered":"2024-04-24T23:03:01.285959+0000","last_unstale":"2024-04-24T23:03:01.286419+0000","last_undegraded":"2024-04-24T23:03:01.286419+0000","last_fullsized":"2024-04-24T23:03:01.286419+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:49.422527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.307044+0000","last_change":"2024-04-24T23:03:00.269391+0000","last_active":"2024-04-24T23:03:01.307044+0000","last_peered":"2024-04-24T23:03:01.307044+0000","last_clean":"2024-04-24T23:03:01.307044+0000","last_became_active":"2024-04-24T23:03:00.268694+0000","last_became_peered":"2024-04-24T23:03:00.268694+0000","last_unstale":"2024-04-24T23:03:01.307044+0000","last_undegraded":"2024-04-24T23:03:01.307044+0000","last_fullsized":"2024-04-24T23:03:01.307044+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:21:17.505190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.286029+0000","last_change":"2024-04-24T23:03:01.285357+0000","last_active":"2024-04-24T23:03:01.286029+0000","last_peered":"2024-04-24T23:03:01.286029+0000","last_clean":"2024-04-24T23:03:01.286029+0000","last_became_active":"2024-04-24T23:03:01.285092+0000","last_became_peered":"2024-04-24T23:03:01.285092+0000","last_unstale":"2024-04-24T23:03:01.286029+0000","last_undegraded":"2024-04-24T23:03:01.286029+0000","last_fullsized":"2024-04-24T23:03:01.286029+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:11.553958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.278721+0000","last_change":"2024-04-24T23:03:00.273292+0000","last_active":"2024-04-24T23:03:01.278721+0000","last_peered":"2024-04-24T23:03:01.278721+0000","last_clean":"2024-04-24T23:03:01.278721+0000","last_became_active":"2024-04-24T23:03:00.273011+0000","last_became_peered":"2024-04-24T23:03:00.273011+0000","last_unstale":"2024-04-24T23:03:01.278721+0000","last_undegraded":"2024-04-24T23:03:01.278721+0000","last_fullsized":"2024-04-24T23:03:01.278721+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:09:31.631003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.295234+0000","last_change":"2024-04-24T23:03:01.295108+0000","last_active":"2024-04-24T23:03:01.295234+0000","last_peered":"2024-04-24T23:03:01.295234+0000","last_clean":"2024-04-24T23:03:01.295234+0000","last_became_active":"2024-04-24T23:03:01.294522+0000","last_became_peered":"2024-04-24T23:03:01.294522+0000","last_unstale":"2024-04-24T23:03:01.295234+0000","last_undegraded":"2024-04-24T23:03:01.295234+0000","last_fullsized":"2024-04-24T23:03:01.295234+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:04:42.673547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.279009+0000","last_change":"2024-04-24T23:03:00.270321+0000","last_active":"2024-04-24T23:03:01.279009+0000","last_peered":"2024-04-24T23:03:01.279009+0000","last_clean":"2024-04-24T23:03:01.279009+0000","last_became_active":"2024-04-24T23:03:00.269233+0000","last_became_peered":"2024-04-24T23:03:00.269233+0000","last_unstale":"2024-04-24T23:03:01.279009+0000","last_undegraded":"2024-04-24T23:03:01.279009+0000","last_fullsized":"2024-04-24T23:03:01.279009+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:38:10.621299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.294009+0000","last_change":"2024-04-24T23:03:01.293480+0000","last_active":"2024-04-24T23:03:01.294009+0000","last_peered":"2024-04-24T23:03:01.294009+0000","last_clean":"2024-04-24T23:03:01.294009+0000","last_became_active":"2024-04-24T23:03:01.293076+0000","last_became_peered":"2024-04-24T23:03:01.293076+0000","last_unstale":"2024-04-24T23:03:01.294009+0000","last_undegraded":"2024-04-24T23:03:01.294009+0000","last_fullsized":"2024-04-24T23:03:01.294009+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:03:28.843850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.279314+0000","last_change":"2024-04-24T23:03:00.269513+0000","last_active":"2024-04-24T23:03:01.279314+0000","last_peered":"2024-04-24T23:03:01.279314+0000","last_clean":"2024-04-24T23:03:01.279314+0000","last_became_active":"2024-04-24T23:03:00.268244+0000","last_became_peered":"2024-04-24T23:03:00.268244+0000","last_unstale":"2024-04-24T23:03:01.279314+0000","last_undegraded":"2024-04-24T23:03:01.279314+0000","last_fullsized":"2024-04-24T23:03:01.279314+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:31.425497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.297412+0000","last_change":"2024-04-24T23:03:01.297290+0000","last_active":"2024-04-24T23:03:01.297412+0000","last_peered":"2024-04-24T23:03:01.297412+0000","last_clean":"2024-04-24T23:03:01.297412+0000","last_became_active":"2024-04-24T23:03:01.297044+0000","last_became_peered":"2024-04-24T23:03:01.297044+0000","last_unstale":"2024-04-24T23:03:01.297412+0000","last_undegraded":"2024-04-24T23:03:01.297412+0000","last_fullsized":"2024-04-24T23:03:01.297412+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:37:10.506350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.281961+0000","last_change":"2024-04-24T23:03:00.272578+0000","last_active":"2024-04-24T23:03:01.281961+0000","last_peered":"2024-04-24T23:03:01.281961+0000","last_clean":"2024-04-24T23:03:01.281961+0000","last_became_active":"2024-04-24T23:03:00.271566+0000","last_became_peered":"2024-04-24T23:03:00.271566+0000","last_unstale":"2024-04-24T23:03:01.281961+0000","last_undegraded":"2024-04-24T23:03:01.281961+0000","last_fullsized":"2024-04-24T23:03:01.281961+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:26:29.926479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.295652+0000","last_change":"2024-04-24T23:03:01.295111+0000","last_active":"2024-04-24T23:03:01.295652+0000","last_peered":"2024-04-24T23:03:01.295652+0000","last_clean":"2024-04-24T23:03:01.295652+0000","last_became_active":"2024-04-24T23:03:01.294851+0000","last_became_peered":"2024-04-24T23:03:01.294851+0000","last_unstale":"2024-04-24T23:03:01.295652+0000","last_undegraded":"2024-04-24T23:03:01.295652+0000","last_fullsized":"2024-04-24T23:03:01.295652+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:05:17.948073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"86'3","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.306995+0000","last_change":"2024-04-24T23:03:00.272302+0000","last_active":"2024-04-24T23:03:01.306995+0000","last_peered":"2024-04-24T23:03:01.306995+0000","last_clean":"2024-04-24T23:03:01.306995+0000","last_became_active":"2024-04-24T23:03:00.272059+0000","last_became_peered":"2024-04-24T23:03:00.272059+0000","last_unstale":"2024-04-24T23:03:01.306995+0000","last_undegraded":"2024-04-24T23:03:01.306995+0000","last_fullsized":"2024-04-24T23:03:01.306995+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:04:20.663069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.291002+0000","last_change":"2024-04-24T23:03:01.290878+0000","last_active":"2024-04-24T23:03:01.291002+0000","last_peered":"2024-04-24T23:03:01.291002+0000","last_clean":"2024-04-24T23:03:01.291002+0000","last_became_active":"2024-04-24T23:03:01.290045+0000","last_became_peered":"2024-04-24T23:03:01.290045+0000","last_unstale":"2024-04-24T23:03:01.291002+0000","last_undegraded":"2024-04-24T23:03:01.291002+0000","last_fullsized":"2024-04-24T23:03:01.291002+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:47:39.488434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"86'6","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.307727+0000","last_change":"2024-04-24T23:03:00.269471+0000","last_active":"2024-04-24T23:03:01.307727+0000","last_peered":"2024-04-24T23:03:01.307727+0000","last_clean":"2024-04-24T23:03:01.307727+0000","last_became_active":"2024-04-24T23:03:00.269198+0000","last_became_peered":"2024-04-24T23:03:00.269198+0000","last_unstale":"2024-04-24T23:03:01.307727+0000","last_undegraded":"2024-04-24T23:03:01.307727+0000","last_fullsized":"2024-04-24T23:03:01.307727+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:40:42.391518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.284085+0000","last_change":"2024-04-24T23:03:01.283963+0000","last_active":"2024-04-24T23:03:01.284085+0000","last_peered":"2024-04-24T23:03:01.284085+0000","last_clean":"2024-04-24T23:03:01.284085+0000","last_became_active":"2024-04-24T23:03:01.283644+0000","last_became_peered":"2024-04-24T23:03:01.283644+0000","last_unstale":"2024-04-24T23:03:01.284085+0000","last_undegraded":"2024-04-24T23:03:01.284085+0000","last_fullsized":"2024-04-24T23:03:01.284085+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:11.646502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.306852+0000","last_change":"2024-04-24T23:03:00.271649+0000","last_active":"2024-04-24T23:03:01.306852+0000","last_peered":"2024-04-24T23:03:01.306852+0000","last_clean":"2024-04-24T23:03:01.306852+0000","last_became_active":"2024-04-24T23:03:00.271387+0000","last_became_peered":"2024-04-24T23:03:00.271387+0000","last_unstale":"2024-04-24T23:03:01.306852+0000","last_undegraded":"2024-04-24T23:03:01.306852+0000","last_fullsized":"2024-04-24T23:03:01.306852+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:11.107776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.295048+0000","last_change":"2024-04-24T23:03:01.294910+0000","last_active":"2024-04-24T23:03:01.295048+0000","last_peered":"2024-04-24T23:03:01.295048+0000","last_clean":"2024-04-24T23:03:01.295048+0000","last_became_active":"2024-04-24T23:03:01.293260+0000","last_became_peered":"2024-04-24T23:03:01.293260+0000","last_unstale":"2024-04-24T23:03:01.295048+0000","last_undegraded":"2024-04-24T23:03:01.295048+0000","last_fullsized":"2024-04-24T23:03:01.295048+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:42:30.609881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.287451+0000","last_change":"2024-04-24T23:03:00.269136+0000","last_active":"2024-04-24T23:03:01.287451+0000","last_peered":"2024-04-24T23:03:01.287451+0000","last_clean":"2024-04-24T23:03:01.287451+0000","last_became_active":"2024-04-24T23:03:00.268885+0000","last_became_peered":"2024-04-24T23:03:00.268885+0000","last_unstale":"2024-04-24T23:03:01.287451+0000","last_undegraded":"2024-04-24T23:03:01.287451+0000","last_fullsized":"2024-04-24T23:03:01.287451+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:36.232639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.292047+0000","last_change":"2024-04-24T23:03:01.291934+0000","last_active":"2024-04-24T23:03:01.292047+0000","last_peered":"2024-04-24T23:03:01.292047+0000","last_clean":"2024-04-24T23:03:01.292047+0000","last_became_active":"2024-04-24T23:03:01.291714+0000","last_became_peered":"2024-04-24T23:03:01.291714+0000","last_unstale":"2024-04-24T23:03:01.292047+0000","last_undegraded":"2024-04-24T23:03:01.292047+0000","last_fullsized":"2024-04-24T23:03:01.292047+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:04:41.265768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.282958+0000","last_change":"2024-04-24T23:03:00.270636+0000","last_active":"2024-04-24T23:03:01.282958+0000","last_peered":"2024-04-24T23:03:01.282958+0000","last_clean":"2024-04-24T23:03:01.282958+0000","last_became_active":"2024-04-24T23:03:00.270384+0000","last_became_peered":"2024-04-24T23:03:00.270384+0000","last_unstale":"2024-04-24T23:03:01.282958+0000","last_undegraded":"2024-04-24T23:03:01.282958+0000","last_fullsized":"2024-04-24T23:03:01.282958+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:55:17.047770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.290462+0000","last_change":"2024-04-24T23:03:01.290170+0000","last_active":"2024-04-24T23:03:01.290462+0000","last_peered":"2024-04-24T23:03:01.290462+0000","last_clean":"2024-04-24T23:03:01.290462+0000","last_became_active":"2024-04-24T23:03:01.289928+0000","last_became_peered":"2024-04-24T23:03:01.289928+0000","last_unstale":"2024-04-24T23:03:01.290462+0000","last_undegraded":"2024-04-24T23:03:01.290462+0000","last_fullsized":"2024-04-24T23:03:01.290462+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:11:41.609894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"86'8","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.306765+0000","last_change":"2024-04-24T23:03:00.269110+0000","last_active":"2024-04-24T23:03:01.306765+0000","last_peered":"2024-04-24T23:03:01.306765+0000","last_clean":"2024-04-24T23:03:01.306765+0000","last_became_active":"2024-04-24T23:03:00.268085+0000","last_became_peered":"2024-04-24T23:03:00.268085+0000","last_unstale":"2024-04-24T23:03:01.306765+0000","last_undegraded":"2024-04-24T23:03:01.306765+0000","last_fullsized":"2024-04-24T23:03:01.306765+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:34:47.213313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.289243+0000","last_change":"2024-04-24T23:03:01.289081+0000","last_active":"2024-04-24T23:03:01.289243+0000","last_peered":"2024-04-24T23:03:01.289243+0000","last_clean":"2024-04-24T23:03:01.289243+0000","last_became_active":"2024-04-24T23:03:01.288241+0000","last_became_peered":"2024-04-24T23:03:01.288241+0000","last_unstale":"2024-04-24T23:03:01.289243+0000","last_undegraded":"2024-04-24T23:03:01.289243+0000","last_fullsized":"2024-04-24T23:03:01.289243+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:05:33.715670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.301371+0000","last_change":"2024-04-24T23:03:00.266103+0000","last_active":"2024-04-24T23:03:01.301371+0000","last_peered":"2024-04-24T23:03:01.301371+0000","last_clean":"2024-04-24T23:03:01.301371+0000","last_became_active":"2024-04-24T23:03:00.265836+0000","last_became_peered":"2024-04-24T23:03:00.265836+0000","last_unstale":"2024-04-24T23:03:01.301371+0000","last_undegraded":"2024-04-24T23:03:01.301371+0000","last_fullsized":"2024-04-24T23:03:01.301371+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:18:06.416870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.288252+0000","last_change":"2024-04-24T23:03:01.288117+0000","last_active":"2024-04-24T23:03:01.288252+0000","last_peered":"2024-04-24T23:03:01.288252+0000","last_clean":"2024-04-24T23:03:01.288252+0000","last_became_active":"2024-04-24T23:03:01.287794+0000","last_became_peered":"2024-04-24T23:03:01.287794+0000","last_unstale":"2024-04-24T23:03:01.288252+0000","last_undegraded":"2024-04-24T23:03:01.288252+0000","last_fullsized":"2024-04-24T23:03:01.288252+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:59:38.865133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.306224+0000","last_change":"2024-04-24T23:03:00.271671+0000","last_active":"2024-04-24T23:03:01.306224+0000","last_peered":"2024-04-24T23:03:01.306224+0000","last_clean":"2024-04-24T23:03:01.306224+0000","last_became_active":"2024-04-24T23:03:00.270672+0000","last_became_peered":"2024-04-24T23:03:00.270672+0000","last_unstale":"2024-04-24T23:03:01.306224+0000","last_undegraded":"2024-04-24T23:03:01.306224+0000","last_fullsized":"2024-04-24T23:03:01.306224+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:12:25.077100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.288389+0000","last_change":"2024-04-24T23:03:01.288264+0000","last_active":"2024-04-24T23:03:01.288389+0000","last_peered":"2024-04-24T23:03:01.288389+0000","last_clean":"2024-04-24T23:03:01.288389+0000","last_became_active":"2024-04-24T23:03:01.287905+0000","last_became_peered":"2024-04-24T23:03:01.287905+0000","last_unstale":"2024-04-24T23:03:01.288389+0000","last_undegraded":"2024-04-24T23:03:01.288389+0000","last_fullsized":"2024-04-24T23:03:01.288389+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:46:15.873614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.306773+0000","last_change":"2024-04-24T23:03:00.275516+0000","last_active":"2024-04-24T23:03:01.306773+0000","last_peered":"2024-04-24T23:03:01.306773+0000","last_clean":"2024-04-24T23:03:01.306773+0000","last_became_active":"2024-04-24T23:03:00.275063+0000","last_became_peered":"2024-04-24T23:03:00.275063+0000","last_unstale":"2024-04-24T23:03:01.306773+0000","last_undegraded":"2024-04-24T23:03:01.306773+0000","last_fullsized":"2024-04-24T23:03:01.306773+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:00.798452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.289551+0000","last_change":"2024-04-24T23:03:01.289155+0000","last_active":"2024-04-24T23:03:01.289551+0000","last_peered":"2024-04-24T23:03:01.289551+0000","last_clean":"2024-04-24T23:03:01.289551+0000","last_became_active":"2024-04-24T23:03:01.288908+0000","last_became_peered":"2024-04-24T23:03:01.288908+0000","last_unstale":"2024-04-24T23:03:01.289551+0000","last_undegraded":"2024-04-24T23:03:01.289551+0000","last_fullsized":"2024-04-24T23:03:01.289551+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:47.863641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.283271+0000","last_change":"2024-04-24T23:03:00.271647+0000","last_active":"2024-04-24T23:03:01.283271+0000","last_peered":"2024-04-24T23:03:01.283271+0000","last_clean":"2024-04-24T23:03:01.283271+0000","last_became_active":"2024-04-24T23:03:00.271397+0000","last_became_peered":"2024-04-24T23:03:00.271397+0000","last_unstale":"2024-04-24T23:03:01.283271+0000","last_undegraded":"2024-04-24T23:03:01.283271+0000","last_fullsized":"2024-04-24T23:03:01.283271+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:22.043771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.286783+0000","last_change":"2024-04-24T23:03:01.285830+0000","last_active":"2024-04-24T23:03:01.286783+0000","last_peered":"2024-04-24T23:03:01.286783+0000","last_clean":"2024-04-24T23:03:01.286783+0000","last_became_active":"2024-04-24T23:03:01.285504+0000","last_became_peered":"2024-04-24T23:03:01.285504+0000","last_unstale":"2024-04-24T23:03:01.286783+0000","last_undegraded":"2024-04-24T23:03:01.286783+0000","last_fullsized":"2024-04-24T23:03:01.286783+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:44:53.420886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.284061+0000","last_change":"2024-04-24T23:03:00.273580+0000","last_active":"2024-04-24T23:03:01.284061+0000","last_peered":"2024-04-24T23:03:01.284061+0000","last_clean":"2024-04-24T23:03:01.284061+0000","last_became_active":"2024-04-24T23:03:00.273018+0000","last_became_peered":"2024-04-24T23:03:00.273018+0000","last_unstale":"2024-04-24T23:03:01.284061+0000","last_undegraded":"2024-04-24T23:03:01.284061+0000","last_fullsized":"2024-04-24T23:03:01.284061+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:02:51.326512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.286399+0000","last_change":"2024-04-24T23:03:01.286132+0000","last_active":"2024-04-24T23:03:01.286399+0000","last_peered":"2024-04-24T23:03:01.286399+0000","last_clean":"2024-04-24T23:03:01.286399+0000","last_became_active":"2024-04-24T23:03:01.285659+0000","last_became_peered":"2024-04-24T23:03:01.285659+0000","last_unstale":"2024-04-24T23:03:01.286399+0000","last_undegraded":"2024-04-24T23:03:01.286399+0000","last_fullsized":"2024-04-24T23:03:01.286399+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:24:17.354917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.282158+0000","last_change":"2024-04-24T23:03:00.274527+0000","last_active":"2024-04-24T23:03:01.282158+0000","last_peered":"2024-04-24T23:03:01.282158+0000","last_clean":"2024-04-24T23:03:01.282158+0000","last_became_active":"2024-04-24T23:03:00.274285+0000","last_became_peered":"2024-04-24T23:03:00.274285+0000","last_unstale":"2024-04-24T23:03:01.282158+0000","last_undegraded":"2024-04-24T23:03:01.282158+0000","last_fullsized":"2024-04-24T23:03:01.282158+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:35.739293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.286262+0000","last_change":"2024-04-24T23:03:01.286112+0000","last_active":"2024-04-24T23:03:01.286262+0000","last_peered":"2024-04-24T23:03:01.286262+0000","last_clean":"2024-04-24T23:03:01.286262+0000","last_became_active":"2024-04-24T23:03:01.285864+0000","last_became_peered":"2024-04-24T23:03:01.285864+0000","last_unstale":"2024-04-24T23:03:01.286262+0000","last_undegraded":"2024-04-24T23:03:01.286262+0000","last_fullsized":"2024-04-24T23:03:01.286262+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:04:26.827759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.281237+0000","last_change":"2024-04-24T23:03:00.268748+0000","last_active":"2024-04-24T23:03:01.281237+0000","last_peered":"2024-04-24T23:03:01.281237+0000","last_clean":"2024-04-24T23:03:01.281237+0000","last_became_active":"2024-04-24T23:03:00.268454+0000","last_became_peered":"2024-04-24T23:03:00.268454+0000","last_unstale":"2024-04-24T23:03:01.281237+0000","last_undegraded":"2024-04-24T23:03:01.281237+0000","last_fullsized":"2024-04-24T23:03:01.281237+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:49.701035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.291258+0000","last_change":"2024-04-24T23:03:01.291116+0000","last_active":"2024-04-24T23:03:01.291258+0000","last_peered":"2024-04-24T23:03:01.291258+0000","last_clean":"2024-04-24T23:03:01.291258+0000","last_became_active":"2024-04-24T23:03:01.290614+0000","last_became_peered":"2024-04-24T23:03:01.290614+0000","last_unstale":"2024-04-24T23:03:01.291258+0000","last_undegraded":"2024-04-24T23:03:01.291258+0000","last_fullsized":"2024-04-24T23:03:01.291258+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:58:47.826986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.304325+0000","last_change":"2024-04-24T23:03:00.269801+0000","last_active":"2024-04-24T23:03:01.304325+0000","last_peered":"2024-04-24T23:03:01.304325+0000","last_clean":"2024-04-24T23:03:01.304325+0000","last_became_active":"2024-04-24T23:03:00.269553+0000","last_became_peered":"2024-04-24T23:03:00.269553+0000","last_unstale":"2024-04-24T23:03:01.304325+0000","last_undegraded":"2024-04-24T23:03:01.304325+0000","last_fullsized":"2024-04-24T23:03:01.304325+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:47:15.087313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.291727+0000","last_change":"2024-04-24T23:03:01.289252+0000","last_active":"2024-04-24T23:03:01.291727+0000","last_peered":"2024-04-24T23:03:01.291727+0000","last_clean":"2024-04-24T23:03:01.291727+0000","last_became_active":"2024-04-24T23:03:01.289013+0000","last_became_peered":"2024-04-24T23:03:01.289013+0000","last_unstale":"2024-04-24T23:03:01.291727+0000","last_undegraded":"2024-04-24T23:03:01.291727+0000","last_fullsized":"2024-04-24T23:03:01.291727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:36:24.322251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.278208+0000","last_change":"2024-04-24T23:03:00.268185+0000","last_active":"2024-04-24T23:03:01.278208+0000","last_peered":"2024-04-24T23:03:01.278208+0000","last_clean":"2024-04-24T23:03:01.278208+0000","last_became_active":"2024-04-24T23:03:00.267255+0000","last_became_peered":"2024-04-24T23:03:00.267255+0000","last_unstale":"2024-04-24T23:03:01.278208+0000","last_undegraded":"2024-04-24T23:03:01.278208+0000","last_fullsized":"2024-04-24T23:03:01.278208+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:58.240612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.289465+0000","last_change":"2024-04-24T23:03:01.289331+0000","last_active":"2024-04-24T23:03:01.289465+0000","last_peered":"2024-04-24T23:03:01.289465+0000","last_clean":"2024-04-24T23:03:01.289465+0000","last_became_active":"2024-04-24T23:03:01.288677+0000","last_became_peered":"2024-04-24T23:03:01.288677+0000","last_unstale":"2024-04-24T23:03:01.289465+0000","last_undegraded":"2024-04-24T23:03:01.289465+0000","last_fullsized":"2024-04-24T23:03:01.289465+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:07.386974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.280438+0000","last_change":"2024-04-24T23:03:00.272666+0000","last_active":"2024-04-24T23:03:01.280438+0000","last_peered":"2024-04-24T23:03:01.280438+0000","last_clean":"2024-04-24T23:03:01.280438+0000","last_became_active":"2024-04-24T23:03:00.271406+0000","last_became_peered":"2024-04-24T23:03:00.271406+0000","last_unstale":"2024-04-24T23:03:01.280438+0000","last_undegraded":"2024-04-24T23:03:01.280438+0000","last_fullsized":"2024-04-24T23:03:01.280438+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:45:59.370519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.294747+0000","last_change":"2024-04-24T23:03:01.294628+0000","last_active":"2024-04-24T23:03:01.294747+0000","last_peered":"2024-04-24T23:03:01.294747+0000","last_clean":"2024-04-24T23:03:01.294747+0000","last_became_active":"2024-04-24T23:03:01.294366+0000","last_became_peered":"2024-04-24T23:03:01.294366+0000","last_unstale":"2024-04-24T23:03:01.294747+0000","last_undegraded":"2024-04-24T23:03:01.294747+0000","last_fullsized":"2024-04-24T23:03:01.294747+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:21.992711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.279148+0000","last_change":"2024-04-24T23:03:00.268434+0000","last_active":"2024-04-24T23:03:01.279148+0000","last_peered":"2024-04-24T23:03:01.279148+0000","last_clean":"2024-04-24T23:03:01.279148+0000","last_became_active":"2024-04-24T23:03:00.268134+0000","last_became_peered":"2024-04-24T23:03:00.268134+0000","last_unstale":"2024-04-24T23:03:01.279148+0000","last_undegraded":"2024-04-24T23:03:01.279148+0000","last_fullsized":"2024-04-24T23:03:01.279148+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:42:29.503384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.286101+0000","last_change":"2024-04-24T23:03:01.285990+0000","last_active":"2024-04-24T23:03:01.286101+0000","last_peered":"2024-04-24T23:03:01.286101+0000","last_clean":"2024-04-24T23:03:01.286101+0000","last_became_active":"2024-04-24T23:03:01.285571+0000","last_became_peered":"2024-04-24T23:03:01.285571+0000","last_unstale":"2024-04-24T23:03:01.286101+0000","last_undegraded":"2024-04-24T23:03:01.286101+0000","last_fullsized":"2024-04-24T23:03:01.286101+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:59:34.804951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.278483+0000","last_change":"2024-04-24T23:03:00.269313+0000","last_active":"2024-04-24T23:03:01.278483+0000","last_peered":"2024-04-24T23:03:01.278483+0000","last_clean":"2024-04-24T23:03:01.278483+0000","last_became_active":"2024-04-24T23:03:00.269076+0000","last_became_peered":"2024-04-24T23:03:00.269076+0000","last_unstale":"2024-04-24T23:03:01.278483+0000","last_undegraded":"2024-04-24T23:03:01.278483+0000","last_fullsized":"2024-04-24T23:03:01.278483+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:08.306657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.291830+0000","last_change":"2024-04-24T23:03:01.291334+0000","last_active":"2024-04-24T23:03:01.291830+0000","last_peered":"2024-04-24T23:03:01.291830+0000","last_clean":"2024-04-24T23:03:01.291830+0000","last_became_active":"2024-04-24T23:03:01.289947+0000","last_became_peered":"2024-04-24T23:03:01.289947+0000","last_unstale":"2024-04-24T23:03:01.291830+0000","last_undegraded":"2024-04-24T23:03:01.291830+0000","last_fullsized":"2024-04-24T23:03:01.291830+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:49:21.158646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.281336+0000","last_change":"2024-04-24T23:03:00.267882+0000","last_active":"2024-04-24T23:03:01.281336+0000","last_peered":"2024-04-24T23:03:01.281336+0000","last_clean":"2024-04-24T23:03:01.281336+0000","last_became_active":"2024-04-24T23:03:00.267600+0000","last_became_peered":"2024-04-24T23:03:00.267600+0000","last_unstale":"2024-04-24T23:03:01.281336+0000","last_undegraded":"2024-04-24T23:03:01.281336+0000","last_fullsized":"2024-04-24T23:03:01.281336+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:58.367970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.290711+0000","last_change":"2024-04-24T23:03:01.290474+0000","last_active":"2024-04-24T23:03:01.290711+0000","last_peered":"2024-04-24T23:03:01.290711+0000","last_clean":"2024-04-24T23:03:01.290711+0000","last_became_active":"2024-04-24T23:03:01.290221+0000","last_became_peered":"2024-04-24T23:03:01.290221+0000","last_unstale":"2024-04-24T23:03:01.290711+0000","last_undegraded":"2024-04-24T23:03:01.290711+0000","last_fullsized":"2024-04-24T23:03:01.290711+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:31:10.422483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.282157+0000","last_change":"2024-04-24T23:03:00.272260+0000","last_active":"2024-04-24T23:03:01.282157+0000","last_peered":"2024-04-24T23:03:01.282157+0000","last_clean":"2024-04-24T23:03:01.282157+0000","last_became_active":"2024-04-24T23:03:00.271163+0000","last_became_peered":"2024-04-24T23:03:00.271163+0000","last_unstale":"2024-04-24T23:03:01.282157+0000","last_undegraded":"2024-04-24T23:03:01.282157+0000","last_fullsized":"2024-04-24T23:03:01.282157+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:01:46.026496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.289402+0000","last_change":"2024-04-24T23:03:01.289277+0000","last_active":"2024-04-24T23:03:01.289402+0000","last_peered":"2024-04-24T23:03:01.289402+0000","last_clean":"2024-04-24T23:03:01.289402+0000","last_became_active":"2024-04-24T23:03:01.289021+0000","last_became_peered":"2024-04-24T23:03:01.289021+0000","last_unstale":"2024-04-24T23:03:01.289402+0000","last_undegraded":"2024-04-24T23:03:01.289402+0000","last_fullsized":"2024-04-24T23:03:01.289402+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:24.651133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.284258+0000","last_change":"2024-04-24T23:03:00.270686+0000","last_active":"2024-04-24T23:03:01.284258+0000","last_peered":"2024-04-24T23:03:01.284258+0000","last_clean":"2024-04-24T23:03:01.284258+0000","last_became_active":"2024-04-24T23:03:00.270334+0000","last_became_peered":"2024-04-24T23:03:00.270334+0000","last_unstale":"2024-04-24T23:03:01.284258+0000","last_undegraded":"2024-04-24T23:03:01.284258+0000","last_fullsized":"2024-04-24T23:03:01.284258+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:12:17.932580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.285484+0000","last_change":"2024-04-24T23:03:01.285374+0000","last_active":"2024-04-24T23:03:01.285484+0000","last_peered":"2024-04-24T23:03:01.285484+0000","last_clean":"2024-04-24T23:03:01.285484+0000","last_became_active":"2024-04-24T23:03:01.285116+0000","last_became_peered":"2024-04-24T23:03:01.285116+0000","last_unstale":"2024-04-24T23:03:01.285484+0000","last_undegraded":"2024-04-24T23:03:01.285484+0000","last_fullsized":"2024-04-24T23:03:01.285484+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:46:43.074985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.306891+0000","last_change":"2024-04-24T23:03:00.270687+0000","last_active":"2024-04-24T23:03:01.306891+0000","last_peered":"2024-04-24T23:03:01.306891+0000","last_clean":"2024-04-24T23:03:01.306891+0000","last_became_active":"2024-04-24T23:03:00.270437+0000","last_became_peered":"2024-04-24T23:03:00.270437+0000","last_unstale":"2024-04-24T23:03:01.306891+0000","last_undegraded":"2024-04-24T23:03:01.306891+0000","last_fullsized":"2024-04-24T23:03:01.306891+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:35.640764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":297,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.291115+0000","last_change":"2024-04-24T22:58:55.138810+0000","last_active":"2024-04-24T23:03:01.291115+0000","last_peered":"2024-04-24T23:03:01.291115+0000","last_clean":"2024-04-24T23:03:01.291115+0000","last_became_active":"2024-04-24T22:58:55.138627+0000","last_became_peered":"2024-04-24T22:58:55.138627+0000","last_unstale":"2024-04-24T23:03:01.291115+0000","last_undegraded":"2024-04-24T23:03:01.291115+0000","last_fullsized":"2024-04-24T23:03:01.291115+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_clean_scrub_stamp":"2024-04-24T22:58:54.127235+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:32:00.228724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574900,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":449,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194863,"statfs":{"total":200000000,"available":199539957,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574900,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":449,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194863,"statfs":{"total":200000000,"available":199539923,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574900,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1024,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194288,"statfs":{"total":200000000,"available":198951111,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574900,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":441,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194871,"statfs":{"total":200000000,"available":199548344,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574900,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":447,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194864,"statfs":{"total":200000000,"available":199541443,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574900,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":448,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194864,"statfs":{"total":200000000,"available":199540987,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574900,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1024,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194288,"statfs":{"total":200000000,"available":198951201,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574900,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":447,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194864,"statfs":{"total":200000000,"available":199541499,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-24T23:03:05.294 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:03:05.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.430+0000 7f45d8734640 1 Processor -- start 2024-04-24T23:03:05.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.430+0000 7f45d8734640 1 -- start start 2024-04-24T23:03:05.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.431+0000 7f45d8734640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f45d015b480 0x7f45d00b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:05.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.431+0000 7f45d8734640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f45d015fd60 0x7f45d006c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:05.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.431+0000 7f45d8734640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f45d006cb30 0x7f45d00ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:05.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.431+0000 7f45d8734640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f45d00b4ff0 con 0x7f45d015b480 2024-04-24T23:03:05.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.431+0000 7f45d8734640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f45d00b46c0 con 0x7f45d015fd60 2024-04-24T23:03:05.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.431+0000 7f45d8734640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f45d00b49a0 con 0x7f45d006cb30 2024-04-24T23:03:05.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.431+0000 7f45d64d0640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f45d015b480 0x7f45d00b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:05.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.431+0000 7f45d64d0640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f45d015b480 0x7f45d00b5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38462/0 (socket says 172.21.15.32:38462) 2024-04-24T23:03:05.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.431+0000 7f45d64d0640 1 -- 172.21.15.32:0/915307889 learned_addr learned my addr 172.21.15.32:0/915307889 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:05.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.431+0000 7f45d5ccf640 1 --2- 172.21.15.32:0/915307889 >> v2:172.21.15.121:3300/0 conn(0x7f45d015fd60 0x7f45d006c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:05.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.431+0000 7f45d6cd1640 1 --2- 172.21.15.32:0/915307889 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f45d006cb30 0x7f45d00ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:05.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.431+0000 7f45d64d0640 1 -- 172.21.15.32:0/915307889 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f45d006cb30 msgr2=0x7f45d00ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:05.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.431+0000 7f45d64d0640 1 --2- 172.21.15.32:0/915307889 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f45d006cb30 0x7f45d00ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:05.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.431+0000 7f45d64d0640 1 -- 172.21.15.32:0/915307889 >> v2:172.21.15.121:3300/0 conn(0x7f45d015fd60 msgr2=0x7f45d006c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:05.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.431+0000 7f45d64d0640 1 --2- 172.21.15.32:0/915307889 >> v2:172.21.15.121:3300/0 conn(0x7f45d015fd60 0x7f45d006c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:05.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.431+0000 7f45d64d0640 1 -- 172.21.15.32:0/915307889 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45d00b4c80 con 0x7f45d015b480 2024-04-24T23:03:05.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.431+0000 7f45d5ccf640 1 --2- 172.21.15.32:0/915307889 >> v2:172.21.15.121:3300/0 conn(0x7f45d015fd60 0x7f45d006c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:03:05.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.432+0000 7f45d64d0640 1 --2- 172.21.15.32:0/915307889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45d015b480 0x7f45d00b5c60 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f45d006ee10 tx=0x7f45cc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7ae39355fa605a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:05.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.432+0000 7f45d54ce640 1 -- 172.21.15.32:0/915307889 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45cc037070 con 0x7f45d015b480 2024-04-24T23:03:05.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.432+0000 7f45d54ce640 1 -- 172.21.15.32:0/915307889 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45cc032440 con 0x7f45d015b480 2024-04-24T23:03:05.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.432+0000 7f45d54ce640 1 -- 172.21.15.32:0/915307889 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45cc032810 con 0x7f45d015b480 2024-04-24T23:03:05.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.432+0000 7f45d8734640 1 -- 172.21.15.32:0/915307889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45d015b480 msgr2=0x7f45d00b5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:05.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.432+0000 7f45d8734640 1 --2- 172.21.15.32:0/915307889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45d015b480 0x7f45d00b5c60 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f45d006ee10 tx=0x7f45cc02a1e0 comp rx=0 tx=0).stop 2024-04-24T23:03:05.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.433+0000 7f45d8734640 1 -- 172.21.15.32:0/915307889 shutdown_connections 2024-04-24T23:03:05.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.433+0000 7f45d8734640 1 --2- 172.21.15.32:0/915307889 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f45d006cb30 0x7f45d00ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:05.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.433+0000 7f45d8734640 1 --2- 172.21.15.32:0/915307889 >> v2:172.21.15.121:3300/0 conn(0x7f45d015fd60 0x7f45d006c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:05.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.433+0000 7f45d8734640 1 --2- 172.21.15.32:0/915307889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45d015b480 0x7f45d00b5c60 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:05.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.433+0000 7f45d8734640 1 -- 172.21.15.32:0/915307889 >> 172.21.15.32:0/915307889 conn(0x7f45d006b110 msgr2=0x7f45d006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:05.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.433+0000 7f45d8734640 1 -- 172.21.15.32:0/915307889 shutdown_connections 2024-04-24T23:03:05.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.433+0000 7f45d8734640 1 -- 172.21.15.32:0/915307889 wait complete. 2024-04-24T23:03:05.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.434+0000 7f45d8734640 1 Processor -- start 2024-04-24T23:03:05.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.434+0000 7f45d8734640 1 -- start start 2024-04-24T23:03:05.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.435+0000 7f45d8734640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45d006cb30 0x7f45d014d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:05.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.435+0000 7f45d8734640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45d015b480 0x7f45d014f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:05.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.435+0000 7f45d8734640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f45d015fd60 0x7f45d015d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:05.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.435+0000 7f45d8734640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f45d015b860 con 0x7f45d015b480 2024-04-24T23:03:05.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.435+0000 7f45d8734640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f45d00b5480 con 0x7f45d006cb30 2024-04-24T23:03:05.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.435+0000 7f45d8734640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f45d0150f10 con 0x7f45d015fd60 2024-04-24T23:03:05.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.435+0000 7f45d5ccf640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45d015b480 0x7f45d014f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:05.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.435+0000 7f45d5ccf640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45d015b480 0x7f45d014f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38468/0 (socket says 172.21.15.32:38468) 2024-04-24T23:03:05.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.435+0000 7f45d5ccf640 1 -- 172.21.15.32:0/1856048783 learned_addr learned my addr 172.21.15.32:0/1856048783 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:05.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.435+0000 7f45d64d0640 1 --2- 172.21.15.32:0/1856048783 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45d006cb30 0x7f45d014d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:05.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.435+0000 7f45d6cd1640 1 --2- 172.21.15.32:0/1856048783 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f45d015fd60 0x7f45d015d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:05.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.435+0000 7f45d5ccf640 1 -- 172.21.15.32:0/1856048783 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f45d015fd60 msgr2=0x7f45d015d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:05.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.435+0000 7f45d5ccf640 1 --2- 172.21.15.32:0/1856048783 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f45d015fd60 0x7f45d015d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:05.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.435+0000 7f45d5ccf640 1 -- 172.21.15.32:0/1856048783 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45d006cb30 msgr2=0x7f45d014d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:05.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.435+0000 7f45d5ccf640 1 --2- 172.21.15.32:0/1856048783 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45d006cb30 0x7f45d014d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:05.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.435+0000 7f45d5ccf640 1 -- 172.21.15.32:0/1856048783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45d0150300 con 0x7f45d015b480 2024-04-24T23:03:05.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.435+0000 7f45d5ccf640 1 --2- 172.21.15.32:0/1856048783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45d015b480 0x7f45d014f910 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f45c000d800 tx=0x7f45c000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:05.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.435+0000 7f45c6ffd640 1 -- 172.21.15.32:0/1856048783 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c0004030 con 0x7f45d015b480 2024-04-24T23:03:05.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.435+0000 7f45c6ffd640 1 -- 172.21.15.32:0/1856048783 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45c00041d0 con 0x7f45d015b480 2024-04-24T23:03:05.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.435+0000 7f45d8734640 1 -- 172.21.15.32:0/1856048783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45d01505e0 con 0x7f45d015b480 2024-04-24T23:03:05.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.435+0000 7f45c6ffd640 1 -- 172.21.15.32:0/1856048783 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c00096a0 con 0x7f45d015b480 2024-04-24T23:03:05.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.435+0000 7f45d8734640 1 -- 172.21.15.32:0/1856048783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45d0150ba0 con 0x7f45d015b480 2024-04-24T23:03:05.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.436+0000 7f45d8734640 1 -- 172.21.15.32:0/1856048783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45d0068430 con 0x7f45d015b480 2024-04-24T23:03:05.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.437+0000 7f45c6ffd640 1 -- 172.21.15.32:0/1856048783 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f45c0009840 con 0x7f45d015b480 2024-04-24T23:03:05.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.437+0000 7f45c6ffd640 1 --2- 172.21.15.32:0/1856048783 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f45b406be30 0x7f45b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:05.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.437+0000 7f45c6ffd640 1 -- 172.21.15.32:0/1856048783 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7f45c008afe0 con 0x7f45d015b480 2024-04-24T23:03:05.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.438+0000 7f45d64d0640 1 --2- 172.21.15.32:0/1856048783 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f45b406be30 0x7f45b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:05.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.438+0000 7f45d64d0640 1 --2- 172.21.15.32:0/1856048783 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f45b406be30 0x7f45b406e2f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f45cc034000 tx=0x7f45cc02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:05.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.442+0000 7f45c6ffd640 1 -- 172.21.15.32:0/1856048783 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f45c0059340 con 0x7f45d015b480 2024-04-24T23:03:05.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.622+0000 7f45d8734640 1 -- 172.21.15.32:0/1856048783 --> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f45d006c000 con 0x7f45b406be30 2024-04-24T23:03:05.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.628+0000 7f45c6ffd640 1 -- 172.21.15.32:0/1856048783 <== mgr.4108 v2:172.21.15.121:6832/1848157943 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247524 (secure 0 0 0) 0x7f45d006c000 con 0x7f45b406be30 2024-04-24T23:03:05.631 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:03:05.634 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-24T23:03:05.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.631+0000 7f45d8734640 1 -- 172.21.15.32:0/1856048783 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f45b406be30 msgr2=0x7f45b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:05.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.631+0000 7f45d8734640 1 --2- 172.21.15.32:0/1856048783 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f45b406be30 0x7f45b406e2f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f45cc034000 tx=0x7f45cc02e000 comp rx=0 tx=0).stop 2024-04-24T23:03:05.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.631+0000 7f45d8734640 1 -- 172.21.15.32:0/1856048783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45d015b480 msgr2=0x7f45d014f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:05.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.631+0000 7f45d8734640 1 --2- 172.21.15.32:0/1856048783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45d015b480 0x7f45d014f910 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f45c000d800 tx=0x7f45c000dcc0 comp rx=0 tx=0).stop 2024-04-24T23:03:05.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.631+0000 7f45d8734640 1 -- 172.21.15.32:0/1856048783 shutdown_connections 2024-04-24T23:03:05.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.631+0000 7f45d8734640 1 --2- 172.21.15.32:0/1856048783 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f45b406be30 0x7f45b406e2f0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:05.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.631+0000 7f45d8734640 1 --2- 172.21.15.32:0/1856048783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45d015b480 0x7f45d014f910 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:05.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.631+0000 7f45d8734640 1 --2- 172.21.15.32:0/1856048783 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f45d015fd60 0x7f45d015d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:05.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.631+0000 7f45d8734640 1 --2- 172.21.15.32:0/1856048783 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45d006cb30 0x7f45d014d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:05.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.631+0000 7f45d8734640 1 -- 172.21.15.32:0/1856048783 >> 172.21.15.32:0/1856048783 conn(0x7f45d006b110 msgr2=0x7f45d00b3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:05.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.631+0000 7f45d8734640 1 -- 172.21.15.32:0/1856048783 shutdown_connections 2024-04-24T23:03:05.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.631+0000 7f45d8734640 1 -- 172.21.15.32:0/1856048783 wait complete. 2024-04-24T23:03:05.643 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":205,"stamp":"2024-04-24T23:03:05.182532+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":99,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":4731,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1557768,"statfs":{"total":1600000000,"available":1595154465,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.923742"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.288518+0000","last_change":"2024-04-24T23:03:01.287997+0000","last_active":"2024-04-24T23:03:01.288518+0000","last_peered":"2024-04-24T23:03:01.288518+0000","last_clean":"2024-04-24T23:03:01.288518+0000","last_became_active":"2024-04-24T23:03:01.285345+0000","last_became_peered":"2024-04-24T23:03:01.285345+0000","last_unstale":"2024-04-24T23:03:01.288518+0000","last_undegraded":"2024-04-24T23:03:01.288518+0000","last_fullsized":"2024-04-24T23:03:01.288518+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:53:18.209022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.288041+0000","last_change":"2024-04-24T23:03:01.287833+0000","last_active":"2024-04-24T23:03:01.288041+0000","last_peered":"2024-04-24T23:03:01.288041+0000","last_clean":"2024-04-24T23:03:01.288041+0000","last_became_active":"2024-04-24T23:03:01.287515+0000","last_became_peered":"2024-04-24T23:03:01.287515+0000","last_unstale":"2024-04-24T23:03:01.288041+0000","last_undegraded":"2024-04-24T23:03:01.288041+0000","last_fullsized":"2024-04-24T23:03:01.288041+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:56:52.361384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.294494+0000","last_change":"2024-04-24T23:03:01.294363+0000","last_active":"2024-04-24T23:03:01.294494+0000","last_peered":"2024-04-24T23:03:01.294494+0000","last_clean":"2024-04-24T23:03:01.294494+0000","last_became_active":"2024-04-24T23:03:01.293671+0000","last_became_peered":"2024-04-24T23:03:01.293671+0000","last_unstale":"2024-04-24T23:03:01.294494+0000","last_undegraded":"2024-04-24T23:03:01.294494+0000","last_fullsized":"2024-04-24T23:03:01.294494+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:39:46.041609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.294861+0000","last_change":"2024-04-24T23:03:01.294730+0000","last_active":"2024-04-24T23:03:01.294861+0000","last_peered":"2024-04-24T23:03:01.294861+0000","last_clean":"2024-04-24T23:03:01.294861+0000","last_became_active":"2024-04-24T23:03:01.294460+0000","last_became_peered":"2024-04-24T23:03:01.294460+0000","last_unstale":"2024-04-24T23:03:01.294861+0000","last_undegraded":"2024-04-24T23:03:01.294861+0000","last_fullsized":"2024-04-24T23:03:01.294861+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:58.260140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.289993+0000","last_change":"2024-04-24T23:03:01.289873+0000","last_active":"2024-04-24T23:03:01.289993+0000","last_peered":"2024-04-24T23:03:01.289993+0000","last_clean":"2024-04-24T23:03:01.289993+0000","last_became_active":"2024-04-24T23:03:01.289591+0000","last_became_peered":"2024-04-24T23:03:01.289591+0000","last_unstale":"2024-04-24T23:03:01.289993+0000","last_undegraded":"2024-04-24T23:03:01.289993+0000","last_fullsized":"2024-04-24T23:03:01.289993+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:23:22.139609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.291856+0000","last_change":"2024-04-24T23:03:01.291161+0000","last_active":"2024-04-24T23:03:01.291856+0000","last_peered":"2024-04-24T23:03:01.291856+0000","last_clean":"2024-04-24T23:03:01.291856+0000","last_became_active":"2024-04-24T23:03:01.290732+0000","last_became_peered":"2024-04-24T23:03:01.290732+0000","last_unstale":"2024-04-24T23:03:01.291856+0000","last_undegraded":"2024-04-24T23:03:01.291856+0000","last_fullsized":"2024-04-24T23:03:01.291856+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:07:11.209559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.286069+0000","last_change":"2024-04-24T23:03:01.285942+0000","last_active":"2024-04-24T23:03:01.286069+0000","last_peered":"2024-04-24T23:03:01.286069+0000","last_clean":"2024-04-24T23:03:01.286069+0000","last_became_active":"2024-04-24T23:03:01.285678+0000","last_became_peered":"2024-04-24T23:03:01.285678+0000","last_unstale":"2024-04-24T23:03:01.286069+0000","last_undegraded":"2024-04-24T23:03:01.286069+0000","last_fullsized":"2024-04-24T23:03:01.286069+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:08.412117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.291029+0000","last_change":"2024-04-24T23:03:01.290909+0000","last_active":"2024-04-24T23:03:01.291029+0000","last_peered":"2024-04-24T23:03:01.291029+0000","last_clean":"2024-04-24T23:03:01.291029+0000","last_became_active":"2024-04-24T23:03:01.290455+0000","last_became_peered":"2024-04-24T23:03:01.290455+0000","last_unstale":"2024-04-24T23:03:01.291029+0000","last_undegraded":"2024-04-24T23:03:01.291029+0000","last_fullsized":"2024-04-24T23:03:01.291029+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:56:27.406947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.287693+0000","last_change":"2024-04-24T23:03:01.287556+0000","last_active":"2024-04-24T23:03:01.287693+0000","last_peered":"2024-04-24T23:03:01.287693+0000","last_clean":"2024-04-24T23:03:01.287693+0000","last_became_active":"2024-04-24T23:03:01.287309+0000","last_became_peered":"2024-04-24T23:03:01.287309+0000","last_unstale":"2024-04-24T23:03:01.287693+0000","last_undegraded":"2024-04-24T23:03:01.287693+0000","last_fullsized":"2024-04-24T23:03:01.287693+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:58:02.114473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.294320+0000","last_change":"2024-04-24T23:03:01.294147+0000","last_active":"2024-04-24T23:03:01.294320+0000","last_peered":"2024-04-24T23:03:01.294320+0000","last_clean":"2024-04-24T23:03:01.294320+0000","last_became_active":"2024-04-24T23:03:01.292898+0000","last_became_peered":"2024-04-24T23:03:01.292898+0000","last_unstale":"2024-04-24T23:03:01.294320+0000","last_undegraded":"2024-04-24T23:03:01.294320+0000","last_fullsized":"2024-04-24T23:03:01.294320+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:20:57.212758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.288501+0000","last_change":"2024-04-24T23:03:01.287783+0000","last_active":"2024-04-24T23:03:01.288501+0000","last_peered":"2024-04-24T23:03:01.288501+0000","last_clean":"2024-04-24T23:03:01.288501+0000","last_became_active":"2024-04-24T23:03:01.284879+0000","last_became_peered":"2024-04-24T23:03:01.284879+0000","last_unstale":"2024-04-24T23:03:01.288501+0000","last_undegraded":"2024-04-24T23:03:01.288501+0000","last_fullsized":"2024-04-24T23:03:01.288501+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:47:33.729406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.291839+0000","last_change":"2024-04-24T23:03:01.291585+0000","last_active":"2024-04-24T23:03:01.291839+0000","last_peered":"2024-04-24T23:03:01.291839+0000","last_clean":"2024-04-24T23:03:01.291839+0000","last_became_active":"2024-04-24T23:03:01.291366+0000","last_became_peered":"2024-04-24T23:03:01.291366+0000","last_unstale":"2024-04-24T23:03:01.291839+0000","last_undegraded":"2024-04-24T23:03:01.291839+0000","last_fullsized":"2024-04-24T23:03:01.291839+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:32:52.765176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.294695+0000","last_change":"2024-04-24T23:03:01.294589+0000","last_active":"2024-04-24T23:03:01.294695+0000","last_peered":"2024-04-24T23:03:01.294695+0000","last_clean":"2024-04-24T23:03:01.294695+0000","last_became_active":"2024-04-24T23:03:01.294103+0000","last_became_peered":"2024-04-24T23:03:01.294103+0000","last_unstale":"2024-04-24T23:03:01.294695+0000","last_undegraded":"2024-04-24T23:03:01.294695+0000","last_fullsized":"2024-04-24T23:03:01.294695+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:31.145086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.295410+0000","last_change":"2024-04-24T23:03:01.295302+0000","last_active":"2024-04-24T23:03:01.295410+0000","last_peered":"2024-04-24T23:03:01.295410+0000","last_clean":"2024-04-24T23:03:01.295410+0000","last_became_active":"2024-04-24T23:03:01.294840+0000","last_became_peered":"2024-04-24T23:03:01.294840+0000","last_unstale":"2024-04-24T23:03:01.295410+0000","last_undegraded":"2024-04-24T23:03:01.295410+0000","last_fullsized":"2024-04-24T23:03:01.295410+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:28:36.529735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.284589+0000","last_change":"2024-04-24T23:03:01.284429+0000","last_active":"2024-04-24T23:03:01.284589+0000","last_peered":"2024-04-24T23:03:01.284589+0000","last_clean":"2024-04-24T23:03:01.284589+0000","last_became_active":"2024-04-24T23:03:01.284164+0000","last_became_peered":"2024-04-24T23:03:01.284164+0000","last_unstale":"2024-04-24T23:03:01.284589+0000","last_undegraded":"2024-04-24T23:03:01.284589+0000","last_fullsized":"2024-04-24T23:03:01.284589+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:17:09.127180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.290893+0000","last_change":"2024-04-24T23:03:01.290183+0000","last_active":"2024-04-24T23:03:01.290893+0000","last_peered":"2024-04-24T23:03:01.290893+0000","last_clean":"2024-04-24T23:03:01.290893+0000","last_became_active":"2024-04-24T23:03:01.289155+0000","last_became_peered":"2024-04-24T23:03:01.289155+0000","last_unstale":"2024-04-24T23:03:01.290893+0000","last_undegraded":"2024-04-24T23:03:01.290893+0000","last_fullsized":"2024-04-24T23:03:01.290893+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:42:33.723426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.285906+0000","last_change":"2024-04-24T23:03:01.285797+0000","last_active":"2024-04-24T23:03:01.285906+0000","last_peered":"2024-04-24T23:03:01.285906+0000","last_clean":"2024-04-24T23:03:01.285906+0000","last_became_active":"2024-04-24T23:03:01.285281+0000","last_became_peered":"2024-04-24T23:03:01.285281+0000","last_unstale":"2024-04-24T23:03:01.285906+0000","last_undegraded":"2024-04-24T23:03:01.285906+0000","last_fullsized":"2024-04-24T23:03:01.285906+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:25:37.338406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.290442+0000","last_change":"2024-04-24T23:03:01.290320+0000","last_active":"2024-04-24T23:03:01.290442+0000","last_peered":"2024-04-24T23:03:01.290442+0000","last_clean":"2024-04-24T23:03:01.290442+0000","last_became_active":"2024-04-24T23:03:01.289055+0000","last_became_peered":"2024-04-24T23:03:01.289055+0000","last_unstale":"2024-04-24T23:03:01.290442+0000","last_undegraded":"2024-04-24T23:03:01.290442+0000","last_fullsized":"2024-04-24T23:03:01.290442+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:03:29.101082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.288369+0000","last_change":"2024-04-24T23:03:01.286441+0000","last_active":"2024-04-24T23:03:01.288369+0000","last_peered":"2024-04-24T23:03:01.288369+0000","last_clean":"2024-04-24T23:03:01.288369+0000","last_became_active":"2024-04-24T23:03:01.285596+0000","last_became_peered":"2024-04-24T23:03:01.285596+0000","last_unstale":"2024-04-24T23:03:01.288369+0000","last_undegraded":"2024-04-24T23:03:01.288369+0000","last_fullsized":"2024-04-24T23:03:01.288369+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:05:10.160284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.287301+0000","last_change":"2024-04-24T23:03:01.287161+0000","last_active":"2024-04-24T23:03:01.287301+0000","last_peered":"2024-04-24T23:03:01.287301+0000","last_clean":"2024-04-24T23:03:01.287301+0000","last_became_active":"2024-04-24T23:03:01.286880+0000","last_became_peered":"2024-04-24T23:03:01.286880+0000","last_unstale":"2024-04-24T23:03:01.287301+0000","last_undegraded":"2024-04-24T23:03:01.287301+0000","last_fullsized":"2024-04-24T23:03:01.287301+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:18:18.763038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.288825+0000","last_change":"2024-04-24T23:03:01.288705+0000","last_active":"2024-04-24T23:03:01.288825+0000","last_peered":"2024-04-24T23:03:01.288825+0000","last_clean":"2024-04-24T23:03:01.288825+0000","last_became_active":"2024-04-24T23:03:01.288448+0000","last_became_peered":"2024-04-24T23:03:01.288448+0000","last_unstale":"2024-04-24T23:03:01.288825+0000","last_undegraded":"2024-04-24T23:03:01.288825+0000","last_fullsized":"2024-04-24T23:03:01.288825+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:30:36.497458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.287084+0000","last_change":"2024-04-24T23:03:01.286961+0000","last_active":"2024-04-24T23:03:01.287084+0000","last_peered":"2024-04-24T23:03:01.287084+0000","last_clean":"2024-04-24T23:03:01.287084+0000","last_became_active":"2024-04-24T23:03:01.286716+0000","last_became_peered":"2024-04-24T23:03:01.286716+0000","last_unstale":"2024-04-24T23:03:01.287084+0000","last_undegraded":"2024-04-24T23:03:01.287084+0000","last_fullsized":"2024-04-24T23:03:01.287084+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:32:49.520863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.298171+0000","last_change":"2024-04-24T23:03:01.298000+0000","last_active":"2024-04-24T23:03:01.298171+0000","last_peered":"2024-04-24T23:03:01.298171+0000","last_clean":"2024-04-24T23:03:01.298171+0000","last_became_active":"2024-04-24T23:03:01.294249+0000","last_became_peered":"2024-04-24T23:03:01.294249+0000","last_unstale":"2024-04-24T23:03:01.298171+0000","last_undegraded":"2024-04-24T23:03:01.298171+0000","last_fullsized":"2024-04-24T23:03:01.298171+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:26:35.747390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.288406+0000","last_change":"2024-04-24T23:03:01.286637+0000","last_active":"2024-04-24T23:03:01.288406+0000","last_peered":"2024-04-24T23:03:01.288406+0000","last_clean":"2024-04-24T23:03:01.288406+0000","last_became_active":"2024-04-24T23:03:01.286078+0000","last_became_peered":"2024-04-24T23:03:01.286078+0000","last_unstale":"2024-04-24T23:03:01.288406+0000","last_undegraded":"2024-04-24T23:03:01.288406+0000","last_fullsized":"2024-04-24T23:03:01.288406+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:19:34.997934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.296320+0000","last_change":"2024-04-24T23:03:01.296132+0000","last_active":"2024-04-24T23:03:01.296320+0000","last_peered":"2024-04-24T23:03:01.296320+0000","last_clean":"2024-04-24T23:03:01.296320+0000","last_became_active":"2024-04-24T23:03:01.294583+0000","last_became_peered":"2024-04-24T23:03:01.294583+0000","last_unstale":"2024-04-24T23:03:01.296320+0000","last_undegraded":"2024-04-24T23:03:01.296320+0000","last_fullsized":"2024-04-24T23:03:01.296320+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:58:27.227491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.297365+0000","last_change":"2024-04-24T23:03:01.297093+0000","last_active":"2024-04-24T23:03:01.297365+0000","last_peered":"2024-04-24T23:03:01.297365+0000","last_clean":"2024-04-24T23:03:01.297365+0000","last_became_active":"2024-04-24T23:03:01.296835+0000","last_became_peered":"2024-04-24T23:03:01.296835+0000","last_unstale":"2024-04-24T23:03:01.297365+0000","last_undegraded":"2024-04-24T23:03:01.297365+0000","last_fullsized":"2024-04-24T23:03:01.297365+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:20:50.261924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.289391+0000","last_change":"2024-04-24T23:03:01.289214+0000","last_active":"2024-04-24T23:03:01.289391+0000","last_peered":"2024-04-24T23:03:01.289391+0000","last_clean":"2024-04-24T23:03:01.289391+0000","last_became_active":"2024-04-24T23:03:01.288966+0000","last_became_peered":"2024-04-24T23:03:01.288966+0000","last_unstale":"2024-04-24T23:03:01.289391+0000","last_undegraded":"2024-04-24T23:03:01.289391+0000","last_fullsized":"2024-04-24T23:03:01.289391+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:33.116279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.287854+0000","last_change":"2024-04-24T23:03:01.287682+0000","last_active":"2024-04-24T23:03:01.287854+0000","last_peered":"2024-04-24T23:03:01.287854+0000","last_clean":"2024-04-24T23:03:01.287854+0000","last_became_active":"2024-04-24T23:03:01.287343+0000","last_became_peered":"2024-04-24T23:03:01.287343+0000","last_unstale":"2024-04-24T23:03:01.287854+0000","last_undegraded":"2024-04-24T23:03:01.287854+0000","last_fullsized":"2024-04-24T23:03:01.287854+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:56:48.889426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.284882+0000","last_change":"2024-04-24T23:03:01.284749+0000","last_active":"2024-04-24T23:03:01.284882+0000","last_peered":"2024-04-24T23:03:01.284882+0000","last_clean":"2024-04-24T23:03:01.284882+0000","last_became_active":"2024-04-24T23:03:01.284512+0000","last_became_peered":"2024-04-24T23:03:01.284512+0000","last_unstale":"2024-04-24T23:03:01.284882+0000","last_undegraded":"2024-04-24T23:03:01.284882+0000","last_fullsized":"2024-04-24T23:03:01.284882+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:59:35.976941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.288474+0000","last_change":"2024-04-24T23:03:01.287311+0000","last_active":"2024-04-24T23:03:01.288474+0000","last_peered":"2024-04-24T23:03:01.288474+0000","last_clean":"2024-04-24T23:03:01.288474+0000","last_became_active":"2024-04-24T23:03:01.286497+0000","last_became_peered":"2024-04-24T23:03:01.286497+0000","last_unstale":"2024-04-24T23:03:01.288474+0000","last_undegraded":"2024-04-24T23:03:01.288474+0000","last_fullsized":"2024-04-24T23:03:01.288474+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:37.134636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.292497+0000","last_change":"2024-04-24T23:03:01.292358+0000","last_active":"2024-04-24T23:03:01.292497+0000","last_peered":"2024-04-24T23:03:01.292497+0000","last_clean":"2024-04-24T23:03:01.292497+0000","last_became_active":"2024-04-24T23:03:01.292123+0000","last_became_peered":"2024-04-24T23:03:01.292123+0000","last_unstale":"2024-04-24T23:03:01.292497+0000","last_undegraded":"2024-04-24T23:03:01.292497+0000","last_fullsized":"2024-04-24T23:03:01.292497+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:34:14.584173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.291792+0000","last_change":"2024-04-24T23:03:01.291660+0000","last_active":"2024-04-24T23:03:01.291792+0000","last_peered":"2024-04-24T23:03:01.291792+0000","last_clean":"2024-04-24T23:03:01.291792+0000","last_became_active":"2024-04-24T23:03:01.291396+0000","last_became_peered":"2024-04-24T23:03:01.291396+0000","last_unstale":"2024-04-24T23:03:01.291792+0000","last_undegraded":"2024-04-24T23:03:01.291792+0000","last_fullsized":"2024-04-24T23:03:01.291792+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:08:31.720241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.289555+0000","last_change":"2024-04-24T23:03:01.289384+0000","last_active":"2024-04-24T23:03:01.289555+0000","last_peered":"2024-04-24T23:03:01.289555+0000","last_clean":"2024-04-24T23:03:01.289555+0000","last_became_active":"2024-04-24T23:03:01.289061+0000","last_became_peered":"2024-04-24T23:03:01.289061+0000","last_unstale":"2024-04-24T23:03:01.289555+0000","last_undegraded":"2024-04-24T23:03:01.289555+0000","last_fullsized":"2024-04-24T23:03:01.289555+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:15.537113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"86'4","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.307358+0000","last_change":"2024-04-24T23:03:00.273081+0000","last_active":"2024-04-24T23:03:01.307358+0000","last_peered":"2024-04-24T23:03:01.307358+0000","last_clean":"2024-04-24T23:03:01.307358+0000","last_became_active":"2024-04-24T23:03:00.272403+0000","last_became_peered":"2024-04-24T23:03:00.272403+0000","last_unstale":"2024-04-24T23:03:01.307358+0000","last_undegraded":"2024-04-24T23:03:01.307358+0000","last_fullsized":"2024-04-24T23:03:01.307358+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:02:45.041959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.288562+0000","last_change":"2024-04-24T23:03:01.288421+0000","last_active":"2024-04-24T23:03:01.288562+0000","last_peered":"2024-04-24T23:03:01.288562+0000","last_clean":"2024-04-24T23:03:01.288562+0000","last_became_active":"2024-04-24T23:03:01.287681+0000","last_became_peered":"2024-04-24T23:03:01.287681+0000","last_unstale":"2024-04-24T23:03:01.288562+0000","last_undegraded":"2024-04-24T23:03:01.288562+0000","last_fullsized":"2024-04-24T23:03:01.288562+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:53:37.318204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.305530+0000","last_change":"2024-04-24T23:03:00.270732+0000","last_active":"2024-04-24T23:03:01.305530+0000","last_peered":"2024-04-24T23:03:01.305530+0000","last_clean":"2024-04-24T23:03:01.305530+0000","last_became_active":"2024-04-24T23:03:00.270403+0000","last_became_peered":"2024-04-24T23:03:00.270403+0000","last_unstale":"2024-04-24T23:03:01.305530+0000","last_undegraded":"2024-04-24T23:03:01.305530+0000","last_fullsized":"2024-04-24T23:03:01.305530+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:16.585968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.291834+0000","last_change":"2024-04-24T23:03:01.290966+0000","last_active":"2024-04-24T23:03:01.291834+0000","last_peered":"2024-04-24T23:03:01.291834+0000","last_clean":"2024-04-24T23:03:01.291834+0000","last_became_active":"2024-04-24T23:03:01.290354+0000","last_became_peered":"2024-04-24T23:03:01.290354+0000","last_unstale":"2024-04-24T23:03:01.291834+0000","last_undegraded":"2024-04-24T23:03:01.291834+0000","last_fullsized":"2024-04-24T23:03:01.291834+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:34:27.208184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.305962+0000","last_change":"2024-04-24T23:03:00.268939+0000","last_active":"2024-04-24T23:03:01.305962+0000","last_peered":"2024-04-24T23:03:01.305962+0000","last_clean":"2024-04-24T23:03:01.305962+0000","last_became_active":"2024-04-24T23:03:00.268698+0000","last_became_peered":"2024-04-24T23:03:00.268698+0000","last_unstale":"2024-04-24T23:03:01.305962+0000","last_undegraded":"2024-04-24T23:03:01.305962+0000","last_fullsized":"2024-04-24T23:03:01.305962+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:58:20.761137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.295966+0000","last_change":"2024-04-24T23:03:01.295826+0000","last_active":"2024-04-24T23:03:01.295966+0000","last_peered":"2024-04-24T23:03:01.295966+0000","last_clean":"2024-04-24T23:03:01.295966+0000","last_became_active":"2024-04-24T23:03:01.293764+0000","last_became_peered":"2024-04-24T23:03:01.293764+0000","last_unstale":"2024-04-24T23:03:01.295966+0000","last_undegraded":"2024-04-24T23:03:01.295966+0000","last_fullsized":"2024-04-24T23:03:01.295966+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:30:58.068795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.304856+0000","last_change":"2024-04-24T23:03:00.267222+0000","last_active":"2024-04-24T23:03:01.304856+0000","last_peered":"2024-04-24T23:03:01.304856+0000","last_clean":"2024-04-24T23:03:01.304856+0000","last_became_active":"2024-04-24T23:03:00.266899+0000","last_became_peered":"2024-04-24T23:03:00.266899+0000","last_unstale":"2024-04-24T23:03:01.304856+0000","last_undegraded":"2024-04-24T23:03:01.304856+0000","last_fullsized":"2024-04-24T23:03:01.304856+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:12:37.789738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.290924+0000","last_change":"2024-04-24T23:03:01.290685+0000","last_active":"2024-04-24T23:03:01.290924+0000","last_peered":"2024-04-24T23:03:01.290924+0000","last_clean":"2024-04-24T23:03:01.290924+0000","last_became_active":"2024-04-24T23:03:01.290127+0000","last_became_peered":"2024-04-24T23:03:01.290127+0000","last_unstale":"2024-04-24T23:03:01.290924+0000","last_undegraded":"2024-04-24T23:03:01.290924+0000","last_fullsized":"2024-04-24T23:03:01.290924+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:32:12.500767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.306194+0000","last_change":"2024-04-24T23:03:00.273984+0000","last_active":"2024-04-24T23:03:01.306194+0000","last_peered":"2024-04-24T23:03:01.306194+0000","last_clean":"2024-04-24T23:03:01.306194+0000","last_became_active":"2024-04-24T23:03:00.273768+0000","last_became_peered":"2024-04-24T23:03:00.273768+0000","last_unstale":"2024-04-24T23:03:01.306194+0000","last_undegraded":"2024-04-24T23:03:01.306194+0000","last_fullsized":"2024-04-24T23:03:01.306194+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:25:11.779263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.286419+0000","last_change":"2024-04-24T23:03:01.286217+0000","last_active":"2024-04-24T23:03:01.286419+0000","last_peered":"2024-04-24T23:03:01.286419+0000","last_clean":"2024-04-24T23:03:01.286419+0000","last_became_active":"2024-04-24T23:03:01.285959+0000","last_became_peered":"2024-04-24T23:03:01.285959+0000","last_unstale":"2024-04-24T23:03:01.286419+0000","last_undegraded":"2024-04-24T23:03:01.286419+0000","last_fullsized":"2024-04-24T23:03:01.286419+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:49.422527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.307044+0000","last_change":"2024-04-24T23:03:00.269391+0000","last_active":"2024-04-24T23:03:01.307044+0000","last_peered":"2024-04-24T23:03:01.307044+0000","last_clean":"2024-04-24T23:03:01.307044+0000","last_became_active":"2024-04-24T23:03:00.268694+0000","last_became_peered":"2024-04-24T23:03:00.268694+0000","last_unstale":"2024-04-24T23:03:01.307044+0000","last_undegraded":"2024-04-24T23:03:01.307044+0000","last_fullsized":"2024-04-24T23:03:01.307044+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:21:17.505190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.286029+0000","last_change":"2024-04-24T23:03:01.285357+0000","last_active":"2024-04-24T23:03:01.286029+0000","last_peered":"2024-04-24T23:03:01.286029+0000","last_clean":"2024-04-24T23:03:01.286029+0000","last_became_active":"2024-04-24T23:03:01.285092+0000","last_became_peered":"2024-04-24T23:03:01.285092+0000","last_unstale":"2024-04-24T23:03:01.286029+0000","last_undegraded":"2024-04-24T23:03:01.286029+0000","last_fullsized":"2024-04-24T23:03:01.286029+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:11.553958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.278721+0000","last_change":"2024-04-24T23:03:00.273292+0000","last_active":"2024-04-24T23:03:01.278721+0000","last_peered":"2024-04-24T23:03:01.278721+0000","last_clean":"2024-04-24T23:03:01.278721+0000","last_became_active":"2024-04-24T23:03:00.273011+0000","last_became_peered":"2024-04-24T23:03:00.273011+0000","last_unstale":"2024-04-24T23:03:01.278721+0000","last_undegraded":"2024-04-24T23:03:01.278721+0000","last_fullsized":"2024-04-24T23:03:01.278721+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:09:31.631003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.295234+0000","last_change":"2024-04-24T23:03:01.295108+0000","last_active":"2024-04-24T23:03:01.295234+0000","last_peered":"2024-04-24T23:03:01.295234+0000","last_clean":"2024-04-24T23:03:01.295234+0000","last_became_active":"2024-04-24T23:03:01.294522+0000","last_became_peered":"2024-04-24T23:03:01.294522+0000","last_unstale":"2024-04-24T23:03:01.295234+0000","last_undegraded":"2024-04-24T23:03:01.295234+0000","last_fullsized":"2024-04-24T23:03:01.295234+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:04:42.673547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.279009+0000","last_change":"2024-04-24T23:03:00.270321+0000","last_active":"2024-04-24T23:03:01.279009+0000","last_peered":"2024-04-24T23:03:01.279009+0000","last_clean":"2024-04-24T23:03:01.279009+0000","last_became_active":"2024-04-24T23:03:00.269233+0000","last_became_peered":"2024-04-24T23:03:00.269233+0000","last_unstale":"2024-04-24T23:03:01.279009+0000","last_undegraded":"2024-04-24T23:03:01.279009+0000","last_fullsized":"2024-04-24T23:03:01.279009+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:38:10.621299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.294009+0000","last_change":"2024-04-24T23:03:01.293480+0000","last_active":"2024-04-24T23:03:01.294009+0000","last_peered":"2024-04-24T23:03:01.294009+0000","last_clean":"2024-04-24T23:03:01.294009+0000","last_became_active":"2024-04-24T23:03:01.293076+0000","last_became_peered":"2024-04-24T23:03:01.293076+0000","last_unstale":"2024-04-24T23:03:01.294009+0000","last_undegraded":"2024-04-24T23:03:01.294009+0000","last_fullsized":"2024-04-24T23:03:01.294009+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:03:28.843850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.279314+0000","last_change":"2024-04-24T23:03:00.269513+0000","last_active":"2024-04-24T23:03:01.279314+0000","last_peered":"2024-04-24T23:03:01.279314+0000","last_clean":"2024-04-24T23:03:01.279314+0000","last_became_active":"2024-04-24T23:03:00.268244+0000","last_became_peered":"2024-04-24T23:03:00.268244+0000","last_unstale":"2024-04-24T23:03:01.279314+0000","last_undegraded":"2024-04-24T23:03:01.279314+0000","last_fullsized":"2024-04-24T23:03:01.279314+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:31.425497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.297412+0000","last_change":"2024-04-24T23:03:01.297290+0000","last_active":"2024-04-24T23:03:01.297412+0000","last_peered":"2024-04-24T23:03:01.297412+0000","last_clean":"2024-04-24T23:03:01.297412+0000","last_became_active":"2024-04-24T23:03:01.297044+0000","last_became_peered":"2024-04-24T23:03:01.297044+0000","last_unstale":"2024-04-24T23:03:01.297412+0000","last_undegraded":"2024-04-24T23:03:01.297412+0000","last_fullsized":"2024-04-24T23:03:01.297412+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:37:10.506350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.281961+0000","last_change":"2024-04-24T23:03:00.272578+0000","last_active":"2024-04-24T23:03:01.281961+0000","last_peered":"2024-04-24T23:03:01.281961+0000","last_clean":"2024-04-24T23:03:01.281961+0000","last_became_active":"2024-04-24T23:03:00.271566+0000","last_became_peered":"2024-04-24T23:03:00.271566+0000","last_unstale":"2024-04-24T23:03:01.281961+0000","last_undegraded":"2024-04-24T23:03:01.281961+0000","last_fullsized":"2024-04-24T23:03:01.281961+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:26:29.926479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.295652+0000","last_change":"2024-04-24T23:03:01.295111+0000","last_active":"2024-04-24T23:03:01.295652+0000","last_peered":"2024-04-24T23:03:01.295652+0000","last_clean":"2024-04-24T23:03:01.295652+0000","last_became_active":"2024-04-24T23:03:01.294851+0000","last_became_peered":"2024-04-24T23:03:01.294851+0000","last_unstale":"2024-04-24T23:03:01.295652+0000","last_undegraded":"2024-04-24T23:03:01.295652+0000","last_fullsized":"2024-04-24T23:03:01.295652+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:05:17.948073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"86'3","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.306995+0000","last_change":"2024-04-24T23:03:00.272302+0000","last_active":"2024-04-24T23:03:01.306995+0000","last_peered":"2024-04-24T23:03:01.306995+0000","last_clean":"2024-04-24T23:03:01.306995+0000","last_became_active":"2024-04-24T23:03:00.272059+0000","last_became_peered":"2024-04-24T23:03:00.272059+0000","last_unstale":"2024-04-24T23:03:01.306995+0000","last_undegraded":"2024-04-24T23:03:01.306995+0000","last_fullsized":"2024-04-24T23:03:01.306995+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:04:20.663069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.291002+0000","last_change":"2024-04-24T23:03:01.290878+0000","last_active":"2024-04-24T23:03:01.291002+0000","last_peered":"2024-04-24T23:03:01.291002+0000","last_clean":"2024-04-24T23:03:01.291002+0000","last_became_active":"2024-04-24T23:03:01.290045+0000","last_became_peered":"2024-04-24T23:03:01.290045+0000","last_unstale":"2024-04-24T23:03:01.291002+0000","last_undegraded":"2024-04-24T23:03:01.291002+0000","last_fullsized":"2024-04-24T23:03:01.291002+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:47:39.488434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"86'6","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.307727+0000","last_change":"2024-04-24T23:03:00.269471+0000","last_active":"2024-04-24T23:03:01.307727+0000","last_peered":"2024-04-24T23:03:01.307727+0000","last_clean":"2024-04-24T23:03:01.307727+0000","last_became_active":"2024-04-24T23:03:00.269198+0000","last_became_peered":"2024-04-24T23:03:00.269198+0000","last_unstale":"2024-04-24T23:03:01.307727+0000","last_undegraded":"2024-04-24T23:03:01.307727+0000","last_fullsized":"2024-04-24T23:03:01.307727+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:40:42.391518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.284085+0000","last_change":"2024-04-24T23:03:01.283963+0000","last_active":"2024-04-24T23:03:01.284085+0000","last_peered":"2024-04-24T23:03:01.284085+0000","last_clean":"2024-04-24T23:03:01.284085+0000","last_became_active":"2024-04-24T23:03:01.283644+0000","last_became_peered":"2024-04-24T23:03:01.283644+0000","last_unstale":"2024-04-24T23:03:01.284085+0000","last_undegraded":"2024-04-24T23:03:01.284085+0000","last_fullsized":"2024-04-24T23:03:01.284085+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:11.646502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.306852+0000","last_change":"2024-04-24T23:03:00.271649+0000","last_active":"2024-04-24T23:03:01.306852+0000","last_peered":"2024-04-24T23:03:01.306852+0000","last_clean":"2024-04-24T23:03:01.306852+0000","last_became_active":"2024-04-24T23:03:00.271387+0000","last_became_peered":"2024-04-24T23:03:00.271387+0000","last_unstale":"2024-04-24T23:03:01.306852+0000","last_undegraded":"2024-04-24T23:03:01.306852+0000","last_fullsized":"2024-04-24T23:03:01.306852+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:11.107776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.295048+0000","last_change":"2024-04-24T23:03:01.294910+0000","last_active":"2024-04-24T23:03:01.295048+0000","last_peered":"2024-04-24T23:03:01.295048+0000","last_clean":"2024-04-24T23:03:01.295048+0000","last_became_active":"2024-04-24T23:03:01.293260+0000","last_became_peered":"2024-04-24T23:03:01.293260+0000","last_unstale":"2024-04-24T23:03:01.295048+0000","last_undegraded":"2024-04-24T23:03:01.295048+0000","last_fullsized":"2024-04-24T23:03:01.295048+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:42:30.609881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.287451+0000","last_change":"2024-04-24T23:03:00.269136+0000","last_active":"2024-04-24T23:03:01.287451+0000","last_peered":"2024-04-24T23:03:01.287451+0000","last_clean":"2024-04-24T23:03:01.287451+0000","last_became_active":"2024-04-24T23:03:00.268885+0000","last_became_peered":"2024-04-24T23:03:00.268885+0000","last_unstale":"2024-04-24T23:03:01.287451+0000","last_undegraded":"2024-04-24T23:03:01.287451+0000","last_fullsized":"2024-04-24T23:03:01.287451+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:36.232639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.292047+0000","last_change":"2024-04-24T23:03:01.291934+0000","last_active":"2024-04-24T23:03:01.292047+0000","last_peered":"2024-04-24T23:03:01.292047+0000","last_clean":"2024-04-24T23:03:01.292047+0000","last_became_active":"2024-04-24T23:03:01.291714+0000","last_became_peered":"2024-04-24T23:03:01.291714+0000","last_unstale":"2024-04-24T23:03:01.292047+0000","last_undegraded":"2024-04-24T23:03:01.292047+0000","last_fullsized":"2024-04-24T23:03:01.292047+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:04:41.265768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.282958+0000","last_change":"2024-04-24T23:03:00.270636+0000","last_active":"2024-04-24T23:03:01.282958+0000","last_peered":"2024-04-24T23:03:01.282958+0000","last_clean":"2024-04-24T23:03:01.282958+0000","last_became_active":"2024-04-24T23:03:00.270384+0000","last_became_peered":"2024-04-24T23:03:00.270384+0000","last_unstale":"2024-04-24T23:03:01.282958+0000","last_undegraded":"2024-04-24T23:03:01.282958+0000","last_fullsized":"2024-04-24T23:03:01.282958+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:55:17.047770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.290462+0000","last_change":"2024-04-24T23:03:01.290170+0000","last_active":"2024-04-24T23:03:01.290462+0000","last_peered":"2024-04-24T23:03:01.290462+0000","last_clean":"2024-04-24T23:03:01.290462+0000","last_became_active":"2024-04-24T23:03:01.289928+0000","last_became_peered":"2024-04-24T23:03:01.289928+0000","last_unstale":"2024-04-24T23:03:01.290462+0000","last_undegraded":"2024-04-24T23:03:01.290462+0000","last_fullsized":"2024-04-24T23:03:01.290462+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:11:41.609894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"86'8","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.306765+0000","last_change":"2024-04-24T23:03:00.269110+0000","last_active":"2024-04-24T23:03:01.306765+0000","last_peered":"2024-04-24T23:03:01.306765+0000","last_clean":"2024-04-24T23:03:01.306765+0000","last_became_active":"2024-04-24T23:03:00.268085+0000","last_became_peered":"2024-04-24T23:03:00.268085+0000","last_unstale":"2024-04-24T23:03:01.306765+0000","last_undegraded":"2024-04-24T23:03:01.306765+0000","last_fullsized":"2024-04-24T23:03:01.306765+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:34:47.213313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.289243+0000","last_change":"2024-04-24T23:03:01.289081+0000","last_active":"2024-04-24T23:03:01.289243+0000","last_peered":"2024-04-24T23:03:01.289243+0000","last_clean":"2024-04-24T23:03:01.289243+0000","last_became_active":"2024-04-24T23:03:01.288241+0000","last_became_peered":"2024-04-24T23:03:01.288241+0000","last_unstale":"2024-04-24T23:03:01.289243+0000","last_undegraded":"2024-04-24T23:03:01.289243+0000","last_fullsized":"2024-04-24T23:03:01.289243+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:05:33.715670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.301371+0000","last_change":"2024-04-24T23:03:00.266103+0000","last_active":"2024-04-24T23:03:01.301371+0000","last_peered":"2024-04-24T23:03:01.301371+0000","last_clean":"2024-04-24T23:03:01.301371+0000","last_became_active":"2024-04-24T23:03:00.265836+0000","last_became_peered":"2024-04-24T23:03:00.265836+0000","last_unstale":"2024-04-24T23:03:01.301371+0000","last_undegraded":"2024-04-24T23:03:01.301371+0000","last_fullsized":"2024-04-24T23:03:01.301371+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:18:06.416870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.288252+0000","last_change":"2024-04-24T23:03:01.288117+0000","last_active":"2024-04-24T23:03:01.288252+0000","last_peered":"2024-04-24T23:03:01.288252+0000","last_clean":"2024-04-24T23:03:01.288252+0000","last_became_active":"2024-04-24T23:03:01.287794+0000","last_became_peered":"2024-04-24T23:03:01.287794+0000","last_unstale":"2024-04-24T23:03:01.288252+0000","last_undegraded":"2024-04-24T23:03:01.288252+0000","last_fullsized":"2024-04-24T23:03:01.288252+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:59:38.865133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.306224+0000","last_change":"2024-04-24T23:03:00.271671+0000","last_active":"2024-04-24T23:03:01.306224+0000","last_peered":"2024-04-24T23:03:01.306224+0000","last_clean":"2024-04-24T23:03:01.306224+0000","last_became_active":"2024-04-24T23:03:00.270672+0000","last_became_peered":"2024-04-24T23:03:00.270672+0000","last_unstale":"2024-04-24T23:03:01.306224+0000","last_undegraded":"2024-04-24T23:03:01.306224+0000","last_fullsized":"2024-04-24T23:03:01.306224+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:12:25.077100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.288389+0000","last_change":"2024-04-24T23:03:01.288264+0000","last_active":"2024-04-24T23:03:01.288389+0000","last_peered":"2024-04-24T23:03:01.288389+0000","last_clean":"2024-04-24T23:03:01.288389+0000","last_became_active":"2024-04-24T23:03:01.287905+0000","last_became_peered":"2024-04-24T23:03:01.287905+0000","last_unstale":"2024-04-24T23:03:01.288389+0000","last_undegraded":"2024-04-24T23:03:01.288389+0000","last_fullsized":"2024-04-24T23:03:01.288389+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:46:15.873614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.306773+0000","last_change":"2024-04-24T23:03:00.275516+0000","last_active":"2024-04-24T23:03:01.306773+0000","last_peered":"2024-04-24T23:03:01.306773+0000","last_clean":"2024-04-24T23:03:01.306773+0000","last_became_active":"2024-04-24T23:03:00.275063+0000","last_became_peered":"2024-04-24T23:03:00.275063+0000","last_unstale":"2024-04-24T23:03:01.306773+0000","last_undegraded":"2024-04-24T23:03:01.306773+0000","last_fullsized":"2024-04-24T23:03:01.306773+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:00.798452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.289551+0000","last_change":"2024-04-24T23:03:01.289155+0000","last_active":"2024-04-24T23:03:01.289551+0000","last_peered":"2024-04-24T23:03:01.289551+0000","last_clean":"2024-04-24T23:03:01.289551+0000","last_became_active":"2024-04-24T23:03:01.288908+0000","last_became_peered":"2024-04-24T23:03:01.288908+0000","last_unstale":"2024-04-24T23:03:01.289551+0000","last_undegraded":"2024-04-24T23:03:01.289551+0000","last_fullsized":"2024-04-24T23:03:01.289551+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:47.863641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.283271+0000","last_change":"2024-04-24T23:03:00.271647+0000","last_active":"2024-04-24T23:03:01.283271+0000","last_peered":"2024-04-24T23:03:01.283271+0000","last_clean":"2024-04-24T23:03:01.283271+0000","last_became_active":"2024-04-24T23:03:00.271397+0000","last_became_peered":"2024-04-24T23:03:00.271397+0000","last_unstale":"2024-04-24T23:03:01.283271+0000","last_undegraded":"2024-04-24T23:03:01.283271+0000","last_fullsized":"2024-04-24T23:03:01.283271+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:22.043771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.286783+0000","last_change":"2024-04-24T23:03:01.285830+0000","last_active":"2024-04-24T23:03:01.286783+0000","last_peered":"2024-04-24T23:03:01.286783+0000","last_clean":"2024-04-24T23:03:01.286783+0000","last_became_active":"2024-04-24T23:03:01.285504+0000","last_became_peered":"2024-04-24T23:03:01.285504+0000","last_unstale":"2024-04-24T23:03:01.286783+0000","last_undegraded":"2024-04-24T23:03:01.286783+0000","last_fullsized":"2024-04-24T23:03:01.286783+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:44:53.420886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.284061+0000","last_change":"2024-04-24T23:03:00.273580+0000","last_active":"2024-04-24T23:03:01.284061+0000","last_peered":"2024-04-24T23:03:01.284061+0000","last_clean":"2024-04-24T23:03:01.284061+0000","last_became_active":"2024-04-24T23:03:00.273018+0000","last_became_peered":"2024-04-24T23:03:00.273018+0000","last_unstale":"2024-04-24T23:03:01.284061+0000","last_undegraded":"2024-04-24T23:03:01.284061+0000","last_fullsized":"2024-04-24T23:03:01.284061+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:02:51.326512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.286399+0000","last_change":"2024-04-24T23:03:01.286132+0000","last_active":"2024-04-24T23:03:01.286399+0000","last_peered":"2024-04-24T23:03:01.286399+0000","last_clean":"2024-04-24T23:03:01.286399+0000","last_became_active":"2024-04-24T23:03:01.285659+0000","last_became_peered":"2024-04-24T23:03:01.285659+0000","last_unstale":"2024-04-24T23:03:01.286399+0000","last_undegraded":"2024-04-24T23:03:01.286399+0000","last_fullsized":"2024-04-24T23:03:01.286399+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:24:17.354917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.282158+0000","last_change":"2024-04-24T23:03:00.274527+0000","last_active":"2024-04-24T23:03:01.282158+0000","last_peered":"2024-04-24T23:03:01.282158+0000","last_clean":"2024-04-24T23:03:01.282158+0000","last_became_active":"2024-04-24T23:03:00.274285+0000","last_became_peered":"2024-04-24T23:03:00.274285+0000","last_unstale":"2024-04-24T23:03:01.282158+0000","last_undegraded":"2024-04-24T23:03:01.282158+0000","last_fullsized":"2024-04-24T23:03:01.282158+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:35.739293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.286262+0000","last_change":"2024-04-24T23:03:01.286112+0000","last_active":"2024-04-24T23:03:01.286262+0000","last_peered":"2024-04-24T23:03:01.286262+0000","last_clean":"2024-04-24T23:03:01.286262+0000","last_became_active":"2024-04-24T23:03:01.285864+0000","last_became_peered":"2024-04-24T23:03:01.285864+0000","last_unstale":"2024-04-24T23:03:01.286262+0000","last_undegraded":"2024-04-24T23:03:01.286262+0000","last_fullsized":"2024-04-24T23:03:01.286262+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:04:26.827759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.281237+0000","last_change":"2024-04-24T23:03:00.268748+0000","last_active":"2024-04-24T23:03:01.281237+0000","last_peered":"2024-04-24T23:03:01.281237+0000","last_clean":"2024-04-24T23:03:01.281237+0000","last_became_active":"2024-04-24T23:03:00.268454+0000","last_became_peered":"2024-04-24T23:03:00.268454+0000","last_unstale":"2024-04-24T23:03:01.281237+0000","last_undegraded":"2024-04-24T23:03:01.281237+0000","last_fullsized":"2024-04-24T23:03:01.281237+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:49.701035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.291258+0000","last_change":"2024-04-24T23:03:01.291116+0000","last_active":"2024-04-24T23:03:01.291258+0000","last_peered":"2024-04-24T23:03:01.291258+0000","last_clean":"2024-04-24T23:03:01.291258+0000","last_became_active":"2024-04-24T23:03:01.290614+0000","last_became_peered":"2024-04-24T23:03:01.290614+0000","last_unstale":"2024-04-24T23:03:01.291258+0000","last_undegraded":"2024-04-24T23:03:01.291258+0000","last_fullsized":"2024-04-24T23:03:01.291258+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:58:47.826986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.304325+0000","last_change":"2024-04-24T23:03:00.269801+0000","last_active":"2024-04-24T23:03:01.304325+0000","last_peered":"2024-04-24T23:03:01.304325+0000","last_clean":"2024-04-24T23:03:01.304325+0000","last_became_active":"2024-04-24T23:03:00.269553+0000","last_became_peered":"2024-04-24T23:03:00.269553+0000","last_unstale":"2024-04-24T23:03:01.304325+0000","last_undegraded":"2024-04-24T23:03:01.304325+0000","last_fullsized":"2024-04-24T23:03:01.304325+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:47:15.087313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.291727+0000","last_change":"2024-04-24T23:03:01.289252+0000","last_active":"2024-04-24T23:03:01.291727+0000","last_peered":"2024-04-24T23:03:01.291727+0000","last_clean":"2024-04-24T23:03:01.291727+0000","last_became_active":"2024-04-24T23:03:01.289013+0000","last_became_peered":"2024-04-24T23:03:01.289013+0000","last_unstale":"2024-04-24T23:03:01.291727+0000","last_undegraded":"2024-04-24T23:03:01.291727+0000","last_fullsized":"2024-04-24T23:03:01.291727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:36:24.322251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.278208+0000","last_change":"2024-04-24T23:03:00.268185+0000","last_active":"2024-04-24T23:03:01.278208+0000","last_peered":"2024-04-24T23:03:01.278208+0000","last_clean":"2024-04-24T23:03:01.278208+0000","last_became_active":"2024-04-24T23:03:00.267255+0000","last_became_peered":"2024-04-24T23:03:00.267255+0000","last_unstale":"2024-04-24T23:03:01.278208+0000","last_undegraded":"2024-04-24T23:03:01.278208+0000","last_fullsized":"2024-04-24T23:03:01.278208+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:58.240612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.289465+0000","last_change":"2024-04-24T23:03:01.289331+0000","last_active":"2024-04-24T23:03:01.289465+0000","last_peered":"2024-04-24T23:03:01.289465+0000","last_clean":"2024-04-24T23:03:01.289465+0000","last_became_active":"2024-04-24T23:03:01.288677+0000","last_became_peered":"2024-04-24T23:03:01.288677+0000","last_unstale":"2024-04-24T23:03:01.289465+0000","last_undegraded":"2024-04-24T23:03:01.289465+0000","last_fullsized":"2024-04-24T23:03:01.289465+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:07.386974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.280438+0000","last_change":"2024-04-24T23:03:00.272666+0000","last_active":"2024-04-24T23:03:01.280438+0000","last_peered":"2024-04-24T23:03:01.280438+0000","last_clean":"2024-04-24T23:03:01.280438+0000","last_became_active":"2024-04-24T23:03:00.271406+0000","last_became_peered":"2024-04-24T23:03:00.271406+0000","last_unstale":"2024-04-24T23:03:01.280438+0000","last_undegraded":"2024-04-24T23:03:01.280438+0000","last_fullsized":"2024-04-24T23:03:01.280438+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:45:59.370519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.294747+0000","last_change":"2024-04-24T23:03:01.294628+0000","last_active":"2024-04-24T23:03:01.294747+0000","last_peered":"2024-04-24T23:03:01.294747+0000","last_clean":"2024-04-24T23:03:01.294747+0000","last_became_active":"2024-04-24T23:03:01.294366+0000","last_became_peered":"2024-04-24T23:03:01.294366+0000","last_unstale":"2024-04-24T23:03:01.294747+0000","last_undegraded":"2024-04-24T23:03:01.294747+0000","last_fullsized":"2024-04-24T23:03:01.294747+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:21.992711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.279148+0000","last_change":"2024-04-24T23:03:00.268434+0000","last_active":"2024-04-24T23:03:01.279148+0000","last_peered":"2024-04-24T23:03:01.279148+0000","last_clean":"2024-04-24T23:03:01.279148+0000","last_became_active":"2024-04-24T23:03:00.268134+0000","last_became_peered":"2024-04-24T23:03:00.268134+0000","last_unstale":"2024-04-24T23:03:01.279148+0000","last_undegraded":"2024-04-24T23:03:01.279148+0000","last_fullsized":"2024-04-24T23:03:01.279148+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:42:29.503384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.286101+0000","last_change":"2024-04-24T23:03:01.285990+0000","last_active":"2024-04-24T23:03:01.286101+0000","last_peered":"2024-04-24T23:03:01.286101+0000","last_clean":"2024-04-24T23:03:01.286101+0000","last_became_active":"2024-04-24T23:03:01.285571+0000","last_became_peered":"2024-04-24T23:03:01.285571+0000","last_unstale":"2024-04-24T23:03:01.286101+0000","last_undegraded":"2024-04-24T23:03:01.286101+0000","last_fullsized":"2024-04-24T23:03:01.286101+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:59:34.804951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.278483+0000","last_change":"2024-04-24T23:03:00.269313+0000","last_active":"2024-04-24T23:03:01.278483+0000","last_peered":"2024-04-24T23:03:01.278483+0000","last_clean":"2024-04-24T23:03:01.278483+0000","last_became_active":"2024-04-24T23:03:00.269076+0000","last_became_peered":"2024-04-24T23:03:00.269076+0000","last_unstale":"2024-04-24T23:03:01.278483+0000","last_undegraded":"2024-04-24T23:03:01.278483+0000","last_fullsized":"2024-04-24T23:03:01.278483+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:08.306657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.291830+0000","last_change":"2024-04-24T23:03:01.291334+0000","last_active":"2024-04-24T23:03:01.291830+0000","last_peered":"2024-04-24T23:03:01.291830+0000","last_clean":"2024-04-24T23:03:01.291830+0000","last_became_active":"2024-04-24T23:03:01.289947+0000","last_became_peered":"2024-04-24T23:03:01.289947+0000","last_unstale":"2024-04-24T23:03:01.291830+0000","last_undegraded":"2024-04-24T23:03:01.291830+0000","last_fullsized":"2024-04-24T23:03:01.291830+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:49:21.158646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.281336+0000","last_change":"2024-04-24T23:03:00.267882+0000","last_active":"2024-04-24T23:03:01.281336+0000","last_peered":"2024-04-24T23:03:01.281336+0000","last_clean":"2024-04-24T23:03:01.281336+0000","last_became_active":"2024-04-24T23:03:00.267600+0000","last_became_peered":"2024-04-24T23:03:00.267600+0000","last_unstale":"2024-04-24T23:03:01.281336+0000","last_undegraded":"2024-04-24T23:03:01.281336+0000","last_fullsized":"2024-04-24T23:03:01.281336+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:58.367970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.290711+0000","last_change":"2024-04-24T23:03:01.290474+0000","last_active":"2024-04-24T23:03:01.290711+0000","last_peered":"2024-04-24T23:03:01.290711+0000","last_clean":"2024-04-24T23:03:01.290711+0000","last_became_active":"2024-04-24T23:03:01.290221+0000","last_became_peered":"2024-04-24T23:03:01.290221+0000","last_unstale":"2024-04-24T23:03:01.290711+0000","last_undegraded":"2024-04-24T23:03:01.290711+0000","last_fullsized":"2024-04-24T23:03:01.290711+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:31:10.422483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.282157+0000","last_change":"2024-04-24T23:03:00.272260+0000","last_active":"2024-04-24T23:03:01.282157+0000","last_peered":"2024-04-24T23:03:01.282157+0000","last_clean":"2024-04-24T23:03:01.282157+0000","last_became_active":"2024-04-24T23:03:00.271163+0000","last_became_peered":"2024-04-24T23:03:00.271163+0000","last_unstale":"2024-04-24T23:03:01.282157+0000","last_undegraded":"2024-04-24T23:03:01.282157+0000","last_fullsized":"2024-04-24T23:03:01.282157+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:01:46.026496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.289402+0000","last_change":"2024-04-24T23:03:01.289277+0000","last_active":"2024-04-24T23:03:01.289402+0000","last_peered":"2024-04-24T23:03:01.289402+0000","last_clean":"2024-04-24T23:03:01.289402+0000","last_became_active":"2024-04-24T23:03:01.289021+0000","last_became_peered":"2024-04-24T23:03:01.289021+0000","last_unstale":"2024-04-24T23:03:01.289402+0000","last_undegraded":"2024-04-24T23:03:01.289402+0000","last_fullsized":"2024-04-24T23:03:01.289402+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:24.651133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.284258+0000","last_change":"2024-04-24T23:03:00.270686+0000","last_active":"2024-04-24T23:03:01.284258+0000","last_peered":"2024-04-24T23:03:01.284258+0000","last_clean":"2024-04-24T23:03:01.284258+0000","last_became_active":"2024-04-24T23:03:00.270334+0000","last_became_peered":"2024-04-24T23:03:00.270334+0000","last_unstale":"2024-04-24T23:03:01.284258+0000","last_undegraded":"2024-04-24T23:03:01.284258+0000","last_fullsized":"2024-04-24T23:03:01.284258+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:12:17.932580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.285484+0000","last_change":"2024-04-24T23:03:01.285374+0000","last_active":"2024-04-24T23:03:01.285484+0000","last_peered":"2024-04-24T23:03:01.285484+0000","last_clean":"2024-04-24T23:03:01.285484+0000","last_became_active":"2024-04-24T23:03:01.285116+0000","last_became_peered":"2024-04-24T23:03:01.285116+0000","last_unstale":"2024-04-24T23:03:01.285484+0000","last_undegraded":"2024-04-24T23:03:01.285484+0000","last_fullsized":"2024-04-24T23:03:01.285484+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:00.255589+0000","last_clean_scrub_stamp":"2024-04-24T23:03:00.255589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:46:43.074985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.306891+0000","last_change":"2024-04-24T23:03:00.270687+0000","last_active":"2024-04-24T23:03:01.306891+0000","last_peered":"2024-04-24T23:03:01.306891+0000","last_clean":"2024-04-24T23:03:01.306891+0000","last_became_active":"2024-04-24T23:03:00.270437+0000","last_became_peered":"2024-04-24T23:03:00.270437+0000","last_unstale":"2024-04-24T23:03:01.306891+0000","last_undegraded":"2024-04-24T23:03:01.306891+0000","last_fullsized":"2024-04-24T23:03:01.306891+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:02:59.251523+0000","last_clean_scrub_stamp":"2024-04-24T23:02:59.251523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:35.640764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":297,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:03:01.291115+0000","last_change":"2024-04-24T22:58:55.138810+0000","last_active":"2024-04-24T23:03:01.291115+0000","last_peered":"2024-04-24T23:03:01.291115+0000","last_clean":"2024-04-24T23:03:01.291115+0000","last_became_active":"2024-04-24T22:58:55.138627+0000","last_became_peered":"2024-04-24T22:58:55.138627+0000","last_unstale":"2024-04-24T23:03:01.291115+0000","last_undegraded":"2024-04-24T23:03:01.291115+0000","last_fullsized":"2024-04-24T23:03:01.291115+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_clean_scrub_stamp":"2024-04-24T22:58:54.127235+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:32:00.228724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574900,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":449,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194863,"statfs":{"total":200000000,"available":199539957,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574900,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":449,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194863,"statfs":{"total":200000000,"available":199539923,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574900,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1024,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194288,"statfs":{"total":200000000,"available":198951111,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574900,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":441,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194871,"statfs":{"total":200000000,"available":199548344,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574900,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":447,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194864,"statfs":{"total":200000000,"available":199541443,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574900,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":448,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194864,"statfs":{"total":200000000,"available":199540987,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574900,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1024,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194288,"statfs":{"total":200000000,"available":198951201,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574900,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":447,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194864,"statfs":{"total":200000000,"available":199541499,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-24T23:03:05.647 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-24T23:03:05.647 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-24T23:03:05.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.789+0000 7fc98ff53640 1 Processor -- start 2024-04-24T23:03:05.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.790+0000 7fc98ff53640 1 -- start start 2024-04-24T23:03:05.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.790+0000 7fc98ff53640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc98806c5f0 0x7fc98806c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:05.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.790+0000 7fc98ff53640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9880b6430 con 0x7fc9880af890 2024-04-24T23:03:05.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.790+0000 7fc98ff53640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc988156f80 con 0x7fc9880b44d0 2024-04-24T23:03:05.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.790+0000 7fc98ff53640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc9880b6150 con 0x7fc98806c5f0 2024-04-24T23:03:05.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.791+0000 7fc98e4f0640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc9880af890 0x7fc98815cac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57622/0 (socket says 172.21.15.32:57622) 2024-04-24T23:03:05.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.791+0000 7fc98e4f0640 1 -- 172.21.15.32:0/2699989734 learned_addr learned my addr 172.21.15.32:0/2699989734 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:03:05.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.791+0000 7fc98dcef640 1 --2- 172.21.15.32:0/2699989734 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc98806c5f0 0x7fc98806c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:05.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.791+0000 7fc98cced640 1 -- 172.21.15.32:0/2699989734 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2840300356 0 0) 0x7fc9880b6430 con 0x7fc9880af890 2024-04-24T23:03:05.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.791+0000 7fc98cced640 1 -- 172.21.15.32:0/2699989734 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc970003520 con 0x7fc9880af890 2024-04-24T23:03:05.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.791+0000 7fc98cced640 1 -- 172.21.15.32:0/2699989734 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2642734248 0 0) 0x7fc970003520 con 0x7fc9880af890 2024-04-24T23:03:05.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.791+0000 7fc98cced640 1 -- 172.21.15.32:0/2699989734 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc98806c5f0 msgr2=0x7fc98806c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:05.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.791+0000 7fc98cced640 1 --2- 172.21.15.32:0/2699989734 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc98806c5f0 0x7fc98806c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:05.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.791+0000 7fc98cced640 1 -- 172.21.15.32:0/2699989734 >> v1:172.21.15.121:6789/0 conn(0x7fc9880b44d0 legacy=0x7fc9880b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:05.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.791+0000 7fc98cced640 1 -- 172.21.15.32:0/2699989734 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9880b39f0 con 0x7fc9880af890 2024-04-24T23:03:05.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.791+0000 7fc98cced640 1 -- 172.21.15.32:0/2699989734 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7fc9780019f0 con 0x7fc9880af890 2024-04-24T23:03:05.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.791+0000 7fc98cced640 1 -- 172.21.15.32:0/2699989734 >> v1:172.21.15.32:6789/0 conn(0x7fc9880af890 legacy=0x7fc98815cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:05.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.791+0000 7fc98cced640 1 --2- 172.21.15.32:0/2699989734 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc98806c5f0 0x7fc9700039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:05.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.791+0000 7fc98cced640 1 --2- 172.21.15.32:0/2699989734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc970003f60 0x7fc970004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:05.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.791+0000 7fc98cced640 1 --2- 172.21.15.32:0/2699989734 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc970004d40 0x7fc9700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:05.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.791+0000 7fc98cced640 1 -- 172.21.15.32:0/2699989734 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc9880b6150 con 0x7fc970003f60 2024-04-24T23:03:05.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.791+0000 7fc98cced640 1 -- 172.21.15.32:0/2699989734 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc97000bb50 con 0x7fc970004d40 2024-04-24T23:03:05.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.791+0000 7fc98cced640 1 -- 172.21.15.32:0/2699989734 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc97000bd50 con 0x7fc98806c5f0 2024-04-24T23:03:05.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.791+0000 7fc98d4ee640 1 --2- 172.21.15.32:0/2699989734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc970003f60 0x7fc970004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:05.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.792+0000 7fc98dcef640 1 --2- 172.21.15.32:0/2699989734 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc98806c5f0 0x7fc9700039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:05.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.792+0000 7fc98e4f0640 1 --2- 172.21.15.32:0/2699989734 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc970004d40 0x7fc9700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:05.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.792+0000 7fc98d4ee640 1 -- 172.21.15.32:0/2699989734 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc98806c5f0 msgr2=0x7fc9700039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:05.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.792+0000 7fc98d4ee640 1 --2- 172.21.15.32:0/2699989734 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc98806c5f0 0x7fc9700039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:05.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.792+0000 7fc98d4ee640 1 -- 172.21.15.32:0/2699989734 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc970004d40 msgr2=0x7fc9700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:05.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.792+0000 7fc98d4ee640 1 --2- 172.21.15.32:0/2699989734 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc970004d40 0x7fc9700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:05.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.792+0000 7fc98d4ee640 1 -- 172.21.15.32:0/2699989734 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc970003520 con 0x7fc970003f60 2024-04-24T23:03:05.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.792+0000 7fc98d4ee640 1 --2- 172.21.15.32:0/2699989734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc970003f60 0x7fc970004800 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fc97c001720 tx=0x7fc970005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fc066b1e22e1ab server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:05.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.793+0000 7fc98cced640 1 -- 172.21.15.32:0/2699989734 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97c001370 con 0x7fc970003f60 2024-04-24T23:03:05.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.793+0000 7fc98cced640 1 -- 172.21.15.32:0/2699989734 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc97c0052d0 con 0x7fc970003f60 2024-04-24T23:03:05.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.793+0000 7fc98cced640 1 -- 172.21.15.32:0/2699989734 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97c02abc0 con 0x7fc970003f60 2024-04-24T23:03:05.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.793+0000 7fc98ff53640 1 -- 172.21.15.32:0/2699989734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc970003f60 msgr2=0x7fc970004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:05.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.793+0000 7fc98ff53640 1 --2- 172.21.15.32:0/2699989734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc970003f60 0x7fc970004800 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fc97c001720 tx=0x7fc970005830 comp rx=0 tx=0).stop 2024-04-24T23:03:05.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.793+0000 7fc98e4f0640 1 -- 172.21.15.32:0/2699989734 reap_dead start 2024-04-24T23:03:05.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.793+0000 7fc98ff53640 1 -- 172.21.15.32:0/2699989734 shutdown_connections 2024-04-24T23:03:05.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.793+0000 7fc98ff53640 1 -- 172.21.15.32:0/2699989734 >> 172.21.15.32:0/2699989734 conn(0x7fc98806b110 msgr2=0x7fc98806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:05.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.793+0000 7fc98ff53640 1 -- 172.21.15.32:0/2699989734 shutdown_connections 2024-04-24T23:03:05.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.793+0000 7fc98ff53640 1 -- 172.21.15.32:0/2699989734 wait complete. 2024-04-24T23:03:05.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.794+0000 7fc98ff53640 1 Processor -- start 2024-04-24T23:03:05.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.794+0000 7fc98ff53640 1 -- start start 2024-04-24T23:03:05.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.795+0000 7fc98ff53640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc970003f60 0x7fc988140d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:05.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.795+0000 7fc98ff53640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9881f3840 0x7fc9881f69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:05.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.795+0000 7fc98ff53640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9881507f0 0x7fc988150ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:05.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.795+0000 7fc98ff53640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc98806c9d0 con 0x7fc9881f3840 2024-04-24T23:03:05.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.795+0000 7fc98ff53640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc9880b3fb0 con 0x7fc970003f60 2024-04-24T23:03:05.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.795+0000 7fc98ff53640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc98806cb50 con 0x7fc9881507f0 2024-04-24T23:03:05.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.795+0000 7fc98d4ee640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9881f3840 0x7fc9881f69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:05.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.795+0000 7fc98d4ee640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9881f3840 0x7fc9881f69b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38490/0 (socket says 172.21.15.32:38490) 2024-04-24T23:03:05.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.795+0000 7fc98d4ee640 1 -- 172.21.15.32:0/2784696275 learned_addr learned my addr 172.21.15.32:0/2784696275 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:05.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.795+0000 7fc98dcef640 1 --2- 172.21.15.32:0/2784696275 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc970003f60 0x7fc988140d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:05.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.795+0000 7fc98e4f0640 1 --2- 172.21.15.32:0/2784696275 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9881507f0 0x7fc988150ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:05.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.796+0000 7fc98d4ee640 1 -- 172.21.15.32:0/2784696275 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9881507f0 msgr2=0x7fc988150ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:05.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.796+0000 7fc98d4ee640 1 --2- 172.21.15.32:0/2784696275 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9881507f0 0x7fc988150ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:05.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.796+0000 7fc98d4ee640 1 -- 172.21.15.32:0/2784696275 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc970003f60 msgr2=0x7fc988140d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:05.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.796+0000 7fc98d4ee640 1 --2- 172.21.15.32:0/2784696275 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc970003f60 0x7fc988140d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:05.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.796+0000 7fc98d4ee640 1 -- 172.21.15.32:0/2784696275 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc988141240 con 0x7fc9881f3840 2024-04-24T23:03:05.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.796+0000 7fc98d4ee640 1 --2- 172.21.15.32:0/2784696275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9881f3840 0x7fc9881f69b0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fc97c001840 tx=0x7fc97c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:05.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.796+0000 7fc9867fc640 1 -- 172.21.15.32:0/2784696275 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97c007a50 con 0x7fc9881f3840 2024-04-24T23:03:05.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.796+0000 7fc9867fc640 1 -- 172.21.15.32:0/2784696275 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc97c041070 con 0x7fc9881f3840 2024-04-24T23:03:05.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.796+0000 7fc98ff53640 1 -- 172.21.15.32:0/2784696275 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc988152960 con 0x7fc9881f3840 2024-04-24T23:03:05.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.796+0000 7fc9867fc640 1 -- 172.21.15.32:0/2784696275 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97c001370 con 0x7fc9881f3840 2024-04-24T23:03:05.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.796+0000 7fc98ff53640 1 -- 172.21.15.32:0/2784696275 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc988152f20 con 0x7fc9881f3840 2024-04-24T23:03:05.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.797+0000 7fc98ff53640 1 -- 172.21.15.32:0/2784696275 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc988068430 con 0x7fc9881f3840 2024-04-24T23:03:05.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.797+0000 7fc9867fc640 1 -- 172.21.15.32:0/2784696275 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc97c0052d0 con 0x7fc9881f3840 2024-04-24T23:03:05.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.798+0000 7fc9867fc640 1 --2- 172.21.15.32:0/2784696275 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc97406be30 0x7fc97406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:05.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.798+0000 7fc9867fc640 1 -- 172.21.15.32:0/2784696275 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7fc97c0aaf80 con 0x7fc9881f3840 2024-04-24T23:03:05.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.798+0000 7fc98dcef640 1 --2- 172.21.15.32:0/2784696275 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc97406be30 0x7fc97406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:05.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.799+0000 7fc98dcef640 1 --2- 172.21.15.32:0/2784696275 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc97406be30 0x7fc97406e2f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc980009bc0 tx=0x7fc980009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:05.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:05.802+0000 7fc9867fc640 1 -- 172.21.15.32:0/2784696275 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc97c079930 con 0x7fc9881f3840 2024-04-24T23:03:06.034 INFO:tasks.ceph.mds.a.smithi032.stderr:2024-04-24T23:03:06.033+0000 7f97e59db640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T23:03:06.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.055+0000 7fc98ff53640 1 -- 172.21.15.32:0/2784696275 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fc9880cc7b0 con 0x7fc9881f3840 2024-04-24T23:03:06.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.057+0000 7fc9867fc640 1 -- 172.21.15.32:0/2784696275 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) ==== 190+0+147 (secure 0 0 0) 0x7fc97c079230 con 0x7fc9881f3840 2024-04-24T23:03:06.058 INFO:teuthology.orchestra.run.smithi032.stdout:[client.0] 2024-04-24T23:03:06.059 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQCejilm5oeiGRAAdLXwwqrYhV3HHo7Dw/2ZqQ== 2024-04-24T23:03:06.059 INFO:teuthology.orchestra.run.smithi032.stdout: caps mds = "allow" 2024-04-24T23:03:06.059 INFO:teuthology.orchestra.run.smithi032.stdout: caps mon = "allow r" 2024-04-24T23:03:06.059 INFO:teuthology.orchestra.run.smithi032.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-24T23:03:06.059 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2024-04-24T23:03:06.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.059+0000 7fc98ff53640 1 -- 172.21.15.32:0/2784696275 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc97406be30 msgr2=0x7fc97406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:06.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.059+0000 7fc98ff53640 1 --2- 172.21.15.32:0/2784696275 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc97406be30 0x7fc97406e2f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc980009bc0 tx=0x7fc980009190 comp rx=0 tx=0).stop 2024-04-24T23:03:06.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.059+0000 7fc98ff53640 1 -- 172.21.15.32:0/2784696275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9881f3840 msgr2=0x7fc9881f69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:06.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.059+0000 7fc98ff53640 1 --2- 172.21.15.32:0/2784696275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9881f3840 0x7fc9881f69b0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fc97c001840 tx=0x7fc97c001120 comp rx=0 tx=0).stop 2024-04-24T23:03:06.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.059+0000 7fc98ff53640 1 -- 172.21.15.32:0/2784696275 shutdown_connections 2024-04-24T23:03:06.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.059+0000 7fc98ff53640 1 --2- 172.21.15.32:0/2784696275 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc97406be30 0x7fc97406e2f0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:06.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.059+0000 7fc98ff53640 1 --2- 172.21.15.32:0/2784696275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9881f3840 0x7fc9881f69b0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:06.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.059+0000 7fc98ff53640 1 --2- 172.21.15.32:0/2784696275 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9881507f0 0x7fc988150ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:06.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.059+0000 7fc98ff53640 1 --2- 172.21.15.32:0/2784696275 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc970003f60 0x7fc988140d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:06.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.059+0000 7fc98ff53640 1 -- 172.21.15.32:0/2784696275 >> 172.21.15.32:0/2784696275 conn(0x7fc98806b110 msgr2=0x7fc988072800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:06.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.059+0000 7fc98ff53640 1 -- 172.21.15.32:0/2784696275 shutdown_connections 2024-04-24T23:03:06.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.059+0000 7fc98ff53640 1 -- 172.21.15.32:0/2784696275 wait complete. 2024-04-24T23:03:06.071 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:03:06.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.170+0000 7fe7a1e41640 1 Processor -- start 2024-04-24T23:03:06.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.170+0000 7fe7a1e41640 1 -- start start 2024-04-24T23:03:06.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.171+0000 7fe7a1e41640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe79c0d4210 0x7fe79c0d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:06.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.171+0000 7fe7a1e41640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe79c0d3170 con 0x7fe79c06b820 2024-04-24T23:03:06.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.171+0000 7fe7a1e41640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe79c0d3450 con 0x7fe79c15e9b0 2024-04-24T23:03:06.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.171+0000 7fe7a1e41640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe79c0d3730 con 0x7fe79c0d4210 2024-04-24T23:03:06.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.171+0000 7fe79affd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe79c06b820 0x7fe79c0b78f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57628/0 (socket says 172.21.15.32:57628) 2024-04-24T23:03:06.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.171+0000 7fe79affd640 1 -- 172.21.15.32:0/1297888998 learned_addr learned my addr 172.21.15.32:0/1297888998 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:03:06.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.172+0000 7fe79bfff640 1 --2- 172.21.15.32:0/1297888998 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe79c0d4210 0x7fe79c0d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:06.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.172+0000 7fe79a7fc640 1 -- 172.21.15.32:0/1297888998 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3263416874 0 0) 0x7fe79c0d3170 con 0x7fe79c06b820 2024-04-24T23:03:06.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.172+0000 7fe79a7fc640 1 -- 172.21.15.32:0/1297888998 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe784003520 con 0x7fe79c06b820 2024-04-24T23:03:06.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.172+0000 7fe79a7fc640 1 -- 172.21.15.32:0/1297888998 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 779764664 0 0) 0x7fe784003520 con 0x7fe79c06b820 2024-04-24T23:03:06.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.172+0000 7fe79a7fc640 1 -- 172.21.15.32:0/1297888998 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe79c0d4210 msgr2=0x7fe79c0d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:06.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.172+0000 7fe79a7fc640 1 --2- 172.21.15.32:0/1297888998 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe79c0d4210 0x7fe79c0d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:06.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.172+0000 7fe79a7fc640 1 -- 172.21.15.32:0/1297888998 >> v1:172.21.15.121:6789/0 conn(0x7fe79c15e9b0 legacy=0x7fe79c06b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:06.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.172+0000 7fe79a7fc640 1 -- 172.21.15.32:0/1297888998 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe79c0d3a10 con 0x7fe79c06b820 2024-04-24T23:03:06.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.172+0000 7fe79a7fc640 1 -- 172.21.15.32:0/1297888998 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7fe790001670 con 0x7fe79c06b820 2024-04-24T23:03:06.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.172+0000 7fe79a7fc640 1 -- 172.21.15.32:0/1297888998 >> v1:172.21.15.32:6789/0 conn(0x7fe79c06b820 legacy=0x7fe79c0b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:06.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.172+0000 7fe79a7fc640 1 --2- 172.21.15.32:0/1297888998 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe79c0d4210 0x7fe7840039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:06.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.172+0000 7fe79a7fc640 1 --2- 172.21.15.32:0/1297888998 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe784003f20 0x7fe784004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:06.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.172+0000 7fe79a7fc640 1 --2- 172.21.15.32:0/1297888998 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe784004d40 0x7fe7840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:06.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.172+0000 7fe79a7fc640 1 -- 172.21.15.32:0/1297888998 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c0d3730 con 0x7fe784004d40 2024-04-24T23:03:06.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.172+0000 7fe79a7fc640 1 -- 172.21.15.32:0/1297888998 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe78400ba20 con 0x7fe784003f20 2024-04-24T23:03:06.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.172+0000 7fe79a7fc640 1 -- 172.21.15.32:0/1297888998 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe78400bc20 con 0x7fe79c0d4210 2024-04-24T23:03:06.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.172+0000 7fe79bfff640 1 --2- 172.21.15.32:0/1297888998 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe784004d40 0x7fe7840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:06.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.172+0000 7fe79affd640 1 --2- 172.21.15.32:0/1297888998 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe784003f20 0x7fe784004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:06.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.172+0000 7fe79b7fe640 1 --2- 172.21.15.32:0/1297888998 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe79c0d4210 0x7fe7840039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:06.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.172+0000 7fe79bfff640 1 -- 172.21.15.32:0/1297888998 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe79c0d4210 msgr2=0x7fe7840039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:06.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.172+0000 7fe79bfff640 1 --2- 172.21.15.32:0/1297888998 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe79c0d4210 0x7fe7840039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:06.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.172+0000 7fe79bfff640 1 -- 172.21.15.32:0/1297888998 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe784003f20 msgr2=0x7fe784004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:06.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.172+0000 7fe79bfff640 1 --2- 172.21.15.32:0/1297888998 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe784003f20 0x7fe784004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:06.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.172+0000 7fe79bfff640 1 -- 172.21.15.32:0/1297888998 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe784003520 con 0x7fe784004d40 2024-04-24T23:03:06.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.172+0000 7fe79b7fe640 1 --2- 172.21.15.32:0/1297888998 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe79c0d4210 0x7fe7840039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:03:06.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.172+0000 7fe79affd640 1 --2- 172.21.15.32:0/1297888998 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe784003f20 0x7fe784004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:03:06.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.173+0000 7fe79bfff640 1 --2- 172.21.15.32:0/1297888998 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe784004d40 0x7fe7840051c0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fe794009f90 tx=0x7fe784005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f276195d8521f85e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:06.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.173+0000 7fe79a7fc640 1 -- 172.21.15.32:0/1297888998 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe794009c10 con 0x7fe784004d40 2024-04-24T23:03:06.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.173+0000 7fe79a7fc640 1 -- 172.21.15.32:0/1297888998 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe794036070 con 0x7fe784004d40 2024-04-24T23:03:06.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.173+0000 7fe79a7fc640 1 -- 172.21.15.32:0/1297888998 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe794031810 con 0x7fe784004d40 2024-04-24T23:03:06.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.174+0000 7fe7a1e41640 1 -- 172.21.15.32:0/1297888998 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe784004d40 msgr2=0x7fe7840051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:06.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.174+0000 7fe7a1e41640 1 --2- 172.21.15.32:0/1297888998 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe784004d40 0x7fe7840051c0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fe794009f90 tx=0x7fe784005830 comp rx=0 tx=0).stop 2024-04-24T23:03:06.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.174+0000 7fe79bfff640 1 -- 172.21.15.32:0/1297888998 reap_dead start 2024-04-24T23:03:06.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.174+0000 7fe7a1e41640 1 -- 172.21.15.32:0/1297888998 shutdown_connections 2024-04-24T23:03:06.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.174+0000 7fe7a1e41640 1 -- 172.21.15.32:0/1297888998 >> 172.21.15.32:0/1297888998 conn(0x7fe79c0d11d0 msgr2=0x7fe79c0d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:06.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.174+0000 7fe7a1e41640 1 -- 172.21.15.32:0/1297888998 shutdown_connections 2024-04-24T23:03:06.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.174+0000 7fe7a1e41640 1 -- 172.21.15.32:0/1297888998 wait complete. 2024-04-24T23:03:06.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.175+0000 7fe7a1e41640 1 Processor -- start 2024-04-24T23:03:06.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.175+0000 7fe7a1e41640 1 -- start start 2024-04-24T23:03:06.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.176+0000 7fe7a1e41640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe784004d40 0x7fe79c153ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:06.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.176+0000 7fe7a1e41640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe79c0cfa50 0x7fe79c0cfed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:06.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.176+0000 7fe7a1e41640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe79c0cd7b0 0x7fe79c0cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:06.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.176+0000 7fe7a1e41640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c0ba0e0 con 0x7fe79c0cd7b0 2024-04-24T23:03:06.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.176+0000 7fe7a1e41640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c0d2400 con 0x7fe784004d40 2024-04-24T23:03:06.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.176+0000 7fe7a1e41640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe79c0ba260 con 0x7fe79c0cfa50 2024-04-24T23:03:06.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.176+0000 7fe79bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe79c0cd7b0 0x7fe79c0cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:06.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.176+0000 7fe79bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe79c0cd7b0 0x7fe79c0cdc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38502/0 (socket says 172.21.15.32:38502) 2024-04-24T23:03:06.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.176+0000 7fe79bfff640 1 -- 172.21.15.32:0/2519210939 learned_addr learned my addr 172.21.15.32:0/2519210939 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:06.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.176+0000 7fe79b7fe640 1 --2- 172.21.15.32:0/2519210939 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe784004d40 0x7fe79c153ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:06.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.176+0000 7fe79affd640 1 --2- 172.21.15.32:0/2519210939 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe79c0cfa50 0x7fe79c0cfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:06.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.176+0000 7fe79bfff640 1 -- 172.21.15.32:0/2519210939 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe79c0cfa50 msgr2=0x7fe79c0cfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:06.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.176+0000 7fe79bfff640 1 --2- 172.21.15.32:0/2519210939 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe79c0cfa50 0x7fe79c0cfed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:06.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.176+0000 7fe79bfff640 1 -- 172.21.15.32:0/2519210939 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe784004d40 msgr2=0x7fe79c153ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:06.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.176+0000 7fe79bfff640 1 --2- 172.21.15.32:0/2519210939 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe784004d40 0x7fe79c153ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:06.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.176+0000 7fe79bfff640 1 -- 172.21.15.32:0/2519210939 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe79c0cb9a0 con 0x7fe79c0cd7b0 2024-04-24T23:03:06.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.176+0000 7fe79bfff640 1 --2- 172.21.15.32:0/2519210939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe79c0cd7b0 0x7fe79c0cdc30 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fe794033040 tx=0x7fe794009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:06.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.177+0000 7fe783fff640 1 -- 172.21.15.32:0/2519210939 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe794009c10 con 0x7fe79c0cd7b0 2024-04-24T23:03:06.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.177+0000 7fe783fff640 1 -- 172.21.15.32:0/2519210939 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe794045430 con 0x7fe79c0cd7b0 2024-04-24T23:03:06.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.177+0000 7fe7a1e41640 1 -- 172.21.15.32:0/2519210939 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe79c0cbc80 con 0x7fe79c0cd7b0 2024-04-24T23:03:06.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.177+0000 7fe783fff640 1 -- 172.21.15.32:0/2519210939 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe794045820 con 0x7fe79c0cd7b0 2024-04-24T23:03:06.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.177+0000 7fe7a1e41640 1 -- 172.21.15.32:0/2519210939 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe79c0cc760 con 0x7fe79c0cd7b0 2024-04-24T23:03:06.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.178+0000 7fe7a1e41640 1 -- 172.21.15.32:0/2519210939 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe79c068430 con 0x7fe79c0cd7b0 2024-04-24T23:03:06.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.178+0000 7fe783fff640 1 -- 172.21.15.32:0/2519210939 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe794005d10 con 0x7fe79c0cd7b0 2024-04-24T23:03:06.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.179+0000 7fe783fff640 1 --2- 172.21.15.32:0/2519210939 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe78806be30 0x7fe78806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:06.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.179+0000 7fe783fff640 1 -- 172.21.15.32:0/2519210939 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7fe7940abfc0 con 0x7fe79c0cd7b0 2024-04-24T23:03:06.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.179+0000 7fe79b7fe640 1 --2- 172.21.15.32:0/2519210939 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe78806be30 0x7fe78806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:06.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.179+0000 7fe79b7fe640 1 --2- 172.21.15.32:0/2519210939 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe78806be30 0x7fe78806e2f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe78c0018f0 tx=0x7fe78c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:06.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.183+0000 7fe783fff640 1 -- 172.21.15.32:0/2519210939 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe79402e050 con 0x7fe79c0cd7b0 2024-04-24T23:03:06.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.403+0000 7fe7a1e41640 1 -- 172.21.15.32:0/2519210939 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe79c072bf0 con 0x7fe79c0cd7b0 2024-04-24T23:03:06.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.404+0000 7fe783fff640 1 -- 172.21.15.32:0/2519210939 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5186 (secure 0 0 0) 0x7fe79407a320 con 0x7fe79c0cd7b0 2024-04-24T23:03:06.405 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:03:06.405 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":36,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5486,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/3550206194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3550206194},{"type":"v1","addr":"172.21.15.32:6834","nonce":3550206194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5905,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1706188546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1706188546},{"type":"v1","addr":"172.21.15.121:6835","nonce":1706188546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5911,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2669443498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2669443498},{"type":"v1","addr":"172.21.15.121:6837","nonce":2669443498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:03:00.630938+0000","modified":"2024-04-24T23:03:03.639570+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6474":{"gid":6474,"name":"a","rank":0,"incarnation":33,"state":"up:active","state_seq":4,"addr":"172.21.15.32:6835/464246210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":464246210},{"type":"v1","addr":"172.21.15.32:6835","nonce":464246210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6474,"qdb_cluster":[6474]},"id":5}]} 2024-04-24T23:03:06.406 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 36 2024-04-24T23:03:06.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.406+0000 7fe7a1e41640 1 -- 172.21.15.32:0/2519210939 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe78806be30 msgr2=0x7fe78806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:06.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.406+0000 7fe7a1e41640 1 --2- 172.21.15.32:0/2519210939 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe78806be30 0x7fe78806e2f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe78c0018f0 tx=0x7fe78c009040 comp rx=0 tx=0).stop 2024-04-24T23:03:06.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.406+0000 7fe7a1e41640 1 -- 172.21.15.32:0/2519210939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe79c0cd7b0 msgr2=0x7fe79c0cdc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:06.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.406+0000 7fe7a1e41640 1 --2- 172.21.15.32:0/2519210939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe79c0cd7b0 0x7fe79c0cdc30 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fe794033040 tx=0x7fe794009750 comp rx=0 tx=0).stop 2024-04-24T23:03:06.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.406+0000 7fe7a1e41640 1 -- 172.21.15.32:0/2519210939 shutdown_connections 2024-04-24T23:03:06.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.406+0000 7fe7a1e41640 1 --2- 172.21.15.32:0/2519210939 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe78806be30 0x7fe78806e2f0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:06.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.406+0000 7fe7a1e41640 1 --2- 172.21.15.32:0/2519210939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe79c0cd7b0 0x7fe79c0cdc30 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:06.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.406+0000 7fe7a1e41640 1 --2- 172.21.15.32:0/2519210939 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe79c0cfa50 0x7fe79c0cfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:06.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.406+0000 7fe7a1e41640 1 --2- 172.21.15.32:0/2519210939 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe784004d40 0x7fe79c153ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:06.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.406+0000 7fe7a1e41640 1 -- 172.21.15.32:0/2519210939 >> 172.21.15.32:0/2519210939 conn(0x7fe79c0d11d0 msgr2=0x7fe79c0b78f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:06.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.406+0000 7fe7a1e41640 1 -- 172.21.15.32:0/2519210939 shutdown_connections 2024-04-24T23:03:06.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.406+0000 7fe7a1e41640 1 -- 172.21.15.32:0/2519210939 wait complete. 2024-04-24T23:03:06.535 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:03:06.538 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:03:06.821 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 36, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T23:03:00.630938+0000', 'modified': '2024-04-24T23:03:03.639570+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6474}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6474': {'gid': 6474, 'name': 'a', 'rank': 0, 'incarnation': 33, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.32:6835/464246210', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6833', 'nonce': 464246210}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 464246210}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6474, 'qdb_cluster': [6474]} 2024-04-24T23:03:06.822 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-24T23:03:06.822 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-24T23:03:06.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.923+0000 7fa02888c640 1 Processor -- start 2024-04-24T23:03:06.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.923+0000 7fa02888c640 1 -- start start 2024-04-24T23:03:06.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.923+0000 7fa02888c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa02415e9b0 0x7fa0240d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:06.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.923+0000 7fa02888c640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa02406b820 0x7fa0240b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:06.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.923+0000 7fa02888c640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0240d3170 con 0x7fa0240d4210 2024-04-24T23:03:06.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.923+0000 7fa02888c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fa0240d3450 con 0x7fa02415e9b0 2024-04-24T23:03:06.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.923+0000 7fa02888c640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fa0240d3730 con 0x7fa02406b820 2024-04-24T23:03:06.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.924+0000 7fa02259c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa02415e9b0 0x7fa0240d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:06.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.924+0000 7fa01bfff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa0240d4210 0x7fa02406b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57642/0 (socket says 172.21.15.32:57642) 2024-04-24T23:03:06.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.924+0000 7fa02259c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa02415e9b0 0x7fa0240d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38516/0 (socket says 172.21.15.32:38516) 2024-04-24T23:03:06.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.924+0000 7fa01bfff640 1 -- 172.21.15.32:0/1468096954 learned_addr learned my addr 172.21.15.32:0/1468096954 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:03:06.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.924+0000 7fa022d9d640 1 --2- 172.21.15.32:0/1468096954 >> v2:172.21.15.121:3300/0 conn(0x7fa02406b820 0x7fa0240b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:06.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.924+0000 7fa02259c640 1 -- 172.21.15.32:0/1468096954 >> v2:172.21.15.121:3300/0 conn(0x7fa02406b820 msgr2=0x7fa0240b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:06.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.924+0000 7fa02259c640 1 --2- 172.21.15.32:0/1468096954 >> v2:172.21.15.121:3300/0 conn(0x7fa02406b820 0x7fa0240b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:06.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.924+0000 7fa02259c640 1 -- 172.21.15.32:0/1468096954 >> v1:172.21.15.32:6789/0 conn(0x7fa0240d4210 legacy=0x7fa02406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:06.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.924+0000 7fa02259c640 1 -- 172.21.15.32:0/1468096954 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0240d3a10 con 0x7fa02415e9b0 2024-04-24T23:03:06.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.924+0000 7fa022d9d640 1 --2- 172.21.15.32:0/1468096954 >> v2:172.21.15.121:3300/0 conn(0x7fa02406b820 0x7fa0240b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:03:06.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.925+0000 7fa02259c640 1 --2- 172.21.15.32:0/1468096954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa02415e9b0 0x7fa0240d2b90 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fa0241622f0 tx=0x7fa01402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21e30a7cf9ffa531 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:06.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.925+0000 7fa021d9b640 1 -- 172.21.15.32:0/1468096954 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa014037070 con 0x7fa02415e9b0 2024-04-24T23:03:06.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.925+0000 7fa021d9b640 1 -- 172.21.15.32:0/1468096954 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa014032440 con 0x7fa02415e9b0 2024-04-24T23:03:06.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.925+0000 7fa021d9b640 1 -- 172.21.15.32:0/1468096954 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa01402ae60 con 0x7fa02415e9b0 2024-04-24T23:03:06.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.925+0000 7fa02888c640 1 -- 172.21.15.32:0/1468096954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa02415e9b0 msgr2=0x7fa0240d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:06.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.925+0000 7fa02888c640 1 --2- 172.21.15.32:0/1468096954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa02415e9b0 0x7fa0240d2b90 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fa0241622f0 tx=0x7fa01402a110 comp rx=0 tx=0).stop 2024-04-24T23:03:06.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.925+0000 7fa02888c640 1 -- 172.21.15.32:0/1468096954 shutdown_connections 2024-04-24T23:03:06.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.925+0000 7fa02888c640 1 --2- 172.21.15.32:0/1468096954 >> v2:172.21.15.121:3300/0 conn(0x7fa02406b820 0x7fa0240b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:06.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.925+0000 7fa02888c640 1 --2- 172.21.15.32:0/1468096954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa02415e9b0 0x7fa0240d2b90 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:06.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.925+0000 7fa02888c640 1 -- 172.21.15.32:0/1468096954 >> 172.21.15.32:0/1468096954 conn(0x7fa0240d11d0 msgr2=0x7fa0240d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:06.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.925+0000 7fa02888c640 1 -- 172.21.15.32:0/1468096954 shutdown_connections 2024-04-24T23:03:06.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.925+0000 7fa02888c640 1 -- 172.21.15.32:0/1468096954 wait complete. 2024-04-24T23:03:06.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.927+0000 7fa02888c640 1 Processor -- start 2024-04-24T23:03:06.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.927+0000 7fa02888c640 1 -- start start 2024-04-24T23:03:06.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.927+0000 7fa02888c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa02406b820 0x7fa0241564b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:06.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.927+0000 7fa02888c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0240d4210 0x7fa02414b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:06.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.927+0000 7fa02888c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa02415e9b0 0x7fa0241f44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:06.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.927+0000 7fa02888c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa0240ba0e0 con 0x7fa02415e9b0 2024-04-24T23:03:06.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.927+0000 7fa02888c640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa0240d2400 con 0x7fa0240d4210 2024-04-24T23:03:06.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.927+0000 7fa02888c640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa0240ba260 con 0x7fa02406b820 2024-04-24T23:03:06.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.928+0000 7fa022d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa02415e9b0 0x7fa0241f44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:06.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.928+0000 7fa022d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa02415e9b0 0x7fa0241f44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38528/0 (socket says 172.21.15.32:38528) 2024-04-24T23:03:06.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.928+0000 7fa022d9d640 1 -- 172.21.15.32:0/874208265 learned_addr learned my addr 172.21.15.32:0/874208265 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:06.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.928+0000 7fa01bfff640 1 --2- 172.21.15.32:0/874208265 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0240d4210 0x7fa02414b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:06.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.928+0000 7fa02259c640 1 --2- 172.21.15.32:0/874208265 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa02406b820 0x7fa0241564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:06.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.928+0000 7fa022d9d640 1 -- 172.21.15.32:0/874208265 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa02406b820 msgr2=0x7fa0241564b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:06.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.928+0000 7fa022d9d640 1 --2- 172.21.15.32:0/874208265 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa02406b820 0x7fa0241564b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:06.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.928+0000 7fa022d9d640 1 -- 172.21.15.32:0/874208265 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0240d4210 msgr2=0x7fa02414b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:06.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.928+0000 7fa022d9d640 1 --2- 172.21.15.32:0/874208265 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0240d4210 0x7fa02414b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:06.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.928+0000 7fa022d9d640 1 -- 172.21.15.32:0/874208265 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0240cc950 con 0x7fa02415e9b0 2024-04-24T23:03:06.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.928+0000 7fa022d9d640 1 --2- 172.21.15.32:0/874208265 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa02415e9b0 0x7fa0241f44e0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fa01c00cef0 tx=0x7fa01c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:06.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.928+0000 7fa01affd640 1 -- 172.21.15.32:0/874208265 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa01c01a070 con 0x7fa02415e9b0 2024-04-24T23:03:06.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.928+0000 7fa01affd640 1 -- 172.21.15.32:0/874208265 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa01c00d860 con 0x7fa02415e9b0 2024-04-24T23:03:06.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.928+0000 7fa02888c640 1 -- 172.21.15.32:0/874208265 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0240ccc30 con 0x7fa02415e9b0 2024-04-24T23:03:06.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.928+0000 7fa01affd640 1 -- 172.21.15.32:0/874208265 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa01c00dc50 con 0x7fa02415e9b0 2024-04-24T23:03:06.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.928+0000 7fa02888c640 1 -- 172.21.15.32:0/874208265 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0240cd1f0 con 0x7fa02415e9b0 2024-04-24T23:03:06.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.930+0000 7fa01affd640 1 -- 172.21.15.32:0/874208265 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa01c01d070 con 0x7fa02415e9b0 2024-04-24T23:03:06.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.930+0000 7fa01affd640 1 --2- 172.21.15.32:0/874208265 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa00406be30 0x7fa00406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:06.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.930+0000 7fa01affd640 1 -- 172.21.15.32:0/874208265 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7fa01c08b5e0 con 0x7fa02415e9b0 2024-04-24T23:03:06.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.931+0000 7fa02259c640 1 --2- 172.21.15.32:0/874208265 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa00406be30 0x7fa00406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:06.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.931+0000 7fa02b59a740 1 Processor -- start 2024-04-24T23:03:06.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.931+0000 7fa02259c640 1 --2- 172.21.15.32:0/874208265 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa00406be30 0x7fa00406e2f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa014034000 tx=0x7fa01402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:06.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.931+0000 7fa02b59a740 1 -- start start 2024-04-24T23:03:06.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.932+0000 7fa02b59a740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564a8200bb60 0x564a81fbe4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:06.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.932+0000 7fa02b59a740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564a81feec80 0x564a81fee000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:06.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.932+0000 7fa02b59a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a81fee540 0x564a81fc3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:06.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.932+0000 7fa02b59a740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x564a81f26f00 con 0x564a8200bb60 2024-04-24T23:03:06.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.932+0000 7fa02b59a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x564a81ecf750 con 0x564a81fee540 2024-04-24T23:03:06.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.932+0000 7fa02b59a740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x564a81f22ec0 con 0x564a81feec80 2024-04-24T23:03:06.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.932+0000 7fa02259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564a8200bb60 0x564a81fbe4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:06.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.932+0000 7fa02259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564a8200bb60 0x564a81fbe4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38534/0 (socket says 172.21.15.32:38534) 2024-04-24T23:03:06.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.932+0000 7fa02259c640 1 -- 172.21.15.32:0/1407815699 learned_addr learned my addr 172.21.15.32:0/1407815699 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:06.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.932+0000 7fa022d9d640 1 --2- 172.21.15.32:0/1407815699 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a81fee540 0x564a81fc3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:06.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.932+0000 7fa01bfff640 1 --2- 172.21.15.32:0/1407815699 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564a81feec80 0x564a81fee000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:06.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.932+0000 7fa02259c640 1 -- 172.21.15.32:0/1407815699 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564a81feec80 msgr2=0x564a81fee000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:06.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.932+0000 7fa02259c640 1 --2- 172.21.15.32:0/1407815699 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564a81feec80 0x564a81fee000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:06.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.932+0000 7fa02259c640 1 -- 172.21.15.32:0/1407815699 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a81fee540 msgr2=0x564a81fc3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:06.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.932+0000 7fa02259c640 1 --2- 172.21.15.32:0/1407815699 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a81fee540 0x564a81fc3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:06.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.932+0000 7fa02259c640 1 -- 172.21.15.32:0/1407815699 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a81c3b2d0 con 0x564a8200bb60 2024-04-24T23:03:06.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.933+0000 7fa02259c640 1 --2- 172.21.15.32:0/1407815699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564a8200bb60 0x564a81fbe4f0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fa014009860 tx=0x7fa01402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edf7cd8897d20930 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:06.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.933+0000 7fa003fff640 1 -- 172.21.15.32:0/1407815699 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa014037040 con 0x564a8200bb60 2024-04-24T23:03:06.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.933+0000 7fa003fff640 1 -- 172.21.15.32:0/1407815699 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa01404e070 con 0x564a8200bb60 2024-04-24T23:03:06.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.933+0000 7fa003fff640 1 -- 172.21.15.32:0/1407815699 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa01403c440 con 0x564a8200bb60 2024-04-24T23:03:06.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.933+0000 7fa02b59a740 1 -- 172.21.15.32:0/1407815699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564a8200bb60 msgr2=0x564a81fbe4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:06.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.933+0000 7fa02b59a740 1 --2- 172.21.15.32:0/1407815699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564a8200bb60 0x564a81fbe4f0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fa014009860 tx=0x7fa01402c000 comp rx=0 tx=0).stop 2024-04-24T23:03:06.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.934+0000 7fa02b59a740 1 -- 172.21.15.32:0/1407815699 shutdown_connections 2024-04-24T23:03:06.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.934+0000 7fa02b59a740 1 --2- 172.21.15.32:0/1407815699 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a81fee540 0x564a81fc3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:06.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.934+0000 7fa02b59a740 1 --2- 172.21.15.32:0/1407815699 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564a81feec80 0x564a81fee000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:06.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.934+0000 7fa02b59a740 1 --2- 172.21.15.32:0/1407815699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564a8200bb60 0x564a81fbe4f0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:06.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.934+0000 7fa02b59a740 1 -- 172.21.15.32:0/1407815699 >> 172.21.15.32:0/1407815699 conn(0x564a81fc0760 msgr2=0x564a82009000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:06.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.934+0000 7fa02b59a740 1 -- 172.21.15.32:0/1407815699 shutdown_connections 2024-04-24T23:03:06.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.934+0000 7fa02b59a740 1 -- 172.21.15.32:0/1407815699 wait complete. 2024-04-24T23:03:06.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.934+0000 7fa02b59a740 1 -- start start 2024-04-24T23:03:06.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.934+0000 7fa02b59a740 1 Processor -- start 2024-04-24T23:03:06.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.935+0000 7fa02b59a740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564a81feec80 0x564a81ff0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:06.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.935+0000 7fa02b59a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a8200bb60 0x564a820017a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:06.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.935+0000 7fa02b59a740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564a81fe99b0 0x564a82018d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:06.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.935+0000 7fa02b59a740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x564a81fc2910 con 0x564a81feec80 2024-04-24T23:03:06.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.935+0000 7fa02b59a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x564a81ee2cf0 con 0x564a8200bb60 2024-04-24T23:03:06.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.935+0000 7fa02b59a740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x564a81fc1d00 con 0x564a81fe99b0 2024-04-24T23:03:06.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.935+0000 7fa02b59a740 10 client.0 _ll_register_callbacks cb 0x564a81ee26e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:03:06.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.935+0000 7fa02259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564a81feec80 0x564a81ff0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:06.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.935+0000 7fa02259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564a81feec80 0x564a81ff0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38540/0 (socket says 172.21.15.32:38540) 2024-04-24T23:03:06.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.935+0000 7fa02259c640 1 -- 172.21.15.32:0/72312263 learned_addr learned my addr 172.21.15.32:0/72312263 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:06.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.935+0000 7fa022d9d640 1 --2- 172.21.15.32:0/72312263 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564a81fe99b0 0x564a82018d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:06.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.935+0000 7fa01bfff640 1 --2- 172.21.15.32:0/72312263 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a8200bb60 0x564a820017a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:06.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.936+0000 7fa02259c640 1 -- 172.21.15.32:0/72312263 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564a81fe99b0 msgr2=0x564a82018d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:06.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.936+0000 7fa02259c640 1 --2- 172.21.15.32:0/72312263 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564a81fe99b0 0x564a82018d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:06.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.936+0000 7fa02259c640 1 -- 172.21.15.32:0/72312263 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a8200bb60 msgr2=0x564a820017a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:06.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.936+0000 7fa02259c640 1 --2- 172.21.15.32:0/72312263 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a8200bb60 0x564a820017a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:06.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.936+0000 7fa02259c640 1 -- 172.21.15.32:0/72312263 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a81d056b0 con 0x564a81feec80 2024-04-24T23:03:06.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.936+0000 7fa02259c640 1 --2- 172.21.15.32:0/72312263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564a81feec80 0x564a81ff0b70 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fa014009560 tx=0x7fa014004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87aa3ee5d567f58a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:06.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.936+0000 7fa0017fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-24T23:03:06.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.936+0000 7fa0017fa640 1 -- 172.21.15.32:0/72312263 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa014058070 con 0x564a81feec80 2024-04-24T23:03:06.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.936+0000 7fa0017fa640 1 -- 172.21.15.32:0/72312263 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7fa01403ca80 con 0x564a81feec80 2024-04-24T23:03:06.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.936+0000 7fa02b59a740 1 -- 172.21.15.32:0/72312263 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa01404e070 con 0x564a81feec80 2024-04-24T23:03:06.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.936+0000 7fa0017fa640 1 -- 172.21.15.32:0/72312263 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa014062050 con 0x564a81feec80 2024-04-24T23:03:06.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.936+0000 7fa02b59a740 10 client.6618 fetch_fsmap learned FSMap version 36 2024-04-24T23:03:06.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.936+0000 7fa02b59a740 1 -- 172.21.15.32:0/72312263 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x564a82001280 con 0x564a81feec80 2024-04-24T23:03:06.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.936+0000 7fa0017fa640 1 -- 172.21.15.32:0/72312263 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa01404e070 con 0x564a81feec80 2024-04-24T23:03:06.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.936+0000 7fa0017fa640 1 -- 172.21.15.32:0/72312263 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 36) ==== 3494+0+0 (secure 0 0 0) 0x7fa014065020 con 0x564a81feec80 2024-04-24T23:03:06.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.936+0000 7fa02b59a740 10 client.6618 fetch_fsmap finished waiting for FSMap version 36 2024-04-24T23:03:06.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.936+0000 7fa02b59a740 10 client.6618 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T23:03:06.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.936+0000 7fa02b59a740 20 client.6618 populate_metadata read hostname 'smithi032' 2024-04-24T23:03:06.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.936+0000 7fa02b59a740 1 --2- 172.21.15.32:0/72312263 >> [v2:172.21.15.32:6833/464246210,v1:172.21.15.32:6835/464246210] conn(0x564a81fef530 0x564a81fef910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:06.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.936+0000 7fa02b59a740 4 client.6618 mds_command: new command op to 6474 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:03:06.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.937+0000 7fa02b59a740 1 -- 172.21.15.32:0/72312263 --> [v2:172.21.15.32:6833/464246210,v1:172.21.15.32:6835/464246210] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a81f34440 con 0x564a81fef530 2024-04-24T23:03:06.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.937+0000 7fa01bfff640 1 --2- 172.21.15.32:0/72312263 >> [v2:172.21.15.32:6833/464246210,v1:172.21.15.32:6835/464246210] conn(0x564a81fef530 0x564a81fef910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:06.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.937+0000 7fa0017fa640 10 client.6618 ms_handle_connect on v2:172.21.15.32:6833/464246210 2024-04-24T23:03:06.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.937+0000 7fa01bfff640 1 --2- 172.21.15.32:0/72312263 >> [v2:172.21.15.32:6833/464246210,v1:172.21.15.32:6835/464246210] conn(0x564a81fef530 0x564a81fef910 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3d3f2e2f0922adc server_cookie=3769dc3f304ee4ff in_seq=0 out_seq=0 2024-04-24T23:03:06.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.938+0000 7fa0017fa640 1 -- 172.21.15.32:0/72312263 <== mds.0 v2:172.21.15.32:6833/464246210 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564a81f34440 con 0x564a81fef530 2024-04-24T23:03:06.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.938+0000 7fa0017fa640 10 client.6618 handle_command_reply: tid=0 2024-04-24T23:03:06.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.938+0000 7fa02b59a740 1 client.6618 shutdown 2024-04-24T23:03:06.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.938+0000 7fa02b59a740 1 -- 172.21.15.32:0/72312263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564a81feec80 msgr2=0x564a81ff0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:06.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.938+0000 7fa02b59a740 1 --2- 172.21.15.32:0/72312263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564a81feec80 0x564a81ff0b70 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fa014009560 tx=0x7fa014004730 comp rx=0 tx=0).stop 2024-04-24T23:03:06.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.938+0000 7fa02b59a740 1 -- 172.21.15.32:0/72312263 shutdown_connections 2024-04-24T23:03:06.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.938+0000 7fa02b59a740 1 --2- 172.21.15.32:0/72312263 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564a81fe99b0 0x564a82018d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:06.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.938+0000 7fa02b59a740 1 --2- 172.21.15.32:0/72312263 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a8200bb60 0x564a820017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:06.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.938+0000 7fa02b59a740 1 --2- 172.21.15.32:0/72312263 >> [v2:172.21.15.32:6833/464246210,v1:172.21.15.32:6835/464246210] conn(0x564a81fef530 0x564a81fef910 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:06.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.938+0000 7fa02b59a740 1 --2- 172.21.15.32:0/72312263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564a81feec80 0x564a81ff0b70 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:06.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.938+0000 7fa02b59a740 1 -- 172.21.15.32:0/72312263 >> 172.21.15.32:0/72312263 conn(0x564a81fee540 msgr2=0x564a81fbf230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:06.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.938+0000 7fa0017fa640 0 client.6618 ms_handle_reset on v2:172.21.15.32:6833/464246210 2024-04-24T23:03:06.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.938+0000 7fa02b59a740 1 -- 172.21.15.32:0/72312263 shutdown_connections 2024-04-24T23:03:06.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.938+0000 7fa02b59a740 1 -- 172.21.15.32:0/72312263 wait complete. 2024-04-24T23:03:06.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.938+0000 7fa02b59a740 20 client.6618 trim_cache size 0 max 16384 2024-04-24T23:03:06.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.956+0000 7fa02b59a740 1 Processor -- start 2024-04-24T23:03:06.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.956+0000 7fa02b59a740 1 -- start start 2024-04-24T23:03:06.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.957+0000 7fa02b59a740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564a81fe99b0 0x564a81fc2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:06.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.957+0000 7fa02b59a740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564a81feec80 0x564a81fddd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:06.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.957+0000 7fa02b59a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a8200bb60 0x564a81fbf230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:06.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.957+0000 7fa02b59a740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa0241ffcd0 con 0x564a81fe99b0 2024-04-24T23:03:06.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.957+0000 7fa02b59a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa024160bf0 con 0x564a8200bb60 2024-04-24T23:03:06.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.957+0000 7fa02b59a740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa024161240 con 0x564a81feec80 2024-04-24T23:03:06.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.957+0000 7fa02259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564a81fe99b0 0x564a81fc2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:06.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.957+0000 7fa02259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564a81fe99b0 0x564a81fc2020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38554/0 (socket says 172.21.15.32:38554) 2024-04-24T23:03:06.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.957+0000 7fa02259c640 1 -- 172.21.15.32:0/4256997823 learned_addr learned my addr 172.21.15.32:0/4256997823 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:06.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.957+0000 7fa022d9d640 1 --2- 172.21.15.32:0/4256997823 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a8200bb60 0x564a81fbf230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:06.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.957+0000 7fa01bfff640 1 --2- 172.21.15.32:0/4256997823 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564a81feec80 0x564a81fddd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:06.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.957+0000 7fa02259c640 1 -- 172.21.15.32:0/4256997823 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564a81feec80 msgr2=0x564a81fddd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:06.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.957+0000 7fa02259c640 1 --2- 172.21.15.32:0/4256997823 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564a81feec80 0x564a81fddd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:06.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.957+0000 7fa02259c640 1 -- 172.21.15.32:0/4256997823 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a8200bb60 msgr2=0x564a81fbf230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:06.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.957+0000 7fa02259c640 1 --2- 172.21.15.32:0/4256997823 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a8200bb60 0x564a81fbf230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:06.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.957+0000 7fa02259c640 1 -- 172.21.15.32:0/4256997823 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a81f31a60 con 0x564a81fe99b0 2024-04-24T23:03:06.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.957+0000 7fa01bfff640 1 --2- 172.21.15.32:0/4256997823 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564a81feec80 0x564a81fddd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:03:06.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.957+0000 7fa022d9d640 1 --2- 172.21.15.32:0/4256997823 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a8200bb60 0x564a81fbf230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:03:06.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.958+0000 7fa02259c640 1 --2- 172.21.15.32:0/4256997823 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564a81fe99b0 0x564a81fc2020 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fa01403ca90 tx=0x7fa014051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49be11a593fa1c94 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:06.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.958+0000 7fa0037fe640 1 -- 172.21.15.32:0/4256997823 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa014058070 con 0x564a81fe99b0 2024-04-24T23:03:06.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.958+0000 7fa0037fe640 1 -- 172.21.15.32:0/4256997823 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa01404e040 con 0x564a81fe99b0 2024-04-24T23:03:06.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.958+0000 7fa0037fe640 1 -- 172.21.15.32:0/4256997823 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa014037040 con 0x564a81fe99b0 2024-04-24T23:03:06.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.958+0000 7fa02b59a740 1 -- 172.21.15.32:0/4256997823 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564a81fe99b0 msgr2=0x564a81fc2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:06.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.958+0000 7fa02b59a740 1 --2- 172.21.15.32:0/4256997823 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564a81fe99b0 0x564a81fc2020 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fa01403ca90 tx=0x7fa014051000 comp rx=0 tx=0).stop 2024-04-24T23:03:06.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.958+0000 7fa02b59a740 1 -- 172.21.15.32:0/4256997823 shutdown_connections 2024-04-24T23:03:06.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.958+0000 7fa02b59a740 1 --2- 172.21.15.32:0/4256997823 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a8200bb60 0x564a81fbf230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:06.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.958+0000 7fa02b59a740 1 --2- 172.21.15.32:0/4256997823 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564a81feec80 0x564a81fddd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:06.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.958+0000 7fa02b59a740 1 --2- 172.21.15.32:0/4256997823 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564a81fe99b0 0x564a81fc2020 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:06.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.958+0000 7fa02b59a740 1 -- 172.21.15.32:0/4256997823 >> 172.21.15.32:0/4256997823 conn(0x564a81fee540 msgr2=0x564a81feb4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:06.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.958+0000 7fa02b59a740 1 -- 172.21.15.32:0/4256997823 shutdown_connections 2024-04-24T23:03:06.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.958+0000 7fa02b59a740 1 -- 172.21.15.32:0/4256997823 wait complete. 2024-04-24T23:03:06.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.959+0000 7fa02b59a740 1 -- start start 2024-04-24T23:03:06.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.959+0000 7fa02b59a740 1 Processor -- start 2024-04-24T23:03:06.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.960+0000 7fa02b59a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a81fe99b0 0x564a82082900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:06.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.960+0000 7fa02b59a740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564a81feec80 0x564a82082e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:06.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.960+0000 7fa02b59a740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564a8200bb60 0x564a81fbf230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:06.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.960+0000 7fa02b59a740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x564a82019420 con 0x564a81feec80 2024-04-24T23:03:06.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.960+0000 7fa02b59a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x564a82019820 con 0x564a81fe99b0 2024-04-24T23:03:06.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.960+0000 7fa02b59a740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x564a8200c600 con 0x564a8200bb60 2024-04-24T23:03:06.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.960+0000 7fa02b59a740 10 client.0 _ll_register_callbacks cb 0x7fa01403c440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:03:06.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.960+0000 7fa01bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564a81feec80 0x564a82082e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:06.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.960+0000 7fa01bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564a81feec80 0x564a82082e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38558/0 (socket says 172.21.15.32:38558) 2024-04-24T23:03:06.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.960+0000 7fa01bfff640 1 -- 172.21.15.32:0/2786124234 learned_addr learned my addr 172.21.15.32:0/2786124234 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:06.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.960+0000 7fa01bfff640 1 -- 172.21.15.32:0/2786124234 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564a8200bb60 msgr2=0x564a81fbf230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:06.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.960+0000 7fa01bfff640 1 --2- 172.21.15.32:0/2786124234 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564a8200bb60 0x564a81fbf230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:06.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.960+0000 7fa01bfff640 1 -- 172.21.15.32:0/2786124234 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a81fe99b0 msgr2=0x564a82082900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:06.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.960+0000 7fa01bfff640 1 --2- 172.21.15.32:0/2786124234 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a81fe99b0 0x564a82082900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:06.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.960+0000 7fa01bfff640 1 -- 172.21.15.32:0/2786124234 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a82022ba0 con 0x564a81feec80 2024-04-24T23:03:06.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.960+0000 7fa01bfff640 1 --2- 172.21.15.32:0/2786124234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564a81feec80 0x564a82082e40 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fa02415c060 tx=0x7fa00c007e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f462e75d4112806 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:06.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.960+0000 7fa0017fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-24T23:03:06.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.961+0000 7fa0017fa640 1 -- 172.21.15.32:0/2786124234 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa00c025450 con 0x564a81feec80 2024-04-24T23:03:06.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.961+0000 7fa0017fa640 1 -- 172.21.15.32:0/2786124234 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7fa00c0255f0 con 0x564a81feec80 2024-04-24T23:03:06.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.961+0000 7fa02b59a740 1 -- 172.21.15.32:0/2786124234 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa01404e040 con 0x564a81feec80 2024-04-24T23:03:06.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.961+0000 7fa0017fa640 1 -- 172.21.15.32:0/2786124234 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa00c02d050 con 0x564a81feec80 2024-04-24T23:03:06.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.961+0000 7fa02b59a740 10 client.6624 fetch_fsmap learned FSMap version 36 2024-04-24T23:03:06.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.961+0000 7fa02b59a740 1 -- 172.21.15.32:0/2786124234 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x564a82083430 con 0x564a81feec80 2024-04-24T23:03:06.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.961+0000 7fa0017fa640 1 -- 172.21.15.32:0/2786124234 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa01404e040 con 0x564a81feec80 2024-04-24T23:03:06.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.961+0000 7fa0017fa640 1 -- 172.21.15.32:0/2786124234 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 36) ==== 3494+0+0 (secure 0 0 0) 0x7fa00c016b00 con 0x564a81feec80 2024-04-24T23:03:06.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.961+0000 7fa02b59a740 10 client.6624 fetch_fsmap finished waiting for FSMap version 36 2024-04-24T23:03:06.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.961+0000 7fa02b59a740 10 client.6624 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T23:03:06.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.961+0000 7fa02b59a740 20 client.6624 populate_metadata read hostname 'smithi032' 2024-04-24T23:03:06.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.961+0000 7fa02b59a740 1 --2- 172.21.15.32:0/2786124234 >> [v2:172.21.15.32:6833/464246210,v1:172.21.15.32:6835/464246210] conn(0x564a81fc3770 0x564a81fef910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:06.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.961+0000 7fa02b59a740 4 client.6624 mds_command: new command op to 6474 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:03:06.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.961+0000 7fa02b59a740 1 -- 172.21.15.32:0/2786124234 --> [v2:172.21.15.32:6833/464246210,v1:172.21.15.32:6835/464246210] -- command(tid 0: {"prefix": "status"}) -- 0x564a82002d50 con 0x564a81fc3770 2024-04-24T23:03:06.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.961+0000 7fa02259c640 1 --2- 172.21.15.32:0/2786124234 >> [v2:172.21.15.32:6833/464246210,v1:172.21.15.32:6835/464246210] conn(0x564a81fc3770 0x564a81fef910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:06.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.961+0000 7fa0017fa640 10 client.6624 ms_handle_connect on v2:172.21.15.32:6833/464246210 2024-04-24T23:03:06.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.961+0000 7fa02259c640 1 --2- 172.21.15.32:0/2786124234 >> [v2:172.21.15.32:6833/464246210,v1:172.21.15.32:6835/464246210] conn(0x564a81fc3770 0x564a81fef910 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=577cd168a93ea3b1 server_cookie=64b1720768a63a8d in_seq=0 out_seq=0 2024-04-24T23:03:06.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.962+0000 7fa0017fa640 1 -- 172.21.15.32:0/2786124234 <== mds.0 v2:172.21.15.32:6833/464246210 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x564a82002d50 con 0x564a81fc3770 2024-04-24T23:03:06.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.962+0000 7fa0017fa640 10 client.6624 handle_command_reply: tid=0 2024-04-24T23:03:06.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.962+0000 7fa02b59a740 1 client.6624 shutdown 2024-04-24T23:03:06.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.962+0000 7fa02b59a740 1 -- 172.21.15.32:0/2786124234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564a81feec80 msgr2=0x564a82082e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:06.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.962+0000 7fa02b59a740 1 --2- 172.21.15.32:0/2786124234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564a81feec80 0x564a82082e40 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fa02415c060 tx=0x7fa00c007e80 comp rx=0 tx=0).stop 2024-04-24T23:03:06.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.962+0000 7fa02b59a740 1 -- 172.21.15.32:0/2786124234 shutdown_connections 2024-04-24T23:03:06.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.962+0000 7fa02b59a740 1 --2- 172.21.15.32:0/2786124234 >> [v2:172.21.15.32:6833/464246210,v1:172.21.15.32:6835/464246210] conn(0x564a81fc3770 0x564a81fef910 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:06.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.962+0000 7fa02b59a740 1 --2- 172.21.15.32:0/2786124234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564a81feec80 0x564a82082e40 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:06.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.962+0000 7fa02b59a740 1 --2- 172.21.15.32:0/2786124234 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564a8200bb60 0x564a81fbf230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:06.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.962+0000 7fa02b59a740 1 --2- 172.21.15.32:0/2786124234 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a81fe99b0 0x564a82082900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:06.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.962+0000 7fa02b59a740 1 -- 172.21.15.32:0/2786124234 >> 172.21.15.32:0/2786124234 conn(0x564a81fee540 msgr2=0x564a81fddd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:06.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.962+0000 7fa0017fa640 0 client.6624 ms_handle_reset on v2:172.21.15.32:6833/464246210 2024-04-24T23:03:06.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.962+0000 7fa02b59a740 1 -- 172.21.15.32:0/2786124234 shutdown_connections 2024-04-24T23:03:06.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.962+0000 7fa02b59a740 1 -- 172.21.15.32:0/2786124234 wait complete. 2024-04-24T23:03:06.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.962+0000 7fa02b59a740 20 client.6624 trim_cache size 0 max 16384 2024-04-24T23:03:06.969 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:03:06.969 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", 2024-04-24T23:03:06.969 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2024-04-24T23:03:06.969 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 6474, 2024-04-24T23:03:06.969 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-04-24T23:03:06.969 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-04-24T23:03:06.970 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-04-24T23:03:06.970 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 5.6711098629999999, 2024-04-24T23:03:06.970 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 36, 2024-04-24T23:03:06.970 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 87, 2024-04-24T23:03:06.970 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 87, 2024-04-24T23:03:06.970 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 16.826905391 2024-04-24T23:03:06.970 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:03:06.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.962+0000 7fa02888c640 1 -- 172.21.15.32:0/874208265 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa00406be30 msgr2=0x7fa00406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:06.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.963+0000 7fa02888c640 1 --2- 172.21.15.32:0/874208265 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa00406be30 0x7fa00406e2f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa014034000 tx=0x7fa01402e000 comp rx=0 tx=0).stop 2024-04-24T23:03:06.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.963+0000 7fa02888c640 1 -- 172.21.15.32:0/874208265 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa02415e9b0 msgr2=0x7fa0241f44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:06.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.963+0000 7fa02888c640 1 --2- 172.21.15.32:0/874208265 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa02415e9b0 0x7fa0241f44e0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fa01c00cef0 tx=0x7fa01c00a710 comp rx=0 tx=0).stop 2024-04-24T23:03:06.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.963+0000 7fa02888c640 1 -- 172.21.15.32:0/874208265 shutdown_connections 2024-04-24T23:03:06.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.963+0000 7fa02888c640 1 --2- 172.21.15.32:0/874208265 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa00406be30 0x7fa00406e2f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:06.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.963+0000 7fa02888c640 1 --2- 172.21.15.32:0/874208265 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa02415e9b0 0x7fa0241f44e0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:06.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.963+0000 7fa02888c640 1 --2- 172.21.15.32:0/874208265 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0240d4210 0x7fa02414b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:06.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.963+0000 7fa02888c640 1 --2- 172.21.15.32:0/874208265 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa02406b820 0x7fa0241564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:06.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.963+0000 7fa02888c640 1 -- 172.21.15.32:0/874208265 >> 172.21.15.32:0/874208265 conn(0x7fa0240d11d0 msgr2=0x7fa02415c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:06.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.963+0000 7fa02888c640 1 -- 172.21.15.32:0/874208265 shutdown_connections 2024-04-24T23:03:06.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:06.963+0000 7fa02888c640 1 -- 172.21.15.32:0/874208265 wait complete. 2024-04-24T23:03:06.975 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T23:03:07.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.074+0000 7fd1d1ce1640 1 Processor -- start 2024-04-24T23:03:07.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.074+0000 7fd1d1ce1640 1 -- start start 2024-04-24T23:03:07.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.075+0000 7fd1d1ce1640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd1cc074210 0x7fd1cc06c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:07.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.075+0000 7fd1d1ce1640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd1cc0b0b80 0x7fd1cc0745f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:07.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.075+0000 7fd1d1ce1640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1cc0b0770 con 0x7fd1cc0ae140 2024-04-24T23:03:07.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.075+0000 7fd1d1ce1640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fd1cc0b1c20 con 0x7fd1cc074210 2024-04-24T23:03:07.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.075+0000 7fd1d1ce1640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd1cc074b30 con 0x7fd1cc0b0b80 2024-04-24T23:03:07.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.075+0000 7fd1cbfff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd1cc0ae140 0x7fd1cc0b20b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57644/0 (socket says 172.21.15.32:57644) 2024-04-24T23:03:07.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.075+0000 7fd1cbfff640 1 -- 172.21.15.32:0/2032412338 learned_addr learned my addr 172.21.15.32:0/2032412338 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:03:07.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.075+0000 7fd1caffd640 1 --2- 172.21.15.32:0/2032412338 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd1cc0b0b80 0x7fd1cc0745f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:07.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.075+0000 7fd1cb7fe640 1 --2- 172.21.15.32:0/2032412338 >> v2:172.21.15.121:3300/0 conn(0x7fd1cc074210 0x7fd1cc06c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:07.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.075+0000 7fd1ca7fc640 1 -- 172.21.15.32:0/2032412338 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4264808422 0 0) 0x7fd1cc0b0770 con 0x7fd1cc0ae140 2024-04-24T23:03:07.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.075+0000 7fd1ca7fc640 1 -- 172.21.15.32:0/2032412338 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd1b8003520 con 0x7fd1cc0ae140 2024-04-24T23:03:07.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.075+0000 7fd1ca7fc640 1 -- 172.21.15.32:0/2032412338 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2179836212 0 0) 0x7fd1b8003520 con 0x7fd1cc0ae140 2024-04-24T23:03:07.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.075+0000 7fd1ca7fc640 1 -- 172.21.15.32:0/2032412338 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd1cc0b0b80 msgr2=0x7fd1cc0745f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:07.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.075+0000 7fd1ca7fc640 1 --2- 172.21.15.32:0/2032412338 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd1cc0b0b80 0x7fd1cc0745f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:07.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.075+0000 7fd1ca7fc640 1 -- 172.21.15.32:0/2032412338 >> v2:172.21.15.121:3300/0 conn(0x7fd1cc074210 msgr2=0x7fd1cc06c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:07.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.075+0000 7fd1ca7fc640 1 --2- 172.21.15.32:0/2032412338 >> v2:172.21.15.121:3300/0 conn(0x7fd1cc074210 0x7fd1cc06c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:07.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.075+0000 7fd1ca7fc640 1 -- 172.21.15.32:0/2032412338 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1cc0b1010 con 0x7fd1cc0ae140 2024-04-24T23:03:07.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.075+0000 7fd1ca7fc640 1 -- 172.21.15.32:0/2032412338 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7fd1c40019f0 con 0x7fd1cc0ae140 2024-04-24T23:03:07.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.075+0000 7fd1ca7fc640 1 -- 172.21.15.32:0/2032412338 >> v1:172.21.15.32:6789/0 conn(0x7fd1cc0ae140 legacy=0x7fd1cc0b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:07.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.075+0000 7fd1ca7fc640 1 --2- 172.21.15.32:0/2032412338 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1b80039b0 0x7fd1b8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:07.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.075+0000 7fd1ca7fc640 1 --2- 172.21.15.32:0/2032412338 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd1b8004800 0x7fd1b8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:07.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.075+0000 7fd1ca7fc640 1 --2- 172.21.15.32:0/2032412338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1b8005180 0x7fd1b800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:07.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.076+0000 7fd1ca7fc640 1 -- 172.21.15.32:0/2032412338 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc0b1c20 con 0x7fd1b80039b0 2024-04-24T23:03:07.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.076+0000 7fd1ca7fc640 1 -- 172.21.15.32:0/2032412338 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc074b30 con 0x7fd1b8005180 2024-04-24T23:03:07.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.076+0000 7fd1ca7fc640 1 -- 172.21.15.32:0/2032412338 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd1b8004600 con 0x7fd1b8004800 2024-04-24T23:03:07.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.076+0000 7fd1cb7fe640 1 --2- 172.21.15.32:0/2032412338 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1b80039b0 0x7fd1b8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:07.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.076+0000 7fd1caffd640 1 --2- 172.21.15.32:0/2032412338 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd1b8004800 0x7fd1b8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:07.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.076+0000 7fd1cbfff640 1 --2- 172.21.15.32:0/2032412338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1b8005180 0x7fd1b800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:07.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.076+0000 7fd1cb7fe640 1 -- 172.21.15.32:0/2032412338 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd1b8004800 msgr2=0x7fd1b8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:07.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.076+0000 7fd1cb7fe640 1 --2- 172.21.15.32:0/2032412338 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd1b8004800 0x7fd1b8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:07.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.076+0000 7fd1cb7fe640 1 -- 172.21.15.32:0/2032412338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1b8005180 msgr2=0x7fd1b800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:07.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.076+0000 7fd1cb7fe640 1 --2- 172.21.15.32:0/2032412338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1b8005180 0x7fd1b800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:07.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.076+0000 7fd1cb7fe640 1 -- 172.21.15.32:0/2032412338 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1b8003520 con 0x7fd1b80039b0 2024-04-24T23:03:07.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.076+0000 7fd1cbfff640 1 --2- 172.21.15.32:0/2032412338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1b8005180 0x7fd1b800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:03:07.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.076+0000 7fd1cb7fe640 1 --2- 172.21.15.32:0/2032412338 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1b80039b0 0x7fd1b8003dd0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4009ac0 tx=0x7fd1b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=785072a5f1edd9a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:07.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.077+0000 7fd1ca7fc640 1 -- 172.21.15.32:0/2032412338 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4036070 con 0x7fd1b80039b0 2024-04-24T23:03:07.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.077+0000 7fd1ca7fc640 1 -- 172.21.15.32:0/2032412338 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1b4031400 con 0x7fd1b80039b0 2024-04-24T23:03:07.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.077+0000 7fd1ca7fc640 1 -- 172.21.15.32:0/2032412338 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4009690 con 0x7fd1b80039b0 2024-04-24T23:03:07.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.077+0000 7fd1d1ce1640 1 -- 172.21.15.32:0/2032412338 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1b80039b0 msgr2=0x7fd1b8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:07.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.077+0000 7fd1d1ce1640 1 --2- 172.21.15.32:0/2032412338 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1b80039b0 0x7fd1b8003dd0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4009ac0 tx=0x7fd1b8005830 comp rx=0 tx=0).stop 2024-04-24T23:03:07.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.077+0000 7fd1cbfff640 1 -- 172.21.15.32:0/2032412338 reap_dead start 2024-04-24T23:03:07.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.077+0000 7fd1d1ce1640 1 -- 172.21.15.32:0/2032412338 shutdown_connections 2024-04-24T23:03:07.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.077+0000 7fd1d1ce1640 1 -- 172.21.15.32:0/2032412338 >> 172.21.15.32:0/2032412338 conn(0x7fd1cc0b2e60 msgr2=0x7fd1cc0b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:07.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.077+0000 7fd1d1ce1640 1 -- 172.21.15.32:0/2032412338 shutdown_connections 2024-04-24T23:03:07.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.077+0000 7fd1d1ce1640 1 -- 172.21.15.32:0/2032412338 wait complete. 2024-04-24T23:03:07.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.078+0000 7fd1d1ce1640 1 Processor -- start 2024-04-24T23:03:07.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.079+0000 7fd1d1ce1640 1 -- start start 2024-04-24T23:03:07.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.079+0000 7fd1d1ce1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1b80039b0 0x7fd1cc09e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:07.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.079+0000 7fd1d1ce1640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1cc15af80 0x7fd1cc09e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:07.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.079+0000 7fd1d1ce1640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd1cc09bcb0 0x7fd1cc09c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:07.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.079+0000 7fd1d1ce1640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc0b18b0 con 0x7fd1b80039b0 2024-04-24T23:03:07.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.079+0000 7fd1d1ce1640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc0b15d0 con 0x7fd1cc15af80 2024-04-24T23:03:07.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.079+0000 7fd1d1ce1640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd1cc0b3ba0 con 0x7fd1cc09bcb0 2024-04-24T23:03:07.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.079+0000 7fd1cb7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1b80039b0 0x7fd1cc09e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:07.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.079+0000 7fd1cb7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1b80039b0 0x7fd1cc09e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38578/0 (socket says 172.21.15.32:38578) 2024-04-24T23:03:07.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.079+0000 7fd1cb7fe640 1 -- 172.21.15.32:0/1952441162 learned_addr learned my addr 172.21.15.32:0/1952441162 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:07.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.080+0000 7fd1cb7fe640 1 -- 172.21.15.32:0/1952441162 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd1cc09bcb0 msgr2=0x7fd1cc09c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:07.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.080+0000 7fd1cb7fe640 1 --2- 172.21.15.32:0/1952441162 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd1cc09bcb0 0x7fd1cc09c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:07.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.080+0000 7fd1cb7fe640 1 -- 172.21.15.32:0/1952441162 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1cc15af80 msgr2=0x7fd1cc09e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:07.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.080+0000 7fd1cb7fe640 1 --2- 172.21.15.32:0/1952441162 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1cc15af80 0x7fd1cc09e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:07.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.080+0000 7fd1cb7fe640 1 -- 172.21.15.32:0/1952441162 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1cc09c8c0 con 0x7fd1b80039b0 2024-04-24T23:03:07.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.080+0000 7fd1cb7fe640 1 --2- 172.21.15.32:0/1952441162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1b80039b0 0x7fd1cc09e350 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4009ac0 tx=0x7fd1b4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:07.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.080+0000 7fd1c3fff640 1 -- 172.21.15.32:0/1952441162 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4040020 con 0x7fd1b80039b0 2024-04-24T23:03:07.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.080+0000 7fd1c3fff640 1 -- 172.21.15.32:0/1952441162 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1b4003ac0 con 0x7fd1b80039b0 2024-04-24T23:03:07.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.080+0000 7fd1d1ce1640 1 -- 172.21.15.32:0/1952441162 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1cc09cba0 con 0x7fd1b80039b0 2024-04-24T23:03:07.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.080+0000 7fd1c3fff640 1 -- 172.21.15.32:0/1952441162 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4036040 con 0x7fd1b80039b0 2024-04-24T23:03:07.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.080+0000 7fd1d1ce1640 1 -- 172.21.15.32:0/1952441162 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1cc15f580 con 0x7fd1b80039b0 2024-04-24T23:03:07.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.081+0000 7fd1d1ce1640 1 -- 172.21.15.32:0/1952441162 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1cc09f1e0 con 0x7fd1b80039b0 2024-04-24T23:03:07.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.081+0000 7fd1c3fff640 1 -- 172.21.15.32:0/1952441162 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd1b4043020 con 0x7fd1b80039b0 2024-04-24T23:03:07.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.082+0000 7fd1c3fff640 1 --2- 172.21.15.32:0/1952441162 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd1ac06be60 0x7fd1ac06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:07.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.082+0000 7fd1c3fff640 1 -- 172.21.15.32:0/1952441162 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7fd1b40aaee0 con 0x7fd1b80039b0 2024-04-24T23:03:07.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.082+0000 7fd1caffd640 1 --2- 172.21.15.32:0/1952441162 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd1ac06be60 0x7fd1ac06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:07.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.083+0000 7fd1caffd640 1 --2- 172.21.15.32:0/1952441162 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd1ac06be60 0x7fd1ac06e320 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc0b2c10 tx=0x7fd1bc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:07.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.086+0000 7fd1c3fff640 1 -- 172.21.15.32:0/1952441162 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd1b404a070 con 0x7fd1b80039b0 2024-04-24T23:03:07.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.308+0000 7fd1d1ce1640 1 -- 172.21.15.32:0/1952441162 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd1cc15eb70 con 0x7fd1b80039b0 2024-04-24T23:03:07.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.308+0000 7fd1c3fff640 1 -- 172.21.15.32:0/1952441162 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v36) ==== 53+0+73 (secure 0 0 0) 0x7fd1b40044e0 con 0x7fd1b80039b0 2024-04-24T23:03:07.310 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T23:03:07.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.310+0000 7fd1d1ce1640 1 -- 172.21.15.32:0/1952441162 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd1ac06be60 msgr2=0x7fd1ac06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:07.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.310+0000 7fd1d1ce1640 1 --2- 172.21.15.32:0/1952441162 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd1ac06be60 0x7fd1ac06e320 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc0b2c10 tx=0x7fd1bc008040 comp rx=0 tx=0).stop 2024-04-24T23:03:07.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.310+0000 7fd1d1ce1640 1 -- 172.21.15.32:0/1952441162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1b80039b0 msgr2=0x7fd1cc09e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:07.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.310+0000 7fd1d1ce1640 1 --2- 172.21.15.32:0/1952441162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1b80039b0 0x7fd1cc09e350 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4009ac0 tx=0x7fd1b4004730 comp rx=0 tx=0).stop 2024-04-24T23:03:07.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.310+0000 7fd1d1ce1640 1 -- 172.21.15.32:0/1952441162 shutdown_connections 2024-04-24T23:03:07.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.310+0000 7fd1d1ce1640 1 --2- 172.21.15.32:0/1952441162 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd1ac06be60 0x7fd1ac06e320 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:07.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.310+0000 7fd1d1ce1640 1 --2- 172.21.15.32:0/1952441162 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd1cc09bcb0 0x7fd1cc09c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:07.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.310+0000 7fd1d1ce1640 1 --2- 172.21.15.32:0/1952441162 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1cc15af80 0x7fd1cc09e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:07.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.310+0000 7fd1d1ce1640 1 --2- 172.21.15.32:0/1952441162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1b80039b0 0x7fd1cc09e350 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:07.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.310+0000 7fd1d1ce1640 1 -- 172.21.15.32:0/1952441162 >> 172.21.15.32:0/1952441162 conn(0x7fd1cc0b2e60 msgr2=0x7fd1cc0aeea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:07.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.310+0000 7fd1d1ce1640 1 -- 172.21.15.32:0/1952441162 shutdown_connections 2024-04-24T23:03:07.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07.310+0000 7fd1d1ce1640 1 -- 172.21.15.32:0/1952441162 wait complete. 2024-04-24T23:03:07.322 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-24T23:03:07.322 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T23:03:07.322 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2024-04-24T23:03:07.323 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-24T23:03:07.323 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T23:03:07.323 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-24T23:03:07.323 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-24T23:03:07.323 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T23:03:07.323 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:03:07.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:03:07.323 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2024-04-24T23:03:07.339 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T23:03:07.339 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T23:03:07.339 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T23:03:07.339 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:03:07.339 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2024-04-24T23:03:07.339 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:03:07.339 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:03:07.339 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2024-04-24T23:03:07.339 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2024-04-24T23:03:07.339 INFO:teuthology.orchestra.run.smithi032.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:03:07.339 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2024-04-24T23:03:07.340 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2024-04-24T23:03:07.340 INFO:teuthology.orchestra.run.smithi032.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T23:03:07.340 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:03:07.340 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2024-04-24T23:03:07.340 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-24T23:03:07.340 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 42521sec preferred_lft 42521sec 2024-04-24T23:03:07.340 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link noprefixroute 2024-04-24T23:03:07.340 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:03:07.340 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:03:07.340 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:03:07.340 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2024-04-24T23:03:07.340 INFO:teuthology.orchestra.run.smithi032.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T23:03:07.340 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 72:d4:96:00:43:8c brd ff:ff:ff:ff:ff:ff 2024-04-24T23:03:07.340 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-24T23:03:07.340 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:03:07.341 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::f4c6:20ff:fe22:2647/64 scope link 2024-04-24T23:03:07.341 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:03:07.341 INFO:teuthology.orchestra.run.smithi032.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:03:07.341 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 72:d4:96:00:43:8c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T23:03:07.341 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::70d4:96ff:fe00:438c/64 scope link 2024-04-24T23:03:07.341 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:03:07.341 INFO:teuthology.orchestra.run.smithi032.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:03:07.341 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether e6:6b:2a:90:1a:f8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T23:03:07.341 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::e46b:2aff:fe90:1af8/64 scope link 2024-04-24T23:03:07.341 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:03:07.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:03:07.342 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T23:03:07.402 INFO:teuthology.orchestra.run.smithi032.stdout:1 2024-04-24T23:03:07.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:03:07.404 DEBUG:teuthology.orchestra.run.smithi032:> route 2024-04-24T23:03:07.460 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2024-04-24T23:03:07.460 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T23:03:07.460 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-24T23:03:07.460 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-24T23:03:07.460 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T23:03:07.460 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:03:07.461 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-24T23:03:07.461 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-24T23:03:07.461 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-24T23:03:07.461 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-24T23:03:07.461 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-24T23:03:07.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:03:07.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:03:07.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:03:07.587 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-04-24T23:03:07.602 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-24T23:03:07.602 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-24T23:03:07.602 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:03:07.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:03:07.603 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2024-04-24T23:03:07.666 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:03:07.667 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-24T23:03:07.722 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:03:07.723 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:03:07.723 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-24T23:03:07.777 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-24T23:03:07.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:03:07.777 DEBUG:teuthology.orchestra.run.smithi032:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:03:07.860 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:03:07.861 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-24T23:03:07.867 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:03:07.867 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-24T23:03:07.867 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.121:3300,172.21.15.121:3301" 2024-04-24T23:03:07.867 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: trying mount with new device syntax: 0@eb1b7a01-6f65-4534-9a12-83011862ba51.cephfs=/ 2024-04-24T23:03:07.867 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.32:3300/172.21.15.121:3300/172.21.15.121:3301" will pass to kernel 2024-04-24T23:03:07.868 INFO:teuthology.orchestra.run.smithi032.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-24T23:03:07.868 INFO:teuthology.orchestra.run.smithi032.stdout: You just mounted a file system that supports labels which does not 2024-04-24T23:03:07.868 INFO:teuthology.orchestra.run.smithi032.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-24T23:03:07.868 INFO:teuthology.orchestra.run.smithi032.stdout: applications will generate AVC messages and not be allowed access to 2024-04-24T23:03:07.868 INFO:teuthology.orchestra.run.smithi032.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-24T23:03:07.869 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-24T23:03:07.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:03:07.870 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-24T23:03:07.935 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:03:08.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:03:08.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:03:08.037 INFO:teuthology.orchestra.run.smithi032.stdout:client6633 2024-04-24T23:03:08.037 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:03:08.037 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:03:08.053 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T23:03:08.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.191+0000 7f5bff8da640 1 Processor -- start 2024-04-24T23:03:08.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.192+0000 7f5bff8da640 1 -- start start 2024-04-24T23:03:08.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.192+0000 7f5bff8da640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5bf815e9b0 0x7f5bf80d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:08.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.192+0000 7f5bff8da640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5bf80d4210 0x7f5bf806b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:08.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.192+0000 7f5bff8da640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bf80d3170 con 0x7f5bf80ba0e0 2024-04-24T23:03:08.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.192+0000 7f5bff8da640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f5bf80b8900 con 0x7f5bf80d4210 2024-04-24T23:03:08.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.192+0000 7f5bff8da640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f5bf80d3450 con 0x7f5bf815e9b0 2024-04-24T23:03:08.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.193+0000 7f5bfce75640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5bf80d4210 0x7f5bf806b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:08.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.193+0000 7f5bfde77640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5bf80ba0e0 0x7f5bf8071730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57660/0 (socket says 172.21.15.32:57660) 2024-04-24T23:03:08.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.193+0000 7f5bfde77640 1 -- 172.21.15.32:0/612079761 learned_addr learned my addr 172.21.15.32:0/612079761 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:03:08.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.193+0000 7f5bfd676640 1 --2- 172.21.15.32:0/612079761 >> v2:172.21.15.121:3300/0 conn(0x7f5bf815e9b0 0x7f5bf80d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:08.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.193+0000 7f5bfce75640 1 -- 172.21.15.32:0/612079761 >> v2:172.21.15.121:3300/0 conn(0x7f5bf815e9b0 msgr2=0x7f5bf80d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:08.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.193+0000 7f5bfce75640 1 --2- 172.21.15.32:0/612079761 >> v2:172.21.15.121:3300/0 conn(0x7f5bf815e9b0 0x7f5bf80d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:08.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.193+0000 7f5bfce75640 1 -- 172.21.15.32:0/612079761 >> v1:172.21.15.32:6789/0 conn(0x7f5bf80ba0e0 legacy=0x7f5bf8071730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:08.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.193+0000 7f5bf7fff640 1 -- 172.21.15.32:0/612079761 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3037132313 0 0) 0x7f5bf80d3170 con 0x7f5bf80ba0e0 2024-04-24T23:03:08.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.193+0000 7f5bfce75640 1 -- 172.21.15.32:0/612079761 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bf80d3730 con 0x7f5bf80d4210 2024-04-24T23:03:08.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.194+0000 7f5bfce75640 1 --2- 172.21.15.32:0/612079761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bf80d4210 0x7f5bf806b110 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f5bf806be20 tx=0x7f5bec02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78a500e94f2b83fe server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:08.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.194+0000 7f5bf7fff640 1 -- 172.21.15.32:0/612079761 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bec037070 con 0x7f5bf80d4210 2024-04-24T23:03:08.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.194+0000 7f5bf7fff640 1 -- 172.21.15.32:0/612079761 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bec032440 con 0x7f5bf80d4210 2024-04-24T23:03:08.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.194+0000 7f5bf7fff640 1 -- 172.21.15.32:0/612079761 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bec02ae60 con 0x7f5bf80d4210 2024-04-24T23:03:08.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.194+0000 7f5bff8da640 1 -- 172.21.15.32:0/612079761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bf80d4210 msgr2=0x7f5bf806b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:08.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.194+0000 7f5bff8da640 1 --2- 172.21.15.32:0/612079761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bf80d4210 0x7f5bf806b110 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f5bf806be20 tx=0x7f5bec02a110 comp rx=0 tx=0).stop 2024-04-24T23:03:08.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.194+0000 7f5bff8da640 1 -- 172.21.15.32:0/612079761 shutdown_connections 2024-04-24T23:03:08.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.194+0000 7f5bff8da640 1 --2- 172.21.15.32:0/612079761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bf80d4210 0x7f5bf806b110 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:08.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.194+0000 7f5bff8da640 1 --2- 172.21.15.32:0/612079761 >> v2:172.21.15.121:3300/0 conn(0x7f5bf815e9b0 0x7f5bf80d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:08.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.194+0000 7f5bff8da640 1 -- 172.21.15.32:0/612079761 >> 172.21.15.32:0/612079761 conn(0x7f5bf80d11d0 msgr2=0x7f5bf80d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:08.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.195+0000 7f5bff8da640 1 -- 172.21.15.32:0/612079761 shutdown_connections 2024-04-24T23:03:08.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.195+0000 7f5bff8da640 1 -- 172.21.15.32:0/612079761 wait complete. 2024-04-24T23:03:08.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.196+0000 7f5bff8da640 1 Processor -- start 2024-04-24T23:03:08.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.196+0000 7f5bff8da640 1 -- start start 2024-04-24T23:03:08.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.196+0000 7f5bff8da640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5bf80ba0e0 0x7f5bf8158aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:08.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.196+0000 7f5bff8da640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5bf80d4210 0x7f5bf8152e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:08.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.196+0000 7f5bff8da640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bf815e9b0 0x7f5bf815d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:08.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.196+0000 7f5bff8da640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf80d2400 con 0x7f5bf815e9b0 2024-04-24T23:03:08.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.196+0000 7f5bff8da640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf80d3cf0 con 0x7f5bf80d4210 2024-04-24T23:03:08.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.196+0000 7f5bff8da640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f5bf806b650 con 0x7f5bf80ba0e0 2024-04-24T23:03:08.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.197+0000 7f5bfde77640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bf815e9b0 0x7f5bf815d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:08.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.197+0000 7f5bfde77640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bf815e9b0 0x7f5bf815d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38598/0 (socket says 172.21.15.32:38598) 2024-04-24T23:03:08.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.197+0000 7f5bfde77640 1 -- 172.21.15.32:0/2569667648 learned_addr learned my addr 172.21.15.32:0/2569667648 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:08.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.197+0000 7f5bfd676640 1 --2- 172.21.15.32:0/2569667648 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5bf80ba0e0 0x7f5bf8158aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:08.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.197+0000 7f5bfce75640 1 --2- 172.21.15.32:0/2569667648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5bf80d4210 0x7f5bf8152e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:08.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.197+0000 7f5bfde77640 1 -- 172.21.15.32:0/2569667648 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5bf80ba0e0 msgr2=0x7f5bf8158aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:08.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.197+0000 7f5bfde77640 1 --2- 172.21.15.32:0/2569667648 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5bf80ba0e0 0x7f5bf8158aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:08.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.197+0000 7f5bfde77640 1 -- 172.21.15.32:0/2569667648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5bf80d4210 msgr2=0x7f5bf8152e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:08.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.197+0000 7f5bfde77640 1 --2- 172.21.15.32:0/2569667648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5bf80d4210 0x7f5bf8152e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:08.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.197+0000 7f5bfde77640 1 -- 172.21.15.32:0/2569667648 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bf80ccc50 con 0x7f5bf815e9b0 2024-04-24T23:03:08.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.197+0000 7f5bfce75640 1 --2- 172.21.15.32:0/2569667648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5bf80d4210 0x7f5bf8152e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:03:08.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.197+0000 7f5bfd676640 1 --2- 172.21.15.32:0/2569667648 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5bf80ba0e0 0x7f5bf8158aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:03:08.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.197+0000 7f5bfde77640 1 --2- 172.21.15.32:0/2569667648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bf815e9b0 0x7f5bf815d5f0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f5bf000c480 tx=0x7f5bf000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:08.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.197+0000 7f5bf5ffb640 1 -- 172.21.15.32:0/2569667648 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf0018070 con 0x7f5bf815e9b0 2024-04-24T23:03:08.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.197+0000 7f5bf5ffb640 1 -- 172.21.15.32:0/2569667648 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bf0007500 con 0x7f5bf815e9b0 2024-04-24T23:03:08.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.197+0000 7f5bf5ffb640 1 -- 172.21.15.32:0/2569667648 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf0004a20 con 0x7f5bf815e9b0 2024-04-24T23:03:08.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.197+0000 7f5bff8da640 1 -- 172.21.15.32:0/2569667648 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bf80ccf30 con 0x7f5bf815e9b0 2024-04-24T23:03:08.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.197+0000 7f5bff8da640 1 -- 172.21.15.32:0/2569667648 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bf80cd4f0 con 0x7f5bf815e9b0 2024-04-24T23:03:08.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.198+0000 7f5bff8da640 1 -- 172.21.15.32:0/2569667648 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bf8068430 con 0x7f5bf815e9b0 2024-04-24T23:03:08.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.199+0000 7f5bf5ffb640 1 -- 172.21.15.32:0/2569667648 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5bf001b070 con 0x7f5bf815e9b0 2024-04-24T23:03:08.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.199+0000 7f5bf5ffb640 1 --2- 172.21.15.32:0/2569667648 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5be406be30 0x7f5be406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:08.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.199+0000 7f5bf5ffb640 1 -- 172.21.15.32:0/2569667648 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7f5bf008a140 con 0x7f5bf815e9b0 2024-04-24T23:03:08.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.199+0000 7f5bfd676640 1 --2- 172.21.15.32:0/2569667648 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5be406be30 0x7f5be406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:08.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.200+0000 7f5bfd676640 1 --2- 172.21.15.32:0/2569667648 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5be406be30 0x7f5be406e2f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5be8009fd0 tx=0x7f5be800b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:08.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.203+0000 7f5bf5ffb640 1 -- 172.21.15.32:0/2569667648 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5bf0058b80 con 0x7f5bf815e9b0 2024-04-24T23:03:08.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.429+0000 7f5bff8da640 1 -- 172.21.15.32:0/2569667648 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5bf80724c0 con 0x7f5bf815e9b0 2024-04-24T23:03:08.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.429+0000 7f5bf5ffb640 1 -- 172.21.15.32:0/2569667648 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v36) ==== 53+0+73 (secure 0 0 0) 0x7f5bf0058480 con 0x7f5bf815e9b0 2024-04-24T23:03:08.431 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T23:03:08.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.431+0000 7f5bff8da640 1 -- 172.21.15.32:0/2569667648 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5be406be30 msgr2=0x7f5be406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:08.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.431+0000 7f5bff8da640 1 --2- 172.21.15.32:0/2569667648 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5be406be30 0x7f5be406e2f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5be8009fd0 tx=0x7f5be800b040 comp rx=0 tx=0).stop 2024-04-24T23:03:08.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.431+0000 7f5bff8da640 1 -- 172.21.15.32:0/2569667648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bf815e9b0 msgr2=0x7f5bf815d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:08.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.431+0000 7f5bff8da640 1 --2- 172.21.15.32:0/2569667648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bf815e9b0 0x7f5bf815d5f0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f5bf000c480 tx=0x7f5bf000c940 comp rx=0 tx=0).stop 2024-04-24T23:03:08.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.431+0000 7f5bff8da640 1 -- 172.21.15.32:0/2569667648 shutdown_connections 2024-04-24T23:03:08.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.431+0000 7f5bff8da640 1 --2- 172.21.15.32:0/2569667648 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5be406be30 0x7f5be406e2f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:08.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.431+0000 7f5bff8da640 1 --2- 172.21.15.32:0/2569667648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bf815e9b0 0x7f5bf815d5f0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:08.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.431+0000 7f5bff8da640 1 --2- 172.21.15.32:0/2569667648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5bf80d4210 0x7f5bf8152e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:08.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.431+0000 7f5bff8da640 1 --2- 172.21.15.32:0/2569667648 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5bf80ba0e0 0x7f5bf8158aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:08.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.431+0000 7f5bff8da640 1 -- 172.21.15.32:0/2569667648 >> 172.21.15.32:0/2569667648 conn(0x7f5bf80d11d0 msgr2=0x7f5bf80b8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:08.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.431+0000 7f5bff8da640 1 -- 172.21.15.32:0/2569667648 shutdown_connections 2024-04-24T23:03:08.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:08.431+0000 7f5bff8da640 1 -- 172.21.15.32:0/2569667648 wait complete. 2024-04-24T23:03:08.443 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-24T23:03:08.443 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T23:03:08.443 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi121.front.sepia.ceph.com 2024-04-24T23:03:08.443 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-24T23:03:08.443 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-24T23:03:08.443 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-24T23:03:08.443 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-24T23:03:08.443 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T23:03:08.444 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-24T23:03:08.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:03:08.444 DEBUG:teuthology.orchestra.run.smithi121:> ip addr 2024-04-24T23:03:08.463 INFO:teuthology.orchestra.run.smithi121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T23:03:08.463 INFO:teuthology.orchestra.run.smithi121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T23:03:08.463 INFO:teuthology.orchestra.run.smithi121.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T23:03:08.463 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:03:08.463 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 ::1/128 scope host 2024-04-24T23:03:08.463 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:03:08.463 INFO:teuthology.orchestra.run.smithi121.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:03:08.464 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:82 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:03:08.464 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f0 2024-04-24T23:03:08.464 INFO:teuthology.orchestra.run.smithi121.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:03:08.464 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:83 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:03:08.464 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f1 2024-04-24T23:03:08.464 INFO:teuthology.orchestra.run.smithi121.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:03:08.464 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a2 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:03:08.464 INFO:teuthology.orchestra.run.smithi121.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T23:03:08.464 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a3 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:03:08.464 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.21.15.121/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-24T23:03:08.464 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft 42543sec preferred_lft 42543sec 2024-04-24T23:03:08.464 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::93dd:8489:4369:e127/64 scope link noprefixroute 2024-04-24T23:03:08.464 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:03:08.464 INFO:teuthology.orchestra.run.smithi121.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T23:03:08.465 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether c6:c0:c7:45:ae:ba brd ff:ff:ff:ff:ff:ff 2024-04-24T23:03:08.465 INFO:teuthology.orchestra.run.smithi121.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-24T23:03:08.465 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:03:08.465 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::80fe:d2ff:fe13:d1/64 scope link 2024-04-24T23:03:08.465 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:03:08.465 INFO:teuthology.orchestra.run.smithi121.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:03:08.465 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether e2:ca:b9:e9:9e:bb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T23:03:08.465 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::e0ca:b9ff:fee9:9ebb/64 scope link 2024-04-24T23:03:08.465 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:03:08.465 INFO:teuthology.orchestra.run.smithi121.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:03:08.465 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether c6:c0:c7:45:ae:ba brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-24T23:03:08.465 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::c4c0:c7ff:fe45:aeba/64 scope link 2024-04-24T23:03:08.465 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:03:08.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:03:08.466 DEBUG:teuthology.orchestra.run.smithi121:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T23:03:08.529 INFO:teuthology.orchestra.run.smithi121.stdout:1 2024-04-24T23:03:08.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:03:08.531 DEBUG:teuthology.orchestra.run.smithi121:> route 2024-04-24T23:03:08.589 INFO:teuthology.orchestra.run.smithi121.stdout:Kernel IP routing table 2024-04-24T23:03:08.589 INFO:teuthology.orchestra.run.smithi121.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T23:03:08.589 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-24T23:03:08.589 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-24T23:03:08.589 INFO:teuthology.orchestra.run.smithi121.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T23:03:08.590 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:03:08.590 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2024-04-24T23:03:08.590 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-24T23:03:08.590 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-24T23:03:08.590 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-24T23:03:08.590 DEBUG:teuthology.orchestra.run.smithi121:> ') 2024-04-24T23:03:08.682 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T23:03:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:03:08.713 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T23:03:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:03:08.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:03:08.720 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2024-04-24T23:03:08.735 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-24T23:03:08.735 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-24T23:03:08.735 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-24T23:03:08.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:03:08.735 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set brx.0 up 2024-04-24T23:03:08.799 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:03:08.800 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-24T23:03:08.854 INFO:teuthology.orchestra.run.smithi121.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-24T23:03:08.855 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:03:08.855 DEBUG:teuthology.orchestra.run.smithi121:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-24T23:03:08.909 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-24T23:03:08.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:03:08.910 DEBUG:teuthology.orchestra.run.smithi121:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:03:08.991 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:03:08.991 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-24T23:03:08.998 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:03:08.998 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-24T23:03:08.998 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.121:3300,172.21.15.121:3301" 2024-04-24T23:03:08.998 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: trying mount with new device syntax: 1@eb1b7a01-6f65-4534-9a12-83011862ba51.cephfs=/ 2024-04-24T23:03:08.998 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.32:3300/172.21.15.121:3300/172.21.15.121:3301" will pass to kernel 2024-04-24T23:03:08.999 INFO:teuthology.orchestra.run.smithi121.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-24T23:03:08.999 INFO:teuthology.orchestra.run.smithi121.stdout: You just mounted a file system that supports labels which does not 2024-04-24T23:03:08.999 INFO:teuthology.orchestra.run.smithi121.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-24T23:03:08.999 INFO:teuthology.orchestra.run.smithi121.stdout: applications will generate AVC messages and not be allowed access to 2024-04-24T23:03:08.999 INFO:teuthology.orchestra.run.smithi121.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-24T23:03:09.001 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-24T23:03:09.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:03:09.001 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-24T23:03:09.066 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:03:09.154 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T23:03:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:03:09.156 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T23:03:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:03:09.162 INFO:teuthology.orchestra.run.smithi121.stdout:client5588 2024-04-24T23:03:09.162 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:03:09.162 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:03:09.177 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:03:09.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.284+0000 7fd9ab163640 1 Processor -- start 2024-04-24T23:03:09.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.284+0000 7fd9ab163640 1 -- start start 2024-04-24T23:03:09.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.285+0000 7fd9ab163640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd9a415e9b0 0x7fd9a40d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:09.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.285+0000 7fd9ab163640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd9a406b820 0x7fd9a40b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:09.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.285+0000 7fd9ab163640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9a40d3170 con 0x7fd9a40d4210 2024-04-24T23:03:09.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.285+0000 7fd9ab163640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fd9a40d3450 con 0x7fd9a415e9b0 2024-04-24T23:03:09.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.285+0000 7fd9ab163640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd9a40d3730 con 0x7fd9a406b820 2024-04-24T23:03:09.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.285+0000 7fd9a8eff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd9a415e9b0 0x7fd9a40d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:09.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.285+0000 7fd9a3fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd9a40d4210 0x7fd9a406b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57676/0 (socket says 172.21.15.32:57676) 2024-04-24T23:03:09.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.285+0000 7fd9a8eff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd9a415e9b0 0x7fd9a40d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38614/0 (socket says 172.21.15.32:38614) 2024-04-24T23:03:09.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.285+0000 7fd9a3fff640 1 -- 172.21.15.32:0/472546036 learned_addr learned my addr 172.21.15.32:0/472546036 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:03:09.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.285+0000 7fd9a9700640 1 --2- 172.21.15.32:0/472546036 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd9a406b820 0x7fd9a40b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:09.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.286+0000 7fd9a8eff640 1 -- 172.21.15.32:0/472546036 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd9a406b820 msgr2=0x7fd9a40b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:09.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.286+0000 7fd9a8eff640 1 --2- 172.21.15.32:0/472546036 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd9a406b820 0x7fd9a40b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:09.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.286+0000 7fd9a8eff640 1 -- 172.21.15.32:0/472546036 >> v1:172.21.15.32:6789/0 conn(0x7fd9a40d4210 legacy=0x7fd9a406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:09.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.286+0000 7fd9a8eff640 1 -- 172.21.15.32:0/472546036 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9a40d3a10 con 0x7fd9a415e9b0 2024-04-24T23:03:09.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.286+0000 7fd9a37fe640 1 -- 172.21.15.32:0/472546036 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2355828284 0 0) 0x7fd9a40d3170 con 0x7fd9a40d4210 2024-04-24T23:03:09.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.286+0000 7fd9a8eff640 1 --2- 172.21.15.32:0/472546036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9a415e9b0 0x7fd9a40d2b90 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fd9980097a0 tx=0x7fd99802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c172e6a5b764587 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:09.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.287+0000 7fd9a37fe640 1 -- 172.21.15.32:0/472546036 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd998037070 con 0x7fd9a415e9b0 2024-04-24T23:03:09.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.287+0000 7fd9a37fe640 1 -- 172.21.15.32:0/472546036 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd998032440 con 0x7fd9a415e9b0 2024-04-24T23:03:09.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.287+0000 7fd9a37fe640 1 -- 172.21.15.32:0/472546036 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd998032810 con 0x7fd9a415e9b0 2024-04-24T23:03:09.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.287+0000 7fd9ab163640 1 -- 172.21.15.32:0/472546036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9a415e9b0 msgr2=0x7fd9a40d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:09.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.287+0000 7fd9ab163640 1 --2- 172.21.15.32:0/472546036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9a415e9b0 0x7fd9a40d2b90 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fd9980097a0 tx=0x7fd99802a180 comp rx=0 tx=0).stop 2024-04-24T23:03:09.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.287+0000 7fd9ab163640 1 -- 172.21.15.32:0/472546036 shutdown_connections 2024-04-24T23:03:09.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.287+0000 7fd9ab163640 1 --2- 172.21.15.32:0/472546036 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd9a406b820 0x7fd9a40b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:09.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.287+0000 7fd9ab163640 1 --2- 172.21.15.32:0/472546036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9a415e9b0 0x7fd9a40d2b90 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:09.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.287+0000 7fd9ab163640 1 -- 172.21.15.32:0/472546036 >> 172.21.15.32:0/472546036 conn(0x7fd9a40d11d0 msgr2=0x7fd9a40d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:09.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.287+0000 7fd9ab163640 1 -- 172.21.15.32:0/472546036 shutdown_connections 2024-04-24T23:03:09.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.287+0000 7fd9ab163640 1 -- 172.21.15.32:0/472546036 wait complete. 2024-04-24T23:03:09.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.289+0000 7fd9ab163640 1 Processor -- start 2024-04-24T23:03:09.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.289+0000 7fd9ab163640 1 -- start start 2024-04-24T23:03:09.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.290+0000 7fd9ab163640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9a406b820 0x7fd9a4154e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:09.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.290+0000 7fd9ab163640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd9a40d4210 0x7fd9a41f75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:09.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.290+0000 7fd9ab163640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9a415e9b0 0x7fd9a4158f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:09.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.290+0000 7fd9ab163640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a40ba0e0 con 0x7fd9a406b820 2024-04-24T23:03:09.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.290+0000 7fd9ab163640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a40d2400 con 0x7fd9a415e9b0 2024-04-24T23:03:09.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.290+0000 7fd9ab163640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd9a40ba260 con 0x7fd9a40d4210 2024-04-24T23:03:09.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.290+0000 7fd9a8eff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9a406b820 0x7fd9a4154e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:09.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.290+0000 7fd9a8eff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9a406b820 0x7fd9a4154e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38618/0 (socket says 172.21.15.32:38618) 2024-04-24T23:03:09.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.290+0000 7fd9a8eff640 1 -- 172.21.15.32:0/585307678 learned_addr learned my addr 172.21.15.32:0/585307678 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:09.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.290+0000 7fd9a3fff640 1 --2- 172.21.15.32:0/585307678 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd9a40d4210 0x7fd9a41f75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:09.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.290+0000 7fd9a9700640 1 --2- 172.21.15.32:0/585307678 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9a415e9b0 0x7fd9a4158f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:09.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.290+0000 7fd9a8eff640 1 -- 172.21.15.32:0/585307678 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd9a40d4210 msgr2=0x7fd9a41f75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:09.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.290+0000 7fd9a8eff640 1 --2- 172.21.15.32:0/585307678 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd9a40d4210 0x7fd9a41f75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:09.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.290+0000 7fd9a8eff640 1 -- 172.21.15.32:0/585307678 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9a415e9b0 msgr2=0x7fd9a4158f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:09.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.290+0000 7fd9a8eff640 1 --2- 172.21.15.32:0/585307678 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9a415e9b0 0x7fd9a4158f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:09.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.290+0000 7fd9a8eff640 1 -- 172.21.15.32:0/585307678 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9a41f7f60 con 0x7fd9a406b820 2024-04-24T23:03:09.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.291+0000 7fd9a8eff640 1 --2- 172.21.15.32:0/585307678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9a406b820 0x7fd9a4154e10 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fd9980097a0 tx=0x7fd9980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:09.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.291+0000 7fd9a17fa640 1 -- 172.21.15.32:0/585307678 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd998037050 con 0x7fd9a406b820 2024-04-24T23:03:09.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.291+0000 7fd9a17fa640 1 -- 172.21.15.32:0/585307678 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd998038040 con 0x7fd9a406b820 2024-04-24T23:03:09.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.291+0000 7fd9a17fa640 1 -- 172.21.15.32:0/585307678 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd998004490 con 0x7fd9a406b820 2024-04-24T23:03:09.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.291+0000 7fd9ab163640 1 -- 172.21.15.32:0/585307678 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9a41f8240 con 0x7fd9a406b820 2024-04-24T23:03:09.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.291+0000 7fd9ab163640 1 -- 172.21.15.32:0/585307678 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9a4158a40 con 0x7fd9a406b820 2024-04-24T23:03:09.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.292+0000 7fd9ab163640 1 -- 172.21.15.32:0/585307678 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9a4068430 con 0x7fd9a406b820 2024-04-24T23:03:09.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.292+0000 7fd9a17fa640 1 -- 172.21.15.32:0/585307678 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd99803c6d0 con 0x7fd9a406b820 2024-04-24T23:03:09.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.293+0000 7fd9a17fa640 1 --2- 172.21.15.32:0/585307678 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd98806be30 0x7fd98806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:09.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.293+0000 7fd9a17fa640 1 -- 172.21.15.32:0/585307678 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7fd9980ab6e0 con 0x7fd9a406b820 2024-04-24T23:03:09.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.293+0000 7fd9a3fff640 1 --2- 172.21.15.32:0/585307678 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd98806be30 0x7fd98806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:09.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.294+0000 7fd9a3fff640 1 --2- 172.21.15.32:0/585307678 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd98806be30 0x7fd98806e2f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4156830 tx=0x7fd990007d50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:09.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.298+0000 7fd9a17fa640 1 -- 172.21.15.32:0/585307678 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd998079ad0 con 0x7fd9a406b820 2024-04-24T23:03:09.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.518+0000 7fd9ab163640 1 -- 172.21.15.32:0/585307678 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd9a415dd20 con 0x7fd9a406b820 2024-04-24T23:03:09.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.519+0000 7fd9a17fa640 1 -- 172.21.15.32:0/585307678 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5186 (secure 0 0 0) 0x7fd99807ddd0 con 0x7fd9a406b820 2024-04-24T23:03:09.520 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:03:09.521 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":36,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5486,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/3550206194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3550206194},{"type":"v1","addr":"172.21.15.32:6834","nonce":3550206194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5905,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1706188546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1706188546},{"type":"v1","addr":"172.21.15.121:6835","nonce":1706188546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5911,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2669443498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2669443498},{"type":"v1","addr":"172.21.15.121:6837","nonce":2669443498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:03:00.630938+0000","modified":"2024-04-24T23:03:03.639570+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6474":{"gid":6474,"name":"a","rank":0,"incarnation":33,"state":"up:active","state_seq":4,"addr":"172.21.15.32:6835/464246210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":464246210},{"type":"v1","addr":"172.21.15.32:6835","nonce":464246210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6474,"qdb_cluster":[6474]},"id":5}]} 2024-04-24T23:03:09.521 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 36 2024-04-24T23:03:09.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.521+0000 7fd9ab163640 1 -- 172.21.15.32:0/585307678 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd98806be30 msgr2=0x7fd98806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:09.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.521+0000 7fd9ab163640 1 --2- 172.21.15.32:0/585307678 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd98806be30 0x7fd98806e2f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4156830 tx=0x7fd990007d50 comp rx=0 tx=0).stop 2024-04-24T23:03:09.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.521+0000 7fd9ab163640 1 -- 172.21.15.32:0/585307678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9a406b820 msgr2=0x7fd9a4154e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:09.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.521+0000 7fd9ab163640 1 --2- 172.21.15.32:0/585307678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9a406b820 0x7fd9a4154e10 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fd9980097a0 tx=0x7fd9980046c0 comp rx=0 tx=0).stop 2024-04-24T23:03:09.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.521+0000 7fd9ab163640 1 -- 172.21.15.32:0/585307678 shutdown_connections 2024-04-24T23:03:09.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.521+0000 7fd9ab163640 1 --2- 172.21.15.32:0/585307678 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd98806be30 0x7fd98806e2f0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:09.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.521+0000 7fd9ab163640 1 --2- 172.21.15.32:0/585307678 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9a415e9b0 0x7fd9a4158f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:09.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.521+0000 7fd9ab163640 1 --2- 172.21.15.32:0/585307678 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd9a40d4210 0x7fd9a41f75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:09.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.521+0000 7fd9ab163640 1 --2- 172.21.15.32:0/585307678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9a406b820 0x7fd9a4154e10 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:09.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.521+0000 7fd9ab163640 1 -- 172.21.15.32:0/585307678 >> 172.21.15.32:0/585307678 conn(0x7fd9a40d11d0 msgr2=0x7fd9a415d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:09.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.521+0000 7fd9ab163640 1 -- 172.21.15.32:0/585307678 shutdown_connections 2024-04-24T23:03:09.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:09.521+0000 7fd9ab163640 1 -- 172.21.15.32:0/585307678 wait complete. 2024-04-24T23:03:09.533 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-04-24T23:03:09.653 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","whoami":0,"id":6474,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":8.3521043709999994,"mdsmap_epoch":36,"osdmap_epoch":87,"osdmap_epoch_barrier":87,"uptime":19.507900868} 2024-04-24T23:03:09.654 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", "fs_name": "cephfs", "id": 6474, "mdsmap_epoch": 36, "osdmap_epoch": 87, "osdmap_epoch_barrier": 87, "rank_uptime": 8.352104371, "state": "up:active", "uptime": 19.507900868, "want_state": "up:active", "whoami": 0 } 2024-04-24T23:03:09.654 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-04-24T23:03:09.767 INFO:teuthology.orchestra.run.smithi032.stdout:{"osd_mon_report_interval":"5"} 2024-04-24T23:03:09.768 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-04-24T23:03:09.768 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2024-04-24T23:03:09.875 INFO:teuthology.orchestra.run.smithi032.stdout:{"mon_tick_interval":"5"} 2024-04-24T23:03:09.875 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2024-04-24T23:03:09.875 INFO:tasks.cephfs.test_full:pool capacity 557082176, 531MB should be enough to fill it 2024-04-24T23:03:09.875 DEBUG:teuthology.orchestra.run.smithi032:> stdin-killer -- python3 -c ' 2024-04-24T23:03:09.876 DEBUG:teuthology.orchestra.run.smithi032:> import time 2024-04-24T23:03:09.876 DEBUG:teuthology.orchestra.run.smithi032:> import datetime 2024-04-24T23:03:09.876 DEBUG:teuthology.orchestra.run.smithi032:> import subprocess 2024-04-24T23:03:09.876 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-24T23:03:09.876 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-24T23:03:09.876 DEBUG:teuthology.orchestra.run.smithi032:> # Write some buffered data through before going full, all should be well 2024-04-24T23:03:09.876 DEBUG:teuthology.orchestra.run.smithi032:> print("writing some data through which we expect to succeed") 2024-04-24T23:03:09.876 DEBUG:teuthology.orchestra.run.smithi032:> bytes = 0 2024-04-24T23:03:09.876 DEBUG:teuthology.orchestra.run.smithi032:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2024-04-24T23:03:09.876 DEBUG:teuthology.orchestra.run.smithi032:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2024-04-24T23:03:09.876 DEBUG:teuthology.orchestra.run.smithi032:> bytes += os.write(f, b'"'"'a'"'"' * 4096) 2024-04-24T23:03:09.876 DEBUG:teuthology.orchestra.run.smithi032:> os.fsync(f) 2024-04-24T23:03:09.876 DEBUG:teuthology.orchestra.run.smithi032:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2024-04-24T23:03:09.876 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-24T23:03:09.876 DEBUG:teuthology.orchestra.run.smithi032:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2024-04-24T23:03:09.877 DEBUG:teuthology.orchestra.run.smithi032:> # writing until we see an error from fsync. As long as we'"'"'re doing 2024-04-24T23:03:09.877 DEBUG:teuthology.orchestra.run.smithi032:> # buffered IO, the error should always only appear from fsync and not 2024-04-24T23:03:09.877 DEBUG:teuthology.orchestra.run.smithi032:> # from write 2024-04-24T23:03:09.877 DEBUG:teuthology.orchestra.run.smithi032:> full = False 2024-04-24T23:03:09.877 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-24T23:03:09.877 DEBUG:teuthology.orchestra.run.smithi032:> for n in range(0, int(531 * 1.1)): 2024-04-24T23:03:09.877 DEBUG:teuthology.orchestra.run.smithi032:> try: 2024-04-24T23:03:09.877 DEBUG:teuthology.orchestra.run.smithi032:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2024-04-24T23:03:09.877 DEBUG:teuthology.orchestra.run.smithi032:> print("wrote bytes via buffered write, moving on to fsync") 2024-04-24T23:03:09.877 DEBUG:teuthology.orchestra.run.smithi032:> except OSError as e: 2024-04-24T23:03:09.877 DEBUG:teuthology.orchestra.run.smithi032:> if False: 2024-04-24T23:03:09.877 DEBUG:teuthology.orchestra.run.smithi032:> print("Unexpected error %s from write() instead of fsync()" % e) 2024-04-24T23:03:09.877 DEBUG:teuthology.orchestra.run.smithi032:> raise 2024-04-24T23:03:09.877 DEBUG:teuthology.orchestra.run.smithi032:> else: 2024-04-24T23:03:09.877 DEBUG:teuthology.orchestra.run.smithi032:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2024-04-24T23:03:09.877 DEBUG:teuthology.orchestra.run.smithi032:> full = True 2024-04-24T23:03:09.877 DEBUG:teuthology.orchestra.run.smithi032:> break 2024-04-24T23:03:09.878 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-24T23:03:09.878 DEBUG:teuthology.orchestra.run.smithi032:> try: 2024-04-24T23:03:09.878 DEBUG:teuthology.orchestra.run.smithi032:> os.fsync(f) 2024-04-24T23:03:09.878 DEBUG:teuthology.orchestra.run.smithi032:> print("fsync'"'"'ed successfully") 2024-04-24T23:03:09.878 DEBUG:teuthology.orchestra.run.smithi032:> except OSError as e: 2024-04-24T23:03:09.878 DEBUG:teuthology.orchestra.run.smithi032:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2024-04-24T23:03:09.878 DEBUG:teuthology.orchestra.run.smithi032:> full = True 2024-04-24T23:03:09.878 DEBUG:teuthology.orchestra.run.smithi032:> break 2024-04-24T23:03:09.878 DEBUG:teuthology.orchestra.run.smithi032:> else: 2024-04-24T23:03:09.878 DEBUG:teuthology.orchestra.run.smithi032:> print("Not full yet after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2024-04-24T23:03:09.878 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-24T23:03:09.878 DEBUG:teuthology.orchestra.run.smithi032:> if n > 531 * 0.9: 2024-04-24T23:03:09.878 DEBUG:teuthology.orchestra.run.smithi032:> # Be cautious in the last region where we expect to hit 2024-04-24T23:03:09.878 DEBUG:teuthology.orchestra.run.smithi032:> # the full condition, so that we don'"'"'t overshoot too dramatically 2024-04-24T23:03:09.878 DEBUG:teuthology.orchestra.run.smithi032:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2024-04-24T23:03:09.878 DEBUG:teuthology.orchestra.run.smithi032:> time.sleep(15.0) 2024-04-24T23:03:09.879 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-24T23:03:09.879 DEBUG:teuthology.orchestra.run.smithi032:> if not full: 2024-04-24T23:03:09.879 DEBUG:teuthology.orchestra.run.smithi032:> raise RuntimeError("Failed to reach fullness after writing %d bytes" % bytes) 2024-04-24T23:03:09.879 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-24T23:03:09.879 DEBUG:teuthology.orchestra.run.smithi032:> # close() should not raise an error because we already caught it in 2024-04-24T23:03:09.879 DEBUG:teuthology.orchestra.run.smithi032:> # fsync. There shouldn'"'"'t have been any more writeback errors 2024-04-24T23:03:09.879 DEBUG:teuthology.orchestra.run.smithi032:> # since then because all IOs got cancelled on the full flag. 2024-04-24T23:03:09.879 DEBUG:teuthology.orchestra.run.smithi032:> print("calling close") 2024-04-24T23:03:09.879 DEBUG:teuthology.orchestra.run.smithi032:> os.close(f) 2024-04-24T23:03:09.879 DEBUG:teuthology.orchestra.run.smithi032:> print("close() did not raise error") 2024-04-24T23:03:09.879 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-24T23:03:09.879 DEBUG:teuthology.orchestra.run.smithi032:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2024-04-24T23:03:09.879 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-24T23:03:10.405 INFO:teuthology.orchestra.run.smithi032.stdout:writing some data through which we expect to succeed 2024-04-24T23:03:10.405 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed data successfully, will now attempt to fill fs 2024-04-24T23:03:10.405 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.406 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.406 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 1.00 MB 2024-04-24T23:03:10.406 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.406 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.406 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 2.00 MB 2024-04-24T23:03:10.406 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.406 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.406 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 3.00 MB 2024-04-24T23:03:10.406 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.406 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.406 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 4.00 MB 2024-04-24T23:03:10.406 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.407 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.407 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 5.00 MB 2024-04-24T23:03:10.407 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.407 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.407 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 6.00 MB 2024-04-24T23:03:10.407 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.407 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.407 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 7.00 MB 2024-04-24T23:03:10.407 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.407 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.407 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 8.00 MB 2024-04-24T23:03:10.407 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.407 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.408 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 9.00 MB 2024-04-24T23:03:10.408 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.408 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.408 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 10.00 MB 2024-04-24T23:03:10.408 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.408 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.408 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 11.00 MB 2024-04-24T23:03:10.408 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.408 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.408 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 12.00 MB 2024-04-24T23:03:10.408 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.408 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.409 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 13.00 MB 2024-04-24T23:03:10.409 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.409 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.409 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 14.00 MB 2024-04-24T23:03:10.409 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.409 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.409 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 15.00 MB 2024-04-24T23:03:10.409 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.410 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.410 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 16.00 MB 2024-04-24T23:03:10.410 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.410 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.410 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 17.00 MB 2024-04-24T23:03:10.410 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.410 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.410 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 18.00 MB 2024-04-24T23:03:10.410 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.410 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.410 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 19.00 MB 2024-04-24T23:03:10.411 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.411 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.411 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 20.00 MB 2024-04-24T23:03:10.411 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.411 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.411 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 21.00 MB 2024-04-24T23:03:10.411 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.411 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.411 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 22.00 MB 2024-04-24T23:03:10.411 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.411 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.411 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 23.00 MB 2024-04-24T23:03:10.411 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.412 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.412 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 24.00 MB 2024-04-24T23:03:10.412 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.412 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.412 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 25.00 MB 2024-04-24T23:03:10.412 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.412 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.412 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 26.00 MB 2024-04-24T23:03:10.412 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.412 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.412 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 27.00 MB 2024-04-24T23:03:10.412 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.412 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.413 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 28.00 MB 2024-04-24T23:03:10.413 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.413 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.413 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 29.00 MB 2024-04-24T23:03:10.413 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.413 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.413 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 30.00 MB 2024-04-24T23:03:10.413 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.413 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.413 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 31.00 MB 2024-04-24T23:03:10.413 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.413 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.413 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 32.00 MB 2024-04-24T23:03:10.413 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.414 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.414 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 33.00 MB 2024-04-24T23:03:10.414 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.414 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.414 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 34.00 MB 2024-04-24T23:03:10.414 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.414 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.414 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 35.00 MB 2024-04-24T23:03:10.414 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.414 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.414 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 36.00 MB 2024-04-24T23:03:10.414 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.415 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.415 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 37.00 MB 2024-04-24T23:03:10.415 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.415 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.415 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 38.00 MB 2024-04-24T23:03:10.415 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.415 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.415 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 39.00 MB 2024-04-24T23:03:10.415 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.415 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.415 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 40.00 MB 2024-04-24T23:03:10.415 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.415 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.416 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 41.00 MB 2024-04-24T23:03:10.416 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.416 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.416 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 42.00 MB 2024-04-24T23:03:10.416 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.416 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.416 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 43.00 MB 2024-04-24T23:03:10.416 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.416 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.416 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 44.00 MB 2024-04-24T23:03:10.416 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.416 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.417 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 45.00 MB 2024-04-24T23:03:10.417 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.417 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.417 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 46.00 MB 2024-04-24T23:03:10.417 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.417 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.417 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 47.00 MB 2024-04-24T23:03:10.417 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.417 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.417 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 48.00 MB 2024-04-24T23:03:10.417 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.417 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.418 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 49.00 MB 2024-04-24T23:03:10.418 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.418 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.418 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 50.00 MB 2024-04-24T23:03:10.418 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.418 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.418 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 51.00 MB 2024-04-24T23:03:10.418 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.418 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.418 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 52.00 MB 2024-04-24T23:03:10.418 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.418 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.419 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 53.00 MB 2024-04-24T23:03:10.419 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.419 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.419 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 54.00 MB 2024-04-24T23:03:10.419 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.419 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.419 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 55.00 MB 2024-04-24T23:03:10.419 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.419 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.419 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 56.00 MB 2024-04-24T23:03:10.419 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.419 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.419 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 57.00 MB 2024-04-24T23:03:10.420 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.420 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.420 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 58.00 MB 2024-04-24T23:03:10.420 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.420 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.420 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 59.00 MB 2024-04-24T23:03:10.420 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.420 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.420 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 60.00 MB 2024-04-24T23:03:10.420 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.420 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.420 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 61.00 MB 2024-04-24T23:03:10.421 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.421 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.421 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 62.00 MB 2024-04-24T23:03:10.421 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.421 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.421 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 63.00 MB 2024-04-24T23:03:10.421 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.421 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.421 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 64.00 MB 2024-04-24T23:03:10.421 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.421 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.421 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 65.00 MB 2024-04-24T23:03:10.421 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.421 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.422 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 66.00 MB 2024-04-24T23:03:10.422 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.422 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.422 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 67.00 MB 2024-04-24T23:03:10.422 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.422 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.422 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 68.00 MB 2024-04-24T23:03:10.422 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.422 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.422 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 69.00 MB 2024-04-24T23:03:10.422 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.422 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.422 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 70.00 MB 2024-04-24T23:03:10.422 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.423 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.423 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 71.00 MB 2024-04-24T23:03:10.423 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.423 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.423 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 72.00 MB 2024-04-24T23:03:10.423 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.423 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.423 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 73.00 MB 2024-04-24T23:03:10.423 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.423 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.423 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 74.00 MB 2024-04-24T23:03:10.423 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.424 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.424 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 75.00 MB 2024-04-24T23:03:10.424 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.424 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.424 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 76.00 MB 2024-04-24T23:03:10.424 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.424 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.424 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 77.00 MB 2024-04-24T23:03:10.424 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.424 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.424 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 78.00 MB 2024-04-24T23:03:10.424 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.424 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.424 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 79.00 MB 2024-04-24T23:03:10.425 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.425 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.425 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 80.00 MB 2024-04-24T23:03:10.652 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.653 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.653 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 81.00 MB 2024-04-24T23:03:10.653 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.653 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.653 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 82.00 MB 2024-04-24T23:03:10.653 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.653 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.653 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 83.00 MB 2024-04-24T23:03:10.654 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.654 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.654 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 84.00 MB 2024-04-24T23:03:10.654 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.654 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.654 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 85.00 MB 2024-04-24T23:03:10.654 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.654 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.654 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 86.00 MB 2024-04-24T23:03:10.654 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.654 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.655 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 87.00 MB 2024-04-24T23:03:10.655 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.655 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.655 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 88.00 MB 2024-04-24T23:03:10.655 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.655 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.655 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 89.00 MB 2024-04-24T23:03:10.655 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.655 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.655 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 90.00 MB 2024-04-24T23:03:10.655 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.655 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.656 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 91.00 MB 2024-04-24T23:03:10.656 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.656 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.656 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 92.00 MB 2024-04-24T23:03:10.656 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.656 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.656 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 93.00 MB 2024-04-24T23:03:10.656 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.657 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.657 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 94.00 MB 2024-04-24T23:03:10.657 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.657 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.657 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 95.00 MB 2024-04-24T23:03:10.657 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.657 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.657 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 96.00 MB 2024-04-24T23:03:10.657 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.657 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.657 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 97.00 MB 2024-04-24T23:03:10.658 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.658 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.658 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 98.00 MB 2024-04-24T23:03:10.658 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.658 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.658 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 99.00 MB 2024-04-24T23:03:10.658 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.658 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.658 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 100.00 MB 2024-04-24T23:03:10.658 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.658 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.659 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 101.00 MB 2024-04-24T23:03:10.659 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.659 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.659 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 102.00 MB 2024-04-24T23:03:10.659 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.659 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.659 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 103.00 MB 2024-04-24T23:03:10.659 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.660 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.660 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 104.00 MB 2024-04-24T23:03:10.660 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.660 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.660 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 105.00 MB 2024-04-24T23:03:10.660 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.660 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.660 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 106.00 MB 2024-04-24T23:03:10.660 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.660 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.660 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 107.00 MB 2024-04-24T23:03:10.661 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.661 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.661 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 108.00 MB 2024-04-24T23:03:10.661 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.661 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.661 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 109.00 MB 2024-04-24T23:03:10.661 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.661 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.661 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 110.00 MB 2024-04-24T23:03:10.661 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.661 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.662 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 111.00 MB 2024-04-24T23:03:10.662 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.662 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.662 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 112.00 MB 2024-04-24T23:03:10.662 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.662 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.662 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 113.00 MB 2024-04-24T23:03:10.662 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.662 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.662 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 114.00 MB 2024-04-24T23:03:10.663 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.663 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.663 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 115.00 MB 2024-04-24T23:03:10.663 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.663 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.663 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 116.00 MB 2024-04-24T23:03:10.663 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.663 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.663 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 117.00 MB 2024-04-24T23:03:10.663 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.664 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.664 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 118.00 MB 2024-04-24T23:03:10.664 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.664 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.664 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 119.00 MB 2024-04-24T23:03:10.664 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.664 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.664 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 120.00 MB 2024-04-24T23:03:10.664 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.664 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.665 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 121.00 MB 2024-04-24T23:03:10.665 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.665 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.665 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 122.00 MB 2024-04-24T23:03:10.665 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.665 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.665 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 123.00 MB 2024-04-24T23:03:10.665 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.665 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.665 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 124.00 MB 2024-04-24T23:03:10.665 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.666 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.666 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 125.00 MB 2024-04-24T23:03:10.666 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.666 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.666 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 126.00 MB 2024-04-24T23:03:10.666 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.666 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.666 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 127.00 MB 2024-04-24T23:03:10.666 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.666 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.667 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 128.00 MB 2024-04-24T23:03:10.667 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.667 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.667 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 129.00 MB 2024-04-24T23:03:10.667 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.667 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.667 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 130.00 MB 2024-04-24T23:03:10.667 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.667 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.667 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 131.00 MB 2024-04-24T23:03:10.668 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.668 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.668 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 132.00 MB 2024-04-24T23:03:10.668 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.668 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.668 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 133.00 MB 2024-04-24T23:03:10.668 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.668 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.668 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 134.00 MB 2024-04-24T23:03:10.668 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.669 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.669 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 135.00 MB 2024-04-24T23:03:10.669 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.669 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.669 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 136.00 MB 2024-04-24T23:03:10.669 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.669 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.669 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 137.00 MB 2024-04-24T23:03:10.669 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.669 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.669 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 138.00 MB 2024-04-24T23:03:10.670 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.670 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.670 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 139.00 MB 2024-04-24T23:03:10.670 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.670 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.670 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 140.00 MB 2024-04-24T23:03:10.670 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.670 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.670 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 141.00 MB 2024-04-24T23:03:10.670 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.671 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.671 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 142.00 MB 2024-04-24T23:03:10.671 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.671 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.671 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 143.00 MB 2024-04-24T23:03:10.671 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.671 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.671 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 144.00 MB 2024-04-24T23:03:10.671 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.671 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.671 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 145.00 MB 2024-04-24T23:03:10.672 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.672 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.672 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 146.00 MB 2024-04-24T23:03:10.672 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.672 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.672 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 147.00 MB 2024-04-24T23:03:10.672 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.672 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.672 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 148.00 MB 2024-04-24T23:03:10.672 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.673 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.673 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 149.00 MB 2024-04-24T23:03:10.673 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.673 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.673 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 150.00 MB 2024-04-24T23:03:10.673 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.673 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.673 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 151.00 MB 2024-04-24T23:03:10.674 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.674 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.674 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 152.00 MB 2024-04-24T23:03:10.674 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.674 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.674 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 153.00 MB 2024-04-24T23:03:10.674 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.674 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.674 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 154.00 MB 2024-04-24T23:03:10.675 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.675 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.675 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 155.00 MB 2024-04-24T23:03:10.675 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.675 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.675 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 156.00 MB 2024-04-24T23:03:10.675 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.675 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.675 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 157.00 MB 2024-04-24T23:03:10.675 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.675 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.675 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 158.00 MB 2024-04-24T23:03:10.676 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.676 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.676 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 159.00 MB 2024-04-24T23:03:10.676 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.676 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.676 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 160.00 MB 2024-04-24T23:03:10.676 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.900 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.900 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 161.00 MB 2024-04-24T23:03:10.900 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.901 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.901 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 162.00 MB 2024-04-24T23:03:10.901 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.901 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.901 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 163.00 MB 2024-04-24T23:03:10.901 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.901 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.901 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 164.00 MB 2024-04-24T23:03:10.901 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.901 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.901 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 165.00 MB 2024-04-24T23:03:10.901 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.901 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.901 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 166.00 MB 2024-04-24T23:03:10.901 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.902 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.902 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 167.00 MB 2024-04-24T23:03:10.902 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.902 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.902 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 168.00 MB 2024-04-24T23:03:10.902 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.902 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.902 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 169.00 MB 2024-04-24T23:03:10.902 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.902 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.902 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 170.00 MB 2024-04-24T23:03:10.902 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.902 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.902 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 171.00 MB 2024-04-24T23:03:10.903 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.903 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.903 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 172.00 MB 2024-04-24T23:03:10.903 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.903 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.903 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 173.00 MB 2024-04-24T23:03:10.903 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.903 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.903 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 174.00 MB 2024-04-24T23:03:10.903 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.903 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.903 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 175.00 MB 2024-04-24T23:03:10.903 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.903 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.903 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 176.00 MB 2024-04-24T23:03:10.904 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.904 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.904 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 177.00 MB 2024-04-24T23:03:10.904 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.904 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.904 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 178.00 MB 2024-04-24T23:03:10.904 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.904 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.904 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 179.00 MB 2024-04-24T23:03:10.904 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.904 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.904 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 180.00 MB 2024-04-24T23:03:10.904 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.904 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.904 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 181.00 MB 2024-04-24T23:03:10.905 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.905 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.905 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 182.00 MB 2024-04-24T23:03:10.905 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.905 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.905 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 183.00 MB 2024-04-24T23:03:10.905 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.905 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.905 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 184.00 MB 2024-04-24T23:03:10.905 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.905 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.905 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 185.00 MB 2024-04-24T23:03:10.905 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.905 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.905 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 186.00 MB 2024-04-24T23:03:10.906 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.906 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.906 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 187.00 MB 2024-04-24T23:03:10.906 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.906 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.906 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 188.00 MB 2024-04-24T23:03:10.906 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.906 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.906 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 189.00 MB 2024-04-24T23:03:10.906 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.906 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.906 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 190.00 MB 2024-04-24T23:03:10.906 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.906 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.907 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 191.00 MB 2024-04-24T23:03:10.907 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.907 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.907 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 192.00 MB 2024-04-24T23:03:10.907 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.907 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.907 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 193.00 MB 2024-04-24T23:03:10.907 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.907 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.907 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 194.00 MB 2024-04-24T23:03:10.907 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.907 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.907 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 195.00 MB 2024-04-24T23:03:10.907 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.908 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.908 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 196.00 MB 2024-04-24T23:03:10.908 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.908 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.908 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 197.00 MB 2024-04-24T23:03:10.908 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.908 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.908 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 198.00 MB 2024-04-24T23:03:10.908 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.908 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.908 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 199.00 MB 2024-04-24T23:03:10.908 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.908 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.908 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 200.00 MB 2024-04-24T23:03:10.909 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.909 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.909 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 201.00 MB 2024-04-24T23:03:10.909 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.909 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.909 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 202.00 MB 2024-04-24T23:03:10.909 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.909 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.909 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 203.00 MB 2024-04-24T23:03:10.909 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.909 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.909 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 204.00 MB 2024-04-24T23:03:10.909 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.909 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.910 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 205.00 MB 2024-04-24T23:03:10.910 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.910 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.910 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 206.00 MB 2024-04-24T23:03:10.910 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.910 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.910 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 207.00 MB 2024-04-24T23:03:10.910 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.910 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.911 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 208.00 MB 2024-04-24T23:03:10.911 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.911 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.911 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 209.00 MB 2024-04-24T23:03:10.911 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.911 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.911 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 210.00 MB 2024-04-24T23:03:10.911 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.911 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.911 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 211.00 MB 2024-04-24T23:03:10.911 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.911 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.911 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 212.00 MB 2024-04-24T23:03:10.911 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.912 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.912 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 213.00 MB 2024-04-24T23:03:10.912 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.912 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.912 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 214.00 MB 2024-04-24T23:03:10.912 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.912 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.912 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 215.00 MB 2024-04-24T23:03:10.912 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.912 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.912 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 216.00 MB 2024-04-24T23:03:10.913 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.913 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.913 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 217.00 MB 2024-04-24T23:03:10.913 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.913 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.913 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 218.00 MB 2024-04-24T23:03:10.913 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.913 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.913 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 219.00 MB 2024-04-24T23:03:10.913 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.913 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.913 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 220.00 MB 2024-04-24T23:03:10.913 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.913 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.914 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 221.00 MB 2024-04-24T23:03:10.914 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.914 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.914 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 222.00 MB 2024-04-24T23:03:10.914 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.914 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.914 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 223.00 MB 2024-04-24T23:03:10.914 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.914 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.914 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 224.00 MB 2024-04-24T23:03:10.914 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.915 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.915 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 225.00 MB 2024-04-24T23:03:10.915 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.915 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.915 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 226.00 MB 2024-04-24T23:03:10.915 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.915 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.915 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 227.00 MB 2024-04-24T23:03:10.915 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.915 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.915 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 228.00 MB 2024-04-24T23:03:10.915 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.915 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.915 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 229.00 MB 2024-04-24T23:03:10.916 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.916 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.916 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 230.00 MB 2024-04-24T23:03:10.916 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.916 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.916 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 231.00 MB 2024-04-24T23:03:10.916 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.916 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.916 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 232.00 MB 2024-04-24T23:03:10.916 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.917 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.917 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 233.00 MB 2024-04-24T23:03:10.917 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.917 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.917 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 234.00 MB 2024-04-24T23:03:10.917 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.917 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.917 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 235.00 MB 2024-04-24T23:03:10.917 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.917 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.917 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 236.00 MB 2024-04-24T23:03:10.917 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.917 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.917 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 237.00 MB 2024-04-24T23:03:10.918 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.918 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.918 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 238.00 MB 2024-04-24T23:03:10.918 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.918 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.918 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 239.00 MB 2024-04-24T23:03:10.918 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.918 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:10.918 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 240.00 MB 2024-04-24T23:03:10.918 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:10.918 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.148 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 241.00 MB 2024-04-24T23:03:11.148 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.148 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.148 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 242.00 MB 2024-04-24T23:03:11.148 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.148 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.148 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 243.00 MB 2024-04-24T23:03:11.148 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.148 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.149 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 244.00 MB 2024-04-24T23:03:11.149 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.149 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.149 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 245.00 MB 2024-04-24T23:03:11.149 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.149 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.149 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 246.00 MB 2024-04-24T23:03:11.149 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.149 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.149 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 247.00 MB 2024-04-24T23:03:11.149 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.150 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.150 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 248.00 MB 2024-04-24T23:03:11.150 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.150 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.150 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 249.00 MB 2024-04-24T23:03:11.150 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.150 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.150 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 250.00 MB 2024-04-24T23:03:11.150 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.150 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.150 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 251.00 MB 2024-04-24T23:03:11.150 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.150 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.151 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 252.00 MB 2024-04-24T23:03:11.151 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.151 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.151 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 253.00 MB 2024-04-24T23:03:11.151 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.151 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.151 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 254.00 MB 2024-04-24T23:03:11.151 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.151 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.151 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 255.00 MB 2024-04-24T23:03:11.151 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.151 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.151 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 256.00 MB 2024-04-24T23:03:11.151 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.152 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.152 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 257.00 MB 2024-04-24T23:03:11.152 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.152 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.152 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 258.00 MB 2024-04-24T23:03:11.152 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.152 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.152 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 259.00 MB 2024-04-24T23:03:11.152 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.152 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.152 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 260.00 MB 2024-04-24T23:03:11.152 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.152 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.152 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 261.00 MB 2024-04-24T23:03:11.152 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.153 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.153 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 262.00 MB 2024-04-24T23:03:11.153 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.153 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.153 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 263.00 MB 2024-04-24T23:03:11.153 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.153 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.153 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 264.00 MB 2024-04-24T23:03:11.153 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.153 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.153 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 265.00 MB 2024-04-24T23:03:11.153 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.153 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.153 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 266.00 MB 2024-04-24T23:03:11.154 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.154 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.154 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 267.00 MB 2024-04-24T23:03:11.154 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.154 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.154 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 268.00 MB 2024-04-24T23:03:11.154 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.154 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.154 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 269.00 MB 2024-04-24T23:03:11.154 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.154 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.154 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 270.00 MB 2024-04-24T23:03:11.154 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.154 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.154 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 271.00 MB 2024-04-24T23:03:11.155 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.155 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.155 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 272.00 MB 2024-04-24T23:03:11.155 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.155 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.156 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 273.00 MB 2024-04-24T23:03:11.156 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.156 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.156 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 274.00 MB 2024-04-24T23:03:11.156 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.156 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.156 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 275.00 MB 2024-04-24T23:03:11.156 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.156 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.156 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 276.00 MB 2024-04-24T23:03:11.156 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.156 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.156 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 277.00 MB 2024-04-24T23:03:11.156 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.157 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.157 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 278.00 MB 2024-04-24T23:03:11.157 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.157 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.157 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 279.00 MB 2024-04-24T23:03:11.157 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.157 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.157 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 280.00 MB 2024-04-24T23:03:11.157 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.157 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.157 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 281.00 MB 2024-04-24T23:03:11.157 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.157 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.157 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 282.00 MB 2024-04-24T23:03:11.157 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.158 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.158 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 283.00 MB 2024-04-24T23:03:11.158 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.158 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.158 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 284.00 MB 2024-04-24T23:03:11.158 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.158 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.158 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 285.00 MB 2024-04-24T23:03:11.158 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.158 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.158 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 286.00 MB 2024-04-24T23:03:11.158 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.158 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.158 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 287.00 MB 2024-04-24T23:03:11.159 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.159 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.159 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 288.00 MB 2024-04-24T23:03:11.159 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.159 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.159 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 289.00 MB 2024-04-24T23:03:11.159 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.159 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.159 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 290.00 MB 2024-04-24T23:03:11.159 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.159 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.159 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 291.00 MB 2024-04-24T23:03:11.159 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.159 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.160 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 292.00 MB 2024-04-24T23:03:11.160 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.160 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.160 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 293.00 MB 2024-04-24T23:03:11.160 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.160 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.160 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 294.00 MB 2024-04-24T23:03:11.160 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.160 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.160 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 295.00 MB 2024-04-24T23:03:11.160 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.160 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.160 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 296.00 MB 2024-04-24T23:03:11.160 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.161 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.161 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 297.00 MB 2024-04-24T23:03:11.161 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.161 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.161 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 298.00 MB 2024-04-24T23:03:11.161 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.161 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.161 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 299.00 MB 2024-04-24T23:03:11.161 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.161 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.161 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 300.00 MB 2024-04-24T23:03:11.161 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.161 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.161 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 301.00 MB 2024-04-24T23:03:11.162 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.162 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.162 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 302.00 MB 2024-04-24T23:03:11.162 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.162 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.162 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 303.00 MB 2024-04-24T23:03:11.162 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.162 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.162 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 304.00 MB 2024-04-24T23:03:11.162 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.162 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.162 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 305.00 MB 2024-04-24T23:03:11.162 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.162 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.163 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 306.00 MB 2024-04-24T23:03:11.163 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.163 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.163 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 307.00 MB 2024-04-24T23:03:11.163 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.163 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.163 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 308.00 MB 2024-04-24T23:03:11.163 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.163 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.163 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 309.00 MB 2024-04-24T23:03:11.163 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.163 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.163 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 310.00 MB 2024-04-24T23:03:11.163 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.164 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.164 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 311.00 MB 2024-04-24T23:03:11.164 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.164 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.164 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 312.00 MB 2024-04-24T23:03:11.164 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.164 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.164 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 313.00 MB 2024-04-24T23:03:11.164 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.164 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.164 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 314.00 MB 2024-04-24T23:03:11.164 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.164 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.165 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 315.00 MB 2024-04-24T23:03:11.165 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.165 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.165 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 316.00 MB 2024-04-24T23:03:11.165 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.165 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.165 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 317.00 MB 2024-04-24T23:03:11.165 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.165 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.165 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 318.00 MB 2024-04-24T23:03:11.165 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.165 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.165 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 319.00 MB 2024-04-24T23:03:11.165 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.166 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.166 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 320.00 MB 2024-04-24T23:03:11.166 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.166 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.166 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 321.00 MB 2024-04-24T23:03:11.394 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.394 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.394 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 322.00 MB 2024-04-24T23:03:11.394 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.394 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.395 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 323.00 MB 2024-04-24T23:03:11.395 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.395 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.395 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 324.00 MB 2024-04-24T23:03:11.395 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.395 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.395 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 325.00 MB 2024-04-24T23:03:11.395 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.395 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.395 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 326.00 MB 2024-04-24T23:03:11.395 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.395 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.395 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 327.00 MB 2024-04-24T23:03:11.396 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.396 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.396 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 328.00 MB 2024-04-24T23:03:11.396 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.396 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.396 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 329.00 MB 2024-04-24T23:03:11.396 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.396 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.396 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 330.00 MB 2024-04-24T23:03:11.396 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.396 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.396 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 331.00 MB 2024-04-24T23:03:11.396 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.397 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.397 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 332.00 MB 2024-04-24T23:03:11.397 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.397 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.397 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 333.00 MB 2024-04-24T23:03:11.397 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.397 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.397 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 334.00 MB 2024-04-24T23:03:11.397 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.397 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.397 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 335.00 MB 2024-04-24T23:03:11.397 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.397 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.397 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 336.00 MB 2024-04-24T23:03:11.398 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.398 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.398 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 337.00 MB 2024-04-24T23:03:11.398 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.398 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.398 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 338.00 MB 2024-04-24T23:03:11.398 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.398 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.398 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 339.00 MB 2024-04-24T23:03:11.398 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.398 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.398 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 340.00 MB 2024-04-24T23:03:11.398 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.399 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.399 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 341.00 MB 2024-04-24T23:03:11.399 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.399 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.399 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 342.00 MB 2024-04-24T23:03:11.399 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.399 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.399 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 343.00 MB 2024-04-24T23:03:11.400 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.400 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.400 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 344.00 MB 2024-04-24T23:03:11.400 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.400 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.400 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 345.00 MB 2024-04-24T23:03:11.400 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.400 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.400 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 346.00 MB 2024-04-24T23:03:11.400 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.400 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.400 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 347.00 MB 2024-04-24T23:03:11.400 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.401 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.401 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 348.00 MB 2024-04-24T23:03:11.401 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.401 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.401 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 349.00 MB 2024-04-24T23:03:11.401 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.401 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.401 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 350.00 MB 2024-04-24T23:03:11.401 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.402 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.402 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 351.00 MB 2024-04-24T23:03:11.402 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.402 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.402 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 352.00 MB 2024-04-24T23:03:11.402 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.402 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.402 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 353.00 MB 2024-04-24T23:03:11.402 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.402 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.402 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 354.00 MB 2024-04-24T23:03:11.402 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.402 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.402 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 355.00 MB 2024-04-24T23:03:11.403 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.403 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.403 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 356.00 MB 2024-04-24T23:03:11.403 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.403 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.403 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 357.00 MB 2024-04-24T23:03:11.403 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.403 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.404 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 358.00 MB 2024-04-24T23:03:11.404 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.404 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.404 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 359.00 MB 2024-04-24T23:03:11.404 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.404 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.404 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 360.00 MB 2024-04-24T23:03:11.404 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.404 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.404 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 361.00 MB 2024-04-24T23:03:11.404 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.404 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.404 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 362.00 MB 2024-04-24T23:03:11.405 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.405 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.405 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 363.00 MB 2024-04-24T23:03:11.405 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.405 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.405 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 364.00 MB 2024-04-24T23:03:11.405 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.405 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.405 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 365.00 MB 2024-04-24T23:03:11.405 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.405 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.406 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 366.00 MB 2024-04-24T23:03:11.406 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.406 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.406 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 367.00 MB 2024-04-24T23:03:11.406 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.406 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.406 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 368.00 MB 2024-04-24T23:03:11.406 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.406 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.406 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 369.00 MB 2024-04-24T23:03:11.406 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.407 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.407 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 370.00 MB 2024-04-24T23:03:11.407 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.407 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.407 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 371.00 MB 2024-04-24T23:03:11.407 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.407 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.407 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 372.00 MB 2024-04-24T23:03:11.407 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.408 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.408 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 373.00 MB 2024-04-24T23:03:11.408 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.408 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.408 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 374.00 MB 2024-04-24T23:03:11.408 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.408 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.408 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 375.00 MB 2024-04-24T23:03:11.408 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.408 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.409 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 376.00 MB 2024-04-24T23:03:11.409 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.409 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.409 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 377.00 MB 2024-04-24T23:03:11.409 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.409 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.409 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 378.00 MB 2024-04-24T23:03:11.409 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.409 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.409 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 379.00 MB 2024-04-24T23:03:11.409 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.410 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.410 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 380.00 MB 2024-04-24T23:03:11.410 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.410 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.410 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 381.00 MB 2024-04-24T23:03:11.410 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.410 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.410 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 382.00 MB 2024-04-24T23:03:11.410 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.410 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.411 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 383.00 MB 2024-04-24T23:03:11.411 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.411 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.411 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 384.00 MB 2024-04-24T23:03:11.411 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.411 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.411 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 385.00 MB 2024-04-24T23:03:11.411 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.411 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.411 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 386.00 MB 2024-04-24T23:03:11.411 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.412 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.412 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 387.00 MB 2024-04-24T23:03:11.412 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.412 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.412 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 388.00 MB 2024-04-24T23:03:11.412 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.412 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.412 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 389.00 MB 2024-04-24T23:03:11.412 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.412 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.412 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 390.00 MB 2024-04-24T23:03:11.412 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.412 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.413 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 391.00 MB 2024-04-24T23:03:11.413 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.413 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.413 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 392.00 MB 2024-04-24T23:03:11.413 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.413 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.413 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 393.00 MB 2024-04-24T23:03:11.413 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.413 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.413 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 394.00 MB 2024-04-24T23:03:11.413 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.413 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.413 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 395.00 MB 2024-04-24T23:03:11.414 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.414 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.414 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 396.00 MB 2024-04-24T23:03:11.414 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.414 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.414 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 397.00 MB 2024-04-24T23:03:11.414 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.414 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.414 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 398.00 MB 2024-04-24T23:03:11.414 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.414 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.414 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 399.00 MB 2024-04-24T23:03:11.414 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.414 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.415 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 400.00 MB 2024-04-24T23:03:11.415 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:11.415 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:11.415 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 401.00 MB 2024-04-24T23:03:15.317 INFO:tasks.ceph.mon.a.smithi032.stderr:2024-04-24T23:03:15.315+0000 7f3b831cb640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 full osd(s) (OSD_FULL) 2024-04-24T23:03:21.788 INFO:tasks.ceph.mon.a.smithi032.stderr:2024-04-24T23:03:21.786+0000 7f3b831cb640 -1 log_channel(cluster) log [ERR] : Health check update: 2 full osd(s) (OSD_FULL) 2024-04-24T23:03:36.599 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:03:36.603 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:03:41.663 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.664 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.664 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 402.00 MB 2024-04-24T23:03:41.664 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.664 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.664 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 403.00 MB 2024-04-24T23:03:41.664 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.664 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.664 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 404.00 MB 2024-04-24T23:03:41.664 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.665 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.665 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 405.00 MB 2024-04-24T23:03:41.665 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.665 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.665 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 406.00 MB 2024-04-24T23:03:41.665 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.665 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.665 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 407.00 MB 2024-04-24T23:03:41.665 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.665 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.665 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 408.00 MB 2024-04-24T23:03:41.665 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.665 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.666 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 409.00 MB 2024-04-24T23:03:41.666 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.666 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.666 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 410.00 MB 2024-04-24T23:03:41.666 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.666 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.666 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 411.00 MB 2024-04-24T23:03:41.666 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.666 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.666 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 412.00 MB 2024-04-24T23:03:41.666 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.667 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.667 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 413.00 MB 2024-04-24T23:03:41.667 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.667 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.667 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 414.00 MB 2024-04-24T23:03:41.667 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.667 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.667 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 415.00 MB 2024-04-24T23:03:41.667 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.667 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.667 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 416.00 MB 2024-04-24T23:03:41.668 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.668 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.668 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 417.00 MB 2024-04-24T23:03:41.668 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.668 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.668 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 418.00 MB 2024-04-24T23:03:41.668 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.668 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.668 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 419.00 MB 2024-04-24T23:03:41.668 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.668 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.668 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 420.00 MB 2024-04-24T23:03:41.668 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.668 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.669 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 421.00 MB 2024-04-24T23:03:41.669 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.669 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.669 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 422.00 MB 2024-04-24T23:03:41.669 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.669 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.669 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 423.00 MB 2024-04-24T23:03:41.669 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.669 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.669 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 424.00 MB 2024-04-24T23:03:41.669 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.670 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.670 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 425.00 MB 2024-04-24T23:03:41.670 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.670 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.670 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 426.00 MB 2024-04-24T23:03:41.670 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.670 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.670 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 427.00 MB 2024-04-24T23:03:41.670 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.670 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.670 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 428.00 MB 2024-04-24T23:03:41.670 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.671 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.671 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 429.00 MB 2024-04-24T23:03:41.671 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.671 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.671 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 430.00 MB 2024-04-24T23:03:41.671 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.671 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.671 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 431.00 MB 2024-04-24T23:03:41.671 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.671 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.671 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 432.00 MB 2024-04-24T23:03:41.671 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.671 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.672 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 433.00 MB 2024-04-24T23:03:41.672 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.672 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.672 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 434.00 MB 2024-04-24T23:03:41.672 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.672 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.672 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 435.00 MB 2024-04-24T23:03:41.672 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.672 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.672 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 436.00 MB 2024-04-24T23:03:41.672 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.672 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.673 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 437.00 MB 2024-04-24T23:03:41.673 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.673 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.673 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 438.00 MB 2024-04-24T23:03:41.673 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.673 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.673 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 439.00 MB 2024-04-24T23:03:41.674 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.674 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.674 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 440.00 MB 2024-04-24T23:03:41.674 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.674 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.674 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 441.00 MB 2024-04-24T23:03:41.674 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.674 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.674 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 442.00 MB 2024-04-24T23:03:41.674 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.674 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.675 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 443.00 MB 2024-04-24T23:03:41.675 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.675 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.675 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 444.00 MB 2024-04-24T23:03:41.675 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.675 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.675 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 445.00 MB 2024-04-24T23:03:41.675 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.675 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.675 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 446.00 MB 2024-04-24T23:03:41.675 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.675 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.676 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 447.00 MB 2024-04-24T23:03:41.676 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.676 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.676 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 448.00 MB 2024-04-24T23:03:41.676 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.676 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.676 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 449.00 MB 2024-04-24T23:03:41.676 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.676 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.676 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 450.00 MB 2024-04-24T23:03:41.676 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.676 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.677 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 451.00 MB 2024-04-24T23:03:41.677 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.677 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.677 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 452.00 MB 2024-04-24T23:03:41.677 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.677 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.677 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 453.00 MB 2024-04-24T23:03:41.677 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.677 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.677 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 454.00 MB 2024-04-24T23:03:41.677 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.677 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.678 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 455.00 MB 2024-04-24T23:03:41.678 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.678 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.678 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 456.00 MB 2024-04-24T23:03:41.678 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.678 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.678 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 457.00 MB 2024-04-24T23:03:41.678 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.678 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.678 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 458.00 MB 2024-04-24T23:03:41.678 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.678 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.679 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 459.00 MB 2024-04-24T23:03:41.679 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.679 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.679 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 460.00 MB 2024-04-24T23:03:41.679 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.679 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.679 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 461.00 MB 2024-04-24T23:03:41.679 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.679 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.679 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 462.00 MB 2024-04-24T23:03:41.679 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.680 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.680 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 463.00 MB 2024-04-24T23:03:41.680 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.680 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.680 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 464.00 MB 2024-04-24T23:03:41.680 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.680 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.680 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 465.00 MB 2024-04-24T23:03:41.680 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.680 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.680 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 466.00 MB 2024-04-24T23:03:41.681 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.681 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.681 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 467.00 MB 2024-04-24T23:03:41.681 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.681 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.681 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 468.00 MB 2024-04-24T23:03:41.681 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.681 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.681 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 469.00 MB 2024-04-24T23:03:41.681 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.682 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.682 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 470.00 MB 2024-04-24T23:03:41.682 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.682 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.682 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 471.00 MB 2024-04-24T23:03:41.682 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.682 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.682 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 472.00 MB 2024-04-24T23:03:41.682 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.682 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.683 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 473.00 MB 2024-04-24T23:03:41.683 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.683 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.683 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 474.00 MB 2024-04-24T23:03:41.683 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.683 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.683 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 475.00 MB 2024-04-24T23:03:41.683 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.683 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.683 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 476.00 MB 2024-04-24T23:03:41.683 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.684 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.684 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 477.00 MB 2024-04-24T23:03:41.684 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.684 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.684 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 478.00 MB 2024-04-24T23:03:41.684 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.684 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.684 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 479.00 MB 2024-04-24T23:03:41.684 INFO:teuthology.orchestra.run.smithi032.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-04-24T23:03:41.685 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:03:41.685 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:03:41.685 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 480.00 MB 2024-04-24T23:03:41.685 INFO:teuthology.orchestra.run.smithi032.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-04-24T23:03:41.691 INFO:teuthology.orchestra.run.smithi032.stdout:Reached fullness after 480.00 MB 2024-04-24T23:03:41.691 INFO:teuthology.orchestra.run.smithi032.stdout:calling close 2024-04-24T23:03:41.692 INFO:teuthology.orchestra.run.smithi032.stdout:close() did not raise error 2024-04-24T23:03:41.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:03:41.705 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-24T23:03:41.705 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:03:41.758 DEBUG:teuthology.orchestra.run.smithi121:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:03:41.799 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-24T23:03:41.799 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:03:41.800 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:03:41.800 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:03:41.818 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:03:41.818 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:03:41.876 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:03:42.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.021+0000 7f4965f63640 1 Processor -- start 2024-04-24T23:03:42.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.021+0000 7f4965f63640 1 -- start start 2024-04-24T23:03:42.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.022+0000 7f4965f63640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f496015b480 0x7f49600b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:42.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.022+0000 7f4965f63640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f496015fd60 0x7f496006c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:42.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.022+0000 7f4965f63640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f496006cb30 0x7f49600ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:42.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.022+0000 7f4965f63640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f49600b4ff0 con 0x7f496015b480 2024-04-24T23:03:42.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.022+0000 7f4965f63640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f49600b46c0 con 0x7f496015fd60 2024-04-24T23:03:42.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.022+0000 7f4965f63640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f49600b49a0 con 0x7f496006cb30 2024-04-24T23:03:42.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.022+0000 7f495f7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f496015b480 0x7f49600b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:42.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.022+0000 7f495f7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f496015b480 0x7f49600b5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53020/0 (socket says 172.21.15.32:53020) 2024-04-24T23:03:42.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.022+0000 7f495f7fe640 1 -- 172.21.15.32:0/1545099186 learned_addr learned my addr 172.21.15.32:0/1545099186 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:42.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.022+0000 7f495effd640 1 --2- 172.21.15.32:0/1545099186 >> v2:172.21.15.121:3300/0 conn(0x7f496015fd60 0x7f496006c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:42.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.022+0000 7f495f7fe640 1 -- 172.21.15.32:0/1545099186 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f496006cb30 msgr2=0x7f49600ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:42.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.022+0000 7f495f7fe640 1 --2- 172.21.15.32:0/1545099186 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f496006cb30 0x7f49600ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:42.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.022+0000 7f495f7fe640 1 -- 172.21.15.32:0/1545099186 >> v2:172.21.15.121:3300/0 conn(0x7f496015fd60 msgr2=0x7f496006c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:42.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.022+0000 7f495f7fe640 1 --2- 172.21.15.32:0/1545099186 >> v2:172.21.15.121:3300/0 conn(0x7f496015fd60 0x7f496006c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:42.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.022+0000 7f495f7fe640 1 -- 172.21.15.32:0/1545099186 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49600b4c80 con 0x7f496015b480 2024-04-24T23:03:42.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.023+0000 7f495f7fe640 1 --2- 172.21.15.32:0/1545099186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f496015b480 0x7f49600b5c60 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f496006ee10 tx=0x7f495002c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffaec265d6023be3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:42.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.023+0000 7f495e7fc640 1 -- 172.21.15.32:0/1545099186 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4950032790 con 0x7f496015b480 2024-04-24T23:03:42.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.023+0000 7f495e7fc640 1 -- 172.21.15.32:0/1545099186 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4950038070 con 0x7f496015b480 2024-04-24T23:03:42.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.023+0000 7f495e7fc640 1 -- 172.21.15.32:0/1545099186 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4950032b50 con 0x7f496015b480 2024-04-24T23:03:42.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.023+0000 7f4965f63640 1 -- 172.21.15.32:0/1545099186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f496015b480 msgr2=0x7f49600b5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:42.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.023+0000 7f4965f63640 1 --2- 172.21.15.32:0/1545099186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f496015b480 0x7f49600b5c60 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f496006ee10 tx=0x7f495002c890 comp rx=0 tx=0).stop 2024-04-24T23:03:42.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.024+0000 7f4965f63640 1 -- 172.21.15.32:0/1545099186 shutdown_connections 2024-04-24T23:03:42.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.024+0000 7f4965f63640 1 --2- 172.21.15.32:0/1545099186 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f496006cb30 0x7f49600ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:42.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.024+0000 7f4965f63640 1 --2- 172.21.15.32:0/1545099186 >> v2:172.21.15.121:3300/0 conn(0x7f496015fd60 0x7f496006c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:42.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.024+0000 7f4965f63640 1 --2- 172.21.15.32:0/1545099186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f496015b480 0x7f49600b5c60 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:42.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.024+0000 7f4965f63640 1 -- 172.21.15.32:0/1545099186 >> 172.21.15.32:0/1545099186 conn(0x7f496006b110 msgr2=0x7f496006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:42.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.024+0000 7f4965f63640 1 -- 172.21.15.32:0/1545099186 shutdown_connections 2024-04-24T23:03:42.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.024+0000 7f4965f63640 1 -- 172.21.15.32:0/1545099186 wait complete. 2024-04-24T23:03:42.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.025+0000 7f4965f63640 1 Processor -- start 2024-04-24T23:03:42.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.025+0000 7f4965f63640 1 -- start start 2024-04-24T23:03:42.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.026+0000 7f4965f63640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f496006cb30 0x7f4960151c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:42.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.026+0000 7f4965f63640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f496015b480 0x7f496015e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:42.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.026+0000 7f4965f63640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f496015fd60 0x7f49600ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:42.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.026+0000 7f4965f63640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f496015b860 con 0x7f496006cb30 2024-04-24T23:03:42.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.026+0000 7f4965f63640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f49600b5480 con 0x7f496015fd60 2024-04-24T23:03:42.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.026+0000 7f4965f63640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f496015e9d0 con 0x7f496015b480 2024-04-24T23:03:42.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.026+0000 7f495f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f496006cb30 0x7f4960151c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:42.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.026+0000 7f495f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f496006cb30 0x7f4960151c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53028/0 (socket says 172.21.15.32:53028) 2024-04-24T23:03:42.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.026+0000 7f495f7fe640 1 -- 172.21.15.32:0/1036777115 learned_addr learned my addr 172.21.15.32:0/1036777115 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:42.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.026+0000 7f495effd640 1 --2- 172.21.15.32:0/1036777115 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f496015b480 0x7f496015e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:42.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.026+0000 7f495ffff640 1 --2- 172.21.15.32:0/1036777115 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f496015fd60 0x7f49600ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:42.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.026+0000 7f495f7fe640 1 -- 172.21.15.32:0/1036777115 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f496015b480 msgr2=0x7f496015e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:42.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.026+0000 7f495f7fe640 1 --2- 172.21.15.32:0/1036777115 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f496015b480 0x7f496015e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:42.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.026+0000 7f495f7fe640 1 -- 172.21.15.32:0/1036777115 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f496015fd60 msgr2=0x7f49600ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:42.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.026+0000 7f495f7fe640 1 --2- 172.21.15.32:0/1036777115 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f496015fd60 0x7f49600ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:42.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.026+0000 7f495f7fe640 1 -- 172.21.15.32:0/1036777115 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49601525c0 con 0x7f496006cb30 2024-04-24T23:03:42.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.026+0000 7f495f7fe640 1 --2- 172.21.15.32:0/1036777115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f496006cb30 0x7f4960151c00 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f495000b2c0 tx=0x7f4950004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:42.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.027+0000 7f4947fff640 1 -- 172.21.15.32:0/1036777115 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4950043070 con 0x7f496006cb30 2024-04-24T23:03:42.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.027+0000 7f4947fff640 1 -- 172.21.15.32:0/1036777115 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f495003c620 con 0x7f496006cb30 2024-04-24T23:03:42.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.027+0000 7f4965f63640 1 -- 172.21.15.32:0/1036777115 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f496015c050 con 0x7f496006cb30 2024-04-24T23:03:42.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.027+0000 7f4947fff640 1 -- 172.21.15.32:0/1036777115 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495003c9f0 con 0x7f496006cb30 2024-04-24T23:03:42.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.027+0000 7f4965f63640 1 -- 172.21.15.32:0/1036777115 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f496015c610 con 0x7f496006cb30 2024-04-24T23:03:42.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.028+0000 7f4965f63640 1 -- 172.21.15.32:0/1036777115 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4960068430 con 0x7f496006cb30 2024-04-24T23:03:42.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.028+0000 7f4947fff640 1 -- 172.21.15.32:0/1036777115 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4950046070 con 0x7f496006cb30 2024-04-24T23:03:42.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.029+0000 7f4947fff640 1 --2- 172.21.15.32:0/1036777115 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f494c06be30 0x7f494c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:42.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.029+0000 7f4947fff640 1 -- 172.21.15.32:0/1036777115 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f49500ab430 con 0x7f496006cb30 2024-04-24T23:03:42.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.029+0000 7f4947fff640 1 -- 172.21.15.32:0/1036777115 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=91}) -- 0x7f494c073960 con 0x7f496006cb30 2024-04-24T23:03:42.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.029+0000 7f495effd640 1 --2- 172.21.15.32:0/1036777115 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f494c06be30 0x7f494c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:42.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.029+0000 7f495effd640 1 --2- 172.21.15.32:0/1036777115 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f494c06be30 0x7f494c06e2f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4960157440 tx=0x7f4954008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:42.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.033+0000 7f4947fff640 1 -- 172.21.15.32:0/1036777115 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f49601525c0 con 0x7f496006cb30 2024-04-24T23:03:42.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.218+0000 7f4965f63640 1 -- 172.21.15.32:0/1036777115 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f49601557e0 con 0x7f496006cb30 2024-04-24T23:03:42.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.219+0000 7f4947fff640 1 -- 172.21.15.32:0/1036777115 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+16709 (secure 0 0 0) 0x7f4950079790 con 0x7f496006cb30 2024-04-24T23:03:42.221 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:03:42.221 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":90,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:03:16.320059+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:02:58.930222+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-24T23:02:59.607070+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["backfillfull","exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","full","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["backfillfull","exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["backfillfull","exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["backfillfull","exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:03:42.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.221+0000 7f4965f63640 1 -- 172.21.15.32:0/1036777115 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f494c06be30 msgr2=0x7f494c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:42.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.221+0000 7f4965f63640 1 --2- 172.21.15.32:0/1036777115 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f494c06be30 0x7f494c06e2f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4960157440 tx=0x7f4954008040 comp rx=0 tx=0).stop 2024-04-24T23:03:42.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.221+0000 7f4965f63640 1 -- 172.21.15.32:0/1036777115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f496006cb30 msgr2=0x7f4960151c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:42.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.221+0000 7f4965f63640 1 --2- 172.21.15.32:0/1036777115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f496006cb30 0x7f4960151c00 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f495000b2c0 tx=0x7f4950004730 comp rx=0 tx=0).stop 2024-04-24T23:03:42.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.221+0000 7f4965f63640 1 -- 172.21.15.32:0/1036777115 shutdown_connections 2024-04-24T23:03:42.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.221+0000 7f4965f63640 1 --2- 172.21.15.32:0/1036777115 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f494c06be30 0x7f494c06e2f0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:42.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.221+0000 7f4965f63640 1 --2- 172.21.15.32:0/1036777115 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f496015fd60 0x7f49600ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:42.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.221+0000 7f4965f63640 1 --2- 172.21.15.32:0/1036777115 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f496015b480 0x7f496015e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:42.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.221+0000 7f4965f63640 1 --2- 172.21.15.32:0/1036777115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f496006cb30 0x7f4960151c00 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:42.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.221+0000 7f4965f63640 1 -- 172.21.15.32:0/1036777115 >> 172.21.15.32:0/1036777115 conn(0x7f496006b110 msgr2=0x7f49600b0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:42.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.221+0000 7f4965f63640 1 -- 172.21.15.32:0/1036777115 shutdown_connections 2024-04-24T23:03:42.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.221+0000 7f4965f63640 1 -- 172.21.15.32:0/1036777115 wait complete. 2024-04-24T23:03:42.234 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:02:58.930222+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:02:59.607070+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:03:42.234 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:03:42.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.337+0000 7f1bbaa83640 1 Processor -- start 2024-04-24T23:03:42.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.337+0000 7f1bbaa83640 1 -- start start 2024-04-24T23:03:42.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.338+0000 7f1bbaa83640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1bb406c5f0 0x7f1bb406c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:42.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.338+0000 7f1bbaa83640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1bb40b44d0 0x7f1bb40b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:42.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.338+0000 7f1bbaa83640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1bb40af890 0x7f1bb40cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:42.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.338+0000 7f1bbaa83640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f1bb40b6430 con 0x7f1bb406c5f0 2024-04-24T23:03:42.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.338+0000 7f1bbaa83640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f1bb4156f80 con 0x7f1bb40b44d0 2024-04-24T23:03:42.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.338+0000 7f1bbaa83640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f1bb40b6150 con 0x7f1bb40af890 2024-04-24T23:03:42.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.338+0000 7f1bb881f640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1bb406c5f0 0x7f1bb406c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:42.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.338+0000 7f1bb881f640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1bb406c5f0 0x7f1bb406c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53044/0 (socket says 172.21.15.32:53044) 2024-04-24T23:03:42.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.338+0000 7f1bb881f640 1 -- 172.21.15.32:0/1842821771 learned_addr learned my addr 172.21.15.32:0/1842821771 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:42.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.338+0000 7f1babfff640 1 --2- 172.21.15.32:0/1842821771 >> v2:172.21.15.121:3300/0 conn(0x7f1bb40b44d0 0x7f1bb40b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:42.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.338+0000 7f1bb9020640 1 --2- 172.21.15.32:0/1842821771 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1bb40af890 0x7f1bb40cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:42.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.338+0000 7f1bb881f640 1 -- 172.21.15.32:0/1842821771 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1bb40af890 msgr2=0x7f1bb40cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:42.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.338+0000 7f1bb881f640 1 --2- 172.21.15.32:0/1842821771 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1bb40af890 0x7f1bb40cef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:42.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.338+0000 7f1bb881f640 1 -- 172.21.15.32:0/1842821771 >> v2:172.21.15.121:3300/0 conn(0x7f1bb40b44d0 msgr2=0x7f1bb40b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:42.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.338+0000 7f1bb881f640 1 --2- 172.21.15.32:0/1842821771 >> v2:172.21.15.121:3300/0 conn(0x7f1bb40b44d0 0x7f1bb40b5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:42.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.338+0000 7f1bb881f640 1 -- 172.21.15.32:0/1842821771 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bb40b39f0 con 0x7f1bb406c5f0 2024-04-24T23:03:42.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.339+0000 7f1bb881f640 1 --2- 172.21.15.32:0/1842821771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1bb406c5f0 0x7f1bb406c9d0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4073080 tx=0x7f1bb002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41ced3c15c2642e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.339+0000 7f1bab7fe640 1 -- 172.21.15.32:0/1842821771 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb0037070 con 0x7f1bb406c5f0 2024-04-24T23:03:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.339+0000 7f1bab7fe640 1 -- 172.21.15.32:0/1842821771 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bb0032440 con 0x7f1bb406c5f0 2024-04-24T23:03:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.339+0000 7f1bab7fe640 1 -- 172.21.15.32:0/1842821771 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb0032810 con 0x7f1bb406c5f0 2024-04-24T23:03:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.339+0000 7f1bbaa83640 1 -- 172.21.15.32:0/1842821771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1bb406c5f0 msgr2=0x7f1bb406c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.339+0000 7f1bbaa83640 1 --2- 172.21.15.32:0/1842821771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1bb406c5f0 0x7f1bb406c9d0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4073080 tx=0x7f1bb002a140 comp rx=0 tx=0).stop 2024-04-24T23:03:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.340+0000 7f1bbaa83640 1 -- 172.21.15.32:0/1842821771 shutdown_connections 2024-04-24T23:03:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.340+0000 7f1bbaa83640 1 --2- 172.21.15.32:0/1842821771 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1bb40af890 0x7f1bb40cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.340+0000 7f1bbaa83640 1 --2- 172.21.15.32:0/1842821771 >> v2:172.21.15.121:3300/0 conn(0x7f1bb40b44d0 0x7f1bb40b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.340+0000 7f1bbaa83640 1 --2- 172.21.15.32:0/1842821771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1bb406c5f0 0x7f1bb406c9d0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.340+0000 7f1bbaa83640 1 -- 172.21.15.32:0/1842821771 >> 172.21.15.32:0/1842821771 conn(0x7f1bb406b110 msgr2=0x7f1bb406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.340+0000 7f1bbaa83640 1 -- 172.21.15.32:0/1842821771 shutdown_connections 2024-04-24T23:03:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.340+0000 7f1bbaa83640 1 -- 172.21.15.32:0/1842821771 wait complete. 2024-04-24T23:03:42.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.341+0000 7f1bbaa83640 1 Processor -- start 2024-04-24T23:03:42.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.341+0000 7f1bbaa83640 1 -- start start 2024-04-24T23:03:42.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.342+0000 7f1bbaa83640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1bb406c5f0 0x7f1bb40aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:42.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.342+0000 7f1bbaa83640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1bb40af890 0x7f1bb4149fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:42.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.342+0000 7f1bbaa83640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1bb40b44d0 0x7f1bb4151b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:42.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.342+0000 7f1bbaa83640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb406cf10 con 0x7f1bb40af890 2024-04-24T23:03:42.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.342+0000 7f1bbaa83640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb40b3fb0 con 0x7f1bb40b44d0 2024-04-24T23:03:42.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.342+0000 7f1bbaa83640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f1bb40b5dd0 con 0x7f1bb406c5f0 2024-04-24T23:03:42.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.342+0000 7f1babfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1bb40af890 0x7f1bb4149fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:42.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.342+0000 7f1babfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1bb40af890 0x7f1bb4149fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53054/0 (socket says 172.21.15.32:53054) 2024-04-24T23:03:42.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.342+0000 7f1babfff640 1 -- 172.21.15.32:0/1069336220 learned_addr learned my addr 172.21.15.32:0/1069336220 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:42.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.342+0000 7f1bb9020640 1 --2- 172.21.15.32:0/1069336220 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1bb40b44d0 0x7f1bb4151b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:42.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.342+0000 7f1bb881f640 1 --2- 172.21.15.32:0/1069336220 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1bb406c5f0 0x7f1bb40aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:42.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.342+0000 7f1babfff640 1 -- 172.21.15.32:0/1069336220 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1bb406c5f0 msgr2=0x7f1bb40aeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:42.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.342+0000 7f1babfff640 1 --2- 172.21.15.32:0/1069336220 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1bb406c5f0 0x7f1bb40aeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:42.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.342+0000 7f1babfff640 1 -- 172.21.15.32:0/1069336220 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1bb40b44d0 msgr2=0x7f1bb4151b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:42.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.342+0000 7f1babfff640 1 --2- 172.21.15.32:0/1069336220 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1bb40b44d0 0x7f1bb4151b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:42.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.342+0000 7f1babfff640 1 -- 172.21.15.32:0/1069336220 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bb40cd1f0 con 0x7f1bb40af890 2024-04-24T23:03:42.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.342+0000 7f1bb9020640 1 --2- 172.21.15.32:0/1069336220 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1bb40b44d0 0x7f1bb4151b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:03:42.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.342+0000 7f1babfff640 1 --2- 172.21.15.32:0/1069336220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1bb40af890 0x7f1bb4149fb0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f1ba400c7f0 tx=0x7f1ba400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:42.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.342+0000 7f1ba97fa640 1 -- 172.21.15.32:0/1069336220 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba4004230 con 0x7f1bb40af890 2024-04-24T23:03:42.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.343+0000 7f1ba97fa640 1 -- 172.21.15.32:0/1069336220 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ba40043d0 con 0x7f1bb40af890 2024-04-24T23:03:42.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.343+0000 7f1bbaa83640 1 -- 172.21.15.32:0/1069336220 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bb40cd4d0 con 0x7f1bb40af890 2024-04-24T23:03:42.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.343+0000 7f1ba97fa640 1 -- 172.21.15.32:0/1069336220 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba4007840 con 0x7f1bb40af890 2024-04-24T23:03:42.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.343+0000 7f1bbaa83640 1 -- 172.21.15.32:0/1069336220 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bb40cda90 con 0x7f1bb40af890 2024-04-24T23:03:42.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.344+0000 7f1bbaa83640 1 -- 172.21.15.32:0/1069336220 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bb4068430 con 0x7f1bb40af890 2024-04-24T23:03:42.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.344+0000 7f1ba97fa640 1 -- 172.21.15.32:0/1069336220 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1ba4007bf0 con 0x7f1bb40af890 2024-04-24T23:03:42.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.344+0000 7f1ba97fa640 1 --2- 172.21.15.32:0/1069336220 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1ba006be30 0x7f1ba006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:42.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.345+0000 7f1ba97fa640 1 -- 172.21.15.32:0/1069336220 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f1ba408b130 con 0x7f1bb40af890 2024-04-24T23:03:42.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.345+0000 7f1ba97fa640 1 -- 172.21.15.32:0/1069336220 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=91}) -- 0x7f1ba0073960 con 0x7f1bb40af890 2024-04-24T23:03:42.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.345+0000 7f1bb881f640 1 --2- 172.21.15.32:0/1069336220 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1ba006be30 0x7f1ba006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:42.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.345+0000 7f1bb881f640 1 --2- 172.21.15.32:0/1069336220 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1ba006be30 0x7f1ba006e2f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0034000 tx=0x7f1bb002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:42.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.349+0000 7f1ba97fa640 1 -- 172.21.15.32:0/1069336220 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1ba4015020 con 0x7f1bb40af890 2024-04-24T23:03:42.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.531+0000 7f1bbaa83640 1 -- 172.21.15.32:0/1069336220 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1bb40b5670 con 0x7f1bb40af890 2024-04-24T23:03:42.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.531+0000 7f1ba97fa640 1 -- 172.21.15.32:0/1069336220 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f1ba40599d0 con 0x7f1bb40af890 2024-04-24T23:03:42.533 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T23:03:42.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.533+0000 7f1bbaa83640 1 -- 172.21.15.32:0/1069336220 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1ba006be30 msgr2=0x7f1ba006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:42.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.533+0000 7f1bbaa83640 1 --2- 172.21.15.32:0/1069336220 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1ba006be30 0x7f1ba006e2f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0034000 tx=0x7f1bb002e000 comp rx=0 tx=0).stop 2024-04-24T23:03:42.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.533+0000 7f1bbaa83640 1 -- 172.21.15.32:0/1069336220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1bb40af890 msgr2=0x7f1bb4149fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:42.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.533+0000 7f1bbaa83640 1 --2- 172.21.15.32:0/1069336220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1bb40af890 0x7f1bb4149fb0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f1ba400c7f0 tx=0x7f1ba400ccb0 comp rx=0 tx=0).stop 2024-04-24T23:03:42.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.533+0000 7f1bbaa83640 1 -- 172.21.15.32:0/1069336220 shutdown_connections 2024-04-24T23:03:42.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.533+0000 7f1bbaa83640 1 --2- 172.21.15.32:0/1069336220 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1ba006be30 0x7f1ba006e2f0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:42.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.533+0000 7f1bbaa83640 1 --2- 172.21.15.32:0/1069336220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1bb40af890 0x7f1bb4149fb0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:42.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.533+0000 7f1bbaa83640 1 --2- 172.21.15.32:0/1069336220 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1bb40b44d0 0x7f1bb4151b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:42.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.533+0000 7f1bbaa83640 1 --2- 172.21.15.32:0/1069336220 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1bb406c5f0 0x7f1bb40aeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:42.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.533+0000 7f1bbaa83640 1 -- 172.21.15.32:0/1069336220 >> 172.21.15.32:0/1069336220 conn(0x7f1bb406b110 msgr2=0x7f1bb40b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:42.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.533+0000 7f1bbaa83640 1 -- 172.21.15.32:0/1069336220 shutdown_connections 2024-04-24T23:03:42.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.533+0000 7f1bbaa83640 1 -- 172.21.15.32:0/1069336220 wait complete. 2024-04-24T23:03:42.545 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:03:42.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.648+0000 7f5fbf909640 1 Processor -- start 2024-04-24T23:03:42.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.648+0000 7f5fbf909640 1 -- start start 2024-04-24T23:03:42.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.649+0000 7f5fbf909640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5fb815c6f0 0x7f5fb806c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:42.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.649+0000 7f5fbf909640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5fb806b820 0x7f5fb8070d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:42.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.649+0000 7f5fbf909640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fb80cf2f0 con 0x7f5fb806d2e0 2024-04-24T23:03:42.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.649+0000 7f5fbf909640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f5fb80afcd0 con 0x7f5fb815c6f0 2024-04-24T23:03:42.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.649+0000 7f5fbf909640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f5fb80d01d0 con 0x7f5fb806b820 2024-04-24T23:03:42.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.649+0000 7f5fbd6a5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5fb815c6f0 0x7f5fb806c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:42.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.649+0000 7f5fbd6a5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5fb815c6f0 0x7f5fb806c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53068/0 (socket says 172.21.15.32:53068) 2024-04-24T23:03:42.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.649+0000 7f5fbd6a5640 1 -- 172.21.15.32:0/394899967 learned_addr learned my addr 172.21.15.32:0/394899967 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:42.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.649+0000 7f5fbdea6640 1 --2- 172.21.15.32:0/394899967 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5fb806b820 0x7f5fb8070d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:42.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.649+0000 7f5fbd6a5640 1 -- 172.21.15.32:0/394899967 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5fb806b820 msgr2=0x7f5fb8070d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:42.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.649+0000 7f5fbd6a5640 1 --2- 172.21.15.32:0/394899967 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5fb806b820 0x7f5fb8070d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:42.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.649+0000 7f5fbd6a5640 1 -- 172.21.15.32:0/394899967 >> v1:172.21.15.121:6789/0 conn(0x7f5fb806d2e0 legacy=0x7f5fb806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:42.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.649+0000 7f5fbd6a5640 1 -- 172.21.15.32:0/394899967 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fb80cf590 con 0x7f5fb815c6f0 2024-04-24T23:03:42.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.649+0000 7f5fbdea6640 1 --2- 172.21.15.32:0/394899967 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5fb806b820 0x7f5fb8070d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:03:42.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.650+0000 7f5fbd6a5640 1 --2- 172.21.15.32:0/394899967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fb815c6f0 0x7f5fb806c7b0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4009870 tx=0x7f5fb402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b84ac0ba063dadfe server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:42.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.650+0000 7f5faffff640 1 -- 172.21.15.32:0/394899967 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb4037070 con 0x7f5fb815c6f0 2024-04-24T23:03:42.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.650+0000 7f5faffff640 1 -- 172.21.15.32:0/394899967 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fb4032440 con 0x7f5fb815c6f0 2024-04-24T23:03:42.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.650+0000 7f5faffff640 1 -- 172.21.15.32:0/394899967 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb4032810 con 0x7f5fb815c6f0 2024-04-24T23:03:42.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.651+0000 7f5fbf909640 1 -- 172.21.15.32:0/394899967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fb815c6f0 msgr2=0x7f5fb806c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:42.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.651+0000 7f5fbf909640 1 --2- 172.21.15.32:0/394899967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fb815c6f0 0x7f5fb806c7b0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4009870 tx=0x7f5fb402a180 comp rx=0 tx=0).stop 2024-04-24T23:03:42.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.651+0000 7f5fbf909640 1 -- 172.21.15.32:0/394899967 shutdown_connections 2024-04-24T23:03:42.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.651+0000 7f5fbf909640 1 --2- 172.21.15.32:0/394899967 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5fb806b820 0x7f5fb8070d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:42.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.651+0000 7f5fbf909640 1 --2- 172.21.15.32:0/394899967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fb815c6f0 0x7f5fb806c7b0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:42.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.651+0000 7f5fbf909640 1 -- 172.21.15.32:0/394899967 >> 172.21.15.32:0/394899967 conn(0x7f5fb80d0730 msgr2=0x7f5fb80d0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:42.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.651+0000 7f5fbf909640 1 -- 172.21.15.32:0/394899967 shutdown_connections 2024-04-24T23:03:42.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.651+0000 7f5fbf909640 1 -- 172.21.15.32:0/394899967 wait complete. 2024-04-24T23:03:42.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.652+0000 7f5fbf909640 1 Processor -- start 2024-04-24T23:03:42.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.652+0000 7f5fbf909640 1 -- start start 2024-04-24T23:03:42.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.653+0000 7f5fbf909640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5fb806b820 0x7f5fb81e4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:42.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.653+0000 7f5fbf909640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fb806d2e0 0x7f5fb81464d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:42.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.653+0000 7f5fbf909640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5fb815c6f0 0x7f5fb81f5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:42.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.653+0000 7f5fbf909640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb80d04b0 con 0x7f5fb806d2e0 2024-04-24T23:03:42.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.653+0000 7f5fbf909640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb815cad0 con 0x7f5fb806b820 2024-04-24T23:03:42.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.653+0000 7f5fbf909640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f5fb806ccf0 con 0x7f5fb815c6f0 2024-04-24T23:03:42.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.653+0000 7f5fbcea4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fb806d2e0 0x7f5fb81464d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:42.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.653+0000 7f5fbcea4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fb806d2e0 0x7f5fb81464d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53080/0 (socket says 172.21.15.32:53080) 2024-04-24T23:03:42.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.653+0000 7f5fbcea4640 1 -- 172.21.15.32:0/4245518914 learned_addr learned my addr 172.21.15.32:0/4245518914 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:42.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.653+0000 7f5fbd6a5640 1 --2- 172.21.15.32:0/4245518914 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5fb806b820 0x7f5fb81e4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:42.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.653+0000 7f5fbdea6640 1 --2- 172.21.15.32:0/4245518914 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5fb815c6f0 0x7f5fb81f5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:42.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.653+0000 7f5fbcea4640 1 -- 172.21.15.32:0/4245518914 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5fb815c6f0 msgr2=0x7f5fb81f5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:42.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.653+0000 7f5fbcea4640 1 --2- 172.21.15.32:0/4245518914 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5fb815c6f0 0x7f5fb81f5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:42.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.653+0000 7f5fbcea4640 1 -- 172.21.15.32:0/4245518914 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5fb806b820 msgr2=0x7f5fb81e4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:42.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.653+0000 7f5fbcea4640 1 --2- 172.21.15.32:0/4245518914 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5fb806b820 0x7f5fb81e4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:42.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.653+0000 7f5fbcea4640 1 -- 172.21.15.32:0/4245518914 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fb8143a70 con 0x7f5fb806d2e0 2024-04-24T23:03:42.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.653+0000 7f5fbcea4640 1 --2- 172.21.15.32:0/4245518914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fb806d2e0 0x7f5fb81464d0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8004640 tx=0x7f5fa800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:42.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.653+0000 7f5fadffb640 1 -- 172.21.15.32:0/4245518914 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa8011790 con 0x7f5fb806d2e0 2024-04-24T23:03:42.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.654+0000 7f5fadffb640 1 -- 172.21.15.32:0/4245518914 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fa8017070 con 0x7f5fb806d2e0 2024-04-24T23:03:42.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.654+0000 7f5fbf909640 1 -- 172.21.15.32:0/4245518914 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fb81e50a0 con 0x7f5fb806d2e0 2024-04-24T23:03:42.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.654+0000 7f5fadffb640 1 -- 172.21.15.32:0/4245518914 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa8011bb0 con 0x7f5fb806d2e0 2024-04-24T23:03:42.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.654+0000 7f5fbf909640 1 -- 172.21.15.32:0/4245518914 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fb81536e0 con 0x7f5fb806d2e0 2024-04-24T23:03:42.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.655+0000 7f5fbf909640 1 -- 172.21.15.32:0/4245518914 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fb8068430 con 0x7f5fb806d2e0 2024-04-24T23:03:42.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.655+0000 7f5fadffb640 1 -- 172.21.15.32:0/4245518914 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5fa8011d50 con 0x7f5fb806d2e0 2024-04-24T23:03:42.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.655+0000 7f5fadffb640 1 --2- 172.21.15.32:0/4245518914 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5f9c06be30 0x7f5f9c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:42.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.656+0000 7f5fadffb640 1 -- 172.21.15.32:0/4245518914 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f5fa808ac00 con 0x7f5fb806d2e0 2024-04-24T23:03:42.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.656+0000 7f5fadffb640 1 -- 172.21.15.32:0/4245518914 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=91}) -- 0x7f5f9c073960 con 0x7f5fb806d2e0 2024-04-24T23:03:42.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.656+0000 7f5fbd6a5640 1 --2- 172.21.15.32:0/4245518914 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5f9c06be30 0x7f5f9c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:42.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.656+0000 7f5fbd6a5640 1 --2- 172.21.15.32:0/4245518914 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5f9c06be30 0x7f5f9c06e2f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4034000 tx=0x7f5fb402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:42.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.660+0000 7f5fadffb640 1 -- 172.21.15.32:0/4245518914 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5fa8091080 con 0x7f5fb806d2e0 2024-04-24T23:03:42.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.842+0000 7f5fbf909640 1 -- 172.21.15.32:0/4245518914 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5fb815e220 con 0x7f5fb806d2e0 2024-04-24T23:03:42.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.842+0000 7f5fadffb640 1 -- 172.21.15.32:0/4245518914 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f5fb8143a70 con 0x7f5fb806d2e0 2024-04-24T23:03:42.844 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-24T23:03:42.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.844+0000 7f5fbf909640 1 -- 172.21.15.32:0/4245518914 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5f9c06be30 msgr2=0x7f5f9c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:42.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.844+0000 7f5fbf909640 1 --2- 172.21.15.32:0/4245518914 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5f9c06be30 0x7f5f9c06e2f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4034000 tx=0x7f5fb402e000 comp rx=0 tx=0).stop 2024-04-24T23:03:42.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.844+0000 7f5fbf909640 1 -- 172.21.15.32:0/4245518914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fb806d2e0 msgr2=0x7f5fb81464d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:42.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.844+0000 7f5fbf909640 1 --2- 172.21.15.32:0/4245518914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fb806d2e0 0x7f5fb81464d0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8004640 tx=0x7f5fa800c8e0 comp rx=0 tx=0).stop 2024-04-24T23:03:42.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.844+0000 7f5fbf909640 1 -- 172.21.15.32:0/4245518914 shutdown_connections 2024-04-24T23:03:42.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.844+0000 7f5fbf909640 1 --2- 172.21.15.32:0/4245518914 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5f9c06be30 0x7f5f9c06e2f0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:42.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.844+0000 7f5fbf909640 1 --2- 172.21.15.32:0/4245518914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fb806d2e0 0x7f5fb81464d0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:42.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.844+0000 7f5fbf909640 1 --2- 172.21.15.32:0/4245518914 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5fb815c6f0 0x7f5fb81f5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:42.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.844+0000 7f5fbf909640 1 --2- 172.21.15.32:0/4245518914 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5fb806b820 0x7f5fb81e4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:42.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.844+0000 7f5fbf909640 1 -- 172.21.15.32:0/4245518914 >> 172.21.15.32:0/4245518914 conn(0x7f5fb80d0730 msgr2=0x7f5fb815b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:42.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.845+0000 7f5fbf909640 1 -- 172.21.15.32:0/4245518914 shutdown_connections 2024-04-24T23:03:42.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.845+0000 7f5fbf909640 1 -- 172.21.15.32:0/4245518914 wait complete. 2024-04-24T23:03:42.857 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:03:42.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.959+0000 7f7fc9b96640 1 Processor -- start 2024-04-24T23:03:42.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.959+0000 7f7fc9b96640 1 -- start start 2024-04-24T23:03:42.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.959+0000 7f7fc9b96640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7fc406c5f0 0x7f7fc406c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:42.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.959+0000 7f7fc9b96640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7fc40b44d0 0x7f7fc40b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:42.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.960+0000 7f7fc9b96640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fc40b6430 con 0x7f7fc40af890 2024-04-24T23:03:42.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.960+0000 7f7fc9b96640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f7fc4156f80 con 0x7f7fc40b44d0 2024-04-24T23:03:42.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.960+0000 7f7fc9b96640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f7fc40b6150 con 0x7f7fc406c5f0 2024-04-24T23:03:42.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.960+0000 7f7fc2ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7fc40b44d0 0x7f7fc40b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:42.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.960+0000 7f7fc2ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7fc40b44d0 0x7f7fc40b5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53092/0 (socket says 172.21.15.32:53092) 2024-04-24T23:03:42.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.960+0000 7f7fc2ffd640 1 -- 172.21.15.32:0/1475039425 learned_addr learned my addr 172.21.15.32:0/1475039425 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:42.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.960+0000 7f7fc2ffd640 1 -- 172.21.15.32:0/1475039425 >> v2:172.21.15.121:3300/0 conn(0x7f7fc406c5f0 msgr2=0x7f7fc406c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:42.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.960+0000 7f7fc37fe640 1 --2- 172.21.15.32:0/1475039425 >> v2:172.21.15.121:3300/0 conn(0x7f7fc406c5f0 0x7f7fc406c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:42.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.960+0000 7f7fc2ffd640 1 --2- 172.21.15.32:0/1475039425 >> v2:172.21.15.121:3300/0 conn(0x7f7fc406c5f0 0x7f7fc406c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:42.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.960+0000 7f7fc2ffd640 1 -- 172.21.15.32:0/1475039425 >> v1:172.21.15.121:6789/0 conn(0x7f7fc40af890 legacy=0x7f7fc40cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:42.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.960+0000 7f7fc2ffd640 1 -- 172.21.15.32:0/1475039425 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fc40b39f0 con 0x7f7fc40b44d0 2024-04-24T23:03:42.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.960+0000 7f7fc37fe640 1 --2- 172.21.15.32:0/1475039425 >> v2:172.21.15.121:3300/0 conn(0x7f7fc406c5f0 0x7f7fc406c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:03:42.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.961+0000 7f7fc2ffd640 1 --2- 172.21.15.32:0/1475039425 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7fc40b44d0 0x7f7fc40b5890 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f7fc406c1f0 tx=0x7f7fb402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83589d5b50255f93 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:42.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.961+0000 7f7fc27fc640 1 -- 172.21.15.32:0/1475039425 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb4037070 con 0x7f7fc40b44d0 2024-04-24T23:03:42.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.961+0000 7f7fc27fc640 1 -- 172.21.15.32:0/1475039425 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fb4032440 con 0x7f7fc40b44d0 2024-04-24T23:03:42.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.961+0000 7f7fc27fc640 1 -- 172.21.15.32:0/1475039425 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb4032810 con 0x7f7fc40b44d0 2024-04-24T23:03:42.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.961+0000 7f7fc9b96640 1 -- 172.21.15.32:0/1475039425 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7fc40b44d0 msgr2=0x7f7fc40b5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:42.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.961+0000 7f7fc9b96640 1 --2- 172.21.15.32:0/1475039425 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7fc40b44d0 0x7f7fc40b5890 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f7fc406c1f0 tx=0x7f7fb402a230 comp rx=0 tx=0).stop 2024-04-24T23:03:42.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.961+0000 7f7fc9b96640 1 -- 172.21.15.32:0/1475039425 shutdown_connections 2024-04-24T23:03:42.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.961+0000 7f7fc9b96640 1 --2- 172.21.15.32:0/1475039425 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7fc40b44d0 0x7f7fc40b5890 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:42.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.961+0000 7f7fc9b96640 1 --2- 172.21.15.32:0/1475039425 >> v2:172.21.15.121:3300/0 conn(0x7f7fc406c5f0 0x7f7fc406c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:42.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.961+0000 7f7fc9b96640 1 -- 172.21.15.32:0/1475039425 >> 172.21.15.32:0/1475039425 conn(0x7f7fc406b110 msgr2=0x7f7fc406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:42.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.962+0000 7f7fc9b96640 1 -- 172.21.15.32:0/1475039425 shutdown_connections 2024-04-24T23:03:42.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.962+0000 7f7fc9b96640 1 -- 172.21.15.32:0/1475039425 wait complete. 2024-04-24T23:03:42.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.963+0000 7f7fc9b96640 1 Processor -- start 2024-04-24T23:03:42.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.963+0000 7f7fc9b96640 1 -- start start 2024-04-24T23:03:42.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.963+0000 7f7fc9b96640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7fc406c5f0 0x7f7fc4145c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:42.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.963+0000 7f7fc9b96640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7fc40af890 0x7f7fc40b1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:42.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.963+0000 7f7fc9b96640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7fc40b44d0 0x7f7fc40b1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:42.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.964+0000 7f7fc9b96640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc406cf10 con 0x7f7fc40b44d0 2024-04-24T23:03:42.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.964+0000 7f7fc9b96640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc40b3fb0 con 0x7f7fc406c5f0 2024-04-24T23:03:42.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.964+0000 7f7fc9b96640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f7fc40b5dd0 con 0x7f7fc40af890 2024-04-24T23:03:42.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.964+0000 7f7fc3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7fc40b44d0 0x7f7fc40b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:42.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.964+0000 7f7fc3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7fc40b44d0 0x7f7fc40b1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53102/0 (socket says 172.21.15.32:53102) 2024-04-24T23:03:42.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.964+0000 7f7fc3fff640 1 -- 172.21.15.32:0/1159160165 learned_addr learned my addr 172.21.15.32:0/1159160165 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:42.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.964+0000 7f7fc37fe640 1 --2- 172.21.15.32:0/1159160165 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7fc406c5f0 0x7f7fc4145c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:42.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.964+0000 7f7fc2ffd640 1 --2- 172.21.15.32:0/1159160165 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7fc40af890 0x7f7fc40b1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:42.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.964+0000 7f7fc3fff640 1 -- 172.21.15.32:0/1159160165 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7fc40af890 msgr2=0x7f7fc40b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:42.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.964+0000 7f7fc3fff640 1 --2- 172.21.15.32:0/1159160165 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7fc40af890 0x7f7fc40b1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:42.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.964+0000 7f7fc3fff640 1 -- 172.21.15.32:0/1159160165 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7fc406c5f0 msgr2=0x7f7fc4145c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:42.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.964+0000 7f7fc3fff640 1 --2- 172.21.15.32:0/1159160165 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7fc406c5f0 0x7f7fc4145c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:42.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.964+0000 7f7fc3fff640 1 -- 172.21.15.32:0/1159160165 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fc4157e90 con 0x7f7fc40b44d0 2024-04-24T23:03:42.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.964+0000 7f7fc3fff640 1 --2- 172.21.15.32:0/1159160165 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7fc40b44d0 0x7f7fc40b1640 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc00b990 tx=0x7f7fbc00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:42.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.964+0000 7f7fbbfff640 1 -- 172.21.15.32:0/1159160165 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc019070 con 0x7f7fc40b44d0 2024-04-24T23:03:42.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.964+0000 7f7fbbfff640 1 -- 172.21.15.32:0/1159160165 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fbc00cba0 con 0x7f7fc40b44d0 2024-04-24T23:03:42.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.964+0000 7f7fc9b96640 1 -- 172.21.15.32:0/1159160165 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fc415a780 con 0x7f7fc40b44d0 2024-04-24T23:03:42.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.964+0000 7f7fbbfff640 1 -- 172.21.15.32:0/1159160165 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc014440 con 0x7f7fc40b44d0 2024-04-24T23:03:42.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.964+0000 7f7fc9b96640 1 -- 172.21.15.32:0/1159160165 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fc4146190 con 0x7f7fc40b44d0 2024-04-24T23:03:42.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.965+0000 7f7fc9b96640 1 -- 172.21.15.32:0/1159160165 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fc4068430 con 0x7f7fc40b44d0 2024-04-24T23:03:42.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.966+0000 7f7fbbfff640 1 -- 172.21.15.32:0/1159160165 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7fbc01c070 con 0x7f7fc40b44d0 2024-04-24T23:03:42.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.966+0000 7f7fbbfff640 1 --2- 172.21.15.32:0/1159160165 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7fa406be30 0x7f7fa406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:42.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.966+0000 7f7fbbfff640 1 -- 172.21.15.32:0/1159160165 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f7fbc08ac00 con 0x7f7fc40b44d0 2024-04-24T23:03:42.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.966+0000 7f7fbbfff640 1 -- 172.21.15.32:0/1159160165 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=91}) -- 0x7f7fa4073960 con 0x7f7fc40b44d0 2024-04-24T23:03:42.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.966+0000 7f7fc37fe640 1 --2- 172.21.15.32:0/1159160165 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7fa406be30 0x7f7fa406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:42.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.967+0000 7f7fc37fe640 1 --2- 172.21.15.32:0/1159160165 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7fa406be30 0x7f7fa406e2f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7fac00a950 tx=0x7f7fac008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:42.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:42.971+0000 7f7fbbfff640 1 -- 172.21.15.32:0/1159160165 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7fbc091080 con 0x7f7fc40b44d0 2024-04-24T23:03:43.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.152+0000 7f7fc9b96640 1 -- 172.21.15.32:0/1159160165 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7fc406b7d0 con 0x7f7fc40b44d0 2024-04-24T23:03:43.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.152+0000 7f7fbbfff640 1 -- 172.21.15.32:0/1159160165 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f7fbc0595b0 con 0x7f7fc40b44d0 2024-04-24T23:03:43.154 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-24T23:03:43.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.155+0000 7f7fc9b96640 1 -- 172.21.15.32:0/1159160165 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7fa406be30 msgr2=0x7f7fa406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:43.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.155+0000 7f7fc9b96640 1 --2- 172.21.15.32:0/1159160165 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7fa406be30 0x7f7fa406e2f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7fac00a950 tx=0x7f7fac008040 comp rx=0 tx=0).stop 2024-04-24T23:03:43.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.155+0000 7f7fc9b96640 1 -- 172.21.15.32:0/1159160165 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7fc40b44d0 msgr2=0x7f7fc40b1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:43.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.155+0000 7f7fc9b96640 1 --2- 172.21.15.32:0/1159160165 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7fc40b44d0 0x7f7fc40b1640 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc00b990 tx=0x7f7fbc00be50 comp rx=0 tx=0).stop 2024-04-24T23:03:43.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.155+0000 7f7fc9b96640 1 -- 172.21.15.32:0/1159160165 shutdown_connections 2024-04-24T23:03:43.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.155+0000 7f7fc9b96640 1 --2- 172.21.15.32:0/1159160165 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7fa406be30 0x7f7fa406e2f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:43.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.155+0000 7f7fc9b96640 1 --2- 172.21.15.32:0/1159160165 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7fc40b44d0 0x7f7fc40b1640 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:43.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.155+0000 7f7fc9b96640 1 --2- 172.21.15.32:0/1159160165 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7fc40af890 0x7f7fc40b1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:43.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.155+0000 7f7fc9b96640 1 --2- 172.21.15.32:0/1159160165 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7fc406c5f0 0x7f7fc4145c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:43.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.155+0000 7f7fc9b96640 1 -- 172.21.15.32:0/1159160165 >> 172.21.15.32:0/1159160165 conn(0x7f7fc406b110 msgr2=0x7f7fc4071750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:43.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.155+0000 7f7fc9b96640 1 -- 172.21.15.32:0/1159160165 shutdown_connections 2024-04-24T23:03:43.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.155+0000 7f7fc9b96640 1 -- 172.21.15.32:0/1159160165 wait complete. 2024-04-24T23:03:43.167 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T23:03:43.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.268+0000 7f3de12ba640 1 Processor -- start 2024-04-24T23:03:43.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.268+0000 7f3de12ba640 1 -- start start 2024-04-24T23:03:43.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.268+0000 7f3de12ba640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3ddc06cd00 0x7f3ddc0ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:43.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.268+0000 7f3de12ba640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3ddc15cb00 0x7f3ddc0ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:43.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.268+0000 7f3de12ba640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ddc0b4520 con 0x7f3ddc15b380 2024-04-24T23:03:43.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.268+0000 7f3de12ba640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f3ddc0b4800 con 0x7f3ddc15cb00 2024-04-24T23:03:43.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.268+0000 7f3de12ba640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f3ddc0b4ae0 con 0x7f3ddc06cd00 2024-04-24T23:03:43.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.269+0000 7f3ddb7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3ddc15cb00 0x7f3ddc0ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:43.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.269+0000 7f3ddaffd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3ddc15b380 0x7f3ddc06c5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54928/0 (socket says 172.21.15.32:54928) 2024-04-24T23:03:43.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.269+0000 7f3ddaffd640 1 -- 172.21.15.32:0/800918752 learned_addr learned my addr 172.21.15.32:0/800918752 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:03:43.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.269+0000 7f3ddb7fe640 1 -- 172.21.15.32:0/800918752 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3ddc06cd00 msgr2=0x7f3ddc0ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:43.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.269+0000 7f3dda7fc640 1 --2- 172.21.15.32:0/800918752 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3ddc06cd00 0x7f3ddc0ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:43.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.269+0000 7f3ddb7fe640 1 --2- 172.21.15.32:0/800918752 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3ddc06cd00 0x7f3ddc0ce430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:43.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.269+0000 7f3ddb7fe640 1 -- 172.21.15.32:0/800918752 >> v1:172.21.15.32:6789/0 conn(0x7f3ddc15b380 legacy=0x7f3ddc06c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:43.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.269+0000 7f3dd9ffb640 1 -- 172.21.15.32:0/800918752 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3781618200 0 0) 0x7f3ddc0b4520 con 0x7f3ddc15b380 2024-04-24T23:03:43.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.269+0000 7f3ddb7fe640 1 -- 172.21.15.32:0/800918752 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ddc0b4dc0 con 0x7f3ddc15cb00 2024-04-24T23:03:43.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.270+0000 7f3ddb7fe640 1 --2- 172.21.15.32:0/800918752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ddc15cb00 0x7f3ddc0ce970 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc009d90 tx=0x7f3dcc02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64f0a1f0506e4f28 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:43.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.270+0000 7f3dd9ffb640 1 -- 172.21.15.32:0/800918752 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc031600 con 0x7f3ddc15cb00 2024-04-24T23:03:43.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.270+0000 7f3dd9ffb640 1 -- 172.21.15.32:0/800918752 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dcc0317a0 con 0x7f3ddc15cb00 2024-04-24T23:03:43.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.270+0000 7f3dd9ffb640 1 -- 172.21.15.32:0/800918752 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc031b70 con 0x7f3ddc15cb00 2024-04-24T23:03:43.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.270+0000 7f3de12ba640 1 -- 172.21.15.32:0/800918752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ddc15cb00 msgr2=0x7f3ddc0ce970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:43.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.270+0000 7f3de12ba640 1 --2- 172.21.15.32:0/800918752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ddc15cb00 0x7f3ddc0ce970 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc009d90 tx=0x7f3dcc02a5d0 comp rx=0 tx=0).stop 2024-04-24T23:03:43.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.270+0000 7f3de12ba640 1 -- 172.21.15.32:0/800918752 shutdown_connections 2024-04-24T23:03:43.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.270+0000 7f3de12ba640 1 --2- 172.21.15.32:0/800918752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ddc15cb00 0x7f3ddc0ce970 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:43.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.270+0000 7f3de12ba640 1 --2- 172.21.15.32:0/800918752 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3ddc06cd00 0x7f3ddc0ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:43.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.270+0000 7f3de12ba640 1 -- 172.21.15.32:0/800918752 >> 172.21.15.32:0/800918752 conn(0x7f3ddc06b110 msgr2=0x7f3ddc06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:43.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.270+0000 7f3de12ba640 1 -- 172.21.15.32:0/800918752 shutdown_connections 2024-04-24T23:03:43.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.270+0000 7f3de12ba640 1 -- 172.21.15.32:0/800918752 wait complete. 2024-04-24T23:03:43.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.272+0000 7f3de12ba640 1 Processor -- start 2024-04-24T23:03:43.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.272+0000 7f3de12ba640 1 -- start start 2024-04-24T23:03:43.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.272+0000 7f3de12ba640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ddc06cd00 0x7f3ddc15bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:43.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.272+0000 7f3de12ba640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ddc15b380 0x7f3ddc1e4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:43.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.272+0000 7f3de12ba640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3ddc15cb00 0x7f3ddc070a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:43.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.272+0000 7f3de12ba640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3ddc15b760 con 0x7f3ddc06cd00 2024-04-24T23:03:43.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.272+0000 7f3de12ba640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3ddc0ceeb0 con 0x7f3ddc15b380 2024-04-24T23:03:43.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.272+0000 7f3de12ba640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f3ddc0b55c0 con 0x7f3ddc15cb00 2024-04-24T23:03:43.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.273+0000 7f3ddaffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ddc06cd00 0x7f3ddc15bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:43.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.273+0000 7f3ddaffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ddc06cd00 0x7f3ddc15bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53114/0 (socket says 172.21.15.32:53114) 2024-04-24T23:03:43.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.273+0000 7f3ddaffd640 1 -- 172.21.15.32:0/3646510689 learned_addr learned my addr 172.21.15.32:0/3646510689 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:43.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.273+0000 7f3dda7fc640 1 --2- 172.21.15.32:0/3646510689 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ddc15b380 0x7f3ddc1e4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:43.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.273+0000 7f3ddb7fe640 1 --2- 172.21.15.32:0/3646510689 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3ddc15cb00 0x7f3ddc070a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:43.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.273+0000 7f3ddaffd640 1 -- 172.21.15.32:0/3646510689 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3ddc15cb00 msgr2=0x7f3ddc070a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:43.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.273+0000 7f3ddaffd640 1 --2- 172.21.15.32:0/3646510689 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3ddc15cb00 0x7f3ddc070a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:43.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.273+0000 7f3ddaffd640 1 -- 172.21.15.32:0/3646510689 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ddc15b380 msgr2=0x7f3ddc1e4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:43.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.273+0000 7f3ddaffd640 1 --2- 172.21.15.32:0/3646510689 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ddc15b380 0x7f3ddc1e4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:43.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.273+0000 7f3ddaffd640 1 -- 172.21.15.32:0/3646510689 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ddc155ac0 con 0x7f3ddc06cd00 2024-04-24T23:03:43.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.273+0000 7f3dda7fc640 1 --2- 172.21.15.32:0/3646510689 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ddc15b380 0x7f3ddc1e4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:03:43.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.273+0000 7f3ddaffd640 1 --2- 172.21.15.32:0/3646510689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ddc06cd00 0x7f3ddc15bb50 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f3dd400c4d0 tx=0x7f3dd400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:43.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.273+0000 7f3dd37fe640 1 -- 172.21.15.32:0/3646510689 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd4018070 con 0x7f3ddc06cd00 2024-04-24T23:03:43.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.273+0000 7f3dd37fe640 1 -- 172.21.15.32:0/3646510689 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dd4007500 con 0x7f3ddc06cd00 2024-04-24T23:03:43.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.273+0000 7f3de12ba640 1 -- 172.21.15.32:0/3646510689 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ddc155da0 con 0x7f3ddc06cd00 2024-04-24T23:03:43.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.273+0000 7f3dd37fe640 1 -- 172.21.15.32:0/3646510689 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd4004a20 con 0x7f3ddc06cd00 2024-04-24T23:03:43.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.273+0000 7f3de12ba640 1 -- 172.21.15.32:0/3646510689 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ddc156360 con 0x7f3ddc06cd00 2024-04-24T23:03:43.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.274+0000 7f3de12ba640 1 -- 172.21.15.32:0/3646510689 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ddc068430 con 0x7f3ddc06cd00 2024-04-24T23:03:43.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.275+0000 7f3dd37fe640 1 -- 172.21.15.32:0/3646510689 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3dd401b070 con 0x7f3ddc06cd00 2024-04-24T23:03:43.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.275+0000 7f3dd37fe640 1 --2- 172.21.15.32:0/3646510689 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3dbc06be30 0x7f3dbc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:43.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.275+0000 7f3dd37fe640 1 -- 172.21.15.32:0/3646510689 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f3dd408a1f0 con 0x7f3ddc06cd00 2024-04-24T23:03:43.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.275+0000 7f3dda7fc640 1 --2- 172.21.15.32:0/3646510689 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3dbc06be30 0x7f3dbc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:43.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.275+0000 7f3dd37fe640 1 -- 172.21.15.32:0/3646510689 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=91}) -- 0x7f3dbc0739c0 con 0x7f3ddc06cd00 2024-04-24T23:03:43.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.276+0000 7f3dda7fc640 1 --2- 172.21.15.32:0/3646510689 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3dbc06be30 0x7f3dbc06e2f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc1547b0 tx=0x7f3dc4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:43.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.279+0000 7f3dd37fe640 1 -- 172.21.15.32:0/3646510689 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3dd4058ba0 con 0x7f3ddc06cd00 2024-04-24T23:03:43.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.476+0000 7f3de12ba640 1 -- 172.21.15.32:0/3646510689 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3ddc0afd60 con 0x7f3ddc06cd00 2024-04-24T23:03:43.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.476+0000 7f3dd37fe640 1 -- 172.21.15.32:0/3646510689 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v90) ==== 80+0+0 (secure 0 0 0) 0x7f3dd40584a0 con 0x7f3ddc06cd00 2024-04-24T23:03:43.478 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2024-04-24T23:03:43.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.478+0000 7f3de12ba640 1 -- 172.21.15.32:0/3646510689 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3dbc06be30 msgr2=0x7f3dbc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:43.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.478+0000 7f3de12ba640 1 --2- 172.21.15.32:0/3646510689 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3dbc06be30 0x7f3dbc06e2f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc1547b0 tx=0x7f3dc4008040 comp rx=0 tx=0).stop 2024-04-24T23:03:43.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.478+0000 7f3de12ba640 1 -- 172.21.15.32:0/3646510689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ddc06cd00 msgr2=0x7f3ddc15bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:43.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.478+0000 7f3de12ba640 1 --2- 172.21.15.32:0/3646510689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ddc06cd00 0x7f3ddc15bb50 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f3dd400c4d0 tx=0x7f3dd400c990 comp rx=0 tx=0).stop 2024-04-24T23:03:43.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.478+0000 7f3de12ba640 1 -- 172.21.15.32:0/3646510689 shutdown_connections 2024-04-24T23:03:43.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.478+0000 7f3de12ba640 1 --2- 172.21.15.32:0/3646510689 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3dbc06be30 0x7f3dbc06e2f0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:43.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.478+0000 7f3de12ba640 1 --2- 172.21.15.32:0/3646510689 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3ddc15cb00 0x7f3ddc070a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:43.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.478+0000 7f3de12ba640 1 --2- 172.21.15.32:0/3646510689 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ddc15b380 0x7f3ddc1e4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:43.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.478+0000 7f3de12ba640 1 --2- 172.21.15.32:0/3646510689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ddc06cd00 0x7f3ddc15bb50 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:43.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.478+0000 7f3de12ba640 1 -- 172.21.15.32:0/3646510689 >> 172.21.15.32:0/3646510689 conn(0x7f3ddc06b110 msgr2=0x7f3ddc0cf1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:43.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.479+0000 7f3de12ba640 1 -- 172.21.15.32:0/3646510689 shutdown_connections 2024-04-24T23:03:43.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.479+0000 7f3de12ba640 1 -- 172.21.15.32:0/3646510689 wait complete. 2024-04-24T23:03:43.491 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-24T23:03:43.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:03:43.491 DEBUG:teuthology.orchestra.run.smithi032:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-24T23:03:43.520 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2024-04-24T23:03:43.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:03:43.521 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-24T23:03:43.576 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:03:43.576 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:03:43.576 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:03:43.595 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:03:43.595 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:03:43.653 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:03:43.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.752+0000 7fe57c570640 1 Processor -- start 2024-04-24T23:03:43.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.752+0000 7fe57c570640 1 -- start start 2024-04-24T23:03:43.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.753+0000 7fe57c570640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe5740b44d0 0x7fe5740b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:43.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.753+0000 7fe57c570640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5740b6430 con 0x7fe57406c5f0 2024-04-24T23:03:43.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.753+0000 7fe57c570640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe574156f80 con 0x7fe5740af890 2024-04-24T23:03:43.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.753+0000 7fe57c570640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fe5740b6150 con 0x7fe5740b44d0 2024-04-24T23:03:43.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.753+0000 7fe579b0b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe5740b44d0 0x7fe5740b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:43.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.753+0000 7fe579b0b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe5740b44d0 0x7fe5740b5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53122/0 (socket says 172.21.15.32:53122) 2024-04-24T23:03:43.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.753+0000 7fe57a30c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe57406c5f0 0x7fe57406c9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54940/0 (socket says 172.21.15.32:54940) 2024-04-24T23:03:43.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.753+0000 7fe579b0b640 1 -- 172.21.15.32:0/1520959989 learned_addr learned my addr 172.21.15.32:0/1520959989 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:43.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.753+0000 7fe579b0b640 1 -- 172.21.15.32:0/1520959989 >> v1:172.21.15.121:6789/0 conn(0x7fe5740af890 legacy=0x7fe57415cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:43.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.753+0000 7fe579b0b640 1 -- 172.21.15.32:0/1520959989 >> v1:172.21.15.32:6789/0 conn(0x7fe57406c5f0 legacy=0x7fe57406c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:43.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.753+0000 7fe579b0b640 1 -- 172.21.15.32:0/1520959989 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5740b39f0 con 0x7fe5740b44d0 2024-04-24T23:03:43.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.753+0000 7fe57930a640 1 -- 172.21.15.32:0/1520959989 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 467694940 0 0) 0x7fe5740b6430 con 0x7fe57406c5f0 2024-04-24T23:03:43.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.754+0000 7fe579b0b640 1 --2- 172.21.15.32:0/1520959989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5740b44d0 0x7fe5740b5890 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fe564009870 tx=0x7fe56402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4883e9b076b82f80 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:43.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.754+0000 7fe57930a640 1 -- 172.21.15.32:0/1520959989 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe564037070 con 0x7fe5740b44d0 2024-04-24T23:03:43.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.754+0000 7fe57930a640 1 -- 172.21.15.32:0/1520959989 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe564032440 con 0x7fe5740b44d0 2024-04-24T23:03:43.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.754+0000 7fe57930a640 1 -- 172.21.15.32:0/1520959989 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe564032810 con 0x7fe5740b44d0 2024-04-24T23:03:43.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.755+0000 7fe57c570640 1 -- 172.21.15.32:0/1520959989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5740b44d0 msgr2=0x7fe5740b5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:43.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.755+0000 7fe57c570640 1 --2- 172.21.15.32:0/1520959989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5740b44d0 0x7fe5740b5890 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fe564009870 tx=0x7fe56402a2a0 comp rx=0 tx=0).stop 2024-04-24T23:03:43.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.755+0000 7fe57c570640 1 -- 172.21.15.32:0/1520959989 shutdown_connections 2024-04-24T23:03:43.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.755+0000 7fe57c570640 1 --2- 172.21.15.32:0/1520959989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5740b44d0 0x7fe5740b5890 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:43.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.755+0000 7fe57c570640 1 -- 172.21.15.32:0/1520959989 >> 172.21.15.32:0/1520959989 conn(0x7fe57406b110 msgr2=0x7fe57406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:43.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.755+0000 7fe57c570640 1 -- 172.21.15.32:0/1520959989 shutdown_connections 2024-04-24T23:03:43.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.755+0000 7fe57c570640 1 -- 172.21.15.32:0/1520959989 wait complete. 2024-04-24T23:03:43.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.756+0000 7fe57c570640 1 Processor -- start 2024-04-24T23:03:43.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.756+0000 7fe57c570640 1 -- start start 2024-04-24T23:03:43.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.757+0000 7fe57c570640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe57406c5f0 0x7fe57414c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:43.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.757+0000 7fe57c570640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5740af890 0x7fe574158650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:43.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.757+0000 7fe57c570640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5740b44d0 0x7fe5740b0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:43.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.757+0000 7fe57c570640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe5740b5dd0 con 0x7fe5740af890 2024-04-24T23:03:43.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.757+0000 7fe57c570640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe5740b3fb0 con 0x7fe5740b44d0 2024-04-24T23:03:43.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.757+0000 7fe57c570640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe57415d1d0 con 0x7fe57406c5f0 2024-04-24T23:03:43.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.757+0000 7fe579b0b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5740af890 0x7fe574158650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:43.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.757+0000 7fe579b0b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5740af890 0x7fe574158650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53136/0 (socket says 172.21.15.32:53136) 2024-04-24T23:03:43.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.757+0000 7fe579b0b640 1 -- 172.21.15.32:0/573986360 learned_addr learned my addr 172.21.15.32:0/573986360 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:43.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.757+0000 7fe57ab0d640 1 --2- 172.21.15.32:0/573986360 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5740b44d0 0x7fe5740b0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:43.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.757+0000 7fe57a30c640 1 --2- 172.21.15.32:0/573986360 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe57406c5f0 0x7fe57414c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:43.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.757+0000 7fe579b0b640 1 -- 172.21.15.32:0/573986360 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe57406c5f0 msgr2=0x7fe57414c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:43.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.757+0000 7fe579b0b640 1 --2- 172.21.15.32:0/573986360 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe57406c5f0 0x7fe57414c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:43.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.757+0000 7fe579b0b640 1 -- 172.21.15.32:0/573986360 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5740b44d0 msgr2=0x7fe5740b0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:43.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.757+0000 7fe579b0b640 1 --2- 172.21.15.32:0/573986360 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5740b44d0 0x7fe5740b0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:43.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.757+0000 7fe579b0b640 1 -- 172.21.15.32:0/573986360 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe574159130 con 0x7fe5740af890 2024-04-24T23:03:43.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.757+0000 7fe57ab0d640 1 --2- 172.21.15.32:0/573986360 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5740b44d0 0x7fe5740b0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:03:43.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.757+0000 7fe579b0b640 1 --2- 172.21.15.32:0/573986360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5740af890 0x7fe574158650 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fe564009870 tx=0x7fe5640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:43.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.757+0000 7fe56affd640 1 -- 172.21.15.32:0/573986360 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe564041070 con 0x7fe5740af890 2024-04-24T23:03:43.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.757+0000 7fe56affd640 1 -- 172.21.15.32:0/573986360 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe56403c870 con 0x7fe5740af890 2024-04-24T23:03:43.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.757+0000 7fe57c570640 1 -- 172.21.15.32:0/573986360 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe574159410 con 0x7fe5740af890 2024-04-24T23:03:43.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.757+0000 7fe56affd640 1 -- 172.21.15.32:0/573986360 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe564037040 con 0x7fe5740af890 2024-04-24T23:03:43.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.757+0000 7fe57c570640 1 -- 172.21.15.32:0/573986360 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5741599d0 con 0x7fe5740af890 2024-04-24T23:03:43.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.758+0000 7fe57c570640 1 -- 172.21.15.32:0/573986360 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe574068430 con 0x7fe5740af890 2024-04-24T23:03:43.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.759+0000 7fe56affd640 1 -- 172.21.15.32:0/573986360 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe564044070 con 0x7fe5740af890 2024-04-24T23:03:43.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.759+0000 7fe56affd640 1 --2- 172.21.15.32:0/573986360 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe55806be30 0x7fe55806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:43.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.759+0000 7fe56affd640 1 -- 172.21.15.32:0/573986360 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7fe5640abae0 con 0x7fe5740af890 2024-04-24T23:03:43.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.759+0000 7fe56affd640 1 -- 172.21.15.32:0/573986360 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=91}) -- 0x7fe558073960 con 0x7fe5740af890 2024-04-24T23:03:43.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.759+0000 7fe57a30c640 1 --2- 172.21.15.32:0/573986360 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe55806be30 0x7fe55806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:43.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.760+0000 7fe57a30c640 1 --2- 172.21.15.32:0/573986360 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe55806be30 0x7fe55806e2f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe570000f60 tx=0x7fe570007b50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:43.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.764+0000 7fe56affd640 1 -- 172.21.15.32:0/573986360 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe56407a640 con 0x7fe5740af890 2024-04-24T23:03:43.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.950+0000 7fe57c570640 1 -- 172.21.15.32:0/573986360 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe5740b5670 con 0x7fe5740af890 2024-04-24T23:03:43.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.951+0000 7fe56affd640 1 -- 172.21.15.32:0/573986360 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+16709 (secure 0 0 0) 0x7fe564079f40 con 0x7fe5740af890 2024-04-24T23:03:43.952 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:03:43.953 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":90,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:03:16.320059+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:02:58.930222+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-24T23:02:59.607070+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["backfillfull","exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","full","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["backfillfull","exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["backfillfull","exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["backfillfull","exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:03:43.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.953+0000 7fe57c570640 1 -- 172.21.15.32:0/573986360 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe55806be30 msgr2=0x7fe55806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:43.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.953+0000 7fe57c570640 1 --2- 172.21.15.32:0/573986360 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe55806be30 0x7fe55806e2f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe570000f60 tx=0x7fe570007b50 comp rx=0 tx=0).stop 2024-04-24T23:03:43.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.953+0000 7fe57c570640 1 -- 172.21.15.32:0/573986360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5740af890 msgr2=0x7fe574158650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:43.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.953+0000 7fe57c570640 1 --2- 172.21.15.32:0/573986360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5740af890 0x7fe574158650 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fe564009870 tx=0x7fe5640046c0 comp rx=0 tx=0).stop 2024-04-24T23:03:43.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.953+0000 7fe57c570640 1 -- 172.21.15.32:0/573986360 shutdown_connections 2024-04-24T23:03:43.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.953+0000 7fe57c570640 1 --2- 172.21.15.32:0/573986360 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe55806be30 0x7fe55806e2f0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:43.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.953+0000 7fe57c570640 1 --2- 172.21.15.32:0/573986360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5740af890 0x7fe574158650 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:43.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.953+0000 7fe57c570640 1 --2- 172.21.15.32:0/573986360 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5740b44d0 0x7fe5740b0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:43.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.953+0000 7fe57c570640 1 --2- 172.21.15.32:0/573986360 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe57406c5f0 0x7fe57414c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:43.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.953+0000 7fe57c570640 1 -- 172.21.15.32:0/573986360 >> 172.21.15.32:0/573986360 conn(0x7fe57406b110 msgr2=0x7fe5740b4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:43.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.953+0000 7fe57c570640 1 -- 172.21.15.32:0/573986360 shutdown_connections 2024-04-24T23:03:43.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:43.953+0000 7fe57c570640 1 -- 172.21.15.32:0/573986360 wait complete. 2024-04-24T23:03:43.965 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:02:58.930222+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:02:59.607070+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:03:43.966 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:03:44.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.068+0000 7ff66c0dd640 1 Processor -- start 2024-04-24T23:03:44.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.068+0000 7ff66c0dd640 1 -- start start 2024-04-24T23:03:44.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.069+0000 7ff66c0dd640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6640b4320 0x7ff6640b6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:44.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.069+0000 7ff66c0dd640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6640cd910 con 0x7ff66415edd0 2024-04-24T23:03:44.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.069+0000 7ff66c0dd640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6640cdbf0 con 0x7ff6640b5560 2024-04-24T23:03:44.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.069+0000 7ff66c0dd640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff6640cded0 con 0x7ff6640b4320 2024-04-24T23:03:44.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.069+0000 7ff669678640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff66415edd0 0x7ff6640b5940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54954/0 (socket says 172.21.15.32:54954) 2024-04-24T23:03:44.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.069+0000 7ff669678640 1 -- 172.21.15.32:0/365317145 learned_addr learned my addr 172.21.15.32:0/365317145 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:03:44.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.069+0000 7ff668e77640 1 -- 172.21.15.32:0/365317145 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1707030081 0 0) 0x7ff6640cd910 con 0x7ff66415edd0 2024-04-24T23:03:44.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.069+0000 7ff668e77640 1 -- 172.21.15.32:0/365317145 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff650003440 con 0x7ff66415edd0 2024-04-24T23:03:44.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.069+0000 7ff66a67a640 1 --2- 172.21.15.32:0/365317145 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6640b4320 0x7ff6640b6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:44.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.069+0000 7ff668e77640 1 -- 172.21.15.32:0/365317145 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 474893217 0 0) 0x7ff650003440 con 0x7ff66415edd0 2024-04-24T23:03:44.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.069+0000 7ff668e77640 1 -- 172.21.15.32:0/365317145 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6640b4320 msgr2=0x7ff6640b6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:44.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.069+0000 7ff668e77640 1 --2- 172.21.15.32:0/365317145 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6640b4320 0x7ff6640b6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:44.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.069+0000 7ff668e77640 1 -- 172.21.15.32:0/365317145 >> v1:172.21.15.121:6789/0 conn(0x7ff6640b5560 legacy=0x7ff66415e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:44.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.069+0000 7ff668e77640 1 -- 172.21.15.32:0/365317145 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6640ce1b0 con 0x7ff66415edd0 2024-04-24T23:03:44.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.070+0000 7ff66a67a640 1 --2- 172.21.15.32:0/365317145 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6640b4320 0x7ff6640b6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:03:44.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.070+0000 7ff668e77640 1 -- 172.21.15.32:0/365317145 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7ff654001690 con 0x7ff66415edd0 2024-04-24T23:03:44.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.070+0000 7ff668e77640 1 -- 172.21.15.32:0/365317145 >> v1:172.21.15.32:6789/0 conn(0x7ff66415edd0 legacy=0x7ff6640b5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:44.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.070+0000 7ff668e77640 1 --2- 172.21.15.32:0/365317145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff650003ad0 0x7ff650003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:44.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.070+0000 7ff668e77640 1 --2- 172.21.15.32:0/365317145 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff650008850 0x7ff65000acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:44.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.070+0000 7ff668e77640 1 --2- 172.21.15.32:0/365317145 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6640b4320 0x7ff65000d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:44.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.070+0000 7ff668e77640 1 -- 172.21.15.32:0/365317145 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff6640cded0 con 0x7ff650003ad0 2024-04-24T23:03:44.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.070+0000 7ff668e77640 1 -- 172.21.15.32:0/365317145 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff65000dc60 con 0x7ff650008850 2024-04-24T23:03:44.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.070+0000 7ff668e77640 1 -- 172.21.15.32:0/365317145 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff65000de60 con 0x7ff6640b4320 2024-04-24T23:03:44.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.070+0000 7ff669e79640 1 --2- 172.21.15.32:0/365317145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff650003ad0 0x7ff650003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:44.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.070+0000 7ff669678640 1 --2- 172.21.15.32:0/365317145 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff650008850 0x7ff65000acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:44.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.070+0000 7ff66a67a640 1 --2- 172.21.15.32:0/365317145 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6640b4320 0x7ff65000d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:44.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.070+0000 7ff669e79640 1 -- 172.21.15.32:0/365317145 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6640b4320 msgr2=0x7ff65000d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:44.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.070+0000 7ff669e79640 1 --2- 172.21.15.32:0/365317145 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6640b4320 0x7ff65000d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:44.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.070+0000 7ff669e79640 1 -- 172.21.15.32:0/365317145 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff650008850 msgr2=0x7ff65000acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:44.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.070+0000 7ff669e79640 1 --2- 172.21.15.32:0/365317145 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff650008850 0x7ff65000acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:44.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.070+0000 7ff669e79640 1 -- 172.21.15.32:0/365317145 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff650003440 con 0x7ff650003ad0 2024-04-24T23:03:44.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.071+0000 7ff669e79640 1 --2- 172.21.15.32:0/365317145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff650003ad0 0x7ff650003eb0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7ff660001860 tx=0x7ff650003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b59ce065c6445ae server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:44.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.071+0000 7ff668e77640 1 -- 172.21.15.32:0/365317145 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff660037070 con 0x7ff650003ad0 2024-04-24T23:03:44.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.071+0000 7ff668e77640 1 -- 172.21.15.32:0/365317145 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff66002ac30 con 0x7ff650003ad0 2024-04-24T23:03:44.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.071+0000 7ff668e77640 1 -- 172.21.15.32:0/365317145 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff660001430 con 0x7ff650003ad0 2024-04-24T23:03:44.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.071+0000 7ff66c0dd640 1 -- 172.21.15.32:0/365317145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff650003ad0 msgr2=0x7ff650003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:44.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.071+0000 7ff66c0dd640 1 --2- 172.21.15.32:0/365317145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff650003ad0 0x7ff650003eb0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7ff660001860 tx=0x7ff650003730 comp rx=0 tx=0).stop 2024-04-24T23:03:44.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.071+0000 7ff66a67a640 1 -- 172.21.15.32:0/365317145 reap_dead start 2024-04-24T23:03:44.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.072+0000 7ff66c0dd640 1 -- 172.21.15.32:0/365317145 shutdown_connections 2024-04-24T23:03:44.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.072+0000 7ff66c0dd640 1 -- 172.21.15.32:0/365317145 >> 172.21.15.32:0/365317145 conn(0x7ff66406b110 msgr2=0x7ff66406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:44.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.072+0000 7ff66c0dd640 1 -- 172.21.15.32:0/365317145 shutdown_connections 2024-04-24T23:03:44.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.072+0000 7ff66c0dd640 1 -- 172.21.15.32:0/365317145 wait complete. 2024-04-24T23:03:44.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.073+0000 7ff66c0dd640 1 Processor -- start 2024-04-24T23:03:44.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.073+0000 7ff66c0dd640 1 -- start start 2024-04-24T23:03:44.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.073+0000 7ff66c0dd640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff650003ad0 0x7ff664148e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:44.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.073+0000 7ff66c0dd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff664070270 0x7ff6641573c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:44.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.074+0000 7ff66c0dd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6640717f0 0x7ff664071c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:44.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.074+0000 7ff66c0dd640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff6640cd540 con 0x7ff664070270 2024-04-24T23:03:44.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.074+0000 7ff66c0dd640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff6640cd3c0 con 0x7ff6640717f0 2024-04-24T23:03:44.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.074+0000 7ff66c0dd640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff6640cd6c0 con 0x7ff650003ad0 2024-04-24T23:03:44.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.074+0000 7ff669678640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff664070270 0x7ff6641573c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:44.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.074+0000 7ff669678640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff664070270 0x7ff6641573c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53152/0 (socket says 172.21.15.32:53152) 2024-04-24T23:03:44.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.074+0000 7ff669678640 1 -- 172.21.15.32:0/1014944069 learned_addr learned my addr 172.21.15.32:0/1014944069 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:44.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.074+0000 7ff669e79640 1 --2- 172.21.15.32:0/1014944069 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff650003ad0 0x7ff664148e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:44.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.074+0000 7ff66a67a640 1 --2- 172.21.15.32:0/1014944069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6640717f0 0x7ff664071c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:44.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.074+0000 7ff669678640 1 -- 172.21.15.32:0/1014944069 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff650003ad0 msgr2=0x7ff664148e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:44.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.074+0000 7ff669678640 1 --2- 172.21.15.32:0/1014944069 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff650003ad0 0x7ff664148e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:44.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.074+0000 7ff669678640 1 -- 172.21.15.32:0/1014944069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6640717f0 msgr2=0x7ff664071c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:44.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.074+0000 7ff669678640 1 --2- 172.21.15.32:0/1014944069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6640717f0 0x7ff664071c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:44.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.074+0000 7ff669678640 1 -- 172.21.15.32:0/1014944069 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff664070750 con 0x7ff664070270 2024-04-24T23:03:44.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.074+0000 7ff669678640 1 --2- 172.21.15.32:0/1014944069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff664070270 0x7ff6641573c0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7ff6540072c0 tx=0x7ff654007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:44.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.074+0000 7ff65a7fc640 1 -- 172.21.15.32:0/1014944069 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65400f940 con 0x7ff664070270 2024-04-24T23:03:44.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.074+0000 7ff65a7fc640 1 -- 172.21.15.32:0/1014944069 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff654018070 con 0x7ff664070270 2024-04-24T23:03:44.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.074+0000 7ff66c0dd640 1 -- 172.21.15.32:0/1014944069 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff664070a30 con 0x7ff664070270 2024-04-24T23:03:44.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.074+0000 7ff65a7fc640 1 -- 172.21.15.32:0/1014944069 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65400fd20 con 0x7ff664070270 2024-04-24T23:03:44.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.074+0000 7ff66c0dd640 1 -- 172.21.15.32:0/1014944069 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff664070ff0 con 0x7ff664070270 2024-04-24T23:03:44.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.075+0000 7ff66c0dd640 1 -- 172.21.15.32:0/1014944069 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff664068430 con 0x7ff664070270 2024-04-24T23:03:44.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.076+0000 7ff65a7fc640 1 -- 172.21.15.32:0/1014944069 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff65401d460 con 0x7ff664070270 2024-04-24T23:03:44.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.076+0000 7ff65a7fc640 1 --2- 172.21.15.32:0/1014944069 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff64806be60 0x7ff64806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:44.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.076+0000 7ff65a7fc640 1 -- 172.21.15.32:0/1014944069 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7ff65408ae40 con 0x7ff664070270 2024-04-24T23:03:44.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.076+0000 7ff65a7fc640 1 -- 172.21.15.32:0/1014944069 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=91}) -- 0x7ff6480739f0 con 0x7ff664070270 2024-04-24T23:03:44.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.077+0000 7ff669e79640 1 --2- 172.21.15.32:0/1014944069 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff64806be60 0x7ff64806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:44.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.077+0000 7ff669e79640 1 --2- 172.21.15.32:0/1014944069 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff64806be60 0x7ff64806e320 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff660034000 tx=0x7ff660009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:44.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.081+0000 7ff65a7fc640 1 -- 172.21.15.32:0/1014944069 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff6540591a0 con 0x7ff664070270 2024-04-24T23:03:44.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.262+0000 7ff66c0dd640 1 -- 172.21.15.32:0/1014944069 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff6640afd60 con 0x7ff664070270 2024-04-24T23:03:44.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.262+0000 7ff65a7fc640 1 -- 172.21.15.32:0/1014944069 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7ff65405d4a0 con 0x7ff664070270 2024-04-24T23:03:44.264 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T23:03:44.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.264+0000 7ff66c0dd640 1 -- 172.21.15.32:0/1014944069 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff64806be60 msgr2=0x7ff64806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:44.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.264+0000 7ff66c0dd640 1 --2- 172.21.15.32:0/1014944069 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff64806be60 0x7ff64806e320 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff660034000 tx=0x7ff660009030 comp rx=0 tx=0).stop 2024-04-24T23:03:44.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.264+0000 7ff66c0dd640 1 -- 172.21.15.32:0/1014944069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff664070270 msgr2=0x7ff6641573c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:44.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.264+0000 7ff66c0dd640 1 --2- 172.21.15.32:0/1014944069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff664070270 0x7ff6641573c0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7ff6540072c0 tx=0x7ff654007780 comp rx=0 tx=0).stop 2024-04-24T23:03:44.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.264+0000 7ff66c0dd640 1 -- 172.21.15.32:0/1014944069 shutdown_connections 2024-04-24T23:03:44.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.264+0000 7ff66c0dd640 1 --2- 172.21.15.32:0/1014944069 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff64806be60 0x7ff64806e320 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:44.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.264+0000 7ff66c0dd640 1 --2- 172.21.15.32:0/1014944069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff664070270 0x7ff6641573c0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:44.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.264+0000 7ff66c0dd640 1 --2- 172.21.15.32:0/1014944069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6640717f0 0x7ff664071c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:44.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.264+0000 7ff66c0dd640 1 --2- 172.21.15.32:0/1014944069 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff650003ad0 0x7ff664148e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:44.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.264+0000 7ff66c0dd640 1 -- 172.21.15.32:0/1014944069 >> 172.21.15.32:0/1014944069 conn(0x7ff66406b110 msgr2=0x7ff664154480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:44.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.264+0000 7ff66c0dd640 1 -- 172.21.15.32:0/1014944069 shutdown_connections 2024-04-24T23:03:44.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.264+0000 7ff66c0dd640 1 -- 172.21.15.32:0/1014944069 wait complete. 2024-04-24T23:03:44.277 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:03:44.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.424+0000 7f76a3458640 1 Processor -- start 2024-04-24T23:03:44.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.425+0000 7f76a3458640 1 -- start start 2024-04-24T23:03:44.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.426+0000 7f76a3458640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f769c15d3b0 0x7f769c0d8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:44.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.426+0000 7f76a3458640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f769c0d9230 0x7f769c071950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:44.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.426+0000 7f76a3458640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f769c166920 con 0x7f769c071e90 2024-04-24T23:03:44.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.426+0000 7f76a3458640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f769c0d9690 con 0x7f769c0d9230 2024-04-24T23:03:44.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.426+0000 7f76a3458640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f769c167530 con 0x7f769c15d3b0 2024-04-24T23:03:44.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.426+0000 7f76a19f5640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f769c071e90 0x7f769c0d55f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54960/0 (socket says 172.21.15.32:54960) 2024-04-24T23:03:44.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.426+0000 7f76a19f5640 1 -- 172.21.15.32:0/3564648394 learned_addr learned my addr 172.21.15.32:0/3564648394 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:03:44.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.426+0000 7f76a11f4640 1 --2- 172.21.15.32:0/3564648394 >> v2:172.21.15.121:3300/0 conn(0x7f769c15d3b0 0x7f769c0d8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:44.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.426+0000 7f76a09f3640 1 --2- 172.21.15.32:0/3564648394 >> v2:172.21.15.32:3300/0 conn(0x7f769c0d9230 0x7f769c071950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:44.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.427+0000 7f76a09f3640 1 -- 172.21.15.32:0/3564648394 >> v2:172.21.15.121:3300/0 conn(0x7f769c15d3b0 msgr2=0x7f769c0d8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:44.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.427+0000 7f76a09f3640 1 --2- 172.21.15.32:0/3564648394 >> v2:172.21.15.121:3300/0 conn(0x7f769c15d3b0 0x7f769c0d8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:44.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.427+0000 7f76a09f3640 1 -- 172.21.15.32:0/3564648394 >> v1:172.21.15.32:6789/0 conn(0x7f769c071e90 legacy=0x7f769c0d55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:44.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.427+0000 7f76a09f3640 1 -- 172.21.15.32:0/3564648394 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f769c166c00 con 0x7f769c0d9230 2024-04-24T23:03:44.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.428+0000 7f76a09f3640 1 --2- 172.21.15.32:0/3564648394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f769c0d9230 0x7f769c071950 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f769c0d9d90 tx=0x7f768c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8b18c541e244682 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:44.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.428+0000 7f7693fff640 1 -- 172.21.15.32:0/3564648394 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768c037070 con 0x7f769c0d9230 2024-04-24T23:03:44.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.428+0000 7f7693fff640 1 -- 172.21.15.32:0/3564648394 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f768c032440 con 0x7f769c0d9230 2024-04-24T23:03:44.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.428+0000 7f7693fff640 1 -- 172.21.15.32:0/3564648394 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768c032810 con 0x7f769c0d9230 2024-04-24T23:03:44.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.429+0000 7f76a3458640 1 -- 172.21.15.32:0/3564648394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f769c0d9230 msgr2=0x7f769c071950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:44.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.429+0000 7f76a3458640 1 --2- 172.21.15.32:0/3564648394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f769c0d9230 0x7f769c071950 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f769c0d9d90 tx=0x7f768c02a1a0 comp rx=0 tx=0).stop 2024-04-24T23:03:44.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.430+0000 7f76a3458640 1 -- 172.21.15.32:0/3564648394 shutdown_connections 2024-04-24T23:03:44.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.430+0000 7f76a3458640 1 --2- 172.21.15.32:0/3564648394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f769c0d9230 0x7f769c071950 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:44.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.430+0000 7f76a3458640 1 --2- 172.21.15.32:0/3564648394 >> v2:172.21.15.121:3300/0 conn(0x7f769c15d3b0 0x7f769c0d8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:44.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.430+0000 7f76a3458640 1 -- 172.21.15.32:0/3564648394 >> 172.21.15.32:0/3564648394 conn(0x7f769c066ea0 msgr2=0x7f769c0672b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:44.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.430+0000 7f76a3458640 1 -- 172.21.15.32:0/3564648394 shutdown_connections 2024-04-24T23:03:44.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.430+0000 7f76a3458640 1 -- 172.21.15.32:0/3564648394 wait complete. 2024-04-24T23:03:44.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.432+0000 7f76a3458640 1 Processor -- start 2024-04-24T23:03:44.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.432+0000 7f76a3458640 1 -- start start 2024-04-24T23:03:44.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.434+0000 7f76a3458640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f769c071e90 0x7f769c06ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:44.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.434+0000 7f76a3458640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f769c0d9230 0x7f769c0ba550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:44.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.434+0000 7f76a3458640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f769c15d3b0 0x7f769c0baa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:44.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.434+0000 7f76a11f4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f769c071e90 0x7f769c06ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:44.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.434+0000 7f76a11f4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f769c071e90 0x7f769c06ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53168/0 (socket says 172.21.15.32:53168) 2024-04-24T23:03:44.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.434+0000 7f76a11f4640 1 -- 172.21.15.32:0/2191512015 learned_addr learned my addr 172.21.15.32:0/2191512015 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:44.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.434+0000 7f76a3458640 1 -- 172.21.15.32:0/2191512015 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f769c1671c0 con 0x7f769c071e90 2024-04-24T23:03:44.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.434+0000 7f76a19f5640 1 --2- 172.21.15.32:0/2191512015 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f769c15d3b0 0x7f769c0baa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:44.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.435+0000 7f76a3458640 1 -- 172.21.15.32:0/2191512015 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f769c15d790 con 0x7f769c15d3b0 2024-04-24T23:03:44.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.435+0000 7f76a3458640 1 -- 172.21.15.32:0/2191512015 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f769c072320 con 0x7f769c0d9230 2024-04-24T23:03:44.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.435+0000 7f76a19f5640 1 -- 172.21.15.32:0/2191512015 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f769c0d9230 msgr2=0x7f769c0ba550 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:03:44.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.436+0000 7f76a09f3640 1 --2- 172.21.15.32:0/2191512015 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f769c0d9230 0x7f769c0ba550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:44.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.436+0000 7f76a19f5640 1 --2- 172.21.15.32:0/2191512015 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f769c0d9230 0x7f769c0ba550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:44.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.436+0000 7f76a19f5640 1 -- 172.21.15.32:0/2191512015 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f769c071e90 msgr2=0x7f769c06ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:44.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.436+0000 7f76a19f5640 1 --2- 172.21.15.32:0/2191512015 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f769c071e90 0x7f769c06ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:44.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.436+0000 7f76a19f5640 1 -- 172.21.15.32:0/2191512015 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f769c06d860 con 0x7f769c15d3b0 2024-04-24T23:03:44.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.436+0000 7f76a11f4640 1 --2- 172.21.15.32:0/2191512015 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f769c071e90 0x7f769c06ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:03:44.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.436+0000 7f76a19f5640 1 --2- 172.21.15.32:0/2191512015 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f769c15d3b0 0x7f769c0baa90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f769800c3c0 tx=0x7f769800c7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:44.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.436+0000 7f7691ffb640 1 -- 172.21.15.32:0/2191512015 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7698017070 con 0x7f769c15d3b0 2024-04-24T23:03:44.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.436+0000 7f7691ffb640 1 -- 172.21.15.32:0/2191512015 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7698005da0 con 0x7f769c15d3b0 2024-04-24T23:03:44.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.436+0000 7f7691ffb640 1 -- 172.21.15.32:0/2191512015 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76980072d0 con 0x7f769c15d3b0 2024-04-24T23:03:44.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.436+0000 7f76a3458640 1 -- 172.21.15.32:0/2191512015 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f769c06db40 con 0x7f769c15d3b0 2024-04-24T23:03:44.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.437+0000 7f76a3458640 1 -- 172.21.15.32:0/2191512015 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f769c145420 con 0x7f769c15d3b0 2024-04-24T23:03:44.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.439+0000 7f7691ffb640 1 -- 172.21.15.32:0/2191512015 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f769801a070 con 0x7f769c15d3b0 2024-04-24T23:03:44.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.439+0000 7f76a3458640 1 -- 172.21.15.32:0/2191512015 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f769c068430 con 0x7f769c15d3b0 2024-04-24T23:03:44.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.439+0000 7f7691ffb640 1 --2- 172.21.15.32:0/2191512015 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f768006be30 0x7f768006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:44.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.439+0000 7f7691ffb640 1 -- 172.21.15.32:0/2191512015 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f769808a160 con 0x7f769c15d3b0 2024-04-24T23:03:44.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.440+0000 7f7691ffb640 1 -- 172.21.15.32:0/2191512015 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=91}) -- 0x7f7680073960 con 0x7f769c15d3b0 2024-04-24T23:03:44.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.440+0000 7f76a11f4640 1 --2- 172.21.15.32:0/2191512015 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f768006be30 0x7f768006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:44.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.441+0000 7f76a11f4640 1 --2- 172.21.15.32:0/2191512015 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f768006be30 0x7f768006e2f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f76940094c0 tx=0x7f769400b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:44.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.449+0000 7f7691ffb640 1 -- 172.21.15.32:0/2191512015 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7698058420 con 0x7f769c15d3b0 2024-04-24T23:03:44.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.636+0000 7f76a3458640 1 -- 172.21.15.32:0/2191512015 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f769c0689c0 con 0x7f769c15d3b0 2024-04-24T23:03:44.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.637+0000 7f7691ffb640 1 -- 172.21.15.32:0/2191512015 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f769805c720 con 0x7f769c15d3b0 2024-04-24T23:03:44.638 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-24T23:03:44.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.639+0000 7f76a3458640 1 -- 172.21.15.32:0/2191512015 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f768006be30 msgr2=0x7f768006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:44.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.639+0000 7f76a3458640 1 --2- 172.21.15.32:0/2191512015 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f768006be30 0x7f768006e2f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f76940094c0 tx=0x7f769400b040 comp rx=0 tx=0).stop 2024-04-24T23:03:44.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.639+0000 7f76a3458640 1 -- 172.21.15.32:0/2191512015 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f769c15d3b0 msgr2=0x7f769c0baa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:44.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.639+0000 7f76a3458640 1 --2- 172.21.15.32:0/2191512015 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f769c15d3b0 0x7f769c0baa90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f769800c3c0 tx=0x7f769800c7c0 comp rx=0 tx=0).stop 2024-04-24T23:03:44.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.639+0000 7f76a3458640 1 -- 172.21.15.32:0/2191512015 shutdown_connections 2024-04-24T23:03:44.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.639+0000 7f76a3458640 1 --2- 172.21.15.32:0/2191512015 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f768006be30 0x7f768006e2f0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:44.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.639+0000 7f76a3458640 1 --2- 172.21.15.32:0/2191512015 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f769c15d3b0 0x7f769c0baa90 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:44.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.639+0000 7f76a3458640 1 --2- 172.21.15.32:0/2191512015 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f769c0d9230 0x7f769c0ba550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:44.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.639+0000 7f76a3458640 1 --2- 172.21.15.32:0/2191512015 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f769c071e90 0x7f769c06ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:44.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.639+0000 7f76a3458640 1 -- 172.21.15.32:0/2191512015 >> 172.21.15.32:0/2191512015 conn(0x7f769c066ea0 msgr2=0x7f769c0d8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:44.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.639+0000 7f76a3458640 1 -- 172.21.15.32:0/2191512015 shutdown_connections 2024-04-24T23:03:44.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.639+0000 7f76a3458640 1 -- 172.21.15.32:0/2191512015 wait complete. 2024-04-24T23:03:44.651 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:03:44.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.753+0000 7feaa9948640 1 Processor -- start 2024-04-24T23:03:44.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.753+0000 7feaa9948640 1 -- start start 2024-04-24T23:03:44.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.754+0000 7feaa9948640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7feaa41476d0 0x7feaa40ce630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:44.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.754+0000 7feaa9948640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7feaa40cfb70 0x7feaa406b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:44.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.754+0000 7feaa9948640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feaa40afdd0 con 0x7feaa406b650 2024-04-24T23:03:44.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.754+0000 7feaa9948640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7feaa406bac0 con 0x7feaa40cfb70 2024-04-24T23:03:44.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.754+0000 7feaa9948640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7feaa40aedc0 con 0x7feaa41476d0 2024-04-24T23:03:44.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.754+0000 7feaa27fc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7feaa40cfb70 0x7feaa406b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:44.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.754+0000 7feaa37fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7feaa406b650 0x7feaa40cd180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54970/0 (socket says 172.21.15.32:54970) 2024-04-24T23:03:44.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.754+0000 7feaa37fe640 1 -- 172.21.15.32:0/625272079 learned_addr learned my addr 172.21.15.32:0/625272079 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:03:44.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.754+0000 7feaa2ffd640 1 --2- 172.21.15.32:0/625272079 >> v2:172.21.15.121:3300/0 conn(0x7feaa41476d0 0x7feaa40ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:44.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.754+0000 7feaa27fc640 1 -- 172.21.15.32:0/625272079 >> v2:172.21.15.121:3300/0 conn(0x7feaa41476d0 msgr2=0x7feaa40ce630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:44.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.754+0000 7feaa27fc640 1 --2- 172.21.15.32:0/625272079 >> v2:172.21.15.121:3300/0 conn(0x7feaa41476d0 0x7feaa40ce630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:44.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.754+0000 7feaa27fc640 1 -- 172.21.15.32:0/625272079 >> v1:172.21.15.32:6789/0 conn(0x7feaa406b650 legacy=0x7feaa40cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:44.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.754+0000 7feaa27fc640 1 -- 172.21.15.32:0/625272079 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaa40af010 con 0x7feaa40cfb70 2024-04-24T23:03:44.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.754+0000 7feaa2ffd640 1 --2- 172.21.15.32:0/625272079 >> v2:172.21.15.121:3300/0 conn(0x7feaa41476d0 0x7feaa40ce630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:03:44.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.755+0000 7feaa27fc640 1 --2- 172.21.15.32:0/625272079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feaa40cfb70 0x7feaa406b110 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7feaa406bd60 tx=0x7fea8c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa5ad15257618c49 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:44.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.755+0000 7feaa1ffb640 1 -- 172.21.15.32:0/625272079 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8c037070 con 0x7feaa40cfb70 2024-04-24T23:03:44.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.755+0000 7feaa1ffb640 1 -- 172.21.15.32:0/625272079 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea8c032440 con 0x7feaa40cfb70 2024-04-24T23:03:44.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.755+0000 7feaa1ffb640 1 -- 172.21.15.32:0/625272079 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8c02ae60 con 0x7feaa40cfb70 2024-04-24T23:03:44.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.756+0000 7feaa9948640 1 -- 172.21.15.32:0/625272079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feaa40cfb70 msgr2=0x7feaa406b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:44.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.756+0000 7feaa9948640 1 --2- 172.21.15.32:0/625272079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feaa40cfb70 0x7feaa406b110 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7feaa406bd60 tx=0x7fea8c02a110 comp rx=0 tx=0).stop 2024-04-24T23:03:44.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.756+0000 7feaa9948640 1 -- 172.21.15.32:0/625272079 shutdown_connections 2024-04-24T23:03:44.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.756+0000 7feaa9948640 1 --2- 172.21.15.32:0/625272079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feaa40cfb70 0x7feaa406b110 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:44.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.756+0000 7feaa9948640 1 --2- 172.21.15.32:0/625272079 >> v2:172.21.15.121:3300/0 conn(0x7feaa41476d0 0x7feaa40ce630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:44.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.756+0000 7feaa9948640 1 -- 172.21.15.32:0/625272079 >> 172.21.15.32:0/625272079 conn(0x7feaa40d0730 msgr2=0x7feaa40d0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:44.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.756+0000 7feaa9948640 1 -- 172.21.15.32:0/625272079 shutdown_connections 2024-04-24T23:03:44.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.756+0000 7feaa9948640 1 -- 172.21.15.32:0/625272079 wait complete. 2024-04-24T23:03:44.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.757+0000 7feaa9948640 1 Processor -- start 2024-04-24T23:03:44.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.757+0000 7feaa9948640 1 -- start start 2024-04-24T23:03:44.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.758+0000 7feaa9948640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feaa406b650 0x7feaa415f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:44.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.758+0000 7feaa9948640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feaa40cfb70 0x7feaa40b1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:44.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.758+0000 7feaa9948640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feaa41476d0 0x7feaa415e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:44.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.758+0000 7feaa9948640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7feaa40af5d0 con 0x7feaa40cfb70 2024-04-24T23:03:44.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.758+0000 7feaa9948640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7feaa40af940 con 0x7feaa406b650 2024-04-24T23:03:44.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.758+0000 7feaa9948640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7feaa4147ab0 con 0x7feaa41476d0 2024-04-24T23:03:44.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.758+0000 7feaa27fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feaa40cfb70 0x7feaa40b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:44.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.758+0000 7feaa27fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feaa40cfb70 0x7feaa40b1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53190/0 (socket says 172.21.15.32:53190) 2024-04-24T23:03:44.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.758+0000 7feaa27fc640 1 -- 172.21.15.32:0/2349903649 learned_addr learned my addr 172.21.15.32:0/2349903649 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:44.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.758+0000 7feaa37fe640 1 --2- 172.21.15.32:0/2349903649 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feaa41476d0 0x7feaa415e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:44.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.758+0000 7feaa2ffd640 1 --2- 172.21.15.32:0/2349903649 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feaa406b650 0x7feaa415f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:44.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.758+0000 7feaa27fc640 1 -- 172.21.15.32:0/2349903649 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feaa41476d0 msgr2=0x7feaa415e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:44.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.758+0000 7feaa27fc640 1 --2- 172.21.15.32:0/2349903649 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feaa41476d0 0x7feaa415e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:44.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.758+0000 7feaa27fc640 1 -- 172.21.15.32:0/2349903649 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feaa406b650 msgr2=0x7feaa415f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:44.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.758+0000 7feaa27fc640 1 --2- 172.21.15.32:0/2349903649 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feaa406b650 0x7feaa415f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:44.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.758+0000 7feaa27fc640 1 -- 172.21.15.32:0/2349903649 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaa40b1e60 con 0x7feaa40cfb70 2024-04-24T23:03:44.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.759+0000 7feaa27fc640 1 --2- 172.21.15.32:0/2349903649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feaa40cfb70 0x7feaa40b1630 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fea8c000c00 tx=0x7fea8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:44.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.759+0000 7fea9bfff640 1 -- 172.21.15.32:0/2349903649 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8c037040 con 0x7feaa40cfb70 2024-04-24T23:03:44.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.759+0000 7fea9bfff640 1 -- 172.21.15.32:0/2349903649 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea8c042070 con 0x7feaa40cfb70 2024-04-24T23:03:44.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.759+0000 7feaa9948640 1 -- 172.21.15.32:0/2349903649 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaa40b2140 con 0x7feaa40cfb70 2024-04-24T23:03:44.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.759+0000 7fea9bfff640 1 -- 172.21.15.32:0/2349903649 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8c030bd0 con 0x7feaa40cfb70 2024-04-24T23:03:44.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.759+0000 7feaa9948640 1 -- 172.21.15.32:0/2349903649 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaa415f320 con 0x7feaa40cfb70 2024-04-24T23:03:44.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.760+0000 7feaa9948640 1 -- 172.21.15.32:0/2349903649 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaa406bd60 con 0x7feaa40cfb70 2024-04-24T23:03:44.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.760+0000 7fea9bfff640 1 -- 172.21.15.32:0/2349903649 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fea8c047460 con 0x7feaa40cfb70 2024-04-24T23:03:44.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.761+0000 7fea9bfff640 1 --2- 172.21.15.32:0/2349903649 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fea8406be30 0x7fea8406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:44.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.761+0000 7fea9bfff640 1 -- 172.21.15.32:0/2349903649 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7fea8c0abab0 con 0x7feaa40cfb70 2024-04-24T23:03:44.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.761+0000 7fea9bfff640 1 -- 172.21.15.32:0/2349903649 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=91}) -- 0x7fea84073960 con 0x7feaa40cfb70 2024-04-24T23:03:44.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.761+0000 7feaa2ffd640 1 --2- 172.21.15.32:0/2349903649 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fea8406be30 0x7fea8406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:44.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.761+0000 7feaa2ffd640 1 --2- 172.21.15.32:0/2349903649 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fea8406be30 0x7fea8406e2f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fea9c006fd0 tx=0x7fea9c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:44.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.765+0000 7fea9bfff640 1 -- 172.21.15.32:0/2349903649 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7feaa406bd60 con 0x7feaa40cfb70 2024-04-24T23:03:44.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.945+0000 7feaa9948640 1 -- 172.21.15.32:0/2349903649 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7feaa4068430 con 0x7feaa40cfb70 2024-04-24T23:03:44.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.946+0000 7fea9bfff640 1 -- 172.21.15.32:0/2349903649 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7fea8c032e20 con 0x7feaa40cfb70 2024-04-24T23:03:44.947 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-24T23:03:44.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.948+0000 7feaa9948640 1 -- 172.21.15.32:0/2349903649 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fea8406be30 msgr2=0x7fea8406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:44.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.948+0000 7feaa9948640 1 --2- 172.21.15.32:0/2349903649 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fea8406be30 0x7fea8406e2f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fea9c006fd0 tx=0x7fea9c008040 comp rx=0 tx=0).stop 2024-04-24T23:03:44.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.948+0000 7feaa9948640 1 -- 172.21.15.32:0/2349903649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feaa40cfb70 msgr2=0x7feaa40b1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:44.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.948+0000 7feaa9948640 1 --2- 172.21.15.32:0/2349903649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feaa40cfb70 0x7feaa40b1630 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fea8c000c00 tx=0x7fea8c0046c0 comp rx=0 tx=0).stop 2024-04-24T23:03:44.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.948+0000 7feaa9948640 1 -- 172.21.15.32:0/2349903649 shutdown_connections 2024-04-24T23:03:44.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.948+0000 7feaa9948640 1 --2- 172.21.15.32:0/2349903649 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fea8406be30 0x7fea8406e2f0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:44.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.948+0000 7feaa9948640 1 --2- 172.21.15.32:0/2349903649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feaa40cfb70 0x7feaa40b1630 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:44.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.948+0000 7feaa9948640 1 --2- 172.21.15.32:0/2349903649 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feaa41476d0 0x7feaa415e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:44.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.948+0000 7feaa9948640 1 --2- 172.21.15.32:0/2349903649 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feaa406b650 0x7feaa415f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:44.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.948+0000 7feaa9948640 1 -- 172.21.15.32:0/2349903649 >> 172.21.15.32:0/2349903649 conn(0x7feaa40d0730 msgr2=0x7feaa4145dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:44.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.948+0000 7feaa9948640 1 -- 172.21.15.32:0/2349903649 shutdown_connections 2024-04-24T23:03:44.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:44.948+0000 7feaa9948640 1 -- 172.21.15.32:0/2349903649 wait complete. 2024-04-24T23:03:44.960 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T23:03:45.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.060+0000 7f4929128640 1 Processor -- start 2024-04-24T23:03:45.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.060+0000 7f4929128640 1 -- start start 2024-04-24T23:03:45.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.061+0000 7f4929128640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f492406b820 0x7f49240d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:45.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.061+0000 7f4929128640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49240d3170 con 0x7f492415e9b0 2024-04-24T23:03:45.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.061+0000 7f4929128640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49240d3450 con 0x7f49240d4210 2024-04-24T23:03:45.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.061+0000 7f4929128640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f49240d3730 con 0x7f492406b820 2024-04-24T23:03:45.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.061+0000 7f4922d9d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f492415e9b0 0x7f492406b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54976/0 (socket says 172.21.15.32:54976) 2024-04-24T23:03:45.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.061+0000 7f4922d9d640 1 -- 172.21.15.32:0/2585385481 learned_addr learned my addr 172.21.15.32:0/2585385481 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:03:45.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.062+0000 7f492259c640 1 --2- 172.21.15.32:0/2585385481 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f492406b820 0x7f49240d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:45.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.062+0000 7f4921d9b640 1 -- 172.21.15.32:0/2585385481 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3034804860 0 0) 0x7f49240d3170 con 0x7f492415e9b0 2024-04-24T23:03:45.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.062+0000 7f4921d9b640 1 -- 172.21.15.32:0/2585385481 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4910003520 con 0x7f492415e9b0 2024-04-24T23:03:45.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.062+0000 7f4921d9b640 1 -- 172.21.15.32:0/2585385481 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 201140024 0 0) 0x7f4910003520 con 0x7f492415e9b0 2024-04-24T23:03:45.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.062+0000 7f4921d9b640 1 -- 172.21.15.32:0/2585385481 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f492406b820 msgr2=0x7f49240d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:45.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.062+0000 7f4921d9b640 1 --2- 172.21.15.32:0/2585385481 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f492406b820 0x7f49240d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:45.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.062+0000 7f4921d9b640 1 -- 172.21.15.32:0/2585385481 >> v1:172.21.15.121:6789/0 conn(0x7f49240d4210 legacy=0x7f49240b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:45.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.062+0000 7f4921d9b640 1 -- 172.21.15.32:0/2585385481 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49240d3a10 con 0x7f492415e9b0 2024-04-24T23:03:45.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.062+0000 7f4921d9b640 1 -- 172.21.15.32:0/2585385481 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f491c001670 con 0x7f492415e9b0 2024-04-24T23:03:45.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.062+0000 7f4921d9b640 1 -- 172.21.15.32:0/2585385481 >> v1:172.21.15.32:6789/0 conn(0x7f492415e9b0 legacy=0x7f492406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:45.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.062+0000 7f4921d9b640 1 --2- 172.21.15.32:0/2585385481 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49100039e0 0x7f4910003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:45.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.062+0000 7f4921d9b640 1 --2- 172.21.15.32:0/2585385481 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4910004390 0x7f4910004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:45.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.062+0000 7f4921d9b640 1 --2- 172.21.15.32:0/2585385481 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f492406b820 0x7f49100050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:45.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.062+0000 7f4921d9b640 1 -- 172.21.15.32:0/2585385481 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f49240d3730 con 0x7f49100039e0 2024-04-24T23:03:45.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.062+0000 7f4921d9b640 1 -- 172.21.15.32:0/2585385481 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f491000ba20 con 0x7f4910004390 2024-04-24T23:03:45.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.062+0000 7f4921d9b640 1 -- 172.21.15.32:0/2585385481 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f491000bc20 con 0x7f492406b820 2024-04-24T23:03:45.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.062+0000 7f4922d9d640 1 --2- 172.21.15.32:0/2585385481 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49100039e0 0x7f4910003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:45.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.062+0000 7f492259c640 1 --2- 172.21.15.32:0/2585385481 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4910004390 0x7f4910004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:45.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.062+0000 7f492359e640 1 --2- 172.21.15.32:0/2585385481 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f492406b820 0x7f49100050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:45.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.062+0000 7f4922d9d640 1 -- 172.21.15.32:0/2585385481 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f492406b820 msgr2=0x7f49100050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:45.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.062+0000 7f4922d9d640 1 --2- 172.21.15.32:0/2585385481 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f492406b820 0x7f49100050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:45.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.062+0000 7f4922d9d640 1 -- 172.21.15.32:0/2585385481 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4910004390 msgr2=0x7f4910004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:45.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.062+0000 7f4922d9d640 1 --2- 172.21.15.32:0/2585385481 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4910004390 0x7f4910004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:45.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.062+0000 7f4922d9d640 1 -- 172.21.15.32:0/2585385481 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4910003520 con 0x7f49100039e0 2024-04-24T23:03:45.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.062+0000 7f492359e640 1 --2- 172.21.15.32:0/2585385481 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f492406b820 0x7f49100050c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:03:45.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.063+0000 7f4922d9d640 1 --2- 172.21.15.32:0/2585385481 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49100039e0 0x7f4910003dc0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f491c00c210 tx=0x7f4910005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b7460345ce10744 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:45.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.063+0000 7f4921d9b640 1 -- 172.21.15.32:0/2585385481 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f491c007d10 con 0x7f49100039e0 2024-04-24T23:03:45.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.063+0000 7f4921d9b640 1 -- 172.21.15.32:0/2585385481 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f491c005ad0 con 0x7f49100039e0 2024-04-24T23:03:45.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.063+0000 7f4921d9b640 1 -- 172.21.15.32:0/2585385481 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f491c007070 con 0x7f49100039e0 2024-04-24T23:03:45.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.064+0000 7f4929128640 1 -- 172.21.15.32:0/2585385481 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49100039e0 msgr2=0x7f4910003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:45.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.064+0000 7f4929128640 1 --2- 172.21.15.32:0/2585385481 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49100039e0 0x7f4910003dc0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f491c00c210 tx=0x7f4910005830 comp rx=0 tx=0).stop 2024-04-24T23:03:45.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.064+0000 7f492359e640 1 -- 172.21.15.32:0/2585385481 reap_dead start 2024-04-24T23:03:45.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.064+0000 7f4929128640 1 -- 172.21.15.32:0/2585385481 shutdown_connections 2024-04-24T23:03:45.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.064+0000 7f4929128640 1 -- 172.21.15.32:0/2585385481 >> 172.21.15.32:0/2585385481 conn(0x7f49240d11d0 msgr2=0x7f49240d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:45.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.064+0000 7f4929128640 1 -- 172.21.15.32:0/2585385481 shutdown_connections 2024-04-24T23:03:45.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.064+0000 7f4929128640 1 -- 172.21.15.32:0/2585385481 wait complete. 2024-04-24T23:03:45.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.065+0000 7f4929128640 1 Processor -- start 2024-04-24T23:03:45.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.065+0000 7f4929128640 1 -- start start 2024-04-24T23:03:45.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.066+0000 7f4929128640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49100039e0 0x7f49241f4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:45.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.066+0000 7f4929128640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4924152f50 0x7f492415ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:45.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.066+0000 7f4929128640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f49240ce4d0 0x7f49240b8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:45.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.066+0000 7f4929128640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f49240ba0e0 con 0x7f49100039e0 2024-04-24T23:03:45.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.066+0000 7f4929128640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f49240d2400 con 0x7f49240ce4d0 2024-04-24T23:03:45.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.066+0000 7f4929128640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f49240ba260 con 0x7f4924152f50 2024-04-24T23:03:45.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.066+0000 7f4922d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49100039e0 0x7f49241f4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:45.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.066+0000 7f4922d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49100039e0 0x7f49241f4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53198/0 (socket says 172.21.15.32:53198) 2024-04-24T23:03:45.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.066+0000 7f4922d9d640 1 -- 172.21.15.32:0/600609951 learned_addr learned my addr 172.21.15.32:0/600609951 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:45.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.066+0000 7f492259c640 1 --2- 172.21.15.32:0/600609951 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4924152f50 0x7f492415ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:45.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.066+0000 7f492359e640 1 --2- 172.21.15.32:0/600609951 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f49240ce4d0 0x7f49240b8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:45.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.066+0000 7f4922d9d640 1 -- 172.21.15.32:0/600609951 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4924152f50 msgr2=0x7f492415ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:45.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.066+0000 7f4922d9d640 1 --2- 172.21.15.32:0/600609951 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4924152f50 0x7f492415ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:45.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.066+0000 7f4922d9d640 1 -- 172.21.15.32:0/600609951 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f49240ce4d0 msgr2=0x7f49240b8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:45.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.066+0000 7f4922d9d640 1 --2- 172.21.15.32:0/600609951 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f49240ce4d0 0x7f49240b8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:45.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.066+0000 7f4922d9d640 1 -- 172.21.15.32:0/600609951 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49241f4710 con 0x7f49100039e0 2024-04-24T23:03:45.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.066+0000 7f492359e640 1 --2- 172.21.15.32:0/600609951 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f49240ce4d0 0x7f49240b8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:03:45.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.066+0000 7f492259c640 1 --2- 172.21.15.32:0/600609951 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4924152f50 0x7f492415ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:03:45.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.067+0000 7f4922d9d640 1 --2- 172.21.15.32:0/600609951 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49100039e0 0x7f49241f4120 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f491c001970 tx=0x7f49240d3620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:45.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.067+0000 7f491b7fe640 1 -- 172.21.15.32:0/600609951 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f491c040020 con 0x7f49100039e0 2024-04-24T23:03:45.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.067+0000 7f491b7fe640 1 -- 172.21.15.32:0/600609951 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f491c001430 con 0x7f49100039e0 2024-04-24T23:03:45.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.067+0000 7f491b7fe640 1 -- 172.21.15.32:0/600609951 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f491c037040 con 0x7f49100039e0 2024-04-24T23:03:45.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.067+0000 7f4929128640 1 -- 172.21.15.32:0/600609951 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49241f49f0 con 0x7f49100039e0 2024-04-24T23:03:45.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.067+0000 7f4929128640 1 -- 172.21.15.32:0/600609951 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49241f4fb0 con 0x7f49100039e0 2024-04-24T23:03:45.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.068+0000 7f4929128640 1 -- 172.21.15.32:0/600609951 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4924068430 con 0x7f49100039e0 2024-04-24T23:03:45.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.068+0000 7f491b7fe640 1 -- 172.21.15.32:0/600609951 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f491c043020 con 0x7f49100039e0 2024-04-24T23:03:45.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.069+0000 7f491b7fe640 1 --2- 172.21.15.32:0/600609951 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f490406be30 0x7f490406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:45.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.069+0000 7f491b7fe640 1 -- 172.21.15.32:0/600609951 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f491c0aa260 con 0x7f49100039e0 2024-04-24T23:03:45.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.069+0000 7f491b7fe640 1 -- 172.21.15.32:0/600609951 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=91}) -- 0x7f4904073960 con 0x7f49100039e0 2024-04-24T23:03:45.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.069+0000 7f492259c640 1 --2- 172.21.15.32:0/600609951 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f490406be30 0x7f490406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:45.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.069+0000 7f492259c640 1 --2- 172.21.15.32:0/600609951 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f490406be30 0x7f490406e2f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f49240b61a0 tx=0x7f490c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:45.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.073+0000 7f491b7fe640 1 -- 172.21.15.32:0/600609951 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f491c005aa0 con 0x7f49100039e0 2024-04-24T23:03:45.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.267+0000 7f4929128640 1 -- 172.21.15.32:0/600609951 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4924155f00 con 0x7f49100039e0 2024-04-24T23:03:45.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.267+0000 7f491b7fe640 1 -- 172.21.15.32:0/600609951 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v90) ==== 80+0+0 (secure 0 0 0) 0x7f491c078c10 con 0x7f49100039e0 2024-04-24T23:03:45.269 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2024-04-24T23:03:45.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.269+0000 7f4929128640 1 -- 172.21.15.32:0/600609951 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f490406be30 msgr2=0x7f490406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:45.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.269+0000 7f4929128640 1 --2- 172.21.15.32:0/600609951 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f490406be30 0x7f490406e2f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f49240b61a0 tx=0x7f490c008040 comp rx=0 tx=0).stop 2024-04-24T23:03:45.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.269+0000 7f4929128640 1 -- 172.21.15.32:0/600609951 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49100039e0 msgr2=0x7f49241f4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:45.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.269+0000 7f4929128640 1 --2- 172.21.15.32:0/600609951 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49100039e0 0x7f49241f4120 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f491c001970 tx=0x7f49240d3620 comp rx=0 tx=0).stop 2024-04-24T23:03:45.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.269+0000 7f4929128640 1 -- 172.21.15.32:0/600609951 shutdown_connections 2024-04-24T23:03:45.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.269+0000 7f4929128640 1 --2- 172.21.15.32:0/600609951 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f490406be30 0x7f490406e2f0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:45.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.269+0000 7f4929128640 1 --2- 172.21.15.32:0/600609951 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f49240ce4d0 0x7f49240b8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:45.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.269+0000 7f4929128640 1 --2- 172.21.15.32:0/600609951 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4924152f50 0x7f492415ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:45.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.269+0000 7f4929128640 1 --2- 172.21.15.32:0/600609951 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49100039e0 0x7f49241f4120 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:45.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.269+0000 7f4929128640 1 -- 172.21.15.32:0/600609951 >> 172.21.15.32:0/600609951 conn(0x7f49240d11d0 msgr2=0x7f49240b78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:45.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.269+0000 7f4929128640 1 -- 172.21.15.32:0/600609951 shutdown_connections 2024-04-24T23:03:45.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.269+0000 7f4929128640 1 -- 172.21.15.32:0/600609951 wait complete. 2024-04-24T23:03:45.281 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-24T23:03:45.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:03:45.282 DEBUG:teuthology.orchestra.run.smithi121:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-24T23:03:45.314 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2024-04-24T23:03:45.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:03:45.314 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-24T23:03:45.373 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:03:45.374 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:03:45.374 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:03:45.390 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=413 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T23:03:45.390 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:03:45.390 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:03:45.390 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:03:45.432 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3271 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T23:03:45.432 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:03:45.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.530+0000 7f42829b7640 1 Processor -- start 2024-04-24T23:03:45.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.530+0000 7f42829b7640 1 -- start start 2024-04-24T23:03:45.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.531+0000 7f42829b7640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f427c074210 0x7f427c06c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:45.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.531+0000 7f42829b7640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f427c0b0b80 0x7f427c0745f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:45.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.531+0000 7f42829b7640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f427c0b0770 con 0x7f427c0ae140 2024-04-24T23:03:45.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.531+0000 7f42829b7640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f427c0b1c20 con 0x7f427c0b0b80 2024-04-24T23:03:45.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.531+0000 7f42829b7640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f427c074b30 con 0x7f427c074210 2024-04-24T23:03:45.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.532+0000 7f427bfff640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f427c074210 0x7f427c06c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:45.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.532+0000 7f427b7fe640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f427c0b0b80 0x7f427c0745f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:45.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.532+0000 7f427bfff640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f427c074210 0x7f427c06c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.32:37562/0 (socket says 172.21.15.32:37562) 2024-04-24T23:03:45.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.532+0000 7f427bfff640 1 -- 172.21.15.32:0/745634962 learned_addr learned my addr 172.21.15.32:0/745634962 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:45.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.532+0000 7f4280f54640 1 --1- 172.21.15.32:0/745634962 >> v1:172.21.15.121:6789/0 conn(0x7f427c0ae140 0x7f427c0b20b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:54302/0 (socket says 172.21.15.32:54302) 2024-04-24T23:03:45.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.532+0000 7f427bfff640 1 -- 172.21.15.32:0/745634962 >> v2:172.21.15.121:3300/0 conn(0x7f427c0b0b80 msgr2=0x7f427c0745f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:45.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.532+0000 7f427bfff640 1 --2- 172.21.15.32:0/745634962 >> v2:172.21.15.121:3300/0 conn(0x7f427c0b0b80 0x7f427c0745f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:45.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.532+0000 7f427bfff640 1 -- 172.21.15.32:0/745634962 >> v1:172.21.15.121:6789/0 conn(0x7f427c0ae140 legacy=0x7f427c0b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:45.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.532+0000 7f427affd640 1 -- 172.21.15.32:0/745634962 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1177795283 0 0) 0x7f427c0b0770 con 0x7f427c0ae140 2024-04-24T23:03:45.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.532+0000 7f427bfff640 1 -- 172.21.15.32:0/745634962 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f427c0b1010 con 0x7f427c074210 2024-04-24T23:03:45.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.532+0000 7f427b7fe640 1 --2- 172.21.15.32:0/745634962 >> v2:172.21.15.121:3300/0 conn(0x7f427c0b0b80 0x7f427c0745f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:03:45.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.533+0000 7f427bfff640 1 --2- 172.21.15.32:0/745634962 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f427c074210 0x7f427c06c480 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f426c009870 tx=0x7f426c02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87473c8e8e96c2ff server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:45.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.533+0000 7f427affd640 1 -- 172.21.15.32:0/745634962 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426c037070 con 0x7f427c074210 2024-04-24T23:03:45.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.533+0000 7f427affd640 1 -- 172.21.15.32:0/745634962 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f426c032440 con 0x7f427c074210 2024-04-24T23:03:45.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.533+0000 7f427affd640 1 -- 172.21.15.32:0/745634962 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426c032810 con 0x7f427c074210 2024-04-24T23:03:45.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.533+0000 7f42829b7640 1 -- 172.21.15.32:0/745634962 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f427c074210 msgr2=0x7f427c06c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:45.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.533+0000 7f42829b7640 1 --2- 172.21.15.32:0/745634962 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f427c074210 0x7f427c06c480 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f426c009870 tx=0x7f426c02a270 comp rx=0 tx=0).stop 2024-04-24T23:03:45.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.534+0000 7f42829b7640 1 -- 172.21.15.32:0/745634962 shutdown_connections 2024-04-24T23:03:45.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.534+0000 7f42829b7640 1 --2- 172.21.15.32:0/745634962 >> v2:172.21.15.121:3300/0 conn(0x7f427c0b0b80 0x7f427c0745f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:45.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.534+0000 7f42829b7640 1 --2- 172.21.15.32:0/745634962 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f427c074210 0x7f427c06c480 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:45.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.534+0000 7f42829b7640 1 -- 172.21.15.32:0/745634962 >> 172.21.15.32:0/745634962 conn(0x7f427c0b2e60 msgr2=0x7f427c0b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:45.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.534+0000 7f42829b7640 1 -- 172.21.15.32:0/745634962 shutdown_connections 2024-04-24T23:03:45.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.534+0000 7f42829b7640 1 -- 172.21.15.32:0/745634962 wait complete. 2024-04-24T23:03:45.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.535+0000 7f42829b7640 1 Processor -- start 2024-04-24T23:03:45.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.535+0000 7f42829b7640 1 -- start start 2024-04-24T23:03:45.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.536+0000 7f42829b7640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f427c074210 0x7f427c0cad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:45.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.536+0000 7f42829b7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f427c0ae140 0x7f427c158530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:45.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.536+0000 7f42829b7640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f427c0c9410 0x7f427c15d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:45.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.536+0000 7f42829b7640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f427c0b18b0 con 0x7f427c0ae140 2024-04-24T23:03:45.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.536+0000 7f42829b7640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f427c0b15d0 con 0x7f427c074210 2024-04-24T23:03:45.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.536+0000 7f42829b7640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f427c0b3ba0 con 0x7f427c0c9410 2024-04-24T23:03:45.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.536+0000 7f427b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f427c0ae140 0x7f427c158530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:45.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.536+0000 7f427b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f427c0ae140 0x7f427c158530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53200/0 (socket says 172.21.15.32:53200) 2024-04-24T23:03:45.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.536+0000 7f427b7fe640 1 -- 172.21.15.32:0/341088245 learned_addr learned my addr 172.21.15.32:0/341088245 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:45.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.536+0000 7f427bfff640 1 --2- 172.21.15.32:0/341088245 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f427c074210 0x7f427c0cad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:45.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.536+0000 7f4280f54640 1 --2- 172.21.15.32:0/341088245 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f427c0c9410 0x7f427c15d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:45.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.536+0000 7f427b7fe640 1 -- 172.21.15.32:0/341088245 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f427c0c9410 msgr2=0x7f427c15d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:45.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.536+0000 7f427b7fe640 1 --2- 172.21.15.32:0/341088245 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f427c0c9410 0x7f427c15d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:45.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.536+0000 7f427b7fe640 1 -- 172.21.15.32:0/341088245 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f427c074210 msgr2=0x7f427c0cad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:45.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.536+0000 7f427b7fe640 1 --2- 172.21.15.32:0/341088245 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f427c074210 0x7f427c0cad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:45.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.536+0000 7f427b7fe640 1 -- 172.21.15.32:0/341088245 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f427c158a70 con 0x7f427c0ae140 2024-04-24T23:03:45.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.536+0000 7f427b7fe640 1 --2- 172.21.15.32:0/341088245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f427c0ae140 0x7f427c158530 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f427000ec10 tx=0x7f427000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:45.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.536+0000 7f4278ff9640 1 -- 172.21.15.32:0/341088245 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4270017070 con 0x7f427c0ae140 2024-04-24T23:03:45.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.536+0000 7f4278ff9640 1 -- 172.21.15.32:0/341088245 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f427000cc30 con 0x7f427c0ae140 2024-04-24T23:03:45.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.536+0000 7f42829b7640 1 -- 172.21.15.32:0/341088245 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f427c0ca860 con 0x7f427c0ae140 2024-04-24T23:03:45.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.537+0000 7f4278ff9640 1 -- 172.21.15.32:0/341088245 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4270013470 con 0x7f427c0ae140 2024-04-24T23:03:45.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.537+0000 7f42829b7640 1 -- 172.21.15.32:0/341088245 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f427c0c9bf0 con 0x7f427c0ae140 2024-04-24T23:03:45.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.538+0000 7f42829b7640 1 -- 172.21.15.32:0/341088245 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f427c068430 con 0x7f427c0ae140 2024-04-24T23:03:45.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.538+0000 7f4278ff9640 1 -- 172.21.15.32:0/341088245 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f427001a070 con 0x7f427c0ae140 2024-04-24T23:03:45.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.538+0000 7f4278ff9640 1 --2- 172.21.15.32:0/341088245 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f426806be30 0x7f426806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:45.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.538+0000 7f4278ff9640 1 -- 172.21.15.32:0/341088245 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5709+0+0 (secure 0 0 0) 0x7f427008b0c0 con 0x7f427c0ae140 2024-04-24T23:03:45.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.539+0000 7f4278ff9640 1 -- 172.21.15.32:0/341088245 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=92}) -- 0x7f42680739c0 con 0x7f427c0ae140 2024-04-24T23:03:45.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.539+0000 7f427bfff640 1 --2- 172.21.15.32:0/341088245 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f426806be30 0x7f426806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:45.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.539+0000 7f427bfff640 1 --2- 172.21.15.32:0/341088245 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f426806be30 0x7f426806e2f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f426c034000 tx=0x7f426c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:45.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.543+0000 7f4278ff9640 1 -- 172.21.15.32:0/341088245 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4270059430 con 0x7f427c0ae140 2024-04-24T23:03:45.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.769+0000 7f42829b7640 1 -- 172.21.15.32:0/341088245 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f427c162200 con 0x7f427c0ae140 2024-04-24T23:03:45.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.769+0000 7f4278ff9640 1 -- 172.21.15.32:0/341088245 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5186 (secure 0 0 0) 0x7f427005d730 con 0x7f427c0ae140 2024-04-24T23:03:45.771 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:03:45.771 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":36,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5486,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/3550206194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3550206194},{"type":"v1","addr":"172.21.15.32:6834","nonce":3550206194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5905,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1706188546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1706188546},{"type":"v1","addr":"172.21.15.121:6835","nonce":1706188546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5911,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2669443498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2669443498},{"type":"v1","addr":"172.21.15.121:6837","nonce":2669443498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:03:00.630938+0000","modified":"2024-04-24T23:03:03.639570+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6474":{"gid":6474,"name":"a","rank":0,"incarnation":33,"state":"up:active","state_seq":4,"addr":"172.21.15.32:6835/464246210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":464246210},{"type":"v1","addr":"172.21.15.32:6835","nonce":464246210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6474,"qdb_cluster":[6474]},"id":5}]} 2024-04-24T23:03:45.772 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 36 2024-04-24T23:03:45.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.772+0000 7f42829b7640 1 -- 172.21.15.32:0/341088245 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f426806be30 msgr2=0x7f426806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:45.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.772+0000 7f42829b7640 1 --2- 172.21.15.32:0/341088245 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f426806be30 0x7f426806e2f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f426c034000 tx=0x7f426c02e000 comp rx=0 tx=0).stop 2024-04-24T23:03:45.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.772+0000 7f42829b7640 1 -- 172.21.15.32:0/341088245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f427c0ae140 msgr2=0x7f427c158530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:45.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.772+0000 7f42829b7640 1 --2- 172.21.15.32:0/341088245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f427c0ae140 0x7f427c158530 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f427000ec10 tx=0x7f427000c520 comp rx=0 tx=0).stop 2024-04-24T23:03:45.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.772+0000 7f42829b7640 1 -- 172.21.15.32:0/341088245 shutdown_connections 2024-04-24T23:03:45.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.772+0000 7f42829b7640 1 --2- 172.21.15.32:0/341088245 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f426806be30 0x7f426806e2f0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:45.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.772+0000 7f42829b7640 1 --2- 172.21.15.32:0/341088245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f427c0ae140 0x7f427c158530 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:45.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.772+0000 7f42829b7640 1 --2- 172.21.15.32:0/341088245 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f427c0c9410 0x7f427c15d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:45.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.772+0000 7f42829b7640 1 --2- 172.21.15.32:0/341088245 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f427c074210 0x7f427c0cad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:45.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.772+0000 7f42829b7640 1 -- 172.21.15.32:0/341088245 >> 172.21.15.32:0/341088245 conn(0x7f427c0b2e60 msgr2=0x7f427c06e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:45.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.772+0000 7f42829b7640 1 -- 172.21.15.32:0/341088245 shutdown_connections 2024-04-24T23:03:45.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.772+0000 7f42829b7640 1 -- 172.21.15.32:0/341088245 wait complete. 2024-04-24T23:03:45.784 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:03:45.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.883+0000 7fd8899c8640 1 Processor -- start 2024-04-24T23:03:45.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.884+0000 7fd8899c8640 1 -- start start 2024-04-24T23:03:45.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.884+0000 7fd8899c8640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd884162e10 0x7fd8840ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:45.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.884+0000 7fd8899c8640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8840cd270 con 0x7fd884162320 2024-04-24T23:03:45.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.884+0000 7fd8899c8640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd884071610 con 0x7fd8840b5af0 2024-04-24T23:03:45.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.884+0000 7fd8899c8640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fd8840b5660 con 0x7fd884162e10 2024-04-24T23:03:45.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.884+0000 7fd8837fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd884162e10 0x7fd8840ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:45.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.885+0000 7fd882ffd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd884162320 0x7fd884162700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54990/0 (socket says 172.21.15.32:54990) 2024-04-24T23:03:45.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.885+0000 7fd882ffd640 1 -- 172.21.15.32:0/633561136 learned_addr learned my addr 172.21.15.32:0/633561136 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:03:45.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.885+0000 7fd8837fe640 1 -- 172.21.15.32:0/633561136 >> v1:172.21.15.121:6789/0 conn(0x7fd8840b5af0 legacy=0x7fd88415ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:45.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.885+0000 7fd8837fe640 1 -- 172.21.15.32:0/633561136 >> v1:172.21.15.32:6789/0 conn(0x7fd884162320 legacy=0x7fd884162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:45.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.885+0000 7fd881ffb640 1 -- 172.21.15.32:0/633561136 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2956221273 0 0) 0x7fd8840cd270 con 0x7fd884162320 2024-04-24T23:03:45.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.885+0000 7fd8837fe640 1 -- 172.21.15.32:0/633561136 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8840718f0 con 0x7fd884162e10 2024-04-24T23:03:45.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.886+0000 7fd8837fe640 1 --2- 172.21.15.32:0/633561136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd884162e10 0x7fd8840ccc90 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fd87c009d90 tx=0x7fd87c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9973a51a08ec7076 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:45.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.886+0000 7fd881ffb640 1 -- 172.21.15.32:0/633561136 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87c007740 con 0x7fd884162e10 2024-04-24T23:03:45.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.886+0000 7fd881ffb640 1 -- 172.21.15.32:0/633561136 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd87c0078e0 con 0x7fd884162e10 2024-04-24T23:03:45.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.886+0000 7fd881ffb640 1 -- 172.21.15.32:0/633561136 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87c007cb0 con 0x7fd884162e10 2024-04-24T23:03:45.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.886+0000 7fd8899c8640 1 -- 172.21.15.32:0/633561136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd884162e10 msgr2=0x7fd8840ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:45.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.886+0000 7fd8899c8640 1 --2- 172.21.15.32:0/633561136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd884162e10 0x7fd8840ccc90 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fd87c009d90 tx=0x7fd87c02a660 comp rx=0 tx=0).stop 2024-04-24T23:03:45.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.886+0000 7fd8899c8640 1 -- 172.21.15.32:0/633561136 shutdown_connections 2024-04-24T23:03:45.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.886+0000 7fd8899c8640 1 --2- 172.21.15.32:0/633561136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd884162e10 0x7fd8840ccc90 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:45.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.886+0000 7fd8899c8640 1 -- 172.21.15.32:0/633561136 >> 172.21.15.32:0/633561136 conn(0x7fd8840d0ae0 msgr2=0x7fd8840d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:45.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.886+0000 7fd8899c8640 1 -- 172.21.15.32:0/633561136 shutdown_connections 2024-04-24T23:03:45.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.886+0000 7fd8899c8640 1 -- 172.21.15.32:0/633561136 wait complete. 2024-04-24T23:03:45.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.887+0000 7fd8899c8640 1 Processor -- start 2024-04-24T23:03:45.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.888+0000 7fd8899c8640 1 -- start start 2024-04-24T23:03:45.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.888+0000 7fd8899c8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8840b5af0 0x7fd88414e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:45.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.888+0000 7fd8899c8640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd884162320 0x7fd8840cefd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:45.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.888+0000 7fd8899c8640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd884162e10 0x7fd8840cf510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:45.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.888+0000 7fd8899c8640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd884071bd0 con 0x7fd8840b5af0 2024-04-24T23:03:45.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.888+0000 7fd8899c8640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd884067950 con 0x7fd884162e10 2024-04-24T23:03:45.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.888+0000 7fd8899c8640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd884071d50 con 0x7fd884162320 2024-04-24T23:03:45.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.888+0000 7fd882ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8840b5af0 0x7fd88414e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:45.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.888+0000 7fd882ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8840b5af0 0x7fd88414e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53228/0 (socket says 172.21.15.32:53228) 2024-04-24T23:03:45.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.888+0000 7fd882ffd640 1 -- 172.21.15.32:0/1019329053 learned_addr learned my addr 172.21.15.32:0/1019329053 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:45.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.889+0000 7fd8837fe640 1 --2- 172.21.15.32:0/1019329053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd884162e10 0x7fd8840cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:45.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.889+0000 7fd8827fc640 1 --2- 172.21.15.32:0/1019329053 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd884162320 0x7fd8840cefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:45.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.889+0000 7fd882ffd640 1 -- 172.21.15.32:0/1019329053 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd884162320 msgr2=0x7fd8840cefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:45.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.889+0000 7fd882ffd640 1 --2- 172.21.15.32:0/1019329053 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd884162320 0x7fd8840cefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:45.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.889+0000 7fd882ffd640 1 -- 172.21.15.32:0/1019329053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd884162e10 msgr2=0x7fd8840cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:45.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.889+0000 7fd882ffd640 1 --2- 172.21.15.32:0/1019329053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd884162e10 0x7fd8840cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:45.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.889+0000 7fd882ffd640 1 -- 172.21.15.32:0/1019329053 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8840cd590 con 0x7fd8840b5af0 2024-04-24T23:03:45.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.889+0000 7fd8837fe640 1 --2- 172.21.15.32:0/1019329053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd884162e10 0x7fd8840cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:03:45.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.889+0000 7fd882ffd640 1 --2- 172.21.15.32:0/1019329053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8840b5af0 0x7fd88414e400 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fd87400c4a0 tx=0x7fd87400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:45.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.889+0000 7fd87bfff640 1 -- 172.21.15.32:0/1019329053 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd874018070 con 0x7fd8840b5af0 2024-04-24T23:03:45.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.889+0000 7fd87bfff640 1 -- 172.21.15.32:0/1019329053 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8740074e0 con 0x7fd8840b5af0 2024-04-24T23:03:45.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.889+0000 7fd8899c8640 1 -- 172.21.15.32:0/1019329053 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8840cd870 con 0x7fd8840b5af0 2024-04-24T23:03:45.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.889+0000 7fd87bfff640 1 -- 172.21.15.32:0/1019329053 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd874004a20 con 0x7fd8840b5af0 2024-04-24T23:03:45.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.889+0000 7fd8899c8640 1 -- 172.21.15.32:0/1019329053 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8840cde30 con 0x7fd8840b5af0 2024-04-24T23:03:45.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.890+0000 7fd8899c8640 1 -- 172.21.15.32:0/1019329053 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd884068430 con 0x7fd8840b5af0 2024-04-24T23:03:45.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.891+0000 7fd87bfff640 1 -- 172.21.15.32:0/1019329053 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd87401b070 con 0x7fd8840b5af0 2024-04-24T23:03:45.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.891+0000 7fd87bfff640 1 --2- 172.21.15.32:0/1019329053 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd86406be30 0x7fd86406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:45.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.891+0000 7fd87bfff640 1 -- 172.21.15.32:0/1019329053 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5709+0+0 (secure 0 0 0) 0x7fd87408a0d0 con 0x7fd8840b5af0 2024-04-24T23:03:45.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.891+0000 7fd87bfff640 1 -- 172.21.15.32:0/1019329053 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=92}) -- 0x7fd864073960 con 0x7fd8840b5af0 2024-04-24T23:03:45.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.891+0000 7fd8827fc640 1 --2- 172.21.15.32:0/1019329053 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd86406be30 0x7fd86406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:45.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.892+0000 7fd8827fc640 1 --2- 172.21.15.32:0/1019329053 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd86406be30 0x7fd86406e2f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd8840b0410 tx=0x7fd86c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:45.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:45.895+0000 7fd87bfff640 1 -- 172.21.15.32:0/1019329053 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd874058b10 con 0x7fd8840b5af0 2024-04-24T23:03:46.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.082+0000 7fd8899c8640 1 -- 172.21.15.32:0/1019329053 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd8840ce250 con 0x7fd8840b5af0 2024-04-24T23:03:46.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.083+0000 7fd87bfff640 1 -- 172.21.15.32:0/1019329053 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+16694 (secure 0 0 0) 0x7fd874058410 con 0x7fd8840b5af0 2024-04-24T23:03:46.085 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:03:46.085 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":91,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:03:45.367429+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:02:58.930222+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-24T23:02:59.607070+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","full","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["backfillfull","exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["backfillfull","exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["backfillfull","exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:03:46.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.085+0000 7fd8899c8640 1 -- 172.21.15.32:0/1019329053 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd86406be30 msgr2=0x7fd86406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:46.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.085+0000 7fd8899c8640 1 --2- 172.21.15.32:0/1019329053 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd86406be30 0x7fd86406e2f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd8840b0410 tx=0x7fd86c009040 comp rx=0 tx=0).stop 2024-04-24T23:03:46.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.085+0000 7fd8899c8640 1 -- 172.21.15.32:0/1019329053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8840b5af0 msgr2=0x7fd88414e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:46.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.085+0000 7fd8899c8640 1 --2- 172.21.15.32:0/1019329053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8840b5af0 0x7fd88414e400 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fd87400c4a0 tx=0x7fd87400c960 comp rx=0 tx=0).stop 2024-04-24T23:03:46.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.085+0000 7fd8899c8640 1 -- 172.21.15.32:0/1019329053 shutdown_connections 2024-04-24T23:03:46.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.085+0000 7fd8899c8640 1 --2- 172.21.15.32:0/1019329053 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd86406be30 0x7fd86406e2f0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:46.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.085+0000 7fd8899c8640 1 --2- 172.21.15.32:0/1019329053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd884162e10 0x7fd8840cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:46.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.085+0000 7fd8899c8640 1 --2- 172.21.15.32:0/1019329053 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd884162320 0x7fd8840cefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:46.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.085+0000 7fd8899c8640 1 --2- 172.21.15.32:0/1019329053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8840b5af0 0x7fd88414e400 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:46.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.085+0000 7fd8899c8640 1 -- 172.21.15.32:0/1019329053 >> 172.21.15.32:0/1019329053 conn(0x7fd8840d0ae0 msgr2=0x7fd8840b43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:46.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.085+0000 7fd8899c8640 1 -- 172.21.15.32:0/1019329053 shutdown_connections 2024-04-24T23:03:46.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.085+0000 7fd8899c8640 1 -- 172.21.15.32:0/1019329053 wait complete. 2024-04-24T23:03:46.098 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:02:58.930222+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:02:59.607070+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:03:46.099 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:03:46.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.201+0000 7fc0d1a58640 1 Processor -- start 2024-04-24T23:03:46.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.201+0000 7fc0d1a58640 1 -- start start 2024-04-24T23:03:46.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.202+0000 7fc0d1a58640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc0cc0b5af0 0x7fc0cc0ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:46.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.202+0000 7fc0d1a58640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0cc0cd270 con 0x7fc0cc162e10 2024-04-24T23:03:46.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.202+0000 7fc0d1a58640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0cc071610 con 0x7fc0cc162320 2024-04-24T23:03:46.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.202+0000 7fc0d1a58640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fc0cc0b5660 con 0x7fc0cc0b5af0 2024-04-24T23:03:46.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.202+0000 7fc0ca7fc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc0cc0b5af0 0x7fc0cc0ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:46.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.202+0000 7fc0ca7fc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc0cc0b5af0 0x7fc0cc0ccc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53238/0 (socket says 172.21.15.32:53238) 2024-04-24T23:03:46.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.202+0000 7fc0ca7fc640 1 -- 172.21.15.32:0/1704017541 learned_addr learned my addr 172.21.15.32:0/1704017541 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:46.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.202+0000 7fc0ca7fc640 1 -- 172.21.15.32:0/1704017541 >> v1:172.21.15.121:6789/0 conn(0x7fc0cc162320 legacy=0x7fc0cc162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:46.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.202+0000 7fc0ca7fc640 1 -- 172.21.15.32:0/1704017541 >> v1:172.21.15.32:6789/0 conn(0x7fc0cc162e10 legacy=0x7fc0cc15ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:46.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.202+0000 7fc0c9ffb640 1 -- 172.21.15.32:0/1704017541 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1611233594 0 0) 0x7fc0cc0cd270 con 0x7fc0cc162e10 2024-04-24T23:03:46.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.202+0000 7fc0ca7fc640 1 -- 172.21.15.32:0/1704017541 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0cc0718f0 con 0x7fc0cc0b5af0 2024-04-24T23:03:46.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.203+0000 7fc0ca7fc640 1 --2- 172.21.15.32:0/1704017541 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0cc0b5af0 0x7fc0cc0ccc90 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0009870 tx=0x7fc0c002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1de1c30d2a4a0ed server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:46.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.203+0000 7fc0c9ffb640 1 -- 172.21.15.32:0/1704017541 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c0037070 con 0x7fc0cc0b5af0 2024-04-24T23:03:46.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.203+0000 7fc0c9ffb640 1 -- 172.21.15.32:0/1704017541 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0c0032440 con 0x7fc0cc0b5af0 2024-04-24T23:03:46.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.203+0000 7fc0c9ffb640 1 -- 172.21.15.32:0/1704017541 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c0032810 con 0x7fc0cc0b5af0 2024-04-24T23:03:46.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.203+0000 7fc0d1a58640 1 -- 172.21.15.32:0/1704017541 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0cc0b5af0 msgr2=0x7fc0cc0ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:46.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.203+0000 7fc0d1a58640 1 --2- 172.21.15.32:0/1704017541 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0cc0b5af0 0x7fc0cc0ccc90 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0009870 tx=0x7fc0c002a270 comp rx=0 tx=0).stop 2024-04-24T23:03:46.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.203+0000 7fc0d1a58640 1 -- 172.21.15.32:0/1704017541 shutdown_connections 2024-04-24T23:03:46.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.204+0000 7fc0d1a58640 1 --2- 172.21.15.32:0/1704017541 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0cc0b5af0 0x7fc0cc0ccc90 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:46.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.204+0000 7fc0d1a58640 1 -- 172.21.15.32:0/1704017541 >> 172.21.15.32:0/1704017541 conn(0x7fc0cc0d0ae0 msgr2=0x7fc0cc0d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:46.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.204+0000 7fc0d1a58640 1 -- 172.21.15.32:0/1704017541 shutdown_connections 2024-04-24T23:03:46.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.204+0000 7fc0d1a58640 1 -- 172.21.15.32:0/1704017541 wait complete. 2024-04-24T23:03:46.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.205+0000 7fc0d1a58640 1 Processor -- start 2024-04-24T23:03:46.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.205+0000 7fc0d1a58640 1 -- start start 2024-04-24T23:03:46.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.206+0000 7fc0d1a58640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0cc0b5af0 0x7fc0cc0cc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:46.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.206+0000 7fc0d1a58640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0cc162320 0x7fc0cc14b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:46.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.206+0000 7fc0d1a58640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc0cc162e10 0x7fc0cc0ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:46.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.206+0000 7fc0d1a58640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc0cc071bd0 con 0x7fc0cc0b5af0 2024-04-24T23:03:46.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.206+0000 7fc0d1a58640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc0cc067950 con 0x7fc0cc162320 2024-04-24T23:03:46.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.206+0000 7fc0d1a58640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc0cc071d50 con 0x7fc0cc162e10 2024-04-24T23:03:46.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.206+0000 7fc0caffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0cc0b5af0 0x7fc0cc0cc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:46.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.206+0000 7fc0caffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0cc0b5af0 0x7fc0cc0cc9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53242/0 (socket says 172.21.15.32:53242) 2024-04-24T23:03:46.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.206+0000 7fc0caffd640 1 -- 172.21.15.32:0/1678547743 learned_addr learned my addr 172.21.15.32:0/1678547743 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:46.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.206+0000 7fc0ca7fc640 1 --2- 172.21.15.32:0/1678547743 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0cc162320 0x7fc0cc14b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:46.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.206+0000 7fc0cb7fe640 1 --2- 172.21.15.32:0/1678547743 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc0cc162e10 0x7fc0cc0ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:46.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.206+0000 7fc0caffd640 1 -- 172.21.15.32:0/1678547743 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc0cc162e10 msgr2=0x7fc0cc0ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:46.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.206+0000 7fc0caffd640 1 --2- 172.21.15.32:0/1678547743 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc0cc162e10 0x7fc0cc0ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:46.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.206+0000 7fc0caffd640 1 -- 172.21.15.32:0/1678547743 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0cc162320 msgr2=0x7fc0cc14b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:46.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.206+0000 7fc0caffd640 1 --2- 172.21.15.32:0/1678547743 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0cc162320 0x7fc0cc14b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:46.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.206+0000 7fc0caffd640 1 -- 172.21.15.32:0/1678547743 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0cc14bd30 con 0x7fc0cc0b5af0 2024-04-24T23:03:46.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.206+0000 7fc0caffd640 1 --2- 172.21.15.32:0/1678547743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0cc0b5af0 0x7fc0cc0cc9c0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc00c450 tx=0x7fc0bc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:46.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.206+0000 7fc0b3fff640 1 -- 172.21.15.32:0/1678547743 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc0052d0 con 0x7fc0cc0b5af0 2024-04-24T23:03:46.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.206+0000 7fc0b3fff640 1 -- 172.21.15.32:0/1678547743 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0bc005470 con 0x7fc0cc0b5af0 2024-04-24T23:03:46.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.206+0000 7fc0b3fff640 1 -- 172.21.15.32:0/1678547743 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc012870 con 0x7fc0cc0b5af0 2024-04-24T23:03:46.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.206+0000 7fc0d1a58640 1 -- 172.21.15.32:0/1678547743 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0cc1490d0 con 0x7fc0cc0b5af0 2024-04-24T23:03:46.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.207+0000 7fc0d1a58640 1 -- 172.21.15.32:0/1678547743 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0cc149690 con 0x7fc0cc0b5af0 2024-04-24T23:03:46.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.207+0000 7fc0d1a58640 1 -- 172.21.15.32:0/1678547743 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0cc068430 con 0x7fc0cc0b5af0 2024-04-24T23:03:46.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.208+0000 7fc0b3fff640 1 -- 172.21.15.32:0/1678547743 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc0bc012a10 con 0x7fc0cc0b5af0 2024-04-24T23:03:46.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.208+0000 7fc0b3fff640 1 --2- 172.21.15.32:0/1678547743 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc0b806be30 0x7fc0b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:46.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.208+0000 7fc0b3fff640 1 -- 172.21.15.32:0/1678547743 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5709+0+0 (secure 0 0 0) 0x7fc0bc08a910 con 0x7fc0cc0b5af0 2024-04-24T23:03:46.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.209+0000 7fc0b3fff640 1 -- 172.21.15.32:0/1678547743 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=92}) -- 0x7fc0b80739c0 con 0x7fc0cc0b5af0 2024-04-24T23:03:46.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.209+0000 7fc0ca7fc640 1 --2- 172.21.15.32:0/1678547743 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc0b806be30 0x7fc0b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:46.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.209+0000 7fc0ca7fc640 1 --2- 172.21.15.32:0/1678547743 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc0b806be30 0x7fc0b806e2f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0034000 tx=0x7fc0c002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:46.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.213+0000 7fc0b3fff640 1 -- 172.21.15.32:0/1678547743 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc0bc058c70 con 0x7fc0cc0b5af0 2024-04-24T23:03:46.372 INFO:tasks.ceph.mon.a.smithi032.stderr:2024-04-24T23:03:46.370+0000 7f3b831cb640 -1 log_channel(cluster) log [ERR] : Health check update: 1 full osd(s) (OSD_FULL) 2024-04-24T23:03:46.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.376+0000 7fc0b3fff640 1 -- 172.21.15.32:0/1678547743 <== mon.0 v2:172.21.15.32:3300/0 7 ==== osd_map(92..92 src has 1..92) ==== 304+0+0 (secure 0 0 0) 0x7fc0bc088f50 con 0x7fc0cc0b5af0 2024-04-24T23:03:46.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.376+0000 7fc0b3fff640 1 -- 172.21.15.32:0/1678547743 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=93}) -- 0x7fc0b80741d0 con 0x7fc0cc0b5af0 2024-04-24T23:03:46.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.403+0000 7fc0d1a58640 1 -- 172.21.15.32:0/1678547743 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc0cc067730 con 0x7fc0cc0b5af0 2024-04-24T23:03:46.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.403+0000 7fc0b3fff640 1 -- 172.21.15.32:0/1678547743 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7fc0bc05cf70 con 0x7fc0cc0b5af0 2024-04-24T23:03:46.405 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T23:03:46.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.405+0000 7fc0d1a58640 1 -- 172.21.15.32:0/1678547743 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc0b806be30 msgr2=0x7fc0b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:46.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.405+0000 7fc0d1a58640 1 --2- 172.21.15.32:0/1678547743 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc0b806be30 0x7fc0b806e2f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0034000 tx=0x7fc0c002e000 comp rx=0 tx=0).stop 2024-04-24T23:03:46.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.405+0000 7fc0d1a58640 1 -- 172.21.15.32:0/1678547743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0cc0b5af0 msgr2=0x7fc0cc0cc9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:46.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.405+0000 7fc0d1a58640 1 --2- 172.21.15.32:0/1678547743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0cc0b5af0 0x7fc0cc0cc9c0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc00c450 tx=0x7fc0bc00c910 comp rx=0 tx=0).stop 2024-04-24T23:03:46.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.405+0000 7fc0d1a58640 1 -- 172.21.15.32:0/1678547743 shutdown_connections 2024-04-24T23:03:46.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.405+0000 7fc0d1a58640 1 --2- 172.21.15.32:0/1678547743 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc0b806be30 0x7fc0b806e2f0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:46.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.405+0000 7fc0d1a58640 1 --2- 172.21.15.32:0/1678547743 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc0cc162e10 0x7fc0cc0ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:46.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.405+0000 7fc0d1a58640 1 --2- 172.21.15.32:0/1678547743 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0cc162320 0x7fc0cc14b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:46.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.405+0000 7fc0d1a58640 1 --2- 172.21.15.32:0/1678547743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0cc0b5af0 0x7fc0cc0cc9c0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:46.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.405+0000 7fc0d1a58640 1 -- 172.21.15.32:0/1678547743 >> 172.21.15.32:0/1678547743 conn(0x7fc0cc0d0ae0 msgr2=0x7fc0cc06f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:46.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.405+0000 7fc0d1a58640 1 -- 172.21.15.32:0/1678547743 shutdown_connections 2024-04-24T23:03:46.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.406+0000 7fc0d1a58640 1 -- 172.21.15.32:0/1678547743 wait complete. 2024-04-24T23:03:46.418 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:03:46.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.521+0000 7f915f1a5640 1 Processor -- start 2024-04-24T23:03:46.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.521+0000 7f915f1a5640 1 -- start start 2024-04-24T23:03:46.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.521+0000 7f915f1a5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f91580af890 0x7f91581670f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:46.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.521+0000 7f915f1a5640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91580b6430 con 0x7f915806c5f0 2024-04-24T23:03:46.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.521+0000 7f915f1a5640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9158156f80 con 0x7f91580b44d0 2024-04-24T23:03:46.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.521+0000 7f915f1a5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f91580b6150 con 0x7f91580af890 2024-04-24T23:03:46.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.522+0000 7f915d742640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f91580af890 0x7f91581670f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:46.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.522+0000 7f915cf41640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f915806c5f0 0x7f915806c9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55016/0 (socket says 172.21.15.32:55016) 2024-04-24T23:03:46.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.522+0000 7f915cf41640 1 -- 172.21.15.32:0/3955091703 learned_addr learned my addr 172.21.15.32:0/3955091703 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:03:46.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.522+0000 7f915d742640 1 -- 172.21.15.32:0/3955091703 >> v1:172.21.15.121:6789/0 conn(0x7f91580b44d0 legacy=0x7f91580b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:46.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.522+0000 7f915d742640 1 -- 172.21.15.32:0/3955091703 >> v1:172.21.15.32:6789/0 conn(0x7f915806c5f0 legacy=0x7f915806c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:46.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.522+0000 7f915d742640 1 -- 172.21.15.32:0/3955091703 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91580b39f0 con 0x7f91580af890 2024-04-24T23:03:46.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.523+0000 7f915d742640 1 --2- 172.21.15.32:0/3955091703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91580af890 0x7f91581670f0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f9150009d90 tx=0x7f915002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e1db28e06f4f9a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:46.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.523+0000 7f91577fe640 1 -- 172.21.15.32:0/3955091703 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9150007740 con 0x7f91580af890 2024-04-24T23:03:46.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.523+0000 7f91577fe640 1 -- 172.21.15.32:0/3955091703 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91500078e0 con 0x7f91580af890 2024-04-24T23:03:46.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.523+0000 7f91577fe640 1 -- 172.21.15.32:0/3955091703 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9150007cb0 con 0x7f91580af890 2024-04-24T23:03:46.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.523+0000 7f915f1a5640 1 -- 172.21.15.32:0/3955091703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91580af890 msgr2=0x7f91581670f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:46.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.523+0000 7f915f1a5640 1 --2- 172.21.15.32:0/3955091703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91580af890 0x7f91581670f0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f9150009d90 tx=0x7f915002a6a0 comp rx=0 tx=0).stop 2024-04-24T23:03:46.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.523+0000 7f915f1a5640 1 -- 172.21.15.32:0/3955091703 shutdown_connections 2024-04-24T23:03:46.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.523+0000 7f915f1a5640 1 --2- 172.21.15.32:0/3955091703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91580af890 0x7f91581670f0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:46.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.523+0000 7f915f1a5640 1 -- 172.21.15.32:0/3955091703 >> 172.21.15.32:0/3955091703 conn(0x7f915806b110 msgr2=0x7f915806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:46.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.523+0000 7f915f1a5640 1 -- 172.21.15.32:0/3955091703 shutdown_connections 2024-04-24T23:03:46.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.523+0000 7f915f1a5640 1 -- 172.21.15.32:0/3955091703 wait complete. 2024-04-24T23:03:46.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.524+0000 7f915f1a5640 1 Processor -- start 2024-04-24T23:03:46.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.524+0000 7f915f1a5640 1 -- start start 2024-04-24T23:03:46.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.525+0000 7f915f1a5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f915806c5f0 0x7f915813ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:46.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.525+0000 7f915f1a5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91580af890 0x7f9158150af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:46.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.525+0000 7f915f1a5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f91580b44d0 0x7f9158151030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:46.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.525+0000 7f915f1a5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9158167630 con 0x7f91580af890 2024-04-24T23:03:46.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.525+0000 7f915f1a5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f91580b3fb0 con 0x7f915806c5f0 2024-04-24T23:03:46.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.525+0000 7f915f1a5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f91581677b0 con 0x7f91580b44d0 2024-04-24T23:03:46.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.525+0000 7f9157fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91580af890 0x7f9158150af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:46.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.525+0000 7f9157fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91580af890 0x7f9158150af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53254/0 (socket says 172.21.15.32:53254) 2024-04-24T23:03:46.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.525+0000 7f9157fff640 1 -- 172.21.15.32:0/2672670369 learned_addr learned my addr 172.21.15.32:0/2672670369 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:46.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.526+0000 7f915cf41640 1 --2- 172.21.15.32:0/2672670369 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f915806c5f0 0x7f915813ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:46.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.526+0000 7f915d742640 1 --2- 172.21.15.32:0/2672670369 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f91580b44d0 0x7f9158151030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:46.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.526+0000 7f9157fff640 1 -- 172.21.15.32:0/2672670369 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f91580b44d0 msgr2=0x7f9158151030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:46.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.526+0000 7f9157fff640 1 --2- 172.21.15.32:0/2672670369 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f91580b44d0 0x7f9158151030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:46.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.526+0000 7f9157fff640 1 -- 172.21.15.32:0/2672670369 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f915806c5f0 msgr2=0x7f915813ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:46.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.526+0000 7f9157fff640 1 --2- 172.21.15.32:0/2672670369 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f915806c5f0 0x7f915813ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:46.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.526+0000 7f9157fff640 1 -- 172.21.15.32:0/2672670369 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91580ab1c0 con 0x7f91580af890 2024-04-24T23:03:46.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.526+0000 7f9157fff640 1 --2- 172.21.15.32:0/2672670369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91580af890 0x7f9158150af0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f9144009fd0 tx=0x7f914400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:46.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.526+0000 7f91557fa640 1 -- 172.21.15.32:0/2672670369 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9144015020 con 0x7f91580af890 2024-04-24T23:03:46.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.526+0000 7f91557fa640 1 -- 172.21.15.32:0/2672670369 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9144011830 con 0x7f91580af890 2024-04-24T23:03:46.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.526+0000 7f915f1a5640 1 -- 172.21.15.32:0/2672670369 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9158154a80 con 0x7f91580af890 2024-04-24T23:03:46.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.526+0000 7f91557fa640 1 -- 172.21.15.32:0/2672670369 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9144011c00 con 0x7f91580af890 2024-04-24T23:03:46.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.526+0000 7f915f1a5640 1 -- 172.21.15.32:0/2672670369 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9158155040 con 0x7f91580af890 2024-04-24T23:03:46.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.527+0000 7f915f1a5640 1 -- 172.21.15.32:0/2672670369 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91580acd80 con 0x7f91580af890 2024-04-24T23:03:46.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.528+0000 7f91557fa640 1 -- 172.21.15.32:0/2672670369 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9144018020 con 0x7f91580af890 2024-04-24T23:03:46.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.528+0000 7f91557fa640 1 --2- 172.21.15.32:0/2672670369 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f913c06be30 0x7f913c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:46.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.528+0000 7f91557fa640 1 -- 172.21.15.32:0/2672670369 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f914408aed0 con 0x7f91580af890 2024-04-24T23:03:46.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.528+0000 7f91557fa640 1 -- 172.21.15.32:0/2672670369 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=93}) -- 0x7f913c073960 con 0x7f91580af890 2024-04-24T23:03:46.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.529+0000 7f915cf41640 1 --2- 172.21.15.32:0/2672670369 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f913c06be30 0x7f913c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:46.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.529+0000 7f915cf41640 1 --2- 172.21.15.32:0/2672670369 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f913c06be30 0x7f913c06e2f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f914c000f60 tx=0x7f914c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:46.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.532+0000 7f91557fa640 1 -- 172.21.15.32:0/2672670369 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f91580acd80 con 0x7f91580af890 2024-04-24T23:03:46.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.714+0000 7f915f1a5640 1 -- 172.21.15.32:0/2672670369 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f91581563b0 con 0x7f91580af890 2024-04-24T23:03:46.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.714+0000 7f91557fa640 1 -- 172.21.15.32:0/2672670369 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7f914401f020 con 0x7f91580af890 2024-04-24T23:03:46.716 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-24T23:03:46.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.716+0000 7f915f1a5640 1 -- 172.21.15.32:0/2672670369 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f913c06be30 msgr2=0x7f913c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:46.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.716+0000 7f915f1a5640 1 --2- 172.21.15.32:0/2672670369 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f913c06be30 0x7f913c06e2f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f914c000f60 tx=0x7f914c009040 comp rx=0 tx=0).stop 2024-04-24T23:03:46.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.716+0000 7f915f1a5640 1 -- 172.21.15.32:0/2672670369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91580af890 msgr2=0x7f9158150af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:46.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.716+0000 7f915f1a5640 1 --2- 172.21.15.32:0/2672670369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91580af890 0x7f9158150af0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f9144009fd0 tx=0x7f914400b870 comp rx=0 tx=0).stop 2024-04-24T23:03:46.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.716+0000 7f915f1a5640 1 -- 172.21.15.32:0/2672670369 shutdown_connections 2024-04-24T23:03:46.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.716+0000 7f915f1a5640 1 --2- 172.21.15.32:0/2672670369 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f913c06be30 0x7f913c06e2f0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:46.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.716+0000 7f915f1a5640 1 --2- 172.21.15.32:0/2672670369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91580af890 0x7f9158150af0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:46.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.716+0000 7f915f1a5640 1 --2- 172.21.15.32:0/2672670369 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f91580b44d0 0x7f9158151030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:46.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.716+0000 7f915f1a5640 1 --2- 172.21.15.32:0/2672670369 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f915806c5f0 0x7f915813ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:46.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.716+0000 7f915f1a5640 1 -- 172.21.15.32:0/2672670369 >> 172.21.15.32:0/2672670369 conn(0x7f915806b110 msgr2=0x7f915806b640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:46.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.716+0000 7f915f1a5640 1 -- 172.21.15.32:0/2672670369 shutdown_connections 2024-04-24T23:03:46.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.716+0000 7f915f1a5640 1 -- 172.21.15.32:0/2672670369 wait complete. 2024-04-24T23:03:46.728 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:03:46.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.831+0000 7fdba125d640 1 Processor -- start 2024-04-24T23:03:46.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.831+0000 7fdba125d640 1 -- start start 2024-04-24T23:03:46.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.831+0000 7fdba125d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdb9c0ae140 0x7fdb9c0ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:46.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.831+0000 7fdba125d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb9c074f00 con 0x7fdb9c074920 2024-04-24T23:03:46.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.831+0000 7fdba125d640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb9c0b0de0 con 0x7fdb9c15c8a0 2024-04-24T23:03:46.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.831+0000 7fdba125d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fdb9c06ca70 con 0x7fdb9c0ae140 2024-04-24T23:03:46.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.832+0000 7fdb9b59e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdb9c0ae140 0x7fdb9c0ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:46.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.832+0000 7fdb9a59c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fdb9c074920 0x7fdb9c0b1ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55028/0 (socket says 172.21.15.32:55028) 2024-04-24T23:03:46.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.832+0000 7fdb9b59e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdb9c0ae140 0x7fdb9c0ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53262/0 (socket says 172.21.15.32:53262) 2024-04-24T23:03:46.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.832+0000 7fdb9a59c640 1 -- 172.21.15.32:0/3039219502 learned_addr learned my addr 172.21.15.32:0/3039219502 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:03:46.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.832+0000 7fdb9b59e640 1 -- 172.21.15.32:0/3039219502 >> v1:172.21.15.121:6789/0 conn(0x7fdb9c15c8a0 legacy=0x7fdb9c074210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:46.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.832+0000 7fdb9b59e640 1 -- 172.21.15.32:0/3039219502 >> v1:172.21.15.32:6789/0 conn(0x7fdb9c074920 legacy=0x7fdb9c0b1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:46.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.832+0000 7fdb9b59e640 1 -- 172.21.15.32:0/3039219502 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb9c0b10c0 con 0x7fdb9c0ae140 2024-04-24T23:03:46.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.832+0000 7fdb99d9b640 1 -- 172.21.15.32:0/3039219502 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1606500069 0 0) 0x7fdb9c074f00 con 0x7fdb9c074920 2024-04-24T23:03:46.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.833+0000 7fdb9b59e640 1 --2- 172.21.15.32:0/3039219502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb9c0ae140 0x7fdb9c0ce7a0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fdb94009d90 tx=0x7fdb9402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b9945d6890ba386 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:46.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.833+0000 7fdb99d9b640 1 -- 172.21.15.32:0/3039219502 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb94007740 con 0x7fdb9c0ae140 2024-04-24T23:03:46.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.833+0000 7fdb99d9b640 1 -- 172.21.15.32:0/3039219502 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb940078e0 con 0x7fdb9c0ae140 2024-04-24T23:03:46.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.833+0000 7fdb99d9b640 1 -- 172.21.15.32:0/3039219502 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb94007cb0 con 0x7fdb9c0ae140 2024-04-24T23:03:46.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.833+0000 7fdba125d640 1 -- 172.21.15.32:0/3039219502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb9c0ae140 msgr2=0x7fdb9c0ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:46.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.833+0000 7fdba125d640 1 --2- 172.21.15.32:0/3039219502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb9c0ae140 0x7fdb9c0ce7a0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fdb94009d90 tx=0x7fdb9402a660 comp rx=0 tx=0).stop 2024-04-24T23:03:46.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.833+0000 7fdba125d640 1 -- 172.21.15.32:0/3039219502 shutdown_connections 2024-04-24T23:03:46.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.833+0000 7fdba125d640 1 --2- 172.21.15.32:0/3039219502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb9c0ae140 0x7fdb9c0ce7a0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:46.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.833+0000 7fdba125d640 1 -- 172.21.15.32:0/3039219502 >> 172.21.15.32:0/3039219502 conn(0x7fdb9c0b2e60 msgr2=0x7fdb9c0b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:46.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.833+0000 7fdba125d640 1 -- 172.21.15.32:0/3039219502 shutdown_connections 2024-04-24T23:03:46.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.833+0000 7fdba125d640 1 -- 172.21.15.32:0/3039219502 wait complete. 2024-04-24T23:03:46.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.835+0000 7fdba125d640 1 Processor -- start 2024-04-24T23:03:46.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.835+0000 7fdba125d640 1 -- start start 2024-04-24T23:03:46.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.836+0000 7fdba125d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdb9c074920 0x7fdb9c0cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:46.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.836+0000 7fdba125d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb9c0ae140 0x7fdb9c0cabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:46.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.836+0000 7fdba125d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb9c15c8a0 0x7fdb9c0d16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:46.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.836+0000 7fdba125d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c15cc80 con 0x7fdb9c0ae140 2024-04-24T23:03:46.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.836+0000 7fdba125d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c0b1680 con 0x7fdb9c15c8a0 2024-04-24T23:03:46.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.836+0000 7fdba125d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fdb9c0d1ff0 con 0x7fdb9c074920 2024-04-24T23:03:46.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.836+0000 7fdb9a59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb9c0ae140 0x7fdb9c0cabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:46.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.836+0000 7fdb9a59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb9c0ae140 0x7fdb9c0cabe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53272/0 (socket says 172.21.15.32:53272) 2024-04-24T23:03:46.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.836+0000 7fdb9a59c640 1 -- 172.21.15.32:0/3786706170 learned_addr learned my addr 172.21.15.32:0/3786706170 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:46.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.836+0000 7fdb9b59e640 1 --2- 172.21.15.32:0/3786706170 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb9c15c8a0 0x7fdb9c0d16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:46.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.836+0000 7fdb9ad9d640 1 --2- 172.21.15.32:0/3786706170 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdb9c074920 0x7fdb9c0cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:46.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.836+0000 7fdb9a59c640 1 -- 172.21.15.32:0/3786706170 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdb9c074920 msgr2=0x7fdb9c0cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:46.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.836+0000 7fdb9a59c640 1 --2- 172.21.15.32:0/3786706170 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdb9c074920 0x7fdb9c0cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:46.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.836+0000 7fdb9a59c640 1 -- 172.21.15.32:0/3786706170 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb9c15c8a0 msgr2=0x7fdb9c0d16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:46.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.836+0000 7fdb9a59c640 1 --2- 172.21.15.32:0/3786706170 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb9c15c8a0 0x7fdb9c0d16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:46.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.836+0000 7fdb9a59c640 1 -- 172.21.15.32:0/3786706170 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb9c0cb940 con 0x7fdb9c0ae140 2024-04-24T23:03:46.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.836+0000 7fdb9b59e640 1 --2- 172.21.15.32:0/3786706170 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb9c15c8a0 0x7fdb9c0d16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:03:46.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.836+0000 7fdb9ad9d640 1 --2- 172.21.15.32:0/3786706170 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdb9c074920 0x7fdb9c0cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:03:46.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.836+0000 7fdb9a59c640 1 --2- 172.21.15.32:0/3786706170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb9c0ae140 0x7fdb9c0cabe0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fdb9000c4d0 tx=0x7fdb9000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:46.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.836+0000 7fdb837fe640 1 -- 172.21.15.32:0/3786706170 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb90018070 con 0x7fdb9c0ae140 2024-04-24T23:03:46.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.836+0000 7fdb837fe640 1 -- 172.21.15.32:0/3786706170 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb900074e0 con 0x7fdb9c0ae140 2024-04-24T23:03:46.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.837+0000 7fdba125d640 1 -- 172.21.15.32:0/3786706170 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb9c0cbc20 con 0x7fdb9c0ae140 2024-04-24T23:03:46.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.837+0000 7fdb837fe640 1 -- 172.21.15.32:0/3786706170 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb90004a20 con 0x7fdb9c0ae140 2024-04-24T23:03:46.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.837+0000 7fdba125d640 1 -- 172.21.15.32:0/3786706170 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb9c0cc1e0 con 0x7fdb9c0ae140 2024-04-24T23:03:46.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.838+0000 7fdba125d640 1 -- 172.21.15.32:0/3786706170 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb9c068430 con 0x7fdb9c0ae140 2024-04-24T23:03:46.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.838+0000 7fdb837fe640 1 -- 172.21.15.32:0/3786706170 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdb9001b070 con 0x7fdb9c0ae140 2024-04-24T23:03:46.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.839+0000 7fdb837fe640 1 --2- 172.21.15.32:0/3786706170 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdb8806be30 0x7fdb8806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:46.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.839+0000 7fdb837fe640 1 -- 172.21.15.32:0/3786706170 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7fdb9008a1e0 con 0x7fdb9c0ae140 2024-04-24T23:03:46.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.839+0000 7fdb837fe640 1 -- 172.21.15.32:0/3786706170 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=93}) -- 0x7fdb88073960 con 0x7fdb9c0ae140 2024-04-24T23:03:46.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.839+0000 7fdb9ad9d640 1 --2- 172.21.15.32:0/3786706170 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdb8806be30 0x7fdb8806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:46.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.839+0000 7fdb9ad9d640 1 --2- 172.21.15.32:0/3786706170 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdb8806be30 0x7fdb8806e2f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c0afb10 tx=0x7fdb8c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:46.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:46.843+0000 7fdb837fe640 1 -- 172.21.15.32:0/3786706170 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdb90058b90 con 0x7fdb9c0ae140 2024-04-24T23:03:47.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.024+0000 7fdba125d640 1 -- 172.21.15.32:0/3786706170 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdb9c0ce370 con 0x7fdb9c0ae140 2024-04-24T23:03:47.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.024+0000 7fdb837fe640 1 -- 172.21.15.32:0/3786706170 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7fdb90058490 con 0x7fdb9c0ae140 2024-04-24T23:03:47.026 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-24T23:03:47.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.027+0000 7fdba125d640 1 -- 172.21.15.32:0/3786706170 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdb8806be30 msgr2=0x7fdb8806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:47.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.027+0000 7fdba125d640 1 --2- 172.21.15.32:0/3786706170 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdb8806be30 0x7fdb8806e2f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c0afb10 tx=0x7fdb8c009040 comp rx=0 tx=0).stop 2024-04-24T23:03:47.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.027+0000 7fdba125d640 1 -- 172.21.15.32:0/3786706170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb9c0ae140 msgr2=0x7fdb9c0cabe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:47.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.027+0000 7fdba125d640 1 --2- 172.21.15.32:0/3786706170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb9c0ae140 0x7fdb9c0cabe0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fdb9000c4d0 tx=0x7fdb9000c990 comp rx=0 tx=0).stop 2024-04-24T23:03:47.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.027+0000 7fdba125d640 1 -- 172.21.15.32:0/3786706170 shutdown_connections 2024-04-24T23:03:47.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.027+0000 7fdba125d640 1 --2- 172.21.15.32:0/3786706170 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdb8806be30 0x7fdb8806e2f0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:47.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.027+0000 7fdba125d640 1 --2- 172.21.15.32:0/3786706170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb9c0ae140 0x7fdb9c0cabe0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:47.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.027+0000 7fdba125d640 1 --2- 172.21.15.32:0/3786706170 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb9c15c8a0 0x7fdb9c0d16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:47.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.027+0000 7fdba125d640 1 --2- 172.21.15.32:0/3786706170 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdb9c074920 0x7fdb9c0cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:47.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.027+0000 7fdba125d640 1 -- 172.21.15.32:0/3786706170 >> 172.21.15.32:0/3786706170 conn(0x7fdb9c0b2e60 msgr2=0x7fdb9c073010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:47.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.027+0000 7fdba125d640 1 -- 172.21.15.32:0/3786706170 shutdown_connections 2024-04-24T23:03:47.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.027+0000 7fdba125d640 1 -- 172.21.15.32:0/3786706170 wait complete. 2024-04-24T23:03:47.039 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:03:47.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.138+0000 7f6fc9bf9640 1 Processor -- start 2024-04-24T23:03:47.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.138+0000 7f6fc9bf9640 1 -- start start 2024-04-24T23:03:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.139+0000 7f6fc9bf9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6fc40cc790 0x7f6fc40b52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.139+0000 7f6fc9bf9640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6fc415f7d0 0x7f6fc40b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.139+0000 7f6fc9bf9640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fc40b4b40 con 0x7f6fc40b3e20 2024-04-24T23:03:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.139+0000 7f6fc9bf9640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f6fc40b58b0 con 0x7f6fc40cc790 2024-04-24T23:03:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.139+0000 7f6fc9bf9640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f6fc40b5b90 con 0x7f6fc415f7d0 2024-04-24T23:03:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.139+0000 7f6fc37fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6fc40cc790 0x7f6fc40b52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.139+0000 7f6fc37fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6fc40cc790 0x7f6fc40b52d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53288/0 (socket says 172.21.15.32:53288) 2024-04-24T23:03:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.139+0000 7f6fc37fe640 1 -- 172.21.15.32:0/3896248579 learned_addr learned my addr 172.21.15.32:0/3896248579 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.139+0000 7f6fc2ffd640 1 --2- 172.21.15.32:0/3896248579 >> v2:172.21.15.121:3300/0 conn(0x7f6fc415f7d0 0x7f6fc40b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.139+0000 7f6fc37fe640 1 -- 172.21.15.32:0/3896248579 >> v2:172.21.15.121:3300/0 conn(0x7f6fc415f7d0 msgr2=0x7f6fc40b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.139+0000 7f6fc37fe640 1 --2- 172.21.15.32:0/3896248579 >> v2:172.21.15.121:3300/0 conn(0x7f6fc415f7d0 0x7f6fc40b38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:47.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.139+0000 7f6fc37fe640 1 -- 172.21.15.32:0/3896248579 >> v1:172.21.15.121:6789/0 conn(0x7f6fc40b3e20 legacy=0x7f6fc406c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:47.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.139+0000 7f6fc37fe640 1 -- 172.21.15.32:0/3896248579 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fc415fc60 con 0x7f6fc40cc790 2024-04-24T23:03:47.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.139+0000 7f6fc2ffd640 1 --2- 172.21.15.32:0/3896248579 >> v2:172.21.15.121:3300/0 conn(0x7f6fc415f7d0 0x7f6fc40b38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:03:47.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.140+0000 7f6fc37fe640 1 --2- 172.21.15.32:0/3896248579 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fc40cc790 0x7f6fc40b52d0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f6fc40cf440 tx=0x7f6fb402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91b6f4a83e175338 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:47.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.140+0000 7f6fc27fc640 1 -- 172.21.15.32:0/3896248579 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb4037070 con 0x7f6fc40cc790 2024-04-24T23:03:47.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.140+0000 7f6fc27fc640 1 -- 172.21.15.32:0/3896248579 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fb4032440 con 0x7f6fc40cc790 2024-04-24T23:03:47.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.140+0000 7f6fc27fc640 1 -- 172.21.15.32:0/3896248579 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb4032810 con 0x7f6fc40cc790 2024-04-24T23:03:47.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.141+0000 7f6fc9bf9640 1 -- 172.21.15.32:0/3896248579 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fc40cc790 msgr2=0x7f6fc40b52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:47.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.141+0000 7f6fc9bf9640 1 --2- 172.21.15.32:0/3896248579 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fc40cc790 0x7f6fc40b52d0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f6fc40cf440 tx=0x7f6fb402a270 comp rx=0 tx=0).stop 2024-04-24T23:03:47.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.141+0000 7f6fc9bf9640 1 -- 172.21.15.32:0/3896248579 shutdown_connections 2024-04-24T23:03:47.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.141+0000 7f6fc9bf9640 1 --2- 172.21.15.32:0/3896248579 >> v2:172.21.15.121:3300/0 conn(0x7f6fc415f7d0 0x7f6fc40b38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:47.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.141+0000 7f6fc9bf9640 1 --2- 172.21.15.32:0/3896248579 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fc40cc790 0x7f6fc40b52d0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:47.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.141+0000 7f6fc9bf9640 1 -- 172.21.15.32:0/3896248579 >> 172.21.15.32:0/3896248579 conn(0x7f6fc406b110 msgr2=0x7f6fc406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:47.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.141+0000 7f6fc9bf9640 1 -- 172.21.15.32:0/3896248579 shutdown_connections 2024-04-24T23:03:47.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.141+0000 7f6fc9bf9640 1 -- 172.21.15.32:0/3896248579 wait complete. 2024-04-24T23:03:47.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.142+0000 7f6fc9bf9640 1 Processor -- start 2024-04-24T23:03:47.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.142+0000 7f6fc9bf9640 1 -- start start 2024-04-24T23:03:47.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.143+0000 7f6fc9bf9640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6fc40cc790 0x7f6fc4072100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:47.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.143+0000 7f6fc9bf9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fc415f7d0 0x7f6fc41f4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:47.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.143+0000 7f6fc9bf9640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6fc41f48c0 0x7f6fc4156690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:47.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.143+0000 7f6fc9bf9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc40b6150 con 0x7f6fc415f7d0 2024-04-24T23:03:47.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.143+0000 7f6fc9bf9640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc40ae950 con 0x7f6fc40cc790 2024-04-24T23:03:47.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.143+0000 7f6fc9bf9640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f6fc40b62d0 con 0x7f6fc41f48c0 2024-04-24T23:03:47.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.143+0000 7f6fc2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fc415f7d0 0x7f6fc41f4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:47.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.143+0000 7f6fc2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fc415f7d0 0x7f6fc41f4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53290/0 (socket says 172.21.15.32:53290) 2024-04-24T23:03:47.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.143+0000 7f6fc2ffd640 1 -- 172.21.15.32:0/4004657753 learned_addr learned my addr 172.21.15.32:0/4004657753 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:47.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.143+0000 7f6fc3fff640 1 --2- 172.21.15.32:0/4004657753 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6fc41f48c0 0x7f6fc4156690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:47.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.143+0000 7f6fc37fe640 1 --2- 172.21.15.32:0/4004657753 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6fc40cc790 0x7f6fc4072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:47.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.143+0000 7f6fc2ffd640 1 -- 172.21.15.32:0/4004657753 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6fc41f48c0 msgr2=0x7f6fc4156690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:47.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.143+0000 7f6fc2ffd640 1 --2- 172.21.15.32:0/4004657753 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6fc41f48c0 0x7f6fc4156690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:47.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.144+0000 7f6fc2ffd640 1 -- 172.21.15.32:0/4004657753 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6fc40cc790 msgr2=0x7f6fc4072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:47.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.144+0000 7f6fc2ffd640 1 --2- 172.21.15.32:0/4004657753 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6fc40cc790 0x7f6fc4072100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:47.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.144+0000 7f6fc2ffd640 1 -- 172.21.15.32:0/4004657753 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fc41f4d50 con 0x7f6fc415f7d0 2024-04-24T23:03:47.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.144+0000 7f6fc37fe640 1 --2- 172.21.15.32:0/4004657753 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6fc40cc790 0x7f6fc4072100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:03:47.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.144+0000 7f6fc2ffd640 1 --2- 172.21.15.32:0/4004657753 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fc415f7d0 0x7f6fc41f4380 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f6fb800e810 tx=0x7f6fb800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:47.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.144+0000 7f6fabfff640 1 -- 172.21.15.32:0/4004657753 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb800f040 con 0x7f6fc415f7d0 2024-04-24T23:03:47.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.144+0000 7f6fabfff640 1 -- 172.21.15.32:0/4004657753 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fb8004030 con 0x7f6fc415f7d0 2024-04-24T23:03:47.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.144+0000 7f6fabfff640 1 -- 172.21.15.32:0/4004657753 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb80043e0 con 0x7f6fc415f7d0 2024-04-24T23:03:47.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.144+0000 7f6fc9bf9640 1 -- 172.21.15.32:0/4004657753 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fc41573c0 con 0x7f6fc415f7d0 2024-04-24T23:03:47.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.144+0000 7f6fc9bf9640 1 -- 172.21.15.32:0/4004657753 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fc4148800 con 0x7f6fc415f7d0 2024-04-24T23:03:47.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.145+0000 7f6fc9bf9640 1 -- 172.21.15.32:0/4004657753 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fc4068430 con 0x7f6fc415f7d0 2024-04-24T23:03:47.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.145+0000 7f6fabfff640 1 -- 172.21.15.32:0/4004657753 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6fb800c6d0 con 0x7f6fc415f7d0 2024-04-24T23:03:47.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.146+0000 7f6fabfff640 1 --2- 172.21.15.32:0/4004657753 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6fb006be30 0x7f6fb006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:47.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.146+0000 7f6fabfff640 1 -- 172.21.15.32:0/4004657753 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f6fb808aba0 con 0x7f6fc415f7d0 2024-04-24T23:03:47.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.146+0000 7f6fabfff640 1 -- 172.21.15.32:0/4004657753 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=93}) -- 0x7f6fb0073960 con 0x7f6fc415f7d0 2024-04-24T23:03:47.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.146+0000 7f6fc37fe640 1 --2- 172.21.15.32:0/4004657753 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6fb006be30 0x7f6fb006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:47.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.147+0000 7f6fc37fe640 1 --2- 172.21.15.32:0/4004657753 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6fb006be30 0x7f6fb006e2f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4034000 tx=0x7f6fb402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:47.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.150+0000 7f6fabfff640 1 -- 172.21.15.32:0/4004657753 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6fb8017020 con 0x7f6fc415f7d0 2024-04-24T23:03:47.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.374+0000 7f6fc9bf9640 1 -- 172.21.15.32:0/4004657753 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6fc406d2d0 con 0x7f6fc415f7d0 2024-04-24T23:03:47.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.375+0000 7f6fabfff640 1 -- 172.21.15.32:0/4004657753 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5186 (secure 0 0 0) 0x7f6fb8058f00 con 0x7f6fc415f7d0 2024-04-24T23:03:47.377 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:03:47.377 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":36,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5486,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/3550206194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3550206194},{"type":"v1","addr":"172.21.15.32:6834","nonce":3550206194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5905,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1706188546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1706188546},{"type":"v1","addr":"172.21.15.121:6835","nonce":1706188546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5911,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2669443498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2669443498},{"type":"v1","addr":"172.21.15.121:6837","nonce":2669443498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:03:00.630938+0000","modified":"2024-04-24T23:03:03.639570+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6474":{"gid":6474,"name":"a","rank":0,"incarnation":33,"state":"up:active","state_seq":4,"addr":"172.21.15.32:6835/464246210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":464246210},{"type":"v1","addr":"172.21.15.32:6835","nonce":464246210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6474,"qdb_cluster":[6474]},"id":5}]} 2024-04-24T23:03:47.379 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 36 2024-04-24T23:03:47.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.377+0000 7f6fc9bf9640 1 -- 172.21.15.32:0/4004657753 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6fb006be30 msgr2=0x7f6fb006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:47.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.377+0000 7f6fc9bf9640 1 --2- 172.21.15.32:0/4004657753 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6fb006be30 0x7f6fb006e2f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4034000 tx=0x7f6fb402e000 comp rx=0 tx=0).stop 2024-04-24T23:03:47.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.377+0000 7f6fc9bf9640 1 -- 172.21.15.32:0/4004657753 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fc415f7d0 msgr2=0x7f6fc41f4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:47.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.377+0000 7f6fc9bf9640 1 --2- 172.21.15.32:0/4004657753 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fc415f7d0 0x7f6fc41f4380 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f6fb800e810 tx=0x7f6fb800ecd0 comp rx=0 tx=0).stop 2024-04-24T23:03:47.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.377+0000 7f6fc9bf9640 1 -- 172.21.15.32:0/4004657753 shutdown_connections 2024-04-24T23:03:47.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.377+0000 7f6fc9bf9640 1 --2- 172.21.15.32:0/4004657753 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6fb006be30 0x7f6fb006e2f0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:47.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.377+0000 7f6fc9bf9640 1 --2- 172.21.15.32:0/4004657753 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fc415f7d0 0x7f6fc41f4380 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:47.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.377+0000 7f6fc9bf9640 1 --2- 172.21.15.32:0/4004657753 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6fc41f48c0 0x7f6fc4156690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:47.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.377+0000 7f6fc9bf9640 1 --2- 172.21.15.32:0/4004657753 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6fc40cc790 0x7f6fc4072100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:47.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.377+0000 7f6fc9bf9640 1 -- 172.21.15.32:0/4004657753 >> 172.21.15.32:0/4004657753 conn(0x7f6fc406b110 msgr2=0x7f6fc4161610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:47.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.377+0000 7f6fc9bf9640 1 -- 172.21.15.32:0/4004657753 shutdown_connections 2024-04-24T23:03:47.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.377+0000 7f6fc9bf9640 1 -- 172.21.15.32:0/4004657753 wait complete. 2024-04-24T23:03:47.389 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:03:47.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.489+0000 7fa31e742640 1 Processor -- start 2024-04-24T23:03:47.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.489+0000 7fa31e742640 1 -- start start 2024-04-24T23:03:47.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.489+0000 7fa31e742640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa31815cc40 0x7fa31815d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:47.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.490+0000 7fa31e742640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3180b4ee0 con 0x7fa31815d560 2024-04-24T23:03:47.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.490+0000 7fa31e742640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3180b51c0 con 0x7fa3180d05b0 2024-04-24T23:03:47.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.490+0000 7fa31e742640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fa31806cca0 con 0x7fa31815cc40 2024-04-24T23:03:47.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.490+0000 7fa317fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa31815cc40 0x7fa31815d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:47.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.490+0000 7fa31ccdf640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa31815d560 0x7fa3180cf340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55030/0 (socket says 172.21.15.32:55030) 2024-04-24T23:03:47.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.490+0000 7fa31ccdf640 1 -- 172.21.15.32:0/1584773290 learned_addr learned my addr 172.21.15.32:0/1584773290 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:03:47.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.490+0000 7fa317fff640 1 -- 172.21.15.32:0/1584773290 >> v1:172.21.15.121:6789/0 conn(0x7fa3180d05b0 legacy=0x7fa3180cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:47.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.490+0000 7fa317fff640 1 -- 172.21.15.32:0/1584773290 >> v1:172.21.15.32:6789/0 conn(0x7fa31815d560 legacy=0x7fa3180cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:47.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.490+0000 7fa317fff640 1 -- 172.21.15.32:0/1584773290 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3180b54a0 con 0x7fa31815cc40 2024-04-24T23:03:47.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.490+0000 7fa316ffd640 1 -- 172.21.15.32:0/1584773290 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3601512129 0 0) 0x7fa3180b4ee0 con 0x7fa31815d560 2024-04-24T23:03:47.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.491+0000 7fa317fff640 1 --2- 172.21.15.32:0/1584773290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa31815cc40 0x7fa31815d020 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fa3000097a0 tx=0x7fa30002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b01fb8bea4b7a46a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:47.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.491+0000 7fa316ffd640 1 -- 172.21.15.32:0/1584773290 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa300037070 con 0x7fa31815cc40 2024-04-24T23:03:47.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.491+0000 7fa316ffd640 1 -- 172.21.15.32:0/1584773290 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa300032440 con 0x7fa31815cc40 2024-04-24T23:03:47.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.491+0000 7fa316ffd640 1 -- 172.21.15.32:0/1584773290 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa300032810 con 0x7fa31815cc40 2024-04-24T23:03:47.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.491+0000 7fa31e742640 1 -- 172.21.15.32:0/1584773290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa31815cc40 msgr2=0x7fa31815d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:47.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.491+0000 7fa31e742640 1 --2- 172.21.15.32:0/1584773290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa31815cc40 0x7fa31815d020 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fa3000097a0 tx=0x7fa30002a1e0 comp rx=0 tx=0).stop 2024-04-24T23:03:47.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.491+0000 7fa31e742640 1 -- 172.21.15.32:0/1584773290 shutdown_connections 2024-04-24T23:03:47.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.491+0000 7fa31e742640 1 --2- 172.21.15.32:0/1584773290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa31815cc40 0x7fa31815d020 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:47.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.491+0000 7fa31e742640 1 -- 172.21.15.32:0/1584773290 >> 172.21.15.32:0/1584773290 conn(0x7fa3180d0ae0 msgr2=0x7fa3180d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:47.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.491+0000 7fa31e742640 1 -- 172.21.15.32:0/1584773290 shutdown_connections 2024-04-24T23:03:47.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.491+0000 7fa31e742640 1 -- 172.21.15.32:0/1584773290 wait complete. 2024-04-24T23:03:47.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.493+0000 7fa31e742640 1 Processor -- start 2024-04-24T23:03:47.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.493+0000 7fa31e742640 1 -- start start 2024-04-24T23:03:47.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.493+0000 7fa31e742640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa3180d05b0 0x7fa31815b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:47.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.494+0000 7fa31e742640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa31815cc40 0x7fa3180cf730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:47.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.494+0000 7fa31e742640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa31815d560 0x7fa318151e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:47.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.494+0000 7fa31e742640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa3180b5a60 con 0x7fa31815d560 2024-04-24T23:03:47.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.494+0000 7fa31e742640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa3180ce5f0 con 0x7fa3180d05b0 2024-04-24T23:03:47.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.494+0000 7fa31e742640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa31815d9b0 con 0x7fa31815cc40 2024-04-24T23:03:47.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.494+0000 7fa31ccdf640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa31815d560 0x7fa318151e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:47.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.494+0000 7fa31ccdf640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa31815d560 0x7fa318151e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53312/0 (socket says 172.21.15.32:53312) 2024-04-24T23:03:47.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.494+0000 7fa31ccdf640 1 -- 172.21.15.32:0/947665382 learned_addr learned my addr 172.21.15.32:0/947665382 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:47.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.494+0000 7fa3177fe640 1 --2- 172.21.15.32:0/947665382 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa31815cc40 0x7fa3180cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:47.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.494+0000 7fa317fff640 1 --2- 172.21.15.32:0/947665382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa3180d05b0 0x7fa31815b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:47.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.494+0000 7fa31ccdf640 1 -- 172.21.15.32:0/947665382 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa31815cc40 msgr2=0x7fa3180cf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:47.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.494+0000 7fa31ccdf640 1 --2- 172.21.15.32:0/947665382 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa31815cc40 0x7fa3180cf730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:47.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.494+0000 7fa31ccdf640 1 -- 172.21.15.32:0/947665382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa3180d05b0 msgr2=0x7fa31815b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:47.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.494+0000 7fa31ccdf640 1 --2- 172.21.15.32:0/947665382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa3180d05b0 0x7fa31815b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:47.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.494+0000 7fa31ccdf640 1 -- 172.21.15.32:0/947665382 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa31815baf0 con 0x7fa31815d560 2024-04-24T23:03:47.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.494+0000 7fa31ccdf640 1 --2- 172.21.15.32:0/947665382 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa31815d560 0x7fa318151e90 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fa31000c7d0 tx=0x7fa31000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:47.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.494+0000 7fa314ff9640 1 -- 172.21.15.32:0/947665382 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa310018070 con 0x7fa31815d560 2024-04-24T23:03:47.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.494+0000 7fa314ff9640 1 -- 172.21.15.32:0/947665382 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa31000a640 con 0x7fa31815d560 2024-04-24T23:03:47.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.494+0000 7fa31e742640 1 -- 172.21.15.32:0/947665382 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa31815c0b0 con 0x7fa31815d560 2024-04-24T23:03:47.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.494+0000 7fa314ff9640 1 -- 172.21.15.32:0/947665382 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31000aa70 con 0x7fa31815d560 2024-04-24T23:03:47.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.494+0000 7fa31e742640 1 -- 172.21.15.32:0/947665382 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa31815c670 con 0x7fa31815d560 2024-04-24T23:03:47.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.495+0000 7fa31e742640 1 -- 172.21.15.32:0/947665382 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa318068430 con 0x7fa31815d560 2024-04-24T23:03:47.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.496+0000 7fa314ff9640 1 -- 172.21.15.32:0/947665382 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa31001b070 con 0x7fa31815d560 2024-04-24T23:03:47.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.496+0000 7fa314ff9640 1 --2- 172.21.15.32:0/947665382 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa2f806be30 0x7fa2f806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:47.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.496+0000 7fa314ff9640 1 -- 172.21.15.32:0/947665382 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7fa31008a400 con 0x7fa31815d560 2024-04-24T23:03:47.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.497+0000 7fa314ff9640 1 -- 172.21.15.32:0/947665382 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=93}) -- 0x7fa2f8073960 con 0x7fa31815d560 2024-04-24T23:03:47.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.497+0000 7fa317fff640 1 --2- 172.21.15.32:0/947665382 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa2f806be30 0x7fa2f806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:47.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.497+0000 7fa317fff640 1 --2- 172.21.15.32:0/947665382 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa2f806be30 0x7fa2f806e2f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa300034000 tx=0x7fa30002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:47.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.501+0000 7fa314ff9640 1 -- 172.21.15.32:0/947665382 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa310058760 con 0x7fa31815d560 2024-04-24T23:03:47.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.682+0000 7fa31e742640 1 -- 172.21.15.32:0/947665382 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa318072f20 con 0x7fa31815d560 2024-04-24T23:03:47.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.684+0000 7fa314ff9640 1 -- 172.21.15.32:0/947665382 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+16672 (secure 0 0 0) 0x7fa31005ca60 con 0x7fa31815d560 2024-04-24T23:03:47.685 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:03:47.685 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":92,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:03:46.371395+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:02:58.930222+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-24T23:02:59.607070+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","full","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["backfillfull","exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["backfillfull","exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:03:47.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.685+0000 7fa31e742640 1 -- 172.21.15.32:0/947665382 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa2f806be30 msgr2=0x7fa2f806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:47.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.685+0000 7fa31e742640 1 --2- 172.21.15.32:0/947665382 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa2f806be30 0x7fa2f806e2f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa300034000 tx=0x7fa30002e000 comp rx=0 tx=0).stop 2024-04-24T23:03:47.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.685+0000 7fa31e742640 1 -- 172.21.15.32:0/947665382 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa31815d560 msgr2=0x7fa318151e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:47.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.685+0000 7fa31e742640 1 --2- 172.21.15.32:0/947665382 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa31815d560 0x7fa318151e90 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fa31000c7d0 tx=0x7fa31000cc90 comp rx=0 tx=0).stop 2024-04-24T23:03:47.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.685+0000 7fa31e742640 1 -- 172.21.15.32:0/947665382 shutdown_connections 2024-04-24T23:03:47.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.685+0000 7fa31e742640 1 --2- 172.21.15.32:0/947665382 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa2f806be30 0x7fa2f806e2f0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:47.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.685+0000 7fa31e742640 1 --2- 172.21.15.32:0/947665382 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa31815d560 0x7fa318151e90 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:47.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.685+0000 7fa31e742640 1 --2- 172.21.15.32:0/947665382 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa31815cc40 0x7fa3180cf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:47.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.685+0000 7fa31e742640 1 --2- 172.21.15.32:0/947665382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa3180d05b0 0x7fa31815b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:47.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.685+0000 7fa31e742640 1 -- 172.21.15.32:0/947665382 >> 172.21.15.32:0/947665382 conn(0x7fa3180d0ae0 msgr2=0x7fa3181505b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:47.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.686+0000 7fa31e742640 1 -- 172.21.15.32:0/947665382 shutdown_connections 2024-04-24T23:03:47.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.686+0000 7fa31e742640 1 -- 172.21.15.32:0/947665382 wait complete. 2024-04-24T23:03:47.698 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-24T23:03:47.698 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:03:47.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.797+0000 7fa87997e640 1 Processor -- start 2024-04-24T23:03:47.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.797+0000 7fa87997e640 1 -- start start 2024-04-24T23:03:47.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.798+0000 7fa87997e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa87406b820 0x7fa8740d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:47.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.798+0000 7fa87997e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa8740d4210 0x7fa8740b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:47.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.798+0000 7fa87997e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8740d3170 con 0x7fa87415e9b0 2024-04-24T23:03:47.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.798+0000 7fa87997e640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fa8740d3450 con 0x7fa8740d4210 2024-04-24T23:03:47.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.798+0000 7fa87997e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa8740d3730 con 0x7fa87406b820 2024-04-24T23:03:47.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.798+0000 7fa8737fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa8740d4210 0x7fa8740b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:47.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.798+0000 7fa8737fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa8740d4210 0x7fa8740b78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53318/0 (socket says 172.21.15.32:53318) 2024-04-24T23:03:47.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.798+0000 7fa872ffd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa87415e9b0 0x7fa87406b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55036/0 (socket says 172.21.15.32:55036) 2024-04-24T23:03:47.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.798+0000 7fa8737fe640 1 -- 172.21.15.32:0/430181193 learned_addr learned my addr 172.21.15.32:0/430181193 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:47.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.799+0000 7fa8727fc640 1 --2- 172.21.15.32:0/430181193 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa87406b820 0x7fa8740d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:47.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.799+0000 7fa8737fe640 1 -- 172.21.15.32:0/430181193 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa87406b820 msgr2=0x7fa8740d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:47.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.799+0000 7fa8737fe640 1 --2- 172.21.15.32:0/430181193 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa87406b820 0x7fa8740d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:47.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.799+0000 7fa8737fe640 1 -- 172.21.15.32:0/430181193 >> v1:172.21.15.32:6789/0 conn(0x7fa87415e9b0 legacy=0x7fa87406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:47.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.799+0000 7fa8737fe640 1 -- 172.21.15.32:0/430181193 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8740d3a10 con 0x7fa8740d4210 2024-04-24T23:03:47.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.799+0000 7fa871ffb640 1 -- 172.21.15.32:0/430181193 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1168076382 0 0) 0x7fa8740d3170 con 0x7fa87415e9b0 2024-04-24T23:03:47.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.799+0000 7fa8737fe640 1 --2- 172.21.15.32:0/430181193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8740d4210 0x7fa8740b78f0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fa86c009870 tx=0x7fa86c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88263ef46d47a760 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:47.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.800+0000 7fa871ffb640 1 -- 172.21.15.32:0/430181193 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86c037070 con 0x7fa8740d4210 2024-04-24T23:03:47.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.800+0000 7fa871ffb640 1 -- 172.21.15.32:0/430181193 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa86c032440 con 0x7fa8740d4210 2024-04-24T23:03:47.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.800+0000 7fa871ffb640 1 -- 172.21.15.32:0/430181193 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86c032810 con 0x7fa8740d4210 2024-04-24T23:03:47.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.800+0000 7fa87997e640 1 -- 172.21.15.32:0/430181193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8740d4210 msgr2=0x7fa8740b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:47.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.800+0000 7fa87997e640 1 --2- 172.21.15.32:0/430181193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8740d4210 0x7fa8740b78f0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fa86c009870 tx=0x7fa86c02a270 comp rx=0 tx=0).stop 2024-04-24T23:03:47.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.800+0000 7fa87997e640 1 -- 172.21.15.32:0/430181193 shutdown_connections 2024-04-24T23:03:47.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.800+0000 7fa87997e640 1 --2- 172.21.15.32:0/430181193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8740d4210 0x7fa8740b78f0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:47.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.800+0000 7fa87997e640 1 --2- 172.21.15.32:0/430181193 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa87406b820 0x7fa8740d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:47.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.800+0000 7fa87997e640 1 -- 172.21.15.32:0/430181193 >> 172.21.15.32:0/430181193 conn(0x7fa8740d11d0 msgr2=0x7fa8740d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:47.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.800+0000 7fa87997e640 1 -- 172.21.15.32:0/430181193 shutdown_connections 2024-04-24T23:03:47.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.800+0000 7fa87997e640 1 -- 172.21.15.32:0/430181193 wait complete. 2024-04-24T23:03:47.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.801+0000 7fa87997e640 1 Processor -- start 2024-04-24T23:03:47.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.801+0000 7fa87997e640 1 -- start start 2024-04-24T23:03:47.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.802+0000 7fa87997e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa87406b820 0x7fa87415bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:47.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.802+0000 7fa87997e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa8740d4210 0x7fa8741564b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:47.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.802+0000 7fa87997e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa87415e9b0 0x7fa8740cb820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:47.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.802+0000 7fa87997e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa8740ba0e0 con 0x7fa87406b820 2024-04-24T23:03:47.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.802+0000 7fa87997e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa8740d2400 con 0x7fa87415e9b0 2024-04-24T23:03:47.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.802+0000 7fa87997e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa8740ba260 con 0x7fa8740d4210 2024-04-24T23:03:47.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.802+0000 7fa872ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa87406b820 0x7fa87415bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:47.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.802+0000 7fa872ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa87406b820 0x7fa87415bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53322/0 (socket says 172.21.15.32:53322) 2024-04-24T23:03:47.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.802+0000 7fa872ffd640 1 -- 172.21.15.32:0/2969874444 learned_addr learned my addr 172.21.15.32:0/2969874444 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:47.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.803+0000 7fa8727fc640 1 --2- 172.21.15.32:0/2969874444 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa8740d4210 0x7fa8741564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:47.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.803+0000 7fa8737fe640 1 --2- 172.21.15.32:0/2969874444 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa87415e9b0 0x7fa8740cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:47.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.803+0000 7fa872ffd640 1 -- 172.21.15.32:0/2969874444 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa8740d4210 msgr2=0x7fa8741564b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:47.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.803+0000 7fa872ffd640 1 --2- 172.21.15.32:0/2969874444 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa8740d4210 0x7fa8741564b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:47.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.803+0000 7fa872ffd640 1 -- 172.21.15.32:0/2969874444 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa87415e9b0 msgr2=0x7fa8740cb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:47.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.803+0000 7fa872ffd640 1 --2- 172.21.15.32:0/2969874444 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa87415e9b0 0x7fa8740cb820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:47.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.803+0000 7fa872ffd640 1 -- 172.21.15.32:0/2969874444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa87415c240 con 0x7fa87406b820 2024-04-24T23:03:47.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.803+0000 7fa8727fc640 1 --2- 172.21.15.32:0/2969874444 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa8740d4210 0x7fa8741564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:03:47.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.803+0000 7fa872ffd640 1 --2- 172.21.15.32:0/2969874444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa87406b820 0x7fa87415bbb0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fa86400c7c0 tx=0x7fa86400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:47.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.803+0000 7fa86bfff640 1 -- 172.21.15.32:0/2969874444 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa864018070 con 0x7fa87406b820 2024-04-24T23:03:47.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.803+0000 7fa86bfff640 1 -- 172.21.15.32:0/2969874444 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa86400a5e0 con 0x7fa87406b820 2024-04-24T23:03:47.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.803+0000 7fa86bfff640 1 -- 172.21.15.32:0/2969874444 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86400a9d0 con 0x7fa87406b820 2024-04-24T23:03:47.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.803+0000 7fa87997e640 1 -- 172.21.15.32:0/2969874444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa874159bb0 con 0x7fa87406b820 2024-04-24T23:03:47.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.803+0000 7fa87997e640 1 -- 172.21.15.32:0/2969874444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa87415a170 con 0x7fa87406b820 2024-04-24T23:03:47.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.804+0000 7fa87997e640 1 -- 172.21.15.32:0/2969874444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa874068430 con 0x7fa87406b820 2024-04-24T23:03:47.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.805+0000 7fa86bfff640 1 -- 172.21.15.32:0/2969874444 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa86401b070 con 0x7fa87406b820 2024-04-24T23:03:47.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.805+0000 7fa86bfff640 1 --2- 172.21.15.32:0/2969874444 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa85406be30 0x7fa85406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:47.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.805+0000 7fa86bfff640 1 -- 172.21.15.32:0/2969874444 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7fa86408a3d0 con 0x7fa87406b820 2024-04-24T23:03:47.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.805+0000 7fa86bfff640 1 -- 172.21.15.32:0/2969874444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=93}) -- 0x7fa854073960 con 0x7fa87406b820 2024-04-24T23:03:47.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.805+0000 7fa8727fc640 1 --2- 172.21.15.32:0/2969874444 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa85406be30 0x7fa85406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:47.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.806+0000 7fa8727fc640 1 --2- 172.21.15.32:0/2969874444 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa85406be30 0x7fa85406e2f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa8740b99c0 tx=0x7fa85c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:47.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:47.809+0000 7fa86bfff640 1 -- 172.21.15.32:0/2969874444 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa864058e10 con 0x7fa87406b820 2024-04-24T23:03:48.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.034+0000 7fa87997e640 1 -- 172.21.15.32:0/2969874444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa8740c7e70 con 0x7fa87406b820 2024-04-24T23:03:48.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.034+0000 7fa86bfff640 1 -- 172.21.15.32:0/2969874444 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5186 (secure 0 0 0) 0x7fa864058710 con 0x7fa87406b820 2024-04-24T23:03:48.036 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:03:48.036 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":36,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5486,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/3550206194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3550206194},{"type":"v1","addr":"172.21.15.32:6834","nonce":3550206194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5905,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1706188546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1706188546},{"type":"v1","addr":"172.21.15.121:6835","nonce":1706188546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5911,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2669443498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2669443498},{"type":"v1","addr":"172.21.15.121:6837","nonce":2669443498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:03:00.630938+0000","modified":"2024-04-24T23:03:03.639570+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6474":{"gid":6474,"name":"a","rank":0,"incarnation":33,"state":"up:active","state_seq":4,"addr":"172.21.15.32:6835/464246210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":464246210},{"type":"v1","addr":"172.21.15.32:6835","nonce":464246210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6474,"qdb_cluster":[6474]},"id":5}]} 2024-04-24T23:03:48.036 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 36 2024-04-24T23:03:48.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.036+0000 7fa87997e640 1 -- 172.21.15.32:0/2969874444 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa85406be30 msgr2=0x7fa85406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:48.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.036+0000 7fa87997e640 1 --2- 172.21.15.32:0/2969874444 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa85406be30 0x7fa85406e2f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa8740b99c0 tx=0x7fa85c008040 comp rx=0 tx=0).stop 2024-04-24T23:03:48.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.036+0000 7fa87997e640 1 -- 172.21.15.32:0/2969874444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa87406b820 msgr2=0x7fa87415bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:48.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.036+0000 7fa87997e640 1 --2- 172.21.15.32:0/2969874444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa87406b820 0x7fa87415bbb0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fa86400c7c0 tx=0x7fa86400cc80 comp rx=0 tx=0).stop 2024-04-24T23:03:48.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.036+0000 7fa87997e640 1 -- 172.21.15.32:0/2969874444 shutdown_connections 2024-04-24T23:03:48.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.036+0000 7fa87997e640 1 --2- 172.21.15.32:0/2969874444 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa85406be30 0x7fa85406e2f0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:48.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.036+0000 7fa87997e640 1 --2- 172.21.15.32:0/2969874444 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa87415e9b0 0x7fa8740cb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:48.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.036+0000 7fa87997e640 1 --2- 172.21.15.32:0/2969874444 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa8740d4210 0x7fa8741564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:48.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.036+0000 7fa87997e640 1 --2- 172.21.15.32:0/2969874444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa87406b820 0x7fa87415bbb0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:48.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.036+0000 7fa87997e640 1 -- 172.21.15.32:0/2969874444 >> 172.21.15.32:0/2969874444 conn(0x7fa8740d11d0 msgr2=0x7fa87406b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:48.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.036+0000 7fa87997e640 1 -- 172.21.15.32:0/2969874444 shutdown_connections 2024-04-24T23:03:48.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.036+0000 7fa87997e640 1 -- 172.21.15.32:0/2969874444 wait complete. 2024-04-24T23:03:48.049 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:03:48.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.147+0000 7f7d3c47b640 1 Processor -- start 2024-04-24T23:03:48.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.147+0000 7f7d3c47b640 1 -- start start 2024-04-24T23:03:48.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.148+0000 7f7d3c47b640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7d3415b380 0x7f7d3406c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:48.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.148+0000 7f7d3c47b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7d340ceb40 0x7f7d3406cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:48.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.148+0000 7f7d3c47b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d340cef50 con 0x7f7d3415cb00 2024-04-24T23:03:48.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.148+0000 7f7d3c47b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f7d340b4520 con 0x7f7d340ceb40 2024-04-24T23:03:48.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.148+0000 7f7d3c47b640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f7d340b4800 con 0x7f7d3415b380 2024-04-24T23:03:48.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.148+0000 7f7d3aa18640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7d340ceb40 0x7f7d3406cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:48.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.148+0000 7f7d39a16640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7d3415cb00 0x7f7d340ce430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55040/0 (socket says 172.21.15.32:55040) 2024-04-24T23:03:48.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.148+0000 7f7d39a16640 1 -- 172.21.15.32:0/2667448630 learned_addr learned my addr 172.21.15.32:0/2667448630 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:03:48.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.148+0000 7f7d3a217640 1 --2- 172.21.15.32:0/2667448630 >> v2:172.21.15.121:3300/0 conn(0x7f7d3415b380 0x7f7d3406c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:48.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.148+0000 7f7d3aa18640 1 -- 172.21.15.32:0/2667448630 >> v2:172.21.15.121:3300/0 conn(0x7f7d3415b380 msgr2=0x7f7d3406c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:48.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.148+0000 7f7d3aa18640 1 --2- 172.21.15.32:0/2667448630 >> v2:172.21.15.121:3300/0 conn(0x7f7d3415b380 0x7f7d3406c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:48.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.148+0000 7f7d3aa18640 1 -- 172.21.15.32:0/2667448630 >> v1:172.21.15.32:6789/0 conn(0x7f7d3415cb00 legacy=0x7f7d340ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:48.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.149+0000 7f7d3aa18640 1 -- 172.21.15.32:0/2667448630 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d340b4ae0 con 0x7f7d340ceb40 2024-04-24T23:03:48.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.148+0000 7f7d39215640 1 -- 172.21.15.32:0/2667448630 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3425693781 0 0) 0x7f7d28001110 con 0x7f7d3415cb00 2024-04-24T23:03:48.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.149+0000 7f7d3aa18640 1 --2- 172.21.15.32:0/2667448630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d340ceb40 0x7f7d3406cb30 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f7d3406eed0 tx=0x7f7d2402c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c186e63d78d31ba1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:48.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.149+0000 7f7d39215640 1 -- 172.21.15.32:0/2667448630 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d24038070 con 0x7f7d340ceb40 2024-04-24T23:03:48.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.149+0000 7f7d39215640 1 -- 172.21.15.32:0/2667448630 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d24009540 con 0x7f7d340ceb40 2024-04-24T23:03:48.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.149+0000 7f7d39215640 1 -- 172.21.15.32:0/2667448630 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d24009910 con 0x7f7d340ceb40 2024-04-24T23:03:48.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.150+0000 7f7d3c47b640 1 -- 172.21.15.32:0/2667448630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d340ceb40 msgr2=0x7f7d3406cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:48.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.150+0000 7f7d3c47b640 1 --2- 172.21.15.32:0/2667448630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d340ceb40 0x7f7d3406cb30 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f7d3406eed0 tx=0x7f7d2402c760 comp rx=0 tx=0).stop 2024-04-24T23:03:48.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.150+0000 7f7d3c47b640 1 -- 172.21.15.32:0/2667448630 shutdown_connections 2024-04-24T23:03:48.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.150+0000 7f7d3c47b640 1 --2- 172.21.15.32:0/2667448630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d340ceb40 0x7f7d3406cb30 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:48.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.150+0000 7f7d3c47b640 1 --2- 172.21.15.32:0/2667448630 >> v2:172.21.15.121:3300/0 conn(0x7f7d3415b380 0x7f7d3406c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:48.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.150+0000 7f7d3c47b640 1 -- 172.21.15.32:0/2667448630 >> 172.21.15.32:0/2667448630 conn(0x7f7d3406b110 msgr2=0x7f7d3406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:48.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.150+0000 7f7d3c47b640 1 -- 172.21.15.32:0/2667448630 shutdown_connections 2024-04-24T23:03:48.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.150+0000 7f7d3c47b640 1 -- 172.21.15.32:0/2667448630 wait complete. 2024-04-24T23:03:48.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.151+0000 7f7d3c47b640 1 Processor -- start 2024-04-24T23:03:48.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.151+0000 7f7d3c47b640 1 -- start start 2024-04-24T23:03:48.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.152+0000 7f7d3c47b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d340ceb40 0x7f7d3414aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:48.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.152+0000 7f7d3c47b640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d3415b380 0x7f7d340ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:48.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.152+0000 7f7d3c47b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d3415cb00 0x7f7d340aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:48.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.152+0000 7f7d3c47b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3415b760 con 0x7f7d3415cb00 2024-04-24T23:03:48.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.152+0000 7f7d3c47b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7d340b50a0 con 0x7f7d340ceb40 2024-04-24T23:03:48.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.152+0000 7f7d3c47b640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f7d340b55c0 con 0x7f7d3415b380 2024-04-24T23:03:48.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.152+0000 7f7d3aa18640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d3415cb00 0x7f7d340aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:48.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.152+0000 7f7d3aa18640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d3415cb00 0x7f7d340aedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53332/0 (socket says 172.21.15.32:53332) 2024-04-24T23:03:48.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.152+0000 7f7d3aa18640 1 -- 172.21.15.32:0/183039080 learned_addr learned my addr 172.21.15.32:0/183039080 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:48.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.152+0000 7f7d39a16640 1 --2- 172.21.15.32:0/183039080 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d3415b380 0x7f7d340ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:48.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.152+0000 7f7d3a217640 1 --2- 172.21.15.32:0/183039080 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d340ceb40 0x7f7d3414aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:48.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.152+0000 7f7d3aa18640 1 -- 172.21.15.32:0/183039080 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d3415b380 msgr2=0x7f7d340ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:48.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.152+0000 7f7d3aa18640 1 --2- 172.21.15.32:0/183039080 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d3415b380 0x7f7d340ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:48.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.152+0000 7f7d3aa18640 1 -- 172.21.15.32:0/183039080 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d340ceb40 msgr2=0x7f7d3414aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:48.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.152+0000 7f7d3aa18640 1 --2- 172.21.15.32:0/183039080 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d340ceb40 0x7f7d3414aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:48.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.152+0000 7f7d3aa18640 1 -- 172.21.15.32:0/183039080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d341529a0 con 0x7f7d3415cb00 2024-04-24T23:03:48.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.153+0000 7f7d3aa18640 1 --2- 172.21.15.32:0/183039080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d3415cb00 0x7f7d340aedf0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f7d2400d140 tx=0x7f7d24004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:48.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.153+0000 7f7d32ffd640 1 -- 172.21.15.32:0/183039080 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d24038040 con 0x7f7d3415cb00 2024-04-24T23:03:48.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.153+0000 7f7d32ffd640 1 -- 172.21.15.32:0/183039080 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d240092b0 con 0x7f7d3415cb00 2024-04-24T23:03:48.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.153+0000 7f7d32ffd640 1 -- 172.21.15.32:0/183039080 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2403e870 con 0x7f7d3415cb00 2024-04-24T23:03:48.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.153+0000 7f7d3c47b640 1 -- 172.21.15.32:0/183039080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d341f4e60 con 0x7f7d3415cb00 2024-04-24T23:03:48.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.153+0000 7f7d3c47b640 1 -- 172.21.15.32:0/183039080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d3415a720 con 0x7f7d3415cb00 2024-04-24T23:03:48.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.154+0000 7f7d3c47b640 1 -- 172.21.15.32:0/183039080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d34068430 con 0x7f7d3415cb00 2024-04-24T23:03:48.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.154+0000 7f7d32ffd640 1 -- 172.21.15.32:0/183039080 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7d2403ea10 con 0x7f7d3415cb00 2024-04-24T23:03:48.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.155+0000 7f7d32ffd640 1 --2- 172.21.15.32:0/183039080 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7d2006be30 0x7f7d2006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:48.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.155+0000 7f7d32ffd640 1 -- 172.21.15.32:0/183039080 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f7d240abb10 con 0x7f7d3415cb00 2024-04-24T23:03:48.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.155+0000 7f7d3a217640 1 --2- 172.21.15.32:0/183039080 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7d2006be30 0x7f7d2006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:48.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.155+0000 7f7d32ffd640 1 -- 172.21.15.32:0/183039080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=93}) -- 0x7f7d20073960 con 0x7f7d3415cb00 2024-04-24T23:03:48.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.156+0000 7f7d3a217640 1 --2- 172.21.15.32:0/183039080 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7d2006be30 0x7f7d2006e2f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c006fd0 tx=0x7f7d2c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:48.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.159+0000 7f7d32ffd640 1 -- 172.21.15.32:0/183039080 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7d341529a0 con 0x7f7d3415cb00 2024-04-24T23:03:48.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.377+0000 7f7d32ffd640 1 -- 172.21.15.32:0/183039080 <== mon.0 v2:172.21.15.32:3300/0 7 ==== osd_map(93..93 src has 1..93) ==== 1484+0+0 (secure 0 0 0) 0x7f7d240aa150 con 0x7f7d3415cb00 2024-04-24T23:03:48.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.377+0000 7f7d32ffd640 1 -- 172.21.15.32:0/183039080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=94}) -- 0x7f7d20074af0 con 0x7f7d3415cb00 2024-04-24T23:03:48.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.384+0000 7f7d3c47b640 1 -- 172.21.15.32:0/183039080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d3406c000 con 0x7f7d3415cb00 2024-04-24T23:03:48.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.385+0000 7f7d32ffd640 1 -- 172.21.15.32:0/183039080 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5186 (secure 0 0 0) 0x7f7d2403ed00 con 0x7f7d3415cb00 2024-04-24T23:03:48.388 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:03:48.388 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":36,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5486,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/3550206194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3550206194},{"type":"v1","addr":"172.21.15.32:6834","nonce":3550206194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5905,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1706188546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1706188546},{"type":"v1","addr":"172.21.15.121:6835","nonce":1706188546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5911,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2669443498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2669443498},{"type":"v1","addr":"172.21.15.121:6837","nonce":2669443498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:03:00.630938+0000","modified":"2024-04-24T23:03:03.639570+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6474":{"gid":6474,"name":"a","rank":0,"incarnation":33,"state":"up:active","state_seq":4,"addr":"172.21.15.32:6835/464246210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":464246210},{"type":"v1","addr":"172.21.15.32:6835","nonce":464246210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6474,"qdb_cluster":[6474]},"id":5}]} 2024-04-24T23:03:48.389 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 36 2024-04-24T23:03:48.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.387+0000 7f7d3c47b640 1 -- 172.21.15.32:0/183039080 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7d2006be30 msgr2=0x7f7d2006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:48.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.387+0000 7f7d3c47b640 1 --2- 172.21.15.32:0/183039080 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7d2006be30 0x7f7d2006e2f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c006fd0 tx=0x7f7d2c008040 comp rx=0 tx=0).stop 2024-04-24T23:03:48.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.387+0000 7f7d3c47b640 1 -- 172.21.15.32:0/183039080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d3415cb00 msgr2=0x7f7d340aedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:48.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.387+0000 7f7d3c47b640 1 --2- 172.21.15.32:0/183039080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d3415cb00 0x7f7d340aedf0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f7d2400d140 tx=0x7f7d24004a40 comp rx=0 tx=0).stop 2024-04-24T23:03:48.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.387+0000 7f7d3c47b640 1 -- 172.21.15.32:0/183039080 shutdown_connections 2024-04-24T23:03:48.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.387+0000 7f7d3c47b640 1 --2- 172.21.15.32:0/183039080 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7d2006be30 0x7f7d2006e2f0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:48.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.387+0000 7f7d3c47b640 1 --2- 172.21.15.32:0/183039080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d3415cb00 0x7f7d340aedf0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:48.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.387+0000 7f7d3c47b640 1 --2- 172.21.15.32:0/183039080 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d3415b380 0x7f7d340ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:48.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.387+0000 7f7d3c47b640 1 --2- 172.21.15.32:0/183039080 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d340ceb40 0x7f7d3414aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:48.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.387+0000 7f7d3c47b640 1 -- 172.21.15.32:0/183039080 >> 172.21.15.32:0/183039080 conn(0x7f7d3406b110 msgr2=0x7f7d3406c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:48.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.387+0000 7f7d3c47b640 1 -- 172.21.15.32:0/183039080 shutdown_connections 2024-04-24T23:03:48.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.387+0000 7f7d3c47b640 1 -- 172.21.15.32:0/183039080 wait complete. 2024-04-24T23:03:48.401 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:03:48.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.499+0000 7fc4f0b4f640 1 Processor -- start 2024-04-24T23:03:48.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.499+0000 7fc4f0b4f640 1 -- start start 2024-04-24T23:03:48.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.500+0000 7fc4f0b4f640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc4ec15fd60 0x7fc4ec06c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:48.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.500+0000 7fc4f0b4f640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc4ec06cb30 0x7fc4ec0cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:48.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.500+0000 7fc4f0b4f640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4ec0b46c0 con 0x7fc4ec15b480 2024-04-24T23:03:48.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.500+0000 7fc4f0b4f640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fc4ec0b4ff0 con 0x7fc4ec15fd60 2024-04-24T23:03:48.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.500+0000 7fc4f0b4f640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc4ec0b49a0 con 0x7fc4ec06cb30 2024-04-24T23:03:48.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.500+0000 7fc4e9d9b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc4ec15fd60 0x7fc4ec06c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:48.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.500+0000 7fc4e9d9b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc4ec15fd60 0x7fc4ec06c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53346/0 (socket says 172.21.15.32:53346) 2024-04-24T23:03:48.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.500+0000 7fc4e9d9b640 1 -- 172.21.15.32:0/1963913113 learned_addr learned my addr 172.21.15.32:0/1963913113 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:48.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.500+0000 7fc4ead9d640 1 --2- 172.21.15.32:0/1963913113 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc4ec06cb30 0x7fc4ec0cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:48.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.501+0000 7fc4e9d9b640 1 -- 172.21.15.32:0/1963913113 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc4ec06cb30 msgr2=0x7fc4ec0cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:48.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.501+0000 7fc4e9d9b640 1 --2- 172.21.15.32:0/1963913113 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc4ec06cb30 0x7fc4ec0cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:48.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.501+0000 7fc4e9d9b640 1 -- 172.21.15.32:0/1963913113 >> v1:172.21.15.121:6789/0 conn(0x7fc4ec15b480 legacy=0x7fc4ec0b5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:48.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.501+0000 7fc4e9d9b640 1 -- 172.21.15.32:0/1963913113 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4ec0b4c80 con 0x7fc4ec15fd60 2024-04-24T23:03:48.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.501+0000 7fc4ead9d640 1 --2- 172.21.15.32:0/1963913113 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc4ec06cb30 0x7fc4ec0cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:03:48.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.501+0000 7fc4e9d9b640 1 --2- 172.21.15.32:0/1963913113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4ec15fd60 0x7fc4ec06c5f0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fc4d400bb10 tx=0x7fc4d402c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cff8f36882aa56d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:48.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.501+0000 7fc4e959a640 1 -- 172.21.15.32:0/1963913113 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4032790 con 0x7fc4ec15fd60 2024-04-24T23:03:48.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.501+0000 7fc4e959a640 1 -- 172.21.15.32:0/1963913113 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4d4038070 con 0x7fc4ec15fd60 2024-04-24T23:03:48.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.501+0000 7fc4e959a640 1 -- 172.21.15.32:0/1963913113 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4032b50 con 0x7fc4ec15fd60 2024-04-24T23:03:48.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.502+0000 7fc4f0b4f640 1 -- 172.21.15.32:0/1963913113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4ec15fd60 msgr2=0x7fc4ec06c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:48.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.502+0000 7fc4f0b4f640 1 --2- 172.21.15.32:0/1963913113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4ec15fd60 0x7fc4ec06c5f0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fc4d400bb10 tx=0x7fc4d402c8d0 comp rx=0 tx=0).stop 2024-04-24T23:03:48.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.502+0000 7fc4f0b4f640 1 -- 172.21.15.32:0/1963913113 shutdown_connections 2024-04-24T23:03:48.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.502+0000 7fc4f0b4f640 1 --2- 172.21.15.32:0/1963913113 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc4ec06cb30 0x7fc4ec0cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:48.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.502+0000 7fc4f0b4f640 1 --2- 172.21.15.32:0/1963913113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4ec15fd60 0x7fc4ec06c5f0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:48.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.502+0000 7fc4f0b4f640 1 -- 172.21.15.32:0/1963913113 >> 172.21.15.32:0/1963913113 conn(0x7fc4ec06b110 msgr2=0x7fc4ec06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:48.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.502+0000 7fc4f0b4f640 1 -- 172.21.15.32:0/1963913113 shutdown_connections 2024-04-24T23:03:48.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.502+0000 7fc4f0b4f640 1 -- 172.21.15.32:0/1963913113 wait complete. 2024-04-24T23:03:48.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.503+0000 7fc4f0b4f640 1 Processor -- start 2024-04-24T23:03:48.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.503+0000 7fc4f0b4f640 1 -- start start 2024-04-24T23:03:48.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.504+0000 7fc4f0b4f640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc4ec06cb30 0x7fc4ec158eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:48.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.504+0000 7fc4f0b4f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4ec15b480 0x7fc4ec141370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:48.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.504+0000 7fc4f0b4f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4ec15fd60 0x7fc4ec153d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:48.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.504+0000 7fc4f0b4f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ec15b860 con 0x7fc4ec15fd60 2024-04-24T23:03:48.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.504+0000 7fc4f0b4f640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ec0ce4c0 con 0x7fc4ec15b480 2024-04-24T23:03:48.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.504+0000 7fc4f0b4f640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc4ec15bb70 con 0x7fc4ec06cb30 2024-04-24T23:03:48.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.504+0000 7fc4ead9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4ec15fd60 0x7fc4ec153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:48.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.504+0000 7fc4ead9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4ec15fd60 0x7fc4ec153d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53350/0 (socket says 172.21.15.32:53350) 2024-04-24T23:03:48.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.504+0000 7fc4ead9d640 1 -- 172.21.15.32:0/2147924003 learned_addr learned my addr 172.21.15.32:0/2147924003 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:48.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.504+0000 7fc4e9d9b640 1 --2- 172.21.15.32:0/2147924003 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4ec15b480 0x7fc4ec141370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:48.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.504+0000 7fc4ea59c640 1 --2- 172.21.15.32:0/2147924003 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc4ec06cb30 0x7fc4ec158eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:48.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.504+0000 7fc4ead9d640 1 -- 172.21.15.32:0/2147924003 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc4ec06cb30 msgr2=0x7fc4ec158eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:48.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.504+0000 7fc4ead9d640 1 --2- 172.21.15.32:0/2147924003 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc4ec06cb30 0x7fc4ec158eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:48.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.504+0000 7fc4ead9d640 1 -- 172.21.15.32:0/2147924003 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4ec15b480 msgr2=0x7fc4ec141370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:48.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.504+0000 7fc4ead9d640 1 --2- 172.21.15.32:0/2147924003 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4ec15b480 0x7fc4ec141370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:48.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.505+0000 7fc4ead9d640 1 -- 172.21.15.32:0/2147924003 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4ec15bdc0 con 0x7fc4ec15fd60 2024-04-24T23:03:48.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.505+0000 7fc4ea59c640 1 --2- 172.21.15.32:0/2147924003 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc4ec06cb30 0x7fc4ec158eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:03:48.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.505+0000 7fc4ead9d640 1 --2- 172.21.15.32:0/2147924003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4ec15fd60 0x7fc4ec153d20 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc00cef0 tx=0x7fc4dc00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:48.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.505+0000 7fc4e2ffd640 1 -- 172.21.15.32:0/2147924003 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc01a070 con 0x7fc4ec15fd60 2024-04-24T23:03:48.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.505+0000 7fc4e2ffd640 1 -- 172.21.15.32:0/2147924003 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4dc00d860 con 0x7fc4ec15fd60 2024-04-24T23:03:48.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.505+0000 7fc4f0b4f640 1 -- 172.21.15.32:0/2147924003 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4ec15c0a0 con 0x7fc4ec15fd60 2024-04-24T23:03:48.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.505+0000 7fc4e2ffd640 1 -- 172.21.15.32:0/2147924003 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc00dc50 con 0x7fc4ec15fd60 2024-04-24T23:03:48.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.505+0000 7fc4f0b4f640 1 -- 172.21.15.32:0/2147924003 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4ec15c660 con 0x7fc4ec15fd60 2024-04-24T23:03:48.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.506+0000 7fc4f0b4f640 1 -- 172.21.15.32:0/2147924003 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4ec068430 con 0x7fc4ec15fd60 2024-04-24T23:03:48.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.506+0000 7fc4e2ffd640 1 -- 172.21.15.32:0/2147924003 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc4dc01d070 con 0x7fc4ec15fd60 2024-04-24T23:03:48.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.507+0000 7fc4e2ffd640 1 --2- 172.21.15.32:0/2147924003 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc4cc06be30 0x7fc4cc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:48.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.507+0000 7fc4e2ffd640 1 -- 172.21.15.32:0/2147924003 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5709+0+0 (secure 0 0 0) 0x7fc4dc08b700 con 0x7fc4ec15fd60 2024-04-24T23:03:48.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.507+0000 7fc4ea59c640 1 --2- 172.21.15.32:0/2147924003 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc4cc06be30 0x7fc4cc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:48.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.507+0000 7fc4ea59c640 1 --2- 172.21.15.32:0/2147924003 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc4cc06be30 0x7fc4cc06e2f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4003eb0 tx=0x7fc4e4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:48.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.511+0000 7fc4e2ffd640 1 -- 172.21.15.32:0/2147924003 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc4dc05a170 con 0x7fc4ec15fd60 2024-04-24T23:03:48.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.698+0000 7fc4f0b4f640 1 -- 172.21.15.32:0/2147924003 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc4ec06c000 con 0x7fc4ec15fd60 2024-04-24T23:03:48.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.699+0000 7fc4e2ffd640 1 -- 172.21.15.32:0/2147924003 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+16620 (secure 0 0 0) 0x7fc4dc059a70 con 0x7fc4ec15fd60 2024-04-24T23:03:48.702 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:03:48.702 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":93,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:03:48.377702+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:02:58.930222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-24T23:02:59.607070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:03:48.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.701+0000 7fc4f0b4f640 1 -- 172.21.15.32:0/2147924003 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc4cc06be30 msgr2=0x7fc4cc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:48.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.701+0000 7fc4f0b4f640 1 --2- 172.21.15.32:0/2147924003 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc4cc06be30 0x7fc4cc06e2f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4003eb0 tx=0x7fc4e4009040 comp rx=0 tx=0).stop 2024-04-24T23:03:48.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.701+0000 7fc4f0b4f640 1 -- 172.21.15.32:0/2147924003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4ec15fd60 msgr2=0x7fc4ec153d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:48.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.701+0000 7fc4f0b4f640 1 --2- 172.21.15.32:0/2147924003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4ec15fd60 0x7fc4ec153d20 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc00cef0 tx=0x7fc4dc00a710 comp rx=0 tx=0).stop 2024-04-24T23:03:48.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.701+0000 7fc4f0b4f640 1 -- 172.21.15.32:0/2147924003 shutdown_connections 2024-04-24T23:03:48.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.701+0000 7fc4f0b4f640 1 --2- 172.21.15.32:0/2147924003 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc4cc06be30 0x7fc4cc06e2f0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:48.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.701+0000 7fc4f0b4f640 1 --2- 172.21.15.32:0/2147924003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4ec15fd60 0x7fc4ec153d20 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:48.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.701+0000 7fc4f0b4f640 1 --2- 172.21.15.32:0/2147924003 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4ec15b480 0x7fc4ec141370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:48.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.701+0000 7fc4f0b4f640 1 --2- 172.21.15.32:0/2147924003 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc4ec06cb30 0x7fc4ec158eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:48.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.701+0000 7fc4f0b4f640 1 -- 172.21.15.32:0/2147924003 >> 172.21.15.32:0/2147924003 conn(0x7fc4ec06b110 msgr2=0x7fc4ec0b0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:48.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.701+0000 7fc4f0b4f640 1 -- 172.21.15.32:0/2147924003 shutdown_connections 2024-04-24T23:03:48.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.701+0000 7fc4f0b4f640 1 -- 172.21.15.32:0/2147924003 wait complete. 2024-04-24T23:03:48.714 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-24T23:03:48.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.814+0000 7f21781ee640 1 Processor -- start 2024-04-24T23:03:48.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.814+0000 7f21781ee640 1 -- start start 2024-04-24T23:03:48.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.815+0000 7f21781ee640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f21700b5560 0x7f217015e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:48.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.815+0000 7f21781ee640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f21700cd3c0 0x7f217015ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:48.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.815+0000 7f21781ee640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21700cd910 con 0x7f21700b5940 2024-04-24T23:03:48.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.815+0000 7f21781ee640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f21700b6470 con 0x7f21700cd3c0 2024-04-24T23:03:48.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.815+0000 7f21781ee640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f21700cdbf0 con 0x7f21700b5560 2024-04-24T23:03:48.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.815+0000 7f2175789640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f21700cd3c0 0x7f217015ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:48.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.815+0000 7f217678b640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f21700b5940 0x7f21700b5d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55050/0 (socket says 172.21.15.32:55050) 2024-04-24T23:03:48.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.815+0000 7f2175789640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f21700cd3c0 0x7f217015ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53358/0 (socket says 172.21.15.32:53358) 2024-04-24T23:03:48.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.815+0000 7f217678b640 1 -- 172.21.15.32:0/1794112956 learned_addr learned my addr 172.21.15.32:0/1794112956 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:03:48.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.815+0000 7f2175f8a640 1 --2- 172.21.15.32:0/1794112956 >> v2:172.21.15.121:3300/0 conn(0x7f21700b5560 0x7f217015e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:48.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.815+0000 7f2175789640 1 -- 172.21.15.32:0/1794112956 >> v2:172.21.15.121:3300/0 conn(0x7f21700b5560 msgr2=0x7f217015e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:48.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.815+0000 7f2175789640 1 --2- 172.21.15.32:0/1794112956 >> v2:172.21.15.121:3300/0 conn(0x7f21700b5560 0x7f217015e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:48.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.815+0000 7f2175789640 1 -- 172.21.15.32:0/1794112956 >> v1:172.21.15.32:6789/0 conn(0x7f21700b5940 legacy=0x7f21700b5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:48.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.815+0000 7f2175789640 1 -- 172.21.15.32:0/1794112956 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21700cded0 con 0x7f21700cd3c0 2024-04-24T23:03:48.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.815+0000 7f2174f88640 1 -- 172.21.15.32:0/1794112956 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1775501350 0 0) 0x7f21700cd910 con 0x7f21700b5940 2024-04-24T23:03:48.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.815+0000 7f2175f8a640 1 --2- 172.21.15.32:0/1794112956 >> v2:172.21.15.121:3300/0 conn(0x7f21700b5560 0x7f217015e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:03:48.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.816+0000 7f2175789640 1 --2- 172.21.15.32:0/1794112956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21700cd3c0 0x7f217015ec00 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f217006ede0 tx=0x7f216002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85e833ac87928a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:48.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.816+0000 7f2174f88640 1 -- 172.21.15.32:0/1794112956 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2160037070 con 0x7f21700cd3c0 2024-04-24T23:03:48.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.816+0000 7f2174f88640 1 -- 172.21.15.32:0/1794112956 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2160032440 con 0x7f21700cd3c0 2024-04-24T23:03:48.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.816+0000 7f2174f88640 1 -- 172.21.15.32:0/1794112956 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2160032810 con 0x7f21700cd3c0 2024-04-24T23:03:48.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.817+0000 7f21781ee640 1 -- 172.21.15.32:0/1794112956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21700cd3c0 msgr2=0x7f217015ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:48.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.817+0000 7f21781ee640 1 --2- 172.21.15.32:0/1794112956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21700cd3c0 0x7f217015ec00 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f217006ede0 tx=0x7f216002a1a0 comp rx=0 tx=0).stop 2024-04-24T23:03:48.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.817+0000 7f21781ee640 1 -- 172.21.15.32:0/1794112956 shutdown_connections 2024-04-24T23:03:48.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.817+0000 7f21781ee640 1 --2- 172.21.15.32:0/1794112956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21700cd3c0 0x7f217015ec00 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:48.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.817+0000 7f21781ee640 1 --2- 172.21.15.32:0/1794112956 >> v2:172.21.15.121:3300/0 conn(0x7f21700b5560 0x7f217015e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:48.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.817+0000 7f21781ee640 1 -- 172.21.15.32:0/1794112956 >> 172.21.15.32:0/1794112956 conn(0x7f217006b110 msgr2=0x7f217006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:48.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.817+0000 7f21781ee640 1 -- 172.21.15.32:0/1794112956 shutdown_connections 2024-04-24T23:03:48.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.817+0000 7f21781ee640 1 -- 172.21.15.32:0/1794112956 wait complete. 2024-04-24T23:03:48.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.818+0000 7f21781ee640 1 Processor -- start 2024-04-24T23:03:48.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.818+0000 7f21781ee640 1 -- start start 2024-04-24T23:03:48.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.819+0000 7f21781ee640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f21700b5560 0x7f21700b3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:48.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.819+0000 7f21781ee640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21700b5940 0x7f2170157b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:48.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.819+0000 7f21781ee640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21700cd3c0 0x7f2170140a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:48.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.819+0000 7f21781ee640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f217006c5f0 con 0x7f21700b5940 2024-04-24T23:03:48.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.819+0000 7f21781ee640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f21700ce490 con 0x7f21700cd3c0 2024-04-24T23:03:48.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.819+0000 7f21781ee640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f217006c770 con 0x7f21700b5560 2024-04-24T23:03:48.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.819+0000 7f2175789640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21700b5940 0x7f2170157b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:48.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.819+0000 7f2175789640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21700b5940 0x7f2170157b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53372/0 (socket says 172.21.15.32:53372) 2024-04-24T23:03:48.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.819+0000 7f2175789640 1 -- 172.21.15.32:0/3471644079 learned_addr learned my addr 172.21.15.32:0/3471644079 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:48.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.819+0000 7f2175f8a640 1 --2- 172.21.15.32:0/3471644079 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f21700b5560 0x7f21700b3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:48.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.819+0000 7f217678b640 1 --2- 172.21.15.32:0/3471644079 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21700cd3c0 0x7f2170140a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:48.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.819+0000 7f2175789640 1 -- 172.21.15.32:0/3471644079 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f21700b5560 msgr2=0x7f21700b3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:48.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.819+0000 7f2175789640 1 --2- 172.21.15.32:0/3471644079 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f21700b5560 0x7f21700b3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:48.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.819+0000 7f2175789640 1 -- 172.21.15.32:0/3471644079 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21700cd3c0 msgr2=0x7f2170140a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:48.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.819+0000 7f2175789640 1 --2- 172.21.15.32:0/3471644079 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21700cd3c0 0x7f2170140a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:48.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.819+0000 7f2175789640 1 -- 172.21.15.32:0/3471644079 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2170156960 con 0x7f21700b5940 2024-04-24T23:03:48.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.819+0000 7f2175789640 1 --2- 172.21.15.32:0/3471644079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21700b5940 0x7f2170157b60 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f2160000c00 tx=0x7f21600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:48.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.819+0000 7f21667fc640 1 -- 172.21.15.32:0/3471644079 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2160037050 con 0x7f21700b5940 2024-04-24T23:03:48.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.819+0000 7f21667fc640 1 -- 172.21.15.32:0/3471644079 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2160038040 con 0x7f21700b5940 2024-04-24T23:03:48.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.819+0000 7f21781ee640 1 -- 172.21.15.32:0/3471644079 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2170156c40 con 0x7f21700b5940 2024-04-24T23:03:48.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.819+0000 7f21667fc640 1 -- 172.21.15.32:0/3471644079 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2160004450 con 0x7f21700b5940 2024-04-24T23:03:48.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.819+0000 7f21781ee640 1 -- 172.21.15.32:0/3471644079 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2170157200 con 0x7f21700b5940 2024-04-24T23:03:48.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.820+0000 7f21781ee640 1 -- 172.21.15.32:0/3471644079 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2170068430 con 0x7f21700b5940 2024-04-24T23:03:48.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.821+0000 7f21667fc640 1 -- 172.21.15.32:0/3471644079 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2160030980 con 0x7f21700b5940 2024-04-24T23:03:48.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.821+0000 7f21667fc640 1 --2- 172.21.15.32:0/3471644079 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f215406be30 0x7f215406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:48.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.821+0000 7f21667fc640 1 -- 172.21.15.32:0/3471644079 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5709+0+0 (secure 0 0 0) 0x7f21600abb90 con 0x7f21700b5940 2024-04-24T23:03:48.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.821+0000 7f2175f8a640 1 --2- 172.21.15.32:0/3471644079 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f215406be30 0x7f215406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:48.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.822+0000 7f2175f8a640 1 --2- 172.21.15.32:0/3471644079 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f215406be30 0x7f215406e2f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f216c00a990 tx=0x7f216c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:48.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:48.826+0000 7f21667fc640 1 -- 172.21.15.32:0/3471644079 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f216007a660 con 0x7f21700b5940 2024-04-24T23:03:49.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.045+0000 7f21781ee640 1 -- 172.21.15.32:0/3471644079 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f21700b3520 con 0x7f21700b5940 2024-04-24T23:03:49.048 INFO:tasks.ceph.mon.a.smithi032.stderr:2024-04-24T23:03:49.046+0000 7f3b809c6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T23:03:49.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.047+0000 7f21667fc640 1 -- 172.21.15.32:0/3471644079 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v37) ==== 161+0+0 (secure 0 0 0) 0x7f2160079f60 con 0x7f21700b5940 2024-04-24T23:03:49.049 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-24T23:03:49.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.051+0000 7f21781ee640 1 -- 172.21.15.32:0/3471644079 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f215406be30 msgr2=0x7f215406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:49.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.051+0000 7f21781ee640 1 --2- 172.21.15.32:0/3471644079 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f215406be30 0x7f215406e2f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f216c00a990 tx=0x7f216c008040 comp rx=0 tx=0).stop 2024-04-24T23:03:49.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.051+0000 7f21781ee640 1 -- 172.21.15.32:0/3471644079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21700b5940 msgr2=0x7f2170157b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:49.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.051+0000 7f21781ee640 1 --2- 172.21.15.32:0/3471644079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21700b5940 0x7f2170157b60 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f2160000c00 tx=0x7f21600046c0 comp rx=0 tx=0).stop 2024-04-24T23:03:49.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.052+0000 7f21781ee640 1 -- 172.21.15.32:0/3471644079 shutdown_connections 2024-04-24T23:03:49.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.052+0000 7f21781ee640 1 --2- 172.21.15.32:0/3471644079 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f215406be30 0x7f215406e2f0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:49.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.052+0000 7f21781ee640 1 --2- 172.21.15.32:0/3471644079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21700b5940 0x7f2170157b60 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:49.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.052+0000 7f21781ee640 1 --2- 172.21.15.32:0/3471644079 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21700cd3c0 0x7f2170140a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:49.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.052+0000 7f21781ee640 1 --2- 172.21.15.32:0/3471644079 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f21700b5560 0x7f21700b3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:49.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.052+0000 7f21781ee640 1 -- 172.21.15.32:0/3471644079 >> 172.21.15.32:0/3471644079 conn(0x7f217006b110 msgr2=0x7f217006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:49.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.052+0000 7f21781ee640 1 -- 172.21.15.32:0/3471644079 shutdown_connections 2024-04-24T23:03:49.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.052+0000 7f21781ee640 1 -- 172.21.15.32:0/3471644079 wait complete. 2024-04-24T23:03:49.066 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-24T23:03:49.091 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2024-04-24T23:03:49.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.168+0000 7f61a4bc1640 1 Processor -- start 2024-04-24T23:03:49.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.168+0000 7f61a4bc1640 1 -- start start 2024-04-24T23:03:49.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.169+0000 7f61a4bc1640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f61a015b380 0x7f61a006c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:49.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.169+0000 7f61a4bc1640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f61a015cb00 0x7f61a006cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:49.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.169+0000 7f61a4bc1640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61a00b4520 con 0x7f61a00ce430 2024-04-24T23:03:49.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.169+0000 7f61a4bc1640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f61a00cef90 con 0x7f61a015cb00 2024-04-24T23:03:49.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.169+0000 7f61a4bc1640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f61a00b4800 con 0x7f61a015b380 2024-04-24T23:03:49.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.170+0000 7f619dd9b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f61a015cb00 0x7f61a006cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:49.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.170+0000 7f619dd9b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f61a015cb00 0x7f61a006cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53400/0 (socket says 172.21.15.32:53400) 2024-04-24T23:03:49.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.170+0000 7f619dd9b640 1 -- 172.21.15.32:0/3534962586 learned_addr learned my addr 172.21.15.32:0/3534962586 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:49.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.170+0000 7f619e59c640 1 --2- 172.21.15.32:0/3534962586 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f61a015b380 0x7f61a006c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:49.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.170+0000 7f619dd9b640 1 -- 172.21.15.32:0/3534962586 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f61a015b380 msgr2=0x7f61a006c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:49.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.170+0000 7f619dd9b640 1 --2- 172.21.15.32:0/3534962586 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f61a015b380 0x7f61a006c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:49.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.170+0000 7f619dd9b640 1 -- 172.21.15.32:0/3534962586 >> v1:172.21.15.121:6789/0 conn(0x7f61a00ce430 legacy=0x7f61a00ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:49.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.170+0000 7f619dd9b640 1 -- 172.21.15.32:0/3534962586 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61a00b4ae0 con 0x7f61a015cb00 2024-04-24T23:03:49.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.171+0000 7f619dd9b640 1 --2- 172.21.15.32:0/3534962586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61a015cb00 0x7f61a006cb30 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f6188009870 tx=0x7f618802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50add8f848493ab0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:49.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.171+0000 7f619d59a640 1 -- 172.21.15.32:0/3534962586 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6188037070 con 0x7f61a015cb00 2024-04-24T23:03:49.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.171+0000 7f619d59a640 1 -- 172.21.15.32:0/3534962586 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6188032440 con 0x7f61a015cb00 2024-04-24T23:03:49.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.171+0000 7f619d59a640 1 -- 172.21.15.32:0/3534962586 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6188032810 con 0x7f61a015cb00 2024-04-24T23:03:49.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.171+0000 7f61a4bc1640 1 -- 172.21.15.32:0/3534962586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61a015cb00 msgr2=0x7f61a006cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:49.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.171+0000 7f61a4bc1640 1 --2- 172.21.15.32:0/3534962586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61a015cb00 0x7f61a006cb30 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f6188009870 tx=0x7f618802a1c0 comp rx=0 tx=0).stop 2024-04-24T23:03:49.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.171+0000 7f61a4bc1640 1 -- 172.21.15.32:0/3534962586 shutdown_connections 2024-04-24T23:03:49.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.171+0000 7f61a4bc1640 1 --2- 172.21.15.32:0/3534962586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61a015cb00 0x7f61a006cb30 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:49.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.171+0000 7f61a4bc1640 1 --2- 172.21.15.32:0/3534962586 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f61a015b380 0x7f61a006c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:49.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.171+0000 7f61a4bc1640 1 -- 172.21.15.32:0/3534962586 >> 172.21.15.32:0/3534962586 conn(0x7f61a006b110 msgr2=0x7f61a006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:49.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.171+0000 7f61a4bc1640 1 -- 172.21.15.32:0/3534962586 shutdown_connections 2024-04-24T23:03:49.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.172+0000 7f61a4bc1640 1 -- 172.21.15.32:0/3534962586 wait complete. 2024-04-24T23:03:49.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.173+0000 7f61a4bc1640 1 Processor -- start 2024-04-24T23:03:49.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.173+0000 7f61a4bc1640 1 -- start start 2024-04-24T23:03:49.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.173+0000 7f61a4bc1640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f61a00ce430 0x7f61a0143d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:49.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.174+0000 7f61a4bc1640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f61a015b380 0x7f61a01e6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:49.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.174+0000 7f61a4bc1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61a015cb00 0x7f61a01599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:49.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.174+0000 7f61a4bc1640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f61a015b760 con 0x7f61a015cb00 2024-04-24T23:03:49.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.174+0000 7f61a4bc1640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f61a00b50a0 con 0x7f61a015b380 2024-04-24T23:03:49.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.174+0000 7f61a4bc1640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f61a00b55c0 con 0x7f61a00ce430 2024-04-24T23:03:49.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.174+0000 7f619ed9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61a015cb00 0x7f61a01599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:49.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.174+0000 7f619ed9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61a015cb00 0x7f61a01599c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53406/0 (socket says 172.21.15.32:53406) 2024-04-24T23:03:49.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.174+0000 7f619ed9d640 1 -- 172.21.15.32:0/350991911 learned_addr learned my addr 172.21.15.32:0/350991911 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:49.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.174+0000 7f619dd9b640 1 --2- 172.21.15.32:0/350991911 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f61a015b380 0x7f61a01e6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:49.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.174+0000 7f619e59c640 1 --2- 172.21.15.32:0/350991911 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f61a00ce430 0x7f61a0143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:49.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.174+0000 7f619ed9d640 1 -- 172.21.15.32:0/350991911 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f61a00ce430 msgr2=0x7f61a0143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:49.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.174+0000 7f619ed9d640 1 --2- 172.21.15.32:0/350991911 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f61a00ce430 0x7f61a0143d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:49.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.174+0000 7f619ed9d640 1 -- 172.21.15.32:0/350991911 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f61a015b380 msgr2=0x7f61a01e6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:49.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.174+0000 7f619ed9d640 1 --2- 172.21.15.32:0/350991911 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f61a015b380 0x7f61a01e6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:49.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.174+0000 7f619ed9d640 1 -- 172.21.15.32:0/350991911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61a014ffb0 con 0x7f61a015cb00 2024-04-24T23:03:49.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.174+0000 7f619ed9d640 1 --2- 172.21.15.32:0/350991911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61a015cb00 0x7f61a01599c0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f619800c7a0 tx=0x7f619800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:49.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.174+0000 7f6196ffd640 1 -- 172.21.15.32:0/350991911 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6198003b70 con 0x7f61a015cb00 2024-04-24T23:03:49.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.174+0000 7f6196ffd640 1 -- 172.21.15.32:0/350991911 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6198003d30 con 0x7f61a015cb00 2024-04-24T23:03:49.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.174+0000 7f6196ffd640 1 -- 172.21.15.32:0/350991911 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f619800aa90 con 0x7f61a015cb00 2024-04-24T23:03:49.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.174+0000 7f61a4bc1640 1 -- 172.21.15.32:0/350991911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61a01e67c0 con 0x7f61a015cb00 2024-04-24T23:03:49.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.174+0000 7f61a4bc1640 1 -- 172.21.15.32:0/350991911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61a014ef30 con 0x7f61a015cb00 2024-04-24T23:03:49.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.175+0000 7f61a4bc1640 1 -- 172.21.15.32:0/350991911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61a0068430 con 0x7f61a015cb00 2024-04-24T23:03:49.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.176+0000 7f6196ffd640 1 -- 172.21.15.32:0/350991911 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f619800ac30 con 0x7f61a015cb00 2024-04-24T23:03:49.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.176+0000 7f6196ffd640 1 --2- 172.21.15.32:0/350991911 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f618006be30 0x7f618006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:49.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.176+0000 7f6196ffd640 1 -- 172.21.15.32:0/350991911 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5795+0+0 (secure 0 0 0) 0x7f619808ac40 con 0x7f61a015cb00 2024-04-24T23:03:49.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.177+0000 7f619e59c640 1 --2- 172.21.15.32:0/350991911 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f618006be30 0x7f618006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:49.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.177+0000 7f619e59c640 1 --2- 172.21.15.32:0/350991911 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f618006be30 0x7f618006e2f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6190009b40 tx=0x7f6190009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:49.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.181+0000 7f6196ffd640 1 -- 172.21.15.32:0/350991911 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f61a014ffb0 con 0x7f61a015cb00 2024-04-24T23:03:49.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:49.407+0000 7f61a4bc1640 1 -- 172.21.15.32:0/350991911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f61a015e110 con 0x7f61a015cb00 2024-04-24T23:03:50.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.048+0000 7f6196ffd640 1 -- 172.21.15.32:0/350991911 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v38) ==== 104+0+0 (secure 0 0 0) 0x7f6198058fd0 con 0x7f61a015cb00 2024-04-24T23:03:50.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.051+0000 7f61a4bc1640 1 -- 172.21.15.32:0/350991911 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f618006be30 msgr2=0x7f618006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:50.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.051+0000 7f61a4bc1640 1 --2- 172.21.15.32:0/350991911 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f618006be30 0x7f618006e2f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6190009b40 tx=0x7f6190009190 comp rx=0 tx=0).stop 2024-04-24T23:03:50.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.051+0000 7f61a4bc1640 1 -- 172.21.15.32:0/350991911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61a015cb00 msgr2=0x7f61a01599c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:50.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.051+0000 7f61a4bc1640 1 --2- 172.21.15.32:0/350991911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61a015cb00 0x7f61a01599c0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f619800c7a0 tx=0x7f619800cc60 comp rx=0 tx=0).stop 2024-04-24T23:03:50.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.051+0000 7f61a4bc1640 1 -- 172.21.15.32:0/350991911 shutdown_connections 2024-04-24T23:03:50.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.051+0000 7f61a4bc1640 1 --2- 172.21.15.32:0/350991911 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f618006be30 0x7f618006e2f0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:50.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.051+0000 7f61a4bc1640 1 --2- 172.21.15.32:0/350991911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61a015cb00 0x7f61a01599c0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:50.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.051+0000 7f61a4bc1640 1 --2- 172.21.15.32:0/350991911 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f61a015b380 0x7f61a01e6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:50.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.051+0000 7f61a4bc1640 1 --2- 172.21.15.32:0/350991911 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f61a00ce430 0x7f61a0143d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:50.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.051+0000 7f61a4bc1640 1 -- 172.21.15.32:0/350991911 >> 172.21.15.32:0/350991911 conn(0x7f61a006b110 msgr2=0x7f61a015e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:50.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.051+0000 7f61a4bc1640 1 -- 172.21.15.32:0/350991911 shutdown_connections 2024-04-24T23:03:50.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.051+0000 7f61a4bc1640 1 -- 172.21.15.32:0/350991911 wait complete. 2024-04-24T23:03:50.067 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-24T23:03:50.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.217+0000 7f239fa12640 1 Processor -- start 2024-04-24T23:03:50.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.217+0000 7f239fa12640 1 -- start start 2024-04-24T23:03:50.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.218+0000 7f239fa12640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2398143a70 0x7f239806f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:50.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.218+0000 7f239fa12640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f23981445d0 0x7f23980b4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:50.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.218+0000 7f239fa12640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f23980b5060 0x7f239815e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:50.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.218+0000 7f239fa12640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f23980b54c0 con 0x7f23980b5060 2024-04-24T23:03:50.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.218+0000 7f239fa12640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f2398143e50 con 0x7f23981445d0 2024-04-24T23:03:50.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.218+0000 7f239fa12640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f23980b5cf0 con 0x7f2398143a70 2024-04-24T23:03:50.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.218+0000 7f239dfaf640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f23980b5060 0x7f239815e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:50.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.218+0000 7f239dfaf640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f23980b5060 0x7f239815e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36842/0 (socket says 172.21.15.32:36842) 2024-04-24T23:03:50.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.218+0000 7f239dfaf640 1 -- 172.21.15.32:0/161252741 learned_addr learned my addr 172.21.15.32:0/161252741 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:50.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.219+0000 7f239dfaf640 1 -- 172.21.15.32:0/161252741 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2398143a70 msgr2=0x7f239806f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:50.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.219+0000 7f239dfaf640 1 --2- 172.21.15.32:0/161252741 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2398143a70 0x7f239806f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:50.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.219+0000 7f239cfad640 1 --2- 172.21.15.32:0/161252741 >> v2:172.21.15.121:3300/0 conn(0x7f23981445d0 0x7f23980b4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:50.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.219+0000 7f239dfaf640 1 -- 172.21.15.32:0/161252741 >> v2:172.21.15.121:3300/0 conn(0x7f23981445d0 msgr2=0x7f23980b4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:50.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.219+0000 7f239dfaf640 1 --2- 172.21.15.32:0/161252741 >> v2:172.21.15.121:3300/0 conn(0x7f23981445d0 0x7f23980b4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:50.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.219+0000 7f239dfaf640 1 -- 172.21.15.32:0/161252741 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2398144220 con 0x7f23980b5060 2024-04-24T23:03:50.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.219+0000 7f239cfad640 1 --2- 172.21.15.32:0/161252741 >> v2:172.21.15.121:3300/0 conn(0x7f23981445d0 0x7f23980b4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:03:50.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.219+0000 7f239dfaf640 1 --2- 172.21.15.32:0/161252741 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23980b5060 0x7f239815e8b0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f23980735d0 tx=0x7f238802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ca874e67c841d2d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:50.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.220+0000 7f2397fff640 1 -- 172.21.15.32:0/161252741 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2388036070 con 0x7f23980b5060 2024-04-24T23:03:50.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.220+0000 7f2397fff640 1 -- 172.21.15.32:0/161252741 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2388031600 con 0x7f23980b5060 2024-04-24T23:03:50.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.220+0000 7f2397fff640 1 -- 172.21.15.32:0/161252741 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23880319d0 con 0x7f23980b5060 2024-04-24T23:03:50.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.220+0000 7f239fa12640 1 -- 172.21.15.32:0/161252741 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23980b5060 msgr2=0x7f239815e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:50.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.220+0000 7f239fa12640 1 --2- 172.21.15.32:0/161252741 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23980b5060 0x7f239815e8b0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f23980735d0 tx=0x7f238802a6f0 comp rx=0 tx=0).stop 2024-04-24T23:03:50.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.220+0000 7f239fa12640 1 -- 172.21.15.32:0/161252741 shutdown_connections 2024-04-24T23:03:50.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.220+0000 7f239fa12640 1 --2- 172.21.15.32:0/161252741 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23980b5060 0x7f239815e8b0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:50.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.220+0000 7f239fa12640 1 --2- 172.21.15.32:0/161252741 >> v2:172.21.15.121:3300/0 conn(0x7f23981445d0 0x7f23980b4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:50.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.220+0000 7f239fa12640 1 --2- 172.21.15.32:0/161252741 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2398143a70 0x7f239806f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:50.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.220+0000 7f239fa12640 1 -- 172.21.15.32:0/161252741 >> 172.21.15.32:0/161252741 conn(0x7f23980d0a60 msgr2=0x7f23980d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:50.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.220+0000 7f239fa12640 1 -- 172.21.15.32:0/161252741 shutdown_connections 2024-04-24T23:03:50.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.220+0000 7f239fa12640 1 -- 172.21.15.32:0/161252741 wait complete. 2024-04-24T23:03:50.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.221+0000 7f239fa12640 1 Processor -- start 2024-04-24T23:03:50.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.221+0000 7f239fa12640 1 -- start start 2024-04-24T23:03:50.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.222+0000 7f239fa12640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23980b5060 0x7f2398151b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:50.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.222+0000 7f239fa12640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2398143a70 0x7f239814f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:50.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.222+0000 7f239fa12640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f23981445d0 0x7f23981532f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:50.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.222+0000 7f239fa12640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2398067730 con 0x7f23980b5060 2024-04-24T23:03:50.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.222+0000 7f239fa12640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f239806dba0 con 0x7f2398143a70 2024-04-24T23:03:50.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.222+0000 7f239fa12640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f23980678b0 con 0x7f23981445d0 2024-04-24T23:03:50.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.222+0000 7f239d7ae640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23980b5060 0x7f2398151b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:50.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.222+0000 7f239d7ae640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23980b5060 0x7f2398151b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36854/0 (socket says 172.21.15.32:36854) 2024-04-24T23:03:50.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.222+0000 7f239d7ae640 1 -- 172.21.15.32:0/1063580065 learned_addr learned my addr 172.21.15.32:0/1063580065 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:50.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.223+0000 7f239d7ae640 1 -- 172.21.15.32:0/1063580065 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f23981445d0 msgr2=0x7f23981532f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:50.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.223+0000 7f239d7ae640 1 --2- 172.21.15.32:0/1063580065 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f23981445d0 0x7f23981532f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:50.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.223+0000 7f239cfad640 1 --2- 172.21.15.32:0/1063580065 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2398143a70 0x7f239814f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:50.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.223+0000 7f239d7ae640 1 -- 172.21.15.32:0/1063580065 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2398143a70 msgr2=0x7f239814f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:50.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.223+0000 7f239d7ae640 1 --2- 172.21.15.32:0/1063580065 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2398143a70 0x7f239814f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:50.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.223+0000 7f239d7ae640 1 -- 172.21.15.32:0/1063580065 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2398150ac0 con 0x7f23980b5060 2024-04-24T23:03:50.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.223+0000 7f239cfad640 1 --2- 172.21.15.32:0/1063580065 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2398143a70 0x7f239814f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:03:50.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.223+0000 7f239d7ae640 1 --2- 172.21.15.32:0/1063580065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23980b5060 0x7f2398151b40 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f239000b4a0 tx=0x7f239000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:50.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.223+0000 7f2395ffb640 1 -- 172.21.15.32:0/1063580065 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2390014070 con 0x7f23980b5060 2024-04-24T23:03:50.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.223+0000 7f2395ffb640 1 -- 172.21.15.32:0/1063580065 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2390010610 con 0x7f23980b5060 2024-04-24T23:03:50.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.223+0000 7f239fa12640 1 -- 172.21.15.32:0/1063580065 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2398150da0 con 0x7f23980b5060 2024-04-24T23:03:50.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.223+0000 7f2395ffb640 1 -- 172.21.15.32:0/1063580065 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2390010a00 con 0x7f23980b5060 2024-04-24T23:03:50.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.223+0000 7f239fa12640 1 -- 172.21.15.32:0/1063580065 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2398151360 con 0x7f23980b5060 2024-04-24T23:03:50.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.224+0000 7f239fa12640 1 -- 172.21.15.32:0/1063580065 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2398068430 con 0x7f23980b5060 2024-04-24T23:03:50.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.224+0000 7f2395ffb640 1 -- 172.21.15.32:0/1063580065 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2390017070 con 0x7f23980b5060 2024-04-24T23:03:50.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.225+0000 7f2395ffb640 1 --2- 172.21.15.32:0/1063580065 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f238406be30 0x7f238406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:50.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.225+0000 7f2395ffb640 1 -- 172.21.15.32:0/1063580065 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5795+0+0 (secure 0 0 0) 0x7f239008a8e0 con 0x7f23980b5060 2024-04-24T23:03:50.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.225+0000 7f239cfad640 1 --2- 172.21.15.32:0/1063580065 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f238406be30 0x7f238406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:50.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.226+0000 7f239cfad640 1 --2- 172.21.15.32:0/1063580065 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f238406be30 0x7f238406e2f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f238c006fd0 tx=0x7f238c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:50.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.229+0000 7f2395ffb640 1 -- 172.21.15.32:0/1063580065 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2390059230 con 0x7f23980b5060 2024-04-24T23:03:50.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:50.412+0000 7f239fa12640 1 -- 172.21.15.32:0/1063580065 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f23980d18d0 con 0x7f23980b5060 2024-04-24T23:03:51.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.381+0000 7f2395ffb640 1 -- 172.21.15.32:0/1063580065 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v95) ==== 181+0+0 (secure 0 0 0) 0x7f2390058b30 con 0x7f23980b5060 2024-04-24T23:03:51.383 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2024-04-24T23:03:51.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.384+0000 7f239fa12640 1 -- 172.21.15.32:0/1063580065 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f238406be30 msgr2=0x7f238406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:51.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.384+0000 7f239fa12640 1 --2- 172.21.15.32:0/1063580065 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f238406be30 0x7f238406e2f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f238c006fd0 tx=0x7f238c008040 comp rx=0 tx=0).stop 2024-04-24T23:03:51.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.384+0000 7f239fa12640 1 -- 172.21.15.32:0/1063580065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23980b5060 msgr2=0x7f2398151b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:51.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.385+0000 7f239fa12640 1 --2- 172.21.15.32:0/1063580065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23980b5060 0x7f2398151b40 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f239000b4a0 tx=0x7f239000b960 comp rx=0 tx=0).stop 2024-04-24T23:03:51.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.385+0000 7f239fa12640 1 -- 172.21.15.32:0/1063580065 shutdown_connections 2024-04-24T23:03:51.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.385+0000 7f239fa12640 1 --2- 172.21.15.32:0/1063580065 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f238406be30 0x7f238406e2f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:51.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.385+0000 7f239fa12640 1 --2- 172.21.15.32:0/1063580065 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f23981445d0 0x7f23981532f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:51.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.385+0000 7f239fa12640 1 --2- 172.21.15.32:0/1063580065 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2398143a70 0x7f239814f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:51.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.385+0000 7f239fa12640 1 --2- 172.21.15.32:0/1063580065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23980b5060 0x7f2398151b40 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:51.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.385+0000 7f239fa12640 1 -- 172.21.15.32:0/1063580065 >> 172.21.15.32:0/1063580065 conn(0x7f23980d0a60 msgr2=0x7f2398142320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:51.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.385+0000 7f239fa12640 1 -- 172.21.15.32:0/1063580065 shutdown_connections 2024-04-24T23:03:51.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.385+0000 7f239fa12640 1 -- 172.21.15.32:0/1063580065 wait complete. 2024-04-24T23:03:51.400 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-24T23:03:51.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.512+0000 7f8ef4b29640 1 Processor -- start 2024-04-24T23:03:51.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.512+0000 7f8ef4b29640 1 -- start start 2024-04-24T23:03:51.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.513+0000 7f8ef4b29640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8ef015b960 0x7f8ef00cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:51.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.513+0000 7f8ef4b29640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ef015bde0 con 0x7f8ef00b58e0 2024-04-24T23:03:51.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.513+0000 7f8ef4b29640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ef015c0c0 con 0x7f8ef015b580 2024-04-24T23:03:51.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.513+0000 7f8ef4b29640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f8ef015c9f0 con 0x7f8ef015b960 2024-04-24T23:03:51.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.513+0000 7f8eeed9d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8ef00b58e0 0x7f8ef015f340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60464/0 (socket says 172.21.15.32:60464) 2024-04-24T23:03:51.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.513+0000 7f8eeed9d640 1 -- 172.21.15.32:0/2721998077 learned_addr learned my addr 172.21.15.32:0/2721998077 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:03:51.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.513+0000 7f8eedd9b640 1 -- 172.21.15.32:0/2721998077 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1037994344 0 0) 0x7f8ef015bde0 con 0x7f8ef00b58e0 2024-04-24T23:03:51.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.513+0000 7f8eedd9b640 1 -- 172.21.15.32:0/2721998077 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8edc003520 con 0x7f8ef00b58e0 2024-04-24T23:03:51.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.513+0000 7f8ee7fff640 1 --2- 172.21.15.32:0/2721998077 >> v2:172.21.15.121:3300/0 conn(0x7f8ef015b960 0x7f8ef00cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:51.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.513+0000 7f8eedd9b640 1 -- 172.21.15.32:0/2721998077 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 936624093 0 0) 0x7f8edc003520 con 0x7f8ef00b58e0 2024-04-24T23:03:51.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.513+0000 7f8eedd9b640 1 -- 172.21.15.32:0/2721998077 >> v2:172.21.15.121:3300/0 conn(0x7f8ef015b960 msgr2=0x7f8ef00cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:51.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.513+0000 7f8eedd9b640 1 --2- 172.21.15.32:0/2721998077 >> v2:172.21.15.121:3300/0 conn(0x7f8ef015b960 0x7f8ef00cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:51.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.513+0000 7f8eedd9b640 1 -- 172.21.15.32:0/2721998077 >> v1:172.21.15.121:6789/0 conn(0x7f8ef015b580 legacy=0x7f8ef006cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:51.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.513+0000 7f8eedd9b640 1 -- 172.21.15.32:0/2721998077 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ef015c3a0 con 0x7f8ef00b58e0 2024-04-24T23:03:51.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.513+0000 7f8eedd9b640 1 -- 172.21.15.32:0/2721998077 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f8ee80019f0 con 0x7f8ef00b58e0 2024-04-24T23:03:51.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.514+0000 7f8eedd9b640 1 -- 172.21.15.32:0/2721998077 >> v1:172.21.15.32:6789/0 conn(0x7f8ef00b58e0 legacy=0x7f8ef015f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:51.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.514+0000 7f8eedd9b640 1 --2- 172.21.15.32:0/2721998077 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8edc0039b0 0x7f8edc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:51.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.514+0000 7f8eedd9b640 1 --2- 172.21.15.32:0/2721998077 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8edc004800 0x7f8edc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:51.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.514+0000 7f8eedd9b640 1 --2- 172.21.15.32:0/2721998077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8edc0051c0 0x7f8edc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:51.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.514+0000 7f8eedd9b640 1 -- 172.21.15.32:0/2721998077 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef015c9f0 con 0x7f8edc0051c0 2024-04-24T23:03:51.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.514+0000 7f8eedd9b640 1 -- 172.21.15.32:0/2721998077 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8edc00c060 con 0x7f8edc0039b0 2024-04-24T23:03:51.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.514+0000 7f8eedd9b640 1 -- 172.21.15.32:0/2721998077 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f8edc00c260 con 0x7f8edc004800 2024-04-24T23:03:51.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.514+0000 7f8eeed9d640 1 --2- 172.21.15.32:0/2721998077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8edc0051c0 0x7f8edc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:51.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.514+0000 7f8eee59c640 1 --2- 172.21.15.32:0/2721998077 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8edc0039b0 0x7f8edc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:51.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.514+0000 7f8ee7fff640 1 --2- 172.21.15.32:0/2721998077 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8edc004800 0x7f8edc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:51.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.514+0000 7f8eeed9d640 1 -- 172.21.15.32:0/2721998077 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8edc004800 msgr2=0x7f8edc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:51.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.514+0000 7f8eeed9d640 1 --2- 172.21.15.32:0/2721998077 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8edc004800 0x7f8edc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:51.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.514+0000 7f8eeed9d640 1 -- 172.21.15.32:0/2721998077 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8edc0039b0 msgr2=0x7f8edc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:51.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.514+0000 7f8eeed9d640 1 --2- 172.21.15.32:0/2721998077 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8edc0039b0 0x7f8edc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:51.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.514+0000 7f8eeed9d640 1 -- 172.21.15.32:0/2721998077 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8edc003520 con 0x7f8edc0051c0 2024-04-24T23:03:51.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.515+0000 7f8eeed9d640 1 -- 172.21.15.32:0/2721998077 reap_dead start 2024-04-24T23:03:51.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.515+0000 7f8eeed9d640 1 --2- 172.21.15.32:0/2721998077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8edc0051c0 0x7f8edc00b9c0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f8ee800a840 tx=0x7f8edc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f850539b62048e4b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:51.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.515+0000 7f8eedd9b640 1 -- 172.21.15.32:0/2721998077 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee800a460 con 0x7f8edc0051c0 2024-04-24T23:03:51.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.515+0000 7f8eedd9b640 1 -- 172.21.15.32:0/2721998077 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ee802f070 con 0x7f8edc0051c0 2024-04-24T23:03:51.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.515+0000 7f8eedd9b640 1 -- 172.21.15.32:0/2721998077 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee802bcb0 con 0x7f8edc0051c0 2024-04-24T23:03:51.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.515+0000 7f8ef4b29640 1 -- 172.21.15.32:0/2721998077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8edc0051c0 msgr2=0x7f8edc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:51.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.515+0000 7f8ef4b29640 1 --2- 172.21.15.32:0/2721998077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8edc0051c0 0x7f8edc00b9c0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f8ee800a840 tx=0x7f8edc005830 comp rx=0 tx=0).stop 2024-04-24T23:03:51.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.515+0000 7f8ef4b29640 1 -- 172.21.15.32:0/2721998077 shutdown_connections 2024-04-24T23:03:51.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.515+0000 7f8ef4b29640 1 --2- 172.21.15.32:0/2721998077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8edc0051c0 0x7f8edc00b9c0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:51.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.515+0000 7f8ef4b29640 1 -- 172.21.15.32:0/2721998077 >> 172.21.15.32:0/2721998077 conn(0x7f8ef00b1f10 msgr2=0x7f8ef00b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:51.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.516+0000 7f8ef4b29640 1 -- 172.21.15.32:0/2721998077 shutdown_connections 2024-04-24T23:03:51.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.516+0000 7f8ef4b29640 1 -- 172.21.15.32:0/2721998077 wait complete. 2024-04-24T23:03:51.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.517+0000 7f8ef4b29640 1 Processor -- start 2024-04-24T23:03:51.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.517+0000 7f8ef4b29640 1 -- start start 2024-04-24T23:03:51.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.518+0000 7f8ef4b29640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8edc0051c0 0x7f8ef01542f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:51.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.518+0000 7f8ef4b29640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ef014da60 0x7f8ef014d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:51.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.518+0000 7f8ef4b29640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8ef01f3240 0x7f8ef01522a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:51.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.518+0000 7f8ef4b29640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef006c920 con 0x7f8ef014da60 2024-04-24T23:03:51.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.518+0000 7f8ef4b29640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef006c7a0 con 0x7f8edc0051c0 2024-04-24T23:03:51.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.518+0000 7f8ef4b29640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f8ef00cde80 con 0x7f8ef01f3240 2024-04-24T23:03:51.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.518+0000 7f8ee7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ef014da60 0x7f8ef014d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:51.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.518+0000 7f8ee7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ef014da60 0x7f8ef014d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36876/0 (socket says 172.21.15.32:36876) 2024-04-24T23:03:51.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.518+0000 7f8ee7fff640 1 -- 172.21.15.32:0/1399656897 learned_addr learned my addr 172.21.15.32:0/1399656897 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:51.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.518+0000 7f8eee59c640 1 --2- 172.21.15.32:0/1399656897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8edc0051c0 0x7f8ef01542f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:51.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.518+0000 7f8eeed9d640 1 --2- 172.21.15.32:0/1399656897 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8ef01f3240 0x7f8ef01522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:51.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.518+0000 7f8ee7fff640 1 -- 172.21.15.32:0/1399656897 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8ef01f3240 msgr2=0x7f8ef01522a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:51.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.518+0000 7f8ee7fff640 1 --2- 172.21.15.32:0/1399656897 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8ef01f3240 0x7f8ef01522a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:51.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.518+0000 7f8ee7fff640 1 -- 172.21.15.32:0/1399656897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8edc0051c0 msgr2=0x7f8ef01542f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:51.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.518+0000 7f8ee7fff640 1 --2- 172.21.15.32:0/1399656897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8edc0051c0 0x7f8ef01542f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:51.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.518+0000 7f8ee7fff640 1 -- 172.21.15.32:0/1399656897 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ef01f3720 con 0x7f8ef014da60 2024-04-24T23:03:51.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.518+0000 7f8eeed9d640 1 --2- 172.21.15.32:0/1399656897 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8ef01f3240 0x7f8ef01522a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:03:51.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.518+0000 7f8ee7fff640 1 --2- 172.21.15.32:0/1399656897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ef014da60 0x7f8ef014d210 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8011930 tx=0x7f8ed8011df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:51.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.518+0000 7f8ee6ffd640 1 -- 172.21.15.32:0/1399656897 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed800fa30 con 0x7f8ef014da60 2024-04-24T23:03:51.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.518+0000 7f8ee6ffd640 1 -- 172.21.15.32:0/1399656897 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ed800fbd0 con 0x7f8ef014da60 2024-04-24T23:03:51.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.518+0000 7f8ef4b29640 1 -- 172.21.15.32:0/1399656897 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ef01f3a00 con 0x7f8ef014da60 2024-04-24T23:03:51.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.518+0000 7f8ee6ffd640 1 -- 172.21.15.32:0/1399656897 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed8014440 con 0x7f8ef014da60 2024-04-24T23:03:51.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.518+0000 7f8ef4b29640 1 -- 172.21.15.32:0/1399656897 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ef01f3fc0 con 0x7f8ef014da60 2024-04-24T23:03:51.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.520+0000 7f8ef4b29640 1 -- 172.21.15.32:0/1399656897 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ef0068430 con 0x7f8ef014da60 2024-04-24T23:03:51.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.520+0000 7f8ee6ffd640 1 -- 172.21.15.32:0/1399656897 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8ed8012040 con 0x7f8ef014da60 2024-04-24T23:03:51.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.520+0000 7f8ee6ffd640 1 --2- 172.21.15.32:0/1399656897 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8ed006be30 0x7f8ed006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:51.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.520+0000 7f8ee6ffd640 1 -- 172.21.15.32:0/1399656897 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5357+0+0 (secure 0 0 0) 0x7f8ed808b280 con 0x7f8ef014da60 2024-04-24T23:03:51.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.520+0000 7f8eee59c640 1 --2- 172.21.15.32:0/1399656897 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8ed006be30 0x7f8ed006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:51.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.521+0000 7f8eee59c640 1 --2- 172.21.15.32:0/1399656897 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8ed006be30 0x7f8ed006e2f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0001760 tx=0x7f8ee0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:51.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.524+0000 7f8ee6ffd640 1 -- 172.21.15.32:0/1399656897 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8ed8019030 con 0x7f8ef014da60 2024-04-24T23:03:51.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:51.707+0000 7f8ef4b29640 1 -- 172.21.15.32:0/1399656897 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8ef015ff00 con 0x7f8ef014da60 2024-04-24T23:03:52.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.382+0000 7f8ee6ffd640 1 -- 172.21.15.32:0/1399656897 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v96) ==== 169+0+0 (secure 0 0 0) 0x7f8ed8059e20 con 0x7f8ef014da60 2024-04-24T23:03:52.385 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2024-04-24T23:03:52.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.386+0000 7f8ef4b29640 1 -- 172.21.15.32:0/1399656897 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8ed006be30 msgr2=0x7f8ed006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:52.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.386+0000 7f8ef4b29640 1 --2- 172.21.15.32:0/1399656897 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8ed006be30 0x7f8ed006e2f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0001760 tx=0x7f8ee0009040 comp rx=0 tx=0).stop 2024-04-24T23:03:52.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.386+0000 7f8ef4b29640 1 -- 172.21.15.32:0/1399656897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ef014da60 msgr2=0x7f8ef014d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:52.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.386+0000 7f8ef4b29640 1 --2- 172.21.15.32:0/1399656897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ef014da60 0x7f8ef014d210 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8011930 tx=0x7f8ed8011df0 comp rx=0 tx=0).stop 2024-04-24T23:03:52.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.386+0000 7f8ef4b29640 1 -- 172.21.15.32:0/1399656897 shutdown_connections 2024-04-24T23:03:52.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.386+0000 7f8ef4b29640 1 --2- 172.21.15.32:0/1399656897 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8ed006be30 0x7f8ed006e2f0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:52.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.386+0000 7f8ef4b29640 1 --2- 172.21.15.32:0/1399656897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ef014da60 0x7f8ef014d210 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:52.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.386+0000 7f8ef4b29640 1 --2- 172.21.15.32:0/1399656897 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8ef01f3240 0x7f8ef01522a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:52.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.386+0000 7f8ef4b29640 1 --2- 172.21.15.32:0/1399656897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8edc0051c0 0x7f8ef01542f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:52.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.386+0000 7f8ef4b29640 1 -- 172.21.15.32:0/1399656897 >> 172.21.15.32:0/1399656897 conn(0x7f8ef00b1f10 msgr2=0x7f8ef006caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:52.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.386+0000 7f8ef4b29640 1 -- 172.21.15.32:0/1399656897 shutdown_connections 2024-04-24T23:03:52.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.386+0000 7f8ef4b29640 1 -- 172.21.15.32:0/1399656897 wait complete. 2024-04-24T23:03:52.403 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-24T23:03:52.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.509+0000 7f09bf238640 1 Processor -- start 2024-04-24T23:03:52.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.509+0000 7f09bf238640 1 -- start start 2024-04-24T23:03:52.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.509+0000 7f09bf238640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f09b815c5b0 0x7f09b80b4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:52.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.509+0000 7f09bf238640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09b806cd00 0x7f09b80b4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:52.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.509+0000 7f09bf238640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09b815c990 con 0x7f09b8072cf0 2024-04-24T23:03:52.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.509+0000 7f09bf238640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f09b8072860 con 0x7f09b815c5b0 2024-04-24T23:03:52.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.509+0000 7f09bf238640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f09b8071c50 con 0x7f09b806cd00 2024-04-24T23:03:52.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.509+0000 7f09bd7d5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09b806cd00 0x7f09b80b4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:52.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.509+0000 7f09bcfd4640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f09b815c5b0 0x7f09b80b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:52.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.509+0000 7f09bd7d5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09b806cd00 0x7f09b80b4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.32:54312/0 (socket says 172.21.15.32:54312) 2024-04-24T23:03:52.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.509+0000 7f09b7fff640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f09b8072cf0 0x7f09b806c5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:53958/0 (socket says 172.21.15.32:53958) 2024-04-24T23:03:52.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.509+0000 7f09bd7d5640 1 -- 172.21.15.32:0/3671562574 learned_addr learned my addr 172.21.15.32:0/3671562574 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:52.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.509+0000 7f09bd7d5640 1 -- 172.21.15.32:0/3671562574 >> v2:172.21.15.121:3300/0 conn(0x7f09b815c5b0 msgr2=0x7f09b80b4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:52.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.509+0000 7f09bd7d5640 1 --2- 172.21.15.32:0/3671562574 >> v2:172.21.15.121:3300/0 conn(0x7f09b815c5b0 0x7f09b80b4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:52.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.509+0000 7f09bd7d5640 1 -- 172.21.15.32:0/3671562574 >> v1:172.21.15.121:6789/0 conn(0x7f09b8072cf0 legacy=0x7f09b806c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:52.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.509+0000 7f09bd7d5640 1 -- 172.21.15.32:0/3671562574 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09b8071f30 con 0x7f09b806cd00 2024-04-24T23:03:52.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.511+0000 7f09bd7d5640 1 --2- 172.21.15.32:0/3671562574 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09b806cd00 0x7f09b80b4b40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f09b0009e50 tx=0x7f09b002a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c2a1ee6f1feeb61 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:52.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.511+0000 7f09b77fe640 1 -- 172.21.15.32:0/3671562574 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b0036070 con 0x7f09b806cd00 2024-04-24T23:03:52.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.511+0000 7f09b77fe640 1 -- 172.21.15.32:0/3671562574 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09b0031600 con 0x7f09b806cd00 2024-04-24T23:03:52.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.511+0000 7f09b77fe640 1 -- 172.21.15.32:0/3671562574 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b00319d0 con 0x7f09b806cd00 2024-04-24T23:03:52.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.512+0000 7f09bf238640 1 -- 172.21.15.32:0/3671562574 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09b806cd00 msgr2=0x7f09b80b4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:52.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.512+0000 7f09bf238640 1 --2- 172.21.15.32:0/3671562574 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09b806cd00 0x7f09b80b4b40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f09b0009e50 tx=0x7f09b002a680 comp rx=0 tx=0).stop 2024-04-24T23:03:52.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.512+0000 7f09bf238640 1 -- 172.21.15.32:0/3671562574 shutdown_connections 2024-04-24T23:03:52.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.512+0000 7f09bf238640 1 --2- 172.21.15.32:0/3671562574 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09b806cd00 0x7f09b80b4b40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:52.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.512+0000 7f09bf238640 1 --2- 172.21.15.32:0/3671562574 >> v2:172.21.15.121:3300/0 conn(0x7f09b815c5b0 0x7f09b80b4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:52.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.512+0000 7f09bf238640 1 -- 172.21.15.32:0/3671562574 >> 172.21.15.32:0/3671562574 conn(0x7f09b806b110 msgr2=0x7f09b806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:52.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.512+0000 7f09bf238640 1 -- 172.21.15.32:0/3671562574 shutdown_connections 2024-04-24T23:03:52.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.512+0000 7f09bf238640 1 -- 172.21.15.32:0/3671562574 wait complete. 2024-04-24T23:03:52.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.512+0000 7f09bf238640 1 Processor -- start 2024-04-24T23:03:52.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.512+0000 7f09bf238640 1 -- start start 2024-04-24T23:03:52.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.513+0000 7f09bf238640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09b806cd00 0x7f09b8155c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:52.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.513+0000 7f09bf238640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09b8072cf0 0x7f09b814ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:52.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.513+0000 7f09bf238640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09b815c5b0 0x7f09b8151d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:52.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.513+0000 7f09bf238640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f09b80b4570 con 0x7f09b815c5b0 2024-04-24T23:03:52.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.513+0000 7f09bf238640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f09b80724f0 con 0x7f09b806cd00 2024-04-24T23:03:52.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.513+0000 7f09bf238640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f09b80b46f0 con 0x7f09b8072cf0 2024-04-24T23:03:52.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.513+0000 7f09bd7d5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09b815c5b0 0x7f09b8151d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:52.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.513+0000 7f09bd7d5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09b815c5b0 0x7f09b8151d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36884/0 (socket says 172.21.15.32:36884) 2024-04-24T23:03:52.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.513+0000 7f09bd7d5640 1 -- 172.21.15.32:0/2475755722 learned_addr learned my addr 172.21.15.32:0/2475755722 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:52.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.513+0000 7f09b7fff640 1 --2- 172.21.15.32:0/2475755722 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09b8072cf0 0x7f09b814ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:52.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.513+0000 7f09bcfd4640 1 --2- 172.21.15.32:0/2475755722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09b806cd00 0x7f09b8155c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:52.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.513+0000 7f09bd7d5640 1 -- 172.21.15.32:0/2475755722 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09b8072cf0 msgr2=0x7f09b814ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:52.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.513+0000 7f09bd7d5640 1 --2- 172.21.15.32:0/2475755722 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09b8072cf0 0x7f09b814ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:52.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.515+0000 7f09bd7d5640 1 -- 172.21.15.32:0/2475755722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09b806cd00 msgr2=0x7f09b8155c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:52.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.515+0000 7f09bd7d5640 1 --2- 172.21.15.32:0/2475755722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09b806cd00 0x7f09b8155c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:52.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.515+0000 7f09bd7d5640 1 -- 172.21.15.32:0/2475755722 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09b815f7e0 con 0x7f09b815c5b0 2024-04-24T23:03:52.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.515+0000 7f09bd7d5640 1 --2- 172.21.15.32:0/2475755722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09b815c5b0 0x7f09b8151d80 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f09b0034040 tx=0x7f09b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:52.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.515+0000 7f09b57fa640 1 -- 172.21.15.32:0/2475755722 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b0036050 con 0x7f09b815c5b0 2024-04-24T23:03:52.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.515+0000 7f09b57fa640 1 -- 172.21.15.32:0/2475755722 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09b0042070 con 0x7f09b815c5b0 2024-04-24T23:03:52.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.515+0000 7f09bf238640 1 -- 172.21.15.32:0/2475755722 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09b8150a20 con 0x7f09b815c5b0 2024-04-24T23:03:52.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.515+0000 7f09b57fa640 1 -- 172.21.15.32:0/2475755722 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b0009510 con 0x7f09b815c5b0 2024-04-24T23:03:52.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.515+0000 7f09bf238640 1 -- 172.21.15.32:0/2475755722 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09b8150fe0 con 0x7f09b815c5b0 2024-04-24T23:03:52.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.516+0000 7f09bf238640 1 -- 172.21.15.32:0/2475755722 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09b81511e0 con 0x7f09b815c5b0 2024-04-24T23:03:52.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.516+0000 7f09b57fa640 1 -- 172.21.15.32:0/2475755722 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f09b0007bf0 con 0x7f09b815c5b0 2024-04-24T23:03:52.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.517+0000 7f09b57fa640 1 --2- 172.21.15.32:0/2475755722 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f099c06be60 0x7f099c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:52.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.517+0000 7f09b57fa640 1 -- 172.21.15.32:0/2475755722 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 4943+0+0 (secure 0 0 0) 0x7f09b00ab960 con 0x7f09b815c5b0 2024-04-24T23:03:52.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.517+0000 7f09bcfd4640 1 --2- 172.21.15.32:0/2475755722 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f099c06be60 0x7f099c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:52.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.517+0000 7f09bcfd4640 1 --2- 172.21.15.32:0/2475755722 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f099c06be60 0x7f099c06e320 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f09b8152e50 tx=0x7f09ac008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:52.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.521+0000 7f09b57fa640 1 -- 172.21.15.32:0/2475755722 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f09b007a590 con 0x7f09b815c5b0 2024-04-24T23:03:52.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.703+0000 7f09bf238640 1 -- 172.21.15.32:0/2475755722 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f09b8068430 con 0x7f09b815c5b0 2024-04-24T23:03:52.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.706+0000 7f09b57fa640 1 -- 172.21.15.32:0/2475755722 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v9) ==== 104+0+0 (secure 0 0 0) 0x7f09b0079e90 con 0x7f09b815c5b0 2024-04-24T23:03:52.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.708+0000 7f09bf238640 1 -- 172.21.15.32:0/2475755722 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f099c06be60 msgr2=0x7f099c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:52.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.708+0000 7f09bf238640 1 --2- 172.21.15.32:0/2475755722 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f099c06be60 0x7f099c06e320 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f09b8152e50 tx=0x7f09ac008040 comp rx=0 tx=0).stop 2024-04-24T23:03:52.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.708+0000 7f09bf238640 1 -- 172.21.15.32:0/2475755722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09b815c5b0 msgr2=0x7f09b8151d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:52.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.708+0000 7f09bf238640 1 --2- 172.21.15.32:0/2475755722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09b815c5b0 0x7f09b8151d80 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f09b0034040 tx=0x7f09b0004a40 comp rx=0 tx=0).stop 2024-04-24T23:03:52.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.708+0000 7f09bf238640 1 -- 172.21.15.32:0/2475755722 shutdown_connections 2024-04-24T23:03:52.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.708+0000 7f09bf238640 1 --2- 172.21.15.32:0/2475755722 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f099c06be60 0x7f099c06e320 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:52.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.708+0000 7f09bf238640 1 --2- 172.21.15.32:0/2475755722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09b815c5b0 0x7f09b8151d80 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:52.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.708+0000 7f09bf238640 1 --2- 172.21.15.32:0/2475755722 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09b8072cf0 0x7f09b814ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:52.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.708+0000 7f09bf238640 1 --2- 172.21.15.32:0/2475755722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09b806cd00 0x7f09b8155c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:52.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.708+0000 7f09bf238640 1 -- 172.21.15.32:0/2475755722 >> 172.21.15.32:0/2475755722 conn(0x7f09b806b110 msgr2=0x7f09b80b0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:52.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.708+0000 7f09bf238640 1 -- 172.21.15.32:0/2475755722 shutdown_connections 2024-04-24T23:03:52.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.708+0000 7f09bf238640 1 -- 172.21.15.32:0/2475755722 wait complete. 2024-04-24T23:03:52.720 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fsync' 2024-04-24T23:03:52.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.820+0000 7ff5e2c1c640 1 Processor -- start 2024-04-24T23:03:52.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.820+0000 7ff5e2c1c640 1 -- start start 2024-04-24T23:03:52.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.821+0000 7ff5e2c1c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff5dc0cc790 0x7ff5dc0b52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:52.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.821+0000 7ff5e2c1c640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff5dc15f7d0 0x7ff5dc0b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:52.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.821+0000 7ff5e2c1c640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5dc0b4b40 con 0x7ff5dc0b3e20 2024-04-24T23:03:52.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.821+0000 7ff5e2c1c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7ff5dc0b58b0 con 0x7ff5dc0cc790 2024-04-24T23:03:52.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.821+0000 7ff5e2c1c640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7ff5dc0b5b90 con 0x7ff5dc15f7d0 2024-04-24T23:03:52.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.821+0000 7ff5e09b8640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff5dc0cc790 0x7ff5dc0b52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:52.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.821+0000 7ff5e09b8640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff5dc0cc790 0x7ff5dc0b52d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36900/0 (socket says 172.21.15.32:36900) 2024-04-24T23:03:52.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.821+0000 7ff5e09b8640 1 -- 172.21.15.32:0/3180936338 learned_addr learned my addr 172.21.15.32:0/3180936338 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:52.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.821+0000 7ff5e09b8640 1 -- 172.21.15.32:0/3180936338 >> v2:172.21.15.121:3300/0 conn(0x7ff5dc15f7d0 msgr2=0x7ff5dc0b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:52.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.821+0000 7ff5dbfff640 1 --2- 172.21.15.32:0/3180936338 >> v2:172.21.15.121:3300/0 conn(0x7ff5dc15f7d0 0x7ff5dc0b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:52.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.821+0000 7ff5e09b8640 1 --2- 172.21.15.32:0/3180936338 >> v2:172.21.15.121:3300/0 conn(0x7ff5dc15f7d0 0x7ff5dc0b38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:52.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.821+0000 7ff5e09b8640 1 -- 172.21.15.32:0/3180936338 >> v1:172.21.15.121:6789/0 conn(0x7ff5dc0b3e20 legacy=0x7ff5dc06c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:52.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.821+0000 7ff5e09b8640 1 -- 172.21.15.32:0/3180936338 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5dc15fc60 con 0x7ff5dc0cc790 2024-04-24T23:03:52.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.821+0000 7ff5dbfff640 1 --2- 172.21.15.32:0/3180936338 >> v2:172.21.15.121:3300/0 conn(0x7ff5dc15f7d0 0x7ff5dc0b38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:03:52.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.822+0000 7ff5e09b8640 1 --2- 172.21.15.32:0/3180936338 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5dc0cc790 0x7ff5dc0b52d0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc0cf440 tx=0x7ff5d002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce914aa80f387ae server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:52.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.822+0000 7ff5db7fe640 1 -- 172.21.15.32:0/3180936338 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d0037070 con 0x7ff5dc0cc790 2024-04-24T23:03:52.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.822+0000 7ff5db7fe640 1 -- 172.21.15.32:0/3180936338 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5d0032440 con 0x7ff5dc0cc790 2024-04-24T23:03:52.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.822+0000 7ff5db7fe640 1 -- 172.21.15.32:0/3180936338 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d0032810 con 0x7ff5dc0cc790 2024-04-24T23:03:52.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.823+0000 7ff5e2c1c640 1 -- 172.21.15.32:0/3180936338 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5dc0cc790 msgr2=0x7ff5dc0b52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:52.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.823+0000 7ff5e2c1c640 1 --2- 172.21.15.32:0/3180936338 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5dc0cc790 0x7ff5dc0b52d0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc0cf440 tx=0x7ff5d002a230 comp rx=0 tx=0).stop 2024-04-24T23:03:52.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.823+0000 7ff5e2c1c640 1 -- 172.21.15.32:0/3180936338 shutdown_connections 2024-04-24T23:03:52.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.823+0000 7ff5e2c1c640 1 --2- 172.21.15.32:0/3180936338 >> v2:172.21.15.121:3300/0 conn(0x7ff5dc15f7d0 0x7ff5dc0b38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:52.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.823+0000 7ff5e2c1c640 1 --2- 172.21.15.32:0/3180936338 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5dc0cc790 0x7ff5dc0b52d0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:52.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.823+0000 7ff5e2c1c640 1 -- 172.21.15.32:0/3180936338 >> 172.21.15.32:0/3180936338 conn(0x7ff5dc06b110 msgr2=0x7ff5dc06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:52.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.823+0000 7ff5e2c1c640 1 -- 172.21.15.32:0/3180936338 shutdown_connections 2024-04-24T23:03:52.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.823+0000 7ff5e2c1c640 1 -- 172.21.15.32:0/3180936338 wait complete. 2024-04-24T23:03:52.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.824+0000 7ff5e2c1c640 1 Processor -- start 2024-04-24T23:03:52.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.824+0000 7ff5e2c1c640 1 -- start start 2024-04-24T23:03:52.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.825+0000 7ff5e2c1c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff5dc0cc790 0x7ff5dc150660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:52.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.825+0000 7ff5e2c1c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5dc15f7d0 0x7ff5dc0b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:52.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.825+0000 7ff5e2c1c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff5dc150c80 0x7ff5dc1584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:52.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.825+0000 7ff5e2c1c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff5dc0b6150 con 0x7ff5dc15f7d0 2024-04-24T23:03:52.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.825+0000 7ff5e2c1c640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff5dc0ae950 con 0x7ff5dc0cc790 2024-04-24T23:03:52.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.825+0000 7ff5e2c1c640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff5dc0b62d0 con 0x7ff5dc150c80 2024-04-24T23:03:52.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.825+0000 7ff5dbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5dc15f7d0 0x7ff5dc0b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:52.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.825+0000 7ff5dbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5dc15f7d0 0x7ff5dc0b01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36916/0 (socket says 172.21.15.32:36916) 2024-04-24T23:03:52.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.825+0000 7ff5dbfff640 1 -- 172.21.15.32:0/405302320 learned_addr learned my addr 172.21.15.32:0/405302320 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:52.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.825+0000 7ff5e11b9640 1 --2- 172.21.15.32:0/405302320 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff5dc150c80 0x7ff5dc1584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:52.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.825+0000 7ff5e09b8640 1 --2- 172.21.15.32:0/405302320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff5dc0cc790 0x7ff5dc150660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:52.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.825+0000 7ff5dbfff640 1 -- 172.21.15.32:0/405302320 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff5dc150c80 msgr2=0x7ff5dc1584a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:52.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.825+0000 7ff5dbfff640 1 --2- 172.21.15.32:0/405302320 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff5dc150c80 0x7ff5dc1584a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:52.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.825+0000 7ff5dbfff640 1 -- 172.21.15.32:0/405302320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff5dc0cc790 msgr2=0x7ff5dc150660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:52.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.825+0000 7ff5dbfff640 1 --2- 172.21.15.32:0/405302320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff5dc0cc790 0x7ff5dc150660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:52.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.825+0000 7ff5dbfff640 1 -- 172.21.15.32:0/405302320 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5dc0af100 con 0x7ff5dc15f7d0 2024-04-24T23:03:52.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.825+0000 7ff5dbfff640 1 --2- 172.21.15.32:0/405302320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5dc15f7d0 0x7ff5dc0b01a0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7ff5c800c7f0 tx=0x7ff5c800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:52.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.825+0000 7ff5d97fa640 1 -- 172.21.15.32:0/405302320 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c8015070 con 0x7ff5dc15f7d0 2024-04-24T23:03:52.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.825+0000 7ff5d97fa640 1 -- 172.21.15.32:0/405302320 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5c8007a30 con 0x7ff5dc15f7d0 2024-04-24T23:03:52.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.825+0000 7ff5d97fa640 1 -- 172.21.15.32:0/405302320 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c8007e00 con 0x7ff5dc15f7d0 2024-04-24T23:03:52.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.825+0000 7ff5e2c1c640 1 -- 172.21.15.32:0/405302320 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5dc0af3e0 con 0x7ff5dc15f7d0 2024-04-24T23:03:52.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.826+0000 7ff5e2c1c640 1 -- 172.21.15.32:0/405302320 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5dc0af9a0 con 0x7ff5dc15f7d0 2024-04-24T23:03:52.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.827+0000 7ff5e2c1c640 1 -- 172.21.15.32:0/405302320 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5dc068430 con 0x7ff5dc15f7d0 2024-04-24T23:03:52.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.827+0000 7ff5d97fa640 1 -- 172.21.15.32:0/405302320 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff5c8018070 con 0x7ff5dc15f7d0 2024-04-24T23:03:52.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.827+0000 7ff5d97fa640 1 --2- 172.21.15.32:0/405302320 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff5c006be30 0x7ff5c006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:52.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.828+0000 7ff5d97fa640 1 -- 172.21.15.32:0/405302320 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 4943+0+0 (secure 0 0 0) 0x7ff5c808aaa0 con 0x7ff5dc15f7d0 2024-04-24T23:03:52.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.828+0000 7ff5e09b8640 1 --2- 172.21.15.32:0/405302320 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff5c006be30 0x7ff5c006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:52.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.828+0000 7ff5e09b8640 1 --2- 172.21.15.32:0/405302320 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff5c006be30 0x7ff5c006e2f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff5d0034000 tx=0x7ff5d002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:52.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:52.832+0000 7ff5d97fa640 1 -- 172.21.15.32:0/405302320 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff5c8059100 con 0x7ff5dc15f7d0 2024-04-24T23:03:53.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.085+0000 7ff5e2c1c640 1 -- 172.21.15.32:0/405302320 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]} v 0) -- 0x7ff5dc06d2d0 con 0x7ff5dc15f7d0 2024-04-24T23:03:53.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.390+0000 7ff5d97fa640 1 -- 172.21.15.32:0/405302320 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]}]=0 v253) ==== 133+0+0 (secure 0 0 0) 0x7ff5c805d400 con 0x7ff5dc15f7d0 2024-04-24T23:03:53.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.393+0000 7ff5e2c1c640 1 -- 172.21.15.32:0/405302320 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff5c006be30 msgr2=0x7ff5c006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:53.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.393+0000 7ff5e2c1c640 1 --2- 172.21.15.32:0/405302320 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff5c006be30 0x7ff5c006e2f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff5d0034000 tx=0x7ff5d002e000 comp rx=0 tx=0).stop 2024-04-24T23:03:53.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.393+0000 7ff5e2c1c640 1 -- 172.21.15.32:0/405302320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5dc15f7d0 msgr2=0x7ff5dc0b01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:53.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.393+0000 7ff5e2c1c640 1 --2- 172.21.15.32:0/405302320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5dc15f7d0 0x7ff5dc0b01a0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7ff5c800c7f0 tx=0x7ff5c800cbf0 comp rx=0 tx=0).stop 2024-04-24T23:03:53.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.393+0000 7ff5e2c1c640 1 -- 172.21.15.32:0/405302320 shutdown_connections 2024-04-24T23:03:53.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.393+0000 7ff5e2c1c640 1 --2- 172.21.15.32:0/405302320 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff5c006be30 0x7ff5c006e2f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:53.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.393+0000 7ff5e2c1c640 1 --2- 172.21.15.32:0/405302320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5dc15f7d0 0x7ff5dc0b01a0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:53.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.393+0000 7ff5e2c1c640 1 --2- 172.21.15.32:0/405302320 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff5dc150c80 0x7ff5dc1584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:53.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.393+0000 7ff5e2c1c640 1 --2- 172.21.15.32:0/405302320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff5dc0cc790 0x7ff5dc150660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:53.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.393+0000 7ff5e2c1c640 1 -- 172.21.15.32:0/405302320 >> 172.21.15.32:0/405302320 conn(0x7ff5dc06b110 msgr2=0x7ff5dc1616a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:53.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.393+0000 7ff5e2c1c640 1 -- 172.21.15.32:0/405302320 shutdown_connections 2024-04-24T23:03:53.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.393+0000 7ff5e2c1c640 1 -- 172.21.15.32:0/405302320 wait complete. 2024-04-24T23:03:53.406 INFO:tasks.cephfs_test_runner:That when the full flag is encountered during asynchronous ... ok 2024-04-24T23:03:53.406 INFO:tasks.cephfs_test_runner:Starting test: test_full_same_file (tasks.cephfs.test_full.TestClusterFull) 2024-04-24T23:03:53.406 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:03:53.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.507+0000 7f166b62c640 1 Processor -- start 2024-04-24T23:03:53.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.507+0000 7f166b62c640 1 -- start start 2024-04-24T23:03:53.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.508+0000 7f166b62c640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f16640cd3b0 0x7f16640b1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:53.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.508+0000 7f166b62c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f166406bbc0 0x7f16640cb940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:53.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.508+0000 7f166b62c640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16640b1d70 con 0x7f16640b6180 2024-04-24T23:03:53.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.508+0000 7f166b62c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f16640b2050 con 0x7f166406bbc0 2024-04-24T23:03:53.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.508+0000 7f166b62c640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f16640b2330 con 0x7f16640cd3b0 2024-04-24T23:03:53.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.508+0000 7f1669bc9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f166406bbc0 0x7f16640cb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:53.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.508+0000 7f1669bc9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f166406bbc0 0x7f16640cb940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36922/0 (socket says 172.21.15.32:36922) 2024-04-24T23:03:53.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.508+0000 7f1669bc9640 1 -- 172.21.15.32:0/1557729459 learned_addr learned my addr 172.21.15.32:0/1557729459 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:53.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.508+0000 7f16693c8640 1 --2- 172.21.15.32:0/1557729459 >> v2:172.21.15.121:3300/0 conn(0x7f16640cd3b0 0x7f16640b1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:53.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.508+0000 7f1669bc9640 1 -- 172.21.15.32:0/1557729459 >> v2:172.21.15.121:3300/0 conn(0x7f16640cd3b0 msgr2=0x7f16640b1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:53.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.508+0000 7f1669bc9640 1 --2- 172.21.15.32:0/1557729459 >> v2:172.21.15.121:3300/0 conn(0x7f16640cd3b0 0x7f16640b1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:53.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.508+0000 7f1669bc9640 1 -- 172.21.15.32:0/1557729459 >> v1:172.21.15.121:6789/0 conn(0x7f16640b6180 legacy=0x7f166406b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:53.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.508+0000 7f1669bc9640 1 -- 172.21.15.32:0/1557729459 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16640b2610 con 0x7f166406bbc0 2024-04-24T23:03:53.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.508+0000 7f16693c8640 1 --2- 172.21.15.32:0/1557729459 >> v2:172.21.15.121:3300/0 conn(0x7f16640cd3b0 0x7f16640b1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:03:53.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.509+0000 7f1669bc9640 1 --2- 172.21.15.32:0/1557729459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f166406bbc0 0x7f16640cb940 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f1664070a40 tx=0x7f165c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80a3079e1c085d2a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:53.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.509+0000 7f165bfff640 1 -- 172.21.15.32:0/1557729459 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165c036070 con 0x7f166406bbc0 2024-04-24T23:03:53.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.509+0000 7f165bfff640 1 -- 172.21.15.32:0/1557729459 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f165c031600 con 0x7f166406bbc0 2024-04-24T23:03:53.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.509+0000 7f165bfff640 1 -- 172.21.15.32:0/1557729459 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165c0319d0 con 0x7f166406bbc0 2024-04-24T23:03:53.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.510+0000 7f166b62c640 1 -- 172.21.15.32:0/1557729459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f166406bbc0 msgr2=0x7f16640cb940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:53.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.510+0000 7f166b62c640 1 --2- 172.21.15.32:0/1557729459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f166406bbc0 0x7f16640cb940 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f1664070a40 tx=0x7f165c02a6a0 comp rx=0 tx=0).stop 2024-04-24T23:03:53.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.510+0000 7f166b62c640 1 -- 172.21.15.32:0/1557729459 shutdown_connections 2024-04-24T23:03:53.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.510+0000 7f166b62c640 1 --2- 172.21.15.32:0/1557729459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f166406bbc0 0x7f16640cb940 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:53.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.510+0000 7f166b62c640 1 --2- 172.21.15.32:0/1557729459 >> v2:172.21.15.121:3300/0 conn(0x7f16640cd3b0 0x7f16640b1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:53.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.510+0000 7f166b62c640 1 -- 172.21.15.32:0/1557729459 >> 172.21.15.32:0/1557729459 conn(0x7f16640b2e60 msgr2=0x7f16640b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:53.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.510+0000 7f166b62c640 1 -- 172.21.15.32:0/1557729459 shutdown_connections 2024-04-24T23:03:53.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.510+0000 7f166b62c640 1 -- 172.21.15.32:0/1557729459 wait complete. 2024-04-24T23:03:53.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.511+0000 7f166b62c640 1 Processor -- start 2024-04-24T23:03:53.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.511+0000 7f166b62c640 1 -- start start 2024-04-24T23:03:53.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.511+0000 7f166b62c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f166406bbc0 0x7f16640d1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:53.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.512+0000 7f166b62c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16640b6180 0x7f16641f4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:53.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.512+0000 7f166b62c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16640cd3b0 0x7f16640c6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:53.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.512+0000 7f166b62c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f16640b65f0 con 0x7f166406bbc0 2024-04-24T23:03:53.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.512+0000 7f166b62c640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f16640cd790 con 0x7f16640cd3b0 2024-04-24T23:03:53.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.512+0000 7f166b62c640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f16640b6770 con 0x7f16640b6180 2024-04-24T23:03:53.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.512+0000 7f16693c8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f166406bbc0 0x7f16640d1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:53.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.512+0000 7f16693c8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f166406bbc0 0x7f16640d1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36928/0 (socket says 172.21.15.32:36928) 2024-04-24T23:03:53.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.512+0000 7f16693c8640 1 -- 172.21.15.32:0/3062977349 learned_addr learned my addr 172.21.15.32:0/3062977349 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:53.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.512+0000 7f1669bc9640 1 --2- 172.21.15.32:0/3062977349 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16640cd3b0 0x7f16640c6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:53.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.512+0000 7f1668bc7640 1 --2- 172.21.15.32:0/3062977349 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16640b6180 0x7f16641f4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:53.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.512+0000 7f16693c8640 1 -- 172.21.15.32:0/3062977349 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16640b6180 msgr2=0x7f16641f4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:53.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.512+0000 7f16693c8640 1 --2- 172.21.15.32:0/3062977349 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16640b6180 0x7f16641f4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:53.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.512+0000 7f16693c8640 1 -- 172.21.15.32:0/3062977349 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16640cd3b0 msgr2=0x7f16640c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:53.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.512+0000 7f16693c8640 1 --2- 172.21.15.32:0/3062977349 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16640cd3b0 0x7f16640c6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:53.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.512+0000 7f16693c8640 1 -- 172.21.15.32:0/3062977349 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f166415e890 con 0x7f166406bbc0 2024-04-24T23:03:53.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.512+0000 7f1669bc9640 1 --2- 172.21.15.32:0/3062977349 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16640cd3b0 0x7f16640c6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:03:53.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.512+0000 7f16693c8640 1 --2- 172.21.15.32:0/3062977349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f166406bbc0 0x7f16640d1cb0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f166000e730 tx=0x7f166000ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:53.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.512+0000 7f1659ffb640 1 -- 172.21.15.32:0/3062977349 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166000f040 con 0x7f166406bbc0 2024-04-24T23:03:53.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.512+0000 7f1659ffb640 1 -- 172.21.15.32:0/3062977349 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1660004230 con 0x7f166406bbc0 2024-04-24T23:03:53.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.512+0000 7f166b62c640 1 -- 172.21.15.32:0/3062977349 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16640ae2c0 con 0x7f166406bbc0 2024-04-24T23:03:53.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.512+0000 7f1659ffb640 1 -- 172.21.15.32:0/3062977349 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166000ee40 con 0x7f166406bbc0 2024-04-24T23:03:53.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.513+0000 7f166b62c640 1 -- 172.21.15.32:0/3062977349 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16640ae880 con 0x7f166406bbc0 2024-04-24T23:03:53.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.513+0000 7f166b62c640 1 -- 172.21.15.32:0/3062977349 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1664068430 con 0x7f166406bbc0 2024-04-24T23:03:53.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.514+0000 7f1659ffb640 1 -- 172.21.15.32:0/3062977349 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f166000c6d0 con 0x7f166406bbc0 2024-04-24T23:03:53.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.514+0000 7f1659ffb640 1 --2- 172.21.15.32:0/3062977349 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f164806be30 0x7f164806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:53.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.514+0000 7f1659ffb640 1 -- 172.21.15.32:0/3062977349 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 4943+0+0 (secure 0 0 0) 0x7f166008a820 con 0x7f166406bbc0 2024-04-24T23:03:53.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.515+0000 7f1668bc7640 1 --2- 172.21.15.32:0/3062977349 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f164806be30 0x7f164806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:53.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.515+0000 7f1668bc7640 1 --2- 172.21.15.32:0/3062977349 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f164806be30 0x7f164806e2f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f16640d21f0 tx=0x7f1654009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:53.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.518+0000 7f1659ffb640 1 -- 172.21.15.32:0/3062977349 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1660017020 con 0x7f166406bbc0 2024-04-24T23:03:53.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.706+0000 7f166b62c640 1 -- 172.21.15.32:0/3062977349 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f16640d2220 con 0x7f166406bbc0 2024-04-24T23:03:53.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.708+0000 7f1659ffb640 1 -- 172.21.15.32:0/3062977349 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+13031 (secure 0 0 0) 0x7f1660059560 con 0x7f166406bbc0 2024-04-24T23:03:53.709 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:03:53.709 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":96,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:03:52.382967+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6835/464246210":"2024-04-25T23:03:49.047826+0000","172.21.15.32:6833/464246210":"2024-04-25T23:03:49.047826+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:03:53.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.710+0000 7f166b62c640 1 -- 172.21.15.32:0/3062977349 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f164806be30 msgr2=0x7f164806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:53.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.710+0000 7f166b62c640 1 --2- 172.21.15.32:0/3062977349 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f164806be30 0x7f164806e2f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f16640d21f0 tx=0x7f1654009040 comp rx=0 tx=0).stop 2024-04-24T23:03:53.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.710+0000 7f166b62c640 1 -- 172.21.15.32:0/3062977349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f166406bbc0 msgr2=0x7f16640d1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:53.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.710+0000 7f166b62c640 1 --2- 172.21.15.32:0/3062977349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f166406bbc0 0x7f16640d1cb0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f166000e730 tx=0x7f166000ebf0 comp rx=0 tx=0).stop 2024-04-24T23:03:53.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.710+0000 7f166b62c640 1 -- 172.21.15.32:0/3062977349 shutdown_connections 2024-04-24T23:03:53.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.710+0000 7f166b62c640 1 --2- 172.21.15.32:0/3062977349 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f164806be30 0x7f164806e2f0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:53.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.710+0000 7f166b62c640 1 --2- 172.21.15.32:0/3062977349 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16640cd3b0 0x7f16640c6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:53.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.710+0000 7f166b62c640 1 --2- 172.21.15.32:0/3062977349 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16640b6180 0x7f16641f4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:53.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.710+0000 7f166b62c640 1 --2- 172.21.15.32:0/3062977349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f166406bbc0 0x7f16640d1cb0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:53.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.710+0000 7f166b62c640 1 -- 172.21.15.32:0/3062977349 >> 172.21.15.32:0/3062977349 conn(0x7f16640b2e60 msgr2=0x7f1664074210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:53.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.710+0000 7f166b62c640 1 -- 172.21.15.32:0/3062977349 shutdown_connections 2024-04-24T23:03:53.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.710+0000 7f166b62c640 1 -- 172.21.15.32:0/3062977349 wait complete. 2024-04-24T23:03:53.722 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '93', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:03:53.723 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:03:53.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.825+0000 7f097224d640 1 Processor -- start 2024-04-24T23:03:53.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.825+0000 7f097224d640 1 -- start start 2024-04-24T23:03:53.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.825+0000 7f097224d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f096c15cb00 0x7f096c0ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:53.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.825+0000 7f097224d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f096c0b4520 con 0x7f096c06cd00 2024-04-24T23:03:53.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.825+0000 7f097224d640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f096c0b4800 con 0x7f096c15b380 2024-04-24T23:03:53.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.825+0000 7f097224d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f096c0b4ae0 con 0x7f096c15cb00 2024-04-24T23:03:53.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.826+0000 7f096bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f096c15cb00 0x7f096c0ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:53.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.826+0000 7f096affd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f096c06cd00 0x7f096c0ce430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60480/0 (socket says 172.21.15.32:60480) 2024-04-24T23:03:53.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.826+0000 7f096affd640 1 -- 172.21.15.32:0/1030480996 learned_addr learned my addr 172.21.15.32:0/1030480996 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:03:53.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.826+0000 7f096bfff640 1 -- 172.21.15.32:0/1030480996 >> v1:172.21.15.121:6789/0 conn(0x7f096c15b380 legacy=0x7f096c06c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:53.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.826+0000 7f096bfff640 1 -- 172.21.15.32:0/1030480996 >> v1:172.21.15.32:6789/0 conn(0x7f096c06cd00 legacy=0x7f096c0ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:53.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.826+0000 7f096bfff640 1 -- 172.21.15.32:0/1030480996 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f096c0b4dc0 con 0x7f096c15cb00 2024-04-24T23:03:53.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.826+0000 7f096a7fc640 1 -- 172.21.15.32:0/1030480996 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3442224204 0 0) 0x7f096c0b4520 con 0x7f096c06cd00 2024-04-24T23:03:53.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.827+0000 7f096bfff640 1 --2- 172.21.15.32:0/1030480996 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f096c15cb00 0x7f096c0ceb40 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f0964009d90 tx=0x7f096402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b042e5552eb38b8f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:53.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.827+0000 7f096a7fc640 1 -- 172.21.15.32:0/1030480996 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0964007740 con 0x7f096c15cb00 2024-04-24T23:03:53.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.827+0000 7f096a7fc640 1 -- 172.21.15.32:0/1030480996 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09640078e0 con 0x7f096c15cb00 2024-04-24T23:03:53.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.827+0000 7f096a7fc640 1 -- 172.21.15.32:0/1030480996 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0964007cb0 con 0x7f096c15cb00 2024-04-24T23:03:53.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.827+0000 7f097224d640 1 -- 172.21.15.32:0/1030480996 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f096c15cb00 msgr2=0x7f096c0ceb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:53.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.827+0000 7f097224d640 1 --2- 172.21.15.32:0/1030480996 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f096c15cb00 0x7f096c0ceb40 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f0964009d90 tx=0x7f096402a660 comp rx=0 tx=0).stop 2024-04-24T23:03:53.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.827+0000 7f097224d640 1 -- 172.21.15.32:0/1030480996 shutdown_connections 2024-04-24T23:03:53.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.827+0000 7f097224d640 1 --2- 172.21.15.32:0/1030480996 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f096c15cb00 0x7f096c0ceb40 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:53.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.827+0000 7f097224d640 1 -- 172.21.15.32:0/1030480996 >> 172.21.15.32:0/1030480996 conn(0x7f096c06b110 msgr2=0x7f096c06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:53.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.827+0000 7f097224d640 1 -- 172.21.15.32:0/1030480996 shutdown_connections 2024-04-24T23:03:53.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.827+0000 7f097224d640 1 -- 172.21.15.32:0/1030480996 wait complete. 2024-04-24T23:03:53.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.828+0000 7f097224d640 1 Processor -- start 2024-04-24T23:03:53.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.829+0000 7f097224d640 1 -- start start 2024-04-24T23:03:53.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.829+0000 7f097224d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f096c06cd00 0x7f096c070680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:53.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.829+0000 7f097224d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f096c15b380 0x7f096c1410d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:53.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.829+0000 7f097224d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f096c15cb00 0x7f096c0aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:53.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.829+0000 7f097224d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f096c0b55c0 con 0x7f096c06cd00 2024-04-24T23:03:53.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.829+0000 7f097224d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f096c15b760 con 0x7f096c15b380 2024-04-24T23:03:53.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.829+0000 7f097224d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f096c0b5740 con 0x7f096c15cb00 2024-04-24T23:03:53.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.829+0000 7f096b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f096c06cd00 0x7f096c070680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:53.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.829+0000 7f096b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f096c06cd00 0x7f096c070680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36950/0 (socket says 172.21.15.32:36950) 2024-04-24T23:03:53.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.829+0000 7f096b7fe640 1 -- 172.21.15.32:0/3097499332 learned_addr learned my addr 172.21.15.32:0/3097499332 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:53.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.830+0000 7f096affd640 1 --2- 172.21.15.32:0/3097499332 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f096c15b380 0x7f096c1410d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:53.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.830+0000 7f096bfff640 1 --2- 172.21.15.32:0/3097499332 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f096c15cb00 0x7f096c0aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:53.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.830+0000 7f096b7fe640 1 -- 172.21.15.32:0/3097499332 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f096c15cb00 msgr2=0x7f096c0aeb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:53.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.830+0000 7f096b7fe640 1 --2- 172.21.15.32:0/3097499332 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f096c15cb00 0x7f096c0aeb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:53.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.830+0000 7f096b7fe640 1 -- 172.21.15.32:0/3097499332 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f096c15b380 msgr2=0x7f096c1410d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:53.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.830+0000 7f096b7fe640 1 --2- 172.21.15.32:0/3097499332 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f096c15b380 0x7f096c1410d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:53.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.830+0000 7f096b7fe640 1 -- 172.21.15.32:0/3097499332 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f096c144ef0 con 0x7f096c06cd00 2024-04-24T23:03:53.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.830+0000 7f096b7fe640 1 --2- 172.21.15.32:0/3097499332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f096c06cd00 0x7f096c070680 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f095c00c450 tx=0x7f095c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:53.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.830+0000 7f0953fff640 1 -- 172.21.15.32:0/3097499332 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095c0052d0 con 0x7f096c06cd00 2024-04-24T23:03:53.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.830+0000 7f0953fff640 1 -- 172.21.15.32:0/3097499332 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f095c005470 con 0x7f096c06cd00 2024-04-24T23:03:53.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.830+0000 7f097224d640 1 -- 172.21.15.32:0/3097499332 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f096c1419d0 con 0x7f096c06cd00 2024-04-24T23:03:53.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.830+0000 7f0953fff640 1 -- 172.21.15.32:0/3097499332 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095c012870 con 0x7f096c06cd00 2024-04-24T23:03:53.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.830+0000 7f097224d640 1 -- 172.21.15.32:0/3097499332 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f096c141f90 con 0x7f096c06cd00 2024-04-24T23:03:53.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.831+0000 7f097224d640 1 -- 172.21.15.32:0/3097499332 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f096c068430 con 0x7f096c06cd00 2024-04-24T23:03:53.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.832+0000 7f0953fff640 1 -- 172.21.15.32:0/3097499332 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f095c012a10 con 0x7f096c06cd00 2024-04-24T23:03:53.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.832+0000 7f0953fff640 1 --2- 172.21.15.32:0/3097499332 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f095806be30 0x7f095806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:53.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.832+0000 7f0953fff640 1 -- 172.21.15.32:0/3097499332 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 4943+0+0 (secure 0 0 0) 0x7f095c08a6f0 con 0x7f096c06cd00 2024-04-24T23:03:53.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.832+0000 7f096affd640 1 --2- 172.21.15.32:0/3097499332 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f095806be30 0x7f095806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:53.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.833+0000 7f096affd640 1 --2- 172.21.15.32:0/3097499332 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f095806be30 0x7f095806e2f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0960007f70 tx=0x7f0960007f00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:53.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:53.836+0000 7f0953fff640 1 -- 172.21.15.32:0/3097499332 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f095c0593a0 con 0x7f096c06cd00 2024-04-24T23:03:54.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.018+0000 7f097224d640 1 -- 172.21.15.32:0/3097499332 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f096c06c000 con 0x7f096c06cd00 2024-04-24T23:03:54.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.018+0000 7f0953fff640 1 -- 172.21.15.32:0/3097499332 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) ==== 93+0+10 (secure 0 0 0) 0x7f095c058ca0 con 0x7f096c06cd00 2024-04-24T23:03:54.020 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T23:03:54.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.021+0000 7f097224d640 1 -- 172.21.15.32:0/3097499332 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f095806be30 msgr2=0x7f095806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:54.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.021+0000 7f097224d640 1 --2- 172.21.15.32:0/3097499332 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f095806be30 0x7f095806e2f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0960007f70 tx=0x7f0960007f00 comp rx=0 tx=0).stop 2024-04-24T23:03:54.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.021+0000 7f097224d640 1 -- 172.21.15.32:0/3097499332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f096c06cd00 msgr2=0x7f096c070680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:54.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.021+0000 7f097224d640 1 --2- 172.21.15.32:0/3097499332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f096c06cd00 0x7f096c070680 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f095c00c450 tx=0x7f095c00c910 comp rx=0 tx=0).stop 2024-04-24T23:03:54.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.021+0000 7f097224d640 1 -- 172.21.15.32:0/3097499332 shutdown_connections 2024-04-24T23:03:54.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.021+0000 7f097224d640 1 --2- 172.21.15.32:0/3097499332 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f095806be30 0x7f095806e2f0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:54.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.021+0000 7f097224d640 1 --2- 172.21.15.32:0/3097499332 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f096c15cb00 0x7f096c0aeb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:54.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.021+0000 7f097224d640 1 --2- 172.21.15.32:0/3097499332 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f096c15b380 0x7f096c1410d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:54.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.021+0000 7f097224d640 1 --2- 172.21.15.32:0/3097499332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f096c06cd00 0x7f096c070680 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:54.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.021+0000 7f097224d640 1 -- 172.21.15.32:0/3097499332 >> 172.21.15.32:0/3097499332 conn(0x7f096c06b110 msgr2=0x7f096c0ce640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:54.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.021+0000 7f097224d640 1 -- 172.21.15.32:0/3097499332 shutdown_connections 2024-04-24T23:03:54.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.021+0000 7f097224d640 1 -- 172.21.15.32:0/3097499332 wait complete. 2024-04-24T23:03:54.034 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_same_file' 2024-04-24T23:03:54.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.133+0000 7fdd5907d640 1 Processor -- start 2024-04-24T23:03:54.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.133+0000 7fdd5907d640 1 -- start start 2024-04-24T23:03:54.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.134+0000 7fdd5907d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdd5406b820 0x7fdd540d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:54.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.134+0000 7fdd5907d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdd540d4210 0x7fdd540b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:54.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.134+0000 7fdd5907d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd540d3170 con 0x7fdd5415e9b0 2024-04-24T23:03:54.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.134+0000 7fdd5907d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fdd540d3450 con 0x7fdd5406b820 2024-04-24T23:03:54.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.134+0000 7fdd5907d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fdd540d3730 con 0x7fdd540d4210 2024-04-24T23:03:54.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.134+0000 7fdd5259c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdd5406b820 0x7fdd540d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:54.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.134+0000 7fdd52d9d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fdd5415e9b0 0x7fdd5406b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60484/0 (socket says 172.21.15.32:60484) 2024-04-24T23:03:54.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.134+0000 7fdd52d9d640 1 -- 172.21.15.32:0/1646370162 learned_addr learned my addr 172.21.15.32:0/1646370162 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:03:54.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.134+0000 7fdd5359e640 1 --2- 172.21.15.32:0/1646370162 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdd540d4210 0x7fdd540b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:54.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.135+0000 7fdd5259c640 1 -- 172.21.15.32:0/1646370162 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdd540d4210 msgr2=0x7fdd540b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:54.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.135+0000 7fdd5259c640 1 --2- 172.21.15.32:0/1646370162 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdd540d4210 0x7fdd540b78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:54.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.135+0000 7fdd5259c640 1 -- 172.21.15.32:0/1646370162 >> v1:172.21.15.32:6789/0 conn(0x7fdd5415e9b0 legacy=0x7fdd5406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:54.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.135+0000 7fdd5259c640 1 -- 172.21.15.32:0/1646370162 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd540d3a10 con 0x7fdd5406b820 2024-04-24T23:03:54.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.135+0000 7fdd51d9b640 1 -- 172.21.15.32:0/1646370162 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1204608633 0 0) 0x7fdd540d3170 con 0x7fdd5415e9b0 2024-04-24T23:03:54.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.135+0000 7fdd5259c640 1 --2- 172.21.15.32:0/1646370162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd5406b820 0x7fdd540d2b90 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fdd480097a0 tx=0x7fdd4802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d473d45eaf3ee83 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:54.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.135+0000 7fdd51d9b640 1 -- 172.21.15.32:0/1646370162 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd48037070 con 0x7fdd5406b820 2024-04-24T23:03:54.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.135+0000 7fdd51d9b640 1 -- 172.21.15.32:0/1646370162 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd48032440 con 0x7fdd5406b820 2024-04-24T23:03:54.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.135+0000 7fdd51d9b640 1 -- 172.21.15.32:0/1646370162 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd48032810 con 0x7fdd5406b820 2024-04-24T23:03:54.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.136+0000 7fdd5907d640 1 -- 172.21.15.32:0/1646370162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd5406b820 msgr2=0x7fdd540d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:54.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.136+0000 7fdd5907d640 1 --2- 172.21.15.32:0/1646370162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd5406b820 0x7fdd540d2b90 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fdd480097a0 tx=0x7fdd4802a180 comp rx=0 tx=0).stop 2024-04-24T23:03:54.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.136+0000 7fdd5907d640 1 -- 172.21.15.32:0/1646370162 shutdown_connections 2024-04-24T23:03:54.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.136+0000 7fdd5907d640 1 --2- 172.21.15.32:0/1646370162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd5406b820 0x7fdd540d2b90 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:54.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.136+0000 7fdd5907d640 1 --2- 172.21.15.32:0/1646370162 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdd540d4210 0x7fdd540b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:54.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.136+0000 7fdd5907d640 1 -- 172.21.15.32:0/1646370162 >> 172.21.15.32:0/1646370162 conn(0x7fdd540d11d0 msgr2=0x7fdd540d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:54.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.136+0000 7fdd5907d640 1 -- 172.21.15.32:0/1646370162 shutdown_connections 2024-04-24T23:03:54.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.136+0000 7fdd5907d640 1 -- 172.21.15.32:0/1646370162 wait complete. 2024-04-24T23:03:54.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.137+0000 7fdd5907d640 1 Processor -- start 2024-04-24T23:03:54.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.138+0000 7fdd5907d640 1 -- start start 2024-04-24T23:03:54.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.138+0000 7fdd5907d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdd5406b820 0x7fdd54158850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:54.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.138+0000 7fdd5907d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd540d4210 0x7fdd541f4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:54.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.138+0000 7fdd5907d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd5415e9b0 0x7fdd5415b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:54.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.138+0000 7fdd5907d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdd540ba0e0 con 0x7fdd5415e9b0 2024-04-24T23:03:54.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.138+0000 7fdd5907d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdd540d2400 con 0x7fdd540d4210 2024-04-24T23:03:54.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.138+0000 7fdd5907d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fdd540ba260 con 0x7fdd5406b820 2024-04-24T23:03:54.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.138+0000 7fdd5359e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd5415e9b0 0x7fdd5415b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:54.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.138+0000 7fdd5359e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd5415e9b0 0x7fdd5415b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36978/0 (socket says 172.21.15.32:36978) 2024-04-24T23:03:54.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.138+0000 7fdd5359e640 1 -- 172.21.15.32:0/2526873402 learned_addr learned my addr 172.21.15.32:0/2526873402 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:54.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.138+0000 7fdd52d9d640 1 --2- 172.21.15.32:0/2526873402 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdd5406b820 0x7fdd54158850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:54.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.139+0000 7fdd5259c640 1 --2- 172.21.15.32:0/2526873402 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd540d4210 0x7fdd541f4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:54.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.139+0000 7fdd5359e640 1 -- 172.21.15.32:0/2526873402 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdd5406b820 msgr2=0x7fdd54158850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:54.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.139+0000 7fdd5359e640 1 --2- 172.21.15.32:0/2526873402 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdd5406b820 0x7fdd54158850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:54.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.139+0000 7fdd5359e640 1 -- 172.21.15.32:0/2526873402 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd540d4210 msgr2=0x7fdd541f4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:54.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.139+0000 7fdd5359e640 1 --2- 172.21.15.32:0/2526873402 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd540d4210 0x7fdd541f4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:54.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.139+0000 7fdd5359e640 1 -- 172.21.15.32:0/2526873402 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd541f49d0 con 0x7fdd5415e9b0 2024-04-24T23:03:54.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.139+0000 7fdd5359e640 1 --2- 172.21.15.32:0/2526873402 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd5415e9b0 0x7fdd5415b2e0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fdd4400cd60 tx=0x7fdd4400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:54.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.139+0000 7fdd3b7fe640 1 -- 172.21.15.32:0/2526873402 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd44003b90 con 0x7fdd5415e9b0 2024-04-24T23:03:54.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.139+0000 7fdd3b7fe640 1 -- 172.21.15.32:0/2526873402 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd44003d50 con 0x7fdd5415e9b0 2024-04-24T23:03:54.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.139+0000 7fdd5907d640 1 -- 172.21.15.32:0/2526873402 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd541f4cb0 con 0x7fdd5415e9b0 2024-04-24T23:03:54.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.139+0000 7fdd3b7fe640 1 -- 172.21.15.32:0/2526873402 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd44004260 con 0x7fdd5415e9b0 2024-04-24T23:03:54.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.139+0000 7fdd5907d640 1 -- 172.21.15.32:0/2526873402 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd5415adc0 con 0x7fdd5415e9b0 2024-04-24T23:03:54.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.140+0000 7fdd5907d640 1 -- 172.21.15.32:0/2526873402 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd54068430 con 0x7fdd5415e9b0 2024-04-24T23:03:54.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.141+0000 7fdd3b7fe640 1 -- 172.21.15.32:0/2526873402 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdd44004400 con 0x7fdd5415e9b0 2024-04-24T23:03:54.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.141+0000 7fdd3b7fe640 1 --2- 172.21.15.32:0/2526873402 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdd4006be30 0x7fdd4006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:54.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.141+0000 7fdd3b7fe640 1 -- 172.21.15.32:0/2526873402 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 4943+0+0 (secure 0 0 0) 0x7fdd4408b4c0 con 0x7fdd5415e9b0 2024-04-24T23:03:54.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.141+0000 7fdd52d9d640 1 --2- 172.21.15.32:0/2526873402 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdd4006be30 0x7fdd4006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:54.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.142+0000 7fdd52d9d640 1 --2- 172.21.15.32:0/2526873402 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdd4006be30 0x7fdd4006e2f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c000f60 tx=0x7fdd4c007d50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:54.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.145+0000 7fdd3b7fe640 1 -- 172.21.15.32:0/2526873402 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdd44059b20 con 0x7fdd5415e9b0 2024-04-24T23:03:54.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:54.397+0000 7fdd5907d640 1 -- 172.21.15.32:0/2526873402 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]} v 0) -- 0x7fdd5415abb0 con 0x7fdd5415e9b0 2024-04-24T23:03:55.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.397+0000 7fdd3b7fe640 1 -- 172.21.15.32:0/2526873402 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]}]=0 v255) ==== 140+0+0 (secure 0 0 0) 0x7fdd4405de20 con 0x7fdd5415e9b0 2024-04-24T23:03:55.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.400+0000 7fdd5907d640 1 -- 172.21.15.32:0/2526873402 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdd4006be30 msgr2=0x7fdd4006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:55.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.400+0000 7fdd5907d640 1 --2- 172.21.15.32:0/2526873402 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdd4006be30 0x7fdd4006e2f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c000f60 tx=0x7fdd4c007d50 comp rx=0 tx=0).stop 2024-04-24T23:03:55.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.400+0000 7fdd5907d640 1 -- 172.21.15.32:0/2526873402 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd5415e9b0 msgr2=0x7fdd5415b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:55.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.400+0000 7fdd5907d640 1 --2- 172.21.15.32:0/2526873402 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd5415e9b0 0x7fdd5415b2e0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fdd4400cd60 tx=0x7fdd4400a660 comp rx=0 tx=0).stop 2024-04-24T23:03:55.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.400+0000 7fdd5907d640 1 -- 172.21.15.32:0/2526873402 shutdown_connections 2024-04-24T23:03:55.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.400+0000 7fdd5907d640 1 --2- 172.21.15.32:0/2526873402 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdd4006be30 0x7fdd4006e2f0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:55.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.400+0000 7fdd5907d640 1 --2- 172.21.15.32:0/2526873402 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd5415e9b0 0x7fdd5415b2e0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:55.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.400+0000 7fdd5907d640 1 --2- 172.21.15.32:0/2526873402 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd540d4210 0x7fdd541f4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:55.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.400+0000 7fdd5907d640 1 --2- 172.21.15.32:0/2526873402 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdd5406b820 0x7fdd54158850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:55.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.400+0000 7fdd5907d640 1 -- 172.21.15.32:0/2526873402 >> 172.21.15.32:0/2526873402 conn(0x7fdd540d11d0 msgr2=0x7fdd54154a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:55.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.400+0000 7fdd5907d640 1 -- 172.21.15.32:0/2526873402 shutdown_connections 2024-04-24T23:03:55.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.400+0000 7fdd5907d640 1 -- 172.21.15.32:0/2526873402 wait complete. 2024-04-24T23:03:55.415 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2024-04-24T23:03:55.535 INFO:teuthology.orchestra.run.smithi032.stdout:{"osd_objectstore":"memstore"} 2024-04-24T23:03:55.535 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2024-04-24T23:03:55.536 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-24T23:03:55.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.637+0000 7f67d6f1d640 1 Processor -- start 2024-04-24T23:03:55.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.637+0000 7f67d6f1d640 1 -- start start 2024-04-24T23:03:55.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.638+0000 7f67d6f1d640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f67d006d390 0x7f67d006b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:55.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.638+0000 7f67d6f1d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67d006c7b0 con 0x7f67d006b650 2024-04-24T23:03:55.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.638+0000 7f67d6f1d640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67d006ca90 con 0x7f67d015c7b0 2024-04-24T23:03:55.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.638+0000 7f67d6f1d640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f67d015cb90 con 0x7f67d006d390 2024-04-24T23:03:55.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.638+0000 7f67d54ba640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f67d006b650 0x7f67d0070d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60500/0 (socket says 172.21.15.32:60500) 2024-04-24T23:03:55.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.638+0000 7f67d54ba640 1 -- 172.21.15.32:0/312942218 learned_addr learned my addr 172.21.15.32:0/312942218 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:03:55.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.639+0000 7f67cf7fe640 1 -- 172.21.15.32:0/312942218 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2446280641 0 0) 0x7f67d006c7b0 con 0x7f67d006b650 2024-04-24T23:03:55.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.639+0000 7f67cf7fe640 1 -- 172.21.15.32:0/312942218 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f67b8003520 con 0x7f67d006b650 2024-04-24T23:03:55.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.639+0000 7f67cffff640 1 --2- 172.21.15.32:0/312942218 >> v2:172.21.15.121:3300/0 conn(0x7f67d006d390 0x7f67d006b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:55.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.639+0000 7f67cf7fe640 1 -- 172.21.15.32:0/312942218 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1666239121 0 0) 0x7f67b8003520 con 0x7f67d006b650 2024-04-24T23:03:55.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.639+0000 7f67cf7fe640 1 -- 172.21.15.32:0/312942218 >> v2:172.21.15.121:3300/0 conn(0x7f67d006d390 msgr2=0x7f67d006b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:55.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.639+0000 7f67cf7fe640 1 --2- 172.21.15.32:0/312942218 >> v2:172.21.15.121:3300/0 conn(0x7f67d006d390 0x7f67d006b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:55.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.639+0000 7f67cf7fe640 1 -- 172.21.15.32:0/312942218 >> v1:172.21.15.121:6789/0 conn(0x7f67d015c7b0 legacy=0x7f67d00cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:55.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.639+0000 7f67cf7fe640 1 -- 172.21.15.32:0/312942218 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67d006cd70 con 0x7f67d006b650 2024-04-24T23:03:55.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.639+0000 7f67cf7fe640 1 -- 172.21.15.32:0/312942218 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f67c0001670 con 0x7f67d006b650 2024-04-24T23:03:55.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.639+0000 7f67cf7fe640 1 -- 172.21.15.32:0/312942218 >> v1:172.21.15.32:6789/0 conn(0x7f67d006b650 legacy=0x7f67d0070d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:55.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.639+0000 7f67cf7fe640 1 --2- 172.21.15.32:0/312942218 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f67b80039b0 0x7f67b8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:55.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.639+0000 7f67cf7fe640 1 --2- 172.21.15.32:0/312942218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67b8004800 0x7f67b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:55.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.639+0000 7f67cf7fe640 1 --2- 172.21.15.32:0/312942218 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67b80051c0 0x7f67b800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:55.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.639+0000 7f67cf7fe640 1 -- 172.21.15.32:0/312942218 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f67b800c030 con 0x7f67b8004800 2024-04-24T23:03:55.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.639+0000 7f67cf7fe640 1 -- 172.21.15.32:0/312942218 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f67b800c1e0 con 0x7f67b80051c0 2024-04-24T23:03:55.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.639+0000 7f67cf7fe640 1 -- 172.21.15.32:0/312942218 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f67b800c3e0 con 0x7f67b80039b0 2024-04-24T23:03:55.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.639+0000 7f67cffff640 1 --2- 172.21.15.32:0/312942218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67b8004800 0x7f67b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:55.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.639+0000 7f67d4cb9640 1 --2- 172.21.15.32:0/312942218 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f67b80039b0 0x7f67b8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:55.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.639+0000 7f67d54ba640 1 --2- 172.21.15.32:0/312942218 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67b80051c0 0x7f67b800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:55.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.640+0000 7f67cffff640 1 -- 172.21.15.32:0/312942218 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f67b80039b0 msgr2=0x7f67b8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:55.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.640+0000 7f67cffff640 1 --2- 172.21.15.32:0/312942218 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f67b80039b0 0x7f67b8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:55.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.640+0000 7f67cffff640 1 -- 172.21.15.32:0/312942218 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67b80051c0 msgr2=0x7f67b800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:55.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.640+0000 7f67cffff640 1 --2- 172.21.15.32:0/312942218 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67b80051c0 0x7f67b800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:55.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.640+0000 7f67cffff640 1 -- 172.21.15.32:0/312942218 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67b8003520 con 0x7f67b8004800 2024-04-24T23:03:55.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.640+0000 7f67d54ba640 1 --2- 172.21.15.32:0/312942218 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67b80051c0 0x7f67b800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:03:55.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.640+0000 7f67d54ba640 1 -- 172.21.15.32:0/312942218 reap_dead start 2024-04-24T23:03:55.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.640+0000 7f67cffff640 1 --2- 172.21.15.32:0/312942218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67b8004800 0x7f67b8004c80 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f67c4009ab0 tx=0x7f67b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e90dccd3edd17279 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:55.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.640+0000 7f67cf7fe640 1 -- 172.21.15.32:0/312942218 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c4036070 con 0x7f67b8004800 2024-04-24T23:03:55.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.640+0000 7f67cf7fe640 1 -- 172.21.15.32:0/312942218 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67c4005990 con 0x7f67b8004800 2024-04-24T23:03:55.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.640+0000 7f67cf7fe640 1 -- 172.21.15.32:0/312942218 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c4009680 con 0x7f67b8004800 2024-04-24T23:03:55.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.641+0000 7f67d6f1d640 1 -- 172.21.15.32:0/312942218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67b8004800 msgr2=0x7f67b8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:55.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.641+0000 7f67d6f1d640 1 --2- 172.21.15.32:0/312942218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67b8004800 0x7f67b8004c80 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f67c4009ab0 tx=0x7f67b8005830 comp rx=0 tx=0).stop 2024-04-24T23:03:55.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.641+0000 7f67d6f1d640 1 -- 172.21.15.32:0/312942218 shutdown_connections 2024-04-24T23:03:55.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.641+0000 7f67d6f1d640 1 --2- 172.21.15.32:0/312942218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67b8004800 0x7f67b8004c80 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:55.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.641+0000 7f67d6f1d640 1 -- 172.21.15.32:0/312942218 >> 172.21.15.32:0/312942218 conn(0x7f67d00d0730 msgr2=0x7f67d00d0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:55.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.641+0000 7f67d6f1d640 1 -- 172.21.15.32:0/312942218 shutdown_connections 2024-04-24T23:03:55.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.641+0000 7f67d6f1d640 1 -- 172.21.15.32:0/312942218 wait complete. 2024-04-24T23:03:55.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.642+0000 7f67d6f1d640 1 Processor -- start 2024-04-24T23:03:55.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.642+0000 7f67d6f1d640 1 -- start start 2024-04-24T23:03:55.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.643+0000 7f67d6f1d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67b8004800 0x7f67d00aee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:55.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.643+0000 7f67d6f1d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f67d01f2150 0x7f67d01561e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:55.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.643+0000 7f67d6f1d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67d00aa4a0 0x7f67d00717b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:55.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.643+0000 7f67d6f1d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f67d00d0450 con 0x7f67b8004800 2024-04-24T23:03:55.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.643+0000 7f67d6f1d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f67d00d0170 con 0x7f67d00aa4a0 2024-04-24T23:03:55.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.643+0000 7f67d6f1d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f67d00afc30 con 0x7f67d01f2150 2024-04-24T23:03:55.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.643+0000 7f67d4cb9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67b8004800 0x7f67d00aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:55.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.643+0000 7f67d4cb9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67b8004800 0x7f67d00aee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36984/0 (socket says 172.21.15.32:36984) 2024-04-24T23:03:55.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.643+0000 7f67d4cb9640 1 -- 172.21.15.32:0/4185676878 learned_addr learned my addr 172.21.15.32:0/4185676878 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:55.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.643+0000 7f67d54ba640 1 --2- 172.21.15.32:0/4185676878 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67d00aa4a0 0x7f67d00717b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:55.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.643+0000 7f67cffff640 1 --2- 172.21.15.32:0/4185676878 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f67d01f2150 0x7f67d01561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:55.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.643+0000 7f67d4cb9640 1 -- 172.21.15.32:0/4185676878 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f67d01f2150 msgr2=0x7f67d01561e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:55.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.643+0000 7f67d4cb9640 1 --2- 172.21.15.32:0/4185676878 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f67d01f2150 0x7f67d01561e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:55.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.643+0000 7f67d4cb9640 1 -- 172.21.15.32:0/4185676878 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67d00aa4a0 msgr2=0x7f67d00717b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:55.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.643+0000 7f67d4cb9640 1 --2- 172.21.15.32:0/4185676878 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67d00aa4a0 0x7f67d00717b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:55.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.643+0000 7f67d4cb9640 1 -- 172.21.15.32:0/4185676878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67d01f28c0 con 0x7f67b8004800 2024-04-24T23:03:55.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.644+0000 7f67d4cb9640 1 --2- 172.21.15.32:0/4185676878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67b8004800 0x7f67d00aee70 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f67c800fbc0 tx=0x7f67c800d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:55.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.644+0000 7f67cd7fa640 1 -- 172.21.15.32:0/4185676878 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c800dc30 con 0x7f67b8004800 2024-04-24T23:03:55.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.644+0000 7f67cd7fa640 1 -- 172.21.15.32:0/4185676878 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67c8018070 con 0x7f67b8004800 2024-04-24T23:03:55.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.644+0000 7f67d6f1d640 1 -- 172.21.15.32:0/4185676878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67d01f2ba0 con 0x7f67b8004800 2024-04-24T23:03:55.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.644+0000 7f67cd7fa640 1 -- 172.21.15.32:0/4185676878 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c8003970 con 0x7f67b8004800 2024-04-24T23:03:55.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.644+0000 7f67d6f1d640 1 -- 172.21.15.32:0/4185676878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67d00b0fe0 con 0x7f67b8004800 2024-04-24T23:03:55.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.645+0000 7f67d6f1d640 1 -- 172.21.15.32:0/4185676878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67d0068430 con 0x7f67b8004800 2024-04-24T23:03:55.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.645+0000 7f67cd7fa640 1 -- 172.21.15.32:0/4185676878 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f67c8003b10 con 0x7f67b8004800 2024-04-24T23:03:55.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.646+0000 7f67cd7fa640 1 --2- 172.21.15.32:0/4185676878 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f67bc06be60 0x7f67bc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:55.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.646+0000 7f67cd7fa640 1 -- 172.21.15.32:0/4185676878 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 4943+0+0 (secure 0 0 0) 0x7f67c808ac40 con 0x7f67b8004800 2024-04-24T23:03:55.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.646+0000 7f67cffff640 1 --2- 172.21.15.32:0/4185676878 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f67bc06be60 0x7f67bc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:55.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.647+0000 7f67cffff640 1 --2- 172.21.15.32:0/4185676878 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f67bc06be60 0x7f67bc06e320 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f67c4033000 tx=0x7f67c4008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:55.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.650+0000 7f67cd7fa640 1 -- 172.21.15.32:0/4185676878 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f67c80592a0 con 0x7f67b8004800 2024-04-24T23:03:55.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.835+0000 7f67d6f1d640 1 -- 172.21.15.32:0/4185676878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f67d0159d40 con 0x7f67b8004800 2024-04-24T23:03:55.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.838+0000 7f67cd7fa640 1 -- 172.21.15.32:0/4185676878 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f67c805d5a0 con 0x7f67b8004800 2024-04-24T23:03:55.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.840+0000 7f67d6f1d640 1 -- 172.21.15.32:0/4185676878 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f67bc06be60 msgr2=0x7f67bc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:55.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.840+0000 7f67d6f1d640 1 --2- 172.21.15.32:0/4185676878 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f67bc06be60 0x7f67bc06e320 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f67c4033000 tx=0x7f67c4008030 comp rx=0 tx=0).stop 2024-04-24T23:03:55.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.840+0000 7f67d6f1d640 1 -- 172.21.15.32:0/4185676878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67b8004800 msgr2=0x7f67d00aee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:55.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.840+0000 7f67d6f1d640 1 --2- 172.21.15.32:0/4185676878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67b8004800 0x7f67d00aee70 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f67c800fbc0 tx=0x7f67c800d520 comp rx=0 tx=0).stop 2024-04-24T23:03:55.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.840+0000 7f67d6f1d640 1 -- 172.21.15.32:0/4185676878 shutdown_connections 2024-04-24T23:03:55.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.840+0000 7f67d6f1d640 1 --2- 172.21.15.32:0/4185676878 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f67bc06be60 0x7f67bc06e320 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:55.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.840+0000 7f67d6f1d640 1 --2- 172.21.15.32:0/4185676878 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67d00aa4a0 0x7f67d00717b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:55.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.840+0000 7f67d6f1d640 1 --2- 172.21.15.32:0/4185676878 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f67d01f2150 0x7f67d01561e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:55.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.840+0000 7f67d6f1d640 1 --2- 172.21.15.32:0/4185676878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67b8004800 0x7f67d00aee70 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:55.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.840+0000 7f67d6f1d640 1 -- 172.21.15.32:0/4185676878 >> 172.21.15.32:0/4185676878 conn(0x7f67d00d0730 msgr2=0x7f67d015f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:55.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.840+0000 7f67d6f1d640 1 -- 172.21.15.32:0/4185676878 shutdown_connections 2024-04-24T23:03:55.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:55.840+0000 7f67d6f1d640 1 -- 172.21.15.32:0/4185676878 wait complete. 2024-04-24T23:03:55.852 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-24T23:03:55.853 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:03:55.903 DEBUG:teuthology.orchestra.run.smithi121:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:03:55.947 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-24T23:03:55.947 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:03:55.947 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:03:55.963 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=413 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T23:03:55.963 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:03:55.963 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:03:56.005 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3271 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T23:03:56.005 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:03:56.005 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:03:56.020 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=413 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T23:03:56.021 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:03:56.021 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:03:56.062 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3271 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T23:03:56.063 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-24T23:03:56.063 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:03:56.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.161+0000 7f5385cf5640 1 Processor -- start 2024-04-24T23:03:56.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.161+0000 7f5385cf5640 1 -- start start 2024-04-24T23:03:56.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.162+0000 7f5385cf5640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f538015e9b0 0x7f53800d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:56.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.162+0000 7f5385cf5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f538006b820 0x7f53800b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:56.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.162+0000 7f5385cf5640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53800d3170 con 0x7f53800d4210 2024-04-24T23:03:56.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.162+0000 7f5385cf5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f53800d3450 con 0x7f538006b820 2024-04-24T23:03:56.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.162+0000 7f5385cf5640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f53800d3730 con 0x7f538015e9b0 2024-04-24T23:03:56.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.162+0000 7f537ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f538006b820 0x7f53800b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:56.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.162+0000 7f537ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f538006b820 0x7f53800b78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36998/0 (socket says 172.21.15.32:36998) 2024-04-24T23:03:56.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.162+0000 7f537ffff640 1 -- 172.21.15.32:0/496890044 learned_addr learned my addr 172.21.15.32:0/496890044 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:56.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.162+0000 7f537f7fe640 1 --2- 172.21.15.32:0/496890044 >> v2:172.21.15.121:3300/0 conn(0x7f538015e9b0 0x7f53800d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:56.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.163+0000 7f537ffff640 1 -- 172.21.15.32:0/496890044 >> v2:172.21.15.121:3300/0 conn(0x7f538015e9b0 msgr2=0x7f53800d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:56.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.163+0000 7f537ffff640 1 --2- 172.21.15.32:0/496890044 >> v2:172.21.15.121:3300/0 conn(0x7f538015e9b0 0x7f53800d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:56.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.163+0000 7f537ffff640 1 -- 172.21.15.32:0/496890044 >> v1:172.21.15.121:6789/0 conn(0x7f53800d4210 legacy=0x7f538006b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:56.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.163+0000 7f537ffff640 1 -- 172.21.15.32:0/496890044 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53800d3a10 con 0x7f538006b820 2024-04-24T23:03:56.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.163+0000 7f537ffff640 1 --2- 172.21.15.32:0/496890044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f538006b820 0x7f53800b78f0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f538006be20 tx=0x7f537802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cca773fa58c41853 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:56.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.164+0000 7f537e7fc640 1 -- 172.21.15.32:0/496890044 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5378036070 con 0x7f538006b820 2024-04-24T23:03:56.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.164+0000 7f537e7fc640 1 -- 172.21.15.32:0/496890044 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5378031600 con 0x7f538006b820 2024-04-24T23:03:56.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.164+0000 7f537e7fc640 1 -- 172.21.15.32:0/496890044 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53780319d0 con 0x7f538006b820 2024-04-24T23:03:56.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.164+0000 7f5385cf5640 1 -- 172.21.15.32:0/496890044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f538006b820 msgr2=0x7f53800b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:56.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.164+0000 7f5385cf5640 1 --2- 172.21.15.32:0/496890044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f538006b820 0x7f53800b78f0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f538006be20 tx=0x7f537802a6a0 comp rx=0 tx=0).stop 2024-04-24T23:03:56.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.164+0000 7f5385cf5640 1 -- 172.21.15.32:0/496890044 shutdown_connections 2024-04-24T23:03:56.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.164+0000 7f5385cf5640 1 --2- 172.21.15.32:0/496890044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f538006b820 0x7f53800b78f0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:56.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.164+0000 7f5385cf5640 1 --2- 172.21.15.32:0/496890044 >> v2:172.21.15.121:3300/0 conn(0x7f538015e9b0 0x7f53800d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:56.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.164+0000 7f5385cf5640 1 -- 172.21.15.32:0/496890044 >> 172.21.15.32:0/496890044 conn(0x7f53800d11d0 msgr2=0x7f53800d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:56.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.164+0000 7f5385cf5640 1 -- 172.21.15.32:0/496890044 shutdown_connections 2024-04-24T23:03:56.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.164+0000 7f5385cf5640 1 -- 172.21.15.32:0/496890044 wait complete. 2024-04-24T23:03:56.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.165+0000 7f5385cf5640 1 Processor -- start 2024-04-24T23:03:56.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.165+0000 7f5385cf5640 1 -- start start 2024-04-24T23:03:56.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.166+0000 7f5385cf5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f538006b820 0x7f5380156120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:56.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.166+0000 7f5385cf5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53800d4210 0x7f53800cd460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:56.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.166+0000 7f5385cf5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f538015e9b0 0x7f53800cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:56.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.166+0000 7f5385cf5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f53800ba0e0 con 0x7f538015e9b0 2024-04-24T23:03:56.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.166+0000 7f5385cf5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f53800d2400 con 0x7f538006b820 2024-04-24T23:03:56.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.166+0000 7f5385cf5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f53800ba260 con 0x7f53800d4210 2024-04-24T23:03:56.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.166+0000 7f537ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f538015e9b0 0x7f53800cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:56.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.166+0000 7f537ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f538015e9b0 0x7f53800cd9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37000/0 (socket says 172.21.15.32:37000) 2024-04-24T23:03:56.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.166+0000 7f537ffff640 1 -- 172.21.15.32:0/2654442293 learned_addr learned my addr 172.21.15.32:0/2654442293 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:56.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.166+0000 7f537ffff640 1 -- 172.21.15.32:0/2654442293 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53800d4210 msgr2=0x7f53800cd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:56.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.167+0000 7f537ffff640 1 --2- 172.21.15.32:0/2654442293 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53800d4210 0x7f53800cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:56.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.167+0000 7f537ffff640 1 -- 172.21.15.32:0/2654442293 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f538006b820 msgr2=0x7f5380156120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:56.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.167+0000 7f537f7fe640 1 --2- 172.21.15.32:0/2654442293 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f538006b820 0x7f5380156120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:56.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.167+0000 7f537ffff640 1 --2- 172.21.15.32:0/2654442293 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f538006b820 0x7f5380156120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:56.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.167+0000 7f537ffff640 1 -- 172.21.15.32:0/2654442293 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5380156660 con 0x7f538015e9b0 2024-04-24T23:03:56.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.167+0000 7f537ffff640 1 --2- 172.21.15.32:0/2654442293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f538015e9b0 0x7f53800cd9a0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f5378009f40 tx=0x7f5378004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:56.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.167+0000 7f5367fff640 1 -- 172.21.15.32:0/2654442293 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5378036050 con 0x7f538015e9b0 2024-04-24T23:03:56.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.167+0000 7f5367fff640 1 -- 172.21.15.32:0/2654442293 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5378042070 con 0x7f538015e9b0 2024-04-24T23:03:56.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.167+0000 7f5367fff640 1 -- 172.21.15.32:0/2654442293 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5378009510 con 0x7f538015e9b0 2024-04-24T23:03:56.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.167+0000 7f5385cf5640 1 -- 172.21.15.32:0/2654442293 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5380154610 con 0x7f538015e9b0 2024-04-24T23:03:56.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.167+0000 7f5385cf5640 1 -- 172.21.15.32:0/2654442293 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5380154bd0 con 0x7f538015e9b0 2024-04-24T23:03:56.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.168+0000 7f5385cf5640 1 -- 172.21.15.32:0/2654442293 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5380068430 con 0x7f538015e9b0 2024-04-24T23:03:56.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.168+0000 7f5367fff640 1 -- 172.21.15.32:0/2654442293 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5378007bd0 con 0x7f538015e9b0 2024-04-24T23:03:56.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.169+0000 7f5367fff640 1 --2- 172.21.15.32:0/2654442293 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f536c06be30 0x7f536c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:56.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.169+0000 7f5367fff640 1 -- 172.21.15.32:0/2654442293 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 4943+0+0 (secure 0 0 0) 0x7f53780abaf0 con 0x7f538015e9b0 2024-04-24T23:03:56.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.169+0000 7f537f7fe640 1 --2- 172.21.15.32:0/2654442293 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f536c06be30 0x7f536c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:56.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.170+0000 7f537f7fe640 1 --2- 172.21.15.32:0/2654442293 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f536c06be30 0x7f536c06e2f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5370006fd0 tx=0x7f5370008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:56.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.173+0000 7f5367fff640 1 -- 172.21.15.32:0/2654442293 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f537807a700 con 0x7f538015e9b0 2024-04-24T23:03:56.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.393+0000 7f5385cf5640 1 -- 172.21.15.32:0/2654442293 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5380156c10 con 0x7f538015e9b0 2024-04-24T23:03:56.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.394+0000 7f5367fff640 1 -- 172.21.15.32:0/2654442293 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+3875 (secure 0 0 0) 0x7f537807a000 con 0x7f538015e9b0 2024-04-24T23:03:56.395 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:03:56.395 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":38,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5486,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/3550206194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3550206194},{"type":"v1","addr":"172.21.15.32:6834","nonce":3550206194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5905,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1706188546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1706188546},{"type":"v1","addr":"172.21.15.121:6835","nonce":1706188546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5911,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2669443498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2669443498},{"type":"v1","addr":"172.21.15.121:6837","nonce":2669443498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":6816,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3010950858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":3010950858},{"type":"v1","addr":"172.21.15.32:6835","nonce":3010950858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[]} 2024-04-24T23:03:56.396 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 38 2024-04-24T23:03:56.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.396+0000 7f5385cf5640 1 -- 172.21.15.32:0/2654442293 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f536c06be30 msgr2=0x7f536c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:56.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.396+0000 7f5385cf5640 1 --2- 172.21.15.32:0/2654442293 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f536c06be30 0x7f536c06e2f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5370006fd0 tx=0x7f5370008040 comp rx=0 tx=0).stop 2024-04-24T23:03:56.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.396+0000 7f5385cf5640 1 -- 172.21.15.32:0/2654442293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f538015e9b0 msgr2=0x7f53800cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:56.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.396+0000 7f5385cf5640 1 --2- 172.21.15.32:0/2654442293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f538015e9b0 0x7f53800cd9a0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f5378009f40 tx=0x7f5378004a40 comp rx=0 tx=0).stop 2024-04-24T23:03:56.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.396+0000 7f5385cf5640 1 -- 172.21.15.32:0/2654442293 shutdown_connections 2024-04-24T23:03:56.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.396+0000 7f5385cf5640 1 --2- 172.21.15.32:0/2654442293 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f536c06be30 0x7f536c06e2f0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:56.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.396+0000 7f5385cf5640 1 --2- 172.21.15.32:0/2654442293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f538015e9b0 0x7f53800cd9a0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:56.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.396+0000 7f5385cf5640 1 --2- 172.21.15.32:0/2654442293 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53800d4210 0x7f53800cd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:56.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.396+0000 7f5385cf5640 1 --2- 172.21.15.32:0/2654442293 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f538006b820 0x7f5380156120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:56.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.396+0000 7f5385cf5640 1 -- 172.21.15.32:0/2654442293 >> 172.21.15.32:0/2654442293 conn(0x7f53800d11d0 msgr2=0x7f5380071730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:56.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.396+0000 7f5385cf5640 1 -- 172.21.15.32:0/2654442293 shutdown_connections 2024-04-24T23:03:56.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.396+0000 7f5385cf5640 1 -- 172.21.15.32:0/2654442293 wait complete. 2024-04-24T23:03:56.409 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-24T23:03:56.409 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-24T23:03:56.409 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-24T23:03:56.409 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:03:56.409 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-24T23:03:56.409 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-24T23:03:56.410 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-24T23:03:56.410 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:03:56.410 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-24T23:03:56.410 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-24T23:03:56.410 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-24T23:03:56.410 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:03:56.411 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-24T23:03:56.411 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-24T23:03:56.411 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-24T23:03:56.411 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:03:56.418 INFO:tasks.ceph.mds.a:Stopped 2024-04-24T23:03:56.419 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-24T23:03:56.419 INFO:tasks.ceph.mds.c:Stopped 2024-04-24T23:03:56.420 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-24T23:03:56.422 INFO:tasks.ceph.mds.b:Stopped 2024-04-24T23:03:56.422 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-24T23:03:56.423 INFO:tasks.ceph.mds.d:Stopped 2024-04-24T23:03:56.423 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-24T23:03:56.462 INFO:tasks.ceph.mds.a:Started 2024-04-24T23:03:56.462 DEBUG:teuthology.parallel:result is None 2024-04-24T23:03:56.463 INFO:tasks.ceph.mds.c:Started 2024-04-24T23:03:56.463 DEBUG:teuthology.parallel:result is None 2024-04-24T23:03:56.465 INFO:tasks.ceph.mds.b:Started 2024-04-24T23:03:56.465 DEBUG:teuthology.parallel:result is None 2024-04-24T23:03:56.465 INFO:tasks.ceph.mds.d:Started 2024-04-24T23:03:56.466 DEBUG:teuthology.parallel:result is None 2024-04-24T23:03:56.466 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-24T23:03:56.538 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2024-04-24T23:03:56.538 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2024-04-24T23:03:56.545 INFO:tasks.ceph.mds.b.smithi121.stdout:starting mds.b at 2024-04-24T23:03:56.545 INFO:tasks.ceph.mds.d.smithi121.stdout:starting mds.d at 2024-04-24T23:03:56.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.571+0000 7fb8a155a640 1 Processor -- start 2024-04-24T23:03:56.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.571+0000 7fb8a155a640 1 -- start start 2024-04-24T23:03:56.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.571+0000 7fb8a155a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb89c15fd60 0x7fb89c06c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:56.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.571+0000 7fb8a155a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb89c06cb30 0x7fb89c0cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:56.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.571+0000 7fb8a155a640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb89c0b46c0 con 0x7fb89c15b480 2024-04-24T23:03:56.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.571+0000 7fb8a155a640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fb89c0b4ff0 con 0x7fb89c06cb30 2024-04-24T23:03:56.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.571+0000 7fb8a155a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fb89c0b49a0 con 0x7fb89c15fd60 2024-04-24T23:03:56.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.572+0000 7fb89b7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb89c06cb30 0x7fb89c0cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:56.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.572+0000 7fb89b7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb89c06cb30 0x7fb89c0cdf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37040/0 (socket says 172.21.15.32:37040) 2024-04-24T23:03:56.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.572+0000 7fb89b7fe640 1 -- 172.21.15.32:0/1256605363 learned_addr learned my addr 172.21.15.32:0/1256605363 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:56.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.572+0000 7fb89a7fc640 1 --2- 172.21.15.32:0/1256605363 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb89c15fd60 0x7fb89c06c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:56.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.572+0000 7fb89b7fe640 1 -- 172.21.15.32:0/1256605363 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb89c15fd60 msgr2=0x7fb89c06c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:56.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.572+0000 7fb89b7fe640 1 --2- 172.21.15.32:0/1256605363 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb89c15fd60 0x7fb89c06c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:56.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.572+0000 7fb89b7fe640 1 -- 172.21.15.32:0/1256605363 >> v1:172.21.15.121:6789/0 conn(0x7fb89c15b480 legacy=0x7fb89c0b5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:56.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.572+0000 7fb89b7fe640 1 -- 172.21.15.32:0/1256605363 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb89c0b4c80 con 0x7fb89c06cb30 2024-04-24T23:03:56.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.573+0000 7fb89b7fe640 1 --2- 172.21.15.32:0/1256605363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb89c06cb30 0x7fb89c0cdf80 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fb894009e50 tx=0x7fb89402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d677086767b8d0e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:56.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.573+0000 7fb899ffb640 1 -- 172.21.15.32:0/1256605363 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb894036070 con 0x7fb89c06cb30 2024-04-24T23:03:56.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.573+0000 7fb899ffb640 1 -- 172.21.15.32:0/1256605363 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb894031600 con 0x7fb89c06cb30 2024-04-24T23:03:56.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.573+0000 7fb899ffb640 1 -- 172.21.15.32:0/1256605363 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8940319d0 con 0x7fb89c06cb30 2024-04-24T23:03:56.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.573+0000 7fb8a155a640 1 -- 172.21.15.32:0/1256605363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb89c06cb30 msgr2=0x7fb89c0cdf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:56.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.573+0000 7fb8a155a640 1 --2- 172.21.15.32:0/1256605363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb89c06cb30 0x7fb89c0cdf80 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fb894009e50 tx=0x7fb89402a710 comp rx=0 tx=0).stop 2024-04-24T23:03:56.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.573+0000 7fb8a155a640 1 -- 172.21.15.32:0/1256605363 shutdown_connections 2024-04-24T23:03:56.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.573+0000 7fb8a155a640 1 --2- 172.21.15.32:0/1256605363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb89c06cb30 0x7fb89c0cdf80 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:56.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.573+0000 7fb8a155a640 1 --2- 172.21.15.32:0/1256605363 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb89c15fd60 0x7fb89c06c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:56.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.573+0000 7fb8a155a640 1 -- 172.21.15.32:0/1256605363 >> 172.21.15.32:0/1256605363 conn(0x7fb89c06b110 msgr2=0x7fb89c06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:56.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.573+0000 7fb8a155a640 1 -- 172.21.15.32:0/1256605363 shutdown_connections 2024-04-24T23:03:56.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.573+0000 7fb8a155a640 1 -- 172.21.15.32:0/1256605363 wait complete. 2024-04-24T23:03:56.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.575+0000 7fb8a155a640 1 Processor -- start 2024-04-24T23:03:56.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.575+0000 7fb8a155a640 1 -- start start 2024-04-24T23:03:56.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.575+0000 7fb8a155a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb89c06cb30 0x7fb89c1f87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:56.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.575+0000 7fb8a155a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb89c15b480 0x7fb89c148ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:56.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.575+0000 7fb8a155a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb89c15fd60 0x7fb89c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:56.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.575+0000 7fb8a155a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb89c15b860 con 0x7fb89c15b480 2024-04-24T23:03:56.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.575+0000 7fb8a155a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb89c0ce4c0 con 0x7fb89c15fd60 2024-04-24T23:03:56.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.575+0000 7fb8a155a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fb89c1f8cf0 con 0x7fb89c06cb30 2024-04-24T23:03:56.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.575+0000 7fb89a7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb89c15b480 0x7fb89c148ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:56.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.576+0000 7fb89a7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb89c15b480 0x7fb89c148ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37048/0 (socket says 172.21.15.32:37048) 2024-04-24T23:03:56.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.576+0000 7fb89a7fc640 1 -- 172.21.15.32:0/1439696761 learned_addr learned my addr 172.21.15.32:0/1439696761 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:56.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.576+0000 7fb89b7fe640 1 --2- 172.21.15.32:0/1439696761 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb89c15fd60 0x7fb89c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:56.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.576+0000 7fb89affd640 1 --2- 172.21.15.32:0/1439696761 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb89c06cb30 0x7fb89c1f87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:56.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.576+0000 7fb89a7fc640 1 -- 172.21.15.32:0/1439696761 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb89c06cb30 msgr2=0x7fb89c1f87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:56.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.576+0000 7fb89a7fc640 1 --2- 172.21.15.32:0/1439696761 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb89c06cb30 0x7fb89c1f87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:56.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.576+0000 7fb89a7fc640 1 -- 172.21.15.32:0/1439696761 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb89c15fd60 msgr2=0x7fb89c0706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:56.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.576+0000 7fb89a7fc640 1 --2- 172.21.15.32:0/1439696761 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb89c15fd60 0x7fb89c0706c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:56.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.576+0000 7fb89a7fc640 1 -- 172.21.15.32:0/1439696761 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb89c14c870 con 0x7fb89c15b480 2024-04-24T23:03:56.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.576+0000 7fb89a7fc640 1 --2- 172.21.15.32:0/1439696761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb89c15b480 0x7fb89c148ea0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fb88c00e870 tx=0x7fb88c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:56.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.576+0000 7fb8837fe640 1 -- 172.21.15.32:0/1439696761 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb88c017070 con 0x7fb89c15b480 2024-04-24T23:03:56.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.576+0000 7fb8837fe640 1 -- 172.21.15.32:0/1439696761 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb88c00c8e0 con 0x7fb89c15b480 2024-04-24T23:03:56.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.576+0000 7fb8a155a640 1 -- 172.21.15.32:0/1439696761 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb89c1e5c90 con 0x7fb89c15b480 2024-04-24T23:03:56.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.576+0000 7fb8837fe640 1 -- 172.21.15.32:0/1439696761 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb88c00ccd0 con 0x7fb89c15b480 2024-04-24T23:03:56.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.576+0000 7fb8a155a640 1 -- 172.21.15.32:0/1439696761 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb89c158eb0 con 0x7fb89c15b480 2024-04-24T23:03:56.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.577+0000 7fb8a155a640 1 -- 172.21.15.32:0/1439696761 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb89c068430 con 0x7fb89c15b480 2024-04-24T23:03:56.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.578+0000 7fb8837fe640 1 -- 172.21.15.32:0/1439696761 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb88c01a070 con 0x7fb89c15b480 2024-04-24T23:03:56.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.578+0000 7fb8837fe640 1 --2- 172.21.15.32:0/1439696761 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb88806be30 0x7fb88806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:56.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.578+0000 7fb8837fe640 1 -- 172.21.15.32:0/1439696761 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 4943+0+0 (secure 0 0 0) 0x7fb88c08aca0 con 0x7fb89c15b480 2024-04-24T23:03:56.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.578+0000 7fb89affd640 1 --2- 172.21.15.32:0/1439696761 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb88806be30 0x7fb88806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:56.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.579+0000 7fb89affd640 1 --2- 172.21.15.32:0/1439696761 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb88806be30 0x7fb88806e2f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb890001780 tx=0x7fb890009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:56.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.583+0000 7fb8837fe640 1 -- 172.21.15.32:0/1439696761 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb88c059300 con 0x7fb89c15b480 2024-04-24T23:03:56.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:56.781+0000 7fb8a155a640 1 -- 172.21.15.32:0/1439696761 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fb89c06c000 con 0x7fb89c15b480 2024-04-24T23:03:57.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.407+0000 7fb8837fe640 1 -- 172.21.15.32:0/1439696761 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v98) ==== 97+0+0 (secure 0 0 0) 0x7fb88c05d600 con 0x7fb89c15b480 2024-04-24T23:03:57.409 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blocklist entries 2024-04-24T23:03:57.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.410+0000 7fb8a155a640 1 -- 172.21.15.32:0/1439696761 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb88806be30 msgr2=0x7fb88806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:57.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.410+0000 7fb8a155a640 1 --2- 172.21.15.32:0/1439696761 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb88806be30 0x7fb88806e2f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb890001780 tx=0x7fb890009040 comp rx=0 tx=0).stop 2024-04-24T23:03:57.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.410+0000 7fb8a155a640 1 -- 172.21.15.32:0/1439696761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb89c15b480 msgr2=0x7fb89c148ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:57.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.410+0000 7fb8a155a640 1 --2- 172.21.15.32:0/1439696761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb89c15b480 0x7fb89c148ea0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fb88c00e870 tx=0x7fb88c00ed30 comp rx=0 tx=0).stop 2024-04-24T23:03:57.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.410+0000 7fb8a155a640 1 -- 172.21.15.32:0/1439696761 shutdown_connections 2024-04-24T23:03:57.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.410+0000 7fb8a155a640 1 --2- 172.21.15.32:0/1439696761 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb88806be30 0x7fb88806e2f0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:57.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.410+0000 7fb8a155a640 1 --2- 172.21.15.32:0/1439696761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb89c15b480 0x7fb89c148ea0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:57.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.410+0000 7fb8a155a640 1 --2- 172.21.15.32:0/1439696761 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb89c15fd60 0x7fb89c0706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:57.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.410+0000 7fb8a155a640 1 --2- 172.21.15.32:0/1439696761 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb89c06cb30 0x7fb89c1f87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:57.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.410+0000 7fb8a155a640 1 -- 172.21.15.32:0/1439696761 >> 172.21.15.32:0/1439696761 conn(0x7fb89c06b110 msgr2=0x7fb89c0cc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:57.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.410+0000 7fb8a155a640 1 -- 172.21.15.32:0/1439696761 shutdown_connections 2024-04-24T23:03:57.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.410+0000 7fb8a155a640 1 -- 172.21.15.32:0/1439696761 wait complete. 2024-04-24T23:03:57.426 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-24T23:03:57.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.532+0000 7f07d78ba640 1 Processor -- start 2024-04-24T23:03:57.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.532+0000 7f07d78ba640 1 -- start start 2024-04-24T23:03:57.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.533+0000 7f07d78ba640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f07d006c5f0 0x7f07d006c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:57.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.533+0000 7f07d78ba640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f07d00b44d0 0x7f07d00b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:57.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.533+0000 7f07d78ba640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f07d00af890 0x7f07d00cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:57.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.533+0000 7f07d78ba640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f07d00b6430 con 0x7f07d00b44d0 2024-04-24T23:03:57.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.533+0000 7f07d78ba640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f07d0156f80 con 0x7f07d00af890 2024-04-24T23:03:57.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.533+0000 7f07d78ba640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f07d00b6150 con 0x7f07d006c5f0 2024-04-24T23:03:57.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.533+0000 7f07d4e55640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f07d00b44d0 0x7f07d00b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:57.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.533+0000 7f07d4e55640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f07d00b44d0 0x7f07d00b5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37052/0 (socket says 172.21.15.32:37052) 2024-04-24T23:03:57.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.533+0000 7f07d4e55640 1 -- 172.21.15.32:0/1560768769 learned_addr learned my addr 172.21.15.32:0/1560768769 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:57.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.533+0000 7f07d5e57640 1 --2- 172.21.15.32:0/1560768769 >> v2:172.21.15.121:3300/0 conn(0x7f07d00af890 0x7f07d00cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:57.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.533+0000 7f07d5656640 1 --2- 172.21.15.32:0/1560768769 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f07d006c5f0 0x7f07d006c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:57.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.533+0000 7f07d4e55640 1 -- 172.21.15.32:0/1560768769 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f07d006c5f0 msgr2=0x7f07d006c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:57.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.533+0000 7f07d4e55640 1 --2- 172.21.15.32:0/1560768769 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f07d006c5f0 0x7f07d006c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:57.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.533+0000 7f07d4e55640 1 -- 172.21.15.32:0/1560768769 >> v2:172.21.15.121:3300/0 conn(0x7f07d00af890 msgr2=0x7f07d00cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:57.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.533+0000 7f07d4e55640 1 --2- 172.21.15.32:0/1560768769 >> v2:172.21.15.121:3300/0 conn(0x7f07d00af890 0x7f07d00cef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:57.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.534+0000 7f07d4e55640 1 -- 172.21.15.32:0/1560768769 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07d00b39f0 con 0x7f07d00b44d0 2024-04-24T23:03:57.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.534+0000 7f07d5656640 1 --2- 172.21.15.32:0/1560768769 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f07d006c5f0 0x7f07d006c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:03:57.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.534+0000 7f07d4e55640 1 --2- 172.21.15.32:0/1560768769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07d00b44d0 0x7f07d00b5890 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f07d00aff30 tx=0x7f07c002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fb40150a6b6f1b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:57.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.534+0000 7f07c7fff640 1 -- 172.21.15.32:0/1560768769 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c0037070 con 0x7f07d00b44d0 2024-04-24T23:03:57.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.534+0000 7f07c7fff640 1 -- 172.21.15.32:0/1560768769 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07c0032440 con 0x7f07d00b44d0 2024-04-24T23:03:57.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.534+0000 7f07c7fff640 1 -- 172.21.15.32:0/1560768769 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c0032810 con 0x7f07d00b44d0 2024-04-24T23:03:57.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.535+0000 7f07d78ba640 1 -- 172.21.15.32:0/1560768769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07d00b44d0 msgr2=0x7f07d00b5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:57.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.535+0000 7f07d78ba640 1 --2- 172.21.15.32:0/1560768769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07d00b44d0 0x7f07d00b5890 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f07d00aff30 tx=0x7f07c002a140 comp rx=0 tx=0).stop 2024-04-24T23:03:57.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.535+0000 7f07d78ba640 1 -- 172.21.15.32:0/1560768769 shutdown_connections 2024-04-24T23:03:57.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.535+0000 7f07d78ba640 1 --2- 172.21.15.32:0/1560768769 >> v2:172.21.15.121:3300/0 conn(0x7f07d00af890 0x7f07d00cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:57.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.535+0000 7f07d78ba640 1 --2- 172.21.15.32:0/1560768769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07d00b44d0 0x7f07d00b5890 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:57.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.535+0000 7f07d78ba640 1 --2- 172.21.15.32:0/1560768769 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f07d006c5f0 0x7f07d006c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:57.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.535+0000 7f07d78ba640 1 -- 172.21.15.32:0/1560768769 >> 172.21.15.32:0/1560768769 conn(0x7f07d006b110 msgr2=0x7f07d006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:57.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.535+0000 7f07d78ba640 1 -- 172.21.15.32:0/1560768769 shutdown_connections 2024-04-24T23:03:57.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.535+0000 7f07d78ba640 1 -- 172.21.15.32:0/1560768769 wait complete. 2024-04-24T23:03:57.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.536+0000 7f07d78ba640 1 Processor -- start 2024-04-24T23:03:57.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.536+0000 7f07d78ba640 1 -- start start 2024-04-24T23:03:57.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.537+0000 7f07d78ba640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f07d006c5f0 0x7f07d01f1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:57.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.537+0000 7f07d78ba640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f07d00af890 0x7f07d0158200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:57.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.537+0000 7f07d78ba640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07d00b44d0 0x7f07d00b0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:57.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.537+0000 7f07d78ba640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f07d006cf10 con 0x7f07d00b44d0 2024-04-24T23:03:57.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.537+0000 7f07d78ba640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f07d00b3fb0 con 0x7f07d006c5f0 2024-04-24T23:03:57.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.537+0000 7f07d78ba640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f07d00b5dd0 con 0x7f07d00af890 2024-04-24T23:03:57.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.537+0000 7f07d5e57640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07d00b44d0 0x7f07d00b0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:57.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.537+0000 7f07d5e57640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07d00b44d0 0x7f07d00b0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37064/0 (socket says 172.21.15.32:37064) 2024-04-24T23:03:57.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.537+0000 7f07d5e57640 1 -- 172.21.15.32:0/2755120883 learned_addr learned my addr 172.21.15.32:0/2755120883 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:57.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.537+0000 7f07d4e55640 1 --2- 172.21.15.32:0/2755120883 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f07d00af890 0x7f07d0158200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:57.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.537+0000 7f07d5656640 1 --2- 172.21.15.32:0/2755120883 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f07d006c5f0 0x7f07d01f1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:57.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.537+0000 7f07d5e57640 1 -- 172.21.15.32:0/2755120883 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f07d00af890 msgr2=0x7f07d0158200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:57.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.537+0000 7f07d5e57640 1 --2- 172.21.15.32:0/2755120883 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f07d00af890 0x7f07d0158200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:57.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.537+0000 7f07d5e57640 1 -- 172.21.15.32:0/2755120883 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f07d006c5f0 msgr2=0x7f07d01f1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:57.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.537+0000 7f07d5e57640 1 --2- 172.21.15.32:0/2755120883 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f07d006c5f0 0x7f07d01f1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:57.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.537+0000 7f07d5e57640 1 -- 172.21.15.32:0/2755120883 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07d0158740 con 0x7f07d00b44d0 2024-04-24T23:03:57.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.538+0000 7f07d5e57640 1 --2- 172.21.15.32:0/2755120883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07d00b44d0 0x7f07d00b0ed0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f07cc00e820 tx=0x7f07cc00ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:57.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.538+0000 7f07c5ffb640 1 -- 172.21.15.32:0/2755120883 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07cc017070 con 0x7f07d00b44d0 2024-04-24T23:03:57.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.538+0000 7f07c5ffb640 1 -- 172.21.15.32:0/2755120883 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07cc00ee70 con 0x7f07d00b44d0 2024-04-24T23:03:57.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.538+0000 7f07c5ffb640 1 -- 172.21.15.32:0/2755120883 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07cc00cb60 con 0x7f07d00b44d0 2024-04-24T23:03:57.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.538+0000 7f07d78ba640 1 -- 172.21.15.32:0/2755120883 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07d0157160 con 0x7f07d00b44d0 2024-04-24T23:03:57.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.538+0000 7f07d78ba640 1 -- 172.21.15.32:0/2755120883 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07d0157720 con 0x7f07d00b44d0 2024-04-24T23:03:57.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.539+0000 7f07d78ba640 1 -- 172.21.15.32:0/2755120883 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07d0068430 con 0x7f07d00b44d0 2024-04-24T23:03:57.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.539+0000 7f07c5ffb640 1 -- 172.21.15.32:0/2755120883 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f07cc01a070 con 0x7f07d00b44d0 2024-04-24T23:03:57.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.540+0000 7f07c5ffb640 1 --2- 172.21.15.32:0/2755120883 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f07b406be30 0x7f07b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:57.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.540+0000 7f07c5ffb640 1 -- 172.21.15.32:0/2755120883 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 4857+0+0 (secure 0 0 0) 0x7f07cc08ac20 con 0x7f07d00b44d0 2024-04-24T23:03:57.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.540+0000 7f07d5656640 1 --2- 172.21.15.32:0/2755120883 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f07b406be30 0x7f07b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:57.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.541+0000 7f07d5656640 1 --2- 172.21.15.32:0/2755120883 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f07b406be30 0x7f07b406e2f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f07c8003fd0 tx=0x7f07c80039f0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:57.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.544+0000 7f07c5ffb640 1 -- 172.21.15.32:0/2755120883 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f07cc0599b0 con 0x7f07d00b44d0 2024-04-24T23:03:57.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.794+0000 7f07d78ba640 1 -- 172.21.15.32:0/2755120883 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f07d00b5670 con 0x7f07d00b44d0 2024-04-24T23:03:57.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.794+0000 7f07c5ffb640 1 -- 172.21.15.32:0/2755120883 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) ==== 80+0+5304 (secure 0 0 0) 0x7f07cc0592b0 con 0x7f07d00b44d0 2024-04-24T23:03:57.796 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:03:57.796 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:03:57.797 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2024-04-24T23:03:57.797 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:03:57.797 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2024-04-24T23:03:57.797 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmXRdxAxAA/EYEn1C38XhG7DC+0X4y7A==", 2024-04-24T23:03:57.797 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:03:57.798 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:03:57.798 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-24T23:03:57.798 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-24T23:03:57.798 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:03:57.798 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:03:57.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:03:57.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:03:57.798 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2024-04-24T23:03:57.798 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmak7bDhAA0he+Jp/gkkrfBjYKSObjcw==", 2024-04-24T23:03:57.798 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:03:57.798 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:03:57.799 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-24T23:03:57.799 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-24T23:03:57.799 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:03:57.799 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:03:57.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:03:57.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:03:57.799 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.c", 2024-04-24T23:03:57.799 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmZXFdChAA8TwwkA5EIXHBzPsQSFvXcQ==", 2024-04-24T23:03:57.799 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:03:57.799 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:03:57.800 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-24T23:03:57.800 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-24T23:03:57.800 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:03:57.800 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:03:57.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:03:57.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:03:57.800 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.d", 2024-04-24T23:03:57.800 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmLbGuFRAAe91fZlulziWdWiDdLBLWNA==", 2024-04-24T23:03:57.800 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:03:57.800 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:03:57.800 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-24T23:03:57.800 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-24T23:03:57.800 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:03:57.800 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:03:57.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:03:57.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:03:57.801 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2024-04-24T23:03:57.801 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCfjilmhTwoOBAAig98xa3a2qnv4rriUO9VCQ==", 2024-04-24T23:03:57.801 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:03:57.801 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:03:57.801 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:03:57.801 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:03:57.801 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:03:57.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:03:57.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:03:57.801 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2024-04-24T23:03:57.801 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCgjilmwRaAAxAAUqDOyR9HV6DEnmbyn3WAjA==", 2024-04-24T23:03:57.801 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:03:57.801 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:03:57.801 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:03:57.802 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:03:57.802 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:03:57.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:03:57.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:03:57.802 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2024-04-24T23:03:57.802 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCgjilmDl2qDBAAF1bhTOaSOc612wmFRd8+XA==", 2024-04-24T23:03:57.802 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:03:57.802 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:03:57.802 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:03:57.802 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:03:57.802 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:03:57.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:03:57.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:03:57.802 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2024-04-24T23:03:57.803 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCgjilmAxHwFRAA9a0v/YwngRxq1vmMRsmEkQ==", 2024-04-24T23:03:57.803 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:03:57.803 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:03:57.803 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:03:57.803 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:03:57.803 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:03:57.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:03:57.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:03:57.803 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2024-04-24T23:03:57.803 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCjjilmDPmJIxAASZlEo1ZJHOhIAFGukIj1tQ==", 2024-04-24T23:03:57.803 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:03:57.803 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:03:57.803 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:03:57.803 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:03:57.803 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:03:57.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:03:57.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:03:57.804 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2024-04-24T23:03:57.804 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCjjilmgcpuKhAAedSq0ubNB30s3bz0fY38hw==", 2024-04-24T23:03:57.804 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:03:57.804 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:03:57.804 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:03:57.804 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:03:57.804 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:03:57.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:03:57.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:03:57.804 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2024-04-24T23:03:57.804 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCjjilmFv2VMxAAM+mUc4I0sjawy2uwuUEKYg==", 2024-04-24T23:03:57.804 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:03:57.804 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:03:57.804 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:03:57.805 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:03:57.805 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:03:57.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:03:57.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:03:57.805 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2024-04-24T23:03:57.805 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCkjilmLvEbARAAnaOvQ5Ee0clN/A9SVoKuBQ==", 2024-04-24T23:03:57.805 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:03:57.805 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:03:57.805 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:03:57.805 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:03:57.805 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:03:57.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:03:57.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:03:57.805 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2024-04-24T23:03:57.805 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilm5oeiGRAAdLXwwqrYhV3HHo7Dw/2ZqQ==", 2024-04-24T23:03:57.806 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:03:57.806 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:03:57.806 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2024-04-24T23:03:57.806 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-24T23:03:57.806 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:03:57.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:03:57.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:03:57.806 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2024-04-24T23:03:57.806 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmyWBDIhAARtVP4Unk3HHlQYZ2QZ3GzQ==", 2024-04-24T23:03:57.806 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:03:57.806 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:03:57.806 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-24T23:03:57.806 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-24T23:03:57.806 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-24T23:03:57.807 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:03:57.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:03:57.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:03:57.807 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.2", 2024-04-24T23:03:57.807 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilm4QElHxAAXgE316aRCosxX7XER5gtKQ==", 2024-04-24T23:03:57.807 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:03:57.807 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:03:57.807 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-24T23:03:57.807 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-24T23:03:57.807 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-24T23:03:57.807 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:03:57.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:03:57.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:03:57.807 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.3", 2024-04-24T23:03:57.807 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmMZihJxAAoWaAAFY3vVfZnC4/GoT0+g==", 2024-04-24T23:03:57.807 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:03:57.808 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:03:57.808 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-24T23:03:57.808 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-24T23:03:57.808 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-24T23:03:57.808 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:03:57.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:03:57.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:03:57.808 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2024-04-24T23:03:57.808 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCdjilmdVXeIxAAwGGaAfshx4pHcoc5+3kRww==", 2024-04-24T23:03:57.808 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:03:57.808 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2024-04-24T23:03:57.808 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-24T23:03:57.808 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-24T23:03:57.808 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:03:57.808 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:03:57.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:03:57.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:03:57.809 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2024-04-24T23:03:57.809 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmn3FJABAAGnvNAoNLzK20cQF6beihmw==", 2024-04-24T23:03:57.809 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:03:57.809 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2024-04-24T23:03:57.809 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2024-04-24T23:03:57.809 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:03:57.809 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:03:57.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:03:57.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:03:57.809 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2024-04-24T23:03:57.809 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCdjilmMqs2OBAAOe3bLpnCq+HK2EbeXnwY8Q==", 2024-04-24T23:03:57.809 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:03:57.810 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2024-04-24T23:03:57.810 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2024-04-24T23:03:57.810 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:03:57.810 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:03:57.810 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:03:57.810 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:03:57.810 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:03:57.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.796+0000 7f07d78ba640 1 -- 172.21.15.32:0/2755120883 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f07b406be30 msgr2=0x7f07b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:57.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.796+0000 7f07d78ba640 1 --2- 172.21.15.32:0/2755120883 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f07b406be30 0x7f07b406e2f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f07c8003fd0 tx=0x7f07c80039f0 comp rx=0 tx=0).stop 2024-04-24T23:03:57.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.796+0000 7f07d78ba640 1 -- 172.21.15.32:0/2755120883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07d00b44d0 msgr2=0x7f07d00b0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:57.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.796+0000 7f07d78ba640 1 --2- 172.21.15.32:0/2755120883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07d00b44d0 0x7f07d00b0ed0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f07cc00e820 tx=0x7f07cc00ec20 comp rx=0 tx=0).stop 2024-04-24T23:03:57.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.796+0000 7f07d78ba640 1 -- 172.21.15.32:0/2755120883 shutdown_connections 2024-04-24T23:03:57.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.796+0000 7f07d78ba640 1 --2- 172.21.15.32:0/2755120883 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f07b406be30 0x7f07b406e2f0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:57.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.796+0000 7f07d78ba640 1 --2- 172.21.15.32:0/2755120883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07d00b44d0 0x7f07d00b0ed0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:57.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.796+0000 7f07d78ba640 1 --2- 172.21.15.32:0/2755120883 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f07d00af890 0x7f07d0158200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:57.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.796+0000 7f07d78ba640 1 --2- 172.21.15.32:0/2755120883 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f07d006c5f0 0x7f07d01f1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:57.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.796+0000 7f07d78ba640 1 -- 172.21.15.32:0/2755120883 >> 172.21.15.32:0/2755120883 conn(0x7f07d006b110 msgr2=0x7f07d0072180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:57.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.796+0000 7f07d78ba640 1 -- 172.21.15.32:0/2755120883 shutdown_connections 2024-04-24T23:03:57.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.796+0000 7f07d78ba640 1 -- 172.21.15.32:0/2755120883 wait complete. 2024-04-24T23:03:57.813 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:03:57.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.912+0000 7fe4e1641640 1 Processor -- start 2024-04-24T23:03:57.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.912+0000 7fe4e1641640 1 -- start start 2024-04-24T23:03:57.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.913+0000 7fe4e1641640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe4dc15e9b0 0x7fe4dc0d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:57.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.913+0000 7fe4e1641640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4dc0d3170 con 0x7fe4dc06b820 2024-04-24T23:03:57.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.913+0000 7fe4e1641640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4dc0d3450 con 0x7fe4dc0d4210 2024-04-24T23:03:57.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.913+0000 7fe4e1641640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fe4dc0d3730 con 0x7fe4dc15e9b0 2024-04-24T23:03:57.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.913+0000 7fe4daffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe4dc15e9b0 0x7fe4dc0d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:57.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.913+0000 7fe4daffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe4dc15e9b0 0x7fe4dc0d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37080/0 (socket says 172.21.15.32:37080) 2024-04-24T23:03:57.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.913+0000 7fe4db7fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe4dc06b820 0x7fe4dc0b78f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60516/0 (socket says 172.21.15.32:60516) 2024-04-24T23:03:57.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.913+0000 7fe4daffd640 1 -- 172.21.15.32:0/2410864766 learned_addr learned my addr 172.21.15.32:0/2410864766 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:57.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.914+0000 7fe4daffd640 1 -- 172.21.15.32:0/2410864766 >> v1:172.21.15.121:6789/0 conn(0x7fe4dc0d4210 legacy=0x7fe4dc06b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:57.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.914+0000 7fe4daffd640 1 -- 172.21.15.32:0/2410864766 >> v1:172.21.15.32:6789/0 conn(0x7fe4dc06b820 legacy=0x7fe4dc0b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:57.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.914+0000 7fe4d9ffb640 1 -- 172.21.15.32:0/2410864766 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 468333483 0 0) 0x7fe4dc0d3170 con 0x7fe4dc06b820 2024-04-24T23:03:57.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.914+0000 7fe4daffd640 1 -- 172.21.15.32:0/2410864766 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4dc0d3a10 con 0x7fe4dc15e9b0 2024-04-24T23:03:57.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.914+0000 7fe4daffd640 1 --2- 172.21.15.32:0/2410864766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4dc15e9b0 0x7fe4dc0d2b90 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4009870 tx=0x7fe4d402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92c00d89b273a851 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:57.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.915+0000 7fe4d9ffb640 1 -- 172.21.15.32:0/2410864766 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d4037070 con 0x7fe4dc15e9b0 2024-04-24T23:03:57.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.915+0000 7fe4d9ffb640 1 -- 172.21.15.32:0/2410864766 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4d4032440 con 0x7fe4dc15e9b0 2024-04-24T23:03:57.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.915+0000 7fe4d9ffb640 1 -- 172.21.15.32:0/2410864766 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d4032810 con 0x7fe4dc15e9b0 2024-04-24T23:03:57.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.915+0000 7fe4e1641640 1 -- 172.21.15.32:0/2410864766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4dc15e9b0 msgr2=0x7fe4dc0d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:57.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.915+0000 7fe4e1641640 1 --2- 172.21.15.32:0/2410864766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4dc15e9b0 0x7fe4dc0d2b90 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4009870 tx=0x7fe4d402a270 comp rx=0 tx=0).stop 2024-04-24T23:03:57.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.915+0000 7fe4e1641640 1 -- 172.21.15.32:0/2410864766 shutdown_connections 2024-04-24T23:03:57.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.915+0000 7fe4e1641640 1 --2- 172.21.15.32:0/2410864766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4dc15e9b0 0x7fe4dc0d2b90 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:57.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.915+0000 7fe4e1641640 1 -- 172.21.15.32:0/2410864766 >> 172.21.15.32:0/2410864766 conn(0x7fe4dc0d11d0 msgr2=0x7fe4dc0d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:57.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.915+0000 7fe4e1641640 1 -- 172.21.15.32:0/2410864766 shutdown_connections 2024-04-24T23:03:57.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.915+0000 7fe4e1641640 1 -- 172.21.15.32:0/2410864766 wait complete. 2024-04-24T23:03:57.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.916+0000 7fe4e1641640 1 Processor -- start 2024-04-24T23:03:57.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.917+0000 7fe4e1641640 1 -- start start 2024-04-24T23:03:57.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.917+0000 7fe4e1641640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe4dc06b820 0x7fe4dc1f4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:57.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.917+0000 7fe4e1641640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe4dc0d4210 0x7fe4dc160e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:57.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.917+0000 7fe4e1641640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4dc15e9b0 0x7fe4dc1613a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:57.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.917+0000 7fe4e1641640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc0ba0e0 con 0x7fe4dc15e9b0 2024-04-24T23:03:57.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.917+0000 7fe4e1641640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc0d2400 con 0x7fe4dc06b820 2024-04-24T23:03:57.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.917+0000 7fe4e1641640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe4dc0ba260 con 0x7fe4dc0d4210 2024-04-24T23:03:57.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.917+0000 7fe4db7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4dc15e9b0 0x7fe4dc1613a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:57.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.917+0000 7fe4db7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4dc15e9b0 0x7fe4dc1613a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37086/0 (socket says 172.21.15.32:37086) 2024-04-24T23:03:57.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.917+0000 7fe4db7fe640 1 -- 172.21.15.32:0/2132120847 learned_addr learned my addr 172.21.15.32:0/2132120847 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:57.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.918+0000 7fe4daffd640 1 --2- 172.21.15.32:0/2132120847 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe4dc06b820 0x7fe4dc1f4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:57.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.918+0000 7fe4da7fc640 1 --2- 172.21.15.32:0/2132120847 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe4dc0d4210 0x7fe4dc160e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:57.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.918+0000 7fe4db7fe640 1 -- 172.21.15.32:0/2132120847 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe4dc0d4210 msgr2=0x7fe4dc160e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:57.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.918+0000 7fe4db7fe640 1 --2- 172.21.15.32:0/2132120847 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe4dc0d4210 0x7fe4dc160e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:57.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.918+0000 7fe4db7fe640 1 -- 172.21.15.32:0/2132120847 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe4dc06b820 msgr2=0x7fe4dc1f4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:57.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.918+0000 7fe4db7fe640 1 --2- 172.21.15.32:0/2132120847 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe4dc06b820 0x7fe4dc1f4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:57.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.918+0000 7fe4db7fe640 1 -- 172.21.15.32:0/2132120847 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4dc1618e0 con 0x7fe4dc15e9b0 2024-04-24T23:03:57.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.918+0000 7fe4daffd640 1 --2- 172.21.15.32:0/2132120847 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe4dc06b820 0x7fe4dc1f4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:03:57.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.918+0000 7fe4da7fc640 1 --2- 172.21.15.32:0/2132120847 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe4dc0d4210 0x7fe4dc160e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:03:57.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.918+0000 7fe4db7fe640 1 --2- 172.21.15.32:0/2132120847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4dc15e9b0 0x7fe4dc1613a0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc00c7a0 tx=0x7fe4cc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:57.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.918+0000 7fe4c37fe640 1 -- 172.21.15.32:0/2132120847 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc018070 con 0x7fe4dc15e9b0 2024-04-24T23:03:57.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.918+0000 7fe4c37fe640 1 -- 172.21.15.32:0/2132120847 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4cc00a640 con 0x7fe4dc15e9b0 2024-04-24T23:03:57.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.918+0000 7fe4c37fe640 1 -- 172.21.15.32:0/2132120847 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc00aa70 con 0x7fe4dc15e9b0 2024-04-24T23:03:57.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.918+0000 7fe4e1641640 1 -- 172.21.15.32:0/2132120847 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4dc160410 con 0x7fe4dc15e9b0 2024-04-24T23:03:57.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.918+0000 7fe4e1641640 1 -- 172.21.15.32:0/2132120847 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4dc15f650 con 0x7fe4dc15e9b0 2024-04-24T23:03:57.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.919+0000 7fe4e1641640 1 -- 172.21.15.32:0/2132120847 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4dc068430 con 0x7fe4dc15e9b0 2024-04-24T23:03:57.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.920+0000 7fe4c37fe640 1 -- 172.21.15.32:0/2132120847 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe4cc01b070 con 0x7fe4dc15e9b0 2024-04-24T23:03:57.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.920+0000 7fe4c37fe640 1 --2- 172.21.15.32:0/2132120847 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe4c806be30 0x7fe4c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:57.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.920+0000 7fe4c37fe640 1 -- 172.21.15.32:0/2132120847 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 4857+0+0 (secure 0 0 0) 0x7fe4cc08a0e0 con 0x7fe4dc15e9b0 2024-04-24T23:03:57.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.920+0000 7fe4daffd640 1 --2- 172.21.15.32:0/2132120847 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe4c806be30 0x7fe4c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:57.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.921+0000 7fe4daffd640 1 --2- 172.21.15.32:0/2132120847 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe4c806be30 0x7fe4c806e2f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4034000 tx=0x7fe4d402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:57.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:57.924+0000 7fe4c37fe640 1 -- 172.21.15.32:0/2132120847 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe4dc1618e0 con 0x7fe4dc15e9b0 2024-04-24T23:03:58.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.108+0000 7fe4e1641640 1 -- 172.21.15.32:0/2132120847 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe4dc15f040 con 0x7fe4dc15e9b0 2024-04-24T23:03:58.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.109+0000 7fe4c37fe640 1 -- 172.21.15.32:0/2132120847 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+12905 (secure 0 0 0) 0x7fe4cc058de0 con 0x7fe4dc15e9b0 2024-04-24T23:03:58.111 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:03:58.111 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":100,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:03:57.418515+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:03:58.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.111+0000 7fe4e1641640 1 -- 172.21.15.32:0/2132120847 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe4c806be30 msgr2=0x7fe4c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:58.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.111+0000 7fe4e1641640 1 --2- 172.21.15.32:0/2132120847 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe4c806be30 0x7fe4c806e2f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4034000 tx=0x7fe4d402e000 comp rx=0 tx=0).stop 2024-04-24T23:03:58.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.111+0000 7fe4e1641640 1 -- 172.21.15.32:0/2132120847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4dc15e9b0 msgr2=0x7fe4dc1613a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:58.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.111+0000 7fe4e1641640 1 --2- 172.21.15.32:0/2132120847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4dc15e9b0 0x7fe4dc1613a0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc00c7a0 tx=0x7fe4cc00cc60 comp rx=0 tx=0).stop 2024-04-24T23:03:58.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.111+0000 7fe4e1641640 1 -- 172.21.15.32:0/2132120847 shutdown_connections 2024-04-24T23:03:58.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.111+0000 7fe4e1641640 1 --2- 172.21.15.32:0/2132120847 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe4c806be30 0x7fe4c806e2f0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:58.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.111+0000 7fe4e1641640 1 --2- 172.21.15.32:0/2132120847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4dc15e9b0 0x7fe4dc1613a0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:58.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.111+0000 7fe4e1641640 1 --2- 172.21.15.32:0/2132120847 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe4dc0d4210 0x7fe4dc160e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:58.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.111+0000 7fe4e1641640 1 --2- 172.21.15.32:0/2132120847 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe4dc06b820 0x7fe4dc1f4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:58.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.111+0000 7fe4e1641640 1 -- 172.21.15.32:0/2132120847 >> 172.21.15.32:0/2132120847 conn(0x7fe4dc0d11d0 msgr2=0x7fe4dc15d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:58.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.111+0000 7fe4e1641640 1 -- 172.21.15.32:0/2132120847 shutdown_connections 2024-04-24T23:03:58.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.111+0000 7fe4e1641640 1 -- 172.21.15.32:0/2132120847 wait complete. 2024-04-24T23:03:58.124 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '93', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:03:58.124 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:03:58.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.226+0000 7fe9536b3640 1 Processor -- start 2024-04-24T23:03:58.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.226+0000 7fe9536b3640 1 -- start start 2024-04-24T23:03:58.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.227+0000 7fe9536b3640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe94c074210 0x7fe94c06c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:58.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.227+0000 7fe9536b3640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe94c0b0b80 0x7fe94c0745f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:58.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.227+0000 7fe9536b3640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe94c0b0770 con 0x7fe94c0ae140 2024-04-24T23:03:58.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.227+0000 7fe9536b3640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fe94c0b1c20 con 0x7fe94c074210 2024-04-24T23:03:58.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.227+0000 7fe9536b3640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fe94c074b30 con 0x7fe94c0b0b80 2024-04-24T23:03:58.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.227+0000 7fe95144f640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe94c074210 0x7fe94c06c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:58.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.227+0000 7fe95144f640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe94c074210 0x7fe94c06c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37100/0 (socket says 172.21.15.32:37100) 2024-04-24T23:03:58.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.227+0000 7fe95144f640 1 -- 172.21.15.32:0/190152320 learned_addr learned my addr 172.21.15.32:0/190152320 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:58.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.227+0000 7fe95144f640 1 -- 172.21.15.32:0/190152320 >> v2:172.21.15.121:3300/0 conn(0x7fe94c0b0b80 msgr2=0x7fe94c0745f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:58.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.227+0000 7fe95144f640 1 --2- 172.21.15.32:0/190152320 >> v2:172.21.15.121:3300/0 conn(0x7fe94c0b0b80 0x7fe94c0745f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:58.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.227+0000 7fe95144f640 1 -- 172.21.15.32:0/190152320 >> v1:172.21.15.121:6789/0 conn(0x7fe94c0ae140 legacy=0x7fe94c0b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:58.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.227+0000 7fe95144f640 1 -- 172.21.15.32:0/190152320 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe94c0b1010 con 0x7fe94c074210 2024-04-24T23:03:58.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.228+0000 7fe95144f640 1 --2- 172.21.15.32:0/190152320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe94c074210 0x7fe94c06c480 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fe94c0722d0 tx=0x7fe93c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da0705399db7a0e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:58.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.228+0000 7fe94bfff640 1 -- 172.21.15.32:0/190152320 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93c036070 con 0x7fe94c074210 2024-04-24T23:03:58.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.228+0000 7fe94bfff640 1 -- 172.21.15.32:0/190152320 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe93c031600 con 0x7fe94c074210 2024-04-24T23:03:58.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.228+0000 7fe94bfff640 1 -- 172.21.15.32:0/190152320 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93c0319d0 con 0x7fe94c074210 2024-04-24T23:03:58.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.229+0000 7fe9536b3640 1 -- 172.21.15.32:0/190152320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe94c074210 msgr2=0x7fe94c06c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:58.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.229+0000 7fe9536b3640 1 --2- 172.21.15.32:0/190152320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe94c074210 0x7fe94c06c480 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fe94c0722d0 tx=0x7fe93c02a6f0 comp rx=0 tx=0).stop 2024-04-24T23:03:58.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.229+0000 7fe9536b3640 1 -- 172.21.15.32:0/190152320 shutdown_connections 2024-04-24T23:03:58.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.229+0000 7fe9536b3640 1 --2- 172.21.15.32:0/190152320 >> v2:172.21.15.121:3300/0 conn(0x7fe94c0b0b80 0x7fe94c0745f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:58.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.229+0000 7fe9536b3640 1 --2- 172.21.15.32:0/190152320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe94c074210 0x7fe94c06c480 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:58.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.229+0000 7fe9536b3640 1 -- 172.21.15.32:0/190152320 >> 172.21.15.32:0/190152320 conn(0x7fe94c0b2e60 msgr2=0x7fe94c0b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:58.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.229+0000 7fe9536b3640 1 -- 172.21.15.32:0/190152320 shutdown_connections 2024-04-24T23:03:58.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.229+0000 7fe9536b3640 1 -- 172.21.15.32:0/190152320 wait complete. 2024-04-24T23:03:58.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.230+0000 7fe9536b3640 1 Processor -- start 2024-04-24T23:03:58.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.230+0000 7fe9536b3640 1 -- start start 2024-04-24T23:03:58.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.231+0000 7fe9536b3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe94c074210 0x7fe94c1f4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:58.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.231+0000 7fe9536b3640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe94c0ae140 0x7fe94c0ca590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:58.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.231+0000 7fe9536b3640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe94c0caad0 0x7fe94c0cc330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:58.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.231+0000 7fe9536b3640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe94c0b18b0 con 0x7fe94c074210 2024-04-24T23:03:58.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.231+0000 7fe9536b3640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe94c0b15d0 con 0x7fe94c0ae140 2024-04-24T23:03:58.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.231+0000 7fe9536b3640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe94c0b3ba0 con 0x7fe94c0caad0 2024-04-24T23:03:58.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.231+0000 7fe95144f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe94c074210 0x7fe94c1f4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:58.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.231+0000 7fe95144f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe94c074210 0x7fe94c1f4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37116/0 (socket says 172.21.15.32:37116) 2024-04-24T23:03:58.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.231+0000 7fe95144f640 1 -- 172.21.15.32:0/567210004 learned_addr learned my addr 172.21.15.32:0/567210004 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:58.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.231+0000 7fe95144f640 1 -- 172.21.15.32:0/567210004 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe94c0caad0 msgr2=0x7fe94c0cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:58.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.231+0000 7fe950c4e640 1 --2- 172.21.15.32:0/567210004 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe94c0ae140 0x7fe94c0ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:58.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.231+0000 7fe95144f640 1 --2- 172.21.15.32:0/567210004 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe94c0caad0 0x7fe94c0cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:58.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.231+0000 7fe95144f640 1 -- 172.21.15.32:0/567210004 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe94c0ae140 msgr2=0x7fe94c0ca590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:58.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.231+0000 7fe95144f640 1 --2- 172.21.15.32:0/567210004 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe94c0ae140 0x7fe94c0ca590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:58.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.231+0000 7fe95144f640 1 -- 172.21.15.32:0/567210004 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe94c0ccf80 con 0x7fe94c074210 2024-04-24T23:03:58.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.231+0000 7fe950c4e640 1 --2- 172.21.15.32:0/567210004 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe94c0ae140 0x7fe94c0ca590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:03:58.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.231+0000 7fe95144f640 1 --2- 172.21.15.32:0/567210004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe94c074210 0x7fe94c1f4420 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fe93c034040 tx=0x7fe93c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:58.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.232+0000 7fe949ffb640 1 -- 172.21.15.32:0/567210004 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93c041020 con 0x7fe94c074210 2024-04-24T23:03:58.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.232+0000 7fe949ffb640 1 -- 172.21.15.32:0/567210004 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe93c007800 con 0x7fe94c074210 2024-04-24T23:03:58.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.232+0000 7fe9536b3640 1 -- 172.21.15.32:0/567210004 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe94c0cd260 con 0x7fe94c074210 2024-04-24T23:03:58.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.232+0000 7fe949ffb640 1 -- 172.21.15.32:0/567210004 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93c036040 con 0x7fe94c074210 2024-04-24T23:03:58.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.232+0000 7fe9536b3640 1 -- 172.21.15.32:0/567210004 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe94c0c51e0 con 0x7fe94c074210 2024-04-24T23:03:58.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.233+0000 7fe9536b3640 1 -- 172.21.15.32:0/567210004 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe94c0caf20 con 0x7fe94c074210 2024-04-24T23:03:58.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.233+0000 7fe949ffb640 1 -- 172.21.15.32:0/567210004 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe93c044020 con 0x7fe94c074210 2024-04-24T23:03:58.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.234+0000 7fe949ffb640 1 --2- 172.21.15.32:0/567210004 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe93806be60 0x7fe93806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:58.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.234+0000 7fe949ffb640 1 -- 172.21.15.32:0/567210004 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 4857+0+0 (secure 0 0 0) 0x7fe93c0ab800 con 0x7fe94c074210 2024-04-24T23:03:58.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.234+0000 7fe950c4e640 1 --2- 172.21.15.32:0/567210004 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe93806be60 0x7fe93806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:58.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.234+0000 7fe950c4e640 1 --2- 172.21.15.32:0/567210004 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe93806be60 0x7fe93806e320 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe940006fd0 tx=0x7fe940008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:58.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.238+0000 7fe949ffb640 1 -- 172.21.15.32:0/567210004 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe93c04b070 con 0x7fe94c074210 2024-04-24T23:03:58.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.423+0000 7fe9536b3640 1 -- 172.21.15.32:0/567210004 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe94c068430 con 0x7fe94c074210 2024-04-24T23:03:58.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.423+0000 7fe949ffb640 1 -- 172.21.15.32:0/567210004 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) ==== 93+0+10 (secure 0 0 0) 0x7fe93c07a530 con 0x7fe94c074210 2024-04-24T23:03:58.425 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T23:03:58.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.425+0000 7fe9536b3640 1 -- 172.21.15.32:0/567210004 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe93806be60 msgr2=0x7fe93806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:58.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.425+0000 7fe9536b3640 1 --2- 172.21.15.32:0/567210004 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe93806be60 0x7fe93806e320 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe940006fd0 tx=0x7fe940008040 comp rx=0 tx=0).stop 2024-04-24T23:03:58.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.425+0000 7fe9536b3640 1 -- 172.21.15.32:0/567210004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe94c074210 msgr2=0x7fe94c1f4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:58.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.425+0000 7fe9536b3640 1 --2- 172.21.15.32:0/567210004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe94c074210 0x7fe94c1f4420 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fe93c034040 tx=0x7fe93c004a40 comp rx=0 tx=0).stop 2024-04-24T23:03:58.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.426+0000 7fe9536b3640 1 -- 172.21.15.32:0/567210004 shutdown_connections 2024-04-24T23:03:58.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.426+0000 7fe9536b3640 1 --2- 172.21.15.32:0/567210004 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe93806be60 0x7fe93806e320 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:58.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.426+0000 7fe9536b3640 1 --2- 172.21.15.32:0/567210004 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe94c0caad0 0x7fe94c0cc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:58.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.426+0000 7fe9536b3640 1 --2- 172.21.15.32:0/567210004 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe94c0ae140 0x7fe94c0ca590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:58.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.426+0000 7fe9536b3640 1 --2- 172.21.15.32:0/567210004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe94c074210 0x7fe94c1f4420 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:58.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.426+0000 7fe9536b3640 1 -- 172.21.15.32:0/567210004 >> 172.21.15.32:0/567210004 conn(0x7fe94c0b2e60 msgr2=0x7fe94c06ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:58.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.426+0000 7fe9536b3640 1 -- 172.21.15.32:0/567210004 shutdown_connections 2024-04-24T23:03:58.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.426+0000 7fe9536b3640 1 -- 172.21.15.32:0/567210004 wait complete. 2024-04-24T23:03:58.438 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-24T23:03:58.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.539+0000 7f7d73476640 1 Processor -- start 2024-04-24T23:03:58.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.539+0000 7f7d73476640 1 -- start start 2024-04-24T23:03:58.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.540+0000 7f7d73476640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d6c06c480 0x7f7d6c0b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:58.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.540+0000 7f7d73476640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d6c0b0770 con 0x7f7d6c0b0b80 2024-04-24T23:03:58.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.540+0000 7f7d73476640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d6c0b1010 con 0x7f7d6c074210 2024-04-24T23:03:58.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.540+0000 7f7d73476640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f7d6c0b1c20 con 0x7f7d6c06c480 2024-04-24T23:03:58.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.540+0000 7f7d70a11640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7d6c0b0b80 0x7f7d6c0ae140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60528/0 (socket says 172.21.15.32:60528) 2024-04-24T23:03:58.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.540+0000 7f7d70a11640 1 -- 172.21.15.32:0/3582332366 learned_addr learned my addr 172.21.15.32:0/3582332366 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:03:58.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.540+0000 7f7d71a13640 1 --2- 172.21.15.32:0/3582332366 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d6c06c480 0x7f7d6c0b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:58.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.540+0000 7f7d6bfff640 1 -- 172.21.15.32:0/3582332366 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3367217962 0 0) 0x7f7d6c0b0770 con 0x7f7d6c0b0b80 2024-04-24T23:03:58.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.541+0000 7f7d6bfff640 1 -- 172.21.15.32:0/3582332366 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d54003520 con 0x7f7d6c0b0b80 2024-04-24T23:03:58.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.541+0000 7f7d6bfff640 1 -- 172.21.15.32:0/3582332366 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2935749900 0 0) 0x7f7d6c0b1010 con 0x7f7d6c074210 2024-04-24T23:03:58.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.541+0000 7f7d6bfff640 1 -- 172.21.15.32:0/3582332366 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d6c0b0770 con 0x7f7d6c074210 2024-04-24T23:03:58.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.541+0000 7f7d6bfff640 1 -- 172.21.15.32:0/3582332366 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2308583812 0 0) 0x7f7d54003520 con 0x7f7d6c0b0b80 2024-04-24T23:03:58.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.541+0000 7f7d6bfff640 1 -- 172.21.15.32:0/3582332366 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d6c06c480 msgr2=0x7f7d6c0b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:58.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.541+0000 7f7d6bfff640 1 --2- 172.21.15.32:0/3582332366 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d6c06c480 0x7f7d6c0b20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:58.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.541+0000 7f7d6bfff640 1 -- 172.21.15.32:0/3582332366 >> v1:172.21.15.121:6789/0 conn(0x7f7d6c074210 legacy=0x7f7d6c0745f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:58.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.541+0000 7f7d6bfff640 1 -- 172.21.15.32:0/3582332366 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d6c0b12f0 con 0x7f7d6c0b0b80 2024-04-24T23:03:58.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.541+0000 7f7d6bfff640 1 -- 172.21.15.32:0/3582332366 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f7d60001670 con 0x7f7d6c0b0b80 2024-04-24T23:03:58.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.541+0000 7f7d6bfff640 1 -- 172.21.15.32:0/3582332366 >> v1:172.21.15.32:6789/0 conn(0x7f7d6c0b0b80 legacy=0x7f7d6c0ae140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:58.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.541+0000 7f7d6bfff640 1 --2- 172.21.15.32:0/3582332366 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d6c06c480 0x7f7d54003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:58.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.541+0000 7f7d6bfff640 1 --2- 172.21.15.32:0/3582332366 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d540040f0 0x7f7d54004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:58.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.541+0000 7f7d6bfff640 1 --2- 172.21.15.32:0/3582332366 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d54004d40 0x7f7d540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:58.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.541+0000 7f7d6bfff640 1 -- 172.21.15.32:0/3582332366 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6c0b1c20 con 0x7f7d54004d40 2024-04-24T23:03:58.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.541+0000 7f7d6bfff640 1 -- 172.21.15.32:0/3582332366 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5400baa0 con 0x7f7d540040f0 2024-04-24T23:03:58.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.541+0000 7f7d6bfff640 1 -- 172.21.15.32:0/3582332366 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f7d5400bca0 con 0x7f7d6c06c480 2024-04-24T23:03:58.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.541+0000 7f7d71a13640 1 --2- 172.21.15.32:0/3582332366 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d54004d40 0x7f7d540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:58.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.541+0000 7f7d70a11640 1 --2- 172.21.15.32:0/3582332366 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d540040f0 0x7f7d54004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:58.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.541+0000 7f7d71212640 1 --2- 172.21.15.32:0/3582332366 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d6c06c480 0x7f7d54003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:58.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.541+0000 7f7d71a13640 1 -- 172.21.15.32:0/3582332366 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d6c06c480 msgr2=0x7f7d54003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:58.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.541+0000 7f7d71a13640 1 --2- 172.21.15.32:0/3582332366 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d6c06c480 0x7f7d54003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:58.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.541+0000 7f7d71a13640 1 -- 172.21.15.32:0/3582332366 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d540040f0 msgr2=0x7f7d54004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:58.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.541+0000 7f7d71a13640 1 --2- 172.21.15.32:0/3582332366 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d540040f0 0x7f7d54004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:58.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.541+0000 7f7d71a13640 1 -- 172.21.15.32:0/3582332366 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d54003520 con 0x7f7d54004d40 2024-04-24T23:03:58.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.542+0000 7f7d71a13640 1 --2- 172.21.15.32:0/3582332366 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d54004d40 0x7f7d540051c0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f7d64009ac0 tx=0x7f7d54005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5982d553d335736 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:58.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.542+0000 7f7d6bfff640 1 -- 172.21.15.32:0/3582332366 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d64036070 con 0x7f7d54004d40 2024-04-24T23:03:58.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.542+0000 7f7d6bfff640 1 -- 172.21.15.32:0/3582332366 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d64031400 con 0x7f7d54004d40 2024-04-24T23:03:58.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.542+0000 7f7d6bfff640 1 -- 172.21.15.32:0/3582332366 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d64009690 con 0x7f7d54004d40 2024-04-24T23:03:58.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.542+0000 7f7d73476640 1 -- 172.21.15.32:0/3582332366 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d54004d40 msgr2=0x7f7d540051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:58.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.542+0000 7f7d73476640 1 --2- 172.21.15.32:0/3582332366 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d54004d40 0x7f7d540051c0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f7d64009ac0 tx=0x7f7d54005830 comp rx=0 tx=0).stop 2024-04-24T23:03:58.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.543+0000 7f7d71a13640 1 -- 172.21.15.32:0/3582332366 reap_dead start 2024-04-24T23:03:58.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.543+0000 7f7d73476640 1 -- 172.21.15.32:0/3582332366 shutdown_connections 2024-04-24T23:03:58.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.543+0000 7f7d73476640 1 -- 172.21.15.32:0/3582332366 >> 172.21.15.32:0/3582332366 conn(0x7f7d6c0b2e60 msgr2=0x7f7d6c0b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:58.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.543+0000 7f7d73476640 1 -- 172.21.15.32:0/3582332366 shutdown_connections 2024-04-24T23:03:58.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.543+0000 7f7d73476640 1 -- 172.21.15.32:0/3582332366 wait complete. 2024-04-24T23:03:58.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.544+0000 7f7d73476640 1 Processor -- start 2024-04-24T23:03:58.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.544+0000 7f7d73476640 1 -- start start 2024-04-24T23:03:58.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.545+0000 7f7d73476640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d54004d40 0x7f7d6c162500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:58.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.545+0000 7f7d73476640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d6c09e990 0x7f7d6c0c77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:58.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.545+0000 7f7d73476640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d6c09c980 0x7f7d6c09ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:58.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.545+0000 7f7d73476640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6c0ae850 con 0x7f7d6c09c980 2024-04-24T23:03:58.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.545+0000 7f7d73476640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6c0b18b0 con 0x7f7d6c09e990 2024-04-24T23:03:58.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.545+0000 7f7d73476640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f7d6c0b3ba0 con 0x7f7d54004d40 2024-04-24T23:03:58.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.545+0000 7f7d71a13640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d6c09c980 0x7f7d6c09ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:58.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.545+0000 7f7d71a13640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d6c09c980 0x7f7d6c09ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37140/0 (socket says 172.21.15.32:37140) 2024-04-24T23:03:58.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.545+0000 7f7d71a13640 1 -- 172.21.15.32:0/3433191577 learned_addr learned my addr 172.21.15.32:0/3433191577 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:58.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.545+0000 7f7d71212640 1 --2- 172.21.15.32:0/3433191577 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d54004d40 0x7f7d6c162500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:58.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.545+0000 7f7d70a11640 1 --2- 172.21.15.32:0/3433191577 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d6c09e990 0x7f7d6c0c77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:58.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.545+0000 7f7d71a13640 1 -- 172.21.15.32:0/3433191577 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d54004d40 msgr2=0x7f7d6c162500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:58.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.545+0000 7f7d71a13640 1 --2- 172.21.15.32:0/3433191577 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d54004d40 0x7f7d6c162500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:58.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.545+0000 7f7d71a13640 1 -- 172.21.15.32:0/3433191577 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d6c09e990 msgr2=0x7f7d6c0c77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:58.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.545+0000 7f7d71a13640 1 --2- 172.21.15.32:0/3433191577 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d6c09e990 0x7f7d6c0c77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:58.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.545+0000 7f7d71a13640 1 -- 172.21.15.32:0/3433191577 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d6c09f230 con 0x7f7d6c09c980 2024-04-24T23:03:58.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.545+0000 7f7d71a13640 1 --2- 172.21.15.32:0/3433191577 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d6c09c980 0x7f7d6c09ce00 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f7d64009be0 tx=0x7f7d64004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:58.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.545+0000 7f7d69ffb640 1 -- 172.21.15.32:0/3433191577 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d64036070 con 0x7f7d6c09c980 2024-04-24T23:03:58.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.545+0000 7f7d69ffb640 1 -- 172.21.15.32:0/3433191577 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d64041070 con 0x7f7d6c09c980 2024-04-24T23:03:58.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.545+0000 7f7d69ffb640 1 -- 172.21.15.32:0/3433191577 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d64009690 con 0x7f7d6c09c980 2024-04-24T23:03:58.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.545+0000 7f7d73476640 1 -- 172.21.15.32:0/3433191577 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d6c09ef50 con 0x7f7d6c09c980 2024-04-24T23:03:58.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.546+0000 7f7d73476640 1 -- 172.21.15.32:0/3433191577 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d6c14f830 con 0x7f7d6c09c980 2024-04-24T23:03:58.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.547+0000 7f7d73476640 1 -- 172.21.15.32:0/3433191577 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d6c0691d0 con 0x7f7d6c09c980 2024-04-24T23:03:58.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.547+0000 7f7d69ffb640 1 -- 172.21.15.32:0/3433191577 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7d6403b840 con 0x7f7d6c09c980 2024-04-24T23:03:58.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.547+0000 7f7d69ffb640 1 --2- 172.21.15.32:0/3433191577 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7d5806be30 0x7f7d5806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:58.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.548+0000 7f7d69ffb640 1 -- 172.21.15.32:0/3433191577 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 4857+0+0 (secure 0 0 0) 0x7f7d640aa6b0 con 0x7f7d6c09c980 2024-04-24T23:03:58.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.548+0000 7f7d71212640 1 --2- 172.21.15.32:0/3433191577 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7d5806be30 0x7f7d5806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:58.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.548+0000 7f7d71212640 1 --2- 172.21.15.32:0/3433191577 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7d5806be30 0x7f7d5806e2f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c003f30 tx=0x7f7d5c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:58.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.552+0000 7f7d69ffb640 1 -- 172.21.15.32:0/3433191577 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7d6c0691d0 con 0x7f7d6c09c980 2024-04-24T23:03:58.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.735+0000 7f7d73476640 1 -- 172.21.15.32:0/3433191577 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f7d6c068430 con 0x7f7d6c09c980 2024-04-24T23:03:58.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.735+0000 7f7d69ffb640 1 -- 172.21.15.32:0/3433191577 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v100) ==== 84+0+65 (secure 0 0 0) 0x7f7d6404a020 con 0x7f7d6c09c980 2024-04-24T23:03:58.737 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:03:58.737 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2024-04-24T23:03:58.737 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:03:58.737 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2024-04-24T23:03:58.737 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": ".mgr" 2024-04-24T23:03:58.737 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:03:58.737 INFO:teuthology.orchestra.run.smithi032.stdout:] 2024-04-24T23:03:58.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.737+0000 7f7d73476640 1 -- 172.21.15.32:0/3433191577 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7d5806be30 msgr2=0x7f7d5806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:58.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.737+0000 7f7d73476640 1 --2- 172.21.15.32:0/3433191577 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7d5806be30 0x7f7d5806e2f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c003f30 tx=0x7f7d5c009040 comp rx=0 tx=0).stop 2024-04-24T23:03:58.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.737+0000 7f7d73476640 1 -- 172.21.15.32:0/3433191577 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d6c09c980 msgr2=0x7f7d6c09ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:58.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.737+0000 7f7d73476640 1 --2- 172.21.15.32:0/3433191577 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d6c09c980 0x7f7d6c09ce00 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f7d64009be0 tx=0x7f7d64004730 comp rx=0 tx=0).stop 2024-04-24T23:03:58.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.737+0000 7f7d73476640 1 -- 172.21.15.32:0/3433191577 shutdown_connections 2024-04-24T23:03:58.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.737+0000 7f7d73476640 1 --2- 172.21.15.32:0/3433191577 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7d5806be30 0x7f7d5806e2f0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:58.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.737+0000 7f7d73476640 1 --2- 172.21.15.32:0/3433191577 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d6c09c980 0x7f7d6c09ce00 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:58.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.737+0000 7f7d73476640 1 --2- 172.21.15.32:0/3433191577 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d6c09e990 0x7f7d6c0c77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:58.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.737+0000 7f7d73476640 1 --2- 172.21.15.32:0/3433191577 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d54004d40 0x7f7d6c162500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:58.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.737+0000 7f7d73476640 1 -- 172.21.15.32:0/3433191577 >> 172.21.15.32:0/3433191577 conn(0x7f7d6c0b2e60 msgr2=0x7f7d6c06ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:58.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.737+0000 7f7d73476640 1 -- 172.21.15.32:0/3433191577 shutdown_connections 2024-04-24T23:03:58.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.737+0000 7f7d73476640 1 -- 172.21.15.32:0/3433191577 wait complete. 2024-04-24T23:03:58.749 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-24T23:03:58.749 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-24T23:03:58.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.853+0000 7fc9b58bd640 1 Processor -- start 2024-04-24T23:03:58.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.853+0000 7fc9b58bd640 1 -- start start 2024-04-24T23:03:58.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.854+0000 7fc9b58bd640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9b015e9a0 0x7fc9b00cf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:58.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.854+0000 7fc9b58bd640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9b015f1c0 con 0x7fc9b00cfd90 2024-04-24T23:03:58.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.854+0000 7fc9b58bd640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9b015f4a0 con 0x7fc9b00d0d60 2024-04-24T23:03:58.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.854+0000 7fc9b58bd640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc9b015faf0 con 0x7fc9b015e9a0 2024-04-24T23:03:58.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.854+0000 7fc9a7fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc9b00cfd90 0x7fc9b00d0650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60542/0 (socket says 172.21.15.32:60542) 2024-04-24T23:03:58.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.854+0000 7fc9a7fff640 1 -- 172.21.15.32:0/2107428677 learned_addr learned my addr 172.21.15.32:0/2107428677 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:03:58.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.854+0000 7fc9aeffd640 1 --2- 172.21.15.32:0/2107428677 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9b015e9a0 0x7fc9b00cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:58.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.854+0000 7fc9ae7fc640 1 -- 172.21.15.32:0/2107428677 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2255019213 0 0) 0x7fc9b015f1c0 con 0x7fc9b00cfd90 2024-04-24T23:03:58.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.854+0000 7fc9ae7fc640 1 -- 172.21.15.32:0/2107428677 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc99c003520 con 0x7fc9b00cfd90 2024-04-24T23:03:58.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.854+0000 7fc9ae7fc640 1 -- 172.21.15.32:0/2107428677 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2347262237 0 0) 0x7fc99c003520 con 0x7fc9b00cfd90 2024-04-24T23:03:58.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.854+0000 7fc9ae7fc640 1 -- 172.21.15.32:0/2107428677 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9b015e9a0 msgr2=0x7fc9b00cf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:58.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.854+0000 7fc9ae7fc640 1 --2- 172.21.15.32:0/2107428677 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9b015e9a0 0x7fc9b00cf850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:58.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.854+0000 7fc9ae7fc640 1 -- 172.21.15.32:0/2107428677 >> v1:172.21.15.121:6789/0 conn(0x7fc9b00d0d60 legacy=0x7fc9b0070710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:58.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.854+0000 7fc9ae7fc640 1 -- 172.21.15.32:0/2107428677 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9b015f780 con 0x7fc9b00cfd90 2024-04-24T23:03:58.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.854+0000 7fc9ae7fc640 1 -- 172.21.15.32:0/2107428677 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7fc998001670 con 0x7fc9b00cfd90 2024-04-24T23:03:58.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.854+0000 7fc9ae7fc640 1 -- 172.21.15.32:0/2107428677 >> v1:172.21.15.32:6789/0 conn(0x7fc9b00cfd90 legacy=0x7fc9b00d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:58.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.854+0000 7fc9ae7fc640 1 --2- 172.21.15.32:0/2107428677 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc99c0039e0 0x7fc99c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:58.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.854+0000 7fc9ae7fc640 1 --2- 172.21.15.32:0/2107428677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc99c004390 0x7fc99c004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:58.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.855+0000 7fc9ae7fc640 1 --2- 172.21.15.32:0/2107428677 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9b015e9a0 0x7fc99c005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:58.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.855+0000 7fc9ae7fc640 1 -- 172.21.15.32:0/2107428677 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b015faf0 con 0x7fc99c004390 2024-04-24T23:03:58.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.855+0000 7fc9ae7fc640 1 -- 172.21.15.32:0/2107428677 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c00ba70 con 0x7fc99c0039e0 2024-04-24T23:03:58.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.855+0000 7fc9ae7fc640 1 -- 172.21.15.32:0/2107428677 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc99c00bc70 con 0x7fc9b015e9a0 2024-04-24T23:03:58.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.855+0000 7fc9a7fff640 1 --2- 172.21.15.32:0/2107428677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc99c004390 0x7fc99c004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:58.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.855+0000 7fc9aeffd640 1 --2- 172.21.15.32:0/2107428677 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc99c0039e0 0x7fc99c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:58.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.855+0000 7fc9af7fe640 1 --2- 172.21.15.32:0/2107428677 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9b015e9a0 0x7fc99c005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:58.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.855+0000 7fc9a7fff640 1 -- 172.21.15.32:0/2107428677 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9b015e9a0 msgr2=0x7fc99c005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:58.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.855+0000 7fc9a7fff640 1 --2- 172.21.15.32:0/2107428677 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9b015e9a0 0x7fc99c005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:58.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.855+0000 7fc9a7fff640 1 -- 172.21.15.32:0/2107428677 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc99c0039e0 msgr2=0x7fc99c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:58.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.855+0000 7fc9a7fff640 1 --2- 172.21.15.32:0/2107428677 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc99c0039e0 0x7fc99c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:58.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.855+0000 7fc9a7fff640 1 -- 172.21.15.32:0/2107428677 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc99c003520 con 0x7fc99c004390 2024-04-24T23:03:58.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.855+0000 7fc9af7fe640 1 --2- 172.21.15.32:0/2107428677 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9b015e9a0 0x7fc99c005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:03:58.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.856+0000 7fc9a7fff640 1 --2- 172.21.15.32:0/2107428677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc99c004390 0x7fc99c004860 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fc99800c240 tx=0x7fc99c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=846b05988acefaf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:58.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.856+0000 7fc9ae7fc640 1 -- 172.21.15.32:0/2107428677 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc998007d10 con 0x7fc99c004390 2024-04-24T23:03:58.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.856+0000 7fc9ae7fc640 1 -- 172.21.15.32:0/2107428677 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9980058d0 con 0x7fc99c004390 2024-04-24T23:03:58.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.856+0000 7fc9ae7fc640 1 -- 172.21.15.32:0/2107428677 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc998005ce0 con 0x7fc99c004390 2024-04-24T23:03:58.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.856+0000 7fc9b58bd640 1 -- 172.21.15.32:0/2107428677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc99c004390 msgr2=0x7fc99c004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:58.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.856+0000 7fc9b58bd640 1 --2- 172.21.15.32:0/2107428677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc99c004390 0x7fc99c004860 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fc99800c240 tx=0x7fc99c005830 comp rx=0 tx=0).stop 2024-04-24T23:03:58.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.856+0000 7fc9af7fe640 1 -- 172.21.15.32:0/2107428677 reap_dead start 2024-04-24T23:03:58.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.856+0000 7fc9b58bd640 1 -- 172.21.15.32:0/2107428677 shutdown_connections 2024-04-24T23:03:58.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.856+0000 7fc9b58bd640 1 -- 172.21.15.32:0/2107428677 >> 172.21.15.32:0/2107428677 conn(0x7fc9b006b110 msgr2=0x7fc9b006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:58.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.856+0000 7fc9b58bd640 1 -- 172.21.15.32:0/2107428677 shutdown_connections 2024-04-24T23:03:58.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.856+0000 7fc9b58bd640 1 -- 172.21.15.32:0/2107428677 wait complete. 2024-04-24T23:03:58.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.857+0000 7fc9b58bd640 1 Processor -- start 2024-04-24T23:03:58.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.858+0000 7fc9b58bd640 1 -- start start 2024-04-24T23:03:58.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.858+0000 7fc9b58bd640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc99c004390 0x7fc9b0158020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:58.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.858+0000 7fc9b58bd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9b0150260 0x7fc9b01506c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:58.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.858+0000 7fc9b58bd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9b0151400 0x7fc9b0151880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:58.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.858+0000 7fc9b58bd640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b015ef00 con 0x7fc9b0150260 2024-04-24T23:03:58.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.858+0000 7fc9b58bd640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b015ed80 con 0x7fc9b0151400 2024-04-24T23:03:58.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.858+0000 7fc9b58bd640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc9b00cf850 con 0x7fc99c004390 2024-04-24T23:03:58.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.858+0000 7fc9a7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9b0150260 0x7fc9b01506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:58.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.858+0000 7fc9a7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9b0150260 0x7fc9b01506c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37160/0 (socket says 172.21.15.32:37160) 2024-04-24T23:03:58.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.858+0000 7fc9a7fff640 1 -- 172.21.15.32:0/4148045897 learned_addr learned my addr 172.21.15.32:0/4148045897 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:58.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.859+0000 7fc9af7fe640 1 --2- 172.21.15.32:0/4148045897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9b0151400 0x7fc9b0151880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:58.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.859+0000 7fc9aeffd640 1 --2- 172.21.15.32:0/4148045897 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc99c004390 0x7fc9b0158020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:58.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.859+0000 7fc9a7fff640 1 -- 172.21.15.32:0/4148045897 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc99c004390 msgr2=0x7fc9b0158020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:58.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.859+0000 7fc9a7fff640 1 --2- 172.21.15.32:0/4148045897 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc99c004390 0x7fc9b0158020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:58.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.859+0000 7fc9a7fff640 1 -- 172.21.15.32:0/4148045897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9b0151400 msgr2=0x7fc9b0151880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:58.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.859+0000 7fc9a7fff640 1 --2- 172.21.15.32:0/4148045897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9b0151400 0x7fc9b0151880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:58.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.859+0000 7fc9a7fff640 1 -- 172.21.15.32:0/4148045897 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9b01537e0 con 0x7fc9b0150260 2024-04-24T23:03:58.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.859+0000 7fc9af7fe640 1 --2- 172.21.15.32:0/4148045897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9b0151400 0x7fc9b0151880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:03:58.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.859+0000 7fc9aeffd640 1 --2- 172.21.15.32:0/4148045897 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc99c004390 0x7fc9b0158020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:03:58.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.859+0000 7fc9a7fff640 1 --2- 172.21.15.32:0/4148045897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9b0150260 0x7fc9b01506c0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fc99800c360 tx=0x7fc9b015f390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:58.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.859+0000 7fc9acff9640 1 -- 172.21.15.32:0/4148045897 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc998038040 con 0x7fc9b0150260 2024-04-24T23:03:58.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.859+0000 7fc9acff9640 1 -- 172.21.15.32:0/4148045897 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc998042070 con 0x7fc9b0150260 2024-04-24T23:03:58.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.859+0000 7fc9b58bd640 1 -- 172.21.15.32:0/4148045897 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9b0150ce0 con 0x7fc9b0150260 2024-04-24T23:03:58.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.859+0000 7fc9acff9640 1 -- 172.21.15.32:0/4148045897 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc998001c50 con 0x7fc9b0150260 2024-04-24T23:03:58.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.859+0000 7fc9b58bd640 1 -- 172.21.15.32:0/4148045897 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9b0153d60 con 0x7fc9b0150260 2024-04-24T23:03:58.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.860+0000 7fc9b58bd640 1 -- 172.21.15.32:0/4148045897 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9b0068430 con 0x7fc9b0150260 2024-04-24T23:03:58.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.860+0000 7fc9acff9640 1 -- 172.21.15.32:0/4148045897 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc998007290 con 0x7fc9b0150260 2024-04-24T23:03:58.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.861+0000 7fc9acff9640 1 --2- 172.21.15.32:0/4148045897 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc99006be60 0x7fc99006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:58.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.861+0000 7fc9acff9640 1 -- 172.21.15.32:0/4148045897 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 4857+0+0 (secure 0 0 0) 0x7fc9980aaa30 con 0x7fc9b0150260 2024-04-24T23:03:58.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.861+0000 7fc9aeffd640 1 --2- 172.21.15.32:0/4148045897 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc99006be60 0x7fc99006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:58.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.861+0000 7fc9aeffd640 1 --2- 172.21.15.32:0/4148045897 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc99006be60 0x7fc99006e320 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0009ab0 tx=0x7fc9a0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:58.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:58.865+0000 7fc9acff9640 1 -- 172.21.15.32:0/4148045897 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc998079850 con 0x7fc9b0150260 2024-04-24T23:03:59.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.047+0000 7fc9b58bd640 1 -- 172.21.15.32:0/4148045897 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fc9b00ce8a0 con 0x7fc9b0150260 2024-04-24T23:03:59.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.409+0000 7fc9acff9640 1 -- 172.21.15.32:0/4148045897 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v101) ==== 138+0+0 (secure 0 0 0) 0x7fc998079150 con 0x7fc9b0150260 2024-04-24T23:03:59.411 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2024-04-24T23:03:59.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.412+0000 7fc9b58bd640 1 -- 172.21.15.32:0/4148045897 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc99006be60 msgr2=0x7fc99006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:59.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.412+0000 7fc9b58bd640 1 --2- 172.21.15.32:0/4148045897 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc99006be60 0x7fc99006e320 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0009ab0 tx=0x7fc9a0008040 comp rx=0 tx=0).stop 2024-04-24T23:03:59.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.412+0000 7fc9b58bd640 1 -- 172.21.15.32:0/4148045897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9b0150260 msgr2=0x7fc9b01506c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:59.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.412+0000 7fc9b58bd640 1 --2- 172.21.15.32:0/4148045897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9b0150260 0x7fc9b01506c0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fc99800c360 tx=0x7fc9b015f390 comp rx=0 tx=0).stop 2024-04-24T23:03:59.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.414+0000 7fc9b58bd640 1 -- 172.21.15.32:0/4148045897 shutdown_connections 2024-04-24T23:03:59.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.414+0000 7fc9b58bd640 1 --2- 172.21.15.32:0/4148045897 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc99006be60 0x7fc99006e320 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:59.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.414+0000 7fc9b58bd640 1 --2- 172.21.15.32:0/4148045897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9b0150260 0x7fc9b01506c0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:59.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.414+0000 7fc9b58bd640 1 --2- 172.21.15.32:0/4148045897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9b0151400 0x7fc9b0151880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:59.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.414+0000 7fc9b58bd640 1 --2- 172.21.15.32:0/4148045897 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc99c004390 0x7fc9b0158020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:59.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.414+0000 7fc9b58bd640 1 -- 172.21.15.32:0/4148045897 >> 172.21.15.32:0/4148045897 conn(0x7fc9b006b110 msgr2=0x7fc9b00d0650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:59.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.414+0000 7fc9b58bd640 1 -- 172.21.15.32:0/4148045897 shutdown_connections 2024-04-24T23:03:59.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.414+0000 7fc9b58bd640 1 -- 172.21.15.32:0/4148045897 wait complete. 2024-04-24T23:03:59.428 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-24T23:03:59.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.535+0000 7f133702a640 1 Processor -- start 2024-04-24T23:03:59.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.535+0000 7f133702a640 1 -- start start 2024-04-24T23:03:59.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.535+0000 7f133702a640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f13301476d0 0x7f13300ce630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:59.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.535+0000 7f133702a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f133006b820 0x7f133006d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:59.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.535+0000 7f133702a640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13300afdd0 con 0x7f13300cfb70 2024-04-24T23:03:59.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.535+0000 7f133702a640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f13300aedc0 con 0x7f13301476d0 2024-04-24T23:03:59.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.535+0000 7f133702a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f13300af940 con 0x7f133006b820 2024-04-24T23:03:59.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.536+0000 7f132ffff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f13300cfb70 0x7f133006b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60548/0 (socket says 172.21.15.32:60548) 2024-04-24T23:03:59.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.536+0000 7f132ffff640 1 -- 172.21.15.32:0/3753030865 learned_addr learned my addr 172.21.15.32:0/3753030865 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:03:59.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.536+0000 7f13355c7640 1 --2- 172.21.15.32:0/3753030865 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f133006b820 0x7f133006d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:59.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.536+0000 7f1334dc6640 1 --2- 172.21.15.32:0/3753030865 >> v2:172.21.15.121:3300/0 conn(0x7f13301476d0 0x7f13300ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:59.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.536+0000 7f132f7fe640 1 -- 172.21.15.32:0/3753030865 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 994584637 0 0) 0x7f13300afdd0 con 0x7f13300cfb70 2024-04-24T23:03:59.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.536+0000 7f132f7fe640 1 -- 172.21.15.32:0/3753030865 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1320003520 con 0x7f13300cfb70 2024-04-24T23:03:59.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.536+0000 7f1334dc6640 1 -- 172.21.15.32:0/3753030865 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f133006b820 msgr2=0x7f133006d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:03:59.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.536+0000 7f1334dc6640 1 --2- 172.21.15.32:0/3753030865 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f133006b820 0x7f133006d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:59.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.536+0000 7f1334dc6640 1 -- 172.21.15.32:0/3753030865 >> v1:172.21.15.32:6789/0 conn(0x7f13300cfb70 legacy=0x7f133006b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:59.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.536+0000 7f1334dc6640 1 -- 172.21.15.32:0/3753030865 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13300af010 con 0x7f13301476d0 2024-04-24T23:03:59.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.536+0000 7f13355c7640 1 --2- 172.21.15.32:0/3753030865 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f133006b820 0x7f133006d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:03:59.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.537+0000 7f1334dc6640 1 --2- 172.21.15.32:0/3753030865 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13301476d0 0x7f13300ce630 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f13240097a0 tx=0x7f132402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a44eb1ebc3cda64d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:59.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.537+0000 7f132f7fe640 1 -- 172.21.15.32:0/3753030865 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1324037070 con 0x7f13301476d0 2024-04-24T23:03:59.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.537+0000 7f132f7fe640 1 -- 172.21.15.32:0/3753030865 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f132402ae50 con 0x7f13301476d0 2024-04-24T23:03:59.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.537+0000 7f132f7fe640 1 -- 172.21.15.32:0/3753030865 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1324032660 con 0x7f13301476d0 2024-04-24T23:03:59.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.538+0000 7f133702a640 1 -- 172.21.15.32:0/3753030865 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13301476d0 msgr2=0x7f13300ce630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:59.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.538+0000 7f133702a640 1 --2- 172.21.15.32:0/3753030865 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13301476d0 0x7f13300ce630 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f13240097a0 tx=0x7f132402a100 comp rx=0 tx=0).stop 2024-04-24T23:03:59.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.538+0000 7f133702a640 1 -- 172.21.15.32:0/3753030865 shutdown_connections 2024-04-24T23:03:59.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.538+0000 7f133702a640 1 --2- 172.21.15.32:0/3753030865 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f133006b820 0x7f133006d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:59.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.538+0000 7f133702a640 1 --2- 172.21.15.32:0/3753030865 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13301476d0 0x7f13300ce630 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:59.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.538+0000 7f133702a640 1 -- 172.21.15.32:0/3753030865 >> 172.21.15.32:0/3753030865 conn(0x7f13300d0730 msgr2=0x7f13300d0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:59.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.538+0000 7f133702a640 1 -- 172.21.15.32:0/3753030865 shutdown_connections 2024-04-24T23:03:59.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.538+0000 7f133702a640 1 -- 172.21.15.32:0/3753030865 wait complete. 2024-04-24T23:03:59.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.539+0000 7f133702a640 1 Processor -- start 2024-04-24T23:03:59.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.539+0000 7f133702a640 1 -- start start 2024-04-24T23:03:59.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.540+0000 7f133702a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f133006b820 0x7f13301f7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:59.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.540+0000 7f133702a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13300cfb70 0x7f1330141100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:59.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.540+0000 7f133702a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f13301476d0 0x7f13301f4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:59.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.540+0000 7f133702a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1330147ab0 con 0x7f13300cfb70 2024-04-24T23:03:59.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.540+0000 7f133702a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f13300af5d0 con 0x7f133006b820 2024-04-24T23:03:59.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.540+0000 7f133702a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f1330148470 con 0x7f13301476d0 2024-04-24T23:03:59.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.540+0000 7f132ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13300cfb70 0x7f1330141100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:59.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.540+0000 7f132ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13300cfb70 0x7f1330141100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37170/0 (socket says 172.21.15.32:37170) 2024-04-24T23:03:59.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.540+0000 7f132ffff640 1 -- 172.21.15.32:0/2887029800 learned_addr learned my addr 172.21.15.32:0/2887029800 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:03:59.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.540+0000 7f13355c7640 1 --2- 172.21.15.32:0/2887029800 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f13301476d0 0x7f13301f4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:59.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.540+0000 7f1334dc6640 1 --2- 172.21.15.32:0/2887029800 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f133006b820 0x7f13301f7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:59.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.540+0000 7f132ffff640 1 -- 172.21.15.32:0/2887029800 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f13301476d0 msgr2=0x7f13301f4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:59.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.540+0000 7f132ffff640 1 --2- 172.21.15.32:0/2887029800 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f13301476d0 0x7f13301f4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:59.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.540+0000 7f132ffff640 1 -- 172.21.15.32:0/2887029800 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f133006b820 msgr2=0x7f13301f7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:59.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.540+0000 7f132ffff640 1 --2- 172.21.15.32:0/2887029800 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f133006b820 0x7f13301f7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:59.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.540+0000 7f132ffff640 1 -- 172.21.15.32:0/2887029800 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13301f8290 con 0x7f13300cfb70 2024-04-24T23:03:59.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.541+0000 7f132ffff640 1 --2- 172.21.15.32:0/2887029800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13300cfb70 0x7f1330141100 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f131c0010e0 tx=0x7f131c00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:59.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.541+0000 7f132d7fa640 1 -- 172.21.15.32:0/2887029800 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131c017070 con 0x7f13300cfb70 2024-04-24T23:03:59.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.541+0000 7f132d7fa640 1 -- 172.21.15.32:0/2887029800 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f131c007270 con 0x7f13300cfb70 2024-04-24T23:03:59.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.541+0000 7f133702a640 1 -- 172.21.15.32:0/2887029800 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13301f8570 con 0x7f13300cfb70 2024-04-24T23:03:59.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.541+0000 7f132d7fa640 1 -- 172.21.15.32:0/2887029800 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131c001d40 con 0x7f13300cfb70 2024-04-24T23:03:59.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.541+0000 7f133702a640 1 -- 172.21.15.32:0/2887029800 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f133015f580 con 0x7f13300cfb70 2024-04-24T23:03:59.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.542+0000 7f133702a640 1 -- 172.21.15.32:0/2887029800 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1330068430 con 0x7f13300cfb70 2024-04-24T23:03:59.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.542+0000 7f132d7fa640 1 -- 172.21.15.32:0/2887029800 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f131c01a070 con 0x7f13300cfb70 2024-04-24T23:03:59.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.543+0000 7f132d7fa640 1 --2- 172.21.15.32:0/2887029800 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f131406be30 0x7f131406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:59.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.543+0000 7f132d7fa640 1 -- 172.21.15.32:0/2887029800 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5227+0+0 (secure 0 0 0) 0x7f131c089fe0 con 0x7f13300cfb70 2024-04-24T23:03:59.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.543+0000 7f1334dc6640 1 --2- 172.21.15.32:0/2887029800 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f131406be30 0x7f131406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:59.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.543+0000 7f1334dc6640 1 --2- 172.21.15.32:0/2887029800 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f131406be30 0x7f131406e2f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1324034000 tx=0x7f132402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:59.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.547+0000 7f132d7fa640 1 -- 172.21.15.32:0/2887029800 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f131c058460 con 0x7f13300cfb70 2024-04-24T23:03:59.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:03:59.733+0000 7f133702a640 1 -- 172.21.15.32:0/2887029800 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f1330071450 con 0x7f13300cfb70 2024-04-24T23:04:00.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.412+0000 7f132d7fa640 1 -- 172.21.15.32:0/2887029800 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v102) ==== 170+0+0 (secure 0 0 0) 0x7f131c05c760 con 0x7f13300cfb70 2024-04-24T23:04:00.414 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2024-04-24T23:04:00.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.416+0000 7f133702a640 1 -- 172.21.15.32:0/2887029800 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f131406be30 msgr2=0x7f131406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:00.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.416+0000 7f133702a640 1 --2- 172.21.15.32:0/2887029800 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f131406be30 0x7f131406e2f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1324034000 tx=0x7f132402e000 comp rx=0 tx=0).stop 2024-04-24T23:04:00.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.416+0000 7f133702a640 1 -- 172.21.15.32:0/2887029800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13300cfb70 msgr2=0x7f1330141100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:00.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.416+0000 7f133702a640 1 --2- 172.21.15.32:0/2887029800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13300cfb70 0x7f1330141100 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f131c0010e0 tx=0x7f131c00cb90 comp rx=0 tx=0).stop 2024-04-24T23:04:00.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.416+0000 7f133702a640 1 -- 172.21.15.32:0/2887029800 shutdown_connections 2024-04-24T23:04:00.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.416+0000 7f133702a640 1 --2- 172.21.15.32:0/2887029800 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f131406be30 0x7f131406e2f0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:00.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.416+0000 7f133702a640 1 --2- 172.21.15.32:0/2887029800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13300cfb70 0x7f1330141100 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:00.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.416+0000 7f133702a640 1 --2- 172.21.15.32:0/2887029800 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f13301476d0 0x7f13301f4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:00.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.416+0000 7f133702a640 1 --2- 172.21.15.32:0/2887029800 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f133006b820 0x7f13301f7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:00.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.416+0000 7f133702a640 1 -- 172.21.15.32:0/2887029800 >> 172.21.15.32:0/2887029800 conn(0x7f13300d0730 msgr2=0x7f13300cef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:00.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.419+0000 7f133702a640 1 -- 172.21.15.32:0/2887029800 shutdown_connections 2024-04-24T23:04:00.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.419+0000 7f133702a640 1 -- 172.21.15.32:0/2887029800 wait complete. 2024-04-24T23:04:00.440 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-24T23:04:00.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.547+0000 7f58dca8c640 1 Processor -- start 2024-04-24T23:04:00.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.547+0000 7f58dca8c640 1 -- start start 2024-04-24T23:04:00.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.548+0000 7f58dca8c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f58d80d05b0 0x7f58d81641e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:00.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.548+0000 7f58dca8c640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f58d815d730 0x7f58d8166730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:00.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.548+0000 7f58dca8c640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58d80b4ee0 con 0x7f58d815cc40 2024-04-24T23:04:00.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.548+0000 7f58dca8c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f58d806cca0 con 0x7f58d80d05b0 2024-04-24T23:04:00.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.548+0000 7f58dca8c640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f58d80ce5f0 con 0x7f58d815d730 2024-04-24T23:04:00.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.548+0000 7f58d5d9b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f58d80d05b0 0x7f58d81641e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:00.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.548+0000 7f58d5d9b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f58d80d05b0 0x7f58d81641e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60104/0 (socket says 172.21.15.32:60104) 2024-04-24T23:04:00.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.548+0000 7f58d5d9b640 1 -- 172.21.15.32:0/1450823724 learned_addr learned my addr 172.21.15.32:0/1450823724 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:00.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.548+0000 7f58d6d9d640 1 --2- 172.21.15.32:0/1450823724 >> v2:172.21.15.121:3300/0 conn(0x7f58d815d730 0x7f58d8166730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:00.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.548+0000 7f58d5d9b640 1 -- 172.21.15.32:0/1450823724 >> v2:172.21.15.121:3300/0 conn(0x7f58d815d730 msgr2=0x7f58d8166730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:00.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.548+0000 7f58d5d9b640 1 --2- 172.21.15.32:0/1450823724 >> v2:172.21.15.121:3300/0 conn(0x7f58d815d730 0x7f58d8166730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:00.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.548+0000 7f58d5d9b640 1 -- 172.21.15.32:0/1450823724 >> v1:172.21.15.121:6789/0 conn(0x7f58d815cc40 legacy=0x7f58d815d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:00.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.548+0000 7f58d5d9b640 1 -- 172.21.15.32:0/1450823724 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58d80b51c0 con 0x7f58d80d05b0 2024-04-24T23:04:00.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.549+0000 7f58d5d9b640 1 --2- 172.21.15.32:0/1450823724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58d80d05b0 0x7f58d81641e0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f58d80b3aa0 tx=0x7f58cc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f3522797601ee59 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:00.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.549+0000 7f58d559a640 1 -- 172.21.15.32:0/1450823724 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58cc037070 con 0x7f58d80d05b0 2024-04-24T23:04:00.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.549+0000 7f58d559a640 1 -- 172.21.15.32:0/1450823724 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58cc032440 con 0x7f58d80d05b0 2024-04-24T23:04:00.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.549+0000 7f58d559a640 1 -- 172.21.15.32:0/1450823724 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58cc032810 con 0x7f58d80d05b0 2024-04-24T23:04:00.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.550+0000 7f58dca8c640 1 -- 172.21.15.32:0/1450823724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58d80d05b0 msgr2=0x7f58d81641e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:00.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.550+0000 7f58dca8c640 1 --2- 172.21.15.32:0/1450823724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58d80d05b0 0x7f58d81641e0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f58d80b3aa0 tx=0x7f58cc02a270 comp rx=0 tx=0).stop 2024-04-24T23:04:00.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.550+0000 7f58dca8c640 1 -- 172.21.15.32:0/1450823724 shutdown_connections 2024-04-24T23:04:00.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.550+0000 7f58dca8c640 1 --2- 172.21.15.32:0/1450823724 >> v2:172.21.15.121:3300/0 conn(0x7f58d815d730 0x7f58d8166730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:00.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.550+0000 7f58dca8c640 1 --2- 172.21.15.32:0/1450823724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58d80d05b0 0x7f58d81641e0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:00.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.550+0000 7f58dca8c640 1 -- 172.21.15.32:0/1450823724 >> 172.21.15.32:0/1450823724 conn(0x7f58d80d0ae0 msgr2=0x7f58d80d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:00.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.550+0000 7f58dca8c640 1 -- 172.21.15.32:0/1450823724 shutdown_connections 2024-04-24T23:04:00.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.550+0000 7f58dca8c640 1 -- 172.21.15.32:0/1450823724 wait complete. 2024-04-24T23:04:00.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.551+0000 7f58dca8c640 1 Processor -- start 2024-04-24T23:04:00.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.551+0000 7f58dca8c640 1 -- start start 2024-04-24T23:04:00.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.552+0000 7f58dca8c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58d80d05b0 0x7f58d8152610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:00.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.552+0000 7f58dca8c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58d815cc40 0x7f58d8144fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:00.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.552+0000 7f58dca8c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f58d815d730 0x7f58d81454f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:00.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.552+0000 7f58dca8c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f58d80b5a60 con 0x7f58d815cc40 2024-04-24T23:04:00.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.552+0000 7f58dca8c640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f58d80b5780 con 0x7f58d80d05b0 2024-04-24T23:04:00.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.552+0000 7f58dca8c640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f58d8067940 con 0x7f58d815d730 2024-04-24T23:04:00.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.552+0000 7f58d5d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58d815cc40 0x7f58d8144fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:00.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.552+0000 7f58d5d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58d815cc40 0x7f58d8144fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60106/0 (socket says 172.21.15.32:60106) 2024-04-24T23:04:00.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.552+0000 7f58d5d9b640 1 -- 172.21.15.32:0/1906076494 learned_addr learned my addr 172.21.15.32:0/1906076494 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:00.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.552+0000 7f58d6d9d640 1 --2- 172.21.15.32:0/1906076494 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f58d815d730 0x7f58d81454f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:00.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.552+0000 7f58d659c640 1 --2- 172.21.15.32:0/1906076494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58d80d05b0 0x7f58d8152610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:00.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.553+0000 7f58d5d9b640 1 -- 172.21.15.32:0/1906076494 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f58d815d730 msgr2=0x7f58d81454f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:00.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.553+0000 7f58d5d9b640 1 --2- 172.21.15.32:0/1906076494 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f58d815d730 0x7f58d81454f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:00.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.553+0000 7f58d5d9b640 1 -- 172.21.15.32:0/1906076494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58d80d05b0 msgr2=0x7f58d8152610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:00.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.553+0000 7f58d5d9b640 1 --2- 172.21.15.32:0/1906076494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58d80d05b0 0x7f58d8152610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:00.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.553+0000 7f58d5d9b640 1 -- 172.21.15.32:0/1906076494 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58d8070c60 con 0x7f58d815cc40 2024-04-24T23:04:00.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.553+0000 7f58d6d9d640 1 --2- 172.21.15.32:0/1906076494 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f58d815d730 0x7f58d81454f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:04:00.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.553+0000 7f58d659c640 1 --2- 172.21.15.32:0/1906076494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58d80d05b0 0x7f58d8152610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:04:00.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.553+0000 7f58d5d9b640 1 --2- 172.21.15.32:0/1906076494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58d815cc40 0x7f58d8144fb0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f58cc02a2b0 tx=0x7f58cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:00.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.553+0000 7f58c6ffd640 1 -- 172.21.15.32:0/1906076494 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58cc037040 con 0x7f58d815cc40 2024-04-24T23:04:00.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.553+0000 7f58c6ffd640 1 -- 172.21.15.32:0/1906076494 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58cc042070 con 0x7f58d815cc40 2024-04-24T23:04:00.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.553+0000 7f58dca8c640 1 -- 172.21.15.32:0/1906076494 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58d8070f40 con 0x7f58d815cc40 2024-04-24T23:04:00.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.553+0000 7f58c6ffd640 1 -- 172.21.15.32:0/1906076494 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58cc030880 con 0x7f58d815cc40 2024-04-24T23:04:00.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.553+0000 7f58dca8c640 1 -- 172.21.15.32:0/1906076494 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58d8071500 con 0x7f58d815cc40 2024-04-24T23:04:00.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.554+0000 7f58dca8c640 1 -- 172.21.15.32:0/1906076494 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58d8068430 con 0x7f58d815cc40 2024-04-24T23:04:00.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.554+0000 7f58c6ffd640 1 -- 172.21.15.32:0/1906076494 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f58cc030a20 con 0x7f58d815cc40 2024-04-24T23:04:00.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.555+0000 7f58c6ffd640 1 --2- 172.21.15.32:0/1906076494 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f58b806be30 0x7f58b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:00.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.555+0000 7f58c6ffd640 1 -- 172.21.15.32:0/1906076494 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5609+0+0 (secure 0 0 0) 0x7f58cc0ab620 con 0x7f58d815cc40 2024-04-24T23:04:00.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.555+0000 7f58d659c640 1 --2- 172.21.15.32:0/1906076494 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f58b806be30 0x7f58b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:00.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.556+0000 7f58d659c640 1 --2- 172.21.15.32:0/1906076494 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f58b806be30 0x7f58b806e2f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f58d0001660 tx=0x7f58d0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:00.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.559+0000 7f58c6ffd640 1 -- 172.21.15.32:0/1906076494 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f58cc07a0c0 con 0x7f58d815cc40 2024-04-24T23:04:00.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:00.781+0000 7f58dca8c640 1 -- 172.21.15.32:0/1906076494 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f58d80d1600 con 0x7f58d815cc40 2024-04-24T23:04:01.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.438+0000 7f58c6ffd640 1 -- 172.21.15.32:0/1906076494 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v41) ==== 174+0+0 (secure 0 0 0) 0x7f58cc0799c0 con 0x7f58d815cc40 2024-04-24T23:04:01.444 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 12 and data pool 13 2024-04-24T23:04:01.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.447+0000 7f58dca8c640 1 -- 172.21.15.32:0/1906076494 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f58b806be30 msgr2=0x7f58b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:01.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.447+0000 7f58dca8c640 1 --2- 172.21.15.32:0/1906076494 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f58b806be30 0x7f58b806e2f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f58d0001660 tx=0x7f58d0009040 comp rx=0 tx=0).stop 2024-04-24T23:04:01.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.447+0000 7f58dca8c640 1 -- 172.21.15.32:0/1906076494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58d815cc40 msgr2=0x7f58d8144fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:01.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.447+0000 7f58dca8c640 1 --2- 172.21.15.32:0/1906076494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58d815cc40 0x7f58d8144fb0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f58cc02a2b0 tx=0x7f58cc0046c0 comp rx=0 tx=0).stop 2024-04-24T23:04:01.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.448+0000 7f58dca8c640 1 -- 172.21.15.32:0/1906076494 shutdown_connections 2024-04-24T23:04:01.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.448+0000 7f58dca8c640 1 --2- 172.21.15.32:0/1906076494 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f58b806be30 0x7f58b806e2f0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:01.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.448+0000 7f58dca8c640 1 --2- 172.21.15.32:0/1906076494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58d815cc40 0x7f58d8144fb0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:01.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.448+0000 7f58dca8c640 1 --2- 172.21.15.32:0/1906076494 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f58d815d730 0x7f58d81454f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:01.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.448+0000 7f58dca8c640 1 --2- 172.21.15.32:0/1906076494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58d80d05b0 0x7f58d8152610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:01.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.448+0000 7f58dca8c640 1 -- 172.21.15.32:0/1906076494 >> 172.21.15.32:0/1906076494 conn(0x7f58d80d0ae0 msgr2=0x7f58d80b1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:01.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.448+0000 7f58dca8c640 1 -- 172.21.15.32:0/1906076494 shutdown_connections 2024-04-24T23:04:01.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.448+0000 7f58dca8c640 1 -- 172.21.15.32:0/1906076494 wait complete. 2024-04-24T23:04:01.466 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:04:01.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.569+0000 7f9d8997e640 1 Processor -- start 2024-04-24T23:04:01.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.569+0000 7f9d8997e640 1 -- start start 2024-04-24T23:04:01.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.570+0000 7f9d8997e640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9d8406c5f0 0x7f9d8406c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:01.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.570+0000 7f9d8997e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9d840b44d0 0x7f9d840b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:01.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.570+0000 7f9d8997e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d840b6430 con 0x7f9d840af890 2024-04-24T23:04:01.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.570+0000 7f9d8997e640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f9d84156f80 con 0x7f9d8406c5f0 2024-04-24T23:04:01.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.570+0000 7f9d8997e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f9d840b6150 con 0x7f9d840b44d0 2024-04-24T23:04:01.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.570+0000 7f9d837fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9d840af890 0x7f9d840cef70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42580/0 (socket says 172.21.15.32:42580) 2024-04-24T23:04:01.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.570+0000 7f9d837fe640 1 -- 172.21.15.32:0/2159125919 learned_addr learned my addr 172.21.15.32:0/2159125919 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:04:01.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.570+0000 7f9d81ffb640 1 -- 172.21.15.32:0/2159125919 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1585479198 0 0) 0x7f9d840b6430 con 0x7f9d840af890 2024-04-24T23:04:01.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.570+0000 7f9d82ffd640 1 --2- 172.21.15.32:0/2159125919 >> v2:172.21.15.121:3300/0 conn(0x7f9d8406c5f0 0x7f9d8406c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:01.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.570+0000 7f9d81ffb640 1 -- 172.21.15.32:0/2159125919 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d70003520 con 0x7f9d840af890 2024-04-24T23:04:01.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.570+0000 7f9d827fc640 1 --2- 172.21.15.32:0/2159125919 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9d840b44d0 0x7f9d840b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:01.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.571+0000 7f9d81ffb640 1 -- 172.21.15.32:0/2159125919 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3854166056 0 0) 0x7f9d70003520 con 0x7f9d840af890 2024-04-24T23:04:01.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.571+0000 7f9d81ffb640 1 -- 172.21.15.32:0/2159125919 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9d840b44d0 msgr2=0x7f9d840b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:01.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.571+0000 7f9d81ffb640 1 --2- 172.21.15.32:0/2159125919 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9d840b44d0 0x7f9d840b5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:01.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.571+0000 7f9d81ffb640 1 -- 172.21.15.32:0/2159125919 >> v2:172.21.15.121:3300/0 conn(0x7f9d8406c5f0 msgr2=0x7f9d8406c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:01.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.571+0000 7f9d81ffb640 1 --2- 172.21.15.32:0/2159125919 >> v2:172.21.15.121:3300/0 conn(0x7f9d8406c5f0 0x7f9d8406c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:01.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.571+0000 7f9d81ffb640 1 -- 172.21.15.32:0/2159125919 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d840b39f0 con 0x7f9d840af890 2024-04-24T23:04:01.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.571+0000 7f9d81ffb640 1 -- 172.21.15.32:0/2159125919 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f9d740019f0 con 0x7f9d840af890 2024-04-24T23:04:01.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.571+0000 7f9d81ffb640 1 -- 172.21.15.32:0/2159125919 >> v1:172.21.15.32:6789/0 conn(0x7f9d840af890 legacy=0x7f9d840cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:01.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.571+0000 7f9d81ffb640 1 --2- 172.21.15.32:0/2159125919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d700039b0 0x7f9d70003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:01.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.571+0000 7f9d81ffb640 1 --2- 172.21.15.32:0/2159125919 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9d70004800 0x7f9d70004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:01.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.571+0000 7f9d81ffb640 1 --2- 172.21.15.32:0/2159125919 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d700051c0 0x7f9d7000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:01.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.571+0000 7f9d81ffb640 1 -- 172.21.15.32:0/2159125919 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9d84156f80 con 0x7f9d700039b0 2024-04-24T23:04:01.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.571+0000 7f9d81ffb640 1 -- 172.21.15.32:0/2159125919 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9d840b6150 con 0x7f9d700051c0 2024-04-24T23:04:01.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.571+0000 7f9d81ffb640 1 -- 172.21.15.32:0/2159125919 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f9d7000bfe0 con 0x7f9d70004800 2024-04-24T23:04:01.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.571+0000 7f9d827fc640 1 --2- 172.21.15.32:0/2159125919 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9d840b44d0 0x7f9d840b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:04:01.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.571+0000 7f9d82ffd640 1 --2- 172.21.15.32:0/2159125919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d700039b0 0x7f9d70003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:01.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.571+0000 7f9d837fe640 1 --2- 172.21.15.32:0/2159125919 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d700051c0 0x7f9d7000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:01.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.571+0000 7f9d827fc640 1 --2- 172.21.15.32:0/2159125919 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9d70004800 0x7f9d70004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:01.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.571+0000 7f9d82ffd640 1 -- 172.21.15.32:0/2159125919 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9d70004800 msgr2=0x7f9d70004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:01.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.571+0000 7f9d82ffd640 1 --2- 172.21.15.32:0/2159125919 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9d70004800 0x7f9d70004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:01.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.571+0000 7f9d82ffd640 1 -- 172.21.15.32:0/2159125919 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d700051c0 msgr2=0x7f9d7000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:01.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.571+0000 7f9d82ffd640 1 --2- 172.21.15.32:0/2159125919 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d700051c0 0x7f9d7000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:01.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.571+0000 7f9d82ffd640 1 -- 172.21.15.32:0/2159125919 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d70003520 con 0x7f9d700039b0 2024-04-24T23:04:01.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.571+0000 7f9d827fc640 1 --2- 172.21.15.32:0/2159125919 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9d70004800 0x7f9d70004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:04:01.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.572+0000 7f9d82ffd640 1 --2- 172.21.15.32:0/2159125919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d700039b0 0x7f9d70003dd0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c009ac0 tx=0x7f9d70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39397f19f6a9c983 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:01.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.572+0000 7f9d81ffb640 1 -- 172.21.15.32:0/2159125919 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c036070 con 0x7f9d700039b0 2024-04-24T23:04:01.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.572+0000 7f9d81ffb640 1 -- 172.21.15.32:0/2159125919 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d7c031400 con 0x7f9d700039b0 2024-04-24T23:04:01.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.572+0000 7f9d81ffb640 1 -- 172.21.15.32:0/2159125919 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c009690 con 0x7f9d700039b0 2024-04-24T23:04:01.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.572+0000 7f9d8997e640 1 -- 172.21.15.32:0/2159125919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d700039b0 msgr2=0x7f9d70003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:01.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.572+0000 7f9d8997e640 1 --2- 172.21.15.32:0/2159125919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d700039b0 0x7f9d70003dd0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c009ac0 tx=0x7f9d70005830 comp rx=0 tx=0).stop 2024-04-24T23:04:01.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.572+0000 7f9d837fe640 1 -- 172.21.15.32:0/2159125919 reap_dead start 2024-04-24T23:04:01.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.572+0000 7f9d8997e640 1 -- 172.21.15.32:0/2159125919 shutdown_connections 2024-04-24T23:04:01.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.572+0000 7f9d8997e640 1 -- 172.21.15.32:0/2159125919 >> 172.21.15.32:0/2159125919 conn(0x7f9d8406b110 msgr2=0x7f9d8406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:01.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.573+0000 7f9d8997e640 1 -- 172.21.15.32:0/2159125919 shutdown_connections 2024-04-24T23:04:01.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.573+0000 7f9d8997e640 1 -- 172.21.15.32:0/2159125919 wait complete. 2024-04-24T23:04:01.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.574+0000 7f9d8997e640 1 Processor -- start 2024-04-24T23:04:01.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.574+0000 7f9d8997e640 1 -- start start 2024-04-24T23:04:01.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.575+0000 7f9d8997e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d700039b0 0x7f9d8415f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:01.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.575+0000 7f9d8997e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9d8415b9a0 0x7f9d84158c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:01.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.575+0000 7f9d8997e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d8415be20 0x7f9d8414bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:01.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.575+0000 7f9d8997e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8406cf10 con 0x7f9d700039b0 2024-04-24T23:04:01.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.575+0000 7f9d8997e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9d840b3fb0 con 0x7f9d8415be20 2024-04-24T23:04:01.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.575+0000 7f9d8997e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f9d840b5890 con 0x7f9d8415b9a0 2024-04-24T23:04:01.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.575+0000 7f9d82ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d700039b0 0x7f9d8415f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:01.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.575+0000 7f9d82ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d700039b0 0x7f9d8415f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60126/0 (socket says 172.21.15.32:60126) 2024-04-24T23:04:01.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.575+0000 7f9d82ffd640 1 -- 172.21.15.32:0/326439602 learned_addr learned my addr 172.21.15.32:0/326439602 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:01.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.575+0000 7f9d827fc640 1 --2- 172.21.15.32:0/326439602 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9d8415b9a0 0x7f9d84158c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:01.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.575+0000 7f9d837fe640 1 --2- 172.21.15.32:0/326439602 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d8415be20 0x7f9d8414bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:01.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.575+0000 7f9d82ffd640 1 -- 172.21.15.32:0/326439602 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9d8415b9a0 msgr2=0x7f9d84158c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:01.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.575+0000 7f9d82ffd640 1 --2- 172.21.15.32:0/326439602 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9d8415b9a0 0x7f9d84158c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:01.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.575+0000 7f9d82ffd640 1 -- 172.21.15.32:0/326439602 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d8415be20 msgr2=0x7f9d8414bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:01.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.575+0000 7f9d82ffd640 1 --2- 172.21.15.32:0/326439602 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d8415be20 0x7f9d8414bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:01.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.575+0000 7f9d82ffd640 1 -- 172.21.15.32:0/326439602 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d84157c10 con 0x7f9d700039b0 2024-04-24T23:04:01.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.575+0000 7f9d82ffd640 1 --2- 172.21.15.32:0/326439602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d700039b0 0x7f9d8415f800 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c034040 tx=0x7f9d7c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:01.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.575+0000 7f9d7bfff640 1 -- 172.21.15.32:0/326439602 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c040020 con 0x7f9d700039b0 2024-04-24T23:04:01.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.575+0000 7f9d7bfff640 1 -- 172.21.15.32:0/326439602 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d7c003950 con 0x7f9d700039b0 2024-04-24T23:04:01.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.575+0000 7f9d8997e640 1 -- 172.21.15.32:0/326439602 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d84157f10 con 0x7f9d700039b0 2024-04-24T23:04:01.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.575+0000 7f9d7bfff640 1 -- 172.21.15.32:0/326439602 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c036040 con 0x7f9d700039b0 2024-04-24T23:04:01.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.575+0000 7f9d8997e640 1 -- 172.21.15.32:0/326439602 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d84158750 con 0x7f9d700039b0 2024-04-24T23:04:01.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.576+0000 7f9d8997e640 1 -- 172.21.15.32:0/326439602 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d840b5c60 con 0x7f9d700039b0 2024-04-24T23:04:01.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.577+0000 7f9d7bfff640 1 -- 172.21.15.32:0/326439602 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9d7c043020 con 0x7f9d700039b0 2024-04-24T23:04:01.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.577+0000 7f9d7bfff640 1 --2- 172.21.15.32:0/326439602 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9d6406be60 0x7f9d6406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:01.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.577+0000 7f9d7bfff640 1 -- 172.21.15.32:0/326439602 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f9d7c0aaeb0 con 0x7f9d700039b0 2024-04-24T23:04:01.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.578+0000 7f9d827fc640 1 --2- 172.21.15.32:0/326439602 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9d6406be60 0x7f9d6406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:01.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.578+0000 7f9d827fc640 1 --2- 172.21.15.32:0/326439602 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9d6406be60 0x7f9d6406e320 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9d84150130 tx=0x7f9d6c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:01.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.582+0000 7f9d7bfff640 1 -- 172.21.15.32:0/326439602 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9d840b5c60 con 0x7f9d700039b0 2024-04-24T23:04:01.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.765+0000 7f9d8997e640 1 -- 172.21.15.32:0/326439602 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9d84068430 con 0x7f9d700039b0 2024-04-24T23:04:01.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.767+0000 7f9d7bfff640 1 -- 172.21.15.32:0/326439602 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+16708 (secure 0 0 0) 0x7f9d7c04a070 con 0x7f9d700039b0 2024-04-24T23:04:01.768 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:04:01.768 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":103,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:04:00.783201+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:03:59.079157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-24T23:03:59.765179+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:04:01.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.769+0000 7f9d8997e640 1 -- 172.21.15.32:0/326439602 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9d6406be60 msgr2=0x7f9d6406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:01.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.769+0000 7f9d8997e640 1 --2- 172.21.15.32:0/326439602 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9d6406be60 0x7f9d6406e320 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9d84150130 tx=0x7f9d6c008040 comp rx=0 tx=0).stop 2024-04-24T23:04:01.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.769+0000 7f9d8997e640 1 -- 172.21.15.32:0/326439602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d700039b0 msgr2=0x7f9d8415f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:01.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.769+0000 7f9d8997e640 1 --2- 172.21.15.32:0/326439602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d700039b0 0x7f9d8415f800 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c034040 tx=0x7f9d7c004730 comp rx=0 tx=0).stop 2024-04-24T23:04:01.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.769+0000 7f9d8997e640 1 -- 172.21.15.32:0/326439602 shutdown_connections 2024-04-24T23:04:01.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.769+0000 7f9d8997e640 1 --2- 172.21.15.32:0/326439602 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9d6406be60 0x7f9d6406e320 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:01.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.769+0000 7f9d8997e640 1 --2- 172.21.15.32:0/326439602 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d8415be20 0x7f9d8414bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:01.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.769+0000 7f9d8997e640 1 --2- 172.21.15.32:0/326439602 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9d8415b9a0 0x7f9d84158c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:01.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.769+0000 7f9d8997e640 1 --2- 172.21.15.32:0/326439602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d700039b0 0x7f9d8415f800 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:01.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.769+0000 7f9d8997e640 1 -- 172.21.15.32:0/326439602 >> 172.21.15.32:0/326439602 conn(0x7f9d8406b110 msgr2=0x7f9d84071750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:01.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.769+0000 7f9d8997e640 1 -- 172.21.15.32:0/326439602 shutdown_connections 2024-04-24T23:04:01.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.769+0000 7f9d8997e640 1 -- 172.21.15.32:0/326439602 wait complete. 2024-04-24T23:04:01.782 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:04:01.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.881+0000 7f3f895d5640 1 Processor -- start 2024-04-24T23:04:01.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.881+0000 7f3f895d5640 1 -- start start 2024-04-24T23:04:01.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.882+0000 7f3f895d5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3f8415c5d0 0x7f3f840b5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:01.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.882+0000 7f3f895d5640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3f84072e70 0x7f3f84073300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:01.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.882+0000 7f3f895d5640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f840b5cf0 con 0x7f3f840d1c10 2024-04-24T23:04:01.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.882+0000 7f3f895d5640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f3f84161160 con 0x7f3f84072e70 2024-04-24T23:04:01.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.882+0000 7f3f895d5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f3f840b5fd0 con 0x7f3f8415c5d0 2024-04-24T23:04:01.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.882+0000 7f3f827fc640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3f840d1c10 0x7f3f84160a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42596/0 (socket says 172.21.15.32:42596) 2024-04-24T23:04:01.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.882+0000 7f3f827fc640 1 -- 172.21.15.32:0/1840968058 learned_addr learned my addr 172.21.15.32:0/1840968058 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:04:01.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.883+0000 7f3f82ffd640 1 --2- 172.21.15.32:0/1840968058 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3f8415c5d0 0x7f3f840b5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:01.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.883+0000 7f3f837fe640 1 --2- 172.21.15.32:0/1840968058 >> v2:172.21.15.121:3300/0 conn(0x7f3f84072e70 0x7f3f84073300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:01.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.883+0000 7f3f81ffb640 1 -- 172.21.15.32:0/1840968058 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1525989315 0 0) 0x7f3f840b5cf0 con 0x7f3f840d1c10 2024-04-24T23:04:01.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.883+0000 7f3f81ffb640 1 -- 172.21.15.32:0/1840968058 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f70003520 con 0x7f3f840d1c10 2024-04-24T23:04:01.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.883+0000 7f3f82ffd640 1 -- 172.21.15.32:0/1840968058 >> v2:172.21.15.121:3300/0 conn(0x7f3f84072e70 msgr2=0x7f3f84073300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:01.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.883+0000 7f3f82ffd640 1 --2- 172.21.15.32:0/1840968058 >> v2:172.21.15.121:3300/0 conn(0x7f3f84072e70 0x7f3f84073300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:01.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.883+0000 7f3f82ffd640 1 -- 172.21.15.32:0/1840968058 >> v1:172.21.15.32:6789/0 conn(0x7f3f840d1c10 legacy=0x7f3f84160a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:01.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.883+0000 7f3f82ffd640 1 -- 172.21.15.32:0/1840968058 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f840b62b0 con 0x7f3f8415c5d0 2024-04-24T23:04:01.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.884+0000 7f3f82ffd640 1 --2- 172.21.15.32:0/1840968058 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3f8415c5d0 0x7f3f840b5410 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c0097a0 tx=0x7f3f7c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70bcbc69aafa2ee1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:01.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.884+0000 7f3f81ffb640 1 -- 172.21.15.32:0/1840968058 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c037070 con 0x7f3f8415c5d0 2024-04-24T23:04:01.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.884+0000 7f3f81ffb640 1 -- 172.21.15.32:0/1840968058 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f7c032440 con 0x7f3f8415c5d0 2024-04-24T23:04:01.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.884+0000 7f3f81ffb640 1 -- 172.21.15.32:0/1840968058 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c0327f0 con 0x7f3f8415c5d0 2024-04-24T23:04:01.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.884+0000 7f3f895d5640 1 -- 172.21.15.32:0/1840968058 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3f8415c5d0 msgr2=0x7f3f840b5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:01.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.884+0000 7f3f895d5640 1 --2- 172.21.15.32:0/1840968058 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3f8415c5d0 0x7f3f840b5410 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c0097a0 tx=0x7f3f7c02a180 comp rx=0 tx=0).stop 2024-04-24T23:04:01.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.884+0000 7f3f895d5640 1 -- 172.21.15.32:0/1840968058 shutdown_connections 2024-04-24T23:04:01.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.884+0000 7f3f895d5640 1 --2- 172.21.15.32:0/1840968058 >> v2:172.21.15.121:3300/0 conn(0x7f3f84072e70 0x7f3f84073300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:01.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.884+0000 7f3f895d5640 1 --2- 172.21.15.32:0/1840968058 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3f8415c5d0 0x7f3f840b5410 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:01.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.884+0000 7f3f895d5640 1 -- 172.21.15.32:0/1840968058 >> 172.21.15.32:0/1840968058 conn(0x7f3f840d24c0 msgr2=0x7f3f840d28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:01.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.884+0000 7f3f895d5640 1 -- 172.21.15.32:0/1840968058 shutdown_connections 2024-04-24T23:04:01.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.884+0000 7f3f895d5640 1 -- 172.21.15.32:0/1840968058 wait complete. 2024-04-24T23:04:01.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.886+0000 7f3f895d5640 1 Processor -- start 2024-04-24T23:04:01.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.886+0000 7f3f895d5640 1 -- start start 2024-04-24T23:04:01.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.886+0000 7f3f895d5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3f84072e70 0x7f3f8415e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:01.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.887+0000 7f3f895d5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3f840d1c10 0x7f3f84156790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:01.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.887+0000 7f3f895d5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f8415c5d0 0x7f3f841f34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:01.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.887+0000 7f3f895d5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3f840b5950 con 0x7f3f8415c5d0 2024-04-24T23:04:01.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.887+0000 7f3f895d5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8415c9b0 con 0x7f3f84072e70 2024-04-24T23:04:01.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.887+0000 7f3f895d5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f3f840b5ad0 con 0x7f3f840d1c10 2024-04-24T23:04:01.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.887+0000 7f3f837fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f8415c5d0 0x7f3f841f34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:01.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.887+0000 7f3f837fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f8415c5d0 0x7f3f841f34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60142/0 (socket says 172.21.15.32:60142) 2024-04-24T23:04:01.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.887+0000 7f3f837fe640 1 -- 172.21.15.32:0/224721227 learned_addr learned my addr 172.21.15.32:0/224721227 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:01.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.887+0000 7f3f82ffd640 1 --2- 172.21.15.32:0/224721227 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3f84072e70 0x7f3f8415e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:01.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.887+0000 7f3f827fc640 1 --2- 172.21.15.32:0/224721227 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3f840d1c10 0x7f3f84156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:01.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.887+0000 7f3f837fe640 1 -- 172.21.15.32:0/224721227 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3f840d1c10 msgr2=0x7f3f84156790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:01.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.887+0000 7f3f837fe640 1 --2- 172.21.15.32:0/224721227 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3f840d1c10 0x7f3f84156790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:01.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.887+0000 7f3f837fe640 1 -- 172.21.15.32:0/224721227 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3f84072e70 msgr2=0x7f3f8415e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:01.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.887+0000 7f3f837fe640 1 --2- 172.21.15.32:0/224721227 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3f84072e70 0x7f3f8415e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:01.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.887+0000 7f3f837fe640 1 -- 172.21.15.32:0/224721227 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f8415cf80 con 0x7f3f8415c5d0 2024-04-24T23:04:01.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.887+0000 7f3f82ffd640 1 --2- 172.21.15.32:0/224721227 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3f84072e70 0x7f3f8415e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:04:01.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.887+0000 7f3f837fe640 1 --2- 172.21.15.32:0/224721227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f8415c5d0 0x7f3f841f34c0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f3f74009dd0 tx=0x7f3f7400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:01.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.887+0000 7f3f7b7fe640 1 -- 172.21.15.32:0/224721227 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f74018020 con 0x7f3f8415c5d0 2024-04-24T23:04:01.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.887+0000 7f3f7b7fe640 1 -- 172.21.15.32:0/224721227 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f74014480 con 0x7f3f8415c5d0 2024-04-24T23:04:01.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.887+0000 7f3f895d5640 1 -- 172.21.15.32:0/224721227 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f8415d260 con 0x7f3f8415c5d0 2024-04-24T23:04:01.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.887+0000 7f3f7b7fe640 1 -- 172.21.15.32:0/224721227 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f74014870 con 0x7f3f8415c5d0 2024-04-24T23:04:01.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.887+0000 7f3f895d5640 1 -- 172.21.15.32:0/224721227 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f8415db00 con 0x7f3f8415c5d0 2024-04-24T23:04:01.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.888+0000 7f3f895d5640 1 -- 172.21.15.32:0/224721227 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f84068430 con 0x7f3f8415c5d0 2024-04-24T23:04:01.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.889+0000 7f3f7b7fe640 1 -- 172.21.15.32:0/224721227 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3f74005d10 con 0x7f3f8415c5d0 2024-04-24T23:04:01.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.889+0000 7f3f7b7fe640 1 --2- 172.21.15.32:0/224721227 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3f6406be30 0x7f3f6406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:01.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.889+0000 7f3f7b7fe640 1 -- 172.21.15.32:0/224721227 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f3f7408b900 con 0x7f3f8415c5d0 2024-04-24T23:04:01.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.889+0000 7f3f82ffd640 1 --2- 172.21.15.32:0/224721227 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3f6406be30 0x7f3f6406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:01.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.890+0000 7f3f82ffd640 1 --2- 172.21.15.32:0/224721227 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3f6406be30 0x7f3f6406e2f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c034000 tx=0x7f3f7c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:01.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:01.894+0000 7f3f7b7fe640 1 -- 172.21.15.32:0/224721227 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3f74022070 con 0x7f3f8415c5d0 2024-04-24T23:04:02.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.079+0000 7f3f895d5640 1 -- 172.21.15.32:0/224721227 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3f84072a20 con 0x7f3f8415c5d0 2024-04-24T23:04:02.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.080+0000 7f3f7b7fe640 1 -- 172.21.15.32:0/224721227 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+16708 (secure 0 0 0) 0x7f3f74010040 con 0x7f3f8415c5d0 2024-04-24T23:04:02.082 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:04:02.082 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":103,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:04:00.783201+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:03:59.079157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-24T23:03:59.765179+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:04:02.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.083+0000 7f3f895d5640 1 -- 172.21.15.32:0/224721227 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3f6406be30 msgr2=0x7f3f6406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:02.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.083+0000 7f3f895d5640 1 --2- 172.21.15.32:0/224721227 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3f6406be30 0x7f3f6406e2f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c034000 tx=0x7f3f7c02e000 comp rx=0 tx=0).stop 2024-04-24T23:04:02.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.083+0000 7f3f895d5640 1 -- 172.21.15.32:0/224721227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f8415c5d0 msgr2=0x7f3f841f34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:02.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.083+0000 7f3f895d5640 1 --2- 172.21.15.32:0/224721227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f8415c5d0 0x7f3f841f34c0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f3f74009dd0 tx=0x7f3f7400c490 comp rx=0 tx=0).stop 2024-04-24T23:04:02.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.083+0000 7f3f895d5640 1 -- 172.21.15.32:0/224721227 shutdown_connections 2024-04-24T23:04:02.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.083+0000 7f3f895d5640 1 --2- 172.21.15.32:0/224721227 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3f6406be30 0x7f3f6406e2f0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:02.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.083+0000 7f3f895d5640 1 --2- 172.21.15.32:0/224721227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f8415c5d0 0x7f3f841f34c0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:02.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.083+0000 7f3f895d5640 1 --2- 172.21.15.32:0/224721227 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3f840d1c10 0x7f3f84156790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:02.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.083+0000 7f3f895d5640 1 --2- 172.21.15.32:0/224721227 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3f84072e70 0x7f3f8415e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:02.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.083+0000 7f3f895d5640 1 -- 172.21.15.32:0/224721227 >> 172.21.15.32:0/224721227 conn(0x7f3f840d24c0 msgr2=0x7f3f8415a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:02.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.083+0000 7f3f895d5640 1 -- 172.21.15.32:0/224721227 shutdown_connections 2024-04-24T23:04:02.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.083+0000 7f3f895d5640 1 -- 172.21.15.32:0/224721227 wait complete. 2024-04-24T23:04:02.095 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-24T23:04:02.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.198+0000 7f685223f640 1 Processor -- start 2024-04-24T23:04:02.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.198+0000 7f685223f640 1 -- start start 2024-04-24T23:04:02.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.198+0000 7f685223f640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f684c15e9b0 0x7f684c0d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:02.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.198+0000 7f685223f640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f684c06b820 0x7f684c0b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:02.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.198+0000 7f685223f640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f684c0d3170 con 0x7f684c0d4210 2024-04-24T23:04:02.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.198+0000 7f685223f640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f684c0d3450 con 0x7f684c06b820 2024-04-24T23:04:02.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.198+0000 7f685223f640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f684c0d3730 con 0x7f684c15e9b0 2024-04-24T23:04:02.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.199+0000 7f684affd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f684c0d4210 0x7f684c06b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42604/0 (socket says 172.21.15.32:42604) 2024-04-24T23:04:02.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.199+0000 7f684affd640 1 -- 172.21.15.32:0/314354155 learned_addr learned my addr 172.21.15.32:0/314354155 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:04:02.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.199+0000 7f684bfff640 1 --2- 172.21.15.32:0/314354155 >> v2:172.21.15.121:3300/0 conn(0x7f684c06b820 0x7f684c0b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:02.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.199+0000 7f684a7fc640 1 -- 172.21.15.32:0/314354155 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2511934003 0 0) 0x7f684c0d3170 con 0x7f684c0d4210 2024-04-24T23:04:02.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.199+0000 7f684a7fc640 1 -- 172.21.15.32:0/314354155 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6838003520 con 0x7f684c0d4210 2024-04-24T23:04:02.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.199+0000 7f684b7fe640 1 --2- 172.21.15.32:0/314354155 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f684c15e9b0 0x7f684c0d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:02.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.199+0000 7f684a7fc640 1 -- 172.21.15.32:0/314354155 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 149949229 0 0) 0x7f6838003520 con 0x7f684c0d4210 2024-04-24T23:04:02.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.199+0000 7f684a7fc640 1 -- 172.21.15.32:0/314354155 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f684c15e9b0 msgr2=0x7f684c0d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:02.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.199+0000 7f684a7fc640 1 --2- 172.21.15.32:0/314354155 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f684c15e9b0 0x7f684c0d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:02.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.199+0000 7f684a7fc640 1 -- 172.21.15.32:0/314354155 >> v2:172.21.15.121:3300/0 conn(0x7f684c06b820 msgr2=0x7f684c0b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:02.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.199+0000 7f684a7fc640 1 --2- 172.21.15.32:0/314354155 >> v2:172.21.15.121:3300/0 conn(0x7f684c06b820 0x7f684c0b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:02.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.199+0000 7f684a7fc640 1 -- 172.21.15.32:0/314354155 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f684c0d3a10 con 0x7f684c0d4210 2024-04-24T23:04:02.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.199+0000 7f684a7fc640 1 -- 172.21.15.32:0/314354155 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f6834001670 con 0x7f684c0d4210 2024-04-24T23:04:02.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.199+0000 7f684a7fc640 1 -- 172.21.15.32:0/314354155 >> v1:172.21.15.32:6789/0 conn(0x7f684c0d4210 legacy=0x7f684c06b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:02.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.199+0000 7f684a7fc640 1 --2- 172.21.15.32:0/314354155 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68380039b0 0x7f6838003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:02.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.199+0000 7f684a7fc640 1 --2- 172.21.15.32:0/314354155 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f684c15e9b0 0x7f6838004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:02.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.199+0000 7f684a7fc640 1 --2- 172.21.15.32:0/314354155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6838004d40 0x7f68380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:02.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.199+0000 7f684a7fc640 1 -- 172.21.15.32:0/314354155 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f684c0d3450 con 0x7f68380039b0 2024-04-24T23:04:02.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.199+0000 7f684a7fc640 1 -- 172.21.15.32:0/314354155 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f684c0d3730 con 0x7f6838004d40 2024-04-24T23:04:02.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.199+0000 7f684a7fc640 1 -- 172.21.15.32:0/314354155 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f68380141d0 con 0x7f684c15e9b0 2024-04-24T23:04:02.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.199+0000 7f684b7fe640 1 --2- 172.21.15.32:0/314354155 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68380039b0 0x7f6838003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:02.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.199+0000 7f684affd640 1 --2- 172.21.15.32:0/314354155 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f684c15e9b0 0x7f6838004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:02.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.199+0000 7f684bfff640 1 --2- 172.21.15.32:0/314354155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6838004d40 0x7f68380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:02.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.200+0000 7f684b7fe640 1 -- 172.21.15.32:0/314354155 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f684c15e9b0 msgr2=0x7f6838004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:02.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.200+0000 7f684b7fe640 1 --2- 172.21.15.32:0/314354155 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f684c15e9b0 0x7f6838004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:02.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.200+0000 7f684b7fe640 1 -- 172.21.15.32:0/314354155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6838004d40 msgr2=0x7f68380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:02.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.200+0000 7f684b7fe640 1 --2- 172.21.15.32:0/314354155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6838004d40 0x7f68380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:02.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.200+0000 7f684b7fe640 1 -- 172.21.15.32:0/314354155 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6838003520 con 0x7f68380039b0 2024-04-24T23:04:02.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.200+0000 7f684b7fe640 1 --2- 172.21.15.32:0/314354155 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68380039b0 0x7f6838003dd0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f683c009ac0 tx=0x7f6838005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cc939982aa4c34c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:02.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.200+0000 7f684a7fc640 1 -- 172.21.15.32:0/314354155 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f683c036070 con 0x7f68380039b0 2024-04-24T23:04:02.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.200+0000 7f684a7fc640 1 -- 172.21.15.32:0/314354155 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f683c031400 con 0x7f68380039b0 2024-04-24T23:04:02.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.200+0000 7f684a7fc640 1 -- 172.21.15.32:0/314354155 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f683c009690 con 0x7f68380039b0 2024-04-24T23:04:02.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.201+0000 7f685223f640 1 -- 172.21.15.32:0/314354155 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68380039b0 msgr2=0x7f6838003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:02.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.201+0000 7f685223f640 1 --2- 172.21.15.32:0/314354155 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68380039b0 0x7f6838003dd0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f683c009ac0 tx=0x7f6838005830 comp rx=0 tx=0).stop 2024-04-24T23:04:02.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.201+0000 7f684bfff640 1 -- 172.21.15.32:0/314354155 reap_dead start 2024-04-24T23:04:02.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.201+0000 7f685223f640 1 -- 172.21.15.32:0/314354155 shutdown_connections 2024-04-24T23:04:02.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.201+0000 7f685223f640 1 -- 172.21.15.32:0/314354155 >> 172.21.15.32:0/314354155 conn(0x7f684c0d11d0 msgr2=0x7f684c0d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:02.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.201+0000 7f685223f640 1 -- 172.21.15.32:0/314354155 shutdown_connections 2024-04-24T23:04:02.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.201+0000 7f685223f640 1 -- 172.21.15.32:0/314354155 wait complete. 2024-04-24T23:04:02.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.202+0000 7f685223f640 1 Processor -- start 2024-04-24T23:04:02.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.202+0000 7f685223f640 1 -- start start 2024-04-24T23:04:02.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.203+0000 7f685223f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68380039b0 0x7f684c0cc380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:02.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.203+0000 7f685223f640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f684c160620 0x7f684c0ce440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:02.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.203+0000 7f685223f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f684c0ce980 0x7f684c1531b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:02.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.203+0000 7f685223f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f684c0ba0e0 con 0x7f684c0ce980 2024-04-24T23:04:02.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.203+0000 7f685223f640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f684c0d2400 con 0x7f68380039b0 2024-04-24T23:04:02.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.203+0000 7f685223f640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f684c0ba260 con 0x7f684c160620 2024-04-24T23:04:02.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.203+0000 7f684bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f684c0ce980 0x7f684c1531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:02.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.203+0000 7f684bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f684c0ce980 0x7f684c1531b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60164/0 (socket says 172.21.15.32:60164) 2024-04-24T23:04:02.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.203+0000 7f684bfff640 1 -- 172.21.15.32:0/2278290686 learned_addr learned my addr 172.21.15.32:0/2278290686 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:02.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.203+0000 7f684b7fe640 1 --2- 172.21.15.32:0/2278290686 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68380039b0 0x7f684c0cc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:02.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.203+0000 7f684affd640 1 --2- 172.21.15.32:0/2278290686 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f684c160620 0x7f684c0ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:02.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.203+0000 7f684bfff640 1 -- 172.21.15.32:0/2278290686 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f684c160620 msgr2=0x7f684c0ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:02.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.203+0000 7f684bfff640 1 --2- 172.21.15.32:0/2278290686 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f684c160620 0x7f684c0ce440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:02.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.203+0000 7f684bfff640 1 -- 172.21.15.32:0/2278290686 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68380039b0 msgr2=0x7f684c0cc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:02.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.203+0000 7f684bfff640 1 --2- 172.21.15.32:0/2278290686 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68380039b0 0x7f684c0cc380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:02.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.204+0000 7f684bfff640 1 -- 172.21.15.32:0/2278290686 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f684c15f580 con 0x7f684c0ce980 2024-04-24T23:04:02.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.204+0000 7f684b7fe640 1 --2- 172.21.15.32:0/2278290686 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68380039b0 0x7f684c0cc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:04:02.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.204+0000 7f684bfff640 1 --2- 172.21.15.32:0/2278290686 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f684c0ce980 0x7f684c1531b0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f684c073110 tx=0x7f684400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:02.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.204+0000 7f6843fff640 1 -- 172.21.15.32:0/2278290686 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f684400fde0 con 0x7f684c0ce980 2024-04-24T23:04:02.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.204+0000 7f6843fff640 1 -- 172.21.15.32:0/2278290686 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f684401d070 con 0x7f684c0ce980 2024-04-24T23:04:02.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.204+0000 7f685223f640 1 -- 172.21.15.32:0/2278290686 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f684c15f860 con 0x7f684c0ce980 2024-04-24T23:04:02.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.204+0000 7f6843fff640 1 -- 172.21.15.32:0/2278290686 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6844015670 con 0x7f684c0ce980 2024-04-24T23:04:02.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.204+0000 7f685223f640 1 -- 172.21.15.32:0/2278290686 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f684c15fe20 con 0x7f684c0ce980 2024-04-24T23:04:02.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.205+0000 7f685223f640 1 -- 172.21.15.32:0/2278290686 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f684c068430 con 0x7f684c0ce980 2024-04-24T23:04:02.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.206+0000 7f6843fff640 1 -- 172.21.15.32:0/2278290686 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6844015810 con 0x7f684c0ce980 2024-04-24T23:04:02.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.206+0000 7f6843fff640 1 --2- 172.21.15.32:0/2278290686 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f682c06be30 0x7f682c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:02.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.206+0000 7f6843fff640 1 -- 172.21.15.32:0/2278290686 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f684408b720 con 0x7f684c0ce980 2024-04-24T23:04:02.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.206+0000 7f684b7fe640 1 --2- 172.21.15.32:0/2278290686 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f682c06be30 0x7f682c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:02.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.207+0000 7f684b7fe640 1 --2- 172.21.15.32:0/2278290686 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f682c06be30 0x7f682c06e2f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f683c033000 tx=0x7f683c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:02.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.210+0000 7f6843fff640 1 -- 172.21.15.32:0/2278290686 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6844010040 con 0x7f684c0ce980 2024-04-24T23:04:02.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:02.431+0000 7f685223f640 1 -- 172.21.15.32:0/2278290686 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f684c15bde0 con 0x7f684c0ce980 2024-04-24T23:04:03.412 INFO:tasks.ceph.mds.a.smithi032.stderr:2024-04-24T23:04:03.410+0000 7f6056bdf640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T23:04:03.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.429+0000 7f6843fff640 1 -- 172.21.15.32:0/2278290686 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v43) ==== 118+0+0 (secure 0 0 0) 0x7f684405a0d0 con 0x7f684c0ce980 2024-04-24T23:04:03.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.432+0000 7f685223f640 1 -- 172.21.15.32:0/2278290686 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f682c06be30 msgr2=0x7f682c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:03.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.432+0000 7f685223f640 1 --2- 172.21.15.32:0/2278290686 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f682c06be30 0x7f682c06e2f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f683c033000 tx=0x7f683c008030 comp rx=0 tx=0).stop 2024-04-24T23:04:03.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.432+0000 7f685223f640 1 -- 172.21.15.32:0/2278290686 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f684c0ce980 msgr2=0x7f684c1531b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:03.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.432+0000 7f685223f640 1 --2- 172.21.15.32:0/2278290686 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f684c0ce980 0x7f684c1531b0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f684c073110 tx=0x7f684400f580 comp rx=0 tx=0).stop 2024-04-24T23:04:03.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.432+0000 7f685223f640 1 -- 172.21.15.32:0/2278290686 shutdown_connections 2024-04-24T23:04:03.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.432+0000 7f685223f640 1 --2- 172.21.15.32:0/2278290686 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f682c06be30 0x7f682c06e2f0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:03.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.432+0000 7f685223f640 1 --2- 172.21.15.32:0/2278290686 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f684c0ce980 0x7f684c1531b0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:03.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.432+0000 7f685223f640 1 --2- 172.21.15.32:0/2278290686 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f684c160620 0x7f684c0ce440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:03.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.432+0000 7f685223f640 1 --2- 172.21.15.32:0/2278290686 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68380039b0 0x7f684c0cc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:03.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.432+0000 7f685223f640 1 -- 172.21.15.32:0/2278290686 >> 172.21.15.32:0/2278290686 conn(0x7f684c0d11d0 msgr2=0x7f684c156b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:03.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.433+0000 7f685223f640 1 -- 172.21.15.32:0/2278290686 shutdown_connections 2024-04-24T23:04:03.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.433+0000 7f685223f640 1 -- 172.21.15.32:0/2278290686 wait complete. 2024-04-24T23:04:03.447 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-24T23:04:03.448 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-24T23:04:03.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.556+0000 7f9774963640 1 Processor -- start 2024-04-24T23:04:03.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.556+0000 7f9774963640 1 -- start start 2024-04-24T23:04:03.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.557+0000 7f9774963640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f977006cd00 0x7f97700ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:03.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.557+0000 7f9774963640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97700b4520 con 0x7f977015cb00 2024-04-24T23:04:03.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.557+0000 7f9774963640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97700b4800 con 0x7f977015b380 2024-04-24T23:04:03.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.557+0000 7f9774963640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f97700b4ae0 con 0x7f977006cd00 2024-04-24T23:04:03.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.557+0000 7f976ed9d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f977015cb00 0x7f97700ce970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42610/0 (socket says 172.21.15.32:42610) 2024-04-24T23:04:03.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.557+0000 7f976ed9d640 1 -- 172.21.15.32:0/1287495172 learned_addr learned my addr 172.21.15.32:0/1287495172 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:04:03.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.557+0000 7f976dd9b640 1 --2- 172.21.15.32:0/1287495172 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f977006cd00 0x7f97700ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:03.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.557+0000 7f976d59a640 1 -- 172.21.15.32:0/1287495172 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4134693160 0 0) 0x7f97700b4520 con 0x7f977015cb00 2024-04-24T23:04:03.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.557+0000 7f976d59a640 1 -- 172.21.15.32:0/1287495172 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9758003520 con 0x7f977015cb00 2024-04-24T23:04:03.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.557+0000 7f976d59a640 1 -- 172.21.15.32:0/1287495172 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 336236376 0 0) 0x7f97700b4800 con 0x7f977015b380 2024-04-24T23:04:03.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.557+0000 7f976d59a640 1 -- 172.21.15.32:0/1287495172 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f97700b4520 con 0x7f977015b380 2024-04-24T23:04:03.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.557+0000 7f976d59a640 1 -- 172.21.15.32:0/1287495172 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 612637941 0 0) 0x7f9758003520 con 0x7f977015cb00 2024-04-24T23:04:03.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.557+0000 7f976d59a640 1 -- 172.21.15.32:0/1287495172 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f977006cd00 msgr2=0x7f97700ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:03.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.557+0000 7f976d59a640 1 --2- 172.21.15.32:0/1287495172 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f977006cd00 0x7f97700ce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:03.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.557+0000 7f976d59a640 1 -- 172.21.15.32:0/1287495172 >> v1:172.21.15.121:6789/0 conn(0x7f977015b380 legacy=0x7f977006c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:03.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.557+0000 7f976d59a640 1 -- 172.21.15.32:0/1287495172 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97700b4dc0 con 0x7f977015cb00 2024-04-24T23:04:03.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.557+0000 7f976d59a640 1 -- 172.21.15.32:0/1287495172 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f97680019f0 con 0x7f977015cb00 2024-04-24T23:04:03.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.557+0000 7f976dd9b640 1 --2- 172.21.15.32:0/1287495172 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f977006cd00 0x7f97700ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:04:03.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.557+0000 7f976d59a640 1 -- 172.21.15.32:0/1287495172 >> v1:172.21.15.32:6789/0 conn(0x7f977015cb00 legacy=0x7f97700ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:03.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.557+0000 7f976d59a640 1 --2- 172.21.15.32:0/1287495172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9758003b70 0x7f9758003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:03.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.557+0000 7f976d59a640 1 --2- 172.21.15.32:0/1287495172 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9758004800 0x7f9758004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:03.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.558+0000 7f976d59a640 1 --2- 172.21.15.32:0/1287495172 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f977006cd00 0x7f97580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:03.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.558+0000 7f976d59a640 1 -- 172.21.15.32:0/1287495172 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f97700b4ae0 con 0x7f9758003b70 2024-04-24T23:04:03.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.558+0000 7f976d59a640 1 -- 172.21.15.32:0/1287495172 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f975800bad0 con 0x7f9758004800 2024-04-24T23:04:03.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.558+0000 7f976d59a640 1 -- 172.21.15.32:0/1287495172 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f975800bcd0 con 0x7f977006cd00 2024-04-24T23:04:03.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.558+0000 7f976e59c640 1 --2- 172.21.15.32:0/1287495172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9758003b70 0x7f9758003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:03.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.558+0000 7f976ed9d640 1 --2- 172.21.15.32:0/1287495172 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f977006cd00 0x7f97580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:03.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.558+0000 7f976dd9b640 1 --2- 172.21.15.32:0/1287495172 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9758004800 0x7f9758004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:03.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.558+0000 7f976e59c640 1 -- 172.21.15.32:0/1287495172 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f977006cd00 msgr2=0x7f97580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:03.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.558+0000 7f976e59c640 1 --2- 172.21.15.32:0/1287495172 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f977006cd00 0x7f97580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:03.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.558+0000 7f976e59c640 1 -- 172.21.15.32:0/1287495172 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9758004800 msgr2=0x7f9758004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:03.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.558+0000 7f976e59c640 1 --2- 172.21.15.32:0/1287495172 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9758004800 0x7f9758004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:03.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.558+0000 7f976e59c640 1 -- 172.21.15.32:0/1287495172 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9758003520 con 0x7f9758003b70 2024-04-24T23:04:03.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.558+0000 7f976ed9d640 1 --2- 172.21.15.32:0/1287495172 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f977006cd00 0x7f97580051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:04:03.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.558+0000 7f976dd9b640 1 --2- 172.21.15.32:0/1287495172 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9758004800 0x7f9758004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:04:03.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.559+0000 7f976e59c640 1 --2- 172.21.15.32:0/1287495172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9758003b70 0x7f9758003f70 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f97600018a0 tx=0x7f9758005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b80b6f568d4a3c7a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:03.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.559+0000 7f976d59a640 1 -- 172.21.15.32:0/1287495172 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97600014f0 con 0x7f9758003b70 2024-04-24T23:04:03.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.559+0000 7f976d59a640 1 -- 172.21.15.32:0/1287495172 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97600052d0 con 0x7f9758003b70 2024-04-24T23:04:03.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.559+0000 7f976d59a640 1 -- 172.21.15.32:0/1287495172 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976002ac40 con 0x7f9758003b70 2024-04-24T23:04:03.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.559+0000 7f9774963640 1 -- 172.21.15.32:0/1287495172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9758003b70 msgr2=0x7f9758003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:03.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.559+0000 7f9774963640 1 --2- 172.21.15.32:0/1287495172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9758003b70 0x7f9758003f70 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f97600018a0 tx=0x7f9758005830 comp rx=0 tx=0).stop 2024-04-24T23:04:03.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.559+0000 7f976ed9d640 1 -- 172.21.15.32:0/1287495172 reap_dead start 2024-04-24T23:04:03.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.559+0000 7f9774963640 1 -- 172.21.15.32:0/1287495172 shutdown_connections 2024-04-24T23:04:03.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.559+0000 7f9774963640 1 -- 172.21.15.32:0/1287495172 >> 172.21.15.32:0/1287495172 conn(0x7f977006b110 msgr2=0x7f977006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:03.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.559+0000 7f9774963640 1 -- 172.21.15.32:0/1287495172 shutdown_connections 2024-04-24T23:04:03.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.559+0000 7f9774963640 1 -- 172.21.15.32:0/1287495172 wait complete. 2024-04-24T23:04:03.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.561+0000 7f9774963640 1 Processor -- start 2024-04-24T23:04:03.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.561+0000 7f9774963640 1 -- start start 2024-04-24T23:04:03.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.562+0000 7f9774963640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9758003b70 0x7f9770149bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:03.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.562+0000 7f9774963640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97700ae8b0 0x7f97700aed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:03.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.562+0000 7f9774963640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f97700708b0 0x7f9770070d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:03.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.562+0000 7f9774963640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f97700ce430 con 0x7f97700ae8b0 2024-04-24T23:04:03.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.562+0000 7f9774963640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f977015b760 con 0x7f97700708b0 2024-04-24T23:04:03.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.562+0000 7f9774963640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f97700ce5b0 con 0x7f9758003b70 2024-04-24T23:04:03.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.562+0000 7f976dd9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97700ae8b0 0x7f97700aed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:03.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.562+0000 7f976dd9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97700ae8b0 0x7f97700aed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60182/0 (socket says 172.21.15.32:60182) 2024-04-24T23:04:03.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.562+0000 7f976dd9b640 1 -- 172.21.15.32:0/3666949676 learned_addr learned my addr 172.21.15.32:0/3666949676 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:03.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.562+0000 7f976ed9d640 1 --2- 172.21.15.32:0/3666949676 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f97700708b0 0x7f9770070d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:03.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.562+0000 7f976e59c640 1 --2- 172.21.15.32:0/3666949676 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9758003b70 0x7f9770149bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:03.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.562+0000 7f976dd9b640 1 -- 172.21.15.32:0/3666949676 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9758003b70 msgr2=0x7f9770149bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:03.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.562+0000 7f976dd9b640 1 --2- 172.21.15.32:0/3666949676 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9758003b70 0x7f9770149bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:03.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.562+0000 7f976dd9b640 1 -- 172.21.15.32:0/3666949676 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f97700708b0 msgr2=0x7f9770070d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:03.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.562+0000 7f976dd9b640 1 --2- 172.21.15.32:0/3666949676 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f97700708b0 0x7f9770070d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:03.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.562+0000 7f976dd9b640 1 -- 172.21.15.32:0/3666949676 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97700715e0 con 0x7f97700ae8b0 2024-04-24T23:04:03.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.562+0000 7f976dd9b640 1 --2- 172.21.15.32:0/3666949676 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97700ae8b0 0x7f97700aed30 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f9764011650 tx=0x7f9764011b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:03.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.562+0000 7f975effd640 1 -- 172.21.15.32:0/3666949676 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976400f940 con 0x7f97700ae8b0 2024-04-24T23:04:03.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.562+0000 7f975effd640 1 -- 172.21.15.32:0/3666949676 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f976400fae0 con 0x7f97700ae8b0 2024-04-24T23:04:03.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.562+0000 7f9774963640 1 -- 172.21.15.32:0/3666949676 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97700718c0 con 0x7f97700ae8b0 2024-04-24T23:04:03.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.562+0000 7f975effd640 1 -- 172.21.15.32:0/3666949676 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9764015460 con 0x7f97700ae8b0 2024-04-24T23:04:03.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.562+0000 7f9774963640 1 -- 172.21.15.32:0/3666949676 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9770071e80 con 0x7f97700ae8b0 2024-04-24T23:04:03.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.563+0000 7f9774963640 1 -- 172.21.15.32:0/3666949676 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9770068430 con 0x7f97700ae8b0 2024-04-24T23:04:03.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.564+0000 7f975effd640 1 -- 172.21.15.32:0/3666949676 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9764015600 con 0x7f97700ae8b0 2024-04-24T23:04:03.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.564+0000 7f975effd640 1 --2- 172.21.15.32:0/3666949676 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f975006be60 0x7f975006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:03.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.564+0000 7f975effd640 1 -- 172.21.15.32:0/3666949676 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7f976408b710 con 0x7f97700ae8b0 2024-04-24T23:04:03.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.565+0000 7f976e59c640 1 --2- 172.21.15.32:0/3666949676 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f975006be60 0x7f975006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:03.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.565+0000 7f976e59c640 1 --2- 172.21.15.32:0/3666949676 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f975006be60 0x7f975006e320 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f9760032000 tx=0x7f9760009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:03.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.569+0000 7f975effd640 1 -- 172.21.15.32:0/3666949676 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9764019030 con 0x7f97700ae8b0 2024-04-24T23:04:03.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:03.789+0000 7f9774963640 1 -- 172.21.15.32:0/3666949676 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f97700afd60 con 0x7f97700ae8b0 2024-04-24T23:04:04.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.437+0000 7f975effd640 1 -- 172.21.15.32:0/3666949676 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v44) ==== 122+0+0 (secure 0 0 0) 0x7f97640599d0 con 0x7f97700ae8b0 2024-04-24T23:04:04.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.440+0000 7f9774963640 1 -- 172.21.15.32:0/3666949676 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f975006be60 msgr2=0x7f975006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:04.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.440+0000 7f9774963640 1 --2- 172.21.15.32:0/3666949676 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f975006be60 0x7f975006e320 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f9760032000 tx=0x7f9760009030 comp rx=0 tx=0).stop 2024-04-24T23:04:04.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.440+0000 7f9774963640 1 -- 172.21.15.32:0/3666949676 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97700ae8b0 msgr2=0x7f97700aed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:04.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.440+0000 7f9774963640 1 --2- 172.21.15.32:0/3666949676 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97700ae8b0 0x7f97700aed30 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f9764011650 tx=0x7f9764011b10 comp rx=0 tx=0).stop 2024-04-24T23:04:04.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.440+0000 7f9774963640 1 -- 172.21.15.32:0/3666949676 shutdown_connections 2024-04-24T23:04:04.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.440+0000 7f9774963640 1 --2- 172.21.15.32:0/3666949676 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f975006be60 0x7f975006e320 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:04.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.440+0000 7f9774963640 1 --2- 172.21.15.32:0/3666949676 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97700ae8b0 0x7f97700aed30 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:04.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.440+0000 7f9774963640 1 --2- 172.21.15.32:0/3666949676 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f97700708b0 0x7f9770070d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:04.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.440+0000 7f9774963640 1 --2- 172.21.15.32:0/3666949676 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9758003b70 0x7f9770149bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:04.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.440+0000 7f9774963640 1 -- 172.21.15.32:0/3666949676 >> 172.21.15.32:0/3666949676 conn(0x7f977006b110 msgr2=0x7f977015bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:04.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.441+0000 7f9774963640 1 -- 172.21.15.32:0/3666949676 shutdown_connections 2024-04-24T23:04:04.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.441+0000 7f9774963640 1 -- 172.21.15.32:0/3666949676 wait complete. 2024-04-24T23:04:04.455 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-24T23:04:04.455 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:04:04.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.559+0000 7fb96b390640 1 Processor -- start 2024-04-24T23:04:04.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.559+0000 7fb96b390640 1 -- start start 2024-04-24T23:04:04.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.560+0000 7fb96b390640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb96406b820 0x7fb9640d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:04.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.560+0000 7fb96b390640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb9640d4210 0x7fb9640b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:04.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.560+0000 7fb96b390640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9640d3170 con 0x7fb96415e9b0 2024-04-24T23:04:04.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.560+0000 7fb96b390640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fb9640d3450 con 0x7fb96406b820 2024-04-24T23:04:04.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.560+0000 7fb96b390640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fb9640d3730 con 0x7fb9640d4210 2024-04-24T23:04:04.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.560+0000 7fb96892b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb96406b820 0x7fb9640d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:04.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.560+0000 7fb96892b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb96406b820 0x7fb9640d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60186/0 (socket says 172.21.15.32:60186) 2024-04-24T23:04:04.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.560+0000 7fb96892b640 1 -- 172.21.15.32:0/903268646 learned_addr learned my addr 172.21.15.32:0/903268646 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:04.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.561+0000 7fb96992d640 1 --2- 172.21.15.32:0/903268646 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb9640d4210 0x7fb9640b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:04.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.561+0000 7fb96892b640 1 -- 172.21.15.32:0/903268646 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb9640d4210 msgr2=0x7fb9640b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:04.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.561+0000 7fb96892b640 1 --2- 172.21.15.32:0/903268646 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb9640d4210 0x7fb9640b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:04.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.561+0000 7fb96892b640 1 -- 172.21.15.32:0/903268646 >> v1:172.21.15.121:6789/0 conn(0x7fb96415e9b0 legacy=0x7fb96406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:04.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.561+0000 7fb96892b640 1 -- 172.21.15.32:0/903268646 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9640d3a10 con 0x7fb96406b820 2024-04-24T23:04:04.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.561+0000 7fb96992d640 1 --2- 172.21.15.32:0/903268646 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb9640d4210 0x7fb9640b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:04:04.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.561+0000 7fb96892b640 1 --2- 172.21.15.32:0/903268646 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb96406b820 0x7fb9640d2b90 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fb958009870 tx=0x7fb95802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd425083e462459a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:04.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.562+0000 7fb963fff640 1 -- 172.21.15.32:0/903268646 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb958037070 con 0x7fb96406b820 2024-04-24T23:04:04.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.562+0000 7fb963fff640 1 -- 172.21.15.32:0/903268646 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb958032440 con 0x7fb96406b820 2024-04-24T23:04:04.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.562+0000 7fb963fff640 1 -- 172.21.15.32:0/903268646 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb958032810 con 0x7fb96406b820 2024-04-24T23:04:04.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.562+0000 7fb96b390640 1 -- 172.21.15.32:0/903268646 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb96406b820 msgr2=0x7fb9640d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:04.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.562+0000 7fb96b390640 1 --2- 172.21.15.32:0/903268646 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb96406b820 0x7fb9640d2b90 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fb958009870 tx=0x7fb95802a270 comp rx=0 tx=0).stop 2024-04-24T23:04:04.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.562+0000 7fb96b390640 1 -- 172.21.15.32:0/903268646 shutdown_connections 2024-04-24T23:04:04.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.562+0000 7fb96b390640 1 --2- 172.21.15.32:0/903268646 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb9640d4210 0x7fb9640b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:04.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.562+0000 7fb96b390640 1 --2- 172.21.15.32:0/903268646 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb96406b820 0x7fb9640d2b90 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:04.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.562+0000 7fb96b390640 1 -- 172.21.15.32:0/903268646 >> 172.21.15.32:0/903268646 conn(0x7fb9640d11d0 msgr2=0x7fb9640d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:04.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.562+0000 7fb96b390640 1 -- 172.21.15.32:0/903268646 shutdown_connections 2024-04-24T23:04:04.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.562+0000 7fb96b390640 1 -- 172.21.15.32:0/903268646 wait complete. 2024-04-24T23:04:04.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.563+0000 7fb96b390640 1 Processor -- start 2024-04-24T23:04:04.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.564+0000 7fb96b390640 1 -- start start 2024-04-24T23:04:04.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.564+0000 7fb96b390640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb96406b820 0x7fb964154530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:04.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.564+0000 7fb96b390640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9640d4210 0x7fb9641617f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:04.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.564+0000 7fb96b390640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb96415e9b0 0x7fb9641579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:04.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.564+0000 7fb96b390640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb9640ba0e0 con 0x7fb96415e9b0 2024-04-24T23:04:04.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.564+0000 7fb96b390640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb9640d2400 con 0x7fb9640d4210 2024-04-24T23:04:04.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.564+0000 7fb96b390640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fb9640ba260 con 0x7fb96406b820 2024-04-24T23:04:04.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.564+0000 7fb96992d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb96415e9b0 0x7fb9641579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:04.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.564+0000 7fb96992d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb96415e9b0 0x7fb9641579c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60194/0 (socket says 172.21.15.32:60194) 2024-04-24T23:04:04.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.564+0000 7fb96992d640 1 -- 172.21.15.32:0/1623217350 learned_addr learned my addr 172.21.15.32:0/1623217350 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:04.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.565+0000 7fb96892b640 1 --2- 172.21.15.32:0/1623217350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9640d4210 0x7fb9641617f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:04.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.565+0000 7fb96992d640 1 -- 172.21.15.32:0/1623217350 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb96406b820 msgr2=0x7fb964154530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:04.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.565+0000 7fb96912c640 1 --2- 172.21.15.32:0/1623217350 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb96406b820 0x7fb964154530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:04.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.565+0000 7fb96992d640 1 --2- 172.21.15.32:0/1623217350 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb96406b820 0x7fb964154530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:04.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.565+0000 7fb96992d640 1 -- 172.21.15.32:0/1623217350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9640d4210 msgr2=0x7fb9641617f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:04.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.565+0000 7fb96992d640 1 --2- 172.21.15.32:0/1623217350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9640d4210 0x7fb9641617f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:04.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.565+0000 7fb96992d640 1 -- 172.21.15.32:0/1623217350 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb964156920 con 0x7fb96415e9b0 2024-04-24T23:04:04.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.565+0000 7fb96892b640 1 --2- 172.21.15.32:0/1623217350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9640d4210 0x7fb9641617f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:04:04.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.565+0000 7fb96912c640 1 --2- 172.21.15.32:0/1623217350 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb96406b820 0x7fb964154530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:04:04.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.565+0000 7fb96992d640 1 --2- 172.21.15.32:0/1623217350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb96415e9b0 0x7fb9641579c0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fb954007e70 tx=0x7fb95400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:04.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.565+0000 7fb961ffb640 1 -- 172.21.15.32:0/1623217350 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb95401a070 con 0x7fb96415e9b0 2024-04-24T23:04:04.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.565+0000 7fb961ffb640 1 -- 172.21.15.32:0/1623217350 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb954015600 con 0x7fb96415e9b0 2024-04-24T23:04:04.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.565+0000 7fb96b390640 1 -- 172.21.15.32:0/1623217350 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb964156c00 con 0x7fb96415e9b0 2024-04-24T23:04:04.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.565+0000 7fb961ffb640 1 -- 172.21.15.32:0/1623217350 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9540159f0 con 0x7fb96415e9b0 2024-04-24T23:04:04.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.565+0000 7fb96b390640 1 -- 172.21.15.32:0/1623217350 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9641571c0 con 0x7fb96415e9b0 2024-04-24T23:04:04.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.566+0000 7fb96b390640 1 -- 172.21.15.32:0/1623217350 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb964068430 con 0x7fb96415e9b0 2024-04-24T23:04:04.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.567+0000 7fb961ffb640 1 -- 172.21.15.32:0/1623217350 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb95401d070 con 0x7fb96415e9b0 2024-04-24T23:04:04.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.567+0000 7fb961ffb640 1 --2- 172.21.15.32:0/1623217350 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb95006be30 0x7fb95006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:04.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.567+0000 7fb961ffb640 1 -- 172.21.15.32:0/1623217350 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7fb95408b170 con 0x7fb96415e9b0 2024-04-24T23:04:04.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.567+0000 7fb96912c640 1 --2- 172.21.15.32:0/1623217350 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb95006be30 0x7fb95006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:04.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.568+0000 7fb96912c640 1 --2- 172.21.15.32:0/1623217350 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb95006be30 0x7fb95006e2f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb95c0015e0 tx=0x7fb95c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:04.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.571+0000 7fb961ffb640 1 -- 172.21.15.32:0/1623217350 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb954010070 con 0x7fb96415e9b0 2024-04-24T23:04:04.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.793+0000 7fb96b390640 1 -- 172.21.15.32:0/1623217350 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb9640b8880 con 0x7fb96415e9b0 2024-04-24T23:04:04.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.794+0000 7fb961ffb640 1 -- 172.21.15.32:0/1623217350 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5186 (secure 0 0 0) 0x7fb954059b20 con 0x7fb96415e9b0 2024-04-24T23:04:04.795 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:04:04.795 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":44,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6175,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/434273738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":434273738},{"type":"v1","addr":"172.21.15.121:6836","nonce":434273738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6178,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1104557785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1104557785},{"type":"v1","addr":"172.21.15.121:6837","nonce":1104557785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6888,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/3137388886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3137388886},{"type":"v1","addr":"172.21.15.32:6834","nonce":3137388886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:04:00.783579+0000","modified":"2024-04-24T23:04:03.790658+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6891},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6891":{"gid":6891,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/3327203898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":3327203898},{"type":"v1","addr":"172.21.15.32:6835","nonce":3327203898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6891,"qdb_cluster":[6891]},"id":6}]} 2024-04-24T23:04:04.796 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 44 2024-04-24T23:04:04.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.795+0000 7fb96b390640 1 -- 172.21.15.32:0/1623217350 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb95006be30 msgr2=0x7fb95006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:04.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.795+0000 7fb96b390640 1 --2- 172.21.15.32:0/1623217350 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb95006be30 0x7fb95006e2f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb95c0015e0 tx=0x7fb95c009040 comp rx=0 tx=0).stop 2024-04-24T23:04:04.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.796+0000 7fb96b390640 1 -- 172.21.15.32:0/1623217350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb96415e9b0 msgr2=0x7fb9641579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:04.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.796+0000 7fb96b390640 1 --2- 172.21.15.32:0/1623217350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb96415e9b0 0x7fb9641579c0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fb954007e70 tx=0x7fb95400d4a0 comp rx=0 tx=0).stop 2024-04-24T23:04:04.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.796+0000 7fb96b390640 1 -- 172.21.15.32:0/1623217350 shutdown_connections 2024-04-24T23:04:04.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.796+0000 7fb96b390640 1 --2- 172.21.15.32:0/1623217350 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb95006be30 0x7fb95006e2f0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:04.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.796+0000 7fb96b390640 1 --2- 172.21.15.32:0/1623217350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb96415e9b0 0x7fb9641579c0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:04.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.796+0000 7fb96b390640 1 --2- 172.21.15.32:0/1623217350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9640d4210 0x7fb9641617f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:04.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.796+0000 7fb96b390640 1 --2- 172.21.15.32:0/1623217350 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb96406b820 0x7fb964154530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:04.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.796+0000 7fb96b390640 1 -- 172.21.15.32:0/1623217350 >> 172.21.15.32:0/1623217350 conn(0x7fb9640d11d0 msgr2=0x7fb9640b78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:04.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.796+0000 7fb96b390640 1 -- 172.21.15.32:0/1623217350 shutdown_connections 2024-04-24T23:04:04.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.796+0000 7fb96b390640 1 -- 172.21.15.32:0/1623217350 wait complete. 2024-04-24T23:04:04.808 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:04:04.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.907+0000 7fa9ac888640 1 Processor -- start 2024-04-24T23:04:04.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.907+0000 7fa9ac888640 1 -- start start 2024-04-24T23:04:04.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.908+0000 7fa9ac888640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9a8158dd0 0x7fa9a806b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:04.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.908+0000 7fa9ac888640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa9a806b860 0x7fa9a806d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:04.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.908+0000 7fa9ac888640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9a80b6d10 con 0x7fa9a806da90 2024-04-24T23:04:04.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.908+0000 7fa9ac888640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fa9a815ef10 con 0x7fa9a806b860 2024-04-24T23:04:04.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.908+0000 7fa9ac888640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa9a815f1f0 con 0x7fa9a8158dd0 2024-04-24T23:04:04.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.908+0000 7fa9a5d9b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa9a806b860 0x7fa9a806d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:04.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.908+0000 7fa9a5d9b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa9a806b860 0x7fa9a806d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60196/0 (socket says 172.21.15.32:60196) 2024-04-24T23:04:04.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.908+0000 7fa9a5d9b640 1 -- 172.21.15.32:0/2979874985 learned_addr learned my addr 172.21.15.32:0/2979874985 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:04.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.909+0000 7fa9a659c640 1 --2- 172.21.15.32:0/2979874985 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9a8158dd0 0x7fa9a806b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:04.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.909+0000 7fa9a5d9b640 1 -- 172.21.15.32:0/2979874985 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9a8158dd0 msgr2=0x7fa9a806b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:04.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.909+0000 7fa9a5d9b640 1 --2- 172.21.15.32:0/2979874985 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9a8158dd0 0x7fa9a806b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:04.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.909+0000 7fa9a5d9b640 1 -- 172.21.15.32:0/2979874985 >> v1:172.21.15.121:6789/0 conn(0x7fa9a806da90 legacy=0x7fa9a8072580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:04.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.909+0000 7fa9a5d9b640 1 -- 172.21.15.32:0/2979874985 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a815f4d0 con 0x7fa9a806b860 2024-04-24T23:04:04.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.909+0000 7fa9a5d9b640 1 --2- 172.21.15.32:0/2979874985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9a806b860 0x7fa9a806d550 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fa990009870 tx=0x7fa99002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c35dd6d57e2c165 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:04.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.909+0000 7fa9a559a640 1 -- 172.21.15.32:0/2979874985 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa990037070 con 0x7fa9a806b860 2024-04-24T23:04:04.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.910+0000 7fa9a559a640 1 -- 172.21.15.32:0/2979874985 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa990032440 con 0x7fa9a806b860 2024-04-24T23:04:04.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.910+0000 7fa9a559a640 1 -- 172.21.15.32:0/2979874985 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa990032810 con 0x7fa9a806b860 2024-04-24T23:04:04.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.910+0000 7fa9ac888640 1 -- 172.21.15.32:0/2979874985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9a806b860 msgr2=0x7fa9a806d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:04.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.910+0000 7fa9ac888640 1 --2- 172.21.15.32:0/2979874985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9a806b860 0x7fa9a806d550 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fa990009870 tx=0x7fa99002a210 comp rx=0 tx=0).stop 2024-04-24T23:04:04.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.910+0000 7fa9ac888640 1 -- 172.21.15.32:0/2979874985 shutdown_connections 2024-04-24T23:04:04.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.910+0000 7fa9ac888640 1 --2- 172.21.15.32:0/2979874985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9a806b860 0x7fa9a806d550 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:04.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.910+0000 7fa9ac888640 1 --2- 172.21.15.32:0/2979874985 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9a8158dd0 0x7fa9a806b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:04.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.910+0000 7fa9ac888640 1 -- 172.21.15.32:0/2979874985 >> 172.21.15.32:0/2979874985 conn(0x7fa9a806e800 msgr2=0x7fa9a806ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:04.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.910+0000 7fa9ac888640 1 -- 172.21.15.32:0/2979874985 shutdown_connections 2024-04-24T23:04:04.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.910+0000 7fa9ac888640 1 -- 172.21.15.32:0/2979874985 wait complete. 2024-04-24T23:04:04.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.911+0000 7fa9ac888640 1 Processor -- start 2024-04-24T23:04:04.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.912+0000 7fa9ac888640 1 -- start start 2024-04-24T23:04:04.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.912+0000 7fa9ac888640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9a806b860 0x7fa9a80b4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:04.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.912+0000 7fa9ac888640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9a806da90 0x7fa9a814d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:04.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.912+0000 7fa9ac888640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9a8158dd0 0x7fa9a81ea620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:04.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.912+0000 7fa9ac888640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a815fcd0 con 0x7fa9a8158dd0 2024-04-24T23:04:04.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.912+0000 7fa9ac888640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a806bce0 con 0x7fa9a806b860 2024-04-24T23:04:04.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.912+0000 7fa9ac888640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa9a806ded0 con 0x7fa9a806da90 2024-04-24T23:04:04.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.912+0000 7fa9a6d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9a8158dd0 0x7fa9a81ea620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:04.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.912+0000 7fa9a6d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9a8158dd0 0x7fa9a81ea620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60206/0 (socket says 172.21.15.32:60206) 2024-04-24T23:04:04.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.912+0000 7fa9a6d9d640 1 -- 172.21.15.32:0/3453610847 learned_addr learned my addr 172.21.15.32:0/3453610847 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:04.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.912+0000 7fa9a659c640 1 --2- 172.21.15.32:0/3453610847 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9a806b860 0x7fa9a80b4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:04.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.913+0000 7fa9a5d9b640 1 --2- 172.21.15.32:0/3453610847 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9a806da90 0x7fa9a814d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:04.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.913+0000 7fa9a6d9d640 1 -- 172.21.15.32:0/3453610847 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9a806da90 msgr2=0x7fa9a814d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:04.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.913+0000 7fa9a6d9d640 1 --2- 172.21.15.32:0/3453610847 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9a806da90 0x7fa9a814d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:04.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.913+0000 7fa9a6d9d640 1 -- 172.21.15.32:0/3453610847 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9a806b860 msgr2=0x7fa9a80b4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:04.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.913+0000 7fa9a6d9d640 1 --2- 172.21.15.32:0/3453610847 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9a806b860 0x7fa9a80b4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:04.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.913+0000 7fa9a6d9d640 1 -- 172.21.15.32:0/3453610847 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a814e0d0 con 0x7fa9a8158dd0 2024-04-24T23:04:04.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.913+0000 7fa9a659c640 1 --2- 172.21.15.32:0/3453610847 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9a806b860 0x7fa9a80b4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:04:04.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.913+0000 7fa9a5d9b640 1 --2- 172.21.15.32:0/3453610847 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9a806da90 0x7fa9a814d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:04:04.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.913+0000 7fa9a6d9d640 1 --2- 172.21.15.32:0/3453610847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9a8158dd0 0x7fa9a81ea620 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fa99800c780 tx=0x7fa99800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:04.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.913+0000 7fa99effd640 1 -- 172.21.15.32:0/3453610847 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa998004b60 con 0x7fa9a8158dd0 2024-04-24T23:04:04.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.913+0000 7fa99effd640 1 -- 172.21.15.32:0/3453610847 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa998004d00 con 0x7fa9a8158dd0 2024-04-24T23:04:04.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.913+0000 7fa9ac888640 1 -- 172.21.15.32:0/3453610847 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9a8151310 con 0x7fa9a8158dd0 2024-04-24T23:04:04.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.913+0000 7fa99effd640 1 -- 172.21.15.32:0/3453610847 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99800aa90 con 0x7fa9a8158dd0 2024-04-24T23:04:04.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.913+0000 7fa9ac888640 1 -- 172.21.15.32:0/3453610847 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9a80b2f20 con 0x7fa9a8158dd0 2024-04-24T23:04:04.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.914+0000 7fa9ac888640 1 -- 172.21.15.32:0/3453610847 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9a8070230 con 0x7fa9a8158dd0 2024-04-24T23:04:04.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.915+0000 7fa99effd640 1 -- 172.21.15.32:0/3453610847 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa99800ac30 con 0x7fa9a8158dd0 2024-04-24T23:04:04.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.915+0000 7fa99effd640 1 --2- 172.21.15.32:0/3453610847 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa98806be30 0x7fa98806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:04.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.915+0000 7fa99effd640 1 -- 172.21.15.32:0/3453610847 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7fa99808b0f0 con 0x7fa9a8158dd0 2024-04-24T23:04:04.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.915+0000 7fa9a659c640 1 --2- 172.21.15.32:0/3453610847 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa98806be30 0x7fa98806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:04.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.916+0000 7fa9a659c640 1 --2- 172.21.15.32:0/3453610847 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa98806be30 0x7fa98806e2f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa9a0006fd0 tx=0x7fa9a0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:04.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:04.919+0000 7fa99effd640 1 -- 172.21.15.32:0/3453610847 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa9a8070230 con 0x7fa9a8158dd0 2024-04-24T23:04:05.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.104+0000 7fa9ac888640 1 -- 172.21.15.32:0/3453610847 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa9a8152930 con 0x7fa9a8158dd0 2024-04-24T23:04:05.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.105+0000 7fa99effd640 1 -- 172.21.15.32:0/3453610847 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) ==== 74+0+16695 (secure 0 0 0) 0x7fa998020070 con 0x7fa9a8158dd0 2024-04-24T23:04:05.106 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:04:05.107 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":104,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:04:02.427694+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:03:59.079157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-24T23:03:59.765179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:04:05.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.107+0000 7fa9ac888640 1 -- 172.21.15.32:0/3453610847 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa98806be30 msgr2=0x7fa98806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:05.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.107+0000 7fa9ac888640 1 --2- 172.21.15.32:0/3453610847 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa98806be30 0x7fa98806e2f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa9a0006fd0 tx=0x7fa9a0008040 comp rx=0 tx=0).stop 2024-04-24T23:04:05.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.107+0000 7fa9ac888640 1 -- 172.21.15.32:0/3453610847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9a8158dd0 msgr2=0x7fa9a81ea620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:05.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.107+0000 7fa9ac888640 1 --2- 172.21.15.32:0/3453610847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9a8158dd0 0x7fa9a81ea620 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fa99800c780 tx=0x7fa99800cc40 comp rx=0 tx=0).stop 2024-04-24T23:04:05.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.107+0000 7fa9ac888640 1 -- 172.21.15.32:0/3453610847 shutdown_connections 2024-04-24T23:04:05.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.107+0000 7fa9ac888640 1 --2- 172.21.15.32:0/3453610847 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa98806be30 0x7fa98806e2f0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:05.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.107+0000 7fa9ac888640 1 --2- 172.21.15.32:0/3453610847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9a8158dd0 0x7fa9a81ea620 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:05.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.107+0000 7fa9ac888640 1 --2- 172.21.15.32:0/3453610847 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9a806da90 0x7fa9a814d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:05.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.107+0000 7fa9ac888640 1 --2- 172.21.15.32:0/3453610847 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9a806b860 0x7fa9a80b4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:05.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.107+0000 7fa9ac888640 1 -- 172.21.15.32:0/3453610847 >> 172.21.15.32:0/3453610847 conn(0x7fa9a806e800 msgr2=0x7fa9a815fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:05.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.107+0000 7fa9ac888640 1 -- 172.21.15.32:0/3453610847 shutdown_connections 2024-04-24T23:04:05.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.107+0000 7fa9ac888640 1 -- 172.21.15.32:0/3453610847 wait complete. 2024-04-24T23:04:05.119 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-24T23:04:05.120 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:04:05.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.220+0000 7fbf9357e640 1 Processor -- start 2024-04-24T23:04:05.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.220+0000 7fbf9357e640 1 -- start start 2024-04-24T23:04:05.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.221+0000 7fbf9357e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf8c15b380 0x7fbf8c06c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:05.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.221+0000 7fbf9357e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbf8c0ceb40 0x7fbf8c06cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:05.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.221+0000 7fbf9357e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf8c0b4520 con 0x7fbf8c15cb00 2024-04-24T23:04:05.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.221+0000 7fbf9357e640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fbf8c0cef90 con 0x7fbf8c0ceb40 2024-04-24T23:04:05.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.221+0000 7fbf9357e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fbf8c0b4800 con 0x7fbf8c15b380 2024-04-24T23:04:05.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.221+0000 7fbf91b1b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbf8c0ceb40 0x7fbf8c06cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:05.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.221+0000 7fbf91b1b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbf8c0ceb40 0x7fbf8c06cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60212/0 (socket says 172.21.15.32:60212) 2024-04-24T23:04:05.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.221+0000 7fbf91b1b640 1 -- 172.21.15.32:0/528000105 learned_addr learned my addr 172.21.15.32:0/528000105 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:05.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.221+0000 7fbf90b19640 1 --1- 172.21.15.32:0/528000105 >> v1:172.21.15.32:6789/0 conn(0x7fbf8c15cb00 0x7fbf8c0ce430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42614/0 (socket says 172.21.15.32:42614) 2024-04-24T23:04:05.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.221+0000 7fbf9131a640 1 --2- 172.21.15.32:0/528000105 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf8c15b380 0x7fbf8c06c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:05.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.221+0000 7fbf91b1b640 1 -- 172.21.15.32:0/528000105 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf8c15b380 msgr2=0x7fbf8c06c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:05.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.221+0000 7fbf91b1b640 1 --2- 172.21.15.32:0/528000105 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf8c15b380 0x7fbf8c06c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:05.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.221+0000 7fbf91b1b640 1 -- 172.21.15.32:0/528000105 >> v1:172.21.15.32:6789/0 conn(0x7fbf8c15cb00 legacy=0x7fbf8c0ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:05.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.221+0000 7fbf91b1b640 1 -- 172.21.15.32:0/528000105 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf8c0b4ae0 con 0x7fbf8c0ceb40 2024-04-24T23:04:05.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.221+0000 7fbf8bfff640 1 -- 172.21.15.32:0/528000105 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 495470365 0 0) 0x7fbf8c0b4520 con 0x7fbf8c15cb00 2024-04-24T23:04:05.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.222+0000 7fbf91b1b640 1 --2- 172.21.15.32:0/528000105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf8c0ceb40 0x7fbf8c06cb30 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c009870 tx=0x7fbf7c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6db76ebd14b2726b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:05.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.222+0000 7fbf8bfff640 1 -- 172.21.15.32:0/528000105 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c037070 con 0x7fbf8c0ceb40 2024-04-24T23:04:05.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.222+0000 7fbf8bfff640 1 -- 172.21.15.32:0/528000105 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf7c032440 con 0x7fbf8c0ceb40 2024-04-24T23:04:05.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.222+0000 7fbf8bfff640 1 -- 172.21.15.32:0/528000105 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c032810 con 0x7fbf8c0ceb40 2024-04-24T23:04:05.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.222+0000 7fbf9357e640 1 -- 172.21.15.32:0/528000105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf8c0ceb40 msgr2=0x7fbf8c06cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:05.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.222+0000 7fbf9357e640 1 --2- 172.21.15.32:0/528000105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf8c0ceb40 0x7fbf8c06cb30 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c009870 tx=0x7fbf7c02a1c0 comp rx=0 tx=0).stop 2024-04-24T23:04:05.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.222+0000 7fbf9357e640 1 -- 172.21.15.32:0/528000105 shutdown_connections 2024-04-24T23:04:05.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.222+0000 7fbf9357e640 1 --2- 172.21.15.32:0/528000105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf8c0ceb40 0x7fbf8c06cb30 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:05.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.222+0000 7fbf9357e640 1 --2- 172.21.15.32:0/528000105 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf8c15b380 0x7fbf8c06c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:05.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.222+0000 7fbf9357e640 1 -- 172.21.15.32:0/528000105 >> 172.21.15.32:0/528000105 conn(0x7fbf8c06b110 msgr2=0x7fbf8c06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:05.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.223+0000 7fbf9357e640 1 -- 172.21.15.32:0/528000105 shutdown_connections 2024-04-24T23:04:05.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.223+0000 7fbf9357e640 1 -- 172.21.15.32:0/528000105 wait complete. 2024-04-24T23:04:05.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.224+0000 7fbf9357e640 1 Processor -- start 2024-04-24T23:04:05.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.224+0000 7fbf9357e640 1 -- start start 2024-04-24T23:04:05.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.224+0000 7fbf9357e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf8c0ceb40 0x7fbf8c158970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:05.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.224+0000 7fbf9357e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf8c15b380 0x7fbf8c151470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:05.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.224+0000 7fbf9357e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf8c15cb00 0x7fbf8c159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:05.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.224+0000 7fbf9357e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbf8c15b760 con 0x7fbf8c15cb00 2024-04-24T23:04:05.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.224+0000 7fbf9357e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fbf8c0b50a0 con 0x7fbf8c15b380 2024-04-24T23:04:05.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.225+0000 7fbf9357e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fbf8c0b55c0 con 0x7fbf8c0ceb40 2024-04-24T23:04:05.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.225+0000 7fbf91b1b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf8c15cb00 0x7fbf8c159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:05.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.225+0000 7fbf91b1b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf8c15cb00 0x7fbf8c159ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60228/0 (socket says 172.21.15.32:60228) 2024-04-24T23:04:05.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.225+0000 7fbf91b1b640 1 -- 172.21.15.32:0/4083403987 learned_addr learned my addr 172.21.15.32:0/4083403987 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:05.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.225+0000 7fbf9131a640 1 --2- 172.21.15.32:0/4083403987 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf8c0ceb40 0x7fbf8c158970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:05.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.225+0000 7fbf90b19640 1 --2- 172.21.15.32:0/4083403987 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf8c15b380 0x7fbf8c151470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:05.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.225+0000 7fbf91b1b640 1 -- 172.21.15.32:0/4083403987 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf8c0ceb40 msgr2=0x7fbf8c158970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:05.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.225+0000 7fbf91b1b640 1 --2- 172.21.15.32:0/4083403987 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf8c0ceb40 0x7fbf8c158970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:05.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.225+0000 7fbf91b1b640 1 -- 172.21.15.32:0/4083403987 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf8c15b380 msgr2=0x7fbf8c151470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:05.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.225+0000 7fbf91b1b640 1 --2- 172.21.15.32:0/4083403987 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf8c15b380 0x7fbf8c151470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:05.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.225+0000 7fbf91b1b640 1 -- 172.21.15.32:0/4083403987 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf8c158f50 con 0x7fbf8c15cb00 2024-04-24T23:04:05.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.225+0000 7fbf90b19640 1 --2- 172.21.15.32:0/4083403987 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf8c15b380 0x7fbf8c151470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:04:05.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.225+0000 7fbf91b1b640 1 --2- 172.21.15.32:0/4083403987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf8c15cb00 0x7fbf8c159ff0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c009870 tx=0x7fbf7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:05.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.225+0000 7fbf89ffb640 1 -- 172.21.15.32:0/4083403987 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c037040 con 0x7fbf8c15cb00 2024-04-24T23:04:05.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.225+0000 7fbf89ffb640 1 -- 172.21.15.32:0/4083403987 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf7c004230 con 0x7fbf8c15cb00 2024-04-24T23:04:05.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.225+0000 7fbf9357e640 1 -- 172.21.15.32:0/4083403987 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf8c159230 con 0x7fbf8c15cb00 2024-04-24T23:04:05.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.225+0000 7fbf89ffb640 1 -- 172.21.15.32:0/4083403987 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c003610 con 0x7fbf8c15cb00 2024-04-24T23:04:05.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.226+0000 7fbf9357e640 1 -- 172.21.15.32:0/4083403987 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf8c1597f0 con 0x7fbf8c15cb00 2024-04-24T23:04:05.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.227+0000 7fbf9357e640 1 -- 172.21.15.32:0/4083403987 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf8c068430 con 0x7fbf8c15cb00 2024-04-24T23:04:05.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.227+0000 7fbf89ffb640 1 -- 172.21.15.32:0/4083403987 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbf7c02ad10 con 0x7fbf8c15cb00 2024-04-24T23:04:05.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.227+0000 7fbf89ffb640 1 --2- 172.21.15.32:0/4083403987 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fbf7806be30 0x7fbf7806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:05.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.227+0000 7fbf89ffb640 1 -- 172.21.15.32:0/4083403987 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7fbf7c0abb50 con 0x7fbf8c15cb00 2024-04-24T23:04:05.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.228+0000 7fbf9131a640 1 --2- 172.21.15.32:0/4083403987 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fbf7806be30 0x7fbf7806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:05.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.228+0000 7fbf9131a640 1 --2- 172.21.15.32:0/4083403987 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fbf7806be30 0x7fbf7806e2f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fbf84003b80 tx=0x7fbf840098d0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:05.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.232+0000 7fbf89ffb640 1 -- 172.21.15.32:0/4083403987 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbf7c07a580 con 0x7fbf8c15cb00 2024-04-24T23:04:05.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.410+0000 7fbf9357e640 1 -- 172.21.15.32:0/4083403987 --> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbf8c0afd60 con 0x7fbf7806be30 2024-04-24T23:04:05.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.416+0000 7fbf89ffb640 1 -- 172.21.15.32:0/4083403987 <== mgr.4108 v2:172.21.15.121:6832/1848157943 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247923 (secure 0 0 0) 0x7fbf8c0afd60 con 0x7fbf7806be30 2024-04-24T23:04:05.420 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:04:05.422 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-24T23:04:05.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.419+0000 7fbf9357e640 1 -- 172.21.15.32:0/4083403987 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fbf7806be30 msgr2=0x7fbf7806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:05.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.419+0000 7fbf9357e640 1 --2- 172.21.15.32:0/4083403987 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fbf7806be30 0x7fbf7806e2f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fbf84003b80 tx=0x7fbf840098d0 comp rx=0 tx=0).stop 2024-04-24T23:04:05.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.419+0000 7fbf9357e640 1 -- 172.21.15.32:0/4083403987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf8c15cb00 msgr2=0x7fbf8c159ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:05.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.419+0000 7fbf9357e640 1 --2- 172.21.15.32:0/4083403987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf8c15cb00 0x7fbf8c159ff0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c009870 tx=0x7fbf7c0046c0 comp rx=0 tx=0).stop 2024-04-24T23:04:05.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.419+0000 7fbf9357e640 1 -- 172.21.15.32:0/4083403987 shutdown_connections 2024-04-24T23:04:05.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.419+0000 7fbf9357e640 1 --2- 172.21.15.32:0/4083403987 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fbf7806be30 0x7fbf7806e2f0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:05.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.419+0000 7fbf9357e640 1 --2- 172.21.15.32:0/4083403987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf8c15cb00 0x7fbf8c159ff0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:05.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.419+0000 7fbf9357e640 1 --2- 172.21.15.32:0/4083403987 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf8c15b380 0x7fbf8c151470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:05.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.419+0000 7fbf9357e640 1 --2- 172.21.15.32:0/4083403987 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf8c0ceb40 0x7fbf8c158970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:05.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.419+0000 7fbf9357e640 1 -- 172.21.15.32:0/4083403987 >> 172.21.15.32:0/4083403987 conn(0x7fbf8c06b110 msgr2=0x7fbf8c15f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:05.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.419+0000 7fbf9357e640 1 -- 172.21.15.32:0/4083403987 shutdown_connections 2024-04-24T23:04:05.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.419+0000 7fbf9357e640 1 -- 172.21.15.32:0/4083403987 wait complete. 2024-04-24T23:04:05.431 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":252,"stamp":"2024-04-24T23:04:05.196714+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":100,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":5509,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1556990,"statfs":{"total":1600000000,"available":1594358410,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.785343"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.439768+0000","last_change":"2024-04-24T23:04:01.439624+0000","last_active":"2024-04-24T23:04:01.439768+0000","last_peered":"2024-04-24T23:04:01.439768+0000","last_clean":"2024-04-24T23:04:01.439768+0000","last_became_active":"2024-04-24T23:04:01.439345+0000","last_became_peered":"2024-04-24T23:04:01.439345+0000","last_unstale":"2024-04-24T23:04:01.439768+0000","last_undegraded":"2024-04-24T23:04:01.439768+0000","last_fullsized":"2024-04-24T23:04:01.439768+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:59.368837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.439396+0000","last_change":"2024-04-24T23:04:01.439230+0000","last_active":"2024-04-24T23:04:01.439396+0000","last_peered":"2024-04-24T23:04:01.439396+0000","last_clean":"2024-04-24T23:04:01.439396+0000","last_became_active":"2024-04-24T23:04:01.438999+0000","last_became_peered":"2024-04-24T23:04:01.438999+0000","last_unstale":"2024-04-24T23:04:01.439396+0000","last_undegraded":"2024-04-24T23:04:01.439396+0000","last_fullsized":"2024-04-24T23:04:01.439396+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:18:48.331110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.446183+0000","last_change":"2024-04-24T23:04:01.446078+0000","last_active":"2024-04-24T23:04:01.446183+0000","last_peered":"2024-04-24T23:04:01.446183+0000","last_clean":"2024-04-24T23:04:01.446183+0000","last_became_active":"2024-04-24T23:04:01.445687+0000","last_became_peered":"2024-04-24T23:04:01.445687+0000","last_unstale":"2024-04-24T23:04:01.446183+0000","last_undegraded":"2024-04-24T23:04:01.446183+0000","last_fullsized":"2024-04-24T23:04:01.446183+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:06:44.287698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.443401+0000","last_change":"2024-04-24T23:04:01.443285+0000","last_active":"2024-04-24T23:04:01.443401+0000","last_peered":"2024-04-24T23:04:01.443401+0000","last_clean":"2024-04-24T23:04:01.443401+0000","last_became_active":"2024-04-24T23:04:01.443033+0000","last_became_peered":"2024-04-24T23:04:01.443033+0000","last_unstale":"2024-04-24T23:04:01.443401+0000","last_undegraded":"2024-04-24T23:04:01.443401+0000","last_fullsized":"2024-04-24T23:04:01.443401+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:54:04.126575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.439061+0000","last_change":"2024-04-24T23:04:01.438932+0000","last_active":"2024-04-24T23:04:01.439061+0000","last_peered":"2024-04-24T23:04:01.439061+0000","last_clean":"2024-04-24T23:04:01.439061+0000","last_became_active":"2024-04-24T23:04:01.438680+0000","last_became_peered":"2024-04-24T23:04:01.438680+0000","last_unstale":"2024-04-24T23:04:01.439061+0000","last_undegraded":"2024-04-24T23:04:01.439061+0000","last_fullsized":"2024-04-24T23:04:01.439061+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:39:56.940903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.444933+0000","last_change":"2024-04-24T23:04:01.444749+0000","last_active":"2024-04-24T23:04:01.444933+0000","last_peered":"2024-04-24T23:04:01.444933+0000","last_clean":"2024-04-24T23:04:01.444933+0000","last_became_active":"2024-04-24T23:04:01.443804+0000","last_became_peered":"2024-04-24T23:04:01.443804+0000","last_unstale":"2024-04-24T23:04:01.444933+0000","last_undegraded":"2024-04-24T23:04:01.444933+0000","last_fullsized":"2024-04-24T23:04:01.444933+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:13.518609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.443843+0000","last_change":"2024-04-24T23:04:01.443243+0000","last_active":"2024-04-24T23:04:01.443843+0000","last_peered":"2024-04-24T23:04:01.443843+0000","last_clean":"2024-04-24T23:04:01.443843+0000","last_became_active":"2024-04-24T23:04:01.443033+0000","last_became_peered":"2024-04-24T23:04:01.443033+0000","last_unstale":"2024-04-24T23:04:01.443843+0000","last_undegraded":"2024-04-24T23:04:01.443843+0000","last_fullsized":"2024-04-24T23:04:01.443843+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:39:54.313053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.434610+0000","last_change":"2024-04-24T23:04:01.434481+0000","last_active":"2024-04-24T23:04:01.434610+0000","last_peered":"2024-04-24T23:04:01.434610+0000","last_clean":"2024-04-24T23:04:01.434610+0000","last_became_active":"2024-04-24T23:04:01.434114+0000","last_became_peered":"2024-04-24T23:04:01.434114+0000","last_unstale":"2024-04-24T23:04:01.434610+0000","last_undegraded":"2024-04-24T23:04:01.434610+0000","last_fullsized":"2024-04-24T23:04:01.434610+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:01:58.242511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.438350+0000","last_change":"2024-04-24T23:04:01.438222+0000","last_active":"2024-04-24T23:04:01.438350+0000","last_peered":"2024-04-24T23:04:01.438350+0000","last_clean":"2024-04-24T23:04:01.438350+0000","last_became_active":"2024-04-24T23:04:01.437639+0000","last_became_peered":"2024-04-24T23:04:01.437639+0000","last_unstale":"2024-04-24T23:04:01.438350+0000","last_undegraded":"2024-04-24T23:04:01.438350+0000","last_fullsized":"2024-04-24T23:04:01.438350+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:09:45.413068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.437805+0000","last_change":"2024-04-24T23:04:01.437693+0000","last_active":"2024-04-24T23:04:01.437805+0000","last_peered":"2024-04-24T23:04:01.437805+0000","last_clean":"2024-04-24T23:04:01.437805+0000","last_became_active":"2024-04-24T23:04:01.437409+0000","last_became_peered":"2024-04-24T23:04:01.437409+0000","last_unstale":"2024-04-24T23:04:01.437805+0000","last_undegraded":"2024-04-24T23:04:01.437805+0000","last_fullsized":"2024-04-24T23:04:01.437805+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:25.922876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.442567+0000","last_change":"2024-04-24T23:04:01.442083+0000","last_active":"2024-04-24T23:04:01.442567+0000","last_peered":"2024-04-24T23:04:01.442567+0000","last_clean":"2024-04-24T23:04:01.442567+0000","last_became_active":"2024-04-24T23:04:01.441474+0000","last_became_peered":"2024-04-24T23:04:01.441474+0000","last_unstale":"2024-04-24T23:04:01.442567+0000","last_undegraded":"2024-04-24T23:04:01.442567+0000","last_fullsized":"2024-04-24T23:04:01.442567+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:49.893907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.447195+0000","last_change":"2024-04-24T23:04:01.446747+0000","last_active":"2024-04-24T23:04:01.447195+0000","last_peered":"2024-04-24T23:04:01.447195+0000","last_clean":"2024-04-24T23:04:01.447195+0000","last_became_active":"2024-04-24T23:04:01.445743+0000","last_became_peered":"2024-04-24T23:04:01.445743+0000","last_unstale":"2024-04-24T23:04:01.447195+0000","last_undegraded":"2024-04-24T23:04:01.447195+0000","last_fullsized":"2024-04-24T23:04:01.447195+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:14:39.847591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.442669+0000","last_change":"2024-04-24T23:04:01.442551+0000","last_active":"2024-04-24T23:04:01.442669+0000","last_peered":"2024-04-24T23:04:01.442669+0000","last_clean":"2024-04-24T23:04:01.442669+0000","last_became_active":"2024-04-24T23:04:01.442011+0000","last_became_peered":"2024-04-24T23:04:01.442011+0000","last_unstale":"2024-04-24T23:04:01.442669+0000","last_undegraded":"2024-04-24T23:04:01.442669+0000","last_fullsized":"2024-04-24T23:04:01.442669+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:26.269220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.440827+0000","last_change":"2024-04-24T23:04:01.440301+0000","last_active":"2024-04-24T23:04:01.440827+0000","last_peered":"2024-04-24T23:04:01.440827+0000","last_clean":"2024-04-24T23:04:01.440827+0000","last_became_active":"2024-04-24T23:04:01.440011+0000","last_became_peered":"2024-04-24T23:04:01.440011+0000","last_unstale":"2024-04-24T23:04:01.440827+0000","last_undegraded":"2024-04-24T23:04:01.440827+0000","last_fullsized":"2024-04-24T23:04:01.440827+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:18:54.054872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.443822+0000","last_change":"2024-04-24T23:04:01.441574+0000","last_active":"2024-04-24T23:04:01.443822+0000","last_peered":"2024-04-24T23:04:01.443822+0000","last_clean":"2024-04-24T23:04:01.443822+0000","last_became_active":"2024-04-24T23:04:01.441305+0000","last_became_peered":"2024-04-24T23:04:01.441305+0000","last_unstale":"2024-04-24T23:04:01.443822+0000","last_undegraded":"2024-04-24T23:04:01.443822+0000","last_fullsized":"2024-04-24T23:04:01.443822+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:37:57.283801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.447833+0000","last_change":"2024-04-24T23:04:01.447695+0000","last_active":"2024-04-24T23:04:01.447833+0000","last_peered":"2024-04-24T23:04:01.447833+0000","last_clean":"2024-04-24T23:04:01.447833+0000","last_became_active":"2024-04-24T23:04:01.447453+0000","last_became_peered":"2024-04-24T23:04:01.447453+0000","last_unstale":"2024-04-24T23:04:01.447833+0000","last_undegraded":"2024-04-24T23:04:01.447833+0000","last_fullsized":"2024-04-24T23:04:01.447833+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:39:02.050156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.446586+0000","last_change":"2024-04-24T23:04:01.445980+0000","last_active":"2024-04-24T23:04:01.446586+0000","last_peered":"2024-04-24T23:04:01.446586+0000","last_clean":"2024-04-24T23:04:01.446586+0000","last_became_active":"2024-04-24T23:04:01.445412+0000","last_became_peered":"2024-04-24T23:04:01.445412+0000","last_unstale":"2024-04-24T23:04:01.446586+0000","last_undegraded":"2024-04-24T23:04:01.446586+0000","last_fullsized":"2024-04-24T23:04:01.446586+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:42:47.271129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.433700+0000","last_change":"2024-04-24T23:04:01.433444+0000","last_active":"2024-04-24T23:04:01.433700+0000","last_peered":"2024-04-24T23:04:01.433700+0000","last_clean":"2024-04-24T23:04:01.433700+0000","last_became_active":"2024-04-24T23:04:01.433094+0000","last_became_peered":"2024-04-24T23:04:01.433094+0000","last_unstale":"2024-04-24T23:04:01.433700+0000","last_undegraded":"2024-04-24T23:04:01.433700+0000","last_fullsized":"2024-04-24T23:04:01.433700+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:28:14.277049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.443053+0000","last_change":"2024-04-24T23:04:01.442672+0000","last_active":"2024-04-24T23:04:01.443053+0000","last_peered":"2024-04-24T23:04:01.443053+0000","last_clean":"2024-04-24T23:04:01.443053+0000","last_became_active":"2024-04-24T23:04:01.442264+0000","last_became_peered":"2024-04-24T23:04:01.442264+0000","last_unstale":"2024-04-24T23:04:01.443053+0000","last_undegraded":"2024-04-24T23:04:01.443053+0000","last_fullsized":"2024-04-24T23:04:01.443053+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:57.766064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.441995+0000","last_change":"2024-04-24T23:04:01.441881+0000","last_active":"2024-04-24T23:04:01.441995+0000","last_peered":"2024-04-24T23:04:01.441995+0000","last_clean":"2024-04-24T23:04:01.441995+0000","last_became_active":"2024-04-24T23:04:01.441244+0000","last_became_peered":"2024-04-24T23:04:01.441244+0000","last_unstale":"2024-04-24T23:04:01.441995+0000","last_undegraded":"2024-04-24T23:04:01.441995+0000","last_fullsized":"2024-04-24T23:04:01.441995+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:25:37.159164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.435271+0000","last_change":"2024-04-24T23:04:01.435125+0000","last_active":"2024-04-24T23:04:01.435271+0000","last_peered":"2024-04-24T23:04:01.435271+0000","last_clean":"2024-04-24T23:04:01.435271+0000","last_became_active":"2024-04-24T23:04:01.434477+0000","last_became_peered":"2024-04-24T23:04:01.434477+0000","last_unstale":"2024-04-24T23:04:01.435271+0000","last_undegraded":"2024-04-24T23:04:01.435271+0000","last_fullsized":"2024-04-24T23:04:01.435271+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:12:54.123074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.446195+0000","last_change":"2024-04-24T23:04:01.446059+0000","last_active":"2024-04-24T23:04:01.446195+0000","last_peered":"2024-04-24T23:04:01.446195+0000","last_clean":"2024-04-24T23:04:01.446195+0000","last_became_active":"2024-04-24T23:04:01.445701+0000","last_became_peered":"2024-04-24T23:04:01.445701+0000","last_unstale":"2024-04-24T23:04:01.446195+0000","last_undegraded":"2024-04-24T23:04:01.446195+0000","last_fullsized":"2024-04-24T23:04:01.446195+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:46:19.706069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.442715+0000","last_change":"2024-04-24T23:04:01.442023+0000","last_active":"2024-04-24T23:04:01.442715+0000","last_peered":"2024-04-24T23:04:01.442715+0000","last_clean":"2024-04-24T23:04:01.442715+0000","last_became_active":"2024-04-24T23:04:01.441742+0000","last_became_peered":"2024-04-24T23:04:01.441742+0000","last_unstale":"2024-04-24T23:04:01.442715+0000","last_undegraded":"2024-04-24T23:04:01.442715+0000","last_fullsized":"2024-04-24T23:04:01.442715+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:08:21.524405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.441838+0000","last_change":"2024-04-24T23:04:01.441669+0000","last_active":"2024-04-24T23:04:01.441838+0000","last_peered":"2024-04-24T23:04:01.441838+0000","last_clean":"2024-04-24T23:04:01.441838+0000","last_became_active":"2024-04-24T23:04:01.439126+0000","last_became_peered":"2024-04-24T23:04:01.439126+0000","last_unstale":"2024-04-24T23:04:01.441838+0000","last_undegraded":"2024-04-24T23:04:01.441838+0000","last_fullsized":"2024-04-24T23:04:01.441838+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:48:04.843640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.432874+0000","last_change":"2024-04-24T23:04:01.432728+0000","last_active":"2024-04-24T23:04:01.432874+0000","last_peered":"2024-04-24T23:04:01.432874+0000","last_clean":"2024-04-24T23:04:01.432874+0000","last_became_active":"2024-04-24T23:04:01.432266+0000","last_became_peered":"2024-04-24T23:04:01.432266+0000","last_unstale":"2024-04-24T23:04:01.432874+0000","last_undegraded":"2024-04-24T23:04:01.432874+0000","last_fullsized":"2024-04-24T23:04:01.432874+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:26:51.472758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.436987+0000","last_change":"2024-04-24T23:04:01.435903+0000","last_active":"2024-04-24T23:04:01.436987+0000","last_peered":"2024-04-24T23:04:01.436987+0000","last_clean":"2024-04-24T23:04:01.436987+0000","last_became_active":"2024-04-24T23:04:01.435640+0000","last_became_peered":"2024-04-24T23:04:01.435640+0000","last_unstale":"2024-04-24T23:04:01.436987+0000","last_undegraded":"2024-04-24T23:04:01.436987+0000","last_fullsized":"2024-04-24T23:04:01.436987+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:19:59.023039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.437218+0000","last_change":"2024-04-24T23:04:01.436705+0000","last_active":"2024-04-24T23:04:01.437218+0000","last_peered":"2024-04-24T23:04:01.437218+0000","last_clean":"2024-04-24T23:04:01.437218+0000","last_became_active":"2024-04-24T23:04:01.436219+0000","last_became_peered":"2024-04-24T23:04:01.436219+0000","last_unstale":"2024-04-24T23:04:01.437218+0000","last_undegraded":"2024-04-24T23:04:01.437218+0000","last_fullsized":"2024-04-24T23:04:01.437218+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:50:03.902587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.445100+0000","last_change":"2024-04-24T23:04:01.444371+0000","last_active":"2024-04-24T23:04:01.445100+0000","last_peered":"2024-04-24T23:04:01.445100+0000","last_clean":"2024-04-24T23:04:01.445100+0000","last_became_active":"2024-04-24T23:04:01.442437+0000","last_became_peered":"2024-04-24T23:04:01.442437+0000","last_unstale":"2024-04-24T23:04:01.445100+0000","last_undegraded":"2024-04-24T23:04:01.445100+0000","last_fullsized":"2024-04-24T23:04:01.445100+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:54:28.694301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.440415+0000","last_change":"2024-04-24T23:04:01.440316+0000","last_active":"2024-04-24T23:04:01.440415+0000","last_peered":"2024-04-24T23:04:01.440415+0000","last_clean":"2024-04-24T23:04:01.440415+0000","last_became_active":"2024-04-24T23:04:01.439710+0000","last_became_peered":"2024-04-24T23:04:01.439710+0000","last_unstale":"2024-04-24T23:04:01.440415+0000","last_undegraded":"2024-04-24T23:04:01.440415+0000","last_fullsized":"2024-04-24T23:04:01.440415+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:48.560864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.441768+0000","last_change":"2024-04-24T23:04:01.441622+0000","last_active":"2024-04-24T23:04:01.441768+0000","last_peered":"2024-04-24T23:04:01.441768+0000","last_clean":"2024-04-24T23:04:01.441768+0000","last_became_active":"2024-04-24T23:04:01.440954+0000","last_became_peered":"2024-04-24T23:04:01.440954+0000","last_unstale":"2024-04-24T23:04:01.441768+0000","last_undegraded":"2024-04-24T23:04:01.441768+0000","last_fullsized":"2024-04-24T23:04:01.441768+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:48:17.112726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.444514+0000","last_change":"2024-04-24T23:04:01.444342+0000","last_active":"2024-04-24T23:04:01.444514+0000","last_peered":"2024-04-24T23:04:01.444514+0000","last_clean":"2024-04-24T23:04:01.444514+0000","last_became_active":"2024-04-24T23:04:01.443485+0000","last_became_peered":"2024-04-24T23:04:01.443485+0000","last_unstale":"2024-04-24T23:04:01.444514+0000","last_undegraded":"2024-04-24T23:04:01.444514+0000","last_fullsized":"2024-04-24T23:04:01.444514+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:17:38.086016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.446927+0000","last_change":"2024-04-24T23:04:01.446631+0000","last_active":"2024-04-24T23:04:01.446927+0000","last_peered":"2024-04-24T23:04:01.446927+0000","last_clean":"2024-04-24T23:04:01.446927+0000","last_became_active":"2024-04-24T23:04:01.444554+0000","last_became_peered":"2024-04-24T23:04:01.444554+0000","last_unstale":"2024-04-24T23:04:01.446927+0000","last_undegraded":"2024-04-24T23:04:01.446927+0000","last_fullsized":"2024-04-24T23:04:01.446927+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:54.200388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.433877+0000","last_change":"2024-04-24T23:04:01.432857+0000","last_active":"2024-04-24T23:04:01.433877+0000","last_peered":"2024-04-24T23:04:01.433877+0000","last_clean":"2024-04-24T23:04:01.433877+0000","last_became_active":"2024-04-24T23:04:01.432621+0000","last_became_peered":"2024-04-24T23:04:01.432621+0000","last_unstale":"2024-04-24T23:04:01.433877+0000","last_undegraded":"2024-04-24T23:04:01.433877+0000","last_fullsized":"2024-04-24T23:04:01.433877+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:27:15.385203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"103'4","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.452299+0000","last_change":"2024-04-24T23:04:00.420780+0000","last_active":"2024-04-24T23:04:01.452299+0000","last_peered":"2024-04-24T23:04:01.452299+0000","last_clean":"2024-04-24T23:04:01.452299+0000","last_became_active":"2024-04-24T23:04:00.420491+0000","last_became_peered":"2024-04-24T23:04:00.420491+0000","last_unstale":"2024-04-24T23:04:01.452299+0000","last_undegraded":"2024-04-24T23:04:01.452299+0000","last_fullsized":"2024-04-24T23:04:01.452299+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:13.110238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.443281+0000","last_change":"2024-04-24T23:04:01.442675+0000","last_active":"2024-04-24T23:04:01.443281+0000","last_peered":"2024-04-24T23:04:01.443281+0000","last_clean":"2024-04-24T23:04:01.443281+0000","last_became_active":"2024-04-24T23:04:01.441778+0000","last_became_peered":"2024-04-24T23:04:01.441778+0000","last_unstale":"2024-04-24T23:04:01.443281+0000","last_undegraded":"2024-04-24T23:04:01.443281+0000","last_fullsized":"2024-04-24T23:04:01.443281+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:40:59.394072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"103'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.453265+0000","last_change":"2024-04-24T23:04:00.421631+0000","last_active":"2024-04-24T23:04:01.453265+0000","last_peered":"2024-04-24T23:04:01.453265+0000","last_clean":"2024-04-24T23:04:01.453265+0000","last_became_active":"2024-04-24T23:04:00.421382+0000","last_became_peered":"2024-04-24T23:04:00.421382+0000","last_unstale":"2024-04-24T23:04:01.453265+0000","last_undegraded":"2024-04-24T23:04:01.453265+0000","last_fullsized":"2024-04-24T23:04:01.453265+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:08:38.371647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.437447+0000","last_change":"2024-04-24T23:04:01.436247+0000","last_active":"2024-04-24T23:04:01.437447+0000","last_peered":"2024-04-24T23:04:01.437447+0000","last_clean":"2024-04-24T23:04:01.437447+0000","last_became_active":"2024-04-24T23:04:01.434655+0000","last_became_peered":"2024-04-24T23:04:01.434655+0000","last_unstale":"2024-04-24T23:04:01.437447+0000","last_undegraded":"2024-04-24T23:04:01.437447+0000","last_fullsized":"2024-04-24T23:04:01.437447+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:51:58.897857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"103'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.451511+0000","last_change":"2024-04-24T23:04:00.424522+0000","last_active":"2024-04-24T23:04:01.451511+0000","last_peered":"2024-04-24T23:04:01.451511+0000","last_clean":"2024-04-24T23:04:01.451511+0000","last_became_active":"2024-04-24T23:04:00.424168+0000","last_became_peered":"2024-04-24T23:04:00.424168+0000","last_unstale":"2024-04-24T23:04:01.451511+0000","last_undegraded":"2024-04-24T23:04:01.451511+0000","last_fullsized":"2024-04-24T23:04:01.451511+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:39.737703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.440170+0000","last_change":"2024-04-24T23:04:01.440036+0000","last_active":"2024-04-24T23:04:01.440170+0000","last_peered":"2024-04-24T23:04:01.440170+0000","last_clean":"2024-04-24T23:04:01.440170+0000","last_became_active":"2024-04-24T23:04:01.439005+0000","last_became_peered":"2024-04-24T23:04:01.439005+0000","last_unstale":"2024-04-24T23:04:01.440170+0000","last_undegraded":"2024-04-24T23:04:01.440170+0000","last_fullsized":"2024-04-24T23:04:01.440170+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:08:37.486298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"103'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.451081+0000","last_change":"2024-04-24T23:04:00.425085+0000","last_active":"2024-04-24T23:04:01.451081+0000","last_peered":"2024-04-24T23:04:01.451081+0000","last_clean":"2024-04-24T23:04:01.451081+0000","last_became_active":"2024-04-24T23:04:00.424750+0000","last_became_peered":"2024-04-24T23:04:00.424750+0000","last_unstale":"2024-04-24T23:04:01.451081+0000","last_undegraded":"2024-04-24T23:04:01.451081+0000","last_fullsized":"2024-04-24T23:04:01.451081+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:15:01.161630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.439121+0000","last_change":"2024-04-24T23:04:01.436590+0000","last_active":"2024-04-24T23:04:01.439121+0000","last_peered":"2024-04-24T23:04:01.439121+0000","last_clean":"2024-04-24T23:04:01.439121+0000","last_became_active":"2024-04-24T23:04:01.436367+0000","last_became_peered":"2024-04-24T23:04:01.436367+0000","last_unstale":"2024-04-24T23:04:01.439121+0000","last_undegraded":"2024-04-24T23:04:01.439121+0000","last_fullsized":"2024-04-24T23:04:01.439121+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:15.053717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"103'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.452087+0000","last_change":"2024-04-24T23:04:00.427288+0000","last_active":"2024-04-24T23:04:01.452087+0000","last_peered":"2024-04-24T23:04:01.452087+0000","last_clean":"2024-04-24T23:04:01.452087+0000","last_became_active":"2024-04-24T23:04:00.427047+0000","last_became_peered":"2024-04-24T23:04:00.427047+0000","last_unstale":"2024-04-24T23:04:01.452087+0000","last_undegraded":"2024-04-24T23:04:01.452087+0000","last_fullsized":"2024-04-24T23:04:01.452087+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:10:55.535313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.447406+0000","last_change":"2024-04-24T23:04:01.447092+0000","last_active":"2024-04-24T23:04:01.447406+0000","last_peered":"2024-04-24T23:04:01.447406+0000","last_clean":"2024-04-24T23:04:01.447406+0000","last_became_active":"2024-04-24T23:04:01.444885+0000","last_became_peered":"2024-04-24T23:04:01.444885+0000","last_unstale":"2024-04-24T23:04:01.447406+0000","last_undegraded":"2024-04-24T23:04:01.447406+0000","last_fullsized":"2024-04-24T23:04:01.447406+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:35:39.696962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"103'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.453176+0000","last_change":"2024-04-24T23:04:00.428309+0000","last_active":"2024-04-24T23:04:01.453176+0000","last_peered":"2024-04-24T23:04:01.453176+0000","last_clean":"2024-04-24T23:04:01.453176+0000","last_became_active":"2024-04-24T23:04:00.427969+0000","last_became_peered":"2024-04-24T23:04:00.427969+0000","last_unstale":"2024-04-24T23:04:01.453176+0000","last_undegraded":"2024-04-24T23:04:01.453176+0000","last_fullsized":"2024-04-24T23:04:01.453176+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:00:45.798299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.439844+0000","last_change":"2024-04-24T23:04:01.439730+0000","last_active":"2024-04-24T23:04:01.439844+0000","last_peered":"2024-04-24T23:04:01.439844+0000","last_clean":"2024-04-24T23:04:01.439844+0000","last_became_active":"2024-04-24T23:04:01.438529+0000","last_became_peered":"2024-04-24T23:04:01.438529+0000","last_unstale":"2024-04-24T23:04:01.439844+0000","last_undegraded":"2024-04-24T23:04:01.439844+0000","last_fullsized":"2024-04-24T23:04:01.439844+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:02.275002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.438944+0000","last_change":"2024-04-24T23:04:00.423871+0000","last_active":"2024-04-24T23:04:01.438944+0000","last_peered":"2024-04-24T23:04:01.438944+0000","last_clean":"2024-04-24T23:04:01.438944+0000","last_became_active":"2024-04-24T23:04:00.423635+0000","last_became_peered":"2024-04-24T23:04:00.423635+0000","last_unstale":"2024-04-24T23:04:01.438944+0000","last_undegraded":"2024-04-24T23:04:01.438944+0000","last_fullsized":"2024-04-24T23:04:01.438944+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:18:21.732139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.445668+0000","last_change":"2024-04-24T23:04:01.445562+0000","last_active":"2024-04-24T23:04:01.445668+0000","last_peered":"2024-04-24T23:04:01.445668+0000","last_clean":"2024-04-24T23:04:01.445668+0000","last_became_active":"2024-04-24T23:04:01.445307+0000","last_became_peered":"2024-04-24T23:04:01.445307+0000","last_unstale":"2024-04-24T23:04:01.445668+0000","last_undegraded":"2024-04-24T23:04:01.445668+0000","last_fullsized":"2024-04-24T23:04:01.445668+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:06:54.614918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.437734+0000","last_change":"2024-04-24T23:04:00.425623+0000","last_active":"2024-04-24T23:04:01.437734+0000","last_peered":"2024-04-24T23:04:01.437734+0000","last_clean":"2024-04-24T23:04:01.437734+0000","last_became_active":"2024-04-24T23:04:00.425153+0000","last_became_peered":"2024-04-24T23:04:00.425153+0000","last_unstale":"2024-04-24T23:04:01.437734+0000","last_undegraded":"2024-04-24T23:04:01.437734+0000","last_fullsized":"2024-04-24T23:04:01.437734+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:40.735104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.445377+0000","last_change":"2024-04-24T23:04:01.444449+0000","last_active":"2024-04-24T23:04:01.445377+0000","last_peered":"2024-04-24T23:04:01.445377+0000","last_clean":"2024-04-24T23:04:01.445377+0000","last_became_active":"2024-04-24T23:04:01.444234+0000","last_became_peered":"2024-04-24T23:04:01.444234+0000","last_unstale":"2024-04-24T23:04:01.445377+0000","last_undegraded":"2024-04-24T23:04:01.445377+0000","last_fullsized":"2024-04-24T23:04:01.445377+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:18.408660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.434197+0000","last_change":"2024-04-24T23:04:00.422891+0000","last_active":"2024-04-24T23:04:01.434197+0000","last_peered":"2024-04-24T23:04:01.434197+0000","last_clean":"2024-04-24T23:04:01.434197+0000","last_became_active":"2024-04-24T23:04:00.422654+0000","last_became_peered":"2024-04-24T23:04:00.422654+0000","last_unstale":"2024-04-24T23:04:01.434197+0000","last_undegraded":"2024-04-24T23:04:01.434197+0000","last_fullsized":"2024-04-24T23:04:01.434197+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:07.066782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.443658+0000","last_change":"2024-04-24T23:04:01.443506+0000","last_active":"2024-04-24T23:04:01.443658+0000","last_peered":"2024-04-24T23:04:01.443658+0000","last_clean":"2024-04-24T23:04:01.443658+0000","last_became_active":"2024-04-24T23:04:01.442780+0000","last_became_peered":"2024-04-24T23:04:01.442780+0000","last_unstale":"2024-04-24T23:04:01.443658+0000","last_undegraded":"2024-04-24T23:04:01.443658+0000","last_fullsized":"2024-04-24T23:04:01.443658+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:12:53.050406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.433839+0000","last_change":"2024-04-24T23:04:00.421012+0000","last_active":"2024-04-24T23:04:01.433839+0000","last_peered":"2024-04-24T23:04:01.433839+0000","last_clean":"2024-04-24T23:04:01.433839+0000","last_became_active":"2024-04-24T23:04:00.420711+0000","last_became_peered":"2024-04-24T23:04:00.420711+0000","last_unstale":"2024-04-24T23:04:01.433839+0000","last_undegraded":"2024-04-24T23:04:01.433839+0000","last_fullsized":"2024-04-24T23:04:01.433839+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:04:04.004076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.441817+0000","last_change":"2024-04-24T23:04:01.441469+0000","last_active":"2024-04-24T23:04:01.441817+0000","last_peered":"2024-04-24T23:04:01.441817+0000","last_clean":"2024-04-24T23:04:01.441817+0000","last_became_active":"2024-04-24T23:04:01.437858+0000","last_became_peered":"2024-04-24T23:04:01.437858+0000","last_unstale":"2024-04-24T23:04:01.441817+0000","last_undegraded":"2024-04-24T23:04:01.441817+0000","last_fullsized":"2024-04-24T23:04:01.441817+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:35.502849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"103'3","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.452388+0000","last_change":"2024-04-24T23:04:00.427467+0000","last_active":"2024-04-24T23:04:01.452388+0000","last_peered":"2024-04-24T23:04:01.452388+0000","last_clean":"2024-04-24T23:04:01.452388+0000","last_became_active":"2024-04-24T23:04:00.426744+0000","last_became_peered":"2024-04-24T23:04:00.426744+0000","last_unstale":"2024-04-24T23:04:01.452388+0000","last_undegraded":"2024-04-24T23:04:01.452388+0000","last_fullsized":"2024-04-24T23:04:01.452388+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:15:53.179365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.441226+0000","last_change":"2024-04-24T23:04:01.441077+0000","last_active":"2024-04-24T23:04:01.441226+0000","last_peered":"2024-04-24T23:04:01.441226+0000","last_clean":"2024-04-24T23:04:01.441226+0000","last_became_active":"2024-04-24T23:04:01.436902+0000","last_became_peered":"2024-04-24T23:04:01.436902+0000","last_unstale":"2024-04-24T23:04:01.441226+0000","last_undegraded":"2024-04-24T23:04:01.441226+0000","last_fullsized":"2024-04-24T23:04:01.441226+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:37:52.956621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"103'6","reported_seq":18,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.454461+0000","last_change":"2024-04-24T23:04:00.426128+0000","last_active":"2024-04-24T23:04:01.454461+0000","last_peered":"2024-04-24T23:04:01.454461+0000","last_clean":"2024-04-24T23:04:01.454461+0000","last_became_active":"2024-04-24T23:04:00.425847+0000","last_became_peered":"2024-04-24T23:04:00.425847+0000","last_unstale":"2024-04-24T23:04:01.454461+0000","last_undegraded":"2024-04-24T23:04:01.454461+0000","last_fullsized":"2024-04-24T23:04:01.454461+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:10.460002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.441787+0000","last_change":"2024-04-24T23:04:01.441143+0000","last_active":"2024-04-24T23:04:01.441787+0000","last_peered":"2024-04-24T23:04:01.441787+0000","last_clean":"2024-04-24T23:04:01.441787+0000","last_became_active":"2024-04-24T23:04:01.439771+0000","last_became_peered":"2024-04-24T23:04:01.439771+0000","last_unstale":"2024-04-24T23:04:01.441787+0000","last_undegraded":"2024-04-24T23:04:01.441787+0000","last_fullsized":"2024-04-24T23:04:01.441787+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:04:14.106634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"103'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.451056+0000","last_change":"2024-04-24T23:04:00.425812+0000","last_active":"2024-04-24T23:04:01.451056+0000","last_peered":"2024-04-24T23:04:01.451056+0000","last_clean":"2024-04-24T23:04:01.451056+0000","last_became_active":"2024-04-24T23:04:00.425544+0000","last_became_peered":"2024-04-24T23:04:00.425544+0000","last_unstale":"2024-04-24T23:04:01.451056+0000","last_undegraded":"2024-04-24T23:04:01.451056+0000","last_fullsized":"2024-04-24T23:04:01.451056+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:43.816546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.439522+0000","last_change":"2024-04-24T23:04:01.439362+0000","last_active":"2024-04-24T23:04:01.439522+0000","last_peered":"2024-04-24T23:04:01.439522+0000","last_clean":"2024-04-24T23:04:01.439522+0000","last_became_active":"2024-04-24T23:04:01.438026+0000","last_became_peered":"2024-04-24T23:04:01.438026+0000","last_unstale":"2024-04-24T23:04:01.439522+0000","last_undegraded":"2024-04-24T23:04:01.439522+0000","last_fullsized":"2024-04-24T23:04:01.439522+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:32.274596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.433045+0000","last_change":"2024-04-24T23:04:00.422482+0000","last_active":"2024-04-24T23:04:01.433045+0000","last_peered":"2024-04-24T23:04:01.433045+0000","last_clean":"2024-04-24T23:04:01.433045+0000","last_became_active":"2024-04-24T23:04:00.422217+0000","last_became_peered":"2024-04-24T23:04:00.422217+0000","last_unstale":"2024-04-24T23:04:01.433045+0000","last_undegraded":"2024-04-24T23:04:01.433045+0000","last_fullsized":"2024-04-24T23:04:01.433045+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:46:50.571382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.444473+0000","last_change":"2024-04-24T23:04:01.444370+0000","last_active":"2024-04-24T23:04:01.444473+0000","last_peered":"2024-04-24T23:04:01.444473+0000","last_clean":"2024-04-24T23:04:01.444473+0000","last_became_active":"2024-04-24T23:04:01.444132+0000","last_became_peered":"2024-04-24T23:04:01.444132+0000","last_unstale":"2024-04-24T23:04:01.444473+0000","last_undegraded":"2024-04-24T23:04:01.444473+0000","last_fullsized":"2024-04-24T23:04:01.444473+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:25:39.221241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.433988+0000","last_change":"2024-04-24T23:04:00.417450+0000","last_active":"2024-04-24T23:04:01.433988+0000","last_peered":"2024-04-24T23:04:01.433988+0000","last_clean":"2024-04-24T23:04:01.433988+0000","last_became_active":"2024-04-24T23:04:00.417215+0000","last_became_peered":"2024-04-24T23:04:00.417215+0000","last_unstale":"2024-04-24T23:04:01.433988+0000","last_undegraded":"2024-04-24T23:04:01.433988+0000","last_fullsized":"2024-04-24T23:04:01.433988+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:28.317728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.436963+0000","last_change":"2024-04-24T23:04:01.436739+0000","last_active":"2024-04-24T23:04:01.436963+0000","last_peered":"2024-04-24T23:04:01.436963+0000","last_clean":"2024-04-24T23:04:01.436963+0000","last_became_active":"2024-04-24T23:04:01.436487+0000","last_became_peered":"2024-04-24T23:04:01.436487+0000","last_unstale":"2024-04-24T23:04:01.436963+0000","last_undegraded":"2024-04-24T23:04:01.436963+0000","last_fullsized":"2024-04-24T23:04:01.436963+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:13:46.686701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"103'8","reported_seq":20,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.455259+0000","last_change":"2024-04-24T23:04:00.417399+0000","last_active":"2024-04-24T23:04:01.455259+0000","last_peered":"2024-04-24T23:04:01.455259+0000","last_clean":"2024-04-24T23:04:01.455259+0000","last_became_active":"2024-04-24T23:04:00.417112+0000","last_became_peered":"2024-04-24T23:04:00.417112+0000","last_unstale":"2024-04-24T23:04:01.455259+0000","last_undegraded":"2024-04-24T23:04:01.455259+0000","last_fullsized":"2024-04-24T23:04:01.455259+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:33:35.187214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.438111+0000","last_change":"2024-04-24T23:04:01.437962+0000","last_active":"2024-04-24T23:04:01.438111+0000","last_peered":"2024-04-24T23:04:01.438111+0000","last_clean":"2024-04-24T23:04:01.438111+0000","last_became_active":"2024-04-24T23:04:01.437711+0000","last_became_peered":"2024-04-24T23:04:01.437711+0000","last_unstale":"2024-04-24T23:04:01.438111+0000","last_undegraded":"2024-04-24T23:04:01.438111+0000","last_fullsized":"2024-04-24T23:04:01.438111+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:52:56.434359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"103'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.451101+0000","last_change":"2024-04-24T23:04:00.427302+0000","last_active":"2024-04-24T23:04:01.451101+0000","last_peered":"2024-04-24T23:04:01.451101+0000","last_clean":"2024-04-24T23:04:01.451101+0000","last_became_active":"2024-04-24T23:04:00.425742+0000","last_became_peered":"2024-04-24T23:04:00.425742+0000","last_unstale":"2024-04-24T23:04:01.451101+0000","last_undegraded":"2024-04-24T23:04:01.451101+0000","last_fullsized":"2024-04-24T23:04:01.451101+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:10:30.189368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.445993+0000","last_change":"2024-04-24T23:04:01.445862+0000","last_active":"2024-04-24T23:04:01.445993+0000","last_peered":"2024-04-24T23:04:01.445993+0000","last_clean":"2024-04-24T23:04:01.445993+0000","last_became_active":"2024-04-24T23:04:01.445603+0000","last_became_peered":"2024-04-24T23:04:01.445603+0000","last_unstale":"2024-04-24T23:04:01.445993+0000","last_undegraded":"2024-04-24T23:04:01.445993+0000","last_fullsized":"2024-04-24T23:04:01.445993+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:51:52.447452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"103'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.451619+0000","last_change":"2024-04-24T23:04:00.424791+0000","last_active":"2024-04-24T23:04:01.451619+0000","last_peered":"2024-04-24T23:04:01.451619+0000","last_clean":"2024-04-24T23:04:01.451619+0000","last_became_active":"2024-04-24T23:04:00.424552+0000","last_became_peered":"2024-04-24T23:04:00.424552+0000","last_unstale":"2024-04-24T23:04:01.451619+0000","last_undegraded":"2024-04-24T23:04:01.451619+0000","last_fullsized":"2024-04-24T23:04:01.451619+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:32:42.658581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.437468+0000","last_change":"2024-04-24T23:04:01.436772+0000","last_active":"2024-04-24T23:04:01.437468+0000","last_peered":"2024-04-24T23:04:01.437468+0000","last_clean":"2024-04-24T23:04:01.437468+0000","last_became_active":"2024-04-24T23:04:01.436545+0000","last_became_peered":"2024-04-24T23:04:01.436545+0000","last_unstale":"2024-04-24T23:04:01.437468+0000","last_undegraded":"2024-04-24T23:04:01.437468+0000","last_fullsized":"2024-04-24T23:04:01.437468+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:17:28.904528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.440745+0000","last_change":"2024-04-24T23:04:00.421402+0000","last_active":"2024-04-24T23:04:01.440745+0000","last_peered":"2024-04-24T23:04:01.440745+0000","last_clean":"2024-04-24T23:04:01.440745+0000","last_became_active":"2024-04-24T23:04:00.421153+0000","last_became_peered":"2024-04-24T23:04:00.421153+0000","last_unstale":"2024-04-24T23:04:01.440745+0000","last_undegraded":"2024-04-24T23:04:01.440745+0000","last_fullsized":"2024-04-24T23:04:01.440745+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:53:16.507098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.437268+0000","last_change":"2024-04-24T23:04:01.436904+0000","last_active":"2024-04-24T23:04:01.437268+0000","last_peered":"2024-04-24T23:04:01.437268+0000","last_clean":"2024-04-24T23:04:01.437268+0000","last_became_active":"2024-04-24T23:04:01.436558+0000","last_became_peered":"2024-04-24T23:04:01.436558+0000","last_unstale":"2024-04-24T23:04:01.437268+0000","last_undegraded":"2024-04-24T23:04:01.437268+0000","last_fullsized":"2024-04-24T23:04:01.437268+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:02.112969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"103'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.451274+0000","last_change":"2024-04-24T23:04:00.423493+0000","last_active":"2024-04-24T23:04:01.451274+0000","last_peered":"2024-04-24T23:04:01.451274+0000","last_clean":"2024-04-24T23:04:01.451274+0000","last_became_active":"2024-04-24T23:04:00.423208+0000","last_became_peered":"2024-04-24T23:04:00.423208+0000","last_unstale":"2024-04-24T23:04:01.451274+0000","last_undegraded":"2024-04-24T23:04:01.451274+0000","last_fullsized":"2024-04-24T23:04:01.451274+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:52:05.454025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.441384+0000","last_change":"2024-04-24T23:04:01.441272+0000","last_active":"2024-04-24T23:04:01.441384+0000","last_peered":"2024-04-24T23:04:01.441384+0000","last_clean":"2024-04-24T23:04:01.441384+0000","last_became_active":"2024-04-24T23:04:01.439999+0000","last_became_peered":"2024-04-24T23:04:01.439999+0000","last_unstale":"2024-04-24T23:04:01.441384+0000","last_undegraded":"2024-04-24T23:04:01.441384+0000","last_fullsized":"2024-04-24T23:04:01.441384+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:32:47.278398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.434745+0000","last_change":"2024-04-24T23:04:00.421533+0000","last_active":"2024-04-24T23:04:01.434745+0000","last_peered":"2024-04-24T23:04:01.434745+0000","last_clean":"2024-04-24T23:04:01.434745+0000","last_became_active":"2024-04-24T23:04:00.421314+0000","last_became_peered":"2024-04-24T23:04:00.421314+0000","last_unstale":"2024-04-24T23:04:01.434745+0000","last_undegraded":"2024-04-24T23:04:01.434745+0000","last_fullsized":"2024-04-24T23:04:01.434745+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:24.258166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.435967+0000","last_change":"2024-04-24T23:04:01.435838+0000","last_active":"2024-04-24T23:04:01.435967+0000","last_peered":"2024-04-24T23:04:01.435967+0000","last_clean":"2024-04-24T23:04:01.435967+0000","last_became_active":"2024-04-24T23:04:01.435492+0000","last_became_peered":"2024-04-24T23:04:01.435492+0000","last_unstale":"2024-04-24T23:04:01.435967+0000","last_undegraded":"2024-04-24T23:04:01.435967+0000","last_fullsized":"2024-04-24T23:04:01.435967+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:55:23.490533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.430108+0000","last_change":"2024-04-24T23:04:00.420668+0000","last_active":"2024-04-24T23:04:01.430108+0000","last_peered":"2024-04-24T23:04:01.430108+0000","last_clean":"2024-04-24T23:04:01.430108+0000","last_became_active":"2024-04-24T23:04:00.420390+0000","last_became_peered":"2024-04-24T23:04:00.420390+0000","last_unstale":"2024-04-24T23:04:01.430108+0000","last_undegraded":"2024-04-24T23:04:01.430108+0000","last_fullsized":"2024-04-24T23:04:01.430108+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:00.214334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.434754+0000","last_change":"2024-04-24T23:04:01.434622+0000","last_active":"2024-04-24T23:04:01.434754+0000","last_peered":"2024-04-24T23:04:01.434754+0000","last_clean":"2024-04-24T23:04:01.434754+0000","last_became_active":"2024-04-24T23:04:01.434369+0000","last_became_peered":"2024-04-24T23:04:01.434369+0000","last_unstale":"2024-04-24T23:04:01.434754+0000","last_undegraded":"2024-04-24T23:04:01.434754+0000","last_fullsized":"2024-04-24T23:04:01.434754+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:09:39.366875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.435290+0000","last_change":"2024-04-24T23:04:00.425589+0000","last_active":"2024-04-24T23:04:01.435290+0000","last_peered":"2024-04-24T23:04:01.435290+0000","last_clean":"2024-04-24T23:04:01.435290+0000","last_became_active":"2024-04-24T23:04:00.425318+0000","last_became_peered":"2024-04-24T23:04:00.425318+0000","last_unstale":"2024-04-24T23:04:01.435290+0000","last_undegraded":"2024-04-24T23:04:01.435290+0000","last_fullsized":"2024-04-24T23:04:01.435290+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:56:55.854071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.442428+0000","last_change":"2024-04-24T23:04:01.442263+0000","last_active":"2024-04-24T23:04:01.442428+0000","last_peered":"2024-04-24T23:04:01.442428+0000","last_clean":"2024-04-24T23:04:01.442428+0000","last_became_active":"2024-04-24T23:04:01.441680+0000","last_became_peered":"2024-04-24T23:04:01.441680+0000","last_unstale":"2024-04-24T23:04:01.442428+0000","last_undegraded":"2024-04-24T23:04:01.442428+0000","last_fullsized":"2024-04-24T23:04:01.442428+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:23:41.534098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"103'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.453246+0000","last_change":"2024-04-24T23:04:00.423120+0000","last_active":"2024-04-24T23:04:01.453246+0000","last_peered":"2024-04-24T23:04:01.453246+0000","last_clean":"2024-04-24T23:04:01.453246+0000","last_became_active":"2024-04-24T23:04:00.422772+0000","last_became_peered":"2024-04-24T23:04:00.422772+0000","last_unstale":"2024-04-24T23:04:01.453246+0000","last_undegraded":"2024-04-24T23:04:01.453246+0000","last_fullsized":"2024-04-24T23:04:01.453246+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:08:54.119256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.437015+0000","last_change":"2024-04-24T23:04:01.436508+0000","last_active":"2024-04-24T23:04:01.437015+0000","last_peered":"2024-04-24T23:04:01.437015+0000","last_clean":"2024-04-24T23:04:01.437015+0000","last_became_active":"2024-04-24T23:04:01.435455+0000","last_became_peered":"2024-04-24T23:04:01.435455+0000","last_unstale":"2024-04-24T23:04:01.437015+0000","last_undegraded":"2024-04-24T23:04:01.437015+0000","last_fullsized":"2024-04-24T23:04:01.437015+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:29:05.440823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.430461+0000","last_change":"2024-04-24T23:04:00.419277+0000","last_active":"2024-04-24T23:04:01.430461+0000","last_peered":"2024-04-24T23:04:01.430461+0000","last_clean":"2024-04-24T23:04:01.430461+0000","last_became_active":"2024-04-24T23:04:00.418982+0000","last_became_peered":"2024-04-24T23:04:00.418982+0000","last_unstale":"2024-04-24T23:04:01.430461+0000","last_undegraded":"2024-04-24T23:04:01.430461+0000","last_fullsized":"2024-04-24T23:04:01.430461+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:14:39.381967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.437240+0000","last_change":"2024-04-24T23:04:01.436680+0000","last_active":"2024-04-24T23:04:01.437240+0000","last_peered":"2024-04-24T23:04:01.437240+0000","last_clean":"2024-04-24T23:04:01.437240+0000","last_became_active":"2024-04-24T23:04:01.436417+0000","last_became_peered":"2024-04-24T23:04:01.436417+0000","last_unstale":"2024-04-24T23:04:01.437240+0000","last_undegraded":"2024-04-24T23:04:01.437240+0000","last_fullsized":"2024-04-24T23:04:01.437240+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:27.715921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.430176+0000","last_change":"2024-04-24T23:04:00.425178+0000","last_active":"2024-04-24T23:04:01.430176+0000","last_peered":"2024-04-24T23:04:01.430176+0000","last_clean":"2024-04-24T23:04:01.430176+0000","last_became_active":"2024-04-24T23:04:00.424823+0000","last_became_peered":"2024-04-24T23:04:00.424823+0000","last_unstale":"2024-04-24T23:04:01.430176+0000","last_undegraded":"2024-04-24T23:04:01.430176+0000","last_fullsized":"2024-04-24T23:04:01.430176+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:48:08.435444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.441031+0000","last_change":"2024-04-24T23:04:01.440874+0000","last_active":"2024-04-24T23:04:01.441031+0000","last_peered":"2024-04-24T23:04:01.441031+0000","last_clean":"2024-04-24T23:04:01.441031+0000","last_became_active":"2024-04-24T23:04:01.440612+0000","last_became_peered":"2024-04-24T23:04:01.440612+0000","last_unstale":"2024-04-24T23:04:01.441031+0000","last_undegraded":"2024-04-24T23:04:01.441031+0000","last_fullsized":"2024-04-24T23:04:01.441031+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:06.229980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.433230+0000","last_change":"2024-04-24T23:04:00.424131+0000","last_active":"2024-04-24T23:04:01.433230+0000","last_peered":"2024-04-24T23:04:01.433230+0000","last_clean":"2024-04-24T23:04:01.433230+0000","last_became_active":"2024-04-24T23:04:00.423892+0000","last_became_peered":"2024-04-24T23:04:00.423892+0000","last_unstale":"2024-04-24T23:04:01.433230+0000","last_undegraded":"2024-04-24T23:04:01.433230+0000","last_fullsized":"2024-04-24T23:04:01.433230+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:42:06.589207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.441443+0000","last_change":"2024-04-24T23:04:01.441284+0000","last_active":"2024-04-24T23:04:01.441443+0000","last_peered":"2024-04-24T23:04:01.441443+0000","last_clean":"2024-04-24T23:04:01.441443+0000","last_became_active":"2024-04-24T23:04:01.440502+0000","last_became_peered":"2024-04-24T23:04:01.440502+0000","last_unstale":"2024-04-24T23:04:01.441443+0000","last_undegraded":"2024-04-24T23:04:01.441443+0000","last_fullsized":"2024-04-24T23:04:01.441443+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:44:18.526320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.433205+0000","last_change":"2024-04-24T23:04:00.420535+0000","last_active":"2024-04-24T23:04:01.433205+0000","last_peered":"2024-04-24T23:04:01.433205+0000","last_clean":"2024-04-24T23:04:01.433205+0000","last_became_active":"2024-04-24T23:04:00.420297+0000","last_became_peered":"2024-04-24T23:04:00.420297+0000","last_unstale":"2024-04-24T23:04:01.433205+0000","last_undegraded":"2024-04-24T23:04:01.433205+0000","last_fullsized":"2024-04-24T23:04:01.433205+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:08:19.176173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.441898+0000","last_change":"2024-04-24T23:04:01.441769+0000","last_active":"2024-04-24T23:04:01.441898+0000","last_peered":"2024-04-24T23:04:01.441898+0000","last_clean":"2024-04-24T23:04:01.441898+0000","last_became_active":"2024-04-24T23:04:01.440669+0000","last_became_peered":"2024-04-24T23:04:01.440669+0000","last_unstale":"2024-04-24T23:04:01.441898+0000","last_undegraded":"2024-04-24T23:04:01.441898+0000","last_fullsized":"2024-04-24T23:04:01.441898+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:11:48.038878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.434134+0000","last_change":"2024-04-24T23:04:00.422871+0000","last_active":"2024-04-24T23:04:01.434134+0000","last_peered":"2024-04-24T23:04:01.434134+0000","last_clean":"2024-04-24T23:04:01.434134+0000","last_became_active":"2024-04-24T23:04:00.422597+0000","last_became_peered":"2024-04-24T23:04:00.422597+0000","last_unstale":"2024-04-24T23:04:01.434134+0000","last_undegraded":"2024-04-24T23:04:01.434134+0000","last_fullsized":"2024-04-24T23:04:01.434134+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:36:02.204842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.438595+0000","last_change":"2024-04-24T23:04:01.438478+0000","last_active":"2024-04-24T23:04:01.438595+0000","last_peered":"2024-04-24T23:04:01.438595+0000","last_clean":"2024-04-24T23:04:01.438595+0000","last_became_active":"2024-04-24T23:04:01.437979+0000","last_became_peered":"2024-04-24T23:04:01.437979+0000","last_unstale":"2024-04-24T23:04:01.438595+0000","last_undegraded":"2024-04-24T23:04:01.438595+0000","last_fullsized":"2024-04-24T23:04:01.438595+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:14:39.523826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.436676+0000","last_change":"2024-04-24T23:04:00.427010+0000","last_active":"2024-04-24T23:04:01.436676+0000","last_peered":"2024-04-24T23:04:01.436676+0000","last_clean":"2024-04-24T23:04:01.436676+0000","last_became_active":"2024-04-24T23:04:00.426200+0000","last_became_peered":"2024-04-24T23:04:00.426200+0000","last_unstale":"2024-04-24T23:04:01.436676+0000","last_undegraded":"2024-04-24T23:04:01.436676+0000","last_fullsized":"2024-04-24T23:04:01.436676+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:14.674089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.444160+0000","last_change":"2024-04-24T23:04:01.443960+0000","last_active":"2024-04-24T23:04:01.444160+0000","last_peered":"2024-04-24T23:04:01.444160+0000","last_clean":"2024-04-24T23:04:01.444160+0000","last_became_active":"2024-04-24T23:04:01.442098+0000","last_became_peered":"2024-04-24T23:04:01.442098+0000","last_unstale":"2024-04-24T23:04:01.444160+0000","last_undegraded":"2024-04-24T23:04:01.444160+0000","last_fullsized":"2024-04-24T23:04:01.444160+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:13.239828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.430318+0000","last_change":"2024-04-24T23:04:00.427378+0000","last_active":"2024-04-24T23:04:01.430318+0000","last_peered":"2024-04-24T23:04:01.430318+0000","last_clean":"2024-04-24T23:04:01.430318+0000","last_became_active":"2024-04-24T23:04:00.426725+0000","last_became_peered":"2024-04-24T23:04:00.426725+0000","last_unstale":"2024-04-24T23:04:01.430318+0000","last_undegraded":"2024-04-24T23:04:01.430318+0000","last_fullsized":"2024-04-24T23:04:01.430318+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:02:25.647213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.443760+0000","last_change":"2024-04-24T23:04:01.443631+0000","last_active":"2024-04-24T23:04:01.443760+0000","last_peered":"2024-04-24T23:04:01.443760+0000","last_clean":"2024-04-24T23:04:01.443760+0000","last_became_active":"2024-04-24T23:04:01.443335+0000","last_became_peered":"2024-04-24T23:04:01.443335+0000","last_unstale":"2024-04-24T23:04:01.443760+0000","last_undegraded":"2024-04-24T23:04:01.443760+0000","last_fullsized":"2024-04-24T23:04:01.443760+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:43:40.099887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"103'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.451433+0000","last_change":"2024-04-24T23:04:00.422932+0000","last_active":"2024-04-24T23:04:01.451433+0000","last_peered":"2024-04-24T23:04:01.451433+0000","last_clean":"2024-04-24T23:04:01.451433+0000","last_became_active":"2024-04-24T23:04:00.422679+0000","last_became_peered":"2024-04-24T23:04:00.422679+0000","last_unstale":"2024-04-24T23:04:01.451433+0000","last_undegraded":"2024-04-24T23:04:01.451433+0000","last_fullsized":"2024-04-24T23:04:01.451433+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:37.706560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":332,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.433460+0000","last_change":"2024-04-24T22:58:55.138810+0000","last_active":"2024-04-24T23:04:01.433460+0000","last_peered":"2024-04-24T23:04:01.433460+0000","last_clean":"2024-04-24T23:04:01.433460+0000","last_became_active":"2024-04-24T22:58:55.138627+0000","last_became_peered":"2024-04-24T22:58:55.138627+0000","last_unstale":"2024-04-24T23:04:01.433460+0000","last_undegraded":"2024-04-24T23:04:01.433460+0000","last_fullsized":"2024-04-24T23:04:01.433460+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_clean_scrub_stamp":"2024-04-24T22:58:54.127235+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:24:10.033365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574912,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":547,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194764,"statfs":{"total":200000000,"available":199439233,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574912,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":540,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194771,"statfs":{"total":200000000,"available":199446078,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574912,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1117,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194195,"statfs":{"total":200000000,"available":198855710,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574912,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":540,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194771,"statfs":{"total":200000000,"available":199446078,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574912,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":548,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194764,"statfs":{"total":200000000,"available":199438629,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574912,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":547,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194764,"statfs":{"total":200000000,"available":199439303,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1117,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194195,"statfs":{"total":200000000,"available":198855710,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574912,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":549,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194763,"statfs":{"total":200000000,"available":199437669,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-24T23:04:05.436 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:04:05.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.571+0000 7f4046b7e640 1 Processor -- start 2024-04-24T23:04:05.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.571+0000 7f4046b7e640 1 -- start start 2024-04-24T23:04:05.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.572+0000 7f4046b7e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f404006b820 0x7f40400d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:05.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.572+0000 7f4046b7e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f40400d4210 0x7f40400b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:05.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.572+0000 7f4046b7e640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40400d3170 con 0x7f404015e9b0 2024-04-24T23:04:05.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.572+0000 7f4046b7e640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f40400d3450 con 0x7f40400d4210 2024-04-24T23:04:05.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.572+0000 7f4046b7e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f40400d3730 con 0x7f404006b820 2024-04-24T23:04:05.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.572+0000 7f404511b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f40400d4210 0x7f40400b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:05.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.572+0000 7f404511b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f40400d4210 0x7f40400b78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60230/0 (socket says 172.21.15.32:60230) 2024-04-24T23:04:05.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.572+0000 7f404511b640 1 -- 172.21.15.32:0/3662254696 learned_addr learned my addr 172.21.15.32:0/3662254696 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:05.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.572+0000 7f403ffff640 1 --2- 172.21.15.32:0/3662254696 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f404006b820 0x7f40400d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:05.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.572+0000 7f404511b640 1 -- 172.21.15.32:0/3662254696 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f404006b820 msgr2=0x7f40400d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:05.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.572+0000 7f404511b640 1 --2- 172.21.15.32:0/3662254696 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f404006b820 0x7f40400d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:05.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.572+0000 7f404511b640 1 -- 172.21.15.32:0/3662254696 >> v1:172.21.15.121:6789/0 conn(0x7f404015e9b0 legacy=0x7f404006b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:05.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.572+0000 7f404511b640 1 -- 172.21.15.32:0/3662254696 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40400d3a10 con 0x7f40400d4210 2024-04-24T23:04:05.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.573+0000 7f404511b640 1 --2- 172.21.15.32:0/3662254696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40400d4210 0x7f40400b78f0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f4038009e50 tx=0x7f403802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8bc983ee870107d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:05.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.573+0000 7f403f7fe640 1 -- 172.21.15.32:0/3662254696 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4038036070 con 0x7f40400d4210 2024-04-24T23:04:05.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.573+0000 7f403f7fe640 1 -- 172.21.15.32:0/3662254696 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4038031600 con 0x7f40400d4210 2024-04-24T23:04:05.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.573+0000 7f403f7fe640 1 -- 172.21.15.32:0/3662254696 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40380319d0 con 0x7f40400d4210 2024-04-24T23:04:05.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.574+0000 7f4046b7e640 1 -- 172.21.15.32:0/3662254696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40400d4210 msgr2=0x7f40400b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:05.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.574+0000 7f4046b7e640 1 --2- 172.21.15.32:0/3662254696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40400d4210 0x7f40400b78f0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f4038009e50 tx=0x7f403802a710 comp rx=0 tx=0).stop 2024-04-24T23:04:05.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.574+0000 7f4046b7e640 1 -- 172.21.15.32:0/3662254696 shutdown_connections 2024-04-24T23:04:05.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.574+0000 7f4046b7e640 1 --2- 172.21.15.32:0/3662254696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40400d4210 0x7f40400b78f0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:05.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.574+0000 7f4046b7e640 1 --2- 172.21.15.32:0/3662254696 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f404006b820 0x7f40400d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:05.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.574+0000 7f4046b7e640 1 -- 172.21.15.32:0/3662254696 >> 172.21.15.32:0/3662254696 conn(0x7f40400d11d0 msgr2=0x7f40400d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:05.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.574+0000 7f4046b7e640 1 -- 172.21.15.32:0/3662254696 shutdown_connections 2024-04-24T23:04:05.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.574+0000 7f4046b7e640 1 -- 172.21.15.32:0/3662254696 wait complete. 2024-04-24T23:04:05.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.575+0000 7f4046b7e640 1 Processor -- start 2024-04-24T23:04:05.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.575+0000 7f4046b7e640 1 -- start start 2024-04-24T23:04:05.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.576+0000 7f4046b7e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f404006b820 0x7f4040157640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:05.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.576+0000 7f4046b7e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40400d4210 0x7f404014fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:05.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.576+0000 7f4046b7e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f404015e9b0 0x7f4040158d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:05.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.576+0000 7f4046b7e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f40400ba0e0 con 0x7f404015e9b0 2024-04-24T23:04:05.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.576+0000 7f4046b7e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f40400d2400 con 0x7f40400d4210 2024-04-24T23:04:05.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.576+0000 7f4046b7e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f40400ba260 con 0x7f404006b820 2024-04-24T23:04:05.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.576+0000 7f404511b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f404015e9b0 0x7f4040158d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:05.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.576+0000 7f404511b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f404015e9b0 0x7f4040158d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60234/0 (socket says 172.21.15.32:60234) 2024-04-24T23:04:05.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.576+0000 7f404511b640 1 -- 172.21.15.32:0/1092923530 learned_addr learned my addr 172.21.15.32:0/1092923530 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:05.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.576+0000 7f404491a640 1 --2- 172.21.15.32:0/1092923530 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f404006b820 0x7f4040157640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:05.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.576+0000 7f403ffff640 1 --2- 172.21.15.32:0/1092923530 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40400d4210 0x7f404014fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:05.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.576+0000 7f404511b640 1 -- 172.21.15.32:0/1092923530 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f404006b820 msgr2=0x7f4040157640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:05.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.576+0000 7f404511b640 1 --2- 172.21.15.32:0/1092923530 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f404006b820 0x7f4040157640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:05.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.576+0000 7f404511b640 1 -- 172.21.15.32:0/1092923530 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40400d4210 msgr2=0x7f404014fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:05.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.576+0000 7f404511b640 1 --2- 172.21.15.32:0/1092923530 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40400d4210 0x7f404014fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:05.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.576+0000 7f404511b640 1 -- 172.21.15.32:0/1092923530 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40401565e0 con 0x7f404015e9b0 2024-04-24T23:04:05.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.576+0000 7f404491a640 1 --2- 172.21.15.32:0/1092923530 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f404006b820 0x7f4040157640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:04:05.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.577+0000 7f404511b640 1 --2- 172.21.15.32:0/1092923530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f404015e9b0 0x7f4040158d80 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f4038034040 tx=0x7f4038004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:05.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.577+0000 7f403d7fa640 1 -- 172.21.15.32:0/1092923530 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4038036050 con 0x7f404015e9b0 2024-04-24T23:04:05.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.577+0000 7f403d7fa640 1 -- 172.21.15.32:0/1092923530 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4038042070 con 0x7f404015e9b0 2024-04-24T23:04:05.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.577+0000 7f4046b7e640 1 -- 172.21.15.32:0/1092923530 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4040156880 con 0x7f404015e9b0 2024-04-24T23:04:05.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.577+0000 7f403d7fa640 1 -- 172.21.15.32:0/1092923530 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4038009510 con 0x7f404015e9b0 2024-04-24T23:04:05.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.577+0000 7f4046b7e640 1 -- 172.21.15.32:0/1092923530 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4040156e40 con 0x7f404015e9b0 2024-04-24T23:04:05.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.578+0000 7f4046b7e640 1 -- 172.21.15.32:0/1092923530 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4040068430 con 0x7f404015e9b0 2024-04-24T23:04:05.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.578+0000 7f403d7fa640 1 -- 172.21.15.32:0/1092923530 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4038007bd0 con 0x7f404015e9b0 2024-04-24T23:04:05.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.579+0000 7f403d7fa640 1 --2- 172.21.15.32:0/1092923530 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f402406be30 0x7f402406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:05.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.579+0000 7f403d7fa640 1 -- 172.21.15.32:0/1092923530 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7f40380abcb0 con 0x7f404015e9b0 2024-04-24T23:04:05.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.579+0000 7f404491a640 1 --2- 172.21.15.32:0/1092923530 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f402406be30 0x7f402406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:05.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.580+0000 7f404491a640 1 --2- 172.21.15.32:0/1092923530 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f402406be30 0x7f402406e2f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4034001660 tx=0x7f4034009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:05.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.583+0000 7f403d7fa640 1 -- 172.21.15.32:0/1092923530 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f403807a650 con 0x7f404015e9b0 2024-04-24T23:04:05.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.761+0000 7f4046b7e640 1 -- 172.21.15.32:0/1092923530 --> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4040156050 con 0x7f402406be30 2024-04-24T23:04:05.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.767+0000 7f403d7fa640 1 -- 172.21.15.32:0/1092923530 <== mgr.4108 v2:172.21.15.121:6832/1848157943 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247923 (secure 0 0 0) 0x7f4040156050 con 0x7f402406be30 2024-04-24T23:04:05.771 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:04:05.772 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-24T23:04:05.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.769+0000 7f4046b7e640 1 -- 172.21.15.32:0/1092923530 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f402406be30 msgr2=0x7f402406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:05.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.769+0000 7f4046b7e640 1 --2- 172.21.15.32:0/1092923530 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f402406be30 0x7f402406e2f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4034001660 tx=0x7f4034009040 comp rx=0 tx=0).stop 2024-04-24T23:04:05.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.769+0000 7f4046b7e640 1 -- 172.21.15.32:0/1092923530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f404015e9b0 msgr2=0x7f4040158d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:05.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.769+0000 7f4046b7e640 1 --2- 172.21.15.32:0/1092923530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f404015e9b0 0x7f4040158d80 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f4038034040 tx=0x7f4038004a40 comp rx=0 tx=0).stop 2024-04-24T23:04:05.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.770+0000 7f4046b7e640 1 -- 172.21.15.32:0/1092923530 shutdown_connections 2024-04-24T23:04:05.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.770+0000 7f4046b7e640 1 --2- 172.21.15.32:0/1092923530 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f402406be30 0x7f402406e2f0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:05.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.770+0000 7f4046b7e640 1 --2- 172.21.15.32:0/1092923530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f404015e9b0 0x7f4040158d80 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:05.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.770+0000 7f4046b7e640 1 --2- 172.21.15.32:0/1092923530 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40400d4210 0x7f404014fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:05.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.770+0000 7f4046b7e640 1 --2- 172.21.15.32:0/1092923530 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f404006b820 0x7f4040157640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:05.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.770+0000 7f4046b7e640 1 -- 172.21.15.32:0/1092923530 >> 172.21.15.32:0/1092923530 conn(0x7f40400d11d0 msgr2=0x7f40400caba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:05.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.770+0000 7f4046b7e640 1 -- 172.21.15.32:0/1092923530 shutdown_connections 2024-04-24T23:04:05.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.770+0000 7f4046b7e640 1 -- 172.21.15.32:0/1092923530 wait complete. 2024-04-24T23:04:05.782 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":252,"stamp":"2024-04-24T23:04:05.196714+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":100,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":5509,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1556990,"statfs":{"total":1600000000,"available":1594358410,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.785343"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.439768+0000","last_change":"2024-04-24T23:04:01.439624+0000","last_active":"2024-04-24T23:04:01.439768+0000","last_peered":"2024-04-24T23:04:01.439768+0000","last_clean":"2024-04-24T23:04:01.439768+0000","last_became_active":"2024-04-24T23:04:01.439345+0000","last_became_peered":"2024-04-24T23:04:01.439345+0000","last_unstale":"2024-04-24T23:04:01.439768+0000","last_undegraded":"2024-04-24T23:04:01.439768+0000","last_fullsized":"2024-04-24T23:04:01.439768+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:59.368837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.439396+0000","last_change":"2024-04-24T23:04:01.439230+0000","last_active":"2024-04-24T23:04:01.439396+0000","last_peered":"2024-04-24T23:04:01.439396+0000","last_clean":"2024-04-24T23:04:01.439396+0000","last_became_active":"2024-04-24T23:04:01.438999+0000","last_became_peered":"2024-04-24T23:04:01.438999+0000","last_unstale":"2024-04-24T23:04:01.439396+0000","last_undegraded":"2024-04-24T23:04:01.439396+0000","last_fullsized":"2024-04-24T23:04:01.439396+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:18:48.331110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.446183+0000","last_change":"2024-04-24T23:04:01.446078+0000","last_active":"2024-04-24T23:04:01.446183+0000","last_peered":"2024-04-24T23:04:01.446183+0000","last_clean":"2024-04-24T23:04:01.446183+0000","last_became_active":"2024-04-24T23:04:01.445687+0000","last_became_peered":"2024-04-24T23:04:01.445687+0000","last_unstale":"2024-04-24T23:04:01.446183+0000","last_undegraded":"2024-04-24T23:04:01.446183+0000","last_fullsized":"2024-04-24T23:04:01.446183+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:06:44.287698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.443401+0000","last_change":"2024-04-24T23:04:01.443285+0000","last_active":"2024-04-24T23:04:01.443401+0000","last_peered":"2024-04-24T23:04:01.443401+0000","last_clean":"2024-04-24T23:04:01.443401+0000","last_became_active":"2024-04-24T23:04:01.443033+0000","last_became_peered":"2024-04-24T23:04:01.443033+0000","last_unstale":"2024-04-24T23:04:01.443401+0000","last_undegraded":"2024-04-24T23:04:01.443401+0000","last_fullsized":"2024-04-24T23:04:01.443401+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:54:04.126575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.439061+0000","last_change":"2024-04-24T23:04:01.438932+0000","last_active":"2024-04-24T23:04:01.439061+0000","last_peered":"2024-04-24T23:04:01.439061+0000","last_clean":"2024-04-24T23:04:01.439061+0000","last_became_active":"2024-04-24T23:04:01.438680+0000","last_became_peered":"2024-04-24T23:04:01.438680+0000","last_unstale":"2024-04-24T23:04:01.439061+0000","last_undegraded":"2024-04-24T23:04:01.439061+0000","last_fullsized":"2024-04-24T23:04:01.439061+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:39:56.940903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.444933+0000","last_change":"2024-04-24T23:04:01.444749+0000","last_active":"2024-04-24T23:04:01.444933+0000","last_peered":"2024-04-24T23:04:01.444933+0000","last_clean":"2024-04-24T23:04:01.444933+0000","last_became_active":"2024-04-24T23:04:01.443804+0000","last_became_peered":"2024-04-24T23:04:01.443804+0000","last_unstale":"2024-04-24T23:04:01.444933+0000","last_undegraded":"2024-04-24T23:04:01.444933+0000","last_fullsized":"2024-04-24T23:04:01.444933+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:13.518609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.443843+0000","last_change":"2024-04-24T23:04:01.443243+0000","last_active":"2024-04-24T23:04:01.443843+0000","last_peered":"2024-04-24T23:04:01.443843+0000","last_clean":"2024-04-24T23:04:01.443843+0000","last_became_active":"2024-04-24T23:04:01.443033+0000","last_became_peered":"2024-04-24T23:04:01.443033+0000","last_unstale":"2024-04-24T23:04:01.443843+0000","last_undegraded":"2024-04-24T23:04:01.443843+0000","last_fullsized":"2024-04-24T23:04:01.443843+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:39:54.313053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.434610+0000","last_change":"2024-04-24T23:04:01.434481+0000","last_active":"2024-04-24T23:04:01.434610+0000","last_peered":"2024-04-24T23:04:01.434610+0000","last_clean":"2024-04-24T23:04:01.434610+0000","last_became_active":"2024-04-24T23:04:01.434114+0000","last_became_peered":"2024-04-24T23:04:01.434114+0000","last_unstale":"2024-04-24T23:04:01.434610+0000","last_undegraded":"2024-04-24T23:04:01.434610+0000","last_fullsized":"2024-04-24T23:04:01.434610+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:01:58.242511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.438350+0000","last_change":"2024-04-24T23:04:01.438222+0000","last_active":"2024-04-24T23:04:01.438350+0000","last_peered":"2024-04-24T23:04:01.438350+0000","last_clean":"2024-04-24T23:04:01.438350+0000","last_became_active":"2024-04-24T23:04:01.437639+0000","last_became_peered":"2024-04-24T23:04:01.437639+0000","last_unstale":"2024-04-24T23:04:01.438350+0000","last_undegraded":"2024-04-24T23:04:01.438350+0000","last_fullsized":"2024-04-24T23:04:01.438350+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:09:45.413068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.437805+0000","last_change":"2024-04-24T23:04:01.437693+0000","last_active":"2024-04-24T23:04:01.437805+0000","last_peered":"2024-04-24T23:04:01.437805+0000","last_clean":"2024-04-24T23:04:01.437805+0000","last_became_active":"2024-04-24T23:04:01.437409+0000","last_became_peered":"2024-04-24T23:04:01.437409+0000","last_unstale":"2024-04-24T23:04:01.437805+0000","last_undegraded":"2024-04-24T23:04:01.437805+0000","last_fullsized":"2024-04-24T23:04:01.437805+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:25.922876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.442567+0000","last_change":"2024-04-24T23:04:01.442083+0000","last_active":"2024-04-24T23:04:01.442567+0000","last_peered":"2024-04-24T23:04:01.442567+0000","last_clean":"2024-04-24T23:04:01.442567+0000","last_became_active":"2024-04-24T23:04:01.441474+0000","last_became_peered":"2024-04-24T23:04:01.441474+0000","last_unstale":"2024-04-24T23:04:01.442567+0000","last_undegraded":"2024-04-24T23:04:01.442567+0000","last_fullsized":"2024-04-24T23:04:01.442567+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:49.893907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.447195+0000","last_change":"2024-04-24T23:04:01.446747+0000","last_active":"2024-04-24T23:04:01.447195+0000","last_peered":"2024-04-24T23:04:01.447195+0000","last_clean":"2024-04-24T23:04:01.447195+0000","last_became_active":"2024-04-24T23:04:01.445743+0000","last_became_peered":"2024-04-24T23:04:01.445743+0000","last_unstale":"2024-04-24T23:04:01.447195+0000","last_undegraded":"2024-04-24T23:04:01.447195+0000","last_fullsized":"2024-04-24T23:04:01.447195+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:14:39.847591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.442669+0000","last_change":"2024-04-24T23:04:01.442551+0000","last_active":"2024-04-24T23:04:01.442669+0000","last_peered":"2024-04-24T23:04:01.442669+0000","last_clean":"2024-04-24T23:04:01.442669+0000","last_became_active":"2024-04-24T23:04:01.442011+0000","last_became_peered":"2024-04-24T23:04:01.442011+0000","last_unstale":"2024-04-24T23:04:01.442669+0000","last_undegraded":"2024-04-24T23:04:01.442669+0000","last_fullsized":"2024-04-24T23:04:01.442669+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:26.269220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.440827+0000","last_change":"2024-04-24T23:04:01.440301+0000","last_active":"2024-04-24T23:04:01.440827+0000","last_peered":"2024-04-24T23:04:01.440827+0000","last_clean":"2024-04-24T23:04:01.440827+0000","last_became_active":"2024-04-24T23:04:01.440011+0000","last_became_peered":"2024-04-24T23:04:01.440011+0000","last_unstale":"2024-04-24T23:04:01.440827+0000","last_undegraded":"2024-04-24T23:04:01.440827+0000","last_fullsized":"2024-04-24T23:04:01.440827+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:18:54.054872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.443822+0000","last_change":"2024-04-24T23:04:01.441574+0000","last_active":"2024-04-24T23:04:01.443822+0000","last_peered":"2024-04-24T23:04:01.443822+0000","last_clean":"2024-04-24T23:04:01.443822+0000","last_became_active":"2024-04-24T23:04:01.441305+0000","last_became_peered":"2024-04-24T23:04:01.441305+0000","last_unstale":"2024-04-24T23:04:01.443822+0000","last_undegraded":"2024-04-24T23:04:01.443822+0000","last_fullsized":"2024-04-24T23:04:01.443822+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:37:57.283801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.447833+0000","last_change":"2024-04-24T23:04:01.447695+0000","last_active":"2024-04-24T23:04:01.447833+0000","last_peered":"2024-04-24T23:04:01.447833+0000","last_clean":"2024-04-24T23:04:01.447833+0000","last_became_active":"2024-04-24T23:04:01.447453+0000","last_became_peered":"2024-04-24T23:04:01.447453+0000","last_unstale":"2024-04-24T23:04:01.447833+0000","last_undegraded":"2024-04-24T23:04:01.447833+0000","last_fullsized":"2024-04-24T23:04:01.447833+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:39:02.050156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.446586+0000","last_change":"2024-04-24T23:04:01.445980+0000","last_active":"2024-04-24T23:04:01.446586+0000","last_peered":"2024-04-24T23:04:01.446586+0000","last_clean":"2024-04-24T23:04:01.446586+0000","last_became_active":"2024-04-24T23:04:01.445412+0000","last_became_peered":"2024-04-24T23:04:01.445412+0000","last_unstale":"2024-04-24T23:04:01.446586+0000","last_undegraded":"2024-04-24T23:04:01.446586+0000","last_fullsized":"2024-04-24T23:04:01.446586+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:42:47.271129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.433700+0000","last_change":"2024-04-24T23:04:01.433444+0000","last_active":"2024-04-24T23:04:01.433700+0000","last_peered":"2024-04-24T23:04:01.433700+0000","last_clean":"2024-04-24T23:04:01.433700+0000","last_became_active":"2024-04-24T23:04:01.433094+0000","last_became_peered":"2024-04-24T23:04:01.433094+0000","last_unstale":"2024-04-24T23:04:01.433700+0000","last_undegraded":"2024-04-24T23:04:01.433700+0000","last_fullsized":"2024-04-24T23:04:01.433700+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:28:14.277049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.443053+0000","last_change":"2024-04-24T23:04:01.442672+0000","last_active":"2024-04-24T23:04:01.443053+0000","last_peered":"2024-04-24T23:04:01.443053+0000","last_clean":"2024-04-24T23:04:01.443053+0000","last_became_active":"2024-04-24T23:04:01.442264+0000","last_became_peered":"2024-04-24T23:04:01.442264+0000","last_unstale":"2024-04-24T23:04:01.443053+0000","last_undegraded":"2024-04-24T23:04:01.443053+0000","last_fullsized":"2024-04-24T23:04:01.443053+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:57.766064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.441995+0000","last_change":"2024-04-24T23:04:01.441881+0000","last_active":"2024-04-24T23:04:01.441995+0000","last_peered":"2024-04-24T23:04:01.441995+0000","last_clean":"2024-04-24T23:04:01.441995+0000","last_became_active":"2024-04-24T23:04:01.441244+0000","last_became_peered":"2024-04-24T23:04:01.441244+0000","last_unstale":"2024-04-24T23:04:01.441995+0000","last_undegraded":"2024-04-24T23:04:01.441995+0000","last_fullsized":"2024-04-24T23:04:01.441995+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:25:37.159164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.435271+0000","last_change":"2024-04-24T23:04:01.435125+0000","last_active":"2024-04-24T23:04:01.435271+0000","last_peered":"2024-04-24T23:04:01.435271+0000","last_clean":"2024-04-24T23:04:01.435271+0000","last_became_active":"2024-04-24T23:04:01.434477+0000","last_became_peered":"2024-04-24T23:04:01.434477+0000","last_unstale":"2024-04-24T23:04:01.435271+0000","last_undegraded":"2024-04-24T23:04:01.435271+0000","last_fullsized":"2024-04-24T23:04:01.435271+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:12:54.123074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.446195+0000","last_change":"2024-04-24T23:04:01.446059+0000","last_active":"2024-04-24T23:04:01.446195+0000","last_peered":"2024-04-24T23:04:01.446195+0000","last_clean":"2024-04-24T23:04:01.446195+0000","last_became_active":"2024-04-24T23:04:01.445701+0000","last_became_peered":"2024-04-24T23:04:01.445701+0000","last_unstale":"2024-04-24T23:04:01.446195+0000","last_undegraded":"2024-04-24T23:04:01.446195+0000","last_fullsized":"2024-04-24T23:04:01.446195+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:46:19.706069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.442715+0000","last_change":"2024-04-24T23:04:01.442023+0000","last_active":"2024-04-24T23:04:01.442715+0000","last_peered":"2024-04-24T23:04:01.442715+0000","last_clean":"2024-04-24T23:04:01.442715+0000","last_became_active":"2024-04-24T23:04:01.441742+0000","last_became_peered":"2024-04-24T23:04:01.441742+0000","last_unstale":"2024-04-24T23:04:01.442715+0000","last_undegraded":"2024-04-24T23:04:01.442715+0000","last_fullsized":"2024-04-24T23:04:01.442715+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:08:21.524405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.441838+0000","last_change":"2024-04-24T23:04:01.441669+0000","last_active":"2024-04-24T23:04:01.441838+0000","last_peered":"2024-04-24T23:04:01.441838+0000","last_clean":"2024-04-24T23:04:01.441838+0000","last_became_active":"2024-04-24T23:04:01.439126+0000","last_became_peered":"2024-04-24T23:04:01.439126+0000","last_unstale":"2024-04-24T23:04:01.441838+0000","last_undegraded":"2024-04-24T23:04:01.441838+0000","last_fullsized":"2024-04-24T23:04:01.441838+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:48:04.843640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.432874+0000","last_change":"2024-04-24T23:04:01.432728+0000","last_active":"2024-04-24T23:04:01.432874+0000","last_peered":"2024-04-24T23:04:01.432874+0000","last_clean":"2024-04-24T23:04:01.432874+0000","last_became_active":"2024-04-24T23:04:01.432266+0000","last_became_peered":"2024-04-24T23:04:01.432266+0000","last_unstale":"2024-04-24T23:04:01.432874+0000","last_undegraded":"2024-04-24T23:04:01.432874+0000","last_fullsized":"2024-04-24T23:04:01.432874+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:26:51.472758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.436987+0000","last_change":"2024-04-24T23:04:01.435903+0000","last_active":"2024-04-24T23:04:01.436987+0000","last_peered":"2024-04-24T23:04:01.436987+0000","last_clean":"2024-04-24T23:04:01.436987+0000","last_became_active":"2024-04-24T23:04:01.435640+0000","last_became_peered":"2024-04-24T23:04:01.435640+0000","last_unstale":"2024-04-24T23:04:01.436987+0000","last_undegraded":"2024-04-24T23:04:01.436987+0000","last_fullsized":"2024-04-24T23:04:01.436987+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:19:59.023039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.437218+0000","last_change":"2024-04-24T23:04:01.436705+0000","last_active":"2024-04-24T23:04:01.437218+0000","last_peered":"2024-04-24T23:04:01.437218+0000","last_clean":"2024-04-24T23:04:01.437218+0000","last_became_active":"2024-04-24T23:04:01.436219+0000","last_became_peered":"2024-04-24T23:04:01.436219+0000","last_unstale":"2024-04-24T23:04:01.437218+0000","last_undegraded":"2024-04-24T23:04:01.437218+0000","last_fullsized":"2024-04-24T23:04:01.437218+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:50:03.902587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.445100+0000","last_change":"2024-04-24T23:04:01.444371+0000","last_active":"2024-04-24T23:04:01.445100+0000","last_peered":"2024-04-24T23:04:01.445100+0000","last_clean":"2024-04-24T23:04:01.445100+0000","last_became_active":"2024-04-24T23:04:01.442437+0000","last_became_peered":"2024-04-24T23:04:01.442437+0000","last_unstale":"2024-04-24T23:04:01.445100+0000","last_undegraded":"2024-04-24T23:04:01.445100+0000","last_fullsized":"2024-04-24T23:04:01.445100+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:54:28.694301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.440415+0000","last_change":"2024-04-24T23:04:01.440316+0000","last_active":"2024-04-24T23:04:01.440415+0000","last_peered":"2024-04-24T23:04:01.440415+0000","last_clean":"2024-04-24T23:04:01.440415+0000","last_became_active":"2024-04-24T23:04:01.439710+0000","last_became_peered":"2024-04-24T23:04:01.439710+0000","last_unstale":"2024-04-24T23:04:01.440415+0000","last_undegraded":"2024-04-24T23:04:01.440415+0000","last_fullsized":"2024-04-24T23:04:01.440415+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:48.560864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.441768+0000","last_change":"2024-04-24T23:04:01.441622+0000","last_active":"2024-04-24T23:04:01.441768+0000","last_peered":"2024-04-24T23:04:01.441768+0000","last_clean":"2024-04-24T23:04:01.441768+0000","last_became_active":"2024-04-24T23:04:01.440954+0000","last_became_peered":"2024-04-24T23:04:01.440954+0000","last_unstale":"2024-04-24T23:04:01.441768+0000","last_undegraded":"2024-04-24T23:04:01.441768+0000","last_fullsized":"2024-04-24T23:04:01.441768+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:48:17.112726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.444514+0000","last_change":"2024-04-24T23:04:01.444342+0000","last_active":"2024-04-24T23:04:01.444514+0000","last_peered":"2024-04-24T23:04:01.444514+0000","last_clean":"2024-04-24T23:04:01.444514+0000","last_became_active":"2024-04-24T23:04:01.443485+0000","last_became_peered":"2024-04-24T23:04:01.443485+0000","last_unstale":"2024-04-24T23:04:01.444514+0000","last_undegraded":"2024-04-24T23:04:01.444514+0000","last_fullsized":"2024-04-24T23:04:01.444514+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:17:38.086016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.446927+0000","last_change":"2024-04-24T23:04:01.446631+0000","last_active":"2024-04-24T23:04:01.446927+0000","last_peered":"2024-04-24T23:04:01.446927+0000","last_clean":"2024-04-24T23:04:01.446927+0000","last_became_active":"2024-04-24T23:04:01.444554+0000","last_became_peered":"2024-04-24T23:04:01.444554+0000","last_unstale":"2024-04-24T23:04:01.446927+0000","last_undegraded":"2024-04-24T23:04:01.446927+0000","last_fullsized":"2024-04-24T23:04:01.446927+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:54.200388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.433877+0000","last_change":"2024-04-24T23:04:01.432857+0000","last_active":"2024-04-24T23:04:01.433877+0000","last_peered":"2024-04-24T23:04:01.433877+0000","last_clean":"2024-04-24T23:04:01.433877+0000","last_became_active":"2024-04-24T23:04:01.432621+0000","last_became_peered":"2024-04-24T23:04:01.432621+0000","last_unstale":"2024-04-24T23:04:01.433877+0000","last_undegraded":"2024-04-24T23:04:01.433877+0000","last_fullsized":"2024-04-24T23:04:01.433877+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:27:15.385203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"103'4","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.452299+0000","last_change":"2024-04-24T23:04:00.420780+0000","last_active":"2024-04-24T23:04:01.452299+0000","last_peered":"2024-04-24T23:04:01.452299+0000","last_clean":"2024-04-24T23:04:01.452299+0000","last_became_active":"2024-04-24T23:04:00.420491+0000","last_became_peered":"2024-04-24T23:04:00.420491+0000","last_unstale":"2024-04-24T23:04:01.452299+0000","last_undegraded":"2024-04-24T23:04:01.452299+0000","last_fullsized":"2024-04-24T23:04:01.452299+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:13.110238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.443281+0000","last_change":"2024-04-24T23:04:01.442675+0000","last_active":"2024-04-24T23:04:01.443281+0000","last_peered":"2024-04-24T23:04:01.443281+0000","last_clean":"2024-04-24T23:04:01.443281+0000","last_became_active":"2024-04-24T23:04:01.441778+0000","last_became_peered":"2024-04-24T23:04:01.441778+0000","last_unstale":"2024-04-24T23:04:01.443281+0000","last_undegraded":"2024-04-24T23:04:01.443281+0000","last_fullsized":"2024-04-24T23:04:01.443281+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:40:59.394072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"103'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.453265+0000","last_change":"2024-04-24T23:04:00.421631+0000","last_active":"2024-04-24T23:04:01.453265+0000","last_peered":"2024-04-24T23:04:01.453265+0000","last_clean":"2024-04-24T23:04:01.453265+0000","last_became_active":"2024-04-24T23:04:00.421382+0000","last_became_peered":"2024-04-24T23:04:00.421382+0000","last_unstale":"2024-04-24T23:04:01.453265+0000","last_undegraded":"2024-04-24T23:04:01.453265+0000","last_fullsized":"2024-04-24T23:04:01.453265+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:08:38.371647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.437447+0000","last_change":"2024-04-24T23:04:01.436247+0000","last_active":"2024-04-24T23:04:01.437447+0000","last_peered":"2024-04-24T23:04:01.437447+0000","last_clean":"2024-04-24T23:04:01.437447+0000","last_became_active":"2024-04-24T23:04:01.434655+0000","last_became_peered":"2024-04-24T23:04:01.434655+0000","last_unstale":"2024-04-24T23:04:01.437447+0000","last_undegraded":"2024-04-24T23:04:01.437447+0000","last_fullsized":"2024-04-24T23:04:01.437447+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:51:58.897857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"103'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.451511+0000","last_change":"2024-04-24T23:04:00.424522+0000","last_active":"2024-04-24T23:04:01.451511+0000","last_peered":"2024-04-24T23:04:01.451511+0000","last_clean":"2024-04-24T23:04:01.451511+0000","last_became_active":"2024-04-24T23:04:00.424168+0000","last_became_peered":"2024-04-24T23:04:00.424168+0000","last_unstale":"2024-04-24T23:04:01.451511+0000","last_undegraded":"2024-04-24T23:04:01.451511+0000","last_fullsized":"2024-04-24T23:04:01.451511+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:39.737703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.440170+0000","last_change":"2024-04-24T23:04:01.440036+0000","last_active":"2024-04-24T23:04:01.440170+0000","last_peered":"2024-04-24T23:04:01.440170+0000","last_clean":"2024-04-24T23:04:01.440170+0000","last_became_active":"2024-04-24T23:04:01.439005+0000","last_became_peered":"2024-04-24T23:04:01.439005+0000","last_unstale":"2024-04-24T23:04:01.440170+0000","last_undegraded":"2024-04-24T23:04:01.440170+0000","last_fullsized":"2024-04-24T23:04:01.440170+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:08:37.486298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"103'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.451081+0000","last_change":"2024-04-24T23:04:00.425085+0000","last_active":"2024-04-24T23:04:01.451081+0000","last_peered":"2024-04-24T23:04:01.451081+0000","last_clean":"2024-04-24T23:04:01.451081+0000","last_became_active":"2024-04-24T23:04:00.424750+0000","last_became_peered":"2024-04-24T23:04:00.424750+0000","last_unstale":"2024-04-24T23:04:01.451081+0000","last_undegraded":"2024-04-24T23:04:01.451081+0000","last_fullsized":"2024-04-24T23:04:01.451081+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:15:01.161630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.439121+0000","last_change":"2024-04-24T23:04:01.436590+0000","last_active":"2024-04-24T23:04:01.439121+0000","last_peered":"2024-04-24T23:04:01.439121+0000","last_clean":"2024-04-24T23:04:01.439121+0000","last_became_active":"2024-04-24T23:04:01.436367+0000","last_became_peered":"2024-04-24T23:04:01.436367+0000","last_unstale":"2024-04-24T23:04:01.439121+0000","last_undegraded":"2024-04-24T23:04:01.439121+0000","last_fullsized":"2024-04-24T23:04:01.439121+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:15.053717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"103'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.452087+0000","last_change":"2024-04-24T23:04:00.427288+0000","last_active":"2024-04-24T23:04:01.452087+0000","last_peered":"2024-04-24T23:04:01.452087+0000","last_clean":"2024-04-24T23:04:01.452087+0000","last_became_active":"2024-04-24T23:04:00.427047+0000","last_became_peered":"2024-04-24T23:04:00.427047+0000","last_unstale":"2024-04-24T23:04:01.452087+0000","last_undegraded":"2024-04-24T23:04:01.452087+0000","last_fullsized":"2024-04-24T23:04:01.452087+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:10:55.535313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.447406+0000","last_change":"2024-04-24T23:04:01.447092+0000","last_active":"2024-04-24T23:04:01.447406+0000","last_peered":"2024-04-24T23:04:01.447406+0000","last_clean":"2024-04-24T23:04:01.447406+0000","last_became_active":"2024-04-24T23:04:01.444885+0000","last_became_peered":"2024-04-24T23:04:01.444885+0000","last_unstale":"2024-04-24T23:04:01.447406+0000","last_undegraded":"2024-04-24T23:04:01.447406+0000","last_fullsized":"2024-04-24T23:04:01.447406+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:35:39.696962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"103'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.453176+0000","last_change":"2024-04-24T23:04:00.428309+0000","last_active":"2024-04-24T23:04:01.453176+0000","last_peered":"2024-04-24T23:04:01.453176+0000","last_clean":"2024-04-24T23:04:01.453176+0000","last_became_active":"2024-04-24T23:04:00.427969+0000","last_became_peered":"2024-04-24T23:04:00.427969+0000","last_unstale":"2024-04-24T23:04:01.453176+0000","last_undegraded":"2024-04-24T23:04:01.453176+0000","last_fullsized":"2024-04-24T23:04:01.453176+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:00:45.798299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.439844+0000","last_change":"2024-04-24T23:04:01.439730+0000","last_active":"2024-04-24T23:04:01.439844+0000","last_peered":"2024-04-24T23:04:01.439844+0000","last_clean":"2024-04-24T23:04:01.439844+0000","last_became_active":"2024-04-24T23:04:01.438529+0000","last_became_peered":"2024-04-24T23:04:01.438529+0000","last_unstale":"2024-04-24T23:04:01.439844+0000","last_undegraded":"2024-04-24T23:04:01.439844+0000","last_fullsized":"2024-04-24T23:04:01.439844+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:02.275002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.438944+0000","last_change":"2024-04-24T23:04:00.423871+0000","last_active":"2024-04-24T23:04:01.438944+0000","last_peered":"2024-04-24T23:04:01.438944+0000","last_clean":"2024-04-24T23:04:01.438944+0000","last_became_active":"2024-04-24T23:04:00.423635+0000","last_became_peered":"2024-04-24T23:04:00.423635+0000","last_unstale":"2024-04-24T23:04:01.438944+0000","last_undegraded":"2024-04-24T23:04:01.438944+0000","last_fullsized":"2024-04-24T23:04:01.438944+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:18:21.732139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.445668+0000","last_change":"2024-04-24T23:04:01.445562+0000","last_active":"2024-04-24T23:04:01.445668+0000","last_peered":"2024-04-24T23:04:01.445668+0000","last_clean":"2024-04-24T23:04:01.445668+0000","last_became_active":"2024-04-24T23:04:01.445307+0000","last_became_peered":"2024-04-24T23:04:01.445307+0000","last_unstale":"2024-04-24T23:04:01.445668+0000","last_undegraded":"2024-04-24T23:04:01.445668+0000","last_fullsized":"2024-04-24T23:04:01.445668+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:06:54.614918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.437734+0000","last_change":"2024-04-24T23:04:00.425623+0000","last_active":"2024-04-24T23:04:01.437734+0000","last_peered":"2024-04-24T23:04:01.437734+0000","last_clean":"2024-04-24T23:04:01.437734+0000","last_became_active":"2024-04-24T23:04:00.425153+0000","last_became_peered":"2024-04-24T23:04:00.425153+0000","last_unstale":"2024-04-24T23:04:01.437734+0000","last_undegraded":"2024-04-24T23:04:01.437734+0000","last_fullsized":"2024-04-24T23:04:01.437734+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:40.735104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.445377+0000","last_change":"2024-04-24T23:04:01.444449+0000","last_active":"2024-04-24T23:04:01.445377+0000","last_peered":"2024-04-24T23:04:01.445377+0000","last_clean":"2024-04-24T23:04:01.445377+0000","last_became_active":"2024-04-24T23:04:01.444234+0000","last_became_peered":"2024-04-24T23:04:01.444234+0000","last_unstale":"2024-04-24T23:04:01.445377+0000","last_undegraded":"2024-04-24T23:04:01.445377+0000","last_fullsized":"2024-04-24T23:04:01.445377+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:18.408660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.434197+0000","last_change":"2024-04-24T23:04:00.422891+0000","last_active":"2024-04-24T23:04:01.434197+0000","last_peered":"2024-04-24T23:04:01.434197+0000","last_clean":"2024-04-24T23:04:01.434197+0000","last_became_active":"2024-04-24T23:04:00.422654+0000","last_became_peered":"2024-04-24T23:04:00.422654+0000","last_unstale":"2024-04-24T23:04:01.434197+0000","last_undegraded":"2024-04-24T23:04:01.434197+0000","last_fullsized":"2024-04-24T23:04:01.434197+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:07.066782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.443658+0000","last_change":"2024-04-24T23:04:01.443506+0000","last_active":"2024-04-24T23:04:01.443658+0000","last_peered":"2024-04-24T23:04:01.443658+0000","last_clean":"2024-04-24T23:04:01.443658+0000","last_became_active":"2024-04-24T23:04:01.442780+0000","last_became_peered":"2024-04-24T23:04:01.442780+0000","last_unstale":"2024-04-24T23:04:01.443658+0000","last_undegraded":"2024-04-24T23:04:01.443658+0000","last_fullsized":"2024-04-24T23:04:01.443658+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:12:53.050406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.433839+0000","last_change":"2024-04-24T23:04:00.421012+0000","last_active":"2024-04-24T23:04:01.433839+0000","last_peered":"2024-04-24T23:04:01.433839+0000","last_clean":"2024-04-24T23:04:01.433839+0000","last_became_active":"2024-04-24T23:04:00.420711+0000","last_became_peered":"2024-04-24T23:04:00.420711+0000","last_unstale":"2024-04-24T23:04:01.433839+0000","last_undegraded":"2024-04-24T23:04:01.433839+0000","last_fullsized":"2024-04-24T23:04:01.433839+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:04:04.004076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.441817+0000","last_change":"2024-04-24T23:04:01.441469+0000","last_active":"2024-04-24T23:04:01.441817+0000","last_peered":"2024-04-24T23:04:01.441817+0000","last_clean":"2024-04-24T23:04:01.441817+0000","last_became_active":"2024-04-24T23:04:01.437858+0000","last_became_peered":"2024-04-24T23:04:01.437858+0000","last_unstale":"2024-04-24T23:04:01.441817+0000","last_undegraded":"2024-04-24T23:04:01.441817+0000","last_fullsized":"2024-04-24T23:04:01.441817+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:35.502849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"103'3","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.452388+0000","last_change":"2024-04-24T23:04:00.427467+0000","last_active":"2024-04-24T23:04:01.452388+0000","last_peered":"2024-04-24T23:04:01.452388+0000","last_clean":"2024-04-24T23:04:01.452388+0000","last_became_active":"2024-04-24T23:04:00.426744+0000","last_became_peered":"2024-04-24T23:04:00.426744+0000","last_unstale":"2024-04-24T23:04:01.452388+0000","last_undegraded":"2024-04-24T23:04:01.452388+0000","last_fullsized":"2024-04-24T23:04:01.452388+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:15:53.179365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.441226+0000","last_change":"2024-04-24T23:04:01.441077+0000","last_active":"2024-04-24T23:04:01.441226+0000","last_peered":"2024-04-24T23:04:01.441226+0000","last_clean":"2024-04-24T23:04:01.441226+0000","last_became_active":"2024-04-24T23:04:01.436902+0000","last_became_peered":"2024-04-24T23:04:01.436902+0000","last_unstale":"2024-04-24T23:04:01.441226+0000","last_undegraded":"2024-04-24T23:04:01.441226+0000","last_fullsized":"2024-04-24T23:04:01.441226+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:37:52.956621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"103'6","reported_seq":18,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.454461+0000","last_change":"2024-04-24T23:04:00.426128+0000","last_active":"2024-04-24T23:04:01.454461+0000","last_peered":"2024-04-24T23:04:01.454461+0000","last_clean":"2024-04-24T23:04:01.454461+0000","last_became_active":"2024-04-24T23:04:00.425847+0000","last_became_peered":"2024-04-24T23:04:00.425847+0000","last_unstale":"2024-04-24T23:04:01.454461+0000","last_undegraded":"2024-04-24T23:04:01.454461+0000","last_fullsized":"2024-04-24T23:04:01.454461+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:10.460002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.441787+0000","last_change":"2024-04-24T23:04:01.441143+0000","last_active":"2024-04-24T23:04:01.441787+0000","last_peered":"2024-04-24T23:04:01.441787+0000","last_clean":"2024-04-24T23:04:01.441787+0000","last_became_active":"2024-04-24T23:04:01.439771+0000","last_became_peered":"2024-04-24T23:04:01.439771+0000","last_unstale":"2024-04-24T23:04:01.441787+0000","last_undegraded":"2024-04-24T23:04:01.441787+0000","last_fullsized":"2024-04-24T23:04:01.441787+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:04:14.106634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"103'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.451056+0000","last_change":"2024-04-24T23:04:00.425812+0000","last_active":"2024-04-24T23:04:01.451056+0000","last_peered":"2024-04-24T23:04:01.451056+0000","last_clean":"2024-04-24T23:04:01.451056+0000","last_became_active":"2024-04-24T23:04:00.425544+0000","last_became_peered":"2024-04-24T23:04:00.425544+0000","last_unstale":"2024-04-24T23:04:01.451056+0000","last_undegraded":"2024-04-24T23:04:01.451056+0000","last_fullsized":"2024-04-24T23:04:01.451056+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:43.816546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.439522+0000","last_change":"2024-04-24T23:04:01.439362+0000","last_active":"2024-04-24T23:04:01.439522+0000","last_peered":"2024-04-24T23:04:01.439522+0000","last_clean":"2024-04-24T23:04:01.439522+0000","last_became_active":"2024-04-24T23:04:01.438026+0000","last_became_peered":"2024-04-24T23:04:01.438026+0000","last_unstale":"2024-04-24T23:04:01.439522+0000","last_undegraded":"2024-04-24T23:04:01.439522+0000","last_fullsized":"2024-04-24T23:04:01.439522+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:32.274596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.433045+0000","last_change":"2024-04-24T23:04:00.422482+0000","last_active":"2024-04-24T23:04:01.433045+0000","last_peered":"2024-04-24T23:04:01.433045+0000","last_clean":"2024-04-24T23:04:01.433045+0000","last_became_active":"2024-04-24T23:04:00.422217+0000","last_became_peered":"2024-04-24T23:04:00.422217+0000","last_unstale":"2024-04-24T23:04:01.433045+0000","last_undegraded":"2024-04-24T23:04:01.433045+0000","last_fullsized":"2024-04-24T23:04:01.433045+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:46:50.571382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.444473+0000","last_change":"2024-04-24T23:04:01.444370+0000","last_active":"2024-04-24T23:04:01.444473+0000","last_peered":"2024-04-24T23:04:01.444473+0000","last_clean":"2024-04-24T23:04:01.444473+0000","last_became_active":"2024-04-24T23:04:01.444132+0000","last_became_peered":"2024-04-24T23:04:01.444132+0000","last_unstale":"2024-04-24T23:04:01.444473+0000","last_undegraded":"2024-04-24T23:04:01.444473+0000","last_fullsized":"2024-04-24T23:04:01.444473+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:25:39.221241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.433988+0000","last_change":"2024-04-24T23:04:00.417450+0000","last_active":"2024-04-24T23:04:01.433988+0000","last_peered":"2024-04-24T23:04:01.433988+0000","last_clean":"2024-04-24T23:04:01.433988+0000","last_became_active":"2024-04-24T23:04:00.417215+0000","last_became_peered":"2024-04-24T23:04:00.417215+0000","last_unstale":"2024-04-24T23:04:01.433988+0000","last_undegraded":"2024-04-24T23:04:01.433988+0000","last_fullsized":"2024-04-24T23:04:01.433988+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:28.317728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.436963+0000","last_change":"2024-04-24T23:04:01.436739+0000","last_active":"2024-04-24T23:04:01.436963+0000","last_peered":"2024-04-24T23:04:01.436963+0000","last_clean":"2024-04-24T23:04:01.436963+0000","last_became_active":"2024-04-24T23:04:01.436487+0000","last_became_peered":"2024-04-24T23:04:01.436487+0000","last_unstale":"2024-04-24T23:04:01.436963+0000","last_undegraded":"2024-04-24T23:04:01.436963+0000","last_fullsized":"2024-04-24T23:04:01.436963+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:13:46.686701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"103'8","reported_seq":20,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.455259+0000","last_change":"2024-04-24T23:04:00.417399+0000","last_active":"2024-04-24T23:04:01.455259+0000","last_peered":"2024-04-24T23:04:01.455259+0000","last_clean":"2024-04-24T23:04:01.455259+0000","last_became_active":"2024-04-24T23:04:00.417112+0000","last_became_peered":"2024-04-24T23:04:00.417112+0000","last_unstale":"2024-04-24T23:04:01.455259+0000","last_undegraded":"2024-04-24T23:04:01.455259+0000","last_fullsized":"2024-04-24T23:04:01.455259+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:33:35.187214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.438111+0000","last_change":"2024-04-24T23:04:01.437962+0000","last_active":"2024-04-24T23:04:01.438111+0000","last_peered":"2024-04-24T23:04:01.438111+0000","last_clean":"2024-04-24T23:04:01.438111+0000","last_became_active":"2024-04-24T23:04:01.437711+0000","last_became_peered":"2024-04-24T23:04:01.437711+0000","last_unstale":"2024-04-24T23:04:01.438111+0000","last_undegraded":"2024-04-24T23:04:01.438111+0000","last_fullsized":"2024-04-24T23:04:01.438111+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:52:56.434359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"103'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.451101+0000","last_change":"2024-04-24T23:04:00.427302+0000","last_active":"2024-04-24T23:04:01.451101+0000","last_peered":"2024-04-24T23:04:01.451101+0000","last_clean":"2024-04-24T23:04:01.451101+0000","last_became_active":"2024-04-24T23:04:00.425742+0000","last_became_peered":"2024-04-24T23:04:00.425742+0000","last_unstale":"2024-04-24T23:04:01.451101+0000","last_undegraded":"2024-04-24T23:04:01.451101+0000","last_fullsized":"2024-04-24T23:04:01.451101+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:10:30.189368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.445993+0000","last_change":"2024-04-24T23:04:01.445862+0000","last_active":"2024-04-24T23:04:01.445993+0000","last_peered":"2024-04-24T23:04:01.445993+0000","last_clean":"2024-04-24T23:04:01.445993+0000","last_became_active":"2024-04-24T23:04:01.445603+0000","last_became_peered":"2024-04-24T23:04:01.445603+0000","last_unstale":"2024-04-24T23:04:01.445993+0000","last_undegraded":"2024-04-24T23:04:01.445993+0000","last_fullsized":"2024-04-24T23:04:01.445993+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:51:52.447452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"103'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.451619+0000","last_change":"2024-04-24T23:04:00.424791+0000","last_active":"2024-04-24T23:04:01.451619+0000","last_peered":"2024-04-24T23:04:01.451619+0000","last_clean":"2024-04-24T23:04:01.451619+0000","last_became_active":"2024-04-24T23:04:00.424552+0000","last_became_peered":"2024-04-24T23:04:00.424552+0000","last_unstale":"2024-04-24T23:04:01.451619+0000","last_undegraded":"2024-04-24T23:04:01.451619+0000","last_fullsized":"2024-04-24T23:04:01.451619+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:32:42.658581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.437468+0000","last_change":"2024-04-24T23:04:01.436772+0000","last_active":"2024-04-24T23:04:01.437468+0000","last_peered":"2024-04-24T23:04:01.437468+0000","last_clean":"2024-04-24T23:04:01.437468+0000","last_became_active":"2024-04-24T23:04:01.436545+0000","last_became_peered":"2024-04-24T23:04:01.436545+0000","last_unstale":"2024-04-24T23:04:01.437468+0000","last_undegraded":"2024-04-24T23:04:01.437468+0000","last_fullsized":"2024-04-24T23:04:01.437468+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:17:28.904528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.440745+0000","last_change":"2024-04-24T23:04:00.421402+0000","last_active":"2024-04-24T23:04:01.440745+0000","last_peered":"2024-04-24T23:04:01.440745+0000","last_clean":"2024-04-24T23:04:01.440745+0000","last_became_active":"2024-04-24T23:04:00.421153+0000","last_became_peered":"2024-04-24T23:04:00.421153+0000","last_unstale":"2024-04-24T23:04:01.440745+0000","last_undegraded":"2024-04-24T23:04:01.440745+0000","last_fullsized":"2024-04-24T23:04:01.440745+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:53:16.507098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.437268+0000","last_change":"2024-04-24T23:04:01.436904+0000","last_active":"2024-04-24T23:04:01.437268+0000","last_peered":"2024-04-24T23:04:01.437268+0000","last_clean":"2024-04-24T23:04:01.437268+0000","last_became_active":"2024-04-24T23:04:01.436558+0000","last_became_peered":"2024-04-24T23:04:01.436558+0000","last_unstale":"2024-04-24T23:04:01.437268+0000","last_undegraded":"2024-04-24T23:04:01.437268+0000","last_fullsized":"2024-04-24T23:04:01.437268+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:02.112969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"103'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.451274+0000","last_change":"2024-04-24T23:04:00.423493+0000","last_active":"2024-04-24T23:04:01.451274+0000","last_peered":"2024-04-24T23:04:01.451274+0000","last_clean":"2024-04-24T23:04:01.451274+0000","last_became_active":"2024-04-24T23:04:00.423208+0000","last_became_peered":"2024-04-24T23:04:00.423208+0000","last_unstale":"2024-04-24T23:04:01.451274+0000","last_undegraded":"2024-04-24T23:04:01.451274+0000","last_fullsized":"2024-04-24T23:04:01.451274+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:52:05.454025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.441384+0000","last_change":"2024-04-24T23:04:01.441272+0000","last_active":"2024-04-24T23:04:01.441384+0000","last_peered":"2024-04-24T23:04:01.441384+0000","last_clean":"2024-04-24T23:04:01.441384+0000","last_became_active":"2024-04-24T23:04:01.439999+0000","last_became_peered":"2024-04-24T23:04:01.439999+0000","last_unstale":"2024-04-24T23:04:01.441384+0000","last_undegraded":"2024-04-24T23:04:01.441384+0000","last_fullsized":"2024-04-24T23:04:01.441384+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:32:47.278398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.434745+0000","last_change":"2024-04-24T23:04:00.421533+0000","last_active":"2024-04-24T23:04:01.434745+0000","last_peered":"2024-04-24T23:04:01.434745+0000","last_clean":"2024-04-24T23:04:01.434745+0000","last_became_active":"2024-04-24T23:04:00.421314+0000","last_became_peered":"2024-04-24T23:04:00.421314+0000","last_unstale":"2024-04-24T23:04:01.434745+0000","last_undegraded":"2024-04-24T23:04:01.434745+0000","last_fullsized":"2024-04-24T23:04:01.434745+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:24.258166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.435967+0000","last_change":"2024-04-24T23:04:01.435838+0000","last_active":"2024-04-24T23:04:01.435967+0000","last_peered":"2024-04-24T23:04:01.435967+0000","last_clean":"2024-04-24T23:04:01.435967+0000","last_became_active":"2024-04-24T23:04:01.435492+0000","last_became_peered":"2024-04-24T23:04:01.435492+0000","last_unstale":"2024-04-24T23:04:01.435967+0000","last_undegraded":"2024-04-24T23:04:01.435967+0000","last_fullsized":"2024-04-24T23:04:01.435967+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:55:23.490533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.430108+0000","last_change":"2024-04-24T23:04:00.420668+0000","last_active":"2024-04-24T23:04:01.430108+0000","last_peered":"2024-04-24T23:04:01.430108+0000","last_clean":"2024-04-24T23:04:01.430108+0000","last_became_active":"2024-04-24T23:04:00.420390+0000","last_became_peered":"2024-04-24T23:04:00.420390+0000","last_unstale":"2024-04-24T23:04:01.430108+0000","last_undegraded":"2024-04-24T23:04:01.430108+0000","last_fullsized":"2024-04-24T23:04:01.430108+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:00.214334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.434754+0000","last_change":"2024-04-24T23:04:01.434622+0000","last_active":"2024-04-24T23:04:01.434754+0000","last_peered":"2024-04-24T23:04:01.434754+0000","last_clean":"2024-04-24T23:04:01.434754+0000","last_became_active":"2024-04-24T23:04:01.434369+0000","last_became_peered":"2024-04-24T23:04:01.434369+0000","last_unstale":"2024-04-24T23:04:01.434754+0000","last_undegraded":"2024-04-24T23:04:01.434754+0000","last_fullsized":"2024-04-24T23:04:01.434754+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:09:39.366875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.435290+0000","last_change":"2024-04-24T23:04:00.425589+0000","last_active":"2024-04-24T23:04:01.435290+0000","last_peered":"2024-04-24T23:04:01.435290+0000","last_clean":"2024-04-24T23:04:01.435290+0000","last_became_active":"2024-04-24T23:04:00.425318+0000","last_became_peered":"2024-04-24T23:04:00.425318+0000","last_unstale":"2024-04-24T23:04:01.435290+0000","last_undegraded":"2024-04-24T23:04:01.435290+0000","last_fullsized":"2024-04-24T23:04:01.435290+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:56:55.854071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.442428+0000","last_change":"2024-04-24T23:04:01.442263+0000","last_active":"2024-04-24T23:04:01.442428+0000","last_peered":"2024-04-24T23:04:01.442428+0000","last_clean":"2024-04-24T23:04:01.442428+0000","last_became_active":"2024-04-24T23:04:01.441680+0000","last_became_peered":"2024-04-24T23:04:01.441680+0000","last_unstale":"2024-04-24T23:04:01.442428+0000","last_undegraded":"2024-04-24T23:04:01.442428+0000","last_fullsized":"2024-04-24T23:04:01.442428+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:23:41.534098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"103'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.453246+0000","last_change":"2024-04-24T23:04:00.423120+0000","last_active":"2024-04-24T23:04:01.453246+0000","last_peered":"2024-04-24T23:04:01.453246+0000","last_clean":"2024-04-24T23:04:01.453246+0000","last_became_active":"2024-04-24T23:04:00.422772+0000","last_became_peered":"2024-04-24T23:04:00.422772+0000","last_unstale":"2024-04-24T23:04:01.453246+0000","last_undegraded":"2024-04-24T23:04:01.453246+0000","last_fullsized":"2024-04-24T23:04:01.453246+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:08:54.119256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.437015+0000","last_change":"2024-04-24T23:04:01.436508+0000","last_active":"2024-04-24T23:04:01.437015+0000","last_peered":"2024-04-24T23:04:01.437015+0000","last_clean":"2024-04-24T23:04:01.437015+0000","last_became_active":"2024-04-24T23:04:01.435455+0000","last_became_peered":"2024-04-24T23:04:01.435455+0000","last_unstale":"2024-04-24T23:04:01.437015+0000","last_undegraded":"2024-04-24T23:04:01.437015+0000","last_fullsized":"2024-04-24T23:04:01.437015+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:29:05.440823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.430461+0000","last_change":"2024-04-24T23:04:00.419277+0000","last_active":"2024-04-24T23:04:01.430461+0000","last_peered":"2024-04-24T23:04:01.430461+0000","last_clean":"2024-04-24T23:04:01.430461+0000","last_became_active":"2024-04-24T23:04:00.418982+0000","last_became_peered":"2024-04-24T23:04:00.418982+0000","last_unstale":"2024-04-24T23:04:01.430461+0000","last_undegraded":"2024-04-24T23:04:01.430461+0000","last_fullsized":"2024-04-24T23:04:01.430461+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:14:39.381967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.437240+0000","last_change":"2024-04-24T23:04:01.436680+0000","last_active":"2024-04-24T23:04:01.437240+0000","last_peered":"2024-04-24T23:04:01.437240+0000","last_clean":"2024-04-24T23:04:01.437240+0000","last_became_active":"2024-04-24T23:04:01.436417+0000","last_became_peered":"2024-04-24T23:04:01.436417+0000","last_unstale":"2024-04-24T23:04:01.437240+0000","last_undegraded":"2024-04-24T23:04:01.437240+0000","last_fullsized":"2024-04-24T23:04:01.437240+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:27.715921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.430176+0000","last_change":"2024-04-24T23:04:00.425178+0000","last_active":"2024-04-24T23:04:01.430176+0000","last_peered":"2024-04-24T23:04:01.430176+0000","last_clean":"2024-04-24T23:04:01.430176+0000","last_became_active":"2024-04-24T23:04:00.424823+0000","last_became_peered":"2024-04-24T23:04:00.424823+0000","last_unstale":"2024-04-24T23:04:01.430176+0000","last_undegraded":"2024-04-24T23:04:01.430176+0000","last_fullsized":"2024-04-24T23:04:01.430176+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:48:08.435444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.441031+0000","last_change":"2024-04-24T23:04:01.440874+0000","last_active":"2024-04-24T23:04:01.441031+0000","last_peered":"2024-04-24T23:04:01.441031+0000","last_clean":"2024-04-24T23:04:01.441031+0000","last_became_active":"2024-04-24T23:04:01.440612+0000","last_became_peered":"2024-04-24T23:04:01.440612+0000","last_unstale":"2024-04-24T23:04:01.441031+0000","last_undegraded":"2024-04-24T23:04:01.441031+0000","last_fullsized":"2024-04-24T23:04:01.441031+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:06.229980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.433230+0000","last_change":"2024-04-24T23:04:00.424131+0000","last_active":"2024-04-24T23:04:01.433230+0000","last_peered":"2024-04-24T23:04:01.433230+0000","last_clean":"2024-04-24T23:04:01.433230+0000","last_became_active":"2024-04-24T23:04:00.423892+0000","last_became_peered":"2024-04-24T23:04:00.423892+0000","last_unstale":"2024-04-24T23:04:01.433230+0000","last_undegraded":"2024-04-24T23:04:01.433230+0000","last_fullsized":"2024-04-24T23:04:01.433230+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:42:06.589207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.441443+0000","last_change":"2024-04-24T23:04:01.441284+0000","last_active":"2024-04-24T23:04:01.441443+0000","last_peered":"2024-04-24T23:04:01.441443+0000","last_clean":"2024-04-24T23:04:01.441443+0000","last_became_active":"2024-04-24T23:04:01.440502+0000","last_became_peered":"2024-04-24T23:04:01.440502+0000","last_unstale":"2024-04-24T23:04:01.441443+0000","last_undegraded":"2024-04-24T23:04:01.441443+0000","last_fullsized":"2024-04-24T23:04:01.441443+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:44:18.526320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.433205+0000","last_change":"2024-04-24T23:04:00.420535+0000","last_active":"2024-04-24T23:04:01.433205+0000","last_peered":"2024-04-24T23:04:01.433205+0000","last_clean":"2024-04-24T23:04:01.433205+0000","last_became_active":"2024-04-24T23:04:00.420297+0000","last_became_peered":"2024-04-24T23:04:00.420297+0000","last_unstale":"2024-04-24T23:04:01.433205+0000","last_undegraded":"2024-04-24T23:04:01.433205+0000","last_fullsized":"2024-04-24T23:04:01.433205+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:08:19.176173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.441898+0000","last_change":"2024-04-24T23:04:01.441769+0000","last_active":"2024-04-24T23:04:01.441898+0000","last_peered":"2024-04-24T23:04:01.441898+0000","last_clean":"2024-04-24T23:04:01.441898+0000","last_became_active":"2024-04-24T23:04:01.440669+0000","last_became_peered":"2024-04-24T23:04:01.440669+0000","last_unstale":"2024-04-24T23:04:01.441898+0000","last_undegraded":"2024-04-24T23:04:01.441898+0000","last_fullsized":"2024-04-24T23:04:01.441898+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:11:48.038878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.434134+0000","last_change":"2024-04-24T23:04:00.422871+0000","last_active":"2024-04-24T23:04:01.434134+0000","last_peered":"2024-04-24T23:04:01.434134+0000","last_clean":"2024-04-24T23:04:01.434134+0000","last_became_active":"2024-04-24T23:04:00.422597+0000","last_became_peered":"2024-04-24T23:04:00.422597+0000","last_unstale":"2024-04-24T23:04:01.434134+0000","last_undegraded":"2024-04-24T23:04:01.434134+0000","last_fullsized":"2024-04-24T23:04:01.434134+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:36:02.204842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.438595+0000","last_change":"2024-04-24T23:04:01.438478+0000","last_active":"2024-04-24T23:04:01.438595+0000","last_peered":"2024-04-24T23:04:01.438595+0000","last_clean":"2024-04-24T23:04:01.438595+0000","last_became_active":"2024-04-24T23:04:01.437979+0000","last_became_peered":"2024-04-24T23:04:01.437979+0000","last_unstale":"2024-04-24T23:04:01.438595+0000","last_undegraded":"2024-04-24T23:04:01.438595+0000","last_fullsized":"2024-04-24T23:04:01.438595+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:14:39.523826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.436676+0000","last_change":"2024-04-24T23:04:00.427010+0000","last_active":"2024-04-24T23:04:01.436676+0000","last_peered":"2024-04-24T23:04:01.436676+0000","last_clean":"2024-04-24T23:04:01.436676+0000","last_became_active":"2024-04-24T23:04:00.426200+0000","last_became_peered":"2024-04-24T23:04:00.426200+0000","last_unstale":"2024-04-24T23:04:01.436676+0000","last_undegraded":"2024-04-24T23:04:01.436676+0000","last_fullsized":"2024-04-24T23:04:01.436676+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:14.674089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.444160+0000","last_change":"2024-04-24T23:04:01.443960+0000","last_active":"2024-04-24T23:04:01.444160+0000","last_peered":"2024-04-24T23:04:01.444160+0000","last_clean":"2024-04-24T23:04:01.444160+0000","last_became_active":"2024-04-24T23:04:01.442098+0000","last_became_peered":"2024-04-24T23:04:01.442098+0000","last_unstale":"2024-04-24T23:04:01.444160+0000","last_undegraded":"2024-04-24T23:04:01.444160+0000","last_fullsized":"2024-04-24T23:04:01.444160+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:13.239828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.430318+0000","last_change":"2024-04-24T23:04:00.427378+0000","last_active":"2024-04-24T23:04:01.430318+0000","last_peered":"2024-04-24T23:04:01.430318+0000","last_clean":"2024-04-24T23:04:01.430318+0000","last_became_active":"2024-04-24T23:04:00.426725+0000","last_became_peered":"2024-04-24T23:04:00.426725+0000","last_unstale":"2024-04-24T23:04:01.430318+0000","last_undegraded":"2024-04-24T23:04:01.430318+0000","last_fullsized":"2024-04-24T23:04:01.430318+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:02:25.647213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.443760+0000","last_change":"2024-04-24T23:04:01.443631+0000","last_active":"2024-04-24T23:04:01.443760+0000","last_peered":"2024-04-24T23:04:01.443760+0000","last_clean":"2024-04-24T23:04:01.443760+0000","last_became_active":"2024-04-24T23:04:01.443335+0000","last_became_peered":"2024-04-24T23:04:01.443335+0000","last_unstale":"2024-04-24T23:04:01.443760+0000","last_undegraded":"2024-04-24T23:04:01.443760+0000","last_fullsized":"2024-04-24T23:04:01.443760+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:00.410574+0000","last_clean_scrub_stamp":"2024-04-24T23:04:00.410574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:43:40.099887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"103'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.451433+0000","last_change":"2024-04-24T23:04:00.422932+0000","last_active":"2024-04-24T23:04:01.451433+0000","last_peered":"2024-04-24T23:04:01.451433+0000","last_clean":"2024-04-24T23:04:01.451433+0000","last_became_active":"2024-04-24T23:04:00.422679+0000","last_became_peered":"2024-04-24T23:04:00.422679+0000","last_unstale":"2024-04-24T23:04:01.451433+0000","last_undegraded":"2024-04-24T23:04:01.451433+0000","last_fullsized":"2024-04-24T23:04:01.451433+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:03:59.406525+0000","last_clean_scrub_stamp":"2024-04-24T23:03:59.406525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:37.706560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":332,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T23:04:01.433460+0000","last_change":"2024-04-24T22:58:55.138810+0000","last_active":"2024-04-24T23:04:01.433460+0000","last_peered":"2024-04-24T23:04:01.433460+0000","last_clean":"2024-04-24T23:04:01.433460+0000","last_became_active":"2024-04-24T22:58:55.138627+0000","last_became_peered":"2024-04-24T22:58:55.138627+0000","last_unstale":"2024-04-24T23:04:01.433460+0000","last_undegraded":"2024-04-24T23:04:01.433460+0000","last_fullsized":"2024-04-24T23:04:01.433460+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_clean_scrub_stamp":"2024-04-24T22:58:54.127235+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:24:10.033365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574912,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":547,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194764,"statfs":{"total":200000000,"available":199439233,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574912,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":540,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194771,"statfs":{"total":200000000,"available":199446078,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574912,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1117,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194195,"statfs":{"total":200000000,"available":198855710,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574912,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":540,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194771,"statfs":{"total":200000000,"available":199446078,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574912,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":548,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194764,"statfs":{"total":200000000,"available":199438629,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574912,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":547,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194764,"statfs":{"total":200000000,"available":199439303,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1117,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194195,"statfs":{"total":200000000,"available":198855710,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574912,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":549,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194763,"statfs":{"total":200000000,"available":199437669,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-24T23:04:05.786 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-24T23:04:05.786 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-24T23:04:05.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.928+0000 7f1236729640 1 Processor -- start 2024-04-24T23:04:05.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.928+0000 7f1236729640 1 -- start start 2024-04-24T23:04:05.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.929+0000 7f1236729640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f123006cd00 0x7f12300ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:05.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.929+0000 7f1236729640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f123015cb00 0x7f12300ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:05.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.929+0000 7f1236729640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12300b4520 con 0x7f123015b380 2024-04-24T23:04:05.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.929+0000 7f1236729640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f12300b4800 con 0x7f123006cd00 2024-04-24T23:04:05.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.929+0000 7f1236729640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f12300b4ae0 con 0x7f123015cb00 2024-04-24T23:04:05.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.929+0000 7f122f7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f123006cd00 0x7f12300ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:05.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.929+0000 7f122f7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f123006cd00 0x7f12300ce430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60236/0 (socket says 172.21.15.32:60236) 2024-04-24T23:04:05.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.929+0000 7f122f7fe640 1 -- 172.21.15.32:0/3939295567 learned_addr learned my addr 172.21.15.32:0/3939295567 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:05.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.930+0000 7f1234cc6640 1 --2- 172.21.15.32:0/3939295567 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f123015cb00 0x7f12300ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:05.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.930+0000 7f122f7fe640 1 -- 172.21.15.32:0/3939295567 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f123015cb00 msgr2=0x7f12300ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:05.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.930+0000 7f122f7fe640 1 --2- 172.21.15.32:0/3939295567 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f123015cb00 0x7f12300ce970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:05.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.930+0000 7f122f7fe640 1 -- 172.21.15.32:0/3939295567 >> v1:172.21.15.121:6789/0 conn(0x7f123015b380 legacy=0x7f123006c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:05.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.930+0000 7f122f7fe640 1 -- 172.21.15.32:0/3939295567 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12300b4dc0 con 0x7f123006cd00 2024-04-24T23:04:05.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.930+0000 7f1234cc6640 1 --2- 172.21.15.32:0/3939295567 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f123015cb00 0x7f12300ce970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:04:05.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.930+0000 7f122f7fe640 1 --2- 172.21.15.32:0/3939295567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f123006cd00 0x7f12300ce430 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f122400bb10 tx=0x7f122402c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=433013e3861475c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:05.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.931+0000 7f122effd640 1 -- 172.21.15.32:0/3939295567 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1224032790 con 0x7f123006cd00 2024-04-24T23:04:05.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.931+0000 7f122effd640 1 -- 172.21.15.32:0/3939295567 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1224038070 con 0x7f123006cd00 2024-04-24T23:04:05.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.931+0000 7f122effd640 1 -- 172.21.15.32:0/3939295567 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1224032b50 con 0x7f123006cd00 2024-04-24T23:04:05.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.931+0000 7f1236729640 1 -- 172.21.15.32:0/3939295567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f123006cd00 msgr2=0x7f12300ce430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:05.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.931+0000 7f1236729640 1 --2- 172.21.15.32:0/3939295567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f123006cd00 0x7f12300ce430 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f122400bb10 tx=0x7f122402c890 comp rx=0 tx=0).stop 2024-04-24T23:04:05.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.931+0000 7f1236729640 1 -- 172.21.15.32:0/3939295567 shutdown_connections 2024-04-24T23:04:05.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.931+0000 7f1236729640 1 --2- 172.21.15.32:0/3939295567 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f123015cb00 0x7f12300ce970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:05.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.931+0000 7f1236729640 1 --2- 172.21.15.32:0/3939295567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f123006cd00 0x7f12300ce430 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:05.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.931+0000 7f1236729640 1 -- 172.21.15.32:0/3939295567 >> 172.21.15.32:0/3939295567 conn(0x7f123006b110 msgr2=0x7f123006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:05.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.931+0000 7f1236729640 1 -- 172.21.15.32:0/3939295567 shutdown_connections 2024-04-24T23:04:05.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.931+0000 7f1236729640 1 -- 172.21.15.32:0/3939295567 wait complete. 2024-04-24T23:04:05.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.932+0000 7f1236729640 1 Processor -- start 2024-04-24T23:04:05.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.933+0000 7f1236729640 1 -- start start 2024-04-24T23:04:05.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.933+0000 7f1236729640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f123006cd00 0x7f12301e7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:05.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.933+0000 7f1236729640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f123015b380 0x7f12300ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:05.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.933+0000 7f1236729640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f123015cb00 0x7f12300aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:05.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.933+0000 7f1236729640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f123015b760 con 0x7f123015b380 2024-04-24T23:04:05.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.933+0000 7f1236729640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f12300ceeb0 con 0x7f123006cd00 2024-04-24T23:04:05.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.933+0000 7f1236729640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f12300b55c0 con 0x7f123015cb00 2024-04-24T23:04:05.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.933+0000 7f122f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f123015b380 0x7f12300ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:05.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.933+0000 7f122f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f123015b380 0x7f12300ae8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60246/0 (socket says 172.21.15.32:60246) 2024-04-24T23:04:05.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.933+0000 7f122f7fe640 1 -- 172.21.15.32:0/3844900407 learned_addr learned my addr 172.21.15.32:0/3844900407 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:05.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.934+0000 7f1234cc6640 1 --2- 172.21.15.32:0/3844900407 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f123015cb00 0x7f12300aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:05.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.934+0000 7f122ffff640 1 --2- 172.21.15.32:0/3844900407 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f123006cd00 0x7f12301e7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:05.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.934+0000 7f122f7fe640 1 -- 172.21.15.32:0/3844900407 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f123015cb00 msgr2=0x7f12300aedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:05.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.934+0000 7f122f7fe640 1 --2- 172.21.15.32:0/3844900407 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f123015cb00 0x7f12300aedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:05.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.934+0000 7f122f7fe640 1 -- 172.21.15.32:0/3844900407 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f123006cd00 msgr2=0x7f12301e7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:05.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.934+0000 7f122f7fe640 1 --2- 172.21.15.32:0/3844900407 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f123006cd00 0x7f12301e7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:05.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.934+0000 7f122f7fe640 1 -- 172.21.15.32:0/3844900407 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12301f81b0 con 0x7f123015b380 2024-04-24T23:04:05.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.934+0000 7f122ffff640 1 --2- 172.21.15.32:0/3844900407 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f123006cd00 0x7f12301e7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:04:05.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.934+0000 7f122f7fe640 1 --2- 172.21.15.32:0/3844900407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f123015b380 0x7f12300ae8b0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f122402c8d0 tx=0x7f1224004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:05.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.934+0000 7f122cff9640 1 -- 172.21.15.32:0/3844900407 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1224043070 con 0x7f123015b380 2024-04-24T23:04:05.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.934+0000 7f122cff9640 1 -- 172.21.15.32:0/3844900407 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f122403c830 con 0x7f123015b380 2024-04-24T23:04:05.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.934+0000 7f1236729640 1 -- 172.21.15.32:0/3844900407 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12301f8420 con 0x7f123015b380 2024-04-24T23:04:05.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.934+0000 7f122cff9640 1 -- 172.21.15.32:0/3844900407 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122403cc20 con 0x7f123015b380 2024-04-24T23:04:05.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.934+0000 7f1236729640 1 -- 172.21.15.32:0/3844900407 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12301f89e0 con 0x7f123015b380 2024-04-24T23:04:05.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.935+0000 7f1236729640 1 -- 172.21.15.32:0/3844900407 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12301e7c10 con 0x7f123015b380 2024-04-24T23:04:05.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.935+0000 7f122cff9640 1 -- 172.21.15.32:0/3844900407 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1224046070 con 0x7f123015b380 2024-04-24T23:04:05.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.936+0000 7f122cff9640 1 --2- 172.21.15.32:0/3844900407 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f121c06be30 0x7f121c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:05.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.936+0000 7f122cff9640 1 -- 172.21.15.32:0/3844900407 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7f12240ab4e0 con 0x7f123015b380 2024-04-24T23:04:05.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.936+0000 7f122ffff640 1 --2- 172.21.15.32:0/3844900407 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f121c06be30 0x7f121c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:05.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.937+0000 7f122ffff640 1 --2- 172.21.15.32:0/3844900407 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f121c06be30 0x7f121c06e2f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f1220001680 tx=0x7f1220009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:05.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:05.940+0000 7f122cff9640 1 -- 172.21.15.32:0/3844900407 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f12301e7c10 con 0x7f123015b380 2024-04-24T23:04:06.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.196+0000 7f1236729640 1 -- 172.21.15.32:0/3844900407 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f1230068430 con 0x7f123015b380 2024-04-24T23:04:06.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.198+0000 7f122cff9640 1 -- 172.21.15.32:0/3844900407 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) ==== 190+0+147 (secure 0 0 0) 0x7f1224079f20 con 0x7f123015b380 2024-04-24T23:04:06.199 INFO:teuthology.orchestra.run.smithi032.stdout:[client.0] 2024-04-24T23:04:06.200 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQCejilm5oeiGRAAdLXwwqrYhV3HHo7Dw/2ZqQ== 2024-04-24T23:04:06.200 INFO:teuthology.orchestra.run.smithi032.stdout: caps mds = "allow" 2024-04-24T23:04:06.200 INFO:teuthology.orchestra.run.smithi032.stdout: caps mon = "allow r" 2024-04-24T23:04:06.200 INFO:teuthology.orchestra.run.smithi032.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-24T23:04:06.200 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2024-04-24T23:04:06.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.200+0000 7f1236729640 1 -- 172.21.15.32:0/3844900407 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f121c06be30 msgr2=0x7f121c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:06.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.200+0000 7f1236729640 1 --2- 172.21.15.32:0/3844900407 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f121c06be30 0x7f121c06e2f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f1220001680 tx=0x7f1220009040 comp rx=0 tx=0).stop 2024-04-24T23:04:06.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.200+0000 7f1236729640 1 -- 172.21.15.32:0/3844900407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f123015b380 msgr2=0x7f12300ae8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:06.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.200+0000 7f1236729640 1 --2- 172.21.15.32:0/3844900407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f123015b380 0x7f12300ae8b0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f122402c8d0 tx=0x7f1224004730 comp rx=0 tx=0).stop 2024-04-24T23:04:06.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.200+0000 7f1236729640 1 -- 172.21.15.32:0/3844900407 shutdown_connections 2024-04-24T23:04:06.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.200+0000 7f1236729640 1 --2- 172.21.15.32:0/3844900407 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f121c06be30 0x7f121c06e2f0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:06.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.200+0000 7f1236729640 1 --2- 172.21.15.32:0/3844900407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f123015b380 0x7f12300ae8b0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:06.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.200+0000 7f1236729640 1 --2- 172.21.15.32:0/3844900407 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f123015cb00 0x7f12300aedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:06.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.200+0000 7f1236729640 1 --2- 172.21.15.32:0/3844900407 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f123006cd00 0x7f12301e7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:06.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.200+0000 7f1236729640 1 -- 172.21.15.32:0/3844900407 >> 172.21.15.32:0/3844900407 conn(0x7f123006b110 msgr2=0x7f123006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:06.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.200+0000 7f1236729640 1 -- 172.21.15.32:0/3844900407 shutdown_connections 2024-04-24T23:04:06.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.200+0000 7f1236729640 1 -- 172.21.15.32:0/3844900407 wait complete. 2024-04-24T23:04:06.213 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:04:06.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.312+0000 7f5d401e7640 1 Processor -- start 2024-04-24T23:04:06.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.312+0000 7f5d401e7640 1 -- start start 2024-04-24T23:04:06.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.313+0000 7f5d401e7640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5d3806b820 0x7f5d380d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:06.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.313+0000 7f5d401e7640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5d380d4210 0x7f5d380b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:06.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.313+0000 7f5d401e7640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d380d3170 con 0x7f5d3815e9b0 2024-04-24T23:04:06.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.313+0000 7f5d401e7640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f5d380d3450 con 0x7f5d3806b820 2024-04-24T23:04:06.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.313+0000 7f5d401e7640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f5d380d3730 con 0x7f5d380d4210 2024-04-24T23:04:06.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.314+0000 7f5d3e784640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5d380d4210 0x7f5d380b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:06.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.314+0000 7f5d3e784640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5d380d4210 0x7f5d380b78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.32:33818/0 (socket says 172.21.15.32:33818) 2024-04-24T23:04:06.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.314+0000 7f5d3e784640 1 -- 172.21.15.32:0/3006495805 learned_addr learned my addr 172.21.15.32:0/3006495805 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:06.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.314+0000 7f5d3d782640 1 --2- 172.21.15.32:0/3006495805 >> v2:172.21.15.32:3300/0 conn(0x7f5d3806b820 0x7f5d380d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:06.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.314+0000 7f5d3d782640 1 -- 172.21.15.32:0/3006495805 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5d380d4210 msgr2=0x7f5d380b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:06.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.314+0000 7f5d3d782640 1 --2- 172.21.15.32:0/3006495805 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5d380d4210 0x7f5d380b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:06.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.314+0000 7f5d3d782640 1 -- 172.21.15.32:0/3006495805 >> v1:172.21.15.32:6789/0 conn(0x7f5d3815e9b0 legacy=0x7f5d3806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:06.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.314+0000 7f5d3d782640 1 -- 172.21.15.32:0/3006495805 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d380d3a10 con 0x7f5d3806b820 2024-04-24T23:04:06.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.314+0000 7f5d3e784640 1 --2- 172.21.15.32:0/3006495805 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5d380d4210 0x7f5d380b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:04:06.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.315+0000 7f5d3d782640 1 --2- 172.21.15.32:0/3006495805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d3806b820 0x7f5d380d2b90 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c0097a0 tx=0x7f5d2c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=214f5a0abc040c00 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:06.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.315+0000 7f5d3cf81640 1 -- 172.21.15.32:0/3006495805 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c037070 con 0x7f5d3806b820 2024-04-24T23:04:06.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.315+0000 7f5d3cf81640 1 -- 172.21.15.32:0/3006495805 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d2c032440 con 0x7f5d3806b820 2024-04-24T23:04:06.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.315+0000 7f5d3cf81640 1 -- 172.21.15.32:0/3006495805 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c032810 con 0x7f5d3806b820 2024-04-24T23:04:06.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.316+0000 7f5d401e7640 1 -- 172.21.15.32:0/3006495805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d3806b820 msgr2=0x7f5d380d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:06.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.316+0000 7f5d401e7640 1 --2- 172.21.15.32:0/3006495805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d3806b820 0x7f5d380d2b90 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c0097a0 tx=0x7f5d2c02a180 comp rx=0 tx=0).stop 2024-04-24T23:04:06.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.316+0000 7f5d401e7640 1 -- 172.21.15.32:0/3006495805 shutdown_connections 2024-04-24T23:04:06.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.316+0000 7f5d401e7640 1 --2- 172.21.15.32:0/3006495805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d3806b820 0x7f5d380d2b90 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:06.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.316+0000 7f5d401e7640 1 --2- 172.21.15.32:0/3006495805 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5d380d4210 0x7f5d380b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:06.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.316+0000 7f5d401e7640 1 -- 172.21.15.32:0/3006495805 >> 172.21.15.32:0/3006495805 conn(0x7f5d380d11d0 msgr2=0x7f5d380d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:06.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.316+0000 7f5d401e7640 1 -- 172.21.15.32:0/3006495805 shutdown_connections 2024-04-24T23:04:06.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.316+0000 7f5d401e7640 1 -- 172.21.15.32:0/3006495805 wait complete. 2024-04-24T23:04:06.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.317+0000 7f5d401e7640 1 Processor -- start 2024-04-24T23:04:06.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.317+0000 7f5d401e7640 1 -- start start 2024-04-24T23:04:06.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.318+0000 7f5d401e7640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5d3806b820 0x7f5d380cd500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:06.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.318+0000 7f5d401e7640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d380d4210 0x7f5d381f4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:06.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.318+0000 7f5d401e7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d3815e9b0 0x7f5d38154e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:06.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.318+0000 7f5d401e7640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5d380ba0e0 con 0x7f5d3815e9b0 2024-04-24T23:04:06.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.318+0000 7f5d401e7640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5d380d2400 con 0x7f5d380d4210 2024-04-24T23:04:06.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.318+0000 7f5d401e7640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f5d380ba260 con 0x7f5d3806b820 2024-04-24T23:04:06.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.318+0000 7f5d3e784640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d3815e9b0 0x7f5d38154e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:06.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.318+0000 7f5d3e784640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d3815e9b0 0x7f5d38154e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60266/0 (socket says 172.21.15.32:60266) 2024-04-24T23:04:06.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.318+0000 7f5d3e784640 1 -- 172.21.15.32:0/318310396 learned_addr learned my addr 172.21.15.32:0/318310396 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:06.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.318+0000 7f5d3df83640 1 --2- 172.21.15.32:0/318310396 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5d3806b820 0x7f5d380cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:06.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.318+0000 7f5d3d782640 1 --2- 172.21.15.32:0/318310396 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d380d4210 0x7f5d381f4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:06.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.318+0000 7f5d3e784640 1 -- 172.21.15.32:0/318310396 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5d3806b820 msgr2=0x7f5d380cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:06.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.318+0000 7f5d3e784640 1 --2- 172.21.15.32:0/318310396 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5d3806b820 0x7f5d380cd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:06.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.318+0000 7f5d3e784640 1 -- 172.21.15.32:0/318310396 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d380d4210 msgr2=0x7f5d381f4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:06.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.318+0000 7f5d3e784640 1 --2- 172.21.15.32:0/318310396 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d380d4210 0x7f5d381f4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:06.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.318+0000 7f5d3e784640 1 -- 172.21.15.32:0/318310396 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d381f49b0 con 0x7f5d3815e9b0 2024-04-24T23:04:06.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.318+0000 7f5d3e784640 1 --2- 172.21.15.32:0/318310396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d3815e9b0 0x7f5d38154e10 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f5d3000efc0 tx=0x7f5d3000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:06.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.319+0000 7f5d2a7fc640 1 -- 172.21.15.32:0/318310396 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3000cdc0 con 0x7f5d3815e9b0 2024-04-24T23:04:06.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.319+0000 7f5d2a7fc640 1 -- 172.21.15.32:0/318310396 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d30019070 con 0x7f5d3815e9b0 2024-04-24T23:04:06.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.319+0000 7f5d2a7fc640 1 -- 172.21.15.32:0/318310396 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d300146c0 con 0x7f5d3815e9b0 2024-04-24T23:04:06.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.319+0000 7f5d401e7640 1 -- 172.21.15.32:0/318310396 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d381f4c50 con 0x7f5d3815e9b0 2024-04-24T23:04:06.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.319+0000 7f5d401e7640 1 -- 172.21.15.32:0/318310396 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d381f5210 con 0x7f5d3815e9b0 2024-04-24T23:04:06.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.320+0000 7f5d401e7640 1 -- 172.21.15.32:0/318310396 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d3815a440 con 0x7f5d3815e9b0 2024-04-24T23:04:06.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.320+0000 7f5d2a7fc640 1 -- 172.21.15.32:0/318310396 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5d30005d10 con 0x7f5d3815e9b0 2024-04-24T23:04:06.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.321+0000 7f5d2a7fc640 1 --2- 172.21.15.32:0/318310396 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5d1c06be30 0x7f5d1c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:06.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.321+0000 7f5d2a7fc640 1 -- 172.21.15.32:0/318310396 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7f5d3008b1c0 con 0x7f5d3815e9b0 2024-04-24T23:04:06.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.321+0000 7f5d3df83640 1 --2- 172.21.15.32:0/318310396 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5d1c06be30 0x7f5d1c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:06.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.321+0000 7f5d3df83640 1 --2- 172.21.15.32:0/318310396 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5d1c06be30 0x7f5d1c06e2f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f5d340016e0 tx=0x7f5d34009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:06.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.329+0000 7f5d2a7fc640 1 -- 172.21.15.32:0/318310396 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5d3815a440 con 0x7f5d3815e9b0 2024-04-24T23:04:06.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.552+0000 7f5d401e7640 1 -- 172.21.15.32:0/318310396 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5d38068430 con 0x7f5d3815e9b0 2024-04-24T23:04:06.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.553+0000 7f5d2a7fc640 1 -- 172.21.15.32:0/318310396 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5186 (secure 0 0 0) 0x7f5d30010040 con 0x7f5d3815e9b0 2024-04-24T23:04:06.555 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:04:06.555 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":44,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6175,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/434273738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":434273738},{"type":"v1","addr":"172.21.15.121:6836","nonce":434273738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6178,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1104557785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1104557785},{"type":"v1","addr":"172.21.15.121:6837","nonce":1104557785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6888,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/3137388886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3137388886},{"type":"v1","addr":"172.21.15.32:6834","nonce":3137388886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:04:00.783579+0000","modified":"2024-04-24T23:04:03.790658+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6891},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6891":{"gid":6891,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/3327203898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":3327203898},{"type":"v1","addr":"172.21.15.32:6835","nonce":3327203898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6891,"qdb_cluster":[6891]},"id":6}]} 2024-04-24T23:04:06.555 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 44 2024-04-24T23:04:06.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.555+0000 7f5d401e7640 1 -- 172.21.15.32:0/318310396 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5d1c06be30 msgr2=0x7f5d1c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:06.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.555+0000 7f5d401e7640 1 --2- 172.21.15.32:0/318310396 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5d1c06be30 0x7f5d1c06e2f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f5d340016e0 tx=0x7f5d34009040 comp rx=0 tx=0).stop 2024-04-24T23:04:06.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.555+0000 7f5d401e7640 1 -- 172.21.15.32:0/318310396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d3815e9b0 msgr2=0x7f5d38154e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:06.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.555+0000 7f5d401e7640 1 --2- 172.21.15.32:0/318310396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d3815e9b0 0x7f5d38154e10 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f5d3000efc0 tx=0x7f5d3000c490 comp rx=0 tx=0).stop 2024-04-24T23:04:06.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.555+0000 7f5d401e7640 1 -- 172.21.15.32:0/318310396 shutdown_connections 2024-04-24T23:04:06.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.555+0000 7f5d401e7640 1 --2- 172.21.15.32:0/318310396 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5d1c06be30 0x7f5d1c06e2f0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:06.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.555+0000 7f5d401e7640 1 --2- 172.21.15.32:0/318310396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d3815e9b0 0x7f5d38154e10 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:06.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.555+0000 7f5d401e7640 1 --2- 172.21.15.32:0/318310396 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d380d4210 0x7f5d381f4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:06.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.555+0000 7f5d401e7640 1 --2- 172.21.15.32:0/318310396 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5d3806b820 0x7f5d380cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:06.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.555+0000 7f5d401e7640 1 -- 172.21.15.32:0/318310396 >> 172.21.15.32:0/318310396 conn(0x7f5d380d11d0 msgr2=0x7f5d38159d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:06.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.555+0000 7f5d401e7640 1 -- 172.21.15.32:0/318310396 shutdown_connections 2024-04-24T23:04:06.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:06.555+0000 7f5d401e7640 1 -- 172.21.15.32:0/318310396 wait complete. 2024-04-24T23:04:06.755 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:04:06.757 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:04:06.970 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 44, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T23:04:00.783579+0000', 'modified': '2024-04-24T23:04:03.790658+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6891}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6891': {'gid': 6891, 'name': 'a', 'rank': 0, 'incarnation': 41, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/3327203898', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6833', 'nonce': 3327203898}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 3327203898}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6891, 'qdb_cluster': [6891]} 2024-04-24T23:04:06.970 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-24T23:04:06.971 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-24T23:04:07.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.071+0000 7fcd7fde9640 1 Processor -- start 2024-04-24T23:04:07.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.072+0000 7fcd7fde9640 1 -- start start 2024-04-24T23:04:07.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.072+0000 7fcd7fde9640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcd7806b820 0x7fcd780d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:07.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.072+0000 7fcd7fde9640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fcd780d4210 0x7fcd780b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:07.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.072+0000 7fcd7fde9640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd780d3170 con 0x7fcd7815e9b0 2024-04-24T23:04:07.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.072+0000 7fcd7fde9640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fcd780d3450 con 0x7fcd780d4210 2024-04-24T23:04:07.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.072+0000 7fcd7fde9640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fcd780d3730 con 0x7fcd7806b820 2024-04-24T23:04:07.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.072+0000 7fcd7db85640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcd7815e9b0 0x7fcd7806b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42640/0 (socket says 172.21.15.32:42640) 2024-04-24T23:04:07.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.073+0000 7fcd7db85640 1 -- 172.21.15.32:0/1077633511 learned_addr learned my addr 172.21.15.32:0/1077633511 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:04:07.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.073+0000 7fcd7e386640 1 --2- 172.21.15.32:0/1077633511 >> v2:172.21.15.121:3300/0 conn(0x7fcd780d4210 0x7fcd780b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:07.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.073+0000 7fcd7d384640 1 --2- 172.21.15.32:0/1077633511 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcd7806b820 0x7fcd780d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:07.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.073+0000 7fcd7cb83640 1 -- 172.21.15.32:0/1077633511 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 295088093 0 0) 0x7fcd780d3170 con 0x7fcd7815e9b0 2024-04-24T23:04:07.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.073+0000 7fcd7cb83640 1 -- 172.21.15.32:0/1077633511 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcd64003520 con 0x7fcd7815e9b0 2024-04-24T23:04:07.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.073+0000 7fcd7cb83640 1 -- 172.21.15.32:0/1077633511 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3035222573 0 0) 0x7fcd64003520 con 0x7fcd7815e9b0 2024-04-24T23:04:07.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.073+0000 7fcd7cb83640 1 -- 172.21.15.32:0/1077633511 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcd7806b820 msgr2=0x7fcd780d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:07.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.073+0000 7fcd7cb83640 1 --2- 172.21.15.32:0/1077633511 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcd7806b820 0x7fcd780d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:07.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.073+0000 7fcd7cb83640 1 -- 172.21.15.32:0/1077633511 >> v2:172.21.15.121:3300/0 conn(0x7fcd780d4210 msgr2=0x7fcd780b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:07.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.073+0000 7fcd7cb83640 1 --2- 172.21.15.32:0/1077633511 >> v2:172.21.15.121:3300/0 conn(0x7fcd780d4210 0x7fcd780b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:07.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.073+0000 7fcd7cb83640 1 -- 172.21.15.32:0/1077633511 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd780d3a10 con 0x7fcd7815e9b0 2024-04-24T23:04:07.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.073+0000 7fcd7cb83640 1 -- 172.21.15.32:0/1077633511 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7fcd74001670 con 0x7fcd7815e9b0 2024-04-24T23:04:07.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.073+0000 7fcd7d384640 1 --2- 172.21.15.32:0/1077633511 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcd7806b820 0x7fcd780d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:04:07.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.073+0000 7fcd7cb83640 1 -- 172.21.15.32:0/1077633511 >> v1:172.21.15.32:6789/0 conn(0x7fcd7815e9b0 legacy=0x7fcd7806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:07.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.073+0000 7fcd7cb83640 1 --2- 172.21.15.32:0/1077633511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd640039b0 0x7fcd64003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:07.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.073+0000 7fcd7cb83640 1 --2- 172.21.15.32:0/1077633511 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcd7806b820 0x7fcd64004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:07.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.073+0000 7fcd7cb83640 1 --2- 172.21.15.32:0/1077633511 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd64004d40 0x7fcd640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:07.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.073+0000 7fcd7cb83640 1 -- 172.21.15.32:0/1077633511 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcd780d3450 con 0x7fcd640039b0 2024-04-24T23:04:07.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.073+0000 7fcd7cb83640 1 -- 172.21.15.32:0/1077633511 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fcd780d3730 con 0x7fcd64004d40 2024-04-24T23:04:07.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.073+0000 7fcd7cb83640 1 -- 172.21.15.32:0/1077633511 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fcd6400b9c0 con 0x7fcd7806b820 2024-04-24T23:04:07.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.073+0000 7fcd7db85640 1 --2- 172.21.15.32:0/1077633511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd640039b0 0x7fcd64003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:07.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.073+0000 7fcd7d384640 1 --2- 172.21.15.32:0/1077633511 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcd7806b820 0x7fcd64004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:07.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.073+0000 7fcd7e386640 1 --2- 172.21.15.32:0/1077633511 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd64004d40 0x7fcd640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:07.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.074+0000 7fcd7db85640 1 -- 172.21.15.32:0/1077633511 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcd7806b820 msgr2=0x7fcd64004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:07.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.074+0000 7fcd7db85640 1 --2- 172.21.15.32:0/1077633511 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcd7806b820 0x7fcd64004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:07.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.074+0000 7fcd7db85640 1 -- 172.21.15.32:0/1077633511 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd64004d40 msgr2=0x7fcd640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:07.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.074+0000 7fcd7db85640 1 --2- 172.21.15.32:0/1077633511 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd64004d40 0x7fcd640051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:07.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.074+0000 7fcd7db85640 1 -- 172.21.15.32:0/1077633511 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd64003520 con 0x7fcd640039b0 2024-04-24T23:04:07.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.074+0000 7fcd7e386640 1 --2- 172.21.15.32:0/1077633511 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd64004d40 0x7fcd640051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:04:07.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.074+0000 7fcd7db85640 1 --2- 172.21.15.32:0/1077633511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd640039b0 0x7fcd64003dd0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fcd7400c240 tx=0x7fcd64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e6a5e79d91976c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:07.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.074+0000 7fcd7cb83640 1 -- 172.21.15.32:0/1077633511 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7402e070 con 0x7fcd640039b0 2024-04-24T23:04:07.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.074+0000 7fcd7cb83640 1 -- 172.21.15.32:0/1077633511 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd74005af0 con 0x7fcd640039b0 2024-04-24T23:04:07.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.074+0000 7fcd7cb83640 1 -- 172.21.15.32:0/1077633511 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd74007d30 con 0x7fcd640039b0 2024-04-24T23:04:07.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.075+0000 7fcd7fde9640 1 -- 172.21.15.32:0/1077633511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd640039b0 msgr2=0x7fcd64003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:07.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.075+0000 7fcd7fde9640 1 --2- 172.21.15.32:0/1077633511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd640039b0 0x7fcd64003dd0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fcd7400c240 tx=0x7fcd64005830 comp rx=0 tx=0).stop 2024-04-24T23:04:07.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.075+0000 7fcd7e386640 1 -- 172.21.15.32:0/1077633511 reap_dead start 2024-04-24T23:04:07.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.075+0000 7fcd7fde9640 1 -- 172.21.15.32:0/1077633511 shutdown_connections 2024-04-24T23:04:07.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.075+0000 7fcd7fde9640 1 -- 172.21.15.32:0/1077633511 >> 172.21.15.32:0/1077633511 conn(0x7fcd780d11d0 msgr2=0x7fcd780d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:07.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.075+0000 7fcd7fde9640 1 -- 172.21.15.32:0/1077633511 shutdown_connections 2024-04-24T23:04:07.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.075+0000 7fcd7fde9640 1 -- 172.21.15.32:0/1077633511 wait complete. 2024-04-24T23:04:07.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.076+0000 7fcd7fde9640 1 Processor -- start 2024-04-24T23:04:07.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.076+0000 7fcd7fde9640 1 -- start start 2024-04-24T23:04:07.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.077+0000 7fcd7fde9640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd640039b0 0x7fcd781529c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:07.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.077+0000 7fcd7fde9640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcd7815b790 0x7fcd781f7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:07.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.077+0000 7fcd7fde9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd78158870 0x7fcd78160f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:07.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.077+0000 7fcd7fde9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcd780ba0e0 con 0x7fcd78158870 2024-04-24T23:04:07.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.077+0000 7fcd7fde9640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fcd780d2400 con 0x7fcd640039b0 2024-04-24T23:04:07.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.077+0000 7fcd7fde9640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fcd780ba260 con 0x7fcd7815b790 2024-04-24T23:04:07.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.077+0000 7fcd7e386640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd78158870 0x7fcd78160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:07.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.077+0000 7fcd7e386640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd78158870 0x7fcd78160f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60276/0 (socket says 172.21.15.32:60276) 2024-04-24T23:04:07.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.077+0000 7fcd7e386640 1 -- 172.21.15.32:0/44031835 learned_addr learned my addr 172.21.15.32:0/44031835 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:07.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.077+0000 7fcd7d384640 1 --2- 172.21.15.32:0/44031835 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcd7815b790 0x7fcd781f7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:07.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.077+0000 7fcd7db85640 1 --2- 172.21.15.32:0/44031835 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd640039b0 0x7fcd781529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:07.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.078+0000 7fcd7e386640 1 -- 172.21.15.32:0/44031835 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcd7815b790 msgr2=0x7fcd781f7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:07.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.078+0000 7fcd7e386640 1 --2- 172.21.15.32:0/44031835 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcd7815b790 0x7fcd781f7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:07.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.078+0000 7fcd7e386640 1 -- 172.21.15.32:0/44031835 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd640039b0 msgr2=0x7fcd781529c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:07.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.078+0000 7fcd7e386640 1 --2- 172.21.15.32:0/44031835 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd640039b0 0x7fcd781529c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:07.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.078+0000 7fcd7e386640 1 -- 172.21.15.32:0/44031835 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd781f8430 con 0x7fcd78158870 2024-04-24T23:04:07.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.078+0000 7fcd7db85640 1 --2- 172.21.15.32:0/44031835 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd640039b0 0x7fcd781529c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:04:07.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.078+0000 7fcd7e386640 1 --2- 172.21.15.32:0/44031835 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd78158870 0x7fcd78160f60 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fcd781622f0 tx=0x7fcd70010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:07.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.078+0000 7fcd6e7fc640 1 -- 172.21.15.32:0/44031835 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7001a030 con 0x7fcd78158870 2024-04-24T23:04:07.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.078+0000 7fcd6e7fc640 1 -- 172.21.15.32:0/44031835 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd70010d80 con 0x7fcd78158870 2024-04-24T23:04:07.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.078+0000 7fcd7fde9640 1 -- 172.21.15.32:0/44031835 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd78152fa0 con 0x7fcd78158870 2024-04-24T23:04:07.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.078+0000 7fcd6e7fc640 1 -- 172.21.15.32:0/44031835 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd70003bd0 con 0x7fcd78158870 2024-04-24T23:04:07.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.078+0000 7fcd7fde9640 1 -- 172.21.15.32:0/44031835 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd78153560 con 0x7fcd78158870 2024-04-24T23:04:07.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.079+0000 7fcd6e7fc640 1 -- 172.21.15.32:0/44031835 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcd70005d10 con 0x7fcd78158870 2024-04-24T23:04:07.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.080+0000 7fcd6e7fc640 1 --2- 172.21.15.32:0/44031835 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcd5c06be30 0x7fcd5c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:07.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.080+0000 7fcd6e7fc640 1 -- 172.21.15.32:0/44031835 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7fcd7008b820 con 0x7fcd78158870 2024-04-24T23:04:07.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.080+0000 7fcd7db85640 1 --2- 172.21.15.32:0/44031835 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcd5c06be30 0x7fcd5c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:07.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.081+0000 7fcd7db85640 1 --2- 172.21.15.32:0/44031835 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcd5c06be30 0x7fcd5c06e2f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fcd74033000 tx=0x7fcd74009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:07.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.081+0000 7fcd82bcb740 1 Processor -- start 2024-04-24T23:04:07.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.081+0000 7fcd82bcb740 1 -- start start 2024-04-24T23:04:07.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.082+0000 7fcd82bcb740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d508e38b60 0x55d508deb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:07.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.082+0000 7fcd82bcb740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d508e1bc80 0x55d508e1b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:07.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.082+0000 7fcd82bcb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d508e1b540 0x55d508df0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:07.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.082+0000 7fcd82bcb740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55d508d53f00 con 0x55d508e1bc80 2024-04-24T23:04:07.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.082+0000 7fcd82bcb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d508cfc750 con 0x55d508e1b540 2024-04-24T23:04:07.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.082+0000 7fcd82bcb740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x55d508d4fec0 con 0x55d508e38b60 2024-04-24T23:04:07.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.082+0000 7fcd7d384640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d508e1bc80 0x55d508e1b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:07.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.082+0000 7fcd7d384640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d508e1bc80 0x55d508e1b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60292/0 (socket says 172.21.15.32:60292) 2024-04-24T23:04:07.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.082+0000 7fcd7d384640 1 -- 172.21.15.32:0/275314455 learned_addr learned my addr 172.21.15.32:0/275314455 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:07.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.082+0000 7fcd7e386640 1 --2- 172.21.15.32:0/275314455 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d508e1b540 0x55d508df0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:07.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.082+0000 7fcd7db85640 1 --2- 172.21.15.32:0/275314455 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d508e38b60 0x55d508deb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:07.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.082+0000 7fcd7d384640 1 -- 172.21.15.32:0/275314455 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d508e38b60 msgr2=0x55d508deb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:07.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.082+0000 7fcd7d384640 1 --2- 172.21.15.32:0/275314455 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d508e38b60 0x55d508deb4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:07.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.082+0000 7fcd7d384640 1 -- 172.21.15.32:0/275314455 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d508e1b540 msgr2=0x55d508df0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:07.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.082+0000 7fcd7d384640 1 --2- 172.21.15.32:0/275314455 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d508e1b540 0x55d508df0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:07.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.082+0000 7fcd7d384640 1 -- 172.21.15.32:0/275314455 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d508a682d0 con 0x55d508e1bc80 2024-04-24T23:04:07.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.082+0000 7fcd7d384640 1 --2- 172.21.15.32:0/275314455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d508e1bc80 0x55d508e1b000 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fcd68009ce0 tx=0x7fcd68014780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba2bb4522debe88 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:07.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.083+0000 7fcd5b7fe640 1 -- 172.21.15.32:0/275314455 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd68004020 con 0x55d508e1bc80 2024-04-24T23:04:07.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.083+0000 7fcd5b7fe640 1 -- 172.21.15.32:0/275314455 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd68019070 con 0x55d508e1bc80 2024-04-24T23:04:07.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.083+0000 7fcd5b7fe640 1 -- 172.21.15.32:0/275314455 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd680043e0 con 0x55d508e1bc80 2024-04-24T23:04:07.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.083+0000 7fcd82bcb740 1 -- 172.21.15.32:0/275314455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d508e1bc80 msgr2=0x55d508e1b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:07.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.083+0000 7fcd82bcb740 1 --2- 172.21.15.32:0/275314455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d508e1bc80 0x55d508e1b000 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fcd68009ce0 tx=0x7fcd68014780 comp rx=0 tx=0).stop 2024-04-24T23:04:07.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.083+0000 7fcd82bcb740 1 -- 172.21.15.32:0/275314455 shutdown_connections 2024-04-24T23:04:07.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.083+0000 7fcd82bcb740 1 --2- 172.21.15.32:0/275314455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d508e1bc80 0x55d508e1b000 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:07.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.083+0000 7fcd82bcb740 1 --2- 172.21.15.32:0/275314455 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d508e1b540 0x55d508df0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:07.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.083+0000 7fcd82bcb740 1 --2- 172.21.15.32:0/275314455 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d508e38b60 0x55d508deb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:07.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.083+0000 7fcd82bcb740 1 -- 172.21.15.32:0/275314455 >> 172.21.15.32:0/275314455 conn(0x55d508ded760 msgr2=0x55d508e36000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:07.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.083+0000 7fcd82bcb740 1 -- 172.21.15.32:0/275314455 shutdown_connections 2024-04-24T23:04:07.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.083+0000 7fcd82bcb740 1 -- 172.21.15.32:0/275314455 wait complete. 2024-04-24T23:04:07.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.084+0000 7fcd82bcb740 1 -- start start 2024-04-24T23:04:07.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.084+0000 7fcd82bcb740 1 Processor -- start 2024-04-24T23:04:07.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.085+0000 7fcd82bcb740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d508e1bc80 0x55d508e12f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:07.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.085+0000 7fcd82bcb740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d508e38b60 0x55d508e4a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:07.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.085+0000 7fcd82bcb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d508e2d420 0x55d508defd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:07.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.085+0000 7fcd82bcb740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55d508def910 con 0x55d508e1bc80 2024-04-24T23:04:07.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.085+0000 7fcd82bcb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d508d0fcf0 con 0x55d508e2d420 2024-04-24T23:04:07.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.085+0000 7fcd82bcb740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x55d508deed00 con 0x55d508e38b60 2024-04-24T23:04:07.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.085+0000 7fcd82bcb740 10 client.0 _ll_register_callbacks cb 0x55d508d0f6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:04:07.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.085+0000 7fcd7db85640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d508e1bc80 0x55d508e12f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:07.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.085+0000 7fcd7db85640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d508e1bc80 0x55d508e12f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60308/0 (socket says 172.21.15.32:60308) 2024-04-24T23:04:07.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.085+0000 7fcd7db85640 1 -- 172.21.15.32:0/1545371442 learned_addr learned my addr 172.21.15.32:0/1545371442 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:07.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.085+0000 7fcd7d384640 1 --2- 172.21.15.32:0/1545371442 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d508e38b60 0x55d508e4a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:07.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.085+0000 7fcd7e386640 1 --2- 172.21.15.32:0/1545371442 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d508e2d420 0x55d508defd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:07.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.085+0000 7fcd7db85640 1 -- 172.21.15.32:0/1545371442 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d508e38b60 msgr2=0x55d508e4a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:07.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.085+0000 7fcd7db85640 1 --2- 172.21.15.32:0/1545371442 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d508e38b60 0x55d508e4a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:07.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.085+0000 7fcd7db85640 1 -- 172.21.15.32:0/1545371442 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d508e2d420 msgr2=0x55d508defd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:07.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.085+0000 7fcd7db85640 1 --2- 172.21.15.32:0/1545371442 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d508e2d420 0x55d508defd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:07.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.085+0000 7fcd7db85640 1 -- 172.21.15.32:0/1545371442 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d508b326b0 con 0x55d508e1bc80 2024-04-24T23:04:07.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.086+0000 7fcd7db85640 1 --2- 172.21.15.32:0/1545371442 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d508e1bc80 0x55d508e12f20 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fcd7402ae30 tx=0x7fcd780d3340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee2faa67bdfdfedf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:07.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.086+0000 7fcd58ff9640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-24T23:04:07.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.086+0000 7fcd58ff9640 1 -- 172.21.15.32:0/1545371442 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd74043070 con 0x55d508e1bc80 2024-04-24T23:04:07.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.086+0000 7fcd58ff9640 1 -- 172.21.15.32:0/1545371442 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7fcd7403f630 con 0x55d508e1bc80 2024-04-24T23:04:07.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.086+0000 7fcd82bcb740 1 -- 172.21.15.32:0/1545371442 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd68019070 con 0x55d508e1bc80 2024-04-24T23:04:07.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.086+0000 7fcd58ff9640 1 -- 172.21.15.32:0/1545371442 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fcd7403f830 con 0x55d508e1bc80 2024-04-24T23:04:07.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.086+0000 7fcd82bcb740 10 client.7035 fetch_fsmap learned FSMap version 44 2024-04-24T23:04:07.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.086+0000 7fcd82bcb740 1 -- 172.21.15.32:0/1545371442 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) -- 0x55d508e2e640 con 0x55d508e1bc80 2024-04-24T23:04:07.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.086+0000 7fcd58ff9640 1 -- 172.21.15.32:0/1545371442 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd68019070 con 0x55d508e1bc80 2024-04-24T23:04:07.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.086+0000 7fcd58ff9640 1 -- 172.21.15.32:0/1545371442 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 44) ==== 3494+0+0 (secure 0 0 0) 0x7fcd74047070 con 0x55d508e1bc80 2024-04-24T23:04:07.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.086+0000 7fcd82bcb740 10 client.7035 fetch_fsmap finished waiting for FSMap version 44 2024-04-24T23:04:07.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.086+0000 7fcd82bcb740 10 client.7035 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T23:04:07.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.086+0000 7fcd82bcb740 20 client.7035 populate_metadata read hostname 'smithi032' 2024-04-24T23:04:07.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.086+0000 7fcd82bcb740 1 --2- 172.21.15.32:0/1545371442 >> [v2:172.21.15.32:6833/3327203898,v1:172.21.15.32:6835/3327203898] conn(0x55d508e1c520 0x55d508e1de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:07.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.086+0000 7fcd82bcb740 4 client.7035 mds_command: new command op to 6891 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:04:07.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.086+0000 7fcd82bcb740 1 -- 172.21.15.32:0/1545371442 --> [v2:172.21.15.32:6833/3327203898,v1:172.21.15.32:6835/3327203898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d508d61440 con 0x55d508e1c520 2024-04-24T23:04:07.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.086+0000 7fcd7d384640 1 --2- 172.21.15.32:0/1545371442 >> [v2:172.21.15.32:6833/3327203898,v1:172.21.15.32:6835/3327203898] conn(0x55d508e1c520 0x55d508e1de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:07.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.087+0000 7fcd58ff9640 10 client.7035 ms_handle_connect on v2:172.21.15.32:6833/3327203898 2024-04-24T23:04:07.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.087+0000 7fcd7d384640 1 --2- 172.21.15.32:0/1545371442 >> [v2:172.21.15.32:6833/3327203898,v1:172.21.15.32:6835/3327203898] conn(0x55d508e1c520 0x55d508e1de40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cc59ce370692a05 server_cookie=a6bb233af5221845 in_seq=0 out_seq=0 2024-04-24T23:04:07.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.088+0000 7fcd58ff9640 1 -- 172.21.15.32:0/1545371442 <== mds.0 v2:172.21.15.32:6833/3327203898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d508d61440 con 0x55d508e1c520 2024-04-24T23:04:07.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.088+0000 7fcd58ff9640 10 client.7035 handle_command_reply: tid=0 2024-04-24T23:04:07.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.088+0000 7fcd82bcb740 1 client.7035 shutdown 2024-04-24T23:04:07.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.088+0000 7fcd82bcb740 1 -- 172.21.15.32:0/1545371442 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d508e1bc80 msgr2=0x55d508e12f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:07.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.088+0000 7fcd82bcb740 1 --2- 172.21.15.32:0/1545371442 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d508e1bc80 0x55d508e12f20 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fcd7402ae30 tx=0x7fcd780d3340 comp rx=0 tx=0).stop 2024-04-24T23:04:07.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.088+0000 7fcd82bcb740 1 -- 172.21.15.32:0/1545371442 shutdown_connections 2024-04-24T23:04:07.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.088+0000 7fcd82bcb740 1 --2- 172.21.15.32:0/1545371442 >> [v2:172.21.15.32:6833/3327203898,v1:172.21.15.32:6835/3327203898] conn(0x55d508e1c520 0x55d508e1de40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:07.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.088+0000 7fcd82bcb740 1 --2- 172.21.15.32:0/1545371442 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d508e2d420 0x55d508defd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:07.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.088+0000 7fcd82bcb740 1 --2- 172.21.15.32:0/1545371442 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d508e38b60 0x55d508e4a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:07.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.088+0000 7fcd58ff9640 0 client.7035 ms_handle_reset on v2:172.21.15.32:6833/3327203898 2024-04-24T23:04:07.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.088+0000 7fcd82bcb740 1 --2- 172.21.15.32:0/1545371442 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d508e1bc80 0x55d508e12f20 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:07.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.088+0000 7fcd82bcb740 1 -- 172.21.15.32:0/1545371442 >> 172.21.15.32:0/1545371442 conn(0x55d508e1b540 msgr2=0x55d508deabe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:07.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.088+0000 7fcd82bcb740 1 -- 172.21.15.32:0/1545371442 shutdown_connections 2024-04-24T23:04:07.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.088+0000 7fcd82bcb740 1 -- 172.21.15.32:0/1545371442 wait complete. 2024-04-24T23:04:07.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.088+0000 7fcd82bcb740 20 client.7035 trim_cache size 0 max 16384 2024-04-24T23:04:07.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.106+0000 7fcd82bcb740 1 Processor -- start 2024-04-24T23:04:07.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.106+0000 7fcd82bcb740 1 -- start start 2024-04-24T23:04:07.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.107+0000 7fcd82bcb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d508e1bc80 0x55d508e38460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:07.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.107+0000 7fcd82bcb740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d508e2d420 0x55d508e35100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:07.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.107+0000 7fcd82bcb740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d508e38b60 0x55d508e2bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:07.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.107+0000 7fcd82bcb740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcd782003e0 con 0x55d508e2d420 2024-04-24T23:04:07.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.107+0000 7fcd82bcb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fcd781ff870 con 0x55d508e1bc80 2024-04-24T23:04:07.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.107+0000 7fcd82bcb740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fcd781ffdc0 con 0x55d508e38b60 2024-04-24T23:04:07.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.107+0000 7fcd7d384640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d508e2d420 0x55d508e35100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:07.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.107+0000 7fcd7d384640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d508e2d420 0x55d508e35100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60314/0 (socket says 172.21.15.32:60314) 2024-04-24T23:04:07.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.107+0000 7fcd7d384640 1 -- 172.21.15.32:0/1787141186 learned_addr learned my addr 172.21.15.32:0/1787141186 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:07.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.107+0000 7fcd7d384640 1 -- 172.21.15.32:0/1787141186 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d508e38b60 msgr2=0x55d508e2bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:07.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.107+0000 7fcd7d384640 1 --2- 172.21.15.32:0/1787141186 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d508e38b60 0x55d508e2bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:07.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.107+0000 7fcd7d384640 1 -- 172.21.15.32:0/1787141186 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d508e1bc80 msgr2=0x55d508e38460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:07.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.107+0000 7fcd7db85640 1 --2- 172.21.15.32:0/1787141186 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d508e1bc80 0x55d508e38460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:07.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.107+0000 7fcd7d384640 1 --2- 172.21.15.32:0/1787141186 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d508e1bc80 0x55d508e38460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:07.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.108+0000 7fcd7d384640 1 -- 172.21.15.32:0/1787141186 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d508dee8a0 con 0x55d508e2d420 2024-04-24T23:04:07.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.108+0000 7fcd7d384640 1 --2- 172.21.15.32:0/1787141186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d508e2d420 0x55d508e35100 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fcd6801b000 tx=0x7fcd68017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5daf34703c7c20b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:07.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.108+0000 7fcd5affd640 1 -- 172.21.15.32:0/1787141186 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd680040c0 con 0x55d508e2d420 2024-04-24T23:04:07.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.108+0000 7fcd5affd640 1 -- 172.21.15.32:0/1787141186 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd68019040 con 0x55d508e2d420 2024-04-24T23:04:07.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.108+0000 7fcd5affd640 1 -- 172.21.15.32:0/1787141186 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd68004480 con 0x55d508e2d420 2024-04-24T23:04:07.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.108+0000 7fcd82bcb740 1 -- 172.21.15.32:0/1787141186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d508e2d420 msgr2=0x55d508e35100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:07.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.108+0000 7fcd82bcb740 1 --2- 172.21.15.32:0/1787141186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d508e2d420 0x55d508e35100 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fcd6801b000 tx=0x7fcd68017000 comp rx=0 tx=0).stop 2024-04-24T23:04:07.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.108+0000 7fcd82bcb740 1 -- 172.21.15.32:0/1787141186 shutdown_connections 2024-04-24T23:04:07.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.108+0000 7fcd82bcb740 1 --2- 172.21.15.32:0/1787141186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d508e2d420 0x55d508e35100 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:07.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.108+0000 7fcd82bcb740 1 --2- 172.21.15.32:0/1787141186 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d508e38b60 0x55d508e2bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:07.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.108+0000 7fcd82bcb740 1 --2- 172.21.15.32:0/1787141186 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d508e1bc80 0x55d508e38460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:07.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.108+0000 7fcd82bcb740 1 -- 172.21.15.32:0/1787141186 >> 172.21.15.32:0/1787141186 conn(0x55d508e1b540 msgr2=0x55d508de6690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:07.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.109+0000 7fcd82bcb740 1 -- 172.21.15.32:0/1787141186 shutdown_connections 2024-04-24T23:04:07.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.109+0000 7fcd82bcb740 1 -- 172.21.15.32:0/1787141186 wait complete. 2024-04-24T23:04:07.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.109+0000 7fcd82bcb740 1 -- start start 2024-04-24T23:04:07.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.109+0000 7fcd82bcb740 1 Processor -- start 2024-04-24T23:04:07.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.110+0000 7fcd82bcb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d508e1bc80 0x55d508deef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:07.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.110+0000 7fcd82bcb740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d508e2d420 0x55d508def480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:07.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.110+0000 7fcd82bcb740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d508e38b60 0x55d508e174b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:07.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.110+0000 7fcd82bcb740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55d508e1c3a0 con 0x55d508e38b60 2024-04-24T23:04:07.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.110+0000 7fcd82bcb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d508e1e9c0 con 0x55d508e1bc80 2024-04-24T23:04:07.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.110+0000 7fcd82bcb740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x55d508e1c930 con 0x55d508e2d420 2024-04-24T23:04:07.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.110+0000 7fcd82bcb740 10 client.0 _ll_register_callbacks cb 0x7fcd68009060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:04:07.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.110+0000 7fcd7e386640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d508e38b60 0x55d508e174b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:07.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.110+0000 7fcd7e386640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d508e38b60 0x55d508e174b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60330/0 (socket says 172.21.15.32:60330) 2024-04-24T23:04:07.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.110+0000 7fcd7e386640 1 -- 172.21.15.32:0/2312988710 learned_addr learned my addr 172.21.15.32:0/2312988710 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:07.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.110+0000 7fcd7d384640 1 --2- 172.21.15.32:0/2312988710 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d508e2d420 0x55d508def480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:07.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.110+0000 7fcd7db85640 1 --2- 172.21.15.32:0/2312988710 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d508e1bc80 0x55d508deef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:07.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.110+0000 7fcd7e386640 1 -- 172.21.15.32:0/2312988710 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d508e2d420 msgr2=0x55d508def480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:07.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.110+0000 7fcd7e386640 1 --2- 172.21.15.32:0/2312988710 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d508e2d420 0x55d508def480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:07.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.110+0000 7fcd7e386640 1 -- 172.21.15.32:0/2312988710 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d508e1bc80 msgr2=0x55d508deef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:07.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.110+0000 7fcd7e386640 1 --2- 172.21.15.32:0/2312988710 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d508e1bc80 0x55d508deef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:07.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.110+0000 7fcd7e386640 1 -- 172.21.15.32:0/2312988710 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d508e170c0 con 0x55d508e38b60 2024-04-24T23:04:07.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.111+0000 7fcd7e386640 1 --2- 172.21.15.32:0/2312988710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d508e38b60 0x55d508e174b0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fcd70007300 tx=0x7fcd7005a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6111ef9cbcce048d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:07.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.111+0000 7fcd58ff9640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-24T23:04:07.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.111+0000 7fcd58ff9640 1 -- 172.21.15.32:0/2312988710 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd700626a0 con 0x55d508e38b60 2024-04-24T23:04:07.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.111+0000 7fcd58ff9640 1 -- 172.21.15.32:0/2312988710 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7fcd70059d20 con 0x55d508e38b60 2024-04-24T23:04:07.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.111+0000 7fcd82bcb740 1 -- 172.21.15.32:0/2312988710 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd68019040 con 0x55d508e38b60 2024-04-24T23:04:07.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.111+0000 7fcd58ff9640 1 -- 172.21.15.32:0/2312988710 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fcd70078e60 con 0x55d508e38b60 2024-04-24T23:04:07.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.111+0000 7fcd82bcb740 10 client.7041 fetch_fsmap learned FSMap version 44 2024-04-24T23:04:07.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.111+0000 7fcd82bcb740 1 -- 172.21.15.32:0/2312988710 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) -- 0x55d508e59bf0 con 0x55d508e38b60 2024-04-24T23:04:07.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.111+0000 7fcd58ff9640 1 -- 172.21.15.32:0/2312988710 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd68019040 con 0x55d508e38b60 2024-04-24T23:04:07.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.111+0000 7fcd58ff9640 1 -- 172.21.15.32:0/2312988710 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 44) ==== 3494+0+0 (secure 0 0 0) 0x7fcd70062210 con 0x55d508e38b60 2024-04-24T23:04:07.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.111+0000 7fcd82bcb740 10 client.7041 fetch_fsmap finished waiting for FSMap version 44 2024-04-24T23:04:07.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.111+0000 7fcd82bcb740 10 client.7041 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T23:04:07.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.111+0000 7fcd82bcb740 20 client.7041 populate_metadata read hostname 'smithi032' 2024-04-24T23:04:07.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.111+0000 7fcd82bcb740 1 --2- 172.21.15.32:0/2312988710 >> [v2:172.21.15.32:6833/3327203898,v1:172.21.15.32:6835/3327203898] conn(0x55d508e58bb0 0x55d508e58f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:07.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.111+0000 7fcd82bcb740 4 client.7041 mds_command: new command op to 6891 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:04:07.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.111+0000 7fcd82bcb740 1 -- 172.21.15.32:0/2312988710 --> [v2:172.21.15.32:6833/3327203898,v1:172.21.15.32:6835/3327203898] -- command(tid 0: {"prefix": "status"}) -- 0x55d508d5b790 con 0x55d508e58bb0 2024-04-24T23:04:07.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.111+0000 7fcd7db85640 1 --2- 172.21.15.32:0/2312988710 >> [v2:172.21.15.32:6833/3327203898,v1:172.21.15.32:6835/3327203898] conn(0x55d508e58bb0 0x55d508e58f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:07.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.112+0000 7fcd58ff9640 10 client.7041 ms_handle_connect on v2:172.21.15.32:6833/3327203898 2024-04-24T23:04:07.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.112+0000 7fcd7db85640 1 --2- 172.21.15.32:0/2312988710 >> [v2:172.21.15.32:6833/3327203898,v1:172.21.15.32:6835/3327203898] conn(0x55d508e58bb0 0x55d508e58f90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4cd3bfb09083253 server_cookie=6053cf86bb6d7b6e in_seq=0 out_seq=0 2024-04-24T23:04:07.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.112+0000 7fcd58ff9640 1 -- 172.21.15.32:0/2312988710 <== mds.0 v2:172.21.15.32:6833/3327203898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x55d508d5b790 con 0x55d508e58bb0 2024-04-24T23:04:07.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.112+0000 7fcd58ff9640 10 client.7041 handle_command_reply: tid=0 2024-04-24T23:04:07.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.112+0000 7fcd82bcb740 1 client.7041 shutdown 2024-04-24T23:04:07.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.112+0000 7fcd82bcb740 1 -- 172.21.15.32:0/2312988710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d508e38b60 msgr2=0x55d508e174b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:07.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.112+0000 7fcd82bcb740 1 --2- 172.21.15.32:0/2312988710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d508e38b60 0x55d508e174b0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fcd70007300 tx=0x7fcd7005a2d0 comp rx=0 tx=0).stop 2024-04-24T23:04:07.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.112+0000 7fcd82bcb740 1 -- 172.21.15.32:0/2312988710 shutdown_connections 2024-04-24T23:04:07.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.112+0000 7fcd82bcb740 1 --2- 172.21.15.32:0/2312988710 >> [v2:172.21.15.32:6833/3327203898,v1:172.21.15.32:6835/3327203898] conn(0x55d508e58bb0 0x55d508e58f90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:07.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.112+0000 7fcd82bcb740 1 --2- 172.21.15.32:0/2312988710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d508e38b60 0x55d508e174b0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:07.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.112+0000 7fcd82bcb740 1 --2- 172.21.15.32:0/2312988710 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d508e2d420 0x55d508def480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:07.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.112+0000 7fcd58ff9640 0 client.7041 ms_handle_reset on v2:172.21.15.32:6833/3327203898 2024-04-24T23:04:07.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.112+0000 7fcd82bcb740 1 --2- 172.21.15.32:0/2312988710 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d508e1bc80 0x55d508deef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:07.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.112+0000 7fcd82bcb740 1 -- 172.21.15.32:0/2312988710 >> 172.21.15.32:0/2312988710 conn(0x55d508e1b540 msgr2=0x55d508e35100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:07.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.112+0000 7fcd82bcb740 1 -- 172.21.15.32:0/2312988710 shutdown_connections 2024-04-24T23:04:07.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.112+0000 7fcd82bcb740 1 -- 172.21.15.32:0/2312988710 wait complete. 2024-04-24T23:04:07.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.112+0000 7fcd82bcb740 20 client.7041 trim_cache size 0 max 16384 2024-04-24T23:04:07.119 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:04:07.119 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", 2024-04-24T23:04:07.120 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2024-04-24T23:04:07.120 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 6891, 2024-04-24T23:04:07.120 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-04-24T23:04:07.120 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-04-24T23:04:07.120 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-04-24T23:04:07.120 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 5.6744528729999999, 2024-04-24T23:04:07.120 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 44, 2024-04-24T23:04:07.120 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 104, 2024-04-24T23:04:07.120 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 104, 2024-04-24T23:04:07.120 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 10.573425070000001 2024-04-24T23:04:07.120 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:04:07.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.113+0000 7fcd7fde9640 1 -- 172.21.15.32:0/44031835 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcd5c06be30 msgr2=0x7fcd5c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:07.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.113+0000 7fcd7fde9640 1 --2- 172.21.15.32:0/44031835 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcd5c06be30 0x7fcd5c06e2f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fcd74033000 tx=0x7fcd74009030 comp rx=0 tx=0).stop 2024-04-24T23:04:07.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.113+0000 7fcd7fde9640 1 -- 172.21.15.32:0/44031835 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd78158870 msgr2=0x7fcd78160f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:07.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.113+0000 7fcd7fde9640 1 --2- 172.21.15.32:0/44031835 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd78158870 0x7fcd78160f60 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fcd781622f0 tx=0x7fcd70010590 comp rx=0 tx=0).stop 2024-04-24T23:04:07.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.113+0000 7fcd7fde9640 1 -- 172.21.15.32:0/44031835 shutdown_connections 2024-04-24T23:04:07.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.113+0000 7fcd7fde9640 1 --2- 172.21.15.32:0/44031835 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcd5c06be30 0x7fcd5c06e2f0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:07.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.113+0000 7fcd7fde9640 1 --2- 172.21.15.32:0/44031835 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd78158870 0x7fcd78160f60 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:07.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.113+0000 7fcd7fde9640 1 --2- 172.21.15.32:0/44031835 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcd7815b790 0x7fcd781f7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:07.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.113+0000 7fcd7fde9640 1 --2- 172.21.15.32:0/44031835 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd640039b0 0x7fcd781529c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:07.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.113+0000 7fcd7fde9640 1 -- 172.21.15.32:0/44031835 >> 172.21.15.32:0/44031835 conn(0x7fcd780d11d0 msgr2=0x7fcd780b8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:07.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.113+0000 7fcd7fde9640 1 -- 172.21.15.32:0/44031835 shutdown_connections 2024-04-24T23:04:07.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.113+0000 7fcd7fde9640 1 -- 172.21.15.32:0/44031835 wait complete. 2024-04-24T23:04:07.125 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T23:04:07.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.225+0000 7f808d928640 1 Processor -- start 2024-04-24T23:04:07.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.225+0000 7f808d928640 1 -- start start 2024-04-24T23:04:07.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.226+0000 7f808d928640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f80880ba390 0x7f80880d40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:07.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.226+0000 7f808d928640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80880d3010 con 0x7f808815e9e0 2024-04-24T23:04:07.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.226+0000 7f808d928640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80880d32f0 con 0x7f808806b110 2024-04-24T23:04:07.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.226+0000 7f808d928640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f80880d2db0 con 0x7f80880ba390 2024-04-24T23:04:07.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.226+0000 7f80867fc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f80880ba390 0x7f80880d40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:07.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.226+0000 7f8086ffd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f808815e9e0 0x7f80880d26a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42646/0 (socket says 172.21.15.32:42646) 2024-04-24T23:04:07.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.226+0000 7f80867fc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f80880ba390 0x7f80880d40b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60344/0 (socket says 172.21.15.32:60344) 2024-04-24T23:04:07.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.226+0000 7f8086ffd640 1 -- 172.21.15.32:0/3252992198 learned_addr learned my addr 172.21.15.32:0/3252992198 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:04:07.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.226+0000 7f80867fc640 1 -- 172.21.15.32:0/3252992198 >> v1:172.21.15.121:6789/0 conn(0x7f808806b110 legacy=0x7f80880b7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:07.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.226+0000 7f80867fc640 1 -- 172.21.15.32:0/3252992198 >> v1:172.21.15.32:6789/0 conn(0x7f808815e9e0 legacy=0x7f80880d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:07.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.226+0000 7f8085ffb640 1 -- 172.21.15.32:0/3252992198 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 160021515 0 0) 0x7f80880d3010 con 0x7f808815e9e0 2024-04-24T23:04:07.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.226+0000 7f80867fc640 1 -- 172.21.15.32:0/3252992198 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80880d35d0 con 0x7f80880ba390 2024-04-24T23:04:07.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.227+0000 7f80867fc640 1 --2- 172.21.15.32:0/3252992198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80880ba390 0x7f80880d40b0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f80700097a0 tx=0x7f807002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3b3225b1c6009d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:07.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.227+0000 7f8085ffb640 1 -- 172.21.15.32:0/3252992198 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8070037070 con 0x7f80880ba390 2024-04-24T23:04:07.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.227+0000 7f8085ffb640 1 -- 172.21.15.32:0/3252992198 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8070032440 con 0x7f80880ba390 2024-04-24T23:04:07.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.227+0000 7f8085ffb640 1 -- 172.21.15.32:0/3252992198 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8070032810 con 0x7f80880ba390 2024-04-24T23:04:07.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.228+0000 7f808d928640 1 -- 172.21.15.32:0/3252992198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80880ba390 msgr2=0x7f80880d40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:07.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.228+0000 7f808d928640 1 --2- 172.21.15.32:0/3252992198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80880ba390 0x7f80880d40b0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f80700097a0 tx=0x7f807002a230 comp rx=0 tx=0).stop 2024-04-24T23:04:07.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.228+0000 7f808d928640 1 -- 172.21.15.32:0/3252992198 shutdown_connections 2024-04-24T23:04:07.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.228+0000 7f808d928640 1 --2- 172.21.15.32:0/3252992198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80880ba390 0x7f80880d40b0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:07.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.228+0000 7f808d928640 1 -- 172.21.15.32:0/3252992198 >> 172.21.15.32:0/3252992198 conn(0x7f80880d12e0 msgr2=0x7f80880d16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:07.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.228+0000 7f808d928640 1 -- 172.21.15.32:0/3252992198 shutdown_connections 2024-04-24T23:04:07.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.228+0000 7f808d928640 1 -- 172.21.15.32:0/3252992198 wait complete. 2024-04-24T23:04:07.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.229+0000 7f808d928640 1 Processor -- start 2024-04-24T23:04:07.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.229+0000 7f808d928640 1 -- start start 2024-04-24T23:04:07.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.230+0000 7f808d928640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f808806b110 0x7f80881f52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:07.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.230+0000 7f808d928640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80880ba390 0x7f808815f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:07.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.230+0000 7f808d928640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f808815e9e0 0x7f80880b82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:07.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.230+0000 7f808d928640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f808806b5c0 con 0x7f808815e9e0 2024-04-24T23:04:07.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.230+0000 7f808d928640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f80880d3b90 con 0x7f80880ba390 2024-04-24T23:04:07.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.230+0000 7f808d928640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f808806b740 con 0x7f808806b110 2024-04-24T23:04:07.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.230+0000 7f80877fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f808815e9e0 0x7f80880b82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:07.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.230+0000 7f80877fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f808815e9e0 0x7f80880b82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60346/0 (socket says 172.21.15.32:60346) 2024-04-24T23:04:07.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.230+0000 7f80877fe640 1 -- 172.21.15.32:0/3866448652 learned_addr learned my addr 172.21.15.32:0/3866448652 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:07.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.230+0000 7f80867fc640 1 --2- 172.21.15.32:0/3866448652 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80880ba390 0x7f808815f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:07.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.230+0000 7f8086ffd640 1 --2- 172.21.15.32:0/3866448652 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f808806b110 0x7f80881f52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:07.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.230+0000 7f80877fe640 1 -- 172.21.15.32:0/3866448652 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f808806b110 msgr2=0x7f80881f52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:07.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.230+0000 7f80877fe640 1 --2- 172.21.15.32:0/3866448652 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f808806b110 0x7f80881f52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:07.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.230+0000 7f80877fe640 1 -- 172.21.15.32:0/3866448652 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80880ba390 msgr2=0x7f808815f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:07.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.230+0000 7f80877fe640 1 --2- 172.21.15.32:0/3866448652 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80880ba390 0x7f808815f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:07.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.230+0000 7f80877fe640 1 -- 172.21.15.32:0/3866448652 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f808815fd10 con 0x7f808815e9e0 2024-04-24T23:04:07.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.231+0000 7f80877fe640 1 --2- 172.21.15.32:0/3866448652 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f808815e9e0 0x7f80880b82a0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f808000c7a0 tx=0x7f808000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:07.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.231+0000 7f807ffff640 1 -- 172.21.15.32:0/3866448652 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8080003b70 con 0x7f808815e9e0 2024-04-24T23:04:07.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.231+0000 7f807ffff640 1 -- 172.21.15.32:0/3866448652 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8080003d30 con 0x7f808815e9e0 2024-04-24T23:04:07.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.231+0000 7f808d928640 1 -- 172.21.15.32:0/3866448652 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8088160ab0 con 0x7f808815e9e0 2024-04-24T23:04:07.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.231+0000 7f807ffff640 1 -- 172.21.15.32:0/3866448652 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808000aa90 con 0x7f808815e9e0 2024-04-24T23:04:07.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.231+0000 7f808d928640 1 -- 172.21.15.32:0/3866448652 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80881f49c0 con 0x7f808815e9e0 2024-04-24T23:04:07.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.232+0000 7f808d928640 1 -- 172.21.15.32:0/3866448652 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8088160c80 con 0x7f808815e9e0 2024-04-24T23:04:07.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.232+0000 7f807ffff640 1 -- 172.21.15.32:0/3866448652 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f808000ac30 con 0x7f808815e9e0 2024-04-24T23:04:07.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.233+0000 7f807ffff640 1 --2- 172.21.15.32:0/3866448652 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f806806be30 0x7f806806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:07.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.233+0000 7f807ffff640 1 -- 172.21.15.32:0/3866448652 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7f808008ac70 con 0x7f808815e9e0 2024-04-24T23:04:07.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.233+0000 7f8086ffd640 1 --2- 172.21.15.32:0/3866448652 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f806806be30 0x7f806806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:07.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.234+0000 7f8086ffd640 1 --2- 172.21.15.32:0/3866448652 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f806806be30 0x7f806806e2f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8078000f60 tx=0x7f8078007e40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:07.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.237+0000 7f807ffff640 1 -- 172.21.15.32:0/3866448652 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f808815fd10 con 0x7f808815e9e0 2024-04-24T23:04:07.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.491+0000 7f808d928640 1 -- 172.21.15.32:0/3866448652 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8088072390 con 0x7f808815e9e0 2024-04-24T23:04:07.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.491+0000 7f807ffff640 1 -- 172.21.15.32:0/3866448652 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v44) ==== 53+0+73 (secure 0 0 0) 0x7f8080058fd0 con 0x7f808815e9e0 2024-04-24T23:04:07.493 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T23:04:07.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.493+0000 7f808d928640 1 -- 172.21.15.32:0/3866448652 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f806806be30 msgr2=0x7f806806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:07.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.493+0000 7f808d928640 1 --2- 172.21.15.32:0/3866448652 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f806806be30 0x7f806806e2f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8078000f60 tx=0x7f8078007e40 comp rx=0 tx=0).stop 2024-04-24T23:04:07.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.493+0000 7f808d928640 1 -- 172.21.15.32:0/3866448652 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f808815e9e0 msgr2=0x7f80880b82a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:07.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.493+0000 7f808d928640 1 --2- 172.21.15.32:0/3866448652 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f808815e9e0 0x7f80880b82a0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f808000c7a0 tx=0x7f808000cc60 comp rx=0 tx=0).stop 2024-04-24T23:04:07.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.493+0000 7f808d928640 1 -- 172.21.15.32:0/3866448652 shutdown_connections 2024-04-24T23:04:07.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.493+0000 7f808d928640 1 --2- 172.21.15.32:0/3866448652 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f806806be30 0x7f806806e2f0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:07.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.493+0000 7f808d928640 1 --2- 172.21.15.32:0/3866448652 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f808815e9e0 0x7f80880b82a0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:07.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.493+0000 7f808d928640 1 --2- 172.21.15.32:0/3866448652 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80880ba390 0x7f808815f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:07.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.493+0000 7f808d928640 1 --2- 172.21.15.32:0/3866448652 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f808806b110 0x7f80881f52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:07.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.493+0000 7f808d928640 1 -- 172.21.15.32:0/3866448652 >> 172.21.15.32:0/3866448652 conn(0x7f80880d12e0 msgr2=0x7f8088159c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:07.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.494+0000 7f808d928640 1 -- 172.21.15.32:0/3866448652 shutdown_connections 2024-04-24T23:04:07.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07.494+0000 7f808d928640 1 -- 172.21.15.32:0/3866448652 wait complete. 2024-04-24T23:04:07.506 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-24T23:04:07.506 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T23:04:07.506 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2024-04-24T23:04:07.506 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-24T23:04:07.506 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T23:04:07.506 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-24T23:04:07.506 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-24T23:04:07.506 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T23:04:07.506 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:04:07.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:04:07.507 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2024-04-24T23:04:07.523 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T23:04:07.523 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T23:04:07.523 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T23:04:07.523 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:04:07.523 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2024-04-24T23:04:07.523 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:04:07.523 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:04:07.524 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2024-04-24T23:04:07.524 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2024-04-24T23:04:07.524 INFO:teuthology.orchestra.run.smithi032.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:04:07.524 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2024-04-24T23:04:07.524 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2024-04-24T23:04:07.524 INFO:teuthology.orchestra.run.smithi032.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T23:04:07.524 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:04:07.524 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2024-04-24T23:04:07.524 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-24T23:04:07.524 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 42461sec preferred_lft 42461sec 2024-04-24T23:04:07.524 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link noprefixroute 2024-04-24T23:04:07.525 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:04:07.525 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:04:07.525 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:04:07.525 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2024-04-24T23:04:07.525 INFO:teuthology.orchestra.run.smithi032.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T23:04:07.525 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 72:d4:96:00:43:8c brd ff:ff:ff:ff:ff:ff 2024-04-24T23:04:07.525 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-24T23:04:07.525 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:04:07.525 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::f4c6:20ff:fe22:2647/64 scope link 2024-04-24T23:04:07.525 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:04:07.526 INFO:teuthology.orchestra.run.smithi032.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:04:07.526 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 72:d4:96:00:43:8c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T23:04:07.526 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::70d4:96ff:fe00:438c/64 scope link 2024-04-24T23:04:07.526 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:04:07.526 INFO:teuthology.orchestra.run.smithi032.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:04:07.526 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether e6:6b:2a:90:1a:f8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T23:04:07.526 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::e46b:2aff:fe90:1af8/64 scope link 2024-04-24T23:04:07.526 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:04:07.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:04:07.527 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T23:04:07.586 INFO:teuthology.orchestra.run.smithi032.stdout:1 2024-04-24T23:04:07.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:04:07.588 DEBUG:teuthology.orchestra.run.smithi032:> route 2024-04-24T23:04:07.644 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2024-04-24T23:04:07.644 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T23:04:07.644 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-24T23:04:07.644 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-24T23:04:07.644 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T23:04:07.645 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:04:07.645 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-24T23:04:07.645 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-24T23:04:07.645 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-24T23:04:07.645 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-24T23:04:07.645 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-24T23:04:07.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:04:07.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:04:07.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:04:07.771 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-04-24T23:04:07.786 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-24T23:04:07.786 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-24T23:04:07.786 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:04:07.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:04:07.786 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2024-04-24T23:04:07.851 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:04:07.851 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-24T23:04:07.906 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:04:07.906 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:04:07.907 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-24T23:04:07.961 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-24T23:04:07.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:04:07.961 DEBUG:teuthology.orchestra.run.smithi032:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:04:08.044 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:04:08.044 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-24T23:04:08.054 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:04:08.054 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-24T23:04:08.054 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.121:3300,172.21.15.121:3301" 2024-04-24T23:04:08.054 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: trying mount with new device syntax: 0@eb1b7a01-6f65-4534-9a12-83011862ba51.cephfs=/ 2024-04-24T23:04:08.054 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.32:3300/172.21.15.121:3300/172.21.15.121:3301" will pass to kernel 2024-04-24T23:04:08.055 INFO:teuthology.orchestra.run.smithi032.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-24T23:04:08.055 INFO:teuthology.orchestra.run.smithi032.stdout: You just mounted a file system that supports labels which does not 2024-04-24T23:04:08.055 INFO:teuthology.orchestra.run.smithi032.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-24T23:04:08.055 INFO:teuthology.orchestra.run.smithi032.stdout: applications will generate AVC messages and not be allowed access to 2024-04-24T23:04:08.055 INFO:teuthology.orchestra.run.smithi032.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-24T23:04:08.061 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-24T23:04:08.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:04:08.061 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-24T23:04:08.126 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:04:08.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:04:08.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:04:08.227 INFO:teuthology.orchestra.run.smithi032.stdout:client5807 2024-04-24T23:04:08.227 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:04:08.227 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:04:08.243 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T23:04:08.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.382+0000 7f3b071d5640 1 Processor -- start 2024-04-24T23:04:08.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.382+0000 7f3b071d5640 1 -- start start 2024-04-24T23:04:08.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.383+0000 7f3b071d5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b0006c5f0 0x7f3b0006c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:08.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.383+0000 7f3b071d5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3b000b44d0 0x7f3b000b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:08.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.383+0000 7f3b071d5640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3b000af890 0x7f3b000cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:08.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.383+0000 7f3b071d5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f3b000b6430 con 0x7f3b000b44d0 2024-04-24T23:04:08.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.383+0000 7f3b071d5640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f3b00156f80 con 0x7f3b000af890 2024-04-24T23:04:08.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.383+0000 7f3b071d5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f3b000b6150 con 0x7f3b0006c5f0 2024-04-24T23:04:08.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.383+0000 7f3af7fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3b000b44d0 0x7f3b000b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:08.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.383+0000 7f3af7fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3b000b44d0 0x7f3b000b5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60352/0 (socket says 172.21.15.32:60352) 2024-04-24T23:04:08.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.383+0000 7f3af7fff640 1 -- 172.21.15.32:0/686032364 learned_addr learned my addr 172.21.15.32:0/686032364 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:08.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.383+0000 7f3b05772640 1 --2- 172.21.15.32:0/686032364 >> v2:172.21.15.121:3300/0 conn(0x7f3b000af890 0x7f3b000cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:08.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.383+0000 7f3b04f71640 1 --2- 172.21.15.32:0/686032364 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b0006c5f0 0x7f3b0006c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:08.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.383+0000 7f3af7fff640 1 -- 172.21.15.32:0/686032364 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b0006c5f0 msgr2=0x7f3b0006c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:08.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.383+0000 7f3af7fff640 1 --2- 172.21.15.32:0/686032364 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b0006c5f0 0x7f3b0006c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:08.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.383+0000 7f3af7fff640 1 -- 172.21.15.32:0/686032364 >> v2:172.21.15.121:3300/0 conn(0x7f3b000af890 msgr2=0x7f3b000cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:08.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.383+0000 7f3af7fff640 1 --2- 172.21.15.32:0/686032364 >> v2:172.21.15.121:3300/0 conn(0x7f3b000af890 0x7f3b000cef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:08.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.383+0000 7f3af7fff640 1 -- 172.21.15.32:0/686032364 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b000b39f0 con 0x7f3b000b44d0 2024-04-24T23:04:08.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.383+0000 7f3b05772640 1 --2- 172.21.15.32:0/686032364 >> v2:172.21.15.121:3300/0 conn(0x7f3b000af890 0x7f3b000cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:04:08.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.384+0000 7f3af7fff640 1 --2- 172.21.15.32:0/686032364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b000b44d0 0x7f3b000b5890 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f3b000aff30 tx=0x7f3af002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4c84f7a42967622 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:08.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.384+0000 7f3af77fe640 1 -- 172.21.15.32:0/686032364 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af0037070 con 0x7f3b000b44d0 2024-04-24T23:04:08.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.384+0000 7f3af77fe640 1 -- 172.21.15.32:0/686032364 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3af0032440 con 0x7f3b000b44d0 2024-04-24T23:04:08.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.384+0000 7f3af77fe640 1 -- 172.21.15.32:0/686032364 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af0032810 con 0x7f3b000b44d0 2024-04-24T23:04:08.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.385+0000 7f3b071d5640 1 -- 172.21.15.32:0/686032364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b000b44d0 msgr2=0x7f3b000b5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:08.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.385+0000 7f3b071d5640 1 --2- 172.21.15.32:0/686032364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b000b44d0 0x7f3b000b5890 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f3b000aff30 tx=0x7f3af002a140 comp rx=0 tx=0).stop 2024-04-24T23:04:08.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.385+0000 7f3b071d5640 1 -- 172.21.15.32:0/686032364 shutdown_connections 2024-04-24T23:04:08.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.385+0000 7f3b071d5640 1 --2- 172.21.15.32:0/686032364 >> v2:172.21.15.121:3300/0 conn(0x7f3b000af890 0x7f3b000cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:08.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.385+0000 7f3b071d5640 1 --2- 172.21.15.32:0/686032364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b000b44d0 0x7f3b000b5890 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:08.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.385+0000 7f3b071d5640 1 --2- 172.21.15.32:0/686032364 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b0006c5f0 0x7f3b0006c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:08.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.385+0000 7f3b071d5640 1 -- 172.21.15.32:0/686032364 >> 172.21.15.32:0/686032364 conn(0x7f3b0006b110 msgr2=0x7f3b0006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:08.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.385+0000 7f3b071d5640 1 -- 172.21.15.32:0/686032364 shutdown_connections 2024-04-24T23:04:08.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.385+0000 7f3b071d5640 1 -- 172.21.15.32:0/686032364 wait complete. 2024-04-24T23:04:08.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.386+0000 7f3b071d5640 1 Processor -- start 2024-04-24T23:04:08.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.386+0000 7f3b071d5640 1 -- start start 2024-04-24T23:04:08.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.387+0000 7f3b071d5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b0006c5f0 0x7f3b0014ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:08.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.387+0000 7f3b071d5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b000af890 0x7f3b000ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:08.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.387+0000 7f3b071d5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b000b44d0 0x7f3b000aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:08.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.387+0000 7f3b071d5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0006cf10 con 0x7f3b000b44d0 2024-04-24T23:04:08.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.387+0000 7f3b071d5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3b000b3fb0 con 0x7f3b000af890 2024-04-24T23:04:08.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.387+0000 7f3b071d5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f3b000b5dd0 con 0x7f3b0006c5f0 2024-04-24T23:04:08.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.387+0000 7f3b05772640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b000b44d0 0x7f3b000aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:08.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.387+0000 7f3b05772640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b000b44d0 0x7f3b000aedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60354/0 (socket says 172.21.15.32:60354) 2024-04-24T23:04:08.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.387+0000 7f3b05772640 1 -- 172.21.15.32:0/1842467538 learned_addr learned my addr 172.21.15.32:0/1842467538 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:08.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.387+0000 7f3b04f71640 1 --2- 172.21.15.32:0/1842467538 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b0006c5f0 0x7f3b0014ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:08.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.387+0000 7f3af7fff640 1 --2- 172.21.15.32:0/1842467538 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b000af890 0x7f3b000ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:08.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.387+0000 7f3b05772640 1 -- 172.21.15.32:0/1842467538 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b0006c5f0 msgr2=0x7f3b0014ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:08.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.387+0000 7f3b05772640 1 --2- 172.21.15.32:0/1842467538 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b0006c5f0 0x7f3b0014ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:08.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.387+0000 7f3b05772640 1 -- 172.21.15.32:0/1842467538 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b000af890 msgr2=0x7f3b000ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:08.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.387+0000 7f3b05772640 1 --2- 172.21.15.32:0/1842467538 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b000af890 0x7f3b000ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:08.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.387+0000 7f3b05772640 1 -- 172.21.15.32:0/1842467538 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b00158780 con 0x7f3b000b44d0 2024-04-24T23:04:08.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.387+0000 7f3af7fff640 1 --2- 172.21.15.32:0/1842467538 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b000af890 0x7f3b000ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:04:08.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.387+0000 7f3b04f71640 1 --2- 172.21.15.32:0/1842467538 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b0006c5f0 0x7f3b0014ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:04:08.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.387+0000 7f3b05772640 1 --2- 172.21.15.32:0/1842467538 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b000b44d0 0x7f3b000aedf0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f3af800cd50 tx=0x7f3af800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:08.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.387+0000 7f3af57fa640 1 -- 172.21.15.32:0/1842467538 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af800f680 con 0x7f3b000b44d0 2024-04-24T23:04:08.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.388+0000 7f3af57fa640 1 -- 172.21.15.32:0/1842467538 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3af800f840 con 0x7f3b000b44d0 2024-04-24T23:04:08.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.388+0000 7f3b071d5640 1 -- 172.21.15.32:0/1842467538 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b000ce050 con 0x7f3b000b44d0 2024-04-24T23:04:08.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.388+0000 7f3af57fa640 1 -- 172.21.15.32:0/1842467538 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af800fc30 con 0x7f3b000b44d0 2024-04-24T23:04:08.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.388+0000 7f3b071d5640 1 -- 172.21.15.32:0/1842467538 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b0014d200 con 0x7f3b000b44d0 2024-04-24T23:04:08.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.389+0000 7f3b071d5640 1 -- 172.21.15.32:0/1842467538 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b00068430 con 0x7f3b000b44d0 2024-04-24T23:04:08.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.389+0000 7f3af57fa640 1 -- 172.21.15.32:0/1842467538 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3af80213f0 con 0x7f3b000b44d0 2024-04-24T23:04:08.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.389+0000 7f3af57fa640 1 --2- 172.21.15.32:0/1842467538 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3aec06be30 0x7f3aec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:08.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.389+0000 7f3af57fa640 1 -- 172.21.15.32:0/1842467538 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7f3af808bb10 con 0x7f3b000b44d0 2024-04-24T23:04:08.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.390+0000 7f3b04f71640 1 --2- 172.21.15.32:0/1842467538 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3aec06be30 0x7f3aec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:08.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.390+0000 7f3b04f71640 1 --2- 172.21.15.32:0/1842467538 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3aec06be30 0x7f3aec06e2f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3afc006fd0 tx=0x7f3afc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:08.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.394+0000 7f3af57fa640 1 -- 172.21.15.32:0/1842467538 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3af8014030 con 0x7f3b000b44d0 2024-04-24T23:04:08.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.620+0000 7f3b071d5640 1 -- 172.21.15.32:0/1842467538 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3b000b5670 con 0x7f3b000b44d0 2024-04-24T23:04:08.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.620+0000 7f3af57fa640 1 -- 172.21.15.32:0/1842467538 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v44) ==== 53+0+73 (secure 0 0 0) 0x7f3af8059f00 con 0x7f3b000b44d0 2024-04-24T23:04:08.622 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T23:04:08.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.622+0000 7f3b071d5640 1 -- 172.21.15.32:0/1842467538 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3aec06be30 msgr2=0x7f3aec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:08.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.622+0000 7f3b071d5640 1 --2- 172.21.15.32:0/1842467538 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3aec06be30 0x7f3aec06e2f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3afc006fd0 tx=0x7f3afc008040 comp rx=0 tx=0).stop 2024-04-24T23:04:08.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.622+0000 7f3b071d5640 1 -- 172.21.15.32:0/1842467538 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b000b44d0 msgr2=0x7f3b000aedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:08.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.622+0000 7f3b071d5640 1 --2- 172.21.15.32:0/1842467538 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b000b44d0 0x7f3b000aedf0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f3af800cd50 tx=0x7f3af800a650 comp rx=0 tx=0).stop 2024-04-24T23:04:08.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.622+0000 7f3b071d5640 1 -- 172.21.15.32:0/1842467538 shutdown_connections 2024-04-24T23:04:08.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.622+0000 7f3b071d5640 1 --2- 172.21.15.32:0/1842467538 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3aec06be30 0x7f3aec06e2f0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:08.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.622+0000 7f3b071d5640 1 --2- 172.21.15.32:0/1842467538 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b000b44d0 0x7f3b000aedf0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:08.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.622+0000 7f3b071d5640 1 --2- 172.21.15.32:0/1842467538 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b000af890 0x7f3b000ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:08.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.622+0000 7f3b071d5640 1 --2- 172.21.15.32:0/1842467538 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b0006c5f0 0x7f3b0014ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:08.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.622+0000 7f3b071d5640 1 -- 172.21.15.32:0/1842467538 >> 172.21.15.32:0/1842467538 conn(0x7f3b0006b110 msgr2=0x7f3b0015c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:08.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.622+0000 7f3b071d5640 1 -- 172.21.15.32:0/1842467538 shutdown_connections 2024-04-24T23:04:08.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:08.622+0000 7f3b071d5640 1 -- 172.21.15.32:0/1842467538 wait complete. 2024-04-24T23:04:08.634 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-24T23:04:08.634 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T23:04:08.634 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi121.front.sepia.ceph.com 2024-04-24T23:04:08.635 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-24T23:04:08.635 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-24T23:04:08.635 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-24T23:04:08.635 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-24T23:04:08.635 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T23:04:08.635 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-24T23:04:08.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:04:08.635 DEBUG:teuthology.orchestra.run.smithi121:> ip addr 2024-04-24T23:04:08.654 INFO:teuthology.orchestra.run.smithi121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T23:04:08.654 INFO:teuthology.orchestra.run.smithi121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T23:04:08.654 INFO:teuthology.orchestra.run.smithi121.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T23:04:08.654 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:04:08.654 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 ::1/128 scope host 2024-04-24T23:04:08.654 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:04:08.654 INFO:teuthology.orchestra.run.smithi121.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:04:08.654 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:82 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:04:08.654 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f0 2024-04-24T23:04:08.654 INFO:teuthology.orchestra.run.smithi121.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:04:08.655 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:83 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:04:08.655 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f1 2024-04-24T23:04:08.655 INFO:teuthology.orchestra.run.smithi121.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:04:08.655 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a2 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:04:08.655 INFO:teuthology.orchestra.run.smithi121.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T23:04:08.655 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a3 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:04:08.655 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.21.15.121/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-24T23:04:08.655 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft 42482sec preferred_lft 42482sec 2024-04-24T23:04:08.655 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::93dd:8489:4369:e127/64 scope link noprefixroute 2024-04-24T23:04:08.655 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:04:08.655 INFO:teuthology.orchestra.run.smithi121.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T23:04:08.655 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether c6:c0:c7:45:ae:ba brd ff:ff:ff:ff:ff:ff 2024-04-24T23:04:08.655 INFO:teuthology.orchestra.run.smithi121.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-24T23:04:08.655 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:04:08.656 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::80fe:d2ff:fe13:d1/64 scope link 2024-04-24T23:04:08.656 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:04:08.656 INFO:teuthology.orchestra.run.smithi121.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:04:08.656 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether e2:ca:b9:e9:9e:bb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T23:04:08.656 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::e0ca:b9ff:fee9:9ebb/64 scope link 2024-04-24T23:04:08.656 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:04:08.656 INFO:teuthology.orchestra.run.smithi121.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:04:08.656 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether c6:c0:c7:45:ae:ba brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-24T23:04:08.656 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::c4c0:c7ff:fe45:aeba/64 scope link 2024-04-24T23:04:08.656 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:04:08.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:04:08.657 DEBUG:teuthology.orchestra.run.smithi121:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T23:04:08.720 INFO:teuthology.orchestra.run.smithi121.stdout:1 2024-04-24T23:04:08.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:04:08.722 DEBUG:teuthology.orchestra.run.smithi121:> route 2024-04-24T23:04:08.779 INFO:teuthology.orchestra.run.smithi121.stdout:Kernel IP routing table 2024-04-24T23:04:08.780 INFO:teuthology.orchestra.run.smithi121.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T23:04:08.780 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-24T23:04:08.780 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-24T23:04:08.780 INFO:teuthology.orchestra.run.smithi121.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T23:04:08.780 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:04:08.780 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2024-04-24T23:04:08.780 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-24T23:04:08.780 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-24T23:04:08.780 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-24T23:04:08.780 DEBUG:teuthology.orchestra.run.smithi121:> ') 2024-04-24T23:04:08.872 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T23:04:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:04:08.903 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T23:04:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:04:08.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:04:08.909 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2024-04-24T23:04:08.923 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-24T23:04:08.924 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-24T23:04:08.924 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-24T23:04:08.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:04:08.924 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set brx.0 up 2024-04-24T23:04:08.988 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:04:08.988 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-24T23:04:09.043 INFO:teuthology.orchestra.run.smithi121.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-24T23:04:09.043 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:04:09.043 DEBUG:teuthology.orchestra.run.smithi121:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-24T23:04:09.097 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-24T23:04:09.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:04:09.098 DEBUG:teuthology.orchestra.run.smithi121:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:04:09.178 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:04:09.178 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-24T23:04:09.184 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:04:09.185 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-24T23:04:09.185 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.121:3300,172.21.15.121:3301" 2024-04-24T23:04:09.185 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: trying mount with new device syntax: 1@eb1b7a01-6f65-4534-9a12-83011862ba51.cephfs=/ 2024-04-24T23:04:09.185 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.32:3300/172.21.15.121:3300/172.21.15.121:3301" will pass to kernel 2024-04-24T23:04:09.185 INFO:teuthology.orchestra.run.smithi121.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-24T23:04:09.185 INFO:teuthology.orchestra.run.smithi121.stdout: You just mounted a file system that supports labels which does not 2024-04-24T23:04:09.185 INFO:teuthology.orchestra.run.smithi121.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-24T23:04:09.186 INFO:teuthology.orchestra.run.smithi121.stdout: applications will generate AVC messages and not be allowed access to 2024-04-24T23:04:09.186 INFO:teuthology.orchestra.run.smithi121.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-24T23:04:09.187 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-24T23:04:09.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:04:09.187 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-24T23:04:09.251 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:04:09.339 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T23:04:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:04:09.341 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T23:04:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:04:09.346 INFO:teuthology.orchestra.run.smithi121.stdout:client6283 2024-04-24T23:04:09.347 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:04:09.347 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:04:09.362 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:04:09.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.477+0000 7f091aa5d640 1 Processor -- start 2024-04-24T23:04:09.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.477+0000 7f091aa5d640 1 -- start start 2024-04-24T23:04:09.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.478+0000 7f091aa5d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0914162320 0x7f09140ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:09.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.478+0000 7f091aa5d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09140b5af0 0x7f0914162700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:09.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.478+0000 7f091aa5d640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09140cd270 con 0x7f0914162c40 2024-04-24T23:04:09.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.478+0000 7f091aa5d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f09140b5660 con 0x7f0914162320 2024-04-24T23:04:09.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.478+0000 7f091aa5d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f0914071610 con 0x7f09140b5af0 2024-04-24T23:04:09.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.478+0000 7f0913fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0914162320 0x7f09140ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:09.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.478+0000 7f0913fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0914162320 0x7f09140ccc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60356/0 (socket says 172.21.15.32:60356) 2024-04-24T23:04:09.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.478+0000 7f0913fff640 1 -- 172.21.15.32:0/4078456615 learned_addr learned my addr 172.21.15.32:0/4078456615 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:09.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.478+0000 7f09137fe640 1 --2- 172.21.15.32:0/4078456615 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09140b5af0 0x7f0914162700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:09.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.478+0000 7f0913fff640 1 -- 172.21.15.32:0/4078456615 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09140b5af0 msgr2=0x7f0914162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:09.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.478+0000 7f0913fff640 1 --2- 172.21.15.32:0/4078456615 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09140b5af0 0x7f0914162700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:09.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.478+0000 7f0913fff640 1 -- 172.21.15.32:0/4078456615 >> v1:172.21.15.121:6789/0 conn(0x7f0914162c40 legacy=0x7f09140ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:09.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.478+0000 7f0913fff640 1 -- 172.21.15.32:0/4078456615 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09140718f0 con 0x7f0914162320 2024-04-24T23:04:09.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.478+0000 7f09137fe640 1 --2- 172.21.15.32:0/4078456615 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09140b5af0 0x7f0914162700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:04:09.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.479+0000 7f0913fff640 1 --2- 172.21.15.32:0/4078456615 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0914162320 0x7f09140ccc90 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f08fc009870 tx=0x7f08fc02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57c89d388872bc79 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:09.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.479+0000 7f0912ffd640 1 -- 172.21.15.32:0/4078456615 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08fc037070 con 0x7f0914162320 2024-04-24T23:04:09.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.479+0000 7f0912ffd640 1 -- 172.21.15.32:0/4078456615 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08fc032440 con 0x7f0914162320 2024-04-24T23:04:09.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.479+0000 7f0912ffd640 1 -- 172.21.15.32:0/4078456615 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08fc032810 con 0x7f0914162320 2024-04-24T23:04:09.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.480+0000 7f091aa5d640 1 -- 172.21.15.32:0/4078456615 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0914162320 msgr2=0x7f09140ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:09.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.480+0000 7f091aa5d640 1 --2- 172.21.15.32:0/4078456615 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0914162320 0x7f09140ccc90 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f08fc009870 tx=0x7f08fc02a1c0 comp rx=0 tx=0).stop 2024-04-24T23:04:09.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.480+0000 7f091aa5d640 1 -- 172.21.15.32:0/4078456615 shutdown_connections 2024-04-24T23:04:09.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.480+0000 7f091aa5d640 1 --2- 172.21.15.32:0/4078456615 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09140b5af0 0x7f0914162700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:09.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.480+0000 7f091aa5d640 1 --2- 172.21.15.32:0/4078456615 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0914162320 0x7f09140ccc90 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:09.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.480+0000 7f091aa5d640 1 -- 172.21.15.32:0/4078456615 >> 172.21.15.32:0/4078456615 conn(0x7f09140d0ae0 msgr2=0x7f09140d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:09.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.480+0000 7f091aa5d640 1 -- 172.21.15.32:0/4078456615 shutdown_connections 2024-04-24T23:04:09.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.480+0000 7f091aa5d640 1 -- 172.21.15.32:0/4078456615 wait complete. 2024-04-24T23:04:09.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.481+0000 7f091aa5d640 1 Processor -- start 2024-04-24T23:04:09.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.481+0000 7f091aa5d640 1 -- start start 2024-04-24T23:04:09.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.482+0000 7f091aa5d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09140b5af0 0x7f09140cf250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:09.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.482+0000 7f091aa5d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0914162320 0x7f091414e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:09.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.482+0000 7f091aa5d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0914162c40 0x7f09141f8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:09.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.482+0000 7f091aa5d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0914067950 con 0x7f0914162c40 2024-04-24T23:04:09.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.482+0000 7f091aa5d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f091415c630 con 0x7f09140b5af0 2024-04-24T23:04:09.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.482+0000 7f091aa5d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f0914071bd0 con 0x7f0914162320 2024-04-24T23:04:09.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.482+0000 7f0918ffa640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0914162c40 0x7f09141f8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:09.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.482+0000 7f0918ffa640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0914162c40 0x7f09141f8c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60366/0 (socket says 172.21.15.32:60366) 2024-04-24T23:04:09.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.482+0000 7f0918ffa640 1 -- 172.21.15.32:0/3319697117 learned_addr learned my addr 172.21.15.32:0/3319697117 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:09.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.482+0000 7f0913fff640 1 --2- 172.21.15.32:0/3319697117 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09140b5af0 0x7f09140cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:09.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.482+0000 7f09137fe640 1 --2- 172.21.15.32:0/3319697117 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0914162320 0x7f091414e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:09.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.483+0000 7f0918ffa640 1 -- 172.21.15.32:0/3319697117 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0914162320 msgr2=0x7f091414e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:09.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.483+0000 7f0918ffa640 1 --2- 172.21.15.32:0/3319697117 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0914162320 0x7f091414e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:09.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.483+0000 7f0918ffa640 1 -- 172.21.15.32:0/3319697117 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09140b5af0 msgr2=0x7f09140cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:09.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.483+0000 7f0918ffa640 1 --2- 172.21.15.32:0/3319697117 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09140b5af0 0x7f09140cf250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:09.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.483+0000 7f0918ffa640 1 -- 172.21.15.32:0/3319697117 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09141f4c60 con 0x7f0914162c40 2024-04-24T23:04:09.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.483+0000 7f09137fe640 1 --2- 172.21.15.32:0/3319697117 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0914162320 0x7f091414e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:04:09.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.483+0000 7f0913fff640 1 --2- 172.21.15.32:0/3319697117 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09140b5af0 0x7f09140cf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:04:09.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.483+0000 7f0918ffa640 1 --2- 172.21.15.32:0/3319697117 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0914162c40 0x7f09141f8c20 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f090c00c6b0 tx=0x7f090c00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:09.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.483+0000 7f0910ff9640 1 -- 172.21.15.32:0/3319697117 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090c003b70 con 0x7f0914162c40 2024-04-24T23:04:09.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.483+0000 7f0910ff9640 1 -- 172.21.15.32:0/3319697117 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f090c003d30 con 0x7f0914162c40 2024-04-24T23:04:09.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.483+0000 7f091aa5d640 1 -- 172.21.15.32:0/3319697117 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09141e8390 con 0x7f0914162c40 2024-04-24T23:04:09.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.483+0000 7f0910ff9640 1 -- 172.21.15.32:0/3319697117 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090c005310 con 0x7f0914162c40 2024-04-24T23:04:09.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.483+0000 7f091aa5d640 1 -- 172.21.15.32:0/3319697117 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09141e8950 con 0x7f0914162c40 2024-04-24T23:04:09.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.484+0000 7f091aa5d640 1 -- 172.21.15.32:0/3319697117 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0914068430 con 0x7f0914162c40 2024-04-24T23:04:09.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.485+0000 7f0910ff9640 1 -- 172.21.15.32:0/3319697117 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f090c0054b0 con 0x7f0914162c40 2024-04-24T23:04:09.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.485+0000 7f0910ff9640 1 --2- 172.21.15.32:0/3319697117 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f08f406be30 0x7f08f406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:09.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.485+0000 7f0910ff9640 1 -- 172.21.15.32:0/3319697117 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7f090c08ac10 con 0x7f0914162c40 2024-04-24T23:04:09.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.485+0000 7f0913fff640 1 --2- 172.21.15.32:0/3319697117 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f08f406be30 0x7f08f406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:09.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.486+0000 7f0913fff640 1 --2- 172.21.15.32:0/3319697117 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f08f406be30 0x7f08f406e2f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f08fc034000 tx=0x7f08fc02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:09.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.489+0000 7f0910ff9640 1 -- 172.21.15.32:0/3319697117 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f090c059650 con 0x7f0914162c40 2024-04-24T23:04:09.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.707+0000 7f091aa5d640 1 -- 172.21.15.32:0/3319697117 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f09140cf9c0 con 0x7f0914162c40 2024-04-24T23:04:09.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.708+0000 7f0910ff9640 1 -- 172.21.15.32:0/3319697117 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5186 (secure 0 0 0) 0x7f090c058f50 con 0x7f0914162c40 2024-04-24T23:04:09.710 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:04:09.710 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":44,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6175,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/434273738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":434273738},{"type":"v1","addr":"172.21.15.121:6836","nonce":434273738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6178,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1104557785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1104557785},{"type":"v1","addr":"172.21.15.121:6837","nonce":1104557785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6888,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/3137388886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3137388886},{"type":"v1","addr":"172.21.15.32:6834","nonce":3137388886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:04:00.783579+0000","modified":"2024-04-24T23:04:03.790658+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6891},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6891":{"gid":6891,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/3327203898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":3327203898},{"type":"v1","addr":"172.21.15.32:6835","nonce":3327203898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6891,"qdb_cluster":[6891]},"id":6}]} 2024-04-24T23:04:09.710 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 44 2024-04-24T23:04:09.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.710+0000 7f091aa5d640 1 -- 172.21.15.32:0/3319697117 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f08f406be30 msgr2=0x7f08f406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:09.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.710+0000 7f091aa5d640 1 --2- 172.21.15.32:0/3319697117 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f08f406be30 0x7f08f406e2f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f08fc034000 tx=0x7f08fc02e000 comp rx=0 tx=0).stop 2024-04-24T23:04:09.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.710+0000 7f091aa5d640 1 -- 172.21.15.32:0/3319697117 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0914162c40 msgr2=0x7f09141f8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:09.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.710+0000 7f091aa5d640 1 --2- 172.21.15.32:0/3319697117 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0914162c40 0x7f09141f8c20 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f090c00c6b0 tx=0x7f090c00cb70 comp rx=0 tx=0).stop 2024-04-24T23:04:09.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.710+0000 7f091aa5d640 1 -- 172.21.15.32:0/3319697117 shutdown_connections 2024-04-24T23:04:09.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.710+0000 7f091aa5d640 1 --2- 172.21.15.32:0/3319697117 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f08f406be30 0x7f08f406e2f0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:09.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.710+0000 7f091aa5d640 1 --2- 172.21.15.32:0/3319697117 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0914162c40 0x7f09141f8c20 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:09.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.710+0000 7f091aa5d640 1 --2- 172.21.15.32:0/3319697117 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0914162320 0x7f091414e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:09.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.710+0000 7f091aa5d640 1 --2- 172.21.15.32:0/3319697117 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09140b5af0 0x7f09140cf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:09.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.710+0000 7f091aa5d640 1 -- 172.21.15.32:0/3319697117 >> 172.21.15.32:0/3319697117 conn(0x7f09140d0ae0 msgr2=0x7f091406ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:09.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.710+0000 7f091aa5d640 1 -- 172.21.15.32:0/3319697117 shutdown_connections 2024-04-24T23:04:09.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:09.710+0000 7f091aa5d640 1 -- 172.21.15.32:0/3319697117 wait complete. 2024-04-24T23:04:09.723 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-04-24T23:04:09.843 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","whoami":0,"id":6891,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":8.3949764719999997,"mdsmap_epoch":44,"osdmap_epoch":104,"osdmap_epoch_barrier":104,"uptime":13.293950776999999} 2024-04-24T23:04:09.843 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", "fs_name": "cephfs", "id": 6891, "mdsmap_epoch": 44, "osdmap_epoch": 104, "osdmap_epoch_barrier": 104, "rank_uptime": 8.394976472, "state": "up:active", "uptime": 13.293950777, "want_state": "up:active", "whoami": 0 } 2024-04-24T23:04:09.843 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-04-24T23:04:09.956 INFO:teuthology.orchestra.run.smithi032.stdout:{"osd_mon_report_interval":"5"} 2024-04-24T23:04:09.956 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-04-24T23:04:09.956 INFO:tasks.cephfs.test_full:Writing 531MB should fill this cluster 2024-04-24T23:04:09.957 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:04:09.957 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:04:09.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:04:09.972 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=265 seek=0) 2024-04-24T23:04:10.929 INFO:teuthology.orchestra.run.smithi032.stderr:265+0 records in 2024-04-24T23:04:10.929 INFO:teuthology.orchestra.run.smithi032.stderr:265+0 records out 2024-04-24T23:04:10.929 INFO:teuthology.orchestra.run.smithi032.stderr:277872640 bytes (278 MB, 265 MiB) copied, 0.898948 s, 309 MB/s 2024-04-24T23:04:10.930 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:04:10.930 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:04:10.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:04:10.947 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=292 seek=0) 2024-04-24T23:04:12.065 INFO:teuthology.orchestra.run.smithi032.stderr:292+0 records in 2024-04-24T23:04:12.066 INFO:teuthology.orchestra.run.smithi032.stderr:292+0 records out 2024-04-24T23:04:12.066 INFO:teuthology.orchestra.run.smithi032.stderr:306184192 bytes (306 MB, 292 MiB) copied, 1.0614 s, 288 MB/s 2024-04-24T23:04:12.066 INFO:tasks.cephfs.test_full:Writing file B succeeded (full status will happen soon) 2024-04-24T23:04:12.066 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-24T23:04:12.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.166+0000 7fd4fba38640 1 Processor -- start 2024-04-24T23:04:12.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.166+0000 7fd4fba38640 1 -- start start 2024-04-24T23:04:12.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.167+0000 7fd4fba38640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd4f40b5af0 0x7fd4f40ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:12.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.167+0000 7fd4fba38640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4f40cd270 con 0x7fd4f4162e10 2024-04-24T23:04:12.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.167+0000 7fd4fba38640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4f4071610 con 0x7fd4f4162320 2024-04-24T23:04:12.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.167+0000 7fd4fba38640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fd4f40b5660 con 0x7fd4f40b5af0 2024-04-24T23:04:12.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.167+0000 7fd4f9fd5640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd4f4162e10 0x7fd4f415ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47212/0 (socket says 172.21.15.32:47212) 2024-04-24T23:04:12.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.167+0000 7fd4f9fd5640 1 -- 172.21.15.32:0/1479414193 learned_addr learned my addr 172.21.15.32:0/1479414193 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:04:12.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.167+0000 7fd4f8fd3640 1 --2- 172.21.15.32:0/1479414193 >> v2:172.21.15.121:3300/0 conn(0x7fd4f40b5af0 0x7fd4f40ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:12.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.167+0000 7fd4e7fff640 1 -- 172.21.15.32:0/1479414193 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1856331502 0 0) 0x7fd4f40cd270 con 0x7fd4f4162e10 2024-04-24T23:04:12.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.167+0000 7fd4e7fff640 1 -- 172.21.15.32:0/1479414193 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd4e0003520 con 0x7fd4f4162e10 2024-04-24T23:04:12.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.167+0000 7fd4e7fff640 1 -- 172.21.15.32:0/1479414193 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2573969573 0 0) 0x7fd4f4071610 con 0x7fd4f4162320 2024-04-24T23:04:12.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.167+0000 7fd4e7fff640 1 -- 172.21.15.32:0/1479414193 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd4f40cd270 con 0x7fd4f4162320 2024-04-24T23:04:12.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.167+0000 7fd4f8fd3640 1 -- 172.21.15.32:0/1479414193 >> v1:172.21.15.121:6789/0 conn(0x7fd4f4162320 legacy=0x7fd4f4162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:12.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.167+0000 7fd4f8fd3640 1 -- 172.21.15.32:0/1479414193 >> v1:172.21.15.32:6789/0 conn(0x7fd4f4162e10 legacy=0x7fd4f415ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:12.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.167+0000 7fd4f8fd3640 1 -- 172.21.15.32:0/1479414193 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4f40718f0 con 0x7fd4f40b5af0 2024-04-24T23:04:12.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.168+0000 7fd4f8fd3640 1 --2- 172.21.15.32:0/1479414193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4f40b5af0 0x7fd4f40ccc90 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd4e80097a0 tx=0x7fd4e802a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5e7b5dee70f1b5d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:12.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.168+0000 7fd4e7fff640 1 -- 172.21.15.32:0/1479414193 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e8037070 con 0x7fd4f40b5af0 2024-04-24T23:04:12.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.168+0000 7fd4e7fff640 1 -- 172.21.15.32:0/1479414193 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4e8032440 con 0x7fd4f40b5af0 2024-04-24T23:04:12.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.168+0000 7fd4e7fff640 1 -- 172.21.15.32:0/1479414193 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e8032810 con 0x7fd4f40b5af0 2024-04-24T23:04:12.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.169+0000 7fd4fba38640 1 -- 172.21.15.32:0/1479414193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4f40b5af0 msgr2=0x7fd4f40ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:12.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.169+0000 7fd4fba38640 1 --2- 172.21.15.32:0/1479414193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4f40b5af0 0x7fd4f40ccc90 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd4e80097a0 tx=0x7fd4e802a1a0 comp rx=0 tx=0).stop 2024-04-24T23:04:12.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.169+0000 7fd4fba38640 1 -- 172.21.15.32:0/1479414193 shutdown_connections 2024-04-24T23:04:12.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.169+0000 7fd4fba38640 1 --2- 172.21.15.32:0/1479414193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4f40b5af0 0x7fd4f40ccc90 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:12.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.169+0000 7fd4fba38640 1 -- 172.21.15.32:0/1479414193 >> 172.21.15.32:0/1479414193 conn(0x7fd4f40d0ae0 msgr2=0x7fd4f40d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:12.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.169+0000 7fd4fba38640 1 -- 172.21.15.32:0/1479414193 shutdown_connections 2024-04-24T23:04:12.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.169+0000 7fd4fba38640 1 -- 172.21.15.32:0/1479414193 wait complete. 2024-04-24T23:04:12.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.170+0000 7fd4fba38640 1 Processor -- start 2024-04-24T23:04:12.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.170+0000 7fd4fba38640 1 -- start start 2024-04-24T23:04:12.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.171+0000 7fd4fba38640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4f40b5af0 0x7fd4f40b0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:12.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.171+0000 7fd4fba38640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd4f4162320 0x7fd4f414b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:12.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.171+0000 7fd4fba38640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4f4162e10 0x7fd4f414b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:12.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.171+0000 7fd4fba38640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f4071bd0 con 0x7fd4f4162e10 2024-04-24T23:04:12.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.171+0000 7fd4fba38640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f4067950 con 0x7fd4f40b5af0 2024-04-24T23:04:12.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.171+0000 7fd4fba38640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd4f4071d50 con 0x7fd4f4162320 2024-04-24T23:04:12.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.171+0000 7fd4f9fd5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4f4162e10 0x7fd4f414b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:12.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.171+0000 7fd4f9fd5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4f4162e10 0x7fd4f414b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57088/0 (socket says 172.21.15.32:57088) 2024-04-24T23:04:12.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.171+0000 7fd4f9fd5640 1 -- 172.21.15.32:0/3591522133 learned_addr learned my addr 172.21.15.32:0/3591522133 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:12.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.171+0000 7fd4f97d4640 1 --2- 172.21.15.32:0/3591522133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4f40b5af0 0x7fd4f40b0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:12.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.171+0000 7fd4f8fd3640 1 --2- 172.21.15.32:0/3591522133 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd4f4162320 0x7fd4f414b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:12.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.171+0000 7fd4f9fd5640 1 -- 172.21.15.32:0/3591522133 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd4f4162320 msgr2=0x7fd4f414b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:12.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.171+0000 7fd4f9fd5640 1 --2- 172.21.15.32:0/3591522133 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd4f4162320 0x7fd4f414b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:12.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.171+0000 7fd4f9fd5640 1 -- 172.21.15.32:0/3591522133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4f40b5af0 msgr2=0x7fd4f40b0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:12.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.171+0000 7fd4f9fd5640 1 --2- 172.21.15.32:0/3591522133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4f40b5af0 0x7fd4f40b0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:12.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.171+0000 7fd4f9fd5640 1 -- 172.21.15.32:0/3591522133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4f41e82e0 con 0x7fd4f4162e10 2024-04-24T23:04:12.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.171+0000 7fd4f9fd5640 1 --2- 172.21.15.32:0/3591522133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4f4162e10 0x7fd4f414b6c0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec00cb20 tx=0x7fd4ec00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:12.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.171+0000 7fd4e5ffb640 1 -- 172.21.15.32:0/3591522133 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec017070 con 0x7fd4f4162e10 2024-04-24T23:04:12.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.171+0000 7fd4e5ffb640 1 -- 172.21.15.32:0/3591522133 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4ec00aca0 con 0x7fd4f4162e10 2024-04-24T23:04:12.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.172+0000 7fd4fba38640 1 -- 172.21.15.32:0/3591522133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4f41e85c0 con 0x7fd4f4162e10 2024-04-24T23:04:12.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.172+0000 7fd4e5ffb640 1 -- 172.21.15.32:0/3591522133 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec012600 con 0x7fd4f4162e10 2024-04-24T23:04:12.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.172+0000 7fd4fba38640 1 -- 172.21.15.32:0/3591522133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4f41e8b80 con 0x7fd4f4162e10 2024-04-24T23:04:12.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.173+0000 7fd4fba38640 1 -- 172.21.15.32:0/3591522133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4f4071610 con 0x7fd4f4162e10 2024-04-24T23:04:12.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.173+0000 7fd4e5ffb640 1 -- 172.21.15.32:0/3591522133 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd4ec01a070 con 0x7fd4f4162e10 2024-04-24T23:04:12.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.174+0000 7fd4e5ffb640 1 --2- 172.21.15.32:0/3591522133 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd4d806be30 0x7fd4d806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:12.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.174+0000 7fd4e5ffb640 1 -- 172.21.15.32:0/3591522133 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7fd4ec08ab00 con 0x7fd4f4162e10 2024-04-24T23:04:12.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.174+0000 7fd4f97d4640 1 --2- 172.21.15.32:0/3591522133 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd4d806be30 0x7fd4d806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:12.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.174+0000 7fd4f97d4640 1 --2- 172.21.15.32:0/3591522133 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd4d806be30 0x7fd4d806e2f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0007fd0 tx=0x7fd4f0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:12.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.178+0000 7fd4e5ffb640 1 -- 172.21.15.32:0/3591522133 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd4f4071610 con 0x7fd4f4162e10 2024-04-24T23:04:12.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.364+0000 7fd4fba38640 1 -- 172.21.15.32:0/3591522133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7fd4f4067730 con 0x7fd4f4162e10 2024-04-24T23:04:12.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.365+0000 7fd4e5ffb640 1 -- 172.21.15.32:0/3591522133 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v104) ==== 81+0+34184 (secure 0 0 0) 0x7fd4ec058e60 con 0x7fd4f4162e10 2024-04-24T23:04:12.367 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:04:12.368 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:04:12.368 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 104, 2024-04-24T23:04:12.368 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", 2024-04-24T23:04:12.368 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2024-04-24T22:58:46.964851+0000", 2024-04-24T23:04:12.368 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2024-04-24T23:04:02.427694+0000", 2024-04-24T23:04:12.368 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2024-04-24T22:58:52.048787+0000", 2024-04-24T23:04:12.368 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2024-04-24T22:58:49.045609+0000", 2024-04-24T23:04:12.368 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-24T23:04:12.368 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2024-04-24T23:04:12.368 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2024-04-24T23:04:12.368 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2024-04-24T23:04:12.369 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2024-04-24T23:04:12.369 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2024-04-24T23:04:12.369 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2024-04-24T23:04:12.369 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:04:12.369 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 4, 2024-04-24T23:04:12.369 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.69999998807907104, 2024-04-24T23:04:12.369 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-24T23:04:12.369 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-24T23:04:12.369 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2024-04-24T23:04:12.370 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 13, 2024-04-24T23:04:12.370 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2024-04-24T23:04:12.370 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "luminous", 2024-04-24T23:04:12.370 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "jewel", 2024-04-24T23:04:12.370 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "squid", 2024-04-24T23:04:12.370 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_crimson": false, 2024-04-24T23:04:12.370 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2024-04-24T23:04:12.370 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.370 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2024-04-24T23:04:12.370 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": ".mgr", 2024-04-24T23:04:12.370 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T22:58:53.171408+0000", 2024-04-24T23:04:12.370 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:04:12.370 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:04:12.371 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:04:12.371 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:04:12.371 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:04:12.371 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:04:12.371 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:04:12.371 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:04:12.371 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:04:12.371 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:04:12.372 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:04:12.372 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:04:12.372 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2024-04-24T23:04:12.372 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2024-04-24T23:04:12.372 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2024-04-24T23:04:12.372 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2024-04-24T23:04:12.372 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2024-04-24T23:04:12.372 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:04:12.372 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:04:12.372 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:04:12.372 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:04:12.372 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:04:12.372 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:04:12.373 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:04:12.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.373 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "93", 2024-04-24T23:04:12.373 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:04:12.373 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:04:12.373 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:04:12.373 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:04:12.373 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:04:12.373 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:04:12.374 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:04:12.374 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:04:12.374 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:04:12.374 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:04:12.374 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:04:12.374 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:04:12.374 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:04:12.374 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:04:12.374 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:04:12.374 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:04:12.374 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:04:12.374 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:04:12.375 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:04:12.375 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:04:12.375 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:04:12.375 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:04:12.375 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:04:12.375 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:04:12.375 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:04:12.375 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:04:12.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.375 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:04:12.376 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:04:12.376 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:04:12.376 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:04:12.376 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:04:12.376 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:04:12.376 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:04:12.376 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:04:12.376 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:04:12.376 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:04:12.376 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:04:12.376 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:04:12.377 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_max": 32, 2024-04-24T23:04:12.377 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2024-04-24T23:04:12.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.377 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:04:12.377 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": {} 2024-04-24T23:04:12.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.377 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:04:12.377 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:04:12.378 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 8, 2024-04-24T23:04:12.378 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 8, 2024-04-24T23:04:12.378 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.25, 2024-04-24T23:04:12.378 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-04-24T23:04:12.378 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-04-24T23:04:12.378 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:04:12.378 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:04:12.378 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:04:12.378 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:12.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.379 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 12, 2024-04-24T23:04:12.379 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2024-04-24T23:04:12.379 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:03:59.079157+0000", 2024-04-24T23:04:12.379 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:04:12.379 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:04:12.379 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:04:12.379 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:04:12.379 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:04:12.379 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:04:12.380 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:04:12.380 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:04:12.380 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:04:12.380 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:04:12.380 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:04:12.380 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:04:12.380 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 32, 2024-04-24T23:04:12.380 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 32, 2024-04-24T23:04:12.381 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 32, 2024-04-24T23:04:12.381 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 32, 2024-04-24T23:04:12.381 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 32, 2024-04-24T23:04:12.381 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:04:12.381 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:04:12.381 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:04:12.381 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:04:12.382 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:04:12.382 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:04:12.382 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:04:12.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.382 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "103", 2024-04-24T23:04:12.382 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:04:12.382 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:04:12.383 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:04:12.383 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:04:12.383 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:04:12.383 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:04:12.383 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:04:12.383 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:04:12.383 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:04:12.384 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:04:12.384 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:04:12.384 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:04:12.384 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:04:12.384 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:04:12.385 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:04:12.385 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:04:12.385 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:04:12.385 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:04:12.385 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:04:12.385 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:04:12.386 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:04:12.386 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:04:12.386 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:04:12.386 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:04:12.386 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:04:12.387 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:04:12.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.387 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:04:12.387 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:04:12.387 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:04:12.388 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:04:12.388 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:04:12.388 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:04:12.388 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:04:12.388 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:04:12.388 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:04:12.389 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:04:12.389 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:04:12.389 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:04:12.389 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2024-04-24T23:04:12.389 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2024-04-24T23:04:12.389 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2024-04-24T23:04:12.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.390 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:04:12.390 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:04:12.390 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2024-04-24T23:04:12.390 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:12.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.391 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:04:12.391 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:04:12.391 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.75, 2024-04-24T23:04:12.391 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.75, 2024-04-24T23:04:12.391 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:04:12.391 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.75, 2024-04-24T23:04:12.392 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.75, 2024-04-24T23:04:12.392 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:04:12.392 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:04:12.392 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:04:12.392 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:12.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.393 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.393 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 13, 2024-04-24T23:04:12.393 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2024-04-24T23:04:12.393 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:03:59.765179+0000", 2024-04-24T23:04:12.393 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:04:12.393 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:04:12.394 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:04:12.394 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:04:12.394 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:04:12.394 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:04:12.394 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:04:12.394 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:04:12.395 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:04:12.395 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:04:12.395 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:04:12.395 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:04:12.395 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 64, 2024-04-24T23:04:12.395 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 64, 2024-04-24T23:04:12.396 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 64, 2024-04-24T23:04:12.396 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 64, 2024-04-24T23:04:12.396 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 64, 2024-04-24T23:04:12.396 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:04:12.396 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:04:12.396 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:04:12.397 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:04:12.397 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:04:12.397 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:04:12.397 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:04:12.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.397 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "104", 2024-04-24T23:04:12.398 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:04:12.398 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:04:12.398 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:04:12.401 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:04:12.401 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:04:12.402 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:04:12.402 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:04:12.402 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:04:12.402 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:04:12.402 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:04:12.403 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:04:12.403 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:04:12.403 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:04:12.403 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:04:12.403 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:04:12.403 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:04:12.404 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:04:12.404 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:04:12.404 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:04:12.404 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:04:12.404 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:04:12.404 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:04:12.405 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:04:12.405 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:04:12.405 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:04:12.405 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:04:12.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.405 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:04:12.406 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:04:12.406 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:04:12.406 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:04:12.406 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:04:12.406 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:04:12.406 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:04:12.407 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:04:12.407 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:04:12.407 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:04:12.407 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:04:12.407 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:04:12.407 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 64, 2024-04-24T23:04:12.408 INFO:teuthology.orchestra.run.smithi032.stdout: "target_size_ratio": 0.89999997615814209 2024-04-24T23:04:12.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.408 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:04:12.408 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:04:12.408 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2024-04-24T23:04:12.408 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:12.409 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.409 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:04:12.409 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:04:12.409 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.3799999952316284, 2024-04-24T23:04:12.409 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.3799999952316284, 2024-04-24T23:04:12.409 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:04:12.410 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.3799999952316284, 2024-04-24T23:04:12.410 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.3799999952316284, 2024-04-24T23:04:12.410 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:04:12.410 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:04:12.410 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:04:12.411 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:12.411 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:12.411 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:04:12.411 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-04-24T23:04:12.411 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.411 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:04:12.412 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:12.412 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:12.412 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:12.412 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:12.412 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:12.412 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:12.412 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:12.413 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:12.413 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:12.413 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:12.413 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:12.413 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:12.413 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:12.414 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.414 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:12.414 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2024-04-24T23:04:12.414 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:12.414 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.415 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.415 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:12.415 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2024-04-24T23:04:12.415 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:12.416 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:12.416 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:12.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.416 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:12.417 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:12.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.417 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:12.417 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2024-04-24T23:04:12.417 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:12.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.418 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.418 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:12.418 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2024-04-24T23:04:12.418 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:12.418 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:12.418 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:12.419 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.419 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:12.419 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:12.419 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.419 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:12.419 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2024-04-24T23:04:12.420 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:12.420 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.420 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.420 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:12.420 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2024-04-24T23:04:12.420 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:12.420 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:12.421 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:12.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.421 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:12.421 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:12.421 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.421 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:12.422 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2024-04-24T23:04:12.422 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:12.422 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.422 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.422 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:12.422 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2024-04-24T23:04:12.423 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:12.423 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:12.423 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:12.423 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.423 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/1479395426", 2024-04-24T23:04:12.424 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/1479395426", 2024-04-24T23:04:12.424 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/1479395426", 2024-04-24T23:04:12.424 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/1479395426", 2024-04-24T23:04:12.424 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:12.424 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:12.424 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:12.425 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:12.425 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.425 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.425 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:04:12.425 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:12.425 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:12.426 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:12.426 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:12.426 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:12.426 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:12.426 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:12.426 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:12.427 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:12.427 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:12.427 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:12.427 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:12.427 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:12.427 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.427 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:12.427 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2024-04-24T23:04:12.427 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:12.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.427 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.427 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:12.427 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2024-04-24T23:04:12.428 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:12.428 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:12.428 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:12.428 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.428 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:12.428 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:12.428 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.428 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:12.428 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2024-04-24T23:04:12.428 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:12.428 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.428 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.428 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:12.428 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2024-04-24T23:04:12.429 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:12.429 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:12.429 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:12.429 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.429 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:12.429 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:12.429 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.429 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:12.429 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2024-04-24T23:04:12.429 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:12.429 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.429 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.429 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:12.429 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2024-04-24T23:04:12.430 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:12.430 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:12.430 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:12.430 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.430 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:12.430 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:12.430 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.430 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:12.430 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2024-04-24T23:04:12.430 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:12.430 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.430 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.430 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:12.430 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2024-04-24T23:04:12.430 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:12.431 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:12.431 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:12.431 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.431 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/1340633718", 2024-04-24T23:04:12.431 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/1340633718", 2024-04-24T23:04:12.431 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/1340633718", 2024-04-24T23:04:12.431 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/1340633718", 2024-04-24T23:04:12.431 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:12.431 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:12.431 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:12.431 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:12.431 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.431 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.432 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:04:12.432 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:12.432 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:12.432 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:12.432 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:12.432 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:12.432 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:12.432 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:12.432 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:12.432 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:12.433 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:12.433 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:12.433 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:12.433 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:12.433 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.433 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:12.433 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2024-04-24T23:04:12.434 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:12.434 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.434 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.434 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:12.434 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2024-04-24T23:04:12.434 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:12.434 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:12.434 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:12.434 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.434 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:12.434 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:12.434 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.434 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:12.435 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2024-04-24T23:04:12.435 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:12.435 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.435 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.435 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:12.435 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2024-04-24T23:04:12.435 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:12.435 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:12.436 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:12.436 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.436 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:12.436 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:12.436 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.436 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:12.436 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2024-04-24T23:04:12.436 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:12.436 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.436 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.437 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:12.437 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2024-04-24T23:04:12.437 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:12.437 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:12.437 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:12.437 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.437 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:12.437 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:12.438 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.438 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:12.438 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2024-04-24T23:04:12.438 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:12.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.438 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.438 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:12.438 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2024-04-24T23:04:12.438 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:12.438 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:12.438 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:12.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.439 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/3405899334", 2024-04-24T23:04:12.439 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6820/3405899334", 2024-04-24T23:04:12.439 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6828/3405899334", 2024-04-24T23:04:12.439 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6824/3405899334", 2024-04-24T23:04:12.439 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:12.439 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:12.439 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:12.440 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:12.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.440 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:04:12.440 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:12.440 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:12.440 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:12.440 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:12.440 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:12.440 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:12.440 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:12.441 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:12.441 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:12.441 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:12.441 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:12.441 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:12.441 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:12.442 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.442 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:12.442 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2024-04-24T23:04:12.442 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:12.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.442 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.443 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:12.443 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2024-04-24T23:04:12.443 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:12.443 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:12.443 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:12.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.443 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:12.443 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:12.443 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.443 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:12.444 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2024-04-24T23:04:12.444 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:12.444 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.444 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.444 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:12.444 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2024-04-24T23:04:12.445 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:12.445 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:12.445 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:12.445 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.445 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:12.445 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:12.445 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.445 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:12.446 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2024-04-24T23:04:12.446 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:12.446 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.446 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.446 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:12.446 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2024-04-24T23:04:12.446 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:12.446 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:12.446 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:12.447 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.447 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:12.447 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:12.447 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.447 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:12.447 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2024-04-24T23:04:12.447 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:12.447 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.447 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.447 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:12.447 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2024-04-24T23:04:12.447 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:12.447 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:12.447 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:12.448 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.448 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6821/3381057190", 2024-04-24T23:04:12.448 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6825/3381057190", 2024-04-24T23:04:12.448 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/3381057190", 2024-04-24T23:04:12.448 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/3381057190", 2024-04-24T23:04:12.448 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:12.448 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:12.448 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:12.448 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:12.448 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.448 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.448 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:04:12.448 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:12.449 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:12.449 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:12.449 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:12.449 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:12.449 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:12.449 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:12.449 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:12.449 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:12.449 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:12.449 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:12.449 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:12.449 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:12.449 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.449 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:12.449 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6800", 2024-04-24T23:04:12.450 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:12.450 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.450 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.450 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:12.450 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6801", 2024-04-24T23:04:12.450 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:12.450 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:12.450 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:12.450 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.450 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:12.450 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:12.450 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.450 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:12.451 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6802", 2024-04-24T23:04:12.451 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:12.451 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.451 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.451 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:12.451 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6803", 2024-04-24T23:04:12.451 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:12.451 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:12.451 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:12.451 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.451 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:12.451 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:12.451 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.452 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:12.452 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6806", 2024-04-24T23:04:12.452 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:12.452 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.452 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.452 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:12.452 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6807", 2024-04-24T23:04:12.452 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:12.452 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:12.452 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:12.452 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.452 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:12.452 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:12.452 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.452 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:12.453 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6804", 2024-04-24T23:04:12.453 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:12.453 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.453 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.453 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:12.453 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6805", 2024-04-24T23:04:12.453 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:12.453 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:12.453 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:12.453 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.453 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6801/4010553979", 2024-04-24T23:04:12.453 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6803/4010553979", 2024-04-24T23:04:12.453 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6807/4010553979", 2024-04-24T23:04:12.454 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6805/4010553979", 2024-04-24T23:04:12.454 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:12.454 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:12.454 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:12.454 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:12.454 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.454 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.454 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:04:12.454 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:12.454 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:12.454 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:12.454 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:12.454 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:12.455 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:12.455 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:12.455 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:12.455 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:12.455 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:12.455 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:12.455 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:12.455 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:12.455 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.455 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:12.455 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6808", 2024-04-24T23:04:12.455 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:12.455 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.456 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.456 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:12.456 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6809", 2024-04-24T23:04:12.456 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:12.456 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:12.456 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:12.456 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.456 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:12.456 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:12.456 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.456 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:12.456 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6810", 2024-04-24T23:04:12.456 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:12.456 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.456 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.457 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:12.457 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6811", 2024-04-24T23:04:12.457 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:12.457 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:12.457 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:12.457 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.457 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:12.457 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:12.457 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.457 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:12.457 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6814", 2024-04-24T23:04:12.457 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:12.458 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.458 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.458 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:12.458 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6815", 2024-04-24T23:04:12.458 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:12.458 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:12.458 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:12.458 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.458 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:12.458 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:12.458 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.458 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:12.458 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6812", 2024-04-24T23:04:12.459 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:12.459 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.459 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.459 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:12.459 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6813", 2024-04-24T23:04:12.459 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:12.459 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:12.459 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:12.459 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.459 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6809/3794084091", 2024-04-24T23:04:12.459 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6811/3794084091", 2024-04-24T23:04:12.459 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6815/3794084091", 2024-04-24T23:04:12.459 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6813/3794084091", 2024-04-24T23:04:12.459 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:12.459 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:12.460 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:12.460 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:12.460 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.460 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.460 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:04:12.460 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:12.460 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:12.460 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:12.460 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:12.460 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:12.460 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:12.460 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:12.460 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:12.460 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:12.461 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:12.461 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:12.461 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:12.461 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:12.461 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.461 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:12.461 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6823", 2024-04-24T23:04:12.461 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:12.461 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.461 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.461 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:12.461 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6825", 2024-04-24T23:04:12.461 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:12.461 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:12.462 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:12.462 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.462 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:12.462 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:12.462 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.462 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:12.462 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6826", 2024-04-24T23:04:12.462 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:12.462 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.462 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.462 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:12.462 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6827", 2024-04-24T23:04:12.463 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:12.463 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:12.463 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:12.463 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.463 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:12.463 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:12.463 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.463 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:12.463 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6830", 2024-04-24T23:04:12.463 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:12.463 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.463 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.463 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:12.463 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6831", 2024-04-24T23:04:12.464 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:12.464 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:12.464 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:12.464 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.464 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:12.464 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:12.464 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.464 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:12.464 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6828", 2024-04-24T23:04:12.464 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:12.464 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.464 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.464 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:12.464 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6829", 2024-04-24T23:04:12.464 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:12.465 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:12.465 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:12.465 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.465 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6825/1348640258", 2024-04-24T23:04:12.465 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6827/1348640258", 2024-04-24T23:04:12.465 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6831/1348640258", 2024-04-24T23:04:12.465 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6829/1348640258", 2024-04-24T23:04:12.465 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:12.465 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:12.465 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:12.465 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:12.465 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.465 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.465 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:04:12.465 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:12.466 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:12.466 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:12.466 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:12.466 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:12.466 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:12.466 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:12.466 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:12.466 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:12.466 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:12.466 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:12.466 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:12.466 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:12.466 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.466 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:12.467 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6816", 2024-04-24T23:04:12.467 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:12.467 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.467 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.467 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:12.467 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6817", 2024-04-24T23:04:12.467 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:12.467 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:12.467 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:12.467 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.467 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:12.467 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:12.467 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.467 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:12.467 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6818", 2024-04-24T23:04:12.468 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:12.468 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.468 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.468 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:12.468 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6819", 2024-04-24T23:04:12.468 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:12.468 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:12.468 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:12.468 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.468 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:12.468 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:12.468 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.468 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:12.468 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6822", 2024-04-24T23:04:12.469 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:12.469 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.469 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.469 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:12.469 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6824", 2024-04-24T23:04:12.469 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:12.469 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:12.469 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:12.469 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.469 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:12.469 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:12.469 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.469 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:12.469 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6820", 2024-04-24T23:04:12.470 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:12.470 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.470 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.470 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:12.470 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6821", 2024-04-24T23:04:12.470 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:12.471 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:12.471 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:12.471 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.471 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6817/3250005122", 2024-04-24T23:04:12.471 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6819/3250005122", 2024-04-24T23:04:12.471 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6824/3250005122", 2024-04-24T23:04:12.471 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6821/3250005122", 2024-04-24T23:04:12.471 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:12.471 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:12.471 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:12.471 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:12.471 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:12.471 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:04:12.472 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2024-04-24T23:04:12.472 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.472 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:04:12.472 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:12.472 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:12.472 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:12.472 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:12.472 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:12.472 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.156184+0000", 2024-04-24T23:04:12.472 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:12.473 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.473 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.473 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:04:12.473 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:12.473 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:12.473 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:12.473 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:12.473 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:12.473 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240774+0000", 2024-04-24T23:04:12.474 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:12.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.474 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.474 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:04:12.474 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:12.474 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:12.474 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:12.474 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:12.474 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:12.474 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240949+0000", 2024-04-24T23:04:12.474 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:12.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.475 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.475 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:04:12.475 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:12.475 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:12.475 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:12.475 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:12.475 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:12.476 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.171672+0000", 2024-04-24T23:04:12.476 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:12.476 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.476 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.476 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:04:12.476 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:12.476 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:12.476 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:12.476 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:12.477 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:12.477 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.220379+0000", 2024-04-24T23:04:12.477 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:12.477 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.477 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.477 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:04:12.477 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:12.477 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:12.477 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:12.477 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:12.477 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:12.478 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.204963+0000", 2024-04-24T23:04:12.478 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:12.478 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.478 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.478 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:04:12.478 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:12.478 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:12.478 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:12.478 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:12.479 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:12.479 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.239748+0000", 2024-04-24T23:04:12.479 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:12.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.479 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:12.479 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:04:12.479 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:12.479 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:12.480 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:12.480 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:12.480 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:12.480 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.154104+0000", 2024-04-24T23:04:12.480 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:12.480 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:12.480 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:04:12.480 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2024-04-24T23:04:12.481 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2024-04-24T23:04:12.481 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_primaries": [], 2024-04-24T23:04:12.481 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2024-04-24T23:04:12.481 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2024-04-24T23:04:12.481 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklist": {}, 2024-04-24T23:04:12.481 INFO:teuthology.orchestra.run.smithi032.stdout: "range_blocklist": {}, 2024-04-24T23:04:12.481 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2024-04-24T23:04:12.482 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2024-04-24T23:04:12.482 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2024-04-24T23:04:12.482 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2024-04-24T23:04:12.482 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2024-04-24T23:04:12.482 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2024-04-24T23:04:12.482 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2024-04-24T23:04:12.483 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:12.483 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:12.483 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2024-04-24T23:04:12.483 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2024-04-24T23:04:12.483 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2024-04-24T23:04:12.483 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2024-04-24T23:04:12.483 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {}, 2024-04-24T23:04:12.483 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode": { 2024-04-24T23:04:12.484 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_enabled": false, 2024-04-24T23:04:12.484 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_bucket_count": 0, 2024-04-24T23:04:12.484 INFO:teuthology.orchestra.run.smithi032.stdout: "degraded_stretch_mode": 0, 2024-04-24T23:04:12.484 INFO:teuthology.orchestra.run.smithi032.stdout: "recovering_stretch_mode": 0, 2024-04-24T23:04:12.484 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_bucket": 0 2024-04-24T23:04:12.484 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:12.484 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:04:12.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.367+0000 7fd4fba38640 1 -- 172.21.15.32:0/3591522133 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd4d806be30 msgr2=0x7fd4d806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:12.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.367+0000 7fd4fba38640 1 --2- 172.21.15.32:0/3591522133 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd4d806be30 0x7fd4d806e2f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0007fd0 tx=0x7fd4f0009040 comp rx=0 tx=0).stop 2024-04-24T23:04:12.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.367+0000 7fd4fba38640 1 -- 172.21.15.32:0/3591522133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4f4162e10 msgr2=0x7fd4f414b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:12.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.367+0000 7fd4fba38640 1 --2- 172.21.15.32:0/3591522133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4f4162e10 0x7fd4f414b6c0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec00cb20 tx=0x7fd4ec00a060 comp rx=0 tx=0).stop 2024-04-24T23:04:12.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.367+0000 7fd4fba38640 1 -- 172.21.15.32:0/3591522133 shutdown_connections 2024-04-24T23:04:12.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.367+0000 7fd4fba38640 1 --2- 172.21.15.32:0/3591522133 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd4d806be30 0x7fd4d806e2f0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:12.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.367+0000 7fd4fba38640 1 --2- 172.21.15.32:0/3591522133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4f4162e10 0x7fd4f414b6c0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:12.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.367+0000 7fd4fba38640 1 --2- 172.21.15.32:0/3591522133 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd4f4162320 0x7fd4f414b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:12.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.367+0000 7fd4fba38640 1 --2- 172.21.15.32:0/3591522133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4f40b5af0 0x7fd4f40b0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:12.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.367+0000 7fd4fba38640 1 -- 172.21.15.32:0/3591522133 >> 172.21.15.32:0/3591522133 conn(0x7fd4f40d0ae0 msgr2=0x7fd4f415dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:12.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.367+0000 7fd4fba38640 1 -- 172.21.15.32:0/3591522133 shutdown_connections 2024-04-24T23:04:12.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:12.367+0000 7fd4fba38640 1 -- 172.21.15.32:0/3591522133 wait complete. 2024-04-24T23:04:12.487 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-04-24T23:04:14.459 INFO:tasks.ceph.mon.a.smithi032.stderr:2024-04-24T23:04:14.457+0000 7f3b831cb640 -1 log_channel(cluster) log [ERR] : Health check failed: 2 full osd(s) (OSD_FULL) 2024-04-24T23:04:17.488 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-24T23:04:17.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.592+0000 7facf8aa5640 1 Processor -- start 2024-04-24T23:04:17.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.592+0000 7facf8aa5640 1 -- start start 2024-04-24T23:04:17.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.593+0000 7facf8aa5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7facf406c5f0 0x7facf406c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:17.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.593+0000 7facf8aa5640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7facf40af890 0x7facf415cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:17.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.593+0000 7facf8aa5640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facf40b6430 con 0x7facf40b44d0 2024-04-24T23:04:17.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.593+0000 7facf8aa5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7facf4156f80 con 0x7facf406c5f0 2024-04-24T23:04:17.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.593+0000 7facf8aa5640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7facf40b6150 con 0x7facf40af890 2024-04-24T23:04:17.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.593+0000 7facf259c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7facf406c5f0 0x7facf406c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:17.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.593+0000 7facf259c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7facf406c5f0 0x7facf406c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57090/0 (socket says 172.21.15.32:57090) 2024-04-24T23:04:17.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.593+0000 7facf259c640 1 -- 172.21.15.32:0/1873022490 learned_addr learned my addr 172.21.15.32:0/1873022490 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:17.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.593+0000 7facf259c640 1 -- 172.21.15.32:0/1873022490 >> v2:172.21.15.121:3300/0 conn(0x7facf40af890 msgr2=0x7facf415cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:17.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.593+0000 7facf259c640 1 --2- 172.21.15.32:0/1873022490 >> v2:172.21.15.121:3300/0 conn(0x7facf40af890 0x7facf415cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:17.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.593+0000 7facf259c640 1 -- 172.21.15.32:0/1873022490 >> v1:172.21.15.121:6789/0 conn(0x7facf40b44d0 legacy=0x7facf40b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:17.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.593+0000 7facf259c640 1 -- 172.21.15.32:0/1873022490 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facf40b39f0 con 0x7facf406c5f0 2024-04-24T23:04:17.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.594+0000 7facf259c640 1 --2- 172.21.15.32:0/1873022490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facf406c5f0 0x7facf406c9d0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7facf4073960 tx=0x7face402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca7294db274e7874 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:17.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.594+0000 7facf1d9b640 1 -- 172.21.15.32:0/1873022490 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face4037070 con 0x7facf406c5f0 2024-04-24T23:04:17.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.594+0000 7facf1d9b640 1 -- 172.21.15.32:0/1873022490 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7face4032440 con 0x7facf406c5f0 2024-04-24T23:04:17.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.594+0000 7facf1d9b640 1 -- 172.21.15.32:0/1873022490 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face4032810 con 0x7facf406c5f0 2024-04-24T23:04:17.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.595+0000 7facf8aa5640 1 -- 172.21.15.32:0/1873022490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facf406c5f0 msgr2=0x7facf406c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:17.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.595+0000 7facf8aa5640 1 --2- 172.21.15.32:0/1873022490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facf406c5f0 0x7facf406c9d0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7facf4073960 tx=0x7face402a230 comp rx=0 tx=0).stop 2024-04-24T23:04:17.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.595+0000 7facf8aa5640 1 -- 172.21.15.32:0/1873022490 shutdown_connections 2024-04-24T23:04:17.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.595+0000 7facf8aa5640 1 --2- 172.21.15.32:0/1873022490 >> v2:172.21.15.121:3300/0 conn(0x7facf40af890 0x7facf415cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:17.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.595+0000 7facf8aa5640 1 --2- 172.21.15.32:0/1873022490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facf406c5f0 0x7facf406c9d0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:17.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.595+0000 7facf8aa5640 1 -- 172.21.15.32:0/1873022490 >> 172.21.15.32:0/1873022490 conn(0x7facf406b110 msgr2=0x7facf406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:17.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.595+0000 7facf8aa5640 1 -- 172.21.15.32:0/1873022490 shutdown_connections 2024-04-24T23:04:17.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.595+0000 7facf8aa5640 1 -- 172.21.15.32:0/1873022490 wait complete. 2024-04-24T23:04:17.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.596+0000 7facf8aa5640 1 Processor -- start 2024-04-24T23:04:17.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.596+0000 7facf8aa5640 1 -- start start 2024-04-24T23:04:17.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.597+0000 7facf8aa5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7facf406c5f0 0x7facf41423c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:17.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.597+0000 7facf8aa5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7facf40af890 0x7facf41e76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:17.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.597+0000 7facf8aa5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facf40b44d0 0x7facf41e7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:17.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.597+0000 7facf8aa5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7facf406cf10 con 0x7facf40b44d0 2024-04-24T23:04:17.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.597+0000 7facf8aa5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7facf40b3fb0 con 0x7facf406c5f0 2024-04-24T23:04:17.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.597+0000 7facf8aa5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7facf415d000 con 0x7facf40af890 2024-04-24T23:04:17.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.597+0000 7facf2d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facf40b44d0 0x7facf41e7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:17.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.597+0000 7facf2d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facf40b44d0 0x7facf41e7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57098/0 (socket says 172.21.15.32:57098) 2024-04-24T23:04:17.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.597+0000 7facf2d9d640 1 -- 172.21.15.32:0/3175999864 learned_addr learned my addr 172.21.15.32:0/3175999864 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:17.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.597+0000 7facf259c640 1 --2- 172.21.15.32:0/3175999864 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7facf406c5f0 0x7facf41423c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:17.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.597+0000 7facebfff640 1 --2- 172.21.15.32:0/3175999864 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7facf40af890 0x7facf41e76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:17.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.597+0000 7facf2d9d640 1 -- 172.21.15.32:0/3175999864 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7facf40af890 msgr2=0x7facf41e76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:17.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.597+0000 7facf2d9d640 1 --2- 172.21.15.32:0/3175999864 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7facf40af890 0x7facf41e76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:17.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.597+0000 7facf2d9d640 1 -- 172.21.15.32:0/3175999864 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7facf406c5f0 msgr2=0x7facf41423c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:17.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.597+0000 7facf2d9d640 1 --2- 172.21.15.32:0/3175999864 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7facf406c5f0 0x7facf41423c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:17.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.597+0000 7facf2d9d640 1 -- 172.21.15.32:0/3175999864 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facf41598f0 con 0x7facf40b44d0 2024-04-24T23:04:17.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.597+0000 7facf2d9d640 1 --2- 172.21.15.32:0/3175999864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facf40b44d0 0x7facf41e7c00 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7facec00b970 tx=0x7facec00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:17.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.598+0000 7faceaffd640 1 -- 172.21.15.32:0/3175999864 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facec019070 con 0x7facf40b44d0 2024-04-24T23:04:17.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.598+0000 7faceaffd640 1 -- 172.21.15.32:0/3175999864 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facec00cb60 con 0x7facf40b44d0 2024-04-24T23:04:17.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.598+0000 7facf8aa5640 1 -- 172.21.15.32:0/3175999864 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facf4142e00 con 0x7facf40b44d0 2024-04-24T23:04:17.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.598+0000 7faceaffd640 1 -- 172.21.15.32:0/3175999864 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facec014440 con 0x7facf40b44d0 2024-04-24T23:04:17.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.598+0000 7facf8aa5640 1 -- 172.21.15.32:0/3175999864 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facf41433c0 con 0x7facf40b44d0 2024-04-24T23:04:17.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.599+0000 7facf8aa5640 1 -- 172.21.15.32:0/3175999864 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facf41435c0 con 0x7facf40b44d0 2024-04-24T23:04:17.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.599+0000 7faceaffd640 1 -- 172.21.15.32:0/3175999864 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7facec01c070 con 0x7facf40b44d0 2024-04-24T23:04:17.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.599+0000 7faceaffd640 1 --2- 172.21.15.32:0/3175999864 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7facd406be60 0x7facd406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:17.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.600+0000 7faceaffd640 1 -- 172.21.15.32:0/3175999864 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5709+0+0 (secure 0 0 0) 0x7facec08ab80 con 0x7facf40b44d0 2024-04-24T23:04:17.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.600+0000 7facf259c640 1 --2- 172.21.15.32:0/3175999864 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7facd406be60 0x7facd406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:17.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.600+0000 7faceaffd640 1 -- 172.21.15.32:0/3175999864 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=109}) -- 0x7facd4073990 con 0x7facf40b44d0 2024-04-24T23:04:17.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.600+0000 7facf259c640 1 --2- 172.21.15.32:0/3175999864 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7facd406be60 0x7facd406e320 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7face4034000 tx=0x7face402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:17.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.604+0000 7faceaffd640 1 -- 172.21.15.32:0/3175999864 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7facf41435c0 con 0x7facf40b44d0 2024-04-24T23:04:17.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.788+0000 7facf8aa5640 1 -- 172.21.15.32:0/3175999864 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7facf4068430 con 0x7facf40b44d0 2024-04-24T23:04:17.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.789+0000 7faceaffd640 1 -- 172.21.15.32:0/3175999864 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v108) ==== 81+0+34360 (secure 0 0 0) 0x7facec058e40 con 0x7facf40b44d0 2024-04-24T23:04:17.791 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:04:17.791 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:04:17.791 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 108, 2024-04-24T23:04:17.791 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", 2024-04-24T23:04:17.792 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2024-04-24T22:58:46.964851+0000", 2024-04-24T23:04:17.792 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2024-04-24T23:04:17.477163+0000", 2024-04-24T23:04:17.792 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2024-04-24T22:58:52.048787+0000", 2024-04-24T23:04:17.792 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2024-04-24T22:58:49.045609+0000", 2024-04-24T23:04:17.792 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-24T23:04:17.792 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2024-04-24T23:04:17.792 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2024-04-24T23:04:17.792 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2024-04-24T23:04:17.792 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2024-04-24T23:04:17.792 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2024-04-24T23:04:17.792 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2024-04-24T23:04:17.792 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:04:17.792 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 4, 2024-04-24T23:04:17.792 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.69999998807907104, 2024-04-24T23:04:17.793 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-24T23:04:17.793 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-24T23:04:17.793 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2024-04-24T23:04:17.793 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 13, 2024-04-24T23:04:17.793 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2024-04-24T23:04:17.793 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "luminous", 2024-04-24T23:04:17.793 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "jewel", 2024-04-24T23:04:17.793 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "squid", 2024-04-24T23:04:17.793 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_crimson": false, 2024-04-24T23:04:17.793 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2024-04-24T23:04:17.793 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.793 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2024-04-24T23:04:17.793 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": ".mgr", 2024-04-24T23:04:17.793 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T22:58:53.171408+0000", 2024-04-24T23:04:17.794 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 3, 2024-04-24T23:04:17.794 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,full", 2024-04-24T23:04:17.794 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:04:17.794 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:04:17.794 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:04:17.794 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:04:17.795 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:04:17.795 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:04:17.795 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:04:17.795 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:04:17.795 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:04:17.795 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:04:17.796 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2024-04-24T23:04:17.796 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2024-04-24T23:04:17.796 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2024-04-24T23:04:17.796 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2024-04-24T23:04:17.796 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2024-04-24T23:04:17.796 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:04:17.796 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:04:17.796 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:04:17.797 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:04:17.797 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:04:17.797 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:04:17.797 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:04:17.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.797 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "105", 2024-04-24T23:04:17.797 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:04:17.797 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:04:17.797 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:04:17.797 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:04:17.797 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:04:17.797 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:04:17.797 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:04:17.797 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:04:17.797 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:04:17.798 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:04:17.798 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:04:17.798 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:04:17.798 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:04:17.798 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:04:17.798 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:04:17.798 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:04:17.798 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:04:17.798 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:04:17.798 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:04:17.798 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:04:17.798 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:04:17.798 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:04:17.798 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:04:17.799 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:04:17.799 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:04:17.799 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:04:17.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.799 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:04:17.799 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:04:17.799 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:04:17.799 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:04:17.799 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:04:17.799 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:04:17.799 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:04:17.799 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:04:17.799 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:04:17.799 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:04:17.799 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:04:17.799 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:04:17.800 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_max": 32, 2024-04-24T23:04:17.800 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2024-04-24T23:04:17.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.800 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:04:17.800 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": {} 2024-04-24T23:04:17.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.800 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:04:17.800 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:04:17.800 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 8, 2024-04-24T23:04:17.800 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 8, 2024-04-24T23:04:17.800 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.25, 2024-04-24T23:04:17.800 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-04-24T23:04:17.800 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-04-24T23:04:17.800 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:04:17.800 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:04:17.801 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:04:17.801 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:17.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.801 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 12, 2024-04-24T23:04:17.801 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2024-04-24T23:04:17.801 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:03:59.079157+0000", 2024-04-24T23:04:17.801 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 3, 2024-04-24T23:04:17.801 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,full", 2024-04-24T23:04:17.801 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:04:17.801 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:04:17.801 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:04:17.801 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:04:17.801 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:04:17.801 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:04:17.802 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:04:17.802 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:04:17.802 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:04:17.802 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:04:17.802 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 32, 2024-04-24T23:04:17.802 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 32, 2024-04-24T23:04:17.802 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 32, 2024-04-24T23:04:17.802 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 32, 2024-04-24T23:04:17.802 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 32, 2024-04-24T23:04:17.802 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:04:17.802 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:04:17.803 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:04:17.803 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:04:17.803 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:04:17.803 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:04:17.803 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:04:17.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.803 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "105", 2024-04-24T23:04:17.803 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:04:17.803 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:04:17.803 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:04:17.803 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:04:17.803 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:04:17.803 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:04:17.803 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:04:17.804 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:04:17.804 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:04:17.804 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:04:17.804 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:04:17.804 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:04:17.804 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:04:17.804 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:04:17.804 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:04:17.804 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:04:17.804 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:04:17.804 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:04:17.804 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:04:17.804 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:04:17.804 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:04:17.805 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:04:17.805 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:04:17.805 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:04:17.805 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:04:17.805 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:04:17.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.805 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:04:17.805 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:04:17.805 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:04:17.805 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:04:17.805 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:04:17.806 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:04:17.806 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:04:17.806 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:04:17.806 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:04:17.806 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:04:17.806 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:04:17.806 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:04:17.806 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2024-04-24T23:04:17.806 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2024-04-24T23:04:17.806 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2024-04-24T23:04:17.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.806 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:04:17.807 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:04:17.807 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2024-04-24T23:04:17.807 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:17.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.808 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:04:17.808 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:04:17.808 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.75, 2024-04-24T23:04:17.808 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.75, 2024-04-24T23:04:17.808 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:04:17.809 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.75, 2024-04-24T23:04:17.809 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.75, 2024-04-24T23:04:17.809 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:04:17.809 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:04:17.810 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:04:17.810 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:17.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.811 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 13, 2024-04-24T23:04:17.811 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2024-04-24T23:04:17.811 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:03:59.765179+0000", 2024-04-24T23:04:17.811 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 3, 2024-04-24T23:04:17.812 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,full", 2024-04-24T23:04:17.812 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:04:17.812 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:04:17.812 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:04:17.813 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:04:17.813 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:04:17.813 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:04:17.813 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:04:17.814 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:04:17.814 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:04:17.814 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:04:17.815 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 64, 2024-04-24T23:04:17.815 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 64, 2024-04-24T23:04:17.815 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 64, 2024-04-24T23:04:17.815 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 64, 2024-04-24T23:04:17.815 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 64, 2024-04-24T23:04:17.815 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:04:17.815 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:04:17.816 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:04:17.816 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:04:17.816 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:04:17.816 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:04:17.816 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:04:17.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.817 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "105", 2024-04-24T23:04:17.817 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:04:17.817 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:04:17.817 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:04:17.817 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:04:17.817 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:04:17.817 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:04:17.818 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:04:17.818 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:04:17.818 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:04:17.818 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:04:17.818 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:04:17.818 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:04:17.818 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:04:17.819 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:04:17.819 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:04:17.819 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:04:17.819 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:04:17.819 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:04:17.819 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:04:17.819 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:04:17.819 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:04:17.820 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:04:17.820 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:04:17.820 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:04:17.820 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:04:17.820 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:04:17.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.820 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:04:17.821 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:04:17.821 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:04:17.821 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:04:17.821 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:04:17.821 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:04:17.821 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:04:17.822 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:04:17.822 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:04:17.822 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:04:17.822 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:04:17.822 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:04:17.822 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 64, 2024-04-24T23:04:17.823 INFO:teuthology.orchestra.run.smithi032.stdout: "target_size_ratio": 0.89999997615814209 2024-04-24T23:04:17.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.823 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:04:17.823 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:04:17.823 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2024-04-24T23:04:17.823 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:17.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.824 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:04:17.824 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:04:17.824 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.3799999952316284, 2024-04-24T23:04:17.824 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.3799999952316284, 2024-04-24T23:04:17.824 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:04:17.824 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.3799999952316284, 2024-04-24T23:04:17.824 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.3799999952316284, 2024-04-24T23:04:17.824 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:04:17.825 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:04:17.825 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:04:17.825 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:17.825 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:17.825 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:04:17.825 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-04-24T23:04:17.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.825 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:04:17.826 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:17.826 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:17.826 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:17.826 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:17.826 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:17.826 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:17.826 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:17.826 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:17.827 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:17.827 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:17.827 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:17.827 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:17.827 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:17.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.827 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:17.828 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2024-04-24T23:04:17.828 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:17.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.828 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:17.828 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2024-04-24T23:04:17.829 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:17.829 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:17.829 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:17.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.829 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:17.829 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:17.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.830 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:17.830 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2024-04-24T23:04:17.830 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:17.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.831 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:17.831 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2024-04-24T23:04:17.831 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:17.831 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:17.831 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:17.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.832 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:17.832 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:17.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.832 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:17.832 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2024-04-24T23:04:17.832 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:17.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.833 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:17.833 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2024-04-24T23:04:17.833 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:17.833 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:17.833 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:17.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.834 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:17.834 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:17.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.834 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:17.834 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2024-04-24T23:04:17.835 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:17.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.835 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:17.835 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2024-04-24T23:04:17.835 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:17.835 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:17.836 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:17.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.836 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/1479395426", 2024-04-24T23:04:17.836 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/1479395426", 2024-04-24T23:04:17.836 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/1479395426", 2024-04-24T23:04:17.836 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/1479395426", 2024-04-24T23:04:17.836 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:17.836 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:17.836 INFO:teuthology.orchestra.run.smithi032.stdout: "full", 2024-04-24T23:04:17.836 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:17.836 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:17.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.837 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:04:17.837 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:17.837 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:17.837 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:17.837 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:17.837 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:17.837 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:17.837 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:17.837 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:17.837 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:17.837 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:17.838 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:17.838 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:17.838 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:17.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.838 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:17.838 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2024-04-24T23:04:17.838 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:17.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.838 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:17.838 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2024-04-24T23:04:17.838 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:17.839 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:17.839 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:17.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.839 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:17.839 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:17.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.839 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:17.839 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2024-04-24T23:04:17.839 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:17.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.839 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:17.839 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2024-04-24T23:04:17.839 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:17.840 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:17.840 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:17.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.840 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:17.840 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:17.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.840 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:17.840 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2024-04-24T23:04:17.840 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:17.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.841 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:17.841 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2024-04-24T23:04:17.841 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:17.841 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:17.841 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:17.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.841 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:17.841 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:17.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.841 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:17.841 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2024-04-24T23:04:17.841 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:17.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.842 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:17.842 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2024-04-24T23:04:17.842 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:17.842 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:17.842 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:17.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.842 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/1340633718", 2024-04-24T23:04:17.842 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/1340633718", 2024-04-24T23:04:17.842 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/1340633718", 2024-04-24T23:04:17.843 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/1340633718", 2024-04-24T23:04:17.843 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:17.843 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull", 2024-04-24T23:04:17.843 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:17.843 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:17.843 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:17.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.843 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:04:17.843 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:17.843 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:17.843 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:17.844 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:17.844 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:17.844 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:17.844 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:17.844 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:17.844 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:17.844 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:17.844 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:17.844 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:17.844 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:17.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.844 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:17.845 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2024-04-24T23:04:17.845 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:17.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.845 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:17.845 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2024-04-24T23:04:17.845 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:17.845 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:17.845 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:17.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.845 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:17.846 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:17.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.846 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:17.846 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2024-04-24T23:04:17.847 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:17.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.847 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:17.847 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2024-04-24T23:04:17.848 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:17.848 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:17.848 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:17.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.848 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:17.849 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:17.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.849 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:17.849 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2024-04-24T23:04:17.849 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:17.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.849 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:17.849 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2024-04-24T23:04:17.849 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:17.849 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:17.849 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:17.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.850 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:17.850 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:17.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.850 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:17.850 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2024-04-24T23:04:17.850 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:17.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.851 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:17.851 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2024-04-24T23:04:17.852 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:17.852 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:17.852 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:17.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.853 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/3405899334", 2024-04-24T23:04:17.853 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6820/3405899334", 2024-04-24T23:04:17.853 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6828/3405899334", 2024-04-24T23:04:17.853 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6824/3405899334", 2024-04-24T23:04:17.853 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:17.854 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull", 2024-04-24T23:04:17.854 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:17.854 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:17.854 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:17.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.855 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:04:17.855 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:17.855 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:17.855 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:17.855 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:17.855 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:17.856 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:17.856 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:17.856 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:17.856 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:17.856 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:17.856 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:17.856 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:17.857 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:17.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.857 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:17.857 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2024-04-24T23:04:17.857 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:17.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.858 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:17.858 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2024-04-24T23:04:17.858 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:17.858 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:17.858 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:17.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.858 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:17.859 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:17.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.859 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:17.859 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2024-04-24T23:04:17.859 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:17.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.860 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:17.860 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2024-04-24T23:04:17.860 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:17.860 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:17.860 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:17.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.860 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:17.861 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:17.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.861 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:17.861 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2024-04-24T23:04:17.861 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:17.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.861 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:17.862 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2024-04-24T23:04:17.862 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:17.862 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:17.862 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:17.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.862 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:17.862 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:17.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.863 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:17.863 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2024-04-24T23:04:17.863 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:17.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.863 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:17.864 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2024-04-24T23:04:17.864 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:17.864 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:17.864 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:17.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.864 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6821/3381057190", 2024-04-24T23:04:17.865 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6825/3381057190", 2024-04-24T23:04:17.865 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/3381057190", 2024-04-24T23:04:17.865 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/3381057190", 2024-04-24T23:04:17.865 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:17.865 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:17.865 INFO:teuthology.orchestra.run.smithi032.stdout: "full", 2024-04-24T23:04:17.865 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:17.866 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:17.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.866 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.866 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:04:17.866 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:17.866 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:17.866 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:17.866 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:17.867 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:17.867 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:17.867 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:17.867 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:17.867 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:17.867 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:17.867 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:17.867 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:17.867 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:17.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.868 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:17.868 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6800", 2024-04-24T23:04:17.868 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:17.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.868 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:17.868 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6801", 2024-04-24T23:04:17.869 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:17.869 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:17.869 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:17.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.869 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:17.869 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:17.870 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.870 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:17.870 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6802", 2024-04-24T23:04:17.870 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:17.870 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.870 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.870 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:17.871 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6803", 2024-04-24T23:04:17.871 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:17.871 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:17.871 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:17.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.871 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:17.872 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:17.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.872 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:17.872 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6806", 2024-04-24T23:04:17.872 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:17.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.873 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:17.873 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6807", 2024-04-24T23:04:17.873 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:17.873 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:17.873 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:17.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.874 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:17.874 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:17.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.874 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:17.875 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6804", 2024-04-24T23:04:17.875 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:17.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.876 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:17.876 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6805", 2024-04-24T23:04:17.876 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:17.876 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:17.876 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:17.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.876 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6801/4010553979", 2024-04-24T23:04:17.876 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6803/4010553979", 2024-04-24T23:04:17.876 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6807/4010553979", 2024-04-24T23:04:17.876 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6805/4010553979", 2024-04-24T23:04:17.876 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:17.876 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:17.876 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:17.876 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:17.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.877 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:04:17.877 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:17.877 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:17.877 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:17.877 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:17.877 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:17.877 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:17.877 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:17.878 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:17.878 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:17.878 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:17.878 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:17.878 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:17.878 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:17.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.878 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:17.878 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6808", 2024-04-24T23:04:17.878 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:17.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.878 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:17.878 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6809", 2024-04-24T23:04:17.879 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:17.879 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:17.879 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:17.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.879 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:17.879 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:17.879 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.879 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:17.879 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6810", 2024-04-24T23:04:17.879 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:17.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.880 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:17.880 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6811", 2024-04-24T23:04:17.880 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:17.880 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:17.880 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:17.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.880 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:17.880 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:17.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.880 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:17.880 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6814", 2024-04-24T23:04:17.880 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:17.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.881 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:17.881 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6815", 2024-04-24T23:04:17.881 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:17.881 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:17.881 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:17.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.881 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:17.881 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:17.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.882 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:17.882 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6812", 2024-04-24T23:04:17.882 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:17.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.882 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:17.882 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6813", 2024-04-24T23:04:17.882 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:17.882 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:17.882 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:17.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.883 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6809/3794084091", 2024-04-24T23:04:17.883 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6811/3794084091", 2024-04-24T23:04:17.883 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6815/3794084091", 2024-04-24T23:04:17.883 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6813/3794084091", 2024-04-24T23:04:17.883 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:17.883 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:17.883 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:17.883 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:17.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.883 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:04:17.883 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:17.884 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:17.884 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:17.884 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:17.884 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:17.884 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:17.884 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:17.884 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:17.884 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:17.884 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:17.884 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:17.884 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:17.885 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:17.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.885 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:17.885 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6823", 2024-04-24T23:04:17.885 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:17.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.885 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:17.885 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6825", 2024-04-24T23:04:17.885 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:17.885 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:17.885 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:17.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.886 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:17.886 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:17.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.887 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:17.887 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6826", 2024-04-24T23:04:17.887 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:17.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.888 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.888 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:17.888 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6827", 2024-04-24T23:04:17.888 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:17.888 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:17.888 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:17.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.889 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:17.889 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:17.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.889 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:17.889 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6830", 2024-04-24T23:04:17.890 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:17.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.890 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:17.890 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6831", 2024-04-24T23:04:17.890 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:17.890 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:17.890 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:17.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.890 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:17.891 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:17.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.891 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:17.891 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6828", 2024-04-24T23:04:17.891 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:17.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.891 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:17.891 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6829", 2024-04-24T23:04:17.891 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:17.891 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:17.892 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:17.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.892 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6825/1348640258", 2024-04-24T23:04:17.892 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6827/1348640258", 2024-04-24T23:04:17.892 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6831/1348640258", 2024-04-24T23:04:17.892 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6829/1348640258", 2024-04-24T23:04:17.892 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:17.892 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:17.892 INFO:teuthology.orchestra.run.smithi032.stdout: "full", 2024-04-24T23:04:17.892 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:17.892 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:17.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.892 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.892 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:04:17.893 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:17.893 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:17.893 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:17.893 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:17.893 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:17.893 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:17.893 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:17.893 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:17.893 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:17.893 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:17.893 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:17.893 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:17.893 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:17.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.893 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:17.894 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6816", 2024-04-24T23:04:17.894 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:17.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.894 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.894 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:17.894 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6817", 2024-04-24T23:04:17.894 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:17.894 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:17.894 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:17.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.894 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:17.894 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:17.894 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.894 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:17.895 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6818", 2024-04-24T23:04:17.895 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:17.895 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.895 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.895 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:17.895 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6819", 2024-04-24T23:04:17.895 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:17.895 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:17.895 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:17.895 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.895 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:17.895 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:17.895 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.895 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:17.895 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6822", 2024-04-24T23:04:17.895 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:17.896 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.896 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.896 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:17.896 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6824", 2024-04-24T23:04:17.896 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:17.896 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:17.896 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:17.896 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.896 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:17.896 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:17.896 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.896 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:17.896 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6820", 2024-04-24T23:04:17.896 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:17.896 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.897 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.897 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:17.897 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6821", 2024-04-24T23:04:17.897 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:17.897 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:17.897 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:17.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.897 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6817/3250005122", 2024-04-24T23:04:17.897 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6819/3250005122", 2024-04-24T23:04:17.897 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6824/3250005122", 2024-04-24T23:04:17.897 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6821/3250005122", 2024-04-24T23:04:17.897 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:17.897 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:17.897 INFO:teuthology.orchestra.run.smithi032.stdout: "full", 2024-04-24T23:04:17.897 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:17.898 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:17.898 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:17.898 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:04:17.898 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2024-04-24T23:04:17.898 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.898 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:04:17.898 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:17.898 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:17.898 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:17.898 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:17.898 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:17.898 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.156184+0000", 2024-04-24T23:04:17.898 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:17.898 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.898 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.899 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:04:17.899 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:17.899 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:17.899 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:17.899 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:17.899 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:17.899 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240774+0000", 2024-04-24T23:04:17.899 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:17.899 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.899 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.899 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:04:17.899 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:17.899 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:17.899 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:17.899 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:17.900 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:17.900 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240949+0000", 2024-04-24T23:04:17.900 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:17.900 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.900 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.900 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:04:17.900 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:17.900 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:17.900 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:17.900 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:17.900 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:17.900 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.171672+0000", 2024-04-24T23:04:17.900 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:17.900 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.900 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.900 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:04:17.901 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:17.901 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:17.901 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:17.901 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:17.901 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:17.901 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.220379+0000", 2024-04-24T23:04:17.901 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:17.901 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.901 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.901 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:04:17.901 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:17.901 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:17.901 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:17.901 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:17.901 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:17.902 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.204963+0000", 2024-04-24T23:04:17.902 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:17.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.902 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.902 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:04:17.902 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:17.902 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:17.902 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:17.902 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:17.902 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:17.902 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.239748+0000", 2024-04-24T23:04:17.902 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:17.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.902 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:17.902 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:04:17.903 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:17.903 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:17.903 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:17.903 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:17.903 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:17.903 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.154104+0000", 2024-04-24T23:04:17.903 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:17.903 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:17.903 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:04:17.903 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2024-04-24T23:04:17.903 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2024-04-24T23:04:17.903 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_primaries": [], 2024-04-24T23:04:17.903 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2024-04-24T23:04:17.903 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2024-04-24T23:04:17.903 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklist": {}, 2024-04-24T23:04:17.904 INFO:teuthology.orchestra.run.smithi032.stdout: "range_blocklist": {}, 2024-04-24T23:04:17.904 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2024-04-24T23:04:17.904 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2024-04-24T23:04:17.904 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2024-04-24T23:04:17.904 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2024-04-24T23:04:17.904 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2024-04-24T23:04:17.904 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2024-04-24T23:04:17.904 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2024-04-24T23:04:17.904 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:17.904 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:17.904 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2024-04-24T23:04:17.904 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2024-04-24T23:04:17.904 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2024-04-24T23:04:17.904 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2024-04-24T23:04:17.904 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {}, 2024-04-24T23:04:17.905 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode": { 2024-04-24T23:04:17.905 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_enabled": false, 2024-04-24T23:04:17.905 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_bucket_count": 0, 2024-04-24T23:04:17.905 INFO:teuthology.orchestra.run.smithi032.stdout: "degraded_stretch_mode": 0, 2024-04-24T23:04:17.905 INFO:teuthology.orchestra.run.smithi032.stdout: "recovering_stretch_mode": 0, 2024-04-24T23:04:17.905 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_bucket": 0 2024-04-24T23:04:17.905 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:17.905 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:04:17.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.791+0000 7facf8aa5640 1 -- 172.21.15.32:0/3175999864 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7facd406be60 msgr2=0x7facd406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:17.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.791+0000 7facf8aa5640 1 --2- 172.21.15.32:0/3175999864 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7facd406be60 0x7facd406e320 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7face4034000 tx=0x7face402e000 comp rx=0 tx=0).stop 2024-04-24T23:04:17.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.791+0000 7facf8aa5640 1 -- 172.21.15.32:0/3175999864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facf40b44d0 msgr2=0x7facf41e7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:17.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.791+0000 7facf8aa5640 1 --2- 172.21.15.32:0/3175999864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facf40b44d0 0x7facf41e7c00 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7facec00b970 tx=0x7facec00be30 comp rx=0 tx=0).stop 2024-04-24T23:04:17.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.791+0000 7facf8aa5640 1 -- 172.21.15.32:0/3175999864 shutdown_connections 2024-04-24T23:04:17.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.791+0000 7facf8aa5640 1 --2- 172.21.15.32:0/3175999864 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7facd406be60 0x7facd406e320 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:17.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.791+0000 7facf8aa5640 1 --2- 172.21.15.32:0/3175999864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facf40b44d0 0x7facf41e7c00 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:17.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.791+0000 7facf8aa5640 1 --2- 172.21.15.32:0/3175999864 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7facf40af890 0x7facf41e76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:17.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.791+0000 7facf8aa5640 1 --2- 172.21.15.32:0/3175999864 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7facf406c5f0 0x7facf41423c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:17.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.791+0000 7facf8aa5640 1 -- 172.21.15.32:0/3175999864 >> 172.21.15.32:0/3175999864 conn(0x7facf406b110 msgr2=0x7facf40b4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:17.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.791+0000 7facf8aa5640 1 -- 172.21.15.32:0/3175999864 shutdown_connections 2024-04-24T23:04:17.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:17.791+0000 7facf8aa5640 1 -- 172.21.15.32:0/3175999864 wait complete. 2024-04-24T23:04:17.908 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-04-24T23:04:17.908 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:04:17.909 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:04:17.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:04:17.925 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=265) 2024-04-24T23:04:18.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:04:18.122 INFO:teuthology.orchestra.run.smithi032.stderr:dd: fdatasync failed for 'large_file_b': No space left on device 2024-04-24T23:04:18.122 INFO:teuthology.orchestra.run.smithi032.stderr:50+0 records in 2024-04-24T23:04:18.122 INFO:teuthology.orchestra.run.smithi032.stderr:50+0 records out 2024-04-24T23:04:18.122 INFO:teuthology.orchestra.run.smithi032.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.136688 s, 384 MB/s 2024-04-24T23:04:18.123 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-24T23:04:18.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.222+0000 7efcf1dce640 1 Processor -- start 2024-04-24T23:04:18.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.222+0000 7efcf1dce640 1 -- start start 2024-04-24T23:04:18.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.222+0000 7efcf1dce640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efcec074210 0x7efcec06c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:18.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.222+0000 7efcf1dce640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efcec0b0b80 0x7efcec0745f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:18.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.222+0000 7efcf1dce640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcec0b0770 con 0x7efcec0ae140 2024-04-24T23:04:18.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.222+0000 7efcf1dce640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7efcec0b1c20 con 0x7efcec0b0b80 2024-04-24T23:04:18.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.222+0000 7efcf1dce640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7efcec074b30 con 0x7efcec074210 2024-04-24T23:04:18.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.222+0000 7efceaffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efcec0b0b80 0x7efcec0745f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:18.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.223+0000 7efceaffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efcec0b0b80 0x7efcec0745f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57110/0 (socket says 172.21.15.32:57110) 2024-04-24T23:04:18.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.223+0000 7efcebfff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7efcec0ae140 0x7efcec0b20b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47228/0 (socket says 172.21.15.32:47228) 2024-04-24T23:04:18.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.223+0000 7efceaffd640 1 -- 172.21.15.32:0/3987603252 learned_addr learned my addr 172.21.15.32:0/3987603252 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:18.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.223+0000 7efceb7fe640 1 --2- 172.21.15.32:0/3987603252 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efcec074210 0x7efcec06c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:18.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.223+0000 7efceaffd640 1 -- 172.21.15.32:0/3987603252 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efcec074210 msgr2=0x7efcec06c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:18.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.223+0000 7efceaffd640 1 --2- 172.21.15.32:0/3987603252 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efcec074210 0x7efcec06c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:18.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.223+0000 7efceaffd640 1 -- 172.21.15.32:0/3987603252 >> v1:172.21.15.32:6789/0 conn(0x7efcec0ae140 legacy=0x7efcec0b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:18.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.223+0000 7efcea7fc640 1 -- 172.21.15.32:0/3987603252 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 340450743 0 0) 0x7efcec0b0770 con 0x7efcec0ae140 2024-04-24T23:04:18.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.223+0000 7efceaffd640 1 -- 172.21.15.32:0/3987603252 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcec0b1010 con 0x7efcec0b0b80 2024-04-24T23:04:18.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.224+0000 7efceaffd640 1 --2- 172.21.15.32:0/3987603252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcec0b0b80 0x7efcec0745f0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7efce0009870 tx=0x7efce002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a5f1c09cad9a2c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:18.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.224+0000 7efcea7fc640 1 -- 172.21.15.32:0/3987603252 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce0037070 con 0x7efcec0b0b80 2024-04-24T23:04:18.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.224+0000 7efcea7fc640 1 -- 172.21.15.32:0/3987603252 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efce0032440 con 0x7efcec0b0b80 2024-04-24T23:04:18.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.224+0000 7efcea7fc640 1 -- 172.21.15.32:0/3987603252 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce0032810 con 0x7efcec0b0b80 2024-04-24T23:04:18.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.224+0000 7efcf1dce640 1 -- 172.21.15.32:0/3987603252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcec0b0b80 msgr2=0x7efcec0745f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:18.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.224+0000 7efcf1dce640 1 --2- 172.21.15.32:0/3987603252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcec0b0b80 0x7efcec0745f0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7efce0009870 tx=0x7efce002a270 comp rx=0 tx=0).stop 2024-04-24T23:04:18.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.224+0000 7efcf1dce640 1 -- 172.21.15.32:0/3987603252 shutdown_connections 2024-04-24T23:04:18.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.224+0000 7efcf1dce640 1 --2- 172.21.15.32:0/3987603252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcec0b0b80 0x7efcec0745f0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:18.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.224+0000 7efcf1dce640 1 --2- 172.21.15.32:0/3987603252 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efcec074210 0x7efcec06c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:18.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.224+0000 7efcf1dce640 1 -- 172.21.15.32:0/3987603252 >> 172.21.15.32:0/3987603252 conn(0x7efcec0b2e60 msgr2=0x7efcec0b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:18.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.224+0000 7efcf1dce640 1 -- 172.21.15.32:0/3987603252 shutdown_connections 2024-04-24T23:04:18.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.224+0000 7efcf1dce640 1 -- 172.21.15.32:0/3987603252 wait complete. 2024-04-24T23:04:18.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.225+0000 7efcf1dce640 1 Processor -- start 2024-04-24T23:04:18.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.226+0000 7efcf1dce640 1 -- start start 2024-04-24T23:04:18.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.226+0000 7efcf1dce640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcec074210 0x7efcec15dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:18.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.226+0000 7efcf1dce640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcec0ae140 0x7efcec0cd720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:18.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.226+0000 7efcf1dce640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efcec0cd190 0x7efcec0cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:18.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.226+0000 7efcf1dce640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7efcec0b18b0 con 0x7efcec074210 2024-04-24T23:04:18.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.226+0000 7efcf1dce640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7efcec0b15d0 con 0x7efcec0ae140 2024-04-24T23:04:18.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.226+0000 7efcf1dce640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7efcec0b3ba0 con 0x7efcec0cd190 2024-04-24T23:04:18.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.226+0000 7efceb7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcec074210 0x7efcec15dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:18.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.227+0000 7efceb7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcec074210 0x7efcec15dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57112/0 (socket says 172.21.15.32:57112) 2024-04-24T23:04:18.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.227+0000 7efceb7fe640 1 -- 172.21.15.32:0/2399263215 learned_addr learned my addr 172.21.15.32:0/2399263215 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:18.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.227+0000 7efceb7fe640 1 -- 172.21.15.32:0/2399263215 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efcec0cd190 msgr2=0x7efcec0cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:18.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.227+0000 7efcebfff640 1 --2- 172.21.15.32:0/2399263215 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efcec0cd190 0x7efcec0cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:18.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.227+0000 7efceb7fe640 1 --2- 172.21.15.32:0/2399263215 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efcec0cd190 0x7efcec0cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:18.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.227+0000 7efceb7fe640 1 -- 172.21.15.32:0/2399263215 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcec0ae140 msgr2=0x7efcec0cd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:18.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.227+0000 7efceaffd640 1 --2- 172.21.15.32:0/2399263215 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcec0ae140 0x7efcec0cd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:18.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.227+0000 7efceb7fe640 1 --2- 172.21.15.32:0/2399263215 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcec0ae140 0x7efcec0cd720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:18.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.227+0000 7efceb7fe640 1 -- 172.21.15.32:0/2399263215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcec15e4f0 con 0x7efcec074210 2024-04-24T23:04:18.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.227+0000 7efcebfff640 1 --2- 172.21.15.32:0/2399263215 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efcec0cd190 0x7efcec0cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:04:18.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.227+0000 7efceaffd640 1 --2- 172.21.15.32:0/2399263215 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcec0ae140 0x7efcec0cd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:04:18.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.227+0000 7efceb7fe640 1 --2- 172.21.15.32:0/2399263215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcec074210 0x7efcec15dfb0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7efcdc00d5d0 tx=0x7efcdc00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:18.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.227+0000 7efcd3fff640 1 -- 172.21.15.32:0/2399263215 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcdc0125f0 con 0x7efcec074210 2024-04-24T23:04:18.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.227+0000 7efcd3fff640 1 -- 172.21.15.32:0/2399263215 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcdc019070 con 0x7efcec074210 2024-04-24T23:04:18.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.227+0000 7efcf1dce640 1 -- 172.21.15.32:0/2399263215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcec0c6a00 con 0x7efcec074210 2024-04-24T23:04:18.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.227+0000 7efcd3fff640 1 -- 172.21.15.32:0/2399263215 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcdc0129f0 con 0x7efcec074210 2024-04-24T23:04:18.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.227+0000 7efcf1dce640 1 -- 172.21.15.32:0/2399263215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcec0c6fc0 con 0x7efcec074210 2024-04-24T23:04:18.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.228+0000 7efcf1dce640 1 -- 172.21.15.32:0/2399263215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcec06ee00 con 0x7efcec074210 2024-04-24T23:04:18.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.229+0000 7efcd3fff640 1 -- 172.21.15.32:0/2399263215 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efcdc012b90 con 0x7efcec074210 2024-04-24T23:04:18.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.229+0000 7efcd3fff640 1 --2- 172.21.15.32:0/2399263215 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7efcd806be30 0x7efcd806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:18.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.229+0000 7efcd3fff640 1 -- 172.21.15.32:0/2399263215 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5709+0+0 (secure 0 0 0) 0x7efcdc08b240 con 0x7efcec074210 2024-04-24T23:04:18.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.229+0000 7efcd3fff640 1 -- 172.21.15.32:0/2399263215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=109}) -- 0x7efcd80739c0 con 0x7efcec074210 2024-04-24T23:04:18.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.230+0000 7efceaffd640 1 --2- 172.21.15.32:0/2399263215 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7efcd806be30 0x7efcd806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:18.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.230+0000 7efceaffd640 1 --2- 172.21.15.32:0/2399263215 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7efcd806be30 0x7efcd806e2f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7efce0034000 tx=0x7efce002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:18.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.234+0000 7efcd3fff640 1 -- 172.21.15.32:0/2399263215 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7efcec06ee00 con 0x7efcec074210 2024-04-24T23:04:18.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.416+0000 7efcf1dce640 1 -- 172.21.15.32:0/2399263215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7efcec068430 con 0x7efcec074210 2024-04-24T23:04:18.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.417+0000 7efcd3fff640 1 -- 172.21.15.32:0/2399263215 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v108) ==== 81+0+34360 (secure 0 0 0) 0x7efcec15e4f0 con 0x7efcec074210 2024-04-24T23:04:18.419 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:04:18.420 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:04:18.420 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 108, 2024-04-24T23:04:18.420 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", 2024-04-24T23:04:18.420 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2024-04-24T22:58:46.964851+0000", 2024-04-24T23:04:18.420 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2024-04-24T23:04:17.477163+0000", 2024-04-24T23:04:18.420 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2024-04-24T22:58:52.048787+0000", 2024-04-24T23:04:18.420 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2024-04-24T22:58:49.045609+0000", 2024-04-24T23:04:18.420 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-24T23:04:18.420 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2024-04-24T23:04:18.420 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2024-04-24T23:04:18.420 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2024-04-24T23:04:18.420 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2024-04-24T23:04:18.420 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2024-04-24T23:04:18.420 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2024-04-24T23:04:18.421 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:04:18.421 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 4, 2024-04-24T23:04:18.421 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.69999998807907104, 2024-04-24T23:04:18.421 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-24T23:04:18.421 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-24T23:04:18.421 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2024-04-24T23:04:18.421 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 13, 2024-04-24T23:04:18.421 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2024-04-24T23:04:18.421 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "luminous", 2024-04-24T23:04:18.421 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "jewel", 2024-04-24T23:04:18.421 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "squid", 2024-04-24T23:04:18.421 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_crimson": false, 2024-04-24T23:04:18.421 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2024-04-24T23:04:18.421 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.421 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2024-04-24T23:04:18.422 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": ".mgr", 2024-04-24T23:04:18.422 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T22:58:53.171408+0000", 2024-04-24T23:04:18.422 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 3, 2024-04-24T23:04:18.422 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,full", 2024-04-24T23:04:18.422 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:04:18.422 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:04:18.422 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:04:18.422 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:04:18.422 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:04:18.422 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:04:18.423 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:04:18.423 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:04:18.423 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:04:18.423 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:04:18.423 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2024-04-24T23:04:18.423 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2024-04-24T23:04:18.423 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2024-04-24T23:04:18.423 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2024-04-24T23:04:18.423 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2024-04-24T23:04:18.423 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:04:18.423 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:04:18.423 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:04:18.423 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:04:18.423 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:04:18.424 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:04:18.424 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:04:18.424 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.424 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "105", 2024-04-24T23:04:18.424 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:04:18.424 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:04:18.424 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:04:18.424 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:04:18.424 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:04:18.424 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:04:18.424 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:04:18.424 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:04:18.424 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:04:18.424 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:04:18.424 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:04:18.425 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:04:18.425 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:04:18.425 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:04:18.425 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:04:18.425 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:04:18.425 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:04:18.425 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:04:18.425 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:04:18.425 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:04:18.425 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:04:18.425 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:04:18.425 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:04:18.425 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:04:18.425 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:04:18.425 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:04:18.426 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.426 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:04:18.426 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:04:18.426 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:04:18.426 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:04:18.426 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:04:18.426 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:04:18.426 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:04:18.426 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:04:18.426 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:04:18.426 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:04:18.426 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:04:18.426 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:04:18.426 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_max": 32, 2024-04-24T23:04:18.426 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2024-04-24T23:04:18.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.427 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:04:18.427 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": {} 2024-04-24T23:04:18.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.427 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:04:18.427 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:04:18.427 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 8, 2024-04-24T23:04:18.427 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 8, 2024-04-24T23:04:18.427 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.25, 2024-04-24T23:04:18.427 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-04-24T23:04:18.427 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-04-24T23:04:18.427 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:04:18.427 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:04:18.427 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:04:18.427 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:18.428 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.428 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.428 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 12, 2024-04-24T23:04:18.428 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2024-04-24T23:04:18.428 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:03:59.079157+0000", 2024-04-24T23:04:18.428 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 3, 2024-04-24T23:04:18.428 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,full", 2024-04-24T23:04:18.428 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:04:18.428 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:04:18.428 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:04:18.428 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:04:18.428 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:04:18.428 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:04:18.428 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:04:18.428 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:04:18.429 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:04:18.429 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:04:18.429 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 32, 2024-04-24T23:04:18.429 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 32, 2024-04-24T23:04:18.429 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 32, 2024-04-24T23:04:18.429 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 32, 2024-04-24T23:04:18.429 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 32, 2024-04-24T23:04:18.429 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:04:18.429 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:04:18.429 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:04:18.429 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:04:18.429 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:04:18.429 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:04:18.429 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:04:18.429 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.429 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "105", 2024-04-24T23:04:18.430 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:04:18.430 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:04:18.430 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:04:18.430 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:04:18.430 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:04:18.430 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:04:18.430 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:04:18.430 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:04:18.430 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:04:18.430 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:04:18.430 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:04:18.430 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:04:18.430 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:04:18.430 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:04:18.431 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:04:18.431 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:04:18.431 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:04:18.431 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:04:18.431 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:04:18.431 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:04:18.431 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:04:18.431 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:04:18.431 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:04:18.431 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:04:18.431 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:04:18.431 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:04:18.431 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.431 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:04:18.431 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:04:18.432 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:04:18.432 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:04:18.432 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:04:18.432 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:04:18.432 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:04:18.432 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:04:18.432 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:04:18.432 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:04:18.432 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:04:18.432 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:04:18.432 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2024-04-24T23:04:18.432 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2024-04-24T23:04:18.432 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2024-04-24T23:04:18.432 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.432 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:04:18.432 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:04:18.433 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2024-04-24T23:04:18.433 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:18.433 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.433 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:04:18.433 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:04:18.433 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.75, 2024-04-24T23:04:18.433 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.75, 2024-04-24T23:04:18.433 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:04:18.433 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.75, 2024-04-24T23:04:18.433 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.75, 2024-04-24T23:04:18.433 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:04:18.433 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:04:18.433 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:04:18.433 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:18.433 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.434 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.434 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 13, 2024-04-24T23:04:18.434 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2024-04-24T23:04:18.434 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:03:59.765179+0000", 2024-04-24T23:04:18.434 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 3, 2024-04-24T23:04:18.434 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,full", 2024-04-24T23:04:18.434 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:04:18.434 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:04:18.434 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:04:18.434 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:04:18.434 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:04:18.434 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:04:18.434 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:04:18.434 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:04:18.434 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:04:18.435 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:04:18.435 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 64, 2024-04-24T23:04:18.435 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 64, 2024-04-24T23:04:18.435 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 64, 2024-04-24T23:04:18.435 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 64, 2024-04-24T23:04:18.435 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 64, 2024-04-24T23:04:18.435 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:04:18.435 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:04:18.435 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:04:18.435 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:04:18.435 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:04:18.435 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:04:18.435 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:04:18.435 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.436 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "105", 2024-04-24T23:04:18.436 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:04:18.436 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:04:18.436 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:04:18.436 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:04:18.436 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:04:18.436 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:04:18.436 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:04:18.436 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:04:18.436 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:04:18.436 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:04:18.436 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:04:18.436 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:04:18.436 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:04:18.436 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:04:18.437 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:04:18.437 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:04:18.437 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:04:18.437 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:04:18.437 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:04:18.437 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:04:18.437 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:04:18.437 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:04:18.437 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:04:18.437 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:04:18.437 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:04:18.437 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:04:18.437 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.437 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:04:18.437 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:04:18.437 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:04:18.438 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:04:18.438 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:04:18.438 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:04:18.438 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:04:18.438 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:04:18.438 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:04:18.438 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:04:18.438 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:04:18.438 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:04:18.438 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 64, 2024-04-24T23:04:18.438 INFO:teuthology.orchestra.run.smithi032.stdout: "target_size_ratio": 0.89999997615814209 2024-04-24T23:04:18.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.438 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:04:18.438 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:04:18.439 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2024-04-24T23:04:18.439 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:18.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.439 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:04:18.439 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:04:18.439 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.3799999952316284, 2024-04-24T23:04:18.439 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.3799999952316284, 2024-04-24T23:04:18.439 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:04:18.439 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.3799999952316284, 2024-04-24T23:04:18.439 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.3799999952316284, 2024-04-24T23:04:18.439 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:04:18.439 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:04:18.439 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:04:18.439 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:18.439 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:18.439 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:04:18.440 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-04-24T23:04:18.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.440 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:04:18.440 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:18.440 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:18.440 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:18.440 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:18.440 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:18.440 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:18.440 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:18.440 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:18.440 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:18.440 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:18.440 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:18.440 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:18.441 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:18.441 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.441 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:18.441 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2024-04-24T23:04:18.441 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:18.441 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.441 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.441 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:18.441 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2024-04-24T23:04:18.441 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:18.441 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:18.441 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:18.441 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.441 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:18.441 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:18.442 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.442 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:18.442 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2024-04-24T23:04:18.442 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:18.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.442 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.442 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:18.442 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2024-04-24T23:04:18.442 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:18.442 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:18.442 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:18.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.442 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:18.442 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:18.442 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.443 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:18.443 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2024-04-24T23:04:18.443 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:18.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.443 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.443 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:18.443 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2024-04-24T23:04:18.443 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:18.443 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:18.443 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:18.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.443 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:18.443 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:18.443 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.443 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:18.444 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2024-04-24T23:04:18.444 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:18.444 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.444 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.444 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:18.444 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2024-04-24T23:04:18.444 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:18.444 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:18.444 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:18.444 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.444 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/1479395426", 2024-04-24T23:04:18.444 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/1479395426", 2024-04-24T23:04:18.445 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/1479395426", 2024-04-24T23:04:18.445 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/1479395426", 2024-04-24T23:04:18.445 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:18.445 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:18.445 INFO:teuthology.orchestra.run.smithi032.stdout: "full", 2024-04-24T23:04:18.445 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:18.445 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:18.445 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.445 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.445 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:04:18.445 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:18.445 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:18.445 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:18.445 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:18.446 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:18.446 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:18.446 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:18.446 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:18.446 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:18.446 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:18.446 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:18.446 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:18.446 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:18.446 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.446 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:18.446 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2024-04-24T23:04:18.446 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:18.446 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.446 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.447 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:18.447 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2024-04-24T23:04:18.447 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:18.447 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:18.447 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:18.447 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.447 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:18.447 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:18.447 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.447 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:18.447 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2024-04-24T23:04:18.447 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:18.447 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.447 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.447 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:18.448 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2024-04-24T23:04:18.448 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:18.448 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:18.448 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:18.448 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.448 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:18.448 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:18.448 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.448 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:18.448 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2024-04-24T23:04:18.448 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:18.448 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.448 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.448 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:18.448 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2024-04-24T23:04:18.448 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:18.449 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:18.449 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:18.449 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.449 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:18.449 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:18.449 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.449 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:18.449 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2024-04-24T23:04:18.449 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:18.449 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.449 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.449 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:18.449 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2024-04-24T23:04:18.449 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:18.449 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:18.450 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:18.450 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.450 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/1340633718", 2024-04-24T23:04:18.450 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/1340633718", 2024-04-24T23:04:18.450 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/1340633718", 2024-04-24T23:04:18.450 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/1340633718", 2024-04-24T23:04:18.450 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:18.450 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull", 2024-04-24T23:04:18.450 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:18.450 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:18.450 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:18.450 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.450 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.450 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:04:18.450 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:18.451 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:18.451 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:18.451 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:18.451 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:18.451 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:18.451 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:18.451 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:18.451 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:18.451 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:18.451 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:18.451 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:18.451 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:18.451 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.452 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:18.452 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2024-04-24T23:04:18.452 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:18.452 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.452 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.452 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:18.452 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2024-04-24T23:04:18.452 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:18.452 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:18.452 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:18.452 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.452 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:18.452 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:18.452 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.453 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:18.453 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2024-04-24T23:04:18.453 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:18.453 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.453 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.453 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:18.453 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2024-04-24T23:04:18.453 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:18.453 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:18.453 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:18.453 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.453 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:18.453 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:18.453 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.453 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:18.454 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2024-04-24T23:04:18.454 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:18.454 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.454 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.454 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:18.454 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2024-04-24T23:04:18.454 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:18.454 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:18.454 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:18.454 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.454 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:18.454 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:18.454 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.454 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:18.454 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2024-04-24T23:04:18.455 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:18.455 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.455 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.455 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:18.455 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2024-04-24T23:04:18.455 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:18.455 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:18.455 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:18.455 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.455 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/3405899334", 2024-04-24T23:04:18.455 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6820/3405899334", 2024-04-24T23:04:18.455 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6828/3405899334", 2024-04-24T23:04:18.455 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6824/3405899334", 2024-04-24T23:04:18.455 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:18.455 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull", 2024-04-24T23:04:18.456 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:18.456 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:18.456 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:18.456 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.456 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.456 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:04:18.456 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:18.456 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:18.456 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:18.456 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:18.456 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:18.456 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:18.456 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:18.456 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:18.456 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:18.457 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:18.457 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:18.457 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:18.457 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:18.457 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.457 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:18.457 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2024-04-24T23:04:18.457 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:18.457 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.457 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.457 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:18.457 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2024-04-24T23:04:18.457 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:18.457 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:18.457 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:18.457 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.458 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:18.458 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:18.458 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.458 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:18.458 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2024-04-24T23:04:18.458 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:18.458 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.458 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.458 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:18.458 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2024-04-24T23:04:18.458 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:18.458 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:18.458 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:18.458 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.458 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:18.459 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:18.459 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.459 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:18.459 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2024-04-24T23:04:18.459 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:18.459 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.459 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.459 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:18.459 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2024-04-24T23:04:18.459 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:18.459 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:18.459 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:18.459 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.459 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:18.460 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:18.460 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.460 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:18.460 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2024-04-24T23:04:18.460 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:18.460 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.460 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.460 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:18.460 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2024-04-24T23:04:18.460 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:18.460 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:18.460 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:18.460 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.460 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6821/3381057190", 2024-04-24T23:04:18.460 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6825/3381057190", 2024-04-24T23:04:18.460 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/3381057190", 2024-04-24T23:04:18.461 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/3381057190", 2024-04-24T23:04:18.461 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:18.461 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:18.461 INFO:teuthology.orchestra.run.smithi032.stdout: "full", 2024-04-24T23:04:18.461 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:18.461 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:18.461 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.461 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.461 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:04:18.461 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:18.461 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:18.461 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:18.461 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:18.461 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:18.461 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:18.462 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:18.462 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:18.462 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:18.462 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:18.462 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:18.462 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:18.462 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:18.462 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.462 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:18.462 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6800", 2024-04-24T23:04:18.462 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:18.462 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.462 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.462 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:18.462 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6801", 2024-04-24T23:04:18.463 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:18.463 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:18.463 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:18.463 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.463 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:18.463 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:18.463 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.463 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:18.463 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6802", 2024-04-24T23:04:18.463 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:18.463 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.463 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.463 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:18.463 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6803", 2024-04-24T23:04:18.463 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:18.463 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:18.464 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:18.464 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.464 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:18.464 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:18.464 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.464 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:18.464 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6806", 2024-04-24T23:04:18.464 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:18.464 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.464 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.464 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:18.464 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6807", 2024-04-24T23:04:18.464 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:18.464 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:18.464 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:18.465 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.465 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:18.465 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:18.465 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.465 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:18.465 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6804", 2024-04-24T23:04:18.465 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:18.465 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.465 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.465 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:18.465 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6805", 2024-04-24T23:04:18.465 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:18.465 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:18.465 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:18.465 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.465 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6801/4010553979", 2024-04-24T23:04:18.466 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6803/4010553979", 2024-04-24T23:04:18.466 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6807/4010553979", 2024-04-24T23:04:18.466 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6805/4010553979", 2024-04-24T23:04:18.466 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:18.466 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:18.466 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:18.466 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:18.466 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.466 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.466 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:04:18.466 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:18.466 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:18.466 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:18.466 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:18.466 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:18.467 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:18.467 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:18.467 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:18.467 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:18.467 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:18.467 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:18.467 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:18.467 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:18.467 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.467 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:18.467 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6808", 2024-04-24T23:04:18.467 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:18.467 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.467 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.467 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:18.468 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6809", 2024-04-24T23:04:18.468 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:18.468 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:18.468 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:18.468 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.468 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:18.468 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:18.468 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.468 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:18.468 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6810", 2024-04-24T23:04:18.468 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:18.468 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.468 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.468 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:18.468 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6811", 2024-04-24T23:04:18.469 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:18.469 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:18.469 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:18.469 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.469 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:18.469 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:18.469 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.469 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:18.469 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6814", 2024-04-24T23:04:18.469 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:18.469 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.469 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.469 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:18.469 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6815", 2024-04-24T23:04:18.469 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:18.470 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:18.470 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:18.470 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.470 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:18.470 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:18.470 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.470 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:18.470 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6812", 2024-04-24T23:04:18.470 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:18.470 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.470 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.470 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:18.470 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6813", 2024-04-24T23:04:18.470 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:18.471 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:18.471 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:18.471 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.471 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6809/3794084091", 2024-04-24T23:04:18.471 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6811/3794084091", 2024-04-24T23:04:18.471 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6815/3794084091", 2024-04-24T23:04:18.471 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6813/3794084091", 2024-04-24T23:04:18.471 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:18.471 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:18.471 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:18.471 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:18.471 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.471 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.471 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:04:18.471 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:18.472 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:18.472 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:18.472 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:18.472 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:18.472 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:18.472 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:18.472 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:18.472 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:18.472 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:18.472 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:18.472 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:18.472 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:18.472 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.472 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:18.472 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6823", 2024-04-24T23:04:18.472 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:18.473 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.473 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.473 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:18.473 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6825", 2024-04-24T23:04:18.473 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:18.473 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:18.473 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:18.473 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.473 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:18.473 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:18.473 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.473 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:18.473 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6826", 2024-04-24T23:04:18.473 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:18.473 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.474 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.474 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:18.474 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6827", 2024-04-24T23:04:18.474 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:18.474 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:18.474 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:18.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.474 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:18.474 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:18.474 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.474 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:18.474 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6830", 2024-04-24T23:04:18.474 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:18.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.474 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.475 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:18.475 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6831", 2024-04-24T23:04:18.475 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:18.475 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:18.475 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:18.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.475 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:18.475 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:18.475 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.475 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:18.475 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6828", 2024-04-24T23:04:18.475 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:18.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.475 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.475 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:18.476 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6829", 2024-04-24T23:04:18.476 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:18.476 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:18.476 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:18.476 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.476 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6825/1348640258", 2024-04-24T23:04:18.476 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6827/1348640258", 2024-04-24T23:04:18.476 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6831/1348640258", 2024-04-24T23:04:18.476 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6829/1348640258", 2024-04-24T23:04:18.476 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:18.476 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:18.476 INFO:teuthology.orchestra.run.smithi032.stdout: "full", 2024-04-24T23:04:18.476 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:18.476 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:18.476 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.476 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.477 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:04:18.477 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:18.477 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:18.477 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:18.477 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:18.477 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:18.477 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:18.477 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:18.477 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:18.477 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:18.477 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:18.477 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:18.477 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:18.477 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:18.477 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.478 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:18.478 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6816", 2024-04-24T23:04:18.478 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:18.478 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.478 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.478 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:18.478 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6817", 2024-04-24T23:04:18.478 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:18.478 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:18.478 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:18.478 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.478 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:18.478 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:18.478 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.478 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:18.479 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6818", 2024-04-24T23:04:18.479 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:18.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.479 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.479 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:18.479 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6819", 2024-04-24T23:04:18.479 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:18.479 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:18.479 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:18.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.479 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:18.479 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:18.479 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.479 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:18.479 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6822", 2024-04-24T23:04:18.480 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:18.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.480 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.480 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:18.480 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6824", 2024-04-24T23:04:18.480 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:18.480 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:18.480 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:18.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.480 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:18.480 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:18.480 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.480 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:18.480 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6820", 2024-04-24T23:04:18.480 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:18.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.481 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.481 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:18.481 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6821", 2024-04-24T23:04:18.481 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:18.481 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:18.481 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:18.481 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.481 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6817/3250005122", 2024-04-24T23:04:18.481 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6819/3250005122", 2024-04-24T23:04:18.481 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6824/3250005122", 2024-04-24T23:04:18.481 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6821/3250005122", 2024-04-24T23:04:18.481 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:18.481 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:18.481 INFO:teuthology.orchestra.run.smithi032.stdout: "full", 2024-04-24T23:04:18.481 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:18.482 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:18.482 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:18.482 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:04:18.482 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2024-04-24T23:04:18.482 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.482 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:04:18.482 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:18.482 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:18.482 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:18.482 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:18.482 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:18.482 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.156184+0000", 2024-04-24T23:04:18.482 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:18.482 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.482 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.483 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:04:18.483 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:18.483 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:18.483 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:18.483 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:18.483 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:18.483 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240774+0000", 2024-04-24T23:04:18.483 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:18.483 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.483 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.483 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:04:18.483 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:18.483 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:18.483 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:18.483 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:18.483 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:18.484 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240949+0000", 2024-04-24T23:04:18.484 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:18.484 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.484 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.484 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:04:18.484 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:18.484 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:18.484 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:18.484 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:18.484 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:18.484 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.171672+0000", 2024-04-24T23:04:18.484 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:18.484 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.484 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.484 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:04:18.485 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:18.485 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:18.485 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:18.485 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:18.485 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:18.485 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.220379+0000", 2024-04-24T23:04:18.485 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:18.485 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.485 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.485 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:04:18.485 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:18.485 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:18.485 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:18.485 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:18.485 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:18.486 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.204963+0000", 2024-04-24T23:04:18.486 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:18.486 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.486 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.486 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:04:18.486 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:18.486 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:18.486 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:18.486 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:18.486 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:18.486 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.239748+0000", 2024-04-24T23:04:18.486 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:18.486 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.486 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:18.486 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:04:18.487 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:18.487 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:18.487 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:18.487 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:18.487 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:18.487 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.154104+0000", 2024-04-24T23:04:18.487 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:18.487 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:18.487 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:04:18.487 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2024-04-24T23:04:18.487 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2024-04-24T23:04:18.487 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_primaries": [], 2024-04-24T23:04:18.487 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2024-04-24T23:04:18.487 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2024-04-24T23:04:18.487 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklist": {}, 2024-04-24T23:04:18.488 INFO:teuthology.orchestra.run.smithi032.stdout: "range_blocklist": {}, 2024-04-24T23:04:18.488 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2024-04-24T23:04:18.488 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2024-04-24T23:04:18.488 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2024-04-24T23:04:18.488 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2024-04-24T23:04:18.488 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2024-04-24T23:04:18.488 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2024-04-24T23:04:18.488 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2024-04-24T23:04:18.488 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:18.488 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:18.488 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2024-04-24T23:04:18.488 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2024-04-24T23:04:18.488 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2024-04-24T23:04:18.488 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2024-04-24T23:04:18.488 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {}, 2024-04-24T23:04:18.489 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode": { 2024-04-24T23:04:18.489 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_enabled": false, 2024-04-24T23:04:18.489 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_bucket_count": 0, 2024-04-24T23:04:18.489 INFO:teuthology.orchestra.run.smithi032.stdout: "degraded_stretch_mode": 0, 2024-04-24T23:04:18.489 INFO:teuthology.orchestra.run.smithi032.stdout: "recovering_stretch_mode": 0, 2024-04-24T23:04:18.489 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_bucket": 0 2024-04-24T23:04:18.489 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:18.489 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:04:18.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.420+0000 7efcf1dce640 1 -- 172.21.15.32:0/2399263215 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7efcd806be30 msgr2=0x7efcd806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:18.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.420+0000 7efcf1dce640 1 --2- 172.21.15.32:0/2399263215 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7efcd806be30 0x7efcd806e2f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7efce0034000 tx=0x7efce002e000 comp rx=0 tx=0).stop 2024-04-24T23:04:18.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.420+0000 7efcf1dce640 1 -- 172.21.15.32:0/2399263215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcec074210 msgr2=0x7efcec15dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:18.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.420+0000 7efcf1dce640 1 --2- 172.21.15.32:0/2399263215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcec074210 0x7efcec15dfb0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7efcdc00d5d0 tx=0x7efcdc00da90 comp rx=0 tx=0).stop 2024-04-24T23:04:18.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.420+0000 7efcf1dce640 1 -- 172.21.15.32:0/2399263215 shutdown_connections 2024-04-24T23:04:18.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.420+0000 7efcf1dce640 1 --2- 172.21.15.32:0/2399263215 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7efcd806be30 0x7efcd806e2f0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:18.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.420+0000 7efcf1dce640 1 --2- 172.21.15.32:0/2399263215 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efcec0cd190 0x7efcec0cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:18.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.420+0000 7efcf1dce640 1 --2- 172.21.15.32:0/2399263215 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcec0ae140 0x7efcec0cd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:18.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.420+0000 7efcf1dce640 1 --2- 172.21.15.32:0/2399263215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcec074210 0x7efcec15dfb0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:18.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.420+0000 7efcf1dce640 1 -- 172.21.15.32:0/2399263215 >> 172.21.15.32:0/2399263215 conn(0x7efcec0b2e60 msgr2=0x7efcec06e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:18.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.420+0000 7efcf1dce640 1 -- 172.21.15.32:0/2399263215 shutdown_connections 2024-04-24T23:04:18.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.420+0000 7efcf1dce640 1 -- 172.21.15.32:0/2399263215 wait complete. 2024-04-24T23:04:18.492 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:04:18.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.590+0000 7f33239c1640 1 Processor -- start 2024-04-24T23:04:18.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.591+0000 7f33239c1640 1 -- start start 2024-04-24T23:04:18.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.591+0000 7f33239c1640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f331c15e9b0 0x7f331c0d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:18.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.591+0000 7f33239c1640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f331c0d4210 0x7f331c06b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:18.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.591+0000 7f33239c1640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f331c0ba0e0 0x7f331c06b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:18.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.591+0000 7f33239c1640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f331c0d3170 con 0x7f331c0ba0e0 2024-04-24T23:04:18.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.591+0000 7f33239c1640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f331c0d3450 con 0x7f331c0d4210 2024-04-24T23:04:18.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.591+0000 7f33239c1640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f331c0d3730 con 0x7f331c15e9b0 2024-04-24T23:04:18.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.592+0000 7f3321f5e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f331c0ba0e0 0x7f331c06b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:18.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.592+0000 7f3321f5e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f331c0ba0e0 0x7f331c06b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57126/0 (socket says 172.21.15.32:57126) 2024-04-24T23:04:18.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.592+0000 7f3321f5e640 1 -- 172.21.15.32:0/1516637010 learned_addr learned my addr 172.21.15.32:0/1516637010 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:18.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.592+0000 7f3320f5c640 1 --2- 172.21.15.32:0/1516637010 >> v2:172.21.15.121:3300/0 conn(0x7f331c0d4210 0x7f331c06b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:18.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.592+0000 7f332175d640 1 --2- 172.21.15.32:0/1516637010 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f331c15e9b0 0x7f331c0d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:18.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.592+0000 7f3321f5e640 1 -- 172.21.15.32:0/1516637010 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f331c15e9b0 msgr2=0x7f331c0d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:18.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.592+0000 7f3321f5e640 1 --2- 172.21.15.32:0/1516637010 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f331c15e9b0 0x7f331c0d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:18.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.592+0000 7f3321f5e640 1 -- 172.21.15.32:0/1516637010 >> v2:172.21.15.121:3300/0 conn(0x7f331c0d4210 msgr2=0x7f331c06b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:18.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.592+0000 7f3321f5e640 1 --2- 172.21.15.32:0/1516637010 >> v2:172.21.15.121:3300/0 conn(0x7f331c0d4210 0x7f331c06b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:18.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.592+0000 7f3321f5e640 1 -- 172.21.15.32:0/1516637010 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f331c0d3a10 con 0x7f331c0ba0e0 2024-04-24T23:04:18.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.592+0000 7f3320f5c640 1 --2- 172.21.15.32:0/1516637010 >> v2:172.21.15.121:3300/0 conn(0x7f331c0d4210 0x7f331c06b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:04:18.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.592+0000 7f332175d640 1 --2- 172.21.15.32:0/1516637010 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f331c15e9b0 0x7f331c0d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:04:18.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.593+0000 7f3321f5e640 1 --2- 172.21.15.32:0/1516637010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f331c0ba0e0 0x7f331c06b650 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f331c06bc40 tx=0x7f331402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fbe87f1aca58ac3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:18.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.593+0000 7f331bfff640 1 -- 172.21.15.32:0/1516637010 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3314037070 con 0x7f331c0ba0e0 2024-04-24T23:04:18.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.593+0000 7f331bfff640 1 -- 172.21.15.32:0/1516637010 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3314032440 con 0x7f331c0ba0e0 2024-04-24T23:04:18.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.593+0000 7f331bfff640 1 -- 172.21.15.32:0/1516637010 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3314032810 con 0x7f331c0ba0e0 2024-04-24T23:04:18.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.593+0000 7f33239c1640 1 -- 172.21.15.32:0/1516637010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f331c0ba0e0 msgr2=0x7f331c06b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:18.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.593+0000 7f33239c1640 1 --2- 172.21.15.32:0/1516637010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f331c0ba0e0 0x7f331c06b650 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f331c06bc40 tx=0x7f331402a1e0 comp rx=0 tx=0).stop 2024-04-24T23:04:18.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.593+0000 7f33239c1640 1 -- 172.21.15.32:0/1516637010 shutdown_connections 2024-04-24T23:04:18.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.593+0000 7f33239c1640 1 --2- 172.21.15.32:0/1516637010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f331c0ba0e0 0x7f331c06b650 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:18.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.593+0000 7f33239c1640 1 --2- 172.21.15.32:0/1516637010 >> v2:172.21.15.121:3300/0 conn(0x7f331c0d4210 0x7f331c06b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:18.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.593+0000 7f33239c1640 1 --2- 172.21.15.32:0/1516637010 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f331c15e9b0 0x7f331c0d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:18.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.593+0000 7f33239c1640 1 -- 172.21.15.32:0/1516637010 >> 172.21.15.32:0/1516637010 conn(0x7f331c0d11d0 msgr2=0x7f331c0d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:18.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.593+0000 7f33239c1640 1 -- 172.21.15.32:0/1516637010 shutdown_connections 2024-04-24T23:04:18.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.593+0000 7f33239c1640 1 -- 172.21.15.32:0/1516637010 wait complete. 2024-04-24T23:04:18.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.595+0000 7f33239c1640 1 Processor -- start 2024-04-24T23:04:18.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.595+0000 7f33239c1640 1 -- start start 2024-04-24T23:04:18.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.595+0000 7f33239c1640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f331c0ba0e0 0x7f331c159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:18.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.595+0000 7f33239c1640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f331c0d4210 0x7f331c1f3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:18.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.595+0000 7f33239c1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f331c15e9b0 0x7f331c158680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:18.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.595+0000 7f33239c1640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f331c071730 con 0x7f331c15e9b0 2024-04-24T23:04:18.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.595+0000 7f33239c1640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f331c0d2400 con 0x7f331c0d4210 2024-04-24T23:04:18.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.595+0000 7f33239c1640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f331c0718b0 con 0x7f331c0ba0e0 2024-04-24T23:04:18.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.595+0000 7f3321f5e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f331c15e9b0 0x7f331c158680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:18.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.595+0000 7f3321f5e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f331c15e9b0 0x7f331c158680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57136/0 (socket says 172.21.15.32:57136) 2024-04-24T23:04:18.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.595+0000 7f3321f5e640 1 -- 172.21.15.32:0/81408123 learned_addr learned my addr 172.21.15.32:0/81408123 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:18.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.596+0000 7f332175d640 1 --2- 172.21.15.32:0/81408123 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f331c0ba0e0 0x7f331c159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:18.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.596+0000 7f3320f5c640 1 --2- 172.21.15.32:0/81408123 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f331c0d4210 0x7f331c1f3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:18.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.596+0000 7f3321f5e640 1 -- 172.21.15.32:0/81408123 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f331c0ba0e0 msgr2=0x7f331c159bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:18.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.596+0000 7f3321f5e640 1 --2- 172.21.15.32:0/81408123 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f331c0ba0e0 0x7f331c159bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:18.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.596+0000 7f3321f5e640 1 -- 172.21.15.32:0/81408123 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f331c0d4210 msgr2=0x7f331c1f3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:18.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.596+0000 7f3321f5e640 1 --2- 172.21.15.32:0/81408123 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f331c0d4210 0x7f331c1f3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:18.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.596+0000 7f3321f5e640 1 -- 172.21.15.32:0/81408123 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f331c0c8420 con 0x7f331c15e9b0 2024-04-24T23:04:18.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.596+0000 7f3321f5e640 1 --2- 172.21.15.32:0/81408123 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f331c15e9b0 0x7f331c158680 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f33140090b0 tx=0x7f33140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:18.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.596+0000 7f3319ffb640 1 -- 172.21.15.32:0/81408123 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3314037040 con 0x7f331c15e9b0 2024-04-24T23:04:18.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.596+0000 7f3319ffb640 1 -- 172.21.15.32:0/81408123 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3314042070 con 0x7f331c15e9b0 2024-04-24T23:04:18.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.596+0000 7f33239c1640 1 -- 172.21.15.32:0/81408123 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f331c0c8700 con 0x7f331c15e9b0 2024-04-24T23:04:18.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.596+0000 7f3319ffb640 1 -- 172.21.15.32:0/81408123 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3314030bd0 con 0x7f331c15e9b0 2024-04-24T23:04:18.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.596+0000 7f33239c1640 1 -- 172.21.15.32:0/81408123 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f331c0c8cc0 con 0x7f331c15e9b0 2024-04-24T23:04:18.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.597+0000 7f33239c1640 1 -- 172.21.15.32:0/81408123 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f331c0d2b10 con 0x7f331c15e9b0 2024-04-24T23:04:18.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.598+0000 7f3319ffb640 1 -- 172.21.15.32:0/81408123 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3314047460 con 0x7f331c15e9b0 2024-04-24T23:04:18.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.598+0000 7f3319ffb640 1 --2- 172.21.15.32:0/81408123 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f330806be30 0x7f330806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:18.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.598+0000 7f3319ffb640 1 -- 172.21.15.32:0/81408123 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5709+0+0 (secure 0 0 0) 0x7f33140abb30 con 0x7f331c15e9b0 2024-04-24T23:04:18.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.598+0000 7f3319ffb640 1 -- 172.21.15.32:0/81408123 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=109}) -- 0x7f33080739c0 con 0x7f331c15e9b0 2024-04-24T23:04:18.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.598+0000 7f332175d640 1 --2- 172.21.15.32:0/81408123 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f330806be30 0x7f330806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:18.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.599+0000 7f332175d640 1 --2- 172.21.15.32:0/81408123 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f330806be30 0x7f330806e2f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f330c007510 tx=0x7f330c0074a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:18.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.602+0000 7f3319ffb640 1 -- 172.21.15.32:0/81408123 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f331407a450 con 0x7f331c15e9b0 2024-04-24T23:04:18.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.820+0000 7f33239c1640 1 -- 172.21.15.32:0/81408123 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f331c068430 con 0x7f331c15e9b0 2024-04-24T23:04:18.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.821+0000 7f3319ffb640 1 -- 172.21.15.32:0/81408123 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5186 (secure 0 0 0) 0x7f3314079d50 con 0x7f331c15e9b0 2024-04-24T23:04:18.823 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:04:18.823 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":44,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6175,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/434273738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":434273738},{"type":"v1","addr":"172.21.15.121:6836","nonce":434273738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6178,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1104557785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1104557785},{"type":"v1","addr":"172.21.15.121:6837","nonce":1104557785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6888,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/3137388886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3137388886},{"type":"v1","addr":"172.21.15.32:6834","nonce":3137388886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:04:00.783579+0000","modified":"2024-04-24T23:04:03.790658+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6891},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6891":{"gid":6891,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/3327203898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":3327203898},{"type":"v1","addr":"172.21.15.32:6835","nonce":3327203898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6891,"qdb_cluster":[6891]},"id":6}]} 2024-04-24T23:04:18.823 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 44 2024-04-24T23:04:18.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.823+0000 7f33239c1640 1 -- 172.21.15.32:0/81408123 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f330806be30 msgr2=0x7f330806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:18.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.823+0000 7f33239c1640 1 --2- 172.21.15.32:0/81408123 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f330806be30 0x7f330806e2f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f330c007510 tx=0x7f330c0074a0 comp rx=0 tx=0).stop 2024-04-24T23:04:18.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.823+0000 7f33239c1640 1 -- 172.21.15.32:0/81408123 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f331c15e9b0 msgr2=0x7f331c158680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:18.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.823+0000 7f33239c1640 1 --2- 172.21.15.32:0/81408123 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f331c15e9b0 0x7f331c158680 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f33140090b0 tx=0x7f33140046c0 comp rx=0 tx=0).stop 2024-04-24T23:04:18.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.823+0000 7f33239c1640 1 -- 172.21.15.32:0/81408123 shutdown_connections 2024-04-24T23:04:18.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.823+0000 7f33239c1640 1 --2- 172.21.15.32:0/81408123 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f330806be30 0x7f330806e2f0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:18.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.823+0000 7f33239c1640 1 --2- 172.21.15.32:0/81408123 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f331c15e9b0 0x7f331c158680 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:18.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.823+0000 7f33239c1640 1 --2- 172.21.15.32:0/81408123 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f331c0d4210 0x7f331c1f3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:18.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.823+0000 7f33239c1640 1 --2- 172.21.15.32:0/81408123 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f331c0ba0e0 0x7f331c159bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:18.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.823+0000 7f33239c1640 1 -- 172.21.15.32:0/81408123 >> 172.21.15.32:0/81408123 conn(0x7f331c0d11d0 msgr2=0x7f331c0b7ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:18.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.823+0000 7f33239c1640 1 -- 172.21.15.32:0/81408123 shutdown_connections 2024-04-24T23:04:18.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:18.823+0000 7f33239c1640 1 -- 172.21.15.32:0/81408123 wait complete. 2024-04-24T23:04:18.835 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-04-24T23:04:18.955 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","whoami":0,"id":6891,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":17.506910719,"mdsmap_epoch":44,"osdmap_epoch":108,"osdmap_epoch_barrier":108,"uptime":22.405886223} 2024-04-24T23:04:18.956 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", "fs_name": "cephfs", "id": 6891, "mdsmap_epoch": 44, "osdmap_epoch": 108, "osdmap_epoch_barrier": 108, "rank_uptime": 17.506910719, "state": "up:active", "uptime": 22.405886223, "want_state": "up:active", "whoami": 0 } 2024-04-24T23:04:18.957 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-24T23:04:18.957 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:04:18.957 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:04:18.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:04:18.973 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=small_file_1 bs=1M conv=fdatasync count=0 seek=0) 2024-04-24T23:04:19.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:04:19.029 INFO:teuthology.orchestra.run.smithi032.stderr:dd: failed to open 'small_file_1': Permission denied 2024-04-24T23:04:19.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:04:19.030 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2024-04-24T23:04:19.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:04:19.086 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2024-04-24T23:04:19.163 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-24T23:04:19.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.302+0000 7fd17f730640 1 Processor -- start 2024-04-24T23:04:19.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.303+0000 7fd17f730640 1 -- start start 2024-04-24T23:04:19.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.303+0000 7fd17f730640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd17806b820 0x7fd1780d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:19.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.303+0000 7fd17f730640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd1780d4210 0x7fd1780b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:19.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.303+0000 7fd17f730640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1780d3170 con 0x7fd17815e9b0 2024-04-24T23:04:19.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.303+0000 7fd17f730640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fd1780d3450 con 0x7fd17806b820 2024-04-24T23:04:19.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.303+0000 7fd17f730640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fd1780d3730 con 0x7fd1780d4210 2024-04-24T23:04:19.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.304+0000 7fd17cccb640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd17806b820 0x7fd1780d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:19.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.304+0000 7fd17cccb640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd17806b820 0x7fd1780d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57138/0 (socket says 172.21.15.32:57138) 2024-04-24T23:04:19.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.304+0000 7fd17cccb640 1 -- 172.21.15.32:0/1124735330 learned_addr learned my addr 172.21.15.32:0/1124735330 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:19.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.304+0000 7fd17dccd640 1 --2- 172.21.15.32:0/1124735330 >> v2:172.21.15.121:3300/0 conn(0x7fd1780d4210 0x7fd1780b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:19.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.304+0000 7fd17cccb640 1 -- 172.21.15.32:0/1124735330 >> v2:172.21.15.121:3300/0 conn(0x7fd1780d4210 msgr2=0x7fd1780b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:19.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.304+0000 7fd17cccb640 1 --2- 172.21.15.32:0/1124735330 >> v2:172.21.15.121:3300/0 conn(0x7fd1780d4210 0x7fd1780b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:19.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.304+0000 7fd17cccb640 1 -- 172.21.15.32:0/1124735330 >> v1:172.21.15.121:6789/0 conn(0x7fd17815e9b0 legacy=0x7fd17806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:19.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.304+0000 7fd17cccb640 1 -- 172.21.15.32:0/1124735330 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1780d3a10 con 0x7fd17806b820 2024-04-24T23:04:19.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.305+0000 7fd17cccb640 1 --2- 172.21.15.32:0/1124735330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd17806b820 0x7fd1780d2b90 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fd1781622f0 tx=0x7fd16c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f88526f3f97f943 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:19.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.305+0000 7fd177fff640 1 -- 172.21.15.32:0/1124735330 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16c037070 con 0x7fd17806b820 2024-04-24T23:04:19.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.305+0000 7fd177fff640 1 -- 172.21.15.32:0/1124735330 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd16c032440 con 0x7fd17806b820 2024-04-24T23:04:19.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.305+0000 7fd177fff640 1 -- 172.21.15.32:0/1124735330 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16c032810 con 0x7fd17806b820 2024-04-24T23:04:19.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.305+0000 7fd17f730640 1 -- 172.21.15.32:0/1124735330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd17806b820 msgr2=0x7fd1780d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:19.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.305+0000 7fd17f730640 1 --2- 172.21.15.32:0/1124735330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd17806b820 0x7fd1780d2b90 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fd1781622f0 tx=0x7fd16c02a1a0 comp rx=0 tx=0).stop 2024-04-24T23:04:19.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.305+0000 7fd17f730640 1 -- 172.21.15.32:0/1124735330 shutdown_connections 2024-04-24T23:04:19.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.305+0000 7fd17f730640 1 --2- 172.21.15.32:0/1124735330 >> v2:172.21.15.121:3300/0 conn(0x7fd1780d4210 0x7fd1780b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:19.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.305+0000 7fd17f730640 1 --2- 172.21.15.32:0/1124735330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd17806b820 0x7fd1780d2b90 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:19.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.305+0000 7fd17f730640 1 -- 172.21.15.32:0/1124735330 >> 172.21.15.32:0/1124735330 conn(0x7fd1780d11d0 msgr2=0x7fd1780d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:19.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.305+0000 7fd17f730640 1 -- 172.21.15.32:0/1124735330 shutdown_connections 2024-04-24T23:04:19.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.305+0000 7fd17f730640 1 -- 172.21.15.32:0/1124735330 wait complete. 2024-04-24T23:04:19.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.307+0000 7fd17f730640 1 Processor -- start 2024-04-24T23:04:19.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.307+0000 7fd17f730640 1 -- start start 2024-04-24T23:04:19.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.307+0000 7fd17f730640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd17806b820 0x7fd178159490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:19.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.307+0000 7fd17f730640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1780d4210 0x7fd178160390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:19.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.307+0000 7fd17f730640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd17815e9b0 0x7fd1780ca560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:19.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.307+0000 7fd17f730640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd1780ba0e0 con 0x7fd1780d4210 2024-04-24T23:04:19.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.307+0000 7fd17f730640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd1780d2400 con 0x7fd17806b820 2024-04-24T23:04:19.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.307+0000 7fd17f730640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd1780ba260 con 0x7fd17815e9b0 2024-04-24T23:04:19.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.307+0000 7fd17cccb640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1780d4210 0x7fd178160390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:19.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.307+0000 7fd17cccb640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1780d4210 0x7fd178160390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57144/0 (socket says 172.21.15.32:57144) 2024-04-24T23:04:19.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.307+0000 7fd17cccb640 1 -- 172.21.15.32:0/3930566197 learned_addr learned my addr 172.21.15.32:0/3930566197 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:19.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.308+0000 7fd17cccb640 1 -- 172.21.15.32:0/3930566197 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd17815e9b0 msgr2=0x7fd1780ca560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:19.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.308+0000 7fd17cccb640 1 --2- 172.21.15.32:0/3930566197 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd17815e9b0 0x7fd1780ca560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:19.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.308+0000 7fd17cccb640 1 -- 172.21.15.32:0/3930566197 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd17806b820 msgr2=0x7fd178159490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:19.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.308+0000 7fd17cccb640 1 --2- 172.21.15.32:0/3930566197 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd17806b820 0x7fd178159490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:19.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.308+0000 7fd17cccb640 1 -- 172.21.15.32:0/3930566197 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd178160b30 con 0x7fd1780d4210 2024-04-24T23:04:19.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.308+0000 7fd17cccb640 1 --2- 172.21.15.32:0/3930566197 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1780d4210 0x7fd178160390 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fd16c000c00 tx=0x7fd16c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:19.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.308+0000 7fd175ffb640 1 -- 172.21.15.32:0/3930566197 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16c041020 con 0x7fd1780d4210 2024-04-24T23:04:19.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.308+0000 7fd175ffb640 1 -- 172.21.15.32:0/3930566197 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd16c0306c0 con 0x7fd1780d4210 2024-04-24T23:04:19.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.308+0000 7fd17f730640 1 -- 172.21.15.32:0/3930566197 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd178160e10 con 0x7fd1780d4210 2024-04-24T23:04:19.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.308+0000 7fd175ffb640 1 -- 172.21.15.32:0/3930566197 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16c037040 con 0x7fd1780d4210 2024-04-24T23:04:19.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.308+0000 7fd17f730640 1 -- 172.21.15.32:0/3930566197 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1781613d0 con 0x7fd1780d4210 2024-04-24T23:04:19.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.309+0000 7fd17f730640 1 -- 172.21.15.32:0/3930566197 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd178068430 con 0x7fd1780d4210 2024-04-24T23:04:19.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.310+0000 7fd175ffb640 1 -- 172.21.15.32:0/3930566197 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd16c044020 con 0x7fd1780d4210 2024-04-24T23:04:19.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.310+0000 7fd175ffb640 1 --2- 172.21.15.32:0/3930566197 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd16406be30 0x7fd16406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:19.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.310+0000 7fd175ffb640 1 -- 172.21.15.32:0/3930566197 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5709+0+0 (secure 0 0 0) 0x7fd16c0ab6f0 con 0x7fd1780d4210 2024-04-24T23:04:19.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.310+0000 7fd175ffb640 1 -- 172.21.15.32:0/3930566197 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=109}) -- 0x7fd164073960 con 0x7fd1780d4210 2024-04-24T23:04:19.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.310+0000 7fd17d4cc640 1 --2- 172.21.15.32:0/3930566197 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd16406be30 0x7fd16406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:19.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.311+0000 7fd17d4cc640 1 --2- 172.21.15.32:0/3930566197 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd16406be30 0x7fd16406e2f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd170006fd0 tx=0x7fd170006bc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:19.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.314+0000 7fd175ffb640 1 -- 172.21.15.32:0/3930566197 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd16c04b030 con 0x7fd1780d4210 2024-04-24T23:04:19.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.501+0000 7fd17f730640 1 -- 172.21.15.32:0/3930566197 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7fd17815c9e0 con 0x7fd1780d4210 2024-04-24T23:04:19.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.502+0000 7fd175ffb640 1 -- 172.21.15.32:0/3930566197 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v108) ==== 81+0+34360 (secure 0 0 0) 0x7fd16c07a0a0 con 0x7fd1780d4210 2024-04-24T23:04:19.504 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:04:19.504 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:04:19.504 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 108, 2024-04-24T23:04:19.504 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", 2024-04-24T23:04:19.504 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2024-04-24T22:58:46.964851+0000", 2024-04-24T23:04:19.504 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2024-04-24T23:04:17.477163+0000", 2024-04-24T23:04:19.504 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2024-04-24T22:58:52.048787+0000", 2024-04-24T23:04:19.504 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2024-04-24T22:58:49.045609+0000", 2024-04-24T23:04:19.504 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-24T23:04:19.505 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2024-04-24T23:04:19.505 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2024-04-24T23:04:19.505 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2024-04-24T23:04:19.505 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2024-04-24T23:04:19.505 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2024-04-24T23:04:19.505 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2024-04-24T23:04:19.505 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:04:19.505 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 4, 2024-04-24T23:04:19.505 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.69999998807907104, 2024-04-24T23:04:19.505 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-24T23:04:19.505 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-24T23:04:19.505 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2024-04-24T23:04:19.505 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 13, 2024-04-24T23:04:19.505 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2024-04-24T23:04:19.506 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "luminous", 2024-04-24T23:04:19.506 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "jewel", 2024-04-24T23:04:19.506 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "squid", 2024-04-24T23:04:19.506 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_crimson": false, 2024-04-24T23:04:19.506 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2024-04-24T23:04:19.506 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.506 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2024-04-24T23:04:19.506 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": ".mgr", 2024-04-24T23:04:19.506 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T22:58:53.171408+0000", 2024-04-24T23:04:19.506 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 3, 2024-04-24T23:04:19.506 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,full", 2024-04-24T23:04:19.506 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:04:19.506 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:04:19.506 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:04:19.507 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:04:19.507 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:04:19.507 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:04:19.507 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:04:19.507 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:04:19.507 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:04:19.507 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:04:19.507 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2024-04-24T23:04:19.507 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2024-04-24T23:04:19.507 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2024-04-24T23:04:19.507 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2024-04-24T23:04:19.507 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2024-04-24T23:04:19.507 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:04:19.507 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:04:19.508 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:04:19.508 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:04:19.508 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:04:19.508 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:04:19.508 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:04:19.508 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.508 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "105", 2024-04-24T23:04:19.508 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:04:19.508 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:04:19.508 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:04:19.508 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:04:19.508 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:04:19.508 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:04:19.508 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:04:19.509 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:04:19.509 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:04:19.509 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:04:19.509 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:04:19.509 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:04:19.509 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:04:19.509 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:04:19.509 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:04:19.509 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:04:19.509 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:04:19.509 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:04:19.509 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:04:19.509 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:04:19.509 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:04:19.509 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:04:19.510 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:04:19.510 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:04:19.510 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:04:19.510 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:04:19.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.510 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:04:19.510 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:04:19.510 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:04:19.510 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:04:19.510 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:04:19.510 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:04:19.510 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:04:19.510 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:04:19.511 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:04:19.511 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:04:19.511 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:04:19.511 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:04:19.511 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_max": 32, 2024-04-24T23:04:19.511 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2024-04-24T23:04:19.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.511 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:04:19.511 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": {} 2024-04-24T23:04:19.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.511 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:04:19.511 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:04:19.511 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 8, 2024-04-24T23:04:19.511 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 8, 2024-04-24T23:04:19.511 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.25, 2024-04-24T23:04:19.512 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-04-24T23:04:19.512 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-04-24T23:04:19.512 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:04:19.512 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:04:19.512 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:04:19.512 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:19.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.512 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 12, 2024-04-24T23:04:19.512 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2024-04-24T23:04:19.512 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:03:59.079157+0000", 2024-04-24T23:04:19.512 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 3, 2024-04-24T23:04:19.512 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,full", 2024-04-24T23:04:19.512 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:04:19.512 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:04:19.513 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:04:19.513 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:04:19.513 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:04:19.513 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:04:19.513 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:04:19.513 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:04:19.513 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:04:19.513 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:04:19.513 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 32, 2024-04-24T23:04:19.513 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 32, 2024-04-24T23:04:19.513 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 32, 2024-04-24T23:04:19.513 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 32, 2024-04-24T23:04:19.513 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 32, 2024-04-24T23:04:19.513 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:04:19.514 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:04:19.514 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:04:19.514 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:04:19.514 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:04:19.514 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:04:19.514 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:04:19.514 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.514 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "105", 2024-04-24T23:04:19.514 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:04:19.514 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:04:19.514 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:04:19.514 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:04:19.514 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:04:19.515 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:04:19.515 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:04:19.515 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:04:19.515 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:04:19.515 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:04:19.515 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:04:19.515 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:04:19.515 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:04:19.515 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:04:19.515 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:04:19.515 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:04:19.515 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:04:19.515 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:04:19.515 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:04:19.515 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:04:19.516 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:04:19.516 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:04:19.516 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:04:19.516 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:04:19.516 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:04:19.516 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:04:19.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.516 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:04:19.516 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:04:19.516 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:04:19.516 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:04:19.516 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:04:19.516 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:04:19.516 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:04:19.517 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:04:19.517 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:04:19.517 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:04:19.517 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:04:19.517 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:04:19.517 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2024-04-24T23:04:19.517 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2024-04-24T23:04:19.517 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2024-04-24T23:04:19.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.517 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:04:19.517 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:04:19.517 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2024-04-24T23:04:19.517 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:19.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.518 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:04:19.518 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:04:19.518 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.75, 2024-04-24T23:04:19.518 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.75, 2024-04-24T23:04:19.518 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:04:19.518 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.75, 2024-04-24T23:04:19.518 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.75, 2024-04-24T23:04:19.518 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:04:19.518 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:04:19.518 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:04:19.518 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:19.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.518 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.518 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 13, 2024-04-24T23:04:19.518 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2024-04-24T23:04:19.519 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:03:59.765179+0000", 2024-04-24T23:04:19.519 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 3, 2024-04-24T23:04:19.519 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,full", 2024-04-24T23:04:19.519 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:04:19.519 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:04:19.519 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:04:19.519 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:04:19.519 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:04:19.519 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:04:19.519 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:04:19.519 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:04:19.519 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:04:19.519 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:04:19.520 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 64, 2024-04-24T23:04:19.520 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 64, 2024-04-24T23:04:19.520 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 64, 2024-04-24T23:04:19.520 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 64, 2024-04-24T23:04:19.520 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 64, 2024-04-24T23:04:19.520 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:04:19.520 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:04:19.520 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:04:19.520 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:04:19.520 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:04:19.520 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:04:19.520 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:04:19.520 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.520 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "105", 2024-04-24T23:04:19.520 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:04:19.521 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:04:19.521 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:04:19.521 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:04:19.521 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:04:19.521 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:04:19.521 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:04:19.521 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:04:19.521 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:04:19.521 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:04:19.521 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:04:19.521 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:04:19.521 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:04:19.521 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:04:19.521 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:04:19.522 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:04:19.522 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:04:19.522 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:04:19.522 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:04:19.522 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:04:19.522 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:04:19.522 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:04:19.522 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:04:19.522 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:04:19.522 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:04:19.522 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:04:19.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.522 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:04:19.523 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:04:19.523 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:04:19.523 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:04:19.523 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:04:19.523 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:04:19.523 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:04:19.523 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:04:19.523 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:04:19.523 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:04:19.523 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:04:19.523 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:04:19.523 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 64, 2024-04-24T23:04:19.523 INFO:teuthology.orchestra.run.smithi032.stdout: "target_size_ratio": 0.89999997615814209 2024-04-24T23:04:19.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.523 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:04:19.524 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:04:19.524 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2024-04-24T23:04:19.524 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:19.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.524 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:04:19.524 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:04:19.524 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.3799999952316284, 2024-04-24T23:04:19.524 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.3799999952316284, 2024-04-24T23:04:19.524 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:04:19.524 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.3799999952316284, 2024-04-24T23:04:19.524 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.3799999952316284, 2024-04-24T23:04:19.524 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:04:19.524 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:04:19.524 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:04:19.525 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:19.525 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:19.525 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:04:19.525 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-04-24T23:04:19.525 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.525 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:04:19.525 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:19.525 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:19.525 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:19.525 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:19.525 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:19.525 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:19.525 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:19.525 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:19.525 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:19.526 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:19.526 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:19.526 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:19.526 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:19.526 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.526 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:19.526 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2024-04-24T23:04:19.526 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:19.526 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.526 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.526 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:19.526 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2024-04-24T23:04:19.526 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:19.527 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:19.527 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:19.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.527 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:19.527 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:19.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.527 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:19.527 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2024-04-24T23:04:19.527 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:19.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.527 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:19.527 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2024-04-24T23:04:19.527 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:19.528 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:19.528 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:19.528 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.528 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:19.528 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:19.528 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.528 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:19.528 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2024-04-24T23:04:19.528 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:19.528 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.528 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.528 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:19.528 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2024-04-24T23:04:19.528 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:19.529 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:19.529 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:19.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.529 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:19.529 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:19.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.529 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:19.529 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2024-04-24T23:04:19.529 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:19.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.529 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:19.529 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2024-04-24T23:04:19.529 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:19.529 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:19.530 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:19.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.530 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/1479395426", 2024-04-24T23:04:19.530 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/1479395426", 2024-04-24T23:04:19.530 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/1479395426", 2024-04-24T23:04:19.530 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/1479395426", 2024-04-24T23:04:19.530 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:19.530 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:19.530 INFO:teuthology.orchestra.run.smithi032.stdout: "full", 2024-04-24T23:04:19.530 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:19.530 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:19.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.531 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:04:19.531 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:19.531 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:19.531 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:19.531 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:19.531 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:19.531 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:19.531 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:19.531 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:19.531 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:19.531 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:19.531 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:19.531 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:19.531 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:19.532 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.532 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:19.532 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2024-04-24T23:04:19.532 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:19.532 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.532 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.532 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:19.532 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2024-04-24T23:04:19.532 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:19.532 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:19.532 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:19.532 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.532 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:19.532 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:19.533 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.533 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:19.533 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2024-04-24T23:04:19.533 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:19.533 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.533 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.533 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:19.533 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2024-04-24T23:04:19.533 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:19.533 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:19.533 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:19.533 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.533 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:19.533 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:19.534 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.534 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:19.534 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2024-04-24T23:04:19.534 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:19.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.534 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.534 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:19.534 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2024-04-24T23:04:19.534 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:19.534 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:19.534 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:19.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.534 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:19.534 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:19.535 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.535 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:19.535 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2024-04-24T23:04:19.535 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:19.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.535 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.535 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:19.535 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2024-04-24T23:04:19.535 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:19.535 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:19.535 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:19.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.535 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/1340633718", 2024-04-24T23:04:19.535 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/1340633718", 2024-04-24T23:04:19.535 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/1340633718", 2024-04-24T23:04:19.536 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/1340633718", 2024-04-24T23:04:19.536 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:19.536 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull", 2024-04-24T23:04:19.536 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:19.536 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:19.536 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:19.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.536 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.536 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:04:19.536 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:19.536 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:19.536 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:19.537 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:19.537 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:19.537 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:19.537 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:19.537 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:19.537 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:19.537 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:19.537 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:19.537 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:19.537 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:19.537 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.537 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:19.537 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2024-04-24T23:04:19.537 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:19.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.538 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:19.538 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2024-04-24T23:04:19.538 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:19.538 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:19.538 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:19.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.539 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:19.539 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:19.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.539 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:19.539 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2024-04-24T23:04:19.539 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:19.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.539 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:19.540 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2024-04-24T23:04:19.540 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:19.540 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:19.540 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:19.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.540 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:19.540 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:19.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.540 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:19.540 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2024-04-24T23:04:19.540 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:19.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.541 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:19.541 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2024-04-24T23:04:19.541 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:19.541 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:19.541 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:19.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.541 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:19.541 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:19.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.541 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:19.541 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2024-04-24T23:04:19.541 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:19.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.541 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:19.542 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2024-04-24T23:04:19.542 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:19.542 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:19.542 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:19.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.542 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/3405899334", 2024-04-24T23:04:19.542 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6820/3405899334", 2024-04-24T23:04:19.542 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6828/3405899334", 2024-04-24T23:04:19.542 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6824/3405899334", 2024-04-24T23:04:19.542 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:19.542 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull", 2024-04-24T23:04:19.543 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:19.543 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:19.543 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:19.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.543 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:04:19.543 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:19.543 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:19.543 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:19.543 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:19.543 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:19.543 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:19.543 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:19.543 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:19.543 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:19.544 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:19.544 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:19.544 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:19.544 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:19.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.544 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:19.544 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2024-04-24T23:04:19.544 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:19.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.544 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:19.544 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2024-04-24T23:04:19.544 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:19.544 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:19.545 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:19.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.545 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:19.545 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:19.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.545 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:19.545 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2024-04-24T23:04:19.545 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:19.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.545 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:19.545 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2024-04-24T23:04:19.545 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:19.546 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:19.546 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:19.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.546 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:19.546 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:19.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.546 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:19.546 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2024-04-24T23:04:19.546 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:19.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.547 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:19.547 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2024-04-24T23:04:19.547 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:19.547 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:19.547 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:19.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.547 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:19.547 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:19.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.548 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:19.548 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2024-04-24T23:04:19.548 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:19.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.548 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:19.548 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2024-04-24T23:04:19.548 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:19.548 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:19.548 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:19.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.549 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6821/3381057190", 2024-04-24T23:04:19.549 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6825/3381057190", 2024-04-24T23:04:19.549 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/3381057190", 2024-04-24T23:04:19.549 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/3381057190", 2024-04-24T23:04:19.549 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:19.549 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:19.549 INFO:teuthology.orchestra.run.smithi032.stdout: "full", 2024-04-24T23:04:19.549 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:19.549 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:19.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.549 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:04:19.549 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:19.550 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:19.550 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:19.550 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:19.550 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:19.550 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:19.550 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:19.550 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:19.550 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:19.550 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:19.550 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:19.550 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:19.550 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:19.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.551 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:19.551 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6800", 2024-04-24T23:04:19.551 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:19.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.551 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:19.551 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6801", 2024-04-24T23:04:19.551 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:19.551 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:19.551 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:19.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.551 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:19.551 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:19.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.552 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:19.552 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6802", 2024-04-24T23:04:19.552 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:19.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.552 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:19.552 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6803", 2024-04-24T23:04:19.552 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:19.552 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:19.552 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:19.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.552 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:19.552 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:19.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.553 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:19.553 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6806", 2024-04-24T23:04:19.553 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:19.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.553 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:19.553 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6807", 2024-04-24T23:04:19.553 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:19.553 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:19.553 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:19.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.553 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:19.554 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:19.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.554 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:19.554 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6804", 2024-04-24T23:04:19.554 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:19.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.554 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:19.554 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6805", 2024-04-24T23:04:19.554 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:19.554 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:19.554 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:19.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.554 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6801/4010553979", 2024-04-24T23:04:19.555 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6803/4010553979", 2024-04-24T23:04:19.555 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6807/4010553979", 2024-04-24T23:04:19.555 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6805/4010553979", 2024-04-24T23:04:19.555 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:19.555 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:19.555 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:19.555 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:19.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.555 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:04:19.555 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:19.555 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:19.555 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:19.555 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:19.555 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:19.556 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:19.556 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:19.556 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:19.556 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:19.556 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:19.556 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:19.556 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:19.556 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:19.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.556 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:19.557 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6808", 2024-04-24T23:04:19.557 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:19.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.557 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:19.557 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6809", 2024-04-24T23:04:19.557 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:19.557 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:19.557 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:19.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.557 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:19.557 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:19.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.558 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:19.558 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6810", 2024-04-24T23:04:19.558 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:19.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.558 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:19.558 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6811", 2024-04-24T23:04:19.558 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:19.558 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:19.559 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:19.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.559 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:19.559 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:19.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.559 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:19.559 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6814", 2024-04-24T23:04:19.559 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:19.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.559 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:19.559 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6815", 2024-04-24T23:04:19.559 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:19.560 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:19.560 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:19.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.560 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:19.560 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:19.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.560 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:19.560 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6812", 2024-04-24T23:04:19.560 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:19.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.560 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:19.560 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6813", 2024-04-24T23:04:19.560 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:19.561 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:19.561 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:19.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.561 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6809/3794084091", 2024-04-24T23:04:19.561 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6811/3794084091", 2024-04-24T23:04:19.561 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6815/3794084091", 2024-04-24T23:04:19.561 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6813/3794084091", 2024-04-24T23:04:19.561 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:19.561 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:19.561 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:19.561 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:19.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.561 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:04:19.562 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:19.562 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:19.562 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:19.562 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:19.562 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:19.562 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:19.562 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:19.562 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:19.562 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:19.562 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:19.562 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:19.562 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:19.562 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:19.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.563 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:19.563 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6823", 2024-04-24T23:04:19.563 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:19.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.566 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:19.566 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6825", 2024-04-24T23:04:19.567 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:19.567 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:19.567 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:19.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.567 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:19.567 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:19.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.567 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:19.567 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6826", 2024-04-24T23:04:19.567 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:19.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.568 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:19.568 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6827", 2024-04-24T23:04:19.568 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:19.568 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:19.568 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:19.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.568 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:19.568 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:19.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.568 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:19.568 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6830", 2024-04-24T23:04:19.569 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:19.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.569 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:19.569 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6831", 2024-04-24T23:04:19.569 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:19.569 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:19.569 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:19.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.569 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:19.570 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:19.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.570 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:19.570 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6828", 2024-04-24T23:04:19.570 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:19.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.570 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:19.570 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6829", 2024-04-24T23:04:19.570 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:19.570 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:19.570 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:19.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.571 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6825/1348640258", 2024-04-24T23:04:19.571 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6827/1348640258", 2024-04-24T23:04:19.571 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6831/1348640258", 2024-04-24T23:04:19.571 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6829/1348640258", 2024-04-24T23:04:19.571 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:19.571 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:19.571 INFO:teuthology.orchestra.run.smithi032.stdout: "full", 2024-04-24T23:04:19.571 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:19.571 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:19.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.572 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:04:19.572 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:19.572 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:19.572 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:19.572 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:19.572 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:19.572 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:19.572 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:19.572 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:19.572 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:19.573 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:19.573 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:19.573 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:19.573 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:19.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.573 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:19.573 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6816", 2024-04-24T23:04:19.573 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:19.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.573 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:19.574 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6817", 2024-04-24T23:04:19.574 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:19.574 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:19.574 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:19.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.574 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:19.574 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:19.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.574 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:19.574 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6818", 2024-04-24T23:04:19.574 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:19.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.574 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:19.575 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6819", 2024-04-24T23:04:19.575 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:19.575 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:19.575 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:19.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.575 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:19.575 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:19.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.576 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:19.576 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6822", 2024-04-24T23:04:19.576 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:19.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.576 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:19.576 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6824", 2024-04-24T23:04:19.576 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:19.576 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:19.576 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:19.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.577 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:19.577 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:19.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.577 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:19.577 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6820", 2024-04-24T23:04:19.577 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:19.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.577 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:19.577 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6821", 2024-04-24T23:04:19.577 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:19.578 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:19.578 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:19.578 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.578 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6817/3250005122", 2024-04-24T23:04:19.578 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6819/3250005122", 2024-04-24T23:04:19.578 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6824/3250005122", 2024-04-24T23:04:19.578 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6821/3250005122", 2024-04-24T23:04:19.578 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:19.579 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:19.579 INFO:teuthology.orchestra.run.smithi032.stdout: "full", 2024-04-24T23:04:19.579 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:19.579 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:19.579 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:19.579 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:04:19.579 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2024-04-24T23:04:19.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.579 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:04:19.579 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:19.580 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:19.580 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:19.580 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:19.580 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:19.580 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.156184+0000", 2024-04-24T23:04:19.580 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:19.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.580 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:04:19.580 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:19.580 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:19.581 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:19.581 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:19.581 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:19.581 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240774+0000", 2024-04-24T23:04:19.581 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:19.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.581 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:04:19.581 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:19.581 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:19.581 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:19.581 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:19.581 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:19.582 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240949+0000", 2024-04-24T23:04:19.582 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:19.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.582 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.582 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:04:19.582 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:19.582 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:19.582 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:19.582 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:19.582 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:19.582 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.171672+0000", 2024-04-24T23:04:19.582 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:19.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.582 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.583 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:04:19.583 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:19.583 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:19.583 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:19.583 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:19.583 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:19.583 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.220379+0000", 2024-04-24T23:04:19.583 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:19.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.583 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:04:19.583 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:19.584 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:19.584 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:19.584 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:19.584 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:19.584 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.204963+0000", 2024-04-24T23:04:19.584 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:19.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.584 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:04:19.584 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:19.584 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:19.585 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:19.585 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:19.585 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:19.585 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.239748+0000", 2024-04-24T23:04:19.585 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:19.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:19.585 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:04:19.585 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:19.585 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:19.585 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:19.585 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:19.585 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:19.585 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.154104+0000", 2024-04-24T23:04:19.586 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:19.586 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:19.586 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:04:19.586 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2024-04-24T23:04:19.586 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2024-04-24T23:04:19.586 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_primaries": [], 2024-04-24T23:04:19.586 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2024-04-24T23:04:19.586 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2024-04-24T23:04:19.586 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklist": {}, 2024-04-24T23:04:19.586 INFO:teuthology.orchestra.run.smithi032.stdout: "range_blocklist": {}, 2024-04-24T23:04:19.586 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2024-04-24T23:04:19.586 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2024-04-24T23:04:19.587 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2024-04-24T23:04:19.587 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2024-04-24T23:04:19.587 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2024-04-24T23:04:19.587 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2024-04-24T23:04:19.587 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2024-04-24T23:04:19.587 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:19.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:19.587 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2024-04-24T23:04:19.587 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2024-04-24T23:04:19.587 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2024-04-24T23:04:19.587 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2024-04-24T23:04:19.587 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {}, 2024-04-24T23:04:19.587 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode": { 2024-04-24T23:04:19.587 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_enabled": false, 2024-04-24T23:04:19.587 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_bucket_count": 0, 2024-04-24T23:04:19.588 INFO:teuthology.orchestra.run.smithi032.stdout: "degraded_stretch_mode": 0, 2024-04-24T23:04:19.588 INFO:teuthology.orchestra.run.smithi032.stdout: "recovering_stretch_mode": 0, 2024-04-24T23:04:19.588 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_bucket": 0 2024-04-24T23:04:19.588 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:19.588 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:04:19.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.504+0000 7fd17f730640 1 -- 172.21.15.32:0/3930566197 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd16406be30 msgr2=0x7fd16406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:19.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.504+0000 7fd17f730640 1 --2- 172.21.15.32:0/3930566197 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd16406be30 0x7fd16406e2f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd170006fd0 tx=0x7fd170006bc0 comp rx=0 tx=0).stop 2024-04-24T23:04:19.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.504+0000 7fd17f730640 1 -- 172.21.15.32:0/3930566197 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1780d4210 msgr2=0x7fd178160390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:19.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.504+0000 7fd17f730640 1 --2- 172.21.15.32:0/3930566197 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1780d4210 0x7fd178160390 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fd16c000c00 tx=0x7fd16c0046c0 comp rx=0 tx=0).stop 2024-04-24T23:04:19.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.504+0000 7fd17f730640 1 -- 172.21.15.32:0/3930566197 shutdown_connections 2024-04-24T23:04:19.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.504+0000 7fd17f730640 1 --2- 172.21.15.32:0/3930566197 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd16406be30 0x7fd16406e2f0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:19.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.504+0000 7fd17f730640 1 --2- 172.21.15.32:0/3930566197 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1780d4210 0x7fd178160390 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:19.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.504+0000 7fd17f730640 1 --2- 172.21.15.32:0/3930566197 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd17815e9b0 0x7fd1780ca560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:19.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.504+0000 7fd17f730640 1 --2- 172.21.15.32:0/3930566197 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd17806b820 0x7fd178159490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:19.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.504+0000 7fd17f730640 1 -- 172.21.15.32:0/3930566197 >> 172.21.15.32:0/3930566197 conn(0x7fd1780d11d0 msgr2=0x7fd178155c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:19.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.504+0000 7fd17f730640 1 -- 172.21.15.32:0/3930566197 shutdown_connections 2024-04-24T23:04:19.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:19.504+0000 7fd17f730640 1 -- 172.21.15.32:0/3930566197 wait complete. 2024-04-24T23:04:19.591 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-04-24T23:04:21.796 INFO:tasks.ceph.mon.a.smithi032.stderr:2024-04-24T23:04:21.794+0000 7f3b831cb640 -1 log_channel(cluster) log [ERR] : Health check update: 4 full osd(s) (OSD_FULL) 2024-04-24T23:04:24.592 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-24T23:04:24.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.704+0000 7f95256a5640 1 Processor -- start 2024-04-24T23:04:24.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.705+0000 7f95256a5640 1 -- start start 2024-04-24T23:04:24.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.705+0000 7f95256a5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f952006b820 0x7f95200d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:24.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.705+0000 7f95256a5640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f95200d4210 0x7f95200b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:24.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.705+0000 7f95256a5640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95200d3170 con 0x7f952015e9b0 2024-04-24T23:04:24.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.705+0000 7f95256a5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f95200d3450 con 0x7f952006b820 2024-04-24T23:04:24.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.705+0000 7f95256a5640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f95200d3730 con 0x7f95200d4210 2024-04-24T23:04:24.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.705+0000 7f951effd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f952015e9b0 0x7f952006b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40006/0 (socket says 172.21.15.32:40006) 2024-04-24T23:04:24.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.705+0000 7f951effd640 1 -- 172.21.15.32:0/2957439078 learned_addr learned my addr 172.21.15.32:0/2957439078 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:04:24.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.705+0000 7f951e7fc640 1 --2- 172.21.15.32:0/2957439078 >> v2:172.21.15.32:3300/0 conn(0x7f952006b820 0x7f95200d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:24.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.706+0000 7f951f7fe640 1 --2- 172.21.15.32:0/2957439078 >> v2:172.21.15.121:3300/0 conn(0x7f95200d4210 0x7f95200b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:24.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.706+0000 7f951e7fc640 1 -- 172.21.15.32:0/2957439078 >> v2:172.21.15.121:3300/0 conn(0x7f95200d4210 msgr2=0x7f95200b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:24.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.706+0000 7f951e7fc640 1 --2- 172.21.15.32:0/2957439078 >> v2:172.21.15.121:3300/0 conn(0x7f95200d4210 0x7f95200b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:24.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.706+0000 7f951e7fc640 1 -- 172.21.15.32:0/2957439078 >> v1:172.21.15.32:6789/0 conn(0x7f952015e9b0 legacy=0x7f952006b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:24.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.706+0000 7f951dffb640 1 -- 172.21.15.32:0/2957439078 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 320425609 0 0) 0x7f95200d3170 con 0x7f952015e9b0 2024-04-24T23:04:24.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.706+0000 7f951e7fc640 1 -- 172.21.15.32:0/2957439078 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95200d3a10 con 0x7f952006b820 2024-04-24T23:04:24.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.706+0000 7f951f7fe640 1 --2- 172.21.15.32:0/2957439078 >> v2:172.21.15.121:3300/0 conn(0x7f95200d4210 0x7f95200b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:04:24.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.707+0000 7f951e7fc640 1 --2- 172.21.15.32:0/2957439078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f952006b820 0x7f95200d2b90 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f95201622f0 tx=0x7f950802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d382503252b86f8b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:24.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.707+0000 7f951dffb640 1 -- 172.21.15.32:0/2957439078 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9508037070 con 0x7f952006b820 2024-04-24T23:04:24.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.707+0000 7f951dffb640 1 -- 172.21.15.32:0/2957439078 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9508032440 con 0x7f952006b820 2024-04-24T23:04:24.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.707+0000 7f951dffb640 1 -- 172.21.15.32:0/2957439078 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f950802ae60 con 0x7f952006b820 2024-04-24T23:04:24.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.707+0000 7f95256a5640 1 -- 172.21.15.32:0/2957439078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f952006b820 msgr2=0x7f95200d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:24.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.707+0000 7f95256a5640 1 --2- 172.21.15.32:0/2957439078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f952006b820 0x7f95200d2b90 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f95201622f0 tx=0x7f950802a110 comp rx=0 tx=0).stop 2024-04-24T23:04:24.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.707+0000 7f95256a5640 1 -- 172.21.15.32:0/2957439078 shutdown_connections 2024-04-24T23:04:24.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.707+0000 7f95256a5640 1 --2- 172.21.15.32:0/2957439078 >> v2:172.21.15.121:3300/0 conn(0x7f95200d4210 0x7f95200b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:24.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.707+0000 7f95256a5640 1 --2- 172.21.15.32:0/2957439078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f952006b820 0x7f95200d2b90 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:24.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.707+0000 7f95256a5640 1 -- 172.21.15.32:0/2957439078 >> 172.21.15.32:0/2957439078 conn(0x7f95200d11d0 msgr2=0x7f95200d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:24.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.707+0000 7f95256a5640 1 -- 172.21.15.32:0/2957439078 shutdown_connections 2024-04-24T23:04:24.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.707+0000 7f95256a5640 1 -- 172.21.15.32:0/2957439078 wait complete. 2024-04-24T23:04:24.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.709+0000 7f95256a5640 1 Processor -- start 2024-04-24T23:04:24.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.709+0000 7f95256a5640 1 -- start start 2024-04-24T23:04:24.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.709+0000 7f95256a5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f952006b820 0x7f95200cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:24.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.709+0000 7f95256a5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f95200d4210 0x7f9520154da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:24.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.709+0000 7f95256a5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f952015e9b0 0x7f952014a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:24.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.710+0000 7f95256a5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f95200ba0e0 con 0x7f952015e9b0 2024-04-24T23:04:24.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.710+0000 7f95256a5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f95200d2400 con 0x7f952006b820 2024-04-24T23:04:24.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.710+0000 7f95256a5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f95200ba260 con 0x7f95200d4210 2024-04-24T23:04:24.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.710+0000 7f951f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f952015e9b0 0x7f952014a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:24.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.710+0000 7f951f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f952015e9b0 0x7f952014a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58366/0 (socket says 172.21.15.32:58366) 2024-04-24T23:04:24.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.710+0000 7f951f7fe640 1 -- 172.21.15.32:0/1869900064 learned_addr learned my addr 172.21.15.32:0/1869900064 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:24.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.710+0000 7f951effd640 1 --2- 172.21.15.32:0/1869900064 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f952006b820 0x7f95200cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:24.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.710+0000 7f951e7fc640 1 --2- 172.21.15.32:0/1869900064 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f95200d4210 0x7f9520154da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:24.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.710+0000 7f951f7fe640 1 -- 172.21.15.32:0/1869900064 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f95200d4210 msgr2=0x7f9520154da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:24.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.710+0000 7f951f7fe640 1 --2- 172.21.15.32:0/1869900064 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f95200d4210 0x7f9520154da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:24.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.710+0000 7f951f7fe640 1 -- 172.21.15.32:0/1869900064 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f952006b820 msgr2=0x7f95200cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:24.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.710+0000 7f951f7fe640 1 --2- 172.21.15.32:0/1869900064 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f952006b820 0x7f95200cc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:24.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.710+0000 7f951f7fe640 1 -- 172.21.15.32:0/1869900064 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95200cde40 con 0x7f952015e9b0 2024-04-24T23:04:24.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.710+0000 7f951f7fe640 1 --2- 172.21.15.32:0/1869900064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f952015e9b0 0x7f952014a740 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f951000cef0 tx=0x7f951000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:24.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.710+0000 7f95177fe640 1 -- 172.21.15.32:0/1869900064 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951001a070 con 0x7f952015e9b0 2024-04-24T23:04:24.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.710+0000 7f95177fe640 1 -- 172.21.15.32:0/1869900064 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f951000d860 con 0x7f952015e9b0 2024-04-24T23:04:24.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.710+0000 7f95256a5640 1 -- 172.21.15.32:0/1869900064 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95200cb820 con 0x7f952015e9b0 2024-04-24T23:04:24.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.710+0000 7f95177fe640 1 -- 172.21.15.32:0/1869900064 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951000dc50 con 0x7f952015e9b0 2024-04-24T23:04:24.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.711+0000 7f95256a5640 1 -- 172.21.15.32:0/1869900064 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95200cbde0 con 0x7f952015e9b0 2024-04-24T23:04:24.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.712+0000 7f95256a5640 1 -- 172.21.15.32:0/1869900064 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9520068430 con 0x7f952015e9b0 2024-04-24T23:04:24.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.712+0000 7f95177fe640 1 -- 172.21.15.32:0/1869900064 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f951001d070 con 0x7f952015e9b0 2024-04-24T23:04:24.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.713+0000 7f95177fe640 1 --2- 172.21.15.32:0/1869900064 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f950006be30 0x7f950006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:24.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.713+0000 7f95177fe640 1 -- 172.21.15.32:0/1869900064 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f951008b650 con 0x7f952015e9b0 2024-04-24T23:04:24.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.713+0000 7f951effd640 1 --2- 172.21.15.32:0/1869900064 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f950006be30 0x7f950006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:24.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.713+0000 7f95177fe640 1 -- 172.21.15.32:0/1869900064 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=110}) -- 0x7f9500073960 con 0x7f952015e9b0 2024-04-24T23:04:24.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.713+0000 7f951effd640 1 --2- 172.21.15.32:0/1869900064 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f950006be30 0x7f950006e2f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f9518007fd0 tx=0x7f9518007de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:24.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.717+0000 7f95177fe640 1 -- 172.21.15.32:0/1869900064 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9510059a70 con 0x7f952015e9b0 2024-04-24T23:04:24.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.903+0000 7f95256a5640 1 -- 172.21.15.32:0/1869900064 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f95200728d0 con 0x7f952015e9b0 2024-04-24T23:04:24.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.905+0000 7f95177fe640 1 -- 172.21.15.32:0/1869900064 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v109) ==== 81+0+34296 (secure 0 0 0) 0x7f951005dd70 con 0x7f952015e9b0 2024-04-24T23:04:24.907 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:04:24.907 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:04:24.907 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 109, 2024-04-24T23:04:24.907 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", 2024-04-24T23:04:24.907 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2024-04-24T22:58:46.964851+0000", 2024-04-24T23:04:24.907 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2024-04-24T23:04:23.492033+0000", 2024-04-24T23:04:24.907 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2024-04-24T22:58:52.048787+0000", 2024-04-24T23:04:24.907 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2024-04-24T22:58:49.045609+0000", 2024-04-24T23:04:24.907 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-24T23:04:24.907 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2024-04-24T23:04:24.908 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2024-04-24T23:04:24.908 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2024-04-24T23:04:24.908 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2024-04-24T23:04:24.908 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2024-04-24T23:04:24.908 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2024-04-24T23:04:24.908 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:04:24.908 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 4, 2024-04-24T23:04:24.908 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.69999998807907104, 2024-04-24T23:04:24.908 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-24T23:04:24.908 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-24T23:04:24.908 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2024-04-24T23:04:24.908 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 13, 2024-04-24T23:04:24.908 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2024-04-24T23:04:24.908 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "luminous", 2024-04-24T23:04:24.909 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "jewel", 2024-04-24T23:04:24.909 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "squid", 2024-04-24T23:04:24.909 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_crimson": false, 2024-04-24T23:04:24.909 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2024-04-24T23:04:24.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.909 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2024-04-24T23:04:24.909 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": ".mgr", 2024-04-24T23:04:24.909 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T22:58:53.171408+0000", 2024-04-24T23:04:24.909 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 3, 2024-04-24T23:04:24.909 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,full", 2024-04-24T23:04:24.909 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:04:24.909 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:04:24.909 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:04:24.909 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:04:24.910 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:04:24.910 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:04:24.910 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:04:24.910 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:04:24.910 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:04:24.910 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:04:24.910 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2024-04-24T23:04:24.910 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2024-04-24T23:04:24.910 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2024-04-24T23:04:24.910 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2024-04-24T23:04:24.910 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2024-04-24T23:04:24.910 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:04:24.910 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:04:24.910 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:04:24.910 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:04:24.911 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:04:24.911 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:04:24.911 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:04:24.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.911 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "105", 2024-04-24T23:04:24.911 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:04:24.911 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:04:24.911 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:04:24.911 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:04:24.911 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:04:24.911 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:04:24.911 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:04:24.911 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:04:24.911 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:04:24.911 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:04:24.912 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:04:24.912 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:04:24.912 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:04:24.912 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:04:24.912 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:04:24.912 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:04:24.912 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:04:24.912 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:04:24.912 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:04:24.912 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:04:24.912 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:04:24.912 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:04:24.912 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:04:24.912 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:04:24.912 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:04:24.913 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:04:24.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.913 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:04:24.913 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:04:24.913 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:04:24.913 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:04:24.913 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:04:24.913 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:04:24.913 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:04:24.913 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:04:24.914 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:04:24.914 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:04:24.914 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:04:24.914 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:04:24.914 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_max": 32, 2024-04-24T23:04:24.914 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2024-04-24T23:04:24.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.914 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:04:24.914 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": {} 2024-04-24T23:04:24.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.914 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:04:24.914 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:04:24.914 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 8, 2024-04-24T23:04:24.915 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 8, 2024-04-24T23:04:24.915 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.25, 2024-04-24T23:04:24.915 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-04-24T23:04:24.915 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-04-24T23:04:24.915 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:04:24.915 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:04:24.915 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:04:24.915 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:24.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.915 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 12, 2024-04-24T23:04:24.915 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2024-04-24T23:04:24.915 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:03:59.079157+0000", 2024-04-24T23:04:24.916 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 3, 2024-04-24T23:04:24.916 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,full", 2024-04-24T23:04:24.916 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:04:24.916 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:04:24.916 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:04:24.916 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:04:24.916 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:04:24.916 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:04:24.916 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:04:24.917 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:04:24.917 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:04:24.917 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:04:24.917 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 32, 2024-04-24T23:04:24.917 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 32, 2024-04-24T23:04:24.917 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 32, 2024-04-24T23:04:24.917 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 32, 2024-04-24T23:04:24.917 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 32, 2024-04-24T23:04:24.917 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:04:24.917 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:04:24.917 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:04:24.917 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:04:24.917 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:04:24.917 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:04:24.918 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:04:24.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.918 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "105", 2024-04-24T23:04:24.918 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:04:24.918 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:04:24.918 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:04:24.918 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:04:24.918 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:04:24.918 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:04:24.919 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:04:24.919 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:04:24.919 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:04:24.919 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:04:24.919 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:04:24.919 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:04:24.919 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:04:24.919 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:04:24.919 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:04:24.919 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:04:24.919 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:04:24.919 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:04:24.919 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:04:24.919 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:04:24.920 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:04:24.920 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:04:24.920 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:04:24.920 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:04:24.920 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:04:24.920 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:04:24.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.920 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:04:24.920 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:04:24.920 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:04:24.921 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:04:24.921 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:04:24.921 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:04:24.921 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:04:24.921 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:04:24.921 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:04:24.921 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:04:24.921 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:04:24.921 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:04:24.921 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2024-04-24T23:04:24.921 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2024-04-24T23:04:24.921 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2024-04-24T23:04:24.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.922 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:04:24.922 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:04:24.922 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2024-04-24T23:04:24.922 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:24.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.922 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:04:24.922 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:04:24.922 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.75, 2024-04-24T23:04:24.922 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.75, 2024-04-24T23:04:24.923 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:04:24.923 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.75, 2024-04-24T23:04:24.923 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.75, 2024-04-24T23:04:24.923 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:04:24.923 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:04:24.923 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:04:24.923 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:24.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.924 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 13, 2024-04-24T23:04:24.924 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2024-04-24T23:04:24.924 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:03:59.765179+0000", 2024-04-24T23:04:24.924 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 3, 2024-04-24T23:04:24.924 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,full", 2024-04-24T23:04:24.924 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:04:24.924 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:04:24.924 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:04:24.925 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:04:24.925 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:04:24.925 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:04:24.925 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:04:24.925 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:04:24.925 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:04:24.925 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:04:24.925 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 64, 2024-04-24T23:04:24.925 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 64, 2024-04-24T23:04:24.925 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 64, 2024-04-24T23:04:24.926 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 64, 2024-04-24T23:04:24.926 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 64, 2024-04-24T23:04:24.926 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:04:24.926 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:04:24.926 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:04:24.926 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:04:24.926 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:04:24.926 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:04:24.926 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:04:24.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.926 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "105", 2024-04-24T23:04:24.927 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:04:24.927 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:04:24.927 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:04:24.927 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:04:24.927 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:04:24.927 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:04:24.927 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:04:24.927 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:04:24.927 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:04:24.927 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:04:24.927 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:04:24.927 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:04:24.927 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:04:24.927 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:04:24.927 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:04:24.928 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:04:24.928 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:04:24.928 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:04:24.928 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:04:24.928 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:04:24.928 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:04:24.928 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:04:24.928 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:04:24.928 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:04:24.928 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:04:24.928 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:04:24.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.928 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:04:24.928 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:04:24.928 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:04:24.929 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:04:24.929 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:04:24.929 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:04:24.929 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:04:24.929 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:04:24.929 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:04:24.929 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:04:24.929 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:04:24.929 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:04:24.929 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 64, 2024-04-24T23:04:24.929 INFO:teuthology.orchestra.run.smithi032.stdout: "target_size_ratio": 0.89999997615814209 2024-04-24T23:04:24.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.929 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:04:24.929 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:04:24.930 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2024-04-24T23:04:24.930 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:24.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.930 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:04:24.930 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:04:24.930 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.3799999952316284, 2024-04-24T23:04:24.930 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.3799999952316284, 2024-04-24T23:04:24.930 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:04:24.930 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.3799999952316284, 2024-04-24T23:04:24.930 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.3799999952316284, 2024-04-24T23:04:24.930 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:04:24.930 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:04:24.930 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:04:24.930 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:24.930 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:24.931 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:04:24.931 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-04-24T23:04:24.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.931 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:04:24.931 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:24.931 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:24.931 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:24.931 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:24.931 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:24.931 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:24.931 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:24.931 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:24.931 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:24.931 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:24.932 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:24.932 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:24.932 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:24.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.932 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:24.932 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2024-04-24T23:04:24.932 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:24.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.932 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:24.932 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2024-04-24T23:04:24.932 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:24.932 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:24.932 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:24.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.933 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:24.933 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:24.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.933 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:24.933 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2024-04-24T23:04:24.933 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:24.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.933 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:24.933 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2024-04-24T23:04:24.933 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:24.933 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:24.933 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:24.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.934 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:24.934 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:24.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.934 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:24.934 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2024-04-24T23:04:24.934 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:24.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.934 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:24.934 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2024-04-24T23:04:24.934 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:24.934 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:24.934 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:24.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.935 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:24.935 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:24.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.935 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:24.935 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2024-04-24T23:04:24.935 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:24.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.935 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:24.935 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2024-04-24T23:04:24.935 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:24.935 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:24.935 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:24.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.935 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/1479395426", 2024-04-24T23:04:24.936 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/1479395426", 2024-04-24T23:04:24.936 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/1479395426", 2024-04-24T23:04:24.936 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/1479395426", 2024-04-24T23:04:24.936 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:24.936 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:24.936 INFO:teuthology.orchestra.run.smithi032.stdout: "full", 2024-04-24T23:04:24.936 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:24.936 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:24.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.936 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:04:24.936 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:24.936 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:24.936 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:24.936 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:24.937 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:24.937 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:24.937 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:24.937 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:24.937 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:24.937 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:24.937 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:24.937 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:24.937 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:24.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.937 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:24.937 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2024-04-24T23:04:24.937 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:24.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.938 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:24.938 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2024-04-24T23:04:24.938 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:24.938 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:24.938 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:24.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.938 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:24.938 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:24.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.938 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:24.938 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2024-04-24T23:04:24.938 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:24.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.939 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:24.939 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2024-04-24T23:04:24.939 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:24.939 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:24.939 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:24.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.939 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:24.939 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:24.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.939 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:24.939 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2024-04-24T23:04:24.939 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:24.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.940 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:24.940 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2024-04-24T23:04:24.940 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:24.940 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:24.940 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:24.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.940 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:24.940 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:24.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.940 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:24.940 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2024-04-24T23:04:24.940 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:24.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.940 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:24.941 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2024-04-24T23:04:24.941 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:24.941 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:24.941 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:24.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.941 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/1340633718", 2024-04-24T23:04:24.941 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/1340633718", 2024-04-24T23:04:24.941 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/1340633718", 2024-04-24T23:04:24.941 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/1340633718", 2024-04-24T23:04:24.941 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:24.941 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:24.941 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:24.941 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:24.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.942 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:04:24.942 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:24.942 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:24.942 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:24.942 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:24.942 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:24.942 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:24.942 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:24.942 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:24.942 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:24.942 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:24.942 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:24.942 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:24.942 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:24.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.943 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:24.943 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2024-04-24T23:04:24.943 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:24.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.943 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:24.943 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2024-04-24T23:04:24.943 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:24.943 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:24.943 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:24.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.944 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:24.944 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:24.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.944 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:24.944 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2024-04-24T23:04:24.944 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:24.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.944 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:24.944 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2024-04-24T23:04:24.944 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:24.944 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:24.944 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:24.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.945 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:24.945 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:24.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.945 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:24.945 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2024-04-24T23:04:24.945 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:24.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.945 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:24.945 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2024-04-24T23:04:24.945 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:24.945 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:24.945 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:24.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.946 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:24.946 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:24.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.946 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:24.946 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2024-04-24T23:04:24.946 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:24.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.946 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:24.946 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2024-04-24T23:04:24.946 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:24.946 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:24.946 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:24.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.947 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/3405899334", 2024-04-24T23:04:24.947 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6820/3405899334", 2024-04-24T23:04:24.947 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6828/3405899334", 2024-04-24T23:04:24.947 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6824/3405899334", 2024-04-24T23:04:24.947 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:24.947 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:24.947 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:24.947 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:24.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.947 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:04:24.947 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:24.947 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:24.947 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:24.947 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:24.948 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:24.948 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:24.948 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:24.948 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:24.948 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:24.948 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:24.948 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:24.948 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:24.948 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:24.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.948 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:24.948 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2024-04-24T23:04:24.948 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:24.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.949 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:24.949 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2024-04-24T23:04:24.949 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:24.949 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:24.949 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:24.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.949 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:24.949 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:24.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.949 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:24.949 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2024-04-24T23:04:24.949 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:24.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.950 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:24.950 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2024-04-24T23:04:24.950 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:24.950 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:24.950 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:24.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.950 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:24.950 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:24.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.951 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:24.951 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2024-04-24T23:04:24.951 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:24.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.951 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:24.951 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2024-04-24T23:04:24.951 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:24.951 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:24.951 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:24.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.952 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:24.952 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:24.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.952 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:24.952 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2024-04-24T23:04:24.952 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:24.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.952 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:24.952 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2024-04-24T23:04:24.952 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:24.952 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:24.952 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:24.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.953 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6821/3381057190", 2024-04-24T23:04:24.953 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6825/3381057190", 2024-04-24T23:04:24.953 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/3381057190", 2024-04-24T23:04:24.953 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/3381057190", 2024-04-24T23:04:24.953 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:24.953 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:24.953 INFO:teuthology.orchestra.run.smithi032.stdout: "full", 2024-04-24T23:04:24.953 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:24.953 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:24.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.953 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:04:24.954 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:24.954 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:24.954 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:24.954 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:24.954 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:24.954 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:24.954 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:24.954 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:24.954 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:24.954 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:24.954 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:24.955 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:24.955 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:24.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.955 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:24.955 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6800", 2024-04-24T23:04:24.955 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:24.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.955 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:24.955 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6801", 2024-04-24T23:04:24.955 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:24.955 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:24.956 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:24.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.956 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:24.956 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:24.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.956 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:24.956 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6802", 2024-04-24T23:04:24.956 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:24.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.956 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:24.956 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6803", 2024-04-24T23:04:24.957 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:24.957 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:24.957 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:24.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.957 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:24.957 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:24.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.957 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:24.957 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6806", 2024-04-24T23:04:24.958 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:24.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.958 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:24.958 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6807", 2024-04-24T23:04:24.958 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:24.958 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:24.958 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:24.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.959 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:24.959 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:24.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.959 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:24.959 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6804", 2024-04-24T23:04:24.959 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:24.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.959 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:24.959 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6805", 2024-04-24T23:04:24.959 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:24.959 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:24.959 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:24.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.960 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6801/4010553979", 2024-04-24T23:04:24.960 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6803/4010553979", 2024-04-24T23:04:24.960 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6807/4010553979", 2024-04-24T23:04:24.960 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6805/4010553979", 2024-04-24T23:04:24.960 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:24.960 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:24.960 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:24.960 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:24.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.960 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:04:24.960 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:24.961 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:24.961 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:24.961 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:24.961 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:24.961 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:24.961 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:24.961 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:24.961 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:24.961 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:24.961 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:24.961 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:24.961 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:24.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.962 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:24.962 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6808", 2024-04-24T23:04:24.962 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:24.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.962 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:24.962 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6809", 2024-04-24T23:04:24.962 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:24.962 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:24.962 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:24.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.962 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:24.962 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:24.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.963 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:24.963 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6810", 2024-04-24T23:04:24.963 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:24.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.963 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:24.963 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6811", 2024-04-24T23:04:24.963 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:24.963 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:24.963 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:24.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.964 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:24.964 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:24.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.964 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:24.964 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6814", 2024-04-24T23:04:24.964 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:24.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.964 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:24.965 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6815", 2024-04-24T23:04:24.965 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:24.965 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:24.965 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:24.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.965 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:24.965 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:24.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.965 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:24.965 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6812", 2024-04-24T23:04:24.965 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:24.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.966 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:24.966 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6813", 2024-04-24T23:04:24.966 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:24.966 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:24.966 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:24.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.966 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6809/3794084091", 2024-04-24T23:04:24.966 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6811/3794084091", 2024-04-24T23:04:24.967 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6815/3794084091", 2024-04-24T23:04:24.967 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6813/3794084091", 2024-04-24T23:04:24.967 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:24.967 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:24.967 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:24.967 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:24.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.967 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.967 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:04:24.967 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:24.967 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:24.967 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:24.967 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:24.968 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:24.968 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:24.968 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:24.968 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:24.968 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:24.968 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:24.968 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:24.968 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:24.968 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:24.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.968 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:24.968 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6823", 2024-04-24T23:04:24.968 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:24.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.969 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:24.969 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6825", 2024-04-24T23:04:24.969 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:24.969 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:24.969 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:24.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.969 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:24.969 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:24.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.969 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:24.969 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6826", 2024-04-24T23:04:24.969 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:24.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.969 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:24.970 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6827", 2024-04-24T23:04:24.970 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:24.970 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:24.970 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:24.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.970 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:24.970 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:24.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.970 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:24.970 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6830", 2024-04-24T23:04:24.970 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:24.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.970 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:24.971 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6831", 2024-04-24T23:04:24.971 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:24.971 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:24.971 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:24.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.971 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:24.971 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:24.971 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.971 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:24.971 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6828", 2024-04-24T23:04:24.971 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:24.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.971 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.971 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:24.971 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6829", 2024-04-24T23:04:24.972 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:24.972 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:24.972 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:24.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.972 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6825/1348640258", 2024-04-24T23:04:24.972 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6827/1348640258", 2024-04-24T23:04:24.972 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6831/1348640258", 2024-04-24T23:04:24.972 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6829/1348640258", 2024-04-24T23:04:24.972 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:24.972 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:24.972 INFO:teuthology.orchestra.run.smithi032.stdout: "full", 2024-04-24T23:04:24.972 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:24.972 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:24.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.973 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:04:24.973 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:24.973 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:24.973 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:24.973 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:24.973 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:24.973 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:24.973 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:24.973 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:24.973 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:24.973 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:24.973 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:24.974 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:24.974 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:24.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.974 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:24.974 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6816", 2024-04-24T23:04:24.974 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:24.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.974 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:24.974 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6817", 2024-04-24T23:04:24.974 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:24.974 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:24.974 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:24.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.975 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:24.975 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:24.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.975 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:24.975 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6818", 2024-04-24T23:04:24.975 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:24.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.975 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:24.975 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6819", 2024-04-24T23:04:24.975 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:24.975 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:24.975 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:24.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.976 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:24.976 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:24.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.976 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:24.976 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6822", 2024-04-24T23:04:24.976 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:24.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.976 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:24.976 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6824", 2024-04-24T23:04:24.976 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:24.976 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:24.976 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:24.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.976 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:24.977 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:24.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.977 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:24.977 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6820", 2024-04-24T23:04:24.977 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:24.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.977 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:24.977 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6821", 2024-04-24T23:04:24.977 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:24.977 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:24.977 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:24.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.977 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6817/3250005122", 2024-04-24T23:04:24.977 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6819/3250005122", 2024-04-24T23:04:24.978 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6824/3250005122", 2024-04-24T23:04:24.978 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6821/3250005122", 2024-04-24T23:04:24.978 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:24.978 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:24.978 INFO:teuthology.orchestra.run.smithi032.stdout: "full", 2024-04-24T23:04:24.978 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:24.978 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:24.978 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:24.978 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:04:24.978 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2024-04-24T23:04:24.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.978 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:04:24.978 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:24.978 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:24.979 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:24.979 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:24.979 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:24.979 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.156184+0000", 2024-04-24T23:04:24.979 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:24.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.979 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:04:24.979 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:24.979 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:24.979 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:24.979 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:24.979 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:24.979 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240774+0000", 2024-04-24T23:04:24.979 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:24.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.980 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:04:24.980 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:24.980 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:24.980 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:24.980 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:24.980 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:24.980 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240949+0000", 2024-04-24T23:04:24.980 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:24.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.980 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:04:24.980 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:24.980 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:24.981 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:24.981 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:24.981 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:24.981 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.171672+0000", 2024-04-24T23:04:24.981 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:24.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.981 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.981 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:04:24.981 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:24.981 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:24.981 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:24.981 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:24.981 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:24.981 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.220379+0000", 2024-04-24T23:04:24.981 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:24.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.982 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:04:24.982 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:24.982 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:24.982 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:24.982 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:24.982 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:24.982 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.204963+0000", 2024-04-24T23:04:24.982 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:24.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.982 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:04:24.982 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:24.983 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:24.983 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:24.983 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:24.983 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:24.983 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.239748+0000", 2024-04-24T23:04:24.983 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:24.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:24.983 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:04:24.983 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:24.983 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:24.983 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:24.983 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:24.983 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:24.983 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.154104+0000", 2024-04-24T23:04:24.984 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:24.984 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:24.984 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:04:24.984 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2024-04-24T23:04:24.984 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2024-04-24T23:04:24.984 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_primaries": [], 2024-04-24T23:04:24.984 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2024-04-24T23:04:24.984 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2024-04-24T23:04:24.984 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklist": {}, 2024-04-24T23:04:24.984 INFO:teuthology.orchestra.run.smithi032.stdout: "range_blocklist": {}, 2024-04-24T23:04:24.984 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2024-04-24T23:04:24.984 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2024-04-24T23:04:24.984 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2024-04-24T23:04:24.984 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2024-04-24T23:04:24.984 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2024-04-24T23:04:24.985 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2024-04-24T23:04:24.985 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2024-04-24T23:04:24.985 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:24.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:24.985 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2024-04-24T23:04:24.985 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2024-04-24T23:04:24.985 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2024-04-24T23:04:24.985 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2024-04-24T23:04:24.985 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {}, 2024-04-24T23:04:24.985 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode": { 2024-04-24T23:04:24.985 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_enabled": false, 2024-04-24T23:04:24.986 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_bucket_count": 0, 2024-04-24T23:04:24.986 INFO:teuthology.orchestra.run.smithi032.stdout: "degraded_stretch_mode": 0, 2024-04-24T23:04:24.986 INFO:teuthology.orchestra.run.smithi032.stdout: "recovering_stretch_mode": 0, 2024-04-24T23:04:24.986 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_bucket": 0 2024-04-24T23:04:24.986 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:24.986 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:04:24.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.907+0000 7f95256a5640 1 -- 172.21.15.32:0/1869900064 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f950006be30 msgr2=0x7f950006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:24.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.907+0000 7f95256a5640 1 --2- 172.21.15.32:0/1869900064 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f950006be30 0x7f950006e2f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f9518007fd0 tx=0x7f9518007de0 comp rx=0 tx=0).stop 2024-04-24T23:04:24.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.907+0000 7f95256a5640 1 -- 172.21.15.32:0/1869900064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f952015e9b0 msgr2=0x7f952014a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:24.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.907+0000 7f95256a5640 1 --2- 172.21.15.32:0/1869900064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f952015e9b0 0x7f952014a740 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f951000cef0 tx=0x7f951000a710 comp rx=0 tx=0).stop 2024-04-24T23:04:24.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.907+0000 7f95256a5640 1 -- 172.21.15.32:0/1869900064 shutdown_connections 2024-04-24T23:04:24.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.907+0000 7f95256a5640 1 --2- 172.21.15.32:0/1869900064 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f950006be30 0x7f950006e2f0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:24.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.907+0000 7f95256a5640 1 --2- 172.21.15.32:0/1869900064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f952015e9b0 0x7f952014a740 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:24.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.907+0000 7f95256a5640 1 --2- 172.21.15.32:0/1869900064 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f95200d4210 0x7f9520154da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:24.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.907+0000 7f95256a5640 1 --2- 172.21.15.32:0/1869900064 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f952006b820 0x7f95200cc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:24.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.907+0000 7f95256a5640 1 -- 172.21.15.32:0/1869900064 >> 172.21.15.32:0/1869900064 conn(0x7f95200d11d0 msgr2=0x7f95200c9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:24.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.907+0000 7f95256a5640 1 -- 172.21.15.32:0/1869900064 shutdown_connections 2024-04-24T23:04:24.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:24.907+0000 7f95256a5640 1 -- 172.21.15.32:0/1869900064 wait complete. 2024-04-24T23:04:24.989 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-04-24T23:04:27.552 INFO:tasks.ceph.mon.a.smithi032.stderr:2024-04-24T23:04:27.550+0000 7f3b831cb640 -1 log_channel(cluster) log [ERR] : Health check update: 3 full osd(s) (OSD_FULL) 2024-04-24T23:04:29.990 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-24T23:04:30.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.103+0000 7f37e1e95640 1 Processor -- start 2024-04-24T23:04:30.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.103+0000 7f37e1e95640 1 -- start start 2024-04-24T23:04:30.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.104+0000 7f37e1e95640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f37dc15b380 0x7f37dc06c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:30.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.104+0000 7f37e1e95640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f37dc15cb00 0x7f37dc06cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:30.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.104+0000 7f37e1e95640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37dc0b4520 con 0x7f37dc0ce430 2024-04-24T23:04:30.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.104+0000 7f37e1e95640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f37dc0cef90 con 0x7f37dc15cb00 2024-04-24T23:04:30.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.104+0000 7f37e1e95640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f37dc0b4800 con 0x7f37dc15b380 2024-04-24T23:04:30.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.104+0000 7f37daffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f37dc15cb00 0x7f37dc06cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:30.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.104+0000 7f37dbfff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f37dc0ce430 0x7f37dc0ce880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39154/0 (socket says 172.21.15.32:39154) 2024-04-24T23:04:30.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.104+0000 7f37daffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f37dc15cb00 0x7f37dc06cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43934/0 (socket says 172.21.15.32:43934) 2024-04-24T23:04:30.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.104+0000 7f37dbfff640 1 -- 172.21.15.32:0/647605991 learned_addr learned my addr 172.21.15.32:0/647605991 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:04:30.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.105+0000 7f37db7fe640 1 --2- 172.21.15.32:0/647605991 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f37dc15b380 0x7f37dc06c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:30.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.105+0000 7f37daffd640 1 -- 172.21.15.32:0/647605991 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f37dc15b380 msgr2=0x7f37dc06c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:30.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.105+0000 7f37daffd640 1 --2- 172.21.15.32:0/647605991 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f37dc15b380 0x7f37dc06c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:30.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.105+0000 7f37daffd640 1 -- 172.21.15.32:0/647605991 >> v1:172.21.15.32:6789/0 conn(0x7f37dc0ce430 legacy=0x7f37dc0ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:30.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.105+0000 7f37daffd640 1 -- 172.21.15.32:0/647605991 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37dc0b4ae0 con 0x7f37dc15cb00 2024-04-24T23:04:30.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.105+0000 7f37da7fc640 1 -- 172.21.15.32:0/647605991 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3690519971 0 0) 0x7f37dc0b4520 con 0x7f37dc0ce430 2024-04-24T23:04:30.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.105+0000 7f37db7fe640 1 --2- 172.21.15.32:0/647605991 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f37dc15b380 0x7f37dc06c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:04:30.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.106+0000 7f37daffd640 1 --2- 172.21.15.32:0/647605991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37dc15cb00 0x7f37dc06cb30 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f37d00097a0 tx=0x7f37d002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ae958394f995aca server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:30.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.106+0000 7f37da7fc640 1 -- 172.21.15.32:0/647605991 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d0037070 con 0x7f37dc15cb00 2024-04-24T23:04:30.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.106+0000 7f37da7fc640 1 -- 172.21.15.32:0/647605991 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37d002ae50 con 0x7f37dc15cb00 2024-04-24T23:04:30.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.106+0000 7f37da7fc640 1 -- 172.21.15.32:0/647605991 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d0032660 con 0x7f37dc15cb00 2024-04-24T23:04:30.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.106+0000 7f37e1e95640 1 -- 172.21.15.32:0/647605991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37dc15cb00 msgr2=0x7f37dc06cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:30.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.106+0000 7f37e1e95640 1 --2- 172.21.15.32:0/647605991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37dc15cb00 0x7f37dc06cb30 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f37d00097a0 tx=0x7f37d002a100 comp rx=0 tx=0).stop 2024-04-24T23:04:30.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.106+0000 7f37e1e95640 1 -- 172.21.15.32:0/647605991 shutdown_connections 2024-04-24T23:04:30.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.106+0000 7f37e1e95640 1 --2- 172.21.15.32:0/647605991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37dc15cb00 0x7f37dc06cb30 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:30.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.106+0000 7f37e1e95640 1 --2- 172.21.15.32:0/647605991 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f37dc15b380 0x7f37dc06c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:30.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.106+0000 7f37e1e95640 1 -- 172.21.15.32:0/647605991 >> 172.21.15.32:0/647605991 conn(0x7f37dc06b110 msgr2=0x7f37dc06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:30.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.106+0000 7f37e1e95640 1 -- 172.21.15.32:0/647605991 shutdown_connections 2024-04-24T23:04:30.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.106+0000 7f37e1e95640 1 -- 172.21.15.32:0/647605991 wait complete. 2024-04-24T23:04:30.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.108+0000 7f37e1e95640 1 Processor -- start 2024-04-24T23:04:30.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.108+0000 7f37e1e95640 1 -- start start 2024-04-24T23:04:30.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.108+0000 7f37e1e95640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37dc0ce430 0x7f37dc1e6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:30.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.108+0000 7f37e1e95640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37dc15b380 0x7f37dc143cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:30.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.108+0000 7f37e1e95640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f37dc15cb00 0x7f37dc155c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:30.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.108+0000 7f37e1e95640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f37dc15b760 con 0x7f37dc15b380 2024-04-24T23:04:30.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.108+0000 7f37e1e95640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f37dc0b50a0 con 0x7f37dc0ce430 2024-04-24T23:04:30.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.108+0000 7f37e1e95640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f37dc0b55c0 con 0x7f37dc15cb00 2024-04-24T23:04:30.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.109+0000 7f37daffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37dc15b380 0x7f37dc143cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:30.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.109+0000 7f37daffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37dc15b380 0x7f37dc143cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43938/0 (socket says 172.21.15.32:43938) 2024-04-24T23:04:30.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.109+0000 7f37daffd640 1 -- 172.21.15.32:0/4142357588 learned_addr learned my addr 172.21.15.32:0/4142357588 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:30.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.109+0000 7f37db7fe640 1 --2- 172.21.15.32:0/4142357588 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37dc0ce430 0x7f37dc1e6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:30.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.109+0000 7f37dbfff640 1 --2- 172.21.15.32:0/4142357588 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f37dc15cb00 0x7f37dc155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:30.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.109+0000 7f37daffd640 1 -- 172.21.15.32:0/4142357588 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f37dc15cb00 msgr2=0x7f37dc155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:30.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.109+0000 7f37daffd640 1 --2- 172.21.15.32:0/4142357588 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f37dc15cb00 0x7f37dc155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:30.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.109+0000 7f37daffd640 1 -- 172.21.15.32:0/4142357588 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37dc0ce430 msgr2=0x7f37dc1e6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:30.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.109+0000 7f37daffd640 1 --2- 172.21.15.32:0/4142357588 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37dc0ce430 0x7f37dc1e6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:30.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.109+0000 7f37daffd640 1 -- 172.21.15.32:0/4142357588 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37dc144200 con 0x7f37dc15b380 2024-04-24T23:04:30.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.109+0000 7f37db7fe640 1 --2- 172.21.15.32:0/4142357588 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37dc0ce430 0x7f37dc1e6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:04:30.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.109+0000 7f37daffd640 1 --2- 172.21.15.32:0/4142357588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37dc15b380 0x7f37dc143cc0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f37d00098c0 tx=0x7f37d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:30.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.109+0000 7f37c3fff640 1 -- 172.21.15.32:0/4142357588 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d0041020 con 0x7f37dc15b380 2024-04-24T23:04:30.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.109+0000 7f37c3fff640 1 -- 172.21.15.32:0/4142357588 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37d003c670 con 0x7f37dc15b380 2024-04-24T23:04:30.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.109+0000 7f37e1e95640 1 -- 172.21.15.32:0/4142357588 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37dc1e5c60 con 0x7f37dc15b380 2024-04-24T23:04:30.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.109+0000 7f37c3fff640 1 -- 172.21.15.32:0/4142357588 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d0037040 con 0x7f37dc15b380 2024-04-24T23:04:30.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.109+0000 7f37e1e95640 1 -- 172.21.15.32:0/4142357588 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37dc1e6220 con 0x7f37dc15b380 2024-04-24T23:04:30.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.110+0000 7f37e1e95640 1 -- 172.21.15.32:0/4142357588 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37dc068430 con 0x7f37dc15b380 2024-04-24T23:04:30.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.111+0000 7f37c3fff640 1 -- 172.21.15.32:0/4142357588 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f37d0044020 con 0x7f37dc15b380 2024-04-24T23:04:30.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.111+0000 7f37c3fff640 1 --2- 172.21.15.32:0/4142357588 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f37c806be30 0x7f37c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:30.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.111+0000 7f37c3fff640 1 -- 172.21.15.32:0/4142357588 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5709+0+0 (secure 0 0 0) 0x7f37d00ab600 con 0x7f37dc15b380 2024-04-24T23:04:30.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.111+0000 7f37c3fff640 1 -- 172.21.15.32:0/4142357588 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=112}) -- 0x7f37c8073960 con 0x7f37dc15b380 2024-04-24T23:04:30.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.112+0000 7f37db7fe640 1 --2- 172.21.15.32:0/4142357588 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f37c806be30 0x7f37c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:30.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.112+0000 7f37db7fe640 1 --2- 172.21.15.32:0/4142357588 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f37c806be30 0x7f37c806e2f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f37cc006fd0 tx=0x7f37cc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:30.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.116+0000 7f37c3fff640 1 -- 172.21.15.32:0/4142357588 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f37d004b030 con 0x7f37dc15b380 2024-04-24T23:04:30.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.300+0000 7f37e1e95640 1 -- 172.21.15.32:0/4142357588 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f37dc0afd60 con 0x7f37dc15b380 2024-04-24T23:04:30.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.301+0000 7f37c3fff640 1 -- 172.21.15.32:0/4142357588 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v111) ==== 81+0+34248 (secure 0 0 0) 0x7f37dc144200 con 0x7f37dc15b380 2024-04-24T23:04:30.304 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:04:30.304 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:04:30.304 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 111, 2024-04-24T23:04:30.304 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", 2024-04-24T23:04:30.304 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2024-04-24T22:58:46.964851+0000", 2024-04-24T23:04:30.304 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2024-04-24T23:04:29.604993+0000", 2024-04-24T23:04:30.304 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2024-04-24T22:58:52.048787+0000", 2024-04-24T23:04:30.304 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2024-04-24T22:58:49.045609+0000", 2024-04-24T23:04:30.305 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-24T23:04:30.305 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2024-04-24T23:04:30.305 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2024-04-24T23:04:30.305 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2024-04-24T23:04:30.305 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2024-04-24T23:04:30.305 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2024-04-24T23:04:30.305 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2024-04-24T23:04:30.305 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:04:30.305 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 4, 2024-04-24T23:04:30.305 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.69999998807907104, 2024-04-24T23:04:30.305 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-24T23:04:30.305 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-24T23:04:30.305 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2024-04-24T23:04:30.305 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 13, 2024-04-24T23:04:30.306 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2024-04-24T23:04:30.306 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "luminous", 2024-04-24T23:04:30.306 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "jewel", 2024-04-24T23:04:30.306 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "squid", 2024-04-24T23:04:30.306 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_crimson": false, 2024-04-24T23:04:30.306 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2024-04-24T23:04:30.306 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.306 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2024-04-24T23:04:30.306 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": ".mgr", 2024-04-24T23:04:30.306 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T22:58:53.171408+0000", 2024-04-24T23:04:30.306 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 3, 2024-04-24T23:04:30.307 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,full", 2024-04-24T23:04:30.307 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:04:30.307 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:04:30.307 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:04:30.307 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:04:30.307 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:04:30.307 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:04:30.307 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:04:30.307 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:04:30.307 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:04:30.307 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:04:30.307 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2024-04-24T23:04:30.307 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2024-04-24T23:04:30.308 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2024-04-24T23:04:30.308 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2024-04-24T23:04:30.308 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2024-04-24T23:04:30.308 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:04:30.308 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:04:30.308 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:04:30.308 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:04:30.308 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:04:30.308 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:04:30.308 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:04:30.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.308 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "105", 2024-04-24T23:04:30.309 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:04:30.309 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:04:30.309 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:04:30.309 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:04:30.309 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:04:30.309 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:04:30.309 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:04:30.309 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:04:30.309 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:04:30.309 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:04:30.310 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:04:30.310 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:04:30.310 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:04:30.310 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:04:30.310 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:04:30.310 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:04:30.310 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:04:30.310 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:04:30.310 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:04:30.310 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:04:30.310 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:04:30.310 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:04:30.311 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:04:30.311 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:04:30.311 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:04:30.311 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:04:30.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.311 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:04:30.311 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:04:30.311 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:04:30.311 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:04:30.311 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:04:30.312 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:04:30.312 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:04:30.312 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:04:30.312 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:04:30.312 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:04:30.312 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:04:30.312 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:04:30.312 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_max": 32, 2024-04-24T23:04:30.312 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2024-04-24T23:04:30.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.312 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:04:30.312 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": {} 2024-04-24T23:04:30.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.313 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:04:30.313 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:04:30.313 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 8, 2024-04-24T23:04:30.313 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 8, 2024-04-24T23:04:30.313 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.25, 2024-04-24T23:04:30.313 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-04-24T23:04:30.313 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-04-24T23:04:30.313 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:04:30.314 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:04:30.314 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:04:30.314 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:30.314 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.314 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.314 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 12, 2024-04-24T23:04:30.314 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2024-04-24T23:04:30.314 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:03:59.079157+0000", 2024-04-24T23:04:30.314 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 3, 2024-04-24T23:04:30.314 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,full", 2024-04-24T23:04:30.314 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:04:30.314 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:04:30.315 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:04:30.315 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:04:30.315 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:04:30.315 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:04:30.315 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:04:30.315 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:04:30.315 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:04:30.315 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:04:30.315 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 32, 2024-04-24T23:04:30.316 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 32, 2024-04-24T23:04:30.316 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 32, 2024-04-24T23:04:30.316 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 32, 2024-04-24T23:04:30.316 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 32, 2024-04-24T23:04:30.316 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:04:30.316 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:04:30.316 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:04:30.316 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:04:30.316 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:04:30.316 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:04:30.316 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:04:30.316 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.317 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "105", 2024-04-24T23:04:30.317 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:04:30.317 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:04:30.317 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:04:30.317 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:04:30.317 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:04:30.317 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:04:30.317 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:04:30.317 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:04:30.317 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:04:30.317 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:04:30.317 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:04:30.317 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:04:30.317 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:04:30.318 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:04:30.318 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:04:30.318 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:04:30.318 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:04:30.318 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:04:30.318 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:04:30.318 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:04:30.318 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:04:30.318 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:04:30.318 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:04:30.319 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:04:30.319 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:04:30.319 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:04:30.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.319 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:04:30.319 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:04:30.319 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:04:30.319 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:04:30.319 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:04:30.319 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:04:30.319 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:04:30.319 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:04:30.320 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:04:30.320 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:04:30.320 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:04:30.320 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:04:30.320 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2024-04-24T23:04:30.320 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2024-04-24T23:04:30.320 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2024-04-24T23:04:30.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.320 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:04:30.320 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:04:30.320 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2024-04-24T23:04:30.320 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:30.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.321 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:04:30.321 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:04:30.321 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.75, 2024-04-24T23:04:30.321 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.75, 2024-04-24T23:04:30.321 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:04:30.321 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.75, 2024-04-24T23:04:30.321 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.75, 2024-04-24T23:04:30.321 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:04:30.321 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:04:30.321 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:04:30.321 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:30.321 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.321 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.322 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 13, 2024-04-24T23:04:30.322 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2024-04-24T23:04:30.322 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:03:59.765179+0000", 2024-04-24T23:04:30.322 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 3, 2024-04-24T23:04:30.322 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,full", 2024-04-24T23:04:30.322 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:04:30.322 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:04:30.322 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:04:30.322 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:04:30.323 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:04:30.323 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:04:30.323 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:04:30.323 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:04:30.323 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:04:30.323 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:04:30.323 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 64, 2024-04-24T23:04:30.323 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 64, 2024-04-24T23:04:30.323 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 64, 2024-04-24T23:04:30.323 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 64, 2024-04-24T23:04:30.324 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 64, 2024-04-24T23:04:30.324 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:04:30.324 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:04:30.324 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:04:30.324 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:04:30.324 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:04:30.324 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:04:30.324 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:04:30.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.325 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "105", 2024-04-24T23:04:30.325 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:04:30.325 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:04:30.325 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:04:30.325 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:04:30.325 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:04:30.325 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:04:30.326 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:04:30.326 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:04:30.326 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:04:30.326 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:04:30.326 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:04:30.326 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:04:30.326 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:04:30.326 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:04:30.326 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:04:30.326 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:04:30.327 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:04:30.327 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:04:30.327 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:04:30.327 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:04:30.327 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:04:30.327 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:04:30.327 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:04:30.327 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:04:30.327 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:04:30.327 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:04:30.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.327 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:04:30.327 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:04:30.327 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:04:30.327 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:04:30.328 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:04:30.328 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:04:30.328 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:04:30.328 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:04:30.328 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:04:30.328 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:04:30.328 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:04:30.328 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:04:30.328 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 64, 2024-04-24T23:04:30.328 INFO:teuthology.orchestra.run.smithi032.stdout: "target_size_ratio": 0.89999997615814209 2024-04-24T23:04:30.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.328 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:04:30.329 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:04:30.329 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2024-04-24T23:04:30.329 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:30.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.329 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:04:30.329 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:04:30.329 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.3799999952316284, 2024-04-24T23:04:30.329 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.3799999952316284, 2024-04-24T23:04:30.329 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:04:30.329 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.3799999952316284, 2024-04-24T23:04:30.329 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.3799999952316284, 2024-04-24T23:04:30.329 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:04:30.329 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:04:30.329 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:04:30.330 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:30.330 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:30.330 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:04:30.330 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-04-24T23:04:30.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.330 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:04:30.330 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:30.330 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:30.330 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:30.330 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:30.330 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:30.330 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:30.330 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:30.330 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:30.331 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:30.331 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:30.331 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:30.331 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:30.331 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:30.331 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.331 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:30.331 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2024-04-24T23:04:30.331 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:30.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.331 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.331 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:30.332 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2024-04-24T23:04:30.332 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:30.332 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:30.332 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:30.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.332 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:30.332 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:30.332 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.332 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:30.332 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2024-04-24T23:04:30.332 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:30.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.332 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.332 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:30.333 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2024-04-24T23:04:30.333 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:30.333 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:30.333 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:30.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.333 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:30.333 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:30.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.333 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:30.333 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2024-04-24T23:04:30.333 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:30.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.333 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:30.334 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2024-04-24T23:04:30.334 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:30.334 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:30.334 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:30.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.334 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:30.334 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:30.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.334 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:30.334 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2024-04-24T23:04:30.334 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:30.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.335 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:30.335 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2024-04-24T23:04:30.335 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:30.335 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:30.335 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:30.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.335 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/1479395426", 2024-04-24T23:04:30.335 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/1479395426", 2024-04-24T23:04:30.335 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/1479395426", 2024-04-24T23:04:30.335 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/1479395426", 2024-04-24T23:04:30.335 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:30.335 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:30.336 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:30.336 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:30.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.336 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:04:30.336 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:30.336 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:30.336 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:30.336 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:30.336 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:30.336 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:30.336 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:30.336 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:30.336 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:30.337 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:30.337 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:30.337 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:30.337 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:30.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.337 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:30.337 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2024-04-24T23:04:30.337 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:30.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.337 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:30.337 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2024-04-24T23:04:30.338 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:30.338 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:30.338 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:30.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.338 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:30.338 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:30.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.338 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:30.338 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2024-04-24T23:04:30.338 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:30.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.338 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:30.339 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2024-04-24T23:04:30.339 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:30.339 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:30.339 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:30.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.339 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:30.339 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:30.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.339 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:30.339 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2024-04-24T23:04:30.339 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:30.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.339 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:30.340 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2024-04-24T23:04:30.340 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:30.340 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:30.340 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:30.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.340 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:30.340 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:30.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.340 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:30.340 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2024-04-24T23:04:30.340 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:30.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.341 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:30.341 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2024-04-24T23:04:30.341 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:30.341 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:30.341 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:30.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.341 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/1340633718", 2024-04-24T23:04:30.341 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/1340633718", 2024-04-24T23:04:30.341 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/1340633718", 2024-04-24T23:04:30.341 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/1340633718", 2024-04-24T23:04:30.341 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:30.341 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:30.342 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:30.342 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:30.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.342 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:04:30.342 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:30.342 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:30.342 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:30.342 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:30.342 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:30.342 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:30.342 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:30.342 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:30.342 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:30.343 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:30.343 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:30.343 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:30.343 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:30.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.343 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:30.343 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2024-04-24T23:04:30.343 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:30.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.343 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:30.343 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2024-04-24T23:04:30.344 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:30.344 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:30.344 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:30.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.344 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:30.344 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:30.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.344 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:30.344 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2024-04-24T23:04:30.345 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:30.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.345 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:30.345 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2024-04-24T23:04:30.345 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:30.345 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:30.345 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:30.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.345 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:30.345 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:30.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.345 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:30.345 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2024-04-24T23:04:30.346 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:30.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.346 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:30.346 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2024-04-24T23:04:30.346 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:30.346 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:30.347 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:30.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.347 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:30.347 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:30.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.347 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:30.347 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2024-04-24T23:04:30.347 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:30.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.347 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:30.347 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2024-04-24T23:04:30.347 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:30.348 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:30.348 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:30.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.348 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/3405899334", 2024-04-24T23:04:30.348 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6820/3405899334", 2024-04-24T23:04:30.348 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6828/3405899334", 2024-04-24T23:04:30.348 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6824/3405899334", 2024-04-24T23:04:30.348 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:30.348 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:30.348 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:30.348 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:30.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.348 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:04:30.349 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:30.349 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:30.349 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:30.349 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:30.349 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:30.349 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:30.349 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:30.349 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:30.349 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:30.349 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:30.349 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:30.349 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:30.350 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:30.350 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.350 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:30.350 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2024-04-24T23:04:30.350 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:30.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.350 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.350 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:30.350 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2024-04-24T23:04:30.350 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:30.350 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:30.350 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:30.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.351 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:30.351 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:30.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.351 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:30.351 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2024-04-24T23:04:30.351 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:30.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.351 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:30.351 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2024-04-24T23:04:30.351 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:30.351 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:30.351 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:30.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.352 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:30.352 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:30.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.352 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:30.352 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2024-04-24T23:04:30.352 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:30.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.352 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:30.353 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2024-04-24T23:04:30.353 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:30.353 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:30.353 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:30.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.353 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:30.353 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:30.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.353 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:30.353 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2024-04-24T23:04:30.353 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:30.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.354 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:30.354 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2024-04-24T23:04:30.354 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:30.354 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:30.354 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:30.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.354 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6821/3381057190", 2024-04-24T23:04:30.354 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6825/3381057190", 2024-04-24T23:04:30.354 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/3381057190", 2024-04-24T23:04:30.354 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/3381057190", 2024-04-24T23:04:30.355 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:30.355 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:30.355 INFO:teuthology.orchestra.run.smithi032.stdout: "full", 2024-04-24T23:04:30.355 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:30.355 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:30.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.355 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.355 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:04:30.355 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:30.356 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:30.356 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:30.356 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:30.356 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:30.356 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:30.356 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:30.356 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:30.356 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:30.356 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:30.356 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:30.356 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:30.356 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:30.356 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.356 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:30.357 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6800", 2024-04-24T23:04:30.357 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:30.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.357 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:30.357 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6801", 2024-04-24T23:04:30.357 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:30.357 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:30.357 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:30.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.358 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:30.358 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:30.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.358 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:30.358 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6802", 2024-04-24T23:04:30.358 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:30.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.358 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:30.358 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6803", 2024-04-24T23:04:30.358 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:30.359 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:30.359 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:30.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.359 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:30.359 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:30.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.359 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:30.359 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6806", 2024-04-24T23:04:30.359 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:30.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.360 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:30.360 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6807", 2024-04-24T23:04:30.360 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:30.360 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:30.360 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:30.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.360 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:30.360 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:30.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.361 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:30.361 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6804", 2024-04-24T23:04:30.361 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:30.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.361 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.361 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:30.361 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6805", 2024-04-24T23:04:30.361 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:30.361 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:30.361 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:30.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.361 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6801/4010553979", 2024-04-24T23:04:30.362 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6803/4010553979", 2024-04-24T23:04:30.362 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6807/4010553979", 2024-04-24T23:04:30.362 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6805/4010553979", 2024-04-24T23:04:30.362 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:30.362 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:30.362 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:30.362 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:30.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.363 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:04:30.363 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:30.363 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:30.363 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:30.363 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:30.363 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:30.363 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:30.363 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:30.363 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:30.364 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:30.364 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:30.364 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:30.364 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:30.364 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:30.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.364 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:30.364 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6808", 2024-04-24T23:04:30.364 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:30.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.365 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:30.365 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6809", 2024-04-24T23:04:30.365 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:30.365 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:30.365 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:30.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.365 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:30.366 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:30.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.366 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:30.366 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6810", 2024-04-24T23:04:30.366 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:30.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.366 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:30.366 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6811", 2024-04-24T23:04:30.366 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:30.366 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:30.366 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:30.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.367 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:30.367 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:30.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.367 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:30.367 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6814", 2024-04-24T23:04:30.367 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:30.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.367 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:30.367 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6815", 2024-04-24T23:04:30.367 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:30.368 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:30.368 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:30.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.368 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:30.368 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:30.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.368 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:30.368 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6812", 2024-04-24T23:04:30.368 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:30.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.369 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:30.369 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6813", 2024-04-24T23:04:30.369 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:30.369 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:30.369 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:30.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.369 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6809/3794084091", 2024-04-24T23:04:30.369 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6811/3794084091", 2024-04-24T23:04:30.369 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6815/3794084091", 2024-04-24T23:04:30.369 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6813/3794084091", 2024-04-24T23:04:30.369 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:30.369 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:30.370 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:30.370 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:30.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.370 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.370 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:04:30.370 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:30.370 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:30.370 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:30.370 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:30.370 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:30.370 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:30.370 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:30.370 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:30.371 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:30.371 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:30.371 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:30.371 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:30.371 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:30.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.371 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:30.371 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6823", 2024-04-24T23:04:30.371 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:30.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.371 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:30.371 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6825", 2024-04-24T23:04:30.371 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:30.372 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:30.372 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:30.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.372 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:30.372 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:30.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.372 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:30.372 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6826", 2024-04-24T23:04:30.372 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:30.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.372 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:30.372 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6827", 2024-04-24T23:04:30.373 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:30.373 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:30.373 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:30.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.373 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:30.373 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:30.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.373 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:30.373 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6830", 2024-04-24T23:04:30.373 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:30.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.373 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:30.374 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6831", 2024-04-24T23:04:30.374 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:30.374 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:30.374 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:30.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.374 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:30.374 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:30.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.374 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:30.374 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6828", 2024-04-24T23:04:30.374 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:30.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.375 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:30.375 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6829", 2024-04-24T23:04:30.375 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:30.375 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:30.375 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:30.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.375 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6825/1348640258", 2024-04-24T23:04:30.375 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6827/1348640258", 2024-04-24T23:04:30.375 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6831/1348640258", 2024-04-24T23:04:30.375 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6829/1348640258", 2024-04-24T23:04:30.375 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:30.375 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:30.375 INFO:teuthology.orchestra.run.smithi032.stdout: "full", 2024-04-24T23:04:30.375 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:30.376 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:30.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.376 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:04:30.376 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:30.376 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:30.376 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:30.376 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:30.376 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:30.376 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:30.376 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:30.376 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:30.376 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:30.377 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:30.377 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:30.377 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:30.377 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:30.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.377 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:30.377 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6816", 2024-04-24T23:04:30.377 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:30.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.377 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:30.377 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6817", 2024-04-24T23:04:30.377 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:30.377 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:30.378 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:30.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.378 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:30.378 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:30.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.378 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:30.378 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6818", 2024-04-24T23:04:30.378 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:30.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.378 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:30.378 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6819", 2024-04-24T23:04:30.379 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:30.379 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:30.379 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:30.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.379 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:30.379 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:30.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.379 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:30.379 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6822", 2024-04-24T23:04:30.379 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:30.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.379 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:30.380 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6824", 2024-04-24T23:04:30.380 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:30.380 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:30.380 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:30.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.380 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:30.380 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:30.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.380 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:30.380 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6820", 2024-04-24T23:04:30.380 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:30.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.381 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:30.381 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6821", 2024-04-24T23:04:30.381 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:30.381 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:30.381 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:30.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.381 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6817/3250005122", 2024-04-24T23:04:30.381 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6819/3250005122", 2024-04-24T23:04:30.381 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6824/3250005122", 2024-04-24T23:04:30.382 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6821/3250005122", 2024-04-24T23:04:30.382 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:30.382 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:30.382 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:30.382 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:30.382 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:30.382 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:04:30.382 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2024-04-24T23:04:30.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.382 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:04:30.382 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:30.382 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:30.383 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:30.383 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:30.383 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:30.383 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.156184+0000", 2024-04-24T23:04:30.383 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:30.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.383 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:04:30.383 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:30.383 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:30.383 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:30.383 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:30.383 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:30.383 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240774+0000", 2024-04-24T23:04:30.384 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:30.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.384 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:04:30.384 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:30.384 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:30.384 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:30.384 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:30.384 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:30.384 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240949+0000", 2024-04-24T23:04:30.384 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:30.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.385 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:04:30.385 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:30.385 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:30.385 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:30.385 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:30.385 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:30.385 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.171672+0000", 2024-04-24T23:04:30.385 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:30.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.385 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:04:30.385 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:30.385 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:30.386 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:30.386 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:30.386 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:30.386 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.220379+0000", 2024-04-24T23:04:30.386 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:30.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.386 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:04:30.386 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:30.386 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:30.386 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:30.386 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:30.386 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:30.387 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.204963+0000", 2024-04-24T23:04:30.387 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:30.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.387 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:04:30.387 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:30.387 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:30.387 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:30.387 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:30.387 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:30.387 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.239748+0000", 2024-04-24T23:04:30.387 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:30.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:30.388 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:04:30.388 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:30.388 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:30.388 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:30.388 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:30.388 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:30.388 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.154104+0000", 2024-04-24T23:04:30.388 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:30.388 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:30.388 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:04:30.388 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2024-04-24T23:04:30.388 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2024-04-24T23:04:30.389 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_primaries": [], 2024-04-24T23:04:30.389 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2024-04-24T23:04:30.389 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2024-04-24T23:04:30.389 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklist": {}, 2024-04-24T23:04:30.389 INFO:teuthology.orchestra.run.smithi032.stdout: "range_blocklist": {}, 2024-04-24T23:04:30.389 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2024-04-24T23:04:30.389 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2024-04-24T23:04:30.389 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2024-04-24T23:04:30.389 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2024-04-24T23:04:30.389 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2024-04-24T23:04:30.389 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2024-04-24T23:04:30.389 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2024-04-24T23:04:30.390 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:30.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:30.390 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2024-04-24T23:04:30.390 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2024-04-24T23:04:30.390 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2024-04-24T23:04:30.390 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2024-04-24T23:04:30.390 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {}, 2024-04-24T23:04:30.391 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode": { 2024-04-24T23:04:30.391 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_enabled": false, 2024-04-24T23:04:30.391 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_bucket_count": 0, 2024-04-24T23:04:30.391 INFO:teuthology.orchestra.run.smithi032.stdout: "degraded_stretch_mode": 0, 2024-04-24T23:04:30.391 INFO:teuthology.orchestra.run.smithi032.stdout: "recovering_stretch_mode": 0, 2024-04-24T23:04:30.391 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_bucket": 0 2024-04-24T23:04:30.391 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:30.391 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:04:30.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.303+0000 7f37e1e95640 1 -- 172.21.15.32:0/4142357588 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f37c806be30 msgr2=0x7f37c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:30.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.303+0000 7f37e1e95640 1 --2- 172.21.15.32:0/4142357588 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f37c806be30 0x7f37c806e2f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f37cc006fd0 tx=0x7f37cc008040 comp rx=0 tx=0).stop 2024-04-24T23:04:30.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.303+0000 7f37e1e95640 1 -- 172.21.15.32:0/4142357588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37dc15b380 msgr2=0x7f37dc143cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:30.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.303+0000 7f37e1e95640 1 --2- 172.21.15.32:0/4142357588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37dc15b380 0x7f37dc143cc0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f37d00098c0 tx=0x7f37d00046c0 comp rx=0 tx=0).stop 2024-04-24T23:04:30.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.303+0000 7f37e1e95640 1 -- 172.21.15.32:0/4142357588 shutdown_connections 2024-04-24T23:04:30.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.303+0000 7f37e1e95640 1 --2- 172.21.15.32:0/4142357588 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f37c806be30 0x7f37c806e2f0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:30.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.303+0000 7f37e1e95640 1 --2- 172.21.15.32:0/4142357588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37dc15b380 0x7f37dc143cc0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:30.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.303+0000 7f37e1e95640 1 --2- 172.21.15.32:0/4142357588 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f37dc15cb00 0x7f37dc155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:30.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.303+0000 7f37e1e95640 1 --2- 172.21.15.32:0/4142357588 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37dc0ce430 0x7f37dc1e6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:30.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.303+0000 7f37e1e95640 1 -- 172.21.15.32:0/4142357588 >> 172.21.15.32:0/4142357588 conn(0x7f37dc06b110 msgr2=0x7f37dc0cf1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:30.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.303+0000 7f37e1e95640 1 -- 172.21.15.32:0/4142357588 shutdown_connections 2024-04-24T23:04:30.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:30.303+0000 7f37e1e95640 1 -- 172.21.15.32:0/4142357588 wait complete. 2024-04-24T23:04:30.395 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-04-24T23:04:35.396 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-24T23:04:35.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.508+0000 7f618164d640 1 Processor -- start 2024-04-24T23:04:35.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.508+0000 7f618164d640 1 -- start start 2024-04-24T23:04:35.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.509+0000 7f618164d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f617c15c8a0 0x7f617c0ae140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:35.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.509+0000 7f618164d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f617c074920 0x7f617c06c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:35.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.509+0000 7f618164d640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f617c074f00 con 0x7f617c0ce7a0 2024-04-24T23:04:35.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.509+0000 7f618164d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f617c06ca70 con 0x7f617c074920 2024-04-24T23:04:35.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.509+0000 7f618164d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f617c0b0de0 con 0x7f617c15c8a0 2024-04-24T23:04:35.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.509+0000 7f617b7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f617c074920 0x7f617c06c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:35.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.509+0000 7f617b7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f617c074920 0x7f617c06c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43940/0 (socket says 172.21.15.32:43940) 2024-04-24T23:04:35.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.509+0000 7f617b7fe640 1 -- 172.21.15.32:0/2348350454 learned_addr learned my addr 172.21.15.32:0/2348350454 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:35.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.509+0000 7f617affd640 1 --2- 172.21.15.32:0/2348350454 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f617c15c8a0 0x7f617c0ae140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:35.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.509+0000 7f617b7fe640 1 -- 172.21.15.32:0/2348350454 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f617c15c8a0 msgr2=0x7f617c0ae140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:35.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.510+0000 7f617b7fe640 1 --2- 172.21.15.32:0/2348350454 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f617c15c8a0 0x7f617c0ae140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:35.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.510+0000 7f617b7fe640 1 -- 172.21.15.32:0/2348350454 >> v1:172.21.15.121:6789/0 conn(0x7f617c0ce7a0 legacy=0x7f617c074210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:35.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.510+0000 7f617b7fe640 1 -- 172.21.15.32:0/2348350454 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f617c0b10c0 con 0x7f617c074920 2024-04-24T23:04:35.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.510+0000 7f617b7fe640 1 --2- 172.21.15.32:0/2348350454 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f617c074920 0x7f617c06c390 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f6174009e50 tx=0x7f617402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8feb2097eba7931b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:35.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.510+0000 7f6179ffb640 1 -- 172.21.15.32:0/2348350454 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6174036070 con 0x7f617c074920 2024-04-24T23:04:35.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.510+0000 7f6179ffb640 1 -- 172.21.15.32:0/2348350454 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6174031600 con 0x7f617c074920 2024-04-24T23:04:35.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.510+0000 7f6179ffb640 1 -- 172.21.15.32:0/2348350454 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61740319d0 con 0x7f617c074920 2024-04-24T23:04:35.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.511+0000 7f618164d640 1 -- 172.21.15.32:0/2348350454 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f617c074920 msgr2=0x7f617c06c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:35.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.511+0000 7f618164d640 1 --2- 172.21.15.32:0/2348350454 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f617c074920 0x7f617c06c390 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f6174009e50 tx=0x7f617402a680 comp rx=0 tx=0).stop 2024-04-24T23:04:35.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.511+0000 7f618164d640 1 -- 172.21.15.32:0/2348350454 shutdown_connections 2024-04-24T23:04:35.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.511+0000 7f618164d640 1 --2- 172.21.15.32:0/2348350454 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f617c074920 0x7f617c06c390 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:35.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.511+0000 7f618164d640 1 --2- 172.21.15.32:0/2348350454 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f617c15c8a0 0x7f617c0ae140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:35.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.511+0000 7f618164d640 1 -- 172.21.15.32:0/2348350454 >> 172.21.15.32:0/2348350454 conn(0x7f617c0b2e60 msgr2=0x7f617c0b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:35.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.511+0000 7f618164d640 1 -- 172.21.15.32:0/2348350454 shutdown_connections 2024-04-24T23:04:35.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.511+0000 7f618164d640 1 -- 172.21.15.32:0/2348350454 wait complete. 2024-04-24T23:04:35.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.512+0000 7f618164d640 1 Processor -- start 2024-04-24T23:04:35.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.513+0000 7f618164d640 1 -- start start 2024-04-24T23:04:35.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.513+0000 7f618164d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f617c074920 0x7f617c0d08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:35.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.513+0000 7f618164d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f617c0ce7a0 0x7f617c0c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:35.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.513+0000 7f618164d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f617c15c8a0 0x7f617c0b6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:35.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.513+0000 7f618164d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f617c15cc80 con 0x7f617c15c8a0 2024-04-24T23:04:35.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.513+0000 7f618164d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f617c0b1680 con 0x7f617c074920 2024-04-24T23:04:35.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.513+0000 7f618164d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f617c0d1eb0 con 0x7f617c0ce7a0 2024-04-24T23:04:35.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.513+0000 7f617b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f617c15c8a0 0x7f617c0b6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:35.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.513+0000 7f617b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f617c15c8a0 0x7f617c0b6330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43952/0 (socket says 172.21.15.32:43952) 2024-04-24T23:04:35.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.513+0000 7f617b7fe640 1 -- 172.21.15.32:0/2479713531 learned_addr learned my addr 172.21.15.32:0/2479713531 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:35.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.514+0000 7f617affd640 1 --2- 172.21.15.32:0/2479713531 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f617c074920 0x7f617c0d08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:35.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.514+0000 7f617a7fc640 1 --2- 172.21.15.32:0/2479713531 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f617c0ce7a0 0x7f617c0c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:35.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.514+0000 7f617b7fe640 1 -- 172.21.15.32:0/2479713531 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f617c0ce7a0 msgr2=0x7f617c0c83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:35.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.514+0000 7f617b7fe640 1 --2- 172.21.15.32:0/2479713531 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f617c0ce7a0 0x7f617c0c83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:35.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.514+0000 7f617b7fe640 1 -- 172.21.15.32:0/2479713531 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f617c074920 msgr2=0x7f617c0d08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:35.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.514+0000 7f617b7fe640 1 --2- 172.21.15.32:0/2479713531 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f617c074920 0x7f617c0d08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:35.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.514+0000 7f617b7fe640 1 -- 172.21.15.32:0/2479713531 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f617c1e6250 con 0x7f617c15c8a0 2024-04-24T23:04:35.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.514+0000 7f617affd640 1 --2- 172.21.15.32:0/2479713531 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f617c074920 0x7f617c0d08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:04:35.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.514+0000 7f617a7fc640 1 --2- 172.21.15.32:0/2479713531 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f617c0ce7a0 0x7f617c0c83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:04:35.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.514+0000 7f617b7fe640 1 --2- 172.21.15.32:0/2479713531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f617c15c8a0 0x7f617c0b6330 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f6174009e50 tx=0x7f6174004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:35.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.514+0000 7f61637fe640 1 -- 172.21.15.32:0/2479713531 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6174036050 con 0x7f617c15c8a0 2024-04-24T23:04:35.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.514+0000 7f61637fe640 1 -- 172.21.15.32:0/2479713531 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6174042070 con 0x7f617c15c8a0 2024-04-24T23:04:35.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.514+0000 7f618164d640 1 -- 172.21.15.32:0/2479713531 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f617c1e6530 con 0x7f617c15c8a0 2024-04-24T23:04:35.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.514+0000 7f61637fe640 1 -- 172.21.15.32:0/2479713531 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6174009510 con 0x7f617c15c8a0 2024-04-24T23:04:35.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.514+0000 7f618164d640 1 -- 172.21.15.32:0/2479713531 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f617c1575f0 con 0x7f617c15c8a0 2024-04-24T23:04:35.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.515+0000 7f618164d640 1 -- 172.21.15.32:0/2479713531 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f617c068430 con 0x7f617c15c8a0 2024-04-24T23:04:35.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.516+0000 7f61637fe640 1 -- 172.21.15.32:0/2479713531 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6174007bf0 con 0x7f617c15c8a0 2024-04-24T23:04:35.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.516+0000 7f61637fe640 1 --2- 172.21.15.32:0/2479713531 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f616806be30 0x7f616806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:35.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.516+0000 7f61637fe640 1 -- 172.21.15.32:0/2479713531 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7f61740abc30 con 0x7f617c15c8a0 2024-04-24T23:04:35.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.517+0000 7f617affd640 1 --2- 172.21.15.32:0/2479713531 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f616806be30 0x7f616806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:35.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.517+0000 7f617affd640 1 --2- 172.21.15.32:0/2479713531 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f616806be30 0x7f616806e2f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f616c009a20 tx=0x7f616c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:35.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.521+0000 7f61637fe640 1 -- 172.21.15.32:0/2479713531 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f617407a5e0 con 0x7f617c15c8a0 2024-04-24T23:04:35.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.705+0000 7f618164d640 1 -- 172.21.15.32:0/2479713531 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f617c0742b0 con 0x7f617c15c8a0 2024-04-24T23:04:35.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.706+0000 7f61637fe640 1 -- 172.21.15.32:0/2479713531 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v113) ==== 81+0+34185 (secure 0 0 0) 0x7f6174079ee0 con 0x7f617c15c8a0 2024-04-24T23:04:35.708 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:04:35.708 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:04:35.709 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 113, 2024-04-24T23:04:35.709 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", 2024-04-24T23:04:35.709 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2024-04-24T22:58:46.964851+0000", 2024-04-24T23:04:35.709 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2024-04-24T23:04:31.618095+0000", 2024-04-24T23:04:35.709 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2024-04-24T22:58:52.048787+0000", 2024-04-24T23:04:35.709 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2024-04-24T22:58:49.045609+0000", 2024-04-24T23:04:35.709 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-24T23:04:35.709 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2024-04-24T23:04:35.709 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2024-04-24T23:04:35.710 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2024-04-24T23:04:35.710 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2024-04-24T23:04:35.710 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2024-04-24T23:04:35.710 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2024-04-24T23:04:35.710 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:04:35.710 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 4, 2024-04-24T23:04:35.710 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.69999998807907104, 2024-04-24T23:04:35.710 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-24T23:04:35.710 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-24T23:04:35.710 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2024-04-24T23:04:35.710 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 13, 2024-04-24T23:04:35.710 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2024-04-24T23:04:35.711 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "luminous", 2024-04-24T23:04:35.711 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "jewel", 2024-04-24T23:04:35.711 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "squid", 2024-04-24T23:04:35.711 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_crimson": false, 2024-04-24T23:04:35.711 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2024-04-24T23:04:35.711 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.711 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2024-04-24T23:04:35.711 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": ".mgr", 2024-04-24T23:04:35.712 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T22:58:53.171408+0000", 2024-04-24T23:04:35.712 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:04:35.712 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:04:35.712 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:04:35.712 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:04:35.712 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:04:35.712 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:04:35.712 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:04:35.713 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:04:35.713 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:04:35.713 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:04:35.713 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:04:35.713 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:04:35.713 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2024-04-24T23:04:35.713 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2024-04-24T23:04:35.713 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2024-04-24T23:04:35.713 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2024-04-24T23:04:35.714 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2024-04-24T23:04:35.714 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:04:35.714 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:04:35.714 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:04:35.714 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:04:35.714 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:04:35.714 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:04:35.714 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:04:35.714 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.714 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "113", 2024-04-24T23:04:35.715 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:04:35.715 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:04:35.715 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:04:35.715 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:04:35.715 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:04:35.715 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:04:35.715 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:04:35.715 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:04:35.715 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:04:35.715 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:04:35.715 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:04:35.716 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:04:35.716 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:04:35.716 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:04:35.716 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:04:35.716 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:04:35.716 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:04:35.716 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:04:35.716 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:04:35.716 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:04:35.716 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:04:35.716 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:04:35.716 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:04:35.716 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:04:35.716 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:04:35.716 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:04:35.717 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.717 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:04:35.717 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:04:35.717 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:04:35.717 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:04:35.717 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:04:35.717 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:04:35.717 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:04:35.717 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:04:35.717 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:04:35.717 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:04:35.717 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:04:35.717 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:04:35.718 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_max": 32, 2024-04-24T23:04:35.718 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2024-04-24T23:04:35.718 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.718 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:04:35.718 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": {} 2024-04-24T23:04:35.718 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.718 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:04:35.718 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:04:35.719 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 8, 2024-04-24T23:04:35.719 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 8, 2024-04-24T23:04:35.719 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.25, 2024-04-24T23:04:35.719 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-04-24T23:04:35.719 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-04-24T23:04:35.719 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:04:35.719 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:04:35.719 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:04:35.719 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:35.719 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.719 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.719 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 12, 2024-04-24T23:04:35.719 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2024-04-24T23:04:35.719 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:03:59.079157+0000", 2024-04-24T23:04:35.719 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:04:35.720 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:04:35.720 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:04:35.720 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:04:35.720 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:04:35.720 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:04:35.720 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:04:35.720 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:04:35.720 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:04:35.720 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:04:35.720 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:04:35.720 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:04:35.721 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 32, 2024-04-24T23:04:35.721 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 32, 2024-04-24T23:04:35.721 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 32, 2024-04-24T23:04:35.721 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 32, 2024-04-24T23:04:35.721 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 32, 2024-04-24T23:04:35.721 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:04:35.721 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:04:35.721 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:04:35.721 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:04:35.721 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:04:35.721 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:04:35.721 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:04:35.721 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.722 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "113", 2024-04-24T23:04:35.722 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:04:35.722 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:04:35.722 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:04:35.722 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:04:35.722 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:04:35.722 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:04:35.722 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:04:35.722 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:04:35.722 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:04:35.723 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:04:35.723 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:04:35.723 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:04:35.723 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:04:35.723 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:04:35.723 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:04:35.723 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:04:35.723 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:04:35.723 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:04:35.723 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:04:35.723 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:04:35.723 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:04:35.723 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:04:35.723 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:04:35.724 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:04:35.724 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:04:35.724 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:04:35.724 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.724 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:04:35.724 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:04:35.724 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:04:35.724 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:04:35.724 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:04:35.724 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:04:35.724 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:04:35.725 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:04:35.725 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:04:35.725 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:04:35.725 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:04:35.725 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:04:35.725 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2024-04-24T23:04:35.725 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2024-04-24T23:04:35.725 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2024-04-24T23:04:35.725 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.725 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:04:35.726 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:04:35.726 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2024-04-24T23:04:35.726 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:35.726 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.726 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:04:35.726 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:04:35.726 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.75, 2024-04-24T23:04:35.726 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.75, 2024-04-24T23:04:35.726 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:04:35.726 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.75, 2024-04-24T23:04:35.727 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.75, 2024-04-24T23:04:35.727 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:04:35.727 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:04:35.727 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:04:35.727 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:35.727 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.727 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.727 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 13, 2024-04-24T23:04:35.727 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2024-04-24T23:04:35.727 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:03:59.765179+0000", 2024-04-24T23:04:35.728 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:04:35.728 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:04:35.728 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:04:35.729 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:04:35.729 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:04:35.729 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:04:35.729 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:04:35.730 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:04:35.730 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:04:35.730 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:04:35.730 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:04:35.730 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:04:35.730 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 64, 2024-04-24T23:04:35.730 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 64, 2024-04-24T23:04:35.730 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 64, 2024-04-24T23:04:35.730 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 64, 2024-04-24T23:04:35.730 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 64, 2024-04-24T23:04:35.730 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:04:35.730 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:04:35.730 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:04:35.730 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:04:35.731 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:04:35.731 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:04:35.731 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:04:35.731 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.731 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "113", 2024-04-24T23:04:35.731 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:04:35.731 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:04:35.731 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:04:35.731 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:04:35.731 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:04:35.731 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:04:35.731 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:04:35.731 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:04:35.731 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:04:35.732 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:04:35.732 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:04:35.732 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:04:35.732 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:04:35.732 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:04:35.732 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:04:35.732 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:04:35.732 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:04:35.732 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:04:35.732 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:04:35.732 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:04:35.732 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:04:35.732 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:04:35.732 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:04:35.733 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:04:35.733 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:04:35.733 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:04:35.733 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.733 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:04:35.733 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:04:35.733 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:04:35.733 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:04:35.733 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:04:35.733 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:04:35.733 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:04:35.733 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:04:35.733 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:04:35.734 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:04:35.734 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:04:35.734 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:04:35.734 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 64, 2024-04-24T23:04:35.734 INFO:teuthology.orchestra.run.smithi032.stdout: "target_size_ratio": 0.89999997615814209 2024-04-24T23:04:35.734 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.734 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:04:35.734 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:04:35.734 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2024-04-24T23:04:35.734 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:35.734 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.734 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:04:35.734 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:04:35.734 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.3799999952316284, 2024-04-24T23:04:35.735 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.3799999952316284, 2024-04-24T23:04:35.735 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:04:35.735 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.3799999952316284, 2024-04-24T23:04:35.735 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.3799999952316284, 2024-04-24T23:04:35.735 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:04:35.735 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:04:35.735 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:04:35.735 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:35.735 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:35.735 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:04:35.735 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-04-24T23:04:35.735 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.735 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:04:35.735 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:35.736 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:35.736 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:35.736 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:35.736 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:35.736 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:35.736 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:35.736 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:35.736 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:35.736 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:35.736 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:35.736 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:35.736 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:35.736 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.737 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:35.737 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2024-04-24T23:04:35.737 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:35.737 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.737 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.737 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:35.737 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2024-04-24T23:04:35.737 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:35.737 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:35.737 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:35.737 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.737 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:35.737 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:35.737 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.738 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:35.738 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2024-04-24T23:04:35.738 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:35.738 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.738 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.738 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:35.738 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2024-04-24T23:04:35.738 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:35.738 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:35.738 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:35.738 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.738 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:35.738 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:35.738 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.739 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:35.739 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2024-04-24T23:04:35.739 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:35.739 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.739 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.739 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:35.739 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2024-04-24T23:04:35.739 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:35.739 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:35.739 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:35.739 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.739 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:35.739 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:35.739 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.740 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:35.740 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2024-04-24T23:04:35.740 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:35.740 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.740 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.740 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:35.740 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2024-04-24T23:04:35.740 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:35.740 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:35.740 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:35.740 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.740 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/1479395426", 2024-04-24T23:04:35.740 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/1479395426", 2024-04-24T23:04:35.741 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/1479395426", 2024-04-24T23:04:35.741 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/1479395426", 2024-04-24T23:04:35.741 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:35.741 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:35.741 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:35.741 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:35.741 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.741 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.741 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:04:35.741 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:35.741 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:35.741 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:35.741 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:35.741 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:35.742 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:35.742 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:35.742 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:35.742 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:35.742 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:35.742 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:35.742 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:35.742 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:35.742 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.742 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:35.743 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2024-04-24T23:04:35.743 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:35.743 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.743 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.743 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:35.743 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2024-04-24T23:04:35.743 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:35.743 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:35.743 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:35.743 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.743 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:35.743 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:35.743 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.743 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:35.744 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2024-04-24T23:04:35.744 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:35.744 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.744 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.744 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:35.744 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2024-04-24T23:04:35.744 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:35.744 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:35.744 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:35.744 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.744 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:35.745 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:35.745 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.745 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:35.745 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2024-04-24T23:04:35.745 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:35.745 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.745 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.745 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:35.745 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2024-04-24T23:04:35.745 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:35.745 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:35.745 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:35.745 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.746 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:35.746 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:35.746 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.746 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:35.746 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2024-04-24T23:04:35.746 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:35.746 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.746 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.746 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:35.746 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2024-04-24T23:04:35.746 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:35.747 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:35.747 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:35.747 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.747 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/1340633718", 2024-04-24T23:04:35.747 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/1340633718", 2024-04-24T23:04:35.747 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/1340633718", 2024-04-24T23:04:35.747 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/1340633718", 2024-04-24T23:04:35.747 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:35.747 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:35.747 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:35.747 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:35.747 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.748 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.748 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:04:35.748 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:35.748 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:35.748 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:35.748 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:35.748 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:35.748 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:35.748 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:35.748 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:35.748 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:35.748 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:35.749 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:35.749 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:35.749 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:35.749 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.749 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:35.749 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2024-04-24T23:04:35.749 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:35.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.749 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.749 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:35.749 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2024-04-24T23:04:35.749 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:35.749 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:35.749 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:35.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.750 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:35.750 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:35.750 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.750 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:35.750 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2024-04-24T23:04:35.750 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:35.750 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.750 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.750 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:35.750 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2024-04-24T23:04:35.750 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:35.750 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:35.750 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:35.750 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.751 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:35.751 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:35.751 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.751 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:35.751 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2024-04-24T23:04:35.751 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:35.751 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.751 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.751 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:35.751 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2024-04-24T23:04:35.751 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:35.751 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:35.751 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:35.751 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.751 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:35.752 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:35.752 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.752 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:35.752 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2024-04-24T23:04:35.752 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:35.752 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.752 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.752 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:35.752 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2024-04-24T23:04:35.752 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:35.752 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:35.752 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:35.752 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.752 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/3405899334", 2024-04-24T23:04:35.752 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6820/3405899334", 2024-04-24T23:04:35.753 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6828/3405899334", 2024-04-24T23:04:35.753 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6824/3405899334", 2024-04-24T23:04:35.753 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:35.753 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:35.753 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:35.753 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:35.753 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.753 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.753 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:04:35.753 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:35.753 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:35.753 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:35.753 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:35.753 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:35.754 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:35.754 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:35.754 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:35.754 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:35.754 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:35.754 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:35.754 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:35.754 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:35.755 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.755 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:35.755 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2024-04-24T23:04:35.755 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:35.755 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.755 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.755 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:35.755 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2024-04-24T23:04:35.755 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:35.755 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:35.756 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:35.756 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.756 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:35.756 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:35.756 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.756 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:35.756 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2024-04-24T23:04:35.756 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:35.756 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.757 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.757 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:35.757 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2024-04-24T23:04:35.757 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:35.757 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:35.757 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:35.757 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.757 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:35.757 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:35.757 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.758 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:35.758 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2024-04-24T23:04:35.758 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:35.758 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.758 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.758 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:35.758 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2024-04-24T23:04:35.758 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:35.758 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:35.758 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:35.759 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.759 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:35.759 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:35.759 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.759 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:35.759 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2024-04-24T23:04:35.759 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:35.759 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.759 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.759 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:35.759 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2024-04-24T23:04:35.759 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:35.760 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:35.760 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:35.760 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.760 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6821/3381057190", 2024-04-24T23:04:35.760 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6825/3381057190", 2024-04-24T23:04:35.760 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/3381057190", 2024-04-24T23:04:35.760 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/3381057190", 2024-04-24T23:04:35.760 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:35.760 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:35.760 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:35.760 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:35.761 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.761 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.761 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:04:35.761 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:35.761 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:35.761 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:35.761 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:35.761 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:35.761 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:35.761 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:35.761 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:35.761 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:35.761 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:35.761 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:35.762 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:35.762 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:35.762 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.762 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:35.762 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6800", 2024-04-24T23:04:35.762 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:35.762 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.762 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.762 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:35.762 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6801", 2024-04-24T23:04:35.762 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:35.763 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:35.763 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:35.763 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.763 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:35.763 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:35.763 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.763 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:35.763 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6802", 2024-04-24T23:04:35.763 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:35.763 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.763 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.764 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:35.764 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6803", 2024-04-24T23:04:35.764 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:35.764 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:35.764 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:35.764 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.764 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:35.764 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:35.764 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.764 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:35.764 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6806", 2024-04-24T23:04:35.764 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:35.765 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.765 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.765 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:35.765 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6807", 2024-04-24T23:04:35.765 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:35.765 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:35.765 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:35.765 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.765 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:35.766 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:35.766 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.766 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:35.766 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6804", 2024-04-24T23:04:35.766 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:35.766 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.766 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.766 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:35.766 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6805", 2024-04-24T23:04:35.766 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:35.767 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:35.767 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:35.767 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.767 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6801/4010553979", 2024-04-24T23:04:35.767 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6803/4010553979", 2024-04-24T23:04:35.767 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6807/4010553979", 2024-04-24T23:04:35.767 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6805/4010553979", 2024-04-24T23:04:35.767 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:35.767 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:35.767 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:35.767 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:35.768 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.768 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.768 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:04:35.768 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:35.768 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:35.768 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:35.768 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:35.768 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:35.768 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:35.768 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:35.768 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:35.768 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:35.768 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:35.768 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:35.768 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:35.769 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:35.769 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.769 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:35.769 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6808", 2024-04-24T23:04:35.769 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:35.769 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.769 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.769 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:35.769 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6809", 2024-04-24T23:04:35.769 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:35.769 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:35.769 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:35.769 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.769 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:35.769 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:35.770 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.770 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:35.770 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6810", 2024-04-24T23:04:35.770 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:35.770 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.770 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.770 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:35.770 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6811", 2024-04-24T23:04:35.770 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:35.770 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:35.770 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:35.770 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.770 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:35.770 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:35.771 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.771 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:35.771 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6814", 2024-04-24T23:04:35.771 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:35.771 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.771 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.771 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:35.771 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6815", 2024-04-24T23:04:35.771 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:35.771 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:35.771 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:35.771 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.771 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:35.771 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:35.772 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.772 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:35.772 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6812", 2024-04-24T23:04:35.772 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:35.772 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.772 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.772 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:35.772 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6813", 2024-04-24T23:04:35.772 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:35.772 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:35.772 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:35.772 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.772 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6809/3794084091", 2024-04-24T23:04:35.772 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6811/3794084091", 2024-04-24T23:04:35.773 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6815/3794084091", 2024-04-24T23:04:35.773 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6813/3794084091", 2024-04-24T23:04:35.773 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:35.773 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:35.773 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:35.773 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:35.773 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.773 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.773 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:04:35.773 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:35.773 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:35.773 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:35.773 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:35.773 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:35.773 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:35.774 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:35.774 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:35.774 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:35.774 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:35.774 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:35.774 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:35.774 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:35.774 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.774 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:35.774 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6823", 2024-04-24T23:04:35.774 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:35.774 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.774 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.774 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:35.774 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6825", 2024-04-24T23:04:35.775 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:35.775 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:35.775 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:35.775 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.775 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:35.775 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:35.775 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.775 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:35.775 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6826", 2024-04-24T23:04:35.775 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:35.775 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.775 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.775 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:35.775 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6827", 2024-04-24T23:04:35.775 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:35.776 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:35.776 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:35.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.776 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:35.776 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:35.776 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.776 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:35.776 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6830", 2024-04-24T23:04:35.776 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:35.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.776 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.776 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:35.776 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6831", 2024-04-24T23:04:35.776 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:35.776 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:35.777 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:35.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.777 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:35.777 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:35.777 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.777 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:35.777 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6828", 2024-04-24T23:04:35.777 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:35.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.777 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.777 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:35.777 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6829", 2024-04-24T23:04:35.777 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:35.777 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:35.777 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:35.778 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.778 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6825/1348640258", 2024-04-24T23:04:35.778 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6827/1348640258", 2024-04-24T23:04:35.778 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6831/1348640258", 2024-04-24T23:04:35.778 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6829/1348640258", 2024-04-24T23:04:35.778 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:35.778 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:35.778 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:35.778 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:35.778 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.778 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.778 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:04:35.778 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:35.778 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:35.778 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:35.779 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:35.779 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:35.779 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:35.779 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:35.779 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:35.779 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:35.779 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:35.779 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:35.779 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:35.779 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:35.779 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.779 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:35.779 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6816", 2024-04-24T23:04:35.779 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:35.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.780 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.780 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:35.780 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6817", 2024-04-24T23:04:35.780 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:35.780 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:35.780 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:35.780 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.780 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:35.780 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:35.780 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.780 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:35.780 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6818", 2024-04-24T23:04:35.780 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:35.780 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.780 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.781 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:35.781 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6819", 2024-04-24T23:04:35.781 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:35.781 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:35.781 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:35.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.781 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:35.781 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:35.781 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.781 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:35.781 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6822", 2024-04-24T23:04:35.782 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:35.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.782 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.782 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:35.782 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6824", 2024-04-24T23:04:35.782 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:35.782 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:35.782 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:35.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.782 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:35.782 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:35.782 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.783 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:35.783 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6820", 2024-04-24T23:04:35.783 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:35.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.783 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.783 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:35.783 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6821", 2024-04-24T23:04:35.783 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:35.783 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:35.783 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:35.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.783 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6817/3250005122", 2024-04-24T23:04:35.783 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6819/3250005122", 2024-04-24T23:04:35.783 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6824/3250005122", 2024-04-24T23:04:35.784 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6821/3250005122", 2024-04-24T23:04:35.784 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:35.784 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:35.784 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:35.784 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:35.784 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:35.784 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:04:35.784 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2024-04-24T23:04:35.784 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.784 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:04:35.785 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:35.785 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:35.785 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:35.785 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:35.785 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:35.785 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.156184+0000", 2024-04-24T23:04:35.785 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:35.785 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.785 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.785 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:04:35.785 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:35.785 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:35.785 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:35.785 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:35.785 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:35.786 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240774+0000", 2024-04-24T23:04:35.786 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:35.786 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.786 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.786 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:04:35.786 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:35.786 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:35.786 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:35.786 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:35.786 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:35.787 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240949+0000", 2024-04-24T23:04:35.787 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:35.787 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.787 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.787 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:04:35.787 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:35.787 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:35.787 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:35.787 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:35.787 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:35.787 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.171672+0000", 2024-04-24T23:04:35.787 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:35.787 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.788 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.788 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:04:35.788 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:35.788 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:35.788 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:35.788 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:35.788 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:35.788 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.220379+0000", 2024-04-24T23:04:35.788 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:35.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.788 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.789 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:04:35.789 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:35.789 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:35.789 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:35.789 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:35.789 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:35.789 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.204963+0000", 2024-04-24T23:04:35.789 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:35.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.789 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.789 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:04:35.790 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:35.790 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:35.790 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:35.790 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:35.790 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:35.790 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.239748+0000", 2024-04-24T23:04:35.791 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:35.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.791 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:35.791 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:04:35.791 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:35.791 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:35.791 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:35.791 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:35.791 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:35.791 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.154104+0000", 2024-04-24T23:04:35.791 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:35.792 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:35.792 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:04:35.792 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2024-04-24T23:04:35.792 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2024-04-24T23:04:35.792 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_primaries": [], 2024-04-24T23:04:35.792 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2024-04-24T23:04:35.793 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2024-04-24T23:04:35.793 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklist": {}, 2024-04-24T23:04:35.793 INFO:teuthology.orchestra.run.smithi032.stdout: "range_blocklist": {}, 2024-04-24T23:04:35.793 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2024-04-24T23:04:35.793 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2024-04-24T23:04:35.793 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2024-04-24T23:04:35.793 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2024-04-24T23:04:35.793 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2024-04-24T23:04:35.793 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2024-04-24T23:04:35.793 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2024-04-24T23:04:35.794 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:35.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:35.794 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2024-04-24T23:04:35.794 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2024-04-24T23:04:35.794 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2024-04-24T23:04:35.795 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2024-04-24T23:04:35.795 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {}, 2024-04-24T23:04:35.795 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode": { 2024-04-24T23:04:35.795 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_enabled": false, 2024-04-24T23:04:35.795 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_bucket_count": 0, 2024-04-24T23:04:35.795 INFO:teuthology.orchestra.run.smithi032.stdout: "degraded_stretch_mode": 0, 2024-04-24T23:04:35.795 INFO:teuthology.orchestra.run.smithi032.stdout: "recovering_stretch_mode": 0, 2024-04-24T23:04:35.795 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_bucket": 0 2024-04-24T23:04:35.795 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:35.795 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:04:35.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.708+0000 7f618164d640 1 -- 172.21.15.32:0/2479713531 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f616806be30 msgr2=0x7f616806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:35.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.708+0000 7f618164d640 1 --2- 172.21.15.32:0/2479713531 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f616806be30 0x7f616806e2f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f616c009a20 tx=0x7f616c008040 comp rx=0 tx=0).stop 2024-04-24T23:04:35.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.708+0000 7f618164d640 1 -- 172.21.15.32:0/2479713531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f617c15c8a0 msgr2=0x7f617c0b6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:35.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.708+0000 7f618164d640 1 --2- 172.21.15.32:0/2479713531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f617c15c8a0 0x7f617c0b6330 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f6174009e50 tx=0x7f6174004a40 comp rx=0 tx=0).stop 2024-04-24T23:04:35.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.708+0000 7f618164d640 1 -- 172.21.15.32:0/2479713531 shutdown_connections 2024-04-24T23:04:35.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.708+0000 7f618164d640 1 --2- 172.21.15.32:0/2479713531 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f616806be30 0x7f616806e2f0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:35.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.708+0000 7f618164d640 1 --2- 172.21.15.32:0/2479713531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f617c15c8a0 0x7f617c0b6330 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:35.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.708+0000 7f618164d640 1 --2- 172.21.15.32:0/2479713531 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f617c0ce7a0 0x7f617c0c83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:35.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.708+0000 7f618164d640 1 --2- 172.21.15.32:0/2479713531 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f617c074920 0x7f617c0d08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:35.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.708+0000 7f618164d640 1 -- 172.21.15.32:0/2479713531 >> 172.21.15.32:0/2479713531 conn(0x7f617c0b2e60 msgr2=0x7f617c15b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:35.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.708+0000 7f618164d640 1 -- 172.21.15.32:0/2479713531 shutdown_connections 2024-04-24T23:04:35.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.708+0000 7f618164d640 1 -- 172.21.15.32:0/2479713531 wait complete. 2024-04-24T23:04:35.798 DEBUG:tasks.ceph_test_case:wait_until_true: success in 15s and 0 retries 2024-04-24T23:04:35.799 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-24T23:04:35.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.898+0000 7f5a3c076640 1 Processor -- start 2024-04-24T23:04:35.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.898+0000 7f5a3c076640 1 -- start start 2024-04-24T23:04:35.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.899+0000 7f5a3c076640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5a3415b480 0x7f5a340b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:35.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.899+0000 7f5a3c076640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a340b46c0 con 0x7f5a3406cd00 2024-04-24T23:04:35.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.899+0000 7f5a3c076640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a340b49a0 con 0x7f5a3415fd60 2024-04-24T23:04:35.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.899+0000 7f5a3c076640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f5a340b4ff0 con 0x7f5a3415b480 2024-04-24T23:04:35.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.899+0000 7f5a3a613640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5a3406cd00 0x7f5a340cdf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39156/0 (socket says 172.21.15.32:39156) 2024-04-24T23:04:35.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.899+0000 7f5a3a613640 1 -- 172.21.15.32:0/1897074953 learned_addr learned my addr 172.21.15.32:0/1897074953 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:04:35.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.899+0000 7f5a39e12640 1 --2- 172.21.15.32:0/1897074953 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5a3415b480 0x7f5a340b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:35.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.899+0000 7f5a38e10640 1 -- 172.21.15.32:0/1897074953 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2176100466 0 0) 0x7f5a340b46c0 con 0x7f5a3406cd00 2024-04-24T23:04:35.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.899+0000 7f5a38e10640 1 -- 172.21.15.32:0/1897074953 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a20003520 con 0x7f5a3406cd00 2024-04-24T23:04:35.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.900+0000 7f5a38e10640 1 -- 172.21.15.32:0/1897074953 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 834085141 0 0) 0x7f5a340b49a0 con 0x7f5a3415fd60 2024-04-24T23:04:35.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.900+0000 7f5a38e10640 1 -- 172.21.15.32:0/1897074953 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a340b46c0 con 0x7f5a3415fd60 2024-04-24T23:04:35.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.900+0000 7f5a38e10640 1 -- 172.21.15.32:0/1897074953 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1002533787 0 0) 0x7f5a20003520 con 0x7f5a3406cd00 2024-04-24T23:04:35.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.900+0000 7f5a39e12640 1 -- 172.21.15.32:0/1897074953 >> v1:172.21.15.121:6789/0 conn(0x7f5a3415fd60 legacy=0x7f5a3406c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:35.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.900+0000 7f5a39e12640 1 -- 172.21.15.32:0/1897074953 >> v1:172.21.15.32:6789/0 conn(0x7f5a3406cd00 legacy=0x7f5a340cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:35.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.900+0000 7f5a39e12640 1 -- 172.21.15.32:0/1897074953 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a340b4c80 con 0x7f5a3415b480 2024-04-24T23:04:35.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.900+0000 7f5a39e12640 1 --2- 172.21.15.32:0/1897074953 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5a3415b480 0x7f5a340b5c60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5a300097d0 tx=0x7f5a3002a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97c2171e2ff1b187 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:35.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.901+0000 7f5a38e10640 1 -- 172.21.15.32:0/1897074953 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a30037070 con 0x7f5a3415b480 2024-04-24T23:04:35.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.901+0000 7f5a38e10640 1 -- 172.21.15.32:0/1897074953 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a3002ac30 con 0x7f5a3415b480 2024-04-24T23:04:35.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.901+0000 7f5a38e10640 1 -- 172.21.15.32:0/1897074953 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c001970 con 0x7f5a3415b480 2024-04-24T23:04:35.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.901+0000 7f5a3c076640 1 -- 172.21.15.32:0/1897074953 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5a3415b480 msgr2=0x7f5a340b5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:35.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.901+0000 7f5a3c076640 1 --2- 172.21.15.32:0/1897074953 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5a3415b480 0x7f5a340b5c60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5a300097d0 tx=0x7f5a3002a1a0 comp rx=0 tx=0).stop 2024-04-24T23:04:35.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.901+0000 7f5a3c076640 1 -- 172.21.15.32:0/1897074953 shutdown_connections 2024-04-24T23:04:35.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.901+0000 7f5a3c076640 1 --2- 172.21.15.32:0/1897074953 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5a3415b480 0x7f5a340b5c60 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:35.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.901+0000 7f5a3c076640 1 -- 172.21.15.32:0/1897074953 >> 172.21.15.32:0/1897074953 conn(0x7f5a3406b110 msgr2=0x7f5a3406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:35.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.901+0000 7f5a3c076640 1 -- 172.21.15.32:0/1897074953 shutdown_connections 2024-04-24T23:04:35.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.901+0000 7f5a3c076640 1 -- 172.21.15.32:0/1897074953 wait complete. 2024-04-24T23:04:35.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.902+0000 7f5a3c076640 1 Processor -- start 2024-04-24T23:04:35.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.902+0000 7f5a3c076640 1 -- start start 2024-04-24T23:04:35.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.903+0000 7f5a3c076640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5a3406cd00 0x7f5a34149080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:35.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.903+0000 7f5a3c076640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a3415b480 0x7f5a340ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:35.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.903+0000 7f5a3c076640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a3415fd60 0x7f5a340cd400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:35.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.903+0000 7f5a3c076640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5a3415b860 con 0x7f5a3415b480 2024-04-24T23:04:35.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.903+0000 7f5a3c076640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5a340b5480 con 0x7f5a3415fd60 2024-04-24T23:04:35.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.903+0000 7f5a3c076640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f5a341495c0 con 0x7f5a3406cd00 2024-04-24T23:04:35.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.903+0000 7f5a39611640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a3415b480 0x7f5a340ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:35.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.903+0000 7f5a39611640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a3415b480 0x7f5a340ccec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43964/0 (socket says 172.21.15.32:43964) 2024-04-24T23:04:35.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.903+0000 7f5a39611640 1 -- 172.21.15.32:0/3293465312 learned_addr learned my addr 172.21.15.32:0/3293465312 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:35.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.904+0000 7f5a3a613640 1 --2- 172.21.15.32:0/3293465312 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a3415fd60 0x7f5a340cd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:35.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.904+0000 7f5a39e12640 1 --2- 172.21.15.32:0/3293465312 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5a3406cd00 0x7f5a34149080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:35.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.904+0000 7f5a39611640 1 -- 172.21.15.32:0/3293465312 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5a3406cd00 msgr2=0x7f5a34149080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:35.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.904+0000 7f5a39611640 1 --2- 172.21.15.32:0/3293465312 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5a3406cd00 0x7f5a34149080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:35.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.904+0000 7f5a39611640 1 -- 172.21.15.32:0/3293465312 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a3415fd60 msgr2=0x7f5a340cd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:35.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.904+0000 7f5a39611640 1 --2- 172.21.15.32:0/3293465312 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a3415fd60 0x7f5a340cd400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:35.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.904+0000 7f5a39611640 1 -- 172.21.15.32:0/3293465312 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a341572d0 con 0x7f5a3415b480 2024-04-24T23:04:35.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.904+0000 7f5a3a613640 1 --2- 172.21.15.32:0/3293465312 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a3415fd60 0x7f5a340cd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:04:35.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.904+0000 7f5a39e12640 1 --2- 172.21.15.32:0/3293465312 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5a3406cd00 0x7f5a34149080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:04:35.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.904+0000 7f5a39611640 1 --2- 172.21.15.32:0/3293465312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a3415b480 0x7f5a340ccec0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f5a240010e0 tx=0x7f5a2400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:35.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.904+0000 7f5a2a7fc640 1 -- 172.21.15.32:0/3293465312 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a24017070 con 0x7f5a3415b480 2024-04-24T23:04:35.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.904+0000 7f5a2a7fc640 1 -- 172.21.15.32:0/3293465312 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a240052f0 con 0x7f5a3415b480 2024-04-24T23:04:35.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.904+0000 7f5a3c076640 1 -- 172.21.15.32:0/3293465312 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a341575b0 con 0x7f5a3415b480 2024-04-24T23:04:35.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.904+0000 7f5a2a7fc640 1 -- 172.21.15.32:0/3293465312 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a24012870 con 0x7f5a3415b480 2024-04-24T23:04:35.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.904+0000 7f5a3c076640 1 -- 172.21.15.32:0/3293465312 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a34157b70 con 0x7f5a3415b480 2024-04-24T23:04:35.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.905+0000 7f5a3c076640 1 -- 172.21.15.32:0/3293465312 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a34068430 con 0x7f5a3415b480 2024-04-24T23:04:35.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.905+0000 7f5a2a7fc640 1 -- 172.21.15.32:0/3293465312 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5a2401a070 con 0x7f5a3415b480 2024-04-24T23:04:35.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.906+0000 7f5a2a7fc640 1 --2- 172.21.15.32:0/3293465312 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5a1806be30 0x7f5a1806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:35.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.906+0000 7f5a2a7fc640 1 -- 172.21.15.32:0/3293465312 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7f5a2408a240 con 0x7f5a3415b480 2024-04-24T23:04:35.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.906+0000 7f5a39e12640 1 --2- 172.21.15.32:0/3293465312 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5a1806be30 0x7f5a1806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:35.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.907+0000 7f5a39e12640 1 --2- 172.21.15.32:0/3293465312 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5a1806be30 0x7f5a1806e2f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5a30034000 tx=0x7f5a3002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:35.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:35.910+0000 7f5a2a7fc640 1 -- 172.21.15.32:0/3293465312 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5a24058c00 con 0x7f5a3415b480 2024-04-24T23:04:36.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.097+0000 7f5a3c076640 1 -- 172.21.15.32:0/3293465312 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f5a3406c000 con 0x7f5a3415b480 2024-04-24T23:04:36.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.098+0000 7f5a2a7fc640 1 -- 172.21.15.32:0/3293465312 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v113) ==== 81+0+34185 (secure 0 0 0) 0x7f5a24058500 con 0x7f5a3415b480 2024-04-24T23:04:36.100 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:04:36.100 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:04:36.101 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 113, 2024-04-24T23:04:36.101 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", 2024-04-24T23:04:36.101 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2024-04-24T22:58:46.964851+0000", 2024-04-24T23:04:36.101 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2024-04-24T23:04:31.618095+0000", 2024-04-24T23:04:36.101 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2024-04-24T22:58:52.048787+0000", 2024-04-24T23:04:36.101 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2024-04-24T22:58:49.045609+0000", 2024-04-24T23:04:36.101 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-24T23:04:36.101 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2024-04-24T23:04:36.101 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2024-04-24T23:04:36.102 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2024-04-24T23:04:36.102 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2024-04-24T23:04:36.102 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2024-04-24T23:04:36.102 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2024-04-24T23:04:36.102 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:04:36.102 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 4, 2024-04-24T23:04:36.102 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.69999998807907104, 2024-04-24T23:04:36.102 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-24T23:04:36.103 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-24T23:04:36.103 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2024-04-24T23:04:36.103 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 13, 2024-04-24T23:04:36.103 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2024-04-24T23:04:36.103 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "luminous", 2024-04-24T23:04:36.103 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "jewel", 2024-04-24T23:04:36.103 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "squid", 2024-04-24T23:04:36.103 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_crimson": false, 2024-04-24T23:04:36.103 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2024-04-24T23:04:36.103 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.103 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2024-04-24T23:04:36.104 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": ".mgr", 2024-04-24T23:04:36.104 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T22:58:53.171408+0000", 2024-04-24T23:04:36.104 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:04:36.104 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:04:36.104 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:04:36.104 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:04:36.104 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:04:36.104 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:04:36.104 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:04:36.105 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:04:36.105 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:04:36.105 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:04:36.105 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:04:36.105 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:04:36.105 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2024-04-24T23:04:36.105 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2024-04-24T23:04:36.105 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2024-04-24T23:04:36.105 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2024-04-24T23:04:36.105 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2024-04-24T23:04:36.105 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:04:36.105 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:04:36.105 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:04:36.105 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:04:36.106 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:04:36.106 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:04:36.106 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:04:36.106 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.106 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "113", 2024-04-24T23:04:36.106 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:04:36.106 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:04:36.106 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:04:36.106 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:04:36.106 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:04:36.107 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:04:36.107 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:04:36.107 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:04:36.107 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:04:36.107 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:04:36.107 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:04:36.107 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:04:36.107 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:04:36.107 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:04:36.107 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:04:36.108 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:04:36.108 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:04:36.108 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:04:36.108 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:04:36.108 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:04:36.108 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:04:36.108 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:04:36.108 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:04:36.108 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:04:36.108 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:04:36.108 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:04:36.108 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.108 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:04:36.109 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:04:36.109 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:04:36.109 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:04:36.109 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:04:36.109 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:04:36.109 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:04:36.109 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:04:36.109 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:04:36.109 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:04:36.109 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:04:36.109 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:04:36.110 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_max": 32, 2024-04-24T23:04:36.110 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2024-04-24T23:04:36.110 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.110 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:04:36.110 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": {} 2024-04-24T23:04:36.110 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.110 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:04:36.110 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:04:36.110 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 8, 2024-04-24T23:04:36.110 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 8, 2024-04-24T23:04:36.110 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.25, 2024-04-24T23:04:36.111 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-04-24T23:04:36.111 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-04-24T23:04:36.111 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:04:36.111 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:04:36.111 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:04:36.111 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:36.111 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.111 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.111 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 12, 2024-04-24T23:04:36.111 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2024-04-24T23:04:36.111 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:03:59.079157+0000", 2024-04-24T23:04:36.111 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:04:36.111 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:04:36.111 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:04:36.111 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:04:36.112 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:04:36.112 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:04:36.112 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:04:36.112 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:04:36.112 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:04:36.112 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:04:36.112 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:04:36.112 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:04:36.112 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 32, 2024-04-24T23:04:36.112 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 32, 2024-04-24T23:04:36.112 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 32, 2024-04-24T23:04:36.112 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 32, 2024-04-24T23:04:36.112 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 32, 2024-04-24T23:04:36.112 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:04:36.112 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:04:36.113 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:04:36.113 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:04:36.113 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:04:36.113 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:04:36.113 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:04:36.113 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.113 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "113", 2024-04-24T23:04:36.113 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:04:36.113 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:04:36.113 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:04:36.113 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:04:36.113 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:04:36.113 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:04:36.113 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:04:36.113 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:04:36.113 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:04:36.114 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:04:36.114 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:04:36.114 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:04:36.114 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:04:36.114 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:04:36.114 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:04:36.114 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:04:36.114 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:04:36.114 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:04:36.114 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:04:36.114 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:04:36.114 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:04:36.114 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:04:36.115 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:04:36.115 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:04:36.115 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:04:36.115 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:04:36.115 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.115 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:04:36.115 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:04:36.115 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:04:36.115 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:04:36.115 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:04:36.115 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:04:36.115 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:04:36.115 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:04:36.115 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:04:36.115 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:04:36.116 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:04:36.116 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:04:36.116 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2024-04-24T23:04:36.116 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2024-04-24T23:04:36.116 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2024-04-24T23:04:36.116 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.116 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:04:36.116 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:04:36.116 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2024-04-24T23:04:36.116 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:36.116 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.116 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:04:36.116 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:04:36.116 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.75, 2024-04-24T23:04:36.116 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.75, 2024-04-24T23:04:36.117 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:04:36.117 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.75, 2024-04-24T23:04:36.117 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.75, 2024-04-24T23:04:36.117 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:04:36.117 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:04:36.117 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:04:36.117 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:36.117 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.117 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.117 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 13, 2024-04-24T23:04:36.117 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2024-04-24T23:04:36.117 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:03:59.765179+0000", 2024-04-24T23:04:36.117 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:04:36.117 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:04:36.117 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:04:36.118 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:04:36.118 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:04:36.118 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:04:36.118 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:04:36.118 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:04:36.118 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:04:36.118 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:04:36.118 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:04:36.118 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:04:36.118 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 64, 2024-04-24T23:04:36.118 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 64, 2024-04-24T23:04:36.118 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 64, 2024-04-24T23:04:36.118 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 64, 2024-04-24T23:04:36.118 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 64, 2024-04-24T23:04:36.118 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:04:36.119 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:04:36.119 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:04:36.119 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:04:36.119 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:04:36.119 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:04:36.119 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:04:36.119 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.119 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "113", 2024-04-24T23:04:36.119 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:04:36.119 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:04:36.119 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:04:36.119 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:04:36.119 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:04:36.119 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:04:36.119 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:04:36.120 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:04:36.120 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:04:36.120 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:04:36.120 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:04:36.120 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:04:36.120 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:04:36.120 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:04:36.120 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:04:36.120 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:04:36.120 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:04:36.120 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:04:36.120 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:04:36.120 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:04:36.120 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:04:36.120 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:04:36.121 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:04:36.121 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:04:36.121 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:04:36.121 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:04:36.121 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.121 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:04:36.121 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:04:36.121 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:04:36.121 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:04:36.121 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:04:36.121 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:04:36.121 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:04:36.121 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:04:36.121 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:04:36.121 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:04:36.121 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:04:36.122 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:04:36.122 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 64, 2024-04-24T23:04:36.122 INFO:teuthology.orchestra.run.smithi032.stdout: "target_size_ratio": 0.89999997615814209 2024-04-24T23:04:36.122 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.122 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:04:36.122 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:04:36.122 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2024-04-24T23:04:36.122 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:36.122 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.122 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:04:36.122 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:04:36.122 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.3799999952316284, 2024-04-24T23:04:36.122 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.3799999952316284, 2024-04-24T23:04:36.122 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:04:36.122 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.3799999952316284, 2024-04-24T23:04:36.123 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.3799999952316284, 2024-04-24T23:04:36.123 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:04:36.123 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:04:36.123 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:04:36.123 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:36.123 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:36.123 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:04:36.123 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-04-24T23:04:36.123 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.123 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:04:36.123 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:36.123 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:36.123 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:36.123 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:36.124 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:36.124 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:36.124 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:36.124 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:36.124 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:36.124 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:36.124 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:36.124 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:36.124 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:36.124 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.124 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:36.124 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2024-04-24T23:04:36.124 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:36.124 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.124 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.124 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:36.125 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2024-04-24T23:04:36.125 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:36.125 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:36.125 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:36.125 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.125 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:36.125 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:36.125 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.125 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:36.125 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2024-04-24T23:04:36.125 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:36.125 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.125 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.125 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:36.125 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2024-04-24T23:04:36.126 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:36.126 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:36.126 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:36.126 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.126 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:36.126 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:36.126 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.126 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:36.126 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2024-04-24T23:04:36.126 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:36.126 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.126 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.126 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:36.126 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2024-04-24T23:04:36.126 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:36.127 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:36.127 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:36.127 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.127 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:36.127 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:36.127 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.127 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:36.127 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2024-04-24T23:04:36.127 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:36.127 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.128 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.128 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:36.128 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2024-04-24T23:04:36.128 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:04:36.128 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:36.128 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:36.128 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.128 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/1479395426", 2024-04-24T23:04:36.128 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/1479395426", 2024-04-24T23:04:36.129 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/1479395426", 2024-04-24T23:04:36.129 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/1479395426", 2024-04-24T23:04:36.129 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:36.129 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:36.129 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:36.129 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:36.129 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.129 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.129 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:04:36.129 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:36.129 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:36.129 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:36.129 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:36.129 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:36.130 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:36.130 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:36.130 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:36.130 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:36.130 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:36.130 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:36.130 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:36.130 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:36.130 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.130 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:36.130 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2024-04-24T23:04:36.131 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:36.131 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.131 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.131 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:36.131 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2024-04-24T23:04:36.131 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:36.131 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:36.131 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:36.131 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.131 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:36.132 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:36.132 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.132 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:36.132 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2024-04-24T23:04:36.132 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:36.132 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.133 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.133 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:36.133 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2024-04-24T23:04:36.133 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:36.133 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:36.133 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:36.133 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.133 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:36.133 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:36.133 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.134 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:36.134 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2024-04-24T23:04:36.134 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:36.134 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.134 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.134 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:36.134 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2024-04-24T23:04:36.134 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:36.134 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:36.134 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:36.134 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.135 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:36.135 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:36.135 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.135 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:36.135 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2024-04-24T23:04:36.135 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:36.135 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.135 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.135 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:36.135 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2024-04-24T23:04:36.135 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:04:36.137 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:36.137 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:36.137 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.137 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/1340633718", 2024-04-24T23:04:36.138 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/1340633718", 2024-04-24T23:04:36.138 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/1340633718", 2024-04-24T23:04:36.138 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/1340633718", 2024-04-24T23:04:36.138 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:36.138 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:36.139 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:36.139 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:36.139 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.139 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.139 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:04:36.139 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:36.139 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:36.139 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:36.140 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:36.140 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:36.140 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:36.140 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:36.140 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:36.140 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:36.141 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:36.141 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:36.141 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:36.141 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:36.141 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.141 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:36.141 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2024-04-24T23:04:36.141 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:36.142 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.142 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.142 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:36.142 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2024-04-24T23:04:36.142 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:36.143 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:36.143 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:36.143 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.143 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:36.143 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:36.143 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.143 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:36.143 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2024-04-24T23:04:36.144 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:36.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.144 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.144 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:36.144 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2024-04-24T23:04:36.144 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:36.144 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:36.144 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:36.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.144 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:36.144 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:36.144 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.144 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:36.145 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2024-04-24T23:04:36.145 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:36.145 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.145 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.145 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:36.145 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2024-04-24T23:04:36.145 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:36.145 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:36.145 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:36.145 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.145 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:36.146 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:36.146 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.146 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:36.146 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2024-04-24T23:04:36.146 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:36.146 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.146 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.146 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:36.147 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2024-04-24T23:04:36.147 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:04:36.147 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:36.147 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:36.147 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.147 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/3405899334", 2024-04-24T23:04:36.147 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6820/3405899334", 2024-04-24T23:04:36.147 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6828/3405899334", 2024-04-24T23:04:36.147 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6824/3405899334", 2024-04-24T23:04:36.147 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:36.148 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:36.148 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:36.148 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:36.148 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.148 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.148 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:04:36.148 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:36.148 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:36.148 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:36.148 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:36.148 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:36.148 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:36.149 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:36.149 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:36.149 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:36.149 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:36.149 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:36.149 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:36.149 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:36.149 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.149 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:36.149 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2024-04-24T23:04:36.150 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:36.150 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.150 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.150 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:36.150 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2024-04-24T23:04:36.150 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:36.150 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:36.150 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:36.150 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.151 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:36.151 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:36.151 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.151 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:36.151 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2024-04-24T23:04:36.151 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:36.151 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.151 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.151 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:36.151 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2024-04-24T23:04:36.152 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:36.152 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:36.152 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:36.152 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.152 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:36.152 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:36.152 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.152 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:36.152 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2024-04-24T23:04:36.152 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:36.152 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.152 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.152 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:36.153 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2024-04-24T23:04:36.153 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:36.153 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:36.153 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:36.153 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.153 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:36.153 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:36.153 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.153 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:36.153 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2024-04-24T23:04:36.153 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:36.153 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.153 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.153 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:36.153 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2024-04-24T23:04:36.154 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:04:36.154 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:36.154 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:36.154 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.154 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6821/3381057190", 2024-04-24T23:04:36.154 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6825/3381057190", 2024-04-24T23:04:36.154 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/3381057190", 2024-04-24T23:04:36.154 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/3381057190", 2024-04-24T23:04:36.154 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:36.154 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:36.154 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:36.154 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:36.154 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.154 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.154 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:04:36.155 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:36.155 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:36.155 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:36.155 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:36.155 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:36.155 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:36.155 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:36.155 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:36.155 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:36.155 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:36.155 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:36.155 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:36.155 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:36.155 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.155 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:36.156 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6800", 2024-04-24T23:04:36.156 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:36.156 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.156 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.156 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:36.156 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6801", 2024-04-24T23:04:36.156 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:36.156 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:36.156 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:36.156 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.156 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:36.156 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:36.156 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.157 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:36.157 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6802", 2024-04-24T23:04:36.157 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:36.157 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.157 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.157 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:36.157 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6803", 2024-04-24T23:04:36.157 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:36.157 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:36.157 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:36.157 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.157 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:36.157 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:36.158 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.158 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:36.158 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6806", 2024-04-24T23:04:36.158 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:36.158 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.158 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.158 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:36.158 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6807", 2024-04-24T23:04:36.158 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:36.158 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:36.158 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:36.158 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.158 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:36.158 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:36.158 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.159 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:36.159 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6804", 2024-04-24T23:04:36.159 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:36.159 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.159 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.159 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:36.159 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6805", 2024-04-24T23:04:36.159 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:04:36.159 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:36.159 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:36.159 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.159 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6801/4010553979", 2024-04-24T23:04:36.159 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6803/4010553979", 2024-04-24T23:04:36.159 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6807/4010553979", 2024-04-24T23:04:36.160 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6805/4010553979", 2024-04-24T23:04:36.160 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:36.160 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:36.160 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:36.160 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:36.160 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.160 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.160 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:04:36.160 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:36.160 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:36.160 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:36.160 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:36.160 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:36.160 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:36.160 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:36.161 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:36.161 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:36.161 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:36.161 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:36.161 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:36.161 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:36.161 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.161 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:36.161 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6808", 2024-04-24T23:04:36.161 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:36.161 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.161 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.161 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:36.161 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6809", 2024-04-24T23:04:36.162 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:36.162 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:36.162 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:36.162 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.162 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:36.162 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:36.162 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.162 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:36.162 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6810", 2024-04-24T23:04:36.162 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:36.162 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.162 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.162 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:36.163 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6811", 2024-04-24T23:04:36.163 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:36.163 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:36.163 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:36.163 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.163 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:36.163 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:36.163 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.163 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:36.163 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6814", 2024-04-24T23:04:36.163 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:36.163 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.163 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.163 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:36.163 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6815", 2024-04-24T23:04:36.164 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:36.164 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:36.164 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:36.164 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.164 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:36.164 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:36.164 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.164 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:36.164 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6812", 2024-04-24T23:04:36.164 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:36.164 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.164 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.164 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:36.164 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6813", 2024-04-24T23:04:36.164 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:04:36.165 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:36.165 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:36.165 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.165 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6809/3794084091", 2024-04-24T23:04:36.165 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6811/3794084091", 2024-04-24T23:04:36.165 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6815/3794084091", 2024-04-24T23:04:36.165 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6813/3794084091", 2024-04-24T23:04:36.165 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:36.165 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:36.165 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:36.165 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:36.165 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.165 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.165 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:04:36.165 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:36.166 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:36.166 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:36.166 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:36.166 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:36.166 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:36.166 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:36.166 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:36.166 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:36.166 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:36.166 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:36.166 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:36.167 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:36.167 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.167 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:36.167 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6823", 2024-04-24T23:04:36.167 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:36.167 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.167 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.167 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:36.167 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6825", 2024-04-24T23:04:36.167 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:36.167 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:36.167 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:36.168 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.168 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:36.168 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:36.168 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.168 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:36.168 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6826", 2024-04-24T23:04:36.168 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:36.168 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.168 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.168 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:36.168 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6827", 2024-04-24T23:04:36.168 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:36.169 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:36.169 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:36.169 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.169 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:36.169 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:36.169 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.169 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:36.169 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6830", 2024-04-24T23:04:36.169 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:36.169 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.169 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.169 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:36.170 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6831", 2024-04-24T23:04:36.170 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:36.170 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:36.170 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:36.170 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.170 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:36.170 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:36.170 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.170 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:36.170 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6828", 2024-04-24T23:04:36.170 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:36.170 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.170 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.171 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:36.171 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6829", 2024-04-24T23:04:36.171 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:04:36.171 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:36.171 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:36.171 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.171 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6825/1348640258", 2024-04-24T23:04:36.171 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6827/1348640258", 2024-04-24T23:04:36.171 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6831/1348640258", 2024-04-24T23:04:36.171 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6829/1348640258", 2024-04-24T23:04:36.172 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:36.172 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:36.172 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:36.172 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:36.172 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.172 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.172 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:04:36.172 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:04:36.172 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:04:36.172 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:04:36.172 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:04:36.172 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:04:36.172 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:04:36.172 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:04:36.172 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:04:36.172 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 102, 2024-04-24T23:04:36.173 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:04:36.173 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:04:36.173 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:04:36.173 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:36.173 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.173 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:36.173 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6816", 2024-04-24T23:04:36.173 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:36.173 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.173 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.173 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:36.173 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6817", 2024-04-24T23:04:36.173 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:36.173 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:36.173 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:36.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.174 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:04:36.174 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:36.174 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.174 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:36.174 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6818", 2024-04-24T23:04:36.174 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:36.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.174 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.174 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:36.174 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6819", 2024-04-24T23:04:36.174 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:36.174 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:36.174 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:36.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.175 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:04:36.175 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:36.175 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.175 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:36.175 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6822", 2024-04-24T23:04:36.175 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:36.175 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.175 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.175 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:36.175 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6824", 2024-04-24T23:04:36.176 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:36.176 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:36.176 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:36.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.176 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:04:36.176 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:04:36.176 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.176 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:04:36.176 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6820", 2024-04-24T23:04:36.176 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:36.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.176 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.176 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:04:36.176 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6821", 2024-04-24T23:04:36.176 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:04:36.177 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:36.177 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:36.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.177 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6817/3250005122", 2024-04-24T23:04:36.177 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6819/3250005122", 2024-04-24T23:04:36.177 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6824/3250005122", 2024-04-24T23:04:36.177 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6821/3250005122", 2024-04-24T23:04:36.177 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:04:36.177 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:04:36.178 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:04:36.178 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:36.178 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:36.178 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:04:36.178 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2024-04-24T23:04:36.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.178 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:04:36.178 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:36.178 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:36.178 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:36.179 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:36.179 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:36.179 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.156184+0000", 2024-04-24T23:04:36.179 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:36.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.179 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.179 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:04:36.179 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:36.179 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:36.180 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:36.180 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:36.180 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:36.180 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240774+0000", 2024-04-24T23:04:36.180 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:36.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.180 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.181 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:04:36.181 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:36.181 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:36.181 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:36.181 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:36.181 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:36.181 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240949+0000", 2024-04-24T23:04:36.181 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:36.181 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.181 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.182 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:04:36.182 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:36.182 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:36.182 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:36.182 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:36.182 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:36.182 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.171672+0000", 2024-04-24T23:04:36.183 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:36.183 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.183 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.183 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:04:36.183 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:36.183 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:36.183 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:36.183 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:36.184 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:36.184 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.220379+0000", 2024-04-24T23:04:36.184 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:36.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.184 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.184 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:04:36.184 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:36.184 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:36.184 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:36.184 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:36.184 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:36.184 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.204963+0000", 2024-04-24T23:04:36.185 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:36.185 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.185 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.185 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:04:36.185 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:36.185 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:36.186 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:36.186 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:36.186 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:36.186 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.239748+0000", 2024-04-24T23:04:36.186 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:36.186 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.187 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:36.187 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:04:36.187 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:04:36.187 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:04:36.187 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:04:36.187 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:04:36.187 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:04:36.187 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.154104+0000", 2024-04-24T23:04:36.188 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:04:36.188 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:36.188 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:04:36.188 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2024-04-24T23:04:36.188 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2024-04-24T23:04:36.188 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_primaries": [], 2024-04-24T23:04:36.188 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2024-04-24T23:04:36.188 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2024-04-24T23:04:36.188 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklist": {}, 2024-04-24T23:04:36.188 INFO:teuthology.orchestra.run.smithi032.stdout: "range_blocklist": {}, 2024-04-24T23:04:36.188 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2024-04-24T23:04:36.188 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2024-04-24T23:04:36.188 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2024-04-24T23:04:36.188 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2024-04-24T23:04:36.189 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2024-04-24T23:04:36.189 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2024-04-24T23:04:36.189 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2024-04-24T23:04:36.189 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:36.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:36.189 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2024-04-24T23:04:36.189 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2024-04-24T23:04:36.189 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2024-04-24T23:04:36.189 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2024-04-24T23:04:36.189 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {}, 2024-04-24T23:04:36.189 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode": { 2024-04-24T23:04:36.189 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_enabled": false, 2024-04-24T23:04:36.189 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_bucket_count": 0, 2024-04-24T23:04:36.189 INFO:teuthology.orchestra.run.smithi032.stdout: "degraded_stretch_mode": 0, 2024-04-24T23:04:36.189 INFO:teuthology.orchestra.run.smithi032.stdout: "recovering_stretch_mode": 0, 2024-04-24T23:04:36.190 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_bucket": 0 2024-04-24T23:04:36.190 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:36.190 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:04:36.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.100+0000 7f5a3c076640 1 -- 172.21.15.32:0/3293465312 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5a1806be30 msgr2=0x7f5a1806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:36.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.100+0000 7f5a3c076640 1 --2- 172.21.15.32:0/3293465312 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5a1806be30 0x7f5a1806e2f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5a30034000 tx=0x7f5a3002e000 comp rx=0 tx=0).stop 2024-04-24T23:04:36.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.100+0000 7f5a3c076640 1 -- 172.21.15.32:0/3293465312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a3415b480 msgr2=0x7f5a340ccec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:36.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.100+0000 7f5a3c076640 1 --2- 172.21.15.32:0/3293465312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a3415b480 0x7f5a340ccec0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f5a240010e0 tx=0x7f5a2400c960 comp rx=0 tx=0).stop 2024-04-24T23:04:36.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.100+0000 7f5a3c076640 1 -- 172.21.15.32:0/3293465312 shutdown_connections 2024-04-24T23:04:36.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.100+0000 7f5a3c076640 1 --2- 172.21.15.32:0/3293465312 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5a1806be30 0x7f5a1806e2f0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:36.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.100+0000 7f5a3c076640 1 --2- 172.21.15.32:0/3293465312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a3415b480 0x7f5a340ccec0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:36.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.100+0000 7f5a3c076640 1 --2- 172.21.15.32:0/3293465312 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a3415fd60 0x7f5a340cd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:36.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.100+0000 7f5a3c076640 1 --2- 172.21.15.32:0/3293465312 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5a3406cd00 0x7f5a34149080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:36.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.100+0000 7f5a3c076640 1 -- 172.21.15.32:0/3293465312 >> 172.21.15.32:0/3293465312 conn(0x7f5a3406b110 msgr2=0x7f5a3415a590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:36.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.100+0000 7f5a3c076640 1 -- 172.21.15.32:0/3293465312 shutdown_connections 2024-04-24T23:04:36.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.100+0000 7f5a3c076640 1 -- 172.21.15.32:0/3293465312 wait complete. 2024-04-24T23:04:36.193 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:04:36.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.292+0000 7ff3eca5d640 1 Processor -- start 2024-04-24T23:04:36.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.292+0000 7ff3eca5d640 1 -- start start 2024-04-24T23:04:36.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.293+0000 7ff3eca5d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff3e815e9b0 0x7ff3e80d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:36.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.293+0000 7ff3eca5d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff3e806b820 0x7ff3e80b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:36.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.293+0000 7ff3eca5d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3e80d3170 con 0x7ff3e80d4210 2024-04-24T23:04:36.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.293+0000 7ff3eca5d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7ff3e80d3450 con 0x7ff3e815e9b0 2024-04-24T23:04:36.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.293+0000 7ff3eca5d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff3e80d3730 con 0x7ff3e806b820 2024-04-24T23:04:36.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.293+0000 7ff3e659c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff3e815e9b0 0x7ff3e80d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:36.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.293+0000 7ff3e5d9b640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff3e80d4210 0x7ff3e806b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39172/0 (socket says 172.21.15.32:39172) 2024-04-24T23:04:36.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.294+0000 7ff3e5d9b640 1 -- 172.21.15.32:0/1937022237 learned_addr learned my addr 172.21.15.32:0/1937022237 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:04:36.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.294+0000 7ff3e659c640 1 -- 172.21.15.32:0/1937022237 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff3e806b820 msgr2=0x7ff3e80b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:36.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.294+0000 7ff3e6d9d640 1 --2- 172.21.15.32:0/1937022237 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff3e806b820 0x7ff3e80b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:36.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.294+0000 7ff3e559a640 1 -- 172.21.15.32:0/1937022237 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3842999875 0 0) 0x7ff3e80d3170 con 0x7ff3e80d4210 2024-04-24T23:04:36.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.294+0000 7ff3e659c640 1 --2- 172.21.15.32:0/1937022237 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff3e806b820 0x7ff3e80b78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:36.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.294+0000 7ff3e659c640 1 -- 172.21.15.32:0/1937022237 >> v1:172.21.15.32:6789/0 conn(0x7ff3e80d4210 legacy=0x7ff3e806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:36.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.294+0000 7ff3e659c640 1 -- 172.21.15.32:0/1937022237 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3e80d3a10 con 0x7ff3e815e9b0 2024-04-24T23:04:36.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.295+0000 7ff3e659c640 1 --2- 172.21.15.32:0/1937022237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3e815e9b0 0x7ff3e80d2b90 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8009d90 tx=0x7ff3d802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=707c7e46389c0ddc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:36.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.295+0000 7ff3e559a640 1 -- 172.21.15.32:0/1937022237 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d8007740 con 0x7ff3e815e9b0 2024-04-24T23:04:36.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.295+0000 7ff3e559a640 1 -- 172.21.15.32:0/1937022237 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3d80078e0 con 0x7ff3e815e9b0 2024-04-24T23:04:36.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.295+0000 7ff3e559a640 1 -- 172.21.15.32:0/1937022237 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d8007cb0 con 0x7ff3e815e9b0 2024-04-24T23:04:36.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.295+0000 7ff3eca5d640 1 -- 172.21.15.32:0/1937022237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3e815e9b0 msgr2=0x7ff3e80d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:36.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.295+0000 7ff3eca5d640 1 --2- 172.21.15.32:0/1937022237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3e815e9b0 0x7ff3e80d2b90 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8009d90 tx=0x7ff3d802a660 comp rx=0 tx=0).stop 2024-04-24T23:04:36.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.295+0000 7ff3eca5d640 1 -- 172.21.15.32:0/1937022237 shutdown_connections 2024-04-24T23:04:36.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.295+0000 7ff3eca5d640 1 --2- 172.21.15.32:0/1937022237 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff3e806b820 0x7ff3e80b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:36.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.295+0000 7ff3eca5d640 1 --2- 172.21.15.32:0/1937022237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3e815e9b0 0x7ff3e80d2b90 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:36.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.295+0000 7ff3eca5d640 1 -- 172.21.15.32:0/1937022237 >> 172.21.15.32:0/1937022237 conn(0x7ff3e80d11d0 msgr2=0x7ff3e80d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:36.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.295+0000 7ff3eca5d640 1 -- 172.21.15.32:0/1937022237 shutdown_connections 2024-04-24T23:04:36.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.295+0000 7ff3eca5d640 1 -- 172.21.15.32:0/1937022237 wait complete. 2024-04-24T23:04:36.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.297+0000 7ff3eca5d640 1 Processor -- start 2024-04-24T23:04:36.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.297+0000 7ff3eca5d640 1 -- start start 2024-04-24T23:04:36.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.297+0000 7ff3eca5d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3e806b820 0x7ff3e80cd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:36.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.297+0000 7ff3eca5d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff3e80d4210 0x7ff3e8159d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:36.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.297+0000 7ff3eca5d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3e815e9b0 0x7ff3e81f4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:36.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.297+0000 7ff3eca5d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e80ba0e0 con 0x7ff3e815e9b0 2024-04-24T23:04:36.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.297+0000 7ff3eca5d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e80d2400 con 0x7ff3e806b820 2024-04-24T23:04:36.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.297+0000 7ff3eca5d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff3e80ba260 con 0x7ff3e80d4210 2024-04-24T23:04:36.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.297+0000 7ff3e6d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3e815e9b0 0x7ff3e81f4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:36.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.297+0000 7ff3e6d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3e815e9b0 0x7ff3e81f4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43978/0 (socket says 172.21.15.32:43978) 2024-04-24T23:04:36.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.297+0000 7ff3e6d9d640 1 -- 172.21.15.32:0/2896083408 learned_addr learned my addr 172.21.15.32:0/2896083408 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:36.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.298+0000 7ff3e5d9b640 1 --2- 172.21.15.32:0/2896083408 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff3e80d4210 0x7ff3e8159d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:36.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.298+0000 7ff3e659c640 1 --2- 172.21.15.32:0/2896083408 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3e806b820 0x7ff3e80cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:36.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.298+0000 7ff3e6d9d640 1 -- 172.21.15.32:0/2896083408 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff3e80d4210 msgr2=0x7ff3e8159d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:36.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.298+0000 7ff3e6d9d640 1 --2- 172.21.15.32:0/2896083408 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff3e80d4210 0x7ff3e8159d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:36.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.298+0000 7ff3e6d9d640 1 -- 172.21.15.32:0/2896083408 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3e806b820 msgr2=0x7ff3e80cd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:36.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.298+0000 7ff3e6d9d640 1 --2- 172.21.15.32:0/2896083408 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3e806b820 0x7ff3e80cd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:36.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.298+0000 7ff3e6d9d640 1 -- 172.21.15.32:0/2896083408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3e815a270 con 0x7ff3e815e9b0 2024-04-24T23:04:36.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.298+0000 7ff3e6d9d640 1 --2- 172.21.15.32:0/2896083408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3e815e9b0 0x7ff3e81f4800 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7ff3e000c7c0 tx=0x7ff3e000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:36.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.298+0000 7ff3d6ffd640 1 -- 172.21.15.32:0/2896083408 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e0015070 con 0x7ff3e815e9b0 2024-04-24T23:04:36.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.298+0000 7ff3d6ffd640 1 -- 172.21.15.32:0/2896083408 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3e0004220 con 0x7ff3e815e9b0 2024-04-24T23:04:36.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.298+0000 7ff3eca5d640 1 -- 172.21.15.32:0/2896083408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3e81f4d40 con 0x7ff3e815e9b0 2024-04-24T23:04:36.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.298+0000 7ff3d6ffd640 1 -- 172.21.15.32:0/2896083408 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e0007620 con 0x7ff3e815e9b0 2024-04-24T23:04:36.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.298+0000 7ff3eca5d640 1 -- 172.21.15.32:0/2896083408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3e80ce130 con 0x7ff3e815e9b0 2024-04-24T23:04:36.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.299+0000 7ff3eca5d640 1 -- 172.21.15.32:0/2896083408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3e81566b0 con 0x7ff3e815e9b0 2024-04-24T23:04:36.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.300+0000 7ff3d6ffd640 1 -- 172.21.15.32:0/2896083408 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff3e00077c0 con 0x7ff3e815e9b0 2024-04-24T23:04:36.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.300+0000 7ff3d6ffd640 1 --2- 172.21.15.32:0/2896083408 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff3c806be30 0x7ff3c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:36.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.300+0000 7ff3d6ffd640 1 -- 172.21.15.32:0/2896083408 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7ff3e008abf0 con 0x7ff3e815e9b0 2024-04-24T23:04:36.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.300+0000 7ff3e659c640 1 --2- 172.21.15.32:0/2896083408 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff3c806be30 0x7ff3c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:36.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.301+0000 7ff3e659c640 1 --2- 172.21.15.32:0/2896083408 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff3c806be30 0x7ff3c806e2f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8034000 tx=0x7ff3d802f000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:36.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.304+0000 7ff3d6ffd640 1 -- 172.21.15.32:0/2896083408 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff3e81566b0 con 0x7ff3e815e9b0 2024-04-24T23:04:36.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.523+0000 7ff3eca5d640 1 -- 172.21.15.32:0/2896083408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff3e8068430 con 0x7ff3e815e9b0 2024-04-24T23:04:36.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.524+0000 7ff3d6ffd640 1 -- 172.21.15.32:0/2896083408 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5186 (secure 0 0 0) 0x7ff3e00596f0 con 0x7ff3e815e9b0 2024-04-24T23:04:36.526 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:04:36.526 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":44,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6175,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/434273738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":434273738},{"type":"v1","addr":"172.21.15.121:6836","nonce":434273738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6178,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1104557785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1104557785},{"type":"v1","addr":"172.21.15.121:6837","nonce":1104557785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6888,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/3137388886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3137388886},{"type":"v1","addr":"172.21.15.32:6834","nonce":3137388886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:04:00.783579+0000","modified":"2024-04-24T23:04:03.790658+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6891},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6891":{"gid":6891,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/3327203898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":3327203898},{"type":"v1","addr":"172.21.15.32:6835","nonce":3327203898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6891,"qdb_cluster":[6891]},"id":6}]} 2024-04-24T23:04:36.527 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 44 2024-04-24T23:04:36.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.526+0000 7ff3eca5d640 1 -- 172.21.15.32:0/2896083408 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff3c806be30 msgr2=0x7ff3c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:36.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.526+0000 7ff3eca5d640 1 --2- 172.21.15.32:0/2896083408 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff3c806be30 0x7ff3c806e2f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8034000 tx=0x7ff3d802f000 comp rx=0 tx=0).stop 2024-04-24T23:04:36.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.526+0000 7ff3eca5d640 1 -- 172.21.15.32:0/2896083408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3e815e9b0 msgr2=0x7ff3e81f4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:36.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.526+0000 7ff3eca5d640 1 --2- 172.21.15.32:0/2896083408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3e815e9b0 0x7ff3e81f4800 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7ff3e000c7c0 tx=0x7ff3e000cc80 comp rx=0 tx=0).stop 2024-04-24T23:04:36.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.526+0000 7ff3eca5d640 1 -- 172.21.15.32:0/2896083408 shutdown_connections 2024-04-24T23:04:36.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.526+0000 7ff3eca5d640 1 --2- 172.21.15.32:0/2896083408 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff3c806be30 0x7ff3c806e2f0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:36.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.526+0000 7ff3eca5d640 1 --2- 172.21.15.32:0/2896083408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3e815e9b0 0x7ff3e81f4800 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:36.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.526+0000 7ff3eca5d640 1 --2- 172.21.15.32:0/2896083408 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff3e80d4210 0x7ff3e8159d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:36.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.526+0000 7ff3eca5d640 1 --2- 172.21.15.32:0/2896083408 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3e806b820 0x7ff3e80cd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:36.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.526+0000 7ff3eca5d640 1 -- 172.21.15.32:0/2896083408 >> 172.21.15.32:0/2896083408 conn(0x7ff3e80d11d0 msgr2=0x7ff3e8155d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:36.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.526+0000 7ff3eca5d640 1 -- 172.21.15.32:0/2896083408 shutdown_connections 2024-04-24T23:04:36.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:36.526+0000 7ff3eca5d640 1 -- 172.21.15.32:0/2896083408 wait complete. 2024-04-24T23:04:36.539 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-04-24T23:04:36.659 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","whoami":0,"id":6891,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":35.211632977999997,"mdsmap_epoch":44,"osdmap_epoch":113,"osdmap_epoch_barrier":108,"uptime":40.110606672000003} 2024-04-24T23:04:36.660 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", "fs_name": "cephfs", "id": 6891, "mdsmap_epoch": 44, "osdmap_epoch": 113, "osdmap_epoch_barrier": 108, "rank_uptime": 35.211632978, "state": "up:active", "uptime": 40.110606672, "want_state": "up:active", "whoami": 0 } 2024-04-24T23:04:36.660 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-24T23:04:36.660 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:04:36.660 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:04:36.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:04:36.677 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2024-04-24T23:04:36.839 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:04:36.841 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:04:36.920 INFO:teuthology.orchestra.run.smithi032.stderr:50+0 records in 2024-04-24T23:04:36.921 INFO:teuthology.orchestra.run.smithi032.stderr:50+0 records out 2024-04-24T23:04:36.921 INFO:teuthology.orchestra.run.smithi032.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.185607 s, 282 MB/s 2024-04-24T23:04:36.921 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-24T23:04:36.921 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:04:37.004 DEBUG:teuthology.orchestra.run.smithi121:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:04:37.039 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-24T23:04:37.039 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:04:37.039 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:04:37.039 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:04:37.060 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:04:37.060 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:04:37.116 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:04:37.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.256+0000 7f518a04d640 1 Processor -- start 2024-04-24T23:04:37.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.256+0000 7f518a04d640 1 -- start start 2024-04-24T23:04:37.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.257+0000 7f518a04d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f518415b380 0x7f518406c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:37.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.257+0000 7f518a04d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f518415cb00 0x7f518406cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:37.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.257+0000 7f518a04d640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51840b4520 con 0x7f51840ce430 2024-04-24T23:04:37.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.257+0000 7f518a04d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f51840cef90 con 0x7f518415cb00 2024-04-24T23:04:37.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.257+0000 7f518a04d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f51840b4800 con 0x7f518415b380 2024-04-24T23:04:37.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.257+0000 7f5182ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f518415cb00 0x7f518406cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:37.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.257+0000 7f5182ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f518415cb00 0x7f518406cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43986/0 (socket says 172.21.15.32:43986) 2024-04-24T23:04:37.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.257+0000 7f5182ffd640 1 -- 172.21.15.32:0/2264628119 learned_addr learned my addr 172.21.15.32:0/2264628119 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:37.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.257+0000 7f51837fe640 1 --2- 172.21.15.32:0/2264628119 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f518415b380 0x7f518406c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:37.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.257+0000 7f5182ffd640 1 -- 172.21.15.32:0/2264628119 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f518415b380 msgr2=0x7f518406c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:37.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.257+0000 7f5182ffd640 1 --2- 172.21.15.32:0/2264628119 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f518415b380 0x7f518406c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:37.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.257+0000 7f5182ffd640 1 -- 172.21.15.32:0/2264628119 >> v1:172.21.15.121:6789/0 conn(0x7f51840ce430 legacy=0x7f51840ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:37.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.257+0000 7f5182ffd640 1 -- 172.21.15.32:0/2264628119 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51840b4ae0 con 0x7f518415cb00 2024-04-24T23:04:37.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.258+0000 7f5182ffd640 1 --2- 172.21.15.32:0/2264628119 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f518415cb00 0x7f518406cb30 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f5178009870 tx=0x7f517802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=573ded2ce845f618 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:37.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.258+0000 7f51827fc640 1 -- 172.21.15.32:0/2264628119 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5178037070 con 0x7f518415cb00 2024-04-24T23:04:37.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.258+0000 7f51827fc640 1 -- 172.21.15.32:0/2264628119 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5178032440 con 0x7f518415cb00 2024-04-24T23:04:37.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.258+0000 7f51827fc640 1 -- 172.21.15.32:0/2264628119 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5178032810 con 0x7f518415cb00 2024-04-24T23:04:37.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.259+0000 7f518a04d640 1 -- 172.21.15.32:0/2264628119 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f518415cb00 msgr2=0x7f518406cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:37.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.259+0000 7f518a04d640 1 --2- 172.21.15.32:0/2264628119 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f518415cb00 0x7f518406cb30 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f5178009870 tx=0x7f517802a180 comp rx=0 tx=0).stop 2024-04-24T23:04:37.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.259+0000 7f518a04d640 1 -- 172.21.15.32:0/2264628119 shutdown_connections 2024-04-24T23:04:37.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.259+0000 7f518a04d640 1 --2- 172.21.15.32:0/2264628119 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f518415cb00 0x7f518406cb30 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:37.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.259+0000 7f518a04d640 1 --2- 172.21.15.32:0/2264628119 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f518415b380 0x7f518406c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:37.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.259+0000 7f518a04d640 1 -- 172.21.15.32:0/2264628119 >> 172.21.15.32:0/2264628119 conn(0x7f518406b110 msgr2=0x7f518406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:37.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.259+0000 7f518a04d640 1 -- 172.21.15.32:0/2264628119 shutdown_connections 2024-04-24T23:04:37.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.259+0000 7f518a04d640 1 -- 172.21.15.32:0/2264628119 wait complete. 2024-04-24T23:04:37.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.260+0000 7f518a04d640 1 Processor -- start 2024-04-24T23:04:37.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.260+0000 7f518a04d640 1 -- start start 2024-04-24T23:04:37.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.261+0000 7f518a04d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51840ce430 0x7f5184146ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:37.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.261+0000 7f518a04d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f518415b380 0x7f5184071510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:37.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.261+0000 7f518a04d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f518415cb00 0x7f5184155a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:37.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.261+0000 7f518a04d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f518415b760 con 0x7f518415b380 2024-04-24T23:04:37.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.261+0000 7f518a04d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f51840b50a0 con 0x7f518415cb00 2024-04-24T23:04:37.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.261+0000 7f518a04d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f51840b55c0 con 0x7f51840ce430 2024-04-24T23:04:37.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.261+0000 7f5182ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f518415b380 0x7f5184071510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:37.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.261+0000 7f5182ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f518415b380 0x7f5184071510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44000/0 (socket says 172.21.15.32:44000) 2024-04-24T23:04:37.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.261+0000 7f5182ffd640 1 -- 172.21.15.32:0/1257898787 learned_addr learned my addr 172.21.15.32:0/1257898787 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:37.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.261+0000 7f51837fe640 1 --2- 172.21.15.32:0/1257898787 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51840ce430 0x7f5184146ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:37.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.261+0000 7f5183fff640 1 --2- 172.21.15.32:0/1257898787 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f518415cb00 0x7f5184155a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:37.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.261+0000 7f5182ffd640 1 -- 172.21.15.32:0/1257898787 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51840ce430 msgr2=0x7f5184146ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:37.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.261+0000 7f5182ffd640 1 --2- 172.21.15.32:0/1257898787 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51840ce430 0x7f5184146ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:37.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.261+0000 7f5182ffd640 1 -- 172.21.15.32:0/1257898787 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f518415cb00 msgr2=0x7f5184155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:37.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.261+0000 7f5182ffd640 1 --2- 172.21.15.32:0/1257898787 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f518415cb00 0x7f5184155a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:37.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.261+0000 7f5182ffd640 1 -- 172.21.15.32:0/1257898787 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5184071fd0 con 0x7f518415b380 2024-04-24T23:04:37.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.262+0000 7f5182ffd640 1 --2- 172.21.15.32:0/1257898787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f518415b380 0x7f5184071510 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f51780090b0 tx=0x7f51780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:37.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.262+0000 7f516bfff640 1 -- 172.21.15.32:0/1257898787 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5178041020 con 0x7f518415b380 2024-04-24T23:04:37.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.262+0000 7f516bfff640 1 -- 172.21.15.32:0/1257898787 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51780306c0 con 0x7f518415b380 2024-04-24T23:04:37.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.262+0000 7f518a04d640 1 -- 172.21.15.32:0/1257898787 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51840722b0 con 0x7f518415b380 2024-04-24T23:04:37.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.262+0000 7f516bfff640 1 -- 172.21.15.32:0/1257898787 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5178037040 con 0x7f518415b380 2024-04-24T23:04:37.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.262+0000 7f518a04d640 1 -- 172.21.15.32:0/1257898787 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51841470e0 con 0x7f518415b380 2024-04-24T23:04:37.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.263+0000 7f518a04d640 1 -- 172.21.15.32:0/1257898787 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5184068430 con 0x7f518415b380 2024-04-24T23:04:37.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.263+0000 7f516bfff640 1 -- 172.21.15.32:0/1257898787 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5178044020 con 0x7f518415b380 2024-04-24T23:04:37.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.264+0000 7f516bfff640 1 --2- 172.21.15.32:0/1257898787 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f517006be30 0x7f517006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:37.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.264+0000 7f516bfff640 1 -- 172.21.15.32:0/1257898787 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7f51780ab560 con 0x7f518415b380 2024-04-24T23:04:37.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.264+0000 7f51837fe640 1 --2- 172.21.15.32:0/1257898787 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f517006be30 0x7f517006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:37.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.264+0000 7f51837fe640 1 --2- 172.21.15.32:0/1257898787 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f517006be30 0x7f517006e2f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5174006fd0 tx=0x7f5174008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:37.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.268+0000 7f516bfff640 1 -- 172.21.15.32:0/1257898787 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f517804b030 con 0x7f518415b380 2024-04-24T23:04:37.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.454+0000 7f518a04d640 1 -- 172.21.15.32:0/1257898787 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f51840afd60 con 0x7f518415b380 2024-04-24T23:04:37.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.455+0000 7f516bfff640 1 -- 172.21.15.32:0/1257898787 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) ==== 74+0+16696 (secure 0 0 0) 0x7f5178079950 con 0x7f518415b380 2024-04-24T23:04:37.457 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:04:37.457 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":113,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:04:31.618095+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:03:59.079157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-24T23:03:59.765179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:04:37.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.457+0000 7f518a04d640 1 -- 172.21.15.32:0/1257898787 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f517006be30 msgr2=0x7f517006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:37.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.457+0000 7f518a04d640 1 --2- 172.21.15.32:0/1257898787 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f517006be30 0x7f517006e2f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5174006fd0 tx=0x7f5174008040 comp rx=0 tx=0).stop 2024-04-24T23:04:37.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.457+0000 7f518a04d640 1 -- 172.21.15.32:0/1257898787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f518415b380 msgr2=0x7f5184071510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:37.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.457+0000 7f518a04d640 1 --2- 172.21.15.32:0/1257898787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f518415b380 0x7f5184071510 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f51780090b0 tx=0x7f51780046c0 comp rx=0 tx=0).stop 2024-04-24T23:04:37.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.457+0000 7f518a04d640 1 -- 172.21.15.32:0/1257898787 shutdown_connections 2024-04-24T23:04:37.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.457+0000 7f518a04d640 1 --2- 172.21.15.32:0/1257898787 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f517006be30 0x7f517006e2f0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:37.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.457+0000 7f518a04d640 1 --2- 172.21.15.32:0/1257898787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f518415b380 0x7f5184071510 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:37.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.457+0000 7f518a04d640 1 --2- 172.21.15.32:0/1257898787 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f518415cb00 0x7f5184155a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:37.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.457+0000 7f518a04d640 1 --2- 172.21.15.32:0/1257898787 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51840ce430 0x7f5184146ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:37.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.457+0000 7f518a04d640 1 -- 172.21.15.32:0/1257898787 >> 172.21.15.32:0/1257898787 conn(0x7f518406b110 msgr2=0x7f518415e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:37.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.457+0000 7f518a04d640 1 -- 172.21.15.32:0/1257898787 shutdown_connections 2024-04-24T23:04:37.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.457+0000 7f518a04d640 1 -- 172.21.15.32:0/1257898787 wait complete. 2024-04-24T23:04:37.469 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:03:59.079157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:03:59.765179+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:04:37.470 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:04:37.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.572+0000 7f669b516640 1 Processor -- start 2024-04-24T23:04:37.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.572+0000 7f669b516640 1 -- start start 2024-04-24T23:04:37.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.573+0000 7f669b516640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f669415c5b0 0x7f66940b4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:37.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.573+0000 7f669b516640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6694072cf0 0x7f669406c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:37.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.573+0000 7f669b516640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6694071770 0x7f669406cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:37.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.573+0000 7f669b516640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f6694072860 con 0x7f669415c5b0 2024-04-24T23:04:37.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.573+0000 7f669b516640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f669415c990 con 0x7f6694072cf0 2024-04-24T23:04:37.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.573+0000 7f669b516640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f6694071c50 con 0x7f6694071770 2024-04-24T23:04:37.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.573+0000 7f66992b2640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f669415c5b0 0x7f66940b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:37.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.573+0000 7f66992b2640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f669415c5b0 0x7f66940b4030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44002/0 (socket says 172.21.15.32:44002) 2024-04-24T23:04:37.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.573+0000 7f66992b2640 1 -- 172.21.15.32:0/895289183 learned_addr learned my addr 172.21.15.32:0/895289183 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:37.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.573+0000 7f6699ab3640 1 --2- 172.21.15.32:0/895289183 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6694071770 0x7f669406cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:37.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.573+0000 7f66992b2640 1 -- 172.21.15.32:0/895289183 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6694071770 msgr2=0x7f669406cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:37.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.573+0000 7f66992b2640 1 --2- 172.21.15.32:0/895289183 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6694071770 0x7f669406cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:37.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.573+0000 7f66992b2640 1 -- 172.21.15.32:0/895289183 >> v2:172.21.15.121:3300/0 conn(0x7f6694072cf0 msgr2=0x7f669406c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:37.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.573+0000 7f6698ab1640 1 --2- 172.21.15.32:0/895289183 >> v2:172.21.15.121:3300/0 conn(0x7f6694072cf0 0x7f669406c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:37.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.573+0000 7f66992b2640 1 --2- 172.21.15.32:0/895289183 >> v2:172.21.15.121:3300/0 conn(0x7f6694072cf0 0x7f669406c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:37.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.573+0000 7f66992b2640 1 -- 172.21.15.32:0/895289183 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6694071f30 con 0x7f669415c5b0 2024-04-24T23:04:37.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.574+0000 7f66992b2640 1 --2- 172.21.15.32:0/895289183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f669415c5b0 0x7f66940b4030 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f6694160050 tx=0x7f669002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ad14db49b44d4f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:37.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.574+0000 7f668bfff640 1 -- 172.21.15.32:0/895289183 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6690037070 con 0x7f669415c5b0 2024-04-24T23:04:37.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.574+0000 7f668bfff640 1 -- 172.21.15.32:0/895289183 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6690032440 con 0x7f669415c5b0 2024-04-24T23:04:37.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.574+0000 7f668bfff640 1 -- 172.21.15.32:0/895289183 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6690032810 con 0x7f669415c5b0 2024-04-24T23:04:37.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.575+0000 7f669b516640 1 -- 172.21.15.32:0/895289183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f669415c5b0 msgr2=0x7f66940b4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:37.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.575+0000 7f669b516640 1 --2- 172.21.15.32:0/895289183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f669415c5b0 0x7f66940b4030 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f6694160050 tx=0x7f669002a140 comp rx=0 tx=0).stop 2024-04-24T23:04:37.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.575+0000 7f669b516640 1 -- 172.21.15.32:0/895289183 shutdown_connections 2024-04-24T23:04:37.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.575+0000 7f669b516640 1 --2- 172.21.15.32:0/895289183 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6694071770 0x7f669406cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:37.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.575+0000 7f669b516640 1 --2- 172.21.15.32:0/895289183 >> v2:172.21.15.121:3300/0 conn(0x7f6694072cf0 0x7f669406c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:37.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.575+0000 7f669b516640 1 --2- 172.21.15.32:0/895289183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f669415c5b0 0x7f66940b4030 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:37.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.575+0000 7f669b516640 1 -- 172.21.15.32:0/895289183 >> 172.21.15.32:0/895289183 conn(0x7f669406b110 msgr2=0x7f669406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:37.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.575+0000 7f669b516640 1 -- 172.21.15.32:0/895289183 shutdown_connections 2024-04-24T23:04:37.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.575+0000 7f669b516640 1 -- 172.21.15.32:0/895289183 wait complete. 2024-04-24T23:04:37.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.576+0000 7f669b516640 1 Processor -- start 2024-04-24T23:04:37.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.576+0000 7f669b516640 1 -- start start 2024-04-24T23:04:37.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.577+0000 7f669b516640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6694071770 0x7f66941f5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:37.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.577+0000 7f669b516640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6694072cf0 0x7f66941544e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:37.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.577+0000 7f669b516640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f669415c5b0 0x7f66940b0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:37.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.577+0000 7f669b516640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f66940b4570 con 0x7f669415c5b0 2024-04-24T23:04:37.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.577+0000 7f669b516640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f66940724f0 con 0x7f6694072cf0 2024-04-24T23:04:37.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.577+0000 7f669b516640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f66940b46f0 con 0x7f6694071770 2024-04-24T23:04:37.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.577+0000 7f6699ab3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f669415c5b0 0x7f66940b0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:37.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.577+0000 7f6699ab3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f669415c5b0 0x7f66940b0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44010/0 (socket says 172.21.15.32:44010) 2024-04-24T23:04:37.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.577+0000 7f6699ab3640 1 -- 172.21.15.32:0/1670731900 learned_addr learned my addr 172.21.15.32:0/1670731900 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:37.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.577+0000 7f66992b2640 1 --2- 172.21.15.32:0/1670731900 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6694071770 0x7f66941f5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:37.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.577+0000 7f6698ab1640 1 --2- 172.21.15.32:0/1670731900 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6694072cf0 0x7f66941544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:37.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.577+0000 7f6699ab3640 1 -- 172.21.15.32:0/1670731900 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6694071770 msgr2=0x7f66941f5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:37.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.577+0000 7f6699ab3640 1 --2- 172.21.15.32:0/1670731900 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6694071770 0x7f66941f5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:37.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.577+0000 7f6699ab3640 1 -- 172.21.15.32:0/1670731900 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6694072cf0 msgr2=0x7f66941544e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:37.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.577+0000 7f6699ab3640 1 --2- 172.21.15.32:0/1670731900 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6694072cf0 0x7f66941544e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:37.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.577+0000 7f6699ab3640 1 -- 172.21.15.32:0/1670731900 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f669415e4a0 con 0x7f669415c5b0 2024-04-24T23:04:37.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.578+0000 7f6699ab3640 1 --2- 172.21.15.32:0/1670731900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f669415c5b0 0x7f66940b0a10 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f668c00cd20 tx=0x7f668c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:37.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.578+0000 7f6689ffb640 1 -- 172.21.15.32:0/1670731900 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668c003b90 con 0x7f669415c5b0 2024-04-24T23:04:37.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.578+0000 7f6689ffb640 1 -- 172.21.15.32:0/1670731900 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f668c003d30 con 0x7f669415c5b0 2024-04-24T23:04:37.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.578+0000 7f6689ffb640 1 -- 172.21.15.32:0/1670731900 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668c004240 con 0x7f669415c5b0 2024-04-24T23:04:37.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.578+0000 7f669b516640 1 -- 172.21.15.32:0/1670731900 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66940ccf00 con 0x7f669415c5b0 2024-04-24T23:04:37.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.578+0000 7f669b516640 1 -- 172.21.15.32:0/1670731900 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66940cd4c0 con 0x7f669415c5b0 2024-04-24T23:04:37.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.579+0000 7f669b516640 1 -- 172.21.15.32:0/1670731900 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6694068430 con 0x7f669415c5b0 2024-04-24T23:04:37.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.579+0000 7f6689ffb640 1 -- 172.21.15.32:0/1670731900 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f668c0043e0 con 0x7f669415c5b0 2024-04-24T23:04:37.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.580+0000 7f6689ffb640 1 --2- 172.21.15.32:0/1670731900 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f667806be60 0x7f667806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:37.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.580+0000 7f6689ffb640 1 -- 172.21.15.32:0/1670731900 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7f668c08b7c0 con 0x7f669415c5b0 2024-04-24T23:04:37.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.580+0000 7f66992b2640 1 --2- 172.21.15.32:0/1670731900 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f667806be60 0x7f667806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:37.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.580+0000 7f66992b2640 1 --2- 172.21.15.32:0/1670731900 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f667806be60 0x7f667806e320 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f6690034000 tx=0x7f669002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:37.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.584+0000 7f6689ffb640 1 -- 172.21.15.32:0/1670731900 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f668c059b20 con 0x7f669415c5b0 2024-04-24T23:04:37.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.766+0000 7f669b516640 1 -- 172.21.15.32:0/1670731900 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f66940b5500 con 0x7f669415c5b0 2024-04-24T23:04:37.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.767+0000 7f6689ffb640 1 -- 172.21.15.32:0/1670731900 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v113) ==== 93+0+10 (secure 0 0 0) 0x7f668c05de20 con 0x7f669415c5b0 2024-04-24T23:04:37.771 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T23:04:37.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.769+0000 7f669b516640 1 -- 172.21.15.32:0/1670731900 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f667806be60 msgr2=0x7f667806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:37.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.769+0000 7f669b516640 1 --2- 172.21.15.32:0/1670731900 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f667806be60 0x7f667806e320 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f6690034000 tx=0x7f669002e000 comp rx=0 tx=0).stop 2024-04-24T23:04:37.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.769+0000 7f669b516640 1 -- 172.21.15.32:0/1670731900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f669415c5b0 msgr2=0x7f66940b0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:37.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.769+0000 7f669b516640 1 --2- 172.21.15.32:0/1670731900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f669415c5b0 0x7f66940b0a10 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f668c00cd20 tx=0x7f668c00a660 comp rx=0 tx=0).stop 2024-04-24T23:04:37.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.769+0000 7f669b516640 1 -- 172.21.15.32:0/1670731900 shutdown_connections 2024-04-24T23:04:37.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.769+0000 7f669b516640 1 --2- 172.21.15.32:0/1670731900 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f667806be60 0x7f667806e320 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:37.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.769+0000 7f669b516640 1 --2- 172.21.15.32:0/1670731900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f669415c5b0 0x7f66940b0a10 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:37.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.769+0000 7f669b516640 1 --2- 172.21.15.32:0/1670731900 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6694072cf0 0x7f66941544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:37.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.769+0000 7f669b516640 1 --2- 172.21.15.32:0/1670731900 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6694071770 0x7f66941f5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:37.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.769+0000 7f669b516640 1 -- 172.21.15.32:0/1670731900 >> 172.21.15.32:0/1670731900 conn(0x7f669406b110 msgr2=0x7f66940b5d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:37.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.769+0000 7f669b516640 1 -- 172.21.15.32:0/1670731900 shutdown_connections 2024-04-24T23:04:37.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.769+0000 7f669b516640 1 -- 172.21.15.32:0/1670731900 wait complete. 2024-04-24T23:04:37.781 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:04:37.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.882+0000 7f37c39b0640 1 Processor -- start 2024-04-24T23:04:37.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.882+0000 7f37c39b0640 1 -- start start 2024-04-24T23:04:37.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.883+0000 7f37c39b0640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f37bc06b820 0x7f37bc0d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:37.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.883+0000 7f37c39b0640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f37bc0d4210 0x7f37bc0b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:37.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.883+0000 7f37c39b0640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37bc0d3170 con 0x7f37bc15e9b0 2024-04-24T23:04:37.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.883+0000 7f37c39b0640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f37bc0d3450 con 0x7f37bc0d4210 2024-04-24T23:04:37.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.883+0000 7f37c39b0640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f37bc0d3730 con 0x7f37bc06b820 2024-04-24T23:04:37.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.883+0000 7f37c1f4d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f37bc0d4210 0x7f37bc0b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:37.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.883+0000 7f37c174c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f37bc15e9b0 0x7f37bc06b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39174/0 (socket says 172.21.15.32:39174) 2024-04-24T23:04:37.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.883+0000 7f37c1f4d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f37bc0d4210 0x7f37bc0b78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44024/0 (socket says 172.21.15.32:44024) 2024-04-24T23:04:37.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.883+0000 7f37c174c640 1 -- 172.21.15.32:0/1419927103 learned_addr learned my addr 172.21.15.32:0/1419927103 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:04:37.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.884+0000 7f37c1f4d640 1 -- 172.21.15.32:0/1419927103 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f37bc06b820 msgr2=0x7f37bc0d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:37.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.884+0000 7f37c0f4b640 1 --2- 172.21.15.32:0/1419927103 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f37bc06b820 0x7f37bc0d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:37.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.884+0000 7f37bbfff640 1 -- 172.21.15.32:0/1419927103 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 381242410 0 0) 0x7f37bc0d3170 con 0x7f37bc15e9b0 2024-04-24T23:04:37.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.884+0000 7f37c1f4d640 1 --2- 172.21.15.32:0/1419927103 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f37bc06b820 0x7f37bc0d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:37.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.884+0000 7f37c1f4d640 1 -- 172.21.15.32:0/1419927103 >> v1:172.21.15.32:6789/0 conn(0x7f37bc15e9b0 legacy=0x7f37bc06b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:37.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.884+0000 7f37c1f4d640 1 -- 172.21.15.32:0/1419927103 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37bc0d3a10 con 0x7f37bc0d4210 2024-04-24T23:04:37.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.884+0000 7f37c0f4b640 1 --2- 172.21.15.32:0/1419927103 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f37bc06b820 0x7f37bc0d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:04:37.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.884+0000 7f37c1f4d640 1 --2- 172.21.15.32:0/1419927103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37bc0d4210 0x7f37bc0b78f0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f37ac009d90 tx=0x7f37ac02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cff53037148f7ab3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:37.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.885+0000 7f37bbfff640 1 -- 172.21.15.32:0/1419927103 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ac007740 con 0x7f37bc0d4210 2024-04-24T23:04:37.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.885+0000 7f37bbfff640 1 -- 172.21.15.32:0/1419927103 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37ac0078e0 con 0x7f37bc0d4210 2024-04-24T23:04:37.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.885+0000 7f37bbfff640 1 -- 172.21.15.32:0/1419927103 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ac007cb0 con 0x7f37bc0d4210 2024-04-24T23:04:37.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.885+0000 7f37c39b0640 1 -- 172.21.15.32:0/1419927103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37bc0d4210 msgr2=0x7f37bc0b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:37.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.885+0000 7f37c39b0640 1 --2- 172.21.15.32:0/1419927103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37bc0d4210 0x7f37bc0b78f0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f37ac009d90 tx=0x7f37ac02a660 comp rx=0 tx=0).stop 2024-04-24T23:04:37.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.885+0000 7f37c39b0640 1 -- 172.21.15.32:0/1419927103 shutdown_connections 2024-04-24T23:04:37.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.885+0000 7f37c39b0640 1 --2- 172.21.15.32:0/1419927103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37bc0d4210 0x7f37bc0b78f0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:37.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.885+0000 7f37c39b0640 1 --2- 172.21.15.32:0/1419927103 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f37bc06b820 0x7f37bc0d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:37.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.885+0000 7f37c39b0640 1 -- 172.21.15.32:0/1419927103 >> 172.21.15.32:0/1419927103 conn(0x7f37bc0d11d0 msgr2=0x7f37bc0d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:37.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.885+0000 7f37c39b0640 1 -- 172.21.15.32:0/1419927103 shutdown_connections 2024-04-24T23:04:37.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.885+0000 7f37c39b0640 1 -- 172.21.15.32:0/1419927103 wait complete. 2024-04-24T23:04:37.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.886+0000 7f37c39b0640 1 Processor -- start 2024-04-24T23:04:37.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.886+0000 7f37c39b0640 1 -- start start 2024-04-24T23:04:37.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.887+0000 7f37c39b0640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37bc06b820 0x7f37bc0cf240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:37.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.887+0000 7f37c39b0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37bc0d4210 0x7f37bc1f3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:37.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.887+0000 7f37c39b0640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f37bc15e9b0 0x7f37bc0cb820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:37.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.887+0000 7f37c39b0640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc0ba0e0 con 0x7f37bc0d4210 2024-04-24T23:04:37.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.887+0000 7f37c39b0640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc0d2400 con 0x7f37bc06b820 2024-04-24T23:04:37.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.887+0000 7f37c39b0640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f37bc0ba260 con 0x7f37bc15e9b0 2024-04-24T23:04:37.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.887+0000 7f37c0f4b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37bc0d4210 0x7f37bc1f3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:37.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.887+0000 7f37c0f4b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37bc0d4210 0x7f37bc1f3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44038/0 (socket says 172.21.15.32:44038) 2024-04-24T23:04:37.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.887+0000 7f37c0f4b640 1 -- 172.21.15.32:0/1860875924 learned_addr learned my addr 172.21.15.32:0/1860875924 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:37.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.887+0000 7f37c1f4d640 1 --2- 172.21.15.32:0/1860875924 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f37bc15e9b0 0x7f37bc0cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:37.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.887+0000 7f37c174c640 1 --2- 172.21.15.32:0/1860875924 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37bc06b820 0x7f37bc0cf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:37.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.887+0000 7f37c0f4b640 1 -- 172.21.15.32:0/1860875924 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f37bc15e9b0 msgr2=0x7f37bc0cb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:37.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.887+0000 7f37c0f4b640 1 --2- 172.21.15.32:0/1860875924 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f37bc15e9b0 0x7f37bc0cb820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:37.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.888+0000 7f37c0f4b640 1 -- 172.21.15.32:0/1860875924 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37bc06b820 msgr2=0x7f37bc0cf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:37.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.888+0000 7f37c0f4b640 1 --2- 172.21.15.32:0/1860875924 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37bc06b820 0x7f37bc0cf240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:37.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.888+0000 7f37c0f4b640 1 -- 172.21.15.32:0/1860875924 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37bc0cc390 con 0x7f37bc0d4210 2024-04-24T23:04:37.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.888+0000 7f37c1f4d640 1 --2- 172.21.15.32:0/1860875924 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f37bc15e9b0 0x7f37bc0cb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:04:37.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.888+0000 7f37c0f4b640 1 --2- 172.21.15.32:0/1860875924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37bc0d4210 0x7f37bc1f3dc0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f37b000c820 tx=0x7f37b000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:37.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.888+0000 7f37b9ffb640 1 -- 172.21.15.32:0/1860875924 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b0015070 con 0x7f37bc0d4210 2024-04-24T23:04:37.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.888+0000 7f37b9ffb640 1 -- 172.21.15.32:0/1860875924 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37b00079d0 con 0x7f37bc0d4210 2024-04-24T23:04:37.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.888+0000 7f37c39b0640 1 -- 172.21.15.32:0/1860875924 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37bc0cc670 con 0x7f37bc0d4210 2024-04-24T23:04:37.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.888+0000 7f37b9ffb640 1 -- 172.21.15.32:0/1860875924 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b0007da0 con 0x7f37bc0d4210 2024-04-24T23:04:37.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.888+0000 7f37c39b0640 1 -- 172.21.15.32:0/1860875924 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37bc0ccc30 con 0x7f37bc0d4210 2024-04-24T23:04:37.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.889+0000 7f37c39b0640 1 -- 172.21.15.32:0/1860875924 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37bc068430 con 0x7f37bc0d4210 2024-04-24T23:04:37.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.889+0000 7f37b9ffb640 1 -- 172.21.15.32:0/1860875924 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f37b0018070 con 0x7f37bc0d4210 2024-04-24T23:04:37.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.890+0000 7f37b9ffb640 1 --2- 172.21.15.32:0/1860875924 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f37a806be30 0x7f37a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:37.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.890+0000 7f37b9ffb640 1 -- 172.21.15.32:0/1860875924 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7f37b008ad40 con 0x7f37bc0d4210 2024-04-24T23:04:37.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.890+0000 7f37c174c640 1 --2- 172.21.15.32:0/1860875924 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f37a806be30 0x7f37a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:37.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.890+0000 7f37c174c640 1 --2- 172.21.15.32:0/1860875924 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f37a806be30 0x7f37a806e2f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f37b4000f60 tx=0x7f37b4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:37.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:37.894+0000 7f37b9ffb640 1 -- 172.21.15.32:0/1860875924 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f37b0059780 con 0x7f37bc0d4210 2024-04-24T23:04:38.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.075+0000 7f37c39b0640 1 -- 172.21.15.32:0/1860875924 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f37bc0723e0 con 0x7f37bc0d4210 2024-04-24T23:04:38.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.075+0000 7f37b9ffb640 1 -- 172.21.15.32:0/1860875924 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v113) ==== 104+0+11 (secure 0 0 0) 0x7f37b0059080 con 0x7f37bc0d4210 2024-04-24T23:04:38.077 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-24T23:04:38.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.077+0000 7f37c39b0640 1 -- 172.21.15.32:0/1860875924 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f37a806be30 msgr2=0x7f37a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:38.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.077+0000 7f37c39b0640 1 --2- 172.21.15.32:0/1860875924 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f37a806be30 0x7f37a806e2f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f37b4000f60 tx=0x7f37b4009040 comp rx=0 tx=0).stop 2024-04-24T23:04:38.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.077+0000 7f37c39b0640 1 -- 172.21.15.32:0/1860875924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37bc0d4210 msgr2=0x7f37bc1f3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:38.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.077+0000 7f37c39b0640 1 --2- 172.21.15.32:0/1860875924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37bc0d4210 0x7f37bc1f3dc0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f37b000c820 tx=0x7f37b000cc20 comp rx=0 tx=0).stop 2024-04-24T23:04:38.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.077+0000 7f37c39b0640 1 -- 172.21.15.32:0/1860875924 shutdown_connections 2024-04-24T23:04:38.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.077+0000 7f37c39b0640 1 --2- 172.21.15.32:0/1860875924 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f37a806be30 0x7f37a806e2f0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:38.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.077+0000 7f37c39b0640 1 --2- 172.21.15.32:0/1860875924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37bc0d4210 0x7f37bc1f3dc0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:38.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.077+0000 7f37c39b0640 1 --2- 172.21.15.32:0/1860875924 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f37bc15e9b0 0x7f37bc0cb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:38.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.077+0000 7f37c39b0640 1 --2- 172.21.15.32:0/1860875924 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37bc06b820 0x7f37bc0cf240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:38.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.077+0000 7f37c39b0640 1 -- 172.21.15.32:0/1860875924 >> 172.21.15.32:0/1860875924 conn(0x7f37bc0d11d0 msgr2=0x7f37bc0d15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:38.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.077+0000 7f37c39b0640 1 -- 172.21.15.32:0/1860875924 shutdown_connections 2024-04-24T23:04:38.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.077+0000 7f37c39b0640 1 -- 172.21.15.32:0/1860875924 wait complete. 2024-04-24T23:04:38.090 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:04:38.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.191+0000 7f1feb142640 1 Processor -- start 2024-04-24T23:04:38.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.191+0000 7f1feb142640 1 -- start start 2024-04-24T23:04:38.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.192+0000 7f1feb142640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1fe415b380 0x7f1fe406c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:38.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.192+0000 7f1feb142640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1fe40ceb40 0x7f1fe406cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:38.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.192+0000 7f1feb142640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fe40b4520 con 0x7f1fe415cb00 2024-04-24T23:04:38.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.192+0000 7f1feb142640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f1fe40cef70 con 0x7f1fe415b380 2024-04-24T23:04:38.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.192+0000 7f1feb142640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f1fe40b4800 con 0x7f1fe40ceb40 2024-04-24T23:04:38.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.192+0000 7f1fe8ede640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1fe415b380 0x7f1fe406c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:38.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.192+0000 7f1fe8ede640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1fe415b380 0x7f1fe406c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44054/0 (socket says 172.21.15.32:44054) 2024-04-24T23:04:38.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.192+0000 7f1fe8ede640 1 -- 172.21.15.32:0/1371670156 learned_addr learned my addr 172.21.15.32:0/1371670156 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:38.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.192+0000 7f1fe96df640 1 --2- 172.21.15.32:0/1371670156 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1fe40ceb40 0x7f1fe406cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:38.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.192+0000 7f1fe8ede640 1 -- 172.21.15.32:0/1371670156 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1fe40ceb40 msgr2=0x7f1fe406cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:38.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.192+0000 7f1fe8ede640 1 --2- 172.21.15.32:0/1371670156 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1fe40ceb40 0x7f1fe406cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:38.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.192+0000 7f1fe8ede640 1 -- 172.21.15.32:0/1371670156 >> v1:172.21.15.121:6789/0 conn(0x7f1fe415cb00 legacy=0x7f1fe40ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:38.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.192+0000 7f1fe8ede640 1 -- 172.21.15.32:0/1371670156 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe40b4ae0 con 0x7f1fe415b380 2024-04-24T23:04:38.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.192+0000 7f1fe96df640 1 --2- 172.21.15.32:0/1371670156 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1fe40ceb40 0x7f1fe406cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:04:38.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.193+0000 7f1fe8ede640 1 --2- 172.21.15.32:0/1371670156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fe415b380 0x7f1fe406c5f0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc009e50 tx=0x7f1fdc02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f219555cf20e081 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:38.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.193+0000 7f1fe37fe640 1 -- 172.21.15.32:0/1371670156 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc036070 con 0x7f1fe415b380 2024-04-24T23:04:38.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.193+0000 7f1fe37fe640 1 -- 172.21.15.32:0/1371670156 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fdc031600 con 0x7f1fe415b380 2024-04-24T23:04:38.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.193+0000 7f1fe37fe640 1 -- 172.21.15.32:0/1371670156 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc0319d0 con 0x7f1fe415b380 2024-04-24T23:04:38.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.194+0000 7f1feb142640 1 -- 172.21.15.32:0/1371670156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fe415b380 msgr2=0x7f1fe406c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:38.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.194+0000 7f1feb142640 1 --2- 172.21.15.32:0/1371670156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fe415b380 0x7f1fe406c5f0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc009e50 tx=0x7f1fdc02a5f0 comp rx=0 tx=0).stop 2024-04-24T23:04:38.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.194+0000 7f1feb142640 1 -- 172.21.15.32:0/1371670156 shutdown_connections 2024-04-24T23:04:38.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.194+0000 7f1feb142640 1 --2- 172.21.15.32:0/1371670156 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1fe40ceb40 0x7f1fe406cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:38.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.194+0000 7f1feb142640 1 --2- 172.21.15.32:0/1371670156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fe415b380 0x7f1fe406c5f0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:38.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.194+0000 7f1feb142640 1 -- 172.21.15.32:0/1371670156 >> 172.21.15.32:0/1371670156 conn(0x7f1fe406b110 msgr2=0x7f1fe406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:38.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.194+0000 7f1feb142640 1 -- 172.21.15.32:0/1371670156 shutdown_connections 2024-04-24T23:04:38.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.194+0000 7f1feb142640 1 -- 172.21.15.32:0/1371670156 wait complete. 2024-04-24T23:04:38.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.195+0000 7f1feb142640 1 Processor -- start 2024-04-24T23:04:38.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.195+0000 7f1feb142640 1 -- start start 2024-04-24T23:04:38.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.196+0000 7f1feb142640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fe40ceb40 0x7f1fe4148570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:38.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.196+0000 7f1feb142640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fe415b380 0x7f1fe415d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:38.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.196+0000 7f1feb142640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1fe415cb00 0x7f1fe415d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:38.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.196+0000 7f1feb142640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe415b760 con 0x7f1fe40ceb40 2024-04-24T23:04:38.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.196+0000 7f1feb142640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe40b50a0 con 0x7f1fe415b380 2024-04-24T23:04:38.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.196+0000 7f1feb142640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f1fe40b55c0 con 0x7f1fe415cb00 2024-04-24T23:04:38.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.196+0000 7f1fe8ede640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fe40ceb40 0x7f1fe4148570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:38.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.196+0000 7f1fe8ede640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fe40ceb40 0x7f1fe4148570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44066/0 (socket says 172.21.15.32:44066) 2024-04-24T23:04:38.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.196+0000 7f1fe8ede640 1 -- 172.21.15.32:0/3513411145 learned_addr learned my addr 172.21.15.32:0/3513411145 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:38.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.196+0000 7f1fe8ede640 1 -- 172.21.15.32:0/3513411145 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1fe415cb00 msgr2=0x7f1fe415d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:38.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.196+0000 7f1fe8ede640 1 --2- 172.21.15.32:0/3513411145 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1fe415cb00 0x7f1fe415d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:38.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.196+0000 7f1fe8ede640 1 -- 172.21.15.32:0/3513411145 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fe415b380 msgr2=0x7f1fe415d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:38.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.196+0000 7f1fe8ede640 1 --2- 172.21.15.32:0/3513411145 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fe415b380 0x7f1fe415d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:38.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.196+0000 7f1fe8ede640 1 -- 172.21.15.32:0/3513411145 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe4154fc0 con 0x7f1fe40ceb40 2024-04-24T23:04:38.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.196+0000 7f1fe8ede640 1 --2- 172.21.15.32:0/3513411145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fe40ceb40 0x7f1fe4148570 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc034040 tx=0x7f1fdc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:38.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.197+0000 7f1fe17fa640 1 -- 172.21.15.32:0/3513411145 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc036050 con 0x7f1fe40ceb40 2024-04-24T23:04:38.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.197+0000 7f1fe17fa640 1 -- 172.21.15.32:0/3513411145 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fdc042070 con 0x7f1fe40ceb40 2024-04-24T23:04:38.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.197+0000 7f1fe17fa640 1 -- 172.21.15.32:0/3513411145 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc007c50 con 0x7f1fe40ceb40 2024-04-24T23:04:38.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.197+0000 7f1feb142640 1 -- 172.21.15.32:0/3513411145 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fe41552a0 con 0x7f1fe40ceb40 2024-04-24T23:04:38.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.197+0000 7f1feb142640 1 -- 172.21.15.32:0/3513411145 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fe4155860 con 0x7f1fe40ceb40 2024-04-24T23:04:38.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.198+0000 7f1feb142640 1 -- 172.21.15.32:0/3513411145 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fe4068430 con 0x7f1fe40ceb40 2024-04-24T23:04:38.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.198+0000 7f1fe17fa640 1 -- 172.21.15.32:0/3513411145 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1fdc047460 con 0x7f1fe40ceb40 2024-04-24T23:04:38.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.199+0000 7f1fe17fa640 1 --2- 172.21.15.32:0/3513411145 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1fc406be30 0x7f1fc406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:38.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.199+0000 7f1fe17fa640 1 -- 172.21.15.32:0/3513411145 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7f1fdc0abce0 con 0x7f1fe40ceb40 2024-04-24T23:04:38.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.199+0000 7f1fe3fff640 1 --2- 172.21.15.32:0/3513411145 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1fc406be30 0x7f1fc406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:38.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.199+0000 7f1fe3fff640 1 --2- 172.21.15.32:0/3513411145 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1fc406be30 0x7f1fc406e2f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc000f60 tx=0x7f1fcc0079d0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:38.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.203+0000 7f1fe17fa640 1 -- 172.21.15.32:0/3513411145 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1fdc07a720 con 0x7f1fe40ceb40 2024-04-24T23:04:38.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.383+0000 7f1feb142640 1 -- 172.21.15.32:0/3513411145 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1fe40afd60 con 0x7f1fe40ceb40 2024-04-24T23:04:38.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.383+0000 7f1fe17fa640 1 -- 172.21.15.32:0/3513411145 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v113) ==== 100+0+11 (secure 0 0 0) 0x7f1fdc07a020 con 0x7f1fe40ceb40 2024-04-24T23:04:38.385 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-24T23:04:38.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.385+0000 7f1feb142640 1 -- 172.21.15.32:0/3513411145 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1fc406be30 msgr2=0x7f1fc406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:38.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.385+0000 7f1feb142640 1 --2- 172.21.15.32:0/3513411145 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1fc406be30 0x7f1fc406e2f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc000f60 tx=0x7f1fcc0079d0 comp rx=0 tx=0).stop 2024-04-24T23:04:38.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.385+0000 7f1feb142640 1 -- 172.21.15.32:0/3513411145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fe40ceb40 msgr2=0x7f1fe4148570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:38.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.385+0000 7f1feb142640 1 --2- 172.21.15.32:0/3513411145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fe40ceb40 0x7f1fe4148570 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc034040 tx=0x7f1fdc004a40 comp rx=0 tx=0).stop 2024-04-24T23:04:38.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.385+0000 7f1feb142640 1 -- 172.21.15.32:0/3513411145 shutdown_connections 2024-04-24T23:04:38.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.385+0000 7f1feb142640 1 --2- 172.21.15.32:0/3513411145 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1fc406be30 0x7f1fc406e2f0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:38.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.385+0000 7f1feb142640 1 --2- 172.21.15.32:0/3513411145 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1fe415cb00 0x7f1fe415d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:38.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.385+0000 7f1feb142640 1 --2- 172.21.15.32:0/3513411145 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fe415b380 0x7f1fe415d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:38.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.385+0000 7f1feb142640 1 --2- 172.21.15.32:0/3513411145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fe40ceb40 0x7f1fe4148570 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:38.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.385+0000 7f1feb142640 1 -- 172.21.15.32:0/3513411145 >> 172.21.15.32:0/3513411145 conn(0x7f1fe406b110 msgr2=0x7f1fe40b5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:38.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.385+0000 7f1feb142640 1 -- 172.21.15.32:0/3513411145 shutdown_connections 2024-04-24T23:04:38.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.385+0000 7f1feb142640 1 -- 172.21.15.32:0/3513411145 wait complete. 2024-04-24T23:04:38.398 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T23:04:38.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.498+0000 7f93f1410640 1 Processor -- start 2024-04-24T23:04:38.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.498+0000 7f93f1410640 1 -- start start 2024-04-24T23:04:38.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.498+0000 7f93f1410640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f93ec06cd00 0x7f93ec0ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:38.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.498+0000 7f93f1410640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f93ec15cb00 0x7f93ec0ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:38.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.498+0000 7f93f1410640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93ec0b4520 con 0x7f93ec15b380 2024-04-24T23:04:38.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.498+0000 7f93f1410640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f93ec0b4800 con 0x7f93ec06cd00 2024-04-24T23:04:38.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.498+0000 7f93f1410640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f93ec0b4ae0 con 0x7f93ec15cb00 2024-04-24T23:04:38.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.499+0000 7f93ea7fc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f93ec06cd00 0x7f93ec0ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:38.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.499+0000 7f93ea7fc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f93ec06cd00 0x7f93ec0ce430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44078/0 (socket says 172.21.15.32:44078) 2024-04-24T23:04:38.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.499+0000 7f93ea7fc640 1 -- 172.21.15.32:0/2058866492 learned_addr learned my addr 172.21.15.32:0/2058866492 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:38.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.499+0000 7f93eb7fe640 1 --2- 172.21.15.32:0/2058866492 >> v2:172.21.15.121:3300/0 conn(0x7f93ec15cb00 0x7f93ec0ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:38.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.499+0000 7f93ea7fc640 1 -- 172.21.15.32:0/2058866492 >> v2:172.21.15.121:3300/0 conn(0x7f93ec15cb00 msgr2=0x7f93ec0ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:38.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.499+0000 7f93ea7fc640 1 --2- 172.21.15.32:0/2058866492 >> v2:172.21.15.121:3300/0 conn(0x7f93ec15cb00 0x7f93ec0ce970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:38.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.499+0000 7f93ea7fc640 1 -- 172.21.15.32:0/2058866492 >> v1:172.21.15.121:6789/0 conn(0x7f93ec15b380 legacy=0x7f93ec06c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:38.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.499+0000 7f93ea7fc640 1 -- 172.21.15.32:0/2058866492 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93ec0b4dc0 con 0x7f93ec06cd00 2024-04-24T23:04:38.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.500+0000 7f93ea7fc640 1 --2- 172.21.15.32:0/2058866492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93ec06cd00 0x7f93ec0ce430 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f93ec0cdd20 tx=0x7f93e002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a04c96708721bfa3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:38.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.500+0000 7f93e9ffb640 1 -- 172.21.15.32:0/2058866492 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e0037070 con 0x7f93ec06cd00 2024-04-24T23:04:38.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.500+0000 7f93e9ffb640 1 -- 172.21.15.32:0/2058866492 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93e0032440 con 0x7f93ec06cd00 2024-04-24T23:04:38.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.500+0000 7f93e9ffb640 1 -- 172.21.15.32:0/2058866492 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e0032810 con 0x7f93ec06cd00 2024-04-24T23:04:38.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.500+0000 7f93f1410640 1 -- 172.21.15.32:0/2058866492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93ec06cd00 msgr2=0x7f93ec0ce430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:38.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.500+0000 7f93f1410640 1 --2- 172.21.15.32:0/2058866492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93ec06cd00 0x7f93ec0ce430 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f93ec0cdd20 tx=0x7f93e002a1e0 comp rx=0 tx=0).stop 2024-04-24T23:04:38.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.500+0000 7f93f1410640 1 -- 172.21.15.32:0/2058866492 shutdown_connections 2024-04-24T23:04:38.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.500+0000 7f93f1410640 1 --2- 172.21.15.32:0/2058866492 >> v2:172.21.15.121:3300/0 conn(0x7f93ec15cb00 0x7f93ec0ce970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:38.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.500+0000 7f93f1410640 1 --2- 172.21.15.32:0/2058866492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93ec06cd00 0x7f93ec0ce430 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:38.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.500+0000 7f93f1410640 1 -- 172.21.15.32:0/2058866492 >> 172.21.15.32:0/2058866492 conn(0x7f93ec06b110 msgr2=0x7f93ec06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:38.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.500+0000 7f93f1410640 1 -- 172.21.15.32:0/2058866492 shutdown_connections 2024-04-24T23:04:38.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.500+0000 7f93f1410640 1 -- 172.21.15.32:0/2058866492 wait complete. 2024-04-24T23:04:38.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.502+0000 7f93f1410640 1 Processor -- start 2024-04-24T23:04:38.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.502+0000 7f93f1410640 1 -- start start 2024-04-24T23:04:38.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.502+0000 7f93f1410640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93ec06cd00 0x7f93ec156690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:38.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.502+0000 7f93f1410640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93ec15b380 0x7f93ec140a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:38.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.502+0000 7f93f1410640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93ec15cb00 0x7f93ec154b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:38.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.502+0000 7f93f1410640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f93ec15b760 con 0x7f93ec15cb00 2024-04-24T23:04:38.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.502+0000 7f93f1410640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f93ec0ceeb0 con 0x7f93ec06cd00 2024-04-24T23:04:38.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.502+0000 7f93f1410640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f93ec0b55c0 con 0x7f93ec15b380 2024-04-24T23:04:38.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.503+0000 7f93eb7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93ec15cb00 0x7f93ec154b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:38.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.503+0000 7f93eb7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93ec15cb00 0x7f93ec154b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44094/0 (socket says 172.21.15.32:44094) 2024-04-24T23:04:38.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.503+0000 7f93eb7fe640 1 -- 172.21.15.32:0/3147659756 learned_addr learned my addr 172.21.15.32:0/3147659756 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:38.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.503+0000 7f93ea7fc640 1 --2- 172.21.15.32:0/3147659756 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93ec15b380 0x7f93ec140a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:38.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.503+0000 7f93eaffd640 1 --2- 172.21.15.32:0/3147659756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93ec06cd00 0x7f93ec156690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:38.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.503+0000 7f93eb7fe640 1 -- 172.21.15.32:0/3147659756 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93ec15b380 msgr2=0x7f93ec140a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:38.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.503+0000 7f93eb7fe640 1 --2- 172.21.15.32:0/3147659756 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93ec15b380 0x7f93ec140a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:38.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.503+0000 7f93eb7fe640 1 -- 172.21.15.32:0/3147659756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93ec06cd00 msgr2=0x7f93ec156690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:38.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.503+0000 7f93eb7fe640 1 --2- 172.21.15.32:0/3147659756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93ec06cd00 0x7f93ec156690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:38.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.503+0000 7f93eb7fe640 1 -- 172.21.15.32:0/3147659756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93ec1555f0 con 0x7f93ec15cb00 2024-04-24T23:04:38.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.503+0000 7f93eb7fe640 1 --2- 172.21.15.32:0/3147659756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93ec15cb00 0x7f93ec154b30 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f93dc007e60 tx=0x7f93dc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:38.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.503+0000 7f93d37fe640 1 -- 172.21.15.32:0/3147659756 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93dc014420 con 0x7f93ec15cb00 2024-04-24T23:04:38.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.503+0000 7f93d37fe640 1 -- 172.21.15.32:0/3147659756 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93dc019070 con 0x7f93ec15cb00 2024-04-24T23:04:38.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.503+0000 7f93f1410640 1 -- 172.21.15.32:0/3147659756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93ec1558d0 con 0x7f93ec15cb00 2024-04-24T23:04:38.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.503+0000 7f93d37fe640 1 -- 172.21.15.32:0/3147659756 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93dc014820 con 0x7f93ec15cb00 2024-04-24T23:04:38.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.503+0000 7f93f1410640 1 -- 172.21.15.32:0/3147659756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93ec155e90 con 0x7f93ec15cb00 2024-04-24T23:04:38.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.504+0000 7f93f1410640 1 -- 172.21.15.32:0/3147659756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93ec068430 con 0x7f93ec15cb00 2024-04-24T23:04:38.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.505+0000 7f93d37fe640 1 -- 172.21.15.32:0/3147659756 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f93dc0149c0 con 0x7f93ec15cb00 2024-04-24T23:04:38.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.505+0000 7f93d37fe640 1 --2- 172.21.15.32:0/3147659756 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f93d806be30 0x7f93d806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:38.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.505+0000 7f93d37fe640 1 -- 172.21.15.32:0/3147659756 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7f93dc08b5c0 con 0x7f93ec15cb00 2024-04-24T23:04:38.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.506+0000 7f93eaffd640 1 --2- 172.21.15.32:0/3147659756 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f93d806be30 0x7f93d806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:38.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.506+0000 7f93eaffd640 1 --2- 172.21.15.32:0/3147659756 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f93d806be30 0x7f93d806e2f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f93e4001780 tx=0x7f93e4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:38.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.510+0000 7f93d37fe640 1 -- 172.21.15.32:0/3147659756 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f93dc059890 con 0x7f93ec15cb00 2024-04-24T23:04:38.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.708+0000 7f93f1410640 1 -- 172.21.15.32:0/3147659756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f93ec158540 con 0x7f93ec15cb00 2024-04-24T23:04:38.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.708+0000 7f93d37fe640 1 -- 172.21.15.32:0/3147659756 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v113) ==== 80+0+0 (secure 0 0 0) 0x7f93dc05db90 con 0x7f93ec15cb00 2024-04-24T23:04:38.710 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2024-04-24T23:04:38.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.710+0000 7f93f1410640 1 -- 172.21.15.32:0/3147659756 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f93d806be30 msgr2=0x7f93d806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:38.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.710+0000 7f93f1410640 1 --2- 172.21.15.32:0/3147659756 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f93d806be30 0x7f93d806e2f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f93e4001780 tx=0x7f93e4009040 comp rx=0 tx=0).stop 2024-04-24T23:04:38.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.710+0000 7f93f1410640 1 -- 172.21.15.32:0/3147659756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93ec15cb00 msgr2=0x7f93ec154b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:38.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.710+0000 7f93f1410640 1 --2- 172.21.15.32:0/3147659756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93ec15cb00 0x7f93ec154b30 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f93dc007e60 tx=0x7f93dc00e4b0 comp rx=0 tx=0).stop 2024-04-24T23:04:38.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.711+0000 7f93f1410640 1 -- 172.21.15.32:0/3147659756 shutdown_connections 2024-04-24T23:04:38.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.711+0000 7f93f1410640 1 --2- 172.21.15.32:0/3147659756 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f93d806be30 0x7f93d806e2f0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:38.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.711+0000 7f93f1410640 1 --2- 172.21.15.32:0/3147659756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93ec15cb00 0x7f93ec154b30 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:38.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.711+0000 7f93f1410640 1 --2- 172.21.15.32:0/3147659756 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93ec15b380 0x7f93ec140a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:38.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.711+0000 7f93f1410640 1 --2- 172.21.15.32:0/3147659756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93ec06cd00 0x7f93ec156690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:38.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.711+0000 7f93f1410640 1 -- 172.21.15.32:0/3147659756 >> 172.21.15.32:0/3147659756 conn(0x7f93ec06b110 msgr2=0x7f93ec15f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:38.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.711+0000 7f93f1410640 1 -- 172.21.15.32:0/3147659756 shutdown_connections 2024-04-24T23:04:38.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.711+0000 7f93f1410640 1 -- 172.21.15.32:0/3147659756 wait complete. 2024-04-24T23:04:38.723 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-24T23:04:38.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:04:38.723 DEBUG:teuthology.orchestra.run.smithi032:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-24T23:04:38.767 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2024-04-24T23:04:38.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:04:38.767 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-24T23:04:38.823 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:04:38.823 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:04:38.823 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:04:38.841 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:04:38.841 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:04:38.898 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:04:38.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.998+0000 7f9fd559e640 1 Processor -- start 2024-04-24T23:04:39.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.998+0000 7f9fd559e640 1 -- start start 2024-04-24T23:04:39.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.999+0000 7f9fd559e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9fd015c2e0 0x7f9fd00d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:39.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.999+0000 7f9fd559e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fd015f670 con 0x7f9fd00b3ee0 2024-04-24T23:04:39.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.999+0000 7f9fd559e640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fd015f950 con 0x7f9fd015bf00 2024-04-24T23:04:39.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.999+0000 7f9fd559e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f9fd00d2140 con 0x7f9fd015c2e0 2024-04-24T23:04:39.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.999+0000 7f9fd4d9d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9fd00b3ee0 0x7f9fd01621a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39182/0 (socket says 172.21.15.32:39182) 2024-04-24T23:04:39.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.999+0000 7f9fd4d9d640 1 -- 172.21.15.32:0/2413201850 learned_addr learned my addr 172.21.15.32:0/2413201850 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:04:39.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.999+0000 7f9fcf7fe640 1 --2- 172.21.15.32:0/2413201850 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9fd015c2e0 0x7f9fd00d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:39.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.999+0000 7f9fceffd640 1 -- 172.21.15.32:0/2413201850 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3086269601 0 0) 0x7f9fd015f670 con 0x7f9fd00b3ee0 2024-04-24T23:04:39.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.999+0000 7f9fceffd640 1 -- 172.21.15.32:0/2413201850 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9fb8003520 con 0x7f9fd00b3ee0 2024-04-24T23:04:39.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.999+0000 7f9fceffd640 1 -- 172.21.15.32:0/2413201850 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 838558448 0 0) 0x7f9fd015f950 con 0x7f9fd015bf00 2024-04-24T23:04:39.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.999+0000 7f9fceffd640 1 -- 172.21.15.32:0/2413201850 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9fd015f670 con 0x7f9fd015bf00 2024-04-24T23:04:39.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.999+0000 7f9fceffd640 1 -- 172.21.15.32:0/2413201850 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 319286410 0 0) 0x7f9fb8003520 con 0x7f9fd00b3ee0 2024-04-24T23:04:39.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.999+0000 7f9fceffd640 1 -- 172.21.15.32:0/2413201850 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9fd015c2e0 msgr2=0x7f9fd00d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:39.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.999+0000 7f9fceffd640 1 --2- 172.21.15.32:0/2413201850 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9fd015c2e0 0x7f9fd00d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:39.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.999+0000 7f9fceffd640 1 -- 172.21.15.32:0/2413201850 >> v1:172.21.15.121:6789/0 conn(0x7f9fd015bf00 legacy=0x7f9fd015ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:39.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.999+0000 7f9fceffd640 1 -- 172.21.15.32:0/2413201850 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fd00d2390 con 0x7f9fd00b3ee0 2024-04-24T23:04:39.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.999+0000 7f9fceffd640 1 -- 172.21.15.32:0/2413201850 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f9fc80019f0 con 0x7f9fd00b3ee0 2024-04-24T23:04:39.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.999+0000 7f9fceffd640 1 -- 172.21.15.32:0/2413201850 >> v1:172.21.15.32:6789/0 conn(0x7f9fd00b3ee0 legacy=0x7f9fd01621a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:39.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.999+0000 7f9fceffd640 1 --2- 172.21.15.32:0/2413201850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fb8003b70 0x7f9fb8003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:39.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.999+0000 7f9fceffd640 1 --2- 172.21.15.32:0/2413201850 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9fb8004800 0x7f9fb8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:39.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.999+0000 7f9fceffd640 1 --2- 172.21.15.32:0/2413201850 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9fb80051c0 0x7f9fb800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:39.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.999+0000 7f9fceffd640 1 -- 172.21.15.32:0/2413201850 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd00d2140 con 0x7f9fb8003b70 2024-04-24T23:04:39.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.999+0000 7f9fceffd640 1 -- 172.21.15.32:0/2413201850 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb800bfb0 con 0x7f9fb8004800 2024-04-24T23:04:39.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:38.999+0000 7f9fceffd640 1 -- 172.21.15.32:0/2413201850 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f9fb800c1b0 con 0x7f9fb80051c0 2024-04-24T23:04:39.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.000+0000 7f9fcf7fe640 1 --2- 172.21.15.32:0/2413201850 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9fd015c2e0 0x7f9fd00d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:04:39.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.000+0000 7f9fcffff640 1 --2- 172.21.15.32:0/2413201850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fb8003b70 0x7f9fb8003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:39.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.000+0000 7f9fd4d9d640 1 --2- 172.21.15.32:0/2413201850 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9fb80051c0 0x7f9fb800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:39.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.000+0000 7f9fcf7fe640 1 --2- 172.21.15.32:0/2413201850 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9fb8004800 0x7f9fb8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:39.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.000+0000 7f9fcffff640 1 -- 172.21.15.32:0/2413201850 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9fb80051c0 msgr2=0x7f9fb800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:39.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.000+0000 7f9fcffff640 1 --2- 172.21.15.32:0/2413201850 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9fb80051c0 0x7f9fb800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:39.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.000+0000 7f9fcffff640 1 -- 172.21.15.32:0/2413201850 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9fb8004800 msgr2=0x7f9fb8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:39.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.000+0000 7f9fcffff640 1 --2- 172.21.15.32:0/2413201850 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9fb8004800 0x7f9fb8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:39.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.000+0000 7f9fcffff640 1 -- 172.21.15.32:0/2413201850 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fb8003520 con 0x7f9fb8003b70 2024-04-24T23:04:39.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.001+0000 7f9fcffff640 1 --2- 172.21.15.32:0/2413201850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fb8003b70 0x7f9fb8003f70 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f9fc00018c0 tx=0x7f9fb8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a64536867aeb4e1f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:39.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.001+0000 7f9fceffd640 1 -- 172.21.15.32:0/2413201850 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc00014e0 con 0x7f9fb8003b70 2024-04-24T23:04:39.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.001+0000 7f9fceffd640 1 -- 172.21.15.32:0/2413201850 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fc00052f0 con 0x7f9fb8003b70 2024-04-24T23:04:39.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.001+0000 7f9fceffd640 1 -- 172.21.15.32:0/2413201850 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc002ac90 con 0x7f9fb8003b70 2024-04-24T23:04:39.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.001+0000 7f9fd559e640 1 -- 172.21.15.32:0/2413201850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fb8003b70 msgr2=0x7f9fb8003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:39.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.001+0000 7f9fd559e640 1 --2- 172.21.15.32:0/2413201850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fb8003b70 0x7f9fb8003f70 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f9fc00018c0 tx=0x7f9fb8005830 comp rx=0 tx=0).stop 2024-04-24T23:04:39.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.001+0000 7f9fd4d9d640 1 -- 172.21.15.32:0/2413201850 reap_dead start 2024-04-24T23:04:39.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.001+0000 7f9fd559e640 1 -- 172.21.15.32:0/2413201850 shutdown_connections 2024-04-24T23:04:39.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.001+0000 7f9fd559e640 1 -- 172.21.15.32:0/2413201850 >> 172.21.15.32:0/2413201850 conn(0x7f9fd00743f0 msgr2=0x7f9fd006d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:39.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.001+0000 7f9fd559e640 1 -- 172.21.15.32:0/2413201850 shutdown_connections 2024-04-24T23:04:39.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.001+0000 7f9fd559e640 1 -- 172.21.15.32:0/2413201850 wait complete. 2024-04-24T23:04:39.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.003+0000 7f9fd559e640 1 Processor -- start 2024-04-24T23:04:39.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.003+0000 7f9fd559e640 1 -- start start 2024-04-24T23:04:39.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.004+0000 7f9fd559e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9fd0144340 0x7f9fd01623e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:39.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.004+0000 7f9fd559e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9fd0144720 0x7f9fd00d1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:39.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.004+0000 7f9fd559e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fd0160e40 0x7f9fd00cec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:39.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.004+0000 7f9fd559e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd015f2c0 con 0x7f9fd0160e40 2024-04-24T23:04:39.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.004+0000 7f9fd559e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd0072da0 con 0x7f9fd0144340 2024-04-24T23:04:39.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.004+0000 7f9fd559e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f9fd0161f50 con 0x7f9fd0144720 2024-04-24T23:04:39.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.004+0000 7f9fd4d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fd0160e40 0x7f9fd00cec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:39.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.004+0000 7f9fd4d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fd0160e40 0x7f9fd00cec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44104/0 (socket says 172.21.15.32:44104) 2024-04-24T23:04:39.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.004+0000 7f9fd4d9d640 1 -- 172.21.15.32:0/2605037940 learned_addr learned my addr 172.21.15.32:0/2605037940 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:39.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.004+0000 7f9fcffff640 1 --2- 172.21.15.32:0/2605037940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9fd0144340 0x7f9fd01623e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:39.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.004+0000 7f9fcf7fe640 1 --2- 172.21.15.32:0/2605037940 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9fd0144720 0x7f9fd00d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:39.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.004+0000 7f9fd4d9d640 1 -- 172.21.15.32:0/2605037940 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9fd0144720 msgr2=0x7f9fd00d1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:39.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.004+0000 7f9fd4d9d640 1 --2- 172.21.15.32:0/2605037940 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9fd0144720 0x7f9fd00d1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:39.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.004+0000 7f9fd4d9d640 1 -- 172.21.15.32:0/2605037940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9fd0144340 msgr2=0x7f9fd01623e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:39.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.004+0000 7f9fd4d9d640 1 --2- 172.21.15.32:0/2605037940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9fd0144340 0x7f9fd01623e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:39.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.004+0000 7f9fd4d9d640 1 -- 172.21.15.32:0/2605037940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fd0161340 con 0x7f9fd0160e40 2024-04-24T23:04:39.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.004+0000 7f9fd4d9d640 1 --2- 172.21.15.32:0/2605037940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fd0160e40 0x7f9fd00cec30 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8000f80 tx=0x7f9fc800a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:39.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.004+0000 7f9fccff9640 1 -- 172.21.15.32:0/2605037940 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc80133f0 con 0x7f9fd0160e40 2024-04-24T23:04:39.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.004+0000 7f9fccff9640 1 -- 172.21.15.32:0/2605037940 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fc8019070 con 0x7f9fd0160e40 2024-04-24T23:04:39.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.004+0000 7f9fccff9640 1 -- 172.21.15.32:0/2605037940 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc8013810 con 0x7f9fd0160e40 2024-04-24T23:04:39.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.004+0000 7f9fd559e640 1 -- 172.21.15.32:0/2605037940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fd0161620 con 0x7f9fd0160e40 2024-04-24T23:04:39.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.005+0000 7f9fd559e640 1 -- 172.21.15.32:0/2605037940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fd0161be0 con 0x7f9fd0160e40 2024-04-24T23:04:39.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.005+0000 7f9fd559e640 1 -- 172.21.15.32:0/2605037940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fd00682b0 con 0x7f9fd0160e40 2024-04-24T23:04:39.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.006+0000 7f9fccff9640 1 -- 172.21.15.32:0/2605037940 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9fc80139b0 con 0x7f9fd0160e40 2024-04-24T23:04:39.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.006+0000 7f9fccff9640 1 --2- 172.21.15.32:0/2605037940 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9fbc06be30 0x7f9fbc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:39.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.006+0000 7f9fccff9640 1 -- 172.21.15.32:0/2605037940 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7f9fc808b7a0 con 0x7f9fd0160e40 2024-04-24T23:04:39.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.007+0000 7f9fcffff640 1 --2- 172.21.15.32:0/2605037940 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9fbc06be30 0x7f9fbc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:39.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.007+0000 7f9fcffff640 1 --2- 172.21.15.32:0/2605037940 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9fbc06be30 0x7f9fbc06e2f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0032000 tx=0x7f9fc002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:39.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.011+0000 7f9fccff9640 1 -- 172.21.15.32:0/2605037940 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9fc805a150 con 0x7f9fd0160e40 2024-04-24T23:04:39.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.197+0000 7f9fd559e640 1 -- 172.21.15.32:0/2605037940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9fd006ca20 con 0x7f9fd0160e40 2024-04-24T23:04:39.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.199+0000 7f9fccff9640 1 -- 172.21.15.32:0/2605037940 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) ==== 74+0+16696 (secure 0 0 0) 0x7f9fc8059a50 con 0x7f9fd0160e40 2024-04-24T23:04:39.201 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:04:39.201 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":113,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:04:31.618095+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:03:59.079157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-24T23:03:59.765179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:04:39.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.201+0000 7f9fd559e640 1 -- 172.21.15.32:0/2605037940 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9fbc06be30 msgr2=0x7f9fbc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:39.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.201+0000 7f9fd559e640 1 --2- 172.21.15.32:0/2605037940 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9fbc06be30 0x7f9fbc06e2f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0032000 tx=0x7f9fc002e000 comp rx=0 tx=0).stop 2024-04-24T23:04:39.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.201+0000 7f9fd559e640 1 -- 172.21.15.32:0/2605037940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fd0160e40 msgr2=0x7f9fd00cec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:39.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.201+0000 7f9fd559e640 1 --2- 172.21.15.32:0/2605037940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fd0160e40 0x7f9fd00cec30 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8000f80 tx=0x7f9fc800a6a0 comp rx=0 tx=0).stop 2024-04-24T23:04:39.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.201+0000 7f9fd559e640 1 -- 172.21.15.32:0/2605037940 shutdown_connections 2024-04-24T23:04:39.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.201+0000 7f9fd559e640 1 --2- 172.21.15.32:0/2605037940 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9fbc06be30 0x7f9fbc06e2f0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:39.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.201+0000 7f9fd559e640 1 --2- 172.21.15.32:0/2605037940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fd0160e40 0x7f9fd00cec30 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:39.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.201+0000 7f9fd559e640 1 --2- 172.21.15.32:0/2605037940 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9fd0144720 0x7f9fd00d1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:39.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.201+0000 7f9fd559e640 1 --2- 172.21.15.32:0/2605037940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9fd0144340 0x7f9fd01623e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:39.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.201+0000 7f9fd559e640 1 -- 172.21.15.32:0/2605037940 >> 172.21.15.32:0/2605037940 conn(0x7f9fd00743f0 msgr2=0x7f9fd015c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:39.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.201+0000 7f9fd559e640 1 -- 172.21.15.32:0/2605037940 shutdown_connections 2024-04-24T23:04:39.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.201+0000 7f9fd559e640 1 -- 172.21.15.32:0/2605037940 wait complete. 2024-04-24T23:04:39.214 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:03:59.079157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:03:59.765179+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:04:39.214 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:04:39.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.317+0000 7f1e6b1b0640 1 Processor -- start 2024-04-24T23:04:39.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.317+0000 7f1e6b1b0640 1 -- start start 2024-04-24T23:04:39.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.317+0000 7f1e6b1b0640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1e640b44d0 0x7f1e640b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:39.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.317+0000 7f1e6b1b0640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e640b6430 con 0x7f1e6406c5f0 2024-04-24T23:04:39.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.317+0000 7f1e6b1b0640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e64156f80 con 0x7f1e640af890 2024-04-24T23:04:39.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.317+0000 7f1e6b1b0640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f1e640b6150 con 0x7f1e640b44d0 2024-04-24T23:04:39.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.318+0000 7f1e63fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1e640b44d0 0x7f1e640b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:39.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.318+0000 7f1e63fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1e640b44d0 0x7f1e640b5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44116/0 (socket says 172.21.15.32:44116) 2024-04-24T23:04:39.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.318+0000 7f1e63fff640 1 -- 172.21.15.32:0/3975405955 learned_addr learned my addr 172.21.15.32:0/3975405955 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:39.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.318+0000 7f1e68f4c640 1 --1- 172.21.15.32:0/3975405955 >> v1:172.21.15.32:6789/0 conn(0x7f1e6406c5f0 0x7f1e6406c9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39190/0 (socket says 172.21.15.32:39190) 2024-04-24T23:04:39.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.318+0000 7f1e63fff640 1 -- 172.21.15.32:0/3975405955 >> v1:172.21.15.121:6789/0 conn(0x7f1e640af890 legacy=0x7f1e6415cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:39.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.318+0000 7f1e63fff640 1 -- 172.21.15.32:0/3975405955 >> v1:172.21.15.32:6789/0 conn(0x7f1e6406c5f0 legacy=0x7f1e6406c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:39.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.318+0000 7f1e63fff640 1 -- 172.21.15.32:0/3975405955 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e640b39f0 con 0x7f1e640b44d0 2024-04-24T23:04:39.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.318+0000 7f1e637fe640 1 -- 172.21.15.32:0/3975405955 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3859012862 0 0) 0x7f1e640b6430 con 0x7f1e6406c5f0 2024-04-24T23:04:39.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.319+0000 7f1e63fff640 1 --2- 172.21.15.32:0/3975405955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e640b44d0 0x7f1e640b5890 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f1e50009870 tx=0x7f1e5002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f4e95420318f0f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:39.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.319+0000 7f1e637fe640 1 -- 172.21.15.32:0/3975405955 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e50037070 con 0x7f1e640b44d0 2024-04-24T23:04:39.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.319+0000 7f1e637fe640 1 -- 172.21.15.32:0/3975405955 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e50032440 con 0x7f1e640b44d0 2024-04-24T23:04:39.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.319+0000 7f1e637fe640 1 -- 172.21.15.32:0/3975405955 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e50032810 con 0x7f1e640b44d0 2024-04-24T23:04:39.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.319+0000 7f1e6b1b0640 1 -- 172.21.15.32:0/3975405955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e640b44d0 msgr2=0x7f1e640b5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:39.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.319+0000 7f1e6b1b0640 1 --2- 172.21.15.32:0/3975405955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e640b44d0 0x7f1e640b5890 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f1e50009870 tx=0x7f1e5002a2a0 comp rx=0 tx=0).stop 2024-04-24T23:04:39.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.319+0000 7f1e6b1b0640 1 -- 172.21.15.32:0/3975405955 shutdown_connections 2024-04-24T23:04:39.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.319+0000 7f1e6b1b0640 1 --2- 172.21.15.32:0/3975405955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e640b44d0 0x7f1e640b5890 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:39.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.319+0000 7f1e6b1b0640 1 -- 172.21.15.32:0/3975405955 >> 172.21.15.32:0/3975405955 conn(0x7f1e6406b110 msgr2=0x7f1e6406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:39.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.319+0000 7f1e6b1b0640 1 -- 172.21.15.32:0/3975405955 shutdown_connections 2024-04-24T23:04:39.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.319+0000 7f1e6b1b0640 1 -- 172.21.15.32:0/3975405955 wait complete. 2024-04-24T23:04:39.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.320+0000 7f1e6b1b0640 1 Processor -- start 2024-04-24T23:04:39.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.321+0000 7f1e6b1b0640 1 -- start start 2024-04-24T23:04:39.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.321+0000 7f1e6b1b0640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e6406c5f0 0x7f1e6414c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:39.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.321+0000 7f1e6b1b0640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1e640af890 0x7f1e64158650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:39.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.321+0000 7f1e6b1b0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e640b44d0 0x7f1e640b0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:39.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.321+0000 7f1e6b1b0640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1e640b5dd0 con 0x7f1e640b44d0 2024-04-24T23:04:39.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.321+0000 7f1e6b1b0640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1e640b3fb0 con 0x7f1e6406c5f0 2024-04-24T23:04:39.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.321+0000 7f1e6b1b0640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f1e6415d1d0 con 0x7f1e640af890 2024-04-24T23:04:39.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.321+0000 7f1e6974d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e640b44d0 0x7f1e640b0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:39.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.321+0000 7f1e6974d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e640b44d0 0x7f1e640b0fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44122/0 (socket says 172.21.15.32:44122) 2024-04-24T23:04:39.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.322+0000 7f1e6974d640 1 -- 172.21.15.32:0/894566313 learned_addr learned my addr 172.21.15.32:0/894566313 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:39.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.322+0000 7f1e68f4c640 1 --2- 172.21.15.32:0/894566313 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e6406c5f0 0x7f1e6414c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:39.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.322+0000 7f1e63fff640 1 --2- 172.21.15.32:0/894566313 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1e640af890 0x7f1e64158650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:39.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.322+0000 7f1e6974d640 1 -- 172.21.15.32:0/894566313 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1e640af890 msgr2=0x7f1e64158650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:39.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.322+0000 7f1e6974d640 1 --2- 172.21.15.32:0/894566313 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1e640af890 0x7f1e64158650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:39.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.322+0000 7f1e6974d640 1 -- 172.21.15.32:0/894566313 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e6406c5f0 msgr2=0x7f1e6414c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:39.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.322+0000 7f1e6974d640 1 --2- 172.21.15.32:0/894566313 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e6406c5f0 0x7f1e6414c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:39.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.322+0000 7f1e6974d640 1 -- 172.21.15.32:0/894566313 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e64159130 con 0x7f1e640b44d0 2024-04-24T23:04:39.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.322+0000 7f1e6974d640 1 --2- 172.21.15.32:0/894566313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e640b44d0 0x7f1e640b0fb0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c00c780 tx=0x7f1e5c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:39.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.322+0000 7f1e617fa640 1 -- 172.21.15.32:0/894566313 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c004b60 con 0x7f1e640b44d0 2024-04-24T23:04:39.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.322+0000 7f1e617fa640 1 -- 172.21.15.32:0/894566313 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e5c004d00 con 0x7f1e640b44d0 2024-04-24T23:04:39.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.322+0000 7f1e617fa640 1 -- 172.21.15.32:0/894566313 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c00aa90 con 0x7f1e640b44d0 2024-04-24T23:04:39.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.322+0000 7f1e6b1b0640 1 -- 172.21.15.32:0/894566313 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e64159410 con 0x7f1e640b44d0 2024-04-24T23:04:39.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.322+0000 7f1e6b1b0640 1 -- 172.21.15.32:0/894566313 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e641599d0 con 0x7f1e640b44d0 2024-04-24T23:04:39.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.323+0000 7f1e6b1b0640 1 -- 172.21.15.32:0/894566313 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e64068430 con 0x7f1e640b44d0 2024-04-24T23:04:39.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.324+0000 7f1e617fa640 1 -- 172.21.15.32:0/894566313 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1e5c00ac30 con 0x7f1e640b44d0 2024-04-24T23:04:39.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.324+0000 7f1e617fa640 1 --2- 172.21.15.32:0/894566313 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1e4806be30 0x7f1e4806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:39.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.324+0000 7f1e617fa640 1 -- 172.21.15.32:0/894566313 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7f1e5c08b0d0 con 0x7f1e640b44d0 2024-04-24T23:04:39.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.324+0000 7f1e68f4c640 1 --2- 172.21.15.32:0/894566313 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1e4806be30 0x7f1e4806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:39.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.325+0000 7f1e68f4c640 1 --2- 172.21.15.32:0/894566313 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1e4806be30 0x7f1e4806e2f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f1e58000f60 tx=0x7f1e58007b50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:39.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.329+0000 7f1e617fa640 1 -- 172.21.15.32:0/894566313 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1e5c020070 con 0x7f1e640b44d0 2024-04-24T23:04:39.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.510+0000 7f1e6b1b0640 1 -- 172.21.15.32:0/894566313 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1e640b5670 con 0x7f1e640b44d0 2024-04-24T23:04:39.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.510+0000 7f1e617fa640 1 -- 172.21.15.32:0/894566313 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v113) ==== 93+0+10 (secure 0 0 0) 0x7f1e5c059b10 con 0x7f1e640b44d0 2024-04-24T23:04:39.512 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T23:04:39.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.512+0000 7f1e6b1b0640 1 -- 172.21.15.32:0/894566313 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1e4806be30 msgr2=0x7f1e4806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:39.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.512+0000 7f1e6b1b0640 1 --2- 172.21.15.32:0/894566313 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1e4806be30 0x7f1e4806e2f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f1e58000f60 tx=0x7f1e58007b50 comp rx=0 tx=0).stop 2024-04-24T23:04:39.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.512+0000 7f1e6b1b0640 1 -- 172.21.15.32:0/894566313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e640b44d0 msgr2=0x7f1e640b0fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:39.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.512+0000 7f1e6b1b0640 1 --2- 172.21.15.32:0/894566313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e640b44d0 0x7f1e640b0fb0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c00c780 tx=0x7f1e5c00cc40 comp rx=0 tx=0).stop 2024-04-24T23:04:39.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.512+0000 7f1e6b1b0640 1 -- 172.21.15.32:0/894566313 shutdown_connections 2024-04-24T23:04:39.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.512+0000 7f1e6b1b0640 1 --2- 172.21.15.32:0/894566313 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1e4806be30 0x7f1e4806e2f0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:39.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.512+0000 7f1e6b1b0640 1 --2- 172.21.15.32:0/894566313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e640b44d0 0x7f1e640b0fb0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:39.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.512+0000 7f1e6b1b0640 1 --2- 172.21.15.32:0/894566313 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1e640af890 0x7f1e64158650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:39.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.512+0000 7f1e6b1b0640 1 --2- 172.21.15.32:0/894566313 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e6406c5f0 0x7f1e6414c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:39.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.512+0000 7f1e6b1b0640 1 -- 172.21.15.32:0/894566313 >> 172.21.15.32:0/894566313 conn(0x7f1e6406b110 msgr2=0x7f1e640b4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:39.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.512+0000 7f1e6b1b0640 1 -- 172.21.15.32:0/894566313 shutdown_connections 2024-04-24T23:04:39.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.512+0000 7f1e6b1b0640 1 -- 172.21.15.32:0/894566313 wait complete. 2024-04-24T23:04:39.525 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:04:39.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.627+0000 7f56a1edf640 1 Processor -- start 2024-04-24T23:04:39.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.627+0000 7f56a1edf640 1 -- start start 2024-04-24T23:04:39.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.628+0000 7f56a1edf640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f569c0b5af0 0x7f569c0ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:39.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.628+0000 7f56a1edf640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f569c162e10 0x7f569c15ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:39.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.628+0000 7f56a1edf640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f569c0cd270 con 0x7f569c162320 2024-04-24T23:04:39.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.628+0000 7f56a1edf640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f569c0b5660 con 0x7f569c162e10 2024-04-24T23:04:39.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.628+0000 7f56a1edf640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f569c071610 con 0x7f569c0b5af0 2024-04-24T23:04:39.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.628+0000 7f569bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f569c162e10 0x7f569c15ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:39.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.628+0000 7f569bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f569c162e10 0x7f569c15ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44126/0 (socket says 172.21.15.32:44126) 2024-04-24T23:04:39.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.628+0000 7f569bfff640 1 -- 172.21.15.32:0/3291720441 learned_addr learned my addr 172.21.15.32:0/3291720441 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:39.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.628+0000 7f569affd640 1 --2- 172.21.15.32:0/3291720441 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f569c0b5af0 0x7f569c0ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:39.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.628+0000 7f569bfff640 1 -- 172.21.15.32:0/3291720441 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f569c0b5af0 msgr2=0x7f569c0ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:39.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.628+0000 7f569bfff640 1 --2- 172.21.15.32:0/3291720441 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f569c0b5af0 0x7f569c0ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:39.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.628+0000 7f569bfff640 1 -- 172.21.15.32:0/3291720441 >> v1:172.21.15.121:6789/0 conn(0x7f569c162320 legacy=0x7f569c162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:39.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.628+0000 7f569bfff640 1 -- 172.21.15.32:0/3291720441 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f569c0718f0 con 0x7f569c162e10 2024-04-24T23:04:39.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.628+0000 7f569affd640 1 --2- 172.21.15.32:0/3291720441 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f569c0b5af0 0x7f569c0ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:04:39.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.629+0000 7f569bfff640 1 --2- 172.21.15.32:0/3291720441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f569c162e10 0x7f569c15ed30 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f5694009e50 tx=0x7f569402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e90844fa17b39f6f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:39.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.629+0000 7f569a7fc640 1 -- 172.21.15.32:0/3291720441 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5694036070 con 0x7f569c162e10 2024-04-24T23:04:39.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.629+0000 7f569a7fc640 1 -- 172.21.15.32:0/3291720441 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5694031600 con 0x7f569c162e10 2024-04-24T23:04:39.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.629+0000 7f569a7fc640 1 -- 172.21.15.32:0/3291720441 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56940319d0 con 0x7f569c162e10 2024-04-24T23:04:39.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.630+0000 7f56a1edf640 1 -- 172.21.15.32:0/3291720441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f569c162e10 msgr2=0x7f569c15ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:39.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.630+0000 7f56a1edf640 1 --2- 172.21.15.32:0/3291720441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f569c162e10 0x7f569c15ed30 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f5694009e50 tx=0x7f569402a5f0 comp rx=0 tx=0).stop 2024-04-24T23:04:39.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.630+0000 7f56a1edf640 1 -- 172.21.15.32:0/3291720441 shutdown_connections 2024-04-24T23:04:39.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.630+0000 7f56a1edf640 1 --2- 172.21.15.32:0/3291720441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f569c162e10 0x7f569c15ed30 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:39.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.630+0000 7f56a1edf640 1 --2- 172.21.15.32:0/3291720441 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f569c0b5af0 0x7f569c0ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:39.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.630+0000 7f56a1edf640 1 -- 172.21.15.32:0/3291720441 >> 172.21.15.32:0/3291720441 conn(0x7f569c0d0ae0 msgr2=0x7f569c0d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:39.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.630+0000 7f56a1edf640 1 -- 172.21.15.32:0/3291720441 shutdown_connections 2024-04-24T23:04:39.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.630+0000 7f56a1edf640 1 -- 172.21.15.32:0/3291720441 wait complete. 2024-04-24T23:04:39.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.631+0000 7f56a1edf640 1 Processor -- start 2024-04-24T23:04:39.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.631+0000 7f56a1edf640 1 -- start start 2024-04-24T23:04:39.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.632+0000 7f56a1edf640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f569c0b5af0 0x7f569c15b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:39.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.632+0000 7f56a1edf640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f569c162320 0x7f569c15aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:39.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.632+0000 7f56a1edf640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f569c162e10 0x7f569c1e91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:39.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.632+0000 7f56a1edf640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f569c071bd0 con 0x7f569c162320 2024-04-24T23:04:39.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.632+0000 7f56a1edf640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f569c067950 con 0x7f569c0b5af0 2024-04-24T23:04:39.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.632+0000 7f56a1edf640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f569c071d50 con 0x7f569c162e10 2024-04-24T23:04:39.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.632+0000 7f569affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f569c162320 0x7f569c15aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:39.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.632+0000 7f569affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f569c162320 0x7f569c15aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44140/0 (socket says 172.21.15.32:44140) 2024-04-24T23:04:39.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.632+0000 7f569affd640 1 -- 172.21.15.32:0/2332432079 learned_addr learned my addr 172.21.15.32:0/2332432079 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:39.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.632+0000 7f569bfff640 1 --2- 172.21.15.32:0/2332432079 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f569c162e10 0x7f569c1e91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:39.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.632+0000 7f569b7fe640 1 --2- 172.21.15.32:0/2332432079 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f569c0b5af0 0x7f569c15b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:39.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.633+0000 7f569affd640 1 -- 172.21.15.32:0/2332432079 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f569c162e10 msgr2=0x7f569c1e91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:39.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.633+0000 7f569affd640 1 --2- 172.21.15.32:0/2332432079 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f569c162e10 0x7f569c1e91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:39.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.633+0000 7f569affd640 1 -- 172.21.15.32:0/2332432079 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f569c0b5af0 msgr2=0x7f569c15b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:39.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.633+0000 7f569affd640 1 --2- 172.21.15.32:0/2332432079 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f569c0b5af0 0x7f569c15b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:39.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.633+0000 7f569affd640 1 -- 172.21.15.32:0/2332432079 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f569c0ce450 con 0x7f569c162320 2024-04-24T23:04:39.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.633+0000 7f569affd640 1 --2- 172.21.15.32:0/2332432079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f569c162320 0x7f569c15aa10 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f569000c7f0 tx=0x7f569000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:39.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.633+0000 7f5683fff640 1 -- 172.21.15.32:0/2332432079 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5690015070 con 0x7f569c162320 2024-04-24T23:04:39.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.633+0000 7f5683fff640 1 -- 172.21.15.32:0/2332432079 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5690007a30 con 0x7f569c162320 2024-04-24T23:04:39.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.633+0000 7f56a1edf640 1 -- 172.21.15.32:0/2332432079 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f569c0ce730 con 0x7f569c162320 2024-04-24T23:04:39.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.633+0000 7f5683fff640 1 -- 172.21.15.32:0/2332432079 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5690007e00 con 0x7f569c162320 2024-04-24T23:04:39.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.633+0000 7f56a1edf640 1 -- 172.21.15.32:0/2332432079 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f569c0cefd0 con 0x7f569c162320 2024-04-24T23:04:39.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.634+0000 7f56a1edf640 1 -- 172.21.15.32:0/2332432079 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f569c0cf1d0 con 0x7f569c162320 2024-04-24T23:04:39.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.635+0000 7f5683fff640 1 -- 172.21.15.32:0/2332432079 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5690018070 con 0x7f569c162320 2024-04-24T23:04:39.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.635+0000 7f5683fff640 1 --2- 172.21.15.32:0/2332432079 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f568806be30 0x7f568806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:39.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.635+0000 7f5683fff640 1 -- 172.21.15.32:0/2332432079 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7f569008ad60 con 0x7f569c162320 2024-04-24T23:04:39.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.635+0000 7f569b7fe640 1 --2- 172.21.15.32:0/2332432079 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f568806be30 0x7f568806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:39.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.636+0000 7f569b7fe640 1 --2- 172.21.15.32:0/2332432079 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f568806be30 0x7f568806e2f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f569c1f7c60 tx=0x7f568c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:39.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.640+0000 7f5683fff640 1 -- 172.21.15.32:0/2332432079 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f569c0cf1d0 con 0x7f569c162320 2024-04-24T23:04:39.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.823+0000 7f56a1edf640 1 -- 172.21.15.32:0/2332432079 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f569c068430 con 0x7f569c162320 2024-04-24T23:04:39.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.823+0000 7f5683fff640 1 -- 172.21.15.32:0/2332432079 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v113) ==== 104+0+11 (secure 0 0 0) 0x7f56900590c0 con 0x7f569c162320 2024-04-24T23:04:39.826 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-24T23:04:39.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.825+0000 7f56a1edf640 1 -- 172.21.15.32:0/2332432079 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f568806be30 msgr2=0x7f568806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:39.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.825+0000 7f56a1edf640 1 --2- 172.21.15.32:0/2332432079 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f568806be30 0x7f568806e2f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f569c1f7c60 tx=0x7f568c009040 comp rx=0 tx=0).stop 2024-04-24T23:04:39.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.825+0000 7f56a1edf640 1 -- 172.21.15.32:0/2332432079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f569c162320 msgr2=0x7f569c15aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:39.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.825+0000 7f56a1edf640 1 --2- 172.21.15.32:0/2332432079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f569c162320 0x7f569c15aa10 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f569000c7f0 tx=0x7f569000ccb0 comp rx=0 tx=0).stop 2024-04-24T23:04:39.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.825+0000 7f56a1edf640 1 -- 172.21.15.32:0/2332432079 shutdown_connections 2024-04-24T23:04:39.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.825+0000 7f56a1edf640 1 --2- 172.21.15.32:0/2332432079 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f568806be30 0x7f568806e2f0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:39.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.825+0000 7f56a1edf640 1 --2- 172.21.15.32:0/2332432079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f569c162320 0x7f569c15aa10 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:39.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.825+0000 7f56a1edf640 1 --2- 172.21.15.32:0/2332432079 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f569c162e10 0x7f569c1e91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:39.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.825+0000 7f56a1edf640 1 --2- 172.21.15.32:0/2332432079 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f569c0b5af0 0x7f569c15b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:39.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.825+0000 7f56a1edf640 1 -- 172.21.15.32:0/2332432079 >> 172.21.15.32:0/2332432079 conn(0x7f569c0d0ae0 msgr2=0x7f569c06ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:39.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.825+0000 7f56a1edf640 1 -- 172.21.15.32:0/2332432079 shutdown_connections 2024-04-24T23:04:39.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.825+0000 7f56a1edf640 1 -- 172.21.15.32:0/2332432079 wait complete. 2024-04-24T23:04:39.838 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:04:39.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.939+0000 7f819a590640 1 Processor -- start 2024-04-24T23:04:39.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.939+0000 7f819a590640 1 -- start start 2024-04-24T23:04:39.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.940+0000 7f819a590640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f81940b44d0 0x7f81940b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:39.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.940+0000 7f819a590640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f81940af890 0x7f819415cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:39.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.940+0000 7f819a590640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81940b6430 con 0x7f819406c5f0 2024-04-24T23:04:39.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.940+0000 7f819a590640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f8194156f80 con 0x7f81940af890 2024-04-24T23:04:39.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.940+0000 7f819a590640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f81940b6150 con 0x7f81940b44d0 2024-04-24T23:04:39.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.940+0000 7f8198b2d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f81940af890 0x7f819415cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:39.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.940+0000 7f8198b2d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f81940af890 0x7f819415cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46890/0 (socket says 172.21.15.32:46890) 2024-04-24T23:04:39.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.940+0000 7f8198b2d640 1 -- 172.21.15.32:0/3924650716 learned_addr learned my addr 172.21.15.32:0/3924650716 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:39.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.940+0000 7f81937fe640 1 --2- 172.21.15.32:0/3924650716 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f81940b44d0 0x7f81940b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:39.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.941+0000 7f8198b2d640 1 -- 172.21.15.32:0/3924650716 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f81940b44d0 msgr2=0x7f81940b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:39.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.941+0000 7f8198b2d640 1 --2- 172.21.15.32:0/3924650716 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f81940b44d0 0x7f81940b5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:39.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.941+0000 7f8198b2d640 1 -- 172.21.15.32:0/3924650716 >> v1:172.21.15.121:6789/0 conn(0x7f819406c5f0 legacy=0x7f819406c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:39.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.941+0000 7f8198b2d640 1 -- 172.21.15.32:0/3924650716 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81940b39f0 con 0x7f81940af890 2024-04-24T23:04:39.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.941+0000 7f81937fe640 1 --2- 172.21.15.32:0/3924650716 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f81940b44d0 0x7f81940b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:04:39.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.941+0000 7f8198b2d640 1 --2- 172.21.15.32:0/3924650716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81940af890 0x7f819415cac0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f818c009e50 tx=0x7f818c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31511659bfe139e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:39.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.942+0000 7f8192ffd640 1 -- 172.21.15.32:0/3924650716 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818c036070 con 0x7f81940af890 2024-04-24T23:04:39.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.942+0000 7f8192ffd640 1 -- 172.21.15.32:0/3924650716 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f818c031600 con 0x7f81940af890 2024-04-24T23:04:39.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.942+0000 7f8192ffd640 1 -- 172.21.15.32:0/3924650716 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818c0319d0 con 0x7f81940af890 2024-04-24T23:04:39.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.942+0000 7f819a590640 1 -- 172.21.15.32:0/3924650716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81940af890 msgr2=0x7f819415cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:39.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.942+0000 7f819a590640 1 --2- 172.21.15.32:0/3924650716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81940af890 0x7f819415cac0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f818c009e50 tx=0x7f818c02a680 comp rx=0 tx=0).stop 2024-04-24T23:04:39.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.942+0000 7f819a590640 1 -- 172.21.15.32:0/3924650716 shutdown_connections 2024-04-24T23:04:39.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.942+0000 7f819a590640 1 --2- 172.21.15.32:0/3924650716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81940af890 0x7f819415cac0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:39.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.942+0000 7f819a590640 1 --2- 172.21.15.32:0/3924650716 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f81940b44d0 0x7f81940b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:39.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.942+0000 7f819a590640 1 -- 172.21.15.32:0/3924650716 >> 172.21.15.32:0/3924650716 conn(0x7f819406b110 msgr2=0x7f819406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:39.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.942+0000 7f819a590640 1 -- 172.21.15.32:0/3924650716 shutdown_connections 2024-04-24T23:04:39.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.942+0000 7f819a590640 1 -- 172.21.15.32:0/3924650716 wait complete. 2024-04-24T23:04:39.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.943+0000 7f819a590640 1 Processor -- start 2024-04-24T23:04:39.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.944+0000 7f819a590640 1 -- start start 2024-04-24T23:04:39.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.944+0000 7f819a590640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f819406c5f0 0x7f81941f2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:39.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.944+0000 7f819a590640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f81940af890 0x7f81941ef5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:39.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.944+0000 7f819a590640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81940b44d0 0x7f8194152220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:39.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.944+0000 7f819a590640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f81940b5dd0 con 0x7f81940b44d0 2024-04-24T23:04:39.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.944+0000 7f819a590640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f81940b3fb0 con 0x7f819406c5f0 2024-04-24T23:04:39.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.944+0000 7f819a590640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f819415d000 con 0x7f81940af890 2024-04-24T23:04:39.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.944+0000 7f8198b2d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81940b44d0 0x7f8194152220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:39.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.944+0000 7f8198b2d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81940b44d0 0x7f8194152220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46896/0 (socket says 172.21.15.32:46896) 2024-04-24T23:04:39.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.944+0000 7f8198b2d640 1 -- 172.21.15.32:0/529861797 learned_addr learned my addr 172.21.15.32:0/529861797 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:39.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.944+0000 7f81937fe640 1 --2- 172.21.15.32:0/529861797 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f81940af890 0x7f81941ef5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:39.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.944+0000 7f8193fff640 1 --2- 172.21.15.32:0/529861797 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f819406c5f0 0x7f81941f2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:39.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.945+0000 7f8198b2d640 1 -- 172.21.15.32:0/529861797 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f81940af890 msgr2=0x7f81941ef5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:39.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.945+0000 7f8198b2d640 1 --2- 172.21.15.32:0/529861797 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f81940af890 0x7f81941ef5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:39.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.945+0000 7f8198b2d640 1 -- 172.21.15.32:0/529861797 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f819406c5f0 msgr2=0x7f81941f2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:39.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.945+0000 7f8198b2d640 1 --2- 172.21.15.32:0/529861797 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f819406c5f0 0x7f81941f2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:39.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.945+0000 7f8198b2d640 1 -- 172.21.15.32:0/529861797 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81941f5670 con 0x7f81940b44d0 2024-04-24T23:04:39.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.945+0000 7f8198b2d640 1 --2- 172.21.15.32:0/529861797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81940b44d0 0x7f8194152220 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f818c009f70 tx=0x7f818c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:39.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.945+0000 7f8190ff9640 1 -- 172.21.15.32:0/529861797 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818c03f020 con 0x7f81940b44d0 2024-04-24T23:04:39.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.945+0000 7f8190ff9640 1 -- 172.21.15.32:0/529861797 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f818c037040 con 0x7f81940b44d0 2024-04-24T23:04:39.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.945+0000 7f819a590640 1 -- 172.21.15.32:0/529861797 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81941f5910 con 0x7f81940b44d0 2024-04-24T23:04:39.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.945+0000 7f8190ff9640 1 -- 172.21.15.32:0/529861797 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818c036070 con 0x7f81940b44d0 2024-04-24T23:04:39.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.945+0000 7f819a590640 1 -- 172.21.15.32:0/529861797 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81941f5ed0 con 0x7f81940b44d0 2024-04-24T23:04:39.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.946+0000 7f819a590640 1 -- 172.21.15.32:0/529861797 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8194068430 con 0x7f81940b44d0 2024-04-24T23:04:39.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.947+0000 7f8190ff9640 1 -- 172.21.15.32:0/529861797 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f818c042020 con 0x7f81940b44d0 2024-04-24T23:04:39.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.947+0000 7f8190ff9640 1 --2- 172.21.15.32:0/529861797 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f818006be30 0x7f818006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:39.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.947+0000 7f8190ff9640 1 -- 172.21.15.32:0/529861797 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7f818c0ab230 con 0x7f81940b44d0 2024-04-24T23:04:39.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.947+0000 7f8193fff640 1 --2- 172.21.15.32:0/529861797 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f818006be30 0x7f818006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:39.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.948+0000 7f8193fff640 1 --2- 172.21.15.32:0/529861797 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f818006be30 0x7f818006e2f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8188001780 tx=0x7f8188009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:39.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:39.951+0000 7f8190ff9640 1 -- 172.21.15.32:0/529861797 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f818c049030 con 0x7f81940b44d0 2024-04-24T23:04:40.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.135+0000 7f819a590640 1 -- 172.21.15.32:0/529861797 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8194156350 con 0x7f81940b44d0 2024-04-24T23:04:40.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.136+0000 7f8190ff9640 1 -- 172.21.15.32:0/529861797 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v113) ==== 100+0+11 (secure 0 0 0) 0x7f818c079590 con 0x7f81940b44d0 2024-04-24T23:04:40.138 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-24T23:04:40.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.138+0000 7f819a590640 1 -- 172.21.15.32:0/529861797 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f818006be30 msgr2=0x7f818006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:40.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.138+0000 7f819a590640 1 --2- 172.21.15.32:0/529861797 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f818006be30 0x7f818006e2f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8188001780 tx=0x7f8188009040 comp rx=0 tx=0).stop 2024-04-24T23:04:40.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.138+0000 7f819a590640 1 -- 172.21.15.32:0/529861797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81940b44d0 msgr2=0x7f8194152220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:40.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.138+0000 7f819a590640 1 --2- 172.21.15.32:0/529861797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81940b44d0 0x7f8194152220 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f818c009f70 tx=0x7f818c004a40 comp rx=0 tx=0).stop 2024-04-24T23:04:40.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.138+0000 7f819a590640 1 -- 172.21.15.32:0/529861797 shutdown_connections 2024-04-24T23:04:40.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.138+0000 7f819a590640 1 --2- 172.21.15.32:0/529861797 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f818006be30 0x7f818006e2f0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:40.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.138+0000 7f819a590640 1 --2- 172.21.15.32:0/529861797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81940b44d0 0x7f8194152220 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:40.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.138+0000 7f819a590640 1 --2- 172.21.15.32:0/529861797 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f81940af890 0x7f81941ef5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:40.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.138+0000 7f819a590640 1 --2- 172.21.15.32:0/529861797 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f819406c5f0 0x7f81941f2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:40.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.138+0000 7f819a590640 1 -- 172.21.15.32:0/529861797 >> 172.21.15.32:0/529861797 conn(0x7f819406b110 msgr2=0x7f8194072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:40.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.138+0000 7f819a590640 1 -- 172.21.15.32:0/529861797 shutdown_connections 2024-04-24T23:04:40.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.138+0000 7f819a590640 1 -- 172.21.15.32:0/529861797 wait complete. 2024-04-24T23:04:40.152 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T23:04:40.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.250+0000 7f2e5735a640 1 Processor -- start 2024-04-24T23:04:40.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.251+0000 7f2e5735a640 1 -- start start 2024-04-24T23:04:40.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.251+0000 7f2e5735a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2e50162320 0x7f2e500ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:40.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.251+0000 7f2e5735a640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2e500b5af0 0x7f2e50162700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:40.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.252+0000 7f2e5735a640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e500cd270 con 0x7f2e50162c40 2024-04-24T23:04:40.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.252+0000 7f2e5735a640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f2e500b5660 con 0x7f2e50162320 2024-04-24T23:04:40.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.252+0000 7f2e5735a640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f2e50071610 con 0x7f2e500b5af0 2024-04-24T23:04:40.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.252+0000 7f2e550f6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2e50162320 0x7f2e500ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:40.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.252+0000 7f2e558f7640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2e50162c40 0x7f2e500ae8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60758/0 (socket says 172.21.15.32:60758) 2024-04-24T23:04:40.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.252+0000 7f2e550f6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2e50162320 0x7f2e500ccc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46898/0 (socket says 172.21.15.32:46898) 2024-04-24T23:04:40.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.252+0000 7f2e558f7640 1 -- 172.21.15.32:0/2889763217 learned_addr learned my addr 172.21.15.32:0/2889763217 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:04:40.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.252+0000 7f2e548f5640 1 --2- 172.21.15.32:0/2889763217 >> v2:172.21.15.121:3300/0 conn(0x7f2e500b5af0 0x7f2e50162700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:40.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.252+0000 7f2e550f6640 1 -- 172.21.15.32:0/2889763217 >> v2:172.21.15.121:3300/0 conn(0x7f2e500b5af0 msgr2=0x7f2e50162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:40.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.252+0000 7f2e550f6640 1 --2- 172.21.15.32:0/2889763217 >> v2:172.21.15.121:3300/0 conn(0x7f2e500b5af0 0x7f2e50162700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:40.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.252+0000 7f2e550f6640 1 -- 172.21.15.32:0/2889763217 >> v1:172.21.15.32:6789/0 conn(0x7f2e50162c40 legacy=0x7f2e500ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:40.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.252+0000 7f2e4ffff640 1 -- 172.21.15.32:0/2889763217 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3708405819 0 0) 0x7f2e500cd270 con 0x7f2e50162c40 2024-04-24T23:04:40.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.252+0000 7f2e550f6640 1 -- 172.21.15.32:0/2889763217 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e500718f0 con 0x7f2e50162320 2024-04-24T23:04:40.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.253+0000 7f2e550f6640 1 --2- 172.21.15.32:0/2889763217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e50162320 0x7f2e500ccc90 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f2e500d1bc0 tx=0x7f2e4802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f26f3ce78055dc3b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:40.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.253+0000 7f2e4ffff640 1 -- 172.21.15.32:0/2889763217 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e48037070 con 0x7f2e50162320 2024-04-24T23:04:40.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.253+0000 7f2e4ffff640 1 -- 172.21.15.32:0/2889763217 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e48032440 con 0x7f2e50162320 2024-04-24T23:04:40.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.253+0000 7f2e4ffff640 1 -- 172.21.15.32:0/2889763217 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4802ae60 con 0x7f2e50162320 2024-04-24T23:04:40.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.253+0000 7f2e5735a640 1 -- 172.21.15.32:0/2889763217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e50162320 msgr2=0x7f2e500ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:40.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.253+0000 7f2e5735a640 1 --2- 172.21.15.32:0/2889763217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e50162320 0x7f2e500ccc90 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f2e500d1bc0 tx=0x7f2e4802a110 comp rx=0 tx=0).stop 2024-04-24T23:04:40.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.253+0000 7f2e5735a640 1 -- 172.21.15.32:0/2889763217 shutdown_connections 2024-04-24T23:04:40.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.253+0000 7f2e5735a640 1 --2- 172.21.15.32:0/2889763217 >> v2:172.21.15.121:3300/0 conn(0x7f2e500b5af0 0x7f2e50162700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:40.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.253+0000 7f2e5735a640 1 --2- 172.21.15.32:0/2889763217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e50162320 0x7f2e500ccc90 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:40.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.253+0000 7f2e5735a640 1 -- 172.21.15.32:0/2889763217 >> 172.21.15.32:0/2889763217 conn(0x7f2e500d0ae0 msgr2=0x7f2e500d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:40.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.254+0000 7f2e5735a640 1 -- 172.21.15.32:0/2889763217 shutdown_connections 2024-04-24T23:04:40.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.254+0000 7f2e5735a640 1 -- 172.21.15.32:0/2889763217 wait complete. 2024-04-24T23:04:40.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.255+0000 7f2e5735a640 1 Processor -- start 2024-04-24T23:04:40.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.255+0000 7f2e5735a640 1 -- start start 2024-04-24T23:04:40.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.255+0000 7f2e5735a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e500b5af0 0x7f2e50155200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:40.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.255+0000 7f2e5735a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e50162320 0x7f2e50161b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:40.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.255+0000 7f2e5735a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e50162c40 0x7f2e501f4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:40.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.255+0000 7f2e5735a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2e50163070 con 0x7f2e500b5af0 2024-04-24T23:04:40.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.255+0000 7f2e5735a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2e50067950 con 0x7f2e50162c40 2024-04-24T23:04:40.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.255+0000 7f2e5735a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f2e50071bd0 con 0x7f2e50162320 2024-04-24T23:04:40.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.255+0000 7f2e550f6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e500b5af0 0x7f2e50155200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:40.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.256+0000 7f2e550f6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e500b5af0 0x7f2e50155200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46910/0 (socket says 172.21.15.32:46910) 2024-04-24T23:04:40.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.256+0000 7f2e550f6640 1 -- 172.21.15.32:0/675514374 learned_addr learned my addr 172.21.15.32:0/675514374 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:40.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.256+0000 7f2e548f5640 1 --2- 172.21.15.32:0/675514374 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e50162320 0x7f2e50161b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:40.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.256+0000 7f2e558f7640 1 --2- 172.21.15.32:0/675514374 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e50162c40 0x7f2e501f4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:40.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.256+0000 7f2e550f6640 1 -- 172.21.15.32:0/675514374 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e50162320 msgr2=0x7f2e50161b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:40.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.256+0000 7f2e550f6640 1 --2- 172.21.15.32:0/675514374 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e50162320 0x7f2e50161b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:40.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.256+0000 7f2e550f6640 1 -- 172.21.15.32:0/675514374 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e50162c40 msgr2=0x7f2e501f4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:40.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.256+0000 7f2e550f6640 1 --2- 172.21.15.32:0/675514374 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e50162c40 0x7f2e501f4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:40.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.256+0000 7f2e550f6640 1 -- 172.21.15.32:0/675514374 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e501594e0 con 0x7f2e500b5af0 2024-04-24T23:04:40.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.256+0000 7f2e550f6640 1 --2- 172.21.15.32:0/675514374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e500b5af0 0x7f2e50155200 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f2e48000c00 tx=0x7f2e480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:40.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.256+0000 7f2e4dffb640 1 -- 172.21.15.32:0/675514374 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e48037040 con 0x7f2e500b5af0 2024-04-24T23:04:40.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.256+0000 7f2e4dffb640 1 -- 172.21.15.32:0/675514374 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e48042070 con 0x7f2e500b5af0 2024-04-24T23:04:40.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.256+0000 7f2e5735a640 1 -- 172.21.15.32:0/675514374 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e50162050 con 0x7f2e500b5af0 2024-04-24T23:04:40.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.256+0000 7f2e4dffb640 1 -- 172.21.15.32:0/675514374 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e48030bd0 con 0x7f2e500b5af0 2024-04-24T23:04:40.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.256+0000 7f2e5735a640 1 -- 172.21.15.32:0/675514374 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e500cfcd0 con 0x7f2e500b5af0 2024-04-24T23:04:40.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.257+0000 7f2e5735a640 1 -- 172.21.15.32:0/675514374 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e50068430 con 0x7f2e500b5af0 2024-04-24T23:04:40.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.258+0000 7f2e4dffb640 1 -- 172.21.15.32:0/675514374 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2e48047460 con 0x7f2e500b5af0 2024-04-24T23:04:40.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.258+0000 7f2e4dffb640 1 --2- 172.21.15.32:0/675514374 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2e3c06be30 0x7f2e3c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:40.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.258+0000 7f2e4dffb640 1 -- 172.21.15.32:0/675514374 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7f2e480ab890 con 0x7f2e500b5af0 2024-04-24T23:04:40.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.258+0000 7f2e548f5640 1 --2- 172.21.15.32:0/675514374 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2e3c06be30 0x7f2e3c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:40.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.259+0000 7f2e548f5640 1 --2- 172.21.15.32:0/675514374 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2e3c06be30 0x7f2e3c06e2f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2e5006f030 tx=0x7f2e44008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:40.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.263+0000 7f2e4dffb640 1 -- 172.21.15.32:0/675514374 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2e48032e20 con 0x7f2e500b5af0 2024-04-24T23:04:40.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.459+0000 7f2e5735a640 1 -- 172.21.15.32:0/675514374 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2e50067730 con 0x7f2e500b5af0 2024-04-24T23:04:40.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.459+0000 7f2e4dffb640 1 -- 172.21.15.32:0/675514374 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v113) ==== 80+0+0 (secure 0 0 0) 0x7f2e48079bf0 con 0x7f2e500b5af0 2024-04-24T23:04:40.462 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2024-04-24T23:04:40.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.461+0000 7f2e5735a640 1 -- 172.21.15.32:0/675514374 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2e3c06be30 msgr2=0x7f2e3c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:40.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.461+0000 7f2e5735a640 1 --2- 172.21.15.32:0/675514374 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2e3c06be30 0x7f2e3c06e2f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2e5006f030 tx=0x7f2e44008040 comp rx=0 tx=0).stop 2024-04-24T23:04:40.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.461+0000 7f2e5735a640 1 -- 172.21.15.32:0/675514374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e500b5af0 msgr2=0x7f2e50155200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:40.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.461+0000 7f2e5735a640 1 --2- 172.21.15.32:0/675514374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e500b5af0 0x7f2e50155200 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f2e48000c00 tx=0x7f2e480046c0 comp rx=0 tx=0).stop 2024-04-24T23:04:40.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.461+0000 7f2e5735a640 1 -- 172.21.15.32:0/675514374 shutdown_connections 2024-04-24T23:04:40.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.461+0000 7f2e5735a640 1 --2- 172.21.15.32:0/675514374 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2e3c06be30 0x7f2e3c06e2f0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:40.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.461+0000 7f2e5735a640 1 --2- 172.21.15.32:0/675514374 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e50162c40 0x7f2e501f4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:40.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.461+0000 7f2e5735a640 1 --2- 172.21.15.32:0/675514374 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e50162320 0x7f2e50161b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:40.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.461+0000 7f2e5735a640 1 --2- 172.21.15.32:0/675514374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e500b5af0 0x7f2e50155200 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:40.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.461+0000 7f2e5735a640 1 -- 172.21.15.32:0/675514374 >> 172.21.15.32:0/675514374 conn(0x7f2e500d0ae0 msgr2=0x7f2e500b0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:40.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.461+0000 7f2e5735a640 1 -- 172.21.15.32:0/675514374 shutdown_connections 2024-04-24T23:04:40.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.461+0000 7f2e5735a640 1 -- 172.21.15.32:0/675514374 wait complete. 2024-04-24T23:04:40.475 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-24T23:04:40.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:04:40.475 DEBUG:teuthology.orchestra.run.smithi121:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-24T23:04:40.507 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2024-04-24T23:04:40.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:04:40.507 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-24T23:04:40.565 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:04:40.565 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:04:40.565 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:04:40.581 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=413 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T23:04:40.581 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:04:40.581 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:04:40.581 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:04:40.622 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3271 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T23:04:40.622 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:04:40.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.720+0000 7f247d39e640 1 Processor -- start 2024-04-24T23:04:40.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.720+0000 7f247d39e640 1 -- start start 2024-04-24T23:04:40.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.721+0000 7f247d39e640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f247815e9b0 0x7f24780d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:40.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.721+0000 7f247d39e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f24780d4210 0x7f247806b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:40.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.721+0000 7f247d39e640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24780d3170 con 0x7f24780ba0e0 2024-04-24T23:04:40.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.721+0000 7f247d39e640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f24780b8900 con 0x7f247815e9b0 2024-04-24T23:04:40.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.721+0000 7f247d39e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f24780d3450 con 0x7f24780d4210 2024-04-24T23:04:40.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.722+0000 7f24767fc640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f24780d4210 0x7f247806b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:40.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.722+0000 7f2476ffd640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f247815e9b0 0x7f24780d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:40.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.722+0000 7f24767fc640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f24780d4210 0x7f247806b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.32:34746/0 (socket says 172.21.15.32:34746) 2024-04-24T23:04:40.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.722+0000 7f24777fe640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f24780ba0e0 0x7f2478071730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:38798/0 (socket says 172.21.15.32:38798) 2024-04-24T23:04:40.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.722+0000 7f24767fc640 1 -- 172.21.15.32:0/2187235092 learned_addr learned my addr 172.21.15.32:0/2187235092 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:40.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.722+0000 7f24767fc640 1 -- 172.21.15.32:0/2187235092 >> v2:172.21.15.121:3300/0 conn(0x7f247815e9b0 msgr2=0x7f24780d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:40.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.722+0000 7f24767fc640 1 --2- 172.21.15.32:0/2187235092 >> v2:172.21.15.121:3300/0 conn(0x7f247815e9b0 0x7f24780d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:40.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.722+0000 7f24767fc640 1 -- 172.21.15.32:0/2187235092 >> v1:172.21.15.121:6789/0 conn(0x7f24780ba0e0 legacy=0x7f2478071730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:40.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.722+0000 7f24767fc640 1 -- 172.21.15.32:0/2187235092 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24780d3730 con 0x7f24780d4210 2024-04-24T23:04:40.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.722+0000 7f2476ffd640 1 --2- 172.21.15.32:0/2187235092 >> v2:172.21.15.121:3300/0 conn(0x7f247815e9b0 0x7f24780d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:04:40.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.723+0000 7f24767fc640 1 --2- 172.21.15.32:0/2187235092 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f24780d4210 0x7f247806b110 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2460009870 tx=0x7f246002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d82fd1f37645b921 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:40.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.723+0000 7f2475ffb640 1 -- 172.21.15.32:0/2187235092 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2460037070 con 0x7f24780d4210 2024-04-24T23:04:40.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.723+0000 7f2475ffb640 1 -- 172.21.15.32:0/2187235092 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2460032440 con 0x7f24780d4210 2024-04-24T23:04:40.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.723+0000 7f2475ffb640 1 -- 172.21.15.32:0/2187235092 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2460032810 con 0x7f24780d4210 2024-04-24T23:04:40.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.723+0000 7f247d39e640 1 -- 172.21.15.32:0/2187235092 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f24780d4210 msgr2=0x7f247806b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:40.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.723+0000 7f247d39e640 1 --2- 172.21.15.32:0/2187235092 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f24780d4210 0x7f247806b110 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2460009870 tx=0x7f246002a270 comp rx=0 tx=0).stop 2024-04-24T23:04:40.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.723+0000 7f247d39e640 1 -- 172.21.15.32:0/2187235092 shutdown_connections 2024-04-24T23:04:40.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.723+0000 7f247d39e640 1 --2- 172.21.15.32:0/2187235092 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f24780d4210 0x7f247806b110 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:40.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.723+0000 7f247d39e640 1 --2- 172.21.15.32:0/2187235092 >> v2:172.21.15.121:3300/0 conn(0x7f247815e9b0 0x7f24780d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:40.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.723+0000 7f247d39e640 1 -- 172.21.15.32:0/2187235092 >> 172.21.15.32:0/2187235092 conn(0x7f24780d11d0 msgr2=0x7f24780d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:40.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.723+0000 7f247d39e640 1 -- 172.21.15.32:0/2187235092 shutdown_connections 2024-04-24T23:04:40.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.723+0000 7f247d39e640 1 -- 172.21.15.32:0/2187235092 wait complete. 2024-04-24T23:04:40.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.725+0000 7f247d39e640 1 Processor -- start 2024-04-24T23:04:40.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.725+0000 7f247d39e640 1 -- start start 2024-04-24T23:04:40.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.725+0000 7f247d39e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f24780ba0e0 0x7f2478158c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:40.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.725+0000 7f247d39e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24780d4210 0x7f247815ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:40.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.725+0000 7f247d39e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f247815e9b0 0x7f247815f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:40.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.725+0000 7f247d39e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f24780d2400 con 0x7f247815e9b0 2024-04-24T23:04:40.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.725+0000 7f247d39e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f24780d3cf0 con 0x7f24780d4210 2024-04-24T23:04:40.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.725+0000 7f247d39e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f247806b650 con 0x7f24780ba0e0 2024-04-24T23:04:40.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.726+0000 7f24777fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f247815e9b0 0x7f247815f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:40.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.726+0000 7f24777fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f247815e9b0 0x7f247815f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46918/0 (socket says 172.21.15.32:46918) 2024-04-24T23:04:40.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.726+0000 7f24777fe640 1 -- 172.21.15.32:0/1009869890 learned_addr learned my addr 172.21.15.32:0/1009869890 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:40.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.726+0000 7f24767fc640 1 --2- 172.21.15.32:0/1009869890 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24780d4210 0x7f247815ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:40.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.726+0000 7f2476ffd640 1 --2- 172.21.15.32:0/1009869890 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f24780ba0e0 0x7f2478158c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:40.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.726+0000 7f24777fe640 1 -- 172.21.15.32:0/1009869890 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f24780ba0e0 msgr2=0x7f2478158c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:40.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.726+0000 7f24777fe640 1 --2- 172.21.15.32:0/1009869890 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f24780ba0e0 0x7f2478158c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:40.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.726+0000 7f24777fe640 1 -- 172.21.15.32:0/1009869890 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24780d4210 msgr2=0x7f247815ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:40.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.726+0000 7f24777fe640 1 --2- 172.21.15.32:0/1009869890 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24780d4210 0x7f247815ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:40.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.726+0000 7f24777fe640 1 -- 172.21.15.32:0/1009869890 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2478156550 con 0x7f247815e9b0 2024-04-24T23:04:40.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.726+0000 7f24767fc640 1 --2- 172.21.15.32:0/1009869890 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24780d4210 0x7f247815ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:04:40.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.726+0000 7f24777fe640 1 --2- 172.21.15.32:0/1009869890 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f247815e9b0 0x7f247815f2d0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f246800c780 tx=0x7f246800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:40.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.726+0000 7f246f7fe640 1 -- 172.21.15.32:0/1009869890 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2468017070 con 0x7f247815e9b0 2024-04-24T23:04:40.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.726+0000 7f246f7fe640 1 -- 172.21.15.32:0/1009869890 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f246800a640 con 0x7f247815e9b0 2024-04-24T23:04:40.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.726+0000 7f246f7fe640 1 -- 172.21.15.32:0/1009869890 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246800aa30 con 0x7f247815e9b0 2024-04-24T23:04:40.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.726+0000 7f247d39e640 1 -- 172.21.15.32:0/1009869890 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24781567f0 con 0x7f247815e9b0 2024-04-24T23:04:40.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.726+0000 7f247d39e640 1 -- 172.21.15.32:0/1009869890 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24780ce1e0 con 0x7f247815e9b0 2024-04-24T23:04:40.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.727+0000 7f247d39e640 1 -- 172.21.15.32:0/1009869890 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2478156fa0 con 0x7f247815e9b0 2024-04-24T23:04:40.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.728+0000 7f246f7fe640 1 -- 172.21.15.32:0/1009869890 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f246801a070 con 0x7f247815e9b0 2024-04-24T23:04:40.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.728+0000 7f246f7fe640 1 --2- 172.21.15.32:0/1009869890 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f245806be30 0x7f245806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:40.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.728+0000 7f246f7fe640 1 -- 172.21.15.32:0/1009869890 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7f246808a3d0 con 0x7f247815e9b0 2024-04-24T23:04:40.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.728+0000 7f2476ffd640 1 --2- 172.21.15.32:0/1009869890 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f245806be30 0x7f245806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:40.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.729+0000 7f2476ffd640 1 --2- 172.21.15.32:0/1009869890 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f245806be30 0x7f245806e2f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f2470009a30 tx=0x7f2470008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:40.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.732+0000 7f246f7fe640 1 -- 172.21.15.32:0/1009869890 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f24680586b0 con 0x7f247815e9b0 2024-04-24T23:04:40.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.954+0000 7f247d39e640 1 -- 172.21.15.32:0/1009869890 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f24780c7bd0 con 0x7f247815e9b0 2024-04-24T23:04:40.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.954+0000 7f246f7fe640 1 -- 172.21.15.32:0/1009869890 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5186 (secure 0 0 0) 0x7f246805c9b0 con 0x7f247815e9b0 2024-04-24T23:04:40.957 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:04:40.957 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":44,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6175,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/434273738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":434273738},{"type":"v1","addr":"172.21.15.121:6836","nonce":434273738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6178,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1104557785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1104557785},{"type":"v1","addr":"172.21.15.121:6837","nonce":1104557785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6888,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/3137388886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3137388886},{"type":"v1","addr":"172.21.15.32:6834","nonce":3137388886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:04:00.783579+0000","modified":"2024-04-24T23:04:03.790658+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6891},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6891":{"gid":6891,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/3327203898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":3327203898},{"type":"v1","addr":"172.21.15.32:6835","nonce":3327203898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6891,"qdb_cluster":[6891]},"id":6}]} 2024-04-24T23:04:40.957 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 44 2024-04-24T23:04:40.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.956+0000 7f247d39e640 1 -- 172.21.15.32:0/1009869890 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f245806be30 msgr2=0x7f245806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:40.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.956+0000 7f247d39e640 1 --2- 172.21.15.32:0/1009869890 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f245806be30 0x7f245806e2f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f2470009a30 tx=0x7f2470008040 comp rx=0 tx=0).stop 2024-04-24T23:04:40.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.956+0000 7f247d39e640 1 -- 172.21.15.32:0/1009869890 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f247815e9b0 msgr2=0x7f247815f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:40.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.956+0000 7f247d39e640 1 --2- 172.21.15.32:0/1009869890 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f247815e9b0 0x7f247815f2d0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f246800c780 tx=0x7f246800cc40 comp rx=0 tx=0).stop 2024-04-24T23:04:40.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.957+0000 7f247d39e640 1 -- 172.21.15.32:0/1009869890 shutdown_connections 2024-04-24T23:04:40.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.957+0000 7f247d39e640 1 --2- 172.21.15.32:0/1009869890 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f245806be30 0x7f245806e2f0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:40.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.957+0000 7f247d39e640 1 --2- 172.21.15.32:0/1009869890 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f247815e9b0 0x7f247815f2d0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:40.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.957+0000 7f247d39e640 1 --2- 172.21.15.32:0/1009869890 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24780d4210 0x7f247815ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:40.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.957+0000 7f247d39e640 1 --2- 172.21.15.32:0/1009869890 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f24780ba0e0 0x7f2478158c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:40.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.957+0000 7f247d39e640 1 -- 172.21.15.32:0/1009869890 >> 172.21.15.32:0/1009869890 conn(0x7f24780d11d0 msgr2=0x7f24780d15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:40.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.957+0000 7f247d39e640 1 -- 172.21.15.32:0/1009869890 shutdown_connections 2024-04-24T23:04:40.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:40.957+0000 7f247d39e640 1 -- 172.21.15.32:0/1009869890 wait complete. 2024-04-24T23:04:40.970 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:04:41.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.068+0000 7fe5ab35c640 1 Processor -- start 2024-04-24T23:04:41.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.068+0000 7fe5ab35c640 1 -- start start 2024-04-24T23:04:41.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.069+0000 7fe5ab35c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe5a406b820 0x7fe5a40d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:41.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.069+0000 7fe5ab35c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe5a40d4210 0x7fe5a40b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:41.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.069+0000 7fe5ab35c640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5a40d3170 con 0x7fe5a415e9b0 2024-04-24T23:04:41.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.069+0000 7fe5ab35c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fe5a40d3450 con 0x7fe5a40d4210 2024-04-24T23:04:41.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.069+0000 7fe5ab35c640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe5a40d3730 con 0x7fe5a406b820 2024-04-24T23:04:41.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.069+0000 7fe5a90f8640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe5a415e9b0 0x7fe5a406b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60772/0 (socket says 172.21.15.32:60772) 2024-04-24T23:04:41.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.069+0000 7fe5a90f8640 1 -- 172.21.15.32:0/3219076771 learned_addr learned my addr 172.21.15.32:0/3219076771 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:04:41.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.069+0000 7fe5a98f9640 1 --2- 172.21.15.32:0/3219076771 >> v2:172.21.15.32:3300/0 conn(0x7fe5a40d4210 0x7fe5a40b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:41.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.070+0000 7fe5a3fff640 1 -- 172.21.15.32:0/3219076771 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 171709161 0 0) 0x7fe5a40d3170 con 0x7fe5a415e9b0 2024-04-24T23:04:41.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.070+0000 7fe5a3fff640 1 -- 172.21.15.32:0/3219076771 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe58c003520 con 0x7fe5a415e9b0 2024-04-24T23:04:41.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.070+0000 7fe5a88f7640 1 --2- 172.21.15.32:0/3219076771 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe5a406b820 0x7fe5a40d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:41.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.070+0000 7fe5a98f9640 1 -- 172.21.15.32:0/3219076771 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe5a406b820 msgr2=0x7fe5a40d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:41.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.070+0000 7fe5a98f9640 1 --2- 172.21.15.32:0/3219076771 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe5a406b820 0x7fe5a40d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:41.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.070+0000 7fe5a98f9640 1 -- 172.21.15.32:0/3219076771 >> v1:172.21.15.32:6789/0 conn(0x7fe5a415e9b0 legacy=0x7fe5a406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:41.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.070+0000 7fe5a98f9640 1 -- 172.21.15.32:0/3219076771 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5a40d3a10 con 0x7fe5a40d4210 2024-04-24T23:04:41.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.070+0000 7fe5a88f7640 1 --2- 172.21.15.32:0/3219076771 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe5a406b820 0x7fe5a40d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:04:41.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.070+0000 7fe5a98f9640 1 --2- 172.21.15.32:0/3219076771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5a40d4210 0x7fe5a40b78f0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fe594009d90 tx=0x7fe59402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54f3209177f33c4b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:41.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.071+0000 7fe5a3fff640 1 -- 172.21.15.32:0/3219076771 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe594007740 con 0x7fe5a40d4210 2024-04-24T23:04:41.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.071+0000 7fe5a3fff640 1 -- 172.21.15.32:0/3219076771 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5940078e0 con 0x7fe5a40d4210 2024-04-24T23:04:41.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.071+0000 7fe5a3fff640 1 -- 172.21.15.32:0/3219076771 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe594007c90 con 0x7fe5a40d4210 2024-04-24T23:04:41.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.071+0000 7fe5ab35c640 1 -- 172.21.15.32:0/3219076771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5a40d4210 msgr2=0x7fe5a40b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:41.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.071+0000 7fe5ab35c640 1 --2- 172.21.15.32:0/3219076771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5a40d4210 0x7fe5a40b78f0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fe594009d90 tx=0x7fe59402a660 comp rx=0 tx=0).stop 2024-04-24T23:04:41.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.071+0000 7fe5ab35c640 1 -- 172.21.15.32:0/3219076771 shutdown_connections 2024-04-24T23:04:41.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.071+0000 7fe5ab35c640 1 --2- 172.21.15.32:0/3219076771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5a40d4210 0x7fe5a40b78f0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:41.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.071+0000 7fe5ab35c640 1 --2- 172.21.15.32:0/3219076771 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe5a406b820 0x7fe5a40d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:41.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.071+0000 7fe5ab35c640 1 -- 172.21.15.32:0/3219076771 >> 172.21.15.32:0/3219076771 conn(0x7fe5a40d11d0 msgr2=0x7fe5a40d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:41.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.071+0000 7fe5ab35c640 1 -- 172.21.15.32:0/3219076771 shutdown_connections 2024-04-24T23:04:41.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.071+0000 7fe5ab35c640 1 -- 172.21.15.32:0/3219076771 wait complete. 2024-04-24T23:04:41.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.072+0000 7fe5ab35c640 1 Processor -- start 2024-04-24T23:04:41.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.072+0000 7fe5ab35c640 1 -- start start 2024-04-24T23:04:41.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.073+0000 7fe5ab35c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5a406b820 0x7fe5a4161700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:41.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.073+0000 7fe5ab35c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5a40d4210 0x7fe5a40c94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:41.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.073+0000 7fe5ab35c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe5a415e9b0 0x7fe5a41f6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:41.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.073+0000 7fe5ab35c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a40ba0e0 con 0x7fe5a406b820 2024-04-24T23:04:41.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.073+0000 7fe5ab35c640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a40d2400 con 0x7fe5a40d4210 2024-04-24T23:04:41.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.073+0000 7fe5ab35c640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe5a40ba260 con 0x7fe5a415e9b0 2024-04-24T23:04:41.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.073+0000 7fe5a90f8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5a406b820 0x7fe5a4161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:41.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.073+0000 7fe5a90f8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5a406b820 0x7fe5a4161700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46934/0 (socket says 172.21.15.32:46934) 2024-04-24T23:04:41.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.073+0000 7fe5a90f8640 1 -- 172.21.15.32:0/2455707001 learned_addr learned my addr 172.21.15.32:0/2455707001 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:41.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.073+0000 7fe5a88f7640 1 --2- 172.21.15.32:0/2455707001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5a40d4210 0x7fe5a40c94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:41.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.073+0000 7fe5a98f9640 1 --2- 172.21.15.32:0/2455707001 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe5a415e9b0 0x7fe5a41f6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:41.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.073+0000 7fe5a90f8640 1 -- 172.21.15.32:0/2455707001 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe5a415e9b0 msgr2=0x7fe5a41f6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:41.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.073+0000 7fe5a90f8640 1 --2- 172.21.15.32:0/2455707001 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe5a415e9b0 0x7fe5a41f6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:41.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.073+0000 7fe5a90f8640 1 -- 172.21.15.32:0/2455707001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5a40d4210 msgr2=0x7fe5a40c94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:41.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.073+0000 7fe5a90f8640 1 --2- 172.21.15.32:0/2455707001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5a40d4210 0x7fe5a40c94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:41.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.074+0000 7fe5a90f8640 1 -- 172.21.15.32:0/2455707001 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5a40c99f0 con 0x7fe5a406b820 2024-04-24T23:04:41.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.074+0000 7fe5a98f9640 1 --2- 172.21.15.32:0/2455707001 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe5a415e9b0 0x7fe5a41f6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:04:41.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.074+0000 7fe5a88f7640 1 --2- 172.21.15.32:0/2455707001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5a40d4210 0x7fe5a40c94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:04:41.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.074+0000 7fe5a90f8640 1 --2- 172.21.15.32:0/2455707001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5a406b820 0x7fe5a4161700 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fe59c00c390 tx=0x7fe59c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:41.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.074+0000 7fe5a1ffb640 1 -- 172.21.15.32:0/2455707001 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59c017070 con 0x7fe5a406b820 2024-04-24T23:04:41.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.074+0000 7fe5a1ffb640 1 -- 172.21.15.32:0/2455707001 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe59c005ce0 con 0x7fe5a406b820 2024-04-24T23:04:41.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.074+0000 7fe5a1ffb640 1 -- 172.21.15.32:0/2455707001 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59c0072f0 con 0x7fe5a406b820 2024-04-24T23:04:41.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.074+0000 7fe5ab35c640 1 -- 172.21.15.32:0/2455707001 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5a4160660 con 0x7fe5a406b820 2024-04-24T23:04:41.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.074+0000 7fe5ab35c640 1 -- 172.21.15.32:0/2455707001 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5a4160c20 con 0x7fe5a406b820 2024-04-24T23:04:41.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.075+0000 7fe5ab35c640 1 -- 172.21.15.32:0/2455707001 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5a415b820 con 0x7fe5a406b820 2024-04-24T23:04:41.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.075+0000 7fe5a1ffb640 1 -- 172.21.15.32:0/2455707001 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe59c01a070 con 0x7fe5a406b820 2024-04-24T23:04:41.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.076+0000 7fe5a1ffb640 1 --2- 172.21.15.32:0/2455707001 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe59006be30 0x7fe59006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:41.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.076+0000 7fe5a1ffb640 1 -- 172.21.15.32:0/2455707001 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7fe59c08a260 con 0x7fe5a406b820 2024-04-24T23:04:41.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.076+0000 7fe5a88f7640 1 --2- 172.21.15.32:0/2455707001 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe59006be30 0x7fe59006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:41.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.077+0000 7fe5a88f7640 1 --2- 172.21.15.32:0/2455707001 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe59006be30 0x7fe59006e2f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe5a40caa80 tx=0x7fe598008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:41.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.080+0000 7fe5a1ffb640 1 -- 172.21.15.32:0/2455707001 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe5a415b820 con 0x7fe5a406b820 2024-04-24T23:04:41.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.262+0000 7fe5ab35c640 1 -- 172.21.15.32:0/2455707001 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe5a4068430 con 0x7fe5a406b820 2024-04-24T23:04:41.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.263+0000 7fe5a1ffb640 1 -- 172.21.15.32:0/2455707001 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) ==== 74+0+16696 (secure 0 0 0) 0x7fe59c058c10 con 0x7fe5a406b820 2024-04-24T23:04:41.269 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:04:41.269 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":113,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:04:31.618095+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:03:59.079157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-24T23:03:59.765179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:04:41.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.265+0000 7fe5ab35c640 1 -- 172.21.15.32:0/2455707001 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe59006be30 msgr2=0x7fe59006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:41.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.265+0000 7fe5ab35c640 1 --2- 172.21.15.32:0/2455707001 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe59006be30 0x7fe59006e2f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe5a40caa80 tx=0x7fe598008040 comp rx=0 tx=0).stop 2024-04-24T23:04:41.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.265+0000 7fe5ab35c640 1 -- 172.21.15.32:0/2455707001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5a406b820 msgr2=0x7fe5a4161700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:41.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.265+0000 7fe5ab35c640 1 --2- 172.21.15.32:0/2455707001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5a406b820 0x7fe5a4161700 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fe59c00c390 tx=0x7fe59c00c850 comp rx=0 tx=0).stop 2024-04-24T23:04:41.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.265+0000 7fe5ab35c640 1 -- 172.21.15.32:0/2455707001 shutdown_connections 2024-04-24T23:04:41.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.265+0000 7fe5ab35c640 1 --2- 172.21.15.32:0/2455707001 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe59006be30 0x7fe59006e2f0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:41.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.265+0000 7fe5ab35c640 1 --2- 172.21.15.32:0/2455707001 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe5a415e9b0 0x7fe5a41f6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:41.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.265+0000 7fe5ab35c640 1 --2- 172.21.15.32:0/2455707001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5a40d4210 0x7fe5a40c94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:41.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.265+0000 7fe5ab35c640 1 --2- 172.21.15.32:0/2455707001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5a406b820 0x7fe5a4161700 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:41.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.265+0000 7fe5ab35c640 1 -- 172.21.15.32:0/2455707001 >> 172.21.15.32:0/2455707001 conn(0x7fe5a40d11d0 msgr2=0x7fe5a4159e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:41.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.265+0000 7fe5ab35c640 1 -- 172.21.15.32:0/2455707001 shutdown_connections 2024-04-24T23:04:41.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.265+0000 7fe5ab35c640 1 -- 172.21.15.32:0/2455707001 wait complete. 2024-04-24T23:04:41.279 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:03:59.079157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:03:59.765179+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:04:41.280 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:04:41.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.382+0000 7fe560b6a640 1 Processor -- start 2024-04-24T23:04:41.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.382+0000 7fe560b6a640 1 -- start start 2024-04-24T23:04:41.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.383+0000 7fe560b6a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe55c06b820 0x7fe55c0d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:41.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.383+0000 7fe560b6a640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe55c0d4210 0x7fe55c0b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:41.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.383+0000 7fe560b6a640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe55c0d3170 con 0x7fe55c15e9b0 2024-04-24T23:04:41.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.383+0000 7fe560b6a640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fe55c0d3450 con 0x7fe55c0d4210 2024-04-24T23:04:41.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.383+0000 7fe560b6a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe55c0d3730 con 0x7fe55c06b820 2024-04-24T23:04:41.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.383+0000 7fe55a59c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe55c15e9b0 0x7fe55c06b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60788/0 (socket says 172.21.15.32:60788) 2024-04-24T23:04:41.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.383+0000 7fe55a59c640 1 -- 172.21.15.32:0/1458355064 learned_addr learned my addr 172.21.15.32:0/1458355064 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:04:41.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.383+0000 7fe55ad9d640 1 --2- 172.21.15.32:0/1458355064 >> v2:172.21.15.121:3300/0 conn(0x7fe55c0d4210 0x7fe55c0b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:41.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.383+0000 7fe559d9b640 1 --2- 172.21.15.32:0/1458355064 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe55c06b820 0x7fe55c0d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:41.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.383+0000 7fe55959a640 1 -- 172.21.15.32:0/1458355064 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1118118374 0 0) 0x7fe55c0d3170 con 0x7fe55c15e9b0 2024-04-24T23:04:41.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.384+0000 7fe55959a640 1 -- 172.21.15.32:0/1458355064 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe548003520 con 0x7fe55c15e9b0 2024-04-24T23:04:41.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.384+0000 7fe55959a640 1 -- 172.21.15.32:0/1458355064 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1304006220 0 0) 0x7fe548003520 con 0x7fe55c15e9b0 2024-04-24T23:04:41.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.384+0000 7fe559d9b640 1 -- 172.21.15.32:0/1458355064 >> v2:172.21.15.121:3300/0 conn(0x7fe55c0d4210 msgr2=0x7fe55c0b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:41.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.384+0000 7fe559d9b640 1 --2- 172.21.15.32:0/1458355064 >> v2:172.21.15.121:3300/0 conn(0x7fe55c0d4210 0x7fe55c0b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:41.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.384+0000 7fe559d9b640 1 -- 172.21.15.32:0/1458355064 >> v1:172.21.15.32:6789/0 conn(0x7fe55c15e9b0 legacy=0x7fe55c06b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:41.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.384+0000 7fe559d9b640 1 -- 172.21.15.32:0/1458355064 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe55c0d3a10 con 0x7fe55c06b820 2024-04-24T23:04:41.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.385+0000 7fe559d9b640 1 --2- 172.21.15.32:0/1458355064 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe55c06b820 0x7fe55c0d2b90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe5440097a0 tx=0x7fe54402a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a548844ee2ad20eb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:41.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.385+0000 7fe55959a640 1 -- 172.21.15.32:0/1458355064 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe544037070 con 0x7fe55c06b820 2024-04-24T23:04:41.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.385+0000 7fe55959a640 1 -- 172.21.15.32:0/1458355064 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe54402aba0 con 0x7fe55c06b820 2024-04-24T23:04:41.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.385+0000 7fe55959a640 1 -- 172.21.15.32:0/1458355064 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54c0019f0 con 0x7fe55c06b820 2024-04-24T23:04:41.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.385+0000 7fe560b6a640 1 -- 172.21.15.32:0/1458355064 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe55c06b820 msgr2=0x7fe55c0d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:41.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.385+0000 7fe560b6a640 1 --2- 172.21.15.32:0/1458355064 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe55c06b820 0x7fe55c0d2b90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe5440097a0 tx=0x7fe54402a110 comp rx=0 tx=0).stop 2024-04-24T23:04:41.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.385+0000 7fe560b6a640 1 -- 172.21.15.32:0/1458355064 shutdown_connections 2024-04-24T23:04:41.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.385+0000 7fe560b6a640 1 --2- 172.21.15.32:0/1458355064 >> v2:172.21.15.121:3300/0 conn(0x7fe55c0d4210 0x7fe55c0b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:41.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.385+0000 7fe560b6a640 1 --2- 172.21.15.32:0/1458355064 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe55c06b820 0x7fe55c0d2b90 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:41.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.385+0000 7fe560b6a640 1 -- 172.21.15.32:0/1458355064 >> 172.21.15.32:0/1458355064 conn(0x7fe55c0d11d0 msgr2=0x7fe55c0d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:41.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.385+0000 7fe560b6a640 1 -- 172.21.15.32:0/1458355064 shutdown_connections 2024-04-24T23:04:41.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.385+0000 7fe560b6a640 1 -- 172.21.15.32:0/1458355064 wait complete. 2024-04-24T23:04:41.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.387+0000 7fe560b6a640 1 Processor -- start 2024-04-24T23:04:41.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.387+0000 7fe560b6a640 1 -- start start 2024-04-24T23:04:41.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.387+0000 7fe560b6a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe55c06b820 0x7fe55c154f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:41.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.387+0000 7fe560b6a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe55c0d4210 0x7fe55c1f73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:41.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.387+0000 7fe560b6a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe55c15e9b0 0x7fe55c158680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:41.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.387+0000 7fe560b6a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe55c0ba0e0 con 0x7fe55c06b820 2024-04-24T23:04:41.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.387+0000 7fe560b6a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe55c0d2400 con 0x7fe55c0d4210 2024-04-24T23:04:41.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.387+0000 7fe560b6a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe55c0ba260 con 0x7fe55c15e9b0 2024-04-24T23:04:41.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.388+0000 7fe55a59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe55c06b820 0x7fe55c154f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:41.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.388+0000 7fe55a59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe55c06b820 0x7fe55c154f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46946/0 (socket says 172.21.15.32:46946) 2024-04-24T23:04:41.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.388+0000 7fe55a59c640 1 -- 172.21.15.32:0/1335345443 learned_addr learned my addr 172.21.15.32:0/1335345443 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:41.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.388+0000 7fe55a59c640 1 -- 172.21.15.32:0/1335345443 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe55c15e9b0 msgr2=0x7fe55c158680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:41.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.388+0000 7fe55a59c640 1 --2- 172.21.15.32:0/1335345443 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe55c15e9b0 0x7fe55c158680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:41.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.388+0000 7fe55a59c640 1 -- 172.21.15.32:0/1335345443 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe55c0d4210 msgr2=0x7fe55c1f73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:41.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.388+0000 7fe559d9b640 1 --2- 172.21.15.32:0/1335345443 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe55c0d4210 0x7fe55c1f73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:41.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.388+0000 7fe55a59c640 1 --2- 172.21.15.32:0/1335345443 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe55c0d4210 0x7fe55c1f73a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:41.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.388+0000 7fe55a59c640 1 -- 172.21.15.32:0/1335345443 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe55c1f7d80 con 0x7fe55c06b820 2024-04-24T23:04:41.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.388+0000 7fe55a59c640 1 --2- 172.21.15.32:0/1335345443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe55c06b820 0x7fe55c154f80 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fe54c00cc40 tx=0x7fe54c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:41.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.388+0000 7fe552ffd640 1 -- 172.21.15.32:0/1335345443 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54c017070 con 0x7fe55c06b820 2024-04-24T23:04:41.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.388+0000 7fe552ffd640 1 -- 172.21.15.32:0/1335345443 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe54c00ce50 con 0x7fe55c06b820 2024-04-24T23:04:41.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.388+0000 7fe560b6a640 1 -- 172.21.15.32:0/1335345443 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe55c1f8060 con 0x7fe55c06b820 2024-04-24T23:04:41.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.388+0000 7fe552ffd640 1 -- 172.21.15.32:0/1335345443 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54c012600 con 0x7fe55c06b820 2024-04-24T23:04:41.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.388+0000 7fe560b6a640 1 -- 172.21.15.32:0/1335345443 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe55c0c97c0 con 0x7fe55c06b820 2024-04-24T23:04:41.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.389+0000 7fe560b6a640 1 -- 172.21.15.32:0/1335345443 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe55c068430 con 0x7fe55c06b820 2024-04-24T23:04:41.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.390+0000 7fe552ffd640 1 -- 172.21.15.32:0/1335345443 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe54c01a070 con 0x7fe55c06b820 2024-04-24T23:04:41.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.390+0000 7fe552ffd640 1 --2- 172.21.15.32:0/1335345443 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe53c06be30 0x7fe53c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:41.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.390+0000 7fe552ffd640 1 -- 172.21.15.32:0/1335345443 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7fe54c08ac20 con 0x7fe55c06b820 2024-04-24T23:04:41.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.391+0000 7fe559d9b640 1 --2- 172.21.15.32:0/1335345443 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe53c06be30 0x7fe53c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:41.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.391+0000 7fe559d9b640 1 --2- 172.21.15.32:0/1335345443 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe53c06be30 0x7fe53c06e2f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe544034000 tx=0x7fe54402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:41.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.395+0000 7fe552ffd640 1 -- 172.21.15.32:0/1335345443 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe54c059010 con 0x7fe55c06b820 2024-04-24T23:04:41.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.580+0000 7fe560b6a640 1 -- 172.21.15.32:0/1335345443 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe55c1f8230 con 0x7fe55c06b820 2024-04-24T23:04:41.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.580+0000 7fe552ffd640 1 -- 172.21.15.32:0/1335345443 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v113) ==== 93+0+10 (secure 0 0 0) 0x7fe54c05d310 con 0x7fe55c06b820 2024-04-24T23:04:41.583 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T23:04:41.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.582+0000 7fe560b6a640 1 -- 172.21.15.32:0/1335345443 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe53c06be30 msgr2=0x7fe53c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:41.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.582+0000 7fe560b6a640 1 --2- 172.21.15.32:0/1335345443 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe53c06be30 0x7fe53c06e2f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe544034000 tx=0x7fe54402e000 comp rx=0 tx=0).stop 2024-04-24T23:04:41.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.582+0000 7fe560b6a640 1 -- 172.21.15.32:0/1335345443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe55c06b820 msgr2=0x7fe55c154f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:41.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.582+0000 7fe560b6a640 1 --2- 172.21.15.32:0/1335345443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe55c06b820 0x7fe55c154f80 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fe54c00cc40 tx=0x7fe54c00a430 comp rx=0 tx=0).stop 2024-04-24T23:04:41.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.582+0000 7fe560b6a640 1 -- 172.21.15.32:0/1335345443 shutdown_connections 2024-04-24T23:04:41.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.582+0000 7fe560b6a640 1 --2- 172.21.15.32:0/1335345443 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe53c06be30 0x7fe53c06e2f0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:41.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.582+0000 7fe560b6a640 1 --2- 172.21.15.32:0/1335345443 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe55c15e9b0 0x7fe55c158680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:41.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.582+0000 7fe560b6a640 1 --2- 172.21.15.32:0/1335345443 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe55c0d4210 0x7fe55c1f73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:41.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.582+0000 7fe560b6a640 1 --2- 172.21.15.32:0/1335345443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe55c06b820 0x7fe55c154f80 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:41.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.582+0000 7fe560b6a640 1 -- 172.21.15.32:0/1335345443 >> 172.21.15.32:0/1335345443 conn(0x7fe55c0d11d0 msgr2=0x7fe55c071fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:41.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.582+0000 7fe560b6a640 1 -- 172.21.15.32:0/1335345443 shutdown_connections 2024-04-24T23:04:41.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.582+0000 7fe560b6a640 1 -- 172.21.15.32:0/1335345443 wait complete. 2024-04-24T23:04:41.596 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:04:41.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.704+0000 7f4e0ce6b640 1 Processor -- start 2024-04-24T23:04:41.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.704+0000 7f4e0ce6b640 1 -- start start 2024-04-24T23:04:41.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.705+0000 7f4e0ce6b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4e0815e9b0 0x7f4e080d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:41.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.705+0000 7f4e0ce6b640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4e0806b820 0x7f4e080b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:41.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.705+0000 7f4e0ce6b640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e080d3170 con 0x7f4e080d4210 2024-04-24T23:04:41.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.705+0000 7f4e0ce6b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f4e080d3450 con 0x7f4e0815e9b0 2024-04-24T23:04:41.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.705+0000 7f4e0ce6b640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f4e080d3730 con 0x7f4e0806b820 2024-04-24T23:04:41.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.705+0000 7f4e0659c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4e0815e9b0 0x7f4e080d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:41.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.706+0000 7f4e0659c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4e0815e9b0 0x7f4e080d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46958/0 (socket says 172.21.15.32:46958) 2024-04-24T23:04:41.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.706+0000 7f4e0659c640 1 -- 172.21.15.32:0/548646421 learned_addr learned my addr 172.21.15.32:0/548646421 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:41.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.706+0000 7f4e0659c640 1 -- 172.21.15.32:0/548646421 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4e0806b820 msgr2=0x7f4e080b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:41.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.706+0000 7f4e06d9d640 1 --2- 172.21.15.32:0/548646421 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4e0806b820 0x7f4e080b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:41.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.706+0000 7f4e0659c640 1 --2- 172.21.15.32:0/548646421 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4e0806b820 0x7f4e080b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:41.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.706+0000 7f4e0659c640 1 -- 172.21.15.32:0/548646421 >> v1:172.21.15.121:6789/0 conn(0x7f4e080d4210 legacy=0x7f4e0806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:41.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.706+0000 7f4e0659c640 1 -- 172.21.15.32:0/548646421 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e080d3a10 con 0x7f4e0815e9b0 2024-04-24T23:04:41.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.706+0000 7f4e06d9d640 1 --2- 172.21.15.32:0/548646421 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4e0806b820 0x7f4e080b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:04:41.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.707+0000 7f4e0659c640 1 --2- 172.21.15.32:0/548646421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e0815e9b0 0x7f4e080d2b90 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f4e00009870 tx=0x7f4e0002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c96e10390c213a95 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:41.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.707+0000 7f4e0559a640 1 -- 172.21.15.32:0/548646421 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e00037070 con 0x7f4e0815e9b0 2024-04-24T23:04:41.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.707+0000 7f4e0559a640 1 -- 172.21.15.32:0/548646421 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e00032440 con 0x7f4e0815e9b0 2024-04-24T23:04:41.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.707+0000 7f4e0559a640 1 -- 172.21.15.32:0/548646421 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e00032810 con 0x7f4e0815e9b0 2024-04-24T23:04:41.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.707+0000 7f4e0ce6b640 1 -- 172.21.15.32:0/548646421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e0815e9b0 msgr2=0x7f4e080d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:41.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.707+0000 7f4e0ce6b640 1 --2- 172.21.15.32:0/548646421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e0815e9b0 0x7f4e080d2b90 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f4e00009870 tx=0x7f4e0002a270 comp rx=0 tx=0).stop 2024-04-24T23:04:41.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.707+0000 7f4e0ce6b640 1 -- 172.21.15.32:0/548646421 shutdown_connections 2024-04-24T23:04:41.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.707+0000 7f4e0ce6b640 1 --2- 172.21.15.32:0/548646421 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4e0806b820 0x7f4e080b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:41.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.707+0000 7f4e0ce6b640 1 --2- 172.21.15.32:0/548646421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e0815e9b0 0x7f4e080d2b90 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:41.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.707+0000 7f4e0ce6b640 1 -- 172.21.15.32:0/548646421 >> 172.21.15.32:0/548646421 conn(0x7f4e080d11d0 msgr2=0x7f4e080d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:41.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.707+0000 7f4e0ce6b640 1 -- 172.21.15.32:0/548646421 shutdown_connections 2024-04-24T23:04:41.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.707+0000 7f4e0ce6b640 1 -- 172.21.15.32:0/548646421 wait complete. 2024-04-24T23:04:41.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.709+0000 7f4e0ce6b640 1 Processor -- start 2024-04-24T23:04:41.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.709+0000 7f4e0ce6b640 1 -- start start 2024-04-24T23:04:41.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.709+0000 7f4e0ce6b640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4e0806b820 0x7f4e080c9190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:41.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.709+0000 7f4e0ce6b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4e080d4210 0x7f4e080cc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:41.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.709+0000 7f4e0ce6b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e0815e9b0 0x7f4e081f80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:41.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.709+0000 7f4e0ce6b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4e080ba0e0 con 0x7f4e0815e9b0 2024-04-24T23:04:41.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.709+0000 7f4e0ce6b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4e080d2400 con 0x7f4e080d4210 2024-04-24T23:04:41.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.709+0000 7f4e0ce6b640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f4e080ba260 con 0x7f4e0806b820 2024-04-24T23:04:41.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.710+0000 7f4e06d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e0815e9b0 0x7f4e081f80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:41.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.710+0000 7f4e06d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e0815e9b0 0x7f4e081f80f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46960/0 (socket says 172.21.15.32:46960) 2024-04-24T23:04:41.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.710+0000 7f4e06d9d640 1 -- 172.21.15.32:0/2479921896 learned_addr learned my addr 172.21.15.32:0/2479921896 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:41.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.710+0000 7f4e05d9b640 1 --2- 172.21.15.32:0/2479921896 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4e080d4210 0x7f4e080cc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:41.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.710+0000 7f4e0659c640 1 --2- 172.21.15.32:0/2479921896 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4e0806b820 0x7f4e080c9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:41.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.710+0000 7f4e06d9d640 1 -- 172.21.15.32:0/2479921896 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4e0806b820 msgr2=0x7f4e080c9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:41.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.710+0000 7f4e06d9d640 1 --2- 172.21.15.32:0/2479921896 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4e0806b820 0x7f4e080c9190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:41.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.710+0000 7f4e06d9d640 1 -- 172.21.15.32:0/2479921896 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4e080d4210 msgr2=0x7f4e080cc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:41.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.710+0000 7f4e06d9d640 1 --2- 172.21.15.32:0/2479921896 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4e080d4210 0x7f4e080cc8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:41.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.710+0000 7f4e06d9d640 1 -- 172.21.15.32:0/2479921896 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e0815fab0 con 0x7f4e0815e9b0 2024-04-24T23:04:41.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.710+0000 7f4e06d9d640 1 --2- 172.21.15.32:0/2479921896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e0815e9b0 0x7f4e081f80f0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f4df800cea0 tx=0x7f4df800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:41.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.710+0000 7f4dfeffd640 1 -- 172.21.15.32:0/2479921896 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df8003b90 con 0x7f4e0815e9b0 2024-04-24T23:04:41.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.710+0000 7f4dfeffd640 1 -- 172.21.15.32:0/2479921896 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4df8003d50 con 0x7f4e0815e9b0 2024-04-24T23:04:41.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.710+0000 7f4e0ce6b640 1 -- 172.21.15.32:0/2479921896 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e0815fd90 con 0x7f4e0815e9b0 2024-04-24T23:04:41.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.710+0000 7f4dfeffd640 1 -- 172.21.15.32:0/2479921896 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df8004260 con 0x7f4e0815e9b0 2024-04-24T23:04:41.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.710+0000 7f4e0ce6b640 1 -- 172.21.15.32:0/2479921896 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e0815f730 con 0x7f4e0815e9b0 2024-04-24T23:04:41.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.711+0000 7f4e0ce6b640 1 -- 172.21.15.32:0/2479921896 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e08068430 con 0x7f4e0815e9b0 2024-04-24T23:04:41.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.712+0000 7f4dfeffd640 1 -- 172.21.15.32:0/2479921896 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4df8004400 con 0x7f4e0815e9b0 2024-04-24T23:04:41.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.712+0000 7f4dfeffd640 1 --2- 172.21.15.32:0/2479921896 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4de806be30 0x7f4de806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:41.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.712+0000 7f4dfeffd640 1 -- 172.21.15.32:0/2479921896 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7f4df808b730 con 0x7f4e0815e9b0 2024-04-24T23:04:41.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.712+0000 7f4e0659c640 1 --2- 172.21.15.32:0/2479921896 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4de806be30 0x7f4de806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:41.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.713+0000 7f4e0659c640 1 --2- 172.21.15.32:0/2479921896 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4de806be30 0x7f4de806e2f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4e00034000 tx=0x7f4e0002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:41.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.716+0000 7f4dfeffd640 1 -- 172.21.15.32:0/2479921896 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4df8059b20 con 0x7f4e0815e9b0 2024-04-24T23:04:41.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.904+0000 7f4e0ce6b640 1 -- 172.21.15.32:0/2479921896 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4e080ca950 con 0x7f4e0815e9b0 2024-04-24T23:04:41.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.904+0000 7f4dfeffd640 1 -- 172.21.15.32:0/2479921896 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v113) ==== 104+0+11 (secure 0 0 0) 0x7f4df805de20 con 0x7f4e0815e9b0 2024-04-24T23:04:41.907 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-24T23:04:41.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.906+0000 7f4e0ce6b640 1 -- 172.21.15.32:0/2479921896 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4de806be30 msgr2=0x7f4de806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:41.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.906+0000 7f4e0ce6b640 1 --2- 172.21.15.32:0/2479921896 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4de806be30 0x7f4de806e2f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4e00034000 tx=0x7f4e0002e000 comp rx=0 tx=0).stop 2024-04-24T23:04:41.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.906+0000 7f4e0ce6b640 1 -- 172.21.15.32:0/2479921896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e0815e9b0 msgr2=0x7f4e081f80f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:41.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.906+0000 7f4e0ce6b640 1 --2- 172.21.15.32:0/2479921896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e0815e9b0 0x7f4e081f80f0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f4df800cea0 tx=0x7f4df800cf90 comp rx=0 tx=0).stop 2024-04-24T23:04:41.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.906+0000 7f4e0ce6b640 1 -- 172.21.15.32:0/2479921896 shutdown_connections 2024-04-24T23:04:41.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.906+0000 7f4e0ce6b640 1 --2- 172.21.15.32:0/2479921896 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4de806be30 0x7f4de806e2f0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:41.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.906+0000 7f4e0ce6b640 1 --2- 172.21.15.32:0/2479921896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e0815e9b0 0x7f4e081f80f0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:41.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.906+0000 7f4e0ce6b640 1 --2- 172.21.15.32:0/2479921896 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4e080d4210 0x7f4e080cc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:41.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.906+0000 7f4e0ce6b640 1 --2- 172.21.15.32:0/2479921896 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4e0806b820 0x7f4e080c9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:41.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.906+0000 7f4e0ce6b640 1 -- 172.21.15.32:0/2479921896 >> 172.21.15.32:0/2479921896 conn(0x7f4e080d11d0 msgr2=0x7f4e0815d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:41.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.906+0000 7f4e0ce6b640 1 -- 172.21.15.32:0/2479921896 shutdown_connections 2024-04-24T23:04:41.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:41.906+0000 7f4e0ce6b640 1 -- 172.21.15.32:0/2479921896 wait complete. 2024-04-24T23:04:41.920 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:04:42.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.026+0000 7efd3d207640 1 Processor -- start 2024-04-24T23:04:42.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.026+0000 7efd3d207640 1 -- start start 2024-04-24T23:04:42.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.027+0000 7efd3d207640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7efd38158dd0 0x7efd3806b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:42.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.027+0000 7efd3d207640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efd3806dc60 0x7efd380b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:42.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.027+0000 7efd3d207640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd3806bd10 con 0x7efd3806b860 2024-04-24T23:04:42.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.027+0000 7efd3d207640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7efd3815ef10 con 0x7efd38158dd0 2024-04-24T23:04:42.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.027+0000 7efd3d207640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7efd3815f1f0 con 0x7efd3806dc60 2024-04-24T23:04:42.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.027+0000 7efd36d9d640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7efd38158dd0 0x7efd3806b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:42.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.027+0000 7efd3759e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efd3806dc60 0x7efd380b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:42.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.027+0000 7efd36d9d640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7efd38158dd0 0x7efd3806b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.32:51414/0 (socket says 172.21.15.32:51414) 2024-04-24T23:04:42.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.027+0000 7efd36d9d640 1 -- 172.21.15.32:0/3619147047 learned_addr learned my addr 172.21.15.32:0/3619147047 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:42.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.027+0000 7efd2ffff640 1 --1- 172.21.15.32:0/3619147047 >> v1:172.21.15.121:6789/0 conn(0x7efd3806b860 0x7efd3806d550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:38820/0 (socket says 172.21.15.32:38820) 2024-04-24T23:04:42.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.027+0000 7efd36d9d640 1 -- 172.21.15.32:0/3619147047 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efd3806dc60 msgr2=0x7efd380b5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:42.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.027+0000 7efd36d9d640 1 --2- 172.21.15.32:0/3619147047 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efd3806dc60 0x7efd380b5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:42.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.027+0000 7efd36d9d640 1 -- 172.21.15.32:0/3619147047 >> v1:172.21.15.121:6789/0 conn(0x7efd3806b860 legacy=0x7efd3806d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:42.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.027+0000 7efd36d9d640 1 -- 172.21.15.32:0/3619147047 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd3815f4d0 con 0x7efd38158dd0 2024-04-24T23:04:42.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.028+0000 7efd36d9d640 1 --2- 172.21.15.32:0/3619147047 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efd38158dd0 0x7efd3806b320 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7efd28009870 tx=0x7efd2802a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2167fc17de3b64c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:42.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.028+0000 7efd3659c640 1 -- 172.21.15.32:0/3619147047 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd28037070 con 0x7efd38158dd0 2024-04-24T23:04:42.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.028+0000 7efd3659c640 1 -- 172.21.15.32:0/3619147047 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd28032440 con 0x7efd38158dd0 2024-04-24T23:04:42.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.028+0000 7efd3659c640 1 -- 172.21.15.32:0/3619147047 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd28032810 con 0x7efd38158dd0 2024-04-24T23:04:42.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.029+0000 7efd3d207640 1 -- 172.21.15.32:0/3619147047 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efd38158dd0 msgr2=0x7efd3806b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:42.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.029+0000 7efd3d207640 1 --2- 172.21.15.32:0/3619147047 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efd38158dd0 0x7efd3806b320 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7efd28009870 tx=0x7efd2802a270 comp rx=0 tx=0).stop 2024-04-24T23:04:42.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.029+0000 7efd3d207640 1 -- 172.21.15.32:0/3619147047 shutdown_connections 2024-04-24T23:04:42.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.029+0000 7efd3d207640 1 --2- 172.21.15.32:0/3619147047 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efd3806dc60 0x7efd380b5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:42.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.029+0000 7efd3d207640 1 --2- 172.21.15.32:0/3619147047 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efd38158dd0 0x7efd3806b320 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:42.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.029+0000 7efd3d207640 1 -- 172.21.15.32:0/3619147047 >> 172.21.15.32:0/3619147047 conn(0x7efd3806e800 msgr2=0x7efd3806ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:42.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.029+0000 7efd3d207640 1 -- 172.21.15.32:0/3619147047 shutdown_connections 2024-04-24T23:04:42.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.029+0000 7efd3d207640 1 -- 172.21.15.32:0/3619147047 wait complete. 2024-04-24T23:04:42.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.030+0000 7efd3d207640 1 Processor -- start 2024-04-24T23:04:42.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.030+0000 7efd3d207640 1 -- start start 2024-04-24T23:04:42.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.031+0000 7efd3d207640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efd3806b860 0x7efd3815bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:42.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.031+0000 7efd3d207640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efd3806dc60 0x7efd38157400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:42.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.031+0000 7efd3d207640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd38158dd0 0x7efd38143510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:42.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.031+0000 7efd3d207640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7efd3806bfe0 con 0x7efd38158dd0 2024-04-24T23:04:42.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.031+0000 7efd3d207640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7efd3815fcd0 con 0x7efd3806dc60 2024-04-24T23:04:42.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.031+0000 7efd3d207640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7efd381591b0 con 0x7efd3806b860 2024-04-24T23:04:42.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.031+0000 7efd3759e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd38158dd0 0x7efd38143510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:42.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.031+0000 7efd3759e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd38158dd0 0x7efd38143510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46964/0 (socket says 172.21.15.32:46964) 2024-04-24T23:04:42.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.031+0000 7efd3759e640 1 -- 172.21.15.32:0/446632623 learned_addr learned my addr 172.21.15.32:0/446632623 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:42.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.031+0000 7efd2ffff640 1 --2- 172.21.15.32:0/446632623 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efd3806dc60 0x7efd38157400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:42.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.031+0000 7efd36d9d640 1 --2- 172.21.15.32:0/446632623 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efd3806b860 0x7efd3815bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:42.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.032+0000 7efd3759e640 1 -- 172.21.15.32:0/446632623 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efd3806b860 msgr2=0x7efd3815bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:42.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.032+0000 7efd3759e640 1 --2- 172.21.15.32:0/446632623 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efd3806b860 0x7efd3815bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:42.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.032+0000 7efd3759e640 1 -- 172.21.15.32:0/446632623 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efd3806dc60 msgr2=0x7efd38157400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:42.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.032+0000 7efd3759e640 1 --2- 172.21.15.32:0/446632623 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efd3806dc60 0x7efd38157400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:42.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.032+0000 7efd3759e640 1 -- 172.21.15.32:0/446632623 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd3815c530 con 0x7efd38158dd0 2024-04-24T23:04:42.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.032+0000 7efd2ffff640 1 --2- 172.21.15.32:0/446632623 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efd3806dc60 0x7efd38157400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:04:42.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.032+0000 7efd3759e640 1 --2- 172.21.15.32:0/446632623 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd38158dd0 0x7efd38143510 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7efd3000efc0 tx=0x7efd3000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:42.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.032+0000 7efd2f7fe640 1 -- 172.21.15.32:0/446632623 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd30018020 con 0x7efd38158dd0 2024-04-24T23:04:42.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.032+0000 7efd2f7fe640 1 -- 172.21.15.32:0/446632623 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd30014480 con 0x7efd38158dd0 2024-04-24T23:04:42.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.032+0000 7efd3d207640 1 -- 172.21.15.32:0/446632623 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd381490a0 con 0x7efd38158dd0 2024-04-24T23:04:42.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.032+0000 7efd2f7fe640 1 -- 172.21.15.32:0/446632623 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd30014870 con 0x7efd38158dd0 2024-04-24T23:04:42.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.032+0000 7efd3d207640 1 -- 172.21.15.32:0/446632623 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd38149660 con 0x7efd38158dd0 2024-04-24T23:04:42.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.033+0000 7efd3d207640 1 -- 172.21.15.32:0/446632623 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd38068430 con 0x7efd38158dd0 2024-04-24T23:04:42.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.033+0000 7efd2f7fe640 1 -- 172.21.15.32:0/446632623 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efd30005d10 con 0x7efd38158dd0 2024-04-24T23:04:42.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.034+0000 7efd2f7fe640 1 --2- 172.21.15.32:0/446632623 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7efd1806be60 0x7efd1806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:42.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.034+0000 7efd2f7fe640 1 -- 172.21.15.32:0/446632623 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7efd3008b930 con 0x7efd38158dd0 2024-04-24T23:04:42.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.034+0000 7efd36d9d640 1 --2- 172.21.15.32:0/446632623 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7efd1806be60 0x7efd1806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:42.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.035+0000 7efd36d9d640 1 --2- 172.21.15.32:0/446632623 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7efd1806be60 0x7efd1806e320 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7efd28034000 tx=0x7efd2802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:42.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.038+0000 7efd2f7fe640 1 -- 172.21.15.32:0/446632623 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7efd30022070 con 0x7efd38158dd0 2024-04-24T23:04:42.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.222+0000 7efd3d207640 1 -- 172.21.15.32:0/446632623 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7efd3814f160 con 0x7efd38158dd0 2024-04-24T23:04:42.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.223+0000 7efd2f7fe640 1 -- 172.21.15.32:0/446632623 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v113) ==== 100+0+11 (secure 0 0 0) 0x7efd30010040 con 0x7efd38158dd0 2024-04-24T23:04:42.226 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-24T23:04:42.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.225+0000 7efd3d207640 1 -- 172.21.15.32:0/446632623 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7efd1806be60 msgr2=0x7efd1806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:42.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.225+0000 7efd3d207640 1 --2- 172.21.15.32:0/446632623 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7efd1806be60 0x7efd1806e320 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7efd28034000 tx=0x7efd2802e000 comp rx=0 tx=0).stop 2024-04-24T23:04:42.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.225+0000 7efd3d207640 1 -- 172.21.15.32:0/446632623 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd38158dd0 msgr2=0x7efd38143510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:42.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.225+0000 7efd3d207640 1 --2- 172.21.15.32:0/446632623 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd38158dd0 0x7efd38143510 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7efd3000efc0 tx=0x7efd3000c490 comp rx=0 tx=0).stop 2024-04-24T23:04:42.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.225+0000 7efd3d207640 1 -- 172.21.15.32:0/446632623 shutdown_connections 2024-04-24T23:04:42.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.225+0000 7efd3d207640 1 --2- 172.21.15.32:0/446632623 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7efd1806be60 0x7efd1806e320 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:42.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.225+0000 7efd3d207640 1 --2- 172.21.15.32:0/446632623 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd38158dd0 0x7efd38143510 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:42.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.225+0000 7efd3d207640 1 --2- 172.21.15.32:0/446632623 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efd3806dc60 0x7efd38157400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:42.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.225+0000 7efd3d207640 1 --2- 172.21.15.32:0/446632623 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efd3806b860 0x7efd3815bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:42.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.225+0000 7efd3d207640 1 -- 172.21.15.32:0/446632623 >> 172.21.15.32:0/446632623 conn(0x7efd3806e800 msgr2=0x7efd38161940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:42.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.225+0000 7efd3d207640 1 -- 172.21.15.32:0/446632623 shutdown_connections 2024-04-24T23:04:42.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.225+0000 7efd3d207640 1 -- 172.21.15.32:0/446632623 wait complete. 2024-04-24T23:04:42.238 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:04:42.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.336+0000 7f58f5f9d640 1 Processor -- start 2024-04-24T23:04:42.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.336+0000 7f58f5f9d640 1 -- start start 2024-04-24T23:04:42.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.337+0000 7f58f5f9d640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f58f00b44d0 0x7f58f00b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:42.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.337+0000 7f58f5f9d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f58f00af890 0x7f58f015cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:42.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.337+0000 7f58f5f9d640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58f00b6430 con 0x7f58f006c5f0 2024-04-24T23:04:42.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.337+0000 7f58f5f9d640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f58f0156f80 con 0x7f58f00b44d0 2024-04-24T23:04:42.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.337+0000 7f58f5f9d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f58f00b6150 con 0x7f58f00af890 2024-04-24T23:04:42.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.337+0000 7f58effff640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f58f00af890 0x7f58f015cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:42.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.337+0000 7f58eeffd640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f58f00b44d0 0x7f58f00b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:42.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.337+0000 7f58ef7fe640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f58f006c5f0 0x7f58f006c9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:38836/0 (socket says 172.21.15.32:38836) 2024-04-24T23:04:42.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.337+0000 7f58ef7fe640 1 -- 172.21.15.32:0/55276574 learned_addr learned my addr 172.21.15.32:0/55276574 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:04:42.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.338+0000 7f58ee7fc640 1 -- 172.21.15.32:0/55276574 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3284801733 0 0) 0x7f58f00b6430 con 0x7f58f006c5f0 2024-04-24T23:04:42.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.338+0000 7f58ee7fc640 1 -- 172.21.15.32:0/55276574 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f58d8003520 con 0x7f58f006c5f0 2024-04-24T23:04:42.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.338+0000 7f58eeffd640 1 -- 172.21.15.32:0/55276574 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f58f00af890 msgr2=0x7f58f015cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:42.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.338+0000 7f58eeffd640 1 --2- 172.21.15.32:0/55276574 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f58f00af890 0x7f58f015cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:42.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.338+0000 7f58eeffd640 1 -- 172.21.15.32:0/55276574 >> v1:172.21.15.121:6789/0 conn(0x7f58f006c5f0 legacy=0x7f58f006c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:42.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.338+0000 7f58eeffd640 1 -- 172.21.15.32:0/55276574 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58f00b39f0 con 0x7f58f00b44d0 2024-04-24T23:04:42.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.338+0000 7f58effff640 1 --2- 172.21.15.32:0/55276574 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f58f00af890 0x7f58f015cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:04:42.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.338+0000 7f58eeffd640 1 --2- 172.21.15.32:0/55276574 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58f00b44d0 0x7f58f00b5890 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f58e40097a0 tx=0x7f58e402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f60f8d2095ba3b6d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.339+0000 7f58ee7fc640 1 -- 172.21.15.32:0/55276574 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e4037070 con 0x7f58f00b44d0 2024-04-24T23:04:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.339+0000 7f58ee7fc640 1 -- 172.21.15.32:0/55276574 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58e402ae50 con 0x7f58f00b44d0 2024-04-24T23:04:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.339+0000 7f58ee7fc640 1 -- 172.21.15.32:0/55276574 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e4032660 con 0x7f58f00b44d0 2024-04-24T23:04:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.339+0000 7f58f5f9d640 1 -- 172.21.15.32:0/55276574 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58f00b44d0 msgr2=0x7f58f00b5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.339+0000 7f58f5f9d640 1 --2- 172.21.15.32:0/55276574 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58f00b44d0 0x7f58f00b5890 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f58e40097a0 tx=0x7f58e402a100 comp rx=0 tx=0).stop 2024-04-24T23:04:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.339+0000 7f58f5f9d640 1 -- 172.21.15.32:0/55276574 shutdown_connections 2024-04-24T23:04:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.339+0000 7f58f5f9d640 1 --2- 172.21.15.32:0/55276574 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f58f00af890 0x7f58f015cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.339+0000 7f58f5f9d640 1 --2- 172.21.15.32:0/55276574 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58f00b44d0 0x7f58f00b5890 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.339+0000 7f58f5f9d640 1 -- 172.21.15.32:0/55276574 >> 172.21.15.32:0/55276574 conn(0x7f58f006b110 msgr2=0x7f58f006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.339+0000 7f58f5f9d640 1 -- 172.21.15.32:0/55276574 shutdown_connections 2024-04-24T23:04:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.339+0000 7f58f5f9d640 1 -- 172.21.15.32:0/55276574 wait complete. 2024-04-24T23:04:42.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.340+0000 7f58f5f9d640 1 Processor -- start 2024-04-24T23:04:42.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.340+0000 7f58f5f9d640 1 -- start start 2024-04-24T23:04:42.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.341+0000 7f58f5f9d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f58f006c5f0 0x7f58f014fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:42.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.341+0000 7f58f5f9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f00af890 0x7f58f00cd770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:42.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.341+0000 7f58f5f9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58f00b44d0 0x7f58f01f4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:42.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.341+0000 7f58f5f9d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f58f00b5dd0 con 0x7f58f00af890 2024-04-24T23:04:42.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.341+0000 7f58f5f9d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f58f00b3fb0 con 0x7f58f00b44d0 2024-04-24T23:04:42.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.341+0000 7f58f5f9d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f58f015d000 con 0x7f58f006c5f0 2024-04-24T23:04:42.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.341+0000 7f58eeffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f00af890 0x7f58f00cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:42.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.341+0000 7f58eeffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f00af890 0x7f58f00cd770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46966/0 (socket says 172.21.15.32:46966) 2024-04-24T23:04:42.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.341+0000 7f58eeffd640 1 -- 172.21.15.32:0/2090523071 learned_addr learned my addr 172.21.15.32:0/2090523071 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:42.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.342+0000 7f58effff640 1 --2- 172.21.15.32:0/2090523071 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58f00b44d0 0x7f58f01f4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:42.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.342+0000 7f58ef7fe640 1 --2- 172.21.15.32:0/2090523071 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f58f006c5f0 0x7f58f014fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:42.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.342+0000 7f58eeffd640 1 -- 172.21.15.32:0/2090523071 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f58f006c5f0 msgr2=0x7f58f014fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:42.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.342+0000 7f58eeffd640 1 --2- 172.21.15.32:0/2090523071 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f58f006c5f0 0x7f58f014fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:42.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.342+0000 7f58eeffd640 1 -- 172.21.15.32:0/2090523071 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58f00b44d0 msgr2=0x7f58f01f4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:42.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.342+0000 7f58eeffd640 1 --2- 172.21.15.32:0/2090523071 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58f00b44d0 0x7f58f01f4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:42.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.342+0000 7f58eeffd640 1 -- 172.21.15.32:0/2090523071 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58f01503a0 con 0x7f58f00af890 2024-04-24T23:04:42.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.342+0000 7f58ef7fe640 1 --2- 172.21.15.32:0/2090523071 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f58f006c5f0 0x7f58f014fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:04:42.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.342+0000 7f58eeffd640 1 --2- 172.21.15.32:0/2090523071 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f00af890 0x7f58f00cd770 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f58e40098c0 tx=0x7f58e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:42.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.342+0000 7f58d7fff640 1 -- 172.21.15.32:0/2090523071 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e4041030 con 0x7f58f00af890 2024-04-24T23:04:42.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.342+0000 7f58d7fff640 1 -- 172.21.15.32:0/2090523071 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58e402ac50 con 0x7f58f00af890 2024-04-24T23:04:42.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.342+0000 7f58f5f9d640 1 -- 172.21.15.32:0/2090523071 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58f01f4cc0 con 0x7f58f00af890 2024-04-24T23:04:42.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.342+0000 7f58d7fff640 1 -- 172.21.15.32:0/2090523071 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e4037040 con 0x7f58f00af890 2024-04-24T23:04:42.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.342+0000 7f58f5f9d640 1 -- 172.21.15.32:0/2090523071 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58f00ccf70 con 0x7f58f00af890 2024-04-24T23:04:42.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.343+0000 7f58f5f9d640 1 -- 172.21.15.32:0/2090523071 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58f00ccd60 con 0x7f58f00af890 2024-04-24T23:04:42.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.344+0000 7f58d7fff640 1 -- 172.21.15.32:0/2090523071 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f58e4044070 con 0x7f58f00af890 2024-04-24T23:04:42.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.344+0000 7f58d7fff640 1 --2- 172.21.15.32:0/2090523071 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f58dc06be60 0x7f58dc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:42.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.344+0000 7f58d7fff640 1 -- 172.21.15.32:0/2090523071 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7f58e40ab570 con 0x7f58f00af890 2024-04-24T23:04:42.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.344+0000 7f58ef7fe640 1 --2- 172.21.15.32:0/2090523071 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f58dc06be60 0x7f58dc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:42.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.345+0000 7f58ef7fe640 1 --2- 172.21.15.32:0/2090523071 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f58dc06be60 0x7f58dc06e320 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f58f01f8840 tx=0x7f58e0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:42.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.348+0000 7f58d7fff640 1 -- 172.21.15.32:0/2090523071 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f58e4044e40 con 0x7f58f00af890 2024-04-24T23:04:42.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.569+0000 7f58f5f9d640 1 -- 172.21.15.32:0/2090523071 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58f0068430 con 0x7f58f00af890 2024-04-24T23:04:42.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.570+0000 7f58d7fff640 1 -- 172.21.15.32:0/2090523071 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5186 (secure 0 0 0) 0x7f58e407a040 con 0x7f58f00af890 2024-04-24T23:04:42.572 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:04:42.572 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":44,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6175,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/434273738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":434273738},{"type":"v1","addr":"172.21.15.121:6836","nonce":434273738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6178,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1104557785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1104557785},{"type":"v1","addr":"172.21.15.121:6837","nonce":1104557785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6888,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/3137388886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3137388886},{"type":"v1","addr":"172.21.15.32:6834","nonce":3137388886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:04:00.783579+0000","modified":"2024-04-24T23:04:03.790658+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6891},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6891":{"gid":6891,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/3327203898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":3327203898},{"type":"v1","addr":"172.21.15.32:6835","nonce":3327203898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6891,"qdb_cluster":[6891]},"id":6}]} 2024-04-24T23:04:42.572 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 44 2024-04-24T23:04:42.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.571+0000 7f58f5f9d640 1 -- 172.21.15.32:0/2090523071 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f58dc06be60 msgr2=0x7f58dc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:42.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.571+0000 7f58f5f9d640 1 --2- 172.21.15.32:0/2090523071 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f58dc06be60 0x7f58dc06e320 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f58f01f8840 tx=0x7f58e0009040 comp rx=0 tx=0).stop 2024-04-24T23:04:42.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.571+0000 7f58f5f9d640 1 -- 172.21.15.32:0/2090523071 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f00af890 msgr2=0x7f58f00cd770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:42.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.571+0000 7f58f5f9d640 1 --2- 172.21.15.32:0/2090523071 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f00af890 0x7f58f00cd770 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f58e40098c0 tx=0x7f58e40046c0 comp rx=0 tx=0).stop 2024-04-24T23:04:42.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.571+0000 7f58f5f9d640 1 -- 172.21.15.32:0/2090523071 shutdown_connections 2024-04-24T23:04:42.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.571+0000 7f58f5f9d640 1 --2- 172.21.15.32:0/2090523071 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f58dc06be60 0x7f58dc06e320 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:42.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.571+0000 7f58f5f9d640 1 --2- 172.21.15.32:0/2090523071 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f00af890 0x7f58f00cd770 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:42.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.571+0000 7f58f5f9d640 1 --2- 172.21.15.32:0/2090523071 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58f00b44d0 0x7f58f01f4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:42.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.571+0000 7f58f5f9d640 1 --2- 172.21.15.32:0/2090523071 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f58f006c5f0 0x7f58f014fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:42.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.571+0000 7f58f5f9d640 1 -- 172.21.15.32:0/2090523071 >> 172.21.15.32:0/2090523071 conn(0x7f58f006b110 msgr2=0x7f58f006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:42.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.572+0000 7f58f5f9d640 1 -- 172.21.15.32:0/2090523071 shutdown_connections 2024-04-24T23:04:42.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.572+0000 7f58f5f9d640 1 -- 172.21.15.32:0/2090523071 wait complete. 2024-04-24T23:04:42.585 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:04:42.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.684+0000 7f3231770640 1 Processor -- start 2024-04-24T23:04:42.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.684+0000 7f3231770640 1 -- start start 2024-04-24T23:04:42.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.684+0000 7f3231770640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f322c06c5f0 0x7f322c06c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:42.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.684+0000 7f3231770640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f322c0b6430 con 0x7f322c0af890 2024-04-24T23:04:42.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.684+0000 7f3231770640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f322c156f80 con 0x7f322c0b44d0 2024-04-24T23:04:42.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.684+0000 7f3231770640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f322c0b6150 con 0x7f322c06c5f0 2024-04-24T23:04:42.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.685+0000 7f322b7fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f322c0af890 0x7f322c15cac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60790/0 (socket says 172.21.15.32:60790) 2024-04-24T23:04:42.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.685+0000 7f322b7fe640 1 -- 172.21.15.32:0/2784908281 learned_addr learned my addr 172.21.15.32:0/2784908281 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:04:42.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.685+0000 7f322affd640 1 --2- 172.21.15.32:0/2784908281 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f322c06c5f0 0x7f322c06c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:42.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.685+0000 7f322a7fc640 1 -- 172.21.15.32:0/2784908281 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1318807407 0 0) 0x7f322c0b6430 con 0x7f322c0af890 2024-04-24T23:04:42.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.685+0000 7f322a7fc640 1 -- 172.21.15.32:0/2784908281 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3218003520 con 0x7f322c0af890 2024-04-24T23:04:42.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.685+0000 7f322a7fc640 1 -- 172.21.15.32:0/2784908281 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4026590329 0 0) 0x7f322c156f80 con 0x7f322c0b44d0 2024-04-24T23:04:42.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.685+0000 7f322a7fc640 1 -- 172.21.15.32:0/2784908281 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f322c0b6430 con 0x7f322c0b44d0 2024-04-24T23:04:42.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.685+0000 7f322a7fc640 1 -- 172.21.15.32:0/2784908281 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2399926156 0 0) 0x7f3218003520 con 0x7f322c0af890 2024-04-24T23:04:42.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.685+0000 7f322a7fc640 1 -- 172.21.15.32:0/2784908281 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f322c06c5f0 msgr2=0x7f322c06c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:42.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.685+0000 7f322a7fc640 1 --2- 172.21.15.32:0/2784908281 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f322c06c5f0 0x7f322c06c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:42.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.685+0000 7f322a7fc640 1 -- 172.21.15.32:0/2784908281 >> v1:172.21.15.121:6789/0 conn(0x7f322c0b44d0 legacy=0x7f322c0b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:42.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.685+0000 7f322a7fc640 1 -- 172.21.15.32:0/2784908281 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f322c0b39f0 con 0x7f322c0af890 2024-04-24T23:04:42.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.685+0000 7f322a7fc640 1 -- 172.21.15.32:0/2784908281 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f32240019f0 con 0x7f322c0af890 2024-04-24T23:04:42.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.685+0000 7f322a7fc640 1 -- 172.21.15.32:0/2784908281 >> v1:172.21.15.32:6789/0 conn(0x7f322c0af890 legacy=0x7f322c15cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:42.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.685+0000 7f322a7fc640 1 --2- 172.21.15.32:0/2784908281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3218003b70 0x7f3218003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:42.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.685+0000 7f322a7fc640 1 --2- 172.21.15.32:0/2784908281 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3218004800 0x7f3218004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:42.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.685+0000 7f322a7fc640 1 --2- 172.21.15.32:0/2784908281 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f322c06c5f0 0x7f3218005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:42.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.685+0000 7f322a7fc640 1 -- 172.21.15.32:0/2784908281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f322c0b6150 con 0x7f3218003b70 2024-04-24T23:04:42.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.685+0000 7f322a7fc640 1 -- 172.21.15.32:0/2784908281 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f321800bb50 con 0x7f3218004800 2024-04-24T23:04:42.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.685+0000 7f322a7fc640 1 -- 172.21.15.32:0/2784908281 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f321800bd20 con 0x7f322c06c5f0 2024-04-24T23:04:42.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.685+0000 7f322affd640 1 --2- 172.21.15.32:0/2784908281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3218003b70 0x7f3218003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:42.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.685+0000 7f3223fff640 1 --2- 172.21.15.32:0/2784908281 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3218004800 0x7f3218004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:42.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.685+0000 7f322b7fe640 1 --2- 172.21.15.32:0/2784908281 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f322c06c5f0 0x7f3218005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:42.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.686+0000 7f322affd640 1 -- 172.21.15.32:0/2784908281 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f322c06c5f0 msgr2=0x7f3218005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:42.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.686+0000 7f322affd640 1 --2- 172.21.15.32:0/2784908281 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f322c06c5f0 0x7f3218005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:42.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.686+0000 7f322affd640 1 -- 172.21.15.32:0/2784908281 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3218004800 msgr2=0x7f3218004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:42.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.686+0000 7f322affd640 1 --2- 172.21.15.32:0/2784908281 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3218004800 0x7f3218004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:42.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.686+0000 7f322affd640 1 -- 172.21.15.32:0/2784908281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3218003520 con 0x7f3218003b70 2024-04-24T23:04:42.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.686+0000 7f322affd640 1 --2- 172.21.15.32:0/2784908281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3218003b70 0x7f3218003f70 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f321c009af0 tx=0x7f3218005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b64a268fa32d5c27 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:42.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.686+0000 7f322a7fc640 1 -- 172.21.15.32:0/2784908281 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321c036070 con 0x7f3218003b70 2024-04-24T23:04:42.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.686+0000 7f322a7fc640 1 -- 172.21.15.32:0/2784908281 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f321c031400 con 0x7f3218003b70 2024-04-24T23:04:42.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.686+0000 7f322a7fc640 1 -- 172.21.15.32:0/2784908281 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321c009690 con 0x7f3218003b70 2024-04-24T23:04:42.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.687+0000 7f3231770640 1 -- 172.21.15.32:0/2784908281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3218003b70 msgr2=0x7f3218003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:42.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.687+0000 7f3231770640 1 --2- 172.21.15.32:0/2784908281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3218003b70 0x7f3218003f70 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f321c009af0 tx=0x7f3218005830 comp rx=0 tx=0).stop 2024-04-24T23:04:42.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.687+0000 7f322b7fe640 1 -- 172.21.15.32:0/2784908281 reap_dead start 2024-04-24T23:04:42.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.687+0000 7f3231770640 1 -- 172.21.15.32:0/2784908281 shutdown_connections 2024-04-24T23:04:42.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.687+0000 7f3231770640 1 -- 172.21.15.32:0/2784908281 >> 172.21.15.32:0/2784908281 conn(0x7f322c06b110 msgr2=0x7f322c06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:42.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.687+0000 7f3231770640 1 -- 172.21.15.32:0/2784908281 shutdown_connections 2024-04-24T23:04:42.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.687+0000 7f3231770640 1 -- 172.21.15.32:0/2784908281 wait complete. 2024-04-24T23:04:42.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.688+0000 7f3231770640 1 Processor -- start 2024-04-24T23:04:42.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.688+0000 7f3231770640 1 -- start start 2024-04-24T23:04:42.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.689+0000 7f3231770640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3218003b70 0x7f322c148c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:42.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.689+0000 7f3231770640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f322c0aff80 0x7f322c0b0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:42.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.689+0000 7f3231770640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f322c1e7370 0x7f322c1e77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:42.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.689+0000 7f3231770640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f322c06c9d0 con 0x7f3218003b70 2024-04-24T23:04:42.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.689+0000 7f3231770640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f322c0b3fb0 con 0x7f322c1e7370 2024-04-24T23:04:42.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.689+0000 7f3231770640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f322c06cb50 con 0x7f322c0aff80 2024-04-24T23:04:42.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.689+0000 7f322affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3218003b70 0x7f322c148c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:42.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.689+0000 7f322affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3218003b70 0x7f322c148c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46990/0 (socket says 172.21.15.32:46990) 2024-04-24T23:04:42.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.689+0000 7f322affd640 1 -- 172.21.15.32:0/843033173 learned_addr learned my addr 172.21.15.32:0/843033173 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:42.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.689+0000 7f3223fff640 1 --2- 172.21.15.32:0/843033173 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f322c0aff80 0x7f322c0b0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:42.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.689+0000 7f322b7fe640 1 --2- 172.21.15.32:0/843033173 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f322c1e7370 0x7f322c1e77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:42.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.689+0000 7f322affd640 1 -- 172.21.15.32:0/843033173 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f322c0aff80 msgr2=0x7f322c0b0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:42.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.689+0000 7f322affd640 1 --2- 172.21.15.32:0/843033173 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f322c0aff80 0x7f322c0b0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:42.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.689+0000 7f322affd640 1 -- 172.21.15.32:0/843033173 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f322c1e7370 msgr2=0x7f322c1e77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:42.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.689+0000 7f322affd640 1 --2- 172.21.15.32:0/843033173 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f322c1e7370 0x7f322c1e77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:42.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.689+0000 7f322affd640 1 -- 172.21.15.32:0/843033173 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f322c1f2ae0 con 0x7f3218003b70 2024-04-24T23:04:42.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.690+0000 7f322affd640 1 --2- 172.21.15.32:0/843033173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3218003b70 0x7f322c148c80 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f321c034040 tx=0x7f321c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:42.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.690+0000 7f32237fe640 1 -- 172.21.15.32:0/843033173 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321c041020 con 0x7f3218003b70 2024-04-24T23:04:42.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.690+0000 7f32237fe640 1 -- 172.21.15.32:0/843033173 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f321c03c470 con 0x7f3218003b70 2024-04-24T23:04:42.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.690+0000 7f3231770640 1 -- 172.21.15.32:0/843033173 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f322c1f2dc0 con 0x7f3218003b70 2024-04-24T23:04:42.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.690+0000 7f32237fe640 1 -- 172.21.15.32:0/843033173 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321c036040 con 0x7f3218003b70 2024-04-24T23:04:42.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.690+0000 7f3231770640 1 -- 172.21.15.32:0/843033173 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f322c1f3380 con 0x7f3218003b70 2024-04-24T23:04:42.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.691+0000 7f3231770640 1 -- 172.21.15.32:0/843033173 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f322c0b0940 con 0x7f3218003b70 2024-04-24T23:04:42.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.691+0000 7f32237fe640 1 -- 172.21.15.32:0/843033173 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f321c044020 con 0x7f3218003b70 2024-04-24T23:04:42.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.692+0000 7f32237fe640 1 --2- 172.21.15.32:0/843033173 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f320c06be30 0x7f320c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:42.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.692+0000 7f32237fe640 1 -- 172.21.15.32:0/843033173 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7f321c0ab5d0 con 0x7f3218003b70 2024-04-24T23:04:42.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.692+0000 7f3223fff640 1 --2- 172.21.15.32:0/843033173 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f320c06be30 0x7f320c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:42.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.692+0000 7f3223fff640 1 --2- 172.21.15.32:0/843033173 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f320c06be30 0x7f320c06e2f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f322c1f0550 tx=0x7f3214009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:42.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.696+0000 7f32237fe640 1 -- 172.21.15.32:0/843033173 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f321c04b030 con 0x7f3218003b70 2024-04-24T23:04:42.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.882+0000 7f3231770640 1 -- 172.21.15.32:0/843033173 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f322c068430 con 0x7f3218003b70 2024-04-24T23:04:42.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.883+0000 7f32237fe640 1 -- 172.21.15.32:0/843033173 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) ==== 74+0+16696 (secure 0 0 0) 0x7f321c079920 con 0x7f3218003b70 2024-04-24T23:04:42.886 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:04:42.886 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":113,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:04:31.618095+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:03:59.079157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-24T23:03:59.765179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:04:42.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.885+0000 7f3231770640 1 -- 172.21.15.32:0/843033173 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f320c06be30 msgr2=0x7f320c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:42.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.885+0000 7f3231770640 1 --2- 172.21.15.32:0/843033173 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f320c06be30 0x7f320c06e2f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f322c1f0550 tx=0x7f3214009040 comp rx=0 tx=0).stop 2024-04-24T23:04:42.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.885+0000 7f3231770640 1 -- 172.21.15.32:0/843033173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3218003b70 msgr2=0x7f322c148c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:42.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.885+0000 7f3231770640 1 --2- 172.21.15.32:0/843033173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3218003b70 0x7f322c148c80 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f321c034040 tx=0x7f321c004730 comp rx=0 tx=0).stop 2024-04-24T23:04:42.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.885+0000 7f3231770640 1 -- 172.21.15.32:0/843033173 shutdown_connections 2024-04-24T23:04:42.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.885+0000 7f3231770640 1 --2- 172.21.15.32:0/843033173 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f320c06be30 0x7f320c06e2f0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:42.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.885+0000 7f3231770640 1 --2- 172.21.15.32:0/843033173 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f322c1e7370 0x7f322c1e77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:42.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.885+0000 7f3231770640 1 --2- 172.21.15.32:0/843033173 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f322c0aff80 0x7f322c0b0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:42.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.885+0000 7f3231770640 1 --2- 172.21.15.32:0/843033173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3218003b70 0x7f322c148c80 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:42.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.885+0000 7f3231770640 1 -- 172.21.15.32:0/843033173 >> 172.21.15.32:0/843033173 conn(0x7f322c06b110 msgr2=0x7f322c0b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:42.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.885+0000 7f3231770640 1 -- 172.21.15.32:0/843033173 shutdown_connections 2024-04-24T23:04:42.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.885+0000 7f3231770640 1 -- 172.21.15.32:0/843033173 wait complete. 2024-04-24T23:04:42.899 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-24T23:04:42.899 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:04:42.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.997+0000 7f381199d640 1 Processor -- start 2024-04-24T23:04:43.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.997+0000 7f381199d640 1 -- start start 2024-04-24T23:04:43.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.998+0000 7f381199d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f380c15b380 0x7f380c06c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:43.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.998+0000 7f381199d640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f380c0ceb40 0x7f380c06cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:43.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.998+0000 7f381199d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f380c0b4520 con 0x7f380c15cb00 2024-04-24T23:04:43.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.998+0000 7f381199d640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f380c0cef90 con 0x7f380c0ceb40 2024-04-24T23:04:43.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.998+0000 7f381199d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f380c0b4800 con 0x7f380c15b380 2024-04-24T23:04:43.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.998+0000 7f380a7fc640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f380c15cb00 0x7f380c0ce430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60802/0 (socket says 172.21.15.32:60802) 2024-04-24T23:04:43.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.998+0000 7f380a7fc640 1 -- 172.21.15.32:0/3078410110 learned_addr learned my addr 172.21.15.32:0/3078410110 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:04:43.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.998+0000 7f380b7fe640 1 --2- 172.21.15.32:0/3078410110 >> v2:172.21.15.121:3300/0 conn(0x7f380c0ceb40 0x7f380c06cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:43.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.998+0000 7f380affd640 1 --2- 172.21.15.32:0/3078410110 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f380c15b380 0x7f380c06c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:43.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.998+0000 7f3809ffb640 1 -- 172.21.15.32:0/3078410110 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3404639524 0 0) 0x7f380c0b4520 con 0x7f380c15cb00 2024-04-24T23:04:43.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.998+0000 7f3809ffb640 1 -- 172.21.15.32:0/3078410110 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f37f8003520 con 0x7f380c15cb00 2024-04-24T23:04:43.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.998+0000 7f380b7fe640 1 -- 172.21.15.32:0/3078410110 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f380c15b380 msgr2=0x7f380c06c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:43.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.998+0000 7f380b7fe640 1 --2- 172.21.15.32:0/3078410110 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f380c15b380 0x7f380c06c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:43.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.998+0000 7f380b7fe640 1 -- 172.21.15.32:0/3078410110 >> v1:172.21.15.32:6789/0 conn(0x7f380c15cb00 legacy=0x7f380c0ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:43.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.998+0000 7f380b7fe640 1 -- 172.21.15.32:0/3078410110 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f380c0b4ae0 con 0x7f380c0ceb40 2024-04-24T23:04:43.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.999+0000 7f380b7fe640 1 --2- 172.21.15.32:0/3078410110 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f380c0ceb40 0x7f380c06cb30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f37fc0097a0 tx=0x7f37fc02a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fc8128a515406da server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:43.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.999+0000 7f3809ffb640 1 -- 172.21.15.32:0/3078410110 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37fc037070 con 0x7f380c0ceb40 2024-04-24T23:04:43.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.999+0000 7f3809ffb640 1 -- 172.21.15.32:0/3078410110 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37fc02ae50 con 0x7f380c0ceb40 2024-04-24T23:04:43.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:42.999+0000 7f3809ffb640 1 -- 172.21.15.32:0/3078410110 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37fc032640 con 0x7f380c0ceb40 2024-04-24T23:04:43.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.000+0000 7f381199d640 1 -- 172.21.15.32:0/3078410110 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f380c0ceb40 msgr2=0x7f380c06cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:43.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.000+0000 7f381199d640 1 --2- 172.21.15.32:0/3078410110 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f380c0ceb40 0x7f380c06cb30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f37fc0097a0 tx=0x7f37fc02a100 comp rx=0 tx=0).stop 2024-04-24T23:04:43.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.000+0000 7f381199d640 1 -- 172.21.15.32:0/3078410110 shutdown_connections 2024-04-24T23:04:43.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.000+0000 7f381199d640 1 --2- 172.21.15.32:0/3078410110 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f380c0ceb40 0x7f380c06cb30 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:43.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.000+0000 7f381199d640 1 --2- 172.21.15.32:0/3078410110 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f380c15b380 0x7f380c06c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:43.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.000+0000 7f381199d640 1 -- 172.21.15.32:0/3078410110 >> 172.21.15.32:0/3078410110 conn(0x7f380c06b110 msgr2=0x7f380c06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:43.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.000+0000 7f381199d640 1 -- 172.21.15.32:0/3078410110 shutdown_connections 2024-04-24T23:04:43.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.000+0000 7f381199d640 1 -- 172.21.15.32:0/3078410110 wait complete. 2024-04-24T23:04:43.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.001+0000 7f381199d640 1 Processor -- start 2024-04-24T23:04:43.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.001+0000 7f381199d640 1 -- start start 2024-04-24T23:04:43.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.002+0000 7f381199d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f380c0ceb40 0x7f380c158a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:43.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.002+0000 7f381199d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f380c15b380 0x7f380c15a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:43.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.002+0000 7f381199d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f380c15cb00 0x7f380c070c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:43.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.002+0000 7f381199d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f380c15b760 con 0x7f380c15cb00 2024-04-24T23:04:43.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.002+0000 7f381199d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f380c0b50a0 con 0x7f380c0ceb40 2024-04-24T23:04:43.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.002+0000 7f381199d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f380c0b55c0 con 0x7f380c15b380 2024-04-24T23:04:43.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.002+0000 7f380b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f380c15cb00 0x7f380c070c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:43.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.002+0000 7f380b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f380c15cb00 0x7f380c070c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47004/0 (socket says 172.21.15.32:47004) 2024-04-24T23:04:43.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.002+0000 7f380b7fe640 1 -- 172.21.15.32:0/1036741044 learned_addr learned my addr 172.21.15.32:0/1036741044 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:43.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.002+0000 7f380affd640 1 --2- 172.21.15.32:0/1036741044 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f380c0ceb40 0x7f380c158a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:43.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.002+0000 7f380a7fc640 1 --2- 172.21.15.32:0/1036741044 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f380c15b380 0x7f380c15a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:43.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.002+0000 7f380b7fe640 1 -- 172.21.15.32:0/1036741044 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f380c15b380 msgr2=0x7f380c15a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:43.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.002+0000 7f380b7fe640 1 --2- 172.21.15.32:0/1036741044 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f380c15b380 0x7f380c15a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:43.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.003+0000 7f380b7fe640 1 -- 172.21.15.32:0/1036741044 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f380c0ceb40 msgr2=0x7f380c158a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:43.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.003+0000 7f380b7fe640 1 --2- 172.21.15.32:0/1036741044 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f380c0ceb40 0x7f380c158a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:43.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.003+0000 7f380b7fe640 1 -- 172.21.15.32:0/1036741044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f380c0b4540 con 0x7f380c15cb00 2024-04-24T23:04:43.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.003+0000 7f380a7fc640 1 --2- 172.21.15.32:0/1036741044 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f380c15b380 0x7f380c15a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:04:43.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.003+0000 7f380b7fe640 1 --2- 172.21.15.32:0/1036741044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f380c15cb00 0x7f380c070c20 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f37fc035040 tx=0x7f37fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:43.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.003+0000 7f3803fff640 1 -- 172.21.15.32:0/1036741044 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37fc041070 con 0x7f380c15cb00 2024-04-24T23:04:43.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.003+0000 7f3803fff640 1 -- 172.21.15.32:0/1036741044 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37fc03c460 con 0x7f380c15cb00 2024-04-24T23:04:43.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.003+0000 7f381199d640 1 -- 172.21.15.32:0/1036741044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f380c159090 con 0x7f380c15cb00 2024-04-24T23:04:43.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.003+0000 7f3803fff640 1 -- 172.21.15.32:0/1036741044 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37fc037040 con 0x7f380c15cb00 2024-04-24T23:04:43.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.003+0000 7f381199d640 1 -- 172.21.15.32:0/1036741044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f380c159650 con 0x7f380c15cb00 2024-04-24T23:04:43.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.004+0000 7f381199d640 1 -- 172.21.15.32:0/1036741044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f380c068430 con 0x7f380c15cb00 2024-04-24T23:04:43.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.004+0000 7f3803fff640 1 -- 172.21.15.32:0/1036741044 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f37fc03a070 con 0x7f380c15cb00 2024-04-24T23:04:43.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.005+0000 7f3803fff640 1 --2- 172.21.15.32:0/1036741044 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f37ec06be30 0x7f37ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:43.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.005+0000 7f3803fff640 1 -- 172.21.15.32:0/1036741044 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7f37fc0ab0e0 con 0x7f380c15cb00 2024-04-24T23:04:43.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.005+0000 7f380affd640 1 --2- 172.21.15.32:0/1036741044 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f37ec06be30 0x7f37ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:43.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.006+0000 7f380affd640 1 --2- 172.21.15.32:0/1036741044 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f37ec06be30 0x7f37ec06e2f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f38040074a0 tx=0x7f380400b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:43.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.009+0000 7f3803fff640 1 -- 172.21.15.32:0/1036741044 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f380c0b4540 con 0x7f380c15cb00 2024-04-24T23:04:43.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.230+0000 7f381199d640 1 -- 172.21.15.32:0/1036741044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f380c06c000 con 0x7f380c15cb00 2024-04-24T23:04:43.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.231+0000 7f3803fff640 1 -- 172.21.15.32:0/1036741044 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5186 (secure 0 0 0) 0x7f37fc079b10 con 0x7f380c15cb00 2024-04-24T23:04:43.234 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:04:43.234 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":44,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6175,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/434273738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":434273738},{"type":"v1","addr":"172.21.15.121:6836","nonce":434273738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6178,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1104557785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1104557785},{"type":"v1","addr":"172.21.15.121:6837","nonce":1104557785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6888,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/3137388886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3137388886},{"type":"v1","addr":"172.21.15.32:6834","nonce":3137388886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:04:00.783579+0000","modified":"2024-04-24T23:04:03.790658+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6891},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6891":{"gid":6891,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/3327203898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":3327203898},{"type":"v1","addr":"172.21.15.32:6835","nonce":3327203898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6891,"qdb_cluster":[6891]},"id":6}]} 2024-04-24T23:04:43.234 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 44 2024-04-24T23:04:43.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.233+0000 7f381199d640 1 -- 172.21.15.32:0/1036741044 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f37ec06be30 msgr2=0x7f37ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:43.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.233+0000 7f381199d640 1 --2- 172.21.15.32:0/1036741044 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f37ec06be30 0x7f37ec06e2f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f38040074a0 tx=0x7f380400b040 comp rx=0 tx=0).stop 2024-04-24T23:04:43.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.233+0000 7f381199d640 1 -- 172.21.15.32:0/1036741044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f380c15cb00 msgr2=0x7f380c070c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:43.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.233+0000 7f381199d640 1 --2- 172.21.15.32:0/1036741044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f380c15cb00 0x7f380c070c20 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f37fc035040 tx=0x7f37fc0046c0 comp rx=0 tx=0).stop 2024-04-24T23:04:43.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.233+0000 7f381199d640 1 -- 172.21.15.32:0/1036741044 shutdown_connections 2024-04-24T23:04:43.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.233+0000 7f381199d640 1 --2- 172.21.15.32:0/1036741044 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f37ec06be30 0x7f37ec06e2f0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:43.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.233+0000 7f381199d640 1 --2- 172.21.15.32:0/1036741044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f380c15cb00 0x7f380c070c20 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:43.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.233+0000 7f381199d640 1 --2- 172.21.15.32:0/1036741044 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f380c15b380 0x7f380c15a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:43.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.233+0000 7f381199d640 1 --2- 172.21.15.32:0/1036741044 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f380c0ceb40 0x7f380c158a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:43.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.233+0000 7f381199d640 1 -- 172.21.15.32:0/1036741044 >> 172.21.15.32:0/1036741044 conn(0x7f380c06b110 msgr2=0x7f380c15e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:43.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.233+0000 7f381199d640 1 -- 172.21.15.32:0/1036741044 shutdown_connections 2024-04-24T23:04:43.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.233+0000 7f381199d640 1 -- 172.21.15.32:0/1036741044 wait complete. 2024-04-24T23:04:43.246 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:04:43.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.344+0000 7ff0acfe7640 1 Processor -- start 2024-04-24T23:04:43.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.344+0000 7ff0acfe7640 1 -- start start 2024-04-24T23:04:43.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.345+0000 7ff0acfe7640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff0a815c5b0 0x7ff0a80b4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:43.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.345+0000 7ff0acfe7640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff0a8072cf0 0x7ff0a806c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:43.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.345+0000 7ff0acfe7640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0a815c990 con 0x7ff0a8071770 2024-04-24T23:04:43.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.345+0000 7ff0acfe7640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7ff0a8072860 con 0x7ff0a815c5b0 2024-04-24T23:04:43.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.345+0000 7ff0acfe7640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7ff0a8071c50 con 0x7ff0a8072cf0 2024-04-24T23:04:43.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.345+0000 7ff0a659c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff0a815c5b0 0x7ff0a80b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:43.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.345+0000 7ff0a6d9d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff0a8071770 0x7ff0a80cef70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60816/0 (socket says 172.21.15.32:60816) 2024-04-24T23:04:43.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.345+0000 7ff0a659c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff0a815c5b0 0x7ff0a80b4030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47016/0 (socket says 172.21.15.32:47016) 2024-04-24T23:04:43.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.345+0000 7ff0a6d9d640 1 -- 172.21.15.32:0/1846992089 learned_addr learned my addr 172.21.15.32:0/1846992089 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:04:43.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.345+0000 7ff0a5d9b640 1 --2- 172.21.15.32:0/1846992089 >> v2:172.21.15.121:3300/0 conn(0x7ff0a8072cf0 0x7ff0a806c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:43.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.345+0000 7ff0a659c640 1 -- 172.21.15.32:0/1846992089 >> v2:172.21.15.121:3300/0 conn(0x7ff0a8072cf0 msgr2=0x7ff0a806c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:43.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.345+0000 7ff0a659c640 1 --2- 172.21.15.32:0/1846992089 >> v2:172.21.15.121:3300/0 conn(0x7ff0a8072cf0 0x7ff0a806c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:43.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.345+0000 7ff0a659c640 1 -- 172.21.15.32:0/1846992089 >> v1:172.21.15.32:6789/0 conn(0x7ff0a8071770 legacy=0x7ff0a80cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:43.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.346+0000 7ff0a659c640 1 -- 172.21.15.32:0/1846992089 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0a8071f30 con 0x7ff0a815c5b0 2024-04-24T23:04:43.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.345+0000 7ff0a559a640 1 -- 172.21.15.32:0/1846992089 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1517085792 0 0) 0x7ff098001490 con 0x7ff0a8071770 2024-04-24T23:04:43.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.346+0000 7ff0a5d9b640 1 --2- 172.21.15.32:0/1846992089 >> v2:172.21.15.121:3300/0 conn(0x7ff0a8072cf0 0x7ff0a806c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:04:43.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.346+0000 7ff0a659c640 1 --2- 172.21.15.32:0/1846992089 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0a815c5b0 0x7ff0a80b4030 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7ff0a8160050 tx=0x7ff0a002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4279b6f17fcdc118 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:43.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.346+0000 7ff0a559a640 1 -- 172.21.15.32:0/1846992089 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a0037070 con 0x7ff0a815c5b0 2024-04-24T23:04:43.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.346+0000 7ff0a559a640 1 -- 172.21.15.32:0/1846992089 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0a0032440 con 0x7ff0a815c5b0 2024-04-24T23:04:43.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.346+0000 7ff0a559a640 1 -- 172.21.15.32:0/1846992089 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a002ae60 con 0x7ff0a815c5b0 2024-04-24T23:04:43.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.347+0000 7ff0acfe7640 1 -- 172.21.15.32:0/1846992089 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0a815c5b0 msgr2=0x7ff0a80b4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:43.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.347+0000 7ff0acfe7640 1 --2- 172.21.15.32:0/1846992089 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0a815c5b0 0x7ff0a80b4030 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7ff0a8160050 tx=0x7ff0a002a110 comp rx=0 tx=0).stop 2024-04-24T23:04:43.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.347+0000 7ff0acfe7640 1 -- 172.21.15.32:0/1846992089 shutdown_connections 2024-04-24T23:04:43.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.347+0000 7ff0acfe7640 1 --2- 172.21.15.32:0/1846992089 >> v2:172.21.15.121:3300/0 conn(0x7ff0a8072cf0 0x7ff0a806c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:43.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.347+0000 7ff0acfe7640 1 --2- 172.21.15.32:0/1846992089 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0a815c5b0 0x7ff0a80b4030 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:43.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.347+0000 7ff0acfe7640 1 -- 172.21.15.32:0/1846992089 >> 172.21.15.32:0/1846992089 conn(0x7ff0a806b110 msgr2=0x7ff0a806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:43.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.347+0000 7ff0acfe7640 1 -- 172.21.15.32:0/1846992089 shutdown_connections 2024-04-24T23:04:43.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.347+0000 7ff0acfe7640 1 -- 172.21.15.32:0/1846992089 wait complete. 2024-04-24T23:04:43.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.348+0000 7ff0acfe7640 1 Processor -- start 2024-04-24T23:04:43.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.348+0000 7ff0acfe7640 1 -- start start 2024-04-24T23:04:43.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.349+0000 7ff0acfe7640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff0a8071770 0x7ff0a815b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:43.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.349+0000 7ff0acfe7640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0a8072cf0 0x7ff0a81f2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:43.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.349+0000 7ff0acfe7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0a815c5b0 0x7ff0a813dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:43.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.349+0000 7ff0acfe7640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a80b4570 con 0x7ff0a815c5b0 2024-04-24T23:04:43.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.349+0000 7ff0acfe7640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a80724f0 con 0x7ff0a8072cf0 2024-04-24T23:04:43.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.349+0000 7ff0acfe7640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff0a80b46f0 con 0x7ff0a8071770 2024-04-24T23:04:43.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.349+0000 7ff0a6d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0a815c5b0 0x7ff0a813dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:43.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.349+0000 7ff0a6d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0a815c5b0 0x7ff0a813dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47032/0 (socket says 172.21.15.32:47032) 2024-04-24T23:04:43.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.349+0000 7ff0a6d9d640 1 -- 172.21.15.32:0/357432881 learned_addr learned my addr 172.21.15.32:0/357432881 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:43.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.349+0000 7ff0a5d9b640 1 --2- 172.21.15.32:0/357432881 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0a8072cf0 0x7ff0a81f2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:43.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.349+0000 7ff0a659c640 1 --2- 172.21.15.32:0/357432881 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff0a8071770 0x7ff0a815b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:43.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.349+0000 7ff0a6d9d640 1 -- 172.21.15.32:0/357432881 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff0a8071770 msgr2=0x7ff0a815b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:43.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.350+0000 7ff0a6d9d640 1 --2- 172.21.15.32:0/357432881 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff0a8071770 0x7ff0a815b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:43.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.350+0000 7ff0a6d9d640 1 -- 172.21.15.32:0/357432881 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0a8072cf0 msgr2=0x7ff0a81f2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:43.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.350+0000 7ff0a6d9d640 1 --2- 172.21.15.32:0/357432881 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0a8072cf0 0x7ff0a81f2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:43.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.350+0000 7ff0a6d9d640 1 -- 172.21.15.32:0/357432881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0a81f2f90 con 0x7ff0a815c5b0 2024-04-24T23:04:43.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.350+0000 7ff0a5d9b640 1 --2- 172.21.15.32:0/357432881 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0a8072cf0 0x7ff0a81f2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:04:43.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.350+0000 7ff0a659c640 1 --2- 172.21.15.32:0/357432881 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff0a8071770 0x7ff0a815b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:04:43.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.350+0000 7ff0a6d9d640 1 --2- 172.21.15.32:0/357432881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0a815c5b0 0x7ff0a813dbd0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7ff09800c7c0 tx=0x7ff09800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:43.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.350+0000 7ff096ffd640 1 -- 172.21.15.32:0/357432881 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff098018070 con 0x7ff0a815c5b0 2024-04-24T23:04:43.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.350+0000 7ff096ffd640 1 -- 172.21.15.32:0/357432881 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff09800a5e0 con 0x7ff0a815c5b0 2024-04-24T23:04:43.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.350+0000 7ff0acfe7640 1 -- 172.21.15.32:0/357432881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0a815a500 con 0x7ff0a815c5b0 2024-04-24T23:04:43.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.350+0000 7ff096ffd640 1 -- 172.21.15.32:0/357432881 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09800a9d0 con 0x7ff0a815c5b0 2024-04-24T23:04:43.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.350+0000 7ff0acfe7640 1 -- 172.21.15.32:0/357432881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0a815aac0 con 0x7ff0a815c5b0 2024-04-24T23:04:43.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.351+0000 7ff0acfe7640 1 -- 172.21.15.32:0/357432881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0a8068430 con 0x7ff0a815c5b0 2024-04-24T23:04:43.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.351+0000 7ff096ffd640 1 -- 172.21.15.32:0/357432881 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff09800ab70 con 0x7ff0a815c5b0 2024-04-24T23:04:43.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.352+0000 7ff096ffd640 1 --2- 172.21.15.32:0/357432881 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff08806be60 0x7ff08806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:43.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.352+0000 7ff096ffd640 1 -- 172.21.15.32:0/357432881 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7ff09808a2f0 con 0x7ff0a815c5b0 2024-04-24T23:04:43.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.352+0000 7ff0a659c640 1 --2- 172.21.15.32:0/357432881 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff08806be60 0x7ff08806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:43.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.353+0000 7ff0a659c640 1 --2- 172.21.15.32:0/357432881 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff08806be60 0x7ff08806e320 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0034000 tx=0x7ff0a002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:43.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.357+0000 7ff096ffd640 1 -- 172.21.15.32:0/357432881 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff0a815c990 con 0x7ff0a815c5b0 2024-04-24T23:04:43.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.577+0000 7ff0acfe7640 1 -- 172.21.15.32:0/357432881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff0a80b5020 con 0x7ff0a815c5b0 2024-04-24T23:04:43.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.578+0000 7ff096ffd640 1 -- 172.21.15.32:0/357432881 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5186 (secure 0 0 0) 0x7ff098058cb0 con 0x7ff0a815c5b0 2024-04-24T23:04:43.580 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:04:43.580 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":44,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6175,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/434273738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":434273738},{"type":"v1","addr":"172.21.15.121:6836","nonce":434273738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6178,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1104557785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1104557785},{"type":"v1","addr":"172.21.15.121:6837","nonce":1104557785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6888,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/3137388886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3137388886},{"type":"v1","addr":"172.21.15.32:6834","nonce":3137388886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:04:00.783579+0000","modified":"2024-04-24T23:04:03.790658+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6891},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6891":{"gid":6891,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/3327203898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":3327203898},{"type":"v1","addr":"172.21.15.32:6835","nonce":3327203898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6891,"qdb_cluster":[6891]},"id":6}]} 2024-04-24T23:04:43.580 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 44 2024-04-24T23:04:43.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.579+0000 7ff0acfe7640 1 -- 172.21.15.32:0/357432881 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff08806be60 msgr2=0x7ff08806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:43.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.579+0000 7ff0acfe7640 1 --2- 172.21.15.32:0/357432881 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff08806be60 0x7ff08806e320 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0034000 tx=0x7ff0a002e000 comp rx=0 tx=0).stop 2024-04-24T23:04:43.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.579+0000 7ff0acfe7640 1 -- 172.21.15.32:0/357432881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0a815c5b0 msgr2=0x7ff0a813dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:43.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.579+0000 7ff0acfe7640 1 --2- 172.21.15.32:0/357432881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0a815c5b0 0x7ff0a813dbd0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7ff09800c7c0 tx=0x7ff09800cc80 comp rx=0 tx=0).stop 2024-04-24T23:04:43.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.579+0000 7ff0acfe7640 1 -- 172.21.15.32:0/357432881 shutdown_connections 2024-04-24T23:04:43.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.579+0000 7ff0acfe7640 1 --2- 172.21.15.32:0/357432881 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff08806be60 0x7ff08806e320 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:43.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.579+0000 7ff0acfe7640 1 --2- 172.21.15.32:0/357432881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0a815c5b0 0x7ff0a813dbd0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:43.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.579+0000 7ff0acfe7640 1 --2- 172.21.15.32:0/357432881 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0a8072cf0 0x7ff0a81f2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:43.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.579+0000 7ff0acfe7640 1 --2- 172.21.15.32:0/357432881 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff0a8071770 0x7ff0a815b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:43.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.579+0000 7ff0acfe7640 1 -- 172.21.15.32:0/357432881 >> 172.21.15.32:0/357432881 conn(0x7ff0a806b110 msgr2=0x7ff0a806c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:43.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.580+0000 7ff0acfe7640 1 -- 172.21.15.32:0/357432881 shutdown_connections 2024-04-24T23:04:43.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.580+0000 7ff0acfe7640 1 -- 172.21.15.32:0/357432881 wait complete. 2024-04-24T23:04:43.593 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:04:43.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.693+0000 7f3502eb3640 1 Processor -- start 2024-04-24T23:04:43.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.693+0000 7f3502eb3640 1 -- start start 2024-04-24T23:04:43.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.693+0000 7f3502eb3640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f34fc15b480 0x7f34fc0b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:43.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.693+0000 7f3502eb3640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f34fc06cd00 0x7f34fc0cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:43.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.693+0000 7f3502eb3640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34fc0b46c0 con 0x7f34fc15fd60 2024-04-24T23:04:43.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.693+0000 7f3502eb3640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f34fc0b4ff0 con 0x7f34fc06cd00 2024-04-24T23:04:43.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.693+0000 7f3502eb3640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f34fc0b49a0 con 0x7f34fc15b480 2024-04-24T23:04:43.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.693+0000 7f3501450640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f34fc06cd00 0x7f34fc0cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:43.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.693+0000 7f3501450640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f34fc06cd00 0x7f34fc0cdf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47036/0 (socket says 172.21.15.32:47036) 2024-04-24T23:04:43.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.693+0000 7f3501450640 1 -- 172.21.15.32:0/1075691679 learned_addr learned my addr 172.21.15.32:0/1075691679 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:43.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.693+0000 7f3500c4f640 1 --2- 172.21.15.32:0/1075691679 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f34fc15b480 0x7f34fc0b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:43.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.693+0000 7f3501450640 1 -- 172.21.15.32:0/1075691679 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f34fc15b480 msgr2=0x7f34fc0b5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:43.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.693+0000 7f3501450640 1 --2- 172.21.15.32:0/1075691679 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f34fc15b480 0x7f34fc0b5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:43.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.693+0000 7f3501450640 1 -- 172.21.15.32:0/1075691679 >> v1:172.21.15.121:6789/0 conn(0x7f34fc15fd60 legacy=0x7f34fc06c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:43.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.693+0000 7f3501450640 1 -- 172.21.15.32:0/1075691679 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34fc0b4c80 con 0x7f34fc06cd00 2024-04-24T23:04:43.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.694+0000 7f3501450640 1 --2- 172.21.15.32:0/1075691679 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34fc06cd00 0x7f34fc0cdf80 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f34f400d150 tx=0x7f34f402c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb6c06de14c0f752 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:43.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.694+0000 7f34fb7fe640 1 -- 172.21.15.32:0/1075691679 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f4009600 con 0x7f34fc06cd00 2024-04-24T23:04:43.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.694+0000 7f34fb7fe640 1 -- 172.21.15.32:0/1075691679 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34f4037070 con 0x7f34fc06cd00 2024-04-24T23:04:43.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.694+0000 7f34fb7fe640 1 -- 172.21.15.32:0/1075691679 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f40099c0 con 0x7f34fc06cd00 2024-04-24T23:04:43.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.694+0000 7f3502eb3640 1 -- 172.21.15.32:0/1075691679 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34fc06cd00 msgr2=0x7f34fc0cdf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:43.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.694+0000 7f3502eb3640 1 --2- 172.21.15.32:0/1075691679 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34fc06cd00 0x7f34fc0cdf80 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f34f400d150 tx=0x7f34f402c930 comp rx=0 tx=0).stop 2024-04-24T23:04:43.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.695+0000 7f3502eb3640 1 -- 172.21.15.32:0/1075691679 shutdown_connections 2024-04-24T23:04:43.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.695+0000 7f3502eb3640 1 --2- 172.21.15.32:0/1075691679 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34fc06cd00 0x7f34fc0cdf80 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:43.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.695+0000 7f3502eb3640 1 --2- 172.21.15.32:0/1075691679 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f34fc15b480 0x7f34fc0b5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:43.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.695+0000 7f3502eb3640 1 -- 172.21.15.32:0/1075691679 >> 172.21.15.32:0/1075691679 conn(0x7f34fc06b110 msgr2=0x7f34fc06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:43.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.695+0000 7f3502eb3640 1 -- 172.21.15.32:0/1075691679 shutdown_connections 2024-04-24T23:04:43.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.695+0000 7f3502eb3640 1 -- 172.21.15.32:0/1075691679 wait complete. 2024-04-24T23:04:43.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.696+0000 7f3502eb3640 1 Processor -- start 2024-04-24T23:04:43.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.696+0000 7f3502eb3640 1 -- start start 2024-04-24T23:04:43.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.697+0000 7f3502eb3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34fc06cd00 0x7f34fc1403a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:43.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.697+0000 7f3502eb3640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f34fc15b480 0x7f34fc159f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:43.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.697+0000 7f3502eb3640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f34fc15fd60 0x7f34fc0721f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:43.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.697+0000 7f3502eb3640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f34fc15b860 con 0x7f34fc06cd00 2024-04-24T23:04:43.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.697+0000 7f3502eb3640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f34fc0b5480 con 0x7f34fc15fd60 2024-04-24T23:04:43.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.697+0000 7f3502eb3640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f34fc158e60 con 0x7f34fc15b480 2024-04-24T23:04:43.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.697+0000 7f3500c4f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34fc06cd00 0x7f34fc1403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:43.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.697+0000 7f3500c4f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34fc06cd00 0x7f34fc1403a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47044/0 (socket says 172.21.15.32:47044) 2024-04-24T23:04:43.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.697+0000 7f3500c4f640 1 -- 172.21.15.32:0/342908744 learned_addr learned my addr 172.21.15.32:0/342908744 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:43.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.697+0000 7f3501450640 1 --2- 172.21.15.32:0/342908744 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f34fc15fd60 0x7f34fc0721f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:43.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.697+0000 7f34fbfff640 1 --2- 172.21.15.32:0/342908744 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f34fc15b480 0x7f34fc159f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:43.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.697+0000 7f3500c4f640 1 -- 172.21.15.32:0/342908744 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f34fc15b480 msgr2=0x7f34fc159f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:43.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.697+0000 7f3500c4f640 1 --2- 172.21.15.32:0/342908744 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f34fc15b480 0x7f34fc159f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:43.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.697+0000 7f3500c4f640 1 -- 172.21.15.32:0/342908744 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f34fc15fd60 msgr2=0x7f34fc0721f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:43.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.697+0000 7f3500c4f640 1 --2- 172.21.15.32:0/342908744 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f34fc15fd60 0x7f34fc0721f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:43.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.697+0000 7f3500c4f640 1 -- 172.21.15.32:0/342908744 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34fc159140 con 0x7f34fc06cd00 2024-04-24T23:04:43.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.697+0000 7f34fbfff640 1 --2- 172.21.15.32:0/342908744 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f34fc15b480 0x7f34fc159f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:04:43.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.697+0000 7f3501450640 1 --2- 172.21.15.32:0/342908744 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f34fc15fd60 0x7f34fc0721f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:04:43.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.697+0000 7f3500c4f640 1 --2- 172.21.15.32:0/342908744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34fc06cd00 0x7f34fc1403a0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f34f000d800 tx=0x7f34f000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:43.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.697+0000 7f34f97fa640 1 -- 172.21.15.32:0/342908744 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f000b6c0 con 0x7f34fc06cd00 2024-04-24T23:04:43.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.697+0000 7f34f97fa640 1 -- 172.21.15.32:0/342908744 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34f0019070 con 0x7f34fc06cd00 2024-04-24T23:04:43.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.697+0000 7f3502eb3640 1 -- 172.21.15.32:0/342908744 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34fc159420 con 0x7f34fc06cd00 2024-04-24T23:04:43.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.697+0000 7f34f97fa640 1 -- 172.21.15.32:0/342908744 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f000ba80 con 0x7f34fc06cd00 2024-04-24T23:04:43.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.697+0000 7f3502eb3640 1 -- 172.21.15.32:0/342908744 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34fc1599e0 con 0x7f34fc06cd00 2024-04-24T23:04:43.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.699+0000 7f3502eb3640 1 -- 172.21.15.32:0/342908744 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34fc068430 con 0x7f34fc06cd00 2024-04-24T23:04:43.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.699+0000 7f34f97fa640 1 -- 172.21.15.32:0/342908744 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f34f000bc20 con 0x7f34fc06cd00 2024-04-24T23:04:43.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.699+0000 7f34f97fa640 1 --2- 172.21.15.32:0/342908744 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f34e006be30 0x7f34e006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:43.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.699+0000 7f34f97fa640 1 -- 172.21.15.32:0/342908744 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7f34f008b310 con 0x7f34fc06cd00 2024-04-24T23:04:43.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.700+0000 7f34fbfff640 1 --2- 172.21.15.32:0/342908744 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f34e006be30 0x7f34e006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:43.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.700+0000 7f34fbfff640 1 --2- 172.21.15.32:0/342908744 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f34e006be30 0x7f34e006e2f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f34fc0b0340 tx=0x7f34e8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:43.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.704+0000 7f34f97fa640 1 -- 172.21.15.32:0/342908744 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f34f0059cc0 con 0x7f34fc06cd00 2024-04-24T23:04:43.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.886+0000 7f3502eb3640 1 -- 172.21.15.32:0/342908744 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f34fc06c000 con 0x7f34fc06cd00 2024-04-24T23:04:43.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.887+0000 7f34f97fa640 1 -- 172.21.15.32:0/342908744 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) ==== 74+0+16696 (secure 0 0 0) 0x7f34f00595c0 con 0x7f34fc06cd00 2024-04-24T23:04:43.890 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:04:43.890 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":113,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:04:31.618095+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:03:59.079157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-24T23:03:59.765179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:04:43.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.889+0000 7f3502eb3640 1 -- 172.21.15.32:0/342908744 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f34e006be30 msgr2=0x7f34e006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:43.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.889+0000 7f3502eb3640 1 --2- 172.21.15.32:0/342908744 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f34e006be30 0x7f34e006e2f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f34fc0b0340 tx=0x7f34e8009040 comp rx=0 tx=0).stop 2024-04-24T23:04:43.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.889+0000 7f3502eb3640 1 -- 172.21.15.32:0/342908744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34fc06cd00 msgr2=0x7f34fc1403a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:43.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.889+0000 7f3502eb3640 1 --2- 172.21.15.32:0/342908744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34fc06cd00 0x7f34fc1403a0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f34f000d800 tx=0x7f34f000dcc0 comp rx=0 tx=0).stop 2024-04-24T23:04:43.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.889+0000 7f3502eb3640 1 -- 172.21.15.32:0/342908744 shutdown_connections 2024-04-24T23:04:43.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.889+0000 7f3502eb3640 1 --2- 172.21.15.32:0/342908744 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f34e006be30 0x7f34e006e2f0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:43.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.889+0000 7f3502eb3640 1 --2- 172.21.15.32:0/342908744 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f34fc15fd60 0x7f34fc0721f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:43.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.889+0000 7f3502eb3640 1 --2- 172.21.15.32:0/342908744 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f34fc15b480 0x7f34fc159f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:43.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.889+0000 7f3502eb3640 1 --2- 172.21.15.32:0/342908744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34fc06cd00 0x7f34fc1403a0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:43.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.889+0000 7f3502eb3640 1 -- 172.21.15.32:0/342908744 >> 172.21.15.32:0/342908744 conn(0x7f34fc06b110 msgr2=0x7f34fc0b04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:43.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.889+0000 7f3502eb3640 1 -- 172.21.15.32:0/342908744 shutdown_connections 2024-04-24T23:04:43.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:43.889+0000 7f3502eb3640 1 -- 172.21.15.32:0/342908744 wait complete. 2024-04-24T23:04:43.903 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-24T23:04:44.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.003+0000 7f55aad42640 1 Processor -- start 2024-04-24T23:04:44.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.003+0000 7f55aad42640 1 -- start start 2024-04-24T23:04:44.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.004+0000 7f55aad42640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f55a406cd00 0x7f55a4166440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:44.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.004+0000 7f55aad42640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55a40cef90 con 0x7f55a415fd60 2024-04-24T23:04:44.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.004+0000 7f55aad42640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55a40b46c0 con 0x7f55a415b480 2024-04-24T23:04:44.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.004+0000 7f55aad42640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f55a40b4ff0 con 0x7f55a406cd00 2024-04-24T23:04:44.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.004+0000 7f55a3fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f55a415fd60 0x7f55a406c5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60824/0 (socket says 172.21.15.32:60824) 2024-04-24T23:04:44.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.004+0000 7f55a3fff640 1 -- 172.21.15.32:0/4005130964 learned_addr learned my addr 172.21.15.32:0/4005130964 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:04:44.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.004+0000 7f55a37fe640 1 -- 172.21.15.32:0/4005130964 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 301413110 0 0) 0x7f55a40cef90 con 0x7f55a415fd60 2024-04-24T23:04:44.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.004+0000 7f55a92df640 1 --2- 172.21.15.32:0/4005130964 >> v2:172.21.15.121:3300/0 conn(0x7f55a406cd00 0x7f55a4166440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:44.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.004+0000 7f55a37fe640 1 -- 172.21.15.32:0/4005130964 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5594005730 con 0x7f55a415fd60 2024-04-24T23:04:44.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.004+0000 7f55a37fe640 1 -- 172.21.15.32:0/4005130964 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4106225907 0 0) 0x7f5594005730 con 0x7f55a415fd60 2024-04-24T23:04:44.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.004+0000 7f55a37fe640 1 -- 172.21.15.32:0/4005130964 >> v2:172.21.15.121:3300/0 conn(0x7f55a406cd00 msgr2=0x7f55a4166440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:44.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.004+0000 7f55a37fe640 1 --2- 172.21.15.32:0/4005130964 >> v2:172.21.15.121:3300/0 conn(0x7f55a406cd00 0x7f55a4166440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:44.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.004+0000 7f55a37fe640 1 -- 172.21.15.32:0/4005130964 >> v1:172.21.15.121:6789/0 conn(0x7f55a415b480 legacy=0x7f55a40b5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:44.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.004+0000 7f55a37fe640 1 -- 172.21.15.32:0/4005130964 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55a40b49a0 con 0x7f55a415fd60 2024-04-24T23:04:44.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.004+0000 7f55a37fe640 1 -- 172.21.15.32:0/4005130964 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f5590001670 con 0x7f55a415fd60 2024-04-24T23:04:44.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.004+0000 7f55a37fe640 1 -- 172.21.15.32:0/4005130964 >> v1:172.21.15.32:6789/0 conn(0x7f55a415fd60 legacy=0x7f55a406c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:44.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.004+0000 7f55a37fe640 1 --2- 172.21.15.32:0/4005130964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5594005bc0 0x7f5594005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:44.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.004+0000 7f55a37fe640 1 --2- 172.21.15.32:0/4005130964 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5594006a10 0x7f5594006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:44.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.004+0000 7f55a37fe640 1 --2- 172.21.15.32:0/4005130964 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55940073d0 0x7f559400dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:44.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.004+0000 7f55a37fe640 1 -- 172.21.15.32:0/4005130964 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f55a40b4ff0 con 0x7f5594005bc0 2024-04-24T23:04:44.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.004+0000 7f55a37fe640 1 -- 172.21.15.32:0/4005130964 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f559400e270 con 0x7f5594006a10 2024-04-24T23:04:44.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.004+0000 7f55a37fe640 1 -- 172.21.15.32:0/4005130964 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f559400e470 con 0x7f55940073d0 2024-04-24T23:04:44.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.004+0000 7f55a8ade640 1 --2- 172.21.15.32:0/4005130964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5594005bc0 0x7f5594005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:44.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.004+0000 7f55a3fff640 1 --2- 172.21.15.32:0/4005130964 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5594006a10 0x7f5594006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:44.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.004+0000 7f55a92df640 1 --2- 172.21.15.32:0/4005130964 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55940073d0 0x7f559400dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:44.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.004+0000 7f55a8ade640 1 -- 172.21.15.32:0/4005130964 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55940073d0 msgr2=0x7f559400dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:44.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.004+0000 7f55a8ade640 1 --2- 172.21.15.32:0/4005130964 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55940073d0 0x7f559400dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:44.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.004+0000 7f55a8ade640 1 -- 172.21.15.32:0/4005130964 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5594006a10 msgr2=0x7f5594006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:44.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.004+0000 7f55a8ade640 1 --2- 172.21.15.32:0/4005130964 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5594006a10 0x7f5594006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:44.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.004+0000 7f55a8ade640 1 -- 172.21.15.32:0/4005130964 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5594005730 con 0x7f5594005bc0 2024-04-24T23:04:44.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.004+0000 7f55a92df640 1 -- 172.21.15.32:0/4005130964 reap_dead start 2024-04-24T23:04:44.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.005+0000 7f55a8ade640 1 --2- 172.21.15.32:0/4005130964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5594005bc0 0x7f5594005fe0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f5598001750 tx=0x7f5594007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a3f52be9da71bcb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:44.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.005+0000 7f55a37fe640 1 -- 172.21.15.32:0/4005130964 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5598001370 con 0x7f5594005bc0 2024-04-24T23:04:44.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.005+0000 7f55a37fe640 1 -- 172.21.15.32:0/4005130964 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55980052d0 con 0x7f5594005bc0 2024-04-24T23:04:44.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.005+0000 7f55a37fe640 1 -- 172.21.15.32:0/4005130964 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559802abc0 con 0x7f5594005bc0 2024-04-24T23:04:44.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.005+0000 7f55aad42640 1 -- 172.21.15.32:0/4005130964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5594005bc0 msgr2=0x7f5594005fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:44.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.005+0000 7f55aad42640 1 --2- 172.21.15.32:0/4005130964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5594005bc0 0x7f5594005fe0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f5598001750 tx=0x7f5594007a40 comp rx=0 tx=0).stop 2024-04-24T23:04:44.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.005+0000 7f55aad42640 1 -- 172.21.15.32:0/4005130964 shutdown_connections 2024-04-24T23:04:44.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.005+0000 7f55aad42640 1 --2- 172.21.15.32:0/4005130964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5594005bc0 0x7f5594005fe0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:44.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.005+0000 7f55aad42640 1 -- 172.21.15.32:0/4005130964 >> 172.21.15.32:0/4005130964 conn(0x7f55a406b110 msgr2=0x7f55a406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:44.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.005+0000 7f55aad42640 1 -- 172.21.15.32:0/4005130964 shutdown_connections 2024-04-24T23:04:44.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.005+0000 7f55aad42640 1 -- 172.21.15.32:0/4005130964 wait complete. 2024-04-24T23:04:44.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.008+0000 7f55aad42640 1 Processor -- start 2024-04-24T23:04:44.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.008+0000 7f55aad42640 1 -- start start 2024-04-24T23:04:44.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.008+0000 7f55aad42640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5594005bc0 0x7f55a415da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:44.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.008+0000 7f55aad42640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55a41f3c70 0x7f55a4072400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:44.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.008+0000 7f55aad42640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55a415bb70 0x7f55a40b3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:44.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.008+0000 7f55aad42640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f55a415b860 con 0x7f55a415bb70 2024-04-24T23:04:44.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.008+0000 7f55aad42640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f55a40b5480 con 0x7f5594005bc0 2024-04-24T23:04:44.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.008+0000 7f55aad42640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f55a4070820 con 0x7f55a41f3c70 2024-04-24T23:04:44.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.008+0000 7f55a92df640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55a415bb70 0x7f55a40b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:44.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.008+0000 7f55a8ade640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5594005bc0 0x7f55a415da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:44.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.008+0000 7f55a92df640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55a415bb70 0x7f55a40b3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47050/0 (socket says 172.21.15.32:47050) 2024-04-24T23:04:44.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.008+0000 7f55a92df640 1 -- 172.21.15.32:0/652359512 learned_addr learned my addr 172.21.15.32:0/652359512 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:44.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.008+0000 7f55a3fff640 1 --2- 172.21.15.32:0/652359512 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55a41f3c70 0x7f55a4072400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:44.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.008+0000 7f55a92df640 1 -- 172.21.15.32:0/652359512 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55a41f3c70 msgr2=0x7f55a4072400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:44.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.008+0000 7f55a92df640 1 --2- 172.21.15.32:0/652359512 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55a41f3c70 0x7f55a4072400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:44.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.008+0000 7f55a92df640 1 -- 172.21.15.32:0/652359512 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5594005bc0 msgr2=0x7f55a415da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:44.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.008+0000 7f55a92df640 1 --2- 172.21.15.32:0/652359512 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5594005bc0 0x7f55a415da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:44.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.008+0000 7f55a92df640 1 -- 172.21.15.32:0/652359512 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55a4072940 con 0x7f55a415bb70 2024-04-24T23:04:44.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.008+0000 7f55a92df640 1 --2- 172.21.15.32:0/652359512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55a415bb70 0x7f55a40b3290 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f55a406e880 tx=0x7f559c0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:44.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.008+0000 7f55a17fa640 1 -- 172.21.15.32:0/652359512 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559c011e40 con 0x7f55a415bb70 2024-04-24T23:04:44.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.008+0000 7f55a17fa640 1 -- 172.21.15.32:0/652359512 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f559c01b070 con 0x7f55a415bb70 2024-04-24T23:04:44.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.008+0000 7f55aad42640 1 -- 172.21.15.32:0/652359512 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55a41f41f0 con 0x7f55a415bb70 2024-04-24T23:04:44.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.008+0000 7f55a17fa640 1 -- 172.21.15.32:0/652359512 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559c003990 con 0x7f55a415bb70 2024-04-24T23:04:44.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.008+0000 7f55aad42640 1 -- 172.21.15.32:0/652359512 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55a41f47b0 con 0x7f55a415bb70 2024-04-24T23:04:44.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.009+0000 7f55aad42640 1 -- 172.21.15.32:0/652359512 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55a4068430 con 0x7f55a415bb70 2024-04-24T23:04:44.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.010+0000 7f55a17fa640 1 -- 172.21.15.32:0/652359512 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f559c005d10 con 0x7f55a415bb70 2024-04-24T23:04:44.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.010+0000 7f55a17fa640 1 --2- 172.21.15.32:0/652359512 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f558806be30 0x7f558806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:44.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.010+0000 7f55a17fa640 1 -- 172.21.15.32:0/652359512 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7f559c08b050 con 0x7f55a415bb70 2024-04-24T23:04:44.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.011+0000 7f55a8ade640 1 --2- 172.21.15.32:0/652359512 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f558806be30 0x7f558806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:44.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.011+0000 7f55a8ade640 1 --2- 172.21.15.32:0/652359512 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f558806be30 0x7f558806e2f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f5598032000 tx=0x7f559802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:44.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.015+0000 7f55a17fa640 1 -- 172.21.15.32:0/652359512 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f559c0593b0 con 0x7f55a415bb70 2024-04-24T23:04:44.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.241+0000 7f55aad42640 1 -- 172.21.15.32:0/652359512 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f55a406c000 con 0x7f55a415bb70 2024-04-24T23:04:44.243 INFO:tasks.ceph.mon.a.smithi032.stderr:2024-04-24T23:04:44.242+0000 7f3b809c6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T23:04:44.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.243+0000 7f55a17fa640 1 -- 172.21.15.32:0/652359512 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v45) ==== 161+0+0 (secure 0 0 0) 0x7f559c05d6b0 con 0x7f55a415bb70 2024-04-24T23:04:44.244 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-24T23:04:44.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.247+0000 7f55aad42640 1 -- 172.21.15.32:0/652359512 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f558806be30 msgr2=0x7f558806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:44.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.247+0000 7f55aad42640 1 --2- 172.21.15.32:0/652359512 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f558806be30 0x7f558806e2f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f5598032000 tx=0x7f559802e000 comp rx=0 tx=0).stop 2024-04-24T23:04:44.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.247+0000 7f55aad42640 1 -- 172.21.15.32:0/652359512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55a415bb70 msgr2=0x7f55a40b3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:44.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.247+0000 7f55aad42640 1 --2- 172.21.15.32:0/652359512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55a415bb70 0x7f55a40b3290 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f55a406e880 tx=0x7f559c0115a0 comp rx=0 tx=0).stop 2024-04-24T23:04:44.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.247+0000 7f55aad42640 1 -- 172.21.15.32:0/652359512 shutdown_connections 2024-04-24T23:04:44.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.247+0000 7f55aad42640 1 --2- 172.21.15.32:0/652359512 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f558806be30 0x7f558806e2f0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:44.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.247+0000 7f55aad42640 1 --2- 172.21.15.32:0/652359512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55a415bb70 0x7f55a40b3290 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:44.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.247+0000 7f55aad42640 1 --2- 172.21.15.32:0/652359512 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55a41f3c70 0x7f55a4072400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:44.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.247+0000 7f55aad42640 1 --2- 172.21.15.32:0/652359512 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5594005bc0 0x7f55a415da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:44.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.247+0000 7f55aad42640 1 -- 172.21.15.32:0/652359512 >> 172.21.15.32:0/652359512 conn(0x7f55a406b110 msgr2=0x7f55a40aff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:44.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.247+0000 7f55aad42640 1 -- 172.21.15.32:0/652359512 shutdown_connections 2024-04-24T23:04:44.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.248+0000 7f55aad42640 1 -- 172.21.15.32:0/652359512 wait complete. 2024-04-24T23:04:44.261 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-24T23:04:44.285 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2024-04-24T23:04:44.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.364+0000 7f7db4f09640 1 Processor -- start 2024-04-24T23:04:44.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.364+0000 7f7db4f09640 1 -- start start 2024-04-24T23:04:44.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.364+0000 7f7db4f09640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7db006cd00 0x7f7db00ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:44.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.365+0000 7f7db4f09640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7db00b4520 con 0x7f7db015cb00 2024-04-24T23:04:44.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.365+0000 7f7db4f09640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7db00b4800 con 0x7f7db015b380 2024-04-24T23:04:44.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.365+0000 7f7db4f09640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f7db00b4ae0 con 0x7f7db006cd00 2024-04-24T23:04:44.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.365+0000 7f7da7fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7db006cd00 0x7f7db00ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:44.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.365+0000 7f7daed9d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7db015cb00 0x7f7db00ce970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60828/0 (socket says 172.21.15.32:60828) 2024-04-24T23:04:44.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.365+0000 7f7da7fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7db006cd00 0x7f7db00ce430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47076/0 (socket says 172.21.15.32:47076) 2024-04-24T23:04:44.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.365+0000 7f7daed9d640 1 -- 172.21.15.32:0/1808220410 learned_addr learned my addr 172.21.15.32:0/1808220410 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:04:44.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.365+0000 7f7da7fff640 1 -- 172.21.15.32:0/1808220410 >> v1:172.21.15.121:6789/0 conn(0x7f7db015b380 legacy=0x7f7db006c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:44.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.365+0000 7f7da7fff640 1 -- 172.21.15.32:0/1808220410 >> v1:172.21.15.32:6789/0 conn(0x7f7db015cb00 legacy=0x7f7db00ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:44.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.365+0000 7f7da7fff640 1 -- 172.21.15.32:0/1808220410 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7db00b4dc0 con 0x7f7db006cd00 2024-04-24T23:04:44.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.365+0000 7f7dadd9b640 1 -- 172.21.15.32:0/1808220410 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2929203560 0 0) 0x7f7db00b4520 con 0x7f7db015cb00 2024-04-24T23:04:44.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.366+0000 7f7da7fff640 1 --2- 172.21.15.32:0/1808220410 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7db006cd00 0x7f7db00ce430 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f7d980097a0 tx=0x7f7d9802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b00e12b217a53f3b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:44.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.366+0000 7f7dadd9b640 1 -- 172.21.15.32:0/1808220410 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d98037070 con 0x7f7db006cd00 2024-04-24T23:04:44.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.366+0000 7f7dadd9b640 1 -- 172.21.15.32:0/1808220410 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d98032440 con 0x7f7db006cd00 2024-04-24T23:04:44.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.366+0000 7f7dadd9b640 1 -- 172.21.15.32:0/1808220410 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d98032810 con 0x7f7db006cd00 2024-04-24T23:04:44.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.366+0000 7f7db4f09640 1 -- 172.21.15.32:0/1808220410 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7db006cd00 msgr2=0x7f7db00ce430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:44.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.366+0000 7f7db4f09640 1 --2- 172.21.15.32:0/1808220410 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7db006cd00 0x7f7db00ce430 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f7d980097a0 tx=0x7f7d9802a1a0 comp rx=0 tx=0).stop 2024-04-24T23:04:44.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.366+0000 7f7db4f09640 1 -- 172.21.15.32:0/1808220410 shutdown_connections 2024-04-24T23:04:44.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.366+0000 7f7db4f09640 1 --2- 172.21.15.32:0/1808220410 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7db006cd00 0x7f7db00ce430 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:44.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.366+0000 7f7db4f09640 1 -- 172.21.15.32:0/1808220410 >> 172.21.15.32:0/1808220410 conn(0x7f7db006b110 msgr2=0x7f7db006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:44.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.366+0000 7f7db4f09640 1 -- 172.21.15.32:0/1808220410 shutdown_connections 2024-04-24T23:04:44.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.367+0000 7f7db4f09640 1 -- 172.21.15.32:0/1808220410 wait complete. 2024-04-24T23:04:44.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.368+0000 7f7db4f09640 1 Processor -- start 2024-04-24T23:04:44.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.368+0000 7f7db4f09640 1 -- start start 2024-04-24T23:04:44.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.369+0000 7f7db4f09640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7db006cd00 0x7f7db0142c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:44.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.369+0000 7f7db4f09640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7db015b380 0x7f7db00b3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:44.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.369+0000 7f7db4f09640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7db015cb00 0x7f7db00b3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:44.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.369+0000 7f7db4f09640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7db00b55c0 con 0x7f7db015b380 2024-04-24T23:04:44.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.369+0000 7f7db4f09640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7db015b760 con 0x7f7db006cd00 2024-04-24T23:04:44.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.369+0000 7f7db4f09640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f7db00b5740 con 0x7f7db015cb00 2024-04-24T23:04:44.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.369+0000 7f7da7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7db015b380 0x7f7db00b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:44.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.369+0000 7f7da7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7db015b380 0x7f7db00b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47084/0 (socket says 172.21.15.32:47084) 2024-04-24T23:04:44.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.369+0000 7f7da7fff640 1 -- 172.21.15.32:0/2070288387 learned_addr learned my addr 172.21.15.32:0/2070288387 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:44.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.369+0000 7f7dae59c640 1 --2- 172.21.15.32:0/2070288387 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7db006cd00 0x7f7db0142c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:44.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.369+0000 7f7daed9d640 1 --2- 172.21.15.32:0/2070288387 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7db015cb00 0x7f7db00b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:44.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.369+0000 7f7da7fff640 1 -- 172.21.15.32:0/2070288387 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7db015cb00 msgr2=0x7f7db00b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:44.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.369+0000 7f7da7fff640 1 --2- 172.21.15.32:0/2070288387 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7db015cb00 0x7f7db00b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:44.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.369+0000 7f7da7fff640 1 -- 172.21.15.32:0/2070288387 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7db006cd00 msgr2=0x7f7db0142c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:44.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.369+0000 7f7da7fff640 1 --2- 172.21.15.32:0/2070288387 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7db006cd00 0x7f7db0142c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:44.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.369+0000 7f7da7fff640 1 -- 172.21.15.32:0/2070288387 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7db0152e80 con 0x7f7db015b380 2024-04-24T23:04:44.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.369+0000 7f7da7fff640 1 --2- 172.21.15.32:0/2070288387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7db015b380 0x7f7db00b3410 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f7d980097a0 tx=0x7f7d980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:44.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.369+0000 7f7da6ffd640 1 -- 172.21.15.32:0/2070288387 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d98037050 con 0x7f7db015b380 2024-04-24T23:04:44.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.369+0000 7f7da6ffd640 1 -- 172.21.15.32:0/2070288387 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d98038040 con 0x7f7db015b380 2024-04-24T23:04:44.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.369+0000 7f7da6ffd640 1 -- 172.21.15.32:0/2070288387 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d98004490 con 0x7f7db015b380 2024-04-24T23:04:44.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.369+0000 7f7db4f09640 1 -- 172.21.15.32:0/2070288387 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7db01528c0 con 0x7f7db015b380 2024-04-24T23:04:44.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.370+0000 7f7db4f09640 1 -- 172.21.15.32:0/2070288387 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7db0154930 con 0x7f7db015b380 2024-04-24T23:04:44.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.371+0000 7f7db4f09640 1 -- 172.21.15.32:0/2070288387 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7db0068430 con 0x7f7db015b380 2024-04-24T23:04:44.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.371+0000 7f7da6ffd640 1 -- 172.21.15.32:0/2070288387 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7d98030900 con 0x7f7db015b380 2024-04-24T23:04:44.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.371+0000 7f7da6ffd640 1 --2- 172.21.15.32:0/2070288387 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7d9006be30 0x7f7d9006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:44.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.372+0000 7f7da6ffd640 1 -- 172.21.15.32:0/2070288387 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5795+0+0 (secure 0 0 0) 0x7f7d980ab9e0 con 0x7f7db015b380 2024-04-24T23:04:44.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.372+0000 7f7dae59c640 1 --2- 172.21.15.32:0/2070288387 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7d9006be30 0x7f7d9006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:44.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.372+0000 7f7dae59c640 1 --2- 172.21.15.32:0/2070288387 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7d9006be30 0x7f7d9006e2f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f7da0000f60 tx=0x7f7da0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:44.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.376+0000 7f7da6ffd640 1 -- 172.21.15.32:0/2070288387 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7db0152e80 con 0x7f7db015b380 2024-04-24T23:04:44.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:44.603+0000 7f7db4f09640 1 -- 172.21.15.32:0/2070288387 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f7db00afd60 con 0x7f7db015b380 2024-04-24T23:04:45.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.244+0000 7f7da6ffd640 1 -- 172.21.15.32:0/2070288387 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v46) ==== 104+0+0 (secure 0 0 0) 0x7f7d98079e90 con 0x7f7db015b380 2024-04-24T23:04:45.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.247+0000 7f7db4f09640 1 -- 172.21.15.32:0/2070288387 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7d9006be30 msgr2=0x7f7d9006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:45.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.247+0000 7f7db4f09640 1 --2- 172.21.15.32:0/2070288387 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7d9006be30 0x7f7d9006e2f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f7da0000f60 tx=0x7f7da0009040 comp rx=0 tx=0).stop 2024-04-24T23:04:45.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.247+0000 7f7db4f09640 1 -- 172.21.15.32:0/2070288387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7db015b380 msgr2=0x7f7db00b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:45.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.247+0000 7f7db4f09640 1 --2- 172.21.15.32:0/2070288387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7db015b380 0x7f7db00b3410 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f7d980097a0 tx=0x7f7d980046c0 comp rx=0 tx=0).stop 2024-04-24T23:04:45.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.247+0000 7f7db4f09640 1 -- 172.21.15.32:0/2070288387 shutdown_connections 2024-04-24T23:04:45.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.247+0000 7f7db4f09640 1 --2- 172.21.15.32:0/2070288387 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7d9006be30 0x7f7d9006e2f0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:45.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.247+0000 7f7db4f09640 1 --2- 172.21.15.32:0/2070288387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7db015b380 0x7f7db00b3410 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:45.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.247+0000 7f7db4f09640 1 --2- 172.21.15.32:0/2070288387 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7db015cb00 0x7f7db00b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:45.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.247+0000 7f7db4f09640 1 --2- 172.21.15.32:0/2070288387 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7db006cd00 0x7f7db0142c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:45.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.247+0000 7f7db4f09640 1 -- 172.21.15.32:0/2070288387 >> 172.21.15.32:0/2070288387 conn(0x7f7db006b110 msgr2=0x7f7db006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:45.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.248+0000 7f7db4f09640 1 -- 172.21.15.32:0/2070288387 shutdown_connections 2024-04-24T23:04:45.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.248+0000 7f7db4f09640 1 -- 172.21.15.32:0/2070288387 wait complete. 2024-04-24T23:04:45.263 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-24T23:04:45.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.413+0000 7f7d7599e640 1 Processor -- start 2024-04-24T23:04:45.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.413+0000 7f7d7599e640 1 -- start start 2024-04-24T23:04:45.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.414+0000 7f7d7599e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d700b3910 0x7f7d700d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:45.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.414+0000 7f7d7599e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d700b3cf0 con 0x7f7d700cfcb0 2024-04-24T23:04:45.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.414+0000 7f7d7599e640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d7006c000 con 0x7f7d700b6080 2024-04-24T23:04:45.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.414+0000 7f7d7599e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f7d700d2ae0 con 0x7f7d700b3910 2024-04-24T23:04:45.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.414+0000 7f7d7519d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7d700cfcb0 0x7f7d700b7b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60834/0 (socket says 172.21.15.32:60834) 2024-04-24T23:04:45.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.414+0000 7f7d7519d640 1 -- 172.21.15.32:0/4037844087 learned_addr learned my addr 172.21.15.32:0/4037844087 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:04:45.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.414+0000 7f7d7499c640 1 --2- 172.21.15.32:0/4037844087 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d700b3910 0x7f7d700d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:45.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.414+0000 7f7d6f7fe640 1 -- 172.21.15.32:0/4037844087 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 193070926 0 0) 0x7f7d700b3cf0 con 0x7f7d700cfcb0 2024-04-24T23:04:45.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.414+0000 7f7d6f7fe640 1 -- 172.21.15.32:0/4037844087 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d60003520 con 0x7f7d700cfcb0 2024-04-24T23:04:45.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.414+0000 7f7d6f7fe640 1 -- 172.21.15.32:0/4037844087 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 897535693 0 0) 0x7f7d7006c000 con 0x7f7d700b6080 2024-04-24T23:04:45.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.414+0000 7f7d6f7fe640 1 -- 172.21.15.32:0/4037844087 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d700b3cf0 con 0x7f7d700b6080 2024-04-24T23:04:45.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.414+0000 7f7d6f7fe640 1 -- 172.21.15.32:0/4037844087 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1585303218 0 0) 0x7f7d60003520 con 0x7f7d700cfcb0 2024-04-24T23:04:45.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.414+0000 7f7d7499c640 1 -- 172.21.15.32:0/4037844087 >> v1:172.21.15.121:6789/0 conn(0x7f7d700b6080 legacy=0x7f7d700cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:45.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.414+0000 7f7d7499c640 1 -- 172.21.15.32:0/4037844087 >> v1:172.21.15.32:6789/0 conn(0x7f7d700cfcb0 legacy=0x7f7d700b7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:45.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.414+0000 7f7d7499c640 1 -- 172.21.15.32:0/4037844087 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d700d2dc0 con 0x7f7d700b3910 2024-04-24T23:04:45.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.415+0000 7f7d7499c640 1 --2- 172.21.15.32:0/4037844087 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d700b3910 0x7f7d700d38a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7d640097d0 tx=0x7f7d6402a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be6318d0be5aefe5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:45.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.415+0000 7f7d6f7fe640 1 -- 172.21.15.32:0/4037844087 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d64037070 con 0x7f7d700b3910 2024-04-24T23:04:45.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.415+0000 7f7d6f7fe640 1 -- 172.21.15.32:0/4037844087 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d6402ac30 con 0x7f7d700b3910 2024-04-24T23:04:45.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.415+0000 7f7d6f7fe640 1 -- 172.21.15.32:0/4037844087 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d680019f0 con 0x7f7d700b3910 2024-04-24T23:04:45.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.416+0000 7f7d7599e640 1 -- 172.21.15.32:0/4037844087 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d700b3910 msgr2=0x7f7d700d38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:45.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.416+0000 7f7d7599e640 1 --2- 172.21.15.32:0/4037844087 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d700b3910 0x7f7d700d38a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7d640097d0 tx=0x7f7d6402a1a0 comp rx=0 tx=0).stop 2024-04-24T23:04:45.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.416+0000 7f7d7599e640 1 -- 172.21.15.32:0/4037844087 shutdown_connections 2024-04-24T23:04:45.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.416+0000 7f7d7599e640 1 --2- 172.21.15.32:0/4037844087 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d700b3910 0x7f7d700d38a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:45.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.416+0000 7f7d7599e640 1 -- 172.21.15.32:0/4037844087 >> 172.21.15.32:0/4037844087 conn(0x7f7d700d4540 msgr2=0x7f7d700d4950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:45.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.416+0000 7f7d7599e640 1 -- 172.21.15.32:0/4037844087 shutdown_connections 2024-04-24T23:04:45.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.416+0000 7f7d7599e640 1 -- 172.21.15.32:0/4037844087 wait complete. 2024-04-24T23:04:45.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.417+0000 7f7d7599e640 1 Processor -- start 2024-04-24T23:04:45.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.417+0000 7f7d7599e640 1 -- start start 2024-04-24T23:04:45.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.418+0000 7f7d7599e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d700b3910 0x7f7d7015c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:45.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.418+0000 7f7d7599e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d700b6080 0x7f7d700ac130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:45.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.418+0000 7f7d7599e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d700cfcb0 0x7f7d700cd390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:45.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.418+0000 7f7d7599e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7d700b6bf0 con 0x7f7d700b6080 2024-04-24T23:04:45.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.418+0000 7f7d7599e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7d700d3de0 con 0x7f7d700cfcb0 2024-04-24T23:04:45.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.418+0000 7f7d7599e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f7d7015ecb0 con 0x7f7d700b3910 2024-04-24T23:04:45.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.418+0000 7f7d6ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d700b6080 0x7f7d700ac130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:45.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.418+0000 7f7d6ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d700b6080 0x7f7d700ac130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47096/0 (socket says 172.21.15.32:47096) 2024-04-24T23:04:45.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.418+0000 7f7d6ffff640 1 -- 172.21.15.32:0/2016156049 learned_addr learned my addr 172.21.15.32:0/2016156049 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:45.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.418+0000 7f7d7519d640 1 --2- 172.21.15.32:0/2016156049 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d700cfcb0 0x7f7d700cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:45.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.419+0000 7f7d7499c640 1 --2- 172.21.15.32:0/2016156049 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d700b3910 0x7f7d7015c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:45.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.419+0000 7f7d6ffff640 1 -- 172.21.15.32:0/2016156049 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d700b3910 msgr2=0x7f7d7015c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:45.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.419+0000 7f7d6ffff640 1 --2- 172.21.15.32:0/2016156049 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d700b3910 0x7f7d7015c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:45.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.419+0000 7f7d6ffff640 1 -- 172.21.15.32:0/2016156049 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d700cfcb0 msgr2=0x7f7d700cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:45.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.419+0000 7f7d6ffff640 1 --2- 172.21.15.32:0/2016156049 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d700cfcb0 0x7f7d700cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:45.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.419+0000 7f7d6ffff640 1 -- 172.21.15.32:0/2016156049 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d7015ef90 con 0x7f7d700b6080 2024-04-24T23:04:45.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.419+0000 7f7d6ffff640 1 --2- 172.21.15.32:0/2016156049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d700b6080 0x7f7d700ac130 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c00c490 tx=0x7f7d5c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:45.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.419+0000 7f7d6d7fa640 1 -- 172.21.15.32:0/2016156049 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c0052d0 con 0x7f7d700b6080 2024-04-24T23:04:45.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.419+0000 7f7d6d7fa640 1 -- 172.21.15.32:0/2016156049 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d5c005470 con 0x7f7d700b6080 2024-04-24T23:04:45.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.419+0000 7f7d6d7fa640 1 -- 172.21.15.32:0/2016156049 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c012850 con 0x7f7d700b6080 2024-04-24T23:04:45.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.419+0000 7f7d7599e640 1 -- 172.21.15.32:0/2016156049 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d7015f270 con 0x7f7d700b6080 2024-04-24T23:04:45.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.419+0000 7f7d7599e640 1 -- 172.21.15.32:0/2016156049 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d7015f830 con 0x7f7d700b6080 2024-04-24T23:04:45.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.420+0000 7f7d7599e640 1 -- 172.21.15.32:0/2016156049 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d700682b0 con 0x7f7d700b6080 2024-04-24T23:04:45.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.421+0000 7f7d6d7fa640 1 -- 172.21.15.32:0/2016156049 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7d5c0129f0 con 0x7f7d700b6080 2024-04-24T23:04:45.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.421+0000 7f7d6d7fa640 1 --2- 172.21.15.32:0/2016156049 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7d5406be30 0x7f7d5406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:45.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.421+0000 7f7d6d7fa640 1 -- 172.21.15.32:0/2016156049 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5795+0+0 (secure 0 0 0) 0x7f7d5c08a9a0 con 0x7f7d700b6080 2024-04-24T23:04:45.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.421+0000 7f7d7499c640 1 --2- 172.21.15.32:0/2016156049 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7d5406be30 0x7f7d5406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:45.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.422+0000 7f7d7499c640 1 --2- 172.21.15.32:0/2016156049 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7d5406be30 0x7f7d5406e2f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f7d64034000 tx=0x7f7d6402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:45.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.426+0000 7f7d6d7fa640 1 -- 172.21.15.32:0/2016156049 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7d5c058d30 con 0x7f7d700b6080 2024-04-24T23:04:45.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:45.611+0000 7f7d7599e640 1 -- 172.21.15.32:0/2016156049 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7d700cfa60 con 0x7f7d700b6080 2024-04-24T23:04:46.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.246+0000 7f7d6d7fa640 1 -- 172.21.15.32:0/2016156049 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v115) ==== 181+0+0 (secure 0 0 0) 0x7f7d5c05d030 con 0x7f7d700b6080 2024-04-24T23:04:46.248 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2024-04-24T23:04:46.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.249+0000 7f7d7599e640 1 -- 172.21.15.32:0/2016156049 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7d5406be30 msgr2=0x7f7d5406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:46.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.249+0000 7f7d7599e640 1 --2- 172.21.15.32:0/2016156049 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7d5406be30 0x7f7d5406e2f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f7d64034000 tx=0x7f7d6402e000 comp rx=0 tx=0).stop 2024-04-24T23:04:46.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.249+0000 7f7d7599e640 1 -- 172.21.15.32:0/2016156049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d700b6080 msgr2=0x7f7d700ac130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:46.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.249+0000 7f7d7599e640 1 --2- 172.21.15.32:0/2016156049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d700b6080 0x7f7d700ac130 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c00c490 tx=0x7f7d5c00c950 comp rx=0 tx=0).stop 2024-04-24T23:04:46.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.249+0000 7f7d7599e640 1 -- 172.21.15.32:0/2016156049 shutdown_connections 2024-04-24T23:04:46.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.249+0000 7f7d7599e640 1 --2- 172.21.15.32:0/2016156049 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7d5406be30 0x7f7d5406e2f0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:46.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.249+0000 7f7d7599e640 1 --2- 172.21.15.32:0/2016156049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d700b6080 0x7f7d700ac130 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:46.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.249+0000 7f7d7599e640 1 --2- 172.21.15.32:0/2016156049 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d700cfcb0 0x7f7d700cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:46.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.249+0000 7f7d7599e640 1 --2- 172.21.15.32:0/2016156049 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d700b3910 0x7f7d7015c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:46.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.249+0000 7f7d7599e640 1 -- 172.21.15.32:0/2016156049 >> 172.21.15.32:0/2016156049 conn(0x7f7d700d4540 msgr2=0x7f7d700d4920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:46.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.249+0000 7f7d7599e640 1 -- 172.21.15.32:0/2016156049 shutdown_connections 2024-04-24T23:04:46.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.249+0000 7f7d7599e640 1 -- 172.21.15.32:0/2016156049 wait complete. 2024-04-24T23:04:46.263 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-24T23:04:46.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.371+0000 7f5358ddb640 1 Processor -- start 2024-04-24T23:04:46.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.371+0000 7f5358ddb640 1 -- start start 2024-04-24T23:04:46.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.372+0000 7f5358ddb640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f53540b5390 0x7f53540b5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:46.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.372+0000 7f5358ddb640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53540b5d60 0x7f53540b36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:46.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.372+0000 7f5358ddb640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5354160390 con 0x7f535415c420 2024-04-24T23:04:46.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.372+0000 7f5358ddb640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f5354160140 con 0x7f53540b5390 2024-04-24T23:04:46.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.372+0000 7f5358ddb640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f5354160fa0 con 0x7f53540b5d60 2024-04-24T23:04:46.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.372+0000 7f535259c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f535415c420 0x7f53540af700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60848/0 (socket says 172.21.15.32:60848) 2024-04-24T23:04:46.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.372+0000 7f535259c640 1 -- 172.21.15.32:0/3989795795 learned_addr learned my addr 172.21.15.32:0/3989795795 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:04:46.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.372+0000 7f5351d9b640 1 -- 172.21.15.32:0/3989795795 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4049707256 0 0) 0x7f5354160390 con 0x7f535415c420 2024-04-24T23:04:46.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.372+0000 7f5351d9b640 1 -- 172.21.15.32:0/3989795795 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5340003520 con 0x7f535415c420 2024-04-24T23:04:46.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.372+0000 7f534bfff640 1 --2- 172.21.15.32:0/3989795795 >> v2:172.21.15.121:3300/0 conn(0x7f53540b5390 0x7f53540b5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:46.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.372+0000 7f5352d9d640 1 --2- 172.21.15.32:0/3989795795 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53540b5d60 0x7f53540b36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:46.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.372+0000 7f5351d9b640 1 -- 172.21.15.32:0/3989795795 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1769216501 0 0) 0x7f5340003520 con 0x7f535415c420 2024-04-24T23:04:46.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.372+0000 7f5351d9b640 1 -- 172.21.15.32:0/3989795795 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53540b5d60 msgr2=0x7f53540b36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:46.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.372+0000 7f5351d9b640 1 --2- 172.21.15.32:0/3989795795 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53540b5d60 0x7f53540b36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:46.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.373+0000 7f5351d9b640 1 -- 172.21.15.32:0/3989795795 >> v2:172.21.15.121:3300/0 conn(0x7f53540b5390 msgr2=0x7f53540b5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:46.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.373+0000 7f5351d9b640 1 --2- 172.21.15.32:0/3989795795 >> v2:172.21.15.121:3300/0 conn(0x7f53540b5390 0x7f53540b5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:46.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.373+0000 7f5351d9b640 1 -- 172.21.15.32:0/3989795795 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5354160670 con 0x7f535415c420 2024-04-24T23:04:46.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.373+0000 7f5351d9b640 1 -- 172.21.15.32:0/3989795795 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f5344001670 con 0x7f535415c420 2024-04-24T23:04:46.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.373+0000 7f5351d9b640 1 -- 172.21.15.32:0/3989795795 >> v1:172.21.15.32:6789/0 conn(0x7f535415c420 legacy=0x7f53540af700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:46.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.373+0000 7f5351d9b640 1 --2- 172.21.15.32:0/3989795795 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53540b5d60 0x7f53400039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:46.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.373+0000 7f5351d9b640 1 --2- 172.21.15.32:0/3989795795 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5340003f30 0x7f5340004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:46.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.373+0000 7f5351d9b640 1 --2- 172.21.15.32:0/3989795795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5340004d40 0x7f53400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:46.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.373+0000 7f5351d9b640 1 -- 172.21.15.32:0/3989795795 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5354160140 con 0x7f5340004d40 2024-04-24T23:04:46.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.373+0000 7f5351d9b640 1 -- 172.21.15.32:0/3989795795 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5354160fa0 con 0x7f5340003f30 2024-04-24T23:04:46.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.373+0000 7f5351d9b640 1 -- 172.21.15.32:0/3989795795 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f534000b9c0 con 0x7f53540b5d60 2024-04-24T23:04:46.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.373+0000 7f5352d9d640 1 --2- 172.21.15.32:0/3989795795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5340004d40 0x7f53400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:46.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.373+0000 7f535259c640 1 --2- 172.21.15.32:0/3989795795 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53540b5d60 0x7f53400039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:46.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.373+0000 7f534bfff640 1 --2- 172.21.15.32:0/3989795795 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5340003f30 0x7f5340004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:46.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.373+0000 7f5352d9d640 1 -- 172.21.15.32:0/3989795795 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53540b5d60 msgr2=0x7f53400039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:46.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.373+0000 7f5352d9d640 1 --2- 172.21.15.32:0/3989795795 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53540b5d60 0x7f53400039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:46.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.373+0000 7f5352d9d640 1 -- 172.21.15.32:0/3989795795 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5340003f30 msgr2=0x7f5340004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:46.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.373+0000 7f5352d9d640 1 --2- 172.21.15.32:0/3989795795 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5340003f30 0x7f5340004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:46.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.373+0000 7f5352d9d640 1 -- 172.21.15.32:0/3989795795 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5340003520 con 0x7f5340004d40 2024-04-24T23:04:46.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.374+0000 7f5352d9d640 1 --2- 172.21.15.32:0/3989795795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5340004d40 0x7f53400051c0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f534c009fc0 tx=0x7f5340005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71add7d651b4e45b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:46.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.374+0000 7f5351d9b640 1 -- 172.21.15.32:0/3989795795 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534c009c10 con 0x7f5340004d40 2024-04-24T23:04:46.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.374+0000 7f5351d9b640 1 -- 172.21.15.32:0/3989795795 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f534c036070 con 0x7f5340004d40 2024-04-24T23:04:46.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.374+0000 7f5351d9b640 1 -- 172.21.15.32:0/3989795795 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534c031670 con 0x7f5340004d40 2024-04-24T23:04:46.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.375+0000 7f5358ddb640 1 -- 172.21.15.32:0/3989795795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5340004d40 msgr2=0x7f53400051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:46.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.375+0000 7f5358ddb640 1 --2- 172.21.15.32:0/3989795795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5340004d40 0x7f53400051c0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f534c009fc0 tx=0x7f5340005830 comp rx=0 tx=0).stop 2024-04-24T23:04:46.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.375+0000 7f5352d9d640 1 -- 172.21.15.32:0/3989795795 reap_dead start 2024-04-24T23:04:46.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.375+0000 7f5358ddb640 1 -- 172.21.15.32:0/3989795795 shutdown_connections 2024-04-24T23:04:46.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.375+0000 7f5358ddb640 1 -- 172.21.15.32:0/3989795795 >> 172.21.15.32:0/3989795795 conn(0x7f53540b1f10 msgr2=0x7f53540b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:46.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.375+0000 7f5358ddb640 1 -- 172.21.15.32:0/3989795795 shutdown_connections 2024-04-24T23:04:46.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.375+0000 7f5358ddb640 1 -- 172.21.15.32:0/3989795795 wait complete. 2024-04-24T23:04:46.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.376+0000 7f5358ddb640 1 Processor -- start 2024-04-24T23:04:46.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.376+0000 7f5358ddb640 1 -- start start 2024-04-24T23:04:46.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.377+0000 7f5358ddb640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53541449a0 0x7f53541e5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:46.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.377+0000 7f5358ddb640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5354144d80 0x7f535415eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:46.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.377+0000 7f5358ddb640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53541e7750 0x7f5354156ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:46.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.377+0000 7f5358ddb640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f53540717b0 con 0x7f53541449a0 2024-04-24T23:04:46.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.377+0000 7f5358ddb640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5354160c30 con 0x7f53541e7750 2024-04-24T23:04:46.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.377+0000 7f5358ddb640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f5354071930 con 0x7f5354144d80 2024-04-24T23:04:46.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.377+0000 7f535259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53541449a0 0x7f53541e5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:46.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.377+0000 7f535259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53541449a0 0x7f53541e5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47104/0 (socket says 172.21.15.32:47104) 2024-04-24T23:04:46.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.377+0000 7f535259c640 1 -- 172.21.15.32:0/227847756 learned_addr learned my addr 172.21.15.32:0/227847756 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:46.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.377+0000 7f5352d9d640 1 --2- 172.21.15.32:0/227847756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53541e7750 0x7f5354156ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:46.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.377+0000 7f534bfff640 1 --2- 172.21.15.32:0/227847756 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5354144d80 0x7f535415eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:46.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.377+0000 7f535259c640 1 -- 172.21.15.32:0/227847756 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5354144d80 msgr2=0x7f535415eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:46.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.377+0000 7f535259c640 1 --2- 172.21.15.32:0/227847756 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5354144d80 0x7f535415eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:46.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.377+0000 7f535259c640 1 -- 172.21.15.32:0/227847756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53541e7750 msgr2=0x7f5354156ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:46.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.377+0000 7f535259c640 1 --2- 172.21.15.32:0/227847756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53541e7750 0x7f5354156ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:46.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.377+0000 7f535259c640 1 -- 172.21.15.32:0/227847756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53541e64b0 con 0x7f53541449a0 2024-04-24T23:04:46.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.377+0000 7f534bfff640 1 --2- 172.21.15.32:0/227847756 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5354144d80 0x7f535415eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:04:46.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.377+0000 7f535259c640 1 --2- 172.21.15.32:0/227847756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53541449a0 0x7f53541e5f70 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f5344007310 tx=0x7f53440077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:46.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.377+0000 7f534affd640 1 -- 172.21.15.32:0/227847756 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534400f9a0 con 0x7f53541449a0 2024-04-24T23:04:46.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.377+0000 7f534affd640 1 -- 172.21.15.32:0/227847756 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5344018070 con 0x7f53541449a0 2024-04-24T23:04:46.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.377+0000 7f5358ddb640 1 -- 172.21.15.32:0/227847756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f535415df50 con 0x7f53541449a0 2024-04-24T23:04:46.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.378+0000 7f534affd640 1 -- 172.21.15.32:0/227847756 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534400fda0 con 0x7f53541449a0 2024-04-24T23:04:46.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.378+0000 7f5358ddb640 1 -- 172.21.15.32:0/227847756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f535415e510 con 0x7f53541449a0 2024-04-24T23:04:46.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.378+0000 7f5358ddb640 1 -- 172.21.15.32:0/227847756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5354068430 con 0x7f53541449a0 2024-04-24T23:04:46.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.379+0000 7f534affd640 1 -- 172.21.15.32:0/227847756 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f534401d460 con 0x7f53541449a0 2024-04-24T23:04:46.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.379+0000 7f534affd640 1 --2- 172.21.15.32:0/227847756 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f533406be30 0x7f533406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:46.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.380+0000 7f534affd640 1 -- 172.21.15.32:0/227847756 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5357+0+0 (secure 0 0 0) 0x7f534408ae80 con 0x7f53541449a0 2024-04-24T23:04:46.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.380+0000 7f534bfff640 1 --2- 172.21.15.32:0/227847756 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f533406be30 0x7f533406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:46.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.380+0000 7f534bfff640 1 --2- 172.21.15.32:0/227847756 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f533406be30 0x7f533406e2f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f53541526c0 tx=0x7f533c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:46.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.384+0000 7f534affd640 1 -- 172.21.15.32:0/227847756 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f53440598f0 con 0x7f53541449a0 2024-04-24T23:04:46.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:46.566+0000 7f5358ddb640 1 -- 172.21.15.32:0/227847756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f53540730d0 con 0x7f53541449a0 2024-04-24T23:04:47.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.248+0000 7f534affd640 1 -- 172.21.15.32:0/227847756 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v116) ==== 169+0+0 (secure 0 0 0) 0x7f53440591f0 con 0x7f53541449a0 2024-04-24T23:04:47.250 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2024-04-24T23:04:47.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.251+0000 7f5358ddb640 1 -- 172.21.15.32:0/227847756 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f533406be30 msgr2=0x7f533406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:47.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.251+0000 7f5358ddb640 1 --2- 172.21.15.32:0/227847756 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f533406be30 0x7f533406e2f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f53541526c0 tx=0x7f533c008040 comp rx=0 tx=0).stop 2024-04-24T23:04:47.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.251+0000 7f5358ddb640 1 -- 172.21.15.32:0/227847756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53541449a0 msgr2=0x7f53541e5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:47.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.251+0000 7f5358ddb640 1 --2- 172.21.15.32:0/227847756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53541449a0 0x7f53541e5f70 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f5344007310 tx=0x7f53440077d0 comp rx=0 tx=0).stop 2024-04-24T23:04:47.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.251+0000 7f5358ddb640 1 -- 172.21.15.32:0/227847756 shutdown_connections 2024-04-24T23:04:47.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.251+0000 7f5358ddb640 1 --2- 172.21.15.32:0/227847756 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f533406be30 0x7f533406e2f0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:47.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.251+0000 7f5358ddb640 1 --2- 172.21.15.32:0/227847756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53541e7750 0x7f5354156ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:47.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.251+0000 7f5358ddb640 1 --2- 172.21.15.32:0/227847756 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5354144d80 0x7f535415eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:47.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.251+0000 7f5358ddb640 1 --2- 172.21.15.32:0/227847756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53541449a0 0x7f53541e5f70 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:47.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.251+0000 7f5358ddb640 1 -- 172.21.15.32:0/227847756 >> 172.21.15.32:0/227847756 conn(0x7f53540b1f10 msgr2=0x7f53540cee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:47.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.251+0000 7f5358ddb640 1 -- 172.21.15.32:0/227847756 shutdown_connections 2024-04-24T23:04:47.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.251+0000 7f5358ddb640 1 -- 172.21.15.32:0/227847756 wait complete. 2024-04-24T23:04:47.268 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-24T23:04:47.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.378+0000 7f96746bd640 1 Processor -- start 2024-04-24T23:04:47.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.378+0000 7f96746bd640 1 -- start start 2024-04-24T23:04:47.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.379+0000 7f96746bd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f966c162320 0x7f966c0ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:47.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.379+0000 7f96746bd640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f966c162e10 0x7f966c15ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:47.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.379+0000 7f96746bd640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f966c0cd270 con 0x7f966c0b5af0 2024-04-24T23:04:47.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.379+0000 7f96746bd640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f966c0b5660 con 0x7f966c162320 2024-04-24T23:04:47.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.379+0000 7f96746bd640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f966c071610 con 0x7f966c162e10 2024-04-24T23:04:47.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.379+0000 7f9672459640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f966c162320 0x7f966c0ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:47.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.379+0000 7f9672459640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f966c162320 0x7f966c0ccc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47116/0 (socket says 172.21.15.32:47116) 2024-04-24T23:04:47.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.379+0000 7f9672459640 1 -- 172.21.15.32:0/890841788 learned_addr learned my addr 172.21.15.32:0/890841788 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:47.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.379+0000 7f9672459640 1 -- 172.21.15.32:0/890841788 >> v2:172.21.15.121:3300/0 conn(0x7f966c162e10 msgr2=0x7f966c15ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:47.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.379+0000 7f9672c5a640 1 --2- 172.21.15.32:0/890841788 >> v2:172.21.15.121:3300/0 conn(0x7f966c162e10 0x7f966c15ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:47.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.379+0000 7f9672459640 1 --2- 172.21.15.32:0/890841788 >> v2:172.21.15.121:3300/0 conn(0x7f966c162e10 0x7f966c15ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:47.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.379+0000 7f9672459640 1 -- 172.21.15.32:0/890841788 >> v1:172.21.15.121:6789/0 conn(0x7f966c0b5af0 legacy=0x7f966c162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:47.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.379+0000 7f9672459640 1 -- 172.21.15.32:0/890841788 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f966c0718f0 con 0x7f966c162320 2024-04-24T23:04:47.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.379+0000 7f9672c5a640 1 --2- 172.21.15.32:0/890841788 >> v2:172.21.15.121:3300/0 conn(0x7f966c162e10 0x7f966c15ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:04:47.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.380+0000 7f9672459640 1 --2- 172.21.15.32:0/890841788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f966c162320 0x7f966c0ccc90 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f966c0696a0 tx=0x7f966802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aea1640f10dd59a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:47.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.380+0000 7f9671457640 1 -- 172.21.15.32:0/890841788 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9668037070 con 0x7f966c162320 2024-04-24T23:04:47.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.380+0000 7f9671457640 1 -- 172.21.15.32:0/890841788 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9668032440 con 0x7f966c162320 2024-04-24T23:04:47.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.380+0000 7f9671457640 1 -- 172.21.15.32:0/890841788 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9668032810 con 0x7f966c162320 2024-04-24T23:04:47.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.381+0000 7f96746bd640 1 -- 172.21.15.32:0/890841788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f966c162320 msgr2=0x7f966c0ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:47.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.381+0000 7f96746bd640 1 --2- 172.21.15.32:0/890841788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f966c162320 0x7f966c0ccc90 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f966c0696a0 tx=0x7f966802a1a0 comp rx=0 tx=0).stop 2024-04-24T23:04:47.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.381+0000 7f96746bd640 1 -- 172.21.15.32:0/890841788 shutdown_connections 2024-04-24T23:04:47.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.381+0000 7f96746bd640 1 --2- 172.21.15.32:0/890841788 >> v2:172.21.15.121:3300/0 conn(0x7f966c162e10 0x7f966c15ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:47.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.381+0000 7f96746bd640 1 --2- 172.21.15.32:0/890841788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f966c162320 0x7f966c0ccc90 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:47.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.381+0000 7f96746bd640 1 -- 172.21.15.32:0/890841788 >> 172.21.15.32:0/890841788 conn(0x7f966c0d0ae0 msgr2=0x7f966c0d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:47.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.381+0000 7f96746bd640 1 -- 172.21.15.32:0/890841788 shutdown_connections 2024-04-24T23:04:47.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.381+0000 7f96746bd640 1 -- 172.21.15.32:0/890841788 wait complete. 2024-04-24T23:04:47.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.382+0000 7f96746bd640 1 Processor -- start 2024-04-24T23:04:47.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.382+0000 7f96746bd640 1 -- start start 2024-04-24T23:04:47.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.383+0000 7f96746bd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f966c0b5af0 0x7f966c0cf390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:47.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.383+0000 7f96746bd640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f966c162320 0x7f966c06e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:47.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.383+0000 7f96746bd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f966c162e10 0x7f966c14e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:47.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.383+0000 7f96746bd640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f966c071bd0 con 0x7f966c0b5af0 2024-04-24T23:04:47.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.383+0000 7f96746bd640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f966c067950 con 0x7f966c162e10 2024-04-24T23:04:47.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.383+0000 7f96746bd640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f966c071d50 con 0x7f966c162320 2024-04-24T23:04:47.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.383+0000 7f9672459640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f966c0b5af0 0x7f966c0cf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:47.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.383+0000 7f9672459640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f966c0b5af0 0x7f966c0cf390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47118/0 (socket says 172.21.15.32:47118) 2024-04-24T23:04:47.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.383+0000 7f9672459640 1 -- 172.21.15.32:0/3678440962 learned_addr learned my addr 172.21.15.32:0/3678440962 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:47.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.383+0000 7f9671c58640 1 --2- 172.21.15.32:0/3678440962 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f966c162320 0x7f966c06e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:47.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.383+0000 7f9672c5a640 1 --2- 172.21.15.32:0/3678440962 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f966c162e10 0x7f966c14e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:47.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.383+0000 7f9672459640 1 -- 172.21.15.32:0/3678440962 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f966c162320 msgr2=0x7f966c06e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:47.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.383+0000 7f9672459640 1 --2- 172.21.15.32:0/3678440962 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f966c162320 0x7f966c06e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:47.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.383+0000 7f9672459640 1 -- 172.21.15.32:0/3678440962 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f966c162e10 msgr2=0x7f966c14e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:47.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.383+0000 7f9672459640 1 --2- 172.21.15.32:0/3678440962 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f966c162e10 0x7f966c14e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:47.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.383+0000 7f9672459640 1 -- 172.21.15.32:0/3678440962 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f966c1e9060 con 0x7f966c0b5af0 2024-04-24T23:04:47.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.383+0000 7f9672459640 1 --2- 172.21.15.32:0/3678440962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f966c0b5af0 0x7f966c0cf390 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f9668000c00 tx=0x7f96680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:47.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.384+0000 7f9662ffd640 1 -- 172.21.15.32:0/3678440962 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9668041020 con 0x7f966c0b5af0 2024-04-24T23:04:47.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.384+0000 7f9662ffd640 1 -- 172.21.15.32:0/3678440962 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9668030970 con 0x7f966c0b5af0 2024-04-24T23:04:47.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.384+0000 7f96746bd640 1 -- 172.21.15.32:0/3678440962 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f966c1e9340 con 0x7f966c0b5af0 2024-04-24T23:04:47.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.384+0000 7f9662ffd640 1 -- 172.21.15.32:0/3678440962 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9668037040 con 0x7f966c0b5af0 2024-04-24T23:04:47.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.384+0000 7f96746bd640 1 -- 172.21.15.32:0/3678440962 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f966c1e9be0 con 0x7f966c0b5af0 2024-04-24T23:04:47.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.385+0000 7f96746bd640 1 -- 172.21.15.32:0/3678440962 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f966c1e9510 con 0x7f966c0b5af0 2024-04-24T23:04:47.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.385+0000 7f9662ffd640 1 -- 172.21.15.32:0/3678440962 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9668044020 con 0x7f966c0b5af0 2024-04-24T23:04:47.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.386+0000 7f9662ffd640 1 --2- 172.21.15.32:0/3678440962 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f965006be30 0x7f965006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:47.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.386+0000 7f9662ffd640 1 -- 172.21.15.32:0/3678440962 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 4943+0+0 (secure 0 0 0) 0x7f96680ab640 con 0x7f966c0b5af0 2024-04-24T23:04:47.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.386+0000 7f9671c58640 1 --2- 172.21.15.32:0/3678440962 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f965006be30 0x7f965006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:47.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.386+0000 7f9671c58640 1 --2- 172.21.15.32:0/3678440962 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f965006be30 0x7f965006e2f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f966c06f8b0 tx=0x7f965c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:47.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.390+0000 7f9662ffd640 1 -- 172.21.15.32:0/3678440962 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f966804b030 con 0x7f966c0b5af0 2024-04-24T23:04:47.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.571+0000 7f96746bd640 1 -- 172.21.15.32:0/3678440962 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f966c068430 con 0x7f966c0b5af0 2024-04-24T23:04:47.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.574+0000 7f9662ffd640 1 -- 172.21.15.32:0/3678440962 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f966807a250 con 0x7f966c0b5af0 2024-04-24T23:04:47.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.576+0000 7f96746bd640 1 -- 172.21.15.32:0/3678440962 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f965006be30 msgr2=0x7f965006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:47.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.576+0000 7f96746bd640 1 --2- 172.21.15.32:0/3678440962 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f965006be30 0x7f965006e2f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f966c06f8b0 tx=0x7f965c009040 comp rx=0 tx=0).stop 2024-04-24T23:04:47.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.576+0000 7f96746bd640 1 -- 172.21.15.32:0/3678440962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f966c0b5af0 msgr2=0x7f966c0cf390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:47.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.576+0000 7f96746bd640 1 --2- 172.21.15.32:0/3678440962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f966c0b5af0 0x7f966c0cf390 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f9668000c00 tx=0x7f96680046c0 comp rx=0 tx=0).stop 2024-04-24T23:04:47.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.576+0000 7f96746bd640 1 -- 172.21.15.32:0/3678440962 shutdown_connections 2024-04-24T23:04:47.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.576+0000 7f96746bd640 1 --2- 172.21.15.32:0/3678440962 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f965006be30 0x7f965006e2f0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:47.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.576+0000 7f96746bd640 1 --2- 172.21.15.32:0/3678440962 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f966c162e10 0x7f966c14e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:47.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.576+0000 7f96746bd640 1 --2- 172.21.15.32:0/3678440962 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f966c162320 0x7f966c06e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:47.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.576+0000 7f96746bd640 1 --2- 172.21.15.32:0/3678440962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f966c0b5af0 0x7f966c0cf390 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:47.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.576+0000 7f96746bd640 1 -- 172.21.15.32:0/3678440962 >> 172.21.15.32:0/3678440962 conn(0x7f966c0d0ae0 msgr2=0x7f966c15fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:47.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.576+0000 7f96746bd640 1 -- 172.21.15.32:0/3678440962 shutdown_connections 2024-04-24T23:04:47.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.576+0000 7f96746bd640 1 -- 172.21.15.32:0/3678440962 wait complete. 2024-04-24T23:04:47.588 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_same_file' 2024-04-24T23:04:47.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.688+0000 7f2587e10640 1 Processor -- start 2024-04-24T23:04:47.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.688+0000 7f2587e10640 1 -- start start 2024-04-24T23:04:47.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.689+0000 7f2587e10640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f258006c5f0 0x7f258006c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:47.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.689+0000 7f2587e10640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f25800af890 0x7f258015cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:47.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.689+0000 7f2587e10640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25800b6430 con 0x7f25800b44d0 2024-04-24T23:04:47.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.689+0000 7f2587e10640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f2580156f80 con 0x7f258006c5f0 2024-04-24T23:04:47.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.689+0000 7f2587e10640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f25800b6150 con 0x7f25800af890 2024-04-24T23:04:47.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.689+0000 7f2585bac640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f258006c5f0 0x7f258006c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:47.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.689+0000 7f25853ab640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f25800b44d0 0x7f25800b5890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60858/0 (socket says 172.21.15.32:60858) 2024-04-24T23:04:47.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.689+0000 7f25853ab640 1 -- 172.21.15.32:0/125431136 learned_addr learned my addr 172.21.15.32:0/125431136 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:04:47.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.689+0000 7f25863ad640 1 --2- 172.21.15.32:0/125431136 >> v2:172.21.15.121:3300/0 conn(0x7f25800af890 0x7f258015cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:47.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.689+0000 7f2585bac640 1 -- 172.21.15.32:0/125431136 >> v2:172.21.15.121:3300/0 conn(0x7f25800af890 msgr2=0x7f258015cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:47.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.689+0000 7f2585bac640 1 --2- 172.21.15.32:0/125431136 >> v2:172.21.15.121:3300/0 conn(0x7f25800af890 0x7f258015cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:47.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.689+0000 7f2585bac640 1 -- 172.21.15.32:0/125431136 >> v1:172.21.15.32:6789/0 conn(0x7f25800b44d0 legacy=0x7f25800b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:47.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.690+0000 7f2585bac640 1 -- 172.21.15.32:0/125431136 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25800b39f0 con 0x7f258006c5f0 2024-04-24T23:04:47.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.690+0000 7f25863ad640 1 --2- 172.21.15.32:0/125431136 >> v2:172.21.15.121:3300/0 conn(0x7f25800af890 0x7f258015cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:04:47.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.690+0000 7f2585bac640 1 --2- 172.21.15.32:0/125431136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f258006c5f0 0x7f258006c9d0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f2580073960 tx=0x7f257802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a6b9bbeb3d02c27 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:47.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.690+0000 7f2584baa640 1 -- 172.21.15.32:0/125431136 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2578037070 con 0x7f258006c5f0 2024-04-24T23:04:47.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.690+0000 7f2584baa640 1 -- 172.21.15.32:0/125431136 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2578032440 con 0x7f258006c5f0 2024-04-24T23:04:47.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.690+0000 7f2584baa640 1 -- 172.21.15.32:0/125431136 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2578032810 con 0x7f258006c5f0 2024-04-24T23:04:47.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.691+0000 7f2587e10640 1 -- 172.21.15.32:0/125431136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f258006c5f0 msgr2=0x7f258006c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:47.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.691+0000 7f2587e10640 1 --2- 172.21.15.32:0/125431136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f258006c5f0 0x7f258006c9d0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f2580073960 tx=0x7f257802a1a0 comp rx=0 tx=0).stop 2024-04-24T23:04:47.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.691+0000 7f2587e10640 1 -- 172.21.15.32:0/125431136 shutdown_connections 2024-04-24T23:04:47.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.691+0000 7f2587e10640 1 --2- 172.21.15.32:0/125431136 >> v2:172.21.15.121:3300/0 conn(0x7f25800af890 0x7f258015cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:47.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.691+0000 7f2587e10640 1 --2- 172.21.15.32:0/125431136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f258006c5f0 0x7f258006c9d0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:47.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.691+0000 7f2587e10640 1 -- 172.21.15.32:0/125431136 >> 172.21.15.32:0/125431136 conn(0x7f258006b110 msgr2=0x7f258006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:47.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.691+0000 7f2587e10640 1 -- 172.21.15.32:0/125431136 shutdown_connections 2024-04-24T23:04:47.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.691+0000 7f2587e10640 1 -- 172.21.15.32:0/125431136 wait complete. 2024-04-24T23:04:47.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.692+0000 7f2587e10640 1 Processor -- start 2024-04-24T23:04:47.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.692+0000 7f2587e10640 1 -- start start 2024-04-24T23:04:47.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.693+0000 7f2587e10640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f258006c5f0 0x7f2580153050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:47.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.693+0000 7f2587e10640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25800af890 0x7f2580146c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:47.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.693+0000 7f2587e10640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25800b44d0 0x7f2580151760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:47.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.693+0000 7f2587e10640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f258006cf10 con 0x7f25800b44d0 2024-04-24T23:04:47.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.693+0000 7f2587e10640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f25800b3fb0 con 0x7f25800af890 2024-04-24T23:04:47.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.693+0000 7f2587e10640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f258015d000 con 0x7f258006c5f0 2024-04-24T23:04:47.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.693+0000 7f25863ad640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25800b44d0 0x7f2580151760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:47.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.693+0000 7f25863ad640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25800b44d0 0x7f2580151760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47142/0 (socket says 172.21.15.32:47142) 2024-04-24T23:04:47.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.693+0000 7f25863ad640 1 -- 172.21.15.32:0/375731705 learned_addr learned my addr 172.21.15.32:0/375731705 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:47.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.693+0000 7f25853ab640 1 --2- 172.21.15.32:0/375731705 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25800af890 0x7f2580146c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:47.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.693+0000 7f2585bac640 1 --2- 172.21.15.32:0/375731705 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f258006c5f0 0x7f2580153050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:47.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.693+0000 7f25863ad640 1 -- 172.21.15.32:0/375731705 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f258006c5f0 msgr2=0x7f2580153050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:47.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.693+0000 7f25863ad640 1 --2- 172.21.15.32:0/375731705 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f258006c5f0 0x7f2580153050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:47.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.693+0000 7f25863ad640 1 -- 172.21.15.32:0/375731705 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25800af890 msgr2=0x7f2580146c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:47.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.693+0000 7f25863ad640 1 --2- 172.21.15.32:0/375731705 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25800af890 0x7f2580146c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:47.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.693+0000 7f25863ad640 1 -- 172.21.15.32:0/375731705 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2580151fb0 con 0x7f25800b44d0 2024-04-24T23:04:47.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.693+0000 7f2585bac640 1 --2- 172.21.15.32:0/375731705 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f258006c5f0 0x7f2580153050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:04:47.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.694+0000 7f25863ad640 1 --2- 172.21.15.32:0/375731705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25800b44d0 0x7f2580151760 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f2570009fd0 tx=0x7f257000d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:47.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.694+0000 7f257e7fc640 1 -- 172.21.15.32:0/375731705 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257001a070 con 0x7f25800b44d0 2024-04-24T23:04:47.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.694+0000 7f257e7fc640 1 -- 172.21.15.32:0/375731705 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2570003bb0 con 0x7f25800b44d0 2024-04-24T23:04:47.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.694+0000 7f2587e10640 1 -- 172.21.15.32:0/375731705 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2580152290 con 0x7f25800b44d0 2024-04-24T23:04:47.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.694+0000 7f257e7fc640 1 -- 172.21.15.32:0/375731705 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2570004030 con 0x7f25800b44d0 2024-04-24T23:04:47.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.694+0000 7f2587e10640 1 -- 172.21.15.32:0/375731705 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2580152850 con 0x7f25800b44d0 2024-04-24T23:04:47.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.695+0000 7f2587e10640 1 -- 172.21.15.32:0/375731705 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2580068430 con 0x7f25800b44d0 2024-04-24T23:04:47.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.695+0000 7f257e7fc640 1 -- 172.21.15.32:0/375731705 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f257001d070 con 0x7f25800b44d0 2024-04-24T23:04:47.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.696+0000 7f257e7fc640 1 --2- 172.21.15.32:0/375731705 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f256c06be30 0x7f256c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:47.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.696+0000 7f257e7fc640 1 -- 172.21.15.32:0/375731705 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 4943+0+0 (secure 0 0 0) 0x7f257008af60 con 0x7f25800b44d0 2024-04-24T23:04:47.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.696+0000 7f2585bac640 1 --2- 172.21.15.32:0/375731705 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f256c06be30 0x7f256c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:47.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.696+0000 7f2585bac640 1 --2- 172.21.15.32:0/375731705 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f256c06be30 0x7f256c06e2f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f2578034000 tx=0x7f257802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:47.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.700+0000 7f257e7fc640 1 -- 172.21.15.32:0/375731705 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2570010070 con 0x7f25800b44d0 2024-04-24T23:04:47.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:47.964+0000 7f2587e10640 1 -- 172.21.15.32:0/375731705 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]} v 0) -- 0x7f258015fc70 con 0x7f25800b44d0 2024-04-24T23:04:48.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.249+0000 7f257e7fc640 1 -- 172.21.15.32:0/375731705 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]}]=0 v300) ==== 137+0+0 (secure 0 0 0) 0x7f2570059b80 con 0x7f25800b44d0 2024-04-24T23:04:48.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.252+0000 7f2587e10640 1 -- 172.21.15.32:0/375731705 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f256c06be30 msgr2=0x7f256c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:48.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.252+0000 7f2587e10640 1 --2- 172.21.15.32:0/375731705 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f256c06be30 0x7f256c06e2f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f2578034000 tx=0x7f257802e000 comp rx=0 tx=0).stop 2024-04-24T23:04:48.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.252+0000 7f2587e10640 1 -- 172.21.15.32:0/375731705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25800b44d0 msgr2=0x7f2580151760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:48.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.252+0000 7f2587e10640 1 --2- 172.21.15.32:0/375731705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25800b44d0 0x7f2580151760 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f2570009fd0 tx=0x7f257000d560 comp rx=0 tx=0).stop 2024-04-24T23:04:48.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.252+0000 7f2587e10640 1 -- 172.21.15.32:0/375731705 shutdown_connections 2024-04-24T23:04:48.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.252+0000 7f2587e10640 1 --2- 172.21.15.32:0/375731705 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f256c06be30 0x7f256c06e2f0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:48.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.252+0000 7f2587e10640 1 --2- 172.21.15.32:0/375731705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25800b44d0 0x7f2580151760 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:48.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.252+0000 7f2587e10640 1 --2- 172.21.15.32:0/375731705 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25800af890 0x7f2580146c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:48.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.252+0000 7f2587e10640 1 --2- 172.21.15.32:0/375731705 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f258006c5f0 0x7f2580153050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:48.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.252+0000 7f2587e10640 1 -- 172.21.15.32:0/375731705 >> 172.21.15.32:0/375731705 conn(0x7f258006b110 msgr2=0x7f258015a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:48.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.252+0000 7f2587e10640 1 -- 172.21.15.32:0/375731705 shutdown_connections 2024-04-24T23:04:48.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.252+0000 7f2587e10640 1 -- 172.21.15.32:0/375731705 wait complete. 2024-04-24T23:04:48.264 INFO:tasks.cephfs_test_runner:test_full_same_file (tasks.cephfs.test_full.TestClusterFull) ... ok 2024-04-24T23:04:48.265 INFO:tasks.cephfs_test_runner:Starting test: test_barrier (tasks.cephfs.test_full.TestQuotaFull) That when an OSD epoch barrier is set on an MDS, subsequently 2024-04-24T23:04:48.265 INFO:tasks.cephfs_test_runner:test_barrier (tasks.cephfs.test_full.TestQuotaFull) 2024-04-24T23:04:48.265 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:04:48.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.364+0000 7fcbffef5640 1 Processor -- start 2024-04-24T23:04:48.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.365+0000 7fcbffef5640 1 -- start start 2024-04-24T23:04:48.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.365+0000 7fcbffef5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcbf815b480 0x7fcbf80b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:48.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.365+0000 7fcbffef5640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fcbf815fd60 0x7fcbf806c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:48.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.366+0000 7fcbffef5640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbf80cef90 con 0x7fcbf806cb30 2024-04-24T23:04:48.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.366+0000 7fcbffef5640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fcbf80b4ff0 con 0x7fcbf815fd60 2024-04-24T23:04:48.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.366+0000 7fcbffef5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fcbf80b46c0 con 0x7fcbf815b480 2024-04-24T23:04:48.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.366+0000 7fcbfdc91640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcbf815b480 0x7fcbf80b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:48.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.366+0000 7fcbfd490640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fcbf815fd60 0x7fcbf806c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:48.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.366+0000 7fcbfd490640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fcbf815fd60 0x7fcbf806c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.32:51636/0 (socket says 172.21.15.32:51636) 2024-04-24T23:04:48.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.366+0000 7fcbfdc91640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcbf815b480 0x7fcbf80b5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.32:35052/0 (socket says 172.21.15.32:35052) 2024-04-24T23:04:48.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.366+0000 7fcbfe492640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fcbf806cb30 0x7fcbf80ccc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:38916/0 (socket says 172.21.15.32:38916) 2024-04-24T23:04:48.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.366+0000 7fcbfd490640 1 -- 172.21.15.32:0/4138185849 learned_addr learned my addr 172.21.15.32:0/4138185849 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:48.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.366+0000 7fcbfd490640 1 -- 172.21.15.32:0/4138185849 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcbf815b480 msgr2=0x7fcbf80b5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:48.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.366+0000 7fcbfd490640 1 --2- 172.21.15.32:0/4138185849 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcbf815b480 0x7fcbf80b5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:48.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.366+0000 7fcbfd490640 1 -- 172.21.15.32:0/4138185849 >> v1:172.21.15.121:6789/0 conn(0x7fcbf806cb30 legacy=0x7fcbf80ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:48.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.366+0000 7fcbfd490640 1 -- 172.21.15.32:0/4138185849 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbf80b49a0 con 0x7fcbf815fd60 2024-04-24T23:04:48.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.367+0000 7fcbfd490640 1 --2- 172.21.15.32:0/4138185849 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcbf815fd60 0x7fcbf806c5f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8009870 tx=0x7fcbe802a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=360cf580c160b508 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:48.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.367+0000 7fcbfcc8f640 1 -- 172.21.15.32:0/4138185849 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe8037070 con 0x7fcbf815fd60 2024-04-24T23:04:48.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.367+0000 7fcbfcc8f640 1 -- 172.21.15.32:0/4138185849 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbe8032440 con 0x7fcbf815fd60 2024-04-24T23:04:48.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.367+0000 7fcbfcc8f640 1 -- 172.21.15.32:0/4138185849 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe8032810 con 0x7fcbf815fd60 2024-04-24T23:04:48.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.368+0000 7fcbffef5640 1 -- 172.21.15.32:0/4138185849 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcbf815fd60 msgr2=0x7fcbf806c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:48.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.368+0000 7fcbffef5640 1 --2- 172.21.15.32:0/4138185849 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcbf815fd60 0x7fcbf806c5f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8009870 tx=0x7fcbe802a270 comp rx=0 tx=0).stop 2024-04-24T23:04:48.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.368+0000 7fcbffef5640 1 -- 172.21.15.32:0/4138185849 shutdown_connections 2024-04-24T23:04:48.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.368+0000 7fcbffef5640 1 --2- 172.21.15.32:0/4138185849 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcbf815fd60 0x7fcbf806c5f0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:48.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.368+0000 7fcbffef5640 1 --2- 172.21.15.32:0/4138185849 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcbf815b480 0x7fcbf80b5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:48.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.368+0000 7fcbffef5640 1 -- 172.21.15.32:0/4138185849 >> 172.21.15.32:0/4138185849 conn(0x7fcbf806b110 msgr2=0x7fcbf806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:48.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.368+0000 7fcbffef5640 1 -- 172.21.15.32:0/4138185849 shutdown_connections 2024-04-24T23:04:48.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.368+0000 7fcbffef5640 1 -- 172.21.15.32:0/4138185849 wait complete. 2024-04-24T23:04:48.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.369+0000 7fcbffef5640 1 Processor -- start 2024-04-24T23:04:48.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.369+0000 7fcbffef5640 1 -- start start 2024-04-24T23:04:48.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.370+0000 7fcbffef5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcbf806cb30 0x7fcbf80706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:48.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.370+0000 7fcbffef5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbf815b480 0x7fcbf8072640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:48.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.370+0000 7fcbffef5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcbf815fd60 0x7fcbf81479a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:48.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.370+0000 7fcbffef5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf80b61a0 con 0x7fcbf815b480 2024-04-24T23:04:48.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.370+0000 7fcbffef5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf806cf40 con 0x7fcbf806cb30 2024-04-24T23:04:48.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.370+0000 7fcbffef5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fcbf815b860 con 0x7fcbf815fd60 2024-04-24T23:04:48.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.370+0000 7fcbfd490640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbf815b480 0x7fcbf8072640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:48.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.370+0000 7fcbfd490640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbf815b480 0x7fcbf8072640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47150/0 (socket says 172.21.15.32:47150) 2024-04-24T23:04:48.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.370+0000 7fcbfd490640 1 -- 172.21.15.32:0/3377845019 learned_addr learned my addr 172.21.15.32:0/3377845019 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:48.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.370+0000 7fcbfdc91640 1 --2- 172.21.15.32:0/3377845019 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcbf806cb30 0x7fcbf80706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:48.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.370+0000 7fcbfe492640 1 --2- 172.21.15.32:0/3377845019 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcbf815fd60 0x7fcbf81479a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:48.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.370+0000 7fcbfd490640 1 -- 172.21.15.32:0/3377845019 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcbf815fd60 msgr2=0x7fcbf81479a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:48.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.370+0000 7fcbfd490640 1 --2- 172.21.15.32:0/3377845019 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcbf815fd60 0x7fcbf81479a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:48.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.370+0000 7fcbfd490640 1 -- 172.21.15.32:0/3377845019 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcbf806cb30 msgr2=0x7fcbf80706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:48.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.370+0000 7fcbfd490640 1 --2- 172.21.15.32:0/3377845019 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcbf806cb30 0x7fcbf80706c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:48.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.370+0000 7fcbfd490640 1 -- 172.21.15.32:0/3377845019 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbf8070ca0 con 0x7fcbf815b480 2024-04-24T23:04:48.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.370+0000 7fcbfd490640 1 --2- 172.21.15.32:0/3377845019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbf815b480 0x7fcbf8072640 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8000c00 tx=0x7fcbe80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:48.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.371+0000 7fcbee7fc640 1 -- 172.21.15.32:0/3377845019 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe8037040 con 0x7fcbf815b480 2024-04-24T23:04:48.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.371+0000 7fcbee7fc640 1 -- 172.21.15.32:0/3377845019 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbe8042070 con 0x7fcbf815b480 2024-04-24T23:04:48.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.371+0000 7fcbee7fc640 1 -- 172.21.15.32:0/3377845019 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe8030880 con 0x7fcbf815b480 2024-04-24T23:04:48.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.371+0000 7fcbffef5640 1 -- 172.21.15.32:0/3377845019 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbf8070f80 con 0x7fcbf815b480 2024-04-24T23:04:48.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.371+0000 7fcbffef5640 1 -- 172.21.15.32:0/3377845019 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbf8071540 con 0x7fcbf815b480 2024-04-24T23:04:48.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.372+0000 7fcbffef5640 1 -- 172.21.15.32:0/3377845019 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbf8068430 con 0x7fcbf815b480 2024-04-24T23:04:48.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.372+0000 7fcbee7fc640 1 -- 172.21.15.32:0/3377845019 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcbe8030a20 con 0x7fcbf815b480 2024-04-24T23:04:48.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.372+0000 7fcbee7fc640 1 --2- 172.21.15.32:0/3377845019 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcbdc06be30 0x7fcbdc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:48.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.373+0000 7fcbee7fc640 1 -- 172.21.15.32:0/3377845019 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 4943+0+0 (secure 0 0 0) 0x7fcbe80ab180 con 0x7fcbf815b480 2024-04-24T23:04:48.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.373+0000 7fcbfdc91640 1 --2- 172.21.15.32:0/3377845019 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcbdc06be30 0x7fcbdc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:48.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.373+0000 7fcbfdc91640 1 --2- 172.21.15.32:0/3377845019 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcbdc06be30 0x7fcbdc06e2f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4009b40 tx=0x7fcbf4009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:48.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.377+0000 7fcbee7fc640 1 -- 172.21.15.32:0/3377845019 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcbe8079870 con 0x7fcbf815b480 2024-04-24T23:04:48.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.560+0000 7fcbffef5640 1 -- 172.21.15.32:0/3377845019 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcbf806c830 con 0x7fcbf815b480 2024-04-24T23:04:48.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.560+0000 7fcbee7fc640 1 -- 172.21.15.32:0/3377845019 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) ==== 74+0+13043 (secure 0 0 0) 0x7fcbe807db70 con 0x7fcbf815b480 2024-04-24T23:04:48.562 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:04:48.562 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":116,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:04:47.247585+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6835/3327203898":"2024-04-25T23:04:44.242923+0000","172.21.15.32:6833/3327203898":"2024-04-25T23:04:44.242923+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:04:48.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.563+0000 7fcbffef5640 1 -- 172.21.15.32:0/3377845019 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcbdc06be30 msgr2=0x7fcbdc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:48.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.563+0000 7fcbffef5640 1 --2- 172.21.15.32:0/3377845019 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcbdc06be30 0x7fcbdc06e2f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4009b40 tx=0x7fcbf4009190 comp rx=0 tx=0).stop 2024-04-24T23:04:48.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.563+0000 7fcbffef5640 1 -- 172.21.15.32:0/3377845019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbf815b480 msgr2=0x7fcbf8072640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:48.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.563+0000 7fcbffef5640 1 --2- 172.21.15.32:0/3377845019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbf815b480 0x7fcbf8072640 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8000c00 tx=0x7fcbe80046c0 comp rx=0 tx=0).stop 2024-04-24T23:04:48.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.563+0000 7fcbffef5640 1 -- 172.21.15.32:0/3377845019 shutdown_connections 2024-04-24T23:04:48.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.563+0000 7fcbffef5640 1 --2- 172.21.15.32:0/3377845019 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcbdc06be30 0x7fcbdc06e2f0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:48.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.563+0000 7fcbffef5640 1 --2- 172.21.15.32:0/3377845019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbf815b480 0x7fcbf8072640 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:48.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.563+0000 7fcbffef5640 1 --2- 172.21.15.32:0/3377845019 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcbf815fd60 0x7fcbf81479a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:48.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.563+0000 7fcbffef5640 1 --2- 172.21.15.32:0/3377845019 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcbf806cb30 0x7fcbf80706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:48.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.563+0000 7fcbffef5640 1 -- 172.21.15.32:0/3377845019 >> 172.21.15.32:0/3377845019 conn(0x7fcbf806b110 msgr2=0x7fcbf80b5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:48.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.563+0000 7fcbffef5640 1 -- 172.21.15.32:0/3377845019 shutdown_connections 2024-04-24T23:04:48.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.563+0000 7fcbffef5640 1 -- 172.21.15.32:0/3377845019 wait complete. 2024-04-24T23:04:48.575 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:04:48.575 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:04:48.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.677+0000 7f318e031640 1 Processor -- start 2024-04-24T23:04:48.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.678+0000 7f318e031640 1 -- start start 2024-04-24T23:04:48.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.678+0000 7f318e031640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3188072cf0 0x7f318806c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:48.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.678+0000 7f318e031640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3188071770 0x7f318806cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:48.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.678+0000 7f318e031640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f318815c990 con 0x7f318815c5b0 2024-04-24T23:04:48.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.678+0000 7f318e031640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f31880b4740 con 0x7f3188072cf0 2024-04-24T23:04:48.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.678+0000 7f318e031640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f3188072860 con 0x7f3188071770 2024-04-24T23:04:48.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.678+0000 7f31877fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f318815c5b0 0x7f31880b4030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60872/0 (socket says 172.21.15.32:60872) 2024-04-24T23:04:48.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.679+0000 7f31877fe640 1 -- 172.21.15.32:0/2338586752 learned_addr learned my addr 172.21.15.32:0/2338586752 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:04:48.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.679+0000 7f3186ffd640 1 --2- 172.21.15.32:0/2338586752 >> v2:172.21.15.32:3300/0 conn(0x7f3188072cf0 0x7f318806c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:48.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.679+0000 7f3187fff640 1 --2- 172.21.15.32:0/2338586752 >> v2:172.21.15.121:3300/0 conn(0x7f3188071770 0x7f318806cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:48.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.679+0000 7f3186ffd640 1 -- 172.21.15.32:0/2338586752 >> v2:172.21.15.121:3300/0 conn(0x7f3188071770 msgr2=0x7f318806cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:48.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.679+0000 7f3186ffd640 1 --2- 172.21.15.32:0/2338586752 >> v2:172.21.15.121:3300/0 conn(0x7f3188071770 0x7f318806cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:48.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.679+0000 7f3186ffd640 1 -- 172.21.15.32:0/2338586752 >> v1:172.21.15.32:6789/0 conn(0x7f318815c5b0 legacy=0x7f31880b4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:48.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.679+0000 7f31867fc640 1 -- 172.21.15.32:0/2338586752 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3093213468 0 0) 0x7f3178001110 con 0x7f318815c5b0 2024-04-24T23:04:48.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.679+0000 7f3186ffd640 1 -- 172.21.15.32:0/2338586752 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3188071c50 con 0x7f3188072cf0 2024-04-24T23:04:48.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.679+0000 7f3187fff640 1 --2- 172.21.15.32:0/2338586752 >> v2:172.21.15.121:3300/0 conn(0x7f3188071770 0x7f318806cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:04:48.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.680+0000 7f3186ffd640 1 --2- 172.21.15.32:0/2338586752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3188072cf0 0x7f318806c5f0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f31880aff50 tx=0x7f317c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bea7f81a90865990 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:48.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.680+0000 7f31867fc640 1 -- 172.21.15.32:0/2338586752 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f317c037070 con 0x7f3188072cf0 2024-04-24T23:04:48.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.680+0000 7f31867fc640 1 -- 172.21.15.32:0/2338586752 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f317c032440 con 0x7f3188072cf0 2024-04-24T23:04:48.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.680+0000 7f31867fc640 1 -- 172.21.15.32:0/2338586752 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f317c02ae60 con 0x7f3188072cf0 2024-04-24T23:04:48.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.680+0000 7f318e031640 1 -- 172.21.15.32:0/2338586752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3188072cf0 msgr2=0x7f318806c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:48.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.680+0000 7f318e031640 1 --2- 172.21.15.32:0/2338586752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3188072cf0 0x7f318806c5f0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f31880aff50 tx=0x7f317c02a110 comp rx=0 tx=0).stop 2024-04-24T23:04:48.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.680+0000 7f318e031640 1 -- 172.21.15.32:0/2338586752 shutdown_connections 2024-04-24T23:04:48.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.680+0000 7f318e031640 1 --2- 172.21.15.32:0/2338586752 >> v2:172.21.15.121:3300/0 conn(0x7f3188071770 0x7f318806cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:48.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.680+0000 7f318e031640 1 --2- 172.21.15.32:0/2338586752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3188072cf0 0x7f318806c5f0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:48.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.680+0000 7f318e031640 1 -- 172.21.15.32:0/2338586752 >> 172.21.15.32:0/2338586752 conn(0x7f318806b110 msgr2=0x7f318806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:48.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.680+0000 7f318e031640 1 -- 172.21.15.32:0/2338586752 shutdown_connections 2024-04-24T23:04:48.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.680+0000 7f318e031640 1 -- 172.21.15.32:0/2338586752 wait complete. 2024-04-24T23:04:48.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.681+0000 7f318e031640 1 Processor -- start 2024-04-24T23:04:48.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.682+0000 7f318e031640 1 -- start start 2024-04-24T23:04:48.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.682+0000 7f318e031640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3188071770 0x7f3188150a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:48.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.682+0000 7f318e031640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3188072cf0 0x7f318814ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:48.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.682+0000 7f318e031640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f318815c5b0 0x7f3188152350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:48.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.682+0000 7f318e031640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f31880724f0 con 0x7f318815c5b0 2024-04-24T23:04:48.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.682+0000 7f318e031640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3188072210 con 0x7f3188072cf0 2024-04-24T23:04:48.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.682+0000 7f318e031640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f31880b4b40 con 0x7f3188071770 2024-04-24T23:04:48.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.683+0000 7f3187fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f318815c5b0 0x7f3188152350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:48.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.683+0000 7f3187fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f318815c5b0 0x7f3188152350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47170/0 (socket says 172.21.15.32:47170) 2024-04-24T23:04:48.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.683+0000 7f3187fff640 1 -- 172.21.15.32:0/1963525306 learned_addr learned my addr 172.21.15.32:0/1963525306 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:48.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.683+0000 7f3186ffd640 1 --2- 172.21.15.32:0/1963525306 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3188072cf0 0x7f318814ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:48.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.683+0000 7f31877fe640 1 --2- 172.21.15.32:0/1963525306 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3188071770 0x7f3188150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:48.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.683+0000 7f3187fff640 1 -- 172.21.15.32:0/1963525306 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3188071770 msgr2=0x7f3188150a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:48.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.683+0000 7f3187fff640 1 --2- 172.21.15.32:0/1963525306 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3188071770 0x7f3188150a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:48.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.683+0000 7f3187fff640 1 -- 172.21.15.32:0/1963525306 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3188072cf0 msgr2=0x7f318814ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:48.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.683+0000 7f3187fff640 1 --2- 172.21.15.32:0/1963525306 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3188072cf0 0x7f318814ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:48.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.683+0000 7f3187fff640 1 -- 172.21.15.32:0/1963525306 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3188151200 con 0x7f318815c5b0 2024-04-24T23:04:48.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.683+0000 7f3187fff640 1 --2- 172.21.15.32:0/1963525306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f318815c5b0 0x7f3188152350 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f318000ce80 tx=0x7f318000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:48.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.683+0000 7f316ffff640 1 -- 172.21.15.32:0/1963525306 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318001a070 con 0x7f318815c5b0 2024-04-24T23:04:48.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.683+0000 7f316ffff640 1 -- 172.21.15.32:0/1963525306 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f318000d880 con 0x7f318815c5b0 2024-04-24T23:04:48.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.683+0000 7f318e031640 1 -- 172.21.15.32:0/1963525306 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31881514e0 con 0x7f318815c5b0 2024-04-24T23:04:48.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.683+0000 7f316ffff640 1 -- 172.21.15.32:0/1963525306 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318000dc70 con 0x7f318815c5b0 2024-04-24T23:04:48.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.683+0000 7f318e031640 1 -- 172.21.15.32:0/1963525306 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f318815cfd0 con 0x7f318815c5b0 2024-04-24T23:04:48.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.684+0000 7f318e031640 1 -- 172.21.15.32:0/1963525306 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3188068430 con 0x7f318815c5b0 2024-04-24T23:04:48.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.685+0000 7f316ffff640 1 -- 172.21.15.32:0/1963525306 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f318001d070 con 0x7f318815c5b0 2024-04-24T23:04:48.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.685+0000 7f316ffff640 1 --2- 172.21.15.32:0/1963525306 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f317406be60 0x7f317406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:48.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.685+0000 7f316ffff640 1 -- 172.21.15.32:0/1963525306 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 4943+0+0 (secure 0 0 0) 0x7f318008b1f0 con 0x7f318815c5b0 2024-04-24T23:04:48.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.685+0000 7f31877fe640 1 --2- 172.21.15.32:0/1963525306 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f317406be60 0x7f317406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:48.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.686+0000 7f31877fe640 1 --2- 172.21.15.32:0/1963525306 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f317406be60 0x7f317406e320 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f3178007fd0 tx=0x7f3178007de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:48.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.690+0000 7f316ffff640 1 -- 172.21.15.32:0/1963525306 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f31800598e0 con 0x7f318815c5b0 2024-04-24T23:04:48.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.872+0000 7f318e031640 1 -- 172.21.15.32:0/1963525306 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f318815bd60 con 0x7f318815c5b0 2024-04-24T23:04:48.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.872+0000 7f316ffff640 1 -- 172.21.15.32:0/1963525306 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v116) ==== 93+0+10 (secure 0 0 0) 0x7f318005dbe0 con 0x7f318815c5b0 2024-04-24T23:04:48.874 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T23:04:48.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.874+0000 7f318e031640 1 -- 172.21.15.32:0/1963525306 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f317406be60 msgr2=0x7f317406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:48.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.874+0000 7f318e031640 1 --2- 172.21.15.32:0/1963525306 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f317406be60 0x7f317406e320 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f3178007fd0 tx=0x7f3178007de0 comp rx=0 tx=0).stop 2024-04-24T23:04:48.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.874+0000 7f318e031640 1 -- 172.21.15.32:0/1963525306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f318815c5b0 msgr2=0x7f3188152350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:48.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.874+0000 7f318e031640 1 --2- 172.21.15.32:0/1963525306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f318815c5b0 0x7f3188152350 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f318000ce80 tx=0x7f318000a710 comp rx=0 tx=0).stop 2024-04-24T23:04:48.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.874+0000 7f318e031640 1 -- 172.21.15.32:0/1963525306 shutdown_connections 2024-04-24T23:04:48.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.874+0000 7f318e031640 1 --2- 172.21.15.32:0/1963525306 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f317406be60 0x7f317406e320 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:48.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.874+0000 7f318e031640 1 --2- 172.21.15.32:0/1963525306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f318815c5b0 0x7f3188152350 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:48.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.874+0000 7f318e031640 1 --2- 172.21.15.32:0/1963525306 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3188072cf0 0x7f318814ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:48.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.874+0000 7f318e031640 1 --2- 172.21.15.32:0/1963525306 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3188071770 0x7f3188150a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:48.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.874+0000 7f318e031640 1 -- 172.21.15.32:0/1963525306 >> 172.21.15.32:0/1963525306 conn(0x7f318806b110 msgr2=0x7f31880b03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:48.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.874+0000 7f318e031640 1 -- 172.21.15.32:0/1963525306 shutdown_connections 2024-04-24T23:04:48.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.874+0000 7f318e031640 1 -- 172.21.15.32:0/1963525306 wait complete. 2024-04-24T23:04:48.887 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_barrier' 2024-04-24T23:04:48.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.986+0000 7f7219790640 1 Processor -- start 2024-04-24T23:04:48.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.986+0000 7f7219790640 1 -- start start 2024-04-24T23:04:48.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.987+0000 7f7219790640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f721415d730 0x7f7214167910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:48.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.987+0000 7f7219790640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72140b4ee0 con 0x7f72140d05b0 2024-04-24T23:04:48.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.987+0000 7f7219790640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72140b51c0 con 0x7f721415cc40 2024-04-24T23:04:48.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.987+0000 7f7219790640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f721406cca0 con 0x7f721415d730 2024-04-24T23:04:48.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.987+0000 7f72127fc640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f72140d05b0 0x7f72141641e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60888/0 (socket says 172.21.15.32:60888) 2024-04-24T23:04:48.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.987+0000 7f72127fc640 1 -- 172.21.15.32:0/2564233588 learned_addr learned my addr 172.21.15.32:0/2564233588 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:04:48.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.987+0000 7f72137fe640 1 --2- 172.21.15.32:0/2564233588 >> v2:172.21.15.121:3300/0 conn(0x7f721415d730 0x7f7214167910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:48.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.987+0000 7f7211ffb640 1 -- 172.21.15.32:0/2564233588 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1251058377 0 0) 0x7f72140b4ee0 con 0x7f72140d05b0 2024-04-24T23:04:48.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.987+0000 7f7211ffb640 1 -- 172.21.15.32:0/2564233588 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f71fc003520 con 0x7f72140d05b0 2024-04-24T23:04:48.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.987+0000 7f72137fe640 1 -- 172.21.15.32:0/2564233588 >> v1:172.21.15.121:6789/0 conn(0x7f721415cc40 legacy=0x7f721415d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:48.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.987+0000 7f7211ffb640 1 -- 172.21.15.32:0/2564233588 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 174632417 0 0) 0x7f72140b51c0 con 0x7f721415cc40 2024-04-24T23:04:48.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.987+0000 7f72137fe640 1 -- 172.21.15.32:0/2564233588 >> v1:172.21.15.32:6789/0 conn(0x7f72140d05b0 legacy=0x7f72141641e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:48.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.987+0000 7f72137fe640 1 -- 172.21.15.32:0/2564233588 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72140b54a0 con 0x7f721415d730 2024-04-24T23:04:48.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.988+0000 7f72137fe640 1 --2- 172.21.15.32:0/2564233588 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f721415d730 0x7f7214167910 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f7204009d90 tx=0x7f720402a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e0e0f3481d9fbcf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:48.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.988+0000 7f7211ffb640 1 -- 172.21.15.32:0/2564233588 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7204007740 con 0x7f721415d730 2024-04-24T23:04:48.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.988+0000 7f7211ffb640 1 -- 172.21.15.32:0/2564233588 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72040078e0 con 0x7f721415d730 2024-04-24T23:04:48.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.988+0000 7f7211ffb640 1 -- 172.21.15.32:0/2564233588 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7204007cb0 con 0x7f721415d730 2024-04-24T23:04:48.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.989+0000 7f7219790640 1 -- 172.21.15.32:0/2564233588 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f721415d730 msgr2=0x7f7214167910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:48.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.989+0000 7f7219790640 1 --2- 172.21.15.32:0/2564233588 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f721415d730 0x7f7214167910 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f7204009d90 tx=0x7f720402a6f0 comp rx=0 tx=0).stop 2024-04-24T23:04:48.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.989+0000 7f7219790640 1 -- 172.21.15.32:0/2564233588 shutdown_connections 2024-04-24T23:04:48.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.989+0000 7f7219790640 1 --2- 172.21.15.32:0/2564233588 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f721415d730 0x7f7214167910 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:48.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.989+0000 7f7219790640 1 -- 172.21.15.32:0/2564233588 >> 172.21.15.32:0/2564233588 conn(0x7f72140d0ae0 msgr2=0x7f72140d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:48.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.989+0000 7f7219790640 1 -- 172.21.15.32:0/2564233588 shutdown_connections 2024-04-24T23:04:48.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.989+0000 7f7219790640 1 -- 172.21.15.32:0/2564233588 wait complete. 2024-04-24T23:04:48.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.990+0000 7f7219790640 1 Processor -- start 2024-04-24T23:04:48.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.990+0000 7f7219790640 1 -- start start 2024-04-24T23:04:48.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.991+0000 7f7219790640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f72140d05b0 0x7f7214153560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:48.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.991+0000 7f7219790640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f721415cc40 0x7f72141e7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:48.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.991+0000 7f7219790640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f721415d730 0x7f7214158140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:48.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.991+0000 7f7219790640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f72140b5a60 con 0x7f721415cc40 2024-04-24T23:04:48.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.991+0000 7f7219790640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f72140ce5f0 con 0x7f72140d05b0 2024-04-24T23:04:48.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.991+0000 7f7219790640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f7214067940 con 0x7f721415d730 2024-04-24T23:04:48.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.991+0000 7f72127fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f721415cc40 0x7f72141e7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:48.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.991+0000 7f72127fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f721415cc40 0x7f72141e7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47186/0 (socket says 172.21.15.32:47186) 2024-04-24T23:04:48.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.991+0000 7f72127fc640 1 -- 172.21.15.32:0/3004809623 learned_addr learned my addr 172.21.15.32:0/3004809623 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:48.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.991+0000 7f7212ffd640 1 --2- 172.21.15.32:0/3004809623 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f72140d05b0 0x7f7214153560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:48.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.991+0000 7f72137fe640 1 --2- 172.21.15.32:0/3004809623 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f721415d730 0x7f7214158140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:48.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.992+0000 7f72127fc640 1 -- 172.21.15.32:0/3004809623 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f721415d730 msgr2=0x7f7214158140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:48.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.992+0000 7f72127fc640 1 --2- 172.21.15.32:0/3004809623 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f721415d730 0x7f7214158140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:48.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.992+0000 7f72127fc640 1 -- 172.21.15.32:0/3004809623 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f72140d05b0 msgr2=0x7f7214153560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:48.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.992+0000 7f72127fc640 1 --2- 172.21.15.32:0/3004809623 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f72140d05b0 0x7f7214153560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:48.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.992+0000 7f72127fc640 1 -- 172.21.15.32:0/3004809623 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72141589c0 con 0x7f721415cc40 2024-04-24T23:04:48.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.992+0000 7f7212ffd640 1 --2- 172.21.15.32:0/3004809623 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f72140d05b0 0x7f7214153560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:04:48.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.992+0000 7f72127fc640 1 --2- 172.21.15.32:0/3004809623 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f721415cc40 0x7f72141e7220 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f7208001eb0 tx=0x7f720800c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:48.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.992+0000 7f71fb7fe640 1 -- 172.21.15.32:0/3004809623 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7208017070 con 0x7f721415cc40 2024-04-24T23:04:48.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.992+0000 7f71fb7fe640 1 -- 172.21.15.32:0/3004809623 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7208005d00 con 0x7f721415cc40 2024-04-24T23:04:48.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.992+0000 7f7219790640 1 -- 172.21.15.32:0/3004809623 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f721415b910 con 0x7f721415cc40 2024-04-24T23:04:48.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.992+0000 7f71fb7fe640 1 -- 172.21.15.32:0/3004809623 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7208007310 con 0x7f721415cc40 2024-04-24T23:04:48.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.992+0000 7f7219790640 1 -- 172.21.15.32:0/3004809623 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f721415bed0 con 0x7f721415cc40 2024-04-24T23:04:48.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.993+0000 7f7219790640 1 -- 172.21.15.32:0/3004809623 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7214068430 con 0x7f721415cc40 2024-04-24T23:04:48.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.993+0000 7f71fb7fe640 1 -- 172.21.15.32:0/3004809623 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f720801a070 con 0x7f721415cc40 2024-04-24T23:04:48.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.994+0000 7f71fb7fe640 1 --2- 172.21.15.32:0/3004809623 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f720006be30 0x7f720006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:48.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.994+0000 7f71fb7fe640 1 -- 172.21.15.32:0/3004809623 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 4943+0+0 (secure 0 0 0) 0x7f7208089f80 con 0x7f721415cc40 2024-04-24T23:04:48.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.994+0000 7f7212ffd640 1 --2- 172.21.15.32:0/3004809623 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f720006be30 0x7f720006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:48.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.995+0000 7f7212ffd640 1 --2- 172.21.15.32:0/3004809623 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f720006be30 0x7f720006e2f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f720c000f60 tx=0x7f720c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:49.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:48.998+0000 7f71fb7fe640 1 -- 172.21.15.32:0/3004809623 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7208058c30 con 0x7f721415cc40 2024-04-24T23:04:49.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:49.250+0000 7f7219790640 1 -- 172.21.15.32:0/3004809623 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]} v 0) -- 0x7f72140d1600 con 0x7f721415cc40 2024-04-24T23:04:50.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.258+0000 7f71fb7fe640 1 -- 172.21.15.32:0/3004809623 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]}]=0 v302) ==== 131+0+0 (secure 0 0 0) 0x7f7208058530 con 0x7f721415cc40 2024-04-24T23:04:50.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.260+0000 7f7219790640 1 -- 172.21.15.32:0/3004809623 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f720006be30 msgr2=0x7f720006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:50.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.261+0000 7f7219790640 1 --2- 172.21.15.32:0/3004809623 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f720006be30 0x7f720006e2f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f720c000f60 tx=0x7f720c009040 comp rx=0 tx=0).stop 2024-04-24T23:04:50.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.261+0000 7f7219790640 1 -- 172.21.15.32:0/3004809623 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f721415cc40 msgr2=0x7f72141e7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:50.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.261+0000 7f7219790640 1 --2- 172.21.15.32:0/3004809623 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f721415cc40 0x7f72141e7220 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f7208001eb0 tx=0x7f720800c820 comp rx=0 tx=0).stop 2024-04-24T23:04:50.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.261+0000 7f7219790640 1 -- 172.21.15.32:0/3004809623 shutdown_connections 2024-04-24T23:04:50.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.261+0000 7f7219790640 1 --2- 172.21.15.32:0/3004809623 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f720006be30 0x7f720006e2f0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:50.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.261+0000 7f7219790640 1 --2- 172.21.15.32:0/3004809623 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f721415cc40 0x7f72141e7220 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:50.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.261+0000 7f7219790640 1 --2- 172.21.15.32:0/3004809623 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f721415d730 0x7f7214158140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:50.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.261+0000 7f7219790640 1 --2- 172.21.15.32:0/3004809623 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f72140d05b0 0x7f7214153560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:50.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.261+0000 7f7219790640 1 -- 172.21.15.32:0/3004809623 >> 172.21.15.32:0/3004809623 conn(0x7f72140d0ae0 msgr2=0x7f72140d0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:50.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.261+0000 7f7219790640 1 -- 172.21.15.32:0/3004809623 shutdown_connections 2024-04-24T23:04:50.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.261+0000 7f7219790640 1 -- 172.21.15.32:0/3004809623 wait complete. 2024-04-24T23:04:50.276 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-24T23:04:50.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.384+0000 7fe6a8f42640 1 Processor -- start 2024-04-24T23:04:50.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.384+0000 7fe6a8f42640 1 -- start start 2024-04-24T23:04:50.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.385+0000 7fe6a8f42640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe6a406cd00 0x7fe6a40ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:50.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.385+0000 7fe6a8f42640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe6a415cb00 0x7fe6a40ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:50.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.385+0000 7fe6a8f42640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6a40b4520 con 0x7fe6a415b380 2024-04-24T23:04:50.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.385+0000 7fe6a8f42640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fe6a40b4800 con 0x7fe6a415cb00 2024-04-24T23:04:50.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.385+0000 7fe6a8f42640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fe6a40b4ae0 con 0x7fe6a406cd00 2024-04-24T23:04:50.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.385+0000 7fe6a2d9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe6a415cb00 0x7fe6a40ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:50.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.385+0000 7fe6a259c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe6a415b380 0x7fe6a406c5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49454/0 (socket says 172.21.15.32:49454) 2024-04-24T23:04:50.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.385+0000 7fe6a259c640 1 -- 172.21.15.32:0/3328884406 learned_addr learned my addr 172.21.15.32:0/3328884406 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:04:50.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.385+0000 7fe6a1d9b640 1 --2- 172.21.15.32:0/3328884406 >> v2:172.21.15.121:3300/0 conn(0x7fe6a406cd00 0x7fe6a40ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:50.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.386+0000 7fe6a2d9d640 1 -- 172.21.15.32:0/3328884406 >> v2:172.21.15.121:3300/0 conn(0x7fe6a406cd00 msgr2=0x7fe6a40ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:50.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.386+0000 7fe6a2d9d640 1 --2- 172.21.15.32:0/3328884406 >> v2:172.21.15.121:3300/0 conn(0x7fe6a406cd00 0x7fe6a40ce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:50.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.386+0000 7fe6a2d9d640 1 -- 172.21.15.32:0/3328884406 >> v1:172.21.15.32:6789/0 conn(0x7fe6a415b380 legacy=0x7fe6a406c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:50.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.386+0000 7fe6a2d9d640 1 -- 172.21.15.32:0/3328884406 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6a40b4dc0 con 0x7fe6a415cb00 2024-04-24T23:04:50.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.386+0000 7fe6a2d9d640 1 --2- 172.21.15.32:0/3328884406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6a415cb00 0x7fe6a40ce970 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fe6a40b33c0 tx=0x7fe69c02c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cba90dc6e0ee31eb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:50.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.386+0000 7fe6a159a640 1 -- 172.21.15.32:0/3328884406 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69c009540 con 0x7fe6a415cb00 2024-04-24T23:04:50.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.387+0000 7fe6a159a640 1 -- 172.21.15.32:0/3328884406 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe69c0096e0 con 0x7fe6a415cb00 2024-04-24T23:04:50.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.387+0000 7fe6a159a640 1 -- 172.21.15.32:0/3328884406 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69c009ab0 con 0x7fe6a415cb00 2024-04-24T23:04:50.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.387+0000 7fe6a8f42640 1 -- 172.21.15.32:0/3328884406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6a415cb00 msgr2=0x7fe6a40ce970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:50.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.387+0000 7fe6a8f42640 1 --2- 172.21.15.32:0/3328884406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6a415cb00 0x7fe6a40ce970 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fe6a40b33c0 tx=0x7fe69c02c7f0 comp rx=0 tx=0).stop 2024-04-24T23:04:50.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.387+0000 7fe6a8f42640 1 -- 172.21.15.32:0/3328884406 shutdown_connections 2024-04-24T23:04:50.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.387+0000 7fe6a8f42640 1 --2- 172.21.15.32:0/3328884406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6a415cb00 0x7fe6a40ce970 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:50.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.387+0000 7fe6a8f42640 1 --2- 172.21.15.32:0/3328884406 >> v2:172.21.15.121:3300/0 conn(0x7fe6a406cd00 0x7fe6a40ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:50.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.387+0000 7fe6a8f42640 1 -- 172.21.15.32:0/3328884406 >> 172.21.15.32:0/3328884406 conn(0x7fe6a406b110 msgr2=0x7fe6a406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:50.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.387+0000 7fe6a8f42640 1 -- 172.21.15.32:0/3328884406 shutdown_connections 2024-04-24T23:04:50.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.387+0000 7fe6a8f42640 1 -- 172.21.15.32:0/3328884406 wait complete. 2024-04-24T23:04:50.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.388+0000 7fe6a8f42640 1 Processor -- start 2024-04-24T23:04:50.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.388+0000 7fe6a8f42640 1 -- start start 2024-04-24T23:04:50.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.389+0000 7fe6a8f42640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6a406cd00 0x7fe6a414c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:50.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.389+0000 7fe6a8f42640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6a415b380 0x7fe6a414ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:50.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.389+0000 7fe6a8f42640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6a415cb00 0x7fe6a41f0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:50.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.389+0000 7fe6a8f42640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a415b760 con 0x7fe6a406cd00 2024-04-24T23:04:50.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.389+0000 7fe6a8f42640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a40ceeb0 con 0x7fe6a415cb00 2024-04-24T23:04:50.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.389+0000 7fe6a8f42640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe6a40b55c0 con 0x7fe6a415b380 2024-04-24T23:04:50.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.389+0000 7fe6a259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6a406cd00 0x7fe6a414c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:50.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.389+0000 7fe6a259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6a406cd00 0x7fe6a414c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56876/0 (socket says 172.21.15.32:56876) 2024-04-24T23:04:50.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.389+0000 7fe6a259c640 1 -- 172.21.15.32:0/1218003968 learned_addr learned my addr 172.21.15.32:0/1218003968 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:50.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.390+0000 7fe6a2d9d640 1 --2- 172.21.15.32:0/1218003968 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6a415cb00 0x7fe6a41f0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:50.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.390+0000 7fe6a259c640 1 -- 172.21.15.32:0/1218003968 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6a415b380 msgr2=0x7fe6a414ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:50.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.390+0000 7fe6a259c640 1 --2- 172.21.15.32:0/1218003968 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6a415b380 0x7fe6a414ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:50.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.390+0000 7fe6a259c640 1 -- 172.21.15.32:0/1218003968 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6a415cb00 msgr2=0x7fe6a41f0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:50.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.390+0000 7fe6a259c640 1 --2- 172.21.15.32:0/1218003968 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6a415cb00 0x7fe6a41f0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:50.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.390+0000 7fe6a259c640 1 -- 172.21.15.32:0/1218003968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6a41531c0 con 0x7fe6a406cd00 2024-04-24T23:04:50.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.390+0000 7fe6a259c640 1 --2- 172.21.15.32:0/1218003968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6a406cd00 0x7fe6a414c920 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fe69400c400 tx=0x7fe69400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:50.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.390+0000 7fe692ffd640 1 -- 172.21.15.32:0/1218003968 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe694017070 con 0x7fe6a406cd00 2024-04-24T23:04:50.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.390+0000 7fe692ffd640 1 -- 172.21.15.32:0/1218003968 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe694005ce0 con 0x7fe6a406cd00 2024-04-24T23:04:50.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.390+0000 7fe6a8f42640 1 -- 172.21.15.32:0/1218003968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6a41534a0 con 0x7fe6a406cd00 2024-04-24T23:04:50.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.390+0000 7fe692ffd640 1 -- 172.21.15.32:0/1218003968 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6940072f0 con 0x7fe6a406cd00 2024-04-24T23:04:50.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.390+0000 7fe6a8f42640 1 -- 172.21.15.32:0/1218003968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6a4153a60 con 0x7fe6a406cd00 2024-04-24T23:04:50.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.391+0000 7fe6a8f42640 1 -- 172.21.15.32:0/1218003968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6a41540b0 con 0x7fe6a406cd00 2024-04-24T23:04:50.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.391+0000 7fe692ffd640 1 -- 172.21.15.32:0/1218003968 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe69401a070 con 0x7fe6a406cd00 2024-04-24T23:04:50.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.392+0000 7fe692ffd640 1 --2- 172.21.15.32:0/1218003968 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe68406be30 0x7fe68406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:50.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.392+0000 7fe692ffd640 1 -- 172.21.15.32:0/1218003968 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 4943+0+0 (secure 0 0 0) 0x7fe694089f00 con 0x7fe6a406cd00 2024-04-24T23:04:50.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.392+0000 7fe6a1d9b640 1 --2- 172.21.15.32:0/1218003968 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe68406be30 0x7fe68406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:50.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.393+0000 7fe6a1d9b640 1 --2- 172.21.15.32:0/1218003968 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe68406be30 0x7fe68406e2f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe6a414b850 tx=0x7fe698008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:50.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.396+0000 7fe692ffd640 1 -- 172.21.15.32:0/1218003968 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe694058c40 con 0x7fe6a406cd00 2024-04-24T23:04:50.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.578+0000 7fe6a8f42640 1 -- 172.21.15.32:0/1218003968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fe6a4068430 con 0x7fe6a406cd00 2024-04-24T23:04:50.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.582+0000 7fe692ffd640 1 -- 172.21.15.32:0/1218003968 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7fe694058540 con 0x7fe6a406cd00 2024-04-24T23:04:50.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.584+0000 7fe6a8f42640 1 -- 172.21.15.32:0/1218003968 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe68406be30 msgr2=0x7fe68406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:50.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.584+0000 7fe6a8f42640 1 --2- 172.21.15.32:0/1218003968 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe68406be30 0x7fe68406e2f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe6a414b850 tx=0x7fe698008040 comp rx=0 tx=0).stop 2024-04-24T23:04:50.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.584+0000 7fe6a8f42640 1 -- 172.21.15.32:0/1218003968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6a406cd00 msgr2=0x7fe6a414c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:50.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.584+0000 7fe6a8f42640 1 --2- 172.21.15.32:0/1218003968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6a406cd00 0x7fe6a414c920 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fe69400c400 tx=0x7fe69400c8c0 comp rx=0 tx=0).stop 2024-04-24T23:04:50.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.584+0000 7fe6a8f42640 1 -- 172.21.15.32:0/1218003968 shutdown_connections 2024-04-24T23:04:50.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.584+0000 7fe6a8f42640 1 --2- 172.21.15.32:0/1218003968 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe68406be30 0x7fe68406e2f0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:50.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.584+0000 7fe6a8f42640 1 --2- 172.21.15.32:0/1218003968 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6a415cb00 0x7fe6a41f0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:50.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.584+0000 7fe6a8f42640 1 --2- 172.21.15.32:0/1218003968 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6a415b380 0x7fe6a414ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:50.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.584+0000 7fe6a8f42640 1 --2- 172.21.15.32:0/1218003968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6a406cd00 0x7fe6a414c920 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:50.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.584+0000 7fe6a8f42640 1 -- 172.21.15.32:0/1218003968 >> 172.21.15.32:0/1218003968 conn(0x7fe6a406b110 msgr2=0x7fe6a415f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:50.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.584+0000 7fe6a8f42640 1 -- 172.21.15.32:0/1218003968 shutdown_connections 2024-04-24T23:04:50.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.584+0000 7fe6a8f42640 1 -- 172.21.15.32:0/1218003968 wait complete. 2024-04-24T23:04:50.596 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-24T23:04:50.596 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:04:50.648 DEBUG:teuthology.orchestra.run.smithi121:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:04:50.699 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-24T23:04:50.699 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:04:50.699 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:04:50.715 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=413 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T23:04:50.715 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:04:50.715 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:04:50.732 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3271 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T23:04:50.732 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:04:50.732 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:04:50.771 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=413 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T23:04:50.771 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:04:50.771 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:04:50.789 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3271 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T23:04:50.789 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-24T23:04:50.789 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:04:50.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.910+0000 7ffb2951d640 1 Processor -- start 2024-04-24T23:04:50.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.910+0000 7ffb2951d640 1 -- start start 2024-04-24T23:04:50.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.911+0000 7ffb2951d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb2415e9b0 0x7ffb240d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:50.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.911+0000 7ffb2951d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffb2406b820 0x7ffb240b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:50.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.911+0000 7ffb2951d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb240d3170 con 0x7ffb240d4210 2024-04-24T23:04:50.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.911+0000 7ffb2951d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7ffb240d3450 con 0x7ffb2406b820 2024-04-24T23:04:50.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.911+0000 7ffb2951d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ffb240d3730 con 0x7ffb2415e9b0 2024-04-24T23:04:50.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.911+0000 7ffb237fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffb2406b820 0x7ffb240b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:50.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.911+0000 7ffb227fc640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ffb240d4210 0x7ffb2406b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49456/0 (socket says 172.21.15.32:49456) 2024-04-24T23:04:50.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.911+0000 7ffb227fc640 1 -- 172.21.15.32:0/1662804690 learned_addr learned my addr 172.21.15.32:0/1662804690 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:04:50.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.911+0000 7ffb237fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffb2406b820 0x7ffb240b78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56890/0 (socket says 172.21.15.32:56890) 2024-04-24T23:04:50.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.912+0000 7ffb22ffd640 1 --2- 172.21.15.32:0/1662804690 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb2415e9b0 0x7ffb240d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:50.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.912+0000 7ffb237fe640 1 -- 172.21.15.32:0/1662804690 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb2415e9b0 msgr2=0x7ffb240d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:50.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.912+0000 7ffb237fe640 1 --2- 172.21.15.32:0/1662804690 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb2415e9b0 0x7ffb240d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:50.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.912+0000 7ffb237fe640 1 -- 172.21.15.32:0/1662804690 >> v1:172.21.15.32:6789/0 conn(0x7ffb240d4210 legacy=0x7ffb2406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:50.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.912+0000 7ffb237fe640 1 -- 172.21.15.32:0/1662804690 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb240d3a10 con 0x7ffb2406b820 2024-04-24T23:04:50.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.912+0000 7ffb21ffb640 1 -- 172.21.15.32:0/1662804690 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3524107653 0 0) 0x7ffb240d3170 con 0x7ffb240d4210 2024-04-24T23:04:50.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.912+0000 7ffb237fe640 1 --2- 172.21.15.32:0/1662804690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb2406b820 0x7ffb240b78f0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7ffb14009d90 tx=0x7ffb1402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9158d9dbfb2cee3a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:50.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.912+0000 7ffb21ffb640 1 -- 172.21.15.32:0/1662804690 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb14007740 con 0x7ffb2406b820 2024-04-24T23:04:50.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.913+0000 7ffb21ffb640 1 -- 172.21.15.32:0/1662804690 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb140078e0 con 0x7ffb2406b820 2024-04-24T23:04:50.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.913+0000 7ffb21ffb640 1 -- 172.21.15.32:0/1662804690 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb14007cb0 con 0x7ffb2406b820 2024-04-24T23:04:50.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.913+0000 7ffb2951d640 1 -- 172.21.15.32:0/1662804690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb2406b820 msgr2=0x7ffb240b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:50.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.913+0000 7ffb2951d640 1 --2- 172.21.15.32:0/1662804690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb2406b820 0x7ffb240b78f0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7ffb14009d90 tx=0x7ffb1402a660 comp rx=0 tx=0).stop 2024-04-24T23:04:50.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.913+0000 7ffb2951d640 1 -- 172.21.15.32:0/1662804690 shutdown_connections 2024-04-24T23:04:50.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.913+0000 7ffb2951d640 1 --2- 172.21.15.32:0/1662804690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb2406b820 0x7ffb240b78f0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:50.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.913+0000 7ffb2951d640 1 --2- 172.21.15.32:0/1662804690 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb2415e9b0 0x7ffb240d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:50.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.913+0000 7ffb2951d640 1 -- 172.21.15.32:0/1662804690 >> 172.21.15.32:0/1662804690 conn(0x7ffb240d11d0 msgr2=0x7ffb240d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:50.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.913+0000 7ffb2951d640 1 -- 172.21.15.32:0/1662804690 shutdown_connections 2024-04-24T23:04:50.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.913+0000 7ffb2951d640 1 -- 172.21.15.32:0/1662804690 wait complete. 2024-04-24T23:04:50.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.914+0000 7ffb2951d640 1 Processor -- start 2024-04-24T23:04:50.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.914+0000 7ffb2951d640 1 -- start start 2024-04-24T23:04:50.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.915+0000 7ffb2951d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb2406b820 0x7ffb241522a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:50.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.915+0000 7ffb2951d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb240d4210 0x7ffb240cc730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:50.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.915+0000 7ffb2951d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb2415e9b0 0x7ffb240ccc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:50.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.915+0000 7ffb2951d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ffb240ba0e0 con 0x7ffb240d4210 2024-04-24T23:04:50.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.915+0000 7ffb2951d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ffb240d2400 con 0x7ffb2415e9b0 2024-04-24T23:04:50.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.915+0000 7ffb2951d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ffb240ba260 con 0x7ffb2406b820 2024-04-24T23:04:50.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.915+0000 7ffb227fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb240d4210 0x7ffb240cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:50.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.915+0000 7ffb227fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb240d4210 0x7ffb240cc730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56902/0 (socket says 172.21.15.32:56902) 2024-04-24T23:04:50.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.915+0000 7ffb227fc640 1 -- 172.21.15.32:0/2053827244 learned_addr learned my addr 172.21.15.32:0/2053827244 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:50.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.915+0000 7ffb22ffd640 1 --2- 172.21.15.32:0/2053827244 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb2406b820 0x7ffb241522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:50.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.916+0000 7ffb237fe640 1 --2- 172.21.15.32:0/2053827244 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb2415e9b0 0x7ffb240ccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:50.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.916+0000 7ffb227fc640 1 -- 172.21.15.32:0/2053827244 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb2406b820 msgr2=0x7ffb241522a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:50.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.916+0000 7ffb227fc640 1 --2- 172.21.15.32:0/2053827244 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb2406b820 0x7ffb241522a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:50.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.916+0000 7ffb227fc640 1 -- 172.21.15.32:0/2053827244 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb2415e9b0 msgr2=0x7ffb240ccc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:50.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.916+0000 7ffb227fc640 1 --2- 172.21.15.32:0/2053827244 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb2415e9b0 0x7ffb240ccc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:50.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.916+0000 7ffb227fc640 1 -- 172.21.15.32:0/2053827244 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb240cd2d0 con 0x7ffb240d4210 2024-04-24T23:04:50.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.916+0000 7ffb237fe640 1 --2- 172.21.15.32:0/2053827244 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb2415e9b0 0x7ffb240ccc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:04:50.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.916+0000 7ffb227fc640 1 --2- 172.21.15.32:0/2053827244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb240d4210 0x7ffb240cc730 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7ffb1800c4d0 tx=0x7ffb1800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:50.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.916+0000 7ffb0b7fe640 1 -- 172.21.15.32:0/2053827244 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb18018070 con 0x7ffb240d4210 2024-04-24T23:04:50.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.916+0000 7ffb0b7fe640 1 -- 172.21.15.32:0/2053827244 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb18007500 con 0x7ffb240d4210 2024-04-24T23:04:50.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.916+0000 7ffb2951d640 1 -- 172.21.15.32:0/2053827244 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb240cd5b0 con 0x7ffb240d4210 2024-04-24T23:04:50.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.916+0000 7ffb0b7fe640 1 -- 172.21.15.32:0/2053827244 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb18004a20 con 0x7ffb240d4210 2024-04-24T23:04:50.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.916+0000 7ffb2951d640 1 -- 172.21.15.32:0/2053827244 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb240c8420 con 0x7ffb240d4210 2024-04-24T23:04:50.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.917+0000 7ffb2951d640 1 -- 172.21.15.32:0/2053827244 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb24068430 con 0x7ffb240d4210 2024-04-24T23:04:50.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.918+0000 7ffb0b7fe640 1 -- 172.21.15.32:0/2053827244 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffb1801b070 con 0x7ffb240d4210 2024-04-24T23:04:50.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.918+0000 7ffb0b7fe640 1 --2- 172.21.15.32:0/2053827244 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ffb1006be30 0x7ffb1006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:50.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.918+0000 7ffb0b7fe640 1 -- 172.21.15.32:0/2053827244 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 4943+0+0 (secure 0 0 0) 0x7ffb18089f80 con 0x7ffb240d4210 2024-04-24T23:04:50.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.918+0000 7ffb22ffd640 1 --2- 172.21.15.32:0/2053827244 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ffb1006be30 0x7ffb1006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:50.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.919+0000 7ffb22ffd640 1 --2- 172.21.15.32:0/2053827244 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ffb1006be30 0x7ffb1006e2f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c006fd0 tx=0x7ffb1c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:50.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:50.922+0000 7ffb0b7fe640 1 -- 172.21.15.32:0/2053827244 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ffb18058b90 con 0x7ffb240d4210 2024-04-24T23:04:51.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.142+0000 7ffb2951d640 1 -- 172.21.15.32:0/2053827244 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffb240d2e30 con 0x7ffb240d4210 2024-04-24T23:04:51.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.143+0000 7ffb0b7fe640 1 -- 172.21.15.32:0/2053827244 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+3869 (secure 0 0 0) 0x7ffb18058490 con 0x7ffb240d4210 2024-04-24T23:04:51.144 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:04:51.144 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":46,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6175,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/434273738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":434273738},{"type":"v1","addr":"172.21.15.121:6836","nonce":434273738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6178,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1104557785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1104557785},{"type":"v1","addr":"172.21.15.121:6837","nonce":1104557785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6888,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/3137388886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3137388886},{"type":"v1","addr":"172.21.15.32:6834","nonce":3137388886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":7272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/419234983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":419234983},{"type":"v1","addr":"172.21.15.32:6835","nonce":419234983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46}],"filesystems":[]} 2024-04-24T23:04:51.145 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 46 2024-04-24T23:04:51.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.145+0000 7ffb2951d640 1 -- 172.21.15.32:0/2053827244 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ffb1006be30 msgr2=0x7ffb1006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:51.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.145+0000 7ffb2951d640 1 --2- 172.21.15.32:0/2053827244 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ffb1006be30 0x7ffb1006e2f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c006fd0 tx=0x7ffb1c008040 comp rx=0 tx=0).stop 2024-04-24T23:04:51.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.145+0000 7ffb2951d640 1 -- 172.21.15.32:0/2053827244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb240d4210 msgr2=0x7ffb240cc730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:51.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.145+0000 7ffb2951d640 1 --2- 172.21.15.32:0/2053827244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb240d4210 0x7ffb240cc730 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7ffb1800c4d0 tx=0x7ffb1800c990 comp rx=0 tx=0).stop 2024-04-24T23:04:51.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.145+0000 7ffb2951d640 1 -- 172.21.15.32:0/2053827244 shutdown_connections 2024-04-24T23:04:51.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.145+0000 7ffb2951d640 1 --2- 172.21.15.32:0/2053827244 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ffb1006be30 0x7ffb1006e2f0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:51.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.145+0000 7ffb2951d640 1 --2- 172.21.15.32:0/2053827244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb240d4210 0x7ffb240cc730 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:51.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.145+0000 7ffb2951d640 1 --2- 172.21.15.32:0/2053827244 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb2415e9b0 0x7ffb240ccc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:51.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.145+0000 7ffb2951d640 1 --2- 172.21.15.32:0/2053827244 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb2406b820 0x7ffb241522a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:51.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.145+0000 7ffb2951d640 1 -- 172.21.15.32:0/2053827244 >> 172.21.15.32:0/2053827244 conn(0x7ffb240d11d0 msgr2=0x7ffb240b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:51.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.145+0000 7ffb2951d640 1 -- 172.21.15.32:0/2053827244 shutdown_connections 2024-04-24T23:04:51.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.145+0000 7ffb2951d640 1 -- 172.21.15.32:0/2053827244 wait complete. 2024-04-24T23:04:51.157 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-24T23:04:51.157 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-24T23:04:51.158 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-24T23:04:51.158 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:04:51.158 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-24T23:04:51.158 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-24T23:04:51.158 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-24T23:04:51.158 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:04:51.158 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-24T23:04:51.159 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-24T23:04:51.159 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-24T23:04:51.159 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:04:51.159 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-24T23:04:51.159 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-24T23:04:51.159 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-24T23:04:51.159 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:04:51.167 INFO:tasks.ceph.mds.a:Stopped 2024-04-24T23:04:51.167 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-24T23:04:51.168 INFO:tasks.ceph.mds.c:Stopped 2024-04-24T23:04:51.168 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-24T23:04:51.170 INFO:tasks.ceph.mds.b:Stopped 2024-04-24T23:04:51.171 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-24T23:04:51.172 INFO:tasks.ceph.mds.d:Stopped 2024-04-24T23:04:51.172 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-24T23:04:51.209 INFO:tasks.ceph.mds.a:Started 2024-04-24T23:04:51.210 DEBUG:teuthology.parallel:result is None 2024-04-24T23:04:51.210 INFO:tasks.ceph.mds.c:Started 2024-04-24T23:04:51.211 DEBUG:teuthology.parallel:result is None 2024-04-24T23:04:51.214 INFO:tasks.ceph.mds.b:Started 2024-04-24T23:04:51.214 DEBUG:teuthology.parallel:result is None 2024-04-24T23:04:51.215 INFO:tasks.ceph.mds.d:Started 2024-04-24T23:04:51.216 DEBUG:teuthology.parallel:result is None 2024-04-24T23:04:51.216 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-24T23:04:51.286 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2024-04-24T23:04:51.287 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2024-04-24T23:04:51.293 INFO:tasks.ceph.mds.d.smithi121.stdout:starting mds.d at 2024-04-24T23:04:51.294 INFO:tasks.ceph.mds.b.smithi121.stdout:starting mds.b at 2024-04-24T23:04:51.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.321+0000 7ff3575ce640 1 Processor -- start 2024-04-24T23:04:51.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.321+0000 7ff3575ce640 1 -- start start 2024-04-24T23:04:51.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.322+0000 7ff3575ce640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff35006b820 0x7ff3500d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:51.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.322+0000 7ff3575ce640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff3500d4210 0x7ff3500b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:51.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.322+0000 7ff3575ce640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3500d3170 con 0x7ff35015e9b0 2024-04-24T23:04:51.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.322+0000 7ff3575ce640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7ff3500d3450 con 0x7ff3500d4210 2024-04-24T23:04:51.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.322+0000 7ff3575ce640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7ff3500d3730 con 0x7ff35006b820 2024-04-24T23:04:51.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.322+0000 7ff355b6b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff3500d4210 0x7ff3500b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:51.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.322+0000 7ff355b6b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff3500d4210 0x7ff3500b78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56940/0 (socket says 172.21.15.32:56940) 2024-04-24T23:04:51.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.322+0000 7ff355b6b640 1 -- 172.21.15.32:0/3304955558 learned_addr learned my addr 172.21.15.32:0/3304955558 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:51.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.322+0000 7ff35536a640 1 --1- 172.21.15.32:0/3304955558 >> v1:172.21.15.32:6789/0 conn(0x7ff35015e9b0 0x7ff35006b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49482/0 (socket says 172.21.15.32:49482) 2024-04-24T23:04:51.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.322+0000 7ff354b69640 1 --2- 172.21.15.32:0/3304955558 >> v2:172.21.15.121:3300/0 conn(0x7ff35006b820 0x7ff3500d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:51.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.322+0000 7ff355b6b640 1 -- 172.21.15.32:0/3304955558 >> v2:172.21.15.121:3300/0 conn(0x7ff35006b820 msgr2=0x7ff3500d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:51.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.322+0000 7ff355b6b640 1 --2- 172.21.15.32:0/3304955558 >> v2:172.21.15.121:3300/0 conn(0x7ff35006b820 0x7ff3500d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:51.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.322+0000 7ff355b6b640 1 -- 172.21.15.32:0/3304955558 >> v1:172.21.15.32:6789/0 conn(0x7ff35015e9b0 legacy=0x7ff35006b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:51.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.322+0000 7ff343fff640 1 -- 172.21.15.32:0/3304955558 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2924689500 0 0) 0x7ff3500d3170 con 0x7ff35015e9b0 2024-04-24T23:04:51.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.322+0000 7ff355b6b640 1 -- 172.21.15.32:0/3304955558 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3500d3a10 con 0x7ff3500d4210 2024-04-24T23:04:51.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.323+0000 7ff355b6b640 1 --2- 172.21.15.32:0/3304955558 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3500d4210 0x7ff3500b78f0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7ff3500730f0 tx=0x7ff34802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e92ee091647b1fe2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:51.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.323+0000 7ff343fff640 1 -- 172.21.15.32:0/3304955558 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff348036070 con 0x7ff3500d4210 2024-04-24T23:04:51.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.323+0000 7ff343fff640 1 -- 172.21.15.32:0/3304955558 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff348031600 con 0x7ff3500d4210 2024-04-24T23:04:51.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.323+0000 7ff343fff640 1 -- 172.21.15.32:0/3304955558 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3480319d0 con 0x7ff3500d4210 2024-04-24T23:04:51.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.323+0000 7ff3575ce640 1 -- 172.21.15.32:0/3304955558 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3500d4210 msgr2=0x7ff3500b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:51.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.323+0000 7ff3575ce640 1 --2- 172.21.15.32:0/3304955558 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3500d4210 0x7ff3500b78f0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7ff3500730f0 tx=0x7ff34802a6a0 comp rx=0 tx=0).stop 2024-04-24T23:04:51.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.323+0000 7ff3575ce640 1 -- 172.21.15.32:0/3304955558 shutdown_connections 2024-04-24T23:04:51.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.323+0000 7ff3575ce640 1 --2- 172.21.15.32:0/3304955558 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3500d4210 0x7ff3500b78f0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:51.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.323+0000 7ff3575ce640 1 --2- 172.21.15.32:0/3304955558 >> v2:172.21.15.121:3300/0 conn(0x7ff35006b820 0x7ff3500d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:51.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.323+0000 7ff3575ce640 1 -- 172.21.15.32:0/3304955558 >> 172.21.15.32:0/3304955558 conn(0x7ff3500d11d0 msgr2=0x7ff3500d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:51.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.324+0000 7ff3575ce640 1 -- 172.21.15.32:0/3304955558 shutdown_connections 2024-04-24T23:04:51.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.324+0000 7ff3575ce640 1 -- 172.21.15.32:0/3304955558 wait complete. 2024-04-24T23:04:51.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.325+0000 7ff3575ce640 1 Processor -- start 2024-04-24T23:04:51.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.325+0000 7ff3575ce640 1 -- start start 2024-04-24T23:04:51.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.325+0000 7ff3575ce640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff35006b820 0x7ff3501f7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:51.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.325+0000 7ff3575ce640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3500d4210 0x7ff35015ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:51.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.325+0000 7ff3575ce640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff35015e9b0 0x7ff35015b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:51.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.325+0000 7ff3575ce640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff3500ba0e0 con 0x7ff35006b820 2024-04-24T23:04:51.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.325+0000 7ff3575ce640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff3500d2400 con 0x7ff3500d4210 2024-04-24T23:04:51.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.325+0000 7ff3575ce640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff3500ba260 con 0x7ff35015e9b0 2024-04-24T23:04:51.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.326+0000 7ff35536a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff35006b820 0x7ff3501f7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:51.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.326+0000 7ff35536a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff35006b820 0x7ff3501f7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56956/0 (socket says 172.21.15.32:56956) 2024-04-24T23:04:51.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.326+0000 7ff35536a640 1 -- 172.21.15.32:0/4099065230 learned_addr learned my addr 172.21.15.32:0/4099065230 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:51.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.326+0000 7ff354b69640 1 --2- 172.21.15.32:0/4099065230 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3500d4210 0x7ff35015ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:51.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.326+0000 7ff355b6b640 1 --2- 172.21.15.32:0/4099065230 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff35015e9b0 0x7ff35015b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:51.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.326+0000 7ff35536a640 1 -- 172.21.15.32:0/4099065230 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff35015e9b0 msgr2=0x7ff35015b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:51.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.326+0000 7ff35536a640 1 --2- 172.21.15.32:0/4099065230 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff35015e9b0 0x7ff35015b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:51.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.326+0000 7ff35536a640 1 -- 172.21.15.32:0/4099065230 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3500d4210 msgr2=0x7ff35015ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:51.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.326+0000 7ff35536a640 1 --2- 172.21.15.32:0/4099065230 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3500d4210 0x7ff35015ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:51.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.326+0000 7ff35536a640 1 -- 172.21.15.32:0/4099065230 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3501e78d0 con 0x7ff35006b820 2024-04-24T23:04:51.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.326+0000 7ff354b69640 1 --2- 172.21.15.32:0/4099065230 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3500d4210 0x7ff35015ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:04:51.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.326+0000 7ff35536a640 1 --2- 172.21.15.32:0/4099065230 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff35006b820 0x7ff3501f7270 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7ff34c00c430 tx=0x7ff34c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:51.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.326+0000 7ff341ffb640 1 -- 172.21.15.32:0/4099065230 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff34c0074e0 con 0x7ff35006b820 2024-04-24T23:04:51.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.326+0000 7ff341ffb640 1 -- 172.21.15.32:0/4099065230 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff34c0047e0 con 0x7ff35006b820 2024-04-24T23:04:51.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.326+0000 7ff341ffb640 1 -- 172.21.15.32:0/4099065230 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff34c004bd0 con 0x7ff35006b820 2024-04-24T23:04:51.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.326+0000 7ff3575ce640 1 -- 172.21.15.32:0/4099065230 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3501e7bb0 con 0x7ff35006b820 2024-04-24T23:04:51.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.327+0000 7ff3575ce640 1 -- 172.21.15.32:0/4099065230 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3501e8170 con 0x7ff35006b820 2024-04-24T23:04:51.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.328+0000 7ff3575ce640 1 -- 172.21.15.32:0/4099065230 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff350068430 con 0x7ff35006b820 2024-04-24T23:04:51.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.328+0000 7ff341ffb640 1 -- 172.21.15.32:0/4099065230 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff34c011070 con 0x7ff35006b820 2024-04-24T23:04:51.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.329+0000 7ff341ffb640 1 --2- 172.21.15.32:0/4099065230 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff33406be60 0x7ff33406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:51.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.329+0000 7ff341ffb640 1 -- 172.21.15.32:0/4099065230 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 4943+0+0 (secure 0 0 0) 0x7ff34c08a7e0 con 0x7ff35006b820 2024-04-24T23:04:51.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.329+0000 7ff354b69640 1 --2- 172.21.15.32:0/4099065230 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff33406be60 0x7ff33406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:51.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.329+0000 7ff354b69640 1 --2- 172.21.15.32:0/4099065230 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff33406be60 0x7ff33406e320 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff3500b6210 tx=0x7ff344008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:51.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.333+0000 7ff341ffb640 1 -- 172.21.15.32:0/4099065230 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff34c059480 con 0x7ff35006b820 2024-04-24T23:04:51.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:51.529+0000 7ff3575ce640 1 -- 172.21.15.32:0/4099065230 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7ff35006b5e0 con 0x7ff35006b820 2024-04-24T23:04:52.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.610+0000 7ff341ffb640 1 -- 172.21.15.32:0/4099065230 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v121) ==== 97+0+0 (secure 0 0 0) 0x7ff34c058d80 con 0x7ff35006b820 2024-04-24T23:04:52.612 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blocklist entries 2024-04-24T23:04:52.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.613+0000 7ff3575ce640 1 -- 172.21.15.32:0/4099065230 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff33406be60 msgr2=0x7ff33406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:52.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.613+0000 7ff3575ce640 1 --2- 172.21.15.32:0/4099065230 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff33406be60 0x7ff33406e320 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff3500b6210 tx=0x7ff344008040 comp rx=0 tx=0).stop 2024-04-24T23:04:52.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.613+0000 7ff3575ce640 1 -- 172.21.15.32:0/4099065230 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff35006b820 msgr2=0x7ff3501f7270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:52.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.613+0000 7ff3575ce640 1 --2- 172.21.15.32:0/4099065230 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff35006b820 0x7ff3501f7270 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7ff34c00c430 tx=0x7ff34c00c8f0 comp rx=0 tx=0).stop 2024-04-24T23:04:52.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.613+0000 7ff3575ce640 1 -- 172.21.15.32:0/4099065230 shutdown_connections 2024-04-24T23:04:52.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.613+0000 7ff3575ce640 1 --2- 172.21.15.32:0/4099065230 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff33406be60 0x7ff33406e320 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:52.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.613+0000 7ff3575ce640 1 --2- 172.21.15.32:0/4099065230 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff35015e9b0 0x7ff35015b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:52.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.613+0000 7ff3575ce640 1 --2- 172.21.15.32:0/4099065230 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3500d4210 0x7ff35015ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:52.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.613+0000 7ff3575ce640 1 --2- 172.21.15.32:0/4099065230 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff35006b820 0x7ff3501f7270 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:52.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.613+0000 7ff3575ce640 1 -- 172.21.15.32:0/4099065230 >> 172.21.15.32:0/4099065230 conn(0x7ff3500d11d0 msgr2=0x7ff3500b78f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:52.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.614+0000 7ff3575ce640 1 -- 172.21.15.32:0/4099065230 shutdown_connections 2024-04-24T23:04:52.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.614+0000 7ff3575ce640 1 -- 172.21.15.32:0/4099065230 wait complete. 2024-04-24T23:04:52.628 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-24T23:04:52.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.734+0000 7f08a8009640 1 Processor -- start 2024-04-24T23:04:52.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.735+0000 7f08a8009640 1 -- start start 2024-04-24T23:04:52.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.735+0000 7f08a8009640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f08a00b44d0 0x7f08a00b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:52.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.735+0000 7f08a8009640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f08a00af890 0x7f08a015cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:52.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.735+0000 7f08a8009640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08a00b6430 con 0x7f08a006c5f0 2024-04-24T23:04:52.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.735+0000 7f08a8009640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f08a0156f80 con 0x7f08a00b44d0 2024-04-24T23:04:52.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.735+0000 7f08a8009640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f08a00b6150 con 0x7f08a00af890 2024-04-24T23:04:52.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.736+0000 7f08a55a4640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f08a00b44d0 0x7f08a00b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:52.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.736+0000 7f08a55a4640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f08a00b44d0 0x7f08a00b5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56958/0 (socket says 172.21.15.32:56958) 2024-04-24T23:04:52.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.736+0000 7f08a55a4640 1 -- 172.21.15.32:0/974760408 learned_addr learned my addr 172.21.15.32:0/974760408 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:52.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.736+0000 7f08a65a6640 1 --2- 172.21.15.32:0/974760408 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f08a00af890 0x7f08a015cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:52.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.736+0000 7f08a55a4640 1 -- 172.21.15.32:0/974760408 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f08a00af890 msgr2=0x7f08a015cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:52.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.736+0000 7f08a55a4640 1 --2- 172.21.15.32:0/974760408 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f08a00af890 0x7f08a015cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:52.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.736+0000 7f08a55a4640 1 -- 172.21.15.32:0/974760408 >> v1:172.21.15.121:6789/0 conn(0x7f08a006c5f0 legacy=0x7f08a006c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:52.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.736+0000 7f08a55a4640 1 -- 172.21.15.32:0/974760408 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08a00b39f0 con 0x7f08a00b44d0 2024-04-24T23:04:52.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.737+0000 7f08a55a4640 1 --2- 172.21.15.32:0/974760408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08a00b44d0 0x7f08a00b5890 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f0890009870 tx=0x7f089002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a5871c0bfc8bc98 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:52.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.737+0000 7f08a4da3640 1 -- 172.21.15.32:0/974760408 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0890037070 con 0x7f08a00b44d0 2024-04-24T23:04:52.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.737+0000 7f08a4da3640 1 -- 172.21.15.32:0/974760408 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0890032440 con 0x7f08a00b44d0 2024-04-24T23:04:52.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.737+0000 7f08a4da3640 1 -- 172.21.15.32:0/974760408 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0890032810 con 0x7f08a00b44d0 2024-04-24T23:04:52.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.737+0000 7f08a8009640 1 -- 172.21.15.32:0/974760408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08a00b44d0 msgr2=0x7f08a00b5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:52.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.737+0000 7f08a8009640 1 --2- 172.21.15.32:0/974760408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08a00b44d0 0x7f08a00b5890 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f0890009870 tx=0x7f089002a1c0 comp rx=0 tx=0).stop 2024-04-24T23:04:52.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.737+0000 7f08a8009640 1 -- 172.21.15.32:0/974760408 shutdown_connections 2024-04-24T23:04:52.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.738+0000 7f08a8009640 1 --2- 172.21.15.32:0/974760408 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f08a00af890 0x7f08a015cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:52.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.738+0000 7f08a8009640 1 --2- 172.21.15.32:0/974760408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08a00b44d0 0x7f08a00b5890 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:52.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.738+0000 7f08a8009640 1 -- 172.21.15.32:0/974760408 >> 172.21.15.32:0/974760408 conn(0x7f08a006b110 msgr2=0x7f08a006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:52.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.738+0000 7f08a8009640 1 -- 172.21.15.32:0/974760408 shutdown_connections 2024-04-24T23:04:52.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.738+0000 7f08a8009640 1 -- 172.21.15.32:0/974760408 wait complete. 2024-04-24T23:04:52.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.739+0000 7f08a8009640 1 Processor -- start 2024-04-24T23:04:52.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.739+0000 7f08a8009640 1 -- start start 2024-04-24T23:04:52.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.739+0000 7f08a8009640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f08a006c5f0 0x7f08a01f50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:52.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.739+0000 7f08a8009640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08a00af890 0x7f08a00aa5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:52.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.739+0000 7f08a8009640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08a00b44d0 0x7f08a01e6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:52.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.739+0000 7f08a8009640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f08a00b5dd0 con 0x7f08a00b44d0 2024-04-24T23:04:52.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.739+0000 7f08a8009640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f08a00b3fb0 con 0x7f08a00af890 2024-04-24T23:04:52.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.739+0000 7f08a8009640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f08a015d000 con 0x7f08a006c5f0 2024-04-24T23:04:52.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.740+0000 7f08a65a6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08a00b44d0 0x7f08a01e6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:52.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.740+0000 7f08a65a6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08a00b44d0 0x7f08a01e6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56974/0 (socket says 172.21.15.32:56974) 2024-04-24T23:04:52.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.740+0000 7f08a65a6640 1 -- 172.21.15.32:0/3899233766 learned_addr learned my addr 172.21.15.32:0/3899233766 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:52.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.740+0000 7f08a5da5640 1 --2- 172.21.15.32:0/3899233766 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f08a006c5f0 0x7f08a01f50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:52.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.740+0000 7f08a55a4640 1 --2- 172.21.15.32:0/3899233766 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08a00af890 0x7f08a00aa5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:52.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.740+0000 7f08a65a6640 1 -- 172.21.15.32:0/3899233766 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f08a006c5f0 msgr2=0x7f08a01f50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:52.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.740+0000 7f08a65a6640 1 --2- 172.21.15.32:0/3899233766 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f08a006c5f0 0x7f08a01f50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:52.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.740+0000 7f08a65a6640 1 -- 172.21.15.32:0/3899233766 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08a00af890 msgr2=0x7f08a00aa5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:52.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.740+0000 7f08a65a6640 1 --2- 172.21.15.32:0/3899233766 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08a00af890 0x7f08a00aa5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:52.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.740+0000 7f08a65a6640 1 -- 172.21.15.32:0/3899233766 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08a01e6e50 con 0x7f08a00b44d0 2024-04-24T23:04:52.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.740+0000 7f08a65a6640 1 --2- 172.21.15.32:0/3899233766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08a00b44d0 0x7f08a01e6910 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f0898009dd0 tx=0x7f089800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:52.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.740+0000 7f08967fc640 1 -- 172.21.15.32:0/3899233766 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0898018020 con 0x7f08a00b44d0 2024-04-24T23:04:52.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.740+0000 7f08967fc640 1 -- 172.21.15.32:0/3899233766 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0898014480 con 0x7f08a00b44d0 2024-04-24T23:04:52.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.740+0000 7f08a8009640 1 -- 172.21.15.32:0/3899233766 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08a01e70d0 con 0x7f08a00b44d0 2024-04-24T23:04:52.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.740+0000 7f08967fc640 1 -- 172.21.15.32:0/3899233766 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0898014870 con 0x7f08a00b44d0 2024-04-24T23:04:52.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.740+0000 7f08a8009640 1 -- 172.21.15.32:0/3899233766 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08a01e7690 con 0x7f08a00b44d0 2024-04-24T23:04:52.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.741+0000 7f08a8009640 1 -- 172.21.15.32:0/3899233766 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08a0068430 con 0x7f08a00b44d0 2024-04-24T23:04:52.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.742+0000 7f08967fc640 1 -- 172.21.15.32:0/3899233766 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0898005d10 con 0x7f08a00b44d0 2024-04-24T23:04:52.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.742+0000 7f08967fc640 1 --2- 172.21.15.32:0/3899233766 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f088406be30 0x7f088406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:52.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.743+0000 7f08967fc640 1 -- 172.21.15.32:0/3899233766 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 4857+0+0 (secure 0 0 0) 0x7f089808b4b0 con 0x7f08a00b44d0 2024-04-24T23:04:52.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.743+0000 7f08a5da5640 1 --2- 172.21.15.32:0/3899233766 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f088406be30 0x7f088406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:52.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.743+0000 7f08a5da5640 1 --2- 172.21.15.32:0/3899233766 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f088406be30 0x7f088406e2f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f089c000f60 tx=0x7f089c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:52.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.747+0000 7f08967fc640 1 -- 172.21.15.32:0/3899233766 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0898022070 con 0x7f08a00b44d0 2024-04-24T23:04:52.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.996+0000 7f08a8009640 1 -- 172.21.15.32:0/3899233766 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f08a00b5670 con 0x7f08a00b44d0 2024-04-24T23:04:52.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.997+0000 7f08967fc640 1 -- 172.21.15.32:0/3899233766 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) ==== 80+0+5304 (secure 0 0 0) 0x7f08a01e6e50 con 0x7f08a00b44d0 2024-04-24T23:04:52.999 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:04:52.999 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:04:52.999 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2024-04-24T23:04:52.999 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:52.999 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2024-04-24T23:04:52.999 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmXRdxAxAA/EYEn1C38XhG7DC+0X4y7A==", 2024-04-24T23:04:52.999 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:04:52.999 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:04:52.999 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-24T23:04:52.999 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-24T23:04:52.999 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:04:52.999 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:53.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:53.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:53.000 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2024-04-24T23:04:53.000 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmak7bDhAA0he+Jp/gkkrfBjYKSObjcw==", 2024-04-24T23:04:53.000 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:04:53.000 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:04:53.000 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-24T23:04:53.000 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-24T23:04:53.000 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:04:53.000 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:53.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:53.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:53.000 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.c", 2024-04-24T23:04:53.000 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmZXFdChAA8TwwkA5EIXHBzPsQSFvXcQ==", 2024-04-24T23:04:53.000 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:04:53.001 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:04:53.001 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-24T23:04:53.001 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-24T23:04:53.001 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:04:53.001 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:53.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:53.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:53.001 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.d", 2024-04-24T23:04:53.001 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmLbGuFRAAe91fZlulziWdWiDdLBLWNA==", 2024-04-24T23:04:53.001 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:04:53.001 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:04:53.001 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-24T23:04:53.001 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-24T23:04:53.001 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:04:53.001 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:53.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:53.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:53.002 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2024-04-24T23:04:53.002 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCfjilmhTwoOBAAig98xa3a2qnv4rriUO9VCQ==", 2024-04-24T23:04:53.002 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:04:53.002 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:04:53.002 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:04:53.002 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:04:53.002 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:53.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:53.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:53.002 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2024-04-24T23:04:53.002 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCgjilmwRaAAxAAUqDOyR9HV6DEnmbyn3WAjA==", 2024-04-24T23:04:53.002 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:04:53.003 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:04:53.003 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:04:53.003 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:04:53.003 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:53.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:53.003 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:53.003 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2024-04-24T23:04:53.003 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCgjilmDl2qDBAAF1bhTOaSOc612wmFRd8+XA==", 2024-04-24T23:04:53.003 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:04:53.003 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:04:53.003 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:04:53.003 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:04:53.003 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:53.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:53.003 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:53.004 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2024-04-24T23:04:53.004 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCgjilmAxHwFRAA9a0v/YwngRxq1vmMRsmEkQ==", 2024-04-24T23:04:53.004 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:04:53.004 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:04:53.004 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:04:53.004 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:04:53.004 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:53.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:53.004 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:53.004 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2024-04-24T23:04:53.004 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCjjilmDPmJIxAASZlEo1ZJHOhIAFGukIj1tQ==", 2024-04-24T23:04:53.004 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:04:53.004 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:04:53.004 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:04:53.005 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:04:53.005 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:53.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:53.005 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:53.005 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2024-04-24T23:04:53.005 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCjjilmgcpuKhAAedSq0ubNB30s3bz0fY38hw==", 2024-04-24T23:04:53.005 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:04:53.005 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:04:53.005 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:04:53.005 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:04:53.005 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:53.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:53.005 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:53.005 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2024-04-24T23:04:53.005 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCjjilmFv2VMxAAM+mUc4I0sjawy2uwuUEKYg==", 2024-04-24T23:04:53.006 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:04:53.006 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:04:53.006 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:04:53.006 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:04:53.006 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:53.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:53.006 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:53.006 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2024-04-24T23:04:53.006 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCkjilmLvEbARAAnaOvQ5Ee0clN/A9SVoKuBQ==", 2024-04-24T23:04:53.006 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:04:53.006 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:04:53.006 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:04:53.006 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:04:53.006 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:53.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:53.007 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:53.007 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2024-04-24T23:04:53.007 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilm5oeiGRAAdLXwwqrYhV3HHo7Dw/2ZqQ==", 2024-04-24T23:04:53.007 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:04:53.007 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:04:53.007 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2024-04-24T23:04:53.007 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-24T23:04:53.007 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:53.007 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:53.007 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:53.007 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2024-04-24T23:04:53.007 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmyWBDIhAARtVP4Unk3HHlQYZ2QZ3GzQ==", 2024-04-24T23:04:53.007 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:04:53.007 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:04:53.007 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-24T23:04:53.008 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-24T23:04:53.008 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-24T23:04:53.008 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:53.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:53.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:53.008 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.2", 2024-04-24T23:04:53.008 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilm4QElHxAAXgE316aRCosxX7XER5gtKQ==", 2024-04-24T23:04:53.008 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:04:53.008 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:04:53.008 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-24T23:04:53.008 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-24T23:04:53.008 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-24T23:04:53.008 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:53.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:53.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:53.009 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.3", 2024-04-24T23:04:53.009 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmMZihJxAAoWaAAFY3vVfZnC4/GoT0+g==", 2024-04-24T23:04:53.009 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:04:53.009 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:04:53.009 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-24T23:04:53.009 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-24T23:04:53.009 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-24T23:04:53.009 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:53.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:53.009 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:53.009 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2024-04-24T23:04:53.009 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCdjilmdVXeIxAAwGGaAfshx4pHcoc5+3kRww==", 2024-04-24T23:04:53.009 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:04:53.009 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2024-04-24T23:04:53.010 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-24T23:04:53.010 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-24T23:04:53.010 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:04:53.010 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:53.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:53.010 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:53.010 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2024-04-24T23:04:53.010 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmn3FJABAAGnvNAoNLzK20cQF6beihmw==", 2024-04-24T23:04:53.010 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:04:53.010 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2024-04-24T23:04:53.010 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2024-04-24T23:04:53.010 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:04:53.010 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:53.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:04:53.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:53.011 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2024-04-24T23:04:53.011 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCdjilmMqs2OBAAOe3bLpnCq+HK2EbeXnwY8Q==", 2024-04-24T23:04:53.011 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:04:53.011 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2024-04-24T23:04:53.011 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2024-04-24T23:04:53.011 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:04:53.011 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:53.011 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:53.011 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:04:53.011 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:04:53.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.999+0000 7f08a8009640 1 -- 172.21.15.32:0/3899233766 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f088406be30 msgr2=0x7f088406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:53.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.999+0000 7f08a8009640 1 --2- 172.21.15.32:0/3899233766 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f088406be30 0x7f088406e2f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f089c000f60 tx=0x7f089c009040 comp rx=0 tx=0).stop 2024-04-24T23:04:53.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.999+0000 7f08a8009640 1 -- 172.21.15.32:0/3899233766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08a00b44d0 msgr2=0x7f08a01e6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:53.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.999+0000 7f08a8009640 1 --2- 172.21.15.32:0/3899233766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08a00b44d0 0x7f08a01e6910 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f0898009dd0 tx=0x7f089800c490 comp rx=0 tx=0).stop 2024-04-24T23:04:53.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.999+0000 7f08a8009640 1 -- 172.21.15.32:0/3899233766 shutdown_connections 2024-04-24T23:04:53.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.999+0000 7f08a8009640 1 --2- 172.21.15.32:0/3899233766 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f088406be30 0x7f088406e2f0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:53.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.999+0000 7f08a8009640 1 --2- 172.21.15.32:0/3899233766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08a00b44d0 0x7f08a01e6910 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:53.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.999+0000 7f08a8009640 1 --2- 172.21.15.32:0/3899233766 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08a00af890 0x7f08a00aa5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:53.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.999+0000 7f08a8009640 1 --2- 172.21.15.32:0/3899233766 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f08a006c5f0 0x7f08a01f50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:53.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.999+0000 7f08a8009640 1 -- 172.21.15.32:0/3899233766 >> 172.21.15.32:0/3899233766 conn(0x7f08a006b110 msgr2=0x7f08a0072a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:53.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.999+0000 7f08a8009640 1 -- 172.21.15.32:0/3899233766 shutdown_connections 2024-04-24T23:04:53.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:52.999+0000 7f08a8009640 1 -- 172.21.15.32:0/3899233766 wait complete. 2024-04-24T23:04:53.014 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:04:53.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.113+0000 7fd0fbc7d640 1 Processor -- start 2024-04-24T23:04:53.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.113+0000 7fd0fbc7d640 1 -- start start 2024-04-24T23:04:53.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.114+0000 7fd0fbc7d640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd0f4162320 0x7fd0f40ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:53.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.114+0000 7fd0fbc7d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd0f4162e10 0x7fd0f415ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:53.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.114+0000 7fd0fbc7d640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0f40cd270 con 0x7fd0f40b5af0 2024-04-24T23:04:53.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.114+0000 7fd0fbc7d640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fd0f40b5660 con 0x7fd0f4162320 2024-04-24T23:04:53.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.114+0000 7fd0fbc7d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd0f4071610 con 0x7fd0f4162e10 2024-04-24T23:04:53.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.114+0000 7fd0fa21a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd0f4162e10 0x7fd0f415ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:53.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.114+0000 7fd0f9a19640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd0f4162320 0x7fd0f40ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:53.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.114+0000 7fd0fa21a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd0f4162e10 0x7fd0f415ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.32:35080/0 (socket says 172.21.15.32:35080) 2024-04-24T23:04:53.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.114+0000 7fd0f9218640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fd0f40b5af0 0x7fd0f4162700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:34972/0 (socket says 172.21.15.32:34972) 2024-04-24T23:04:53.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.114+0000 7fd0fa21a640 1 -- 172.21.15.32:0/751237465 learned_addr learned my addr 172.21.15.32:0/751237465 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:53.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.114+0000 7fd0f9a19640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd0f4162320 0x7fd0f40ccc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.32:53646/0 (socket says 172.21.15.32:53646) 2024-04-24T23:04:53.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.114+0000 7fd0fa21a640 1 -- 172.21.15.32:0/751237465 >> v2:172.21.15.121:3300/0 conn(0x7fd0f4162320 msgr2=0x7fd0f40ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:53.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.114+0000 7fd0fa21a640 1 --2- 172.21.15.32:0/751237465 >> v2:172.21.15.121:3300/0 conn(0x7fd0f4162320 0x7fd0f40ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:53.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.114+0000 7fd0fa21a640 1 -- 172.21.15.32:0/751237465 >> v1:172.21.15.121:6789/0 conn(0x7fd0f40b5af0 legacy=0x7fd0f4162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:53.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.115+0000 7fd0fa21a640 1 -- 172.21.15.32:0/751237465 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0f40718f0 con 0x7fd0f4162e10 2024-04-24T23:04:53.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.115+0000 7fd0fa21a640 1 --2- 172.21.15.32:0/751237465 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd0f4162e10 0x7fd0f415ed30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec009e50 tx=0x7fd0ec02a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6862407673a0fad server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:53.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.115+0000 7fd0f8a17640 1 -- 172.21.15.32:0/751237465 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ec036070 con 0x7fd0f4162e10 2024-04-24T23:04:53.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.115+0000 7fd0f8a17640 1 -- 172.21.15.32:0/751237465 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0ec031600 con 0x7fd0f4162e10 2024-04-24T23:04:53.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.115+0000 7fd0f8a17640 1 -- 172.21.15.32:0/751237465 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ec0319d0 con 0x7fd0f4162e10 2024-04-24T23:04:53.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.116+0000 7fd0fbc7d640 1 -- 172.21.15.32:0/751237465 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd0f4162e10 msgr2=0x7fd0f415ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:53.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.116+0000 7fd0fbc7d640 1 --2- 172.21.15.32:0/751237465 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd0f4162e10 0x7fd0f415ed30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec009e50 tx=0x7fd0ec02a5f0 comp rx=0 tx=0).stop 2024-04-24T23:04:53.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.116+0000 7fd0fbc7d640 1 -- 172.21.15.32:0/751237465 shutdown_connections 2024-04-24T23:04:53.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.116+0000 7fd0fbc7d640 1 --2- 172.21.15.32:0/751237465 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd0f4162e10 0x7fd0f415ed30 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:53.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.116+0000 7fd0fbc7d640 1 --2- 172.21.15.32:0/751237465 >> v2:172.21.15.121:3300/0 conn(0x7fd0f4162320 0x7fd0f40ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:53.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.116+0000 7fd0fbc7d640 1 -- 172.21.15.32:0/751237465 >> 172.21.15.32:0/751237465 conn(0x7fd0f40d0ae0 msgr2=0x7fd0f40d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:53.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.116+0000 7fd0fbc7d640 1 -- 172.21.15.32:0/751237465 shutdown_connections 2024-04-24T23:04:53.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.116+0000 7fd0fbc7d640 1 -- 172.21.15.32:0/751237465 wait complete. 2024-04-24T23:04:53.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.117+0000 7fd0fbc7d640 1 Processor -- start 2024-04-24T23:04:53.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.117+0000 7fd0fbc7d640 1 -- start start 2024-04-24T23:04:53.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.118+0000 7fd0fbc7d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd0f40b5af0 0x7fd0f40b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:53.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.118+0000 7fd0fbc7d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0f4162320 0x7fd0f41f8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:53.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.118+0000 7fd0fbc7d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0f4162e10 0x7fd0f41f8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:53.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.118+0000 7fd0fbc7d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f4071bd0 con 0x7fd0f4162320 2024-04-24T23:04:53.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.118+0000 7fd0fbc7d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f4067950 con 0x7fd0f4162e10 2024-04-24T23:04:53.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.118+0000 7fd0fbc7d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd0f4071d50 con 0x7fd0f40b5af0 2024-04-24T23:04:53.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.118+0000 7fd0f9218640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0f4162320 0x7fd0f41f8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:53.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.118+0000 7fd0f9218640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0f4162320 0x7fd0f41f8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56984/0 (socket says 172.21.15.32:56984) 2024-04-24T23:04:53.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.118+0000 7fd0f9218640 1 -- 172.21.15.32:0/2068962658 learned_addr learned my addr 172.21.15.32:0/2068962658 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:53.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.118+0000 7fd0fa21a640 1 --2- 172.21.15.32:0/2068962658 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0f4162e10 0x7fd0f41f8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:53.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.118+0000 7fd0f9a19640 1 --2- 172.21.15.32:0/2068962658 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd0f40b5af0 0x7fd0f40b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:53.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.118+0000 7fd0f9218640 1 -- 172.21.15.32:0/2068962658 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd0f40b5af0 msgr2=0x7fd0f40b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:53.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.118+0000 7fd0f9218640 1 --2- 172.21.15.32:0/2068962658 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd0f40b5af0 0x7fd0f40b1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:53.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.118+0000 7fd0f9218640 1 -- 172.21.15.32:0/2068962658 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0f4162e10 msgr2=0x7fd0f41f8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:53.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.118+0000 7fd0f9218640 1 --2- 172.21.15.32:0/2068962658 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0f4162e10 0x7fd0f41f8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:53.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.118+0000 7fd0f9218640 1 -- 172.21.15.32:0/2068962658 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0f41f8bb0 con 0x7fd0f4162320 2024-04-24T23:04:53.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.119+0000 7fd0f9218640 1 --2- 172.21.15.32:0/2068962658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0f4162320 0x7fd0f41f8130 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fd0e400c430 tx=0x7fd0e400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:53.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.119+0000 7fd0ea7fc640 1 -- 172.21.15.32:0/2068962658 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e40074e0 con 0x7fd0f4162320 2024-04-24T23:04:53.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.119+0000 7fd0ea7fc640 1 -- 172.21.15.32:0/2068962658 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0e4007680 con 0x7fd0f4162320 2024-04-24T23:04:53.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.119+0000 7fd0fbc7d640 1 -- 172.21.15.32:0/2068962658 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0f41f8e40 con 0x7fd0f4162320 2024-04-24T23:04:53.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.119+0000 7fd0ea7fc640 1 -- 172.21.15.32:0/2068962658 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e4004a00 con 0x7fd0f4162320 2024-04-24T23:04:53.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.119+0000 7fd0fbc7d640 1 -- 172.21.15.32:0/2068962658 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0f41e85c0 con 0x7fd0f4162320 2024-04-24T23:04:53.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.120+0000 7fd0fbc7d640 1 -- 172.21.15.32:0/2068962658 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0f4068430 con 0x7fd0f4162320 2024-04-24T23:04:53.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.120+0000 7fd0ea7fc640 1 -- 172.21.15.32:0/2068962658 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd0e401a020 con 0x7fd0f4162320 2024-04-24T23:04:53.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.121+0000 7fd0ea7fc640 1 --2- 172.21.15.32:0/2068962658 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd0d806be30 0x7fd0d806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:53.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.121+0000 7fd0ea7fc640 1 -- 172.21.15.32:0/2068962658 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 4857+0+0 (secure 0 0 0) 0x7fd0e408a1e0 con 0x7fd0f4162320 2024-04-24T23:04:53.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.121+0000 7fd0f9a19640 1 --2- 172.21.15.32:0/2068962658 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd0d806be30 0x7fd0d806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:53.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.121+0000 7fd0f9a19640 1 --2- 172.21.15.32:0/2068962658 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd0d806be30 0x7fd0d806e2f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0009b40 tx=0x7fd0f0009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:53.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.125+0000 7fd0ea7fc640 1 -- 172.21.15.32:0/2068962658 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd0e4058890 con 0x7fd0f4162320 2024-04-24T23:04:53.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.312+0000 7fd0fbc7d640 1 -- 172.21.15.32:0/2068962658 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd0f4067730 con 0x7fd0f4162320 2024-04-24T23:04:53.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.312+0000 7fd0ea7fc640 1 -- 172.21.15.32:0/2068962658 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) ==== 74+0+12914 (secure 0 0 0) 0x7fd0e405cb90 con 0x7fd0f4162320 2024-04-24T23:04:53.314 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:04:53.314 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":121,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:04:52.609875+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:04:53.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.314+0000 7fd0fbc7d640 1 -- 172.21.15.32:0/2068962658 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd0d806be30 msgr2=0x7fd0d806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:53.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.314+0000 7fd0fbc7d640 1 --2- 172.21.15.32:0/2068962658 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd0d806be30 0x7fd0d806e2f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0009b40 tx=0x7fd0f0009190 comp rx=0 tx=0).stop 2024-04-24T23:04:53.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.314+0000 7fd0fbc7d640 1 -- 172.21.15.32:0/2068962658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0f4162320 msgr2=0x7fd0f41f8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:53.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.314+0000 7fd0fbc7d640 1 --2- 172.21.15.32:0/2068962658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0f4162320 0x7fd0f41f8130 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fd0e400c430 tx=0x7fd0e400c8f0 comp rx=0 tx=0).stop 2024-04-24T23:04:53.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.314+0000 7fd0fbc7d640 1 -- 172.21.15.32:0/2068962658 shutdown_connections 2024-04-24T23:04:53.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.314+0000 7fd0fbc7d640 1 --2- 172.21.15.32:0/2068962658 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd0d806be30 0x7fd0d806e2f0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:53.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.314+0000 7fd0fbc7d640 1 --2- 172.21.15.32:0/2068962658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0f4162320 0x7fd0f41f8130 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:53.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.314+0000 7fd0fbc7d640 1 --2- 172.21.15.32:0/2068962658 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0f4162e10 0x7fd0f41f8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:53.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.314+0000 7fd0fbc7d640 1 --2- 172.21.15.32:0/2068962658 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd0f40b5af0 0x7fd0f40b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:53.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.314+0000 7fd0fbc7d640 1 -- 172.21.15.32:0/2068962658 >> 172.21.15.32:0/2068962658 conn(0x7fd0f40d0ae0 msgr2=0x7fd0f415d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:53.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.315+0000 7fd0fbc7d640 1 -- 172.21.15.32:0/2068962658 shutdown_connections 2024-04-24T23:04:53.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.315+0000 7fd0fbc7d640 1 -- 172.21.15.32:0/2068962658 wait complete. 2024-04-24T23:04:53.327 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:04:53.327 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:04:53.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.429+0000 7fcbc6d34640 1 Processor -- start 2024-04-24T23:04:53.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.430+0000 7fcbc6d34640 1 -- start start 2024-04-24T23:04:53.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.430+0000 7fcbc6d34640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcbc015e9b0 0x7fcbc00d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:53.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.430+0000 7fcbc6d34640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcbc00d4210 0x7fcbc006b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:53.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.430+0000 7fcbc6d34640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbc00d3170 con 0x7fcbc00ba0e0 2024-04-24T23:04:53.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.430+0000 7fcbc6d34640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fcbc00b8900 con 0x7fcbc015e9b0 2024-04-24T23:04:53.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.430+0000 7fcbc6d34640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fcbc00d3450 con 0x7fcbc00d4210 2024-04-24T23:04:53.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.430+0000 7fcbc4ad0640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcbc015e9b0 0x7fcbc00d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:53.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.430+0000 7fcbc4ad0640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcbc015e9b0 0x7fcbc00d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56990/0 (socket says 172.21.15.32:56990) 2024-04-24T23:04:53.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.430+0000 7fcbc4ad0640 1 -- 172.21.15.32:0/477518580 learned_addr learned my addr 172.21.15.32:0/477518580 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:53.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.430+0000 7fcbc52d1640 1 --1- 172.21.15.32:0/477518580 >> v1:172.21.15.32:6789/0 conn(0x7fcbc00ba0e0 0x7fcbc0071730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49488/0 (socket says 172.21.15.32:49488) 2024-04-24T23:04:53.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.431+0000 7fcbc4ad0640 1 -- 172.21.15.32:0/477518580 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcbc00d4210 msgr2=0x7fcbc006b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:53.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.431+0000 7fcbc4ad0640 1 --2- 172.21.15.32:0/477518580 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcbc00d4210 0x7fcbc006b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:53.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.431+0000 7fcbc4ad0640 1 -- 172.21.15.32:0/477518580 >> v1:172.21.15.32:6789/0 conn(0x7fcbc00ba0e0 legacy=0x7fcbc0071730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:53.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.431+0000 7fcbc4ad0640 1 -- 172.21.15.32:0/477518580 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbc00d3730 con 0x7fcbc015e9b0 2024-04-24T23:04:53.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.431+0000 7fcbbf7fe640 1 -- 172.21.15.32:0/477518580 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2271045167 0 0) 0x7fcbc00d3170 con 0x7fcbc00ba0e0 2024-04-24T23:04:53.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.431+0000 7fcbc4ad0640 1 --2- 172.21.15.32:0/477518580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbc015e9b0 0x7fcbc00d2b90 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8009e50 tx=0x7fcbb802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f932304b373c6a13 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:53.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.432+0000 7fcbbf7fe640 1 -- 172.21.15.32:0/477518580 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb8036070 con 0x7fcbc015e9b0 2024-04-24T23:04:53.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.432+0000 7fcbbf7fe640 1 -- 172.21.15.32:0/477518580 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbb8031600 con 0x7fcbc015e9b0 2024-04-24T23:04:53.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.432+0000 7fcbbf7fe640 1 -- 172.21.15.32:0/477518580 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb80319d0 con 0x7fcbc015e9b0 2024-04-24T23:04:53.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.432+0000 7fcbc6d34640 1 -- 172.21.15.32:0/477518580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbc015e9b0 msgr2=0x7fcbc00d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:53.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.432+0000 7fcbc6d34640 1 --2- 172.21.15.32:0/477518580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbc015e9b0 0x7fcbc00d2b90 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8009e50 tx=0x7fcbb802a710 comp rx=0 tx=0).stop 2024-04-24T23:04:53.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.432+0000 7fcbc6d34640 1 -- 172.21.15.32:0/477518580 shutdown_connections 2024-04-24T23:04:53.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.432+0000 7fcbc6d34640 1 --2- 172.21.15.32:0/477518580 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcbc00d4210 0x7fcbc006b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:53.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.432+0000 7fcbc6d34640 1 --2- 172.21.15.32:0/477518580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbc015e9b0 0x7fcbc00d2b90 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:53.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.432+0000 7fcbc6d34640 1 -- 172.21.15.32:0/477518580 >> 172.21.15.32:0/477518580 conn(0x7fcbc00d11d0 msgr2=0x7fcbc00d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:53.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.432+0000 7fcbc6d34640 1 -- 172.21.15.32:0/477518580 shutdown_connections 2024-04-24T23:04:53.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.432+0000 7fcbc6d34640 1 -- 172.21.15.32:0/477518580 wait complete. 2024-04-24T23:04:53.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.433+0000 7fcbc6d34640 1 Processor -- start 2024-04-24T23:04:53.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.433+0000 7fcbc6d34640 1 -- start start 2024-04-24T23:04:53.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.434+0000 7fcbc6d34640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbc00ba0e0 0x7fcbc01501f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:53.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.434+0000 7fcbc6d34640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcbc00d4210 0x7fcbc0153360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:53.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.434+0000 7fcbc6d34640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcbc015e9b0 0x7fcbc01538a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:53.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.434+0000 7fcbc6d34640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc00d2400 con 0x7fcbc00ba0e0 2024-04-24T23:04:53.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.434+0000 7fcbc6d34640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc00d3cf0 con 0x7fcbc015e9b0 2024-04-24T23:04:53.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.434+0000 7fcbc6d34640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fcbc006b650 con 0x7fcbc00d4210 2024-04-24T23:04:53.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.434+0000 7fcbc4ad0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbc00ba0e0 0x7fcbc01501f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:53.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.434+0000 7fcbc4ad0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbc00ba0e0 0x7fcbc01501f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57004/0 (socket says 172.21.15.32:57004) 2024-04-24T23:04:53.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.434+0000 7fcbc4ad0640 1 -- 172.21.15.32:0/4212037540 learned_addr learned my addr 172.21.15.32:0/4212037540 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:53.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.435+0000 7fcbbffff640 1 --2- 172.21.15.32:0/4212037540 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcbc00d4210 0x7fcbc0153360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:53.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.435+0000 7fcbc52d1640 1 --2- 172.21.15.32:0/4212037540 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcbc015e9b0 0x7fcbc01538a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:53.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.435+0000 7fcbc4ad0640 1 -- 172.21.15.32:0/4212037540 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcbc00d4210 msgr2=0x7fcbc0153360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:53.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.435+0000 7fcbc4ad0640 1 --2- 172.21.15.32:0/4212037540 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcbc00d4210 0x7fcbc0153360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:53.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.435+0000 7fcbc4ad0640 1 -- 172.21.15.32:0/4212037540 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcbc015e9b0 msgr2=0x7fcbc01538a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:53.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.435+0000 7fcbc4ad0640 1 --2- 172.21.15.32:0/4212037540 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcbc015e9b0 0x7fcbc01538a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:53.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.435+0000 7fcbc4ad0640 1 -- 172.21.15.32:0/4212037540 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbc015f350 con 0x7fcbc00ba0e0 2024-04-24T23:04:53.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.435+0000 7fcbc4ad0640 1 --2- 172.21.15.32:0/4212037540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbc00ba0e0 0x7fcbc01501f0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8009e50 tx=0x7fcbb8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:53.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.435+0000 7fcbbd7fa640 1 -- 172.21.15.32:0/4212037540 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb8041020 con 0x7fcbc00ba0e0 2024-04-24T23:04:53.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.435+0000 7fcbbd7fa640 1 -- 172.21.15.32:0/4212037540 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbb8007800 con 0x7fcbc00ba0e0 2024-04-24T23:04:53.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.435+0000 7fcbc6d34640 1 -- 172.21.15.32:0/4212037540 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbc015f630 con 0x7fcbc00ba0e0 2024-04-24T23:04:53.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.435+0000 7fcbbd7fa640 1 -- 172.21.15.32:0/4212037540 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb8036040 con 0x7fcbc00ba0e0 2024-04-24T23:04:53.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.435+0000 7fcbc6d34640 1 -- 172.21.15.32:0/4212037540 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbc015fbf0 con 0x7fcbc00ba0e0 2024-04-24T23:04:53.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.436+0000 7fcbc6d34640 1 -- 172.21.15.32:0/4212037540 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbc0068430 con 0x7fcbc00ba0e0 2024-04-24T23:04:53.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.437+0000 7fcbbd7fa640 1 -- 172.21.15.32:0/4212037540 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcbb8044020 con 0x7fcbc00ba0e0 2024-04-24T23:04:53.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.437+0000 7fcbbd7fa640 1 --2- 172.21.15.32:0/4212037540 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcba006be30 0x7fcba006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:53.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.437+0000 7fcbbd7fa640 1 -- 172.21.15.32:0/4212037540 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 4857+0+0 (secure 0 0 0) 0x7fcbb80ab940 con 0x7fcbc00ba0e0 2024-04-24T23:04:53.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.437+0000 7fcbbffff640 1 --2- 172.21.15.32:0/4212037540 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcba006be30 0x7fcba006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:53.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.438+0000 7fcbbffff640 1 --2- 172.21.15.32:0/4212037540 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcba006be30 0x7fcba006e2f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fcbc015bb20 tx=0x7fcba8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:53.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.441+0000 7fcbbd7fa640 1 -- 172.21.15.32:0/4212037540 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcbb804b030 con 0x7fcbc00ba0e0 2024-04-24T23:04:53.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.622+0000 7fcbc6d34640 1 -- 172.21.15.32:0/4212037540 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcbc00cad30 con 0x7fcbc00ba0e0 2024-04-24T23:04:53.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.623+0000 7fcbbd7fa640 1 -- 172.21.15.32:0/4212037540 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v121) ==== 93+0+10 (secure 0 0 0) 0x7fcbb807a640 con 0x7fcbc00ba0e0 2024-04-24T23:04:53.624 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T23:04:53.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.625+0000 7fcbc6d34640 1 -- 172.21.15.32:0/4212037540 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcba006be30 msgr2=0x7fcba006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:53.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.625+0000 7fcbc6d34640 1 --2- 172.21.15.32:0/4212037540 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcba006be30 0x7fcba006e2f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fcbc015bb20 tx=0x7fcba8008040 comp rx=0 tx=0).stop 2024-04-24T23:04:53.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.625+0000 7fcbc6d34640 1 -- 172.21.15.32:0/4212037540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbc00ba0e0 msgr2=0x7fcbc01501f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:53.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.625+0000 7fcbc6d34640 1 --2- 172.21.15.32:0/4212037540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbc00ba0e0 0x7fcbc01501f0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8009e50 tx=0x7fcbb8004a40 comp rx=0 tx=0).stop 2024-04-24T23:04:53.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.625+0000 7fcbc6d34640 1 -- 172.21.15.32:0/4212037540 shutdown_connections 2024-04-24T23:04:53.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.625+0000 7fcbc6d34640 1 --2- 172.21.15.32:0/4212037540 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcba006be30 0x7fcba006e2f0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:53.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.625+0000 7fcbc6d34640 1 --2- 172.21.15.32:0/4212037540 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcbc015e9b0 0x7fcbc01538a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:53.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.625+0000 7fcbc6d34640 1 --2- 172.21.15.32:0/4212037540 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcbc00d4210 0x7fcbc0153360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:53.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.625+0000 7fcbc6d34640 1 --2- 172.21.15.32:0/4212037540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbc00ba0e0 0x7fcbc01501f0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:53.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.625+0000 7fcbc6d34640 1 -- 172.21.15.32:0/4212037540 >> 172.21.15.32:0/4212037540 conn(0x7fcbc00d11d0 msgr2=0x7fcbc0071730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:53.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.625+0000 7fcbc6d34640 1 -- 172.21.15.32:0/4212037540 shutdown_connections 2024-04-24T23:04:53.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.625+0000 7fcbc6d34640 1 -- 172.21.15.32:0/4212037540 wait complete. 2024-04-24T23:04:53.637 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-24T23:04:53.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.736+0000 7f52e8da0640 1 Processor -- start 2024-04-24T23:04:53.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.736+0000 7f52e8da0640 1 -- start start 2024-04-24T23:04:53.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.737+0000 7f52e8da0640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f52e40b45f0 0x7f52e40b6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:53.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.737+0000 7f52e8da0640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f52e40b6ab0 0x7f52e415e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:53.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.737+0000 7f52e8da0640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52e40b7260 con 0x7f52e4159f10 2024-04-24T23:04:53.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.737+0000 7f52e8da0640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f52e415a2f0 con 0x7f52e40b6ab0 2024-04-24T23:04:53.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.737+0000 7f52e8da0640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f52e415dbd0 con 0x7f52e40b45f0 2024-04-24T23:04:53.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.737+0000 7f52e3fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f52e40b6ab0 0x7f52e415e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:53.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.738+0000 7f52e3fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f52e40b6ab0 0x7f52e415e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57018/0 (socket says 172.21.15.32:57018) 2024-04-24T23:04:53.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.738+0000 7f52e3fff640 1 -- 172.21.15.32:0/2345277848 learned_addr learned my addr 172.21.15.32:0/2345277848 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:53.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.738+0000 7f52e2ffd640 1 --2- 172.21.15.32:0/2345277848 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f52e40b45f0 0x7f52e40b6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:53.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.738+0000 7f52e3fff640 1 -- 172.21.15.32:0/2345277848 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f52e40b45f0 msgr2=0x7f52e40b6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:53.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.738+0000 7f52e3fff640 1 --2- 172.21.15.32:0/2345277848 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f52e40b45f0 0x7f52e40b6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:53.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.738+0000 7f52e3fff640 1 -- 172.21.15.32:0/2345277848 >> v1:172.21.15.121:6789/0 conn(0x7f52e4159f10 legacy=0x7f52e40b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:53.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.738+0000 7f52e3fff640 1 -- 172.21.15.32:0/2345277848 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52e415deb0 con 0x7f52e40b6ab0 2024-04-24T23:04:53.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.739+0000 7f52e3fff640 1 --2- 172.21.15.32:0/2345277848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52e40b6ab0 0x7f52e415e6b0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f52dc009e50 tx=0x7f52dc02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a783380f91a0dbff server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:53.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.739+0000 7f52e27fc640 1 -- 172.21.15.32:0/2345277848 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52dc036070 con 0x7f52e40b6ab0 2024-04-24T23:04:53.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.739+0000 7f52e27fc640 1 -- 172.21.15.32:0/2345277848 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52dc031600 con 0x7f52e40b6ab0 2024-04-24T23:04:53.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.739+0000 7f52e27fc640 1 -- 172.21.15.32:0/2345277848 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52dc0319d0 con 0x7f52e40b6ab0 2024-04-24T23:04:53.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.739+0000 7f52e8da0640 1 -- 172.21.15.32:0/2345277848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52e40b6ab0 msgr2=0x7f52e415e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:53.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.739+0000 7f52e8da0640 1 --2- 172.21.15.32:0/2345277848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52e40b6ab0 0x7f52e415e6b0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f52dc009e50 tx=0x7f52dc02a710 comp rx=0 tx=0).stop 2024-04-24T23:04:53.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.739+0000 7f52e8da0640 1 -- 172.21.15.32:0/2345277848 shutdown_connections 2024-04-24T23:04:53.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.739+0000 7f52e8da0640 1 --2- 172.21.15.32:0/2345277848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52e40b6ab0 0x7f52e415e6b0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:53.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.739+0000 7f52e8da0640 1 --2- 172.21.15.32:0/2345277848 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f52e40b45f0 0x7f52e40b6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:53.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.739+0000 7f52e8da0640 1 -- 172.21.15.32:0/2345277848 >> 172.21.15.32:0/2345277848 conn(0x7f52e40743f0 msgr2=0x7f52e406d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:53.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.739+0000 7f52e8da0640 1 -- 172.21.15.32:0/2345277848 shutdown_connections 2024-04-24T23:04:53.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.739+0000 7f52e8da0640 1 -- 172.21.15.32:0/2345277848 wait complete. 2024-04-24T23:04:53.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.741+0000 7f52e8da0640 1 Processor -- start 2024-04-24T23:04:53.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.741+0000 7f52e8da0640 1 -- start start 2024-04-24T23:04:53.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.741+0000 7f52e8da0640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f52e40b45f0 0x7f52e41f4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:53.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.741+0000 7f52e8da0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52e40b6ab0 0x7f52e40d0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:53.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.741+0000 7f52e8da0640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52e4159f10 0x7f52e4155dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:53.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.742+0000 7f52e8da0640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f52e415ebf0 con 0x7f52e40b6ab0 2024-04-24T23:04:53.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.742+0000 7f52e8da0640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f52e40b6f40 con 0x7f52e4159f10 2024-04-24T23:04:53.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.742+0000 7f52e8da0640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f52e415ed70 con 0x7f52e40b45f0 2024-04-24T23:04:53.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.742+0000 7f52e2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52e40b6ab0 0x7f52e40d0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:53.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.742+0000 7f52e2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52e40b6ab0 0x7f52e40d0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57020/0 (socket says 172.21.15.32:57020) 2024-04-24T23:04:53.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.742+0000 7f52e2ffd640 1 -- 172.21.15.32:0/4048498470 learned_addr learned my addr 172.21.15.32:0/4048498470 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:53.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.742+0000 7f52e37fe640 1 --2- 172.21.15.32:0/4048498470 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f52e40b45f0 0x7f52e41f4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:53.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.742+0000 7f52e3fff640 1 --2- 172.21.15.32:0/4048498470 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52e4159f10 0x7f52e4155dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:53.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.742+0000 7f52e2ffd640 1 -- 172.21.15.32:0/4048498470 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f52e40b45f0 msgr2=0x7f52e41f4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:53.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.742+0000 7f52e2ffd640 1 --2- 172.21.15.32:0/4048498470 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f52e40b45f0 0x7f52e41f4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:53.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.742+0000 7f52e2ffd640 1 -- 172.21.15.32:0/4048498470 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52e4159f10 msgr2=0x7f52e4155dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:53.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.742+0000 7f52e2ffd640 1 --2- 172.21.15.32:0/4048498470 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52e4159f10 0x7f52e4155dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:53.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.742+0000 7f52e2ffd640 1 -- 172.21.15.32:0/4048498470 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52e406b960 con 0x7f52e40b6ab0 2024-04-24T23:04:53.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.742+0000 7f52e2ffd640 1 --2- 172.21.15.32:0/4048498470 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52e40b6ab0 0x7f52e40d0d30 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f52d800d800 tx=0x7f52d800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:53.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.742+0000 7f52cbfff640 1 -- 172.21.15.32:0/4048498470 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d800b6c0 con 0x7f52e40b6ab0 2024-04-24T23:04:53.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.742+0000 7f52cbfff640 1 -- 172.21.15.32:0/4048498470 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52d8019070 con 0x7f52e40b6ab0 2024-04-24T23:04:53.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.742+0000 7f52cbfff640 1 -- 172.21.15.32:0/4048498470 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d800bac0 con 0x7f52e40b6ab0 2024-04-24T23:04:53.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.742+0000 7f52e8da0640 1 -- 172.21.15.32:0/4048498470 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52e40ae8b0 con 0x7f52e40b6ab0 2024-04-24T23:04:53.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.743+0000 7f52e8da0640 1 -- 172.21.15.32:0/4048498470 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52e406bc00 con 0x7f52e40b6ab0 2024-04-24T23:04:53.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.744+0000 7f52e8da0640 1 -- 172.21.15.32:0/4048498470 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52e40682b0 con 0x7f52e40b6ab0 2024-04-24T23:04:53.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.744+0000 7f52cbfff640 1 -- 172.21.15.32:0/4048498470 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f52d800bc60 con 0x7f52e40b6ab0 2024-04-24T23:04:53.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.744+0000 7f52cbfff640 1 --2- 172.21.15.32:0/4048498470 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f52d006be30 0x7f52d006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:53.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.744+0000 7f52cbfff640 1 -- 172.21.15.32:0/4048498470 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 4857+0+0 (secure 0 0 0) 0x7f52d808ae80 con 0x7f52e40b6ab0 2024-04-24T23:04:53.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.744+0000 7f52e37fe640 1 --2- 172.21.15.32:0/4048498470 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f52d006be30 0x7f52d006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:53.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.745+0000 7f52e37fe640 1 --2- 172.21.15.32:0/4048498470 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f52d006be30 0x7f52d006e2f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f52d40016e0 tx=0x7f52d4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:53.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.749+0000 7f52cbfff640 1 -- 172.21.15.32:0/4048498470 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f52d8059530 con 0x7f52e40b6ab0 2024-04-24T23:04:53.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.934+0000 7f52e8da0640 1 -- 172.21.15.32:0/4048498470 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f52e40b1510 con 0x7f52e40b6ab0 2024-04-24T23:04:53.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.934+0000 7f52cbfff640 1 -- 172.21.15.32:0/4048498470 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v121) ==== 84+0+65 (secure 0 0 0) 0x7f52d805d830 con 0x7f52e40b6ab0 2024-04-24T23:04:53.936 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:04:53.936 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2024-04-24T23:04:53.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:04:53.936 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2024-04-24T23:04:53.936 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": ".mgr" 2024-04-24T23:04:53.936 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:04:53.936 INFO:teuthology.orchestra.run.smithi032.stdout:] 2024-04-24T23:04:53.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.936+0000 7f52e8da0640 1 -- 172.21.15.32:0/4048498470 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f52d006be30 msgr2=0x7f52d006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:53.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.936+0000 7f52e8da0640 1 --2- 172.21.15.32:0/4048498470 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f52d006be30 0x7f52d006e2f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f52d40016e0 tx=0x7f52d4009040 comp rx=0 tx=0).stop 2024-04-24T23:04:53.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.936+0000 7f52e8da0640 1 -- 172.21.15.32:0/4048498470 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52e40b6ab0 msgr2=0x7f52e40d0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:53.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.936+0000 7f52e8da0640 1 --2- 172.21.15.32:0/4048498470 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52e40b6ab0 0x7f52e40d0d30 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f52d800d800 tx=0x7f52d800dcc0 comp rx=0 tx=0).stop 2024-04-24T23:04:53.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.936+0000 7f52e8da0640 1 -- 172.21.15.32:0/4048498470 shutdown_connections 2024-04-24T23:04:53.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.936+0000 7f52e8da0640 1 --2- 172.21.15.32:0/4048498470 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f52d006be30 0x7f52d006e2f0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:53.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.936+0000 7f52e8da0640 1 --2- 172.21.15.32:0/4048498470 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52e40b6ab0 0x7f52e40d0d30 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:53.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.936+0000 7f52e8da0640 1 --2- 172.21.15.32:0/4048498470 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52e4159f10 0x7f52e4155dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:53.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.936+0000 7f52e8da0640 1 --2- 172.21.15.32:0/4048498470 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f52e40b45f0 0x7f52e41f4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:53.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.936+0000 7f52e8da0640 1 -- 172.21.15.32:0/4048498470 >> 172.21.15.32:0/4048498470 conn(0x7f52e40743f0 msgr2=0x7f52e406d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:53.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.936+0000 7f52e8da0640 1 -- 172.21.15.32:0/4048498470 shutdown_connections 2024-04-24T23:04:53.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:53.936+0000 7f52e8da0640 1 -- 172.21.15.32:0/4048498470 wait complete. 2024-04-24T23:04:53.949 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-24T23:04:53.949 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-24T23:04:54.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.052+0000 7f9499f34640 1 Processor -- start 2024-04-24T23:04:54.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.052+0000 7f9499f34640 1 -- start start 2024-04-24T23:04:54.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.053+0000 7f9499f34640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f94940b9ff0 0x7f94940d40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:54.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.053+0000 7f9499f34640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f94940b6450 0x7f94940b68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:54.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.053+0000 7f9499f34640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94940d3000 con 0x7f949415e770 2024-04-24T23:04:54.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.053+0000 7f9499f34640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f94940b6df0 con 0x7f94940b6450 2024-04-24T23:04:54.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.053+0000 7f9499f34640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f94940d32e0 con 0x7f94940b9ff0 2024-04-24T23:04:54.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.054+0000 7f9493fff640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f94940b6450 0x7f94940b68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:54.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.054+0000 7f9492ffd640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f94940b9ff0 0x7f94940d40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:54.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.054+0000 7f9492ffd640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f94940b9ff0 0x7f94940d40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.32:35126/0 (socket says 172.21.15.32:35126) 2024-04-24T23:04:54.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.054+0000 7f9492ffd640 1 -- 172.21.15.32:0/4073127530 learned_addr learned my addr 172.21.15.32:0/4073127530 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:54.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.054+0000 7f94937fe640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f949415e770 0x7f94940d2700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:34990/0 (socket says 172.21.15.32:34990) 2024-04-24T23:04:54.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.054+0000 7f9493fff640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f94940b6450 0x7f94940b68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.32:53698/0 (socket says 172.21.15.32:53698) 2024-04-24T23:04:54.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.054+0000 7f9492ffd640 1 -- 172.21.15.32:0/4073127530 >> v2:172.21.15.121:3300/0 conn(0x7f94940b6450 msgr2=0x7f94940b68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:54.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.054+0000 7f9492ffd640 1 --2- 172.21.15.32:0/4073127530 >> v2:172.21.15.121:3300/0 conn(0x7f94940b6450 0x7f94940b68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:54.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.054+0000 7f9492ffd640 1 -- 172.21.15.32:0/4073127530 >> v1:172.21.15.121:6789/0 conn(0x7f949415e770 legacy=0x7f94940d2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:54.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.054+0000 7f9492ffd640 1 -- 172.21.15.32:0/4073127530 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94940d35c0 con 0x7f94940b9ff0 2024-04-24T23:04:54.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.055+0000 7f9492ffd640 1 --2- 172.21.15.32:0/4073127530 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f94940b9ff0 0x7f94940d40a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9488009870 tx=0x7f948802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91e911cff17ef756 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:54.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.055+0000 7f94927fc640 1 -- 172.21.15.32:0/4073127530 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9488037070 con 0x7f94940b9ff0 2024-04-24T23:04:54.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.055+0000 7f94927fc640 1 -- 172.21.15.32:0/4073127530 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9488032440 con 0x7f94940b9ff0 2024-04-24T23:04:54.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.055+0000 7f94927fc640 1 -- 172.21.15.32:0/4073127530 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9488032810 con 0x7f94940b9ff0 2024-04-24T23:04:54.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.055+0000 7f9499f34640 1 -- 172.21.15.32:0/4073127530 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f94940b9ff0 msgr2=0x7f94940d40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:54.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.055+0000 7f9499f34640 1 --2- 172.21.15.32:0/4073127530 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f94940b9ff0 0x7f94940d40a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9488009870 tx=0x7f948802a270 comp rx=0 tx=0).stop 2024-04-24T23:04:54.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.055+0000 7f9499f34640 1 -- 172.21.15.32:0/4073127530 shutdown_connections 2024-04-24T23:04:54.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.055+0000 7f9499f34640 1 --2- 172.21.15.32:0/4073127530 >> v2:172.21.15.121:3300/0 conn(0x7f94940b6450 0x7f94940b68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:54.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.055+0000 7f9499f34640 1 --2- 172.21.15.32:0/4073127530 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f94940b9ff0 0x7f94940d40a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:54.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.055+0000 7f9499f34640 1 -- 172.21.15.32:0/4073127530 >> 172.21.15.32:0/4073127530 conn(0x7f9494071910 msgr2=0x7f9494071d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:54.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.055+0000 7f9499f34640 1 -- 172.21.15.32:0/4073127530 shutdown_connections 2024-04-24T23:04:54.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.055+0000 7f9499f34640 1 -- 172.21.15.32:0/4073127530 wait complete. 2024-04-24T23:04:54.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.057+0000 7f9499f34640 1 Processor -- start 2024-04-24T23:04:54.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.057+0000 7f9499f34640 1 -- start start 2024-04-24T23:04:54.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.057+0000 7f9499f34640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94940b6450 0x7f9494154d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:54.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.057+0000 7f9499f34640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94940b9ff0 0x7f9494160c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:54.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.057+0000 7f9499f34640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f949415e770 0x7f949415fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:54.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.057+0000 7f9499f34640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f949415eb50 con 0x7f94940b6450 2024-04-24T23:04:54.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.057+0000 7f9499f34640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f94940d3b80 con 0x7f94940b9ff0 2024-04-24T23:04:54.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.058+0000 7f9499f34640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f949415ecd0 con 0x7f949415e770 2024-04-24T23:04:54.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.058+0000 7f94937fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94940b6450 0x7f9494154d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:54.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.058+0000 7f94937fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94940b6450 0x7f9494154d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57032/0 (socket says 172.21.15.32:57032) 2024-04-24T23:04:54.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.058+0000 7f94937fe640 1 -- 172.21.15.32:0/2964158590 learned_addr learned my addr 172.21.15.32:0/2964158590 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:54.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.058+0000 7f9492ffd640 1 --2- 172.21.15.32:0/2964158590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94940b9ff0 0x7f9494160c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:54.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.058+0000 7f9493fff640 1 --2- 172.21.15.32:0/2964158590 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f949415e770 0x7f949415fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:54.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.058+0000 7f94937fe640 1 -- 172.21.15.32:0/2964158590 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f949415e770 msgr2=0x7f949415fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:54.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.058+0000 7f94937fe640 1 --2- 172.21.15.32:0/2964158590 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f949415e770 0x7f949415fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:54.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.058+0000 7f94937fe640 1 -- 172.21.15.32:0/2964158590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94940b9ff0 msgr2=0x7f9494160c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:54.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.058+0000 7f94937fe640 1 --2- 172.21.15.32:0/2964158590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94940b9ff0 0x7f9494160c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:54.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.058+0000 7f94937fe640 1 -- 172.21.15.32:0/2964158590 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f949415efe0 con 0x7f94940b6450 2024-04-24T23:04:54.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.058+0000 7f94937fe640 1 --2- 172.21.15.32:0/2964158590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94940b6450 0x7f9494154d20 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f948400c3f0 tx=0x7f948400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:54.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.058+0000 7f947bfff640 1 -- 172.21.15.32:0/2964158590 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94840074e0 con 0x7f94940b6450 2024-04-24T23:04:54.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.058+0000 7f947bfff640 1 -- 172.21.15.32:0/2964158590 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9484007680 con 0x7f94940b6450 2024-04-24T23:04:54.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.058+0000 7f947bfff640 1 -- 172.21.15.32:0/2964158590 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9484004a00 con 0x7f94940b6450 2024-04-24T23:04:54.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.058+0000 7f9499f34640 1 -- 172.21.15.32:0/2964158590 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94941614e0 con 0x7f94940b6450 2024-04-24T23:04:54.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.058+0000 7f9499f34640 1 -- 172.21.15.32:0/2964158590 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f949415f560 con 0x7f94940b6450 2024-04-24T23:04:54.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.059+0000 7f9499f34640 1 -- 172.21.15.32:0/2964158590 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9494068430 con 0x7f94940b6450 2024-04-24T23:04:54.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.060+0000 7f947bfff640 1 -- 172.21.15.32:0/2964158590 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f948401a020 con 0x7f94940b6450 2024-04-24T23:04:54.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.060+0000 7f947bfff640 1 --2- 172.21.15.32:0/2964158590 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f948006be30 0x7f948006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:54.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.061+0000 7f947bfff640 1 -- 172.21.15.32:0/2964158590 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 4857+0+0 (secure 0 0 0) 0x7f948408a310 con 0x7f94940b6450 2024-04-24T23:04:54.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.061+0000 7f9492ffd640 1 --2- 172.21.15.32:0/2964158590 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f948006be30 0x7f948006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:54.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.061+0000 7f9492ffd640 1 --2- 172.21.15.32:0/2964158590 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f948006be30 0x7f948006e2f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f9488034000 tx=0x7f948802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:54.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.065+0000 7f947bfff640 1 -- 172.21.15.32:0/2964158590 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9484010300 con 0x7f94940b6450 2024-04-24T23:04:54.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.249+0000 7f9499f34640 1 -- 172.21.15.32:0/2964158590 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f949415df00 con 0x7f94940b6450 2024-04-24T23:04:54.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.613+0000 7f947bfff640 1 -- 172.21.15.32:0/2964158590 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v122) ==== 138+0+0 (secure 0 0 0) 0x7f9484059000 con 0x7f94940b6450 2024-04-24T23:04:54.615 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2024-04-24T23:04:54.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.616+0000 7f9499f34640 1 -- 172.21.15.32:0/2964158590 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f948006be30 msgr2=0x7f948006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:54.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.616+0000 7f9499f34640 1 --2- 172.21.15.32:0/2964158590 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f948006be30 0x7f948006e2f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f9488034000 tx=0x7f948802e000 comp rx=0 tx=0).stop 2024-04-24T23:04:54.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.616+0000 7f9499f34640 1 -- 172.21.15.32:0/2964158590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94940b6450 msgr2=0x7f9494154d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:54.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.616+0000 7f9499f34640 1 --2- 172.21.15.32:0/2964158590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94940b6450 0x7f9494154d20 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f948400c3f0 tx=0x7f948400c8b0 comp rx=0 tx=0).stop 2024-04-24T23:04:54.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.617+0000 7f9499f34640 1 -- 172.21.15.32:0/2964158590 shutdown_connections 2024-04-24T23:04:54.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.617+0000 7f9499f34640 1 --2- 172.21.15.32:0/2964158590 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f948006be30 0x7f948006e2f0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:54.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.617+0000 7f9499f34640 1 --2- 172.21.15.32:0/2964158590 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f949415e770 0x7f949415fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:54.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.617+0000 7f9499f34640 1 --2- 172.21.15.32:0/2964158590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94940b9ff0 0x7f9494160c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:54.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.617+0000 7f9499f34640 1 --2- 172.21.15.32:0/2964158590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94940b6450 0x7f9494154d20 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:54.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.617+0000 7f9499f34640 1 -- 172.21.15.32:0/2964158590 >> 172.21.15.32:0/2964158590 conn(0x7f9494071910 msgr2=0x7f94940d3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:54.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.617+0000 7f9499f34640 1 -- 172.21.15.32:0/2964158590 shutdown_connections 2024-04-24T23:04:54.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.617+0000 7f9499f34640 1 -- 172.21.15.32:0/2964158590 wait complete. 2024-04-24T23:04:54.630 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-24T23:04:54.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.738+0000 7faf3302a640 1 Processor -- start 2024-04-24T23:04:54.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.739+0000 7faf3302a640 1 -- start start 2024-04-24T23:04:54.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.739+0000 7faf3302a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faf2c06b820 0x7faf2c0d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:54.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.739+0000 7faf3302a640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7faf2c0d4210 0x7faf2c0b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:54.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.739+0000 7faf3302a640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf2c0d3170 con 0x7faf2c15e9b0 2024-04-24T23:04:54.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.739+0000 7faf3302a640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7faf2c0d3450 con 0x7faf2c0d4210 2024-04-24T23:04:54.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.739+0000 7faf3302a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7faf2c0d3730 con 0x7faf2c06b820 2024-04-24T23:04:54.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.739+0000 7faf30dc6640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7faf2c15e9b0 0x7faf2c06b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49500/0 (socket says 172.21.15.32:49500) 2024-04-24T23:04:54.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.740+0000 7faf30dc6640 1 -- 172.21.15.32:0/3917722546 learned_addr learned my addr 172.21.15.32:0/3917722546 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:04:54.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.740+0000 7faf315c7640 1 --2- 172.21.15.32:0/3917722546 >> v2:172.21.15.121:3300/0 conn(0x7faf2c0d4210 0x7faf2c0b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:54.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.740+0000 7faf2bfff640 1 --2- 172.21.15.32:0/3917722546 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faf2c06b820 0x7faf2c0d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:54.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.740+0000 7faf2b7fe640 1 -- 172.21.15.32:0/3917722546 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2449783447 0 0) 0x7faf2c0d3170 con 0x7faf2c15e9b0 2024-04-24T23:04:54.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.740+0000 7faf2b7fe640 1 -- 172.21.15.32:0/3917722546 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faf1c003520 con 0x7faf2c15e9b0 2024-04-24T23:04:54.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.740+0000 7faf2b7fe640 1 -- 172.21.15.32:0/3917722546 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3477190347 0 0) 0x7faf1c003520 con 0x7faf2c15e9b0 2024-04-24T23:04:54.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.740+0000 7faf2b7fe640 1 -- 172.21.15.32:0/3917722546 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faf2c06b820 msgr2=0x7faf2c0d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:54.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.740+0000 7faf2b7fe640 1 --2- 172.21.15.32:0/3917722546 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faf2c06b820 0x7faf2c0d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:54.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.740+0000 7faf2b7fe640 1 -- 172.21.15.32:0/3917722546 >> v2:172.21.15.121:3300/0 conn(0x7faf2c0d4210 msgr2=0x7faf2c0b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:54.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.740+0000 7faf2b7fe640 1 --2- 172.21.15.32:0/3917722546 >> v2:172.21.15.121:3300/0 conn(0x7faf2c0d4210 0x7faf2c0b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:54.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.740+0000 7faf2b7fe640 1 -- 172.21.15.32:0/3917722546 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf2c0d3a10 con 0x7faf2c15e9b0 2024-04-24T23:04:54.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.740+0000 7faf2b7fe640 1 -- 172.21.15.32:0/3917722546 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7faf20001670 con 0x7faf2c15e9b0 2024-04-24T23:04:54.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.740+0000 7faf2b7fe640 1 -- 172.21.15.32:0/3917722546 >> v1:172.21.15.32:6789/0 conn(0x7faf2c15e9b0 legacy=0x7faf2c06b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:54.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.740+0000 7faf2b7fe640 1 --2- 172.21.15.32:0/3917722546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf1c0039b0 0x7faf1c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:54.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.740+0000 7faf2b7fe640 1 --2- 172.21.15.32:0/3917722546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf1c004800 0x7faf1c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:54.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.740+0000 7faf2b7fe640 1 --2- 172.21.15.32:0/3917722546 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faf2c06b820 0x7faf1c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:54.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.740+0000 7faf2b7fe640 1 -- 172.21.15.32:0/3917722546 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7faf2c0d3450 con 0x7faf1c0039b0 2024-04-24T23:04:54.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.740+0000 7faf2b7fe640 1 -- 172.21.15.32:0/3917722546 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7faf2c0d3730 con 0x7faf1c004800 2024-04-24T23:04:54.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.740+0000 7faf2b7fe640 1 -- 172.21.15.32:0/3917722546 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7faf1c00b9c0 con 0x7faf2c06b820 2024-04-24T23:04:54.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.740+0000 7faf30dc6640 1 --2- 172.21.15.32:0/3917722546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf1c0039b0 0x7faf1c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:54.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.740+0000 7faf2bfff640 1 --2- 172.21.15.32:0/3917722546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf1c004800 0x7faf1c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:54.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.740+0000 7faf315c7640 1 --2- 172.21.15.32:0/3917722546 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faf2c06b820 0x7faf1c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:54.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.740+0000 7faf30dc6640 1 -- 172.21.15.32:0/3917722546 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faf2c06b820 msgr2=0x7faf1c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:54.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.740+0000 7faf30dc6640 1 --2- 172.21.15.32:0/3917722546 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faf2c06b820 0x7faf1c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:54.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.740+0000 7faf30dc6640 1 -- 172.21.15.32:0/3917722546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf1c004800 msgr2=0x7faf1c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:54.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.740+0000 7faf30dc6640 1 --2- 172.21.15.32:0/3917722546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf1c004800 0x7faf1c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:54.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.740+0000 7faf30dc6640 1 -- 172.21.15.32:0/3917722546 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf1c003520 con 0x7faf1c0039b0 2024-04-24T23:04:54.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.740+0000 7faf2bfff640 1 --2- 172.21.15.32:0/3917722546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf1c004800 0x7faf1c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:04:54.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.741+0000 7faf30dc6640 1 --2- 172.21.15.32:0/3917722546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf1c0039b0 0x7faf1c003dd0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7faf2000c270 tx=0x7faf1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fa2643dbe3d1606 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:54.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.741+0000 7faf2b7fe640 1 -- 172.21.15.32:0/3917722546 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf20007d30 con 0x7faf1c0039b0 2024-04-24T23:04:54.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.741+0000 7faf2b7fe640 1 -- 172.21.15.32:0/3917722546 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf20005ad0 con 0x7faf1c0039b0 2024-04-24T23:04:54.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.741+0000 7faf2b7fe640 1 -- 172.21.15.32:0/3917722546 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf20007070 con 0x7faf1c0039b0 2024-04-24T23:04:54.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.742+0000 7faf3302a640 1 -- 172.21.15.32:0/3917722546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf1c0039b0 msgr2=0x7faf1c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:54.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.742+0000 7faf3302a640 1 --2- 172.21.15.32:0/3917722546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf1c0039b0 0x7faf1c003dd0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7faf2000c270 tx=0x7faf1c005830 comp rx=0 tx=0).stop 2024-04-24T23:04:54.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.742+0000 7faf315c7640 1 -- 172.21.15.32:0/3917722546 reap_dead start 2024-04-24T23:04:54.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.742+0000 7faf3302a640 1 -- 172.21.15.32:0/3917722546 shutdown_connections 2024-04-24T23:04:54.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.742+0000 7faf3302a640 1 -- 172.21.15.32:0/3917722546 >> 172.21.15.32:0/3917722546 conn(0x7faf2c0d11d0 msgr2=0x7faf2c0d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:54.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.742+0000 7faf3302a640 1 -- 172.21.15.32:0/3917722546 shutdown_connections 2024-04-24T23:04:54.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.742+0000 7faf3302a640 1 -- 172.21.15.32:0/3917722546 wait complete. 2024-04-24T23:04:54.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.743+0000 7faf3302a640 1 Processor -- start 2024-04-24T23:04:54.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.743+0000 7faf3302a640 1 -- start start 2024-04-24T23:04:54.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.744+0000 7faf3302a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf1c0039b0 0x7faf2c0ce3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:54.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.744+0000 7faf3302a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf2c1432f0 0x7faf2c1f3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:54.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.744+0000 7faf3302a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faf2c143770 0x7faf2c1f3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:54.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.744+0000 7faf3302a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7faf2c0ba0e0 con 0x7faf2c1432f0 2024-04-24T23:04:54.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.744+0000 7faf3302a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7faf2c0d2400 con 0x7faf1c0039b0 2024-04-24T23:04:54.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.744+0000 7faf3302a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7faf2c0ba260 con 0x7faf2c143770 2024-04-24T23:04:54.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.744+0000 7faf2bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf2c1432f0 0x7faf2c1f3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:54.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.744+0000 7faf2bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf2c1432f0 0x7faf2c1f3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57048/0 (socket says 172.21.15.32:57048) 2024-04-24T23:04:54.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.744+0000 7faf2bfff640 1 -- 172.21.15.32:0/707418352 learned_addr learned my addr 172.21.15.32:0/707418352 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:54.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.744+0000 7faf315c7640 1 --2- 172.21.15.32:0/707418352 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faf2c143770 0x7faf2c1f3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:54.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.744+0000 7faf30dc6640 1 --2- 172.21.15.32:0/707418352 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf1c0039b0 0x7faf2c0ce3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:54.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.744+0000 7faf2bfff640 1 -- 172.21.15.32:0/707418352 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faf2c143770 msgr2=0x7faf2c1f3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:54.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.744+0000 7faf2bfff640 1 --2- 172.21.15.32:0/707418352 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faf2c143770 0x7faf2c1f3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:54.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.744+0000 7faf2bfff640 1 -- 172.21.15.32:0/707418352 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf1c0039b0 msgr2=0x7faf2c0ce3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:54.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.744+0000 7faf2bfff640 1 --2- 172.21.15.32:0/707418352 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf1c0039b0 0x7faf2c0ce3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:54.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.744+0000 7faf2bfff640 1 -- 172.21.15.32:0/707418352 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf2c1f44f0 con 0x7faf2c1432f0 2024-04-24T23:04:54.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.745+0000 7faf2bfff640 1 --2- 172.21.15.32:0/707418352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf2c1432f0 0x7faf2c1f3970 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7faf18011570 tx=0x7faf18011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:54.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.745+0000 7faf297fa640 1 -- 172.21.15.32:0/707418352 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf18015870 con 0x7faf2c1432f0 2024-04-24T23:04:54.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.745+0000 7faf297fa640 1 -- 172.21.15.32:0/707418352 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf18015a10 con 0x7faf2c1432f0 2024-04-24T23:04:54.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.745+0000 7faf3302a640 1 -- 172.21.15.32:0/707418352 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf2c1f47d0 con 0x7faf2c1432f0 2024-04-24T23:04:54.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.745+0000 7faf297fa640 1 -- 172.21.15.32:0/707418352 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf18015dc0 con 0x7faf2c1432f0 2024-04-24T23:04:54.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.745+0000 7faf3302a640 1 -- 172.21.15.32:0/707418352 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf2c0cbae0 con 0x7faf2c1432f0 2024-04-24T23:04:54.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.746+0000 7faf3302a640 1 -- 172.21.15.32:0/707418352 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf2c068430 con 0x7faf2c1432f0 2024-04-24T23:04:54.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.746+0000 7faf297fa640 1 -- 172.21.15.32:0/707418352 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faf1801d400 con 0x7faf2c1432f0 2024-04-24T23:04:54.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.747+0000 7faf297fa640 1 --2- 172.21.15.32:0/707418352 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7faf1006be30 0x7faf1006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:54.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.747+0000 7faf297fa640 1 -- 172.21.15.32:0/707418352 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5227+0+0 (secure 0 0 0) 0x7faf1808b490 con 0x7faf2c1432f0 2024-04-24T23:04:54.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.747+0000 7faf30dc6640 1 --2- 172.21.15.32:0/707418352 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7faf1006be30 0x7faf1006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:54.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.748+0000 7faf30dc6640 1 --2- 172.21.15.32:0/707418352 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7faf1006be30 0x7faf1006e2f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7faf20033000 tx=0x7faf20009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:54.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.751+0000 7faf297fa640 1 -- 172.21.15.32:0/707418352 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7faf18019030 con 0x7faf2c1432f0 2024-04-24T23:04:54.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:54.934+0000 7faf3302a640 1 -- 172.21.15.32:0/707418352 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7faf2c074df0 con 0x7faf2c1432f0 2024-04-24T23:04:55.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.622+0000 7faf297fa640 1 -- 172.21.15.32:0/707418352 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v123) ==== 170+0+0 (secure 0 0 0) 0x7faf1805a020 con 0x7faf2c1432f0 2024-04-24T23:04:55.624 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2024-04-24T23:04:55.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.626+0000 7faf3302a640 1 -- 172.21.15.32:0/707418352 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7faf1006be30 msgr2=0x7faf1006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:55.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.626+0000 7faf3302a640 1 --2- 172.21.15.32:0/707418352 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7faf1006be30 0x7faf1006e2f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7faf20033000 tx=0x7faf20009030 comp rx=0 tx=0).stop 2024-04-24T23:04:55.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.626+0000 7faf3302a640 1 -- 172.21.15.32:0/707418352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf2c1432f0 msgr2=0x7faf2c1f3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:55.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.626+0000 7faf3302a640 1 --2- 172.21.15.32:0/707418352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf2c1432f0 0x7faf2c1f3970 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7faf18011570 tx=0x7faf18011a30 comp rx=0 tx=0).stop 2024-04-24T23:04:55.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.627+0000 7faf3302a640 1 -- 172.21.15.32:0/707418352 shutdown_connections 2024-04-24T23:04:55.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.627+0000 7faf3302a640 1 --2- 172.21.15.32:0/707418352 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7faf1006be30 0x7faf1006e2f0 secure :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7faf20033000 tx=0x7faf20009030 comp rx=0 tx=0).stop 2024-04-24T23:04:55.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.627+0000 7faf3302a640 1 --2- 172.21.15.32:0/707418352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf2c1432f0 0x7faf2c1f3970 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:55.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.627+0000 7faf3302a640 1 --2- 172.21.15.32:0/707418352 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faf2c143770 0x7faf2c1f3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:55.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.627+0000 7faf3302a640 1 --2- 172.21.15.32:0/707418352 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf1c0039b0 0x7faf2c0ce3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:55.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.627+0000 7faf3302a640 1 -- 172.21.15.32:0/707418352 >> 172.21.15.32:0/707418352 conn(0x7faf2c0d11d0 msgr2=0x7faf2c06b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:55.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.628+0000 7faf3302a640 1 -- 172.21.15.32:0/707418352 shutdown_connections 2024-04-24T23:04:55.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.630+0000 7faf3302a640 1 -- 172.21.15.32:0/707418352 wait complete. 2024-04-24T23:04:55.650 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-24T23:04:55.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.757+0000 7ff137f10640 1 Processor -- start 2024-04-24T23:04:55.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.757+0000 7ff137f10640 1 -- start start 2024-04-24T23:04:55.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.758+0000 7ff137f10640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff13015e9b0 0x7ff1300d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:55.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.758+0000 7ff137f10640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff1300d4210 0x7ff13006b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:55.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.758+0000 7ff137f10640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff1300ba0e0 0x7ff13006b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:55.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.758+0000 7ff137f10640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7ff1300d3170 con 0x7ff1300ba0e0 2024-04-24T23:04:55.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.758+0000 7ff137f10640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7ff1300d3450 con 0x7ff13015e9b0 2024-04-24T23:04:55.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.758+0000 7ff137f10640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff1300d3730 con 0x7ff1300d4210 2024-04-24T23:04:55.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.758+0000 7ff1364ad640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff1300ba0e0 0x7ff13006b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:55.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.758+0000 7ff1364ad640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff1300ba0e0 0x7ff13006b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57064/0 (socket says 172.21.15.32:57064) 2024-04-24T23:04:55.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.758+0000 7ff1364ad640 1 -- 172.21.15.32:0/2248561909 learned_addr learned my addr 172.21.15.32:0/2248561909 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:55.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.759+0000 7ff135cac640 1 --2- 172.21.15.32:0/2248561909 >> v2:172.21.15.121:3300/0 conn(0x7ff13015e9b0 0x7ff1300d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:55.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.759+0000 7ff1354ab640 1 --2- 172.21.15.32:0/2248561909 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff1300d4210 0x7ff13006b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:55.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.759+0000 7ff1364ad640 1 -- 172.21.15.32:0/2248561909 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff1300d4210 msgr2=0x7ff13006b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:55.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.759+0000 7ff1364ad640 1 --2- 172.21.15.32:0/2248561909 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff1300d4210 0x7ff13006b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:55.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.759+0000 7ff1364ad640 1 -- 172.21.15.32:0/2248561909 >> v2:172.21.15.121:3300/0 conn(0x7ff13015e9b0 msgr2=0x7ff1300d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:55.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.759+0000 7ff1364ad640 1 --2- 172.21.15.32:0/2248561909 >> v2:172.21.15.121:3300/0 conn(0x7ff13015e9b0 0x7ff1300d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:55.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.759+0000 7ff1364ad640 1 -- 172.21.15.32:0/2248561909 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1300d3a10 con 0x7ff1300ba0e0 2024-04-24T23:04:55.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.759+0000 7ff1364ad640 1 --2- 172.21.15.32:0/2248561909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1300ba0e0 0x7ff13006b650 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7ff13006be20 tx=0x7ff12002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62225077ed1a15f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:55.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.760+0000 7ff134caa640 1 -- 172.21.15.32:0/2248561909 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff120036070 con 0x7ff1300ba0e0 2024-04-24T23:04:55.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.760+0000 7ff134caa640 1 -- 172.21.15.32:0/2248561909 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff120031600 con 0x7ff1300ba0e0 2024-04-24T23:04:55.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.760+0000 7ff134caa640 1 -- 172.21.15.32:0/2248561909 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1200319d0 con 0x7ff1300ba0e0 2024-04-24T23:04:55.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.760+0000 7ff137f10640 1 -- 172.21.15.32:0/2248561909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1300ba0e0 msgr2=0x7ff13006b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:55.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.760+0000 7ff137f10640 1 --2- 172.21.15.32:0/2248561909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1300ba0e0 0x7ff13006b650 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7ff13006be20 tx=0x7ff12002a6a0 comp rx=0 tx=0).stop 2024-04-24T23:04:55.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.760+0000 7ff137f10640 1 -- 172.21.15.32:0/2248561909 shutdown_connections 2024-04-24T23:04:55.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.760+0000 7ff137f10640 1 --2- 172.21.15.32:0/2248561909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1300ba0e0 0x7ff13006b650 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:55.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.760+0000 7ff137f10640 1 --2- 172.21.15.32:0/2248561909 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff1300d4210 0x7ff13006b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:55.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.760+0000 7ff137f10640 1 --2- 172.21.15.32:0/2248561909 >> v2:172.21.15.121:3300/0 conn(0x7ff13015e9b0 0x7ff1300d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:55.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.760+0000 7ff137f10640 1 -- 172.21.15.32:0/2248561909 >> 172.21.15.32:0/2248561909 conn(0x7ff1300d11d0 msgr2=0x7ff1300d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:55.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.760+0000 7ff137f10640 1 -- 172.21.15.32:0/2248561909 shutdown_connections 2024-04-24T23:04:55.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.760+0000 7ff137f10640 1 -- 172.21.15.32:0/2248561909 wait complete. 2024-04-24T23:04:55.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.761+0000 7ff137f10640 1 Processor -- start 2024-04-24T23:04:55.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.762+0000 7ff137f10640 1 -- start start 2024-04-24T23:04:55.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.762+0000 7ff137f10640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff1300ba0e0 0x7ff1301f4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:55.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.762+0000 7ff137f10640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1300d4210 0x7ff1300cd550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:55.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.762+0000 7ff137f10640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff13015e9b0 0x7ff1300cda90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:55.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.762+0000 7ff137f10640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff130071730 con 0x7ff1300d4210 2024-04-24T23:04:55.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.762+0000 7ff137f10640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff1300d2400 con 0x7ff1300ba0e0 2024-04-24T23:04:55.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.762+0000 7ff137f10640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff1300718b0 con 0x7ff13015e9b0 2024-04-24T23:04:55.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.762+0000 7ff1354ab640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1300d4210 0x7ff1300cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:55.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.762+0000 7ff1354ab640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1300d4210 0x7ff1300cd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57078/0 (socket says 172.21.15.32:57078) 2024-04-24T23:04:55.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.762+0000 7ff1354ab640 1 -- 172.21.15.32:0/888554530 learned_addr learned my addr 172.21.15.32:0/888554530 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:55.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.763+0000 7ff1364ad640 1 --2- 172.21.15.32:0/888554530 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff13015e9b0 0x7ff1300cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:55.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.763+0000 7ff135cac640 1 --2- 172.21.15.32:0/888554530 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff1300ba0e0 0x7ff1301f4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:55.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.763+0000 7ff1354ab640 1 -- 172.21.15.32:0/888554530 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff13015e9b0 msgr2=0x7ff1300cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:55.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.763+0000 7ff1354ab640 1 --2- 172.21.15.32:0/888554530 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff13015e9b0 0x7ff1300cda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:55.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.763+0000 7ff1354ab640 1 -- 172.21.15.32:0/888554530 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff1300ba0e0 msgr2=0x7ff1301f4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:55.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.763+0000 7ff1354ab640 1 --2- 172.21.15.32:0/888554530 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff1300ba0e0 0x7ff1301f4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:55.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.763+0000 7ff1354ab640 1 -- 172.21.15.32:0/888554530 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1300ce660 con 0x7ff1300d4210 2024-04-24T23:04:55.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.763+0000 7ff135cac640 1 --2- 172.21.15.32:0/888554530 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff1300ba0e0 0x7ff1301f4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:04:55.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.763+0000 7ff1364ad640 1 --2- 172.21.15.32:0/888554530 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff13015e9b0 0x7ff1300cda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:04:55.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.763+0000 7ff1354ab640 1 --2- 172.21.15.32:0/888554530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1300d4210 0x7ff1300cd550 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7ff12400d830 tx=0x7ff12400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:55.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.763+0000 7ff12e7fc640 1 -- 172.21.15.32:0/888554530 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff124018020 con 0x7ff1300d4210 2024-04-24T23:04:55.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.763+0000 7ff12e7fc640 1 -- 172.21.15.32:0/888554530 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1240096a0 con 0x7ff1300d4210 2024-04-24T23:04:55.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.763+0000 7ff137f10640 1 -- 172.21.15.32:0/888554530 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1300c83b0 con 0x7ff1300d4210 2024-04-24T23:04:55.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.763+0000 7ff12e7fc640 1 -- 172.21.15.32:0/888554530 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff124009a70 con 0x7ff1300d4210 2024-04-24T23:04:55.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.763+0000 7ff137f10640 1 -- 172.21.15.32:0/888554530 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1300ce380 con 0x7ff1300d4210 2024-04-24T23:04:55.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.764+0000 7ff137f10640 1 -- 172.21.15.32:0/888554530 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff130068430 con 0x7ff1300d4210 2024-04-24T23:04:55.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.765+0000 7ff12e7fc640 1 -- 172.21.15.32:0/888554530 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff12401b020 con 0x7ff1300d4210 2024-04-24T23:04:55.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.765+0000 7ff12e7fc640 1 --2- 172.21.15.32:0/888554530 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff11c06be30 0x7ff11c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:55.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.765+0000 7ff12e7fc640 1 -- 172.21.15.32:0/888554530 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5609+0+0 (secure 0 0 0) 0x7ff12408b1c0 con 0x7ff1300d4210 2024-04-24T23:04:55.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.765+0000 7ff135cac640 1 --2- 172.21.15.32:0/888554530 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff11c06be30 0x7ff11c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:55.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.766+0000 7ff135cac640 1 --2- 172.21.15.32:0/888554530 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff11c06be30 0x7ff11c06e2f0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7ff128006fd0 tx=0x7ff128008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:55.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.770+0000 7ff12e7fc640 1 -- 172.21.15.32:0/888554530 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff124022030 con 0x7ff1300d4210 2024-04-24T23:04:55.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:55.996+0000 7ff137f10640 1 -- 172.21.15.32:0/888554530 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7ff13014f3b0 con 0x7ff1300d4210 2024-04-24T23:04:56.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.623+0000 7ff12e7fc640 1 -- 172.21.15.32:0/888554530 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v48) ==== 174+0+0 (secure 0 0 0) 0x7ff1240102e0 con 0x7ff1300d4210 2024-04-24T23:04:56.625 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 14 and data pool 15 2024-04-24T23:04:56.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.626+0000 7ff137f10640 1 -- 172.21.15.32:0/888554530 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff11c06be30 msgr2=0x7ff11c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:56.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.626+0000 7ff137f10640 1 --2- 172.21.15.32:0/888554530 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff11c06be30 0x7ff11c06e2f0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7ff128006fd0 tx=0x7ff128008040 comp rx=0 tx=0).stop 2024-04-24T23:04:56.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.626+0000 7ff137f10640 1 -- 172.21.15.32:0/888554530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1300d4210 msgr2=0x7ff1300cd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:56.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.626+0000 7ff137f10640 1 --2- 172.21.15.32:0/888554530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1300d4210 0x7ff1300cd550 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7ff12400d830 tx=0x7ff12400dcf0 comp rx=0 tx=0).stop 2024-04-24T23:04:56.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.626+0000 7ff137f10640 1 -- 172.21.15.32:0/888554530 shutdown_connections 2024-04-24T23:04:56.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.626+0000 7ff137f10640 1 --2- 172.21.15.32:0/888554530 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff11c06be30 0x7ff11c06e2f0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:56.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.626+0000 7ff137f10640 1 --2- 172.21.15.32:0/888554530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1300d4210 0x7ff1300cd550 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:56.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.626+0000 7ff137f10640 1 --2- 172.21.15.32:0/888554530 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff13015e9b0 0x7ff1300cda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:56.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.626+0000 7ff137f10640 1 --2- 172.21.15.32:0/888554530 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff1300ba0e0 0x7ff1301f4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:56.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.626+0000 7ff137f10640 1 -- 172.21.15.32:0/888554530 >> 172.21.15.32:0/888554530 conn(0x7ff1300d11d0 msgr2=0x7ff1300b78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:56.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.627+0000 7ff137f10640 1 -- 172.21.15.32:0/888554530 shutdown_connections 2024-04-24T23:04:56.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.627+0000 7ff137f10640 1 -- 172.21.15.32:0/888554530 wait complete. 2024-04-24T23:04:56.643 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:04:56.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.744+0000 7f7058641640 1 Processor -- start 2024-04-24T23:04:56.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.744+0000 7f7058641640 1 -- start start 2024-04-24T23:04:56.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.745+0000 7f7058641640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f705006c5f0 0x7f705006c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:56.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.745+0000 7f7058641640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f70500af890 0x7f705015cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:56.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.745+0000 7f7058641640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70500b6430 con 0x7f70500b44d0 2024-04-24T23:04:56.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.745+0000 7f7058641640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f7050156f80 con 0x7f70500af890 2024-04-24T23:04:56.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.745+0000 7f7058641640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f70500b6150 con 0x7f705006c5f0 2024-04-24T23:04:56.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.745+0000 7f7056bde640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f70500af890 0x7f705015cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:56.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.745+0000 7f7056bde640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f70500af890 0x7f705015cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57084/0 (socket says 172.21.15.32:57084) 2024-04-24T23:04:56.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.745+0000 7f7056bde640 1 -- 172.21.15.32:0/3021868461 learned_addr learned my addr 172.21.15.32:0/3021868461 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:56.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.745+0000 7f70563dd640 1 --2- 172.21.15.32:0/3021868461 >> v2:172.21.15.121:3300/0 conn(0x7f705006c5f0 0x7f705006c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:56.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.745+0000 7f7056bde640 1 -- 172.21.15.32:0/3021868461 >> v2:172.21.15.121:3300/0 conn(0x7f705006c5f0 msgr2=0x7f705006c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:56.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.745+0000 7f7056bde640 1 --2- 172.21.15.32:0/3021868461 >> v2:172.21.15.121:3300/0 conn(0x7f705006c5f0 0x7f705006c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:56.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.745+0000 7f7056bde640 1 -- 172.21.15.32:0/3021868461 >> v1:172.21.15.121:6789/0 conn(0x7f70500b44d0 legacy=0x7f70500b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:56.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.745+0000 7f7056bde640 1 -- 172.21.15.32:0/3021868461 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70500b39f0 con 0x7f70500af890 2024-04-24T23:04:56.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.746+0000 7f7056bde640 1 --2- 172.21.15.32:0/3021868461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70500af890 0x7f705015cac0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f705006c1f0 tx=0x7f704802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9f1786c306fd552 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:56.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.746+0000 7f70553db640 1 -- 172.21.15.32:0/3021868461 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7048036070 con 0x7f70500af890 2024-04-24T23:04:56.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.746+0000 7f70553db640 1 -- 172.21.15.32:0/3021868461 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7048031600 con 0x7f70500af890 2024-04-24T23:04:56.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.746+0000 7f70553db640 1 -- 172.21.15.32:0/3021868461 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70480319d0 con 0x7f70500af890 2024-04-24T23:04:56.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.747+0000 7f7058641640 1 -- 172.21.15.32:0/3021868461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70500af890 msgr2=0x7f705015cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:56.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.747+0000 7f7058641640 1 --2- 172.21.15.32:0/3021868461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70500af890 0x7f705015cac0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f705006c1f0 tx=0x7f704802a730 comp rx=0 tx=0).stop 2024-04-24T23:04:56.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.747+0000 7f7058641640 1 -- 172.21.15.32:0/3021868461 shutdown_connections 2024-04-24T23:04:56.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.747+0000 7f7058641640 1 --2- 172.21.15.32:0/3021868461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70500af890 0x7f705015cac0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:56.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.747+0000 7f7058641640 1 --2- 172.21.15.32:0/3021868461 >> v2:172.21.15.121:3300/0 conn(0x7f705006c5f0 0x7f705006c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:56.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.747+0000 7f7058641640 1 -- 172.21.15.32:0/3021868461 >> 172.21.15.32:0/3021868461 conn(0x7f705006b110 msgr2=0x7f705006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:56.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.747+0000 7f7058641640 1 -- 172.21.15.32:0/3021868461 shutdown_connections 2024-04-24T23:04:56.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.747+0000 7f7058641640 1 -- 172.21.15.32:0/3021868461 wait complete. 2024-04-24T23:04:56.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.748+0000 7f7058641640 1 Processor -- start 2024-04-24T23:04:56.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.748+0000 7f7058641640 1 -- start start 2024-04-24T23:04:56.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.749+0000 7f7058641640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f705006c5f0 0x7f7050145de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:56.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.749+0000 7f7058641640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f70500af890 0x7f705015e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:56.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.749+0000 7f7058641640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70500b44d0 0x7f7050150db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:56.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.749+0000 7f7058641640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f705006cf10 con 0x7f70500b44d0 2024-04-24T23:04:56.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.749+0000 7f7058641640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f70500b3fb0 con 0x7f705006c5f0 2024-04-24T23:04:56.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.749+0000 7f7058641640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f705015d000 con 0x7f70500af890 2024-04-24T23:04:56.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.749+0000 7f7056bde640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70500b44d0 0x7f7050150db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:56.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.749+0000 7f7056bde640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70500b44d0 0x7f7050150db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57096/0 (socket says 172.21.15.32:57096) 2024-04-24T23:04:56.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.749+0000 7f7056bde640 1 -- 172.21.15.32:0/1178669989 learned_addr learned my addr 172.21.15.32:0/1178669989 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:56.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.749+0000 7f70563dd640 1 --2- 172.21.15.32:0/1178669989 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f705006c5f0 0x7f7050145de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:56.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.749+0000 7f7055bdc640 1 --2- 172.21.15.32:0/1178669989 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f70500af890 0x7f705015e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:56.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.749+0000 7f7056bde640 1 -- 172.21.15.32:0/1178669989 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f70500af890 msgr2=0x7f705015e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:56.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.749+0000 7f7056bde640 1 --2- 172.21.15.32:0/1178669989 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f70500af890 0x7f705015e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:56.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.749+0000 7f7056bde640 1 -- 172.21.15.32:0/1178669989 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f705006c5f0 msgr2=0x7f7050145de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:56.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.749+0000 7f7056bde640 1 --2- 172.21.15.32:0/1178669989 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f705006c5f0 0x7f7050145de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:56.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.749+0000 7f7056bde640 1 -- 172.21.15.32:0/1178669989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f705015f030 con 0x7f70500b44d0 2024-04-24T23:04:56.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.749+0000 7f70563dd640 1 --2- 172.21.15.32:0/1178669989 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f705006c5f0 0x7f7050145de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:04:56.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.749+0000 7f7056bde640 1 --2- 172.21.15.32:0/1178669989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70500b44d0 0x7f7050150db0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f704802ad20 tx=0x7f7048004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:56.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.750+0000 7f7046ffd640 1 -- 172.21.15.32:0/1178669989 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7048036050 con 0x7f70500b44d0 2024-04-24T23:04:56.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.750+0000 7f7046ffd640 1 -- 172.21.15.32:0/1178669989 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7048042070 con 0x7f70500b44d0 2024-04-24T23:04:56.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.750+0000 7f7058641640 1 -- 172.21.15.32:0/1178669989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f705015f310 con 0x7f70500b44d0 2024-04-24T23:04:56.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.750+0000 7f7046ffd640 1 -- 172.21.15.32:0/1178669989 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7048009510 con 0x7f70500b44d0 2024-04-24T23:04:56.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.750+0000 7f7058641640 1 -- 172.21.15.32:0/1178669989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7050146320 con 0x7f70500b44d0 2024-04-24T23:04:56.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.751+0000 7f7058641640 1 -- 172.21.15.32:0/1178669989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7050068430 con 0x7f70500b44d0 2024-04-24T23:04:56.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.751+0000 7f7046ffd640 1 -- 172.21.15.32:0/1178669989 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7048007bf0 con 0x7f70500b44d0 2024-04-24T23:04:56.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.751+0000 7f7046ffd640 1 --2- 172.21.15.32:0/1178669989 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f703406be30 0x7f703406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:56.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.751+0000 7f7046ffd640 1 -- 172.21.15.32:0/1178669989 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5709+0+0 (secure 0 0 0) 0x7f70480abbc0 con 0x7f70500b44d0 2024-04-24T23:04:56.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.752+0000 7f70563dd640 1 --2- 172.21.15.32:0/1178669989 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f703406be30 0x7f703406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:56.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.752+0000 7f70563dd640 1 --2- 172.21.15.32:0/1178669989 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f703406be30 0x7f703406e2f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f704c006fd0 tx=0x7f704c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:56.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.756+0000 7f7046ffd640 1 -- 172.21.15.32:0/1178669989 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f704807a600 con 0x7f70500b44d0 2024-04-24T23:04:56.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.944+0000 7f7058641640 1 -- 172.21.15.32:0/1178669989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f70500b5670 con 0x7f70500b44d0 2024-04-24T23:04:56.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.945+0000 7f7046ffd640 1 -- 172.21.15.32:0/1178669989 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) ==== 74+0+16645 (secure 0 0 0) 0x7f7048079f00 con 0x7f70500b44d0 2024-04-24T23:04:56.947 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:04:56.947 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":124,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:04:55.997703+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:04:54.280785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-24T23:04:54.966294+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:04:56.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.947+0000 7f7058641640 1 -- 172.21.15.32:0/1178669989 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f703406be30 msgr2=0x7f703406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:56.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.947+0000 7f7058641640 1 --2- 172.21.15.32:0/1178669989 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f703406be30 0x7f703406e2f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f704c006fd0 tx=0x7f704c008040 comp rx=0 tx=0).stop 2024-04-24T23:04:56.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.947+0000 7f7058641640 1 -- 172.21.15.32:0/1178669989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70500b44d0 msgr2=0x7f7050150db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:56.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.947+0000 7f7058641640 1 --2- 172.21.15.32:0/1178669989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70500b44d0 0x7f7050150db0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f704802ad20 tx=0x7f7048004a40 comp rx=0 tx=0).stop 2024-04-24T23:04:56.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.947+0000 7f7058641640 1 -- 172.21.15.32:0/1178669989 shutdown_connections 2024-04-24T23:04:56.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.947+0000 7f7058641640 1 --2- 172.21.15.32:0/1178669989 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f703406be30 0x7f703406e2f0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:56.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.947+0000 7f7058641640 1 --2- 172.21.15.32:0/1178669989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70500b44d0 0x7f7050150db0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:56.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.947+0000 7f7058641640 1 --2- 172.21.15.32:0/1178669989 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f70500af890 0x7f705015e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:56.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.947+0000 7f7058641640 1 --2- 172.21.15.32:0/1178669989 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f705006c5f0 0x7f7050145de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:56.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.947+0000 7f7058641640 1 -- 172.21.15.32:0/1178669989 >> 172.21.15.32:0/1178669989 conn(0x7f705006b110 msgr2=0x7f705015d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:56.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.947+0000 7f7058641640 1 -- 172.21.15.32:0/1178669989 shutdown_connections 2024-04-24T23:04:56.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:56.947+0000 7f7058641640 1 -- 172.21.15.32:0/1178669989 wait complete. 2024-04-24T23:04:56.963 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:04:57.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.063+0000 7fadbcb14640 1 Processor -- start 2024-04-24T23:04:57.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.063+0000 7fadbcb14640 1 -- start start 2024-04-24T23:04:57.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.064+0000 7fadbcb14640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fadb80b5af0 0x7fadb80ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:57.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.064+0000 7fadbcb14640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadb80cd270 con 0x7fadb8162e10 2024-04-24T23:04:57.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.064+0000 7fadbcb14640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadb8071610 con 0x7fadb8162320 2024-04-24T23:04:57.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.064+0000 7fadbcb14640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fadb80b5660 con 0x7fadb80b5af0 2024-04-24T23:04:57.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.064+0000 7fadaffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fadb80b5af0 0x7fadb80ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:57.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.064+0000 7fadaffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fadb80b5af0 0x7fadb80ccc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57106/0 (socket says 172.21.15.32:57106) 2024-04-24T23:04:57.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.064+0000 7fadaffff640 1 -- 172.21.15.32:0/4156520932 learned_addr learned my addr 172.21.15.32:0/4156520932 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:57.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.064+0000 7fadaffff640 1 -- 172.21.15.32:0/4156520932 >> v1:172.21.15.121:6789/0 conn(0x7fadb8162320 legacy=0x7fadb8162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:57.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.064+0000 7fadaffff640 1 -- 172.21.15.32:0/4156520932 >> v1:172.21.15.32:6789/0 conn(0x7fadb8162e10 legacy=0x7fadb815ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:57.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.064+0000 7fadaffff640 1 -- 172.21.15.32:0/4156520932 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadb80718f0 con 0x7fadb80b5af0 2024-04-24T23:04:57.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.064+0000 7fadb5d9b640 1 -- 172.21.15.32:0/4156520932 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2416793325 0 0) 0x7fadb80cd270 con 0x7fadb8162e10 2024-04-24T23:04:57.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.065+0000 7fadaffff640 1 --2- 172.21.15.32:0/4156520932 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fadb80b5af0 0x7fadb80ccc90 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fada0009870 tx=0x7fada002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f66d9f0ed418763f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:57.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.065+0000 7fadb5d9b640 1 -- 172.21.15.32:0/4156520932 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada0037070 con 0x7fadb80b5af0 2024-04-24T23:04:57.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.065+0000 7fadb5d9b640 1 -- 172.21.15.32:0/4156520932 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fada0032440 con 0x7fadb80b5af0 2024-04-24T23:04:57.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.065+0000 7fadb5d9b640 1 -- 172.21.15.32:0/4156520932 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada0032810 con 0x7fadb80b5af0 2024-04-24T23:04:57.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.066+0000 7fadbcb14640 1 -- 172.21.15.32:0/4156520932 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fadb80b5af0 msgr2=0x7fadb80ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:57.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.066+0000 7fadbcb14640 1 --2- 172.21.15.32:0/4156520932 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fadb80b5af0 0x7fadb80ccc90 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fada0009870 tx=0x7fada002a270 comp rx=0 tx=0).stop 2024-04-24T23:04:57.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.066+0000 7fadbcb14640 1 -- 172.21.15.32:0/4156520932 shutdown_connections 2024-04-24T23:04:57.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.066+0000 7fadbcb14640 1 --2- 172.21.15.32:0/4156520932 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fadb80b5af0 0x7fadb80ccc90 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:57.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.066+0000 7fadbcb14640 1 -- 172.21.15.32:0/4156520932 >> 172.21.15.32:0/4156520932 conn(0x7fadb80d0ae0 msgr2=0x7fadb80d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:57.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.066+0000 7fadbcb14640 1 -- 172.21.15.32:0/4156520932 shutdown_connections 2024-04-24T23:04:57.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.066+0000 7fadbcb14640 1 -- 172.21.15.32:0/4156520932 wait complete. 2024-04-24T23:04:57.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.067+0000 7fadbcb14640 1 Processor -- start 2024-04-24T23:04:57.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.067+0000 7fadbcb14640 1 -- start start 2024-04-24T23:04:57.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.068+0000 7fadbcb14640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fadb80b5af0 0x7fadb814e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:57.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.068+0000 7fadbcb14640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fadb8162320 0x7fadb80cf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:57.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.068+0000 7fadbcb14640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fadb8162e10 0x7fadb80cfce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:57.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.068+0000 7fadbcb14640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fadb8071bd0 con 0x7fadb8162e10 2024-04-24T23:04:57.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.068+0000 7fadbcb14640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fadb8067950 con 0x7fadb80b5af0 2024-04-24T23:04:57.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.068+0000 7fadbcb14640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fadb8071d50 con 0x7fadb8162320 2024-04-24T23:04:57.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.068+0000 7fadb6d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fadb8162e10 0x7fadb80cfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:57.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.068+0000 7fadb6d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fadb8162e10 0x7fadb80cfce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57118/0 (socket says 172.21.15.32:57118) 2024-04-24T23:04:57.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.068+0000 7fadb6d9d640 1 -- 172.21.15.32:0/1123239508 learned_addr learned my addr 172.21.15.32:0/1123239508 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:57.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.068+0000 7fadaffff640 1 --2- 172.21.15.32:0/1123239508 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fadb8162320 0x7fadb80cf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:57.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.068+0000 7fadb659c640 1 --2- 172.21.15.32:0/1123239508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fadb80b5af0 0x7fadb814e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:57.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.068+0000 7fadb6d9d640 1 -- 172.21.15.32:0/1123239508 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fadb8162320 msgr2=0x7fadb80cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:57.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.068+0000 7fadb6d9d640 1 --2- 172.21.15.32:0/1123239508 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fadb8162320 0x7fadb80cf7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:57.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.068+0000 7fadb6d9d640 1 -- 172.21.15.32:0/1123239508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fadb80b5af0 msgr2=0x7fadb814e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:57.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.068+0000 7fadb6d9d640 1 --2- 172.21.15.32:0/1123239508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fadb80b5af0 0x7fadb814e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:57.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.068+0000 7fadb6d9d640 1 -- 172.21.15.32:0/1123239508 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadb814ea70 con 0x7fadb8162e10 2024-04-24T23:04:57.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.068+0000 7fadb6d9d640 1 --2- 172.21.15.32:0/1123239508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fadb8162e10 0x7fadb80cfce0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fadb000c7b0 tx=0x7fadb000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:57.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.068+0000 7fadaeffd640 1 -- 172.21.15.32:0/1123239508 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb0003ba0 con 0x7fadb8162e10 2024-04-24T23:04:57.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.068+0000 7fadaeffd640 1 -- 172.21.15.32:0/1123239508 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadb0003d40 con 0x7fadb8162e10 2024-04-24T23:04:57.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.068+0000 7fadbcb14640 1 -- 172.21.15.32:0/1123239508 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadb81e8270 con 0x7fadb8162e10 2024-04-24T23:04:57.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.068+0000 7fadaeffd640 1 -- 172.21.15.32:0/1123239508 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb0005270 con 0x7fadb8162e10 2024-04-24T23:04:57.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.068+0000 7fadbcb14640 1 -- 172.21.15.32:0/1123239508 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadb81e8830 con 0x7fadb8162e10 2024-04-24T23:04:57.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.070+0000 7fadbcb14640 1 -- 172.21.15.32:0/1123239508 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadb8068430 con 0x7fadb8162e10 2024-04-24T23:04:57.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.070+0000 7fadaeffd640 1 -- 172.21.15.32:0/1123239508 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fadb0005410 con 0x7fadb8162e10 2024-04-24T23:04:57.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.070+0000 7fadaeffd640 1 --2- 172.21.15.32:0/1123239508 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fad9806be30 0x7fad9806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:57.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.070+0000 7fadaeffd640 1 -- 172.21.15.32:0/1123239508 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5709+0+0 (secure 0 0 0) 0x7fadb008abf0 con 0x7fadb8162e10 2024-04-24T23:04:57.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.071+0000 7fadb659c640 1 --2- 172.21.15.32:0/1123239508 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fad9806be30 0x7fad9806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:57.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.071+0000 7fadb659c640 1 --2- 172.21.15.32:0/1123239508 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fad9806be30 0x7fad9806e2f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fada8000f60 tx=0x7fada8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:57.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.075+0000 7fadaeffd640 1 -- 172.21.15.32:0/1123239508 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fadb0059630 con 0x7fadb8162e10 2024-04-24T23:04:57.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.260+0000 7fadbcb14640 1 -- 172.21.15.32:0/1123239508 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fadb80d0850 con 0x7fadb8162e10 2024-04-24T23:04:57.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.261+0000 7fadaeffd640 1 -- 172.21.15.32:0/1123239508 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) ==== 74+0+16645 (secure 0 0 0) 0x7fadb0058f30 con 0x7fadb8162e10 2024-04-24T23:04:57.263 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:04:57.263 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":124,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:04:55.997703+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:04:54.280785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-24T23:04:54.966294+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:04:57.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.263+0000 7fadbcb14640 1 -- 172.21.15.32:0/1123239508 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fad9806be30 msgr2=0x7fad9806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:57.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.263+0000 7fadbcb14640 1 --2- 172.21.15.32:0/1123239508 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fad9806be30 0x7fad9806e2f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fada8000f60 tx=0x7fada8009040 comp rx=0 tx=0).stop 2024-04-24T23:04:57.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.263+0000 7fadbcb14640 1 -- 172.21.15.32:0/1123239508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fadb8162e10 msgr2=0x7fadb80cfce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:57.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.263+0000 7fadbcb14640 1 --2- 172.21.15.32:0/1123239508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fadb8162e10 0x7fadb80cfce0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fadb000c7b0 tx=0x7fadb000cbb0 comp rx=0 tx=0).stop 2024-04-24T23:04:57.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.263+0000 7fadbcb14640 1 -- 172.21.15.32:0/1123239508 shutdown_connections 2024-04-24T23:04:57.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.263+0000 7fadbcb14640 1 --2- 172.21.15.32:0/1123239508 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fad9806be30 0x7fad9806e2f0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:57.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.263+0000 7fadbcb14640 1 --2- 172.21.15.32:0/1123239508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fadb8162e10 0x7fadb80cfce0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:57.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.263+0000 7fadbcb14640 1 --2- 172.21.15.32:0/1123239508 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fadb8162320 0x7fadb80cf7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:57.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.263+0000 7fadbcb14640 1 --2- 172.21.15.32:0/1123239508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fadb80b5af0 0x7fadb814e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:57.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.263+0000 7fadbcb14640 1 -- 172.21.15.32:0/1123239508 >> 172.21.15.32:0/1123239508 conn(0x7fadb80d0ae0 msgr2=0x7fadb8161350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:57.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.263+0000 7fadbcb14640 1 -- 172.21.15.32:0/1123239508 shutdown_connections 2024-04-24T23:04:57.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.263+0000 7fadbcb14640 1 -- 172.21.15.32:0/1123239508 wait complete. 2024-04-24T23:04:57.270 INFO:tasks.ceph.mds.a.smithi032.stderr:2024-04-24T23:04:57.269+0000 7f4b6dd7d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T23:04:57.276 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-24T23:04:57.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.380+0000 7f222a9ab640 1 Processor -- start 2024-04-24T23:04:57.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.380+0000 7f222a9ab640 1 -- start start 2024-04-24T23:04:57.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.381+0000 7f222a9ab640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f222406c5f0 0x7f222406c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:57.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.381+0000 7f222a9ab640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22240af890 0x7f222415cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:57.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.381+0000 7f222a9ab640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22240b6430 con 0x7f22240b44d0 2024-04-24T23:04:57.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.381+0000 7f222a9ab640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f2224156f80 con 0x7f222406c5f0 2024-04-24T23:04:57.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.381+0000 7f222a9ab640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f22240b6150 con 0x7f22240af890 2024-04-24T23:04:57.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.381+0000 7f22237fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f22240b44d0 0x7f22240b5890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49512/0 (socket says 172.21.15.32:49512) 2024-04-24T23:04:57.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.381+0000 7f22237fe640 1 -- 172.21.15.32:0/674153041 learned_addr learned my addr 172.21.15.32:0/674153041 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:04:57.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.381+0000 7f2228f48640 1 --2- 172.21.15.32:0/674153041 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22240af890 0x7f222415cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:57.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.381+0000 7f2223fff640 1 --2- 172.21.15.32:0/674153041 >> v2:172.21.15.121:3300/0 conn(0x7f222406c5f0 0x7f222406c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:57.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.381+0000 7f2222ffd640 1 -- 172.21.15.32:0/674153041 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3541329554 0 0) 0x7f22240b6430 con 0x7f22240b44d0 2024-04-24T23:04:57.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.381+0000 7f2222ffd640 1 -- 172.21.15.32:0/674153041 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f220c003520 con 0x7f22240b44d0 2024-04-24T23:04:57.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.381+0000 7f2228f48640 1 -- 172.21.15.32:0/674153041 >> v2:172.21.15.121:3300/0 conn(0x7f222406c5f0 msgr2=0x7f222406c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:57.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.381+0000 7f2228f48640 1 --2- 172.21.15.32:0/674153041 >> v2:172.21.15.121:3300/0 conn(0x7f222406c5f0 0x7f222406c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:57.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.381+0000 7f2228f48640 1 -- 172.21.15.32:0/674153041 >> v1:172.21.15.32:6789/0 conn(0x7f22240b44d0 legacy=0x7f22240b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:57.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.381+0000 7f2222ffd640 1 -- 172.21.15.32:0/674153041 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4267401861 0 0) 0x7f220c003520 con 0x7f22240b44d0 2024-04-24T23:04:57.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.381+0000 7f2228f48640 1 -- 172.21.15.32:0/674153041 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22240b39f0 con 0x7f22240af890 2024-04-24T23:04:57.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.381+0000 7f2223fff640 1 --2- 172.21.15.32:0/674153041 >> v2:172.21.15.121:3300/0 conn(0x7f222406c5f0 0x7f222406c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:04:57.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.382+0000 7f2228f48640 1 --2- 172.21.15.32:0/674153041 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22240af890 0x7f222415cac0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f221c009d90 tx=0x7f221c02a560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93dca3d0259d2d56 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:57.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.382+0000 7f2222ffd640 1 -- 172.21.15.32:0/674153041 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2214001690 con 0x7f22240af890 2024-04-24T23:04:57.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.382+0000 7f2222ffd640 1 -- 172.21.15.32:0/674153041 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f221c007540 con 0x7f22240af890 2024-04-24T23:04:57.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.382+0000 7f2222ffd640 1 -- 172.21.15.32:0/674153041 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221c007910 con 0x7f22240af890 2024-04-24T23:04:57.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.383+0000 7f222a9ab640 1 -- 172.21.15.32:0/674153041 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22240af890 msgr2=0x7f222415cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:57.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.383+0000 7f222a9ab640 1 --2- 172.21.15.32:0/674153041 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22240af890 0x7f222415cac0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f221c009d90 tx=0x7f221c02a560 comp rx=0 tx=0).stop 2024-04-24T23:04:57.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.383+0000 7f222a9ab640 1 -- 172.21.15.32:0/674153041 shutdown_connections 2024-04-24T23:04:57.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.383+0000 7f222a9ab640 1 --2- 172.21.15.32:0/674153041 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22240af890 0x7f222415cac0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:57.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.383+0000 7f222a9ab640 1 --2- 172.21.15.32:0/674153041 >> v2:172.21.15.121:3300/0 conn(0x7f222406c5f0 0x7f222406c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:57.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.383+0000 7f222a9ab640 1 -- 172.21.15.32:0/674153041 >> 172.21.15.32:0/674153041 conn(0x7f222406b110 msgr2=0x7f222406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:57.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.383+0000 7f222a9ab640 1 -- 172.21.15.32:0/674153041 shutdown_connections 2024-04-24T23:04:57.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.383+0000 7f222a9ab640 1 -- 172.21.15.32:0/674153041 wait complete. 2024-04-24T23:04:57.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.384+0000 7f222a9ab640 1 Processor -- start 2024-04-24T23:04:57.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.384+0000 7f222a9ab640 1 -- start start 2024-04-24T23:04:57.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.385+0000 7f222a9ab640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f222406c5f0 0x7f222414c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:57.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.385+0000 7f222a9ab640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22240af890 0x7f222414c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:57.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.385+0000 7f222a9ab640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22240b44d0 0x7f22240ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:57.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.385+0000 7f222a9ab640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f222406cf10 con 0x7f222406c5f0 2024-04-24T23:04:57.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.385+0000 7f222a9ab640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f22240b3fb0 con 0x7f22240af890 2024-04-24T23:04:57.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.385+0000 7f222a9ab640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f222415d000 con 0x7f22240b44d0 2024-04-24T23:04:57.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.385+0000 7f2223fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f222406c5f0 0x7f222414c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:57.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.385+0000 7f2223fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f222406c5f0 0x7f222414c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57130/0 (socket says 172.21.15.32:57130) 2024-04-24T23:04:57.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.385+0000 7f2223fff640 1 -- 172.21.15.32:0/520455121 learned_addr learned my addr 172.21.15.32:0/520455121 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:57.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.385+0000 7f2228f48640 1 --2- 172.21.15.32:0/520455121 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22240b44d0 0x7f22240ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:57.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.385+0000 7f22237fe640 1 --2- 172.21.15.32:0/520455121 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22240af890 0x7f222414c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:57.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.386+0000 7f2223fff640 1 -- 172.21.15.32:0/520455121 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22240b44d0 msgr2=0x7f22240ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:57.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.386+0000 7f2223fff640 1 --2- 172.21.15.32:0/520455121 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22240b44d0 0x7f22240ae8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:57.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.386+0000 7f2223fff640 1 -- 172.21.15.32:0/520455121 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22240af890 msgr2=0x7f222414c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:57.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.386+0000 7f2223fff640 1 --2- 172.21.15.32:0/520455121 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22240af890 0x7f222414c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:57.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.386+0000 7f2223fff640 1 -- 172.21.15.32:0/520455121 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f222415a2c0 con 0x7f222406c5f0 2024-04-24T23:04:57.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.386+0000 7f2223fff640 1 --2- 172.21.15.32:0/520455121 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f222406c5f0 0x7f222414c780 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f221800ebe0 tx=0x7f221800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:57.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.386+0000 7f2220ff9640 1 -- 172.21.15.32:0/520455121 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2218017070 con 0x7f222406c5f0 2024-04-24T23:04:57.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.386+0000 7f2220ff9640 1 -- 172.21.15.32:0/520455121 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f221800cc30 con 0x7f222406c5f0 2024-04-24T23:04:57.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.386+0000 7f222a9ab640 1 -- 172.21.15.32:0/520455121 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22240ce3d0 con 0x7f222406c5f0 2024-04-24T23:04:57.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.386+0000 7f2220ff9640 1 -- 172.21.15.32:0/520455121 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2218013470 con 0x7f222406c5f0 2024-04-24T23:04:57.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.386+0000 7f222a9ab640 1 -- 172.21.15.32:0/520455121 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22240ce080 con 0x7f222406c5f0 2024-04-24T23:04:57.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.387+0000 7f222a9ab640 1 -- 172.21.15.32:0/520455121 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2224068430 con 0x7f222406c5f0 2024-04-24T23:04:57.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.387+0000 7f2220ff9640 1 -- 172.21.15.32:0/520455121 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f221801a070 con 0x7f222406c5f0 2024-04-24T23:04:57.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.388+0000 7f2220ff9640 1 --2- 172.21.15.32:0/520455121 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f221006be60 0x7f221006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:57.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.388+0000 7f2220ff9640 1 -- 172.21.15.32:0/520455121 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5709+0+0 (secure 0 0 0) 0x7f221808b150 con 0x7f222406c5f0 2024-04-24T23:04:57.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.388+0000 7f22237fe640 1 --2- 172.21.15.32:0/520455121 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f221006be60 0x7f221006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:57.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.389+0000 7f22237fe640 1 --2- 172.21.15.32:0/520455121 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f221006be60 0x7f221006e320 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f22240b67c0 tx=0x7f2214009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:57.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.392+0000 7f2220ff9640 1 -- 172.21.15.32:0/520455121 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2218059b00 con 0x7f222406c5f0 2024-04-24T23:04:57.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:57.613+0000 7f222a9ab640 1 -- 172.21.15.32:0/520455121 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f222415d9e0 con 0x7f222406c5f0 2024-04-24T23:04:58.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.295+0000 7f2220ff9640 1 -- 172.21.15.32:0/520455121 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v50) ==== 118+0+0 (secure 0 0 0) 0x7f2218059400 con 0x7f222406c5f0 2024-04-24T23:04:58.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.297+0000 7f222a9ab640 1 -- 172.21.15.32:0/520455121 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f221006be60 msgr2=0x7f221006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:58.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.297+0000 7f222a9ab640 1 --2- 172.21.15.32:0/520455121 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f221006be60 0x7f221006e320 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f22240b67c0 tx=0x7f2214009040 comp rx=0 tx=0).stop 2024-04-24T23:04:58.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.297+0000 7f222a9ab640 1 -- 172.21.15.32:0/520455121 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f222406c5f0 msgr2=0x7f222414c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:58.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.297+0000 7f222a9ab640 1 --2- 172.21.15.32:0/520455121 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f222406c5f0 0x7f222414c780 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f221800ebe0 tx=0x7f221800c520 comp rx=0 tx=0).stop 2024-04-24T23:04:58.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.298+0000 7f222a9ab640 1 -- 172.21.15.32:0/520455121 shutdown_connections 2024-04-24T23:04:58.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.298+0000 7f222a9ab640 1 --2- 172.21.15.32:0/520455121 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f221006be60 0x7f221006e320 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:58.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.298+0000 7f222a9ab640 1 --2- 172.21.15.32:0/520455121 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22240b44d0 0x7f22240ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:58.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.298+0000 7f222a9ab640 1 --2- 172.21.15.32:0/520455121 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22240af890 0x7f222414c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:58.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.298+0000 7f222a9ab640 1 --2- 172.21.15.32:0/520455121 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f222406c5f0 0x7f222414c780 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:58.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.298+0000 7f222a9ab640 1 -- 172.21.15.32:0/520455121 >> 172.21.15.32:0/520455121 conn(0x7f222406b110 msgr2=0x7f222406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:58.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.298+0000 7f222a9ab640 1 -- 172.21.15.32:0/520455121 shutdown_connections 2024-04-24T23:04:58.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.298+0000 7f222a9ab640 1 -- 172.21.15.32:0/520455121 wait complete. 2024-04-24T23:04:58.312 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-24T23:04:58.313 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-24T23:04:58.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.422+0000 7f2fe4309640 1 Processor -- start 2024-04-24T23:04:58.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.422+0000 7f2fe4309640 1 -- start start 2024-04-24T23:04:58.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.423+0000 7f2fe4309640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2fdc15b480 0x7f2fdc0b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:58.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.423+0000 7f2fe4309640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fdc0b46c0 con 0x7f2fdc15fd60 2024-04-24T23:04:58.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.423+0000 7f2fe4309640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fdc0b49a0 con 0x7f2fdc06cd00 2024-04-24T23:04:58.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.423+0000 7f2fe4309640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f2fdc0b4ff0 con 0x7f2fdc15b480 2024-04-24T23:04:58.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.423+0000 7f2fe20a5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2fdc15b480 0x7f2fdc0b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:58.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.423+0000 7f2fe20a5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2fdc15b480 0x7f2fdc0b5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57146/0 (socket says 172.21.15.32:57146) 2024-04-24T23:04:58.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.423+0000 7f2fe20a5640 1 -- 172.21.15.32:0/2246750380 learned_addr learned my addr 172.21.15.32:0/2246750380 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:58.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.423+0000 7f2fe18a4640 1 --1- 172.21.15.32:0/2246750380 >> v1:172.21.15.32:6789/0 conn(0x7f2fdc15fd60 0x7f2fdc06c5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49518/0 (socket says 172.21.15.32:49518) 2024-04-24T23:04:58.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.423+0000 7f2fe20a5640 1 -- 172.21.15.32:0/2246750380 >> v1:172.21.15.121:6789/0 conn(0x7f2fdc06cd00 legacy=0x7f2fdc0cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:58.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.423+0000 7f2fe20a5640 1 -- 172.21.15.32:0/2246750380 >> v1:172.21.15.32:6789/0 conn(0x7f2fdc15fd60 legacy=0x7f2fdc06c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:58.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.423+0000 7f2fe20a5640 1 -- 172.21.15.32:0/2246750380 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fdc0b4c80 con 0x7f2fdc15b480 2024-04-24T23:04:58.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.423+0000 7f2fe10a3640 1 -- 172.21.15.32:0/2246750380 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3557036486 0 0) 0x7f2fdc0b46c0 con 0x7f2fdc15fd60 2024-04-24T23:04:58.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.424+0000 7f2fe20a5640 1 --2- 172.21.15.32:0/2246750380 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fdc15b480 0x7f2fdc0b5c60 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc0b5170 tx=0x7f2fd802a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1638d96183ac1321 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:58.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.424+0000 7f2fe10a3640 1 -- 172.21.15.32:0/2246750380 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd8037070 con 0x7f2fdc15b480 2024-04-24T23:04:58.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.424+0000 7f2fe10a3640 1 -- 172.21.15.32:0/2246750380 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fd8032440 con 0x7f2fdc15b480 2024-04-24T23:04:58.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.424+0000 7f2fe10a3640 1 -- 172.21.15.32:0/2246750380 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd8032810 con 0x7f2fdc15b480 2024-04-24T23:04:58.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.425+0000 7f2fe4309640 1 -- 172.21.15.32:0/2246750380 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fdc15b480 msgr2=0x7f2fdc0b5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:58.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.425+0000 7f2fe4309640 1 --2- 172.21.15.32:0/2246750380 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fdc15b480 0x7f2fdc0b5c60 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc0b5170 tx=0x7f2fd802a290 comp rx=0 tx=0).stop 2024-04-24T23:04:58.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.425+0000 7f2fe4309640 1 -- 172.21.15.32:0/2246750380 shutdown_connections 2024-04-24T23:04:58.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.425+0000 7f2fe4309640 1 --2- 172.21.15.32:0/2246750380 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fdc15b480 0x7f2fdc0b5c60 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:58.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.425+0000 7f2fe4309640 1 -- 172.21.15.32:0/2246750380 >> 172.21.15.32:0/2246750380 conn(0x7f2fdc06b110 msgr2=0x7f2fdc06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:58.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.425+0000 7f2fe4309640 1 -- 172.21.15.32:0/2246750380 shutdown_connections 2024-04-24T23:04:58.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.425+0000 7f2fe4309640 1 -- 172.21.15.32:0/2246750380 wait complete. 2024-04-24T23:04:58.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.426+0000 7f2fe4309640 1 Processor -- start 2024-04-24T23:04:58.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.426+0000 7f2fe4309640 1 -- start start 2024-04-24T23:04:58.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.427+0000 7f2fe4309640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fdc06cd00 0x7f2fdc14a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:58.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.427+0000 7f2fe4309640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2fdc15b480 0x7f2fdc14bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:58.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.427+0000 7f2fe4309640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2fdc15fd60 0x7f2fdc0ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:58.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.427+0000 7f2fe4309640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2fdc15b860 con 0x7f2fdc06cd00 2024-04-24T23:04:58.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.427+0000 7f2fe4309640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2fdc0b5480 con 0x7f2fdc15fd60 2024-04-24T23:04:58.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.427+0000 7f2fe4309640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f2fdc15ea10 con 0x7f2fdc15b480 2024-04-24T23:04:58.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.427+0000 7f2fe20a5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fdc06cd00 0x7f2fdc14a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:58.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.427+0000 7f2fe20a5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fdc06cd00 0x7f2fdc14a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57148/0 (socket says 172.21.15.32:57148) 2024-04-24T23:04:58.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.427+0000 7f2fe20a5640 1 -- 172.21.15.32:0/1084258853 learned_addr learned my addr 172.21.15.32:0/1084258853 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:58.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.427+0000 7f2fe28a6640 1 --2- 172.21.15.32:0/1084258853 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2fdc15fd60 0x7f2fdc0ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:58.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.427+0000 7f2fe18a4640 1 --2- 172.21.15.32:0/1084258853 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2fdc15b480 0x7f2fdc14bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:58.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.427+0000 7f2fe20a5640 1 -- 172.21.15.32:0/1084258853 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2fdc15b480 msgr2=0x7f2fdc14bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:58.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.427+0000 7f2fe20a5640 1 --2- 172.21.15.32:0/1084258853 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2fdc15b480 0x7f2fdc14bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:58.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.427+0000 7f2fe20a5640 1 -- 172.21.15.32:0/1084258853 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2fdc15fd60 msgr2=0x7f2fdc0ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:58.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.427+0000 7f2fe20a5640 1 --2- 172.21.15.32:0/1084258853 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2fdc15fd60 0x7f2fdc0ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:58.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.427+0000 7f2fe20a5640 1 -- 172.21.15.32:0/1084258853 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fdc14db50 con 0x7f2fdc06cd00 2024-04-24T23:04:58.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.427+0000 7f2fe18a4640 1 --2- 172.21.15.32:0/1084258853 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2fdc15b480 0x7f2fdc14bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:04:58.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.427+0000 7f2fe20a5640 1 --2- 172.21.15.32:0/1084258853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fdc06cd00 0x7f2fdc14a6b0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8000c00 tx=0x7f2fd80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:58.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.427+0000 7f2fd2ffd640 1 -- 172.21.15.32:0/1084258853 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd8041070 con 0x7f2fdc06cd00 2024-04-24T23:04:58.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.427+0000 7f2fd2ffd640 1 -- 172.21.15.32:0/1084258853 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fd803c870 con 0x7f2fdc06cd00 2024-04-24T23:04:58.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.427+0000 7f2fd2ffd640 1 -- 172.21.15.32:0/1084258853 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd8037040 con 0x7f2fdc06cd00 2024-04-24T23:04:58.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.427+0000 7f2fe4309640 1 -- 172.21.15.32:0/1084258853 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fdc14de30 con 0x7f2fdc06cd00 2024-04-24T23:04:58.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.427+0000 7f2fe4309640 1 -- 172.21.15.32:0/1084258853 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fdc14e3f0 con 0x7f2fdc06cd00 2024-04-24T23:04:58.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.428+0000 7f2fe4309640 1 -- 172.21.15.32:0/1084258853 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fdc068430 con 0x7f2fdc06cd00 2024-04-24T23:04:58.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.429+0000 7f2fd2ffd640 1 -- 172.21.15.32:0/1084258853 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2fd8044070 con 0x7f2fdc06cd00 2024-04-24T23:04:58.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.429+0000 7f2fd2ffd640 1 --2- 172.21.15.32:0/1084258853 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2fc006be30 0x7f2fc006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:58.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.429+0000 7f2fd2ffd640 1 -- 172.21.15.32:0/1084258853 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5709+0+0 (secure 0 0 0) 0x7f2fd80ab910 con 0x7f2fdc06cd00 2024-04-24T23:04:58.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.430+0000 7f2fe18a4640 1 --2- 172.21.15.32:0/1084258853 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2fc006be30 0x7f2fc006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:58.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.430+0000 7f2fe18a4640 1 --2- 172.21.15.32:0/1084258853 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2fc006be30 0x7f2fc006e2f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc14b250 tx=0x7f2fcc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:58.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.434+0000 7f2fd2ffd640 1 -- 172.21.15.32:0/1084258853 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2fd8079d00 con 0x7f2fdc06cd00 2024-04-24T23:04:58.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:58.656+0000 7f2fe4309640 1 -- 172.21.15.32:0/1084258853 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f2fdc06c000 con 0x7f2fdc06cd00 2024-04-24T23:04:59.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.299+0000 7f2fd2ffd640 1 -- 172.21.15.32:0/1084258853 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v51) ==== 122+0+0 (secure 0 0 0) 0x7f2fd807e000 con 0x7f2fdc06cd00 2024-04-24T23:04:59.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.302+0000 7f2fe4309640 1 -- 172.21.15.32:0/1084258853 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2fc006be30 msgr2=0x7f2fc006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:59.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.302+0000 7f2fe4309640 1 --2- 172.21.15.32:0/1084258853 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2fc006be30 0x7f2fc006e2f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc14b250 tx=0x7f2fcc009040 comp rx=0 tx=0).stop 2024-04-24T23:04:59.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.302+0000 7f2fe4309640 1 -- 172.21.15.32:0/1084258853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fdc06cd00 msgr2=0x7f2fdc14a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:59.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.302+0000 7f2fe4309640 1 --2- 172.21.15.32:0/1084258853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fdc06cd00 0x7f2fdc14a6b0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8000c00 tx=0x7f2fd80046c0 comp rx=0 tx=0).stop 2024-04-24T23:04:59.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.302+0000 7f2fe4309640 1 -- 172.21.15.32:0/1084258853 shutdown_connections 2024-04-24T23:04:59.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.302+0000 7f2fe4309640 1 --2- 172.21.15.32:0/1084258853 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2fc006be30 0x7f2fc006e2f0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:59.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.302+0000 7f2fe4309640 1 --2- 172.21.15.32:0/1084258853 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2fdc15fd60 0x7f2fdc0ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:59.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.302+0000 7f2fe4309640 1 --2- 172.21.15.32:0/1084258853 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2fdc15b480 0x7f2fdc14bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:59.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.302+0000 7f2fe4309640 1 --2- 172.21.15.32:0/1084258853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fdc06cd00 0x7f2fdc14a6b0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:59.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.302+0000 7f2fe4309640 1 -- 172.21.15.32:0/1084258853 >> 172.21.15.32:0/1084258853 conn(0x7f2fdc06b110 msgr2=0x7f2fdc15edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:59.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.302+0000 7f2fe4309640 1 -- 172.21.15.32:0/1084258853 shutdown_connections 2024-04-24T23:04:59.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.302+0000 7f2fe4309640 1 -- 172.21.15.32:0/1084258853 wait complete. 2024-04-24T23:04:59.315 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-24T23:04:59.316 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:04:59.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.418+0000 7fd7c1a3f640 1 Processor -- start 2024-04-24T23:04:59.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.419+0000 7fd7c1a3f640 1 -- start start 2024-04-24T23:04:59.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.419+0000 7fd7c1a3f640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd7bc06c5f0 0x7fd7bc06c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:59.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.419+0000 7fd7c1a3f640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7bc0b6430 con 0x7fd7bc0af890 2024-04-24T23:04:59.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.419+0000 7fd7c1a3f640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7bc156f80 con 0x7fd7bc0b44d0 2024-04-24T23:04:59.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.419+0000 7fd7c1a3f640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fd7bc0b6150 con 0x7fd7bc06c5f0 2024-04-24T23:04:59.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.419+0000 7fd7bb7fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd7bc0af890 0x7fd7bc15cac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49530/0 (socket says 172.21.15.32:49530) 2024-04-24T23:04:59.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.420+0000 7fd7bb7fe640 1 -- 172.21.15.32:0/300196349 learned_addr learned my addr 172.21.15.32:0/300196349 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:04:59.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.420+0000 7fd7b9ffb640 1 -- 172.21.15.32:0/300196349 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 594016026 0 0) 0x7fd7bc0b6430 con 0x7fd7bc0af890 2024-04-24T23:04:59.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.420+0000 7fd7baffd640 1 --2- 172.21.15.32:0/300196349 >> v2:172.21.15.121:3300/0 conn(0x7fd7bc06c5f0 0x7fd7bc06c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:59.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.420+0000 7fd7b9ffb640 1 -- 172.21.15.32:0/300196349 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd7a8003520 con 0x7fd7bc0af890 2024-04-24T23:04:59.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.420+0000 7fd7b9ffb640 1 -- 172.21.15.32:0/300196349 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3215078878 0 0) 0x7fd7a8003520 con 0x7fd7bc0af890 2024-04-24T23:04:59.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.420+0000 7fd7b9ffb640 1 -- 172.21.15.32:0/300196349 >> v2:172.21.15.121:3300/0 conn(0x7fd7bc06c5f0 msgr2=0x7fd7bc06c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:59.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.420+0000 7fd7b9ffb640 1 --2- 172.21.15.32:0/300196349 >> v2:172.21.15.121:3300/0 conn(0x7fd7bc06c5f0 0x7fd7bc06c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:59.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.420+0000 7fd7b9ffb640 1 -- 172.21.15.32:0/300196349 >> v1:172.21.15.121:6789/0 conn(0x7fd7bc0b44d0 legacy=0x7fd7bc0b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:59.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.420+0000 7fd7b9ffb640 1 -- 172.21.15.32:0/300196349 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7bc0b39f0 con 0x7fd7bc0af890 2024-04-24T23:04:59.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.420+0000 7fd7b9ffb640 1 -- 172.21.15.32:0/300196349 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7fd7ac0019f0 con 0x7fd7bc0af890 2024-04-24T23:04:59.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.420+0000 7fd7b9ffb640 1 -- 172.21.15.32:0/300196349 >> v1:172.21.15.32:6789/0 conn(0x7fd7bc0af890 legacy=0x7fd7bc15cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:59.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.420+0000 7fd7b9ffb640 1 --2- 172.21.15.32:0/300196349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7a80039b0 0x7fd7a8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:59.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.420+0000 7fd7b9ffb640 1 --2- 172.21.15.32:0/300196349 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7a8004800 0x7fd7a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:59.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.420+0000 7fd7b9ffb640 1 --2- 172.21.15.32:0/300196349 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7a80051c0 0x7fd7a800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:59.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.420+0000 7fd7b9ffb640 1 -- 172.21.15.32:0/300196349 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd7bc0b6150 con 0x7fd7a80039b0 2024-04-24T23:04:59.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.420+0000 7fd7b9ffb640 1 -- 172.21.15.32:0/300196349 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a8010470 con 0x7fd7a8004800 2024-04-24T23:04:59.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.420+0000 7fd7b9ffb640 1 -- 172.21.15.32:0/300196349 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd7a8010670 con 0x7fd7a80051c0 2024-04-24T23:04:59.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.420+0000 7fd7baffd640 1 --2- 172.21.15.32:0/300196349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7a80039b0 0x7fd7a8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:59.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.420+0000 7fd7ba7fc640 1 --2- 172.21.15.32:0/300196349 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7a8004800 0x7fd7a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:59.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.420+0000 7fd7bb7fe640 1 --2- 172.21.15.32:0/300196349 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7a80051c0 0x7fd7a800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:59.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.421+0000 7fd7baffd640 1 -- 172.21.15.32:0/300196349 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7a80051c0 msgr2=0x7fd7a800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:59.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.421+0000 7fd7baffd640 1 --2- 172.21.15.32:0/300196349 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7a80051c0 0x7fd7a800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:59.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.421+0000 7fd7baffd640 1 -- 172.21.15.32:0/300196349 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7a8004800 msgr2=0x7fd7a8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:59.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.421+0000 7fd7baffd640 1 --2- 172.21.15.32:0/300196349 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7a8004800 0x7fd7a8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:59.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.421+0000 7fd7baffd640 1 -- 172.21.15.32:0/300196349 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7a8003520 con 0x7fd7a80039b0 2024-04-24T23:04:59.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.421+0000 7fd7bb7fe640 1 --2- 172.21.15.32:0/300196349 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7a80051c0 0x7fd7a800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:04:59.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.421+0000 7fd7bb7fe640 1 -- 172.21.15.32:0/300196349 reap_dead start 2024-04-24T23:04:59.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.421+0000 7fd7baffd640 1 --2- 172.21.15.32:0/300196349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7a80039b0 0x7fd7a8003dd0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4014f00 tx=0x7fd7a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=600d60879c44e870 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:59.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.421+0000 7fd7b9ffb640 1 -- 172.21.15.32:0/300196349 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b403b070 con 0x7fd7a80039b0 2024-04-24T23:04:59.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.421+0000 7fd7b9ffb640 1 -- 172.21.15.32:0/300196349 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7b4037650 con 0x7fd7a80039b0 2024-04-24T23:04:59.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.421+0000 7fd7b9ffb640 1 -- 172.21.15.32:0/300196349 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b4014b50 con 0x7fd7a80039b0 2024-04-24T23:04:59.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.422+0000 7fd7c1a3f640 1 -- 172.21.15.32:0/300196349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7a80039b0 msgr2=0x7fd7a8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:59.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.422+0000 7fd7c1a3f640 1 --2- 172.21.15.32:0/300196349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7a80039b0 0x7fd7a8003dd0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4014f00 tx=0x7fd7a8005830 comp rx=0 tx=0).stop 2024-04-24T23:04:59.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.422+0000 7fd7c1a3f640 1 -- 172.21.15.32:0/300196349 shutdown_connections 2024-04-24T23:04:59.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.422+0000 7fd7c1a3f640 1 --2- 172.21.15.32:0/300196349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7a80039b0 0x7fd7a8003dd0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:59.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.422+0000 7fd7c1a3f640 1 -- 172.21.15.32:0/300196349 >> 172.21.15.32:0/300196349 conn(0x7fd7bc06b110 msgr2=0x7fd7bc06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:59.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.422+0000 7fd7c1a3f640 1 -- 172.21.15.32:0/300196349 shutdown_connections 2024-04-24T23:04:59.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.422+0000 7fd7c1a3f640 1 -- 172.21.15.32:0/300196349 wait complete. 2024-04-24T23:04:59.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.423+0000 7fd7c1a3f640 1 Processor -- start 2024-04-24T23:04:59.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.424+0000 7fd7c1a3f640 1 -- start start 2024-04-24T23:04:59.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.424+0000 7fd7c1a3f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7a80039b0 0x7fd7bc150320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:59.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.424+0000 7fd7c1a3f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7bc1f0130 0x7fd7bc0b0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:59.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.424+0000 7fd7c1a3f640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7bc1508f0 0x7fd7bc157180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:59.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.424+0000 7fd7c1a3f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd7bc06c9d0 con 0x7fd7a80039b0 2024-04-24T23:04:59.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.424+0000 7fd7c1a3f640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd7bc0b3fb0 con 0x7fd7bc1f0130 2024-04-24T23:04:59.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.424+0000 7fd7c1a3f640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd7bc06cb50 con 0x7fd7bc1508f0 2024-04-24T23:04:59.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.424+0000 7fd7baffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7a80039b0 0x7fd7bc150320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:59.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.424+0000 7fd7baffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7a80039b0 0x7fd7bc150320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57174/0 (socket says 172.21.15.32:57174) 2024-04-24T23:04:59.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.424+0000 7fd7baffd640 1 -- 172.21.15.32:0/2544872174 learned_addr learned my addr 172.21.15.32:0/2544872174 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:59.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.424+0000 7fd7ba7fc640 1 --2- 172.21.15.32:0/2544872174 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7bc1f0130 0x7fd7bc0b0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:59.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.425+0000 7fd7bb7fe640 1 --2- 172.21.15.32:0/2544872174 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7bc1508f0 0x7fd7bc157180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:59.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.425+0000 7fd7baffd640 1 -- 172.21.15.32:0/2544872174 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7bc1508f0 msgr2=0x7fd7bc157180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:59.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.425+0000 7fd7baffd640 1 --2- 172.21.15.32:0/2544872174 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7bc1508f0 0x7fd7bc157180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:59.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.425+0000 7fd7baffd640 1 -- 172.21.15.32:0/2544872174 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7bc1f0130 msgr2=0x7fd7bc0b0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:59.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.425+0000 7fd7baffd640 1 --2- 172.21.15.32:0/2544872174 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7bc1f0130 0x7fd7bc0b0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:59.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.425+0000 7fd7baffd640 1 -- 172.21.15.32:0/2544872174 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7bc159b80 con 0x7fd7a80039b0 2024-04-24T23:04:59.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.425+0000 7fd7ba7fc640 1 --2- 172.21.15.32:0/2544872174 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7bc1f0130 0x7fd7bc0b0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:04:59.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.425+0000 7fd7baffd640 1 --2- 172.21.15.32:0/2544872174 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7a80039b0 0x7fd7bc150320 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4014f00 tx=0x7fd7b4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:59.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.425+0000 7fd7b3fff640 1 -- 172.21.15.32:0/2544872174 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b403b050 con 0x7fd7a80039b0 2024-04-24T23:04:59.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.425+0000 7fd7b3fff640 1 -- 172.21.15.32:0/2544872174 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7b40044e0 con 0x7fd7a80039b0 2024-04-24T23:04:59.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.425+0000 7fd7c1a3f640 1 -- 172.21.15.32:0/2544872174 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7bc159e60 con 0x7fd7a80039b0 2024-04-24T23:04:59.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.425+0000 7fd7b3fff640 1 -- 172.21.15.32:0/2544872174 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b4014b50 con 0x7fd7a80039b0 2024-04-24T23:04:59.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.425+0000 7fd7c1a3f640 1 -- 172.21.15.32:0/2544872174 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7bc158270 con 0x7fd7a80039b0 2024-04-24T23:04:59.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.426+0000 7fd7c1a3f640 1 -- 172.21.15.32:0/2544872174 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7bc15a030 con 0x7fd7a80039b0 2024-04-24T23:04:59.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.427+0000 7fd7b3fff640 1 -- 172.21.15.32:0/2544872174 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd7b403e070 con 0x7fd7a80039b0 2024-04-24T23:04:59.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.427+0000 7fd7b3fff640 1 --2- 172.21.15.32:0/2544872174 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd79c06be30 0x7fd79c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:59.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.427+0000 7fd7b3fff640 1 -- 172.21.15.32:0/2544872174 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5709+0+0 (secure 0 0 0) 0x7fd7b40b3d40 con 0x7fd7a80039b0 2024-04-24T23:04:59.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.427+0000 7fd7ba7fc640 1 --2- 172.21.15.32:0/2544872174 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd79c06be30 0x7fd79c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:59.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.428+0000 7fd7ba7fc640 1 --2- 172.21.15.32:0/2544872174 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd79c06be30 0x7fd79c06e2f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc15bd00 tx=0x7fd7a4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:59.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.431+0000 7fd7b3fff640 1 -- 172.21.15.32:0/2544872174 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd7b40826f0 con 0x7fd7a80039b0 2024-04-24T23:04:59.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.650+0000 7fd7c1a3f640 1 -- 172.21.15.32:0/2544872174 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd7bc0b5670 con 0x7fd7a80039b0 2024-04-24T23:04:59.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.650+0000 7fd7b3fff640 1 -- 172.21.15.32:0/2544872174 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+5189 (secure 0 0 0) 0x7fd7b4081ff0 con 0x7fd7a80039b0 2024-04-24T23:04:59.652 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:04:59.652 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":51,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5972,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3336404028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3336404028},{"type":"v1","addr":"172.21.15.121:6837","nonce":3336404028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6505,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/1068557774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1068557774},{"type":"v1","addr":"172.21.15.121:6836","nonce":1068557774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":7353,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/2118719195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":2118719195},{"type":"v1","addr":"172.21.15.32:6834","nonce":2118719195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:04:55.998022+0000","modified":"2024-04-24T23:04:58.657258+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7356},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7356":{"gid":7356,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/3015290487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":3015290487},{"type":"v1","addr":"172.21.15.32:6835","nonce":3015290487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7356,"qdb_cluster":[7356]},"id":7}]} 2024-04-24T23:04:59.652 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 51 2024-04-24T23:04:59.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.652+0000 7fd7c1a3f640 1 -- 172.21.15.32:0/2544872174 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd79c06be30 msgr2=0x7fd79c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:59.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.652+0000 7fd7c1a3f640 1 --2- 172.21.15.32:0/2544872174 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd79c06be30 0x7fd79c06e2f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc15bd00 tx=0x7fd7a4009040 comp rx=0 tx=0).stop 2024-04-24T23:04:59.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.652+0000 7fd7c1a3f640 1 -- 172.21.15.32:0/2544872174 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7a80039b0 msgr2=0x7fd7bc150320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:59.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.652+0000 7fd7c1a3f640 1 --2- 172.21.15.32:0/2544872174 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7a80039b0 0x7fd7bc150320 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4014f00 tx=0x7fd7b4004730 comp rx=0 tx=0).stop 2024-04-24T23:04:59.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.652+0000 7fd7c1a3f640 1 -- 172.21.15.32:0/2544872174 shutdown_connections 2024-04-24T23:04:59.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.652+0000 7fd7c1a3f640 1 --2- 172.21.15.32:0/2544872174 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd79c06be30 0x7fd79c06e2f0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:59.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.652+0000 7fd7c1a3f640 1 --2- 172.21.15.32:0/2544872174 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7bc1508f0 0x7fd7bc157180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:59.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.652+0000 7fd7c1a3f640 1 --2- 172.21.15.32:0/2544872174 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7bc1f0130 0x7fd7bc0b0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:59.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.652+0000 7fd7c1a3f640 1 --2- 172.21.15.32:0/2544872174 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7a80039b0 0x7fd7bc150320 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:59.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.652+0000 7fd7c1a3f640 1 -- 172.21.15.32:0/2544872174 >> 172.21.15.32:0/2544872174 conn(0x7fd7bc06b110 msgr2=0x7fd7bc153e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:59.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.652+0000 7fd7c1a3f640 1 -- 172.21.15.32:0/2544872174 shutdown_connections 2024-04-24T23:04:59.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.652+0000 7fd7c1a3f640 1 -- 172.21.15.32:0/2544872174 wait complete. 2024-04-24T23:04:59.665 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:04:59.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.764+0000 7f23313ed640 1 Processor -- start 2024-04-24T23:04:59.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.764+0000 7f23313ed640 1 -- start start 2024-04-24T23:04:59.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.764+0000 7f23313ed640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f232c162320 0x7f232c0ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:59.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.764+0000 7f23313ed640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f232c162e10 0x7f232c15ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:59.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.765+0000 7f23313ed640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f232c0cd270 con 0x7f232c0b5af0 2024-04-24T23:04:59.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.765+0000 7f23313ed640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f232c0b5660 con 0x7f232c162320 2024-04-24T23:04:59.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.765+0000 7f23313ed640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f232c071610 con 0x7f232c162e10 2024-04-24T23:04:59.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.765+0000 7f232affd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f232c162320 0x7f232c0ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:59.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.765+0000 7f232affd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f232c162320 0x7f232c0ccc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52558/0 (socket says 172.21.15.32:52558) 2024-04-24T23:04:59.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.765+0000 7f232affd640 1 -- 172.21.15.32:0/4202971347 learned_addr learned my addr 172.21.15.32:0/4202971347 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:59.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.765+0000 7f232affd640 1 -- 172.21.15.32:0/4202971347 >> v2:172.21.15.121:3300/0 conn(0x7f232c162e10 msgr2=0x7f232c15ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:59.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.765+0000 7f232b7fe640 1 --2- 172.21.15.32:0/4202971347 >> v2:172.21.15.121:3300/0 conn(0x7f232c162e10 0x7f232c15ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:59.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.765+0000 7f232affd640 1 --2- 172.21.15.32:0/4202971347 >> v2:172.21.15.121:3300/0 conn(0x7f232c162e10 0x7f232c15ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:59.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.765+0000 7f232affd640 1 -- 172.21.15.32:0/4202971347 >> v1:172.21.15.121:6789/0 conn(0x7f232c0b5af0 legacy=0x7f232c162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:04:59.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.765+0000 7f232affd640 1 -- 172.21.15.32:0/4202971347 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f232c0718f0 con 0x7f232c162320 2024-04-24T23:04:59.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.766+0000 7f232affd640 1 --2- 172.21.15.32:0/4202971347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f232c162320 0x7f232c0ccc90 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f232c0696a0 tx=0x7f231c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f869c0cdec62186 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:59.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.766+0000 7f2329ffb640 1 -- 172.21.15.32:0/4202971347 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231c037070 con 0x7f232c162320 2024-04-24T23:04:59.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.766+0000 7f2329ffb640 1 -- 172.21.15.32:0/4202971347 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f231c032440 con 0x7f232c162320 2024-04-24T23:04:59.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.766+0000 7f2329ffb640 1 -- 172.21.15.32:0/4202971347 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231c032810 con 0x7f232c162320 2024-04-24T23:04:59.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.766+0000 7f23313ed640 1 -- 172.21.15.32:0/4202971347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f232c162320 msgr2=0x7f232c0ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:59.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.766+0000 7f23313ed640 1 --2- 172.21.15.32:0/4202971347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f232c162320 0x7f232c0ccc90 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f232c0696a0 tx=0x7f231c02a1a0 comp rx=0 tx=0).stop 2024-04-24T23:04:59.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.766+0000 7f23313ed640 1 -- 172.21.15.32:0/4202971347 shutdown_connections 2024-04-24T23:04:59.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.766+0000 7f23313ed640 1 --2- 172.21.15.32:0/4202971347 >> v2:172.21.15.121:3300/0 conn(0x7f232c162e10 0x7f232c15ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:59.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.766+0000 7f23313ed640 1 --2- 172.21.15.32:0/4202971347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f232c162320 0x7f232c0ccc90 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:59.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.766+0000 7f23313ed640 1 -- 172.21.15.32:0/4202971347 >> 172.21.15.32:0/4202971347 conn(0x7f232c0d0ae0 msgr2=0x7f232c0d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:59.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.766+0000 7f23313ed640 1 -- 172.21.15.32:0/4202971347 shutdown_connections 2024-04-24T23:04:59.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.767+0000 7f23313ed640 1 -- 172.21.15.32:0/4202971347 wait complete. 2024-04-24T23:04:59.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.768+0000 7f23313ed640 1 Processor -- start 2024-04-24T23:04:59.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.768+0000 7f23313ed640 1 -- start start 2024-04-24T23:04:59.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.768+0000 7f23313ed640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f232c0b5af0 0x7f232c1f5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:59.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.769+0000 7f23313ed640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f232c162320 0x7f232c0cd720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:59.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.769+0000 7f23313ed640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f232c162e10 0x7f232c152590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:59.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.769+0000 7f23313ed640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f232c071bd0 con 0x7f232c162e10 2024-04-24T23:04:59.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.769+0000 7f23313ed640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f232c067950 con 0x7f232c0b5af0 2024-04-24T23:04:59.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.769+0000 7f23313ed640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f232c071d50 con 0x7f232c162320 2024-04-24T23:04:59.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.769+0000 7f232b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f232c162e10 0x7f232c152590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:59.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.769+0000 7f232b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f232c162e10 0x7f232c152590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52574/0 (socket says 172.21.15.32:52574) 2024-04-24T23:04:59.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.769+0000 7f232b7fe640 1 -- 172.21.15.32:0/3777775794 learned_addr learned my addr 172.21.15.32:0/3777775794 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:04:59.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.769+0000 7f232affd640 1 --2- 172.21.15.32:0/3777775794 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f232c0b5af0 0x7f232c1f5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:59.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.769+0000 7f232a7fc640 1 --2- 172.21.15.32:0/3777775794 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f232c162320 0x7f232c0cd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:59.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.769+0000 7f232b7fe640 1 -- 172.21.15.32:0/3777775794 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f232c162320 msgr2=0x7f232c0cd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:59.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.769+0000 7f232b7fe640 1 --2- 172.21.15.32:0/3777775794 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f232c162320 0x7f232c0cd720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:59.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.769+0000 7f232b7fe640 1 -- 172.21.15.32:0/3777775794 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f232c0b5af0 msgr2=0x7f232c1f5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:59.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.769+0000 7f232b7fe640 1 --2- 172.21.15.32:0/3777775794 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f232c0b5af0 0x7f232c1f5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:59.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.769+0000 7f232b7fe640 1 -- 172.21.15.32:0/3777775794 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f232c152ad0 con 0x7f232c162e10 2024-04-24T23:04:59.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.769+0000 7f232b7fe640 1 --2- 172.21.15.32:0/3777775794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f232c162e10 0x7f232c152590 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f232400cc30 tx=0x7f232400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:59.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.769+0000 7f23137fe640 1 -- 172.21.15.32:0/3777775794 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f232400ce40 con 0x7f232c162e10 2024-04-24T23:04:59.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.769+0000 7f23137fe640 1 -- 172.21.15.32:0/3777775794 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2324003990 con 0x7f232c162e10 2024-04-24T23:04:59.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.769+0000 7f23137fe640 1 -- 172.21.15.32:0/3777775794 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2324003d40 con 0x7f232c162e10 2024-04-24T23:04:59.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.769+0000 7f23313ed640 1 -- 172.21.15.32:0/3777775794 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f232c0cdea0 con 0x7f232c162e10 2024-04-24T23:04:59.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.769+0000 7f23313ed640 1 -- 172.21.15.32:0/3777775794 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f232c0ce460 con 0x7f232c162e10 2024-04-24T23:04:59.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.770+0000 7f23313ed640 1 -- 172.21.15.32:0/3777775794 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f232c0b1ca0 con 0x7f232c162e10 2024-04-24T23:04:59.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.771+0000 7f23137fe640 1 -- 172.21.15.32:0/3777775794 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2324003ee0 con 0x7f232c162e10 2024-04-24T23:04:59.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.771+0000 7f23137fe640 1 --2- 172.21.15.32:0/3777775794 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f231806be30 0x7f231806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:59.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.771+0000 7f23137fe640 1 -- 172.21.15.32:0/3777775794 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5709+0+0 (secure 0 0 0) 0x7f232408b2b0 con 0x7f232c162e10 2024-04-24T23:04:59.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.771+0000 7f232affd640 1 --2- 172.21.15.32:0/3777775794 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f231806be30 0x7f231806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:59.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.772+0000 7f232affd640 1 --2- 172.21.15.32:0/3777775794 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f231806be30 0x7f231806e2f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f231c034000 tx=0x7f231c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:59.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.775+0000 7f23137fe640 1 -- 172.21.15.32:0/3777775794 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f232c0b1ca0 con 0x7f232c162e10 2024-04-24T23:04:59.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.960+0000 7f23313ed640 1 -- 172.21.15.32:0/3777775794 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f232c0cef20 con 0x7f232c162e10 2024-04-24T23:04:59.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.961+0000 7f23137fe640 1 -- 172.21.15.32:0/3777775794 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) ==== 74+0+16632 (secure 0 0 0) 0x7f232c0cef20 con 0x7f232c162e10 2024-04-24T23:04:59.963 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:04:59.963 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":125,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:04:57.285152+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:04:54.280785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-24T23:04:54.966294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:04:59.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.963+0000 7f23313ed640 1 -- 172.21.15.32:0/3777775794 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f231806be30 msgr2=0x7f231806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:59.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.963+0000 7f23313ed640 1 --2- 172.21.15.32:0/3777775794 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f231806be30 0x7f231806e2f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f231c034000 tx=0x7f231c02e000 comp rx=0 tx=0).stop 2024-04-24T23:04:59.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.963+0000 7f23313ed640 1 -- 172.21.15.32:0/3777775794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f232c162e10 msgr2=0x7f232c152590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:59.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.963+0000 7f23313ed640 1 --2- 172.21.15.32:0/3777775794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f232c162e10 0x7f232c152590 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f232400cc30 tx=0x7f232400a5c0 comp rx=0 tx=0).stop 2024-04-24T23:04:59.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.963+0000 7f23313ed640 1 -- 172.21.15.32:0/3777775794 shutdown_connections 2024-04-24T23:04:59.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.963+0000 7f23313ed640 1 --2- 172.21.15.32:0/3777775794 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f231806be30 0x7f231806e2f0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:59.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.963+0000 7f23313ed640 1 --2- 172.21.15.32:0/3777775794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f232c162e10 0x7f232c152590 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:59.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.963+0000 7f23313ed640 1 --2- 172.21.15.32:0/3777775794 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f232c162320 0x7f232c0cd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:59.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.963+0000 7f23313ed640 1 --2- 172.21.15.32:0/3777775794 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f232c0b5af0 0x7f232c1f5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:59.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.963+0000 7f23313ed640 1 -- 172.21.15.32:0/3777775794 >> 172.21.15.32:0/3777775794 conn(0x7f232c0d0ae0 msgr2=0x7f232c0ae8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:59.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.963+0000 7f23313ed640 1 -- 172.21.15.32:0/3777775794 shutdown_connections 2024-04-24T23:04:59.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:04:59.963+0000 7f23313ed640 1 -- 172.21.15.32:0/3777775794 wait complete. 2024-04-24T23:04:59.976 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-24T23:04:59.976 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:05:00.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.076+0000 7ff8596ba640 1 Processor -- start 2024-04-24T23:05:00.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.076+0000 7ff8596ba640 1 -- start start 2024-04-24T23:05:00.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.076+0000 7ff8596ba640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff85415e9b0 0x7ff8540d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:00.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.076+0000 7ff8596ba640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff8540d4210 0x7ff85406b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:00.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.076+0000 7ff8596ba640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8540d3170 con 0x7ff8540ba0e0 2024-04-24T23:05:00.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.076+0000 7ff8596ba640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7ff8540b8900 con 0x7ff85415e9b0 2024-04-24T23:05:00.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.076+0000 7ff8596ba640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7ff8540d3450 con 0x7ff8540d4210 2024-04-24T23:05:00.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.077+0000 7ff852ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff85415e9b0 0x7ff8540d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:00.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.077+0000 7ff852ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff85415e9b0 0x7ff8540d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52584/0 (socket says 172.21.15.32:52584) 2024-04-24T23:05:00.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.077+0000 7ff852ffd640 1 -- 172.21.15.32:0/3007891939 learned_addr learned my addr 172.21.15.32:0/3007891939 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:00.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.077+0000 7ff8527fc640 1 --2- 172.21.15.32:0/3007891939 >> v2:172.21.15.121:3300/0 conn(0x7ff8540d4210 0x7ff85406b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:00.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.077+0000 7ff852ffd640 1 -- 172.21.15.32:0/3007891939 >> v2:172.21.15.121:3300/0 conn(0x7ff8540d4210 msgr2=0x7ff85406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:00.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.077+0000 7ff852ffd640 1 --2- 172.21.15.32:0/3007891939 >> v2:172.21.15.121:3300/0 conn(0x7ff8540d4210 0x7ff85406b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:00.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.077+0000 7ff852ffd640 1 -- 172.21.15.32:0/3007891939 >> v1:172.21.15.121:6789/0 conn(0x7ff8540ba0e0 legacy=0x7ff854071730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:00.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.077+0000 7ff852ffd640 1 -- 172.21.15.32:0/3007891939 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8540d3730 con 0x7ff85415e9b0 2024-04-24T23:05:00.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.077+0000 7ff8527fc640 1 --2- 172.21.15.32:0/3007891939 >> v2:172.21.15.121:3300/0 conn(0x7ff8540d4210 0x7ff85406b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:05:00.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.078+0000 7ff852ffd640 1 --2- 172.21.15.32:0/3007891939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff85415e9b0 0x7ff8540d2b90 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7ff8540730f0 tx=0x7ff84c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da980c6cfe52cf38 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:00.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.078+0000 7ff851ffb640 1 -- 172.21.15.32:0/3007891939 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84c037070 con 0x7ff85415e9b0 2024-04-24T23:05:00.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.078+0000 7ff851ffb640 1 -- 172.21.15.32:0/3007891939 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff84c032440 con 0x7ff85415e9b0 2024-04-24T23:05:00.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.078+0000 7ff851ffb640 1 -- 172.21.15.32:0/3007891939 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84c032810 con 0x7ff85415e9b0 2024-04-24T23:05:00.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.078+0000 7ff8596ba640 1 -- 172.21.15.32:0/3007891939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff85415e9b0 msgr2=0x7ff8540d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:00.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.078+0000 7ff8596ba640 1 --2- 172.21.15.32:0/3007891939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff85415e9b0 0x7ff8540d2b90 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7ff8540730f0 tx=0x7ff84c02a1e0 comp rx=0 tx=0).stop 2024-04-24T23:05:00.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.078+0000 7ff8596ba640 1 -- 172.21.15.32:0/3007891939 shutdown_connections 2024-04-24T23:05:00.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.078+0000 7ff8596ba640 1 --2- 172.21.15.32:0/3007891939 >> v2:172.21.15.121:3300/0 conn(0x7ff8540d4210 0x7ff85406b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:00.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.078+0000 7ff8596ba640 1 --2- 172.21.15.32:0/3007891939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff85415e9b0 0x7ff8540d2b90 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:00.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.078+0000 7ff8596ba640 1 -- 172.21.15.32:0/3007891939 >> 172.21.15.32:0/3007891939 conn(0x7ff8540d11d0 msgr2=0x7ff8540d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:00.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.078+0000 7ff8596ba640 1 -- 172.21.15.32:0/3007891939 shutdown_connections 2024-04-24T23:05:00.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.078+0000 7ff8596ba640 1 -- 172.21.15.32:0/3007891939 wait complete. 2024-04-24T23:05:00.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.080+0000 7ff8596ba640 1 Processor -- start 2024-04-24T23:05:00.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.080+0000 7ff8596ba640 1 -- start start 2024-04-24T23:05:00.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.080+0000 7ff8596ba640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8540ba0e0 0x7ff8540c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:00.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.080+0000 7ff8596ba640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8540d4210 0x7ff8541f8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:00.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.080+0000 7ff8596ba640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff85415e9b0 0x7ff8540cd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:00.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.080+0000 7ff8596ba640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff8540d2400 con 0x7ff8540ba0e0 2024-04-24T23:05:00.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.080+0000 7ff8596ba640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff8540d3cf0 con 0x7ff8540d4210 2024-04-24T23:05:00.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.080+0000 7ff8596ba640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff85406b650 con 0x7ff85415e9b0 2024-04-24T23:05:00.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.080+0000 7ff852ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8540ba0e0 0x7ff8540c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:00.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.081+0000 7ff852ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8540ba0e0 0x7ff8540c9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52590/0 (socket says 172.21.15.32:52590) 2024-04-24T23:05:00.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.081+0000 7ff852ffd640 1 -- 172.21.15.32:0/252251429 learned_addr learned my addr 172.21.15.32:0/252251429 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:00.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.081+0000 7ff852ffd640 1 -- 172.21.15.32:0/252251429 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff85415e9b0 msgr2=0x7ff8540cd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:00.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.081+0000 7ff852ffd640 1 --2- 172.21.15.32:0/252251429 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff85415e9b0 0x7ff8540cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:00.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.081+0000 7ff852ffd640 1 -- 172.21.15.32:0/252251429 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8540d4210 msgr2=0x7ff8541f8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:00.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.081+0000 7ff852ffd640 1 --2- 172.21.15.32:0/252251429 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8540d4210 0x7ff8541f8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:00.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.081+0000 7ff852ffd640 1 -- 172.21.15.32:0/252251429 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8541f3f70 con 0x7ff8540ba0e0 2024-04-24T23:05:00.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.081+0000 7ff852ffd640 1 --2- 172.21.15.32:0/252251429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8540ba0e0 0x7ff8540c9c50 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7ff84c02a220 tx=0x7ff84c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:00.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.081+0000 7ff83b7fe640 1 -- 172.21.15.32:0/252251429 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84c041020 con 0x7ff8540ba0e0 2024-04-24T23:05:00.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.081+0000 7ff83b7fe640 1 -- 172.21.15.32:0/252251429 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff84c0306c0 con 0x7ff8540ba0e0 2024-04-24T23:05:00.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.081+0000 7ff8596ba640 1 -- 172.21.15.32:0/252251429 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8541f85b0 con 0x7ff8540ba0e0 2024-04-24T23:05:00.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.081+0000 7ff83b7fe640 1 -- 172.21.15.32:0/252251429 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84c037040 con 0x7ff8540ba0e0 2024-04-24T23:05:00.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.081+0000 7ff8596ba640 1 -- 172.21.15.32:0/252251429 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff85415af50 con 0x7ff8540ba0e0 2024-04-24T23:05:00.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.082+0000 7ff8596ba640 1 -- 172.21.15.32:0/252251429 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff854068430 con 0x7ff8540ba0e0 2024-04-24T23:05:00.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.083+0000 7ff83b7fe640 1 -- 172.21.15.32:0/252251429 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff84c044020 con 0x7ff8540ba0e0 2024-04-24T23:05:00.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.083+0000 7ff83b7fe640 1 --2- 172.21.15.32:0/252251429 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff84006be30 0x7ff84006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:00.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.083+0000 7ff83b7fe640 1 -- 172.21.15.32:0/252251429 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5709+0+0 (secure 0 0 0) 0x7ff84c0ab6d0 con 0x7ff8540ba0e0 2024-04-24T23:05:00.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.083+0000 7ff8527fc640 1 --2- 172.21.15.32:0/252251429 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff84006be30 0x7ff84006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:00.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.084+0000 7ff8527fc640 1 --2- 172.21.15.32:0/252251429 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff84006be30 0x7ff84006e2f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ff848006fd0 tx=0x7ff848008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:00.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.087+0000 7ff83b7fe640 1 -- 172.21.15.32:0/252251429 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff84c04b030 con 0x7ff8540ba0e0 2024-04-24T23:05:00.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.271+0000 7ff8596ba640 1 -- 172.21.15.32:0/252251429 --> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff85415dff0 con 0x7ff84006be30 2024-04-24T23:05:00.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.278+0000 7ff83b7fe640 1 -- 172.21.15.32:0/252251429 <== mgr.4108 v2:172.21.15.121:6832/1848157943 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247923 (secure 0 0 0) 0x7ff85415dff0 con 0x7ff84006be30 2024-04-24T23:05:00.282 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:05:00.283 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-24T23:05:00.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.281+0000 7ff8596ba640 1 -- 172.21.15.32:0/252251429 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff84006be30 msgr2=0x7ff84006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:00.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.281+0000 7ff8596ba640 1 --2- 172.21.15.32:0/252251429 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff84006be30 0x7ff84006e2f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ff848006fd0 tx=0x7ff848008040 comp rx=0 tx=0).stop 2024-04-24T23:05:00.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.281+0000 7ff8596ba640 1 -- 172.21.15.32:0/252251429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8540ba0e0 msgr2=0x7ff8540c9c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:00.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.281+0000 7ff8596ba640 1 --2- 172.21.15.32:0/252251429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8540ba0e0 0x7ff8540c9c50 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7ff84c02a220 tx=0x7ff84c0046c0 comp rx=0 tx=0).stop 2024-04-24T23:05:00.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.281+0000 7ff8596ba640 1 -- 172.21.15.32:0/252251429 shutdown_connections 2024-04-24T23:05:00.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.281+0000 7ff8596ba640 1 --2- 172.21.15.32:0/252251429 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff84006be30 0x7ff84006e2f0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:00.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.281+0000 7ff8596ba640 1 --2- 172.21.15.32:0/252251429 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff85415e9b0 0x7ff8540cd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:00.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.281+0000 7ff8596ba640 1 --2- 172.21.15.32:0/252251429 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8540d4210 0x7ff8541f8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:00.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.281+0000 7ff8596ba640 1 --2- 172.21.15.32:0/252251429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8540ba0e0 0x7ff8540c9c50 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:00.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.281+0000 7ff8596ba640 1 -- 172.21.15.32:0/252251429 >> 172.21.15.32:0/252251429 conn(0x7ff8540d11d0 msgr2=0x7ff85415d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:00.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.281+0000 7ff8596ba640 1 -- 172.21.15.32:0/252251429 shutdown_connections 2024-04-24T23:05:00.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.281+0000 7ff8596ba640 1 -- 172.21.15.32:0/252251429 wait complete. 2024-04-24T23:05:00.293 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":300,"stamp":"2024-04-24T23:04:59.209736+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":107,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6491,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1556008,"statfs":{"total":1600000000,"available":1593352218,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.594290"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.299417+0000","last_change":"2024-04-24T23:04:56.299069+0000","last_active":"2024-04-24T23:04:56.299417+0000","last_peered":"2024-04-24T23:04:56.299417+0000","last_clean":"2024-04-24T23:04:56.299417+0000","last_became_active":"2024-04-24T23:04:56.296532+0000","last_became_peered":"2024-04-24T23:04:56.296532+0000","last_unstale":"2024-04-24T23:04:56.299417+0000","last_undegraded":"2024-04-24T23:04:56.299417+0000","last_fullsized":"2024-04-24T23:04:56.299417+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:07:58.343128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.298738+0000","last_change":"2024-04-24T23:04:56.298610+0000","last_active":"2024-04-24T23:04:56.298738+0000","last_peered":"2024-04-24T23:04:56.298738+0000","last_clean":"2024-04-24T23:04:56.298738+0000","last_became_active":"2024-04-24T23:04:56.298063+0000","last_became_peered":"2024-04-24T23:04:56.298063+0000","last_unstale":"2024-04-24T23:04:56.298738+0000","last_undegraded":"2024-04-24T23:04:56.298738+0000","last_fullsized":"2024-04-24T23:04:56.298738+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:29:13.057885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.301954+0000","last_change":"2024-04-24T23:04:56.301838+0000","last_active":"2024-04-24T23:04:56.301954+0000","last_peered":"2024-04-24T23:04:56.301954+0000","last_clean":"2024-04-24T23:04:56.301954+0000","last_became_active":"2024-04-24T23:04:56.301608+0000","last_became_peered":"2024-04-24T23:04:56.301608+0000","last_unstale":"2024-04-24T23:04:56.301954+0000","last_undegraded":"2024-04-24T23:04:56.301954+0000","last_fullsized":"2024-04-24T23:04:56.301954+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:57.464174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.299897+0000","last_change":"2024-04-24T23:04:56.299648+0000","last_active":"2024-04-24T23:04:56.299897+0000","last_peered":"2024-04-24T23:04:56.299897+0000","last_clean":"2024-04-24T23:04:56.299897+0000","last_became_active":"2024-04-24T23:04:56.299337+0000","last_became_peered":"2024-04-24T23:04:56.299337+0000","last_unstale":"2024-04-24T23:04:56.299897+0000","last_undegraded":"2024-04-24T23:04:56.299897+0000","last_fullsized":"2024-04-24T23:04:56.299897+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:51.404191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.296489+0000","last_change":"2024-04-24T23:04:56.296306+0000","last_active":"2024-04-24T23:04:56.296489+0000","last_peered":"2024-04-24T23:04:56.296489+0000","last_clean":"2024-04-24T23:04:56.296489+0000","last_became_active":"2024-04-24T23:04:56.296021+0000","last_became_peered":"2024-04-24T23:04:56.296021+0000","last_unstale":"2024-04-24T23:04:56.296489+0000","last_undegraded":"2024-04-24T23:04:56.296489+0000","last_fullsized":"2024-04-24T23:04:56.296489+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:20:31.166345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.299993+0000","last_change":"2024-04-24T23:04:56.298933+0000","last_active":"2024-04-24T23:04:56.299993+0000","last_peered":"2024-04-24T23:04:56.299993+0000","last_clean":"2024-04-24T23:04:56.299993+0000","last_became_active":"2024-04-24T23:04:56.298480+0000","last_became_peered":"2024-04-24T23:04:56.298480+0000","last_unstale":"2024-04-24T23:04:56.299993+0000","last_undegraded":"2024-04-24T23:04:56.299993+0000","last_fullsized":"2024-04-24T23:04:56.299993+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:40:22.228542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.295471+0000","last_change":"2024-04-24T23:04:56.294829+0000","last_active":"2024-04-24T23:04:56.295471+0000","last_peered":"2024-04-24T23:04:56.295471+0000","last_clean":"2024-04-24T23:04:56.295471+0000","last_became_active":"2024-04-24T23:04:56.294572+0000","last_became_peered":"2024-04-24T23:04:56.294572+0000","last_unstale":"2024-04-24T23:04:56.295471+0000","last_undegraded":"2024-04-24T23:04:56.295471+0000","last_fullsized":"2024-04-24T23:04:56.295471+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:22:05.535544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.301333+0000","last_change":"2024-04-24T23:04:56.301218+0000","last_active":"2024-04-24T23:04:56.301333+0000","last_peered":"2024-04-24T23:04:56.301333+0000","last_clean":"2024-04-24T23:04:56.301333+0000","last_became_active":"2024-04-24T23:04:56.300935+0000","last_became_peered":"2024-04-24T23:04:56.300935+0000","last_unstale":"2024-04-24T23:04:56.301333+0000","last_undegraded":"2024-04-24T23:04:56.301333+0000","last_fullsized":"2024-04-24T23:04:56.301333+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:47:47.452807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.299959+0000","last_change":"2024-04-24T23:04:56.298604+0000","last_active":"2024-04-24T23:04:56.299959+0000","last_peered":"2024-04-24T23:04:56.299959+0000","last_clean":"2024-04-24T23:04:56.299959+0000","last_became_active":"2024-04-24T23:04:56.298374+0000","last_became_peered":"2024-04-24T23:04:56.298374+0000","last_unstale":"2024-04-24T23:04:56.299959+0000","last_undegraded":"2024-04-24T23:04:56.299959+0000","last_fullsized":"2024-04-24T23:04:56.299959+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:04.889874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.297123+0000","last_change":"2024-04-24T23:04:56.297011+0000","last_active":"2024-04-24T23:04:56.297123+0000","last_peered":"2024-04-24T23:04:56.297123+0000","last_clean":"2024-04-24T23:04:56.297123+0000","last_became_active":"2024-04-24T23:04:56.296508+0000","last_became_peered":"2024-04-24T23:04:56.296508+0000","last_unstale":"2024-04-24T23:04:56.297123+0000","last_undegraded":"2024-04-24T23:04:56.297123+0000","last_fullsized":"2024-04-24T23:04:56.297123+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:14:29.838138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.299976+0000","last_change":"2024-04-24T23:04:56.298747+0000","last_active":"2024-04-24T23:04:56.299976+0000","last_peered":"2024-04-24T23:04:56.299976+0000","last_clean":"2024-04-24T23:04:56.299976+0000","last_became_active":"2024-04-24T23:04:56.298159+0000","last_became_peered":"2024-04-24T23:04:56.298159+0000","last_unstale":"2024-04-24T23:04:56.299976+0000","last_undegraded":"2024-04-24T23:04:56.299976+0000","last_fullsized":"2024-04-24T23:04:56.299976+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:26:30.517186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.297252+0000","last_change":"2024-04-24T23:04:56.296477+0000","last_active":"2024-04-24T23:04:56.297252+0000","last_peered":"2024-04-24T23:04:56.297252+0000","last_clean":"2024-04-24T23:04:56.297252+0000","last_became_active":"2024-04-24T23:04:56.296252+0000","last_became_peered":"2024-04-24T23:04:56.296252+0000","last_unstale":"2024-04-24T23:04:56.297252+0000","last_undegraded":"2024-04-24T23:04:56.297252+0000","last_fullsized":"2024-04-24T23:04:56.297252+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:56:06.843889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.306078+0000","last_change":"2024-04-24T23:04:56.305944+0000","last_active":"2024-04-24T23:04:56.306078+0000","last_peered":"2024-04-24T23:04:56.306078+0000","last_clean":"2024-04-24T23:04:56.306078+0000","last_became_active":"2024-04-24T23:04:56.305502+0000","last_became_peered":"2024-04-24T23:04:56.305502+0000","last_unstale":"2024-04-24T23:04:56.306078+0000","last_undegraded":"2024-04-24T23:04:56.306078+0000","last_fullsized":"2024-04-24T23:04:56.306078+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:08:56.967524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.305696+0000","last_change":"2024-04-24T23:04:56.305576+0000","last_active":"2024-04-24T23:04:56.305696+0000","last_peered":"2024-04-24T23:04:56.305696+0000","last_clean":"2024-04-24T23:04:56.305696+0000","last_became_active":"2024-04-24T23:04:56.305325+0000","last_became_peered":"2024-04-24T23:04:56.305325+0000","last_unstale":"2024-04-24T23:04:56.305696+0000","last_undegraded":"2024-04-24T23:04:56.305696+0000","last_fullsized":"2024-04-24T23:04:56.305696+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:58.990812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.304817+0000","last_change":"2024-04-24T23:04:56.304395+0000","last_active":"2024-04-24T23:04:56.304817+0000","last_peered":"2024-04-24T23:04:56.304817+0000","last_clean":"2024-04-24T23:04:56.304817+0000","last_became_active":"2024-04-24T23:04:56.304032+0000","last_became_peered":"2024-04-24T23:04:56.304032+0000","last_unstale":"2024-04-24T23:04:56.304817+0000","last_undegraded":"2024-04-24T23:04:56.304817+0000","last_fullsized":"2024-04-24T23:04:56.304817+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:11:03.231462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.295553+0000","last_change":"2024-04-24T23:04:56.295452+0000","last_active":"2024-04-24T23:04:56.295553+0000","last_peered":"2024-04-24T23:04:56.295553+0000","last_clean":"2024-04-24T23:04:56.295553+0000","last_became_active":"2024-04-24T23:04:56.295226+0000","last_became_peered":"2024-04-24T23:04:56.295226+0000","last_unstale":"2024-04-24T23:04:56.295553+0000","last_undegraded":"2024-04-24T23:04:56.295553+0000","last_fullsized":"2024-04-24T23:04:56.295553+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:04:17.234668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.304838+0000","last_change":"2024-04-24T23:04:56.304379+0000","last_active":"2024-04-24T23:04:56.304838+0000","last_peered":"2024-04-24T23:04:56.304838+0000","last_clean":"2024-04-24T23:04:56.304838+0000","last_became_active":"2024-04-24T23:04:56.304141+0000","last_became_peered":"2024-04-24T23:04:56.304141+0000","last_unstale":"2024-04-24T23:04:56.304838+0000","last_undegraded":"2024-04-24T23:04:56.304838+0000","last_fullsized":"2024-04-24T23:04:56.304838+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:09:15.954439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.299370+0000","last_change":"2024-04-24T23:04:56.299066+0000","last_active":"2024-04-24T23:04:56.299370+0000","last_peered":"2024-04-24T23:04:56.299370+0000","last_clean":"2024-04-24T23:04:56.299370+0000","last_became_active":"2024-04-24T23:04:56.298777+0000","last_became_peered":"2024-04-24T23:04:56.298777+0000","last_unstale":"2024-04-24T23:04:56.299370+0000","last_undegraded":"2024-04-24T23:04:56.299370+0000","last_fullsized":"2024-04-24T23:04:56.299370+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:31:24.305630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.300569+0000","last_change":"2024-04-24T23:04:56.299070+0000","last_active":"2024-04-24T23:04:56.300569+0000","last_peered":"2024-04-24T23:04:56.300569+0000","last_clean":"2024-04-24T23:04:56.300569+0000","last_became_active":"2024-04-24T23:04:56.298605+0000","last_became_peered":"2024-04-24T23:04:56.298605+0000","last_unstale":"2024-04-24T23:04:56.300569+0000","last_undegraded":"2024-04-24T23:04:56.300569+0000","last_fullsized":"2024-04-24T23:04:56.300569+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:55:26.412963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.306039+0000","last_change":"2024-04-24T23:04:56.305938+0000","last_active":"2024-04-24T23:04:56.306039+0000","last_peered":"2024-04-24T23:04:56.306039+0000","last_clean":"2024-04-24T23:04:56.306039+0000","last_became_active":"2024-04-24T23:04:56.305699+0000","last_became_peered":"2024-04-24T23:04:56.305699+0000","last_unstale":"2024-04-24T23:04:56.306039+0000","last_undegraded":"2024-04-24T23:04:56.306039+0000","last_fullsized":"2024-04-24T23:04:56.306039+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:29:53.907588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.305124+0000","last_change":"2024-04-24T23:04:56.304956+0000","last_active":"2024-04-24T23:04:56.305124+0000","last_peered":"2024-04-24T23:04:56.305124+0000","last_clean":"2024-04-24T23:04:56.305124+0000","last_became_active":"2024-04-24T23:04:56.303397+0000","last_became_peered":"2024-04-24T23:04:56.303397+0000","last_unstale":"2024-04-24T23:04:56.305124+0000","last_undegraded":"2024-04-24T23:04:56.305124+0000","last_fullsized":"2024-04-24T23:04:56.305124+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:03:37.194732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.299987+0000","last_change":"2024-04-24T23:04:56.298680+0000","last_active":"2024-04-24T23:04:56.299987+0000","last_peered":"2024-04-24T23:04:56.299987+0000","last_clean":"2024-04-24T23:04:56.299987+0000","last_became_active":"2024-04-24T23:04:56.297332+0000","last_became_peered":"2024-04-24T23:04:56.297332+0000","last_unstale":"2024-04-24T23:04:56.299987+0000","last_undegraded":"2024-04-24T23:04:56.299987+0000","last_fullsized":"2024-04-24T23:04:56.299987+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:05.152014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.304838+0000","last_change":"2024-04-24T23:04:56.304457+0000","last_active":"2024-04-24T23:04:56.304838+0000","last_peered":"2024-04-24T23:04:56.304838+0000","last_clean":"2024-04-24T23:04:56.304838+0000","last_became_active":"2024-04-24T23:04:56.304142+0000","last_became_peered":"2024-04-24T23:04:56.304142+0000","last_unstale":"2024-04-24T23:04:56.304838+0000","last_undegraded":"2024-04-24T23:04:56.304838+0000","last_fullsized":"2024-04-24T23:04:56.304838+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:42:08.073443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.292732+0000","last_change":"2024-04-24T23:04:56.292630+0000","last_active":"2024-04-24T23:04:56.292732+0000","last_peered":"2024-04-24T23:04:56.292732+0000","last_clean":"2024-04-24T23:04:56.292732+0000","last_became_active":"2024-04-24T23:04:56.292363+0000","last_became_peered":"2024-04-24T23:04:56.292363+0000","last_unstale":"2024-04-24T23:04:56.292732+0000","last_undegraded":"2024-04-24T23:04:56.292732+0000","last_fullsized":"2024-04-24T23:04:56.292732+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:00:31.940037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.299766+0000","last_change":"2024-04-24T23:04:56.299627+0000","last_active":"2024-04-24T23:04:56.299766+0000","last_peered":"2024-04-24T23:04:56.299766+0000","last_clean":"2024-04-24T23:04:56.299766+0000","last_became_active":"2024-04-24T23:04:56.299375+0000","last_became_peered":"2024-04-24T23:04:56.299375+0000","last_unstale":"2024-04-24T23:04:56.299766+0000","last_undegraded":"2024-04-24T23:04:56.299766+0000","last_fullsized":"2024-04-24T23:04:56.299766+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:57.144816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.303657+0000","last_change":"2024-04-24T23:04:56.303514+0000","last_active":"2024-04-24T23:04:56.303657+0000","last_peered":"2024-04-24T23:04:56.303657+0000","last_clean":"2024-04-24T23:04:56.303657+0000","last_became_active":"2024-04-24T23:04:56.303264+0000","last_became_peered":"2024-04-24T23:04:56.303264+0000","last_unstale":"2024-04-24T23:04:56.303657+0000","last_undegraded":"2024-04-24T23:04:56.303657+0000","last_fullsized":"2024-04-24T23:04:56.303657+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:24:11.103964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.297294+0000","last_change":"2024-04-24T23:04:56.297126+0000","last_active":"2024-04-24T23:04:56.297294+0000","last_peered":"2024-04-24T23:04:56.297294+0000","last_clean":"2024-04-24T23:04:56.297294+0000","last_became_active":"2024-04-24T23:04:56.296856+0000","last_became_peered":"2024-04-24T23:04:56.296856+0000","last_unstale":"2024-04-24T23:04:56.297294+0000","last_undegraded":"2024-04-24T23:04:56.297294+0000","last_fullsized":"2024-04-24T23:04:56.297294+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:27:09.210347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.298561+0000","last_change":"2024-04-24T23:04:56.298261+0000","last_active":"2024-04-24T23:04:56.298561+0000","last_peered":"2024-04-24T23:04:56.298561+0000","last_clean":"2024-04-24T23:04:56.298561+0000","last_became_active":"2024-04-24T23:04:56.297574+0000","last_became_peered":"2024-04-24T23:04:56.297574+0000","last_unstale":"2024-04-24T23:04:56.298561+0000","last_undegraded":"2024-04-24T23:04:56.298561+0000","last_fullsized":"2024-04-24T23:04:56.298561+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:18.121624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.306632+0000","last_change":"2024-04-24T23:04:56.306507+0000","last_active":"2024-04-24T23:04:56.306632+0000","last_peered":"2024-04-24T23:04:56.306632+0000","last_clean":"2024-04-24T23:04:56.306632+0000","last_became_active":"2024-04-24T23:04:56.306144+0000","last_became_peered":"2024-04-24T23:04:56.306144+0000","last_unstale":"2024-04-24T23:04:56.306632+0000","last_undegraded":"2024-04-24T23:04:56.306632+0000","last_fullsized":"2024-04-24T23:04:56.306632+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:07:23.208212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.306612+0000","last_change":"2024-04-24T23:04:56.306502+0000","last_active":"2024-04-24T23:04:56.306612+0000","last_peered":"2024-04-24T23:04:56.306612+0000","last_clean":"2024-04-24T23:04:56.306612+0000","last_became_active":"2024-04-24T23:04:56.305814+0000","last_became_peered":"2024-04-24T23:04:56.305814+0000","last_unstale":"2024-04-24T23:04:56.306612+0000","last_undegraded":"2024-04-24T23:04:56.306612+0000","last_fullsized":"2024-04-24T23:04:56.306612+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:30:14.988093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.304810+0000","last_change":"2024-04-24T23:04:56.304695+0000","last_active":"2024-04-24T23:04:56.304810+0000","last_peered":"2024-04-24T23:04:56.304810+0000","last_clean":"2024-04-24T23:04:56.304810+0000","last_became_active":"2024-04-24T23:04:56.302930+0000","last_became_peered":"2024-04-24T23:04:56.302930+0000","last_unstale":"2024-04-24T23:04:56.304810+0000","last_undegraded":"2024-04-24T23:04:56.304810+0000","last_fullsized":"2024-04-24T23:04:56.304810+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:27.550752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.296877+0000","last_change":"2024-04-24T23:04:56.296717+0000","last_active":"2024-04-24T23:04:56.296877+0000","last_peered":"2024-04-24T23:04:56.296877+0000","last_clean":"2024-04-24T23:04:56.296877+0000","last_became_active":"2024-04-24T23:04:56.296428+0000","last_became_peered":"2024-04-24T23:04:56.296428+0000","last_unstale":"2024-04-24T23:04:56.296877+0000","last_undegraded":"2024-04-24T23:04:56.296877+0000","last_fullsized":"2024-04-24T23:04:56.296877+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:30.891284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.300546+0000","last_change":"2024-04-24T23:04:56.298871+0000","last_active":"2024-04-24T23:04:56.300546+0000","last_peered":"2024-04-24T23:04:56.300546+0000","last_clean":"2024-04-24T23:04:56.300546+0000","last_became_active":"2024-04-24T23:04:56.298333+0000","last_became_peered":"2024-04-24T23:04:56.298333+0000","last_unstale":"2024-04-24T23:04:56.300546+0000","last_undegraded":"2024-04-24T23:04:56.300546+0000","last_fullsized":"2024-04-24T23:04:56.300546+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:46:54.546573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"124'4","reported_seq":16,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.633192+0000","last_change":"2024-04-24T23:04:55.640119+0000","last_active":"2024-04-24T23:04:56.633192+0000","last_peered":"2024-04-24T23:04:56.633192+0000","last_clean":"2024-04-24T23:04:56.633192+0000","last_became_active":"2024-04-24T23:04:55.639812+0000","last_became_peered":"2024-04-24T23:04:55.639812+0000","last_unstale":"2024-04-24T23:04:56.633192+0000","last_undegraded":"2024-04-24T23:04:56.633192+0000","last_fullsized":"2024-04-24T23:04:56.633192+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:04:50.132779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.305167+0000","last_change":"2024-04-24T23:04:56.302433+0000","last_active":"2024-04-24T23:04:56.305167+0000","last_peered":"2024-04-24T23:04:56.305167+0000","last_clean":"2024-04-24T23:04:56.305167+0000","last_became_active":"2024-04-24T23:04:56.302207+0000","last_became_peered":"2024-04-24T23:04:56.302207+0000","last_unstale":"2024-04-24T23:04:56.305167+0000","last_undegraded":"2024-04-24T23:04:56.305167+0000","last_fullsized":"2024-04-24T23:04:56.305167+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:46:05.959174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"124'1","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.631597+0000","last_change":"2024-04-24T23:04:55.632889+0000","last_active":"2024-04-24T23:04:56.631597+0000","last_peered":"2024-04-24T23:04:56.631597+0000","last_clean":"2024-04-24T23:04:56.631597+0000","last_became_active":"2024-04-24T23:04:55.632655+0000","last_became_peered":"2024-04-24T23:04:55.632655+0000","last_unstale":"2024-04-24T23:04:56.631597+0000","last_undegraded":"2024-04-24T23:04:56.631597+0000","last_fullsized":"2024-04-24T23:04:56.631597+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:35:34.066639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.297359+0000","last_change":"2024-04-24T23:04:56.297138+0000","last_active":"2024-04-24T23:04:56.297359+0000","last_peered":"2024-04-24T23:04:56.297359+0000","last_clean":"2024-04-24T23:04:56.297359+0000","last_became_active":"2024-04-24T23:04:56.295653+0000","last_became_peered":"2024-04-24T23:04:56.295653+0000","last_unstale":"2024-04-24T23:04:56.297359+0000","last_undegraded":"2024-04-24T23:04:56.297359+0000","last_fullsized":"2024-04-24T23:04:56.297359+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:56:37.458428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"124'2","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.632513+0000","last_change":"2024-04-24T23:04:55.639981+0000","last_active":"2024-04-24T23:04:56.632513+0000","last_peered":"2024-04-24T23:04:56.632513+0000","last_clean":"2024-04-24T23:04:56.632513+0000","last_became_active":"2024-04-24T23:04:55.639262+0000","last_became_peered":"2024-04-24T23:04:55.639262+0000","last_unstale":"2024-04-24T23:04:56.632513+0000","last_undegraded":"2024-04-24T23:04:56.632513+0000","last_fullsized":"2024-04-24T23:04:56.632513+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:22:56.035112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.296892+0000","last_change":"2024-04-24T23:04:56.296784+0000","last_active":"2024-04-24T23:04:56.296892+0000","last_peered":"2024-04-24T23:04:56.296892+0000","last_clean":"2024-04-24T23:04:56.296892+0000","last_became_active":"2024-04-24T23:04:56.296194+0000","last_became_peered":"2024-04-24T23:04:56.296194+0000","last_unstale":"2024-04-24T23:04:56.296892+0000","last_undegraded":"2024-04-24T23:04:56.296892+0000","last_fullsized":"2024-04-24T23:04:56.296892+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:48.318106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"124'2","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.633189+0000","last_change":"2024-04-24T23:04:55.641448+0000","last_active":"2024-04-24T23:04:56.633189+0000","last_peered":"2024-04-24T23:04:56.633189+0000","last_clean":"2024-04-24T23:04:56.633189+0000","last_became_active":"2024-04-24T23:04:55.640749+0000","last_became_peered":"2024-04-24T23:04:55.640749+0000","last_unstale":"2024-04-24T23:04:56.633189+0000","last_undegraded":"2024-04-24T23:04:56.633189+0000","last_fullsized":"2024-04-24T23:04:56.633189+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:26:42.681740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.301472+0000","last_change":"2024-04-24T23:04:56.301322+0000","last_active":"2024-04-24T23:04:56.301472+0000","last_peered":"2024-04-24T23:04:56.301472+0000","last_clean":"2024-04-24T23:04:56.301472+0000","last_became_active":"2024-04-24T23:04:56.301081+0000","last_became_peered":"2024-04-24T23:04:56.301081+0000","last_unstale":"2024-04-24T23:04:56.301472+0000","last_undegraded":"2024-04-24T23:04:56.301472+0000","last_fullsized":"2024-04-24T23:04:56.301472+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:43:43.227101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"124'1","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.632445+0000","last_change":"2024-04-24T23:04:55.634733+0000","last_active":"2024-04-24T23:04:56.632445+0000","last_peered":"2024-04-24T23:04:56.632445+0000","last_clean":"2024-04-24T23:04:56.632445+0000","last_became_active":"2024-04-24T23:04:55.634440+0000","last_became_peered":"2024-04-24T23:04:55.634440+0000","last_unstale":"2024-04-24T23:04:56.632445+0000","last_undegraded":"2024-04-24T23:04:56.632445+0000","last_fullsized":"2024-04-24T23:04:56.632445+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:45:23.578055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.299938+0000","last_change":"2024-04-24T23:04:56.298387+0000","last_active":"2024-04-24T23:04:56.299938+0000","last_peered":"2024-04-24T23:04:56.299938+0000","last_clean":"2024-04-24T23:04:56.299938+0000","last_became_active":"2024-04-24T23:04:56.298018+0000","last_became_peered":"2024-04-24T23:04:56.298018+0000","last_unstale":"2024-04-24T23:04:56.299938+0000","last_undegraded":"2024-04-24T23:04:56.299938+0000","last_fullsized":"2024-04-24T23:04:56.299938+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:14:04.049719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"124'1","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.632689+0000","last_change":"2024-04-24T23:04:55.641651+0000","last_active":"2024-04-24T23:04:56.632689+0000","last_peered":"2024-04-24T23:04:56.632689+0000","last_clean":"2024-04-24T23:04:56.632689+0000","last_became_active":"2024-04-24T23:04:55.641141+0000","last_became_peered":"2024-04-24T23:04:55.641141+0000","last_unstale":"2024-04-24T23:04:56.632689+0000","last_undegraded":"2024-04-24T23:04:56.632689+0000","last_fullsized":"2024-04-24T23:04:56.632689+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:05:41.689378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.306280+0000","last_change":"2024-04-24T23:04:56.306160+0000","last_active":"2024-04-24T23:04:56.306280+0000","last_peered":"2024-04-24T23:04:56.306280+0000","last_clean":"2024-04-24T23:04:56.306280+0000","last_became_active":"2024-04-24T23:04:56.305413+0000","last_became_peered":"2024-04-24T23:04:56.305413+0000","last_unstale":"2024-04-24T23:04:56.306280+0000","last_undegraded":"2024-04-24T23:04:56.306280+0000","last_fullsized":"2024-04-24T23:04:56.306280+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:06:22.157421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.287011+0000","last_change":"2024-04-24T23:04:55.635238+0000","last_active":"2024-04-24T23:04:56.287011+0000","last_peered":"2024-04-24T23:04:56.287011+0000","last_clean":"2024-04-24T23:04:56.287011+0000","last_became_active":"2024-04-24T23:04:55.634924+0000","last_became_peered":"2024-04-24T23:04:55.634924+0000","last_unstale":"2024-04-24T23:04:56.287011+0000","last_undegraded":"2024-04-24T23:04:56.287011+0000","last_fullsized":"2024-04-24T23:04:56.287011+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:40:47.430362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.298044+0000","last_change":"2024-04-24T23:04:56.297932+0000","last_active":"2024-04-24T23:04:56.298044+0000","last_peered":"2024-04-24T23:04:56.298044+0000","last_clean":"2024-04-24T23:04:56.298044+0000","last_became_active":"2024-04-24T23:04:56.297694+0000","last_became_peered":"2024-04-24T23:04:56.297694+0000","last_unstale":"2024-04-24T23:04:56.298044+0000","last_undegraded":"2024-04-24T23:04:56.298044+0000","last_fullsized":"2024-04-24T23:04:56.298044+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:17:20.878502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.293312+0000","last_change":"2024-04-24T23:04:55.636020+0000","last_active":"2024-04-24T23:04:56.293312+0000","last_peered":"2024-04-24T23:04:56.293312+0000","last_clean":"2024-04-24T23:04:56.293312+0000","last_became_active":"2024-04-24T23:04:55.635763+0000","last_became_peered":"2024-04-24T23:04:55.635763+0000","last_unstale":"2024-04-24T23:04:56.293312+0000","last_undegraded":"2024-04-24T23:04:56.293312+0000","last_fullsized":"2024-04-24T23:04:56.293312+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:17:45.085895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.301003+0000","last_change":"2024-04-24T23:04:56.300873+0000","last_active":"2024-04-24T23:04:56.301003+0000","last_peered":"2024-04-24T23:04:56.301003+0000","last_clean":"2024-04-24T23:04:56.301003+0000","last_became_active":"2024-04-24T23:04:56.300140+0000","last_became_peered":"2024-04-24T23:04:56.300140+0000","last_unstale":"2024-04-24T23:04:56.301003+0000","last_undegraded":"2024-04-24T23:04:56.301003+0000","last_fullsized":"2024-04-24T23:04:56.301003+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:26:30.312012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.299950+0000","last_change":"2024-04-24T23:04:55.640569+0000","last_active":"2024-04-24T23:04:56.299950+0000","last_peered":"2024-04-24T23:04:56.299950+0000","last_clean":"2024-04-24T23:04:56.299950+0000","last_became_active":"2024-04-24T23:04:55.639955+0000","last_became_peered":"2024-04-24T23:04:55.639955+0000","last_unstale":"2024-04-24T23:04:56.299950+0000","last_undegraded":"2024-04-24T23:04:56.299950+0000","last_fullsized":"2024-04-24T23:04:56.299950+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:52:55.132318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.300025+0000","last_change":"2024-04-24T23:04:56.299898+0000","last_active":"2024-04-24T23:04:56.300025+0000","last_peered":"2024-04-24T23:04:56.300025+0000","last_clean":"2024-04-24T23:04:56.300025+0000","last_became_active":"2024-04-24T23:04:56.296789+0000","last_became_peered":"2024-04-24T23:04:56.296789+0000","last_unstale":"2024-04-24T23:04:56.300025+0000","last_undegraded":"2024-04-24T23:04:56.300025+0000","last_fullsized":"2024-04-24T23:04:56.300025+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:09.824794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.292019+0000","last_change":"2024-04-24T23:04:55.638415+0000","last_active":"2024-04-24T23:04:56.292019+0000","last_peered":"2024-04-24T23:04:56.292019+0000","last_clean":"2024-04-24T23:04:56.292019+0000","last_became_active":"2024-04-24T23:04:55.638142+0000","last_became_peered":"2024-04-24T23:04:55.638142+0000","last_unstale":"2024-04-24T23:04:56.292019+0000","last_undegraded":"2024-04-24T23:04:56.292019+0000","last_fullsized":"2024-04-24T23:04:56.292019+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:41:15.782557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.292646+0000","last_change":"2024-04-24T23:04:56.292499+0000","last_active":"2024-04-24T23:04:56.292646+0000","last_peered":"2024-04-24T23:04:56.292646+0000","last_clean":"2024-04-24T23:04:56.292646+0000","last_became_active":"2024-04-24T23:04:56.292252+0000","last_became_peered":"2024-04-24T23:04:56.292252+0000","last_unstale":"2024-04-24T23:04:56.292646+0000","last_undegraded":"2024-04-24T23:04:56.292646+0000","last_fullsized":"2024-04-24T23:04:56.292646+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:00:58.925068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"124'3","reported_seq":15,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.632767+0000","last_change":"2024-04-24T23:04:55.636011+0000","last_active":"2024-04-24T23:04:56.632767+0000","last_peered":"2024-04-24T23:04:56.632767+0000","last_clean":"2024-04-24T23:04:56.632767+0000","last_became_active":"2024-04-24T23:04:55.635750+0000","last_became_peered":"2024-04-24T23:04:55.635750+0000","last_unstale":"2024-04-24T23:04:56.632767+0000","last_undegraded":"2024-04-24T23:04:56.632767+0000","last_fullsized":"2024-04-24T23:04:56.632767+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:34:14.335998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.295895+0000","last_change":"2024-04-24T23:04:56.295318+0000","last_active":"2024-04-24T23:04:56.295895+0000","last_peered":"2024-04-24T23:04:56.295895+0000","last_clean":"2024-04-24T23:04:56.295895+0000","last_became_active":"2024-04-24T23:04:56.295088+0000","last_became_peered":"2024-04-24T23:04:56.295088+0000","last_unstale":"2024-04-24T23:04:56.295895+0000","last_undegraded":"2024-04-24T23:04:56.295895+0000","last_fullsized":"2024-04-24T23:04:56.295895+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:41:28.277210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"124'6","reported_seq":18,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.634845+0000","last_change":"2024-04-24T23:04:55.638235+0000","last_active":"2024-04-24T23:04:56.634845+0000","last_peered":"2024-04-24T23:04:56.634845+0000","last_clean":"2024-04-24T23:04:56.634845+0000","last_became_active":"2024-04-24T23:04:55.637985+0000","last_became_peered":"2024-04-24T23:04:55.637985+0000","last_unstale":"2024-04-24T23:04:56.634845+0000","last_undegraded":"2024-04-24T23:04:56.634845+0000","last_fullsized":"2024-04-24T23:04:56.634845+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:55:16.497642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.296301+0000","last_change":"2024-04-24T23:04:56.296154+0000","last_active":"2024-04-24T23:04:56.296301+0000","last_peered":"2024-04-24T23:04:56.296301+0000","last_clean":"2024-04-24T23:04:56.296301+0000","last_became_active":"2024-04-24T23:04:56.295135+0000","last_became_peered":"2024-04-24T23:04:56.295135+0000","last_unstale":"2024-04-24T23:04:56.296301+0000","last_undegraded":"2024-04-24T23:04:56.296301+0000","last_fullsized":"2024-04-24T23:04:56.296301+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:49:48.073966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"124'2","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.633003+0000","last_change":"2024-04-24T23:04:55.635279+0000","last_active":"2024-04-24T23:04:56.633003+0000","last_peered":"2024-04-24T23:04:56.633003+0000","last_clean":"2024-04-24T23:04:56.633003+0000","last_became_active":"2024-04-24T23:04:55.635018+0000","last_became_peered":"2024-04-24T23:04:55.635018+0000","last_unstale":"2024-04-24T23:04:56.633003+0000","last_undegraded":"2024-04-24T23:04:56.633003+0000","last_fullsized":"2024-04-24T23:04:56.633003+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:59.921601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.306061+0000","last_change":"2024-04-24T23:04:56.305946+0000","last_active":"2024-04-24T23:04:56.306061+0000","last_peered":"2024-04-24T23:04:56.306061+0000","last_clean":"2024-04-24T23:04:56.306061+0000","last_became_active":"2024-04-24T23:04:56.305580+0000","last_became_peered":"2024-04-24T23:04:56.305580+0000","last_unstale":"2024-04-24T23:04:56.306061+0000","last_undegraded":"2024-04-24T23:04:56.306061+0000","last_fullsized":"2024-04-24T23:04:56.306061+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:22.488951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.294816+0000","last_change":"2024-04-24T23:04:55.638955+0000","last_active":"2024-04-24T23:04:56.294816+0000","last_peered":"2024-04-24T23:04:56.294816+0000","last_clean":"2024-04-24T23:04:56.294816+0000","last_became_active":"2024-04-24T23:04:55.636523+0000","last_became_peered":"2024-04-24T23:04:55.636523+0000","last_unstale":"2024-04-24T23:04:56.294816+0000","last_undegraded":"2024-04-24T23:04:56.294816+0000","last_fullsized":"2024-04-24T23:04:56.294816+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:53:47.040506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.300011+0000","last_change":"2024-04-24T23:04:56.299109+0000","last_active":"2024-04-24T23:04:56.300011+0000","last_peered":"2024-04-24T23:04:56.300011+0000","last_clean":"2024-04-24T23:04:56.300011+0000","last_became_active":"2024-04-24T23:04:56.298886+0000","last_became_peered":"2024-04-24T23:04:56.298886+0000","last_unstale":"2024-04-24T23:04:56.300011+0000","last_undegraded":"2024-04-24T23:04:56.300011+0000","last_fullsized":"2024-04-24T23:04:56.300011+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:12:12.807753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.294643+0000","last_change":"2024-04-24T23:04:55.636814+0000","last_active":"2024-04-24T23:04:56.294643+0000","last_peered":"2024-04-24T23:04:56.294643+0000","last_clean":"2024-04-24T23:04:56.294643+0000","last_became_active":"2024-04-24T23:04:55.636564+0000","last_became_peered":"2024-04-24T23:04:55.636564+0000","last_unstale":"2024-04-24T23:04:56.294643+0000","last_undegraded":"2024-04-24T23:04:56.294643+0000","last_fullsized":"2024-04-24T23:04:56.294643+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:57:27.084042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.301735+0000","last_change":"2024-04-24T23:04:56.301592+0000","last_active":"2024-04-24T23:04:56.301735+0000","last_peered":"2024-04-24T23:04:56.301735+0000","last_clean":"2024-04-24T23:04:56.301735+0000","last_became_active":"2024-04-24T23:04:56.301316+0000","last_became_peered":"2024-04-24T23:04:56.301316+0000","last_unstale":"2024-04-24T23:04:56.301735+0000","last_undegraded":"2024-04-24T23:04:56.301735+0000","last_fullsized":"2024-04-24T23:04:56.301735+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:07:48.640162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"124'8","reported_seq":20,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.635268+0000","last_change":"2024-04-24T23:04:55.640903+0000","last_active":"2024-04-24T23:04:56.635268+0000","last_peered":"2024-04-24T23:04:56.635268+0000","last_clean":"2024-04-24T23:04:56.635268+0000","last_became_active":"2024-04-24T23:04:55.635883+0000","last_became_peered":"2024-04-24T23:04:55.635883+0000","last_unstale":"2024-04-24T23:04:56.635268+0000","last_undegraded":"2024-04-24T23:04:56.635268+0000","last_fullsized":"2024-04-24T23:04:56.635268+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:30.760629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.295372+0000","last_change":"2024-04-24T23:04:56.295214+0000","last_active":"2024-04-24T23:04:56.295372+0000","last_peered":"2024-04-24T23:04:56.295372+0000","last_clean":"2024-04-24T23:04:56.295372+0000","last_became_active":"2024-04-24T23:04:56.294977+0000","last_became_peered":"2024-04-24T23:04:56.294977+0000","last_unstale":"2024-04-24T23:04:56.295372+0000","last_undegraded":"2024-04-24T23:04:56.295372+0000","last_fullsized":"2024-04-24T23:04:56.295372+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:15.499923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"124'2","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.632480+0000","last_change":"2024-04-24T23:04:55.638319+0000","last_active":"2024-04-24T23:04:56.632480+0000","last_peered":"2024-04-24T23:04:56.632480+0000","last_clean":"2024-04-24T23:04:56.632480+0000","last_became_active":"2024-04-24T23:04:55.638027+0000","last_became_peered":"2024-04-24T23:04:55.638027+0000","last_unstale":"2024-04-24T23:04:56.632480+0000","last_undegraded":"2024-04-24T23:04:56.632480+0000","last_fullsized":"2024-04-24T23:04:56.632480+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:24:10.334282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.299099+0000","last_change":"2024-04-24T23:04:56.298844+0000","last_active":"2024-04-24T23:04:56.299099+0000","last_peered":"2024-04-24T23:04:56.299099+0000","last_clean":"2024-04-24T23:04:56.299099+0000","last_became_active":"2024-04-24T23:04:56.297635+0000","last_became_peered":"2024-04-24T23:04:56.297635+0000","last_unstale":"2024-04-24T23:04:56.299099+0000","last_undegraded":"2024-04-24T23:04:56.299099+0000","last_fullsized":"2024-04-24T23:04:56.299099+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:56:12.264690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"124'1","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.632026+0000","last_change":"2024-04-24T23:04:55.634143+0000","last_active":"2024-04-24T23:04:56.632026+0000","last_peered":"2024-04-24T23:04:56.632026+0000","last_clean":"2024-04-24T23:04:56.632026+0000","last_became_active":"2024-04-24T23:04:55.633714+0000","last_became_peered":"2024-04-24T23:04:55.633714+0000","last_unstale":"2024-04-24T23:04:56.632026+0000","last_undegraded":"2024-04-24T23:04:56.632026+0000","last_fullsized":"2024-04-24T23:04:56.632026+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:37:02.355090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.303540+0000","last_change":"2024-04-24T23:04:56.303436+0000","last_active":"2024-04-24T23:04:56.303540+0000","last_peered":"2024-04-24T23:04:56.303540+0000","last_clean":"2024-04-24T23:04:56.303540+0000","last_became_active":"2024-04-24T23:04:56.303256+0000","last_became_peered":"2024-04-24T23:04:56.303256+0000","last_unstale":"2024-04-24T23:04:56.303540+0000","last_undegraded":"2024-04-24T23:04:56.303540+0000","last_fullsized":"2024-04-24T23:04:56.303540+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:24:55.532403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.291870+0000","last_change":"2024-04-24T23:04:55.638108+0000","last_active":"2024-04-24T23:04:56.291870+0000","last_peered":"2024-04-24T23:04:56.291870+0000","last_clean":"2024-04-24T23:04:56.291870+0000","last_became_active":"2024-04-24T23:04:55.637880+0000","last_became_peered":"2024-04-24T23:04:55.637880+0000","last_unstale":"2024-04-24T23:04:56.291870+0000","last_undegraded":"2024-04-24T23:04:56.291870+0000","last_fullsized":"2024-04-24T23:04:56.291870+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:08.229039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.304745+0000","last_change":"2024-04-24T23:04:56.302906+0000","last_active":"2024-04-24T23:04:56.304745+0000","last_peered":"2024-04-24T23:04:56.304745+0000","last_clean":"2024-04-24T23:04:56.304745+0000","last_became_active":"2024-04-24T23:04:56.302662+0000","last_became_peered":"2024-04-24T23:04:56.302662+0000","last_unstale":"2024-04-24T23:04:56.304745+0000","last_undegraded":"2024-04-24T23:04:56.304745+0000","last_fullsized":"2024-04-24T23:04:56.304745+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:32.738827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"124'2","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.635841+0000","last_change":"2024-04-24T23:04:55.641264+0000","last_active":"2024-04-24T23:04:56.635841+0000","last_peered":"2024-04-24T23:04:56.635841+0000","last_clean":"2024-04-24T23:04:56.635841+0000","last_became_active":"2024-04-24T23:04:55.639584+0000","last_became_peered":"2024-04-24T23:04:55.639584+0000","last_unstale":"2024-04-24T23:04:56.635841+0000","last_undegraded":"2024-04-24T23:04:56.635841+0000","last_fullsized":"2024-04-24T23:04:56.635841+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:10:21.463856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.300836+0000","last_change":"2024-04-24T23:04:56.299539+0000","last_active":"2024-04-24T23:04:56.300836+0000","last_peered":"2024-04-24T23:04:56.300836+0000","last_clean":"2024-04-24T23:04:56.300836+0000","last_became_active":"2024-04-24T23:04:56.298309+0000","last_became_peered":"2024-04-24T23:04:56.298309+0000","last_unstale":"2024-04-24T23:04:56.300836+0000","last_undegraded":"2024-04-24T23:04:56.300836+0000","last_fullsized":"2024-04-24T23:04:56.300836+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:48:25.659158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.289644+0000","last_change":"2024-04-24T23:04:55.634554+0000","last_active":"2024-04-24T23:04:56.289644+0000","last_peered":"2024-04-24T23:04:56.289644+0000","last_clean":"2024-04-24T23:04:56.289644+0000","last_became_active":"2024-04-24T23:04:55.633943+0000","last_became_peered":"2024-04-24T23:04:55.633943+0000","last_unstale":"2024-04-24T23:04:56.289644+0000","last_undegraded":"2024-04-24T23:04:56.289644+0000","last_fullsized":"2024-04-24T23:04:56.289644+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:04:06.826836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.299875+0000","last_change":"2024-04-24T23:04:56.299592+0000","last_active":"2024-04-24T23:04:56.299875+0000","last_peered":"2024-04-24T23:04:56.299875+0000","last_clean":"2024-04-24T23:04:56.299875+0000","last_became_active":"2024-04-24T23:04:56.297889+0000","last_became_peered":"2024-04-24T23:04:56.297889+0000","last_unstale":"2024-04-24T23:04:56.299875+0000","last_undegraded":"2024-04-24T23:04:56.299875+0000","last_fullsized":"2024-04-24T23:04:56.299875+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:41:11.995488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.294048+0000","last_change":"2024-04-24T23:04:55.634641+0000","last_active":"2024-04-24T23:04:56.294048+0000","last_peered":"2024-04-24T23:04:56.294048+0000","last_clean":"2024-04-24T23:04:56.294048+0000","last_became_active":"2024-04-24T23:04:55.633996+0000","last_became_peered":"2024-04-24T23:04:55.633996+0000","last_unstale":"2024-04-24T23:04:56.294048+0000","last_undegraded":"2024-04-24T23:04:56.294048+0000","last_fullsized":"2024-04-24T23:04:56.294048+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:57:15.362988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.302888+0000","last_change":"2024-04-24T23:04:56.302804+0000","last_active":"2024-04-24T23:04:56.302888+0000","last_peered":"2024-04-24T23:04:56.302888+0000","last_clean":"2024-04-24T23:04:56.302888+0000","last_became_active":"2024-04-24T23:04:56.302608+0000","last_became_peered":"2024-04-24T23:04:56.302608+0000","last_unstale":"2024-04-24T23:04:56.302888+0000","last_undegraded":"2024-04-24T23:04:56.302888+0000","last_fullsized":"2024-04-24T23:04:56.302888+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:59:19.113955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.300760+0000","last_change":"2024-04-24T23:04:55.640758+0000","last_active":"2024-04-24T23:04:56.300760+0000","last_peered":"2024-04-24T23:04:56.300760+0000","last_clean":"2024-04-24T23:04:56.300760+0000","last_became_active":"2024-04-24T23:04:55.640334+0000","last_became_peered":"2024-04-24T23:04:55.640334+0000","last_unstale":"2024-04-24T23:04:56.300760+0000","last_undegraded":"2024-04-24T23:04:56.300760+0000","last_fullsized":"2024-04-24T23:04:56.300760+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:14:05.309948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.297787+0000","last_change":"2024-04-24T23:04:56.294014+0000","last_active":"2024-04-24T23:04:56.297787+0000","last_peered":"2024-04-24T23:04:56.297787+0000","last_clean":"2024-04-24T23:04:56.297787+0000","last_became_active":"2024-04-24T23:04:56.293757+0000","last_became_peered":"2024-04-24T23:04:56.293757+0000","last_unstale":"2024-04-24T23:04:56.297787+0000","last_undegraded":"2024-04-24T23:04:56.297787+0000","last_fullsized":"2024-04-24T23:04:56.297787+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:04:01.666435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"124'2","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.632382+0000","last_change":"2024-04-24T23:04:55.636245+0000","last_active":"2024-04-24T23:04:56.632382+0000","last_peered":"2024-04-24T23:04:56.632382+0000","last_clean":"2024-04-24T23:04:56.632382+0000","last_became_active":"2024-04-24T23:04:55.635890+0000","last_became_peered":"2024-04-24T23:04:55.635890+0000","last_unstale":"2024-04-24T23:04:56.632382+0000","last_undegraded":"2024-04-24T23:04:56.632382+0000","last_fullsized":"2024-04-24T23:04:56.632382+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:42:07.230037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.298308+0000","last_change":"2024-04-24T23:04:56.296966+0000","last_active":"2024-04-24T23:04:56.298308+0000","last_peered":"2024-04-24T23:04:56.298308+0000","last_clean":"2024-04-24T23:04:56.298308+0000","last_became_active":"2024-04-24T23:04:56.296204+0000","last_became_peered":"2024-04-24T23:04:56.296204+0000","last_unstale":"2024-04-24T23:04:56.298308+0000","last_undegraded":"2024-04-24T23:04:56.298308+0000","last_fullsized":"2024-04-24T23:04:56.298308+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:23:39.368377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.291492+0000","last_change":"2024-04-24T23:04:55.640732+0000","last_active":"2024-04-24T23:04:56.291492+0000","last_peered":"2024-04-24T23:04:56.291492+0000","last_clean":"2024-04-24T23:04:56.291492+0000","last_became_active":"2024-04-24T23:04:55.640351+0000","last_became_peered":"2024-04-24T23:04:55.640351+0000","last_unstale":"2024-04-24T23:04:56.291492+0000","last_undegraded":"2024-04-24T23:04:56.291492+0000","last_fullsized":"2024-04-24T23:04:56.291492+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:36:12.441787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.304885+0000","last_change":"2024-04-24T23:04:56.304731+0000","last_active":"2024-04-24T23:04:56.304885+0000","last_peered":"2024-04-24T23:04:56.304885+0000","last_clean":"2024-04-24T23:04:56.304885+0000","last_became_active":"2024-04-24T23:04:56.303061+0000","last_became_peered":"2024-04-24T23:04:56.303061+0000","last_unstale":"2024-04-24T23:04:56.304885+0000","last_undegraded":"2024-04-24T23:04:56.304885+0000","last_fullsized":"2024-04-24T23:04:56.304885+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:57.785936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.293944+0000","last_change":"2024-04-24T23:04:55.636039+0000","last_active":"2024-04-24T23:04:56.293944+0000","last_peered":"2024-04-24T23:04:56.293944+0000","last_clean":"2024-04-24T23:04:56.293944+0000","last_became_active":"2024-04-24T23:04:55.635633+0000","last_became_peered":"2024-04-24T23:04:55.635633+0000","last_unstale":"2024-04-24T23:04:56.293944+0000","last_undegraded":"2024-04-24T23:04:56.293944+0000","last_fullsized":"2024-04-24T23:04:56.293944+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:29:19.736267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.295777+0000","last_change":"2024-04-24T23:04:56.295661+0000","last_active":"2024-04-24T23:04:56.295777+0000","last_peered":"2024-04-24T23:04:56.295777+0000","last_clean":"2024-04-24T23:04:56.295777+0000","last_became_active":"2024-04-24T23:04:56.294758+0000","last_became_peered":"2024-04-24T23:04:56.294758+0000","last_unstale":"2024-04-24T23:04:56.295777+0000","last_undegraded":"2024-04-24T23:04:56.295777+0000","last_fullsized":"2024-04-24T23:04:56.295777+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:55.860923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.289365+0000","last_change":"2024-04-24T23:04:55.632726+0000","last_active":"2024-04-24T23:04:56.289365+0000","last_peered":"2024-04-24T23:04:56.289365+0000","last_clean":"2024-04-24T23:04:56.289365+0000","last_became_active":"2024-04-24T23:04:55.632469+0000","last_became_peered":"2024-04-24T23:04:55.632469+0000","last_unstale":"2024-04-24T23:04:56.289365+0000","last_undegraded":"2024-04-24T23:04:56.289365+0000","last_fullsized":"2024-04-24T23:04:56.289365+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:33:40.335636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.298286+0000","last_change":"2024-04-24T23:04:56.296450+0000","last_active":"2024-04-24T23:04:56.298286+0000","last_peered":"2024-04-24T23:04:56.298286+0000","last_clean":"2024-04-24T23:04:56.298286+0000","last_became_active":"2024-04-24T23:04:56.295739+0000","last_became_peered":"2024-04-24T23:04:56.295739+0000","last_unstale":"2024-04-24T23:04:56.298286+0000","last_undegraded":"2024-04-24T23:04:56.298286+0000","last_fullsized":"2024-04-24T23:04:56.298286+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:51:07.700024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.300943+0000","last_change":"2024-04-24T23:04:55.641993+0000","last_active":"2024-04-24T23:04:56.300943+0000","last_peered":"2024-04-24T23:04:56.300943+0000","last_clean":"2024-04-24T23:04:56.300943+0000","last_became_active":"2024-04-24T23:04:55.641729+0000","last_became_peered":"2024-04-24T23:04:55.641729+0000","last_unstale":"2024-04-24T23:04:56.300943+0000","last_undegraded":"2024-04-24T23:04:56.300943+0000","last_fullsized":"2024-04-24T23:04:56.300943+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:14:55.485408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.301726+0000","last_change":"2024-04-24T23:04:56.301585+0000","last_active":"2024-04-24T23:04:56.301726+0000","last_peered":"2024-04-24T23:04:56.301726+0000","last_clean":"2024-04-24T23:04:56.301726+0000","last_became_active":"2024-04-24T23:04:56.301376+0000","last_became_peered":"2024-04-24T23:04:56.301376+0000","last_unstale":"2024-04-24T23:04:56.301726+0000","last_undegraded":"2024-04-24T23:04:56.301726+0000","last_fullsized":"2024-04-24T23:04:56.301726+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:40:25.671360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.289663+0000","last_change":"2024-04-24T23:04:55.634810+0000","last_active":"2024-04-24T23:04:56.289663+0000","last_peered":"2024-04-24T23:04:56.289663+0000","last_clean":"2024-04-24T23:04:56.289663+0000","last_became_active":"2024-04-24T23:04:55.634556+0000","last_became_peered":"2024-04-24T23:04:55.634556+0000","last_unstale":"2024-04-24T23:04:56.289663+0000","last_undegraded":"2024-04-24T23:04:56.289663+0000","last_fullsized":"2024-04-24T23:04:56.289663+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:12.941676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.297829+0000","last_change":"2024-04-24T23:04:56.295255+0000","last_active":"2024-04-24T23:04:56.297829+0000","last_peered":"2024-04-24T23:04:56.297829+0000","last_clean":"2024-04-24T23:04:56.297829+0000","last_became_active":"2024-04-24T23:04:56.295021+0000","last_became_peered":"2024-04-24T23:04:56.295021+0000","last_unstale":"2024-04-24T23:04:56.297829+0000","last_undegraded":"2024-04-24T23:04:56.297829+0000","last_fullsized":"2024-04-24T23:04:56.297829+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:00.086790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.297648+0000","last_change":"2024-04-24T23:04:55.639514+0000","last_active":"2024-04-24T23:04:56.297648+0000","last_peered":"2024-04-24T23:04:56.297648+0000","last_clean":"2024-04-24T23:04:56.297648+0000","last_became_active":"2024-04-24T23:04:55.638828+0000","last_became_peered":"2024-04-24T23:04:55.638828+0000","last_unstale":"2024-04-24T23:04:56.297648+0000","last_undegraded":"2024-04-24T23:04:56.297648+0000","last_fullsized":"2024-04-24T23:04:56.297648+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:49.020530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.300825+0000","last_change":"2024-04-24T23:04:56.300721+0000","last_active":"2024-04-24T23:04:56.300825+0000","last_peered":"2024-04-24T23:04:56.300825+0000","last_clean":"2024-04-24T23:04:56.300825+0000","last_became_active":"2024-04-24T23:04:56.300034+0000","last_became_peered":"2024-04-24T23:04:56.300034+0000","last_unstale":"2024-04-24T23:04:56.300825+0000","last_undegraded":"2024-04-24T23:04:56.300825+0000","last_fullsized":"2024-04-24T23:04:56.300825+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:26:08.592050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.296429+0000","last_change":"2024-04-24T23:04:55.640513+0000","last_active":"2024-04-24T23:04:56.296429+0000","last_peered":"2024-04-24T23:04:56.296429+0000","last_clean":"2024-04-24T23:04:56.296429+0000","last_became_active":"2024-04-24T23:04:55.640243+0000","last_became_peered":"2024-04-24T23:04:55.640243+0000","last_unstale":"2024-04-24T23:04:56.296429+0000","last_undegraded":"2024-04-24T23:04:56.296429+0000","last_fullsized":"2024-04-24T23:04:56.296429+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:21:49.757962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.303597+0000","last_change":"2024-04-24T23:04:56.303332+0000","last_active":"2024-04-24T23:04:56.303597+0000","last_peered":"2024-04-24T23:04:56.303597+0000","last_clean":"2024-04-24T23:04:56.303597+0000","last_became_active":"2024-04-24T23:04:56.302704+0000","last_became_peered":"2024-04-24T23:04:56.302704+0000","last_unstale":"2024-04-24T23:04:56.303597+0000","last_undegraded":"2024-04-24T23:04:56.303597+0000","last_fullsized":"2024-04-24T23:04:56.303597+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:03:33.032633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"124'1","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.633214+0000","last_change":"2024-04-24T23:04:55.632584+0000","last_active":"2024-04-24T23:04:56.633214+0000","last_peered":"2024-04-24T23:04:56.633214+0000","last_clean":"2024-04-24T23:04:56.633214+0000","last_became_active":"2024-04-24T23:04:55.632285+0000","last_became_peered":"2024-04-24T23:04:55.632285+0000","last_unstale":"2024-04-24T23:04:56.633214+0000","last_undegraded":"2024-04-24T23:04:56.633214+0000","last_fullsized":"2024-04-24T23:04:56.633214+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:23:24.718858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":375,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.294277+0000","last_change":"2024-04-24T22:58:55.138810+0000","last_active":"2024-04-24T23:04:56.294277+0000","last_peered":"2024-04-24T23:04:56.294277+0000","last_clean":"2024-04-24T23:04:56.294277+0000","last_became_active":"2024-04-24T22:58:55.138627+0000","last_became_peered":"2024-04-24T22:58:55.138627+0000","last_unstale":"2024-04-24T23:04:56.294277+0000","last_undegraded":"2024-04-24T23:04:56.294277+0000","last_fullsized":"2024-04-24T23:04:56.294277+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_clean_scrub_stamp":"2024-04-24T22:58:54.127235+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:51.383182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574923,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":662,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194650,"statfs":{"total":200000000,"available":199321942,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574923,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":668,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194643,"statfs":{"total":200000000,"available":199315167,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574923,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1246,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194065,"statfs":{"total":200000000,"available":198723131,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574923,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":668,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194643,"statfs":{"total":200000000,"available":199315167,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574923,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":668,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194643,"statfs":{"total":200000000,"available":199314973,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574923,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":668,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194643,"statfs":{"total":200000000,"available":199315097,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1238,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194073,"statfs":{"total":200000000,"available":198731574,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574923,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":668,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194643,"statfs":{"total":200000000,"available":199315167,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-24T23:05:00.297 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:05:00.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.432+0000 7f6c86142640 1 Processor -- start 2024-04-24T23:05:00.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.432+0000 7f6c86142640 1 -- start start 2024-04-24T23:05:00.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.433+0000 7f6c86142640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6c8015e9b0 0x7f6c800d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:00.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.433+0000 7f6c86142640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6c800d4210 0x7f6c8006b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:00.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.433+0000 7f6c86142640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c800d3170 con 0x7f6c800ba0e0 2024-04-24T23:05:00.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.433+0000 7f6c86142640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f6c800b8900 con 0x7f6c800d4210 2024-04-24T23:05:00.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.433+0000 7f6c86142640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f6c800d3450 con 0x7f6c8015e9b0 2024-04-24T23:05:00.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.433+0000 7f6c7effd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6c800d4210 0x7f6c8006b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:00.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.433+0000 7f6c7effd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6c800d4210 0x7f6c8006b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52598/0 (socket says 172.21.15.32:52598) 2024-04-24T23:05:00.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.433+0000 7f6c7effd640 1 -- 172.21.15.32:0/1995792882 learned_addr learned my addr 172.21.15.32:0/1995792882 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:00.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.434+0000 7f6c7f7fe640 1 --2- 172.21.15.32:0/1995792882 >> v2:172.21.15.121:3300/0 conn(0x7f6c8015e9b0 0x7f6c800d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:00.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.434+0000 7f6c7effd640 1 -- 172.21.15.32:0/1995792882 >> v2:172.21.15.121:3300/0 conn(0x7f6c8015e9b0 msgr2=0x7f6c800d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:00.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.434+0000 7f6c7effd640 1 --2- 172.21.15.32:0/1995792882 >> v2:172.21.15.121:3300/0 conn(0x7f6c8015e9b0 0x7f6c800d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:00.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.434+0000 7f6c7effd640 1 -- 172.21.15.32:0/1995792882 >> v1:172.21.15.121:6789/0 conn(0x7f6c800ba0e0 legacy=0x7f6c80071730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:00.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.434+0000 7f6c7effd640 1 -- 172.21.15.32:0/1995792882 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c800d3730 con 0x7f6c800d4210 2024-04-24T23:05:00.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.434+0000 7f6c7effd640 1 --2- 172.21.15.32:0/1995792882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c800d4210 0x7f6c8006b110 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f6c8006be20 tx=0x7f6c7402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0c17880989a5fbc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:00.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.435+0000 7f6c7e7fc640 1 -- 172.21.15.32:0/1995792882 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c74037070 con 0x7f6c800d4210 2024-04-24T23:05:00.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.435+0000 7f6c7e7fc640 1 -- 172.21.15.32:0/1995792882 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c74032440 con 0x7f6c800d4210 2024-04-24T23:05:00.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.435+0000 7f6c7e7fc640 1 -- 172.21.15.32:0/1995792882 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c74032810 con 0x7f6c800d4210 2024-04-24T23:05:00.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.435+0000 7f6c86142640 1 -- 172.21.15.32:0/1995792882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c800d4210 msgr2=0x7f6c8006b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:00.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.435+0000 7f6c86142640 1 --2- 172.21.15.32:0/1995792882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c800d4210 0x7f6c8006b110 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f6c8006be20 tx=0x7f6c7402a1e0 comp rx=0 tx=0).stop 2024-04-24T23:05:00.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.435+0000 7f6c86142640 1 -- 172.21.15.32:0/1995792882 shutdown_connections 2024-04-24T23:05:00.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.435+0000 7f6c86142640 1 --2- 172.21.15.32:0/1995792882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c800d4210 0x7f6c8006b110 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:00.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.435+0000 7f6c86142640 1 --2- 172.21.15.32:0/1995792882 >> v2:172.21.15.121:3300/0 conn(0x7f6c8015e9b0 0x7f6c800d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:00.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.435+0000 7f6c86142640 1 -- 172.21.15.32:0/1995792882 >> 172.21.15.32:0/1995792882 conn(0x7f6c800d11d0 msgr2=0x7f6c800d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:00.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.435+0000 7f6c86142640 1 -- 172.21.15.32:0/1995792882 shutdown_connections 2024-04-24T23:05:00.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.435+0000 7f6c86142640 1 -- 172.21.15.32:0/1995792882 wait complete. 2024-04-24T23:05:00.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.436+0000 7f6c86142640 1 Processor -- start 2024-04-24T23:05:00.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.437+0000 7f6c86142640 1 -- start start 2024-04-24T23:05:00.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.437+0000 7f6c86142640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c800ba0e0 0x7f6c80161670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:00.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.437+0000 7f6c86142640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c800d4210 0x7f6c800ca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:00.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.437+0000 7f6c86142640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6c8015e9b0 0x7f6c8015bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:00.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.437+0000 7f6c86142640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6c800d2400 con 0x7f6c800d4210 2024-04-24T23:05:00.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.437+0000 7f6c86142640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6c800d3cf0 con 0x7f6c800ba0e0 2024-04-24T23:05:00.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.437+0000 7f6c86142640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f6c8006b650 con 0x7f6c8015e9b0 2024-04-24T23:05:00.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.438+0000 7f6c7effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c800d4210 0x7f6c800ca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:00.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.438+0000 7f6c7effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c800d4210 0x7f6c800ca9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52600/0 (socket says 172.21.15.32:52600) 2024-04-24T23:05:00.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.438+0000 7f6c7effd640 1 -- 172.21.15.32:0/2323815066 learned_addr learned my addr 172.21.15.32:0/2323815066 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:00.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.438+0000 7f6c7f7fe640 1 --2- 172.21.15.32:0/2323815066 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c800ba0e0 0x7f6c80161670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:00.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.438+0000 7f6c7ffff640 1 --2- 172.21.15.32:0/2323815066 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6c8015e9b0 0x7f6c8015bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:00.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.438+0000 7f6c7effd640 1 -- 172.21.15.32:0/2323815066 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6c8015e9b0 msgr2=0x7f6c8015bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:00.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.438+0000 7f6c7effd640 1 --2- 172.21.15.32:0/2323815066 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6c8015e9b0 0x7f6c8015bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:00.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.438+0000 7f6c7effd640 1 -- 172.21.15.32:0/2323815066 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c800ba0e0 msgr2=0x7f6c80161670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:00.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.438+0000 7f6c7effd640 1 --2- 172.21.15.32:0/2323815066 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c800ba0e0 0x7f6c80161670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:00.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.438+0000 7f6c7effd640 1 -- 172.21.15.32:0/2323815066 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c800cbc00 con 0x7f6c800d4210 2024-04-24T23:05:00.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.438+0000 7f6c7effd640 1 --2- 172.21.15.32:0/2323815066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c800d4210 0x7f6c800ca9f0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f6c7402a220 tx=0x7f6c740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:00.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.438+0000 7f6c67fff640 1 -- 172.21.15.32:0/2323815066 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c74041020 con 0x7f6c800d4210 2024-04-24T23:05:00.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.438+0000 7f6c67fff640 1 -- 172.21.15.32:0/2323815066 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c740306c0 con 0x7f6c800d4210 2024-04-24T23:05:00.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.438+0000 7f6c67fff640 1 -- 172.21.15.32:0/2323815066 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c74037040 con 0x7f6c800d4210 2024-04-24T23:05:00.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.438+0000 7f6c86142640 1 -- 172.21.15.32:0/2323815066 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c800cbee0 con 0x7f6c800d4210 2024-04-24T23:05:00.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.438+0000 7f6c86142640 1 -- 172.21.15.32:0/2323815066 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c800cc4a0 con 0x7f6c800d4210 2024-04-24T23:05:00.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.439+0000 7f6c86142640 1 -- 172.21.15.32:0/2323815066 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c80068430 con 0x7f6c800d4210 2024-04-24T23:05:00.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.440+0000 7f6c67fff640 1 -- 172.21.15.32:0/2323815066 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6c74044020 con 0x7f6c800d4210 2024-04-24T23:05:00.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.440+0000 7f6c67fff640 1 --2- 172.21.15.32:0/2323815066 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6c6c06be30 0x7f6c6c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:00.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.440+0000 7f6c67fff640 1 -- 172.21.15.32:0/2323815066 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5709+0+0 (secure 0 0 0) 0x7f6c740ab560 con 0x7f6c800d4210 2024-04-24T23:05:00.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.441+0000 7f6c7f7fe640 1 --2- 172.21.15.32:0/2323815066 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6c6c06be30 0x7f6c6c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:00.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.441+0000 7f6c7f7fe640 1 --2- 172.21.15.32:0/2323815066 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6c6c06be30 0x7f6c6c06e2f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f6c80152bb0 tx=0x7f6c70008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:00.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.445+0000 7f6c67fff640 1 -- 172.21.15.32:0/2323815066 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6c7404b030 con 0x7f6c800d4210 2024-04-24T23:05:00.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.623+0000 7f6c86142640 1 -- 172.21.15.32:0/2323815066 --> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6c8015d150 con 0x7f6c6c06be30 2024-04-24T23:05:00.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.629+0000 7f6c67fff640 1 -- 172.21.15.32:0/2323815066 <== mgr.4108 v2:172.21.15.121:6832/1848157943 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247923 (secure 0 0 0) 0x7f6c8015d150 con 0x7f6c6c06be30 2024-04-24T23:05:00.633 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:05:00.634 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-24T23:05:00.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.632+0000 7f6c86142640 1 -- 172.21.15.32:0/2323815066 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6c6c06be30 msgr2=0x7f6c6c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:00.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.632+0000 7f6c86142640 1 --2- 172.21.15.32:0/2323815066 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6c6c06be30 0x7f6c6c06e2f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f6c80152bb0 tx=0x7f6c70008040 comp rx=0 tx=0).stop 2024-04-24T23:05:00.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.632+0000 7f6c86142640 1 -- 172.21.15.32:0/2323815066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c800d4210 msgr2=0x7f6c800ca9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:00.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.632+0000 7f6c86142640 1 --2- 172.21.15.32:0/2323815066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c800d4210 0x7f6c800ca9f0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f6c7402a220 tx=0x7f6c740046c0 comp rx=0 tx=0).stop 2024-04-24T23:05:00.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.632+0000 7f6c86142640 1 -- 172.21.15.32:0/2323815066 shutdown_connections 2024-04-24T23:05:00.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.632+0000 7f6c86142640 1 --2- 172.21.15.32:0/2323815066 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6c6c06be30 0x7f6c6c06e2f0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:00.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.632+0000 7f6c86142640 1 --2- 172.21.15.32:0/2323815066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c800d4210 0x7f6c800ca9f0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:00.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.632+0000 7f6c86142640 1 --2- 172.21.15.32:0/2323815066 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6c8015e9b0 0x7f6c8015bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:00.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.632+0000 7f6c86142640 1 --2- 172.21.15.32:0/2323815066 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c800ba0e0 0x7f6c80161670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:00.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.632+0000 7f6c86142640 1 -- 172.21.15.32:0/2323815066 >> 172.21.15.32:0/2323815066 conn(0x7f6c800d11d0 msgr2=0x7f6c800cfa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:00.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.632+0000 7f6c86142640 1 -- 172.21.15.32:0/2323815066 shutdown_connections 2024-04-24T23:05:00.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.632+0000 7f6c86142640 1 -- 172.21.15.32:0/2323815066 wait complete. 2024-04-24T23:05:00.644 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":300,"stamp":"2024-04-24T23:04:59.209736+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":107,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6491,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1556008,"statfs":{"total":1600000000,"available":1593352218,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.594290"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.299417+0000","last_change":"2024-04-24T23:04:56.299069+0000","last_active":"2024-04-24T23:04:56.299417+0000","last_peered":"2024-04-24T23:04:56.299417+0000","last_clean":"2024-04-24T23:04:56.299417+0000","last_became_active":"2024-04-24T23:04:56.296532+0000","last_became_peered":"2024-04-24T23:04:56.296532+0000","last_unstale":"2024-04-24T23:04:56.299417+0000","last_undegraded":"2024-04-24T23:04:56.299417+0000","last_fullsized":"2024-04-24T23:04:56.299417+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:07:58.343128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.298738+0000","last_change":"2024-04-24T23:04:56.298610+0000","last_active":"2024-04-24T23:04:56.298738+0000","last_peered":"2024-04-24T23:04:56.298738+0000","last_clean":"2024-04-24T23:04:56.298738+0000","last_became_active":"2024-04-24T23:04:56.298063+0000","last_became_peered":"2024-04-24T23:04:56.298063+0000","last_unstale":"2024-04-24T23:04:56.298738+0000","last_undegraded":"2024-04-24T23:04:56.298738+0000","last_fullsized":"2024-04-24T23:04:56.298738+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:29:13.057885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.301954+0000","last_change":"2024-04-24T23:04:56.301838+0000","last_active":"2024-04-24T23:04:56.301954+0000","last_peered":"2024-04-24T23:04:56.301954+0000","last_clean":"2024-04-24T23:04:56.301954+0000","last_became_active":"2024-04-24T23:04:56.301608+0000","last_became_peered":"2024-04-24T23:04:56.301608+0000","last_unstale":"2024-04-24T23:04:56.301954+0000","last_undegraded":"2024-04-24T23:04:56.301954+0000","last_fullsized":"2024-04-24T23:04:56.301954+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:57.464174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.299897+0000","last_change":"2024-04-24T23:04:56.299648+0000","last_active":"2024-04-24T23:04:56.299897+0000","last_peered":"2024-04-24T23:04:56.299897+0000","last_clean":"2024-04-24T23:04:56.299897+0000","last_became_active":"2024-04-24T23:04:56.299337+0000","last_became_peered":"2024-04-24T23:04:56.299337+0000","last_unstale":"2024-04-24T23:04:56.299897+0000","last_undegraded":"2024-04-24T23:04:56.299897+0000","last_fullsized":"2024-04-24T23:04:56.299897+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:51.404191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.296489+0000","last_change":"2024-04-24T23:04:56.296306+0000","last_active":"2024-04-24T23:04:56.296489+0000","last_peered":"2024-04-24T23:04:56.296489+0000","last_clean":"2024-04-24T23:04:56.296489+0000","last_became_active":"2024-04-24T23:04:56.296021+0000","last_became_peered":"2024-04-24T23:04:56.296021+0000","last_unstale":"2024-04-24T23:04:56.296489+0000","last_undegraded":"2024-04-24T23:04:56.296489+0000","last_fullsized":"2024-04-24T23:04:56.296489+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:20:31.166345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.299993+0000","last_change":"2024-04-24T23:04:56.298933+0000","last_active":"2024-04-24T23:04:56.299993+0000","last_peered":"2024-04-24T23:04:56.299993+0000","last_clean":"2024-04-24T23:04:56.299993+0000","last_became_active":"2024-04-24T23:04:56.298480+0000","last_became_peered":"2024-04-24T23:04:56.298480+0000","last_unstale":"2024-04-24T23:04:56.299993+0000","last_undegraded":"2024-04-24T23:04:56.299993+0000","last_fullsized":"2024-04-24T23:04:56.299993+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:40:22.228542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.295471+0000","last_change":"2024-04-24T23:04:56.294829+0000","last_active":"2024-04-24T23:04:56.295471+0000","last_peered":"2024-04-24T23:04:56.295471+0000","last_clean":"2024-04-24T23:04:56.295471+0000","last_became_active":"2024-04-24T23:04:56.294572+0000","last_became_peered":"2024-04-24T23:04:56.294572+0000","last_unstale":"2024-04-24T23:04:56.295471+0000","last_undegraded":"2024-04-24T23:04:56.295471+0000","last_fullsized":"2024-04-24T23:04:56.295471+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:22:05.535544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.301333+0000","last_change":"2024-04-24T23:04:56.301218+0000","last_active":"2024-04-24T23:04:56.301333+0000","last_peered":"2024-04-24T23:04:56.301333+0000","last_clean":"2024-04-24T23:04:56.301333+0000","last_became_active":"2024-04-24T23:04:56.300935+0000","last_became_peered":"2024-04-24T23:04:56.300935+0000","last_unstale":"2024-04-24T23:04:56.301333+0000","last_undegraded":"2024-04-24T23:04:56.301333+0000","last_fullsized":"2024-04-24T23:04:56.301333+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:47:47.452807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.299959+0000","last_change":"2024-04-24T23:04:56.298604+0000","last_active":"2024-04-24T23:04:56.299959+0000","last_peered":"2024-04-24T23:04:56.299959+0000","last_clean":"2024-04-24T23:04:56.299959+0000","last_became_active":"2024-04-24T23:04:56.298374+0000","last_became_peered":"2024-04-24T23:04:56.298374+0000","last_unstale":"2024-04-24T23:04:56.299959+0000","last_undegraded":"2024-04-24T23:04:56.299959+0000","last_fullsized":"2024-04-24T23:04:56.299959+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:04.889874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.297123+0000","last_change":"2024-04-24T23:04:56.297011+0000","last_active":"2024-04-24T23:04:56.297123+0000","last_peered":"2024-04-24T23:04:56.297123+0000","last_clean":"2024-04-24T23:04:56.297123+0000","last_became_active":"2024-04-24T23:04:56.296508+0000","last_became_peered":"2024-04-24T23:04:56.296508+0000","last_unstale":"2024-04-24T23:04:56.297123+0000","last_undegraded":"2024-04-24T23:04:56.297123+0000","last_fullsized":"2024-04-24T23:04:56.297123+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:14:29.838138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.299976+0000","last_change":"2024-04-24T23:04:56.298747+0000","last_active":"2024-04-24T23:04:56.299976+0000","last_peered":"2024-04-24T23:04:56.299976+0000","last_clean":"2024-04-24T23:04:56.299976+0000","last_became_active":"2024-04-24T23:04:56.298159+0000","last_became_peered":"2024-04-24T23:04:56.298159+0000","last_unstale":"2024-04-24T23:04:56.299976+0000","last_undegraded":"2024-04-24T23:04:56.299976+0000","last_fullsized":"2024-04-24T23:04:56.299976+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:26:30.517186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.297252+0000","last_change":"2024-04-24T23:04:56.296477+0000","last_active":"2024-04-24T23:04:56.297252+0000","last_peered":"2024-04-24T23:04:56.297252+0000","last_clean":"2024-04-24T23:04:56.297252+0000","last_became_active":"2024-04-24T23:04:56.296252+0000","last_became_peered":"2024-04-24T23:04:56.296252+0000","last_unstale":"2024-04-24T23:04:56.297252+0000","last_undegraded":"2024-04-24T23:04:56.297252+0000","last_fullsized":"2024-04-24T23:04:56.297252+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:56:06.843889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.306078+0000","last_change":"2024-04-24T23:04:56.305944+0000","last_active":"2024-04-24T23:04:56.306078+0000","last_peered":"2024-04-24T23:04:56.306078+0000","last_clean":"2024-04-24T23:04:56.306078+0000","last_became_active":"2024-04-24T23:04:56.305502+0000","last_became_peered":"2024-04-24T23:04:56.305502+0000","last_unstale":"2024-04-24T23:04:56.306078+0000","last_undegraded":"2024-04-24T23:04:56.306078+0000","last_fullsized":"2024-04-24T23:04:56.306078+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:08:56.967524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.305696+0000","last_change":"2024-04-24T23:04:56.305576+0000","last_active":"2024-04-24T23:04:56.305696+0000","last_peered":"2024-04-24T23:04:56.305696+0000","last_clean":"2024-04-24T23:04:56.305696+0000","last_became_active":"2024-04-24T23:04:56.305325+0000","last_became_peered":"2024-04-24T23:04:56.305325+0000","last_unstale":"2024-04-24T23:04:56.305696+0000","last_undegraded":"2024-04-24T23:04:56.305696+0000","last_fullsized":"2024-04-24T23:04:56.305696+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:58.990812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.304817+0000","last_change":"2024-04-24T23:04:56.304395+0000","last_active":"2024-04-24T23:04:56.304817+0000","last_peered":"2024-04-24T23:04:56.304817+0000","last_clean":"2024-04-24T23:04:56.304817+0000","last_became_active":"2024-04-24T23:04:56.304032+0000","last_became_peered":"2024-04-24T23:04:56.304032+0000","last_unstale":"2024-04-24T23:04:56.304817+0000","last_undegraded":"2024-04-24T23:04:56.304817+0000","last_fullsized":"2024-04-24T23:04:56.304817+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:11:03.231462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.295553+0000","last_change":"2024-04-24T23:04:56.295452+0000","last_active":"2024-04-24T23:04:56.295553+0000","last_peered":"2024-04-24T23:04:56.295553+0000","last_clean":"2024-04-24T23:04:56.295553+0000","last_became_active":"2024-04-24T23:04:56.295226+0000","last_became_peered":"2024-04-24T23:04:56.295226+0000","last_unstale":"2024-04-24T23:04:56.295553+0000","last_undegraded":"2024-04-24T23:04:56.295553+0000","last_fullsized":"2024-04-24T23:04:56.295553+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:04:17.234668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.304838+0000","last_change":"2024-04-24T23:04:56.304379+0000","last_active":"2024-04-24T23:04:56.304838+0000","last_peered":"2024-04-24T23:04:56.304838+0000","last_clean":"2024-04-24T23:04:56.304838+0000","last_became_active":"2024-04-24T23:04:56.304141+0000","last_became_peered":"2024-04-24T23:04:56.304141+0000","last_unstale":"2024-04-24T23:04:56.304838+0000","last_undegraded":"2024-04-24T23:04:56.304838+0000","last_fullsized":"2024-04-24T23:04:56.304838+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:09:15.954439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.299370+0000","last_change":"2024-04-24T23:04:56.299066+0000","last_active":"2024-04-24T23:04:56.299370+0000","last_peered":"2024-04-24T23:04:56.299370+0000","last_clean":"2024-04-24T23:04:56.299370+0000","last_became_active":"2024-04-24T23:04:56.298777+0000","last_became_peered":"2024-04-24T23:04:56.298777+0000","last_unstale":"2024-04-24T23:04:56.299370+0000","last_undegraded":"2024-04-24T23:04:56.299370+0000","last_fullsized":"2024-04-24T23:04:56.299370+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:31:24.305630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.300569+0000","last_change":"2024-04-24T23:04:56.299070+0000","last_active":"2024-04-24T23:04:56.300569+0000","last_peered":"2024-04-24T23:04:56.300569+0000","last_clean":"2024-04-24T23:04:56.300569+0000","last_became_active":"2024-04-24T23:04:56.298605+0000","last_became_peered":"2024-04-24T23:04:56.298605+0000","last_unstale":"2024-04-24T23:04:56.300569+0000","last_undegraded":"2024-04-24T23:04:56.300569+0000","last_fullsized":"2024-04-24T23:04:56.300569+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:55:26.412963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.306039+0000","last_change":"2024-04-24T23:04:56.305938+0000","last_active":"2024-04-24T23:04:56.306039+0000","last_peered":"2024-04-24T23:04:56.306039+0000","last_clean":"2024-04-24T23:04:56.306039+0000","last_became_active":"2024-04-24T23:04:56.305699+0000","last_became_peered":"2024-04-24T23:04:56.305699+0000","last_unstale":"2024-04-24T23:04:56.306039+0000","last_undegraded":"2024-04-24T23:04:56.306039+0000","last_fullsized":"2024-04-24T23:04:56.306039+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:29:53.907588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.305124+0000","last_change":"2024-04-24T23:04:56.304956+0000","last_active":"2024-04-24T23:04:56.305124+0000","last_peered":"2024-04-24T23:04:56.305124+0000","last_clean":"2024-04-24T23:04:56.305124+0000","last_became_active":"2024-04-24T23:04:56.303397+0000","last_became_peered":"2024-04-24T23:04:56.303397+0000","last_unstale":"2024-04-24T23:04:56.305124+0000","last_undegraded":"2024-04-24T23:04:56.305124+0000","last_fullsized":"2024-04-24T23:04:56.305124+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:03:37.194732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.299987+0000","last_change":"2024-04-24T23:04:56.298680+0000","last_active":"2024-04-24T23:04:56.299987+0000","last_peered":"2024-04-24T23:04:56.299987+0000","last_clean":"2024-04-24T23:04:56.299987+0000","last_became_active":"2024-04-24T23:04:56.297332+0000","last_became_peered":"2024-04-24T23:04:56.297332+0000","last_unstale":"2024-04-24T23:04:56.299987+0000","last_undegraded":"2024-04-24T23:04:56.299987+0000","last_fullsized":"2024-04-24T23:04:56.299987+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:05.152014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.304838+0000","last_change":"2024-04-24T23:04:56.304457+0000","last_active":"2024-04-24T23:04:56.304838+0000","last_peered":"2024-04-24T23:04:56.304838+0000","last_clean":"2024-04-24T23:04:56.304838+0000","last_became_active":"2024-04-24T23:04:56.304142+0000","last_became_peered":"2024-04-24T23:04:56.304142+0000","last_unstale":"2024-04-24T23:04:56.304838+0000","last_undegraded":"2024-04-24T23:04:56.304838+0000","last_fullsized":"2024-04-24T23:04:56.304838+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:42:08.073443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.292732+0000","last_change":"2024-04-24T23:04:56.292630+0000","last_active":"2024-04-24T23:04:56.292732+0000","last_peered":"2024-04-24T23:04:56.292732+0000","last_clean":"2024-04-24T23:04:56.292732+0000","last_became_active":"2024-04-24T23:04:56.292363+0000","last_became_peered":"2024-04-24T23:04:56.292363+0000","last_unstale":"2024-04-24T23:04:56.292732+0000","last_undegraded":"2024-04-24T23:04:56.292732+0000","last_fullsized":"2024-04-24T23:04:56.292732+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:00:31.940037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.299766+0000","last_change":"2024-04-24T23:04:56.299627+0000","last_active":"2024-04-24T23:04:56.299766+0000","last_peered":"2024-04-24T23:04:56.299766+0000","last_clean":"2024-04-24T23:04:56.299766+0000","last_became_active":"2024-04-24T23:04:56.299375+0000","last_became_peered":"2024-04-24T23:04:56.299375+0000","last_unstale":"2024-04-24T23:04:56.299766+0000","last_undegraded":"2024-04-24T23:04:56.299766+0000","last_fullsized":"2024-04-24T23:04:56.299766+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:57.144816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.303657+0000","last_change":"2024-04-24T23:04:56.303514+0000","last_active":"2024-04-24T23:04:56.303657+0000","last_peered":"2024-04-24T23:04:56.303657+0000","last_clean":"2024-04-24T23:04:56.303657+0000","last_became_active":"2024-04-24T23:04:56.303264+0000","last_became_peered":"2024-04-24T23:04:56.303264+0000","last_unstale":"2024-04-24T23:04:56.303657+0000","last_undegraded":"2024-04-24T23:04:56.303657+0000","last_fullsized":"2024-04-24T23:04:56.303657+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:24:11.103964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.297294+0000","last_change":"2024-04-24T23:04:56.297126+0000","last_active":"2024-04-24T23:04:56.297294+0000","last_peered":"2024-04-24T23:04:56.297294+0000","last_clean":"2024-04-24T23:04:56.297294+0000","last_became_active":"2024-04-24T23:04:56.296856+0000","last_became_peered":"2024-04-24T23:04:56.296856+0000","last_unstale":"2024-04-24T23:04:56.297294+0000","last_undegraded":"2024-04-24T23:04:56.297294+0000","last_fullsized":"2024-04-24T23:04:56.297294+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:27:09.210347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.298561+0000","last_change":"2024-04-24T23:04:56.298261+0000","last_active":"2024-04-24T23:04:56.298561+0000","last_peered":"2024-04-24T23:04:56.298561+0000","last_clean":"2024-04-24T23:04:56.298561+0000","last_became_active":"2024-04-24T23:04:56.297574+0000","last_became_peered":"2024-04-24T23:04:56.297574+0000","last_unstale":"2024-04-24T23:04:56.298561+0000","last_undegraded":"2024-04-24T23:04:56.298561+0000","last_fullsized":"2024-04-24T23:04:56.298561+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:18.121624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.306632+0000","last_change":"2024-04-24T23:04:56.306507+0000","last_active":"2024-04-24T23:04:56.306632+0000","last_peered":"2024-04-24T23:04:56.306632+0000","last_clean":"2024-04-24T23:04:56.306632+0000","last_became_active":"2024-04-24T23:04:56.306144+0000","last_became_peered":"2024-04-24T23:04:56.306144+0000","last_unstale":"2024-04-24T23:04:56.306632+0000","last_undegraded":"2024-04-24T23:04:56.306632+0000","last_fullsized":"2024-04-24T23:04:56.306632+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:07:23.208212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.306612+0000","last_change":"2024-04-24T23:04:56.306502+0000","last_active":"2024-04-24T23:04:56.306612+0000","last_peered":"2024-04-24T23:04:56.306612+0000","last_clean":"2024-04-24T23:04:56.306612+0000","last_became_active":"2024-04-24T23:04:56.305814+0000","last_became_peered":"2024-04-24T23:04:56.305814+0000","last_unstale":"2024-04-24T23:04:56.306612+0000","last_undegraded":"2024-04-24T23:04:56.306612+0000","last_fullsized":"2024-04-24T23:04:56.306612+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:30:14.988093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.304810+0000","last_change":"2024-04-24T23:04:56.304695+0000","last_active":"2024-04-24T23:04:56.304810+0000","last_peered":"2024-04-24T23:04:56.304810+0000","last_clean":"2024-04-24T23:04:56.304810+0000","last_became_active":"2024-04-24T23:04:56.302930+0000","last_became_peered":"2024-04-24T23:04:56.302930+0000","last_unstale":"2024-04-24T23:04:56.304810+0000","last_undegraded":"2024-04-24T23:04:56.304810+0000","last_fullsized":"2024-04-24T23:04:56.304810+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:27.550752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.296877+0000","last_change":"2024-04-24T23:04:56.296717+0000","last_active":"2024-04-24T23:04:56.296877+0000","last_peered":"2024-04-24T23:04:56.296877+0000","last_clean":"2024-04-24T23:04:56.296877+0000","last_became_active":"2024-04-24T23:04:56.296428+0000","last_became_peered":"2024-04-24T23:04:56.296428+0000","last_unstale":"2024-04-24T23:04:56.296877+0000","last_undegraded":"2024-04-24T23:04:56.296877+0000","last_fullsized":"2024-04-24T23:04:56.296877+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:30.891284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.300546+0000","last_change":"2024-04-24T23:04:56.298871+0000","last_active":"2024-04-24T23:04:56.300546+0000","last_peered":"2024-04-24T23:04:56.300546+0000","last_clean":"2024-04-24T23:04:56.300546+0000","last_became_active":"2024-04-24T23:04:56.298333+0000","last_became_peered":"2024-04-24T23:04:56.298333+0000","last_unstale":"2024-04-24T23:04:56.300546+0000","last_undegraded":"2024-04-24T23:04:56.300546+0000","last_fullsized":"2024-04-24T23:04:56.300546+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:46:54.546573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"124'4","reported_seq":16,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.633192+0000","last_change":"2024-04-24T23:04:55.640119+0000","last_active":"2024-04-24T23:04:56.633192+0000","last_peered":"2024-04-24T23:04:56.633192+0000","last_clean":"2024-04-24T23:04:56.633192+0000","last_became_active":"2024-04-24T23:04:55.639812+0000","last_became_peered":"2024-04-24T23:04:55.639812+0000","last_unstale":"2024-04-24T23:04:56.633192+0000","last_undegraded":"2024-04-24T23:04:56.633192+0000","last_fullsized":"2024-04-24T23:04:56.633192+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:04:50.132779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.305167+0000","last_change":"2024-04-24T23:04:56.302433+0000","last_active":"2024-04-24T23:04:56.305167+0000","last_peered":"2024-04-24T23:04:56.305167+0000","last_clean":"2024-04-24T23:04:56.305167+0000","last_became_active":"2024-04-24T23:04:56.302207+0000","last_became_peered":"2024-04-24T23:04:56.302207+0000","last_unstale":"2024-04-24T23:04:56.305167+0000","last_undegraded":"2024-04-24T23:04:56.305167+0000","last_fullsized":"2024-04-24T23:04:56.305167+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:46:05.959174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"124'1","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.631597+0000","last_change":"2024-04-24T23:04:55.632889+0000","last_active":"2024-04-24T23:04:56.631597+0000","last_peered":"2024-04-24T23:04:56.631597+0000","last_clean":"2024-04-24T23:04:56.631597+0000","last_became_active":"2024-04-24T23:04:55.632655+0000","last_became_peered":"2024-04-24T23:04:55.632655+0000","last_unstale":"2024-04-24T23:04:56.631597+0000","last_undegraded":"2024-04-24T23:04:56.631597+0000","last_fullsized":"2024-04-24T23:04:56.631597+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:35:34.066639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.297359+0000","last_change":"2024-04-24T23:04:56.297138+0000","last_active":"2024-04-24T23:04:56.297359+0000","last_peered":"2024-04-24T23:04:56.297359+0000","last_clean":"2024-04-24T23:04:56.297359+0000","last_became_active":"2024-04-24T23:04:56.295653+0000","last_became_peered":"2024-04-24T23:04:56.295653+0000","last_unstale":"2024-04-24T23:04:56.297359+0000","last_undegraded":"2024-04-24T23:04:56.297359+0000","last_fullsized":"2024-04-24T23:04:56.297359+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:56:37.458428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"124'2","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.632513+0000","last_change":"2024-04-24T23:04:55.639981+0000","last_active":"2024-04-24T23:04:56.632513+0000","last_peered":"2024-04-24T23:04:56.632513+0000","last_clean":"2024-04-24T23:04:56.632513+0000","last_became_active":"2024-04-24T23:04:55.639262+0000","last_became_peered":"2024-04-24T23:04:55.639262+0000","last_unstale":"2024-04-24T23:04:56.632513+0000","last_undegraded":"2024-04-24T23:04:56.632513+0000","last_fullsized":"2024-04-24T23:04:56.632513+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:22:56.035112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.296892+0000","last_change":"2024-04-24T23:04:56.296784+0000","last_active":"2024-04-24T23:04:56.296892+0000","last_peered":"2024-04-24T23:04:56.296892+0000","last_clean":"2024-04-24T23:04:56.296892+0000","last_became_active":"2024-04-24T23:04:56.296194+0000","last_became_peered":"2024-04-24T23:04:56.296194+0000","last_unstale":"2024-04-24T23:04:56.296892+0000","last_undegraded":"2024-04-24T23:04:56.296892+0000","last_fullsized":"2024-04-24T23:04:56.296892+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:48.318106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"124'2","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.633189+0000","last_change":"2024-04-24T23:04:55.641448+0000","last_active":"2024-04-24T23:04:56.633189+0000","last_peered":"2024-04-24T23:04:56.633189+0000","last_clean":"2024-04-24T23:04:56.633189+0000","last_became_active":"2024-04-24T23:04:55.640749+0000","last_became_peered":"2024-04-24T23:04:55.640749+0000","last_unstale":"2024-04-24T23:04:56.633189+0000","last_undegraded":"2024-04-24T23:04:56.633189+0000","last_fullsized":"2024-04-24T23:04:56.633189+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:26:42.681740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.301472+0000","last_change":"2024-04-24T23:04:56.301322+0000","last_active":"2024-04-24T23:04:56.301472+0000","last_peered":"2024-04-24T23:04:56.301472+0000","last_clean":"2024-04-24T23:04:56.301472+0000","last_became_active":"2024-04-24T23:04:56.301081+0000","last_became_peered":"2024-04-24T23:04:56.301081+0000","last_unstale":"2024-04-24T23:04:56.301472+0000","last_undegraded":"2024-04-24T23:04:56.301472+0000","last_fullsized":"2024-04-24T23:04:56.301472+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:43:43.227101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"124'1","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.632445+0000","last_change":"2024-04-24T23:04:55.634733+0000","last_active":"2024-04-24T23:04:56.632445+0000","last_peered":"2024-04-24T23:04:56.632445+0000","last_clean":"2024-04-24T23:04:56.632445+0000","last_became_active":"2024-04-24T23:04:55.634440+0000","last_became_peered":"2024-04-24T23:04:55.634440+0000","last_unstale":"2024-04-24T23:04:56.632445+0000","last_undegraded":"2024-04-24T23:04:56.632445+0000","last_fullsized":"2024-04-24T23:04:56.632445+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:45:23.578055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.299938+0000","last_change":"2024-04-24T23:04:56.298387+0000","last_active":"2024-04-24T23:04:56.299938+0000","last_peered":"2024-04-24T23:04:56.299938+0000","last_clean":"2024-04-24T23:04:56.299938+0000","last_became_active":"2024-04-24T23:04:56.298018+0000","last_became_peered":"2024-04-24T23:04:56.298018+0000","last_unstale":"2024-04-24T23:04:56.299938+0000","last_undegraded":"2024-04-24T23:04:56.299938+0000","last_fullsized":"2024-04-24T23:04:56.299938+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:14:04.049719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"124'1","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.632689+0000","last_change":"2024-04-24T23:04:55.641651+0000","last_active":"2024-04-24T23:04:56.632689+0000","last_peered":"2024-04-24T23:04:56.632689+0000","last_clean":"2024-04-24T23:04:56.632689+0000","last_became_active":"2024-04-24T23:04:55.641141+0000","last_became_peered":"2024-04-24T23:04:55.641141+0000","last_unstale":"2024-04-24T23:04:56.632689+0000","last_undegraded":"2024-04-24T23:04:56.632689+0000","last_fullsized":"2024-04-24T23:04:56.632689+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:05:41.689378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.306280+0000","last_change":"2024-04-24T23:04:56.306160+0000","last_active":"2024-04-24T23:04:56.306280+0000","last_peered":"2024-04-24T23:04:56.306280+0000","last_clean":"2024-04-24T23:04:56.306280+0000","last_became_active":"2024-04-24T23:04:56.305413+0000","last_became_peered":"2024-04-24T23:04:56.305413+0000","last_unstale":"2024-04-24T23:04:56.306280+0000","last_undegraded":"2024-04-24T23:04:56.306280+0000","last_fullsized":"2024-04-24T23:04:56.306280+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:06:22.157421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.287011+0000","last_change":"2024-04-24T23:04:55.635238+0000","last_active":"2024-04-24T23:04:56.287011+0000","last_peered":"2024-04-24T23:04:56.287011+0000","last_clean":"2024-04-24T23:04:56.287011+0000","last_became_active":"2024-04-24T23:04:55.634924+0000","last_became_peered":"2024-04-24T23:04:55.634924+0000","last_unstale":"2024-04-24T23:04:56.287011+0000","last_undegraded":"2024-04-24T23:04:56.287011+0000","last_fullsized":"2024-04-24T23:04:56.287011+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:40:47.430362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.298044+0000","last_change":"2024-04-24T23:04:56.297932+0000","last_active":"2024-04-24T23:04:56.298044+0000","last_peered":"2024-04-24T23:04:56.298044+0000","last_clean":"2024-04-24T23:04:56.298044+0000","last_became_active":"2024-04-24T23:04:56.297694+0000","last_became_peered":"2024-04-24T23:04:56.297694+0000","last_unstale":"2024-04-24T23:04:56.298044+0000","last_undegraded":"2024-04-24T23:04:56.298044+0000","last_fullsized":"2024-04-24T23:04:56.298044+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:17:20.878502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.293312+0000","last_change":"2024-04-24T23:04:55.636020+0000","last_active":"2024-04-24T23:04:56.293312+0000","last_peered":"2024-04-24T23:04:56.293312+0000","last_clean":"2024-04-24T23:04:56.293312+0000","last_became_active":"2024-04-24T23:04:55.635763+0000","last_became_peered":"2024-04-24T23:04:55.635763+0000","last_unstale":"2024-04-24T23:04:56.293312+0000","last_undegraded":"2024-04-24T23:04:56.293312+0000","last_fullsized":"2024-04-24T23:04:56.293312+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:17:45.085895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.301003+0000","last_change":"2024-04-24T23:04:56.300873+0000","last_active":"2024-04-24T23:04:56.301003+0000","last_peered":"2024-04-24T23:04:56.301003+0000","last_clean":"2024-04-24T23:04:56.301003+0000","last_became_active":"2024-04-24T23:04:56.300140+0000","last_became_peered":"2024-04-24T23:04:56.300140+0000","last_unstale":"2024-04-24T23:04:56.301003+0000","last_undegraded":"2024-04-24T23:04:56.301003+0000","last_fullsized":"2024-04-24T23:04:56.301003+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:26:30.312012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.299950+0000","last_change":"2024-04-24T23:04:55.640569+0000","last_active":"2024-04-24T23:04:56.299950+0000","last_peered":"2024-04-24T23:04:56.299950+0000","last_clean":"2024-04-24T23:04:56.299950+0000","last_became_active":"2024-04-24T23:04:55.639955+0000","last_became_peered":"2024-04-24T23:04:55.639955+0000","last_unstale":"2024-04-24T23:04:56.299950+0000","last_undegraded":"2024-04-24T23:04:56.299950+0000","last_fullsized":"2024-04-24T23:04:56.299950+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:52:55.132318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.300025+0000","last_change":"2024-04-24T23:04:56.299898+0000","last_active":"2024-04-24T23:04:56.300025+0000","last_peered":"2024-04-24T23:04:56.300025+0000","last_clean":"2024-04-24T23:04:56.300025+0000","last_became_active":"2024-04-24T23:04:56.296789+0000","last_became_peered":"2024-04-24T23:04:56.296789+0000","last_unstale":"2024-04-24T23:04:56.300025+0000","last_undegraded":"2024-04-24T23:04:56.300025+0000","last_fullsized":"2024-04-24T23:04:56.300025+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:09.824794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.292019+0000","last_change":"2024-04-24T23:04:55.638415+0000","last_active":"2024-04-24T23:04:56.292019+0000","last_peered":"2024-04-24T23:04:56.292019+0000","last_clean":"2024-04-24T23:04:56.292019+0000","last_became_active":"2024-04-24T23:04:55.638142+0000","last_became_peered":"2024-04-24T23:04:55.638142+0000","last_unstale":"2024-04-24T23:04:56.292019+0000","last_undegraded":"2024-04-24T23:04:56.292019+0000","last_fullsized":"2024-04-24T23:04:56.292019+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:41:15.782557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.292646+0000","last_change":"2024-04-24T23:04:56.292499+0000","last_active":"2024-04-24T23:04:56.292646+0000","last_peered":"2024-04-24T23:04:56.292646+0000","last_clean":"2024-04-24T23:04:56.292646+0000","last_became_active":"2024-04-24T23:04:56.292252+0000","last_became_peered":"2024-04-24T23:04:56.292252+0000","last_unstale":"2024-04-24T23:04:56.292646+0000","last_undegraded":"2024-04-24T23:04:56.292646+0000","last_fullsized":"2024-04-24T23:04:56.292646+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:00:58.925068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"124'3","reported_seq":15,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.632767+0000","last_change":"2024-04-24T23:04:55.636011+0000","last_active":"2024-04-24T23:04:56.632767+0000","last_peered":"2024-04-24T23:04:56.632767+0000","last_clean":"2024-04-24T23:04:56.632767+0000","last_became_active":"2024-04-24T23:04:55.635750+0000","last_became_peered":"2024-04-24T23:04:55.635750+0000","last_unstale":"2024-04-24T23:04:56.632767+0000","last_undegraded":"2024-04-24T23:04:56.632767+0000","last_fullsized":"2024-04-24T23:04:56.632767+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:34:14.335998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.295895+0000","last_change":"2024-04-24T23:04:56.295318+0000","last_active":"2024-04-24T23:04:56.295895+0000","last_peered":"2024-04-24T23:04:56.295895+0000","last_clean":"2024-04-24T23:04:56.295895+0000","last_became_active":"2024-04-24T23:04:56.295088+0000","last_became_peered":"2024-04-24T23:04:56.295088+0000","last_unstale":"2024-04-24T23:04:56.295895+0000","last_undegraded":"2024-04-24T23:04:56.295895+0000","last_fullsized":"2024-04-24T23:04:56.295895+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:41:28.277210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"124'6","reported_seq":18,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.634845+0000","last_change":"2024-04-24T23:04:55.638235+0000","last_active":"2024-04-24T23:04:56.634845+0000","last_peered":"2024-04-24T23:04:56.634845+0000","last_clean":"2024-04-24T23:04:56.634845+0000","last_became_active":"2024-04-24T23:04:55.637985+0000","last_became_peered":"2024-04-24T23:04:55.637985+0000","last_unstale":"2024-04-24T23:04:56.634845+0000","last_undegraded":"2024-04-24T23:04:56.634845+0000","last_fullsized":"2024-04-24T23:04:56.634845+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:55:16.497642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.296301+0000","last_change":"2024-04-24T23:04:56.296154+0000","last_active":"2024-04-24T23:04:56.296301+0000","last_peered":"2024-04-24T23:04:56.296301+0000","last_clean":"2024-04-24T23:04:56.296301+0000","last_became_active":"2024-04-24T23:04:56.295135+0000","last_became_peered":"2024-04-24T23:04:56.295135+0000","last_unstale":"2024-04-24T23:04:56.296301+0000","last_undegraded":"2024-04-24T23:04:56.296301+0000","last_fullsized":"2024-04-24T23:04:56.296301+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:49:48.073966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"124'2","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.633003+0000","last_change":"2024-04-24T23:04:55.635279+0000","last_active":"2024-04-24T23:04:56.633003+0000","last_peered":"2024-04-24T23:04:56.633003+0000","last_clean":"2024-04-24T23:04:56.633003+0000","last_became_active":"2024-04-24T23:04:55.635018+0000","last_became_peered":"2024-04-24T23:04:55.635018+0000","last_unstale":"2024-04-24T23:04:56.633003+0000","last_undegraded":"2024-04-24T23:04:56.633003+0000","last_fullsized":"2024-04-24T23:04:56.633003+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:59.921601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.306061+0000","last_change":"2024-04-24T23:04:56.305946+0000","last_active":"2024-04-24T23:04:56.306061+0000","last_peered":"2024-04-24T23:04:56.306061+0000","last_clean":"2024-04-24T23:04:56.306061+0000","last_became_active":"2024-04-24T23:04:56.305580+0000","last_became_peered":"2024-04-24T23:04:56.305580+0000","last_unstale":"2024-04-24T23:04:56.306061+0000","last_undegraded":"2024-04-24T23:04:56.306061+0000","last_fullsized":"2024-04-24T23:04:56.306061+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:22.488951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.294816+0000","last_change":"2024-04-24T23:04:55.638955+0000","last_active":"2024-04-24T23:04:56.294816+0000","last_peered":"2024-04-24T23:04:56.294816+0000","last_clean":"2024-04-24T23:04:56.294816+0000","last_became_active":"2024-04-24T23:04:55.636523+0000","last_became_peered":"2024-04-24T23:04:55.636523+0000","last_unstale":"2024-04-24T23:04:56.294816+0000","last_undegraded":"2024-04-24T23:04:56.294816+0000","last_fullsized":"2024-04-24T23:04:56.294816+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:53:47.040506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.300011+0000","last_change":"2024-04-24T23:04:56.299109+0000","last_active":"2024-04-24T23:04:56.300011+0000","last_peered":"2024-04-24T23:04:56.300011+0000","last_clean":"2024-04-24T23:04:56.300011+0000","last_became_active":"2024-04-24T23:04:56.298886+0000","last_became_peered":"2024-04-24T23:04:56.298886+0000","last_unstale":"2024-04-24T23:04:56.300011+0000","last_undegraded":"2024-04-24T23:04:56.300011+0000","last_fullsized":"2024-04-24T23:04:56.300011+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:12:12.807753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.294643+0000","last_change":"2024-04-24T23:04:55.636814+0000","last_active":"2024-04-24T23:04:56.294643+0000","last_peered":"2024-04-24T23:04:56.294643+0000","last_clean":"2024-04-24T23:04:56.294643+0000","last_became_active":"2024-04-24T23:04:55.636564+0000","last_became_peered":"2024-04-24T23:04:55.636564+0000","last_unstale":"2024-04-24T23:04:56.294643+0000","last_undegraded":"2024-04-24T23:04:56.294643+0000","last_fullsized":"2024-04-24T23:04:56.294643+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:57:27.084042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.301735+0000","last_change":"2024-04-24T23:04:56.301592+0000","last_active":"2024-04-24T23:04:56.301735+0000","last_peered":"2024-04-24T23:04:56.301735+0000","last_clean":"2024-04-24T23:04:56.301735+0000","last_became_active":"2024-04-24T23:04:56.301316+0000","last_became_peered":"2024-04-24T23:04:56.301316+0000","last_unstale":"2024-04-24T23:04:56.301735+0000","last_undegraded":"2024-04-24T23:04:56.301735+0000","last_fullsized":"2024-04-24T23:04:56.301735+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:07:48.640162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"124'8","reported_seq":20,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.635268+0000","last_change":"2024-04-24T23:04:55.640903+0000","last_active":"2024-04-24T23:04:56.635268+0000","last_peered":"2024-04-24T23:04:56.635268+0000","last_clean":"2024-04-24T23:04:56.635268+0000","last_became_active":"2024-04-24T23:04:55.635883+0000","last_became_peered":"2024-04-24T23:04:55.635883+0000","last_unstale":"2024-04-24T23:04:56.635268+0000","last_undegraded":"2024-04-24T23:04:56.635268+0000","last_fullsized":"2024-04-24T23:04:56.635268+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:30.760629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.295372+0000","last_change":"2024-04-24T23:04:56.295214+0000","last_active":"2024-04-24T23:04:56.295372+0000","last_peered":"2024-04-24T23:04:56.295372+0000","last_clean":"2024-04-24T23:04:56.295372+0000","last_became_active":"2024-04-24T23:04:56.294977+0000","last_became_peered":"2024-04-24T23:04:56.294977+0000","last_unstale":"2024-04-24T23:04:56.295372+0000","last_undegraded":"2024-04-24T23:04:56.295372+0000","last_fullsized":"2024-04-24T23:04:56.295372+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:15.499923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"124'2","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.632480+0000","last_change":"2024-04-24T23:04:55.638319+0000","last_active":"2024-04-24T23:04:56.632480+0000","last_peered":"2024-04-24T23:04:56.632480+0000","last_clean":"2024-04-24T23:04:56.632480+0000","last_became_active":"2024-04-24T23:04:55.638027+0000","last_became_peered":"2024-04-24T23:04:55.638027+0000","last_unstale":"2024-04-24T23:04:56.632480+0000","last_undegraded":"2024-04-24T23:04:56.632480+0000","last_fullsized":"2024-04-24T23:04:56.632480+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:24:10.334282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.299099+0000","last_change":"2024-04-24T23:04:56.298844+0000","last_active":"2024-04-24T23:04:56.299099+0000","last_peered":"2024-04-24T23:04:56.299099+0000","last_clean":"2024-04-24T23:04:56.299099+0000","last_became_active":"2024-04-24T23:04:56.297635+0000","last_became_peered":"2024-04-24T23:04:56.297635+0000","last_unstale":"2024-04-24T23:04:56.299099+0000","last_undegraded":"2024-04-24T23:04:56.299099+0000","last_fullsized":"2024-04-24T23:04:56.299099+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:56:12.264690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"124'1","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.632026+0000","last_change":"2024-04-24T23:04:55.634143+0000","last_active":"2024-04-24T23:04:56.632026+0000","last_peered":"2024-04-24T23:04:56.632026+0000","last_clean":"2024-04-24T23:04:56.632026+0000","last_became_active":"2024-04-24T23:04:55.633714+0000","last_became_peered":"2024-04-24T23:04:55.633714+0000","last_unstale":"2024-04-24T23:04:56.632026+0000","last_undegraded":"2024-04-24T23:04:56.632026+0000","last_fullsized":"2024-04-24T23:04:56.632026+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:37:02.355090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.303540+0000","last_change":"2024-04-24T23:04:56.303436+0000","last_active":"2024-04-24T23:04:56.303540+0000","last_peered":"2024-04-24T23:04:56.303540+0000","last_clean":"2024-04-24T23:04:56.303540+0000","last_became_active":"2024-04-24T23:04:56.303256+0000","last_became_peered":"2024-04-24T23:04:56.303256+0000","last_unstale":"2024-04-24T23:04:56.303540+0000","last_undegraded":"2024-04-24T23:04:56.303540+0000","last_fullsized":"2024-04-24T23:04:56.303540+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:24:55.532403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.291870+0000","last_change":"2024-04-24T23:04:55.638108+0000","last_active":"2024-04-24T23:04:56.291870+0000","last_peered":"2024-04-24T23:04:56.291870+0000","last_clean":"2024-04-24T23:04:56.291870+0000","last_became_active":"2024-04-24T23:04:55.637880+0000","last_became_peered":"2024-04-24T23:04:55.637880+0000","last_unstale":"2024-04-24T23:04:56.291870+0000","last_undegraded":"2024-04-24T23:04:56.291870+0000","last_fullsized":"2024-04-24T23:04:56.291870+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:08.229039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.304745+0000","last_change":"2024-04-24T23:04:56.302906+0000","last_active":"2024-04-24T23:04:56.304745+0000","last_peered":"2024-04-24T23:04:56.304745+0000","last_clean":"2024-04-24T23:04:56.304745+0000","last_became_active":"2024-04-24T23:04:56.302662+0000","last_became_peered":"2024-04-24T23:04:56.302662+0000","last_unstale":"2024-04-24T23:04:56.304745+0000","last_undegraded":"2024-04-24T23:04:56.304745+0000","last_fullsized":"2024-04-24T23:04:56.304745+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:32.738827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"124'2","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.635841+0000","last_change":"2024-04-24T23:04:55.641264+0000","last_active":"2024-04-24T23:04:56.635841+0000","last_peered":"2024-04-24T23:04:56.635841+0000","last_clean":"2024-04-24T23:04:56.635841+0000","last_became_active":"2024-04-24T23:04:55.639584+0000","last_became_peered":"2024-04-24T23:04:55.639584+0000","last_unstale":"2024-04-24T23:04:56.635841+0000","last_undegraded":"2024-04-24T23:04:56.635841+0000","last_fullsized":"2024-04-24T23:04:56.635841+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:10:21.463856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.300836+0000","last_change":"2024-04-24T23:04:56.299539+0000","last_active":"2024-04-24T23:04:56.300836+0000","last_peered":"2024-04-24T23:04:56.300836+0000","last_clean":"2024-04-24T23:04:56.300836+0000","last_became_active":"2024-04-24T23:04:56.298309+0000","last_became_peered":"2024-04-24T23:04:56.298309+0000","last_unstale":"2024-04-24T23:04:56.300836+0000","last_undegraded":"2024-04-24T23:04:56.300836+0000","last_fullsized":"2024-04-24T23:04:56.300836+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:48:25.659158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.289644+0000","last_change":"2024-04-24T23:04:55.634554+0000","last_active":"2024-04-24T23:04:56.289644+0000","last_peered":"2024-04-24T23:04:56.289644+0000","last_clean":"2024-04-24T23:04:56.289644+0000","last_became_active":"2024-04-24T23:04:55.633943+0000","last_became_peered":"2024-04-24T23:04:55.633943+0000","last_unstale":"2024-04-24T23:04:56.289644+0000","last_undegraded":"2024-04-24T23:04:56.289644+0000","last_fullsized":"2024-04-24T23:04:56.289644+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:04:06.826836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.299875+0000","last_change":"2024-04-24T23:04:56.299592+0000","last_active":"2024-04-24T23:04:56.299875+0000","last_peered":"2024-04-24T23:04:56.299875+0000","last_clean":"2024-04-24T23:04:56.299875+0000","last_became_active":"2024-04-24T23:04:56.297889+0000","last_became_peered":"2024-04-24T23:04:56.297889+0000","last_unstale":"2024-04-24T23:04:56.299875+0000","last_undegraded":"2024-04-24T23:04:56.299875+0000","last_fullsized":"2024-04-24T23:04:56.299875+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:41:11.995488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.294048+0000","last_change":"2024-04-24T23:04:55.634641+0000","last_active":"2024-04-24T23:04:56.294048+0000","last_peered":"2024-04-24T23:04:56.294048+0000","last_clean":"2024-04-24T23:04:56.294048+0000","last_became_active":"2024-04-24T23:04:55.633996+0000","last_became_peered":"2024-04-24T23:04:55.633996+0000","last_unstale":"2024-04-24T23:04:56.294048+0000","last_undegraded":"2024-04-24T23:04:56.294048+0000","last_fullsized":"2024-04-24T23:04:56.294048+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:57:15.362988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.302888+0000","last_change":"2024-04-24T23:04:56.302804+0000","last_active":"2024-04-24T23:04:56.302888+0000","last_peered":"2024-04-24T23:04:56.302888+0000","last_clean":"2024-04-24T23:04:56.302888+0000","last_became_active":"2024-04-24T23:04:56.302608+0000","last_became_peered":"2024-04-24T23:04:56.302608+0000","last_unstale":"2024-04-24T23:04:56.302888+0000","last_undegraded":"2024-04-24T23:04:56.302888+0000","last_fullsized":"2024-04-24T23:04:56.302888+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:59:19.113955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.300760+0000","last_change":"2024-04-24T23:04:55.640758+0000","last_active":"2024-04-24T23:04:56.300760+0000","last_peered":"2024-04-24T23:04:56.300760+0000","last_clean":"2024-04-24T23:04:56.300760+0000","last_became_active":"2024-04-24T23:04:55.640334+0000","last_became_peered":"2024-04-24T23:04:55.640334+0000","last_unstale":"2024-04-24T23:04:56.300760+0000","last_undegraded":"2024-04-24T23:04:56.300760+0000","last_fullsized":"2024-04-24T23:04:56.300760+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:14:05.309948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.297787+0000","last_change":"2024-04-24T23:04:56.294014+0000","last_active":"2024-04-24T23:04:56.297787+0000","last_peered":"2024-04-24T23:04:56.297787+0000","last_clean":"2024-04-24T23:04:56.297787+0000","last_became_active":"2024-04-24T23:04:56.293757+0000","last_became_peered":"2024-04-24T23:04:56.293757+0000","last_unstale":"2024-04-24T23:04:56.297787+0000","last_undegraded":"2024-04-24T23:04:56.297787+0000","last_fullsized":"2024-04-24T23:04:56.297787+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:04:01.666435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"124'2","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.632382+0000","last_change":"2024-04-24T23:04:55.636245+0000","last_active":"2024-04-24T23:04:56.632382+0000","last_peered":"2024-04-24T23:04:56.632382+0000","last_clean":"2024-04-24T23:04:56.632382+0000","last_became_active":"2024-04-24T23:04:55.635890+0000","last_became_peered":"2024-04-24T23:04:55.635890+0000","last_unstale":"2024-04-24T23:04:56.632382+0000","last_undegraded":"2024-04-24T23:04:56.632382+0000","last_fullsized":"2024-04-24T23:04:56.632382+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:42:07.230037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.298308+0000","last_change":"2024-04-24T23:04:56.296966+0000","last_active":"2024-04-24T23:04:56.298308+0000","last_peered":"2024-04-24T23:04:56.298308+0000","last_clean":"2024-04-24T23:04:56.298308+0000","last_became_active":"2024-04-24T23:04:56.296204+0000","last_became_peered":"2024-04-24T23:04:56.296204+0000","last_unstale":"2024-04-24T23:04:56.298308+0000","last_undegraded":"2024-04-24T23:04:56.298308+0000","last_fullsized":"2024-04-24T23:04:56.298308+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:23:39.368377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.291492+0000","last_change":"2024-04-24T23:04:55.640732+0000","last_active":"2024-04-24T23:04:56.291492+0000","last_peered":"2024-04-24T23:04:56.291492+0000","last_clean":"2024-04-24T23:04:56.291492+0000","last_became_active":"2024-04-24T23:04:55.640351+0000","last_became_peered":"2024-04-24T23:04:55.640351+0000","last_unstale":"2024-04-24T23:04:56.291492+0000","last_undegraded":"2024-04-24T23:04:56.291492+0000","last_fullsized":"2024-04-24T23:04:56.291492+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:36:12.441787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.304885+0000","last_change":"2024-04-24T23:04:56.304731+0000","last_active":"2024-04-24T23:04:56.304885+0000","last_peered":"2024-04-24T23:04:56.304885+0000","last_clean":"2024-04-24T23:04:56.304885+0000","last_became_active":"2024-04-24T23:04:56.303061+0000","last_became_peered":"2024-04-24T23:04:56.303061+0000","last_unstale":"2024-04-24T23:04:56.304885+0000","last_undegraded":"2024-04-24T23:04:56.304885+0000","last_fullsized":"2024-04-24T23:04:56.304885+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:57.785936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.293944+0000","last_change":"2024-04-24T23:04:55.636039+0000","last_active":"2024-04-24T23:04:56.293944+0000","last_peered":"2024-04-24T23:04:56.293944+0000","last_clean":"2024-04-24T23:04:56.293944+0000","last_became_active":"2024-04-24T23:04:55.635633+0000","last_became_peered":"2024-04-24T23:04:55.635633+0000","last_unstale":"2024-04-24T23:04:56.293944+0000","last_undegraded":"2024-04-24T23:04:56.293944+0000","last_fullsized":"2024-04-24T23:04:56.293944+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:29:19.736267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.295777+0000","last_change":"2024-04-24T23:04:56.295661+0000","last_active":"2024-04-24T23:04:56.295777+0000","last_peered":"2024-04-24T23:04:56.295777+0000","last_clean":"2024-04-24T23:04:56.295777+0000","last_became_active":"2024-04-24T23:04:56.294758+0000","last_became_peered":"2024-04-24T23:04:56.294758+0000","last_unstale":"2024-04-24T23:04:56.295777+0000","last_undegraded":"2024-04-24T23:04:56.295777+0000","last_fullsized":"2024-04-24T23:04:56.295777+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:55.860923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.289365+0000","last_change":"2024-04-24T23:04:55.632726+0000","last_active":"2024-04-24T23:04:56.289365+0000","last_peered":"2024-04-24T23:04:56.289365+0000","last_clean":"2024-04-24T23:04:56.289365+0000","last_became_active":"2024-04-24T23:04:55.632469+0000","last_became_peered":"2024-04-24T23:04:55.632469+0000","last_unstale":"2024-04-24T23:04:56.289365+0000","last_undegraded":"2024-04-24T23:04:56.289365+0000","last_fullsized":"2024-04-24T23:04:56.289365+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:33:40.335636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.298286+0000","last_change":"2024-04-24T23:04:56.296450+0000","last_active":"2024-04-24T23:04:56.298286+0000","last_peered":"2024-04-24T23:04:56.298286+0000","last_clean":"2024-04-24T23:04:56.298286+0000","last_became_active":"2024-04-24T23:04:56.295739+0000","last_became_peered":"2024-04-24T23:04:56.295739+0000","last_unstale":"2024-04-24T23:04:56.298286+0000","last_undegraded":"2024-04-24T23:04:56.298286+0000","last_fullsized":"2024-04-24T23:04:56.298286+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:51:07.700024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.300943+0000","last_change":"2024-04-24T23:04:55.641993+0000","last_active":"2024-04-24T23:04:56.300943+0000","last_peered":"2024-04-24T23:04:56.300943+0000","last_clean":"2024-04-24T23:04:56.300943+0000","last_became_active":"2024-04-24T23:04:55.641729+0000","last_became_peered":"2024-04-24T23:04:55.641729+0000","last_unstale":"2024-04-24T23:04:56.300943+0000","last_undegraded":"2024-04-24T23:04:56.300943+0000","last_fullsized":"2024-04-24T23:04:56.300943+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:14:55.485408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.301726+0000","last_change":"2024-04-24T23:04:56.301585+0000","last_active":"2024-04-24T23:04:56.301726+0000","last_peered":"2024-04-24T23:04:56.301726+0000","last_clean":"2024-04-24T23:04:56.301726+0000","last_became_active":"2024-04-24T23:04:56.301376+0000","last_became_peered":"2024-04-24T23:04:56.301376+0000","last_unstale":"2024-04-24T23:04:56.301726+0000","last_undegraded":"2024-04-24T23:04:56.301726+0000","last_fullsized":"2024-04-24T23:04:56.301726+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:40:25.671360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.289663+0000","last_change":"2024-04-24T23:04:55.634810+0000","last_active":"2024-04-24T23:04:56.289663+0000","last_peered":"2024-04-24T23:04:56.289663+0000","last_clean":"2024-04-24T23:04:56.289663+0000","last_became_active":"2024-04-24T23:04:55.634556+0000","last_became_peered":"2024-04-24T23:04:55.634556+0000","last_unstale":"2024-04-24T23:04:56.289663+0000","last_undegraded":"2024-04-24T23:04:56.289663+0000","last_fullsized":"2024-04-24T23:04:56.289663+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:12.941676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.297829+0000","last_change":"2024-04-24T23:04:56.295255+0000","last_active":"2024-04-24T23:04:56.297829+0000","last_peered":"2024-04-24T23:04:56.297829+0000","last_clean":"2024-04-24T23:04:56.297829+0000","last_became_active":"2024-04-24T23:04:56.295021+0000","last_became_peered":"2024-04-24T23:04:56.295021+0000","last_unstale":"2024-04-24T23:04:56.297829+0000","last_undegraded":"2024-04-24T23:04:56.297829+0000","last_fullsized":"2024-04-24T23:04:56.297829+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:00.086790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.297648+0000","last_change":"2024-04-24T23:04:55.639514+0000","last_active":"2024-04-24T23:04:56.297648+0000","last_peered":"2024-04-24T23:04:56.297648+0000","last_clean":"2024-04-24T23:04:56.297648+0000","last_became_active":"2024-04-24T23:04:55.638828+0000","last_became_peered":"2024-04-24T23:04:55.638828+0000","last_unstale":"2024-04-24T23:04:56.297648+0000","last_undegraded":"2024-04-24T23:04:56.297648+0000","last_fullsized":"2024-04-24T23:04:56.297648+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:49.020530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.300825+0000","last_change":"2024-04-24T23:04:56.300721+0000","last_active":"2024-04-24T23:04:56.300825+0000","last_peered":"2024-04-24T23:04:56.300825+0000","last_clean":"2024-04-24T23:04:56.300825+0000","last_became_active":"2024-04-24T23:04:56.300034+0000","last_became_peered":"2024-04-24T23:04:56.300034+0000","last_unstale":"2024-04-24T23:04:56.300825+0000","last_undegraded":"2024-04-24T23:04:56.300825+0000","last_fullsized":"2024-04-24T23:04:56.300825+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:26:08.592050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.296429+0000","last_change":"2024-04-24T23:04:55.640513+0000","last_active":"2024-04-24T23:04:56.296429+0000","last_peered":"2024-04-24T23:04:56.296429+0000","last_clean":"2024-04-24T23:04:56.296429+0000","last_became_active":"2024-04-24T23:04:55.640243+0000","last_became_peered":"2024-04-24T23:04:55.640243+0000","last_unstale":"2024-04-24T23:04:56.296429+0000","last_undegraded":"2024-04-24T23:04:56.296429+0000","last_fullsized":"2024-04-24T23:04:56.296429+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:21:49.757962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.303597+0000","last_change":"2024-04-24T23:04:56.303332+0000","last_active":"2024-04-24T23:04:56.303597+0000","last_peered":"2024-04-24T23:04:56.303597+0000","last_clean":"2024-04-24T23:04:56.303597+0000","last_became_active":"2024-04-24T23:04:56.302704+0000","last_became_peered":"2024-04-24T23:04:56.302704+0000","last_unstale":"2024-04-24T23:04:56.303597+0000","last_undegraded":"2024-04-24T23:04:56.303597+0000","last_fullsized":"2024-04-24T23:04:56.303597+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:55.614610+0000","last_clean_scrub_stamp":"2024-04-24T23:04:55.614610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:03:33.032633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"124'1","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.633214+0000","last_change":"2024-04-24T23:04:55.632584+0000","last_active":"2024-04-24T23:04:56.633214+0000","last_peered":"2024-04-24T23:04:56.633214+0000","last_clean":"2024-04-24T23:04:56.633214+0000","last_became_active":"2024-04-24T23:04:55.632285+0000","last_became_peered":"2024-04-24T23:04:55.632285+0000","last_unstale":"2024-04-24T23:04:56.633214+0000","last_undegraded":"2024-04-24T23:04:56.633214+0000","last_fullsized":"2024-04-24T23:04:56.633214+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:04:54.612789+0000","last_clean_scrub_stamp":"2024-04-24T23:04:54.612789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:23:24.718858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":375,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-24T23:04:56.294277+0000","last_change":"2024-04-24T22:58:55.138810+0000","last_active":"2024-04-24T23:04:56.294277+0000","last_peered":"2024-04-24T23:04:56.294277+0000","last_clean":"2024-04-24T23:04:56.294277+0000","last_became_active":"2024-04-24T22:58:55.138627+0000","last_became_peered":"2024-04-24T22:58:55.138627+0000","last_unstale":"2024-04-24T23:04:56.294277+0000","last_undegraded":"2024-04-24T23:04:56.294277+0000","last_fullsized":"2024-04-24T23:04:56.294277+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_clean_scrub_stamp":"2024-04-24T22:58:54.127235+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:51.383182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574923,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":662,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194650,"statfs":{"total":200000000,"available":199321942,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574923,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":668,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194643,"statfs":{"total":200000000,"available":199315167,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574923,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1246,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194065,"statfs":{"total":200000000,"available":198723131,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574923,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":668,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194643,"statfs":{"total":200000000,"available":199315167,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574923,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":668,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194643,"statfs":{"total":200000000,"available":199314973,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574923,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":668,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194643,"statfs":{"total":200000000,"available":199315097,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1238,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194073,"statfs":{"total":200000000,"available":198731574,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574923,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":668,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194643,"statfs":{"total":200000000,"available":199315167,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-24T23:05:00.648 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-24T23:05:00.648 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-24T23:05:00.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.791+0000 7f4af3a8c640 1 Processor -- start 2024-04-24T23:05:00.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.791+0000 7f4af3a8c640 1 -- start start 2024-04-24T23:05:00.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.792+0000 7f4af3a8c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4aec06ba30 0x7f4aec06d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:00.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.792+0000 7f4af3a8c640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4aec15ef10 con 0x7f4aec158dd0 2024-04-24T23:05:00.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.792+0000 7f4af3a8c640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4aec15f1f0 con 0x7f4aec06da90 2024-04-24T23:05:00.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.792+0000 7f4af3a8c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f4aec15f4d0 con 0x7f4aec06ba30 2024-04-24T23:05:00.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.792+0000 7f4af1027640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4aec06ba30 0x7f4aec06d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:00.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.792+0000 7f4af1828640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4aec158dd0 0x7f4aec06b320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56408/0 (socket says 172.21.15.32:56408) 2024-04-24T23:05:00.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.792+0000 7f4af1828640 1 -- 172.21.15.32:0/2301843388 learned_addr learned my addr 172.21.15.32:0/2301843388 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:05:00.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.793+0000 7f4af1027640 1 -- 172.21.15.32:0/2301843388 >> v1:172.21.15.121:6789/0 conn(0x7f4aec06da90 legacy=0x7f4aec0b5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:00.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.793+0000 7f4af1027640 1 -- 172.21.15.32:0/2301843388 >> v1:172.21.15.32:6789/0 conn(0x7f4aec158dd0 legacy=0x7f4aec06b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:00.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.793+0000 7f4af0826640 1 -- 172.21.15.32:0/2301843388 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1740750243 0 0) 0x7f4aec15ef10 con 0x7f4aec158dd0 2024-04-24T23:05:00.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.793+0000 7f4af1027640 1 -- 172.21.15.32:0/2301843388 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aec15f7b0 con 0x7f4aec06ba30 2024-04-24T23:05:00.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.793+0000 7f4af1027640 1 --2- 172.21.15.32:0/2301843388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4aec06ba30 0x7f4aec06d550 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f4ae80097a0 tx=0x7f4ae802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30852f432dafa76f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:00.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.794+0000 7f4af0826640 1 -- 172.21.15.32:0/2301843388 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae8037070 con 0x7f4aec06ba30 2024-04-24T23:05:00.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.794+0000 7f4af0826640 1 -- 172.21.15.32:0/2301843388 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ae8032440 con 0x7f4aec06ba30 2024-04-24T23:05:00.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.794+0000 7f4af0826640 1 -- 172.21.15.32:0/2301843388 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae8032810 con 0x7f4aec06ba30 2024-04-24T23:05:00.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.794+0000 7f4af3a8c640 1 -- 172.21.15.32:0/2301843388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4aec06ba30 msgr2=0x7f4aec06d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:00.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.794+0000 7f4af3a8c640 1 --2- 172.21.15.32:0/2301843388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4aec06ba30 0x7f4aec06d550 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f4ae80097a0 tx=0x7f4ae802a230 comp rx=0 tx=0).stop 2024-04-24T23:05:00.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.794+0000 7f4af3a8c640 1 -- 172.21.15.32:0/2301843388 shutdown_connections 2024-04-24T23:05:00.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.794+0000 7f4af3a8c640 1 --2- 172.21.15.32:0/2301843388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4aec06ba30 0x7f4aec06d550 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:00.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.794+0000 7f4af3a8c640 1 -- 172.21.15.32:0/2301843388 >> 172.21.15.32:0/2301843388 conn(0x7f4aec06e800 msgr2=0x7f4aec06ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:00.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.794+0000 7f4af3a8c640 1 -- 172.21.15.32:0/2301843388 shutdown_connections 2024-04-24T23:05:00.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.794+0000 7f4af3a8c640 1 -- 172.21.15.32:0/2301843388 wait complete. 2024-04-24T23:05:00.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.795+0000 7f4af3a8c640 1 Processor -- start 2024-04-24T23:05:00.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.795+0000 7f4af3a8c640 1 -- start start 2024-04-24T23:05:00.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.796+0000 7f4af3a8c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4aec06ba30 0x7f4aec150030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:00.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.796+0000 7f4af3a8c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4aec06da90 0x7f4aec0b1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:00.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.796+0000 7f4af3a8c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4aec158dd0 0x7f4aec0b2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:00.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.796+0000 7f4af3a8c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4aec06bfe0 con 0x7f4aec06ba30 2024-04-24T23:05:00.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.796+0000 7f4af3a8c640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4aec06ded0 con 0x7f4aec06da90 2024-04-24T23:05:00.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.796+0000 7f4af3a8c640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f4aec1591b0 con 0x7f4aec158dd0 2024-04-24T23:05:00.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.796+0000 7f4af1828640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4aec06ba30 0x7f4aec150030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:00.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.796+0000 7f4af1828640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4aec06ba30 0x7f4aec150030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52612/0 (socket says 172.21.15.32:52612) 2024-04-24T23:05:00.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.796+0000 7f4af1828640 1 -- 172.21.15.32:0/3361737302 learned_addr learned my addr 172.21.15.32:0/3361737302 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:00.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.796+0000 7f4af1027640 1 --2- 172.21.15.32:0/3361737302 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4aec06da90 0x7f4aec0b1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:00.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.796+0000 7f4af2029640 1 --2- 172.21.15.32:0/3361737302 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4aec158dd0 0x7f4aec0b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:00.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.796+0000 7f4af1828640 1 -- 172.21.15.32:0/3361737302 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4aec158dd0 msgr2=0x7f4aec0b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:00.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.796+0000 7f4af1828640 1 --2- 172.21.15.32:0/3361737302 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4aec158dd0 0x7f4aec0b2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:00.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.796+0000 7f4af1828640 1 -- 172.21.15.32:0/3361737302 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4aec06da90 msgr2=0x7f4aec0b1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:00.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.796+0000 7f4af1828640 1 --2- 172.21.15.32:0/3361737302 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4aec06da90 0x7f4aec0b1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:00.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.796+0000 7f4af1828640 1 -- 172.21.15.32:0/3361737302 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aec1f4a80 con 0x7f4aec06ba30 2024-04-24T23:05:00.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.796+0000 7f4af1027640 1 --2- 172.21.15.32:0/3361737302 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4aec06da90 0x7f4aec0b1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:05:00.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.796+0000 7f4af2029640 1 --2- 172.21.15.32:0/3361737302 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4aec158dd0 0x7f4aec0b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:05:00.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.797+0000 7f4af1828640 1 --2- 172.21.15.32:0/3361737302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4aec06ba30 0x7f4aec150030 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f4ae000c3f0 tx=0x7f4ae000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:00.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.797+0000 7f4ade7fc640 1 -- 172.21.15.32:0/3361737302 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae00074e0 con 0x7f4aec06ba30 2024-04-24T23:05:00.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.797+0000 7f4ade7fc640 1 -- 172.21.15.32:0/3361737302 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ae00047e0 con 0x7f4aec06ba30 2024-04-24T23:05:00.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.797+0000 7f4ade7fc640 1 -- 172.21.15.32:0/3361737302 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae0004c10 con 0x7f4aec06ba30 2024-04-24T23:05:00.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.797+0000 7f4af3a8c640 1 -- 172.21.15.32:0/3361737302 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4aec1f4d60 con 0x7f4aec06ba30 2024-04-24T23:05:00.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.797+0000 7f4af3a8c640 1 -- 172.21.15.32:0/3361737302 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4aec150570 con 0x7f4aec06ba30 2024-04-24T23:05:00.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.798+0000 7f4af3a8c640 1 -- 172.21.15.32:0/3361737302 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4aec068430 con 0x7f4aec06ba30 2024-04-24T23:05:00.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.798+0000 7f4ade7fc640 1 -- 172.21.15.32:0/3361737302 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4ae0011070 con 0x7f4aec06ba30 2024-04-24T23:05:00.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.799+0000 7f4ade7fc640 1 --2- 172.21.15.32:0/3361737302 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4ad006be30 0x7f4ad006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:00.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.799+0000 7f4ade7fc640 1 -- 172.21.15.32:0/3361737302 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5709+0+0 (secure 0 0 0) 0x7f4ae008aa30 con 0x7f4aec06ba30 2024-04-24T23:05:00.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.799+0000 7f4af1027640 1 --2- 172.21.15.32:0/3361737302 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4ad006be30 0x7f4ad006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:00.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.800+0000 7f4af1027640 1 --2- 172.21.15.32:0/3361737302 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4ad006be30 0x7f4ad006e2f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8034000 tx=0x7f4ae802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:00.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:00.803+0000 7f4ade7fc640 1 -- 172.21.15.32:0/3361737302 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4ae001a2b0 con 0x7f4aec06ba30 2024-04-24T23:05:01.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.062+0000 7f4af3a8c640 1 -- 172.21.15.32:0/3361737302 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f4aec150800 con 0x7f4aec06ba30 2024-04-24T23:05:01.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.064+0000 7f4ade7fc640 1 -- 172.21.15.32:0/3361737302 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) ==== 190+0+147 (secure 0 0 0) 0x7f4ae00593e0 con 0x7f4aec06ba30 2024-04-24T23:05:01.066 INFO:teuthology.orchestra.run.smithi032.stdout:[client.0] 2024-04-24T23:05:01.066 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQCejilm5oeiGRAAdLXwwqrYhV3HHo7Dw/2ZqQ== 2024-04-24T23:05:01.066 INFO:teuthology.orchestra.run.smithi032.stdout: caps mds = "allow" 2024-04-24T23:05:01.066 INFO:teuthology.orchestra.run.smithi032.stdout: caps mon = "allow r" 2024-04-24T23:05:01.066 INFO:teuthology.orchestra.run.smithi032.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-24T23:05:01.066 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2024-04-24T23:05:01.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.066+0000 7f4af3a8c640 1 -- 172.21.15.32:0/3361737302 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4ad006be30 msgr2=0x7f4ad006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:01.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.066+0000 7f4af3a8c640 1 --2- 172.21.15.32:0/3361737302 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4ad006be30 0x7f4ad006e2f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8034000 tx=0x7f4ae802e000 comp rx=0 tx=0).stop 2024-04-24T23:05:01.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.066+0000 7f4af3a8c640 1 -- 172.21.15.32:0/3361737302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4aec06ba30 msgr2=0x7f4aec150030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:01.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.066+0000 7f4af3a8c640 1 --2- 172.21.15.32:0/3361737302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4aec06ba30 0x7f4aec150030 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f4ae000c3f0 tx=0x7f4ae000c8b0 comp rx=0 tx=0).stop 2024-04-24T23:05:01.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.066+0000 7f4af3a8c640 1 -- 172.21.15.32:0/3361737302 shutdown_connections 2024-04-24T23:05:01.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.066+0000 7f4af3a8c640 1 --2- 172.21.15.32:0/3361737302 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4ad006be30 0x7f4ad006e2f0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:01.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.066+0000 7f4af3a8c640 1 --2- 172.21.15.32:0/3361737302 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4aec158dd0 0x7f4aec0b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:01.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.066+0000 7f4af3a8c640 1 --2- 172.21.15.32:0/3361737302 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4aec06da90 0x7f4aec0b1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:01.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.066+0000 7f4af3a8c640 1 --2- 172.21.15.32:0/3361737302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4aec06ba30 0x7f4aec150030 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:01.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.066+0000 7f4af3a8c640 1 -- 172.21.15.32:0/3361737302 >> 172.21.15.32:0/3361737302 conn(0x7f4aec06e800 msgr2=0x7f4aec160d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:01.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.066+0000 7f4af3a8c640 1 -- 172.21.15.32:0/3361737302 shutdown_connections 2024-04-24T23:05:01.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.066+0000 7f4af3a8c640 1 -- 172.21.15.32:0/3361737302 wait complete. 2024-04-24T23:05:01.079 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:05:01.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.178+0000 7f01c1d28640 1 Processor -- start 2024-04-24T23:05:01.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.178+0000 7f01c1d28640 1 -- start start 2024-04-24T23:05:01.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.179+0000 7f01c1d28640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f01bc074210 0x7f01bc06c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:01.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.179+0000 7f01c1d28640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f01bc0ae140 0x7f01bc0b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:01.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.179+0000 7f01c1d28640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01bc0b0770 con 0x7f01bc0b0b80 2024-04-24T23:05:01.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.179+0000 7f01c1d28640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f01bc0b1c20 con 0x7f01bc074210 2024-04-24T23:05:01.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.179+0000 7f01c1d28640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f01bc0b1010 con 0x7f01bc0ae140 2024-04-24T23:05:01.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.179+0000 7f01bb7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f01bc074210 0x7f01bc06c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:01.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.179+0000 7f01baffd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f01bc0b0b80 0x7f01bc0745f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56412/0 (socket says 172.21.15.32:56412) 2024-04-24T23:05:01.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.179+0000 7f01baffd640 1 -- 172.21.15.32:0/52552299 learned_addr learned my addr 172.21.15.32:0/52552299 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:05:01.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.179+0000 7f01bbfff640 1 --2- 172.21.15.32:0/52552299 >> v2:172.21.15.121:3300/0 conn(0x7f01bc0ae140 0x7f01bc0b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:01.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.179+0000 7f01bb7fe640 1 -- 172.21.15.32:0/52552299 >> v2:172.21.15.121:3300/0 conn(0x7f01bc0ae140 msgr2=0x7f01bc0b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:01.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.179+0000 7f01bb7fe640 1 --2- 172.21.15.32:0/52552299 >> v2:172.21.15.121:3300/0 conn(0x7f01bc0ae140 0x7f01bc0b20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:01.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.179+0000 7f01bb7fe640 1 -- 172.21.15.32:0/52552299 >> v1:172.21.15.32:6789/0 conn(0x7f01bc0b0b80 legacy=0x7f01bc0745f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:01.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.179+0000 7f01bb7fe640 1 -- 172.21.15.32:0/52552299 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01bc0b12f0 con 0x7f01bc074210 2024-04-24T23:05:01.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.179+0000 7f01ba7fc640 1 -- 172.21.15.32:0/52552299 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1599797373 0 0) 0x7f01bc0b0770 con 0x7f01bc0b0b80 2024-04-24T23:05:01.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.180+0000 7f01bb7fe640 1 --2- 172.21.15.32:0/52552299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01bc074210 0x7f01bc06c480 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f01bc072740 tx=0x7f01ac02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b425d53eef11a730 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:01.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.180+0000 7f01ba7fc640 1 -- 172.21.15.32:0/52552299 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ac037070 con 0x7f01bc074210 2024-04-24T23:05:01.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.180+0000 7f01ba7fc640 1 -- 172.21.15.32:0/52552299 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01ac032440 con 0x7f01bc074210 2024-04-24T23:05:01.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.180+0000 7f01ba7fc640 1 -- 172.21.15.32:0/52552299 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ac032810 con 0x7f01bc074210 2024-04-24T23:05:01.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.181+0000 7f01c1d28640 1 -- 172.21.15.32:0/52552299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01bc074210 msgr2=0x7f01bc06c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:01.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.181+0000 7f01c1d28640 1 --2- 172.21.15.32:0/52552299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01bc074210 0x7f01bc06c480 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f01bc072740 tx=0x7f01ac02a1a0 comp rx=0 tx=0).stop 2024-04-24T23:05:01.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.181+0000 7f01c1d28640 1 -- 172.21.15.32:0/52552299 shutdown_connections 2024-04-24T23:05:01.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.181+0000 7f01c1d28640 1 --2- 172.21.15.32:0/52552299 >> v2:172.21.15.121:3300/0 conn(0x7f01bc0ae140 0x7f01bc0b20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:01.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.181+0000 7f01c1d28640 1 --2- 172.21.15.32:0/52552299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01bc074210 0x7f01bc06c480 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:01.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.181+0000 7f01c1d28640 1 -- 172.21.15.32:0/52552299 >> 172.21.15.32:0/52552299 conn(0x7f01bc0b2e60 msgr2=0x7f01bc0b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:01.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.181+0000 7f01c1d28640 1 -- 172.21.15.32:0/52552299 shutdown_connections 2024-04-24T23:05:01.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.181+0000 7f01c1d28640 1 -- 172.21.15.32:0/52552299 wait complete. 2024-04-24T23:05:01.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.182+0000 7f01c1d28640 1 Processor -- start 2024-04-24T23:05:01.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.182+0000 7f01c1d28640 1 -- start start 2024-04-24T23:05:01.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.183+0000 7f01c1d28640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01bc074210 0x7f01bc1582a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:01.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.183+0000 7f01c1d28640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01bc0ae140 0x7f01bc0cb900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:01.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.183+0000 7f01c1d28640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01bc0c6bd0 0x7f01bc06e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:01.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.183+0000 7f01c1d28640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f01bc0b3ba0 con 0x7f01bc0c6bd0 2024-04-24T23:05:01.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.183+0000 7f01c1d28640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f01bc0b18b0 con 0x7f01bc0ae140 2024-04-24T23:05:01.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.183+0000 7f01c1d28640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f01bc0b3d20 con 0x7f01bc074210 2024-04-24T23:05:01.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.183+0000 7f01bbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01bc0c6bd0 0x7f01bc06e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:01.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.183+0000 7f01bbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01bc0c6bd0 0x7f01bc06e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52628/0 (socket says 172.21.15.32:52628) 2024-04-24T23:05:01.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.183+0000 7f01bbfff640 1 -- 172.21.15.32:0/658792319 learned_addr learned my addr 172.21.15.32:0/658792319 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:01.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.183+0000 7f01baffd640 1 --2- 172.21.15.32:0/658792319 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01bc0ae140 0x7f01bc0cb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:01.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.183+0000 7f01bb7fe640 1 --2- 172.21.15.32:0/658792319 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01bc074210 0x7f01bc1582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:01.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.183+0000 7f01bbfff640 1 -- 172.21.15.32:0/658792319 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01bc074210 msgr2=0x7f01bc1582a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:01.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.183+0000 7f01bbfff640 1 --2- 172.21.15.32:0/658792319 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01bc074210 0x7f01bc1582a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:01.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.183+0000 7f01bbfff640 1 -- 172.21.15.32:0/658792319 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01bc0ae140 msgr2=0x7f01bc0cb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:01.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.183+0000 7f01bbfff640 1 --2- 172.21.15.32:0/658792319 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01bc0ae140 0x7f01bc0cb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:01.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.183+0000 7f01bbfff640 1 -- 172.21.15.32:0/658792319 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01bc0cc130 con 0x7f01bc0c6bd0 2024-04-24T23:05:01.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.183+0000 7f01bbfff640 1 --2- 172.21.15.32:0/658792319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01bc0c6bd0 0x7f01bc06e9e0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f01b4007e70 tx=0x7f01b400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:01.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.184+0000 7f01a3fff640 1 -- 172.21.15.32:0/658792319 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b4014620 con 0x7f01bc0c6bd0 2024-04-24T23:05:01.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.184+0000 7f01a3fff640 1 -- 172.21.15.32:0/658792319 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01b4019070 con 0x7f01bc0c6bd0 2024-04-24T23:05:01.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.184+0000 7f01c1d28640 1 -- 172.21.15.32:0/658792319 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01bc0cc410 con 0x7f01bc0c6bd0 2024-04-24T23:05:01.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.184+0000 7f01a3fff640 1 -- 172.21.15.32:0/658792319 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b4014a20 con 0x7f01bc0c6bd0 2024-04-24T23:05:01.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.184+0000 7f01c1d28640 1 -- 172.21.15.32:0/658792319 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01bc0cc9d0 con 0x7f01bc0c6bd0 2024-04-24T23:05:01.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.185+0000 7f01c1d28640 1 -- 172.21.15.32:0/658792319 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01bc068430 con 0x7f01bc0c6bd0 2024-04-24T23:05:01.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.185+0000 7f01a3fff640 1 -- 172.21.15.32:0/658792319 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f01b4014bc0 con 0x7f01bc0c6bd0 2024-04-24T23:05:01.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.186+0000 7f01a3fff640 1 --2- 172.21.15.32:0/658792319 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f01a806be60 0x7f01a806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:01.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.186+0000 7f01a3fff640 1 -- 172.21.15.32:0/658792319 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5709+0+0 (secure 0 0 0) 0x7f01b408b6d0 con 0x7f01bc0c6bd0 2024-04-24T23:05:01.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.186+0000 7f01bb7fe640 1 --2- 172.21.15.32:0/658792319 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f01a806be60 0x7f01a806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:01.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.186+0000 7f01bb7fe640 1 --2- 172.21.15.32:0/658792319 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f01a806be60 0x7f01a806e320 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f01ac034000 tx=0x7f01ac02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:01.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.190+0000 7f01a3fff640 1 -- 172.21.15.32:0/658792319 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f01b405a110 con 0x7f01bc0c6bd0 2024-04-24T23:05:01.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.412+0000 7f01c1d28640 1 -- 172.21.15.32:0/658792319 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01bc06d9f0 con 0x7f01bc0c6bd0 2024-04-24T23:05:01.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.412+0000 7f01a3fff640 1 -- 172.21.15.32:0/658792319 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+5189 (secure 0 0 0) 0x7f01b4059a10 con 0x7f01bc0c6bd0 2024-04-24T23:05:01.414 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:05:01.414 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":51,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5972,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3336404028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3336404028},{"type":"v1","addr":"172.21.15.121:6837","nonce":3336404028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6505,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/1068557774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1068557774},{"type":"v1","addr":"172.21.15.121:6836","nonce":1068557774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":7353,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/2118719195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":2118719195},{"type":"v1","addr":"172.21.15.32:6834","nonce":2118719195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:04:55.998022+0000","modified":"2024-04-24T23:04:58.657258+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7356},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7356":{"gid":7356,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/3015290487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":3015290487},{"type":"v1","addr":"172.21.15.32:6835","nonce":3015290487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7356,"qdb_cluster":[7356]},"id":7}]} 2024-04-24T23:05:01.414 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 51 2024-04-24T23:05:01.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.414+0000 7f01c1d28640 1 -- 172.21.15.32:0/658792319 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f01a806be60 msgr2=0x7f01a806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:01.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.414+0000 7f01c1d28640 1 --2- 172.21.15.32:0/658792319 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f01a806be60 0x7f01a806e320 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f01ac034000 tx=0x7f01ac02e000 comp rx=0 tx=0).stop 2024-04-24T23:05:01.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.414+0000 7f01c1d28640 1 -- 172.21.15.32:0/658792319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01bc0c6bd0 msgr2=0x7f01bc06e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:01.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.414+0000 7f01c1d28640 1 --2- 172.21.15.32:0/658792319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01bc0c6bd0 0x7f01bc06e9e0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f01b4007e70 tx=0x7f01b400e4b0 comp rx=0 tx=0).stop 2024-04-24T23:05:01.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.415+0000 7f01c1d28640 1 -- 172.21.15.32:0/658792319 shutdown_connections 2024-04-24T23:05:01.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.415+0000 7f01c1d28640 1 --2- 172.21.15.32:0/658792319 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f01a806be60 0x7f01a806e320 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:01.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.415+0000 7f01c1d28640 1 --2- 172.21.15.32:0/658792319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01bc0c6bd0 0x7f01bc06e9e0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:01.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.415+0000 7f01c1d28640 1 --2- 172.21.15.32:0/658792319 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01bc0ae140 0x7f01bc0cb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:01.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.415+0000 7f01c1d28640 1 --2- 172.21.15.32:0/658792319 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01bc074210 0x7f01bc1582a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:01.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.415+0000 7f01c1d28640 1 -- 172.21.15.32:0/658792319 >> 172.21.15.32:0/658792319 conn(0x7f01bc0b2e60 msgr2=0x7f01bc0aee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:01.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.415+0000 7f01c1d28640 1 -- 172.21.15.32:0/658792319 shutdown_connections 2024-04-24T23:05:01.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.415+0000 7f01c1d28640 1 -- 172.21.15.32:0/658792319 wait complete. 2024-04-24T23:05:01.830 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 51, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T23:04:55.998022+0000', 'modified': '2024-04-24T23:04:58.657258+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7356}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7356': {'gid': 7356, 'name': 'a', 'rank': 0, 'incarnation': 48, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/3015290487', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6833', 'nonce': 3015290487}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 3015290487}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7356, 'qdb_cluster': [7356]} 2024-04-24T23:05:01.830 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-24T23:05:01.831 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-24T23:05:01.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.934+0000 7fc769fb0640 1 Processor -- start 2024-04-24T23:05:01.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.935+0000 7fc769fb0640 1 -- start start 2024-04-24T23:05:01.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.935+0000 7fc769fb0640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc7640b5560 0x7fc76415e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:01.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.935+0000 7fc769fb0640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc7640cd3c0 0x7fc76415ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:01.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.935+0000 7fc769fb0640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7640cd910 con 0x7fc7640b5940 2024-04-24T23:05:01.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.935+0000 7fc769fb0640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fc7640cdbf0 con 0x7fc7640cd3c0 2024-04-24T23:05:01.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.935+0000 7fc769fb0640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc7640cded0 con 0x7fc7640b5560 2024-04-24T23:05:01.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.936+0000 7fc762ffd640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc7640cd3c0 0x7fc76415ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:01.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.936+0000 7fc7637fe640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc7640b5560 0x7fc76415e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:01.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.936+0000 7fc762ffd640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc7640cd3c0 0x7fc76415ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.32:34406/0 (socket says 172.21.15.32:34406) 2024-04-24T23:05:01.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.936+0000 7fc762ffd640 1 -- 172.21.15.32:0/2388729192 learned_addr learned my addr 172.21.15.32:0/2388729192 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:01.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.936+0000 7fc762ffd640 1 -- 172.21.15.32:0/2388729192 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc7640b5560 msgr2=0x7fc76415e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:01.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.936+0000 7fc762ffd640 1 --2- 172.21.15.32:0/2388729192 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc7640b5560 0x7fc76415e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:01.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.936+0000 7fc762ffd640 1 -- 172.21.15.32:0/2388729192 >> v1:172.21.15.121:6789/0 conn(0x7fc7640b5940 legacy=0x7fc7640b5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:01.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.936+0000 7fc7627fc640 1 -- 172.21.15.32:0/2388729192 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1740597710 0 0) 0x7fc7640cd910 con 0x7fc7640b5940 2024-04-24T23:05:01.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.936+0000 7fc762ffd640 1 -- 172.21.15.32:0/2388729192 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7640ce1b0 con 0x7fc7640cd3c0 2024-04-24T23:05:01.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.936+0000 7fc7637fe640 1 --2- 172.21.15.32:0/2388729192 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc7640b5560 0x7fc76415e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:05:01.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.937+0000 7fc762ffd640 1 --2- 172.21.15.32:0/2388729192 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7640cd3c0 0x7fc76415ec00 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc758009870 tx=0x7fc75802a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=289b42532fe8e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:01.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.937+0000 7fc7627fc640 1 -- 172.21.15.32:0/2388729192 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc758037070 con 0x7fc7640cd3c0 2024-04-24T23:05:01.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.937+0000 7fc7627fc640 1 -- 172.21.15.32:0/2388729192 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc758032440 con 0x7fc7640cd3c0 2024-04-24T23:05:01.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.937+0000 7fc7627fc640 1 -- 172.21.15.32:0/2388729192 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc758032810 con 0x7fc7640cd3c0 2024-04-24T23:05:01.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.938+0000 7fc769fb0640 1 -- 172.21.15.32:0/2388729192 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7640cd3c0 msgr2=0x7fc76415ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:01.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.938+0000 7fc769fb0640 1 --2- 172.21.15.32:0/2388729192 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7640cd3c0 0x7fc76415ec00 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc758009870 tx=0x7fc75802a270 comp rx=0 tx=0).stop 2024-04-24T23:05:01.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.938+0000 7fc769fb0640 1 -- 172.21.15.32:0/2388729192 shutdown_connections 2024-04-24T23:05:01.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.938+0000 7fc769fb0640 1 --2- 172.21.15.32:0/2388729192 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7640cd3c0 0x7fc76415ec00 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:01.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.938+0000 7fc769fb0640 1 --2- 172.21.15.32:0/2388729192 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc7640b5560 0x7fc76415e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:01.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.938+0000 7fc769fb0640 1 -- 172.21.15.32:0/2388729192 >> 172.21.15.32:0/2388729192 conn(0x7fc76406b110 msgr2=0x7fc76406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:01.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.938+0000 7fc769fb0640 1 -- 172.21.15.32:0/2388729192 shutdown_connections 2024-04-24T23:05:01.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.938+0000 7fc769fb0640 1 -- 172.21.15.32:0/2388729192 wait complete. 2024-04-24T23:05:01.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.939+0000 7fc769fb0640 1 Processor -- start 2024-04-24T23:05:01.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.939+0000 7fc769fb0640 1 -- start start 2024-04-24T23:05:01.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.940+0000 7fc769fb0640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7640b5560 0x7fc764156cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:01.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.940+0000 7fc769fb0640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc7640b5940 0x7fc7641474b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:01.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.940+0000 7fc769fb0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7640cd3c0 0x7fc764155790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:01.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.940+0000 7fc769fb0640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc76406c770 con 0x7fc7640cd3c0 2024-04-24T23:05:01.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.940+0000 7fc769fb0640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc76406c5f0 con 0x7fc7640b5560 2024-04-24T23:05:01.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.940+0000 7fc769fb0640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc76406c8f0 con 0x7fc7640b5940 2024-04-24T23:05:01.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.940+0000 7fc763fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7640cd3c0 0x7fc764155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:01.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.940+0000 7fc763fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7640cd3c0 0x7fc764155790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52632/0 (socket says 172.21.15.32:52632) 2024-04-24T23:05:01.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.940+0000 7fc763fff640 1 -- 172.21.15.32:0/1006319712 learned_addr learned my addr 172.21.15.32:0/1006319712 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:01.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.940+0000 7fc762ffd640 1 --2- 172.21.15.32:0/1006319712 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc7640b5940 0x7fc7641474b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:01.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.940+0000 7fc7637fe640 1 --2- 172.21.15.32:0/1006319712 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7640b5560 0x7fc764156cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:01.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.940+0000 7fc763fff640 1 -- 172.21.15.32:0/1006319712 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc7640b5940 msgr2=0x7fc7641474b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:01.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.940+0000 7fc763fff640 1 --2- 172.21.15.32:0/1006319712 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc7640b5940 0x7fc7641474b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:01.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.940+0000 7fc763fff640 1 -- 172.21.15.32:0/1006319712 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7640b5560 msgr2=0x7fc764156cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:01.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.940+0000 7fc763fff640 1 --2- 172.21.15.32:0/1006319712 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7640b5560 0x7fc764156cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:01.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.940+0000 7fc763fff640 1 -- 172.21.15.32:0/1006319712 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc764147db0 con 0x7fc7640cd3c0 2024-04-24T23:05:01.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.940+0000 7fc763fff640 1 --2- 172.21.15.32:0/1006319712 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7640cd3c0 0x7fc764155790 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fc75c00c760 tx=0x7fc75c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:01.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.941+0000 7fc74bfff640 1 -- 172.21.15.32:0/1006319712 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75c003da0 con 0x7fc7640cd3c0 2024-04-24T23:05:01.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.941+0000 7fc74bfff640 1 -- 172.21.15.32:0/1006319712 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc75c005070 con 0x7fc7640cd3c0 2024-04-24T23:05:01.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.941+0000 7fc769fb0640 1 -- 172.21.15.32:0/1006319712 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc764148090 con 0x7fc7640cd3c0 2024-04-24T23:05:01.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.941+0000 7fc74bfff640 1 -- 172.21.15.32:0/1006319712 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75c005460 con 0x7fc7640cd3c0 2024-04-24T23:05:01.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.941+0000 7fc769fb0640 1 -- 172.21.15.32:0/1006319712 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc764148650 con 0x7fc7640cd3c0 2024-04-24T23:05:01.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.942+0000 7fc74bfff640 1 -- 172.21.15.32:0/1006319712 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc75c005600 con 0x7fc7640cd3c0 2024-04-24T23:05:01.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.943+0000 7fc74bfff640 1 --2- 172.21.15.32:0/1006319712 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc75006be60 0x7fc75006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:01.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.943+0000 7fc74bfff640 1 -- 172.21.15.32:0/1006319712 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5709+0+0 (secure 0 0 0) 0x7fc75c08ad90 con 0x7fc7640cd3c0 2024-04-24T23:05:01.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.943+0000 7fc7637fe640 1 --2- 172.21.15.32:0/1006319712 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc75006be60 0x7fc75006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:01.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.944+0000 7fc76cd9e740 1 Processor -- start 2024-04-24T23:05:01.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.944+0000 7fc76cd9e740 1 -- start start 2024-04-24T23:05:01.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.944+0000 7fc7637fe640 1 --2- 172.21.15.32:0/1006319712 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc75006be60 0x7fc75006e320 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fc754009560 tx=0x7fc754008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:01.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.944+0000 7fc76cd9e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce57d7eb60 0x55ce57d314f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:01.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.944+0000 7fc76cd9e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ce57d61c80 0x55ce57d61000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:01.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.944+0000 7fc76cd9e740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ce57d61540 0x55ce57d36350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:01.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.944+0000 7fc76cd9e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ce57c99f00 con 0x55ce57d7eb60 2024-04-24T23:05:01.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.944+0000 7fc76cd9e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55ce57c42750 con 0x55ce57d61c80 2024-04-24T23:05:01.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.944+0000 7fc76cd9e740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x55ce57c95ec0 con 0x55ce57d61540 2024-04-24T23:05:01.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.945+0000 7fc7637fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce57d7eb60 0x55ce57d314f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:01.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.945+0000 7fc7637fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce57d7eb60 0x55ce57d314f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52646/0 (socket says 172.21.15.32:52646) 2024-04-24T23:05:01.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.945+0000 7fc7637fe640 1 -- 172.21.15.32:0/944304506 learned_addr learned my addr 172.21.15.32:0/944304506 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:01.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.945+0000 7fc763fff640 1 --2- 172.21.15.32:0/944304506 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ce57d61540 0x55ce57d36350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:01.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.945+0000 7fc762ffd640 1 --2- 172.21.15.32:0/944304506 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ce57d61c80 0x55ce57d61000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:01.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.945+0000 7fc7637fe640 1 -- 172.21.15.32:0/944304506 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ce57d61540 msgr2=0x55ce57d36350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:01.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.945+0000 7fc7637fe640 1 --2- 172.21.15.32:0/944304506 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ce57d61540 0x55ce57d36350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:01.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.945+0000 7fc7637fe640 1 -- 172.21.15.32:0/944304506 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ce57d61c80 msgr2=0x55ce57d61000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:01.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.945+0000 7fc7637fe640 1 --2- 172.21.15.32:0/944304506 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ce57d61c80 0x55ce57d61000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:01.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.945+0000 7fc7637fe640 1 -- 172.21.15.32:0/944304506 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce579ae2d0 con 0x55ce57d7eb60 2024-04-24T23:05:01.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.945+0000 7fc7637fe640 1 --2- 172.21.15.32:0/944304506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce57d7eb60 0x55ce57d314f0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fc754011df0 tx=0x7fc754014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=612fbd9f85da2e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:01.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.945+0000 7fc7497fa640 1 -- 172.21.15.32:0/944304506 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc754011e20 con 0x55ce57d7eb60 2024-04-24T23:05:01.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.945+0000 7fc7497fa640 1 -- 172.21.15.32:0/944304506 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc754026070 con 0x55ce57d7eb60 2024-04-24T23:05:01.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.945+0000 7fc7497fa640 1 -- 172.21.15.32:0/944304506 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc754021640 con 0x55ce57d7eb60 2024-04-24T23:05:01.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.946+0000 7fc76cd9e740 1 -- 172.21.15.32:0/944304506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce57d7eb60 msgr2=0x55ce57d314f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:01.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.946+0000 7fc76cd9e740 1 --2- 172.21.15.32:0/944304506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce57d7eb60 0x55ce57d314f0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fc754011df0 tx=0x7fc754014000 comp rx=0 tx=0).stop 2024-04-24T23:05:01.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.946+0000 7fc76cd9e740 1 -- 172.21.15.32:0/944304506 shutdown_connections 2024-04-24T23:05:01.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.946+0000 7fc76cd9e740 1 --2- 172.21.15.32:0/944304506 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ce57d61540 0x55ce57d36350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:01.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.946+0000 7fc76cd9e740 1 --2- 172.21.15.32:0/944304506 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ce57d61c80 0x55ce57d61000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:01.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.946+0000 7fc76cd9e740 1 --2- 172.21.15.32:0/944304506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce57d7eb60 0x55ce57d314f0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:01.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.946+0000 7fc76cd9e740 1 -- 172.21.15.32:0/944304506 >> 172.21.15.32:0/944304506 conn(0x55ce57d33760 msgr2=0x55ce57d7c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:01.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.946+0000 7fc76cd9e740 1 -- 172.21.15.32:0/944304506 shutdown_connections 2024-04-24T23:05:01.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.946+0000 7fc76cd9e740 1 -- 172.21.15.32:0/944304506 wait complete. 2024-04-24T23:05:01.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.947+0000 7fc76cd9e740 1 -- start start 2024-04-24T23:05:01.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.947+0000 7fc76cd9e740 1 Processor -- start 2024-04-24T23:05:01.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.947+0000 7fc76cd9e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ce57d61c80 0x55ce57d73cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:01.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.947+0000 7fc76cd9e740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ce57d7eb60 0x55ce57d63eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:01.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.947+0000 7fc76cd9e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce57d4eac0 0x55ce57d4f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:01.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.947+0000 7fc76cd9e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ce57d35910 con 0x55ce57d4eac0 2024-04-24T23:05:01.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.947+0000 7fc76cd9e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55ce57c55cf0 con 0x55ce57d61c80 2024-04-24T23:05:01.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.947+0000 7fc76cd9e740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x55ce57d34d00 con 0x55ce57d7eb60 2024-04-24T23:05:01.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.948+0000 7fc76cd9e740 10 client.0 _ll_register_callbacks cb 0x55ce57c556e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:05:01.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.948+0000 7fc763fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce57d4eac0 0x55ce57d4f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:01.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.948+0000 7fc763fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce57d4eac0 0x55ce57d4f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52662/0 (socket says 172.21.15.32:52662) 2024-04-24T23:05:01.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.948+0000 7fc763fff640 1 -- 172.21.15.32:0/3759699196 learned_addr learned my addr 172.21.15.32:0/3759699196 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:01.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.948+0000 7fc762ffd640 1 --2- 172.21.15.32:0/3759699196 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ce57d7eb60 0x55ce57d63eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:01.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.948+0000 7fc7637fe640 1 --2- 172.21.15.32:0/3759699196 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ce57d61c80 0x55ce57d73cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:01.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.948+0000 7fc763fff640 1 -- 172.21.15.32:0/3759699196 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ce57d7eb60 msgr2=0x55ce57d63eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:01.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.948+0000 7fc763fff640 1 --2- 172.21.15.32:0/3759699196 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ce57d7eb60 0x55ce57d63eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:01.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.948+0000 7fc763fff640 1 -- 172.21.15.32:0/3759699196 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ce57d61c80 msgr2=0x55ce57d73cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:01.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.948+0000 7fc763fff640 1 --2- 172.21.15.32:0/3759699196 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ce57d61c80 0x55ce57d73cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:01.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.948+0000 7fc763fff640 1 -- 172.21.15.32:0/3759699196 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce57a786b0 con 0x55ce57d4eac0 2024-04-24T23:05:01.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.948+0000 7fc763fff640 1 --2- 172.21.15.32:0/3759699196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce57d4eac0 0x55ce57d4f030 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fc75c0892c0 tx=0x7fc75c059750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c25ec8feb056655 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:01.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.948+0000 7fc7427fc640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-24T23:05:01.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.948+0000 7fc7427fc640 1 -- 172.21.15.32:0/3759699196 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75c071070 con 0x55ce57d4eac0 2024-04-24T23:05:01.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.948+0000 7fc7427fc640 1 -- 172.21.15.32:0/3759699196 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 5709+0+0 (secure 0 0 0) 0x7fc75c05d0f0 con 0x55ce57d4eac0 2024-04-24T23:05:01.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.948+0000 7fc76cd9e740 1 -- 172.21.15.32:0/3759699196 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc754026070 con 0x55ce57d4eac0 2024-04-24T23:05:01.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.948+0000 7fc7427fc640 1 -- 172.21.15.32:0/3759699196 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fc75c05d400 con 0x55ce57d4eac0 2024-04-24T23:05:01.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.948+0000 7fc76cd9e740 10 client.7497 fetch_fsmap learned FSMap version 51 2024-04-24T23:05:01.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.948+0000 7fc76cd9e740 1 -- 172.21.15.32:0/3759699196 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x55ce57d734d0 con 0x55ce57d4eac0 2024-04-24T23:05:01.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.949+0000 7fc7427fc640 1 -- 172.21.15.32:0/3759699196 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc754026070 con 0x55ce57d4eac0 2024-04-24T23:05:01.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.949+0000 7fc7427fc640 1 -- 172.21.15.32:0/3759699196 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7fc75c0556c0 con 0x55ce57d4eac0 2024-04-24T23:05:01.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.949+0000 7fc76cd9e740 10 client.7497 fetch_fsmap finished waiting for FSMap version 51 2024-04-24T23:05:01.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.949+0000 7fc76cd9e740 10 client.7497 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T23:05:01.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.949+0000 7fc76cd9e740 20 client.7497 populate_metadata read hostname 'smithi032' 2024-04-24T23:05:01.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.949+0000 7fc76cd9e740 1 --2- 172.21.15.32:0/3759699196 >> [v2:172.21.15.32:6833/3015290487,v1:172.21.15.32:6835/3015290487] conn(0x55ce57d62620 0x55ce57d62a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:01.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.949+0000 7fc76cd9e740 4 client.7497 mds_command: new command op to 7356 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:05:01.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.949+0000 7fc76cd9e740 1 -- 172.21.15.32:0/3759699196 --> [v2:172.21.15.32:6833/3015290487,v1:172.21.15.32:6835/3015290487] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce57ca7440 con 0x55ce57d62620 2024-04-24T23:05:01.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.949+0000 7fc7637fe640 1 --2- 172.21.15.32:0/3759699196 >> [v2:172.21.15.32:6833/3015290487,v1:172.21.15.32:6835/3015290487] conn(0x55ce57d62620 0x55ce57d62a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:01.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.949+0000 7fc7637fe640 1 --2- 172.21.15.32:0/3759699196 >> [v2:172.21.15.32:6833/3015290487,v1:172.21.15.32:6835/3015290487] conn(0x55ce57d62620 0x55ce57d62a00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d5284a593bd64d8 server_cookie=6b2d74b33ddd0aec in_seq=0 out_seq=0 2024-04-24T23:05:01.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.949+0000 7fc7427fc640 10 client.7497 ms_handle_connect on v2:172.21.15.32:6833/3015290487 2024-04-24T23:05:01.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.950+0000 7fc7427fc640 1 -- 172.21.15.32:0/3759699196 <== mds.0 v2:172.21.15.32:6833/3015290487 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ce57ca7440 con 0x55ce57d62620 2024-04-24T23:05:01.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.950+0000 7fc7427fc640 10 client.7497 handle_command_reply: tid=0 2024-04-24T23:05:01.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.950+0000 7fc76cd9e740 1 client.7497 shutdown 2024-04-24T23:05:01.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.950+0000 7fc76cd9e740 1 -- 172.21.15.32:0/3759699196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce57d4eac0 msgr2=0x55ce57d4f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:01.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.950+0000 7fc76cd9e740 1 --2- 172.21.15.32:0/3759699196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce57d4eac0 0x55ce57d4f030 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fc75c0892c0 tx=0x7fc75c059750 comp rx=0 tx=0).stop 2024-04-24T23:05:01.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.950+0000 7fc76cd9e740 1 -- 172.21.15.32:0/3759699196 shutdown_connections 2024-04-24T23:05:01.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.950+0000 7fc76cd9e740 1 --2- 172.21.15.32:0/3759699196 >> [v2:172.21.15.32:6833/3015290487,v1:172.21.15.32:6835/3015290487] conn(0x55ce57d62620 0x55ce57d62a00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:01.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.950+0000 7fc76cd9e740 1 --2- 172.21.15.32:0/3759699196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce57d4eac0 0x55ce57d4f030 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:01.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.950+0000 7fc76cd9e740 1 --2- 172.21.15.32:0/3759699196 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ce57d7eb60 0x55ce57d63eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:01.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.950+0000 7fc76cd9e740 1 --2- 172.21.15.32:0/3759699196 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ce57d61c80 0x55ce57d73cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:01.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.950+0000 7fc7427fc640 0 client.7497 ms_handle_reset on v2:172.21.15.32:6833/3015290487 2024-04-24T23:05:01.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.950+0000 7fc76cd9e740 1 -- 172.21.15.32:0/3759699196 >> 172.21.15.32:0/3759699196 conn(0x55ce57d61540 msgr2=0x55ce57d7e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:01.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.950+0000 7fc76cd9e740 1 -- 172.21.15.32:0/3759699196 shutdown_connections 2024-04-24T23:05:01.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.950+0000 7fc76cd9e740 1 -- 172.21.15.32:0/3759699196 wait complete. 2024-04-24T23:05:01.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.950+0000 7fc76cd9e740 20 client.7497 trim_cache size 0 max 16384 2024-04-24T23:05:01.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.968+0000 7fc76cd9e740 1 Processor -- start 2024-04-24T23:05:01.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.968+0000 7fc76cd9e740 1 -- start start 2024-04-24T23:05:01.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.969+0000 7fc76cd9e740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ce57d4eac0 0x55ce57d65010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:01.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.969+0000 7fc76cd9e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ce57d61c80 0x55ce57d5bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:01.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.969+0000 7fc76cd9e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce57d7eb60 0x55ce57d5c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:01.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.969+0000 7fc76cd9e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc764151b20 con 0x55ce57d7eb60 2024-04-24T23:05:01.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.969+0000 7fc76cd9e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc7641f42b0 con 0x55ce57d61c80 2024-04-24T23:05:01.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.969+0000 7fc76cd9e740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc7641514d0 con 0x55ce57d4eac0 2024-04-24T23:05:01.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.969+0000 7fc763fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce57d7eb60 0x55ce57d5c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:01.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.969+0000 7fc763fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce57d7eb60 0x55ce57d5c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52678/0 (socket says 172.21.15.32:52678) 2024-04-24T23:05:01.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.969+0000 7fc763fff640 1 -- 172.21.15.32:0/3058667570 learned_addr learned my addr 172.21.15.32:0/3058667570 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:01.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.970+0000 7fc762ffd640 1 --2- 172.21.15.32:0/3058667570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ce57d61c80 0x55ce57d5bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:01.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.970+0000 7fc7637fe640 1 --2- 172.21.15.32:0/3058667570 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ce57d4eac0 0x55ce57d65010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:01.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.970+0000 7fc763fff640 1 -- 172.21.15.32:0/3058667570 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ce57d4eac0 msgr2=0x55ce57d65010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:01.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.970+0000 7fc763fff640 1 --2- 172.21.15.32:0/3058667570 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ce57d4eac0 0x55ce57d65010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:01.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.970+0000 7fc763fff640 1 -- 172.21.15.32:0/3058667570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ce57d61c80 msgr2=0x55ce57d5bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:01.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.970+0000 7fc763fff640 1 --2- 172.21.15.32:0/3058667570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ce57d61c80 0x55ce57d5bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:01.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.970+0000 7fc763fff640 1 -- 172.21.15.32:0/3058667570 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce57d348a0 con 0x55ce57d7eb60 2024-04-24T23:05:01.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.970+0000 7fc762ffd640 1 --2- 172.21.15.32:0/3058667570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ce57d61c80 0x55ce57d5bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:05:01.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.970+0000 7fc763fff640 1 --2- 172.21.15.32:0/3058667570 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce57d7eb60 0x55ce57d5c4f0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fc75c058d50 tx=0x7fc75c06e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1cf44f8f6392526 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:01.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.970+0000 7fc748ff9640 1 -- 172.21.15.32:0/3058667570 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75c071040 con 0x55ce57d7eb60 2024-04-24T23:05:01.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.970+0000 7fc748ff9640 1 -- 172.21.15.32:0/3058667570 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc75c061c30 con 0x55ce57d7eb60 2024-04-24T23:05:01.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.970+0000 7fc748ff9640 1 -- 172.21.15.32:0/3058667570 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75c0556c0 con 0x55ce57d7eb60 2024-04-24T23:05:01.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.971+0000 7fc76cd9e740 1 -- 172.21.15.32:0/3058667570 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce57d7eb60 msgr2=0x55ce57d5c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:01.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.971+0000 7fc76cd9e740 1 --2- 172.21.15.32:0/3058667570 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce57d7eb60 0x55ce57d5c4f0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fc75c058d50 tx=0x7fc75c06e000 comp rx=0 tx=0).stop 2024-04-24T23:05:01.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.971+0000 7fc76cd9e740 1 -- 172.21.15.32:0/3058667570 shutdown_connections 2024-04-24T23:05:01.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.971+0000 7fc76cd9e740 1 --2- 172.21.15.32:0/3058667570 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce57d7eb60 0x55ce57d5c4f0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:01.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.971+0000 7fc76cd9e740 1 --2- 172.21.15.32:0/3058667570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ce57d61c80 0x55ce57d5bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:01.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.971+0000 7fc76cd9e740 1 --2- 172.21.15.32:0/3058667570 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ce57d4eac0 0x55ce57d65010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:01.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.971+0000 7fc76cd9e740 1 -- 172.21.15.32:0/3058667570 >> 172.21.15.32:0/3058667570 conn(0x55ce57d61540 msgr2=0x55ce57d2c690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:01.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.971+0000 7fc76cd9e740 1 -- 172.21.15.32:0/3058667570 shutdown_connections 2024-04-24T23:05:01.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.971+0000 7fc76cd9e740 1 -- 172.21.15.32:0/3058667570 wait complete. 2024-04-24T23:05:01.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.971+0000 7fc76cd9e740 1 -- start start 2024-04-24T23:05:01.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.971+0000 7fc76cd9e740 1 Processor -- start 2024-04-24T23:05:01.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.972+0000 7fc76cd9e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ce57d4eac0 0x55ce57d31bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:01.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.972+0000 7fc76cd9e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce57d61c80 0x55ce57d9e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:01.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.972+0000 7fc76cd9e740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ce57d7eb60 0x55ce57d356a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:01.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.972+0000 7fc76cd9e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ce57d4f7a0 con 0x55ce57d61c80 2024-04-24T23:05:01.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.972+0000 7fc76cd9e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55ce57d4fb50 con 0x55ce57d4eac0 2024-04-24T23:05:01.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.972+0000 7fc76cd9e740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x55ce57d30530 con 0x55ce57d7eb60 2024-04-24T23:05:01.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.972+0000 7fc76cd9e740 10 client.0 _ll_register_callbacks cb 0x7fc75c058b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:05:01.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.972+0000 7fc762ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce57d61c80 0x55ce57d9e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:01.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.972+0000 7fc762ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce57d61c80 0x55ce57d9e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52686/0 (socket says 172.21.15.32:52686) 2024-04-24T23:05:01.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.972+0000 7fc762ffd640 1 -- 172.21.15.32:0/3114996730 learned_addr learned my addr 172.21.15.32:0/3114996730 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:01.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.972+0000 7fc7637fe640 1 --2- 172.21.15.32:0/3114996730 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ce57d4eac0 0x55ce57d31bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:01.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.972+0000 7fc763fff640 1 --2- 172.21.15.32:0/3114996730 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ce57d7eb60 0x55ce57d356a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:01.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.972+0000 7fc762ffd640 1 -- 172.21.15.32:0/3114996730 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ce57d7eb60 msgr2=0x55ce57d356a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:01.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.972+0000 7fc762ffd640 1 --2- 172.21.15.32:0/3114996730 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ce57d7eb60 0x55ce57d356a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:01.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.972+0000 7fc762ffd640 1 -- 172.21.15.32:0/3114996730 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ce57d4eac0 msgr2=0x55ce57d31bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:01.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.972+0000 7fc762ffd640 1 --2- 172.21.15.32:0/3114996730 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ce57d4eac0 0x55ce57d31bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:01.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.972+0000 7fc762ffd640 1 -- 172.21.15.32:0/3114996730 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce57d36ca0 con 0x55ce57d61c80 2024-04-24T23:05:01.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.973+0000 7fc762ffd640 1 --2- 172.21.15.32:0/3114996730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce57d61c80 0x55ce57d9e740 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fc7580090b0 tx=0x7fc758004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=679481ed6f6bc9a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:01.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.973+0000 7fc7427fc640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-24T23:05:01.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.973+0000 7fc7427fc640 1 -- 172.21.15.32:0/3114996730 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75804c060 con 0x55ce57d61c80 2024-04-24T23:05:01.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.973+0000 7fc7427fc640 1 -- 172.21.15.32:0/3114996730 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 5709+0+0 (secure 0 0 0) 0x7fc75802adc0 con 0x55ce57d61c80 2024-04-24T23:05:01.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.973+0000 7fc76cd9e740 1 -- 172.21.15.32:0/3114996730 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc75c061c30 con 0x55ce57d61c80 2024-04-24T23:05:01.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.973+0000 7fc7427fc640 1 -- 172.21.15.32:0/3114996730 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fc758052050 con 0x55ce57d61c80 2024-04-24T23:05:01.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.973+0000 7fc76cd9e740 10 client.7503 fetch_fsmap learned FSMap version 51 2024-04-24T23:05:01.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.973+0000 7fc76cd9e740 1 -- 172.21.15.32:0/3114996730 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x55ce57d37590 con 0x55ce57d61c80 2024-04-24T23:05:01.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.973+0000 7fc7427fc640 1 -- 172.21.15.32:0/3114996730 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc75c061c30 con 0x55ce57d61c80 2024-04-24T23:05:01.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.973+0000 7fc7427fc640 1 -- 172.21.15.32:0/3114996730 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7fc758055020 con 0x55ce57d61c80 2024-04-24T23:05:01.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.973+0000 7fc76cd9e740 10 client.7503 fetch_fsmap finished waiting for FSMap version 51 2024-04-24T23:05:01.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.973+0000 7fc76cd9e740 10 client.7503 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T23:05:01.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.973+0000 7fc76cd9e740 20 client.7503 populate_metadata read hostname 'smithi032' 2024-04-24T23:05:01.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.973+0000 7fc76cd9e740 1 --2- 172.21.15.32:0/3114996730 >> [v2:172.21.15.32:6833/3015290487,v1:172.21.15.32:6835/3015290487] conn(0x55ce57d5c7a0 0x55ce57d7a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:01.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.973+0000 7fc76cd9e740 4 client.7503 mds_command: new command op to 7356 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:05:01.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.973+0000 7fc76cd9e740 1 -- 172.21.15.32:0/3114996730 --> [v2:172.21.15.32:6833/3015290487,v1:172.21.15.32:6835/3015290487] -- command(tid 0: {"prefix": "status"}) -- 0x55ce57ca1790 con 0x55ce57d5c7a0 2024-04-24T23:05:01.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.973+0000 7fc7637fe640 1 --2- 172.21.15.32:0/3114996730 >> [v2:172.21.15.32:6833/3015290487,v1:172.21.15.32:6835/3015290487] conn(0x55ce57d5c7a0 0x55ce57d7a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:01.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.974+0000 7fc7427fc640 10 client.7503 ms_handle_connect on v2:172.21.15.32:6833/3015290487 2024-04-24T23:05:01.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.974+0000 7fc7637fe640 1 --2- 172.21.15.32:0/3114996730 >> [v2:172.21.15.32:6833/3015290487,v1:172.21.15.32:6835/3015290487] conn(0x55ce57d5c7a0 0x55ce57d7a480 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64572c759cf4a9a2 server_cookie=f2ac43124c21a5e8 in_seq=0 out_seq=0 2024-04-24T23:05:01.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.974+0000 7fc7427fc640 1 -- 172.21.15.32:0/3114996730 <== mds.0 v2:172.21.15.32:6833/3015290487 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55ce57ca1790 con 0x55ce57d5c7a0 2024-04-24T23:05:01.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.974+0000 7fc7427fc640 10 client.7503 handle_command_reply: tid=0 2024-04-24T23:05:01.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.974+0000 7fc76cd9e740 1 client.7503 shutdown 2024-04-24T23:05:01.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.974+0000 7fc76cd9e740 1 -- 172.21.15.32:0/3114996730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce57d61c80 msgr2=0x55ce57d9e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:01.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.974+0000 7fc76cd9e740 1 --2- 172.21.15.32:0/3114996730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce57d61c80 0x55ce57d9e740 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fc7580090b0 tx=0x7fc758004730 comp rx=0 tx=0).stop 2024-04-24T23:05:01.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.974+0000 7fc76cd9e740 1 -- 172.21.15.32:0/3114996730 shutdown_connections 2024-04-24T23:05:01.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.974+0000 7fc76cd9e740 1 --2- 172.21.15.32:0/3114996730 >> [v2:172.21.15.32:6833/3015290487,v1:172.21.15.32:6835/3015290487] conn(0x55ce57d5c7a0 0x55ce57d7a480 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:01.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.974+0000 7fc76cd9e740 1 --2- 172.21.15.32:0/3114996730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce57d61c80 0x55ce57d9e740 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:01.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.974+0000 7fc76cd9e740 1 --2- 172.21.15.32:0/3114996730 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ce57d7eb60 0x55ce57d356a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:01.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.974+0000 7fc7427fc640 0 client.7503 ms_handle_reset on v2:172.21.15.32:6833/3015290487 2024-04-24T23:05:01.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.974+0000 7fc76cd9e740 1 --2- 172.21.15.32:0/3114996730 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ce57d4eac0 0x55ce57d31bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:01.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.974+0000 7fc76cd9e740 1 -- 172.21.15.32:0/3114996730 >> 172.21.15.32:0/3114996730 conn(0x55ce57d61540 msgr2=0x55ce57d2c690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:01.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.974+0000 7fc76cd9e740 1 -- 172.21.15.32:0/3114996730 shutdown_connections 2024-04-24T23:05:01.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.974+0000 7fc76cd9e740 1 -- 172.21.15.32:0/3114996730 wait complete. 2024-04-24T23:05:01.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.974+0000 7fc76cd9e740 20 client.7503 trim_cache size 0 max 16384 2024-04-24T23:05:01.982 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:05:01.982 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", 2024-04-24T23:05:01.982 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2024-04-24T23:05:01.982 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 7356, 2024-04-24T23:05:01.982 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-04-24T23:05:01.982 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-04-24T23:05:01.983 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-04-24T23:05:01.983 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 5.3498203249999996, 2024-04-24T23:05:01.983 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 51, 2024-04-24T23:05:01.983 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 125, 2024-04-24T23:05:01.983 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 125, 2024-04-24T23:05:01.983 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 10.68723559 2024-04-24T23:05:01.983 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:05:01.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.975+0000 7fc769fb0640 1 -- 172.21.15.32:0/1006319712 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc75006be60 msgr2=0x7fc75006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:01.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.975+0000 7fc769fb0640 1 --2- 172.21.15.32:0/1006319712 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc75006be60 0x7fc75006e320 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fc754009560 tx=0x7fc754008040 comp rx=0 tx=0).stop 2024-04-24T23:05:01.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.975+0000 7fc769fb0640 1 -- 172.21.15.32:0/1006319712 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7640cd3c0 msgr2=0x7fc764155790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:01.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.975+0000 7fc769fb0640 1 --2- 172.21.15.32:0/1006319712 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7640cd3c0 0x7fc764155790 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fc75c00c760 tx=0x7fc75c00cc20 comp rx=0 tx=0).stop 2024-04-24T23:05:01.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.975+0000 7fc769fb0640 1 -- 172.21.15.32:0/1006319712 shutdown_connections 2024-04-24T23:05:01.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.975+0000 7fc769fb0640 1 --2- 172.21.15.32:0/1006319712 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc75006be60 0x7fc75006e320 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:01.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.975+0000 7fc769fb0640 1 --2- 172.21.15.32:0/1006319712 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7640cd3c0 0x7fc764155790 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:01.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.975+0000 7fc769fb0640 1 --2- 172.21.15.32:0/1006319712 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc7640b5940 0x7fc7641474b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:01.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.975+0000 7fc769fb0640 1 --2- 172.21.15.32:0/1006319712 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7640b5560 0x7fc764156cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:01.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.975+0000 7fc769fb0640 1 -- 172.21.15.32:0/1006319712 >> 172.21.15.32:0/1006319712 conn(0x7fc76406b110 msgr2=0x7fc7641496c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:01.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.975+0000 7fc769fb0640 1 -- 172.21.15.32:0/1006319712 shutdown_connections 2024-04-24T23:05:01.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:01.975+0000 7fc769fb0640 1 -- 172.21.15.32:0/1006319712 wait complete. 2024-04-24T23:05:01.987 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T23:05:02.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.086+0000 7f50ad0ba640 1 Processor -- start 2024-04-24T23:05:02.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.086+0000 7f50ad0ba640 1 -- start start 2024-04-24T23:05:02.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.087+0000 7f50ad0ba640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f50a8158dd0 0x7f50a806b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:02.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.087+0000 7f50ad0ba640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f50a806b860 0x7f50a806d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:02.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.087+0000 7f50ad0ba640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f50a806da90 0x7f50a8072580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:02.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.087+0000 7f50ad0ba640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f50a815ef10 con 0x7f50a8158dd0 2024-04-24T23:05:02.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.087+0000 7f50ad0ba640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f50a815f1f0 con 0x7f50a806da90 2024-04-24T23:05:02.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.087+0000 7f50ad0ba640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f50a815f4d0 con 0x7f50a806b860 2024-04-24T23:05:02.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.087+0000 7f50a6d9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f50a8158dd0 0x7f50a806b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:02.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.087+0000 7f50a6d9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f50a8158dd0 0x7f50a806b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52696/0 (socket says 172.21.15.32:52696) 2024-04-24T23:05:02.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.087+0000 7f50a6d9d640 1 -- 172.21.15.32:0/3604268020 learned_addr learned my addr 172.21.15.32:0/3604268020 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:02.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.087+0000 7f509ffff640 1 --2- 172.21.15.32:0/3604268020 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f50a806b860 0x7f50a806d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:02.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.087+0000 7f50a6d9d640 1 -- 172.21.15.32:0/3604268020 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f50a806b860 msgr2=0x7f50a806d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:02.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.087+0000 7f50a6d9d640 1 --2- 172.21.15.32:0/3604268020 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f50a806b860 0x7f50a806d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:02.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.087+0000 7f50a6d9d640 1 -- 172.21.15.32:0/3604268020 >> v2:172.21.15.121:3300/0 conn(0x7f50a806da90 msgr2=0x7f50a8072580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:02.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.087+0000 7f50a759e640 1 --2- 172.21.15.32:0/3604268020 >> v2:172.21.15.121:3300/0 conn(0x7f50a806da90 0x7f50a8072580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:02.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.087+0000 7f50a6d9d640 1 --2- 172.21.15.32:0/3604268020 >> v2:172.21.15.121:3300/0 conn(0x7f50a806da90 0x7f50a8072580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:02.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.087+0000 7f50a6d9d640 1 -- 172.21.15.32:0/3604268020 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50a815f7b0 con 0x7f50a8158dd0 2024-04-24T23:05:02.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.087+0000 7f50a759e640 1 --2- 172.21.15.32:0/3604268020 >> v2:172.21.15.121:3300/0 conn(0x7f50a806da90 0x7f50a8072580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:05:02.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.088+0000 7f50a6d9d640 1 --2- 172.21.15.32:0/3604268020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50a8158dd0 0x7f50a806b320 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f50a8162320 tx=0x7f509802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=584ec761d6d036c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:02.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.088+0000 7f50a659c640 1 -- 172.21.15.32:0/3604268020 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5098037070 con 0x7f50a8158dd0 2024-04-24T23:05:02.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.088+0000 7f50a659c640 1 -- 172.21.15.32:0/3604268020 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5098032440 con 0x7f50a8158dd0 2024-04-24T23:05:02.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.088+0000 7f50a659c640 1 -- 172.21.15.32:0/3604268020 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5098032810 con 0x7f50a8158dd0 2024-04-24T23:05:02.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.089+0000 7f50ad0ba640 1 -- 172.21.15.32:0/3604268020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50a8158dd0 msgr2=0x7f50a806b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:02.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.089+0000 7f50ad0ba640 1 --2- 172.21.15.32:0/3604268020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50a8158dd0 0x7f50a806b320 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f50a8162320 tx=0x7f509802a1e0 comp rx=0 tx=0).stop 2024-04-24T23:05:02.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.089+0000 7f50ad0ba640 1 -- 172.21.15.32:0/3604268020 shutdown_connections 2024-04-24T23:05:02.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.089+0000 7f50ad0ba640 1 --2- 172.21.15.32:0/3604268020 >> v2:172.21.15.121:3300/0 conn(0x7f50a806da90 0x7f50a8072580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:02.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.089+0000 7f50ad0ba640 1 --2- 172.21.15.32:0/3604268020 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f50a806b860 0x7f50a806d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:02.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.089+0000 7f50ad0ba640 1 --2- 172.21.15.32:0/3604268020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50a8158dd0 0x7f50a806b320 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:02.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.089+0000 7f50ad0ba640 1 -- 172.21.15.32:0/3604268020 >> 172.21.15.32:0/3604268020 conn(0x7f50a806e800 msgr2=0x7f50a806ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:02.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.089+0000 7f50ad0ba640 1 -- 172.21.15.32:0/3604268020 shutdown_connections 2024-04-24T23:05:02.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.089+0000 7f50ad0ba640 1 -- 172.21.15.32:0/3604268020 wait complete. 2024-04-24T23:05:02.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.090+0000 7f50ad0ba640 1 Processor -- start 2024-04-24T23:05:02.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.090+0000 7f50ad0ba640 1 -- start start 2024-04-24T23:05:02.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.091+0000 7f50ad0ba640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50a806b860 0x7f50a8156bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:02.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.091+0000 7f50ad0ba640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f50a806da90 0x7f50a815b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:02.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.091+0000 7f50ad0ba640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50a8158dd0 0x7f50a81f4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:02.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.091+0000 7f50ad0ba640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f50a806ded0 con 0x7f50a806b860 2024-04-24T23:05:02.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.091+0000 7f50ad0ba640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f50a815fcd0 con 0x7f50a8158dd0 2024-04-24T23:05:02.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.091+0000 7f50ad0ba640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f50a806bfe0 con 0x7f50a806da90 2024-04-24T23:05:02.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.091+0000 7f50a6d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50a806b860 0x7f50a8156bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:02.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.091+0000 7f50a6d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50a806b860 0x7f50a8156bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52706/0 (socket says 172.21.15.32:52706) 2024-04-24T23:05:02.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.091+0000 7f50a6d9d640 1 -- 172.21.15.32:0/2663131428 learned_addr learned my addr 172.21.15.32:0/2663131428 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:02.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.091+0000 7f509ffff640 1 --2- 172.21.15.32:0/2663131428 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f50a806da90 0x7f50a815b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:02.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.091+0000 7f50a759e640 1 --2- 172.21.15.32:0/2663131428 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50a8158dd0 0x7f50a81f4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:02.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.091+0000 7f50a6d9d640 1 -- 172.21.15.32:0/2663131428 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f50a806da90 msgr2=0x7f50a815b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:02.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.091+0000 7f50a6d9d640 1 --2- 172.21.15.32:0/2663131428 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f50a806da90 0x7f50a815b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:02.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.091+0000 7f50a6d9d640 1 -- 172.21.15.32:0/2663131428 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50a8158dd0 msgr2=0x7f50a81f4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:02.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.091+0000 7f50a6d9d640 1 --2- 172.21.15.32:0/2663131428 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50a8158dd0 0x7f50a81f4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:02.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.091+0000 7f50a6d9d640 1 -- 172.21.15.32:0/2663131428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50a81563f0 con 0x7f50a806b860 2024-04-24T23:05:02.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.091+0000 7f50a6d9d640 1 --2- 172.21.15.32:0/2663131428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50a806b860 0x7f50a8156bd0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f509802a220 tx=0x7f50980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:02.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.091+0000 7f509f7fe640 1 -- 172.21.15.32:0/2663131428 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5098037040 con 0x7f50a806b860 2024-04-24T23:05:02.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.091+0000 7f509f7fe640 1 -- 172.21.15.32:0/2663131428 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5098042070 con 0x7f50a806b860 2024-04-24T23:05:02.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.091+0000 7f50ad0ba640 1 -- 172.21.15.32:0/2663131428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50a81f4d30 con 0x7f50a806b860 2024-04-24T23:05:02.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.092+0000 7f509f7fe640 1 -- 172.21.15.32:0/2663131428 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5098030bb0 con 0x7f50a806b860 2024-04-24T23:05:02.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.092+0000 7f50ad0ba640 1 -- 172.21.15.32:0/2663131428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50a81566b0 con 0x7f50a806b860 2024-04-24T23:05:02.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.093+0000 7f50ad0ba640 1 -- 172.21.15.32:0/2663131428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50a8068430 con 0x7f50a806b860 2024-04-24T23:05:02.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.093+0000 7f509f7fe640 1 -- 172.21.15.32:0/2663131428 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5098047460 con 0x7f50a806b860 2024-04-24T23:05:02.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.093+0000 7f509f7fe640 1 --2- 172.21.15.32:0/2663131428 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f508806be30 0x7f508806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:02.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.094+0000 7f509f7fe640 1 -- 172.21.15.32:0/2663131428 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5709+0+0 (secure 0 0 0) 0x7f50980ab960 con 0x7f50a806b860 2024-04-24T23:05:02.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.094+0000 7f509ffff640 1 --2- 172.21.15.32:0/2663131428 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f508806be30 0x7f508806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:02.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.094+0000 7f509ffff640 1 --2- 172.21.15.32:0/2663131428 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f508806be30 0x7f508806e2f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f50a814ef30 tx=0x7f5090008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:02.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.098+0000 7f509f7fe640 1 -- 172.21.15.32:0/2663131428 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5098079cc0 con 0x7f50a806b860 2024-04-24T23:05:02.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.321+0000 7f50ad0ba640 1 -- 172.21.15.32:0/2663131428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f50a81610f0 con 0x7f50a806b860 2024-04-24T23:05:02.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.321+0000 7f509f7fe640 1 -- 172.21.15.32:0/2663131428 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v51) ==== 53+0+73 (secure 0 0 0) 0x7f509807dfc0 con 0x7f50a806b860 2024-04-24T23:05:02.323 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T23:05:02.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.323+0000 7f50ad0ba640 1 -- 172.21.15.32:0/2663131428 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f508806be30 msgr2=0x7f508806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:02.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.323+0000 7f50ad0ba640 1 --2- 172.21.15.32:0/2663131428 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f508806be30 0x7f508806e2f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f50a814ef30 tx=0x7f5090008040 comp rx=0 tx=0).stop 2024-04-24T23:05:02.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.323+0000 7f50ad0ba640 1 -- 172.21.15.32:0/2663131428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50a806b860 msgr2=0x7f50a8156bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:02.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.323+0000 7f50ad0ba640 1 --2- 172.21.15.32:0/2663131428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50a806b860 0x7f50a8156bd0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f509802a220 tx=0x7f50980046c0 comp rx=0 tx=0).stop 2024-04-24T23:05:02.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.323+0000 7f50ad0ba640 1 -- 172.21.15.32:0/2663131428 shutdown_connections 2024-04-24T23:05:02.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.323+0000 7f50ad0ba640 1 --2- 172.21.15.32:0/2663131428 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f508806be30 0x7f508806e2f0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:02.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.323+0000 7f50ad0ba640 1 --2- 172.21.15.32:0/2663131428 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50a8158dd0 0x7f50a81f4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:02.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.323+0000 7f50ad0ba640 1 --2- 172.21.15.32:0/2663131428 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f50a806da90 0x7f50a815b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:02.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.323+0000 7f50ad0ba640 1 --2- 172.21.15.32:0/2663131428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50a806b860 0x7f50a8156bd0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:02.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.323+0000 7f50ad0ba640 1 -- 172.21.15.32:0/2663131428 >> 172.21.15.32:0/2663131428 conn(0x7f50a806e800 msgr2=0x7f50a80b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:02.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.323+0000 7f50ad0ba640 1 -- 172.21.15.32:0/2663131428 shutdown_connections 2024-04-24T23:05:02.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02.323+0000 7f50ad0ba640 1 -- 172.21.15.32:0/2663131428 wait complete. 2024-04-24T23:05:02.336 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-24T23:05:02.336 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T23:05:02.336 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2024-04-24T23:05:02.336 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-24T23:05:02.336 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T23:05:02.336 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-24T23:05:02.336 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-24T23:05:02.336 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T23:05:02.336 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:05:02.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:05:02.337 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2024-04-24T23:05:02.357 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T23:05:02.357 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T23:05:02.357 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T23:05:02.357 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:05:02.357 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2024-04-24T23:05:02.358 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:05:02.358 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:05:02.358 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2024-04-24T23:05:02.358 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2024-04-24T23:05:02.358 INFO:teuthology.orchestra.run.smithi032.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:05:02.358 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2024-04-24T23:05:02.358 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2024-04-24T23:05:02.358 INFO:teuthology.orchestra.run.smithi032.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T23:05:02.358 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:05:02.358 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2024-04-24T23:05:02.359 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-24T23:05:02.359 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 42406sec preferred_lft 42406sec 2024-04-24T23:05:02.359 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link noprefixroute 2024-04-24T23:05:02.359 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:05:02.359 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:05:02.359 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:05:02.359 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2024-04-24T23:05:02.359 INFO:teuthology.orchestra.run.smithi032.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T23:05:02.359 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 72:d4:96:00:43:8c brd ff:ff:ff:ff:ff:ff 2024-04-24T23:05:02.359 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-24T23:05:02.360 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:05:02.360 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::f4c6:20ff:fe22:2647/64 scope link 2024-04-24T23:05:02.360 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:05:02.360 INFO:teuthology.orchestra.run.smithi032.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:05:02.360 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 72:d4:96:00:43:8c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T23:05:02.360 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::70d4:96ff:fe00:438c/64 scope link 2024-04-24T23:05:02.360 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:05:02.360 INFO:teuthology.orchestra.run.smithi032.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:05:02.360 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether e6:6b:2a:90:1a:f8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T23:05:02.360 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::e46b:2aff:fe90:1af8/64 scope link 2024-04-24T23:05:02.361 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:05:02.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:05:02.361 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T23:05:02.423 INFO:teuthology.orchestra.run.smithi032.stdout:1 2024-04-24T23:05:02.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:05:02.424 DEBUG:teuthology.orchestra.run.smithi032:> route 2024-04-24T23:05:02.481 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2024-04-24T23:05:02.481 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T23:05:02.481 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-24T23:05:02.481 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-24T23:05:02.481 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T23:05:02.481 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:05:02.481 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-24T23:05:02.481 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-24T23:05:02.481 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-24T23:05:02.482 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-24T23:05:02.482 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-24T23:05:02.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:05:02.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:05:02.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:05:02.608 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-04-24T23:05:02.624 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-24T23:05:02.624 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-24T23:05:02.624 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:05:02.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:05:02.624 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2024-04-24T23:05:02.690 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:05:02.690 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-24T23:05:02.746 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:05:02.746 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:05:02.746 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-24T23:05:02.801 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-24T23:05:02.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:05:02.801 DEBUG:teuthology.orchestra.run.smithi032:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:05:02.883 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:05:02.883 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-24T23:05:02.890 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:05:02.890 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-24T23:05:02.890 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.121:3300,172.21.15.121:3301" 2024-04-24T23:05:02.891 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: trying mount with new device syntax: 0@eb1b7a01-6f65-4534-9a12-83011862ba51.cephfs=/ 2024-04-24T23:05:02.891 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.32:3300/172.21.15.121:3300/172.21.15.121:3301" will pass to kernel 2024-04-24T23:05:02.891 INFO:teuthology.orchestra.run.smithi032.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-24T23:05:02.891 INFO:teuthology.orchestra.run.smithi032.stdout: You just mounted a file system that supports labels which does not 2024-04-24T23:05:02.891 INFO:teuthology.orchestra.run.smithi032.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-24T23:05:02.891 INFO:teuthology.orchestra.run.smithi032.stdout: applications will generate AVC messages and not be allowed access to 2024-04-24T23:05:02.891 INFO:teuthology.orchestra.run.smithi032.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-24T23:05:02.892 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-24T23:05:02.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:05:02.893 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-24T23:05:02.959 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:05:03.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:05:03.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:05:03.064 INFO:teuthology.orchestra.run.smithi032.stdout:client6047 2024-04-24T23:05:03.064 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:05:03.064 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:05:03.079 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T23:05:03.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.219+0000 7fd717929640 1 Processor -- start 2024-04-24T23:05:03.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.219+0000 7fd717929640 1 -- start start 2024-04-24T23:05:03.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.220+0000 7fd717929640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd7100b5af0 0x7fd7100ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:03.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.220+0000 7fd717929640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7100cd270 con 0x7fd710162320 2024-04-24T23:05:03.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.220+0000 7fd717929640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd710071610 con 0x7fd710162e10 2024-04-24T23:05:03.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.220+0000 7fd717929640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fd7100b5660 con 0x7fd7100b5af0 2024-04-24T23:05:03.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.220+0000 7fd7156c5640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd710162320 0x7fd710162700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56422/0 (socket says 172.21.15.32:56422) 2024-04-24T23:05:03.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.220+0000 7fd7156c5640 1 -- 172.21.15.32:0/3421961416 learned_addr learned my addr 172.21.15.32:0/3421961416 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:05:03.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.220+0000 7fd714ec4640 1 --2- 172.21.15.32:0/3421961416 >> v2:172.21.15.121:3300/0 conn(0x7fd7100b5af0 0x7fd7100ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:03.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.220+0000 7fd707fff640 1 -- 172.21.15.32:0/3421961416 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2771864821 0 0) 0x7fd7100cd270 con 0x7fd710162320 2024-04-24T23:05:03.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.220+0000 7fd707fff640 1 -- 172.21.15.32:0/3421961416 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6fc003520 con 0x7fd710162320 2024-04-24T23:05:03.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.221+0000 7fd707fff640 1 -- 172.21.15.32:0/3421961416 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 439154357 0 0) 0x7fd6fc003520 con 0x7fd710162320 2024-04-24T23:05:03.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.221+0000 7fd707fff640 1 -- 172.21.15.32:0/3421961416 >> v2:172.21.15.121:3300/0 conn(0x7fd7100b5af0 msgr2=0x7fd7100ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:03.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.221+0000 7fd707fff640 1 --2- 172.21.15.32:0/3421961416 >> v2:172.21.15.121:3300/0 conn(0x7fd7100b5af0 0x7fd7100ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:03.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.221+0000 7fd707fff640 1 -- 172.21.15.32:0/3421961416 >> v1:172.21.15.121:6789/0 conn(0x7fd710162e10 legacy=0x7fd71015ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:03.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.221+0000 7fd707fff640 1 -- 172.21.15.32:0/3421961416 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7100718f0 con 0x7fd710162320 2024-04-24T23:05:03.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.221+0000 7fd707fff640 1 -- 172.21.15.32:0/3421961416 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7fd708001670 con 0x7fd710162320 2024-04-24T23:05:03.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.221+0000 7fd707fff640 1 -- 172.21.15.32:0/3421961416 >> v1:172.21.15.32:6789/0 conn(0x7fd710162320 legacy=0x7fd710162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:03.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.221+0000 7fd707fff640 1 --2- 172.21.15.32:0/3421961416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6fc0039b0 0x7fd6fc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:03.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.221+0000 7fd707fff640 1 --2- 172.21.15.32:0/3421961416 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6fc004800 0x7fd6fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:03.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.221+0000 7fd707fff640 1 --2- 172.21.15.32:0/3421961416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6fc0051c0 0x7fd6fc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:03.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.221+0000 7fd707fff640 1 -- 172.21.15.32:0/3421961416 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd7100b5660 con 0x7fd6fc0039b0 2024-04-24T23:05:03.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.221+0000 7fd707fff640 1 -- 172.21.15.32:0/3421961416 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd6fc00c160 con 0x7fd6fc0051c0 2024-04-24T23:05:03.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.221+0000 7fd707fff640 1 -- 172.21.15.32:0/3421961416 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd6fc00c360 con 0x7fd6fc004800 2024-04-24T23:05:03.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.221+0000 7fd7156c5640 1 --2- 172.21.15.32:0/3421961416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6fc0039b0 0x7fd6fc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:03.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.221+0000 7fd715ec6640 1 --2- 172.21.15.32:0/3421961416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6fc0051c0 0x7fd6fc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:03.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.221+0000 7fd714ec4640 1 --2- 172.21.15.32:0/3421961416 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6fc004800 0x7fd6fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:03.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.221+0000 7fd7156c5640 1 -- 172.21.15.32:0/3421961416 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6fc004800 msgr2=0x7fd6fc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:03.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.221+0000 7fd7156c5640 1 --2- 172.21.15.32:0/3421961416 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6fc004800 0x7fd6fc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:03.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.221+0000 7fd7156c5640 1 -- 172.21.15.32:0/3421961416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6fc0051c0 msgr2=0x7fd6fc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:03.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.221+0000 7fd7156c5640 1 --2- 172.21.15.32:0/3421961416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6fc0051c0 0x7fd6fc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:03.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.221+0000 7fd7156c5640 1 -- 172.21.15.32:0/3421961416 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6fc003520 con 0x7fd6fc0039b0 2024-04-24T23:05:03.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.221+0000 7fd715ec6640 1 --2- 172.21.15.32:0/3421961416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6fc0051c0 0x7fd6fc00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:05:03.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.221+0000 7fd714ec4640 1 --2- 172.21.15.32:0/3421961416 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6fc004800 0x7fd6fc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:05:03.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.221+0000 7fd715ec6640 1 -- 172.21.15.32:0/3421961416 reap_dead start 2024-04-24T23:05:03.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.222+0000 7fd7156c5640 1 --2- 172.21.15.32:0/3421961416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6fc0039b0 0x7fd6fc003dd0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fd70800c210 tx=0x7fd6fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9de15a47f48ce9b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:03.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.222+0000 7fd707fff640 1 -- 172.21.15.32:0/3421961416 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd708007d30 con 0x7fd6fc0039b0 2024-04-24T23:05:03.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.222+0000 7fd707fff640 1 -- 172.21.15.32:0/3421961416 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd708005ad0 con 0x7fd6fc0039b0 2024-04-24T23:05:03.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.222+0000 7fd707fff640 1 -- 172.21.15.32:0/3421961416 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd708007070 con 0x7fd6fc0039b0 2024-04-24T23:05:03.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.223+0000 7fd717929640 1 -- 172.21.15.32:0/3421961416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6fc0039b0 msgr2=0x7fd6fc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:03.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.223+0000 7fd717929640 1 --2- 172.21.15.32:0/3421961416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6fc0039b0 0x7fd6fc003dd0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fd70800c210 tx=0x7fd6fc005830 comp rx=0 tx=0).stop 2024-04-24T23:05:03.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.223+0000 7fd717929640 1 -- 172.21.15.32:0/3421961416 shutdown_connections 2024-04-24T23:05:03.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.223+0000 7fd717929640 1 --2- 172.21.15.32:0/3421961416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6fc0039b0 0x7fd6fc003dd0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:03.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.223+0000 7fd717929640 1 -- 172.21.15.32:0/3421961416 >> 172.21.15.32:0/3421961416 conn(0x7fd7100d0ae0 msgr2=0x7fd7100d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:03.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.223+0000 7fd717929640 1 -- 172.21.15.32:0/3421961416 shutdown_connections 2024-04-24T23:05:03.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.223+0000 7fd717929640 1 -- 172.21.15.32:0/3421961416 wait complete. 2024-04-24T23:05:03.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.224+0000 7fd717929640 1 Processor -- start 2024-04-24T23:05:03.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.224+0000 7fd717929640 1 -- start start 2024-04-24T23:05:03.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.225+0000 7fd717929640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6fc0039b0 0x7fd710150690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:03.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.225+0000 7fd717929640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7101f92a0 0x7fd7101f5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:03.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.225+0000 7fd717929640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7100cdcb0 0x7fd7100ce140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:03.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.225+0000 7fd717929640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd710071bd0 con 0x7fd7101f92a0 2024-04-24T23:05:03.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.225+0000 7fd717929640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd710067950 con 0x7fd6fc0039b0 2024-04-24T23:05:03.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.225+0000 7fd717929640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd710071d50 con 0x7fd7100cdcb0 2024-04-24T23:05:03.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.225+0000 7fd714ec4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7101f92a0 0x7fd7101f5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:03.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.225+0000 7fd714ec4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7101f92a0 0x7fd7101f5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52720/0 (socket says 172.21.15.32:52720) 2024-04-24T23:05:03.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.225+0000 7fd714ec4640 1 -- 172.21.15.32:0/2145354907 learned_addr learned my addr 172.21.15.32:0/2145354907 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:03.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.225+0000 7fd7156c5640 1 --2- 172.21.15.32:0/2145354907 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6fc0039b0 0x7fd710150690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:03.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.225+0000 7fd715ec6640 1 --2- 172.21.15.32:0/2145354907 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7100cdcb0 0x7fd7100ce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:03.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.225+0000 7fd714ec4640 1 -- 172.21.15.32:0/2145354907 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7100cdcb0 msgr2=0x7fd7100ce140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:03.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.225+0000 7fd714ec4640 1 --2- 172.21.15.32:0/2145354907 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7100cdcb0 0x7fd7100ce140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:03.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.225+0000 7fd714ec4640 1 -- 172.21.15.32:0/2145354907 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6fc0039b0 msgr2=0x7fd710150690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:03.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.225+0000 7fd714ec4640 1 --2- 172.21.15.32:0/2145354907 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6fc0039b0 0x7fd710150690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:03.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.225+0000 7fd714ec4640 1 -- 172.21.15.32:0/2145354907 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7100cd4f0 con 0x7fd7101f92a0 2024-04-24T23:05:03.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.225+0000 7fd715ec6640 1 --2- 172.21.15.32:0/2145354907 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7100cdcb0 0x7fd7100ce140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:05:03.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.225+0000 7fd7156c5640 1 --2- 172.21.15.32:0/2145354907 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6fc0039b0 0x7fd710150690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:05:03.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.225+0000 7fd714ec4640 1 --2- 172.21.15.32:0/2145354907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7101f92a0 0x7fd7101f5030 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fd700011510 tx=0x7fd7000119d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:03.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.225+0000 7fd705ffb640 1 -- 172.21.15.32:0/2145354907 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd700015870 con 0x7fd7101f92a0 2024-04-24T23:05:03.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.225+0000 7fd705ffb640 1 -- 172.21.15.32:0/2145354907 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd700015a10 con 0x7fd7101f92a0 2024-04-24T23:05:03.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.225+0000 7fd717929640 1 -- 172.21.15.32:0/2145354907 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd710150bd0 con 0x7fd7101f92a0 2024-04-24T23:05:03.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.225+0000 7fd705ffb640 1 -- 172.21.15.32:0/2145354907 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd700015e00 con 0x7fd7101f92a0 2024-04-24T23:05:03.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.226+0000 7fd717929640 1 -- 172.21.15.32:0/2145354907 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7101f5c10 con 0x7fd7101f92a0 2024-04-24T23:05:03.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.227+0000 7fd717929640 1 -- 172.21.15.32:0/2145354907 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd71015c190 con 0x7fd7101f92a0 2024-04-24T23:05:03.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.227+0000 7fd705ffb640 1 -- 172.21.15.32:0/2145354907 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd70001d400 con 0x7fd7101f92a0 2024-04-24T23:05:03.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.228+0000 7fd705ffb640 1 --2- 172.21.15.32:0/2145354907 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd6f406be60 0x7fd6f406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:03.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.228+0000 7fd705ffb640 1 -- 172.21.15.32:0/2145354907 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5709+0+0 (secure 0 0 0) 0x7fd70008b5a0 con 0x7fd7101f92a0 2024-04-24T23:05:03.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.228+0000 7fd7156c5640 1 --2- 172.21.15.32:0/2145354907 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd6f406be60 0x7fd6f406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:03.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.228+0000 7fd7156c5640 1 --2- 172.21.15.32:0/2145354907 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd6f406be60 0x7fd6f406e320 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd708033000 tx=0x7fd70802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:03.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.232+0000 7fd705ffb640 1 -- 172.21.15.32:0/2145354907 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd71015c190 con 0x7fd7101f92a0 2024-04-24T23:05:03.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.456+0000 7fd717929640 1 -- 172.21.15.32:0/2145354907 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd710068430 con 0x7fd7101f92a0 2024-04-24T23:05:03.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.456+0000 7fd705ffb640 1 -- 172.21.15.32:0/2145354907 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v51) ==== 53+0+73 (secure 0 0 0) 0x7fd70005ee20 con 0x7fd7101f92a0 2024-04-24T23:05:03.458 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T23:05:03.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.458+0000 7fd717929640 1 -- 172.21.15.32:0/2145354907 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd6f406be60 msgr2=0x7fd6f406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:03.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.458+0000 7fd717929640 1 --2- 172.21.15.32:0/2145354907 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd6f406be60 0x7fd6f406e320 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd708033000 tx=0x7fd70802e000 comp rx=0 tx=0).stop 2024-04-24T23:05:03.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.458+0000 7fd717929640 1 -- 172.21.15.32:0/2145354907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7101f92a0 msgr2=0x7fd7101f5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:03.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.458+0000 7fd717929640 1 --2- 172.21.15.32:0/2145354907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7101f92a0 0x7fd7101f5030 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fd700011510 tx=0x7fd7000119d0 comp rx=0 tx=0).stop 2024-04-24T23:05:03.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.458+0000 7fd717929640 1 -- 172.21.15.32:0/2145354907 shutdown_connections 2024-04-24T23:05:03.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.458+0000 7fd717929640 1 --2- 172.21.15.32:0/2145354907 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd6f406be60 0x7fd6f406e320 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:03.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.458+0000 7fd717929640 1 --2- 172.21.15.32:0/2145354907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7101f92a0 0x7fd7101f5030 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:03.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.458+0000 7fd717929640 1 --2- 172.21.15.32:0/2145354907 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7100cdcb0 0x7fd7100ce140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:03.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.458+0000 7fd717929640 1 --2- 172.21.15.32:0/2145354907 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6fc0039b0 0x7fd710150690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:03.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.458+0000 7fd717929640 1 -- 172.21.15.32:0/2145354907 >> 172.21.15.32:0/2145354907 conn(0x7fd7100d0ae0 msgr2=0x7fd71015afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:03.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.459+0000 7fd717929640 1 -- 172.21.15.32:0/2145354907 shutdown_connections 2024-04-24T23:05:03.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:03.459+0000 7fd717929640 1 -- 172.21.15.32:0/2145354907 wait complete. 2024-04-24T23:05:03.471 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-24T23:05:03.471 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T23:05:03.471 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi121.front.sepia.ceph.com 2024-04-24T23:05:03.471 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-24T23:05:03.471 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-24T23:05:03.471 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-24T23:05:03.471 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-24T23:05:03.471 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T23:05:03.471 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-24T23:05:03.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:05:03.471 DEBUG:teuthology.orchestra.run.smithi121:> ip addr 2024-04-24T23:05:03.491 INFO:teuthology.orchestra.run.smithi121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T23:05:03.491 INFO:teuthology.orchestra.run.smithi121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T23:05:03.491 INFO:teuthology.orchestra.run.smithi121.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T23:05:03.491 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:05:03.491 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 ::1/128 scope host 2024-04-24T23:05:03.491 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:05:03.491 INFO:teuthology.orchestra.run.smithi121.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:05:03.491 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:82 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:05:03.491 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f0 2024-04-24T23:05:03.491 INFO:teuthology.orchestra.run.smithi121.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:05:03.491 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:83 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:05:03.491 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f1 2024-04-24T23:05:03.492 INFO:teuthology.orchestra.run.smithi121.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:05:03.492 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a2 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:05:03.492 INFO:teuthology.orchestra.run.smithi121.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T23:05:03.492 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a3 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:05:03.492 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.21.15.121/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-24T23:05:03.492 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft 42428sec preferred_lft 42428sec 2024-04-24T23:05:03.492 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::93dd:8489:4369:e127/64 scope link noprefixroute 2024-04-24T23:05:03.492 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:05:03.492 INFO:teuthology.orchestra.run.smithi121.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T23:05:03.492 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether c6:c0:c7:45:ae:ba brd ff:ff:ff:ff:ff:ff 2024-04-24T23:05:03.492 INFO:teuthology.orchestra.run.smithi121.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-24T23:05:03.492 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:05:03.492 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::80fe:d2ff:fe13:d1/64 scope link 2024-04-24T23:05:03.492 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:05:03.492 INFO:teuthology.orchestra.run.smithi121.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:05:03.493 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether e2:ca:b9:e9:9e:bb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T23:05:03.493 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::e0ca:b9ff:fee9:9ebb/64 scope link 2024-04-24T23:05:03.493 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:05:03.493 INFO:teuthology.orchestra.run.smithi121.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:05:03.493 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether c6:c0:c7:45:ae:ba brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-24T23:05:03.493 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::c4c0:c7ff:fe45:aeba/64 scope link 2024-04-24T23:05:03.493 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:05:03.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:05:03.493 DEBUG:teuthology.orchestra.run.smithi121:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T23:05:03.558 INFO:teuthology.orchestra.run.smithi121.stdout:1 2024-04-24T23:05:03.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:05:03.560 DEBUG:teuthology.orchestra.run.smithi121:> route 2024-04-24T23:05:03.617 INFO:teuthology.orchestra.run.smithi121.stdout:Kernel IP routing table 2024-04-24T23:05:03.617 INFO:teuthology.orchestra.run.smithi121.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T23:05:03.617 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-24T23:05:03.617 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-24T23:05:03.618 INFO:teuthology.orchestra.run.smithi121.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T23:05:03.618 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:05:03.618 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2024-04-24T23:05:03.618 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-24T23:05:03.618 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-24T23:05:03.618 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-24T23:05:03.618 DEBUG:teuthology.orchestra.run.smithi121:> ') 2024-04-24T23:05:03.709 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T23:05:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:05:03.739 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T23:05:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:05:03.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:05:03.746 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2024-04-24T23:05:03.760 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-24T23:05:03.760 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-24T23:05:03.760 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-24T23:05:03.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:05:03.761 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set brx.0 up 2024-04-24T23:05:03.823 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:05:03.824 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-24T23:05:03.879 INFO:teuthology.orchestra.run.smithi121.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-24T23:05:03.879 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:05:03.879 DEBUG:teuthology.orchestra.run.smithi121:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-24T23:05:03.933 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-24T23:05:03.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:05:03.933 DEBUG:teuthology.orchestra.run.smithi121:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:05:04.014 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:05:04.014 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-24T23:05:04.022 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:05:04.022 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-24T23:05:04.022 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.121:3300,172.21.15.121:3301" 2024-04-24T23:05:04.022 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: trying mount with new device syntax: 1@eb1b7a01-6f65-4534-9a12-83011862ba51.cephfs=/ 2024-04-24T23:05:04.022 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.32:3300/172.21.15.121:3300/172.21.15.121:3301" will pass to kernel 2024-04-24T23:05:04.022 INFO:teuthology.orchestra.run.smithi121.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-24T23:05:04.023 INFO:teuthology.orchestra.run.smithi121.stdout: You just mounted a file system that supports labels which does not 2024-04-24T23:05:04.023 INFO:teuthology.orchestra.run.smithi121.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-24T23:05:04.023 INFO:teuthology.orchestra.run.smithi121.stdout: applications will generate AVC messages and not be allowed access to 2024-04-24T23:05:04.023 INFO:teuthology.orchestra.run.smithi121.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-24T23:05:04.024 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-24T23:05:04.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:05:04.024 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-24T23:05:04.086 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:05:04.174 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T23:05:04 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:05:04.176 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T23:05:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:05:04.181 INFO:teuthology.orchestra.run.smithi121.stdout:client7518 2024-04-24T23:05:04.182 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:05:04.182 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:05:04.196 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:05:04.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.307+0000 7f237bc58640 1 Processor -- start 2024-04-24T23:05:04.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.307+0000 7f237bc58640 1 -- start start 2024-04-24T23:05:04.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.308+0000 7f237bc58640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2374162e10 0x7f23740ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:04.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.308+0000 7f237bc58640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23740cd270 con 0x7f2374162320 2024-04-24T23:05:04.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.308+0000 7f237bc58640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2374071610 con 0x7f23740b5af0 2024-04-24T23:05:04.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.308+0000 7f237bc58640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f23740b5660 con 0x7f2374162e10 2024-04-24T23:05:04.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.308+0000 7f23799f4640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2374162320 0x7f2374162700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56426/0 (socket says 172.21.15.32:56426) 2024-04-24T23:05:04.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.308+0000 7f23799f4640 1 -- 172.21.15.32:0/1792973841 learned_addr learned my addr 172.21.15.32:0/1792973841 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:05:04.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.308+0000 7f237a1f5640 1 --2- 172.21.15.32:0/1792973841 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2374162e10 0x7f23740ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:04.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.309+0000 7f237a1f5640 1 -- 172.21.15.32:0/1792973841 >> v1:172.21.15.121:6789/0 conn(0x7f23740b5af0 legacy=0x7f237415ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:04.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.309+0000 7f237a1f5640 1 -- 172.21.15.32:0/1792973841 >> v1:172.21.15.32:6789/0 conn(0x7f2374162320 legacy=0x7f2374162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:04.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.309+0000 7f237a1f5640 1 -- 172.21.15.32:0/1792973841 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23740718f0 con 0x7f2374162e10 2024-04-24T23:05:04.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.309+0000 7f237a1f5640 1 --2- 172.21.15.32:0/1792973841 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2374162e10 0x7f23740ccc90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f236c009dc0 tx=0x7f236c02a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42451bd40538d1db server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:04.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.311+0000 7f23789f2640 1 -- 172.21.15.32:0/1792973841 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f236c02ae60 con 0x7f2374162e10 2024-04-24T23:05:04.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.311+0000 7f23789f2640 1 -- 172.21.15.32:0/1792973841 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f236c007740 con 0x7f2374162e10 2024-04-24T23:05:04.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.311+0000 7f23789f2640 1 -- 172.21.15.32:0/1792973841 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f236c007b10 con 0x7f2374162e10 2024-04-24T23:05:04.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.311+0000 7f237bc58640 1 -- 172.21.15.32:0/1792973841 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2374162e10 msgr2=0x7f23740ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:04.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.311+0000 7f237bc58640 1 --2- 172.21.15.32:0/1792973841 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2374162e10 0x7f23740ccc90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f236c009dc0 tx=0x7f236c02a640 comp rx=0 tx=0).stop 2024-04-24T23:05:04.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.312+0000 7f237bc58640 1 -- 172.21.15.32:0/1792973841 shutdown_connections 2024-04-24T23:05:04.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.312+0000 7f237bc58640 1 --2- 172.21.15.32:0/1792973841 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2374162e10 0x7f23740ccc90 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:04.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.312+0000 7f237bc58640 1 -- 172.21.15.32:0/1792973841 >> 172.21.15.32:0/1792973841 conn(0x7f23740d0ae0 msgr2=0x7f23740d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:04.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.312+0000 7f237bc58640 1 -- 172.21.15.32:0/1792973841 shutdown_connections 2024-04-24T23:05:04.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.312+0000 7f237bc58640 1 -- 172.21.15.32:0/1792973841 wait complete. 2024-04-24T23:05:04.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.313+0000 7f237bc58640 1 Processor -- start 2024-04-24T23:05:04.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.313+0000 7f237bc58640 1 -- start start 2024-04-24T23:05:04.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.314+0000 7f237bc58640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f23740b5af0 0x7f23741e8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:04.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.314+0000 7f237bc58640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2374162320 0x7f23741f8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:04.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.314+0000 7f237bc58640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2374162e10 0x7f23740cfa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:04.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.314+0000 7f237bc58640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2374071bd0 con 0x7f2374162e10 2024-04-24T23:05:04.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.314+0000 7f237bc58640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2374067950 con 0x7f2374162320 2024-04-24T23:05:04.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.314+0000 7f237bc58640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f2374071d50 con 0x7f23740b5af0 2024-04-24T23:05:04.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.314+0000 7f237a1f5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2374162e10 0x7f23740cfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:04.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.314+0000 7f237a1f5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2374162e10 0x7f23740cfa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52722/0 (socket says 172.21.15.32:52722) 2024-04-24T23:05:04.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.314+0000 7f237a1f5640 1 -- 172.21.15.32:0/628107382 learned_addr learned my addr 172.21.15.32:0/628107382 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:04.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.314+0000 7f23799f4640 1 --2- 172.21.15.32:0/628107382 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f23740b5af0 0x7f23741e8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:04.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.314+0000 7f23791f3640 1 --2- 172.21.15.32:0/628107382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2374162320 0x7f23741f8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:04.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.314+0000 7f23799f4640 1 -- 172.21.15.32:0/628107382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2374162320 msgr2=0x7f23741f8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:04.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.314+0000 7f23799f4640 1 --2- 172.21.15.32:0/628107382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2374162320 0x7f23741f8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:04.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.314+0000 7f23799f4640 1 -- 172.21.15.32:0/628107382 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2374162e10 msgr2=0x7f23740cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:04.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.314+0000 7f23799f4640 1 --2- 172.21.15.32:0/628107382 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2374162e10 0x7f23740cfa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:04.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.314+0000 7f23799f4640 1 -- 172.21.15.32:0/628107382 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23740cd4f0 con 0x7f23740b5af0 2024-04-24T23:05:04.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.314+0000 7f237a1f5640 1 --2- 172.21.15.32:0/628107382 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2374162e10 0x7f23740cfa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:05:04.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.314+0000 7f23799f4640 1 --2- 172.21.15.32:0/628107382 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f23740b5af0 0x7f23741e8e00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f237000c470 tx=0x7f2370001e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:04.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.314+0000 7f236a7fc640 1 -- 172.21.15.32:0/628107382 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2370017070 con 0x7f23740b5af0 2024-04-24T23:05:04.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.314+0000 7f236a7fc640 1 -- 172.21.15.32:0/628107382 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2370005ce0 con 0x7f23740b5af0 2024-04-24T23:05:04.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.314+0000 7f237bc58640 1 -- 172.21.15.32:0/628107382 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23740cd790 con 0x7f23740b5af0 2024-04-24T23:05:04.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.314+0000 7f236a7fc640 1 -- 172.21.15.32:0/628107382 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23700072d0 con 0x7f23740b5af0 2024-04-24T23:05:04.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.315+0000 7f237bc58640 1 -- 172.21.15.32:0/628107382 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f23740cdd50 con 0x7f23740b5af0 2024-04-24T23:05:04.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.315+0000 7f237bc58640 1 -- 172.21.15.32:0/628107382 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2374068430 con 0x7f23740b5af0 2024-04-24T23:05:04.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.316+0000 7f236a7fc640 1 -- 172.21.15.32:0/628107382 <== mon.2 v2:172.21.15.121:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f237001a070 con 0x7f23740b5af0 2024-04-24T23:05:04.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.316+0000 7f236a7fc640 1 --2- 172.21.15.32:0/628107382 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f235806be30 0x7f235806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:04.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.316+0000 7f236a7fc640 1 -- 172.21.15.32:0/628107382 <== mon.2 v2:172.21.15.121:3301/0 5 ==== osd_map(125..125 src has 1..125) ==== 5709+0+0 (secure 0 0 0) 0x7f237008a0e0 con 0x7f23740b5af0 2024-04-24T23:05:04.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.316+0000 7f23791f3640 1 --2- 172.21.15.32:0/628107382 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f235806be30 0x7f235806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:04.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.317+0000 7f23791f3640 1 --2- 172.21.15.32:0/628107382 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f235806be30 0x7f235806e2f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f23740d0f70 tx=0x7f2364009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:04.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.321+0000 7f236a7fc640 1 -- 172.21.15.32:0/628107382 <== mon.2 v2:172.21.15.121:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2370058430 con 0x7f23740b5af0 2024-04-24T23:05:04.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.541+0000 7f237bc58640 1 -- 172.21.15.32:0/628107382 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f23740d0850 con 0x7f23740b5af0 2024-04-24T23:05:04.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.542+0000 7f236a7fc640 1 -- 172.21.15.32:0/628107382 <== mon.2 v2:172.21.15.121:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+5189 (secure 0 0 0) 0x7f237005c730 con 0x7f23740b5af0 2024-04-24T23:05:04.543 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:05:04.544 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":51,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5972,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3336404028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3336404028},{"type":"v1","addr":"172.21.15.121:6837","nonce":3336404028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6505,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/1068557774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1068557774},{"type":"v1","addr":"172.21.15.121:6836","nonce":1068557774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":7353,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/2118719195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":2118719195},{"type":"v1","addr":"172.21.15.32:6834","nonce":2118719195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:04:55.998022+0000","modified":"2024-04-24T23:04:58.657258+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7356},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7356":{"gid":7356,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/3015290487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":3015290487},{"type":"v1","addr":"172.21.15.32:6835","nonce":3015290487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7356,"qdb_cluster":[7356]},"id":7}]} 2024-04-24T23:05:04.544 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 51 2024-04-24T23:05:04.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.544+0000 7f237bc58640 1 -- 172.21.15.32:0/628107382 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f235806be30 msgr2=0x7f235806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:04.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.544+0000 7f237bc58640 1 --2- 172.21.15.32:0/628107382 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f235806be30 0x7f235806e2f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f23740d0f70 tx=0x7f2364009040 comp rx=0 tx=0).stop 2024-04-24T23:05:04.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.544+0000 7f237bc58640 1 -- 172.21.15.32:0/628107382 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f23740b5af0 msgr2=0x7f23741e8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:04.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.544+0000 7f237bc58640 1 --2- 172.21.15.32:0/628107382 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f23740b5af0 0x7f23741e8e00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f237000c470 tx=0x7f2370001e70 comp rx=0 tx=0).stop 2024-04-24T23:05:04.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.544+0000 7f237bc58640 1 -- 172.21.15.32:0/628107382 shutdown_connections 2024-04-24T23:05:04.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.544+0000 7f237bc58640 1 --2- 172.21.15.32:0/628107382 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f235806be30 0x7f235806e2f0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:04.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.544+0000 7f237bc58640 1 --2- 172.21.15.32:0/628107382 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2374162e10 0x7f23740cfa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:04.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.544+0000 7f237bc58640 1 --2- 172.21.15.32:0/628107382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2374162320 0x7f23741f8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:04.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.544+0000 7f237bc58640 1 --2- 172.21.15.32:0/628107382 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f23740b5af0 0x7f23741e8e00 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:04.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.544+0000 7f237bc58640 1 -- 172.21.15.32:0/628107382 >> 172.21.15.32:0/628107382 conn(0x7f23740d0ae0 msgr2=0x7f237415f260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:04.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.544+0000 7f237bc58640 1 -- 172.21.15.32:0/628107382 shutdown_connections 2024-04-24T23:05:04.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.544+0000 7f237bc58640 1 -- 172.21.15.32:0/628107382 wait complete. 2024-04-24T23:05:04.557 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-04-24T23:05:04.677 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","whoami":0,"id":7356,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":8.0422790580000001,"mdsmap_epoch":51,"osdmap_epoch":125,"osdmap_epoch_barrier":125,"uptime":13.379692811} 2024-04-24T23:05:04.677 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", "fs_name": "cephfs", "id": 7356, "mdsmap_epoch": 51, "osdmap_epoch": 125, "osdmap_epoch_barrier": 125, "rank_uptime": 8.042279058, "state": "up:active", "uptime": 13.379692811, "want_state": "up:active", "whoami": 0 } 2024-04-24T23:05:04.677 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2024-04-24T23:05:04.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.781+0000 7f95a1d34640 1 Processor -- start 2024-04-24T23:05:04.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.781+0000 7f95a1d34640 1 -- start start 2024-04-24T23:05:04.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.782+0000 7f95a1d34640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f959c0b5ff0 0x7f959c0cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:04.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.782+0000 7f95a1d34640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f959c15bde0 con 0x7f959c15b580 2024-04-24T23:05:04.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.782+0000 7f95a1d34640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f959c15c0c0 con 0x7f959c15b960 2024-04-24T23:05:04.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.782+0000 7f95a1d34640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f959c15c9f0 con 0x7f959c0b5ff0 2024-04-24T23:05:04.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.782+0000 7f959bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f959c0b5ff0 0x7f959c0cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:04.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.782+0000 7f959b7fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f959c15b580 0x7f959c06cb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56436/0 (socket says 172.21.15.32:56436) 2024-04-24T23:05:04.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.782+0000 7f959b7fe640 1 -- 172.21.15.32:0/885864924 learned_addr learned my addr 172.21.15.32:0/885864924 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:05:04.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.782+0000 7f959bfff640 1 -- 172.21.15.32:0/885864924 >> v1:172.21.15.121:6789/0 conn(0x7f959c15b960 legacy=0x7f959c0b58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:04.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.782+0000 7f959bfff640 1 -- 172.21.15.32:0/885864924 >> v1:172.21.15.32:6789/0 conn(0x7f959c15b580 legacy=0x7f959c06cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:04.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.782+0000 7f959bfff640 1 -- 172.21.15.32:0/885864924 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f959c15c3a0 con 0x7f959c0b5ff0 2024-04-24T23:05:04.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.783+0000 7f959bfff640 1 --2- 172.21.15.32:0/885864924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f959c0b5ff0 0x7f959c0cde80 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f959c15cb70 tx=0x7f959402c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c871f1ac3f9dba9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:04.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.783+0000 7f959a7fc640 1 -- 172.21.15.32:0/885864924 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9594032770 con 0x7f959c0b5ff0 2024-04-24T23:05:04.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.783+0000 7f959a7fc640 1 -- 172.21.15.32:0/885864924 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9594038070 con 0x7f959c0b5ff0 2024-04-24T23:05:04.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.783+0000 7f959a7fc640 1 -- 172.21.15.32:0/885864924 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9594032b30 con 0x7f959c0b5ff0 2024-04-24T23:05:04.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.783+0000 7f95a1d34640 1 -- 172.21.15.32:0/885864924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f959c0b5ff0 msgr2=0x7f959c0cde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:04.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.783+0000 7f95a1d34640 1 --2- 172.21.15.32:0/885864924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f959c0b5ff0 0x7f959c0cde80 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f959c15cb70 tx=0x7f959402c8b0 comp rx=0 tx=0).stop 2024-04-24T23:05:04.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.783+0000 7f95a1d34640 1 -- 172.21.15.32:0/885864924 shutdown_connections 2024-04-24T23:05:04.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.783+0000 7f95a1d34640 1 --2- 172.21.15.32:0/885864924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f959c0b5ff0 0x7f959c0cde80 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:04.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.783+0000 7f95a1d34640 1 -- 172.21.15.32:0/885864924 >> 172.21.15.32:0/885864924 conn(0x7f959c0b1f10 msgr2=0x7f959c0b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:04.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.784+0000 7f95a1d34640 1 -- 172.21.15.32:0/885864924 shutdown_connections 2024-04-24T23:05:04.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.784+0000 7f95a1d34640 1 -- 172.21.15.32:0/885864924 wait complete. 2024-04-24T23:05:04.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.785+0000 7f95a1d34640 1 Processor -- start 2024-04-24T23:05:04.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.785+0000 7f95a1d34640 1 -- start start 2024-04-24T23:05:04.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.786+0000 7f95a1d34640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f959c0b5ff0 0x7f959c147ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:04.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.786+0000 7f95a1d34640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f959c15b580 0x7f959c159d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:04.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.786+0000 7f95a1d34640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f959c15b960 0x7f959c1f0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:04.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.786+0000 7f95a1d34640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f959c06c7a0 con 0x7f959c15b960 2024-04-24T23:05:04.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.786+0000 7f95a1d34640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f959c0ce3c0 con 0x7f959c15b580 2024-04-24T23:05:04.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.786+0000 7f95a1d34640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f959c06c920 con 0x7f959c0b5ff0 2024-04-24T23:05:04.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.786+0000 7f959bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f959c15b960 0x7f959c1f0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:04.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.786+0000 7f959bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f959c15b960 0x7f959c1f0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52740/0 (socket says 172.21.15.32:52740) 2024-04-24T23:05:04.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.786+0000 7f959bfff640 1 -- 172.21.15.32:0/3263854585 learned_addr learned my addr 172.21.15.32:0/3263854585 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:04.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.786+0000 7f959b7fe640 1 --2- 172.21.15.32:0/3263854585 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f959c0b5ff0 0x7f959c147ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:04.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.786+0000 7f959affd640 1 --2- 172.21.15.32:0/3263854585 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f959c15b580 0x7f959c159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:04.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.786+0000 7f959bfff640 1 -- 172.21.15.32:0/3263854585 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f959c0b5ff0 msgr2=0x7f959c147ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:04.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.786+0000 7f959bfff640 1 --2- 172.21.15.32:0/3263854585 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f959c0b5ff0 0x7f959c147ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:04.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.786+0000 7f959bfff640 1 -- 172.21.15.32:0/3263854585 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f959c15b580 msgr2=0x7f959c159d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:04.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.786+0000 7f959bfff640 1 --2- 172.21.15.32:0/3263854585 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f959c15b580 0x7f959c159d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:04.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.786+0000 7f959bfff640 1 -- 172.21.15.32:0/3263854585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f959c071e80 con 0x7f959c15b960 2024-04-24T23:05:04.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.786+0000 7f959bfff640 1 --2- 172.21.15.32:0/3263854585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f959c15b960 0x7f959c1f0610 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f9594035040 tx=0x7f9594004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:04.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.786+0000 7f9583fff640 1 -- 172.21.15.32:0/3263854585 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9594033600 con 0x7f959c15b960 2024-04-24T23:05:04.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.786+0000 7f9583fff640 1 -- 172.21.15.32:0/3263854585 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95940337a0 con 0x7f959c15b960 2024-04-24T23:05:04.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.786+0000 7f95a1d34640 1 -- 172.21.15.32:0/3263854585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f959c072160 con 0x7f959c15b960 2024-04-24T23:05:04.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.786+0000 7f9583fff640 1 -- 172.21.15.32:0/3263854585 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9594033b70 con 0x7f959c15b960 2024-04-24T23:05:04.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.786+0000 7f95a1d34640 1 -- 172.21.15.32:0/3263854585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f959c148010 con 0x7f959c15b960 2024-04-24T23:05:04.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.788+0000 7f95a1d34640 1 -- 172.21.15.32:0/3263854585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f959c068430 con 0x7f959c15b960 2024-04-24T23:05:04.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.788+0000 7f9583fff640 1 -- 172.21.15.32:0/3263854585 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9594032070 con 0x7f959c15b960 2024-04-24T23:05:04.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.788+0000 7f9583fff640 1 --2- 172.21.15.32:0/3263854585 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f958806be30 0x7f958806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:04.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.788+0000 7f9583fff640 1 -- 172.21.15.32:0/3263854585 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5709+0+0 (secure 0 0 0) 0x7f95940add10 con 0x7f959c15b960 2024-04-24T23:05:04.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.788+0000 7f959b7fe640 1 --2- 172.21.15.32:0/3263854585 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f958806be30 0x7f958806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:04.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.789+0000 7f959b7fe640 1 --2- 172.21.15.32:0/3263854585 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f958806be30 0x7f958806e2f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9590007fd0 tx=0x7f9590007e80 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:04.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.793+0000 7f9583fff640 1 -- 172.21.15.32:0/3263854585 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9594042070 con 0x7f959c15b960 2024-04-24T23:05:04.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:04.979+0000 7f95a1d34640 1 -- 172.21.15.32:0/3263854585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) -- 0x7f959c0cc9d0 con 0x7f959c15b960 2024-04-24T23:05:05.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.313+0000 7f9583fff640 1 -- 172.21.15.32:0/3263854585 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v126) ==== 181+0+0 (secure 0 0 0) 0x7f959407c1a0 con 0x7f959c15b960 2024-04-24T23:05:05.314 INFO:teuthology.orchestra.run.smithi032.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2024-04-24T23:05:05.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.316+0000 7f95a1d34640 1 -- 172.21.15.32:0/3263854585 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f958806be30 msgr2=0x7f958806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:05.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.316+0000 7f95a1d34640 1 --2- 172.21.15.32:0/3263854585 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f958806be30 0x7f958806e2f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9590007fd0 tx=0x7f9590007e80 comp rx=0 tx=0).stop 2024-04-24T23:05:05.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.316+0000 7f95a1d34640 1 -- 172.21.15.32:0/3263854585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f959c15b960 msgr2=0x7f959c1f0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:05.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.316+0000 7f95a1d34640 1 --2- 172.21.15.32:0/3263854585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f959c15b960 0x7f959c1f0610 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f9594035040 tx=0x7f9594004a40 comp rx=0 tx=0).stop 2024-04-24T23:05:05.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.316+0000 7f95a1d34640 1 -- 172.21.15.32:0/3263854585 shutdown_connections 2024-04-24T23:05:05.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.316+0000 7f95a1d34640 1 --2- 172.21.15.32:0/3263854585 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f958806be30 0x7f958806e2f0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:05.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.316+0000 7f95a1d34640 1 --2- 172.21.15.32:0/3263854585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f959c15b960 0x7f959c1f0610 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:05.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.316+0000 7f95a1d34640 1 --2- 172.21.15.32:0/3263854585 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f959c15b580 0x7f959c159d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:05.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.316+0000 7f95a1d34640 1 --2- 172.21.15.32:0/3263854585 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f959c0b5ff0 0x7f959c147ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:05.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.316+0000 7f95a1d34640 1 -- 172.21.15.32:0/3263854585 >> 172.21.15.32:0/3263854585 conn(0x7f959c0b1f10 msgr2=0x7f959c15cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:05.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.316+0000 7f95a1d34640 1 -- 172.21.15.32:0/3263854585 shutdown_connections 2024-04-24T23:05:05.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.316+0000 7f95a1d34640 1 -- 172.21.15.32:0/3263854585 wait complete. 2024-04-24T23:05:05.329 DEBUG:teuthology.orchestra.run.smithi032:> stdin-killer -- python3 -c ' 2024-04-24T23:05:05.329 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-24T23:05:05.329 DEBUG:teuthology.orchestra.run.smithi032:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2024-04-24T23:05:05.329 DEBUG:teuthology.orchestra.run.smithi032:> os.fchmod(fd, 0O666) 2024-04-24T23:05:05.329 DEBUG:teuthology.orchestra.run.smithi032:> os.fsync(fd) 2024-04-24T23:05:05.329 DEBUG:teuthology.orchestra.run.smithi032:> os.close(fd) 2024-04-24T23:05:05.329 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-24T23:05:05.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:05:05.412 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:05:05.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:05:05.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:05:05.509 INFO:teuthology.orchestra.run.smithi032.stdout:client6047 2024-04-24T23:05:05.510 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/eb1b7a01-6f65-4534-9a12-83011862ba51.client6047/osdmap') 2024-04-24T23:05:05.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:05:05.568 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 125 barrier 125 flags 0x588000 2024-04-24T23:05:05.568 INFO:teuthology.orchestra.run.smithi032.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-04-24T23:05:05.568 INFO:teuthology.orchestra.run.smithi032.stdout:pool 14 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-04-24T23:05:05.568 INFO:teuthology.orchestra.run.smithi032.stdout:pool 15 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-04-24T23:05:05.568 INFO:teuthology.orchestra.run.smithi032.stdout:osd0 (2)172.21.15.32:6800 100% (exists, up) 100% -1 2024-04-24T23:05:05.568 INFO:teuthology.orchestra.run.smithi032.stdout:osd1 (2)172.21.15.32:6808 100% (exists, up) 100% -1 2024-04-24T23:05:05.568 INFO:teuthology.orchestra.run.smithi032.stdout:osd2 (2)172.21.15.32:6816 100% (exists, up) 100% -1 2024-04-24T23:05:05.568 INFO:teuthology.orchestra.run.smithi032.stdout:osd3 (2)172.21.15.32:6818 100% (exists, up) 100% -1 2024-04-24T23:05:05.568 INFO:teuthology.orchestra.run.smithi032.stdout:osd4 (2)172.21.15.121:6800 100% (exists, up) 100% -1 2024-04-24T23:05:05.569 INFO:teuthology.orchestra.run.smithi032.stdout:osd5 (2)172.21.15.121:6808 100% (exists, up) 100% -1 2024-04-24T23:05:05.569 INFO:teuthology.orchestra.run.smithi032.stdout:osd6 (2)172.21.15.121:6823 100% (exists, up) 100% -1 2024-04-24T23:05:05.569 INFO:teuthology.orchestra.run.smithi032.stdout:osd7 (2)172.21.15.121:6816 100% (exists, up) 100% -1 2024-04-24T23:05:05.569 INFO:teuthology.orchestra.run.smithi032.stderr:1+1 records in 2024-04-24T23:05:05.569 INFO:teuthology.orchestra.run.smithi032.stderr:1+1 records out 2024-04-24T23:05:05.569 INFO:teuthology.orchestra.run.smithi032.stderr:799 bytes copied, 3.5876e-05 s, 22.3 MB/s 2024-04-24T23:05:05.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:05:05.575 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set pause 2024-04-24T23:05:05.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.676+0000 7f46f0088640 1 Processor -- start 2024-04-24T23:05:05.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.676+0000 7f46f0088640 1 -- start start 2024-04-24T23:05:05.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.677+0000 7f46f0088640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f46e80b44d0 0x7f46e80b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:05.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.677+0000 7f46f0088640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f46e80af890 0x7f46e815cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:05.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.677+0000 7f46f0088640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46e80b6430 con 0x7f46e806c5f0 2024-04-24T23:05:05.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.677+0000 7f46f0088640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f46e8156f80 con 0x7f46e80b44d0 2024-04-24T23:05:05.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.677+0000 7f46f0088640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f46e80b6150 con 0x7f46e80af890 2024-04-24T23:05:05.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.677+0000 7f46ede24640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f46e806c5f0 0x7f46e806c9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56446/0 (socket says 172.21.15.32:56446) 2024-04-24T23:05:05.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.677+0000 7f46ede24640 1 -- 172.21.15.32:0/1650823903 learned_addr learned my addr 172.21.15.32:0/1650823903 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:05:05.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.677+0000 7f46ed623640 1 --2- 172.21.15.32:0/1650823903 >> v2:172.21.15.32:3300/0 conn(0x7f46e80b44d0 0x7f46e80b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:05.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.677+0000 7f46ed623640 1 -- 172.21.15.32:0/1650823903 >> v2:172.21.15.121:3300/0 conn(0x7f46e80af890 msgr2=0x7f46e815cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:05.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.677+0000 7f46ee625640 1 --2- 172.21.15.32:0/1650823903 >> v2:172.21.15.121:3300/0 conn(0x7f46e80af890 0x7f46e815cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:05.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.677+0000 7f46ece22640 1 -- 172.21.15.32:0/1650823903 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3116399107 0 0) 0x7f46e80b6430 con 0x7f46e806c5f0 2024-04-24T23:05:05.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.677+0000 7f46ed623640 1 --2- 172.21.15.32:0/1650823903 >> v2:172.21.15.121:3300/0 conn(0x7f46e80af890 0x7f46e815cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:05.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.677+0000 7f46ed623640 1 -- 172.21.15.32:0/1650823903 >> v1:172.21.15.32:6789/0 conn(0x7f46e806c5f0 legacy=0x7f46e806c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:05.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.677+0000 7f46ed623640 1 -- 172.21.15.32:0/1650823903 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46e80b39f0 con 0x7f46e80b44d0 2024-04-24T23:05:05.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.677+0000 7f46ee625640 1 --2- 172.21.15.32:0/1650823903 >> v2:172.21.15.121:3300/0 conn(0x7f46e80af890 0x7f46e815cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:05:05.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.678+0000 7f46ed623640 1 --2- 172.21.15.32:0/1650823903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46e80b44d0 0x7f46e80b5890 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f46e8073960 tx=0x7f46d802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a3d6bb163de488f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:05.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.678+0000 7f46ece22640 1 -- 172.21.15.32:0/1650823903 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d8037070 con 0x7f46e80b44d0 2024-04-24T23:05:05.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.678+0000 7f46ece22640 1 -- 172.21.15.32:0/1650823903 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46d8032440 con 0x7f46e80b44d0 2024-04-24T23:05:05.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.678+0000 7f46ece22640 1 -- 172.21.15.32:0/1650823903 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d8032810 con 0x7f46e80b44d0 2024-04-24T23:05:05.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.679+0000 7f46f0088640 1 -- 172.21.15.32:0/1650823903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46e80b44d0 msgr2=0x7f46e80b5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:05.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.679+0000 7f46f0088640 1 --2- 172.21.15.32:0/1650823903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46e80b44d0 0x7f46e80b5890 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f46e8073960 tx=0x7f46d802a1a0 comp rx=0 tx=0).stop 2024-04-24T23:05:05.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.679+0000 7f46f0088640 1 -- 172.21.15.32:0/1650823903 shutdown_connections 2024-04-24T23:05:05.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.679+0000 7f46f0088640 1 --2- 172.21.15.32:0/1650823903 >> v2:172.21.15.121:3300/0 conn(0x7f46e80af890 0x7f46e815cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:05.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.679+0000 7f46f0088640 1 --2- 172.21.15.32:0/1650823903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46e80b44d0 0x7f46e80b5890 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:05.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.679+0000 7f46f0088640 1 -- 172.21.15.32:0/1650823903 >> 172.21.15.32:0/1650823903 conn(0x7f46e806b110 msgr2=0x7f46e806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:05.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.679+0000 7f46f0088640 1 -- 172.21.15.32:0/1650823903 shutdown_connections 2024-04-24T23:05:05.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.679+0000 7f46f0088640 1 -- 172.21.15.32:0/1650823903 wait complete. 2024-04-24T23:05:05.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.680+0000 7f46f0088640 1 Processor -- start 2024-04-24T23:05:05.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.680+0000 7f46f0088640 1 -- start start 2024-04-24T23:05:05.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.681+0000 7f46f0088640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46e806c5f0 0x7f46e81525c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:05.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.681+0000 7f46f0088640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46e80af890 0x7f46e80aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:05.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.681+0000 7f46f0088640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46e80b44d0 0x7f46e815ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:05.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.681+0000 7f46f0088640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f46e80b5dd0 con 0x7f46e806c5f0 2024-04-24T23:05:05.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.681+0000 7f46f0088640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f46e80b3fb0 con 0x7f46e80af890 2024-04-24T23:05:05.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.681+0000 7f46f0088640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f46e815d000 con 0x7f46e80b44d0 2024-04-24T23:05:05.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.681+0000 7f46ede24640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46e806c5f0 0x7f46e81525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:05.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.681+0000 7f46ede24640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46e806c5f0 0x7f46e81525c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52760/0 (socket says 172.21.15.32:52760) 2024-04-24T23:05:05.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.681+0000 7f46ede24640 1 -- 172.21.15.32:0/2273500970 learned_addr learned my addr 172.21.15.32:0/2273500970 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:05.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.681+0000 7f46ee625640 1 --2- 172.21.15.32:0/2273500970 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46e80b44d0 0x7f46e815ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:05.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.681+0000 7f46ed623640 1 --2- 172.21.15.32:0/2273500970 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46e80af890 0x7f46e80aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:05.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.681+0000 7f46ede24640 1 -- 172.21.15.32:0/2273500970 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46e80b44d0 msgr2=0x7f46e815ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:05.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.681+0000 7f46ede24640 1 --2- 172.21.15.32:0/2273500970 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46e80b44d0 0x7f46e815ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:05.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.681+0000 7f46ede24640 1 -- 172.21.15.32:0/2273500970 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46e80af890 msgr2=0x7f46e80aebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:05.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.681+0000 7f46ede24640 1 --2- 172.21.15.32:0/2273500970 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46e80af890 0x7f46e80aebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:05.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.681+0000 7f46ede24640 1 -- 172.21.15.32:0/2273500970 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46e8158410 con 0x7f46e806c5f0 2024-04-24T23:05:05.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.681+0000 7f46ed623640 1 --2- 172.21.15.32:0/2273500970 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46e80af890 0x7f46e80aebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:05:05.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.681+0000 7f46ee625640 1 --2- 172.21.15.32:0/2273500970 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46e80b44d0 0x7f46e815ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:05:05.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.681+0000 7f46ede24640 1 --2- 172.21.15.32:0/2273500970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46e806c5f0 0x7f46e81525c0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f46e400c4d0 tx=0x7f46e400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:05.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.682+0000 7f46de7fc640 1 -- 172.21.15.32:0/2273500970 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e4018070 con 0x7f46e806c5f0 2024-04-24T23:05:05.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.682+0000 7f46de7fc640 1 -- 172.21.15.32:0/2273500970 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46e4007500 con 0x7f46e806c5f0 2024-04-24T23:05:05.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.682+0000 7f46f0088640 1 -- 172.21.15.32:0/2273500970 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46e8152df0 con 0x7f46e806c5f0 2024-04-24T23:05:05.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.682+0000 7f46de7fc640 1 -- 172.21.15.32:0/2273500970 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e4004a20 con 0x7f46e806c5f0 2024-04-24T23:05:05.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.682+0000 7f46f0088640 1 -- 172.21.15.32:0/2273500970 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46e81533b0 con 0x7f46e806c5f0 2024-04-24T23:05:05.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.683+0000 7f46f0088640 1 -- 172.21.15.32:0/2273500970 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46e8068430 con 0x7f46e806c5f0 2024-04-24T23:05:05.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.683+0000 7f46de7fc640 1 -- 172.21.15.32:0/2273500970 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f46e401b070 con 0x7f46e806c5f0 2024-04-24T23:05:05.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.683+0000 7f46de7fc640 1 --2- 172.21.15.32:0/2273500970 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f46cc06be30 0x7f46cc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:05.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.684+0000 7f46de7fc640 1 -- 172.21.15.32:0/2273500970 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5709+0+0 (secure 0 0 0) 0x7f46e408a240 con 0x7f46e806c5f0 2024-04-24T23:05:05.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.684+0000 7f46ed623640 1 --2- 172.21.15.32:0/2273500970 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f46cc06be30 0x7f46cc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:05.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.684+0000 7f46ed623640 1 --2- 172.21.15.32:0/2273500970 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f46cc06be30 0x7f46cc06e2f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f46d8034000 tx=0x7f46d802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:05.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.688+0000 7f46de7fc640 1 -- 172.21.15.32:0/2273500970 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f46e4058b60 con 0x7f46e806c5f0 2024-04-24T23:05:05.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:05.880+0000 7f46f0088640 1 -- 172.21.15.32:0/2273500970 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd set", "key": "pause"} v 0) -- 0x7f46e80b5670 con 0x7f46e806c5f0 2024-04-24T23:05:06.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.319+0000 7f46de7fc640 1 -- 172.21.15.32:0/2273500970 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "pause"}]=0 pauserd,pausewr is set v127) ==== 93+0+0 (secure 0 0 0) 0x7f46e4058460 con 0x7f46e806c5f0 2024-04-24T23:05:06.320 INFO:teuthology.orchestra.run.smithi032.stderr:pauserd,pausewr is set 2024-04-24T23:05:06.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.322+0000 7f46f0088640 1 -- 172.21.15.32:0/2273500970 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f46cc06be30 msgr2=0x7f46cc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:06.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.322+0000 7f46f0088640 1 --2- 172.21.15.32:0/2273500970 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f46cc06be30 0x7f46cc06e2f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f46d8034000 tx=0x7f46d802e000 comp rx=0 tx=0).stop 2024-04-24T23:05:06.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.322+0000 7f46f0088640 1 -- 172.21.15.32:0/2273500970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46e806c5f0 msgr2=0x7f46e81525c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:06.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.322+0000 7f46f0088640 1 --2- 172.21.15.32:0/2273500970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46e806c5f0 0x7f46e81525c0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f46e400c4d0 tx=0x7f46e400c990 comp rx=0 tx=0).stop 2024-04-24T23:05:06.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.322+0000 7f46f0088640 1 -- 172.21.15.32:0/2273500970 shutdown_connections 2024-04-24T23:05:06.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.322+0000 7f46f0088640 1 --2- 172.21.15.32:0/2273500970 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f46cc06be30 0x7f46cc06e2f0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:06.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.322+0000 7f46f0088640 1 --2- 172.21.15.32:0/2273500970 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46e80b44d0 0x7f46e815ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:06.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.322+0000 7f46f0088640 1 --2- 172.21.15.32:0/2273500970 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46e80af890 0x7f46e80aebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:06.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.322+0000 7f46f0088640 1 --2- 172.21.15.32:0/2273500970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46e806c5f0 0x7f46e81525c0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:06.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.322+0000 7f46f0088640 1 -- 172.21.15.32:0/2273500970 >> 172.21.15.32:0/2273500970 conn(0x7f46e806b110 msgr2=0x7f46e80d0190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:06.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.322+0000 7f46f0088640 1 -- 172.21.15.32:0/2273500970 shutdown_connections 2024-04-24T23:05:06.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.322+0000 7f46f0088640 1 -- 172.21.15.32:0/2273500970 wait complete. 2024-04-24T23:05:06.335 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset pause 2024-04-24T23:05:06.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.438+0000 7fba9c815640 1 Processor -- start 2024-04-24T23:05:06.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.438+0000 7fba9c815640 1 -- start start 2024-04-24T23:05:06.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.439+0000 7fba9c815640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fba9815e800 0x7fba980d2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:06.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.439+0000 7fba9c815640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fba980ba260 0x7fba980b6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:06.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.439+0000 7fba9c815640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba980b8730 con 0x7fba980b6990 2024-04-24T23:05:06.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.439+0000 7fba9c815640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fba980b6e00 con 0x7fba980ba260 2024-04-24T23:05:06.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.439+0000 7fba9c815640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fba980d31a0 con 0x7fba9815e800 2024-04-24T23:05:06.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.439+0000 7fba95d9b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fba980ba260 0x7fba980b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:06.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.439+0000 7fba96d9d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fba980b6990 0x7fba98071730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56448/0 (socket says 172.21.15.32:56448) 2024-04-24T23:05:06.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.439+0000 7fba95d9b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fba980ba260 0x7fba980b6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52772/0 (socket says 172.21.15.32:52772) 2024-04-24T23:05:06.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.439+0000 7fba96d9d640 1 -- 172.21.15.32:0/2441372606 learned_addr learned my addr 172.21.15.32:0/2441372606 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:05:06.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.440+0000 7fba9659c640 1 --2- 172.21.15.32:0/2441372606 >> v2:172.21.15.121:3300/0 conn(0x7fba9815e800 0x7fba980d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:06.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.440+0000 7fba95d9b640 1 -- 172.21.15.32:0/2441372606 >> v2:172.21.15.121:3300/0 conn(0x7fba9815e800 msgr2=0x7fba980d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:06.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.440+0000 7fba95d9b640 1 --2- 172.21.15.32:0/2441372606 >> v2:172.21.15.121:3300/0 conn(0x7fba9815e800 0x7fba980d2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:06.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.440+0000 7fba95d9b640 1 -- 172.21.15.32:0/2441372606 >> v1:172.21.15.32:6789/0 conn(0x7fba980b6990 legacy=0x7fba98071730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:06.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.440+0000 7fba95d9b640 1 -- 172.21.15.32:0/2441372606 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba980d3480 con 0x7fba980ba260 2024-04-24T23:05:06.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.440+0000 7fba9659c640 1 --2- 172.21.15.32:0/2441372606 >> v2:172.21.15.121:3300/0 conn(0x7fba9815e800 0x7fba980d2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:05:06.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.440+0000 7fba95d9b640 1 --2- 172.21.15.32:0/2441372606 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba980ba260 0x7fba980b6450 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fba980758e0 tx=0x7fba8002a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3850f3c8b427102b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:06.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.441+0000 7fba9559a640 1 -- 172.21.15.32:0/2441372606 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba80037070 con 0x7fba980ba260 2024-04-24T23:05:06.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.441+0000 7fba9559a640 1 -- 172.21.15.32:0/2441372606 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba8002ac90 con 0x7fba980ba260 2024-04-24T23:05:06.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.441+0000 7fba9559a640 1 -- 172.21.15.32:0/2441372606 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba80032660 con 0x7fba980ba260 2024-04-24T23:05:06.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.441+0000 7fba9c815640 1 -- 172.21.15.32:0/2441372606 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba980ba260 msgr2=0x7fba980b6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:06.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.441+0000 7fba9c815640 1 --2- 172.21.15.32:0/2441372606 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba980ba260 0x7fba980b6450 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fba980758e0 tx=0x7fba8002a000 comp rx=0 tx=0).stop 2024-04-24T23:05:06.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.441+0000 7fba9c815640 1 -- 172.21.15.32:0/2441372606 shutdown_connections 2024-04-24T23:05:06.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.441+0000 7fba9c815640 1 --2- 172.21.15.32:0/2441372606 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba980ba260 0x7fba980b6450 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:06.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.441+0000 7fba9c815640 1 --2- 172.21.15.32:0/2441372606 >> v2:172.21.15.121:3300/0 conn(0x7fba9815e800 0x7fba980d2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:06.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.441+0000 7fba9c815640 1 -- 172.21.15.32:0/2441372606 >> 172.21.15.32:0/2441372606 conn(0x7fba980d1490 msgr2=0x7fba980d18a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:06.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.441+0000 7fba9c815640 1 -- 172.21.15.32:0/2441372606 shutdown_connections 2024-04-24T23:05:06.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.441+0000 7fba9c815640 1 -- 172.21.15.32:0/2441372606 wait complete. 2024-04-24T23:05:06.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.443+0000 7fba9c815640 1 Processor -- start 2024-04-24T23:05:06.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.443+0000 7fba9c815640 1 -- start start 2024-04-24T23:05:06.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.443+0000 7fba9c815640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba980b6990 0x7fba980ce540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:06.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.443+0000 7fba9c815640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fba980ba260 0x7fba981f4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:06.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.443+0000 7fba9c815640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fba9815e800 0x7fba98160070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:06.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.443+0000 7fba9c815640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fba980d3d20 con 0x7fba980b6990 2024-04-24T23:05:06.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.443+0000 7fba9c815640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fba980d3a40 con 0x7fba9815e800 2024-04-24T23:05:06.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.443+0000 7fba9c815640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fba980d2e90 con 0x7fba980ba260 2024-04-24T23:05:06.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.443+0000 7fba9659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba980b6990 0x7fba980ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:06.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.443+0000 7fba9659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba980b6990 0x7fba980ce540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52774/0 (socket says 172.21.15.32:52774) 2024-04-24T23:05:06.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.444+0000 7fba9659c640 1 -- 172.21.15.32:0/570201269 learned_addr learned my addr 172.21.15.32:0/570201269 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:06.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.444+0000 7fba96d9d640 1 --2- 172.21.15.32:0/570201269 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fba9815e800 0x7fba98160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:06.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.444+0000 7fba95d9b640 1 --2- 172.21.15.32:0/570201269 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fba980ba260 0x7fba981f4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:06.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.444+0000 7fba9659c640 1 -- 172.21.15.32:0/570201269 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fba980ba260 msgr2=0x7fba981f4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:06.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.444+0000 7fba9659c640 1 --2- 172.21.15.32:0/570201269 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fba980ba260 0x7fba981f4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:06.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.444+0000 7fba9659c640 1 -- 172.21.15.32:0/570201269 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fba9815e800 msgr2=0x7fba98160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:06.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.444+0000 7fba9659c640 1 --2- 172.21.15.32:0/570201269 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fba9815e800 0x7fba98160070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:06.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.444+0000 7fba9659c640 1 -- 172.21.15.32:0/570201269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba981f5070 con 0x7fba980b6990 2024-04-24T23:05:06.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.444+0000 7fba96d9d640 1 --2- 172.21.15.32:0/570201269 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fba9815e800 0x7fba98160070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:05:06.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.444+0000 7fba95d9b640 1 --2- 172.21.15.32:0/570201269 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fba980ba260 0x7fba981f4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:05:06.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.444+0000 7fba9659c640 1 --2- 172.21.15.32:0/570201269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba980b6990 0x7fba980ce540 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fba9000c7f0 tx=0x7fba9000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:06.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.444+0000 7fba8effd640 1 -- 172.21.15.32:0/570201269 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba90015070 con 0x7fba980b6990 2024-04-24T23:05:06.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.444+0000 7fba8effd640 1 -- 172.21.15.32:0/570201269 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba90007820 con 0x7fba980b6990 2024-04-24T23:05:06.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.444+0000 7fba9c815640 1 -- 172.21.15.32:0/570201269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba981f5350 con 0x7fba980b6990 2024-04-24T23:05:06.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.444+0000 7fba8effd640 1 -- 172.21.15.32:0/570201269 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba90007bd0 con 0x7fba980b6990 2024-04-24T23:05:06.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.444+0000 7fba9c815640 1 -- 172.21.15.32:0/570201269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba981f5c10 con 0x7fba980b6990 2024-04-24T23:05:06.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.445+0000 7fba9c815640 1 -- 172.21.15.32:0/570201269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba980d1dd0 con 0x7fba980b6990 2024-04-24T23:05:06.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.446+0000 7fba8effd640 1 -- 172.21.15.32:0/570201269 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fba90018070 con 0x7fba980b6990 2024-04-24T23:05:06.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.446+0000 7fba8effd640 1 --2- 172.21.15.32:0/570201269 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fba7806be30 0x7fba7806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:06.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.446+0000 7fba8effd640 1 -- 172.21.15.32:0/570201269 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5709+0+0 (secure 0 0 0) 0x7fba9008af80 con 0x7fba980b6990 2024-04-24T23:05:06.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.446+0000 7fba8effd640 1 -- 172.21.15.32:0/570201269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=128+}) -- 0x7fba78073960 con 0x7fba980b6990 2024-04-24T23:05:06.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.446+0000 7fba95d9b640 1 --2- 172.21.15.32:0/570201269 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fba7806be30 0x7fba7806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:06.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.447+0000 7fba95d9b640 1 --2- 172.21.15.32:0/570201269 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fba7806be30 0x7fba7806e2f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fba80034000 tx=0x7fba8002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:06.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.450+0000 7fba8effd640 1 -- 172.21.15.32:0/570201269 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fba980d1dd0 con 0x7fba980b6990 2024-04-24T23:05:06.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:06.640+0000 7fba9c815640 1 -- 172.21.15.32:0/570201269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd unset", "key": "pause"} v 0) -- 0x7fba98068430 con 0x7fba980b6990 2024-04-24T23:05:06.977 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:05:06.980 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:05:07.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.321+0000 7fba8effd640 1 -- 172.21.15.32:0/570201269 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "pause"}]=0 pauserd,pausewr is unset v128) ==== 97+0+0 (secure 0 0 0) 0x7fba90059940 con 0x7fba980b6990 2024-04-24T23:05:07.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.322+0000 7fba8effd640 1 -- 172.21.15.32:0/570201269 <== mon.0 v2:172.21.15.32:3300/0 8 ==== osd_map(128..128 src has 1..128) ==== 288+0+0 (secure 0 0 0) 0x7fba900895c0 con 0x7fba980b6990 2024-04-24T23:05:07.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.322+0000 7fba8effd640 1 -- 172.21.15.32:0/570201269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=129}) -- 0x7fba780742d0 con 0x7fba980b6990 2024-04-24T23:05:07.323 INFO:teuthology.orchestra.run.smithi032.stderr:pauserd,pausewr is unset 2024-04-24T23:05:07.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.325+0000 7fba9c815640 1 -- 172.21.15.32:0/570201269 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fba7806be30 msgr2=0x7fba7806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:07.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.325+0000 7fba9c815640 1 --2- 172.21.15.32:0/570201269 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fba7806be30 0x7fba7806e2f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fba80034000 tx=0x7fba8002e000 comp rx=0 tx=0).stop 2024-04-24T23:05:07.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.325+0000 7fba9c815640 1 -- 172.21.15.32:0/570201269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba980b6990 msgr2=0x7fba980ce540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:07.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.325+0000 7fba9c815640 1 --2- 172.21.15.32:0/570201269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba980b6990 0x7fba980ce540 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fba9000c7f0 tx=0x7fba9000ccb0 comp rx=0 tx=0).stop 2024-04-24T23:05:07.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.325+0000 7fba9c815640 1 -- 172.21.15.32:0/570201269 shutdown_connections 2024-04-24T23:05:07.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.325+0000 7fba9c815640 1 --2- 172.21.15.32:0/570201269 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fba7806be30 0x7fba7806e2f0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:07.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.325+0000 7fba9c815640 1 --2- 172.21.15.32:0/570201269 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fba9815e800 0x7fba98160070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:07.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.325+0000 7fba9c815640 1 --2- 172.21.15.32:0/570201269 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fba980ba260 0x7fba981f4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:07.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.325+0000 7fba9c815640 1 --2- 172.21.15.32:0/570201269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba980b6990 0x7fba980ce540 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:07.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.325+0000 7fba9c815640 1 -- 172.21.15.32:0/570201269 >> 172.21.15.32:0/570201269 conn(0x7fba980d1490 msgr2=0x7fba98071730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:07.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.325+0000 7fba9c815640 1 -- 172.21.15.32:0/570201269 shutdown_connections 2024-04-24T23:05:07.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.325+0000 7fba9c815640 1 -- 172.21.15.32:0/570201269 wait complete. 2024-04-24T23:05:07.339 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:05:07.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.443+0000 7f9b7c461640 1 Processor -- start 2024-04-24T23:05:07.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.443+0000 7f9b7c461640 1 -- start start 2024-04-24T23:05:07.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.444+0000 7f9b7c461640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9b7406c5f0 0x7f9b7406c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:07.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.444+0000 7f9b7c461640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9b740b44d0 0x7f9b740b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:07.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.444+0000 7f9b7c461640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b740af890 0x7f9b740cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:07.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.444+0000 7f9b7c461640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f9b740b6430 con 0x7f9b740b44d0 2024-04-24T23:05:07.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.444+0000 7f9b7c461640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f9b74156f80 con 0x7f9b7406c5f0 2024-04-24T23:05:07.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.444+0000 7f9b7c461640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f9b740b6150 con 0x7f9b740af890 2024-04-24T23:05:07.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.444+0000 7f9b799fc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9b740b44d0 0x7f9b740b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:07.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.444+0000 7f9b799fc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9b740b44d0 0x7f9b740b5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52782/0 (socket says 172.21.15.32:52782) 2024-04-24T23:05:07.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.444+0000 7f9b799fc640 1 -- 172.21.15.32:0/2590349247 learned_addr learned my addr 172.21.15.32:0/2590349247 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:07.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.445+0000 7f9b799fc640 1 -- 172.21.15.32:0/2590349247 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b740af890 msgr2=0x7f9b740cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:07.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.445+0000 7f9b799fc640 1 --2- 172.21.15.32:0/2590349247 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b740af890 0x7f9b740cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:07.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.445+0000 7f9b799fc640 1 -- 172.21.15.32:0/2590349247 >> v2:172.21.15.121:3300/0 conn(0x7f9b7406c5f0 msgr2=0x7f9b7406c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:07.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.445+0000 7f9b7a1fd640 1 --2- 172.21.15.32:0/2590349247 >> v2:172.21.15.121:3300/0 conn(0x7f9b7406c5f0 0x7f9b7406c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:07.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.445+0000 7f9b799fc640 1 --2- 172.21.15.32:0/2590349247 >> v2:172.21.15.121:3300/0 conn(0x7f9b7406c5f0 0x7f9b7406c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:07.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.445+0000 7f9b799fc640 1 -- 172.21.15.32:0/2590349247 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b740b39f0 con 0x7f9b740b44d0 2024-04-24T23:05:07.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.445+0000 7f9b7a1fd640 1 --2- 172.21.15.32:0/2590349247 >> v2:172.21.15.121:3300/0 conn(0x7f9b7406c5f0 0x7f9b7406c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:05:07.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.445+0000 7f9b799fc640 1 --2- 172.21.15.32:0/2590349247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b740b44d0 0x7f9b740b5890 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f9b7406c1f0 tx=0x7f9b6802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c1340a457a73847 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:07.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.446+0000 7f9b791fb640 1 -- 172.21.15.32:0/2590349247 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b68037070 con 0x7f9b740b44d0 2024-04-24T23:05:07.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.446+0000 7f9b791fb640 1 -- 172.21.15.32:0/2590349247 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b68032440 con 0x7f9b740b44d0 2024-04-24T23:05:07.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.446+0000 7f9b791fb640 1 -- 172.21.15.32:0/2590349247 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b68032810 con 0x7f9b740b44d0 2024-04-24T23:05:07.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.446+0000 7f9b7c461640 1 -- 172.21.15.32:0/2590349247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b740b44d0 msgr2=0x7f9b740b5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:07.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.446+0000 7f9b7c461640 1 --2- 172.21.15.32:0/2590349247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b740b44d0 0x7f9b740b5890 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f9b7406c1f0 tx=0x7f9b6802a140 comp rx=0 tx=0).stop 2024-04-24T23:05:07.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.446+0000 7f9b7c461640 1 -- 172.21.15.32:0/2590349247 shutdown_connections 2024-04-24T23:05:07.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.446+0000 7f9b7c461640 1 --2- 172.21.15.32:0/2590349247 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b740af890 0x7f9b740cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:07.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.446+0000 7f9b7c461640 1 --2- 172.21.15.32:0/2590349247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b740b44d0 0x7f9b740b5890 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:07.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.446+0000 7f9b7c461640 1 --2- 172.21.15.32:0/2590349247 >> v2:172.21.15.121:3300/0 conn(0x7f9b7406c5f0 0x7f9b7406c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:07.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.446+0000 7f9b7c461640 1 -- 172.21.15.32:0/2590349247 >> 172.21.15.32:0/2590349247 conn(0x7f9b7406b110 msgr2=0x7f9b7406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:07.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.446+0000 7f9b7c461640 1 -- 172.21.15.32:0/2590349247 shutdown_connections 2024-04-24T23:05:07.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.446+0000 7f9b7c461640 1 -- 172.21.15.32:0/2590349247 wait complete. 2024-04-24T23:05:07.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.447+0000 7f9b7c461640 1 Processor -- start 2024-04-24T23:05:07.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.447+0000 7f9b7c461640 1 -- start start 2024-04-24T23:05:07.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.448+0000 7f9b7c461640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b7406c5f0 0x7f9b74152c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:07.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.448+0000 7f9b7c461640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b740af890 0x7f9b740ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:07.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.448+0000 7f9b7c461640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b740b44d0 0x7f9b740aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:07.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.448+0000 7f9b7c461640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7406cf10 con 0x7f9b740af890 2024-04-24T23:05:07.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.448+0000 7f9b7c461640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9b740b3fb0 con 0x7f9b7406c5f0 2024-04-24T23:05:07.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.448+0000 7f9b7c461640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f9b740b5dd0 con 0x7f9b740b44d0 2024-04-24T23:05:07.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.448+0000 7f9b799fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b740af890 0x7f9b740ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:07.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.448+0000 7f9b799fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b740af890 0x7f9b740ae8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52784/0 (socket says 172.21.15.32:52784) 2024-04-24T23:05:07.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.448+0000 7f9b799fc640 1 -- 172.21.15.32:0/2764695258 learned_addr learned my addr 172.21.15.32:0/2764695258 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:07.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.448+0000 7f9b7a1fd640 1 --2- 172.21.15.32:0/2764695258 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b7406c5f0 0x7f9b74152c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:07.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.448+0000 7f9b7a9fe640 1 --2- 172.21.15.32:0/2764695258 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b740b44d0 0x7f9b740aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:07.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.449+0000 7f9b799fc640 1 -- 172.21.15.32:0/2764695258 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b740b44d0 msgr2=0x7f9b740aedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:07.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.449+0000 7f9b799fc640 1 --2- 172.21.15.32:0/2764695258 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b740b44d0 0x7f9b740aedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:07.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.449+0000 7f9b799fc640 1 -- 172.21.15.32:0/2764695258 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b7406c5f0 msgr2=0x7f9b74152c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:07.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.449+0000 7f9b799fc640 1 --2- 172.21.15.32:0/2764695258 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b7406c5f0 0x7f9b74152c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:07.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.449+0000 7f9b799fc640 1 -- 172.21.15.32:0/2764695258 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b7415e200 con 0x7f9b740af890 2024-04-24T23:05:07.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.449+0000 7f9b7a9fe640 1 --2- 172.21.15.32:0/2764695258 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b740b44d0 0x7f9b740aedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:05:07.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.449+0000 7f9b7a1fd640 1 --2- 172.21.15.32:0/2764695258 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b7406c5f0 0x7f9b74152c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:05:07.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.449+0000 7f9b799fc640 1 --2- 172.21.15.32:0/2764695258 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b740af890 0x7f9b740ae8b0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f9b68000c00 tx=0x7f9b680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:07.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.449+0000 7f9b72ffd640 1 -- 172.21.15.32:0/2764695258 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b68041020 con 0x7f9b740af890 2024-04-24T23:05:07.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.449+0000 7f9b72ffd640 1 -- 172.21.15.32:0/2764695258 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b68030970 con 0x7f9b740af890 2024-04-24T23:05:07.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.449+0000 7f9b7c461640 1 -- 172.21.15.32:0/2764695258 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b7415e4e0 con 0x7f9b740af890 2024-04-24T23:05:07.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.449+0000 7f9b72ffd640 1 -- 172.21.15.32:0/2764695258 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b68037040 con 0x7f9b740af890 2024-04-24T23:05:07.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.449+0000 7f9b7c461640 1 -- 172.21.15.32:0/2764695258 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b7415eaa0 con 0x7f9b740af890 2024-04-24T23:05:07.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.450+0000 7f9b7c461640 1 -- 172.21.15.32:0/2764695258 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b74068430 con 0x7f9b740af890 2024-04-24T23:05:07.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.450+0000 7f9b72ffd640 1 -- 172.21.15.32:0/2764695258 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9b68044020 con 0x7f9b740af890 2024-04-24T23:05:07.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.451+0000 7f9b72ffd640 1 --2- 172.21.15.32:0/2764695258 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9b6006be30 0x7f9b6006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:07.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.451+0000 7f9b72ffd640 1 -- 172.21.15.32:0/2764695258 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5709+0+0 (secure 0 0 0) 0x7f9b680ab8b0 con 0x7f9b740af890 2024-04-24T23:05:07.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.451+0000 7f9b7a1fd640 1 --2- 172.21.15.32:0/2764695258 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9b6006be30 0x7f9b6006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:07.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.452+0000 7f9b7a1fd640 1 --2- 172.21.15.32:0/2764695258 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9b6006be30 0x7f9b6006e2f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c006fd0 tx=0x7f9b6c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:07.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.455+0000 7f9b72ffd640 1 -- 172.21.15.32:0/2764695258 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9b6804b030 con 0x7f9b740af890 2024-04-24T23:05:07.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.643+0000 7f9b7c461640 1 -- 172.21.15.32:0/2764695258 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9b740b4d80 con 0x7f9b740af890 2024-04-24T23:05:07.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.645+0000 7f9b72ffd640 1 -- 172.21.15.32:0/2764695258 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) ==== 74+0+16639 (secure 0 0 0) 0x7f9b6807a250 con 0x7f9b740af890 2024-04-24T23:05:07.646 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:05:07.646 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":128,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:05:07.320153+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:04:54.280785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-24T23:04:54.966294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:05:07.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.646+0000 7f9b7c461640 1 -- 172.21.15.32:0/2764695258 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9b6006be30 msgr2=0x7f9b6006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:07.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.646+0000 7f9b7c461640 1 --2- 172.21.15.32:0/2764695258 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9b6006be30 0x7f9b6006e2f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c006fd0 tx=0x7f9b6c008040 comp rx=0 tx=0).stop 2024-04-24T23:05:07.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.646+0000 7f9b7c461640 1 -- 172.21.15.32:0/2764695258 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b740af890 msgr2=0x7f9b740ae8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:07.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.646+0000 7f9b7c461640 1 --2- 172.21.15.32:0/2764695258 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b740af890 0x7f9b740ae8b0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f9b68000c00 tx=0x7f9b680046c0 comp rx=0 tx=0).stop 2024-04-24T23:05:07.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.647+0000 7f9b7c461640 1 -- 172.21.15.32:0/2764695258 shutdown_connections 2024-04-24T23:05:07.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.647+0000 7f9b7c461640 1 --2- 172.21.15.32:0/2764695258 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9b6006be30 0x7f9b6006e2f0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:07.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.647+0000 7f9b7c461640 1 --2- 172.21.15.32:0/2764695258 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b740af890 0x7f9b740ae8b0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:07.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.647+0000 7f9b7c461640 1 --2- 172.21.15.32:0/2764695258 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b740b44d0 0x7f9b740aedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:07.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.647+0000 7f9b7c461640 1 --2- 172.21.15.32:0/2764695258 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b7406c5f0 0x7f9b74152c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:07.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.647+0000 7f9b7c461640 1 -- 172.21.15.32:0/2764695258 >> 172.21.15.32:0/2764695258 conn(0x7f9b7406b110 msgr2=0x7f9b74155e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:07.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.647+0000 7f9b7c461640 1 -- 172.21.15.32:0/2764695258 shutdown_connections 2024-04-24T23:05:07.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07.647+0000 7f9b7c461640 1 -- 172.21.15.32:0/2764695258 wait complete. 2024-04-24T23:05:07.660 DEBUG:teuthology.orchestra.run.smithi032:> stdin-killer -- python3 -c ' 2024-04-24T23:05:07.660 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-24T23:05:07.660 DEBUG:teuthology.orchestra.run.smithi032:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2024-04-24T23:05:07.660 DEBUG:teuthology.orchestra.run.smithi032:> os.fchmod(fd, 0O666) 2024-04-24T23:05:07.660 DEBUG:teuthology.orchestra.run.smithi032:> os.fsync(fd) 2024-04-24T23:05:07.660 DEBUG:teuthology.orchestra.run.smithi032:> os.close(fd) 2024-04-24T23:05:07.660 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-24T23:05:07.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:05:07.737 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:05:07.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:05:07.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:05:07.835 INFO:teuthology.orchestra.run.smithi032.stdout:client6047 2024-04-24T23:05:07.835 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/eb1b7a01-6f65-4534-9a12-83011862ba51.client6047/osdmap') 2024-04-24T23:05:07.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:05:07.893 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 125 barrier 125 flags 0x588000 2024-04-24T23:05:07.893 INFO:teuthology.orchestra.run.smithi032.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-04-24T23:05:07.893 INFO:teuthology.orchestra.run.smithi032.stdout:pool 14 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-04-24T23:05:07.894 INFO:teuthology.orchestra.run.smithi032.stdout:pool 15 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-04-24T23:05:07.894 INFO:teuthology.orchestra.run.smithi032.stdout:osd0 (2)172.21.15.32:6800 100% (exists, up) 100% -1 2024-04-24T23:05:07.894 INFO:teuthology.orchestra.run.smithi032.stdout:osd1 (2)172.21.15.32:6808 100% (exists, up) 100% -1 2024-04-24T23:05:07.894 INFO:teuthology.orchestra.run.smithi032.stdout:osd2 (2)172.21.15.32:6816 100% (exists, up) 100% -1 2024-04-24T23:05:07.894 INFO:teuthology.orchestra.run.smithi032.stdout:osd3 (2)172.21.15.32:6818 100% (exists, up) 100% -1 2024-04-24T23:05:07.894 INFO:teuthology.orchestra.run.smithi032.stdout:osd4 (2)172.21.15.121:6800 100% (exists, up) 100% -1 2024-04-24T23:05:07.894 INFO:teuthology.orchestra.run.smithi032.stdout:osd5 (2)172.21.15.121:6808 100% (exists, up) 100% -1 2024-04-24T23:05:07.894 INFO:teuthology.orchestra.run.smithi032.stdout:osd6 (2)172.21.15.121:6823 100% (exists, up) 100% -1 2024-04-24T23:05:07.894 INFO:teuthology.orchestra.run.smithi032.stdout:osd7 (2)172.21.15.121:6816 100% (exists, up) 100% -1 2024-04-24T23:05:07.894 INFO:teuthology.orchestra.run.smithi032.stderr:1+1 records in 2024-04-24T23:05:07.895 INFO:teuthology.orchestra.run.smithi032.stderr:1+1 records out 2024-04-24T23:05:07.895 INFO:teuthology.orchestra.run.smithi032.stderr:799 bytes copied, 4.0416e-05 s, 19.8 MB/s 2024-04-24T23:05:07.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:05:07.901 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:05:08.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.000+0000 7f8ab2563640 1 Processor -- start 2024-04-24T23:05:08.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.000+0000 7f8ab2563640 1 -- start start 2024-04-24T23:05:08.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.001+0000 7f8ab2563640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8aac0cc790 0x7f8aac0b52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:08.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.001+0000 7f8ab2563640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8aac15f7d0 0x7f8aac0b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:08.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.001+0000 7f8ab2563640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8aac0b4b40 con 0x7f8aac0b3e20 2024-04-24T23:05:08.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.001+0000 7f8ab2563640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f8aac0b58b0 con 0x7f8aac0cc790 2024-04-24T23:05:08.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.001+0000 7f8ab2563640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f8aac0b5b90 con 0x7f8aac15f7d0 2024-04-24T23:05:08.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.001+0000 7f8aabfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8aac0cc790 0x7f8aac0b52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:08.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.001+0000 7f8aabfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8aac0cc790 0x7f8aac0b52d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52800/0 (socket says 172.21.15.32:52800) 2024-04-24T23:05:08.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.001+0000 7f8aabfff640 1 -- 172.21.15.32:0/3918709825 learned_addr learned my addr 172.21.15.32:0/3918709825 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:08.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.001+0000 7f8ab0b00640 1 --1- 172.21.15.32:0/3918709825 >> v1:172.21.15.32:6789/0 conn(0x7f8aac0b3e20 0x7f8aac06c5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56462/0 (socket says 172.21.15.32:56462) 2024-04-24T23:05:08.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.001+0000 7f8aab7fe640 1 --2- 172.21.15.32:0/3918709825 >> v2:172.21.15.121:3300/0 conn(0x7f8aac15f7d0 0x7f8aac0b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:08.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.001+0000 7f8aabfff640 1 -- 172.21.15.32:0/3918709825 >> v2:172.21.15.121:3300/0 conn(0x7f8aac15f7d0 msgr2=0x7f8aac0b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:08.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.001+0000 7f8aabfff640 1 --2- 172.21.15.32:0/3918709825 >> v2:172.21.15.121:3300/0 conn(0x7f8aac15f7d0 0x7f8aac0b38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:08.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.001+0000 7f8aabfff640 1 -- 172.21.15.32:0/3918709825 >> v1:172.21.15.32:6789/0 conn(0x7f8aac0b3e20 legacy=0x7f8aac06c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:08.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.001+0000 7f8aabfff640 1 -- 172.21.15.32:0/3918709825 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aac15fc60 con 0x7f8aac0cc790 2024-04-24T23:05:08.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.002+0000 7f8aabfff640 1 --2- 172.21.15.32:0/3918709825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8aac0cc790 0x7f8aac0b52d0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f8aac0cf440 tx=0x7f8aa002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a31cf75878f47d96 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:08.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.002+0000 7f8aaaffd640 1 -- 172.21.15.32:0/3918709825 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa0037070 con 0x7f8aac0cc790 2024-04-24T23:05:08.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.002+0000 7f8aaaffd640 1 -- 172.21.15.32:0/3918709825 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8aa0032440 con 0x7f8aac0cc790 2024-04-24T23:05:08.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.002+0000 7f8aaaffd640 1 -- 172.21.15.32:0/3918709825 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa0032810 con 0x7f8aac0cc790 2024-04-24T23:05:08.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.002+0000 7f8ab2563640 1 -- 172.21.15.32:0/3918709825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8aac0cc790 msgr2=0x7f8aac0b52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:08.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.002+0000 7f8ab2563640 1 --2- 172.21.15.32:0/3918709825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8aac0cc790 0x7f8aac0b52d0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f8aac0cf440 tx=0x7f8aa002a270 comp rx=0 tx=0).stop 2024-04-24T23:05:08.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.003+0000 7f8ab2563640 1 -- 172.21.15.32:0/3918709825 shutdown_connections 2024-04-24T23:05:08.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.003+0000 7f8ab2563640 1 --2- 172.21.15.32:0/3918709825 >> v2:172.21.15.121:3300/0 conn(0x7f8aac15f7d0 0x7f8aac0b38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:08.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.003+0000 7f8ab2563640 1 --2- 172.21.15.32:0/3918709825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8aac0cc790 0x7f8aac0b52d0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:08.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.003+0000 7f8ab2563640 1 -- 172.21.15.32:0/3918709825 >> 172.21.15.32:0/3918709825 conn(0x7f8aac06b110 msgr2=0x7f8aac06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:08.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.003+0000 7f8ab2563640 1 -- 172.21.15.32:0/3918709825 shutdown_connections 2024-04-24T23:05:08.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.003+0000 7f8ab2563640 1 -- 172.21.15.32:0/3918709825 wait complete. 2024-04-24T23:05:08.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.004+0000 7f8ab2563640 1 Processor -- start 2024-04-24T23:05:08.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.004+0000 7f8ab2563640 1 -- start start 2024-04-24T23:05:08.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.005+0000 7f8ab2563640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8aac0cc790 0x7f8aac071d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:08.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.005+0000 7f8ab2563640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8aac15f7d0 0x7f8aac1498f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:08.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.005+0000 7f8ab2563640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8aac149e30 0x7f8aac0c53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:08.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.005+0000 7f8ab2563640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8aac0b6150 con 0x7f8aac15f7d0 2024-04-24T23:05:08.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.005+0000 7f8ab2563640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8aac0ae950 con 0x7f8aac149e30 2024-04-24T23:05:08.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.005+0000 7f8ab2563640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f8aac0b62d0 con 0x7f8aac0cc790 2024-04-24T23:05:08.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.005+0000 7f8aab7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8aac15f7d0 0x7f8aac1498f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:08.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.005+0000 7f8aab7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8aac15f7d0 0x7f8aac1498f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52814/0 (socket says 172.21.15.32:52814) 2024-04-24T23:05:08.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.005+0000 7f8aab7fe640 1 -- 172.21.15.32:0/1280563259 learned_addr learned my addr 172.21.15.32:0/1280563259 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:08.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.005+0000 7f8ab0b00640 1 --2- 172.21.15.32:0/1280563259 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8aac149e30 0x7f8aac0c53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:08.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.005+0000 7f8aabfff640 1 --2- 172.21.15.32:0/1280563259 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8aac0cc790 0x7f8aac071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:08.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.005+0000 7f8aab7fe640 1 -- 172.21.15.32:0/1280563259 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8aac0cc790 msgr2=0x7f8aac071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:08.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.005+0000 7f8aab7fe640 1 --2- 172.21.15.32:0/1280563259 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8aac0cc790 0x7f8aac071d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:08.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.005+0000 7f8aab7fe640 1 -- 172.21.15.32:0/1280563259 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8aac149e30 msgr2=0x7f8aac0c53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:08.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.005+0000 7f8aab7fe640 1 --2- 172.21.15.32:0/1280563259 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8aac149e30 0x7f8aac0c53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:08.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.005+0000 7f8aab7fe640 1 -- 172.21.15.32:0/1280563259 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aac0b00a0 con 0x7f8aac15f7d0 2024-04-24T23:05:08.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.005+0000 7f8ab0b00640 1 --2- 172.21.15.32:0/1280563259 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8aac149e30 0x7f8aac0c53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:05:08.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.005+0000 7f8aab7fe640 1 --2- 172.21.15.32:0/1280563259 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8aac15f7d0 0x7f8aac1498f0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c00e9b0 tx=0x7f8a9c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:08.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.005+0000 7f8aa8ff9640 1 -- 172.21.15.32:0/1280563259 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c017070 con 0x7f8aac15f7d0 2024-04-24T23:05:08.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.005+0000 7f8aa8ff9640 1 -- 172.21.15.32:0/1280563259 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a9c00c9a0 con 0x7f8aac15f7d0 2024-04-24T23:05:08.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.005+0000 7f8aa8ff9640 1 -- 172.21.15.32:0/1280563259 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c00cd90 con 0x7f8aac15f7d0 2024-04-24T23:05:08.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.005+0000 7f8ab2563640 1 -- 172.21.15.32:0/1280563259 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8aac0c6100 con 0x7f8aac15f7d0 2024-04-24T23:05:08.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.005+0000 7f8ab2563640 1 -- 172.21.15.32:0/1280563259 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8aac0b0340 con 0x7f8aac15f7d0 2024-04-24T23:05:08.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.006+0000 7f8ab2563640 1 -- 172.21.15.32:0/1280563259 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8aac068430 con 0x7f8aac15f7d0 2024-04-24T23:05:08.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.007+0000 7f8aa8ff9640 1 -- 172.21.15.32:0/1280563259 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8a9c01a070 con 0x7f8aac15f7d0 2024-04-24T23:05:08.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.007+0000 7f8aa8ff9640 1 --2- 172.21.15.32:0/1280563259 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8a9806be60 0x7f8a9806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:08.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.007+0000 7f8aa8ff9640 1 -- 172.21.15.32:0/1280563259 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5709+0+0 (secure 0 0 0) 0x7f8a9c08aee0 con 0x7f8aac15f7d0 2024-04-24T23:05:08.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.007+0000 7f8aabfff640 1 --2- 172.21.15.32:0/1280563259 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8a9806be60 0x7f8a9806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:08.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.008+0000 7f8aabfff640 1 --2- 172.21.15.32:0/1280563259 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8a9806be60 0x7f8a9806e320 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0034000 tx=0x7f8aa002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:08.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.011+0000 7f8aa8ff9640 1 -- 172.21.15.32:0/1280563259 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8a9c059800 con 0x7f8aac15f7d0 2024-04-24T23:05:08.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.231+0000 7f8ab2563640 1 -- 172.21.15.32:0/1280563259 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8aac06d2d0 con 0x7f8aac15f7d0 2024-04-24T23:05:08.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.231+0000 7f8aa8ff9640 1 -- 172.21.15.32:0/1280563259 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+5189 (secure 0 0 0) 0x7f8a9c059100 con 0x7f8aac15f7d0 2024-04-24T23:05:08.233 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:05:08.233 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":51,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5972,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3336404028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3336404028},{"type":"v1","addr":"172.21.15.121:6837","nonce":3336404028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6505,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/1068557774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1068557774},{"type":"v1","addr":"172.21.15.121:6836","nonce":1068557774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":7353,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/2118719195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":2118719195},{"type":"v1","addr":"172.21.15.32:6834","nonce":2118719195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:04:55.998022+0000","modified":"2024-04-24T23:04:58.657258+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7356},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7356":{"gid":7356,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/3015290487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":3015290487},{"type":"v1","addr":"172.21.15.32:6835","nonce":3015290487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7356,"qdb_cluster":[7356]},"id":7}]} 2024-04-24T23:05:08.233 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 51 2024-04-24T23:05:08.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.233+0000 7f8ab2563640 1 -- 172.21.15.32:0/1280563259 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8a9806be60 msgr2=0x7f8a9806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:08.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.233+0000 7f8ab2563640 1 --2- 172.21.15.32:0/1280563259 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8a9806be60 0x7f8a9806e320 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0034000 tx=0x7f8aa002e000 comp rx=0 tx=0).stop 2024-04-24T23:05:08.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.233+0000 7f8ab2563640 1 -- 172.21.15.32:0/1280563259 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8aac15f7d0 msgr2=0x7f8aac1498f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:08.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.233+0000 7f8ab2563640 1 --2- 172.21.15.32:0/1280563259 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8aac15f7d0 0x7f8aac1498f0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c00e9b0 tx=0x7f8a9c00ee70 comp rx=0 tx=0).stop 2024-04-24T23:05:08.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.233+0000 7f8ab2563640 1 -- 172.21.15.32:0/1280563259 shutdown_connections 2024-04-24T23:05:08.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.233+0000 7f8ab2563640 1 --2- 172.21.15.32:0/1280563259 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8a9806be60 0x7f8a9806e320 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:08.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.233+0000 7f8ab2563640 1 --2- 172.21.15.32:0/1280563259 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8aac15f7d0 0x7f8aac1498f0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:08.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.233+0000 7f8ab2563640 1 --2- 172.21.15.32:0/1280563259 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8aac149e30 0x7f8aac0c53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:08.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.233+0000 7f8ab2563640 1 --2- 172.21.15.32:0/1280563259 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8aac0cc790 0x7f8aac071d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:08.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.233+0000 7f8ab2563640 1 -- 172.21.15.32:0/1280563259 >> 172.21.15.32:0/1280563259 conn(0x7f8aac06b110 msgr2=0x7f8aac06b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:08.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.233+0000 7f8ab2563640 1 -- 172.21.15.32:0/1280563259 shutdown_connections 2024-04-24T23:05:08.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08.233+0000 7f8ab2563640 1 -- 172.21.15.32:0/1280563259 wait complete. 2024-04-24T23:05:08.246 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json osdmap barrier 128 2024-04-24T23:05:08.364 DEBUG:tasks.cephfs.filesystem:_json_asok output empty 2024-04-24T23:05:08.365 DEBUG:teuthology.orchestra.run.smithi032:> stdin-killer -- python3 -c ' 2024-04-24T23:05:08.365 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-24T23:05:08.365 DEBUG:teuthology.orchestra.run.smithi032:> fd = os.open("/home/ubuntu/cephtest/mnt.0/baz", os.O_CREAT | os.O_RDWR, 0O600) 2024-04-24T23:05:08.365 DEBUG:teuthology.orchestra.run.smithi032:> os.fchmod(fd, 0O666) 2024-04-24T23:05:08.365 DEBUG:teuthology.orchestra.run.smithi032:> os.fsync(fd) 2024-04-24T23:05:08.365 DEBUG:teuthology.orchestra.run.smithi032:> os.close(fd) 2024-04-24T23:05:08.365 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-24T23:05:08.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:05:08.445 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:05:08.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:05:08.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:05:08.543 INFO:teuthology.orchestra.run.smithi032.stdout:client6047 2024-04-24T23:05:08.543 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/eb1b7a01-6f65-4534-9a12-83011862ba51.client6047/osdmap') 2024-04-24T23:05:08.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:05:08.601 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 128 barrier 128 flags 0x588000 2024-04-24T23:05:08.601 INFO:teuthology.orchestra.run.smithi032.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-04-24T23:05:08.602 INFO:teuthology.orchestra.run.smithi032.stdout:pool 14 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-04-24T23:05:08.602 INFO:teuthology.orchestra.run.smithi032.stdout:pool 15 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-04-24T23:05:08.602 INFO:teuthology.orchestra.run.smithi032.stdout:osd0 (2)172.21.15.32:6800 100% (exists, up) 100% -1 2024-04-24T23:05:08.602 INFO:teuthology.orchestra.run.smithi032.stdout:osd1 (2)172.21.15.32:6808 100% (exists, up) 100% -1 2024-04-24T23:05:08.602 INFO:teuthology.orchestra.run.smithi032.stdout:osd2 (2)172.21.15.32:6816 100% (exists, up) 100% -1 2024-04-24T23:05:08.602 INFO:teuthology.orchestra.run.smithi032.stdout:osd3 (2)172.21.15.32:6818 100% (exists, up) 100% -1 2024-04-24T23:05:08.602 INFO:teuthology.orchestra.run.smithi032.stdout:osd4 (2)172.21.15.121:6800 100% (exists, up) 100% -1 2024-04-24T23:05:08.602 INFO:teuthology.orchestra.run.smithi032.stdout:osd5 (2)172.21.15.121:6808 100% (exists, up) 100% -1 2024-04-24T23:05:08.602 INFO:teuthology.orchestra.run.smithi032.stdout:osd6 (2)172.21.15.121:6823 100% (exists, up) 100% -1 2024-04-24T23:05:08.602 INFO:teuthology.orchestra.run.smithi032.stdout:osd7 (2)172.21.15.121:6816 100% (exists, up) 100% -1 2024-04-24T23:05:08.603 INFO:teuthology.orchestra.run.smithi032.stderr:1+1 records in 2024-04-24T23:05:08.603 INFO:teuthology.orchestra.run.smithi032.stderr:1+1 records out 2024-04-24T23:05:08.603 INFO:teuthology.orchestra.run.smithi032.stderr:799 bytes copied, 3.5743e-05 s, 22.4 MB/s 2024-04-24T23:05:08.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:05:08.611 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:05:08.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:05:08.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:05:08.668 INFO:teuthology.orchestra.run.smithi032.stdout:client6047 2024-04-24T23:05:08.668 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/eb1b7a01-6f65-4534-9a12-83011862ba51.client6047/osdmap') 2024-04-24T23:05:08.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:05:08.726 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 128 barrier 128 flags 0x588000 2024-04-24T23:05:08.726 INFO:teuthology.orchestra.run.smithi032.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-04-24T23:05:08.727 INFO:teuthology.orchestra.run.smithi032.stdout:pool 14 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-04-24T23:05:08.727 INFO:teuthology.orchestra.run.smithi032.stdout:pool 15 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-04-24T23:05:08.727 INFO:teuthology.orchestra.run.smithi032.stdout:osd0 (2)172.21.15.32:6800 100% (exists, up) 100% -1 2024-04-24T23:05:08.727 INFO:teuthology.orchestra.run.smithi032.stdout:osd1 (2)172.21.15.32:6808 100% (exists, up) 100% -1 2024-04-24T23:05:08.727 INFO:teuthology.orchestra.run.smithi032.stdout:osd2 (2)172.21.15.32:6816 100% (exists, up) 100% -1 2024-04-24T23:05:08.727 INFO:teuthology.orchestra.run.smithi032.stdout:osd3 (2)172.21.15.32:6818 100% (exists, up) 100% -1 2024-04-24T23:05:08.727 INFO:teuthology.orchestra.run.smithi032.stdout:osd4 (2)172.21.15.121:6800 100% (exists, up) 100% -1 2024-04-24T23:05:08.727 INFO:teuthology.orchestra.run.smithi032.stdout:osd5 (2)172.21.15.121:6808 100% (exists, up) 100% -1 2024-04-24T23:05:08.727 INFO:teuthology.orchestra.run.smithi032.stdout:osd6 (2)172.21.15.121:6823 100% (exists, up) 100% -1 2024-04-24T23:05:08.727 INFO:teuthology.orchestra.run.smithi032.stdout:osd7 (2)172.21.15.121:6816 100% (exists, up) 100% -1 2024-04-24T23:05:08.728 INFO:teuthology.orchestra.run.smithi032.stderr:1+1 records in 2024-04-24T23:05:08.728 INFO:teuthology.orchestra.run.smithi032.stderr:1+1 records out 2024-04-24T23:05:08.728 INFO:teuthology.orchestra.run.smithi032.stderr:799 bytes copied, 3.63e-05 s, 22.0 MB/s 2024-04-24T23:05:08.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:05:08.734 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-24T23:05:08.734 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-24T23:05:08.735 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:05:08.781 DEBUG:teuthology.orchestra.run.smithi121:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:05:08.837 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-24T23:05:08.837 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:05:08.837 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:05:08.837 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:05:08.853 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:05:08.853 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:05:08.909 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:05:09.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.049+0000 7fde42076640 1 Processor -- start 2024-04-24T23:05:09.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.049+0000 7fde42076640 1 -- start start 2024-04-24T23:05:09.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.050+0000 7fde42076640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fde3c15e9b0 0x7fde3c0d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:09.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.050+0000 7fde42076640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fde3c0d4210 0x7fde3c06b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:09.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.050+0000 7fde42076640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fde3c0ba0e0 0x7fde3c06b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:09.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.050+0000 7fde42076640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fde3c0d3170 con 0x7fde3c0d4210 2024-04-24T23:05:09.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.050+0000 7fde42076640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fde3c0d3450 con 0x7fde3c0ba0e0 2024-04-24T23:05:09.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.050+0000 7fde42076640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fde3c0d3730 con 0x7fde3c15e9b0 2024-04-24T23:05:09.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.050+0000 7fde3affd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fde3c0d4210 0x7fde3c06b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:09.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.050+0000 7fde3affd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fde3c0d4210 0x7fde3c06b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52818/0 (socket says 172.21.15.32:52818) 2024-04-24T23:05:09.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.050+0000 7fde3affd640 1 -- 172.21.15.32:0/1588004310 learned_addr learned my addr 172.21.15.32:0/1588004310 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:09.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.050+0000 7fde3bfff640 1 --2- 172.21.15.32:0/1588004310 >> v2:172.21.15.121:3300/0 conn(0x7fde3c0ba0e0 0x7fde3c06b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:09.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.050+0000 7fde3b7fe640 1 --2- 172.21.15.32:0/1588004310 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fde3c15e9b0 0x7fde3c0d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:09.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.050+0000 7fde3affd640 1 -- 172.21.15.32:0/1588004310 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fde3c15e9b0 msgr2=0x7fde3c0d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:09.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.050+0000 7fde3affd640 1 --2- 172.21.15.32:0/1588004310 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fde3c15e9b0 0x7fde3c0d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:09.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.050+0000 7fde3affd640 1 -- 172.21.15.32:0/1588004310 >> v2:172.21.15.121:3300/0 conn(0x7fde3c0ba0e0 msgr2=0x7fde3c06b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:09.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.050+0000 7fde3affd640 1 --2- 172.21.15.32:0/1588004310 >> v2:172.21.15.121:3300/0 conn(0x7fde3c0ba0e0 0x7fde3c06b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:09.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.050+0000 7fde3affd640 1 -- 172.21.15.32:0/1588004310 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde3c0d3a10 con 0x7fde3c0d4210 2024-04-24T23:05:09.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.050+0000 7fde3b7fe640 1 --2- 172.21.15.32:0/1588004310 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fde3c15e9b0 0x7fde3c0d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:05:09.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.051+0000 7fde3affd640 1 --2- 172.21.15.32:0/1588004310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde3c0d4210 0x7fde3c06b110 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fde3c0b61d0 tx=0x7fde3002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63d3f3afd0f8eddf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:09.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.051+0000 7fde3a7fc640 1 -- 172.21.15.32:0/1588004310 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde30037070 con 0x7fde3c0d4210 2024-04-24T23:05:09.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.051+0000 7fde3a7fc640 1 -- 172.21.15.32:0/1588004310 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde30032440 con 0x7fde3c0d4210 2024-04-24T23:05:09.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.051+0000 7fde3a7fc640 1 -- 172.21.15.32:0/1588004310 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde30032810 con 0x7fde3c0d4210 2024-04-24T23:05:09.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.052+0000 7fde42076640 1 -- 172.21.15.32:0/1588004310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde3c0d4210 msgr2=0x7fde3c06b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:09.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.052+0000 7fde42076640 1 --2- 172.21.15.32:0/1588004310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde3c0d4210 0x7fde3c06b110 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fde3c0b61d0 tx=0x7fde3002a1e0 comp rx=0 tx=0).stop 2024-04-24T23:05:09.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.052+0000 7fde42076640 1 -- 172.21.15.32:0/1588004310 shutdown_connections 2024-04-24T23:05:09.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.052+0000 7fde42076640 1 --2- 172.21.15.32:0/1588004310 >> v2:172.21.15.121:3300/0 conn(0x7fde3c0ba0e0 0x7fde3c06b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:09.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.052+0000 7fde42076640 1 --2- 172.21.15.32:0/1588004310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde3c0d4210 0x7fde3c06b110 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:09.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.052+0000 7fde42076640 1 --2- 172.21.15.32:0/1588004310 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fde3c15e9b0 0x7fde3c0d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:09.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.052+0000 7fde42076640 1 -- 172.21.15.32:0/1588004310 >> 172.21.15.32:0/1588004310 conn(0x7fde3c0d11d0 msgr2=0x7fde3c0d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:09.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.052+0000 7fde42076640 1 -- 172.21.15.32:0/1588004310 shutdown_connections 2024-04-24T23:05:09.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.052+0000 7fde42076640 1 -- 172.21.15.32:0/1588004310 wait complete. 2024-04-24T23:05:09.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.053+0000 7fde42076640 1 Processor -- start 2024-04-24T23:05:09.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.053+0000 7fde42076640 1 -- start start 2024-04-24T23:05:09.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.054+0000 7fde42076640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde3c0ba0e0 0x7fde3c15ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:09.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.054+0000 7fde42076640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fde3c0d4210 0x7fde3c0cf240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:09.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.054+0000 7fde42076640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fde3c15e9b0 0x7fde3c153fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:09.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.054+0000 7fde42076640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fde3c071730 con 0x7fde3c0ba0e0 2024-04-24T23:05:09.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.054+0000 7fde42076640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fde3c0d2400 con 0x7fde3c15e9b0 2024-04-24T23:05:09.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.054+0000 7fde42076640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fde3c0718b0 con 0x7fde3c0d4210 2024-04-24T23:05:09.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.054+0000 7fde3b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde3c0ba0e0 0x7fde3c15ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:09.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.054+0000 7fde3b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde3c0ba0e0 0x7fde3c15ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52830/0 (socket says 172.21.15.32:52830) 2024-04-24T23:05:09.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.054+0000 7fde3b7fe640 1 -- 172.21.15.32:0/3278401828 learned_addr learned my addr 172.21.15.32:0/3278401828 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:09.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.054+0000 7fde3bfff640 1 --2- 172.21.15.32:0/3278401828 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fde3c15e9b0 0x7fde3c153fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:09.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.054+0000 7fde3affd640 1 --2- 172.21.15.32:0/3278401828 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fde3c0d4210 0x7fde3c0cf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:09.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.054+0000 7fde3b7fe640 1 -- 172.21.15.32:0/3278401828 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fde3c0d4210 msgr2=0x7fde3c0cf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:09.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.054+0000 7fde3b7fe640 1 --2- 172.21.15.32:0/3278401828 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fde3c0d4210 0x7fde3c0cf240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:09.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.054+0000 7fde3b7fe640 1 -- 172.21.15.32:0/3278401828 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fde3c15e9b0 msgr2=0x7fde3c153fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:09.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.054+0000 7fde3b7fe640 1 --2- 172.21.15.32:0/3278401828 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fde3c15e9b0 0x7fde3c153fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:09.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.054+0000 7fde3b7fe640 1 -- 172.21.15.32:0/3278401828 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde3c0ceb90 con 0x7fde3c0ba0e0 2024-04-24T23:05:09.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.054+0000 7fde3b7fe640 1 --2- 172.21.15.32:0/3278401828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde3c0ba0e0 0x7fde3c15ed90 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fde2c00c7f0 tx=0x7fde2c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:09.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.054+0000 7fde23fff640 1 -- 172.21.15.32:0/3278401828 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde2c015070 con 0x7fde3c0ba0e0 2024-04-24T23:05:09.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.054+0000 7fde23fff640 1 -- 172.21.15.32:0/3278401828 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde2c007820 con 0x7fde3c0ba0e0 2024-04-24T23:05:09.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.055+0000 7fde42076640 1 -- 172.21.15.32:0/3278401828 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde3c15f470 con 0x7fde3c0ba0e0 2024-04-24T23:05:09.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.055+0000 7fde23fff640 1 -- 172.21.15.32:0/3278401828 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde2c007bd0 con 0x7fde3c0ba0e0 2024-04-24T23:05:09.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.055+0000 7fde42076640 1 -- 172.21.15.32:0/3278401828 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde3c15fa30 con 0x7fde3c0ba0e0 2024-04-24T23:05:09.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.056+0000 7fde42076640 1 -- 172.21.15.32:0/3278401828 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde3c068430 con 0x7fde3c0ba0e0 2024-04-24T23:05:09.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.056+0000 7fde23fff640 1 -- 172.21.15.32:0/3278401828 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fde2c018070 con 0x7fde3c0ba0e0 2024-04-24T23:05:09.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.057+0000 7fde23fff640 1 --2- 172.21.15.32:0/3278401828 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fde2806be30 0x7fde2806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:09.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.057+0000 7fde23fff640 1 -- 172.21.15.32:0/3278401828 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5709+0+0 (secure 0 0 0) 0x7fde2c08af50 con 0x7fde3c0ba0e0 2024-04-24T23:05:09.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.057+0000 7fde3affd640 1 --2- 172.21.15.32:0/3278401828 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fde2806be30 0x7fde2806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:09.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.057+0000 7fde3affd640 1 --2- 172.21.15.32:0/3278401828 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fde2806be30 0x7fde2806e2f0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fde30034000 tx=0x7fde3002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:09.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.061+0000 7fde23fff640 1 -- 172.21.15.32:0/3278401828 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fde3c0ceb90 con 0x7fde3c0ba0e0 2024-04-24T23:05:09.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.245+0000 7fde42076640 1 -- 172.21.15.32:0/3278401828 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fde3c0ccc30 con 0x7fde3c0ba0e0 2024-04-24T23:05:09.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.246+0000 7fde23fff640 1 -- 172.21.15.32:0/3278401828 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) ==== 74+0+16639 (secure 0 0 0) 0x7fde2c059210 con 0x7fde3c0ba0e0 2024-04-24T23:05:09.248 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:05:09.248 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":128,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:05:07.320153+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:04:54.280785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-24T23:04:54.966294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:05:09.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.249+0000 7fde42076640 1 -- 172.21.15.32:0/3278401828 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fde2806be30 msgr2=0x7fde2806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:09.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.249+0000 7fde42076640 1 --2- 172.21.15.32:0/3278401828 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fde2806be30 0x7fde2806e2f0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fde30034000 tx=0x7fde3002e000 comp rx=0 tx=0).stop 2024-04-24T23:05:09.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.249+0000 7fde42076640 1 -- 172.21.15.32:0/3278401828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde3c0ba0e0 msgr2=0x7fde3c15ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:09.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.249+0000 7fde42076640 1 --2- 172.21.15.32:0/3278401828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde3c0ba0e0 0x7fde3c15ed90 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fde2c00c7f0 tx=0x7fde2c00ccb0 comp rx=0 tx=0).stop 2024-04-24T23:05:09.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.249+0000 7fde42076640 1 -- 172.21.15.32:0/3278401828 shutdown_connections 2024-04-24T23:05:09.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.249+0000 7fde42076640 1 --2- 172.21.15.32:0/3278401828 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fde2806be30 0x7fde2806e2f0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:09.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.249+0000 7fde42076640 1 --2- 172.21.15.32:0/3278401828 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fde3c15e9b0 0x7fde3c153fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:09.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.249+0000 7fde42076640 1 --2- 172.21.15.32:0/3278401828 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fde3c0d4210 0x7fde3c0cf240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:09.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.249+0000 7fde42076640 1 --2- 172.21.15.32:0/3278401828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde3c0ba0e0 0x7fde3c15ed90 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:09.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.249+0000 7fde42076640 1 -- 172.21.15.32:0/3278401828 >> 172.21.15.32:0/3278401828 conn(0x7fde3c0d11d0 msgr2=0x7fde3c155cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:09.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.249+0000 7fde42076640 1 -- 172.21.15.32:0/3278401828 shutdown_connections 2024-04-24T23:05:09.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.249+0000 7fde42076640 1 -- 172.21.15.32:0/3278401828 wait complete. 2024-04-24T23:05:09.261 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:04:54.280785+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '124', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:04:54.966294+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:05:09.261 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:05:09.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.364+0000 7fa9b5189640 1 Processor -- start 2024-04-24T23:05:09.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.364+0000 7fa9b5189640 1 -- start start 2024-04-24T23:05:09.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.365+0000 7fa9b5189640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa9b006ba30 0x7fa9b006d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:09.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.365+0000 7fa9b5189640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9b015ef10 con 0x7fa9b006da90 2024-04-24T23:05:09.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.365+0000 7fa9b5189640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9b015f1f0 con 0x7fa9b0158dd0 2024-04-24T23:05:09.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.365+0000 7fa9b5189640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fa9b015f4d0 con 0x7fa9b006ba30 2024-04-24T23:05:09.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.365+0000 7fa9af59e640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa9b006da90 0x7fa9b00b5d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56478/0 (socket says 172.21.15.32:56478) 2024-04-24T23:05:09.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.365+0000 7fa9af59e640 1 -- 172.21.15.32:0/678241550 learned_addr learned my addr 172.21.15.32:0/678241550 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:05:09.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.365+0000 7fa9a7fff640 1 --2- 172.21.15.32:0/678241550 >> v2:172.21.15.121:3300/0 conn(0x7fa9b006ba30 0x7fa9b006d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:09.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.365+0000 7fa9ae59c640 1 -- 172.21.15.32:0/678241550 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2512677559 0 0) 0x7fa9b015ef10 con 0x7fa9b006da90 2024-04-24T23:05:09.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.365+0000 7fa9ae59c640 1 -- 172.21.15.32:0/678241550 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa99c003520 con 0x7fa9b006da90 2024-04-24T23:05:09.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.365+0000 7fa9a7fff640 1 -- 172.21.15.32:0/678241550 >> v1:172.21.15.121:6789/0 conn(0x7fa9b0158dd0 legacy=0x7fa9b006b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:09.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.365+0000 7fa9a7fff640 1 -- 172.21.15.32:0/678241550 >> v1:172.21.15.32:6789/0 conn(0x7fa9b006da90 legacy=0x7fa9b00b5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:09.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.365+0000 7fa9ae59c640 1 -- 172.21.15.32:0/678241550 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 772269323 0 0) 0x7fa99c003520 con 0x7fa9b006da90 2024-04-24T23:05:09.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.365+0000 7fa9a7fff640 1 -- 172.21.15.32:0/678241550 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9b015f7b0 con 0x7fa9b006ba30 2024-04-24T23:05:09.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.366+0000 7fa9a7fff640 1 --2- 172.21.15.32:0/678241550 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9b006ba30 0x7fa9b006d550 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa9980097a0 tx=0x7fa99802a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=697d8e7fbe3823ce server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:09.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.366+0000 7fa9ae59c640 1 -- 172.21.15.32:0/678241550 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa998037070 con 0x7fa9b006ba30 2024-04-24T23:05:09.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.366+0000 7fa9ae59c640 1 -- 172.21.15.32:0/678241550 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa99802ac50 con 0x7fa9b006ba30 2024-04-24T23:05:09.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.366+0000 7fa9ae59c640 1 -- 172.21.15.32:0/678241550 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a80019f0 con 0x7fa9b006ba30 2024-04-24T23:05:09.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.367+0000 7fa9b5189640 1 -- 172.21.15.32:0/678241550 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9b006ba30 msgr2=0x7fa9b006d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:09.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.367+0000 7fa9b5189640 1 --2- 172.21.15.32:0/678241550 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9b006ba30 0x7fa9b006d550 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa9980097a0 tx=0x7fa99802a1c0 comp rx=0 tx=0).stop 2024-04-24T23:05:09.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.367+0000 7fa9b5189640 1 -- 172.21.15.32:0/678241550 shutdown_connections 2024-04-24T23:05:09.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.367+0000 7fa9b5189640 1 --2- 172.21.15.32:0/678241550 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9b006ba30 0x7fa9b006d550 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:09.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.367+0000 7fa9b5189640 1 -- 172.21.15.32:0/678241550 >> 172.21.15.32:0/678241550 conn(0x7fa9b006e800 msgr2=0x7fa9b006ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:09.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.367+0000 7fa9b5189640 1 -- 172.21.15.32:0/678241550 shutdown_connections 2024-04-24T23:05:09.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.367+0000 7fa9b5189640 1 -- 172.21.15.32:0/678241550 wait complete. 2024-04-24T23:05:09.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.368+0000 7fa9b5189640 1 Processor -- start 2024-04-24T23:05:09.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.368+0000 7fa9b5189640 1 -- start start 2024-04-24T23:05:09.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.369+0000 7fa9b5189640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9b006ba30 0x7fa9b014f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:09.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.369+0000 7fa9b5189640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9b006da90 0x7fa9b01416c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:09.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.369+0000 7fa9b5189640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9b0158dd0 0x7fa9b01f78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:09.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.369+0000 7fa9b5189640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b006bfe0 con 0x7fa9b0158dd0 2024-04-24T23:05:09.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.369+0000 7fa9b5189640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b006ded0 con 0x7fa9b006da90 2024-04-24T23:05:09.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.369+0000 7fa9b5189640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa9b01591b0 con 0x7fa9b006ba30 2024-04-24T23:05:09.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.369+0000 7fa9af59e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9b0158dd0 0x7fa9b01f78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:09.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.369+0000 7fa9af59e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9b0158dd0 0x7fa9b01f78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52846/0 (socket says 172.21.15.32:52846) 2024-04-24T23:05:09.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.369+0000 7fa9af59e640 1 -- 172.21.15.32:0/2963907130 learned_addr learned my addr 172.21.15.32:0/2963907130 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:09.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.369+0000 7fa9a7fff640 1 --2- 172.21.15.32:0/2963907130 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9b006da90 0x7fa9b01416c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:09.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.369+0000 7fa9aed9d640 1 --2- 172.21.15.32:0/2963907130 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9b006ba30 0x7fa9b014f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:09.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.369+0000 7fa9af59e640 1 -- 172.21.15.32:0/2963907130 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9b006ba30 msgr2=0x7fa9b014f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:09.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.369+0000 7fa9af59e640 1 --2- 172.21.15.32:0/2963907130 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9b006ba30 0x7fa9b014f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:09.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.369+0000 7fa9af59e640 1 -- 172.21.15.32:0/2963907130 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9b006da90 msgr2=0x7fa9b01416c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:09.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.369+0000 7fa9af59e640 1 --2- 172.21.15.32:0/2963907130 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9b006da90 0x7fa9b01416c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:09.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.369+0000 7fa9af59e640 1 -- 172.21.15.32:0/2963907130 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9b0141ef0 con 0x7fa9b0158dd0 2024-04-24T23:05:09.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.369+0000 7fa9a7fff640 1 --2- 172.21.15.32:0/2963907130 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9b006da90 0x7fa9b01416c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:05:09.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.369+0000 7fa9aed9d640 1 --2- 172.21.15.32:0/2963907130 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9b006ba30 0x7fa9b014f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:05:09.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.370+0000 7fa9af59e640 1 --2- 172.21.15.32:0/2963907130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9b0158dd0 0x7fa9b01f78a0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8001b60 tx=0x7fa9a800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:09.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.370+0000 7fa9a77fe640 1 -- 172.21.15.32:0/2963907130 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a8017070 con 0x7fa9b0158dd0 2024-04-24T23:05:09.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.370+0000 7fa9a77fe640 1 -- 172.21.15.32:0/2963907130 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9a8012400 con 0x7fa9b0158dd0 2024-04-24T23:05:09.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.370+0000 7fa9b5189640 1 -- 172.21.15.32:0/2963907130 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9b01421d0 con 0x7fa9b0158dd0 2024-04-24T23:05:09.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.370+0000 7fa9a77fe640 1 -- 172.21.15.32:0/2963907130 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a80127b0 con 0x7fa9b0158dd0 2024-04-24T23:05:09.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.370+0000 7fa9b5189640 1 -- 172.21.15.32:0/2963907130 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9b0142a70 con 0x7fa9b0158dd0 2024-04-24T23:05:09.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.371+0000 7fa9b5189640 1 -- 172.21.15.32:0/2963907130 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9b01423a0 con 0x7fa9b0158dd0 2024-04-24T23:05:09.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.371+0000 7fa9a77fe640 1 -- 172.21.15.32:0/2963907130 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa9a801a070 con 0x7fa9b0158dd0 2024-04-24T23:05:09.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.372+0000 7fa9a77fe640 1 --2- 172.21.15.32:0/2963907130 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa99006be30 0x7fa99006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:09.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.372+0000 7fa9a77fe640 1 -- 172.21.15.32:0/2963907130 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5709+0+0 (secure 0 0 0) 0x7fa9a808ad70 con 0x7fa9b0158dd0 2024-04-24T23:05:09.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.372+0000 7fa9aed9d640 1 --2- 172.21.15.32:0/2963907130 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa99006be30 0x7fa99006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:09.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.372+0000 7fa9aed9d640 1 --2- 172.21.15.32:0/2963907130 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa99006be30 0x7fa99006e2f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fa9a0000f60 tx=0x7fa9a0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:09.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.376+0000 7fa9a77fe640 1 -- 172.21.15.32:0/2963907130 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa9a8059030 con 0x7fa9b0158dd0 2024-04-24T23:05:09.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.562+0000 7fa9b5189640 1 -- 172.21.15.32:0/2963907130 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa9b0068430 con 0x7fa9b0158dd0 2024-04-24T23:05:09.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.563+0000 7fa9a77fe640 1 -- 172.21.15.32:0/2963907130 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v128) ==== 93+0+10 (secure 0 0 0) 0x7fa9a805d330 con 0x7fa9b0158dd0 2024-04-24T23:05:09.564 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T23:05:09.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.565+0000 7fa9b5189640 1 -- 172.21.15.32:0/2963907130 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa99006be30 msgr2=0x7fa99006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:09.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.565+0000 7fa9b5189640 1 --2- 172.21.15.32:0/2963907130 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa99006be30 0x7fa99006e2f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fa9a0000f60 tx=0x7fa9a0009040 comp rx=0 tx=0).stop 2024-04-24T23:05:09.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.565+0000 7fa9b5189640 1 -- 172.21.15.32:0/2963907130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9b0158dd0 msgr2=0x7fa9b01f78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:09.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.565+0000 7fa9b5189640 1 --2- 172.21.15.32:0/2963907130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9b0158dd0 0x7fa9b01f78a0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8001b60 tx=0x7fa9a800a430 comp rx=0 tx=0).stop 2024-04-24T23:05:09.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.565+0000 7fa9b5189640 1 -- 172.21.15.32:0/2963907130 shutdown_connections 2024-04-24T23:05:09.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.565+0000 7fa9b5189640 1 --2- 172.21.15.32:0/2963907130 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa99006be30 0x7fa99006e2f0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:09.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.565+0000 7fa9b5189640 1 --2- 172.21.15.32:0/2963907130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9b0158dd0 0x7fa9b01f78a0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:09.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.565+0000 7fa9b5189640 1 --2- 172.21.15.32:0/2963907130 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9b006da90 0x7fa9b01416c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:09.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.565+0000 7fa9b5189640 1 --2- 172.21.15.32:0/2963907130 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9b006ba30 0x7fa9b014f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:09.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.565+0000 7fa9b5189640 1 -- 172.21.15.32:0/2963907130 >> 172.21.15.32:0/2963907130 conn(0x7fa9b006e800 msgr2=0x7fa9b01608e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:09.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.565+0000 7fa9b5189640 1 -- 172.21.15.32:0/2963907130 shutdown_connections 2024-04-24T23:05:09.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.565+0000 7fa9b5189640 1 -- 172.21.15.32:0/2963907130 wait complete. 2024-04-24T23:05:09.577 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:05:09.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.679+0000 7f3b9074f640 1 Processor -- start 2024-04-24T23:05:09.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.679+0000 7f3b9074f640 1 -- start start 2024-04-24T23:05:09.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.680+0000 7f3b9074f640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b8815e9b0 0x7f3b880d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:09.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.680+0000 7f3b9074f640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3b880d4210 0x7f3b8806b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:09.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.680+0000 7f3b9074f640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b880d3170 con 0x7f3b880ba0e0 2024-04-24T23:05:09.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.680+0000 7f3b9074f640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f3b880b8900 con 0x7f3b880d4210 2024-04-24T23:05:09.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.680+0000 7f3b9074f640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f3b880d3450 con 0x7f3b8815e9b0 2024-04-24T23:05:09.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.681+0000 7f3b8dcea640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3b880d4210 0x7f3b8806b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:09.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.681+0000 7f3b8e4eb640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b8815e9b0 0x7f3b880d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:09.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.681+0000 7f3b8e4eb640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b8815e9b0 0x7f3b880d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.32:41312/0 (socket says 172.21.15.32:41312) 2024-04-24T23:05:09.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.681+0000 7f3b8dcea640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3b880d4210 0x7f3b8806b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.32:34644/0 (socket says 172.21.15.32:34644) 2024-04-24T23:05:09.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.681+0000 7f3b8e4eb640 1 -- 172.21.15.32:0/2585315775 learned_addr learned my addr 172.21.15.32:0/2585315775 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:09.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.681+0000 7f3b8e4eb640 1 -- 172.21.15.32:0/2585315775 >> v2:172.21.15.121:3300/0 conn(0x7f3b880d4210 msgr2=0x7f3b8806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:09.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.681+0000 7f3b8e4eb640 1 --2- 172.21.15.32:0/2585315775 >> v2:172.21.15.121:3300/0 conn(0x7f3b880d4210 0x7f3b8806b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:09.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.681+0000 7f3b8e4eb640 1 -- 172.21.15.32:0/2585315775 >> v1:172.21.15.121:6789/0 conn(0x7f3b880ba0e0 legacy=0x7f3b88071730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:09.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.681+0000 7f3b8e4eb640 1 -- 172.21.15.32:0/2585315775 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b880d3730 con 0x7f3b8815e9b0 2024-04-24T23:05:09.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.681+0000 7f3b8d4e9640 1 -- 172.21.15.32:0/2585315775 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 979153319 0 0) 0x7f3b880d3170 con 0x7f3b880ba0e0 2024-04-24T23:05:09.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.681+0000 7f3b8dcea640 1 --2- 172.21.15.32:0/2585315775 >> v2:172.21.15.121:3300/0 conn(0x7f3b880d4210 0x7f3b8806b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:05:09.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.682+0000 7f3b8e4eb640 1 --2- 172.21.15.32:0/2585315775 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b8815e9b0 0x7f3b880d2b90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3b80009870 tx=0x7f3b8002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=879411f0f5fdd947 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:09.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.682+0000 7f3b8d4e9640 1 -- 172.21.15.32:0/2585315775 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b80037070 con 0x7f3b8815e9b0 2024-04-24T23:05:09.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.682+0000 7f3b8d4e9640 1 -- 172.21.15.32:0/2585315775 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b80032440 con 0x7f3b8815e9b0 2024-04-24T23:05:09.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.682+0000 7f3b8d4e9640 1 -- 172.21.15.32:0/2585315775 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b80032810 con 0x7f3b8815e9b0 2024-04-24T23:05:09.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.682+0000 7f3b9074f640 1 -- 172.21.15.32:0/2585315775 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b8815e9b0 msgr2=0x7f3b880d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:09.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.682+0000 7f3b9074f640 1 --2- 172.21.15.32:0/2585315775 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b8815e9b0 0x7f3b880d2b90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3b80009870 tx=0x7f3b8002a270 comp rx=0 tx=0).stop 2024-04-24T23:05:09.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.683+0000 7f3b9074f640 1 -- 172.21.15.32:0/2585315775 shutdown_connections 2024-04-24T23:05:09.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.683+0000 7f3b9074f640 1 --2- 172.21.15.32:0/2585315775 >> v2:172.21.15.121:3300/0 conn(0x7f3b880d4210 0x7f3b8806b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:09.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.683+0000 7f3b9074f640 1 --2- 172.21.15.32:0/2585315775 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b8815e9b0 0x7f3b880d2b90 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:09.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.683+0000 7f3b9074f640 1 -- 172.21.15.32:0/2585315775 >> 172.21.15.32:0/2585315775 conn(0x7f3b880d11d0 msgr2=0x7f3b880d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:09.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.683+0000 7f3b9074f640 1 -- 172.21.15.32:0/2585315775 shutdown_connections 2024-04-24T23:05:09.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.683+0000 7f3b9074f640 1 -- 172.21.15.32:0/2585315775 wait complete. 2024-04-24T23:05:09.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.684+0000 7f3b9074f640 1 Processor -- start 2024-04-24T23:05:09.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.684+0000 7f3b9074f640 1 -- start start 2024-04-24T23:05:09.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.684+0000 7f3b9074f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b880ba0e0 0x7f3b88151520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:09.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.685+0000 7f3b9074f640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b880d4210 0x7f3b8815f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:09.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.685+0000 7f3b9074f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b8815e9b0 0x7f3b88160d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:09.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.685+0000 7f3b9074f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3b880d2400 con 0x7f3b880ba0e0 2024-04-24T23:05:09.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.685+0000 7f3b9074f640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3b880d3cf0 con 0x7f3b8815e9b0 2024-04-24T23:05:09.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.685+0000 7f3b9074f640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f3b8806b650 con 0x7f3b880d4210 2024-04-24T23:05:09.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.685+0000 7f3b8e4eb640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b880ba0e0 0x7f3b88151520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:09.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.685+0000 7f3b8e4eb640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b880ba0e0 0x7f3b88151520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52858/0 (socket says 172.21.15.32:52858) 2024-04-24T23:05:09.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.685+0000 7f3b8e4eb640 1 -- 172.21.15.32:0/2618462865 learned_addr learned my addr 172.21.15.32:0/2618462865 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:09.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.685+0000 7f3b8dcea640 1 --2- 172.21.15.32:0/2618462865 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b880d4210 0x7f3b8815f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:09.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.685+0000 7f3b8ecec640 1 --2- 172.21.15.32:0/2618462865 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b8815e9b0 0x7f3b88160d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:09.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.685+0000 7f3b8e4eb640 1 -- 172.21.15.32:0/2618462865 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b880d4210 msgr2=0x7f3b8815f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:09.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.685+0000 7f3b8e4eb640 1 --2- 172.21.15.32:0/2618462865 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b880d4210 0x7f3b8815f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:09.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.685+0000 7f3b8e4eb640 1 -- 172.21.15.32:0/2618462865 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b8815e9b0 msgr2=0x7f3b88160d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:09.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.685+0000 7f3b8e4eb640 1 --2- 172.21.15.32:0/2618462865 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b8815e9b0 0x7f3b88160d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:09.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.685+0000 7f3b8e4eb640 1 -- 172.21.15.32:0/2618462865 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b880cae30 con 0x7f3b880ba0e0 2024-04-24T23:05:09.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.685+0000 7f3b8e4eb640 1 --2- 172.21.15.32:0/2618462865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b880ba0e0 0x7f3b88151520 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f3b8002a2b0 tx=0x7f3b800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:09.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.685+0000 7f3b86ffd640 1 -- 172.21.15.32:0/2618462865 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b80041030 con 0x7f3b880ba0e0 2024-04-24T23:05:09.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.685+0000 7f3b86ffd640 1 -- 172.21.15.32:0/2618462865 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b80030440 con 0x7f3b880ba0e0 2024-04-24T23:05:09.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.685+0000 7f3b9074f640 1 -- 172.21.15.32:0/2618462865 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b880c83b0 con 0x7f3b880ba0e0 2024-04-24T23:05:09.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.685+0000 7f3b86ffd640 1 -- 172.21.15.32:0/2618462865 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b80037040 con 0x7f3b880ba0e0 2024-04-24T23:05:09.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.685+0000 7f3b9074f640 1 -- 172.21.15.32:0/2618462865 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b88151a60 con 0x7f3b880ba0e0 2024-04-24T23:05:09.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.686+0000 7f3b9074f640 1 -- 172.21.15.32:0/2618462865 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b88068430 con 0x7f3b880ba0e0 2024-04-24T23:05:09.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.687+0000 7f3b86ffd640 1 -- 172.21.15.32:0/2618462865 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3b80044070 con 0x7f3b880ba0e0 2024-04-24T23:05:09.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.687+0000 7f3b86ffd640 1 --2- 172.21.15.32:0/2618462865 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3b7406be30 0x7f3b7406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:09.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.688+0000 7f3b86ffd640 1 -- 172.21.15.32:0/2618462865 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5709+0+0 (secure 0 0 0) 0x7f3b800ab4c0 con 0x7f3b880ba0e0 2024-04-24T23:05:09.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.688+0000 7f3b8dcea640 1 --2- 172.21.15.32:0/2618462865 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3b7406be30 0x7f3b7406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:09.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.688+0000 7f3b8dcea640 1 --2- 172.21.15.32:0/2618462865 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3b7406be30 0x7f3b7406e2f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c009560 tx=0x7f3b7c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:09.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.692+0000 7f3b86ffd640 1 -- 172.21.15.32:0/2618462865 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3b80079f00 con 0x7f3b880ba0e0 2024-04-24T23:05:09.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.875+0000 7f3b9074f640 1 -- 172.21.15.32:0/2618462865 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3b88072340 con 0x7f3b880ba0e0 2024-04-24T23:05:09.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.875+0000 7f3b86ffd640 1 -- 172.21.15.32:0/2618462865 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v128) ==== 104+0+11 (secure 0 0 0) 0x7f3b80079800 con 0x7f3b880ba0e0 2024-04-24T23:05:09.877 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-24T23:05:09.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.877+0000 7f3b9074f640 1 -- 172.21.15.32:0/2618462865 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3b7406be30 msgr2=0x7f3b7406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:09.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.877+0000 7f3b9074f640 1 --2- 172.21.15.32:0/2618462865 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3b7406be30 0x7f3b7406e2f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c009560 tx=0x7f3b7c008040 comp rx=0 tx=0).stop 2024-04-24T23:05:09.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.877+0000 7f3b9074f640 1 -- 172.21.15.32:0/2618462865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b880ba0e0 msgr2=0x7f3b88151520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:09.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.877+0000 7f3b9074f640 1 --2- 172.21.15.32:0/2618462865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b880ba0e0 0x7f3b88151520 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f3b8002a2b0 tx=0x7f3b800046c0 comp rx=0 tx=0).stop 2024-04-24T23:05:09.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.877+0000 7f3b9074f640 1 -- 172.21.15.32:0/2618462865 shutdown_connections 2024-04-24T23:05:09.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.877+0000 7f3b9074f640 1 --2- 172.21.15.32:0/2618462865 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3b7406be30 0x7f3b7406e2f0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:09.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.877+0000 7f3b9074f640 1 --2- 172.21.15.32:0/2618462865 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b8815e9b0 0x7f3b88160d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:09.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.877+0000 7f3b9074f640 1 --2- 172.21.15.32:0/2618462865 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b880d4210 0x7f3b8815f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:09.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.877+0000 7f3b9074f640 1 --2- 172.21.15.32:0/2618462865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b880ba0e0 0x7f3b88151520 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:09.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.877+0000 7f3b9074f640 1 -- 172.21.15.32:0/2618462865 >> 172.21.15.32:0/2618462865 conn(0x7f3b880d11d0 msgr2=0x7f3b880b8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:09.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.878+0000 7f3b9074f640 1 -- 172.21.15.32:0/2618462865 shutdown_connections 2024-04-24T23:05:09.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.878+0000 7f3b9074f640 1 -- 172.21.15.32:0/2618462865 wait complete. 2024-04-24T23:05:09.890 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:05:09.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.991+0000 7f7d863c1640 1 Processor -- start 2024-04-24T23:05:09.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.991+0000 7f7d863c1640 1 -- start start 2024-04-24T23:05:09.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.992+0000 7f7d863c1640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7d800b5560 0x7f7d8015e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:09.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.992+0000 7f7d863c1640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7d800cd3c0 0x7f7d8015ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:09.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.992+0000 7f7d863c1640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d800cd910 con 0x7f7d800b5940 2024-04-24T23:05:09.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.992+0000 7f7d863c1640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f7d800b6470 con 0x7f7d800b5560 2024-04-24T23:05:09.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.992+0000 7f7d863c1640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f7d800cdbf0 con 0x7f7d800cd3c0 2024-04-24T23:05:09.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.992+0000 7f7d7ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7d800b5560 0x7f7d8015e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:09.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.992+0000 7f7d7ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7d800b5560 0x7f7d8015e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55620/0 (socket says 172.21.15.32:55620) 2024-04-24T23:05:09.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.992+0000 7f7d7ffff640 1 -- 172.21.15.32:0/1122908988 learned_addr learned my addr 172.21.15.32:0/1122908988 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:09.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.992+0000 7f7d8495e640 1 --1- 172.21.15.32:0/1122908988 >> v1:172.21.15.32:6789/0 conn(0x7f7d800b5940 0x7f7d800b5d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45120/0 (socket says 172.21.15.32:45120) 2024-04-24T23:05:09.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.993+0000 7f7d7f7fe640 1 --2- 172.21.15.32:0/1122908988 >> v2:172.21.15.121:3300/0 conn(0x7f7d800cd3c0 0x7f7d8015ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:09.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.993+0000 7f7d7ffff640 1 -- 172.21.15.32:0/1122908988 >> v2:172.21.15.121:3300/0 conn(0x7f7d800cd3c0 msgr2=0x7f7d8015ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:09.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.993+0000 7f7d7ffff640 1 --2- 172.21.15.32:0/1122908988 >> v2:172.21.15.121:3300/0 conn(0x7f7d800cd3c0 0x7f7d8015ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:09.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.993+0000 7f7d7ffff640 1 -- 172.21.15.32:0/1122908988 >> v1:172.21.15.32:6789/0 conn(0x7f7d800b5940 legacy=0x7f7d800b5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:09.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.993+0000 7f7d7ffff640 1 -- 172.21.15.32:0/1122908988 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d800cded0 con 0x7f7d800b5560 2024-04-24T23:05:09.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.993+0000 7f7d7f7fe640 1 --2- 172.21.15.32:0/1122908988 >> v2:172.21.15.121:3300/0 conn(0x7f7d800cd3c0 0x7f7d8015ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:05:09.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.993+0000 7f7d7ffff640 1 --2- 172.21.15.32:0/1122908988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d800b5560 0x7f7d8015e6c0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f7d800ccb60 tx=0x7f7d6802c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f44047a37f412c70 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:09.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.993+0000 7f7d7effd640 1 -- 172.21.15.32:0/1122908988 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d68036020 con 0x7f7d800b5560 2024-04-24T23:05:09.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.994+0000 7f7d7effd640 1 -- 172.21.15.32:0/1122908988 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d68032640 con 0x7f7d800b5560 2024-04-24T23:05:09.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.994+0000 7f7d7effd640 1 -- 172.21.15.32:0/1122908988 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d68032a10 con 0x7f7d800b5560 2024-04-24T23:05:09.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.994+0000 7f7d863c1640 1 -- 172.21.15.32:0/1122908988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d800b5560 msgr2=0x7f7d8015e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:09.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.994+0000 7f7d863c1640 1 --2- 172.21.15.32:0/1122908988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d800b5560 0x7f7d8015e6c0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f7d800ccb60 tx=0x7f7d6802c940 comp rx=0 tx=0).stop 2024-04-24T23:05:09.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.994+0000 7f7d863c1640 1 -- 172.21.15.32:0/1122908988 shutdown_connections 2024-04-24T23:05:09.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.994+0000 7f7d863c1640 1 --2- 172.21.15.32:0/1122908988 >> v2:172.21.15.121:3300/0 conn(0x7f7d800cd3c0 0x7f7d8015ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:09.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.994+0000 7f7d863c1640 1 --2- 172.21.15.32:0/1122908988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d800b5560 0x7f7d8015e6c0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:09.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.994+0000 7f7d863c1640 1 -- 172.21.15.32:0/1122908988 >> 172.21.15.32:0/1122908988 conn(0x7f7d8006b110 msgr2=0x7f7d8006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:09.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.994+0000 7f7d863c1640 1 -- 172.21.15.32:0/1122908988 shutdown_connections 2024-04-24T23:05:10.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.994+0000 7f7d863c1640 1 -- 172.21.15.32:0/1122908988 wait complete. 2024-04-24T23:05:10.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.995+0000 7f7d863c1640 1 Processor -- start 2024-04-24T23:05:10.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.995+0000 7f7d863c1640 1 -- start start 2024-04-24T23:05:10.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.996+0000 7f7d863c1640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d800b5560 0x7f7d80155790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:10.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.996+0000 7f7d863c1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d800b5940 0x7f7d8014ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:10.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.996+0000 7f7d863c1640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d800cd3c0 0x7f7d800ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:10.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.996+0000 7f7d863c1640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8006c5f0 con 0x7f7d800b5940 2024-04-24T23:05:10.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.996+0000 7f7d863c1640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7d800ce490 con 0x7f7d800cd3c0 2024-04-24T23:05:10.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.996+0000 7f7d863c1640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f7d8006c770 con 0x7f7d800b5560 2024-04-24T23:05:10.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.996+0000 7f7d7f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d800b5940 0x7f7d8014ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:10.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.996+0000 7f7d7f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d800b5940 0x7f7d8014ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55632/0 (socket says 172.21.15.32:55632) 2024-04-24T23:05:10.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.996+0000 7f7d7f7fe640 1 -- 172.21.15.32:0/1303843722 learned_addr learned my addr 172.21.15.32:0/1303843722 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:10.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.997+0000 7f7d7ffff640 1 --2- 172.21.15.32:0/1303843722 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d800b5560 0x7f7d80155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:10.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.997+0000 7f7d8495e640 1 --2- 172.21.15.32:0/1303843722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d800cd3c0 0x7f7d800ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:10.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.997+0000 7f7d7f7fe640 1 -- 172.21.15.32:0/1303843722 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d800b5560 msgr2=0x7f7d80155790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:10.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.997+0000 7f7d7f7fe640 1 --2- 172.21.15.32:0/1303843722 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d800b5560 0x7f7d80155790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:10.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.997+0000 7f7d7f7fe640 1 -- 172.21.15.32:0/1303843722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d800cd3c0 msgr2=0x7f7d800ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:10.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.997+0000 7f7d7f7fe640 1 --2- 172.21.15.32:0/1303843722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d800cd3c0 0x7f7d800ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:10.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.997+0000 7f7d7f7fe640 1 -- 172.21.15.32:0/1303843722 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d80155fc0 con 0x7f7d800b5940 2024-04-24T23:05:10.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.997+0000 7f7d8495e640 1 --2- 172.21.15.32:0/1303843722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d800cd3c0 0x7f7d800ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:05:10.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.997+0000 7f7d7f7fe640 1 --2- 172.21.15.32:0/1303843722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d800b5940 0x7f7d8014ccc0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f7d7000c7f0 tx=0x7f7d7000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:10.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.997+0000 7f7d7cff9640 1 -- 172.21.15.32:0/1303843722 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d70015070 con 0x7f7d800b5940 2024-04-24T23:05:10.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.997+0000 7f7d7cff9640 1 -- 172.21.15.32:0/1303843722 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d70007820 con 0x7f7d800b5940 2024-04-24T23:05:10.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.997+0000 7f7d7cff9640 1 -- 172.21.15.32:0/1303843722 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d70007bd0 con 0x7f7d800b5940 2024-04-24T23:05:10.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.997+0000 7f7d863c1640 1 -- 172.21.15.32:0/1303843722 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d801562a0 con 0x7f7d800b5940 2024-04-24T23:05:10.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.997+0000 7f7d863c1640 1 -- 172.21.15.32:0/1303843722 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d80156d70 con 0x7f7d800b5940 2024-04-24T23:05:10.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.998+0000 7f7d863c1640 1 -- 172.21.15.32:0/1303843722 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d80156b80 con 0x7f7d800b5940 2024-04-24T23:05:10.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.999+0000 7f7d7cff9640 1 -- 172.21.15.32:0/1303843722 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7d70018070 con 0x7f7d800b5940 2024-04-24T23:05:10.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.999+0000 7f7d7cff9640 1 --2- 172.21.15.32:0/1303843722 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7d6006be60 0x7f7d6006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:10.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.999+0000 7f7d7cff9640 1 -- 172.21.15.32:0/1303843722 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5709+0+0 (secure 0 0 0) 0x7f7d7008af60 con 0x7f7d800b5940 2024-04-24T23:05:10.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:09.999+0000 7f7d7ffff640 1 --2- 172.21.15.32:0/1303843722 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7d6006be60 0x7f7d6006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:10.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.000+0000 7f7d7ffff640 1 --2- 172.21.15.32:0/1303843722 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7d6006be60 0x7f7d6006e320 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f7d68037000 tx=0x7f7d68008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:10.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.003+0000 7f7d7cff9640 1 -- 172.21.15.32:0/1303843722 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7d70059910 con 0x7f7d800b5940 2024-04-24T23:05:10.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.187+0000 7f7d863c1640 1 -- 172.21.15.32:0/1303843722 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7d80068430 con 0x7f7d800b5940 2024-04-24T23:05:10.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.188+0000 7f7d7cff9640 1 -- 172.21.15.32:0/1303843722 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v128) ==== 100+0+11 (secure 0 0 0) 0x7f7d70059210 con 0x7f7d800b5940 2024-04-24T23:05:10.189 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-24T23:05:10.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.189+0000 7f7d863c1640 1 -- 172.21.15.32:0/1303843722 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7d6006be60 msgr2=0x7f7d6006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:10.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.189+0000 7f7d863c1640 1 --2- 172.21.15.32:0/1303843722 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7d6006be60 0x7f7d6006e320 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f7d68037000 tx=0x7f7d68008040 comp rx=0 tx=0).stop 2024-04-24T23:05:10.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.189+0000 7f7d863c1640 1 -- 172.21.15.32:0/1303843722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d800b5940 msgr2=0x7f7d8014ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:10.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.189+0000 7f7d863c1640 1 --2- 172.21.15.32:0/1303843722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d800b5940 0x7f7d8014ccc0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f7d7000c7f0 tx=0x7f7d7000ccb0 comp rx=0 tx=0).stop 2024-04-24T23:05:10.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.189+0000 7f7d863c1640 1 -- 172.21.15.32:0/1303843722 shutdown_connections 2024-04-24T23:05:10.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.189+0000 7f7d863c1640 1 --2- 172.21.15.32:0/1303843722 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7d6006be60 0x7f7d6006e320 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:10.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.189+0000 7f7d863c1640 1 --2- 172.21.15.32:0/1303843722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d800b5940 0x7f7d8014ccc0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:10.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.189+0000 7f7d863c1640 1 --2- 172.21.15.32:0/1303843722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d800cd3c0 0x7f7d800ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:10.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.189+0000 7f7d863c1640 1 --2- 172.21.15.32:0/1303843722 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d800b5560 0x7f7d80155790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:10.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.189+0000 7f7d863c1640 1 -- 172.21.15.32:0/1303843722 >> 172.21.15.32:0/1303843722 conn(0x7f7d8006b110 msgr2=0x7f7d800b0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:10.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.189+0000 7f7d863c1640 1 -- 172.21.15.32:0/1303843722 shutdown_connections 2024-04-24T23:05:10.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.189+0000 7f7d863c1640 1 -- 172.21.15.32:0/1303843722 wait complete. 2024-04-24T23:05:10.202 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T23:05:10.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.302+0000 7fd3e5bc1640 1 Processor -- start 2024-04-24T23:05:10.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.302+0000 7fd3e5bc1640 1 -- start start 2024-04-24T23:05:10.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.303+0000 7fd3e5bc1640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd3e00b5560 0x7fd3e015e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:10.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.303+0000 7fd3e5bc1640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd3e00cd3c0 0x7fd3e015ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:10.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.303+0000 7fd3e5bc1640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3e00cd910 con 0x7fd3e00b5940 2024-04-24T23:05:10.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.303+0000 7fd3e5bc1640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fd3e00b6470 con 0x7fd3e00cd3c0 2024-04-24T23:05:10.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.303+0000 7fd3e5bc1640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fd3e00cdbf0 con 0x7fd3e00b5560 2024-04-24T23:05:10.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.303+0000 7fd3deffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd3e00cd3c0 0x7fd3e015ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:10.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.303+0000 7fd3dffff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd3e00b5940 0x7fd3e00b5d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45128/0 (socket says 172.21.15.32:45128) 2024-04-24T23:05:10.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.303+0000 7fd3dffff640 1 -- 172.21.15.32:0/3769004025 learned_addr learned my addr 172.21.15.32:0/3769004025 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:05:10.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.304+0000 7fd3df7fe640 1 --2- 172.21.15.32:0/3769004025 >> v2:172.21.15.121:3300/0 conn(0x7fd3e00b5560 0x7fd3e015e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:10.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.304+0000 7fd3deffd640 1 -- 172.21.15.32:0/3769004025 >> v2:172.21.15.121:3300/0 conn(0x7fd3e00b5560 msgr2=0x7fd3e015e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:10.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.304+0000 7fd3deffd640 1 --2- 172.21.15.32:0/3769004025 >> v2:172.21.15.121:3300/0 conn(0x7fd3e00b5560 0x7fd3e015e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:10.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.304+0000 7fd3de7fc640 1 -- 172.21.15.32:0/3769004025 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2703773526 0 0) 0x7fd3e00cd910 con 0x7fd3e00b5940 2024-04-24T23:05:10.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.304+0000 7fd3deffd640 1 -- 172.21.15.32:0/3769004025 >> v1:172.21.15.32:6789/0 conn(0x7fd3e00b5940 legacy=0x7fd3e00b5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:10.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.304+0000 7fd3deffd640 1 -- 172.21.15.32:0/3769004025 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3e00cded0 con 0x7fd3e00cd3c0 2024-04-24T23:05:10.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.304+0000 7fd3df7fe640 1 --2- 172.21.15.32:0/3769004025 >> v2:172.21.15.121:3300/0 conn(0x7fd3e00b5560 0x7fd3e015e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:05:10.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.304+0000 7fd3deffd640 1 --2- 172.21.15.32:0/3769004025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3e00cd3c0 0x7fd3e015ec00 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fd3e006ede0 tx=0x7fd3d402c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d79c35f6c2c86c0d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:10.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.305+0000 7fd3de7fc640 1 -- 172.21.15.32:0/3769004025 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d403a070 con 0x7fd3e00cd3c0 2024-04-24T23:05:10.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.305+0000 7fd3de7fc640 1 -- 172.21.15.32:0/3769004025 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3d4032480 con 0x7fd3e00cd3c0 2024-04-24T23:05:10.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.305+0000 7fd3de7fc640 1 -- 172.21.15.32:0/3769004025 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d402ce60 con 0x7fd3e00cd3c0 2024-04-24T23:05:10.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.305+0000 7fd3e5bc1640 1 -- 172.21.15.32:0/3769004025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3e00cd3c0 msgr2=0x7fd3e015ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:10.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.305+0000 7fd3e5bc1640 1 --2- 172.21.15.32:0/3769004025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3e00cd3c0 0x7fd3e015ec00 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fd3e006ede0 tx=0x7fd3d402c640 comp rx=0 tx=0).stop 2024-04-24T23:05:10.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.305+0000 7fd3e5bc1640 1 -- 172.21.15.32:0/3769004025 shutdown_connections 2024-04-24T23:05:10.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.305+0000 7fd3e5bc1640 1 --2- 172.21.15.32:0/3769004025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3e00cd3c0 0x7fd3e015ec00 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:10.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.305+0000 7fd3e5bc1640 1 --2- 172.21.15.32:0/3769004025 >> v2:172.21.15.121:3300/0 conn(0x7fd3e00b5560 0x7fd3e015e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:10.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.305+0000 7fd3e5bc1640 1 -- 172.21.15.32:0/3769004025 >> 172.21.15.32:0/3769004025 conn(0x7fd3e006b110 msgr2=0x7fd3e006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:10.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.305+0000 7fd3e5bc1640 1 -- 172.21.15.32:0/3769004025 shutdown_connections 2024-04-24T23:05:10.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.305+0000 7fd3e5bc1640 1 -- 172.21.15.32:0/3769004025 wait complete. 2024-04-24T23:05:10.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.306+0000 7fd3e5bc1640 1 Processor -- start 2024-04-24T23:05:10.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.307+0000 7fd3e5bc1640 1 -- start start 2024-04-24T23:05:10.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.307+0000 7fd3e5bc1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3e00b5560 0x7fd3e00b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:10.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.307+0000 7fd3e5bc1640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd3e00b5940 0x7fd3e0158e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:10.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.307+0000 7fd3e5bc1640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3e00cd3c0 0x7fd3e00ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:10.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.307+0000 7fd3e5bc1640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e006c5f0 con 0x7fd3e00b5560 2024-04-24T23:05:10.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.307+0000 7fd3e5bc1640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e00ce490 con 0x7fd3e00cd3c0 2024-04-24T23:05:10.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.307+0000 7fd3e5bc1640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd3e006c770 con 0x7fd3e00b5940 2024-04-24T23:05:10.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.307+0000 7fd3df7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3e00b5560 0x7fd3e00b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:10.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.307+0000 7fd3df7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3e00b5560 0x7fd3e00b3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55636/0 (socket says 172.21.15.32:55636) 2024-04-24T23:05:10.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.307+0000 7fd3df7fe640 1 -- 172.21.15.32:0/2905209348 learned_addr learned my addr 172.21.15.32:0/2905209348 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:10.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.308+0000 7fd3df7fe640 1 -- 172.21.15.32:0/2905209348 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd3e00b5940 msgr2=0x7fd3e0158e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:10.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.308+0000 7fd3df7fe640 1 --2- 172.21.15.32:0/2905209348 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd3e00b5940 0x7fd3e0158e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:10.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.308+0000 7fd3df7fe640 1 -- 172.21.15.32:0/2905209348 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3e00cd3c0 msgr2=0x7fd3e00ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:10.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.308+0000 7fd3dffff640 1 --2- 172.21.15.32:0/2905209348 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3e00cd3c0 0x7fd3e00ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:10.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.308+0000 7fd3df7fe640 1 --2- 172.21.15.32:0/2905209348 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3e00cd3c0 0x7fd3e00ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:10.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.308+0000 7fd3df7fe640 1 -- 172.21.15.32:0/2905209348 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3e015af20 con 0x7fd3e00b5560 2024-04-24T23:05:10.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.308+0000 7fd3dffff640 1 --2- 172.21.15.32:0/2905209348 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3e00cd3c0 0x7fd3e00ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:05:10.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.308+0000 7fd3df7fe640 1 --2- 172.21.15.32:0/2905209348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3e00b5560 0x7fd3e00b3b20 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fd3d000c7f0 tx=0x7fd3d000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:10.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.308+0000 7fd3c7fff640 1 -- 172.21.15.32:0/2905209348 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d0015070 con 0x7fd3e00b5560 2024-04-24T23:05:10.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.308+0000 7fd3c7fff640 1 -- 172.21.15.32:0/2905209348 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3d0007a30 con 0x7fd3e00b5560 2024-04-24T23:05:10.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.308+0000 7fd3c7fff640 1 -- 172.21.15.32:0/2905209348 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d0007e00 con 0x7fd3e00b5560 2024-04-24T23:05:10.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.308+0000 7fd3e5bc1640 1 -- 172.21.15.32:0/2905209348 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3e015b200 con 0x7fd3e00b5560 2024-04-24T23:05:10.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.308+0000 7fd3e5bc1640 1 -- 172.21.15.32:0/2905209348 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3e015b7c0 con 0x7fd3e00b5560 2024-04-24T23:05:10.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.309+0000 7fd3e5bc1640 1 -- 172.21.15.32:0/2905209348 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3e015bba0 con 0x7fd3e00b5560 2024-04-24T23:05:10.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.310+0000 7fd3c7fff640 1 -- 172.21.15.32:0/2905209348 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd3d0018070 con 0x7fd3e00b5560 2024-04-24T23:05:10.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.310+0000 7fd3c7fff640 1 --2- 172.21.15.32:0/2905209348 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd3cc06be30 0x7fd3cc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:10.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.310+0000 7fd3c7fff640 1 -- 172.21.15.32:0/2905209348 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5709+0+0 (secure 0 0 0) 0x7fd3d008adc0 con 0x7fd3e00b5560 2024-04-24T23:05:10.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.310+0000 7fd3deffd640 1 --2- 172.21.15.32:0/2905209348 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd3cc06be30 0x7fd3cc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:10.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.311+0000 7fd3deffd640 1 --2- 172.21.15.32:0/2905209348 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd3cc06be30 0x7fd3cc06e2f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4037000 tx=0x7fd3d4032000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:10.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.314+0000 7fd3c7fff640 1 -- 172.21.15.32:0/2905209348 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd3e015bba0 con 0x7fd3e00b5560 2024-04-24T23:05:10.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.508+0000 7fd3e5bc1640 1 -- 172.21.15.32:0/2905209348 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd3e006d5f0 con 0x7fd3e00b5560 2024-04-24T23:05:10.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.509+0000 7fd3c7fff640 1 -- 172.21.15.32:0/2905209348 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v128) ==== 80+0+0 (secure 0 0 0) 0x7fd3d0059800 con 0x7fd3e00b5560 2024-04-24T23:05:10.510 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2024-04-24T23:05:10.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.511+0000 7fd3e5bc1640 1 -- 172.21.15.32:0/2905209348 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd3cc06be30 msgr2=0x7fd3cc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:10.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.511+0000 7fd3e5bc1640 1 --2- 172.21.15.32:0/2905209348 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd3cc06be30 0x7fd3cc06e2f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4037000 tx=0x7fd3d4032000 comp rx=0 tx=0).stop 2024-04-24T23:05:10.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.511+0000 7fd3e5bc1640 1 -- 172.21.15.32:0/2905209348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3e00b5560 msgr2=0x7fd3e00b3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:10.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.511+0000 7fd3e5bc1640 1 --2- 172.21.15.32:0/2905209348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3e00b5560 0x7fd3e00b3b20 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fd3d000c7f0 tx=0x7fd3d000cbf0 comp rx=0 tx=0).stop 2024-04-24T23:05:10.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.511+0000 7fd3e5bc1640 1 -- 172.21.15.32:0/2905209348 shutdown_connections 2024-04-24T23:05:10.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.511+0000 7fd3e5bc1640 1 --2- 172.21.15.32:0/2905209348 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd3cc06be30 0x7fd3cc06e2f0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:10.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.511+0000 7fd3e5bc1640 1 --2- 172.21.15.32:0/2905209348 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3e00cd3c0 0x7fd3e00ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:10.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.511+0000 7fd3e5bc1640 1 --2- 172.21.15.32:0/2905209348 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd3e00b5940 0x7fd3e0158e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:10.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.511+0000 7fd3e5bc1640 1 --2- 172.21.15.32:0/2905209348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3e00b5560 0x7fd3e00b3b20 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:10.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.511+0000 7fd3e5bc1640 1 -- 172.21.15.32:0/2905209348 >> 172.21.15.32:0/2905209348 conn(0x7fd3e006b110 msgr2=0x7fd3e0154750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:10.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.511+0000 7fd3e5bc1640 1 -- 172.21.15.32:0/2905209348 shutdown_connections 2024-04-24T23:05:10.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.511+0000 7fd3e5bc1640 1 -- 172.21.15.32:0/2905209348 wait complete. 2024-04-24T23:05:10.523 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-24T23:05:10.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:05:10.523 DEBUG:teuthology.orchestra.run.smithi032:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-24T23:05:10.561 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2024-04-24T23:05:10.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:05:10.562 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-24T23:05:10.618 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:05:10.618 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:05:10.618 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:05:10.635 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:05:10.635 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:05:10.692 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:05:10.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.791+0000 7f4b96530640 1 Processor -- start 2024-04-24T23:05:10.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.791+0000 7f4b96530640 1 -- start start 2024-04-24T23:05:10.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.792+0000 7f4b96530640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4b9015b380 0x7f4b9006c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:10.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.792+0000 7f4b96530640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4b900ceb40 0x7f4b9006cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:10.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.792+0000 7f4b96530640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b900b4520 con 0x7f4b9015cb00 2024-04-24T23:05:10.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.792+0000 7f4b96530640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f4b900cef70 con 0x7f4b9015b380 2024-04-24T23:05:10.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.792+0000 7f4b96530640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f4b900b4800 con 0x7f4b900ceb40 2024-04-24T23:05:10.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.792+0000 7f4b8ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4b9015b380 0x7f4b9006c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:10.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.792+0000 7f4b8ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4b9015b380 0x7f4b9006c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55640/0 (socket says 172.21.15.32:55640) 2024-04-24T23:05:10.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.792+0000 7f4b8ffff640 1 -- 172.21.15.32:0/4026116638 learned_addr learned my addr 172.21.15.32:0/4026116638 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:10.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.792+0000 7f4b94acd640 1 --2- 172.21.15.32:0/4026116638 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4b900ceb40 0x7f4b9006cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:10.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.793+0000 7f4b8ffff640 1 -- 172.21.15.32:0/4026116638 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4b900ceb40 msgr2=0x7f4b9006cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:10.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.793+0000 7f4b8ffff640 1 --2- 172.21.15.32:0/4026116638 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4b900ceb40 0x7f4b9006cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:10.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.793+0000 7f4b8ffff640 1 -- 172.21.15.32:0/4026116638 >> v1:172.21.15.121:6789/0 conn(0x7f4b9015cb00 legacy=0x7f4b900ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:10.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.793+0000 7f4b8ffff640 1 -- 172.21.15.32:0/4026116638 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b900b4ae0 con 0x7f4b9015b380 2024-04-24T23:05:10.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.793+0000 7f4b8ffff640 1 --2- 172.21.15.32:0/4026116638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b9015b380 0x7f4b9006c5f0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f4b8400bb10 tx=0x7f4b8402c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eade904ac82f9816 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:10.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.793+0000 7f4b8effd640 1 -- 172.21.15.32:0/4026116638 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b84032700 con 0x7f4b9015b380 2024-04-24T23:05:10.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.793+0000 7f4b8effd640 1 -- 172.21.15.32:0/4026116638 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b840328a0 con 0x7f4b9015b380 2024-04-24T23:05:10.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.793+0000 7f4b8effd640 1 -- 172.21.15.32:0/4026116638 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b84032c70 con 0x7f4b9015b380 2024-04-24T23:05:10.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.794+0000 7f4b96530640 1 -- 172.21.15.32:0/4026116638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b9015b380 msgr2=0x7f4b9006c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:10.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.794+0000 7f4b96530640 1 --2- 172.21.15.32:0/4026116638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b9015b380 0x7f4b9006c5f0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f4b8400bb10 tx=0x7f4b8402c800 comp rx=0 tx=0).stop 2024-04-24T23:05:10.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.794+0000 7f4b96530640 1 -- 172.21.15.32:0/4026116638 shutdown_connections 2024-04-24T23:05:10.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.794+0000 7f4b96530640 1 --2- 172.21.15.32:0/4026116638 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4b900ceb40 0x7f4b9006cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:10.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.794+0000 7f4b96530640 1 --2- 172.21.15.32:0/4026116638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b9015b380 0x7f4b9006c5f0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:10.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.794+0000 7f4b96530640 1 -- 172.21.15.32:0/4026116638 >> 172.21.15.32:0/4026116638 conn(0x7f4b9006b110 msgr2=0x7f4b9006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:10.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.794+0000 7f4b96530640 1 -- 172.21.15.32:0/4026116638 shutdown_connections 2024-04-24T23:05:10.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.794+0000 7f4b96530640 1 -- 172.21.15.32:0/4026116638 wait complete. 2024-04-24T23:05:10.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.795+0000 7f4b96530640 1 Processor -- start 2024-04-24T23:05:10.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.795+0000 7f4b96530640 1 -- start start 2024-04-24T23:05:10.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.796+0000 7f4b96530640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4b900ceb40 0x7f4b900b3260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:10.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.796+0000 7f4b96530640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b9015b380 0x7f4b9014fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:10.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.796+0000 7f4b96530640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b9015cb00 0x7f4b901fb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:10.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.796+0000 7f4b96530640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9015b760 con 0x7f4b9015cb00 2024-04-24T23:05:10.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.796+0000 7f4b96530640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4b900b50a0 con 0x7f4b9015b380 2024-04-24T23:05:10.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.796+0000 7f4b96530640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f4b900b55c0 con 0x7f4b900ceb40 2024-04-24T23:05:10.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.796+0000 7f4b94acd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b9015cb00 0x7f4b901fb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:10.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.796+0000 7f4b94acd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b9015cb00 0x7f4b901fb3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55648/0 (socket says 172.21.15.32:55648) 2024-04-24T23:05:10.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.796+0000 7f4b94acd640 1 -- 172.21.15.32:0/395046789 learned_addr learned my addr 172.21.15.32:0/395046789 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:10.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.796+0000 7f4b8ffff640 1 --2- 172.21.15.32:0/395046789 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4b900ceb40 0x7f4b900b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:10.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.796+0000 7f4b8f7fe640 1 --2- 172.21.15.32:0/395046789 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b9015b380 0x7f4b9014fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:10.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.797+0000 7f4b94acd640 1 -- 172.21.15.32:0/395046789 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4b900ceb40 msgr2=0x7f4b900b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:10.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.797+0000 7f4b94acd640 1 --2- 172.21.15.32:0/395046789 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4b900ceb40 0x7f4b900b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:10.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.797+0000 7f4b94acd640 1 -- 172.21.15.32:0/395046789 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b9015b380 msgr2=0x7f4b9014fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:10.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.797+0000 7f4b94acd640 1 --2- 172.21.15.32:0/395046789 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b9015b380 0x7f4b9014fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:10.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.797+0000 7f4b94acd640 1 -- 172.21.15.32:0/395046789 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b901fc1e0 con 0x7f4b9015cb00 2024-04-24T23:05:10.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.797+0000 7f4b94acd640 1 --2- 172.21.15.32:0/395046789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b9015cb00 0x7f4b901fb3f0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f4b88007e70 tx=0x7f4b8800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:10.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.797+0000 7f4b8cff9640 1 -- 172.21.15.32:0/395046789 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b88014620 con 0x7f4b9015cb00 2024-04-24T23:05:10.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.797+0000 7f4b8cff9640 1 -- 172.21.15.32:0/395046789 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b88019070 con 0x7f4b9015cb00 2024-04-24T23:05:10.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.797+0000 7f4b8cff9640 1 -- 172.21.15.32:0/395046789 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b88014a20 con 0x7f4b9015cb00 2024-04-24T23:05:10.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.797+0000 7f4b96530640 1 -- 172.21.15.32:0/395046789 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b901fc4c0 con 0x7f4b9015cb00 2024-04-24T23:05:10.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.797+0000 7f4b96530640 1 -- 172.21.15.32:0/395046789 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b901fca80 con 0x7f4b9015cb00 2024-04-24T23:05:10.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.798+0000 7f4b96530640 1 -- 172.21.15.32:0/395046789 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b90068430 con 0x7f4b9015cb00 2024-04-24T23:05:10.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.799+0000 7f4b8cff9640 1 -- 172.21.15.32:0/395046789 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4b88014bc0 con 0x7f4b9015cb00 2024-04-24T23:05:10.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.799+0000 7f4b8cff9640 1 --2- 172.21.15.32:0/395046789 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4b7c06be30 0x7f4b7c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:10.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.799+0000 7f4b8cff9640 1 -- 172.21.15.32:0/395046789 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5709+0+0 (secure 0 0 0) 0x7f4b8808b5c0 con 0x7f4b9015cb00 2024-04-24T23:05:10.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.799+0000 7f4b8ffff640 1 --2- 172.21.15.32:0/395046789 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4b7c06be30 0x7f4b7c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:10.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.800+0000 7f4b8ffff640 1 --2- 172.21.15.32:0/395046789 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4b7c06be30 0x7f4b7c06e2f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f4b84037000 tx=0x7f4b84008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:10.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.804+0000 7f4b8cff9640 1 -- 172.21.15.32:0/395046789 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4b880599b0 con 0x7f4b9015cb00 2024-04-24T23:05:10.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.994+0000 7f4b96530640 1 -- 172.21.15.32:0/395046789 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4b900afd60 con 0x7f4b9015cb00 2024-04-24T23:05:10.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.995+0000 7f4b8cff9640 1 -- 172.21.15.32:0/395046789 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) ==== 74+0+16639 (secure 0 0 0) 0x7f4b8805dcb0 con 0x7f4b9015cb00 2024-04-24T23:05:10.996 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:05:10.997 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":128,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:05:07.320153+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:04:54.280785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-24T23:04:54.966294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:05:10.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.997+0000 7f4b96530640 1 -- 172.21.15.32:0/395046789 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4b7c06be30 msgr2=0x7f4b7c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:10.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.997+0000 7f4b96530640 1 --2- 172.21.15.32:0/395046789 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4b7c06be30 0x7f4b7c06e2f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f4b84037000 tx=0x7f4b84008040 comp rx=0 tx=0).stop 2024-04-24T23:05:10.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.997+0000 7f4b96530640 1 -- 172.21.15.32:0/395046789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b9015cb00 msgr2=0x7f4b901fb3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:10.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.997+0000 7f4b96530640 1 --2- 172.21.15.32:0/395046789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b9015cb00 0x7f4b901fb3f0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f4b88007e70 tx=0x7f4b8800e4b0 comp rx=0 tx=0).stop 2024-04-24T23:05:10.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.997+0000 7f4b96530640 1 -- 172.21.15.32:0/395046789 shutdown_connections 2024-04-24T23:05:10.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.997+0000 7f4b96530640 1 --2- 172.21.15.32:0/395046789 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4b7c06be30 0x7f4b7c06e2f0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:10.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.997+0000 7f4b96530640 1 --2- 172.21.15.32:0/395046789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b9015cb00 0x7f4b901fb3f0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:10.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.997+0000 7f4b96530640 1 --2- 172.21.15.32:0/395046789 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b9015b380 0x7f4b9014fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:10.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.997+0000 7f4b96530640 1 --2- 172.21.15.32:0/395046789 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4b900ceb40 0x7f4b900b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:10.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.997+0000 7f4b96530640 1 -- 172.21.15.32:0/395046789 >> 172.21.15.32:0/395046789 conn(0x7f4b9006b110 msgr2=0x7f4b900ce430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:10.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.997+0000 7f4b96530640 1 -- 172.21.15.32:0/395046789 shutdown_connections 2024-04-24T23:05:10.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:10.997+0000 7f4b96530640 1 -- 172.21.15.32:0/395046789 wait complete. 2024-04-24T23:05:11.009 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:04:54.280785+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '124', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:04:54.966294+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:05:11.010 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:05:11.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.112+0000 7fc114c58640 1 Processor -- start 2024-04-24T23:05:11.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.112+0000 7fc114c58640 1 -- start start 2024-04-24T23:05:11.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.113+0000 7fc114c58640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc110072cf0 0x7fc11006c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:11.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.113+0000 7fc114c58640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc11015c990 con 0x7fc11015c5b0 2024-04-24T23:05:11.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.113+0000 7fc114c58640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc110071c50 con 0x7fc110071770 2024-04-24T23:05:11.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.113+0000 7fc114c58640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc1100b4740 con 0x7fc110072cf0 2024-04-24T23:05:11.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.113+0000 7fc10e59c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc11015c5b0 0x7fc1100b4030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45142/0 (socket says 172.21.15.32:45142) 2024-04-24T23:05:11.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.113+0000 7fc10e59c640 1 -- 172.21.15.32:0/3262513497 learned_addr learned my addr 172.21.15.32:0/3262513497 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:05:11.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.113+0000 7fc10d59a640 1 -- 172.21.15.32:0/3262513497 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2030246551 0 0) 0x7fc108001110 con 0x7fc11015c5b0 2024-04-24T23:05:11.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.113+0000 7fc10d59a640 1 -- 172.21.15.32:0/3262513497 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc0fc003520 con 0x7fc11015c5b0 2024-04-24T23:05:11.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.113+0000 7fc10dd9b640 1 --2- 172.21.15.32:0/3262513497 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc110072cf0 0x7fc11006c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:11.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.113+0000 7fc10d59a640 1 -- 172.21.15.32:0/3262513497 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 713291271 0 0) 0x7fc0fc003520 con 0x7fc11015c5b0 2024-04-24T23:05:11.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.113+0000 7fc10d59a640 1 -- 172.21.15.32:0/3262513497 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc110072cf0 msgr2=0x7fc11006c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:11.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.113+0000 7fc10d59a640 1 --2- 172.21.15.32:0/3262513497 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc110072cf0 0x7fc11006c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:11.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.114+0000 7fc10d59a640 1 -- 172.21.15.32:0/3262513497 >> v1:172.21.15.121:6789/0 conn(0x7fc110071770 legacy=0x7fc1100b4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:11.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.114+0000 7fc10d59a640 1 -- 172.21.15.32:0/3262513497 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc110071f30 con 0x7fc11015c5b0 2024-04-24T23:05:11.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.114+0000 7fc10d59a640 1 -- 172.21.15.32:0/3262513497 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7fc108001840 con 0x7fc11015c5b0 2024-04-24T23:05:11.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.114+0000 7fc10d59a640 1 -- 172.21.15.32:0/3262513497 >> v1:172.21.15.32:6789/0 conn(0x7fc11015c5b0 legacy=0x7fc1100b4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:11.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.114+0000 7fc10d59a640 1 --2- 172.21.15.32:0/3262513497 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc110072cf0 0x7fc0fc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:11.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.114+0000 7fc10d59a640 1 --2- 172.21.15.32:0/3262513497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0fc004010 0x7fc0fc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:11.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.114+0000 7fc10d59a640 1 --2- 172.21.15.32:0/3262513497 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0fc004d40 0x7fc0fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:11.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.114+0000 7fc10d59a640 1 -- 172.21.15.32:0/3262513497 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc0fc00bbd0 con 0x7fc0fc004010 2024-04-24T23:05:11.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.114+0000 7fc10d59a640 1 -- 172.21.15.32:0/3262513497 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc0fc00bd80 con 0x7fc0fc004d40 2024-04-24T23:05:11.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.114+0000 7fc10d59a640 1 -- 172.21.15.32:0/3262513497 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc0fc00bf80 con 0x7fc110072cf0 2024-04-24T23:05:11.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.114+0000 7fc10dd9b640 1 --2- 172.21.15.32:0/3262513497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0fc004010 0x7fc0fc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:11.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.114+0000 7fc10e59c640 1 --2- 172.21.15.32:0/3262513497 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc110072cf0 0x7fc0fc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:11.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.114+0000 7fc10ed9d640 1 --2- 172.21.15.32:0/3262513497 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0fc004d40 0x7fc0fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:11.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.114+0000 7fc10dd9b640 1 -- 172.21.15.32:0/3262513497 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc110072cf0 msgr2=0x7fc0fc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:11.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.114+0000 7fc10dd9b640 1 --2- 172.21.15.32:0/3262513497 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc110072cf0 0x7fc0fc0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:11.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.114+0000 7fc10dd9b640 1 -- 172.21.15.32:0/3262513497 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0fc004d40 msgr2=0x7fc0fc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:11.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.114+0000 7fc10dd9b640 1 --2- 172.21.15.32:0/3262513497 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0fc004d40 0x7fc0fc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:11.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.114+0000 7fc10dd9b640 1 -- 172.21.15.32:0/3262513497 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0fc003520 con 0x7fc0fc004010 2024-04-24T23:05:11.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.115+0000 7fc10dd9b640 1 --2- 172.21.15.32:0/3262513497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0fc004010 0x7fc0fc004800 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fc0f800ba30 tx=0x7fc0fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47f9c7a01f5be4ad server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:11.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.115+0000 7fc10d59a640 1 -- 172.21.15.32:0/3262513497 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f8030070 con 0x7fc0fc004010 2024-04-24T23:05:11.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.115+0000 7fc10d59a640 1 -- 172.21.15.32:0/3262513497 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0f802ade0 con 0x7fc0fc004010 2024-04-24T23:05:11.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.115+0000 7fc10d59a640 1 -- 172.21.15.32:0/3262513497 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f800b600 con 0x7fc0fc004010 2024-04-24T23:05:11.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.115+0000 7fc114c58640 1 -- 172.21.15.32:0/3262513497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0fc004010 msgr2=0x7fc0fc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:11.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.115+0000 7fc114c58640 1 --2- 172.21.15.32:0/3262513497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0fc004010 0x7fc0fc004800 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fc0f800ba30 tx=0x7fc0fc005830 comp rx=0 tx=0).stop 2024-04-24T23:05:11.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.115+0000 7fc10ed9d640 1 -- 172.21.15.32:0/3262513497 reap_dead start 2024-04-24T23:05:11.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.116+0000 7fc114c58640 1 -- 172.21.15.32:0/3262513497 shutdown_connections 2024-04-24T23:05:11.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.116+0000 7fc114c58640 1 -- 172.21.15.32:0/3262513497 >> 172.21.15.32:0/3262513497 conn(0x7fc11006b110 msgr2=0x7fc11006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:11.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.116+0000 7fc114c58640 1 -- 172.21.15.32:0/3262513497 shutdown_connections 2024-04-24T23:05:11.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.116+0000 7fc114c58640 1 -- 172.21.15.32:0/3262513497 wait complete. 2024-04-24T23:05:11.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.117+0000 7fc114c58640 1 Processor -- start 2024-04-24T23:05:11.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.117+0000 7fc114c58640 1 -- start start 2024-04-24T23:05:11.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.118+0000 7fc114c58640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0fc004010 0x7fc11015f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:11.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.118+0000 7fc114c58640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc11014ef90 0x7fc110150300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:11.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.118+0000 7fc114c58640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc110150840 0x7fc110154920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:11.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.118+0000 7fc114c58640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc1100724f0 con 0x7fc0fc004010 2024-04-24T23:05:11.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.118+0000 7fc114c58640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc110072860 con 0x7fc11014ef90 2024-04-24T23:05:11.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.118+0000 7fc114c58640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc1100b5250 con 0x7fc110150840 2024-04-24T23:05:11.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.118+0000 7fc10e59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0fc004010 0x7fc11015f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:11.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.118+0000 7fc10e59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0fc004010 0x7fc11015f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55674/0 (socket says 172.21.15.32:55674) 2024-04-24T23:05:11.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.118+0000 7fc10e59c640 1 -- 172.21.15.32:0/2193626622 learned_addr learned my addr 172.21.15.32:0/2193626622 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:11.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.118+0000 7fc10dd9b640 1 --2- 172.21.15.32:0/2193626622 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc11014ef90 0x7fc110150300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:11.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.118+0000 7fc10ed9d640 1 --2- 172.21.15.32:0/2193626622 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc110150840 0x7fc110154920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:11.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.118+0000 7fc10e59c640 1 -- 172.21.15.32:0/2193626622 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc110150840 msgr2=0x7fc110154920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:11.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.118+0000 7fc10e59c640 1 --2- 172.21.15.32:0/2193626622 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc110150840 0x7fc110154920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:11.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.118+0000 7fc10e59c640 1 -- 172.21.15.32:0/2193626622 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc11014ef90 msgr2=0x7fc110150300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:11.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.118+0000 7fc10e59c640 1 --2- 172.21.15.32:0/2193626622 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc11014ef90 0x7fc110150300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:11.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.118+0000 7fc10e59c640 1 -- 172.21.15.32:0/2193626622 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc11015d720 con 0x7fc0fc004010 2024-04-24T23:05:11.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.118+0000 7fc10ed9d640 1 --2- 172.21.15.32:0/2193626622 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc110150840 0x7fc110154920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:05:11.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.118+0000 7fc10e59c640 1 --2- 172.21.15.32:0/2193626622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0fc004010 0x7fc11015f630 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fc108007310 tx=0x7fc1080077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:11.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.118+0000 7fc106ffd640 1 -- 172.21.15.32:0/2193626622 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10800f940 con 0x7fc0fc004010 2024-04-24T23:05:11.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.118+0000 7fc106ffd640 1 -- 172.21.15.32:0/2193626622 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc108018070 con 0x7fc0fc004010 2024-04-24T23:05:11.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.118+0000 7fc106ffd640 1 -- 172.21.15.32:0/2193626622 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10800fd60 con 0x7fc0fc004010 2024-04-24T23:05:11.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.118+0000 7fc114c58640 1 -- 172.21.15.32:0/2193626622 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc11015da00 con 0x7fc0fc004010 2024-04-24T23:05:11.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.119+0000 7fc114c58640 1 -- 172.21.15.32:0/2193626622 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc11015dfc0 con 0x7fc0fc004010 2024-04-24T23:05:11.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.120+0000 7fc114c58640 1 -- 172.21.15.32:0/2193626622 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc110068430 con 0x7fc0fc004010 2024-04-24T23:05:11.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.120+0000 7fc106ffd640 1 -- 172.21.15.32:0/2193626622 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc10801d460 con 0x7fc0fc004010 2024-04-24T23:05:11.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.120+0000 7fc106ffd640 1 --2- 172.21.15.32:0/2193626622 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc0f006be30 0x7fc0f006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:11.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.121+0000 7fc106ffd640 1 -- 172.21.15.32:0/2193626622 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5709+0+0 (secure 0 0 0) 0x7fc10808aeb0 con 0x7fc0fc004010 2024-04-24T23:05:11.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.121+0000 7fc10dd9b640 1 --2- 172.21.15.32:0/2193626622 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc0f006be30 0x7fc0f006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:11.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.121+0000 7fc10dd9b640 1 --2- 172.21.15.32:0/2193626622 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc0f006be30 0x7fc0f006e2f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc0f800ba00 tx=0x7fc0f8030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:11.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.125+0000 7fc106ffd640 1 -- 172.21.15.32:0/2193626622 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc11015c990 con 0x7fc0fc004010 2024-04-24T23:05:11.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.307+0000 7fc114c58640 1 -- 172.21.15.32:0/2193626622 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc1101500e0 con 0x7fc0fc004010 2024-04-24T23:05:11.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.308+0000 7fc106ffd640 1 -- 172.21.15.32:0/2193626622 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v128) ==== 93+0+10 (secure 0 0 0) 0x7fc108059860 con 0x7fc0fc004010 2024-04-24T23:05:11.309 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T23:05:11.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.309+0000 7fc114c58640 1 -- 172.21.15.32:0/2193626622 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc0f006be30 msgr2=0x7fc0f006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:11.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.309+0000 7fc114c58640 1 --2- 172.21.15.32:0/2193626622 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc0f006be30 0x7fc0f006e2f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc0f800ba00 tx=0x7fc0f8030000 comp rx=0 tx=0).stop 2024-04-24T23:05:11.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.309+0000 7fc114c58640 1 -- 172.21.15.32:0/2193626622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0fc004010 msgr2=0x7fc11015f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:11.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.309+0000 7fc114c58640 1 --2- 172.21.15.32:0/2193626622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0fc004010 0x7fc11015f630 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fc108007310 tx=0x7fc1080077d0 comp rx=0 tx=0).stop 2024-04-24T23:05:11.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.310+0000 7fc114c58640 1 -- 172.21.15.32:0/2193626622 shutdown_connections 2024-04-24T23:05:11.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.310+0000 7fc114c58640 1 --2- 172.21.15.32:0/2193626622 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc0f006be30 0x7fc0f006e2f0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:11.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.310+0000 7fc114c58640 1 --2- 172.21.15.32:0/2193626622 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc110150840 0x7fc110154920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:11.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.310+0000 7fc114c58640 1 --2- 172.21.15.32:0/2193626622 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc11014ef90 0x7fc110150300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:11.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.310+0000 7fc114c58640 1 --2- 172.21.15.32:0/2193626622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0fc004010 0x7fc11015f630 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:11.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.310+0000 7fc114c58640 1 -- 172.21.15.32:0/2193626622 >> 172.21.15.32:0/2193626622 conn(0x7fc11006b110 msgr2=0x7fc1100b3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:11.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.310+0000 7fc114c58640 1 -- 172.21.15.32:0/2193626622 shutdown_connections 2024-04-24T23:05:11.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.310+0000 7fc114c58640 1 -- 172.21.15.32:0/2193626622 wait complete. 2024-04-24T23:05:11.322 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:05:11.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.425+0000 7fc499b21640 1 Processor -- start 2024-04-24T23:05:11.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.425+0000 7fc499b21640 1 -- start start 2024-04-24T23:05:11.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.426+0000 7fc499b21640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc494162320 0x7fc4940ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:11.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.426+0000 7fc499b21640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc4940b5af0 0x7fc494162700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:11.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.426+0000 7fc499b21640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4940cd270 con 0x7fc494162c40 2024-04-24T23:05:11.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.426+0000 7fc499b21640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fc4940b5660 con 0x7fc4940b5af0 2024-04-24T23:05:11.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.426+0000 7fc499b21640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc494071610 con 0x7fc494162320 2024-04-24T23:05:11.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.426+0000 7fc4937fe640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc494162320 0x7fc4940ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:11.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.426+0000 7fc492ffd640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc4940b5af0 0x7fc494162700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:11.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.426+0000 7fc493fff640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fc494162c40 0x7fc4940ae8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:55726/0 (socket says 172.21.15.32:55726) 2024-04-24T23:05:11.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.426+0000 7fc493fff640 1 -- 172.21.15.32:0/1786438199 learned_addr learned my addr 172.21.15.32:0/1786438199 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:05:11.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.426+0000 7fc4927fc640 1 -- 172.21.15.32:0/1786438199 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 533172146 0 0) 0x7fc4940cd270 con 0x7fc494162c40 2024-04-24T23:05:11.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.426+0000 7fc4927fc640 1 -- 172.21.15.32:0/1786438199 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc47c003520 con 0x7fc494162c40 2024-04-24T23:05:11.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.426+0000 7fc4937fe640 1 -- 172.21.15.32:0/1786438199 >> v2:172.21.15.121:3300/0 conn(0x7fc4940b5af0 msgr2=0x7fc494162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:11.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.427+0000 7fc4937fe640 1 --2- 172.21.15.32:0/1786438199 >> v2:172.21.15.121:3300/0 conn(0x7fc4940b5af0 0x7fc494162700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:11.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.427+0000 7fc4937fe640 1 -- 172.21.15.32:0/1786438199 >> v1:172.21.15.121:6789/0 conn(0x7fc494162c40 legacy=0x7fc4940ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:11.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.427+0000 7fc4937fe640 1 -- 172.21.15.32:0/1786438199 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4940718f0 con 0x7fc494162320 2024-04-24T23:05:11.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.427+0000 7fc492ffd640 1 --2- 172.21.15.32:0/1786438199 >> v2:172.21.15.121:3300/0 conn(0x7fc4940b5af0 0x7fc494162700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:05:11.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.427+0000 7fc4937fe640 1 --2- 172.21.15.32:0/1786438199 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc494162320 0x7fc4940ccc90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc4840097a0 tx=0x7fc48402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7a1d9b44733c9c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:11.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.427+0000 7fc4927fc640 1 -- 172.21.15.32:0/1786438199 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc484037070 con 0x7fc494162320 2024-04-24T23:05:11.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.427+0000 7fc4927fc640 1 -- 172.21.15.32:0/1786438199 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc48402ae50 con 0x7fc494162320 2024-04-24T23:05:11.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.427+0000 7fc4927fc640 1 -- 172.21.15.32:0/1786438199 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc484032640 con 0x7fc494162320 2024-04-24T23:05:11.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.428+0000 7fc499b21640 1 -- 172.21.15.32:0/1786438199 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc494162320 msgr2=0x7fc4940ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:11.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.428+0000 7fc499b21640 1 --2- 172.21.15.32:0/1786438199 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc494162320 0x7fc4940ccc90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc4840097a0 tx=0x7fc48402a100 comp rx=0 tx=0).stop 2024-04-24T23:05:11.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.428+0000 7fc499b21640 1 -- 172.21.15.32:0/1786438199 shutdown_connections 2024-04-24T23:05:11.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.428+0000 7fc499b21640 1 --2- 172.21.15.32:0/1786438199 >> v2:172.21.15.121:3300/0 conn(0x7fc4940b5af0 0x7fc494162700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:11.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.428+0000 7fc499b21640 1 --2- 172.21.15.32:0/1786438199 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc494162320 0x7fc4940ccc90 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:11.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.428+0000 7fc499b21640 1 -- 172.21.15.32:0/1786438199 >> 172.21.15.32:0/1786438199 conn(0x7fc4940d0ae0 msgr2=0x7fc4940d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:11.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.428+0000 7fc499b21640 1 -- 172.21.15.32:0/1786438199 shutdown_connections 2024-04-24T23:05:11.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.428+0000 7fc499b21640 1 -- 172.21.15.32:0/1786438199 wait complete. 2024-04-24T23:05:11.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.429+0000 7fc499b21640 1 Processor -- start 2024-04-24T23:05:11.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.429+0000 7fc499b21640 1 -- start start 2024-04-24T23:05:11.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.430+0000 7fc499b21640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4940b5af0 0x7fc494148350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:11.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.430+0000 7fc499b21640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc494162320 0x7fc4941e8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:11.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.430+0000 7fc499b21640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc494162c40 0x7fc4941e9090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:11.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.430+0000 7fc499b21640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc494067950 con 0x7fc4940b5af0 2024-04-24T23:05:11.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.430+0000 7fc499b21640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc49415c630 con 0x7fc494162c40 2024-04-24T23:05:11.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.430+0000 7fc499b21640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc494071bd0 con 0x7fc494162320 2024-04-24T23:05:11.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.430+0000 7fc4937fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4940b5af0 0x7fc494148350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:11.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.430+0000 7fc4937fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4940b5af0 0x7fc494148350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55680/0 (socket says 172.21.15.32:55680) 2024-04-24T23:05:11.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.430+0000 7fc4937fe640 1 -- 172.21.15.32:0/866032817 learned_addr learned my addr 172.21.15.32:0/866032817 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:11.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.430+0000 7fc492ffd640 1 --2- 172.21.15.32:0/866032817 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc494162320 0x7fc4941e8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:11.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.430+0000 7fc493fff640 1 --2- 172.21.15.32:0/866032817 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc494162c40 0x7fc4941e9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:11.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.431+0000 7fc4937fe640 1 -- 172.21.15.32:0/866032817 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc494162320 msgr2=0x7fc4941e8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:11.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.431+0000 7fc4937fe640 1 --2- 172.21.15.32:0/866032817 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc494162320 0x7fc4941e8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:11.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.431+0000 7fc4937fe640 1 -- 172.21.15.32:0/866032817 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc494162c40 msgr2=0x7fc4941e9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:11.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.431+0000 7fc4937fe640 1 --2- 172.21.15.32:0/866032817 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc494162c40 0x7fc4941e9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:11.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.431+0000 7fc4937fe640 1 -- 172.21.15.32:0/866032817 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4940cdba0 con 0x7fc4940b5af0 2024-04-24T23:05:11.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.431+0000 7fc4937fe640 1 --2- 172.21.15.32:0/866032817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4940b5af0 0x7fc494148350 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fc48402a140 tx=0x7fc4840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:11.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.431+0000 7fc47bfff640 1 -- 172.21.15.32:0/866032817 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc484041070 con 0x7fc4940b5af0 2024-04-24T23:05:11.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.431+0000 7fc47bfff640 1 -- 172.21.15.32:0/866032817 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc48403c460 con 0x7fc4940b5af0 2024-04-24T23:05:11.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.431+0000 7fc499b21640 1 -- 172.21.15.32:0/866032817 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4941e9810 con 0x7fc4940b5af0 2024-04-24T23:05:11.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.431+0000 7fc47bfff640 1 -- 172.21.15.32:0/866032817 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc484037040 con 0x7fc4940b5af0 2024-04-24T23:05:11.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.431+0000 7fc499b21640 1 -- 172.21.15.32:0/866032817 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4941e9dd0 con 0x7fc4940b5af0 2024-04-24T23:05:11.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.432+0000 7fc499b21640 1 -- 172.21.15.32:0/866032817 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc494068430 con 0x7fc4940b5af0 2024-04-24T23:05:11.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.432+0000 7fc47bfff640 1 -- 172.21.15.32:0/866032817 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc484044070 con 0x7fc4940b5af0 2024-04-24T23:05:11.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.433+0000 7fc47bfff640 1 --2- 172.21.15.32:0/866032817 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc48006be30 0x7fc48006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:11.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.433+0000 7fc47bfff640 1 -- 172.21.15.32:0/866032817 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5709+0+0 (secure 0 0 0) 0x7fc4840ab350 con 0x7fc4940b5af0 2024-04-24T23:05:11.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.433+0000 7fc492ffd640 1 --2- 172.21.15.32:0/866032817 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc48006be30 0x7fc48006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:11.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.434+0000 7fc492ffd640 1 --2- 172.21.15.32:0/866032817 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc48006be30 0x7fc48006e2f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc488009560 tx=0x7fc488009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:11.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.437+0000 7fc47bfff640 1 -- 172.21.15.32:0/866032817 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc484079740 con 0x7fc4940b5af0 2024-04-24T23:05:11.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.617+0000 7fc499b21640 1 -- 172.21.15.32:0/866032817 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc494067730 con 0x7fc4940b5af0 2024-04-24T23:05:11.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.618+0000 7fc47bfff640 1 -- 172.21.15.32:0/866032817 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v128) ==== 104+0+11 (secure 0 0 0) 0x7fc48407da40 con 0x7fc4940b5af0 2024-04-24T23:05:11.619 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-24T23:05:11.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.620+0000 7fc499b21640 1 -- 172.21.15.32:0/866032817 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc48006be30 msgr2=0x7fc48006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:11.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.620+0000 7fc499b21640 1 --2- 172.21.15.32:0/866032817 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc48006be30 0x7fc48006e2f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc488009560 tx=0x7fc488009190 comp rx=0 tx=0).stop 2024-04-24T23:05:11.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.620+0000 7fc499b21640 1 -- 172.21.15.32:0/866032817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4940b5af0 msgr2=0x7fc494148350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:11.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.620+0000 7fc499b21640 1 --2- 172.21.15.32:0/866032817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4940b5af0 0x7fc494148350 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fc48402a140 tx=0x7fc4840046c0 comp rx=0 tx=0).stop 2024-04-24T23:05:11.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.620+0000 7fc499b21640 1 -- 172.21.15.32:0/866032817 shutdown_connections 2024-04-24T23:05:11.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.620+0000 7fc499b21640 1 --2- 172.21.15.32:0/866032817 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc48006be30 0x7fc48006e2f0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:11.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.620+0000 7fc499b21640 1 --2- 172.21.15.32:0/866032817 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc494162c40 0x7fc4941e9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:11.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.620+0000 7fc499b21640 1 --2- 172.21.15.32:0/866032817 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc494162320 0x7fc4941e8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:11.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.620+0000 7fc499b21640 1 --2- 172.21.15.32:0/866032817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4940b5af0 0x7fc494148350 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:11.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.620+0000 7fc499b21640 1 -- 172.21.15.32:0/866032817 >> 172.21.15.32:0/866032817 conn(0x7fc4940d0ae0 msgr2=0x7fc4940b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:11.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.620+0000 7fc499b21640 1 -- 172.21.15.32:0/866032817 shutdown_connections 2024-04-24T23:05:11.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.620+0000 7fc499b21640 1 -- 172.21.15.32:0/866032817 wait complete. 2024-04-24T23:05:11.632 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:05:11.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.734+0000 7f01e9390640 1 Processor -- start 2024-04-24T23:05:11.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.734+0000 7f01e9390640 1 -- start start 2024-04-24T23:05:11.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.735+0000 7f01e9390640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01e415c7b0 0x7f01e40cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:11.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.735+0000 7f01e9390640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f01e406b820 0x7f01e4070d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:11.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.735+0000 7f01e9390640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01e406c7b0 con 0x7f01e406d390 2024-04-24T23:05:11.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.735+0000 7f01e9390640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f01e415cb90 con 0x7f01e406b820 2024-04-24T23:05:11.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.735+0000 7f01e9390640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f01e406ca90 con 0x7f01e415c7b0 2024-04-24T23:05:11.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.735+0000 7f01e37fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f01e406b820 0x7f01e4070d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:11.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.735+0000 7f01e27fc640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f01e406d390 0x7f01e406b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45158/0 (socket says 172.21.15.32:45158) 2024-04-24T23:05:11.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.735+0000 7f01e27fc640 1 -- 172.21.15.32:0/2810675992 learned_addr learned my addr 172.21.15.32:0/2810675992 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:05:11.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.735+0000 7f01e37fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f01e406b820 0x7f01e4070d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55692/0 (socket says 172.21.15.32:55692) 2024-04-24T23:05:11.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.735+0000 7f01e2ffd640 1 --2- 172.21.15.32:0/2810675992 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01e415c7b0 0x7f01e40cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:11.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.735+0000 7f01e37fe640 1 -- 172.21.15.32:0/2810675992 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01e415c7b0 msgr2=0x7f01e40cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:11.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.735+0000 7f01e37fe640 1 --2- 172.21.15.32:0/2810675992 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01e415c7b0 0x7f01e40cdbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:11.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.735+0000 7f01e37fe640 1 -- 172.21.15.32:0/2810675992 >> v1:172.21.15.32:6789/0 conn(0x7f01e406d390 legacy=0x7f01e406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:11.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.735+0000 7f01e1ffb640 1 -- 172.21.15.32:0/2810675992 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2383542363 0 0) 0x7f01e406c7b0 con 0x7f01e406d390 2024-04-24T23:05:11.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.735+0000 7f01e37fe640 1 -- 172.21.15.32:0/2810675992 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01e406cd70 con 0x7f01e406b820 2024-04-24T23:05:11.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.736+0000 7f01e37fe640 1 --2- 172.21.15.32:0/2810675992 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01e406b820 0x7f01e4070d70 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f01d4009d90 tx=0x7f01d402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d78fe2dbf0b19532 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:11.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.736+0000 7f01e1ffb640 1 -- 172.21.15.32:0/2810675992 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d4007740 con 0x7f01e406b820 2024-04-24T23:05:11.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.736+0000 7f01e1ffb640 1 -- 172.21.15.32:0/2810675992 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01d40078e0 con 0x7f01e406b820 2024-04-24T23:05:11.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.736+0000 7f01e1ffb640 1 -- 172.21.15.32:0/2810675992 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d4007cb0 con 0x7f01e406b820 2024-04-24T23:05:11.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.737+0000 7f01e9390640 1 -- 172.21.15.32:0/2810675992 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01e406b820 msgr2=0x7f01e4070d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:11.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.737+0000 7f01e9390640 1 --2- 172.21.15.32:0/2810675992 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01e406b820 0x7f01e4070d70 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f01d4009d90 tx=0x7f01d402a660 comp rx=0 tx=0).stop 2024-04-24T23:05:11.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.737+0000 7f01e9390640 1 -- 172.21.15.32:0/2810675992 shutdown_connections 2024-04-24T23:05:11.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.737+0000 7f01e9390640 1 --2- 172.21.15.32:0/2810675992 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01e406b820 0x7f01e4070d70 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:11.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.737+0000 7f01e9390640 1 --2- 172.21.15.32:0/2810675992 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01e415c7b0 0x7f01e40cdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:11.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.737+0000 7f01e9390640 1 -- 172.21.15.32:0/2810675992 >> 172.21.15.32:0/2810675992 conn(0x7f01e40d0730 msgr2=0x7f01e40d0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:11.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.737+0000 7f01e9390640 1 -- 172.21.15.32:0/2810675992 shutdown_connections 2024-04-24T23:05:11.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.737+0000 7f01e9390640 1 -- 172.21.15.32:0/2810675992 wait complete. 2024-04-24T23:05:11.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.738+0000 7f01e9390640 1 Processor -- start 2024-04-24T23:05:11.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.738+0000 7f01e9390640 1 -- start start 2024-04-24T23:05:11.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.739+0000 7f01e9390640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01e406b820 0x7f01e40b24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:11.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.739+0000 7f01e9390640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01e406d390 0x7f01e4155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:11.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.739+0000 7f01e9390640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01e415c7b0 0x7f01e415f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:11.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.739+0000 7f01e9390640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f01e40d0450 con 0x7f01e406b820 2024-04-24T23:05:11.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.739+0000 7f01e9390640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f01e40d0170 con 0x7f01e406d390 2024-04-24T23:05:11.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.739+0000 7f01e9390640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f01e40afc30 con 0x7f01e415c7b0 2024-04-24T23:05:11.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.739+0000 7f01e2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01e406b820 0x7f01e40b24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:11.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.739+0000 7f01e2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01e406b820 0x7f01e40b24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55706/0 (socket says 172.21.15.32:55706) 2024-04-24T23:05:11.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.739+0000 7f01e2ffd640 1 -- 172.21.15.32:0/409902800 learned_addr learned my addr 172.21.15.32:0/409902800 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:11.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.739+0000 7f01e37fe640 1 --2- 172.21.15.32:0/409902800 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01e415c7b0 0x7f01e415f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:11.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.739+0000 7f01e27fc640 1 --2- 172.21.15.32:0/409902800 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01e406d390 0x7f01e4155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:11.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.739+0000 7f01e2ffd640 1 -- 172.21.15.32:0/409902800 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01e415c7b0 msgr2=0x7f01e415f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:11.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.739+0000 7f01e2ffd640 1 --2- 172.21.15.32:0/409902800 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01e415c7b0 0x7f01e415f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:11.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.739+0000 7f01e2ffd640 1 -- 172.21.15.32:0/409902800 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01e406d390 msgr2=0x7f01e4155bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:11.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.739+0000 7f01e2ffd640 1 --2- 172.21.15.32:0/409902800 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01e406d390 0x7f01e4155bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:11.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.739+0000 7f01e2ffd640 1 -- 172.21.15.32:0/409902800 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01e40ae7f0 con 0x7f01e406b820 2024-04-24T23:05:11.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.739+0000 7f01e2ffd640 1 --2- 172.21.15.32:0/409902800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01e406b820 0x7f01e40b24e0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f01dc00d800 tx=0x7f01dc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:11.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.740+0000 7f01cb7fe640 1 -- 172.21.15.32:0/409902800 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01dc00b6c0 con 0x7f01e406b820 2024-04-24T23:05:11.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.740+0000 7f01cb7fe640 1 -- 172.21.15.32:0/409902800 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01dc019070 con 0x7f01e406b820 2024-04-24T23:05:11.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.740+0000 7f01e9390640 1 -- 172.21.15.32:0/409902800 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01e40aead0 con 0x7f01e406b820 2024-04-24T23:05:11.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.740+0000 7f01cb7fe640 1 -- 172.21.15.32:0/409902800 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01dc00bac0 con 0x7f01e406b820 2024-04-24T23:05:11.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.740+0000 7f01e9390640 1 -- 172.21.15.32:0/409902800 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01e40af090 con 0x7f01e406b820 2024-04-24T23:05:11.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.741+0000 7f01e9390640 1 -- 172.21.15.32:0/409902800 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01e4068430 con 0x7f01e406b820 2024-04-24T23:05:11.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.741+0000 7f01cb7fe640 1 -- 172.21.15.32:0/409902800 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f01dc00bc60 con 0x7f01e406b820 2024-04-24T23:05:11.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.741+0000 7f01cb7fe640 1 --2- 172.21.15.32:0/409902800 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f01d006be30 0x7f01d006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:11.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.742+0000 7f01cb7fe640 1 -- 172.21.15.32:0/409902800 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5709+0+0 (secure 0 0 0) 0x7f01dc08b2e0 con 0x7f01e406b820 2024-04-24T23:05:11.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.742+0000 7f01e27fc640 1 --2- 172.21.15.32:0/409902800 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f01d006be30 0x7f01d006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:11.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.742+0000 7f01e27fc640 1 --2- 172.21.15.32:0/409902800 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f01d006be30 0x7f01d006e2f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f01e40cbc70 tx=0x7f01d8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:11.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.746+0000 7f01cb7fe640 1 -- 172.21.15.32:0/409902800 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f01dc059d20 con 0x7f01e406b820 2024-04-24T23:05:11.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.931+0000 7f01e9390640 1 -- 172.21.15.32:0/409902800 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f01e40aa4a0 con 0x7f01e406b820 2024-04-24T23:05:11.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.932+0000 7f01cb7fe640 1 -- 172.21.15.32:0/409902800 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v128) ==== 100+0+11 (secure 0 0 0) 0x7f01dc059620 con 0x7f01e406b820 2024-04-24T23:05:11.933 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-24T23:05:11.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.934+0000 7f01e9390640 1 -- 172.21.15.32:0/409902800 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f01d006be30 msgr2=0x7f01d006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:11.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.934+0000 7f01e9390640 1 --2- 172.21.15.32:0/409902800 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f01d006be30 0x7f01d006e2f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f01e40cbc70 tx=0x7f01d8009040 comp rx=0 tx=0).stop 2024-04-24T23:05:11.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.934+0000 7f01e9390640 1 -- 172.21.15.32:0/409902800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01e406b820 msgr2=0x7f01e40b24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:11.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.934+0000 7f01e9390640 1 --2- 172.21.15.32:0/409902800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01e406b820 0x7f01e40b24e0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f01dc00d800 tx=0x7f01dc00dcc0 comp rx=0 tx=0).stop 2024-04-24T23:05:11.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.934+0000 7f01e9390640 1 -- 172.21.15.32:0/409902800 shutdown_connections 2024-04-24T23:05:11.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.934+0000 7f01e9390640 1 --2- 172.21.15.32:0/409902800 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f01d006be30 0x7f01d006e2f0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:11.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.934+0000 7f01e9390640 1 --2- 172.21.15.32:0/409902800 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01e415c7b0 0x7f01e415f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:11.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.934+0000 7f01e9390640 1 --2- 172.21.15.32:0/409902800 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01e406d390 0x7f01e4155bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:11.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.934+0000 7f01e9390640 1 --2- 172.21.15.32:0/409902800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01e406b820 0x7f01e40b24e0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:11.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.934+0000 7f01e9390640 1 -- 172.21.15.32:0/409902800 >> 172.21.15.32:0/409902800 conn(0x7f01e40d0730 msgr2=0x7f01e4156bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:11.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.934+0000 7f01e9390640 1 -- 172.21.15.32:0/409902800 shutdown_connections 2024-04-24T23:05:11.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:11.934+0000 7f01e9390640 1 -- 172.21.15.32:0/409902800 wait complete. 2024-04-24T23:05:11.946 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T23:05:12.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.046+0000 7f516e55c640 1 Processor -- start 2024-04-24T23:05:12.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.046+0000 7f516e55c640 1 -- start start 2024-04-24T23:05:12.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.047+0000 7f516e55c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f516815b380 0x7f516806c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:12.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.047+0000 7f516e55c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f516815cb00 0x7f516806cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:12.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.047+0000 7f516e55c640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51680b4520 con 0x7f51680ce430 2024-04-24T23:05:12.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.047+0000 7f516e55c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f51680cef90 con 0x7f516815cb00 2024-04-24T23:05:12.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.047+0000 7f516e55c640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f51680b4800 con 0x7f516815b380 2024-04-24T23:05:12.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.047+0000 7f51677fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f516815cb00 0x7f516806cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:12.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.047+0000 7f516caf9640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f51680ce430 0x7f51680ce880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45160/0 (socket says 172.21.15.32:45160) 2024-04-24T23:05:12.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.047+0000 7f516caf9640 1 -- 172.21.15.32:0/1347182097 learned_addr learned my addr 172.21.15.32:0/1347182097 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:05:12.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.047+0000 7f51677fe640 1 -- 172.21.15.32:0/1347182097 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f516815b380 msgr2=0x7f516806c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:12.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.047+0000 7f5167fff640 1 --2- 172.21.15.32:0/1347182097 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f516815b380 0x7f516806c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:12.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.047+0000 7f51677fe640 1 --2- 172.21.15.32:0/1347182097 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f516815b380 0x7f516806c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:12.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.047+0000 7f51677fe640 1 -- 172.21.15.32:0/1347182097 >> v1:172.21.15.32:6789/0 conn(0x7f51680ce430 legacy=0x7f51680ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:12.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.047+0000 7f5166ffd640 1 -- 172.21.15.32:0/1347182097 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4145585007 0 0) 0x7f51680b4520 con 0x7f51680ce430 2024-04-24T23:05:12.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.047+0000 7f51677fe640 1 -- 172.21.15.32:0/1347182097 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51680b4ae0 con 0x7f516815cb00 2024-04-24T23:05:12.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.048+0000 7f51677fe640 1 --2- 172.21.15.32:0/1347182097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f516815cb00 0x7f516806cb30 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f515c0097a0 tx=0x7f515c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e9493c5de916a41 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:12.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.048+0000 7f5166ffd640 1 -- 172.21.15.32:0/1347182097 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515c037070 con 0x7f516815cb00 2024-04-24T23:05:12.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.048+0000 7f5166ffd640 1 -- 172.21.15.32:0/1347182097 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f515c02ae50 con 0x7f516815cb00 2024-04-24T23:05:12.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.048+0000 7f5166ffd640 1 -- 172.21.15.32:0/1347182097 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515c032660 con 0x7f516815cb00 2024-04-24T23:05:12.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.049+0000 7f516e55c640 1 -- 172.21.15.32:0/1347182097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f516815cb00 msgr2=0x7f516806cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:12.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.049+0000 7f516e55c640 1 --2- 172.21.15.32:0/1347182097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f516815cb00 0x7f516806cb30 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f515c0097a0 tx=0x7f515c02a100 comp rx=0 tx=0).stop 2024-04-24T23:05:12.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.049+0000 7f516e55c640 1 -- 172.21.15.32:0/1347182097 shutdown_connections 2024-04-24T23:05:12.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.049+0000 7f516e55c640 1 --2- 172.21.15.32:0/1347182097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f516815cb00 0x7f516806cb30 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:12.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.049+0000 7f516e55c640 1 --2- 172.21.15.32:0/1347182097 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f516815b380 0x7f516806c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:12.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.049+0000 7f516e55c640 1 -- 172.21.15.32:0/1347182097 >> 172.21.15.32:0/1347182097 conn(0x7f516806b110 msgr2=0x7f516806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:12.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.049+0000 7f516e55c640 1 -- 172.21.15.32:0/1347182097 shutdown_connections 2024-04-24T23:05:12.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.049+0000 7f516e55c640 1 -- 172.21.15.32:0/1347182097 wait complete. 2024-04-24T23:05:12.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.050+0000 7f516e55c640 1 Processor -- start 2024-04-24T23:05:12.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.050+0000 7f516e55c640 1 -- start start 2024-04-24T23:05:12.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.051+0000 7f516e55c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51680ce430 0x7f51681f94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:12.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.051+0000 7f516e55c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f516815b380 0x7f51681591f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:12.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.051+0000 7f516e55c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f516815cb00 0x7f51681500c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:12.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.051+0000 7f516e55c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f516815b760 con 0x7f51680ce430 2024-04-24T23:05:12.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.051+0000 7f516e55c640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f51680b50a0 con 0x7f516815cb00 2024-04-24T23:05:12.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.051+0000 7f516e55c640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f51680b55c0 con 0x7f516815b380 2024-04-24T23:05:12.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.051+0000 7f5167fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51680ce430 0x7f51681f94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:12.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.051+0000 7f5167fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51680ce430 0x7f51681f94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55722/0 (socket says 172.21.15.32:55722) 2024-04-24T23:05:12.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.051+0000 7f5167fff640 1 -- 172.21.15.32:0/1939204120 learned_addr learned my addr 172.21.15.32:0/1939204120 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:12.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.051+0000 7f51677fe640 1 --2- 172.21.15.32:0/1939204120 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f516815b380 0x7f51681591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:12.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.051+0000 7f516caf9640 1 --2- 172.21.15.32:0/1939204120 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f516815cb00 0x7f51681500c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:12.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.051+0000 7f5167fff640 1 -- 172.21.15.32:0/1939204120 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f516815b380 msgr2=0x7f51681591f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:12.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.051+0000 7f5167fff640 1 --2- 172.21.15.32:0/1939204120 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f516815b380 0x7f51681591f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:12.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.051+0000 7f5167fff640 1 -- 172.21.15.32:0/1939204120 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f516815cb00 msgr2=0x7f51681500c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:12.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.051+0000 7f5167fff640 1 --2- 172.21.15.32:0/1939204120 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f516815cb00 0x7f51681500c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:12.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.051+0000 7f5167fff640 1 -- 172.21.15.32:0/1939204120 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5168159730 con 0x7f51680ce430 2024-04-24T23:05:12.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.051+0000 7f5167fff640 1 --2- 172.21.15.32:0/1939204120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51680ce430 0x7f51681f94e0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f515800c820 tx=0x7f515800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:12.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.052+0000 7f5164ff9640 1 -- 172.21.15.32:0/1939204120 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5158015070 con 0x7f51680ce430 2024-04-24T23:05:12.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.052+0000 7f5164ff9640 1 -- 172.21.15.32:0/1939204120 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5158004240 con 0x7f51680ce430 2024-04-24T23:05:12.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.052+0000 7f516e55c640 1 -- 172.21.15.32:0/1939204120 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51680707b0 con 0x7f51680ce430 2024-04-24T23:05:12.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.052+0000 7f5164ff9640 1 -- 172.21.15.32:0/1939204120 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515800ce10 con 0x7f51680ce430 2024-04-24T23:05:12.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.052+0000 7f516e55c640 1 -- 172.21.15.32:0/1939204120 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5168070d70 con 0x7f51680ce430 2024-04-24T23:05:12.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.053+0000 7f516e55c640 1 -- 172.21.15.32:0/1939204120 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5168068430 con 0x7f51680ce430 2024-04-24T23:05:12.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.053+0000 7f5164ff9640 1 -- 172.21.15.32:0/1939204120 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5158007830 con 0x7f51680ce430 2024-04-24T23:05:12.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.054+0000 7f5164ff9640 1 --2- 172.21.15.32:0/1939204120 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f515406be30 0x7f515406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:12.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.054+0000 7f5164ff9640 1 -- 172.21.15.32:0/1939204120 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5709+0+0 (secure 0 0 0) 0x7f515808aca0 con 0x7f51680ce430 2024-04-24T23:05:12.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.054+0000 7f51677fe640 1 --2- 172.21.15.32:0/1939204120 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f515406be30 0x7f515406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:12.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.054+0000 7f51677fe640 1 --2- 172.21.15.32:0/1939204120 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f515406be30 0x7f515406e2f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f515c034000 tx=0x7f515c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:12.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.058+0000 7f5164ff9640 1 -- 172.21.15.32:0/1939204120 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f51580596e0 con 0x7f51680ce430 2024-04-24T23:05:12.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.253+0000 7f516e55c640 1 -- 172.21.15.32:0/1939204120 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f51680afd60 con 0x7f51680ce430 2024-04-24T23:05:12.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.254+0000 7f5164ff9640 1 -- 172.21.15.32:0/1939204120 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v128) ==== 80+0+0 (secure 0 0 0) 0x7f5158058fe0 con 0x7f51680ce430 2024-04-24T23:05:12.255 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2024-04-24T23:05:12.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.256+0000 7f516e55c640 1 -- 172.21.15.32:0/1939204120 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f515406be30 msgr2=0x7f515406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:12.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.256+0000 7f516e55c640 1 --2- 172.21.15.32:0/1939204120 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f515406be30 0x7f515406e2f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f515c034000 tx=0x7f515c02e000 comp rx=0 tx=0).stop 2024-04-24T23:05:12.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.256+0000 7f516e55c640 1 -- 172.21.15.32:0/1939204120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51680ce430 msgr2=0x7f51681f94e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:12.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.256+0000 7f516e55c640 1 --2- 172.21.15.32:0/1939204120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51680ce430 0x7f51681f94e0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f515800c820 tx=0x7f515800cce0 comp rx=0 tx=0).stop 2024-04-24T23:05:12.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.256+0000 7f516e55c640 1 -- 172.21.15.32:0/1939204120 shutdown_connections 2024-04-24T23:05:12.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.256+0000 7f516e55c640 1 --2- 172.21.15.32:0/1939204120 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f515406be30 0x7f515406e2f0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:12.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.256+0000 7f516e55c640 1 --2- 172.21.15.32:0/1939204120 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f516815cb00 0x7f51681500c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:12.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.256+0000 7f516e55c640 1 --2- 172.21.15.32:0/1939204120 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f516815b380 0x7f51681591f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:12.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.256+0000 7f516e55c640 1 --2- 172.21.15.32:0/1939204120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51680ce430 0x7f51681f94e0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:12.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.256+0000 7f516e55c640 1 -- 172.21.15.32:0/1939204120 >> 172.21.15.32:0/1939204120 conn(0x7f516806b110 msgr2=0x7f516806b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:12.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.256+0000 7f516e55c640 1 -- 172.21.15.32:0/1939204120 shutdown_connections 2024-04-24T23:05:12.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.256+0000 7f516e55c640 1 -- 172.21.15.32:0/1939204120 wait complete. 2024-04-24T23:05:12.268 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-24T23:05:12.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:05:12.268 DEBUG:teuthology.orchestra.run.smithi121:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-24T23:05:12.300 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2024-04-24T23:05:12.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:05:12.300 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-24T23:05:12.358 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:05:12.358 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:05:12.358 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:05:12.374 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=413 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T23:05:12.374 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:05:12.374 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:05:12.374 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:05:12.415 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3271 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T23:05:12.415 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:05:12.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.515+0000 7f185fdb7640 1 Processor -- start 2024-04-24T23:05:12.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.515+0000 7f185fdb7640 1 -- start start 2024-04-24T23:05:12.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.516+0000 7f185fdb7640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f18580b5560 0x7f185815e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:12.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.516+0000 7f185fdb7640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f18580cd3c0 0x7f185815ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:12.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.516+0000 7f185fdb7640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18580cd910 con 0x7f18580b5940 2024-04-24T23:05:12.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.516+0000 7f185fdb7640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f18580b6470 con 0x7f18580cd3c0 2024-04-24T23:05:12.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.516+0000 7f185fdb7640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f18580cdbf0 con 0x7f18580b5560 2024-04-24T23:05:12.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.516+0000 7f185d352640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f18580cd3c0 0x7f185815ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:12.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.516+0000 7f185e354640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f18580b5940 0x7f18580b5d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45176/0 (socket says 172.21.15.32:45176) 2024-04-24T23:05:12.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.516+0000 7f185d352640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f18580cd3c0 0x7f185815ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55726/0 (socket says 172.21.15.32:55726) 2024-04-24T23:05:12.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.516+0000 7f185e354640 1 -- 172.21.15.32:0/60705682 learned_addr learned my addr 172.21.15.32:0/60705682 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:05:12.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.516+0000 7f185db53640 1 --2- 172.21.15.32:0/60705682 >> v2:172.21.15.121:3300/0 conn(0x7f18580b5560 0x7f185815e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:12.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.516+0000 7f185d352640 1 -- 172.21.15.32:0/60705682 >> v2:172.21.15.121:3300/0 conn(0x7f18580b5560 msgr2=0x7f185815e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:12.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.516+0000 7f185d352640 1 --2- 172.21.15.32:0/60705682 >> v2:172.21.15.121:3300/0 conn(0x7f18580b5560 0x7f185815e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:12.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.516+0000 7f185d352640 1 -- 172.21.15.32:0/60705682 >> v1:172.21.15.32:6789/0 conn(0x7f18580b5940 legacy=0x7f18580b5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:12.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.516+0000 7f185d352640 1 -- 172.21.15.32:0/60705682 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18580cded0 con 0x7f18580cd3c0 2024-04-24T23:05:12.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.516+0000 7f185cb51640 1 -- 172.21.15.32:0/60705682 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1929032248 0 0) 0x7f18580cd910 con 0x7f18580b5940 2024-04-24T23:05:12.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.517+0000 7f185d352640 1 --2- 172.21.15.32:0/60705682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18580cd3c0 0x7f185815ec00 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f185806ede0 tx=0x7f184802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8841a29cf968d22 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:12.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.517+0000 7f185cb51640 1 -- 172.21.15.32:0/60705682 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1848037070 con 0x7f18580cd3c0 2024-04-24T23:05:12.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.517+0000 7f185cb51640 1 -- 172.21.15.32:0/60705682 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1848032440 con 0x7f18580cd3c0 2024-04-24T23:05:12.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.517+0000 7f185cb51640 1 -- 172.21.15.32:0/60705682 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1848032810 con 0x7f18580cd3c0 2024-04-24T23:05:12.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.518+0000 7f185fdb7640 1 -- 172.21.15.32:0/60705682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18580cd3c0 msgr2=0x7f185815ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:12.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.518+0000 7f185fdb7640 1 --2- 172.21.15.32:0/60705682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18580cd3c0 0x7f185815ec00 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f185806ede0 tx=0x7f184802a1a0 comp rx=0 tx=0).stop 2024-04-24T23:05:12.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.518+0000 7f185fdb7640 1 -- 172.21.15.32:0/60705682 shutdown_connections 2024-04-24T23:05:12.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.518+0000 7f185fdb7640 1 --2- 172.21.15.32:0/60705682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18580cd3c0 0x7f185815ec00 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:12.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.518+0000 7f185fdb7640 1 --2- 172.21.15.32:0/60705682 >> v2:172.21.15.121:3300/0 conn(0x7f18580b5560 0x7f185815e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:12.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.518+0000 7f185fdb7640 1 -- 172.21.15.32:0/60705682 >> 172.21.15.32:0/60705682 conn(0x7f185806b110 msgr2=0x7f185806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:12.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.518+0000 7f185fdb7640 1 -- 172.21.15.32:0/60705682 shutdown_connections 2024-04-24T23:05:12.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.518+0000 7f185fdb7640 1 -- 172.21.15.32:0/60705682 wait complete. 2024-04-24T23:05:12.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.519+0000 7f185fdb7640 1 Processor -- start 2024-04-24T23:05:12.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.519+0000 7f185fdb7640 1 -- start start 2024-04-24T23:05:12.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.520+0000 7f185fdb7640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f18580b5560 0x7f185814bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:12.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.520+0000 7f185fdb7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18580b5940 0x7f18581577a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:12.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.520+0000 7f185fdb7640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18580cd3c0 0x7f1858157ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:12.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.520+0000 7f185fdb7640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f185806c5f0 con 0x7f18580b5940 2024-04-24T23:05:12.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.520+0000 7f185fdb7640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f18580ce490 con 0x7f18580cd3c0 2024-04-24T23:05:12.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.520+0000 7f185fdb7640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f185806c770 con 0x7f18580b5560 2024-04-24T23:05:12.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.520+0000 7f185d352640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18580b5940 0x7f18581577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:12.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.520+0000 7f185d352640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18580b5940 0x7f18581577a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55734/0 (socket says 172.21.15.32:55734) 2024-04-24T23:05:12.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.520+0000 7f185d352640 1 -- 172.21.15.32:0/1301871877 learned_addr learned my addr 172.21.15.32:0/1301871877 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:12.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.520+0000 7f185db53640 1 --2- 172.21.15.32:0/1301871877 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f18580b5560 0x7f185814bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:12.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.520+0000 7f185e354640 1 --2- 172.21.15.32:0/1301871877 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18580cd3c0 0x7f1858157ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:12.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.520+0000 7f185d352640 1 -- 172.21.15.32:0/1301871877 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f18580b5560 msgr2=0x7f185814bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:12.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.520+0000 7f185d352640 1 --2- 172.21.15.32:0/1301871877 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f18580b5560 0x7f185814bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:12.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.520+0000 7f185d352640 1 -- 172.21.15.32:0/1301871877 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18580cd3c0 msgr2=0x7f1858157ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:12.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.520+0000 7f185d352640 1 --2- 172.21.15.32:0/1301871877 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18580cd3c0 0x7f1858157ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:12.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.520+0000 7f185d352640 1 -- 172.21.15.32:0/1301871877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1858149350 con 0x7f18580b5940 2024-04-24T23:05:12.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.520+0000 7f185d352640 1 --2- 172.21.15.32:0/1301871877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18580b5940 0x7f18581577a0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f1848000c00 tx=0x7f18480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:12.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.520+0000 7f184e7fc640 1 -- 172.21.15.32:0/1301871877 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1848037050 con 0x7f18580b5940 2024-04-24T23:05:12.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.520+0000 7f184e7fc640 1 -- 172.21.15.32:0/1301871877 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1848038040 con 0x7f18580b5940 2024-04-24T23:05:12.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.520+0000 7f184e7fc640 1 -- 172.21.15.32:0/1301871877 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1848004470 con 0x7f18580b5940 2024-04-24T23:05:12.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.520+0000 7f185fdb7640 1 -- 172.21.15.32:0/1301871877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1858149630 con 0x7f18580b5940 2024-04-24T23:05:12.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.520+0000 7f185fdb7640 1 -- 172.21.15.32:0/1301871877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1858149bf0 con 0x7f18580b5940 2024-04-24T23:05:12.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.521+0000 7f185fdb7640 1 -- 172.21.15.32:0/1301871877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1858068430 con 0x7f18580b5940 2024-04-24T23:05:12.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.522+0000 7f184e7fc640 1 -- 172.21.15.32:0/1301871877 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1848030900 con 0x7f18580b5940 2024-04-24T23:05:12.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.522+0000 7f184e7fc640 1 --2- 172.21.15.32:0/1301871877 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f183c06be30 0x7f183c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:12.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.522+0000 7f184e7fc640 1 -- 172.21.15.32:0/1301871877 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5709+0+0 (secure 0 0 0) 0x7f18480abb10 con 0x7f18580b5940 2024-04-24T23:05:12.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.522+0000 7f185db53640 1 --2- 172.21.15.32:0/1301871877 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f183c06be30 0x7f183c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:12.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.523+0000 7f185db53640 1 --2- 172.21.15.32:0/1301871877 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f183c06be30 0x7f183c06e2f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f1854006fd0 tx=0x7f1854008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:12.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.527+0000 7f184e7fc640 1 -- 172.21.15.32:0/1301871877 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f184807a5e0 con 0x7f18580b5940 2024-04-24T23:05:12.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.747+0000 7f185fdb7640 1 -- 172.21.15.32:0/1301871877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f185815a9a0 con 0x7f18580b5940 2024-04-24T23:05:12.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.747+0000 7f184e7fc640 1 -- 172.21.15.32:0/1301871877 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+5189 (secure 0 0 0) 0x7f1848079ee0 con 0x7f18580b5940 2024-04-24T23:05:12.749 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:05:12.749 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":51,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5972,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3336404028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3336404028},{"type":"v1","addr":"172.21.15.121:6837","nonce":3336404028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6505,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/1068557774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1068557774},{"type":"v1","addr":"172.21.15.121:6836","nonce":1068557774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":7353,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/2118719195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":2118719195},{"type":"v1","addr":"172.21.15.32:6834","nonce":2118719195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:04:55.998022+0000","modified":"2024-04-24T23:04:58.657258+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7356},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7356":{"gid":7356,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/3015290487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":3015290487},{"type":"v1","addr":"172.21.15.32:6835","nonce":3015290487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7356,"qdb_cluster":[7356]},"id":7}]} 2024-04-24T23:05:12.750 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 51 2024-04-24T23:05:12.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.749+0000 7f185fdb7640 1 -- 172.21.15.32:0/1301871877 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f183c06be30 msgr2=0x7f183c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:12.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.749+0000 7f185fdb7640 1 --2- 172.21.15.32:0/1301871877 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f183c06be30 0x7f183c06e2f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f1854006fd0 tx=0x7f1854008040 comp rx=0 tx=0).stop 2024-04-24T23:05:12.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.749+0000 7f185fdb7640 1 -- 172.21.15.32:0/1301871877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18580b5940 msgr2=0x7f18581577a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:12.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.749+0000 7f185fdb7640 1 --2- 172.21.15.32:0/1301871877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18580b5940 0x7f18581577a0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f1848000c00 tx=0x7f18480046c0 comp rx=0 tx=0).stop 2024-04-24T23:05:12.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.749+0000 7f185fdb7640 1 -- 172.21.15.32:0/1301871877 shutdown_connections 2024-04-24T23:05:12.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.749+0000 7f185fdb7640 1 --2- 172.21.15.32:0/1301871877 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f183c06be30 0x7f183c06e2f0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:12.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.749+0000 7f185fdb7640 1 --2- 172.21.15.32:0/1301871877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18580b5940 0x7f18581577a0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:12.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.749+0000 7f185fdb7640 1 --2- 172.21.15.32:0/1301871877 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18580cd3c0 0x7f1858157ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:12.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.749+0000 7f185fdb7640 1 --2- 172.21.15.32:0/1301871877 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f18580b5560 0x7f185814bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:12.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.749+0000 7f185fdb7640 1 -- 172.21.15.32:0/1301871877 >> 172.21.15.32:0/1301871877 conn(0x7f185806b110 msgr2=0x7f18580b0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:12.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.749+0000 7f185fdb7640 1 -- 172.21.15.32:0/1301871877 shutdown_connections 2024-04-24T23:05:12.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.749+0000 7f185fdb7640 1 -- 172.21.15.32:0/1301871877 wait complete. 2024-04-24T23:05:12.762 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:05:12.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.865+0000 7f6bbf109640 1 Processor -- start 2024-04-24T23:05:12.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.865+0000 7f6bbf109640 1 -- start start 2024-04-24T23:05:12.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.866+0000 7f6bbf109640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6bb8074210 0x7f6bb806c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:12.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.866+0000 7f6bbf109640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6bb80b0b80 0x7f6bb80745f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:12.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.866+0000 7f6bbf109640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6bb80b0770 con 0x7f6bb80ae140 2024-04-24T23:05:12.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.866+0000 7f6bbf109640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f6bb80b1c20 con 0x7f6bb80b0b80 2024-04-24T23:05:12.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.866+0000 7f6bbf109640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f6bb8074b30 con 0x7f6bb8074210 2024-04-24T23:05:12.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.866+0000 7f6bbd6a6640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6bb80ae140 0x7f6bb80b20b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45190/0 (socket says 172.21.15.32:45190) 2024-04-24T23:05:12.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.866+0000 7f6bbd6a6640 1 -- 172.21.15.32:0/4058554225 learned_addr learned my addr 172.21.15.32:0/4058554225 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:05:12.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.866+0000 7f6bbcea5640 1 --2- 172.21.15.32:0/4058554225 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6bb8074210 0x7f6bb806c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:12.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.866+0000 7f6bb7fff640 1 --2- 172.21.15.32:0/4058554225 >> v2:172.21.15.121:3300/0 conn(0x7f6bb80b0b80 0x7f6bb80745f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:12.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.866+0000 7f6bb77fe640 1 -- 172.21.15.32:0/4058554225 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 346640442 0 0) 0x7f6bb80b0770 con 0x7f6bb80ae140 2024-04-24T23:05:12.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.866+0000 7f6bb77fe640 1 -- 172.21.15.32:0/4058554225 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ba8003520 con 0x7f6bb80ae140 2024-04-24T23:05:12.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.866+0000 7f6bb77fe640 1 -- 172.21.15.32:0/4058554225 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2693659870 0 0) 0x7f6ba8003520 con 0x7f6bb80ae140 2024-04-24T23:05:12.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.866+0000 7f6bb77fe640 1 -- 172.21.15.32:0/4058554225 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6bb8074210 msgr2=0x7f6bb806c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:12.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.866+0000 7f6bb77fe640 1 --2- 172.21.15.32:0/4058554225 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6bb8074210 0x7f6bb806c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:12.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.866+0000 7f6bb77fe640 1 -- 172.21.15.32:0/4058554225 >> v2:172.21.15.121:3300/0 conn(0x7f6bb80b0b80 msgr2=0x7f6bb80745f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:12.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.866+0000 7f6bb77fe640 1 --2- 172.21.15.32:0/4058554225 >> v2:172.21.15.121:3300/0 conn(0x7f6bb80b0b80 0x7f6bb80745f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:12.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.866+0000 7f6bb77fe640 1 -- 172.21.15.32:0/4058554225 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bb80b1010 con 0x7f6bb80ae140 2024-04-24T23:05:12.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.866+0000 7f6bb77fe640 1 -- 172.21.15.32:0/4058554225 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f6bb00019f0 con 0x7f6bb80ae140 2024-04-24T23:05:12.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.866+0000 7f6bbcea5640 1 --2- 172.21.15.32:0/4058554225 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6bb8074210 0x7f6bb806c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:05:12.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.866+0000 7f6bb77fe640 1 -- 172.21.15.32:0/4058554225 >> v1:172.21.15.32:6789/0 conn(0x7f6bb80ae140 legacy=0x7f6bb80b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:12.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.866+0000 7f6bb77fe640 1 --2- 172.21.15.32:0/4058554225 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6bb8074210 0x7f6ba80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:12.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.867+0000 7f6bb77fe640 1 --2- 172.21.15.32:0/4058554225 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ba8003fa0 0x7f6ba8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:12.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.867+0000 7f6bb77fe640 1 --2- 172.21.15.32:0/4058554225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ba8004d40 0x7f6ba80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:12.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.867+0000 7f6bb77fe640 1 -- 172.21.15.32:0/4058554225 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb80b1c20 con 0x7f6ba8004d40 2024-04-24T23:05:12.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.867+0000 7f6bb77fe640 1 -- 172.21.15.32:0/4058554225 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb8074b30 con 0x7f6ba8003fa0 2024-04-24T23:05:12.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.867+0000 7f6bb77fe640 1 -- 172.21.15.32:0/4058554225 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f6ba800baa0 con 0x7f6bb8074210 2024-04-24T23:05:12.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.867+0000 7f6bb7fff640 1 --2- 172.21.15.32:0/4058554225 >> v2:172.21.15.121:3300/0 conn(0x7f6bb80b0b80 0x7f6bb80745f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:05:12.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.867+0000 7f6bbd6a6640 1 --2- 172.21.15.32:0/4058554225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ba8004d40 0x7f6ba80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:12.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.867+0000 7f6bbcea5640 1 --2- 172.21.15.32:0/4058554225 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6bb8074210 0x7f6ba80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:12.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.867+0000 7f6bb7fff640 1 --2- 172.21.15.32:0/4058554225 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ba8003fa0 0x7f6ba8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:12.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.867+0000 7f6bbd6a6640 1 -- 172.21.15.32:0/4058554225 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6bb8074210 msgr2=0x7f6ba80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:12.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.867+0000 7f6bbd6a6640 1 --2- 172.21.15.32:0/4058554225 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6bb8074210 0x7f6ba80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:12.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.867+0000 7f6bbd6a6640 1 -- 172.21.15.32:0/4058554225 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ba8003fa0 msgr2=0x7f6ba8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:12.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.867+0000 7f6bbd6a6640 1 --2- 172.21.15.32:0/4058554225 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ba8003fa0 0x7f6ba8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:12.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.867+0000 7f6bbd6a6640 1 -- 172.21.15.32:0/4058554225 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba8003520 con 0x7f6ba8004d40 2024-04-24T23:05:12.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.868+0000 7f6bbd6a6640 1 --2- 172.21.15.32:0/4058554225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ba8004d40 0x7f6ba80051c0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f6bb000a920 tx=0x7f6ba8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fa836f21ac016da server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:12.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.868+0000 7f6bb77fe640 1 -- 172.21.15.32:0/4058554225 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb000a4c0 con 0x7f6ba8004d40 2024-04-24T23:05:12.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.868+0000 7f6bb77fe640 1 -- 172.21.15.32:0/4058554225 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bb002f070 con 0x7f6ba8004d40 2024-04-24T23:05:12.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.868+0000 7f6bb77fe640 1 -- 172.21.15.32:0/4058554225 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb002bd40 con 0x7f6ba8004d40 2024-04-24T23:05:12.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.868+0000 7f6bbf109640 1 -- 172.21.15.32:0/4058554225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ba8004d40 msgr2=0x7f6ba80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:12.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.868+0000 7f6bbf109640 1 --2- 172.21.15.32:0/4058554225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ba8004d40 0x7f6ba80051c0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f6bb000a920 tx=0x7f6ba8005830 comp rx=0 tx=0).stop 2024-04-24T23:05:12.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.868+0000 7f6bbd6a6640 1 -- 172.21.15.32:0/4058554225 reap_dead start 2024-04-24T23:05:12.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.868+0000 7f6bbf109640 1 -- 172.21.15.32:0/4058554225 shutdown_connections 2024-04-24T23:05:12.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.868+0000 7f6bbf109640 1 -- 172.21.15.32:0/4058554225 >> 172.21.15.32:0/4058554225 conn(0x7f6bb80b2e60 msgr2=0x7f6bb80b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:12.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.868+0000 7f6bbf109640 1 -- 172.21.15.32:0/4058554225 shutdown_connections 2024-04-24T23:05:12.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.868+0000 7f6bbf109640 1 -- 172.21.15.32:0/4058554225 wait complete. 2024-04-24T23:05:12.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.870+0000 7f6bbf109640 1 Processor -- start 2024-04-24T23:05:12.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.870+0000 7f6bbf109640 1 -- start start 2024-04-24T23:05:12.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.870+0000 7f6bbf109640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ba8004d40 0x7f6bb815e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:12.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.871+0000 7f6bbf109640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6bb809c670 0x7f6bb80c9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:12.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.871+0000 7f6bbf109640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6bb80ca8f0 0x7f6bb80c9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:12.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.871+0000 7f6bbf109640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb80b18b0 con 0x7f6ba8004d40 2024-04-24T23:05:12.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.871+0000 7f6bbf109640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb80b15d0 con 0x7f6bb809c670 2024-04-24T23:05:12.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.871+0000 7f6bbf109640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f6bb80b3ba0 con 0x7f6bb80ca8f0 2024-04-24T23:05:12.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.871+0000 7f6bbcea5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ba8004d40 0x7f6bb815e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:12.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.871+0000 7f6bbcea5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ba8004d40 0x7f6bb815e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55760/0 (socket says 172.21.15.32:55760) 2024-04-24T23:05:12.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.871+0000 7f6bbcea5640 1 -- 172.21.15.32:0/3302617522 learned_addr learned my addr 172.21.15.32:0/3302617522 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:12.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.871+0000 7f6bb7fff640 1 --2- 172.21.15.32:0/3302617522 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6bb809c670 0x7f6bb80c9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:12.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.871+0000 7f6bbd6a6640 1 --2- 172.21.15.32:0/3302617522 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6bb80ca8f0 0x7f6bb80c9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:12.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.871+0000 7f6bbcea5640 1 -- 172.21.15.32:0/3302617522 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6bb80ca8f0 msgr2=0x7f6bb80c9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:12.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.871+0000 7f6bbcea5640 1 --2- 172.21.15.32:0/3302617522 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6bb80ca8f0 0x7f6bb80c9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:12.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.871+0000 7f6bbcea5640 1 -- 172.21.15.32:0/3302617522 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6bb809c670 msgr2=0x7f6bb80c9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:12.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.871+0000 7f6bbcea5640 1 --2- 172.21.15.32:0/3302617522 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6bb809c670 0x7f6bb80c9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:12.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.871+0000 7f6bbcea5640 1 -- 172.21.15.32:0/3302617522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bb80cadd0 con 0x7f6ba8004d40 2024-04-24T23:05:12.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.871+0000 7f6bbcea5640 1 --2- 172.21.15.32:0/3302617522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ba8004d40 0x7f6bb815e0b0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f6bac0115f0 tx=0x7f6bac011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:12.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.871+0000 7f6bb57fa640 1 -- 172.21.15.32:0/3302617522 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bac019070 con 0x7f6ba8004d40 2024-04-24T23:05:12.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.871+0000 7f6bb57fa640 1 -- 172.21.15.32:0/3302617522 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bac004220 con 0x7f6ba8004d40 2024-04-24T23:05:12.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.871+0000 7f6bbf109640 1 -- 172.21.15.32:0/3302617522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bb80cb0b0 con 0x7f6ba8004d40 2024-04-24T23:05:12.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.871+0000 7f6bb57fa640 1 -- 172.21.15.32:0/3302617522 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bac00f040 con 0x7f6ba8004d40 2024-04-24T23:05:12.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.871+0000 7f6bbf109640 1 -- 172.21.15.32:0/3302617522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bb815e5f0 con 0x7f6ba8004d40 2024-04-24T23:05:12.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.872+0000 7f6bbf109640 1 -- 172.21.15.32:0/3302617522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bb8074950 con 0x7f6ba8004d40 2024-04-24T23:05:12.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.873+0000 7f6bb57fa640 1 -- 172.21.15.32:0/3302617522 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6bac00f6f0 con 0x7f6ba8004d40 2024-04-24T23:05:12.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.873+0000 7f6bb57fa640 1 --2- 172.21.15.32:0/3302617522 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6b9c06be30 0x7f6b9c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:12.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.873+0000 7f6bb57fa640 1 -- 172.21.15.32:0/3302617522 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5709+0+0 (secure 0 0 0) 0x7f6bac08b0f0 con 0x7f6ba8004d40 2024-04-24T23:05:12.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.873+0000 7f6bb7fff640 1 --2- 172.21.15.32:0/3302617522 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6b9c06be30 0x7f6b9c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:12.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.874+0000 7f6bb7fff640 1 --2- 172.21.15.32:0/3302617522 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6b9c06be30 0x7f6b9c06e2f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6bb80cea60 tx=0x7f6ba4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:12.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:12.877+0000 7f6bb57fa640 1 -- 172.21.15.32:0/3302617522 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6bb8074950 con 0x7f6ba8004d40 2024-04-24T23:05:13.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.062+0000 7f6bbf109640 1 -- 172.21.15.32:0/3302617522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6bb8068430 con 0x7f6ba8004d40 2024-04-24T23:05:13.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.063+0000 7f6bb57fa640 1 -- 172.21.15.32:0/3302617522 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) ==== 74+0+16639 (secure 0 0 0) 0x7f6bac059b30 con 0x7f6ba8004d40 2024-04-24T23:05:13.065 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:05:13.065 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":128,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:05:07.320153+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:04:54.280785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-24T23:04:54.966294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:05:13.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.065+0000 7f6bbf109640 1 -- 172.21.15.32:0/3302617522 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6b9c06be30 msgr2=0x7f6b9c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:13.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.065+0000 7f6bbf109640 1 --2- 172.21.15.32:0/3302617522 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6b9c06be30 0x7f6b9c06e2f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6bb80cea60 tx=0x7f6ba4008040 comp rx=0 tx=0).stop 2024-04-24T23:05:13.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.065+0000 7f6bbf109640 1 -- 172.21.15.32:0/3302617522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ba8004d40 msgr2=0x7f6bb815e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:13.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.065+0000 7f6bbf109640 1 --2- 172.21.15.32:0/3302617522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ba8004d40 0x7f6bb815e0b0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f6bac0115f0 tx=0x7f6bac011ab0 comp rx=0 tx=0).stop 2024-04-24T23:05:13.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.065+0000 7f6bbf109640 1 -- 172.21.15.32:0/3302617522 shutdown_connections 2024-04-24T23:05:13.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.065+0000 7f6bbf109640 1 --2- 172.21.15.32:0/3302617522 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6b9c06be30 0x7f6b9c06e2f0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:13.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.065+0000 7f6bbf109640 1 --2- 172.21.15.32:0/3302617522 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6bb80ca8f0 0x7f6bb80c9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:13.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.065+0000 7f6bbf109640 1 --2- 172.21.15.32:0/3302617522 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6bb809c670 0x7f6bb80c9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:13.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.065+0000 7f6bbf109640 1 --2- 172.21.15.32:0/3302617522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ba8004d40 0x7f6bb815e0b0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:13.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.065+0000 7f6bbf109640 1 -- 172.21.15.32:0/3302617522 >> 172.21.15.32:0/3302617522 conn(0x7f6bb80b2e60 msgr2=0x7f6bb806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:13.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.065+0000 7f6bbf109640 1 -- 172.21.15.32:0/3302617522 shutdown_connections 2024-04-24T23:05:13.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.065+0000 7f6bbf109640 1 -- 172.21.15.32:0/3302617522 wait complete. 2024-04-24T23:05:13.077 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:04:54.280785+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '124', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:04:54.966294+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:05:13.078 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:05:13.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.180+0000 7f319b589640 1 Processor -- start 2024-04-24T23:05:13.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.180+0000 7f319b589640 1 -- start start 2024-04-24T23:05:13.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.181+0000 7f319b589640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f319406c5f0 0x7f319406c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:13.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.181+0000 7f319b589640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31940b44d0 0x7f31940b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:13.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.181+0000 7f319b589640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f31940af890 0x7f31940cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:13.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.181+0000 7f319b589640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f31940b6430 con 0x7f31940af890 2024-04-24T23:05:13.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.181+0000 7f319b589640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f3194156f80 con 0x7f319406c5f0 2024-04-24T23:05:13.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.181+0000 7f319b589640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f31940b6150 con 0x7f31940b44d0 2024-04-24T23:05:13.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.181+0000 7f3199b26640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f31940af890 0x7f31940cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:13.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.181+0000 7f3199b26640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f31940af890 0x7f31940cef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55772/0 (socket says 172.21.15.32:55772) 2024-04-24T23:05:13.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.181+0000 7f3199b26640 1 -- 172.21.15.32:0/1577102976 learned_addr learned my addr 172.21.15.32:0/1577102976 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:13.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.181+0000 7f3199325640 1 --2- 172.21.15.32:0/1577102976 >> v2:172.21.15.121:3300/0 conn(0x7f319406c5f0 0x7f319406c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:13.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.181+0000 7f3198b24640 1 --2- 172.21.15.32:0/1577102976 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31940b44d0 0x7f31940b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:13.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.181+0000 7f3199b26640 1 -- 172.21.15.32:0/1577102976 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31940b44d0 msgr2=0x7f31940b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:13.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.181+0000 7f3199b26640 1 --2- 172.21.15.32:0/1577102976 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31940b44d0 0x7f31940b5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:13.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.181+0000 7f3199b26640 1 -- 172.21.15.32:0/1577102976 >> v2:172.21.15.121:3300/0 conn(0x7f319406c5f0 msgr2=0x7f319406c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:13.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.181+0000 7f3199b26640 1 --2- 172.21.15.32:0/1577102976 >> v2:172.21.15.121:3300/0 conn(0x7f319406c5f0 0x7f319406c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:13.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.181+0000 7f3199b26640 1 -- 172.21.15.32:0/1577102976 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31940b39f0 con 0x7f31940af890 2024-04-24T23:05:13.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.181+0000 7f3198b24640 1 --2- 172.21.15.32:0/1577102976 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31940b44d0 0x7f31940b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:05:13.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.181+0000 7f3199325640 1 --2- 172.21.15.32:0/1577102976 >> v2:172.21.15.121:3300/0 conn(0x7f319406c5f0 0x7f319406c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:05:13.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.182+0000 7f3199b26640 1 --2- 172.21.15.32:0/1577102976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31940af890 0x7f31940cef70 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f319406c1f0 tx=0x7f318c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f77f38472496ab2d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:13.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.182+0000 7f318bfff640 1 -- 172.21.15.32:0/1577102976 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c037070 con 0x7f31940af890 2024-04-24T23:05:13.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.182+0000 7f318bfff640 1 -- 172.21.15.32:0/1577102976 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f318c032440 con 0x7f31940af890 2024-04-24T23:05:13.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.182+0000 7f318bfff640 1 -- 172.21.15.32:0/1577102976 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c032810 con 0x7f31940af890 2024-04-24T23:05:13.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.183+0000 7f319b589640 1 -- 172.21.15.32:0/1577102976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31940af890 msgr2=0x7f31940cef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:13.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.183+0000 7f319b589640 1 --2- 172.21.15.32:0/1577102976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31940af890 0x7f31940cef70 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f319406c1f0 tx=0x7f318c02a140 comp rx=0 tx=0).stop 2024-04-24T23:05:13.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.183+0000 7f319b589640 1 -- 172.21.15.32:0/1577102976 shutdown_connections 2024-04-24T23:05:13.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.183+0000 7f319b589640 1 --2- 172.21.15.32:0/1577102976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31940af890 0x7f31940cef70 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:13.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.183+0000 7f319b589640 1 --2- 172.21.15.32:0/1577102976 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31940b44d0 0x7f31940b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:13.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.183+0000 7f319b589640 1 --2- 172.21.15.32:0/1577102976 >> v2:172.21.15.121:3300/0 conn(0x7f319406c5f0 0x7f319406c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:13.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.183+0000 7f319b589640 1 -- 172.21.15.32:0/1577102976 >> 172.21.15.32:0/1577102976 conn(0x7f319406b110 msgr2=0x7f319406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:13.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.183+0000 7f319b589640 1 -- 172.21.15.32:0/1577102976 shutdown_connections 2024-04-24T23:05:13.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.183+0000 7f319b589640 1 -- 172.21.15.32:0/1577102976 wait complete. 2024-04-24T23:05:13.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.184+0000 7f319b589640 1 Processor -- start 2024-04-24T23:05:13.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.184+0000 7f319b589640 1 -- start start 2024-04-24T23:05:13.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.185+0000 7f319b589640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f319406c5f0 0x7f31941520c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:13.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.185+0000 7f319b589640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31940af890 0x7f31940b0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:13.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.185+0000 7f319b589640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31940b44d0 0x7f31940ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:13.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.185+0000 7f319b589640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f319406cf10 con 0x7f31940b44d0 2024-04-24T23:05:13.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.185+0000 7f319b589640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f31940b3fb0 con 0x7f319406c5f0 2024-04-24T23:05:13.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.185+0000 7f319b589640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f31940b5dd0 con 0x7f31940af890 2024-04-24T23:05:13.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.185+0000 7f3199b26640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31940b44d0 0x7f31940ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:13.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.185+0000 7f3199325640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f319406c5f0 0x7f31941520c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:13.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.185+0000 7f3199b26640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31940b44d0 0x7f31940ae8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55778/0 (socket says 172.21.15.32:55778) 2024-04-24T23:05:13.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.185+0000 7f3199b26640 1 -- 172.21.15.32:0/947957690 learned_addr learned my addr 172.21.15.32:0/947957690 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:13.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.185+0000 7f3198b24640 1 --2- 172.21.15.32:0/947957690 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31940af890 0x7f31940b0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:13.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.185+0000 7f3199b26640 1 -- 172.21.15.32:0/947957690 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31940af890 msgr2=0x7f31940b0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:13.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.185+0000 7f3199b26640 1 --2- 172.21.15.32:0/947957690 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31940af890 0x7f31940b0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:13.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.185+0000 7f3199b26640 1 -- 172.21.15.32:0/947957690 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f319406c5f0 msgr2=0x7f31941520c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:13.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.185+0000 7f3199b26640 1 --2- 172.21.15.32:0/947957690 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f319406c5f0 0x7f31941520c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:13.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.185+0000 7f3199b26640 1 -- 172.21.15.32:0/947957690 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f319415c2e0 con 0x7f31940b44d0 2024-04-24T23:05:13.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.185+0000 7f3199b26640 1 --2- 172.21.15.32:0/947957690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31940b44d0 0x7f31940ae8b0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f318c0090b0 tx=0x7f318c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:13.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.185+0000 7f3189ffb640 1 -- 172.21.15.32:0/947957690 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c037040 con 0x7f31940b44d0 2024-04-24T23:05:13.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.185+0000 7f3189ffb640 1 -- 172.21.15.32:0/947957690 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f318c042070 con 0x7f31940b44d0 2024-04-24T23:05:13.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.185+0000 7f319b589640 1 -- 172.21.15.32:0/947957690 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3194158f10 con 0x7f31940b44d0 2024-04-24T23:05:13.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.185+0000 7f3189ffb640 1 -- 172.21.15.32:0/947957690 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c030bd0 con 0x7f31940b44d0 2024-04-24T23:05:13.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.186+0000 7f319b589640 1 -- 172.21.15.32:0/947957690 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f319415bf70 con 0x7f31940b44d0 2024-04-24T23:05:13.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.187+0000 7f319b589640 1 -- 172.21.15.32:0/947957690 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31941595d0 con 0x7f31940b44d0 2024-04-24T23:05:13.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.187+0000 7f3189ffb640 1 -- 172.21.15.32:0/947957690 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f318c047460 con 0x7f31940b44d0 2024-04-24T23:05:13.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.187+0000 7f3189ffb640 1 --2- 172.21.15.32:0/947957690 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f317806be30 0x7f317806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:13.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.188+0000 7f3189ffb640 1 -- 172.21.15.32:0/947957690 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5709+0+0 (secure 0 0 0) 0x7f318c0ab930 con 0x7f31940b44d0 2024-04-24T23:05:13.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.188+0000 7f3199325640 1 --2- 172.21.15.32:0/947957690 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f317806be30 0x7f317806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:13.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.188+0000 7f3199325640 1 --2- 172.21.15.32:0/947957690 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f317806be30 0x7f317806e2f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f319000f500 tx=0x7f3190007770 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:13.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.192+0000 7f3189ffb640 1 -- 172.21.15.32:0/947957690 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f31941595d0 con 0x7f31940b44d0 2024-04-24T23:05:13.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.382+0000 7f319b589640 1 -- 172.21.15.32:0/947957690 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3194068430 con 0x7f31940b44d0 2024-04-24T23:05:13.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.383+0000 7f3189ffb640 1 -- 172.21.15.32:0/947957690 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v128) ==== 93+0+10 (secure 0 0 0) 0x7f318c080030 con 0x7f31940b44d0 2024-04-24T23:05:13.384 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T23:05:13.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.385+0000 7f319b589640 1 -- 172.21.15.32:0/947957690 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f317806be30 msgr2=0x7f317806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:13.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.385+0000 7f319b589640 1 --2- 172.21.15.32:0/947957690 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f317806be30 0x7f317806e2f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f319000f500 tx=0x7f3190007770 comp rx=0 tx=0).stop 2024-04-24T23:05:13.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.385+0000 7f319b589640 1 -- 172.21.15.32:0/947957690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31940b44d0 msgr2=0x7f31940ae8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:13.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.385+0000 7f319b589640 1 --2- 172.21.15.32:0/947957690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31940b44d0 0x7f31940ae8b0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f318c0090b0 tx=0x7f318c0046c0 comp rx=0 tx=0).stop 2024-04-24T23:05:13.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.385+0000 7f319b589640 1 -- 172.21.15.32:0/947957690 shutdown_connections 2024-04-24T23:05:13.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.385+0000 7f319b589640 1 --2- 172.21.15.32:0/947957690 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f317806be30 0x7f317806e2f0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:13.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.385+0000 7f319b589640 1 --2- 172.21.15.32:0/947957690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31940b44d0 0x7f31940ae8b0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:13.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.385+0000 7f319b589640 1 --2- 172.21.15.32:0/947957690 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31940af890 0x7f31940b0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:13.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.385+0000 7f319b589640 1 --2- 172.21.15.32:0/947957690 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f319406c5f0 0x7f31941520c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:13.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.385+0000 7f319b589640 1 -- 172.21.15.32:0/947957690 >> 172.21.15.32:0/947957690 conn(0x7f319406b110 msgr2=0x7f31940728b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:13.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.385+0000 7f319b589640 1 -- 172.21.15.32:0/947957690 shutdown_connections 2024-04-24T23:05:13.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.385+0000 7f319b589640 1 -- 172.21.15.32:0/947957690 wait complete. 2024-04-24T23:05:13.397 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:05:13.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.499+0000 7f2747d61640 1 Processor -- start 2024-04-24T23:05:13.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.500+0000 7f2747d61640 1 -- start start 2024-04-24T23:05:13.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.500+0000 7f2747d61640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f27400b5560 0x7f274015e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:13.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.500+0000 7f2747d61640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f27400cd3c0 0x7f274015ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:13.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.500+0000 7f2747d61640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27400cd910 con 0x7f27400b5940 2024-04-24T23:05:13.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.500+0000 7f2747d61640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f27400cdbf0 con 0x7f27400cd3c0 2024-04-24T23:05:13.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.500+0000 7f2747d61640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f27400cded0 con 0x7f27400b5560 2024-04-24T23:05:13.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.501+0000 7f27452fc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f27400cd3c0 0x7f274015ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:13.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.501+0000 7f27452fc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f27400cd3c0 0x7f274015ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55794/0 (socket says 172.21.15.32:55794) 2024-04-24T23:05:13.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.501+0000 7f27452fc640 1 -- 172.21.15.32:0/2130192391 learned_addr learned my addr 172.21.15.32:0/2130192391 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:13.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.501+0000 7f2745afd640 1 --2- 172.21.15.32:0/2130192391 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f27400b5560 0x7f274015e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:13.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.501+0000 7f27452fc640 1 -- 172.21.15.32:0/2130192391 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f27400b5560 msgr2=0x7f274015e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:13.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.501+0000 7f27452fc640 1 --2- 172.21.15.32:0/2130192391 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f27400b5560 0x7f274015e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:13.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.501+0000 7f27452fc640 1 -- 172.21.15.32:0/2130192391 >> v1:172.21.15.121:6789/0 conn(0x7f27400b5940 legacy=0x7f27400b5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:13.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.501+0000 7f27452fc640 1 -- 172.21.15.32:0/2130192391 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27400ce1b0 con 0x7f27400cd3c0 2024-04-24T23:05:13.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.502+0000 7f27452fc640 1 --2- 172.21.15.32:0/2130192391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27400cd3c0 0x7f274015ec00 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f2730009870 tx=0x7f273002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4926e93c0e9b77a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:13.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.502+0000 7f2744afb640 1 -- 172.21.15.32:0/2130192391 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2730037070 con 0x7f27400cd3c0 2024-04-24T23:05:13.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.502+0000 7f2744afb640 1 -- 172.21.15.32:0/2130192391 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2730032440 con 0x7f27400cd3c0 2024-04-24T23:05:13.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.502+0000 7f2744afb640 1 -- 172.21.15.32:0/2130192391 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2730032810 con 0x7f27400cd3c0 2024-04-24T23:05:13.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.502+0000 7f2747d61640 1 -- 172.21.15.32:0/2130192391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27400cd3c0 msgr2=0x7f274015ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:13.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.502+0000 7f2747d61640 1 --2- 172.21.15.32:0/2130192391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27400cd3c0 0x7f274015ec00 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f2730009870 tx=0x7f273002a210 comp rx=0 tx=0).stop 2024-04-24T23:05:13.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.502+0000 7f2747d61640 1 -- 172.21.15.32:0/2130192391 shutdown_connections 2024-04-24T23:05:13.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.502+0000 7f2747d61640 1 --2- 172.21.15.32:0/2130192391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27400cd3c0 0x7f274015ec00 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:13.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.502+0000 7f2747d61640 1 --2- 172.21.15.32:0/2130192391 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f27400b5560 0x7f274015e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:13.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.502+0000 7f2747d61640 1 -- 172.21.15.32:0/2130192391 >> 172.21.15.32:0/2130192391 conn(0x7f274006b110 msgr2=0x7f274006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:13.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.502+0000 7f2747d61640 1 -- 172.21.15.32:0/2130192391 shutdown_connections 2024-04-24T23:05:13.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.502+0000 7f2747d61640 1 -- 172.21.15.32:0/2130192391 wait complete. 2024-04-24T23:05:13.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.504+0000 7f2747d61640 1 Processor -- start 2024-04-24T23:05:13.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.504+0000 7f2747d61640 1 -- start start 2024-04-24T23:05:13.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.504+0000 7f2747d61640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f27400b5560 0x7f274013fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:13.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.504+0000 7f2747d61640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27400b5940 0x7f274014fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:13.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.504+0000 7f2747d61640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27400cd3c0 0x7f2740070940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:13.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.504+0000 7f2747d61640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f274006c770 con 0x7f27400b5940 2024-04-24T23:05:13.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.504+0000 7f2747d61640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f274006c5f0 con 0x7f27400cd3c0 2024-04-24T23:05:13.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.504+0000 7f2747d61640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f274006c8f0 con 0x7f27400b5560 2024-04-24T23:05:13.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.504+0000 7f27452fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27400b5940 0x7f274014fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:13.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.505+0000 7f27452fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27400b5940 0x7f274014fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55806/0 (socket says 172.21.15.32:55806) 2024-04-24T23:05:13.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.505+0000 7f27452fc640 1 -- 172.21.15.32:0/1826976195 learned_addr learned my addr 172.21.15.32:0/1826976195 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:13.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.505+0000 7f2745afd640 1 --2- 172.21.15.32:0/1826976195 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f27400b5560 0x7f274013fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:13.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.505+0000 7f27462fe640 1 --2- 172.21.15.32:0/1826976195 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27400cd3c0 0x7f2740070940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:13.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.505+0000 7f27452fc640 1 -- 172.21.15.32:0/1826976195 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f27400b5560 msgr2=0x7f274013fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:13.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.505+0000 7f27452fc640 1 --2- 172.21.15.32:0/1826976195 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f27400b5560 0x7f274013fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:13.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.505+0000 7f27452fc640 1 -- 172.21.15.32:0/1826976195 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27400cd3c0 msgr2=0x7f2740070940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:13.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.505+0000 7f27452fc640 1 --2- 172.21.15.32:0/1826976195 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27400cd3c0 0x7f2740070940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:13.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.505+0000 7f27452fc640 1 -- 172.21.15.32:0/1826976195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27401428f0 con 0x7f27400b5940 2024-04-24T23:05:13.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.505+0000 7f27452fc640 1 --2- 172.21.15.32:0/1826976195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27400b5940 0x7f274014fd60 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f2730000c00 tx=0x7f27300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:13.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.505+0000 7f27367fc640 1 -- 172.21.15.32:0/1826976195 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2730041020 con 0x7f27400b5940 2024-04-24T23:05:13.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.505+0000 7f27367fc640 1 -- 172.21.15.32:0/1826976195 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27300306e0 con 0x7f27400b5940 2024-04-24T23:05:13.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.505+0000 7f2747d61640 1 -- 172.21.15.32:0/1826976195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27401502a0 con 0x7f27400b5940 2024-04-24T23:05:13.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.505+0000 7f27367fc640 1 -- 172.21.15.32:0/1826976195 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2730037040 con 0x7f27400b5940 2024-04-24T23:05:13.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.505+0000 7f2747d61640 1 -- 172.21.15.32:0/1826976195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2740142610 con 0x7f27400b5940 2024-04-24T23:05:13.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.506+0000 7f2747d61640 1 -- 172.21.15.32:0/1826976195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27400691d0 con 0x7f27400b5940 2024-04-24T23:05:13.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.507+0000 7f27367fc640 1 -- 172.21.15.32:0/1826976195 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2730044020 con 0x7f27400b5940 2024-04-24T23:05:13.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.507+0000 7f27367fc640 1 --2- 172.21.15.32:0/1826976195 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f272406be60 0x7f272406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:13.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.507+0000 7f27367fc640 1 -- 172.21.15.32:0/1826976195 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5709+0+0 (secure 0 0 0) 0x7f27300ab5d0 con 0x7f27400b5940 2024-04-24T23:05:13.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.507+0000 7f2745afd640 1 --2- 172.21.15.32:0/1826976195 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f272406be60 0x7f272406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:13.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.508+0000 7f2745afd640 1 --2- 172.21.15.32:0/1826976195 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f272406be60 0x7f272406e320 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f273c006fd0 tx=0x7f273c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:13.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.511+0000 7f27367fc640 1 -- 172.21.15.32:0/1826976195 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f27400691d0 con 0x7f27400b5940 2024-04-24T23:05:13.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.694+0000 7f2747d61640 1 -- 172.21.15.32:0/1826976195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2740068430 con 0x7f27400b5940 2024-04-24T23:05:13.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.695+0000 7f27367fc640 1 -- 172.21.15.32:0/1826976195 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v128) ==== 104+0+11 (secure 0 0 0) 0x7f273004b030 con 0x7f27400b5940 2024-04-24T23:05:13.696 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-24T23:05:13.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.697+0000 7f2747d61640 1 -- 172.21.15.32:0/1826976195 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f272406be60 msgr2=0x7f272406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:13.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.697+0000 7f2747d61640 1 --2- 172.21.15.32:0/1826976195 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f272406be60 0x7f272406e320 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f273c006fd0 tx=0x7f273c008040 comp rx=0 tx=0).stop 2024-04-24T23:05:13.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.697+0000 7f2747d61640 1 -- 172.21.15.32:0/1826976195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27400b5940 msgr2=0x7f274014fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:13.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.697+0000 7f2747d61640 1 --2- 172.21.15.32:0/1826976195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27400b5940 0x7f274014fd60 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f2730000c00 tx=0x7f27300046c0 comp rx=0 tx=0).stop 2024-04-24T23:05:13.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.697+0000 7f2747d61640 1 -- 172.21.15.32:0/1826976195 shutdown_connections 2024-04-24T23:05:13.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.697+0000 7f2747d61640 1 --2- 172.21.15.32:0/1826976195 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f272406be60 0x7f272406e320 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:13.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.697+0000 7f2747d61640 1 --2- 172.21.15.32:0/1826976195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27400b5940 0x7f274014fd60 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:13.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.697+0000 7f2747d61640 1 --2- 172.21.15.32:0/1826976195 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27400cd3c0 0x7f2740070940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:13.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.697+0000 7f2747d61640 1 --2- 172.21.15.32:0/1826976195 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f27400b5560 0x7f274013fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:13.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.697+0000 7f2747d61640 1 -- 172.21.15.32:0/1826976195 >> 172.21.15.32:0/1826976195 conn(0x7f274006b110 msgr2=0x7f2740154910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:13.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.697+0000 7f2747d61640 1 -- 172.21.15.32:0/1826976195 shutdown_connections 2024-04-24T23:05:13.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.697+0000 7f2747d61640 1 -- 172.21.15.32:0/1826976195 wait complete. 2024-04-24T23:05:13.709 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:05:13.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.811+0000 7f8d23389640 1 Processor -- start 2024-04-24T23:05:13.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.812+0000 7f8d23389640 1 -- start start 2024-04-24T23:05:13.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.812+0000 7f8d23389640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8d1c15e9b0 0x7f8d1c0d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:13.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.812+0000 7f8d23389640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d1c0d4210 0x7f8d1c06b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:13.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.812+0000 7f8d23389640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d1c0d3170 con 0x7f8d1c0ba0e0 2024-04-24T23:05:13.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.812+0000 7f8d23389640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f8d1c0b8900 con 0x7f8d1c15e9b0 2024-04-24T23:05:13.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.812+0000 7f8d23389640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f8d1c0d3450 con 0x7f8d1c0d4210 2024-04-24T23:05:13.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.813+0000 7f8d21125640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8d1c15e9b0 0x7f8d1c0d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:13.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.813+0000 7f8d21926640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8d1c0ba0e0 0x7f8d1c071730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45192/0 (socket says 172.21.15.32:45192) 2024-04-24T23:05:13.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.813+0000 7f8d21926640 1 -- 172.21.15.32:0/1277692439 learned_addr learned my addr 172.21.15.32:0/1277692439 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:05:13.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.813+0000 7f8d20924640 1 --2- 172.21.15.32:0/1277692439 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d1c0d4210 0x7f8d1c06b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:13.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.813+0000 7f8d21125640 1 -- 172.21.15.32:0/1277692439 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d1c0d4210 msgr2=0x7f8d1c06b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:13.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.813+0000 7f8d21125640 1 --2- 172.21.15.32:0/1277692439 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d1c0d4210 0x7f8d1c06b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:13.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.813+0000 7f8d21125640 1 -- 172.21.15.32:0/1277692439 >> v1:172.21.15.32:6789/0 conn(0x7f8d1c0ba0e0 legacy=0x7f8d1c071730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:13.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.813+0000 7f8d21125640 1 -- 172.21.15.32:0/1277692439 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d1c0d3730 con 0x7f8d1c15e9b0 2024-04-24T23:05:13.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.814+0000 7f8d21125640 1 --2- 172.21.15.32:0/1277692439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d1c15e9b0 0x7f8d1c0d2b90 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f8d140097a0 tx=0x7f8d1402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8450d6c39d467247 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:13.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.814+0000 7f8d1bfff640 1 -- 172.21.15.32:0/1277692439 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d14037070 con 0x7f8d1c15e9b0 2024-04-24T23:05:13.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.814+0000 7f8d1bfff640 1 -- 172.21.15.32:0/1277692439 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d14032440 con 0x7f8d1c15e9b0 2024-04-24T23:05:13.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.814+0000 7f8d1bfff640 1 -- 172.21.15.32:0/1277692439 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d14032810 con 0x7f8d1c15e9b0 2024-04-24T23:05:13.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.814+0000 7f8d23389640 1 -- 172.21.15.32:0/1277692439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d1c15e9b0 msgr2=0x7f8d1c0d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:13.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.814+0000 7f8d23389640 1 --2- 172.21.15.32:0/1277692439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d1c15e9b0 0x7f8d1c0d2b90 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f8d140097a0 tx=0x7f8d1402a180 comp rx=0 tx=0).stop 2024-04-24T23:05:13.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.814+0000 7f8d23389640 1 -- 172.21.15.32:0/1277692439 shutdown_connections 2024-04-24T23:05:13.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.814+0000 7f8d23389640 1 --2- 172.21.15.32:0/1277692439 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d1c0d4210 0x7f8d1c06b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:13.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.814+0000 7f8d23389640 1 --2- 172.21.15.32:0/1277692439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d1c15e9b0 0x7f8d1c0d2b90 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:13.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.814+0000 7f8d23389640 1 -- 172.21.15.32:0/1277692439 >> 172.21.15.32:0/1277692439 conn(0x7f8d1c0d11d0 msgr2=0x7f8d1c0d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:13.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.814+0000 7f8d23389640 1 -- 172.21.15.32:0/1277692439 shutdown_connections 2024-04-24T23:05:13.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.814+0000 7f8d23389640 1 -- 172.21.15.32:0/1277692439 wait complete. 2024-04-24T23:05:13.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.816+0000 7f8d23389640 1 Processor -- start 2024-04-24T23:05:13.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.816+0000 7f8d23389640 1 -- start start 2024-04-24T23:05:13.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.816+0000 7f8d23389640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d1c0ba0e0 0x7f8d1c1613a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:13.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.816+0000 7f8d23389640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d1c0d4210 0x7f8d1c1507a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:13.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.816+0000 7f8d23389640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d1c15e9b0 0x7f8d1c150ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:13.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.816+0000 7f8d23389640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1c0d2400 con 0x7f8d1c15e9b0 2024-04-24T23:05:13.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.816+0000 7f8d23389640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1c0d3cf0 con 0x7f8d1c0ba0e0 2024-04-24T23:05:13.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.816+0000 7f8d23389640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f8d1c06b650 con 0x7f8d1c0d4210 2024-04-24T23:05:13.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.816+0000 7f8d21926640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d1c15e9b0 0x7f8d1c150ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:13.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.816+0000 7f8d21926640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d1c15e9b0 0x7f8d1c150ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55826/0 (socket says 172.21.15.32:55826) 2024-04-24T23:05:13.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.816+0000 7f8d21926640 1 -- 172.21.15.32:0/3752761147 learned_addr learned my addr 172.21.15.32:0/3752761147 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:13.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.817+0000 7f8d20924640 1 --2- 172.21.15.32:0/3752761147 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d1c0d4210 0x7f8d1c1507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:13.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.817+0000 7f8d21125640 1 --2- 172.21.15.32:0/3752761147 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d1c0ba0e0 0x7f8d1c1613a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:13.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.817+0000 7f8d21926640 1 -- 172.21.15.32:0/3752761147 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d1c0d4210 msgr2=0x7f8d1c1507a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:13.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.817+0000 7f8d21926640 1 --2- 172.21.15.32:0/3752761147 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d1c0d4210 0x7f8d1c1507a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:13.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.817+0000 7f8d21926640 1 -- 172.21.15.32:0/3752761147 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d1c0ba0e0 msgr2=0x7f8d1c1613a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:13.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.817+0000 7f8d21926640 1 --2- 172.21.15.32:0/3752761147 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d1c0ba0e0 0x7f8d1c1613a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:13.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.817+0000 7f8d21926640 1 -- 172.21.15.32:0/3752761147 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d1c0cb8c0 con 0x7f8d1c15e9b0 2024-04-24T23:05:13.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.817+0000 7f8d21125640 1 --2- 172.21.15.32:0/3752761147 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d1c0ba0e0 0x7f8d1c1613a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:05:13.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.817+0000 7f8d21926640 1 --2- 172.21.15.32:0/3752761147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d1c15e9b0 0x7f8d1c150ce0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c00cbc0 tx=0x7f8d0c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:13.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.817+0000 7f8d19ffb640 1 -- 172.21.15.32:0/3752761147 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c017070 con 0x7f8d1c15e9b0 2024-04-24T23:05:13.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.817+0000 7f8d19ffb640 1 -- 172.21.15.32:0/3752761147 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d0c00ad40 con 0x7f8d1c15e9b0 2024-04-24T23:05:13.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.817+0000 7f8d19ffb640 1 -- 172.21.15.32:0/3752761147 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c012600 con 0x7f8d1c15e9b0 2024-04-24T23:05:13.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.817+0000 7f8d23389640 1 -- 172.21.15.32:0/3752761147 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d1c0cbba0 con 0x7f8d1c15e9b0 2024-04-24T23:05:13.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.817+0000 7f8d23389640 1 -- 172.21.15.32:0/3752761147 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d1c0cc160 con 0x7f8d1c15e9b0 2024-04-24T23:05:13.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.818+0000 7f8d23389640 1 -- 172.21.15.32:0/3752761147 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d1c0cc650 con 0x7f8d1c15e9b0 2024-04-24T23:05:13.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.818+0000 7f8d19ffb640 1 -- 172.21.15.32:0/3752761147 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8d0c01a070 con 0x7f8d1c15e9b0 2024-04-24T23:05:13.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.819+0000 7f8d19ffb640 1 --2- 172.21.15.32:0/3752761147 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8d0806be30 0x7f8d0806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:13.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.819+0000 7f8d19ffb640 1 -- 172.21.15.32:0/3752761147 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5709+0+0 (secure 0 0 0) 0x7f8d0c08aa80 con 0x7f8d1c15e9b0 2024-04-24T23:05:13.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.819+0000 7f8d21125640 1 --2- 172.21.15.32:0/3752761147 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8d0806be30 0x7f8d0806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:13.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.820+0000 7f8d21125640 1 --2- 172.21.15.32:0/3752761147 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8d0806be30 0x7f8d0806e2f0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8d14034000 tx=0x7f8d1402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:13.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:13.823+0000 7f8d19ffb640 1 -- 172.21.15.32:0/3752761147 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8d0c058e70 con 0x7f8d1c15e9b0 2024-04-24T23:05:14.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.006+0000 7f8d23389640 1 -- 172.21.15.32:0/3752761147 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8d1c068430 con 0x7f8d1c15e9b0 2024-04-24T23:05:14.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.007+0000 7f8d19ffb640 1 -- 172.21.15.32:0/3752761147 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v128) ==== 100+0+11 (secure 0 0 0) 0x7f8d0c05d170 con 0x7f8d1c15e9b0 2024-04-24T23:05:14.008 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-24T23:05:14.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.009+0000 7f8d23389640 1 -- 172.21.15.32:0/3752761147 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8d0806be30 msgr2=0x7f8d0806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:14.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.009+0000 7f8d23389640 1 --2- 172.21.15.32:0/3752761147 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8d0806be30 0x7f8d0806e2f0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8d14034000 tx=0x7f8d1402e000 comp rx=0 tx=0).stop 2024-04-24T23:05:14.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.009+0000 7f8d23389640 1 -- 172.21.15.32:0/3752761147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d1c15e9b0 msgr2=0x7f8d1c150ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:14.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.009+0000 7f8d23389640 1 --2- 172.21.15.32:0/3752761147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d1c15e9b0 0x7f8d1c150ce0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c00cbc0 tx=0x7f8d0c00a430 comp rx=0 tx=0).stop 2024-04-24T23:05:14.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.009+0000 7f8d23389640 1 -- 172.21.15.32:0/3752761147 shutdown_connections 2024-04-24T23:05:14.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.009+0000 7f8d23389640 1 --2- 172.21.15.32:0/3752761147 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8d0806be30 0x7f8d0806e2f0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:14.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.009+0000 7f8d23389640 1 --2- 172.21.15.32:0/3752761147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d1c15e9b0 0x7f8d1c150ce0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:14.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.009+0000 7f8d23389640 1 --2- 172.21.15.32:0/3752761147 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d1c0d4210 0x7f8d1c1507a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:14.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.009+0000 7f8d23389640 1 --2- 172.21.15.32:0/3752761147 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d1c0ba0e0 0x7f8d1c1613a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:14.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.009+0000 7f8d23389640 1 -- 172.21.15.32:0/3752761147 >> 172.21.15.32:0/3752761147 conn(0x7f8d1c0d11d0 msgr2=0x7f8d1c15be80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:14.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.009+0000 7f8d23389640 1 -- 172.21.15.32:0/3752761147 shutdown_connections 2024-04-24T23:05:14.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.009+0000 7f8d23389640 1 -- 172.21.15.32:0/3752761147 wait complete. 2024-04-24T23:05:14.021 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:05:14.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.120+0000 7f2e6cf22640 1 Processor -- start 2024-04-24T23:05:14.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.120+0000 7f2e6cf22640 1 -- start start 2024-04-24T23:05:14.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.121+0000 7f2e6cf22640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2e6806b820 0x7f2e680d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:14.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.121+0000 7f2e6cf22640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e680d3170 con 0x7f2e680d4210 2024-04-24T23:05:14.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.121+0000 7f2e6cf22640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e680d3450 con 0x7f2e6815e9b0 2024-04-24T23:05:14.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.121+0000 7f2e6cf22640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f2e680d3730 con 0x7f2e6806b820 2024-04-24T23:05:14.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.121+0000 7f2e65d9b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2e6806b820 0x7f2e680d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:14.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.122+0000 7f2e65d9b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2e6806b820 0x7f2e680d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55834/0 (socket says 172.21.15.32:55834) 2024-04-24T23:05:14.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.122+0000 7f2e65d9b640 1 -- 172.21.15.32:0/1685399031 learned_addr learned my addr 172.21.15.32:0/1685399031 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:14.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.122+0000 7f2e66d9d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2e680d4210 0x7f2e680b78f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45200/0 (socket says 172.21.15.32:45200) 2024-04-24T23:05:14.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.122+0000 7f2e65d9b640 1 -- 172.21.15.32:0/1685399031 >> v1:172.21.15.121:6789/0 conn(0x7f2e6815e9b0 legacy=0x7f2e6806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:14.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.122+0000 7f2e65d9b640 1 -- 172.21.15.32:0/1685399031 >> v1:172.21.15.32:6789/0 conn(0x7f2e680d4210 legacy=0x7f2e680b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:14.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.122+0000 7f2e65d9b640 1 -- 172.21.15.32:0/1685399031 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e680d3a10 con 0x7f2e6806b820 2024-04-24T23:05:14.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.122+0000 7f2e6559a640 1 -- 172.21.15.32:0/1685399031 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3832439671 0 0) 0x7f2e680d3170 con 0x7f2e680d4210 2024-04-24T23:05:14.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.123+0000 7f2e65d9b640 1 --2- 172.21.15.32:0/1685399031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e6806b820 0x7f2e680d2b90 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c009870 tx=0x7f2e5c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6206e7a65e8a94c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:14.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.123+0000 7f2e6559a640 1 -- 172.21.15.32:0/1685399031 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c037070 con 0x7f2e6806b820 2024-04-24T23:05:14.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.123+0000 7f2e6559a640 1 -- 172.21.15.32:0/1685399031 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e5c032440 con 0x7f2e6806b820 2024-04-24T23:05:14.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.123+0000 7f2e6559a640 1 -- 172.21.15.32:0/1685399031 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c032810 con 0x7f2e6806b820 2024-04-24T23:05:14.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.123+0000 7f2e6cf22640 1 -- 172.21.15.32:0/1685399031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e6806b820 msgr2=0x7f2e680d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:14.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.123+0000 7f2e6cf22640 1 --2- 172.21.15.32:0/1685399031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e6806b820 0x7f2e680d2b90 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c009870 tx=0x7f2e5c02a270 comp rx=0 tx=0).stop 2024-04-24T23:05:14.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.123+0000 7f2e6cf22640 1 -- 172.21.15.32:0/1685399031 shutdown_connections 2024-04-24T23:05:14.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.123+0000 7f2e6cf22640 1 --2- 172.21.15.32:0/1685399031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e6806b820 0x7f2e680d2b90 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:14.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.123+0000 7f2e6cf22640 1 -- 172.21.15.32:0/1685399031 >> 172.21.15.32:0/1685399031 conn(0x7f2e680d11d0 msgr2=0x7f2e680d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:14.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.123+0000 7f2e6cf22640 1 -- 172.21.15.32:0/1685399031 shutdown_connections 2024-04-24T23:05:14.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.123+0000 7f2e6cf22640 1 -- 172.21.15.32:0/1685399031 wait complete. 2024-04-24T23:05:14.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.125+0000 7f2e6cf22640 1 Processor -- start 2024-04-24T23:05:14.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.125+0000 7f2e6cf22640 1 -- start start 2024-04-24T23:05:14.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.125+0000 7f2e6cf22640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e6806b820 0x7f2e6815a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:14.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.125+0000 7f2e6cf22640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e680d4210 0x7f2e680ca620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:14.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.125+0000 7f2e6cf22640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e6815e9b0 0x7f2e680ccd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:14.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.125+0000 7f2e6cf22640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2e680ba0e0 con 0x7f2e6806b820 2024-04-24T23:05:14.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.125+0000 7f2e6cf22640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2e680d2400 con 0x7f2e680d4210 2024-04-24T23:05:14.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.125+0000 7f2e6cf22640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f2e680ba260 con 0x7f2e6815e9b0 2024-04-24T23:05:14.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.126+0000 7f2e6659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e6806b820 0x7f2e6815a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:14.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.126+0000 7f2e6659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e6806b820 0x7f2e6815a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55844/0 (socket says 172.21.15.32:55844) 2024-04-24T23:05:14.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.126+0000 7f2e6659c640 1 -- 172.21.15.32:0/1147386027 learned_addr learned my addr 172.21.15.32:0/1147386027 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:14.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.126+0000 7f2e65d9b640 1 --2- 172.21.15.32:0/1147386027 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e680d4210 0x7f2e680ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:14.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.126+0000 7f2e6659c640 1 -- 172.21.15.32:0/1147386027 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e6815e9b0 msgr2=0x7f2e680ccd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:14.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.126+0000 7f2e66d9d640 1 --2- 172.21.15.32:0/1147386027 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e6815e9b0 0x7f2e680ccd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:14.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.126+0000 7f2e6659c640 1 --2- 172.21.15.32:0/1147386027 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e6815e9b0 0x7f2e680ccd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:14.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.126+0000 7f2e6659c640 1 -- 172.21.15.32:0/1147386027 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e680d4210 msgr2=0x7f2e680ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:14.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.126+0000 7f2e6659c640 1 --2- 172.21.15.32:0/1147386027 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e680d4210 0x7f2e680ca620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:14.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.126+0000 7f2e6659c640 1 -- 172.21.15.32:0/1147386027 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e680c8320 con 0x7f2e6806b820 2024-04-24T23:05:14.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.126+0000 7f2e65d9b640 1 --2- 172.21.15.32:0/1147386027 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e680d4210 0x7f2e680ca620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:05:14.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.126+0000 7f2e66d9d640 1 --2- 172.21.15.32:0/1147386027 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e6815e9b0 0x7f2e680ccd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:05:14.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.126+0000 7f2e6659c640 1 --2- 172.21.15.32:0/1147386027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e6806b820 0x7f2e6815a5d0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f2e6000c400 tx=0x7f2e6000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:14.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.126+0000 7f2e56ffd640 1 -- 172.21.15.32:0/1147386027 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e60017070 con 0x7f2e6806b820 2024-04-24T23:05:14.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.126+0000 7f2e56ffd640 1 -- 172.21.15.32:0/1147386027 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e600052d0 con 0x7f2e6806b820 2024-04-24T23:05:14.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.126+0000 7f2e6cf22640 1 -- 172.21.15.32:0/1147386027 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e680cd2a0 con 0x7f2e6806b820 2024-04-24T23:05:14.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.126+0000 7f2e56ffd640 1 -- 172.21.15.32:0/1147386027 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e60003830 con 0x7f2e6806b820 2024-04-24T23:05:14.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.126+0000 7f2e6cf22640 1 -- 172.21.15.32:0/1147386027 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e680c85c0 con 0x7f2e6806b820 2024-04-24T23:05:14.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.127+0000 7f2e6cf22640 1 -- 172.21.15.32:0/1147386027 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e68068430 con 0x7f2e6806b820 2024-04-24T23:05:14.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.128+0000 7f2e56ffd640 1 -- 172.21.15.32:0/1147386027 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2e6001a070 con 0x7f2e6806b820 2024-04-24T23:05:14.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.128+0000 7f2e56ffd640 1 --2- 172.21.15.32:0/1147386027 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2e4806be30 0x7f2e4806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:14.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.128+0000 7f2e56ffd640 1 -- 172.21.15.32:0/1147386027 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5709+0+0 (secure 0 0 0) 0x7f2e6008a1c0 con 0x7f2e6806b820 2024-04-24T23:05:14.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.128+0000 7f2e65d9b640 1 --2- 172.21.15.32:0/1147386027 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2e4806be30 0x7f2e4806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:14.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.129+0000 7f2e65d9b640 1 --2- 172.21.15.32:0/1147386027 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2e4806be30 0x7f2e4806e2f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c034000 tx=0x7f2e5c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:14.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.132+0000 7f2e56ffd640 1 -- 172.21.15.32:0/1147386027 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2e60058b70 con 0x7f2e6806b820 2024-04-24T23:05:14.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.353+0000 7f2e6cf22640 1 -- 172.21.15.32:0/1147386027 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2e68074df0 con 0x7f2e6806b820 2024-04-24T23:05:14.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.353+0000 7f2e56ffd640 1 -- 172.21.15.32:0/1147386027 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+5189 (secure 0 0 0) 0x7f2e60058470 con 0x7f2e6806b820 2024-04-24T23:05:14.355 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:05:14.355 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":51,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5972,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3336404028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3336404028},{"type":"v1","addr":"172.21.15.121:6837","nonce":3336404028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6505,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/1068557774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1068557774},{"type":"v1","addr":"172.21.15.121:6836","nonce":1068557774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":7353,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/2118719195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":2118719195},{"type":"v1","addr":"172.21.15.32:6834","nonce":2118719195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:04:55.998022+0000","modified":"2024-04-24T23:04:58.657258+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7356},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7356":{"gid":7356,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/3015290487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":3015290487},{"type":"v1","addr":"172.21.15.32:6835","nonce":3015290487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7356,"qdb_cluster":[7356]},"id":7}]} 2024-04-24T23:05:14.355 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 51 2024-04-24T23:05:14.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.355+0000 7f2e6cf22640 1 -- 172.21.15.32:0/1147386027 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2e4806be30 msgr2=0x7f2e4806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:14.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.355+0000 7f2e6cf22640 1 --2- 172.21.15.32:0/1147386027 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2e4806be30 0x7f2e4806e2f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c034000 tx=0x7f2e5c02e000 comp rx=0 tx=0).stop 2024-04-24T23:05:14.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.355+0000 7f2e6cf22640 1 -- 172.21.15.32:0/1147386027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e6806b820 msgr2=0x7f2e6815a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:14.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.355+0000 7f2e6cf22640 1 --2- 172.21.15.32:0/1147386027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e6806b820 0x7f2e6815a5d0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f2e6000c400 tx=0x7f2e6000c8c0 comp rx=0 tx=0).stop 2024-04-24T23:05:14.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.355+0000 7f2e6cf22640 1 -- 172.21.15.32:0/1147386027 shutdown_connections 2024-04-24T23:05:14.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.355+0000 7f2e6cf22640 1 --2- 172.21.15.32:0/1147386027 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2e4806be30 0x7f2e4806e2f0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:14.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.355+0000 7f2e6cf22640 1 --2- 172.21.15.32:0/1147386027 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e6815e9b0 0x7f2e680ccd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:14.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.355+0000 7f2e6cf22640 1 --2- 172.21.15.32:0/1147386027 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e680d4210 0x7f2e680ca620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:14.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.355+0000 7f2e6cf22640 1 --2- 172.21.15.32:0/1147386027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e6806b820 0x7f2e6815a5d0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:14.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.355+0000 7f2e6cf22640 1 -- 172.21.15.32:0/1147386027 >> 172.21.15.32:0/1147386027 conn(0x7f2e680d11d0 msgr2=0x7f2e6806b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:14.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.355+0000 7f2e6cf22640 1 -- 172.21.15.32:0/1147386027 shutdown_connections 2024-04-24T23:05:14.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.355+0000 7f2e6cf22640 1 -- 172.21.15.32:0/1147386027 wait complete. 2024-04-24T23:05:14.367 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:05:14.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.466+0000 7f3f033a5640 1 Processor -- start 2024-04-24T23:05:14.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.466+0000 7f3f033a5640 1 -- start start 2024-04-24T23:05:14.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.467+0000 7f3f033a5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3efc15e9b0 0x7f3efc0d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:14.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.467+0000 7f3f033a5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3efc0d4210 0x7f3efc06b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:14.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.467+0000 7f3f033a5640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3efc0ba0e0 0x7f3efc06b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:14.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.467+0000 7f3f033a5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f3efc0d3170 con 0x7f3efc15e9b0 2024-04-24T23:05:14.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.467+0000 7f3f033a5640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f3efc0d3450 con 0x7f3efc0ba0e0 2024-04-24T23:05:14.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.467+0000 7f3f033a5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f3efc0d3730 con 0x7f3efc0d4210 2024-04-24T23:05:14.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.467+0000 7f3f01141640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3efc15e9b0 0x7f3efc0d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:14.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.467+0000 7f3f01141640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3efc15e9b0 0x7f3efc0d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55852/0 (socket says 172.21.15.32:55852) 2024-04-24T23:05:14.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.467+0000 7f3f01141640 1 -- 172.21.15.32:0/492796755 learned_addr learned my addr 172.21.15.32:0/492796755 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:14.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.468+0000 7f3f01141640 1 -- 172.21.15.32:0/492796755 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3efc0d4210 msgr2=0x7f3efc06b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:14.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.468+0000 7f3f01141640 1 --2- 172.21.15.32:0/492796755 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3efc0d4210 0x7f3efc06b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:14.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.468+0000 7f3f01141640 1 -- 172.21.15.32:0/492796755 >> v2:172.21.15.121:3300/0 conn(0x7f3efc0ba0e0 msgr2=0x7f3efc06b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:14.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.468+0000 7f3f01141640 1 --2- 172.21.15.32:0/492796755 >> v2:172.21.15.121:3300/0 conn(0x7f3efc0ba0e0 0x7f3efc06b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:14.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.468+0000 7f3f01141640 1 -- 172.21.15.32:0/492796755 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3efc0d3a10 con 0x7f3efc15e9b0 2024-04-24T23:05:14.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.468+0000 7f3f01141640 1 --2- 172.21.15.32:0/492796755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3efc15e9b0 0x7f3efc0d2b90 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f3efc0b61d0 tx=0x7f3ef402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ed120756c10ea52 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:14.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.468+0000 7f3efbfff640 1 -- 172.21.15.32:0/492796755 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef4037070 con 0x7f3efc15e9b0 2024-04-24T23:05:14.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.469+0000 7f3efbfff640 1 -- 172.21.15.32:0/492796755 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ef4032440 con 0x7f3efc15e9b0 2024-04-24T23:05:14.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.469+0000 7f3efbfff640 1 -- 172.21.15.32:0/492796755 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef4032810 con 0x7f3efc15e9b0 2024-04-24T23:05:14.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.469+0000 7f3f033a5640 1 -- 172.21.15.32:0/492796755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3efc15e9b0 msgr2=0x7f3efc0d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:14.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.469+0000 7f3f033a5640 1 --2- 172.21.15.32:0/492796755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3efc15e9b0 0x7f3efc0d2b90 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f3efc0b61d0 tx=0x7f3ef402a1e0 comp rx=0 tx=0).stop 2024-04-24T23:05:14.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.469+0000 7f3f033a5640 1 -- 172.21.15.32:0/492796755 shutdown_connections 2024-04-24T23:05:14.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.469+0000 7f3f033a5640 1 --2- 172.21.15.32:0/492796755 >> v2:172.21.15.121:3300/0 conn(0x7f3efc0ba0e0 0x7f3efc06b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:14.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.469+0000 7f3f033a5640 1 --2- 172.21.15.32:0/492796755 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3efc0d4210 0x7f3efc06b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:14.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.469+0000 7f3f033a5640 1 --2- 172.21.15.32:0/492796755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3efc15e9b0 0x7f3efc0d2b90 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:14.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.469+0000 7f3f033a5640 1 -- 172.21.15.32:0/492796755 >> 172.21.15.32:0/492796755 conn(0x7f3efc0d11d0 msgr2=0x7f3efc0d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:14.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.469+0000 7f3f033a5640 1 -- 172.21.15.32:0/492796755 shutdown_connections 2024-04-24T23:05:14.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.469+0000 7f3f033a5640 1 -- 172.21.15.32:0/492796755 wait complete. 2024-04-24T23:05:14.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.470+0000 7f3f033a5640 1 Processor -- start 2024-04-24T23:05:14.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.470+0000 7f3f033a5640 1 -- start start 2024-04-24T23:05:14.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.471+0000 7f3f033a5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3efc0ba0e0 0x7f3efc145b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:14.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.471+0000 7f3f033a5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3efc0d4210 0x7f3efc1566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:14.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.471+0000 7f3f033a5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3efc15e9b0 0x7f3efc15b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:14.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.471+0000 7f3f033a5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc071730 con 0x7f3efc0ba0e0 2024-04-24T23:05:14.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.471+0000 7f3f033a5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc0d2400 con 0x7f3efc0d4210 2024-04-24T23:05:14.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.471+0000 7f3f033a5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f3efc0718b0 con 0x7f3efc15e9b0 2024-04-24T23:05:14.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.471+0000 7f3f01141640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3efc0ba0e0 0x7f3efc145b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:14.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.471+0000 7f3f01141640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3efc0ba0e0 0x7f3efc145b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55862/0 (socket says 172.21.15.32:55862) 2024-04-24T23:05:14.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.471+0000 7f3f01141640 1 -- 172.21.15.32:0/1589724131 learned_addr learned my addr 172.21.15.32:0/1589724131 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:14.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.471+0000 7f3f01942640 1 --2- 172.21.15.32:0/1589724131 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3efc15e9b0 0x7f3efc15b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:14.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.471+0000 7f3f00940640 1 --2- 172.21.15.32:0/1589724131 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3efc0d4210 0x7f3efc1566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:14.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.471+0000 7f3f01141640 1 -- 172.21.15.32:0/1589724131 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3efc15e9b0 msgr2=0x7f3efc15b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:14.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.471+0000 7f3f01141640 1 --2- 172.21.15.32:0/1589724131 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3efc15e9b0 0x7f3efc15b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:14.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.471+0000 7f3f01141640 1 -- 172.21.15.32:0/1589724131 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3efc0d4210 msgr2=0x7f3efc1566f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:14.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.471+0000 7f3f01141640 1 --2- 172.21.15.32:0/1589724131 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3efc0d4210 0x7f3efc1566f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:14.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.472+0000 7f3f01141640 1 -- 172.21.15.32:0/1589724131 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3efc1461b0 con 0x7f3efc0ba0e0 2024-04-24T23:05:14.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.472+0000 7f3f00940640 1 --2- 172.21.15.32:0/1589724131 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3efc0d4210 0x7f3efc1566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:05:14.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.472+0000 7f3f01141640 1 --2- 172.21.15.32:0/1589724131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3efc0ba0e0 0x7f3efc145b20 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4000c00 tx=0x7f3ef40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:14.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.472+0000 7f3ef9ffb640 1 -- 172.21.15.32:0/1589724131 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef4041070 con 0x7f3efc0ba0e0 2024-04-24T23:05:14.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.472+0000 7f3ef9ffb640 1 -- 172.21.15.32:0/1589724131 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ef403c870 con 0x7f3efc0ba0e0 2024-04-24T23:05:14.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.472+0000 7f3f033a5640 1 -- 172.21.15.32:0/1589724131 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3efc146490 con 0x7f3efc0ba0e0 2024-04-24T23:05:14.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.472+0000 7f3ef9ffb640 1 -- 172.21.15.32:0/1589724131 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef4037040 con 0x7f3efc0ba0e0 2024-04-24T23:05:14.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.472+0000 7f3f033a5640 1 -- 172.21.15.32:0/1589724131 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3efc146a50 con 0x7f3efc0ba0e0 2024-04-24T23:05:14.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.473+0000 7f3f033a5640 1 -- 172.21.15.32:0/1589724131 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3efc0d15b0 con 0x7f3efc0ba0e0 2024-04-24T23:05:14.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.474+0000 7f3ef9ffb640 1 -- 172.21.15.32:0/1589724131 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3ef4044070 con 0x7f3efc0ba0e0 2024-04-24T23:05:14.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.474+0000 7f3ef9ffb640 1 --2- 172.21.15.32:0/1589724131 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3ee806be30 0x7f3ee806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:14.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.474+0000 7f3ef9ffb640 1 -- 172.21.15.32:0/1589724131 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5709+0+0 (secure 0 0 0) 0x7f3ef40abbf0 con 0x7f3efc0ba0e0 2024-04-24T23:05:14.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.474+0000 7f3f00940640 1 --2- 172.21.15.32:0/1589724131 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3ee806be30 0x7f3ee806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:14.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.475+0000 7f3f00940640 1 --2- 172.21.15.32:0/1589724131 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3ee806be30 0x7f3ee806e2f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0006fd0 tx=0x7f3ef0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:14.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.478+0000 7f3ef9ffb640 1 -- 172.21.15.32:0/1589724131 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3efc0d15b0 con 0x7f3efc0ba0e0 2024-04-24T23:05:14.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.660+0000 7f3f033a5640 1 -- 172.21.15.32:0/1589724131 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3efc068430 con 0x7f3efc0ba0e0 2024-04-24T23:05:14.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.661+0000 7f3ef9ffb640 1 -- 172.21.15.32:0/1589724131 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) ==== 74+0+16639 (secure 0 0 0) 0x7f3ef407a630 con 0x7f3efc0ba0e0 2024-04-24T23:05:14.663 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:05:14.663 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":128,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:05:07.320153+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:04:54.280785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-24T23:04:54.966294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:05:14.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.663+0000 7f3f033a5640 1 -- 172.21.15.32:0/1589724131 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3ee806be30 msgr2=0x7f3ee806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:14.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.663+0000 7f3f033a5640 1 --2- 172.21.15.32:0/1589724131 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3ee806be30 0x7f3ee806e2f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0006fd0 tx=0x7f3ef0008040 comp rx=0 tx=0).stop 2024-04-24T23:05:14.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.663+0000 7f3f033a5640 1 -- 172.21.15.32:0/1589724131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3efc0ba0e0 msgr2=0x7f3efc145b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:14.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.663+0000 7f3f033a5640 1 --2- 172.21.15.32:0/1589724131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3efc0ba0e0 0x7f3efc145b20 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4000c00 tx=0x7f3ef40046c0 comp rx=0 tx=0).stop 2024-04-24T23:05:14.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.663+0000 7f3f033a5640 1 -- 172.21.15.32:0/1589724131 shutdown_connections 2024-04-24T23:05:14.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.663+0000 7f3f033a5640 1 --2- 172.21.15.32:0/1589724131 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3ee806be30 0x7f3ee806e2f0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:14.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.663+0000 7f3f033a5640 1 --2- 172.21.15.32:0/1589724131 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3efc15e9b0 0x7f3efc15b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:14.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.663+0000 7f3f033a5640 1 --2- 172.21.15.32:0/1589724131 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3efc0d4210 0x7f3efc1566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:14.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.663+0000 7f3f033a5640 1 --2- 172.21.15.32:0/1589724131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3efc0ba0e0 0x7f3efc145b20 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:14.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.663+0000 7f3f033a5640 1 -- 172.21.15.32:0/1589724131 >> 172.21.15.32:0/1589724131 conn(0x7f3efc0d11d0 msgr2=0x7f3efc15d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:14.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.663+0000 7f3f033a5640 1 -- 172.21.15.32:0/1589724131 shutdown_connections 2024-04-24T23:05:14.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.663+0000 7f3f033a5640 1 -- 172.21.15.32:0/1589724131 wait complete. 2024-04-24T23:05:14.676 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-24T23:05:14.676 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:05:14.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.775+0000 7f0c76258640 1 Processor -- start 2024-04-24T23:05:14.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.775+0000 7f0c76258640 1 -- start start 2024-04-24T23:05:14.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.776+0000 7f0c76258640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0c7015e9b0 0x7f0c700d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:14.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.776+0000 7f0c76258640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0c700d4210 0x7f0c7006b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:14.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.776+0000 7f0c76258640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0c700ba0e0 0x7f0c7006b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:14.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.776+0000 7f0c76258640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f0c700d3170 con 0x7f0c700ba0e0 2024-04-24T23:05:14.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.776+0000 7f0c76258640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f0c700d3450 con 0x7f0c700d4210 2024-04-24T23:05:14.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.776+0000 7f0c76258640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f0c700d3730 con 0x7f0c7015e9b0 2024-04-24T23:05:14.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.776+0000 7f0c6ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0c700ba0e0 0x7f0c7006b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:14.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.776+0000 7f0c6ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0c700ba0e0 0x7f0c7006b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55866/0 (socket says 172.21.15.32:55866) 2024-04-24T23:05:14.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.776+0000 7f0c6ffff640 1 -- 172.21.15.32:0/2343758880 learned_addr learned my addr 172.21.15.32:0/2343758880 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:14.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.776+0000 7f0c6effd640 1 --2- 172.21.15.32:0/2343758880 >> v2:172.21.15.121:3300/0 conn(0x7f0c700d4210 0x7f0c7006b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:14.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.776+0000 7f0c6f7fe640 1 --2- 172.21.15.32:0/2343758880 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0c7015e9b0 0x7f0c700d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:14.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.776+0000 7f0c6ffff640 1 -- 172.21.15.32:0/2343758880 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0c7015e9b0 msgr2=0x7f0c700d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:14.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.776+0000 7f0c6ffff640 1 --2- 172.21.15.32:0/2343758880 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0c7015e9b0 0x7f0c700d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:14.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.776+0000 7f0c6ffff640 1 -- 172.21.15.32:0/2343758880 >> v2:172.21.15.121:3300/0 conn(0x7f0c700d4210 msgr2=0x7f0c7006b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:14.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.776+0000 7f0c6ffff640 1 --2- 172.21.15.32:0/2343758880 >> v2:172.21.15.121:3300/0 conn(0x7f0c700d4210 0x7f0c7006b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:14.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.776+0000 7f0c6ffff640 1 -- 172.21.15.32:0/2343758880 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c700d3a10 con 0x7f0c700ba0e0 2024-04-24T23:05:14.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.777+0000 7f0c6ffff640 1 --2- 172.21.15.32:0/2343758880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c700ba0e0 0x7f0c7006b650 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f0c7006bc40 tx=0x7f0c6802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2300301736e1b6f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:14.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.777+0000 7f0c6e7fc640 1 -- 172.21.15.32:0/2343758880 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c68036070 con 0x7f0c700ba0e0 2024-04-24T23:05:14.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.777+0000 7f0c6e7fc640 1 -- 172.21.15.32:0/2343758880 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c68031600 con 0x7f0c700ba0e0 2024-04-24T23:05:14.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.777+0000 7f0c6e7fc640 1 -- 172.21.15.32:0/2343758880 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c680319d0 con 0x7f0c700ba0e0 2024-04-24T23:05:14.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.778+0000 7f0c76258640 1 -- 172.21.15.32:0/2343758880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c700ba0e0 msgr2=0x7f0c7006b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:14.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.778+0000 7f0c76258640 1 --2- 172.21.15.32:0/2343758880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c700ba0e0 0x7f0c7006b650 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f0c7006bc40 tx=0x7f0c6802a6a0 comp rx=0 tx=0).stop 2024-04-24T23:05:14.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.778+0000 7f0c76258640 1 -- 172.21.15.32:0/2343758880 shutdown_connections 2024-04-24T23:05:14.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.778+0000 7f0c76258640 1 --2- 172.21.15.32:0/2343758880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c700ba0e0 0x7f0c7006b650 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:14.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.778+0000 7f0c76258640 1 --2- 172.21.15.32:0/2343758880 >> v2:172.21.15.121:3300/0 conn(0x7f0c700d4210 0x7f0c7006b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:14.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.778+0000 7f0c76258640 1 --2- 172.21.15.32:0/2343758880 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0c7015e9b0 0x7f0c700d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:14.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.778+0000 7f0c76258640 1 -- 172.21.15.32:0/2343758880 >> 172.21.15.32:0/2343758880 conn(0x7f0c700d11d0 msgr2=0x7f0c700d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:14.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.778+0000 7f0c76258640 1 -- 172.21.15.32:0/2343758880 shutdown_connections 2024-04-24T23:05:14.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.778+0000 7f0c76258640 1 -- 172.21.15.32:0/2343758880 wait complete. 2024-04-24T23:05:14.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.779+0000 7f0c76258640 1 Processor -- start 2024-04-24T23:05:14.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.779+0000 7f0c76258640 1 -- start start 2024-04-24T23:05:14.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.780+0000 7f0c76258640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0c700ba0e0 0x7f0c70155580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:14.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.780+0000 7f0c76258640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c700d4210 0x7f0c700b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:14.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.780+0000 7f0c76258640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0c7015e9b0 0x7f0c70160380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:14.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.780+0000 7f0c76258640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0c70071730 con 0x7f0c700d4210 2024-04-24T23:05:14.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.780+0000 7f0c76258640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0c700d2400 con 0x7f0c700ba0e0 2024-04-24T23:05:14.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.780+0000 7f0c76258640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f0c700718b0 con 0x7f0c7015e9b0 2024-04-24T23:05:14.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.780+0000 7f0c6effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c700d4210 0x7f0c700b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:14.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.780+0000 7f0c6effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c700d4210 0x7f0c700b4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55868/0 (socket says 172.21.15.32:55868) 2024-04-24T23:05:14.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.780+0000 7f0c6effd640 1 -- 172.21.15.32:0/2488202491 learned_addr learned my addr 172.21.15.32:0/2488202491 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:14.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.780+0000 7f0c6ffff640 1 --2- 172.21.15.32:0/2488202491 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0c7015e9b0 0x7f0c70160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:14.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.780+0000 7f0c6f7fe640 1 --2- 172.21.15.32:0/2488202491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0c700ba0e0 0x7f0c70155580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:14.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.780+0000 7f0c6effd640 1 -- 172.21.15.32:0/2488202491 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0c7015e9b0 msgr2=0x7f0c70160380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:14.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.780+0000 7f0c6effd640 1 --2- 172.21.15.32:0/2488202491 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0c7015e9b0 0x7f0c70160380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:14.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.780+0000 7f0c6effd640 1 -- 172.21.15.32:0/2488202491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0c700ba0e0 msgr2=0x7f0c70155580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:14.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.780+0000 7f0c6effd640 1 --2- 172.21.15.32:0/2488202491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0c700ba0e0 0x7f0c70155580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:14.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.780+0000 7f0c6effd640 1 -- 172.21.15.32:0/2488202491 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c701fbd00 con 0x7f0c700d4210 2024-04-24T23:05:14.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.781+0000 7f0c6effd640 1 --2- 172.21.15.32:0/2488202491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c700d4210 0x7f0c700b4b00 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f0c6400e890 tx=0x7f0c6400ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:14.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.781+0000 7f0c57fff640 1 -- 172.21.15.32:0/2488202491 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c64017070 con 0x7f0c700d4210 2024-04-24T23:05:14.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.781+0000 7f0c57fff640 1 -- 172.21.15.32:0/2488202491 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c6400c940 con 0x7f0c700d4210 2024-04-24T23:05:14.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.781+0000 7f0c76258640 1 -- 172.21.15.32:0/2488202491 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c701fbfe0 con 0x7f0c700d4210 2024-04-24T23:05:14.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.781+0000 7f0c57fff640 1 -- 172.21.15.32:0/2488202491 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c6400cd10 con 0x7f0c700d4210 2024-04-24T23:05:14.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.781+0000 7f0c76258640 1 -- 172.21.15.32:0/2488202491 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c701fc5a0 con 0x7f0c700d4210 2024-04-24T23:05:14.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.782+0000 7f0c76258640 1 -- 172.21.15.32:0/2488202491 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c700b7500 con 0x7f0c700d4210 2024-04-24T23:05:14.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.782+0000 7f0c57fff640 1 -- 172.21.15.32:0/2488202491 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0c6401a070 con 0x7f0c700d4210 2024-04-24T23:05:14.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.783+0000 7f0c57fff640 1 --2- 172.21.15.32:0/2488202491 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0c5c06be30 0x7f0c5c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:14.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.783+0000 7f0c57fff640 1 -- 172.21.15.32:0/2488202491 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5709+0+0 (secure 0 0 0) 0x7f0c6408b140 con 0x7f0c700d4210 2024-04-24T23:05:14.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.783+0000 7f0c6f7fe640 1 --2- 172.21.15.32:0/2488202491 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0c5c06be30 0x7f0c5c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:14.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.784+0000 7f0c6f7fe640 1 --2- 172.21.15.32:0/2488202491 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0c5c06be30 0x7f0c5c06e2f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f0c60009aa0 tx=0x7f0c60009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:14.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:14.787+0000 7f0c57fff640 1 -- 172.21.15.32:0/2488202491 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0c700b7500 con 0x7f0c700d4210 2024-04-24T23:05:15.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.008+0000 7f0c76258640 1 -- 172.21.15.32:0/2488202491 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c70155ac0 con 0x7f0c700d4210 2024-04-24T23:05:15.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.008+0000 7f0c57fff640 1 -- 172.21.15.32:0/2488202491 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+5189 (secure 0 0 0) 0x7f0c64059490 con 0x7f0c700d4210 2024-04-24T23:05:15.010 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:05:15.010 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":51,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5972,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3336404028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3336404028},{"type":"v1","addr":"172.21.15.121:6837","nonce":3336404028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6505,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/1068557774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1068557774},{"type":"v1","addr":"172.21.15.121:6836","nonce":1068557774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":7353,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/2118719195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":2118719195},{"type":"v1","addr":"172.21.15.32:6834","nonce":2118719195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:04:55.998022+0000","modified":"2024-04-24T23:04:58.657258+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7356},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7356":{"gid":7356,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/3015290487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":3015290487},{"type":"v1","addr":"172.21.15.32:6835","nonce":3015290487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7356,"qdb_cluster":[7356]},"id":7}]} 2024-04-24T23:05:15.011 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 51 2024-04-24T23:05:15.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.010+0000 7f0c76258640 1 -- 172.21.15.32:0/2488202491 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0c5c06be30 msgr2=0x7f0c5c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:15.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.010+0000 7f0c76258640 1 --2- 172.21.15.32:0/2488202491 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0c5c06be30 0x7f0c5c06e2f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f0c60009aa0 tx=0x7f0c60009190 comp rx=0 tx=0).stop 2024-04-24T23:05:15.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.010+0000 7f0c76258640 1 -- 172.21.15.32:0/2488202491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c700d4210 msgr2=0x7f0c700b4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:15.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.010+0000 7f0c76258640 1 --2- 172.21.15.32:0/2488202491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c700d4210 0x7f0c700b4b00 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f0c6400e890 tx=0x7f0c6400ed50 comp rx=0 tx=0).stop 2024-04-24T23:05:15.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.010+0000 7f0c76258640 1 -- 172.21.15.32:0/2488202491 shutdown_connections 2024-04-24T23:05:15.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.010+0000 7f0c76258640 1 --2- 172.21.15.32:0/2488202491 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0c5c06be30 0x7f0c5c06e2f0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:15.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.010+0000 7f0c76258640 1 --2- 172.21.15.32:0/2488202491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c700d4210 0x7f0c700b4b00 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:15.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.010+0000 7f0c76258640 1 --2- 172.21.15.32:0/2488202491 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0c7015e9b0 0x7f0c70160380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:15.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.010+0000 7f0c76258640 1 --2- 172.21.15.32:0/2488202491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0c700ba0e0 0x7f0c70155580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:15.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.010+0000 7f0c76258640 1 -- 172.21.15.32:0/2488202491 >> 172.21.15.32:0/2488202491 conn(0x7f0c700d11d0 msgr2=0x7f0c70159a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:15.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.011+0000 7f0c76258640 1 -- 172.21.15.32:0/2488202491 shutdown_connections 2024-04-24T23:05:15.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.011+0000 7f0c76258640 1 -- 172.21.15.32:0/2488202491 wait complete. 2024-04-24T23:05:15.023 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:05:15.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.122+0000 7f3df0970640 1 Processor -- start 2024-04-24T23:05:15.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.122+0000 7f3df0970640 1 -- start start 2024-04-24T23:05:15.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.123+0000 7f3df0970640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3dec0b44d0 0x7f3dec0b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:15.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.123+0000 7f3df0970640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3dec0b6430 con 0x7f3dec06c5f0 2024-04-24T23:05:15.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.123+0000 7f3df0970640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3dec156f80 con 0x7f3dec0af890 2024-04-24T23:05:15.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.123+0000 7f3df0970640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f3dec0b6150 con 0x7f3dec0b44d0 2024-04-24T23:05:15.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.123+0000 7f3dea59c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3dec06c5f0 0x7f3dec06c9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45208/0 (socket says 172.21.15.32:45208) 2024-04-24T23:05:15.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.123+0000 7f3dea59c640 1 -- 172.21.15.32:0/3271330113 learned_addr learned my addr 172.21.15.32:0/3271330113 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:05:15.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.123+0000 7f3de3fff640 1 --2- 172.21.15.32:0/3271330113 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3dec0b44d0 0x7f3dec0b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:15.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.123+0000 7f3de9d9b640 1 -- 172.21.15.32:0/3271330113 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3182419674 0 0) 0x7f3dec0b6430 con 0x7f3dec06c5f0 2024-04-24T23:05:15.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.123+0000 7f3de9d9b640 1 -- 172.21.15.32:0/3271330113 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3dd8003520 con 0x7f3dec06c5f0 2024-04-24T23:05:15.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.123+0000 7f3de3fff640 1 -- 172.21.15.32:0/3271330113 >> v1:172.21.15.121:6789/0 conn(0x7f3dec0af890 legacy=0x7f3dec15cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:15.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.123+0000 7f3de3fff640 1 -- 172.21.15.32:0/3271330113 >> v1:172.21.15.32:6789/0 conn(0x7f3dec06c5f0 legacy=0x7f3dec06c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:15.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.123+0000 7f3de9d9b640 1 -- 172.21.15.32:0/3271330113 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2919227449 0 0) 0x7f3dec156f80 con 0x7f3dec0af890 2024-04-24T23:05:15.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.123+0000 7f3de3fff640 1 -- 172.21.15.32:0/3271330113 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dec0b39f0 con 0x7f3dec0b44d0 2024-04-24T23:05:15.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.124+0000 7f3de3fff640 1 --2- 172.21.15.32:0/3271330113 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3dec0b44d0 0x7f3dec0b5890 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3dd40097d0 tx=0x7f3dd402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=347ef23da2a8376c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:15.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.124+0000 7f3de9d9b640 1 -- 172.21.15.32:0/3271330113 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd4037070 con 0x7f3dec0b44d0 2024-04-24T23:05:15.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.124+0000 7f3de9d9b640 1 -- 172.21.15.32:0/3271330113 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dd4032440 con 0x7f3dec0b44d0 2024-04-24T23:05:15.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.124+0000 7f3de9d9b640 1 -- 172.21.15.32:0/3271330113 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd40327f0 con 0x7f3dec0b44d0 2024-04-24T23:05:15.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.125+0000 7f3df0970640 1 -- 172.21.15.32:0/3271330113 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3dec0b44d0 msgr2=0x7f3dec0b5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:15.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.125+0000 7f3df0970640 1 --2- 172.21.15.32:0/3271330113 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3dec0b44d0 0x7f3dec0b5890 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3dd40097d0 tx=0x7f3dd402a180 comp rx=0 tx=0).stop 2024-04-24T23:05:15.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.125+0000 7f3df0970640 1 -- 172.21.15.32:0/3271330113 shutdown_connections 2024-04-24T23:05:15.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.125+0000 7f3df0970640 1 --2- 172.21.15.32:0/3271330113 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3dec0b44d0 0x7f3dec0b5890 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:15.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.125+0000 7f3df0970640 1 -- 172.21.15.32:0/3271330113 >> 172.21.15.32:0/3271330113 conn(0x7f3dec06b110 msgr2=0x7f3dec06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:15.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.125+0000 7f3df0970640 1 -- 172.21.15.32:0/3271330113 shutdown_connections 2024-04-24T23:05:15.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.125+0000 7f3df0970640 1 -- 172.21.15.32:0/3271330113 wait complete. 2024-04-24T23:05:15.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.126+0000 7f3df0970640 1 Processor -- start 2024-04-24T23:05:15.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.126+0000 7f3df0970640 1 -- start start 2024-04-24T23:05:15.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.127+0000 7f3df0970640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dec06c5f0 0x7f3dec0b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:15.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.127+0000 7f3df0970640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3dec0af890 0x7f3dec1591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:15.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.127+0000 7f3df0970640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3dec0b44d0 0x7f3dec142880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:15.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.127+0000 7f3df0970640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3dec0b5dd0 con 0x7f3dec06c5f0 2024-04-24T23:05:15.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.127+0000 7f3df0970640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3dec0b3fb0 con 0x7f3dec0b44d0 2024-04-24T23:05:15.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.127+0000 7f3df0970640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f3dec15d1d0 con 0x7f3dec0af890 2024-04-24T23:05:15.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.127+0000 7f3dea59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dec06c5f0 0x7f3dec0b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:15.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.127+0000 7f3dea59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dec06c5f0 0x7f3dec0b12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55870/0 (socket says 172.21.15.32:55870) 2024-04-24T23:05:15.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.127+0000 7f3dea59c640 1 -- 172.21.15.32:0/1038579441 learned_addr learned my addr 172.21.15.32:0/1038579441 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:15.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.127+0000 7f3de3fff640 1 --2- 172.21.15.32:0/1038579441 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3dec0af890 0x7f3dec1591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:15.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.127+0000 7f3dead9d640 1 --2- 172.21.15.32:0/1038579441 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3dec0b44d0 0x7f3dec142880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:15.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.127+0000 7f3dea59c640 1 -- 172.21.15.32:0/1038579441 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3dec0af890 msgr2=0x7f3dec1591d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:15.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.127+0000 7f3dea59c640 1 --2- 172.21.15.32:0/1038579441 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3dec0af890 0x7f3dec1591d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:15.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.127+0000 7f3dea59c640 1 -- 172.21.15.32:0/1038579441 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3dec0b44d0 msgr2=0x7f3dec142880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:15.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.127+0000 7f3dea59c640 1 --2- 172.21.15.32:0/1038579441 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3dec0b44d0 0x7f3dec142880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:15.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.127+0000 7f3dea59c640 1 -- 172.21.15.32:0/1038579441 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dec0b1820 con 0x7f3dec06c5f0 2024-04-24T23:05:15.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.127+0000 7f3dea59c640 1 --2- 172.21.15.32:0/1038579441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dec06c5f0 0x7f3dec0b12e0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc00c430 tx=0x7f3ddc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:15.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.128+0000 7f3de2ffd640 1 -- 172.21.15.32:0/1038579441 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc017070 con 0x7f3dec06c5f0 2024-04-24T23:05:15.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.128+0000 7f3de2ffd640 1 -- 172.21.15.32:0/1038579441 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ddc005ce0 con 0x7f3dec06c5f0 2024-04-24T23:05:15.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.128+0000 7f3df0970640 1 -- 172.21.15.32:0/1038579441 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dec15b730 con 0x7f3dec06c5f0 2024-04-24T23:05:15.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.128+0000 7f3de2ffd640 1 -- 172.21.15.32:0/1038579441 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc0072f0 con 0x7f3dec06c5f0 2024-04-24T23:05:15.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.128+0000 7f3df0970640 1 -- 172.21.15.32:0/1038579441 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dec15bcf0 con 0x7f3dec06c5f0 2024-04-24T23:05:15.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.129+0000 7f3df0970640 1 -- 172.21.15.32:0/1038579441 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dec068430 con 0x7f3dec06c5f0 2024-04-24T23:05:15.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.129+0000 7f3de2ffd640 1 -- 172.21.15.32:0/1038579441 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3ddc01a070 con 0x7f3dec06c5f0 2024-04-24T23:05:15.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.129+0000 7f3de2ffd640 1 --2- 172.21.15.32:0/1038579441 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3dcc06be30 0x7f3dcc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:15.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.130+0000 7f3de2ffd640 1 -- 172.21.15.32:0/1038579441 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5709+0+0 (secure 0 0 0) 0x7f3ddc08a360 con 0x7f3dec06c5f0 2024-04-24T23:05:15.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.130+0000 7f3de3fff640 1 --2- 172.21.15.32:0/1038579441 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3dcc06be30 0x7f3dcc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:15.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.130+0000 7f3de3fff640 1 --2- 172.21.15.32:0/1038579441 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3dcc06be30 0x7f3dcc06e2f0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4034000 tx=0x7f3dd402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:15.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.134+0000 7f3de2ffd640 1 -- 172.21.15.32:0/1038579441 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3ddc058d10 con 0x7f3dec06c5f0 2024-04-24T23:05:15.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.353+0000 7f3df0970640 1 -- 172.21.15.32:0/1038579441 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3dec15d550 con 0x7f3dec06c5f0 2024-04-24T23:05:15.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.353+0000 7f3de2ffd640 1 -- 172.21.15.32:0/1038579441 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+5189 (secure 0 0 0) 0x7f3ddc058610 con 0x7f3dec06c5f0 2024-04-24T23:05:15.355 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:05:15.355 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":51,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5972,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3336404028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3336404028},{"type":"v1","addr":"172.21.15.121:6837","nonce":3336404028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6505,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/1068557774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1068557774},{"type":"v1","addr":"172.21.15.121:6836","nonce":1068557774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":7353,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/2118719195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":2118719195},{"type":"v1","addr":"172.21.15.32:6834","nonce":2118719195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:04:55.998022+0000","modified":"2024-04-24T23:04:58.657258+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7356},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7356":{"gid":7356,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/3015290487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":3015290487},{"type":"v1","addr":"172.21.15.32:6835","nonce":3015290487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7356,"qdb_cluster":[7356]},"id":7}]} 2024-04-24T23:05:15.355 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 51 2024-04-24T23:05:15.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.355+0000 7f3df0970640 1 -- 172.21.15.32:0/1038579441 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3dcc06be30 msgr2=0x7f3dcc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:15.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.355+0000 7f3df0970640 1 --2- 172.21.15.32:0/1038579441 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3dcc06be30 0x7f3dcc06e2f0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4034000 tx=0x7f3dd402e000 comp rx=0 tx=0).stop 2024-04-24T23:05:15.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.355+0000 7f3df0970640 1 -- 172.21.15.32:0/1038579441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dec06c5f0 msgr2=0x7f3dec0b12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:15.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.355+0000 7f3df0970640 1 --2- 172.21.15.32:0/1038579441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dec06c5f0 0x7f3dec0b12e0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc00c430 tx=0x7f3ddc00c8f0 comp rx=0 tx=0).stop 2024-04-24T23:05:15.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.355+0000 7f3df0970640 1 -- 172.21.15.32:0/1038579441 shutdown_connections 2024-04-24T23:05:15.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.355+0000 7f3df0970640 1 --2- 172.21.15.32:0/1038579441 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3dcc06be30 0x7f3dcc06e2f0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:15.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.355+0000 7f3df0970640 1 --2- 172.21.15.32:0/1038579441 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3dec0b44d0 0x7f3dec142880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:15.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.355+0000 7f3df0970640 1 --2- 172.21.15.32:0/1038579441 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3dec0af890 0x7f3dec1591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:15.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.355+0000 7f3df0970640 1 --2- 172.21.15.32:0/1038579441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dec06c5f0 0x7f3dec0b12e0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:15.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.355+0000 7f3df0970640 1 -- 172.21.15.32:0/1038579441 >> 172.21.15.32:0/1038579441 conn(0x7f3dec06b110 msgr2=0x7f3dec06c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:15.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.355+0000 7f3df0970640 1 -- 172.21.15.32:0/1038579441 shutdown_connections 2024-04-24T23:05:15.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.355+0000 7f3df0970640 1 -- 172.21.15.32:0/1038579441 wait complete. 2024-04-24T23:05:15.368 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:05:15.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.467+0000 7fc35fc09640 1 Processor -- start 2024-04-24T23:05:15.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.467+0000 7fc35fc09640 1 -- start start 2024-04-24T23:05:15.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.468+0000 7fc35fc09640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc358162320 0x7fc3580ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:15.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.468+0000 7fc35fc09640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc358162e10 0x7fc35815ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:15.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.468+0000 7fc35fc09640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3580cd270 con 0x7fc3580b5af0 2024-04-24T23:05:15.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.468+0000 7fc35fc09640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fc3580b5660 con 0x7fc358162e10 2024-04-24T23:05:15.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.468+0000 7fc35fc09640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc358071610 con 0x7fc358162320 2024-04-24T23:05:15.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.468+0000 7fc35e1a6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc358162e10 0x7fc35815ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:15.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.468+0000 7fc35e1a6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc358162e10 0x7fc35815ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55876/0 (socket says 172.21.15.32:55876) 2024-04-24T23:05:15.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.468+0000 7fc35e1a6640 1 -- 172.21.15.32:0/3179582407 learned_addr learned my addr 172.21.15.32:0/3179582407 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:15.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.468+0000 7fc35d9a5640 1 --2- 172.21.15.32:0/3179582407 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc358162320 0x7fc3580ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:15.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.468+0000 7fc35e1a6640 1 -- 172.21.15.32:0/3179582407 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc358162320 msgr2=0x7fc3580ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:15.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.468+0000 7fc35e1a6640 1 --2- 172.21.15.32:0/3179582407 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc358162320 0x7fc3580ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:15.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.468+0000 7fc35e1a6640 1 -- 172.21.15.32:0/3179582407 >> v1:172.21.15.121:6789/0 conn(0x7fc3580b5af0 legacy=0x7fc358162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:15.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.468+0000 7fc35e1a6640 1 -- 172.21.15.32:0/3179582407 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3580718f0 con 0x7fc358162e10 2024-04-24T23:05:15.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.468+0000 7fc35d9a5640 1 --2- 172.21.15.32:0/3179582407 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc358162320 0x7fc3580ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:05:15.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.469+0000 7fc35e1a6640 1 --2- 172.21.15.32:0/3179582407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc358162e10 0x7fc35815ed30 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fc350009e50 tx=0x7fc35002a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2eee592788b1894 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:15.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.469+0000 7fc35c9a3640 1 -- 172.21.15.32:0/3179582407 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc350036070 con 0x7fc358162e10 2024-04-24T23:05:15.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.469+0000 7fc35c9a3640 1 -- 172.21.15.32:0/3179582407 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc350031600 con 0x7fc358162e10 2024-04-24T23:05:15.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.469+0000 7fc35c9a3640 1 -- 172.21.15.32:0/3179582407 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3500319d0 con 0x7fc358162e10 2024-04-24T23:05:15.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.469+0000 7fc35fc09640 1 -- 172.21.15.32:0/3179582407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc358162e10 msgr2=0x7fc35815ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:15.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.469+0000 7fc35fc09640 1 --2- 172.21.15.32:0/3179582407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc358162e10 0x7fc35815ed30 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fc350009e50 tx=0x7fc35002a640 comp rx=0 tx=0).stop 2024-04-24T23:05:15.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.469+0000 7fc35fc09640 1 -- 172.21.15.32:0/3179582407 shutdown_connections 2024-04-24T23:05:15.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.469+0000 7fc35fc09640 1 --2- 172.21.15.32:0/3179582407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc358162e10 0x7fc35815ed30 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:15.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.469+0000 7fc35fc09640 1 --2- 172.21.15.32:0/3179582407 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc358162320 0x7fc3580ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:15.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.469+0000 7fc35fc09640 1 -- 172.21.15.32:0/3179582407 >> 172.21.15.32:0/3179582407 conn(0x7fc3580d0ae0 msgr2=0x7fc3580d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:15.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.470+0000 7fc35fc09640 1 -- 172.21.15.32:0/3179582407 shutdown_connections 2024-04-24T23:05:15.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.470+0000 7fc35fc09640 1 -- 172.21.15.32:0/3179582407 wait complete. 2024-04-24T23:05:15.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.471+0000 7fc35fc09640 1 Processor -- start 2024-04-24T23:05:15.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.471+0000 7fc35fc09640 1 -- start start 2024-04-24T23:05:15.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.471+0000 7fc35fc09640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc3580b5af0 0x7fc358147c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:15.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.471+0000 7fc35fc09640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc358162320 0x7fc3580cea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:15.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.472+0000 7fc35fc09640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc358162e10 0x7fc3581600d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:15.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.472+0000 7fc35fc09640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc358071bd0 con 0x7fc358162320 2024-04-24T23:05:15.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.472+0000 7fc35fc09640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc358067950 con 0x7fc358162e10 2024-04-24T23:05:15.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.472+0000 7fc35fc09640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc358071d50 con 0x7fc3580b5af0 2024-04-24T23:05:15.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.472+0000 7fc35d1a4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc358162320 0x7fc3580cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:15.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.472+0000 7fc35d1a4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc358162320 0x7fc3580cea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55878/0 (socket says 172.21.15.32:55878) 2024-04-24T23:05:15.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.472+0000 7fc35d1a4640 1 -- 172.21.15.32:0/3335605478 learned_addr learned my addr 172.21.15.32:0/3335605478 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:15.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.472+0000 7fc35e1a6640 1 --2- 172.21.15.32:0/3335605478 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc358162e10 0x7fc3581600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:15.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.472+0000 7fc35d9a5640 1 --2- 172.21.15.32:0/3335605478 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc3580b5af0 0x7fc358147c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:15.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.472+0000 7fc35d1a4640 1 -- 172.21.15.32:0/3335605478 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc3580b5af0 msgr2=0x7fc358147c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:15.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.472+0000 7fc35d1a4640 1 --2- 172.21.15.32:0/3335605478 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc3580b5af0 0x7fc358147c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:15.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.472+0000 7fc35d1a4640 1 -- 172.21.15.32:0/3335605478 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc358162e10 msgr2=0x7fc3581600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:15.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.472+0000 7fc35d1a4640 1 --2- 172.21.15.32:0/3335605478 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc358162e10 0x7fc3581600d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:15.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.472+0000 7fc35d1a4640 1 -- 172.21.15.32:0/3335605478 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc35806ccd0 con 0x7fc358162320 2024-04-24T23:05:15.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.472+0000 7fc35d1a4640 1 --2- 172.21.15.32:0/3335605478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc358162320 0x7fc3580cea80 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fc34800c460 tx=0x7fc34800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:15.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.472+0000 7fc34e7fc640 1 -- 172.21.15.32:0/3335605478 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc348016020 con 0x7fc358162320 2024-04-24T23:05:15.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.472+0000 7fc34e7fc640 1 -- 172.21.15.32:0/3335605478 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc348012870 con 0x7fc358162320 2024-04-24T23:05:15.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.472+0000 7fc35fc09640 1 -- 172.21.15.32:0/3335605478 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc35806cfb0 con 0x7fc358162320 2024-04-24T23:05:15.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.472+0000 7fc34e7fc640 1 -- 172.21.15.32:0/3335605478 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc348012c60 con 0x7fc358162320 2024-04-24T23:05:15.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.473+0000 7fc35fc09640 1 -- 172.21.15.32:0/3335605478 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc35806d570 con 0x7fc358162320 2024-04-24T23:05:15.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.474+0000 7fc35fc09640 1 -- 172.21.15.32:0/3335605478 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc358068430 con 0x7fc358162320 2024-04-24T23:05:15.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.474+0000 7fc34e7fc640 1 -- 172.21.15.32:0/3335605478 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc348019020 con 0x7fc358162320 2024-04-24T23:05:15.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.474+0000 7fc34e7fc640 1 --2- 172.21.15.32:0/3335605478 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc33c06be30 0x7fc33c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:15.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.475+0000 7fc34e7fc640 1 -- 172.21.15.32:0/3335605478 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5709+0+0 (secure 0 0 0) 0x7fc34808aa70 con 0x7fc358162320 2024-04-24T23:05:15.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.475+0000 7fc35d9a5640 1 --2- 172.21.15.32:0/3335605478 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc33c06be30 0x7fc33c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:15.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.475+0000 7fc35d9a5640 1 --2- 172.21.15.32:0/3335605478 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc33c06be30 0x7fc33c06e2f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc35400ab60 tx=0x7fc354008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:15.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.479+0000 7fc34e7fc640 1 -- 172.21.15.32:0/3335605478 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc348020030 con 0x7fc358162320 2024-04-24T23:05:15.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.664+0000 7fc35fc09640 1 -- 172.21.15.32:0/3335605478 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc3580d0850 con 0x7fc358162320 2024-04-24T23:05:15.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.665+0000 7fc34e7fc640 1 -- 172.21.15.32:0/3335605478 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) ==== 74+0+16639 (secure 0 0 0) 0x7fc348058dd0 con 0x7fc358162320 2024-04-24T23:05:15.666 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:05:15.667 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":128,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:05:07.320153+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:04:54.280785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-24T23:04:54.966294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:05:15.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.667+0000 7fc35fc09640 1 -- 172.21.15.32:0/3335605478 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc33c06be30 msgr2=0x7fc33c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:15.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.667+0000 7fc35fc09640 1 --2- 172.21.15.32:0/3335605478 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc33c06be30 0x7fc33c06e2f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc35400ab60 tx=0x7fc354008040 comp rx=0 tx=0).stop 2024-04-24T23:05:15.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.667+0000 7fc35fc09640 1 -- 172.21.15.32:0/3335605478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc358162320 msgr2=0x7fc3580cea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:15.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.667+0000 7fc35fc09640 1 --2- 172.21.15.32:0/3335605478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc358162320 0x7fc3580cea80 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fc34800c460 tx=0x7fc34800c920 comp rx=0 tx=0).stop 2024-04-24T23:05:15.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.667+0000 7fc35fc09640 1 -- 172.21.15.32:0/3335605478 shutdown_connections 2024-04-24T23:05:15.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.667+0000 7fc35fc09640 1 --2- 172.21.15.32:0/3335605478 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc33c06be30 0x7fc33c06e2f0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:15.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.667+0000 7fc35fc09640 1 --2- 172.21.15.32:0/3335605478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc358162320 0x7fc3580cea80 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:15.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.667+0000 7fc35fc09640 1 --2- 172.21.15.32:0/3335605478 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc358162e10 0x7fc3581600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:15.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.667+0000 7fc35fc09640 1 --2- 172.21.15.32:0/3335605478 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc3580b5af0 0x7fc358147c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:15.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.667+0000 7fc35fc09640 1 -- 172.21.15.32:0/3335605478 >> 172.21.15.32:0/3335605478 conn(0x7fc3580d0ae0 msgr2=0x7fc35815c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:15.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.667+0000 7fc35fc09640 1 -- 172.21.15.32:0/3335605478 shutdown_connections 2024-04-24T23:05:15.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.667+0000 7fc35fc09640 1 -- 172.21.15.32:0/3335605478 wait complete. 2024-04-24T23:05:15.679 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-24T23:05:15.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.780+0000 7f041fbd5640 1 Processor -- start 2024-04-24T23:05:15.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.780+0000 7f041fbd5640 1 -- start start 2024-04-24T23:05:15.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.781+0000 7f041fbd5640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f041806c5f0 0x7f041806c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:15.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.781+0000 7f041fbd5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f04180b44d0 0x7f04180b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:15.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.781+0000 7f041fbd5640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04180b6430 con 0x7f04180af890 2024-04-24T23:05:15.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.781+0000 7f041fbd5640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f0418156f80 con 0x7f041806c5f0 2024-04-24T23:05:15.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.781+0000 7f041fbd5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f04180b6150 con 0x7f04180b44d0 2024-04-24T23:05:15.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.781+0000 7f041d971640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f041806c5f0 0x7f041806c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:15.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.781+0000 7f041e172640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f04180af890 0x7f04180cef70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:55764/0 (socket says 172.21.15.32:55764) 2024-04-24T23:05:15.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.781+0000 7f041e172640 1 -- 172.21.15.32:0/3890652441 learned_addr learned my addr 172.21.15.32:0/3890652441 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:05:15.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.781+0000 7f041d170640 1 --2- 172.21.15.32:0/3890652441 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f04180b44d0 0x7f04180b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:15.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.781+0000 7f041d971640 1 -- 172.21.15.32:0/3890652441 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f04180b44d0 msgr2=0x7f04180b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:15.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.781+0000 7f041d971640 1 --2- 172.21.15.32:0/3890652441 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f04180b44d0 0x7f04180b5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:15.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.781+0000 7f041d971640 1 -- 172.21.15.32:0/3890652441 >> v1:172.21.15.121:6789/0 conn(0x7f04180af890 legacy=0x7f04180cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:15.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.781+0000 7f041d971640 1 -- 172.21.15.32:0/3890652441 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04180b39f0 con 0x7f041806c5f0 2024-04-24T23:05:15.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.782+0000 7f041d971640 1 --2- 172.21.15.32:0/3890652441 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f041806c5f0 0x7f041806c9d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0408009d90 tx=0x7f040802a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea9b8c6169d03b78 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:15.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.782+0000 7f041c96f640 1 -- 172.21.15.32:0/3890652441 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0408031600 con 0x7f041806c5f0 2024-04-24T23:05:15.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.782+0000 7f041c96f640 1 -- 172.21.15.32:0/3890652441 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04080317a0 con 0x7f041806c5f0 2024-04-24T23:05:15.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.782+0000 7f041c96f640 1 -- 172.21.15.32:0/3890652441 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0408031b70 con 0x7f041806c5f0 2024-04-24T23:05:15.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.783+0000 7f041fbd5640 1 -- 172.21.15.32:0/3890652441 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f041806c5f0 msgr2=0x7f041806c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:15.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.783+0000 7f041fbd5640 1 --2- 172.21.15.32:0/3890652441 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f041806c5f0 0x7f041806c9d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0408009d90 tx=0x7f040802a5d0 comp rx=0 tx=0).stop 2024-04-24T23:05:15.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.783+0000 7f041fbd5640 1 -- 172.21.15.32:0/3890652441 shutdown_connections 2024-04-24T23:05:15.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.783+0000 7f041fbd5640 1 --2- 172.21.15.32:0/3890652441 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f04180b44d0 0x7f04180b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:15.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.783+0000 7f041fbd5640 1 --2- 172.21.15.32:0/3890652441 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f041806c5f0 0x7f041806c9d0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:15.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.783+0000 7f041fbd5640 1 -- 172.21.15.32:0/3890652441 >> 172.21.15.32:0/3890652441 conn(0x7f041806b110 msgr2=0x7f041806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:15.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.783+0000 7f041fbd5640 1 -- 172.21.15.32:0/3890652441 shutdown_connections 2024-04-24T23:05:15.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.783+0000 7f041fbd5640 1 -- 172.21.15.32:0/3890652441 wait complete. 2024-04-24T23:05:15.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.784+0000 7f041fbd5640 1 Processor -- start 2024-04-24T23:05:15.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.784+0000 7f041fbd5640 1 -- start start 2024-04-24T23:05:15.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.785+0000 7f041fbd5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f041806c5f0 0x7f04180ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:15.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.785+0000 7f041fbd5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04180af890 0x7f0418157160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:15.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.785+0000 7f041fbd5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f04180b44d0 0x7f04180aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:15.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.785+0000 7f041fbd5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f041806cf10 con 0x7f04180af890 2024-04-24T23:05:15.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.785+0000 7f041fbd5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f04180b3fb0 con 0x7f04180b44d0 2024-04-24T23:05:15.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.785+0000 7f041fbd5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f04180b5dd0 con 0x7f041806c5f0 2024-04-24T23:05:15.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.785+0000 7f041d170640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04180af890 0x7f0418157160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:15.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.785+0000 7f041d170640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04180af890 0x7f0418157160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55880/0 (socket says 172.21.15.32:55880) 2024-04-24T23:05:15.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.785+0000 7f041d170640 1 -- 172.21.15.32:0/4121104799 learned_addr learned my addr 172.21.15.32:0/4121104799 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:15.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.785+0000 7f041d971640 1 --2- 172.21.15.32:0/4121104799 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f041806c5f0 0x7f04180ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:15.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.785+0000 7f041e172640 1 --2- 172.21.15.32:0/4121104799 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f04180b44d0 0x7f04180aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:15.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.785+0000 7f041d170640 1 -- 172.21.15.32:0/4121104799 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f041806c5f0 msgr2=0x7f04180ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:15.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.785+0000 7f041d170640 1 --2- 172.21.15.32:0/4121104799 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f041806c5f0 0x7f04180ce9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:15.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.785+0000 7f041d170640 1 -- 172.21.15.32:0/4121104799 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f04180b44d0 msgr2=0x7f04180aeb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:15.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.785+0000 7f041d170640 1 --2- 172.21.15.32:0/4121104799 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f04180b44d0 0x7f04180aeb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:15.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.785+0000 7f041d170640 1 -- 172.21.15.32:0/4121104799 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04181578d0 con 0x7f04180af890 2024-04-24T23:05:15.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.785+0000 7f041e172640 1 --2- 172.21.15.32:0/4121104799 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f04180b44d0 0x7f04180aeb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:05:15.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.785+0000 7f041d170640 1 --2- 172.21.15.32:0/4121104799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04180af890 0x7f0418157160 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f040c00e9b0 tx=0x7f040c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:15.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.786+0000 7f04167fc640 1 -- 172.21.15.32:0/4121104799 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c017070 con 0x7f04180af890 2024-04-24T23:05:15.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.786+0000 7f04167fc640 1 -- 172.21.15.32:0/4121104799 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f040c00c9a0 con 0x7f04180af890 2024-04-24T23:05:15.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.786+0000 7f041fbd5640 1 -- 172.21.15.32:0/4121104799 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0418157bb0 con 0x7f04180af890 2024-04-24T23:05:15.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.786+0000 7f04167fc640 1 -- 172.21.15.32:0/4121104799 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c00cd90 con 0x7f04180af890 2024-04-24T23:05:15.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.786+0000 7f041fbd5640 1 -- 172.21.15.32:0/4121104799 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0418158170 con 0x7f04180af890 2024-04-24T23:05:15.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.786+0000 7f041fbd5640 1 -- 172.21.15.32:0/4121104799 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0418068430 con 0x7f04180af890 2024-04-24T23:05:15.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.787+0000 7f04167fc640 1 -- 172.21.15.32:0/4121104799 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f040c01a070 con 0x7f04180af890 2024-04-24T23:05:15.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.788+0000 7f04167fc640 1 --2- 172.21.15.32:0/4121104799 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f040406be30 0x7f040406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:15.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.788+0000 7f04167fc640 1 -- 172.21.15.32:0/4121104799 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5709+0+0 (secure 0 0 0) 0x7f040c08b0a0 con 0x7f04180af890 2024-04-24T23:05:15.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.788+0000 7f041d971640 1 --2- 172.21.15.32:0/4121104799 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f040406be30 0x7f040406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:15.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.788+0000 7f041d971640 1 --2- 172.21.15.32:0/4121104799 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f040406be30 0x7f040406e2f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f0408034000 tx=0x7f040802f000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:15.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:15.792+0000 7f04167fc640 1 -- 172.21.15.32:0/4121104799 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f040c059490 con 0x7f04180af890 2024-04-24T23:05:16.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.019+0000 7f041fbd5640 1 -- 172.21.15.32:0/4121104799 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f04180b5670 con 0x7f04180af890 2024-04-24T23:05:16.021 INFO:tasks.ceph.mon.a.smithi032.stderr:2024-04-24T23:05:16.019+0000 7f3b809c6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T23:05:16.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.021+0000 7f04167fc640 1 -- 172.21.15.32:0/4121104799 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v52) ==== 161+0+0 (secure 0 0 0) 0x7f040c05d790 con 0x7f04180af890 2024-04-24T23:05:16.022 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-24T23:05:16.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.025+0000 7f041fbd5640 1 -- 172.21.15.32:0/4121104799 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f040406be30 msgr2=0x7f040406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:16.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.025+0000 7f041fbd5640 1 --2- 172.21.15.32:0/4121104799 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f040406be30 0x7f040406e2f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f0408034000 tx=0x7f040802f000 comp rx=0 tx=0).stop 2024-04-24T23:05:16.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.025+0000 7f041fbd5640 1 -- 172.21.15.32:0/4121104799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04180af890 msgr2=0x7f0418157160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:16.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.025+0000 7f041fbd5640 1 --2- 172.21.15.32:0/4121104799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04180af890 0x7f0418157160 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f040c00e9b0 tx=0x7f040c00ee70 comp rx=0 tx=0).stop 2024-04-24T23:05:16.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.025+0000 7f041fbd5640 1 -- 172.21.15.32:0/4121104799 shutdown_connections 2024-04-24T23:05:16.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.025+0000 7f041fbd5640 1 --2- 172.21.15.32:0/4121104799 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f040406be30 0x7f040406e2f0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:16.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.025+0000 7f041fbd5640 1 --2- 172.21.15.32:0/4121104799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04180af890 0x7f0418157160 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:16.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.025+0000 7f041fbd5640 1 --2- 172.21.15.32:0/4121104799 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f04180b44d0 0x7f04180aeb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:16.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.025+0000 7f041fbd5640 1 --2- 172.21.15.32:0/4121104799 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f041806c5f0 0x7f04180ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:16.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.025+0000 7f041fbd5640 1 -- 172.21.15.32:0/4121104799 >> 172.21.15.32:0/4121104799 conn(0x7f041806b110 msgr2=0x7f04180b01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:16.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.025+0000 7f041fbd5640 1 -- 172.21.15.32:0/4121104799 shutdown_connections 2024-04-24T23:05:16.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.025+0000 7f041fbd5640 1 -- 172.21.15.32:0/4121104799 wait complete. 2024-04-24T23:05:16.028 INFO:tasks.ceph.mds.a.smithi032.stderr: -5179> 2024-04-24T23:04:57.269+0000 7f4b6dd7d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T23:05:16.038 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-24T23:05:16.058 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2024-04-24T23:05:16.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.140+0000 7f8941ca5640 1 Processor -- start 2024-04-24T23:05:16.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.140+0000 7f8941ca5640 1 -- start start 2024-04-24T23:05:16.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.141+0000 7f8941ca5640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f893c0b5af0 0x7f893c0ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:16.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.141+0000 7f8941ca5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f893c162e10 0x7f893c15ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:16.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.141+0000 7f8941ca5640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f893c0cd270 con 0x7f893c162320 2024-04-24T23:05:16.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.141+0000 7f8941ca5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f893c0b5660 con 0x7f893c162e10 2024-04-24T23:05:16.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.141+0000 7f8941ca5640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f893c071610 con 0x7f893c0b5af0 2024-04-24T23:05:16.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.141+0000 7f893bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f893c162e10 0x7f893c15ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:16.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.141+0000 7f893bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f893c162e10 0x7f893c15ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55900/0 (socket says 172.21.15.32:55900) 2024-04-24T23:05:16.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.141+0000 7f893bfff640 1 -- 172.21.15.32:0/3461956623 learned_addr learned my addr 172.21.15.32:0/3461956623 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:16.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.141+0000 7f893b7fe640 1 --1- 172.21.15.32:0/3461956623 >> v1:172.21.15.32:6789/0 conn(0x7f893c162320 0x7f893c162700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45224/0 (socket says 172.21.15.32:45224) 2024-04-24T23:05:16.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.141+0000 7f893bfff640 1 -- 172.21.15.32:0/3461956623 >> v2:172.21.15.121:3300/0 conn(0x7f893c0b5af0 msgr2=0x7f893c0ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:16.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.141+0000 7f893bfff640 1 --2- 172.21.15.32:0/3461956623 >> v2:172.21.15.121:3300/0 conn(0x7f893c0b5af0 0x7f893c0ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:16.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.141+0000 7f893bfff640 1 -- 172.21.15.32:0/3461956623 >> v1:172.21.15.32:6789/0 conn(0x7f893c162320 legacy=0x7f893c162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:16.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.141+0000 7f893bfff640 1 -- 172.21.15.32:0/3461956623 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f893c0718f0 con 0x7f893c162e10 2024-04-24T23:05:16.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.141+0000 7f893a7fc640 1 -- 172.21.15.32:0/3461956623 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3971492278 0 0) 0x7f893c0cd270 con 0x7f893c162320 2024-04-24T23:05:16.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.142+0000 7f893bfff640 1 --2- 172.21.15.32:0/3461956623 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f893c162e10 0x7f893c15ed30 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f893c0af8c0 tx=0x7f893402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bfe60cc83116173 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:16.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.142+0000 7f893a7fc640 1 -- 172.21.15.32:0/3461956623 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8934036070 con 0x7f893c162e10 2024-04-24T23:05:16.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.142+0000 7f893a7fc640 1 -- 172.21.15.32:0/3461956623 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8934031600 con 0x7f893c162e10 2024-04-24T23:05:16.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.142+0000 7f893a7fc640 1 -- 172.21.15.32:0/3461956623 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89340319d0 con 0x7f893c162e10 2024-04-24T23:05:16.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.143+0000 7f8941ca5640 1 -- 172.21.15.32:0/3461956623 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f893c162e10 msgr2=0x7f893c15ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:16.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.143+0000 7f8941ca5640 1 --2- 172.21.15.32:0/3461956623 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f893c162e10 0x7f893c15ed30 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f893c0af8c0 tx=0x7f893402a6a0 comp rx=0 tx=0).stop 2024-04-24T23:05:16.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.143+0000 7f8941ca5640 1 -- 172.21.15.32:0/3461956623 shutdown_connections 2024-04-24T23:05:16.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.143+0000 7f8941ca5640 1 --2- 172.21.15.32:0/3461956623 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f893c162e10 0x7f893c15ed30 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:16.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.143+0000 7f8941ca5640 1 --2- 172.21.15.32:0/3461956623 >> v2:172.21.15.121:3300/0 conn(0x7f893c0b5af0 0x7f893c0ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:16.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.143+0000 7f8941ca5640 1 -- 172.21.15.32:0/3461956623 >> 172.21.15.32:0/3461956623 conn(0x7f893c0d0ae0 msgr2=0x7f893c0d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:16.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.143+0000 7f8941ca5640 1 -- 172.21.15.32:0/3461956623 shutdown_connections 2024-04-24T23:05:16.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.143+0000 7f8941ca5640 1 -- 172.21.15.32:0/3461956623 wait complete. 2024-04-24T23:05:16.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.144+0000 7f8941ca5640 1 Processor -- start 2024-04-24T23:05:16.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.144+0000 7f8941ca5640 1 -- start start 2024-04-24T23:05:16.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.145+0000 7f8941ca5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f893c0b5af0 0x7f893c146430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:16.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.145+0000 7f8941ca5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f893c162320 0x7f893c15a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:16.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.145+0000 7f8941ca5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f893c162e10 0x7f893c15ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:16.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.145+0000 7f8941ca5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f893c071bd0 con 0x7f893c162e10 2024-04-24T23:05:16.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.145+0000 7f8941ca5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f893c067950 con 0x7f893c0b5af0 2024-04-24T23:05:16.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.145+0000 7f8941ca5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f893c071d50 con 0x7f893c162320 2024-04-24T23:05:16.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.145+0000 7f893bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f893c162e10 0x7f893c15ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:16.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.145+0000 7f893bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f893c162e10 0x7f893c15ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55908/0 (socket says 172.21.15.32:55908) 2024-04-24T23:05:16.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.145+0000 7f893bfff640 1 -- 172.21.15.32:0/1126833835 learned_addr learned my addr 172.21.15.32:0/1126833835 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:16.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.145+0000 7f893bfff640 1 -- 172.21.15.32:0/1126833835 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f893c162320 msgr2=0x7f893c15a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:16.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.145+0000 7f893bfff640 1 --2- 172.21.15.32:0/1126833835 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f893c162320 0x7f893c15a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:16.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.145+0000 7f893bfff640 1 -- 172.21.15.32:0/1126833835 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f893c0b5af0 msgr2=0x7f893c146430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:16.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.145+0000 7f893bfff640 1 --2- 172.21.15.32:0/1126833835 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f893c0b5af0 0x7f893c146430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:16.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.145+0000 7f893bfff640 1 -- 172.21.15.32:0/1126833835 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f893c1f4aa0 con 0x7f893c162e10 2024-04-24T23:05:16.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.145+0000 7f893bfff640 1 --2- 172.21.15.32:0/1126833835 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f893c162e10 0x7f893c15ab10 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f8934034040 tx=0x7f8934004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:16.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.146+0000 7f8923fff640 1 -- 172.21.15.32:0/1126833835 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8934041020 con 0x7f893c162e10 2024-04-24T23:05:16.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.146+0000 7f8923fff640 1 -- 172.21.15.32:0/1126833835 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89340077a0 con 0x7f893c162e10 2024-04-24T23:05:16.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.146+0000 7f8941ca5640 1 -- 172.21.15.32:0/1126833835 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f893c1f4d80 con 0x7f893c162e10 2024-04-24T23:05:16.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.146+0000 7f8923fff640 1 -- 172.21.15.32:0/1126833835 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8934036040 con 0x7f893c162e10 2024-04-24T23:05:16.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.146+0000 7f8941ca5640 1 -- 172.21.15.32:0/1126833835 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f893c1f5340 con 0x7f893c162e10 2024-04-24T23:05:16.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.147+0000 7f8941ca5640 1 -- 172.21.15.32:0/1126833835 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f893c0b38a0 con 0x7f893c162e10 2024-04-24T23:05:16.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.147+0000 7f8923fff640 1 -- 172.21.15.32:0/1126833835 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8934044020 con 0x7f893c162e10 2024-04-24T23:05:16.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.148+0000 7f8923fff640 1 --2- 172.21.15.32:0/1126833835 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f892806be30 0x7f892806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:16.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.148+0000 7f8923fff640 1 -- 172.21.15.32:0/1126833835 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5795+0+0 (secure 0 0 0) 0x7f89340abd20 con 0x7f893c162e10 2024-04-24T23:05:16.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.148+0000 7f893b7fe640 1 --2- 172.21.15.32:0/1126833835 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f892806be30 0x7f892806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:16.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.148+0000 7f893b7fe640 1 --2- 172.21.15.32:0/1126833835 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f892806be30 0x7f892806e2f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f892c007fd0 tx=0x7f892c007920 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:16.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.152+0000 7f8923fff640 1 -- 172.21.15.32:0/1126833835 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f893c0b38a0 con 0x7f893c162e10 2024-04-24T23:05:16.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:16.376+0000 7f8941ca5640 1 -- 172.21.15.32:0/1126833835 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f893c15f9a0 con 0x7f893c162e10 2024-04-24T23:05:17.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.023+0000 7f8923fff640 1 -- 172.21.15.32:0/1126833835 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v53) ==== 104+0+0 (secure 0 0 0) 0x7f893c15f9a0 con 0x7f893c162e10 2024-04-24T23:05:17.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.025+0000 7f8941ca5640 1 -- 172.21.15.32:0/1126833835 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f892806be30 msgr2=0x7f892806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:17.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.025+0000 7f8941ca5640 1 --2- 172.21.15.32:0/1126833835 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f892806be30 0x7f892806e2f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f892c007fd0 tx=0x7f892c007920 comp rx=0 tx=0).stop 2024-04-24T23:05:17.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.025+0000 7f8941ca5640 1 -- 172.21.15.32:0/1126833835 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f893c162e10 msgr2=0x7f893c15ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:17.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.025+0000 7f8941ca5640 1 --2- 172.21.15.32:0/1126833835 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f893c162e10 0x7f893c15ab10 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f8934034040 tx=0x7f8934004a40 comp rx=0 tx=0).stop 2024-04-24T23:05:17.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.025+0000 7f8941ca5640 1 -- 172.21.15.32:0/1126833835 shutdown_connections 2024-04-24T23:05:17.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.025+0000 7f8941ca5640 1 --2- 172.21.15.32:0/1126833835 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f892806be30 0x7f892806e2f0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:17.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.025+0000 7f8941ca5640 1 --2- 172.21.15.32:0/1126833835 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f893c162e10 0x7f893c15ab10 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:17.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.025+0000 7f8941ca5640 1 --2- 172.21.15.32:0/1126833835 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f893c162320 0x7f893c15a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:17.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.025+0000 7f8941ca5640 1 --2- 172.21.15.32:0/1126833835 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f893c0b5af0 0x7f893c146430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:17.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.025+0000 7f8941ca5640 1 -- 172.21.15.32:0/1126833835 >> 172.21.15.32:0/1126833835 conn(0x7f893c0d0ae0 msgr2=0x7f893c0b0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:17.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.025+0000 7f8941ca5640 1 -- 172.21.15.32:0/1126833835 shutdown_connections 2024-04-24T23:05:17.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.026+0000 7f8941ca5640 1 -- 172.21.15.32:0/1126833835 wait complete. 2024-04-24T23:05:17.041 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-24T23:05:17.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.193+0000 7fd4a4770640 1 Processor -- start 2024-04-24T23:05:17.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.193+0000 7fd4a4770640 1 -- start start 2024-04-24T23:05:17.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.194+0000 7fd4a4770640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd49c0b5ff0 0x7fd49c0cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:17.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.194+0000 7fd4a4770640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd49c15bde0 con 0x7fd49c15b960 2024-04-24T23:05:17.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.194+0000 7fd4a4770640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd49c15c0c0 con 0x7fd49c15b580 2024-04-24T23:05:17.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.194+0000 7fd4a4770640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd49c15c9f0 con 0x7fd49c0b5ff0 2024-04-24T23:05:17.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.194+0000 7fd4a1d0b640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd49c15b960 0x7fd49c0b58e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45232/0 (socket says 172.21.15.32:45232) 2024-04-24T23:05:17.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.194+0000 7fd4a1d0b640 1 -- 172.21.15.32:0/3722017548 learned_addr learned my addr 172.21.15.32:0/3722017548 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:05:17.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.194+0000 7fd4a2d0d640 1 --2- 172.21.15.32:0/3722017548 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd49c0b5ff0 0x7fd49c0cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:17.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.194+0000 7fd4a150a640 1 -- 172.21.15.32:0/3722017548 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3950320033 0 0) 0x7fd49c15bde0 con 0x7fd49c15b960 2024-04-24T23:05:17.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.194+0000 7fd4a150a640 1 -- 172.21.15.32:0/3722017548 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd488003520 con 0x7fd49c15b960 2024-04-24T23:05:17.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.195+0000 7fd4a150a640 1 -- 172.21.15.32:0/3722017548 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2897236313 0 0) 0x7fd488003520 con 0x7fd49c15b960 2024-04-24T23:05:17.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.195+0000 7fd4a150a640 1 -- 172.21.15.32:0/3722017548 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd49c0b5ff0 msgr2=0x7fd49c0cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:17.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.195+0000 7fd4a150a640 1 --2- 172.21.15.32:0/3722017548 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd49c0b5ff0 0x7fd49c0cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:17.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.195+0000 7fd4a150a640 1 -- 172.21.15.32:0/3722017548 >> v1:172.21.15.121:6789/0 conn(0x7fd49c15b580 legacy=0x7fd49c06cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:17.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.195+0000 7fd4a150a640 1 -- 172.21.15.32:0/3722017548 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd49c15c3a0 con 0x7fd49c15b960 2024-04-24T23:05:17.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.195+0000 7fd4a150a640 1 -- 172.21.15.32:0/3722017548 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7fd48c001670 con 0x7fd49c15b960 2024-04-24T23:05:17.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.195+0000 7fd4a150a640 1 -- 172.21.15.32:0/3722017548 >> v1:172.21.15.32:6789/0 conn(0x7fd49c15b960 legacy=0x7fd49c0b58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:17.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.195+0000 7fd4a150a640 1 --2- 172.21.15.32:0/3722017548 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd49c0b5ff0 0x7fd4880039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:17.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.195+0000 7fd4a150a640 1 --2- 172.21.15.32:0/3722017548 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd488003f20 0x7fd488004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:17.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.195+0000 7fd4a150a640 1 --2- 172.21.15.32:0/3722017548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd488004d40 0x7fd4880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:17.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.195+0000 7fd4a150a640 1 -- 172.21.15.32:0/3722017548 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd49c15c9f0 con 0x7fd488004d40 2024-04-24T23:05:17.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.195+0000 7fd4a150a640 1 -- 172.21.15.32:0/3722017548 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd48800ba20 con 0x7fd488003f20 2024-04-24T23:05:17.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.195+0000 7fd4a150a640 1 -- 172.21.15.32:0/3722017548 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd48800bc20 con 0x7fd49c0b5ff0 2024-04-24T23:05:17.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.195+0000 7fd4a2d0d640 1 --2- 172.21.15.32:0/3722017548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd488004d40 0x7fd4880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:17.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.195+0000 7fd4a1d0b640 1 --2- 172.21.15.32:0/3722017548 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd488003f20 0x7fd488004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:17.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.195+0000 7fd4a250c640 1 --2- 172.21.15.32:0/3722017548 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd49c0b5ff0 0x7fd4880039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:17.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.195+0000 7fd4a2d0d640 1 -- 172.21.15.32:0/3722017548 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd49c0b5ff0 msgr2=0x7fd4880039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:17.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.195+0000 7fd4a2d0d640 1 --2- 172.21.15.32:0/3722017548 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd49c0b5ff0 0x7fd4880039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:17.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.195+0000 7fd4a2d0d640 1 -- 172.21.15.32:0/3722017548 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd488003f20 msgr2=0x7fd488004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:17.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.195+0000 7fd4a2d0d640 1 --2- 172.21.15.32:0/3722017548 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd488003f20 0x7fd488004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:17.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.195+0000 7fd4a2d0d640 1 -- 172.21.15.32:0/3722017548 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd488003520 con 0x7fd488004d40 2024-04-24T23:05:17.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.196+0000 7fd4a2d0d640 1 --2- 172.21.15.32:0/3722017548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd488004d40 0x7fd4880051c0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fd494009ac0 tx=0x7fd488005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5e3747ef8607811 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:17.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.196+0000 7fd4a150a640 1 -- 172.21.15.32:0/3722017548 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd494036070 con 0x7fd488004d40 2024-04-24T23:05:17.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.196+0000 7fd4a150a640 1 -- 172.21.15.32:0/3722017548 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd494031400 con 0x7fd488004d40 2024-04-24T23:05:17.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.196+0000 7fd4a150a640 1 -- 172.21.15.32:0/3722017548 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd494009690 con 0x7fd488004d40 2024-04-24T23:05:17.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.197+0000 7fd4a4770640 1 -- 172.21.15.32:0/3722017548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd488004d40 msgr2=0x7fd4880051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:17.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.197+0000 7fd4a4770640 1 --2- 172.21.15.32:0/3722017548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd488004d40 0x7fd4880051c0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fd494009ac0 tx=0x7fd488005830 comp rx=0 tx=0).stop 2024-04-24T23:05:17.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.197+0000 7fd4a2d0d640 1 -- 172.21.15.32:0/3722017548 reap_dead start 2024-04-24T23:05:17.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.197+0000 7fd4a4770640 1 -- 172.21.15.32:0/3722017548 shutdown_connections 2024-04-24T23:05:17.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.197+0000 7fd4a4770640 1 -- 172.21.15.32:0/3722017548 >> 172.21.15.32:0/3722017548 conn(0x7fd49c0b1f10 msgr2=0x7fd49c0b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:17.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.197+0000 7fd4a4770640 1 -- 172.21.15.32:0/3722017548 shutdown_connections 2024-04-24T23:05:17.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.197+0000 7fd4a4770640 1 -- 172.21.15.32:0/3722017548 wait complete. 2024-04-24T23:05:17.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.198+0000 7fd4a4770640 1 Processor -- start 2024-04-24T23:05:17.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.198+0000 7fd4a4770640 1 -- start start 2024-04-24T23:05:17.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.199+0000 7fd4a4770640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd488004d40 0x7fd49c14e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:17.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.199+0000 7fd4a4770640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd49c073170 0x7fd49c14cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:17.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.199+0000 7fd4a4770640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd49c14c170 0x7fd49c14ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:17.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.199+0000 7fd4a4770640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd49c06c920 con 0x7fd49c14c170 2024-04-24T23:05:17.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.199+0000 7fd4a4770640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd49c06c7a0 con 0x7fd49c073170 2024-04-24T23:05:17.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.199+0000 7fd4a4770640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd49c0cde80 con 0x7fd488004d40 2024-04-24T23:05:17.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.199+0000 7fd4a2d0d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd49c14c170 0x7fd49c14ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:17.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.199+0000 7fd4a2d0d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd49c14c170 0x7fd49c14ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55930/0 (socket says 172.21.15.32:55930) 2024-04-24T23:05:17.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.199+0000 7fd4a2d0d640 1 -- 172.21.15.32:0/607459335 learned_addr learned my addr 172.21.15.32:0/607459335 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:17.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.199+0000 7fd4a1d0b640 1 --2- 172.21.15.32:0/607459335 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd49c073170 0x7fd49c14cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:17.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.199+0000 7fd4a250c640 1 --2- 172.21.15.32:0/607459335 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd488004d40 0x7fd49c14e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:17.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.199+0000 7fd4a2d0d640 1 -- 172.21.15.32:0/607459335 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd488004d40 msgr2=0x7fd49c14e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:17.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.199+0000 7fd4a2d0d640 1 --2- 172.21.15.32:0/607459335 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd488004d40 0x7fd49c14e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:17.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.199+0000 7fd4a2d0d640 1 -- 172.21.15.32:0/607459335 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd49c073170 msgr2=0x7fd49c14cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:17.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.199+0000 7fd4a2d0d640 1 --2- 172.21.15.32:0/607459335 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd49c073170 0x7fd49c14cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:17.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.199+0000 7fd4a2d0d640 1 -- 172.21.15.32:0/607459335 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd49c14d2d0 con 0x7fd49c14c170 2024-04-24T23:05:17.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.199+0000 7fd4a250c640 1 --2- 172.21.15.32:0/607459335 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd488004d40 0x7fd49c14e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:05:17.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.199+0000 7fd4a1d0b640 1 --2- 172.21.15.32:0/607459335 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd49c073170 0x7fd49c14cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:05:17.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.199+0000 7fd4a2d0d640 1 --2- 172.21.15.32:0/607459335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd49c14c170 0x7fd49c14ebb0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fd494009ac0 tx=0x7fd494004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:17.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.199+0000 7fd492ffd640 1 -- 172.21.15.32:0/607459335 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd494036040 con 0x7fd49c14c170 2024-04-24T23:05:17.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.199+0000 7fd492ffd640 1 -- 172.21.15.32:0/607459335 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd494041070 con 0x7fd49c14c170 2024-04-24T23:05:17.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.199+0000 7fd4a4770640 1 -- 172.21.15.32:0/607459335 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd49c14d5b0 con 0x7fd49c14c170 2024-04-24T23:05:17.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.199+0000 7fd492ffd640 1 -- 172.21.15.32:0/607459335 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd494003a50 con 0x7fd49c14c170 2024-04-24T23:05:17.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.200+0000 7fd4a4770640 1 -- 172.21.15.32:0/607459335 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd49c14db70 con 0x7fd49c14c170 2024-04-24T23:05:17.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.201+0000 7fd4a4770640 1 -- 172.21.15.32:0/607459335 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd49c068430 con 0x7fd49c14c170 2024-04-24T23:05:17.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.201+0000 7fd492ffd640 1 -- 172.21.15.32:0/607459335 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd494003bf0 con 0x7fd49c14c170 2024-04-24T23:05:17.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.201+0000 7fd492ffd640 1 --2- 172.21.15.32:0/607459335 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd48006be60 0x7fd48006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:17.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.201+0000 7fd492ffd640 1 -- 172.21.15.32:0/607459335 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5795+0+0 (secure 0 0 0) 0x7fd4940aa250 con 0x7fd49c14c170 2024-04-24T23:05:17.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.202+0000 7fd4a250c640 1 --2- 172.21.15.32:0/607459335 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd48006be60 0x7fd48006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:17.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.202+0000 7fd4a250c640 1 --2- 172.21.15.32:0/607459335 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd48006be60 0x7fd48006e320 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fd498001bd0 tx=0x7fd498009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:17.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.206+0000 7fd492ffd640 1 -- 172.21.15.32:0/607459335 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd4940785e0 con 0x7fd49c14c170 2024-04-24T23:05:17.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:17.389+0000 7fd4a4770640 1 -- 172.21.15.32:0/607459335 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd49c0cc9d0 con 0x7fd49c14c170 2024-04-24T23:05:18.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.349+0000 7fd492ffd640 1 -- 172.21.15.32:0/607459335 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v130) ==== 181+0+0 (secure 0 0 0) 0x7fd49407c8e0 con 0x7fd49c14c170 2024-04-24T23:05:18.350 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2024-04-24T23:05:18.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.351+0000 7fd4a4770640 1 -- 172.21.15.32:0/607459335 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd48006be60 msgr2=0x7fd48006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:18.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.351+0000 7fd4a4770640 1 --2- 172.21.15.32:0/607459335 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd48006be60 0x7fd48006e320 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fd498001bd0 tx=0x7fd498009040 comp rx=0 tx=0).stop 2024-04-24T23:05:18.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.351+0000 7fd4a4770640 1 -- 172.21.15.32:0/607459335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd49c14c170 msgr2=0x7fd49c14ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:18.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.351+0000 7fd4a4770640 1 --2- 172.21.15.32:0/607459335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd49c14c170 0x7fd49c14ebb0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fd494009ac0 tx=0x7fd494004730 comp rx=0 tx=0).stop 2024-04-24T23:05:18.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.351+0000 7fd4a4770640 1 -- 172.21.15.32:0/607459335 shutdown_connections 2024-04-24T23:05:18.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.351+0000 7fd4a4770640 1 --2- 172.21.15.32:0/607459335 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd48006be60 0x7fd48006e320 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:18.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.351+0000 7fd4a4770640 1 --2- 172.21.15.32:0/607459335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd49c14c170 0x7fd49c14ebb0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:18.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.351+0000 7fd4a4770640 1 --2- 172.21.15.32:0/607459335 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd49c073170 0x7fd49c14cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:18.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.351+0000 7fd4a4770640 1 --2- 172.21.15.32:0/607459335 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd488004d40 0x7fd49c14e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:18.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.351+0000 7fd4a4770640 1 -- 172.21.15.32:0/607459335 >> 172.21.15.32:0/607459335 conn(0x7fd49c0b1f10 msgr2=0x7fd49c15f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:18.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.352+0000 7fd4a4770640 1 -- 172.21.15.32:0/607459335 shutdown_connections 2024-04-24T23:05:18.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.352+0000 7fd4a4770640 1 -- 172.21.15.32:0/607459335 wait complete. 2024-04-24T23:05:18.369 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-24T23:05:18.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.481+0000 7f2963909640 1 Processor -- start 2024-04-24T23:05:18.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.481+0000 7f2963909640 1 -- start start 2024-04-24T23:05:18.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.482+0000 7f2963909640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f295c0cd3c0 0x7f295c15e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:18.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.482+0000 7f2963909640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f295c15ec80 0x7f295c1513b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:18.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.482+0000 7f2963909640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f295c0cd850 con 0x7f295c0b57c0 2024-04-24T23:05:18.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.482+0000 7f2963909640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f295c15f0d0 con 0x7f295c15ec80 2024-04-24T23:05:18.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.482+0000 7f2963909640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f295c15f5c0 con 0x7f295c0cd3c0 2024-04-24T23:05:18.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.482+0000 7f2961ea6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f295c15ec80 0x7f295c1513b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:18.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.482+0000 7f2961ea6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f295c15ec80 0x7f295c1513b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55932/0 (socket says 172.21.15.32:55932) 2024-04-24T23:05:18.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.482+0000 7f2961ea6640 1 -- 172.21.15.32:0/4171936560 learned_addr learned my addr 172.21.15.32:0/4171936560 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:18.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.482+0000 7f2960ea4640 1 --2- 172.21.15.32:0/4171936560 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f295c0cd3c0 0x7f295c15e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:18.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.482+0000 7f2961ea6640 1 -- 172.21.15.32:0/4171936560 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f295c0cd3c0 msgr2=0x7f295c15e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:18.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.482+0000 7f2961ea6640 1 --2- 172.21.15.32:0/4171936560 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f295c0cd3c0 0x7f295c15e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:18.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.482+0000 7f2961ea6640 1 -- 172.21.15.32:0/4171936560 >> v1:172.21.15.121:6789/0 conn(0x7f295c0b57c0 legacy=0x7f295c0b5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:18.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.482+0000 7f2961ea6640 1 -- 172.21.15.32:0/4171936560 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f295c0cdb30 con 0x7f295c15ec80 2024-04-24T23:05:18.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.482+0000 7f2960ea4640 1 --2- 172.21.15.32:0/4171936560 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f295c0cd3c0 0x7f295c15e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:05:18.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.483+0000 7f2961ea6640 1 --2- 172.21.15.32:0/4171936560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f295c15ec80 0x7f295c1513b0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f295400d150 tx=0x7f295402c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eb79934f4791b8a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:18.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.483+0000 7f2953fff640 1 -- 172.21.15.32:0/4171936560 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29540337a0 con 0x7f295c15ec80 2024-04-24T23:05:18.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.483+0000 7f2953fff640 1 -- 172.21.15.32:0/4171936560 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2954039070 con 0x7f295c15ec80 2024-04-24T23:05:18.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.483+0000 7f2953fff640 1 -- 172.21.15.32:0/4171936560 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2954033b60 con 0x7f295c15ec80 2024-04-24T23:05:18.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.483+0000 7f2963909640 1 -- 172.21.15.32:0/4171936560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f295c15ec80 msgr2=0x7f295c1513b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:18.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.483+0000 7f2963909640 1 --2- 172.21.15.32:0/4171936560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f295c15ec80 0x7f295c1513b0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f295400d150 tx=0x7f295402c930 comp rx=0 tx=0).stop 2024-04-24T23:05:18.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.483+0000 7f2963909640 1 -- 172.21.15.32:0/4171936560 shutdown_connections 2024-04-24T23:05:18.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.484+0000 7f2963909640 1 --2- 172.21.15.32:0/4171936560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f295c15ec80 0x7f295c1513b0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:18.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.484+0000 7f2963909640 1 --2- 172.21.15.32:0/4171936560 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f295c0cd3c0 0x7f295c15e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:18.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.484+0000 7f2963909640 1 -- 172.21.15.32:0/4171936560 >> 172.21.15.32:0/4171936560 conn(0x7f295c0b1f10 msgr2=0x7f295c0b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:18.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.484+0000 7f2963909640 1 -- 172.21.15.32:0/4171936560 shutdown_connections 2024-04-24T23:05:18.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.484+0000 7f2963909640 1 -- 172.21.15.32:0/4171936560 wait complete. 2024-04-24T23:05:18.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.485+0000 7f2963909640 1 Processor -- start 2024-04-24T23:05:18.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.485+0000 7f2963909640 1 -- start start 2024-04-24T23:05:18.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.485+0000 7f2963909640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f295c0b57c0 0x7f295c1f00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:18.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.485+0000 7f2963909640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f295c15ec80 0x7f295c1f0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:18.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.485+0000 7f2963909640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f295c1f36f0 0x7f295c1f3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:18.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.485+0000 7f2963909640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f295c0ce3d0 con 0x7f295c0b57c0 2024-04-24T23:05:18.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.485+0000 7f2963909640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f295c0ce0f0 con 0x7f295c15ec80 2024-04-24T23:05:18.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.485+0000 7f2963909640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f295c0b62b0 con 0x7f295c1f36f0 2024-04-24T23:05:18.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.486+0000 7f29616a5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f295c0b57c0 0x7f295c1f00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:18.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.486+0000 7f29616a5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f295c0b57c0 0x7f295c1f00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55940/0 (socket says 172.21.15.32:55940) 2024-04-24T23:05:18.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.486+0000 7f29616a5640 1 -- 172.21.15.32:0/2870802766 learned_addr learned my addr 172.21.15.32:0/2870802766 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:18.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.486+0000 7f2960ea4640 1 --2- 172.21.15.32:0/2870802766 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f295c15ec80 0x7f295c1f0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:18.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.486+0000 7f2961ea6640 1 --2- 172.21.15.32:0/2870802766 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f295c1f36f0 0x7f295c1f3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:18.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.486+0000 7f29616a5640 1 -- 172.21.15.32:0/2870802766 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f295c1f36f0 msgr2=0x7f295c1f3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:18.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.486+0000 7f29616a5640 1 --2- 172.21.15.32:0/2870802766 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f295c1f36f0 0x7f295c1f3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:18.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.486+0000 7f29616a5640 1 -- 172.21.15.32:0/2870802766 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f295c15ec80 msgr2=0x7f295c1f0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:18.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.486+0000 7f29616a5640 1 --2- 172.21.15.32:0/2870802766 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f295c15ec80 0x7f295c1f0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:18.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.486+0000 7f29616a5640 1 -- 172.21.15.32:0/2870802766 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f295c14e140 con 0x7f295c0b57c0 2024-04-24T23:05:18.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.486+0000 7f29616a5640 1 --2- 172.21.15.32:0/2870802766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f295c0b57c0 0x7f295c1f00f0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f295800c400 tx=0x7f295800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:18.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.486+0000 7f2951ffb640 1 -- 172.21.15.32:0/2870802766 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29580052d0 con 0x7f295c0b57c0 2024-04-24T23:05:18.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.486+0000 7f2951ffb640 1 -- 172.21.15.32:0/2870802766 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2958005490 con 0x7f295c0b57c0 2024-04-24T23:05:18.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.486+0000 7f2963909640 1 -- 172.21.15.32:0/2870802766 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f295c15df00 con 0x7f295c0b57c0 2024-04-24T23:05:18.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.486+0000 7f2951ffb640 1 -- 172.21.15.32:0/2870802766 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2958007260 con 0x7f295c0b57c0 2024-04-24T23:05:18.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.487+0000 7f2963909640 1 -- 172.21.15.32:0/2870802766 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f295c1f1510 con 0x7f295c0b57c0 2024-04-24T23:05:18.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.488+0000 7f2963909640 1 -- 172.21.15.32:0/2870802766 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f295c068430 con 0x7f295c0b57c0 2024-04-24T23:05:18.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.488+0000 7f2951ffb640 1 -- 172.21.15.32:0/2870802766 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2958007400 con 0x7f295c0b57c0 2024-04-24T23:05:18.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.488+0000 7f2951ffb640 1 --2- 172.21.15.32:0/2870802766 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f294006be30 0x7f294006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:18.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.489+0000 7f2951ffb640 1 -- 172.21.15.32:0/2870802766 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5357+0+0 (secure 0 0 0) 0x7f295808a100 con 0x7f295c0b57c0 2024-04-24T23:05:18.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.489+0000 7f2960ea4640 1 --2- 172.21.15.32:0/2870802766 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f294006be30 0x7f294006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:18.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.489+0000 7f2960ea4640 1 --2- 172.21.15.32:0/2870802766 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f294006be30 0x7f294006e2f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f295c073480 tx=0x7f294c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:18.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.493+0000 7f2951ffb640 1 -- 172.21.15.32:0/2870802766 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2958058650 con 0x7f295c0b57c0 2024-04-24T23:05:18.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:18.676+0000 7f2963909640 1 -- 172.21.15.32:0/2870802766 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f295c15a230 con 0x7f295c0b57c0 2024-04-24T23:05:19.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.353+0000 7f2951ffb640 1 -- 172.21.15.32:0/2870802766 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v131) ==== 169+0+0 (secure 0 0 0) 0x7f295805c950 con 0x7f295c0b57c0 2024-04-24T23:05:19.355 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2024-04-24T23:05:19.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.356+0000 7f2963909640 1 -- 172.21.15.32:0/2870802766 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f294006be30 msgr2=0x7f294006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:19.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.356+0000 7f2963909640 1 --2- 172.21.15.32:0/2870802766 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f294006be30 0x7f294006e2f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f295c073480 tx=0x7f294c008040 comp rx=0 tx=0).stop 2024-04-24T23:05:19.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.356+0000 7f2963909640 1 -- 172.21.15.32:0/2870802766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f295c0b57c0 msgr2=0x7f295c1f00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:19.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.356+0000 7f2963909640 1 --2- 172.21.15.32:0/2870802766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f295c0b57c0 0x7f295c1f00f0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f295800c400 tx=0x7f295800c8c0 comp rx=0 tx=0).stop 2024-04-24T23:05:19.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.356+0000 7f2963909640 1 -- 172.21.15.32:0/2870802766 shutdown_connections 2024-04-24T23:05:19.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.356+0000 7f2963909640 1 --2- 172.21.15.32:0/2870802766 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f294006be30 0x7f294006e2f0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:19.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.356+0000 7f2963909640 1 --2- 172.21.15.32:0/2870802766 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f295c1f36f0 0x7f295c1f3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:19.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.356+0000 7f2963909640 1 --2- 172.21.15.32:0/2870802766 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f295c15ec80 0x7f295c1f0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:19.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.356+0000 7f2963909640 1 --2- 172.21.15.32:0/2870802766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f295c0b57c0 0x7f295c1f00f0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:19.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.356+0000 7f2963909640 1 -- 172.21.15.32:0/2870802766 >> 172.21.15.32:0/2870802766 conn(0x7f295c0b1f10 msgr2=0x7f295c0b5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:19.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.356+0000 7f2963909640 1 -- 172.21.15.32:0/2870802766 shutdown_connections 2024-04-24T23:05:19.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.356+0000 7f2963909640 1 -- 172.21.15.32:0/2870802766 wait complete. 2024-04-24T23:05:19.371 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-24T23:05:19.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.479+0000 7f0fe65b7640 1 Processor -- start 2024-04-24T23:05:19.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.479+0000 7f0fe65b7640 1 -- start start 2024-04-24T23:05:19.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.480+0000 7f0fe65b7640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fe006b820 0x7f0fe00d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:19.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.480+0000 7f0fe65b7640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0fe00d4210 0x7f0fe00b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:19.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.480+0000 7f0fe65b7640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fe00d3170 con 0x7f0fe015e9b0 2024-04-24T23:05:19.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.480+0000 7f0fe65b7640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f0fe00d3450 con 0x7f0fe00d4210 2024-04-24T23:05:19.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.480+0000 7f0fe65b7640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f0fe00d3730 con 0x7f0fe006b820 2024-04-24T23:05:19.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.480+0000 7f0fe4b54640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0fe00d4210 0x7f0fe00b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:19.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.480+0000 7f0fdffff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0fe015e9b0 0x7f0fe006b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45248/0 (socket says 172.21.15.32:45248) 2024-04-24T23:05:19.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.480+0000 7f0fe4b54640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0fe00d4210 0x7f0fe00b78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55946/0 (socket says 172.21.15.32:55946) 2024-04-24T23:05:19.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.480+0000 7f0fdffff640 1 -- 172.21.15.32:0/951931345 learned_addr learned my addr 172.21.15.32:0/951931345 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:05:19.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.481+0000 7f0fe4b54640 1 -- 172.21.15.32:0/951931345 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fe006b820 msgr2=0x7f0fe00d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:19.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.481+0000 7f0fe4b54640 1 --2- 172.21.15.32:0/951931345 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fe006b820 0x7f0fe00d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:19.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.481+0000 7f0fdeffd640 1 -- 172.21.15.32:0/951931345 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2479610096 0 0) 0x7f0fe00d3170 con 0x7f0fe015e9b0 2024-04-24T23:05:19.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.481+0000 7f0fe4b54640 1 -- 172.21.15.32:0/951931345 >> v1:172.21.15.32:6789/0 conn(0x7f0fe015e9b0 legacy=0x7f0fe006b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:19.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.481+0000 7f0fe4b54640 1 -- 172.21.15.32:0/951931345 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fe00d3a10 con 0x7f0fe00d4210 2024-04-24T23:05:19.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.481+0000 7f0fe4b54640 1 --2- 172.21.15.32:0/951931345 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fe00d4210 0x7f0fe00b78f0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f0fd8009d90 tx=0x7f0fd802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b7d05344bce710f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:19.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.482+0000 7f0fdeffd640 1 -- 172.21.15.32:0/951931345 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd8007740 con 0x7f0fe00d4210 2024-04-24T23:05:19.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.482+0000 7f0fdeffd640 1 -- 172.21.15.32:0/951931345 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fd80078e0 con 0x7f0fe00d4210 2024-04-24T23:05:19.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.482+0000 7f0fdeffd640 1 -- 172.21.15.32:0/951931345 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd8007cb0 con 0x7f0fe00d4210 2024-04-24T23:05:19.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.482+0000 7f0fe65b7640 1 -- 172.21.15.32:0/951931345 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fe00d4210 msgr2=0x7f0fe00b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:19.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.482+0000 7f0fe65b7640 1 --2- 172.21.15.32:0/951931345 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fe00d4210 0x7f0fe00b78f0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f0fd8009d90 tx=0x7f0fd802a660 comp rx=0 tx=0).stop 2024-04-24T23:05:19.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.482+0000 7f0fe65b7640 1 -- 172.21.15.32:0/951931345 shutdown_connections 2024-04-24T23:05:19.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.482+0000 7f0fe65b7640 1 --2- 172.21.15.32:0/951931345 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fe00d4210 0x7f0fe00b78f0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:19.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.482+0000 7f0fe65b7640 1 --2- 172.21.15.32:0/951931345 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fe006b820 0x7f0fe00d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:19.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.482+0000 7f0fe65b7640 1 -- 172.21.15.32:0/951931345 >> 172.21.15.32:0/951931345 conn(0x7f0fe00d11d0 msgr2=0x7f0fe00d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:19.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.482+0000 7f0fe65b7640 1 -- 172.21.15.32:0/951931345 shutdown_connections 2024-04-24T23:05:19.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.482+0000 7f0fe65b7640 1 -- 172.21.15.32:0/951931345 wait complete. 2024-04-24T23:05:19.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.483+0000 7f0fe65b7640 1 Processor -- start 2024-04-24T23:05:19.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.484+0000 7f0fe65b7640 1 -- start start 2024-04-24T23:05:19.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.484+0000 7f0fe65b7640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fe006b820 0x7f0fe014ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:19.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.484+0000 7f0fe65b7640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fe00d4210 0x7f0fe00cd400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:19.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.484+0000 7f0fe65b7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fe015e9b0 0x7f0fe00ce160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:19.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.484+0000 7f0fe65b7640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe00ba0e0 con 0x7f0fe015e9b0 2024-04-24T23:05:19.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.484+0000 7f0fe65b7640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe00d2400 con 0x7f0fe00d4210 2024-04-24T23:05:19.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.484+0000 7f0fe65b7640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f0fe00ba260 con 0x7f0fe006b820 2024-04-24T23:05:19.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.484+0000 7f0fe4b54640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fe015e9b0 0x7f0fe00ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:19.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.484+0000 7f0fe4b54640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fe015e9b0 0x7f0fe00ce160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55952/0 (socket says 172.21.15.32:55952) 2024-04-24T23:05:19.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.484+0000 7f0fe4b54640 1 -- 172.21.15.32:0/2389385654 learned_addr learned my addr 172.21.15.32:0/2389385654 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:19.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.485+0000 7f0fdffff640 1 --2- 172.21.15.32:0/2389385654 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fe006b820 0x7f0fe014ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:19.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.485+0000 7f0fe4b54640 1 -- 172.21.15.32:0/2389385654 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fe006b820 msgr2=0x7f0fe014ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:19.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.485+0000 7f0fdf7fe640 1 --2- 172.21.15.32:0/2389385654 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fe00d4210 0x7f0fe00cd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:19.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.485+0000 7f0fe4b54640 1 --2- 172.21.15.32:0/2389385654 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fe006b820 0x7f0fe014ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:19.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.485+0000 7f0fe4b54640 1 -- 172.21.15.32:0/2389385654 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fe00d4210 msgr2=0x7f0fe00cd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:19.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.485+0000 7f0fe4b54640 1 --2- 172.21.15.32:0/2389385654 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fe00d4210 0x7f0fe00cd400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:19.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.485+0000 7f0fe4b54640 1 -- 172.21.15.32:0/2389385654 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fe00cc360 con 0x7f0fe015e9b0 2024-04-24T23:05:19.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.485+0000 7f0fdffff640 1 --2- 172.21.15.32:0/2389385654 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fe006b820 0x7f0fe014ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:05:19.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.485+0000 7f0fdf7fe640 1 --2- 172.21.15.32:0/2389385654 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fe00d4210 0x7f0fe00cd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:05:19.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.485+0000 7f0fe4b54640 1 --2- 172.21.15.32:0/2389385654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fe015e9b0 0x7f0fe00ce160 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f0fd802ac50 tx=0x7f0fd8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:19.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.485+0000 7f0fdcff9640 1 -- 172.21.15.32:0/2389385654 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd8040070 con 0x7f0fe015e9b0 2024-04-24T23:05:19.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.485+0000 7f0fdcff9640 1 -- 172.21.15.32:0/2389385654 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fd8031c90 con 0x7f0fe015e9b0 2024-04-24T23:05:19.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.485+0000 7f0fe65b7640 1 -- 172.21.15.32:0/2389385654 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fe00cc640 con 0x7f0fe015e9b0 2024-04-24T23:05:19.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.485+0000 7f0fdcff9640 1 -- 172.21.15.32:0/2389385654 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd8045680 con 0x7f0fe015e9b0 2024-04-24T23:05:19.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.485+0000 7f0fe65b7640 1 -- 172.21.15.32:0/2389385654 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fe00ccc00 con 0x7f0fe015e9b0 2024-04-24T23:05:19.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.486+0000 7f0fe65b7640 1 -- 172.21.15.32:0/2389385654 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fe0068430 con 0x7f0fe015e9b0 2024-04-24T23:05:19.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.487+0000 7f0fdcff9640 1 -- 172.21.15.32:0/2389385654 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0fd8039040 con 0x7f0fe015e9b0 2024-04-24T23:05:19.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.487+0000 7f0fdcff9640 1 --2- 172.21.15.32:0/2389385654 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0fc006be30 0x7f0fc006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:19.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.487+0000 7f0fdcff9640 1 -- 172.21.15.32:0/2389385654 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 4943+0+0 (secure 0 0 0) 0x7f0fd80ac570 con 0x7f0fe015e9b0 2024-04-24T23:05:19.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.488+0000 7f0fdffff640 1 --2- 172.21.15.32:0/2389385654 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0fc006be30 0x7f0fc006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:19.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.488+0000 7f0fdffff640 1 --2- 172.21.15.32:0/2389385654 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0fc006be30 0x7f0fc006e2f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f0fd0000f60 tx=0x7f0fd0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:19.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.492+0000 7f0fdcff9640 1 -- 172.21.15.32:0/2389385654 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0fd8043360 con 0x7f0fe015e9b0 2024-04-24T23:05:19.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.681+0000 7f0fe65b7640 1 -- 172.21.15.32:0/2389385654 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f0fe00d0f90 con 0x7f0fe015e9b0 2024-04-24T23:05:19.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.685+0000 7f0fdcff9640 1 -- 172.21.15.32:0/2389385654 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) ==== 104+0+0 (secure 0 0 0) 0x7f0fd80810a0 con 0x7f0fe015e9b0 2024-04-24T23:05:19.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.687+0000 7f0fe65b7640 1 -- 172.21.15.32:0/2389385654 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0fc006be30 msgr2=0x7f0fc006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:19.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.687+0000 7f0fe65b7640 1 --2- 172.21.15.32:0/2389385654 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0fc006be30 0x7f0fc006e2f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f0fd0000f60 tx=0x7f0fd0009040 comp rx=0 tx=0).stop 2024-04-24T23:05:19.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.687+0000 7f0fe65b7640 1 -- 172.21.15.32:0/2389385654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fe015e9b0 msgr2=0x7f0fe00ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:19.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.687+0000 7f0fe65b7640 1 --2- 172.21.15.32:0/2389385654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fe015e9b0 0x7f0fe00ce160 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f0fd802ac50 tx=0x7f0fd8004a40 comp rx=0 tx=0).stop 2024-04-24T23:05:19.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.687+0000 7f0fe65b7640 1 -- 172.21.15.32:0/2389385654 shutdown_connections 2024-04-24T23:05:19.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.687+0000 7f0fe65b7640 1 --2- 172.21.15.32:0/2389385654 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0fc006be30 0x7f0fc006e2f0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:19.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.687+0000 7f0fe65b7640 1 --2- 172.21.15.32:0/2389385654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fe015e9b0 0x7f0fe00ce160 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:19.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.687+0000 7f0fe65b7640 1 --2- 172.21.15.32:0/2389385654 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fe00d4210 0x7f0fe00cd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:19.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.687+0000 7f0fe65b7640 1 --2- 172.21.15.32:0/2389385654 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fe006b820 0x7f0fe014ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:19.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.687+0000 7f0fe65b7640 1 -- 172.21.15.32:0/2389385654 >> 172.21.15.32:0/2389385654 conn(0x7f0fe00d11d0 msgr2=0x7f0fe0071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:19.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.687+0000 7f0fe65b7640 1 -- 172.21.15.32:0/2389385654 shutdown_connections 2024-04-24T23:05:19.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.687+0000 7f0fe65b7640 1 -- 172.21.15.32:0/2389385654 wait complete. 2024-04-24T23:05:19.699 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_barrier' 2024-04-24T23:05:19.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.801+0000 7f5038676640 1 Processor -- start 2024-04-24T23:05:19.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.801+0000 7f5038676640 1 -- start start 2024-04-24T23:05:19.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.802+0000 7f5038676640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f503015b480 0x7f50300b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:19.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.802+0000 7f5038676640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f503015fd60 0x7f503006c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:19.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.802+0000 7f5038676640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50300cef90 con 0x7f503006cb30 2024-04-24T23:05:19.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.802+0000 7f5038676640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f50300b4ff0 con 0x7f503015b480 2024-04-24T23:05:19.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.802+0000 7f5038676640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f50300b46c0 con 0x7f503015fd60 2024-04-24T23:05:19.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.802+0000 7f5035c11640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f503015fd60 0x7f503006c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:19.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.802+0000 7f5036c13640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f503006cb30 0x7f50300ccc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:39646/0 (socket says 172.21.15.32:39646) 2024-04-24T23:05:19.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.802+0000 7f5036412640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f503015b480 0x7f50300b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:19.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.802+0000 7f5036c13640 1 -- 172.21.15.32:0/3914150538 learned_addr learned my addr 172.21.15.32:0/3914150538 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:05:19.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.802+0000 7f5035c11640 1 -- 172.21.15.32:0/3914150538 >> v2:172.21.15.121:3300/0 conn(0x7f503015b480 msgr2=0x7f50300b5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:19.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.802+0000 7f5035c11640 1 --2- 172.21.15.32:0/3914150538 >> v2:172.21.15.121:3300/0 conn(0x7f503015b480 0x7f50300b5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:19.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.802+0000 7f5035c11640 1 -- 172.21.15.32:0/3914150538 >> v1:172.21.15.121:6789/0 conn(0x7f503006cb30 legacy=0x7f50300ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:19.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.802+0000 7f5035c11640 1 -- 172.21.15.32:0/3914150538 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50300b49a0 con 0x7f503015fd60 2024-04-24T23:05:19.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.802+0000 7f5036412640 1 --2- 172.21.15.32:0/3914150538 >> v2:172.21.15.121:3300/0 conn(0x7f503015b480 0x7f50300b5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:05:19.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.803+0000 7f5035c11640 1 --2- 172.21.15.32:0/3914150538 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f503015fd60 0x7f503006c5f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f50240097a0 tx=0x7f502402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53d12ed3c56f61db server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:19.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.803+0000 7f5035410640 1 -- 172.21.15.32:0/3914150538 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5024037070 con 0x7f503015fd60 2024-04-24T23:05:19.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.803+0000 7f5035410640 1 -- 172.21.15.32:0/3914150538 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5024032440 con 0x7f503015fd60 2024-04-24T23:05:19.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.803+0000 7f5035410640 1 -- 172.21.15.32:0/3914150538 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5024032810 con 0x7f503015fd60 2024-04-24T23:05:19.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.804+0000 7f5038676640 1 -- 172.21.15.32:0/3914150538 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f503015fd60 msgr2=0x7f503006c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:19.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.804+0000 7f5038676640 1 --2- 172.21.15.32:0/3914150538 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f503015fd60 0x7f503006c5f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f50240097a0 tx=0x7f502402a180 comp rx=0 tx=0).stop 2024-04-24T23:05:19.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.804+0000 7f5038676640 1 -- 172.21.15.32:0/3914150538 shutdown_connections 2024-04-24T23:05:19.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.804+0000 7f5038676640 1 --2- 172.21.15.32:0/3914150538 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f503015fd60 0x7f503006c5f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:19.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.804+0000 7f5038676640 1 --2- 172.21.15.32:0/3914150538 >> v2:172.21.15.121:3300/0 conn(0x7f503015b480 0x7f50300b5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:19.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.804+0000 7f5038676640 1 -- 172.21.15.32:0/3914150538 >> 172.21.15.32:0/3914150538 conn(0x7f503006b110 msgr2=0x7f503006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:19.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.804+0000 7f5038676640 1 -- 172.21.15.32:0/3914150538 shutdown_connections 2024-04-24T23:05:19.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.804+0000 7f5038676640 1 -- 172.21.15.32:0/3914150538 wait complete. 2024-04-24T23:05:19.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.805+0000 7f5038676640 1 Processor -- start 2024-04-24T23:05:19.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.805+0000 7f5038676640 1 -- start start 2024-04-24T23:05:19.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.806+0000 7f5038676640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f503006cb30 0x7f5030148d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:19.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.806+0000 7f5038676640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f503015b480 0x7f5030071800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:19.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.806+0000 7f5038676640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f503015fd60 0x7f503014c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:19.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.806+0000 7f5038676640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f50300b5480 con 0x7f503006cb30 2024-04-24T23:05:19.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.806+0000 7f5038676640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f50300b61a0 con 0x7f503015fd60 2024-04-24T23:05:19.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.806+0000 7f5038676640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f503015b860 con 0x7f503015b480 2024-04-24T23:05:19.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.806+0000 7f5036412640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f503006cb30 0x7f5030148d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:19.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.806+0000 7f5036412640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f503006cb30 0x7f5030148d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49692/0 (socket says 172.21.15.32:49692) 2024-04-24T23:05:19.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.806+0000 7f5036412640 1 -- 172.21.15.32:0/634941110 learned_addr learned my addr 172.21.15.32:0/634941110 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:19.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.806+0000 7f5035c11640 1 --2- 172.21.15.32:0/634941110 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f503015b480 0x7f5030071800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:19.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.806+0000 7f5036c13640 1 --2- 172.21.15.32:0/634941110 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f503015fd60 0x7f503014c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:19.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.806+0000 7f5036412640 1 -- 172.21.15.32:0/634941110 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f503015b480 msgr2=0x7f5030071800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:19.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.806+0000 7f5036412640 1 --2- 172.21.15.32:0/634941110 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f503015b480 0x7f5030071800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:19.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.806+0000 7f5036412640 1 -- 172.21.15.32:0/634941110 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f503015fd60 msgr2=0x7f503014c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:19.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.806+0000 7f5036412640 1 --2- 172.21.15.32:0/634941110 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f503015fd60 0x7f503014c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:19.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.806+0000 7f5036412640 1 -- 172.21.15.32:0/634941110 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5030149980 con 0x7f503006cb30 2024-04-24T23:05:19.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.806+0000 7f5036c13640 1 --2- 172.21.15.32:0/634941110 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f503015fd60 0x7f503014c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:05:19.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.807+0000 7f5036412640 1 --2- 172.21.15.32:0/634941110 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f503006cb30 0x7f5030148d10 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f502800ec10 tx=0x7f502800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:19.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.807+0000 7f502effd640 1 -- 172.21.15.32:0/634941110 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5028017070 con 0x7f503006cb30 2024-04-24T23:05:19.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.807+0000 7f502effd640 1 -- 172.21.15.32:0/634941110 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f502800cc30 con 0x7f503006cb30 2024-04-24T23:05:19.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.807+0000 7f502effd640 1 -- 172.21.15.32:0/634941110 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5028013470 con 0x7f503006cb30 2024-04-24T23:05:19.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.807+0000 7f5038676640 1 -- 172.21.15.32:0/634941110 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5030149c60 con 0x7f503006cb30 2024-04-24T23:05:19.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.807+0000 7f5038676640 1 -- 172.21.15.32:0/634941110 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f503014a220 con 0x7f503006cb30 2024-04-24T23:05:19.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.808+0000 7f5038676640 1 -- 172.21.15.32:0/634941110 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50300b1e30 con 0x7f503006cb30 2024-04-24T23:05:19.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.808+0000 7f502effd640 1 -- 172.21.15.32:0/634941110 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f502801a070 con 0x7f503006cb30 2024-04-24T23:05:19.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.809+0000 7f502effd640 1 --2- 172.21.15.32:0/634941110 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f501c06be60 0x7f501c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:19.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.809+0000 7f502effd640 1 -- 172.21.15.32:0/634941110 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 4943+0+0 (secure 0 0 0) 0x7f502808ad70 con 0x7f503006cb30 2024-04-24T23:05:19.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.809+0000 7f5035c11640 1 --2- 172.21.15.32:0/634941110 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f501c06be60 0x7f501c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:19.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.809+0000 7f5035c11640 1 --2- 172.21.15.32:0/634941110 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f501c06be60 0x7f501c06e320 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5024034000 tx=0x7f502402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:19.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:19.813+0000 7f502effd640 1 -- 172.21.15.32:0/634941110 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f50300b1e30 con 0x7f503006cb30 2024-04-24T23:05:20.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.065+0000 7f5038676640 1 -- 172.21.15.32:0/634941110 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]} v 0) -- 0x7f503006c950 con 0x7f503006cb30 2024-04-24T23:05:20.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.355+0000 7f502effd640 1 -- 172.21.15.32:0/634941110 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]}]=0 v332) ==== 128+0+0 (secure 0 0 0) 0x7f503006c950 con 0x7f503006cb30 2024-04-24T23:05:20.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.357+0000 7f5038676640 1 -- 172.21.15.32:0/634941110 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f501c06be60 msgr2=0x7f501c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:20.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.357+0000 7f5038676640 1 --2- 172.21.15.32:0/634941110 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f501c06be60 0x7f501c06e320 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5024034000 tx=0x7f502402e000 comp rx=0 tx=0).stop 2024-04-24T23:05:20.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.357+0000 7f5038676640 1 -- 172.21.15.32:0/634941110 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f503006cb30 msgr2=0x7f5030148d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:20.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.357+0000 7f5038676640 1 --2- 172.21.15.32:0/634941110 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f503006cb30 0x7f5030148d10 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f502800ec10 tx=0x7f502800c520 comp rx=0 tx=0).stop 2024-04-24T23:05:20.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.357+0000 7f5038676640 1 -- 172.21.15.32:0/634941110 shutdown_connections 2024-04-24T23:05:20.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.357+0000 7f5038676640 1 --2- 172.21.15.32:0/634941110 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f501c06be60 0x7f501c06e320 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:20.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.357+0000 7f5038676640 1 --2- 172.21.15.32:0/634941110 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f503015fd60 0x7f503014c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:20.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.357+0000 7f5038676640 1 --2- 172.21.15.32:0/634941110 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f503015b480 0x7f5030071800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:20.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.357+0000 7f5038676640 1 --2- 172.21.15.32:0/634941110 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f503006cb30 0x7f5030148d10 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:20.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.357+0000 7f5038676640 1 -- 172.21.15.32:0/634941110 >> 172.21.15.32:0/634941110 conn(0x7f503006b110 msgr2=0x7f50300cce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:20.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.357+0000 7f5038676640 1 -- 172.21.15.32:0/634941110 shutdown_connections 2024-04-24T23:05:20.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.357+0000 7f5038676640 1 -- 172.21.15.32:0/634941110 wait complete. 2024-04-24T23:05:20.370 INFO:tasks.cephfs_test_runner:That when an OSD epoch barrier is set on an MDS, subsequently ... ok 2024-04-24T23:05:20.370 INFO:tasks.cephfs_test_runner:Starting test: test_full_different_file (tasks.cephfs.test_full.TestQuotaFull) 2024-04-24T23:05:20.370 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:05:20.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.470+0000 7f1f61f09640 1 Processor -- start 2024-04-24T23:05:20.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.470+0000 7f1f61f09640 1 -- start start 2024-04-24T23:05:20.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.471+0000 7f1f61f09640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1f5c15e9b0 0x7f1f5c0d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:20.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.471+0000 7f1f61f09640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1f5c0d4210 0x7f1f5c06b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:20.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.471+0000 7f1f61f09640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f5c0d3170 con 0x7f1f5c0ba0e0 2024-04-24T23:05:20.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.471+0000 7f1f61f09640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f1f5c0b8900 con 0x7f1f5c0d4210 2024-04-24T23:05:20.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.471+0000 7f1f61f09640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f1f5c0d3450 con 0x7f1f5c15e9b0 2024-04-24T23:05:20.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.471+0000 7f1f5bfff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1f5c0ba0e0 0x7f1f5c071730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53130/0 (socket says 172.21.15.32:53130) 2024-04-24T23:05:20.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.471+0000 7f1f5bfff640 1 -- 172.21.15.32:0/2841765913 learned_addr learned my addr 172.21.15.32:0/2841765913 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:05:20.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.471+0000 7f1f5b7fe640 1 --2- 172.21.15.32:0/2841765913 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1f5c15e9b0 0x7f1f5c0d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:20.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.472+0000 7f1f5affd640 1 --2- 172.21.15.32:0/2841765913 >> v2:172.21.15.121:3300/0 conn(0x7f1f5c0d4210 0x7f1f5c06b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:20.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.472+0000 7f1f5a7fc640 1 -- 172.21.15.32:0/2841765913 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3184665939 0 0) 0x7f1f5c0d3170 con 0x7f1f5c0ba0e0 2024-04-24T23:05:20.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.472+0000 7f1f5a7fc640 1 -- 172.21.15.32:0/2841765913 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f44003520 con 0x7f1f5c0ba0e0 2024-04-24T23:05:20.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.472+0000 7f1f5a7fc640 1 -- 172.21.15.32:0/2841765913 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1419199301 0 0) 0x7f1f44003520 con 0x7f1f5c0ba0e0 2024-04-24T23:05:20.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.472+0000 7f1f5a7fc640 1 -- 172.21.15.32:0/2841765913 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1f5c15e9b0 msgr2=0x7f1f5c0d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:20.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.472+0000 7f1f5a7fc640 1 --2- 172.21.15.32:0/2841765913 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1f5c15e9b0 0x7f1f5c0d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:20.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.472+0000 7f1f5a7fc640 1 -- 172.21.15.32:0/2841765913 >> v2:172.21.15.121:3300/0 conn(0x7f1f5c0d4210 msgr2=0x7f1f5c06b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:20.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.472+0000 7f1f5a7fc640 1 --2- 172.21.15.32:0/2841765913 >> v2:172.21.15.121:3300/0 conn(0x7f1f5c0d4210 0x7f1f5c06b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:20.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.472+0000 7f1f5a7fc640 1 -- 172.21.15.32:0/2841765913 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f5c0d3730 con 0x7f1f5c0ba0e0 2024-04-24T23:05:20.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.472+0000 7f1f5a7fc640 1 -- 172.21.15.32:0/2841765913 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f1f540019f0 con 0x7f1f5c0ba0e0 2024-04-24T23:05:20.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.472+0000 7f1f5b7fe640 1 --2- 172.21.15.32:0/2841765913 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1f5c15e9b0 0x7f1f5c0d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:05:20.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.472+0000 7f1f5a7fc640 1 -- 172.21.15.32:0/2841765913 >> v1:172.21.15.32:6789/0 conn(0x7f1f5c0ba0e0 legacy=0x7f1f5c071730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:20.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.472+0000 7f1f5a7fc640 1 --2- 172.21.15.32:0/2841765913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f440039b0 0x7f1f44003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:20.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.472+0000 7f1f5a7fc640 1 --2- 172.21.15.32:0/2841765913 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f44004800 0x7f1f44004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:20.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.472+0000 7f1f5a7fc640 1 --2- 172.21.15.32:0/2841765913 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1f5c15e9b0 0x7f1f440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:20.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.472+0000 7f1f5a7fc640 1 -- 172.21.15.32:0/2841765913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5c0b8900 con 0x7f1f440039b0 2024-04-24T23:05:20.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.472+0000 7f1f5a7fc640 1 -- 172.21.15.32:0/2841765913 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5c0d3450 con 0x7f1f44004800 2024-04-24T23:05:20.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.472+0000 7f1f5a7fc640 1 -- 172.21.15.32:0/2841765913 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f1f4400b9c0 con 0x7f1f5c15e9b0 2024-04-24T23:05:20.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.472+0000 7f1f5affd640 1 --2- 172.21.15.32:0/2841765913 >> v2:172.21.15.121:3300/0 conn(0x7f1f5c0d4210 0x7f1f5c06b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:05:20.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.472+0000 7f1f5b7fe640 1 --2- 172.21.15.32:0/2841765913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f440039b0 0x7f1f44003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:20.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.472+0000 7f1f5bfff640 1 --2- 172.21.15.32:0/2841765913 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1f5c15e9b0 0x7f1f440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:20.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.472+0000 7f1f5affd640 1 --2- 172.21.15.32:0/2841765913 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f44004800 0x7f1f44004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:20.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.472+0000 7f1f5b7fe640 1 -- 172.21.15.32:0/2841765913 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1f5c15e9b0 msgr2=0x7f1f440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:20.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.472+0000 7f1f5b7fe640 1 --2- 172.21.15.32:0/2841765913 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1f5c15e9b0 0x7f1f440051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:20.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.472+0000 7f1f5b7fe640 1 -- 172.21.15.32:0/2841765913 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f44004800 msgr2=0x7f1f44004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:20.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.472+0000 7f1f5b7fe640 1 --2- 172.21.15.32:0/2841765913 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f44004800 0x7f1f44004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:20.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.472+0000 7f1f5b7fe640 1 -- 172.21.15.32:0/2841765913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f44003520 con 0x7f1f440039b0 2024-04-24T23:05:20.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.472+0000 7f1f5affd640 1 --2- 172.21.15.32:0/2841765913 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f44004800 0x7f1f44004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:05:20.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.473+0000 7f1f5b7fe640 1 --2- 172.21.15.32:0/2841765913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f440039b0 0x7f1f44003dd0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c009c90 tx=0x7f1f44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49610e5e6069d4af server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:20.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.473+0000 7f1f5a7fc640 1 -- 172.21.15.32:0/2841765913 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c036070 con 0x7f1f440039b0 2024-04-24T23:05:20.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.473+0000 7f1f5a7fc640 1 -- 172.21.15.32:0/2841765913 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f4c031400 con 0x7f1f440039b0 2024-04-24T23:05:20.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.473+0000 7f1f5a7fc640 1 -- 172.21.15.32:0/2841765913 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c009860 con 0x7f1f440039b0 2024-04-24T23:05:20.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.474+0000 7f1f61f09640 1 -- 172.21.15.32:0/2841765913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f440039b0 msgr2=0x7f1f44003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:20.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.474+0000 7f1f61f09640 1 --2- 172.21.15.32:0/2841765913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f440039b0 0x7f1f44003dd0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c009c90 tx=0x7f1f44005830 comp rx=0 tx=0).stop 2024-04-24T23:05:20.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.474+0000 7f1f5bfff640 1 -- 172.21.15.32:0/2841765913 reap_dead start 2024-04-24T23:05:20.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.474+0000 7f1f61f09640 1 -- 172.21.15.32:0/2841765913 shutdown_connections 2024-04-24T23:05:20.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.474+0000 7f1f61f09640 1 -- 172.21.15.32:0/2841765913 >> 172.21.15.32:0/2841765913 conn(0x7f1f5c0d11d0 msgr2=0x7f1f5c0d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:20.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.474+0000 7f1f61f09640 1 -- 172.21.15.32:0/2841765913 shutdown_connections 2024-04-24T23:05:20.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.474+0000 7f1f61f09640 1 -- 172.21.15.32:0/2841765913 wait complete. 2024-04-24T23:05:20.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.475+0000 7f1f61f09640 1 Processor -- start 2024-04-24T23:05:20.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.475+0000 7f1f61f09640 1 -- start start 2024-04-24T23:05:20.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.476+0000 7f1f61f09640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f440039b0 0x7f1f5c161310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:20.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.476+0000 7f1f61f09640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f5c1531a0 0x7f1f5c157070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:20.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.476+0000 7f1f61f09640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1f5c1575b0 0x7f1f5c155530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:20.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.476+0000 7f1f61f09640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5c0d2400 con 0x7f1f440039b0 2024-04-24T23:05:20.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.476+0000 7f1f61f09640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5c0d3cf0 con 0x7f1f5c1531a0 2024-04-24T23:05:20.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.476+0000 7f1f61f09640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f1f5c0d2b90 con 0x7f1f5c1575b0 2024-04-24T23:05:20.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.476+0000 7f1f5b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f440039b0 0x7f1f5c161310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:20.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.476+0000 7f1f5b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f440039b0 0x7f1f5c161310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49714/0 (socket says 172.21.15.32:49714) 2024-04-24T23:05:20.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.476+0000 7f1f5b7fe640 1 -- 172.21.15.32:0/2064049477 learned_addr learned my addr 172.21.15.32:0/2064049477 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:20.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.476+0000 7f1f5bfff640 1 --2- 172.21.15.32:0/2064049477 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1f5c1575b0 0x7f1f5c155530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:20.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.476+0000 7f1f5affd640 1 --2- 172.21.15.32:0/2064049477 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f5c1531a0 0x7f1f5c157070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:20.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.476+0000 7f1f5b7fe640 1 -- 172.21.15.32:0/2064049477 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1f5c1575b0 msgr2=0x7f1f5c155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:20.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.476+0000 7f1f5b7fe640 1 --2- 172.21.15.32:0/2064049477 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1f5c1575b0 0x7f1f5c155530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:20.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.476+0000 7f1f5b7fe640 1 -- 172.21.15.32:0/2064049477 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f5c1531a0 msgr2=0x7f1f5c157070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:20.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.476+0000 7f1f5b7fe640 1 --2- 172.21.15.32:0/2064049477 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f5c1531a0 0x7f1f5c157070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:20.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.476+0000 7f1f5b7fe640 1 -- 172.21.15.32:0/2064049477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f5c15f7a0 con 0x7f1f440039b0 2024-04-24T23:05:20.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.476+0000 7f1f5b7fe640 1 --2- 172.21.15.32:0/2064049477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f440039b0 0x7f1f5c161310 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c000c00 tx=0x7f1f4c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:20.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.477+0000 7f1f43fff640 1 -- 172.21.15.32:0/2064049477 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c040070 con 0x7f1f440039b0 2024-04-24T23:05:20.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.477+0000 7f1f43fff640 1 -- 172.21.15.32:0/2064049477 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f4c0093e0 con 0x7f1f440039b0 2024-04-24T23:05:20.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.477+0000 7f1f61f09640 1 -- 172.21.15.32:0/2064049477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f5c15fa80 con 0x7f1f440039b0 2024-04-24T23:05:20.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.477+0000 7f1f43fff640 1 -- 172.21.15.32:0/2064049477 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c036040 con 0x7f1f440039b0 2024-04-24T23:05:20.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.477+0000 7f1f61f09640 1 -- 172.21.15.32:0/2064049477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f5c160040 con 0x7f1f440039b0 2024-04-24T23:05:20.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.478+0000 7f1f61f09640 1 -- 172.21.15.32:0/2064049477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f5c068430 con 0x7f1f440039b0 2024-04-24T23:05:20.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.478+0000 7f1f43fff640 1 -- 172.21.15.32:0/2064049477 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1f4c043070 con 0x7f1f440039b0 2024-04-24T23:05:20.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.478+0000 7f1f43fff640 1 --2- 172.21.15.32:0/2064049477 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1f4806be60 0x7f1f4806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:20.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.479+0000 7f1f43fff640 1 -- 172.21.15.32:0/2064049477 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 4943+0+0 (secure 0 0 0) 0x7f1f4c0ab1e0 con 0x7f1f440039b0 2024-04-24T23:05:20.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.479+0000 7f1f5affd640 1 --2- 172.21.15.32:0/2064049477 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1f4806be60 0x7f1f4806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:20.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.479+0000 7f1f5affd640 1 --2- 172.21.15.32:0/2064049477 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1f4806be60 0x7f1f4806e320 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0cbd30 tx=0x7f1f50008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:20.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.483+0000 7f1f43fff640 1 -- 172.21.15.32:0/2064049477 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1f4c0797b0 con 0x7f1f440039b0 2024-04-24T23:05:20.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.668+0000 7f1f61f09640 1 -- 172.21.15.32:0/2064049477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1f5c157990 con 0x7f1f440039b0 2024-04-24T23:05:20.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.669+0000 7f1f43fff640 1 -- 172.21.15.32:0/2064049477 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) ==== 74+0+13043 (secure 0 0 0) 0x7f1f4c07dab0 con 0x7f1f440039b0 2024-04-24T23:05:20.671 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:05:20.671 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":131,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:05:19.350315+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6835/3015290487":"2024-04-25T23:05:16.020562+0000","172.21.15.32:6833/3015290487":"2024-04-25T23:05:16.020562+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:05:20.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.671+0000 7f1f61f09640 1 -- 172.21.15.32:0/2064049477 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1f4806be60 msgr2=0x7f1f4806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:20.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.671+0000 7f1f61f09640 1 --2- 172.21.15.32:0/2064049477 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1f4806be60 0x7f1f4806e320 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0cbd30 tx=0x7f1f50008040 comp rx=0 tx=0).stop 2024-04-24T23:05:20.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.671+0000 7f1f61f09640 1 -- 172.21.15.32:0/2064049477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f440039b0 msgr2=0x7f1f5c161310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:20.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.671+0000 7f1f61f09640 1 --2- 172.21.15.32:0/2064049477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f440039b0 0x7f1f5c161310 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c000c00 tx=0x7f1f4c004730 comp rx=0 tx=0).stop 2024-04-24T23:05:20.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.671+0000 7f1f61f09640 1 -- 172.21.15.32:0/2064049477 shutdown_connections 2024-04-24T23:05:20.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.671+0000 7f1f61f09640 1 --2- 172.21.15.32:0/2064049477 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1f4806be60 0x7f1f4806e320 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:20.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.671+0000 7f1f61f09640 1 --2- 172.21.15.32:0/2064049477 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1f5c1575b0 0x7f1f5c155530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:20.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.671+0000 7f1f61f09640 1 --2- 172.21.15.32:0/2064049477 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f5c1531a0 0x7f1f5c157070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:20.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.671+0000 7f1f61f09640 1 --2- 172.21.15.32:0/2064049477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f440039b0 0x7f1f5c161310 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:20.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.671+0000 7f1f61f09640 1 -- 172.21.15.32:0/2064049477 >> 172.21.15.32:0/2064049477 conn(0x7f1f5c0d11d0 msgr2=0x7f1f5c15dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:20.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.671+0000 7f1f61f09640 1 -- 172.21.15.32:0/2064049477 shutdown_connections 2024-04-24T23:05:20.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.671+0000 7f1f61f09640 1 -- 172.21.15.32:0/2064049477 wait complete. 2024-04-24T23:05:20.684 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:05:20.684 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:05:20.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.787+0000 7f0908e88640 1 Processor -- start 2024-04-24T23:05:20.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.787+0000 7f0908e88640 1 -- start start 2024-04-24T23:05:20.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.787+0000 7f0908e88640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f090406b820 0x7f09040d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:20.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.787+0000 7f0908e88640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09040d4210 0x7f09040b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:20.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.787+0000 7f0908e88640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09040d3170 con 0x7f090415e9b0 2024-04-24T23:05:20.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.787+0000 7f0908e88640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f09040d3450 con 0x7f090406b820 2024-04-24T23:05:20.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.787+0000 7f0908e88640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f09040d3730 con 0x7f09040d4210 2024-04-24T23:05:20.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.788+0000 7f0902d9d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09040d4210 0x7f09040b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:20.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.788+0000 7f08fbfff640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f090406b820 0x7f09040d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:20.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.788+0000 7f090259c640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f090415e9b0 0x7f090406b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:39658/0 (socket says 172.21.15.32:39658) 2024-04-24T23:05:20.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.788+0000 7f0902d9d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09040d4210 0x7f09040b78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.32:55600/0 (socket says 172.21.15.32:55600) 2024-04-24T23:05:20.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.788+0000 7f08fbfff640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f090406b820 0x7f09040d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.32:51768/0 (socket says 172.21.15.32:51768) 2024-04-24T23:05:20.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.788+0000 7f090259c640 1 -- 172.21.15.32:0/638172087 learned_addr learned my addr 172.21.15.32:0/638172087 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:05:20.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.788+0000 7f08fbfff640 1 -- 172.21.15.32:0/638172087 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09040d4210 msgr2=0x7f09040b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:20.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.788+0000 7f08fbfff640 1 --2- 172.21.15.32:0/638172087 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09040d4210 0x7f09040b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:20.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.788+0000 7f08fbfff640 1 -- 172.21.15.32:0/638172087 >> v1:172.21.15.121:6789/0 conn(0x7f090415e9b0 legacy=0x7f090406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:20.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.788+0000 7f08fbfff640 1 -- 172.21.15.32:0/638172087 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09040d3a10 con 0x7f090406b820 2024-04-24T23:05:20.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.788+0000 7f0902d9d640 1 --2- 172.21.15.32:0/638172087 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09040d4210 0x7f09040b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:05:20.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.789+0000 7f08fbfff640 1 --2- 172.21.15.32:0/638172087 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f090406b820 0x7f09040d2b90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f08ec0097a0 tx=0x7f08ec02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3ebfda9a508cb6c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:20.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.789+0000 7f0901d9b640 1 -- 172.21.15.32:0/638172087 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ec037070 con 0x7f090406b820 2024-04-24T23:05:20.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.789+0000 7f0901d9b640 1 -- 172.21.15.32:0/638172087 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08ec032440 con 0x7f090406b820 2024-04-24T23:05:20.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.789+0000 7f0901d9b640 1 -- 172.21.15.32:0/638172087 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ec032810 con 0x7f090406b820 2024-04-24T23:05:20.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.790+0000 7f0908e88640 1 -- 172.21.15.32:0/638172087 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f090406b820 msgr2=0x7f09040d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:20.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.790+0000 7f0908e88640 1 --2- 172.21.15.32:0/638172087 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f090406b820 0x7f09040d2b90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f08ec0097a0 tx=0x7f08ec02a180 comp rx=0 tx=0).stop 2024-04-24T23:05:20.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.790+0000 7f0908e88640 1 -- 172.21.15.32:0/638172087 shutdown_connections 2024-04-24T23:05:20.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.790+0000 7f0908e88640 1 --2- 172.21.15.32:0/638172087 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09040d4210 0x7f09040b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:20.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.790+0000 7f0908e88640 1 --2- 172.21.15.32:0/638172087 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f090406b820 0x7f09040d2b90 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:20.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.790+0000 7f0908e88640 1 -- 172.21.15.32:0/638172087 >> 172.21.15.32:0/638172087 conn(0x7f09040d11d0 msgr2=0x7f09040d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:20.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.790+0000 7f0908e88640 1 -- 172.21.15.32:0/638172087 shutdown_connections 2024-04-24T23:05:20.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.790+0000 7f0908e88640 1 -- 172.21.15.32:0/638172087 wait complete. 2024-04-24T23:05:20.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.791+0000 7f0908e88640 1 Processor -- start 2024-04-24T23:05:20.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.791+0000 7f0908e88640 1 -- start start 2024-04-24T23:05:20.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.792+0000 7f0908e88640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f090406b820 0x7f09041564b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:20.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.792+0000 7f0908e88640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09040d4210 0x7f09041611d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:20.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.792+0000 7f0908e88640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f090415e9b0 0x7f09040cde30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:20.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.792+0000 7f0908e88640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f09040ba0e0 con 0x7f090406b820 2024-04-24T23:05:20.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.792+0000 7f0908e88640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f09040d2400 con 0x7f090415e9b0 2024-04-24T23:05:20.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.792+0000 7f0908e88640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f09040ba260 con 0x7f09040d4210 2024-04-24T23:05:20.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.792+0000 7f090259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f090406b820 0x7f09041564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:20.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.792+0000 7f090259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f090406b820 0x7f09041564b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49716/0 (socket says 172.21.15.32:49716) 2024-04-24T23:05:20.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.792+0000 7f090259c640 1 -- 172.21.15.32:0/1937676927 learned_addr learned my addr 172.21.15.32:0/1937676927 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:20.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.792+0000 7f08fbfff640 1 --2- 172.21.15.32:0/1937676927 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09040d4210 0x7f09041611d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:20.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.792+0000 7f0902d9d640 1 --2- 172.21.15.32:0/1937676927 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f090415e9b0 0x7f09040cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:20.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.792+0000 7f090259c640 1 -- 172.21.15.32:0/1937676927 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09040d4210 msgr2=0x7f09041611d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:20.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.792+0000 7f090259c640 1 --2- 172.21.15.32:0/1937676927 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09040d4210 0x7f09041611d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:20.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.792+0000 7f090259c640 1 -- 172.21.15.32:0/1937676927 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f090415e9b0 msgr2=0x7f09040cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:20.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.792+0000 7f090259c640 1 --2- 172.21.15.32:0/1937676927 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f090415e9b0 0x7f09040cde30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:20.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.792+0000 7f090259c640 1 -- 172.21.15.32:0/1937676927 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09040ccd90 con 0x7f090406b820 2024-04-24T23:05:20.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.792+0000 7f090259c640 1 --2- 172.21.15.32:0/1937676927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f090406b820 0x7f09041564b0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f08f400c400 tx=0x7f08f400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:20.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.793+0000 7f08faffd640 1 -- 172.21.15.32:0/1937676927 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f4017070 con 0x7f090406b820 2024-04-24T23:05:20.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.793+0000 7f08faffd640 1 -- 172.21.15.32:0/1937676927 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08f4005ce0 con 0x7f090406b820 2024-04-24T23:05:20.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.793+0000 7f08faffd640 1 -- 172.21.15.32:0/1937676927 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f40072f0 con 0x7f090406b820 2024-04-24T23:05:20.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.793+0000 7f0908e88640 1 -- 172.21.15.32:0/1937676927 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09040cd070 con 0x7f090406b820 2024-04-24T23:05:20.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.793+0000 7f0908e88640 1 -- 172.21.15.32:0/1937676927 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09040cd910 con 0x7f090406b820 2024-04-24T23:05:20.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.794+0000 7f0908e88640 1 -- 172.21.15.32:0/1937676927 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0904068430 con 0x7f090406b820 2024-04-24T23:05:20.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.794+0000 7f08faffd640 1 -- 172.21.15.32:0/1937676927 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f08f401a070 con 0x7f090406b820 2024-04-24T23:05:20.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.794+0000 7f08faffd640 1 --2- 172.21.15.32:0/1937676927 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f08e406be30 0x7f08e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:20.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.795+0000 7f08faffd640 1 -- 172.21.15.32:0/1937676927 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 4943+0+0 (secure 0 0 0) 0x7f08f4089eb0 con 0x7f090406b820 2024-04-24T23:05:20.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.795+0000 7f08fbfff640 1 --2- 172.21.15.32:0/1937676927 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f08e406be30 0x7f08e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:20.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.795+0000 7f08fbfff640 1 --2- 172.21.15.32:0/1937676927 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f08e406be30 0x7f08e406e2f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f08ec034000 tx=0x7f08ec02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:20.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.799+0000 7f08faffd640 1 -- 172.21.15.32:0/1937676927 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f08f4058510 con 0x7f090406b820 2024-04-24T23:05:20.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.985+0000 7f0908e88640 1 -- 172.21.15.32:0/1937676927 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f09040cd240 con 0x7f090406b820 2024-04-24T23:05:20.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.986+0000 7f08faffd640 1 -- 172.21.15.32:0/1937676927 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v131) ==== 93+0+10 (secure 0 0 0) 0x7f08f405c810 con 0x7f090406b820 2024-04-24T23:05:20.987 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T23:05:20.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.988+0000 7f0908e88640 1 -- 172.21.15.32:0/1937676927 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f08e406be30 msgr2=0x7f08e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:20.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.988+0000 7f0908e88640 1 --2- 172.21.15.32:0/1937676927 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f08e406be30 0x7f08e406e2f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f08ec034000 tx=0x7f08ec02e000 comp rx=0 tx=0).stop 2024-04-24T23:05:20.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.988+0000 7f0908e88640 1 -- 172.21.15.32:0/1937676927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f090406b820 msgr2=0x7f09041564b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:20.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.988+0000 7f0908e88640 1 --2- 172.21.15.32:0/1937676927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f090406b820 0x7f09041564b0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f08f400c400 tx=0x7f08f400c8c0 comp rx=0 tx=0).stop 2024-04-24T23:05:20.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.988+0000 7f0908e88640 1 -- 172.21.15.32:0/1937676927 shutdown_connections 2024-04-24T23:05:20.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.988+0000 7f0908e88640 1 --2- 172.21.15.32:0/1937676927 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f08e406be30 0x7f08e406e2f0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:20.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.988+0000 7f0908e88640 1 --2- 172.21.15.32:0/1937676927 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f090415e9b0 0x7f09040cde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:20.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.988+0000 7f0908e88640 1 --2- 172.21.15.32:0/1937676927 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09040d4210 0x7f09041611d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:20.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.988+0000 7f0908e88640 1 --2- 172.21.15.32:0/1937676927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f090406b820 0x7f09041564b0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:20.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.988+0000 7f0908e88640 1 -- 172.21.15.32:0/1937676927 >> 172.21.15.32:0/1937676927 conn(0x7f09040d11d0 msgr2=0x7f0904159e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:20.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.988+0000 7f0908e88640 1 -- 172.21.15.32:0/1937676927 shutdown_connections 2024-04-24T23:05:20.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:20.988+0000 7f0908e88640 1 -- 172.21.15.32:0/1937676927 wait complete. 2024-04-24T23:05:21.001 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file' 2024-04-24T23:05:21.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.100+0000 7fba6ca58640 1 Processor -- start 2024-04-24T23:05:21.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.100+0000 7fba6ca58640 1 -- start start 2024-04-24T23:05:21.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.101+0000 7fba6ca58640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fba68162320 0x7fba680ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:21.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.101+0000 7fba6ca58640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fba680b5af0 0x7fba68162700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:21.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.101+0000 7fba6ca58640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba680cd270 con 0x7fba68162c40 2024-04-24T23:05:21.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.101+0000 7fba6ca58640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fba680b5660 con 0x7fba680b5af0 2024-04-24T23:05:21.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.101+0000 7fba6ca58640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fba68071610 con 0x7fba68162320 2024-04-24T23:05:21.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.101+0000 7fba5ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fba680b5af0 0x7fba68162700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:21.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.101+0000 7fba5ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fba680b5af0 0x7fba68162700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49726/0 (socket says 172.21.15.32:49726) 2024-04-24T23:05:21.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.101+0000 7fba5ffff640 1 -- 172.21.15.32:0/2649916217 learned_addr learned my addr 172.21.15.32:0/2649916217 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:21.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.101+0000 7fba6659c640 1 --2- 172.21.15.32:0/2649916217 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fba68162320 0x7fba680ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:21.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.101+0000 7fba5ffff640 1 -- 172.21.15.32:0/2649916217 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fba68162320 msgr2=0x7fba680ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:21.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.101+0000 7fba5ffff640 1 --2- 172.21.15.32:0/2649916217 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fba68162320 0x7fba680ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:21.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.101+0000 7fba5ffff640 1 -- 172.21.15.32:0/2649916217 >> v1:172.21.15.121:6789/0 conn(0x7fba68162c40 legacy=0x7fba680ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:21.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.101+0000 7fba5ffff640 1 -- 172.21.15.32:0/2649916217 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba680718f0 con 0x7fba680b5af0 2024-04-24T23:05:21.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.101+0000 7fba6659c640 1 --2- 172.21.15.32:0/2649916217 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fba68162320 0x7fba680ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:05:21.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.102+0000 7fba5ffff640 1 --2- 172.21.15.32:0/2649916217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba680b5af0 0x7fba68162700 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fba50009870 tx=0x7fba5002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5176d3b95ae70c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:21.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.102+0000 7fba65d9b640 1 -- 172.21.15.32:0/2649916217 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba50037070 con 0x7fba680b5af0 2024-04-24T23:05:21.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.102+0000 7fba65d9b640 1 -- 172.21.15.32:0/2649916217 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba50032440 con 0x7fba680b5af0 2024-04-24T23:05:21.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.102+0000 7fba65d9b640 1 -- 172.21.15.32:0/2649916217 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba50032810 con 0x7fba680b5af0 2024-04-24T23:05:21.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.103+0000 7fba6ca58640 1 -- 172.21.15.32:0/2649916217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba680b5af0 msgr2=0x7fba68162700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:21.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.103+0000 7fba6ca58640 1 --2- 172.21.15.32:0/2649916217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba680b5af0 0x7fba68162700 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fba50009870 tx=0x7fba5002a1c0 comp rx=0 tx=0).stop 2024-04-24T23:05:21.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.103+0000 7fba6ca58640 1 -- 172.21.15.32:0/2649916217 shutdown_connections 2024-04-24T23:05:21.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.103+0000 7fba6ca58640 1 --2- 172.21.15.32:0/2649916217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba680b5af0 0x7fba68162700 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:21.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.103+0000 7fba6ca58640 1 --2- 172.21.15.32:0/2649916217 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fba68162320 0x7fba680ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:21.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.103+0000 7fba6ca58640 1 -- 172.21.15.32:0/2649916217 >> 172.21.15.32:0/2649916217 conn(0x7fba680d0ae0 msgr2=0x7fba680d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:21.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.103+0000 7fba6ca58640 1 -- 172.21.15.32:0/2649916217 shutdown_connections 2024-04-24T23:05:21.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.103+0000 7fba6ca58640 1 -- 172.21.15.32:0/2649916217 wait complete. 2024-04-24T23:05:21.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.104+0000 7fba6ca58640 1 Processor -- start 2024-04-24T23:05:21.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.104+0000 7fba6ca58640 1 -- start start 2024-04-24T23:05:21.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.105+0000 7fba6ca58640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fba680b5af0 0x7fba68155090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:21.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.105+0000 7fba6ca58640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fba68162320 0x7fba680cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:21.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.105+0000 7fba6ca58640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba68162c40 0x7fba68144880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:21.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.105+0000 7fba6ca58640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fba68067950 con 0x7fba68162c40 2024-04-24T23:05:21.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.105+0000 7fba6ca58640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fba6815c630 con 0x7fba68162320 2024-04-24T23:05:21.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.105+0000 7fba6ca58640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fba68071bd0 con 0x7fba680b5af0 2024-04-24T23:05:21.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.105+0000 7fba66d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba68162c40 0x7fba68144880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:21.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.105+0000 7fba66d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba68162c40 0x7fba68144880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49742/0 (socket says 172.21.15.32:49742) 2024-04-24T23:05:21.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.105+0000 7fba66d9d640 1 -- 172.21.15.32:0/3011838641 learned_addr learned my addr 172.21.15.32:0/3011838641 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:21.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.105+0000 7fba6659c640 1 --2- 172.21.15.32:0/3011838641 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fba680b5af0 0x7fba68155090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:21.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.105+0000 7fba5ffff640 1 --2- 172.21.15.32:0/3011838641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fba68162320 0x7fba680cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:21.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.105+0000 7fba66d9d640 1 -- 172.21.15.32:0/3011838641 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fba680b5af0 msgr2=0x7fba68155090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:21.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.105+0000 7fba66d9d640 1 --2- 172.21.15.32:0/3011838641 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fba680b5af0 0x7fba68155090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:21.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.105+0000 7fba66d9d640 1 -- 172.21.15.32:0/3011838641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fba68162320 msgr2=0x7fba680cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:21.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.105+0000 7fba66d9d640 1 --2- 172.21.15.32:0/3011838641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fba68162320 0x7fba680cd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:21.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.105+0000 7fba66d9d640 1 -- 172.21.15.32:0/3011838641 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba681437e0 con 0x7fba68162c40 2024-04-24T23:05:21.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.105+0000 7fba5ffff640 1 --2- 172.21.15.32:0/3011838641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fba68162320 0x7fba680cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:05:21.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.105+0000 7fba6659c640 1 --2- 172.21.15.32:0/3011838641 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fba680b5af0 0x7fba68155090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:05:21.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.105+0000 7fba66d9d640 1 --2- 172.21.15.32:0/3011838641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba68162c40 0x7fba68144880 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fba6000c6b0 tx=0x7fba6000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:21.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.106+0000 7fba5effd640 1 -- 172.21.15.32:0/3011838641 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba60003b70 con 0x7fba68162c40 2024-04-24T23:05:21.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.106+0000 7fba5effd640 1 -- 172.21.15.32:0/3011838641 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba60003d30 con 0x7fba68162c40 2024-04-24T23:05:21.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.106+0000 7fba6ca58640 1 -- 172.21.15.32:0/3011838641 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba68143ac0 con 0x7fba68162c40 2024-04-24T23:05:21.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.106+0000 7fba5effd640 1 -- 172.21.15.32:0/3011838641 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba60005310 con 0x7fba68162c40 2024-04-24T23:05:21.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.106+0000 7fba6ca58640 1 -- 172.21.15.32:0/3011838641 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba68144080 con 0x7fba68162c40 2024-04-24T23:05:21.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.107+0000 7fba6ca58640 1 -- 172.21.15.32:0/3011838641 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba68068430 con 0x7fba68162c40 2024-04-24T23:05:21.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.107+0000 7fba5effd640 1 -- 172.21.15.32:0/3011838641 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fba600054b0 con 0x7fba68162c40 2024-04-24T23:05:21.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.108+0000 7fba5effd640 1 --2- 172.21.15.32:0/3011838641 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fba4806be60 0x7fba4806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:21.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.108+0000 7fba5effd640 1 -- 172.21.15.32:0/3011838641 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 4943+0+0 (secure 0 0 0) 0x7fba6008a9b0 con 0x7fba68162c40 2024-04-24T23:05:21.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.108+0000 7fba6659c640 1 --2- 172.21.15.32:0/3011838641 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fba4806be60 0x7fba4806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:21.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.108+0000 7fba6659c640 1 --2- 172.21.15.32:0/3011838641 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fba4806be60 0x7fba4806e320 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fba6815bb30 tx=0x7fba58008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:21.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.112+0000 7fba5effd640 1 -- 172.21.15.32:0/3011838641 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fba60059660 con 0x7fba68162c40 2024-04-24T23:05:21.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:21.370+0000 7fba6ca58640 1 -- 172.21.15.32:0/3011838641 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]} v 0) -- 0x7fba680d0850 con 0x7fba68162c40 2024-04-24T23:05:22.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.361+0000 7fba5effd640 1 -- 172.21.15.32:0/3011838641 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]}]=0 v334) ==== 143+0+0 (secure 0 0 0) 0x7fba60058f60 con 0x7fba68162c40 2024-04-24T23:05:22.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.364+0000 7fba6ca58640 1 -- 172.21.15.32:0/3011838641 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fba4806be60 msgr2=0x7fba4806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:22.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.364+0000 7fba6ca58640 1 --2- 172.21.15.32:0/3011838641 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fba4806be60 0x7fba4806e320 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fba6815bb30 tx=0x7fba58008040 comp rx=0 tx=0).stop 2024-04-24T23:05:22.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.364+0000 7fba6ca58640 1 -- 172.21.15.32:0/3011838641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba68162c40 msgr2=0x7fba68144880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:22.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.364+0000 7fba6ca58640 1 --2- 172.21.15.32:0/3011838641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba68162c40 0x7fba68144880 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fba6000c6b0 tx=0x7fba6000cb70 comp rx=0 tx=0).stop 2024-04-24T23:05:22.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.364+0000 7fba6ca58640 1 -- 172.21.15.32:0/3011838641 shutdown_connections 2024-04-24T23:05:22.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.364+0000 7fba6ca58640 1 --2- 172.21.15.32:0/3011838641 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fba4806be60 0x7fba4806e320 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:22.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.364+0000 7fba6ca58640 1 --2- 172.21.15.32:0/3011838641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba68162c40 0x7fba68144880 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:22.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.364+0000 7fba6ca58640 1 --2- 172.21.15.32:0/3011838641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fba68162320 0x7fba680cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:22.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.364+0000 7fba6ca58640 1 --2- 172.21.15.32:0/3011838641 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fba680b5af0 0x7fba68155090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:22.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.364+0000 7fba6ca58640 1 -- 172.21.15.32:0/3011838641 >> 172.21.15.32:0/3011838641 conn(0x7fba680d0ae0 msgr2=0x7fba680b1630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:22.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.364+0000 7fba6ca58640 1 -- 172.21.15.32:0/3011838641 shutdown_connections 2024-04-24T23:05:22.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.364+0000 7fba6ca58640 1 -- 172.21.15.32:0/3011838641 wait complete. 2024-04-24T23:05:22.378 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-24T23:05:22.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.483+0000 7ff0a635c640 1 Processor -- start 2024-04-24T23:05:22.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.483+0000 7ff0a635c640 1 -- start start 2024-04-24T23:05:22.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.483+0000 7ff0a635c640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff0a006c5f0 0x7ff0a006c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:22.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.483+0000 7ff0a635c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff0a00b44d0 0x7ff0a00b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:22.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.483+0000 7ff0a635c640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0a00b6430 con 0x7ff0a00af890 2024-04-24T23:05:22.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.483+0000 7ff0a635c640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7ff0a0156f80 con 0x7ff0a006c5f0 2024-04-24T23:05:22.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.483+0000 7ff0a635c640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff0a00b6150 con 0x7ff0a00b44d0 2024-04-24T23:05:22.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.484+0000 7ff09f7fe640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff0a00b44d0 0x7ff0a00b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:22.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.484+0000 7ff09ffff640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff0a006c5f0 0x7ff0a006c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:22.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.484+0000 7ff0a48f9640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7ff0a00af890 0x7ff0a00cef70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:39670/0 (socket says 172.21.15.32:39670) 2024-04-24T23:05:22.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.484+0000 7ff09ffff640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff0a006c5f0 0x7ff0a006c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.32:51808/0 (socket says 172.21.15.32:51808) 2024-04-24T23:05:22.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.484+0000 7ff0a48f9640 1 -- 172.21.15.32:0/2411803956 learned_addr learned my addr 172.21.15.32:0/2411803956 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:05:22.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.484+0000 7ff09f7fe640 1 -- 172.21.15.32:0/2411803956 >> v2:172.21.15.121:3300/0 conn(0x7ff0a006c5f0 msgr2=0x7ff0a006c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:22.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.484+0000 7ff09f7fe640 1 --2- 172.21.15.32:0/2411803956 >> v2:172.21.15.121:3300/0 conn(0x7ff0a006c5f0 0x7ff0a006c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:22.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.484+0000 7ff09f7fe640 1 -- 172.21.15.32:0/2411803956 >> v1:172.21.15.121:6789/0 conn(0x7ff0a00af890 legacy=0x7ff0a00cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:22.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.484+0000 7ff09f7fe640 1 -- 172.21.15.32:0/2411803956 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0a00b39f0 con 0x7ff0a00b44d0 2024-04-24T23:05:22.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.484+0000 7ff09ffff640 1 --2- 172.21.15.32:0/2411803956 >> v2:172.21.15.121:3300/0 conn(0x7ff0a006c5f0 0x7ff0a006c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:05:22.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.485+0000 7ff09f7fe640 1 --2- 172.21.15.32:0/2411803956 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff0a00b44d0 0x7ff0a00b5890 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff0940097a0 tx=0x7ff09402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41a4cfb0cfeca473 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:22.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.485+0000 7ff09effd640 1 -- 172.21.15.32:0/2411803956 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff094037070 con 0x7ff0a00b44d0 2024-04-24T23:05:22.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.485+0000 7ff09effd640 1 -- 172.21.15.32:0/2411803956 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff09402ae50 con 0x7ff0a00b44d0 2024-04-24T23:05:22.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.485+0000 7ff09effd640 1 -- 172.21.15.32:0/2411803956 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff094032660 con 0x7ff0a00b44d0 2024-04-24T23:05:22.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.486+0000 7ff0a635c640 1 -- 172.21.15.32:0/2411803956 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff0a00b44d0 msgr2=0x7ff0a00b5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:22.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.486+0000 7ff0a635c640 1 --2- 172.21.15.32:0/2411803956 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff0a00b44d0 0x7ff0a00b5890 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff0940097a0 tx=0x7ff09402a100 comp rx=0 tx=0).stop 2024-04-24T23:05:22.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.486+0000 7ff0a635c640 1 -- 172.21.15.32:0/2411803956 shutdown_connections 2024-04-24T23:05:22.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.486+0000 7ff0a635c640 1 --2- 172.21.15.32:0/2411803956 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff0a00b44d0 0x7ff0a00b5890 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:22.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.486+0000 7ff0a635c640 1 --2- 172.21.15.32:0/2411803956 >> v2:172.21.15.121:3300/0 conn(0x7ff0a006c5f0 0x7ff0a006c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:22.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.486+0000 7ff0a635c640 1 -- 172.21.15.32:0/2411803956 >> 172.21.15.32:0/2411803956 conn(0x7ff0a006b110 msgr2=0x7ff0a006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:22.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.486+0000 7ff0a635c640 1 -- 172.21.15.32:0/2411803956 shutdown_connections 2024-04-24T23:05:22.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.486+0000 7ff0a635c640 1 -- 172.21.15.32:0/2411803956 wait complete. 2024-04-24T23:05:22.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.487+0000 7ff0a635c640 1 Processor -- start 2024-04-24T23:05:22.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.487+0000 7ff0a635c640 1 -- start start 2024-04-24T23:05:22.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.488+0000 7ff0a635c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0a006c5f0 0x7ff0a0143dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:22.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.488+0000 7ff0a635c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0a00af890 0x7ff0a0158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:22.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.488+0000 7ff0a635c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff0a00b44d0 0x7ff0a015ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:22.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.488+0000 7ff0a635c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a006cf10 con 0x7ff0a006c5f0 2024-04-24T23:05:22.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.488+0000 7ff0a635c640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a00b3fb0 con 0x7ff0a00af890 2024-04-24T23:05:22.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.488+0000 7ff0a635c640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff0a00b5dd0 con 0x7ff0a00b44d0 2024-04-24T23:05:22.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.488+0000 7ff09ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0a006c5f0 0x7ff0a0143dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:22.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.488+0000 7ff09ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0a006c5f0 0x7ff0a0143dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49758/0 (socket says 172.21.15.32:49758) 2024-04-24T23:05:22.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.488+0000 7ff09ffff640 1 -- 172.21.15.32:0/4247335626 learned_addr learned my addr 172.21.15.32:0/4247335626 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:22.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.488+0000 7ff09f7fe640 1 --2- 172.21.15.32:0/4247335626 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0a00af890 0x7ff0a0158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:22.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.488+0000 7ff0a48f9640 1 --2- 172.21.15.32:0/4247335626 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff0a00b44d0 0x7ff0a015ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:22.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.488+0000 7ff09ffff640 1 -- 172.21.15.32:0/4247335626 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff0a00b44d0 msgr2=0x7ff0a015ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:22.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.488+0000 7ff09ffff640 1 --2- 172.21.15.32:0/4247335626 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff0a00b44d0 0x7ff0a015ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:22.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.488+0000 7ff09ffff640 1 -- 172.21.15.32:0/4247335626 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0a00af890 msgr2=0x7ff0a0158ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:22.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.488+0000 7ff09ffff640 1 --2- 172.21.15.32:0/4247335626 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0a00af890 0x7ff0a0158ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:22.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.488+0000 7ff09ffff640 1 -- 172.21.15.32:0/4247335626 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0a01f5cf0 con 0x7ff0a006c5f0 2024-04-24T23:05:22.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.488+0000 7ff09ffff640 1 --2- 172.21.15.32:0/4247335626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0a006c5f0 0x7ff0a0143dc0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7ff09000e960 tx=0x7ff09000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:22.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.489+0000 7ff09cff9640 1 -- 172.21.15.32:0/4247335626 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff090017070 con 0x7ff0a006c5f0 2024-04-24T23:05:22.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.489+0000 7ff09cff9640 1 -- 172.21.15.32:0/4247335626 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff09000ca00 con 0x7ff0a006c5f0 2024-04-24T23:05:22.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.489+0000 7ff0a635c640 1 -- 172.21.15.32:0/4247335626 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0a01f5fd0 con 0x7ff0a006c5f0 2024-04-24T23:05:22.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.489+0000 7ff09cff9640 1 -- 172.21.15.32:0/4247335626 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09000cdd0 con 0x7ff0a006c5f0 2024-04-24T23:05:22.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.489+0000 7ff0a635c640 1 -- 172.21.15.32:0/4247335626 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0a01f6590 con 0x7ff0a006c5f0 2024-04-24T23:05:22.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.490+0000 7ff0a635c640 1 -- 172.21.15.32:0/4247335626 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0a00691d0 con 0x7ff0a006c5f0 2024-04-24T23:05:22.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.490+0000 7ff09cff9640 1 -- 172.21.15.32:0/4247335626 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff09001a070 con 0x7ff0a006c5f0 2024-04-24T23:05:22.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.490+0000 7ff09cff9640 1 --2- 172.21.15.32:0/4247335626 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff08c06be30 0x7ff08c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:22.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.491+0000 7ff09cff9640 1 -- 172.21.15.32:0/4247335626 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 4943+0+0 (secure 0 0 0) 0x7ff09008a910 con 0x7ff0a006c5f0 2024-04-24T23:05:22.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.491+0000 7ff09f7fe640 1 --2- 172.21.15.32:0/4247335626 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff08c06be30 0x7ff08c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:22.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.491+0000 7ff09f7fe640 1 --2- 172.21.15.32:0/4247335626 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff08c06be30 0x7ff08c06e2f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ff094034000 tx=0x7ff09402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:22.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.495+0000 7ff09cff9640 1 -- 172.21.15.32:0/4247335626 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff0a00691d0 con 0x7ff0a006c5f0 2024-04-24T23:05:22.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.679+0000 7ff0a635c640 1 -- 172.21.15.32:0/4247335626 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7ff0a0157160 con 0x7ff0a006c5f0 2024-04-24T23:05:22.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.683+0000 7ff09cff9640 1 -- 172.21.15.32:0/4247335626 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7ff090059000 con 0x7ff0a006c5f0 2024-04-24T23:05:22.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.684+0000 7ff0a635c640 1 -- 172.21.15.32:0/4247335626 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff08c06be30 msgr2=0x7ff08c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:22.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.684+0000 7ff0a635c640 1 --2- 172.21.15.32:0/4247335626 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff08c06be30 0x7ff08c06e2f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ff094034000 tx=0x7ff09402e000 comp rx=0 tx=0).stop 2024-04-24T23:05:22.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.684+0000 7ff0a635c640 1 -- 172.21.15.32:0/4247335626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0a006c5f0 msgr2=0x7ff0a0143dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:22.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.684+0000 7ff0a635c640 1 --2- 172.21.15.32:0/4247335626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0a006c5f0 0x7ff0a0143dc0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7ff09000e960 tx=0x7ff09000ee20 comp rx=0 tx=0).stop 2024-04-24T23:05:22.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.685+0000 7ff0a635c640 1 -- 172.21.15.32:0/4247335626 shutdown_connections 2024-04-24T23:05:22.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.685+0000 7ff0a635c640 1 --2- 172.21.15.32:0/4247335626 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff08c06be30 0x7ff08c06e2f0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:22.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.685+0000 7ff0a635c640 1 --2- 172.21.15.32:0/4247335626 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff0a00b44d0 0x7ff0a015ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:22.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.685+0000 7ff0a635c640 1 --2- 172.21.15.32:0/4247335626 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0a00af890 0x7ff0a0158ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:22.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.685+0000 7ff0a635c640 1 --2- 172.21.15.32:0/4247335626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0a006c5f0 0x7ff0a0143dc0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:22.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.685+0000 7ff0a635c640 1 -- 172.21.15.32:0/4247335626 >> 172.21.15.32:0/4247335626 conn(0x7ff0a006b110 msgr2=0x7ff0a00d01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:22.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.685+0000 7ff0a635c640 1 -- 172.21.15.32:0/4247335626 shutdown_connections 2024-04-24T23:05:22.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.685+0000 7ff0a635c640 1 -- 172.21.15.32:0/4247335626 wait complete. 2024-04-24T23:05:22.698 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-24T23:05:22.698 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:05:22.734 DEBUG:teuthology.orchestra.run.smithi121:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:05:22.785 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-24T23:05:22.785 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:05:22.785 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:05:22.801 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=413 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T23:05:22.801 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:05:22.801 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:05:22.819 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3271 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T23:05:22.819 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:05:22.819 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:05:22.857 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=413 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T23:05:22.857 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:05:22.857 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:05:22.876 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3271 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T23:05:22.876 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-24T23:05:22.876 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:05:22.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.997+0000 7fa1df361640 1 Processor -- start 2024-04-24T23:05:22.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.997+0000 7fa1df361640 1 -- start start 2024-04-24T23:05:23.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.998+0000 7fa1df361640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa1d815b380 0x7fa1d806c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:23.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.998+0000 7fa1df361640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa1d80ceb40 0x7fa1d806cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:23.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.998+0000 7fa1df361640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1d80b4520 con 0x7fa1d815cb00 2024-04-24T23:05:23.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.998+0000 7fa1df361640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fa1d80cef90 con 0x7fa1d80ceb40 2024-04-24T23:05:23.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.998+0000 7fa1df361640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa1d80b4800 con 0x7fa1d815b380 2024-04-24T23:05:23.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.998+0000 7fa1dd8fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa1d80ceb40 0x7fa1d806cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:23.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.998+0000 7fa1dd8fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa1d80ceb40 0x7fa1d806cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49770/0 (socket says 172.21.15.32:49770) 2024-04-24T23:05:23.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.998+0000 7fa1dd8fe640 1 -- 172.21.15.32:0/3443430162 learned_addr learned my addr 172.21.15.32:0/3443430162 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:23.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.998+0000 7fa1dd0fd640 1 --2- 172.21.15.32:0/3443430162 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa1d815b380 0x7fa1d806c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:23.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.998+0000 7fa1dd8fe640 1 -- 172.21.15.32:0/3443430162 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa1d815b380 msgr2=0x7fa1d806c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:23.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.998+0000 7fa1dd8fe640 1 --2- 172.21.15.32:0/3443430162 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa1d815b380 0x7fa1d806c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:23.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.998+0000 7fa1dd8fe640 1 -- 172.21.15.32:0/3443430162 >> v1:172.21.15.121:6789/0 conn(0x7fa1d815cb00 legacy=0x7fa1d80ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:23.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.998+0000 7fa1dd8fe640 1 -- 172.21.15.32:0/3443430162 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1d80b4ae0 con 0x7fa1d80ceb40 2024-04-24T23:05:23.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.998+0000 7fa1dd0fd640 1 --2- 172.21.15.32:0/3443430162 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa1d815b380 0x7fa1d806c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:05:23.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.999+0000 7fa1dd8fe640 1 --2- 172.21.15.32:0/3443430162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1d80ceb40 0x7fa1d806cb30 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0009e50 tx=0x7fa1d002a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96ba064142fbb996 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:23.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.999+0000 7fa1cffff640 1 -- 172.21.15.32:0/3443430162 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d0036070 con 0x7fa1d80ceb40 2024-04-24T23:05:23.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.999+0000 7fa1cffff640 1 -- 172.21.15.32:0/3443430162 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1d0031600 con 0x7fa1d80ceb40 2024-04-24T23:05:23.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:22.999+0000 7fa1cffff640 1 -- 172.21.15.32:0/3443430162 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d00319d0 con 0x7fa1d80ceb40 2024-04-24T23:05:23.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:23.000+0000 7fa1df361640 1 -- 172.21.15.32:0/3443430162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1d80ceb40 msgr2=0x7fa1d806cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:23.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:23.000+0000 7fa1df361640 1 --2- 172.21.15.32:0/3443430162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1d80ceb40 0x7fa1d806cb30 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0009e50 tx=0x7fa1d002a630 comp rx=0 tx=0).stop 2024-04-24T23:05:23.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:23.000+0000 7fa1df361640 1 -- 172.21.15.32:0/3443430162 shutdown_connections 2024-04-24T23:05:23.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:23.000+0000 7fa1df361640 1 --2- 172.21.15.32:0/3443430162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1d80ceb40 0x7fa1d806cb30 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:23.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:23.000+0000 7fa1df361640 1 --2- 172.21.15.32:0/3443430162 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa1d815b380 0x7fa1d806c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:23.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:23.000+0000 7fa1df361640 1 -- 172.21.15.32:0/3443430162 >> 172.21.15.32:0/3443430162 conn(0x7fa1d806b110 msgr2=0x7fa1d806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:23.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:23.000+0000 7fa1df361640 1 -- 172.21.15.32:0/3443430162 shutdown_connections 2024-04-24T23:05:23.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:23.000+0000 7fa1df361640 1 -- 172.21.15.32:0/3443430162 wait complete. 2024-04-24T23:05:23.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:23.001+0000 7fa1df361640 1 Processor -- start 2024-04-24T23:05:23.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:23.001+0000 7fa1df361640 1 -- start start 2024-04-24T23:05:23.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:23.002+0000 7fa1df361640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1d80ceb40 0x7fa1d80b3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:23.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:23.002+0000 7fa1df361640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa1d815b380 0x7fa1d81e5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:23.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:23.002+0000 7fa1df361640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1d815cb00 0x7fa1d814f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:23.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:23.002+0000 7fa1df361640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d815b760 con 0x7fa1d815cb00 2024-04-24T23:05:23.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:23.002+0000 7fa1df361640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d80b50a0 con 0x7fa1d80ceb40 2024-04-24T23:05:23.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:23.002+0000 7fa1df361640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa1d80b55c0 con 0x7fa1d815b380 2024-04-24T23:05:23.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:23.002+0000 7fa1dd8fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1d815cb00 0x7fa1d814f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:23.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:23.002+0000 7fa1dd8fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1d815cb00 0x7fa1d814f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49774/0 (socket says 172.21.15.32:49774) 2024-04-24T23:05:23.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:23.002+0000 7fa1dd8fe640 1 -- 172.21.15.32:0/1556642625 learned_addr learned my addr 172.21.15.32:0/1556642625 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:23.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:23.002+0000 7fa1dd0fd640 1 --2- 172.21.15.32:0/1556642625 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1d80ceb40 0x7fa1d80b3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:23.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:23.002+0000 7fa1dc8fc640 1 --2- 172.21.15.32:0/1556642625 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa1d815b380 0x7fa1d81e5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:23.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:23.002+0000 7fa1dd8fe640 1 -- 172.21.15.32:0/1556642625 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa1d815b380 msgr2=0x7fa1d81e5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:23.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:23.002+0000 7fa1dd8fe640 1 --2- 172.21.15.32:0/1556642625 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa1d815b380 0x7fa1d81e5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:23.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:23.002+0000 7fa1dd8fe640 1 -- 172.21.15.32:0/1556642625 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1d80ceb40 msgr2=0x7fa1d80b3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:23.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:23.002+0000 7fa1dd8fe640 1 --2- 172.21.15.32:0/1556642625 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1d80ceb40 0x7fa1d80b3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:23.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:23.002+0000 7fa1dd8fe640 1 -- 172.21.15.32:0/1556642625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1d8159a00 con 0x7fa1d815cb00 2024-04-24T23:05:23.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:23.002+0000 7fa1dc8fc640 1 --2- 172.21.15.32:0/1556642625 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa1d815b380 0x7fa1d81e5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:05:23.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:23.002+0000 7fa1dd0fd640 1 --2- 172.21.15.32:0/1556642625 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1d80ceb40 0x7fa1d80b3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:05:23.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:23.002+0000 7fa1dd8fe640 1 --2- 172.21.15.32:0/1556642625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1d815cb00 0x7fa1d814f7c0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0009e50 tx=0x7fa1d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:23.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:23.003+0000 7fa1cdffb640 1 -- 172.21.15.32:0/1556642625 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d0036050 con 0x7fa1d815cb00 2024-04-24T23:05:23.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:23.003+0000 7fa1cdffb640 1 -- 172.21.15.32:0/1556642625 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1d0042070 con 0x7fa1d815cb00 2024-04-24T23:05:23.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:23.003+0000 7fa1cdffb640 1 -- 172.21.15.32:0/1556642625 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d0009510 con 0x7fa1d815cb00 2024-04-24T23:05:23.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:23.003+0000 7fa1df361640 1 -- 172.21.15.32:0/1556642625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1d8159ce0 con 0x7fa1d815cb00 2024-04-24T23:05:23.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:23.003+0000 7fa1df361640 1 -- 172.21.15.32:0/1556642625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1d815a2a0 con 0x7fa1d815cb00 2024-04-24T23:05:23.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:23.004+0000 7fa1df361640 1 -- 172.21.15.32:0/1556642625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1d8068430 con 0x7fa1d815cb00 2024-04-24T23:05:23.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:23.004+0000 7fa1cdffb640 1 -- 172.21.15.32:0/1556642625 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa1d0007bd0 con 0x7fa1d815cb00 2024-04-24T23:05:23.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:23.005+0000 7fa1cdffb640 1 --2- 172.21.15.32:0/1556642625 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa1bc06be60 0x7fa1bc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:23.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:23.005+0000 7fa1cdffb640 1 -- 172.21.15.32:0/1556642625 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 4943+0+0 (secure 0 0 0) 0x7fa1d00aba00 con 0x7fa1d815cb00 2024-04-24T23:05:23.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:23.005+0000 7fa1dd0fd640 1 --2- 172.21.15.32:0/1556642625 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa1bc06be60 0x7fa1bc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:23.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:23.005+0000 7fa1dd0fd640 1 --2- 172.21.15.32:0/1556642625 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa1bc06be60 0x7fa1bc06e320 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa1d400aa30 tx=0x7fa1d4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:23.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:23.009+0000 7fa1cdffb640 1 -- 172.21.15.32:0/1556642625 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa1d0045360 con 0x7fa1d815cb00 2024-04-24T23:05:23.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:23.229+0000 7fa1df361640 1 -- 172.21.15.32:0/1556642625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa1d806c000 con 0x7fa1d815cb00 2024-04-24T23:05:23.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:23.230+0000 7fa1cdffb640 1 -- 172.21.15.32:0/1556642625 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+3872 (secure 0 0 0) 0x7fa1d007a740 con 0x7fa1d815cb00 2024-04-24T23:05:23.231 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:05:23.231 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":53,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5972,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3336404028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3336404028},{"type":"v1","addr":"172.21.15.121:6837","nonce":3336404028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6505,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/1068557774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1068557774},{"type":"v1","addr":"172.21.15.121:6836","nonce":1068557774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":7353,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/2118719195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":2118719195},{"type":"v1","addr":"172.21.15.32:6834","nonce":2118719195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":7710,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/462071289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":462071289},{"type":"v1","addr":"172.21.15.32:6835","nonce":462071289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53}],"filesystems":[]} 2024-04-24T23:05:23.232 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 53 2024-04-24T23:05:23.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:23.232+0000 7fa1df361640 1 -- 172.21.15.32:0/1556642625 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa1bc06be60 msgr2=0x7fa1bc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:23.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:23.232+0000 7fa1df361640 1 --2- 172.21.15.32:0/1556642625 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa1bc06be60 0x7fa1bc06e320 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa1d400aa30 tx=0x7fa1d4008040 comp rx=0 tx=0).stop 2024-04-24T23:05:23.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:23.232+0000 7fa1df361640 1 -- 172.21.15.32:0/1556642625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1d815cb00 msgr2=0x7fa1d814f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:23.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:23.232+0000 7fa1df361640 1 --2- 172.21.15.32:0/1556642625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1d815cb00 0x7fa1d814f7c0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0009e50 tx=0x7fa1d0004a40 comp rx=0 tx=0).stop 2024-04-24T23:05:23.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:23.232+0000 7fa1df361640 1 -- 172.21.15.32:0/1556642625 shutdown_connections 2024-04-24T23:05:23.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:23.232+0000 7fa1df361640 1 --2- 172.21.15.32:0/1556642625 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa1bc06be60 0x7fa1bc06e320 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:23.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:23.232+0000 7fa1df361640 1 --2- 172.21.15.32:0/1556642625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1d815cb00 0x7fa1d814f7c0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:23.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:23.232+0000 7fa1df361640 1 --2- 172.21.15.32:0/1556642625 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa1d815b380 0x7fa1d81e5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:23.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:23.232+0000 7fa1df361640 1 --2- 172.21.15.32:0/1556642625 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1d80ceb40 0x7fa1d80b3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:23.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:23.232+0000 7fa1df361640 1 -- 172.21.15.32:0/1556642625 >> 172.21.15.32:0/1556642625 conn(0x7fa1d806b110 msgr2=0x7fa1d815ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:23.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:23.232+0000 7fa1df361640 1 -- 172.21.15.32:0/1556642625 shutdown_connections 2024-04-24T23:05:23.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:23.232+0000 7fa1df361640 1 -- 172.21.15.32:0/1556642625 wait complete. 2024-04-24T23:05:23.245 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-24T23:05:23.245 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-24T23:05:23.245 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-24T23:05:23.245 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:05:23.245 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-24T23:05:23.245 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-24T23:05:23.245 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-24T23:05:23.246 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:05:23.246 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-24T23:05:23.246 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-24T23:05:23.246 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-24T23:05:23.246 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:05:23.246 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-24T23:05:23.246 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-24T23:05:23.247 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-24T23:05:23.247 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:05:23.257 INFO:tasks.ceph.mds.c:Stopped 2024-04-24T23:05:23.257 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-24T23:05:23.258 INFO:tasks.ceph.mds.d:Stopped 2024-04-24T23:05:23.258 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-24T23:05:23.259 INFO:tasks.ceph.mds.b:Stopped 2024-04-24T23:05:23.259 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-24T23:05:23.301 INFO:tasks.ceph.mds.c:Started 2024-04-24T23:05:23.301 DEBUG:teuthology.parallel:result is None 2024-04-24T23:05:23.301 INFO:tasks.ceph.mds.d:Started 2024-04-24T23:05:23.302 INFO:tasks.ceph.mds.b:Started 2024-04-24T23:05:23.302 DEBUG:teuthology.parallel:result is None 2024-04-24T23:05:23.302 DEBUG:teuthology.parallel:result is None 2024-04-24T23:05:23.373 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2024-04-24T23:05:23.378 INFO:tasks.ceph.mds.b.smithi121.stdout:starting mds.b at 2024-04-24T23:05:23.378 INFO:tasks.ceph.mds.d.smithi121.stdout:starting mds.d at 2024-04-24T23:05:29.257 INFO:tasks.ceph.mds.a:Stopped 2024-04-24T23:05:29.257 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-24T23:05:29.260 INFO:tasks.ceph.mds.a:Started 2024-04-24T23:05:29.260 DEBUG:teuthology.parallel:result is None 2024-04-24T23:05:29.261 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-24T23:05:29.346 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2024-04-24T23:05:29.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.374+0000 7fd40ba1c640 1 Processor -- start 2024-04-24T23:05:29.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.374+0000 7fd40ba1c640 1 -- start start 2024-04-24T23:05:29.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.375+0000 7fd40ba1c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd4040b5af0 0x7fd4040ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:29.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.375+0000 7fd40ba1c640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd404162e10 0x7fd40415ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:29.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.375+0000 7fd40ba1c640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4040cd270 con 0x7fd404162320 2024-04-24T23:05:29.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.375+0000 7fd40ba1c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fd4040b5660 con 0x7fd4040b5af0 2024-04-24T23:05:29.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.375+0000 7fd40ba1c640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fd404071610 con 0x7fd404162e10 2024-04-24T23:05:29.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.375+0000 7fd408fb7640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd4040b5af0 0x7fd4040ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:29.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.375+0000 7fd4097b8640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd404162320 0x7fd404162700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53156/0 (socket says 172.21.15.32:53156) 2024-04-24T23:05:29.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.375+0000 7fd408fb7640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd4040b5af0 0x7fd4040ccc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49804/0 (socket says 172.21.15.32:49804) 2024-04-24T23:05:29.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.375+0000 7fd4097b8640 1 -- 172.21.15.32:0/757041519 learned_addr learned my addr 172.21.15.32:0/757041519 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:05:29.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.375+0000 7fd409fb9640 1 --2- 172.21.15.32:0/757041519 >> v2:172.21.15.121:3300/0 conn(0x7fd404162e10 0x7fd40415ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:29.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.375+0000 7fd408fb7640 1 -- 172.21.15.32:0/757041519 >> v2:172.21.15.121:3300/0 conn(0x7fd404162e10 msgr2=0x7fd40415ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:29.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.375+0000 7fd408fb7640 1 --2- 172.21.15.32:0/757041519 >> v2:172.21.15.121:3300/0 conn(0x7fd404162e10 0x7fd40415ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:29.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.375+0000 7fd408fb7640 1 -- 172.21.15.32:0/757041519 >> v1:172.21.15.32:6789/0 conn(0x7fd404162320 legacy=0x7fd404162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:29.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.375+0000 7fd3fbfff640 1 -- 172.21.15.32:0/757041519 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 540740729 0 0) 0x7fd4040cd270 con 0x7fd404162320 2024-04-24T23:05:29.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.375+0000 7fd408fb7640 1 -- 172.21.15.32:0/757041519 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4040718f0 con 0x7fd4040b5af0 2024-04-24T23:05:29.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.376+0000 7fd408fb7640 1 --2- 172.21.15.32:0/757041519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4040b5af0 0x7fd4040ccc90 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fd4040d1bc0 tx=0x7fd3f402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2729661801367668 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:29.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.376+0000 7fd3fbfff640 1 -- 172.21.15.32:0/757041519 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f4037070 con 0x7fd4040b5af0 2024-04-24T23:05:29.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.376+0000 7fd3fbfff640 1 -- 172.21.15.32:0/757041519 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3f4032440 con 0x7fd4040b5af0 2024-04-24T23:05:29.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.376+0000 7fd3fbfff640 1 -- 172.21.15.32:0/757041519 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f402ae60 con 0x7fd4040b5af0 2024-04-24T23:05:29.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.377+0000 7fd40ba1c640 1 -- 172.21.15.32:0/757041519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4040b5af0 msgr2=0x7fd4040ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:29.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.377+0000 7fd40ba1c640 1 --2- 172.21.15.32:0/757041519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4040b5af0 0x7fd4040ccc90 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fd4040d1bc0 tx=0x7fd3f402a110 comp rx=0 tx=0).stop 2024-04-24T23:05:29.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.377+0000 7fd40ba1c640 1 -- 172.21.15.32:0/757041519 shutdown_connections 2024-04-24T23:05:29.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.377+0000 7fd40ba1c640 1 --2- 172.21.15.32:0/757041519 >> v2:172.21.15.121:3300/0 conn(0x7fd404162e10 0x7fd40415ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:29.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.377+0000 7fd40ba1c640 1 --2- 172.21.15.32:0/757041519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4040b5af0 0x7fd4040ccc90 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:29.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.377+0000 7fd40ba1c640 1 -- 172.21.15.32:0/757041519 >> 172.21.15.32:0/757041519 conn(0x7fd4040d0ae0 msgr2=0x7fd4040d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:29.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.377+0000 7fd40ba1c640 1 -- 172.21.15.32:0/757041519 shutdown_connections 2024-04-24T23:05:29.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.377+0000 7fd40ba1c640 1 -- 172.21.15.32:0/757041519 wait complete. 2024-04-24T23:05:29.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.378+0000 7fd40ba1c640 1 Processor -- start 2024-04-24T23:05:29.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.379+0000 7fd40ba1c640 1 -- start start 2024-04-24T23:05:29.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.379+0000 7fd40ba1c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4040b5af0 0x7fd4040cf400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:29.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.379+0000 7fd40ba1c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd404162320 0x7fd4041e95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:29.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.379+0000 7fd40ba1c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd404162e10 0x7fd40415b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:29.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.379+0000 7fd40ba1c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd404071bd0 con 0x7fd404162e10 2024-04-24T23:05:29.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.379+0000 7fd40ba1c640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd404067950 con 0x7fd4040b5af0 2024-04-24T23:05:29.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.379+0000 7fd40ba1c640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd404071d50 con 0x7fd404162320 2024-04-24T23:05:29.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.380+0000 7fd409fb9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd404162e10 0x7fd40415b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:29.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.380+0000 7fd409fb9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd404162e10 0x7fd40415b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49812/0 (socket says 172.21.15.32:49812) 2024-04-24T23:05:29.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.380+0000 7fd409fb9640 1 -- 172.21.15.32:0/2049926808 learned_addr learned my addr 172.21.15.32:0/2049926808 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:29.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.380+0000 7fd408fb7640 1 --2- 172.21.15.32:0/2049926808 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd404162320 0x7fd4041e95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:29.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.380+0000 7fd4097b8640 1 --2- 172.21.15.32:0/2049926808 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4040b5af0 0x7fd4040cf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:29.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.380+0000 7fd409fb9640 1 -- 172.21.15.32:0/2049926808 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd404162320 msgr2=0x7fd4041e95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:29.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.380+0000 7fd409fb9640 1 --2- 172.21.15.32:0/2049926808 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd404162320 0x7fd4041e95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:29.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.380+0000 7fd409fb9640 1 -- 172.21.15.32:0/2049926808 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4040b5af0 msgr2=0x7fd4040cf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:29.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.380+0000 7fd409fb9640 1 --2- 172.21.15.32:0/2049926808 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4040b5af0 0x7fd4040cf400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:29.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.380+0000 7fd409fb9640 1 -- 172.21.15.32:0/2049926808 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4041e9b20 con 0x7fd404162e10 2024-04-24T23:05:29.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.380+0000 7fd409fb9640 1 --2- 172.21.15.32:0/2049926808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd404162e10 0x7fd40415b260 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc009d30 tx=0x7fd3fc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:29.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.381+0000 7fd3f9ffb640 1 -- 172.21.15.32:0/2049926808 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc018020 con 0x7fd404162e10 2024-04-24T23:05:29.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.381+0000 7fd40ba1c640 1 -- 172.21.15.32:0/2049926808 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd40415b7a0 con 0x7fd404162e10 2024-04-24T23:05:29.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.381+0000 7fd3f9ffb640 1 -- 172.21.15.32:0/2049926808 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3fc014480 con 0x7fd404162e10 2024-04-24T23:05:29.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.382+0000 7fd40ba1c640 1 -- 172.21.15.32:0/2049926808 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4040cfe10 con 0x7fd404162e10 2024-04-24T23:05:29.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.382+0000 7fd3f9ffb640 1 -- 172.21.15.32:0/2049926808 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc014830 con 0x7fd404162e10 2024-04-24T23:05:29.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.383+0000 7fd40ba1c640 1 -- 172.21.15.32:0/2049926808 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd404068430 con 0x7fd404162e10 2024-04-24T23:05:29.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.383+0000 7fd3f9ffb640 1 -- 172.21.15.32:0/2049926808 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd3fc005d10 con 0x7fd404162e10 2024-04-24T23:05:29.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.383+0000 7fd3f9ffb640 1 --2- 172.21.15.32:0/2049926808 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd3e806be30 0x7fd3e806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:29.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.384+0000 7fd3f9ffb640 1 -- 172.21.15.32:0/2049926808 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 4943+0+0 (secure 0 0 0) 0x7fd3fc08b420 con 0x7fd404162e10 2024-04-24T23:05:29.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.384+0000 7fd4097b8640 1 --2- 172.21.15.32:0/2049926808 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd3e806be30 0x7fd3e806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:29.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.384+0000 7fd4097b8640 1 --2- 172.21.15.32:0/2049926808 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd3e806be30 0x7fd3e806e2f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fd400000f60 tx=0x7fd400007d50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:29.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.388+0000 7fd3f9ffb640 1 -- 172.21.15.32:0/2049926808 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd3fc022070 con 0x7fd404162e10 2024-04-24T23:05:29.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:29.584+0000 7fd40ba1c640 1 -- 172.21.15.32:0/2049926808 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fd404067730 con 0x7fd404162e10 2024-04-24T23:05:30.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.393+0000 7fd3f9ffb640 1 -- 172.21.15.32:0/2049926808 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v136) ==== 97+0+0 (secure 0 0 0) 0x7fd3fc010040 con 0x7fd404162e10 2024-04-24T23:05:30.395 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blocklist entries 2024-04-24T23:05:30.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.396+0000 7fd40ba1c640 1 -- 172.21.15.32:0/2049926808 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd3e806be30 msgr2=0x7fd3e806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:30.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.396+0000 7fd40ba1c640 1 --2- 172.21.15.32:0/2049926808 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd3e806be30 0x7fd3e806e2f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fd400000f60 tx=0x7fd400007d50 comp rx=0 tx=0).stop 2024-04-24T23:05:30.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.396+0000 7fd40ba1c640 1 -- 172.21.15.32:0/2049926808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd404162e10 msgr2=0x7fd40415b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:30.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.396+0000 7fd40ba1c640 1 --2- 172.21.15.32:0/2049926808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd404162e10 0x7fd40415b260 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc009d30 tx=0x7fd3fc00c490 comp rx=0 tx=0).stop 2024-04-24T23:05:30.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.396+0000 7fd40ba1c640 1 -- 172.21.15.32:0/2049926808 shutdown_connections 2024-04-24T23:05:30.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.396+0000 7fd40ba1c640 1 --2- 172.21.15.32:0/2049926808 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd3e806be30 0x7fd3e806e2f0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:30.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.396+0000 7fd40ba1c640 1 --2- 172.21.15.32:0/2049926808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd404162e10 0x7fd40415b260 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:30.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.396+0000 7fd40ba1c640 1 --2- 172.21.15.32:0/2049926808 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd404162320 0x7fd4041e95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:30.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.396+0000 7fd40ba1c640 1 --2- 172.21.15.32:0/2049926808 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4040b5af0 0x7fd4040cf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:30.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.396+0000 7fd40ba1c640 1 -- 172.21.15.32:0/2049926808 >> 172.21.15.32:0/2049926808 conn(0x7fd4040d0ae0 msgr2=0x7fd4040b0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:30.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.396+0000 7fd40ba1c640 1 -- 172.21.15.32:0/2049926808 shutdown_connections 2024-04-24T23:05:30.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.397+0000 7fd40ba1c640 1 -- 172.21.15.32:0/2049926808 wait complete. 2024-04-24T23:05:30.412 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-24T23:05:30.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.517+0000 7efd8f0c3640 1 Processor -- start 2024-04-24T23:05:30.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.517+0000 7efd8f0c3640 1 -- start start 2024-04-24T23:05:30.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.518+0000 7efd8f0c3640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efd88162320 0x7efd880ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:30.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.518+0000 7efd8f0c3640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7efd880b5af0 0x7efd88162700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:30.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.518+0000 7efd8f0c3640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efd88162c40 0x7efd880ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:30.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.518+0000 7efd8f0c3640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7efd880b5660 con 0x7efd88162320 2024-04-24T23:05:30.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.518+0000 7efd8f0c3640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7efd880cd270 con 0x7efd880b5af0 2024-04-24T23:05:30.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.518+0000 7efd8f0c3640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7efd88071610 con 0x7efd88162c40 2024-04-24T23:05:30.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.518+0000 7efd8ce5f640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efd88162320 0x7efd880ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:30.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.518+0000 7efd8ce5f640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efd88162320 0x7efd880ccc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56154/0 (socket says 172.21.15.32:56154) 2024-04-24T23:05:30.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.518+0000 7efd8ce5f640 1 -- 172.21.15.32:0/1039284185 learned_addr learned my addr 172.21.15.32:0/1039284185 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:30.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.518+0000 7efd87fff640 1 --2- 172.21.15.32:0/1039284185 >> v2:172.21.15.121:3300/0 conn(0x7efd880b5af0 0x7efd88162700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:30.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.518+0000 7efd8d660640 1 --2- 172.21.15.32:0/1039284185 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efd88162c40 0x7efd880ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:30.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.518+0000 7efd8ce5f640 1 -- 172.21.15.32:0/1039284185 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efd88162c40 msgr2=0x7efd880ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:30.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.518+0000 7efd8ce5f640 1 --2- 172.21.15.32:0/1039284185 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efd88162c40 0x7efd880ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:30.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.518+0000 7efd8ce5f640 1 -- 172.21.15.32:0/1039284185 >> v2:172.21.15.121:3300/0 conn(0x7efd880b5af0 msgr2=0x7efd88162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:30.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.518+0000 7efd8ce5f640 1 --2- 172.21.15.32:0/1039284185 >> v2:172.21.15.121:3300/0 conn(0x7efd880b5af0 0x7efd88162700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:30.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.518+0000 7efd8ce5f640 1 -- 172.21.15.32:0/1039284185 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd880718f0 con 0x7efd88162320 2024-04-24T23:05:30.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.518+0000 7efd87fff640 1 --2- 172.21.15.32:0/1039284185 >> v2:172.21.15.121:3300/0 conn(0x7efd880b5af0 0x7efd88162700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:05:30.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.519+0000 7efd8ce5f640 1 --2- 172.21.15.32:0/1039284185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd88162320 0x7efd880ccc90 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7efd880d1bc0 tx=0x7efd7c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97883fcd060abf3c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:30.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.519+0000 7efd877fe640 1 -- 172.21.15.32:0/1039284185 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7c037070 con 0x7efd88162320 2024-04-24T23:05:30.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.519+0000 7efd877fe640 1 -- 172.21.15.32:0/1039284185 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd7c032440 con 0x7efd88162320 2024-04-24T23:05:30.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.519+0000 7efd877fe640 1 -- 172.21.15.32:0/1039284185 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7c032810 con 0x7efd88162320 2024-04-24T23:05:30.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.520+0000 7efd8f0c3640 1 -- 172.21.15.32:0/1039284185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd88162320 msgr2=0x7efd880ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:30.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.520+0000 7efd8f0c3640 1 --2- 172.21.15.32:0/1039284185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd88162320 0x7efd880ccc90 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7efd880d1bc0 tx=0x7efd7c02a140 comp rx=0 tx=0).stop 2024-04-24T23:05:30.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.520+0000 7efd8f0c3640 1 -- 172.21.15.32:0/1039284185 shutdown_connections 2024-04-24T23:05:30.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.520+0000 7efd8f0c3640 1 --2- 172.21.15.32:0/1039284185 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efd88162c40 0x7efd880ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:30.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.520+0000 7efd8f0c3640 1 --2- 172.21.15.32:0/1039284185 >> v2:172.21.15.121:3300/0 conn(0x7efd880b5af0 0x7efd88162700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:30.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.520+0000 7efd8f0c3640 1 --2- 172.21.15.32:0/1039284185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd88162320 0x7efd880ccc90 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:30.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.520+0000 7efd8f0c3640 1 -- 172.21.15.32:0/1039284185 >> 172.21.15.32:0/1039284185 conn(0x7efd880d0ae0 msgr2=0x7efd880d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:30.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.520+0000 7efd8f0c3640 1 -- 172.21.15.32:0/1039284185 shutdown_connections 2024-04-24T23:05:30.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.520+0000 7efd8f0c3640 1 -- 172.21.15.32:0/1039284185 wait complete. 2024-04-24T23:05:30.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.521+0000 7efd8f0c3640 1 Processor -- start 2024-04-24T23:05:30.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.521+0000 7efd8f0c3640 1 -- start start 2024-04-24T23:05:30.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.522+0000 7efd8f0c3640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efd880b5af0 0x7efd88153950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:30.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.522+0000 7efd8f0c3640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efd88162320 0x7efd8815a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:30.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.522+0000 7efd8f0c3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd88162c40 0x7efd8815ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:30.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.522+0000 7efd8f0c3640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7efd88067950 con 0x7efd88162c40 2024-04-24T23:05:30.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.522+0000 7efd8f0c3640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7efd8815c630 con 0x7efd88162320 2024-04-24T23:05:30.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.522+0000 7efd8f0c3640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7efd88071bd0 con 0x7efd880b5af0 2024-04-24T23:05:30.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.522+0000 7efd8d660640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd88162c40 0x7efd8815ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:30.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.522+0000 7efd8d660640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd88162c40 0x7efd8815ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56160/0 (socket says 172.21.15.32:56160) 2024-04-24T23:05:30.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.522+0000 7efd8d660640 1 -- 172.21.15.32:0/4038671595 learned_addr learned my addr 172.21.15.32:0/4038671595 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:30.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.522+0000 7efd8ce5f640 1 --2- 172.21.15.32:0/4038671595 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efd880b5af0 0x7efd88153950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:30.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.522+0000 7efd87fff640 1 --2- 172.21.15.32:0/4038671595 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efd88162320 0x7efd8815a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:30.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.522+0000 7efd8d660640 1 -- 172.21.15.32:0/4038671595 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efd880b5af0 msgr2=0x7efd88153950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:30.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.522+0000 7efd8d660640 1 --2- 172.21.15.32:0/4038671595 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efd880b5af0 0x7efd88153950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:30.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.522+0000 7efd8d660640 1 -- 172.21.15.32:0/4038671595 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efd88162320 msgr2=0x7efd8815a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:30.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.522+0000 7efd8d660640 1 --2- 172.21.15.32:0/4038671595 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efd88162320 0x7efd8815a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:30.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.522+0000 7efd8d660640 1 -- 172.21.15.32:0/4038671595 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd88155dc0 con 0x7efd88162c40 2024-04-24T23:05:30.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.522+0000 7efd87fff640 1 --2- 172.21.15.32:0/4038671595 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efd88162320 0x7efd8815a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:05:30.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.522+0000 7efd8d660640 1 --2- 172.21.15.32:0/4038671595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd88162c40 0x7efd8815ad70 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7efd80007e40 tx=0x7efd8000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:30.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.523+0000 7efd857fa640 1 -- 172.21.15.32:0/4038671595 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd80014620 con 0x7efd88162c40 2024-04-24T23:05:30.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.523+0000 7efd857fa640 1 -- 172.21.15.32:0/4038671595 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd80019070 con 0x7efd88162c40 2024-04-24T23:05:30.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.523+0000 7efd8f0c3640 1 -- 172.21.15.32:0/4038671595 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd88155ae0 con 0x7efd88162c40 2024-04-24T23:05:30.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.523+0000 7efd857fa640 1 -- 172.21.15.32:0/4038671595 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd80014a20 con 0x7efd88162c40 2024-04-24T23:05:30.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.523+0000 7efd8f0c3640 1 -- 172.21.15.32:0/4038671595 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd881e9950 con 0x7efd88162c40 2024-04-24T23:05:30.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.524+0000 7efd8f0c3640 1 -- 172.21.15.32:0/4038671595 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd88068430 con 0x7efd88162c40 2024-04-24T23:05:30.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.524+0000 7efd857fa640 1 -- 172.21.15.32:0/4038671595 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efd80014bc0 con 0x7efd88162c40 2024-04-24T23:05:30.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.525+0000 7efd857fa640 1 --2- 172.21.15.32:0/4038671595 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7efd6c06be30 0x7efd6c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:30.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.525+0000 7efd857fa640 1 -- 172.21.15.32:0/4038671595 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 4857+0+0 (secure 0 0 0) 0x7efd8008b310 con 0x7efd88162c40 2024-04-24T23:05:30.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.525+0000 7efd8ce5f640 1 --2- 172.21.15.32:0/4038671595 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7efd6c06be30 0x7efd6c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:30.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.525+0000 7efd8ce5f640 1 --2- 172.21.15.32:0/4038671595 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7efd6c06be30 0x7efd6c06e2f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7efd7c034000 tx=0x7efd7c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:30.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.529+0000 7efd857fa640 1 -- 172.21.15.32:0/4038671595 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7efd8005a080 con 0x7efd88162c40 2024-04-24T23:05:30.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.783+0000 7efd8f0c3640 1 -- 172.21.15.32:0/4038671595 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7efd88067730 con 0x7efd88162c40 2024-04-24T23:05:30.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.783+0000 7efd857fa640 1 -- 172.21.15.32:0/4038671595 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) ==== 80+0+5304 (secure 0 0 0) 0x7efd80059980 con 0x7efd88162c40 2024-04-24T23:05:30.785 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:05:30.785 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:05:30.785 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2024-04-24T23:05:30.785 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:30.785 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2024-04-24T23:05:30.785 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmXRdxAxAA/EYEn1C38XhG7DC+0X4y7A==", 2024-04-24T23:05:30.785 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:05:30.785 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:05:30.785 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-24T23:05:30.785 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-24T23:05:30.785 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:05:30.785 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:30.786 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:30.786 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:30.786 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2024-04-24T23:05:30.786 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmak7bDhAA0he+Jp/gkkrfBjYKSObjcw==", 2024-04-24T23:05:30.786 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:05:30.786 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:05:30.786 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-24T23:05:30.786 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-24T23:05:30.786 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:05:30.786 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:30.786 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:30.786 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:30.786 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.c", 2024-04-24T23:05:30.786 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmZXFdChAA8TwwkA5EIXHBzPsQSFvXcQ==", 2024-04-24T23:05:30.787 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:05:30.787 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:05:30.787 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-24T23:05:30.787 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-24T23:05:30.787 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:05:30.787 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:30.787 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:30.787 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:30.787 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.d", 2024-04-24T23:05:30.787 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmLbGuFRAAe91fZlulziWdWiDdLBLWNA==", 2024-04-24T23:05:30.787 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:05:30.787 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:05:30.787 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-24T23:05:30.787 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-24T23:05:30.787 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:05:30.788 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:30.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:30.788 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:30.788 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2024-04-24T23:05:30.788 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCfjilmhTwoOBAAig98xa3a2qnv4rriUO9VCQ==", 2024-04-24T23:05:30.788 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:05:30.788 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:05:30.788 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:05:30.788 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:05:30.788 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:30.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:30.788 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:30.788 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2024-04-24T23:05:30.788 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCgjilmwRaAAxAAUqDOyR9HV6DEnmbyn3WAjA==", 2024-04-24T23:05:30.788 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:05:30.789 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:05:30.789 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:05:30.789 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:05:30.789 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:30.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:30.789 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:30.789 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2024-04-24T23:05:30.789 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCgjilmDl2qDBAAF1bhTOaSOc612wmFRd8+XA==", 2024-04-24T23:05:30.789 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:05:30.789 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:05:30.789 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:05:30.789 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:05:30.789 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:30.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:30.789 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:30.790 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2024-04-24T23:05:30.790 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCgjilmAxHwFRAA9a0v/YwngRxq1vmMRsmEkQ==", 2024-04-24T23:05:30.790 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:05:30.790 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:05:30.790 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:05:30.790 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:05:30.790 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:30.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:30.790 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:30.790 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2024-04-24T23:05:30.790 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCjjilmDPmJIxAASZlEo1ZJHOhIAFGukIj1tQ==", 2024-04-24T23:05:30.790 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:05:30.790 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:05:30.790 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:05:30.791 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:05:30.791 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:30.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:30.791 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:30.791 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2024-04-24T23:05:30.791 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCjjilmgcpuKhAAedSq0ubNB30s3bz0fY38hw==", 2024-04-24T23:05:30.791 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:05:30.791 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:05:30.791 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:05:30.791 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:05:30.791 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:30.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:30.791 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:30.791 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2024-04-24T23:05:30.791 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCjjilmFv2VMxAAM+mUc4I0sjawy2uwuUEKYg==", 2024-04-24T23:05:30.791 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:05:30.792 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:05:30.792 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:05:30.792 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:05:30.792 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:30.792 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:30.792 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:30.792 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2024-04-24T23:05:30.792 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCkjilmLvEbARAAnaOvQ5Ee0clN/A9SVoKuBQ==", 2024-04-24T23:05:30.792 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:05:30.792 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:05:30.792 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:05:30.792 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:05:30.792 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:30.792 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:30.792 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:30.793 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2024-04-24T23:05:30.793 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilm5oeiGRAAdLXwwqrYhV3HHo7Dw/2ZqQ==", 2024-04-24T23:05:30.793 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:05:30.793 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:05:30.793 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2024-04-24T23:05:30.793 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-24T23:05:30.793 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:30.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:30.793 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:30.793 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2024-04-24T23:05:30.793 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmyWBDIhAARtVP4Unk3HHlQYZ2QZ3GzQ==", 2024-04-24T23:05:30.793 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:05:30.793 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:05:30.793 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-24T23:05:30.793 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-24T23:05:30.793 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-24T23:05:30.794 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:30.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:30.794 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:30.794 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.2", 2024-04-24T23:05:30.794 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilm4QElHxAAXgE316aRCosxX7XER5gtKQ==", 2024-04-24T23:05:30.794 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:05:30.794 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:05:30.794 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-24T23:05:30.794 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-24T23:05:30.794 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-24T23:05:30.794 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:30.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:30.794 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:30.794 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.3", 2024-04-24T23:05:30.794 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmMZihJxAAoWaAAFY3vVfZnC4/GoT0+g==", 2024-04-24T23:05:30.795 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:05:30.795 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:05:30.795 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-24T23:05:30.795 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-24T23:05:30.795 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-24T23:05:30.795 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:30.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:30.795 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:30.795 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2024-04-24T23:05:30.795 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCdjilmdVXeIxAAwGGaAfshx4pHcoc5+3kRww==", 2024-04-24T23:05:30.795 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:05:30.795 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2024-04-24T23:05:30.795 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-24T23:05:30.795 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-24T23:05:30.795 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:05:30.796 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:30.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:30.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:30.796 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2024-04-24T23:05:30.796 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmn3FJABAAGnvNAoNLzK20cQF6beihmw==", 2024-04-24T23:05:30.796 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:05:30.796 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2024-04-24T23:05:30.796 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2024-04-24T23:05:30.796 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:05:30.796 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:30.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:30.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:30.796 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2024-04-24T23:05:30.796 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCdjilmMqs2OBAAOe3bLpnCq+HK2EbeXnwY8Q==", 2024-04-24T23:05:30.796 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:05:30.796 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2024-04-24T23:05:30.797 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2024-04-24T23:05:30.797 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:05:30.797 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:30.797 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:30.797 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:30.797 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:05:30.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.785+0000 7efd8f0c3640 1 -- 172.21.15.32:0/4038671595 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7efd6c06be30 msgr2=0x7efd6c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:30.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.785+0000 7efd8f0c3640 1 --2- 172.21.15.32:0/4038671595 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7efd6c06be30 0x7efd6c06e2f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7efd7c034000 tx=0x7efd7c02e000 comp rx=0 tx=0).stop 2024-04-24T23:05:30.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.785+0000 7efd8f0c3640 1 -- 172.21.15.32:0/4038671595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd88162c40 msgr2=0x7efd8815ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:30.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.785+0000 7efd8f0c3640 1 --2- 172.21.15.32:0/4038671595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd88162c40 0x7efd8815ad70 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7efd80007e40 tx=0x7efd8000e4b0 comp rx=0 tx=0).stop 2024-04-24T23:05:30.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.785+0000 7efd8f0c3640 1 -- 172.21.15.32:0/4038671595 shutdown_connections 2024-04-24T23:05:30.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.785+0000 7efd8f0c3640 1 --2- 172.21.15.32:0/4038671595 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7efd6c06be30 0x7efd6c06e2f0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:30.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.785+0000 7efd8f0c3640 1 --2- 172.21.15.32:0/4038671595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd88162c40 0x7efd8815ad70 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:30.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.785+0000 7efd8f0c3640 1 --2- 172.21.15.32:0/4038671595 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efd88162320 0x7efd8815a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:30.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.785+0000 7efd8f0c3640 1 --2- 172.21.15.32:0/4038671595 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efd880b5af0 0x7efd88153950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:30.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.785+0000 7efd8f0c3640 1 -- 172.21.15.32:0/4038671595 >> 172.21.15.32:0/4038671595 conn(0x7efd880d0ae0 msgr2=0x7efd880b1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:30.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.785+0000 7efd8f0c3640 1 -- 172.21.15.32:0/4038671595 shutdown_connections 2024-04-24T23:05:30.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.785+0000 7efd8f0c3640 1 -- 172.21.15.32:0/4038671595 wait complete. 2024-04-24T23:05:30.800 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:05:30.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.898+0000 7f1e894d4640 1 Processor -- start 2024-04-24T23:05:30.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.899+0000 7f1e894d4640 1 -- start start 2024-04-24T23:05:30.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.899+0000 7f1e894d4640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1e8406c5f0 0x7f1e8406c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:30.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.899+0000 7f1e894d4640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1e840af890 0x7f1e8415cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:30.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.899+0000 7f1e894d4640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e840b6430 con 0x7f1e840b44d0 2024-04-24T23:05:30.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.899+0000 7f1e894d4640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f1e84156f80 con 0x7f1e840af890 2024-04-24T23:05:30.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.899+0000 7f1e894d4640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f1e840b6150 con 0x7f1e8406c5f0 2024-04-24T23:05:30.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.899+0000 7f1e837fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1e840af890 0x7f1e8415cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:30.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.899+0000 7f1e827fc640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1e840b44d0 0x7f1e840b5890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37798/0 (socket says 172.21.15.32:37798) 2024-04-24T23:05:30.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.900+0000 7f1e827fc640 1 -- 172.21.15.32:0/1146330732 learned_addr learned my addr 172.21.15.32:0/1146330732 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:05:30.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.900+0000 7f1e82ffd640 1 --2- 172.21.15.32:0/1146330732 >> v2:172.21.15.121:3300/0 conn(0x7f1e8406c5f0 0x7f1e8406c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:30.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.900+0000 7f1e837fe640 1 -- 172.21.15.32:0/1146330732 >> v2:172.21.15.121:3300/0 conn(0x7f1e8406c5f0 msgr2=0x7f1e8406c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:30.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.900+0000 7f1e837fe640 1 --2- 172.21.15.32:0/1146330732 >> v2:172.21.15.121:3300/0 conn(0x7f1e8406c5f0 0x7f1e8406c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:30.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.900+0000 7f1e837fe640 1 -- 172.21.15.32:0/1146330732 >> v1:172.21.15.32:6789/0 conn(0x7f1e840b44d0 legacy=0x7f1e840b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:30.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.900+0000 7f1e837fe640 1 -- 172.21.15.32:0/1146330732 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e840b39f0 con 0x7f1e840af890 2024-04-24T23:05:30.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.900+0000 7f1e81ffb640 1 -- 172.21.15.32:0/1146330732 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 852942768 0 0) 0x7f1e840b6430 con 0x7f1e840b44d0 2024-04-24T23:05:30.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.901+0000 7f1e837fe640 1 --2- 172.21.15.32:0/1146330732 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e840af890 0x7f1e8415cac0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f1e8406c1f0 tx=0x7f1e7402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9a8e9525a3c191a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:30.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.901+0000 7f1e81ffb640 1 -- 172.21.15.32:0/1146330732 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e74007740 con 0x7f1e840af890 2024-04-24T23:05:30.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.901+0000 7f1e81ffb640 1 -- 172.21.15.32:0/1146330732 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e740078e0 con 0x7f1e840af890 2024-04-24T23:05:30.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.901+0000 7f1e81ffb640 1 -- 172.21.15.32:0/1146330732 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e74007cb0 con 0x7f1e840af890 2024-04-24T23:05:30.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.901+0000 7f1e894d4640 1 -- 172.21.15.32:0/1146330732 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e840af890 msgr2=0x7f1e8415cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:30.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.901+0000 7f1e894d4640 1 --2- 172.21.15.32:0/1146330732 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e840af890 0x7f1e8415cac0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f1e8406c1f0 tx=0x7f1e7402a680 comp rx=0 tx=0).stop 2024-04-24T23:05:30.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.901+0000 7f1e894d4640 1 -- 172.21.15.32:0/1146330732 shutdown_connections 2024-04-24T23:05:30.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.901+0000 7f1e894d4640 1 --2- 172.21.15.32:0/1146330732 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e840af890 0x7f1e8415cac0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:30.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.901+0000 7f1e894d4640 1 --2- 172.21.15.32:0/1146330732 >> v2:172.21.15.121:3300/0 conn(0x7f1e8406c5f0 0x7f1e8406c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:30.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.901+0000 7f1e894d4640 1 -- 172.21.15.32:0/1146330732 >> 172.21.15.32:0/1146330732 conn(0x7f1e8406b110 msgr2=0x7f1e8406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:30.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.901+0000 7f1e894d4640 1 -- 172.21.15.32:0/1146330732 shutdown_connections 2024-04-24T23:05:30.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.901+0000 7f1e894d4640 1 -- 172.21.15.32:0/1146330732 wait complete. 2024-04-24T23:05:30.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.903+0000 7f1e894d4640 1 Processor -- start 2024-04-24T23:05:30.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.903+0000 7f1e894d4640 1 -- start start 2024-04-24T23:05:30.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.903+0000 7f1e894d4640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e8406c5f0 0x7f1e841530f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:30.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.903+0000 7f1e894d4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e840af890 0x7f1e8414ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:30.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.903+0000 7f1e894d4640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1e840b44d0 0x7f1e840ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:30.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.903+0000 7f1e894d4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8406cf10 con 0x7f1e840af890 2024-04-24T23:05:30.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.903+0000 7f1e894d4640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1e840b3fb0 con 0x7f1e8406c5f0 2024-04-24T23:05:30.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.903+0000 7f1e894d4640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f1e8415d000 con 0x7f1e840b44d0 2024-04-24T23:05:30.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.903+0000 7f1e827fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e840af890 0x7f1e8414ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:30.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.903+0000 7f1e827fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e840af890 0x7f1e8414ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56182/0 (socket says 172.21.15.32:56182) 2024-04-24T23:05:30.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.903+0000 7f1e827fc640 1 -- 172.21.15.32:0/3401007751 learned_addr learned my addr 172.21.15.32:0/3401007751 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:30.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.904+0000 7f1e837fe640 1 --2- 172.21.15.32:0/3401007751 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1e840b44d0 0x7f1e840ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:30.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.904+0000 7f1e82ffd640 1 --2- 172.21.15.32:0/3401007751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e8406c5f0 0x7f1e841530f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:30.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.904+0000 7f1e827fc640 1 -- 172.21.15.32:0/3401007751 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1e840b44d0 msgr2=0x7f1e840ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:30.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.904+0000 7f1e827fc640 1 --2- 172.21.15.32:0/3401007751 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1e840b44d0 0x7f1e840ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:30.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.904+0000 7f1e827fc640 1 -- 172.21.15.32:0/3401007751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e8406c5f0 msgr2=0x7f1e841530f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:30.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.904+0000 7f1e827fc640 1 --2- 172.21.15.32:0/3401007751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e8406c5f0 0x7f1e841530f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:30.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.904+0000 7f1e827fc640 1 -- 172.21.15.32:0/3401007751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e84153630 con 0x7f1e840af890 2024-04-24T23:05:30.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.904+0000 7f1e82ffd640 1 --2- 172.21.15.32:0/3401007751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e8406c5f0 0x7f1e841530f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:05:30.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.904+0000 7f1e827fc640 1 --2- 172.21.15.32:0/3401007751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e840af890 0x7f1e8414ddd0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f1e7800c390 tx=0x7f1e7800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:30.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.904+0000 7f1e6b7fe640 1 -- 172.21.15.32:0/3401007751 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e780074e0 con 0x7f1e840af890 2024-04-24T23:05:30.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.904+0000 7f1e6b7fe640 1 -- 172.21.15.32:0/3401007751 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e780047e0 con 0x7f1e840af890 2024-04-24T23:05:30.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.904+0000 7f1e894d4640 1 -- 172.21.15.32:0/3401007751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e841f7730 con 0x7f1e840af890 2024-04-24T23:05:30.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.904+0000 7f1e6b7fe640 1 -- 172.21.15.32:0/3401007751 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e78004bd0 con 0x7f1e840af890 2024-04-24T23:05:30.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.904+0000 7f1e894d4640 1 -- 172.21.15.32:0/3401007751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e841f7cf0 con 0x7f1e840af890 2024-04-24T23:05:30.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.905+0000 7f1e894d4640 1 -- 172.21.15.32:0/3401007751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e84068430 con 0x7f1e840af890 2024-04-24T23:05:30.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.906+0000 7f1e6b7fe640 1 -- 172.21.15.32:0/3401007751 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1e78011070 con 0x7f1e840af890 2024-04-24T23:05:30.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.906+0000 7f1e6b7fe640 1 --2- 172.21.15.32:0/3401007751 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1e7006be30 0x7f1e7006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:30.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.906+0000 7f1e6b7fe640 1 -- 172.21.15.32:0/3401007751 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 4857+0+0 (secure 0 0 0) 0x7f1e7808a610 con 0x7f1e840af890 2024-04-24T23:05:30.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.906+0000 7f1e82ffd640 1 --2- 172.21.15.32:0/3401007751 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1e7006be30 0x7f1e7006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:30.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.907+0000 7f1e82ffd640 1 --2- 172.21.15.32:0/3401007751 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1e7006be30 0x7f1e7006e2f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c006fd0 tx=0x7f1e7c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:30.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:30.910+0000 7f1e6b7fe640 1 -- 172.21.15.32:0/3401007751 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1e780593a0 con 0x7f1e840af890 2024-04-24T23:05:31.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.098+0000 7f1e894d4640 1 -- 172.21.15.32:0/3401007751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1e840b5670 con 0x7f1e840af890 2024-04-24T23:05:31.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.099+0000 7f1e6b7fe640 1 -- 172.21.15.32:0/3401007751 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) ==== 74+0+12914 (secure 0 0 0) 0x7f1e78058ca0 con 0x7f1e840af890 2024-04-24T23:05:31.100 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:05:31.100 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":136,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:05:30.389937+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:05:31.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.100+0000 7f1e894d4640 1 -- 172.21.15.32:0/3401007751 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1e7006be30 msgr2=0x7f1e7006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:31.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.100+0000 7f1e894d4640 1 --2- 172.21.15.32:0/3401007751 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1e7006be30 0x7f1e7006e2f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c006fd0 tx=0x7f1e7c008040 comp rx=0 tx=0).stop 2024-04-24T23:05:31.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.100+0000 7f1e894d4640 1 -- 172.21.15.32:0/3401007751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e840af890 msgr2=0x7f1e8414ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:31.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.100+0000 7f1e894d4640 1 --2- 172.21.15.32:0/3401007751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e840af890 0x7f1e8414ddd0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f1e7800c390 tx=0x7f1e7800c850 comp rx=0 tx=0).stop 2024-04-24T23:05:31.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.100+0000 7f1e894d4640 1 -- 172.21.15.32:0/3401007751 shutdown_connections 2024-04-24T23:05:31.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.100+0000 7f1e894d4640 1 --2- 172.21.15.32:0/3401007751 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1e7006be30 0x7f1e7006e2f0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:31.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.100+0000 7f1e894d4640 1 --2- 172.21.15.32:0/3401007751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e840af890 0x7f1e8414ddd0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:31.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.100+0000 7f1e894d4640 1 --2- 172.21.15.32:0/3401007751 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1e840b44d0 0x7f1e840ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:31.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.100+0000 7f1e894d4640 1 --2- 172.21.15.32:0/3401007751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e8406c5f0 0x7f1e841530f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:31.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.100+0000 7f1e894d4640 1 -- 172.21.15.32:0/3401007751 >> 172.21.15.32:0/3401007751 conn(0x7f1e8406b110 msgr2=0x7f1e8415d360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:31.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.101+0000 7f1e894d4640 1 -- 172.21.15.32:0/3401007751 shutdown_connections 2024-04-24T23:05:31.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.101+0000 7f1e894d4640 1 -- 172.21.15.32:0/3401007751 wait complete. 2024-04-24T23:05:31.113 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:05:31.114 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:05:31.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.215+0000 7f8dc48a5640 1 Processor -- start 2024-04-24T23:05:31.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.215+0000 7f8dc48a5640 1 -- start start 2024-04-24T23:05:31.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.216+0000 7f8dc48a5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8dc006b820 0x7f8dc00d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:31.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.216+0000 7f8dc48a5640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8dc00d3170 con 0x7f8dc00d4210 2024-04-24T23:05:31.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.216+0000 7f8dc48a5640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8dc00d3450 con 0x7f8dc015e9b0 2024-04-24T23:05:31.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.216+0000 7f8dc48a5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f8dc00d3730 con 0x7f8dc006b820 2024-04-24T23:05:31.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.216+0000 7f8db5d9b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8dc006b820 0x7f8dc00d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:31.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.216+0000 7f8dbed9d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8dc00d4210 0x7f8dc00b78f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37802/0 (socket says 172.21.15.32:37802) 2024-04-24T23:05:31.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.216+0000 7f8db5d9b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8dc006b820 0x7f8dc00d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56184/0 (socket says 172.21.15.32:56184) 2024-04-24T23:05:31.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.216+0000 7f8dbed9d640 1 -- 172.21.15.32:0/3229148098 learned_addr learned my addr 172.21.15.32:0/3229148098 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:05:31.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.216+0000 7f8db5d9b640 1 -- 172.21.15.32:0/3229148098 >> v1:172.21.15.121:6789/0 conn(0x7f8dc015e9b0 legacy=0x7f8dc006b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:31.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.216+0000 7f8db5d9b640 1 -- 172.21.15.32:0/3229148098 >> v1:172.21.15.32:6789/0 conn(0x7f8dc00d4210 legacy=0x7f8dc00b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:31.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.216+0000 7f8db5d9b640 1 -- 172.21.15.32:0/3229148098 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dc00d3a10 con 0x7f8dc006b820 2024-04-24T23:05:31.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.216+0000 7f8dbdd9b640 1 -- 172.21.15.32:0/3229148098 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4204648493 0 0) 0x7f8dc00d3170 con 0x7f8dc00d4210 2024-04-24T23:05:31.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.217+0000 7f8db5d9b640 1 --2- 172.21.15.32:0/3229148098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8dc006b820 0x7f8dc00d2b90 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f8da80097a0 tx=0x7f8da802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b331abc2375a48c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:31.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.217+0000 7f8dbdd9b640 1 -- 172.21.15.32:0/3229148098 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da8037070 con 0x7f8dc006b820 2024-04-24T23:05:31.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.217+0000 7f8dbdd9b640 1 -- 172.21.15.32:0/3229148098 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8da8032440 con 0x7f8dc006b820 2024-04-24T23:05:31.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.217+0000 7f8dbdd9b640 1 -- 172.21.15.32:0/3229148098 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da8032810 con 0x7f8dc006b820 2024-04-24T23:05:31.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.217+0000 7f8dc48a5640 1 -- 172.21.15.32:0/3229148098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8dc006b820 msgr2=0x7f8dc00d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:31.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.217+0000 7f8dc48a5640 1 --2- 172.21.15.32:0/3229148098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8dc006b820 0x7f8dc00d2b90 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f8da80097a0 tx=0x7f8da802a1a0 comp rx=0 tx=0).stop 2024-04-24T23:05:31.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.217+0000 7f8dc48a5640 1 -- 172.21.15.32:0/3229148098 shutdown_connections 2024-04-24T23:05:31.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.217+0000 7f8dc48a5640 1 --2- 172.21.15.32:0/3229148098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8dc006b820 0x7f8dc00d2b90 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:31.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.217+0000 7f8dc48a5640 1 -- 172.21.15.32:0/3229148098 >> 172.21.15.32:0/3229148098 conn(0x7f8dc00d11d0 msgr2=0x7f8dc00d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:31.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.218+0000 7f8dc48a5640 1 -- 172.21.15.32:0/3229148098 shutdown_connections 2024-04-24T23:05:31.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.218+0000 7f8dc48a5640 1 -- 172.21.15.32:0/3229148098 wait complete. 2024-04-24T23:05:31.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.219+0000 7f8dc48a5640 1 Processor -- start 2024-04-24T23:05:31.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.219+0000 7f8dc48a5640 1 -- start start 2024-04-24T23:05:31.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.220+0000 7f8dc48a5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8dc006b820 0x7f8dc015a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:31.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.220+0000 7f8dc48a5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8dc00d4210 0x7f8dc00ca620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:31.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.220+0000 7f8dc48a5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8dc015e9b0 0x7f8dc00ccd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:31.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.220+0000 7f8dc48a5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc00ba0e0 con 0x7f8dc015e9b0 2024-04-24T23:05:31.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.220+0000 7f8dc48a5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc00d2400 con 0x7f8dc006b820 2024-04-24T23:05:31.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.220+0000 7f8dc48a5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f8dc00ba260 con 0x7f8dc00d4210 2024-04-24T23:05:31.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.220+0000 7f8dbed9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8dc015e9b0 0x7f8dc00ccd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:31.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.220+0000 7f8dbed9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8dc015e9b0 0x7f8dc00ccd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56200/0 (socket says 172.21.15.32:56200) 2024-04-24T23:05:31.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.220+0000 7f8dbed9d640 1 -- 172.21.15.32:0/3111092871 learned_addr learned my addr 172.21.15.32:0/3111092871 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:31.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.220+0000 7f8dbe59c640 1 --2- 172.21.15.32:0/3111092871 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8dc006b820 0x7f8dc015a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:31.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.220+0000 7f8dbed9d640 1 -- 172.21.15.32:0/3111092871 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8dc00d4210 msgr2=0x7f8dc00ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:31.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.220+0000 7f8db5d9b640 1 --2- 172.21.15.32:0/3111092871 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8dc00d4210 0x7f8dc00ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:31.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.220+0000 7f8dbed9d640 1 --2- 172.21.15.32:0/3111092871 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8dc00d4210 0x7f8dc00ca620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:31.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.220+0000 7f8dbed9d640 1 -- 172.21.15.32:0/3111092871 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8dc006b820 msgr2=0x7f8dc015a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:31.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.220+0000 7f8dbed9d640 1 --2- 172.21.15.32:0/3111092871 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8dc006b820 0x7f8dc015a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:31.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.220+0000 7f8dbed9d640 1 -- 172.21.15.32:0/3111092871 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dc00c8320 con 0x7f8dc015e9b0 2024-04-24T23:05:31.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.220+0000 7f8dbe59c640 1 --2- 172.21.15.32:0/3111092871 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8dc006b820 0x7f8dc015a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:05:31.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.220+0000 7f8dbed9d640 1 --2- 172.21.15.32:0/3111092871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8dc015e9b0 0x7f8dc00ccd60 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f8db800c7a0 tx=0x7f8db800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:31.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.220+0000 7f8db77fe640 1 -- 172.21.15.32:0/3111092871 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db8018070 con 0x7f8dc015e9b0 2024-04-24T23:05:31.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.221+0000 7f8db77fe640 1 -- 172.21.15.32:0/3111092871 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8db800a640 con 0x7f8dc015e9b0 2024-04-24T23:05:31.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.221+0000 7f8db77fe640 1 -- 172.21.15.32:0/3111092871 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db800aab0 con 0x7f8dc015e9b0 2024-04-24T23:05:31.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.221+0000 7f8dc48a5640 1 -- 172.21.15.32:0/3111092871 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dc00cd2a0 con 0x7f8dc015e9b0 2024-04-24T23:05:31.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.221+0000 7f8dc48a5640 1 -- 172.21.15.32:0/3111092871 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dc00c85c0 con 0x7f8dc015e9b0 2024-04-24T23:05:31.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.222+0000 7f8dc48a5640 1 -- 172.21.15.32:0/3111092871 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dc0068430 con 0x7f8dc015e9b0 2024-04-24T23:05:31.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.222+0000 7f8db77fe640 1 -- 172.21.15.32:0/3111092871 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8db801b070 con 0x7f8dc015e9b0 2024-04-24T23:05:31.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.222+0000 7f8db77fe640 1 --2- 172.21.15.32:0/3111092871 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8da006be30 0x7f8da006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:31.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.223+0000 7f8db77fe640 1 -- 172.21.15.32:0/3111092871 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 4857+0+0 (secure 0 0 0) 0x7f8db808a160 con 0x7f8dc015e9b0 2024-04-24T23:05:31.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.223+0000 7f8dbe59c640 1 --2- 172.21.15.32:0/3111092871 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8da006be30 0x7f8da006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:31.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.223+0000 7f8dbe59c640 1 --2- 172.21.15.32:0/3111092871 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8da006be30 0x7f8da006e2f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f8db0000f60 tx=0x7f8db0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:31.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.227+0000 7f8db77fe640 1 -- 172.21.15.32:0/3111092871 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8db8058e60 con 0x7f8dc015e9b0 2024-04-24T23:05:31.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.408+0000 7f8dc48a5640 1 -- 172.21.15.32:0/3111092871 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8dc0074df0 con 0x7f8dc015e9b0 2024-04-24T23:05:31.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.408+0000 7f8db77fe640 1 -- 172.21.15.32:0/3111092871 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v136) ==== 93+0+10 (secure 0 0 0) 0x7f8db8058760 con 0x7f8dc015e9b0 2024-04-24T23:05:31.411 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T23:05:31.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.410+0000 7f8dc48a5640 1 -- 172.21.15.32:0/3111092871 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8da006be30 msgr2=0x7f8da006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:31.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.410+0000 7f8dc48a5640 1 --2- 172.21.15.32:0/3111092871 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8da006be30 0x7f8da006e2f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f8db0000f60 tx=0x7f8db0009040 comp rx=0 tx=0).stop 2024-04-24T23:05:31.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.410+0000 7f8dc48a5640 1 -- 172.21.15.32:0/3111092871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8dc015e9b0 msgr2=0x7f8dc00ccd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:31.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.410+0000 7f8dc48a5640 1 --2- 172.21.15.32:0/3111092871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8dc015e9b0 0x7f8dc00ccd60 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f8db800c7a0 tx=0x7f8db800cc60 comp rx=0 tx=0).stop 2024-04-24T23:05:31.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.410+0000 7f8dc48a5640 1 -- 172.21.15.32:0/3111092871 shutdown_connections 2024-04-24T23:05:31.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.410+0000 7f8dc48a5640 1 --2- 172.21.15.32:0/3111092871 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8da006be30 0x7f8da006e2f0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:31.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.410+0000 7f8dc48a5640 1 --2- 172.21.15.32:0/3111092871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8dc015e9b0 0x7f8dc00ccd60 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:31.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.410+0000 7f8dc48a5640 1 --2- 172.21.15.32:0/3111092871 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8dc00d4210 0x7f8dc00ca620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:31.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.410+0000 7f8dc48a5640 1 --2- 172.21.15.32:0/3111092871 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8dc006b820 0x7f8dc015a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:31.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.410+0000 7f8dc48a5640 1 -- 172.21.15.32:0/3111092871 >> 172.21.15.32:0/3111092871 conn(0x7f8dc00d11d0 msgr2=0x7f8dc006b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:31.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.410+0000 7f8dc48a5640 1 -- 172.21.15.32:0/3111092871 shutdown_connections 2024-04-24T23:05:31.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.410+0000 7f8dc48a5640 1 -- 172.21.15.32:0/3111092871 wait complete. 2024-04-24T23:05:31.424 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-24T23:05:31.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.522+0000 7f830e1f9640 1 Processor -- start 2024-04-24T23:05:31.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.522+0000 7f830e1f9640 1 -- start start 2024-04-24T23:05:31.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.523+0000 7f830e1f9640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f830815b380 0x7f830806c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:31.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.523+0000 7f830e1f9640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f83080ceb40 0x7f830806cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:31.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.523+0000 7f830e1f9640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83080b4520 con 0x7f830815cb00 2024-04-24T23:05:31.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.523+0000 7f830e1f9640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f83080cef70 con 0x7f830815b380 2024-04-24T23:05:31.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.523+0000 7f830e1f9640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f83080b4800 con 0x7f83080ceb40 2024-04-24T23:05:31.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.523+0000 7f8307fff640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f83080ceb40 0x7f830806cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:31.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.523+0000 7f8307fff640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f83080ceb40 0x7f830806cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.32:42072/0 (socket says 172.21.15.32:42072) 2024-04-24T23:05:31.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.523+0000 7f8307fff640 1 -- 172.21.15.32:0/1739758880 learned_addr learned my addr 172.21.15.32:0/1739758880 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:31.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.523+0000 7f83077fe640 1 --2- 172.21.15.32:0/1739758880 >> v2:172.21.15.121:3300/0 conn(0x7f830815b380 0x7f830806c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:31.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.524+0000 7f83077fe640 1 -- 172.21.15.32:0/1739758880 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f83080ceb40 msgr2=0x7f830806cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:31.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.524+0000 7f83077fe640 1 --2- 172.21.15.32:0/1739758880 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f83080ceb40 0x7f830806cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:31.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.524+0000 7f83077fe640 1 -- 172.21.15.32:0/1739758880 >> v1:172.21.15.121:6789/0 conn(0x7f830815cb00 legacy=0x7f83080ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:31.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.524+0000 7f83077fe640 1 -- 172.21.15.32:0/1739758880 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83080b4ae0 con 0x7f830815b380 2024-04-24T23:05:31.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.524+0000 7f83067fc640 1 -- 172.21.15.32:0/1739758880 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3042095714 0 0) 0x7f83080b4520 con 0x7f830815cb00 2024-04-24T23:05:31.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.524+0000 7f8307fff640 1 --2- 172.21.15.32:0/1739758880 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f83080ceb40 0x7f830806cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:05:31.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.524+0000 7f83077fe640 1 --2- 172.21.15.32:0/1739758880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f830815b380 0x7f830806c5f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f82f80097a0 tx=0x7f82f802a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bac6a5d9a44e527 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:31.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.525+0000 7f83067fc640 1 -- 172.21.15.32:0/1739758880 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f8037070 con 0x7f830815b380 2024-04-24T23:05:31.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.525+0000 7f83067fc640 1 -- 172.21.15.32:0/1739758880 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82f802adb0 con 0x7f830815b380 2024-04-24T23:05:31.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.525+0000 7f83067fc640 1 -- 172.21.15.32:0/1739758880 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f8032660 con 0x7f830815b380 2024-04-24T23:05:31.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.525+0000 7f830e1f9640 1 -- 172.21.15.32:0/1739758880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f830815b380 msgr2=0x7f830806c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:31.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.525+0000 7f830e1f9640 1 --2- 172.21.15.32:0/1739758880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f830815b380 0x7f830806c5f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f82f80097a0 tx=0x7f82f802a060 comp rx=0 tx=0).stop 2024-04-24T23:05:31.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.525+0000 7f830e1f9640 1 -- 172.21.15.32:0/1739758880 shutdown_connections 2024-04-24T23:05:31.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.525+0000 7f830e1f9640 1 --2- 172.21.15.32:0/1739758880 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f83080ceb40 0x7f830806cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:31.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.525+0000 7f830e1f9640 1 --2- 172.21.15.32:0/1739758880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f830815b380 0x7f830806c5f0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:31.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.525+0000 7f830e1f9640 1 -- 172.21.15.32:0/1739758880 >> 172.21.15.32:0/1739758880 conn(0x7f830806b110 msgr2=0x7f830806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:31.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.525+0000 7f830e1f9640 1 -- 172.21.15.32:0/1739758880 shutdown_connections 2024-04-24T23:05:31.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.525+0000 7f830e1f9640 1 -- 172.21.15.32:0/1739758880 wait complete. 2024-04-24T23:05:31.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.526+0000 7f830e1f9640 1 Processor -- start 2024-04-24T23:05:31.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.526+0000 7f830e1f9640 1 -- start start 2024-04-24T23:05:31.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.527+0000 7f830e1f9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83080ceb40 0x7f8308072060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:31.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.527+0000 7f830e1f9640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f830815b380 0x7f83080aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:31.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.527+0000 7f830e1f9640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f830815cb00 0x7f83081446a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:31.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.527+0000 7f830e1f9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f830815b760 con 0x7f83080ceb40 2024-04-24T23:05:31.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.527+0000 7f830e1f9640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f83080b50a0 con 0x7f830815b380 2024-04-24T23:05:31.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.527+0000 7f830e1f9640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f83080b55c0 con 0x7f830815cb00 2024-04-24T23:05:31.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.527+0000 7f83077fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83080ceb40 0x7f8308072060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:31.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.527+0000 7f83077fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83080ceb40 0x7f8308072060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56204/0 (socket says 172.21.15.32:56204) 2024-04-24T23:05:31.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.527+0000 7f83077fe640 1 -- 172.21.15.32:0/1042808699 learned_addr learned my addr 172.21.15.32:0/1042808699 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:31.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.528+0000 7f8307fff640 1 --2- 172.21.15.32:0/1042808699 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f830815cb00 0x7f83081446a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:31.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.528+0000 7f8306ffd640 1 --2- 172.21.15.32:0/1042808699 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f830815b380 0x7f83080aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:31.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.528+0000 7f83077fe640 1 -- 172.21.15.32:0/1042808699 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f830815cb00 msgr2=0x7f83081446a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:31.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.528+0000 7f83077fe640 1 --2- 172.21.15.32:0/1042808699 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f830815cb00 0x7f83081446a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:31.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.528+0000 7f83077fe640 1 -- 172.21.15.32:0/1042808699 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f830815b380 msgr2=0x7f83080aeb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:31.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.528+0000 7f83077fe640 1 --2- 172.21.15.32:0/1042808699 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f830815b380 0x7f83080aeb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:31.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.528+0000 7f83077fe640 1 -- 172.21.15.32:0/1042808699 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83081f8410 con 0x7f83080ceb40 2024-04-24T23:05:31.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.528+0000 7f83077fe640 1 --2- 172.21.15.32:0/1042808699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83080ceb40 0x7f8308072060 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f82f80097a0 tx=0x7f82f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:31.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.528+0000 7f82effff640 1 -- 172.21.15.32:0/1042808699 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f8037040 con 0x7f83080ceb40 2024-04-24T23:05:31.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.528+0000 7f82effff640 1 -- 172.21.15.32:0/1042808699 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82f8004030 con 0x7f83080ceb40 2024-04-24T23:05:31.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.528+0000 7f830e1f9640 1 -- 172.21.15.32:0/1042808699 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83081f86f0 con 0x7f83080ceb40 2024-04-24T23:05:31.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.528+0000 7f82effff640 1 -- 172.21.15.32:0/1042808699 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f8004420 con 0x7f83080ceb40 2024-04-24T23:05:31.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.528+0000 7f830e1f9640 1 -- 172.21.15.32:0/1042808699 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83081f8cb0 con 0x7f83080ceb40 2024-04-24T23:05:31.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.529+0000 7f830e1f9640 1 -- 172.21.15.32:0/1042808699 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8308068430 con 0x7f83080ceb40 2024-04-24T23:05:31.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.530+0000 7f82effff640 1 -- 172.21.15.32:0/1042808699 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f82f8003610 con 0x7f83080ceb40 2024-04-24T23:05:31.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.530+0000 7f82effff640 1 --2- 172.21.15.32:0/1042808699 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f82f406be30 0x7f82f406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:31.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.530+0000 7f82effff640 1 -- 172.21.15.32:0/1042808699 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 4857+0+0 (secure 0 0 0) 0x7f82f80ab8f0 con 0x7f83080ceb40 2024-04-24T23:05:31.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.530+0000 7f8306ffd640 1 --2- 172.21.15.32:0/1042808699 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f82f406be30 0x7f82f406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:31.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.531+0000 7f8306ffd640 1 --2- 172.21.15.32:0/1042808699 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f82f406be30 0x7f82f406e2f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f82fc007fd0 tx=0x7f82fc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:31.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.534+0000 7f82effff640 1 -- 172.21.15.32:0/1042808699 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f82f807a5f0 con 0x7f83080ceb40 2024-04-24T23:05:31.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.719+0000 7f830e1f9640 1 -- 172.21.15.32:0/1042808699 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f830806c000 con 0x7f83080ceb40 2024-04-24T23:05:31.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.719+0000 7f82effff640 1 -- 172.21.15.32:0/1042808699 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v136) ==== 84+0+65 (secure 0 0 0) 0x7f82f8079ef0 con 0x7f83080ceb40 2024-04-24T23:05:31.722 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:05:31.722 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2024-04-24T23:05:31.722 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:31.722 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2024-04-24T23:05:31.722 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": ".mgr" 2024-04-24T23:05:31.722 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:31.722 INFO:teuthology.orchestra.run.smithi032.stdout:] 2024-04-24T23:05:31.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.721+0000 7f830e1f9640 1 -- 172.21.15.32:0/1042808699 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f82f406be30 msgr2=0x7f82f406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:31.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.721+0000 7f830e1f9640 1 --2- 172.21.15.32:0/1042808699 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f82f406be30 0x7f82f406e2f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f82fc007fd0 tx=0x7f82fc009040 comp rx=0 tx=0).stop 2024-04-24T23:05:31.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.721+0000 7f830e1f9640 1 -- 172.21.15.32:0/1042808699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83080ceb40 msgr2=0x7f8308072060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:31.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.721+0000 7f830e1f9640 1 --2- 172.21.15.32:0/1042808699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83080ceb40 0x7f8308072060 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f82f80097a0 tx=0x7f82f80046c0 comp rx=0 tx=0).stop 2024-04-24T23:05:31.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.721+0000 7f830e1f9640 1 -- 172.21.15.32:0/1042808699 shutdown_connections 2024-04-24T23:05:31.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.721+0000 7f830e1f9640 1 --2- 172.21.15.32:0/1042808699 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f82f406be30 0x7f82f406e2f0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:31.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.721+0000 7f830e1f9640 1 --2- 172.21.15.32:0/1042808699 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f830815cb00 0x7f83081446a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:31.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.721+0000 7f830e1f9640 1 --2- 172.21.15.32:0/1042808699 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f830815b380 0x7f83080aeb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:31.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.721+0000 7f830e1f9640 1 --2- 172.21.15.32:0/1042808699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83080ceb40 0x7f8308072060 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:31.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.721+0000 7f830e1f9640 1 -- 172.21.15.32:0/1042808699 >> 172.21.15.32:0/1042808699 conn(0x7f830806b110 msgr2=0x7f830806c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:31.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.721+0000 7f830e1f9640 1 -- 172.21.15.32:0/1042808699 shutdown_connections 2024-04-24T23:05:31.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.721+0000 7f830e1f9640 1 -- 172.21.15.32:0/1042808699 wait complete. 2024-04-24T23:05:31.734 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-24T23:05:31.735 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-24T23:05:31.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.838+0000 7ffa36a6b640 1 Processor -- start 2024-04-24T23:05:31.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.838+0000 7ffa36a6b640 1 -- start start 2024-04-24T23:05:31.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.838+0000 7ffa36a6b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffa3015b960 0x7ffa300cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:31.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.838+0000 7ffa36a6b640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa300b58e0 0x7ffa300b5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:31.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.838+0000 7ffa36a6b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa3015bde0 con 0x7ffa3015b580 2024-04-24T23:05:31.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.838+0000 7ffa36a6b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7ffa3015c9f0 con 0x7ffa3015b960 2024-04-24T23:05:31.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.838+0000 7ffa36a6b640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ffa300ce3c0 con 0x7ffa300b58e0 2024-04-24T23:05:31.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.839+0000 7ffa2ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffa3015b960 0x7ffa300cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:31.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.839+0000 7ffa34807640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ffa3015b580 0x7ffa3006cb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37804/0 (socket says 172.21.15.32:37804) 2024-04-24T23:05:31.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.839+0000 7ffa34807640 1 -- 172.21.15.32:0/64349980 learned_addr learned my addr 172.21.15.32:0/64349980 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:05:31.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.839+0000 7ffa35008640 1 --2- 172.21.15.32:0/64349980 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa300b58e0 0x7ffa300b5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:31.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.839+0000 7ffa2ffff640 1 -- 172.21.15.32:0/64349980 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa300b58e0 msgr2=0x7ffa300b5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:31.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.839+0000 7ffa2ffff640 1 --2- 172.21.15.32:0/64349980 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa300b58e0 0x7ffa300b5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:31.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.839+0000 7ffa2ffff640 1 -- 172.21.15.32:0/64349980 >> v1:172.21.15.32:6789/0 conn(0x7ffa3015b580 legacy=0x7ffa3006cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:31.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.839+0000 7ffa2f7fe640 1 -- 172.21.15.32:0/64349980 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2143395779 0 0) 0x7ffa3015bde0 con 0x7ffa3015b580 2024-04-24T23:05:31.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.839+0000 7ffa2ffff640 1 -- 172.21.15.32:0/64349980 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa3015c0c0 con 0x7ffa3015b960 2024-04-24T23:05:31.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.840+0000 7ffa2ffff640 1 --2- 172.21.15.32:0/64349980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa3015b960 0x7ffa300cde80 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c0097a0 tx=0x7ffa1c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1da74bf62e3b78d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:31.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.840+0000 7ffa2f7fe640 1 -- 172.21.15.32:0/64349980 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c037070 con 0x7ffa3015b960 2024-04-24T23:05:31.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.840+0000 7ffa2f7fe640 1 -- 172.21.15.32:0/64349980 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa1c032440 con 0x7ffa3015b960 2024-04-24T23:05:31.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.840+0000 7ffa2f7fe640 1 -- 172.21.15.32:0/64349980 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c032810 con 0x7ffa3015b960 2024-04-24T23:05:31.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.840+0000 7ffa36a6b640 1 -- 172.21.15.32:0/64349980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa3015b960 msgr2=0x7ffa300cde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:31.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.840+0000 7ffa36a6b640 1 --2- 172.21.15.32:0/64349980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa3015b960 0x7ffa300cde80 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c0097a0 tx=0x7ffa1c02a180 comp rx=0 tx=0).stop 2024-04-24T23:05:31.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.840+0000 7ffa36a6b640 1 -- 172.21.15.32:0/64349980 shutdown_connections 2024-04-24T23:05:31.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.840+0000 7ffa36a6b640 1 --2- 172.21.15.32:0/64349980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa3015b960 0x7ffa300cde80 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:31.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.840+0000 7ffa36a6b640 1 --2- 172.21.15.32:0/64349980 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa300b58e0 0x7ffa300b5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:31.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.840+0000 7ffa36a6b640 1 -- 172.21.15.32:0/64349980 >> 172.21.15.32:0/64349980 conn(0x7ffa300b1f10 msgr2=0x7ffa300b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:31.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.840+0000 7ffa36a6b640 1 -- 172.21.15.32:0/64349980 shutdown_connections 2024-04-24T23:05:31.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.840+0000 7ffa36a6b640 1 -- 172.21.15.32:0/64349980 wait complete. 2024-04-24T23:05:31.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.841+0000 7ffa36a6b640 1 Processor -- start 2024-04-24T23:05:31.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.842+0000 7ffa36a6b640 1 -- start start 2024-04-24T23:05:31.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.842+0000 7ffa36a6b640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa300b58e0 0x7ffa301f44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:31.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.842+0000 7ffa36a6b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa3015b580 0x7ffa301f8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:31.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.842+0000 7ffa36a6b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa3015b960 0x7ffa300cba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:31.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.842+0000 7ffa36a6b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3015c680 con 0x7ffa3015b960 2024-04-24T23:05:31.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.842+0000 7ffa36a6b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ffa300b6260 con 0x7ffa3015b580 2024-04-24T23:05:31.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.842+0000 7ffa36a6b640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ffa3006c7a0 con 0x7ffa300b58e0 2024-04-24T23:05:31.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.843+0000 7ffa35008640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa3015b960 0x7ffa300cba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:31.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.843+0000 7ffa35008640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa3015b960 0x7ffa300cba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56222/0 (socket says 172.21.15.32:56222) 2024-04-24T23:05:31.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.843+0000 7ffa35008640 1 -- 172.21.15.32:0/3897877658 learned_addr learned my addr 172.21.15.32:0/3897877658 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:31.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.843+0000 7ffa34807640 1 --2- 172.21.15.32:0/3897877658 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa300b58e0 0x7ffa301f44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:31.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.843+0000 7ffa2ffff640 1 --2- 172.21.15.32:0/3897877658 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa3015b580 0x7ffa301f8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:31.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.843+0000 7ffa35008640 1 -- 172.21.15.32:0/3897877658 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa300b58e0 msgr2=0x7ffa301f44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:31.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.843+0000 7ffa35008640 1 --2- 172.21.15.32:0/3897877658 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa300b58e0 0x7ffa301f44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:31.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.843+0000 7ffa35008640 1 -- 172.21.15.32:0/3897877658 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa3015b580 msgr2=0x7ffa301f8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:31.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.843+0000 7ffa35008640 1 --2- 172.21.15.32:0/3897877658 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa3015b580 0x7ffa301f8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:31.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.843+0000 7ffa35008640 1 -- 172.21.15.32:0/3897877658 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa301f4bb0 con 0x7ffa3015b960 2024-04-24T23:05:31.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.843+0000 7ffa35008640 1 --2- 172.21.15.32:0/3897877658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa3015b960 0x7ffa300cba90 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7ffa28007d90 tx=0x7ffa2800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:31.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.843+0000 7ffa2d7fa640 1 -- 172.21.15.32:0/3897877658 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa28014420 con 0x7ffa3015b960 2024-04-24T23:05:31.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.843+0000 7ffa2d7fa640 1 -- 172.21.15.32:0/3897877658 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa28019070 con 0x7ffa3015b960 2024-04-24T23:05:31.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.843+0000 7ffa2d7fa640 1 -- 172.21.15.32:0/3897877658 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa28014820 con 0x7ffa3015b960 2024-04-24T23:05:31.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.843+0000 7ffa36a6b640 1 -- 172.21.15.32:0/3897877658 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa301f4e90 con 0x7ffa3015b960 2024-04-24T23:05:31.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.843+0000 7ffa36a6b640 1 -- 172.21.15.32:0/3897877658 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa300caa60 con 0x7ffa3015b960 2024-04-24T23:05:31.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.844+0000 7ffa36a6b640 1 -- 172.21.15.32:0/3897877658 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa301f53b0 con 0x7ffa3015b960 2024-04-24T23:05:31.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.845+0000 7ffa2d7fa640 1 -- 172.21.15.32:0/3897877658 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffa280149c0 con 0x7ffa3015b960 2024-04-24T23:05:31.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.845+0000 7ffa2d7fa640 1 --2- 172.21.15.32:0/3897877658 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ffa1406be60 0x7ffa1406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:31.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.845+0000 7ffa2d7fa640 1 -- 172.21.15.32:0/3897877658 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 4857+0+0 (secure 0 0 0) 0x7ffa2808b320 con 0x7ffa3015b960 2024-04-24T23:05:31.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.845+0000 7ffa34807640 1 --2- 172.21.15.32:0/3897877658 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ffa1406be60 0x7ffa1406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:31.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.846+0000 7ffa34807640 1 --2- 172.21.15.32:0/3897877658 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ffa1406be60 0x7ffa1406e320 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ffa24000f60 tx=0x7ffa24007b50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:31.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:31.849+0000 7ffa2d7fa640 1 -- 172.21.15.32:0/3897877658 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ffa301f53b0 con 0x7ffa3015b960 2024-04-24T23:05:32.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.031+0000 7ffa36a6b640 1 -- 172.21.15.32:0/3897877658 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7ffa30068430 con 0x7ffa3015b960 2024-04-24T23:05:32.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.408+0000 7ffa2d7fa640 1 -- 172.21.15.32:0/3897877658 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v137) ==== 138+0+0 (secure 0 0 0) 0x7ffa2805a020 con 0x7ffa3015b960 2024-04-24T23:05:32.410 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2024-04-24T23:05:32.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.411+0000 7ffa36a6b640 1 -- 172.21.15.32:0/3897877658 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ffa1406be60 msgr2=0x7ffa1406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:32.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.411+0000 7ffa36a6b640 1 --2- 172.21.15.32:0/3897877658 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ffa1406be60 0x7ffa1406e320 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ffa24000f60 tx=0x7ffa24007b50 comp rx=0 tx=0).stop 2024-04-24T23:05:32.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.411+0000 7ffa36a6b640 1 -- 172.21.15.32:0/3897877658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa3015b960 msgr2=0x7ffa300cba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:32.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.411+0000 7ffa36a6b640 1 --2- 172.21.15.32:0/3897877658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa3015b960 0x7ffa300cba90 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7ffa28007d90 tx=0x7ffa2800e4b0 comp rx=0 tx=0).stop 2024-04-24T23:05:32.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.411+0000 7ffa36a6b640 1 -- 172.21.15.32:0/3897877658 shutdown_connections 2024-04-24T23:05:32.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.411+0000 7ffa36a6b640 1 --2- 172.21.15.32:0/3897877658 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ffa1406be60 0x7ffa1406e320 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:32.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.411+0000 7ffa36a6b640 1 --2- 172.21.15.32:0/3897877658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa3015b960 0x7ffa300cba90 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:32.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.411+0000 7ffa36a6b640 1 --2- 172.21.15.32:0/3897877658 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa3015b580 0x7ffa301f8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:32.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.411+0000 7ffa36a6b640 1 --2- 172.21.15.32:0/3897877658 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa300b58e0 0x7ffa301f44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:32.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.411+0000 7ffa36a6b640 1 -- 172.21.15.32:0/3897877658 >> 172.21.15.32:0/3897877658 conn(0x7ffa300b1f10 msgr2=0x7ffa300ce570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:32.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.411+0000 7ffa36a6b640 1 -- 172.21.15.32:0/3897877658 shutdown_connections 2024-04-24T23:05:32.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.411+0000 7ffa36a6b640 1 -- 172.21.15.32:0/3897877658 wait complete. 2024-04-24T23:05:32.426 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-24T23:05:32.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.533+0000 7f812c17e640 1 Processor -- start 2024-04-24T23:05:32.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.533+0000 7f812c17e640 1 -- start start 2024-04-24T23:05:32.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.534+0000 7f812c17e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8124162320 0x7f81240ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:32.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.534+0000 7f812c17e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8124162e10 0x7f812415ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:32.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.534+0000 7f812c17e640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81240cd270 con 0x7f81240b5af0 2024-04-24T23:05:32.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.534+0000 7f812c17e640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f81240b5660 con 0x7f8124162e10 2024-04-24T23:05:32.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.534+0000 7f812c17e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f8124071610 con 0x7f8124162320 2024-04-24T23:05:32.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.534+0000 7f812a71b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8124162e10 0x7f812415ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:32.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.534+0000 7f812a71b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8124162e10 0x7f812415ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56234/0 (socket says 172.21.15.32:56234) 2024-04-24T23:05:32.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.534+0000 7f812a71b640 1 -- 172.21.15.32:0/1363689231 learned_addr learned my addr 172.21.15.32:0/1363689231 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:32.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.534+0000 7f8129f1a640 1 --2- 172.21.15.32:0/1363689231 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8124162320 0x7f81240ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:32.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.534+0000 7f812a71b640 1 -- 172.21.15.32:0/1363689231 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8124162320 msgr2=0x7f81240ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:32.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.534+0000 7f812a71b640 1 --2- 172.21.15.32:0/1363689231 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8124162320 0x7f81240ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:32.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.534+0000 7f812a71b640 1 -- 172.21.15.32:0/1363689231 >> v1:172.21.15.121:6789/0 conn(0x7f81240b5af0 legacy=0x7f8124162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:32.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.534+0000 7f812a71b640 1 -- 172.21.15.32:0/1363689231 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81240718f0 con 0x7f8124162e10 2024-04-24T23:05:32.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.535+0000 7f812a71b640 1 --2- 172.21.15.32:0/1363689231 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8124162e10 0x7f812415ed30 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f811c009e50 tx=0x7f811c02a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=913f7eac7a2f43e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:32.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.535+0000 7f8128f18640 1 -- 172.21.15.32:0/1363689231 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811c036070 con 0x7f8124162e10 2024-04-24T23:05:32.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.535+0000 7f8128f18640 1 -- 172.21.15.32:0/1363689231 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f811c031600 con 0x7f8124162e10 2024-04-24T23:05:32.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.535+0000 7f8128f18640 1 -- 172.21.15.32:0/1363689231 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811c0319d0 con 0x7f8124162e10 2024-04-24T23:05:32.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.535+0000 7f812c17e640 1 -- 172.21.15.32:0/1363689231 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8124162e10 msgr2=0x7f812415ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:32.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.535+0000 7f812c17e640 1 --2- 172.21.15.32:0/1363689231 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8124162e10 0x7f812415ed30 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f811c009e50 tx=0x7f811c02a630 comp rx=0 tx=0).stop 2024-04-24T23:05:32.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.535+0000 7f812c17e640 1 -- 172.21.15.32:0/1363689231 shutdown_connections 2024-04-24T23:05:32.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.536+0000 7f812c17e640 1 --2- 172.21.15.32:0/1363689231 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8124162e10 0x7f812415ed30 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:32.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.536+0000 7f812c17e640 1 --2- 172.21.15.32:0/1363689231 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8124162320 0x7f81240ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:32.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.536+0000 7f812c17e640 1 -- 172.21.15.32:0/1363689231 >> 172.21.15.32:0/1363689231 conn(0x7f81240d0ae0 msgr2=0x7f81240d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:32.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.536+0000 7f812c17e640 1 -- 172.21.15.32:0/1363689231 shutdown_connections 2024-04-24T23:05:32.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.536+0000 7f812c17e640 1 -- 172.21.15.32:0/1363689231 wait complete. 2024-04-24T23:05:32.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.537+0000 7f812c17e640 1 Processor -- start 2024-04-24T23:05:32.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.537+0000 7f812c17e640 1 -- start start 2024-04-24T23:05:32.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.538+0000 7f812c17e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f81240b5af0 0x7f81241e9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:32.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.538+0000 7f812c17e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8124162320 0x7f8124158800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:32.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.538+0000 7f812c17e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8124162e10 0x7f81241e8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:32.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.538+0000 7f812c17e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8124071bd0 con 0x7f8124162e10 2024-04-24T23:05:32.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.538+0000 7f812c17e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8124067950 con 0x7f8124162320 2024-04-24T23:05:32.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.538+0000 7f812c17e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f8124071d50 con 0x7f81240b5af0 2024-04-24T23:05:32.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.538+0000 7f812a71b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8124162e10 0x7f81241e8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:32.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.538+0000 7f812a71b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8124162e10 0x7f81241e8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56236/0 (socket says 172.21.15.32:56236) 2024-04-24T23:05:32.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.538+0000 7f812a71b640 1 -- 172.21.15.32:0/445815183 learned_addr learned my addr 172.21.15.32:0/445815183 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:32.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.538+0000 7f8129f1a640 1 --2- 172.21.15.32:0/445815183 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f81240b5af0 0x7f81241e9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:32.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.538+0000 7f8129719640 1 --2- 172.21.15.32:0/445815183 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8124162320 0x7f8124158800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:32.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.538+0000 7f812a71b640 1 -- 172.21.15.32:0/445815183 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f81240b5af0 msgr2=0x7f81241e9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:32.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.538+0000 7f812a71b640 1 --2- 172.21.15.32:0/445815183 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f81240b5af0 0x7f81241e9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:32.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.538+0000 7f812a71b640 1 -- 172.21.15.32:0/445815183 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8124162320 msgr2=0x7f8124158800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:32.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.538+0000 7f812a71b640 1 --2- 172.21.15.32:0/445815183 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8124162320 0x7f8124158800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:32.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.538+0000 7f812a71b640 1 -- 172.21.15.32:0/445815183 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8124158d40 con 0x7f8124162e10 2024-04-24T23:05:32.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.538+0000 7f812a71b640 1 --2- 172.21.15.32:0/445815183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8124162e10 0x7f81241e8140 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f811c034040 tx=0x7f811c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:32.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.539+0000 7f811a7fc640 1 -- 172.21.15.32:0/445815183 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811c041020 con 0x7f8124162e10 2024-04-24T23:05:32.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.539+0000 7f811a7fc640 1 -- 172.21.15.32:0/445815183 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f811c0077a0 con 0x7f8124162e10 2024-04-24T23:05:32.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.539+0000 7f812c17e640 1 -- 172.21.15.32:0/445815183 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81241e8ab0 con 0x7f8124162e10 2024-04-24T23:05:32.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.539+0000 7f811a7fc640 1 -- 172.21.15.32:0/445815183 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811c036040 con 0x7f8124162e10 2024-04-24T23:05:32.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.539+0000 7f812c17e640 1 -- 172.21.15.32:0/445815183 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81241e9070 con 0x7f8124162e10 2024-04-24T23:05:32.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.540+0000 7f812c17e640 1 -- 172.21.15.32:0/445815183 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8124068430 con 0x7f8124162e10 2024-04-24T23:05:32.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.540+0000 7f811a7fc640 1 -- 172.21.15.32:0/445815183 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f811c044020 con 0x7f8124162e10 2024-04-24T23:05:32.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.541+0000 7f811a7fc640 1 --2- 172.21.15.32:0/445815183 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f810806be30 0x7f810806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:32.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.541+0000 7f811a7fc640 1 -- 172.21.15.32:0/445815183 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5227+0+0 (secure 0 0 0) 0x7f811c0ab8e0 con 0x7f8124162e10 2024-04-24T23:05:32.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.541+0000 7f8129f1a640 1 --2- 172.21.15.32:0/445815183 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f810806be30 0x7f810806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:32.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.542+0000 7f8129f1a640 1 --2- 172.21.15.32:0/445815183 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f810806be30 0x7f810806e2f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f8120006fd0 tx=0x7f8120008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:32.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.546+0000 7f811a7fc640 1 -- 172.21.15.32:0/445815183 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f811c04b030 con 0x7f8124162e10 2024-04-24T23:05:32.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:32.728+0000 7f812c17e640 1 -- 172.21.15.32:0/445815183 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f81240d0850 con 0x7f8124162e10 2024-04-24T23:05:33.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.415+0000 7f811a7fc640 1 -- 172.21.15.32:0/445815183 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v138) ==== 170+0+0 (secure 0 0 0) 0x7f811c079d80 con 0x7f8124162e10 2024-04-24T23:05:33.418 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2024-04-24T23:05:33.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.420+0000 7f812c17e640 1 -- 172.21.15.32:0/445815183 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f810806be30 msgr2=0x7f810806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:33.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.420+0000 7f812c17e640 1 --2- 172.21.15.32:0/445815183 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f810806be30 0x7f810806e2f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f8120006fd0 tx=0x7f8120008040 comp rx=0 tx=0).stop 2024-04-24T23:05:33.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.420+0000 7f812c17e640 1 -- 172.21.15.32:0/445815183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8124162e10 msgr2=0x7f81241e8140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:33.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.420+0000 7f812c17e640 1 --2- 172.21.15.32:0/445815183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8124162e10 0x7f81241e8140 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f811c034040 tx=0x7f811c004a40 comp rx=0 tx=0).stop 2024-04-24T23:05:33.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.426+0000 7f812c17e640 1 -- 172.21.15.32:0/445815183 shutdown_connections 2024-04-24T23:05:33.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.426+0000 7f812c17e640 1 --2- 172.21.15.32:0/445815183 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f810806be30 0x7f810806e2f0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:33.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.427+0000 7f812c17e640 1 --2- 172.21.15.32:0/445815183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8124162e10 0x7f81241e8140 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:33.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.427+0000 7f812c17e640 1 --2- 172.21.15.32:0/445815183 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8124162320 0x7f8124158800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:33.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.427+0000 7f812c17e640 1 --2- 172.21.15.32:0/445815183 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f81240b5af0 0x7f81241e9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:33.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.427+0000 7f812c17e640 1 -- 172.21.15.32:0/445815183 >> 172.21.15.32:0/445815183 conn(0x7f81240d0ae0 msgr2=0x7f81240b0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:33.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.427+0000 7f812c17e640 1 -- 172.21.15.32:0/445815183 shutdown_connections 2024-04-24T23:05:33.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.427+0000 7f812c17e640 1 -- 172.21.15.32:0/445815183 wait complete. 2024-04-24T23:05:33.444 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-24T23:05:33.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.549+0000 7fd983cdf640 1 Processor -- start 2024-04-24T23:05:33.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.550+0000 7fd983cdf640 1 -- start start 2024-04-24T23:05:33.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.550+0000 7fd983cdf640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd97c0cd3c0 0x7fd97c06c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:33.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.550+0000 7fd983cdf640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd97c151bc0 0x7fd97c15eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:33.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.550+0000 7fd983cdf640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd97c152010 con 0x7fd97c15e6c0 2024-04-24T23:05:33.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.551+0000 7fd983cdf640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fd97c15f060 con 0x7fd97c151bc0 2024-04-24T23:05:33.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.551+0000 7fd983cdf640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd97c0cd910 con 0x7fd97c0cd3c0 2024-04-24T23:05:33.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.551+0000 7fd981a7b640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd97c0cd3c0 0x7fd97c06c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:33.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.551+0000 7fd98227c640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd97c151bc0 0x7fd97c15eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:33.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.551+0000 7fd98127a640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fd97c15e6c0 0x7fd97c1514b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:37500/0 (socket says 172.21.15.32:37500) 2024-04-24T23:05:33.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.551+0000 7fd98127a640 1 -- 172.21.15.32:0/3660804272 learned_addr learned my addr 172.21.15.32:0/3660804272 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:05:33.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.551+0000 7fd980a79640 1 -- 172.21.15.32:0/3660804272 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3252559596 0 0) 0x7fd96c001110 con 0x7fd97c15e6c0 2024-04-24T23:05:33.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.551+0000 7fd981a7b640 1 -- 172.21.15.32:0/3660804272 >> v2:172.21.15.121:3300/0 conn(0x7fd97c151bc0 msgr2=0x7fd97c15eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:33.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.551+0000 7fd981a7b640 1 --2- 172.21.15.32:0/3660804272 >> v2:172.21.15.121:3300/0 conn(0x7fd97c151bc0 0x7fd97c15eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:33.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.551+0000 7fd981a7b640 1 -- 172.21.15.32:0/3660804272 >> v1:172.21.15.121:6789/0 conn(0x7fd97c15e6c0 legacy=0x7fd97c1514b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:33.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.551+0000 7fd981a7b640 1 -- 172.21.15.32:0/3660804272 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd97c0cdbf0 con 0x7fd97c0cd3c0 2024-04-24T23:05:33.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.551+0000 7fd98227c640 1 --2- 172.21.15.32:0/3660804272 >> v2:172.21.15.121:3300/0 conn(0x7fd97c151bc0 0x7fd97c15eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:05:33.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.552+0000 7fd981a7b640 1 --2- 172.21.15.32:0/3660804272 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd97c0cd3c0 0x7fd97c06c5f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd9780097a0 tx=0x7fd97802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af2ec125105c8494 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:33.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.552+0000 7fd980a79640 1 -- 172.21.15.32:0/3660804272 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd978037070 con 0x7fd97c0cd3c0 2024-04-24T23:05:33.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.552+0000 7fd980a79640 1 -- 172.21.15.32:0/3660804272 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd978032440 con 0x7fd97c0cd3c0 2024-04-24T23:05:33.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.552+0000 7fd980a79640 1 -- 172.21.15.32:0/3660804272 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd978032810 con 0x7fd97c0cd3c0 2024-04-24T23:05:33.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.553+0000 7fd983cdf640 1 -- 172.21.15.32:0/3660804272 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd97c0cd3c0 msgr2=0x7fd97c06c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:33.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.553+0000 7fd983cdf640 1 --2- 172.21.15.32:0/3660804272 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd97c0cd3c0 0x7fd97c06c5f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd9780097a0 tx=0x7fd97802a180 comp rx=0 tx=0).stop 2024-04-24T23:05:33.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.553+0000 7fd983cdf640 1 -- 172.21.15.32:0/3660804272 shutdown_connections 2024-04-24T23:05:33.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.553+0000 7fd983cdf640 1 --2- 172.21.15.32:0/3660804272 >> v2:172.21.15.121:3300/0 conn(0x7fd97c151bc0 0x7fd97c15eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:33.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.553+0000 7fd983cdf640 1 --2- 172.21.15.32:0/3660804272 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd97c0cd3c0 0x7fd97c06c5f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:33.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.553+0000 7fd983cdf640 1 -- 172.21.15.32:0/3660804272 >> 172.21.15.32:0/3660804272 conn(0x7fd97c06b110 msgr2=0x7fd97c06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:33.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.553+0000 7fd983cdf640 1 -- 172.21.15.32:0/3660804272 shutdown_connections 2024-04-24T23:05:33.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.553+0000 7fd983cdf640 1 -- 172.21.15.32:0/3660804272 wait complete. 2024-04-24T23:05:33.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.554+0000 7fd983cdf640 1 Processor -- start 2024-04-24T23:05:33.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.554+0000 7fd983cdf640 1 -- start start 2024-04-24T23:05:33.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.555+0000 7fd983cdf640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd97c0cd3c0 0x7fd97c13f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:33.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.555+0000 7fd983cdf640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd97c151bc0 0x7fd97c13fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:33.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.555+0000 7fd983cdf640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd97c15e6c0 0x7fd97c1f1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:33.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.555+0000 7fd983cdf640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd97c0ce490 con 0x7fd97c151bc0 2024-04-24T23:05:33.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.555+0000 7fd983cdf640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd97c0ce1b0 con 0x7fd97c0cd3c0 2024-04-24T23:05:33.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.555+0000 7fd983cdf640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd97c0b4420 con 0x7fd97c15e6c0 2024-04-24T23:05:33.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.555+0000 7fd98127a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd97c151bc0 0x7fd97c13fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:33.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.555+0000 7fd98127a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd97c151bc0 0x7fd97c13fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56242/0 (socket says 172.21.15.32:56242) 2024-04-24T23:05:33.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.555+0000 7fd98127a640 1 -- 172.21.15.32:0/2465905664 learned_addr learned my addr 172.21.15.32:0/2465905664 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:33.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.555+0000 7fd98227c640 1 --2- 172.21.15.32:0/2465905664 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd97c15e6c0 0x7fd97c1f1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:33.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.555+0000 7fd981a7b640 1 --2- 172.21.15.32:0/2465905664 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd97c0cd3c0 0x7fd97c13f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:33.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.555+0000 7fd98127a640 1 -- 172.21.15.32:0/2465905664 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd97c15e6c0 msgr2=0x7fd97c1f1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:33.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.555+0000 7fd98127a640 1 --2- 172.21.15.32:0/2465905664 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd97c15e6c0 0x7fd97c1f1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:33.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.555+0000 7fd98127a640 1 -- 172.21.15.32:0/2465905664 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd97c0cd3c0 msgr2=0x7fd97c13f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:33.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.555+0000 7fd98127a640 1 --2- 172.21.15.32:0/2465905664 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd97c0cd3c0 0x7fd97c13f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:33.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.555+0000 7fd98127a640 1 -- 172.21.15.32:0/2465905664 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd97c1510d0 con 0x7fd97c151bc0 2024-04-24T23:05:33.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.555+0000 7fd98227c640 1 --2- 172.21.15.32:0/2465905664 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd97c15e6c0 0x7fd97c1f1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:05:33.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.555+0000 7fd98127a640 1 --2- 172.21.15.32:0/2465905664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd97c151bc0 0x7fd97c13fc20 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fd96c00c780 tx=0x7fd96c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:33.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.556+0000 7fd9727fc640 1 -- 172.21.15.32:0/2465905664 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96c018070 con 0x7fd97c151bc0 2024-04-24T23:05:33.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.556+0000 7fd9727fc640 1 -- 172.21.15.32:0/2465905664 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd96c0072d0 con 0x7fd97c151bc0 2024-04-24T23:05:33.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.556+0000 7fd983cdf640 1 -- 172.21.15.32:0/2465905664 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd97c150830 con 0x7fd97c151bc0 2024-04-24T23:05:33.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.556+0000 7fd9727fc640 1 -- 172.21.15.32:0/2465905664 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96c0047e0 con 0x7fd97c151bc0 2024-04-24T23:05:33.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.556+0000 7fd983cdf640 1 -- 172.21.15.32:0/2465905664 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd97c150df0 con 0x7fd97c151bc0 2024-04-24T23:05:33.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.557+0000 7fd983cdf640 1 -- 172.21.15.32:0/2465905664 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd97c068430 con 0x7fd97c151bc0 2024-04-24T23:05:33.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.557+0000 7fd9727fc640 1 -- 172.21.15.32:0/2465905664 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd96c01b070 con 0x7fd97c151bc0 2024-04-24T23:05:33.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.558+0000 7fd9727fc640 1 --2- 172.21.15.32:0/2465905664 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd96006be60 0x7fd96006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:33.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.558+0000 7fd9727fc640 1 -- 172.21.15.32:0/2465905664 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5609+0+0 (secure 0 0 0) 0x7fd96c08a090 con 0x7fd97c151bc0 2024-04-24T23:05:33.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.558+0000 7fd981a7b640 1 --2- 172.21.15.32:0/2465905664 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd96006be60 0x7fd96006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:33.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.558+0000 7fd981a7b640 1 --2- 172.21.15.32:0/2465905664 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd96006be60 0x7fd96006e320 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd978034000 tx=0x7fd97802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:33.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.562+0000 7fd9727fc640 1 -- 172.21.15.32:0/2465905664 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd97c152010 con 0x7fd97c151bc0 2024-04-24T23:05:33.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:33.788+0000 7fd983cdf640 1 -- 172.21.15.32:0/2465905664 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fd97c06d5f0 con 0x7fd97c151bc0 2024-04-24T23:05:34.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.417+0000 7fd9727fc640 1 -- 172.21.15.32:0/2465905664 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v56) ==== 174+0+0 (secure 0 0 0) 0x7fd96c058b30 con 0x7fd97c151bc0 2024-04-24T23:05:34.420 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 16 and data pool 17 2024-04-24T23:05:34.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.420+0000 7fd983cdf640 1 -- 172.21.15.32:0/2465905664 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd96006be60 msgr2=0x7fd96006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:34.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.420+0000 7fd983cdf640 1 --2- 172.21.15.32:0/2465905664 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd96006be60 0x7fd96006e320 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd978034000 tx=0x7fd97802e000 comp rx=0 tx=0).stop 2024-04-24T23:05:34.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.420+0000 7fd983cdf640 1 -- 172.21.15.32:0/2465905664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd97c151bc0 msgr2=0x7fd97c13fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:34.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.420+0000 7fd983cdf640 1 --2- 172.21.15.32:0/2465905664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd97c151bc0 0x7fd97c13fc20 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fd96c00c780 tx=0x7fd96c00cc40 comp rx=0 tx=0).stop 2024-04-24T23:05:34.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.422+0000 7fd983cdf640 1 -- 172.21.15.32:0/2465905664 shutdown_connections 2024-04-24T23:05:34.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.422+0000 7fd983cdf640 1 --2- 172.21.15.32:0/2465905664 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd96006be60 0x7fd96006e320 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:34.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.422+0000 7fd983cdf640 1 --2- 172.21.15.32:0/2465905664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd97c151bc0 0x7fd97c13fc20 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:34.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.422+0000 7fd983cdf640 1 --2- 172.21.15.32:0/2465905664 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd97c15e6c0 0x7fd97c1f1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:34.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.422+0000 7fd983cdf640 1 --2- 172.21.15.32:0/2465905664 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd97c0cd3c0 0x7fd97c13f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:34.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.422+0000 7fd983cdf640 1 -- 172.21.15.32:0/2465905664 >> 172.21.15.32:0/2465905664 conn(0x7fd97c06b110 msgr2=0x7fd97c14f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:34.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.424+0000 7fd983cdf640 1 -- 172.21.15.32:0/2465905664 shutdown_connections 2024-04-24T23:05:34.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.431+0000 7fd983cdf640 1 -- 172.21.15.32:0/2465905664 wait complete. 2024-04-24T23:05:34.454 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:05:34.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.557+0000 7fdfd5c4d640 1 Processor -- start 2024-04-24T23:05:34.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.557+0000 7fdfd5c4d640 1 -- start start 2024-04-24T23:05:34.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.558+0000 7fdfd5c4d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdfd00b44d0 0x7fdfd00b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:34.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.558+0000 7fdfd5c4d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfd00b6430 con 0x7fdfd00af890 2024-04-24T23:05:34.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.558+0000 7fdfd5c4d640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfd0156f80 con 0x7fdfd006c5f0 2024-04-24T23:05:34.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.558+0000 7fdfd5c4d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fdfd00b6150 con 0x7fdfd00b44d0 2024-04-24T23:05:34.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.558+0000 7fdfcffff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fdfd00af890 0x7fdfd015cac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37806/0 (socket says 172.21.15.32:37806) 2024-04-24T23:05:34.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.558+0000 7fdfcffff640 1 -- 172.21.15.32:0/3773647199 learned_addr learned my addr 172.21.15.32:0/3773647199 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:05:34.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.559+0000 7fdfce7fc640 1 -- 172.21.15.32:0/3773647199 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4187377637 0 0) 0x7fdfd00b6430 con 0x7fdfd00af890 2024-04-24T23:05:34.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.559+0000 7fdfceffd640 1 --2- 172.21.15.32:0/3773647199 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdfd00b44d0 0x7fdfd00b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:34.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.559+0000 7fdfce7fc640 1 -- 172.21.15.32:0/3773647199 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdfb8003520 con 0x7fdfd00af890 2024-04-24T23:05:34.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.559+0000 7fdfce7fc640 1 -- 172.21.15.32:0/3773647199 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1512362206 0 0) 0x7fdfb8003520 con 0x7fdfd00af890 2024-04-24T23:05:34.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.559+0000 7fdfce7fc640 1 -- 172.21.15.32:0/3773647199 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdfd00b44d0 msgr2=0x7fdfd00b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:34.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.559+0000 7fdfce7fc640 1 --2- 172.21.15.32:0/3773647199 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdfd00b44d0 0x7fdfd00b5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:34.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.559+0000 7fdfce7fc640 1 -- 172.21.15.32:0/3773647199 >> v1:172.21.15.121:6789/0 conn(0x7fdfd006c5f0 legacy=0x7fdfd006c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:34.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.559+0000 7fdfce7fc640 1 -- 172.21.15.32:0/3773647199 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfd00b39f0 con 0x7fdfd00af890 2024-04-24T23:05:34.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.559+0000 7fdfce7fc640 1 -- 172.21.15.32:0/3773647199 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7fdfc80019f0 con 0x7fdfd00af890 2024-04-24T23:05:34.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.559+0000 7fdfce7fc640 1 -- 172.21.15.32:0/3773647199 >> v1:172.21.15.32:6789/0 conn(0x7fdfd00af890 legacy=0x7fdfd015cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:34.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.559+0000 7fdfce7fc640 1 --2- 172.21.15.32:0/3773647199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfb80039e0 0x7fdfb8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:34.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.559+0000 7fdfce7fc640 1 --2- 172.21.15.32:0/3773647199 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdfb8004390 0x7fdfb8004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:34.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.559+0000 7fdfce7fc640 1 --2- 172.21.15.32:0/3773647199 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdfd00b44d0 0x7fdfb8005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:34.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.559+0000 7fdfce7fc640 1 -- 172.21.15.32:0/3773647199 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd00b6150 con 0x7fdfb80039e0 2024-04-24T23:05:34.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.559+0000 7fdfce7fc640 1 -- 172.21.15.32:0/3773647199 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb800ba70 con 0x7fdfb8004390 2024-04-24T23:05:34.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.559+0000 7fdfce7fc640 1 -- 172.21.15.32:0/3773647199 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fdfb800bc70 con 0x7fdfd00b44d0 2024-04-24T23:05:34.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.559+0000 7fdfcf7fe640 1 --2- 172.21.15.32:0/3773647199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfb80039e0 0x7fdfb8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:34.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.559+0000 7fdfcffff640 1 --2- 172.21.15.32:0/3773647199 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdfd00b44d0 0x7fdfb8005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:34.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.559+0000 7fdfceffd640 1 --2- 172.21.15.32:0/3773647199 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdfb8004390 0x7fdfb8004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:34.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.560+0000 7fdfcf7fe640 1 -- 172.21.15.32:0/3773647199 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdfd00b44d0 msgr2=0x7fdfb8005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:34.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.560+0000 7fdfcf7fe640 1 --2- 172.21.15.32:0/3773647199 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdfd00b44d0 0x7fdfb8005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:34.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.560+0000 7fdfcf7fe640 1 -- 172.21.15.32:0/3773647199 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdfb8004390 msgr2=0x7fdfb8004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:34.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.560+0000 7fdfcf7fe640 1 --2- 172.21.15.32:0/3773647199 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdfb8004390 0x7fdfb8004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:34.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.560+0000 7fdfcf7fe640 1 -- 172.21.15.32:0/3773647199 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfb8003520 con 0x7fdfb80039e0 2024-04-24T23:05:34.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.560+0000 7fdfcf7fe640 1 --2- 172.21.15.32:0/3773647199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfb80039e0 0x7fdfb8003dc0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0001830 tx=0x7fdfb8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c23576657f26e4c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:34.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.560+0000 7fdfce7fc640 1 -- 172.21.15.32:0/3773647199 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc0001430 con 0x7fdfb80039e0 2024-04-24T23:05:34.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.560+0000 7fdfce7fc640 1 -- 172.21.15.32:0/3773647199 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfc00052f0 con 0x7fdfb80039e0 2024-04-24T23:05:34.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.560+0000 7fdfce7fc640 1 -- 172.21.15.32:0/3773647199 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc002ac10 con 0x7fdfb80039e0 2024-04-24T23:05:34.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.561+0000 7fdfd5c4d640 1 -- 172.21.15.32:0/3773647199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfb80039e0 msgr2=0x7fdfb8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:34.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.561+0000 7fdfd5c4d640 1 --2- 172.21.15.32:0/3773647199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfb80039e0 0x7fdfb8003dc0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0001830 tx=0x7fdfb8005830 comp rx=0 tx=0).stop 2024-04-24T23:05:34.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.561+0000 7fdfcffff640 1 -- 172.21.15.32:0/3773647199 reap_dead start 2024-04-24T23:05:34.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.561+0000 7fdfd5c4d640 1 -- 172.21.15.32:0/3773647199 shutdown_connections 2024-04-24T23:05:34.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.561+0000 7fdfd5c4d640 1 -- 172.21.15.32:0/3773647199 >> 172.21.15.32:0/3773647199 conn(0x7fdfd006b110 msgr2=0x7fdfd006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:34.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.561+0000 7fdfd5c4d640 1 -- 172.21.15.32:0/3773647199 shutdown_connections 2024-04-24T23:05:34.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.561+0000 7fdfd5c4d640 1 -- 172.21.15.32:0/3773647199 wait complete. 2024-04-24T23:05:34.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.562+0000 7fdfd5c4d640 1 Processor -- start 2024-04-24T23:05:34.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.562+0000 7fdfd5c4d640 1 -- start start 2024-04-24T23:05:34.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.563+0000 7fdfd5c4d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdfd015bdd0 0x7fdfd015c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:34.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.563+0000 7fdfd5c4d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdfd014afe0 0x7fdfd01585c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:34.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.563+0000 7fdfd5c4d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfd0158b00 0x7fdfd0158f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:34.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.563+0000 7fdfd5c4d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd00b5890 con 0x7fdfd0158b00 2024-04-24T23:05:34.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.563+0000 7fdfd5c4d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd00b3fb0 con 0x7fdfd014afe0 2024-04-24T23:05:34.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.563+0000 7fdfd5c4d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fdfd00b5a10 con 0x7fdfd015bdd0 2024-04-24T23:05:34.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.563+0000 7fdfcffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfd0158b00 0x7fdfd0158f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:34.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.563+0000 7fdfcffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfd0158b00 0x7fdfd0158f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56266/0 (socket says 172.21.15.32:56266) 2024-04-24T23:05:34.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.563+0000 7fdfcffff640 1 -- 172.21.15.32:0/2972039423 learned_addr learned my addr 172.21.15.32:0/2972039423 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:34.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.563+0000 7fdfcf7fe640 1 --2- 172.21.15.32:0/2972039423 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdfd015bdd0 0x7fdfd015c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:34.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.563+0000 7fdfceffd640 1 --2- 172.21.15.32:0/2972039423 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdfd014afe0 0x7fdfd01585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:34.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.564+0000 7fdfcffff640 1 -- 172.21.15.32:0/2972039423 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdfd015bdd0 msgr2=0x7fdfd015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:34.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.564+0000 7fdfcffff640 1 --2- 172.21.15.32:0/2972039423 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdfd015bdd0 0x7fdfd015c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:34.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.564+0000 7fdfcffff640 1 -- 172.21.15.32:0/2972039423 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdfd014afe0 msgr2=0x7fdfd01585c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:34.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.564+0000 7fdfcffff640 1 --2- 172.21.15.32:0/2972039423 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdfd014afe0 0x7fdfd01585c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:34.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.564+0000 7fdfcffff640 1 -- 172.21.15.32:0/2972039423 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfd0149c60 con 0x7fdfd0158b00 2024-04-24T23:05:34.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.564+0000 7fdfcffff640 1 --2- 172.21.15.32:0/2972039423 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfd0158b00 0x7fdfd0158f80 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fdfd0073080 tx=0x7fdfc800a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:34.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.564+0000 7fdfb7fff640 1 -- 172.21.15.32:0/2972039423 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc80133f0 con 0x7fdfd0158b00 2024-04-24T23:05:34.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.564+0000 7fdfb7fff640 1 -- 172.21.15.32:0/2972039423 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfc8019070 con 0x7fdfd0158b00 2024-04-24T23:05:34.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.564+0000 7fdfd5c4d640 1 -- 172.21.15.32:0/2972039423 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfd0149f40 con 0x7fdfd0158b00 2024-04-24T23:05:34.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.564+0000 7fdfb7fff640 1 -- 172.21.15.32:0/2972039423 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc8013790 con 0x7fdfd0158b00 2024-04-24T23:05:34.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.564+0000 7fdfd5c4d640 1 -- 172.21.15.32:0/2972039423 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfd014a500 con 0x7fdfd0158b00 2024-04-24T23:05:34.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.565+0000 7fdfd5c4d640 1 -- 172.21.15.32:0/2972039423 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfd0068430 con 0x7fdfd0158b00 2024-04-24T23:05:34.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.566+0000 7fdfb7fff640 1 -- 172.21.15.32:0/2972039423 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdfc8013930 con 0x7fdfd0158b00 2024-04-24T23:05:34.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.566+0000 7fdfb7fff640 1 --2- 172.21.15.32:0/2972039423 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdfbc06be30 0x7fdfbc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:34.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.566+0000 7fdfb7fff640 1 -- 172.21.15.32:0/2972039423 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 5709+0+0 (secure 0 0 0) 0x7fdfc808b470 con 0x7fdfd0158b00 2024-04-24T23:05:34.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.566+0000 7fdfcf7fe640 1 --2- 172.21.15.32:0/2972039423 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdfbc06be30 0x7fdfbc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:34.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.567+0000 7fdfcf7fe640 1 --2- 172.21.15.32:0/2972039423 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdfbc06be30 0x7fdfbc06e2f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0032000 tx=0x7fdfc0009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:34.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.570+0000 7fdfb7fff640 1 -- 172.21.15.32:0/2972039423 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdfc8059eb0 con 0x7fdfd0158b00 2024-04-24T23:05:34.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.753+0000 7fdfd5c4d640 1 -- 172.21.15.32:0/2972039423 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdfd00b5670 con 0x7fdfd0158b00 2024-04-24T23:05:34.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.754+0000 7fdfb7fff640 1 -- 172.21.15.32:0/2972039423 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) ==== 74+0+16709 (secure 0 0 0) 0x7fdfc8007a90 con 0x7fdfd0158b00 2024-04-24T23:05:34.757 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:05:34.757 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":139,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:05:33.790751+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:05:32.064234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-24T23:05:32.760777+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:05:34.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.756+0000 7fdfd5c4d640 1 -- 172.21.15.32:0/2972039423 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdfbc06be30 msgr2=0x7fdfbc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:34.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.756+0000 7fdfd5c4d640 1 --2- 172.21.15.32:0/2972039423 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdfbc06be30 0x7fdfbc06e2f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0032000 tx=0x7fdfc0009030 comp rx=0 tx=0).stop 2024-04-24T23:05:34.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.756+0000 7fdfd5c4d640 1 -- 172.21.15.32:0/2972039423 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfd0158b00 msgr2=0x7fdfd0158f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:34.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.756+0000 7fdfd5c4d640 1 --2- 172.21.15.32:0/2972039423 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfd0158b00 0x7fdfd0158f80 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fdfd0073080 tx=0x7fdfc800a730 comp rx=0 tx=0).stop 2024-04-24T23:05:34.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.757+0000 7fdfd5c4d640 1 -- 172.21.15.32:0/2972039423 shutdown_connections 2024-04-24T23:05:34.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.757+0000 7fdfd5c4d640 1 --2- 172.21.15.32:0/2972039423 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdfbc06be30 0x7fdfbc06e2f0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:34.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.757+0000 7fdfd5c4d640 1 --2- 172.21.15.32:0/2972039423 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfd0158b00 0x7fdfd0158f80 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:34.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.757+0000 7fdfd5c4d640 1 --2- 172.21.15.32:0/2972039423 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdfd014afe0 0x7fdfd01585c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:34.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.757+0000 7fdfd5c4d640 1 --2- 172.21.15.32:0/2972039423 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdfd015bdd0 0x7fdfd015c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:34.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.757+0000 7fdfd5c4d640 1 -- 172.21.15.32:0/2972039423 >> 172.21.15.32:0/2972039423 conn(0x7fdfd006b110 msgr2=0x7fdfd00b48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:34.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.757+0000 7fdfd5c4d640 1 -- 172.21.15.32:0/2972039423 shutdown_connections 2024-04-24T23:05:34.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.757+0000 7fdfd5c4d640 1 -- 172.21.15.32:0/2972039423 wait complete. 2024-04-24T23:05:34.771 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:05:34.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.871+0000 7f3f7af5a640 1 Processor -- start 2024-04-24T23:05:34.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.871+0000 7f3f7af5a640 1 -- start start 2024-04-24T23:05:34.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.871+0000 7f3f7af5a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3f7415e9b0 0x7f3f740d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:34.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.871+0000 7f3f7af5a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3f740d4210 0x7f3f7406b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:34.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.871+0000 7f3f7af5a640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3f740ba0e0 0x7f3f7406b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:34.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.871+0000 7f3f7af5a640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f3f740d3170 con 0x7f3f740d4210 2024-04-24T23:05:34.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.871+0000 7f3f7af5a640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f3f740d3450 con 0x7f3f740ba0e0 2024-04-24T23:05:34.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.871+0000 7f3f7af5a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f3f740d3730 con 0x7f3f7415e9b0 2024-04-24T23:05:34.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.872+0000 7f3f73fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3f740d4210 0x7f3f7406b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:34.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.872+0000 7f3f73fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3f740d4210 0x7f3f7406b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56280/0 (socket says 172.21.15.32:56280) 2024-04-24T23:05:34.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.872+0000 7f3f73fff640 1 -- 172.21.15.32:0/2402999798 learned_addr learned my addr 172.21.15.32:0/2402999798 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:34.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.872+0000 7f3f794f7640 1 --2- 172.21.15.32:0/2402999798 >> v2:172.21.15.121:3300/0 conn(0x7f3f740ba0e0 0x7f3f7406b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:34.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.872+0000 7f3f78cf6640 1 --2- 172.21.15.32:0/2402999798 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3f7415e9b0 0x7f3f740d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:34.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.872+0000 7f3f73fff640 1 -- 172.21.15.32:0/2402999798 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3f7415e9b0 msgr2=0x7f3f740d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:34.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.872+0000 7f3f73fff640 1 --2- 172.21.15.32:0/2402999798 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3f7415e9b0 0x7f3f740d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:34.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.872+0000 7f3f73fff640 1 -- 172.21.15.32:0/2402999798 >> v2:172.21.15.121:3300/0 conn(0x7f3f740ba0e0 msgr2=0x7f3f7406b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:34.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.872+0000 7f3f73fff640 1 --2- 172.21.15.32:0/2402999798 >> v2:172.21.15.121:3300/0 conn(0x7f3f740ba0e0 0x7f3f7406b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:34.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.872+0000 7f3f73fff640 1 -- 172.21.15.32:0/2402999798 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f740d3a10 con 0x7f3f740d4210 2024-04-24T23:05:34.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.872+0000 7f3f794f7640 1 --2- 172.21.15.32:0/2402999798 >> v2:172.21.15.121:3300/0 conn(0x7f3f740ba0e0 0x7f3f7406b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:05:34.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.872+0000 7f3f78cf6640 1 --2- 172.21.15.32:0/2402999798 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3f7415e9b0 0x7f3f740d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:05:34.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.873+0000 7f3f73fff640 1 --2- 172.21.15.32:0/2402999798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f740d4210 0x7f3f7406b110 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f3f740b61d0 tx=0x7f3f6002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e954e20d623a01e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:34.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.873+0000 7f3f737fe640 1 -- 172.21.15.32:0/2402999798 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f60037070 con 0x7f3f740d4210 2024-04-24T23:05:34.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.873+0000 7f3f737fe640 1 -- 172.21.15.32:0/2402999798 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f60032440 con 0x7f3f740d4210 2024-04-24T23:05:34.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.873+0000 7f3f737fe640 1 -- 172.21.15.32:0/2402999798 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f60032810 con 0x7f3f740d4210 2024-04-24T23:05:34.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.873+0000 7f3f7af5a640 1 -- 172.21.15.32:0/2402999798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f740d4210 msgr2=0x7f3f7406b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:34.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.873+0000 7f3f7af5a640 1 --2- 172.21.15.32:0/2402999798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f740d4210 0x7f3f7406b110 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f3f740b61d0 tx=0x7f3f6002a1e0 comp rx=0 tx=0).stop 2024-04-24T23:05:34.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.873+0000 7f3f7af5a640 1 -- 172.21.15.32:0/2402999798 shutdown_connections 2024-04-24T23:05:34.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.873+0000 7f3f7af5a640 1 --2- 172.21.15.32:0/2402999798 >> v2:172.21.15.121:3300/0 conn(0x7f3f740ba0e0 0x7f3f7406b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:34.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.873+0000 7f3f7af5a640 1 --2- 172.21.15.32:0/2402999798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f740d4210 0x7f3f7406b110 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:34.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.873+0000 7f3f7af5a640 1 --2- 172.21.15.32:0/2402999798 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3f7415e9b0 0x7f3f740d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:34.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.873+0000 7f3f7af5a640 1 -- 172.21.15.32:0/2402999798 >> 172.21.15.32:0/2402999798 conn(0x7f3f740d11d0 msgr2=0x7f3f740d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:34.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.873+0000 7f3f7af5a640 1 -- 172.21.15.32:0/2402999798 shutdown_connections 2024-04-24T23:05:34.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.873+0000 7f3f7af5a640 1 -- 172.21.15.32:0/2402999798 wait complete. 2024-04-24T23:05:34.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.874+0000 7f3f7af5a640 1 Processor -- start 2024-04-24T23:05:34.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.875+0000 7f3f7af5a640 1 -- start start 2024-04-24T23:05:34.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.875+0000 7f3f7af5a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3f740ba0e0 0x7f3f741f3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:34.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.875+0000 7f3f7af5a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3f740d4210 0x7f3f740cd790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:34.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.875+0000 7f3f7af5a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f7415e9b0 0x7f3f740c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:34.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.875+0000 7f3f7af5a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3f74071730 con 0x7f3f7415e9b0 2024-04-24T23:05:34.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.875+0000 7f3f7af5a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3f740d2400 con 0x7f3f740d4210 2024-04-24T23:05:34.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.875+0000 7f3f7af5a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f3f740718b0 con 0x7f3f740ba0e0 2024-04-24T23:05:34.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.876+0000 7f3f794f7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f7415e9b0 0x7f3f740c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:34.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.876+0000 7f3f794f7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f7415e9b0 0x7f3f740c90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56290/0 (socket says 172.21.15.32:56290) 2024-04-24T23:05:34.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.876+0000 7f3f794f7640 1 -- 172.21.15.32:0/624340548 learned_addr learned my addr 172.21.15.32:0/624340548 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:34.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.876+0000 7f3f73fff640 1 --2- 172.21.15.32:0/624340548 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3f740d4210 0x7f3f740cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:34.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.876+0000 7f3f78cf6640 1 --2- 172.21.15.32:0/624340548 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3f740ba0e0 0x7f3f741f3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:34.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.876+0000 7f3f794f7640 1 -- 172.21.15.32:0/624340548 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3f740ba0e0 msgr2=0x7f3f741f3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:34.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.876+0000 7f3f794f7640 1 --2- 172.21.15.32:0/624340548 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3f740ba0e0 0x7f3f741f3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:34.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.876+0000 7f3f794f7640 1 -- 172.21.15.32:0/624340548 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3f740d4210 msgr2=0x7f3f740cd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:34.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.876+0000 7f3f794f7640 1 --2- 172.21.15.32:0/624340548 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3f740d4210 0x7f3f740cd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:34.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.876+0000 7f3f794f7640 1 -- 172.21.15.32:0/624340548 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f740c8320 con 0x7f3f7415e9b0 2024-04-24T23:05:34.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.876+0000 7f3f78cf6640 1 --2- 172.21.15.32:0/624340548 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3f740ba0e0 0x7f3f741f3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:05:34.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.876+0000 7f3f73fff640 1 --2- 172.21.15.32:0/624340548 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3f740d4210 0x7f3f740cd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:05:34.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.876+0000 7f3f794f7640 1 --2- 172.21.15.32:0/624340548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f7415e9b0 0x7f3f740c90a0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c00cef0 tx=0x7f3f6c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:34.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.876+0000 7f3f717fa640 1 -- 172.21.15.32:0/624340548 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c01a070 con 0x7f3f7415e9b0 2024-04-24T23:05:34.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.876+0000 7f3f717fa640 1 -- 172.21.15.32:0/624340548 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f6c00d840 con 0x7f3f7415e9b0 2024-04-24T23:05:34.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.876+0000 7f3f7af5a640 1 -- 172.21.15.32:0/624340548 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f740c85c0 con 0x7f3f7415e9b0 2024-04-24T23:05:34.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.876+0000 7f3f717fa640 1 -- 172.21.15.32:0/624340548 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c00dc30 con 0x7f3f7415e9b0 2024-04-24T23:05:34.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.876+0000 7f3f7af5a640 1 -- 172.21.15.32:0/624340548 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f740ce230 con 0x7f3f7415e9b0 2024-04-24T23:05:34.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.877+0000 7f3f7af5a640 1 -- 172.21.15.32:0/624340548 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f740d2ec0 con 0x7f3f7415e9b0 2024-04-24T23:05:34.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.878+0000 7f3f717fa640 1 -- 172.21.15.32:0/624340548 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3f6c01d070 con 0x7f3f7415e9b0 2024-04-24T23:05:34.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.878+0000 7f3f717fa640 1 --2- 172.21.15.32:0/624340548 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3f5806be60 0x7f3f5806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:34.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.878+0000 7f3f717fa640 1 -- 172.21.15.32:0/624340548 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 5709+0+0 (secure 0 0 0) 0x7f3f6c08b600 con 0x7f3f7415e9b0 2024-04-24T23:05:34.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.878+0000 7f3f78cf6640 1 --2- 172.21.15.32:0/624340548 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3f5806be60 0x7f3f5806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:34.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.879+0000 7f3f78cf6640 1 --2- 172.21.15.32:0/624340548 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3f5806be60 0x7f3f5806e320 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f3f68006fd0 tx=0x7f3f68008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:34.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:34.882+0000 7f3f717fa640 1 -- 172.21.15.32:0/624340548 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3f740d2ec0 con 0x7f3f7415e9b0 2024-04-24T23:05:35.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.065+0000 7f3f7af5a640 1 -- 172.21.15.32:0/624340548 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3f740c8d30 con 0x7f3f7415e9b0 2024-04-24T23:05:35.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.067+0000 7f3f717fa640 1 -- 172.21.15.32:0/624340548 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) ==== 74+0+16709 (secure 0 0 0) 0x7f3f6c059fb0 con 0x7f3f7415e9b0 2024-04-24T23:05:35.069 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:05:35.070 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":139,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:05:33.790751+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:05:32.064234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-24T23:05:32.760777+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:05:35.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.069+0000 7f3f7af5a640 1 -- 172.21.15.32:0/624340548 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3f5806be60 msgr2=0x7f3f5806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:35.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.069+0000 7f3f7af5a640 1 --2- 172.21.15.32:0/624340548 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3f5806be60 0x7f3f5806e320 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f3f68006fd0 tx=0x7f3f68008040 comp rx=0 tx=0).stop 2024-04-24T23:05:35.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.069+0000 7f3f7af5a640 1 -- 172.21.15.32:0/624340548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f7415e9b0 msgr2=0x7f3f740c90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:35.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.069+0000 7f3f7af5a640 1 --2- 172.21.15.32:0/624340548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f7415e9b0 0x7f3f740c90a0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c00cef0 tx=0x7f3f6c00a710 comp rx=0 tx=0).stop 2024-04-24T23:05:35.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.069+0000 7f3f7af5a640 1 -- 172.21.15.32:0/624340548 shutdown_connections 2024-04-24T23:05:35.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.069+0000 7f3f7af5a640 1 --2- 172.21.15.32:0/624340548 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3f5806be60 0x7f3f5806e320 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:35.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.069+0000 7f3f7af5a640 1 --2- 172.21.15.32:0/624340548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f7415e9b0 0x7f3f740c90a0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:35.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.069+0000 7f3f7af5a640 1 --2- 172.21.15.32:0/624340548 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3f740d4210 0x7f3f740cd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:35.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.069+0000 7f3f7af5a640 1 --2- 172.21.15.32:0/624340548 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3f740ba0e0 0x7f3f741f3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:35.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.069+0000 7f3f7af5a640 1 -- 172.21.15.32:0/624340548 >> 172.21.15.32:0/624340548 conn(0x7f3f740d11d0 msgr2=0x7f3f74159a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:35.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.069+0000 7f3f7af5a640 1 -- 172.21.15.32:0/624340548 shutdown_connections 2024-04-24T23:05:35.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.069+0000 7f3f7af5a640 1 -- 172.21.15.32:0/624340548 wait complete. 2024-04-24T23:05:35.083 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-24T23:05:35.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.183+0000 7f10c9ba2640 1 Processor -- start 2024-04-24T23:05:35.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.184+0000 7f10c9ba2640 1 -- start start 2024-04-24T23:05:35.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.184+0000 7f10c9ba2640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f10c415e9b0 0x7f10c40d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:35.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.184+0000 7f10c9ba2640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f10c40d4210 0x7f10c406b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:35.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.184+0000 7f10c9ba2640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10c40d3170 con 0x7f10c40ba0e0 2024-04-24T23:05:35.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.184+0000 7f10c9ba2640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f10c40b8900 con 0x7f10c415e9b0 2024-04-24T23:05:35.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.184+0000 7f10c9ba2640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f10c40d3450 con 0x7f10c40d4210 2024-04-24T23:05:35.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.185+0000 7f10c37fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f10c415e9b0 0x7f10c40d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:35.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.185+0000 7f10c37fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f10c415e9b0 0x7f10c40d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56292/0 (socket says 172.21.15.32:56292) 2024-04-24T23:05:35.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.185+0000 7f10c37fe640 1 -- 172.21.15.32:0/3501979474 learned_addr learned my addr 172.21.15.32:0/3501979474 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:35.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.185+0000 7f10c2ffd640 1 --2- 172.21.15.32:0/3501979474 >> v2:172.21.15.121:3300/0 conn(0x7f10c40d4210 0x7f10c406b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:35.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.185+0000 7f10c37fe640 1 -- 172.21.15.32:0/3501979474 >> v2:172.21.15.121:3300/0 conn(0x7f10c40d4210 msgr2=0x7f10c406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:35.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.185+0000 7f10c37fe640 1 --2- 172.21.15.32:0/3501979474 >> v2:172.21.15.121:3300/0 conn(0x7f10c40d4210 0x7f10c406b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:35.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.185+0000 7f10c37fe640 1 -- 172.21.15.32:0/3501979474 >> v1:172.21.15.121:6789/0 conn(0x7f10c40ba0e0 legacy=0x7f10c4071730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:35.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.185+0000 7f10c37fe640 1 -- 172.21.15.32:0/3501979474 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10c40d3730 con 0x7f10c415e9b0 2024-04-24T23:05:35.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.185+0000 7f10c2ffd640 1 --2- 172.21.15.32:0/3501979474 >> v2:172.21.15.121:3300/0 conn(0x7f10c40d4210 0x7f10c406b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:05:35.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.186+0000 7f10c37fe640 1 --2- 172.21.15.32:0/3501979474 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10c415e9b0 0x7f10c40d2b90 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f10c40730f0 tx=0x7f10b402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f34514a4abb0de2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:35.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.186+0000 7f10c27fc640 1 -- 172.21.15.32:0/3501979474 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b4037070 con 0x7f10c415e9b0 2024-04-24T23:05:35.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.186+0000 7f10c27fc640 1 -- 172.21.15.32:0/3501979474 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10b4032440 con 0x7f10c415e9b0 2024-04-24T23:05:35.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.186+0000 7f10c27fc640 1 -- 172.21.15.32:0/3501979474 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b4032810 con 0x7f10c415e9b0 2024-04-24T23:05:35.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.186+0000 7f10c9ba2640 1 -- 172.21.15.32:0/3501979474 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10c415e9b0 msgr2=0x7f10c40d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:35.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.186+0000 7f10c9ba2640 1 --2- 172.21.15.32:0/3501979474 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10c415e9b0 0x7f10c40d2b90 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f10c40730f0 tx=0x7f10b402a1a0 comp rx=0 tx=0).stop 2024-04-24T23:05:35.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.186+0000 7f10c9ba2640 1 -- 172.21.15.32:0/3501979474 shutdown_connections 2024-04-24T23:05:35.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.186+0000 7f10c9ba2640 1 --2- 172.21.15.32:0/3501979474 >> v2:172.21.15.121:3300/0 conn(0x7f10c40d4210 0x7f10c406b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:35.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.186+0000 7f10c9ba2640 1 --2- 172.21.15.32:0/3501979474 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10c415e9b0 0x7f10c40d2b90 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:35.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.186+0000 7f10c9ba2640 1 -- 172.21.15.32:0/3501979474 >> 172.21.15.32:0/3501979474 conn(0x7f10c40d11d0 msgr2=0x7f10c40d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:35.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.186+0000 7f10c9ba2640 1 -- 172.21.15.32:0/3501979474 shutdown_connections 2024-04-24T23:05:35.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.186+0000 7f10c9ba2640 1 -- 172.21.15.32:0/3501979474 wait complete. 2024-04-24T23:05:35.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.189+0000 7f10c9ba2640 1 Processor -- start 2024-04-24T23:05:35.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.189+0000 7f10c9ba2640 1 -- start start 2024-04-24T23:05:35.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.189+0000 7f10c9ba2640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f10c40ba0e0 0x7f10c4161440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:35.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.189+0000 7f10c9ba2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10c40d4210 0x7f10c4161b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:35.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.189+0000 7f10c9ba2640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10c415e9b0 0x7f10c41f3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:35.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.189+0000 7f10c9ba2640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f10c40d2400 con 0x7f10c40d4210 2024-04-24T23:05:35.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.189+0000 7f10c9ba2640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f10c40d3cf0 con 0x7f10c415e9b0 2024-04-24T23:05:35.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.189+0000 7f10c9ba2640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f10c406b650 con 0x7f10c40ba0e0 2024-04-24T23:05:35.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.189+0000 7f10c2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10c40d4210 0x7f10c4161b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:35.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.189+0000 7f10c2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10c40d4210 0x7f10c4161b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56302/0 (socket says 172.21.15.32:56302) 2024-04-24T23:05:35.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.189+0000 7f10c2ffd640 1 -- 172.21.15.32:0/988925765 learned_addr learned my addr 172.21.15.32:0/988925765 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:35.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.189+0000 7f10c37fe640 1 --2- 172.21.15.32:0/988925765 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f10c40ba0e0 0x7f10c4161440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:35.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.189+0000 7f10c3fff640 1 --2- 172.21.15.32:0/988925765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10c415e9b0 0x7f10c41f3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:35.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.189+0000 7f10c2ffd640 1 -- 172.21.15.32:0/988925765 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f10c40ba0e0 msgr2=0x7f10c4161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:35.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.189+0000 7f10c2ffd640 1 --2- 172.21.15.32:0/988925765 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f10c40ba0e0 0x7f10c4161440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:35.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.189+0000 7f10c2ffd640 1 -- 172.21.15.32:0/988925765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10c415e9b0 msgr2=0x7f10c41f3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:35.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.189+0000 7f10c2ffd640 1 --2- 172.21.15.32:0/988925765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10c415e9b0 0x7f10c41f3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:35.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.189+0000 7f10c2ffd640 1 -- 172.21.15.32:0/988925765 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10c41603a0 con 0x7f10c40d4210 2024-04-24T23:05:35.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.189+0000 7f10c2ffd640 1 --2- 172.21.15.32:0/988925765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10c40d4210 0x7f10c4161b20 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f10b800c7f0 tx=0x7f10b800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:35.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.189+0000 7f10abfff640 1 -- 172.21.15.32:0/988925765 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b8015070 con 0x7f10c40d4210 2024-04-24T23:05:35.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.189+0000 7f10abfff640 1 -- 172.21.15.32:0/988925765 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10b8007a30 con 0x7f10c40d4210 2024-04-24T23:05:35.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.189+0000 7f10c9ba2640 1 -- 172.21.15.32:0/988925765 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10c4160680 con 0x7f10c40d4210 2024-04-24T23:05:35.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.189+0000 7f10abfff640 1 -- 172.21.15.32:0/988925765 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b8007e00 con 0x7f10c40d4210 2024-04-24T23:05:35.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.189+0000 7f10c9ba2640 1 -- 172.21.15.32:0/988925765 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10c4160c40 con 0x7f10c40d4210 2024-04-24T23:05:35.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.190+0000 7f10c9ba2640 1 -- 172.21.15.32:0/988925765 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10c4068430 con 0x7f10c40d4210 2024-04-24T23:05:35.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.191+0000 7f10abfff640 1 -- 172.21.15.32:0/988925765 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f10b8018070 con 0x7f10c40d4210 2024-04-24T23:05:35.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.191+0000 7f10abfff640 1 --2- 172.21.15.32:0/988925765 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f10b006be30 0x7f10b006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:35.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.191+0000 7f10abfff640 1 -- 172.21.15.32:0/988925765 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 5709+0+0 (secure 0 0 0) 0x7f10b808ae50 con 0x7f10c40d4210 2024-04-24T23:05:35.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.192+0000 7f10c37fe640 1 --2- 172.21.15.32:0/988925765 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f10b006be30 0x7f10b006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:35.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.192+0000 7f10c37fe640 1 --2- 172.21.15.32:0/988925765 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f10b006be30 0x7f10b006e2f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f10b4034000 tx=0x7f10b402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:35.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.195+0000 7f10abfff640 1 -- 172.21.15.32:0/988925765 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f10b8059800 con 0x7f10c40d4210 2024-04-24T23:05:35.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.419+0000 7f10c9ba2640 1 -- 172.21.15.32:0/988925765 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f10c40b82c0 con 0x7f10c40d4210 2024-04-24T23:05:35.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.437+0000 7f10abfff640 1 -- 172.21.15.32:0/988925765 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v57) ==== 118+0+0 (secure 0 0 0) 0x7f10b8059100 con 0x7f10c40d4210 2024-04-24T23:05:35.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.439+0000 7f10c9ba2640 1 -- 172.21.15.32:0/988925765 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f10b006be30 msgr2=0x7f10b006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:35.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.439+0000 7f10c9ba2640 1 --2- 172.21.15.32:0/988925765 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f10b006be30 0x7f10b006e2f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f10b4034000 tx=0x7f10b402e000 comp rx=0 tx=0).stop 2024-04-24T23:05:35.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.439+0000 7f10c9ba2640 1 -- 172.21.15.32:0/988925765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10c40d4210 msgr2=0x7f10c4161b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:35.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.439+0000 7f10c9ba2640 1 --2- 172.21.15.32:0/988925765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10c40d4210 0x7f10c4161b20 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f10b800c7f0 tx=0x7f10b800cbf0 comp rx=0 tx=0).stop 2024-04-24T23:05:35.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.439+0000 7f10c9ba2640 1 -- 172.21.15.32:0/988925765 shutdown_connections 2024-04-24T23:05:35.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.439+0000 7f10c9ba2640 1 --2- 172.21.15.32:0/988925765 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f10b006be30 0x7f10b006e2f0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:35.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.439+0000 7f10c9ba2640 1 --2- 172.21.15.32:0/988925765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10c40d4210 0x7f10c4161b20 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:35.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.439+0000 7f10c9ba2640 1 --2- 172.21.15.32:0/988925765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10c415e9b0 0x7f10c41f3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:35.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.439+0000 7f10c9ba2640 1 --2- 172.21.15.32:0/988925765 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f10c40ba0e0 0x7f10c4161440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:35.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.439+0000 7f10c9ba2640 1 -- 172.21.15.32:0/988925765 >> 172.21.15.32:0/988925765 conn(0x7f10c40d11d0 msgr2=0x7f10c40cefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:35.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.439+0000 7f10c9ba2640 1 -- 172.21.15.32:0/988925765 shutdown_connections 2024-04-24T23:05:35.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.439+0000 7f10c9ba2640 1 -- 172.21.15.32:0/988925765 wait complete. 2024-04-24T23:05:35.452 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-24T23:05:35.452 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-24T23:05:35.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.554+0000 7fcb53ea5640 1 Processor -- start 2024-04-24T23:05:35.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.554+0000 7fcb53ea5640 1 -- start start 2024-04-24T23:05:35.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.555+0000 7fcb53ea5640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fcb4c162320 0x7fcb4c0ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:35.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.555+0000 7fcb53ea5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcb4c0b5af0 0x7fcb4c162700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:35.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.555+0000 7fcb53ea5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcb4c162c40 0x7fcb4c0ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:35.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.555+0000 7fcb53ea5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fcb4c0b5660 con 0x7fcb4c162c40 2024-04-24T23:05:35.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.555+0000 7fcb53ea5640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fcb4c0cd270 con 0x7fcb4c162320 2024-04-24T23:05:35.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.555+0000 7fcb53ea5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fcb4c071610 con 0x7fcb4c0b5af0 2024-04-24T23:05:35.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.555+0000 7fcb52442640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcb4c162c40 0x7fcb4c0ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:35.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.555+0000 7fcb52442640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcb4c162c40 0x7fcb4c0ae8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56310/0 (socket says 172.21.15.32:56310) 2024-04-24T23:05:35.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.555+0000 7fcb52442640 1 -- 172.21.15.32:0/888842376 learned_addr learned my addr 172.21.15.32:0/888842376 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:35.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.555+0000 7fcb51c41640 1 --2- 172.21.15.32:0/888842376 >> v2:172.21.15.121:3300/0 conn(0x7fcb4c162320 0x7fcb4c0ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:35.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.555+0000 7fcb51440640 1 --2- 172.21.15.32:0/888842376 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcb4c0b5af0 0x7fcb4c162700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:35.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.555+0000 7fcb52442640 1 -- 172.21.15.32:0/888842376 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcb4c0b5af0 msgr2=0x7fcb4c162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:35.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.555+0000 7fcb52442640 1 --2- 172.21.15.32:0/888842376 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcb4c0b5af0 0x7fcb4c162700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:35.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.555+0000 7fcb52442640 1 -- 172.21.15.32:0/888842376 >> v2:172.21.15.121:3300/0 conn(0x7fcb4c162320 msgr2=0x7fcb4c0ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:35.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.555+0000 7fcb52442640 1 --2- 172.21.15.32:0/888842376 >> v2:172.21.15.121:3300/0 conn(0x7fcb4c162320 0x7fcb4c0ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:35.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.555+0000 7fcb52442640 1 -- 172.21.15.32:0/888842376 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb4c0718f0 con 0x7fcb4c162c40 2024-04-24T23:05:35.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.555+0000 7fcb51c41640 1 --2- 172.21.15.32:0/888842376 >> v2:172.21.15.121:3300/0 conn(0x7fcb4c162320 0x7fcb4c0ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:05:35.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.556+0000 7fcb52442640 1 --2- 172.21.15.32:0/888842376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb4c162c40 0x7fcb4c0ae8b0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c06c5b0 tx=0x7fcb3c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f502d0f743d4cef server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:35.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.556+0000 7fcb50c3f640 1 -- 172.21.15.32:0/888842376 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c036070 con 0x7fcb4c162c40 2024-04-24T23:05:35.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.556+0000 7fcb50c3f640 1 -- 172.21.15.32:0/888842376 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb3c031600 con 0x7fcb4c162c40 2024-04-24T23:05:35.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.556+0000 7fcb50c3f640 1 -- 172.21.15.32:0/888842376 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c0319d0 con 0x7fcb4c162c40 2024-04-24T23:05:35.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.557+0000 7fcb53ea5640 1 -- 172.21.15.32:0/888842376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb4c162c40 msgr2=0x7fcb4c0ae8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:35.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.557+0000 7fcb53ea5640 1 --2- 172.21.15.32:0/888842376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb4c162c40 0x7fcb4c0ae8b0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c06c5b0 tx=0x7fcb3c02a610 comp rx=0 tx=0).stop 2024-04-24T23:05:35.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.557+0000 7fcb53ea5640 1 -- 172.21.15.32:0/888842376 shutdown_connections 2024-04-24T23:05:35.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.557+0000 7fcb53ea5640 1 --2- 172.21.15.32:0/888842376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb4c162c40 0x7fcb4c0ae8b0 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:35.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.557+0000 7fcb53ea5640 1 --2- 172.21.15.32:0/888842376 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcb4c0b5af0 0x7fcb4c162700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:35.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.557+0000 7fcb53ea5640 1 --2- 172.21.15.32:0/888842376 >> v2:172.21.15.121:3300/0 conn(0x7fcb4c162320 0x7fcb4c0ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:35.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.557+0000 7fcb53ea5640 1 -- 172.21.15.32:0/888842376 >> 172.21.15.32:0/888842376 conn(0x7fcb4c0d0ae0 msgr2=0x7fcb4c0d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:35.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.557+0000 7fcb53ea5640 1 -- 172.21.15.32:0/888842376 shutdown_connections 2024-04-24T23:05:35.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.557+0000 7fcb53ea5640 1 -- 172.21.15.32:0/888842376 wait complete. 2024-04-24T23:05:35.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.558+0000 7fcb53ea5640 1 Processor -- start 2024-04-24T23:05:35.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.558+0000 7fcb53ea5640 1 -- start start 2024-04-24T23:05:35.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.559+0000 7fcb53ea5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb4c0b5af0 0x7fcb4c1f7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:35.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.559+0000 7fcb53ea5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcb4c162320 0x7fcb4c06dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:35.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.559+0000 7fcb53ea5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcb4c162c40 0x7fcb4c06e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:35.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.559+0000 7fcb53ea5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4c067950 con 0x7fcb4c0b5af0 2024-04-24T23:05:35.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.559+0000 7fcb53ea5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4c15c630 con 0x7fcb4c162c40 2024-04-24T23:05:35.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.559+0000 7fcb53ea5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fcb4c071bd0 con 0x7fcb4c162320 2024-04-24T23:05:35.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.559+0000 7fcb51c41640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb4c0b5af0 0x7fcb4c1f7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:35.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.559+0000 7fcb51c41640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb4c0b5af0 0x7fcb4c1f7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56316/0 (socket says 172.21.15.32:56316) 2024-04-24T23:05:35.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.559+0000 7fcb51c41640 1 -- 172.21.15.32:0/2920583654 learned_addr learned my addr 172.21.15.32:0/2920583654 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:35.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.559+0000 7fcb52442640 1 --2- 172.21.15.32:0/2920583654 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcb4c162c40 0x7fcb4c06e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:35.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.559+0000 7fcb51440640 1 --2- 172.21.15.32:0/2920583654 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcb4c162320 0x7fcb4c06dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:35.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.559+0000 7fcb51c41640 1 -- 172.21.15.32:0/2920583654 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcb4c162320 msgr2=0x7fcb4c06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:35.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.559+0000 7fcb51c41640 1 --2- 172.21.15.32:0/2920583654 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcb4c162320 0x7fcb4c06dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:35.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.559+0000 7fcb51c41640 1 -- 172.21.15.32:0/2920583654 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcb4c162c40 msgr2=0x7fcb4c06e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:35.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.559+0000 7fcb51c41640 1 --2- 172.21.15.32:0/2920583654 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcb4c162c40 0x7fcb4c06e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:35.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.559+0000 7fcb51c41640 1 -- 172.21.15.32:0/2920583654 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb4c0cd820 con 0x7fcb4c0b5af0 2024-04-24T23:05:35.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.559+0000 7fcb52442640 1 --2- 172.21.15.32:0/2920583654 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcb4c162c40 0x7fcb4c06e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:05:35.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.559+0000 7fcb51c41640 1 --2- 172.21.15.32:0/2920583654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb4c0b5af0 0x7fcb4c1f7c60 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fcb4400e7e0 tx=0x7fcb4400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:35.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.560+0000 7fcb4a7fc640 1 -- 172.21.15.32:0/2920583654 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4400f040 con 0x7fcb4c0b5af0 2024-04-24T23:05:35.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.560+0000 7fcb4a7fc640 1 -- 172.21.15.32:0/2920583654 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb44004030 con 0x7fcb4c0b5af0 2024-04-24T23:05:35.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.560+0000 7fcb53ea5640 1 -- 172.21.15.32:0/2920583654 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb4c160ad0 con 0x7fcb4c0b5af0 2024-04-24T23:05:35.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.560+0000 7fcb4a7fc640 1 -- 172.21.15.32:0/2920583654 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb44004400 con 0x7fcb4c0b5af0 2024-04-24T23:05:35.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.560+0000 7fcb53ea5640 1 -- 172.21.15.32:0/2920583654 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb4c161090 con 0x7fcb4c0b5af0 2024-04-24T23:05:35.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.561+0000 7fcb53ea5640 1 -- 172.21.15.32:0/2920583654 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb4c068430 con 0x7fcb4c0b5af0 2024-04-24T23:05:35.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.561+0000 7fcb4a7fc640 1 -- 172.21.15.32:0/2920583654 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcb4400c6d0 con 0x7fcb4c0b5af0 2024-04-24T23:05:35.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.562+0000 7fcb4a7fc640 1 --2- 172.21.15.32:0/2920583654 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcb3806be30 0x7fcb3806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:35.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.562+0000 7fcb4a7fc640 1 -- 172.21.15.32:0/2920583654 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5709+0+0 (secure 0 0 0) 0x7fcb4408abb0 con 0x7fcb4c0b5af0 2024-04-24T23:05:35.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.562+0000 7fcb51440640 1 --2- 172.21.15.32:0/2920583654 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcb3806be30 0x7fcb3806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:35.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.563+0000 7fcb51440640 1 --2- 172.21.15.32:0/2920583654 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcb3806be30 0x7fcb3806e2f0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fcb40006fd0 tx=0x7fcb40008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:35.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.566+0000 7fcb4a7fc640 1 -- 172.21.15.32:0/2920583654 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcb4c0cd820 con 0x7fcb4c0b5af0 2024-04-24T23:05:35.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:35.791+0000 7fcb53ea5640 1 -- 172.21.15.32:0/2920583654 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fcb4c067730 con 0x7fcb4c0b5af0 2024-04-24T23:05:36.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.443+0000 7fcb4a7fc640 1 -- 172.21.15.32:0/2920583654 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v58) ==== 122+0+0 (secure 0 0 0) 0x7fcb44017020 con 0x7fcb4c0b5af0 2024-04-24T23:05:36.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.446+0000 7fcb53ea5640 1 -- 172.21.15.32:0/2920583654 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcb3806be30 msgr2=0x7fcb3806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:36.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.446+0000 7fcb53ea5640 1 --2- 172.21.15.32:0/2920583654 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcb3806be30 0x7fcb3806e2f0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fcb40006fd0 tx=0x7fcb40008040 comp rx=0 tx=0).stop 2024-04-24T23:05:36.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.446+0000 7fcb53ea5640 1 -- 172.21.15.32:0/2920583654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb4c0b5af0 msgr2=0x7fcb4c1f7c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:36.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.446+0000 7fcb53ea5640 1 --2- 172.21.15.32:0/2920583654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb4c0b5af0 0x7fcb4c1f7c60 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fcb4400e7e0 tx=0x7fcb4400eca0 comp rx=0 tx=0).stop 2024-04-24T23:05:36.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.446+0000 7fcb53ea5640 1 -- 172.21.15.32:0/2920583654 shutdown_connections 2024-04-24T23:05:36.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.446+0000 7fcb53ea5640 1 --2- 172.21.15.32:0/2920583654 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcb3806be30 0x7fcb3806e2f0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:36.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.446+0000 7fcb53ea5640 1 --2- 172.21.15.32:0/2920583654 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcb4c162c40 0x7fcb4c06e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:36.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.446+0000 7fcb53ea5640 1 --2- 172.21.15.32:0/2920583654 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcb4c162320 0x7fcb4c06dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:36.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.446+0000 7fcb53ea5640 1 --2- 172.21.15.32:0/2920583654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb4c0b5af0 0x7fcb4c1f7c60 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:36.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.446+0000 7fcb53ea5640 1 -- 172.21.15.32:0/2920583654 >> 172.21.15.32:0/2920583654 conn(0x7fcb4c0d0ae0 msgr2=0x7fcb4c0af740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:36.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.446+0000 7fcb53ea5640 1 -- 172.21.15.32:0/2920583654 shutdown_connections 2024-04-24T23:05:36.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.446+0000 7fcb53ea5640 1 -- 172.21.15.32:0/2920583654 wait complete. 2024-04-24T23:05:36.461 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-24T23:05:36.461 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:05:36.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.567+0000 7f1841e38640 1 Processor -- start 2024-04-24T23:05:36.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.567+0000 7f1841e38640 1 -- start start 2024-04-24T23:05:36.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.568+0000 7f1841e38640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f183c15b380 0x7f183c06c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:36.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.568+0000 7f1841e38640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f183c0ceb40 0x7f183c06cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:36.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.568+0000 7f1841e38640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f183c0cef50 con 0x7f183c15cb00 2024-04-24T23:05:36.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.568+0000 7f1841e38640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f183c0b4520 con 0x7f183c15b380 2024-04-24T23:05:36.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.568+0000 7f1841e38640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f183c0b4800 con 0x7f183c0ceb40 2024-04-24T23:05:36.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.568+0000 7f183b7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f183c15b380 0x7f183c06c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:36.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.568+0000 7f183b7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f183c15b380 0x7f183c06c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56324/0 (socket says 172.21.15.32:56324) 2024-04-24T23:05:36.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.568+0000 7f183b7fe640 1 -- 172.21.15.32:0/2071796815 learned_addr learned my addr 172.21.15.32:0/2071796815 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:36.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.568+0000 7f183bfff640 1 --2- 172.21.15.32:0/2071796815 >> v2:172.21.15.121:3300/0 conn(0x7f183c0ceb40 0x7f183c06cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:36.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.568+0000 7f183b7fe640 1 -- 172.21.15.32:0/2071796815 >> v2:172.21.15.121:3300/0 conn(0x7f183c0ceb40 msgr2=0x7f183c06cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:36.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.568+0000 7f183b7fe640 1 --2- 172.21.15.32:0/2071796815 >> v2:172.21.15.121:3300/0 conn(0x7f183c0ceb40 0x7f183c06cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:36.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.568+0000 7f183b7fe640 1 -- 172.21.15.32:0/2071796815 >> v1:172.21.15.121:6789/0 conn(0x7f183c15cb00 legacy=0x7f183c0ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:36.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.568+0000 7f183b7fe640 1 -- 172.21.15.32:0/2071796815 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f183c0b4ae0 con 0x7f183c15b380 2024-04-24T23:05:36.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.569+0000 7f183b7fe640 1 --2- 172.21.15.32:0/2071796815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f183c15b380 0x7f183c06c5f0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f183c070490 tx=0x7f182c02c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=720a35b1306ed562 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:36.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.569+0000 7f183a7fc640 1 -- 172.21.15.32:0/2071796815 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f182c031710 con 0x7f183c15b380 2024-04-24T23:05:36.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.569+0000 7f183a7fc640 1 -- 172.21.15.32:0/2071796815 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f182c0318b0 con 0x7f183c15b380 2024-04-24T23:05:36.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.569+0000 7f183a7fc640 1 -- 172.21.15.32:0/2071796815 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f182c031c80 con 0x7f183c15b380 2024-04-24T23:05:36.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.570+0000 7f1841e38640 1 -- 172.21.15.32:0/2071796815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f183c15b380 msgr2=0x7f183c06c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:36.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.570+0000 7f1841e38640 1 --2- 172.21.15.32:0/2071796815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f183c15b380 0x7f183c06c5f0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f183c070490 tx=0x7f182c02c7d0 comp rx=0 tx=0).stop 2024-04-24T23:05:36.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.570+0000 7f1841e38640 1 -- 172.21.15.32:0/2071796815 shutdown_connections 2024-04-24T23:05:36.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.570+0000 7f1841e38640 1 --2- 172.21.15.32:0/2071796815 >> v2:172.21.15.121:3300/0 conn(0x7f183c0ceb40 0x7f183c06cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:36.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.570+0000 7f1841e38640 1 --2- 172.21.15.32:0/2071796815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f183c15b380 0x7f183c06c5f0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:36.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.570+0000 7f1841e38640 1 -- 172.21.15.32:0/2071796815 >> 172.21.15.32:0/2071796815 conn(0x7f183c06b110 msgr2=0x7f183c06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:36.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.570+0000 7f1841e38640 1 -- 172.21.15.32:0/2071796815 shutdown_connections 2024-04-24T23:05:36.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.570+0000 7f1841e38640 1 -- 172.21.15.32:0/2071796815 wait complete. 2024-04-24T23:05:36.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.571+0000 7f1841e38640 1 Processor -- start 2024-04-24T23:05:36.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.571+0000 7f1841e38640 1 -- start start 2024-04-24T23:05:36.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.572+0000 7f1841e38640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f183c0ceb40 0x7f183c072100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:36.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.572+0000 7f1841e38640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f183c15b380 0x7f183c144db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:36.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.572+0000 7f1841e38640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f183c15cb00 0x7f183c070680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:36.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.572+0000 7f1841e38640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f183c15b760 con 0x7f183c15cb00 2024-04-24T23:05:36.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.572+0000 7f1841e38640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f183c0b50a0 con 0x7f183c15b380 2024-04-24T23:05:36.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.572+0000 7f1841e38640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f183c0b55c0 con 0x7f183c0ceb40 2024-04-24T23:05:36.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.572+0000 7f183bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f183c15cb00 0x7f183c070680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:36.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.572+0000 7f183bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f183c15cb00 0x7f183c070680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56328/0 (socket says 172.21.15.32:56328) 2024-04-24T23:05:36.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.572+0000 7f183bfff640 1 -- 172.21.15.32:0/734963700 learned_addr learned my addr 172.21.15.32:0/734963700 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:36.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.572+0000 7f183affd640 1 --2- 172.21.15.32:0/734963700 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f183c15b380 0x7f183c144db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:36.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.572+0000 7f183b7fe640 1 --2- 172.21.15.32:0/734963700 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f183c0ceb40 0x7f183c072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:36.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.572+0000 7f183bfff640 1 -- 172.21.15.32:0/734963700 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f183c0ceb40 msgr2=0x7f183c072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:36.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.572+0000 7f183bfff640 1 --2- 172.21.15.32:0/734963700 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f183c0ceb40 0x7f183c072100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:36.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.572+0000 7f183bfff640 1 -- 172.21.15.32:0/734963700 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f183c15b380 msgr2=0x7f183c144db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:36.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.572+0000 7f183bfff640 1 --2- 172.21.15.32:0/734963700 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f183c15b380 0x7f183c144db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:36.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.572+0000 7f183bfff640 1 -- 172.21.15.32:0/734963700 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f183c1452f0 con 0x7f183c15cb00 2024-04-24T23:05:36.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.573+0000 7f183bfff640 1 --2- 172.21.15.32:0/734963700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f183c15cb00 0x7f183c070680 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f1834007e40 tx=0x7f183400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:36.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.573+0000 7f1823fff640 1 -- 172.21.15.32:0/734963700 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1834014620 con 0x7f183c15cb00 2024-04-24T23:05:36.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.573+0000 7f1823fff640 1 -- 172.21.15.32:0/734963700 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1834019070 con 0x7f183c15cb00 2024-04-24T23:05:36.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.573+0000 7f1841e38640 1 -- 172.21.15.32:0/734963700 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f183c071060 con 0x7f183c15cb00 2024-04-24T23:05:36.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.573+0000 7f1823fff640 1 -- 172.21.15.32:0/734963700 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1834014a20 con 0x7f183c15cb00 2024-04-24T23:05:36.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.573+0000 7f1841e38640 1 -- 172.21.15.32:0/734963700 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f183c071900 con 0x7f183c15cb00 2024-04-24T23:05:36.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.574+0000 7f1841e38640 1 -- 172.21.15.32:0/734963700 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f183c068430 con 0x7f183c15cb00 2024-04-24T23:05:36.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.574+0000 7f1823fff640 1 -- 172.21.15.32:0/734963700 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1834014bc0 con 0x7f183c15cb00 2024-04-24T23:05:36.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.575+0000 7f1823fff640 1 --2- 172.21.15.32:0/734963700 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f182806be30 0x7f182806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:36.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.575+0000 7f1823fff640 1 -- 172.21.15.32:0/734963700 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5709+0+0 (secure 0 0 0) 0x7f183408b810 con 0x7f183c15cb00 2024-04-24T23:05:36.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.575+0000 7f183b7fe640 1 --2- 172.21.15.32:0/734963700 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f182806be30 0x7f182806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:36.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.576+0000 7f183b7fe640 1 --2- 172.21.15.32:0/734963700 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f182806be30 0x7f182806e2f0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f182c036000 tx=0x7f182c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:36.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.579+0000 7f1823fff640 1 -- 172.21.15.32:0/734963700 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f183405a120 con 0x7f183c15cb00 2024-04-24T23:05:36.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.802+0000 7f1841e38640 1 -- 172.21.15.32:0/734963700 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f183c0afd60 con 0x7f183c15cb00 2024-04-24T23:05:36.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.802+0000 7f1823fff640 1 -- 172.21.15.32:0/734963700 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+5189 (secure 0 0 0) 0x7f1834059a20 con 0x7f183c15cb00 2024-04-24T23:05:36.804 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:05:36.804 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":58,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3361752496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3361752496},{"type":"v1","addr":"172.21.15.121:6837","nonce":3361752496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":6799,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3813453544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3813453544},{"type":"v1","addr":"172.21.15.121:6836","nonce":3813453544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7779,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/1352085651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1352085651},{"type":"v1","addr":"172.21.15.32:6833","nonce":1352085651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:05:33.791068+0000","modified":"2024-04-24T23:05:35.792946+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7794},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7794":{"gid":7794,"name":"a","rank":0,"incarnation":56,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/4206823204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":4206823204},{"type":"v1","addr":"172.21.15.32:6835","nonce":4206823204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7794,"qdb_cluster":[7794]},"id":8}]} 2024-04-24T23:05:36.804 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 58 2024-04-24T23:05:36.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.804+0000 7f1841e38640 1 -- 172.21.15.32:0/734963700 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f182806be30 msgr2=0x7f182806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:36.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.804+0000 7f1841e38640 1 --2- 172.21.15.32:0/734963700 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f182806be30 0x7f182806e2f0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f182c036000 tx=0x7f182c008040 comp rx=0 tx=0).stop 2024-04-24T23:05:36.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.804+0000 7f1841e38640 1 -- 172.21.15.32:0/734963700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f183c15cb00 msgr2=0x7f183c070680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:36.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.804+0000 7f1841e38640 1 --2- 172.21.15.32:0/734963700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f183c15cb00 0x7f183c070680 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f1834007e40 tx=0x7f183400e4b0 comp rx=0 tx=0).stop 2024-04-24T23:05:36.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.804+0000 7f1841e38640 1 -- 172.21.15.32:0/734963700 shutdown_connections 2024-04-24T23:05:36.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.804+0000 7f1841e38640 1 --2- 172.21.15.32:0/734963700 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f182806be30 0x7f182806e2f0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:36.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.804+0000 7f1841e38640 1 --2- 172.21.15.32:0/734963700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f183c15cb00 0x7f183c070680 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:36.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.804+0000 7f1841e38640 1 --2- 172.21.15.32:0/734963700 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f183c15b380 0x7f183c144db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:36.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.804+0000 7f1841e38640 1 --2- 172.21.15.32:0/734963700 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f183c0ceb40 0x7f183c072100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:36.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.804+0000 7f1841e38640 1 -- 172.21.15.32:0/734963700 >> 172.21.15.32:0/734963700 conn(0x7f183c06b110 msgr2=0x7f183c0b3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:36.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.804+0000 7f1841e38640 1 -- 172.21.15.32:0/734963700 shutdown_connections 2024-04-24T23:05:36.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.804+0000 7f1841e38640 1 -- 172.21.15.32:0/734963700 wait complete. 2024-04-24T23:05:36.817 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:05:36.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.916+0000 7fd296c9e640 1 Processor -- start 2024-04-24T23:05:36.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.916+0000 7fd296c9e640 1 -- start start 2024-04-24T23:05:36.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.917+0000 7fd296c9e640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd29015fd60 0x7fd29006c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:36.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.917+0000 7fd296c9e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd29006cb30 0x7fd2900cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:36.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.917+0000 7fd296c9e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2900b46c0 con 0x7fd29015b480 2024-04-24T23:05:36.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.917+0000 7fd296c9e640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fd2900b4ff0 con 0x7fd29015fd60 2024-04-24T23:05:36.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.917+0000 7fd296c9e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd2900b49a0 con 0x7fd29006cb30 2024-04-24T23:05:36.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.917+0000 7fd294a3a640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd29015b480 0x7fd2900b5c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37818/0 (socket says 172.21.15.32:37818) 2024-04-24T23:05:36.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.917+0000 7fd294a3a640 1 -- 172.21.15.32:0/1976710491 learned_addr learned my addr 172.21.15.32:0/1976710491 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:05:36.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.917+0000 7fd29523b640 1 --2- 172.21.15.32:0/1976710491 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd29006cb30 0x7fd2900cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:36.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.917+0000 7fd28ffff640 1 --2- 172.21.15.32:0/1976710491 >> v2:172.21.15.121:3300/0 conn(0x7fd29015fd60 0x7fd29006c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:36.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.917+0000 7fd28f7fe640 1 -- 172.21.15.32:0/1976710491 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1060515916 0 0) 0x7fd2900b46c0 con 0x7fd29015b480 2024-04-24T23:05:36.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.917+0000 7fd28f7fe640 1 -- 172.21.15.32:0/1976710491 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd278003520 con 0x7fd29015b480 2024-04-24T23:05:36.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.918+0000 7fd28f7fe640 1 -- 172.21.15.32:0/1976710491 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1549580819 0 0) 0x7fd278003520 con 0x7fd29015b480 2024-04-24T23:05:36.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.918+0000 7fd28f7fe640 1 -- 172.21.15.32:0/1976710491 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd29006cb30 msgr2=0x7fd2900cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:36.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.918+0000 7fd28f7fe640 1 --2- 172.21.15.32:0/1976710491 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd29006cb30 0x7fd2900cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:36.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.918+0000 7fd28f7fe640 1 -- 172.21.15.32:0/1976710491 >> v2:172.21.15.121:3300/0 conn(0x7fd29015fd60 msgr2=0x7fd29006c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:36.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.918+0000 7fd28f7fe640 1 --2- 172.21.15.32:0/1976710491 >> v2:172.21.15.121:3300/0 conn(0x7fd29015fd60 0x7fd29006c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:36.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.918+0000 7fd28f7fe640 1 -- 172.21.15.32:0/1976710491 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2900b4c80 con 0x7fd29015b480 2024-04-24T23:05:36.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.918+0000 7fd28f7fe640 1 -- 172.21.15.32:0/1976710491 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7fd280001670 con 0x7fd29015b480 2024-04-24T23:05:36.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.918+0000 7fd28f7fe640 1 -- 172.21.15.32:0/1976710491 >> v1:172.21.15.32:6789/0 conn(0x7fd29015b480 legacy=0x7fd2900b5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:36.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.918+0000 7fd28f7fe640 1 --2- 172.21.15.32:0/1976710491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd2780039b0 0x7fd278003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:36.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.918+0000 7fd28f7fe640 1 --2- 172.21.15.32:0/1976710491 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd29006cb30 0x7fd278004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:36.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.918+0000 7fd28f7fe640 1 --2- 172.21.15.32:0/1976710491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd278004d40 0x7fd2780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:36.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.918+0000 7fd28f7fe640 1 -- 172.21.15.32:0/1976710491 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd2900b4ff0 con 0x7fd278004d40 2024-04-24T23:05:36.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.918+0000 7fd28f7fe640 1 -- 172.21.15.32:0/1976710491 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd2900b49a0 con 0x7fd2780039b0 2024-04-24T23:05:36.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.918+0000 7fd28f7fe640 1 -- 172.21.15.32:0/1976710491 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd27800b9c0 con 0x7fd29006cb30 2024-04-24T23:05:36.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.918+0000 7fd28ffff640 1 --2- 172.21.15.32:0/1976710491 >> v2:172.21.15.121:3300/0 conn(0x7fd29015fd60 0x7fd29006c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:05:36.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.918+0000 7fd29523b640 1 --2- 172.21.15.32:0/1976710491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd278004d40 0x7fd2780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:36.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.918+0000 7fd294a3a640 1 --2- 172.21.15.32:0/1976710491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd2780039b0 0x7fd278003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:36.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.918+0000 7fd28ffff640 1 --2- 172.21.15.32:0/1976710491 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd29006cb30 0x7fd278004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:36.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.918+0000 7fd29523b640 1 -- 172.21.15.32:0/1976710491 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd29006cb30 msgr2=0x7fd278004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:36.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.918+0000 7fd29523b640 1 --2- 172.21.15.32:0/1976710491 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd29006cb30 0x7fd278004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:36.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.918+0000 7fd29523b640 1 -- 172.21.15.32:0/1976710491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd2780039b0 msgr2=0x7fd278003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:36.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.918+0000 7fd29523b640 1 --2- 172.21.15.32:0/1976710491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd2780039b0 0x7fd278003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:36.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.918+0000 7fd29523b640 1 -- 172.21.15.32:0/1976710491 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd278003520 con 0x7fd278004d40 2024-04-24T23:05:36.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.918+0000 7fd28ffff640 1 --2- 172.21.15.32:0/1976710491 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd29006cb30 0x7fd278004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:05:36.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.919+0000 7fd29523b640 1 --2- 172.21.15.32:0/1976710491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd278004d40 0x7fd2780051c0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fd288009fd0 tx=0x7fd278005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15a46a90640f171f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:36.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.919+0000 7fd28f7fe640 1 -- 172.21.15.32:0/1976710491 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd288009bf0 con 0x7fd278004d40 2024-04-24T23:05:36.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.919+0000 7fd28f7fe640 1 -- 172.21.15.32:0/1976710491 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd288036070 con 0x7fd278004d40 2024-04-24T23:05:36.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.919+0000 7fd28f7fe640 1 -- 172.21.15.32:0/1976710491 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd288030880 con 0x7fd278004d40 2024-04-24T23:05:36.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.919+0000 7fd296c9e640 1 -- 172.21.15.32:0/1976710491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd278004d40 msgr2=0x7fd2780051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:36.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.919+0000 7fd296c9e640 1 --2- 172.21.15.32:0/1976710491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd278004d40 0x7fd2780051c0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fd288009fd0 tx=0x7fd278005830 comp rx=0 tx=0).stop 2024-04-24T23:05:36.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.919+0000 7fd29523b640 1 -- 172.21.15.32:0/1976710491 reap_dead start 2024-04-24T23:05:36.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.920+0000 7fd296c9e640 1 -- 172.21.15.32:0/1976710491 shutdown_connections 2024-04-24T23:05:36.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.920+0000 7fd296c9e640 1 -- 172.21.15.32:0/1976710491 >> 172.21.15.32:0/1976710491 conn(0x7fd29006b110 msgr2=0x7fd29006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:36.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.920+0000 7fd296c9e640 1 -- 172.21.15.32:0/1976710491 shutdown_connections 2024-04-24T23:05:36.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.920+0000 7fd296c9e640 1 -- 172.21.15.32:0/1976710491 wait complete. 2024-04-24T23:05:36.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.921+0000 7fd296c9e640 1 Processor -- start 2024-04-24T23:05:36.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.921+0000 7fd296c9e640 1 -- start start 2024-04-24T23:05:36.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.922+0000 7fd296c9e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd278004d40 0x7fd290138f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:36.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.922+0000 7fd296c9e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd2901387c0 0x7fd29014de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:36.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.922+0000 7fd296c9e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd29014e350 0x7fd290150170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:36.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.922+0000 7fd296c9e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd29015b860 con 0x7fd29014e350 2024-04-24T23:05:36.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.922+0000 7fd296c9e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd2900cdf80 con 0x7fd278004d40 2024-04-24T23:05:36.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.922+0000 7fd296c9e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd2901511a0 con 0x7fd2901387c0 2024-04-24T23:05:36.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.922+0000 7fd29523b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd29014e350 0x7fd290150170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:36.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.922+0000 7fd29523b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd29014e350 0x7fd290150170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56344/0 (socket says 172.21.15.32:56344) 2024-04-24T23:05:36.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.922+0000 7fd29523b640 1 -- 172.21.15.32:0/2981111286 learned_addr learned my addr 172.21.15.32:0/2981111286 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:36.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.922+0000 7fd294a3a640 1 --2- 172.21.15.32:0/2981111286 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd278004d40 0x7fd290138f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:36.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.922+0000 7fd28ffff640 1 --2- 172.21.15.32:0/2981111286 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd2901387c0 0x7fd29014de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:36.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.922+0000 7fd29523b640 1 -- 172.21.15.32:0/2981111286 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd2901387c0 msgr2=0x7fd29014de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:36.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.922+0000 7fd29523b640 1 --2- 172.21.15.32:0/2981111286 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd2901387c0 0x7fd29014de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:36.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.922+0000 7fd29523b640 1 -- 172.21.15.32:0/2981111286 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd278004d40 msgr2=0x7fd290138f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:36.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.922+0000 7fd29523b640 1 --2- 172.21.15.32:0/2981111286 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd278004d40 0x7fd290138f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:36.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.922+0000 7fd29523b640 1 -- 172.21.15.32:0/2981111286 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd290138c40 con 0x7fd29014e350 2024-04-24T23:05:36.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.922+0000 7fd29523b640 1 --2- 172.21.15.32:0/2981111286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd29014e350 0x7fd290150170 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fd28800e230 tx=0x7fd288009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:36.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.922+0000 7fd28d7fa640 1 -- 172.21.15.32:0/2981111286 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd288009bf0 con 0x7fd29014e350 2024-04-24T23:05:36.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.922+0000 7fd28d7fa640 1 -- 172.21.15.32:0/2981111286 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd288031d30 con 0x7fd29014e350 2024-04-24T23:05:36.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.922+0000 7fd296c9e640 1 -- 172.21.15.32:0/2981111286 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd290150ec0 con 0x7fd29014e350 2024-04-24T23:05:36.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.922+0000 7fd28d7fa640 1 -- 172.21.15.32:0/2981111286 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd288045670 con 0x7fd29014e350 2024-04-24T23:05:36.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.923+0000 7fd296c9e640 1 -- 172.21.15.32:0/2981111286 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd29015d040 con 0x7fd29014e350 2024-04-24T23:05:36.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.923+0000 7fd296c9e640 1 -- 172.21.15.32:0/2981111286 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd290068430 con 0x7fd29014e350 2024-04-24T23:05:36.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.924+0000 7fd28d7fa640 1 -- 172.21.15.32:0/2981111286 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd288005d10 con 0x7fd29014e350 2024-04-24T23:05:36.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.924+0000 7fd28d7fa640 1 --2- 172.21.15.32:0/2981111286 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd27c06be60 0x7fd27c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:36.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.924+0000 7fd28d7fa640 1 -- 172.21.15.32:0/2981111286 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5709+0+0 (secure 0 0 0) 0x7fd2880abd90 con 0x7fd29014e350 2024-04-24T23:05:36.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.925+0000 7fd294a3a640 1 --2- 172.21.15.32:0/2981111286 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd27c06be60 0x7fd27c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:36.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.925+0000 7fd294a3a640 1 --2- 172.21.15.32:0/2981111286 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd27c06be60 0x7fd27c06e320 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fd280007fd0 tx=0x7fd280009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:36.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:36.929+0000 7fd28d7fa640 1 -- 172.21.15.32:0/2981111286 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd28807a860 con 0x7fd29014e350 2024-04-24T23:05:37.039 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:05:37.041 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:05:37.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.115+0000 7fd296c9e640 1 -- 172.21.15.32:0/2981111286 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd29006c000 con 0x7fd29014e350 2024-04-24T23:05:37.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.117+0000 7fd28d7fa640 1 -- 172.21.15.32:0/2981111286 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) ==== 74+0+16696 (secure 0 0 0) 0x7fd28807a160 con 0x7fd29014e350 2024-04-24T23:05:37.118 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:05:37.118 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":140,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:05:35.419032+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:05:32.064234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-24T23:05:32.760777+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:05:37.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.118+0000 7fd296c9e640 1 -- 172.21.15.32:0/2981111286 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd27c06be60 msgr2=0x7fd27c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:37.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.118+0000 7fd296c9e640 1 --2- 172.21.15.32:0/2981111286 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd27c06be60 0x7fd27c06e320 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fd280007fd0 tx=0x7fd280009040 comp rx=0 tx=0).stop 2024-04-24T23:05:37.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.118+0000 7fd296c9e640 1 -- 172.21.15.32:0/2981111286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd29014e350 msgr2=0x7fd290150170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:37.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.118+0000 7fd296c9e640 1 --2- 172.21.15.32:0/2981111286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd29014e350 0x7fd290150170 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fd28800e230 tx=0x7fd288009750 comp rx=0 tx=0).stop 2024-04-24T23:05:37.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.119+0000 7fd296c9e640 1 -- 172.21.15.32:0/2981111286 shutdown_connections 2024-04-24T23:05:37.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.119+0000 7fd296c9e640 1 --2- 172.21.15.32:0/2981111286 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd27c06be60 0x7fd27c06e320 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:37.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.119+0000 7fd296c9e640 1 --2- 172.21.15.32:0/2981111286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd29014e350 0x7fd290150170 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:37.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.119+0000 7fd296c9e640 1 --2- 172.21.15.32:0/2981111286 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd2901387c0 0x7fd29014de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:37.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.119+0000 7fd296c9e640 1 --2- 172.21.15.32:0/2981111286 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd278004d40 0x7fd290138f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:37.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.119+0000 7fd296c9e640 1 -- 172.21.15.32:0/2981111286 >> 172.21.15.32:0/2981111286 conn(0x7fd29006b110 msgr2=0x7fd29006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:37.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.119+0000 7fd296c9e640 1 -- 172.21.15.32:0/2981111286 shutdown_connections 2024-04-24T23:05:37.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.119+0000 7fd296c9e640 1 -- 172.21.15.32:0/2981111286 wait complete. 2024-04-24T23:05:37.132 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-24T23:05:37.132 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:05:37.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.271+0000 7ffb068da640 1 Processor -- start 2024-04-24T23:05:37.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.271+0000 7ffb068da640 1 -- start start 2024-04-24T23:05:37.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.272+0000 7ffb068da640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffb0006c5f0 0x7ffb0006c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:37.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.272+0000 7ffb068da640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb000b44d0 0x7ffb000b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:37.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.272+0000 7ffb068da640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ffb000af890 0x7ffb000cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:37.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.272+0000 7ffb068da640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7ffb000b6430 con 0x7ffb0006c5f0 2024-04-24T23:05:37.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.272+0000 7ffb068da640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7ffb00156f80 con 0x7ffb000af890 2024-04-24T23:05:37.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.272+0000 7ffb068da640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ffb000b6150 con 0x7ffb000b44d0 2024-04-24T23:05:37.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.272+0000 7ffafffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffb0006c5f0 0x7ffb0006c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:37.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.272+0000 7ffafffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffb0006c5f0 0x7ffb0006c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56360/0 (socket says 172.21.15.32:56360) 2024-04-24T23:05:37.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.272+0000 7ffafffff640 1 -- 172.21.15.32:0/2155267209 learned_addr learned my addr 172.21.15.32:0/2155267209 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:37.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.272+0000 7ffb04e77640 1 --2- 172.21.15.32:0/2155267209 >> v2:172.21.15.121:3300/0 conn(0x7ffb000af890 0x7ffb000cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:37.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.272+0000 7ffaff7fe640 1 --2- 172.21.15.32:0/2155267209 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb000b44d0 0x7ffb000b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:37.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.272+0000 7ffafffff640 1 -- 172.21.15.32:0/2155267209 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb000b44d0 msgr2=0x7ffb000b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:37.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.272+0000 7ffafffff640 1 --2- 172.21.15.32:0/2155267209 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb000b44d0 0x7ffb000b5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:37.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.272+0000 7ffafffff640 1 -- 172.21.15.32:0/2155267209 >> v2:172.21.15.121:3300/0 conn(0x7ffb000af890 msgr2=0x7ffb000cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:37.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.272+0000 7ffafffff640 1 --2- 172.21.15.32:0/2155267209 >> v2:172.21.15.121:3300/0 conn(0x7ffb000af890 0x7ffb000cef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:37.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.272+0000 7ffafffff640 1 -- 172.21.15.32:0/2155267209 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb000b39f0 con 0x7ffb0006c5f0 2024-04-24T23:05:37.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.272+0000 7ffaff7fe640 1 --2- 172.21.15.32:0/2155267209 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb000b44d0 0x7ffb000b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:05:37.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.273+0000 7ffafffff640 1 --2- 172.21.15.32:0/2155267209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb0006c5f0 0x7ffb0006c9d0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7ffb000aff30 tx=0x7ffaf002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=407f59c24fbf80d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:37.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.273+0000 7ffafeffd640 1 -- 172.21.15.32:0/2155267209 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf0037070 con 0x7ffb0006c5f0 2024-04-24T23:05:37.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.273+0000 7ffafeffd640 1 -- 172.21.15.32:0/2155267209 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffaf0032440 con 0x7ffb0006c5f0 2024-04-24T23:05:37.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.273+0000 7ffafeffd640 1 -- 172.21.15.32:0/2155267209 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf0032810 con 0x7ffb0006c5f0 2024-04-24T23:05:37.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.273+0000 7ffb068da640 1 -- 172.21.15.32:0/2155267209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb0006c5f0 msgr2=0x7ffb0006c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:37.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.273+0000 7ffb068da640 1 --2- 172.21.15.32:0/2155267209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb0006c5f0 0x7ffb0006c9d0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7ffb000aff30 tx=0x7ffaf002a140 comp rx=0 tx=0).stop 2024-04-24T23:05:37.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.273+0000 7ffb068da640 1 -- 172.21.15.32:0/2155267209 shutdown_connections 2024-04-24T23:05:37.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.273+0000 7ffb068da640 1 --2- 172.21.15.32:0/2155267209 >> v2:172.21.15.121:3300/0 conn(0x7ffb000af890 0x7ffb000cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:37.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.273+0000 7ffb068da640 1 --2- 172.21.15.32:0/2155267209 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb000b44d0 0x7ffb000b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:37.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.273+0000 7ffb068da640 1 --2- 172.21.15.32:0/2155267209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb0006c5f0 0x7ffb0006c9d0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:37.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.274+0000 7ffb068da640 1 -- 172.21.15.32:0/2155267209 >> 172.21.15.32:0/2155267209 conn(0x7ffb0006b110 msgr2=0x7ffb0006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:37.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.274+0000 7ffb068da640 1 -- 172.21.15.32:0/2155267209 shutdown_connections 2024-04-24T23:05:37.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.274+0000 7ffb068da640 1 -- 172.21.15.32:0/2155267209 wait complete. 2024-04-24T23:05:37.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.275+0000 7ffb068da640 1 Processor -- start 2024-04-24T23:05:37.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.275+0000 7ffb068da640 1 -- start start 2024-04-24T23:05:37.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.276+0000 7ffb068da640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb0006c5f0 0x7ffb001598a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:37.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.276+0000 7ffb068da640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb000af890 0x7ffb0015cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:37.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.276+0000 7ffb068da640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb000b44d0 0x7ffb00155a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:37.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.276+0000 7ffb068da640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ffb0006cf10 con 0x7ffb000b44d0 2024-04-24T23:05:37.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.276+0000 7ffb068da640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ffb000b3fb0 con 0x7ffb0006c5f0 2024-04-24T23:05:37.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.276+0000 7ffb068da640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ffb000b5dd0 con 0x7ffb000af890 2024-04-24T23:05:37.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.276+0000 7ffb04e77640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb000b44d0 0x7ffb00155a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:37.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.276+0000 7ffb04e77640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb000b44d0 0x7ffb00155a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56366/0 (socket says 172.21.15.32:56366) 2024-04-24T23:05:37.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.276+0000 7ffb04e77640 1 -- 172.21.15.32:0/1156967958 learned_addr learned my addr 172.21.15.32:0/1156967958 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:37.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.276+0000 7ffafffff640 1 --2- 172.21.15.32:0/1156967958 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb0006c5f0 0x7ffb001598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:37.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.276+0000 7ffaff7fe640 1 --2- 172.21.15.32:0/1156967958 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb000af890 0x7ffb0015cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:37.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.276+0000 7ffb04e77640 1 -- 172.21.15.32:0/1156967958 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb000af890 msgr2=0x7ffb0015cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:37.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.276+0000 7ffb04e77640 1 --2- 172.21.15.32:0/1156967958 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb000af890 0x7ffb0015cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:37.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.276+0000 7ffb04e77640 1 -- 172.21.15.32:0/1156967958 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb0006c5f0 msgr2=0x7ffb001598a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:37.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.276+0000 7ffb04e77640 1 --2- 172.21.15.32:0/1156967958 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb0006c5f0 0x7ffb001598a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:37.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.276+0000 7ffb04e77640 1 -- 172.21.15.32:0/1156967958 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb001e4fa0 con 0x7ffb000b44d0 2024-04-24T23:05:37.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.276+0000 7ffb04e77640 1 --2- 172.21.15.32:0/1156967958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb000b44d0 0x7ffb00155a20 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8007e60 tx=0x7ffaf800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:37.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.276+0000 7ffafcff9640 1 -- 172.21.15.32:0/1156967958 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf8014420 con 0x7ffb000b44d0 2024-04-24T23:05:37.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.276+0000 7ffafcff9640 1 -- 172.21.15.32:0/1156967958 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffaf8019070 con 0x7ffb000b44d0 2024-04-24T23:05:37.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.276+0000 7ffb068da640 1 -- 172.21.15.32:0/1156967958 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb001e5280 con 0x7ffb000b44d0 2024-04-24T23:05:37.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.276+0000 7ffafcff9640 1 -- 172.21.15.32:0/1156967958 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf8014820 con 0x7ffb000b44d0 2024-04-24T23:05:37.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.277+0000 7ffb068da640 1 -- 172.21.15.32:0/1156967958 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb001e5840 con 0x7ffb000b44d0 2024-04-24T23:05:37.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.278+0000 7ffb068da640 1 -- 172.21.15.32:0/1156967958 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb00068430 con 0x7ffb000b44d0 2024-04-24T23:05:37.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.278+0000 7ffafcff9640 1 -- 172.21.15.32:0/1156967958 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffaf80149c0 con 0x7ffb000b44d0 2024-04-24T23:05:37.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.278+0000 7ffafcff9640 1 --2- 172.21.15.32:0/1156967958 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ffaec06be30 0x7ffaec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:37.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.278+0000 7ffafcff9640 1 -- 172.21.15.32:0/1156967958 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5709+0+0 (secure 0 0 0) 0x7ffaf808b4a0 con 0x7ffb000b44d0 2024-04-24T23:05:37.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.278+0000 7ffafffff640 1 --2- 172.21.15.32:0/1156967958 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ffaec06be30 0x7ffaec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:37.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.279+0000 7ffafffff640 1 --2- 172.21.15.32:0/1156967958 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ffaec06be30 0x7ffaec06e2f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0034000 tx=0x7ffaf002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:37.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.282+0000 7ffafcff9640 1 -- 172.21.15.32:0/1156967958 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ffaf8059890 con 0x7ffb000b44d0 2024-04-24T23:05:37.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.461+0000 7ffb068da640 1 -- 172.21.15.32:0/1156967958 --> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ffb000b5670 con 0x7ffaec06be30 2024-04-24T23:05:37.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.468+0000 7ffafcff9640 1 -- 172.21.15.32:0/1156967958 <== mgr.4108 v2:172.21.15.121:6832/1848157943 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247992 (secure 0 0 0) 0x7ffb000b5670 con 0x7ffaec06be30 2024-04-24T23:05:37.472 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:05:37.473 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-24T23:05:37.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.470+0000 7ffb068da640 1 -- 172.21.15.32:0/1156967958 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ffaec06be30 msgr2=0x7ffaec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:37.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.470+0000 7ffb068da640 1 --2- 172.21.15.32:0/1156967958 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ffaec06be30 0x7ffaec06e2f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0034000 tx=0x7ffaf002e000 comp rx=0 tx=0).stop 2024-04-24T23:05:37.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.470+0000 7ffb068da640 1 -- 172.21.15.32:0/1156967958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb000b44d0 msgr2=0x7ffb00155a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:37.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.470+0000 7ffb068da640 1 --2- 172.21.15.32:0/1156967958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb000b44d0 0x7ffb00155a20 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8007e60 tx=0x7ffaf800e4b0 comp rx=0 tx=0).stop 2024-04-24T23:05:37.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.471+0000 7ffb068da640 1 -- 172.21.15.32:0/1156967958 shutdown_connections 2024-04-24T23:05:37.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.471+0000 7ffb068da640 1 --2- 172.21.15.32:0/1156967958 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ffaec06be30 0x7ffaec06e2f0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:37.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.471+0000 7ffb068da640 1 --2- 172.21.15.32:0/1156967958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb000b44d0 0x7ffb00155a20 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:37.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.471+0000 7ffb068da640 1 --2- 172.21.15.32:0/1156967958 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb000af890 0x7ffb0015cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:37.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.471+0000 7ffb068da640 1 --2- 172.21.15.32:0/1156967958 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb0006c5f0 0x7ffb001598a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:37.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.471+0000 7ffb068da640 1 -- 172.21.15.32:0/1156967958 >> 172.21.15.32:0/1156967958 conn(0x7ffb0006b110 msgr2=0x7ffb00071750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:37.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.471+0000 7ffb068da640 1 -- 172.21.15.32:0/1156967958 shutdown_connections 2024-04-24T23:05:37.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.471+0000 7ffb068da640 1 -- 172.21.15.32:0/1156967958 wait complete. 2024-04-24T23:05:37.483 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":334,"stamp":"2024-04-24T23:05:37.218178+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":164,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":7227,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555272,"statfs":{"total":1600000000,"available":1592598616,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.802922"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.422504+0000","last_change":"2024-04-24T23:05:34.435206+0000","last_active":"2024-04-24T23:05:35.422504+0000","last_peered":"2024-04-24T23:05:35.422504+0000","last_clean":"2024-04-24T23:05:35.422504+0000","last_became_active":"2024-04-24T23:05:34.434604+0000","last_became_peered":"2024-04-24T23:05:34.434604+0000","last_unstale":"2024-04-24T23:05:35.422504+0000","last_undegraded":"2024-04-24T23:05:35.422504+0000","last_fullsized":"2024-04-24T23:05:35.422504+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:25:11.877792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.758213+0000","last_change":"2024-04-24T23:05:34.436043+0000","last_active":"2024-04-24T23:05:35.758213+0000","last_peered":"2024-04-24T23:05:35.758213+0000","last_clean":"2024-04-24T23:05:35.758213+0000","last_became_active":"2024-04-24T23:05:34.435657+0000","last_became_peered":"2024-04-24T23:05:34.435657+0000","last_unstale":"2024-04-24T23:05:35.758213+0000","last_undegraded":"2024-04-24T23:05:35.758213+0000","last_fullsized":"2024-04-24T23:05:35.758213+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:10:23.227041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:36.754419+0000","last_change":"2024-04-24T23:05:34.438801+0000","last_active":"2024-04-24T23:05:36.754419+0000","last_peered":"2024-04-24T23:05:36.754419+0000","last_clean":"2024-04-24T23:05:36.754419+0000","last_became_active":"2024-04-24T23:05:34.438543+0000","last_became_peered":"2024-04-24T23:05:34.438543+0000","last_unstale":"2024-04-24T23:05:36.754419+0000","last_undegraded":"2024-04-24T23:05:36.754419+0000","last_fullsized":"2024-04-24T23:05:36.754419+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:14:57.282009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.422399+0000","last_change":"2024-04-24T23:05:34.433100+0000","last_active":"2024-04-24T23:05:35.422399+0000","last_peered":"2024-04-24T23:05:35.422399+0000","last_clean":"2024-04-24T23:05:35.422399+0000","last_became_active":"2024-04-24T23:05:34.432382+0000","last_became_peered":"2024-04-24T23:05:34.432382+0000","last_unstale":"2024-04-24T23:05:35.422399+0000","last_undegraded":"2024-04-24T23:05:35.422399+0000","last_fullsized":"2024-04-24T23:05:35.422399+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:09:12.205245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.542410+0000","last_change":"2024-04-24T23:05:34.439317+0000","last_active":"2024-04-24T23:05:35.542410+0000","last_peered":"2024-04-24T23:05:35.542410+0000","last_clean":"2024-04-24T23:05:35.542410+0000","last_became_active":"2024-04-24T23:05:34.438822+0000","last_became_peered":"2024-04-24T23:05:34.438822+0000","last_unstale":"2024-04-24T23:05:35.542410+0000","last_undegraded":"2024-04-24T23:05:35.542410+0000","last_fullsized":"2024-04-24T23:05:35.542410+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:38:50.873838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.758398+0000","last_change":"2024-04-24T23:05:34.427217+0000","last_active":"2024-04-24T23:05:35.758398+0000","last_peered":"2024-04-24T23:05:35.758398+0000","last_clean":"2024-04-24T23:05:35.758398+0000","last_became_active":"2024-04-24T23:05:34.426956+0000","last_became_peered":"2024-04-24T23:05:34.426956+0000","last_unstale":"2024-04-24T23:05:35.758398+0000","last_undegraded":"2024-04-24T23:05:35.758398+0000","last_fullsized":"2024-04-24T23:05:35.758398+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:05:15.703627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.421519+0000","last_change":"2024-04-24T23:05:34.428003+0000","last_active":"2024-04-24T23:05:35.421519+0000","last_peered":"2024-04-24T23:05:35.421519+0000","last_clean":"2024-04-24T23:05:35.421519+0000","last_became_active":"2024-04-24T23:05:34.427722+0000","last_became_peered":"2024-04-24T23:05:34.427722+0000","last_unstale":"2024-04-24T23:05:35.421519+0000","last_undegraded":"2024-04-24T23:05:35.421519+0000","last_fullsized":"2024-04-24T23:05:35.421519+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:53:20.339099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.421479+0000","last_change":"2024-04-24T23:05:34.429921+0000","last_active":"2024-04-24T23:05:35.421479+0000","last_peered":"2024-04-24T23:05:35.421479+0000","last_clean":"2024-04-24T23:05:35.421479+0000","last_became_active":"2024-04-24T23:05:34.429682+0000","last_became_peered":"2024-04-24T23:05:34.429682+0000","last_unstale":"2024-04-24T23:05:35.421479+0000","last_undegraded":"2024-04-24T23:05:35.421479+0000","last_fullsized":"2024-04-24T23:05:35.421479+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:14:53.789433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.422242+0000","last_change":"2024-04-24T23:05:34.436440+0000","last_active":"2024-04-24T23:05:35.422242+0000","last_peered":"2024-04-24T23:05:35.422242+0000","last_clean":"2024-04-24T23:05:35.422242+0000","last_became_active":"2024-04-24T23:05:34.436191+0000","last_became_peered":"2024-04-24T23:05:34.436191+0000","last_unstale":"2024-04-24T23:05:35.422242+0000","last_undegraded":"2024-04-24T23:05:35.422242+0000","last_fullsized":"2024-04-24T23:05:35.422242+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:35.460052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.542230+0000","last_change":"2024-04-24T23:05:34.434349+0000","last_active":"2024-04-24T23:05:35.542230+0000","last_peered":"2024-04-24T23:05:35.542230+0000","last_clean":"2024-04-24T23:05:35.542230+0000","last_became_active":"2024-04-24T23:05:34.433780+0000","last_became_peered":"2024-04-24T23:05:34.433780+0000","last_unstale":"2024-04-24T23:05:35.542230+0000","last_undegraded":"2024-04-24T23:05:35.542230+0000","last_fullsized":"2024-04-24T23:05:35.542230+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:51.615533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.421588+0000","last_change":"2024-04-24T23:05:34.428833+0000","last_active":"2024-04-24T23:05:35.421588+0000","last_peered":"2024-04-24T23:05:35.421588+0000","last_clean":"2024-04-24T23:05:35.421588+0000","last_became_active":"2024-04-24T23:05:34.428559+0000","last_became_peered":"2024-04-24T23:05:34.428559+0000","last_unstale":"2024-04-24T23:05:35.421588+0000","last_undegraded":"2024-04-24T23:05:35.421588+0000","last_fullsized":"2024-04-24T23:05:35.421588+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:02.530795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.541681+0000","last_change":"2024-04-24T23:05:34.433862+0000","last_active":"2024-04-24T23:05:35.541681+0000","last_peered":"2024-04-24T23:05:35.541681+0000","last_clean":"2024-04-24T23:05:35.541681+0000","last_became_active":"2024-04-24T23:05:34.433627+0000","last_became_peered":"2024-04-24T23:05:34.433627+0000","last_unstale":"2024-04-24T23:05:35.541681+0000","last_undegraded":"2024-04-24T23:05:35.541681+0000","last_fullsized":"2024-04-24T23:05:35.541681+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:12:02.947338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.758309+0000","last_change":"2024-04-24T23:05:34.434265+0000","last_active":"2024-04-24T23:05:35.758309+0000","last_peered":"2024-04-24T23:05:35.758309+0000","last_clean":"2024-04-24T23:05:35.758309+0000","last_became_active":"2024-04-24T23:05:34.434021+0000","last_became_peered":"2024-04-24T23:05:34.434021+0000","last_unstale":"2024-04-24T23:05:35.758309+0000","last_undegraded":"2024-04-24T23:05:35.758309+0000","last_fullsized":"2024-04-24T23:05:35.758309+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:47:35.322597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:36.536713+0000","last_change":"2024-04-24T23:05:34.428147+0000","last_active":"2024-04-24T23:05:36.536713+0000","last_peered":"2024-04-24T23:05:36.536713+0000","last_clean":"2024-04-24T23:05:36.536713+0000","last_became_active":"2024-04-24T23:05:34.427234+0000","last_became_peered":"2024-04-24T23:05:34.427234+0000","last_unstale":"2024-04-24T23:05:36.536713+0000","last_undegraded":"2024-04-24T23:05:36.536713+0000","last_fullsized":"2024-04-24T23:05:36.536713+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:01.491784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.422580+0000","last_change":"2024-04-24T23:05:34.436175+0000","last_active":"2024-04-24T23:05:35.422580+0000","last_peered":"2024-04-24T23:05:35.422580+0000","last_clean":"2024-04-24T23:05:35.422580+0000","last_became_active":"2024-04-24T23:05:34.435595+0000","last_became_peered":"2024-04-24T23:05:34.435595+0000","last_unstale":"2024-04-24T23:05:35.422580+0000","last_undegraded":"2024-04-24T23:05:35.422580+0000","last_fullsized":"2024-04-24T23:05:35.422580+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:58:58.766747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.542062+0000","last_change":"2024-04-24T23:05:34.438708+0000","last_active":"2024-04-24T23:05:35.542062+0000","last_peered":"2024-04-24T23:05:35.542062+0000","last_clean":"2024-04-24T23:05:35.542062+0000","last_became_active":"2024-04-24T23:05:34.438477+0000","last_became_peered":"2024-04-24T23:05:34.438477+0000","last_unstale":"2024-04-24T23:05:35.542062+0000","last_undegraded":"2024-04-24T23:05:35.542062+0000","last_fullsized":"2024-04-24T23:05:35.542062+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:50:24.587596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:36.754183+0000","last_change":"2024-04-24T23:05:34.433335+0000","last_active":"2024-04-24T23:05:36.754183+0000","last_peered":"2024-04-24T23:05:36.754183+0000","last_clean":"2024-04-24T23:05:36.754183+0000","last_became_active":"2024-04-24T23:05:34.433073+0000","last_became_peered":"2024-04-24T23:05:34.433073+0000","last_unstale":"2024-04-24T23:05:36.754183+0000","last_undegraded":"2024-04-24T23:05:36.754183+0000","last_fullsized":"2024-04-24T23:05:36.754183+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:19:30.205516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.421646+0000","last_change":"2024-04-24T23:05:34.432655+0000","last_active":"2024-04-24T23:05:35.421646+0000","last_peered":"2024-04-24T23:05:35.421646+0000","last_clean":"2024-04-24T23:05:35.421646+0000","last_became_active":"2024-04-24T23:05:34.432400+0000","last_became_peered":"2024-04-24T23:05:34.432400+0000","last_unstale":"2024-04-24T23:05:35.421646+0000","last_undegraded":"2024-04-24T23:05:35.421646+0000","last_fullsized":"2024-04-24T23:05:35.421646+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:35:38.814104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:36.754209+0000","last_change":"2024-04-24T23:05:34.437757+0000","last_active":"2024-04-24T23:05:36.754209+0000","last_peered":"2024-04-24T23:05:36.754209+0000","last_clean":"2024-04-24T23:05:36.754209+0000","last_became_active":"2024-04-24T23:05:34.437508+0000","last_became_peered":"2024-04-24T23:05:34.437508+0000","last_unstale":"2024-04-24T23:05:36.754209+0000","last_undegraded":"2024-04-24T23:05:36.754209+0000","last_fullsized":"2024-04-24T23:05:36.754209+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:49.557955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:36.754278+0000","last_change":"2024-04-24T23:05:34.442202+0000","last_active":"2024-04-24T23:05:36.754278+0000","last_peered":"2024-04-24T23:05:36.754278+0000","last_clean":"2024-04-24T23:05:36.754278+0000","last_became_active":"2024-04-24T23:05:34.441949+0000","last_became_peered":"2024-04-24T23:05:34.441949+0000","last_unstale":"2024-04-24T23:05:36.754278+0000","last_undegraded":"2024-04-24T23:05:36.754278+0000","last_fullsized":"2024-04-24T23:05:36.754278+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:44:19.263757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.421746+0000","last_change":"2024-04-24T23:05:34.431325+0000","last_active":"2024-04-24T23:05:35.421746+0000","last_peered":"2024-04-24T23:05:35.421746+0000","last_clean":"2024-04-24T23:05:35.421746+0000","last_became_active":"2024-04-24T23:05:34.431001+0000","last_became_peered":"2024-04-24T23:05:34.431001+0000","last_unstale":"2024-04-24T23:05:35.421746+0000","last_undegraded":"2024-04-24T23:05:35.421746+0000","last_fullsized":"2024-04-24T23:05:35.421746+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:53:27.345860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.758401+0000","last_change":"2024-04-24T23:05:34.427574+0000","last_active":"2024-04-24T23:05:35.758401+0000","last_peered":"2024-04-24T23:05:35.758401+0000","last_clean":"2024-04-24T23:05:35.758401+0000","last_became_active":"2024-04-24T23:05:34.427305+0000","last_became_peered":"2024-04-24T23:05:34.427305+0000","last_unstale":"2024-04-24T23:05:35.758401+0000","last_undegraded":"2024-04-24T23:05:35.758401+0000","last_fullsized":"2024-04-24T23:05:35.758401+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:25.951310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:36.536716+0000","last_change":"2024-04-24T23:05:34.431114+0000","last_active":"2024-04-24T23:05:36.536716+0000","last_peered":"2024-04-24T23:05:36.536716+0000","last_clean":"2024-04-24T23:05:36.536716+0000","last_became_active":"2024-04-24T23:05:34.430526+0000","last_became_peered":"2024-04-24T23:05:34.430526+0000","last_unstale":"2024-04-24T23:05:36.536716+0000","last_undegraded":"2024-04-24T23:05:36.536716+0000","last_fullsized":"2024-04-24T23:05:36.536716+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:56:41.938698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.422541+0000","last_change":"2024-04-24T23:05:34.437889+0000","last_active":"2024-04-24T23:05:35.422541+0000","last_peered":"2024-04-24T23:05:35.422541+0000","last_clean":"2024-04-24T23:05:35.422541+0000","last_became_active":"2024-04-24T23:05:34.437525+0000","last_became_peered":"2024-04-24T23:05:34.437525+0000","last_unstale":"2024-04-24T23:05:35.422541+0000","last_undegraded":"2024-04-24T23:05:35.422541+0000","last_fullsized":"2024-04-24T23:05:35.422541+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:31.864268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.421490+0000","last_change":"2024-04-24T23:05:34.436354+0000","last_active":"2024-04-24T23:05:35.421490+0000","last_peered":"2024-04-24T23:05:35.421490+0000","last_clean":"2024-04-24T23:05:35.421490+0000","last_became_active":"2024-04-24T23:05:34.435882+0000","last_became_peered":"2024-04-24T23:05:34.435882+0000","last_unstale":"2024-04-24T23:05:35.421490+0000","last_undegraded":"2024-04-24T23:05:35.421490+0000","last_fullsized":"2024-04-24T23:05:35.421490+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:04.009068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.758737+0000","last_change":"2024-04-24T23:05:34.433774+0000","last_active":"2024-04-24T23:05:35.758737+0000","last_peered":"2024-04-24T23:05:35.758737+0000","last_clean":"2024-04-24T23:05:35.758737+0000","last_became_active":"2024-04-24T23:05:34.433535+0000","last_became_peered":"2024-04-24T23:05:34.433535+0000","last_unstale":"2024-04-24T23:05:35.758737+0000","last_undegraded":"2024-04-24T23:05:35.758737+0000","last_fullsized":"2024-04-24T23:05:35.758737+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:27:51.152470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.423310+0000","last_change":"2024-04-24T23:05:34.435513+0000","last_active":"2024-04-24T23:05:35.423310+0000","last_peered":"2024-04-24T23:05:35.423310+0000","last_clean":"2024-04-24T23:05:35.423310+0000","last_became_active":"2024-04-24T23:05:34.435209+0000","last_became_peered":"2024-04-24T23:05:34.435209+0000","last_unstale":"2024-04-24T23:05:35.423310+0000","last_undegraded":"2024-04-24T23:05:35.423310+0000","last_fullsized":"2024-04-24T23:05:35.423310+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:10:55.262208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.758382+0000","last_change":"2024-04-24T23:05:34.428127+0000","last_active":"2024-04-24T23:05:35.758382+0000","last_peered":"2024-04-24T23:05:35.758382+0000","last_clean":"2024-04-24T23:05:35.758382+0000","last_became_active":"2024-04-24T23:05:34.427831+0000","last_became_peered":"2024-04-24T23:05:34.427831+0000","last_unstale":"2024-04-24T23:05:35.758382+0000","last_undegraded":"2024-04-24T23:05:35.758382+0000","last_fullsized":"2024-04-24T23:05:35.758382+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:52:49.899040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:36.536887+0000","last_change":"2024-04-24T23:05:34.428336+0000","last_active":"2024-04-24T23:05:36.536887+0000","last_peered":"2024-04-24T23:05:36.536887+0000","last_clean":"2024-04-24T23:05:36.536887+0000","last_became_active":"2024-04-24T23:05:34.427582+0000","last_became_peered":"2024-04-24T23:05:34.427582+0000","last_unstale":"2024-04-24T23:05:36.536887+0000","last_undegraded":"2024-04-24T23:05:36.536887+0000","last_fullsized":"2024-04-24T23:05:36.536887+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:51:15.742586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.421648+0000","last_change":"2024-04-24T23:05:34.436116+0000","last_active":"2024-04-24T23:05:35.421648+0000","last_peered":"2024-04-24T23:05:35.421648+0000","last_clean":"2024-04-24T23:05:35.421648+0000","last_became_active":"2024-04-24T23:05:34.435569+0000","last_became_peered":"2024-04-24T23:05:34.435569+0000","last_unstale":"2024-04-24T23:05:35.421648+0000","last_undegraded":"2024-04-24T23:05:35.421648+0000","last_fullsized":"2024-04-24T23:05:35.421648+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:43.290722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.422679+0000","last_change":"2024-04-24T23:05:34.435453+0000","last_active":"2024-04-24T23:05:35.422679+0000","last_peered":"2024-04-24T23:05:35.422679+0000","last_clean":"2024-04-24T23:05:35.422679+0000","last_became_active":"2024-04-24T23:05:34.434970+0000","last_became_peered":"2024-04-24T23:05:34.434970+0000","last_unstale":"2024-04-24T23:05:35.422679+0000","last_undegraded":"2024-04-24T23:05:35.422679+0000","last_fullsized":"2024-04-24T23:05:35.422679+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:25.877884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.542003+0000","last_change":"2024-04-24T23:05:34.441368+0000","last_active":"2024-04-24T23:05:35.542003+0000","last_peered":"2024-04-24T23:05:35.542003+0000","last_clean":"2024-04-24T23:05:35.542003+0000","last_became_active":"2024-04-24T23:05:34.441136+0000","last_became_peered":"2024-04-24T23:05:34.441136+0000","last_unstale":"2024-04-24T23:05:35.542003+0000","last_undegraded":"2024-04-24T23:05:35.542003+0000","last_fullsized":"2024-04-24T23:05:35.542003+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:39.837776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.422411+0000","last_change":"2024-04-24T23:05:34.439810+0000","last_active":"2024-04-24T23:05:35.422411+0000","last_peered":"2024-04-24T23:05:35.422411+0000","last_clean":"2024-04-24T23:05:35.422411+0000","last_became_active":"2024-04-24T23:05:34.439092+0000","last_became_peered":"2024-04-24T23:05:34.439092+0000","last_unstale":"2024-04-24T23:05:35.422411+0000","last_undegraded":"2024-04-24T23:05:35.422411+0000","last_fullsized":"2024-04-24T23:05:35.422411+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:24:11.127522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"139'4","reported_seq":18,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:36.537181+0000","last_change":"2024-04-24T23:05:33.424762+0000","last_active":"2024-04-24T23:05:36.537181+0000","last_peered":"2024-04-24T23:05:36.537181+0000","last_clean":"2024-04-24T23:05:36.537181+0000","last_became_active":"2024-04-24T23:05:33.424523+0000","last_became_peered":"2024-04-24T23:05:33.424523+0000","last_unstale":"2024-04-24T23:05:36.537181+0000","last_undegraded":"2024-04-24T23:05:36.537181+0000","last_fullsized":"2024-04-24T23:05:36.537181+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:28:11.678570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:36.536888+0000","last_change":"2024-04-24T23:05:34.431755+0000","last_active":"2024-04-24T23:05:36.536888+0000","last_peered":"2024-04-24T23:05:36.536888+0000","last_clean":"2024-04-24T23:05:36.536888+0000","last_became_active":"2024-04-24T23:05:34.430852+0000","last_became_peered":"2024-04-24T23:05:34.430852+0000","last_unstale":"2024-04-24T23:05:36.536888+0000","last_undegraded":"2024-04-24T23:05:36.536888+0000","last_fullsized":"2024-04-24T23:05:36.536888+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:36.981128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1e","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.758307+0000","last_change":"2024-04-24T23:05:33.422951+0000","last_active":"2024-04-24T23:05:35.758307+0000","last_peered":"2024-04-24T23:05:35.758307+0000","last_clean":"2024-04-24T23:05:35.758307+0000","last_became_active":"2024-04-24T23:05:33.422697+0000","last_became_peered":"2024-04-24T23:05:33.422697+0000","last_unstale":"2024-04-24T23:05:35.758307+0000","last_undegraded":"2024-04-24T23:05:35.758307+0000","last_fullsized":"2024-04-24T23:05:35.758307+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:56:46.572909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.758567+0000","last_change":"2024-04-24T23:05:34.436891+0000","last_active":"2024-04-24T23:05:35.758567+0000","last_peered":"2024-04-24T23:05:35.758567+0000","last_clean":"2024-04-24T23:05:35.758567+0000","last_became_active":"2024-04-24T23:05:34.436667+0000","last_became_peered":"2024-04-24T23:05:34.436667+0000","last_unstale":"2024-04-24T23:05:35.758567+0000","last_undegraded":"2024-04-24T23:05:35.758567+0000","last_fullsized":"2024-04-24T23:05:35.758567+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:13.649179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1d","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.423684+0000","last_change":"2024-04-24T23:05:33.433064+0000","last_active":"2024-04-24T23:05:35.423684+0000","last_peered":"2024-04-24T23:05:35.423684+0000","last_clean":"2024-04-24T23:05:35.423684+0000","last_became_active":"2024-04-24T23:05:33.432806+0000","last_became_peered":"2024-04-24T23:05:33.432806+0000","last_unstale":"2024-04-24T23:05:35.423684+0000","last_undegraded":"2024-04-24T23:05:35.423684+0000","last_fullsized":"2024-04-24T23:05:35.423684+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:46:24.321544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:36.754429+0000","last_change":"2024-04-24T23:05:34.440019+0000","last_active":"2024-04-24T23:05:36.754429+0000","last_peered":"2024-04-24T23:05:36.754429+0000","last_clean":"2024-04-24T23:05:36.754429+0000","last_became_active":"2024-04-24T23:05:34.439735+0000","last_became_peered":"2024-04-24T23:05:34.439735+0000","last_unstale":"2024-04-24T23:05:36.754429+0000","last_undegraded":"2024-04-24T23:05:36.754429+0000","last_fullsized":"2024-04-24T23:05:36.754429+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:46.855410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.423226+0000","last_change":"2024-04-24T23:05:33.431526+0000","last_active":"2024-04-24T23:05:35.423226+0000","last_peered":"2024-04-24T23:05:35.423226+0000","last_clean":"2024-04-24T23:05:35.423226+0000","last_became_active":"2024-04-24T23:05:33.431291+0000","last_became_peered":"2024-04-24T23:05:33.431291+0000","last_unstale":"2024-04-24T23:05:35.423226+0000","last_undegraded":"2024-04-24T23:05:35.423226+0000","last_fullsized":"2024-04-24T23:05:35.423226+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:50:04.910591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.422933+0000","last_change":"2024-04-24T23:05:34.436634+0000","last_active":"2024-04-24T23:05:35.422933+0000","last_peered":"2024-04-24T23:05:35.422933+0000","last_clean":"2024-04-24T23:05:35.422933+0000","last_became_active":"2024-04-24T23:05:34.436381+0000","last_became_peered":"2024-04-24T23:05:34.436381+0000","last_unstale":"2024-04-24T23:05:35.422933+0000","last_undegraded":"2024-04-24T23:05:35.422933+0000","last_fullsized":"2024-04-24T23:05:35.422933+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:11:55.139151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1b","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:36.754490+0000","last_change":"2024-04-24T23:05:33.426739+0000","last_active":"2024-04-24T23:05:36.754490+0000","last_peered":"2024-04-24T23:05:36.754490+0000","last_clean":"2024-04-24T23:05:36.754490+0000","last_became_active":"2024-04-24T23:05:33.426341+0000","last_became_peered":"2024-04-24T23:05:33.426341+0000","last_unstale":"2024-04-24T23:05:36.754490+0000","last_undegraded":"2024-04-24T23:05:36.754490+0000","last_fullsized":"2024-04-24T23:05:36.754490+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:48:27.126130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.421659+0000","last_change":"2024-04-24T23:05:34.435065+0000","last_active":"2024-04-24T23:05:35.421659+0000","last_peered":"2024-04-24T23:05:35.421659+0000","last_clean":"2024-04-24T23:05:35.421659+0000","last_became_active":"2024-04-24T23:05:34.434780+0000","last_became_peered":"2024-04-24T23:05:34.434780+0000","last_unstale":"2024-04-24T23:05:35.421659+0000","last_undegraded":"2024-04-24T23:05:35.421659+0000","last_fullsized":"2024-04-24T23:05:35.421659+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:51:18.245665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1a","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.422635+0000","last_change":"2024-04-24T23:05:33.427744+0000","last_active":"2024-04-24T23:05:35.422635+0000","last_peered":"2024-04-24T23:05:35.422635+0000","last_clean":"2024-04-24T23:05:35.422635+0000","last_became_active":"2024-04-24T23:05:33.427055+0000","last_became_peered":"2024-04-24T23:05:33.427055+0000","last_unstale":"2024-04-24T23:05:35.422635+0000","last_undegraded":"2024-04-24T23:05:35.422635+0000","last_fullsized":"2024-04-24T23:05:35.422635+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:23.744870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.541910+0000","last_change":"2024-04-24T23:05:34.434024+0000","last_active":"2024-04-24T23:05:35.541910+0000","last_peered":"2024-04-24T23:05:35.541910+0000","last_clean":"2024-04-24T23:05:35.541910+0000","last_became_active":"2024-04-24T23:05:34.433417+0000","last_became_peered":"2024-04-24T23:05:34.433417+0000","last_unstale":"2024-04-24T23:05:35.541910+0000","last_undegraded":"2024-04-24T23:05:35.541910+0000","last_fullsized":"2024-04-24T23:05:35.541910+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:26:37.412259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.421701+0000","last_change":"2024-04-24T23:05:33.425559+0000","last_active":"2024-04-24T23:05:35.421701+0000","last_peered":"2024-04-24T23:05:35.421701+0000","last_clean":"2024-04-24T23:05:35.421701+0000","last_became_active":"2024-04-24T23:05:33.425269+0000","last_became_peered":"2024-04-24T23:05:33.425269+0000","last_unstale":"2024-04-24T23:05:35.421701+0000","last_undegraded":"2024-04-24T23:05:35.421701+0000","last_fullsized":"2024-04-24T23:05:35.421701+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:10:01.141531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.421525+0000","last_change":"2024-04-24T23:05:34.430981+0000","last_active":"2024-04-24T23:05:35.421525+0000","last_peered":"2024-04-24T23:05:35.421525+0000","last_clean":"2024-04-24T23:05:35.421525+0000","last_became_active":"2024-04-24T23:05:34.430535+0000","last_became_peered":"2024-04-24T23:05:34.430535+0000","last_unstale":"2024-04-24T23:05:35.421525+0000","last_undegraded":"2024-04-24T23:05:35.421525+0000","last_fullsized":"2024-04-24T23:05:35.421525+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:31:35.233121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.542325+0000","last_change":"2024-04-24T23:05:33.430778+0000","last_active":"2024-04-24T23:05:35.542325+0000","last_peered":"2024-04-24T23:05:35.542325+0000","last_clean":"2024-04-24T23:05:35.542325+0000","last_became_active":"2024-04-24T23:05:33.429066+0000","last_became_peered":"2024-04-24T23:05:33.429066+0000","last_unstale":"2024-04-24T23:05:35.542325+0000","last_undegraded":"2024-04-24T23:05:35.542325+0000","last_fullsized":"2024-04-24T23:05:35.542325+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:09:05.492627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.758361+0000","last_change":"2024-04-24T23:05:34.433643+0000","last_active":"2024-04-24T23:05:35.758361+0000","last_peered":"2024-04-24T23:05:35.758361+0000","last_clean":"2024-04-24T23:05:35.758361+0000","last_became_active":"2024-04-24T23:05:34.433397+0000","last_became_peered":"2024-04-24T23:05:34.433397+0000","last_unstale":"2024-04-24T23:05:35.758361+0000","last_undegraded":"2024-04-24T23:05:35.758361+0000","last_fullsized":"2024-04-24T23:05:35.758361+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:04.560118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.423348+0000","last_change":"2024-04-24T23:05:33.430918+0000","last_active":"2024-04-24T23:05:35.423348+0000","last_peered":"2024-04-24T23:05:35.423348+0000","last_clean":"2024-04-24T23:05:35.423348+0000","last_became_active":"2024-04-24T23:05:33.430161+0000","last_became_peered":"2024-04-24T23:05:33.430161+0000","last_unstale":"2024-04-24T23:05:35.423348+0000","last_undegraded":"2024-04-24T23:05:35.423348+0000","last_fullsized":"2024-04-24T23:05:35.423348+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:12:14.042047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.758093+0000","last_change":"2024-04-24T23:05:34.434073+0000","last_active":"2024-04-24T23:05:35.758093+0000","last_peered":"2024-04-24T23:05:35.758093+0000","last_clean":"2024-04-24T23:05:35.758093+0000","last_became_active":"2024-04-24T23:05:34.433819+0000","last_became_peered":"2024-04-24T23:05:34.433819+0000","last_unstale":"2024-04-24T23:05:35.758093+0000","last_undegraded":"2024-04-24T23:05:35.758093+0000","last_fullsized":"2024-04-24T23:05:35.758093+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:16:10.393384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.542106+0000","last_change":"2024-04-24T23:05:33.424437+0000","last_active":"2024-04-24T23:05:35.542106+0000","last_peered":"2024-04-24T23:05:35.542106+0000","last_clean":"2024-04-24T23:05:35.542106+0000","last_became_active":"2024-04-24T23:05:33.424052+0000","last_became_peered":"2024-04-24T23:05:33.424052+0000","last_unstale":"2024-04-24T23:05:35.542106+0000","last_undegraded":"2024-04-24T23:05:35.542106+0000","last_fullsized":"2024-04-24T23:05:35.542106+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:03.154085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.758336+0000","last_change":"2024-04-24T23:05:34.429265+0000","last_active":"2024-04-24T23:05:35.758336+0000","last_peered":"2024-04-24T23:05:35.758336+0000","last_clean":"2024-04-24T23:05:35.758336+0000","last_became_active":"2024-04-24T23:05:34.429029+0000","last_became_peered":"2024-04-24T23:05:34.429029+0000","last_unstale":"2024-04-24T23:05:35.758336+0000","last_undegraded":"2024-04-24T23:05:35.758336+0000","last_fullsized":"2024-04-24T23:05:35.758336+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:45:45.885131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"139'3","reported_seq":17,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.541646+0000","last_change":"2024-04-24T23:05:33.431987+0000","last_active":"2024-04-24T23:05:35.541646+0000","last_peered":"2024-04-24T23:05:35.541646+0000","last_clean":"2024-04-24T23:05:35.541646+0000","last_became_active":"2024-04-24T23:05:33.431614+0000","last_became_peered":"2024-04-24T23:05:33.431614+0000","last_unstale":"2024-04-24T23:05:35.541646+0000","last_undegraded":"2024-04-24T23:05:35.541646+0000","last_fullsized":"2024-04-24T23:05:35.541646+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:57:39.840720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.421798+0000","last_change":"2024-04-24T23:05:34.436433+0000","last_active":"2024-04-24T23:05:35.421798+0000","last_peered":"2024-04-24T23:05:35.421798+0000","last_clean":"2024-04-24T23:05:35.421798+0000","last_became_active":"2024-04-24T23:05:34.436187+0000","last_became_peered":"2024-04-24T23:05:34.436187+0000","last_unstale":"2024-04-24T23:05:35.421798+0000","last_undegraded":"2024-04-24T23:05:35.421798+0000","last_fullsized":"2024-04-24T23:05:35.421798+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:44:26.418771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"139'6","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.422720+0000","last_change":"2024-04-24T23:05:33.429393+0000","last_active":"2024-04-24T23:05:35.422720+0000","last_peered":"2024-04-24T23:05:35.422720+0000","last_clean":"2024-04-24T23:05:35.422720+0000","last_became_active":"2024-04-24T23:05:33.428836+0000","last_became_peered":"2024-04-24T23:05:33.428836+0000","last_unstale":"2024-04-24T23:05:35.422720+0000","last_undegraded":"2024-04-24T23:05:35.422720+0000","last_fullsized":"2024-04-24T23:05:35.422720+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:53.541586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.541883+0000","last_change":"2024-04-24T23:05:34.438040+0000","last_active":"2024-04-24T23:05:35.541883+0000","last_peered":"2024-04-24T23:05:35.541883+0000","last_clean":"2024-04-24T23:05:35.541883+0000","last_became_active":"2024-04-24T23:05:34.437807+0000","last_became_peered":"2024-04-24T23:05:34.437807+0000","last_unstale":"2024-04-24T23:05:35.541883+0000","last_undegraded":"2024-04-24T23:05:35.541883+0000","last_fullsized":"2024-04-24T23:05:35.541883+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:30:22.392636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.542474+0000","last_change":"2024-04-24T23:05:33.429287+0000","last_active":"2024-04-24T23:05:35.542474+0000","last_peered":"2024-04-24T23:05:35.542474+0000","last_clean":"2024-04-24T23:05:35.542474+0000","last_became_active":"2024-04-24T23:05:33.428276+0000","last_became_peered":"2024-04-24T23:05:33.428276+0000","last_unstale":"2024-04-24T23:05:35.542474+0000","last_undegraded":"2024-04-24T23:05:35.542474+0000","last_fullsized":"2024-04-24T23:05:35.542474+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:43:33.806921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.423578+0000","last_change":"2024-04-24T23:05:34.435723+0000","last_active":"2024-04-24T23:05:35.423578+0000","last_peered":"2024-04-24T23:05:35.423578+0000","last_clean":"2024-04-24T23:05:35.423578+0000","last_became_active":"2024-04-24T23:05:34.434981+0000","last_became_peered":"2024-04-24T23:05:34.434981+0000","last_unstale":"2024-04-24T23:05:35.423578+0000","last_undegraded":"2024-04-24T23:05:35.423578+0000","last_fullsized":"2024-04-24T23:05:35.423578+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:09:20.362005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:36.754398+0000","last_change":"2024-04-24T23:05:33.432071+0000","last_active":"2024-04-24T23:05:36.754398+0000","last_peered":"2024-04-24T23:05:36.754398+0000","last_clean":"2024-04-24T23:05:36.754398+0000","last_became_active":"2024-04-24T23:05:33.431681+0000","last_became_peered":"2024-04-24T23:05:33.431681+0000","last_unstale":"2024-04-24T23:05:36.754398+0000","last_undegraded":"2024-04-24T23:05:36.754398+0000","last_fullsized":"2024-04-24T23:05:36.754398+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:32:00.321253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.423502+0000","last_change":"2024-04-24T23:05:34.434329+0000","last_active":"2024-04-24T23:05:35.423502+0000","last_peered":"2024-04-24T23:05:35.423502+0000","last_clean":"2024-04-24T23:05:35.423502+0000","last_became_active":"2024-04-24T23:05:34.434085+0000","last_became_peered":"2024-04-24T23:05:34.434085+0000","last_unstale":"2024-04-24T23:05:35.423502+0000","last_undegraded":"2024-04-24T23:05:35.423502+0000","last_fullsized":"2024-04-24T23:05:35.423502+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:59:23.738713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"139'8","reported_seq":22,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.422868+0000","last_change":"2024-04-24T23:05:33.429042+0000","last_active":"2024-04-24T23:05:35.422868+0000","last_peered":"2024-04-24T23:05:35.422868+0000","last_clean":"2024-04-24T23:05:35.422868+0000","last_became_active":"2024-04-24T23:05:33.428789+0000","last_became_peered":"2024-04-24T23:05:33.428789+0000","last_unstale":"2024-04-24T23:05:35.422868+0000","last_undegraded":"2024-04-24T23:05:35.422868+0000","last_fullsized":"2024-04-24T23:05:35.422868+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:36.904618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.422741+0000","last_change":"2024-04-24T23:05:34.430678+0000","last_active":"2024-04-24T23:05:35.422741+0000","last_peered":"2024-04-24T23:05:35.422741+0000","last_clean":"2024-04-24T23:05:35.422741+0000","last_became_active":"2024-04-24T23:05:34.430432+0000","last_became_peered":"2024-04-24T23:05:34.430432+0000","last_unstale":"2024-04-24T23:05:35.422741+0000","last_undegraded":"2024-04-24T23:05:35.422741+0000","last_fullsized":"2024-04-24T23:05:35.422741+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:29:37.052742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.f","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.758469+0000","last_change":"2024-04-24T23:05:33.432463+0000","last_active":"2024-04-24T23:05:35.758469+0000","last_peered":"2024-04-24T23:05:35.758469+0000","last_clean":"2024-04-24T23:05:35.758469+0000","last_became_active":"2024-04-24T23:05:33.431935+0000","last_became_peered":"2024-04-24T23:05:33.431935+0000","last_unstale":"2024-04-24T23:05:35.758469+0000","last_undegraded":"2024-04-24T23:05:35.758469+0000","last_fullsized":"2024-04-24T23:05:35.758469+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:15:05.612832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.422570+0000","last_change":"2024-04-24T23:05:34.440635+0000","last_active":"2024-04-24T23:05:35.422570+0000","last_peered":"2024-04-24T23:05:35.422570+0000","last_clean":"2024-04-24T23:05:35.422570+0000","last_became_active":"2024-04-24T23:05:34.439414+0000","last_became_peered":"2024-04-24T23:05:34.439414+0000","last_unstale":"2024-04-24T23:05:35.422570+0000","last_undegraded":"2024-04-24T23:05:35.422570+0000","last_fullsized":"2024-04-24T23:05:35.422570+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:55:08.361607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.542534+0000","last_change":"2024-04-24T23:05:33.429481+0000","last_active":"2024-04-24T23:05:35.542534+0000","last_peered":"2024-04-24T23:05:35.542534+0000","last_clean":"2024-04-24T23:05:35.542534+0000","last_became_active":"2024-04-24T23:05:33.428605+0000","last_became_peered":"2024-04-24T23:05:33.428605+0000","last_unstale":"2024-04-24T23:05:35.542534+0000","last_undegraded":"2024-04-24T23:05:35.542534+0000","last_fullsized":"2024-04-24T23:05:35.542534+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:14:25.825665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.421798+0000","last_change":"2024-04-24T23:05:34.432876+0000","last_active":"2024-04-24T23:05:35.421798+0000","last_peered":"2024-04-24T23:05:35.421798+0000","last_clean":"2024-04-24T23:05:35.421798+0000","last_became_active":"2024-04-24T23:05:34.432594+0000","last_became_peered":"2024-04-24T23:05:34.432594+0000","last_unstale":"2024-04-24T23:05:35.421798+0000","last_undegraded":"2024-04-24T23:05:35.421798+0000","last_fullsized":"2024-04-24T23:05:35.421798+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:06.169592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.d","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:36.537406+0000","last_change":"2024-04-24T23:05:33.429289+0000","last_active":"2024-04-24T23:05:36.537406+0000","last_peered":"2024-04-24T23:05:36.537406+0000","last_clean":"2024-04-24T23:05:36.537406+0000","last_became_active":"2024-04-24T23:05:33.429063+0000","last_became_peered":"2024-04-24T23:05:33.429063+0000","last_unstale":"2024-04-24T23:05:36.537406+0000","last_undegraded":"2024-04-24T23:05:36.537406+0000","last_fullsized":"2024-04-24T23:05:36.537406+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:03.163984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:36.537026+0000","last_change":"2024-04-24T23:05:34.433294+0000","last_active":"2024-04-24T23:05:36.537026+0000","last_peered":"2024-04-24T23:05:36.537026+0000","last_clean":"2024-04-24T23:05:36.537026+0000","last_became_active":"2024-04-24T23:05:34.432955+0000","last_became_peered":"2024-04-24T23:05:34.432955+0000","last_unstale":"2024-04-24T23:05:36.537026+0000","last_undegraded":"2024-04-24T23:05:36.537026+0000","last_fullsized":"2024-04-24T23:05:36.537026+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:04:19.696935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:36.536929+0000","last_change":"2024-04-24T23:05:33.431817+0000","last_active":"2024-04-24T23:05:36.536929+0000","last_peered":"2024-04-24T23:05:36.536929+0000","last_clean":"2024-04-24T23:05:36.536929+0000","last_became_active":"2024-04-24T23:05:33.431569+0000","last_became_peered":"2024-04-24T23:05:33.431569+0000","last_unstale":"2024-04-24T23:05:36.536929+0000","last_undegraded":"2024-04-24T23:05:36.536929+0000","last_fullsized":"2024-04-24T23:05:36.536929+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:36.333278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:36.754382+0000","last_change":"2024-04-24T23:05:34.439443+0000","last_active":"2024-04-24T23:05:36.754382+0000","last_peered":"2024-04-24T23:05:36.754382+0000","last_clean":"2024-04-24T23:05:36.754382+0000","last_became_active":"2024-04-24T23:05:34.439154+0000","last_became_peered":"2024-04-24T23:05:34.439154+0000","last_unstale":"2024-04-24T23:05:36.754382+0000","last_undegraded":"2024-04-24T23:05:36.754382+0000","last_fullsized":"2024-04-24T23:05:36.754382+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:08:36.733577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.b","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.541764+0000","last_change":"2024-04-24T23:05:33.425107+0000","last_active":"2024-04-24T23:05:35.541764+0000","last_peered":"2024-04-24T23:05:35.541764+0000","last_clean":"2024-04-24T23:05:35.541764+0000","last_became_active":"2024-04-24T23:05:33.424855+0000","last_became_peered":"2024-04-24T23:05:33.424855+0000","last_unstale":"2024-04-24T23:05:35.541764+0000","last_undegraded":"2024-04-24T23:05:35.541764+0000","last_fullsized":"2024-04-24T23:05:35.541764+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:43:59.232440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.422778+0000","last_change":"2024-04-24T23:05:34.433609+0000","last_active":"2024-04-24T23:05:35.422778+0000","last_peered":"2024-04-24T23:05:35.422778+0000","last_clean":"2024-04-24T23:05:35.422778+0000","last_became_active":"2024-04-24T23:05:34.433208+0000","last_became_peered":"2024-04-24T23:05:34.433208+0000","last_unstale":"2024-04-24T23:05:35.422778+0000","last_undegraded":"2024-04-24T23:05:35.422778+0000","last_fullsized":"2024-04-24T23:05:35.422778+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:55:21.103335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.422673+0000","last_change":"2024-04-24T23:05:33.432951+0000","last_active":"2024-04-24T23:05:35.422673+0000","last_peered":"2024-04-24T23:05:35.422673+0000","last_clean":"2024-04-24T23:05:35.422673+0000","last_became_active":"2024-04-24T23:05:33.432689+0000","last_became_peered":"2024-04-24T23:05:33.432689+0000","last_unstale":"2024-04-24T23:05:35.422673+0000","last_undegraded":"2024-04-24T23:05:35.422673+0000","last_fullsized":"2024-04-24T23:05:35.422673+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:19:08.520909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:36.537177+0000","last_change":"2024-04-24T23:05:34.435206+0000","last_active":"2024-04-24T23:05:36.537177+0000","last_peered":"2024-04-24T23:05:36.537177+0000","last_clean":"2024-04-24T23:05:36.537177+0000","last_became_active":"2024-04-24T23:05:34.434699+0000","last_became_peered":"2024-04-24T23:05:34.434699+0000","last_unstale":"2024-04-24T23:05:36.537177+0000","last_undegraded":"2024-04-24T23:05:36.537177+0000","last_fullsized":"2024-04-24T23:05:36.537177+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:15.953037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.423780+0000","last_change":"2024-04-24T23:05:33.431825+0000","last_active":"2024-04-24T23:05:35.423780+0000","last_peered":"2024-04-24T23:05:35.423780+0000","last_clean":"2024-04-24T23:05:35.423780+0000","last_became_active":"2024-04-24T23:05:33.430954+0000","last_became_peered":"2024-04-24T23:05:33.430954+0000","last_unstale":"2024-04-24T23:05:35.423780+0000","last_undegraded":"2024-04-24T23:05:35.423780+0000","last_fullsized":"2024-04-24T23:05:35.423780+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:05:54.412042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.542075+0000","last_change":"2024-04-24T23:05:34.440213+0000","last_active":"2024-04-24T23:05:35.542075+0000","last_peered":"2024-04-24T23:05:35.542075+0000","last_clean":"2024-04-24T23:05:35.542075+0000","last_became_active":"2024-04-24T23:05:34.439991+0000","last_became_peered":"2024-04-24T23:05:34.439991+0000","last_unstale":"2024-04-24T23:05:35.542075+0000","last_undegraded":"2024-04-24T23:05:35.542075+0000","last_fullsized":"2024-04-24T23:05:35.542075+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:00:07.214249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.423462+0000","last_change":"2024-04-24T23:05:33.430669+0000","last_active":"2024-04-24T23:05:35.423462+0000","last_peered":"2024-04-24T23:05:35.423462+0000","last_clean":"2024-04-24T23:05:35.423462+0000","last_became_active":"2024-04-24T23:05:33.429839+0000","last_became_peered":"2024-04-24T23:05:33.429839+0000","last_unstale":"2024-04-24T23:05:35.423462+0000","last_undegraded":"2024-04-24T23:05:35.423462+0000","last_fullsized":"2024-04-24T23:05:35.423462+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:41:13.207524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:36.537067+0000","last_change":"2024-04-24T23:05:34.435311+0000","last_active":"2024-04-24T23:05:36.537067+0000","last_peered":"2024-04-24T23:05:36.537067+0000","last_clean":"2024-04-24T23:05:36.537067+0000","last_became_active":"2024-04-24T23:05:34.435046+0000","last_became_peered":"2024-04-24T23:05:34.435046+0000","last_unstale":"2024-04-24T23:05:36.537067+0000","last_undegraded":"2024-04-24T23:05:36.537067+0000","last_fullsized":"2024-04-24T23:05:36.537067+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:57.399349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.7","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.423674+0000","last_change":"2024-04-24T23:05:33.432007+0000","last_active":"2024-04-24T23:05:35.423674+0000","last_peered":"2024-04-24T23:05:35.423674+0000","last_clean":"2024-04-24T23:05:35.423674+0000","last_became_active":"2024-04-24T23:05:33.430736+0000","last_became_peered":"2024-04-24T23:05:33.430736+0000","last_unstale":"2024-04-24T23:05:35.423674+0000","last_undegraded":"2024-04-24T23:05:35.423674+0000","last_fullsized":"2024-04-24T23:05:35.423674+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:00:21.858605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.758620+0000","last_change":"2024-04-24T23:05:34.429975+0000","last_active":"2024-04-24T23:05:35.758620+0000","last_peered":"2024-04-24T23:05:35.758620+0000","last_clean":"2024-04-24T23:05:35.758620+0000","last_became_active":"2024-04-24T23:05:34.429704+0000","last_became_peered":"2024-04-24T23:05:34.429704+0000","last_unstale":"2024-04-24T23:05:35.758620+0000","last_undegraded":"2024-04-24T23:05:35.758620+0000","last_fullsized":"2024-04-24T23:05:35.758620+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:56:03.351101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"140'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:36.424699+0000","last_change":"2024-04-24T23:05:33.427070+0000","last_active":"2024-04-24T23:05:36.424699+0000","last_peered":"2024-04-24T23:05:36.424699+0000","last_clean":"2024-04-24T23:05:36.424699+0000","last_became_active":"2024-04-24T23:05:33.426791+0000","last_became_peered":"2024-04-24T23:05:33.426791+0000","last_unstale":"2024-04-24T23:05:36.424699+0000","last_undegraded":"2024-04-24T23:05:36.424699+0000","last_fullsized":"2024-04-24T23:05:36.424699+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:33:29.057440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.422891+0000","last_change":"2024-04-24T23:05:34.435945+0000","last_active":"2024-04-24T23:05:35.422891+0000","last_peered":"2024-04-24T23:05:35.422891+0000","last_clean":"2024-04-24T23:05:35.422891+0000","last_became_active":"2024-04-24T23:05:34.435233+0000","last_became_peered":"2024-04-24T23:05:34.435233+0000","last_unstale":"2024-04-24T23:05:35.422891+0000","last_undegraded":"2024-04-24T23:05:35.422891+0000","last_fullsized":"2024-04-24T23:05:35.422891+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:47.494849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.422819+0000","last_change":"2024-04-24T23:05:33.429170+0000","last_active":"2024-04-24T23:05:35.422819+0000","last_peered":"2024-04-24T23:05:35.422819+0000","last_clean":"2024-04-24T23:05:35.422819+0000","last_became_active":"2024-04-24T23:05:33.428500+0000","last_became_peered":"2024-04-24T23:05:33.428500+0000","last_unstale":"2024-04-24T23:05:35.422819+0000","last_undegraded":"2024-04-24T23:05:35.422819+0000","last_fullsized":"2024-04-24T23:05:35.422819+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:48:11.710253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:36.537067+0000","last_change":"2024-04-24T23:05:34.428521+0000","last_active":"2024-04-24T23:05:36.537067+0000","last_peered":"2024-04-24T23:05:36.537067+0000","last_clean":"2024-04-24T23:05:36.537067+0000","last_became_active":"2024-04-24T23:05:34.427910+0000","last_became_peered":"2024-04-24T23:05:34.427910+0000","last_unstale":"2024-04-24T23:05:36.537067+0000","last_undegraded":"2024-04-24T23:05:36.537067+0000","last_fullsized":"2024-04-24T23:05:36.537067+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:54.444606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.423873+0000","last_change":"2024-04-24T23:05:33.431184+0000","last_active":"2024-04-24T23:05:35.423873+0000","last_peered":"2024-04-24T23:05:35.423873+0000","last_clean":"2024-04-24T23:05:35.423873+0000","last_became_active":"2024-04-24T23:05:33.430590+0000","last_became_peered":"2024-04-24T23:05:33.430590+0000","last_unstale":"2024-04-24T23:05:35.423873+0000","last_undegraded":"2024-04-24T23:05:35.423873+0000","last_fullsized":"2024-04-24T23:05:35.423873+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:07:35.055097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:36.754415+0000","last_change":"2024-04-24T23:05:34.442204+0000","last_active":"2024-04-24T23:05:36.754415+0000","last_peered":"2024-04-24T23:05:36.754415+0000","last_clean":"2024-04-24T23:05:36.754415+0000","last_became_active":"2024-04-24T23:05:34.441934+0000","last_became_peered":"2024-04-24T23:05:34.441934+0000","last_unstale":"2024-04-24T23:05:36.754415+0000","last_undegraded":"2024-04-24T23:05:36.754415+0000","last_fullsized":"2024-04-24T23:05:36.754415+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:21:45.238132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.421855+0000","last_change":"2024-04-24T23:05:33.421091+0000","last_active":"2024-04-24T23:05:35.421855+0000","last_peered":"2024-04-24T23:05:35.421855+0000","last_clean":"2024-04-24T23:05:35.421855+0000","last_became_active":"2024-04-24T23:05:33.420817+0000","last_became_peered":"2024-04-24T23:05:33.420817+0000","last_unstale":"2024-04-24T23:05:35.421855+0000","last_undegraded":"2024-04-24T23:05:35.421855+0000","last_fullsized":"2024-04-24T23:05:35.421855+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:29:35.910813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:36.537308+0000","last_change":"2024-04-24T23:05:34.435023+0000","last_active":"2024-04-24T23:05:36.537308+0000","last_peered":"2024-04-24T23:05:36.537308+0000","last_clean":"2024-04-24T23:05:36.537308+0000","last_became_active":"2024-04-24T23:05:34.434269+0000","last_became_peered":"2024-04-24T23:05:34.434269+0000","last_unstale":"2024-04-24T23:05:36.537308+0000","last_undegraded":"2024-04-24T23:05:36.537308+0000","last_fullsized":"2024-04-24T23:05:36.537308+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:31:42.428916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:36.536996+0000","last_change":"2024-04-24T23:05:33.427689+0000","last_active":"2024-04-24T23:05:36.536996+0000","last_peered":"2024-04-24T23:05:36.536996+0000","last_clean":"2024-04-24T23:05:36.536996+0000","last_became_active":"2024-04-24T23:05:33.427406+0000","last_became_peered":"2024-04-24T23:05:33.427406+0000","last_unstale":"2024-04-24T23:05:36.536996+0000","last_undegraded":"2024-04-24T23:05:36.536996+0000","last_fullsized":"2024-04-24T23:05:36.536996+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:27:06.971278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:36.537224+0000","last_change":"2024-04-24T23:05:34.430457+0000","last_active":"2024-04-24T23:05:36.537224+0000","last_peered":"2024-04-24T23:05:36.537224+0000","last_clean":"2024-04-24T23:05:36.537224+0000","last_became_active":"2024-04-24T23:05:34.430197+0000","last_became_peered":"2024-04-24T23:05:34.430197+0000","last_unstale":"2024-04-24T23:05:36.537224+0000","last_undegraded":"2024-04-24T23:05:36.537224+0000","last_fullsized":"2024-04-24T23:05:36.537224+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:02:00.628269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.422975+0000","last_change":"2024-04-24T23:05:33.428478+0000","last_active":"2024-04-24T23:05:35.422975+0000","last_peered":"2024-04-24T23:05:35.422975+0000","last_clean":"2024-04-24T23:05:35.422975+0000","last_became_active":"2024-04-24T23:05:33.427474+0000","last_became_peered":"2024-04-24T23:05:33.427474+0000","last_unstale":"2024-04-24T23:05:35.422975+0000","last_undegraded":"2024-04-24T23:05:35.422975+0000","last_fullsized":"2024-04-24T23:05:35.422975+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:57.778668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.758552+0000","last_change":"2024-04-24T23:05:34.432945+0000","last_active":"2024-04-24T23:05:35.758552+0000","last_peered":"2024-04-24T23:05:35.758552+0000","last_clean":"2024-04-24T23:05:35.758552+0000","last_became_active":"2024-04-24T23:05:34.432651+0000","last_became_peered":"2024-04-24T23:05:34.432651+0000","last_unstale":"2024-04-24T23:05:35.758552+0000","last_undegraded":"2024-04-24T23:05:35.758552+0000","last_fullsized":"2024-04-24T23:05:35.758552+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:24:43.717620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.758616+0000","last_change":"2024-04-24T23:05:33.432773+0000","last_active":"2024-04-24T23:05:35.758616+0000","last_peered":"2024-04-24T23:05:35.758616+0000","last_clean":"2024-04-24T23:05:35.758616+0000","last_became_active":"2024-04-24T23:05:33.432239+0000","last_became_peered":"2024-04-24T23:05:33.432239+0000","last_unstale":"2024-04-24T23:05:35.758616+0000","last_undegraded":"2024-04-24T23:05:35.758616+0000","last_fullsized":"2024-04-24T23:05:35.758616+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:19.820007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.423158+0000","last_change":"2024-04-24T23:05:34.427830+0000","last_active":"2024-04-24T23:05:35.423158+0000","last_peered":"2024-04-24T23:05:35.423158+0000","last_clean":"2024-04-24T23:05:35.423158+0000","last_became_active":"2024-04-24T23:05:34.427574+0000","last_became_peered":"2024-04-24T23:05:34.427574+0000","last_unstale":"2024-04-24T23:05:35.423158+0000","last_undegraded":"2024-04-24T23:05:35.423158+0000","last_fullsized":"2024-04-24T23:05:35.423158+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:32.846474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.758454+0000","last_change":"2024-04-24T23:05:33.431829+0000","last_active":"2024-04-24T23:05:35.758454+0000","last_peered":"2024-04-24T23:05:35.758454+0000","last_clean":"2024-04-24T23:05:35.758454+0000","last_became_active":"2024-04-24T23:05:33.431554+0000","last_became_peered":"2024-04-24T23:05:33.431554+0000","last_unstale":"2024-04-24T23:05:35.758454+0000","last_undegraded":"2024-04-24T23:05:35.758454+0000","last_fullsized":"2024-04-24T23:05:35.758454+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:04:17.756802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":406,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:36.754513+0000","last_change":"2024-04-24T22:58:55.138810+0000","last_active":"2024-04-24T23:05:36.754513+0000","last_peered":"2024-04-24T23:05:36.754513+0000","last_clean":"2024-04-24T23:05:36.754513+0000","last_became_active":"2024-04-24T22:58:55.138627+0000","last_became_peered":"2024-04-24T22:58:55.138627+0000","last_unstale":"2024-04-24T23:05:36.754513+0000","last_undegraded":"2024-04-24T23:05:36.754513+0000","last_fullsized":"2024-04-24T23:05:36.754513+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_clean_scrub_stamp":"2024-04-24T22:58:54.127235+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:51.383182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574931,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":761,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194551,"statfs":{"total":200000000,"available":199220498,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574931,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":759,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194552,"statfs":{"total":200000000,"available":199222234,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574931,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1336,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193976,"statfs":{"total":200000000,"available":198631774,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574931,"num_pgs":33,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":761,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194551,"statfs":{"total":200000000,"available":199220532,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574931,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":759,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194552,"statfs":{"total":200000000,"available":199222234,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574931,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":760,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194552,"statfs":{"total":200000000,"available":199221596,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1329,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193982,"statfs":{"total":200000000,"available":198638166,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574931,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":760,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194552,"statfs":{"total":200000000,"available":199221582,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-24T23:05:37.521 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:05:37.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.623+0000 7f56e1f3b640 1 Processor -- start 2024-04-24T23:05:37.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.623+0000 7f56e1f3b640 1 -- start start 2024-04-24T23:05:37.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.624+0000 7f56e1f3b640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f56dc162320 0x7f56dc0ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:37.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.624+0000 7f56e1f3b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56dc0cd270 con 0x7f56dc0b5af0 2024-04-24T23:05:37.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.624+0000 7f56e1f3b640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56dc071610 con 0x7f56dc162e10 2024-04-24T23:05:37.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.624+0000 7f56e1f3b640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f56dc0b5660 con 0x7f56dc162320 2024-04-24T23:05:37.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.624+0000 7f56daffd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f56dc0b5af0 0x7f56dc162700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37820/0 (socket says 172.21.15.32:37820) 2024-04-24T23:05:37.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.624+0000 7f56daffd640 1 -- 172.21.15.32:0/2059816134 learned_addr learned my addr 172.21.15.32:0/2059816134 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:05:37.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.624+0000 7f56db7fe640 1 --2- 172.21.15.32:0/2059816134 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f56dc162320 0x7f56dc0ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:37.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.624+0000 7f56da7fc640 1 -- 172.21.15.32:0/2059816134 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1190131502 0 0) 0x7f56dc0cd270 con 0x7f56dc0b5af0 2024-04-24T23:05:37.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.624+0000 7f56da7fc640 1 -- 172.21.15.32:0/2059816134 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f56c4003520 con 0x7f56dc0b5af0 2024-04-24T23:05:37.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.624+0000 7f56da7fc640 1 -- 172.21.15.32:0/2059816134 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3600825481 0 0) 0x7f56dc071610 con 0x7f56dc162e10 2024-04-24T23:05:37.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.624+0000 7f56da7fc640 1 -- 172.21.15.32:0/2059816134 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f56dc0cd270 con 0x7f56dc162e10 2024-04-24T23:05:37.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.624+0000 7f56da7fc640 1 -- 172.21.15.32:0/2059816134 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1996638882 0 0) 0x7f56c4003520 con 0x7f56dc0b5af0 2024-04-24T23:05:37.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.624+0000 7f56da7fc640 1 -- 172.21.15.32:0/2059816134 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f56dc162320 msgr2=0x7f56dc0ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:37.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.624+0000 7f56da7fc640 1 --2- 172.21.15.32:0/2059816134 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f56dc162320 0x7f56dc0ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:37.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.624+0000 7f56da7fc640 1 -- 172.21.15.32:0/2059816134 >> v1:172.21.15.121:6789/0 conn(0x7f56dc162e10 legacy=0x7f56dc15ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:37.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.624+0000 7f56da7fc640 1 -- 172.21.15.32:0/2059816134 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56dc0718f0 con 0x7f56dc0b5af0 2024-04-24T23:05:37.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.624+0000 7f56da7fc640 1 -- 172.21.15.32:0/2059816134 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f56d0001670 con 0x7f56dc0b5af0 2024-04-24T23:05:37.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.624+0000 7f56da7fc640 1 -- 172.21.15.32:0/2059816134 >> v1:172.21.15.32:6789/0 conn(0x7f56dc0b5af0 legacy=0x7f56dc162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:37.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.624+0000 7f56da7fc640 1 --2- 172.21.15.32:0/2059816134 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f56dc162320 0x7f56c4003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:37.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.624+0000 7f56da7fc640 1 --2- 172.21.15.32:0/2059816134 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f56c40040d0 0x7f56c4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:37.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.624+0000 7f56da7fc640 1 --2- 172.21.15.32:0/2059816134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56c4004d40 0x7f56c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:37.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.624+0000 7f56da7fc640 1 -- 172.21.15.32:0/2059816134 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f56dc0b5660 con 0x7f56c4004d40 2024-04-24T23:05:37.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.624+0000 7f56da7fc640 1 -- 172.21.15.32:0/2059816134 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f56c400bb50 con 0x7f56c40040d0 2024-04-24T23:05:37.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.624+0000 7f56da7fc640 1 -- 172.21.15.32:0/2059816134 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f56c400bd20 con 0x7f56dc162320 2024-04-24T23:05:37.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.625+0000 7f56dbfff640 1 --2- 172.21.15.32:0/2059816134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56c4004d40 0x7f56c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:37.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.625+0000 7f56daffd640 1 --2- 172.21.15.32:0/2059816134 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f56c40040d0 0x7f56c4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:37.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.625+0000 7f56db7fe640 1 --2- 172.21.15.32:0/2059816134 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f56dc162320 0x7f56c4003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:37.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.625+0000 7f56dbfff640 1 -- 172.21.15.32:0/2059816134 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f56dc162320 msgr2=0x7f56c4003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:37.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.625+0000 7f56dbfff640 1 --2- 172.21.15.32:0/2059816134 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f56dc162320 0x7f56c4003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:37.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.625+0000 7f56dbfff640 1 -- 172.21.15.32:0/2059816134 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f56c40040d0 msgr2=0x7f56c4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:37.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.625+0000 7f56dbfff640 1 --2- 172.21.15.32:0/2059816134 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f56c40040d0 0x7f56c4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:37.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.625+0000 7f56dbfff640 1 -- 172.21.15.32:0/2059816134 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56c4003520 con 0x7f56c4004d40 2024-04-24T23:05:37.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.625+0000 7f56db7fe640 1 --2- 172.21.15.32:0/2059816134 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f56dc162320 0x7f56c4003b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:05:37.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.626+0000 7f56dbfff640 1 --2- 172.21.15.32:0/2059816134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56c4004d40 0x7f56c40051c0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f56d4001fd0 tx=0x7f56c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91a9b7be7417f986 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:37.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.626+0000 7f56da7fc640 1 -- 172.21.15.32:0/2059816134 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d4001bf0 con 0x7f56c4004d40 2024-04-24T23:05:37.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.626+0000 7f56da7fc640 1 -- 172.21.15.32:0/2059816134 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56d4003b70 con 0x7f56c4004d40 2024-04-24T23:05:37.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.626+0000 7f56da7fc640 1 -- 172.21.15.32:0/2059816134 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d40050d0 con 0x7f56c4004d40 2024-04-24T23:05:37.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.626+0000 7f56e1f3b640 1 -- 172.21.15.32:0/2059816134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56c4004d40 msgr2=0x7f56c40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:37.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.626+0000 7f56e1f3b640 1 --2- 172.21.15.32:0/2059816134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56c4004d40 0x7f56c40051c0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f56d4001fd0 tx=0x7f56c4005830 comp rx=0 tx=0).stop 2024-04-24T23:05:37.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.626+0000 7f56dbfff640 1 -- 172.21.15.32:0/2059816134 reap_dead start 2024-04-24T23:05:37.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.626+0000 7f56e1f3b640 1 -- 172.21.15.32:0/2059816134 shutdown_connections 2024-04-24T23:05:37.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.626+0000 7f56e1f3b640 1 -- 172.21.15.32:0/2059816134 >> 172.21.15.32:0/2059816134 conn(0x7f56dc0d0ae0 msgr2=0x7f56dc0d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:37.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.626+0000 7f56e1f3b640 1 -- 172.21.15.32:0/2059816134 shutdown_connections 2024-04-24T23:05:37.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.626+0000 7f56e1f3b640 1 -- 172.21.15.32:0/2059816134 wait complete. 2024-04-24T23:05:37.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.628+0000 7f56e1f3b640 1 Processor -- start 2024-04-24T23:05:37.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.628+0000 7f56e1f3b640 1 -- start start 2024-04-24T23:05:37.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.628+0000 7f56e1f3b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56dc146410 0x7f56dc0cf090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:37.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.628+0000 7f56e1f3b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f56dc0cf5d0 0x7f56dc06dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:37.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.628+0000 7f56e1f3b640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f56dc1467f0 0x7f56dc06e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:37.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.628+0000 7f56e1f3b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f56dc071bd0 con 0x7f56dc146410 2024-04-24T23:05:37.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.628+0000 7f56e1f3b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f56dc067950 con 0x7f56dc0cf5d0 2024-04-24T23:05:37.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.628+0000 7f56e1f3b640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f56dc071d50 con 0x7f56dc1467f0 2024-04-24T23:05:37.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.629+0000 7f56db7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56dc146410 0x7f56dc0cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:37.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.629+0000 7f56db7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56dc146410 0x7f56dc0cf090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56384/0 (socket says 172.21.15.32:56384) 2024-04-24T23:05:37.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.629+0000 7f56db7fe640 1 -- 172.21.15.32:0/2849242164 learned_addr learned my addr 172.21.15.32:0/2849242164 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:37.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.629+0000 7f56daffd640 1 --2- 172.21.15.32:0/2849242164 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f56dc0cf5d0 0x7f56dc06dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:37.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.629+0000 7f56dbfff640 1 --2- 172.21.15.32:0/2849242164 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f56dc1467f0 0x7f56dc06e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:37.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.629+0000 7f56db7fe640 1 -- 172.21.15.32:0/2849242164 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f56dc1467f0 msgr2=0x7f56dc06e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:37.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.629+0000 7f56db7fe640 1 --2- 172.21.15.32:0/2849242164 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f56dc1467f0 0x7f56dc06e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:37.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.629+0000 7f56db7fe640 1 -- 172.21.15.32:0/2849242164 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f56dc0cf5d0 msgr2=0x7f56dc06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:37.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.629+0000 7f56db7fe640 1 --2- 172.21.15.32:0/2849242164 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f56dc0cf5d0 0x7f56dc06dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:37.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.629+0000 7f56db7fe640 1 -- 172.21.15.32:0/2849242164 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56dc0cd670 con 0x7f56dc146410 2024-04-24T23:05:37.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.629+0000 7f56dbfff640 1 --2- 172.21.15.32:0/2849242164 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f56dc1467f0 0x7f56dc06e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:05:37.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.629+0000 7f56db7fe640 1 --2- 172.21.15.32:0/2849242164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56dc146410 0x7f56dc0cf090 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f56cc011ca0 tx=0x7f56cc00f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:37.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.629+0000 7f56c3fff640 1 -- 172.21.15.32:0/2849242164 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56cc00fd20 con 0x7f56dc146410 2024-04-24T23:05:37.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.629+0000 7f56c3fff640 1 -- 172.21.15.32:0/2849242164 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56cc014440 con 0x7f56dc146410 2024-04-24T23:05:37.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.629+0000 7f56e1f3b640 1 -- 172.21.15.32:0/2849242164 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56dc0cd950 con 0x7f56dc146410 2024-04-24T23:05:37.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.629+0000 7f56c3fff640 1 -- 172.21.15.32:0/2849242164 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56cc014830 con 0x7f56dc146410 2024-04-24T23:05:37.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.629+0000 7f56e1f3b640 1 -- 172.21.15.32:0/2849242164 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56dc0cdf10 con 0x7f56dc146410 2024-04-24T23:05:37.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.630+0000 7f56e1f3b640 1 -- 172.21.15.32:0/2849242164 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56dc068430 con 0x7f56dc146410 2024-04-24T23:05:37.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.631+0000 7f56c3fff640 1 -- 172.21.15.32:0/2849242164 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f56cc012040 con 0x7f56dc146410 2024-04-24T23:05:37.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.631+0000 7f56c3fff640 1 --2- 172.21.15.32:0/2849242164 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f56c806be60 0x7f56c806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:37.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.631+0000 7f56c3fff640 1 -- 172.21.15.32:0/2849242164 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5709+0+0 (secure 0 0 0) 0x7f56cc08b710 con 0x7f56dc146410 2024-04-24T23:05:37.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.631+0000 7f56daffd640 1 --2- 172.21.15.32:0/2849242164 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f56c806be60 0x7f56c806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:37.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.632+0000 7f56daffd640 1 --2- 172.21.15.32:0/2849242164 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f56c806be60 0x7f56c806e320 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f56dc0cf9f0 tx=0x7f56d0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:37.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.635+0000 7f56c3fff640 1 -- 172.21.15.32:0/2849242164 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f56cc019030 con 0x7f56dc146410 2024-04-24T23:05:37.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.816+0000 7f56e1f3b640 1 -- 172.21.15.32:0/2849242164 --> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f56dc067730 con 0x7f56c806be60 2024-04-24T23:05:37.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.822+0000 7f56c3fff640 1 -- 172.21.15.32:0/2849242164 <== mgr.4108 v2:172.21.15.121:6832/1848157943 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247992 (secure 0 0 0) 0x7f56dc067730 con 0x7f56c806be60 2024-04-24T23:05:37.826 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:05:37.828 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-24T23:05:37.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.825+0000 7f56e1f3b640 1 -- 172.21.15.32:0/2849242164 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f56c806be60 msgr2=0x7f56c806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:37.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.825+0000 7f56e1f3b640 1 --2- 172.21.15.32:0/2849242164 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f56c806be60 0x7f56c806e320 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f56dc0cf9f0 tx=0x7f56d0009040 comp rx=0 tx=0).stop 2024-04-24T23:05:37.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.825+0000 7f56e1f3b640 1 -- 172.21.15.32:0/2849242164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56dc146410 msgr2=0x7f56dc0cf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:37.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.825+0000 7f56e1f3b640 1 --2- 172.21.15.32:0/2849242164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56dc146410 0x7f56dc0cf090 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f56cc011ca0 tx=0x7f56cc00f610 comp rx=0 tx=0).stop 2024-04-24T23:05:37.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.825+0000 7f56e1f3b640 1 -- 172.21.15.32:0/2849242164 shutdown_connections 2024-04-24T23:05:37.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.825+0000 7f56e1f3b640 1 --2- 172.21.15.32:0/2849242164 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f56c806be60 0x7f56c806e320 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:37.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.825+0000 7f56e1f3b640 1 --2- 172.21.15.32:0/2849242164 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f56dc1467f0 0x7f56dc06e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:37.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.825+0000 7f56e1f3b640 1 --2- 172.21.15.32:0/2849242164 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f56dc0cf5d0 0x7f56dc06dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:37.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.825+0000 7f56e1f3b640 1 --2- 172.21.15.32:0/2849242164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56dc146410 0x7f56dc0cf090 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:37.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.825+0000 7f56e1f3b640 1 -- 172.21.15.32:0/2849242164 >> 172.21.15.32:0/2849242164 conn(0x7f56dc0d0ae0 msgr2=0x7f56dc0af5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:37.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.825+0000 7f56e1f3b640 1 -- 172.21.15.32:0/2849242164 shutdown_connections 2024-04-24T23:05:37.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.826+0000 7f56e1f3b640 1 -- 172.21.15.32:0/2849242164 wait complete. 2024-04-24T23:05:37.838 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":334,"stamp":"2024-04-24T23:05:37.218178+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":164,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":7227,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555272,"statfs":{"total":1600000000,"available":1592598616,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.802922"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.422504+0000","last_change":"2024-04-24T23:05:34.435206+0000","last_active":"2024-04-24T23:05:35.422504+0000","last_peered":"2024-04-24T23:05:35.422504+0000","last_clean":"2024-04-24T23:05:35.422504+0000","last_became_active":"2024-04-24T23:05:34.434604+0000","last_became_peered":"2024-04-24T23:05:34.434604+0000","last_unstale":"2024-04-24T23:05:35.422504+0000","last_undegraded":"2024-04-24T23:05:35.422504+0000","last_fullsized":"2024-04-24T23:05:35.422504+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:25:11.877792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.758213+0000","last_change":"2024-04-24T23:05:34.436043+0000","last_active":"2024-04-24T23:05:35.758213+0000","last_peered":"2024-04-24T23:05:35.758213+0000","last_clean":"2024-04-24T23:05:35.758213+0000","last_became_active":"2024-04-24T23:05:34.435657+0000","last_became_peered":"2024-04-24T23:05:34.435657+0000","last_unstale":"2024-04-24T23:05:35.758213+0000","last_undegraded":"2024-04-24T23:05:35.758213+0000","last_fullsized":"2024-04-24T23:05:35.758213+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:10:23.227041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:36.754419+0000","last_change":"2024-04-24T23:05:34.438801+0000","last_active":"2024-04-24T23:05:36.754419+0000","last_peered":"2024-04-24T23:05:36.754419+0000","last_clean":"2024-04-24T23:05:36.754419+0000","last_became_active":"2024-04-24T23:05:34.438543+0000","last_became_peered":"2024-04-24T23:05:34.438543+0000","last_unstale":"2024-04-24T23:05:36.754419+0000","last_undegraded":"2024-04-24T23:05:36.754419+0000","last_fullsized":"2024-04-24T23:05:36.754419+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:14:57.282009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.422399+0000","last_change":"2024-04-24T23:05:34.433100+0000","last_active":"2024-04-24T23:05:35.422399+0000","last_peered":"2024-04-24T23:05:35.422399+0000","last_clean":"2024-04-24T23:05:35.422399+0000","last_became_active":"2024-04-24T23:05:34.432382+0000","last_became_peered":"2024-04-24T23:05:34.432382+0000","last_unstale":"2024-04-24T23:05:35.422399+0000","last_undegraded":"2024-04-24T23:05:35.422399+0000","last_fullsized":"2024-04-24T23:05:35.422399+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:09:12.205245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.542410+0000","last_change":"2024-04-24T23:05:34.439317+0000","last_active":"2024-04-24T23:05:35.542410+0000","last_peered":"2024-04-24T23:05:35.542410+0000","last_clean":"2024-04-24T23:05:35.542410+0000","last_became_active":"2024-04-24T23:05:34.438822+0000","last_became_peered":"2024-04-24T23:05:34.438822+0000","last_unstale":"2024-04-24T23:05:35.542410+0000","last_undegraded":"2024-04-24T23:05:35.542410+0000","last_fullsized":"2024-04-24T23:05:35.542410+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:38:50.873838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.758398+0000","last_change":"2024-04-24T23:05:34.427217+0000","last_active":"2024-04-24T23:05:35.758398+0000","last_peered":"2024-04-24T23:05:35.758398+0000","last_clean":"2024-04-24T23:05:35.758398+0000","last_became_active":"2024-04-24T23:05:34.426956+0000","last_became_peered":"2024-04-24T23:05:34.426956+0000","last_unstale":"2024-04-24T23:05:35.758398+0000","last_undegraded":"2024-04-24T23:05:35.758398+0000","last_fullsized":"2024-04-24T23:05:35.758398+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:05:15.703627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.421519+0000","last_change":"2024-04-24T23:05:34.428003+0000","last_active":"2024-04-24T23:05:35.421519+0000","last_peered":"2024-04-24T23:05:35.421519+0000","last_clean":"2024-04-24T23:05:35.421519+0000","last_became_active":"2024-04-24T23:05:34.427722+0000","last_became_peered":"2024-04-24T23:05:34.427722+0000","last_unstale":"2024-04-24T23:05:35.421519+0000","last_undegraded":"2024-04-24T23:05:35.421519+0000","last_fullsized":"2024-04-24T23:05:35.421519+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:53:20.339099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.421479+0000","last_change":"2024-04-24T23:05:34.429921+0000","last_active":"2024-04-24T23:05:35.421479+0000","last_peered":"2024-04-24T23:05:35.421479+0000","last_clean":"2024-04-24T23:05:35.421479+0000","last_became_active":"2024-04-24T23:05:34.429682+0000","last_became_peered":"2024-04-24T23:05:34.429682+0000","last_unstale":"2024-04-24T23:05:35.421479+0000","last_undegraded":"2024-04-24T23:05:35.421479+0000","last_fullsized":"2024-04-24T23:05:35.421479+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:14:53.789433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.422242+0000","last_change":"2024-04-24T23:05:34.436440+0000","last_active":"2024-04-24T23:05:35.422242+0000","last_peered":"2024-04-24T23:05:35.422242+0000","last_clean":"2024-04-24T23:05:35.422242+0000","last_became_active":"2024-04-24T23:05:34.436191+0000","last_became_peered":"2024-04-24T23:05:34.436191+0000","last_unstale":"2024-04-24T23:05:35.422242+0000","last_undegraded":"2024-04-24T23:05:35.422242+0000","last_fullsized":"2024-04-24T23:05:35.422242+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:35.460052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.542230+0000","last_change":"2024-04-24T23:05:34.434349+0000","last_active":"2024-04-24T23:05:35.542230+0000","last_peered":"2024-04-24T23:05:35.542230+0000","last_clean":"2024-04-24T23:05:35.542230+0000","last_became_active":"2024-04-24T23:05:34.433780+0000","last_became_peered":"2024-04-24T23:05:34.433780+0000","last_unstale":"2024-04-24T23:05:35.542230+0000","last_undegraded":"2024-04-24T23:05:35.542230+0000","last_fullsized":"2024-04-24T23:05:35.542230+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:51.615533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.421588+0000","last_change":"2024-04-24T23:05:34.428833+0000","last_active":"2024-04-24T23:05:35.421588+0000","last_peered":"2024-04-24T23:05:35.421588+0000","last_clean":"2024-04-24T23:05:35.421588+0000","last_became_active":"2024-04-24T23:05:34.428559+0000","last_became_peered":"2024-04-24T23:05:34.428559+0000","last_unstale":"2024-04-24T23:05:35.421588+0000","last_undegraded":"2024-04-24T23:05:35.421588+0000","last_fullsized":"2024-04-24T23:05:35.421588+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:02.530795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.541681+0000","last_change":"2024-04-24T23:05:34.433862+0000","last_active":"2024-04-24T23:05:35.541681+0000","last_peered":"2024-04-24T23:05:35.541681+0000","last_clean":"2024-04-24T23:05:35.541681+0000","last_became_active":"2024-04-24T23:05:34.433627+0000","last_became_peered":"2024-04-24T23:05:34.433627+0000","last_unstale":"2024-04-24T23:05:35.541681+0000","last_undegraded":"2024-04-24T23:05:35.541681+0000","last_fullsized":"2024-04-24T23:05:35.541681+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:12:02.947338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.758309+0000","last_change":"2024-04-24T23:05:34.434265+0000","last_active":"2024-04-24T23:05:35.758309+0000","last_peered":"2024-04-24T23:05:35.758309+0000","last_clean":"2024-04-24T23:05:35.758309+0000","last_became_active":"2024-04-24T23:05:34.434021+0000","last_became_peered":"2024-04-24T23:05:34.434021+0000","last_unstale":"2024-04-24T23:05:35.758309+0000","last_undegraded":"2024-04-24T23:05:35.758309+0000","last_fullsized":"2024-04-24T23:05:35.758309+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:47:35.322597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:36.536713+0000","last_change":"2024-04-24T23:05:34.428147+0000","last_active":"2024-04-24T23:05:36.536713+0000","last_peered":"2024-04-24T23:05:36.536713+0000","last_clean":"2024-04-24T23:05:36.536713+0000","last_became_active":"2024-04-24T23:05:34.427234+0000","last_became_peered":"2024-04-24T23:05:34.427234+0000","last_unstale":"2024-04-24T23:05:36.536713+0000","last_undegraded":"2024-04-24T23:05:36.536713+0000","last_fullsized":"2024-04-24T23:05:36.536713+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:01.491784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.422580+0000","last_change":"2024-04-24T23:05:34.436175+0000","last_active":"2024-04-24T23:05:35.422580+0000","last_peered":"2024-04-24T23:05:35.422580+0000","last_clean":"2024-04-24T23:05:35.422580+0000","last_became_active":"2024-04-24T23:05:34.435595+0000","last_became_peered":"2024-04-24T23:05:34.435595+0000","last_unstale":"2024-04-24T23:05:35.422580+0000","last_undegraded":"2024-04-24T23:05:35.422580+0000","last_fullsized":"2024-04-24T23:05:35.422580+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:58:58.766747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.542062+0000","last_change":"2024-04-24T23:05:34.438708+0000","last_active":"2024-04-24T23:05:35.542062+0000","last_peered":"2024-04-24T23:05:35.542062+0000","last_clean":"2024-04-24T23:05:35.542062+0000","last_became_active":"2024-04-24T23:05:34.438477+0000","last_became_peered":"2024-04-24T23:05:34.438477+0000","last_unstale":"2024-04-24T23:05:35.542062+0000","last_undegraded":"2024-04-24T23:05:35.542062+0000","last_fullsized":"2024-04-24T23:05:35.542062+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:50:24.587596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:36.754183+0000","last_change":"2024-04-24T23:05:34.433335+0000","last_active":"2024-04-24T23:05:36.754183+0000","last_peered":"2024-04-24T23:05:36.754183+0000","last_clean":"2024-04-24T23:05:36.754183+0000","last_became_active":"2024-04-24T23:05:34.433073+0000","last_became_peered":"2024-04-24T23:05:34.433073+0000","last_unstale":"2024-04-24T23:05:36.754183+0000","last_undegraded":"2024-04-24T23:05:36.754183+0000","last_fullsized":"2024-04-24T23:05:36.754183+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:19:30.205516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.421646+0000","last_change":"2024-04-24T23:05:34.432655+0000","last_active":"2024-04-24T23:05:35.421646+0000","last_peered":"2024-04-24T23:05:35.421646+0000","last_clean":"2024-04-24T23:05:35.421646+0000","last_became_active":"2024-04-24T23:05:34.432400+0000","last_became_peered":"2024-04-24T23:05:34.432400+0000","last_unstale":"2024-04-24T23:05:35.421646+0000","last_undegraded":"2024-04-24T23:05:35.421646+0000","last_fullsized":"2024-04-24T23:05:35.421646+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:35:38.814104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:36.754209+0000","last_change":"2024-04-24T23:05:34.437757+0000","last_active":"2024-04-24T23:05:36.754209+0000","last_peered":"2024-04-24T23:05:36.754209+0000","last_clean":"2024-04-24T23:05:36.754209+0000","last_became_active":"2024-04-24T23:05:34.437508+0000","last_became_peered":"2024-04-24T23:05:34.437508+0000","last_unstale":"2024-04-24T23:05:36.754209+0000","last_undegraded":"2024-04-24T23:05:36.754209+0000","last_fullsized":"2024-04-24T23:05:36.754209+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:49.557955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:36.754278+0000","last_change":"2024-04-24T23:05:34.442202+0000","last_active":"2024-04-24T23:05:36.754278+0000","last_peered":"2024-04-24T23:05:36.754278+0000","last_clean":"2024-04-24T23:05:36.754278+0000","last_became_active":"2024-04-24T23:05:34.441949+0000","last_became_peered":"2024-04-24T23:05:34.441949+0000","last_unstale":"2024-04-24T23:05:36.754278+0000","last_undegraded":"2024-04-24T23:05:36.754278+0000","last_fullsized":"2024-04-24T23:05:36.754278+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:44:19.263757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.421746+0000","last_change":"2024-04-24T23:05:34.431325+0000","last_active":"2024-04-24T23:05:35.421746+0000","last_peered":"2024-04-24T23:05:35.421746+0000","last_clean":"2024-04-24T23:05:35.421746+0000","last_became_active":"2024-04-24T23:05:34.431001+0000","last_became_peered":"2024-04-24T23:05:34.431001+0000","last_unstale":"2024-04-24T23:05:35.421746+0000","last_undegraded":"2024-04-24T23:05:35.421746+0000","last_fullsized":"2024-04-24T23:05:35.421746+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:53:27.345860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.758401+0000","last_change":"2024-04-24T23:05:34.427574+0000","last_active":"2024-04-24T23:05:35.758401+0000","last_peered":"2024-04-24T23:05:35.758401+0000","last_clean":"2024-04-24T23:05:35.758401+0000","last_became_active":"2024-04-24T23:05:34.427305+0000","last_became_peered":"2024-04-24T23:05:34.427305+0000","last_unstale":"2024-04-24T23:05:35.758401+0000","last_undegraded":"2024-04-24T23:05:35.758401+0000","last_fullsized":"2024-04-24T23:05:35.758401+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:25.951310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:36.536716+0000","last_change":"2024-04-24T23:05:34.431114+0000","last_active":"2024-04-24T23:05:36.536716+0000","last_peered":"2024-04-24T23:05:36.536716+0000","last_clean":"2024-04-24T23:05:36.536716+0000","last_became_active":"2024-04-24T23:05:34.430526+0000","last_became_peered":"2024-04-24T23:05:34.430526+0000","last_unstale":"2024-04-24T23:05:36.536716+0000","last_undegraded":"2024-04-24T23:05:36.536716+0000","last_fullsized":"2024-04-24T23:05:36.536716+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:56:41.938698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.422541+0000","last_change":"2024-04-24T23:05:34.437889+0000","last_active":"2024-04-24T23:05:35.422541+0000","last_peered":"2024-04-24T23:05:35.422541+0000","last_clean":"2024-04-24T23:05:35.422541+0000","last_became_active":"2024-04-24T23:05:34.437525+0000","last_became_peered":"2024-04-24T23:05:34.437525+0000","last_unstale":"2024-04-24T23:05:35.422541+0000","last_undegraded":"2024-04-24T23:05:35.422541+0000","last_fullsized":"2024-04-24T23:05:35.422541+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:31.864268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.421490+0000","last_change":"2024-04-24T23:05:34.436354+0000","last_active":"2024-04-24T23:05:35.421490+0000","last_peered":"2024-04-24T23:05:35.421490+0000","last_clean":"2024-04-24T23:05:35.421490+0000","last_became_active":"2024-04-24T23:05:34.435882+0000","last_became_peered":"2024-04-24T23:05:34.435882+0000","last_unstale":"2024-04-24T23:05:35.421490+0000","last_undegraded":"2024-04-24T23:05:35.421490+0000","last_fullsized":"2024-04-24T23:05:35.421490+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:04.009068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.758737+0000","last_change":"2024-04-24T23:05:34.433774+0000","last_active":"2024-04-24T23:05:35.758737+0000","last_peered":"2024-04-24T23:05:35.758737+0000","last_clean":"2024-04-24T23:05:35.758737+0000","last_became_active":"2024-04-24T23:05:34.433535+0000","last_became_peered":"2024-04-24T23:05:34.433535+0000","last_unstale":"2024-04-24T23:05:35.758737+0000","last_undegraded":"2024-04-24T23:05:35.758737+0000","last_fullsized":"2024-04-24T23:05:35.758737+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:27:51.152470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.423310+0000","last_change":"2024-04-24T23:05:34.435513+0000","last_active":"2024-04-24T23:05:35.423310+0000","last_peered":"2024-04-24T23:05:35.423310+0000","last_clean":"2024-04-24T23:05:35.423310+0000","last_became_active":"2024-04-24T23:05:34.435209+0000","last_became_peered":"2024-04-24T23:05:34.435209+0000","last_unstale":"2024-04-24T23:05:35.423310+0000","last_undegraded":"2024-04-24T23:05:35.423310+0000","last_fullsized":"2024-04-24T23:05:35.423310+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:10:55.262208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.758382+0000","last_change":"2024-04-24T23:05:34.428127+0000","last_active":"2024-04-24T23:05:35.758382+0000","last_peered":"2024-04-24T23:05:35.758382+0000","last_clean":"2024-04-24T23:05:35.758382+0000","last_became_active":"2024-04-24T23:05:34.427831+0000","last_became_peered":"2024-04-24T23:05:34.427831+0000","last_unstale":"2024-04-24T23:05:35.758382+0000","last_undegraded":"2024-04-24T23:05:35.758382+0000","last_fullsized":"2024-04-24T23:05:35.758382+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:52:49.899040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:36.536887+0000","last_change":"2024-04-24T23:05:34.428336+0000","last_active":"2024-04-24T23:05:36.536887+0000","last_peered":"2024-04-24T23:05:36.536887+0000","last_clean":"2024-04-24T23:05:36.536887+0000","last_became_active":"2024-04-24T23:05:34.427582+0000","last_became_peered":"2024-04-24T23:05:34.427582+0000","last_unstale":"2024-04-24T23:05:36.536887+0000","last_undegraded":"2024-04-24T23:05:36.536887+0000","last_fullsized":"2024-04-24T23:05:36.536887+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:51:15.742586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.421648+0000","last_change":"2024-04-24T23:05:34.436116+0000","last_active":"2024-04-24T23:05:35.421648+0000","last_peered":"2024-04-24T23:05:35.421648+0000","last_clean":"2024-04-24T23:05:35.421648+0000","last_became_active":"2024-04-24T23:05:34.435569+0000","last_became_peered":"2024-04-24T23:05:34.435569+0000","last_unstale":"2024-04-24T23:05:35.421648+0000","last_undegraded":"2024-04-24T23:05:35.421648+0000","last_fullsized":"2024-04-24T23:05:35.421648+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:43.290722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.422679+0000","last_change":"2024-04-24T23:05:34.435453+0000","last_active":"2024-04-24T23:05:35.422679+0000","last_peered":"2024-04-24T23:05:35.422679+0000","last_clean":"2024-04-24T23:05:35.422679+0000","last_became_active":"2024-04-24T23:05:34.434970+0000","last_became_peered":"2024-04-24T23:05:34.434970+0000","last_unstale":"2024-04-24T23:05:35.422679+0000","last_undegraded":"2024-04-24T23:05:35.422679+0000","last_fullsized":"2024-04-24T23:05:35.422679+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:25.877884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.542003+0000","last_change":"2024-04-24T23:05:34.441368+0000","last_active":"2024-04-24T23:05:35.542003+0000","last_peered":"2024-04-24T23:05:35.542003+0000","last_clean":"2024-04-24T23:05:35.542003+0000","last_became_active":"2024-04-24T23:05:34.441136+0000","last_became_peered":"2024-04-24T23:05:34.441136+0000","last_unstale":"2024-04-24T23:05:35.542003+0000","last_undegraded":"2024-04-24T23:05:35.542003+0000","last_fullsized":"2024-04-24T23:05:35.542003+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:39.837776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.422411+0000","last_change":"2024-04-24T23:05:34.439810+0000","last_active":"2024-04-24T23:05:35.422411+0000","last_peered":"2024-04-24T23:05:35.422411+0000","last_clean":"2024-04-24T23:05:35.422411+0000","last_became_active":"2024-04-24T23:05:34.439092+0000","last_became_peered":"2024-04-24T23:05:34.439092+0000","last_unstale":"2024-04-24T23:05:35.422411+0000","last_undegraded":"2024-04-24T23:05:35.422411+0000","last_fullsized":"2024-04-24T23:05:35.422411+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:24:11.127522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"139'4","reported_seq":18,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:36.537181+0000","last_change":"2024-04-24T23:05:33.424762+0000","last_active":"2024-04-24T23:05:36.537181+0000","last_peered":"2024-04-24T23:05:36.537181+0000","last_clean":"2024-04-24T23:05:36.537181+0000","last_became_active":"2024-04-24T23:05:33.424523+0000","last_became_peered":"2024-04-24T23:05:33.424523+0000","last_unstale":"2024-04-24T23:05:36.537181+0000","last_undegraded":"2024-04-24T23:05:36.537181+0000","last_fullsized":"2024-04-24T23:05:36.537181+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:28:11.678570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:36.536888+0000","last_change":"2024-04-24T23:05:34.431755+0000","last_active":"2024-04-24T23:05:36.536888+0000","last_peered":"2024-04-24T23:05:36.536888+0000","last_clean":"2024-04-24T23:05:36.536888+0000","last_became_active":"2024-04-24T23:05:34.430852+0000","last_became_peered":"2024-04-24T23:05:34.430852+0000","last_unstale":"2024-04-24T23:05:36.536888+0000","last_undegraded":"2024-04-24T23:05:36.536888+0000","last_fullsized":"2024-04-24T23:05:36.536888+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:36.981128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1e","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.758307+0000","last_change":"2024-04-24T23:05:33.422951+0000","last_active":"2024-04-24T23:05:35.758307+0000","last_peered":"2024-04-24T23:05:35.758307+0000","last_clean":"2024-04-24T23:05:35.758307+0000","last_became_active":"2024-04-24T23:05:33.422697+0000","last_became_peered":"2024-04-24T23:05:33.422697+0000","last_unstale":"2024-04-24T23:05:35.758307+0000","last_undegraded":"2024-04-24T23:05:35.758307+0000","last_fullsized":"2024-04-24T23:05:35.758307+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:56:46.572909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.758567+0000","last_change":"2024-04-24T23:05:34.436891+0000","last_active":"2024-04-24T23:05:35.758567+0000","last_peered":"2024-04-24T23:05:35.758567+0000","last_clean":"2024-04-24T23:05:35.758567+0000","last_became_active":"2024-04-24T23:05:34.436667+0000","last_became_peered":"2024-04-24T23:05:34.436667+0000","last_unstale":"2024-04-24T23:05:35.758567+0000","last_undegraded":"2024-04-24T23:05:35.758567+0000","last_fullsized":"2024-04-24T23:05:35.758567+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:13.649179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1d","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.423684+0000","last_change":"2024-04-24T23:05:33.433064+0000","last_active":"2024-04-24T23:05:35.423684+0000","last_peered":"2024-04-24T23:05:35.423684+0000","last_clean":"2024-04-24T23:05:35.423684+0000","last_became_active":"2024-04-24T23:05:33.432806+0000","last_became_peered":"2024-04-24T23:05:33.432806+0000","last_unstale":"2024-04-24T23:05:35.423684+0000","last_undegraded":"2024-04-24T23:05:35.423684+0000","last_fullsized":"2024-04-24T23:05:35.423684+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:46:24.321544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:36.754429+0000","last_change":"2024-04-24T23:05:34.440019+0000","last_active":"2024-04-24T23:05:36.754429+0000","last_peered":"2024-04-24T23:05:36.754429+0000","last_clean":"2024-04-24T23:05:36.754429+0000","last_became_active":"2024-04-24T23:05:34.439735+0000","last_became_peered":"2024-04-24T23:05:34.439735+0000","last_unstale":"2024-04-24T23:05:36.754429+0000","last_undegraded":"2024-04-24T23:05:36.754429+0000","last_fullsized":"2024-04-24T23:05:36.754429+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:46.855410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.423226+0000","last_change":"2024-04-24T23:05:33.431526+0000","last_active":"2024-04-24T23:05:35.423226+0000","last_peered":"2024-04-24T23:05:35.423226+0000","last_clean":"2024-04-24T23:05:35.423226+0000","last_became_active":"2024-04-24T23:05:33.431291+0000","last_became_peered":"2024-04-24T23:05:33.431291+0000","last_unstale":"2024-04-24T23:05:35.423226+0000","last_undegraded":"2024-04-24T23:05:35.423226+0000","last_fullsized":"2024-04-24T23:05:35.423226+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:50:04.910591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.422933+0000","last_change":"2024-04-24T23:05:34.436634+0000","last_active":"2024-04-24T23:05:35.422933+0000","last_peered":"2024-04-24T23:05:35.422933+0000","last_clean":"2024-04-24T23:05:35.422933+0000","last_became_active":"2024-04-24T23:05:34.436381+0000","last_became_peered":"2024-04-24T23:05:34.436381+0000","last_unstale":"2024-04-24T23:05:35.422933+0000","last_undegraded":"2024-04-24T23:05:35.422933+0000","last_fullsized":"2024-04-24T23:05:35.422933+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:11:55.139151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1b","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:36.754490+0000","last_change":"2024-04-24T23:05:33.426739+0000","last_active":"2024-04-24T23:05:36.754490+0000","last_peered":"2024-04-24T23:05:36.754490+0000","last_clean":"2024-04-24T23:05:36.754490+0000","last_became_active":"2024-04-24T23:05:33.426341+0000","last_became_peered":"2024-04-24T23:05:33.426341+0000","last_unstale":"2024-04-24T23:05:36.754490+0000","last_undegraded":"2024-04-24T23:05:36.754490+0000","last_fullsized":"2024-04-24T23:05:36.754490+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:48:27.126130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.421659+0000","last_change":"2024-04-24T23:05:34.435065+0000","last_active":"2024-04-24T23:05:35.421659+0000","last_peered":"2024-04-24T23:05:35.421659+0000","last_clean":"2024-04-24T23:05:35.421659+0000","last_became_active":"2024-04-24T23:05:34.434780+0000","last_became_peered":"2024-04-24T23:05:34.434780+0000","last_unstale":"2024-04-24T23:05:35.421659+0000","last_undegraded":"2024-04-24T23:05:35.421659+0000","last_fullsized":"2024-04-24T23:05:35.421659+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:51:18.245665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1a","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.422635+0000","last_change":"2024-04-24T23:05:33.427744+0000","last_active":"2024-04-24T23:05:35.422635+0000","last_peered":"2024-04-24T23:05:35.422635+0000","last_clean":"2024-04-24T23:05:35.422635+0000","last_became_active":"2024-04-24T23:05:33.427055+0000","last_became_peered":"2024-04-24T23:05:33.427055+0000","last_unstale":"2024-04-24T23:05:35.422635+0000","last_undegraded":"2024-04-24T23:05:35.422635+0000","last_fullsized":"2024-04-24T23:05:35.422635+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:23.744870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.541910+0000","last_change":"2024-04-24T23:05:34.434024+0000","last_active":"2024-04-24T23:05:35.541910+0000","last_peered":"2024-04-24T23:05:35.541910+0000","last_clean":"2024-04-24T23:05:35.541910+0000","last_became_active":"2024-04-24T23:05:34.433417+0000","last_became_peered":"2024-04-24T23:05:34.433417+0000","last_unstale":"2024-04-24T23:05:35.541910+0000","last_undegraded":"2024-04-24T23:05:35.541910+0000","last_fullsized":"2024-04-24T23:05:35.541910+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:26:37.412259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.421701+0000","last_change":"2024-04-24T23:05:33.425559+0000","last_active":"2024-04-24T23:05:35.421701+0000","last_peered":"2024-04-24T23:05:35.421701+0000","last_clean":"2024-04-24T23:05:35.421701+0000","last_became_active":"2024-04-24T23:05:33.425269+0000","last_became_peered":"2024-04-24T23:05:33.425269+0000","last_unstale":"2024-04-24T23:05:35.421701+0000","last_undegraded":"2024-04-24T23:05:35.421701+0000","last_fullsized":"2024-04-24T23:05:35.421701+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:10:01.141531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.421525+0000","last_change":"2024-04-24T23:05:34.430981+0000","last_active":"2024-04-24T23:05:35.421525+0000","last_peered":"2024-04-24T23:05:35.421525+0000","last_clean":"2024-04-24T23:05:35.421525+0000","last_became_active":"2024-04-24T23:05:34.430535+0000","last_became_peered":"2024-04-24T23:05:34.430535+0000","last_unstale":"2024-04-24T23:05:35.421525+0000","last_undegraded":"2024-04-24T23:05:35.421525+0000","last_fullsized":"2024-04-24T23:05:35.421525+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:31:35.233121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.542325+0000","last_change":"2024-04-24T23:05:33.430778+0000","last_active":"2024-04-24T23:05:35.542325+0000","last_peered":"2024-04-24T23:05:35.542325+0000","last_clean":"2024-04-24T23:05:35.542325+0000","last_became_active":"2024-04-24T23:05:33.429066+0000","last_became_peered":"2024-04-24T23:05:33.429066+0000","last_unstale":"2024-04-24T23:05:35.542325+0000","last_undegraded":"2024-04-24T23:05:35.542325+0000","last_fullsized":"2024-04-24T23:05:35.542325+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:09:05.492627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.758361+0000","last_change":"2024-04-24T23:05:34.433643+0000","last_active":"2024-04-24T23:05:35.758361+0000","last_peered":"2024-04-24T23:05:35.758361+0000","last_clean":"2024-04-24T23:05:35.758361+0000","last_became_active":"2024-04-24T23:05:34.433397+0000","last_became_peered":"2024-04-24T23:05:34.433397+0000","last_unstale":"2024-04-24T23:05:35.758361+0000","last_undegraded":"2024-04-24T23:05:35.758361+0000","last_fullsized":"2024-04-24T23:05:35.758361+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:04.560118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.423348+0000","last_change":"2024-04-24T23:05:33.430918+0000","last_active":"2024-04-24T23:05:35.423348+0000","last_peered":"2024-04-24T23:05:35.423348+0000","last_clean":"2024-04-24T23:05:35.423348+0000","last_became_active":"2024-04-24T23:05:33.430161+0000","last_became_peered":"2024-04-24T23:05:33.430161+0000","last_unstale":"2024-04-24T23:05:35.423348+0000","last_undegraded":"2024-04-24T23:05:35.423348+0000","last_fullsized":"2024-04-24T23:05:35.423348+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:12:14.042047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.758093+0000","last_change":"2024-04-24T23:05:34.434073+0000","last_active":"2024-04-24T23:05:35.758093+0000","last_peered":"2024-04-24T23:05:35.758093+0000","last_clean":"2024-04-24T23:05:35.758093+0000","last_became_active":"2024-04-24T23:05:34.433819+0000","last_became_peered":"2024-04-24T23:05:34.433819+0000","last_unstale":"2024-04-24T23:05:35.758093+0000","last_undegraded":"2024-04-24T23:05:35.758093+0000","last_fullsized":"2024-04-24T23:05:35.758093+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:16:10.393384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.542106+0000","last_change":"2024-04-24T23:05:33.424437+0000","last_active":"2024-04-24T23:05:35.542106+0000","last_peered":"2024-04-24T23:05:35.542106+0000","last_clean":"2024-04-24T23:05:35.542106+0000","last_became_active":"2024-04-24T23:05:33.424052+0000","last_became_peered":"2024-04-24T23:05:33.424052+0000","last_unstale":"2024-04-24T23:05:35.542106+0000","last_undegraded":"2024-04-24T23:05:35.542106+0000","last_fullsized":"2024-04-24T23:05:35.542106+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:03.154085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.758336+0000","last_change":"2024-04-24T23:05:34.429265+0000","last_active":"2024-04-24T23:05:35.758336+0000","last_peered":"2024-04-24T23:05:35.758336+0000","last_clean":"2024-04-24T23:05:35.758336+0000","last_became_active":"2024-04-24T23:05:34.429029+0000","last_became_peered":"2024-04-24T23:05:34.429029+0000","last_unstale":"2024-04-24T23:05:35.758336+0000","last_undegraded":"2024-04-24T23:05:35.758336+0000","last_fullsized":"2024-04-24T23:05:35.758336+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:45:45.885131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"139'3","reported_seq":17,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.541646+0000","last_change":"2024-04-24T23:05:33.431987+0000","last_active":"2024-04-24T23:05:35.541646+0000","last_peered":"2024-04-24T23:05:35.541646+0000","last_clean":"2024-04-24T23:05:35.541646+0000","last_became_active":"2024-04-24T23:05:33.431614+0000","last_became_peered":"2024-04-24T23:05:33.431614+0000","last_unstale":"2024-04-24T23:05:35.541646+0000","last_undegraded":"2024-04-24T23:05:35.541646+0000","last_fullsized":"2024-04-24T23:05:35.541646+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:57:39.840720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.421798+0000","last_change":"2024-04-24T23:05:34.436433+0000","last_active":"2024-04-24T23:05:35.421798+0000","last_peered":"2024-04-24T23:05:35.421798+0000","last_clean":"2024-04-24T23:05:35.421798+0000","last_became_active":"2024-04-24T23:05:34.436187+0000","last_became_peered":"2024-04-24T23:05:34.436187+0000","last_unstale":"2024-04-24T23:05:35.421798+0000","last_undegraded":"2024-04-24T23:05:35.421798+0000","last_fullsized":"2024-04-24T23:05:35.421798+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:44:26.418771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"139'6","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.422720+0000","last_change":"2024-04-24T23:05:33.429393+0000","last_active":"2024-04-24T23:05:35.422720+0000","last_peered":"2024-04-24T23:05:35.422720+0000","last_clean":"2024-04-24T23:05:35.422720+0000","last_became_active":"2024-04-24T23:05:33.428836+0000","last_became_peered":"2024-04-24T23:05:33.428836+0000","last_unstale":"2024-04-24T23:05:35.422720+0000","last_undegraded":"2024-04-24T23:05:35.422720+0000","last_fullsized":"2024-04-24T23:05:35.422720+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:53.541586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.541883+0000","last_change":"2024-04-24T23:05:34.438040+0000","last_active":"2024-04-24T23:05:35.541883+0000","last_peered":"2024-04-24T23:05:35.541883+0000","last_clean":"2024-04-24T23:05:35.541883+0000","last_became_active":"2024-04-24T23:05:34.437807+0000","last_became_peered":"2024-04-24T23:05:34.437807+0000","last_unstale":"2024-04-24T23:05:35.541883+0000","last_undegraded":"2024-04-24T23:05:35.541883+0000","last_fullsized":"2024-04-24T23:05:35.541883+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:30:22.392636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.542474+0000","last_change":"2024-04-24T23:05:33.429287+0000","last_active":"2024-04-24T23:05:35.542474+0000","last_peered":"2024-04-24T23:05:35.542474+0000","last_clean":"2024-04-24T23:05:35.542474+0000","last_became_active":"2024-04-24T23:05:33.428276+0000","last_became_peered":"2024-04-24T23:05:33.428276+0000","last_unstale":"2024-04-24T23:05:35.542474+0000","last_undegraded":"2024-04-24T23:05:35.542474+0000","last_fullsized":"2024-04-24T23:05:35.542474+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:43:33.806921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.423578+0000","last_change":"2024-04-24T23:05:34.435723+0000","last_active":"2024-04-24T23:05:35.423578+0000","last_peered":"2024-04-24T23:05:35.423578+0000","last_clean":"2024-04-24T23:05:35.423578+0000","last_became_active":"2024-04-24T23:05:34.434981+0000","last_became_peered":"2024-04-24T23:05:34.434981+0000","last_unstale":"2024-04-24T23:05:35.423578+0000","last_undegraded":"2024-04-24T23:05:35.423578+0000","last_fullsized":"2024-04-24T23:05:35.423578+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:09:20.362005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:36.754398+0000","last_change":"2024-04-24T23:05:33.432071+0000","last_active":"2024-04-24T23:05:36.754398+0000","last_peered":"2024-04-24T23:05:36.754398+0000","last_clean":"2024-04-24T23:05:36.754398+0000","last_became_active":"2024-04-24T23:05:33.431681+0000","last_became_peered":"2024-04-24T23:05:33.431681+0000","last_unstale":"2024-04-24T23:05:36.754398+0000","last_undegraded":"2024-04-24T23:05:36.754398+0000","last_fullsized":"2024-04-24T23:05:36.754398+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:32:00.321253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.423502+0000","last_change":"2024-04-24T23:05:34.434329+0000","last_active":"2024-04-24T23:05:35.423502+0000","last_peered":"2024-04-24T23:05:35.423502+0000","last_clean":"2024-04-24T23:05:35.423502+0000","last_became_active":"2024-04-24T23:05:34.434085+0000","last_became_peered":"2024-04-24T23:05:34.434085+0000","last_unstale":"2024-04-24T23:05:35.423502+0000","last_undegraded":"2024-04-24T23:05:35.423502+0000","last_fullsized":"2024-04-24T23:05:35.423502+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:59:23.738713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"139'8","reported_seq":22,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.422868+0000","last_change":"2024-04-24T23:05:33.429042+0000","last_active":"2024-04-24T23:05:35.422868+0000","last_peered":"2024-04-24T23:05:35.422868+0000","last_clean":"2024-04-24T23:05:35.422868+0000","last_became_active":"2024-04-24T23:05:33.428789+0000","last_became_peered":"2024-04-24T23:05:33.428789+0000","last_unstale":"2024-04-24T23:05:35.422868+0000","last_undegraded":"2024-04-24T23:05:35.422868+0000","last_fullsized":"2024-04-24T23:05:35.422868+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:36.904618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.422741+0000","last_change":"2024-04-24T23:05:34.430678+0000","last_active":"2024-04-24T23:05:35.422741+0000","last_peered":"2024-04-24T23:05:35.422741+0000","last_clean":"2024-04-24T23:05:35.422741+0000","last_became_active":"2024-04-24T23:05:34.430432+0000","last_became_peered":"2024-04-24T23:05:34.430432+0000","last_unstale":"2024-04-24T23:05:35.422741+0000","last_undegraded":"2024-04-24T23:05:35.422741+0000","last_fullsized":"2024-04-24T23:05:35.422741+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:29:37.052742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.f","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.758469+0000","last_change":"2024-04-24T23:05:33.432463+0000","last_active":"2024-04-24T23:05:35.758469+0000","last_peered":"2024-04-24T23:05:35.758469+0000","last_clean":"2024-04-24T23:05:35.758469+0000","last_became_active":"2024-04-24T23:05:33.431935+0000","last_became_peered":"2024-04-24T23:05:33.431935+0000","last_unstale":"2024-04-24T23:05:35.758469+0000","last_undegraded":"2024-04-24T23:05:35.758469+0000","last_fullsized":"2024-04-24T23:05:35.758469+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:15:05.612832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.422570+0000","last_change":"2024-04-24T23:05:34.440635+0000","last_active":"2024-04-24T23:05:35.422570+0000","last_peered":"2024-04-24T23:05:35.422570+0000","last_clean":"2024-04-24T23:05:35.422570+0000","last_became_active":"2024-04-24T23:05:34.439414+0000","last_became_peered":"2024-04-24T23:05:34.439414+0000","last_unstale":"2024-04-24T23:05:35.422570+0000","last_undegraded":"2024-04-24T23:05:35.422570+0000","last_fullsized":"2024-04-24T23:05:35.422570+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:55:08.361607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.542534+0000","last_change":"2024-04-24T23:05:33.429481+0000","last_active":"2024-04-24T23:05:35.542534+0000","last_peered":"2024-04-24T23:05:35.542534+0000","last_clean":"2024-04-24T23:05:35.542534+0000","last_became_active":"2024-04-24T23:05:33.428605+0000","last_became_peered":"2024-04-24T23:05:33.428605+0000","last_unstale":"2024-04-24T23:05:35.542534+0000","last_undegraded":"2024-04-24T23:05:35.542534+0000","last_fullsized":"2024-04-24T23:05:35.542534+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:14:25.825665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.421798+0000","last_change":"2024-04-24T23:05:34.432876+0000","last_active":"2024-04-24T23:05:35.421798+0000","last_peered":"2024-04-24T23:05:35.421798+0000","last_clean":"2024-04-24T23:05:35.421798+0000","last_became_active":"2024-04-24T23:05:34.432594+0000","last_became_peered":"2024-04-24T23:05:34.432594+0000","last_unstale":"2024-04-24T23:05:35.421798+0000","last_undegraded":"2024-04-24T23:05:35.421798+0000","last_fullsized":"2024-04-24T23:05:35.421798+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:06.169592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.d","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:36.537406+0000","last_change":"2024-04-24T23:05:33.429289+0000","last_active":"2024-04-24T23:05:36.537406+0000","last_peered":"2024-04-24T23:05:36.537406+0000","last_clean":"2024-04-24T23:05:36.537406+0000","last_became_active":"2024-04-24T23:05:33.429063+0000","last_became_peered":"2024-04-24T23:05:33.429063+0000","last_unstale":"2024-04-24T23:05:36.537406+0000","last_undegraded":"2024-04-24T23:05:36.537406+0000","last_fullsized":"2024-04-24T23:05:36.537406+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:03.163984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:36.537026+0000","last_change":"2024-04-24T23:05:34.433294+0000","last_active":"2024-04-24T23:05:36.537026+0000","last_peered":"2024-04-24T23:05:36.537026+0000","last_clean":"2024-04-24T23:05:36.537026+0000","last_became_active":"2024-04-24T23:05:34.432955+0000","last_became_peered":"2024-04-24T23:05:34.432955+0000","last_unstale":"2024-04-24T23:05:36.537026+0000","last_undegraded":"2024-04-24T23:05:36.537026+0000","last_fullsized":"2024-04-24T23:05:36.537026+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:04:19.696935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:36.536929+0000","last_change":"2024-04-24T23:05:33.431817+0000","last_active":"2024-04-24T23:05:36.536929+0000","last_peered":"2024-04-24T23:05:36.536929+0000","last_clean":"2024-04-24T23:05:36.536929+0000","last_became_active":"2024-04-24T23:05:33.431569+0000","last_became_peered":"2024-04-24T23:05:33.431569+0000","last_unstale":"2024-04-24T23:05:36.536929+0000","last_undegraded":"2024-04-24T23:05:36.536929+0000","last_fullsized":"2024-04-24T23:05:36.536929+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:36.333278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:36.754382+0000","last_change":"2024-04-24T23:05:34.439443+0000","last_active":"2024-04-24T23:05:36.754382+0000","last_peered":"2024-04-24T23:05:36.754382+0000","last_clean":"2024-04-24T23:05:36.754382+0000","last_became_active":"2024-04-24T23:05:34.439154+0000","last_became_peered":"2024-04-24T23:05:34.439154+0000","last_unstale":"2024-04-24T23:05:36.754382+0000","last_undegraded":"2024-04-24T23:05:36.754382+0000","last_fullsized":"2024-04-24T23:05:36.754382+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:08:36.733577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.b","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.541764+0000","last_change":"2024-04-24T23:05:33.425107+0000","last_active":"2024-04-24T23:05:35.541764+0000","last_peered":"2024-04-24T23:05:35.541764+0000","last_clean":"2024-04-24T23:05:35.541764+0000","last_became_active":"2024-04-24T23:05:33.424855+0000","last_became_peered":"2024-04-24T23:05:33.424855+0000","last_unstale":"2024-04-24T23:05:35.541764+0000","last_undegraded":"2024-04-24T23:05:35.541764+0000","last_fullsized":"2024-04-24T23:05:35.541764+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:43:59.232440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.422778+0000","last_change":"2024-04-24T23:05:34.433609+0000","last_active":"2024-04-24T23:05:35.422778+0000","last_peered":"2024-04-24T23:05:35.422778+0000","last_clean":"2024-04-24T23:05:35.422778+0000","last_became_active":"2024-04-24T23:05:34.433208+0000","last_became_peered":"2024-04-24T23:05:34.433208+0000","last_unstale":"2024-04-24T23:05:35.422778+0000","last_undegraded":"2024-04-24T23:05:35.422778+0000","last_fullsized":"2024-04-24T23:05:35.422778+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:55:21.103335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.422673+0000","last_change":"2024-04-24T23:05:33.432951+0000","last_active":"2024-04-24T23:05:35.422673+0000","last_peered":"2024-04-24T23:05:35.422673+0000","last_clean":"2024-04-24T23:05:35.422673+0000","last_became_active":"2024-04-24T23:05:33.432689+0000","last_became_peered":"2024-04-24T23:05:33.432689+0000","last_unstale":"2024-04-24T23:05:35.422673+0000","last_undegraded":"2024-04-24T23:05:35.422673+0000","last_fullsized":"2024-04-24T23:05:35.422673+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:19:08.520909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:36.537177+0000","last_change":"2024-04-24T23:05:34.435206+0000","last_active":"2024-04-24T23:05:36.537177+0000","last_peered":"2024-04-24T23:05:36.537177+0000","last_clean":"2024-04-24T23:05:36.537177+0000","last_became_active":"2024-04-24T23:05:34.434699+0000","last_became_peered":"2024-04-24T23:05:34.434699+0000","last_unstale":"2024-04-24T23:05:36.537177+0000","last_undegraded":"2024-04-24T23:05:36.537177+0000","last_fullsized":"2024-04-24T23:05:36.537177+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:15.953037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.423780+0000","last_change":"2024-04-24T23:05:33.431825+0000","last_active":"2024-04-24T23:05:35.423780+0000","last_peered":"2024-04-24T23:05:35.423780+0000","last_clean":"2024-04-24T23:05:35.423780+0000","last_became_active":"2024-04-24T23:05:33.430954+0000","last_became_peered":"2024-04-24T23:05:33.430954+0000","last_unstale":"2024-04-24T23:05:35.423780+0000","last_undegraded":"2024-04-24T23:05:35.423780+0000","last_fullsized":"2024-04-24T23:05:35.423780+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:05:54.412042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.542075+0000","last_change":"2024-04-24T23:05:34.440213+0000","last_active":"2024-04-24T23:05:35.542075+0000","last_peered":"2024-04-24T23:05:35.542075+0000","last_clean":"2024-04-24T23:05:35.542075+0000","last_became_active":"2024-04-24T23:05:34.439991+0000","last_became_peered":"2024-04-24T23:05:34.439991+0000","last_unstale":"2024-04-24T23:05:35.542075+0000","last_undegraded":"2024-04-24T23:05:35.542075+0000","last_fullsized":"2024-04-24T23:05:35.542075+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:00:07.214249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.423462+0000","last_change":"2024-04-24T23:05:33.430669+0000","last_active":"2024-04-24T23:05:35.423462+0000","last_peered":"2024-04-24T23:05:35.423462+0000","last_clean":"2024-04-24T23:05:35.423462+0000","last_became_active":"2024-04-24T23:05:33.429839+0000","last_became_peered":"2024-04-24T23:05:33.429839+0000","last_unstale":"2024-04-24T23:05:35.423462+0000","last_undegraded":"2024-04-24T23:05:35.423462+0000","last_fullsized":"2024-04-24T23:05:35.423462+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:41:13.207524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:36.537067+0000","last_change":"2024-04-24T23:05:34.435311+0000","last_active":"2024-04-24T23:05:36.537067+0000","last_peered":"2024-04-24T23:05:36.537067+0000","last_clean":"2024-04-24T23:05:36.537067+0000","last_became_active":"2024-04-24T23:05:34.435046+0000","last_became_peered":"2024-04-24T23:05:34.435046+0000","last_unstale":"2024-04-24T23:05:36.537067+0000","last_undegraded":"2024-04-24T23:05:36.537067+0000","last_fullsized":"2024-04-24T23:05:36.537067+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:57.399349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.7","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.423674+0000","last_change":"2024-04-24T23:05:33.432007+0000","last_active":"2024-04-24T23:05:35.423674+0000","last_peered":"2024-04-24T23:05:35.423674+0000","last_clean":"2024-04-24T23:05:35.423674+0000","last_became_active":"2024-04-24T23:05:33.430736+0000","last_became_peered":"2024-04-24T23:05:33.430736+0000","last_unstale":"2024-04-24T23:05:35.423674+0000","last_undegraded":"2024-04-24T23:05:35.423674+0000","last_fullsized":"2024-04-24T23:05:35.423674+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:00:21.858605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.758620+0000","last_change":"2024-04-24T23:05:34.429975+0000","last_active":"2024-04-24T23:05:35.758620+0000","last_peered":"2024-04-24T23:05:35.758620+0000","last_clean":"2024-04-24T23:05:35.758620+0000","last_became_active":"2024-04-24T23:05:34.429704+0000","last_became_peered":"2024-04-24T23:05:34.429704+0000","last_unstale":"2024-04-24T23:05:35.758620+0000","last_undegraded":"2024-04-24T23:05:35.758620+0000","last_fullsized":"2024-04-24T23:05:35.758620+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:56:03.351101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"140'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:36.424699+0000","last_change":"2024-04-24T23:05:33.427070+0000","last_active":"2024-04-24T23:05:36.424699+0000","last_peered":"2024-04-24T23:05:36.424699+0000","last_clean":"2024-04-24T23:05:36.424699+0000","last_became_active":"2024-04-24T23:05:33.426791+0000","last_became_peered":"2024-04-24T23:05:33.426791+0000","last_unstale":"2024-04-24T23:05:36.424699+0000","last_undegraded":"2024-04-24T23:05:36.424699+0000","last_fullsized":"2024-04-24T23:05:36.424699+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:33:29.057440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.422891+0000","last_change":"2024-04-24T23:05:34.435945+0000","last_active":"2024-04-24T23:05:35.422891+0000","last_peered":"2024-04-24T23:05:35.422891+0000","last_clean":"2024-04-24T23:05:35.422891+0000","last_became_active":"2024-04-24T23:05:34.435233+0000","last_became_peered":"2024-04-24T23:05:34.435233+0000","last_unstale":"2024-04-24T23:05:35.422891+0000","last_undegraded":"2024-04-24T23:05:35.422891+0000","last_fullsized":"2024-04-24T23:05:35.422891+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:47.494849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.422819+0000","last_change":"2024-04-24T23:05:33.429170+0000","last_active":"2024-04-24T23:05:35.422819+0000","last_peered":"2024-04-24T23:05:35.422819+0000","last_clean":"2024-04-24T23:05:35.422819+0000","last_became_active":"2024-04-24T23:05:33.428500+0000","last_became_peered":"2024-04-24T23:05:33.428500+0000","last_unstale":"2024-04-24T23:05:35.422819+0000","last_undegraded":"2024-04-24T23:05:35.422819+0000","last_fullsized":"2024-04-24T23:05:35.422819+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:48:11.710253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:36.537067+0000","last_change":"2024-04-24T23:05:34.428521+0000","last_active":"2024-04-24T23:05:36.537067+0000","last_peered":"2024-04-24T23:05:36.537067+0000","last_clean":"2024-04-24T23:05:36.537067+0000","last_became_active":"2024-04-24T23:05:34.427910+0000","last_became_peered":"2024-04-24T23:05:34.427910+0000","last_unstale":"2024-04-24T23:05:36.537067+0000","last_undegraded":"2024-04-24T23:05:36.537067+0000","last_fullsized":"2024-04-24T23:05:36.537067+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:54.444606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.423873+0000","last_change":"2024-04-24T23:05:33.431184+0000","last_active":"2024-04-24T23:05:35.423873+0000","last_peered":"2024-04-24T23:05:35.423873+0000","last_clean":"2024-04-24T23:05:35.423873+0000","last_became_active":"2024-04-24T23:05:33.430590+0000","last_became_peered":"2024-04-24T23:05:33.430590+0000","last_unstale":"2024-04-24T23:05:35.423873+0000","last_undegraded":"2024-04-24T23:05:35.423873+0000","last_fullsized":"2024-04-24T23:05:35.423873+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:07:35.055097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:36.754415+0000","last_change":"2024-04-24T23:05:34.442204+0000","last_active":"2024-04-24T23:05:36.754415+0000","last_peered":"2024-04-24T23:05:36.754415+0000","last_clean":"2024-04-24T23:05:36.754415+0000","last_became_active":"2024-04-24T23:05:34.441934+0000","last_became_peered":"2024-04-24T23:05:34.441934+0000","last_unstale":"2024-04-24T23:05:36.754415+0000","last_undegraded":"2024-04-24T23:05:36.754415+0000","last_fullsized":"2024-04-24T23:05:36.754415+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:21:45.238132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.421855+0000","last_change":"2024-04-24T23:05:33.421091+0000","last_active":"2024-04-24T23:05:35.421855+0000","last_peered":"2024-04-24T23:05:35.421855+0000","last_clean":"2024-04-24T23:05:35.421855+0000","last_became_active":"2024-04-24T23:05:33.420817+0000","last_became_peered":"2024-04-24T23:05:33.420817+0000","last_unstale":"2024-04-24T23:05:35.421855+0000","last_undegraded":"2024-04-24T23:05:35.421855+0000","last_fullsized":"2024-04-24T23:05:35.421855+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:29:35.910813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:36.537308+0000","last_change":"2024-04-24T23:05:34.435023+0000","last_active":"2024-04-24T23:05:36.537308+0000","last_peered":"2024-04-24T23:05:36.537308+0000","last_clean":"2024-04-24T23:05:36.537308+0000","last_became_active":"2024-04-24T23:05:34.434269+0000","last_became_peered":"2024-04-24T23:05:34.434269+0000","last_unstale":"2024-04-24T23:05:36.537308+0000","last_undegraded":"2024-04-24T23:05:36.537308+0000","last_fullsized":"2024-04-24T23:05:36.537308+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:31:42.428916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:36.536996+0000","last_change":"2024-04-24T23:05:33.427689+0000","last_active":"2024-04-24T23:05:36.536996+0000","last_peered":"2024-04-24T23:05:36.536996+0000","last_clean":"2024-04-24T23:05:36.536996+0000","last_became_active":"2024-04-24T23:05:33.427406+0000","last_became_peered":"2024-04-24T23:05:33.427406+0000","last_unstale":"2024-04-24T23:05:36.536996+0000","last_undegraded":"2024-04-24T23:05:36.536996+0000","last_fullsized":"2024-04-24T23:05:36.536996+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:27:06.971278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:36.537224+0000","last_change":"2024-04-24T23:05:34.430457+0000","last_active":"2024-04-24T23:05:36.537224+0000","last_peered":"2024-04-24T23:05:36.537224+0000","last_clean":"2024-04-24T23:05:36.537224+0000","last_became_active":"2024-04-24T23:05:34.430197+0000","last_became_peered":"2024-04-24T23:05:34.430197+0000","last_unstale":"2024-04-24T23:05:36.537224+0000","last_undegraded":"2024-04-24T23:05:36.537224+0000","last_fullsized":"2024-04-24T23:05:36.537224+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:02:00.628269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.422975+0000","last_change":"2024-04-24T23:05:33.428478+0000","last_active":"2024-04-24T23:05:35.422975+0000","last_peered":"2024-04-24T23:05:35.422975+0000","last_clean":"2024-04-24T23:05:35.422975+0000","last_became_active":"2024-04-24T23:05:33.427474+0000","last_became_peered":"2024-04-24T23:05:33.427474+0000","last_unstale":"2024-04-24T23:05:35.422975+0000","last_undegraded":"2024-04-24T23:05:35.422975+0000","last_fullsized":"2024-04-24T23:05:35.422975+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:57.778668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.758552+0000","last_change":"2024-04-24T23:05:34.432945+0000","last_active":"2024-04-24T23:05:35.758552+0000","last_peered":"2024-04-24T23:05:35.758552+0000","last_clean":"2024-04-24T23:05:35.758552+0000","last_became_active":"2024-04-24T23:05:34.432651+0000","last_became_peered":"2024-04-24T23:05:34.432651+0000","last_unstale":"2024-04-24T23:05:35.758552+0000","last_undegraded":"2024-04-24T23:05:35.758552+0000","last_fullsized":"2024-04-24T23:05:35.758552+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:24:43.717620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.758616+0000","last_change":"2024-04-24T23:05:33.432773+0000","last_active":"2024-04-24T23:05:35.758616+0000","last_peered":"2024-04-24T23:05:35.758616+0000","last_clean":"2024-04-24T23:05:35.758616+0000","last_became_active":"2024-04-24T23:05:33.432239+0000","last_became_peered":"2024-04-24T23:05:33.432239+0000","last_unstale":"2024-04-24T23:05:35.758616+0000","last_undegraded":"2024-04-24T23:05:35.758616+0000","last_fullsized":"2024-04-24T23:05:35.758616+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:19.820007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.423158+0000","last_change":"2024-04-24T23:05:34.427830+0000","last_active":"2024-04-24T23:05:35.423158+0000","last_peered":"2024-04-24T23:05:35.423158+0000","last_clean":"2024-04-24T23:05:35.423158+0000","last_became_active":"2024-04-24T23:05:34.427574+0000","last_became_peered":"2024-04-24T23:05:34.427574+0000","last_unstale":"2024-04-24T23:05:35.423158+0000","last_undegraded":"2024-04-24T23:05:35.423158+0000","last_fullsized":"2024-04-24T23:05:35.423158+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:33.410361+0000","last_clean_scrub_stamp":"2024-04-24T23:05:33.410361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:32.846474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:35.758454+0000","last_change":"2024-04-24T23:05:33.431829+0000","last_active":"2024-04-24T23:05:35.758454+0000","last_peered":"2024-04-24T23:05:35.758454+0000","last_clean":"2024-04-24T23:05:35.758454+0000","last_became_active":"2024-04-24T23:05:33.431554+0000","last_became_peered":"2024-04-24T23:05:33.431554+0000","last_unstale":"2024-04-24T23:05:35.758454+0000","last_undegraded":"2024-04-24T23:05:35.758454+0000","last_fullsized":"2024-04-24T23:05:35.758454+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:05:32.401734+0000","last_clean_scrub_stamp":"2024-04-24T23:05:32.401734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:04:17.756802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":406,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-24T23:05:36.754513+0000","last_change":"2024-04-24T22:58:55.138810+0000","last_active":"2024-04-24T23:05:36.754513+0000","last_peered":"2024-04-24T23:05:36.754513+0000","last_clean":"2024-04-24T23:05:36.754513+0000","last_became_active":"2024-04-24T22:58:55.138627+0000","last_became_peered":"2024-04-24T22:58:55.138627+0000","last_unstale":"2024-04-24T23:05:36.754513+0000","last_undegraded":"2024-04-24T23:05:36.754513+0000","last_fullsized":"2024-04-24T23:05:36.754513+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_clean_scrub_stamp":"2024-04-24T22:58:54.127235+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:51.383182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574931,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":761,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194551,"statfs":{"total":200000000,"available":199220498,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574931,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":759,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194552,"statfs":{"total":200000000,"available":199222234,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574931,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1336,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193976,"statfs":{"total":200000000,"available":198631774,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574931,"num_pgs":33,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":761,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194551,"statfs":{"total":200000000,"available":199220532,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574931,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":759,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194552,"statfs":{"total":200000000,"available":199222234,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574931,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":760,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194552,"statfs":{"total":200000000,"available":199221596,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1329,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193982,"statfs":{"total":200000000,"available":198638166,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574931,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":760,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194552,"statfs":{"total":200000000,"available":199221582,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-24T23:05:37.841 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-24T23:05:37.842 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-24T23:05:37.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.985+0000 7f62095a5640 1 Processor -- start 2024-04-24T23:05:37.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.985+0000 7f62095a5640 1 -- start start 2024-04-24T23:05:37.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.986+0000 7f62095a5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f62040af890 0x7f62041670f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:37.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.986+0000 7f62095a5640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62040b6430 con 0x7f62040b44d0 2024-04-24T23:05:37.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.986+0000 7f62095a5640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6204156f80 con 0x7f620406c5f0 2024-04-24T23:05:37.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.986+0000 7f62095a5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f62040b6150 con 0x7f62040af890 2024-04-24T23:05:37.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.986+0000 7f62037fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f62040af890 0x7f62041670f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:37.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.986+0000 7f62037fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f62040af890 0x7f62041670f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56394/0 (socket says 172.21.15.32:56394) 2024-04-24T23:05:37.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.986+0000 7f61fbfff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f62040b44d0 0x7f62040b5890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37822/0 (socket says 172.21.15.32:37822) 2024-04-24T23:05:37.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.986+0000 7f62037fe640 1 -- 172.21.15.32:0/2517822970 learned_addr learned my addr 172.21.15.32:0/2517822970 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:37.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.986+0000 7f62037fe640 1 -- 172.21.15.32:0/2517822970 >> v1:172.21.15.121:6789/0 conn(0x7f620406c5f0 legacy=0x7f620406c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:37.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.986+0000 7f62037fe640 1 -- 172.21.15.32:0/2517822970 >> v1:172.21.15.32:6789/0 conn(0x7f62040b44d0 legacy=0x7f62040b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:37.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.986+0000 7f62037fe640 1 -- 172.21.15.32:0/2517822970 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62040b39f0 con 0x7f62040af890 2024-04-24T23:05:37.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.987+0000 7f62037fe640 1 --2- 172.21.15.32:0/2517822970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62040af890 0x7f62041670f0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f61fc009e50 tx=0x7f61fc02a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fbd67029877b8ac server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:37.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.987+0000 7f62027fc640 1 -- 172.21.15.32:0/2517822970 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61fc036070 con 0x7f62040af890 2024-04-24T23:05:37.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.987+0000 7f62027fc640 1 -- 172.21.15.32:0/2517822970 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61fc031600 con 0x7f62040af890 2024-04-24T23:05:37.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.987+0000 7f62027fc640 1 -- 172.21.15.32:0/2517822970 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61fc0319d0 con 0x7f62040af890 2024-04-24T23:05:37.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.987+0000 7f62095a5640 1 -- 172.21.15.32:0/2517822970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62040af890 msgr2=0x7f62041670f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:37.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.987+0000 7f62095a5640 1 --2- 172.21.15.32:0/2517822970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62040af890 0x7f62041670f0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f61fc009e50 tx=0x7f61fc02a760 comp rx=0 tx=0).stop 2024-04-24T23:05:37.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.987+0000 7f62095a5640 1 -- 172.21.15.32:0/2517822970 shutdown_connections 2024-04-24T23:05:37.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.988+0000 7f62095a5640 1 --2- 172.21.15.32:0/2517822970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62040af890 0x7f62041670f0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:37.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.988+0000 7f62095a5640 1 -- 172.21.15.32:0/2517822970 >> 172.21.15.32:0/2517822970 conn(0x7f620406b110 msgr2=0x7f620406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:37.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.988+0000 7f62095a5640 1 -- 172.21.15.32:0/2517822970 shutdown_connections 2024-04-24T23:05:37.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.988+0000 7f62095a5640 1 -- 172.21.15.32:0/2517822970 wait complete. 2024-04-24T23:05:37.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.989+0000 7f62095a5640 1 Processor -- start 2024-04-24T23:05:37.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.989+0000 7f62095a5640 1 -- start start 2024-04-24T23:05:37.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.989+0000 7f62095a5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f620406c5f0 0x7f620414c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:37.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.989+0000 7f62095a5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62040af890 0x7f6204153ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:37.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.989+0000 7f62095a5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62040b44d0 0x7f62041543e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:37.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.989+0000 7f62095a5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6204167630 con 0x7f62040b44d0 2024-04-24T23:05:37.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.989+0000 7f62095a5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f62040b3fb0 con 0x7f620406c5f0 2024-04-24T23:05:37.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.989+0000 7f62095a5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f62041677b0 con 0x7f62040af890 2024-04-24T23:05:37.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.989+0000 7f62037fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62040b44d0 0x7f62041543e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:37.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.990+0000 7f62037fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62040b44d0 0x7f62041543e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56408/0 (socket says 172.21.15.32:56408) 2024-04-24T23:05:37.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.990+0000 7f62037fe640 1 -- 172.21.15.32:0/2598647447 learned_addr learned my addr 172.21.15.32:0/2598647447 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:37.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.990+0000 7f62037fe640 1 -- 172.21.15.32:0/2598647447 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62040af890 msgr2=0x7f6204153ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:37.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.990+0000 7f62037fe640 1 --2- 172.21.15.32:0/2598647447 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62040af890 0x7f6204153ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:37.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.990+0000 7f62037fe640 1 -- 172.21.15.32:0/2598647447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f620406c5f0 msgr2=0x7f620414c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:37.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.990+0000 7f6202ffd640 1 --2- 172.21.15.32:0/2598647447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f620406c5f0 0x7f620414c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:37.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.990+0000 7f62037fe640 1 --2- 172.21.15.32:0/2598647447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f620406c5f0 0x7f620414c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:37.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.990+0000 7f62037fe640 1 -- 172.21.15.32:0/2598647447 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f620415fbd0 con 0x7f62040b44d0 2024-04-24T23:05:37.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.990+0000 7f62037fe640 1 --2- 172.21.15.32:0/2598647447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62040b44d0 0x7f62041543e0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f61fc009e50 tx=0x7f61fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:37.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.990+0000 7f61fb7fe640 1 -- 172.21.15.32:0/2598647447 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61fc041020 con 0x7f62040b44d0 2024-04-24T23:05:37.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.990+0000 7f61fb7fe640 1 -- 172.21.15.32:0/2598647447 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61fc007780 con 0x7f62040b44d0 2024-04-24T23:05:37.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.990+0000 7f62095a5640 1 -- 172.21.15.32:0/2598647447 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f620415e670 con 0x7f62040b44d0 2024-04-24T23:05:37.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.990+0000 7f61fb7fe640 1 -- 172.21.15.32:0/2598647447 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61fc036040 con 0x7f62040b44d0 2024-04-24T23:05:37.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.990+0000 7f62095a5640 1 -- 172.21.15.32:0/2598647447 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f620415ec30 con 0x7f62040b44d0 2024-04-24T23:05:37.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.991+0000 7f62095a5640 1 -- 172.21.15.32:0/2598647447 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6204068430 con 0x7f62040b44d0 2024-04-24T23:05:37.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.992+0000 7f61fb7fe640 1 -- 172.21.15.32:0/2598647447 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f61fc044020 con 0x7f62040b44d0 2024-04-24T23:05:37.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.992+0000 7f61fb7fe640 1 --2- 172.21.15.32:0/2598647447 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f61e406be30 0x7f61e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:37.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.992+0000 7f61fb7fe640 1 -- 172.21.15.32:0/2598647447 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5709+0+0 (secure 0 0 0) 0x7f61fc0abb80 con 0x7f62040b44d0 2024-04-24T23:05:37.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.992+0000 7f6202ffd640 1 --2- 172.21.15.32:0/2598647447 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f61e406be30 0x7f61e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:37.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.993+0000 7f6202ffd640 1 --2- 172.21.15.32:0/2598647447 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f61e406be30 0x7f61e406e2f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f62041f5f60 tx=0x7f61f4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:37.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:37.996+0000 7f61fb7fe640 1 -- 172.21.15.32:0/2598647447 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f61fc04b030 con 0x7f62040b44d0 2024-04-24T23:05:38.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.251+0000 7f62095a5640 1 -- 172.21.15.32:0/2598647447 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f6204071050 con 0x7f62040b44d0 2024-04-24T23:05:38.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.253+0000 7f61fb7fe640 1 -- 172.21.15.32:0/2598647447 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) ==== 190+0+147 (secure 0 0 0) 0x7f61fc07a5f0 con 0x7f62040b44d0 2024-04-24T23:05:38.255 INFO:teuthology.orchestra.run.smithi032.stdout:[client.0] 2024-04-24T23:05:38.255 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQCejilm5oeiGRAAdLXwwqrYhV3HHo7Dw/2ZqQ== 2024-04-24T23:05:38.255 INFO:teuthology.orchestra.run.smithi032.stdout: caps mds = "allow" 2024-04-24T23:05:38.255 INFO:teuthology.orchestra.run.smithi032.stdout: caps mon = "allow r" 2024-04-24T23:05:38.255 INFO:teuthology.orchestra.run.smithi032.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-24T23:05:38.255 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2024-04-24T23:05:38.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.255+0000 7f62095a5640 1 -- 172.21.15.32:0/2598647447 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f61e406be30 msgr2=0x7f61e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:38.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.255+0000 7f62095a5640 1 --2- 172.21.15.32:0/2598647447 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f61e406be30 0x7f61e406e2f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f62041f5f60 tx=0x7f61f4009040 comp rx=0 tx=0).stop 2024-04-24T23:05:38.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.255+0000 7f62095a5640 1 -- 172.21.15.32:0/2598647447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62040b44d0 msgr2=0x7f62041543e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:38.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.255+0000 7f62095a5640 1 --2- 172.21.15.32:0/2598647447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62040b44d0 0x7f62041543e0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f61fc009e50 tx=0x7f61fc004a40 comp rx=0 tx=0).stop 2024-04-24T23:05:38.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.255+0000 7f62095a5640 1 -- 172.21.15.32:0/2598647447 shutdown_connections 2024-04-24T23:05:38.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.255+0000 7f62095a5640 1 --2- 172.21.15.32:0/2598647447 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f61e406be30 0x7f61e406e2f0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:38.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.255+0000 7f62095a5640 1 --2- 172.21.15.32:0/2598647447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62040b44d0 0x7f62041543e0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:38.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.255+0000 7f62095a5640 1 --2- 172.21.15.32:0/2598647447 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62040af890 0x7f6204153ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:38.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.255+0000 7f62095a5640 1 --2- 172.21.15.32:0/2598647447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f620406c5f0 0x7f620414c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:38.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.255+0000 7f62095a5640 1 -- 172.21.15.32:0/2598647447 >> 172.21.15.32:0/2598647447 conn(0x7f620406b110 msgr2=0x7f62040cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:38.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.255+0000 7f62095a5640 1 -- 172.21.15.32:0/2598647447 shutdown_connections 2024-04-24T23:05:38.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.255+0000 7f62095a5640 1 -- 172.21.15.32:0/2598647447 wait complete. 2024-04-24T23:05:38.268 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:05:38.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.367+0000 7f677279e640 1 Processor -- start 2024-04-24T23:05:38.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.367+0000 7f677279e640 1 -- start start 2024-04-24T23:05:38.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.368+0000 7f677279e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f676c06b820 0x7f676c0d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:38.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.368+0000 7f677279e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f676c0d4210 0x7f676c0b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:38.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.368+0000 7f677279e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f676c0d3170 con 0x7f676c15e9b0 2024-04-24T23:05:38.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.368+0000 7f677279e640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f676c0d3450 con 0x7f676c06b820 2024-04-24T23:05:38.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.368+0000 7f677279e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f676c0d3730 con 0x7f676c0d4210 2024-04-24T23:05:38.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.368+0000 7f676b7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f676c06b820 0x7f676c0d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:38.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.368+0000 7f676b7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f676c06b820 0x7f676c0d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56420/0 (socket says 172.21.15.32:56420) 2024-04-24T23:05:38.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.368+0000 7f676b7fe640 1 -- 172.21.15.32:0/2451890658 learned_addr learned my addr 172.21.15.32:0/2451890658 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:38.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.368+0000 7f676bfff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f676c15e9b0 0x7f676c06b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37830/0 (socket says 172.21.15.32:37830) 2024-04-24T23:05:38.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.369+0000 7f6770d3b640 1 --2- 172.21.15.32:0/2451890658 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f676c0d4210 0x7f676c0b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:38.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.369+0000 7f676b7fe640 1 -- 172.21.15.32:0/2451890658 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f676c0d4210 msgr2=0x7f676c0b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:38.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.369+0000 7f676b7fe640 1 --2- 172.21.15.32:0/2451890658 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f676c0d4210 0x7f676c0b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:38.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.369+0000 7f676b7fe640 1 -- 172.21.15.32:0/2451890658 >> v1:172.21.15.32:6789/0 conn(0x7f676c15e9b0 legacy=0x7f676c06b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:38.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.369+0000 7f676b7fe640 1 -- 172.21.15.32:0/2451890658 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f676c0d3a10 con 0x7f676c06b820 2024-04-24T23:05:38.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.369+0000 7f676affd640 1 -- 172.21.15.32:0/2451890658 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3986169550 0 0) 0x7f676c0d3170 con 0x7f676c15e9b0 2024-04-24T23:05:38.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.369+0000 7f676b7fe640 1 --2- 172.21.15.32:0/2451890658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f676c06b820 0x7f676c0d2b90 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f6760009870 tx=0x7f676002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcd939813f3bb96e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:38.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.369+0000 7f676affd640 1 -- 172.21.15.32:0/2451890658 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6760037070 con 0x7f676c06b820 2024-04-24T23:05:38.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.370+0000 7f676affd640 1 -- 172.21.15.32:0/2451890658 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6760032440 con 0x7f676c06b820 2024-04-24T23:05:38.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.370+0000 7f676affd640 1 -- 172.21.15.32:0/2451890658 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6760032810 con 0x7f676c06b820 2024-04-24T23:05:38.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.370+0000 7f677279e640 1 -- 172.21.15.32:0/2451890658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f676c06b820 msgr2=0x7f676c0d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:38.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.370+0000 7f677279e640 1 --2- 172.21.15.32:0/2451890658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f676c06b820 0x7f676c0d2b90 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f6760009870 tx=0x7f676002a270 comp rx=0 tx=0).stop 2024-04-24T23:05:38.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.370+0000 7f677279e640 1 -- 172.21.15.32:0/2451890658 shutdown_connections 2024-04-24T23:05:38.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.370+0000 7f677279e640 1 --2- 172.21.15.32:0/2451890658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f676c06b820 0x7f676c0d2b90 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:38.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.370+0000 7f677279e640 1 --2- 172.21.15.32:0/2451890658 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f676c0d4210 0x7f676c0b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:38.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.370+0000 7f677279e640 1 -- 172.21.15.32:0/2451890658 >> 172.21.15.32:0/2451890658 conn(0x7f676c0d11d0 msgr2=0x7f676c0d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:38.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.370+0000 7f677279e640 1 -- 172.21.15.32:0/2451890658 shutdown_connections 2024-04-24T23:05:38.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.370+0000 7f677279e640 1 -- 172.21.15.32:0/2451890658 wait complete. 2024-04-24T23:05:38.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.371+0000 7f677279e640 1 Processor -- start 2024-04-24T23:05:38.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.371+0000 7f677279e640 1 -- start start 2024-04-24T23:05:38.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.372+0000 7f677279e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f676c06b820 0x7f676c0ce3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:38.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.372+0000 7f677279e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f676c0d4210 0x7f676c0cb820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:38.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.372+0000 7f677279e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f676c15e9b0 0x7f676c15b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:38.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.372+0000 7f677279e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f676c0ba0e0 con 0x7f676c06b820 2024-04-24T23:05:38.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.372+0000 7f677279e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f676c0d2400 con 0x7f676c15e9b0 2024-04-24T23:05:38.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.372+0000 7f677279e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f676c0ba260 con 0x7f676c0d4210 2024-04-24T23:05:38.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.372+0000 7f676bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f676c06b820 0x7f676c0ce3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:38.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.372+0000 7f676bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f676c06b820 0x7f676c0ce3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56426/0 (socket says 172.21.15.32:56426) 2024-04-24T23:05:38.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.372+0000 7f676bfff640 1 -- 172.21.15.32:0/3952710995 learned_addr learned my addr 172.21.15.32:0/3952710995 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:38.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.372+0000 7f676b7fe640 1 --2- 172.21.15.32:0/3952710995 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f676c0d4210 0x7f676c0cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:38.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.373+0000 7f6770d3b640 1 --2- 172.21.15.32:0/3952710995 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f676c15e9b0 0x7f676c15b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:38.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.373+0000 7f676bfff640 1 -- 172.21.15.32:0/3952710995 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f676c0d4210 msgr2=0x7f676c0cb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:38.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.373+0000 7f676bfff640 1 --2- 172.21.15.32:0/3952710995 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f676c0d4210 0x7f676c0cb820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:38.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.373+0000 7f676bfff640 1 -- 172.21.15.32:0/3952710995 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f676c15e9b0 msgr2=0x7f676c15b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:38.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.373+0000 7f676bfff640 1 --2- 172.21.15.32:0/3952710995 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f676c15e9b0 0x7f676c15b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:38.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.373+0000 7f676bfff640 1 -- 172.21.15.32:0/3952710995 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f676c0ce920 con 0x7f676c06b820 2024-04-24T23:05:38.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.373+0000 7f6770d3b640 1 --2- 172.21.15.32:0/3952710995 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f676c15e9b0 0x7f676c15b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:05:38.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.373+0000 7f676bfff640 1 --2- 172.21.15.32:0/3952710995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f676c06b820 0x7f676c0ce3e0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f675c00c430 tx=0x7f675c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:38.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.373+0000 7f6768ff9640 1 -- 172.21.15.32:0/3952710995 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675c0074e0 con 0x7f676c06b820 2024-04-24T23:05:38.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.373+0000 7f6768ff9640 1 -- 172.21.15.32:0/3952710995 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f675c0047e0 con 0x7f676c06b820 2024-04-24T23:05:38.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.373+0000 7f6768ff9640 1 -- 172.21.15.32:0/3952710995 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675c004bd0 con 0x7f676c06b820 2024-04-24T23:05:38.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.373+0000 7f677279e640 1 -- 172.21.15.32:0/3952710995 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f676c0cc270 con 0x7f676c06b820 2024-04-24T23:05:38.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.373+0000 7f677279e640 1 -- 172.21.15.32:0/3952710995 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f676c0cc830 con 0x7f676c06b820 2024-04-24T23:05:38.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.374+0000 7f677279e640 1 -- 172.21.15.32:0/3952710995 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f676c068430 con 0x7f676c06b820 2024-04-24T23:05:38.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.374+0000 7f6768ff9640 1 -- 172.21.15.32:0/3952710995 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f675c011070 con 0x7f676c06b820 2024-04-24T23:05:38.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.375+0000 7f6768ff9640 1 --2- 172.21.15.32:0/3952710995 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f675806be30 0x7f675806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:38.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.375+0000 7f6768ff9640 1 -- 172.21.15.32:0/3952710995 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5709+0+0 (secure 0 0 0) 0x7f675c08a960 con 0x7f676c06b820 2024-04-24T23:05:38.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.375+0000 7f676b7fe640 1 --2- 172.21.15.32:0/3952710995 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f675806be30 0x7f675806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:38.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.376+0000 7f676b7fe640 1 --2- 172.21.15.32:0/3952710995 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f675806be30 0x7f675806e2f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f6760034000 tx=0x7f676002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:38.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.379+0000 7f6768ff9640 1 -- 172.21.15.32:0/3952710995 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f675c0593a0 con 0x7f676c06b820 2024-04-24T23:05:38.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.600+0000 7f677279e640 1 -- 172.21.15.32:0/3952710995 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f676c0d16b0 con 0x7f676c06b820 2024-04-24T23:05:38.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.601+0000 7f6768ff9640 1 -- 172.21.15.32:0/3952710995 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+5189 (secure 0 0 0) 0x7f675c058ca0 con 0x7f676c06b820 2024-04-24T23:05:38.602 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:05:38.603 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":58,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3361752496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3361752496},{"type":"v1","addr":"172.21.15.121:6837","nonce":3361752496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":6799,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3813453544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3813453544},{"type":"v1","addr":"172.21.15.121:6836","nonce":3813453544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7779,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/1352085651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1352085651},{"type":"v1","addr":"172.21.15.32:6833","nonce":1352085651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:05:33.791068+0000","modified":"2024-04-24T23:05:35.792946+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7794},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7794":{"gid":7794,"name":"a","rank":0,"incarnation":56,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/4206823204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":4206823204},{"type":"v1","addr":"172.21.15.32:6835","nonce":4206823204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7794,"qdb_cluster":[7794]},"id":8}]} 2024-04-24T23:05:38.603 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 58 2024-04-24T23:05:38.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.602+0000 7f677279e640 1 -- 172.21.15.32:0/3952710995 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f675806be30 msgr2=0x7f675806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:38.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.602+0000 7f677279e640 1 --2- 172.21.15.32:0/3952710995 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f675806be30 0x7f675806e2f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f6760034000 tx=0x7f676002e000 comp rx=0 tx=0).stop 2024-04-24T23:05:38.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.602+0000 7f677279e640 1 -- 172.21.15.32:0/3952710995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f676c06b820 msgr2=0x7f676c0ce3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:38.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.602+0000 7f677279e640 1 --2- 172.21.15.32:0/3952710995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f676c06b820 0x7f676c0ce3e0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f675c00c430 tx=0x7f675c00c8f0 comp rx=0 tx=0).stop 2024-04-24T23:05:38.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.603+0000 7f677279e640 1 -- 172.21.15.32:0/3952710995 shutdown_connections 2024-04-24T23:05:38.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.603+0000 7f677279e640 1 --2- 172.21.15.32:0/3952710995 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f675806be30 0x7f675806e2f0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:38.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.603+0000 7f677279e640 1 --2- 172.21.15.32:0/3952710995 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f676c15e9b0 0x7f676c15b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:38.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.603+0000 7f677279e640 1 --2- 172.21.15.32:0/3952710995 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f676c0d4210 0x7f676c0cb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:38.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.603+0000 7f677279e640 1 --2- 172.21.15.32:0/3952710995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f676c06b820 0x7f676c0ce3e0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:38.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.603+0000 7f677279e640 1 -- 172.21.15.32:0/3952710995 >> 172.21.15.32:0/3952710995 conn(0x7f676c0d11d0 msgr2=0x7f676c0cefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:38.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.603+0000 7f677279e640 1 -- 172.21.15.32:0/3952710995 shutdown_connections 2024-04-24T23:05:38.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:38.603+0000 7f677279e640 1 -- 172.21.15.32:0/3952710995 wait complete. 2024-04-24T23:05:39.017 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 58, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T23:05:33.791068+0000', 'modified': '2024-04-24T23:05:35.792946+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7794}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7794': {'gid': 7794, 'name': 'a', 'rank': 0, 'incarnation': 56, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/4206823204', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 4206823204}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 4206823204}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7794, 'qdb_cluster': [7794]} 2024-04-24T23:05:39.018 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-24T23:05:39.018 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-24T23:05:39.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.122+0000 7f6f6f49e640 1 Processor -- start 2024-04-24T23:05:39.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.122+0000 7f6f6f49e640 1 -- start start 2024-04-24T23:05:39.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.123+0000 7f6f6f49e640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6f68074210 0x7f6f6806c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:39.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.123+0000 7f6f6f49e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6f680b0b80 0x7f6f680745f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:39.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.123+0000 7f6f6f49e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f680b0770 con 0x7f6f680ae140 2024-04-24T23:05:39.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.123+0000 7f6f6f49e640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f6f680b1c20 con 0x7f6f680b0b80 2024-04-24T23:05:39.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.123+0000 7f6f6f49e640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f6f68074b30 con 0x7f6f68074210 2024-04-24T23:05:39.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.123+0000 7f6f6ca39640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6f680b0b80 0x7f6f680745f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:39.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.123+0000 7f6f6da3b640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6f680ae140 0x7f6f680b20b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37836/0 (socket says 172.21.15.32:37836) 2024-04-24T23:05:39.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.123+0000 7f6f6da3b640 1 -- 172.21.15.32:0/163470041 learned_addr learned my addr 172.21.15.32:0/163470041 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:05:39.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.123+0000 7f6f6d23a640 1 --2- 172.21.15.32:0/163470041 >> v2:172.21.15.121:3300/0 conn(0x7f6f68074210 0x7f6f6806c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:39.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.123+0000 7f6f6ca39640 1 -- 172.21.15.32:0/163470041 >> v2:172.21.15.121:3300/0 conn(0x7f6f68074210 msgr2=0x7f6f6806c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:39.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.123+0000 7f6f6ca39640 1 --2- 172.21.15.32:0/163470041 >> v2:172.21.15.121:3300/0 conn(0x7f6f68074210 0x7f6f6806c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:39.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.123+0000 7f6f6ca39640 1 -- 172.21.15.32:0/163470041 >> v1:172.21.15.32:6789/0 conn(0x7f6f680ae140 legacy=0x7f6f680b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:39.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.123+0000 7f6f6ca39640 1 -- 172.21.15.32:0/163470041 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f680b1010 con 0x7f6f680b0b80 2024-04-24T23:05:39.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.123+0000 7f6f67fff640 1 -- 172.21.15.32:0/163470041 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3068128048 0 0) 0x7f6f680b0770 con 0x7f6f680ae140 2024-04-24T23:05:39.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.124+0000 7f6f6ca39640 1 --2- 172.21.15.32:0/163470041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f680b0b80 0x7f6f680745f0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f6f68073960 tx=0x7f6f5c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a87d65fae1d68a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:39.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.124+0000 7f6f67fff640 1 -- 172.21.15.32:0/163470041 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c037070 con 0x7f6f680b0b80 2024-04-24T23:05:39.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.124+0000 7f6f67fff640 1 -- 172.21.15.32:0/163470041 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f5c032440 con 0x7f6f680b0b80 2024-04-24T23:05:39.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.124+0000 7f6f67fff640 1 -- 172.21.15.32:0/163470041 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c032810 con 0x7f6f680b0b80 2024-04-24T23:05:39.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.125+0000 7f6f6f49e640 1 -- 172.21.15.32:0/163470041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f680b0b80 msgr2=0x7f6f680745f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:39.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.125+0000 7f6f6f49e640 1 --2- 172.21.15.32:0/163470041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f680b0b80 0x7f6f680745f0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f6f68073960 tx=0x7f6f5c02a1a0 comp rx=0 tx=0).stop 2024-04-24T23:05:39.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.125+0000 7f6f6f49e640 1 -- 172.21.15.32:0/163470041 shutdown_connections 2024-04-24T23:05:39.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.125+0000 7f6f6f49e640 1 --2- 172.21.15.32:0/163470041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f680b0b80 0x7f6f680745f0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:39.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.125+0000 7f6f6f49e640 1 --2- 172.21.15.32:0/163470041 >> v2:172.21.15.121:3300/0 conn(0x7f6f68074210 0x7f6f6806c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:39.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.125+0000 7f6f6f49e640 1 -- 172.21.15.32:0/163470041 >> 172.21.15.32:0/163470041 conn(0x7f6f680b2e60 msgr2=0x7f6f680b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:39.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.125+0000 7f6f6f49e640 1 -- 172.21.15.32:0/163470041 shutdown_connections 2024-04-24T23:05:39.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.125+0000 7f6f6f49e640 1 -- 172.21.15.32:0/163470041 wait complete. 2024-04-24T23:05:39.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.126+0000 7f6f6f49e640 1 Processor -- start 2024-04-24T23:05:39.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.126+0000 7f6f6f49e640 1 -- start start 2024-04-24T23:05:39.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.127+0000 7f6f6f49e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f68074210 0x7f6f680af380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:39.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.127+0000 7f6f6f49e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f680ae140 0x7f6f681530c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:39.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.127+0000 7f6f6f49e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f6815e3a0 0x7f6f681f4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:39.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.127+0000 7f6f6f49e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6f680b18b0 con 0x7f6f680ae140 2024-04-24T23:05:39.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.127+0000 7f6f6f49e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6f680b15d0 con 0x7f6f6815e3a0 2024-04-24T23:05:39.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.127+0000 7f6f6f49e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f6f680b3ba0 con 0x7f6f68074210 2024-04-24T23:05:39.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.127+0000 7f6f6ca39640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f680ae140 0x7f6f681530c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:39.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.127+0000 7f6f6ca39640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f680ae140 0x7f6f681530c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56432/0 (socket says 172.21.15.32:56432) 2024-04-24T23:05:39.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.127+0000 7f6f6ca39640 1 -- 172.21.15.32:0/999456852 learned_addr learned my addr 172.21.15.32:0/999456852 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:39.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.127+0000 7f6f6ca39640 1 -- 172.21.15.32:0/999456852 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f68074210 msgr2=0x7f6f680af380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:39.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.127+0000 7f6f6ca39640 1 --2- 172.21.15.32:0/999456852 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f68074210 0x7f6f680af380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:39.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.127+0000 7f6f6ca39640 1 -- 172.21.15.32:0/999456852 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f6815e3a0 msgr2=0x7f6f681f4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:39.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.127+0000 7f6f6ca39640 1 --2- 172.21.15.32:0/999456852 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f6815e3a0 0x7f6f681f4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:39.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.127+0000 7f6f6ca39640 1 -- 172.21.15.32:0/999456852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f68161820 con 0x7f6f680ae140 2024-04-24T23:05:39.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.127+0000 7f6f6ca39640 1 --2- 172.21.15.32:0/999456852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f680ae140 0x7f6f681530c0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c000c00 tx=0x7f6f5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:39.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.128+0000 7f6f65ffb640 1 -- 172.21.15.32:0/999456852 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c037040 con 0x7f6f680ae140 2024-04-24T23:05:39.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.128+0000 7f6f65ffb640 1 -- 172.21.15.32:0/999456852 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f5c042070 con 0x7f6f680ae140 2024-04-24T23:05:39.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.128+0000 7f6f6f49e640 1 -- 172.21.15.32:0/999456852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f68161b00 con 0x7f6f680ae140 2024-04-24T23:05:39.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.128+0000 7f6f65ffb640 1 -- 172.21.15.32:0/999456852 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c030bd0 con 0x7f6f680ae140 2024-04-24T23:05:39.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.128+0000 7f6f6f49e640 1 -- 172.21.15.32:0/999456852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f680cd900 con 0x7f6f680ae140 2024-04-24T23:05:39.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.129+0000 7f6f65ffb640 1 -- 172.21.15.32:0/999456852 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6f5c047460 con 0x7f6f680ae140 2024-04-24T23:05:39.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.130+0000 7f6f65ffb640 1 --2- 172.21.15.32:0/999456852 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6f5406be60 0x7f6f5406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:39.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.130+0000 7f6f65ffb640 1 -- 172.21.15.32:0/999456852 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5709+0+0 (secure 0 0 0) 0x7f6f5c0aba60 con 0x7f6f680ae140 2024-04-24T23:05:39.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.130+0000 7f6f6d23a640 1 --2- 172.21.15.32:0/999456852 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6f5406be60 0x7f6f5406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:39.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.131+0000 7f6f721af740 1 Processor -- start 2024-04-24T23:05:39.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.131+0000 7f6f6d23a640 1 --2- 172.21.15.32:0/999456852 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6f5406be60 0x7f6f5406e320 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f6f580070b0 tx=0x7f6f58007040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:39.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.131+0000 7f6f721af740 1 -- start start 2024-04-24T23:05:39.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.131+0000 7f6f721af740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612ae355b60 0x5612ae3084f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:39.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.131+0000 7f6f721af740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612ae338c80 0x5612ae338000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:39.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.131+0000 7f6f721af740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5612ae338540 0x5612ae30d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:39.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.131+0000 7f6f721af740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5612ae270f00 con 0x5612ae338c80 2024-04-24T23:05:39.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.131+0000 7f6f721af740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5612ae219750 con 0x5612ae355b60 2024-04-24T23:05:39.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.131+0000 7f6f721af740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x5612ae26cec0 con 0x5612ae338540 2024-04-24T23:05:39.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.132+0000 7f6f6ca39640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612ae338c80 0x5612ae338000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:39.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.132+0000 7f6f6ca39640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612ae338c80 0x5612ae338000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56444/0 (socket says 172.21.15.32:56444) 2024-04-24T23:05:39.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.132+0000 7f6f6ca39640 1 -- 172.21.15.32:0/749871773 learned_addr learned my addr 172.21.15.32:0/749871773 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:39.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.132+0000 7f6f6d23a640 1 --2- 172.21.15.32:0/749871773 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612ae355b60 0x5612ae3084f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:39.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.132+0000 7f6f6da3b640 1 --2- 172.21.15.32:0/749871773 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5612ae338540 0x5612ae30d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:39.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.132+0000 7f6f6ca39640 1 -- 172.21.15.32:0/749871773 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5612ae338540 msgr2=0x5612ae30d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:39.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.132+0000 7f6f6ca39640 1 --2- 172.21.15.32:0/749871773 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5612ae338540 0x5612ae30d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:39.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.132+0000 7f6f6ca39640 1 -- 172.21.15.32:0/749871773 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612ae355b60 msgr2=0x5612ae3084f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:39.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.132+0000 7f6f6ca39640 1 --2- 172.21.15.32:0/749871773 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612ae355b60 0x5612ae3084f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:39.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.132+0000 7f6f6ca39640 1 -- 172.21.15.32:0/749871773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612adf852d0 con 0x5612ae338c80 2024-04-24T23:05:39.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.132+0000 7f6f6da3b640 1 --2- 172.21.15.32:0/749871773 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5612ae338540 0x5612ae30d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:05:39.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.132+0000 7f6f6ca39640 1 --2- 172.21.15.32:0/749871773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612ae338c80 0x5612ae338000 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c076160 tx=0x7f6f5c02a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbed3a78b782a687 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:39.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.132+0000 7f6f4effd640 1 -- 172.21.15.32:0/749871773 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c079d00 con 0x5612ae338c80 2024-04-24T23:05:39.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.132+0000 7f6f4effd640 1 -- 172.21.15.32:0/749871773 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f5c0a8070 con 0x5612ae338c80 2024-04-24T23:05:39.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.132+0000 7f6f4effd640 1 -- 172.21.15.32:0/749871773 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c07e0b0 con 0x5612ae338c80 2024-04-24T23:05:39.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.133+0000 7f6f721af740 1 -- 172.21.15.32:0/749871773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612ae338c80 msgr2=0x5612ae338000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:39.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.133+0000 7f6f721af740 1 --2- 172.21.15.32:0/749871773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612ae338c80 0x5612ae338000 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c076160 tx=0x7f6f5c02a750 comp rx=0 tx=0).stop 2024-04-24T23:05:39.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.133+0000 7f6f721af740 1 -- 172.21.15.32:0/749871773 shutdown_connections 2024-04-24T23:05:39.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.133+0000 7f6f721af740 1 --2- 172.21.15.32:0/749871773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612ae338c80 0x5612ae338000 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:39.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.133+0000 7f6f721af740 1 --2- 172.21.15.32:0/749871773 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5612ae338540 0x5612ae30d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:39.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.133+0000 7f6f721af740 1 --2- 172.21.15.32:0/749871773 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612ae355b60 0x5612ae3084f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:39.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.133+0000 7f6f721af740 1 -- 172.21.15.32:0/749871773 >> 172.21.15.32:0/749871773 conn(0x5612ae30a760 msgr2=0x5612ae353000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:39.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.133+0000 7f6f721af740 1 -- 172.21.15.32:0/749871773 shutdown_connections 2024-04-24T23:05:39.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.133+0000 7f6f721af740 1 -- 172.21.15.32:0/749871773 wait complete. 2024-04-24T23:05:39.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.133+0000 7f6f721af740 1 -- start start 2024-04-24T23:05:39.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.133+0000 7f6f721af740 1 Processor -- start 2024-04-24T23:05:39.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.134+0000 7f6f721af740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612ae338c80 0x5612ae325930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:39.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.134+0000 7f6f721af740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612ae355b60 0x5612ae366610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:39.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.134+0000 7f6f721af740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5612ae366b50 0x5612ae3546a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:39.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.134+0000 7f6f721af740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5612ae270f00 con 0x5612ae338c80 2024-04-24T23:05:39.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.134+0000 7f6f721af740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5612ae22ccf0 con 0x5612ae355b60 2024-04-24T23:05:39.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.134+0000 7f6f721af740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x5612ae30c910 con 0x5612ae366b50 2024-04-24T23:05:39.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.134+0000 7f6f721af740 10 client.0 _ll_register_callbacks cb 0x5612ae22c6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:05:39.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.134+0000 7f6f6d23a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612ae338c80 0x5612ae325930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:39.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.134+0000 7f6f6d23a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612ae338c80 0x5612ae325930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56452/0 (socket says 172.21.15.32:56452) 2024-04-24T23:05:39.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.134+0000 7f6f6d23a640 1 -- 172.21.15.32:0/4270488524 learned_addr learned my addr 172.21.15.32:0/4270488524 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:39.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.135+0000 7f6f6ca39640 1 --2- 172.21.15.32:0/4270488524 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612ae355b60 0x5612ae366610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:39.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.135+0000 7f6f6da3b640 1 --2- 172.21.15.32:0/4270488524 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5612ae366b50 0x5612ae3546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:39.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.135+0000 7f6f6d23a640 1 -- 172.21.15.32:0/4270488524 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5612ae366b50 msgr2=0x5612ae3546a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:39.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.135+0000 7f6f6d23a640 1 --2- 172.21.15.32:0/4270488524 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5612ae366b50 0x5612ae3546a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:39.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.135+0000 7f6f6d23a640 1 -- 172.21.15.32:0/4270488524 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612ae355b60 msgr2=0x5612ae366610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:39.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.135+0000 7f6f6d23a640 1 --2- 172.21.15.32:0/4270488524 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612ae355b60 0x5612ae366610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:39.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.135+0000 7f6f6d23a640 1 -- 172.21.15.32:0/4270488524 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612ae04f6b0 con 0x5612ae338c80 2024-04-24T23:05:39.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.135+0000 7f6f6d23a640 1 --2- 172.21.15.32:0/4270488524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612ae338c80 0x5612ae325930 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f6f58016e60 tx=0x7f6f58004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb668599a20339ef server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:39.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.135+0000 7f6f47fff640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-24T23:05:39.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.135+0000 7f6f47fff640 1 -- 172.21.15.32:0/4270488524 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f58021600 con 0x5612ae338c80 2024-04-24T23:05:39.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.135+0000 7f6f721af740 1 -- 172.21.15.32:0/4270488524 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f5c0a8070 con 0x5612ae338c80 2024-04-24T23:05:39.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.135+0000 7f6f47fff640 1 -- 172.21.15.32:0/4270488524 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(140..140 src has 1..140) ==== 5709+0+0 (secure 0 0 0) 0x7f6f580217a0 con 0x5612ae338c80 2024-04-24T23:05:39.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.135+0000 7f6f47fff640 1 -- 172.21.15.32:0/4270488524 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f6f580219a0 con 0x5612ae338c80 2024-04-24T23:05:39.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.135+0000 7f6f721af740 10 client.7929 fetch_fsmap learned FSMap version 58 2024-04-24T23:05:39.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.135+0000 7f6f721af740 1 -- 172.21.15.32:0/4270488524 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) -- 0x5612ae34bb30 con 0x5612ae338c80 2024-04-24T23:05:39.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.135+0000 7f6f47fff640 1 -- 172.21.15.32:0/4270488524 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f5c0a8070 con 0x5612ae338c80 2024-04-24T23:05:39.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.135+0000 7f6f47fff640 1 -- 172.21.15.32:0/4270488524 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 58) ==== 3494+0+0 (secure 0 0 0) 0x7f6f58021b40 con 0x5612ae338c80 2024-04-24T23:05:39.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.136+0000 7f6f721af740 10 client.7929 fetch_fsmap finished waiting for FSMap version 58 2024-04-24T23:05:39.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.136+0000 7f6f721af740 10 client.7929 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T23:05:39.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.136+0000 7f6f721af740 20 client.7929 populate_metadata read hostname 'smithi032' 2024-04-24T23:05:39.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.136+0000 7f6f721af740 1 --2- 172.21.15.32:0/4270488524 >> [v2:172.21.15.32:6834/4206823204,v1:172.21.15.32:6835/4206823204] conn(0x5612ae3394d0 0x5612ae34af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:39.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.136+0000 7f6f721af740 4 client.7929 mds_command: new command op to 7794 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:05:39.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.136+0000 7f6f721af740 1 -- 172.21.15.32:0/4270488524 --> [v2:172.21.15.32:6834/4206823204,v1:172.21.15.32:6835/4206823204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612ae27e440 con 0x5612ae3394d0 2024-04-24T23:05:39.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.136+0000 7f6f6ca39640 1 --2- 172.21.15.32:0/4270488524 >> [v2:172.21.15.32:6834/4206823204,v1:172.21.15.32:6835/4206823204] conn(0x5612ae3394d0 0x5612ae34af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:39.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.136+0000 7f6f6ca39640 1 --2- 172.21.15.32:0/4270488524 >> [v2:172.21.15.32:6834/4206823204,v1:172.21.15.32:6835/4206823204] conn(0x5612ae3394d0 0x5612ae34af70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4145ce5616c4f069 server_cookie=6f4998f0d48b4310 in_seq=0 out_seq=0 2024-04-24T23:05:39.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.136+0000 7f6f47fff640 10 client.7929 ms_handle_connect on v2:172.21.15.32:6834/4206823204 2024-04-24T23:05:39.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.137+0000 7f6f47fff640 1 -- 172.21.15.32:0/4270488524 <== mds.0 v2:172.21.15.32:6834/4206823204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5612ae27e440 con 0x5612ae3394d0 2024-04-24T23:05:39.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.137+0000 7f6f47fff640 10 client.7929 handle_command_reply: tid=0 2024-04-24T23:05:39.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.137+0000 7f6f721af740 1 client.7929 shutdown 2024-04-24T23:05:39.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.137+0000 7f6f721af740 1 -- 172.21.15.32:0/4270488524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612ae338c80 msgr2=0x5612ae325930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:39.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.137+0000 7f6f721af740 1 --2- 172.21.15.32:0/4270488524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612ae338c80 0x5612ae325930 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f6f58016e60 tx=0x7f6f58004a40 comp rx=0 tx=0).stop 2024-04-24T23:05:39.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.137+0000 7f6f721af740 1 -- 172.21.15.32:0/4270488524 shutdown_connections 2024-04-24T23:05:39.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.137+0000 7f6f721af740 1 --2- 172.21.15.32:0/4270488524 >> [v2:172.21.15.32:6834/4206823204,v1:172.21.15.32:6835/4206823204] conn(0x5612ae3394d0 0x5612ae34af70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:39.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.137+0000 7f6f721af740 1 --2- 172.21.15.32:0/4270488524 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5612ae366b50 0x5612ae3546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:39.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.137+0000 7f6f721af740 1 --2- 172.21.15.32:0/4270488524 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612ae355b60 0x5612ae366610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:39.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.137+0000 7f6f721af740 1 --2- 172.21.15.32:0/4270488524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612ae338c80 0x5612ae325930 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:39.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.137+0000 7f6f721af740 1 -- 172.21.15.32:0/4270488524 >> 172.21.15.32:0/4270488524 conn(0x5612ae338540 msgr2=0x5612ae356420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:39.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.137+0000 7f6f47fff640 0 client.7929 ms_handle_reset on v2:172.21.15.32:6834/4206823204 2024-04-24T23:05:39.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.137+0000 7f6f721af740 1 -- 172.21.15.32:0/4270488524 shutdown_connections 2024-04-24T23:05:39.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.137+0000 7f6f721af740 1 -- 172.21.15.32:0/4270488524 wait complete. 2024-04-24T23:05:39.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.137+0000 7f6f721af740 20 client.7929 trim_cache size 0 max 16384 2024-04-24T23:05:39.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.155+0000 7f6f721af740 1 Processor -- start 2024-04-24T23:05:39.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.155+0000 7f6f721af740 1 -- start start 2024-04-24T23:05:39.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.156+0000 7f6f721af740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612ae338c80 0x5612ae34b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:39.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.156+0000 7f6f721af740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612ae355b60 0x5612ae354040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:39.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.156+0000 7f6f721af740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5612ae366b50 0x5612ae354580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:39.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.156+0000 7f6f721af740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6f68200cd0 con 0x5612ae355b60 2024-04-24T23:05:39.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.156+0000 7f6f721af740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6f680cc6e0 con 0x5612ae338c80 2024-04-24T23:05:39.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.156+0000 7f6f721af740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f6f680cba40 con 0x5612ae366b50 2024-04-24T23:05:39.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.156+0000 7f6f6ca39640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612ae355b60 0x5612ae354040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:39.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.156+0000 7f6f6ca39640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612ae355b60 0x5612ae354040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56454/0 (socket says 172.21.15.32:56454) 2024-04-24T23:05:39.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.156+0000 7f6f6ca39640 1 -- 172.21.15.32:0/2233271381 learned_addr learned my addr 172.21.15.32:0/2233271381 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:39.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.156+0000 7f6f6da3b640 1 --2- 172.21.15.32:0/2233271381 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5612ae366b50 0x5612ae354580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:39.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.156+0000 7f6f6d23a640 1 --2- 172.21.15.32:0/2233271381 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612ae338c80 0x5612ae34b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:39.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.156+0000 7f6f6ca39640 1 -- 172.21.15.32:0/2233271381 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5612ae366b50 msgr2=0x5612ae354580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:39.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.156+0000 7f6f6ca39640 1 --2- 172.21.15.32:0/2233271381 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5612ae366b50 0x5612ae354580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:39.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.156+0000 7f6f6ca39640 1 -- 172.21.15.32:0/2233271381 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612ae338c80 msgr2=0x5612ae34b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:39.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.156+0000 7f6f6ca39640 1 --2- 172.21.15.32:0/2233271381 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612ae338c80 0x5612ae34b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:39.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.156+0000 7f6f6ca39640 1 -- 172.21.15.32:0/2233271381 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612ae356d50 con 0x5612ae355b60 2024-04-24T23:05:39.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.156+0000 7f6f6ca39640 1 --2- 172.21.15.32:0/2233271381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612ae355b60 0x5612ae354040 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c02a1e0 tx=0x7f6f5c07c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=749ae8b679de681b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:39.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.157+0000 7f6f4e7fc640 1 -- 172.21.15.32:0/2233271381 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c07e0b0 con 0x5612ae355b60 2024-04-24T23:05:39.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.157+0000 7f6f4e7fc640 1 -- 172.21.15.32:0/2233271381 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f5c0a8040 con 0x5612ae355b60 2024-04-24T23:05:39.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.157+0000 7f6f4e7fc640 1 -- 172.21.15.32:0/2233271381 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c079d00 con 0x5612ae355b60 2024-04-24T23:05:39.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.157+0000 7f6f721af740 1 -- 172.21.15.32:0/2233271381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612ae355b60 msgr2=0x5612ae354040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:39.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.157+0000 7f6f721af740 1 --2- 172.21.15.32:0/2233271381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612ae355b60 0x5612ae354040 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c02a1e0 tx=0x7f6f5c07c000 comp rx=0 tx=0).stop 2024-04-24T23:05:39.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.157+0000 7f6f721af740 1 -- 172.21.15.32:0/2233271381 shutdown_connections 2024-04-24T23:05:39.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.157+0000 7f6f721af740 1 --2- 172.21.15.32:0/2233271381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612ae355b60 0x5612ae354040 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:39.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.157+0000 7f6f721af740 1 --2- 172.21.15.32:0/2233271381 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5612ae366b50 0x5612ae354580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:39.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.157+0000 7f6f721af740 1 --2- 172.21.15.32:0/2233271381 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612ae338c80 0x5612ae34b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:39.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.157+0000 7f6f721af740 1 -- 172.21.15.32:0/2233271381 >> 172.21.15.32:0/2233271381 conn(0x5612ae338540 msgr2=0x5612ae33bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:39.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.157+0000 7f6f721af740 1 -- 172.21.15.32:0/2233271381 shutdown_connections 2024-04-24T23:05:39.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.157+0000 7f6f721af740 1 -- 172.21.15.32:0/2233271381 wait complete. 2024-04-24T23:05:39.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.158+0000 7f6f721af740 1 -- start start 2024-04-24T23:05:39.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.158+0000 7f6f721af740 1 Processor -- start 2024-04-24T23:05:39.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.159+0000 7f6f721af740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612ae338c80 0x5612ae348c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:39.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.159+0000 7f6f721af740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612ae355b60 0x5612ae3c6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:39.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.159+0000 7f6f721af740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5612ae366b50 0x5612ae362110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:39.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.159+0000 7f6f721af740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5612ae367370 con 0x5612ae338c80 2024-04-24T23:05:39.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.159+0000 7f6f721af740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6f680cc090 con 0x5612ae355b60 2024-04-24T23:05:39.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.159+0000 7f6f721af740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x5612ae339350 con 0x5612ae366b50 2024-04-24T23:05:39.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.159+0000 7f6f721af740 10 client.0 _ll_register_callbacks cb 0x7f6f58016ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:05:39.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.159+0000 7f6f6d23a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612ae338c80 0x5612ae348c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:39.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.159+0000 7f6f6d23a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612ae338c80 0x5612ae348c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56462/0 (socket says 172.21.15.32:56462) 2024-04-24T23:05:39.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.159+0000 7f6f6d23a640 1 -- 172.21.15.32:0/3889044135 learned_addr learned my addr 172.21.15.32:0/3889044135 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:39.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.159+0000 7f6f6d23a640 1 -- 172.21.15.32:0/3889044135 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5612ae366b50 msgr2=0x5612ae362110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:39.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.159+0000 7f6f6d23a640 1 --2- 172.21.15.32:0/3889044135 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5612ae366b50 0x5612ae362110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:39.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.159+0000 7f6f6d23a640 1 -- 172.21.15.32:0/3889044135 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612ae355b60 msgr2=0x5612ae3c6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:39.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.159+0000 7f6f6d23a640 1 --2- 172.21.15.32:0/3889044135 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612ae355b60 0x5612ae3c6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:39.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.159+0000 7f6f6d23a640 1 -- 172.21.15.32:0/3889044135 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612ae362890 con 0x5612ae338c80 2024-04-24T23:05:39.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.159+0000 7f6f6d23a640 1 --2- 172.21.15.32:0/3889044135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612ae338c80 0x5612ae348c60 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f6f58016e60 tx=0x7f6f58015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=889d65c408950e55 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:39.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.159+0000 7f6f47fff640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-24T23:05:39.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.159+0000 7f6f47fff640 1 -- 172.21.15.32:0/3889044135 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f58036020 con 0x5612ae338c80 2024-04-24T23:05:39.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.160+0000 7f6f47fff640 1 -- 172.21.15.32:0/3889044135 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(140..140 src has 1..140) ==== 5709+0+0 (secure 0 0 0) 0x7f6f58021400 con 0x5612ae338c80 2024-04-24T23:05:39.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.160+0000 7f6f721af740 1 -- 172.21.15.32:0/3889044135 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f5c0a8040 con 0x5612ae338c80 2024-04-24T23:05:39.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.160+0000 7f6f47fff640 1 -- 172.21.15.32:0/3889044135 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f6f58021600 con 0x5612ae338c80 2024-04-24T23:05:39.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.160+0000 7f6f721af740 10 client.7935 fetch_fsmap learned FSMap version 58 2024-04-24T23:05:39.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.160+0000 7f6f721af740 1 -- 172.21.15.32:0/3889044135 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) -- 0x5612ae334f80 con 0x5612ae338c80 2024-04-24T23:05:39.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.160+0000 7f6f47fff640 1 -- 172.21.15.32:0/3889044135 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f5c0a8040 con 0x5612ae338c80 2024-04-24T23:05:39.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.160+0000 7f6f47fff640 1 -- 172.21.15.32:0/3889044135 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 58) ==== 3494+0+0 (secure 0 0 0) 0x7f6f5802a680 con 0x5612ae338c80 2024-04-24T23:05:39.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.160+0000 7f6f721af740 10 client.7935 fetch_fsmap finished waiting for FSMap version 58 2024-04-24T23:05:39.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.160+0000 7f6f721af740 10 client.7935 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T23:05:39.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.160+0000 7f6f721af740 20 client.7935 populate_metadata read hostname 'smithi032' 2024-04-24T23:05:39.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.160+0000 7f6f721af740 1 --2- 172.21.15.32:0/3889044135 >> [v2:172.21.15.32:6834/4206823204,v1:172.21.15.32:6835/4206823204] conn(0x5612ae339b20 0x5612ae339f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:39.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.160+0000 7f6f721af740 4 client.7935 mds_command: new command op to 7794 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:05:39.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.160+0000 7f6f721af740 1 -- 172.21.15.32:0/3889044135 --> [v2:172.21.15.32:6834/4206823204,v1:172.21.15.32:6835/4206823204] -- command(tid 0: {"prefix": "status"}) -- 0x5612ae278790 con 0x5612ae339b20 2024-04-24T23:05:39.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.160+0000 7f6f6ca39640 1 --2- 172.21.15.32:0/3889044135 >> [v2:172.21.15.32:6834/4206823204,v1:172.21.15.32:6835/4206823204] conn(0x5612ae339b20 0x5612ae339f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:39.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.160+0000 7f6f6ca39640 1 --2- 172.21.15.32:0/3889044135 >> [v2:172.21.15.32:6834/4206823204,v1:172.21.15.32:6835/4206823204] conn(0x5612ae339b20 0x5612ae339f00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6124a9d1daab6 server_cookie=424da49ce5aef558 in_seq=0 out_seq=0 2024-04-24T23:05:39.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.160+0000 7f6f47fff640 10 client.7935 ms_handle_connect on v2:172.21.15.32:6834/4206823204 2024-04-24T23:05:39.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.161+0000 7f6f47fff640 1 -- 172.21.15.32:0/3889044135 <== mds.0 v2:172.21.15.32:6834/4206823204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x5612ae278790 con 0x5612ae339b20 2024-04-24T23:05:39.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.161+0000 7f6f47fff640 10 client.7935 handle_command_reply: tid=0 2024-04-24T23:05:39.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.161+0000 7f6f721af740 1 client.7935 shutdown 2024-04-24T23:05:39.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.161+0000 7f6f721af740 1 -- 172.21.15.32:0/3889044135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612ae338c80 msgr2=0x5612ae348c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:39.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.161+0000 7f6f721af740 1 --2- 172.21.15.32:0/3889044135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612ae338c80 0x5612ae348c60 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f6f58016e60 tx=0x7f6f58015000 comp rx=0 tx=0).stop 2024-04-24T23:05:39.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.161+0000 7f6f721af740 1 -- 172.21.15.32:0/3889044135 shutdown_connections 2024-04-24T23:05:39.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.161+0000 7f6f721af740 1 --2- 172.21.15.32:0/3889044135 >> [v2:172.21.15.32:6834/4206823204,v1:172.21.15.32:6835/4206823204] conn(0x5612ae339b20 0x5612ae339f00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:39.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.161+0000 7f6f721af740 1 --2- 172.21.15.32:0/3889044135 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5612ae366b50 0x5612ae362110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:39.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.161+0000 7f6f721af740 1 --2- 172.21.15.32:0/3889044135 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612ae355b60 0x5612ae3c6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:39.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.161+0000 7f6f47fff640 0 client.7935 ms_handle_reset on v2:172.21.15.32:6834/4206823204 2024-04-24T23:05:39.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.161+0000 7f6f721af740 1 --2- 172.21.15.32:0/3889044135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612ae338c80 0x5612ae348c60 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:39.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.161+0000 7f6f721af740 1 -- 172.21.15.32:0/3889044135 >> 172.21.15.32:0/3889044135 conn(0x5612ae338540 msgr2=0x5612ae376c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:39.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.161+0000 7f6f721af740 1 -- 172.21.15.32:0/3889044135 shutdown_connections 2024-04-24T23:05:39.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.161+0000 7f6f721af740 1 -- 172.21.15.32:0/3889044135 wait complete. 2024-04-24T23:05:39.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.161+0000 7f6f721af740 20 client.7935 trim_cache size 0 max 16384 2024-04-24T23:05:39.170 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:05:39.170 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", 2024-04-24T23:05:39.170 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2024-04-24T23:05:39.170 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 7794, 2024-04-24T23:05:39.170 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-04-24T23:05:39.170 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-04-24T23:05:39.170 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-04-24T23:05:39.170 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 4.7421829439999996, 2024-04-24T23:05:39.171 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 58, 2024-04-24T23:05:39.171 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 140, 2024-04-24T23:05:39.171 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 140, 2024-04-24T23:05:39.171 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 9.8136039440000005 2024-04-24T23:05:39.171 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:05:39.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.161+0000 7f6f6f49e640 1 -- 172.21.15.32:0/999456852 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6f5406be60 msgr2=0x7f6f5406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:39.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.161+0000 7f6f6f49e640 1 --2- 172.21.15.32:0/999456852 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6f5406be60 0x7f6f5406e320 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f6f580070b0 tx=0x7f6f58007040 comp rx=0 tx=0).stop 2024-04-24T23:05:39.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.161+0000 7f6f6f49e640 1 -- 172.21.15.32:0/999456852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f680ae140 msgr2=0x7f6f681530c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:39.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.161+0000 7f6f6f49e640 1 --2- 172.21.15.32:0/999456852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f680ae140 0x7f6f681530c0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c000c00 tx=0x7f6f5c0046c0 comp rx=0 tx=0).stop 2024-04-24T23:05:39.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.162+0000 7f6f6f49e640 1 -- 172.21.15.32:0/999456852 shutdown_connections 2024-04-24T23:05:39.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.162+0000 7f6f6f49e640 1 --2- 172.21.15.32:0/999456852 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6f5406be60 0x7f6f5406e320 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:39.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.162+0000 7f6f6f49e640 1 --2- 172.21.15.32:0/999456852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f680ae140 0x7f6f681530c0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:39.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.162+0000 7f6f6f49e640 1 --2- 172.21.15.32:0/999456852 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f6815e3a0 0x7f6f681f4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:39.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.162+0000 7f6f6f49e640 1 --2- 172.21.15.32:0/999456852 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f68074210 0x7f6f680af380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:39.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.162+0000 7f6f6f49e640 1 -- 172.21.15.32:0/999456852 >> 172.21.15.32:0/999456852 conn(0x7f6f680b2e60 msgr2=0x7f6f680aec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:39.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.162+0000 7f6f6f49e640 1 -- 172.21.15.32:0/999456852 shutdown_connections 2024-04-24T23:05:39.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.162+0000 7f6f6f49e640 1 -- 172.21.15.32:0/999456852 wait complete. 2024-04-24T23:05:39.174 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T23:05:39.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.274+0000 7fc1665d5640 1 Processor -- start 2024-04-24T23:05:39.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.274+0000 7fc1665d5640 1 -- start start 2024-04-24T23:05:39.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.275+0000 7fc1665d5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc16015b380 0x7fc16006c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:39.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.275+0000 7fc1665d5640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1600b4520 con 0x7fc16015cb00 2024-04-24T23:05:39.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.275+0000 7fc1665d5640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1600b4800 con 0x7fc1600ceb40 2024-04-24T23:05:39.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.275+0000 7fc1665d5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc1600cef90 con 0x7fc16015b380 2024-04-24T23:05:39.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.275+0000 7fc15f7fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc16015cb00 0x7fc1600ce430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37850/0 (socket says 172.21.15.32:37850) 2024-04-24T23:05:39.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.275+0000 7fc15f7fe640 1 -- 172.21.15.32:0/1448630246 learned_addr learned my addr 172.21.15.32:0/1448630246 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:05:39.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.275+0000 7fc15ffff640 1 --2- 172.21.15.32:0/1448630246 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc16015b380 0x7fc16006c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:39.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.276+0000 7fc15effd640 1 -- 172.21.15.32:0/1448630246 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 945958999 0 0) 0x7fc1600b4520 con 0x7fc16015cb00 2024-04-24T23:05:39.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.276+0000 7fc15effd640 1 -- 172.21.15.32:0/1448630246 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc14c005730 con 0x7fc16015cb00 2024-04-24T23:05:39.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.276+0000 7fc15effd640 1 -- 172.21.15.32:0/1448630246 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1725712602 0 0) 0x7fc1600b4800 con 0x7fc1600ceb40 2024-04-24T23:05:39.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.276+0000 7fc15effd640 1 -- 172.21.15.32:0/1448630246 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc1600b4520 con 0x7fc1600ceb40 2024-04-24T23:05:39.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.276+0000 7fc15ffff640 1 -- 172.21.15.32:0/1448630246 >> v1:172.21.15.121:6789/0 conn(0x7fc1600ceb40 legacy=0x7fc1600b55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:39.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.276+0000 7fc15effd640 1 -- 172.21.15.32:0/1448630246 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2148209148 0 0) 0x7fc14c005730 con 0x7fc16015cb00 2024-04-24T23:05:39.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.276+0000 7fc15ffff640 1 -- 172.21.15.32:0/1448630246 >> v1:172.21.15.32:6789/0 conn(0x7fc16015cb00 legacy=0x7fc1600ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:39.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.276+0000 7fc15ffff640 1 -- 172.21.15.32:0/1448630246 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1600b4ae0 con 0x7fc16015b380 2024-04-24T23:05:39.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.277+0000 7fc15ffff640 1 --2- 172.21.15.32:0/1448630246 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc16015b380 0x7fc16006c5f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc1500097d0 tx=0x7fc15002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8190f7a6a2ce287 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:39.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.277+0000 7fc15effd640 1 -- 172.21.15.32:0/1448630246 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc150037070 con 0x7fc16015b380 2024-04-24T23:05:39.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.277+0000 7fc15effd640 1 -- 172.21.15.32:0/1448630246 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc15002ab90 con 0x7fc16015b380 2024-04-24T23:05:39.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.277+0000 7fc15effd640 1 -- 172.21.15.32:0/1448630246 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc148001670 con 0x7fc16015b380 2024-04-24T23:05:39.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.277+0000 7fc1665d5640 1 -- 172.21.15.32:0/1448630246 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc16015b380 msgr2=0x7fc16006c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:39.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.277+0000 7fc1665d5640 1 --2- 172.21.15.32:0/1448630246 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc16015b380 0x7fc16006c5f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc1500097d0 tx=0x7fc15002a100 comp rx=0 tx=0).stop 2024-04-24T23:05:39.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.277+0000 7fc1665d5640 1 -- 172.21.15.32:0/1448630246 shutdown_connections 2024-04-24T23:05:39.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.277+0000 7fc1665d5640 1 --2- 172.21.15.32:0/1448630246 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc16015b380 0x7fc16006c5f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:39.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.277+0000 7fc1665d5640 1 -- 172.21.15.32:0/1448630246 >> 172.21.15.32:0/1448630246 conn(0x7fc16006b110 msgr2=0x7fc16006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:39.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.277+0000 7fc1665d5640 1 -- 172.21.15.32:0/1448630246 shutdown_connections 2024-04-24T23:05:39.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.277+0000 7fc1665d5640 1 -- 172.21.15.32:0/1448630246 wait complete. 2024-04-24T23:05:39.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.279+0000 7fc1665d5640 1 Processor -- start 2024-04-24T23:05:39.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.279+0000 7fc1665d5640 1 -- start start 2024-04-24T23:05:39.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.279+0000 7fc1665d5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1600ceb40 0x7fc160149170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:39.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.279+0000 7fc1665d5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc16015b380 0x7fc1600b1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:39.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.279+0000 7fc1665d5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc16015cb00 0x7fc1600ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:39.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.279+0000 7fc1665d5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc16015b760 con 0x7fc16015b380 2024-04-24T23:05:39.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.279+0000 7fc1665d5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc1600b50a0 con 0x7fc1600ceb40 2024-04-24T23:05:39.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.279+0000 7fc1665d5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc16006cb30 con 0x7fc16015cb00 2024-04-24T23:05:39.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.280+0000 7fc15f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc16015b380 0x7fc1600b1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:39.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.280+0000 7fc15f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc16015b380 0x7fc1600b1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56476/0 (socket says 172.21.15.32:56476) 2024-04-24T23:05:39.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.280+0000 7fc15f7fe640 1 -- 172.21.15.32:0/508241340 learned_addr learned my addr 172.21.15.32:0/508241340 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:39.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.280+0000 7fc15ffff640 1 --2- 172.21.15.32:0/508241340 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1600ceb40 0x7fc160149170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:39.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.280+0000 7fc164b72640 1 --2- 172.21.15.32:0/508241340 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc16015cb00 0x7fc1600ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:39.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.280+0000 7fc15f7fe640 1 -- 172.21.15.32:0/508241340 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc16015cb00 msgr2=0x7fc1600ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:39.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.280+0000 7fc15f7fe640 1 --2- 172.21.15.32:0/508241340 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc16015cb00 0x7fc1600ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:39.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.280+0000 7fc15f7fe640 1 -- 172.21.15.32:0/508241340 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1600ceb40 msgr2=0x7fc160149170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:39.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.280+0000 7fc15f7fe640 1 --2- 172.21.15.32:0/508241340 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1600ceb40 0x7fc160149170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:39.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.280+0000 7fc15f7fe640 1 -- 172.21.15.32:0/508241340 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc160152b70 con 0x7fc16015b380 2024-04-24T23:05:39.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.280+0000 7fc15f7fe640 1 --2- 172.21.15.32:0/508241340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc16015b380 0x7fc1600b1470 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fc14800c520 tx=0x7fc14800c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:39.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.280+0000 7fc15cff9640 1 -- 172.21.15.32:0/508241340 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc148017070 con 0x7fc16015b380 2024-04-24T23:05:39.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.280+0000 7fc15cff9640 1 -- 172.21.15.32:0/508241340 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc148005ce0 con 0x7fc16015b380 2024-04-24T23:05:39.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.280+0000 7fc1665d5640 1 -- 172.21.15.32:0/508241340 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc160152e50 con 0x7fc16015b380 2024-04-24T23:05:39.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.280+0000 7fc15cff9640 1 -- 172.21.15.32:0/508241340 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1480072b0 con 0x7fc16015b380 2024-04-24T23:05:39.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.280+0000 7fc1665d5640 1 -- 172.21.15.32:0/508241340 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc160070790 con 0x7fc16015b380 2024-04-24T23:05:39.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.281+0000 7fc1665d5640 1 -- 172.21.15.32:0/508241340 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc160068430 con 0x7fc16015b380 2024-04-24T23:05:39.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.282+0000 7fc15cff9640 1 -- 172.21.15.32:0/508241340 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc14801a070 con 0x7fc16015b380 2024-04-24T23:05:39.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.282+0000 7fc15cff9640 1 --2- 172.21.15.32:0/508241340 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc14006be30 0x7fc14006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:39.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.282+0000 7fc15cff9640 1 -- 172.21.15.32:0/508241340 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5709+0+0 (secure 0 0 0) 0x7fc14808a2f0 con 0x7fc16015b380 2024-04-24T23:05:39.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.282+0000 7fc15ffff640 1 --2- 172.21.15.32:0/508241340 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc14006be30 0x7fc14006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:39.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.283+0000 7fc15ffff640 1 --2- 172.21.15.32:0/508241340 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc14006be30 0x7fc14006e2f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc150034000 tx=0x7fc15002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:39.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.287+0000 7fc15cff9640 1 -- 172.21.15.32:0/508241340 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc148058ca0 con 0x7fc16015b380 2024-04-24T23:05:39.396 INFO:tasks.ceph.mds.a.smithi032.stderr:2024-04-24T23:05:39.395+0000 7fb2b9737640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T23:05:39.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.508+0000 7fc1665d5640 1 -- 172.21.15.32:0/508241340 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc1600afd60 con 0x7fc16015b380 2024-04-24T23:05:39.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.508+0000 7fc15cff9640 1 -- 172.21.15.32:0/508241340 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v58) ==== 53+0+73 (secure 0 0 0) 0x7fc1480585a0 con 0x7fc16015b380 2024-04-24T23:05:39.510 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T23:05:39.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.510+0000 7fc1665d5640 1 -- 172.21.15.32:0/508241340 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc14006be30 msgr2=0x7fc14006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:39.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.510+0000 7fc1665d5640 1 --2- 172.21.15.32:0/508241340 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc14006be30 0x7fc14006e2f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc150034000 tx=0x7fc15002e000 comp rx=0 tx=0).stop 2024-04-24T23:05:39.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.510+0000 7fc1665d5640 1 -- 172.21.15.32:0/508241340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc16015b380 msgr2=0x7fc1600b1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:39.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.510+0000 7fc1665d5640 1 --2- 172.21.15.32:0/508241340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc16015b380 0x7fc1600b1470 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fc14800c520 tx=0x7fc14800c9e0 comp rx=0 tx=0).stop 2024-04-24T23:05:39.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.510+0000 7fc1665d5640 1 -- 172.21.15.32:0/508241340 shutdown_connections 2024-04-24T23:05:39.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.510+0000 7fc1665d5640 1 --2- 172.21.15.32:0/508241340 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc14006be30 0x7fc14006e2f0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:39.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.510+0000 7fc1665d5640 1 --2- 172.21.15.32:0/508241340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc16015b380 0x7fc1600b1470 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:39.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.510+0000 7fc1665d5640 1 --2- 172.21.15.32:0/508241340 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc16015cb00 0x7fc1600ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:39.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.510+0000 7fc1665d5640 1 --2- 172.21.15.32:0/508241340 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1600ceb40 0x7fc160149170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:39.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.510+0000 7fc1665d5640 1 -- 172.21.15.32:0/508241340 >> 172.21.15.32:0/508241340 conn(0x7fc16006b110 msgr2=0x7fc1600b3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:39.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.510+0000 7fc1665d5640 1 -- 172.21.15.32:0/508241340 shutdown_connections 2024-04-24T23:05:39.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39.510+0000 7fc1665d5640 1 -- 172.21.15.32:0/508241340 wait complete. 2024-04-24T23:05:39.523 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-24T23:05:39.523 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T23:05:39.523 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2024-04-24T23:05:39.523 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-24T23:05:39.523 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T23:05:39.523 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-24T23:05:39.523 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-24T23:05:39.523 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T23:05:39.523 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:05:39.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:05:39.524 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2024-04-24T23:05:39.539 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T23:05:39.539 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T23:05:39.539 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T23:05:39.539 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:05:39.540 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2024-04-24T23:05:39.540 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:05:39.540 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:05:39.540 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2024-04-24T23:05:39.540 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2024-04-24T23:05:39.540 INFO:teuthology.orchestra.run.smithi032.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:05:39.540 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2024-04-24T23:05:39.540 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2024-04-24T23:05:39.540 INFO:teuthology.orchestra.run.smithi032.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T23:05:39.540 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:05:39.540 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2024-04-24T23:05:39.540 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-24T23:05:39.540 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 42369sec preferred_lft 42369sec 2024-04-24T23:05:39.540 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link noprefixroute 2024-04-24T23:05:39.541 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:05:39.541 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:05:39.541 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:05:39.541 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2024-04-24T23:05:39.541 INFO:teuthology.orchestra.run.smithi032.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T23:05:39.541 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 72:d4:96:00:43:8c brd ff:ff:ff:ff:ff:ff 2024-04-24T23:05:39.541 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-24T23:05:39.541 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:05:39.541 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::f4c6:20ff:fe22:2647/64 scope link 2024-04-24T23:05:39.541 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:05:39.541 INFO:teuthology.orchestra.run.smithi032.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:05:39.541 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 72:d4:96:00:43:8c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T23:05:39.541 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::70d4:96ff:fe00:438c/64 scope link 2024-04-24T23:05:39.541 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:05:39.542 INFO:teuthology.orchestra.run.smithi032.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:05:39.542 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether e6:6b:2a:90:1a:f8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T23:05:39.542 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::e46b:2aff:fe90:1af8/64 scope link 2024-04-24T23:05:39.542 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:05:39.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:05:39.542 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T23:05:39.602 INFO:teuthology.orchestra.run.smithi032.stdout:1 2024-04-24T23:05:39.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:05:39.604 DEBUG:teuthology.orchestra.run.smithi032:> route 2024-04-24T23:05:39.660 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2024-04-24T23:05:39.660 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T23:05:39.660 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-24T23:05:39.660 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-24T23:05:39.660 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T23:05:39.660 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:05:39.660 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-24T23:05:39.660 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-24T23:05:39.661 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-24T23:05:39.661 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-24T23:05:39.661 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-24T23:05:39.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:05:39.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:05:39.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:05:39.786 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-04-24T23:05:39.801 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-24T23:05:39.801 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-24T23:05:39.802 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:05:39.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:05:39.802 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2024-04-24T23:05:39.866 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:05:39.867 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-24T23:05:39.922 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:05:39.923 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:05:39.923 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-24T23:05:39.978 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-24T23:05:39.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:05:39.978 DEBUG:teuthology.orchestra.run.smithi032:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:05:40.061 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:05:40.061 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-24T23:05:40.069 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:05:40.069 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-24T23:05:40.069 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.121:3300,172.21.15.121:3301" 2024-04-24T23:05:40.069 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: trying mount with new device syntax: 0@eb1b7a01-6f65-4534-9a12-83011862ba51.cephfs=/ 2024-04-24T23:05:40.069 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.32:3300/172.21.15.121:3300/172.21.15.121:3301" will pass to kernel 2024-04-24T23:05:40.069 INFO:teuthology.orchestra.run.smithi032.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-24T23:05:40.069 INFO:teuthology.orchestra.run.smithi032.stdout: You just mounted a file system that supports labels which does not 2024-04-24T23:05:40.069 INFO:teuthology.orchestra.run.smithi032.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-24T23:05:40.070 INFO:teuthology.orchestra.run.smithi032.stdout: applications will generate AVC messages and not be allowed access to 2024-04-24T23:05:40.070 INFO:teuthology.orchestra.run.smithi032.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-24T23:05:40.072 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-24T23:05:40.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:05:40.073 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-24T23:05:40.138 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:05:40.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:05:40.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:05:40.239 INFO:teuthology.orchestra.run.smithi032.stdout:client6293 2024-04-24T23:05:40.240 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:05:40.240 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:05:40.256 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T23:05:40.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.395+0000 7f4f003f9640 1 Processor -- start 2024-04-24T23:05:40.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.395+0000 7f4f003f9640 1 -- start start 2024-04-24T23:05:40.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.396+0000 7f4f003f9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4ef815e9b0 0x7f4ef80d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:40.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.396+0000 7f4f003f9640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ef80d3170 con 0x7f4ef80d4210 2024-04-24T23:05:40.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.396+0000 7f4f003f9640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ef80d3450 con 0x7f4ef806b820 2024-04-24T23:05:40.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.396+0000 7f4f003f9640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f4ef80d3730 con 0x7f4ef815e9b0 2024-04-24T23:05:40.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.397+0000 7f4efe195640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4ef815e9b0 0x7f4ef80d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:40.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.397+0000 7f4efd994640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4ef80d4210 0x7f4ef806b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54058/0 (socket says 172.21.15.32:54058) 2024-04-24T23:05:40.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.397+0000 7f4efe195640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4ef815e9b0 0x7f4ef80d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40418/0 (socket says 172.21.15.32:40418) 2024-04-24T23:05:40.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.397+0000 7f4efd994640 1 -- 172.21.15.32:0/1391454813 learned_addr learned my addr 172.21.15.32:0/1391454813 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:05:40.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.397+0000 7f4efe195640 1 -- 172.21.15.32:0/1391454813 >> v1:172.21.15.121:6789/0 conn(0x7f4ef806b820 legacy=0x7f4ef80b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:40.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.397+0000 7f4efe195640 1 -- 172.21.15.32:0/1391454813 >> v1:172.21.15.32:6789/0 conn(0x7f4ef80d4210 legacy=0x7f4ef806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:40.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.397+0000 7f4efe195640 1 -- 172.21.15.32:0/1391454813 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ef80d3a10 con 0x7f4ef815e9b0 2024-04-24T23:05:40.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.397+0000 7f4efd193640 1 -- 172.21.15.32:0/1391454813 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1345311913 0 0) 0x7f4ef80d3170 con 0x7f4ef80d4210 2024-04-24T23:05:40.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.398+0000 7f4efe195640 1 --2- 172.21.15.32:0/1391454813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ef815e9b0 0x7f4ef80d2b90 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f4ef40097a0 tx=0x7f4ef402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=394ded4370f2a9b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:40.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.398+0000 7f4efd193640 1 -- 172.21.15.32:0/1391454813 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef4037070 con 0x7f4ef815e9b0 2024-04-24T23:05:40.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.398+0000 7f4efd193640 1 -- 172.21.15.32:0/1391454813 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ef402ae50 con 0x7f4ef815e9b0 2024-04-24T23:05:40.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.398+0000 7f4efd193640 1 -- 172.21.15.32:0/1391454813 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef4032660 con 0x7f4ef815e9b0 2024-04-24T23:05:40.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.398+0000 7f4f003f9640 1 -- 172.21.15.32:0/1391454813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ef815e9b0 msgr2=0x7f4ef80d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:40.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.398+0000 7f4f003f9640 1 --2- 172.21.15.32:0/1391454813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ef815e9b0 0x7f4ef80d2b90 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f4ef40097a0 tx=0x7f4ef402a140 comp rx=0 tx=0).stop 2024-04-24T23:05:40.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.398+0000 7f4f003f9640 1 -- 172.21.15.32:0/1391454813 shutdown_connections 2024-04-24T23:05:40.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.398+0000 7f4f003f9640 1 --2- 172.21.15.32:0/1391454813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ef815e9b0 0x7f4ef80d2b90 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:40.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.398+0000 7f4f003f9640 1 -- 172.21.15.32:0/1391454813 >> 172.21.15.32:0/1391454813 conn(0x7f4ef80d11d0 msgr2=0x7f4ef80d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:40.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.398+0000 7f4f003f9640 1 -- 172.21.15.32:0/1391454813 shutdown_connections 2024-04-24T23:05:40.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.399+0000 7f4f003f9640 1 -- 172.21.15.32:0/1391454813 wait complete. 2024-04-24T23:05:40.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.400+0000 7f4f003f9640 1 Processor -- start 2024-04-24T23:05:40.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.400+0000 7f4f003f9640 1 -- start start 2024-04-24T23:05:40.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.400+0000 7f4f003f9640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ef806b820 0x7f4ef815aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:40.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.400+0000 7f4f003f9640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ef80d4210 0x7f4ef8153f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:40.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.400+0000 7f4f003f9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ef815e9b0 0x7f4ef815d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:40.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.400+0000 7f4f003f9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef80ba0e0 con 0x7f4ef815e9b0 2024-04-24T23:05:40.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.400+0000 7f4f003f9640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef80d2400 con 0x7f4ef80d4210 2024-04-24T23:05:40.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.400+0000 7f4f003f9640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f4ef80ba260 con 0x7f4ef806b820 2024-04-24T23:05:40.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.401+0000 7f4efe996640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ef815e9b0 0x7f4ef815d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:40.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.401+0000 7f4efe996640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ef815e9b0 0x7f4ef815d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40430/0 (socket says 172.21.15.32:40430) 2024-04-24T23:05:40.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.401+0000 7f4efe996640 1 -- 172.21.15.32:0/1590515483 learned_addr learned my addr 172.21.15.32:0/1590515483 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:40.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.401+0000 7f4efd994640 1 --2- 172.21.15.32:0/1590515483 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ef80d4210 0x7f4ef8153f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:40.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.401+0000 7f4efe195640 1 --2- 172.21.15.32:0/1590515483 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ef806b820 0x7f4ef815aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:40.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.401+0000 7f4efe996640 1 -- 172.21.15.32:0/1590515483 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ef806b820 msgr2=0x7f4ef815aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:40.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.401+0000 7f4efe996640 1 --2- 172.21.15.32:0/1590515483 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ef806b820 0x7f4ef815aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:40.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.401+0000 7f4efe996640 1 -- 172.21.15.32:0/1590515483 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ef80d4210 msgr2=0x7f4ef8153f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:40.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.401+0000 7f4efe996640 1 --2- 172.21.15.32:0/1590515483 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ef80d4210 0x7f4ef8153f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:40.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.401+0000 7f4efe996640 1 -- 172.21.15.32:0/1590515483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ef815ff80 con 0x7f4ef815e9b0 2024-04-24T23:05:40.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.401+0000 7f4efd994640 1 --2- 172.21.15.32:0/1590515483 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ef80d4210 0x7f4ef8153f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:05:40.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.401+0000 7f4efe195640 1 --2- 172.21.15.32:0/1590515483 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ef806b820 0x7f4ef815aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:05:40.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.401+0000 7f4efe996640 1 --2- 172.21.15.32:0/1590515483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ef815e9b0 0x7f4ef815d2f0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f4ef000c7b0 tx=0x7f4ef000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:40.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.401+0000 7f4eeeffd640 1 -- 172.21.15.32:0/1590515483 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef0004b60 con 0x7f4ef815e9b0 2024-04-24T23:05:40.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.401+0000 7f4eeeffd640 1 -- 172.21.15.32:0/1590515483 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ef0004d00 con 0x7f4ef815e9b0 2024-04-24T23:05:40.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.401+0000 7f4eeeffd640 1 -- 172.21.15.32:0/1590515483 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef000aa90 con 0x7f4ef815e9b0 2024-04-24T23:05:40.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.401+0000 7f4f003f9640 1 -- 172.21.15.32:0/1590515483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ef8160260 con 0x7f4ef815e9b0 2024-04-24T23:05:40.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.401+0000 7f4f003f9640 1 -- 172.21.15.32:0/1590515483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ef8160820 con 0x7f4ef815e9b0 2024-04-24T23:05:40.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.402+0000 7f4f003f9640 1 -- 172.21.15.32:0/1590515483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ef8068430 con 0x7f4ef815e9b0 2024-04-24T23:05:40.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.403+0000 7f4eeeffd640 1 -- 172.21.15.32:0/1590515483 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4ef000ac30 con 0x7f4ef815e9b0 2024-04-24T23:05:40.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.403+0000 7f4eeeffd640 1 --2- 172.21.15.32:0/1590515483 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4edc06be30 0x7f4edc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:40.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.403+0000 7f4eeeffd640 1 -- 172.21.15.32:0/1590515483 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5709+0+0 (secure 0 0 0) 0x7f4ef008b100 con 0x7f4ef815e9b0 2024-04-24T23:05:40.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.403+0000 7f4efe195640 1 --2- 172.21.15.32:0/1590515483 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4edc06be30 0x7f4edc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:40.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.404+0000 7f4efe195640 1 --2- 172.21.15.32:0/1590515483 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4edc06be30 0x7f4edc06e2f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f4ef4034000 tx=0x7f4ef402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:40.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.407+0000 7f4eeeffd640 1 -- 172.21.15.32:0/1590515483 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4ef0020070 con 0x7f4ef815e9b0 2024-04-24T23:05:40.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.632+0000 7f4f003f9640 1 -- 172.21.15.32:0/1590515483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f4ef80cadb0 con 0x7f4ef815e9b0 2024-04-24T23:05:40.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.632+0000 7f4eeeffd640 1 -- 172.21.15.32:0/1590515483 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v58) ==== 53+0+73 (secure 0 0 0) 0x7f4ef0059ab0 con 0x7f4ef815e9b0 2024-04-24T23:05:40.634 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T23:05:40.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.634+0000 7f4f003f9640 1 -- 172.21.15.32:0/1590515483 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4edc06be30 msgr2=0x7f4edc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:40.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.634+0000 7f4f003f9640 1 --2- 172.21.15.32:0/1590515483 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4edc06be30 0x7f4edc06e2f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f4ef4034000 tx=0x7f4ef402e000 comp rx=0 tx=0).stop 2024-04-24T23:05:40.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.634+0000 7f4f003f9640 1 -- 172.21.15.32:0/1590515483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ef815e9b0 msgr2=0x7f4ef815d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:40.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.634+0000 7f4f003f9640 1 --2- 172.21.15.32:0/1590515483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ef815e9b0 0x7f4ef815d2f0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f4ef000c7b0 tx=0x7f4ef000cc70 comp rx=0 tx=0).stop 2024-04-24T23:05:40.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.634+0000 7f4f003f9640 1 -- 172.21.15.32:0/1590515483 shutdown_connections 2024-04-24T23:05:40.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.634+0000 7f4f003f9640 1 --2- 172.21.15.32:0/1590515483 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4edc06be30 0x7f4edc06e2f0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:40.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.634+0000 7f4f003f9640 1 --2- 172.21.15.32:0/1590515483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ef815e9b0 0x7f4ef815d2f0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:40.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.634+0000 7f4f003f9640 1 --2- 172.21.15.32:0/1590515483 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ef80d4210 0x7f4ef8153f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:40.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.634+0000 7f4f003f9640 1 --2- 172.21.15.32:0/1590515483 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ef806b820 0x7f4ef815aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:40.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.634+0000 7f4f003f9640 1 -- 172.21.15.32:0/1590515483 >> 172.21.15.32:0/1590515483 conn(0x7f4ef80d11d0 msgr2=0x7f4ef80ca290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:40.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.634+0000 7f4f003f9640 1 -- 172.21.15.32:0/1590515483 shutdown_connections 2024-04-24T23:05:40.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:40.634+0000 7f4f003f9640 1 -- 172.21.15.32:0/1590515483 wait complete. 2024-04-24T23:05:40.646 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-24T23:05:40.646 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T23:05:40.646 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi121.front.sepia.ceph.com 2024-04-24T23:05:40.646 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-24T23:05:40.646 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-24T23:05:40.646 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-24T23:05:40.646 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-24T23:05:40.647 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T23:05:40.647 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-24T23:05:40.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:05:40.647 DEBUG:teuthology.orchestra.run.smithi121:> ip addr 2024-04-24T23:05:40.667 INFO:teuthology.orchestra.run.smithi121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T23:05:40.667 INFO:teuthology.orchestra.run.smithi121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T23:05:40.667 INFO:teuthology.orchestra.run.smithi121.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T23:05:40.668 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:05:40.668 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 ::1/128 scope host 2024-04-24T23:05:40.668 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:05:40.668 INFO:teuthology.orchestra.run.smithi121.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:05:40.668 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:82 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:05:40.669 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f0 2024-04-24T23:05:40.669 INFO:teuthology.orchestra.run.smithi121.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:05:40.669 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:83 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:05:40.669 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f1 2024-04-24T23:05:40.670 INFO:teuthology.orchestra.run.smithi121.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:05:40.670 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a2 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:05:40.670 INFO:teuthology.orchestra.run.smithi121.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T23:05:40.670 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a3 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:05:40.670 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.21.15.121/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-24T23:05:40.671 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft 42390sec preferred_lft 42390sec 2024-04-24T23:05:40.671 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::93dd:8489:4369:e127/64 scope link noprefixroute 2024-04-24T23:05:40.671 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:05:40.671 INFO:teuthology.orchestra.run.smithi121.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T23:05:40.671 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether c6:c0:c7:45:ae:ba brd ff:ff:ff:ff:ff:ff 2024-04-24T23:05:40.672 INFO:teuthology.orchestra.run.smithi121.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-24T23:05:40.672 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:05:40.672 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::80fe:d2ff:fe13:d1/64 scope link 2024-04-24T23:05:40.672 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:05:40.672 INFO:teuthology.orchestra.run.smithi121.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:05:40.672 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether e2:ca:b9:e9:9e:bb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T23:05:40.673 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::e0ca:b9ff:fee9:9ebb/64 scope link 2024-04-24T23:05:40.673 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:05:40.673 INFO:teuthology.orchestra.run.smithi121.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:05:40.673 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether c6:c0:c7:45:ae:ba brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-24T23:05:40.673 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::c4c0:c7ff:fe45:aeba/64 scope link 2024-04-24T23:05:40.674 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:05:40.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:05:40.674 DEBUG:teuthology.orchestra.run.smithi121:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T23:05:40.733 INFO:teuthology.orchestra.run.smithi121.stdout:1 2024-04-24T23:05:40.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:05:40.735 DEBUG:teuthology.orchestra.run.smithi121:> route 2024-04-24T23:05:40.793 INFO:teuthology.orchestra.run.smithi121.stdout:Kernel IP routing table 2024-04-24T23:05:40.793 INFO:teuthology.orchestra.run.smithi121.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T23:05:40.793 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-24T23:05:40.793 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-24T23:05:40.793 INFO:teuthology.orchestra.run.smithi121.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T23:05:40.794 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:05:40.794 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2024-04-24T23:05:40.794 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-24T23:05:40.794 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-24T23:05:40.794 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-24T23:05:40.794 DEBUG:teuthology.orchestra.run.smithi121:> ') 2024-04-24T23:05:40.885 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T23:05:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:05:40.916 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T23:05:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:05:40.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:05:40.922 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2024-04-24T23:05:40.937 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-24T23:05:40.937 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-24T23:05:40.937 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-24T23:05:40.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:05:40.938 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set brx.0 up 2024-04-24T23:05:41.001 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:05:41.001 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-24T23:05:41.056 INFO:teuthology.orchestra.run.smithi121.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-24T23:05:41.056 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:05:41.056 DEBUG:teuthology.orchestra.run.smithi121:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-24T23:05:41.111 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-24T23:05:41.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:05:41.111 DEBUG:teuthology.orchestra.run.smithi121:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:05:41.192 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:05:41.192 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-24T23:05:41.199 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:05:41.199 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-24T23:05:41.199 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.121:3300,172.21.15.121:3301" 2024-04-24T23:05:41.199 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: trying mount with new device syntax: 1@eb1b7a01-6f65-4534-9a12-83011862ba51.cephfs=/ 2024-04-24T23:05:41.199 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.32:3300/172.21.15.121:3300/172.21.15.121:3301" will pass to kernel 2024-04-24T23:05:41.200 INFO:teuthology.orchestra.run.smithi121.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-24T23:05:41.200 INFO:teuthology.orchestra.run.smithi121.stdout: You just mounted a file system that supports labels which does not 2024-04-24T23:05:41.200 INFO:teuthology.orchestra.run.smithi121.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-24T23:05:41.200 INFO:teuthology.orchestra.run.smithi121.stdout: applications will generate AVC messages and not be allowed access to 2024-04-24T23:05:41.201 INFO:teuthology.orchestra.run.smithi121.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-24T23:05:41.202 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-24T23:05:41.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:05:41.202 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-24T23:05:41.266 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:05:41.354 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T23:05:41 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:05:41.356 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T23:05:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:05:41.361 INFO:teuthology.orchestra.run.smithi121.stdout:client6296 2024-04-24T23:05:41.361 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:05:41.361 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:05:41.377 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:05:41.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.493+0000 7fde3de95640 1 Processor -- start 2024-04-24T23:05:41.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.493+0000 7fde3de95640 1 -- start start 2024-04-24T23:05:41.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.494+0000 7fde3de95640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fde3815fd60 0x7fde3806c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:41.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.494+0000 7fde3de95640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fde3806cb30 0x7fde380cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:41.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.494+0000 7fde3de95640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde380b46c0 con 0x7fde3815b480 2024-04-24T23:05:41.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.494+0000 7fde3de95640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fde380b4ff0 con 0x7fde3806cb30 2024-04-24T23:05:41.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.494+0000 7fde3de95640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fde380b49a0 con 0x7fde3815fd60 2024-04-24T23:05:41.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.494+0000 7fde377fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fde3815b480 0x7fde380b5c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54064/0 (socket says 172.21.15.32:54064) 2024-04-24T23:05:41.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.494+0000 7fde377fe640 1 -- 172.21.15.32:0/3952767314 learned_addr learned my addr 172.21.15.32:0/3952767314 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:05:41.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.494+0000 7fde37fff640 1 --2- 172.21.15.32:0/3952767314 >> v2:172.21.15.32:3300/0 conn(0x7fde3806cb30 0x7fde380cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:41.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.494+0000 7fde36ffd640 1 --2- 172.21.15.32:0/3952767314 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fde3815fd60 0x7fde3806c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:41.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.494+0000 7fde367fc640 1 -- 172.21.15.32:0/3952767314 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3060062462 0 0) 0x7fde380b46c0 con 0x7fde3815b480 2024-04-24T23:05:41.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.494+0000 7fde367fc640 1 -- 172.21.15.32:0/3952767314 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde20003520 con 0x7fde3815b480 2024-04-24T23:05:41.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.494+0000 7fde37fff640 1 -- 172.21.15.32:0/3952767314 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fde3815fd60 msgr2=0x7fde3806c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:41.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.494+0000 7fde37fff640 1 --2- 172.21.15.32:0/3952767314 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fde3815fd60 0x7fde3806c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:41.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.494+0000 7fde37fff640 1 -- 172.21.15.32:0/3952767314 >> v1:172.21.15.32:6789/0 conn(0x7fde3815b480 legacy=0x7fde380b5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:41.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.494+0000 7fde37fff640 1 -- 172.21.15.32:0/3952767314 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde380b4c80 con 0x7fde3806cb30 2024-04-24T23:05:41.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.495+0000 7fde37fff640 1 --2- 172.21.15.32:0/3952767314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde3806cb30 0x7fde380cdf80 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fde30009d90 tx=0x7fde3002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=657c996a988788a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:41.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.495+0000 7fde367fc640 1 -- 172.21.15.32:0/3952767314 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde30007740 con 0x7fde3806cb30 2024-04-24T23:05:41.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.495+0000 7fde367fc640 1 -- 172.21.15.32:0/3952767314 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde300078e0 con 0x7fde3806cb30 2024-04-24T23:05:41.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.495+0000 7fde367fc640 1 -- 172.21.15.32:0/3952767314 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde30007cb0 con 0x7fde3806cb30 2024-04-24T23:05:41.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.496+0000 7fde3de95640 1 -- 172.21.15.32:0/3952767314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde3806cb30 msgr2=0x7fde380cdf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:41.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.496+0000 7fde3de95640 1 --2- 172.21.15.32:0/3952767314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde3806cb30 0x7fde380cdf80 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fde30009d90 tx=0x7fde3002a660 comp rx=0 tx=0).stop 2024-04-24T23:05:41.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.496+0000 7fde3de95640 1 -- 172.21.15.32:0/3952767314 shutdown_connections 2024-04-24T23:05:41.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.496+0000 7fde3de95640 1 --2- 172.21.15.32:0/3952767314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde3806cb30 0x7fde380cdf80 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:41.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.496+0000 7fde3de95640 1 --2- 172.21.15.32:0/3952767314 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fde3815fd60 0x7fde3806c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:41.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.496+0000 7fde3de95640 1 -- 172.21.15.32:0/3952767314 >> 172.21.15.32:0/3952767314 conn(0x7fde3806b110 msgr2=0x7fde3806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:41.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.496+0000 7fde3de95640 1 -- 172.21.15.32:0/3952767314 shutdown_connections 2024-04-24T23:05:41.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.496+0000 7fde3de95640 1 -- 172.21.15.32:0/3952767314 wait complete. 2024-04-24T23:05:41.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.497+0000 7fde3de95640 1 Processor -- start 2024-04-24T23:05:41.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.497+0000 7fde3de95640 1 -- start start 2024-04-24T23:05:41.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.498+0000 7fde3de95640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde3806cb30 0x7fde381468a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:41.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.498+0000 7fde3de95640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fde3815b480 0x7fde3814a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:41.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.498+0000 7fde3de95640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fde3815fd60 0x7fde380aeb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:41.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.498+0000 7fde3de95640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fde3815b860 con 0x7fde3806cb30 2024-04-24T23:05:41.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.498+0000 7fde3de95640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fde380ce4c0 con 0x7fde3815fd60 2024-04-24T23:05:41.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.498+0000 7fde3de95640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fde3814a980 con 0x7fde3815b480 2024-04-24T23:05:41.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.498+0000 7fde377fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde3806cb30 0x7fde381468a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:41.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.498+0000 7fde377fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde3806cb30 0x7fde381468a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40442/0 (socket says 172.21.15.32:40442) 2024-04-24T23:05:41.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.498+0000 7fde377fe640 1 -- 172.21.15.32:0/1267337706 learned_addr learned my addr 172.21.15.32:0/1267337706 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:41.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.499+0000 7fde377fe640 1 -- 172.21.15.32:0/1267337706 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fde3815b480 msgr2=0x7fde3814a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:41.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.499+0000 7fde377fe640 1 --2- 172.21.15.32:0/1267337706 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fde3815b480 0x7fde3814a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:41.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.499+0000 7fde377fe640 1 -- 172.21.15.32:0/1267337706 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fde3815fd60 msgr2=0x7fde380aeb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:41.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.499+0000 7fde37fff640 1 --2- 172.21.15.32:0/1267337706 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fde3815fd60 0x7fde380aeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:41.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.499+0000 7fde377fe640 1 --2- 172.21.15.32:0/1267337706 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fde3815fd60 0x7fde380aeb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:41.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.499+0000 7fde377fe640 1 -- 172.21.15.32:0/1267337706 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde381470f0 con 0x7fde3806cb30 2024-04-24T23:05:41.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.499+0000 7fde37fff640 1 --2- 172.21.15.32:0/1267337706 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fde3815fd60 0x7fde380aeb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:05:41.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.499+0000 7fde377fe640 1 --2- 172.21.15.32:0/1267337706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde3806cb30 0x7fde381468a0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fde2800c430 tx=0x7fde2800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:41.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.499+0000 7fde1ffff640 1 -- 172.21.15.32:0/1267337706 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde28017070 con 0x7fde3806cb30 2024-04-24T23:05:41.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.499+0000 7fde1ffff640 1 -- 172.21.15.32:0/1267337706 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde28005ce0 con 0x7fde3806cb30 2024-04-24T23:05:41.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.499+0000 7fde3de95640 1 -- 172.21.15.32:0/1267337706 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde381473d0 con 0x7fde3806cb30 2024-04-24T23:05:41.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.499+0000 7fde1ffff640 1 -- 172.21.15.32:0/1267337706 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde280072f0 con 0x7fde3806cb30 2024-04-24T23:05:41.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.499+0000 7fde3de95640 1 -- 172.21.15.32:0/1267337706 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde38147990 con 0x7fde3806cb30 2024-04-24T23:05:41.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.500+0000 7fde3de95640 1 -- 172.21.15.32:0/1267337706 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde38068430 con 0x7fde3806cb30 2024-04-24T23:05:41.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.501+0000 7fde1ffff640 1 -- 172.21.15.32:0/1267337706 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fde2801a070 con 0x7fde3806cb30 2024-04-24T23:05:41.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.501+0000 7fde1ffff640 1 --2- 172.21.15.32:0/1267337706 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fde2406be30 0x7fde2406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:41.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.501+0000 7fde1ffff640 1 -- 172.21.15.32:0/1267337706 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5709+0+0 (secure 0 0 0) 0x7fde2808a2a0 con 0x7fde3806cb30 2024-04-24T23:05:41.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.502+0000 7fde36ffd640 1 --2- 172.21.15.32:0/1267337706 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fde2406be30 0x7fde2406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:41.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.502+0000 7fde36ffd640 1 --2- 172.21.15.32:0/1267337706 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fde2406be30 0x7fde2406e2f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fde2c006a90 tx=0x7fde2c006a20 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:41.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.506+0000 7fde1ffff640 1 -- 172.21.15.32:0/1267337706 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fde28058ce0 con 0x7fde3806cb30 2024-04-24T23:05:41.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.727+0000 7fde3de95640 1 -- 172.21.15.32:0/1267337706 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fde3815aca0 con 0x7fde3806cb30 2024-04-24T23:05:41.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.728+0000 7fde1ffff640 1 -- 172.21.15.32:0/1267337706 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+5189 (secure 0 0 0) 0x7fde280585e0 con 0x7fde3806cb30 2024-04-24T23:05:41.730 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:05:41.730 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":58,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3361752496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3361752496},{"type":"v1","addr":"172.21.15.121:6837","nonce":3361752496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":6799,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3813453544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3813453544},{"type":"v1","addr":"172.21.15.121:6836","nonce":3813453544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7779,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/1352085651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1352085651},{"type":"v1","addr":"172.21.15.32:6833","nonce":1352085651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:05:33.791068+0000","modified":"2024-04-24T23:05:35.792946+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7794},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7794":{"gid":7794,"name":"a","rank":0,"incarnation":56,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/4206823204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":4206823204},{"type":"v1","addr":"172.21.15.32:6835","nonce":4206823204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7794,"qdb_cluster":[7794]},"id":8}]} 2024-04-24T23:05:41.731 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 58 2024-04-24T23:05:41.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.730+0000 7fde3de95640 1 -- 172.21.15.32:0/1267337706 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fde2406be30 msgr2=0x7fde2406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:41.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.730+0000 7fde3de95640 1 --2- 172.21.15.32:0/1267337706 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fde2406be30 0x7fde2406e2f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fde2c006a90 tx=0x7fde2c006a20 comp rx=0 tx=0).stop 2024-04-24T23:05:41.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.730+0000 7fde3de95640 1 -- 172.21.15.32:0/1267337706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde3806cb30 msgr2=0x7fde381468a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:41.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.730+0000 7fde3de95640 1 --2- 172.21.15.32:0/1267337706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde3806cb30 0x7fde381468a0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fde2800c430 tx=0x7fde2800c8f0 comp rx=0 tx=0).stop 2024-04-24T23:05:41.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.730+0000 7fde3de95640 1 -- 172.21.15.32:0/1267337706 shutdown_connections 2024-04-24T23:05:41.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.730+0000 7fde3de95640 1 --2- 172.21.15.32:0/1267337706 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fde2406be30 0x7fde2406e2f0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:41.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.730+0000 7fde3de95640 1 --2- 172.21.15.32:0/1267337706 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fde3815fd60 0x7fde380aeb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:41.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.730+0000 7fde3de95640 1 --2- 172.21.15.32:0/1267337706 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fde3815b480 0x7fde3814a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:41.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.730+0000 7fde3de95640 1 --2- 172.21.15.32:0/1267337706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde3806cb30 0x7fde381468a0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:41.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.730+0000 7fde3de95640 1 -- 172.21.15.32:0/1267337706 >> 172.21.15.32:0/1267337706 conn(0x7fde3806b110 msgr2=0x7fde3815db80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:41.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.730+0000 7fde3de95640 1 -- 172.21.15.32:0/1267337706 shutdown_connections 2024-04-24T23:05:41.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.730+0000 7fde3de95640 1 -- 172.21.15.32:0/1267337706 wait complete. 2024-04-24T23:05:41.743 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-04-24T23:05:41.862 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","whoami":0,"id":7794,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":7.4337362469999997,"mdsmap_epoch":58,"osdmap_epoch":140,"osdmap_epoch_barrier":140,"uptime":12.505158231999999} 2024-04-24T23:05:41.863 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", "fs_name": "cephfs", "id": 7794, "mdsmap_epoch": 58, "osdmap_epoch": 140, "osdmap_epoch_barrier": 140, "rank_uptime": 7.433736247, "state": "up:active", "uptime": 12.505158232, "want_state": "up:active", "whoami": 0 } 2024-04-24T23:05:41.863 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2024-04-24T23:05:41.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.966+0000 7f9c18bce640 1 Processor -- start 2024-04-24T23:05:41.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.966+0000 7f9c18bce640 1 -- start start 2024-04-24T23:05:41.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.967+0000 7f9c18bce640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9c140b9ff0 0x7f9c140d40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:41.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.967+0000 7f9c18bce640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c140b6450 0x7f9c140b68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:41.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.967+0000 7f9c18bce640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c140d3000 con 0x7f9c1415e770 2024-04-24T23:05:41.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.967+0000 7f9c18bce640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f9c140d32e0 con 0x7f9c140b9ff0 2024-04-24T23:05:41.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.967+0000 7f9c18bce640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f9c140d35c0 con 0x7f9c140b6450 2024-04-24T23:05:41.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.967+0000 7f9c11d9b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9c140b9ff0 0x7f9c140d40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:41.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.967+0000 7f9c1259c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9c1415e770 0x7f9c140d2700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54080/0 (socket says 172.21.15.32:54080) 2024-04-24T23:05:41.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.967+0000 7f9c11d9b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9c140b9ff0 0x7f9c140d40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40450/0 (socket says 172.21.15.32:40450) 2024-04-24T23:05:41.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.968+0000 7f9c11d9b640 1 -- 172.21.15.32:0/4226329220 learned_addr learned my addr 172.21.15.32:0/4226329220 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:41.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.968+0000 7f9c12d9d640 1 --2- 172.21.15.32:0/4226329220 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c140b6450 0x7f9c140b68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:41.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.968+0000 7f9c11d9b640 1 -- 172.21.15.32:0/4226329220 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c140b6450 msgr2=0x7f9c140b68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:41.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.968+0000 7f9c11d9b640 1 --2- 172.21.15.32:0/4226329220 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c140b6450 0x7f9c140b68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:41.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.968+0000 7f9c11d9b640 1 -- 172.21.15.32:0/4226329220 >> v1:172.21.15.32:6789/0 conn(0x7f9c1415e770 legacy=0x7f9c140d2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:41.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.968+0000 7f9c11d9b640 1 -- 172.21.15.32:0/4226329220 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c140d38a0 con 0x7f9c140b9ff0 2024-04-24T23:05:41.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.968+0000 7f9c1159a640 1 -- 172.21.15.32:0/4226329220 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4251328122 0 0) 0x7f9c140d3000 con 0x7f9c1415e770 2024-04-24T23:05:41.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.968+0000 7f9c12d9d640 1 --2- 172.21.15.32:0/4226329220 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c140b6450 0x7f9c140b68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:05:41.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.969+0000 7f9c11d9b640 1 --2- 172.21.15.32:0/4226329220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c140b9ff0 0x7f9c140d40a0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f9c08009870 tx=0x7f9c0802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39ed062888ab3fec server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:41.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.969+0000 7f9c1159a640 1 -- 172.21.15.32:0/4226329220 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c08037070 con 0x7f9c140b9ff0 2024-04-24T23:05:41.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.969+0000 7f9c1159a640 1 -- 172.21.15.32:0/4226329220 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c08032440 con 0x7f9c140b9ff0 2024-04-24T23:05:41.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.969+0000 7f9c1159a640 1 -- 172.21.15.32:0/4226329220 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c08032810 con 0x7f9c140b9ff0 2024-04-24T23:05:41.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.969+0000 7f9c18bce640 1 -- 172.21.15.32:0/4226329220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c140b9ff0 msgr2=0x7f9c140d40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:41.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.969+0000 7f9c18bce640 1 --2- 172.21.15.32:0/4226329220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c140b9ff0 0x7f9c140d40a0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f9c08009870 tx=0x7f9c0802a250 comp rx=0 tx=0).stop 2024-04-24T23:05:41.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.969+0000 7f9c18bce640 1 -- 172.21.15.32:0/4226329220 shutdown_connections 2024-04-24T23:05:41.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.969+0000 7f9c18bce640 1 --2- 172.21.15.32:0/4226329220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c140b9ff0 0x7f9c140d40a0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:41.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.969+0000 7f9c18bce640 1 --2- 172.21.15.32:0/4226329220 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c140b6450 0x7f9c140b68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:41.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.969+0000 7f9c18bce640 1 -- 172.21.15.32:0/4226329220 >> 172.21.15.32:0/4226329220 conn(0x7f9c14071910 msgr2=0x7f9c14071d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:41.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.969+0000 7f9c18bce640 1 -- 172.21.15.32:0/4226329220 shutdown_connections 2024-04-24T23:05:41.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.969+0000 7f9c18bce640 1 -- 172.21.15.32:0/4226329220 wait complete. 2024-04-24T23:05:41.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.971+0000 7f9c18bce640 1 Processor -- start 2024-04-24T23:05:41.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.971+0000 7f9c18bce640 1 -- start start 2024-04-24T23:05:41.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.971+0000 7f9c18bce640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c140b6450 0x7f9c1414fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:41.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.971+0000 7f9c18bce640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c140b9ff0 0x7f9c140aaff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:41.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.971+0000 7f9c18bce640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c1415e770 0x7f9c14150490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:41.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.971+0000 7f9c18bce640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1415ecd0 con 0x7f9c1415e770 2024-04-24T23:05:41.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.971+0000 7f9c18bce640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1415eb50 con 0x7f9c140b9ff0 2024-04-24T23:05:41.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.971+0000 7f9c18bce640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f9c140b7790 con 0x7f9c140b6450 2024-04-24T23:05:41.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.972+0000 7f9c12d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c1415e770 0x7f9c14150490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:41.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.972+0000 7f9c12d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c1415e770 0x7f9c14150490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40454/0 (socket says 172.21.15.32:40454) 2024-04-24T23:05:41.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.972+0000 7f9c12d9d640 1 -- 172.21.15.32:0/1160760262 learned_addr learned my addr 172.21.15.32:0/1160760262 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:41.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.972+0000 7f9c1259c640 1 --2- 172.21.15.32:0/1160760262 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c140b6450 0x7f9c1414fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:41.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.972+0000 7f9c11d9b640 1 --2- 172.21.15.32:0/1160760262 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c140b9ff0 0x7f9c140aaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:41.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.972+0000 7f9c12d9d640 1 -- 172.21.15.32:0/1160760262 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c140b6450 msgr2=0x7f9c1414fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:41.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.972+0000 7f9c12d9d640 1 --2- 172.21.15.32:0/1160760262 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c140b6450 0x7f9c1414fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:41.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.972+0000 7f9c12d9d640 1 -- 172.21.15.32:0/1160760262 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c140b9ff0 msgr2=0x7f9c140aaff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:41.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.972+0000 7f9c12d9d640 1 --2- 172.21.15.32:0/1160760262 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c140b9ff0 0x7f9c140aaff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:41.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.972+0000 7f9c12d9d640 1 -- 172.21.15.32:0/1160760262 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c140ab8d0 con 0x7f9c1415e770 2024-04-24T23:05:41.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.972+0000 7f9c12d9d640 1 --2- 172.21.15.32:0/1160760262 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c1415e770 0x7f9c14150490 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f9c04009fd0 tx=0x7f9c0400d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:41.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.972+0000 7f9c02ffd640 1 -- 172.21.15.32:0/1160760262 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0401a070 con 0x7f9c1415e770 2024-04-24T23:05:41.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.972+0000 7f9c02ffd640 1 -- 172.21.15.32:0/1160760262 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c04003bb0 con 0x7f9c1415e770 2024-04-24T23:05:41.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.972+0000 7f9c02ffd640 1 -- 172.21.15.32:0/1160760262 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c04004030 con 0x7f9c1415e770 2024-04-24T23:05:41.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.972+0000 7f9c18bce640 1 -- 172.21.15.32:0/1160760262 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c140abbb0 con 0x7f9c1415e770 2024-04-24T23:05:41.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.972+0000 7f9c18bce640 1 -- 172.21.15.32:0/1160760262 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c140ac170 con 0x7f9c1415e770 2024-04-24T23:05:41.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.973+0000 7f9c18bce640 1 -- 172.21.15.32:0/1160760262 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c14068430 con 0x7f9c1415e770 2024-04-24T23:05:41.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.974+0000 7f9c02ffd640 1 -- 172.21.15.32:0/1160760262 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9c0401d070 con 0x7f9c1415e770 2024-04-24T23:05:41.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.974+0000 7f9c02ffd640 1 --2- 172.21.15.32:0/1160760262 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9bf406be30 0x7f9bf406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:41.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.974+0000 7f9c02ffd640 1 -- 172.21.15.32:0/1160760262 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5709+0+0 (secure 0 0 0) 0x7f9c0408b090 con 0x7f9c1415e770 2024-04-24T23:05:41.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.974+0000 7f9c1259c640 1 --2- 172.21.15.32:0/1160760262 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9bf406be30 0x7f9bf406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:41.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.975+0000 7f9c1259c640 1 --2- 172.21.15.32:0/1160760262 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9bf406be30 0x7f9bf406e2f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c007fd0 tx=0x7f9c0c007de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:41.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:41.979+0000 7f9c02ffd640 1 -- 172.21.15.32:0/1160760262 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9c04010070 con 0x7f9c1415e770 2024-04-24T23:05:42.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.166+0000 7f9c18bce640 1 -- 172.21.15.32:0/1160760262 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) -- 0x7f9c140673d0 con 0x7f9c1415e770 2024-04-24T23:05:42.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.463+0000 7f9c02ffd640 1 -- 172.21.15.32:0/1160760262 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v141) ==== 181+0+0 (secure 0 0 0) 0x7f9c04059480 con 0x7f9c1415e770 2024-04-24T23:05:42.465 INFO:teuthology.orchestra.run.smithi032.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2024-04-24T23:05:42.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.466+0000 7f9c18bce640 1 -- 172.21.15.32:0/1160760262 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9bf406be30 msgr2=0x7f9bf406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:42.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.466+0000 7f9c18bce640 1 --2- 172.21.15.32:0/1160760262 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9bf406be30 0x7f9bf406e2f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c007fd0 tx=0x7f9c0c007de0 comp rx=0 tx=0).stop 2024-04-24T23:05:42.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.466+0000 7f9c18bce640 1 -- 172.21.15.32:0/1160760262 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c1415e770 msgr2=0x7f9c14150490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:42.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.466+0000 7f9c18bce640 1 --2- 172.21.15.32:0/1160760262 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c1415e770 0x7f9c14150490 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f9c04009fd0 tx=0x7f9c0400d560 comp rx=0 tx=0).stop 2024-04-24T23:05:42.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.466+0000 7f9c18bce640 1 -- 172.21.15.32:0/1160760262 shutdown_connections 2024-04-24T23:05:42.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.466+0000 7f9c18bce640 1 --2- 172.21.15.32:0/1160760262 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9bf406be30 0x7f9bf406e2f0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:42.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.466+0000 7f9c18bce640 1 --2- 172.21.15.32:0/1160760262 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c1415e770 0x7f9c14150490 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:42.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.466+0000 7f9c18bce640 1 --2- 172.21.15.32:0/1160760262 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c140b9ff0 0x7f9c140aaff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:42.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.466+0000 7f9c18bce640 1 --2- 172.21.15.32:0/1160760262 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c140b6450 0x7f9c1414fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:42.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.466+0000 7f9c18bce640 1 -- 172.21.15.32:0/1160760262 >> 172.21.15.32:0/1160760262 conn(0x7f9c14071910 msgr2=0x7f9c140b3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:42.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.466+0000 7f9c18bce640 1 -- 172.21.15.32:0/1160760262 shutdown_connections 2024-04-24T23:05:42.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.466+0000 7f9c18bce640 1 -- 172.21.15.32:0/1160760262 wait complete. 2024-04-24T23:05:42.480 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-04-24T23:05:42.593 INFO:teuthology.orchestra.run.smithi032.stdout:{"osd_mon_report_interval":"5"} 2024-04-24T23:05:42.593 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-04-24T23:05:42.593 INFO:tasks.cephfs.test_full:Writing 32MB should fill this cluster 2024-04-24T23:05:42.593 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:05:42.594 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:05:42.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:05:42.609 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=16 seek=0) 2024-04-24T23:05:42.735 INFO:teuthology.orchestra.run.smithi032.stderr:16+0 records in 2024-04-24T23:05:42.735 INFO:teuthology.orchestra.run.smithi032.stderr:16+0 records out 2024-04-24T23:05:42.735 INFO:teuthology.orchestra.run.smithi032.stderr:16777216 bytes (17 MB, 16 MiB) copied, 0.0656444 s, 256 MB/s 2024-04-24T23:05:42.735 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:05:42.735 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:05:42.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:05:42.751 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=17 seek=0) 2024-04-24T23:05:42.870 INFO:teuthology.orchestra.run.smithi032.stderr:17+0 records in 2024-04-24T23:05:42.871 INFO:teuthology.orchestra.run.smithi032.stderr:17+0 records out 2024-04-24T23:05:42.871 INFO:teuthology.orchestra.run.smithi032.stderr:17825792 bytes (18 MB, 17 MiB) copied, 0.0624639 s, 285 MB/s 2024-04-24T23:05:42.871 INFO:tasks.cephfs.test_full:Writing file B succeeded (full status will happen soon) 2024-04-24T23:05:42.871 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-24T23:05:42.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.970+0000 7f494759e640 1 Processor -- start 2024-04-24T23:05:42.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.970+0000 7f494759e640 1 -- start start 2024-04-24T23:05:42.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.971+0000 7f494759e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f494015bf00 0x7f49400d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:42.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.971+0000 7f494759e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f494015f670 con 0x7f494015c2e0 2024-04-24T23:05:42.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.971+0000 7f494759e640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f494015f950 con 0x7f49400b3ee0 2024-04-24T23:05:42.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.971+0000 7f494759e640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f49400d2140 con 0x7f494015bf00 2024-04-24T23:05:42.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.972+0000 7f494659c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f494015bf00 0x7f49400d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:42.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.972+0000 7f4945d9b640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f494015c2e0 0x7f494015ebb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54084/0 (socket says 172.21.15.32:54084) 2024-04-24T23:05:42.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.972+0000 7f4945d9b640 1 -- 172.21.15.32:0/155768631 learned_addr learned my addr 172.21.15.32:0/155768631 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:05:42.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.972+0000 7f494659c640 1 -- 172.21.15.32:0/155768631 >> v1:172.21.15.121:6789/0 conn(0x7f49400b3ee0 legacy=0x7f49401621a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:42.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.972+0000 7f494659c640 1 -- 172.21.15.32:0/155768631 >> v1:172.21.15.32:6789/0 conn(0x7f494015c2e0 legacy=0x7f494015ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:42.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.972+0000 7f494559a640 1 -- 172.21.15.32:0/155768631 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1315705772 0 0) 0x7f494015f670 con 0x7f494015c2e0 2024-04-24T23:05:42.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.972+0000 7f494659c640 1 -- 172.21.15.32:0/155768631 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49400d2390 con 0x7f494015bf00 2024-04-24T23:05:42.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.973+0000 7f494659c640 1 --2- 172.21.15.32:0/155768631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f494015bf00 0x7f49400d2b90 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f49340097a0 tx=0x7f493402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58adfbf4aa3a9562 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:42.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.973+0000 7f494559a640 1 -- 172.21.15.32:0/155768631 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4934037070 con 0x7f494015bf00 2024-04-24T23:05:42.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.973+0000 7f494559a640 1 -- 172.21.15.32:0/155768631 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4934032440 con 0x7f494015bf00 2024-04-24T23:05:42.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.973+0000 7f494559a640 1 -- 172.21.15.32:0/155768631 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4934032810 con 0x7f494015bf00 2024-04-24T23:05:42.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.973+0000 7f494759e640 1 -- 172.21.15.32:0/155768631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f494015bf00 msgr2=0x7f49400d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:42.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.973+0000 7f494759e640 1 --2- 172.21.15.32:0/155768631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f494015bf00 0x7f49400d2b90 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f49340097a0 tx=0x7f493402a1e0 comp rx=0 tx=0).stop 2024-04-24T23:05:42.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.973+0000 7f494759e640 1 -- 172.21.15.32:0/155768631 shutdown_connections 2024-04-24T23:05:42.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.973+0000 7f494759e640 1 --2- 172.21.15.32:0/155768631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f494015bf00 0x7f49400d2b90 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:42.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.973+0000 7f494759e640 1 -- 172.21.15.32:0/155768631 >> 172.21.15.32:0/155768631 conn(0x7f49400743f0 msgr2=0x7f494006d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:42.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.973+0000 7f494759e640 1 -- 172.21.15.32:0/155768631 shutdown_connections 2024-04-24T23:05:42.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.973+0000 7f494759e640 1 -- 172.21.15.32:0/155768631 wait complete. 2024-04-24T23:05:42.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.975+0000 7f494759e640 1 Processor -- start 2024-04-24T23:05:42.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.975+0000 7f494759e640 1 -- start start 2024-04-24T23:05:42.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.975+0000 7f494759e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f49400b3ee0 0x7f49400cec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:42.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.975+0000 7f494759e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f494015bf00 0x7f49400d0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:42.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.975+0000 7f494759e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f494015c2e0 0x7f49400b1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:42.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.975+0000 7f494759e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f494015f2c0 con 0x7f494015c2e0 2024-04-24T23:05:42.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.975+0000 7f494759e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4940072da0 con 0x7f49400b3ee0 2024-04-24T23:05:42.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.975+0000 7f494759e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f49400b19b0 con 0x7f494015bf00 2024-04-24T23:05:42.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.976+0000 7f4946d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f494015c2e0 0x7f49400b1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:42.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.976+0000 7f4946d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f494015c2e0 0x7f49400b1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40478/0 (socket says 172.21.15.32:40478) 2024-04-24T23:05:42.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.976+0000 7f4946d9d640 1 -- 172.21.15.32:0/2273285434 learned_addr learned my addr 172.21.15.32:0/2273285434 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:42.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.976+0000 7f494659c640 1 --2- 172.21.15.32:0/2273285434 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f49400b3ee0 0x7f49400cec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:42.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.976+0000 7f4945d9b640 1 --2- 172.21.15.32:0/2273285434 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f494015bf00 0x7f49400d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:42.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.976+0000 7f4946d9d640 1 -- 172.21.15.32:0/2273285434 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f494015bf00 msgr2=0x7f49400d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:42.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.976+0000 7f4946d9d640 1 --2- 172.21.15.32:0/2273285434 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f494015bf00 0x7f49400d0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:42.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.976+0000 7f4946d9d640 1 -- 172.21.15.32:0/2273285434 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f49400b3ee0 msgr2=0x7f49400cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:42.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.976+0000 7f4946d9d640 1 --2- 172.21.15.32:0/2273285434 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f49400b3ee0 0x7f49400cec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:42.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.976+0000 7f4946d9d640 1 -- 172.21.15.32:0/2273285434 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49400cf2c0 con 0x7f494015c2e0 2024-04-24T23:05:42.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.976+0000 7f4946d9d640 1 --2- 172.21.15.32:0/2273285434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f494015c2e0 0x7f49400b1470 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f493c00c780 tx=0x7f493c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:42.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.976+0000 7f493affd640 1 -- 172.21.15.32:0/2273285434 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493c004b60 con 0x7f494015c2e0 2024-04-24T23:05:42.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.976+0000 7f493affd640 1 -- 172.21.15.32:0/2273285434 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f493c004d00 con 0x7f494015c2e0 2024-04-24T23:05:42.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.976+0000 7f494759e640 1 -- 172.21.15.32:0/2273285434 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49400cf5a0 con 0x7f494015c2e0 2024-04-24T23:05:42.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.976+0000 7f493affd640 1 -- 172.21.15.32:0/2273285434 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493c00aa90 con 0x7f494015c2e0 2024-04-24T23:05:42.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.976+0000 7f494759e640 1 -- 172.21.15.32:0/2273285434 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49400cfb60 con 0x7f494015c2e0 2024-04-24T23:05:42.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.977+0000 7f494759e640 1 -- 172.21.15.32:0/2273285434 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49400682b0 con 0x7f494015c2e0 2024-04-24T23:05:42.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.978+0000 7f493affd640 1 -- 172.21.15.32:0/2273285434 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f493c00ac30 con 0x7f494015c2e0 2024-04-24T23:05:42.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.978+0000 7f493affd640 1 --2- 172.21.15.32:0/2273285434 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f492406be30 0x7f492406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:42.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.978+0000 7f493affd640 1 -- 172.21.15.32:0/2273285434 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5709+0+0 (secure 0 0 0) 0x7f493c08b000 con 0x7f494015c2e0 2024-04-24T23:05:42.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.978+0000 7f494659c640 1 --2- 172.21.15.32:0/2273285434 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f492406be30 0x7f492406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:42.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.979+0000 7f494659c640 1 --2- 172.21.15.32:0/2273285434 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f492406be30 0x7f492406e2f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f4934034000 tx=0x7f493402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:42.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:42.982+0000 7f493affd640 1 -- 172.21.15.32:0/2273285434 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f493c020070 con 0x7f494015c2e0 2024-04-24T23:05:43.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:43.168+0000 7f494759e640 1 -- 172.21.15.32:0/2273285434 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f49400cfff0 con 0x7f494015c2e0 2024-04-24T23:05:43.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:43.169+0000 7f493affd640 1 -- 172.21.15.32:0/2273285434 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v141) ==== 81+0+34192 (secure 0 0 0) 0x7f493c059360 con 0x7f494015c2e0 2024-04-24T23:05:43.171 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:05:43.171 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:05:43.171 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 141, 2024-04-24T23:05:43.172 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", 2024-04-24T23:05:43.172 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2024-04-24T22:58:46.964851+0000", 2024-04-24T23:05:43.172 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2024-04-24T23:05:42.463180+0000", 2024-04-24T23:05:43.172 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2024-04-24T22:58:52.048787+0000", 2024-04-24T23:05:43.172 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2024-04-24T22:58:49.045609+0000", 2024-04-24T23:05:43.172 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-24T23:05:43.172 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2024-04-24T23:05:43.172 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2024-04-24T23:05:43.172 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2024-04-24T23:05:43.172 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2024-04-24T23:05:43.172 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2024-04-24T23:05:43.172 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2024-04-24T23:05:43.172 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:05:43.172 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 4, 2024-04-24T23:05:43.173 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.69999998807907104, 2024-04-24T23:05:43.173 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-24T23:05:43.173 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-24T23:05:43.173 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2024-04-24T23:05:43.173 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 17, 2024-04-24T23:05:43.173 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2024-04-24T23:05:43.173 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "luminous", 2024-04-24T23:05:43.173 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "jewel", 2024-04-24T23:05:43.173 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "squid", 2024-04-24T23:05:43.173 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_crimson": false, 2024-04-24T23:05:43.173 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2024-04-24T23:05:43.173 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.173 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2024-04-24T23:05:43.173 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": ".mgr", 2024-04-24T23:05:43.174 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T22:58:53.171408+0000", 2024-04-24T23:05:43.174 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:05:43.174 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:05:43.174 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:05:43.174 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:05:43.174 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:05:43.174 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:05:43.174 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:05:43.174 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:05:43.174 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:05:43.174 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:05:43.174 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:05:43.174 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:05:43.174 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2024-04-24T23:05:43.175 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2024-04-24T23:05:43.175 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2024-04-24T23:05:43.175 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2024-04-24T23:05:43.175 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2024-04-24T23:05:43.175 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:05:43.175 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:05:43.175 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:05:43.175 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:05:43.175 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:05:43.175 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:05:43.175 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:05:43.175 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.175 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "113", 2024-04-24T23:05:43.175 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:05:43.176 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:05:43.176 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:05:43.176 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:05:43.176 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:05:43.176 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:05:43.176 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:05:43.176 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:05:43.176 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:05:43.176 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:05:43.176 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:05:43.176 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:05:43.176 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:05:43.176 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:05:43.176 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:05:43.177 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:05:43.177 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:05:43.177 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:05:43.177 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:05:43.177 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:05:43.177 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:05:43.177 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:05:43.177 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:05:43.177 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:05:43.177 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:05:43.177 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:05:43.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.177 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:05:43.177 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:05:43.177 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:05:43.178 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:05:43.178 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:05:43.178 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:05:43.178 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:05:43.178 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:05:43.178 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:05:43.178 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:05:43.178 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:05:43.178 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:05:43.178 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_max": 32, 2024-04-24T23:05:43.178 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2024-04-24T23:05:43.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.178 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:05:43.179 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": {} 2024-04-24T23:05:43.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.179 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:05:43.179 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:05:43.179 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 8, 2024-04-24T23:05:43.179 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 8, 2024-04-24T23:05:43.179 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.25, 2024-04-24T23:05:43.179 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-04-24T23:05:43.179 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-04-24T23:05:43.179 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:05:43.179 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:05:43.179 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:05:43.179 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:43.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.179 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.180 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 16, 2024-04-24T23:05:43.180 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2024-04-24T23:05:43.180 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:05:32.064234+0000", 2024-04-24T23:05:43.180 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:05:43.180 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:05:43.180 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:05:43.180 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:05:43.180 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:05:43.180 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:05:43.180 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:05:43.180 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:05:43.180 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:05:43.180 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:05:43.180 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:05:43.181 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:05:43.181 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 32, 2024-04-24T23:05:43.181 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 32, 2024-04-24T23:05:43.181 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 32, 2024-04-24T23:05:43.181 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 32, 2024-04-24T23:05:43.181 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 32, 2024-04-24T23:05:43.181 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:05:43.181 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:05:43.181 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:05:43.181 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:05:43.181 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:05:43.181 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:05:43.181 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:05:43.181 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.181 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "139", 2024-04-24T23:05:43.182 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:05:43.182 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:05:43.182 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:05:43.182 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:05:43.182 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:05:43.182 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:05:43.182 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:05:43.182 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:05:43.182 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:05:43.182 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:05:43.182 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:05:43.182 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:05:43.182 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:05:43.182 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:05:43.183 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:05:43.183 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:05:43.183 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:05:43.183 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:05:43.183 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:05:43.183 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:05:43.183 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:05:43.183 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:05:43.183 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:05:43.183 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:05:43.183 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:05:43.183 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:05:43.183 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.183 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:05:43.184 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:05:43.184 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:05:43.184 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:05:43.184 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:05:43.184 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:05:43.184 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:05:43.184 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:05:43.184 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:05:43.184 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:05:43.184 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:05:43.184 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:05:43.184 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2024-04-24T23:05:43.184 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2024-04-24T23:05:43.184 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2024-04-24T23:05:43.185 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.185 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:05:43.185 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:05:43.185 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2024-04-24T23:05:43.185 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:43.185 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.185 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:05:43.185 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:05:43.185 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 2.25, 2024-04-24T23:05:43.185 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 2.25, 2024-04-24T23:05:43.185 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:05:43.185 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2.25, 2024-04-24T23:05:43.185 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2.25, 2024-04-24T23:05:43.185 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:05:43.185 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:05:43.186 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:05:43.186 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:43.186 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.186 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.186 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 17, 2024-04-24T23:05:43.186 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2024-04-24T23:05:43.186 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:05:32.760777+0000", 2024-04-24T23:05:43.186 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:05:43.186 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:05:43.186 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:05:43.186 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:05:43.186 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:05:43.187 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:05:43.187 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:05:43.187 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:05:43.187 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:05:43.187 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:05:43.187 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:05:43.187 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:05:43.187 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 64, 2024-04-24T23:05:43.187 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 64, 2024-04-24T23:05:43.187 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 64, 2024-04-24T23:05:43.187 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 64, 2024-04-24T23:05:43.187 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 64, 2024-04-24T23:05:43.187 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:05:43.187 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:05:43.188 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:05:43.188 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:05:43.188 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:05:43.188 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:05:43.188 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:05:43.188 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.188 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "141", 2024-04-24T23:05:43.188 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:05:43.188 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:05:43.188 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:05:43.188 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:05:43.188 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:05:43.188 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:05:43.188 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:05:43.188 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:05:43.189 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:05:43.189 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 33554432, 2024-04-24T23:05:43.189 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:05:43.189 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:05:43.189 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:05:43.189 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:05:43.189 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:05:43.189 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:05:43.189 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:05:43.189 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:05:43.189 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:05:43.189 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:05:43.189 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:05:43.189 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:05:43.189 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:05:43.190 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:05:43.190 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:05:43.190 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:05:43.190 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.190 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:05:43.190 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:05:43.190 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:05:43.190 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:05:43.190 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:05:43.190 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:05:43.190 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:05:43.190 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:05:43.190 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:05:43.190 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:05:43.191 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:05:43.191 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:05:43.191 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 64, 2024-04-24T23:05:43.191 INFO:teuthology.orchestra.run.smithi032.stdout: "target_size_ratio": 0.89999997615814209 2024-04-24T23:05:43.191 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.191 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:05:43.191 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:05:43.191 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2024-04-24T23:05:43.191 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:43.191 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.191 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:05:43.191 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:05:43.191 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.3799999952316284, 2024-04-24T23:05:43.191 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.3799999952316284, 2024-04-24T23:05:43.191 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:05:43.192 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.3799999952316284, 2024-04-24T23:05:43.192 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.3799999952316284, 2024-04-24T23:05:43.192 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:05:43.192 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:05:43.192 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:05:43.192 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:43.192 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:43.192 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:05:43.192 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-04-24T23:05:43.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.192 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:05:43.192 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:05:43.192 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:05:43.192 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:05:43.192 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:05:43.193 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:05:43.193 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:05:43.193 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:05:43.193 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:05:43.193 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 138, 2024-04-24T23:05:43.193 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:05:43.193 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:05:43.193 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:05:43.193 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:43.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.193 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:43.193 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2024-04-24T23:05:43.193 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:05:43.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.194 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.194 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:43.194 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2024-04-24T23:05:43.194 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:05:43.194 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:43.194 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:43.194 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.194 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:05:43.194 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:43.194 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.194 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:43.194 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2024-04-24T23:05:43.194 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:05:43.194 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.195 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.195 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:43.195 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2024-04-24T23:05:43.195 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:05:43.195 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:43.195 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:43.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.195 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:05:43.195 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:43.195 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.195 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:43.195 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2024-04-24T23:05:43.195 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:05:43.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.195 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.196 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:43.196 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2024-04-24T23:05:43.196 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:05:43.196 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:43.196 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:43.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.196 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:05:43.196 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:43.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.196 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:43.196 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2024-04-24T23:05:43.196 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:05:43.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.197 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:43.197 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2024-04-24T23:05:43.197 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:05:43.197 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:43.197 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:43.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.197 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/1479395426", 2024-04-24T23:05:43.197 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/1479395426", 2024-04-24T23:05:43.197 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/1479395426", 2024-04-24T23:05:43.197 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/1479395426", 2024-04-24T23:05:43.197 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:05:43.197 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:05:43.197 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:05:43.197 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:43.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.198 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.198 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:05:43.198 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:05:43.198 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:05:43.198 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:05:43.198 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:05:43.198 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:05:43.198 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:05:43.198 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:05:43.198 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:05:43.198 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 138, 2024-04-24T23:05:43.198 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:05:43.198 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:05:43.199 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:05:43.199 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:43.199 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.199 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:43.199 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2024-04-24T23:05:43.199 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:05:43.199 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.199 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.199 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:43.199 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2024-04-24T23:05:43.199 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:05:43.199 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:43.199 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:43.199 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.199 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:05:43.200 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:43.200 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.200 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:43.200 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2024-04-24T23:05:43.200 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:05:43.200 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.200 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.200 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:43.200 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2024-04-24T23:05:43.200 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:05:43.200 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:43.200 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:43.200 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.200 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:05:43.201 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:43.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.201 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:43.201 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2024-04-24T23:05:43.201 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:05:43.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.201 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:43.201 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2024-04-24T23:05:43.201 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:05:43.201 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:43.201 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:43.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.201 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:05:43.201 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:43.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.202 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:43.202 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2024-04-24T23:05:43.202 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:05:43.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.202 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:43.202 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2024-04-24T23:05:43.202 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:05:43.202 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:43.202 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:43.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.202 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/1340633718", 2024-04-24T23:05:43.202 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/1340633718", 2024-04-24T23:05:43.203 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/1340633718", 2024-04-24T23:05:43.203 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/1340633718", 2024-04-24T23:05:43.203 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:05:43.203 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:05:43.203 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:05:43.203 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:43.203 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.203 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.203 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:05:43.203 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:05:43.203 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:05:43.203 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:05:43.203 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:05:43.203 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:05:43.203 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:05:43.204 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:05:43.204 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:05:43.204 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 138, 2024-04-24T23:05:43.204 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:05:43.204 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:05:43.204 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:05:43.204 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:43.204 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.204 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:43.204 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2024-04-24T23:05:43.204 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:05:43.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.204 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.204 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:43.205 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2024-04-24T23:05:43.205 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:05:43.205 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:43.205 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:43.205 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.205 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:05:43.205 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:43.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.205 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:43.205 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2024-04-24T23:05:43.205 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:05:43.205 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.205 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:43.206 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2024-04-24T23:05:43.206 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:05:43.206 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:43.206 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:43.206 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.206 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:05:43.206 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:43.206 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.206 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:43.206 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2024-04-24T23:05:43.206 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:05:43.206 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.206 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.206 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:43.206 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2024-04-24T23:05:43.207 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:05:43.207 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:43.207 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:43.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.207 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:05:43.207 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:43.207 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.207 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:43.207 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2024-04-24T23:05:43.207 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:05:43.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.207 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.207 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:43.207 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2024-04-24T23:05:43.208 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:05:43.208 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:43.208 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:43.208 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.208 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/3405899334", 2024-04-24T23:05:43.208 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6820/3405899334", 2024-04-24T23:05:43.208 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6828/3405899334", 2024-04-24T23:05:43.208 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6824/3405899334", 2024-04-24T23:05:43.208 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:05:43.208 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:05:43.208 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:05:43.208 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:43.208 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.208 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.208 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:05:43.209 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:05:43.209 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:05:43.209 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:05:43.209 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:05:43.209 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:05:43.209 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:05:43.209 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:05:43.209 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:05:43.209 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 138, 2024-04-24T23:05:43.209 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:05:43.209 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:05:43.209 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:05:43.209 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:43.209 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.209 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:43.210 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2024-04-24T23:05:43.210 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:05:43.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.210 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.210 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:43.210 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2024-04-24T23:05:43.210 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:05:43.210 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:43.210 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:43.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.210 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:05:43.211 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:43.211 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.211 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:43.211 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2024-04-24T23:05:43.211 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:05:43.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.211 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.211 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:43.211 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2024-04-24T23:05:43.211 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:05:43.211 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:43.211 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:43.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.211 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:05:43.212 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:43.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.212 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:43.212 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2024-04-24T23:05:43.212 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:05:43.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.212 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:43.212 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2024-04-24T23:05:43.212 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:05:43.212 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:43.212 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:43.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.212 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:05:43.213 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:43.213 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.213 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:43.213 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2024-04-24T23:05:43.213 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:05:43.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.213 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.213 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:43.213 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2024-04-24T23:05:43.213 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:05:43.213 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:43.213 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:43.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.213 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6821/3381057190", 2024-04-24T23:05:43.214 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6825/3381057190", 2024-04-24T23:05:43.214 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/3381057190", 2024-04-24T23:05:43.214 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/3381057190", 2024-04-24T23:05:43.214 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:05:43.214 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:05:43.214 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:05:43.214 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:43.214 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.214 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.214 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:05:43.214 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:05:43.214 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:05:43.214 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:05:43.214 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:05:43.215 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:05:43.215 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:05:43.215 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:05:43.215 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:05:43.215 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 138, 2024-04-24T23:05:43.215 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:05:43.215 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:05:43.215 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:05:43.215 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:43.215 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.215 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:43.215 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6800", 2024-04-24T23:05:43.215 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:05:43.215 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.215 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.216 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:43.216 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6801", 2024-04-24T23:05:43.216 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:05:43.216 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:43.216 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:43.216 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.216 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:05:43.216 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:43.216 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.216 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:43.216 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6802", 2024-04-24T23:05:43.216 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:05:43.216 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.216 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.216 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:43.217 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6803", 2024-04-24T23:05:43.217 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:05:43.217 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:43.217 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:43.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.217 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:05:43.217 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:43.217 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.217 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:43.217 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6806", 2024-04-24T23:05:43.217 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:05:43.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.217 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.217 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:43.217 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6807", 2024-04-24T23:05:43.218 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:05:43.218 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:43.218 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:43.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.218 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:05:43.218 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:43.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.218 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:43.218 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6804", 2024-04-24T23:05:43.218 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:05:43.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.218 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:43.218 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6805", 2024-04-24T23:05:43.218 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:05:43.219 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:43.219 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:43.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.219 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6801/4010553979", 2024-04-24T23:05:43.219 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6803/4010553979", 2024-04-24T23:05:43.219 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6807/4010553979", 2024-04-24T23:05:43.219 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6805/4010553979", 2024-04-24T23:05:43.219 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:05:43.219 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:05:43.219 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:05:43.219 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:43.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.219 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.219 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:05:43.220 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:05:43.220 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:05:43.220 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:05:43.220 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:05:43.220 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:05:43.220 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:05:43.220 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:05:43.220 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:05:43.220 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 138, 2024-04-24T23:05:43.220 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:05:43.220 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:05:43.220 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:05:43.220 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:43.220 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.221 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:43.221 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6808", 2024-04-24T23:05:43.221 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:05:43.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.221 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:43.221 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6809", 2024-04-24T23:05:43.221 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:05:43.221 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:43.221 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:43.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.221 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:05:43.221 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:43.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.221 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:43.222 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6810", 2024-04-24T23:05:43.222 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:05:43.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.222 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.222 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:43.222 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6811", 2024-04-24T23:05:43.222 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:05:43.222 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:43.222 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:43.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.222 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:05:43.222 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:43.222 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.222 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:43.223 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6814", 2024-04-24T23:05:43.223 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:05:43.223 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.223 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.223 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:43.223 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6815", 2024-04-24T23:05:43.223 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:05:43.223 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:43.223 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:43.223 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.223 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:05:43.223 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:43.223 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.223 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:43.224 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6812", 2024-04-24T23:05:43.224 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:05:43.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.224 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.224 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:43.224 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6813", 2024-04-24T23:05:43.224 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:05:43.224 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:43.224 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:43.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.224 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6809/3794084091", 2024-04-24T23:05:43.224 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6811/3794084091", 2024-04-24T23:05:43.224 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6815/3794084091", 2024-04-24T23:05:43.224 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6813/3794084091", 2024-04-24T23:05:43.225 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:05:43.225 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:05:43.225 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:05:43.225 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:43.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.225 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:05:43.225 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:05:43.225 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:05:43.225 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:05:43.225 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:05:43.225 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:05:43.225 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:05:43.225 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:05:43.225 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:05:43.226 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 138, 2024-04-24T23:05:43.226 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:05:43.226 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:05:43.226 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:05:43.226 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:43.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.226 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:43.226 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6823", 2024-04-24T23:05:43.226 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:05:43.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.226 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:43.226 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6825", 2024-04-24T23:05:43.226 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:05:43.227 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:43.227 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:43.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.227 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:05:43.227 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:43.227 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.227 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:43.227 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6826", 2024-04-24T23:05:43.227 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:05:43.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.227 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.227 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:43.227 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6827", 2024-04-24T23:05:43.227 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:05:43.228 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:43.228 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:43.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.228 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:05:43.228 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:43.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.228 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:43.228 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6830", 2024-04-24T23:05:43.228 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:05:43.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.228 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:43.228 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6831", 2024-04-24T23:05:43.228 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:05:43.228 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:43.229 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:43.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.229 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:05:43.229 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:43.229 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.229 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:43.229 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6828", 2024-04-24T23:05:43.229 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:05:43.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.229 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.229 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:43.229 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6829", 2024-04-24T23:05:43.229 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:05:43.230 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:43.230 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:43.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.230 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6825/1348640258", 2024-04-24T23:05:43.230 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6827/1348640258", 2024-04-24T23:05:43.230 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6831/1348640258", 2024-04-24T23:05:43.230 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6829/1348640258", 2024-04-24T23:05:43.230 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:05:43.230 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:05:43.230 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:05:43.230 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:43.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.230 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:05:43.231 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:05:43.231 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:05:43.231 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:05:43.231 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:05:43.231 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:05:43.231 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:05:43.231 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:05:43.231 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:05:43.231 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 138, 2024-04-24T23:05:43.231 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:05:43.231 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:05:43.231 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:05:43.231 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:43.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.231 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:43.232 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6816", 2024-04-24T23:05:43.232 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:05:43.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.232 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.232 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:43.232 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6817", 2024-04-24T23:05:43.232 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:05:43.232 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:43.232 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:43.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.232 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:05:43.232 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:43.232 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.232 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:43.232 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6818", 2024-04-24T23:05:43.233 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:05:43.233 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.233 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.233 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:43.233 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6819", 2024-04-24T23:05:43.233 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:05:43.233 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:43.233 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:43.233 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.233 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:05:43.233 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:43.233 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.233 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:43.233 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6822", 2024-04-24T23:05:43.234 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:05:43.234 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.234 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.234 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:43.234 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6824", 2024-04-24T23:05:43.234 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:05:43.234 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:43.234 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:43.234 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.234 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:05:43.234 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:43.234 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.234 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:43.235 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6820", 2024-04-24T23:05:43.235 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:05:43.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.235 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:43.235 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6821", 2024-04-24T23:05:43.235 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:05:43.235 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:43.235 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:43.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.235 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6817/3250005122", 2024-04-24T23:05:43.235 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6819/3250005122", 2024-04-24T23:05:43.235 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6824/3250005122", 2024-04-24T23:05:43.235 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6821/3250005122", 2024-04-24T23:05:43.236 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:05:43.236 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:05:43.236 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:05:43.236 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:43.236 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:43.236 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:05:43.236 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2024-04-24T23:05:43.236 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.236 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:05:43.236 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:05:43.236 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:05:43.236 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:05:43.236 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:05:43.236 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:05:43.236 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.156184+0000", 2024-04-24T23:05:43.236 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:05:43.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.237 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.237 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:05:43.237 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:05:43.237 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:05:43.237 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:05:43.237 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:05:43.237 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:05:43.237 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240774+0000", 2024-04-24T23:05:43.237 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:05:43.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.237 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.237 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:05:43.237 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:05:43.237 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:05:43.238 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:05:43.238 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:05:43.238 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:05:43.238 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240949+0000", 2024-04-24T23:05:43.238 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:05:43.238 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.238 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.238 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:05:43.238 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:05:43.238 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:05:43.238 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:05:43.238 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:05:43.238 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:05:43.238 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.171672+0000", 2024-04-24T23:05:43.239 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:05:43.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.239 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.239 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:05:43.239 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:05:43.239 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:05:43.239 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:05:43.239 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:05:43.239 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:05:43.239 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.220379+0000", 2024-04-24T23:05:43.239 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:05:43.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.239 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.239 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:05:43.239 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:05:43.240 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:05:43.240 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:05:43.240 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:05:43.240 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:05:43.240 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.204963+0000", 2024-04-24T23:05:43.240 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:05:43.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.240 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.240 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:05:43.240 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:05:43.240 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:05:43.240 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:05:43.240 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:05:43.241 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:05:43.241 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.239748+0000", 2024-04-24T23:05:43.241 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:05:43.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.241 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:43.241 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:05:43.241 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:05:43.241 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:05:43.241 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:05:43.241 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:05:43.241 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:05:43.241 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.154104+0000", 2024-04-24T23:05:43.241 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:05:43.241 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:43.241 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:05:43.242 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2024-04-24T23:05:43.242 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2024-04-24T23:05:43.242 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_primaries": [], 2024-04-24T23:05:43.242 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2024-04-24T23:05:43.242 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2024-04-24T23:05:43.242 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklist": {}, 2024-04-24T23:05:43.242 INFO:teuthology.orchestra.run.smithi032.stdout: "range_blocklist": {}, 2024-04-24T23:05:43.242 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2024-04-24T23:05:43.242 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2024-04-24T23:05:43.242 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2024-04-24T23:05:43.242 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2024-04-24T23:05:43.242 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2024-04-24T23:05:43.242 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2024-04-24T23:05:43.242 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2024-04-24T23:05:43.243 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:43.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:43.243 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2024-04-24T23:05:43.243 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2024-04-24T23:05:43.243 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2024-04-24T23:05:43.243 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2024-04-24T23:05:43.243 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {}, 2024-04-24T23:05:43.243 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode": { 2024-04-24T23:05:43.243 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_enabled": false, 2024-04-24T23:05:43.243 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_bucket_count": 0, 2024-04-24T23:05:43.243 INFO:teuthology.orchestra.run.smithi032.stdout: "degraded_stretch_mode": 0, 2024-04-24T23:05:43.243 INFO:teuthology.orchestra.run.smithi032.stdout: "recovering_stretch_mode": 0, 2024-04-24T23:05:43.243 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_bucket": 0 2024-04-24T23:05:43.243 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:43.244 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:05:43.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:43.171+0000 7f494759e640 1 -- 172.21.15.32:0/2273285434 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f492406be30 msgr2=0x7f492406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:43.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:43.171+0000 7f494759e640 1 --2- 172.21.15.32:0/2273285434 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f492406be30 0x7f492406e2f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f4934034000 tx=0x7f493402e000 comp rx=0 tx=0).stop 2024-04-24T23:05:43.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:43.171+0000 7f494759e640 1 -- 172.21.15.32:0/2273285434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f494015c2e0 msgr2=0x7f49400b1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:43.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:43.171+0000 7f494759e640 1 --2- 172.21.15.32:0/2273285434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f494015c2e0 0x7f49400b1470 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f493c00c780 tx=0x7f493c00cc40 comp rx=0 tx=0).stop 2024-04-24T23:05:43.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:43.171+0000 7f494759e640 1 -- 172.21.15.32:0/2273285434 shutdown_connections 2024-04-24T23:05:43.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:43.171+0000 7f494759e640 1 --2- 172.21.15.32:0/2273285434 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f492406be30 0x7f492406e2f0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:43.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:43.171+0000 7f494759e640 1 --2- 172.21.15.32:0/2273285434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f494015c2e0 0x7f49400b1470 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:43.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:43.171+0000 7f494759e640 1 --2- 172.21.15.32:0/2273285434 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f494015bf00 0x7f49400d0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:43.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:43.171+0000 7f494759e640 1 --2- 172.21.15.32:0/2273285434 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f49400b3ee0 0x7f49400cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:43.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:43.171+0000 7f494759e640 1 -- 172.21.15.32:0/2273285434 >> 172.21.15.32:0/2273285434 conn(0x7f49400743f0 msgr2=0x7f494006d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:43.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:43.171+0000 7f494759e640 1 -- 172.21.15.32:0/2273285434 shutdown_connections 2024-04-24T23:05:43.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:43.171+0000 7f494759e640 1 -- 172.21.15.32:0/2273285434 wait complete. 2024-04-24T23:05:43.247 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-04-24T23:05:48.248 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-24T23:05:48.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.360+0000 7f0fe9a9a640 1 Processor -- start 2024-04-24T23:05:48.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.360+0000 7f0fe9a9a640 1 -- start start 2024-04-24T23:05:48.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.361+0000 7f0fe9a9a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fe415b380 0x7f0fe406c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:48.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.361+0000 7f0fe9a9a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0fe415cb00 0x7f0fe406cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:48.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.361+0000 7f0fe9a9a640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fe40b4520 con 0x7f0fe40ce430 2024-04-24T23:05:48.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.361+0000 7f0fe9a9a640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f0fe40cef90 con 0x7f0fe415cb00 2024-04-24T23:05:48.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.361+0000 7f0fe9a9a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f0fe40b4800 con 0x7f0fe415b380 2024-04-24T23:05:48.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.361+0000 7f0fe2ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0fe415cb00 0x7f0fe406cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:48.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.361+0000 7f0fe3fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0fe40ce430 0x7f0fe40ce880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54092/0 (socket says 172.21.15.32:54092) 2024-04-24T23:05:48.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.361+0000 7f0fe3fff640 1 -- 172.21.15.32:0/3329172677 learned_addr learned my addr 172.21.15.32:0/3329172677 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:05:48.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.361+0000 7f0fe37fe640 1 --2- 172.21.15.32:0/3329172677 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fe415b380 0x7f0fe406c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:48.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.361+0000 7f0fe2ffd640 1 -- 172.21.15.32:0/3329172677 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fe415b380 msgr2=0x7f0fe406c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:48.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.361+0000 7f0fe2ffd640 1 --2- 172.21.15.32:0/3329172677 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fe415b380 0x7f0fe406c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:48.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.361+0000 7f0fe2ffd640 1 -- 172.21.15.32:0/3329172677 >> v1:172.21.15.32:6789/0 conn(0x7f0fe40ce430 legacy=0x7f0fe40ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:48.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.361+0000 7f0fe2ffd640 1 -- 172.21.15.32:0/3329172677 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fe40b4ae0 con 0x7f0fe415cb00 2024-04-24T23:05:48.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.361+0000 7f0fe27fc640 1 -- 172.21.15.32:0/3329172677 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 936621454 0 0) 0x7f0fe40b4520 con 0x7f0fe40ce430 2024-04-24T23:05:48.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.362+0000 7f0fe2ffd640 1 --2- 172.21.15.32:0/3329172677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fe415cb00 0x7f0fe406cb30 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f0fd80097a0 tx=0x7f0fd802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a2c63fca8006f2b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:48.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.362+0000 7f0fe27fc640 1 -- 172.21.15.32:0/3329172677 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd8037070 con 0x7f0fe415cb00 2024-04-24T23:05:48.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.362+0000 7f0fe27fc640 1 -- 172.21.15.32:0/3329172677 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fd802ae50 con 0x7f0fe415cb00 2024-04-24T23:05:48.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.362+0000 7f0fe27fc640 1 -- 172.21.15.32:0/3329172677 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd8032660 con 0x7f0fe415cb00 2024-04-24T23:05:48.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.363+0000 7f0fe9a9a640 1 -- 172.21.15.32:0/3329172677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fe415cb00 msgr2=0x7f0fe406cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:48.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.363+0000 7f0fe9a9a640 1 --2- 172.21.15.32:0/3329172677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fe415cb00 0x7f0fe406cb30 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f0fd80097a0 tx=0x7f0fd802a100 comp rx=0 tx=0).stop 2024-04-24T23:05:48.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.363+0000 7f0fe9a9a640 1 -- 172.21.15.32:0/3329172677 shutdown_connections 2024-04-24T23:05:48.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.363+0000 7f0fe9a9a640 1 --2- 172.21.15.32:0/3329172677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fe415cb00 0x7f0fe406cb30 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:48.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.363+0000 7f0fe9a9a640 1 --2- 172.21.15.32:0/3329172677 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fe415b380 0x7f0fe406c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:48.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.363+0000 7f0fe9a9a640 1 -- 172.21.15.32:0/3329172677 >> 172.21.15.32:0/3329172677 conn(0x7f0fe406b110 msgr2=0x7f0fe406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:48.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.363+0000 7f0fe9a9a640 1 -- 172.21.15.32:0/3329172677 shutdown_connections 2024-04-24T23:05:48.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.363+0000 7f0fe9a9a640 1 -- 172.21.15.32:0/3329172677 wait complete. 2024-04-24T23:05:48.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.364+0000 7f0fe9a9a640 1 Processor -- start 2024-04-24T23:05:48.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.364+0000 7f0fe9a9a640 1 -- start start 2024-04-24T23:05:48.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.365+0000 7f0fe9a9a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fe40ce430 0x7f0fe4139260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:48.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.365+0000 7f0fe9a9a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fe415b380 0x7f0fe4155490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:48.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.365+0000 7f0fe9a9a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fe415cb00 0x7f0fe415b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:48.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.365+0000 7f0fe9a9a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe415b760 con 0x7f0fe415b380 2024-04-24T23:05:48.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.365+0000 7f0fe9a9a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe40b50a0 con 0x7f0fe415cb00 2024-04-24T23:05:48.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.365+0000 7f0fe9a9a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f0fe40b55c0 con 0x7f0fe40ce430 2024-04-24T23:05:48.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.365+0000 7f0fe2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fe415b380 0x7f0fe4155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:48.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.365+0000 7f0fe2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fe415b380 0x7f0fe4155490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40494/0 (socket says 172.21.15.32:40494) 2024-04-24T23:05:48.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.365+0000 7f0fe2ffd640 1 -- 172.21.15.32:0/2057824128 learned_addr learned my addr 172.21.15.32:0/2057824128 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:48.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.365+0000 7f0fe2ffd640 1 -- 172.21.15.32:0/2057824128 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fe40ce430 msgr2=0x7f0fe4139260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:48.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.365+0000 7f0fe2ffd640 1 --2- 172.21.15.32:0/2057824128 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fe40ce430 0x7f0fe4139260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:48.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.365+0000 7f0fe2ffd640 1 -- 172.21.15.32:0/2057824128 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fe415cb00 msgr2=0x7f0fe415b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:48.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.365+0000 7f0fe3fff640 1 --2- 172.21.15.32:0/2057824128 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fe415cb00 0x7f0fe415b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:48.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.365+0000 7f0fe2ffd640 1 --2- 172.21.15.32:0/2057824128 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fe415cb00 0x7f0fe415b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:48.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.365+0000 7f0fe2ffd640 1 -- 172.21.15.32:0/2057824128 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fe41397a0 con 0x7f0fe415b380 2024-04-24T23:05:48.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.366+0000 7f0fe2ffd640 1 --2- 172.21.15.32:0/2057824128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fe415b380 0x7f0fe4155490 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f0fd80098c0 tx=0x7f0fd80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:48.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.366+0000 7f0fcbfff640 1 -- 172.21.15.32:0/2057824128 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd8041020 con 0x7f0fe415b380 2024-04-24T23:05:48.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.366+0000 7f0fcbfff640 1 -- 172.21.15.32:0/2057824128 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fd803c670 con 0x7f0fe415b380 2024-04-24T23:05:48.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.366+0000 7f0fe9a9a640 1 -- 172.21.15.32:0/2057824128 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fe4155d90 con 0x7f0fe415b380 2024-04-24T23:05:48.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.366+0000 7f0fcbfff640 1 -- 172.21.15.32:0/2057824128 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd8037040 con 0x7f0fe415b380 2024-04-24T23:05:48.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.366+0000 7f0fe9a9a640 1 -- 172.21.15.32:0/2057824128 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fe4156350 con 0x7f0fe415b380 2024-04-24T23:05:48.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.367+0000 7f0fe9a9a640 1 -- 172.21.15.32:0/2057824128 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fe4068430 con 0x7f0fe415b380 2024-04-24T23:05:48.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.367+0000 7f0fcbfff640 1 -- 172.21.15.32:0/2057824128 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0fd8044020 con 0x7f0fe415b380 2024-04-24T23:05:48.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.368+0000 7f0fcbfff640 1 --2- 172.21.15.32:0/2057824128 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0fd006be60 0x7f0fd006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:48.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.368+0000 7f0fcbfff640 1 -- 172.21.15.32:0/2057824128 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5709+0+0 (secure 0 0 0) 0x7f0fd80ab540 con 0x7f0fe415b380 2024-04-24T23:05:48.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.368+0000 7f0fe37fe640 1 --2- 172.21.15.32:0/2057824128 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0fd006be60 0x7f0fd006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:48.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.369+0000 7f0fe37fe640 1 --2- 172.21.15.32:0/2057824128 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0fd006be60 0x7f0fd006e320 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f0fd40098f0 tx=0x7f0fd4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:48.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.373+0000 7f0fcbfff640 1 -- 172.21.15.32:0/2057824128 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0fd804b030 con 0x7f0fe415b380 2024-04-24T23:05:48.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.559+0000 7f0fe9a9a640 1 -- 172.21.15.32:0/2057824128 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f0fe4067150 con 0x7f0fe415b380 2024-04-24T23:05:48.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.560+0000 7f0fcbfff640 1 -- 172.21.15.32:0/2057824128 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v141) ==== 81+0+34192 (secure 0 0 0) 0x7f0fd8079ef0 con 0x7f0fe415b380 2024-04-24T23:05:48.562 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:05:48.562 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:05:48.562 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 141, 2024-04-24T23:05:48.562 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", 2024-04-24T23:05:48.562 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2024-04-24T22:58:46.964851+0000", 2024-04-24T23:05:48.563 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2024-04-24T23:05:42.463180+0000", 2024-04-24T23:05:48.563 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2024-04-24T22:58:52.048787+0000", 2024-04-24T23:05:48.563 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2024-04-24T22:58:49.045609+0000", 2024-04-24T23:05:48.563 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-24T23:05:48.563 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2024-04-24T23:05:48.563 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2024-04-24T23:05:48.563 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2024-04-24T23:05:48.563 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2024-04-24T23:05:48.563 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2024-04-24T23:05:48.563 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2024-04-24T23:05:48.563 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:05:48.563 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 4, 2024-04-24T23:05:48.563 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.69999998807907104, 2024-04-24T23:05:48.563 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-24T23:05:48.563 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-24T23:05:48.564 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2024-04-24T23:05:48.564 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 17, 2024-04-24T23:05:48.564 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2024-04-24T23:05:48.564 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "luminous", 2024-04-24T23:05:48.564 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "jewel", 2024-04-24T23:05:48.564 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "squid", 2024-04-24T23:05:48.564 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_crimson": false, 2024-04-24T23:05:48.564 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2024-04-24T23:05:48.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.564 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2024-04-24T23:05:48.564 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": ".mgr", 2024-04-24T23:05:48.564 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T22:58:53.171408+0000", 2024-04-24T23:05:48.564 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:05:48.564 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:05:48.565 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:05:48.565 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:05:48.565 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:05:48.565 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:05:48.565 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:05:48.565 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:05:48.565 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:05:48.565 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:05:48.565 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:05:48.565 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:05:48.565 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2024-04-24T23:05:48.565 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2024-04-24T23:05:48.565 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2024-04-24T23:05:48.565 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2024-04-24T23:05:48.565 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2024-04-24T23:05:48.566 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:05:48.566 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:05:48.566 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:05:48.566 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:05:48.566 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:05:48.566 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:05:48.566 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:05:48.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.566 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "113", 2024-04-24T23:05:48.566 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:05:48.566 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:05:48.566 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:05:48.566 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:05:48.566 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:05:48.567 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:05:48.567 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:05:48.567 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:05:48.567 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:05:48.567 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:05:48.567 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:05:48.567 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:05:48.567 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:05:48.567 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:05:48.567 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:05:48.567 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:05:48.567 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:05:48.567 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:05:48.567 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:05:48.567 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:05:48.567 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:05:48.568 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:05:48.568 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:05:48.568 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:05:48.568 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:05:48.568 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:05:48.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.568 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:05:48.568 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:05:48.568 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:05:48.568 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:05:48.568 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:05:48.568 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:05:48.568 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:05:48.568 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:05:48.568 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:05:48.569 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:05:48.569 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:05:48.569 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:05:48.569 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_max": 32, 2024-04-24T23:05:48.569 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2024-04-24T23:05:48.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.569 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:05:48.569 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": {} 2024-04-24T23:05:48.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.569 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:05:48.569 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:05:48.569 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 8, 2024-04-24T23:05:48.569 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 8, 2024-04-24T23:05:48.569 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.25, 2024-04-24T23:05:48.569 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-04-24T23:05:48.570 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-04-24T23:05:48.570 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:05:48.570 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:05:48.570 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:05:48.570 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:48.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.570 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 16, 2024-04-24T23:05:48.570 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2024-04-24T23:05:48.570 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:05:32.064234+0000", 2024-04-24T23:05:48.570 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:05:48.570 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:05:48.570 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:05:48.570 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:05:48.571 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:05:48.571 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:05:48.571 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:05:48.571 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:05:48.571 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:05:48.571 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:05:48.571 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:05:48.571 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:05:48.571 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 32, 2024-04-24T23:05:48.571 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 32, 2024-04-24T23:05:48.571 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 32, 2024-04-24T23:05:48.571 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 32, 2024-04-24T23:05:48.571 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 32, 2024-04-24T23:05:48.571 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:05:48.571 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:05:48.572 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:05:48.572 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:05:48.572 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:05:48.572 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:05:48.572 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:05:48.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.572 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "139", 2024-04-24T23:05:48.572 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:05:48.572 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:05:48.572 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:05:48.572 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:05:48.572 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:05:48.572 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:05:48.572 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:05:48.572 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:05:48.573 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:05:48.573 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:05:48.573 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:05:48.573 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:05:48.573 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:05:48.573 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:05:48.573 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:05:48.573 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:05:48.573 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:05:48.573 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:05:48.573 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:05:48.573 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:05:48.573 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:05:48.573 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:05:48.573 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:05:48.574 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:05:48.574 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:05:48.574 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:05:48.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.574 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:05:48.574 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:05:48.574 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:05:48.574 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:05:48.574 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:05:48.574 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:05:48.574 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:05:48.574 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:05:48.574 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:05:48.574 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:05:48.574 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:05:48.575 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:05:48.575 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2024-04-24T23:05:48.575 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2024-04-24T23:05:48.575 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2024-04-24T23:05:48.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.575 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:05:48.575 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:05:48.575 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2024-04-24T23:05:48.575 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:48.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.575 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:05:48.575 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:05:48.575 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 2.25, 2024-04-24T23:05:48.575 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 2.25, 2024-04-24T23:05:48.576 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:05:48.576 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2.25, 2024-04-24T23:05:48.576 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2.25, 2024-04-24T23:05:48.576 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:05:48.576 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:05:48.576 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:05:48.576 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:48.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.576 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 17, 2024-04-24T23:05:48.576 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2024-04-24T23:05:48.576 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:05:32.760777+0000", 2024-04-24T23:05:48.576 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:05:48.576 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:05:48.576 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:05:48.577 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:05:48.577 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:05:48.577 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:05:48.577 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:05:48.577 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:05:48.577 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:05:48.577 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:05:48.577 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:05:48.577 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:05:48.577 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 64, 2024-04-24T23:05:48.577 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 64, 2024-04-24T23:05:48.577 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 64, 2024-04-24T23:05:48.577 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 64, 2024-04-24T23:05:48.577 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 64, 2024-04-24T23:05:48.578 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:05:48.578 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:05:48.578 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:05:48.578 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:05:48.578 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:05:48.578 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:05:48.578 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:05:48.578 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.578 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "141", 2024-04-24T23:05:48.578 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:05:48.578 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:05:48.578 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:05:48.578 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:05:48.578 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:05:48.578 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:05:48.579 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:05:48.579 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:05:48.579 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:05:48.579 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 33554432, 2024-04-24T23:05:48.579 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:05:48.579 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:05:48.579 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:05:48.579 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:05:48.579 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:05:48.579 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:05:48.579 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:05:48.579 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:05:48.579 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:05:48.579 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:05:48.580 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:05:48.580 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:05:48.580 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:05:48.580 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:05:48.580 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:05:48.580 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:05:48.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.580 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:05:48.580 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:05:48.580 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:05:48.580 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:05:48.580 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:05:48.580 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:05:48.580 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:05:48.580 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:05:48.581 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:05:48.581 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:05:48.581 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:05:48.581 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:05:48.581 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 64, 2024-04-24T23:05:48.581 INFO:teuthology.orchestra.run.smithi032.stdout: "target_size_ratio": 0.89999997615814209 2024-04-24T23:05:48.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.581 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:05:48.581 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:05:48.581 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2024-04-24T23:05:48.581 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:48.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.581 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:05:48.581 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:05:48.581 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.3799999952316284, 2024-04-24T23:05:48.582 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.3799999952316284, 2024-04-24T23:05:48.582 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:05:48.582 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.3799999952316284, 2024-04-24T23:05:48.582 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.3799999952316284, 2024-04-24T23:05:48.582 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:05:48.582 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:05:48.582 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:05:48.582 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:48.582 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:48.582 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:05:48.582 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-04-24T23:05:48.582 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.582 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:05:48.582 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:05:48.583 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:05:48.583 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:05:48.583 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:05:48.583 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:05:48.583 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:05:48.583 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:05:48.583 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:05:48.583 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 138, 2024-04-24T23:05:48.583 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:05:48.583 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:05:48.583 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:05:48.583 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:48.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.583 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:48.583 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2024-04-24T23:05:48.584 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:05:48.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.584 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:48.584 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2024-04-24T23:05:48.584 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:05:48.584 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:48.584 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:48.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.584 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:05:48.584 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:48.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.584 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:48.584 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2024-04-24T23:05:48.584 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:05:48.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.585 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:48.585 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2024-04-24T23:05:48.585 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:05:48.585 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:48.585 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:48.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.585 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:05:48.585 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:48.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.585 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:48.585 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2024-04-24T23:05:48.585 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:05:48.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.586 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:48.586 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2024-04-24T23:05:48.586 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:05:48.586 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:48.586 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:48.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.586 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:05:48.586 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:48.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.586 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:48.586 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2024-04-24T23:05:48.586 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:05:48.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.587 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:48.587 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2024-04-24T23:05:48.587 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:05:48.587 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:48.587 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:48.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.587 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/1479395426", 2024-04-24T23:05:48.587 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/1479395426", 2024-04-24T23:05:48.587 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/1479395426", 2024-04-24T23:05:48.587 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/1479395426", 2024-04-24T23:05:48.587 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:05:48.587 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:05:48.587 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:05:48.587 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:48.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.588 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:05:48.588 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:05:48.588 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:05:48.588 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:05:48.588 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:05:48.588 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:05:48.588 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:05:48.588 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:05:48.588 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:05:48.588 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 138, 2024-04-24T23:05:48.588 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:05:48.588 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:05:48.588 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:05:48.588 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:48.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.589 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:48.589 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2024-04-24T23:05:48.589 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:05:48.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.589 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:48.589 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2024-04-24T23:05:48.589 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:05:48.589 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:48.589 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:48.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.589 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:05:48.589 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:48.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.590 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:48.590 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2024-04-24T23:05:48.590 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:05:48.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.590 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:48.590 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2024-04-24T23:05:48.590 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:05:48.590 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:48.590 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:48.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.590 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:05:48.590 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:48.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.591 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:48.591 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2024-04-24T23:05:48.591 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:05:48.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.591 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:48.591 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2024-04-24T23:05:48.591 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:05:48.591 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:48.591 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:48.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.591 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:05:48.591 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:48.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.592 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:48.592 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2024-04-24T23:05:48.592 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:05:48.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.592 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.592 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:48.592 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2024-04-24T23:05:48.592 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:05:48.592 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:48.592 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:48.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.592 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/1340633718", 2024-04-24T23:05:48.592 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/1340633718", 2024-04-24T23:05:48.592 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/1340633718", 2024-04-24T23:05:48.592 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/1340633718", 2024-04-24T23:05:48.593 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:05:48.593 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:05:48.593 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:05:48.593 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:48.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.593 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:05:48.593 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:05:48.593 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:05:48.593 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:05:48.593 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:05:48.593 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:05:48.593 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:05:48.593 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:05:48.594 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:05:48.594 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 138, 2024-04-24T23:05:48.594 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:05:48.594 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:05:48.594 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:05:48.594 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:48.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.594 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:48.594 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2024-04-24T23:05:48.594 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:05:48.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.594 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:48.594 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2024-04-24T23:05:48.595 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:05:48.595 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:48.595 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:48.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.595 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:05:48.595 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:48.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.595 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:48.595 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2024-04-24T23:05:48.595 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:05:48.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.595 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:48.595 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2024-04-24T23:05:48.595 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:05:48.595 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:48.596 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:48.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.596 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:05:48.596 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:48.596 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.596 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:48.596 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2024-04-24T23:05:48.596 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:05:48.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.596 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.596 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:48.596 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2024-04-24T23:05:48.596 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:05:48.596 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:48.596 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:48.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.597 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:05:48.597 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:48.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.597 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:48.597 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2024-04-24T23:05:48.597 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:05:48.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.597 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:48.597 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2024-04-24T23:05:48.597 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:05:48.597 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:48.597 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:48.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.597 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/3405899334", 2024-04-24T23:05:48.598 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6820/3405899334", 2024-04-24T23:05:48.598 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6828/3405899334", 2024-04-24T23:05:48.598 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6824/3405899334", 2024-04-24T23:05:48.598 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:05:48.598 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:05:48.598 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:05:48.598 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:48.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.598 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.598 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:05:48.598 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:05:48.598 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:05:48.598 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:05:48.598 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:05:48.598 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:05:48.599 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:05:48.599 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:05:48.599 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:05:48.599 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 138, 2024-04-24T23:05:48.599 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:05:48.599 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:05:48.599 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:05:48.599 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:48.599 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.599 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:48.599 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2024-04-24T23:05:48.599 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:05:48.599 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.599 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.599 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:48.600 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2024-04-24T23:05:48.600 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:05:48.600 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:48.600 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:48.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.600 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:05:48.600 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:48.600 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.600 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:48.600 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2024-04-24T23:05:48.600 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:05:48.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.600 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.600 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:48.600 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2024-04-24T23:05:48.601 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:05:48.601 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:48.601 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:48.601 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.601 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:05:48.601 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:48.601 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.601 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:48.601 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2024-04-24T23:05:48.601 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:05:48.601 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.601 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.601 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:48.601 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2024-04-24T23:05:48.601 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:05:48.602 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:48.602 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:48.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.602 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:05:48.602 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:48.602 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.602 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:48.602 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2024-04-24T23:05:48.602 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:05:48.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.602 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.602 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:48.602 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2024-04-24T23:05:48.602 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:05:48.603 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:48.603 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:48.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.603 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6821/3381057190", 2024-04-24T23:05:48.603 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6825/3381057190", 2024-04-24T23:05:48.603 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/3381057190", 2024-04-24T23:05:48.603 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/3381057190", 2024-04-24T23:05:48.603 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:05:48.603 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:05:48.603 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:05:48.603 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:48.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.603 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.603 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:05:48.603 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:05:48.604 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:05:48.604 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:05:48.604 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:05:48.604 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:05:48.604 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:05:48.604 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:05:48.604 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:05:48.604 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 138, 2024-04-24T23:05:48.604 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:05:48.604 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:05:48.604 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:05:48.604 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:48.604 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.604 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:48.605 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6800", 2024-04-24T23:05:48.605 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:05:48.605 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.605 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:48.605 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6801", 2024-04-24T23:05:48.605 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:05:48.605 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:48.605 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:48.605 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.605 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:05:48.605 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:48.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.605 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:48.605 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6802", 2024-04-24T23:05:48.606 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:05:48.606 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.606 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.606 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:48.606 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6803", 2024-04-24T23:05:48.606 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:05:48.606 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:48.606 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:48.606 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.606 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:05:48.606 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:48.606 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.606 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:48.606 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6806", 2024-04-24T23:05:48.607 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:05:48.607 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.607 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.607 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:48.607 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6807", 2024-04-24T23:05:48.607 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:05:48.607 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:48.607 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:48.607 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.607 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:05:48.607 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:48.607 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.607 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:48.607 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6804", 2024-04-24T23:05:48.607 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:05:48.607 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.608 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.608 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:48.608 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6805", 2024-04-24T23:05:48.608 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:05:48.608 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:48.608 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:48.608 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.608 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6801/4010553979", 2024-04-24T23:05:48.608 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6803/4010553979", 2024-04-24T23:05:48.608 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6807/4010553979", 2024-04-24T23:05:48.608 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6805/4010553979", 2024-04-24T23:05:48.608 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:05:48.608 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:05:48.608 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:05:48.608 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:48.609 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.609 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.609 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:05:48.609 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:05:48.609 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:05:48.609 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:05:48.609 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:05:48.609 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:05:48.609 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:05:48.609 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:05:48.609 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:05:48.609 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 138, 2024-04-24T23:05:48.609 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:05:48.609 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:05:48.610 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:05:48.610 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:48.610 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.610 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:48.610 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6808", 2024-04-24T23:05:48.610 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:05:48.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.610 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.610 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:48.610 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6809", 2024-04-24T23:05:48.610 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:05:48.610 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:48.610 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:48.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.610 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:05:48.611 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:48.611 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.611 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:48.611 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6810", 2024-04-24T23:05:48.611 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:05:48.611 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.611 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.611 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:48.611 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6811", 2024-04-24T23:05:48.611 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:05:48.611 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:48.611 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:48.611 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.611 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:05:48.611 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:48.612 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.612 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:48.612 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6814", 2024-04-24T23:05:48.612 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:05:48.612 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.612 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.612 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:48.612 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6815", 2024-04-24T23:05:48.612 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:05:48.612 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:48.612 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:48.612 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.612 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:05:48.612 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:48.613 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.613 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:48.613 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6812", 2024-04-24T23:05:48.613 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:05:48.613 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.613 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.613 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:48.613 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6813", 2024-04-24T23:05:48.613 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:05:48.613 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:48.613 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:48.613 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.613 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6809/3794084091", 2024-04-24T23:05:48.613 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6811/3794084091", 2024-04-24T23:05:48.613 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6815/3794084091", 2024-04-24T23:05:48.614 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6813/3794084091", 2024-04-24T23:05:48.614 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:05:48.614 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:05:48.614 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:05:48.614 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:48.614 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.614 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.614 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:05:48.614 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:05:48.614 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:05:48.614 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:05:48.614 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:05:48.614 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:05:48.614 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:05:48.614 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:05:48.615 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:05:48.615 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 138, 2024-04-24T23:05:48.615 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:05:48.615 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:05:48.615 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:05:48.615 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:48.615 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.615 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:48.615 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6823", 2024-04-24T23:05:48.615 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:05:48.615 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.615 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.615 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:48.615 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6825", 2024-04-24T23:05:48.615 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:05:48.616 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:48.616 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:48.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.616 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:05:48.616 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:48.616 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.616 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:48.616 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6826", 2024-04-24T23:05:48.616 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:05:48.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.616 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.616 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:48.616 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6827", 2024-04-24T23:05:48.616 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:05:48.616 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:48.617 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:48.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.617 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:05:48.617 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:48.617 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.617 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:48.617 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6830", 2024-04-24T23:05:48.617 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:05:48.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.617 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.617 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:48.617 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6831", 2024-04-24T23:05:48.617 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:05:48.617 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:48.617 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:48.618 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.618 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:05:48.618 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:48.618 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.618 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:48.618 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6828", 2024-04-24T23:05:48.618 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:05:48.618 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.618 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.618 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:48.618 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6829", 2024-04-24T23:05:48.618 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:05:48.618 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:48.618 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:48.618 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.619 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6825/1348640258", 2024-04-24T23:05:48.619 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6827/1348640258", 2024-04-24T23:05:48.619 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6831/1348640258", 2024-04-24T23:05:48.619 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6829/1348640258", 2024-04-24T23:05:48.619 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:05:48.619 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:05:48.619 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:05:48.619 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:48.619 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.619 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.619 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:05:48.619 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:05:48.619 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:05:48.619 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:05:48.619 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:05:48.620 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:05:48.620 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:05:48.620 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:05:48.620 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:05:48.620 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 138, 2024-04-24T23:05:48.620 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:05:48.620 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:05:48.620 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:05:48.620 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:48.620 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.620 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:48.620 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6816", 2024-04-24T23:05:48.620 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:05:48.620 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.620 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.621 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:48.621 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6817", 2024-04-24T23:05:48.621 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:05:48.621 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:48.621 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:48.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.621 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:05:48.621 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:48.621 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.621 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:48.621 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6818", 2024-04-24T23:05:48.621 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:05:48.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.621 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.621 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:48.622 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6819", 2024-04-24T23:05:48.622 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:05:48.622 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:48.622 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:48.622 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.622 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:05:48.622 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:48.622 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.622 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:48.622 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6822", 2024-04-24T23:05:48.622 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:05:48.622 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.622 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.622 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:48.622 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6824", 2024-04-24T23:05:48.623 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:05:48.623 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:48.623 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:48.623 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.623 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:05:48.623 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:48.623 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.623 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:48.623 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6820", 2024-04-24T23:05:48.623 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:05:48.623 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.623 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.623 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:48.623 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6821", 2024-04-24T23:05:48.623 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:05:48.624 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:48.624 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:48.624 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.624 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6817/3250005122", 2024-04-24T23:05:48.624 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6819/3250005122", 2024-04-24T23:05:48.624 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6824/3250005122", 2024-04-24T23:05:48.624 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6821/3250005122", 2024-04-24T23:05:48.624 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:05:48.624 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:05:48.624 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:05:48.624 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:48.624 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:48.624 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:05:48.624 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2024-04-24T23:05:48.624 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.624 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:05:48.625 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:05:48.625 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:05:48.625 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:05:48.625 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:05:48.625 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:05:48.625 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.156184+0000", 2024-04-24T23:05:48.625 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:05:48.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.625 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.625 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:05:48.625 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:05:48.625 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:05:48.625 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:05:48.625 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:05:48.626 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:05:48.626 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240774+0000", 2024-04-24T23:05:48.626 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:05:48.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.626 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.626 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:05:48.626 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:05:48.626 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:05:48.626 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:05:48.626 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:05:48.626 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:05:48.626 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240949+0000", 2024-04-24T23:05:48.626 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:05:48.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.626 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.627 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:05:48.627 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:05:48.627 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:05:48.627 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:05:48.627 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:05:48.627 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:05:48.627 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.171672+0000", 2024-04-24T23:05:48.627 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:05:48.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.627 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:05:48.627 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:05:48.627 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:05:48.627 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:05:48.627 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:05:48.628 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:05:48.628 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.220379+0000", 2024-04-24T23:05:48.628 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:05:48.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.628 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:05:48.628 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:05:48.628 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:05:48.628 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:05:48.628 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:05:48.628 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:05:48.628 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.204963+0000", 2024-04-24T23:05:48.628 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:05:48.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.629 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:05:48.629 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:05:48.629 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:05:48.629 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:05:48.629 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:05:48.629 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:05:48.629 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.239748+0000", 2024-04-24T23:05:48.629 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:05:48.629 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.629 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:48.629 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:05:48.629 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:05:48.629 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:05:48.629 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:05:48.630 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:05:48.630 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:05:48.630 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.154104+0000", 2024-04-24T23:05:48.630 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:05:48.630 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:48.630 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:05:48.630 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2024-04-24T23:05:48.630 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2024-04-24T23:05:48.630 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_primaries": [], 2024-04-24T23:05:48.630 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2024-04-24T23:05:48.630 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2024-04-24T23:05:48.630 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklist": {}, 2024-04-24T23:05:48.630 INFO:teuthology.orchestra.run.smithi032.stdout: "range_blocklist": {}, 2024-04-24T23:05:48.630 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2024-04-24T23:05:48.631 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2024-04-24T23:05:48.631 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2024-04-24T23:05:48.631 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2024-04-24T23:05:48.631 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2024-04-24T23:05:48.631 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2024-04-24T23:05:48.631 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2024-04-24T23:05:48.631 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:48.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:48.631 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2024-04-24T23:05:48.631 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2024-04-24T23:05:48.631 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2024-04-24T23:05:48.631 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2024-04-24T23:05:48.631 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {}, 2024-04-24T23:05:48.631 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode": { 2024-04-24T23:05:48.631 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_enabled": false, 2024-04-24T23:05:48.632 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_bucket_count": 0, 2024-04-24T23:05:48.632 INFO:teuthology.orchestra.run.smithi032.stdout: "degraded_stretch_mode": 0, 2024-04-24T23:05:48.632 INFO:teuthology.orchestra.run.smithi032.stdout: "recovering_stretch_mode": 0, 2024-04-24T23:05:48.632 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_bucket": 0 2024-04-24T23:05:48.632 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:48.632 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:05:48.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.562+0000 7f0fe9a9a640 1 -- 172.21.15.32:0/2057824128 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0fd006be60 msgr2=0x7f0fd006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:48.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.562+0000 7f0fe9a9a640 1 --2- 172.21.15.32:0/2057824128 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0fd006be60 0x7f0fd006e320 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f0fd40098f0 tx=0x7f0fd4008040 comp rx=0 tx=0).stop 2024-04-24T23:05:48.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.562+0000 7f0fe9a9a640 1 -- 172.21.15.32:0/2057824128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fe415b380 msgr2=0x7f0fe4155490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:48.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.562+0000 7f0fe9a9a640 1 --2- 172.21.15.32:0/2057824128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fe415b380 0x7f0fe4155490 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f0fd80098c0 tx=0x7f0fd80046c0 comp rx=0 tx=0).stop 2024-04-24T23:05:48.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.562+0000 7f0fe9a9a640 1 -- 172.21.15.32:0/2057824128 shutdown_connections 2024-04-24T23:05:48.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.562+0000 7f0fe9a9a640 1 --2- 172.21.15.32:0/2057824128 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0fd006be60 0x7f0fd006e320 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:48.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.562+0000 7f0fe9a9a640 1 --2- 172.21.15.32:0/2057824128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fe415b380 0x7f0fe4155490 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:48.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.562+0000 7f0fe9a9a640 1 --2- 172.21.15.32:0/2057824128 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fe415cb00 0x7f0fe415b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:48.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.562+0000 7f0fe9a9a640 1 --2- 172.21.15.32:0/2057824128 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fe40ce430 0x7f0fe4139260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:48.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.562+0000 7f0fe9a9a640 1 -- 172.21.15.32:0/2057824128 >> 172.21.15.32:0/2057824128 conn(0x7f0fe406b110 msgr2=0x7f0fe40b59e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:48.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.563+0000 7f0fe9a9a640 1 -- 172.21.15.32:0/2057824128 shutdown_connections 2024-04-24T23:05:48.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:48.563+0000 7f0fe9a9a640 1 -- 172.21.15.32:0/2057824128 wait complete. 2024-04-24T23:05:48.635 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-04-24T23:05:53.636 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-24T23:05:53.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.749+0000 7fdd7955a640 1 Processor -- start 2024-04-24T23:05:53.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.749+0000 7fdd7955a640 1 -- start start 2024-04-24T23:05:53.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.750+0000 7fdd7955a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdd7415b380 0x7fdd7406c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:53.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.750+0000 7fdd7955a640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd740cef50 con 0x7fdd7415cb00 2024-04-24T23:05:53.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.750+0000 7fdd7955a640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd740b4520 con 0x7fdd740ceb40 2024-04-24T23:05:53.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.750+0000 7fdd7955a640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fdd740b4800 con 0x7fdd7415b380 2024-04-24T23:05:53.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.750+0000 7fdd72ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdd7415b380 0x7fdd7406c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:53.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.750+0000 7fdd6bfff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fdd7415cb00 0x7fdd740ce430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59984/0 (socket says 172.21.15.32:59984) 2024-04-24T23:05:53.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.750+0000 7fdd6bfff640 1 -- 172.21.15.32:0/924059179 learned_addr learned my addr 172.21.15.32:0/924059179 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:05:53.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.750+0000 7fdd727fc640 1 -- 172.21.15.32:0/924059179 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2044038694 0 0) 0x7fdd5c001110 con 0x7fdd7415cb00 2024-04-24T23:05:53.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.750+0000 7fdd727fc640 1 -- 172.21.15.32:0/924059179 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd60003520 con 0x7fdd7415cb00 2024-04-24T23:05:53.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.750+0000 7fdd72ffd640 1 -- 172.21.15.32:0/924059179 >> v1:172.21.15.121:6789/0 conn(0x7fdd740ceb40 legacy=0x7fdd740b55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:53.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.750+0000 7fdd72ffd640 1 -- 172.21.15.32:0/924059179 >> v1:172.21.15.32:6789/0 conn(0x7fdd7415cb00 legacy=0x7fdd740ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:53.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.750+0000 7fdd72ffd640 1 -- 172.21.15.32:0/924059179 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd740b4ae0 con 0x7fdd7415b380 2024-04-24T23:05:53.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.751+0000 7fdd72ffd640 1 --2- 172.21.15.32:0/924059179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd7415b380 0x7fdd7406c5f0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fdd640097a0 tx=0x7fdd6402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac0b01a45d0864b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:53.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.751+0000 7fdd727fc640 1 -- 172.21.15.32:0/924059179 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd64037070 con 0x7fdd7415b380 2024-04-24T23:05:53.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.751+0000 7fdd727fc640 1 -- 172.21.15.32:0/924059179 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd64032440 con 0x7fdd7415b380 2024-04-24T23:05:53.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.751+0000 7fdd727fc640 1 -- 172.21.15.32:0/924059179 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd6402ae60 con 0x7fdd7415b380 2024-04-24T23:05:53.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.752+0000 7fdd7955a640 1 -- 172.21.15.32:0/924059179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd7415b380 msgr2=0x7fdd7406c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:53.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.752+0000 7fdd7955a640 1 --2- 172.21.15.32:0/924059179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd7415b380 0x7fdd7406c5f0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fdd640097a0 tx=0x7fdd6402a110 comp rx=0 tx=0).stop 2024-04-24T23:05:53.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.752+0000 7fdd7955a640 1 -- 172.21.15.32:0/924059179 shutdown_connections 2024-04-24T23:05:53.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.752+0000 7fdd7955a640 1 --2- 172.21.15.32:0/924059179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd7415b380 0x7fdd7406c5f0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:53.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.752+0000 7fdd7955a640 1 -- 172.21.15.32:0/924059179 >> 172.21.15.32:0/924059179 conn(0x7fdd7406b110 msgr2=0x7fdd7406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:53.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.752+0000 7fdd7955a640 1 -- 172.21.15.32:0/924059179 shutdown_connections 2024-04-24T23:05:53.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.752+0000 7fdd7955a640 1 -- 172.21.15.32:0/924059179 wait complete. 2024-04-24T23:05:53.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.753+0000 7fdd7955a640 1 Processor -- start 2024-04-24T23:05:53.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.753+0000 7fdd7955a640 1 -- start start 2024-04-24T23:05:53.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.754+0000 7fdd7955a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd740ceb40 0x7fdd74142740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:53.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.754+0000 7fdd7955a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdd7415b380 0x7fdd7413f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:53.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.754+0000 7fdd7955a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd7415cb00 0x7fdd74141300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:53.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.754+0000 7fdd7955a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdd7415b760 con 0x7fdd7415cb00 2024-04-24T23:05:53.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.754+0000 7fdd7955a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdd740b50a0 con 0x7fdd740ceb40 2024-04-24T23:05:53.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.754+0000 7fdd7955a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fdd7406cb30 con 0x7fdd7415b380 2024-04-24T23:05:53.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.754+0000 7fdd737fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd7415cb00 0x7fdd74141300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:53.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.754+0000 7fdd737fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd7415cb00 0x7fdd74141300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38338/0 (socket says 172.21.15.32:38338) 2024-04-24T23:05:53.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.754+0000 7fdd737fe640 1 -- 172.21.15.32:0/256666521 learned_addr learned my addr 172.21.15.32:0/256666521 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:53.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.754+0000 7fdd72ffd640 1 --2- 172.21.15.32:0/256666521 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd740ceb40 0x7fdd74142740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:53.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.754+0000 7fdd6bfff640 1 --2- 172.21.15.32:0/256666521 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdd7415b380 0x7fdd7413f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:53.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.754+0000 7fdd737fe640 1 -- 172.21.15.32:0/256666521 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdd7415b380 msgr2=0x7fdd7413f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:53.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.754+0000 7fdd737fe640 1 --2- 172.21.15.32:0/256666521 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdd7415b380 0x7fdd7413f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:53.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.754+0000 7fdd737fe640 1 -- 172.21.15.32:0/256666521 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd740ceb40 msgr2=0x7fdd74142740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:53.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.754+0000 7fdd737fe640 1 --2- 172.21.15.32:0/256666521 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd740ceb40 0x7fdd74142740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:53.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.754+0000 7fdd737fe640 1 -- 172.21.15.32:0/256666521 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd74142c80 con 0x7fdd7415cb00 2024-04-24T23:05:53.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.754+0000 7fdd72ffd640 1 --2- 172.21.15.32:0/256666521 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd740ceb40 0x7fdd74142740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:05:53.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.754+0000 7fdd737fe640 1 --2- 172.21.15.32:0/256666521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd7415cb00 0x7fdd74141300 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c00c7d0 tx=0x7fdd6c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:53.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.754+0000 7fdd6b7fe640 1 -- 172.21.15.32:0/256666521 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c003b70 con 0x7fdd7415cb00 2024-04-24T23:05:53.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.755+0000 7fdd6b7fe640 1 -- 172.21.15.32:0/256666521 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd6c003d30 con 0x7fdd7415cb00 2024-04-24T23:05:53.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.755+0000 7fdd7955a640 1 -- 172.21.15.32:0/256666521 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd74140260 con 0x7fdd7415cb00 2024-04-24T23:05:53.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.755+0000 7fdd6b7fe640 1 -- 172.21.15.32:0/256666521 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c00aa90 con 0x7fdd7415cb00 2024-04-24T23:05:53.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.755+0000 7fdd7955a640 1 -- 172.21.15.32:0/256666521 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd74140820 con 0x7fdd7415cb00 2024-04-24T23:05:53.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.756+0000 7fdd7955a640 1 -- 172.21.15.32:0/256666521 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd74068430 con 0x7fdd7415cb00 2024-04-24T23:05:53.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.756+0000 7fdd6b7fe640 1 -- 172.21.15.32:0/256666521 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdd6c00ac30 con 0x7fdd7415cb00 2024-04-24T23:05:53.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.756+0000 7fdd6b7fe640 1 --2- 172.21.15.32:0/256666521 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdd5406be30 0x7fdd5406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:53.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.757+0000 7fdd6b7fe640 1 -- 172.21.15.32:0/256666521 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5796+0+0 (secure 0 0 0) 0x7fdd6c08ade0 con 0x7fdd7415cb00 2024-04-24T23:05:53.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.757+0000 7fdd6b7fe640 1 -- 172.21.15.32:0/256666521 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=144}) -- 0x7fdd54073a80 con 0x7fdd7415cb00 2024-04-24T23:05:53.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.757+0000 7fdd72ffd640 1 --2- 172.21.15.32:0/256666521 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdd5406be30 0x7fdd5406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:53.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.757+0000 7fdd72ffd640 1 --2- 172.21.15.32:0/256666521 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdd5406be30 0x7fdd5406e2f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fdd64034000 tx=0x7fdd6402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:53.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.761+0000 7fdd6b7fe640 1 -- 172.21.15.32:0/256666521 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdd74142c80 con 0x7fdd7415cb00 2024-04-24T23:05:53.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.814+0000 7fdd6b7fe640 1 -- 172.21.15.32:0/256666521 <== mon.0 v2:172.21.15.32:3300/0 7 ==== osd_map(144..144 src has 1..144) ==== 312+0+0 (secure 0 0 0) 0x7fdd6c0570c0 con 0x7fdd7415cb00 2024-04-24T23:05:53.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.814+0000 7fdd6b7fe640 1 -- 172.21.15.32:0/256666521 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=145}) -- 0x7fdd54074370 con 0x7fdd7415cb00 2024-04-24T23:05:53.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.952+0000 7fdd7955a640 1 -- 172.21.15.32:0/256666521 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7fdd740afd60 con 0x7fdd7415cb00 2024-04-24T23:05:53.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.953+0000 7fdd6b7fe640 1 -- 172.21.15.32:0/256666521 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v144) ==== 81+0+34760 (secure 0 0 0) 0x7fdd6c059720 con 0x7fdd7415cb00 2024-04-24T23:05:53.955 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:05:53.955 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:05:53.955 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 144, 2024-04-24T23:05:53.955 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", 2024-04-24T23:05:53.955 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2024-04-24T22:58:46.964851+0000", 2024-04-24T23:05:53.955 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2024-04-24T23:05:53.811291+0000", 2024-04-24T23:05:53.956 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2024-04-24T22:58:52.048787+0000", 2024-04-24T23:05:53.956 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2024-04-24T22:58:49.045609+0000", 2024-04-24T23:05:53.956 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-24T23:05:53.956 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2024-04-24T23:05:53.956 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2024-04-24T23:05:53.956 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2024-04-24T23:05:53.956 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2024-04-24T23:05:53.956 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2024-04-24T23:05:53.956 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2024-04-24T23:05:53.956 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:05:53.956 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 4, 2024-04-24T23:05:53.956 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.69999998807907104, 2024-04-24T23:05:53.956 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-24T23:05:53.956 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-24T23:05:53.957 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2024-04-24T23:05:53.957 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 17, 2024-04-24T23:05:53.957 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2024-04-24T23:05:53.957 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "luminous", 2024-04-24T23:05:53.957 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "luminous", 2024-04-24T23:05:53.957 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "squid", 2024-04-24T23:05:53.957 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_crimson": false, 2024-04-24T23:05:53.957 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2024-04-24T23:05:53.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:53.957 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2024-04-24T23:05:53.957 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": ".mgr", 2024-04-24T23:05:53.957 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T22:58:53.171408+0000", 2024-04-24T23:05:53.957 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:05:53.957 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:05:53.958 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:05:53.958 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:05:53.958 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:05:53.958 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:05:53.958 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:05:53.958 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:05:53.958 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:05:53.958 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:05:53.958 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:05:53.958 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:05:53.958 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2024-04-24T23:05:53.958 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2024-04-24T23:05:53.958 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2024-04-24T23:05:53.959 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2024-04-24T23:05:53.959 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2024-04-24T23:05:53.959 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:05:53.959 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:05:53.959 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:05:53.959 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:05:53.959 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:05:53.959 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:05:53.959 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:05:53.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:53.959 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "113", 2024-04-24T23:05:53.959 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:05:53.959 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:05:53.959 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:05:53.959 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:05:53.960 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:05:53.960 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:05:53.960 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:05:53.960 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:05:53.960 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:05:53.960 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:05:53.960 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:05:53.960 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:05:53.960 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:05:53.960 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:05:53.960 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:05:53.960 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:05:53.960 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:05:53.960 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:05:53.961 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:05:53.961 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:05:53.961 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:05:53.961 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:05:53.961 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:05:53.961 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:05:53.961 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:05:53.961 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:05:53.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:53.961 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:05:53.961 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:05:53.961 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:05:53.961 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:05:53.961 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:05:53.961 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:05:53.962 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:05:53.962 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:05:53.962 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:05:53.962 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:05:53.962 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:05:53.962 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:05:53.962 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_max": 32, 2024-04-24T23:05:53.962 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2024-04-24T23:05:53.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:53.962 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:05:53.962 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": {} 2024-04-24T23:05:53.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:53.962 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:05:53.962 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:05:53.963 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 8, 2024-04-24T23:05:53.963 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 8, 2024-04-24T23:05:53.963 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.25, 2024-04-24T23:05:53.963 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-04-24T23:05:53.963 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-04-24T23:05:53.963 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:05:53.963 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:05:53.963 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:05:53.963 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:53.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:53.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:53.963 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 16, 2024-04-24T23:05:53.963 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2024-04-24T23:05:53.963 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:05:32.064234+0000", 2024-04-24T23:05:53.964 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:05:53.964 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:05:53.964 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:05:53.964 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:05:53.964 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:05:53.964 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:05:53.964 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:05:53.964 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:05:53.964 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:05:53.964 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:05:53.964 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:05:53.964 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:05:53.964 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 32, 2024-04-24T23:05:53.964 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 32, 2024-04-24T23:05:53.965 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 32, 2024-04-24T23:05:53.965 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 32, 2024-04-24T23:05:53.965 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 32, 2024-04-24T23:05:53.965 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:05:53.965 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:05:53.965 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:05:53.965 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:05:53.965 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:05:53.965 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:05:53.965 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:05:53.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:53.965 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "139", 2024-04-24T23:05:53.965 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:05:53.965 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:05:53.965 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:05:53.966 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:05:53.966 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:05:53.966 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:05:53.966 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:05:53.966 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:05:53.966 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:05:53.966 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:05:53.966 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:05:53.966 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:05:53.966 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:05:53.966 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:05:53.966 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:05:53.966 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:05:53.966 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:05:53.967 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:05:53.967 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:05:53.967 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:05:53.967 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:05:53.967 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:05:53.967 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:05:53.967 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:05:53.967 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:05:53.967 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:05:53.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:53.967 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:05:53.967 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:05:53.967 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:05:53.967 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:05:53.968 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:05:53.968 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:05:53.968 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:05:53.968 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:05:53.968 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:05:53.968 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:05:53.968 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:05:53.968 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:05:53.968 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2024-04-24T23:05:53.968 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2024-04-24T23:05:53.968 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2024-04-24T23:05:53.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:53.968 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:05:53.968 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:05:53.969 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2024-04-24T23:05:53.969 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:53.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:53.969 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:05:53.969 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:05:53.969 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 2, 2024-04-24T23:05:53.969 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 2, 2024-04-24T23:05:53.969 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:05:53.969 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-04-24T23:05:53.969 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-04-24T23:05:53.969 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:05:53.969 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:05:53.969 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:05:53.969 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:53.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:53.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:53.970 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 17, 2024-04-24T23:05:53.970 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2024-04-24T23:05:53.970 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:05:32.760777+0000", 2024-04-24T23:05:53.970 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1027, 2024-04-24T23:05:53.970 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,full,full_quota", 2024-04-24T23:05:53.970 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:05:53.970 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:05:53.970 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:05:53.970 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:05:53.970 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:05:53.970 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:05:53.970 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:05:53.970 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:05:53.970 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:05:53.971 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:05:53.971 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 64, 2024-04-24T23:05:53.971 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 64, 2024-04-24T23:05:53.971 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 64, 2024-04-24T23:05:53.971 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 64, 2024-04-24T23:05:53.971 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 64, 2024-04-24T23:05:53.971 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:05:53.971 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:05:53.971 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:05:53.971 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:05:53.971 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:05:53.971 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:05:53.971 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:05:53.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:53.971 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "142", 2024-04-24T23:05:53.972 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:05:53.972 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:05:53.972 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:05:53.972 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:05:53.972 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:05:53.972 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:05:53.972 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:05:53.972 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:05:53.972 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:05:53.972 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 33554432, 2024-04-24T23:05:53.972 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:05:53.972 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:05:53.972 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:05:53.972 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:05:53.973 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:05:53.973 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:05:53.973 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:05:53.973 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:05:53.973 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:05:53.973 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:05:53.973 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:05:53.973 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:05:53.973 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:05:53.973 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:05:53.973 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:05:53.973 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:05:53.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:53.973 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:05:53.974 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:05:53.974 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:05:53.974 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:05:53.974 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:05:53.974 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:05:53.974 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:05:53.974 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:05:53.974 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:05:53.974 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:05:53.974 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:05:53.974 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:05:53.974 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 64, 2024-04-24T23:05:53.974 INFO:teuthology.orchestra.run.smithi032.stdout: "target_size_ratio": 0.89999997615814209 2024-04-24T23:05:53.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:53.975 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:05:53.975 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:05:53.975 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2024-04-24T23:05:53.975 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:53.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:53.975 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:05:53.975 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:05:53.975 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.3799999952316284, 2024-04-24T23:05:53.975 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.3799999952316284, 2024-04-24T23:05:53.975 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:05:53.975 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.3799999952316284, 2024-04-24T23:05:53.975 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.3799999952316284, 2024-04-24T23:05:53.975 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:05:53.975 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:05:53.976 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:05:53.976 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:53.976 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:53.976 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:05:53.976 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-04-24T23:05:53.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:53.976 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:05:53.976 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:05:53.976 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:05:53.976 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:05:53.976 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:05:53.976 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:05:53.976 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:05:53.976 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:05:53.977 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:05:53.977 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 143, 2024-04-24T23:05:53.977 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:05:53.977 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:05:53.977 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:05:53.977 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:53.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:53.977 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:53.977 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2024-04-24T23:05:53.977 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:05:53.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:53.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:53.977 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:53.977 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2024-04-24T23:05:53.978 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:05:53.978 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:53.978 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:53.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:53.978 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:05:53.978 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:53.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:53.978 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:53.978 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2024-04-24T23:05:53.978 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:05:53.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:53.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:53.978 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:53.978 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2024-04-24T23:05:53.979 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:05:53.979 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:53.979 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:53.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:53.979 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:05:53.979 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:53.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:53.979 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:53.979 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2024-04-24T23:05:53.979 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:05:53.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:53.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:53.979 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:53.979 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2024-04-24T23:05:53.979 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:05:53.980 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:53.980 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:53.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:53.980 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:05:53.980 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:53.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:53.980 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:53.980 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2024-04-24T23:05:53.980 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:05:53.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:53.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:53.980 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:53.980 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2024-04-24T23:05:53.980 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:05:53.980 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:53.981 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:53.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:53.981 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/1479395426", 2024-04-24T23:05:53.981 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/1479395426", 2024-04-24T23:05:53.981 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/1479395426", 2024-04-24T23:05:53.981 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/1479395426", 2024-04-24T23:05:53.981 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:05:53.981 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:05:53.981 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:05:53.981 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:53.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:53.981 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:53.981 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:05:53.981 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:05:53.982 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:05:53.982 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:05:53.982 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:05:53.982 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:05:53.982 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:05:53.982 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:05:53.982 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:05:53.982 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 143, 2024-04-24T23:05:53.982 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:05:53.982 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:05:53.982 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:05:53.982 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:53.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:53.982 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:53.983 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2024-04-24T23:05:53.983 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:05:53.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:53.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:53.983 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:53.983 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2024-04-24T23:05:53.983 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:05:53.983 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:53.983 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:53.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:53.983 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:05:53.983 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:53.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:53.983 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:53.983 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2024-04-24T23:05:53.984 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:05:53.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:53.984 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:53.984 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:53.984 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2024-04-24T23:05:53.984 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:05:53.984 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:53.984 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:53.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:53.984 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:05:53.984 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:53.984 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:53.984 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:53.984 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2024-04-24T23:05:53.985 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:05:53.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:53.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:53.985 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:53.985 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2024-04-24T23:05:53.985 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:05:53.985 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:53.985 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:53.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:53.985 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:05:53.985 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:53.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:53.985 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:53.985 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2024-04-24T23:05:53.986 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:05:53.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:53.986 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:53.986 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:53.986 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2024-04-24T23:05:53.986 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:05:53.986 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:53.986 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:53.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:53.986 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/1340633718", 2024-04-24T23:05:53.986 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/1340633718", 2024-04-24T23:05:53.986 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/1340633718", 2024-04-24T23:05:53.986 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/1340633718", 2024-04-24T23:05:53.986 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:05:53.987 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:05:53.987 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:05:53.987 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:53.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:53.987 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:53.987 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:05:53.987 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:05:53.987 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:05:53.987 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:05:53.987 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:05:53.987 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:05:53.987 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:05:53.987 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:05:53.987 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:05:53.987 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 138, 2024-04-24T23:05:53.988 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:05:53.988 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:05:53.988 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:05:53.988 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:53.988 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:53.988 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:53.988 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2024-04-24T23:05:53.988 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:05:53.988 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:53.988 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:53.988 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:53.988 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2024-04-24T23:05:53.988 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:05:53.988 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:53.988 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:53.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:53.989 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:05:53.989 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:53.989 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:53.989 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:53.989 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2024-04-24T23:05:53.989 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:05:53.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:53.989 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:53.989 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:53.989 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2024-04-24T23:05:53.989 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:05:53.989 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:53.989 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:53.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:53.990 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:05:53.990 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:53.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:53.990 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:53.990 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2024-04-24T23:05:53.990 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:05:53.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:53.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:53.990 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:53.990 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2024-04-24T23:05:53.990 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:05:53.990 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:53.991 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:53.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:53.991 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:05:53.991 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:53.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:53.991 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:53.991 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2024-04-24T23:05:53.991 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:05:53.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:53.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:53.991 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:53.991 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2024-04-24T23:05:53.991 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:05:53.991 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:53.992 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:53.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:53.992 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/3405899334", 2024-04-24T23:05:53.992 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6820/3405899334", 2024-04-24T23:05:53.992 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6828/3405899334", 2024-04-24T23:05:53.992 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6824/3405899334", 2024-04-24T23:05:53.992 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:05:53.992 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:05:53.992 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:05:53.992 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:53.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:53.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:53.992 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:05:53.992 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:05:53.992 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:05:53.993 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:05:53.993 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:05:53.993 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:05:53.993 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:05:53.993 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:05:53.993 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:05:53.993 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 143, 2024-04-24T23:05:53.993 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:05:53.993 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:05:53.993 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:05:53.993 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:53.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:53.993 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:53.993 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2024-04-24T23:05:53.994 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:05:53.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:53.994 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:53.994 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:53.994 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2024-04-24T23:05:53.994 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:05:53.994 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:53.994 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:53.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:53.995 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:05:53.995 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:53.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:53.995 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:53.995 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2024-04-24T23:05:53.995 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:05:53.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:53.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:53.995 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:53.995 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2024-04-24T23:05:53.995 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:05:53.995 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:53.995 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:53.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:53.996 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:05:53.996 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:53.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:53.996 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:53.996 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2024-04-24T23:05:53.996 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:05:53.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:53.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:53.996 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:53.996 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2024-04-24T23:05:53.996 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:05:53.996 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:53.996 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:53.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:53.997 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:05:53.997 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:53.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:53.997 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:53.997 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2024-04-24T23:05:53.997 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:05:53.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:53.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:53.997 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:53.997 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2024-04-24T23:05:53.997 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:05:53.997 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:53.997 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:53.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:53.997 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6821/3381057190", 2024-04-24T23:05:53.998 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6825/3381057190", 2024-04-24T23:05:53.998 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/3381057190", 2024-04-24T23:05:53.998 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/3381057190", 2024-04-24T23:05:53.998 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:05:53.998 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:05:53.998 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:05:53.998 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:53.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:53.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:53.998 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:05:53.998 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:05:53.998 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:05:53.998 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:05:53.998 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:05:53.999 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:05:53.999 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:05:53.999 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:05:53.999 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:05:53.999 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 138, 2024-04-24T23:05:53.999 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:05:53.999 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:05:53.999 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:05:53.999 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:53.999 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:53.999 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:53.999 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6800", 2024-04-24T23:05:53.999 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:05:53.999 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.000 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:54.000 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6801", 2024-04-24T23:05:54.000 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:05:54.000 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.000 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.000 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:05:54.000 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:54.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.000 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:54.000 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6802", 2024-04-24T23:05:54.000 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:05:54.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.001 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:54.001 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6803", 2024-04-24T23:05:54.001 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:05:54.001 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.001 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.001 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:05:54.001 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:54.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.001 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:54.001 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6806", 2024-04-24T23:05:54.001 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:05:54.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.001 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:54.002 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6807", 2024-04-24T23:05:54.002 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:05:54.002 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.002 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.002 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:05:54.002 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:54.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.002 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:54.002 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6804", 2024-04-24T23:05:54.002 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:05:54.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.003 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:54.003 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6805", 2024-04-24T23:05:54.003 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:05:54.003 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.003 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.003 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6801/4010553979", 2024-04-24T23:05:54.003 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6803/4010553979", 2024-04-24T23:05:54.003 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6807/4010553979", 2024-04-24T23:05:54.003 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6805/4010553979", 2024-04-24T23:05:54.003 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:05:54.003 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:05:54.003 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:05:54.003 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.004 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.004 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:05:54.004 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:05:54.004 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:05:54.004 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:05:54.004 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:05:54.004 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:05:54.004 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:05:54.004 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:05:54.004 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:05:54.004 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 138, 2024-04-24T23:05:54.004 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:05:54.004 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:05:54.004 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:05:54.005 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:54.005 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.005 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:54.005 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6808", 2024-04-24T23:05:54.005 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:05:54.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.005 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.005 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:54.005 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6809", 2024-04-24T23:05:54.005 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:05:54.005 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.005 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.005 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:05:54.005 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:54.006 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.006 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:54.006 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6810", 2024-04-24T23:05:54.006 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:05:54.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.006 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.006 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:54.006 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6811", 2024-04-24T23:05:54.006 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:05:54.006 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.006 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.006 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:05:54.006 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:54.007 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.007 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:54.007 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6814", 2024-04-24T23:05:54.007 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:05:54.007 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.007 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.007 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:54.007 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6815", 2024-04-24T23:05:54.007 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:05:54.007 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.007 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.007 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.007 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:05:54.008 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:54.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.008 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:54.008 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6812", 2024-04-24T23:05:54.008 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:05:54.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.008 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:54.008 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6813", 2024-04-24T23:05:54.008 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:05:54.008 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.008 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.008 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6809/3794084091", 2024-04-24T23:05:54.008 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6811/3794084091", 2024-04-24T23:05:54.008 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6815/3794084091", 2024-04-24T23:05:54.009 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6813/3794084091", 2024-04-24T23:05:54.009 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:05:54.009 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:05:54.009 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:05:54.009 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.009 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.009 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:05:54.009 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:05:54.009 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:05:54.009 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:05:54.009 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:05:54.009 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:05:54.009 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:05:54.010 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:05:54.010 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:05:54.010 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 138, 2024-04-24T23:05:54.010 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:05:54.010 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:05:54.010 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:05:54.010 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:54.010 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.010 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:54.010 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6823", 2024-04-24T23:05:54.010 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:05:54.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.010 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.010 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:54.011 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6825", 2024-04-24T23:05:54.011 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:05:54.011 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.011 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.011 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:05:54.011 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:54.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.011 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:54.011 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6826", 2024-04-24T23:05:54.011 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:05:54.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.011 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:54.011 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6827", 2024-04-24T23:05:54.012 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:05:54.012 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.012 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.012 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:05:54.012 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:54.012 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.012 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:54.012 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6830", 2024-04-24T23:05:54.012 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:05:54.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.012 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.012 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:54.012 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6831", 2024-04-24T23:05:54.012 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:05:54.013 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.013 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.013 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:05:54.013 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:54.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.013 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:54.013 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6828", 2024-04-24T23:05:54.013 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:05:54.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.013 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:54.013 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6829", 2024-04-24T23:05:54.013 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:05:54.014 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.014 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.014 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6825/1348640258", 2024-04-24T23:05:54.014 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6827/1348640258", 2024-04-24T23:05:54.014 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6831/1348640258", 2024-04-24T23:05:54.014 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6829/1348640258", 2024-04-24T23:05:54.014 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:05:54.014 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:05:54.014 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:05:54.014 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.014 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.014 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:05:54.015 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:05:54.015 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:05:54.015 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:05:54.015 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:05:54.015 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:05:54.015 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:05:54.015 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:05:54.015 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:05:54.015 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 138, 2024-04-24T23:05:54.015 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:05:54.015 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:05:54.015 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:05:54.015 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:54.015 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.015 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:54.015 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6816", 2024-04-24T23:05:54.016 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:05:54.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.016 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:54.016 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6817", 2024-04-24T23:05:54.016 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:05:54.016 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.016 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.016 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:05:54.016 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:54.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.016 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:54.016 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6818", 2024-04-24T23:05:54.016 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:05:54.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.017 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.017 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:54.017 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6819", 2024-04-24T23:05:54.017 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:05:54.017 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.017 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.017 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:05:54.017 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:54.017 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.017 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:54.017 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6822", 2024-04-24T23:05:54.017 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:05:54.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.018 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.018 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:54.018 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6824", 2024-04-24T23:05:54.018 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:05:54.018 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.018 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.018 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.018 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:05:54.018 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:54.018 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.018 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:54.018 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6820", 2024-04-24T23:05:54.018 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:05:54.018 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.019 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.019 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:54.019 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6821", 2024-04-24T23:05:54.019 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:05:54.019 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.019 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.019 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6817/3250005122", 2024-04-24T23:05:54.019 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6819/3250005122", 2024-04-24T23:05:54.019 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6824/3250005122", 2024-04-24T23:05:54.019 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6821/3250005122", 2024-04-24T23:05:54.019 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:05:54.019 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:05:54.019 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:05:54.019 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.020 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.020 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:05:54.020 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2024-04-24T23:05:54.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.020 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:05:54.020 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:05:54.020 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:05:54.020 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:05:54.020 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:05:54.020 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:05:54.020 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.156184+0000", 2024-04-24T23:05:54.020 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:05:54.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.020 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:05:54.021 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:05:54.021 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:05:54.021 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:05:54.021 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:05:54.021 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:05:54.021 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240774+0000", 2024-04-24T23:05:54.021 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:05:54.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.021 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.021 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:05:54.021 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:05:54.021 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:05:54.021 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:05:54.021 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:05:54.021 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:05:54.022 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240949+0000", 2024-04-24T23:05:54.022 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:05:54.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.022 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:05:54.022 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:05:54.022 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:05:54.022 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:05:54.022 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:05:54.022 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:05:54.022 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.171672+0000", 2024-04-24T23:05:54.022 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:05:54.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.023 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:05:54.023 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:05:54.023 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:05:54.023 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:05:54.023 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:05:54.023 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:05:54.023 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.220379+0000", 2024-04-24T23:05:54.023 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:05:54.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.023 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:05:54.023 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:05:54.023 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:05:54.023 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:05:54.024 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:05:54.024 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:05:54.024 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.204963+0000", 2024-04-24T23:05:54.024 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:05:54.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.024 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.024 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:05:54.024 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:05:54.024 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:05:54.024 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:05:54.024 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:05:54.024 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:05:54.024 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.239748+0000", 2024-04-24T23:05:54.024 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:05:54.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.025 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.025 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:05:54.025 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:05:54.025 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:05:54.025 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:05:54.025 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:05:54.025 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:05:54.025 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.154104+0000", 2024-04-24T23:05:54.025 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:05:54.025 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.025 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:05:54.025 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2024-04-24T23:05:54.025 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [ 2024-04-24T23:05:54.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.026 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "16.17", 2024-04-24T23:05:54.026 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:05:54.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.026 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 4, 2024-04-24T23:05:54.026 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 3 2024-04-24T23:05:54.026 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.026 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.026 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "17.8", 2024-04-24T23:05:54.026 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:05:54.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.026 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 6, 2024-04-24T23:05:54.027 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 5 2024-04-24T23:05:54.027 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.027 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.027 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "17.3f", 2024-04-24T23:05:54.027 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:05:54.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.027 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 6, 2024-04-24T23:05:54.027 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 1 2024-04-24T23:05:54.027 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.027 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.027 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.027 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:05:54.027 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_primaries": [], 2024-04-24T23:05:54.028 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2024-04-24T23:05:54.028 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2024-04-24T23:05:54.028 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklist": {}, 2024-04-24T23:05:54.028 INFO:teuthology.orchestra.run.smithi032.stdout: "range_blocklist": {}, 2024-04-24T23:05:54.028 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2024-04-24T23:05:54.028 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2024-04-24T23:05:54.028 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2024-04-24T23:05:54.028 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2024-04-24T23:05:54.028 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2024-04-24T23:05:54.028 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2024-04-24T23:05:54.028 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2024-04-24T23:05:54.028 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.028 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2024-04-24T23:05:54.028 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2024-04-24T23:05:54.029 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2024-04-24T23:05:54.029 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2024-04-24T23:05:54.029 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {}, 2024-04-24T23:05:54.029 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode": { 2024-04-24T23:05:54.029 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_enabled": false, 2024-04-24T23:05:54.029 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_bucket_count": 0, 2024-04-24T23:05:54.029 INFO:teuthology.orchestra.run.smithi032.stdout: "degraded_stretch_mode": 0, 2024-04-24T23:05:54.029 INFO:teuthology.orchestra.run.smithi032.stdout: "recovering_stretch_mode": 0, 2024-04-24T23:05:54.029 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_bucket": 0 2024-04-24T23:05:54.029 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.029 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:05:54.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.955+0000 7fdd7955a640 1 -- 172.21.15.32:0/256666521 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdd5406be30 msgr2=0x7fdd5406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:54.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.955+0000 7fdd7955a640 1 --2- 172.21.15.32:0/256666521 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdd5406be30 0x7fdd5406e2f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fdd64034000 tx=0x7fdd6402e000 comp rx=0 tx=0).stop 2024-04-24T23:05:54.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.955+0000 7fdd7955a640 1 -- 172.21.15.32:0/256666521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd7415cb00 msgr2=0x7fdd74141300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:54.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.955+0000 7fdd7955a640 1 --2- 172.21.15.32:0/256666521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd7415cb00 0x7fdd74141300 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c00c7d0 tx=0x7fdd6c00cc90 comp rx=0 tx=0).stop 2024-04-24T23:05:54.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.955+0000 7fdd7955a640 1 -- 172.21.15.32:0/256666521 shutdown_connections 2024-04-24T23:05:54.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.955+0000 7fdd7955a640 1 --2- 172.21.15.32:0/256666521 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdd5406be30 0x7fdd5406e2f0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:54.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.955+0000 7fdd7955a640 1 --2- 172.21.15.32:0/256666521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd7415cb00 0x7fdd74141300 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:54.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.955+0000 7fdd7955a640 1 --2- 172.21.15.32:0/256666521 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdd7415b380 0x7fdd7413f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:54.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.955+0000 7fdd7955a640 1 --2- 172.21.15.32:0/256666521 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd740ceb40 0x7fdd74142740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:54.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.955+0000 7fdd7955a640 1 -- 172.21.15.32:0/256666521 >> 172.21.15.32:0/256666521 conn(0x7fdd7406b110 msgr2=0x7fdd740b6060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:54.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.955+0000 7fdd7955a640 1 -- 172.21.15.32:0/256666521 shutdown_connections 2024-04-24T23:05:54.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:53.955+0000 7fdd7955a640 1 -- 172.21.15.32:0/256666521 wait complete. 2024-04-24T23:05:54.032 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2024-04-24T23:05:54.032 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:05:54.033 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:05:54.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:05:54.048 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=16) 2024-04-24T23:05:54.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:05:54.250 INFO:teuthology.orchestra.run.smithi032.stderr:dd: fdatasync failed for 'large_file_b': No space left on device 2024-04-24T23:05:54.250 INFO:teuthology.orchestra.run.smithi032.stderr:50+0 records in 2024-04-24T23:05:54.250 INFO:teuthology.orchestra.run.smithi032.stderr:50+0 records out 2024-04-24T23:05:54.250 INFO:teuthology.orchestra.run.smithi032.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.142908 s, 367 MB/s 2024-04-24T23:05:54.250 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-24T23:05:54.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.349+0000 7f81c4efc640 1 Processor -- start 2024-04-24T23:05:54.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.349+0000 7f81c4efc640 1 -- start start 2024-04-24T23:05:54.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.350+0000 7f81c4efc640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f81c015c5b0 0x7f81c00b4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:54.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.350+0000 7f81c4efc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f81c0072cf0 0x7f81c006c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:54.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.350+0000 7f81c4efc640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81c015c990 con 0x7f81c0071770 2024-04-24T23:05:54.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.350+0000 7f81c4efc640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f81c0072860 con 0x7f81c0072cf0 2024-04-24T23:05:54.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.350+0000 7f81c4efc640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f81c0071c50 con 0x7f81c015c5b0 2024-04-24T23:05:54.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.350+0000 7f81b7fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f81c0072cf0 0x7f81c006c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:54.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.350+0000 7f81b7fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f81c0072cf0 0x7f81c006c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38346/0 (socket says 172.21.15.32:38346) 2024-04-24T23:05:54.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.350+0000 7f81bed9d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f81c0071770 0x7f81c00cef70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59990/0 (socket says 172.21.15.32:59990) 2024-04-24T23:05:54.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.350+0000 7f81b7fff640 1 -- 172.21.15.32:0/1672588661 learned_addr learned my addr 172.21.15.32:0/1672588661 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:54.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.350+0000 7f81be59c640 1 --2- 172.21.15.32:0/1672588661 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f81c015c5b0 0x7f81c00b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:54.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.350+0000 7f81b7fff640 1 -- 172.21.15.32:0/1672588661 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f81c015c5b0 msgr2=0x7f81c00b4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:54.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.350+0000 7f81b7fff640 1 --2- 172.21.15.32:0/1672588661 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f81c015c5b0 0x7f81c00b4030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:54.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.350+0000 7f81b7fff640 1 -- 172.21.15.32:0/1672588661 >> v1:172.21.15.32:6789/0 conn(0x7f81c0071770 legacy=0x7f81c00cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:54.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.350+0000 7f81bdd9b640 1 -- 172.21.15.32:0/1672588661 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1892458643 0 0) 0x7f81b8001490 con 0x7f81c0071770 2024-04-24T23:05:54.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.350+0000 7f81b7fff640 1 -- 172.21.15.32:0/1672588661 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81c0071f30 con 0x7f81c0072cf0 2024-04-24T23:05:54.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.351+0000 7f81b7fff640 1 --2- 172.21.15.32:0/1672588661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81c0072cf0 0x7f81c006c5f0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f81a8009870 tx=0x7f81a802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8df434f7ffb494d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:54.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.351+0000 7f81bdd9b640 1 -- 172.21.15.32:0/1672588661 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a8037070 con 0x7f81c0072cf0 2024-04-24T23:05:54.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.351+0000 7f81bdd9b640 1 -- 172.21.15.32:0/1672588661 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81a8032440 con 0x7f81c0072cf0 2024-04-24T23:05:54.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.351+0000 7f81bdd9b640 1 -- 172.21.15.32:0/1672588661 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a8032810 con 0x7f81c0072cf0 2024-04-24T23:05:54.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.351+0000 7f81c4efc640 1 -- 172.21.15.32:0/1672588661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81c0072cf0 msgr2=0x7f81c006c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:54.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.351+0000 7f81c4efc640 1 --2- 172.21.15.32:0/1672588661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81c0072cf0 0x7f81c006c5f0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f81a8009870 tx=0x7f81a802a1c0 comp rx=0 tx=0).stop 2024-04-24T23:05:54.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.351+0000 7f81c4efc640 1 -- 172.21.15.32:0/1672588661 shutdown_connections 2024-04-24T23:05:54.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.351+0000 7f81c4efc640 1 --2- 172.21.15.32:0/1672588661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81c0072cf0 0x7f81c006c5f0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:54.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.351+0000 7f81c4efc640 1 --2- 172.21.15.32:0/1672588661 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f81c015c5b0 0x7f81c00b4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:54.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.352+0000 7f81c4efc640 1 -- 172.21.15.32:0/1672588661 >> 172.21.15.32:0/1672588661 conn(0x7f81c006b110 msgr2=0x7f81c006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:54.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.352+0000 7f81c4efc640 1 -- 172.21.15.32:0/1672588661 shutdown_connections 2024-04-24T23:05:54.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.352+0000 7f81c4efc640 1 -- 172.21.15.32:0/1672588661 wait complete. 2024-04-24T23:05:54.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.353+0000 7f81c4efc640 1 Processor -- start 2024-04-24T23:05:54.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.353+0000 7f81c4efc640 1 -- start start 2024-04-24T23:05:54.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.353+0000 7f81c4efc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81c0071770 0x7f81c0157af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:54.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.354+0000 7f81c4efc640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81c0072cf0 0x7f81c015cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:54.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.354+0000 7f81c4efc640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f81c015c5b0 0x7f81c00ce4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:54.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.354+0000 7f81c4efc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f81c00b4570 con 0x7f81c0071770 2024-04-24T23:05:54.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.354+0000 7f81c4efc640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f81c00724f0 con 0x7f81c0072cf0 2024-04-24T23:05:54.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.354+0000 7f81c4efc640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f81c00b46f0 con 0x7f81c015c5b0 2024-04-24T23:05:54.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.354+0000 7f81be59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81c0071770 0x7f81c0157af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:54.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.354+0000 7f81be59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81c0071770 0x7f81c0157af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38350/0 (socket says 172.21.15.32:38350) 2024-04-24T23:05:54.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.354+0000 7f81be59c640 1 -- 172.21.15.32:0/110135425 learned_addr learned my addr 172.21.15.32:0/110135425 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:54.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.354+0000 7f81b7fff640 1 --2- 172.21.15.32:0/110135425 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81c0072cf0 0x7f81c015cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:54.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.354+0000 7f81be59c640 1 -- 172.21.15.32:0/110135425 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f81c015c5b0 msgr2=0x7f81c00ce4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:54.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.354+0000 7f81bed9d640 1 --2- 172.21.15.32:0/110135425 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f81c015c5b0 0x7f81c00ce4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:54.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.354+0000 7f81be59c640 1 --2- 172.21.15.32:0/110135425 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f81c015c5b0 0x7f81c00ce4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:54.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.354+0000 7f81be59c640 1 -- 172.21.15.32:0/110135425 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81c0072cf0 msgr2=0x7f81c015cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:54.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.354+0000 7f81be59c640 1 --2- 172.21.15.32:0/110135425 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81c0072cf0 0x7f81c015cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:54.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.354+0000 7f81be59c640 1 -- 172.21.15.32:0/110135425 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81c01580d0 con 0x7f81c0071770 2024-04-24T23:05:54.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.354+0000 7f81be59c640 1 --2- 172.21.15.32:0/110135425 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81c0071770 0x7f81c0157af0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f81b000c7f0 tx=0x7f81b000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:54.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.354+0000 7f81b6ffd640 1 -- 172.21.15.32:0/110135425 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b0015070 con 0x7f81c0071770 2024-04-24T23:05:54.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.354+0000 7f81b6ffd640 1 -- 172.21.15.32:0/110135425 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81b0004220 con 0x7f81c0071770 2024-04-24T23:05:54.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.354+0000 7f81b6ffd640 1 -- 172.21.15.32:0/110135425 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b0007620 con 0x7f81c0071770 2024-04-24T23:05:54.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.354+0000 7f81c4efc640 1 -- 172.21.15.32:0/110135425 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81c01583b0 con 0x7f81c0071770 2024-04-24T23:05:54.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.354+0000 7f81c4efc640 1 -- 172.21.15.32:0/110135425 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81c00cdfc0 con 0x7f81c0071770 2024-04-24T23:05:54.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.355+0000 7f81c4efc640 1 -- 172.21.15.32:0/110135425 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81c0068430 con 0x7f81c0071770 2024-04-24T23:05:54.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.356+0000 7f81b6ffd640 1 -- 172.21.15.32:0/110135425 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f81b00079d0 con 0x7f81c0071770 2024-04-24T23:05:54.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.356+0000 7f81b6ffd640 1 --2- 172.21.15.32:0/110135425 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f81a006be30 0x7f81a006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:54.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.356+0000 7f81b6ffd640 1 -- 172.21.15.32:0/110135425 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5796+0+0 (secure 0 0 0) 0x7f81b008b290 con 0x7f81c0071770 2024-04-24T23:05:54.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.356+0000 7f81b6ffd640 1 -- 172.21.15.32:0/110135425 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=145}) -- 0x7f81a0073ae0 con 0x7f81c0071770 2024-04-24T23:05:54.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.356+0000 7f81b7fff640 1 --2- 172.21.15.32:0/110135425 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f81a006be30 0x7f81a006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:54.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.357+0000 7f81b7fff640 1 --2- 172.21.15.32:0/110135425 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f81a006be30 0x7f81a006e2f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f81a8034000 tx=0x7f81a802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:54.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.360+0000 7f81b6ffd640 1 -- 172.21.15.32:0/110135425 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f81b0059be0 con 0x7f81c0071770 2024-04-24T23:05:54.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.543+0000 7f81c4efc640 1 -- 172.21.15.32:0/110135425 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f81c014e410 con 0x7f81c0071770 2024-04-24T23:05:54.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.544+0000 7f81b6ffd640 1 -- 172.21.15.32:0/110135425 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v144) ==== 81+0+34760 (secure 0 0 0) 0x7f81b00594e0 con 0x7f81c0071770 2024-04-24T23:05:54.546 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:05:54.547 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:05:54.547 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 144, 2024-04-24T23:05:54.547 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", 2024-04-24T23:05:54.547 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2024-04-24T22:58:46.964851+0000", 2024-04-24T23:05:54.547 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2024-04-24T23:05:53.811291+0000", 2024-04-24T23:05:54.547 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2024-04-24T22:58:52.048787+0000", 2024-04-24T23:05:54.547 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2024-04-24T22:58:49.045609+0000", 2024-04-24T23:05:54.547 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-24T23:05:54.547 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2024-04-24T23:05:54.547 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2024-04-24T23:05:54.547 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2024-04-24T23:05:54.547 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2024-04-24T23:05:54.547 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2024-04-24T23:05:54.547 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2024-04-24T23:05:54.548 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:05:54.548 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 4, 2024-04-24T23:05:54.548 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.69999998807907104, 2024-04-24T23:05:54.548 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-24T23:05:54.548 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-24T23:05:54.548 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2024-04-24T23:05:54.548 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 17, 2024-04-24T23:05:54.548 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2024-04-24T23:05:54.548 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "luminous", 2024-04-24T23:05:54.548 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "luminous", 2024-04-24T23:05:54.548 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "squid", 2024-04-24T23:05:54.548 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_crimson": false, 2024-04-24T23:05:54.548 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2024-04-24T23:05:54.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.548 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2024-04-24T23:05:54.549 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": ".mgr", 2024-04-24T23:05:54.549 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T22:58:53.171408+0000", 2024-04-24T23:05:54.549 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:05:54.549 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:05:54.549 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:05:54.549 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:05:54.549 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:05:54.549 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:05:54.549 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:05:54.549 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:05:54.549 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:05:54.549 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:05:54.549 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:05:54.549 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:05:54.549 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2024-04-24T23:05:54.550 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2024-04-24T23:05:54.550 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2024-04-24T23:05:54.550 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2024-04-24T23:05:54.550 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2024-04-24T23:05:54.550 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:05:54.550 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:05:54.550 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:05:54.550 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:05:54.550 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:05:54.550 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:05:54.550 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:05:54.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.550 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "113", 2024-04-24T23:05:54.550 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:05:54.551 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:05:54.551 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:05:54.551 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:05:54.551 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:05:54.551 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:05:54.551 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:05:54.551 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:05:54.551 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:05:54.551 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:05:54.551 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:05:54.551 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:05:54.551 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:05:54.551 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:05:54.551 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:05:54.551 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:05:54.552 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:05:54.552 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:05:54.552 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:05:54.552 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:05:54.552 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:05:54.552 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:05:54.552 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:05:54.552 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:05:54.552 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:05:54.552 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:05:54.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.552 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:05:54.552 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:05:54.552 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:05:54.553 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:05:54.553 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:05:54.553 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:05:54.553 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:05:54.553 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:05:54.553 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:05:54.553 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:05:54.553 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:05:54.553 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:05:54.553 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_max": 32, 2024-04-24T23:05:54.553 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2024-04-24T23:05:54.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.553 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:05:54.553 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": {} 2024-04-24T23:05:54.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.554 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:05:54.554 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:05:54.554 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 8, 2024-04-24T23:05:54.554 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 8, 2024-04-24T23:05:54.554 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.25, 2024-04-24T23:05:54.554 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-04-24T23:05:54.554 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-04-24T23:05:54.554 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:05:54.554 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:05:54.554 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:05:54.554 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.554 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 16, 2024-04-24T23:05:54.555 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2024-04-24T23:05:54.555 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:05:32.064234+0000", 2024-04-24T23:05:54.555 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:05:54.555 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:05:54.555 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:05:54.555 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:05:54.555 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:05:54.555 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:05:54.555 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:05:54.555 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:05:54.555 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:05:54.555 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:05:54.555 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:05:54.555 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:05:54.555 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 32, 2024-04-24T23:05:54.556 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 32, 2024-04-24T23:05:54.556 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 32, 2024-04-24T23:05:54.556 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 32, 2024-04-24T23:05:54.556 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 32, 2024-04-24T23:05:54.556 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:05:54.556 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:05:54.556 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:05:54.556 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:05:54.556 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:05:54.556 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:05:54.556 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:05:54.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.556 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "139", 2024-04-24T23:05:54.556 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:05:54.556 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:05:54.557 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:05:54.557 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:05:54.557 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:05:54.557 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:05:54.557 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:05:54.557 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:05:54.557 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:05:54.557 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:05:54.557 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:05:54.557 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:05:54.557 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:05:54.557 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:05:54.557 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:05:54.557 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:05:54.557 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:05:54.558 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:05:54.558 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:05:54.558 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:05:54.558 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:05:54.558 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:05:54.558 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:05:54.558 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:05:54.558 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:05:54.558 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:05:54.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.558 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:05:54.558 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:05:54.558 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:05:54.559 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:05:54.559 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:05:54.559 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:05:54.559 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:05:54.559 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:05:54.559 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:05:54.559 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:05:54.559 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:05:54.559 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:05:54.559 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2024-04-24T23:05:54.559 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2024-04-24T23:05:54.559 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2024-04-24T23:05:54.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.559 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:05:54.559 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:05:54.559 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2024-04-24T23:05:54.560 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.560 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:05:54.560 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:05:54.560 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 2, 2024-04-24T23:05:54.560 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 2, 2024-04-24T23:05:54.560 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:05:54.560 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-04-24T23:05:54.560 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-04-24T23:05:54.560 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:05:54.560 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:05:54.560 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:05:54.560 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.561 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 17, 2024-04-24T23:05:54.561 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2024-04-24T23:05:54.561 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:05:32.760777+0000", 2024-04-24T23:05:54.561 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1027, 2024-04-24T23:05:54.561 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,full,full_quota", 2024-04-24T23:05:54.561 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:05:54.561 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:05:54.561 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:05:54.561 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:05:54.561 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:05:54.561 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:05:54.561 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:05:54.561 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:05:54.562 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:05:54.562 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:05:54.562 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 64, 2024-04-24T23:05:54.562 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 64, 2024-04-24T23:05:54.562 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 64, 2024-04-24T23:05:54.562 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 64, 2024-04-24T23:05:54.562 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 64, 2024-04-24T23:05:54.562 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:05:54.562 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:05:54.562 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:05:54.562 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:05:54.562 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:05:54.562 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:05:54.562 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:05:54.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.563 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "142", 2024-04-24T23:05:54.563 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:05:54.563 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:05:54.563 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:05:54.563 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:05:54.563 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:05:54.563 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:05:54.563 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:05:54.563 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:05:54.563 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:05:54.563 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 33554432, 2024-04-24T23:05:54.563 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:05:54.563 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:05:54.564 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:05:54.564 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:05:54.564 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:05:54.564 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:05:54.564 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:05:54.564 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:05:54.564 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:05:54.564 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:05:54.564 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:05:54.564 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:05:54.564 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:05:54.564 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:05:54.564 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:05:54.564 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:05:54.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.565 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:05:54.565 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:05:54.565 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:05:54.565 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:05:54.565 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:05:54.565 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:05:54.565 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:05:54.565 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:05:54.565 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:05:54.565 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:05:54.565 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:05:54.565 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:05:54.565 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 64, 2024-04-24T23:05:54.565 INFO:teuthology.orchestra.run.smithi032.stdout: "target_size_ratio": 0.89999997615814209 2024-04-24T23:05:54.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.566 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:05:54.566 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:05:54.566 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2024-04-24T23:05:54.566 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.566 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:05:54.566 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:05:54.566 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.3799999952316284, 2024-04-24T23:05:54.566 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.3799999952316284, 2024-04-24T23:05:54.566 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:05:54.566 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.3799999952316284, 2024-04-24T23:05:54.566 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.3799999952316284, 2024-04-24T23:05:54.566 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:05:54.566 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:05:54.567 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:05:54.567 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.567 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.567 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:05:54.567 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-04-24T23:05:54.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.567 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:05:54.567 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:05:54.567 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:05:54.567 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:05:54.567 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:05:54.567 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:05:54.567 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:05:54.567 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:05:54.568 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:05:54.568 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 143, 2024-04-24T23:05:54.568 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:05:54.568 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:05:54.568 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:05:54.568 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:54.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.568 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:54.568 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2024-04-24T23:05:54.568 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:05:54.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.568 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:54.568 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2024-04-24T23:05:54.568 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:05:54.569 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.569 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.569 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:05:54.569 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:54.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.569 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:54.569 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2024-04-24T23:05:54.569 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:05:54.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.569 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:54.569 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2024-04-24T23:05:54.569 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:05:54.570 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.570 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.570 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:05:54.570 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:54.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.570 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:54.570 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2024-04-24T23:05:54.570 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:05:54.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.570 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:54.570 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2024-04-24T23:05:54.570 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:05:54.570 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.571 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.571 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:05:54.571 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:54.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.571 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:54.571 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2024-04-24T23:05:54.571 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:05:54.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.571 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:54.571 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2024-04-24T23:05:54.571 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:05:54.571 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.572 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.572 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/1479395426", 2024-04-24T23:05:54.572 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/1479395426", 2024-04-24T23:05:54.572 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/1479395426", 2024-04-24T23:05:54.572 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/1479395426", 2024-04-24T23:05:54.572 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:05:54.572 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:05:54.572 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:05:54.572 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.572 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:05:54.572 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:05:54.572 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:05:54.573 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:05:54.573 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:05:54.573 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:05:54.573 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:05:54.573 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:05:54.573 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:05:54.573 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 143, 2024-04-24T23:05:54.573 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:05:54.573 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:05:54.573 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:05:54.573 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:54.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.573 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:54.573 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2024-04-24T23:05:54.573 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:05:54.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.574 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:54.574 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2024-04-24T23:05:54.574 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:05:54.574 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.574 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.574 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:05:54.574 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:54.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.574 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:54.574 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2024-04-24T23:05:54.574 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:05:54.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.575 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:54.575 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2024-04-24T23:05:54.575 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:05:54.575 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.575 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.575 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:05:54.575 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:54.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.575 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:54.575 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2024-04-24T23:05:54.575 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:05:54.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.576 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:54.576 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2024-04-24T23:05:54.576 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:05:54.576 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.576 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.576 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:05:54.576 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:54.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.576 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:54.576 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2024-04-24T23:05:54.576 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:05:54.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.577 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:54.577 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2024-04-24T23:05:54.577 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:05:54.577 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.577 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.577 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/1340633718", 2024-04-24T23:05:54.577 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/1340633718", 2024-04-24T23:05:54.577 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/1340633718", 2024-04-24T23:05:54.577 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/1340633718", 2024-04-24T23:05:54.577 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:05:54.577 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:05:54.577 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:05:54.578 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.578 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.578 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.578 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:05:54.578 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:05:54.578 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:05:54.578 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:05:54.578 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:05:54.578 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:05:54.578 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:05:54.578 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:05:54.578 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:05:54.578 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 138, 2024-04-24T23:05:54.578 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:05:54.578 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:05:54.579 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:05:54.579 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:54.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.579 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:54.579 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2024-04-24T23:05:54.579 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:05:54.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.579 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:54.579 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2024-04-24T23:05:54.579 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:05:54.579 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.579 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.579 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:05:54.580 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:54.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.580 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:54.580 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2024-04-24T23:05:54.580 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:05:54.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.580 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:54.580 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2024-04-24T23:05:54.580 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:05:54.580 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.580 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.580 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:05:54.581 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:54.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.581 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:54.581 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2024-04-24T23:05:54.581 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:05:54.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.581 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:54.581 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2024-04-24T23:05:54.581 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:05:54.581 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.581 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.581 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:05:54.581 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:54.582 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.582 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:54.582 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2024-04-24T23:05:54.582 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:05:54.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.582 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.582 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:54.582 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2024-04-24T23:05:54.582 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:05:54.582 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.582 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.582 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/3405899334", 2024-04-24T23:05:54.582 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6820/3405899334", 2024-04-24T23:05:54.583 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6828/3405899334", 2024-04-24T23:05:54.583 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6824/3405899334", 2024-04-24T23:05:54.583 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:05:54.583 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:05:54.583 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:05:54.583 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.583 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:05:54.583 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:05:54.583 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:05:54.583 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:05:54.583 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:05:54.583 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:05:54.583 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:05:54.583 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:05:54.584 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:05:54.584 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 143, 2024-04-24T23:05:54.584 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:05:54.584 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:05:54.584 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:05:54.584 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:54.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.584 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:54.584 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2024-04-24T23:05:54.584 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:05:54.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.584 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:54.584 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2024-04-24T23:05:54.584 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:05:54.585 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.585 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.585 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:05:54.585 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:54.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.585 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:54.585 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2024-04-24T23:05:54.585 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:05:54.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.585 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:54.585 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2024-04-24T23:05:54.585 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:05:54.585 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.585 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.586 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:05:54.586 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:54.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.586 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:54.586 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2024-04-24T23:05:54.586 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:05:54.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.586 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:54.586 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2024-04-24T23:05:54.586 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:05:54.586 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.586 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.587 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:05:54.587 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:54.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.587 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:54.587 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2024-04-24T23:05:54.587 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:05:54.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.587 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:54.587 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2024-04-24T23:05:54.587 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:05:54.587 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.587 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.587 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6821/3381057190", 2024-04-24T23:05:54.588 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6825/3381057190", 2024-04-24T23:05:54.588 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/3381057190", 2024-04-24T23:05:54.588 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/3381057190", 2024-04-24T23:05:54.588 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:05:54.588 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:05:54.588 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:05:54.588 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.588 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:05:54.588 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:05:54.588 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:05:54.588 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:05:54.588 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:05:54.589 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:05:54.589 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:05:54.589 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:05:54.589 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:05:54.589 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 138, 2024-04-24T23:05:54.589 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:05:54.589 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:05:54.589 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:05:54.589 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:54.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.589 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:54.589 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6800", 2024-04-24T23:05:54.589 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:05:54.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.590 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:54.590 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6801", 2024-04-24T23:05:54.590 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:05:54.590 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.590 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.590 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:05:54.590 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:54.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.590 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:54.590 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6802", 2024-04-24T23:05:54.590 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:05:54.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.591 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:54.591 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6803", 2024-04-24T23:05:54.591 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:05:54.591 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.591 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.591 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:05:54.591 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:54.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.591 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:54.591 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6806", 2024-04-24T23:05:54.591 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:05:54.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.591 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:54.592 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6807", 2024-04-24T23:05:54.592 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:05:54.592 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.592 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.592 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:05:54.592 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:54.592 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.592 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:54.592 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6804", 2024-04-24T23:05:54.592 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:05:54.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.592 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.592 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:54.592 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6805", 2024-04-24T23:05:54.593 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:05:54.593 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.593 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.593 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6801/4010553979", 2024-04-24T23:05:54.593 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6803/4010553979", 2024-04-24T23:05:54.593 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6807/4010553979", 2024-04-24T23:05:54.593 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6805/4010553979", 2024-04-24T23:05:54.593 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:05:54.593 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:05:54.593 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:05:54.593 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.594 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:05:54.594 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:05:54.594 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:05:54.594 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:05:54.594 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:05:54.594 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:05:54.594 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:05:54.594 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:05:54.594 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:05:54.594 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 138, 2024-04-24T23:05:54.594 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:05:54.594 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:05:54.594 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:05:54.595 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:54.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.595 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:54.595 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6808", 2024-04-24T23:05:54.595 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:05:54.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.595 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:54.595 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6809", 2024-04-24T23:05:54.595 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:05:54.595 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.595 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.595 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:05:54.595 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:54.596 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.596 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:54.596 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6810", 2024-04-24T23:05:54.596 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:05:54.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.596 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.596 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:54.596 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6811", 2024-04-24T23:05:54.596 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:05:54.596 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.596 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.596 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:05:54.596 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:54.596 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.597 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:54.597 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6814", 2024-04-24T23:05:54.597 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:05:54.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.597 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:54.597 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6815", 2024-04-24T23:05:54.597 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:05:54.597 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.597 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.597 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:05:54.597 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:54.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.597 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:54.598 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6812", 2024-04-24T23:05:54.598 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:05:54.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.598 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.598 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:54.598 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6813", 2024-04-24T23:05:54.598 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:05:54.598 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.598 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.598 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6809/3794084091", 2024-04-24T23:05:54.598 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6811/3794084091", 2024-04-24T23:05:54.598 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6815/3794084091", 2024-04-24T23:05:54.598 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6813/3794084091", 2024-04-24T23:05:54.599 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:05:54.599 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:05:54.599 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:05:54.599 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.599 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.599 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.599 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:05:54.599 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:05:54.599 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:05:54.599 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:05:54.599 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:05:54.599 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:05:54.599 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:05:54.599 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:05:54.599 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:05:54.600 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 138, 2024-04-24T23:05:54.600 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:05:54.600 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:05:54.600 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:05:54.600 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:54.600 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.600 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:54.600 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6823", 2024-04-24T23:05:54.600 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:05:54.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.600 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.600 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:54.600 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6825", 2024-04-24T23:05:54.600 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:05:54.600 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.601 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.601 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.601 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:05:54.601 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:54.601 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.601 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:54.601 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6826", 2024-04-24T23:05:54.601 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:05:54.601 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.601 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.601 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:54.601 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6827", 2024-04-24T23:05:54.601 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:05:54.601 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.601 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.602 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:05:54.602 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:54.602 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.602 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:54.602 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6830", 2024-04-24T23:05:54.602 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:05:54.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.602 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.602 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:54.602 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6831", 2024-04-24T23:05:54.602 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:05:54.602 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.602 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.603 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:05:54.603 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:54.603 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.603 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:54.603 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6828", 2024-04-24T23:05:54.603 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:05:54.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.603 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.603 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:54.603 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6829", 2024-04-24T23:05:54.603 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:05:54.603 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.603 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.604 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.604 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6825/1348640258", 2024-04-24T23:05:54.604 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6827/1348640258", 2024-04-24T23:05:54.604 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6831/1348640258", 2024-04-24T23:05:54.604 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6829/1348640258", 2024-04-24T23:05:54.604 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:05:54.604 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:05:54.604 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:05:54.604 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.604 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.604 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.604 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:05:54.604 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:05:54.604 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:05:54.605 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:05:54.605 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:05:54.605 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:05:54.605 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:05:54.605 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:05:54.605 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:05:54.605 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 138, 2024-04-24T23:05:54.605 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:05:54.605 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:05:54.605 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:05:54.605 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:54.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.605 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:54.605 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6816", 2024-04-24T23:05:54.606 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:05:54.606 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.606 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.606 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:54.606 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6817", 2024-04-24T23:05:54.606 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:05:54.606 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.606 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.606 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.606 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:05:54.606 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:54.606 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.606 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:54.606 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6818", 2024-04-24T23:05:54.607 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:05:54.607 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.607 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.607 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:54.607 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6819", 2024-04-24T23:05:54.607 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:05:54.607 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.607 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.607 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.607 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:05:54.607 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:54.607 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.607 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:54.607 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6822", 2024-04-24T23:05:54.608 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:05:54.608 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.608 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.608 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:54.608 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6824", 2024-04-24T23:05:54.608 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:05:54.608 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.608 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.608 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.608 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:05:54.608 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:54.608 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.608 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:54.608 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6820", 2024-04-24T23:05:54.608 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:05:54.609 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.609 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.609 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:54.609 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6821", 2024-04-24T23:05:54.609 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:05:54.609 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.609 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.609 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.609 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6817/3250005122", 2024-04-24T23:05:54.609 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6819/3250005122", 2024-04-24T23:05:54.609 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6824/3250005122", 2024-04-24T23:05:54.609 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6821/3250005122", 2024-04-24T23:05:54.609 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:05:54.609 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:05:54.610 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:05:54.610 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.610 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.610 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:05:54.610 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2024-04-24T23:05:54.610 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.610 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:05:54.610 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:05:54.610 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:05:54.610 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:05:54.610 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:05:54.610 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:05:54.610 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.156184+0000", 2024-04-24T23:05:54.611 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:05:54.611 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.611 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.611 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:05:54.611 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:05:54.611 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:05:54.611 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:05:54.611 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:05:54.611 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:05:54.611 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240774+0000", 2024-04-24T23:05:54.611 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:05:54.611 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.611 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.611 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:05:54.611 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:05:54.612 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:05:54.612 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:05:54.612 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:05:54.612 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:05:54.612 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240949+0000", 2024-04-24T23:05:54.612 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:05:54.612 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.612 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.612 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:05:54.612 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:05:54.612 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:05:54.612 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:05:54.612 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:05:54.612 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:05:54.612 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.171672+0000", 2024-04-24T23:05:54.613 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:05:54.613 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.613 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.613 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:05:54.613 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:05:54.613 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:05:54.613 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:05:54.613 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:05:54.613 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:05:54.613 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.220379+0000", 2024-04-24T23:05:54.613 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:05:54.613 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.613 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.613 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:05:54.613 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:05:54.614 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:05:54.614 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:05:54.614 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:05:54.614 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:05:54.614 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.204963+0000", 2024-04-24T23:05:54.614 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:05:54.614 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.614 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.614 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:05:54.614 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:05:54.614 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:05:54.614 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:05:54.614 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:05:54.614 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:05:54.615 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.239748+0000", 2024-04-24T23:05:54.615 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:05:54.615 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.615 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.615 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:05:54.615 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:05:54.615 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:05:54.615 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:05:54.615 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:05:54.615 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:05:54.615 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.154104+0000", 2024-04-24T23:05:54.615 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:05:54.615 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.615 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:05:54.615 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2024-04-24T23:05:54.616 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [ 2024-04-24T23:05:54.616 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.616 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "16.17", 2024-04-24T23:05:54.616 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:05:54.616 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.616 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 4, 2024-04-24T23:05:54.616 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 3 2024-04-24T23:05:54.616 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.616 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.616 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.616 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "17.8", 2024-04-24T23:05:54.616 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:05:54.616 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.616 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 6, 2024-04-24T23:05:54.617 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 5 2024-04-24T23:05:54.617 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.617 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.617 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.617 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "17.3f", 2024-04-24T23:05:54.617 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:05:54.617 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:54.617 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 6, 2024-04-24T23:05:54.617 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 1 2024-04-24T23:05:54.617 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.617 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:54.617 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.617 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:05:54.617 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_primaries": [], 2024-04-24T23:05:54.618 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2024-04-24T23:05:54.618 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2024-04-24T23:05:54.618 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklist": {}, 2024-04-24T23:05:54.618 INFO:teuthology.orchestra.run.smithi032.stdout: "range_blocklist": {}, 2024-04-24T23:05:54.618 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2024-04-24T23:05:54.618 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2024-04-24T23:05:54.618 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2024-04-24T23:05:54.618 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2024-04-24T23:05:54.618 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2024-04-24T23:05:54.618 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2024-04-24T23:05:54.618 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2024-04-24T23:05:54.618 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.618 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:54.619 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2024-04-24T23:05:54.619 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2024-04-24T23:05:54.619 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2024-04-24T23:05:54.619 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2024-04-24T23:05:54.619 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {}, 2024-04-24T23:05:54.619 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode": { 2024-04-24T23:05:54.619 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_enabled": false, 2024-04-24T23:05:54.619 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_bucket_count": 0, 2024-04-24T23:05:54.619 INFO:teuthology.orchestra.run.smithi032.stdout: "degraded_stretch_mode": 0, 2024-04-24T23:05:54.619 INFO:teuthology.orchestra.run.smithi032.stdout: "recovering_stretch_mode": 0, 2024-04-24T23:05:54.619 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_bucket": 0 2024-04-24T23:05:54.619 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:54.619 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:05:54.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.546+0000 7f81c4efc640 1 -- 172.21.15.32:0/110135425 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f81a006be30 msgr2=0x7f81a006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:54.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.546+0000 7f81c4efc640 1 --2- 172.21.15.32:0/110135425 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f81a006be30 0x7f81a006e2f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f81a8034000 tx=0x7f81a802e000 comp rx=0 tx=0).stop 2024-04-24T23:05:54.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.547+0000 7f81c4efc640 1 -- 172.21.15.32:0/110135425 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81c0071770 msgr2=0x7f81c0157af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:54.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.547+0000 7f81c4efc640 1 --2- 172.21.15.32:0/110135425 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81c0071770 0x7f81c0157af0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f81b000c7f0 tx=0x7f81b000ccb0 comp rx=0 tx=0).stop 2024-04-24T23:05:54.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.547+0000 7f81c4efc640 1 -- 172.21.15.32:0/110135425 shutdown_connections 2024-04-24T23:05:54.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.547+0000 7f81c4efc640 1 --2- 172.21.15.32:0/110135425 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f81a006be30 0x7f81a006e2f0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:54.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.547+0000 7f81c4efc640 1 --2- 172.21.15.32:0/110135425 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f81c015c5b0 0x7f81c00ce4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:54.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.547+0000 7f81c4efc640 1 --2- 172.21.15.32:0/110135425 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81c0072cf0 0x7f81c015cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:54.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.547+0000 7f81c4efc640 1 --2- 172.21.15.32:0/110135425 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81c0071770 0x7f81c0157af0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:54.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.547+0000 7f81c4efc640 1 -- 172.21.15.32:0/110135425 >> 172.21.15.32:0/110135425 conn(0x7f81c006b110 msgr2=0x7f81c00b1840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:54.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.547+0000 7f81c4efc640 1 -- 172.21.15.32:0/110135425 shutdown_connections 2024-04-24T23:05:54.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.547+0000 7f81c4efc640 1 -- 172.21.15.32:0/110135425 wait complete. 2024-04-24T23:05:54.622 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:05:54.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.722+0000 7f02c1410640 1 Processor -- start 2024-04-24T23:05:54.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.722+0000 7f02c1410640 1 -- start start 2024-04-24T23:05:54.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.723+0000 7f02c1410640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02bc162320 0x7f02bc0ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:54.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.723+0000 7f02c1410640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02bc0cd270 con 0x7f02bc0b5af0 2024-04-24T23:05:54.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.723+0000 7f02c1410640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02bc071610 con 0x7f02bc162e10 2024-04-24T23:05:54.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.723+0000 7f02c1410640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f02bc0b5660 con 0x7f02bc162320 2024-04-24T23:05:54.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.723+0000 7f02ba7fc640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f02bc0b5af0 0x7f02bc162700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59992/0 (socket says 172.21.15.32:59992) 2024-04-24T23:05:54.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.723+0000 7f02ba7fc640 1 -- 172.21.15.32:0/1469390506 learned_addr learned my addr 172.21.15.32:0/1469390506 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:05:54.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.723+0000 7f02baffd640 1 --2- 172.21.15.32:0/1469390506 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02bc162320 0x7f02bc0ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:54.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.723+0000 7f02b9ffb640 1 -- 172.21.15.32:0/1469390506 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4086290026 0 0) 0x7f02bc0cd270 con 0x7f02bc0b5af0 2024-04-24T23:05:54.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.723+0000 7f02b9ffb640 1 -- 172.21.15.32:0/1469390506 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f02a4003520 con 0x7f02bc0b5af0 2024-04-24T23:05:54.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.723+0000 7f02b9ffb640 1 -- 172.21.15.32:0/1469390506 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3596860704 0 0) 0x7f02a4003520 con 0x7f02bc0b5af0 2024-04-24T23:05:54.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.723+0000 7f02baffd640 1 -- 172.21.15.32:0/1469390506 >> v1:172.21.15.121:6789/0 conn(0x7f02bc162e10 legacy=0x7f02bc15ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:54.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.723+0000 7f02baffd640 1 -- 172.21.15.32:0/1469390506 >> v1:172.21.15.32:6789/0 conn(0x7f02bc0b5af0 legacy=0x7f02bc162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:54.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.723+0000 7f02baffd640 1 -- 172.21.15.32:0/1469390506 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02bc0718f0 con 0x7f02bc162320 2024-04-24T23:05:54.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.724+0000 7f02baffd640 1 --2- 172.21.15.32:0/1469390506 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02bc162320 0x7f02bc0ccc90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f02ac009dc0 tx=0x7f02ac02a500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4266e3edd893341 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:54.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.724+0000 7f02b9ffb640 1 -- 172.21.15.32:0/1469390506 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02ac035020 con 0x7f02bc162320 2024-04-24T23:05:54.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.724+0000 7f02b9ffb640 1 -- 172.21.15.32:0/1469390506 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02ac007540 con 0x7f02bc162320 2024-04-24T23:05:54.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.724+0000 7f02b9ffb640 1 -- 172.21.15.32:0/1469390506 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b0001670 con 0x7f02bc162320 2024-04-24T23:05:54.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.725+0000 7f02c1410640 1 -- 172.21.15.32:0/1469390506 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02bc162320 msgr2=0x7f02bc0ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:54.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.725+0000 7f02c1410640 1 --2- 172.21.15.32:0/1469390506 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02bc162320 0x7f02bc0ccc90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f02ac009dc0 tx=0x7f02ac02a500 comp rx=0 tx=0).stop 2024-04-24T23:05:54.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.725+0000 7f02c1410640 1 -- 172.21.15.32:0/1469390506 shutdown_connections 2024-04-24T23:05:54.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.725+0000 7f02c1410640 1 --2- 172.21.15.32:0/1469390506 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02bc162320 0x7f02bc0ccc90 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:54.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.725+0000 7f02c1410640 1 -- 172.21.15.32:0/1469390506 >> 172.21.15.32:0/1469390506 conn(0x7f02bc0d0ae0 msgr2=0x7f02bc0d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:54.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.725+0000 7f02c1410640 1 -- 172.21.15.32:0/1469390506 shutdown_connections 2024-04-24T23:05:54.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.725+0000 7f02c1410640 1 -- 172.21.15.32:0/1469390506 wait complete. 2024-04-24T23:05:54.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.726+0000 7f02c1410640 1 Processor -- start 2024-04-24T23:05:54.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.726+0000 7f02c1410640 1 -- start start 2024-04-24T23:05:54.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.727+0000 7f02c1410640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02bc0b5af0 0x7f02bc06f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:54.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.727+0000 7f02c1410640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02bc162320 0x7f02bc14d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:54.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.727+0000 7f02c1410640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02bc162e10 0x7f02bc1f5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:54.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.727+0000 7f02c1410640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f02bc071bd0 con 0x7f02bc0b5af0 2024-04-24T23:05:54.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.727+0000 7f02c1410640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f02bc067950 con 0x7f02bc162e10 2024-04-24T23:05:54.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.727+0000 7f02c1410640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f02bc071d50 con 0x7f02bc162320 2024-04-24T23:05:54.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.727+0000 7f02baffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02bc0b5af0 0x7f02bc06f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:54.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.727+0000 7f02baffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02bc0b5af0 0x7f02bc06f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38354/0 (socket says 172.21.15.32:38354) 2024-04-24T23:05:54.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.727+0000 7f02baffd640 1 -- 172.21.15.32:0/1967105779 learned_addr learned my addr 172.21.15.32:0/1967105779 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:54.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.727+0000 7f02baffd640 1 -- 172.21.15.32:0/1967105779 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02bc162320 msgr2=0x7f02bc14d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:54.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.727+0000 7f02baffd640 1 --2- 172.21.15.32:0/1967105779 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02bc162320 0x7f02bc14d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:54.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.727+0000 7f02baffd640 1 -- 172.21.15.32:0/1967105779 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02bc162e10 msgr2=0x7f02bc1f5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:54.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.727+0000 7f02baffd640 1 --2- 172.21.15.32:0/1967105779 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02bc162e10 0x7f02bc1f5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:54.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.727+0000 7f02baffd640 1 -- 172.21.15.32:0/1967105779 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02bc14dc30 con 0x7f02bc0b5af0 2024-04-24T23:05:54.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.728+0000 7f02baffd640 1 --2- 172.21.15.32:0/1967105779 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02bc0b5af0 0x7f02bc06f250 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f02ac009ee0 tx=0x7f02ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:54.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.728+0000 7f02a37fe640 1 -- 172.21.15.32:0/1967105779 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02ac03f070 con 0x7f02bc0b5af0 2024-04-24T23:05:54.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.728+0000 7f02a37fe640 1 -- 172.21.15.32:0/1967105779 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02ac031c60 con 0x7f02bc0b5af0 2024-04-24T23:05:54.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.728+0000 7f02a37fe640 1 -- 172.21.15.32:0/1967105779 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02ac035070 con 0x7f02bc0b5af0 2024-04-24T23:05:54.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.728+0000 7f02c1410640 1 -- 172.21.15.32:0/1967105779 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02bc0c5ca0 con 0x7f02bc0b5af0 2024-04-24T23:05:54.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.728+0000 7f02c1410640 1 -- 172.21.15.32:0/1967105779 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02bc06e1b0 con 0x7f02bc0b5af0 2024-04-24T23:05:54.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.729+0000 7f02c1410640 1 -- 172.21.15.32:0/1967105779 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02bc068430 con 0x7f02bc0b5af0 2024-04-24T23:05:54.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.730+0000 7f02a37fe640 1 -- 172.21.15.32:0/1967105779 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f02ac042070 con 0x7f02bc0b5af0 2024-04-24T23:05:54.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.730+0000 7f02a37fe640 1 --2- 172.21.15.32:0/1967105779 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f02a806be60 0x7f02a806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:54.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.730+0000 7f02a37fe640 1 -- 172.21.15.32:0/1967105779 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5796+0+0 (secure 0 0 0) 0x7f02ac0abee0 con 0x7f02bc0b5af0 2024-04-24T23:05:54.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.730+0000 7f02a37fe640 1 -- 172.21.15.32:0/1967105779 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=145}) -- 0x7f02a8073b10 con 0x7f02bc0b5af0 2024-04-24T23:05:54.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.730+0000 7f02ba7fc640 1 --2- 172.21.15.32:0/1967105779 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f02a806be60 0x7f02a806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:54.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.731+0000 7f02ba7fc640 1 --2- 172.21.15.32:0/1967105779 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f02a806be60 0x7f02a806e320 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f02b00011e0 tx=0x7f02b0001a40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:54.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.734+0000 7f02a37fe640 1 -- 172.21.15.32:0/1967105779 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f02bc14dc30 con 0x7f02bc0b5af0 2024-04-24T23:05:54.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.956+0000 7f02c1410640 1 -- 172.21.15.32:0/1967105779 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f02bc0d0850 con 0x7f02bc0b5af0 2024-04-24T23:05:54.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.957+0000 7f02a37fe640 1 -- 172.21.15.32:0/1967105779 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+5189 (secure 0 0 0) 0x7f02ac07a830 con 0x7f02bc0b5af0 2024-04-24T23:05:54.959 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:05:54.959 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":58,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3361752496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3361752496},{"type":"v1","addr":"172.21.15.121:6837","nonce":3361752496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":6799,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3813453544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3813453544},{"type":"v1","addr":"172.21.15.121:6836","nonce":3813453544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7779,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/1352085651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1352085651},{"type":"v1","addr":"172.21.15.32:6833","nonce":1352085651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:05:33.791068+0000","modified":"2024-04-24T23:05:35.792946+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7794},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7794":{"gid":7794,"name":"a","rank":0,"incarnation":56,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/4206823204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":4206823204},{"type":"v1","addr":"172.21.15.32:6835","nonce":4206823204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7794,"qdb_cluster":[7794]},"id":8}]} 2024-04-24T23:05:54.959 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 58 2024-04-24T23:05:54.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.959+0000 7f02c1410640 1 -- 172.21.15.32:0/1967105779 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f02a806be60 msgr2=0x7f02a806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:54.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.959+0000 7f02c1410640 1 --2- 172.21.15.32:0/1967105779 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f02a806be60 0x7f02a806e320 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f02b00011e0 tx=0x7f02b0001a40 comp rx=0 tx=0).stop 2024-04-24T23:05:54.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.959+0000 7f02c1410640 1 -- 172.21.15.32:0/1967105779 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02bc0b5af0 msgr2=0x7f02bc06f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:54.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.959+0000 7f02c1410640 1 --2- 172.21.15.32:0/1967105779 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02bc0b5af0 0x7f02bc06f250 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f02ac009ee0 tx=0x7f02ac004a40 comp rx=0 tx=0).stop 2024-04-24T23:05:54.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.959+0000 7f02c1410640 1 -- 172.21.15.32:0/1967105779 shutdown_connections 2024-04-24T23:05:54.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.959+0000 7f02c1410640 1 --2- 172.21.15.32:0/1967105779 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f02a806be60 0x7f02a806e320 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:54.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.959+0000 7f02c1410640 1 --2- 172.21.15.32:0/1967105779 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02bc162e10 0x7f02bc1f5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:54.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.959+0000 7f02c1410640 1 --2- 172.21.15.32:0/1967105779 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02bc162320 0x7f02bc14d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:54.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.959+0000 7f02c1410640 1 --2- 172.21.15.32:0/1967105779 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02bc0b5af0 0x7f02bc06f250 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:54.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.959+0000 7f02c1410640 1 -- 172.21.15.32:0/1967105779 >> 172.21.15.32:0/1967105779 conn(0x7f02bc0d0ae0 msgr2=0x7f02bc15b530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:54.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.959+0000 7f02c1410640 1 -- 172.21.15.32:0/1967105779 shutdown_connections 2024-04-24T23:05:54.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:54.959+0000 7f02c1410640 1 -- 172.21.15.32:0/1967105779 wait complete. 2024-04-24T23:05:54.971 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-04-24T23:05:55.091 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","whoami":0,"id":7794,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":20.662206907000002,"mdsmap_epoch":58,"osdmap_epoch":144,"osdmap_epoch_barrier":144,"uptime":25.733629327999999} 2024-04-24T23:05:55.091 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", "fs_name": "cephfs", "id": 7794, "mdsmap_epoch": 58, "osdmap_epoch": 144, "osdmap_epoch_barrier": 144, "rank_uptime": 20.662206907, "state": "up:active", "uptime": 25.733629328, "want_state": "up:active", "whoami": 0 } 2024-04-24T23:05:55.091 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-24T23:05:55.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:05:55.091 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2024-04-24T23:05:55.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:05:55.108 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2024-04-24T23:05:55.167 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-24T23:05:55.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.306+0000 7ff3cc314640 1 Processor -- start 2024-04-24T23:05:55.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.307+0000 7ff3cc314640 1 -- start start 2024-04-24T23:05:55.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.307+0000 7ff3cc314640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff3c406b820 0x7ff3c40d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:55.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.307+0000 7ff3cc314640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff3c40d4210 0x7ff3c40b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:55.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.307+0000 7ff3cc314640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3c40d3170 con 0x7ff3c415e9b0 2024-04-24T23:05:55.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.307+0000 7ff3cc314640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7ff3c40d3450 con 0x7ff3c40d4210 2024-04-24T23:05:55.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.307+0000 7ff3cc314640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7ff3c40d3730 con 0x7ff3c406b820 2024-04-24T23:05:55.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.308+0000 7ff3ca8b1640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff3c40d4210 0x7ff3c40b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:55.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.308+0000 7ff3ca8b1640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff3c40d4210 0x7ff3c40b78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38360/0 (socket says 172.21.15.32:38360) 2024-04-24T23:05:55.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.308+0000 7ff3ca8b1640 1 -- 172.21.15.32:0/4115398211 learned_addr learned my addr 172.21.15.32:0/4115398211 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:55.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.308+0000 7ff3c98af640 1 --2- 172.21.15.32:0/4115398211 >> v2:172.21.15.121:3300/0 conn(0x7ff3c406b820 0x7ff3c40d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:55.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.308+0000 7ff3ca8b1640 1 -- 172.21.15.32:0/4115398211 >> v2:172.21.15.121:3300/0 conn(0x7ff3c406b820 msgr2=0x7ff3c40d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:05:55.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.308+0000 7ff3ca8b1640 1 --2- 172.21.15.32:0/4115398211 >> v2:172.21.15.121:3300/0 conn(0x7ff3c406b820 0x7ff3c40d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:55.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.308+0000 7ff3ca8b1640 1 -- 172.21.15.32:0/4115398211 >> v1:172.21.15.121:6789/0 conn(0x7ff3c415e9b0 legacy=0x7ff3c406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:55.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.308+0000 7ff3ca8b1640 1 -- 172.21.15.32:0/4115398211 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3c40d3a10 con 0x7ff3c40d4210 2024-04-24T23:05:55.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.308+0000 7ff3c98af640 1 --2- 172.21.15.32:0/4115398211 >> v2:172.21.15.121:3300/0 conn(0x7ff3c406b820 0x7ff3c40d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:05:55.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.309+0000 7ff3ca8b1640 1 --2- 172.21.15.32:0/4115398211 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3c40d4210 0x7ff3c40b78f0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7ff3c40730f0 tx=0x7ff3bc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e19a5a4f32e0348 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:55.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.309+0000 7ff3c90ae640 1 -- 172.21.15.32:0/4115398211 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc036070 con 0x7ff3c40d4210 2024-04-24T23:05:55.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.309+0000 7ff3c90ae640 1 -- 172.21.15.32:0/4115398211 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3bc031600 con 0x7ff3c40d4210 2024-04-24T23:05:55.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.309+0000 7ff3c90ae640 1 -- 172.21.15.32:0/4115398211 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc0319d0 con 0x7ff3c40d4210 2024-04-24T23:05:55.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.309+0000 7ff3cc314640 1 -- 172.21.15.32:0/4115398211 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3c40d4210 msgr2=0x7ff3c40b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:55.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.309+0000 7ff3cc314640 1 --2- 172.21.15.32:0/4115398211 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3c40d4210 0x7ff3c40b78f0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7ff3c40730f0 tx=0x7ff3bc02a6a0 comp rx=0 tx=0).stop 2024-04-24T23:05:55.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.309+0000 7ff3cc314640 1 -- 172.21.15.32:0/4115398211 shutdown_connections 2024-04-24T23:05:55.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.309+0000 7ff3cc314640 1 --2- 172.21.15.32:0/4115398211 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3c40d4210 0x7ff3c40b78f0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:55.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.309+0000 7ff3cc314640 1 --2- 172.21.15.32:0/4115398211 >> v2:172.21.15.121:3300/0 conn(0x7ff3c406b820 0x7ff3c40d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:55.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.309+0000 7ff3cc314640 1 -- 172.21.15.32:0/4115398211 >> 172.21.15.32:0/4115398211 conn(0x7ff3c40d11d0 msgr2=0x7ff3c40d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:55.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.310+0000 7ff3cc314640 1 -- 172.21.15.32:0/4115398211 shutdown_connections 2024-04-24T23:05:55.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.310+0000 7ff3cc314640 1 -- 172.21.15.32:0/4115398211 wait complete. 2024-04-24T23:05:55.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.311+0000 7ff3cc314640 1 Processor -- start 2024-04-24T23:05:55.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.311+0000 7ff3cc314640 1 -- start start 2024-04-24T23:05:55.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.311+0000 7ff3cc314640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff3c406b820 0x7ff3c40ce5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:55.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.311+0000 7ff3cc314640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3c40d4210 0x7ff3c41e3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:55.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.311+0000 7ff3cc314640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3c415e9b0 0x7ff3c40ca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:55.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.311+0000 7ff3cc314640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c40ba0e0 con 0x7ff3c415e9b0 2024-04-24T23:05:55.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.311+0000 7ff3cc314640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c40d2400 con 0x7ff3c40d4210 2024-04-24T23:05:55.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.311+0000 7ff3cc314640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff3c40ba260 con 0x7ff3c406b820 2024-04-24T23:05:55.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.312+0000 7ff3ca8b1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3c415e9b0 0x7ff3c40ca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:55.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.312+0000 7ff3ca8b1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3c415e9b0 0x7ff3c40ca9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38372/0 (socket says 172.21.15.32:38372) 2024-04-24T23:05:55.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.312+0000 7ff3ca8b1640 1 -- 172.21.15.32:0/1443769785 learned_addr learned my addr 172.21.15.32:0/1443769785 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:05:55.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.312+0000 7ff3c98af640 1 --2- 172.21.15.32:0/1443769785 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3c40d4210 0x7ff3c41e3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:55.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.312+0000 7ff3ca0b0640 1 --2- 172.21.15.32:0/1443769785 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff3c406b820 0x7ff3c40ce5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:55.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.312+0000 7ff3ca8b1640 1 -- 172.21.15.32:0/1443769785 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff3c406b820 msgr2=0x7ff3c40ce5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:55.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.312+0000 7ff3ca8b1640 1 --2- 172.21.15.32:0/1443769785 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff3c406b820 0x7ff3c40ce5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:55.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.312+0000 7ff3ca8b1640 1 -- 172.21.15.32:0/1443769785 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3c40d4210 msgr2=0x7ff3c41e3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:55.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.312+0000 7ff3ca8b1640 1 --2- 172.21.15.32:0/1443769785 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3c40d4210 0x7ff3c41e3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:55.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.312+0000 7ff3ca8b1640 1 -- 172.21.15.32:0/1443769785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3c40cd550 con 0x7ff3c415e9b0 2024-04-24T23:05:55.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.312+0000 7ff3ca0b0640 1 --2- 172.21.15.32:0/1443769785 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff3c406b820 0x7ff3c40ce5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:05:55.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.312+0000 7ff3ca8b1640 1 --2- 172.21.15.32:0/1443769785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3c415e9b0 0x7ff3c40ca9f0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc009f40 tx=0x7ff3bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:55.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.312+0000 7ff3baffd640 1 -- 172.21.15.32:0/1443769785 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc036050 con 0x7ff3c415e9b0 2024-04-24T23:05:55.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.312+0000 7ff3baffd640 1 -- 172.21.15.32:0/1443769785 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3bc042070 con 0x7ff3c415e9b0 2024-04-24T23:05:55.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.312+0000 7ff3cc314640 1 -- 172.21.15.32:0/1443769785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3c40cd830 con 0x7ff3c415e9b0 2024-04-24T23:05:55.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.312+0000 7ff3baffd640 1 -- 172.21.15.32:0/1443769785 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc009510 con 0x7ff3c415e9b0 2024-04-24T23:05:55.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.312+0000 7ff3cc314640 1 -- 172.21.15.32:0/1443769785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3c40cddf0 con 0x7ff3c415e9b0 2024-04-24T23:05:55.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.313+0000 7ff3cc314640 1 -- 172.21.15.32:0/1443769785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3c40d26a0 con 0x7ff3c415e9b0 2024-04-24T23:05:55.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.314+0000 7ff3baffd640 1 -- 172.21.15.32:0/1443769785 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff3bc007c50 con 0x7ff3c415e9b0 2024-04-24T23:05:55.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.314+0000 7ff3baffd640 1 --2- 172.21.15.32:0/1443769785 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff3a806be30 0x7ff3a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:55.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.314+0000 7ff3baffd640 1 -- 172.21.15.32:0/1443769785 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5796+0+0 (secure 0 0 0) 0x7ff3bc0abdf0 con 0x7ff3c415e9b0 2024-04-24T23:05:55.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.314+0000 7ff3baffd640 1 -- 172.21.15.32:0/1443769785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=145}) -- 0x7ff3a8073a80 con 0x7ff3c415e9b0 2024-04-24T23:05:55.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.314+0000 7ff3ca0b0640 1 --2- 172.21.15.32:0/1443769785 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff3a806be30 0x7ff3a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:55.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.315+0000 7ff3ca0b0640 1 --2- 172.21.15.32:0/1443769785 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff3a806be30 0x7ff3a806e2f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7ff3c00015e0 tx=0x7ff3c0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:55.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.319+0000 7ff3baffd640 1 -- 172.21.15.32:0/1443769785 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff3c40d26a0 con 0x7ff3c415e9b0 2024-04-24T23:05:55.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.507+0000 7ff3cc314640 1 -- 172.21.15.32:0/1443769785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7ff3c4068430 con 0x7ff3c415e9b0 2024-04-24T23:05:55.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.508+0000 7ff3baffd640 1 -- 172.21.15.32:0/1443769785 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v144) ==== 81+0+34760 (secure 0 0 0) 0x7ff3bc07a740 con 0x7ff3c415e9b0 2024-04-24T23:05:55.510 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:05:55.510 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:05:55.510 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 144, 2024-04-24T23:05:55.510 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", 2024-04-24T23:05:55.510 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2024-04-24T22:58:46.964851+0000", 2024-04-24T23:05:55.510 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2024-04-24T23:05:53.811291+0000", 2024-04-24T23:05:55.510 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2024-04-24T22:58:52.048787+0000", 2024-04-24T23:05:55.510 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2024-04-24T22:58:49.045609+0000", 2024-04-24T23:05:55.510 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-24T23:05:55.511 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2024-04-24T23:05:55.511 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2024-04-24T23:05:55.511 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2024-04-24T23:05:55.511 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2024-04-24T23:05:55.511 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2024-04-24T23:05:55.511 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2024-04-24T23:05:55.511 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:05:55.511 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 4, 2024-04-24T23:05:55.511 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.69999998807907104, 2024-04-24T23:05:55.511 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-24T23:05:55.511 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-24T23:05:55.511 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2024-04-24T23:05:55.512 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 17, 2024-04-24T23:05:55.512 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2024-04-24T23:05:55.512 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "luminous", 2024-04-24T23:05:55.512 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "luminous", 2024-04-24T23:05:55.512 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "squid", 2024-04-24T23:05:55.512 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_crimson": false, 2024-04-24T23:05:55.512 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2024-04-24T23:05:55.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.512 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2024-04-24T23:05:55.512 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": ".mgr", 2024-04-24T23:05:55.512 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T22:58:53.171408+0000", 2024-04-24T23:05:55.512 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:05:55.512 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:05:55.512 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:05:55.512 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:05:55.513 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:05:55.513 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:05:55.513 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:05:55.513 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:05:55.513 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:05:55.513 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:05:55.513 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:05:55.513 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:05:55.513 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2024-04-24T23:05:55.513 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2024-04-24T23:05:55.513 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2024-04-24T23:05:55.513 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2024-04-24T23:05:55.513 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2024-04-24T23:05:55.513 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:05:55.513 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:05:55.514 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:05:55.514 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:05:55.514 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:05:55.514 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:05:55.514 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:05:55.514 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.514 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "113", 2024-04-24T23:05:55.514 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:05:55.514 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:05:55.514 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:05:55.514 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:05:55.514 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:05:55.514 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:05:55.515 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:05:55.515 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:05:55.515 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:05:55.515 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:05:55.515 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:05:55.515 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:05:55.515 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:05:55.515 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:05:55.515 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:05:55.515 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:05:55.516 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:05:55.516 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:05:55.516 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:05:55.516 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:05:55.516 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:05:55.516 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:05:55.516 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:05:55.516 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:05:55.516 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:05:55.516 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:05:55.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.516 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:05:55.516 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:05:55.516 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:05:55.517 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:05:55.517 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:05:55.517 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:05:55.517 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:05:55.517 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:05:55.517 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:05:55.517 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:05:55.517 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:05:55.517 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:05:55.517 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_max": 32, 2024-04-24T23:05:55.518 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2024-04-24T23:05:55.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.518 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:05:55.518 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": {} 2024-04-24T23:05:55.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.518 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:05:55.518 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:05:55.518 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 8, 2024-04-24T23:05:55.518 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 8, 2024-04-24T23:05:55.518 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.25, 2024-04-24T23:05:55.518 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-04-24T23:05:55.518 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-04-24T23:05:55.518 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:05:55.519 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:05:55.519 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:05:55.519 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:55.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.519 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.519 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 16, 2024-04-24T23:05:55.519 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2024-04-24T23:05:55.519 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:05:32.064234+0000", 2024-04-24T23:05:55.519 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:05:55.519 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:05:55.519 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:05:55.519 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:05:55.519 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:05:55.520 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:05:55.520 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:05:55.520 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:05:55.520 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:05:55.520 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:05:55.520 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:05:55.520 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:05:55.520 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 32, 2024-04-24T23:05:55.520 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 32, 2024-04-24T23:05:55.520 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 32, 2024-04-24T23:05:55.520 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 32, 2024-04-24T23:05:55.520 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 32, 2024-04-24T23:05:55.520 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:05:55.521 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:05:55.521 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:05:55.521 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:05:55.521 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:05:55.521 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:05:55.521 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:05:55.521 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.521 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "139", 2024-04-24T23:05:55.521 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:05:55.521 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:05:55.521 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:05:55.521 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:05:55.521 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:05:55.521 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:05:55.521 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:05:55.522 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:05:55.522 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:05:55.522 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:05:55.522 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:05:55.522 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:05:55.522 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:05:55.522 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:05:55.522 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:05:55.522 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:05:55.522 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:05:55.522 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:05:55.522 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:05:55.522 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:05:55.522 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:05:55.523 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:05:55.523 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:05:55.523 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:05:55.523 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:05:55.523 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:05:55.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.523 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:05:55.523 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:05:55.523 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:05:55.523 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:05:55.523 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:05:55.523 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:05:55.523 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:05:55.523 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:05:55.524 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:05:55.524 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:05:55.524 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:05:55.524 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:05:55.524 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2024-04-24T23:05:55.524 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2024-04-24T23:05:55.524 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2024-04-24T23:05:55.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.524 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:05:55.524 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:05:55.524 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2024-04-24T23:05:55.524 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:55.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.524 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:05:55.524 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:05:55.525 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 2, 2024-04-24T23:05:55.525 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 2, 2024-04-24T23:05:55.525 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:05:55.525 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-04-24T23:05:55.525 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-04-24T23:05:55.525 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:05:55.525 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:05:55.525 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:05:55.525 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:55.526 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.526 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.526 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 17, 2024-04-24T23:05:55.526 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2024-04-24T23:05:55.526 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:05:32.760777+0000", 2024-04-24T23:05:55.526 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1027, 2024-04-24T23:05:55.526 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,full,full_quota", 2024-04-24T23:05:55.526 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:05:55.526 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:05:55.526 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:05:55.526 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:05:55.527 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:05:55.527 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:05:55.527 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:05:55.527 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:05:55.527 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:05:55.527 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:05:55.527 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 64, 2024-04-24T23:05:55.527 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 64, 2024-04-24T23:05:55.527 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 64, 2024-04-24T23:05:55.527 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 64, 2024-04-24T23:05:55.530 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 64, 2024-04-24T23:05:55.530 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:05:55.530 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:05:55.530 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:05:55.531 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:05:55.531 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:05:55.531 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:05:55.531 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:05:55.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.531 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "142", 2024-04-24T23:05:55.531 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:05:55.531 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:05:55.531 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:05:55.531 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:05:55.531 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:05:55.532 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:05:55.532 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:05:55.532 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:05:55.532 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:05:55.532 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 33554432, 2024-04-24T23:05:55.532 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:05:55.532 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:05:55.532 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:05:55.532 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:05:55.532 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:05:55.532 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:05:55.533 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:05:55.533 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:05:55.533 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:05:55.533 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:05:55.533 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:05:55.533 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:05:55.533 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:05:55.533 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:05:55.533 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:05:55.533 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:05:55.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.534 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:05:55.534 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:05:55.534 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:05:55.534 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:05:55.534 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:05:55.534 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:05:55.534 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:05:55.534 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:05:55.534 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:05:55.535 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:05:55.535 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:05:55.535 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:05:55.535 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 64, 2024-04-24T23:05:55.535 INFO:teuthology.orchestra.run.smithi032.stdout: "target_size_ratio": 0.89999997615814209 2024-04-24T23:05:55.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.535 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:05:55.535 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:05:55.535 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2024-04-24T23:05:55.535 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:55.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.535 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:05:55.536 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:05:55.536 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.3799999952316284, 2024-04-24T23:05:55.536 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.3799999952316284, 2024-04-24T23:05:55.536 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:05:55.536 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.3799999952316284, 2024-04-24T23:05:55.536 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.3799999952316284, 2024-04-24T23:05:55.536 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:05:55.536 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:05:55.536 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:05:55.537 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:55.537 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:55.537 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:05:55.537 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-04-24T23:05:55.537 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.537 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:05:55.537 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:05:55.537 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:05:55.537 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:05:55.537 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:05:55.537 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:05:55.537 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:05:55.537 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:05:55.537 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:05:55.537 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 143, 2024-04-24T23:05:55.538 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:05:55.538 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:05:55.538 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:05:55.538 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:55.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.538 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:55.538 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2024-04-24T23:05:55.538 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:05:55.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.538 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:55.538 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2024-04-24T23:05:55.539 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:05:55.539 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:55.539 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:55.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.539 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:05:55.539 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:55.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.539 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:55.539 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2024-04-24T23:05:55.539 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:05:55.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.539 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:55.539 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2024-04-24T23:05:55.539 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:05:55.540 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:55.540 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:55.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.540 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:05:55.540 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:55.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.540 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:55.541 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2024-04-24T23:05:55.541 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:05:55.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.541 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:55.541 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2024-04-24T23:05:55.541 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:05:55.541 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:55.541 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:55.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.541 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:05:55.541 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:55.542 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.542 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:55.542 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2024-04-24T23:05:55.542 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:05:55.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.542 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.542 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:55.542 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2024-04-24T23:05:55.542 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:05:55.542 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:55.542 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:55.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.543 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/1479395426", 2024-04-24T23:05:55.543 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/1479395426", 2024-04-24T23:05:55.543 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/1479395426", 2024-04-24T23:05:55.543 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/1479395426", 2024-04-24T23:05:55.543 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:05:55.543 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:05:55.543 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:05:55.543 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:55.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.544 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:05:55.544 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:05:55.544 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:05:55.544 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:05:55.544 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:05:55.544 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:05:55.545 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:05:55.545 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:05:55.545 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:05:55.545 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 143, 2024-04-24T23:05:55.545 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:05:55.545 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:05:55.546 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:05:55.546 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:55.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.546 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:55.546 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2024-04-24T23:05:55.547 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:05:55.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.547 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:55.547 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2024-04-24T23:05:55.547 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:05:55.548 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:55.548 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:55.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.548 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:05:55.548 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:55.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.549 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:55.549 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2024-04-24T23:05:55.549 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:05:55.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.550 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:55.550 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2024-04-24T23:05:55.550 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:05:55.550 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:55.550 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:55.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.551 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:05:55.551 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:55.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.551 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:55.551 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2024-04-24T23:05:55.552 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:05:55.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.552 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:55.552 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2024-04-24T23:05:55.552 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:05:55.553 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:55.553 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:55.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.553 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:05:55.553 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:55.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.554 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:55.554 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2024-04-24T23:05:55.554 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:05:55.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.555 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:55.555 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2024-04-24T23:05:55.555 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:05:55.555 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:55.555 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:55.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.555 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/1340633718", 2024-04-24T23:05:55.555 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/1340633718", 2024-04-24T23:05:55.555 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/1340633718", 2024-04-24T23:05:55.555 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/1340633718", 2024-04-24T23:05:55.555 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:05:55.555 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:05:55.555 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:05:55.556 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:55.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.556 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:05:55.556 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:05:55.556 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:05:55.556 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:05:55.556 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:05:55.556 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:05:55.556 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:05:55.556 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:05:55.556 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:05:55.557 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 138, 2024-04-24T23:05:55.557 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:05:55.557 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:05:55.557 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:05:55.557 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:55.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.557 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:55.557 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2024-04-24T23:05:55.558 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:05:55.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.558 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:55.558 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2024-04-24T23:05:55.558 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:05:55.558 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:55.558 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:55.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.558 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:05:55.558 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:55.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.559 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:55.559 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2024-04-24T23:05:55.559 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:05:55.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.559 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:55.559 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2024-04-24T23:05:55.559 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:05:55.559 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:55.559 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:55.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.559 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:05:55.559 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:55.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.559 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:55.560 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2024-04-24T23:05:55.560 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:05:55.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.560 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:55.560 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2024-04-24T23:05:55.560 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:05:55.560 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:55.560 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:55.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.560 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:05:55.561 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:55.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.561 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:55.561 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2024-04-24T23:05:55.561 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:05:55.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.561 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:55.561 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2024-04-24T23:05:55.561 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:05:55.561 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:55.561 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:55.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.562 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/3405899334", 2024-04-24T23:05:55.562 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6820/3405899334", 2024-04-24T23:05:55.562 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6828/3405899334", 2024-04-24T23:05:55.562 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6824/3405899334", 2024-04-24T23:05:55.562 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:05:55.562 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:05:55.562 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:05:55.562 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:55.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.562 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:05:55.562 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:05:55.563 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:05:55.563 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:05:55.563 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:05:55.563 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:05:55.563 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:05:55.563 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:05:55.563 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:05:55.563 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 143, 2024-04-24T23:05:55.564 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:05:55.564 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:05:55.564 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:05:55.564 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:55.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.564 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:55.564 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2024-04-24T23:05:55.564 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:05:55.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.564 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:55.564 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2024-04-24T23:05:55.564 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:05:55.564 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:55.565 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:55.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.565 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:05:55.565 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:55.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.565 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:55.565 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2024-04-24T23:05:55.565 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:05:55.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.566 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:55.566 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2024-04-24T23:05:55.566 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:05:55.566 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:55.566 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:55.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.566 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:05:55.566 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:55.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.566 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:55.566 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2024-04-24T23:05:55.566 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:05:55.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.567 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:55.567 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2024-04-24T23:05:55.567 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:05:55.567 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:55.567 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:55.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.567 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:05:55.567 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:55.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.568 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:55.568 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2024-04-24T23:05:55.568 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:05:55.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.568 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:55.568 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2024-04-24T23:05:55.568 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:05:55.568 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:55.568 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:55.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.568 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6821/3381057190", 2024-04-24T23:05:55.569 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6825/3381057190", 2024-04-24T23:05:55.569 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/3381057190", 2024-04-24T23:05:55.569 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/3381057190", 2024-04-24T23:05:55.569 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:05:55.569 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:05:55.569 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:05:55.569 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:55.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.569 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:05:55.570 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:05:55.570 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:05:55.570 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:05:55.570 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:05:55.570 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:05:55.570 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:05:55.570 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:05:55.570 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:05:55.570 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 138, 2024-04-24T23:05:55.570 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:05:55.571 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:05:55.571 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:05:55.571 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:55.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.571 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:55.571 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6800", 2024-04-24T23:05:55.571 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:05:55.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.571 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:55.571 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6801", 2024-04-24T23:05:55.572 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:05:55.572 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:55.572 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:55.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.572 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:05:55.572 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:55.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.572 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:55.572 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6802", 2024-04-24T23:05:55.572 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:05:55.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.573 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:55.573 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6803", 2024-04-24T23:05:55.573 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:05:55.573 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:55.573 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:55.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.573 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:05:55.573 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:55.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.573 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:55.573 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6806", 2024-04-24T23:05:55.574 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:05:55.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.574 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:55.574 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6807", 2024-04-24T23:05:55.574 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:05:55.574 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:55.574 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:55.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.574 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:05:55.574 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:55.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.574 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:55.575 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6804", 2024-04-24T23:05:55.575 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:05:55.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.575 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:55.575 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6805", 2024-04-24T23:05:55.575 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:05:55.575 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:55.575 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:55.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.576 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6801/4010553979", 2024-04-24T23:05:55.576 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6803/4010553979", 2024-04-24T23:05:55.576 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6807/4010553979", 2024-04-24T23:05:55.576 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6805/4010553979", 2024-04-24T23:05:55.576 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:05:55.576 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:05:55.576 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:05:55.577 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:55.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.577 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:05:55.577 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:05:55.577 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:05:55.577 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:05:55.577 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:05:55.577 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:05:55.577 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:05:55.577 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:05:55.577 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:05:55.578 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 138, 2024-04-24T23:05:55.578 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:05:55.578 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:05:55.578 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:05:55.578 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:55.578 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.578 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:55.578 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6808", 2024-04-24T23:05:55.578 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:05:55.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.579 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:55.579 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6809", 2024-04-24T23:05:55.579 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:05:55.579 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:55.579 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:55.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.580 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:05:55.580 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:55.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.580 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:55.580 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6810", 2024-04-24T23:05:55.580 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:05:55.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.580 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:55.581 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6811", 2024-04-24T23:05:55.581 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:05:55.581 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:55.581 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:55.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.581 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:05:55.581 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:55.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.581 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:55.581 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6814", 2024-04-24T23:05:55.581 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:05:55.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.582 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:55.582 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6815", 2024-04-24T23:05:55.582 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:05:55.582 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:55.582 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:55.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.582 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:05:55.582 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:55.582 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.582 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6812", 2024-04-24T23:05:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:05:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6813", 2024-04-24T23:05:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:05:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6809/3794084091", 2024-04-24T23:05:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6811/3794084091", 2024-04-24T23:05:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6815/3794084091", 2024-04-24T23:05:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6813/3794084091", 2024-04-24T23:05:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:05:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:05:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:05:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:05:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:05:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:05:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:05:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:05:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:05:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:05:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:05:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:05:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 138, 2024-04-24T23:05:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:05:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:05:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:05:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6823", 2024-04-24T23:05:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:05:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6825", 2024-04-24T23:05:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:05:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:05:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6826", 2024-04-24T23:05:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:05:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6827", 2024-04-24T23:05:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:05:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:05:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6830", 2024-04-24T23:05:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:05:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6831", 2024-04-24T23:05:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:05:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:05:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6828", 2024-04-24T23:05:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:05:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6829", 2024-04-24T23:05:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:05:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6825/1348640258", 2024-04-24T23:05:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6827/1348640258", 2024-04-24T23:05:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6831/1348640258", 2024-04-24T23:05:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6829/1348640258", 2024-04-24T23:05:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:05:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:05:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:05:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.590 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:05:55.590 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:05:55.590 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:05:55.590 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:05:55.590 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:05:55.590 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:05:55.590 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:05:55.590 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:05:55.590 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:05:55.590 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 138, 2024-04-24T23:05:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:05:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:05:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:05:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6816", 2024-04-24T23:05:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:05:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6817", 2024-04-24T23:05:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:05:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:05:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6818", 2024-04-24T23:05:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:05:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6819", 2024-04-24T23:05:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:05:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:05:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6822", 2024-04-24T23:05:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:05:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6824", 2024-04-24T23:05:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:05:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:05:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:05:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:05:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6820", 2024-04-24T23:05:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:05:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:05:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6821", 2024-04-24T23:05:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:05:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6817/3250005122", 2024-04-24T23:05:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6819/3250005122", 2024-04-24T23:05:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6824/3250005122", 2024-04-24T23:05:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6821/3250005122", 2024-04-24T23:05:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:05:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:05:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:05:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:05:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2024-04-24T23:05:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:05:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:05:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:05:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:05:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:05:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:05:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.156184+0000", 2024-04-24T23:05:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:05:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:05:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:05:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:05:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:05:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:05:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:05:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240774+0000", 2024-04-24T23:05:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:05:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:05:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:05:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:05:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:05:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:05:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:05:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240949+0000", 2024-04-24T23:05:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:05:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:05:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:05:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:05:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:05:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:05:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:05:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.171672+0000", 2024-04-24T23:05:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:05:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:05:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:05:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:05:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:05:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:05:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:05:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.220379+0000", 2024-04-24T23:05:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:05:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:05:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:05:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:05:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:05:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:05:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:05:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.204963+0000", 2024-04-24T23:05:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:05:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:05:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:05:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:05:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:05:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:05:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:05:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.239748+0000", 2024-04-24T23:05:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:05:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:05:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:05:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:05:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:05:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:05:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:05:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.154104+0000", 2024-04-24T23:05:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:05:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:05:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2024-04-24T23:05:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [ 2024-04-24T23:05:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "16.17", 2024-04-24T23:05:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:05:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 4, 2024-04-24T23:05:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 3 2024-04-24T23:05:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "17.8", 2024-04-24T23:05:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:05:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 6, 2024-04-24T23:05:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 5 2024-04-24T23:05:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "17.3f", 2024-04-24T23:05:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:05:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:05:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 6, 2024-04-24T23:05:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 1 2024-04-24T23:05:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:05:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:05:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_primaries": [], 2024-04-24T23:05:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2024-04-24T23:05:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2024-04-24T23:05:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklist": {}, 2024-04-24T23:05:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: "range_blocklist": {}, 2024-04-24T23:05:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2024-04-24T23:05:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2024-04-24T23:05:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2024-04-24T23:05:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2024-04-24T23:05:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2024-04-24T23:05:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2024-04-24T23:05:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2024-04-24T23:05:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:05:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2024-04-24T23:05:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2024-04-24T23:05:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2024-04-24T23:05:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2024-04-24T23:05:55.606 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {}, 2024-04-24T23:05:55.606 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode": { 2024-04-24T23:05:55.606 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_enabled": false, 2024-04-24T23:05:55.606 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_bucket_count": 0, 2024-04-24T23:05:55.606 INFO:teuthology.orchestra.run.smithi032.stdout: "degraded_stretch_mode": 0, 2024-04-24T23:05:55.606 INFO:teuthology.orchestra.run.smithi032.stdout: "recovering_stretch_mode": 0, 2024-04-24T23:05:55.606 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_bucket": 0 2024-04-24T23:05:55.606 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:05:55.606 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:05:55.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.510+0000 7ff3cc314640 1 -- 172.21.15.32:0/1443769785 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff3a806be30 msgr2=0x7ff3a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:55.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.510+0000 7ff3cc314640 1 --2- 172.21.15.32:0/1443769785 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff3a806be30 0x7ff3a806e2f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7ff3c00015e0 tx=0x7ff3c0009040 comp rx=0 tx=0).stop 2024-04-24T23:05:55.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.510+0000 7ff3cc314640 1 -- 172.21.15.32:0/1443769785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3c415e9b0 msgr2=0x7ff3c40ca9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:55.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.510+0000 7ff3cc314640 1 --2- 172.21.15.32:0/1443769785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3c415e9b0 0x7ff3c40ca9f0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc009f40 tx=0x7ff3bc004a40 comp rx=0 tx=0).stop 2024-04-24T23:05:55.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.510+0000 7ff3cc314640 1 -- 172.21.15.32:0/1443769785 shutdown_connections 2024-04-24T23:05:55.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.510+0000 7ff3cc314640 1 --2- 172.21.15.32:0/1443769785 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff3a806be30 0x7ff3a806e2f0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:55.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.510+0000 7ff3cc314640 1 --2- 172.21.15.32:0/1443769785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3c415e9b0 0x7ff3c40ca9f0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:55.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.510+0000 7ff3cc314640 1 --2- 172.21.15.32:0/1443769785 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3c40d4210 0x7ff3c41e3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:55.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.510+0000 7ff3cc314640 1 --2- 172.21.15.32:0/1443769785 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff3c406b820 0x7ff3c40ce5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:55.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.510+0000 7ff3cc314640 1 -- 172.21.15.32:0/1443769785 >> 172.21.15.32:0/1443769785 conn(0x7ff3c40d11d0 msgr2=0x7ff3c4159ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:55.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.510+0000 7ff3cc314640 1 -- 172.21.15.32:0/1443769785 shutdown_connections 2024-04-24T23:05:55.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:05:55.510+0000 7ff3cc314640 1 -- 172.21.15.32:0/1443769785 wait complete. 2024-04-24T23:05:55.611 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-04-24T23:06:00.613 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-24T23:06:00.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.726+0000 7f9f27c1c640 1 Processor -- start 2024-04-24T23:06:00.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.726+0000 7f9f27c1c640 1 -- start start 2024-04-24T23:06:00.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.727+0000 7f9f27c1c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9f20074210 0x7f9f2006c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:00.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.727+0000 7f9f27c1c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9f200ae140 0x7f9f200b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:00.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.727+0000 7f9f27c1c640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f200b0770 con 0x7f9f200b0b80 2024-04-24T23:06:00.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.727+0000 7f9f27c1c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f9f200b1c20 con 0x7f9f20074210 2024-04-24T23:06:00.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.727+0000 7f9f27c1c640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f9f200b1010 con 0x7f9f200ae140 2024-04-24T23:06:00.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.727+0000 7f9f259b8640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9f20074210 0x7f9f2006c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:00.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.727+0000 7f9f259b8640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9f20074210 0x7f9f2006c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42564/0 (socket says 172.21.15.32:42564) 2024-04-24T23:06:00.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.727+0000 7f9f259b8640 1 -- 172.21.15.32:0/1954807399 learned_addr learned my addr 172.21.15.32:0/1954807399 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:00.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.727+0000 7f9f261b9640 1 --2- 172.21.15.32:0/1954807399 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9f200ae140 0x7f9f200b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:00.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.727+0000 7f9f259b8640 1 -- 172.21.15.32:0/1954807399 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9f200ae140 msgr2=0x7f9f200b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:00.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.727+0000 7f9f259b8640 1 --2- 172.21.15.32:0/1954807399 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9f200ae140 0x7f9f200b20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:00.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.727+0000 7f9f259b8640 1 -- 172.21.15.32:0/1954807399 >> v1:172.21.15.121:6789/0 conn(0x7f9f200b0b80 legacy=0x7f9f200745f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:00.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.727+0000 7f9f259b8640 1 -- 172.21.15.32:0/1954807399 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f200b12f0 con 0x7f9f20074210 2024-04-24T23:06:00.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.728+0000 7f9f259b8640 1 --2- 172.21.15.32:0/1954807399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f20074210 0x7f9f2006c480 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c009870 tx=0x7f9f1c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3b76050c94e0ff3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:00.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.728+0000 7f9f249b6640 1 -- 172.21.15.32:0/1954807399 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c037070 con 0x7f9f20074210 2024-04-24T23:06:00.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.728+0000 7f9f249b6640 1 -- 172.21.15.32:0/1954807399 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f1c032440 con 0x7f9f20074210 2024-04-24T23:06:00.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.728+0000 7f9f249b6640 1 -- 172.21.15.32:0/1954807399 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c032810 con 0x7f9f20074210 2024-04-24T23:06:00.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.729+0000 7f9f27c1c640 1 -- 172.21.15.32:0/1954807399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f20074210 msgr2=0x7f9f2006c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:00.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.729+0000 7f9f27c1c640 1 --2- 172.21.15.32:0/1954807399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f20074210 0x7f9f2006c480 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c009870 tx=0x7f9f1c02a210 comp rx=0 tx=0).stop 2024-04-24T23:06:00.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.729+0000 7f9f27c1c640 1 -- 172.21.15.32:0/1954807399 shutdown_connections 2024-04-24T23:06:00.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.729+0000 7f9f27c1c640 1 --2- 172.21.15.32:0/1954807399 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9f200ae140 0x7f9f200b20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:00.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.729+0000 7f9f27c1c640 1 --2- 172.21.15.32:0/1954807399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f20074210 0x7f9f2006c480 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:00.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.729+0000 7f9f27c1c640 1 -- 172.21.15.32:0/1954807399 >> 172.21.15.32:0/1954807399 conn(0x7f9f200b2e60 msgr2=0x7f9f200b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:00.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.729+0000 7f9f27c1c640 1 -- 172.21.15.32:0/1954807399 shutdown_connections 2024-04-24T23:06:00.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.729+0000 7f9f27c1c640 1 -- 172.21.15.32:0/1954807399 wait complete. 2024-04-24T23:06:00.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.730+0000 7f9f27c1c640 1 Processor -- start 2024-04-24T23:06:00.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.730+0000 7f9f27c1c640 1 -- start start 2024-04-24T23:06:00.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.731+0000 7f9f27c1c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f20074210 0x7f9f2015db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:00.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.731+0000 7f9f27c1c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9f200ae140 0x7f9f201e8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:00.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.731+0000 7f9f27c1c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f200cb920 0x7f9f200ccf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:00.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.731+0000 7f9f27c1c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9f200b3ba0 con 0x7f9f200cb920 2024-04-24T23:06:00.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.731+0000 7f9f27c1c640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9f200b18b0 con 0x7f9f20074210 2024-04-24T23:06:00.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.731+0000 7f9f27c1c640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f9f200b3d20 con 0x7f9f200ae140 2024-04-24T23:06:00.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.731+0000 7f9f261b9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f200cb920 0x7f9f200ccf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:00.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.731+0000 7f9f261b9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f200cb920 0x7f9f200ccf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42578/0 (socket says 172.21.15.32:42578) 2024-04-24T23:06:00.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.731+0000 7f9f261b9640 1 -- 172.21.15.32:0/84652152 learned_addr learned my addr 172.21.15.32:0/84652152 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:00.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.731+0000 7f9f259b8640 1 --2- 172.21.15.32:0/84652152 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f20074210 0x7f9f2015db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:00.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.731+0000 7f9f251b7640 1 --2- 172.21.15.32:0/84652152 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9f200ae140 0x7f9f201e8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:00.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.732+0000 7f9f261b9640 1 -- 172.21.15.32:0/84652152 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9f200ae140 msgr2=0x7f9f201e8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:00.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.732+0000 7f9f261b9640 1 --2- 172.21.15.32:0/84652152 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9f200ae140 0x7f9f201e8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:00.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.732+0000 7f9f261b9640 1 -- 172.21.15.32:0/84652152 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f20074210 msgr2=0x7f9f2015db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:00.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.732+0000 7f9f261b9640 1 --2- 172.21.15.32:0/84652152 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f20074210 0x7f9f2015db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:00.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.732+0000 7f9f261b9640 1 -- 172.21.15.32:0/84652152 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f2015e0a0 con 0x7f9f200cb920 2024-04-24T23:06:00.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.732+0000 7f9f261b9640 1 --2- 172.21.15.32:0/84652152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f200cb920 0x7f9f200ccf10 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f9f1800cda0 tx=0x7f9f1800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:00.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.732+0000 7f9f167fc640 1 -- 172.21.15.32:0/84652152 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f18003b90 con 0x7f9f200cb920 2024-04-24T23:06:00.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.732+0000 7f9f167fc640 1 -- 172.21.15.32:0/84652152 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f18003d30 con 0x7f9f200cb920 2024-04-24T23:06:00.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.732+0000 7f9f167fc640 1 -- 172.21.15.32:0/84652152 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f18004240 con 0x7f9f200cb920 2024-04-24T23:06:00.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.732+0000 7f9f27c1c640 1 -- 172.21.15.32:0/84652152 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f200cbe70 con 0x7f9f200cb920 2024-04-24T23:06:00.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.732+0000 7f9f27c1c640 1 -- 172.21.15.32:0/84652152 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f200cc430 con 0x7f9f200cb920 2024-04-24T23:06:00.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.733+0000 7f9f27c1c640 1 -- 172.21.15.32:0/84652152 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f20068430 con 0x7f9f200cb920 2024-04-24T23:06:00.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.734+0000 7f9f167fc640 1 -- 172.21.15.32:0/84652152 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9f180043e0 con 0x7f9f200cb920 2024-04-24T23:06:00.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.734+0000 7f9f167fc640 1 --2- 172.21.15.32:0/84652152 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9f0406be30 0x7f9f0406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:00.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.734+0000 7f9f167fc640 1 -- 172.21.15.32:0/84652152 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5796+0+0 (secure 0 0 0) 0x7f9f1808b890 con 0x7f9f200cb920 2024-04-24T23:06:00.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.734+0000 7f9f167fc640 1 -- 172.21.15.32:0/84652152 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=145}) -- 0x7f9f04073a80 con 0x7f9f200cb920 2024-04-24T23:06:00.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.734+0000 7f9f259b8640 1 --2- 172.21.15.32:0/84652152 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9f0406be30 0x7f9f0406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:00.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.735+0000 7f9f259b8640 1 --2- 172.21.15.32:0/84652152 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9f0406be30 0x7f9f0406e2f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c034000 tx=0x7f9f1c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:00.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.738+0000 7f9f167fc640 1 -- 172.21.15.32:0/84652152 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9f1805a270 con 0x7f9f200cb920 2024-04-24T23:06:00.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.926+0000 7f9f27c1c640 1 -- 172.21.15.32:0/84652152 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f9f2006b220 con 0x7f9f200cb920 2024-04-24T23:06:00.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.928+0000 7f9f167fc640 1 -- 172.21.15.32:0/84652152 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v144) ==== 81+0+34760 (secure 0 0 0) 0x7f9f18059b70 con 0x7f9f200cb920 2024-04-24T23:06:00.930 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:06:00.930 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:06:00.930 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 144, 2024-04-24T23:06:00.930 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", 2024-04-24T23:06:00.930 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2024-04-24T22:58:46.964851+0000", 2024-04-24T23:06:00.930 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2024-04-24T23:05:53.811291+0000", 2024-04-24T23:06:00.930 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2024-04-24T22:58:52.048787+0000", 2024-04-24T23:06:00.930 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2024-04-24T22:58:49.045609+0000", 2024-04-24T23:06:00.931 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-24T23:06:00.931 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2024-04-24T23:06:00.931 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2024-04-24T23:06:00.931 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2024-04-24T23:06:00.931 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2024-04-24T23:06:00.931 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2024-04-24T23:06:00.931 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2024-04-24T23:06:00.931 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:06:00.931 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 4, 2024-04-24T23:06:00.931 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.69999998807907104, 2024-04-24T23:06:00.931 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-24T23:06:00.932 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-24T23:06:00.932 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2024-04-24T23:06:00.932 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 17, 2024-04-24T23:06:00.932 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2024-04-24T23:06:00.932 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "luminous", 2024-04-24T23:06:00.932 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "luminous", 2024-04-24T23:06:00.932 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "squid", 2024-04-24T23:06:00.932 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_crimson": false, 2024-04-24T23:06:00.932 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2024-04-24T23:06:00.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.932 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2024-04-24T23:06:00.932 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": ".mgr", 2024-04-24T23:06:00.932 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T22:58:53.171408+0000", 2024-04-24T23:06:00.932 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:06:00.932 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:06:00.933 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:06:00.933 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:06:00.933 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:06:00.933 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:06:00.933 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:06:00.933 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:06:00.933 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:06:00.933 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:06:00.933 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:06:00.933 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:06:00.933 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2024-04-24T23:06:00.933 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2024-04-24T23:06:00.933 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2024-04-24T23:06:00.933 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2024-04-24T23:06:00.934 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2024-04-24T23:06:00.934 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:06:00.934 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:06:00.934 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:06:00.934 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:06:00.934 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:06:00.934 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:06:00.934 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:06:00.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.934 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "113", 2024-04-24T23:06:00.934 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:06:00.934 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:06:00.934 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:06:00.934 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:06:00.935 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:06:00.935 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:06:00.935 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:06:00.935 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:06:00.935 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:06:00.935 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:06:00.935 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:06:00.935 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:06:00.935 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:06:00.935 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:06:00.935 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:06:00.935 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:06:00.935 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:06:00.935 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:06:00.935 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:06:00.936 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:06:00.936 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:06:00.936 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:06:00.936 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:06:00.936 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:06:00.936 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:06:00.936 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:06:00.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.936 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:06:00.936 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:06:00.936 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:06:00.936 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:06:00.936 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:06:00.936 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:06:00.936 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:06:00.937 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:06:00.937 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:06:00.937 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:06:00.937 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:06:00.937 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:06:00.937 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_max": 32, 2024-04-24T23:06:00.937 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2024-04-24T23:06:00.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.937 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:06:00.937 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": {} 2024-04-24T23:06:00.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.937 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:06:00.937 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:06:00.937 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 8, 2024-04-24T23:06:00.937 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 8, 2024-04-24T23:06:00.938 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.25, 2024-04-24T23:06:00.938 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-04-24T23:06:00.938 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-04-24T23:06:00.938 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:06:00.938 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:06:00.938 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:06:00.938 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:00.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.938 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 16, 2024-04-24T23:06:00.938 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2024-04-24T23:06:00.938 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:05:32.064234+0000", 2024-04-24T23:06:00.938 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:06:00.939 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:06:00.939 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:06:00.939 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:06:00.939 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:06:00.939 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:06:00.939 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:06:00.939 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:06:00.939 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:06:00.939 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:06:00.939 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:06:00.939 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:06:00.939 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 32, 2024-04-24T23:06:00.939 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 32, 2024-04-24T23:06:00.939 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 32, 2024-04-24T23:06:00.940 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 32, 2024-04-24T23:06:00.940 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 32, 2024-04-24T23:06:00.940 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:06:00.940 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:06:00.940 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:06:00.940 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:06:00.940 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:06:00.940 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:06:00.940 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:06:00.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.940 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "139", 2024-04-24T23:06:00.940 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:06:00.940 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:06:00.940 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:06:00.940 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:06:00.941 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:06:00.941 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:06:00.941 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:06:00.941 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:06:00.941 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:06:00.941 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:06:00.941 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:06:00.941 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:06:00.941 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:06:00.941 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:06:00.941 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:06:00.941 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:06:00.941 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:06:00.941 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:06:00.941 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:06:00.942 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:06:00.942 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:06:00.942 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:06:00.942 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:06:00.942 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:06:00.942 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:06:00.942 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:06:00.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.942 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:06:00.942 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:06:00.942 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:06:00.942 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:06:00.942 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:06:00.942 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:06:00.942 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:06:00.943 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:06:00.943 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:06:00.943 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:06:00.943 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:06:00.943 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:06:00.943 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2024-04-24T23:06:00.943 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2024-04-24T23:06:00.943 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2024-04-24T23:06:00.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.943 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:06:00.943 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:06:00.943 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2024-04-24T23:06:00.943 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:00.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.944 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:06:00.944 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:06:00.944 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 2, 2024-04-24T23:06:00.944 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 2, 2024-04-24T23:06:00.944 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:06:00.944 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-04-24T23:06:00.944 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-04-24T23:06:00.944 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:06:00.944 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:06:00.944 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:06:00.944 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:00.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.944 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 17, 2024-04-24T23:06:00.945 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2024-04-24T23:06:00.945 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:05:32.760777+0000", 2024-04-24T23:06:00.945 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1027, 2024-04-24T23:06:00.945 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,full,full_quota", 2024-04-24T23:06:00.945 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:06:00.945 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:06:00.945 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:06:00.945 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:06:00.945 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:06:00.945 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:06:00.945 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:06:00.945 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:06:00.945 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:06:00.945 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:06:00.945 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 64, 2024-04-24T23:06:00.946 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 64, 2024-04-24T23:06:00.946 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 64, 2024-04-24T23:06:00.946 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 64, 2024-04-24T23:06:00.946 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 64, 2024-04-24T23:06:00.946 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:06:00.946 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:06:00.946 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:06:00.946 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:06:00.946 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:06:00.946 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:06:00.946 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:06:00.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.946 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "142", 2024-04-24T23:06:00.946 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:06:00.947 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:06:00.947 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:06:00.947 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:06:00.947 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:06:00.947 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:06:00.947 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:06:00.947 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:06:00.947 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:06:00.947 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 33554432, 2024-04-24T23:06:00.947 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:06:00.947 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:06:00.947 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:06:00.947 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:06:00.947 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:06:00.948 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:06:00.948 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:06:00.948 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:06:00.948 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:06:00.948 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:06:00.948 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:06:00.948 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:06:00.948 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:06:00.948 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:06:00.948 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:06:00.948 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:06:00.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.948 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:06:00.948 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:06:00.949 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:06:00.949 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:06:00.949 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:06:00.949 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:06:00.949 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:06:00.949 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:06:00.949 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:06:00.949 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:06:00.949 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:06:00.949 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:06:00.949 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 64, 2024-04-24T23:06:00.949 INFO:teuthology.orchestra.run.smithi032.stdout: "target_size_ratio": 0.89999997615814209 2024-04-24T23:06:00.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.949 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:06:00.949 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:06:00.950 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2024-04-24T23:06:00.950 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:00.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.950 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:06:00.950 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:06:00.950 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.3799999952316284, 2024-04-24T23:06:00.950 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.3799999952316284, 2024-04-24T23:06:00.950 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:06:00.950 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.3799999952316284, 2024-04-24T23:06:00.950 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.3799999952316284, 2024-04-24T23:06:00.950 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:06:00.950 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:06:00.950 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:06:00.950 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:00.951 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:00.951 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:06:00.951 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-04-24T23:06:00.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.951 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:06:00.951 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:06:00.951 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:06:00.951 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:06:00.951 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:06:00.951 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:06:00.951 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:06:00.951 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:06:00.951 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:06:00.951 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 143, 2024-04-24T23:06:00.952 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:06:00.952 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:06:00.952 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:06:00.952 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:00.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.952 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:00.952 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2024-04-24T23:06:00.952 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:06:00.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.952 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:00.952 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2024-04-24T23:06:00.952 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:06:00.952 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:00.953 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:00.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.953 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:06:00.953 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:00.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.953 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:00.953 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2024-04-24T23:06:00.953 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:06:00.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.953 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:00.953 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2024-04-24T23:06:00.953 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:06:00.953 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:00.954 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:00.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.954 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:06:00.954 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:00.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.954 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:00.954 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2024-04-24T23:06:00.954 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:06:00.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.954 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:00.954 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2024-04-24T23:06:00.954 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:06:00.954 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:00.955 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:00.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.955 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:06:00.955 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:00.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.955 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:00.955 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2024-04-24T23:06:00.955 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:06:00.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.955 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:00.955 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2024-04-24T23:06:00.955 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:06:00.955 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:00.955 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:00.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.956 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/1479395426", 2024-04-24T23:06:00.956 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/1479395426", 2024-04-24T23:06:00.956 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/1479395426", 2024-04-24T23:06:00.956 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/1479395426", 2024-04-24T23:06:00.956 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:06:00.956 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:06:00.956 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:06:00.956 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:00.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.956 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:06:00.956 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:06:00.956 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:06:00.956 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:06:00.957 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:06:00.957 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:06:00.957 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:06:00.957 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:06:00.957 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:06:00.957 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 143, 2024-04-24T23:06:00.957 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:06:00.957 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:06:00.957 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:06:00.957 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:00.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.957 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:00.957 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2024-04-24T23:06:00.957 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:06:00.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.958 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:00.958 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2024-04-24T23:06:00.958 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:06:00.958 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:00.958 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:00.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.958 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:06:00.958 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:00.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.958 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:00.958 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2024-04-24T23:06:00.958 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:06:00.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.959 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:00.959 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2024-04-24T23:06:00.959 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:06:00.959 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:00.959 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:00.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.959 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:06:00.959 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:00.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.959 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:00.959 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2024-04-24T23:06:00.959 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:06:00.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.960 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:00.960 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2024-04-24T23:06:00.960 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:06:00.960 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:00.960 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:00.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.960 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:06:00.960 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:00.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.960 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:00.960 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2024-04-24T23:06:00.960 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:06:00.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.960 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:00.961 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2024-04-24T23:06:00.961 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:06:00.961 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:00.961 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:00.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.961 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/1340633718", 2024-04-24T23:06:00.961 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/1340633718", 2024-04-24T23:06:00.961 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/1340633718", 2024-04-24T23:06:00.961 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/1340633718", 2024-04-24T23:06:00.961 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:06:00.961 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:06:00.961 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:06:00.961 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:00.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.962 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:06:00.962 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:06:00.962 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:06:00.962 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:06:00.962 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:06:00.962 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:06:00.962 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:06:00.962 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:06:00.962 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:06:00.962 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 138, 2024-04-24T23:06:00.962 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:06:00.962 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:06:00.962 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:06:00.962 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:00.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.963 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:00.963 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2024-04-24T23:06:00.963 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:06:00.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.963 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:00.963 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2024-04-24T23:06:00.963 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:06:00.963 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:00.963 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:00.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.963 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:06:00.963 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:00.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.964 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:00.964 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2024-04-24T23:06:00.964 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:06:00.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.964 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:00.964 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2024-04-24T23:06:00.964 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:06:00.964 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:00.964 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:00.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.964 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:06:00.964 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:00.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.965 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:00.965 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2024-04-24T23:06:00.965 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:06:00.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.965 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:00.965 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2024-04-24T23:06:00.965 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:06:00.965 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:00.965 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:00.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.965 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:06:00.965 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:00.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.966 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:00.966 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2024-04-24T23:06:00.966 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:06:00.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.966 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:00.966 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2024-04-24T23:06:00.966 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:06:00.966 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:00.966 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:00.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.966 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/3405899334", 2024-04-24T23:06:00.966 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6820/3405899334", 2024-04-24T23:06:00.966 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6828/3405899334", 2024-04-24T23:06:00.967 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6824/3405899334", 2024-04-24T23:06:00.967 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:06:00.967 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:06:00.967 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:06:00.967 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:00.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.967 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.967 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:06:00.967 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:06:00.967 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:06:00.967 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:06:00.967 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:06:00.967 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:06:00.967 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:06:00.967 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:06:00.968 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:06:00.968 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 143, 2024-04-24T23:06:00.968 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:06:00.968 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:06:00.968 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:06:00.968 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:00.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.968 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:00.968 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2024-04-24T23:06:00.968 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:06:00.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.968 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:00.968 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2024-04-24T23:06:00.968 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:06:00.969 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:00.969 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:00.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.969 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:06:00.969 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:00.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.969 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:00.969 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2024-04-24T23:06:00.969 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:06:00.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.969 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:00.969 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2024-04-24T23:06:00.969 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:06:00.969 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:00.970 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:00.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.970 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:06:00.970 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:00.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.970 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:00.970 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2024-04-24T23:06:00.970 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:06:00.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.970 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:00.970 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2024-04-24T23:06:00.970 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:06:00.970 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:00.971 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:00.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.971 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:06:00.971 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:00.971 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.971 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:00.971 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2024-04-24T23:06:00.971 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:06:00.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.971 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.971 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:00.971 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2024-04-24T23:06:00.971 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:06:00.971 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:00.971 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:00.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.972 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6821/3381057190", 2024-04-24T23:06:00.972 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6825/3381057190", 2024-04-24T23:06:00.972 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/3381057190", 2024-04-24T23:06:00.972 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/3381057190", 2024-04-24T23:06:00.972 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:06:00.972 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:06:00.972 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:06:00.972 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:00.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.972 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:06:00.972 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:06:00.972 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:06:00.973 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:06:00.973 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:06:00.973 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:06:00.973 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:06:00.973 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:06:00.973 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:06:00.973 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 138, 2024-04-24T23:06:00.973 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:06:00.973 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:06:00.973 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:06:00.973 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:00.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.973 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:00.973 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6800", 2024-04-24T23:06:00.973 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:06:00.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.974 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:00.974 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6801", 2024-04-24T23:06:00.974 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:06:00.974 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:00.974 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:00.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.974 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:06:00.974 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:00.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.974 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:00.974 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6802", 2024-04-24T23:06:00.974 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:06:00.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.975 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:00.975 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6803", 2024-04-24T23:06:00.975 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:06:00.975 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:00.975 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:00.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.975 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:06:00.975 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:00.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.975 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:00.975 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6806", 2024-04-24T23:06:00.975 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:06:00.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.976 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:00.976 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6807", 2024-04-24T23:06:00.976 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:06:00.976 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:00.976 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:00.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.976 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:06:00.976 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:00.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.976 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:00.976 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6804", 2024-04-24T23:06:00.976 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:06:00.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.977 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:00.977 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6805", 2024-04-24T23:06:00.977 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:06:00.977 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:00.977 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:00.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.977 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6801/4010553979", 2024-04-24T23:06:00.977 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6803/4010553979", 2024-04-24T23:06:00.977 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6807/4010553979", 2024-04-24T23:06:00.977 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6805/4010553979", 2024-04-24T23:06:00.977 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:06:00.977 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:06:00.977 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:06:00.977 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:00.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.978 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:06:00.978 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:06:00.978 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:06:00.978 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:06:00.978 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:06:00.978 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:06:00.978 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:06:00.978 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:06:00.978 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:06:00.978 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 138, 2024-04-24T23:06:00.978 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:06:00.978 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:06:00.978 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:06:00.979 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:00.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.979 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:00.979 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6808", 2024-04-24T23:06:00.979 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:06:00.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.979 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:00.979 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6809", 2024-04-24T23:06:00.979 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:06:00.979 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:00.979 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:00.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.979 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:06:00.980 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:00.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.980 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:00.980 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6810", 2024-04-24T23:06:00.980 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:06:00.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.980 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:00.980 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6811", 2024-04-24T23:06:00.980 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:06:00.980 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:00.980 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:00.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.980 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:06:00.981 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:00.981 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.981 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:00.981 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6814", 2024-04-24T23:06:00.981 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:06:00.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.981 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.981 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:00.981 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6815", 2024-04-24T23:06:00.981 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:06:00.981 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:00.981 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:00.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.981 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:06:00.982 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:00.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.982 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:00.982 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6812", 2024-04-24T23:06:00.982 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:06:00.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.982 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:00.982 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6813", 2024-04-24T23:06:00.982 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:06:00.982 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:00.982 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:00.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.983 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6809/3794084091", 2024-04-24T23:06:00.983 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6811/3794084091", 2024-04-24T23:06:00.983 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6815/3794084091", 2024-04-24T23:06:00.983 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6813/3794084091", 2024-04-24T23:06:00.983 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:06:00.983 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:06:00.983 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:06:00.983 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:00.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.983 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:06:00.983 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:06:00.983 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:06:00.983 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:06:00.983 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:06:00.984 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:06:00.984 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:06:00.984 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:06:00.984 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:06:00.984 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 138, 2024-04-24T23:06:00.984 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:06:00.984 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:06:00.984 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:06:00.984 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:00.984 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.984 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:00.984 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6823", 2024-04-24T23:06:00.984 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:06:00.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.985 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:00.985 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6825", 2024-04-24T23:06:00.985 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:06:00.985 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:00.985 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:00.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.985 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:06:00.985 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:00.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.985 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:00.985 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6826", 2024-04-24T23:06:00.985 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:06:00.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.986 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.986 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:00.986 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6827", 2024-04-24T23:06:00.986 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:06:00.986 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:00.986 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:00.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.986 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:06:00.986 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:00.986 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.986 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:00.986 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6830", 2024-04-24T23:06:00.986 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:06:00.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.987 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.987 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:00.987 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6831", 2024-04-24T23:06:00.987 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:06:00.987 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:00.987 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:00.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.987 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:06:00.987 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:00.987 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.987 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:00.987 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6828", 2024-04-24T23:06:00.987 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:06:00.988 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.988 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.988 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:00.988 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6829", 2024-04-24T23:06:00.988 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:06:00.988 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:00.988 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:00.988 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.988 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6825/1348640258", 2024-04-24T23:06:00.988 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6827/1348640258", 2024-04-24T23:06:00.988 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6831/1348640258", 2024-04-24T23:06:00.988 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6829/1348640258", 2024-04-24T23:06:00.988 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:06:00.988 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:06:00.988 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:06:00.989 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:00.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.989 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.989 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:06:00.989 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:06:00.989 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:06:00.989 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:06:00.989 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:06:00.989 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:06:00.989 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:06:00.989 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:06:00.989 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:06:00.989 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 138, 2024-04-24T23:06:00.990 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:06:00.990 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:06:00.990 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:06:00.990 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:00.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.990 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:00.990 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6816", 2024-04-24T23:06:00.990 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:06:00.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.990 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:00.990 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6817", 2024-04-24T23:06:00.990 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:06:00.990 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:00.991 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:00.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.991 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:06:00.991 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:00.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.991 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:00.991 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6818", 2024-04-24T23:06:00.991 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:06:00.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.991 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:00.991 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6819", 2024-04-24T23:06:00.991 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:06:00.991 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:00.991 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:00.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.992 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:06:00.992 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:00.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.992 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:00.992 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6822", 2024-04-24T23:06:00.992 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:06:00.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.992 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:00.992 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6824", 2024-04-24T23:06:00.992 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:06:00.992 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:00.992 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:00.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.993 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:06:00.993 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:00.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.993 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:00.993 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6820", 2024-04-24T23:06:00.993 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:06:00.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.993 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:00.993 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6821", 2024-04-24T23:06:00.993 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:06:00.993 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:00.993 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:00.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.994 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6817/3250005122", 2024-04-24T23:06:00.994 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6819/3250005122", 2024-04-24T23:06:00.994 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6824/3250005122", 2024-04-24T23:06:00.994 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6821/3250005122", 2024-04-24T23:06:00.994 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:06:00.994 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:06:00.994 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:06:00.994 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:00.994 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:00.994 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:06:00.994 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2024-04-24T23:06:00.994 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.994 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:06:00.994 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:06:00.995 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:06:00.995 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:06:00.995 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:06:00.995 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:06:00.995 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.156184+0000", 2024-04-24T23:06:00.995 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:06:00.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.995 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:06:00.995 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:06:00.995 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:06:00.995 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:06:00.995 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:06:00.995 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:06:00.995 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240774+0000", 2024-04-24T23:06:00.996 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:06:00.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.996 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:06:00.996 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:06:00.996 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:06:00.996 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:06:00.996 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:06:00.996 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:06:00.996 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240949+0000", 2024-04-24T23:06:00.996 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:06:00.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.996 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:06:00.996 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:06:00.997 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:06:00.997 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:06:00.997 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:06:00.997 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:06:00.997 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.171672+0000", 2024-04-24T23:06:00.997 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:06:00.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.997 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:06:00.997 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:06:00.997 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:06:00.997 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:06:00.997 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:06:00.997 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:06:00.998 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.220379+0000", 2024-04-24T23:06:00.998 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:06:00.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.998 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:06:00.998 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:06:00.998 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:06:00.998 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:06:00.998 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:06:00.998 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:06:00.998 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.204963+0000", 2024-04-24T23:06:00.998 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:06:00.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.999 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:06:00.999 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:06:00.999 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:06:00.999 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:06:00.999 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:06:00.999 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:06:00.999 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.239748+0000", 2024-04-24T23:06:00.999 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:06:00.999 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:00.999 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:00.999 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:06:00.999 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:06:00.999 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:06:00.999 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:06:00.999 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:06:01.000 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:06:01.000 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.154104+0000", 2024-04-24T23:06:01.000 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:06:01.000 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:01.000 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:06:01.000 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2024-04-24T23:06:01.000 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [ 2024-04-24T23:06:01.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:01.000 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "16.17", 2024-04-24T23:06:01.000 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:06:01.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:01.000 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 4, 2024-04-24T23:06:01.000 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 3 2024-04-24T23:06:01.000 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:01.001 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:01.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:01.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:01.001 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "17.8", 2024-04-24T23:06:01.001 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:06:01.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:01.001 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 6, 2024-04-24T23:06:01.001 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 5 2024-04-24T23:06:01.001 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:01.001 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:01.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:01.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:01.001 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "17.3f", 2024-04-24T23:06:01.001 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:06:01.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:01.002 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 6, 2024-04-24T23:06:01.002 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 1 2024-04-24T23:06:01.002 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:01.002 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:01.002 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:01.002 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:06:01.002 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_primaries": [], 2024-04-24T23:06:01.002 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2024-04-24T23:06:01.002 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2024-04-24T23:06:01.002 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklist": {}, 2024-04-24T23:06:01.002 INFO:teuthology.orchestra.run.smithi032.stdout: "range_blocklist": {}, 2024-04-24T23:06:01.002 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2024-04-24T23:06:01.002 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2024-04-24T23:06:01.002 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2024-04-24T23:06:01.003 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2024-04-24T23:06:01.003 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2024-04-24T23:06:01.003 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2024-04-24T23:06:01.003 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2024-04-24T23:06:01.003 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:01.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:01.003 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2024-04-24T23:06:01.003 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2024-04-24T23:06:01.003 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2024-04-24T23:06:01.003 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2024-04-24T23:06:01.003 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {}, 2024-04-24T23:06:01.003 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode": { 2024-04-24T23:06:01.003 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_enabled": false, 2024-04-24T23:06:01.003 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_bucket_count": 0, 2024-04-24T23:06:01.003 INFO:teuthology.orchestra.run.smithi032.stdout: "degraded_stretch_mode": 0, 2024-04-24T23:06:01.004 INFO:teuthology.orchestra.run.smithi032.stdout: "recovering_stretch_mode": 0, 2024-04-24T23:06:01.004 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_bucket": 0 2024-04-24T23:06:01.004 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:01.004 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:06:01.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.930+0000 7f9f27c1c640 1 -- 172.21.15.32:0/84652152 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9f0406be30 msgr2=0x7f9f0406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:01.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.930+0000 7f9f27c1c640 1 --2- 172.21.15.32:0/84652152 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9f0406be30 0x7f9f0406e2f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c034000 tx=0x7f9f1c02e000 comp rx=0 tx=0).stop 2024-04-24T23:06:01.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.930+0000 7f9f27c1c640 1 -- 172.21.15.32:0/84652152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f200cb920 msgr2=0x7f9f200ccf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:01.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.930+0000 7f9f27c1c640 1 --2- 172.21.15.32:0/84652152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f200cb920 0x7f9f200ccf10 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f9f1800cda0 tx=0x7f9f1800a660 comp rx=0 tx=0).stop 2024-04-24T23:06:01.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.930+0000 7f9f27c1c640 1 -- 172.21.15.32:0/84652152 shutdown_connections 2024-04-24T23:06:01.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.930+0000 7f9f27c1c640 1 --2- 172.21.15.32:0/84652152 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9f0406be30 0x7f9f0406e2f0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:01.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.930+0000 7f9f27c1c640 1 --2- 172.21.15.32:0/84652152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f200cb920 0x7f9f200ccf10 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:01.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.930+0000 7f9f27c1c640 1 --2- 172.21.15.32:0/84652152 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9f200ae140 0x7f9f201e8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:01.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.930+0000 7f9f27c1c640 1 --2- 172.21.15.32:0/84652152 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f20074210 0x7f9f2015db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:01.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.930+0000 7f9f27c1c640 1 -- 172.21.15.32:0/84652152 >> 172.21.15.32:0/84652152 conn(0x7f9f200b2e60 msgr2=0x7f9f200b20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:01.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.930+0000 7f9f27c1c640 1 -- 172.21.15.32:0/84652152 shutdown_connections 2024-04-24T23:06:01.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:00.930+0000 7f9f27c1c640 1 -- 172.21.15.32:0/84652152 wait complete. 2024-04-24T23:06:01.007 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-04-24T23:06:06.008 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-24T23:06:06.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.120+0000 7f247b2bd640 1 Processor -- start 2024-04-24T23:06:06.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.121+0000 7f247b2bd640 1 -- start start 2024-04-24T23:06:06.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.121+0000 7f247b2bd640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f247415e9b0 0x7f24740d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:06.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.121+0000 7f247b2bd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f24740d4210 0x7f247406b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:06.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.121+0000 7f247b2bd640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24740d3170 con 0x7f24740ba0e0 2024-04-24T23:06:06.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.121+0000 7f247b2bd640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f24740b8900 con 0x7f24740d4210 2024-04-24T23:06:06.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.121+0000 7f247b2bd640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f24740d3450 con 0x7f247415e9b0 2024-04-24T23:06:06.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.122+0000 7f2478858640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f24740d4210 0x7f247406b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:06.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.122+0000 7f247985a640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f24740ba0e0 0x7f2474071730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41390/0 (socket says 172.21.15.32:41390) 2024-04-24T23:06:06.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.122+0000 7f2478858640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f24740d4210 0x7f247406b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42580/0 (socket says 172.21.15.32:42580) 2024-04-24T23:06:06.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.122+0000 7f247985a640 1 -- 172.21.15.32:0/3819997775 learned_addr learned my addr 172.21.15.32:0/3819997775 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:06:06.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.122+0000 7f2479059640 1 --2- 172.21.15.32:0/3819997775 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f247415e9b0 0x7f24740d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:06.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.122+0000 7f2478858640 1 -- 172.21.15.32:0/3819997775 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f247415e9b0 msgr2=0x7f24740d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:06.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.122+0000 7f2478858640 1 --2- 172.21.15.32:0/3819997775 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f247415e9b0 0x7f24740d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:06.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.122+0000 7f2478858640 1 -- 172.21.15.32:0/3819997775 >> v1:172.21.15.32:6789/0 conn(0x7f24740ba0e0 legacy=0x7f2474071730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:06.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.122+0000 7f2478858640 1 -- 172.21.15.32:0/3819997775 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24740d3730 con 0x7f24740d4210 2024-04-24T23:06:06.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.122+0000 7f2473fff640 1 -- 172.21.15.32:0/3819997775 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 54301470 0 0) 0x7f24740d3170 con 0x7f24740ba0e0 2024-04-24T23:06:06.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.123+0000 7f2478858640 1 --2- 172.21.15.32:0/3819997775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24740d4210 0x7f247406b110 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f24680097a0 tx=0x7f246802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55b08fd89ea6bc93 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:06.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.123+0000 7f2473fff640 1 -- 172.21.15.32:0/3819997775 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2468037070 con 0x7f24740d4210 2024-04-24T23:06:06.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.123+0000 7f2473fff640 1 -- 172.21.15.32:0/3819997775 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2468032440 con 0x7f24740d4210 2024-04-24T23:06:06.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.123+0000 7f2473fff640 1 -- 172.21.15.32:0/3819997775 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2468032810 con 0x7f24740d4210 2024-04-24T23:06:06.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.123+0000 7f247b2bd640 1 -- 172.21.15.32:0/3819997775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24740d4210 msgr2=0x7f247406b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:06.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.123+0000 7f247b2bd640 1 --2- 172.21.15.32:0/3819997775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24740d4210 0x7f247406b110 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f24680097a0 tx=0x7f246802a180 comp rx=0 tx=0).stop 2024-04-24T23:06:06.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.123+0000 7f247b2bd640 1 -- 172.21.15.32:0/3819997775 shutdown_connections 2024-04-24T23:06:06.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.123+0000 7f247b2bd640 1 --2- 172.21.15.32:0/3819997775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24740d4210 0x7f247406b110 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:06.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.123+0000 7f247b2bd640 1 --2- 172.21.15.32:0/3819997775 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f247415e9b0 0x7f24740d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:06.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.123+0000 7f247b2bd640 1 -- 172.21.15.32:0/3819997775 >> 172.21.15.32:0/3819997775 conn(0x7f24740d11d0 msgr2=0x7f24740d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:06.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.124+0000 7f247b2bd640 1 -- 172.21.15.32:0/3819997775 shutdown_connections 2024-04-24T23:06:06.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.124+0000 7f247b2bd640 1 -- 172.21.15.32:0/3819997775 wait complete. 2024-04-24T23:06:06.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.125+0000 7f247b2bd640 1 Processor -- start 2024-04-24T23:06:06.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.125+0000 7f247b2bd640 1 -- start start 2024-04-24T23:06:06.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.125+0000 7f247b2bd640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f24740ba0e0 0x7f24741f4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:06.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.125+0000 7f247b2bd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24740d4210 0x7f24740ce140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:06.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.125+0000 7f247b2bd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f247415e9b0 0x7f24741f7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:06.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.125+0000 7f247b2bd640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f24740d2400 con 0x7f247415e9b0 2024-04-24T23:06:06.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.125+0000 7f247b2bd640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f24740d3cf0 con 0x7f24740d4210 2024-04-24T23:06:06.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.125+0000 7f247b2bd640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f247406b650 con 0x7f24740ba0e0 2024-04-24T23:06:06.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.126+0000 7f247985a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f247415e9b0 0x7f24741f7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:06.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.126+0000 7f247985a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f247415e9b0 0x7f24741f7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42594/0 (socket says 172.21.15.32:42594) 2024-04-24T23:06:06.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.126+0000 7f247985a640 1 -- 172.21.15.32:0/588298410 learned_addr learned my addr 172.21.15.32:0/588298410 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:06.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.126+0000 7f2479059640 1 --2- 172.21.15.32:0/588298410 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f24740ba0e0 0x7f24741f4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:06.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.126+0000 7f2478858640 1 --2- 172.21.15.32:0/588298410 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24740d4210 0x7f24740ce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:06.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.126+0000 7f247985a640 1 -- 172.21.15.32:0/588298410 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f24740ba0e0 msgr2=0x7f24741f4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:06.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.126+0000 7f247985a640 1 --2- 172.21.15.32:0/588298410 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f24740ba0e0 0x7f24741f4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:06.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.126+0000 7f247985a640 1 -- 172.21.15.32:0/588298410 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24740d4210 msgr2=0x7f24740ce140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:06.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.126+0000 7f247985a640 1 --2- 172.21.15.32:0/588298410 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24740d4210 0x7f24740ce140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:06.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.126+0000 7f247985a640 1 -- 172.21.15.32:0/588298410 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24741f82f0 con 0x7f247415e9b0 2024-04-24T23:06:06.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.126+0000 7f247985a640 1 --2- 172.21.15.32:0/588298410 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f247415e9b0 0x7f24741f7a10 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f246400c7a0 tx=0x7f246400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:06.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.126+0000 7f2471ffb640 1 -- 172.21.15.32:0/588298410 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2464018070 con 0x7f247415e9b0 2024-04-24T23:06:06.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.126+0000 7f2471ffb640 1 -- 172.21.15.32:0/588298410 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f246400a640 con 0x7f247415e9b0 2024-04-24T23:06:06.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.126+0000 7f247b2bd640 1 -- 172.21.15.32:0/588298410 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24741f85d0 con 0x7f247415e9b0 2024-04-24T23:06:06.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.126+0000 7f2471ffb640 1 -- 172.21.15.32:0/588298410 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246400aab0 con 0x7f247415e9b0 2024-04-24T23:06:06.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.127+0000 7f247b2bd640 1 -- 172.21.15.32:0/588298410 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24741f48a0 con 0x7f247415e9b0 2024-04-24T23:06:06.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.128+0000 7f247b2bd640 1 -- 172.21.15.32:0/588298410 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2474068430 con 0x7f247415e9b0 2024-04-24T23:06:06.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.128+0000 7f2471ffb640 1 -- 172.21.15.32:0/588298410 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f246401b070 con 0x7f247415e9b0 2024-04-24T23:06:06.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.129+0000 7f2471ffb640 1 --2- 172.21.15.32:0/588298410 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f246006be30 0x7f246006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:06.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.129+0000 7f2471ffb640 1 -- 172.21.15.32:0/588298410 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5796+0+0 (secure 0 0 0) 0x7f246408a470 con 0x7f247415e9b0 2024-04-24T23:06:06.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.129+0000 7f2471ffb640 1 -- 172.21.15.32:0/588298410 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=145}) -- 0x7f2460073a80 con 0x7f247415e9b0 2024-04-24T23:06:06.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.129+0000 7f2479059640 1 --2- 172.21.15.32:0/588298410 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f246006be30 0x7f246006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:06.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.129+0000 7f2479059640 1 --2- 172.21.15.32:0/588298410 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f246006be30 0x7f246006e2f0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f246c009aa0 tx=0x7f246c009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:06.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.133+0000 7f2471ffb640 1 -- 172.21.15.32:0/588298410 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2464058760 con 0x7f247415e9b0 2024-04-24T23:06:06.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.315+0000 7f247b2bd640 1 -- 172.21.15.32:0/588298410 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f247406ba20 con 0x7f247415e9b0 2024-04-24T23:06:06.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.316+0000 7f2471ffb640 1 -- 172.21.15.32:0/588298410 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v144) ==== 81+0+34760 (secure 0 0 0) 0x7f246405ca60 con 0x7f247415e9b0 2024-04-24T23:06:06.319 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:06:06.319 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:06:06.319 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 144, 2024-04-24T23:06:06.319 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", 2024-04-24T23:06:06.319 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2024-04-24T22:58:46.964851+0000", 2024-04-24T23:06:06.319 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2024-04-24T23:05:53.811291+0000", 2024-04-24T23:06:06.319 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2024-04-24T22:58:52.048787+0000", 2024-04-24T23:06:06.319 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2024-04-24T22:58:49.045609+0000", 2024-04-24T23:06:06.319 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-24T23:06:06.319 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2024-04-24T23:06:06.319 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2024-04-24T23:06:06.319 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2024-04-24T23:06:06.319 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2024-04-24T23:06:06.320 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2024-04-24T23:06:06.320 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2024-04-24T23:06:06.320 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:06:06.320 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 4, 2024-04-24T23:06:06.320 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.69999998807907104, 2024-04-24T23:06:06.320 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-24T23:06:06.320 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-24T23:06:06.320 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2024-04-24T23:06:06.320 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 17, 2024-04-24T23:06:06.320 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2024-04-24T23:06:06.320 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "luminous", 2024-04-24T23:06:06.320 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "luminous", 2024-04-24T23:06:06.320 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "squid", 2024-04-24T23:06:06.321 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_crimson": false, 2024-04-24T23:06:06.321 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2024-04-24T23:06:06.321 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.321 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2024-04-24T23:06:06.321 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": ".mgr", 2024-04-24T23:06:06.321 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T22:58:53.171408+0000", 2024-04-24T23:06:06.321 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:06:06.321 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:06:06.321 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:06:06.321 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:06:06.321 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:06:06.321 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:06:06.321 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:06:06.321 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:06:06.322 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:06:06.322 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:06:06.322 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:06:06.322 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:06:06.322 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2024-04-24T23:06:06.322 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2024-04-24T23:06:06.322 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2024-04-24T23:06:06.322 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2024-04-24T23:06:06.322 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2024-04-24T23:06:06.322 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:06:06.322 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:06:06.322 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:06:06.322 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:06:06.322 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:06:06.323 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:06:06.323 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:06:06.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.323 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "113", 2024-04-24T23:06:06.323 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:06:06.323 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:06:06.323 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:06:06.323 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:06:06.323 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:06:06.323 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:06:06.323 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:06:06.323 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:06:06.323 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:06:06.324 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:06:06.324 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:06:06.324 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:06:06.324 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:06:06.324 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:06:06.324 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:06:06.324 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:06:06.324 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:06:06.324 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:06:06.324 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:06:06.324 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:06:06.324 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:06:06.324 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:06:06.324 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:06:06.325 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:06:06.325 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:06:06.325 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:06:06.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.325 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:06:06.325 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:06:06.325 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:06:06.325 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:06:06.325 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:06:06.325 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:06:06.325 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:06:06.325 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:06:06.325 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:06:06.325 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:06:06.325 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:06:06.326 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:06:06.326 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_max": 32, 2024-04-24T23:06:06.326 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2024-04-24T23:06:06.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.326 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:06:06.326 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": {} 2024-04-24T23:06:06.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.326 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:06:06.326 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:06:06.326 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 8, 2024-04-24T23:06:06.326 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 8, 2024-04-24T23:06:06.326 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.25, 2024-04-24T23:06:06.326 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-04-24T23:06:06.326 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-04-24T23:06:06.327 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:06:06.327 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:06:06.327 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:06:06.327 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:06.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.327 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.327 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 16, 2024-04-24T23:06:06.327 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2024-04-24T23:06:06.327 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:05:32.064234+0000", 2024-04-24T23:06:06.327 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:06:06.327 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:06:06.327 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:06:06.327 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:06:06.327 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:06:06.328 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:06:06.328 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:06:06.328 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:06:06.328 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:06:06.328 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:06:06.328 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:06:06.328 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:06:06.328 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 32, 2024-04-24T23:06:06.328 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 32, 2024-04-24T23:06:06.328 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 32, 2024-04-24T23:06:06.328 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 32, 2024-04-24T23:06:06.328 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 32, 2024-04-24T23:06:06.328 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:06:06.328 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:06:06.329 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:06:06.329 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:06:06.329 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:06:06.329 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:06:06.329 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:06:06.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.329 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "139", 2024-04-24T23:06:06.329 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:06:06.329 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:06:06.329 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:06:06.329 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:06:06.329 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:06:06.329 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:06:06.329 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:06:06.330 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:06:06.330 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:06:06.330 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:06:06.330 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:06:06.330 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:06:06.330 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:06:06.330 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:06:06.330 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:06:06.330 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:06:06.330 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:06:06.330 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:06:06.330 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:06:06.330 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:06:06.331 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:06:06.331 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:06:06.331 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:06:06.331 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:06:06.331 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:06:06.331 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:06:06.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.331 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:06:06.331 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:06:06.331 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:06:06.331 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:06:06.331 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:06:06.331 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:06:06.331 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:06:06.332 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:06:06.332 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:06:06.332 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:06:06.332 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:06:06.332 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:06:06.332 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2024-04-24T23:06:06.332 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2024-04-24T23:06:06.332 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2024-04-24T23:06:06.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.332 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:06:06.332 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:06:06.332 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2024-04-24T23:06:06.332 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:06.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.333 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:06:06.333 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:06:06.333 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 2, 2024-04-24T23:06:06.333 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 2, 2024-04-24T23:06:06.333 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:06:06.333 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-04-24T23:06:06.333 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-04-24T23:06:06.333 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:06:06.333 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:06:06.333 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:06:06.333 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:06.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.333 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 17, 2024-04-24T23:06:06.334 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2024-04-24T23:06:06.334 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:05:32.760777+0000", 2024-04-24T23:06:06.334 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1027, 2024-04-24T23:06:06.334 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,full,full_quota", 2024-04-24T23:06:06.334 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:06:06.334 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:06:06.334 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:06:06.334 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:06:06.334 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:06:06.334 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:06:06.334 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:06:06.334 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:06:06.334 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:06:06.334 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:06:06.334 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 64, 2024-04-24T23:06:06.335 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 64, 2024-04-24T23:06:06.335 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 64, 2024-04-24T23:06:06.335 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 64, 2024-04-24T23:06:06.335 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 64, 2024-04-24T23:06:06.335 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:06:06.335 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:06:06.335 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:06:06.335 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:06:06.335 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:06:06.335 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:06:06.335 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:06:06.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.335 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "142", 2024-04-24T23:06:06.335 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:06:06.336 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:06:06.336 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:06:06.336 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:06:06.336 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:06:06.336 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:06:06.336 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:06:06.336 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:06:06.336 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:06:06.336 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 33554432, 2024-04-24T23:06:06.336 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:06:06.336 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:06:06.336 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:06:06.336 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:06:06.336 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:06:06.337 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:06:06.337 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:06:06.337 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:06:06.337 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:06:06.337 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:06:06.337 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:06:06.337 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:06:06.337 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:06:06.337 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:06:06.337 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:06:06.337 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:06:06.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.337 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:06:06.337 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:06:06.337 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:06:06.338 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:06:06.338 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:06:06.338 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:06:06.338 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:06:06.338 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:06:06.338 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:06:06.338 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:06:06.338 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:06:06.338 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:06:06.338 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 64, 2024-04-24T23:06:06.338 INFO:teuthology.orchestra.run.smithi032.stdout: "target_size_ratio": 0.89999997615814209 2024-04-24T23:06:06.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.338 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:06:06.338 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:06:06.339 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2024-04-24T23:06:06.339 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:06.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.339 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:06:06.339 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:06:06.339 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.3799999952316284, 2024-04-24T23:06:06.339 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.3799999952316284, 2024-04-24T23:06:06.339 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:06:06.339 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.3799999952316284, 2024-04-24T23:06:06.339 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.3799999952316284, 2024-04-24T23:06:06.339 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:06:06.339 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:06:06.339 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:06:06.339 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:06.339 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:06.340 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:06:06.340 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-04-24T23:06:06.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.340 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:06:06.340 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:06:06.340 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:06:06.340 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:06:06.340 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:06:06.340 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:06:06.340 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:06:06.340 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:06:06.340 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:06:06.340 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 143, 2024-04-24T23:06:06.340 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:06:06.340 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:06:06.341 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:06:06.341 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:06.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.341 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:06.341 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2024-04-24T23:06:06.341 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:06:06.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.341 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:06.341 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2024-04-24T23:06:06.341 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:06:06.341 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:06.341 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:06.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.342 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:06:06.342 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:06.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.342 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:06.342 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2024-04-24T23:06:06.342 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:06:06.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.342 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:06.342 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2024-04-24T23:06:06.342 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:06:06.342 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:06.342 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:06.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.342 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:06:06.343 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:06.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.343 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:06.343 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2024-04-24T23:06:06.343 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:06:06.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.343 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:06.343 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2024-04-24T23:06:06.343 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:06:06.343 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:06.343 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:06.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.343 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:06:06.344 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:06.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.344 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:06.344 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2024-04-24T23:06:06.344 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:06:06.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.344 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:06.344 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2024-04-24T23:06:06.344 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:06:06.344 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:06.344 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:06.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.344 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/1479395426", 2024-04-24T23:06:06.345 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/1479395426", 2024-04-24T23:06:06.345 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/1479395426", 2024-04-24T23:06:06.345 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/1479395426", 2024-04-24T23:06:06.345 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:06:06.345 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:06:06.345 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:06:06.345 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:06.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.345 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:06:06.345 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:06:06.345 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:06:06.345 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:06:06.345 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:06:06.346 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:06:06.346 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:06:06.346 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:06:06.346 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:06:06.346 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 143, 2024-04-24T23:06:06.346 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:06:06.346 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:06:06.346 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:06:06.346 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:06.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.346 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:06.346 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2024-04-24T23:06:06.346 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:06:06.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.347 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:06.347 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2024-04-24T23:06:06.347 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:06:06.347 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:06.347 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:06.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.347 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:06:06.347 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:06.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.347 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:06.347 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2024-04-24T23:06:06.347 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:06:06.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.348 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:06.348 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2024-04-24T23:06:06.348 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:06:06.348 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:06.348 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:06.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.348 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:06:06.348 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:06.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.348 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:06.348 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2024-04-24T23:06:06.348 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:06:06.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.348 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:06.349 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2024-04-24T23:06:06.349 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:06:06.349 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:06.349 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:06.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.349 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:06:06.349 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:06.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.349 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:06.349 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2024-04-24T23:06:06.349 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:06:06.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.349 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:06.349 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2024-04-24T23:06:06.350 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:06:06.350 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:06.350 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:06.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.350 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/1340633718", 2024-04-24T23:06:06.350 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/1340633718", 2024-04-24T23:06:06.350 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/1340633718", 2024-04-24T23:06:06.350 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/1340633718", 2024-04-24T23:06:06.350 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:06:06.350 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:06:06.350 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:06:06.350 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:06.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.350 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.350 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:06:06.351 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:06:06.351 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:06:06.351 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:06:06.351 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:06:06.351 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:06:06.351 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:06:06.351 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:06:06.351 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:06:06.351 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 138, 2024-04-24T23:06:06.351 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:06:06.351 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:06:06.351 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:06:06.351 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:06.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.351 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:06.352 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2024-04-24T23:06:06.352 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:06:06.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.352 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:06.352 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2024-04-24T23:06:06.352 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:06:06.352 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:06.352 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:06.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.352 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:06:06.352 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:06.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.352 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:06.352 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2024-04-24T23:06:06.353 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:06:06.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.353 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:06.353 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2024-04-24T23:06:06.353 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:06:06.353 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:06.353 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:06.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.353 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:06:06.353 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:06.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.353 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:06.353 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2024-04-24T23:06:06.353 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:06:06.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.354 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:06.354 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2024-04-24T23:06:06.354 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:06:06.354 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:06.354 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:06.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.354 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:06:06.354 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:06.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.354 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:06.354 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2024-04-24T23:06:06.354 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:06:06.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.355 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.355 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:06.355 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2024-04-24T23:06:06.355 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:06:06.355 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:06.355 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:06.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.355 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/3405899334", 2024-04-24T23:06:06.355 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6820/3405899334", 2024-04-24T23:06:06.355 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6828/3405899334", 2024-04-24T23:06:06.355 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6824/3405899334", 2024-04-24T23:06:06.355 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:06:06.355 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:06:06.355 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:06:06.356 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:06.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.356 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.356 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:06:06.356 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:06:06.356 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:06:06.356 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:06:06.356 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:06:06.356 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:06:06.356 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:06:06.356 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:06:06.356 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:06:06.356 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 143, 2024-04-24T23:06:06.356 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:06:06.357 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:06:06.357 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:06:06.357 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:06.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.357 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:06.357 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2024-04-24T23:06:06.357 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:06:06.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.357 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:06.357 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2024-04-24T23:06:06.357 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:06:06.357 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:06.357 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:06.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.358 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:06:06.358 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:06.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.358 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:06.358 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2024-04-24T23:06:06.358 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:06:06.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.358 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:06.358 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2024-04-24T23:06:06.358 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:06:06.358 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:06.358 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:06.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.358 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:06:06.359 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:06.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.359 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:06.359 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2024-04-24T23:06:06.359 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:06:06.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.359 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:06.359 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2024-04-24T23:06:06.359 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:06:06.359 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:06.359 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:06.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.359 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:06:06.360 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:06.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.360 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:06.360 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2024-04-24T23:06:06.360 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:06:06.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.360 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:06.360 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2024-04-24T23:06:06.360 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:06:06.360 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:06.360 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:06.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.361 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6821/3381057190", 2024-04-24T23:06:06.361 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6825/3381057190", 2024-04-24T23:06:06.361 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/3381057190", 2024-04-24T23:06:06.361 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/3381057190", 2024-04-24T23:06:06.361 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:06:06.361 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:06:06.361 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:06:06.361 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:06.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.362 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:06:06.362 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:06:06.362 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:06:06.362 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:06:06.362 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:06:06.362 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:06:06.362 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:06:06.362 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:06:06.362 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:06:06.362 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 138, 2024-04-24T23:06:06.362 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:06:06.362 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:06:06.362 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:06:06.362 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:06.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.363 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:06.363 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6800", 2024-04-24T23:06:06.363 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:06:06.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.363 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:06.363 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6801", 2024-04-24T23:06:06.363 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:06:06.363 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:06.363 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:06.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.363 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:06:06.363 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:06.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.364 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:06.364 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6802", 2024-04-24T23:06:06.364 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:06:06.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.364 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:06.364 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6803", 2024-04-24T23:06:06.364 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:06:06.364 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:06.364 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:06.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.364 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:06:06.364 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:06.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.365 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:06.365 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6806", 2024-04-24T23:06:06.365 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:06:06.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.365 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:06.365 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6807", 2024-04-24T23:06:06.365 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:06:06.365 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:06.365 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:06.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.365 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:06:06.365 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:06.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.365 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:06.366 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6804", 2024-04-24T23:06:06.366 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:06:06.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.366 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:06.366 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6805", 2024-04-24T23:06:06.366 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:06:06.366 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:06.366 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:06.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.366 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6801/4010553979", 2024-04-24T23:06:06.366 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6803/4010553979", 2024-04-24T23:06:06.366 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6807/4010553979", 2024-04-24T23:06:06.366 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6805/4010553979", 2024-04-24T23:06:06.366 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:06:06.367 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:06:06.367 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:06:06.367 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:06.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.367 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:06:06.367 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:06:06.367 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:06:06.367 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:06:06.367 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:06:06.367 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:06:06.367 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:06:06.367 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:06:06.367 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:06:06.367 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 138, 2024-04-24T23:06:06.368 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:06:06.368 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:06:06.368 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:06:06.368 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:06.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.368 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:06.368 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6808", 2024-04-24T23:06:06.368 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:06:06.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.368 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:06.368 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6809", 2024-04-24T23:06:06.368 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:06:06.368 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:06.368 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:06.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.369 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:06:06.369 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:06.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.369 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:06.369 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6810", 2024-04-24T23:06:06.369 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:06:06.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.369 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:06.369 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6811", 2024-04-24T23:06:06.369 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:06:06.369 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:06.369 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:06.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.370 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:06:06.370 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:06.370 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.370 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:06.370 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6814", 2024-04-24T23:06:06.370 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:06:06.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.370 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.370 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:06.370 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6815", 2024-04-24T23:06:06.370 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:06:06.370 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:06.370 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:06.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.371 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:06:06.371 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:06.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.371 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:06.371 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6812", 2024-04-24T23:06:06.371 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:06:06.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.371 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:06.371 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6813", 2024-04-24T23:06:06.371 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:06:06.371 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:06.371 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:06.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.372 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6809/3794084091", 2024-04-24T23:06:06.372 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6811/3794084091", 2024-04-24T23:06:06.372 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6815/3794084091", 2024-04-24T23:06:06.372 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6813/3794084091", 2024-04-24T23:06:06.372 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:06:06.372 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:06:06.372 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:06:06.372 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:06.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.372 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:06:06.372 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:06:06.372 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:06:06.372 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:06:06.372 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:06:06.373 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:06:06.373 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:06:06.373 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:06:06.373 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:06:06.373 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 138, 2024-04-24T23:06:06.373 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:06:06.373 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:06:06.373 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:06:06.373 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:06.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.373 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:06.373 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6823", 2024-04-24T23:06:06.373 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:06:06.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.374 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:06.374 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6825", 2024-04-24T23:06:06.374 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:06:06.374 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:06.374 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:06.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.374 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:06:06.374 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:06.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.374 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:06.374 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6826", 2024-04-24T23:06:06.374 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:06:06.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.375 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.375 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:06.375 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6827", 2024-04-24T23:06:06.375 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:06:06.375 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:06.375 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:06.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.375 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:06:06.375 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:06.375 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.375 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:06.375 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6830", 2024-04-24T23:06:06.375 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:06:06.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.376 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:06.376 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6831", 2024-04-24T23:06:06.376 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:06:06.376 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:06.376 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:06.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.376 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:06:06.376 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:06.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.376 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:06.376 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6828", 2024-04-24T23:06:06.376 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:06:06.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.377 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:06.377 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6829", 2024-04-24T23:06:06.377 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:06:06.377 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:06.377 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:06.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.377 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6825/1348640258", 2024-04-24T23:06:06.377 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6827/1348640258", 2024-04-24T23:06:06.377 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6831/1348640258", 2024-04-24T23:06:06.377 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6829/1348640258", 2024-04-24T23:06:06.377 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:06:06.377 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:06:06.377 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:06:06.378 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:06.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.378 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:06:06.378 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:06:06.378 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:06:06.378 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:06:06.378 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:06:06.378 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:06:06.378 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:06:06.378 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:06:06.378 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:06:06.378 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 138, 2024-04-24T23:06:06.378 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:06:06.379 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:06:06.379 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:06:06.379 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:06.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.379 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:06.379 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6816", 2024-04-24T23:06:06.379 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:06:06.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.379 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:06.379 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6817", 2024-04-24T23:06:06.379 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:06:06.379 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:06.379 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:06.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.380 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:06:06.380 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:06.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.380 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:06.380 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6818", 2024-04-24T23:06:06.380 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:06:06.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.380 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:06.380 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6819", 2024-04-24T23:06:06.380 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:06:06.380 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:06.380 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:06.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.381 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:06:06.381 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:06.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.381 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:06.381 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6822", 2024-04-24T23:06:06.381 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:06:06.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.381 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:06.381 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6824", 2024-04-24T23:06:06.381 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:06:06.381 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:06.381 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:06.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.382 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:06:06.382 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:06.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.382 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:06.382 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6820", 2024-04-24T23:06:06.382 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:06:06.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.382 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:06.382 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6821", 2024-04-24T23:06:06.382 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:06:06.382 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:06.382 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:06.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.382 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6817/3250005122", 2024-04-24T23:06:06.383 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6819/3250005122", 2024-04-24T23:06:06.383 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6824/3250005122", 2024-04-24T23:06:06.383 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6821/3250005122", 2024-04-24T23:06:06.383 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:06:06.383 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:06:06.383 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:06:06.383 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:06.383 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:06.383 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:06:06.383 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2024-04-24T23:06:06.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.383 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:06:06.383 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:06:06.383 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:06:06.384 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:06:06.384 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:06:06.384 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:06:06.384 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.156184+0000", 2024-04-24T23:06:06.384 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:06:06.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.384 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:06:06.384 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:06:06.384 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:06:06.384 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:06:06.384 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:06:06.384 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:06:06.384 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240774+0000", 2024-04-24T23:06:06.384 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:06:06.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.385 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:06:06.385 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:06:06.385 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:06:06.385 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:06:06.385 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:06:06.385 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:06:06.385 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240949+0000", 2024-04-24T23:06:06.385 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:06:06.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.385 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:06:06.385 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:06:06.385 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:06:06.386 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:06:06.386 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:06:06.386 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:06:06.386 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.171672+0000", 2024-04-24T23:06:06.386 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:06:06.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.386 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:06:06.386 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:06:06.386 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:06:06.386 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:06:06.386 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:06:06.386 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:06:06.386 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.220379+0000", 2024-04-24T23:06:06.387 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:06:06.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.387 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:06:06.387 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:06:06.387 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:06:06.387 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:06:06.387 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:06:06.387 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:06:06.387 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.204963+0000", 2024-04-24T23:06:06.387 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:06:06.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.388 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:06:06.388 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:06:06.388 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:06:06.388 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:06:06.388 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:06:06.388 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:06:06.388 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.239748+0000", 2024-04-24T23:06:06.388 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:06:06.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.388 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:06:06.388 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:06:06.388 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:06:06.388 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:06:06.388 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:06:06.389 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:06:06.389 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.154104+0000", 2024-04-24T23:06:06.389 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:06:06.389 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:06.389 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:06:06.389 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2024-04-24T23:06:06.389 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [ 2024-04-24T23:06:06.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.389 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "16.17", 2024-04-24T23:06:06.389 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:06:06.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.389 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 4, 2024-04-24T23:06:06.389 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 3 2024-04-24T23:06:06.389 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:06.390 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:06.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.390 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "17.8", 2024-04-24T23:06:06.390 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:06:06.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.390 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 6, 2024-04-24T23:06:06.390 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 5 2024-04-24T23:06:06.390 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:06.390 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:06.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.390 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "17.3f", 2024-04-24T23:06:06.390 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:06:06.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:06.391 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 6, 2024-04-24T23:06:06.391 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 1 2024-04-24T23:06:06.391 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:06.391 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:06.391 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:06.391 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:06:06.391 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_primaries": [], 2024-04-24T23:06:06.391 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2024-04-24T23:06:06.391 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2024-04-24T23:06:06.391 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklist": {}, 2024-04-24T23:06:06.391 INFO:teuthology.orchestra.run.smithi032.stdout: "range_blocklist": {}, 2024-04-24T23:06:06.391 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2024-04-24T23:06:06.391 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2024-04-24T23:06:06.392 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2024-04-24T23:06:06.392 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2024-04-24T23:06:06.392 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2024-04-24T23:06:06.392 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2024-04-24T23:06:06.392 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2024-04-24T23:06:06.392 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:06.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:06.392 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2024-04-24T23:06:06.392 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2024-04-24T23:06:06.392 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2024-04-24T23:06:06.392 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2024-04-24T23:06:06.392 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {}, 2024-04-24T23:06:06.392 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode": { 2024-04-24T23:06:06.392 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_enabled": false, 2024-04-24T23:06:06.392 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_bucket_count": 0, 2024-04-24T23:06:06.393 INFO:teuthology.orchestra.run.smithi032.stdout: "degraded_stretch_mode": 0, 2024-04-24T23:06:06.393 INFO:teuthology.orchestra.run.smithi032.stdout: "recovering_stretch_mode": 0, 2024-04-24T23:06:06.393 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_bucket": 0 2024-04-24T23:06:06.393 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:06.393 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:06:06.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.319+0000 7f247b2bd640 1 -- 172.21.15.32:0/588298410 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f246006be30 msgr2=0x7f246006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:06.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.319+0000 7f247b2bd640 1 --2- 172.21.15.32:0/588298410 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f246006be30 0x7f246006e2f0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f246c009aa0 tx=0x7f246c009190 comp rx=0 tx=0).stop 2024-04-24T23:06:06.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.319+0000 7f247b2bd640 1 -- 172.21.15.32:0/588298410 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f247415e9b0 msgr2=0x7f24741f7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:06.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.319+0000 7f247b2bd640 1 --2- 172.21.15.32:0/588298410 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f247415e9b0 0x7f24741f7a10 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f246400c7a0 tx=0x7f246400cc60 comp rx=0 tx=0).stop 2024-04-24T23:06:06.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.319+0000 7f247b2bd640 1 -- 172.21.15.32:0/588298410 shutdown_connections 2024-04-24T23:06:06.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.319+0000 7f247b2bd640 1 --2- 172.21.15.32:0/588298410 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f246006be30 0x7f246006e2f0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:06.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.319+0000 7f247b2bd640 1 --2- 172.21.15.32:0/588298410 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f247415e9b0 0x7f24741f7a10 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:06.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.319+0000 7f247b2bd640 1 --2- 172.21.15.32:0/588298410 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24740d4210 0x7f24740ce140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:06.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.319+0000 7f247b2bd640 1 --2- 172.21.15.32:0/588298410 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f24740ba0e0 0x7f24741f4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:06.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.319+0000 7f247b2bd640 1 -- 172.21.15.32:0/588298410 >> 172.21.15.32:0/588298410 conn(0x7f24740d11d0 msgr2=0x7f2474071fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:06.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.319+0000 7f247b2bd640 1 -- 172.21.15.32:0/588298410 shutdown_connections 2024-04-24T23:06:06.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:06.319+0000 7f247b2bd640 1 -- 172.21.15.32:0/588298410 wait complete. 2024-04-24T23:06:06.396 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-04-24T23:06:07.173 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:06:07.176 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:06:11.397 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-24T23:06:11.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.510+0000 7fb37ee7d640 1 Processor -- start 2024-04-24T23:06:11.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.510+0000 7fb37ee7d640 1 -- start start 2024-04-24T23:06:11.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.511+0000 7fb37ee7d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb37806c5f0 0x7fb37806c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:11.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.511+0000 7fb37ee7d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3780b6430 con 0x7fb3780b44d0 2024-04-24T23:06:11.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.511+0000 7fb37ee7d640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb378156f80 con 0x7fb3780af890 2024-04-24T23:06:11.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.511+0000 7fb37ee7d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fb3780b6150 con 0x7fb37806c5f0 2024-04-24T23:06:11.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.511+0000 7fb377fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb3780b44d0 0x7fb3780b5890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41152/0 (socket says 172.21.15.32:41152) 2024-04-24T23:06:11.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.511+0000 7fb377fff640 1 -- 172.21.15.32:0/3974230325 learned_addr learned my addr 172.21.15.32:0/3974230325 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:06:11.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.511+0000 7fb37cc19640 1 --2- 172.21.15.32:0/3974230325 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb37806c5f0 0x7fb37806c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:11.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.511+0000 7fb3777fe640 1 -- 172.21.15.32:0/3974230325 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3388633236 0 0) 0x7fb3780b6430 con 0x7fb3780b44d0 2024-04-24T23:06:11.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.511+0000 7fb3777fe640 1 -- 172.21.15.32:0/3974230325 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb368003520 con 0x7fb3780b44d0 2024-04-24T23:06:11.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.512+0000 7fb3777fe640 1 -- 172.21.15.32:0/3974230325 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4256638786 0 0) 0x7fb378156f80 con 0x7fb3780af890 2024-04-24T23:06:11.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.512+0000 7fb3777fe640 1 -- 172.21.15.32:0/3974230325 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb3780b6430 con 0x7fb3780af890 2024-04-24T23:06:11.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.512+0000 7fb3777fe640 1 -- 172.21.15.32:0/3974230325 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1196279262 0 0) 0x7fb368003520 con 0x7fb3780b44d0 2024-04-24T23:06:11.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.512+0000 7fb3777fe640 1 -- 172.21.15.32:0/3974230325 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb37806c5f0 msgr2=0x7fb37806c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:11.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.512+0000 7fb3777fe640 1 --2- 172.21.15.32:0/3974230325 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb37806c5f0 0x7fb37806c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:11.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.512+0000 7fb3777fe640 1 -- 172.21.15.32:0/3974230325 >> v1:172.21.15.121:6789/0 conn(0x7fb3780af890 legacy=0x7fb37815cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:11.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.512+0000 7fb3777fe640 1 -- 172.21.15.32:0/3974230325 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3780b39f0 con 0x7fb3780b44d0 2024-04-24T23:06:11.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.512+0000 7fb3777fe640 1 -- 172.21.15.32:0/3974230325 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7fb3640015f0 con 0x7fb3780b44d0 2024-04-24T23:06:11.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.512+0000 7fb3777fe640 1 -- 172.21.15.32:0/3974230325 >> v1:172.21.15.32:6789/0 conn(0x7fb3780b44d0 legacy=0x7fb3780b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:11.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.512+0000 7fb3777fe640 1 --2- 172.21.15.32:0/3974230325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb368003b70 0x7fb368003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:11.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.512+0000 7fb3777fe640 1 --2- 172.21.15.32:0/3974230325 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb368004800 0x7fb368004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:11.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.512+0000 7fb3777fe640 1 --2- 172.21.15.32:0/3974230325 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb37806c5f0 0x7fb368005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:11.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.512+0000 7fb3777fe640 1 -- 172.21.15.32:0/3974230325 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb3780b6150 con 0x7fb368003b70 2024-04-24T23:06:11.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.512+0000 7fb3777fe640 1 -- 172.21.15.32:0/3974230325 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb36800bb50 con 0x7fb368004800 2024-04-24T23:06:11.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.512+0000 7fb3777fe640 1 -- 172.21.15.32:0/3974230325 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fb36800bd20 con 0x7fb37806c5f0 2024-04-24T23:06:11.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.512+0000 7fb37cc19640 1 --2- 172.21.15.32:0/3974230325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb368003b70 0x7fb368003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:11.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.512+0000 7fb377fff640 1 --2- 172.21.15.32:0/3974230325 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb368004800 0x7fb368004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:11.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.512+0000 7fb37d41a640 1 --2- 172.21.15.32:0/3974230325 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb37806c5f0 0x7fb368005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:11.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.512+0000 7fb37cc19640 1 -- 172.21.15.32:0/3974230325 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb37806c5f0 msgr2=0x7fb368005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:11.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.512+0000 7fb37cc19640 1 --2- 172.21.15.32:0/3974230325 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb37806c5f0 0x7fb368005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:11.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.512+0000 7fb37cc19640 1 -- 172.21.15.32:0/3974230325 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb368004800 msgr2=0x7fb368004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:11.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.512+0000 7fb37cc19640 1 --2- 172.21.15.32:0/3974230325 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb368004800 0x7fb368004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:11.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.512+0000 7fb37cc19640 1 -- 172.21.15.32:0/3974230325 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb368003520 con 0x7fb368003b70 2024-04-24T23:06:11.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.512+0000 7fb37d41a640 1 --2- 172.21.15.32:0/3974230325 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb37806c5f0 0x7fb368005220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:06:11.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.513+0000 7fb37cc19640 1 --2- 172.21.15.32:0/3974230325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb368003b70 0x7fb368003f70 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fb36c009af0 tx=0x7fb368005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f54c51b175e13f8f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:11.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.513+0000 7fb3777fe640 1 -- 172.21.15.32:0/3974230325 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36c036070 con 0x7fb368003b70 2024-04-24T23:06:11.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.513+0000 7fb3777fe640 1 -- 172.21.15.32:0/3974230325 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb36c031400 con 0x7fb368003b70 2024-04-24T23:06:11.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.513+0000 7fb3777fe640 1 -- 172.21.15.32:0/3974230325 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36c009690 con 0x7fb368003b70 2024-04-24T23:06:11.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.514+0000 7fb37ee7d640 1 -- 172.21.15.32:0/3974230325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb368003b70 msgr2=0x7fb368003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:11.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.514+0000 7fb37ee7d640 1 --2- 172.21.15.32:0/3974230325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb368003b70 0x7fb368003f70 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fb36c009af0 tx=0x7fb368005830 comp rx=0 tx=0).stop 2024-04-24T23:06:11.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.514+0000 7fb37d41a640 1 -- 172.21.15.32:0/3974230325 reap_dead start 2024-04-24T23:06:11.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.514+0000 7fb37ee7d640 1 -- 172.21.15.32:0/3974230325 shutdown_connections 2024-04-24T23:06:11.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.514+0000 7fb37ee7d640 1 -- 172.21.15.32:0/3974230325 >> 172.21.15.32:0/3974230325 conn(0x7fb37806b110 msgr2=0x7fb37806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:11.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.514+0000 7fb37ee7d640 1 -- 172.21.15.32:0/3974230325 shutdown_connections 2024-04-24T23:06:11.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.514+0000 7fb37ee7d640 1 -- 172.21.15.32:0/3974230325 wait complete. 2024-04-24T23:06:11.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.515+0000 7fb37ee7d640 1 Processor -- start 2024-04-24T23:06:11.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.515+0000 7fb37ee7d640 1 -- start start 2024-04-24T23:06:11.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.516+0000 7fb37ee7d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb368003b70 0x7fb3781507c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:11.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.516+0000 7fb37ee7d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb37815a1e0 0x7fb3781f5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:11.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.516+0000 7fb37ee7d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3781522b0 0x7fb378152730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:11.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.516+0000 7fb37ee7d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb37806c9d0 con 0x7fb37815a1e0 2024-04-24T23:06:11.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.516+0000 7fb37ee7d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb3780b3fb0 con 0x7fb3781522b0 2024-04-24T23:06:11.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.516+0000 7fb37ee7d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fb37806cb50 con 0x7fb368003b70 2024-04-24T23:06:11.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.516+0000 7fb377fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb37815a1e0 0x7fb3781f5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:11.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.516+0000 7fb377fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb37815a1e0 0x7fb3781f5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34720/0 (socket says 172.21.15.32:34720) 2024-04-24T23:06:11.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.516+0000 7fb377fff640 1 -- 172.21.15.32:0/330828543 learned_addr learned my addr 172.21.15.32:0/330828543 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:11.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.516+0000 7fb37cc19640 1 --2- 172.21.15.32:0/330828543 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb368003b70 0x7fb3781507c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:11.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.516+0000 7fb37d41a640 1 --2- 172.21.15.32:0/330828543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3781522b0 0x7fb378152730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:11.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.516+0000 7fb377fff640 1 -- 172.21.15.32:0/330828543 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb368003b70 msgr2=0x7fb3781507c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:11.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.516+0000 7fb377fff640 1 --2- 172.21.15.32:0/330828543 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb368003b70 0x7fb3781507c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:11.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.516+0000 7fb377fff640 1 -- 172.21.15.32:0/330828543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3781522b0 msgr2=0x7fb378152730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:11.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.516+0000 7fb377fff640 1 --2- 172.21.15.32:0/330828543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3781522b0 0x7fb378152730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:11.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.516+0000 7fb377fff640 1 -- 172.21.15.32:0/330828543 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb378153010 con 0x7fb37815a1e0 2024-04-24T23:06:11.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.517+0000 7fb377fff640 1 --2- 172.21.15.32:0/330828543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb37815a1e0 0x7fb3781f5480 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fb364004f50 tx=0x7fb364007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:11.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.517+0000 7fb3757fa640 1 -- 172.21.15.32:0/330828543 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36401a070 con 0x7fb37815a1e0 2024-04-24T23:06:11.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.517+0000 7fb3757fa640 1 -- 172.21.15.32:0/330828543 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb36400f6d0 con 0x7fb37815a1e0 2024-04-24T23:06:11.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.517+0000 7fb37ee7d640 1 -- 172.21.15.32:0/330828543 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3781532f0 con 0x7fb37815a1e0 2024-04-24T23:06:11.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.517+0000 7fb3757fa640 1 -- 172.21.15.32:0/330828543 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36400faa0 con 0x7fb37815a1e0 2024-04-24T23:06:11.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.517+0000 7fb37ee7d640 1 -- 172.21.15.32:0/330828543 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3781538b0 con 0x7fb37815a1e0 2024-04-24T23:06:11.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.518+0000 7fb37ee7d640 1 -- 172.21.15.32:0/330828543 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3781578a0 con 0x7fb37815a1e0 2024-04-24T23:06:11.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.518+0000 7fb3757fa640 1 -- 172.21.15.32:0/330828543 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb36400fc40 con 0x7fb37815a1e0 2024-04-24T23:06:11.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.519+0000 7fb3757fa640 1 --2- 172.21.15.32:0/330828543 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb35c06be60 0x7fb35c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:11.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.519+0000 7fb3757fa640 1 -- 172.21.15.32:0/330828543 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5796+0+0 (secure 0 0 0) 0x7fb36408a380 con 0x7fb37815a1e0 2024-04-24T23:06:11.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.519+0000 7fb37cc19640 1 --2- 172.21.15.32:0/330828543 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb35c06be60 0x7fb35c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:11.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.520+0000 7fb37cc19640 1 --2- 172.21.15.32:0/330828543 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb35c06be60 0x7fb35c06e320 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fb36c033000 tx=0x7fb36c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:11.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.523+0000 7fb3757fa640 1 -- 172.21.15.32:0/330828543 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb3781578a0 con 0x7fb37815a1e0 2024-04-24T23:06:11.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.706+0000 7fb37ee7d640 1 -- 172.21.15.32:0/330828543 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7fb378068430 con 0x7fb37815a1e0 2024-04-24T23:06:11.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.707+0000 7fb3757fa640 1 -- 172.21.15.32:0/330828543 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v145) ==== 81+0+34741 (secure 0 0 0) 0x7fb364058680 con 0x7fb37815a1e0 2024-04-24T23:06:11.709 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:06:11.709 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:06:11.709 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 145, 2024-04-24T23:06:11.710 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", 2024-04-24T23:06:11.710 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2024-04-24T22:58:46.964851+0000", 2024-04-24T23:06:11.710 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2024-04-24T23:06:06.811191+0000", 2024-04-24T23:06:11.710 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2024-04-24T22:58:52.048787+0000", 2024-04-24T23:06:11.710 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2024-04-24T22:58:49.045609+0000", 2024-04-24T23:06:11.710 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-24T23:06:11.710 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2024-04-24T23:06:11.710 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2024-04-24T23:06:11.710 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2024-04-24T23:06:11.710 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2024-04-24T23:06:11.711 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2024-04-24T23:06:11.711 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2024-04-24T23:06:11.711 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:06:11.711 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 4, 2024-04-24T23:06:11.711 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.69999998807907104, 2024-04-24T23:06:11.711 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-24T23:06:11.711 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-24T23:06:11.711 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2024-04-24T23:06:11.711 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 17, 2024-04-24T23:06:11.712 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2024-04-24T23:06:11.712 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "luminous", 2024-04-24T23:06:11.712 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "luminous", 2024-04-24T23:06:11.712 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "squid", 2024-04-24T23:06:11.712 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_crimson": false, 2024-04-24T23:06:11.712 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2024-04-24T23:06:11.712 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.712 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2024-04-24T23:06:11.712 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": ".mgr", 2024-04-24T23:06:11.712 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T22:58:53.171408+0000", 2024-04-24T23:06:11.712 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:06:11.713 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:06:11.713 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:06:11.713 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:06:11.713 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:06:11.713 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:06:11.713 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:06:11.713 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:06:11.713 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:06:11.713 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:06:11.713 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:06:11.713 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:06:11.713 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2024-04-24T23:06:11.714 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2024-04-24T23:06:11.714 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2024-04-24T23:06:11.714 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2024-04-24T23:06:11.714 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2024-04-24T23:06:11.714 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:06:11.714 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:06:11.714 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:06:11.715 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:06:11.715 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:06:11.715 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:06:11.715 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:06:11.715 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.715 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "113", 2024-04-24T23:06:11.715 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:06:11.716 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:06:11.716 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:06:11.716 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:06:11.716 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:06:11.716 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:06:11.716 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:06:11.716 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:06:11.716 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:06:11.716 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:06:11.716 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:06:11.717 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:06:11.717 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:06:11.717 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:06:11.717 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:06:11.718 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:06:11.718 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:06:11.718 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:06:11.718 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:06:11.718 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:06:11.718 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:06:11.718 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:06:11.718 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:06:11.718 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:06:11.718 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:06:11.719 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:06:11.719 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.719 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:06:11.719 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:06:11.719 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:06:11.719 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:06:11.719 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:06:11.719 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:06:11.719 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:06:11.719 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:06:11.720 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:06:11.720 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:06:11.720 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:06:11.720 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:06:11.720 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_max": 32, 2024-04-24T23:06:11.720 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2024-04-24T23:06:11.721 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.721 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:06:11.721 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": {} 2024-04-24T23:06:11.721 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.721 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:06:11.721 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:06:11.721 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 8, 2024-04-24T23:06:11.722 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 8, 2024-04-24T23:06:11.722 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.25, 2024-04-24T23:06:11.722 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-04-24T23:06:11.722 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-04-24T23:06:11.722 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:06:11.722 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:06:11.722 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:06:11.722 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:11.722 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.722 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.723 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 16, 2024-04-24T23:06:11.723 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2024-04-24T23:06:11.723 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:05:32.064234+0000", 2024-04-24T23:06:11.723 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:06:11.723 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:06:11.723 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:06:11.723 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:06:11.723 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:06:11.723 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:06:11.723 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:06:11.723 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:06:11.723 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:06:11.723 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:06:11.723 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:06:11.724 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:06:11.724 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 32, 2024-04-24T23:06:11.724 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 32, 2024-04-24T23:06:11.724 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 32, 2024-04-24T23:06:11.724 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 32, 2024-04-24T23:06:11.724 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 32, 2024-04-24T23:06:11.724 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:06:11.724 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:06:11.724 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:06:11.725 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:06:11.725 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:06:11.725 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:06:11.725 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:06:11.725 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.725 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "139", 2024-04-24T23:06:11.725 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:06:11.725 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:06:11.725 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:06:11.725 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:06:11.726 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:06:11.726 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:06:11.726 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:06:11.726 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:06:11.726 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:06:11.726 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:06:11.726 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:06:11.726 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:06:11.726 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:06:11.727 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:06:11.727 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:06:11.727 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:06:11.727 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:06:11.727 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:06:11.727 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:06:11.727 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:06:11.727 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:06:11.727 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:06:11.727 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:06:11.727 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:06:11.727 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:06:11.728 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:06:11.728 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.728 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:06:11.728 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:06:11.728 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:06:11.728 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:06:11.728 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:06:11.728 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:06:11.728 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:06:11.728 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:06:11.728 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:06:11.729 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:06:11.729 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:06:11.729 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:06:11.729 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2024-04-24T23:06:11.729 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2024-04-24T23:06:11.729 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2024-04-24T23:06:11.729 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.729 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:06:11.729 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:06:11.729 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2024-04-24T23:06:11.730 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:11.730 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.730 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:06:11.730 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:06:11.730 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 2, 2024-04-24T23:06:11.730 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 2, 2024-04-24T23:06:11.731 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:06:11.731 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-04-24T23:06:11.731 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-04-24T23:06:11.731 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:06:11.731 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:06:11.731 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:06:11.731 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:11.731 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.731 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.731 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 17, 2024-04-24T23:06:11.731 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2024-04-24T23:06:11.731 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:05:32.760777+0000", 2024-04-24T23:06:11.732 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:06:11.732 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:06:11.732 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:06:11.732 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:06:11.732 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:06:11.732 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:06:11.732 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:06:11.732 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:06:11.732 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:06:11.732 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:06:11.732 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:06:11.733 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:06:11.733 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 64, 2024-04-24T23:06:11.733 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 64, 2024-04-24T23:06:11.733 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 64, 2024-04-24T23:06:11.733 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 64, 2024-04-24T23:06:11.733 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 64, 2024-04-24T23:06:11.733 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:06:11.733 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:06:11.733 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:06:11.733 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:06:11.733 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:06:11.733 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:06:11.734 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:06:11.734 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.734 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "145", 2024-04-24T23:06:11.734 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:06:11.734 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:06:11.734 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:06:11.734 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:06:11.734 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:06:11.734 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:06:11.734 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:06:11.734 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:06:11.735 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:06:11.735 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 33554432, 2024-04-24T23:06:11.735 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:06:11.735 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:06:11.735 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:06:11.735 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:06:11.735 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:06:11.735 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:06:11.735 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:06:11.735 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:06:11.735 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:06:11.735 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:06:11.736 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:06:11.736 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:06:11.736 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:06:11.736 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:06:11.736 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:06:11.736 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:06:11.736 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.736 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:06:11.736 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:06:11.736 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:06:11.737 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:06:11.737 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:06:11.737 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:06:11.737 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:06:11.737 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:06:11.737 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:06:11.737 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:06:11.737 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:06:11.737 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:06:11.737 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 64, 2024-04-24T23:06:11.737 INFO:teuthology.orchestra.run.smithi032.stdout: "target_size_ratio": 0.89999997615814209 2024-04-24T23:06:11.738 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.738 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:06:11.738 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:06:11.738 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2024-04-24T23:06:11.738 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:11.738 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.738 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:06:11.738 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:06:11.738 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.3799999952316284, 2024-04-24T23:06:11.738 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.3799999952316284, 2024-04-24T23:06:11.738 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:06:11.739 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.3799999952316284, 2024-04-24T23:06:11.739 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.3799999952316284, 2024-04-24T23:06:11.739 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:06:11.739 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:06:11.739 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:06:11.739 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:11.739 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:11.739 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:06:11.739 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-04-24T23:06:11.739 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.739 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:06:11.740 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:06:11.740 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:06:11.740 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:06:11.740 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:06:11.740 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:06:11.740 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:06:11.740 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:06:11.740 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:06:11.740 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 143, 2024-04-24T23:06:11.740 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:06:11.741 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:06:11.741 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:06:11.741 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:11.741 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.741 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:11.741 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2024-04-24T23:06:11.741 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:06:11.741 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.741 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.741 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:11.741 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2024-04-24T23:06:11.741 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:06:11.741 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:11.742 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:11.742 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.742 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:06:11.742 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:11.742 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.742 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:11.742 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2024-04-24T23:06:11.742 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:06:11.742 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.742 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.742 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:11.742 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2024-04-24T23:06:11.742 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:06:11.742 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:11.743 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:11.743 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.743 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:06:11.743 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:11.743 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.743 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:11.743 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2024-04-24T23:06:11.743 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:06:11.743 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.743 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.744 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:11.744 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2024-04-24T23:06:11.744 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:06:11.744 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:11.744 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:11.744 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.744 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:06:11.744 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:11.744 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.744 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:11.744 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2024-04-24T23:06:11.744 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:06:11.744 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.745 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.745 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:11.745 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2024-04-24T23:06:11.745 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:06:11.745 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:11.745 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:11.745 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.745 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/1479395426", 2024-04-24T23:06:11.745 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/1479395426", 2024-04-24T23:06:11.745 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/1479395426", 2024-04-24T23:06:11.745 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/1479395426", 2024-04-24T23:06:11.745 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:06:11.745 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:06:11.746 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:06:11.746 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:11.746 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.746 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.746 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:06:11.746 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:06:11.746 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:06:11.746 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:06:11.746 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:06:11.746 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:06:11.747 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:06:11.747 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:06:11.747 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:06:11.747 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 143, 2024-04-24T23:06:11.747 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:06:11.747 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:06:11.747 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:06:11.747 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:11.747 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.747 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:11.747 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2024-04-24T23:06:11.748 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:06:11.748 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.748 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.748 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:11.748 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2024-04-24T23:06:11.748 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:06:11.748 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:11.748 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:11.748 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.748 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:06:11.748 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:11.748 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.749 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:11.749 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2024-04-24T23:06:11.749 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:06:11.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.749 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.749 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:11.749 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2024-04-24T23:06:11.749 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:06:11.749 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:11.749 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:11.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.749 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:06:11.750 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:11.750 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.750 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:11.750 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2024-04-24T23:06:11.750 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:06:11.750 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.750 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.750 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:11.750 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2024-04-24T23:06:11.751 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:06:11.751 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:11.751 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:11.751 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.751 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:06:11.751 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:11.751 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.751 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:11.751 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2024-04-24T23:06:11.751 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:06:11.752 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.752 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.752 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:11.752 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2024-04-24T23:06:11.752 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:06:11.752 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:11.753 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:11.753 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.753 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/1340633718", 2024-04-24T23:06:11.753 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/1340633718", 2024-04-24T23:06:11.753 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/1340633718", 2024-04-24T23:06:11.753 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/1340633718", 2024-04-24T23:06:11.753 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:06:11.753 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:06:11.753 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:06:11.754 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:11.754 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.754 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.754 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:06:11.754 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:06:11.754 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:06:11.754 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:06:11.754 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:06:11.754 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:06:11.754 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:06:11.754 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:06:11.754 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:06:11.755 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 138, 2024-04-24T23:06:11.755 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:06:11.755 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:06:11.755 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:06:11.755 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:11.755 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.755 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:11.755 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2024-04-24T23:06:11.755 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:06:11.755 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.755 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.755 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:11.756 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2024-04-24T23:06:11.756 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:06:11.756 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:11.756 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:11.756 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.756 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:06:11.756 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:11.756 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.756 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:11.756 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2024-04-24T23:06:11.756 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:06:11.756 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.757 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.757 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:11.757 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2024-04-24T23:06:11.757 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:06:11.757 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:11.757 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:11.757 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.757 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:06:11.757 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:11.757 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.757 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:11.757 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2024-04-24T23:06:11.758 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:06:11.758 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.758 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.758 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:11.758 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2024-04-24T23:06:11.758 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:06:11.758 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:11.758 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:11.758 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.758 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:06:11.758 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:11.758 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.759 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:11.759 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2024-04-24T23:06:11.759 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:06:11.759 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.759 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.759 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:11.759 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2024-04-24T23:06:11.759 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:06:11.759 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:11.759 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:11.759 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.759 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/3405899334", 2024-04-24T23:06:11.759 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6820/3405899334", 2024-04-24T23:06:11.759 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6828/3405899334", 2024-04-24T23:06:11.760 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6824/3405899334", 2024-04-24T23:06:11.760 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:06:11.760 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:06:11.760 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:06:11.760 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:11.760 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.760 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.760 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:06:11.760 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:06:11.760 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:06:11.760 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:06:11.761 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:06:11.761 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:06:11.761 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:06:11.761 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:06:11.761 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:06:11.761 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 143, 2024-04-24T23:06:11.761 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:06:11.761 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:06:11.761 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:06:11.761 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:11.761 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.761 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:11.761 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2024-04-24T23:06:11.762 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:06:11.762 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.762 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.762 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:11.762 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2024-04-24T23:06:11.762 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:06:11.762 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:11.762 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:11.762 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.762 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:06:11.763 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:11.763 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.763 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:11.763 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2024-04-24T23:06:11.763 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:06:11.763 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.763 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.763 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:11.763 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2024-04-24T23:06:11.763 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:06:11.763 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:11.763 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:11.763 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.764 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:06:11.764 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:11.764 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.764 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:11.764 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2024-04-24T23:06:11.764 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:06:11.764 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.764 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.764 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:11.764 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2024-04-24T23:06:11.764 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:06:11.764 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:11.765 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:11.765 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.765 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:06:11.765 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:11.765 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.765 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:11.765 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2024-04-24T23:06:11.765 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:06:11.765 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.765 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.765 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:11.765 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2024-04-24T23:06:11.766 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:06:11.766 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:11.766 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:11.766 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.766 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6821/3381057190", 2024-04-24T23:06:11.766 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6825/3381057190", 2024-04-24T23:06:11.766 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/3381057190", 2024-04-24T23:06:11.766 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/3381057190", 2024-04-24T23:06:11.766 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:06:11.766 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:06:11.766 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:06:11.766 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:11.767 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.767 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.767 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:06:11.767 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:06:11.767 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:06:11.767 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:06:11.767 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:06:11.767 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:06:11.767 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:06:11.767 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:06:11.768 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:06:11.768 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 138, 2024-04-24T23:06:11.768 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:06:11.768 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:06:11.768 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:06:11.768 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:11.768 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.768 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:11.768 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6800", 2024-04-24T23:06:11.768 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:06:11.768 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.769 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.769 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:11.769 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6801", 2024-04-24T23:06:11.769 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:06:11.769 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:11.769 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:11.769 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.769 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:06:11.769 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:11.770 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.770 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:11.770 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6802", 2024-04-24T23:06:11.770 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:06:11.770 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.770 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.770 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:11.770 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6803", 2024-04-24T23:06:11.770 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:06:11.770 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:11.770 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:11.770 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.770 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:06:11.770 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:11.771 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.771 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:11.771 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6806", 2024-04-24T23:06:11.771 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:06:11.771 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.771 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.771 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:11.771 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6807", 2024-04-24T23:06:11.771 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:06:11.771 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:11.771 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:11.771 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.771 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:06:11.771 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:11.771 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.772 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:11.772 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6804", 2024-04-24T23:06:11.772 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:06:11.772 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.772 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.772 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:11.772 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6805", 2024-04-24T23:06:11.772 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:06:11.772 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:11.772 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:11.772 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.772 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6801/4010553979", 2024-04-24T23:06:11.772 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6803/4010553979", 2024-04-24T23:06:11.772 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6807/4010553979", 2024-04-24T23:06:11.773 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6805/4010553979", 2024-04-24T23:06:11.773 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:06:11.773 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:06:11.773 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:06:11.773 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:11.773 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.773 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.773 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:06:11.773 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:06:11.773 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:06:11.773 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:06:11.773 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:06:11.773 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:06:11.773 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:06:11.774 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:06:11.774 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:06:11.774 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 138, 2024-04-24T23:06:11.774 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:06:11.774 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:06:11.774 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:06:11.774 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:11.774 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.774 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:11.774 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6808", 2024-04-24T23:06:11.774 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:06:11.774 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.774 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.774 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:11.775 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6809", 2024-04-24T23:06:11.775 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:06:11.775 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:11.775 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:11.775 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.775 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:06:11.775 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:11.775 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.775 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:11.775 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6810", 2024-04-24T23:06:11.775 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:06:11.775 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.775 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.775 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:11.775 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6811", 2024-04-24T23:06:11.776 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:06:11.776 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:11.776 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:11.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.776 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:06:11.776 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:11.776 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.776 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:11.776 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6814", 2024-04-24T23:06:11.776 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:06:11.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.776 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.776 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:11.776 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6815", 2024-04-24T23:06:11.777 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:06:11.777 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:11.777 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:11.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.777 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:06:11.777 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:11.777 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.777 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:11.777 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6812", 2024-04-24T23:06:11.777 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:06:11.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.777 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.777 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:11.777 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6813", 2024-04-24T23:06:11.777 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:06:11.777 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:11.778 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:11.778 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.778 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6809/3794084091", 2024-04-24T23:06:11.778 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6811/3794084091", 2024-04-24T23:06:11.778 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6815/3794084091", 2024-04-24T23:06:11.778 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6813/3794084091", 2024-04-24T23:06:11.778 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:06:11.778 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:06:11.778 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:06:11.778 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:11.778 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.778 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.778 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:06:11.779 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:06:11.779 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:06:11.779 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:06:11.779 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:06:11.779 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:06:11.779 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:06:11.779 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:06:11.779 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:06:11.779 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 138, 2024-04-24T23:06:11.779 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:06:11.779 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:06:11.779 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:06:11.779 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:11.779 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.779 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:11.780 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6823", 2024-04-24T23:06:11.780 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:06:11.780 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.780 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.780 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:11.780 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6825", 2024-04-24T23:06:11.780 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:06:11.780 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:11.780 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:11.780 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.780 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:06:11.780 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:11.780 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.780 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:11.780 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6826", 2024-04-24T23:06:11.781 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:06:11.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.781 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.781 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:11.781 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6827", 2024-04-24T23:06:11.781 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:06:11.781 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:11.781 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:11.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.781 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:06:11.781 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:11.781 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.781 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:11.781 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6830", 2024-04-24T23:06:11.781 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:06:11.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.782 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.782 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:11.782 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6831", 2024-04-24T23:06:11.782 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:06:11.782 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:11.782 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:11.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.782 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:06:11.782 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:11.782 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.782 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:11.782 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6828", 2024-04-24T23:06:11.782 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:06:11.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.782 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.783 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:11.783 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6829", 2024-04-24T23:06:11.783 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:06:11.783 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:11.783 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:11.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.783 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6825/1348640258", 2024-04-24T23:06:11.783 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6827/1348640258", 2024-04-24T23:06:11.783 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6831/1348640258", 2024-04-24T23:06:11.783 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6829/1348640258", 2024-04-24T23:06:11.783 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:06:11.783 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:06:11.783 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:06:11.783 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:11.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.784 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.784 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:06:11.784 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:06:11.784 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:06:11.784 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:06:11.784 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:06:11.784 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:06:11.784 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:06:11.784 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:06:11.784 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:06:11.784 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 138, 2024-04-24T23:06:11.784 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:06:11.784 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:06:11.784 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:06:11.784 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:11.785 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.785 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:11.785 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6816", 2024-04-24T23:06:11.785 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:06:11.785 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.785 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.785 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:11.785 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6817", 2024-04-24T23:06:11.785 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:06:11.785 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:11.785 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:11.785 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.785 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:06:11.785 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:11.785 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.786 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:11.786 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6818", 2024-04-24T23:06:11.786 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:06:11.786 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.786 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.786 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:11.786 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6819", 2024-04-24T23:06:11.786 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:06:11.786 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:11.786 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:11.786 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.786 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:06:11.786 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:11.786 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.787 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:11.787 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6822", 2024-04-24T23:06:11.787 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:06:11.787 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.787 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.787 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:11.787 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6824", 2024-04-24T23:06:11.787 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:06:11.787 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:11.787 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:11.787 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.787 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:06:11.787 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:11.787 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.787 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:11.788 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6820", 2024-04-24T23:06:11.788 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:06:11.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.788 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.788 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:11.788 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6821", 2024-04-24T23:06:11.788 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:06:11.788 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:11.788 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:11.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.788 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6817/3250005122", 2024-04-24T23:06:11.788 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6819/3250005122", 2024-04-24T23:06:11.788 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6824/3250005122", 2024-04-24T23:06:11.788 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6821/3250005122", 2024-04-24T23:06:11.788 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:06:11.789 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:06:11.789 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:06:11.789 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:11.789 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:11.789 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:06:11.789 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2024-04-24T23:06:11.789 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.789 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:06:11.789 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:06:11.789 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:06:11.789 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:06:11.789 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:06:11.789 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:06:11.789 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.156184+0000", 2024-04-24T23:06:11.789 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:06:11.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.790 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.790 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:06:11.790 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:06:11.790 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:06:11.790 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:06:11.790 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:06:11.790 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:06:11.790 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240774+0000", 2024-04-24T23:06:11.790 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:06:11.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.790 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.790 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:06:11.790 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:06:11.790 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:06:11.791 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:06:11.791 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:06:11.791 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:06:11.791 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240949+0000", 2024-04-24T23:06:11.791 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:06:11.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.791 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.791 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:06:11.791 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:06:11.791 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:06:11.791 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:06:11.791 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:06:11.791 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:06:11.791 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.171672+0000", 2024-04-24T23:06:11.791 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:06:11.792 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.792 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.792 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:06:11.792 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:06:11.792 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:06:11.792 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:06:11.792 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:06:11.792 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:06:11.792 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.220379+0000", 2024-04-24T23:06:11.792 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:06:11.792 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.792 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.792 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:06:11.792 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:06:11.792 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:06:11.793 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:06:11.793 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:06:11.793 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:06:11.793 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.204963+0000", 2024-04-24T23:06:11.793 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:06:11.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.793 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.793 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:06:11.793 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:06:11.793 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:06:11.793 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:06:11.793 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:06:11.793 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:06:11.793 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.239748+0000", 2024-04-24T23:06:11.793 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:06:11.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.794 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.794 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:06:11.794 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:06:11.794 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:06:11.794 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:06:11.794 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:06:11.794 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:06:11.794 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.154104+0000", 2024-04-24T23:06:11.794 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:06:11.794 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:11.794 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:06:11.794 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2024-04-24T23:06:11.794 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [ 2024-04-24T23:06:11.794 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.795 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "16.17", 2024-04-24T23:06:11.795 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:06:11.795 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.795 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 4, 2024-04-24T23:06:11.795 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 3 2024-04-24T23:06:11.795 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:11.796 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:11.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.796 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "17.8", 2024-04-24T23:06:11.796 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:06:11.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.796 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 6, 2024-04-24T23:06:11.796 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 5 2024-04-24T23:06:11.796 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:11.796 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:11.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.796 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "17.3f", 2024-04-24T23:06:11.797 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:06:11.797 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:11.797 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 6, 2024-04-24T23:06:11.797 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 1 2024-04-24T23:06:11.797 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:11.797 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:11.797 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:11.797 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:06:11.797 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_primaries": [], 2024-04-24T23:06:11.797 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2024-04-24T23:06:11.797 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2024-04-24T23:06:11.797 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklist": {}, 2024-04-24T23:06:11.797 INFO:teuthology.orchestra.run.smithi032.stdout: "range_blocklist": {}, 2024-04-24T23:06:11.797 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2024-04-24T23:06:11.797 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2024-04-24T23:06:11.798 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2024-04-24T23:06:11.798 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2024-04-24T23:06:11.798 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2024-04-24T23:06:11.798 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2024-04-24T23:06:11.798 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2024-04-24T23:06:11.798 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:11.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:11.798 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2024-04-24T23:06:11.798 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2024-04-24T23:06:11.798 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2024-04-24T23:06:11.798 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2024-04-24T23:06:11.798 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {}, 2024-04-24T23:06:11.798 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode": { 2024-04-24T23:06:11.798 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_enabled": false, 2024-04-24T23:06:11.798 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_bucket_count": 0, 2024-04-24T23:06:11.799 INFO:teuthology.orchestra.run.smithi032.stdout: "degraded_stretch_mode": 0, 2024-04-24T23:06:11.799 INFO:teuthology.orchestra.run.smithi032.stdout: "recovering_stretch_mode": 0, 2024-04-24T23:06:11.799 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_bucket": 0 2024-04-24T23:06:11.799 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:11.799 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:06:11.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.709+0000 7fb37ee7d640 1 -- 172.21.15.32:0/330828543 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb35c06be60 msgr2=0x7fb35c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:11.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.709+0000 7fb37ee7d640 1 --2- 172.21.15.32:0/330828543 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb35c06be60 0x7fb35c06e320 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fb36c033000 tx=0x7fb36c008030 comp rx=0 tx=0).stop 2024-04-24T23:06:11.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.709+0000 7fb37ee7d640 1 -- 172.21.15.32:0/330828543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb37815a1e0 msgr2=0x7fb3781f5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:11.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.709+0000 7fb37ee7d640 1 --2- 172.21.15.32:0/330828543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb37815a1e0 0x7fb3781f5480 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fb364004f50 tx=0x7fb364007430 comp rx=0 tx=0).stop 2024-04-24T23:06:11.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.709+0000 7fb37ee7d640 1 -- 172.21.15.32:0/330828543 shutdown_connections 2024-04-24T23:06:11.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.709+0000 7fb37ee7d640 1 --2- 172.21.15.32:0/330828543 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb35c06be60 0x7fb35c06e320 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:11.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.709+0000 7fb37ee7d640 1 --2- 172.21.15.32:0/330828543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb37815a1e0 0x7fb3781f5480 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:11.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.709+0000 7fb37ee7d640 1 --2- 172.21.15.32:0/330828543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3781522b0 0x7fb378152730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:11.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.709+0000 7fb37ee7d640 1 --2- 172.21.15.32:0/330828543 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb368003b70 0x7fb3781507c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:11.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.709+0000 7fb37ee7d640 1 -- 172.21.15.32:0/330828543 >> 172.21.15.32:0/330828543 conn(0x7fb37806b110 msgr2=0x7fb37815a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:11.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.709+0000 7fb37ee7d640 1 -- 172.21.15.32:0/330828543 shutdown_connections 2024-04-24T23:06:11.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.709+0000 7fb37ee7d640 1 -- 172.21.15.32:0/330828543 wait complete. 2024-04-24T23:06:11.802 DEBUG:tasks.ceph_test_case:wait_until_true: success in 15s and 0 retries 2024-04-24T23:06:11.802 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-24T23:06:11.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.904+0000 7f0cbefa2640 1 Processor -- start 2024-04-24T23:06:11.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.904+0000 7f0cbefa2640 1 -- start start 2024-04-24T23:06:11.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.905+0000 7f0cbefa2640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0cb815b380 0x7f0cb806c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:11.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.905+0000 7f0cbefa2640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cb80b4520 con 0x7f0cb80ceb40 2024-04-24T23:06:11.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.905+0000 7f0cbefa2640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cb80b4800 con 0x7f0cb815cb00 2024-04-24T23:06:11.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.905+0000 7f0cbefa2640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f0cb80cef90 con 0x7f0cb815b380 2024-04-24T23:06:11.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.905+0000 7f0cbd53f640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0cb80ceb40 0x7f0cb80b55c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41162/0 (socket says 172.21.15.32:41162) 2024-04-24T23:06:11.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.905+0000 7f0cbd53f640 1 -- 172.21.15.32:0/4079914273 learned_addr learned my addr 172.21.15.32:0/4079914273 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:06:11.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.905+0000 7f0cb77fe640 1 -- 172.21.15.32:0/4079914273 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3416609156 0 0) 0x7f0cb80b4520 con 0x7f0cb80ceb40 2024-04-24T23:06:11.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.905+0000 7f0cbcd3e640 1 --2- 172.21.15.32:0/4079914273 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0cb815b380 0x7f0cb806c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:11.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.905+0000 7f0cb77fe640 1 -- 172.21.15.32:0/4079914273 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ca0003520 con 0x7f0cb80ceb40 2024-04-24T23:06:11.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.906+0000 7f0cb77fe640 1 -- 172.21.15.32:0/4079914273 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4184241905 0 0) 0x7f0ca0003520 con 0x7f0cb80ceb40 2024-04-24T23:06:11.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.906+0000 7f0cb77fe640 1 -- 172.21.15.32:0/4079914273 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0cb815b380 msgr2=0x7f0cb806c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:11.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.906+0000 7f0cb77fe640 1 --2- 172.21.15.32:0/4079914273 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0cb815b380 0x7f0cb806c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:11.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.906+0000 7f0cb77fe640 1 -- 172.21.15.32:0/4079914273 >> v1:172.21.15.121:6789/0 conn(0x7f0cb815cb00 legacy=0x7f0cb80ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:11.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.906+0000 7f0cb77fe640 1 -- 172.21.15.32:0/4079914273 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cb80b4ae0 con 0x7f0cb80ceb40 2024-04-24T23:06:11.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.906+0000 7f0cb77fe640 1 -- 172.21.15.32:0/4079914273 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f0ca8001670 con 0x7f0cb80ceb40 2024-04-24T23:06:11.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.906+0000 7f0cb77fe640 1 -- 172.21.15.32:0/4079914273 >> v1:172.21.15.32:6789/0 conn(0x7f0cb80ceb40 legacy=0x7f0cb80b55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:11.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.906+0000 7f0cb77fe640 1 --2- 172.21.15.32:0/4079914273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ca00039e0 0x7f0ca0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:11.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.906+0000 7f0cb77fe640 1 --2- 172.21.15.32:0/4079914273 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0cb815b380 0x7f0ca0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:11.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.906+0000 7f0cb77fe640 1 --2- 172.21.15.32:0/4079914273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ca0004d40 0x7f0ca00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:11.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.906+0000 7f0cb77fe640 1 -- 172.21.15.32:0/4079914273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb80cef90 con 0x7f0ca0004d40 2024-04-24T23:06:11.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.906+0000 7f0cb77fe640 1 -- 172.21.15.32:0/4079914273 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca000baf0 con 0x7f0ca00039e0 2024-04-24T23:06:11.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.906+0000 7f0cb77fe640 1 -- 172.21.15.32:0/4079914273 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f0ca000bcf0 con 0x7f0cb815b380 2024-04-24T23:06:11.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.906+0000 7f0cbd53f640 1 --2- 172.21.15.32:0/4079914273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ca0004d40 0x7f0ca00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:11.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.906+0000 7f0cbcd3e640 1 --2- 172.21.15.32:0/4079914273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ca00039e0 0x7f0ca0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:11.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.906+0000 7f0cb7fff640 1 --2- 172.21.15.32:0/4079914273 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0cb815b380 0x7f0ca0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:11.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.906+0000 7f0cbd53f640 1 -- 172.21.15.32:0/4079914273 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0cb815b380 msgr2=0x7f0ca0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:11.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.906+0000 7f0cbd53f640 1 --2- 172.21.15.32:0/4079914273 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0cb815b380 0x7f0ca0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:11.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.906+0000 7f0cbd53f640 1 -- 172.21.15.32:0/4079914273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ca00039e0 msgr2=0x7f0ca0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:11.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.906+0000 7f0cbd53f640 1 --2- 172.21.15.32:0/4079914273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ca00039e0 0x7f0ca0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:11.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.906+0000 7f0cbd53f640 1 -- 172.21.15.32:0/4079914273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ca0003520 con 0x7f0ca0004d40 2024-04-24T23:06:11.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.907+0000 7f0cbd53f640 1 --2- 172.21.15.32:0/4079914273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ca0004d40 0x7f0ca00051c0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f0ca800c240 tx=0x7f0ca0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a9d7029821d7e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:11.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.907+0000 7f0cb77fe640 1 -- 172.21.15.32:0/4079914273 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca8007d10 con 0x7f0ca0004d40 2024-04-24T23:06:11.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.907+0000 7f0cb77fe640 1 -- 172.21.15.32:0/4079914273 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ca8005ad0 con 0x7f0ca0004d40 2024-04-24T23:06:11.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.907+0000 7f0cb77fe640 1 -- 172.21.15.32:0/4079914273 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca8007070 con 0x7f0ca0004d40 2024-04-24T23:06:11.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.907+0000 7f0cbefa2640 1 -- 172.21.15.32:0/4079914273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ca0004d40 msgr2=0x7f0ca00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:11.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.907+0000 7f0cbefa2640 1 --2- 172.21.15.32:0/4079914273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ca0004d40 0x7f0ca00051c0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f0ca800c240 tx=0x7f0ca0005830 comp rx=0 tx=0).stop 2024-04-24T23:06:11.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.907+0000 7f0cbd53f640 1 -- 172.21.15.32:0/4079914273 reap_dead start 2024-04-24T23:06:11.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.908+0000 7f0cbefa2640 1 -- 172.21.15.32:0/4079914273 shutdown_connections 2024-04-24T23:06:11.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.908+0000 7f0cbefa2640 1 -- 172.21.15.32:0/4079914273 >> 172.21.15.32:0/4079914273 conn(0x7f0cb806b110 msgr2=0x7f0cb806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:11.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.908+0000 7f0cbefa2640 1 -- 172.21.15.32:0/4079914273 shutdown_connections 2024-04-24T23:06:11.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.908+0000 7f0cbefa2640 1 -- 172.21.15.32:0/4079914273 wait complete. 2024-04-24T23:06:11.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.909+0000 7f0cbefa2640 1 Processor -- start 2024-04-24T23:06:11.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.909+0000 7f0cbefa2640 1 -- start start 2024-04-24T23:06:11.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.910+0000 7f0cbefa2640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0cb80aeb20 0x7f0cb8155e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:11.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.910+0000 7f0cbefa2640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0cb80aef00 0x7f0cb8153e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:11.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.910+0000 7f0cbefa2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cb81f5100 0x7f0cb8158e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:11.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.910+0000 7f0cbefa2640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb815b760 con 0x7f0cb81f5100 2024-04-24T23:06:11.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.910+0000 7f0cbefa2640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb80b50a0 con 0x7f0cb80aef00 2024-04-24T23:06:11.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.910+0000 7f0cbefa2640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f0cb80b5cd0 con 0x7f0cb80aeb20 2024-04-24T23:06:11.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.910+0000 7f0cbd53f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cb81f5100 0x7f0cb8158e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:11.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.910+0000 7f0cbd53f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cb81f5100 0x7f0cb8158e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34748/0 (socket says 172.21.15.32:34748) 2024-04-24T23:06:11.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.910+0000 7f0cbd53f640 1 -- 172.21.15.32:0/1575044348 learned_addr learned my addr 172.21.15.32:0/1575044348 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:11.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.910+0000 7f0cbcd3e640 1 --2- 172.21.15.32:0/1575044348 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0cb80aeb20 0x7f0cb8155e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:11.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.910+0000 7f0cb7fff640 1 --2- 172.21.15.32:0/1575044348 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0cb80aef00 0x7f0cb8153e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:11.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.910+0000 7f0cbd53f640 1 -- 172.21.15.32:0/1575044348 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0cb80aeb20 msgr2=0x7f0cb8155e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:11.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.910+0000 7f0cbd53f640 1 --2- 172.21.15.32:0/1575044348 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0cb80aeb20 0x7f0cb8155e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:11.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.910+0000 7f0cbd53f640 1 -- 172.21.15.32:0/1575044348 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0cb80aef00 msgr2=0x7f0cb8153e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:11.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.910+0000 7f0cbd53f640 1 --2- 172.21.15.32:0/1575044348 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0cb80aef00 0x7f0cb8153e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:11.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.910+0000 7f0cbd53f640 1 -- 172.21.15.32:0/1575044348 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cb81543a0 con 0x7f0cb81f5100 2024-04-24T23:06:11.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.910+0000 7f0cbd53f640 1 --2- 172.21.15.32:0/1575044348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cb81f5100 0x7f0cb8158e30 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8034040 tx=0x7f0cb80b46f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:11.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.910+0000 7f0cb57fa640 1 -- 172.21.15.32:0/1575044348 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca8007d10 con 0x7f0cb81f5100 2024-04-24T23:06:11.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.911+0000 7f0cb57fa640 1 -- 172.21.15.32:0/1575044348 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ca8043070 con 0x7f0cb81f5100 2024-04-24T23:06:11.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.911+0000 7f0cbefa2640 1 -- 172.21.15.32:0/1575044348 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cb81565a0 con 0x7f0cb81f5100 2024-04-24T23:06:11.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.911+0000 7f0cb57fa640 1 -- 172.21.15.32:0/1575044348 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca80076c0 con 0x7f0cb81f5100 2024-04-24T23:06:11.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.911+0000 7f0cbefa2640 1 -- 172.21.15.32:0/1575044348 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cb8156b60 con 0x7f0cb81f5100 2024-04-24T23:06:11.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.911+0000 7f0cbefa2640 1 -- 172.21.15.32:0/1575044348 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cb8068430 con 0x7f0cb81f5100 2024-04-24T23:06:11.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.912+0000 7f0cb57fa640 1 -- 172.21.15.32:0/1575044348 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0ca803e470 con 0x7f0cb81f5100 2024-04-24T23:06:11.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.912+0000 7f0cb57fa640 1 --2- 172.21.15.32:0/1575044348 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0ca406be60 0x7f0ca406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:11.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.912+0000 7f0cb57fa640 1 -- 172.21.15.32:0/1575044348 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5796+0+0 (secure 0 0 0) 0x7f0ca80ab270 con 0x7f0cb81f5100 2024-04-24T23:06:11.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.913+0000 7f0cbcd3e640 1 --2- 172.21.15.32:0/1575044348 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0ca406be60 0x7f0ca406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:11.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.913+0000 7f0cbcd3e640 1 --2- 172.21.15.32:0/1575044348 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0ca406be60 0x7f0ca406e320 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f0cb815f100 tx=0x7f0cb000b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:11.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:11.917+0000 7f0cb57fa640 1 -- 172.21.15.32:0/1575044348 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0ca8079720 con 0x7f0cb81f5100 2024-04-24T23:06:12.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.099+0000 7f0cbefa2640 1 -- 172.21.15.32:0/1575044348 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f0cb80afd60 con 0x7f0cb81f5100 2024-04-24T23:06:12.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.101+0000 7f0cb57fa640 1 -- 172.21.15.32:0/1575044348 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v145) ==== 81+0+34741 (secure 0 0 0) 0x7f0ca807da20 con 0x7f0cb81f5100 2024-04-24T23:06:12.102 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:06:12.103 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:06:12.103 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 145, 2024-04-24T23:06:12.103 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", 2024-04-24T23:06:12.103 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2024-04-24T22:58:46.964851+0000", 2024-04-24T23:06:12.103 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2024-04-24T23:06:06.811191+0000", 2024-04-24T23:06:12.103 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2024-04-24T22:58:52.048787+0000", 2024-04-24T23:06:12.103 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2024-04-24T22:58:49.045609+0000", 2024-04-24T23:06:12.103 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-24T23:06:12.103 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2024-04-24T23:06:12.103 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2024-04-24T23:06:12.103 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2024-04-24T23:06:12.103 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2024-04-24T23:06:12.103 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2024-04-24T23:06:12.104 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2024-04-24T23:06:12.104 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:06:12.104 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 4, 2024-04-24T23:06:12.104 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.69999998807907104, 2024-04-24T23:06:12.104 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-24T23:06:12.104 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-24T23:06:12.104 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2024-04-24T23:06:12.104 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 17, 2024-04-24T23:06:12.104 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2024-04-24T23:06:12.104 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "luminous", 2024-04-24T23:06:12.104 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "luminous", 2024-04-24T23:06:12.104 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "squid", 2024-04-24T23:06:12.104 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_crimson": false, 2024-04-24T23:06:12.104 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2024-04-24T23:06:12.105 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.105 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2024-04-24T23:06:12.105 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": ".mgr", 2024-04-24T23:06:12.105 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T22:58:53.171408+0000", 2024-04-24T23:06:12.105 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:06:12.105 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:06:12.105 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:06:12.105 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:06:12.105 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:06:12.105 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:06:12.105 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:06:12.105 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:06:12.105 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:06:12.105 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:06:12.105 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:06:12.106 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:06:12.106 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2024-04-24T23:06:12.106 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2024-04-24T23:06:12.106 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2024-04-24T23:06:12.106 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2024-04-24T23:06:12.106 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2024-04-24T23:06:12.107 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:06:12.107 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:06:12.107 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:06:12.107 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:06:12.107 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:06:12.107 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:06:12.107 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:06:12.107 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.107 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "113", 2024-04-24T23:06:12.108 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:06:12.108 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:06:12.108 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:06:12.108 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:06:12.108 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:06:12.108 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:06:12.108 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:06:12.108 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:06:12.108 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:06:12.108 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:06:12.108 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:06:12.108 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:06:12.108 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:06:12.108 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:06:12.108 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:06:12.109 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:06:12.109 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:06:12.109 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:06:12.109 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:06:12.109 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:06:12.109 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:06:12.109 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:06:12.109 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:06:12.109 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:06:12.109 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:06:12.109 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:06:12.109 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.109 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:06:12.109 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:06:12.109 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:06:12.110 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:06:12.110 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:06:12.110 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:06:12.110 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:06:12.110 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:06:12.110 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:06:12.110 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:06:12.110 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:06:12.110 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:06:12.110 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_max": 32, 2024-04-24T23:06:12.110 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2024-04-24T23:06:12.110 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.110 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:06:12.110 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": {} 2024-04-24T23:06:12.111 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.111 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:06:12.111 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:06:12.111 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 8, 2024-04-24T23:06:12.111 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 8, 2024-04-24T23:06:12.111 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.25, 2024-04-24T23:06:12.111 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-04-24T23:06:12.111 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-04-24T23:06:12.111 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:06:12.111 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:06:12.111 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:06:12.111 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:12.111 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.111 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.111 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 16, 2024-04-24T23:06:12.112 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2024-04-24T23:06:12.112 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:05:32.064234+0000", 2024-04-24T23:06:12.112 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:06:12.112 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:06:12.112 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:06:12.112 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:06:12.112 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:06:12.112 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:06:12.112 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:06:12.112 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:06:12.112 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:06:12.112 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:06:12.112 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:06:12.113 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:06:12.113 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 32, 2024-04-24T23:06:12.113 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 32, 2024-04-24T23:06:12.113 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 32, 2024-04-24T23:06:12.113 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 32, 2024-04-24T23:06:12.113 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 32, 2024-04-24T23:06:12.113 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:06:12.113 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:06:12.113 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:06:12.113 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:06:12.113 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:06:12.113 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:06:12.113 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:06:12.113 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.114 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "139", 2024-04-24T23:06:12.114 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:06:12.114 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:06:12.114 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:06:12.114 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:06:12.114 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:06:12.114 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:06:12.114 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:06:12.114 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:06:12.114 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:06:12.114 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:06:12.114 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:06:12.114 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:06:12.114 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:06:12.114 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:06:12.115 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:06:12.115 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:06:12.115 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:06:12.115 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:06:12.115 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:06:12.115 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:06:12.115 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:06:12.115 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:06:12.115 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:06:12.115 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:06:12.115 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:06:12.115 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:06:12.115 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.115 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:06:12.115 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:06:12.116 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:06:12.116 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:06:12.116 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:06:12.116 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:06:12.116 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:06:12.116 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:06:12.116 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:06:12.116 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:06:12.116 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:06:12.116 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:06:12.116 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2024-04-24T23:06:12.116 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2024-04-24T23:06:12.116 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2024-04-24T23:06:12.116 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.117 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:06:12.117 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:06:12.117 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2024-04-24T23:06:12.117 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:12.117 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.117 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:06:12.117 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:06:12.117 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 2, 2024-04-24T23:06:12.117 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 2, 2024-04-24T23:06:12.117 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:06:12.117 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-04-24T23:06:12.117 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-04-24T23:06:12.117 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:06:12.117 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:06:12.117 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:06:12.118 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:12.118 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.118 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.118 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 17, 2024-04-24T23:06:12.118 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2024-04-24T23:06:12.118 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:05:32.760777+0000", 2024-04-24T23:06:12.118 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:06:12.118 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:06:12.118 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:06:12.118 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:06:12.118 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:06:12.119 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:06:12.119 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:06:12.119 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:06:12.119 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:06:12.119 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:06:12.119 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:06:12.119 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:06:12.119 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 64, 2024-04-24T23:06:12.119 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 64, 2024-04-24T23:06:12.120 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 64, 2024-04-24T23:06:12.120 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 64, 2024-04-24T23:06:12.120 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 64, 2024-04-24T23:06:12.120 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:06:12.120 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:06:12.120 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:06:12.120 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:06:12.120 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:06:12.120 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:06:12.120 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:06:12.120 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.120 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "145", 2024-04-24T23:06:12.121 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:06:12.121 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:06:12.121 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:06:12.121 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:06:12.121 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:06:12.121 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:06:12.121 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:06:12.121 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:06:12.121 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:06:12.121 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 33554432, 2024-04-24T23:06:12.121 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:06:12.122 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:06:12.122 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:06:12.122 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:06:12.122 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:06:12.122 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:06:12.122 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:06:12.122 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:06:12.122 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:06:12.122 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:06:12.122 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:06:12.122 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:06:12.122 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:06:12.122 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:06:12.122 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:06:12.123 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:06:12.123 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.123 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:06:12.123 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:06:12.123 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:06:12.123 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:06:12.123 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:06:12.123 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:06:12.123 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:06:12.123 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:06:12.123 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:06:12.123 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:06:12.123 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:06:12.124 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:06:12.124 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 64, 2024-04-24T23:06:12.124 INFO:teuthology.orchestra.run.smithi032.stdout: "target_size_ratio": 0.89999997615814209 2024-04-24T23:06:12.124 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.124 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:06:12.124 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:06:12.124 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2024-04-24T23:06:12.124 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:12.124 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.124 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:06:12.124 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:06:12.124 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.3799999952316284, 2024-04-24T23:06:12.124 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.3799999952316284, 2024-04-24T23:06:12.124 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:06:12.124 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.3799999952316284, 2024-04-24T23:06:12.124 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.3799999952316284, 2024-04-24T23:06:12.125 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:06:12.125 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:06:12.125 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:06:12.125 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:12.125 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:12.125 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:06:12.125 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-04-24T23:06:12.125 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.125 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:06:12.125 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:06:12.125 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:06:12.125 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:06:12.125 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:06:12.125 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:06:12.126 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:06:12.126 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:06:12.126 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:06:12.126 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 143, 2024-04-24T23:06:12.126 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:06:12.126 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:06:12.126 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:06:12.126 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:12.126 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.126 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:12.126 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2024-04-24T23:06:12.127 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:06:12.127 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.127 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.127 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:12.127 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2024-04-24T23:06:12.127 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:06:12.127 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:12.127 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:12.127 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.127 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:06:12.127 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:12.128 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.128 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:12.128 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2024-04-24T23:06:12.128 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:06:12.128 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.128 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.128 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:12.128 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2024-04-24T23:06:12.128 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:06:12.128 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:12.128 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:12.129 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.129 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:06:12.129 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:12.129 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.129 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:12.129 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2024-04-24T23:06:12.129 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:06:12.129 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.129 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.129 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:12.129 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2024-04-24T23:06:12.129 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:06:12.129 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:12.129 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:12.129 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.130 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:06:12.130 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:12.130 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.130 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:12.130 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2024-04-24T23:06:12.130 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:06:12.130 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.130 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.130 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:12.130 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2024-04-24T23:06:12.130 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:06:12.130 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:12.130 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:12.130 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.130 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/1479395426", 2024-04-24T23:06:12.131 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/1479395426", 2024-04-24T23:06:12.131 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/1479395426", 2024-04-24T23:06:12.131 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/1479395426", 2024-04-24T23:06:12.131 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:06:12.131 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:06:12.131 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:06:12.131 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:12.131 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.131 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.131 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:06:12.131 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:06:12.131 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:06:12.131 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:06:12.131 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:06:12.131 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:06:12.132 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:06:12.132 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:06:12.132 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:06:12.132 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 143, 2024-04-24T23:06:12.132 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:06:12.132 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:06:12.132 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:06:12.132 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:12.132 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.132 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:12.132 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2024-04-24T23:06:12.133 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:06:12.133 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.133 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.133 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:12.133 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2024-04-24T23:06:12.133 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:06:12.133 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:12.133 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:12.133 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.133 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:06:12.133 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:12.133 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.134 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:12.134 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2024-04-24T23:06:12.134 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:06:12.134 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.134 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.134 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:12.134 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2024-04-24T23:06:12.135 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:06:12.135 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:12.135 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:12.135 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.135 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:06:12.135 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:12.135 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.135 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:12.135 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2024-04-24T23:06:12.135 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:06:12.135 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.135 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.135 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:12.135 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2024-04-24T23:06:12.135 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:06:12.136 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:12.136 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:12.136 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.136 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:06:12.136 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:12.136 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.136 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:12.136 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2024-04-24T23:06:12.136 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:06:12.136 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.136 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.136 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:12.136 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2024-04-24T23:06:12.136 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:06:12.136 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:12.136 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:12.137 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.137 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/1340633718", 2024-04-24T23:06:12.137 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/1340633718", 2024-04-24T23:06:12.137 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/1340633718", 2024-04-24T23:06:12.137 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/1340633718", 2024-04-24T23:06:12.137 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:06:12.137 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:06:12.137 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:06:12.137 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:12.137 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.137 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.137 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:06:12.137 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:06:12.137 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:06:12.137 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:06:12.138 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:06:12.138 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:06:12.138 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:06:12.138 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:06:12.138 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:06:12.138 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 138, 2024-04-24T23:06:12.138 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:06:12.138 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:06:12.138 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:06:12.138 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:12.138 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.138 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:12.138 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2024-04-24T23:06:12.138 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:06:12.139 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.139 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.139 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:12.139 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2024-04-24T23:06:12.139 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:06:12.139 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:12.139 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:12.139 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.139 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:06:12.139 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:12.139 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.139 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:12.139 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2024-04-24T23:06:12.139 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:06:12.139 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.140 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.140 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:12.140 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2024-04-24T23:06:12.140 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:06:12.140 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:12.140 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:12.140 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.140 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:06:12.140 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:12.140 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.140 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:12.140 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2024-04-24T23:06:12.140 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:06:12.140 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.140 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.141 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:12.141 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2024-04-24T23:06:12.141 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:06:12.141 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:12.141 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:12.141 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.141 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:06:12.141 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:12.141 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.141 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:12.141 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2024-04-24T23:06:12.141 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:06:12.141 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.141 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.141 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:12.142 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2024-04-24T23:06:12.142 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:06:12.142 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:12.142 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:12.142 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.142 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/3405899334", 2024-04-24T23:06:12.142 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6820/3405899334", 2024-04-24T23:06:12.142 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6828/3405899334", 2024-04-24T23:06:12.142 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6824/3405899334", 2024-04-24T23:06:12.142 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:06:12.142 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:06:12.142 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:06:12.143 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:12.143 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.143 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.143 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:06:12.143 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:06:12.143 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:06:12.143 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:06:12.143 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:06:12.143 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:06:12.143 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:06:12.143 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:06:12.143 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:06:12.143 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 143, 2024-04-24T23:06:12.143 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:06:12.143 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:06:12.144 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:06:12.144 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:12.144 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.144 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:12.144 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2024-04-24T23:06:12.144 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:06:12.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.144 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.144 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:12.144 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2024-04-24T23:06:12.144 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:06:12.144 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:12.144 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:12.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.144 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:06:12.145 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:12.145 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.145 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:12.145 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2024-04-24T23:06:12.145 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:06:12.145 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.145 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.145 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:12.145 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2024-04-24T23:06:12.145 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:06:12.145 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:12.145 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:12.145 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.145 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:06:12.146 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:12.146 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.146 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:12.146 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2024-04-24T23:06:12.146 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:06:12.146 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.146 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.146 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:12.146 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2024-04-24T23:06:12.146 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:06:12.146 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:12.146 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:12.146 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.146 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:06:12.147 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:12.147 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.147 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:12.147 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2024-04-24T23:06:12.147 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:06:12.147 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.147 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.147 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:12.147 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2024-04-24T23:06:12.147 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:06:12.148 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:12.148 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:12.148 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.148 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6821/3381057190", 2024-04-24T23:06:12.148 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6825/3381057190", 2024-04-24T23:06:12.148 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/3381057190", 2024-04-24T23:06:12.148 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/3381057190", 2024-04-24T23:06:12.148 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:06:12.148 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:06:12.148 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:06:12.148 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:12.148 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.148 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.148 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:06:12.148 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:06:12.149 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:06:12.149 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:06:12.149 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:06:12.149 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:06:12.149 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:06:12.149 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:06:12.149 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:06:12.149 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 138, 2024-04-24T23:06:12.149 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:06:12.149 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:06:12.149 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:06:12.149 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:12.149 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.149 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:12.149 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6800", 2024-04-24T23:06:12.150 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:06:12.150 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.150 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.150 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:12.150 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6801", 2024-04-24T23:06:12.150 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:06:12.150 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:12.150 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:12.150 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.150 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:06:12.150 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:12.150 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.150 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:12.150 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6802", 2024-04-24T23:06:12.151 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:06:12.151 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.151 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.151 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:12.151 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6803", 2024-04-24T23:06:12.151 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:06:12.151 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:12.151 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:12.151 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.151 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:06:12.151 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:12.152 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.152 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:12.152 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6806", 2024-04-24T23:06:12.152 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:06:12.152 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.152 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.152 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:12.152 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6807", 2024-04-24T23:06:12.152 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:06:12.152 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:12.152 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:12.152 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.152 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:06:12.152 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:12.153 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.153 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:12.153 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6804", 2024-04-24T23:06:12.153 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:06:12.153 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.153 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.153 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:12.153 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6805", 2024-04-24T23:06:12.153 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:06:12.153 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:12.153 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:12.153 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.153 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6801/4010553979", 2024-04-24T23:06:12.153 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6803/4010553979", 2024-04-24T23:06:12.154 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6807/4010553979", 2024-04-24T23:06:12.154 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6805/4010553979", 2024-04-24T23:06:12.154 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:06:12.154 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:06:12.154 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:06:12.154 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:12.154 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.154 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.154 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:06:12.154 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:06:12.154 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:06:12.154 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:06:12.154 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:06:12.154 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:06:12.155 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:06:12.155 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:06:12.155 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:06:12.155 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 138, 2024-04-24T23:06:12.155 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:06:12.155 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:06:12.155 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:06:12.155 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:12.155 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.155 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:12.155 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6808", 2024-04-24T23:06:12.155 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:06:12.155 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.155 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.155 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:12.156 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6809", 2024-04-24T23:06:12.156 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:06:12.156 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:12.156 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:12.156 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.156 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:06:12.156 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:12.156 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.156 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:12.156 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6810", 2024-04-24T23:06:12.156 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:06:12.157 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.157 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.157 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:12.157 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6811", 2024-04-24T23:06:12.157 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:06:12.157 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:12.157 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:12.157 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.157 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:06:12.157 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:12.157 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.157 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:12.157 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6814", 2024-04-24T23:06:12.157 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:06:12.158 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.158 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.158 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:12.158 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6815", 2024-04-24T23:06:12.158 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:06:12.158 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:12.158 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:12.158 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.158 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:06:12.158 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:12.158 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.158 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:12.158 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6812", 2024-04-24T23:06:12.158 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:06:12.159 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.159 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.159 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:12.159 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6813", 2024-04-24T23:06:12.159 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:06:12.159 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:12.159 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:12.159 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.159 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6809/3794084091", 2024-04-24T23:06:12.159 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6811/3794084091", 2024-04-24T23:06:12.159 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6815/3794084091", 2024-04-24T23:06:12.159 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6813/3794084091", 2024-04-24T23:06:12.159 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:06:12.159 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:06:12.159 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:06:12.160 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:12.160 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.160 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.160 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:06:12.160 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:06:12.160 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:06:12.160 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:06:12.160 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:06:12.160 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:06:12.160 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:06:12.160 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:06:12.160 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:06:12.160 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 138, 2024-04-24T23:06:12.160 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:06:12.161 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:06:12.161 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:06:12.161 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:12.161 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.161 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:12.161 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6823", 2024-04-24T23:06:12.162 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:06:12.162 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.163 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.163 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:12.163 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6825", 2024-04-24T23:06:12.163 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:06:12.163 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:12.164 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:12.164 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.165 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:06:12.165 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:12.165 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.165 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:12.165 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6826", 2024-04-24T23:06:12.166 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:06:12.167 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.167 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.167 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:12.167 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6827", 2024-04-24T23:06:12.167 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:06:12.167 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:12.167 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:12.168 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.168 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:06:12.168 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:12.168 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.168 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:12.168 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6830", 2024-04-24T23:06:12.168 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:06:12.168 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.168 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.168 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:12.168 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6831", 2024-04-24T23:06:12.168 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:06:12.168 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:12.168 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:12.168 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.169 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:06:12.169 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:12.169 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.169 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:12.169 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6828", 2024-04-24T23:06:12.169 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:06:12.169 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.169 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.169 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:12.169 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6829", 2024-04-24T23:06:12.169 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:06:12.169 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:12.169 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:12.169 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.169 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6825/1348640258", 2024-04-24T23:06:12.170 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6827/1348640258", 2024-04-24T23:06:12.170 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6831/1348640258", 2024-04-24T23:06:12.170 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6829/1348640258", 2024-04-24T23:06:12.170 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:06:12.170 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:06:12.170 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:06:12.170 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:12.170 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.170 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.170 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:06:12.170 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:06:12.170 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:06:12.170 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:06:12.170 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:06:12.171 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:06:12.171 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:06:12.171 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:06:12.171 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:06:12.171 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 138, 2024-04-24T23:06:12.171 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:06:12.171 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:06:12.171 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:06:12.171 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:12.171 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.171 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:12.171 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6816", 2024-04-24T23:06:12.171 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:06:12.171 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.171 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.171 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:12.172 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6817", 2024-04-24T23:06:12.172 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:06:12.172 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:12.172 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:12.172 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.172 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:06:12.172 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:12.172 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.172 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:12.172 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6818", 2024-04-24T23:06:12.172 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:06:12.172 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.172 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.172 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:12.173 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6819", 2024-04-24T23:06:12.173 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:06:12.173 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:12.173 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:12.173 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.173 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:06:12.173 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:12.173 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.173 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:12.173 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6822", 2024-04-24T23:06:12.173 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:06:12.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.174 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.174 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:12.174 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6824", 2024-04-24T23:06:12.174 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:06:12.174 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:12.174 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:12.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.174 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:06:12.174 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:06:12.174 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.174 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:06:12.174 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6820", 2024-04-24T23:06:12.174 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:06:12.175 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.175 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.175 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:06:12.175 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6821", 2024-04-24T23:06:12.175 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:06:12.175 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:12.175 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:12.175 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.175 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6817/3250005122", 2024-04-24T23:06:12.175 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6819/3250005122", 2024-04-24T23:06:12.175 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6824/3250005122", 2024-04-24T23:06:12.175 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6821/3250005122", 2024-04-24T23:06:12.175 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:06:12.175 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:06:12.175 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:06:12.176 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:12.176 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:12.176 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:06:12.176 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2024-04-24T23:06:12.176 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.176 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:06:12.176 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:06:12.176 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:06:12.176 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:06:12.176 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:06:12.176 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:06:12.176 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.156184+0000", 2024-04-24T23:06:12.176 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:06:12.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.176 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.177 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:06:12.177 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:06:12.177 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:06:12.177 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:06:12.177 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:06:12.177 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:06:12.177 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240774+0000", 2024-04-24T23:06:12.177 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:06:12.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.177 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.177 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:06:12.177 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:06:12.177 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:06:12.177 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:06:12.177 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:06:12.178 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:06:12.178 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240949+0000", 2024-04-24T23:06:12.178 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:06:12.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.178 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:06:12.178 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:06:12.178 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:06:12.178 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:06:12.178 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:06:12.178 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:06:12.178 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.171672+0000", 2024-04-24T23:06:12.178 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:06:12.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.179 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:06:12.179 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:06:12.179 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:06:12.179 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:06:12.179 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:06:12.179 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:06:12.179 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.220379+0000", 2024-04-24T23:06:12.179 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:06:12.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.179 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.179 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:06:12.179 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:06:12.179 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:06:12.179 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:06:12.179 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:06:12.180 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:06:12.180 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.204963+0000", 2024-04-24T23:06:12.180 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:06:12.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.180 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.180 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:06:12.180 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:06:12.180 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:06:12.180 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:06:12.180 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:06:12.180 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:06:12.180 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.239748+0000", 2024-04-24T23:06:12.180 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:06:12.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.180 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.181 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:06:12.181 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:06:12.181 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:06:12.181 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:06:12.181 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:06:12.181 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:06:12.181 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.154104+0000", 2024-04-24T23:06:12.181 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:06:12.181 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:12.181 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:06:12.181 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2024-04-24T23:06:12.181 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [ 2024-04-24T23:06:12.181 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.181 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "16.17", 2024-04-24T23:06:12.181 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:06:12.182 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.182 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 4, 2024-04-24T23:06:12.182 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 3 2024-04-24T23:06:12.182 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:12.182 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:12.182 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.182 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.182 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "17.8", 2024-04-24T23:06:12.182 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:06:12.182 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.182 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 6, 2024-04-24T23:06:12.182 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 5 2024-04-24T23:06:12.182 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:12.182 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:12.183 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.183 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.183 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "17.3f", 2024-04-24T23:06:12.183 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:06:12.183 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:12.183 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 6, 2024-04-24T23:06:12.183 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 1 2024-04-24T23:06:12.183 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:12.183 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:12.183 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:12.183 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:06:12.183 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_primaries": [], 2024-04-24T23:06:12.183 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2024-04-24T23:06:12.183 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2024-04-24T23:06:12.183 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklist": {}, 2024-04-24T23:06:12.183 INFO:teuthology.orchestra.run.smithi032.stdout: "range_blocklist": {}, 2024-04-24T23:06:12.184 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2024-04-24T23:06:12.184 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2024-04-24T23:06:12.184 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2024-04-24T23:06:12.184 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2024-04-24T23:06:12.184 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2024-04-24T23:06:12.184 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2024-04-24T23:06:12.184 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2024-04-24T23:06:12.184 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:12.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:12.184 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2024-04-24T23:06:12.184 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2024-04-24T23:06:12.184 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2024-04-24T23:06:12.184 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2024-04-24T23:06:12.184 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {}, 2024-04-24T23:06:12.184 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode": { 2024-04-24T23:06:12.185 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_enabled": false, 2024-04-24T23:06:12.185 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_bucket_count": 0, 2024-04-24T23:06:12.185 INFO:teuthology.orchestra.run.smithi032.stdout: "degraded_stretch_mode": 0, 2024-04-24T23:06:12.185 INFO:teuthology.orchestra.run.smithi032.stdout: "recovering_stretch_mode": 0, 2024-04-24T23:06:12.185 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_bucket": 0 2024-04-24T23:06:12.185 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:12.185 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:06:12.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.102+0000 7f0cbefa2640 1 -- 172.21.15.32:0/1575044348 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0ca406be60 msgr2=0x7f0ca406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:12.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.102+0000 7f0cbefa2640 1 --2- 172.21.15.32:0/1575044348 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0ca406be60 0x7f0ca406e320 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f0cb815f100 tx=0x7f0cb000b040 comp rx=0 tx=0).stop 2024-04-24T23:06:12.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.103+0000 7f0cbefa2640 1 -- 172.21.15.32:0/1575044348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cb81f5100 msgr2=0x7f0cb8158e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:12.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.103+0000 7f0cbefa2640 1 --2- 172.21.15.32:0/1575044348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cb81f5100 0x7f0cb8158e30 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8034040 tx=0x7f0cb80b46f0 comp rx=0 tx=0).stop 2024-04-24T23:06:12.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.103+0000 7f0cbefa2640 1 -- 172.21.15.32:0/1575044348 shutdown_connections 2024-04-24T23:06:12.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.103+0000 7f0cbefa2640 1 --2- 172.21.15.32:0/1575044348 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0ca406be60 0x7f0ca406e320 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:12.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.103+0000 7f0cbefa2640 1 --2- 172.21.15.32:0/1575044348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cb81f5100 0x7f0cb8158e30 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:12.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.103+0000 7f0cbefa2640 1 --2- 172.21.15.32:0/1575044348 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0cb80aef00 0x7f0cb8153e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:12.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.103+0000 7f0cbefa2640 1 --2- 172.21.15.32:0/1575044348 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0cb80aeb20 0x7f0cb8155e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:12.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.103+0000 7f0cbefa2640 1 -- 172.21.15.32:0/1575044348 >> 172.21.15.32:0/1575044348 conn(0x7f0cb806b110 msgr2=0x7f0cb806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:12.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.103+0000 7f0cbefa2640 1 -- 172.21.15.32:0/1575044348 shutdown_connections 2024-04-24T23:06:12.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.103+0000 7f0cbefa2640 1 -- 172.21.15.32:0/1575044348 wait complete. 2024-04-24T23:06:12.188 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:06:12.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.294+0000 7f841ad32640 1 Processor -- start 2024-04-24T23:06:12.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.294+0000 7f841ad32640 1 -- start start 2024-04-24T23:06:12.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.295+0000 7f841ad32640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84140b5560 0x7f841415e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:12.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.295+0000 7f841ad32640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f84140cd3c0 0x7f841415ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:12.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.295+0000 7f841ad32640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f84140b5940 0x7f84140b5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:12.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.295+0000 7f841ad32640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f84140cd910 con 0x7f84140cd3c0 2024-04-24T23:06:12.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.295+0000 7f841ad32640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f84140cdbf0 con 0x7f84140b5940 2024-04-24T23:06:12.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.295+0000 7f841ad32640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f84140cded0 con 0x7f84140b5560 2024-04-24T23:06:12.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.295+0000 7f8413fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f84140cd3c0 0x7f841415ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:12.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.295+0000 7f8413fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f84140cd3c0 0x7f841415ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34760/0 (socket says 172.21.15.32:34760) 2024-04-24T23:06:12.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.295+0000 7f8413fff640 1 -- 172.21.15.32:0/2895640446 learned_addr learned my addr 172.21.15.32:0/2895640446 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:12.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.296+0000 7f84192cf640 1 --2- 172.21.15.32:0/2895640446 >> v2:172.21.15.121:3300/0 conn(0x7f84140b5940 0x7f84140b5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:12.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.296+0000 7f8418ace640 1 --2- 172.21.15.32:0/2895640446 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84140b5560 0x7f841415e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:12.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.296+0000 7f8413fff640 1 -- 172.21.15.32:0/2895640446 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84140b5560 msgr2=0x7f841415e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:12.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.296+0000 7f8413fff640 1 --2- 172.21.15.32:0/2895640446 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84140b5560 0x7f841415e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:12.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.296+0000 7f8413fff640 1 -- 172.21.15.32:0/2895640446 >> v2:172.21.15.121:3300/0 conn(0x7f84140b5940 msgr2=0x7f84140b5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:12.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.296+0000 7f8413fff640 1 --2- 172.21.15.32:0/2895640446 >> v2:172.21.15.121:3300/0 conn(0x7f84140b5940 0x7f84140b5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:12.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.296+0000 7f8413fff640 1 -- 172.21.15.32:0/2895640446 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84140ce1b0 con 0x7f84140cd3c0 2024-04-24T23:06:12.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.296+0000 7f8413fff640 1 --2- 172.21.15.32:0/2895640446 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84140cd3c0 0x7f841415ec00 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f84140ccb60 tx=0x7f840802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbdbce8d483ff1e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:12.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.297+0000 7f84137fe640 1 -- 172.21.15.32:0/2895640446 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8408037070 con 0x7f84140cd3c0 2024-04-24T23:06:12.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.297+0000 7f84137fe640 1 -- 172.21.15.32:0/2895640446 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8408032440 con 0x7f84140cd3c0 2024-04-24T23:06:12.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.297+0000 7f84137fe640 1 -- 172.21.15.32:0/2895640446 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8408032810 con 0x7f84140cd3c0 2024-04-24T23:06:12.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.297+0000 7f841ad32640 1 -- 172.21.15.32:0/2895640446 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84140cd3c0 msgr2=0x7f841415ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:12.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.297+0000 7f841ad32640 1 --2- 172.21.15.32:0/2895640446 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84140cd3c0 0x7f841415ec00 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f84140ccb60 tx=0x7f840802a1e0 comp rx=0 tx=0).stop 2024-04-24T23:06:12.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.297+0000 7f841ad32640 1 -- 172.21.15.32:0/2895640446 shutdown_connections 2024-04-24T23:06:12.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.297+0000 7f841ad32640 1 --2- 172.21.15.32:0/2895640446 >> v2:172.21.15.121:3300/0 conn(0x7f84140b5940 0x7f84140b5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:12.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.297+0000 7f841ad32640 1 --2- 172.21.15.32:0/2895640446 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84140cd3c0 0x7f841415ec00 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:12.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.297+0000 7f841ad32640 1 --2- 172.21.15.32:0/2895640446 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84140b5560 0x7f841415e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:12.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.297+0000 7f841ad32640 1 -- 172.21.15.32:0/2895640446 >> 172.21.15.32:0/2895640446 conn(0x7f841406b110 msgr2=0x7f841406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:12.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.297+0000 7f841ad32640 1 -- 172.21.15.32:0/2895640446 shutdown_connections 2024-04-24T23:06:12.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.297+0000 7f841ad32640 1 -- 172.21.15.32:0/2895640446 wait complete. 2024-04-24T23:06:12.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.298+0000 7f841ad32640 1 Processor -- start 2024-04-24T23:06:12.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.298+0000 7f841ad32640 1 -- start start 2024-04-24T23:06:12.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.299+0000 7f841ad32640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84140b5560 0x7f841415a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:12.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.299+0000 7f841ad32640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84140b5940 0x7f841414e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:12.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.299+0000 7f841ad32640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84140cd3c0 0x7f84141e63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:12.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.299+0000 7f841ad32640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f84140b6450 con 0x7f84140b5560 2024-04-24T23:06:12.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.299+0000 7f841ad32640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f84140b62d0 con 0x7f84140b5940 2024-04-24T23:06:12.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.299+0000 7f841ad32640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f841406c5f0 con 0x7f84140cd3c0 2024-04-24T23:06:12.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.299+0000 7f8418ace640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84140b5560 0x7f841415a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:12.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.299+0000 7f8418ace640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84140b5560 0x7f841415a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34764/0 (socket says 172.21.15.32:34764) 2024-04-24T23:06:12.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.299+0000 7f8418ace640 1 -- 172.21.15.32:0/1298683209 learned_addr learned my addr 172.21.15.32:0/1298683209 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:12.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.299+0000 7f84192cf640 1 --2- 172.21.15.32:0/1298683209 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84140cd3c0 0x7f84141e63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:12.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.299+0000 7f8413fff640 1 --2- 172.21.15.32:0/1298683209 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84140b5940 0x7f841414e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:12.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.300+0000 7f8418ace640 1 -- 172.21.15.32:0/1298683209 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84140cd3c0 msgr2=0x7f84141e63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:12.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.300+0000 7f8418ace640 1 --2- 172.21.15.32:0/1298683209 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84140cd3c0 0x7f84141e63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:12.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.300+0000 7f8418ace640 1 -- 172.21.15.32:0/1298683209 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84140b5940 msgr2=0x7f841414e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:12.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.300+0000 7f8418ace640 1 --2- 172.21.15.32:0/1298683209 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84140b5940 0x7f841414e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:12.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.300+0000 7f8418ace640 1 -- 172.21.15.32:0/1298683209 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8414072630 con 0x7f84140b5560 2024-04-24T23:06:12.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.300+0000 7f8418ace640 1 --2- 172.21.15.32:0/1298683209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84140b5560 0x7f841415a890 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f840c007ef0 tx=0x7f840c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:12.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.300+0000 7f84117fa640 1 -- 172.21.15.32:0/1298683209 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840c014620 con 0x7f84140b5560 2024-04-24T23:06:12.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.300+0000 7f84117fa640 1 -- 172.21.15.32:0/1298683209 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f840c019070 con 0x7f84140b5560 2024-04-24T23:06:12.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.300+0000 7f841ad32640 1 -- 172.21.15.32:0/1298683209 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8414070960 con 0x7f84140b5560 2024-04-24T23:06:12.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.300+0000 7f84117fa640 1 -- 172.21.15.32:0/1298683209 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840c014a00 con 0x7f84140b5560 2024-04-24T23:06:12.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.300+0000 7f841ad32640 1 -- 172.21.15.32:0/1298683209 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8414070f20 con 0x7f84140b5560 2024-04-24T23:06:12.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.301+0000 7f841ad32640 1 -- 172.21.15.32:0/1298683209 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8414068430 con 0x7f84140b5560 2024-04-24T23:06:12.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.302+0000 7f84117fa640 1 -- 172.21.15.32:0/1298683209 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f840c014ba0 con 0x7f84140b5560 2024-04-24T23:06:12.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.302+0000 7f84117fa640 1 --2- 172.21.15.32:0/1298683209 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f840006be60 0x7f840006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:12.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.302+0000 7f84117fa640 1 -- 172.21.15.32:0/1298683209 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5796+0+0 (secure 0 0 0) 0x7f840c08b610 con 0x7f84140b5560 2024-04-24T23:06:12.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.302+0000 7f8413fff640 1 --2- 172.21.15.32:0/1298683209 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f840006be60 0x7f840006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:12.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.303+0000 7f8413fff640 1 --2- 172.21.15.32:0/1298683209 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f840006be60 0x7f840006e320 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f8408034000 tx=0x7f840802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:12.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.306+0000 7f84117fa640 1 -- 172.21.15.32:0/1298683209 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f840c0599a0 con 0x7f84140b5560 2024-04-24T23:06:12.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.526+0000 7f841ad32640 1 -- 172.21.15.32:0/1298683209 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f841406d5f0 con 0x7f84140b5560 2024-04-24T23:06:12.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.527+0000 7f84117fa640 1 -- 172.21.15.32:0/1298683209 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+5189 (secure 0 0 0) 0x7f840c05dca0 con 0x7f84140b5560 2024-04-24T23:06:12.528 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:06:12.529 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":58,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3361752496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3361752496},{"type":"v1","addr":"172.21.15.121:6837","nonce":3361752496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":6799,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3813453544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3813453544},{"type":"v1","addr":"172.21.15.121:6836","nonce":3813453544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7779,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/1352085651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1352085651},{"type":"v1","addr":"172.21.15.32:6833","nonce":1352085651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:05:33.791068+0000","modified":"2024-04-24T23:05:35.792946+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7794},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7794":{"gid":7794,"name":"a","rank":0,"incarnation":56,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/4206823204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":4206823204},{"type":"v1","addr":"172.21.15.32:6835","nonce":4206823204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7794,"qdb_cluster":[7794]},"id":8}]} 2024-04-24T23:06:12.529 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 58 2024-04-24T23:06:12.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.528+0000 7f841ad32640 1 -- 172.21.15.32:0/1298683209 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f840006be60 msgr2=0x7f840006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:12.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.528+0000 7f841ad32640 1 --2- 172.21.15.32:0/1298683209 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f840006be60 0x7f840006e320 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f8408034000 tx=0x7f840802e000 comp rx=0 tx=0).stop 2024-04-24T23:06:12.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.528+0000 7f841ad32640 1 -- 172.21.15.32:0/1298683209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84140b5560 msgr2=0x7f841415a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:12.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.528+0000 7f841ad32640 1 --2- 172.21.15.32:0/1298683209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84140b5560 0x7f841415a890 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f840c007ef0 tx=0x7f840c00e4b0 comp rx=0 tx=0).stop 2024-04-24T23:06:12.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.529+0000 7f841ad32640 1 -- 172.21.15.32:0/1298683209 shutdown_connections 2024-04-24T23:06:12.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.529+0000 7f841ad32640 1 --2- 172.21.15.32:0/1298683209 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f840006be60 0x7f840006e320 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:12.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.529+0000 7f841ad32640 1 --2- 172.21.15.32:0/1298683209 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84140cd3c0 0x7f84141e63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:12.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.529+0000 7f841ad32640 1 --2- 172.21.15.32:0/1298683209 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84140b5940 0x7f841414e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:12.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.529+0000 7f841ad32640 1 --2- 172.21.15.32:0/1298683209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84140b5560 0x7f841415a890 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:12.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.529+0000 7f841ad32640 1 -- 172.21.15.32:0/1298683209 >> 172.21.15.32:0/1298683209 conn(0x7f841406b110 msgr2=0x7f84140b3870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:12.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.529+0000 7f841ad32640 1 -- 172.21.15.32:0/1298683209 shutdown_connections 2024-04-24T23:06:12.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:12.529+0000 7f841ad32640 1 -- 172.21.15.32:0/1298683209 wait complete. 2024-04-24T23:06:12.542 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-04-24T23:06:12.662 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","whoami":0,"id":7794,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":38.233117118999999,"mdsmap_epoch":58,"osdmap_epoch":145,"osdmap_epoch_barrier":144,"uptime":43.304539603999999} 2024-04-24T23:06:12.663 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", "fs_name": "cephfs", "id": 7794, "mdsmap_epoch": 58, "osdmap_epoch": 145, "osdmap_epoch_barrier": 144, "rank_uptime": 38.233117119, "state": "up:active", "uptime": 43.304539604, "want_state": "up:active", "whoami": 0 } 2024-04-24T23:06:12.663 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-24T23:06:12.663 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:06:12.663 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:06:12.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:06:12.680 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2024-04-24T23:06:12.913 INFO:teuthology.orchestra.run.smithi032.stderr:50+0 records in 2024-04-24T23:06:12.913 INFO:teuthology.orchestra.run.smithi032.stderr:50+0 records out 2024-04-24T23:06:12.913 INFO:teuthology.orchestra.run.smithi032.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.17074 s, 307 MB/s 2024-04-24T23:06:12.913 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-24T23:06:12.913 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:06:12.961 DEBUG:teuthology.orchestra.run.smithi121:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:06:13.005 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-24T23:06:13.005 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:06:13.005 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:06:13.005 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:06:13.028 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:06:13.029 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:06:13.091 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:06:13.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.237+0000 7fc97c6f7640 1 Processor -- start 2024-04-24T23:06:13.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.237+0000 7fc97c6f7640 1 -- start start 2024-04-24T23:06:13.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.238+0000 7fc97c6f7640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc97415b480 0x7fc9740b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:13.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.238+0000 7fc97c6f7640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc97415fd60 0x7fc97406c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:13.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.238+0000 7fc97c6f7640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc97406cb30 0x7fc9740ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:13.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.238+0000 7fc97c6f7640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fc9740b4ff0 con 0x7fc97406cb30 2024-04-24T23:06:13.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.238+0000 7fc97c6f7640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fc9740b46c0 con 0x7fc97415b480 2024-04-24T23:06:13.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.238+0000 7fc97c6f7640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc9740b49a0 con 0x7fc97415fd60 2024-04-24T23:06:13.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.238+0000 7fc97ac94640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc97406cb30 0x7fc9740ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:13.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.238+0000 7fc97ac94640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc97406cb30 0x7fc9740ccc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34772/0 (socket says 172.21.15.32:34772) 2024-04-24T23:06:13.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.238+0000 7fc97ac94640 1 -- 172.21.15.32:0/1767167097 learned_addr learned my addr 172.21.15.32:0/1767167097 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:13.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.238+0000 7fc979c92640 1 --2- 172.21.15.32:0/1767167097 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc97415fd60 0x7fc97406c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:13.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.238+0000 7fc97a493640 1 --2- 172.21.15.32:0/1767167097 >> v2:172.21.15.121:3300/0 conn(0x7fc97415b480 0x7fc9740b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:13.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.238+0000 7fc97ac94640 1 -- 172.21.15.32:0/1767167097 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc97415fd60 msgr2=0x7fc97406c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:13.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.238+0000 7fc97ac94640 1 --2- 172.21.15.32:0/1767167097 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc97415fd60 0x7fc97406c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:13.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.238+0000 7fc97ac94640 1 -- 172.21.15.32:0/1767167097 >> v2:172.21.15.121:3300/0 conn(0x7fc97415b480 msgr2=0x7fc9740b5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:13.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.238+0000 7fc97ac94640 1 --2- 172.21.15.32:0/1767167097 >> v2:172.21.15.121:3300/0 conn(0x7fc97415b480 0x7fc9740b5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:13.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.239+0000 7fc97ac94640 1 -- 172.21.15.32:0/1767167097 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9740b4c80 con 0x7fc97406cb30 2024-04-24T23:06:13.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.239+0000 7fc97a493640 1 --2- 172.21.15.32:0/1767167097 >> v2:172.21.15.121:3300/0 conn(0x7fc97415b480 0x7fc9740b5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:06:13.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.239+0000 7fc97ac94640 1 --2- 172.21.15.32:0/1767167097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc97406cb30 0x7fc9740ccc90 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fc97406bf90 tx=0x7fc96402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39eced3d7be3cafe server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:13.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.239+0000 7fc979491640 1 -- 172.21.15.32:0/1767167097 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc964037070 con 0x7fc97406cb30 2024-04-24T23:06:13.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.239+0000 7fc979491640 1 -- 172.21.15.32:0/1767167097 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc964032440 con 0x7fc97406cb30 2024-04-24T23:06:13.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.239+0000 7fc979491640 1 -- 172.21.15.32:0/1767167097 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc964032810 con 0x7fc97406cb30 2024-04-24T23:06:13.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.240+0000 7fc97c6f7640 1 -- 172.21.15.32:0/1767167097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc97406cb30 msgr2=0x7fc9740ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:13.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.240+0000 7fc97c6f7640 1 --2- 172.21.15.32:0/1767167097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc97406cb30 0x7fc9740ccc90 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fc97406bf90 tx=0x7fc96402a1e0 comp rx=0 tx=0).stop 2024-04-24T23:06:13.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.240+0000 7fc97c6f7640 1 -- 172.21.15.32:0/1767167097 shutdown_connections 2024-04-24T23:06:13.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.240+0000 7fc97c6f7640 1 --2- 172.21.15.32:0/1767167097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc97406cb30 0x7fc9740ccc90 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:13.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.240+0000 7fc97c6f7640 1 --2- 172.21.15.32:0/1767167097 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc97415fd60 0x7fc97406c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:13.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.240+0000 7fc97c6f7640 1 --2- 172.21.15.32:0/1767167097 >> v2:172.21.15.121:3300/0 conn(0x7fc97415b480 0x7fc9740b5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:13.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.240+0000 7fc97c6f7640 1 -- 172.21.15.32:0/1767167097 >> 172.21.15.32:0/1767167097 conn(0x7fc97406b110 msgr2=0x7fc97406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:13.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.240+0000 7fc97c6f7640 1 -- 172.21.15.32:0/1767167097 shutdown_connections 2024-04-24T23:06:13.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.240+0000 7fc97c6f7640 1 -- 172.21.15.32:0/1767167097 wait complete. 2024-04-24T23:06:13.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.241+0000 7fc97c6f7640 1 Processor -- start 2024-04-24T23:06:13.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.241+0000 7fc97c6f7640 1 -- start start 2024-04-24T23:06:13.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.242+0000 7fc97c6f7640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc97406cb30 0x7fc97415dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:13.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.242+0000 7fc97c6f7640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc97415b480 0x7fc97415a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:13.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.242+0000 7fc97c6f7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc97415fd60 0x7fc97414ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:13.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.242+0000 7fc97c6f7640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc97415b860 con 0x7fc97415fd60 2024-04-24T23:06:13.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.242+0000 7fc97c6f7640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc9740b5480 con 0x7fc97415b480 2024-04-24T23:06:13.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.242+0000 7fc97c6f7640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc974149c00 con 0x7fc97406cb30 2024-04-24T23:06:13.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.242+0000 7fc97ac94640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc97415fd60 0x7fc97414ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:13.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.242+0000 7fc97ac94640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc97415fd60 0x7fc97414ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34782/0 (socket says 172.21.15.32:34782) 2024-04-24T23:06:13.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.242+0000 7fc97ac94640 1 -- 172.21.15.32:0/3080201456 learned_addr learned my addr 172.21.15.32:0/3080201456 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:13.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.242+0000 7fc979c92640 1 --2- 172.21.15.32:0/3080201456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc97415b480 0x7fc97415a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:13.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.242+0000 7fc97a493640 1 --2- 172.21.15.32:0/3080201456 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc97406cb30 0x7fc97415dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:13.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.242+0000 7fc97ac94640 1 -- 172.21.15.32:0/3080201456 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc97406cb30 msgr2=0x7fc97415dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:13.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.242+0000 7fc97ac94640 1 --2- 172.21.15.32:0/3080201456 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc97406cb30 0x7fc97415dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:13.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.242+0000 7fc97ac94640 1 -- 172.21.15.32:0/3080201456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc97415b480 msgr2=0x7fc97415a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:13.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.242+0000 7fc97ac94640 1 --2- 172.21.15.32:0/3080201456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc97415b480 0x7fc97415a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:13.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.242+0000 7fc97ac94640 1 -- 172.21.15.32:0/3080201456 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc97415e950 con 0x7fc97415fd60 2024-04-24T23:06:13.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.242+0000 7fc97a493640 1 --2- 172.21.15.32:0/3080201456 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc97406cb30 0x7fc97415dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:06:13.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.243+0000 7fc97ac94640 1 --2- 172.21.15.32:0/3080201456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc97415fd60 0x7fc97414ff00 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fc9640090b0 tx=0x7fc9640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:13.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.243+0000 7fc972ffd640 1 -- 172.21.15.32:0/3080201456 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc964041070 con 0x7fc97415fd60 2024-04-24T23:06:13.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.243+0000 7fc972ffd640 1 -- 172.21.15.32:0/3080201456 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc96403c870 con 0x7fc97415fd60 2024-04-24T23:06:13.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.243+0000 7fc97c6f7640 1 -- 172.21.15.32:0/3080201456 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc97415e390 con 0x7fc97415fd60 2024-04-24T23:06:13.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.243+0000 7fc972ffd640 1 -- 172.21.15.32:0/3080201456 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc964037040 con 0x7fc97415fd60 2024-04-24T23:06:13.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.243+0000 7fc97c6f7640 1 -- 172.21.15.32:0/3080201456 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc974149890 con 0x7fc97415fd60 2024-04-24T23:06:13.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.244+0000 7fc97c6f7640 1 -- 172.21.15.32:0/3080201456 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc974068430 con 0x7fc97415fd60 2024-04-24T23:06:13.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.244+0000 7fc972ffd640 1 -- 172.21.15.32:0/3080201456 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc964044070 con 0x7fc97415fd60 2024-04-24T23:06:13.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.245+0000 7fc972ffd640 1 --2- 172.21.15.32:0/3080201456 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc96006be30 0x7fc96006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:13.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.245+0000 7fc972ffd640 1 -- 172.21.15.32:0/3080201456 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5796+0+0 (secure 0 0 0) 0x7fc9640aba30 con 0x7fc97415fd60 2024-04-24T23:06:13.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.245+0000 7fc97a493640 1 --2- 172.21.15.32:0/3080201456 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc96006be30 0x7fc96006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:13.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.246+0000 7fc97a493640 1 --2- 172.21.15.32:0/3080201456 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc96006be30 0x7fc96006e2f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc9740af440 tx=0x7fc96c007670 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:13.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.249+0000 7fc972ffd640 1 -- 172.21.15.32:0/3080201456 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc964079e50 con 0x7fc97415fd60 2024-04-24T23:06:13.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.434+0000 7fc97c6f7640 1 -- 172.21.15.32:0/3080201456 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc97414b2a0 con 0x7fc97415fd60 2024-04-24T23:06:13.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.435+0000 7fc972ffd640 1 -- 172.21.15.32:0/3080201456 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) ==== 74+0+16836 (secure 0 0 0) 0x7fc96407e150 con 0x7fc97415fd60 2024-04-24T23:06:13.438 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:06:13.438 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":145,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:06:06.811191+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:05:32.064234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-24T23:05:32.760777+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.17","mappings":[{"from":4,"to":3}]},{"pgid":"17.8","mappings":[{"from":6,"to":5}]},{"pgid":"17.3f","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:06:13.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.437+0000 7fc97c6f7640 1 -- 172.21.15.32:0/3080201456 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc96006be30 msgr2=0x7fc96006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:13.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.437+0000 7fc97c6f7640 1 --2- 172.21.15.32:0/3080201456 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc96006be30 0x7fc96006e2f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc9740af440 tx=0x7fc96c007670 comp rx=0 tx=0).stop 2024-04-24T23:06:13.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.437+0000 7fc97c6f7640 1 -- 172.21.15.32:0/3080201456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc97415fd60 msgr2=0x7fc97414ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:13.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.437+0000 7fc97c6f7640 1 --2- 172.21.15.32:0/3080201456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc97415fd60 0x7fc97414ff00 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fc9640090b0 tx=0x7fc9640046c0 comp rx=0 tx=0).stop 2024-04-24T23:06:13.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.437+0000 7fc97c6f7640 1 -- 172.21.15.32:0/3080201456 shutdown_connections 2024-04-24T23:06:13.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.438+0000 7fc97c6f7640 1 --2- 172.21.15.32:0/3080201456 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc96006be30 0x7fc96006e2f0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:13.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.438+0000 7fc97c6f7640 1 --2- 172.21.15.32:0/3080201456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc97415fd60 0x7fc97414ff00 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:13.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.438+0000 7fc97c6f7640 1 --2- 172.21.15.32:0/3080201456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc97415b480 0x7fc97415a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:13.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.438+0000 7fc97c6f7640 1 --2- 172.21.15.32:0/3080201456 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc97406cb30 0x7fc97415dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:13.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.438+0000 7fc97c6f7640 1 -- 172.21.15.32:0/3080201456 >> 172.21.15.32:0/3080201456 conn(0x7fc97406b110 msgr2=0x7fc9740d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:13.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.438+0000 7fc97c6f7640 1 -- 172.21.15.32:0/3080201456 shutdown_connections 2024-04-24T23:06:13.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.438+0000 7fc97c6f7640 1 -- 172.21.15.32:0/3080201456 wait complete. 2024-04-24T23:06:13.450 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:05:32.064234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '139', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:05:32.760777+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '145', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:06:13.451 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:06:13.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.554+0000 7feac97e7640 1 Processor -- start 2024-04-24T23:06:13.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.554+0000 7feac97e7640 1 -- start start 2024-04-24T23:06:13.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.555+0000 7feac97e7640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7feac406c480 0x7feac40b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:13.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.555+0000 7feac97e7640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feac40b0770 con 0x7feac4074210 2024-04-24T23:06:13.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.555+0000 7feac97e7640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feac40b1010 con 0x7feac40b0b80 2024-04-24T23:06:13.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.555+0000 7feac97e7640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7feac406c8d0 con 0x7feac406c480 2024-04-24T23:06:13.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.555+0000 7feac37fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7feac406c480 0x7feac40b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:13.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.555+0000 7feac37fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7feac406c480 0x7feac40b20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34796/0 (socket says 172.21.15.32:34796) 2024-04-24T23:06:13.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.555+0000 7feac37fe640 1 -- 172.21.15.32:0/1510920900 learned_addr learned my addr 172.21.15.32:0/1510920900 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:13.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.555+0000 7feac2ffd640 1 --1- 172.21.15.32:0/1510920900 >> v1:172.21.15.32:6789/0 conn(0x7feac4074210 0x7feac40745f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41168/0 (socket says 172.21.15.32:41168) 2024-04-24T23:06:13.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.555+0000 7feac37fe640 1 -- 172.21.15.32:0/1510920900 >> v1:172.21.15.121:6789/0 conn(0x7feac40b0b80 legacy=0x7feac40ae140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:13.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.555+0000 7feac37fe640 1 -- 172.21.15.32:0/1510920900 >> v1:172.21.15.32:6789/0 conn(0x7feac4074210 legacy=0x7feac40745f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:13.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.555+0000 7feac27fc640 1 -- 172.21.15.32:0/1510920900 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1459091492 0 0) 0x7feac40b0770 con 0x7feac4074210 2024-04-24T23:06:13.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.555+0000 7feac37fe640 1 -- 172.21.15.32:0/1510920900 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feac40b12f0 con 0x7feac406c480 2024-04-24T23:06:13.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.556+0000 7feac37fe640 1 --2- 172.21.15.32:0/1510920900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feac406c480 0x7feac40b20b0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7feabc009e50 tx=0x7feabc02a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11d2157fa7bac949 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:13.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.556+0000 7feac27fc640 1 -- 172.21.15.32:0/1510920900 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feabc036070 con 0x7feac406c480 2024-04-24T23:06:13.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.556+0000 7feac27fc640 1 -- 172.21.15.32:0/1510920900 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feabc031600 con 0x7feac406c480 2024-04-24T23:06:13.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.556+0000 7feac27fc640 1 -- 172.21.15.32:0/1510920900 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feabc0319d0 con 0x7feac406c480 2024-04-24T23:06:13.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.557+0000 7feac97e7640 1 -- 172.21.15.32:0/1510920900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feac406c480 msgr2=0x7feac40b20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:13.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.557+0000 7feac97e7640 1 --2- 172.21.15.32:0/1510920900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feac406c480 0x7feac40b20b0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7feabc009e50 tx=0x7feabc02a760 comp rx=0 tx=0).stop 2024-04-24T23:06:13.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.557+0000 7feac97e7640 1 -- 172.21.15.32:0/1510920900 shutdown_connections 2024-04-24T23:06:13.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.557+0000 7feac97e7640 1 --2- 172.21.15.32:0/1510920900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feac406c480 0x7feac40b20b0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:13.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.557+0000 7feac97e7640 1 -- 172.21.15.32:0/1510920900 >> 172.21.15.32:0/1510920900 conn(0x7feac40b2e60 msgr2=0x7feac40b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:13.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.557+0000 7feac97e7640 1 -- 172.21.15.32:0/1510920900 shutdown_connections 2024-04-24T23:06:13.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.557+0000 7feac97e7640 1 -- 172.21.15.32:0/1510920900 wait complete. 2024-04-24T23:06:13.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.558+0000 7feac97e7640 1 Processor -- start 2024-04-24T23:06:13.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.558+0000 7feac97e7640 1 -- start start 2024-04-24T23:06:13.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.558+0000 7feac97e7640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feac406c480 0x7feac40cce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:13.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.558+0000 7feac97e7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feac4074210 0x7feac406e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:13.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.558+0000 7feac97e7640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feac40c79e0 0x7feac415d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:13.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.558+0000 7feac97e7640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7feac40b18b0 con 0x7feac4074210 2024-04-24T23:06:13.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.559+0000 7feac97e7640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7feac40b1c20 con 0x7feac40c79e0 2024-04-24T23:06:13.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.559+0000 7feac97e7640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7feac40ae850 con 0x7feac406c480 2024-04-24T23:06:13.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.559+0000 7feabbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feac4074210 0x7feac406e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:13.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.559+0000 7feabbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feac4074210 0x7feac406e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34806/0 (socket says 172.21.15.32:34806) 2024-04-24T23:06:13.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.559+0000 7feabbfff640 1 -- 172.21.15.32:0/3602796742 learned_addr learned my addr 172.21.15.32:0/3602796742 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:13.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.559+0000 7feac37fe640 1 --2- 172.21.15.32:0/3602796742 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feac40c79e0 0x7feac415d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:13.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.559+0000 7feac2ffd640 1 --2- 172.21.15.32:0/3602796742 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feac406c480 0x7feac40cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:13.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.559+0000 7feabbfff640 1 -- 172.21.15.32:0/3602796742 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feac406c480 msgr2=0x7feac40cce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:13.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.559+0000 7feabbfff640 1 --2- 172.21.15.32:0/3602796742 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feac406c480 0x7feac40cce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:13.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.559+0000 7feabbfff640 1 -- 172.21.15.32:0/3602796742 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feac40c79e0 msgr2=0x7feac415d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:13.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.559+0000 7feabbfff640 1 --2- 172.21.15.32:0/3602796742 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feac40c79e0 0x7feac415d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:13.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.559+0000 7feabbfff640 1 -- 172.21.15.32:0/3602796742 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feac41f4c00 con 0x7feac4074210 2024-04-24T23:06:13.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.559+0000 7feabbfff640 1 --2- 172.21.15.32:0/3602796742 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feac4074210 0x7feac406e990 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7feaac00b420 tx=0x7feaac00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:13.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.559+0000 7feabb7fe640 1 -- 172.21.15.32:0/3602796742 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaac011830 con 0x7feac4074210 2024-04-24T23:06:13.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.559+0000 7feabb7fe640 1 -- 172.21.15.32:0/3602796742 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaac016070 con 0x7feac4074210 2024-04-24T23:06:13.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.559+0000 7feac97e7640 1 -- 172.21.15.32:0/3602796742 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feac41f4f20 con 0x7feac4074210 2024-04-24T23:06:13.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.559+0000 7feabb7fe640 1 -- 172.21.15.32:0/3602796742 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaac011c10 con 0x7feac4074210 2024-04-24T23:06:13.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.559+0000 7feac97e7640 1 -- 172.21.15.32:0/3602796742 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feac41f5760 con 0x7feac4074210 2024-04-24T23:06:13.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.560+0000 7feac97e7640 1 -- 172.21.15.32:0/3602796742 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feac4162200 con 0x7feac4074210 2024-04-24T23:06:13.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.561+0000 7feabb7fe640 1 -- 172.21.15.32:0/3602796742 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feaac01b460 con 0x7feac4074210 2024-04-24T23:06:13.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.561+0000 7feabb7fe640 1 --2- 172.21.15.32:0/3602796742 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7feaa406be30 0x7feaa406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:13.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.561+0000 7feabb7fe640 1 -- 172.21.15.32:0/3602796742 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5796+0+0 (secure 0 0 0) 0x7feaac08aa20 con 0x7feac4074210 2024-04-24T23:06:13.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.561+0000 7feac2ffd640 1 --2- 172.21.15.32:0/3602796742 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7feaa406be30 0x7feaa406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:13.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.562+0000 7feac2ffd640 1 --2- 172.21.15.32:0/3602796742 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7feaa406be30 0x7feaa406e2f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7feab4000f60 tx=0x7feab4007b50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:13.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.566+0000 7feabb7fe640 1 -- 172.21.15.32:0/3602796742 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7feaac020020 con 0x7feac4074210 2024-04-24T23:06:13.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.746+0000 7feac97e7640 1 -- 172.21.15.32:0/3602796742 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7feac406b220 con 0x7feac4074210 2024-04-24T23:06:13.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.747+0000 7feabb7fe640 1 -- 172.21.15.32:0/3602796742 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v145) ==== 93+0+10 (secure 0 0 0) 0x7feac41f4f20 con 0x7feac4074210 2024-04-24T23:06:13.749 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T23:06:13.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.748+0000 7feac97e7640 1 -- 172.21.15.32:0/3602796742 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7feaa406be30 msgr2=0x7feaa406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:13.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.748+0000 7feac97e7640 1 --2- 172.21.15.32:0/3602796742 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7feaa406be30 0x7feaa406e2f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7feab4000f60 tx=0x7feab4007b50 comp rx=0 tx=0).stop 2024-04-24T23:06:13.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.748+0000 7feac97e7640 1 -- 172.21.15.32:0/3602796742 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feac4074210 msgr2=0x7feac406e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:13.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.748+0000 7feac97e7640 1 --2- 172.21.15.32:0/3602796742 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feac4074210 0x7feac406e990 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7feaac00b420 tx=0x7feaac00b8e0 comp rx=0 tx=0).stop 2024-04-24T23:06:13.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.748+0000 7feac97e7640 1 -- 172.21.15.32:0/3602796742 shutdown_connections 2024-04-24T23:06:13.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.748+0000 7feac97e7640 1 --2- 172.21.15.32:0/3602796742 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7feaa406be30 0x7feaa406e2f0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:13.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.748+0000 7feac97e7640 1 --2- 172.21.15.32:0/3602796742 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feac4074210 0x7feac406e990 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:13.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.748+0000 7feac97e7640 1 --2- 172.21.15.32:0/3602796742 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feac40c79e0 0x7feac415d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:13.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.748+0000 7feac97e7640 1 --2- 172.21.15.32:0/3602796742 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feac406c480 0x7feac40cce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:13.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.748+0000 7feac97e7640 1 -- 172.21.15.32:0/3602796742 >> 172.21.15.32:0/3602796742 conn(0x7feac40b2e60 msgr2=0x7feac40af450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:13.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.749+0000 7feac97e7640 1 -- 172.21.15.32:0/3602796742 shutdown_connections 2024-04-24T23:06:13.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.749+0000 7feac97e7640 1 -- 172.21.15.32:0/3602796742 wait complete. 2024-04-24T23:06:13.761 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:06:13.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.866+0000 7fd612777640 1 Processor -- start 2024-04-24T23:06:13.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.866+0000 7fd612777640 1 -- start start 2024-04-24T23:06:13.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.866+0000 7fd612777640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd60c06b820 0x7fd60c0d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:13.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.866+0000 7fd612777640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd60c0d4210 0x7fd60c0b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:13.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.866+0000 7fd612777640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd60c0d3170 con 0x7fd60c15e9b0 2024-04-24T23:06:13.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.866+0000 7fd612777640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fd60c0d3450 con 0x7fd60c06b820 2024-04-24T23:06:13.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.866+0000 7fd612777640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fd60c0d3730 con 0x7fd60c0d4210 2024-04-24T23:06:13.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.866+0000 7fd60b7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd60c06b820 0x7fd60c0d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:13.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.867+0000 7fd60b7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd60c06b820 0x7fd60c0d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34812/0 (socket says 172.21.15.32:34812) 2024-04-24T23:06:13.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.867+0000 7fd60b7fe640 1 -- 172.21.15.32:0/3433755802 learned_addr learned my addr 172.21.15.32:0/3433755802 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:13.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.867+0000 7fd610d14640 1 --2- 172.21.15.32:0/3433755802 >> v2:172.21.15.121:3300/0 conn(0x7fd60c0d4210 0x7fd60c0b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:13.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.867+0000 7fd60b7fe640 1 -- 172.21.15.32:0/3433755802 >> v2:172.21.15.121:3300/0 conn(0x7fd60c0d4210 msgr2=0x7fd60c0b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:13.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.867+0000 7fd60b7fe640 1 --2- 172.21.15.32:0/3433755802 >> v2:172.21.15.121:3300/0 conn(0x7fd60c0d4210 0x7fd60c0b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:13.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.867+0000 7fd60b7fe640 1 -- 172.21.15.32:0/3433755802 >> v1:172.21.15.121:6789/0 conn(0x7fd60c15e9b0 legacy=0x7fd60c06b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:13.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.867+0000 7fd60b7fe640 1 -- 172.21.15.32:0/3433755802 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd60c0d3a10 con 0x7fd60c06b820 2024-04-24T23:06:13.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.868+0000 7fd60b7fe640 1 --2- 172.21.15.32:0/3433755802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd60c06b820 0x7fd60c0d2b90 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fd60c1622f0 tx=0x7fd60002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38eb4518585e4044 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:13.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.868+0000 7fd60affd640 1 -- 172.21.15.32:0/3433755802 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd600037070 con 0x7fd60c06b820 2024-04-24T23:06:13.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.868+0000 7fd60affd640 1 -- 172.21.15.32:0/3433755802 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd600032440 con 0x7fd60c06b820 2024-04-24T23:06:13.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.868+0000 7fd60affd640 1 -- 172.21.15.32:0/3433755802 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd600032810 con 0x7fd60c06b820 2024-04-24T23:06:13.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.868+0000 7fd612777640 1 -- 172.21.15.32:0/3433755802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd60c06b820 msgr2=0x7fd60c0d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:13.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.868+0000 7fd612777640 1 --2- 172.21.15.32:0/3433755802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd60c06b820 0x7fd60c0d2b90 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fd60c1622f0 tx=0x7fd60002a1e0 comp rx=0 tx=0).stop 2024-04-24T23:06:13.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.868+0000 7fd612777640 1 -- 172.21.15.32:0/3433755802 shutdown_connections 2024-04-24T23:06:13.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.868+0000 7fd612777640 1 --2- 172.21.15.32:0/3433755802 >> v2:172.21.15.121:3300/0 conn(0x7fd60c0d4210 0x7fd60c0b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:13.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.868+0000 7fd612777640 1 --2- 172.21.15.32:0/3433755802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd60c06b820 0x7fd60c0d2b90 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:13.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.868+0000 7fd612777640 1 -- 172.21.15.32:0/3433755802 >> 172.21.15.32:0/3433755802 conn(0x7fd60c0d11d0 msgr2=0x7fd60c0d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:13.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.868+0000 7fd612777640 1 -- 172.21.15.32:0/3433755802 shutdown_connections 2024-04-24T23:06:13.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.868+0000 7fd612777640 1 -- 172.21.15.32:0/3433755802 wait complete. 2024-04-24T23:06:13.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.869+0000 7fd612777640 1 Processor -- start 2024-04-24T23:06:13.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.870+0000 7fd612777640 1 -- start start 2024-04-24T23:06:13.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.870+0000 7fd612777640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd60c06b820 0x7fd60c158fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:13.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.870+0000 7fd612777640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd60c0d4210 0x7fd60c0cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:13.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.870+0000 7fd612777640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd60c15e9b0 0x7fd60c1f3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:13.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.870+0000 7fd612777640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd60c0ba0e0 con 0x7fd60c06b820 2024-04-24T23:06:13.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.870+0000 7fd612777640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd60c0d2400 con 0x7fd60c0d4210 2024-04-24T23:06:13.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.870+0000 7fd612777640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd60c0ba260 con 0x7fd60c15e9b0 2024-04-24T23:06:13.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.870+0000 7fd60bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd60c06b820 0x7fd60c158fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:13.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.871+0000 7fd60bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd60c06b820 0x7fd60c158fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34818/0 (socket says 172.21.15.32:34818) 2024-04-24T23:06:13.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.871+0000 7fd60bfff640 1 -- 172.21.15.32:0/1825740929 learned_addr learned my addr 172.21.15.32:0/1825740929 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:13.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.871+0000 7fd610d14640 1 --2- 172.21.15.32:0/1825740929 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd60c15e9b0 0x7fd60c1f3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:13.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.871+0000 7fd60b7fe640 1 --2- 172.21.15.32:0/1825740929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd60c0d4210 0x7fd60c0cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:13.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.871+0000 7fd60bfff640 1 -- 172.21.15.32:0/1825740929 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd60c15e9b0 msgr2=0x7fd60c1f3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:13.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.871+0000 7fd60bfff640 1 --2- 172.21.15.32:0/1825740929 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd60c15e9b0 0x7fd60c1f3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:13.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.871+0000 7fd60bfff640 1 -- 172.21.15.32:0/1825740929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd60c0d4210 msgr2=0x7fd60c0cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:13.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.871+0000 7fd60bfff640 1 --2- 172.21.15.32:0/1825740929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd60c0d4210 0x7fd60c0cd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:13.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.871+0000 7fd60bfff640 1 -- 172.21.15.32:0/1825740929 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd60c1581f0 con 0x7fd60c06b820 2024-04-24T23:06:13.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.871+0000 7fd60b7fe640 1 --2- 172.21.15.32:0/1825740929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd60c0d4210 0x7fd60c0cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:06:13.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.871+0000 7fd60bfff640 1 --2- 172.21.15.32:0/1825740929 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd60c06b820 0x7fd60c158fb0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc00c3c0 tx=0x7fd5fc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:13.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.871+0000 7fd608ff9640 1 -- 172.21.15.32:0/1825740929 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc0052d0 con 0x7fd60c06b820 2024-04-24T23:06:13.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.871+0000 7fd608ff9640 1 -- 172.21.15.32:0/1825740929 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5fc005490 con 0x7fd60c06b820 2024-04-24T23:06:13.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.871+0000 7fd612777640 1 -- 172.21.15.32:0/1825740929 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd60c1584d0 con 0x7fd60c06b820 2024-04-24T23:06:13.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.871+0000 7fd608ff9640 1 -- 172.21.15.32:0/1825740929 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc007260 con 0x7fd60c06b820 2024-04-24T23:06:13.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.871+0000 7fd612777640 1 -- 172.21.15.32:0/1825740929 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd60c158a90 con 0x7fd60c06b820 2024-04-24T23:06:13.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.872+0000 7fd612777640 1 -- 172.21.15.32:0/1825740929 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd60c068430 con 0x7fd60c06b820 2024-04-24T23:06:13.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.873+0000 7fd608ff9640 1 -- 172.21.15.32:0/1825740929 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd5fc007400 con 0x7fd60c06b820 2024-04-24T23:06:13.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.873+0000 7fd608ff9640 1 --2- 172.21.15.32:0/1825740929 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd5f806be30 0x7fd5f806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:13.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.873+0000 7fd608ff9640 1 -- 172.21.15.32:0/1825740929 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5796+0+0 (secure 0 0 0) 0x7fd5fc08a430 con 0x7fd60c06b820 2024-04-24T23:06:13.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.873+0000 7fd60b7fe640 1 --2- 172.21.15.32:0/1825740929 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd5f806be30 0x7fd5f806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:13.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.874+0000 7fd60b7fe640 1 --2- 172.21.15.32:0/1825740929 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd5f806be30 0x7fd5f806e2f0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd600034000 tx=0x7fd60002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:13.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:13.877+0000 7fd608ff9640 1 -- 172.21.15.32:0/1825740929 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd5fc058cf0 con 0x7fd60c06b820 2024-04-24T23:06:14.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.057+0000 7fd612777640 1 -- 172.21.15.32:0/1825740929 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd60c158cd0 con 0x7fd60c06b820 2024-04-24T23:06:14.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.058+0000 7fd608ff9640 1 -- 172.21.15.32:0/1825740929 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v145) ==== 104+0+11 (secure 0 0 0) 0x7fd5fc0585f0 con 0x7fd60c06b820 2024-04-24T23:06:14.061 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-24T23:06:14.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.060+0000 7fd612777640 1 -- 172.21.15.32:0/1825740929 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd5f806be30 msgr2=0x7fd5f806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:14.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.060+0000 7fd612777640 1 --2- 172.21.15.32:0/1825740929 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd5f806be30 0x7fd5f806e2f0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd600034000 tx=0x7fd60002e000 comp rx=0 tx=0).stop 2024-04-24T23:06:14.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.060+0000 7fd612777640 1 -- 172.21.15.32:0/1825740929 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd60c06b820 msgr2=0x7fd60c158fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:14.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.060+0000 7fd612777640 1 --2- 172.21.15.32:0/1825740929 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd60c06b820 0x7fd60c158fb0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc00c3c0 tx=0x7fd5fc00c880 comp rx=0 tx=0).stop 2024-04-24T23:06:14.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.060+0000 7fd612777640 1 -- 172.21.15.32:0/1825740929 shutdown_connections 2024-04-24T23:06:14.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.060+0000 7fd612777640 1 --2- 172.21.15.32:0/1825740929 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd5f806be30 0x7fd5f806e2f0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:14.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.060+0000 7fd612777640 1 --2- 172.21.15.32:0/1825740929 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd60c15e9b0 0x7fd60c1f3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:14.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.060+0000 7fd612777640 1 --2- 172.21.15.32:0/1825740929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd60c0d4210 0x7fd60c0cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:14.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.060+0000 7fd612777640 1 --2- 172.21.15.32:0/1825740929 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd60c06b820 0x7fd60c158fb0 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:14.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.060+0000 7fd612777640 1 -- 172.21.15.32:0/1825740929 >> 172.21.15.32:0/1825740929 conn(0x7fd60c0d11d0 msgr2=0x7fd60c0b7bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:14.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.060+0000 7fd612777640 1 -- 172.21.15.32:0/1825740929 shutdown_connections 2024-04-24T23:06:14.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.060+0000 7fd612777640 1 -- 172.21.15.32:0/1825740929 wait complete. 2024-04-24T23:06:14.072 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:06:14.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.179+0000 7fe80e62c640 1 Processor -- start 2024-04-24T23:06:14.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.179+0000 7fe80e62c640 1 -- start start 2024-04-24T23:06:14.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.180+0000 7fe80e62c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe8081476d0 0x7fe8080ce630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:14.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.180+0000 7fe80e62c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe8080cfb70 0x7fe80806b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:14.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.180+0000 7fe80e62c640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8080afdd0 con 0x7fe80806b650 2024-04-24T23:06:14.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.180+0000 7fe80e62c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fe8080aedc0 con 0x7fe8081476d0 2024-04-24T23:06:14.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.180+0000 7fe80e62c640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe8080af940 con 0x7fe8080cfb70 2024-04-24T23:06:14.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.180+0000 7fe807fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe8081476d0 0x7fe8080ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:14.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.180+0000 7fe807fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe8081476d0 0x7fe8080ce630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34826/0 (socket says 172.21.15.32:34826) 2024-04-24T23:06:14.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.180+0000 7fe807fff640 1 -- 172.21.15.32:0/825511068 learned_addr learned my addr 172.21.15.32:0/825511068 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:14.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.180+0000 7fe8077fe640 1 --2- 172.21.15.32:0/825511068 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe8080cfb70 0x7fe80806b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:14.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.180+0000 7fe807fff640 1 -- 172.21.15.32:0/825511068 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe8080cfb70 msgr2=0x7fe80806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:14.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.180+0000 7fe807fff640 1 --2- 172.21.15.32:0/825511068 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe8080cfb70 0x7fe80806b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:14.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.180+0000 7fe807fff640 1 -- 172.21.15.32:0/825511068 >> v1:172.21.15.121:6789/0 conn(0x7fe80806b650 legacy=0x7fe8080cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:14.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.180+0000 7fe807fff640 1 -- 172.21.15.32:0/825511068 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8080af010 con 0x7fe8081476d0 2024-04-24T23:06:14.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.180+0000 7fe8077fe640 1 --2- 172.21.15.32:0/825511068 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe8080cfb70 0x7fe80806b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:06:14.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.181+0000 7fe807fff640 1 --2- 172.21.15.32:0/825511068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8081476d0 0x7fe8080ce630 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0009870 tx=0x7fe7f002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c415aabf44a671d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:14.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.181+0000 7fe806ffd640 1 -- 172.21.15.32:0/825511068 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f0037070 con 0x7fe8081476d0 2024-04-24T23:06:14.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.181+0000 7fe806ffd640 1 -- 172.21.15.32:0/825511068 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7f0032440 con 0x7fe8081476d0 2024-04-24T23:06:14.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.181+0000 7fe806ffd640 1 -- 172.21.15.32:0/825511068 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f0032810 con 0x7fe8081476d0 2024-04-24T23:06:14.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.182+0000 7fe80e62c640 1 -- 172.21.15.32:0/825511068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8081476d0 msgr2=0x7fe8080ce630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:14.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.182+0000 7fe80e62c640 1 --2- 172.21.15.32:0/825511068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8081476d0 0x7fe8080ce630 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0009870 tx=0x7fe7f002a1c0 comp rx=0 tx=0).stop 2024-04-24T23:06:14.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.182+0000 7fe80e62c640 1 -- 172.21.15.32:0/825511068 shutdown_connections 2024-04-24T23:06:14.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.182+0000 7fe80e62c640 1 --2- 172.21.15.32:0/825511068 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe8080cfb70 0x7fe80806b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:14.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.182+0000 7fe80e62c640 1 --2- 172.21.15.32:0/825511068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8081476d0 0x7fe8080ce630 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:14.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.182+0000 7fe80e62c640 1 -- 172.21.15.32:0/825511068 >> 172.21.15.32:0/825511068 conn(0x7fe8080d0730 msgr2=0x7fe8080d0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:14.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.182+0000 7fe80e62c640 1 -- 172.21.15.32:0/825511068 shutdown_connections 2024-04-24T23:06:14.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.182+0000 7fe80e62c640 1 -- 172.21.15.32:0/825511068 wait complete. 2024-04-24T23:06:14.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.183+0000 7fe80e62c640 1 Processor -- start 2024-04-24T23:06:14.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.183+0000 7fe80e62c640 1 -- start start 2024-04-24T23:06:14.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.184+0000 7fe80e62c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe80806b650 0x7fe8080b1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:14.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.184+0000 7fe80e62c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8080cfb70 0x7fe8080cdea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:14.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.184+0000 7fe80e62c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe8081476d0 0x7fe80813f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:14.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.184+0000 7fe80e62c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe808147ab0 con 0x7fe80806b650 2024-04-24T23:06:14.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.184+0000 7fe80e62c640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe8080af5d0 con 0x7fe8080cfb70 2024-04-24T23:06:14.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.184+0000 7fe80e62c640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe808148470 con 0x7fe8081476d0 2024-04-24T23:06:14.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.184+0000 7fe807fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe80806b650 0x7fe8080b1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:14.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.184+0000 7fe807fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe80806b650 0x7fe8080b1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34830/0 (socket says 172.21.15.32:34830) 2024-04-24T23:06:14.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.184+0000 7fe807fff640 1 -- 172.21.15.32:0/3684369645 learned_addr learned my addr 172.21.15.32:0/3684369645 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:14.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.184+0000 7fe80cbc9640 1 --2- 172.21.15.32:0/3684369645 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe8081476d0 0x7fe80813f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:14.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.184+0000 7fe8077fe640 1 --2- 172.21.15.32:0/3684369645 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8080cfb70 0x7fe8080cdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:14.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.184+0000 7fe807fff640 1 -- 172.21.15.32:0/3684369645 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe8081476d0 msgr2=0x7fe80813f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:14.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.184+0000 7fe807fff640 1 --2- 172.21.15.32:0/3684369645 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe8081476d0 0x7fe80813f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:14.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.184+0000 7fe807fff640 1 -- 172.21.15.32:0/3684369645 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8080cfb70 msgr2=0x7fe8080cdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:14.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.184+0000 7fe807fff640 1 --2- 172.21.15.32:0/3684369645 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8080cfb70 0x7fe8080cdea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:14.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.184+0000 7fe807fff640 1 -- 172.21.15.32:0/3684369645 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8080b2490 con 0x7fe80806b650 2024-04-24T23:06:14.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.184+0000 7fe807fff640 1 --2- 172.21.15.32:0/3684369645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe80806b650 0x7fe8080b1eb0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0000c00 tx=0x7fe7f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:14.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.185+0000 7fe804ff9640 1 -- 172.21.15.32:0/3684369645 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f0041020 con 0x7fe80806b650 2024-04-24T23:06:14.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.185+0000 7fe804ff9640 1 -- 172.21.15.32:0/3684369645 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7f00306c0 con 0x7fe80806b650 2024-04-24T23:06:14.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.185+0000 7fe804ff9640 1 -- 172.21.15.32:0/3684369645 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f0037040 con 0x7fe80806b650 2024-04-24T23:06:14.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.185+0000 7fe80e62c640 1 -- 172.21.15.32:0/3684369645 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8080b2770 con 0x7fe80806b650 2024-04-24T23:06:14.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.185+0000 7fe80e62c640 1 -- 172.21.15.32:0/3684369645 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8080b2d30 con 0x7fe80806b650 2024-04-24T23:06:14.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.186+0000 7fe80e62c640 1 -- 172.21.15.32:0/3684369645 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe808068430 con 0x7fe80806b650 2024-04-24T23:06:14.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.186+0000 7fe804ff9640 1 -- 172.21.15.32:0/3684369645 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe7f0044020 con 0x7fe80806b650 2024-04-24T23:06:14.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.186+0000 7fe804ff9640 1 --2- 172.21.15.32:0/3684369645 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe7e806be60 0x7fe7e806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:14.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.187+0000 7fe804ff9640 1 -- 172.21.15.32:0/3684369645 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5796+0+0 (secure 0 0 0) 0x7fe7f00ab710 con 0x7fe80806b650 2024-04-24T23:06:14.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.187+0000 7fe8077fe640 1 --2- 172.21.15.32:0/3684369645 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe7e806be60 0x7fe7e806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:14.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.187+0000 7fe8077fe640 1 --2- 172.21.15.32:0/3684369645 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe7e806be60 0x7fe7e806e320 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fe80813ed30 tx=0x7fe7f8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:14.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.191+0000 7fe804ff9640 1 -- 172.21.15.32:0/3684369645 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe7f004b030 con 0x7fe80806b650 2024-04-24T23:06:14.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.372+0000 7fe80e62c640 1 -- 172.21.15.32:0/3684369645 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe808146650 con 0x7fe80806b650 2024-04-24T23:06:14.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.373+0000 7fe804ff9640 1 -- 172.21.15.32:0/3684369645 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v145) ==== 100+0+11 (secure 0 0 0) 0x7fe7f007a050 con 0x7fe80806b650 2024-04-24T23:06:14.375 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-24T23:06:14.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.375+0000 7fe80e62c640 1 -- 172.21.15.32:0/3684369645 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe7e806be60 msgr2=0x7fe7e806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:14.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.375+0000 7fe80e62c640 1 --2- 172.21.15.32:0/3684369645 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe7e806be60 0x7fe7e806e320 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fe80813ed30 tx=0x7fe7f8008040 comp rx=0 tx=0).stop 2024-04-24T23:06:14.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.375+0000 7fe80e62c640 1 -- 172.21.15.32:0/3684369645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe80806b650 msgr2=0x7fe8080b1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:14.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.375+0000 7fe80e62c640 1 --2- 172.21.15.32:0/3684369645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe80806b650 0x7fe8080b1eb0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0000c00 tx=0x7fe7f00046c0 comp rx=0 tx=0).stop 2024-04-24T23:06:14.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.375+0000 7fe80e62c640 1 -- 172.21.15.32:0/3684369645 shutdown_connections 2024-04-24T23:06:14.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.375+0000 7fe80e62c640 1 --2- 172.21.15.32:0/3684369645 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe7e806be60 0x7fe7e806e320 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:14.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.375+0000 7fe80e62c640 1 --2- 172.21.15.32:0/3684369645 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe8081476d0 0x7fe80813f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:14.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.375+0000 7fe80e62c640 1 --2- 172.21.15.32:0/3684369645 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8080cfb70 0x7fe8080cdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:14.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.375+0000 7fe80e62c640 1 --2- 172.21.15.32:0/3684369645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe80806b650 0x7fe8080b1eb0 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:14.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.375+0000 7fe80e62c640 1 -- 172.21.15.32:0/3684369645 >> 172.21.15.32:0/3684369645 conn(0x7fe8080d0730 msgr2=0x7fe8081469a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:14.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.375+0000 7fe80e62c640 1 -- 172.21.15.32:0/3684369645 shutdown_connections 2024-04-24T23:06:14.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.375+0000 7fe80e62c640 1 -- 172.21.15.32:0/3684369645 wait complete. 2024-04-24T23:06:14.387 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T23:06:14.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.491+0000 7fbf14c7b640 1 Processor -- start 2024-04-24T23:06:14.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.491+0000 7fbf14c7b640 1 -- start start 2024-04-24T23:06:14.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.491+0000 7fbf14c7b640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf1006c5f0 0x7fbf1006c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:14.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.491+0000 7fbf14c7b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbf100b44d0 0x7fbf100b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:14.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.491+0000 7fbf14c7b640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf100b6430 con 0x7fbf100af890 2024-04-24T23:06:14.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.491+0000 7fbf14c7b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fbf10156f80 con 0x7fbf100b44d0 2024-04-24T23:06:14.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.491+0000 7fbf14c7b640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fbf100b6150 con 0x7fbf1006c5f0 2024-04-24T23:06:14.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.492+0000 7fbf07fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbf100b44d0 0x7fbf100b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:14.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.492+0000 7fbf07fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbf100b44d0 0x7fbf100b5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34838/0 (socket says 172.21.15.32:34838) 2024-04-24T23:06:14.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.492+0000 7fbf07fff640 1 -- 172.21.15.32:0/2428605477 learned_addr learned my addr 172.21.15.32:0/2428605477 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:14.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.492+0000 7fbf0e59c640 1 --2- 172.21.15.32:0/2428605477 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf1006c5f0 0x7fbf1006c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:14.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.492+0000 7fbf07fff640 1 -- 172.21.15.32:0/2428605477 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf1006c5f0 msgr2=0x7fbf1006c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:14.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.492+0000 7fbf07fff640 1 --2- 172.21.15.32:0/2428605477 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf1006c5f0 0x7fbf1006c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:14.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.492+0000 7fbf07fff640 1 -- 172.21.15.32:0/2428605477 >> v1:172.21.15.121:6789/0 conn(0x7fbf100af890 legacy=0x7fbf100cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:14.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.492+0000 7fbf07fff640 1 -- 172.21.15.32:0/2428605477 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf100b39f0 con 0x7fbf100b44d0 2024-04-24T23:06:14.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.493+0000 7fbf07fff640 1 --2- 172.21.15.32:0/2428605477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf100b44d0 0x7fbf100b5890 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fbef8009870 tx=0x7fbef802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f61e86bf43cac073 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:14.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.493+0000 7fbf0dd9b640 1 -- 172.21.15.32:0/2428605477 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef8037070 con 0x7fbf100b44d0 2024-04-24T23:06:14.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.493+0000 7fbf0dd9b640 1 -- 172.21.15.32:0/2428605477 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbef8032440 con 0x7fbf100b44d0 2024-04-24T23:06:14.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.493+0000 7fbf0dd9b640 1 -- 172.21.15.32:0/2428605477 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef8032810 con 0x7fbf100b44d0 2024-04-24T23:06:14.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.493+0000 7fbf14c7b640 1 -- 172.21.15.32:0/2428605477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf100b44d0 msgr2=0x7fbf100b5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:14.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.493+0000 7fbf14c7b640 1 --2- 172.21.15.32:0/2428605477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf100b44d0 0x7fbf100b5890 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fbef8009870 tx=0x7fbef802a1c0 comp rx=0 tx=0).stop 2024-04-24T23:06:14.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.493+0000 7fbf14c7b640 1 -- 172.21.15.32:0/2428605477 shutdown_connections 2024-04-24T23:06:14.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.493+0000 7fbf14c7b640 1 --2- 172.21.15.32:0/2428605477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf100b44d0 0x7fbf100b5890 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:14.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.493+0000 7fbf14c7b640 1 --2- 172.21.15.32:0/2428605477 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf1006c5f0 0x7fbf1006c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:14.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.493+0000 7fbf14c7b640 1 -- 172.21.15.32:0/2428605477 >> 172.21.15.32:0/2428605477 conn(0x7fbf1006b110 msgr2=0x7fbf1006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:14.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.493+0000 7fbf14c7b640 1 -- 172.21.15.32:0/2428605477 shutdown_connections 2024-04-24T23:06:14.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.493+0000 7fbf14c7b640 1 -- 172.21.15.32:0/2428605477 wait complete. 2024-04-24T23:06:14.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.495+0000 7fbf14c7b640 1 Processor -- start 2024-04-24T23:06:14.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.495+0000 7fbf14c7b640 1 -- start start 2024-04-24T23:06:14.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.495+0000 7fbf14c7b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf1006c5f0 0x7fbf1015dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:14.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.495+0000 7fbf14c7b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf100af890 0x7fbf100ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:14.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.495+0000 7fbf14c7b640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf100b44d0 0x7fbf100aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:14.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.495+0000 7fbf14c7b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1006cf10 con 0x7fbf100af890 2024-04-24T23:06:14.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.495+0000 7fbf14c7b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fbf100b3fb0 con 0x7fbf1006c5f0 2024-04-24T23:06:14.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.495+0000 7fbf14c7b640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fbf100b5dd0 con 0x7fbf100b44d0 2024-04-24T23:06:14.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.496+0000 7fbf07fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf100af890 0x7fbf100ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:14.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.496+0000 7fbf07fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf100af890 0x7fbf100ae8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34846/0 (socket says 172.21.15.32:34846) 2024-04-24T23:06:14.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.496+0000 7fbf07fff640 1 -- 172.21.15.32:0/3482249566 learned_addr learned my addr 172.21.15.32:0/3482249566 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:14.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.496+0000 7fbf0e59c640 1 --2- 172.21.15.32:0/3482249566 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf1006c5f0 0x7fbf1015dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:14.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.496+0000 7fbf0ed9d640 1 --2- 172.21.15.32:0/3482249566 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf100b44d0 0x7fbf100aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:14.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.496+0000 7fbf07fff640 1 -- 172.21.15.32:0/3482249566 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf100b44d0 msgr2=0x7fbf100aedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:14.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.496+0000 7fbf07fff640 1 --2- 172.21.15.32:0/3482249566 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf100b44d0 0x7fbf100aedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:14.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.496+0000 7fbf07fff640 1 -- 172.21.15.32:0/3482249566 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf1006c5f0 msgr2=0x7fbf1015dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:14.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.496+0000 7fbf07fff640 1 --2- 172.21.15.32:0/3482249566 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf1006c5f0 0x7fbf1015dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:14.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.496+0000 7fbf07fff640 1 -- 172.21.15.32:0/3482249566 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf1015ed10 con 0x7fbf100af890 2024-04-24T23:06:14.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.496+0000 7fbf07fff640 1 --2- 172.21.15.32:0/3482249566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf100af890 0x7fbf100ae8b0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fbef8000c00 tx=0x7fbef80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:14.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.496+0000 7fbf06ffd640 1 -- 172.21.15.32:0/3482249566 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef8037040 con 0x7fbf100af890 2024-04-24T23:06:14.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.496+0000 7fbf06ffd640 1 -- 172.21.15.32:0/3482249566 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbef8042070 con 0x7fbf100af890 2024-04-24T23:06:14.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.496+0000 7fbf14c7b640 1 -- 172.21.15.32:0/3482249566 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf1015eff0 con 0x7fbf100af890 2024-04-24T23:06:14.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.496+0000 7fbf06ffd640 1 -- 172.21.15.32:0/3482249566 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef8030860 con 0x7fbf100af890 2024-04-24T23:06:14.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.496+0000 7fbf14c7b640 1 -- 172.21.15.32:0/3482249566 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf100ccfb0 con 0x7fbf100af890 2024-04-24T23:06:14.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.497+0000 7fbf14c7b640 1 -- 172.21.15.32:0/3482249566 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf100ccdc0 con 0x7fbf100af890 2024-04-24T23:06:14.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.498+0000 7fbf06ffd640 1 -- 172.21.15.32:0/3482249566 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbef802ad10 con 0x7fbf100af890 2024-04-24T23:06:14.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.498+0000 7fbf06ffd640 1 --2- 172.21.15.32:0/3482249566 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fbef006be30 0x7fbef006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:14.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.498+0000 7fbf06ffd640 1 -- 172.21.15.32:0/3482249566 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5796+0+0 (secure 0 0 0) 0x7fbef80ab2e0 con 0x7fbf100af890 2024-04-24T23:06:14.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.498+0000 7fbf0e59c640 1 --2- 172.21.15.32:0/3482249566 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fbef006be30 0x7fbef006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:14.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.499+0000 7fbf0e59c640 1 --2- 172.21.15.32:0/3482249566 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fbef006be30 0x7fbef006e2f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fbf00009aa0 tx=0x7fbf00009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:14.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.502+0000 7fbf06ffd640 1 -- 172.21.15.32:0/3482249566 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbef8079670 con 0x7fbf100af890 2024-04-24T23:06:14.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.695+0000 7fbf14c7b640 1 -- 172.21.15.32:0/3482249566 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbf10068430 con 0x7fbf100af890 2024-04-24T23:06:14.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.695+0000 7fbf06ffd640 1 -- 172.21.15.32:0/3482249566 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v145) ==== 80+0+0 (secure 0 0 0) 0x7fbef807d970 con 0x7fbf100af890 2024-04-24T23:06:14.697 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2024-04-24T23:06:14.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.697+0000 7fbf14c7b640 1 -- 172.21.15.32:0/3482249566 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fbef006be30 msgr2=0x7fbef006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:14.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.697+0000 7fbf14c7b640 1 --2- 172.21.15.32:0/3482249566 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fbef006be30 0x7fbef006e2f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fbf00009aa0 tx=0x7fbf00009190 comp rx=0 tx=0).stop 2024-04-24T23:06:14.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.697+0000 7fbf14c7b640 1 -- 172.21.15.32:0/3482249566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf100af890 msgr2=0x7fbf100ae8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:14.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.697+0000 7fbf14c7b640 1 --2- 172.21.15.32:0/3482249566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf100af890 0x7fbf100ae8b0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fbef8000c00 tx=0x7fbef80046c0 comp rx=0 tx=0).stop 2024-04-24T23:06:14.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.697+0000 7fbf14c7b640 1 -- 172.21.15.32:0/3482249566 shutdown_connections 2024-04-24T23:06:14.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.697+0000 7fbf14c7b640 1 --2- 172.21.15.32:0/3482249566 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fbef006be30 0x7fbef006e2f0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:14.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.697+0000 7fbf14c7b640 1 --2- 172.21.15.32:0/3482249566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf100af890 0x7fbf100ae8b0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:14.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.697+0000 7fbf14c7b640 1 --2- 172.21.15.32:0/3482249566 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf100b44d0 0x7fbf100aedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:14.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.697+0000 7fbf14c7b640 1 --2- 172.21.15.32:0/3482249566 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf1006c5f0 0x7fbf1015dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:14.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.697+0000 7fbf14c7b640 1 -- 172.21.15.32:0/3482249566 >> 172.21.15.32:0/3482249566 conn(0x7fbf1006b110 msgr2=0x7fbf10071820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:14.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.697+0000 7fbf14c7b640 1 -- 172.21.15.32:0/3482249566 shutdown_connections 2024-04-24T23:06:14.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.697+0000 7fbf14c7b640 1 -- 172.21.15.32:0/3482249566 wait complete. 2024-04-24T23:06:14.709 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-24T23:06:14.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:06:14.710 DEBUG:teuthology.orchestra.run.smithi032:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-24T23:06:14.765 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2024-04-24T23:06:14.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:06:14.765 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-24T23:06:14.782 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:06:14.783 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:06:14.783 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:06:14.800 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:06:14.800 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:06:14.857 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:06:14.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.956+0000 7fc44c3c1640 1 Processor -- start 2024-04-24T23:06:14.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.956+0000 7fc44c3c1640 1 -- start start 2024-04-24T23:06:14.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.957+0000 7fc44c3c1640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc44415b480 0x7fc4440b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:14.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.957+0000 7fc44c3c1640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4440b46c0 con 0x7fc44415fd60 2024-04-24T23:06:14.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.957+0000 7fc44c3c1640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4440b49a0 con 0x7fc44406cd00 2024-04-24T23:06:14.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.957+0000 7fc44c3c1640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc4440b4ff0 con 0x7fc44415b480 2024-04-24T23:06:14.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.957+0000 7fc44995c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc44415fd60 0x7fc44406c5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41178/0 (socket says 172.21.15.32:41178) 2024-04-24T23:06:14.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.957+0000 7fc44995c640 1 -- 172.21.15.32:0/1084365688 learned_addr learned my addr 172.21.15.32:0/1084365688 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:06:14.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.957+0000 7fc44a15d640 1 --2- 172.21.15.32:0/1084365688 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc44415b480 0x7fc4440b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:14.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.957+0000 7fc44915b640 1 -- 172.21.15.32:0/1084365688 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3371931340 0 0) 0x7fc4440b46c0 con 0x7fc44415fd60 2024-04-24T23:06:14.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.958+0000 7fc44915b640 1 -- 172.21.15.32:0/1084365688 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc42c005730 con 0x7fc44415fd60 2024-04-24T23:06:14.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.958+0000 7fc44915b640 1 -- 172.21.15.32:0/1084365688 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 833960312 0 0) 0x7fc4440b49a0 con 0x7fc44406cd00 2024-04-24T23:06:14.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.958+0000 7fc44915b640 1 -- 172.21.15.32:0/1084365688 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc4440b46c0 con 0x7fc44406cd00 2024-04-24T23:06:14.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.958+0000 7fc44a15d640 1 -- 172.21.15.32:0/1084365688 >> v1:172.21.15.121:6789/0 conn(0x7fc44406cd00 legacy=0x7fc4440cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:14.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.958+0000 7fc44a15d640 1 -- 172.21.15.32:0/1084365688 >> v1:172.21.15.32:6789/0 conn(0x7fc44415fd60 legacy=0x7fc44406c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:14.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.958+0000 7fc44915b640 1 -- 172.21.15.32:0/1084365688 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 712875100 0 0) 0x7fc42c005730 con 0x7fc44415fd60 2024-04-24T23:06:14.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.958+0000 7fc44a15d640 1 -- 172.21.15.32:0/1084365688 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4440b4c80 con 0x7fc44415b480 2024-04-24T23:06:14.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.958+0000 7fc44a15d640 1 --2- 172.21.15.32:0/1084365688 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc44415b480 0x7fc4440b5c60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc43c0097d0 tx=0x7fc43c02a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10ef78406fc2172b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:14.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.959+0000 7fc44915b640 1 -- 172.21.15.32:0/1084365688 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc43c037070 con 0x7fc44415b480 2024-04-24T23:06:14.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.959+0000 7fc44915b640 1 -- 172.21.15.32:0/1084365688 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc43c02ac30 con 0x7fc44415b480 2024-04-24T23:06:14.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.959+0000 7fc44915b640 1 -- 172.21.15.32:0/1084365688 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc438001670 con 0x7fc44415b480 2024-04-24T23:06:14.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.959+0000 7fc44c3c1640 1 -- 172.21.15.32:0/1084365688 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc44415b480 msgr2=0x7fc4440b5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:14.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.959+0000 7fc44c3c1640 1 --2- 172.21.15.32:0/1084365688 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc44415b480 0x7fc4440b5c60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc43c0097d0 tx=0x7fc43c02a1a0 comp rx=0 tx=0).stop 2024-04-24T23:06:14.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.959+0000 7fc44c3c1640 1 -- 172.21.15.32:0/1084365688 shutdown_connections 2024-04-24T23:06:14.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.959+0000 7fc44c3c1640 1 --2- 172.21.15.32:0/1084365688 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc44415b480 0x7fc4440b5c60 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:14.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.959+0000 7fc44c3c1640 1 -- 172.21.15.32:0/1084365688 >> 172.21.15.32:0/1084365688 conn(0x7fc44406b110 msgr2=0x7fc44406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:14.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.959+0000 7fc44c3c1640 1 -- 172.21.15.32:0/1084365688 shutdown_connections 2024-04-24T23:06:14.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.959+0000 7fc44c3c1640 1 -- 172.21.15.32:0/1084365688 wait complete. 2024-04-24T23:06:14.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.960+0000 7fc44c3c1640 1 Processor -- start 2024-04-24T23:06:14.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.960+0000 7fc44c3c1640 1 -- start start 2024-04-24T23:06:14.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.961+0000 7fc44c3c1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc44406cd00 0x7fc444072700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:14.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.961+0000 7fc44c3c1640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc44415b480 0x7fc4440707e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:14.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.961+0000 7fc44c3c1640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc44415fd60 0x7fc44414b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:14.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.961+0000 7fc44c3c1640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc44415b860 con 0x7fc44406cd00 2024-04-24T23:06:14.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.961+0000 7fc44c3c1640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc4440b5480 con 0x7fc44415b480 2024-04-24T23:06:14.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.961+0000 7fc44c3c1640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc44415bb70 con 0x7fc44415fd60 2024-04-24T23:06:14.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.961+0000 7fc44a15d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc44406cd00 0x7fc444072700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:14.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.961+0000 7fc44a15d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc44406cd00 0x7fc444072700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34862/0 (socket says 172.21.15.32:34862) 2024-04-24T23:06:14.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.961+0000 7fc44a15d640 1 -- 172.21.15.32:0/505967960 learned_addr learned my addr 172.21.15.32:0/505967960 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:14.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.962+0000 7fc44a15d640 1 -- 172.21.15.32:0/505967960 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc44415fd60 msgr2=0x7fc44414b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:14.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.962+0000 7fc44a15d640 1 --2- 172.21.15.32:0/505967960 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc44415fd60 0x7fc44414b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:14.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.962+0000 7fc44a15d640 1 -- 172.21.15.32:0/505967960 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc44415b480 msgr2=0x7fc4440707e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:14.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.962+0000 7fc44a15d640 1 --2- 172.21.15.32:0/505967960 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc44415b480 0x7fc4440707e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:14.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.962+0000 7fc44a15d640 1 -- 172.21.15.32:0/505967960 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc44414b620 con 0x7fc44406cd00 2024-04-24T23:06:14.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.962+0000 7fc44a15d640 1 --2- 172.21.15.32:0/505967960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc44406cd00 0x7fc444072700 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fc43c0097d0 tx=0x7fc43c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:14.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.962+0000 7fc442ffd640 1 -- 172.21.15.32:0/505967960 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc43c041020 con 0x7fc44406cd00 2024-04-24T23:06:14.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.962+0000 7fc442ffd640 1 -- 172.21.15.32:0/505967960 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc43c03d620 con 0x7fc44406cd00 2024-04-24T23:06:14.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.962+0000 7fc44c3c1640 1 -- 172.21.15.32:0/505967960 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc444070e70 con 0x7fc44406cd00 2024-04-24T23:06:14.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.962+0000 7fc442ffd640 1 -- 172.21.15.32:0/505967960 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc43c037040 con 0x7fc44406cd00 2024-04-24T23:06:14.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.962+0000 7fc44c3c1640 1 -- 172.21.15.32:0/505967960 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc444071430 con 0x7fc44406cd00 2024-04-24T23:06:14.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.963+0000 7fc44c3c1640 1 -- 172.21.15.32:0/505967960 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc444068430 con 0x7fc44406cd00 2024-04-24T23:06:14.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.964+0000 7fc442ffd640 1 -- 172.21.15.32:0/505967960 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc43c044020 con 0x7fc44406cd00 2024-04-24T23:06:14.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.964+0000 7fc442ffd640 1 --2- 172.21.15.32:0/505967960 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc43006be60 0x7fc43006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:14.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.964+0000 7fc442ffd640 1 -- 172.21.15.32:0/505967960 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5796+0+0 (secure 0 0 0) 0x7fc43c0ab400 con 0x7fc44406cd00 2024-04-24T23:06:14.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.964+0000 7fc44995c640 1 --2- 172.21.15.32:0/505967960 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc43006be60 0x7fc43006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:14.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.965+0000 7fc44995c640 1 --2- 172.21.15.32:0/505967960 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc43006be60 0x7fc43006e320 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc44414efd0 tx=0x7fc438000f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:14.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:14.968+0000 7fc442ffd640 1 -- 172.21.15.32:0/505967960 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc43c04b030 con 0x7fc44406cd00 2024-04-24T23:06:15.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.152+0000 7fc44c3c1640 1 -- 172.21.15.32:0/505967960 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc44406c000 con 0x7fc44406cd00 2024-04-24T23:06:15.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.153+0000 7fc442ffd640 1 -- 172.21.15.32:0/505967960 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) ==== 74+0+16836 (secure 0 0 0) 0x7fc43c079d50 con 0x7fc44406cd00 2024-04-24T23:06:15.155 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:06:15.155 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":145,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:06:06.811191+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:05:32.064234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-24T23:05:32.760777+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.17","mappings":[{"from":4,"to":3}]},{"pgid":"17.8","mappings":[{"from":6,"to":5}]},{"pgid":"17.3f","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:06:15.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.155+0000 7fc44c3c1640 1 -- 172.21.15.32:0/505967960 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc43006be60 msgr2=0x7fc43006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:15.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.155+0000 7fc44c3c1640 1 --2- 172.21.15.32:0/505967960 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc43006be60 0x7fc43006e320 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc44414efd0 tx=0x7fc438000f90 comp rx=0 tx=0).stop 2024-04-24T23:06:15.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.155+0000 7fc44c3c1640 1 -- 172.21.15.32:0/505967960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc44406cd00 msgr2=0x7fc444072700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:15.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.155+0000 7fc44c3c1640 1 --2- 172.21.15.32:0/505967960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc44406cd00 0x7fc444072700 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fc43c0097d0 tx=0x7fc43c0046c0 comp rx=0 tx=0).stop 2024-04-24T23:06:15.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.155+0000 7fc44c3c1640 1 -- 172.21.15.32:0/505967960 shutdown_connections 2024-04-24T23:06:15.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.155+0000 7fc44c3c1640 1 --2- 172.21.15.32:0/505967960 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc43006be60 0x7fc43006e320 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:15.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.155+0000 7fc44c3c1640 1 --2- 172.21.15.32:0/505967960 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc44415fd60 0x7fc44414b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:15.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.155+0000 7fc44c3c1640 1 --2- 172.21.15.32:0/505967960 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc44415b480 0x7fc4440707e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:15.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.155+0000 7fc44c3c1640 1 --2- 172.21.15.32:0/505967960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc44406cd00 0x7fc444072700 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:15.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.155+0000 7fc44c3c1640 1 -- 172.21.15.32:0/505967960 >> 172.21.15.32:0/505967960 conn(0x7fc44406b110 msgr2=0x7fc444155540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:15.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.155+0000 7fc44c3c1640 1 -- 172.21.15.32:0/505967960 shutdown_connections 2024-04-24T23:06:15.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.155+0000 7fc44c3c1640 1 -- 172.21.15.32:0/505967960 wait complete. 2024-04-24T23:06:15.168 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:05:32.064234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '139', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:05:32.760777+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '145', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:06:15.168 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:06:15.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.271+0000 7f946caf5640 1 Processor -- start 2024-04-24T23:06:15.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.271+0000 7f946caf5640 1 -- start start 2024-04-24T23:06:15.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.272+0000 7f946caf5640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f946815b380 0x7f946806c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:15.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.272+0000 7f946caf5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f946815cb00 0x7f946806cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:15.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.272+0000 7f946caf5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f94680ce430 0x7f94680ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:15.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.272+0000 7f946caf5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f94680b4520 con 0x7f94680ce430 2024-04-24T23:06:15.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.272+0000 7f946caf5640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f94680b4800 con 0x7f946815b380 2024-04-24T23:06:15.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.272+0000 7f946caf5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f94680b4ae0 con 0x7f946815cb00 2024-04-24T23:06:15.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.272+0000 7f9466d9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f94680ce430 0x7f94680ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:15.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.272+0000 7f9466d9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f94680ce430 0x7f94680ce880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34872/0 (socket says 172.21.15.32:34872) 2024-04-24T23:06:15.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.272+0000 7f9466d9d640 1 -- 172.21.15.32:0/2872800320 learned_addr learned my addr 172.21.15.32:0/2872800320 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:15.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.272+0000 7f946659c640 1 --2- 172.21.15.32:0/2872800320 >> v2:172.21.15.121:3300/0 conn(0x7f946815b380 0x7f946806c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:15.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.272+0000 7f9465d9b640 1 --2- 172.21.15.32:0/2872800320 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f946815cb00 0x7f946806cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:15.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.272+0000 7f9466d9d640 1 -- 172.21.15.32:0/2872800320 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f946815cb00 msgr2=0x7f946806cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:15.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.272+0000 7f9466d9d640 1 --2- 172.21.15.32:0/2872800320 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f946815cb00 0x7f946806cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:15.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.272+0000 7f9466d9d640 1 -- 172.21.15.32:0/2872800320 >> v2:172.21.15.121:3300/0 conn(0x7f946815b380 msgr2=0x7f946806c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:15.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.272+0000 7f9466d9d640 1 --2- 172.21.15.32:0/2872800320 >> v2:172.21.15.121:3300/0 conn(0x7f946815b380 0x7f946806c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:15.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.272+0000 7f9466d9d640 1 -- 172.21.15.32:0/2872800320 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94680b4dc0 con 0x7f94680ce430 2024-04-24T23:06:15.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.273+0000 7f9466d9d640 1 --2- 172.21.15.32:0/2872800320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94680ce430 0x7f94680ce880 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f946806eed0 tx=0x7f945802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df1dd3af41f57495 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:15.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.273+0000 7f946559a640 1 -- 172.21.15.32:0/2872800320 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9458037070 con 0x7f94680ce430 2024-04-24T23:06:15.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.273+0000 7f946559a640 1 -- 172.21.15.32:0/2872800320 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9458032440 con 0x7f94680ce430 2024-04-24T23:06:15.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.273+0000 7f946559a640 1 -- 172.21.15.32:0/2872800320 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9458032810 con 0x7f94680ce430 2024-04-24T23:06:15.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.274+0000 7f946caf5640 1 -- 172.21.15.32:0/2872800320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94680ce430 msgr2=0x7f94680ce880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:15.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.274+0000 7f946caf5640 1 --2- 172.21.15.32:0/2872800320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94680ce430 0x7f94680ce880 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f946806eed0 tx=0x7f945802a140 comp rx=0 tx=0).stop 2024-04-24T23:06:15.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.274+0000 7f946caf5640 1 -- 172.21.15.32:0/2872800320 shutdown_connections 2024-04-24T23:06:15.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.274+0000 7f946caf5640 1 --2- 172.21.15.32:0/2872800320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94680ce430 0x7f94680ce880 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:15.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.274+0000 7f946caf5640 1 --2- 172.21.15.32:0/2872800320 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f946815cb00 0x7f946806cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:15.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.274+0000 7f946caf5640 1 --2- 172.21.15.32:0/2872800320 >> v2:172.21.15.121:3300/0 conn(0x7f946815b380 0x7f946806c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:15.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.274+0000 7f946caf5640 1 -- 172.21.15.32:0/2872800320 >> 172.21.15.32:0/2872800320 conn(0x7f946806b110 msgr2=0x7f946806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:15.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.274+0000 7f946caf5640 1 -- 172.21.15.32:0/2872800320 shutdown_connections 2024-04-24T23:06:15.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.274+0000 7f946caf5640 1 -- 172.21.15.32:0/2872800320 wait complete. 2024-04-24T23:06:15.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.275+0000 7f946caf5640 1 Processor -- start 2024-04-24T23:06:15.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.275+0000 7f946caf5640 1 -- start start 2024-04-24T23:06:15.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.276+0000 7f946caf5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94680ce430 0x7f94680ae710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:15.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.276+0000 7f946caf5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f946815b380 0x7f94680aec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:15.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.276+0000 7f946caf5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f946815cb00 0x7f9468159bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:15.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.276+0000 7f946caf5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f94680cedc0 con 0x7f946815b380 2024-04-24T23:06:15.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.276+0000 7f946caf5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f946815b760 con 0x7f94680ce430 2024-04-24T23:06:15.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.276+0000 7f946caf5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f94680cef40 con 0x7f946815cb00 2024-04-24T23:06:15.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.276+0000 7f9465d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f946815b380 0x7f94680aec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:15.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.276+0000 7f9465d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f946815b380 0x7f94680aec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34878/0 (socket says 172.21.15.32:34878) 2024-04-24T23:06:15.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.276+0000 7f9465d9b640 1 -- 172.21.15.32:0/904442162 learned_addr learned my addr 172.21.15.32:0/904442162 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:15.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.276+0000 7f946659c640 1 --2- 172.21.15.32:0/904442162 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94680ce430 0x7f94680ae710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:15.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.276+0000 7f9466d9d640 1 --2- 172.21.15.32:0/904442162 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f946815cb00 0x7f9468159bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:15.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.277+0000 7f9465d9b640 1 -- 172.21.15.32:0/904442162 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f946815cb00 msgr2=0x7f9468159bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:15.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.277+0000 7f9465d9b640 1 --2- 172.21.15.32:0/904442162 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f946815cb00 0x7f9468159bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:15.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.277+0000 7f9465d9b640 1 -- 172.21.15.32:0/904442162 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94680ce430 msgr2=0x7f94680ae710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:15.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.277+0000 7f9465d9b640 1 --2- 172.21.15.32:0/904442162 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94680ce430 0x7f94680ae710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:15.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.277+0000 7f9465d9b640 1 -- 172.21.15.32:0/904442162 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f946814cf40 con 0x7f946815b380 2024-04-24T23:06:15.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.277+0000 7f9465d9b640 1 --2- 172.21.15.32:0/904442162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f946815b380 0x7f94680aec50 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f945c00e9e0 tx=0x7f945c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:15.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.277+0000 7f9456ffd640 1 -- 172.21.15.32:0/904442162 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945c017070 con 0x7f946815b380 2024-04-24T23:06:15.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.277+0000 7f9456ffd640 1 -- 172.21.15.32:0/904442162 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f945c00c9d0 con 0x7f946815b380 2024-04-24T23:06:15.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.277+0000 7f946caf5640 1 -- 172.21.15.32:0/904442162 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f946814d1c0 con 0x7f946815b380 2024-04-24T23:06:15.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.277+0000 7f9456ffd640 1 -- 172.21.15.32:0/904442162 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945c00cdc0 con 0x7f946815b380 2024-04-24T23:06:15.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.277+0000 7f946caf5640 1 -- 172.21.15.32:0/904442162 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f946814da60 con 0x7f946815b380 2024-04-24T23:06:15.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.278+0000 7f946caf5640 1 -- 172.21.15.32:0/904442162 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f946815e2d0 con 0x7f946815b380 2024-04-24T23:06:15.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.278+0000 7f9456ffd640 1 -- 172.21.15.32:0/904442162 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f945c01a070 con 0x7f946815b380 2024-04-24T23:06:15.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.279+0000 7f9456ffd640 1 --2- 172.21.15.32:0/904442162 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f944806be30 0x7f944806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:15.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.279+0000 7f9456ffd640 1 -- 172.21.15.32:0/904442162 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5796+0+0 (secure 0 0 0) 0x7f945c08ad40 con 0x7f946815b380 2024-04-24T23:06:15.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.279+0000 7f946659c640 1 --2- 172.21.15.32:0/904442162 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f944806be30 0x7f944806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:15.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.279+0000 7f946659c640 1 --2- 172.21.15.32:0/904442162 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f944806be30 0x7f944806e2f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f94600113e0 tx=0x7f946000b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:15.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.283+0000 7f9456ffd640 1 -- 172.21.15.32:0/904442162 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f946815e2d0 con 0x7f946815b380 2024-04-24T23:06:15.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.465+0000 7f946caf5640 1 -- 172.21.15.32:0/904442162 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9468068430 con 0x7f946815b380 2024-04-24T23:06:15.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.465+0000 7f9456ffd640 1 -- 172.21.15.32:0/904442162 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v145) ==== 93+0+10 (secure 0 0 0) 0x7f946814cf40 con 0x7f946815b380 2024-04-24T23:06:15.467 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T23:06:15.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.467+0000 7f946caf5640 1 -- 172.21.15.32:0/904442162 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f944806be30 msgr2=0x7f944806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:15.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.467+0000 7f946caf5640 1 --2- 172.21.15.32:0/904442162 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f944806be30 0x7f944806e2f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f94600113e0 tx=0x7f946000b040 comp rx=0 tx=0).stop 2024-04-24T23:06:15.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.467+0000 7f946caf5640 1 -- 172.21.15.32:0/904442162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f946815b380 msgr2=0x7f94680aec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:15.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.467+0000 7f946caf5640 1 --2- 172.21.15.32:0/904442162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f946815b380 0x7f94680aec50 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f945c00e9e0 tx=0x7f945c00eea0 comp rx=0 tx=0).stop 2024-04-24T23:06:15.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.467+0000 7f946caf5640 1 -- 172.21.15.32:0/904442162 shutdown_connections 2024-04-24T23:06:15.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.467+0000 7f946caf5640 1 --2- 172.21.15.32:0/904442162 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f944806be30 0x7f944806e2f0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:15.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.467+0000 7f946caf5640 1 --2- 172.21.15.32:0/904442162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f946815b380 0x7f94680aec50 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:15.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.467+0000 7f946caf5640 1 --2- 172.21.15.32:0/904442162 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f946815cb00 0x7f9468159bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:15.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.467+0000 7f946caf5640 1 --2- 172.21.15.32:0/904442162 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94680ce430 0x7f94680ae710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:15.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.467+0000 7f946caf5640 1 -- 172.21.15.32:0/904442162 >> 172.21.15.32:0/904442162 conn(0x7f946806b110 msgr2=0x7f946815d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:15.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.467+0000 7f946caf5640 1 -- 172.21.15.32:0/904442162 shutdown_connections 2024-04-24T23:06:15.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.467+0000 7f946caf5640 1 -- 172.21.15.32:0/904442162 wait complete. 2024-04-24T23:06:15.479 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:06:15.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.582+0000 7f5574c37640 1 Processor -- start 2024-04-24T23:06:15.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.582+0000 7f5574c37640 1 -- start start 2024-04-24T23:06:15.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.583+0000 7f5574c37640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f55700cfd80 0x7f557006deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:15.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.583+0000 7f5574c37640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f557006e3f0 0x7f55700ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:15.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.583+0000 7f5574c37640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f557015f680 con 0x7f55700cf9a0 2024-04-24T23:06:15.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.583+0000 7f5574c37640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f557006da20 con 0x7f55700cfd80 2024-04-24T23:06:15.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.583+0000 7f5574c37640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f557015f960 con 0x7f557006e3f0 2024-04-24T23:06:15.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.583+0000 7f556e59c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f55700cf9a0 0x7f557015ee50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41192/0 (socket says 172.21.15.32:41192) 2024-04-24T23:06:15.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.583+0000 7f556e59c640 1 -- 172.21.15.32:0/4261433137 learned_addr learned my addr 172.21.15.32:0/4261433137 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:06:15.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.583+0000 7f556dd9b640 1 -- 172.21.15.32:0/4261433137 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 757673786 0 0) 0x7f557015f680 con 0x7f55700cf9a0 2024-04-24T23:06:15.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.583+0000 7f556ed9d640 1 --2- 172.21.15.32:0/4261433137 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f557006e3f0 0x7f55700ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:15.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.583+0000 7f556dd9b640 1 -- 172.21.15.32:0/4261433137 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f555c003520 con 0x7f55700cf9a0 2024-04-24T23:06:15.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.583+0000 7f5567fff640 1 --2- 172.21.15.32:0/4261433137 >> v2:172.21.15.121:3300/0 conn(0x7f55700cfd80 0x7f557006deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:15.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.583+0000 7f556dd9b640 1 -- 172.21.15.32:0/4261433137 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2795888580 0 0) 0x7f555c003520 con 0x7f55700cf9a0 2024-04-24T23:06:15.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.583+0000 7f556dd9b640 1 -- 172.21.15.32:0/4261433137 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f557006e3f0 msgr2=0x7f55700ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:15.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.583+0000 7f556dd9b640 1 --2- 172.21.15.32:0/4261433137 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f557006e3f0 0x7f55700ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:15.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.583+0000 7f556dd9b640 1 -- 172.21.15.32:0/4261433137 >> v2:172.21.15.121:3300/0 conn(0x7f55700cfd80 msgr2=0x7f557006deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:15.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.583+0000 7f556dd9b640 1 --2- 172.21.15.32:0/4261433137 >> v2:172.21.15.121:3300/0 conn(0x7f55700cfd80 0x7f557006deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:15.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.583+0000 7f556dd9b640 1 -- 172.21.15.32:0/4261433137 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f557015fc40 con 0x7f55700cf9a0 2024-04-24T23:06:15.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.583+0000 7f556dd9b640 1 -- 172.21.15.32:0/4261433137 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f5560001670 con 0x7f55700cf9a0 2024-04-24T23:06:15.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.583+0000 7f5567fff640 1 --2- 172.21.15.32:0/4261433137 >> v2:172.21.15.121:3300/0 conn(0x7f55700cfd80 0x7f557006deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:06:15.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.583+0000 7f556dd9b640 1 -- 172.21.15.32:0/4261433137 >> v1:172.21.15.32:6789/0 conn(0x7f55700cf9a0 legacy=0x7f557015ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:15.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.583+0000 7f556dd9b640 1 --2- 172.21.15.32:0/4261433137 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f555c0039b0 0x7f555c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:15.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.583+0000 7f556dd9b640 1 --2- 172.21.15.32:0/4261433137 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f557006e3f0 0x7f555c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:15.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.583+0000 7f556dd9b640 1 --2- 172.21.15.32:0/4261433137 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f555c004d40 0x7f555c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:15.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.583+0000 7f556dd9b640 1 -- 172.21.15.32:0/4261433137 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f557006da20 con 0x7f555c004d40 2024-04-24T23:06:15.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.583+0000 7f556dd9b640 1 -- 172.21.15.32:0/4261433137 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f557015f960 con 0x7f555c0039b0 2024-04-24T23:06:15.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.583+0000 7f556dd9b640 1 -- 172.21.15.32:0/4261433137 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f555c00b9c0 con 0x7f557006e3f0 2024-04-24T23:06:15.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.584+0000 7f556ed9d640 1 --2- 172.21.15.32:0/4261433137 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f555c004d40 0x7f555c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:15.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.584+0000 7f556e59c640 1 --2- 172.21.15.32:0/4261433137 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f555c0039b0 0x7f555c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:15.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.584+0000 7f5567fff640 1 --2- 172.21.15.32:0/4261433137 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f557006e3f0 0x7f555c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:15.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.584+0000 7f556ed9d640 1 -- 172.21.15.32:0/4261433137 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f557006e3f0 msgr2=0x7f555c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:15.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.584+0000 7f556ed9d640 1 --2- 172.21.15.32:0/4261433137 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f557006e3f0 0x7f555c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:15.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.584+0000 7f556ed9d640 1 -- 172.21.15.32:0/4261433137 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f555c0039b0 msgr2=0x7f555c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:15.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.584+0000 7f556ed9d640 1 --2- 172.21.15.32:0/4261433137 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f555c0039b0 0x7f555c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:15.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.584+0000 7f556ed9d640 1 -- 172.21.15.32:0/4261433137 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f555c003520 con 0x7f555c004d40 2024-04-24T23:06:15.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.585+0000 7f556ed9d640 1 --2- 172.21.15.32:0/4261433137 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f555c004d40 0x7f555c0051c0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f556800e170 tx=0x7f555c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a8f708b7410447f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:15.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.585+0000 7f556dd9b640 1 -- 172.21.15.32:0/4261433137 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5568034020 con 0x7f555c004d40 2024-04-24T23:06:15.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.585+0000 7f556dd9b640 1 -- 172.21.15.32:0/4261433137 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55680305f0 con 0x7f555c004d40 2024-04-24T23:06:15.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.585+0000 7f556dd9b640 1 -- 172.21.15.32:0/4261433137 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5568009c10 con 0x7f555c004d40 2024-04-24T23:06:15.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.585+0000 7f5574c37640 1 -- 172.21.15.32:0/4261433137 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f555c004d40 msgr2=0x7f555c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:15.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.585+0000 7f5574c37640 1 --2- 172.21.15.32:0/4261433137 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f555c004d40 0x7f555c0051c0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f556800e170 tx=0x7f555c005830 comp rx=0 tx=0).stop 2024-04-24T23:06:15.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.585+0000 7f556ed9d640 1 -- 172.21.15.32:0/4261433137 reap_dead start 2024-04-24T23:06:15.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.585+0000 7f5574c37640 1 -- 172.21.15.32:0/4261433137 shutdown_connections 2024-04-24T23:06:15.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.585+0000 7f5574c37640 1 -- 172.21.15.32:0/4261433137 >> 172.21.15.32:0/4261433137 conn(0x7f55700d0730 msgr2=0x7f55700d0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:15.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.585+0000 7f5574c37640 1 -- 172.21.15.32:0/4261433137 shutdown_connections 2024-04-24T23:06:15.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.585+0000 7f5574c37640 1 -- 172.21.15.32:0/4261433137 wait complete. 2024-04-24T23:06:15.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.587+0000 7f5574c37640 1 Processor -- start 2024-04-24T23:06:15.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.587+0000 7f5574c37640 1 -- start start 2024-04-24T23:06:15.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.587+0000 7f5574c37640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55701e7270 0x7f557013ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:15.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.587+0000 7f5574c37640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55701e7650 0x7f55701f1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:15.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.587+0000 7f5574c37640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55701f1ac0 0x7f5570140ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:15.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.587+0000 7f5574c37640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f55700afe70 con 0x7f55701e7650 2024-04-24T23:06:15.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.587+0000 7f5574c37640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f55700afcf0 con 0x7f55701f1ac0 2024-04-24T23:06:15.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.587+0000 7f5574c37640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f55700afff0 con 0x7f55701e7270 2024-04-24T23:06:15.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.587+0000 7f5567fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55701e7650 0x7f55701f1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:15.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.587+0000 7f5567fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55701e7650 0x7f55701f1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34892/0 (socket says 172.21.15.32:34892) 2024-04-24T23:06:15.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.587+0000 7f5567fff640 1 -- 172.21.15.32:0/2590378357 learned_addr learned my addr 172.21.15.32:0/2590378357 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:15.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.588+0000 7f556ed9d640 1 --2- 172.21.15.32:0/2590378357 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55701f1ac0 0x7f5570140ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:15.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.588+0000 7f556e59c640 1 --2- 172.21.15.32:0/2590378357 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55701e7270 0x7f557013ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:15.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.588+0000 7f5567fff640 1 -- 172.21.15.32:0/2590378357 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55701e7270 msgr2=0x7f557013ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:15.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.588+0000 7f5567fff640 1 --2- 172.21.15.32:0/2590378357 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55701e7270 0x7f557013ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:15.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.588+0000 7f5567fff640 1 -- 172.21.15.32:0/2590378357 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55701f1ac0 msgr2=0x7f5570140ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:15.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.588+0000 7f5567fff640 1 --2- 172.21.15.32:0/2590378357 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55701f1ac0 0x7f5570140ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:15.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.588+0000 7f5567fff640 1 -- 172.21.15.32:0/2590378357 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55701f1f20 con 0x7f55701e7650 2024-04-24T23:06:15.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.588+0000 7f556ed9d640 1 --2- 172.21.15.32:0/2590378357 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55701f1ac0 0x7f5570140ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:06:15.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.588+0000 7f556e59c640 1 --2- 172.21.15.32:0/2590378357 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55701e7270 0x7f557013ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:06:15.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.588+0000 7f5567fff640 1 --2- 172.21.15.32:0/2590378357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55701e7650 0x7f55701f1580 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f5558011620 tx=0x7f5558011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:15.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.588+0000 7f5566ffd640 1 -- 172.21.15.32:0/2590378357 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5558019070 con 0x7f55701e7650 2024-04-24T23:06:15.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.588+0000 7f5566ffd640 1 -- 172.21.15.32:0/2590378357 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f555800f920 con 0x7f55701e7650 2024-04-24T23:06:15.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.588+0000 7f5574c37640 1 -- 172.21.15.32:0/2590378357 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5570141bd0 con 0x7f55701e7650 2024-04-24T23:06:15.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.588+0000 7f5566ffd640 1 -- 172.21.15.32:0/2590378357 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555800f040 con 0x7f55701e7650 2024-04-24T23:06:15.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.588+0000 7f5574c37640 1 -- 172.21.15.32:0/2590378357 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55700b2800 con 0x7f55701e7650 2024-04-24T23:06:15.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.589+0000 7f5574c37640 1 -- 172.21.15.32:0/2590378357 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5570068430 con 0x7f55701e7650 2024-04-24T23:06:15.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.589+0000 7f5566ffd640 1 -- 172.21.15.32:0/2590378357 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f555801c070 con 0x7f55701e7650 2024-04-24T23:06:15.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.590+0000 7f5566ffd640 1 --2- 172.21.15.32:0/2590378357 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f555006be60 0x7f555006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:15.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.590+0000 7f5566ffd640 1 -- 172.21.15.32:0/2590378357 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5796+0+0 (secure 0 0 0) 0x7f555808b270 con 0x7f55701e7650 2024-04-24T23:06:15.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.590+0000 7f556e59c640 1 --2- 172.21.15.32:0/2590378357 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f555006be60 0x7f555006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:15.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.590+0000 7f556e59c640 1 --2- 172.21.15.32:0/2590378357 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f555006be60 0x7f555006e320 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f5560004840 tx=0x7f5560009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:15.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.594+0000 7f5566ffd640 1 -- 172.21.15.32:0/2590378357 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5558059bc0 con 0x7f55701e7650 2024-04-24T23:06:15.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.780+0000 7f5574c37640 1 -- 172.21.15.32:0/2590378357 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f557015e700 con 0x7f55701e7650 2024-04-24T23:06:15.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.780+0000 7f5566ffd640 1 -- 172.21.15.32:0/2590378357 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v145) ==== 104+0+11 (secure 0 0 0) 0x7f55580594c0 con 0x7f55701e7650 2024-04-24T23:06:15.782 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-24T23:06:15.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.782+0000 7f5574c37640 1 -- 172.21.15.32:0/2590378357 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f555006be60 msgr2=0x7f555006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:15.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.782+0000 7f5574c37640 1 --2- 172.21.15.32:0/2590378357 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f555006be60 0x7f555006e320 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f5560004840 tx=0x7f5560009040 comp rx=0 tx=0).stop 2024-04-24T23:06:15.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.782+0000 7f5574c37640 1 -- 172.21.15.32:0/2590378357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55701e7650 msgr2=0x7f55701f1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:15.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.782+0000 7f5574c37640 1 --2- 172.21.15.32:0/2590378357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55701e7650 0x7f55701f1580 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f5558011620 tx=0x7f5558011ae0 comp rx=0 tx=0).stop 2024-04-24T23:06:15.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.782+0000 7f5574c37640 1 -- 172.21.15.32:0/2590378357 shutdown_connections 2024-04-24T23:06:15.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.782+0000 7f5574c37640 1 --2- 172.21.15.32:0/2590378357 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f555006be60 0x7f555006e320 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:15.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.782+0000 7f5574c37640 1 --2- 172.21.15.32:0/2590378357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55701e7650 0x7f55701f1580 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:15.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.782+0000 7f5574c37640 1 --2- 172.21.15.32:0/2590378357 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55701f1ac0 0x7f5570140ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:15.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.782+0000 7f5574c37640 1 --2- 172.21.15.32:0/2590378357 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55701e7270 0x7f557013ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:15.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.782+0000 7f5574c37640 1 -- 172.21.15.32:0/2590378357 >> 172.21.15.32:0/2590378357 conn(0x7f55700d0730 msgr2=0x7f5570070a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:15.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.782+0000 7f5574c37640 1 -- 172.21.15.32:0/2590378357 shutdown_connections 2024-04-24T23:06:15.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.782+0000 7f5574c37640 1 -- 172.21.15.32:0/2590378357 wait complete. 2024-04-24T23:06:15.795 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:06:15.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.897+0000 7f1ba7f90640 1 Processor -- start 2024-04-24T23:06:15.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.898+0000 7f1ba7f90640 1 -- start start 2024-04-24T23:06:15.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.898+0000 7f1ba7f90640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1ba00d4210 0x7f1ba00d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:15.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.898+0000 7f1ba7f90640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ba00d3170 con 0x7f1ba015e9b0 2024-04-24T23:06:15.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.898+0000 7f1ba7f90640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ba00d3450 con 0x7f1ba006b820 2024-04-24T23:06:15.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.898+0000 7f1ba7f90640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f1ba00d3730 con 0x7f1ba00d4210 2024-04-24T23:06:15.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.898+0000 7f1ba652d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1ba00d4210 0x7f1ba00d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:15.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.898+0000 7f1ba5d2c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1ba015e9b0 0x7f1ba006b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41204/0 (socket says 172.21.15.32:41204) 2024-04-24T23:06:15.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.898+0000 7f1ba5d2c640 1 -- 172.21.15.32:0/2221714913 learned_addr learned my addr 172.21.15.32:0/2221714913 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:06:15.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.899+0000 7f1ba652d640 1 -- 172.21.15.32:0/2221714913 >> v1:172.21.15.121:6789/0 conn(0x7f1ba006b820 legacy=0x7f1ba00b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:15.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.899+0000 7f1ba652d640 1 -- 172.21.15.32:0/2221714913 >> v1:172.21.15.32:6789/0 conn(0x7f1ba015e9b0 legacy=0x7f1ba006b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:15.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.899+0000 7f1ba652d640 1 -- 172.21.15.32:0/2221714913 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba00d3a10 con 0x7f1ba00d4210 2024-04-24T23:06:15.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.899+0000 7f1ba4d2a640 1 -- 172.21.15.32:0/2221714913 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1527477692 0 0) 0x7f1ba00d3170 con 0x7f1ba015e9b0 2024-04-24T23:06:15.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.899+0000 7f1ba652d640 1 --2- 172.21.15.32:0/2221714913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ba00d4210 0x7f1ba00d2b90 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f1b90009d90 tx=0x7f1b9002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47258b5cc72bedd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:15.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.900+0000 7f1ba4d2a640 1 -- 172.21.15.32:0/2221714913 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b90031600 con 0x7f1ba00d4210 2024-04-24T23:06:15.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.900+0000 7f1ba4d2a640 1 -- 172.21.15.32:0/2221714913 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b900317a0 con 0x7f1ba00d4210 2024-04-24T23:06:15.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.900+0000 7f1ba4d2a640 1 -- 172.21.15.32:0/2221714913 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b90031b70 con 0x7f1ba00d4210 2024-04-24T23:06:15.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.900+0000 7f1ba7f90640 1 -- 172.21.15.32:0/2221714913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ba00d4210 msgr2=0x7f1ba00d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:15.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.900+0000 7f1ba7f90640 1 --2- 172.21.15.32:0/2221714913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ba00d4210 0x7f1ba00d2b90 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f1b90009d90 tx=0x7f1b9002a610 comp rx=0 tx=0).stop 2024-04-24T23:06:15.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.900+0000 7f1ba7f90640 1 -- 172.21.15.32:0/2221714913 shutdown_connections 2024-04-24T23:06:15.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.900+0000 7f1ba7f90640 1 --2- 172.21.15.32:0/2221714913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ba00d4210 0x7f1ba00d2b90 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:15.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.900+0000 7f1ba7f90640 1 -- 172.21.15.32:0/2221714913 >> 172.21.15.32:0/2221714913 conn(0x7f1ba00d11d0 msgr2=0x7f1ba00d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:15.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.900+0000 7f1ba7f90640 1 -- 172.21.15.32:0/2221714913 shutdown_connections 2024-04-24T23:06:15.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.900+0000 7f1ba7f90640 1 -- 172.21.15.32:0/2221714913 wait complete. 2024-04-24T23:06:15.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.901+0000 7f1ba7f90640 1 Processor -- start 2024-04-24T23:06:15.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.902+0000 7f1ba7f90640 1 -- start start 2024-04-24T23:06:15.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.902+0000 7f1ba7f90640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1ba006b820 0x7f1ba0155790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:15.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.902+0000 7f1ba7f90640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ba00d4210 0x7f1ba014db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:15.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.902+0000 7f1ba7f90640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ba015e9b0 0x7f1ba01547c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:15.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.902+0000 7f1ba7f90640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba00ba0e0 con 0x7f1ba015e9b0 2024-04-24T23:06:15.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.902+0000 7f1ba7f90640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba00d2400 con 0x7f1ba00d4210 2024-04-24T23:06:15.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.902+0000 7f1ba7f90640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f1ba00ba260 con 0x7f1ba006b820 2024-04-24T23:06:15.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.902+0000 7f1ba652d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ba015e9b0 0x7f1ba01547c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:15.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.902+0000 7f1ba652d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ba015e9b0 0x7f1ba01547c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34904/0 (socket says 172.21.15.32:34904) 2024-04-24T23:06:15.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.902+0000 7f1ba652d640 1 -- 172.21.15.32:0/4186818463 learned_addr learned my addr 172.21.15.32:0/4186818463 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:15.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.903+0000 7f1ba652d640 1 -- 172.21.15.32:0/4186818463 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1ba006b820 msgr2=0x7f1ba0155790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:15.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.903+0000 7f1ba5d2c640 1 --2- 172.21.15.32:0/4186818463 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1ba006b820 0x7f1ba0155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:15.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.903+0000 7f1ba552b640 1 --2- 172.21.15.32:0/4186818463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ba00d4210 0x7f1ba014db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:15.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.903+0000 7f1ba652d640 1 --2- 172.21.15.32:0/4186818463 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1ba006b820 0x7f1ba0155790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:15.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.903+0000 7f1ba652d640 1 -- 172.21.15.32:0/4186818463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ba00d4210 msgr2=0x7f1ba014db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:15.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.903+0000 7f1ba652d640 1 --2- 172.21.15.32:0/4186818463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ba00d4210 0x7f1ba014db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:15.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.903+0000 7f1ba652d640 1 -- 172.21.15.32:0/4186818463 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba01570b0 con 0x7f1ba015e9b0 2024-04-24T23:06:15.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.903+0000 7f1ba552b640 1 --2- 172.21.15.32:0/4186818463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ba00d4210 0x7f1ba014db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:06:15.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.903+0000 7f1ba5d2c640 1 --2- 172.21.15.32:0/4186818463 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1ba006b820 0x7f1ba0155790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:06:15.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.903+0000 7f1ba652d640 1 --2- 172.21.15.32:0/4186818463 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ba015e9b0 0x7f1ba01547c0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f1b90009eb0 tx=0x7f1b90004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:15.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.903+0000 7f1b9e7fc640 1 -- 172.21.15.32:0/4186818463 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b90040070 con 0x7f1ba015e9b0 2024-04-24T23:06:15.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.903+0000 7f1b9e7fc640 1 -- 172.21.15.32:0/4186818463 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b90030b60 con 0x7f1ba015e9b0 2024-04-24T23:06:15.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.903+0000 7f1ba7f90640 1 -- 172.21.15.32:0/4186818463 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ba00cc960 con 0x7f1ba015e9b0 2024-04-24T23:06:15.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.903+0000 7f1b9e7fc640 1 -- 172.21.15.32:0/4186818463 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b90045460 con 0x7f1ba015e9b0 2024-04-24T23:06:15.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.903+0000 7f1ba7f90640 1 -- 172.21.15.32:0/4186818463 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ba014e0c0 con 0x7f1ba015e9b0 2024-04-24T23:06:15.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.904+0000 7f1ba7f90640 1 -- 172.21.15.32:0/4186818463 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ba0068430 con 0x7f1ba015e9b0 2024-04-24T23:06:15.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.904+0000 7f1b9e7fc640 1 -- 172.21.15.32:0/4186818463 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1b90039040 con 0x7f1ba015e9b0 2024-04-24T23:06:15.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.905+0000 7f1b9e7fc640 1 --2- 172.21.15.32:0/4186818463 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1b8c06be60 0x7f1b8c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:15.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.905+0000 7f1b9e7fc640 1 -- 172.21.15.32:0/4186818463 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5796+0+0 (secure 0 0 0) 0x7f1b900ac740 con 0x7f1ba015e9b0 2024-04-24T23:06:15.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.905+0000 7f1ba5d2c640 1 --2- 172.21.15.32:0/4186818463 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1b8c06be60 0x7f1b8c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:15.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.906+0000 7f1ba5d2c640 1 --2- 172.21.15.32:0/4186818463 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1b8c06be60 0x7f1b8c06e320 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f1b98000f60 tx=0x7f1b98009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:15.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:15.909+0000 7f1b9e7fc640 1 -- 172.21.15.32:0/4186818463 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1b9007b1e0 con 0x7f1ba015e9b0 2024-04-24T23:06:16.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.093+0000 7f1ba7f90640 1 -- 172.21.15.32:0/4186818463 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1ba0068bb0 con 0x7f1ba015e9b0 2024-04-24T23:06:16.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.093+0000 7f1b9e7fc640 1 -- 172.21.15.32:0/4186818463 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v145) ==== 100+0+11 (secure 0 0 0) 0x7f1b9007aae0 con 0x7f1ba015e9b0 2024-04-24T23:06:16.095 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-24T23:06:16.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.096+0000 7f1ba7f90640 1 -- 172.21.15.32:0/4186818463 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1b8c06be60 msgr2=0x7f1b8c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:16.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.096+0000 7f1ba7f90640 1 --2- 172.21.15.32:0/4186818463 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1b8c06be60 0x7f1b8c06e320 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f1b98000f60 tx=0x7f1b98009040 comp rx=0 tx=0).stop 2024-04-24T23:06:16.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.096+0000 7f1ba7f90640 1 -- 172.21.15.32:0/4186818463 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ba015e9b0 msgr2=0x7f1ba01547c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:16.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.096+0000 7f1ba7f90640 1 --2- 172.21.15.32:0/4186818463 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ba015e9b0 0x7f1ba01547c0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f1b90009eb0 tx=0x7f1b90004a40 comp rx=0 tx=0).stop 2024-04-24T23:06:16.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.096+0000 7f1ba7f90640 1 -- 172.21.15.32:0/4186818463 shutdown_connections 2024-04-24T23:06:16.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.096+0000 7f1ba7f90640 1 --2- 172.21.15.32:0/4186818463 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1b8c06be60 0x7f1b8c06e320 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:16.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.096+0000 7f1ba7f90640 1 --2- 172.21.15.32:0/4186818463 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ba015e9b0 0x7f1ba01547c0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:16.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.096+0000 7f1ba7f90640 1 --2- 172.21.15.32:0/4186818463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ba00d4210 0x7f1ba014db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:16.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.096+0000 7f1ba7f90640 1 --2- 172.21.15.32:0/4186818463 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1ba006b820 0x7f1ba0155790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:16.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.096+0000 7f1ba7f90640 1 -- 172.21.15.32:0/4186818463 >> 172.21.15.32:0/4186818463 conn(0x7f1ba00d11d0 msgr2=0x7f1ba00d15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:16.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.096+0000 7f1ba7f90640 1 -- 172.21.15.32:0/4186818463 shutdown_connections 2024-04-24T23:06:16.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.096+0000 7f1ba7f90640 1 -- 172.21.15.32:0/4186818463 wait complete. 2024-04-24T23:06:16.108 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T23:06:16.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.208+0000 7f3da9963640 1 Processor -- start 2024-04-24T23:06:16.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.208+0000 7f3da9963640 1 -- start start 2024-04-24T23:06:16.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.209+0000 7f3da9963640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3da40b44d0 0x7f3da40b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:16.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.209+0000 7f3da9963640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3da40af890 0x7f3da415cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:16.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.209+0000 7f3da9963640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3da40b6430 con 0x7f3da406c5f0 2024-04-24T23:06:16.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.209+0000 7f3da9963640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f3da4156f80 con 0x7f3da40b44d0 2024-04-24T23:06:16.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.209+0000 7f3da9963640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f3da40b6150 con 0x7f3da40af890 2024-04-24T23:06:16.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.209+0000 7f3da27fc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3da40b44d0 0x7f3da40b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:16.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.209+0000 7f3da27fc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3da40b44d0 0x7f3da40b5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34908/0 (socket says 172.21.15.32:34908) 2024-04-24T23:06:16.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.209+0000 7f3da27fc640 1 -- 172.21.15.32:0/95975397 learned_addr learned my addr 172.21.15.32:0/95975397 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:16.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.209+0000 7f3da2ffd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3da406c5f0 0x7f3da406c9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41208/0 (socket says 172.21.15.32:41208) 2024-04-24T23:06:16.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.209+0000 7f3da37fe640 1 --2- 172.21.15.32:0/95975397 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3da40af890 0x7f3da415cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:16.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.209+0000 7f3da27fc640 1 -- 172.21.15.32:0/95975397 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3da40af890 msgr2=0x7f3da415cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:16.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.209+0000 7f3da27fc640 1 --2- 172.21.15.32:0/95975397 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3da40af890 0x7f3da415cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:16.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.209+0000 7f3da27fc640 1 -- 172.21.15.32:0/95975397 >> v1:172.21.15.32:6789/0 conn(0x7f3da406c5f0 legacy=0x7f3da406c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:16.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.209+0000 7f3da27fc640 1 -- 172.21.15.32:0/95975397 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3da40b39f0 con 0x7f3da40b44d0 2024-04-24T23:06:16.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.210+0000 7f3da27fc640 1 --2- 172.21.15.32:0/95975397 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3da40b44d0 0x7f3da40b5890 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c009870 tx=0x7f3d8c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82b431f87daaecf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:16.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.210+0000 7f3da1ffb640 1 -- 172.21.15.32:0/95975397 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d8c037070 con 0x7f3da40b44d0 2024-04-24T23:06:16.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.210+0000 7f3da1ffb640 1 -- 172.21.15.32:0/95975397 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d8c032440 con 0x7f3da40b44d0 2024-04-24T23:06:16.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.210+0000 7f3da1ffb640 1 -- 172.21.15.32:0/95975397 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d8c032810 con 0x7f3da40b44d0 2024-04-24T23:06:16.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.211+0000 7f3da9963640 1 -- 172.21.15.32:0/95975397 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3da40b44d0 msgr2=0x7f3da40b5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:16.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.211+0000 7f3da9963640 1 --2- 172.21.15.32:0/95975397 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3da40b44d0 0x7f3da40b5890 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c009870 tx=0x7f3d8c02a1c0 comp rx=0 tx=0).stop 2024-04-24T23:06:16.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.211+0000 7f3da9963640 1 -- 172.21.15.32:0/95975397 shutdown_connections 2024-04-24T23:06:16.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.211+0000 7f3da9963640 1 --2- 172.21.15.32:0/95975397 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3da40b44d0 0x7f3da40b5890 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:16.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.211+0000 7f3da9963640 1 --2- 172.21.15.32:0/95975397 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3da40af890 0x7f3da415cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:16.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.211+0000 7f3da9963640 1 -- 172.21.15.32:0/95975397 >> 172.21.15.32:0/95975397 conn(0x7f3da406b110 msgr2=0x7f3da406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:16.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.211+0000 7f3da9963640 1 -- 172.21.15.32:0/95975397 shutdown_connections 2024-04-24T23:06:16.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.211+0000 7f3da9963640 1 -- 172.21.15.32:0/95975397 wait complete. 2024-04-24T23:06:16.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.212+0000 7f3da9963640 1 Processor -- start 2024-04-24T23:06:16.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.212+0000 7f3da9963640 1 -- start start 2024-04-24T23:06:16.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.213+0000 7f3da9963640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3da406c5f0 0x7f3da40ce620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:16.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.213+0000 7f3da9963640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3da40af890 0x7f3da4152030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:16.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.213+0000 7f3da9963640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3da40b44d0 0x7f3da40ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:16.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.213+0000 7f3da9963640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3da40b5dd0 con 0x7f3da40b44d0 2024-04-24T23:06:16.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.213+0000 7f3da9963640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3da40b3fb0 con 0x7f3da40af890 2024-04-24T23:06:16.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.213+0000 7f3da9963640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f3da415d000 con 0x7f3da406c5f0 2024-04-24T23:06:16.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.213+0000 7f3da37fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3da40b44d0 0x7f3da40ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:16.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.213+0000 7f3da37fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3da40b44d0 0x7f3da40ae8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34922/0 (socket says 172.21.15.32:34922) 2024-04-24T23:06:16.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.213+0000 7f3da37fe640 1 -- 172.21.15.32:0/696474175 learned_addr learned my addr 172.21.15.32:0/696474175 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:16.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.213+0000 7f3da2ffd640 1 --2- 172.21.15.32:0/696474175 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3da406c5f0 0x7f3da40ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:16.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.213+0000 7f3da37fe640 1 -- 172.21.15.32:0/696474175 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3da406c5f0 msgr2=0x7f3da40ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:16.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.213+0000 7f3da37fe640 1 --2- 172.21.15.32:0/696474175 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3da406c5f0 0x7f3da40ce620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:16.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.213+0000 7f3da27fc640 1 --2- 172.21.15.32:0/696474175 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3da40af890 0x7f3da4152030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:16.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.213+0000 7f3da37fe640 1 -- 172.21.15.32:0/696474175 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3da40af890 msgr2=0x7f3da4152030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:16.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.213+0000 7f3da37fe640 1 --2- 172.21.15.32:0/696474175 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3da40af890 0x7f3da4152030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:16.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.213+0000 7f3da37fe640 1 -- 172.21.15.32:0/696474175 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3da41572b0 con 0x7f3da40b44d0 2024-04-24T23:06:16.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.213+0000 7f3da37fe640 1 --2- 172.21.15.32:0/696474175 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3da40b44d0 0x7f3da40ae8b0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f3d94007e40 tx=0x7f3d9400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:16.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.214+0000 7f3d9bfff640 1 -- 172.21.15.32:0/696474175 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d94014620 con 0x7f3da40b44d0 2024-04-24T23:06:16.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.214+0000 7f3d9bfff640 1 -- 172.21.15.32:0/696474175 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d94019070 con 0x7f3da40b44d0 2024-04-24T23:06:16.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.214+0000 7f3da9963640 1 -- 172.21.15.32:0/696474175 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3da4157590 con 0x7f3da40b44d0 2024-04-24T23:06:16.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.214+0000 7f3d9bfff640 1 -- 172.21.15.32:0/696474175 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d94014a20 con 0x7f3da40b44d0 2024-04-24T23:06:16.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.214+0000 7f3da9963640 1 -- 172.21.15.32:0/696474175 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3da4157b50 con 0x7f3da40b44d0 2024-04-24T23:06:16.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.215+0000 7f3da9963640 1 -- 172.21.15.32:0/696474175 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3da415cac0 con 0x7f3da40b44d0 2024-04-24T23:06:16.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.215+0000 7f3d9bfff640 1 -- 172.21.15.32:0/696474175 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3d94014bc0 con 0x7f3da40b44d0 2024-04-24T23:06:16.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.215+0000 7f3d9bfff640 1 --2- 172.21.15.32:0/696474175 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3d8406be30 0x7f3d8406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:16.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.216+0000 7f3d9bfff640 1 -- 172.21.15.32:0/696474175 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5796+0+0 (secure 0 0 0) 0x7f3d9408b740 con 0x7f3da40b44d0 2024-04-24T23:06:16.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.216+0000 7f3da2ffd640 1 --2- 172.21.15.32:0/696474175 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3d8406be30 0x7f3d8406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:16.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.217+0000 7f3da2ffd640 1 --2- 172.21.15.32:0/696474175 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3d8406be30 0x7f3d8406e2f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c007fd0 tx=0x7f3d9c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:16.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.220+0000 7f3d9bfff640 1 -- 172.21.15.32:0/696474175 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3da415cac0 con 0x7f3da40b44d0 2024-04-24T23:06:16.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.418+0000 7f3da9963640 1 -- 172.21.15.32:0/696474175 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3da4068430 con 0x7f3da40b44d0 2024-04-24T23:06:16.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.419+0000 7f3d9bfff640 1 -- 172.21.15.32:0/696474175 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v145) ==== 80+0+0 (secure 0 0 0) 0x7f3d9405a120 con 0x7f3da40b44d0 2024-04-24T23:06:16.420 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2024-04-24T23:06:16.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.421+0000 7f3da9963640 1 -- 172.21.15.32:0/696474175 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3d8406be30 msgr2=0x7f3d8406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:16.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.421+0000 7f3da9963640 1 --2- 172.21.15.32:0/696474175 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3d8406be30 0x7f3d8406e2f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c007fd0 tx=0x7f3d9c009040 comp rx=0 tx=0).stop 2024-04-24T23:06:16.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.421+0000 7f3da9963640 1 -- 172.21.15.32:0/696474175 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3da40b44d0 msgr2=0x7f3da40ae8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:16.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.421+0000 7f3da9963640 1 --2- 172.21.15.32:0/696474175 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3da40b44d0 0x7f3da40ae8b0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f3d94007e40 tx=0x7f3d9400e4b0 comp rx=0 tx=0).stop 2024-04-24T23:06:16.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.421+0000 7f3da9963640 1 -- 172.21.15.32:0/696474175 shutdown_connections 2024-04-24T23:06:16.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.421+0000 7f3da9963640 1 --2- 172.21.15.32:0/696474175 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3d8406be30 0x7f3d8406e2f0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:16.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.421+0000 7f3da9963640 1 --2- 172.21.15.32:0/696474175 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3da40b44d0 0x7f3da40ae8b0 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:16.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.421+0000 7f3da9963640 1 --2- 172.21.15.32:0/696474175 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3da40af890 0x7f3da4152030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:16.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.421+0000 7f3da9963640 1 --2- 172.21.15.32:0/696474175 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3da406c5f0 0x7f3da40ce620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:16.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.421+0000 7f3da9963640 1 -- 172.21.15.32:0/696474175 >> 172.21.15.32:0/696474175 conn(0x7f3da406b110 msgr2=0x7f3da4072a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:16.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.421+0000 7f3da9963640 1 -- 172.21.15.32:0/696474175 shutdown_connections 2024-04-24T23:06:16.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.421+0000 7f3da9963640 1 -- 172.21.15.32:0/696474175 wait complete. 2024-04-24T23:06:16.433 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-24T23:06:16.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:06:16.433 DEBUG:teuthology.orchestra.run.smithi121:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-24T23:06:16.465 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2024-04-24T23:06:16.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:06:16.465 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-24T23:06:16.523 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:06:16.523 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:06:16.523 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:06:16.539 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=413 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T23:06:16.539 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:06:16.539 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:06:16.539 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:06:16.581 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3271 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T23:06:16.581 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:06:16.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.680+0000 7fda8cbe7640 1 Processor -- start 2024-04-24T23:06:16.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.680+0000 7fda8cbe7640 1 -- start start 2024-04-24T23:06:16.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.681+0000 7fda8cbe7640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fda8815b380 0x7fda8806c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:16.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.681+0000 7fda8cbe7640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda880cef50 con 0x7fda8815cb00 2024-04-24T23:06:16.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.681+0000 7fda8cbe7640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda880b4520 con 0x7fda880ceb40 2024-04-24T23:06:16.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.681+0000 7fda8cbe7640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fda880b4800 con 0x7fda8815b380 2024-04-24T23:06:16.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.681+0000 7fda8659c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fda8815b380 0x7fda8806c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:16.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.681+0000 7fda7ffff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fda8815cb00 0x7fda880ce430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41212/0 (socket says 172.21.15.32:41212) 2024-04-24T23:06:16.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.681+0000 7fda8659c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fda8815b380 0x7fda8806c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34938/0 (socket says 172.21.15.32:34938) 2024-04-24T23:06:16.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.681+0000 7fda7ffff640 1 -- 172.21.15.32:0/187583179 learned_addr learned my addr 172.21.15.32:0/187583179 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:06:16.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.681+0000 7fda8659c640 1 -- 172.21.15.32:0/187583179 >> v1:172.21.15.121:6789/0 conn(0x7fda880ceb40 legacy=0x7fda880b55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:16.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.681+0000 7fda8659c640 1 -- 172.21.15.32:0/187583179 >> v1:172.21.15.32:6789/0 conn(0x7fda8815cb00 legacy=0x7fda880ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:16.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.681+0000 7fda8659c640 1 -- 172.21.15.32:0/187583179 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda880b4ae0 con 0x7fda8815b380 2024-04-24T23:06:16.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.681+0000 7fda85d9b640 1 -- 172.21.15.32:0/187583179 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 429427348 0 0) 0x7fda70001110 con 0x7fda8815cb00 2024-04-24T23:06:16.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.682+0000 7fda8659c640 1 --2- 172.21.15.32:0/187583179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda8815b380 0x7fda8806c5f0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fda780097a0 tx=0x7fda7802a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a090e4e10f03fdee server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:16.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.682+0000 7fda85d9b640 1 -- 172.21.15.32:0/187583179 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda78037070 con 0x7fda8815b380 2024-04-24T23:06:16.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.682+0000 7fda85d9b640 1 -- 172.21.15.32:0/187583179 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda7802adc0 con 0x7fda8815b380 2024-04-24T23:06:16.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.682+0000 7fda85d9b640 1 -- 172.21.15.32:0/187583179 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda78032660 con 0x7fda8815b380 2024-04-24T23:06:16.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.683+0000 7fda8cbe7640 1 -- 172.21.15.32:0/187583179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda8815b380 msgr2=0x7fda8806c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:16.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.683+0000 7fda8cbe7640 1 --2- 172.21.15.32:0/187583179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda8815b380 0x7fda8806c5f0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fda780097a0 tx=0x7fda7802a0b0 comp rx=0 tx=0).stop 2024-04-24T23:06:16.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.683+0000 7fda8cbe7640 1 -- 172.21.15.32:0/187583179 shutdown_connections 2024-04-24T23:06:16.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.683+0000 7fda8cbe7640 1 --2- 172.21.15.32:0/187583179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda8815b380 0x7fda8806c5f0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:16.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.683+0000 7fda8cbe7640 1 -- 172.21.15.32:0/187583179 >> 172.21.15.32:0/187583179 conn(0x7fda8806b110 msgr2=0x7fda8806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:16.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.683+0000 7fda8cbe7640 1 -- 172.21.15.32:0/187583179 shutdown_connections 2024-04-24T23:06:16.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.683+0000 7fda8cbe7640 1 -- 172.21.15.32:0/187583179 wait complete. 2024-04-24T23:06:16.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.684+0000 7fda8cbe7640 1 Processor -- start 2024-04-24T23:06:16.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.684+0000 7fda8cbe7640 1 -- start start 2024-04-24T23:06:16.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.685+0000 7fda8cbe7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda880ceb40 0x7fda881553e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:16.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.685+0000 7fda8cbe7640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fda8815b380 0x7fda881527a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:16.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.685+0000 7fda8cbe7640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda8815cb00 0x7fda88140760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:16.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.685+0000 7fda8cbe7640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fda8815b760 con 0x7fda880ceb40 2024-04-24T23:06:16.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.685+0000 7fda8cbe7640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fda880b50a0 con 0x7fda8815cb00 2024-04-24T23:06:16.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.685+0000 7fda8cbe7640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fda8806cb30 con 0x7fda8815b380 2024-04-24T23:06:16.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.685+0000 7fda8659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda880ceb40 0x7fda881553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:16.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.685+0000 7fda8659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda880ceb40 0x7fda881553e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34952/0 (socket says 172.21.15.32:34952) 2024-04-24T23:06:16.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.685+0000 7fda8659c640 1 -- 172.21.15.32:0/3000005140 learned_addr learned my addr 172.21.15.32:0/3000005140 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:16.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.685+0000 7fda7ffff640 1 --2- 172.21.15.32:0/3000005140 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fda8815b380 0x7fda881527a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:16.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.685+0000 7fda86d9d640 1 --2- 172.21.15.32:0/3000005140 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda8815cb00 0x7fda88140760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:16.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.685+0000 7fda8659c640 1 -- 172.21.15.32:0/3000005140 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fda8815b380 msgr2=0x7fda881527a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:16.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.685+0000 7fda8659c640 1 --2- 172.21.15.32:0/3000005140 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fda8815b380 0x7fda881527a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:16.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.685+0000 7fda8659c640 1 -- 172.21.15.32:0/3000005140 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda8815cb00 msgr2=0x7fda88140760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:16.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.685+0000 7fda8659c640 1 --2- 172.21.15.32:0/3000005140 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda8815cb00 0x7fda88140760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:16.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.685+0000 7fda8659c640 1 -- 172.21.15.32:0/3000005140 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda881f3760 con 0x7fda880ceb40 2024-04-24T23:06:16.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.685+0000 7fda7ffff640 1 --2- 172.21.15.32:0/3000005140 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fda8815b380 0x7fda881527a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:06:16.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.685+0000 7fda86d9d640 1 --2- 172.21.15.32:0/3000005140 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda8815cb00 0x7fda88140760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:06:16.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.685+0000 7fda8659c640 1 --2- 172.21.15.32:0/3000005140 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda880ceb40 0x7fda881553e0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fda780097a0 tx=0x7fda780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:16.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.686+0000 7fda7effd640 1 -- 172.21.15.32:0/3000005140 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda78041070 con 0x7fda880ceb40 2024-04-24T23:06:16.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.686+0000 7fda7effd640 1 -- 172.21.15.32:0/3000005140 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda7803c670 con 0x7fda880ceb40 2024-04-24T23:06:16.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.686+0000 7fda8cbe7640 1 -- 172.21.15.32:0/3000005140 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda88070820 con 0x7fda880ceb40 2024-04-24T23:06:16.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.686+0000 7fda7effd640 1 -- 172.21.15.32:0/3000005140 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda78037040 con 0x7fda880ceb40 2024-04-24T23:06:16.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.686+0000 7fda8cbe7640 1 -- 172.21.15.32:0/3000005140 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda88070de0 con 0x7fda880ceb40 2024-04-24T23:06:16.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.687+0000 7fda8cbe7640 1 -- 172.21.15.32:0/3000005140 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda88068430 con 0x7fda880ceb40 2024-04-24T23:06:16.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.687+0000 7fda7effd640 1 -- 172.21.15.32:0/3000005140 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fda78044070 con 0x7fda880ceb40 2024-04-24T23:06:16.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.688+0000 7fda7effd640 1 --2- 172.21.15.32:0/3000005140 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fda6806be30 0x7fda6806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:16.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.688+0000 7fda7effd640 1 -- 172.21.15.32:0/3000005140 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5796+0+0 (secure 0 0 0) 0x7fda780ab720 con 0x7fda880ceb40 2024-04-24T23:06:16.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.688+0000 7fda7ffff640 1 --2- 172.21.15.32:0/3000005140 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fda6806be30 0x7fda6806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:16.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.688+0000 7fda7ffff640 1 --2- 172.21.15.32:0/3000005140 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fda6806be30 0x7fda6806e2f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fda880cdd90 tx=0x7fda70009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:16.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.692+0000 7fda7effd640 1 -- 172.21.15.32:0/3000005140 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fda7807a070 con 0x7fda880ceb40 2024-04-24T23:06:16.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.911+0000 7fda8cbe7640 1 -- 172.21.15.32:0/3000005140 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fda880afd60 con 0x7fda880ceb40 2024-04-24T23:06:16.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.912+0000 7fda7effd640 1 -- 172.21.15.32:0/3000005140 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+5189 (secure 0 0 0) 0x7fda78079970 con 0x7fda880ceb40 2024-04-24T23:06:16.913 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:06:16.914 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":58,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3361752496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3361752496},{"type":"v1","addr":"172.21.15.121:6837","nonce":3361752496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":6799,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3813453544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3813453544},{"type":"v1","addr":"172.21.15.121:6836","nonce":3813453544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7779,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/1352085651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1352085651},{"type":"v1","addr":"172.21.15.32:6833","nonce":1352085651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:05:33.791068+0000","modified":"2024-04-24T23:05:35.792946+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7794},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7794":{"gid":7794,"name":"a","rank":0,"incarnation":56,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/4206823204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":4206823204},{"type":"v1","addr":"172.21.15.32:6835","nonce":4206823204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7794,"qdb_cluster":[7794]},"id":8}]} 2024-04-24T23:06:16.914 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 58 2024-04-24T23:06:16.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.914+0000 7fda8cbe7640 1 -- 172.21.15.32:0/3000005140 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fda6806be30 msgr2=0x7fda6806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:16.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.914+0000 7fda8cbe7640 1 --2- 172.21.15.32:0/3000005140 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fda6806be30 0x7fda6806e2f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fda880cdd90 tx=0x7fda70009040 comp rx=0 tx=0).stop 2024-04-24T23:06:16.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.914+0000 7fda8cbe7640 1 -- 172.21.15.32:0/3000005140 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda880ceb40 msgr2=0x7fda881553e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:16.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.914+0000 7fda8cbe7640 1 --2- 172.21.15.32:0/3000005140 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda880ceb40 0x7fda881553e0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fda780097a0 tx=0x7fda780046c0 comp rx=0 tx=0).stop 2024-04-24T23:06:16.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.914+0000 7fda8cbe7640 1 -- 172.21.15.32:0/3000005140 shutdown_connections 2024-04-24T23:06:16.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.914+0000 7fda8cbe7640 1 --2- 172.21.15.32:0/3000005140 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fda6806be30 0x7fda6806e2f0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:16.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.914+0000 7fda8cbe7640 1 --2- 172.21.15.32:0/3000005140 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda8815cb00 0x7fda88140760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:16.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.914+0000 7fda8cbe7640 1 --2- 172.21.15.32:0/3000005140 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fda8815b380 0x7fda881527a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:16.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.914+0000 7fda8cbe7640 1 --2- 172.21.15.32:0/3000005140 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda880ceb40 0x7fda881553e0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:16.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.914+0000 7fda8cbe7640 1 -- 172.21.15.32:0/3000005140 >> 172.21.15.32:0/3000005140 conn(0x7fda8806b110 msgr2=0x7fda88150db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:16.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.914+0000 7fda8cbe7640 1 -- 172.21.15.32:0/3000005140 shutdown_connections 2024-04-24T23:06:16.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:16.914+0000 7fda8cbe7640 1 -- 172.21.15.32:0/3000005140 wait complete. 2024-04-24T23:06:16.926 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:06:17.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.025+0000 7f0d2fecd640 1 Processor -- start 2024-04-24T23:06:17.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.025+0000 7f0d2fecd640 1 -- start start 2024-04-24T23:06:17.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.026+0000 7f0d2fecd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0d280b0b80 0x7f0d2806c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:17.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.026+0000 7f0d2fecd640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d280b0770 con 0x7f0d28074210 2024-04-24T23:06:17.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.026+0000 7f0d2fecd640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d280b1010 con 0x7f0d280ae140 2024-04-24T23:06:17.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.026+0000 7f0d2fecd640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f0d280b1c20 con 0x7f0d280b0b80 2024-04-24T23:06:17.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.026+0000 7f0d2d468640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0d280b0b80 0x7f0d2806c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:17.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.026+0000 7f0d2dc69640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0d28074210 0x7f0d280745f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41216/0 (socket says 172.21.15.32:41216) 2024-04-24T23:06:17.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.026+0000 7f0d2dc69640 1 -- 172.21.15.32:0/947725293 learned_addr learned my addr 172.21.15.32:0/947725293 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:06:17.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.026+0000 7f0d2d468640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0d280b0b80 0x7f0d2806c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34958/0 (socket says 172.21.15.32:34958) 2024-04-24T23:06:17.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.026+0000 7f0d2d468640 1 -- 172.21.15.32:0/947725293 >> v1:172.21.15.121:6789/0 conn(0x7f0d280ae140 legacy=0x7f0d280b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:17.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.026+0000 7f0d2d468640 1 -- 172.21.15.32:0/947725293 >> v1:172.21.15.32:6789/0 conn(0x7f0d28074210 legacy=0x7f0d280745f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:17.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.026+0000 7f0d2d468640 1 -- 172.21.15.32:0/947725293 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d280b12f0 con 0x7f0d280b0b80 2024-04-24T23:06:17.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.026+0000 7f0d2cc67640 1 -- 172.21.15.32:0/947725293 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1509730463 0 0) 0x7f0d280b0770 con 0x7f0d28074210 2024-04-24T23:06:17.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.027+0000 7f0d2d468640 1 --2- 172.21.15.32:0/947725293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d280b0b80 0x7f0d2806c480 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f0d180097a0 tx=0x7f0d1802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6312902aa32519a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:17.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.027+0000 7f0d2cc67640 1 -- 172.21.15.32:0/947725293 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d18037070 con 0x7f0d280b0b80 2024-04-24T23:06:17.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.027+0000 7f0d2cc67640 1 -- 172.21.15.32:0/947725293 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d18032440 con 0x7f0d280b0b80 2024-04-24T23:06:17.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.027+0000 7f0d2cc67640 1 -- 172.21.15.32:0/947725293 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d18032810 con 0x7f0d280b0b80 2024-04-24T23:06:17.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.028+0000 7f0d2fecd640 1 -- 172.21.15.32:0/947725293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d280b0b80 msgr2=0x7f0d2806c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:17.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.028+0000 7f0d2fecd640 1 --2- 172.21.15.32:0/947725293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d280b0b80 0x7f0d2806c480 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f0d180097a0 tx=0x7f0d1802a1e0 comp rx=0 tx=0).stop 2024-04-24T23:06:17.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.028+0000 7f0d2fecd640 1 -- 172.21.15.32:0/947725293 shutdown_connections 2024-04-24T23:06:17.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.028+0000 7f0d2fecd640 1 --2- 172.21.15.32:0/947725293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d280b0b80 0x7f0d2806c480 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:17.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.028+0000 7f0d2fecd640 1 -- 172.21.15.32:0/947725293 >> 172.21.15.32:0/947725293 conn(0x7f0d280b2e60 msgr2=0x7f0d280b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:17.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.028+0000 7f0d2fecd640 1 -- 172.21.15.32:0/947725293 shutdown_connections 2024-04-24T23:06:17.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.028+0000 7f0d2fecd640 1 -- 172.21.15.32:0/947725293 wait complete. 2024-04-24T23:06:17.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.029+0000 7f0d2fecd640 1 Processor -- start 2024-04-24T23:06:17.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.029+0000 7f0d2fecd640 1 -- start start 2024-04-24T23:06:17.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.030+0000 7f0d2fecd640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d28074210 0x7f0d280d0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:17.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.030+0000 7f0d2fecd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d280ae140 0x7f0d281550e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:17.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.030+0000 7f0d2fecd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d280c98e0 0x7f0d2815db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:17.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.030+0000 7f0d2fecd640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0d280b3ba0 con 0x7f0d280ae140 2024-04-24T23:06:17.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.030+0000 7f0d2fecd640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0d280b18b0 con 0x7f0d280c98e0 2024-04-24T23:06:17.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.030+0000 7f0d2fecd640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f0d280b3d20 con 0x7f0d28074210 2024-04-24T23:06:17.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.030+0000 7f0d2d468640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d280ae140 0x7f0d281550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:17.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.030+0000 7f0d2d468640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d280ae140 0x7f0d281550e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34966/0 (socket says 172.21.15.32:34966) 2024-04-24T23:06:17.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.030+0000 7f0d2d468640 1 -- 172.21.15.32:0/3942812915 learned_addr learned my addr 172.21.15.32:0/3942812915 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:17.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.030+0000 7f0d2dc69640 1 --2- 172.21.15.32:0/3942812915 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d28074210 0x7f0d280d0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:17.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.030+0000 7f0d2d468640 1 -- 172.21.15.32:0/3942812915 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d28074210 msgr2=0x7f0d280d0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:17.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.030+0000 7f0d2e46a640 1 --2- 172.21.15.32:0/3942812915 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d280c98e0 0x7f0d2815db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:17.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.030+0000 7f0d2d468640 1 --2- 172.21.15.32:0/3942812915 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d28074210 0x7f0d280d0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:17.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.030+0000 7f0d2d468640 1 -- 172.21.15.32:0/3942812915 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d280c98e0 msgr2=0x7f0d2815db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:17.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.030+0000 7f0d2d468640 1 --2- 172.21.15.32:0/3942812915 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d280c98e0 0x7f0d2815db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:17.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.030+0000 7f0d2d468640 1 -- 172.21.15.32:0/3942812915 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d28155620 con 0x7f0d280ae140 2024-04-24T23:06:17.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.031+0000 7f0d2d468640 1 --2- 172.21.15.32:0/3942812915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d280ae140 0x7f0d281550e0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f0d180097a0 tx=0x7f0d180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:17.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.031+0000 7f0d1e7fc640 1 -- 172.21.15.32:0/3942812915 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d18041070 con 0x7f0d280ae140 2024-04-24T23:06:17.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.031+0000 7f0d1e7fc640 1 -- 172.21.15.32:0/3942812915 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d1803c870 con 0x7f0d280ae140 2024-04-24T23:06:17.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.031+0000 7f0d2fecd640 1 -- 172.21.15.32:0/3942812915 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d280ca160 con 0x7f0d280ae140 2024-04-24T23:06:17.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.031+0000 7f0d1e7fc640 1 -- 172.21.15.32:0/3942812915 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d18037040 con 0x7f0d280ae140 2024-04-24T23:06:17.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.031+0000 7f0d2fecd640 1 -- 172.21.15.32:0/3942812915 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d280ca720 con 0x7f0d280ae140 2024-04-24T23:06:17.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.032+0000 7f0d2fecd640 1 -- 172.21.15.32:0/3942812915 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d28068430 con 0x7f0d280ae140 2024-04-24T23:06:17.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.032+0000 7f0d1e7fc640 1 -- 172.21.15.32:0/3942812915 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0d18044070 con 0x7f0d280ae140 2024-04-24T23:06:17.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.033+0000 7f0d1e7fc640 1 --2- 172.21.15.32:0/3942812915 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0d0c06be30 0x7f0d0c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:17.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.033+0000 7f0d1e7fc640 1 -- 172.21.15.32:0/3942812915 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5796+0+0 (secure 0 0 0) 0x7f0d180abda0 con 0x7f0d280ae140 2024-04-24T23:06:17.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.033+0000 7f0d2dc69640 1 --2- 172.21.15.32:0/3942812915 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0d0c06be30 0x7f0d0c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:17.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.034+0000 7f0d2dc69640 1 --2- 172.21.15.32:0/3942812915 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0d0c06be30 0x7f0d0c06e2f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f0d240012e0 tx=0x7f0d24009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:17.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.037+0000 7f0d1e7fc640 1 -- 172.21.15.32:0/3942812915 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0d1807a810 con 0x7f0d280ae140 2024-04-24T23:06:17.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.225+0000 7f0d2fecd640 1 -- 172.21.15.32:0/3942812915 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0d2806b220 con 0x7f0d280ae140 2024-04-24T23:06:17.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.226+0000 7f0d1e7fc640 1 -- 172.21.15.32:0/3942812915 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) ==== 74+0+16836 (secure 0 0 0) 0x7f0d1807a110 con 0x7f0d280ae140 2024-04-24T23:06:17.228 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:06:17.228 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":145,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:06:06.811191+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:05:32.064234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-24T23:05:32.760777+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.17","mappings":[{"from":4,"to":3}]},{"pgid":"17.8","mappings":[{"from":6,"to":5}]},{"pgid":"17.3f","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:06:17.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.229+0000 7f0d2fecd640 1 -- 172.21.15.32:0/3942812915 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0d0c06be30 msgr2=0x7f0d0c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:17.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.229+0000 7f0d2fecd640 1 --2- 172.21.15.32:0/3942812915 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0d0c06be30 0x7f0d0c06e2f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f0d240012e0 tx=0x7f0d24009040 comp rx=0 tx=0).stop 2024-04-24T23:06:17.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.229+0000 7f0d2fecd640 1 -- 172.21.15.32:0/3942812915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d280ae140 msgr2=0x7f0d281550e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:17.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.229+0000 7f0d2fecd640 1 --2- 172.21.15.32:0/3942812915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d280ae140 0x7f0d281550e0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f0d180097a0 tx=0x7f0d180046c0 comp rx=0 tx=0).stop 2024-04-24T23:06:17.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.229+0000 7f0d2fecd640 1 -- 172.21.15.32:0/3942812915 shutdown_connections 2024-04-24T23:06:17.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.229+0000 7f0d2fecd640 1 --2- 172.21.15.32:0/3942812915 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0d0c06be30 0x7f0d0c06e2f0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:17.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.229+0000 7f0d2fecd640 1 --2- 172.21.15.32:0/3942812915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d280ae140 0x7f0d281550e0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:17.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.229+0000 7f0d2fecd640 1 --2- 172.21.15.32:0/3942812915 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d280c98e0 0x7f0d2815db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:17.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.229+0000 7f0d2fecd640 1 --2- 172.21.15.32:0/3942812915 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d28074210 0x7f0d280d0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:17.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.229+0000 7f0d2fecd640 1 -- 172.21.15.32:0/3942812915 >> 172.21.15.32:0/3942812915 conn(0x7f0d280b2e60 msgr2=0x7f0d280b20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:17.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.229+0000 7f0d2fecd640 1 -- 172.21.15.32:0/3942812915 shutdown_connections 2024-04-24T23:06:17.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.229+0000 7f0d2fecd640 1 -- 172.21.15.32:0/3942812915 wait complete. 2024-04-24T23:06:17.241 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:05:32.064234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '139', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:05:32.760777+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '145', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:06:17.242 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:06:17.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.344+0000 7f65697a2640 1 Processor -- start 2024-04-24T23:06:17.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.344+0000 7f65697a2640 1 -- start start 2024-04-24T23:06:17.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.345+0000 7f65697a2640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f65640b5560 0x7f656415e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:17.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.345+0000 7f65697a2640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f65640b6050 0x7f656415ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:17.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.345+0000 7f65697a2640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65640cd910 con 0x7f65640cd3c0 2024-04-24T23:06:17.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.345+0000 7f65697a2640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f65640b6470 con 0x7f65640b6050 2024-04-24T23:06:17.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.345+0000 7f65697a2640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f65640cdbf0 con 0x7f65640b5560 2024-04-24T23:06:17.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.345+0000 7f65637fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f65640b6050 0x7f656415ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:17.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.345+0000 7f65637fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f65640b6050 0x7f656415ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34972/0 (socket says 172.21.15.32:34972) 2024-04-24T23:06:17.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.345+0000 7f65637fe640 1 -- 172.21.15.32:0/537699323 learned_addr learned my addr 172.21.15.32:0/537699323 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:17.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.345+0000 7f6562ffd640 1 --2- 172.21.15.32:0/537699323 >> v2:172.21.15.121:3300/0 conn(0x7f65640b5560 0x7f656415e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:17.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.345+0000 7f65637fe640 1 -- 172.21.15.32:0/537699323 >> v2:172.21.15.121:3300/0 conn(0x7f65640b5560 msgr2=0x7f656415e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:17.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.345+0000 7f65637fe640 1 --2- 172.21.15.32:0/537699323 >> v2:172.21.15.121:3300/0 conn(0x7f65640b5560 0x7f656415e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:17.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.345+0000 7f65637fe640 1 -- 172.21.15.32:0/537699323 >> v1:172.21.15.121:6789/0 conn(0x7f65640cd3c0 legacy=0x7f65640b5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:17.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.345+0000 7f65637fe640 1 -- 172.21.15.32:0/537699323 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65640cded0 con 0x7f65640b6050 2024-04-24T23:06:17.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.346+0000 7f65637fe640 1 --2- 172.21.15.32:0/537699323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65640b6050 0x7f656415ec00 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f656406ede0 tx=0x7f655c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc86d3b3bd51c0ba server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:17.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.346+0000 7f6561ffb640 1 -- 172.21.15.32:0/537699323 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655c036070 con 0x7f65640b6050 2024-04-24T23:06:17.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.346+0000 7f6561ffb640 1 -- 172.21.15.32:0/537699323 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f655c031600 con 0x7f65640b6050 2024-04-24T23:06:17.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.346+0000 7f6561ffb640 1 -- 172.21.15.32:0/537699323 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655c0319d0 con 0x7f65640b6050 2024-04-24T23:06:17.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.347+0000 7f65697a2640 1 -- 172.21.15.32:0/537699323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65640b6050 msgr2=0x7f656415ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:17.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.347+0000 7f65697a2640 1 --2- 172.21.15.32:0/537699323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65640b6050 0x7f656415ec00 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f656406ede0 tx=0x7f655c02a730 comp rx=0 tx=0).stop 2024-04-24T23:06:17.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.347+0000 7f65697a2640 1 -- 172.21.15.32:0/537699323 shutdown_connections 2024-04-24T23:06:17.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.347+0000 7f65697a2640 1 --2- 172.21.15.32:0/537699323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65640b6050 0x7f656415ec00 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:17.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.347+0000 7f65697a2640 1 --2- 172.21.15.32:0/537699323 >> v2:172.21.15.121:3300/0 conn(0x7f65640b5560 0x7f656415e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:17.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.347+0000 7f65697a2640 1 -- 172.21.15.32:0/537699323 >> 172.21.15.32:0/537699323 conn(0x7f656406b110 msgr2=0x7f656406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:17.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.347+0000 7f65697a2640 1 -- 172.21.15.32:0/537699323 shutdown_connections 2024-04-24T23:06:17.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.347+0000 7f65697a2640 1 -- 172.21.15.32:0/537699323 wait complete. 2024-04-24T23:06:17.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.348+0000 7f65697a2640 1 Processor -- start 2024-04-24T23:06:17.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.348+0000 7f65697a2640 1 -- start start 2024-04-24T23:06:17.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.349+0000 7f65697a2640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f65640b5560 0x7f656413e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:17.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.349+0000 7f65697a2640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65640b6050 0x7f65641e4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:17.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.349+0000 7f65697a2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65640cd3c0 0x7f6564141700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:17.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.349+0000 7f65697a2640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f65641514b0 con 0x7f65640cd3c0 2024-04-24T23:06:17.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.349+0000 7f65697a2640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f65640ce490 con 0x7f65640b6050 2024-04-24T23:06:17.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.349+0000 7f65697a2640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f6564151630 con 0x7f65640b5560 2024-04-24T23:06:17.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.349+0000 7f65637fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65640cd3c0 0x7f6564141700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:17.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.349+0000 7f65637fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65640cd3c0 0x7f6564141700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34984/0 (socket says 172.21.15.32:34984) 2024-04-24T23:06:17.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.349+0000 7f65637fe640 1 -- 172.21.15.32:0/2956219659 learned_addr learned my addr 172.21.15.32:0/2956219659 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:17.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.349+0000 7f6562ffd640 1 --2- 172.21.15.32:0/2956219659 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f65640b5560 0x7f656413e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:17.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.349+0000 7f65627fc640 1 --2- 172.21.15.32:0/2956219659 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65640b6050 0x7f65641e4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:17.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.349+0000 7f65637fe640 1 -- 172.21.15.32:0/2956219659 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f65640b5560 msgr2=0x7f656413e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:17.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.349+0000 7f65637fe640 1 --2- 172.21.15.32:0/2956219659 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f65640b5560 0x7f656413e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:17.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.349+0000 7f65637fe640 1 -- 172.21.15.32:0/2956219659 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65640b6050 msgr2=0x7f65641e4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:17.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.349+0000 7f65637fe640 1 --2- 172.21.15.32:0/2956219659 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65640b6050 0x7f65641e4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:17.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.349+0000 7f65637fe640 1 -- 172.21.15.32:0/2956219659 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f656415de10 con 0x7f65640cd3c0 2024-04-24T23:06:17.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.350+0000 7f65637fe640 1 --2- 172.21.15.32:0/2956219659 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65640cd3c0 0x7f6564141700 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f655c009f40 tx=0x7f655c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:17.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.350+0000 7f655b7fe640 1 -- 172.21.15.32:0/2956219659 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655c036050 con 0x7f65640cd3c0 2024-04-24T23:06:17.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.350+0000 7f655b7fe640 1 -- 172.21.15.32:0/2956219659 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f655c042070 con 0x7f65640cd3c0 2024-04-24T23:06:17.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.350+0000 7f655b7fe640 1 -- 172.21.15.32:0/2956219659 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655c009510 con 0x7f65640cd3c0 2024-04-24T23:06:17.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.350+0000 7f65697a2640 1 -- 172.21.15.32:0/2956219659 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65641e4d30 con 0x7f65640cd3c0 2024-04-24T23:06:17.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.350+0000 7f65697a2640 1 -- 172.21.15.32:0/2956219659 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65641e52f0 con 0x7f65640cd3c0 2024-04-24T23:06:17.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.351+0000 7f65697a2640 1 -- 172.21.15.32:0/2956219659 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6564068430 con 0x7f65640cd3c0 2024-04-24T23:06:17.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.351+0000 7f655b7fe640 1 -- 172.21.15.32:0/2956219659 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f655c007c70 con 0x7f65640cd3c0 2024-04-24T23:06:17.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.352+0000 7f655b7fe640 1 --2- 172.21.15.32:0/2956219659 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f654406be30 0x7f654406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:17.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.352+0000 7f655b7fe640 1 -- 172.21.15.32:0/2956219659 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5796+0+0 (secure 0 0 0) 0x7f655c0abdb0 con 0x7f65640cd3c0 2024-04-24T23:06:17.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.352+0000 7f6562ffd640 1 --2- 172.21.15.32:0/2956219659 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f654406be30 0x7f654406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:17.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.352+0000 7f6562ffd640 1 --2- 172.21.15.32:0/2956219659 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f654406be30 0x7f654406e2f0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6554006fd0 tx=0x7f6554008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:17.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.356+0000 7f655b7fe640 1 -- 172.21.15.32:0/2956219659 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f655c045360 con 0x7f65640cd3c0 2024-04-24T23:06:17.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.537+0000 7f65697a2640 1 -- 172.21.15.32:0/2956219659 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f656406d5f0 con 0x7f65640cd3c0 2024-04-24T23:06:17.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.537+0000 7f655b7fe640 1 -- 172.21.15.32:0/2956219659 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v145) ==== 93+0+10 (secure 0 0 0) 0x7f655c07a680 con 0x7f65640cd3c0 2024-04-24T23:06:17.539 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T23:06:17.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.539+0000 7f65697a2640 1 -- 172.21.15.32:0/2956219659 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f654406be30 msgr2=0x7f654406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:17.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.539+0000 7f65697a2640 1 --2- 172.21.15.32:0/2956219659 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f654406be30 0x7f654406e2f0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6554006fd0 tx=0x7f6554008040 comp rx=0 tx=0).stop 2024-04-24T23:06:17.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.539+0000 7f65697a2640 1 -- 172.21.15.32:0/2956219659 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65640cd3c0 msgr2=0x7f6564141700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:17.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.539+0000 7f65697a2640 1 --2- 172.21.15.32:0/2956219659 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65640cd3c0 0x7f6564141700 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f655c009f40 tx=0x7f655c004a40 comp rx=0 tx=0).stop 2024-04-24T23:06:17.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.539+0000 7f65697a2640 1 -- 172.21.15.32:0/2956219659 shutdown_connections 2024-04-24T23:06:17.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.539+0000 7f65697a2640 1 --2- 172.21.15.32:0/2956219659 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f654406be30 0x7f654406e2f0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:17.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.539+0000 7f65697a2640 1 --2- 172.21.15.32:0/2956219659 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65640cd3c0 0x7f6564141700 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:17.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.539+0000 7f65697a2640 1 --2- 172.21.15.32:0/2956219659 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65640b6050 0x7f65641e4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:17.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.539+0000 7f65697a2640 1 --2- 172.21.15.32:0/2956219659 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f65640b5560 0x7f656413e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:17.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.539+0000 7f65697a2640 1 -- 172.21.15.32:0/2956219659 >> 172.21.15.32:0/2956219659 conn(0x7f656406b110 msgr2=0x7f65640b1660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:17.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.539+0000 7f65697a2640 1 -- 172.21.15.32:0/2956219659 shutdown_connections 2024-04-24T23:06:17.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.539+0000 7f65697a2640 1 -- 172.21.15.32:0/2956219659 wait complete. 2024-04-24T23:06:17.552 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:06:17.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.653+0000 7f20473ce640 1 Processor -- start 2024-04-24T23:06:17.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.653+0000 7f20473ce640 1 -- start start 2024-04-24T23:06:17.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.654+0000 7f20473ce640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20401476d0 0x7f20400ce630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:17.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.654+0000 7f20473ce640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f20400cfb70 0x7f204006b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:17.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.654+0000 7f20473ce640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20400afdd0 con 0x7f204006b650 2024-04-24T23:06:17.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.654+0000 7f20473ce640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f20400aedc0 con 0x7f20400cfb70 2024-04-24T23:06:17.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.654+0000 7f20473ce640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f20400af940 con 0x7f20401476d0 2024-04-24T23:06:17.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.654+0000 7f2044969640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f20400cfb70 0x7f204006b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:17.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.654+0000 7f204596b640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f204006b650 0x7f20400cd180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41218/0 (socket says 172.21.15.32:41218) 2024-04-24T23:06:17.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.654+0000 7f204596b640 1 -- 172.21.15.32:0/3754612956 learned_addr learned my addr 172.21.15.32:0/3754612956 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:06:17.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.655+0000 7f204516a640 1 --2- 172.21.15.32:0/3754612956 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20401476d0 0x7f20400ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:17.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.655+0000 7f2044969640 1 -- 172.21.15.32:0/3754612956 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20401476d0 msgr2=0x7f20400ce630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:17.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.655+0000 7f2044969640 1 --2- 172.21.15.32:0/3754612956 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20401476d0 0x7f20400ce630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:17.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.655+0000 7f2044969640 1 -- 172.21.15.32:0/3754612956 >> v1:172.21.15.32:6789/0 conn(0x7f204006b650 legacy=0x7f20400cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:17.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.655+0000 7f2044969640 1 -- 172.21.15.32:0/3754612956 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20400af010 con 0x7f20400cfb70 2024-04-24T23:06:17.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.655+0000 7f203ffff640 1 -- 172.21.15.32:0/3754612956 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3124849075 0 0) 0x7f20400afdd0 con 0x7f204006b650 2024-04-24T23:06:17.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.655+0000 7f2044969640 1 --2- 172.21.15.32:0/3754612956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20400cfb70 0x7f204006b110 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f2034009790 tx=0x7f203402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9028608655162da server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:17.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.656+0000 7f203ffff640 1 -- 172.21.15.32:0/3754612956 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2034037070 con 0x7f20400cfb70 2024-04-24T23:06:17.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.656+0000 7f203ffff640 1 -- 172.21.15.32:0/3754612956 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f203402ae50 con 0x7f20400cfb70 2024-04-24T23:06:17.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.656+0000 7f203ffff640 1 -- 172.21.15.32:0/3754612956 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2034032660 con 0x7f20400cfb70 2024-04-24T23:06:17.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.656+0000 7f20473ce640 1 -- 172.21.15.32:0/3754612956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20400cfb70 msgr2=0x7f204006b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:17.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.656+0000 7f20473ce640 1 --2- 172.21.15.32:0/3754612956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20400cfb70 0x7f204006b110 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f2034009790 tx=0x7f203402a100 comp rx=0 tx=0).stop 2024-04-24T23:06:17.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.656+0000 7f20473ce640 1 -- 172.21.15.32:0/3754612956 shutdown_connections 2024-04-24T23:06:17.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.656+0000 7f20473ce640 1 --2- 172.21.15.32:0/3754612956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20400cfb70 0x7f204006b110 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:17.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.656+0000 7f20473ce640 1 --2- 172.21.15.32:0/3754612956 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20401476d0 0x7f20400ce630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:17.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.656+0000 7f20473ce640 1 -- 172.21.15.32:0/3754612956 >> 172.21.15.32:0/3754612956 conn(0x7f20400d0730 msgr2=0x7f20400d0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:17.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.656+0000 7f20473ce640 1 -- 172.21.15.32:0/3754612956 shutdown_connections 2024-04-24T23:06:17.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.656+0000 7f20473ce640 1 -- 172.21.15.32:0/3754612956 wait complete. 2024-04-24T23:06:17.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.657+0000 7f20473ce640 1 Processor -- start 2024-04-24T23:06:17.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.657+0000 7f20473ce640 1 -- start start 2024-04-24T23:06:17.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.658+0000 7f20473ce640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f204006b650 0x7f204013e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:17.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.658+0000 7f20473ce640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20400cfb70 0x7f204015dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:17.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.658+0000 7f20473ce640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20401476d0 0x7f2040140130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:17.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.658+0000 7f20473ce640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2040147ab0 con 0x7f204006b650 2024-04-24T23:06:17.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.658+0000 7f20473ce640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f20400af5d0 con 0x7f20400cfb70 2024-04-24T23:06:17.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.658+0000 7f20473ce640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f2040148470 con 0x7f20401476d0 2024-04-24T23:06:17.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.658+0000 7f204516a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f204006b650 0x7f204013e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:17.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.658+0000 7f204516a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f204006b650 0x7f204013e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34996/0 (socket says 172.21.15.32:34996) 2024-04-24T23:06:17.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.658+0000 7f204516a640 1 -- 172.21.15.32:0/529013194 learned_addr learned my addr 172.21.15.32:0/529013194 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:17.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.658+0000 7f2044969640 1 --2- 172.21.15.32:0/529013194 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20400cfb70 0x7f204015dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:17.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.658+0000 7f204596b640 1 --2- 172.21.15.32:0/529013194 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20401476d0 0x7f2040140130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:17.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.659+0000 7f204516a640 1 -- 172.21.15.32:0/529013194 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20401476d0 msgr2=0x7f2040140130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:17.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.659+0000 7f204516a640 1 --2- 172.21.15.32:0/529013194 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20401476d0 0x7f2040140130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:17.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.659+0000 7f204516a640 1 -- 172.21.15.32:0/529013194 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20400cfb70 msgr2=0x7f204015dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:17.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.659+0000 7f204516a640 1 --2- 172.21.15.32:0/529013194 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20400cfb70 0x7f204015dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:17.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.659+0000 7f204516a640 1 -- 172.21.15.32:0/529013194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f204013d710 con 0x7f204006b650 2024-04-24T23:06:17.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.659+0000 7f2044969640 1 --2- 172.21.15.32:0/529013194 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20400cfb70 0x7f204015dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:06:17.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.659+0000 7f204516a640 1 --2- 172.21.15.32:0/529013194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f204006b650 0x7f204013e810 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f203800e9b0 tx=0x7f203800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:17.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.659+0000 7f203dffb640 1 -- 172.21.15.32:0/529013194 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2038017070 con 0x7f204006b650 2024-04-24T23:06:17.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.659+0000 7f203dffb640 1 -- 172.21.15.32:0/529013194 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f203800c9a0 con 0x7f204006b650 2024-04-24T23:06:17.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.659+0000 7f20473ce640 1 -- 172.21.15.32:0/529013194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f204015f750 con 0x7f204006b650 2024-04-24T23:06:17.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.659+0000 7f203dffb640 1 -- 172.21.15.32:0/529013194 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203800cd90 con 0x7f204006b650 2024-04-24T23:06:17.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.659+0000 7f20473ce640 1 -- 172.21.15.32:0/529013194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f204015e420 con 0x7f204006b650 2024-04-24T23:06:17.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.660+0000 7f20473ce640 1 -- 172.21.15.32:0/529013194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2040068430 con 0x7f204006b650 2024-04-24T23:06:17.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.661+0000 7f203dffb640 1 -- 172.21.15.32:0/529013194 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f203801a070 con 0x7f204006b650 2024-04-24T23:06:17.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.661+0000 7f203dffb640 1 --2- 172.21.15.32:0/529013194 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f202c06be30 0x7f202c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:17.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.661+0000 7f203dffb640 1 -- 172.21.15.32:0/529013194 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5796+0+0 (secure 0 0 0) 0x7f203808b240 con 0x7f204006b650 2024-04-24T23:06:17.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.661+0000 7f2044969640 1 --2- 172.21.15.32:0/529013194 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f202c06be30 0x7f202c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:17.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.662+0000 7f2044969640 1 --2- 172.21.15.32:0/529013194 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f202c06be30 0x7f202c06e2f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f2034034000 tx=0x7f203402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:17.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.665+0000 7f203dffb640 1 -- 172.21.15.32:0/529013194 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2038059b90 con 0x7f204006b650 2024-04-24T23:06:17.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.850+0000 7f20473ce640 1 -- 172.21.15.32:0/529013194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f20401464a0 con 0x7f204006b650 2024-04-24T23:06:17.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.851+0000 7f203dffb640 1 -- 172.21.15.32:0/529013194 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v145) ==== 104+0+11 (secure 0 0 0) 0x7f2038059490 con 0x7f204006b650 2024-04-24T23:06:17.853 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-24T23:06:17.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.853+0000 7f20473ce640 1 -- 172.21.15.32:0/529013194 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f202c06be30 msgr2=0x7f202c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:17.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.853+0000 7f20473ce640 1 --2- 172.21.15.32:0/529013194 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f202c06be30 0x7f202c06e2f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f2034034000 tx=0x7f203402e000 comp rx=0 tx=0).stop 2024-04-24T23:06:17.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.853+0000 7f20473ce640 1 -- 172.21.15.32:0/529013194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f204006b650 msgr2=0x7f204013e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:17.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.853+0000 7f20473ce640 1 --2- 172.21.15.32:0/529013194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f204006b650 0x7f204013e810 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f203800e9b0 tx=0x7f203800ee70 comp rx=0 tx=0).stop 2024-04-24T23:06:17.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.853+0000 7f20473ce640 1 -- 172.21.15.32:0/529013194 shutdown_connections 2024-04-24T23:06:17.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.853+0000 7f20473ce640 1 --2- 172.21.15.32:0/529013194 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f202c06be30 0x7f202c06e2f0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:17.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.853+0000 7f20473ce640 1 --2- 172.21.15.32:0/529013194 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20401476d0 0x7f2040140130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:17.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.853+0000 7f20473ce640 1 --2- 172.21.15.32:0/529013194 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20400cfb70 0x7f204015dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:17.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.853+0000 7f20473ce640 1 --2- 172.21.15.32:0/529013194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f204006b650 0x7f204013e810 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:17.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.853+0000 7f20473ce640 1 -- 172.21.15.32:0/529013194 >> 172.21.15.32:0/529013194 conn(0x7f20400d0730 msgr2=0x7f2040146890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:17.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.853+0000 7f20473ce640 1 -- 172.21.15.32:0/529013194 shutdown_connections 2024-04-24T23:06:17.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.853+0000 7f20473ce640 1 -- 172.21.15.32:0/529013194 wait complete. 2024-04-24T23:06:17.865 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:06:17.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.967+0000 7f7560441640 1 Processor -- start 2024-04-24T23:06:17.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.967+0000 7f7560441640 1 -- start start 2024-04-24T23:06:17.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.968+0000 7f7560441640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f75580b44d0 0x7f75580b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:17.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.968+0000 7f7560441640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f75580af890 0x7f755815cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:17.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.968+0000 7f7560441640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75580b6430 con 0x7f755806c5f0 2024-04-24T23:06:17.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.968+0000 7f7560441640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f7558156f80 con 0x7f75580af890 2024-04-24T23:06:17.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.968+0000 7f7560441640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f75580b6150 con 0x7f75580b44d0 2024-04-24T23:06:17.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.968+0000 7f755e9de640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f75580af890 0x7f755815cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:17.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.968+0000 7f755e1dd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f755806c5f0 0x7f755806c9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41222/0 (socket says 172.21.15.32:41222) 2024-04-24T23:06:17.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.969+0000 7f755e1dd640 1 -- 172.21.15.32:0/812929070 learned_addr learned my addr 172.21.15.32:0/812929070 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:06:17.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.969+0000 7f755d9dc640 1 --2- 172.21.15.32:0/812929070 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f75580b44d0 0x7f75580b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:17.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.969+0000 7f755e9de640 1 -- 172.21.15.32:0/812929070 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f75580b44d0 msgr2=0x7f75580b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:17.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.969+0000 7f755e9de640 1 --2- 172.21.15.32:0/812929070 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f75580b44d0 0x7f75580b5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:17.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.969+0000 7f755e9de640 1 -- 172.21.15.32:0/812929070 >> v1:172.21.15.32:6789/0 conn(0x7f755806c5f0 legacy=0x7f755806c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:17.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.969+0000 7f755e9de640 1 -- 172.21.15.32:0/812929070 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75580b39f0 con 0x7f75580af890 2024-04-24T23:06:17.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.969+0000 7f755d1db640 1 -- 172.21.15.32:0/812929070 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4230595941 0 0) 0x7f75580b6430 con 0x7f755806c5f0 2024-04-24T23:06:17.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.970+0000 7f755e9de640 1 --2- 172.21.15.32:0/812929070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75580af890 0x7f755815cac0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f7550009d90 tx=0x7f755002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba0530106d1b2245 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:17.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.970+0000 7f755d1db640 1 -- 172.21.15.32:0/812929070 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7550031600 con 0x7f75580af890 2024-04-24T23:06:17.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.970+0000 7f755d1db640 1 -- 172.21.15.32:0/812929070 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75500317a0 con 0x7f75580af890 2024-04-24T23:06:17.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.970+0000 7f755d1db640 1 -- 172.21.15.32:0/812929070 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7550031b70 con 0x7f75580af890 2024-04-24T23:06:17.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.970+0000 7f7560441640 1 -- 172.21.15.32:0/812929070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75580af890 msgr2=0x7f755815cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:17.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.970+0000 7f7560441640 1 --2- 172.21.15.32:0/812929070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75580af890 0x7f755815cac0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f7550009d90 tx=0x7f755002a5d0 comp rx=0 tx=0).stop 2024-04-24T23:06:17.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.970+0000 7f7560441640 1 -- 172.21.15.32:0/812929070 shutdown_connections 2024-04-24T23:06:17.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.970+0000 7f7560441640 1 --2- 172.21.15.32:0/812929070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75580af890 0x7f755815cac0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:17.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.970+0000 7f7560441640 1 --2- 172.21.15.32:0/812929070 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f75580b44d0 0x7f75580b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:17.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.970+0000 7f7560441640 1 -- 172.21.15.32:0/812929070 >> 172.21.15.32:0/812929070 conn(0x7f755806b110 msgr2=0x7f755806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:17.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.970+0000 7f7560441640 1 -- 172.21.15.32:0/812929070 shutdown_connections 2024-04-24T23:06:17.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.970+0000 7f7560441640 1 -- 172.21.15.32:0/812929070 wait complete. 2024-04-24T23:06:17.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.972+0000 7f7560441640 1 Processor -- start 2024-04-24T23:06:17.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.972+0000 7f7560441640 1 -- start start 2024-04-24T23:06:17.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.972+0000 7f7560441640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f755806c5f0 0x7f755814ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:17.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.972+0000 7f7560441640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75580af890 0x7f75580ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:17.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.972+0000 7f7560441640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f75580b44d0 0x7f75580aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:17.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.972+0000 7f7560441640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f75580b5dd0 con 0x7f75580af890 2024-04-24T23:06:17.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.972+0000 7f7560441640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f75580b3fb0 con 0x7f755806c5f0 2024-04-24T23:06:17.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.972+0000 7f7560441640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f755815d000 con 0x7f75580b44d0 2024-04-24T23:06:17.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.973+0000 7f755d9dc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75580af890 0x7f75580ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:17.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.973+0000 7f755d9dc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75580af890 0x7f75580ae8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35014/0 (socket says 172.21.15.32:35014) 2024-04-24T23:06:17.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.973+0000 7f755d9dc640 1 -- 172.21.15.32:0/174015135 learned_addr learned my addr 172.21.15.32:0/174015135 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:17.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.973+0000 7f755d9dc640 1 -- 172.21.15.32:0/174015135 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f75580b44d0 msgr2=0x7f75580aedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:17.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.973+0000 7f755d9dc640 1 --2- 172.21.15.32:0/174015135 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f75580b44d0 0x7f75580aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:17.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.973+0000 7f755e1dd640 1 --2- 172.21.15.32:0/174015135 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f755806c5f0 0x7f755814ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:17.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.973+0000 7f755d9dc640 1 -- 172.21.15.32:0/174015135 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f755806c5f0 msgr2=0x7f755814ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:17.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.973+0000 7f755d9dc640 1 --2- 172.21.15.32:0/174015135 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f755806c5f0 0x7f755814ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:17.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.973+0000 7f755d9dc640 1 -- 172.21.15.32:0/174015135 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75580b04b0 con 0x7f75580af890 2024-04-24T23:06:17.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.973+0000 7f755e1dd640 1 --2- 172.21.15.32:0/174015135 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f755806c5f0 0x7f755814ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:06:17.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.973+0000 7f755d9dc640 1 --2- 172.21.15.32:0/174015135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75580af890 0x7f75580ae8b0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f754800d5f0 tx=0x7f754800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:17.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.973+0000 7f754effd640 1 -- 172.21.15.32:0/174015135 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75480125f0 con 0x7f75580af890 2024-04-24T23:06:17.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.973+0000 7f754effd640 1 -- 172.21.15.32:0/174015135 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7548019070 con 0x7f75580af890 2024-04-24T23:06:17.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.973+0000 7f7560441640 1 -- 172.21.15.32:0/174015135 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75580b0790 con 0x7f75580af890 2024-04-24T23:06:17.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.973+0000 7f754effd640 1 -- 172.21.15.32:0/174015135 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75480129d0 con 0x7f75580af890 2024-04-24T23:06:17.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.973+0000 7f7560441640 1 -- 172.21.15.32:0/174015135 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75580b0d50 con 0x7f75580af890 2024-04-24T23:06:17.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.974+0000 7f7560441640 1 -- 172.21.15.32:0/174015135 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7558068430 con 0x7f75580af890 2024-04-24T23:06:17.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.975+0000 7f754effd640 1 -- 172.21.15.32:0/174015135 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7548012b70 con 0x7f75580af890 2024-04-24T23:06:17.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.975+0000 7f754effd640 1 --2- 172.21.15.32:0/174015135 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f753c06be60 0x7f753c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:17.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.975+0000 7f754effd640 1 -- 172.21.15.32:0/174015135 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5796+0+0 (secure 0 0 0) 0x7f754808b200 con 0x7f75580af890 2024-04-24T23:06:17.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.976+0000 7f755e1dd640 1 --2- 172.21.15.32:0/174015135 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f753c06be60 0x7f753c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:17.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.976+0000 7f755e1dd640 1 --2- 172.21.15.32:0/174015135 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f753c06be60 0x7f753c06e320 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f7554000f60 tx=0x7f7554009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:17.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:17.980+0000 7f754effd640 1 -- 172.21.15.32:0/174015135 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7548059be0 con 0x7f75580af890 2024-04-24T23:06:18.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.161+0000 7f7560441640 1 -- 172.21.15.32:0/174015135 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f75580b5670 con 0x7f75580af890 2024-04-24T23:06:18.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.161+0000 7f754effd640 1 -- 172.21.15.32:0/174015135 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v145) ==== 100+0+11 (secure 0 0 0) 0x7f75480594e0 con 0x7f75580af890 2024-04-24T23:06:18.163 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-24T23:06:18.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.163+0000 7f7560441640 1 -- 172.21.15.32:0/174015135 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f753c06be60 msgr2=0x7f753c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:18.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.163+0000 7f7560441640 1 --2- 172.21.15.32:0/174015135 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f753c06be60 0x7f753c06e320 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f7554000f60 tx=0x7f7554009040 comp rx=0 tx=0).stop 2024-04-24T23:06:18.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.163+0000 7f7560441640 1 -- 172.21.15.32:0/174015135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75580af890 msgr2=0x7f75580ae8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:18.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.163+0000 7f7560441640 1 --2- 172.21.15.32:0/174015135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75580af890 0x7f75580ae8b0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f754800d5f0 tx=0x7f754800dab0 comp rx=0 tx=0).stop 2024-04-24T23:06:18.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.163+0000 7f7560441640 1 -- 172.21.15.32:0/174015135 shutdown_connections 2024-04-24T23:06:18.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.163+0000 7f7560441640 1 --2- 172.21.15.32:0/174015135 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f753c06be60 0x7f753c06e320 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:18.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.163+0000 7f7560441640 1 --2- 172.21.15.32:0/174015135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75580af890 0x7f75580ae8b0 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:18.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.163+0000 7f7560441640 1 --2- 172.21.15.32:0/174015135 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f75580b44d0 0x7f75580aedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:18.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.163+0000 7f7560441640 1 --2- 172.21.15.32:0/174015135 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f755806c5f0 0x7f755814ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:18.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.163+0000 7f7560441640 1 -- 172.21.15.32:0/174015135 >> 172.21.15.32:0/174015135 conn(0x7f755806b110 msgr2=0x7f755806c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:18.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.163+0000 7f7560441640 1 -- 172.21.15.32:0/174015135 shutdown_connections 2024-04-24T23:06:18.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.163+0000 7f7560441640 1 -- 172.21.15.32:0/174015135 wait complete. 2024-04-24T23:06:18.175 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:06:18.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.276+0000 7f7ee34da640 1 Processor -- start 2024-04-24T23:06:18.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.276+0000 7f7ee34da640 1 -- start start 2024-04-24T23:06:18.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.277+0000 7f7ee34da640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7edc15b380 0x7f7edc06c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:18.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.277+0000 7f7ee34da640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7edc15cb00 0x7f7edc06cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:18.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.277+0000 7f7ee34da640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7edc0ce430 0x7f7edc0ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:18.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.277+0000 7f7ee34da640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f7edc0b4520 con 0x7f7edc15cb00 2024-04-24T23:06:18.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.277+0000 7f7ee34da640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f7edc0b4800 con 0x7f7edc0ce430 2024-04-24T23:06:18.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.277+0000 7f7ee34da640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f7edc0b4ae0 con 0x7f7edc15b380 2024-04-24T23:06:18.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.277+0000 7f7ee0a75640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7edc15cb00 0x7f7edc06cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:18.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.277+0000 7f7ee0a75640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7edc15cb00 0x7f7edc06cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35022/0 (socket says 172.21.15.32:35022) 2024-04-24T23:06:18.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.277+0000 7f7ee0a75640 1 -- 172.21.15.32:0/2070854579 learned_addr learned my addr 172.21.15.32:0/2070854579 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:18.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.277+0000 7f7ee1a77640 1 --2- 172.21.15.32:0/2070854579 >> v2:172.21.15.121:3300/0 conn(0x7f7edc0ce430 0x7f7edc0ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:18.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.277+0000 7f7ee1276640 1 --2- 172.21.15.32:0/2070854579 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7edc15b380 0x7f7edc06c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:18.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.277+0000 7f7ee0a75640 1 -- 172.21.15.32:0/2070854579 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7edc15b380 msgr2=0x7f7edc06c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:18.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.277+0000 7f7ee0a75640 1 --2- 172.21.15.32:0/2070854579 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7edc15b380 0x7f7edc06c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:18.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.277+0000 7f7ee0a75640 1 -- 172.21.15.32:0/2070854579 >> v2:172.21.15.121:3300/0 conn(0x7f7edc0ce430 msgr2=0x7f7edc0ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:18.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.277+0000 7f7ee0a75640 1 --2- 172.21.15.32:0/2070854579 >> v2:172.21.15.121:3300/0 conn(0x7f7edc0ce430 0x7f7edc0ce880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:18.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.277+0000 7f7ee0a75640 1 -- 172.21.15.32:0/2070854579 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7edc0b4dc0 con 0x7f7edc15cb00 2024-04-24T23:06:18.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.278+0000 7f7ee1a77640 1 --2- 172.21.15.32:0/2070854579 >> v2:172.21.15.121:3300/0 conn(0x7f7edc0ce430 0x7f7edc0ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:06:18.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.278+0000 7f7ee1276640 1 --2- 172.21.15.32:0/2070854579 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7edc15b380 0x7f7edc06c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:06:18.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.278+0000 7f7ee0a75640 1 --2- 172.21.15.32:0/2070854579 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7edc15cb00 0x7f7edc06cb30 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f7edc070130 tx=0x7f7ed002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0ed0815efab61a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:18.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.278+0000 7f7edbfff640 1 -- 172.21.15.32:0/2070854579 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0037070 con 0x7f7edc15cb00 2024-04-24T23:06:18.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.278+0000 7f7edbfff640 1 -- 172.21.15.32:0/2070854579 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ed0032440 con 0x7f7edc15cb00 2024-04-24T23:06:18.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.278+0000 7f7edbfff640 1 -- 172.21.15.32:0/2070854579 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0032810 con 0x7f7edc15cb00 2024-04-24T23:06:18.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.279+0000 7f7ee34da640 1 -- 172.21.15.32:0/2070854579 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7edc15cb00 msgr2=0x7f7edc06cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:18.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.279+0000 7f7ee34da640 1 --2- 172.21.15.32:0/2070854579 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7edc15cb00 0x7f7edc06cb30 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f7edc070130 tx=0x7f7ed002a140 comp rx=0 tx=0).stop 2024-04-24T23:06:18.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.279+0000 7f7ee34da640 1 -- 172.21.15.32:0/2070854579 shutdown_connections 2024-04-24T23:06:18.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.279+0000 7f7ee34da640 1 --2- 172.21.15.32:0/2070854579 >> v2:172.21.15.121:3300/0 conn(0x7f7edc0ce430 0x7f7edc0ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:18.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.279+0000 7f7ee34da640 1 --2- 172.21.15.32:0/2070854579 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7edc15cb00 0x7f7edc06cb30 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:18.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.279+0000 7f7ee34da640 1 --2- 172.21.15.32:0/2070854579 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7edc15b380 0x7f7edc06c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:18.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.279+0000 7f7ee34da640 1 -- 172.21.15.32:0/2070854579 >> 172.21.15.32:0/2070854579 conn(0x7f7edc06b110 msgr2=0x7f7edc06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:18.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.279+0000 7f7ee34da640 1 -- 172.21.15.32:0/2070854579 shutdown_connections 2024-04-24T23:06:18.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.279+0000 7f7ee34da640 1 -- 172.21.15.32:0/2070854579 wait complete. 2024-04-24T23:06:18.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.280+0000 7f7ee34da640 1 Processor -- start 2024-04-24T23:06:18.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.280+0000 7f7ee34da640 1 -- start start 2024-04-24T23:06:18.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.281+0000 7f7ee34da640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7edc0ce430 0x7f7edc1595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:18.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.281+0000 7f7ee34da640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7edc15b380 0x7f7edc141b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:18.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.281+0000 7f7ee34da640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7edc15cb00 0x7f7edc15f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:18.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.281+0000 7f7ee34da640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7edc0cedc0 con 0x7f7edc15cb00 2024-04-24T23:06:18.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.281+0000 7f7ee34da640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7edc15b760 con 0x7f7edc0ce430 2024-04-24T23:06:18.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.281+0000 7f7ee34da640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f7edc0cef40 con 0x7f7edc15b380 2024-04-24T23:06:18.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.281+0000 7f7ee1a77640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7edc15cb00 0x7f7edc15f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:18.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.281+0000 7f7ee1a77640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7edc15cb00 0x7f7edc15f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35036/0 (socket says 172.21.15.32:35036) 2024-04-24T23:06:18.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.281+0000 7f7ee1a77640 1 -- 172.21.15.32:0/3302548601 learned_addr learned my addr 172.21.15.32:0/3302548601 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:18.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.281+0000 7f7ee1276640 1 --2- 172.21.15.32:0/3302548601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7edc0ce430 0x7f7edc1595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:18.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.281+0000 7f7ee0a75640 1 --2- 172.21.15.32:0/3302548601 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7edc15b380 0x7f7edc141b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:18.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.281+0000 7f7ee1a77640 1 -- 172.21.15.32:0/3302548601 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7edc15b380 msgr2=0x7f7edc141b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:18.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.281+0000 7f7ee1a77640 1 --2- 172.21.15.32:0/3302548601 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7edc15b380 0x7f7edc141b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:18.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.281+0000 7f7ee1a77640 1 -- 172.21.15.32:0/3302548601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7edc0ce430 msgr2=0x7f7edc1595b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:18.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.281+0000 7f7ee1a77640 1 --2- 172.21.15.32:0/3302548601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7edc0ce430 0x7f7edc1595b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:18.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.281+0000 7f7ee1a77640 1 -- 172.21.15.32:0/3302548601 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7edc158b10 con 0x7f7edc15cb00 2024-04-24T23:06:18.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.281+0000 7f7ee1276640 1 --2- 172.21.15.32:0/3302548601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7edc0ce430 0x7f7edc1595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:06:18.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.281+0000 7f7ee0a75640 1 --2- 172.21.15.32:0/3302548601 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7edc15b380 0x7f7edc141b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:06:18.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.282+0000 7f7ee1a77640 1 --2- 172.21.15.32:0/3302548601 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7edc15cb00 0x7f7edc15f0b0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc007e70 tx=0x7f7ecc00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:18.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.282+0000 7f7ed9ffb640 1 -- 172.21.15.32:0/3302548601 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc01a070 con 0x7f7edc15cb00 2024-04-24T23:06:18.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.282+0000 7f7ed9ffb640 1 -- 172.21.15.32:0/3302548601 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ecc003bb0 con 0x7f7edc15cb00 2024-04-24T23:06:18.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.282+0000 7f7ee34da640 1 -- 172.21.15.32:0/3302548601 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7edc145220 con 0x7f7edc15cb00 2024-04-24T23:06:18.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.282+0000 7f7ed9ffb640 1 -- 172.21.15.32:0/3302548601 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc004030 con 0x7f7edc15cb00 2024-04-24T23:06:18.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.282+0000 7f7ee34da640 1 -- 172.21.15.32:0/3302548601 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7edc1420b0 con 0x7f7edc15cb00 2024-04-24T23:06:18.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.283+0000 7f7ee34da640 1 -- 172.21.15.32:0/3302548601 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7edc068430 con 0x7f7edc15cb00 2024-04-24T23:06:18.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.283+0000 7f7ed9ffb640 1 -- 172.21.15.32:0/3302548601 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7ecc01d070 con 0x7f7edc15cb00 2024-04-24T23:06:18.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.284+0000 7f7ed9ffb640 1 --2- 172.21.15.32:0/3302548601 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7ec806be60 0x7f7ec806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:18.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.284+0000 7f7ed9ffb640 1 -- 172.21.15.32:0/3302548601 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5796+0+0 (secure 0 0 0) 0x7f7ecc08b150 con 0x7f7edc15cb00 2024-04-24T23:06:18.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.284+0000 7f7ee1276640 1 --2- 172.21.15.32:0/3302548601 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7ec806be60 0x7f7ec806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:18.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.284+0000 7f7ee1276640 1 --2- 172.21.15.32:0/3302548601 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7ec806be60 0x7f7ec806e320 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4006fd0 tx=0x7f7ed4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:18.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.288+0000 7f7ed9ffb640 1 -- 172.21.15.32:0/3302548601 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7ecc010070 con 0x7f7edc15cb00 2024-04-24T23:06:18.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.507+0000 7f7ee34da640 1 -- 172.21.15.32:0/3302548601 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7edc0afd60 con 0x7f7edc15cb00 2024-04-24T23:06:18.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.508+0000 7f7ed9ffb640 1 -- 172.21.15.32:0/3302548601 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+5189 (secure 0 0 0) 0x7f7ecc059aa0 con 0x7f7edc15cb00 2024-04-24T23:06:18.509 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:06:18.510 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":58,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3361752496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3361752496},{"type":"v1","addr":"172.21.15.121:6837","nonce":3361752496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":6799,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3813453544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3813453544},{"type":"v1","addr":"172.21.15.121:6836","nonce":3813453544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7779,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/1352085651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1352085651},{"type":"v1","addr":"172.21.15.32:6833","nonce":1352085651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:05:33.791068+0000","modified":"2024-04-24T23:05:35.792946+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7794},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7794":{"gid":7794,"name":"a","rank":0,"incarnation":56,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/4206823204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":4206823204},{"type":"v1","addr":"172.21.15.32:6835","nonce":4206823204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7794,"qdb_cluster":[7794]},"id":8}]} 2024-04-24T23:06:18.510 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 58 2024-04-24T23:06:18.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.510+0000 7f7ee34da640 1 -- 172.21.15.32:0/3302548601 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7ec806be60 msgr2=0x7f7ec806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:18.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.510+0000 7f7ee34da640 1 --2- 172.21.15.32:0/3302548601 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7ec806be60 0x7f7ec806e320 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4006fd0 tx=0x7f7ed4008040 comp rx=0 tx=0).stop 2024-04-24T23:06:18.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.510+0000 7f7ee34da640 1 -- 172.21.15.32:0/3302548601 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7edc15cb00 msgr2=0x7f7edc15f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:18.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.510+0000 7f7ee34da640 1 --2- 172.21.15.32:0/3302548601 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7edc15cb00 0x7f7edc15f0b0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc007e70 tx=0x7f7ecc00d4a0 comp rx=0 tx=0).stop 2024-04-24T23:06:18.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.510+0000 7f7ee34da640 1 -- 172.21.15.32:0/3302548601 shutdown_connections 2024-04-24T23:06:18.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.510+0000 7f7ee34da640 1 --2- 172.21.15.32:0/3302548601 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7ec806be60 0x7f7ec806e320 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:18.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.510+0000 7f7ee34da640 1 --2- 172.21.15.32:0/3302548601 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7edc15cb00 0x7f7edc15f0b0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:18.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.510+0000 7f7ee34da640 1 --2- 172.21.15.32:0/3302548601 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7edc15b380 0x7f7edc141b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:18.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.510+0000 7f7ee34da640 1 --2- 172.21.15.32:0/3302548601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7edc0ce430 0x7f7edc1595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:18.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.510+0000 7f7ee34da640 1 -- 172.21.15.32:0/3302548601 >> 172.21.15.32:0/3302548601 conn(0x7f7edc06b110 msgr2=0x7f7edc0b5410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:18.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.510+0000 7f7ee34da640 1 -- 172.21.15.32:0/3302548601 shutdown_connections 2024-04-24T23:06:18.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.510+0000 7f7ee34da640 1 -- 172.21.15.32:0/3302548601 wait complete. 2024-04-24T23:06:18.522 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:06:18.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.621+0000 7effc41b0640 1 Processor -- start 2024-04-24T23:06:18.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.622+0000 7effc41b0640 1 -- start start 2024-04-24T23:06:18.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.622+0000 7effc41b0640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7effbc06b820 0x7effbc0d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:18.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.622+0000 7effc41b0640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7effbc0d4210 0x7effbc0b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:18.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.622+0000 7effc41b0640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effbc0d3170 con 0x7effbc15e9b0 2024-04-24T23:06:18.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.622+0000 7effc41b0640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7effbc0d3450 con 0x7effbc06b820 2024-04-24T23:06:18.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.622+0000 7effc41b0640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7effbc0d3730 con 0x7effbc0d4210 2024-04-24T23:06:18.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.623+0000 7effc174b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7effbc06b820 0x7effbc0d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:18.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.623+0000 7effc174b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7effbc06b820 0x7effbc0d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35040/0 (socket says 172.21.15.32:35040) 2024-04-24T23:06:18.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.623+0000 7effc174b640 1 -- 172.21.15.32:0/3814968468 learned_addr learned my addr 172.21.15.32:0/3814968468 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:18.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.623+0000 7effc274d640 1 --2- 172.21.15.32:0/3814968468 >> v2:172.21.15.121:3300/0 conn(0x7effbc0d4210 0x7effbc0b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:18.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.623+0000 7effc174b640 1 -- 172.21.15.32:0/3814968468 >> v2:172.21.15.121:3300/0 conn(0x7effbc0d4210 msgr2=0x7effbc0b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:18.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.623+0000 7effc174b640 1 --2- 172.21.15.32:0/3814968468 >> v2:172.21.15.121:3300/0 conn(0x7effbc0d4210 0x7effbc0b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:18.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.623+0000 7effc174b640 1 -- 172.21.15.32:0/3814968468 >> v1:172.21.15.121:6789/0 conn(0x7effbc15e9b0 legacy=0x7effbc06b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:18.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.623+0000 7effc174b640 1 -- 172.21.15.32:0/3814968468 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effbc0d3a10 con 0x7effbc06b820 2024-04-24T23:06:18.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.624+0000 7effc174b640 1 --2- 172.21.15.32:0/3814968468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effbc06b820 0x7effbc0d2b90 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7effbc1622f0 tx=0x7effac02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=308a9794f3a2cbd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:18.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.624+0000 7effc0f4a640 1 -- 172.21.15.32:0/3814968468 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effac037070 con 0x7effbc06b820 2024-04-24T23:06:18.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.624+0000 7effc0f4a640 1 -- 172.21.15.32:0/3814968468 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effac032440 con 0x7effbc06b820 2024-04-24T23:06:18.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.624+0000 7effc0f4a640 1 -- 172.21.15.32:0/3814968468 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effac032810 con 0x7effbc06b820 2024-04-24T23:06:18.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.624+0000 7effc41b0640 1 -- 172.21.15.32:0/3814968468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effbc06b820 msgr2=0x7effbc0d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:18.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.624+0000 7effc41b0640 1 --2- 172.21.15.32:0/3814968468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effbc06b820 0x7effbc0d2b90 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7effbc1622f0 tx=0x7effac02a1e0 comp rx=0 tx=0).stop 2024-04-24T23:06:18.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.624+0000 7effc41b0640 1 -- 172.21.15.32:0/3814968468 shutdown_connections 2024-04-24T23:06:18.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.624+0000 7effc41b0640 1 --2- 172.21.15.32:0/3814968468 >> v2:172.21.15.121:3300/0 conn(0x7effbc0d4210 0x7effbc0b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:18.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.624+0000 7effc41b0640 1 --2- 172.21.15.32:0/3814968468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effbc06b820 0x7effbc0d2b90 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:18.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.624+0000 7effc41b0640 1 -- 172.21.15.32:0/3814968468 >> 172.21.15.32:0/3814968468 conn(0x7effbc0d11d0 msgr2=0x7effbc0d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:18.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.625+0000 7effc41b0640 1 -- 172.21.15.32:0/3814968468 shutdown_connections 2024-04-24T23:06:18.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.625+0000 7effc41b0640 1 -- 172.21.15.32:0/3814968468 wait complete. 2024-04-24T23:06:18.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.626+0000 7effc41b0640 1 Processor -- start 2024-04-24T23:06:18.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.626+0000 7effc41b0640 1 -- start start 2024-04-24T23:06:18.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.626+0000 7effc41b0640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7effbc06b820 0x7effbc14f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:18.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.626+0000 7effc41b0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effbc0d4210 0x7effbc160dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:18.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.626+0000 7effc41b0640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7effbc15e9b0 0x7effbc161310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:18.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.627+0000 7effc41b0640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7effbc0ba0e0 con 0x7effbc0d4210 2024-04-24T23:06:18.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.627+0000 7effc41b0640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7effbc0d2400 con 0x7effbc15e9b0 2024-04-24T23:06:18.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.627+0000 7effc41b0640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7effbc0ba260 con 0x7effbc06b820 2024-04-24T23:06:18.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.627+0000 7effc174b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effbc0d4210 0x7effbc160dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:18.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.627+0000 7effc174b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effbc0d4210 0x7effbc160dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35056/0 (socket says 172.21.15.32:35056) 2024-04-24T23:06:18.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.627+0000 7effc174b640 1 -- 172.21.15.32:0/868895557 learned_addr learned my addr 172.21.15.32:0/868895557 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:18.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.627+0000 7effc274d640 1 --2- 172.21.15.32:0/868895557 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7effbc15e9b0 0x7effbc161310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:18.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.627+0000 7effc1f4c640 1 --2- 172.21.15.32:0/868895557 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7effbc06b820 0x7effbc14f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:18.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.627+0000 7effc174b640 1 -- 172.21.15.32:0/868895557 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7effbc06b820 msgr2=0x7effbc14f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:18.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.627+0000 7effc174b640 1 --2- 172.21.15.32:0/868895557 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7effbc06b820 0x7effbc14f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:18.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.627+0000 7effc174b640 1 -- 172.21.15.32:0/868895557 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7effbc15e9b0 msgr2=0x7effbc161310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:18.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.627+0000 7effc174b640 1 --2- 172.21.15.32:0/868895557 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7effbc15e9b0 0x7effbc161310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:18.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.627+0000 7effc174b640 1 -- 172.21.15.32:0/868895557 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effbc160580 con 0x7effbc0d4210 2024-04-24T23:06:18.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.627+0000 7effc274d640 1 --2- 172.21.15.32:0/868895557 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7effbc15e9b0 0x7effbc161310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:06:18.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.627+0000 7effc174b640 1 --2- 172.21.15.32:0/868895557 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effbc0d4210 0x7effbc160dd0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7effac000c00 tx=0x7effac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:18.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.627+0000 7effb27fc640 1 -- 172.21.15.32:0/868895557 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effac037040 con 0x7effbc0d4210 2024-04-24T23:06:18.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.627+0000 7effb27fc640 1 -- 172.21.15.32:0/868895557 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effac042070 con 0x7effbc0d4210 2024-04-24T23:06:18.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.627+0000 7effc41b0640 1 -- 172.21.15.32:0/868895557 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effbc0cc5c0 con 0x7effbc0d4210 2024-04-24T23:06:18.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.627+0000 7effb27fc640 1 -- 172.21.15.32:0/868895557 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effac030880 con 0x7effbc0d4210 2024-04-24T23:06:18.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.627+0000 7effc41b0640 1 -- 172.21.15.32:0/868895557 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effbc0cc860 con 0x7effbc0d4210 2024-04-24T23:06:18.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.628+0000 7effc41b0640 1 -- 172.21.15.32:0/868895557 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effbc068430 con 0x7effbc0d4210 2024-04-24T23:06:18.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.629+0000 7effb27fc640 1 -- 172.21.15.32:0/868895557 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7effac030a20 con 0x7effbc0d4210 2024-04-24T23:06:18.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.629+0000 7effb27fc640 1 --2- 172.21.15.32:0/868895557 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7effa006be30 0x7effa006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:18.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.629+0000 7effb27fc640 1 -- 172.21.15.32:0/868895557 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5796+0+0 (secure 0 0 0) 0x7effac0ab760 con 0x7effbc0d4210 2024-04-24T23:06:18.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.629+0000 7effc1f4c640 1 --2- 172.21.15.32:0/868895557 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7effa006be30 0x7effa006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:18.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.630+0000 7effc1f4c640 1 --2- 172.21.15.32:0/868895557 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7effa006be30 0x7effa006e2f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7effb8007fd0 tx=0x7effb8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:18.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.633+0000 7effb27fc640 1 -- 172.21.15.32:0/868895557 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7effac07a0b0 con 0x7effbc0d4210 2024-04-24T23:06:18.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.820+0000 7effc41b0640 1 -- 172.21.15.32:0/868895557 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7effbc0d19c0 con 0x7effbc0d4210 2024-04-24T23:06:18.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.821+0000 7effb27fc640 1 -- 172.21.15.32:0/868895557 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) ==== 74+0+16836 (secure 0 0 0) 0x7effac0799b0 con 0x7effbc0d4210 2024-04-24T23:06:18.822 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:06:18.823 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":145,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:06:06.811191+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:05:32.064234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-24T23:05:32.760777+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.17","mappings":[{"from":4,"to":3}]},{"pgid":"17.8","mappings":[{"from":6,"to":5}]},{"pgid":"17.3f","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:06:18.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.823+0000 7effc41b0640 1 -- 172.21.15.32:0/868895557 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7effa006be30 msgr2=0x7effa006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:18.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.823+0000 7effc41b0640 1 --2- 172.21.15.32:0/868895557 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7effa006be30 0x7effa006e2f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7effb8007fd0 tx=0x7effb8009040 comp rx=0 tx=0).stop 2024-04-24T23:06:18.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.823+0000 7effc41b0640 1 -- 172.21.15.32:0/868895557 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effbc0d4210 msgr2=0x7effbc160dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:18.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.823+0000 7effc41b0640 1 --2- 172.21.15.32:0/868895557 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effbc0d4210 0x7effbc160dd0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7effac000c00 tx=0x7effac0046c0 comp rx=0 tx=0).stop 2024-04-24T23:06:18.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.823+0000 7effc41b0640 1 -- 172.21.15.32:0/868895557 shutdown_connections 2024-04-24T23:06:18.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.823+0000 7effc41b0640 1 --2- 172.21.15.32:0/868895557 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7effa006be30 0x7effa006e2f0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:18.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.823+0000 7effc41b0640 1 --2- 172.21.15.32:0/868895557 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effbc0d4210 0x7effbc160dd0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:18.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.823+0000 7effc41b0640 1 --2- 172.21.15.32:0/868895557 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7effbc15e9b0 0x7effbc161310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:18.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.823+0000 7effc41b0640 1 --2- 172.21.15.32:0/868895557 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7effbc06b820 0x7effbc14f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:18.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.823+0000 7effc41b0640 1 -- 172.21.15.32:0/868895557 >> 172.21.15.32:0/868895557 conn(0x7effbc0d11d0 msgr2=0x7effbc15aee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:18.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.823+0000 7effc41b0640 1 -- 172.21.15.32:0/868895557 shutdown_connections 2024-04-24T23:06:18.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.823+0000 7effc41b0640 1 -- 172.21.15.32:0/868895557 wait complete. 2024-04-24T23:06:18.835 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-24T23:06:18.835 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:06:18.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.934+0000 7f53af929640 1 Processor -- start 2024-04-24T23:06:18.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.935+0000 7f53af929640 1 -- start start 2024-04-24T23:06:18.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.935+0000 7f53af929640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f53a806b820 0x7f53a80ce630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:18.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.935+0000 7f53af929640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53a80afdd0 con 0x7f53a80cfb70 2024-04-24T23:06:18.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.936+0000 7f53af929640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53a80af010 con 0x7f53a81476d0 2024-04-24T23:06:18.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.936+0000 7f53af929640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f53a80aedc0 con 0x7f53a806b820 2024-04-24T23:06:18.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.936+0000 7f53acec4640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f53a806b820 0x7f53a80ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:18.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.936+0000 7f53adec6640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f53a80cfb70 0x7f53a806d5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41236/0 (socket says 172.21.15.32:41236) 2024-04-24T23:06:18.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.936+0000 7f53adec6640 1 -- 172.21.15.32:0/4053724378 learned_addr learned my addr 172.21.15.32:0/4053724378 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:06:18.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.936+0000 7f53acec4640 1 --2- 172.21.15.32:0/4053724378 >> v2:172.21.15.32:3300/0 conn(0x7f53a806b820 0x7f53a80ce630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35072/0 (socket says 172.21.15.32:35072) 2024-04-24T23:06:18.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.936+0000 7f53acec4640 1 -- 172.21.15.32:0/4053724378 >> v1:172.21.15.121:6789/0 conn(0x7f53a81476d0 legacy=0x7f53a806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:18.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.936+0000 7f53acec4640 1 -- 172.21.15.32:0/4053724378 >> v1:172.21.15.32:6789/0 conn(0x7f53a80cfb70 legacy=0x7f53a806d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:18.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.936+0000 7f53acec4640 1 -- 172.21.15.32:0/4053724378 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53a80af2f0 con 0x7f53a806b820 2024-04-24T23:06:18.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.936+0000 7f539ffff640 1 -- 172.21.15.32:0/4053724378 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1989985142 0 0) 0x7f53a80afdd0 con 0x7f53a80cfb70 2024-04-24T23:06:18.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.937+0000 7f53acec4640 1 --2- 172.21.15.32:0/4053724378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53a806b820 0x7f53a80ce630 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f5398009790 tx=0x7f539802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41302b947e3adfe4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:18.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.937+0000 7f539ffff640 1 -- 172.21.15.32:0/4053724378 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5398037070 con 0x7f53a806b820 2024-04-24T23:06:18.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.937+0000 7f539ffff640 1 -- 172.21.15.32:0/4053724378 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5398032440 con 0x7f53a806b820 2024-04-24T23:06:18.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.937+0000 7f539ffff640 1 -- 172.21.15.32:0/4053724378 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5398032810 con 0x7f53a806b820 2024-04-24T23:06:18.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.937+0000 7f53af929640 1 -- 172.21.15.32:0/4053724378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53a806b820 msgr2=0x7f53a80ce630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:18.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.937+0000 7f53af929640 1 --2- 172.21.15.32:0/4053724378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53a806b820 0x7f53a80ce630 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f5398009790 tx=0x7f539802a180 comp rx=0 tx=0).stop 2024-04-24T23:06:18.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.937+0000 7f53af929640 1 -- 172.21.15.32:0/4053724378 shutdown_connections 2024-04-24T23:06:18.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.937+0000 7f53af929640 1 --2- 172.21.15.32:0/4053724378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53a806b820 0x7f53a80ce630 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:18.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.937+0000 7f53af929640 1 -- 172.21.15.32:0/4053724378 >> 172.21.15.32:0/4053724378 conn(0x7f53a80d0730 msgr2=0x7f53a80d0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:18.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.937+0000 7f53af929640 1 -- 172.21.15.32:0/4053724378 shutdown_connections 2024-04-24T23:06:18.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.938+0000 7f53af929640 1 -- 172.21.15.32:0/4053724378 wait complete. 2024-04-24T23:06:18.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.939+0000 7f53af929640 1 Processor -- start 2024-04-24T23:06:18.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.939+0000 7f53af929640 1 -- start start 2024-04-24T23:06:18.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.939+0000 7f53af929640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53a806b820 0x7f53a81f4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:18.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.939+0000 7f53af929640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53a80cfb70 0x7f53a80c6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:18.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.939+0000 7f53af929640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53a81476d0 0x7f53a81f75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:18.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.939+0000 7f53af929640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f53a8147ab0 con 0x7f53a81476d0 2024-04-24T23:06:18.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.939+0000 7f53af929640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f53a80af940 con 0x7f53a80cfb70 2024-04-24T23:06:18.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.939+0000 7f53af929640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f53a8148470 con 0x7f53a806b820 2024-04-24T23:06:18.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.939+0000 7f53adec6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53a81476d0 0x7f53a81f75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:18.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.939+0000 7f53adec6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53a81476d0 0x7f53a81f75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35080/0 (socket says 172.21.15.32:35080) 2024-04-24T23:06:18.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.939+0000 7f53adec6640 1 -- 172.21.15.32:0/1516401291 learned_addr learned my addr 172.21.15.32:0/1516401291 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:18.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.940+0000 7f53ad6c5640 1 --2- 172.21.15.32:0/1516401291 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53a806b820 0x7f53a81f4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:18.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.940+0000 7f53acec4640 1 --2- 172.21.15.32:0/1516401291 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53a80cfb70 0x7f53a80c6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:18.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.940+0000 7f53adec6640 1 -- 172.21.15.32:0/1516401291 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53a806b820 msgr2=0x7f53a81f4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:18.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.940+0000 7f53adec6640 1 --2- 172.21.15.32:0/1516401291 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53a806b820 0x7f53a81f4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:18.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.940+0000 7f53adec6640 1 -- 172.21.15.32:0/1516401291 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53a80cfb70 msgr2=0x7f53a80c6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:18.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.940+0000 7f53adec6640 1 --2- 172.21.15.32:0/1516401291 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53a80cfb70 0x7f53a80c6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:18.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.940+0000 7f53adec6640 1 -- 172.21.15.32:0/1516401291 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53a80b1f80 con 0x7f53a81476d0 2024-04-24T23:06:18.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.940+0000 7f53ad6c5640 1 --2- 172.21.15.32:0/1516401291 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53a806b820 0x7f53a81f4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:06:18.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.940+0000 7f53acec4640 1 --2- 172.21.15.32:0/1516401291 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53a80cfb70 0x7f53a80c6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:06:18.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.940+0000 7f53adec6640 1 --2- 172.21.15.32:0/1516401291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53a81476d0 0x7f53a81f75d0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f53a000c7d0 tx=0x7f53a000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:18.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.940+0000 7f539dffb640 1 -- 172.21.15.32:0/1516401291 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a0018070 con 0x7f53a81476d0 2024-04-24T23:06:18.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.940+0000 7f539dffb640 1 -- 172.21.15.32:0/1516401291 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53a000a640 con 0x7f53a81476d0 2024-04-24T23:06:18.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.940+0000 7f53af929640 1 -- 172.21.15.32:0/1516401291 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53a80b2220 con 0x7f53a81476d0 2024-04-24T23:06:18.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.940+0000 7f539dffb640 1 -- 172.21.15.32:0/1516401291 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a000aa30 con 0x7f53a81476d0 2024-04-24T23:06:18.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.940+0000 7f53af929640 1 -- 172.21.15.32:0/1516401291 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53a80b27e0 con 0x7f53a81476d0 2024-04-24T23:06:18.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.941+0000 7f53af929640 1 -- 172.21.15.32:0/1516401291 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53a8068430 con 0x7f53a81476d0 2024-04-24T23:06:18.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.942+0000 7f539dffb640 1 -- 172.21.15.32:0/1516401291 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f53a001b070 con 0x7f53a81476d0 2024-04-24T23:06:18.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.942+0000 7f539dffb640 1 --2- 172.21.15.32:0/1516401291 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f538c06be30 0x7f538c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:18.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.942+0000 7f539dffb640 1 -- 172.21.15.32:0/1516401291 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5796+0+0 (secure 0 0 0) 0x7f53a008a530 con 0x7f53a81476d0 2024-04-24T23:06:18.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.943+0000 7f53ad6c5640 1 --2- 172.21.15.32:0/1516401291 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f538c06be30 0x7f538c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:18.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.943+0000 7f53ad6c5640 1 --2- 172.21.15.32:0/1516401291 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f538c06be30 0x7f538c06e2f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f53a4007fd0 tx=0x7f53a4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:18.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:18.946+0000 7f539dffb640 1 -- 172.21.15.32:0/1516401291 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f53a0058de0 con 0x7f53a81476d0 2024-04-24T23:06:19.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.167+0000 7f53af929640 1 -- 172.21.15.32:0/1516401291 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f53a80ce0e0 con 0x7f53a81476d0 2024-04-24T23:06:19.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.168+0000 7f539dffb640 1 -- 172.21.15.32:0/1516401291 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+5189 (secure 0 0 0) 0x7f53a00586e0 con 0x7f53a81476d0 2024-04-24T23:06:19.169 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:06:19.170 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":58,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3361752496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3361752496},{"type":"v1","addr":"172.21.15.121:6837","nonce":3361752496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":6799,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3813453544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3813453544},{"type":"v1","addr":"172.21.15.121:6836","nonce":3813453544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7779,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/1352085651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1352085651},{"type":"v1","addr":"172.21.15.32:6833","nonce":1352085651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:05:33.791068+0000","modified":"2024-04-24T23:05:35.792946+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7794},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7794":{"gid":7794,"name":"a","rank":0,"incarnation":56,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/4206823204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":4206823204},{"type":"v1","addr":"172.21.15.32:6835","nonce":4206823204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7794,"qdb_cluster":[7794]},"id":8}]} 2024-04-24T23:06:19.170 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 58 2024-04-24T23:06:19.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.170+0000 7f53af929640 1 -- 172.21.15.32:0/1516401291 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f538c06be30 msgr2=0x7f538c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:19.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.170+0000 7f53af929640 1 --2- 172.21.15.32:0/1516401291 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f538c06be30 0x7f538c06e2f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f53a4007fd0 tx=0x7f53a4009040 comp rx=0 tx=0).stop 2024-04-24T23:06:19.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.170+0000 7f53af929640 1 -- 172.21.15.32:0/1516401291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53a81476d0 msgr2=0x7f53a81f75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:19.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.170+0000 7f53af929640 1 --2- 172.21.15.32:0/1516401291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53a81476d0 0x7f53a81f75d0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f53a000c7d0 tx=0x7f53a000cc90 comp rx=0 tx=0).stop 2024-04-24T23:06:19.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.170+0000 7f53af929640 1 -- 172.21.15.32:0/1516401291 shutdown_connections 2024-04-24T23:06:19.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.170+0000 7f53af929640 1 --2- 172.21.15.32:0/1516401291 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f538c06be30 0x7f538c06e2f0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:19.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.170+0000 7f53af929640 1 --2- 172.21.15.32:0/1516401291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53a81476d0 0x7f53a81f75d0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:19.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.170+0000 7f53af929640 1 --2- 172.21.15.32:0/1516401291 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53a80cfb70 0x7f53a80c6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:19.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.170+0000 7f53af929640 1 --2- 172.21.15.32:0/1516401291 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53a806b820 0x7f53a81f4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:19.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.170+0000 7f53af929640 1 -- 172.21.15.32:0/1516401291 >> 172.21.15.32:0/1516401291 conn(0x7f53a80d0730 msgr2=0x7f53a8145af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:19.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.170+0000 7f53af929640 1 -- 172.21.15.32:0/1516401291 shutdown_connections 2024-04-24T23:06:19.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.170+0000 7f53af929640 1 -- 172.21.15.32:0/1516401291 wait complete. 2024-04-24T23:06:19.182 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:06:19.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.282+0000 7f8409742640 1 Processor -- start 2024-04-24T23:06:19.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.282+0000 7f8409742640 1 -- start start 2024-04-24T23:06:19.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.282+0000 7f8409742640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f840415cc40 0x7f840415d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:19.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.283+0000 7f8409742640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f84040d05b0 0x7f840415d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:19.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.283+0000 7f8409742640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84040cec30 0x7f84040cf0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:19.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.283+0000 7f8409742640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f840406cca0 con 0x7f84040d05b0 2024-04-24T23:06:19.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.283+0000 7f8409742640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f84040ce5f0 con 0x7f840415cc40 2024-04-24T23:06:19.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.283+0000 7f8409742640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f840415daa0 con 0x7f84040cec30 2024-04-24T23:06:19.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.283+0000 7f84027fc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f84040d05b0 0x7f840415d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:19.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.283+0000 7f84027fc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f84040d05b0 0x7f840415d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35088/0 (socket says 172.21.15.32:35088) 2024-04-24T23:06:19.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.283+0000 7f84027fc640 1 -- 172.21.15.32:0/2632695286 learned_addr learned my addr 172.21.15.32:0/2632695286 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:19.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.283+0000 7f84037fe640 1 --2- 172.21.15.32:0/2632695286 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84040cec30 0x7f84040cf0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:19.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.283+0000 7f8402ffd640 1 --2- 172.21.15.32:0/2632695286 >> v2:172.21.15.121:3300/0 conn(0x7f840415cc40 0x7f840415d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:19.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.283+0000 7f84027fc640 1 -- 172.21.15.32:0/2632695286 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84040cec30 msgr2=0x7f84040cf0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:19.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.283+0000 7f84027fc640 1 --2- 172.21.15.32:0/2632695286 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84040cec30 0x7f84040cf0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:19.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.283+0000 7f84027fc640 1 -- 172.21.15.32:0/2632695286 >> v2:172.21.15.121:3300/0 conn(0x7f840415cc40 msgr2=0x7f840415d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:19.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.283+0000 7f84027fc640 1 --2- 172.21.15.32:0/2632695286 >> v2:172.21.15.121:3300/0 conn(0x7f840415cc40 0x7f840415d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:19.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.283+0000 7f84027fc640 1 -- 172.21.15.32:0/2632695286 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84040b4ee0 con 0x7f84040d05b0 2024-04-24T23:06:19.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.283+0000 7f8402ffd640 1 --2- 172.21.15.32:0/2632695286 >> v2:172.21.15.121:3300/0 conn(0x7f840415cc40 0x7f840415d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:06:19.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.284+0000 7f84027fc640 1 --2- 172.21.15.32:0/2632695286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84040d05b0 0x7f840415d560 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f840404f570 tx=0x7f83f802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1039b1a0b692cc30 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:19.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.284+0000 7f8401ffb640 1 -- 172.21.15.32:0/2632695286 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f8037070 con 0x7f84040d05b0 2024-04-24T23:06:19.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.284+0000 7f8401ffb640 1 -- 172.21.15.32:0/2632695286 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83f8032440 con 0x7f84040d05b0 2024-04-24T23:06:19.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.284+0000 7f8401ffb640 1 -- 172.21.15.32:0/2632695286 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f8032810 con 0x7f84040d05b0 2024-04-24T23:06:19.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.284+0000 7f8409742640 1 -- 172.21.15.32:0/2632695286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84040d05b0 msgr2=0x7f840415d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:19.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.284+0000 7f8409742640 1 --2- 172.21.15.32:0/2632695286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84040d05b0 0x7f840415d560 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f840404f570 tx=0x7f83f802a1e0 comp rx=0 tx=0).stop 2024-04-24T23:06:19.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.284+0000 7f8409742640 1 -- 172.21.15.32:0/2632695286 shutdown_connections 2024-04-24T23:06:19.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.284+0000 7f8409742640 1 --2- 172.21.15.32:0/2632695286 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84040cec30 0x7f84040cf0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:19.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.284+0000 7f8409742640 1 --2- 172.21.15.32:0/2632695286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84040d05b0 0x7f840415d560 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:19.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.284+0000 7f8409742640 1 --2- 172.21.15.32:0/2632695286 >> v2:172.21.15.121:3300/0 conn(0x7f840415cc40 0x7f840415d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:19.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.284+0000 7f8409742640 1 -- 172.21.15.32:0/2632695286 >> 172.21.15.32:0/2632695286 conn(0x7f84040d0ae0 msgr2=0x7f84040d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:19.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.285+0000 7f8409742640 1 -- 172.21.15.32:0/2632695286 shutdown_connections 2024-04-24T23:06:19.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.285+0000 7f8409742640 1 -- 172.21.15.32:0/2632695286 wait complete. 2024-04-24T23:06:19.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.286+0000 7f8409742640 1 Processor -- start 2024-04-24T23:06:19.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.286+0000 7f8409742640 1 -- start start 2024-04-24T23:06:19.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.287+0000 7f8409742640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84040cec30 0x7f84041f7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:19.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.287+0000 7f8409742640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84040d05b0 0x7f84041e76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:19.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.287+0000 7f8409742640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f840415cc40 0x7f84041e7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:19.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.287+0000 7f8409742640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f84040b5780 con 0x7f84040d05b0 2024-04-24T23:06:19.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.287+0000 7f8409742640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f84040b54a0 con 0x7f840415cc40 2024-04-24T23:06:19.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.287+0000 7f8409742640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f84040b5a60 con 0x7f84040cec30 2024-04-24T23:06:19.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.287+0000 7f84027fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84040d05b0 0x7f84041e76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:19.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.287+0000 7f84027fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84040d05b0 0x7f84041e76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35102/0 (socket says 172.21.15.32:35102) 2024-04-24T23:06:19.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.287+0000 7f84027fc640 1 -- 172.21.15.32:0/2809261894 learned_addr learned my addr 172.21.15.32:0/2809261894 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:19.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.287+0000 7f84037fe640 1 --2- 172.21.15.32:0/2809261894 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f840415cc40 0x7f84041e7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:19.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.287+0000 7f8402ffd640 1 --2- 172.21.15.32:0/2809261894 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84040cec30 0x7f84041f7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:19.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.287+0000 7f84027fc640 1 -- 172.21.15.32:0/2809261894 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84040cec30 msgr2=0x7f84041f7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:19.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.287+0000 7f84027fc640 1 --2- 172.21.15.32:0/2809261894 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84040cec30 0x7f84041f7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:19.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.287+0000 7f84027fc640 1 -- 172.21.15.32:0/2809261894 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f840415cc40 msgr2=0x7f84041e7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:19.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.287+0000 7f84027fc640 1 --2- 172.21.15.32:0/2809261894 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f840415cc40 0x7f84041e7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:19.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.287+0000 7f84027fc640 1 -- 172.21.15.32:0/2809261894 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84041f8870 con 0x7f84040d05b0 2024-04-24T23:06:19.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.287+0000 7f84027fc640 1 --2- 172.21.15.32:0/2809261894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84040d05b0 0x7f84041e76f0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f83f802a220 tx=0x7f83f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:19.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.287+0000 7f83eb7fe640 1 -- 172.21.15.32:0/2809261894 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f8041020 con 0x7f84040d05b0 2024-04-24T23:06:19.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.287+0000 7f83eb7fe640 1 -- 172.21.15.32:0/2809261894 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83f80306c0 con 0x7f84040d05b0 2024-04-24T23:06:19.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.287+0000 7f8409742640 1 -- 172.21.15.32:0/2809261894 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84041f8b50 con 0x7f84040d05b0 2024-04-24T23:06:19.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.287+0000 7f83eb7fe640 1 -- 172.21.15.32:0/2809261894 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f8037040 con 0x7f84040d05b0 2024-04-24T23:06:19.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.288+0000 7f8409742640 1 -- 172.21.15.32:0/2809261894 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84041e71d0 con 0x7f84040d05b0 2024-04-24T23:06:19.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.289+0000 7f8409742640 1 -- 172.21.15.32:0/2809261894 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8404068430 con 0x7f84040d05b0 2024-04-24T23:06:19.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.289+0000 7f83eb7fe640 1 -- 172.21.15.32:0/2809261894 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f83f8044020 con 0x7f84040d05b0 2024-04-24T23:06:19.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.289+0000 7f83eb7fe640 1 --2- 172.21.15.32:0/2809261894 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f83f006be30 0x7f83f006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:19.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.289+0000 7f83eb7fe640 1 -- 172.21.15.32:0/2809261894 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5796+0+0 (secure 0 0 0) 0x7f83f80ab720 con 0x7f84040d05b0 2024-04-24T23:06:19.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.289+0000 7f8402ffd640 1 --2- 172.21.15.32:0/2809261894 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f83f006be30 0x7f83f006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:19.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.290+0000 7f8402ffd640 1 --2- 172.21.15.32:0/2809261894 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f83f006be30 0x7f83f006e2f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f83f4006fd0 tx=0x7f83f4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:19.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.294+0000 7f83eb7fe640 1 -- 172.21.15.32:0/2809261894 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f83f804b030 con 0x7f84040d05b0 2024-04-24T23:06:19.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.516+0000 7f8409742640 1 -- 172.21.15.32:0/2809261894 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f84040d1600 con 0x7f84040d05b0 2024-04-24T23:06:19.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.517+0000 7f83eb7fe640 1 -- 172.21.15.32:0/2809261894 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+5189 (secure 0 0 0) 0x7f83f8079900 con 0x7f84040d05b0 2024-04-24T23:06:19.518 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:06:19.519 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":58,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3361752496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3361752496},{"type":"v1","addr":"172.21.15.121:6837","nonce":3361752496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":6799,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3813453544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3813453544},{"type":"v1","addr":"172.21.15.121:6836","nonce":3813453544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7779,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/1352085651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1352085651},{"type":"v1","addr":"172.21.15.32:6833","nonce":1352085651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:05:33.791068+0000","modified":"2024-04-24T23:05:35.792946+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7794},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7794":{"gid":7794,"name":"a","rank":0,"incarnation":56,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/4206823204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":4206823204},{"type":"v1","addr":"172.21.15.32:6835","nonce":4206823204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7794,"qdb_cluster":[7794]},"id":8}]} 2024-04-24T23:06:19.519 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 58 2024-04-24T23:06:19.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.519+0000 7f8409742640 1 -- 172.21.15.32:0/2809261894 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f83f006be30 msgr2=0x7f83f006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:19.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.519+0000 7f8409742640 1 --2- 172.21.15.32:0/2809261894 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f83f006be30 0x7f83f006e2f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f83f4006fd0 tx=0x7f83f4008040 comp rx=0 tx=0).stop 2024-04-24T23:06:19.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.519+0000 7f8409742640 1 -- 172.21.15.32:0/2809261894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84040d05b0 msgr2=0x7f84041e76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:19.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.519+0000 7f8409742640 1 --2- 172.21.15.32:0/2809261894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84040d05b0 0x7f84041e76f0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f83f802a220 tx=0x7f83f80046c0 comp rx=0 tx=0).stop 2024-04-24T23:06:19.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.519+0000 7f8409742640 1 -- 172.21.15.32:0/2809261894 shutdown_connections 2024-04-24T23:06:19.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.519+0000 7f8409742640 1 --2- 172.21.15.32:0/2809261894 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f83f006be30 0x7f83f006e2f0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:19.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.519+0000 7f8409742640 1 --2- 172.21.15.32:0/2809261894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84040d05b0 0x7f84041e76f0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:19.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.519+0000 7f8409742640 1 --2- 172.21.15.32:0/2809261894 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f840415cc40 0x7f84041e7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:19.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.519+0000 7f8409742640 1 --2- 172.21.15.32:0/2809261894 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84040cec30 0x7f84041f7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:19.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.519+0000 7f8409742640 1 -- 172.21.15.32:0/2809261894 >> 172.21.15.32:0/2809261894 conn(0x7f84040d0ae0 msgr2=0x7f840415c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:19.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.519+0000 7f8409742640 1 -- 172.21.15.32:0/2809261894 shutdown_connections 2024-04-24T23:06:19.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.519+0000 7f8409742640 1 -- 172.21.15.32:0/2809261894 wait complete. 2024-04-24T23:06:19.531 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:06:19.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.630+0000 7f48bedd5640 1 Processor -- start 2024-04-24T23:06:19.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.630+0000 7f48bedd5640 1 -- start start 2024-04-24T23:06:19.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.631+0000 7f48bedd5640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f48b8162320 0x7f48b80ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:19.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.631+0000 7f48bedd5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f48b8162e10 0x7f48b815ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:19.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.631+0000 7f48bedd5640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48b80cd270 con 0x7f48b80b5af0 2024-04-24T23:06:19.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.631+0000 7f48bedd5640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f48b80b5660 con 0x7f48b8162320 2024-04-24T23:06:19.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.631+0000 7f48bedd5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f48b8071610 con 0x7f48b8162e10 2024-04-24T23:06:19.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.632+0000 7f48bcb71640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f48b8162320 0x7f48b80ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:19.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.632+0000 7f48bd372640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f48b8162e10 0x7f48b815ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:19.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.632+0000 7f48b7fff640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f48b80b5af0 0x7f48b8162700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:57722/0 (socket says 172.21.15.32:57722) 2024-04-24T23:06:19.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.632+0000 7f48b7fff640 1 -- 172.21.15.32:0/3926805581 learned_addr learned my addr 172.21.15.32:0/3926805581 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:06:19.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.632+0000 7f48bcb71640 1 -- 172.21.15.32:0/3926805581 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f48b8162e10 msgr2=0x7f48b815ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:19.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.632+0000 7f48bcb71640 1 --2- 172.21.15.32:0/3926805581 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f48b8162e10 0x7f48b815ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:19.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.632+0000 7f48bcb71640 1 -- 172.21.15.32:0/3926805581 >> v1:172.21.15.121:6789/0 conn(0x7f48b80b5af0 legacy=0x7f48b8162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:19.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.632+0000 7f48b77fe640 1 -- 172.21.15.32:0/3926805581 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4042261264 0 0) 0x7f48b80cd270 con 0x7f48b80b5af0 2024-04-24T23:06:19.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.632+0000 7f48bcb71640 1 -- 172.21.15.32:0/3926805581 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48b80718f0 con 0x7f48b8162320 2024-04-24T23:06:19.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.633+0000 7f48bcb71640 1 --2- 172.21.15.32:0/3926805581 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48b8162320 0x7f48b80ccc90 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f48b0009d90 tx=0x7f48b002a530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10b253d39471c86a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:19.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.633+0000 7f48b77fe640 1 -- 172.21.15.32:0/3926805581 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b0007740 con 0x7f48b8162320 2024-04-24T23:06:19.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.633+0000 7f48b77fe640 1 -- 172.21.15.32:0/3926805581 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48b00078e0 con 0x7f48b8162320 2024-04-24T23:06:19.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.633+0000 7f48b77fe640 1 -- 172.21.15.32:0/3926805581 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b0007cb0 con 0x7f48b8162320 2024-04-24T23:06:19.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.633+0000 7f48bedd5640 1 -- 172.21.15.32:0/3926805581 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48b8162320 msgr2=0x7f48b80ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:19.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.633+0000 7f48bedd5640 1 --2- 172.21.15.32:0/3926805581 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48b8162320 0x7f48b80ccc90 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f48b0009d90 tx=0x7f48b002a530 comp rx=0 tx=0).stop 2024-04-24T23:06:19.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.633+0000 7f48bedd5640 1 -- 172.21.15.32:0/3926805581 shutdown_connections 2024-04-24T23:06:19.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.633+0000 7f48bedd5640 1 --2- 172.21.15.32:0/3926805581 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f48b8162e10 0x7f48b815ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:19.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.633+0000 7f48bedd5640 1 --2- 172.21.15.32:0/3926805581 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48b8162320 0x7f48b80ccc90 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:19.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.633+0000 7f48bedd5640 1 -- 172.21.15.32:0/3926805581 >> 172.21.15.32:0/3926805581 conn(0x7f48b80d0ae0 msgr2=0x7f48b80d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:19.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.633+0000 7f48bedd5640 1 -- 172.21.15.32:0/3926805581 shutdown_connections 2024-04-24T23:06:19.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.633+0000 7f48bedd5640 1 -- 172.21.15.32:0/3926805581 wait complete. 2024-04-24T23:06:19.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.635+0000 7f48bedd5640 1 Processor -- start 2024-04-24T23:06:19.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.635+0000 7f48bedd5640 1 -- start start 2024-04-24T23:06:19.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.635+0000 7f48bedd5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f48b80b5af0 0x7f48b80cf170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:19.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.635+0000 7f48bedd5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48b8162320 0x7f48b80cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:19.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.635+0000 7f48bedd5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48b8162e10 0x7f48b80b1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:19.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.635+0000 7f48bedd5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f48b8071bd0 con 0x7f48b8162e10 2024-04-24T23:06:19.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.635+0000 7f48bedd5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f48b8067950 con 0x7f48b8162320 2024-04-24T23:06:19.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.635+0000 7f48bedd5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f48b8071d50 con 0x7f48b80b5af0 2024-04-24T23:06:19.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.636+0000 7f48bd372640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48b8162e10 0x7f48b80b1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:19.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.636+0000 7f48bd372640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48b8162e10 0x7f48b80b1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35116/0 (socket says 172.21.15.32:35116) 2024-04-24T23:06:19.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.636+0000 7f48bd372640 1 -- 172.21.15.32:0/4213241985 learned_addr learned my addr 172.21.15.32:0/4213241985 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:19.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.636+0000 7f48b7fff640 1 --2- 172.21.15.32:0/4213241985 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48b8162320 0x7f48b80cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:19.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.636+0000 7f48bcb71640 1 --2- 172.21.15.32:0/4213241985 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f48b80b5af0 0x7f48b80cf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:19.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.636+0000 7f48bd372640 1 -- 172.21.15.32:0/4213241985 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f48b80b5af0 msgr2=0x7f48b80cf170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:19.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.636+0000 7f48bd372640 1 --2- 172.21.15.32:0/4213241985 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f48b80b5af0 0x7f48b80cf170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:19.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.636+0000 7f48bd372640 1 -- 172.21.15.32:0/4213241985 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48b8162320 msgr2=0x7f48b80cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:19.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.636+0000 7f48bd372640 1 --2- 172.21.15.32:0/4213241985 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48b8162320 0x7f48b80cd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:19.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.636+0000 7f48bd372640 1 -- 172.21.15.32:0/4213241985 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48b80cdc40 con 0x7f48b8162e10 2024-04-24T23:06:19.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.636+0000 7f48bd372640 1 --2- 172.21.15.32:0/4213241985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48b8162e10 0x7f48b80b1070 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f48a800e820 tx=0x7f48a800ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:19.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.636+0000 7f48b57fa640 1 -- 172.21.15.32:0/4213241985 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a8017070 con 0x7f48b8162e10 2024-04-24T23:06:19.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.636+0000 7f48b57fa640 1 -- 172.21.15.32:0/4213241985 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48a800ee70 con 0x7f48b8162e10 2024-04-24T23:06:19.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.636+0000 7f48bedd5640 1 -- 172.21.15.32:0/4213241985 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48b80cdf20 con 0x7f48b8162e10 2024-04-24T23:06:19.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.636+0000 7f48b57fa640 1 -- 172.21.15.32:0/4213241985 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a800cb60 con 0x7f48b8162e10 2024-04-24T23:06:19.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.636+0000 7f48bedd5640 1 -- 172.21.15.32:0/4213241985 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48b80b15b0 con 0x7f48b8162e10 2024-04-24T23:06:19.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.637+0000 7f48bedd5640 1 -- 172.21.15.32:0/4213241985 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48b80ce440 con 0x7f48b8162e10 2024-04-24T23:06:19.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.638+0000 7f48b57fa640 1 -- 172.21.15.32:0/4213241985 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f48a801a070 con 0x7f48b8162e10 2024-04-24T23:06:19.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.638+0000 7f48b57fa640 1 --2- 172.21.15.32:0/4213241985 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f489806be30 0x7f489806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:19.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.639+0000 7f48b57fa640 1 -- 172.21.15.32:0/4213241985 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5796+0+0 (secure 0 0 0) 0x7f48a808b130 con 0x7f48b8162e10 2024-04-24T23:06:19.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.639+0000 7f48bcb71640 1 --2- 172.21.15.32:0/4213241985 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f489806be30 0x7f489806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:19.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.639+0000 7f48bcb71640 1 --2- 172.21.15.32:0/4213241985 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f489806be30 0x7f489806e2f0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f48b0034000 tx=0x7f48b002f000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:19.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.643+0000 7f48b57fa640 1 -- 172.21.15.32:0/4213241985 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f48b80ce440 con 0x7f48b8162e10 2024-04-24T23:06:19.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.826+0000 7f48bedd5640 1 -- 172.21.15.32:0/4213241985 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f48b81f46c0 con 0x7f48b8162e10 2024-04-24T23:06:19.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.827+0000 7f48b57fa640 1 -- 172.21.15.32:0/4213241985 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) ==== 74+0+16836 (secure 0 0 0) 0x7f48a8059a00 con 0x7f48b8162e10 2024-04-24T23:06:19.828 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:06:19.829 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":145,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:06:06.811191+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:05:32.064234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-24T23:05:32.760777+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.17","mappings":[{"from":4,"to":3}]},{"pgid":"17.8","mappings":[{"from":6,"to":5}]},{"pgid":"17.3f","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:06:19.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.829+0000 7f48bedd5640 1 -- 172.21.15.32:0/4213241985 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f489806be30 msgr2=0x7f489806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:19.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.829+0000 7f48bedd5640 1 --2- 172.21.15.32:0/4213241985 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f489806be30 0x7f489806e2f0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f48b0034000 tx=0x7f48b002f000 comp rx=0 tx=0).stop 2024-04-24T23:06:19.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.829+0000 7f48bedd5640 1 -- 172.21.15.32:0/4213241985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48b8162e10 msgr2=0x7f48b80b1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:19.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.829+0000 7f48bedd5640 1 --2- 172.21.15.32:0/4213241985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48b8162e10 0x7f48b80b1070 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f48a800e820 tx=0x7f48a800ec20 comp rx=0 tx=0).stop 2024-04-24T23:06:19.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.829+0000 7f48bedd5640 1 -- 172.21.15.32:0/4213241985 shutdown_connections 2024-04-24T23:06:19.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.829+0000 7f48bedd5640 1 --2- 172.21.15.32:0/4213241985 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f489806be30 0x7f489806e2f0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:19.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.829+0000 7f48bedd5640 1 --2- 172.21.15.32:0/4213241985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48b8162e10 0x7f48b80b1070 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:19.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.829+0000 7f48bedd5640 1 --2- 172.21.15.32:0/4213241985 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48b8162320 0x7f48b80cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:19.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.829+0000 7f48bedd5640 1 --2- 172.21.15.32:0/4213241985 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f48b80b5af0 0x7f48b80cf170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:19.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.829+0000 7f48bedd5640 1 -- 172.21.15.32:0/4213241985 >> 172.21.15.32:0/4213241985 conn(0x7f48b80d0ae0 msgr2=0x7f48b815dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:19.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.829+0000 7f48bedd5640 1 -- 172.21.15.32:0/4213241985 shutdown_connections 2024-04-24T23:06:19.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.829+0000 7f48bedd5640 1 -- 172.21.15.32:0/4213241985 wait complete. 2024-04-24T23:06:19.843 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-24T23:06:19.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.944+0000 7ff6c5ada640 1 Processor -- start 2024-04-24T23:06:19.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.944+0000 7ff6c5ada640 1 -- start start 2024-04-24T23:06:19.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.945+0000 7ff6c5ada640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff6c0162320 0x7ff6c00ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:19.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.945+0000 7ff6c5ada640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff6c00b5af0 0x7ff6c0162700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:19.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.945+0000 7ff6c5ada640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6c0162c40 0x7ff6c00ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:19.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.945+0000 7ff6c5ada640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7ff6c00b5660 con 0x7ff6c0162320 2024-04-24T23:06:19.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.945+0000 7ff6c5ada640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7ff6c00cd270 con 0x7ff6c00b5af0 2024-04-24T23:06:19.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.945+0000 7ff6c5ada640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff6c0071610 con 0x7ff6c0162c40 2024-04-24T23:06:19.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.945+0000 7ff6bf7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff6c0162320 0x7ff6c00ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:19.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.945+0000 7ff6bf7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff6c0162320 0x7ff6c00ccc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53442/0 (socket says 172.21.15.32:53442) 2024-04-24T23:06:19.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.945+0000 7ff6bf7fe640 1 -- 172.21.15.32:0/2483973019 learned_addr learned my addr 172.21.15.32:0/2483973019 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:19.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.945+0000 7ff6beffd640 1 --2- 172.21.15.32:0/2483973019 >> v2:172.21.15.121:3300/0 conn(0x7ff6c00b5af0 0x7ff6c0162700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:19.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.945+0000 7ff6bffff640 1 --2- 172.21.15.32:0/2483973019 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6c0162c40 0x7ff6c00ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:19.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.946+0000 7ff6bf7fe640 1 -- 172.21.15.32:0/2483973019 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6c0162c40 msgr2=0x7ff6c00ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:19.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.946+0000 7ff6bf7fe640 1 --2- 172.21.15.32:0/2483973019 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6c0162c40 0x7ff6c00ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:19.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.946+0000 7ff6bf7fe640 1 -- 172.21.15.32:0/2483973019 >> v2:172.21.15.121:3300/0 conn(0x7ff6c00b5af0 msgr2=0x7ff6c0162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:19.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.946+0000 7ff6bf7fe640 1 --2- 172.21.15.32:0/2483973019 >> v2:172.21.15.121:3300/0 conn(0x7ff6c00b5af0 0x7ff6c0162700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:19.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.946+0000 7ff6bf7fe640 1 -- 172.21.15.32:0/2483973019 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6c00718f0 con 0x7ff6c0162320 2024-04-24T23:06:19.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.946+0000 7ff6bf7fe640 1 --2- 172.21.15.32:0/2483973019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6c0162320 0x7ff6c00ccc90 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7ff6c00d1bc0 tx=0x7ff6b002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4255cd7c14f5247c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:19.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.946+0000 7ff6be7fc640 1 -- 172.21.15.32:0/2483973019 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b0037070 con 0x7ff6c0162320 2024-04-24T23:06:19.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.946+0000 7ff6be7fc640 1 -- 172.21.15.32:0/2483973019 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6b0032440 con 0x7ff6c0162320 2024-04-24T23:06:19.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.946+0000 7ff6be7fc640 1 -- 172.21.15.32:0/2483973019 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b0032810 con 0x7ff6c0162320 2024-04-24T23:06:19.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.947+0000 7ff6c5ada640 1 -- 172.21.15.32:0/2483973019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6c0162320 msgr2=0x7ff6c00ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:19.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.947+0000 7ff6c5ada640 1 --2- 172.21.15.32:0/2483973019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6c0162320 0x7ff6c00ccc90 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7ff6c00d1bc0 tx=0x7ff6b002a140 comp rx=0 tx=0).stop 2024-04-24T23:06:19.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.947+0000 7ff6c5ada640 1 -- 172.21.15.32:0/2483973019 shutdown_connections 2024-04-24T23:06:19.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.947+0000 7ff6c5ada640 1 --2- 172.21.15.32:0/2483973019 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6c0162c40 0x7ff6c00ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:19.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.947+0000 7ff6c5ada640 1 --2- 172.21.15.32:0/2483973019 >> v2:172.21.15.121:3300/0 conn(0x7ff6c00b5af0 0x7ff6c0162700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:19.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.947+0000 7ff6c5ada640 1 --2- 172.21.15.32:0/2483973019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6c0162320 0x7ff6c00ccc90 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:19.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.947+0000 7ff6c5ada640 1 -- 172.21.15.32:0/2483973019 >> 172.21.15.32:0/2483973019 conn(0x7ff6c00d0ae0 msgr2=0x7ff6c00d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:19.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.947+0000 7ff6c5ada640 1 -- 172.21.15.32:0/2483973019 shutdown_connections 2024-04-24T23:06:19.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.947+0000 7ff6c5ada640 1 -- 172.21.15.32:0/2483973019 wait complete. 2024-04-24T23:06:19.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.948+0000 7ff6c5ada640 1 Processor -- start 2024-04-24T23:06:19.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.948+0000 7ff6c5ada640 1 -- start start 2024-04-24T23:06:19.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.949+0000 7ff6c5ada640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6c00b5af0 0x7ff6c00ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:19.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.949+0000 7ff6c5ada640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6c0162320 0x7ff6c015ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:19.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.949+0000 7ff6c5ada640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6c0162c40 0x7ff6c006dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:19.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.949+0000 7ff6c5ada640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c0067950 con 0x7ff6c00b5af0 2024-04-24T23:06:19.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.949+0000 7ff6c5ada640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c015c630 con 0x7ff6c0162c40 2024-04-24T23:06:19.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.949+0000 7ff6c5ada640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff6c0071bd0 con 0x7ff6c0162320 2024-04-24T23:06:19.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.949+0000 7ff6bf7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6c00b5af0 0x7ff6c00ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:19.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.949+0000 7ff6bf7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6c00b5af0 0x7ff6c00ceb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53448/0 (socket says 172.21.15.32:53448) 2024-04-24T23:06:19.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.949+0000 7ff6bf7fe640 1 -- 172.21.15.32:0/120317373 learned_addr learned my addr 172.21.15.32:0/120317373 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:19.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.949+0000 7ff6beffd640 1 --2- 172.21.15.32:0/120317373 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6c0162320 0x7ff6c015ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:19.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.949+0000 7ff6bf7fe640 1 -- 172.21.15.32:0/120317373 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6c0162320 msgr2=0x7ff6c015ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:19.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.949+0000 7ff6bffff640 1 --2- 172.21.15.32:0/120317373 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6c0162c40 0x7ff6c006dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:19.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.949+0000 7ff6bf7fe640 1 --2- 172.21.15.32:0/120317373 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6c0162320 0x7ff6c015ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:19.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.950+0000 7ff6bf7fe640 1 -- 172.21.15.32:0/120317373 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6c0162c40 msgr2=0x7ff6c006dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:19.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.950+0000 7ff6bf7fe640 1 --2- 172.21.15.32:0/120317373 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6c0162c40 0x7ff6c006dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:19.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.950+0000 7ff6bf7fe640 1 -- 172.21.15.32:0/120317373 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6c015ba30 con 0x7ff6c00b5af0 2024-04-24T23:06:19.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.950+0000 7ff6bffff640 1 --2- 172.21.15.32:0/120317373 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6c0162c40 0x7ff6c006dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:06:19.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.950+0000 7ff6bf7fe640 1 --2- 172.21.15.32:0/120317373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6c00b5af0 0x7ff6c00ceb60 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7ff6b002a180 tx=0x7ff6b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:19.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.950+0000 7ff6a7fff640 1 -- 172.21.15.32:0/120317373 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b0041020 con 0x7ff6c00b5af0 2024-04-24T23:06:19.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.950+0000 7ff6a7fff640 1 -- 172.21.15.32:0/120317373 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6b003d620 con 0x7ff6c00b5af0 2024-04-24T23:06:19.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.950+0000 7ff6c5ada640 1 -- 172.21.15.32:0/120317373 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6c00cf1f0 con 0x7ff6c00b5af0 2024-04-24T23:06:19.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.950+0000 7ff6a7fff640 1 -- 172.21.15.32:0/120317373 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b0037040 con 0x7ff6c00b5af0 2024-04-24T23:06:19.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.950+0000 7ff6c5ada640 1 -- 172.21.15.32:0/120317373 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6c00cf7b0 con 0x7ff6c00b5af0 2024-04-24T23:06:19.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.951+0000 7ff6c5ada640 1 -- 172.21.15.32:0/120317373 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6c0068430 con 0x7ff6c00b5af0 2024-04-24T23:06:19.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.952+0000 7ff6a7fff640 1 -- 172.21.15.32:0/120317373 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff6b0044020 con 0x7ff6c00b5af0 2024-04-24T23:06:19.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.952+0000 7ff6a7fff640 1 --2- 172.21.15.32:0/120317373 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff6ac06be30 0x7ff6ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:19.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.952+0000 7ff6a7fff640 1 -- 172.21.15.32:0/120317373 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5796+0+0 (secure 0 0 0) 0x7ff6b00ab4e0 con 0x7ff6c00b5af0 2024-04-24T23:06:19.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.952+0000 7ff6beffd640 1 --2- 172.21.15.32:0/120317373 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff6ac06be30 0x7ff6ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:19.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.953+0000 7ff6beffd640 1 --2- 172.21.15.32:0/120317373 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff6ac06be30 0x7ff6ac06e2f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ff6c006e7e0 tx=0x7ff6b4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:19.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:19.956+0000 7ff6a7fff640 1 -- 172.21.15.32:0/120317373 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff6b004b030 con 0x7ff6c00b5af0 2024-04-24T23:06:20.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.179+0000 7ff6c5ada640 1 -- 172.21.15.32:0/120317373 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff6c0067730 con 0x7ff6c00b5af0 2024-04-24T23:06:20.182 INFO:tasks.ceph.mon.a.smithi032.stderr:2024-04-24T23:06:20.180+0000 7f3b809c6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T23:06:20.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.181+0000 7ff6a7fff640 1 -- 172.21.15.32:0/120317373 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v59) ==== 161+0+0 (secure 0 0 0) 0x7ff6b0079ec0 con 0x7ff6c00b5af0 2024-04-24T23:06:20.183 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-24T23:06:20.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.185+0000 7ff6c5ada640 1 -- 172.21.15.32:0/120317373 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff6ac06be30 msgr2=0x7ff6ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:20.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.185+0000 7ff6c5ada640 1 --2- 172.21.15.32:0/120317373 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff6ac06be30 0x7ff6ac06e2f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ff6c006e7e0 tx=0x7ff6b4008040 comp rx=0 tx=0).stop 2024-04-24T23:06:20.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.185+0000 7ff6c5ada640 1 -- 172.21.15.32:0/120317373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6c00b5af0 msgr2=0x7ff6c00ceb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:20.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.185+0000 7ff6c5ada640 1 --2- 172.21.15.32:0/120317373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6c00b5af0 0x7ff6c00ceb60 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7ff6b002a180 tx=0x7ff6b00046c0 comp rx=0 tx=0).stop 2024-04-24T23:06:20.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.185+0000 7ff6c5ada640 1 -- 172.21.15.32:0/120317373 shutdown_connections 2024-04-24T23:06:20.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.185+0000 7ff6c5ada640 1 --2- 172.21.15.32:0/120317373 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff6ac06be30 0x7ff6ac06e2f0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:20.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.185+0000 7ff6c5ada640 1 --2- 172.21.15.32:0/120317373 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6c0162c40 0x7ff6c006dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:20.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.185+0000 7ff6c5ada640 1 --2- 172.21.15.32:0/120317373 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6c0162320 0x7ff6c015ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:20.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.185+0000 7ff6c5ada640 1 --2- 172.21.15.32:0/120317373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6c00b5af0 0x7ff6c00ceb60 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:20.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.185+0000 7ff6c5ada640 1 -- 172.21.15.32:0/120317373 >> 172.21.15.32:0/120317373 conn(0x7ff6c00d0ae0 msgr2=0x7ff6c00b09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:20.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.185+0000 7ff6c5ada640 1 -- 172.21.15.32:0/120317373 shutdown_connections 2024-04-24T23:06:20.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.185+0000 7ff6c5ada640 1 -- 172.21.15.32:0/120317373 wait complete. 2024-04-24T23:06:20.188 INFO:tasks.ceph.mds.a.smithi032.stderr: -9925> 2024-04-24T23:05:39.395+0000 7fb2b9737640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T23:06:20.198 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-24T23:06:20.223 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2024-04-24T23:06:20.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.301+0000 7f0e6844c640 1 Processor -- start 2024-04-24T23:06:20.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.301+0000 7f0e6844c640 1 -- start start 2024-04-24T23:06:20.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.302+0000 7f0e6844c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0e600b5560 0x7f0e6015e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:20.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.302+0000 7f0e6844c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e600cd3c0 0x7f0e6015ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:20.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.302+0000 7f0e6844c640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e600cd910 con 0x7f0e600b5940 2024-04-24T23:06:20.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.302+0000 7f0e6844c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f0e600cdbf0 con 0x7f0e600b5560 2024-04-24T23:06:20.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.302+0000 7f0e6844c640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f0e600cded0 con 0x7f0e600cd3c0 2024-04-24T23:06:20.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.302+0000 7f0e661e8640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0e600b5560 0x7f0e6015e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:20.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.303+0000 7f0e661e8640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0e600b5560 0x7f0e6015e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53464/0 (socket says 172.21.15.32:53464) 2024-04-24T23:06:20.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.303+0000 7f0e661e8640 1 -- 172.21.15.32:0/564136068 learned_addr learned my addr 172.21.15.32:0/564136068 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:20.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.303+0000 7f0e659e7640 1 --2- 172.21.15.32:0/564136068 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e600cd3c0 0x7f0e6015ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:20.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.303+0000 7f0e661e8640 1 -- 172.21.15.32:0/564136068 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e600cd3c0 msgr2=0x7f0e6015ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:20.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.303+0000 7f0e661e8640 1 --2- 172.21.15.32:0/564136068 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e600cd3c0 0x7f0e6015ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:20.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.303+0000 7f0e661e8640 1 -- 172.21.15.32:0/564136068 >> v1:172.21.15.121:6789/0 conn(0x7f0e600b5940 legacy=0x7f0e600b5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:20.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.303+0000 7f0e661e8640 1 -- 172.21.15.32:0/564136068 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e600ce1b0 con 0x7f0e600b5560 2024-04-24T23:06:20.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.304+0000 7f0e661e8640 1 --2- 172.21.15.32:0/564136068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e600b5560 0x7f0e6015e6c0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f0e5800bb00 tx=0x7f0e5802c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edd4bf329d59e230 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:20.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.304+0000 7f0e651e6640 1 -- 172.21.15.32:0/564136068 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5803a070 con 0x7f0e600b5560 2024-04-24T23:06:20.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.304+0000 7f0e651e6640 1 -- 172.21.15.32:0/564136068 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e58032480 con 0x7f0e600b5560 2024-04-24T23:06:20.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.304+0000 7f0e651e6640 1 -- 172.21.15.32:0/564136068 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e58032850 con 0x7f0e600b5560 2024-04-24T23:06:20.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.304+0000 7f0e6844c640 1 -- 172.21.15.32:0/564136068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e600b5560 msgr2=0x7f0e6015e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:20.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.304+0000 7f0e6844c640 1 --2- 172.21.15.32:0/564136068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e600b5560 0x7f0e6015e6c0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f0e5800bb00 tx=0x7f0e5802c6e0 comp rx=0 tx=0).stop 2024-04-24T23:06:20.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.304+0000 7f0e6844c640 1 -- 172.21.15.32:0/564136068 shutdown_connections 2024-04-24T23:06:20.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.304+0000 7f0e6844c640 1 --2- 172.21.15.32:0/564136068 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e600cd3c0 0x7f0e6015ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:20.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.304+0000 7f0e6844c640 1 --2- 172.21.15.32:0/564136068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e600b5560 0x7f0e6015e6c0 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:20.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.304+0000 7f0e6844c640 1 -- 172.21.15.32:0/564136068 >> 172.21.15.32:0/564136068 conn(0x7f0e6006b110 msgr2=0x7f0e6006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:20.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.304+0000 7f0e6844c640 1 -- 172.21.15.32:0/564136068 shutdown_connections 2024-04-24T23:06:20.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.304+0000 7f0e6844c640 1 -- 172.21.15.32:0/564136068 wait complete. 2024-04-24T23:06:20.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.306+0000 7f0e6844c640 1 Processor -- start 2024-04-24T23:06:20.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.306+0000 7f0e6844c640 1 -- start start 2024-04-24T23:06:20.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.306+0000 7f0e6844c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e600b5560 0x7f0e6015ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:20.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.306+0000 7f0e6844c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e600b5940 0x7f0e6015cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:20.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.306+0000 7f0e6844c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e600cd3c0 0x7f0e60149e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:20.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.306+0000 7f0e6844c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6006c770 con 0x7f0e600b5940 2024-04-24T23:06:20.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.306+0000 7f0e6844c640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6006c5f0 con 0x7f0e600b5560 2024-04-24T23:06:20.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.306+0000 7f0e6844c640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f0e6006c8f0 con 0x7f0e600cd3c0 2024-04-24T23:06:20.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.307+0000 7f0e659e7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e600b5940 0x7f0e6015cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:20.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.307+0000 7f0e659e7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e600b5940 0x7f0e6015cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53466/0 (socket says 172.21.15.32:53466) 2024-04-24T23:06:20.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.307+0000 7f0e659e7640 1 -- 172.21.15.32:0/3322048879 learned_addr learned my addr 172.21.15.32:0/3322048879 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:20.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.307+0000 7f0e661e8640 1 --2- 172.21.15.32:0/3322048879 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e600b5560 0x7f0e6015ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:20.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.307+0000 7f0e669e9640 1 --2- 172.21.15.32:0/3322048879 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e600cd3c0 0x7f0e60149e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:20.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.307+0000 7f0e659e7640 1 -- 172.21.15.32:0/3322048879 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e600cd3c0 msgr2=0x7f0e60149e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:20.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.307+0000 7f0e659e7640 1 --2- 172.21.15.32:0/3322048879 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e600cd3c0 0x7f0e60149e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:20.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.307+0000 7f0e659e7640 1 -- 172.21.15.32:0/3322048879 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e600b5560 msgr2=0x7f0e6015ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:20.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.307+0000 7f0e659e7640 1 --2- 172.21.15.32:0/3322048879 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e600b5560 0x7f0e6015ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:20.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.307+0000 7f0e659e7640 1 -- 172.21.15.32:0/3322048879 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e6015d140 con 0x7f0e600b5940 2024-04-24T23:06:20.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.307+0000 7f0e669e9640 1 --2- 172.21.15.32:0/3322048879 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e600cd3c0 0x7f0e60149e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:06:20.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.307+0000 7f0e659e7640 1 --2- 172.21.15.32:0/3322048879 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e600b5940 0x7f0e6015cc00 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f0e5400e870 tx=0x7f0e5400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:20.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.307+0000 7f0e5effd640 1 -- 172.21.15.32:0/3322048879 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e54017070 con 0x7f0e600b5940 2024-04-24T23:06:20.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.307+0000 7f0e5effd640 1 -- 172.21.15.32:0/3322048879 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e5400c940 con 0x7f0e600b5940 2024-04-24T23:06:20.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.307+0000 7f0e6844c640 1 -- 172.21.15.32:0/3322048879 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e6015d3b0 con 0x7f0e600b5940 2024-04-24T23:06:20.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.307+0000 7f0e5effd640 1 -- 172.21.15.32:0/3322048879 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5400cd30 con 0x7f0e600b5940 2024-04-24T23:06:20.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.307+0000 7f0e6844c640 1 -- 172.21.15.32:0/3322048879 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e6015dc50 con 0x7f0e600b5940 2024-04-24T23:06:20.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.308+0000 7f0e6844c640 1 -- 172.21.15.32:0/3322048879 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e60068430 con 0x7f0e600b5940 2024-04-24T23:06:20.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.309+0000 7f0e5effd640 1 -- 172.21.15.32:0/3322048879 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0e5401a070 con 0x7f0e600b5940 2024-04-24T23:06:20.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.309+0000 7f0e5effd640 1 --2- 172.21.15.32:0/3322048879 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0e4c06be30 0x7f0e4c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:20.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.309+0000 7f0e5effd640 1 -- 172.21.15.32:0/3322048879 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5882+0+0 (secure 0 0 0) 0x7f0e5408ad10 con 0x7f0e600b5940 2024-04-24T23:06:20.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.309+0000 7f0e661e8640 1 --2- 172.21.15.32:0/3322048879 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0e4c06be30 0x7f0e4c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:20.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.310+0000 7f0e661e8640 1 --2- 172.21.15.32:0/3322048879 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0e4c06be30 0x7f0e4c06e2f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f0e58037000 tx=0x7f0e58030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:20.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.313+0000 7f0e5effd640 1 -- 172.21.15.32:0/3322048879 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0e54058fc0 con 0x7f0e600b5940 2024-04-24T23:06:20.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:20.534+0000 7f0e6844c640 1 -- 172.21.15.32:0/3322048879 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f0e600afd60 con 0x7f0e600b5940 2024-04-24T23:06:21.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.182+0000 7f0e5effd640 1 -- 172.21.15.32:0/3322048879 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v60) ==== 104+0+0 (secure 0 0 0) 0x7f0e5405d2c0 con 0x7f0e600b5940 2024-04-24T23:06:21.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.185+0000 7f0e6844c640 1 -- 172.21.15.32:0/3322048879 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0e4c06be30 msgr2=0x7f0e4c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:21.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.185+0000 7f0e6844c640 1 --2- 172.21.15.32:0/3322048879 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0e4c06be30 0x7f0e4c06e2f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f0e58037000 tx=0x7f0e58030000 comp rx=0 tx=0).stop 2024-04-24T23:06:21.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.185+0000 7f0e6844c640 1 -- 172.21.15.32:0/3322048879 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e600b5940 msgr2=0x7f0e6015cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:21.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.185+0000 7f0e6844c640 1 --2- 172.21.15.32:0/3322048879 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e600b5940 0x7f0e6015cc00 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f0e5400e870 tx=0x7f0e5400ed30 comp rx=0 tx=0).stop 2024-04-24T23:06:21.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.185+0000 7f0e6844c640 1 -- 172.21.15.32:0/3322048879 shutdown_connections 2024-04-24T23:06:21.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.185+0000 7f0e6844c640 1 --2- 172.21.15.32:0/3322048879 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0e4c06be30 0x7f0e4c06e2f0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:21.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.185+0000 7f0e6844c640 1 --2- 172.21.15.32:0/3322048879 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e600b5940 0x7f0e6015cc00 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:21.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.185+0000 7f0e6844c640 1 --2- 172.21.15.32:0/3322048879 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e600cd3c0 0x7f0e60149e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:21.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.185+0000 7f0e6844c640 1 --2- 172.21.15.32:0/3322048879 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e600b5560 0x7f0e6015ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:21.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.185+0000 7f0e6844c640 1 -- 172.21.15.32:0/3322048879 >> 172.21.15.32:0/3322048879 conn(0x7f0e6006b110 msgr2=0x7f0e600b0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:21.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.185+0000 7f0e6844c640 1 -- 172.21.15.32:0/3322048879 shutdown_connections 2024-04-24T23:06:21.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.185+0000 7f0e6844c640 1 -- 172.21.15.32:0/3322048879 wait complete. 2024-04-24T23:06:21.200 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-24T23:06:21.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.350+0000 7fef850b3640 1 Processor -- start 2024-04-24T23:06:21.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.350+0000 7fef850b3640 1 -- start start 2024-04-24T23:06:21.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.351+0000 7fef850b3640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef8015ea40 0x7fef800b4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:21.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.351+0000 7fef850b3640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef800732f0 con 0x7fef800b1ab0 2024-04-24T23:06:21.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.351+0000 7fef850b3640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef800735d0 con 0x7fef800d0550 2024-04-24T23:06:21.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.351+0000 7fef850b3640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fef800b5c00 con 0x7fef8015ea40 2024-04-24T23:06:21.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.351+0000 7fef7e59c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fef800b1ab0 0x7fef800b54f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42700/0 (socket says 172.21.15.32:42700) 2024-04-24T23:06:21.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.351+0000 7fef7e59c640 1 -- 172.21.15.32:0/3320413940 learned_addr learned my addr 172.21.15.32:0/3320413940 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:06:21.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.351+0000 7fef7ed9d640 1 --2- 172.21.15.32:0/3320413940 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef8015ea40 0x7fef800b4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:21.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.351+0000 7fef7dd9b640 1 -- 172.21.15.32:0/3320413940 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2173708049 0 0) 0x7fef800732f0 con 0x7fef800b1ab0 2024-04-24T23:06:21.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.352+0000 7fef7dd9b640 1 -- 172.21.15.32:0/3320413940 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fef6c003520 con 0x7fef800b1ab0 2024-04-24T23:06:21.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.352+0000 7fef7dd9b640 1 -- 172.21.15.32:0/3320413940 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 499407219 0 0) 0x7fef6c003520 con 0x7fef800b1ab0 2024-04-24T23:06:21.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.352+0000 7fef7dd9b640 1 -- 172.21.15.32:0/3320413940 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef8015ea40 msgr2=0x7fef800b4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:21.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.352+0000 7fef7dd9b640 1 --2- 172.21.15.32:0/3320413940 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef8015ea40 0x7fef800b4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:21.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.352+0000 7fef7dd9b640 1 -- 172.21.15.32:0/3320413940 >> v1:172.21.15.121:6789/0 conn(0x7fef800d0550 legacy=0x7fef8006bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:21.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.352+0000 7fef7dd9b640 1 -- 172.21.15.32:0/3320413940 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef8006b590 con 0x7fef800b1ab0 2024-04-24T23:06:21.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.352+0000 7fef7dd9b640 1 -- 172.21.15.32:0/3320413940 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7fef68001670 con 0x7fef800b1ab0 2024-04-24T23:06:21.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.352+0000 7fef7dd9b640 1 -- 172.21.15.32:0/3320413940 >> v1:172.21.15.32:6789/0 conn(0x7fef800b1ab0 legacy=0x7fef800b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:21.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.352+0000 7fef7dd9b640 1 --2- 172.21.15.32:0/3320413940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef6c0039e0 0x7fef6c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:21.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.352+0000 7fef7dd9b640 1 --2- 172.21.15.32:0/3320413940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef6c004390 0x7fef6c004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:21.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.352+0000 7fef7dd9b640 1 --2- 172.21.15.32:0/3320413940 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef8015ea40 0x7fef6c005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:21.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.352+0000 7fef7dd9b640 1 -- 172.21.15.32:0/3320413940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fef800b5c00 con 0x7fef6c0039e0 2024-04-24T23:06:21.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.352+0000 7fef7dd9b640 1 -- 172.21.15.32:0/3320413940 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c00ba70 con 0x7fef6c004390 2024-04-24T23:06:21.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.352+0000 7fef7dd9b640 1 -- 172.21.15.32:0/3320413940 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fef6c00bc70 con 0x7fef8015ea40 2024-04-24T23:06:21.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.352+0000 7fef7ed9d640 1 --2- 172.21.15.32:0/3320413940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef6c0039e0 0x7fef6c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:21.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.352+0000 7fef7e59c640 1 --2- 172.21.15.32:0/3320413940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef6c004390 0x7fef6c004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:21.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.352+0000 7fef7f59e640 1 --2- 172.21.15.32:0/3320413940 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef8015ea40 0x7fef6c005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:21.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.352+0000 7fef7ed9d640 1 -- 172.21.15.32:0/3320413940 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef8015ea40 msgr2=0x7fef6c005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:21.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.352+0000 7fef7ed9d640 1 --2- 172.21.15.32:0/3320413940 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef8015ea40 0x7fef6c005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:21.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.352+0000 7fef7ed9d640 1 -- 172.21.15.32:0/3320413940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef6c004390 msgr2=0x7fef6c004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:21.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.352+0000 7fef7ed9d640 1 --2- 172.21.15.32:0/3320413940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef6c004390 0x7fef6c004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:21.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.352+0000 7fef7ed9d640 1 -- 172.21.15.32:0/3320413940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef6c003520 con 0x7fef6c0039e0 2024-04-24T23:06:21.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.353+0000 7fef7ed9d640 1 --2- 172.21.15.32:0/3320413940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef6c0039e0 0x7fef6c003dc0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fef78009ac0 tx=0x7fef6c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99571755e42029b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:21.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.353+0000 7fef7dd9b640 1 -- 172.21.15.32:0/3320413940 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef78036070 con 0x7fef6c0039e0 2024-04-24T23:06:21.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.353+0000 7fef7dd9b640 1 -- 172.21.15.32:0/3320413940 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef78031400 con 0x7fef6c0039e0 2024-04-24T23:06:21.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.353+0000 7fef7dd9b640 1 -- 172.21.15.32:0/3320413940 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef78009690 con 0x7fef6c0039e0 2024-04-24T23:06:21.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.354+0000 7fef850b3640 1 -- 172.21.15.32:0/3320413940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef6c0039e0 msgr2=0x7fef6c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:21.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.354+0000 7fef850b3640 1 --2- 172.21.15.32:0/3320413940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef6c0039e0 0x7fef6c003dc0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fef78009ac0 tx=0x7fef6c005830 comp rx=0 tx=0).stop 2024-04-24T23:06:21.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.354+0000 7fef7f59e640 1 -- 172.21.15.32:0/3320413940 reap_dead start 2024-04-24T23:06:21.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.354+0000 7fef850b3640 1 -- 172.21.15.32:0/3320413940 shutdown_connections 2024-04-24T23:06:21.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.354+0000 7fef850b3640 1 -- 172.21.15.32:0/3320413940 >> 172.21.15.32:0/3320413940 conn(0x7fef800b41f0 msgr2=0x7fef800b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:21.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.354+0000 7fef850b3640 1 -- 172.21.15.32:0/3320413940 shutdown_connections 2024-04-24T23:06:21.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.354+0000 7fef850b3640 1 -- 172.21.15.32:0/3320413940 wait complete. 2024-04-24T23:06:21.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.355+0000 7fef850b3640 1 Processor -- start 2024-04-24T23:06:21.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.355+0000 7fef850b3640 1 -- start start 2024-04-24T23:06:21.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.356+0000 7fef850b3640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef6c0039e0 0x7fef80152310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:21.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.356+0000 7fef850b3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef801537f0 0x7fef80158170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:21.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.356+0000 7fef850b3640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef80153c70 0x7fef801586b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:21.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.356+0000 7fef850b3640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fef80072f70 con 0x7fef801537f0 2024-04-24T23:06:21.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.356+0000 7fef850b3640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fef8006b220 con 0x7fef6c0039e0 2024-04-24T23:06:21.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.356+0000 7fef850b3640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fef8006c4a0 con 0x7fef80153c70 2024-04-24T23:06:21.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.356+0000 7fef7e59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef801537f0 0x7fef80158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:21.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.356+0000 7fef7e59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef801537f0 0x7fef80158170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53488/0 (socket says 172.21.15.32:53488) 2024-04-24T23:06:21.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.356+0000 7fef7e59c640 1 -- 172.21.15.32:0/1042494557 learned_addr learned my addr 172.21.15.32:0/1042494557 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:21.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.356+0000 7fef7ed9d640 1 --2- 172.21.15.32:0/1042494557 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef6c0039e0 0x7fef80152310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:21.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.356+0000 7fef7f59e640 1 --2- 172.21.15.32:0/1042494557 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef80153c70 0x7fef801586b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:21.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.356+0000 7fef7e59c640 1 -- 172.21.15.32:0/1042494557 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef80153c70 msgr2=0x7fef801586b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:21.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.356+0000 7fef7e59c640 1 --2- 172.21.15.32:0/1042494557 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef80153c70 0x7fef801586b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:21.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.356+0000 7fef7e59c640 1 -- 172.21.15.32:0/1042494557 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef6c0039e0 msgr2=0x7fef80152310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:21.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.356+0000 7fef7e59c640 1 --2- 172.21.15.32:0/1042494557 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef6c0039e0 0x7fef80152310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:21.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.356+0000 7fef7e59c640 1 -- 172.21.15.32:0/1042494557 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef8015ee20 con 0x7fef801537f0 2024-04-24T23:06:21.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.356+0000 7fef7ed9d640 1 --2- 172.21.15.32:0/1042494557 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef6c0039e0 0x7fef80152310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:06:21.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.357+0000 7fef7e59c640 1 --2- 172.21.15.32:0/1042494557 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef801537f0 0x7fef80158170 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fef6800e730 tx=0x7fef6800ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:21.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.357+0000 7fef777fe640 1 -- 172.21.15.32:0/1042494557 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6800c660 con 0x7fef801537f0 2024-04-24T23:06:21.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.357+0000 7fef777fe640 1 -- 172.21.15.32:0/1042494557 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef68018070 con 0x7fef801537f0 2024-04-24T23:06:21.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.357+0000 7fef777fe640 1 -- 172.21.15.32:0/1042494557 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6800ca60 con 0x7fef801537f0 2024-04-24T23:06:21.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.357+0000 7fef850b3640 1 -- 172.21.15.32:0/1042494557 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef8015f100 con 0x7fef801537f0 2024-04-24T23:06:21.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.357+0000 7fef850b3640 1 -- 172.21.15.32:0/1042494557 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef8015f6c0 con 0x7fef801537f0 2024-04-24T23:06:21.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.358+0000 7fef850b3640 1 -- 172.21.15.32:0/1042494557 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef80068430 con 0x7fef801537f0 2024-04-24T23:06:21.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.358+0000 7fef777fe640 1 -- 172.21.15.32:0/1042494557 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fef6800cc00 con 0x7fef801537f0 2024-04-24T23:06:21.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.359+0000 7fef777fe640 1 --2- 172.21.15.32:0/1042494557 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fef6006be60 0x7fef6006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:21.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.359+0000 7fef777fe640 1 -- 172.21.15.32:0/1042494557 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5882+0+0 (secure 0 0 0) 0x7fef6808a8e0 con 0x7fef801537f0 2024-04-24T23:06:21.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.359+0000 7fef7ed9d640 1 --2- 172.21.15.32:0/1042494557 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fef6006be60 0x7fef6006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:21.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.360+0000 7fef7ed9d640 1 --2- 172.21.15.32:0/1042494557 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fef6006be60 0x7fef6006e320 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fef78033000 tx=0x7fef78008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:21.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.363+0000 7fef777fe640 1 -- 172.21.15.32:0/1042494557 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fef680591e0 con 0x7fef801537f0 2024-04-24T23:06:21.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.545+0000 7fef850b3640 1 -- 172.21.15.32:0/1042494557 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fef800b6980 con 0x7fef801537f0 2024-04-24T23:06:21.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.814+0000 7fef777fe640 1 -- 172.21.15.32:0/1042494557 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v147) ==== 181+0+0 (secure 0 0 0) 0x7fef68058ae0 con 0x7fef801537f0 2024-04-24T23:06:21.816 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2024-04-24T23:06:21.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.816+0000 7fef850b3640 1 -- 172.21.15.32:0/1042494557 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fef6006be60 msgr2=0x7fef6006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:21.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.816+0000 7fef850b3640 1 --2- 172.21.15.32:0/1042494557 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fef6006be60 0x7fef6006e320 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fef78033000 tx=0x7fef78008030 comp rx=0 tx=0).stop 2024-04-24T23:06:21.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.816+0000 7fef850b3640 1 -- 172.21.15.32:0/1042494557 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef801537f0 msgr2=0x7fef80158170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:21.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.816+0000 7fef850b3640 1 --2- 172.21.15.32:0/1042494557 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef801537f0 0x7fef80158170 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fef6800e730 tx=0x7fef6800ebf0 comp rx=0 tx=0).stop 2024-04-24T23:06:21.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.816+0000 7fef850b3640 1 -- 172.21.15.32:0/1042494557 shutdown_connections 2024-04-24T23:06:21.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.816+0000 7fef850b3640 1 --2- 172.21.15.32:0/1042494557 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fef6006be60 0x7fef6006e320 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:21.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.816+0000 7fef850b3640 1 --2- 172.21.15.32:0/1042494557 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef801537f0 0x7fef80158170 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:21.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.816+0000 7fef850b3640 1 --2- 172.21.15.32:0/1042494557 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef80153c70 0x7fef801586b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:21.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.816+0000 7fef850b3640 1 --2- 172.21.15.32:0/1042494557 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef6c0039e0 0x7fef80152310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:21.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.816+0000 7fef850b3640 1 -- 172.21.15.32:0/1042494557 >> 172.21.15.32:0/1042494557 conn(0x7fef800b41f0 msgr2=0x7fef8015d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:21.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.817+0000 7fef850b3640 1 -- 172.21.15.32:0/1042494557 shutdown_connections 2024-04-24T23:06:21.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.817+0000 7fef850b3640 1 -- 172.21.15.32:0/1042494557 wait complete. 2024-04-24T23:06:21.829 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-24T23:06:21.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.935+0000 7f6d29529640 1 Processor -- start 2024-04-24T23:06:21.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.935+0000 7f6d29529640 1 -- start start 2024-04-24T23:06:21.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.936+0000 7f6d29529640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6d24161c30 0x7f6d24067980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:21.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.936+0000 7f6d29529640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6d2406d230 0x7f6d240b0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:21.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.936+0000 7f6d29529640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d24071620 con 0x7f6d2406cdc0 2024-04-24T23:06:21.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.936+0000 7f6d29529640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f6d240af5a0 con 0x7f6d2406d230 2024-04-24T23:06:21.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.936+0000 7f6d29529640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f6d240af270 con 0x7f6d24161c30 2024-04-24T23:06:21.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.936+0000 7f6d237fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6d2406d230 0x7f6d240b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:21.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.936+0000 7f6d237fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6d2406d230 0x7f6d240b0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53502/0 (socket says 172.21.15.32:53502) 2024-04-24T23:06:21.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.936+0000 7f6d1bfff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6d2406cdc0 0x7f6d240afa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42714/0 (socket says 172.21.15.32:42714) 2024-04-24T23:06:21.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.936+0000 7f6d237fe640 1 -- 172.21.15.32:0/1221471341 learned_addr learned my addr 172.21.15.32:0/1221471341 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:21.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.936+0000 7f6d22ffd640 1 --2- 172.21.15.32:0/1221471341 >> v2:172.21.15.121:3300/0 conn(0x7f6d24161c30 0x7f6d24067980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:21.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.936+0000 7f6d237fe640 1 -- 172.21.15.32:0/1221471341 >> v2:172.21.15.121:3300/0 conn(0x7f6d24161c30 msgr2=0x7f6d24067980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:21.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.936+0000 7f6d237fe640 1 --2- 172.21.15.32:0/1221471341 >> v2:172.21.15.121:3300/0 conn(0x7f6d24161c30 0x7f6d24067980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:21.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.936+0000 7f6d237fe640 1 -- 172.21.15.32:0/1221471341 >> v1:172.21.15.32:6789/0 conn(0x7f6d2406cdc0 legacy=0x7f6d240afa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:21.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.936+0000 7f6d237fe640 1 -- 172.21.15.32:0/1221471341 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d240718c0 con 0x7f6d2406d230 2024-04-24T23:06:21.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.936+0000 7f6d227fc640 1 -- 172.21.15.32:0/1221471341 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2366905155 0 0) 0x7f6d24071620 con 0x7f6d2406cdc0 2024-04-24T23:06:21.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.937+0000 7f6d237fe640 1 --2- 172.21.15.32:0/1221471341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d2406d230 0x7f6d240b0140 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f6d240717f0 tx=0x7f6d1c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f31fe062970d0f7e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:21.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.937+0000 7f6d227fc640 1 -- 172.21.15.32:0/1221471341 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c036070 con 0x7f6d2406d230 2024-04-24T23:06:21.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.937+0000 7f6d227fc640 1 -- 172.21.15.32:0/1221471341 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d1c031600 con 0x7f6d2406d230 2024-04-24T23:06:21.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.937+0000 7f6d227fc640 1 -- 172.21.15.32:0/1221471341 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c0319d0 con 0x7f6d2406d230 2024-04-24T23:06:21.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.938+0000 7f6d29529640 1 -- 172.21.15.32:0/1221471341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d2406d230 msgr2=0x7f6d240b0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:21.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.938+0000 7f6d29529640 1 --2- 172.21.15.32:0/1221471341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d2406d230 0x7f6d240b0140 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f6d240717f0 tx=0x7f6d1c02a730 comp rx=0 tx=0).stop 2024-04-24T23:06:21.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.938+0000 7f6d29529640 1 -- 172.21.15.32:0/1221471341 shutdown_connections 2024-04-24T23:06:21.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.938+0000 7f6d29529640 1 --2- 172.21.15.32:0/1221471341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d2406d230 0x7f6d240b0140 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:21.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.938+0000 7f6d29529640 1 --2- 172.21.15.32:0/1221471341 >> v2:172.21.15.121:3300/0 conn(0x7f6d24161c30 0x7f6d24067980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:21.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.938+0000 7f6d29529640 1 -- 172.21.15.32:0/1221471341 >> 172.21.15.32:0/1221471341 conn(0x7f6d240d0a60 msgr2=0x7f6d240d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:21.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.938+0000 7f6d29529640 1 -- 172.21.15.32:0/1221471341 shutdown_connections 2024-04-24T23:06:21.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.938+0000 7f6d29529640 1 -- 172.21.15.32:0/1221471341 wait complete. 2024-04-24T23:06:21.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.939+0000 7f6d29529640 1 Processor -- start 2024-04-24T23:06:21.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.939+0000 7f6d29529640 1 -- start start 2024-04-24T23:06:21.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.940+0000 7f6d29529640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d2406cdc0 0x7f6d24142fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:21.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.940+0000 7f6d29529640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d2406d230 0x7f6d2415f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:21.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.940+0000 7f6d29529640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6d24161c30 0x7f6d2415fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:21.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.940+0000 7f6d29529640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6d24070d60 con 0x7f6d2406d230 2024-04-24T23:06:21.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.940+0000 7f6d29529640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6d24162010 con 0x7f6d2406cdc0 2024-04-24T23:06:21.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.940+0000 7f6d29529640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f6d24070ee0 con 0x7f6d24161c30 2024-04-24T23:06:21.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.940+0000 7f6d1bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d2406d230 0x7f6d2415f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:21.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.940+0000 7f6d1bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d2406d230 0x7f6d2415f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53514/0 (socket says 172.21.15.32:53514) 2024-04-24T23:06:21.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.940+0000 7f6d1bfff640 1 -- 172.21.15.32:0/4035265765 learned_addr learned my addr 172.21.15.32:0/4035265765 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:21.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.940+0000 7f6d22ffd640 1 --2- 172.21.15.32:0/4035265765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d2406cdc0 0x7f6d24142fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:21.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.940+0000 7f6d237fe640 1 --2- 172.21.15.32:0/4035265765 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6d24161c30 0x7f6d2415fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:21.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.940+0000 7f6d1bfff640 1 -- 172.21.15.32:0/4035265765 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6d24161c30 msgr2=0x7f6d2415fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:21.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.940+0000 7f6d1bfff640 1 --2- 172.21.15.32:0/4035265765 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6d24161c30 0x7f6d2415fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:21.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.940+0000 7f6d1bfff640 1 -- 172.21.15.32:0/4035265765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d2406cdc0 msgr2=0x7f6d24142fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:21.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.940+0000 7f6d1bfff640 1 --2- 172.21.15.32:0/4035265765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d2406cdc0 0x7f6d24142fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:21.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.940+0000 7f6d1bfff640 1 -- 172.21.15.32:0/4035265765 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d241602a0 con 0x7f6d2406d230 2024-04-24T23:06:21.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.940+0000 7f6d237fe640 1 --2- 172.21.15.32:0/4035265765 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6d24161c30 0x7f6d2415fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:06:21.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.940+0000 7f6d1bfff640 1 --2- 172.21.15.32:0/4035265765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d2406d230 0x7f6d2415f730 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c00c500 tx=0x7f6d0c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:21.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.941+0000 7f6d1b7fe640 1 -- 172.21.15.32:0/4035265765 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c018070 con 0x7f6d2406d230 2024-04-24T23:06:21.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.941+0000 7f6d1b7fe640 1 -- 172.21.15.32:0/4035265765 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d0c007580 con 0x7f6d2406d230 2024-04-24T23:06:21.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.941+0000 7f6d29529640 1 -- 172.21.15.32:0/4035265765 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d24160580 con 0x7f6d2406d230 2024-04-24T23:06:21.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.941+0000 7f6d1b7fe640 1 -- 172.21.15.32:0/4035265765 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c004a20 con 0x7f6d2406d230 2024-04-24T23:06:21.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.941+0000 7f6d29529640 1 -- 172.21.15.32:0/4035265765 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d24160b40 con 0x7f6d2406d230 2024-04-24T23:06:21.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.942+0000 7f6d29529640 1 -- 172.21.15.32:0/4035265765 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d24072df0 con 0x7f6d2406d230 2024-04-24T23:06:21.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.942+0000 7f6d1b7fe640 1 -- 172.21.15.32:0/4035265765 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6d0c01b070 con 0x7f6d2406d230 2024-04-24T23:06:21.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.943+0000 7f6d1b7fe640 1 --2- 172.21.15.32:0/4035265765 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6d0406be30 0x7f6d0406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:21.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.943+0000 7f6d1b7fe640 1 -- 172.21.15.32:0/4035265765 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5415+0+0 (secure 0 0 0) 0x7f6d0c089ff0 con 0x7f6d2406d230 2024-04-24T23:06:21.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.943+0000 7f6d1b7fe640 1 -- 172.21.15.32:0/4035265765 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=148}) -- 0x7f6d04072cc0 con 0x7f6d2406d230 2024-04-24T23:06:21.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.943+0000 7f6d22ffd640 1 --2- 172.21.15.32:0/4035265765 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6d0406be30 0x7f6d0406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:21.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.943+0000 7f6d22ffd640 1 --2- 172.21.15.32:0/4035265765 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6d0406be30 0x7f6d0406e2f0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6d14009a30 tx=0x7f6d14008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:21.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:21.947+0000 7f6d1b7fe640 1 -- 172.21.15.32:0/4035265765 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6d24072df0 con 0x7f6d2406d230 2024-04-24T23:06:22.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:22.131+0000 7f6d29529640 1 -- 172.21.15.32:0/4035265765 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6d24068430 con 0x7f6d2406d230 2024-04-24T23:06:22.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:22.884+0000 7f6d1b7fe640 1 -- 172.21.15.32:0/4035265765 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v148) ==== 169+0+0 (secure 0 0 0) 0x7f6d0c058b50 con 0x7f6d2406d230 2024-04-24T23:06:22.887 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2024-04-24T23:06:22.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:22.884+0000 7f6d1b7fe640 1 -- 172.21.15.32:0/4035265765 <== mon.0 v2:172.21.15.32:3300/0 8 ==== osd_map(148..148 src has 1..148) ==== 330+0+0 (secure 0 0 0) 0x7f6d0c088750 con 0x7f6d2406d230 2024-04-24T23:06:22.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:22.884+0000 7f6d1b7fe640 1 -- 172.21.15.32:0/4035265765 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=149}) -- 0x7f6d04071a50 con 0x7f6d2406d230 2024-04-24T23:06:22.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:22.890+0000 7f6d29529640 1 -- 172.21.15.32:0/4035265765 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6d0406be30 msgr2=0x7f6d0406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:22.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:22.890+0000 7f6d29529640 1 --2- 172.21.15.32:0/4035265765 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6d0406be30 0x7f6d0406e2f0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6d14009a30 tx=0x7f6d14008040 comp rx=0 tx=0).stop 2024-04-24T23:06:22.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:22.890+0000 7f6d29529640 1 -- 172.21.15.32:0/4035265765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d2406d230 msgr2=0x7f6d2415f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:22.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:22.890+0000 7f6d29529640 1 --2- 172.21.15.32:0/4035265765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d2406d230 0x7f6d2415f730 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c00c500 tx=0x7f6d0c00c9c0 comp rx=0 tx=0).stop 2024-04-24T23:06:22.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:22.890+0000 7f6d29529640 1 -- 172.21.15.32:0/4035265765 shutdown_connections 2024-04-24T23:06:22.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:22.890+0000 7f6d29529640 1 --2- 172.21.15.32:0/4035265765 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6d0406be30 0x7f6d0406e2f0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:22.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:22.890+0000 7f6d29529640 1 --2- 172.21.15.32:0/4035265765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d2406d230 0x7f6d2415f730 secure :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c00c500 tx=0x7f6d0c00c9c0 comp rx=0 tx=0).stop 2024-04-24T23:06:22.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:22.890+0000 7f6d29529640 1 --2- 172.21.15.32:0/4035265765 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6d24161c30 0x7f6d2415fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:22.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:22.890+0000 7f6d29529640 1 --2- 172.21.15.32:0/4035265765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d2406cdc0 0x7f6d24142fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:22.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:22.890+0000 7f6d29529640 1 -- 172.21.15.32:0/4035265765 >> 172.21.15.32:0/4035265765 conn(0x7f6d240d0a60 msgr2=0x7f6d240afbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:22.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:22.892+0000 7f6d29529640 1 -- 172.21.15.32:0/4035265765 shutdown_connections 2024-04-24T23:06:22.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:22.892+0000 7f6d29529640 1 -- 172.21.15.32:0/4035265765 wait complete. 2024-04-24T23:06:22.911 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-24T23:06:23.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.015+0000 7f0da4238640 1 Processor -- start 2024-04-24T23:06:23.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.015+0000 7f0da4238640 1 -- start start 2024-04-24T23:06:23.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.016+0000 7f0da4238640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0d9c0ba390 0x7f0d9c0d40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:23.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.016+0000 7f0da4238640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d9c06b110 0x7f0d9c06b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:23.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.016+0000 7f0da4238640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d9c0d3010 con 0x7f0d9c15e9e0 2024-04-24T23:06:23.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.016+0000 7f0da4238640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f0d9c0d2db0 con 0x7f0d9c0ba390 2024-04-24T23:06:23.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.016+0000 7f0da4238640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f0d9c0d32f0 con 0x7f0d9c06b110 2024-04-24T23:06:23.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.016+0000 7f0da1fd4640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0d9c15e9e0 0x7f0d9c0d26a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42722/0 (socket says 172.21.15.32:42722) 2024-04-24T23:06:23.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.016+0000 7f0da1fd4640 1 -- 172.21.15.32:0/111162221 learned_addr learned my addr 172.21.15.32:0/111162221 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:06:23.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.017+0000 7f0da27d5640 1 --2- 172.21.15.32:0/111162221 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d9c06b110 0x7f0d9c06b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:23.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.017+0000 7f0da0fd2640 1 -- 172.21.15.32:0/111162221 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1606851069 0 0) 0x7f0d9c0d3010 con 0x7f0d9c15e9e0 2024-04-24T23:06:23.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.017+0000 7f0da0fd2640 1 -- 172.21.15.32:0/111162221 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0d84003520 con 0x7f0d9c15e9e0 2024-04-24T23:06:23.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.017+0000 7f0da17d3640 1 --2- 172.21.15.32:0/111162221 >> v2:172.21.15.121:3300/0 conn(0x7f0d9c0ba390 0x7f0d9c0d40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:23.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.017+0000 7f0da0fd2640 1 -- 172.21.15.32:0/111162221 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3071034711 0 0) 0x7f0d84003520 con 0x7f0d9c15e9e0 2024-04-24T23:06:23.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.017+0000 7f0da0fd2640 1 -- 172.21.15.32:0/111162221 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d9c06b110 msgr2=0x7f0d9c06b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:23.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.017+0000 7f0da0fd2640 1 --2- 172.21.15.32:0/111162221 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d9c06b110 0x7f0d9c06b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:23.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.017+0000 7f0da0fd2640 1 -- 172.21.15.32:0/111162221 >> v2:172.21.15.121:3300/0 conn(0x7f0d9c0ba390 msgr2=0x7f0d9c0d40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:23.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.017+0000 7f0da0fd2640 1 --2- 172.21.15.32:0/111162221 >> v2:172.21.15.121:3300/0 conn(0x7f0d9c0ba390 0x7f0d9c0d40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:23.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.017+0000 7f0da0fd2640 1 -- 172.21.15.32:0/111162221 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d9c0d35d0 con 0x7f0d9c15e9e0 2024-04-24T23:06:23.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.017+0000 7f0da0fd2640 1 -- 172.21.15.32:0/111162221 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f0d94001670 con 0x7f0d9c15e9e0 2024-04-24T23:06:23.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.017+0000 7f0da0fd2640 1 -- 172.21.15.32:0/111162221 >> v1:172.21.15.32:6789/0 conn(0x7f0d9c15e9e0 legacy=0x7f0d9c0d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:23.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.017+0000 7f0da0fd2640 1 --2- 172.21.15.32:0/111162221 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d9c06b110 0x7f0d840039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:23.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.017+0000 7f0da0fd2640 1 --2- 172.21.15.32:0/111162221 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d84003f30 0x7f0d84004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:23.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.017+0000 7f0da0fd2640 1 --2- 172.21.15.32:0/111162221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d84004d40 0x7f0d840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:23.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.017+0000 7f0da0fd2640 1 -- 172.21.15.32:0/111162221 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c0d2db0 con 0x7f0d84004d40 2024-04-24T23:06:23.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.017+0000 7f0da0fd2640 1 -- 172.21.15.32:0/111162221 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c0d32f0 con 0x7f0d84003f30 2024-04-24T23:06:23.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.017+0000 7f0da0fd2640 1 -- 172.21.15.32:0/111162221 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f0d8400b9c0 con 0x7f0d9c06b110 2024-04-24T23:06:23.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.017+0000 7f0da27d5640 1 --2- 172.21.15.32:0/111162221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d84004d40 0x7f0d840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:23.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.017+0000 7f0da1fd4640 1 --2- 172.21.15.32:0/111162221 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d9c06b110 0x7f0d840039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:23.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.017+0000 7f0da17d3640 1 --2- 172.21.15.32:0/111162221 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d84003f30 0x7f0d84004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:23.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.017+0000 7f0da27d5640 1 -- 172.21.15.32:0/111162221 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d9c06b110 msgr2=0x7f0d840039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:23.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.017+0000 7f0da27d5640 1 --2- 172.21.15.32:0/111162221 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d9c06b110 0x7f0d840039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:23.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.017+0000 7f0da27d5640 1 -- 172.21.15.32:0/111162221 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d84003f30 msgr2=0x7f0d84004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:23.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.017+0000 7f0da27d5640 1 --2- 172.21.15.32:0/111162221 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d84003f30 0x7f0d84004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:23.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.017+0000 7f0da27d5640 1 -- 172.21.15.32:0/111162221 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d84003520 con 0x7f0d84004d40 2024-04-24T23:06:23.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.018+0000 7f0da27d5640 1 --2- 172.21.15.32:0/111162221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d84004d40 0x7f0d840051c0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c009fc0 tx=0x7f0d84005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a31665b01a85295a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:23.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.018+0000 7f0da0fd2640 1 -- 172.21.15.32:0/111162221 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c009c10 con 0x7f0d84004d40 2024-04-24T23:06:23.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.018+0000 7f0da0fd2640 1 -- 172.21.15.32:0/111162221 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d8c036070 con 0x7f0d84004d40 2024-04-24T23:06:23.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.018+0000 7f0da0fd2640 1 -- 172.21.15.32:0/111162221 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c031670 con 0x7f0d84004d40 2024-04-24T23:06:23.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.019+0000 7f0da4238640 1 -- 172.21.15.32:0/111162221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d84004d40 msgr2=0x7f0d840051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:23.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.019+0000 7f0da4238640 1 --2- 172.21.15.32:0/111162221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d84004d40 0x7f0d840051c0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c009fc0 tx=0x7f0d84005830 comp rx=0 tx=0).stop 2024-04-24T23:06:23.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.019+0000 7f0da27d5640 1 -- 172.21.15.32:0/111162221 reap_dead start 2024-04-24T23:06:23.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.019+0000 7f0da4238640 1 -- 172.21.15.32:0/111162221 shutdown_connections 2024-04-24T23:06:23.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.019+0000 7f0da4238640 1 -- 172.21.15.32:0/111162221 >> 172.21.15.32:0/111162221 conn(0x7f0d9c0d12e0 msgr2=0x7f0d9c0d16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:23.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.019+0000 7f0da4238640 1 -- 172.21.15.32:0/111162221 shutdown_connections 2024-04-24T23:06:23.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.019+0000 7f0da4238640 1 -- 172.21.15.32:0/111162221 wait complete. 2024-04-24T23:06:23.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.020+0000 7f0da4238640 1 Processor -- start 2024-04-24T23:06:23.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.020+0000 7f0da4238640 1 -- start start 2024-04-24T23:06:23.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.021+0000 7f0da4238640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d9c0ca300 0x7f0d9c14e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:23.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.021+0000 7f0da4238640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d9c14e9c0 0x7f0d9c1f42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:23.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.021+0000 7f0da4238640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d9c1f4820 0x7f0d9c160830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:23.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.021+0000 7f0da4238640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c06b5c0 con 0x7f0d9c14e9c0 2024-04-24T23:06:23.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.021+0000 7f0da4238640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c0d3b90 con 0x7f0d9c0ca300 2024-04-24T23:06:23.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.021+0000 7f0da4238640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f0d9c06b740 con 0x7f0d9c1f4820 2024-04-24T23:06:23.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.021+0000 7f0da17d3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d9c14e9c0 0x7f0d9c1f42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:23.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.021+0000 7f0da17d3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d9c14e9c0 0x7f0d9c1f42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53528/0 (socket says 172.21.15.32:53528) 2024-04-24T23:06:23.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.021+0000 7f0da17d3640 1 -- 172.21.15.32:0/2750901989 learned_addr learned my addr 172.21.15.32:0/2750901989 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:23.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.021+0000 7f0da27d5640 1 --2- 172.21.15.32:0/2750901989 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d9c1f4820 0x7f0d9c160830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:23.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.021+0000 7f0da1fd4640 1 --2- 172.21.15.32:0/2750901989 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d9c0ca300 0x7f0d9c14e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:23.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.021+0000 7f0da17d3640 1 -- 172.21.15.32:0/2750901989 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d9c1f4820 msgr2=0x7f0d9c160830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:23.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.021+0000 7f0da17d3640 1 --2- 172.21.15.32:0/2750901989 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d9c1f4820 0x7f0d9c160830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:23.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.021+0000 7f0da17d3640 1 -- 172.21.15.32:0/2750901989 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d9c0ca300 msgr2=0x7f0d9c14e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:23.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.021+0000 7f0da17d3640 1 --2- 172.21.15.32:0/2750901989 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d9c0ca300 0x7f0d9c14e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:23.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.021+0000 7f0da17d3640 1 -- 172.21.15.32:0/2750901989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d9c15fc20 con 0x7f0d9c14e9c0 2024-04-24T23:06:23.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.021+0000 7f0da17d3640 1 --2- 172.21.15.32:0/2750901989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d9c14e9c0 0x7f0d9c1f42e0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f0d90011960 tx=0x7f0d90011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:23.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.022+0000 7f0d9a7fc640 1 -- 172.21.15.32:0/2750901989 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9000fa90 con 0x7f0d9c14e9c0 2024-04-24T23:06:23.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.022+0000 7f0d9a7fc640 1 -- 172.21.15.32:0/2750901989 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d9000fc30 con 0x7f0d9c14e9c0 2024-04-24T23:06:23.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.022+0000 7f0da4238640 1 -- 172.21.15.32:0/2750901989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d9c15ee60 con 0x7f0d9c14e9c0 2024-04-24T23:06:23.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.022+0000 7f0d9a7fc640 1 -- 172.21.15.32:0/2750901989 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d90014440 con 0x7f0d9c14e9c0 2024-04-24T23:06:23.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.022+0000 7f0da4238640 1 -- 172.21.15.32:0/2750901989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d9c15f420 con 0x7f0d9c14e9c0 2024-04-24T23:06:23.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.023+0000 7f0da4238640 1 -- 172.21.15.32:0/2750901989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d9c068430 con 0x7f0d9c14e9c0 2024-04-24T23:06:23.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.023+0000 7f0d9a7fc640 1 -- 172.21.15.32:0/2750901989 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0d90012040 con 0x7f0d9c14e9c0 2024-04-24T23:06:23.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.023+0000 7f0d9a7fc640 1 --2- 172.21.15.32:0/2750901989 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0d8806be30 0x7f0d8806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:23.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.024+0000 7f0d9a7fc640 1 -- 172.21.15.32:0/2750901989 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4943+0+0 (secure 0 0 0) 0x7f0d9008b180 con 0x7f0d9c14e9c0 2024-04-24T23:06:23.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.024+0000 7f0da1fd4640 1 --2- 172.21.15.32:0/2750901989 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0d8806be30 0x7f0d8806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:23.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.024+0000 7f0da1fd4640 1 --2- 172.21.15.32:0/2750901989 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0d8806be30 0x7f0d8806e2f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c1531b0 tx=0x7f0d94009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:23.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.028+0000 7f0d9a7fc640 1 -- 172.21.15.32:0/2750901989 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0d90019030 con 0x7f0d9c14e9c0 2024-04-24T23:06:23.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.211+0000 7f0da4238640 1 -- 172.21.15.32:0/2750901989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f0d9c15c3a0 con 0x7f0d9c14e9c0 2024-04-24T23:06:23.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.214+0000 7f0d9a7fc640 1 -- 172.21.15.32:0/2750901989 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) ==== 104+0+0 (secure 0 0 0) 0x7f0d900597e0 con 0x7f0d9c14e9c0 2024-04-24T23:06:23.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.216+0000 7f0da4238640 1 -- 172.21.15.32:0/2750901989 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0d8806be30 msgr2=0x7f0d8806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:23.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.216+0000 7f0da4238640 1 --2- 172.21.15.32:0/2750901989 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0d8806be30 0x7f0d8806e2f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c1531b0 tx=0x7f0d94009040 comp rx=0 tx=0).stop 2024-04-24T23:06:23.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.216+0000 7f0da4238640 1 -- 172.21.15.32:0/2750901989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d9c14e9c0 msgr2=0x7f0d9c1f42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:23.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.216+0000 7f0da4238640 1 --2- 172.21.15.32:0/2750901989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d9c14e9c0 0x7f0d9c1f42e0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f0d90011960 tx=0x7f0d90011e20 comp rx=0 tx=0).stop 2024-04-24T23:06:23.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.216+0000 7f0da4238640 1 -- 172.21.15.32:0/2750901989 shutdown_connections 2024-04-24T23:06:23.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.216+0000 7f0da4238640 1 --2- 172.21.15.32:0/2750901989 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0d8806be30 0x7f0d8806e2f0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:23.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.216+0000 7f0da4238640 1 --2- 172.21.15.32:0/2750901989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d9c14e9c0 0x7f0d9c1f42e0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:23.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.216+0000 7f0da4238640 1 --2- 172.21.15.32:0/2750901989 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d9c1f4820 0x7f0d9c160830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:23.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.216+0000 7f0da4238640 1 --2- 172.21.15.32:0/2750901989 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d9c0ca300 0x7f0d9c14e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:23.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.216+0000 7f0da4238640 1 -- 172.21.15.32:0/2750901989 >> 172.21.15.32:0/2750901989 conn(0x7f0d9c0d12e0 msgr2=0x7f0d9c15b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:23.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.216+0000 7f0da4238640 1 -- 172.21.15.32:0/2750901989 shutdown_connections 2024-04-24T23:06:23.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.216+0000 7f0da4238640 1 -- 172.21.15.32:0/2750901989 wait complete. 2024-04-24T23:06:23.230 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file' 2024-04-24T23:06:23.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.328+0000 7f2353464640 1 Processor -- start 2024-04-24T23:06:23.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.328+0000 7f2353464640 1 -- start start 2024-04-24T23:06:23.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.329+0000 7f2353464640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f234c15e9b0 0x7f234c0d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:23.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.329+0000 7f2353464640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f234c06b820 0x7f234c0b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:23.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.329+0000 7f2353464640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f234c0d3170 con 0x7f234c0d4210 2024-04-24T23:06:23.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.329+0000 7f2353464640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f234c0d3450 con 0x7f234c06b820 2024-04-24T23:06:23.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.329+0000 7f2353464640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f234c0d3730 con 0x7f234c15e9b0 2024-04-24T23:06:23.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.329+0000 7f23509ff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f234c0d4210 0x7f234c06b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42732/0 (socket says 172.21.15.32:42732) 2024-04-24T23:06:23.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.329+0000 7f23509ff640 1 -- 172.21.15.32:0/423486222 learned_addr learned my addr 172.21.15.32:0/423486222 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:06:23.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.330+0000 7f2351a01640 1 --2- 172.21.15.32:0/423486222 >> v2:172.21.15.121:3300/0 conn(0x7f234c06b820 0x7f234c0b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:23.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.330+0000 7f2351200640 1 --2- 172.21.15.32:0/423486222 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f234c15e9b0 0x7f234c0d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:23.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.330+0000 7f2343fff640 1 -- 172.21.15.32:0/423486222 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2741844675 0 0) 0x7f234c0d3170 con 0x7f234c0d4210 2024-04-24T23:06:23.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.330+0000 7f2343fff640 1 -- 172.21.15.32:0/423486222 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2338003520 con 0x7f234c0d4210 2024-04-24T23:06:23.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.330+0000 7f2343fff640 1 -- 172.21.15.32:0/423486222 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3624657073 0 0) 0x7f2338003520 con 0x7f234c0d4210 2024-04-24T23:06:23.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.330+0000 7f2343fff640 1 -- 172.21.15.32:0/423486222 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f234c15e9b0 msgr2=0x7f234c0d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:23.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.330+0000 7f2343fff640 1 --2- 172.21.15.32:0/423486222 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f234c15e9b0 0x7f234c0d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:23.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.330+0000 7f2343fff640 1 -- 172.21.15.32:0/423486222 >> v2:172.21.15.121:3300/0 conn(0x7f234c06b820 msgr2=0x7f234c0b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:23.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.330+0000 7f2343fff640 1 --2- 172.21.15.32:0/423486222 >> v2:172.21.15.121:3300/0 conn(0x7f234c06b820 0x7f234c0b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:23.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.330+0000 7f2343fff640 1 -- 172.21.15.32:0/423486222 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f234c0d3a10 con 0x7f234c0d4210 2024-04-24T23:06:23.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.330+0000 7f2343fff640 1 -- 172.21.15.32:0/423486222 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f233c001670 con 0x7f234c0d4210 2024-04-24T23:06:23.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.330+0000 7f2351a01640 1 --2- 172.21.15.32:0/423486222 >> v2:172.21.15.121:3300/0 conn(0x7f234c06b820 0x7f234c0b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:06:23.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.330+0000 7f2343fff640 1 -- 172.21.15.32:0/423486222 >> v1:172.21.15.32:6789/0 conn(0x7f234c0d4210 legacy=0x7f234c06b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:23.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.330+0000 7f2343fff640 1 --2- 172.21.15.32:0/423486222 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f234c15e9b0 0x7f23380039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:23.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.330+0000 7f2343fff640 1 --2- 172.21.15.32:0/423486222 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2338003f80 0x7f2338004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:23.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.330+0000 7f2343fff640 1 --2- 172.21.15.32:0/423486222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2338004d40 0x7f23380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:23.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.330+0000 7f2343fff640 1 -- 172.21.15.32:0/423486222 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f234c0d3450 con 0x7f2338004d40 2024-04-24T23:06:23.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.330+0000 7f2343fff640 1 -- 172.21.15.32:0/423486222 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f234c0d3730 con 0x7f2338003f80 2024-04-24T23:06:23.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.330+0000 7f2343fff640 1 -- 172.21.15.32:0/423486222 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f233800b9c0 con 0x7f234c15e9b0 2024-04-24T23:06:23.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.330+0000 7f2351a01640 1 --2- 172.21.15.32:0/423486222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2338004d40 0x7f23380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:23.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.331+0000 7f2351200640 1 --2- 172.21.15.32:0/423486222 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f234c15e9b0 0x7f23380039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:23.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.331+0000 7f23509ff640 1 --2- 172.21.15.32:0/423486222 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2338003f80 0x7f2338004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:23.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.331+0000 7f2351a01640 1 -- 172.21.15.32:0/423486222 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f234c15e9b0 msgr2=0x7f23380039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:23.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.331+0000 7f2351a01640 1 --2- 172.21.15.32:0/423486222 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f234c15e9b0 0x7f23380039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:23.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.331+0000 7f2351a01640 1 -- 172.21.15.32:0/423486222 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2338003f80 msgr2=0x7f2338004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:23.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.331+0000 7f2351a01640 1 --2- 172.21.15.32:0/423486222 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2338003f80 0x7f2338004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:23.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.331+0000 7f2351a01640 1 -- 172.21.15.32:0/423486222 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2338003520 con 0x7f2338004d40 2024-04-24T23:06:23.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.331+0000 7f23509ff640 1 --2- 172.21.15.32:0/423486222 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2338003f80 0x7f2338004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:06:23.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.331+0000 7f2351a01640 1 --2- 172.21.15.32:0/423486222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2338004d40 0x7f23380051c0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f2344009c60 tx=0x7f2338005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aba740e7b491f6ae server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:23.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.332+0000 7f2343fff640 1 -- 172.21.15.32:0/423486222 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2344036070 con 0x7f2338004d40 2024-04-24T23:06:23.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.332+0000 7f2343fff640 1 -- 172.21.15.32:0/423486222 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2344031400 con 0x7f2338004d40 2024-04-24T23:06:23.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.332+0000 7f2343fff640 1 -- 172.21.15.32:0/423486222 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2344009860 con 0x7f2338004d40 2024-04-24T23:06:23.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.332+0000 7f2353464640 1 -- 172.21.15.32:0/423486222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2338004d40 msgr2=0x7f23380051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:23.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.332+0000 7f2353464640 1 --2- 172.21.15.32:0/423486222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2338004d40 0x7f23380051c0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f2344009c60 tx=0x7f2338005830 comp rx=0 tx=0).stop 2024-04-24T23:06:23.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.332+0000 7f2351a01640 1 -- 172.21.15.32:0/423486222 reap_dead start 2024-04-24T23:06:23.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.332+0000 7f2353464640 1 -- 172.21.15.32:0/423486222 shutdown_connections 2024-04-24T23:06:23.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.332+0000 7f2353464640 1 -- 172.21.15.32:0/423486222 >> 172.21.15.32:0/423486222 conn(0x7f234c0d11d0 msgr2=0x7f234c0d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:23.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.332+0000 7f2353464640 1 -- 172.21.15.32:0/423486222 shutdown_connections 2024-04-24T23:06:23.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.332+0000 7f2353464640 1 -- 172.21.15.32:0/423486222 wait complete. 2024-04-24T23:06:23.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.334+0000 7f2353464640 1 Processor -- start 2024-04-24T23:06:23.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.334+0000 7f2353464640 1 -- start start 2024-04-24T23:06:23.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.334+0000 7f2353464640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2338004d40 0x7f234c15ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:23.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.334+0000 7f2353464640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f234c0cdb10 0x7f234c160680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:23.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.334+0000 7f2353464640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f234c160bc0 0x7f234c1f4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:23.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.334+0000 7f2353464640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f234c0ba0e0 con 0x7f234c160bc0 2024-04-24T23:06:23.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.334+0000 7f2353464640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f234c0d2400 con 0x7f234c0cdb10 2024-04-24T23:06:23.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.334+0000 7f2353464640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f234c0ba260 con 0x7f2338004d40 2024-04-24T23:06:23.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.334+0000 7f2351a01640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f234c160bc0 0x7f234c1f4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:23.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.335+0000 7f2351a01640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f234c160bc0 0x7f234c1f4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53552/0 (socket says 172.21.15.32:53552) 2024-04-24T23:06:23.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.335+0000 7f2351a01640 1 -- 172.21.15.32:0/3524005337 learned_addr learned my addr 172.21.15.32:0/3524005337 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:23.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.335+0000 7f2351200640 1 --2- 172.21.15.32:0/3524005337 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2338004d40 0x7f234c15ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:23.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.335+0000 7f23509ff640 1 --2- 172.21.15.32:0/3524005337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f234c0cdb10 0x7f234c160680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:23.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.335+0000 7f2351a01640 1 -- 172.21.15.32:0/3524005337 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2338004d40 msgr2=0x7f234c15ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:23.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.335+0000 7f2351a01640 1 --2- 172.21.15.32:0/3524005337 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2338004d40 0x7f234c15ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:23.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.335+0000 7f2351a01640 1 -- 172.21.15.32:0/3524005337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f234c0cdb10 msgr2=0x7f234c160680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:23.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.335+0000 7f2351a01640 1 --2- 172.21.15.32:0/3524005337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f234c0cdb10 0x7f234c160680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:23.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.335+0000 7f2351a01640 1 -- 172.21.15.32:0/3524005337 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f234c15f5e0 con 0x7f234c160bc0 2024-04-24T23:06:23.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.335+0000 7f2351a01640 1 --2- 172.21.15.32:0/3524005337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f234c160bc0 0x7f234c1f4290 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f2344009d80 tx=0x7f2344004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:23.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.335+0000 7f2341ffb640 1 -- 172.21.15.32:0/3524005337 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2344036070 con 0x7f234c160bc0 2024-04-24T23:06:23.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.335+0000 7f2341ffb640 1 -- 172.21.15.32:0/3524005337 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2344041070 con 0x7f234c160bc0 2024-04-24T23:06:23.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.335+0000 7f2341ffb640 1 -- 172.21.15.32:0/3524005337 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2344009860 con 0x7f234c160bc0 2024-04-24T23:06:23.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.335+0000 7f2353464640 1 -- 172.21.15.32:0/3524005337 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f234c15f8c0 con 0x7f234c160bc0 2024-04-24T23:06:23.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.336+0000 7f2353464640 1 -- 172.21.15.32:0/3524005337 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f234c15fe80 con 0x7f234c160bc0 2024-04-24T23:06:23.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.336+0000 7f2353464640 1 -- 172.21.15.32:0/3524005337 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f234c068430 con 0x7f234c160bc0 2024-04-24T23:06:23.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.337+0000 7f2341ffb640 1 -- 172.21.15.32:0/3524005337 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f234403b840 con 0x7f234c160bc0 2024-04-24T23:06:23.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.337+0000 7f2341ffb640 1 --2- 172.21.15.32:0/3524005337 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f233006be30 0x7f233006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:23.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.337+0000 7f2341ffb640 1 -- 172.21.15.32:0/3524005337 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4943+0+0 (secure 0 0 0) 0x7f23440aac10 con 0x7f234c160bc0 2024-04-24T23:06:23.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.338+0000 7f2351200640 1 --2- 172.21.15.32:0/3524005337 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f233006be30 0x7f233006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:23.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.338+0000 7f2351200640 1 --2- 172.21.15.32:0/3524005337 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f233006be30 0x7f233006e2f0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f234c0ceb50 tx=0x7f234800b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:23.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.342+0000 7f2341ffb640 1 -- 172.21.15.32:0/3524005337 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2344079270 con 0x7f234c160bc0 2024-04-24T23:06:23.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.595+0000 7f2353464640 1 -- 172.21.15.32:0/3524005337 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]} v 0) -- 0x7f234c0d5370 con 0x7f234c160bc0 2024-04-24T23:06:23.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.897+0000 7f2341ffb640 1 -- 172.21.15.32:0/3524005337 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]}]=0 v385) ==== 140+0+0 (secure 0 0 0) 0x7f23440ae020 con 0x7f234c160bc0 2024-04-24T23:06:23.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.899+0000 7f2353464640 1 -- 172.21.15.32:0/3524005337 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f233006be30 msgr2=0x7f233006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:23.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.899+0000 7f2353464640 1 --2- 172.21.15.32:0/3524005337 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f233006be30 0x7f233006e2f0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f234c0ceb50 tx=0x7f234800b040 comp rx=0 tx=0).stop 2024-04-24T23:06:23.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.899+0000 7f2353464640 1 -- 172.21.15.32:0/3524005337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f234c160bc0 msgr2=0x7f234c1f4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:23.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.899+0000 7f2353464640 1 --2- 172.21.15.32:0/3524005337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f234c160bc0 0x7f234c1f4290 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f2344009d80 tx=0x7f2344004730 comp rx=0 tx=0).stop 2024-04-24T23:06:23.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.899+0000 7f2353464640 1 -- 172.21.15.32:0/3524005337 shutdown_connections 2024-04-24T23:06:23.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.899+0000 7f2353464640 1 --2- 172.21.15.32:0/3524005337 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f233006be30 0x7f233006e2f0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:23.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.899+0000 7f2353464640 1 --2- 172.21.15.32:0/3524005337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f234c160bc0 0x7f234c1f4290 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:23.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.899+0000 7f2353464640 1 --2- 172.21.15.32:0/3524005337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f234c0cdb10 0x7f234c160680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:23.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.899+0000 7f2353464640 1 --2- 172.21.15.32:0/3524005337 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2338004d40 0x7f234c15ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:23.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.899+0000 7f2353464640 1 -- 172.21.15.32:0/3524005337 >> 172.21.15.32:0/3524005337 conn(0x7f234c0d11d0 msgr2=0x7f234c15be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:23.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.899+0000 7f2353464640 1 -- 172.21.15.32:0/3524005337 shutdown_connections 2024-04-24T23:06:23.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:23.899+0000 7f2353464640 1 -- 172.21.15.32:0/3524005337 wait complete. 2024-04-24T23:06:23.913 INFO:tasks.cephfs_test_runner:test_full_different_file (tasks.cephfs.test_full.TestQuotaFull) ... ok 2024-04-24T23:06:23.913 INFO:tasks.cephfs_test_runner:Starting test: test_full_fclose (tasks.cephfs.test_full.TestQuotaFull) 2024-04-24T23:06:23.914 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:06:24.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.012+0000 7fa79411b640 1 Processor -- start 2024-04-24T23:06:24.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.012+0000 7fa79411b640 1 -- start start 2024-04-24T23:06:24.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.013+0000 7fa79411b640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa78c15fd60 0x7fa78c06c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:24.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.013+0000 7fa79411b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa78c0b46c0 con 0x7fa78c06cb30 2024-04-24T23:06:24.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.013+0000 7fa79411b640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa78c0b49a0 con 0x7fa78c15b480 2024-04-24T23:06:24.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.013+0000 7fa79411b640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa78c0b4ff0 con 0x7fa78c15fd60 2024-04-24T23:06:24.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.013+0000 7fa7926b8640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa78c06cb30 0x7fa78c0cdf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42748/0 (socket says 172.21.15.32:42748) 2024-04-24T23:06:24.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.013+0000 7fa7926b8640 1 -- 172.21.15.32:0/2792439980 learned_addr learned my addr 172.21.15.32:0/2792439980 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:06:24.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.014+0000 7fa7916b6640 1 --2- 172.21.15.32:0/2792439980 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa78c15fd60 0x7fa78c06c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:24.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.014+0000 7fa790eb5640 1 -- 172.21.15.32:0/2792439980 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1465766276 0 0) 0x7fa78c0b46c0 con 0x7fa78c06cb30 2024-04-24T23:06:24.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.014+0000 7fa790eb5640 1 -- 172.21.15.32:0/2792439980 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa778003520 con 0x7fa78c06cb30 2024-04-24T23:06:24.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.014+0000 7fa790eb5640 1 -- 172.21.15.32:0/2792439980 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1607943724 0 0) 0x7fa78c0b49a0 con 0x7fa78c15b480 2024-04-24T23:06:24.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.014+0000 7fa790eb5640 1 -- 172.21.15.32:0/2792439980 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa78c0b46c0 con 0x7fa78c15b480 2024-04-24T23:06:24.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.014+0000 7fa790eb5640 1 -- 172.21.15.32:0/2792439980 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2844857945 0 0) 0x7fa778003520 con 0x7fa78c06cb30 2024-04-24T23:06:24.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.014+0000 7fa790eb5640 1 -- 172.21.15.32:0/2792439980 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa78c15fd60 msgr2=0x7fa78c06c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:24.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.014+0000 7fa790eb5640 1 --2- 172.21.15.32:0/2792439980 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa78c15fd60 0x7fa78c06c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:24.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.014+0000 7fa790eb5640 1 -- 172.21.15.32:0/2792439980 >> v1:172.21.15.121:6789/0 conn(0x7fa78c15b480 legacy=0x7fa78c0b5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:24.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.014+0000 7fa790eb5640 1 -- 172.21.15.32:0/2792439980 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa78c0b4c80 con 0x7fa78c06cb30 2024-04-24T23:06:24.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.014+0000 7fa790eb5640 1 -- 172.21.15.32:0/2792439980 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7fa7880019f0 con 0x7fa78c06cb30 2024-04-24T23:06:24.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.014+0000 7fa7916b6640 1 --2- 172.21.15.32:0/2792439980 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa78c15fd60 0x7fa78c06c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:06:24.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.014+0000 7fa790eb5640 1 -- 172.21.15.32:0/2792439980 >> v1:172.21.15.32:6789/0 conn(0x7fa78c06cb30 legacy=0x7fa78c0cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:24.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.014+0000 7fa790eb5640 1 --2- 172.21.15.32:0/2792439980 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa778003b70 0x7fa778003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:24.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.014+0000 7fa790eb5640 1 --2- 172.21.15.32:0/2792439980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa778004800 0x7fa778004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:24.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.014+0000 7fa790eb5640 1 --2- 172.21.15.32:0/2792439980 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa78c15fd60 0x7fa7780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:24.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.014+0000 7fa790eb5640 1 -- 172.21.15.32:0/2792439980 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa78c0b4ff0 con 0x7fa778004800 2024-04-24T23:06:24.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.014+0000 7fa790eb5640 1 -- 172.21.15.32:0/2792439980 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa77800b9f0 con 0x7fa778003b70 2024-04-24T23:06:24.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.014+0000 7fa790eb5640 1 -- 172.21.15.32:0/2792439980 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa77800bbf0 con 0x7fa78c15fd60 2024-04-24T23:06:24.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.014+0000 7fa7916b6640 1 --2- 172.21.15.32:0/2792439980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa778004800 0x7fa778004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:24.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.014+0000 7fa791eb7640 1 --2- 172.21.15.32:0/2792439980 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa778003b70 0x7fa778003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:24.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.014+0000 7fa7926b8640 1 --2- 172.21.15.32:0/2792439980 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa78c15fd60 0x7fa7780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:24.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.014+0000 7fa7916b6640 1 -- 172.21.15.32:0/2792439980 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa78c15fd60 msgr2=0x7fa7780051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:24.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.014+0000 7fa7916b6640 1 --2- 172.21.15.32:0/2792439980 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa78c15fd60 0x7fa7780051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:24.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.014+0000 7fa7916b6640 1 -- 172.21.15.32:0/2792439980 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa778003b70 msgr2=0x7fa778003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:24.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.014+0000 7fa7916b6640 1 --2- 172.21.15.32:0/2792439980 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa778003b70 0x7fa778003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:24.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.014+0000 7fa7916b6640 1 -- 172.21.15.32:0/2792439980 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa778003520 con 0x7fa778004800 2024-04-24T23:06:24.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.015+0000 7fa7916b6640 1 --2- 172.21.15.32:0/2792439980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa778004800 0x7fa778004c80 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fa77c009c60 tx=0x7fa778005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d01b7aa7b7b890df server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:24.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.015+0000 7fa790eb5640 1 -- 172.21.15.32:0/2792439980 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77c036070 con 0x7fa778004800 2024-04-24T23:06:24.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.015+0000 7fa790eb5640 1 -- 172.21.15.32:0/2792439980 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa77c031400 con 0x7fa778004800 2024-04-24T23:06:24.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.015+0000 7fa790eb5640 1 -- 172.21.15.32:0/2792439980 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77c009860 con 0x7fa778004800 2024-04-24T23:06:24.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.016+0000 7fa79411b640 1 -- 172.21.15.32:0/2792439980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa778004800 msgr2=0x7fa778004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:24.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.016+0000 7fa79411b640 1 --2- 172.21.15.32:0/2792439980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa778004800 0x7fa778004c80 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fa77c009c60 tx=0x7fa778005830 comp rx=0 tx=0).stop 2024-04-24T23:06:24.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.016+0000 7fa7926b8640 1 -- 172.21.15.32:0/2792439980 reap_dead start 2024-04-24T23:06:24.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.016+0000 7fa79411b640 1 -- 172.21.15.32:0/2792439980 shutdown_connections 2024-04-24T23:06:24.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.016+0000 7fa79411b640 1 -- 172.21.15.32:0/2792439980 >> 172.21.15.32:0/2792439980 conn(0x7fa78c06b110 msgr2=0x7fa78c06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:24.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.016+0000 7fa79411b640 1 -- 172.21.15.32:0/2792439980 shutdown_connections 2024-04-24T23:06:24.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.016+0000 7fa79411b640 1 -- 172.21.15.32:0/2792439980 wait complete. 2024-04-24T23:06:24.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.017+0000 7fa79411b640 1 Processor -- start 2024-04-24T23:06:24.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.017+0000 7fa79411b640 1 -- start start 2024-04-24T23:06:24.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.018+0000 7fa79411b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa78c14e5d0 0x7fa78c1f10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:24.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.018+0000 7fa79411b640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa78c15c650 0x7fa78c071df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:24.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.018+0000 7fa79411b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa78c15cad0 0x7fa78c072330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:24.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.018+0000 7fa79411b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa78c15b860 con 0x7fa78c14e5d0 2024-04-24T23:06:24.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.018+0000 7fa79411b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa78c06c5f0 con 0x7fa78c15cad0 2024-04-24T23:06:24.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.018+0000 7fa79411b640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa78c15c1c0 con 0x7fa78c15c650 2024-04-24T23:06:24.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.018+0000 7fa791eb7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa78c14e5d0 0x7fa78c1f10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:24.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.018+0000 7fa791eb7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa78c14e5d0 0x7fa78c1f10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53564/0 (socket says 172.21.15.32:53564) 2024-04-24T23:06:24.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.018+0000 7fa791eb7640 1 -- 172.21.15.32:0/2715622373 learned_addr learned my addr 172.21.15.32:0/2715622373 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:24.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.018+0000 7fa7916b6640 1 --2- 172.21.15.32:0/2715622373 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa78c15c650 0x7fa78c071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:24.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.018+0000 7fa7926b8640 1 --2- 172.21.15.32:0/2715622373 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa78c15cad0 0x7fa78c072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:24.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.018+0000 7fa791eb7640 1 -- 172.21.15.32:0/2715622373 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa78c15c650 msgr2=0x7fa78c071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:24.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.018+0000 7fa791eb7640 1 --2- 172.21.15.32:0/2715622373 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa78c15c650 0x7fa78c071df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:24.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.018+0000 7fa791eb7640 1 -- 172.21.15.32:0/2715622373 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa78c15cad0 msgr2=0x7fa78c072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:24.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.018+0000 7fa791eb7640 1 --2- 172.21.15.32:0/2715622373 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa78c15cad0 0x7fa78c072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:24.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.018+0000 7fa791eb7640 1 -- 172.21.15.32:0/2715622373 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa78c1f1610 con 0x7fa78c14e5d0 2024-04-24T23:06:24.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.019+0000 7fa791eb7640 1 --2- 172.21.15.32:0/2715622373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa78c14e5d0 0x7fa78c1f10d0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fa78400f7a0 tx=0x7fa78400fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:24.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.019+0000 7fa7827fc640 1 -- 172.21.15.32:0/2715622373 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78400fe50 con 0x7fa78c14e5d0 2024-04-24T23:06:24.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.019+0000 7fa7827fc640 1 -- 172.21.15.32:0/2715622373 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa784018070 con 0x7fa78c14e5d0 2024-04-24T23:06:24.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.019+0000 7fa7827fc640 1 -- 172.21.15.32:0/2715622373 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78400d730 con 0x7fa78c14e5d0 2024-04-24T23:06:24.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.019+0000 7fa79411b640 1 -- 172.21.15.32:0/2715622373 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa78c14e9b0 con 0x7fa78c14e5d0 2024-04-24T23:06:24.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.019+0000 7fa79411b640 1 -- 172.21.15.32:0/2715622373 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa78c15dc60 con 0x7fa78c14e5d0 2024-04-24T23:06:24.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.020+0000 7fa79411b640 1 -- 172.21.15.32:0/2715622373 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa78c068430 con 0x7fa78c14e5d0 2024-04-24T23:06:24.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.020+0000 7fa7827fc640 1 -- 172.21.15.32:0/2715622373 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa78400d8d0 con 0x7fa78c14e5d0 2024-04-24T23:06:24.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.020+0000 7fa7827fc640 1 --2- 172.21.15.32:0/2715622373 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa77006be00 0x7fa77006e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:24.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.021+0000 7fa7827fc640 1 -- 172.21.15.32:0/2715622373 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4943+0+0 (secure 0 0 0) 0x7fa78408aba0 con 0x7fa78c14e5d0 2024-04-24T23:06:24.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.021+0000 7fa7916b6640 1 --2- 172.21.15.32:0/2715622373 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa77006be00 0x7fa77006e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:24.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.021+0000 7fa7916b6640 1 --2- 172.21.15.32:0/2715622373 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa77006be00 0x7fa77006e2c0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa77c033000 tx=0x7fa77c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:24.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.025+0000 7fa7827fc640 1 -- 172.21.15.32:0/2715622373 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa78c1f1610 con 0x7fa78c14e5d0 2024-04-24T23:06:24.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.210+0000 7fa79411b640 1 -- 172.21.15.32:0/2715622373 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa78c06c000 con 0x7fa78c14e5d0 2024-04-24T23:06:24.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.210+0000 7fa7827fc640 1 -- 172.21.15.32:0/2715622373 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) ==== 74+0+13043 (secure 0 0 0) 0x7fa78401a310 con 0x7fa78c14e5d0 2024-04-24T23:06:24.213 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:06:24.213 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":148,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:06:22.884892+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6835/4206823204":"2024-04-25T23:06:20.180990+0000","172.21.15.32:6834/4206823204":"2024-04-25T23:06:20.180990+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:06:24.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.213+0000 7fa79411b640 1 -- 172.21.15.32:0/2715622373 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa77006be00 msgr2=0x7fa77006e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:24.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.213+0000 7fa79411b640 1 --2- 172.21.15.32:0/2715622373 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa77006be00 0x7fa77006e2c0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa77c033000 tx=0x7fa77c008030 comp rx=0 tx=0).stop 2024-04-24T23:06:24.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.213+0000 7fa79411b640 1 -- 172.21.15.32:0/2715622373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa78c14e5d0 msgr2=0x7fa78c1f10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:24.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.213+0000 7fa79411b640 1 --2- 172.21.15.32:0/2715622373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa78c14e5d0 0x7fa78c1f10d0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fa78400f7a0 tx=0x7fa78400fc60 comp rx=0 tx=0).stop 2024-04-24T23:06:24.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.213+0000 7fa79411b640 1 -- 172.21.15.32:0/2715622373 shutdown_connections 2024-04-24T23:06:24.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.213+0000 7fa79411b640 1 --2- 172.21.15.32:0/2715622373 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa77006be00 0x7fa77006e2c0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:24.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.213+0000 7fa79411b640 1 --2- 172.21.15.32:0/2715622373 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa78c15cad0 0x7fa78c072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:24.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.213+0000 7fa79411b640 1 --2- 172.21.15.32:0/2715622373 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa78c15c650 0x7fa78c071df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:24.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.213+0000 7fa79411b640 1 --2- 172.21.15.32:0/2715622373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa78c14e5d0 0x7fa78c1f10d0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:24.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.213+0000 7fa79411b640 1 -- 172.21.15.32:0/2715622373 >> 172.21.15.32:0/2715622373 conn(0x7fa78c06b110 msgr2=0x7fa78c0b00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:24.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.213+0000 7fa79411b640 1 -- 172.21.15.32:0/2715622373 shutdown_connections 2024-04-24T23:06:24.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.213+0000 7fa79411b640 1 -- 172.21.15.32:0/2715622373 wait complete. 2024-04-24T23:06:24.227 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:06:24.227 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:06:24.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.327+0000 7f7579dd5640 1 Processor -- start 2024-04-24T23:06:24.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.328+0000 7f7579dd5640 1 -- start start 2024-04-24T23:06:24.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.328+0000 7f7579dd5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f757415c8a0 0x7f75740ae140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:24.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.328+0000 7f7579dd5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f75740ce7a0 0x7f757406c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:24.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.328+0000 7f7579dd5640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7574074210 0x7f75740746a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:24.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.328+0000 7f7579dd5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f757406ca70 con 0x7f757415c8a0 2024-04-24T23:06:24.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.328+0000 7f7579dd5640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f7574074f00 con 0x7f7574074210 2024-04-24T23:06:24.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.328+0000 7f7579dd5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f75740b0de0 con 0x7f75740ce7a0 2024-04-24T23:06:24.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.328+0000 7f75737fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f757415c8a0 0x7f75740ae140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:24.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.328+0000 7f75737fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f757415c8a0 0x7f75740ae140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53576/0 (socket says 172.21.15.32:53576) 2024-04-24T23:06:24.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.328+0000 7f75737fe640 1 -- 172.21.15.32:0/3154050818 learned_addr learned my addr 172.21.15.32:0/3154050818 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:24.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.329+0000 7f7572ffd640 1 --2- 172.21.15.32:0/3154050818 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f75740ce7a0 0x7f757406c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:24.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.329+0000 7f7573fff640 1 --2- 172.21.15.32:0/3154050818 >> v2:172.21.15.121:3300/0 conn(0x7f7574074210 0x7f75740746a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:24.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.329+0000 7f75737fe640 1 -- 172.21.15.32:0/3154050818 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f75740ce7a0 msgr2=0x7f757406c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:24.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.329+0000 7f75737fe640 1 --2- 172.21.15.32:0/3154050818 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f75740ce7a0 0x7f757406c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:24.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.329+0000 7f75737fe640 1 -- 172.21.15.32:0/3154050818 >> v2:172.21.15.121:3300/0 conn(0x7f7574074210 msgr2=0x7f75740746a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:24.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.329+0000 7f75737fe640 1 --2- 172.21.15.32:0/3154050818 >> v2:172.21.15.121:3300/0 conn(0x7f7574074210 0x7f75740746a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:24.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.329+0000 7f75737fe640 1 -- 172.21.15.32:0/3154050818 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75740b10c0 con 0x7f757415c8a0 2024-04-24T23:06:24.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.329+0000 7f7573fff640 1 --2- 172.21.15.32:0/3154050818 >> v2:172.21.15.121:3300/0 conn(0x7f7574074210 0x7f75740746a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:06:24.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.329+0000 7f7572ffd640 1 --2- 172.21.15.32:0/3154050818 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f75740ce7a0 0x7f757406c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:06:24.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.329+0000 7f75737fe640 1 --2- 172.21.15.32:0/3154050818 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f757415c8a0 0x7f75740ae140 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f75740cdc70 tx=0x7f756402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48c7ffed61c7d468 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:24.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.330+0000 7f75727fc640 1 -- 172.21.15.32:0/3154050818 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7564037070 con 0x7f757415c8a0 2024-04-24T23:06:24.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.330+0000 7f75727fc640 1 -- 172.21.15.32:0/3154050818 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7564032440 con 0x7f757415c8a0 2024-04-24T23:06:24.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.330+0000 7f75727fc640 1 -- 172.21.15.32:0/3154050818 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7564032810 con 0x7f757415c8a0 2024-04-24T23:06:24.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.330+0000 7f7579dd5640 1 -- 172.21.15.32:0/3154050818 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f757415c8a0 msgr2=0x7f75740ae140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:24.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.330+0000 7f7579dd5640 1 --2- 172.21.15.32:0/3154050818 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f757415c8a0 0x7f75740ae140 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f75740cdc70 tx=0x7f756402a140 comp rx=0 tx=0).stop 2024-04-24T23:06:24.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.330+0000 7f7579dd5640 1 -- 172.21.15.32:0/3154050818 shutdown_connections 2024-04-24T23:06:24.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.330+0000 7f7579dd5640 1 --2- 172.21.15.32:0/3154050818 >> v2:172.21.15.121:3300/0 conn(0x7f7574074210 0x7f75740746a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:24.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.330+0000 7f7579dd5640 1 --2- 172.21.15.32:0/3154050818 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f75740ce7a0 0x7f757406c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:24.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.330+0000 7f7579dd5640 1 --2- 172.21.15.32:0/3154050818 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f757415c8a0 0x7f75740ae140 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:24.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.330+0000 7f7579dd5640 1 -- 172.21.15.32:0/3154050818 >> 172.21.15.32:0/3154050818 conn(0x7f75740b2e60 msgr2=0x7f75740b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:24.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.330+0000 7f7579dd5640 1 -- 172.21.15.32:0/3154050818 shutdown_connections 2024-04-24T23:06:24.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.330+0000 7f7579dd5640 1 -- 172.21.15.32:0/3154050818 wait complete. 2024-04-24T23:06:24.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.331+0000 7f7579dd5640 1 Processor -- start 2024-04-24T23:06:24.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.332+0000 7f7579dd5640 1 -- start start 2024-04-24T23:06:24.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.332+0000 7f7579dd5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7574074210 0x7f75740d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:24.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.332+0000 7f7579dd5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75740ce7a0 0x7f7574157590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:24.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.332+0000 7f7579dd5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f757415c8a0 0x7f757415f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:24.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.332+0000 7f7579dd5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f757415cc80 con 0x7f75740ce7a0 2024-04-24T23:06:24.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.332+0000 7f7579dd5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f75740b1680 con 0x7f7574074210 2024-04-24T23:06:24.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.332+0000 7f7579dd5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f75740d1a50 con 0x7f757415c8a0 2024-04-24T23:06:24.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.332+0000 7f7572ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75740ce7a0 0x7f7574157590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:24.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.332+0000 7f7572ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75740ce7a0 0x7f7574157590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53580/0 (socket says 172.21.15.32:53580) 2024-04-24T23:06:24.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.332+0000 7f7572ffd640 1 -- 172.21.15.32:0/2765061595 learned_addr learned my addr 172.21.15.32:0/2765061595 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:24.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.333+0000 7f7573fff640 1 --2- 172.21.15.32:0/2765061595 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f757415c8a0 0x7f757415f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:24.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.333+0000 7f75737fe640 1 --2- 172.21.15.32:0/2765061595 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7574074210 0x7f75740d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:24.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.333+0000 7f7572ffd640 1 -- 172.21.15.32:0/2765061595 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f757415c8a0 msgr2=0x7f757415f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:24.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.333+0000 7f7572ffd640 1 --2- 172.21.15.32:0/2765061595 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f757415c8a0 0x7f757415f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:24.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.333+0000 7f7572ffd640 1 -- 172.21.15.32:0/2765061595 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7574074210 msgr2=0x7f75740d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:24.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.333+0000 7f7572ffd640 1 --2- 172.21.15.32:0/2765061595 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7574074210 0x7f75740d0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:24.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.333+0000 7f7572ffd640 1 -- 172.21.15.32:0/2765061595 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75740d1490 con 0x7f75740ce7a0 2024-04-24T23:06:24.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.333+0000 7f7572ffd640 1 --2- 172.21.15.32:0/2765061595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75740ce7a0 0x7f7574157590 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f756800e7e0 tx=0x7f756800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:24.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.333+0000 7f755bfff640 1 -- 172.21.15.32:0/2765061595 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756800f040 con 0x7f75740ce7a0 2024-04-24T23:06:24.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.333+0000 7f755bfff640 1 -- 172.21.15.32:0/2765061595 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7568004030 con 0x7f75740ce7a0 2024-04-24T23:06:24.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.333+0000 7f7579dd5640 1 -- 172.21.15.32:0/2765061595 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75740d1770 con 0x7f75740ce7a0 2024-04-24T23:06:24.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.333+0000 7f755bfff640 1 -- 172.21.15.32:0/2765061595 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7568004400 con 0x7f75740ce7a0 2024-04-24T23:06:24.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.333+0000 7f7579dd5640 1 -- 172.21.15.32:0/2765061595 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f757415f470 con 0x7f75740ce7a0 2024-04-24T23:06:24.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.334+0000 7f7579dd5640 1 -- 172.21.15.32:0/2765061595 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7574068430 con 0x7f75740ce7a0 2024-04-24T23:06:24.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.335+0000 7f755bfff640 1 -- 172.21.15.32:0/2765061595 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f756800c6d0 con 0x7f75740ce7a0 2024-04-24T23:06:24.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.335+0000 7f755bfff640 1 --2- 172.21.15.32:0/2765061595 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f756006be30 0x7f756006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:24.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.335+0000 7f755bfff640 1 -- 172.21.15.32:0/2765061595 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4943+0+0 (secure 0 0 0) 0x7f756808a850 con 0x7f75740ce7a0 2024-04-24T23:06:24.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.335+0000 7f75737fe640 1 --2- 172.21.15.32:0/2765061595 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f756006be30 0x7f756006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:24.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.336+0000 7f75737fe640 1 --2- 172.21.15.32:0/2765061595 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f756006be30 0x7f756006e2f0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f7564034000 tx=0x7f756402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:24.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.339+0000 7f755bfff640 1 -- 172.21.15.32:0/2765061595 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7568017020 con 0x7f75740ce7a0 2024-04-24T23:06:24.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.525+0000 7f7579dd5640 1 -- 172.21.15.32:0/2765061595 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f757406c350 con 0x7f75740ce7a0 2024-04-24T23:06:24.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.525+0000 7f755bfff640 1 -- 172.21.15.32:0/2765061595 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v148) ==== 93+0+10 (secure 0 0 0) 0x7f7568058f40 con 0x7f75740ce7a0 2024-04-24T23:06:24.528 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T23:06:24.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.527+0000 7f7579dd5640 1 -- 172.21.15.32:0/2765061595 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f756006be30 msgr2=0x7f756006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:24.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.527+0000 7f7579dd5640 1 --2- 172.21.15.32:0/2765061595 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f756006be30 0x7f756006e2f0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f7564034000 tx=0x7f756402e000 comp rx=0 tx=0).stop 2024-04-24T23:06:24.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.527+0000 7f7579dd5640 1 -- 172.21.15.32:0/2765061595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75740ce7a0 msgr2=0x7f7574157590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:24.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.527+0000 7f7579dd5640 1 --2- 172.21.15.32:0/2765061595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75740ce7a0 0x7f7574157590 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f756800e7e0 tx=0x7f756800eca0 comp rx=0 tx=0).stop 2024-04-24T23:06:24.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.527+0000 7f7579dd5640 1 -- 172.21.15.32:0/2765061595 shutdown_connections 2024-04-24T23:06:24.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.527+0000 7f7579dd5640 1 --2- 172.21.15.32:0/2765061595 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f756006be30 0x7f756006e2f0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:24.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.527+0000 7f7579dd5640 1 --2- 172.21.15.32:0/2765061595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75740ce7a0 0x7f7574157590 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:24.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.527+0000 7f7579dd5640 1 --2- 172.21.15.32:0/2765061595 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f757415c8a0 0x7f757415f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:24.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.527+0000 7f7579dd5640 1 --2- 172.21.15.32:0/2765061595 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7574074210 0x7f75740d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:24.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.527+0000 7f7579dd5640 1 -- 172.21.15.32:0/2765061595 >> 172.21.15.32:0/2765061595 conn(0x7f75740b2e60 msgr2=0x7f757406ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:24.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.527+0000 7f7579dd5640 1 -- 172.21.15.32:0/2765061595 shutdown_connections 2024-04-24T23:06:24.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.527+0000 7f7579dd5640 1 -- 172.21.15.32:0/2765061595 wait complete. 2024-04-24T23:06:24.543 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose' 2024-04-24T23:06:24.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.641+0000 7ff8a9e5d640 1 Processor -- start 2024-04-24T23:06:24.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.641+0000 7ff8a9e5d640 1 -- start start 2024-04-24T23:06:24.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.642+0000 7ff8a9e5d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff8a415b380 0x7ff8a406c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:24.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.642+0000 7ff8a9e5d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8a40cef50 con 0x7ff8a415cb00 2024-04-24T23:06:24.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.642+0000 7ff8a9e5d640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8a40b4520 con 0x7ff8a40ceb40 2024-04-24T23:06:24.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.642+0000 7ff8a9e5d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7ff8a40b4800 con 0x7ff8a415b380 2024-04-24T23:06:24.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.642+0000 7ff8a37fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff8a415b380 0x7ff8a406c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:24.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.642+0000 7ff8a37fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff8a415b380 0x7ff8a406c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53596/0 (socket says 172.21.15.32:53596) 2024-04-24T23:06:24.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.642+0000 7ff8a2ffd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff8a415cb00 0x7ff8a40ce430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42752/0 (socket says 172.21.15.32:42752) 2024-04-24T23:06:24.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.642+0000 7ff8a37fe640 1 -- 172.21.15.32:0/3558951287 learned_addr learned my addr 172.21.15.32:0/3558951287 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:24.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.643+0000 7ff8a37fe640 1 -- 172.21.15.32:0/3558951287 >> v1:172.21.15.121:6789/0 conn(0x7ff8a40ceb40 legacy=0x7ff8a40b55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:24.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.643+0000 7ff8a37fe640 1 -- 172.21.15.32:0/3558951287 >> v1:172.21.15.32:6789/0 conn(0x7ff8a415cb00 legacy=0x7ff8a40ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:24.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.643+0000 7ff8a37fe640 1 -- 172.21.15.32:0/3558951287 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8a40b4ae0 con 0x7ff8a415b380 2024-04-24T23:06:24.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.643+0000 7ff8a37fe640 1 --2- 172.21.15.32:0/3558951287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8a415b380 0x7ff8a406c5f0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7ff89400bb10 tx=0x7ff89402c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8616daa4747fb63 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:24.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.644+0000 7ff8a27fc640 1 -- 172.21.15.32:0/3558951287 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff894032710 con 0x7ff8a415b380 2024-04-24T23:06:24.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.644+0000 7ff8a27fc640 1 -- 172.21.15.32:0/3558951287 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8940328b0 con 0x7ff8a415b380 2024-04-24T23:06:24.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.644+0000 7ff8a27fc640 1 -- 172.21.15.32:0/3558951287 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff894032ca0 con 0x7ff8a415b380 2024-04-24T23:06:24.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.644+0000 7ff8a9e5d640 1 -- 172.21.15.32:0/3558951287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8a415b380 msgr2=0x7ff8a406c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:24.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.644+0000 7ff8a9e5d640 1 --2- 172.21.15.32:0/3558951287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8a415b380 0x7ff8a406c5f0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7ff89400bb10 tx=0x7ff89402c850 comp rx=0 tx=0).stop 2024-04-24T23:06:24.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.644+0000 7ff8a9e5d640 1 -- 172.21.15.32:0/3558951287 shutdown_connections 2024-04-24T23:06:24.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.644+0000 7ff8a9e5d640 1 --2- 172.21.15.32:0/3558951287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8a415b380 0x7ff8a406c5f0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:24.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.644+0000 7ff8a9e5d640 1 -- 172.21.15.32:0/3558951287 >> 172.21.15.32:0/3558951287 conn(0x7ff8a406b110 msgr2=0x7ff8a406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:24.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.644+0000 7ff8a9e5d640 1 -- 172.21.15.32:0/3558951287 shutdown_connections 2024-04-24T23:06:24.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.644+0000 7ff8a9e5d640 1 -- 172.21.15.32:0/3558951287 wait complete. 2024-04-24T23:06:24.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.645+0000 7ff8a9e5d640 1 Processor -- start 2024-04-24T23:06:24.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.646+0000 7ff8a9e5d640 1 -- start start 2024-04-24T23:06:24.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.646+0000 7ff8a9e5d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8a40ceb40 0x7ff8a4148250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:24.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.646+0000 7ff8a9e5d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8a415b380 0x7ff8a4155ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:24.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.646+0000 7ff8a9e5d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff8a415cb00 0x7ff8a4156530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:24.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.646+0000 7ff8a9e5d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a415b760 con 0x7ff8a415b380 2024-04-24T23:06:24.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.646+0000 7ff8a9e5d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a40b50a0 con 0x7ff8a40ceb40 2024-04-24T23:06:24.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.646+0000 7ff8a9e5d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff8a406cb30 con 0x7ff8a415cb00 2024-04-24T23:06:24.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.646+0000 7ff8a2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8a415b380 0x7ff8a4155ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:24.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.646+0000 7ff8a2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8a415b380 0x7ff8a4155ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53612/0 (socket says 172.21.15.32:53612) 2024-04-24T23:06:24.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.646+0000 7ff8a2ffd640 1 -- 172.21.15.32:0/1766948049 learned_addr learned my addr 172.21.15.32:0/1766948049 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:24.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.647+0000 7ff8a37fe640 1 --2- 172.21.15.32:0/1766948049 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8a40ceb40 0x7ff8a4148250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:24.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.647+0000 7ff8a3fff640 1 --2- 172.21.15.32:0/1766948049 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff8a415cb00 0x7ff8a4156530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:24.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.647+0000 7ff8a2ffd640 1 -- 172.21.15.32:0/1766948049 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff8a415cb00 msgr2=0x7ff8a4156530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:24.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.647+0000 7ff8a2ffd640 1 --2- 172.21.15.32:0/1766948049 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff8a415cb00 0x7ff8a4156530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:24.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.647+0000 7ff8a2ffd640 1 -- 172.21.15.32:0/1766948049 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8a40ceb40 msgr2=0x7ff8a4148250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:24.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.647+0000 7ff8a2ffd640 1 --2- 172.21.15.32:0/1766948049 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8a40ceb40 0x7ff8a4148250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:24.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.647+0000 7ff8a2ffd640 1 -- 172.21.15.32:0/1766948049 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8a4072330 con 0x7ff8a415b380 2024-04-24T23:06:24.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.647+0000 7ff8a3fff640 1 --2- 172.21.15.32:0/1766948049 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff8a415cb00 0x7ff8a4156530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:06:24.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.647+0000 7ff8a2ffd640 1 --2- 172.21.15.32:0/1766948049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8a415b380 0x7ff8a4155ff0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7ff89800c3c0 tx=0x7ff89800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:24.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.647+0000 7ff88bfff640 1 -- 172.21.15.32:0/1766948049 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8980074e0 con 0x7ff8a415b380 2024-04-24T23:06:24.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.647+0000 7ff88bfff640 1 -- 172.21.15.32:0/1766948049 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff898007680 con 0x7ff8a415b380 2024-04-24T23:06:24.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.647+0000 7ff8a9e5d640 1 -- 172.21.15.32:0/1766948049 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8a4071290 con 0x7ff8a415b380 2024-04-24T23:06:24.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.647+0000 7ff88bfff640 1 -- 172.21.15.32:0/1766948049 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff898004a00 con 0x7ff8a415b380 2024-04-24T23:06:24.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.647+0000 7ff8a9e5d640 1 -- 172.21.15.32:0/1766948049 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8a4071850 con 0x7ff8a415b380 2024-04-24T23:06:24.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.648+0000 7ff8a9e5d640 1 -- 172.21.15.32:0/1766948049 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8a4068430 con 0x7ff8a415b380 2024-04-24T23:06:24.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.648+0000 7ff88bfff640 1 -- 172.21.15.32:0/1766948049 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff89801a020 con 0x7ff8a415b380 2024-04-24T23:06:24.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.649+0000 7ff88bfff640 1 --2- 172.21.15.32:0/1766948049 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff89006be30 0x7ff89006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:24.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.649+0000 7ff88bfff640 1 -- 172.21.15.32:0/1766948049 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4943+0+0 (secure 0 0 0) 0x7ff89808a250 con 0x7ff8a415b380 2024-04-24T23:06:24.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.649+0000 7ff8a37fe640 1 --2- 172.21.15.32:0/1766948049 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff89006be30 0x7ff89006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:24.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.650+0000 7ff8a37fe640 1 --2- 172.21.15.32:0/1766948049 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff89006be30 0x7ff89006e2f0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7ff894037000 tx=0x7ff894008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:24.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.653+0000 7ff88bfff640 1 -- 172.21.15.32:0/1766948049 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff8a40cef50 con 0x7ff8a415b380 2024-04-24T23:06:24.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:24.911+0000 7ff8a9e5d640 1 -- 172.21.15.32:0/1766948049 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]} v 0) -- 0x7ff8a406c000 con 0x7ff8a415b380 2024-04-24T23:06:25.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:25.902+0000 7ff88bfff640 1 -- 172.21.15.32:0/1766948049 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]}]=0 v387) ==== 135+0+0 (secure 0 0 0) 0x7ff898058f90 con 0x7ff8a415b380 2024-04-24T23:06:25.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:25.905+0000 7ff8a9e5d640 1 -- 172.21.15.32:0/1766948049 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff89006be30 msgr2=0x7ff89006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:25.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:25.905+0000 7ff8a9e5d640 1 --2- 172.21.15.32:0/1766948049 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff89006be30 0x7ff89006e2f0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7ff894037000 tx=0x7ff894008040 comp rx=0 tx=0).stop 2024-04-24T23:06:25.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:25.905+0000 7ff8a9e5d640 1 -- 172.21.15.32:0/1766948049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8a415b380 msgr2=0x7ff8a4155ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:25.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:25.905+0000 7ff8a9e5d640 1 --2- 172.21.15.32:0/1766948049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8a415b380 0x7ff8a4155ff0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7ff89800c3c0 tx=0x7ff89800c880 comp rx=0 tx=0).stop 2024-04-24T23:06:25.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:25.905+0000 7ff8a9e5d640 1 -- 172.21.15.32:0/1766948049 shutdown_connections 2024-04-24T23:06:25.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:25.905+0000 7ff8a9e5d640 1 --2- 172.21.15.32:0/1766948049 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff89006be30 0x7ff89006e2f0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:25.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:25.905+0000 7ff8a9e5d640 1 --2- 172.21.15.32:0/1766948049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8a415b380 0x7ff8a4155ff0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:25.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:25.905+0000 7ff8a9e5d640 1 --2- 172.21.15.32:0/1766948049 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff8a415cb00 0x7ff8a4156530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:25.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:25.905+0000 7ff8a9e5d640 1 --2- 172.21.15.32:0/1766948049 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8a40ceb40 0x7ff8a4148250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:25.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:25.905+0000 7ff8a9e5d640 1 -- 172.21.15.32:0/1766948049 >> 172.21.15.32:0/1766948049 conn(0x7ff8a406b110 msgr2=0x7ff8a415dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:25.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:25.905+0000 7ff8a9e5d640 1 -- 172.21.15.32:0/1766948049 shutdown_connections 2024-04-24T23:06:25.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:25.905+0000 7ff8a9e5d640 1 -- 172.21.15.32:0/1766948049 wait complete. 2024-04-24T23:06:25.921 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-24T23:06:26.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.029+0000 7f2534c27640 1 Processor -- start 2024-04-24T23:06:26.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.029+0000 7f2534c27640 1 -- start start 2024-04-24T23:06:26.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.030+0000 7f2534c27640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f25300af890 0x7f25301670f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:26.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.030+0000 7f2534c27640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25300b6430 con 0x7f25300b44d0 2024-04-24T23:06:26.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.030+0000 7f2534c27640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2530156f80 con 0x7f253006c5f0 2024-04-24T23:06:26.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.030+0000 7f2534c27640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f25300b6150 con 0x7f25300af890 2024-04-24T23:06:26.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.030+0000 7f2527fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f25300b44d0 0x7f25300b5890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42756/0 (socket says 172.21.15.32:42756) 2024-04-24T23:06:26.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.030+0000 7f2527fff640 1 -- 172.21.15.32:0/3295462820 learned_addr learned my addr 172.21.15.32:0/3295462820 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:06:26.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.030+0000 7f252ed9d640 1 --2- 172.21.15.32:0/3295462820 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f25300af890 0x7f25301670f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:26.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.030+0000 7f252dd9b640 1 -- 172.21.15.32:0/3295462820 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2479891469 0 0) 0x7f25300b6430 con 0x7f25300b44d0 2024-04-24T23:06:26.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.030+0000 7f252dd9b640 1 -- 172.21.15.32:0/3295462820 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f251c003520 con 0x7f25300b44d0 2024-04-24T23:06:26.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.030+0000 7f252dd9b640 1 -- 172.21.15.32:0/3295462820 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 687882707 0 0) 0x7f2530156f80 con 0x7f253006c5f0 2024-04-24T23:06:26.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.030+0000 7f252dd9b640 1 -- 172.21.15.32:0/3295462820 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f25300b6430 con 0x7f253006c5f0 2024-04-24T23:06:26.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.030+0000 7f252dd9b640 1 -- 172.21.15.32:0/3295462820 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1090767107 0 0) 0x7f251c003520 con 0x7f25300b44d0 2024-04-24T23:06:26.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.030+0000 7f252dd9b640 1 -- 172.21.15.32:0/3295462820 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f25300af890 msgr2=0x7f25301670f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:26.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.030+0000 7f252dd9b640 1 --2- 172.21.15.32:0/3295462820 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f25300af890 0x7f25301670f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:26.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.030+0000 7f252dd9b640 1 -- 172.21.15.32:0/3295462820 >> v1:172.21.15.121:6789/0 conn(0x7f253006c5f0 legacy=0x7f253006c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:26.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.030+0000 7f252dd9b640 1 -- 172.21.15.32:0/3295462820 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25300b39f0 con 0x7f25300b44d0 2024-04-24T23:06:26.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.030+0000 7f252dd9b640 1 -- 172.21.15.32:0/3295462820 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f2518001690 con 0x7f25300b44d0 2024-04-24T23:06:26.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.030+0000 7f252dd9b640 1 -- 172.21.15.32:0/3295462820 >> v1:172.21.15.32:6789/0 conn(0x7f25300b44d0 legacy=0x7f25300b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:26.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.030+0000 7f252dd9b640 1 --2- 172.21.15.32:0/3295462820 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f251c003b70 0x7f251c003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:26.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.030+0000 7f252dd9b640 1 --2- 172.21.15.32:0/3295462820 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f25300af890 0x7f251c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:26.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.030+0000 7f252dd9b640 1 --2- 172.21.15.32:0/3295462820 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f251c004d40 0x7f251c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:26.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.030+0000 7f252dd9b640 1 -- 172.21.15.32:0/3295462820 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f25300b6150 con 0x7f251c003b70 2024-04-24T23:06:26.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.030+0000 7f252dd9b640 1 -- 172.21.15.32:0/3295462820 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f251c00bb50 con 0x7f251c004d40 2024-04-24T23:06:26.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.030+0000 7f252dd9b640 1 -- 172.21.15.32:0/3295462820 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f251c00bd20 con 0x7f25300af890 2024-04-24T23:06:26.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.031+0000 7f252e59c640 1 --2- 172.21.15.32:0/3295462820 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f251c003b70 0x7f251c003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:26.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.031+0000 7f2527fff640 1 --2- 172.21.15.32:0/3295462820 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f25300af890 0x7f251c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:26.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.031+0000 7f252ed9d640 1 --2- 172.21.15.32:0/3295462820 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f251c004d40 0x7f251c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:26.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.031+0000 7f252e59c640 1 -- 172.21.15.32:0/3295462820 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f25300af890 msgr2=0x7f251c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:26.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.031+0000 7f252e59c640 1 --2- 172.21.15.32:0/3295462820 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f25300af890 0x7f251c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:26.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.031+0000 7f252e59c640 1 -- 172.21.15.32:0/3295462820 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f251c004d40 msgr2=0x7f251c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:26.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.031+0000 7f252e59c640 1 --2- 172.21.15.32:0/3295462820 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f251c004d40 0x7f251c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:26.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.031+0000 7f252e59c640 1 -- 172.21.15.32:0/3295462820 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f251c003520 con 0x7f251c003b70 2024-04-24T23:06:26.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.032+0000 7f252e59c640 1 --2- 172.21.15.32:0/3295462820 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f251c003b70 0x7f251c003f70 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f2520001900 tx=0x7f251c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae96778a76ad2dcf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:26.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.032+0000 7f252dd9b640 1 -- 172.21.15.32:0/3295462820 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25200014f0 con 0x7f251c003b70 2024-04-24T23:06:26.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.032+0000 7f252dd9b640 1 -- 172.21.15.32:0/3295462820 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25200052d0 con 0x7f251c003b70 2024-04-24T23:06:26.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.032+0000 7f252dd9b640 1 -- 172.21.15.32:0/3295462820 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252002ac80 con 0x7f251c003b70 2024-04-24T23:06:26.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.032+0000 7f2534c27640 1 -- 172.21.15.32:0/3295462820 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f251c003b70 msgr2=0x7f251c003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:26.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.032+0000 7f2534c27640 1 --2- 172.21.15.32:0/3295462820 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f251c003b70 0x7f251c003f70 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f2520001900 tx=0x7f251c005830 comp rx=0 tx=0).stop 2024-04-24T23:06:26.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.032+0000 7f252ed9d640 1 -- 172.21.15.32:0/3295462820 reap_dead start 2024-04-24T23:06:26.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.032+0000 7f2534c27640 1 -- 172.21.15.32:0/3295462820 shutdown_connections 2024-04-24T23:06:26.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.032+0000 7f2534c27640 1 -- 172.21.15.32:0/3295462820 >> 172.21.15.32:0/3295462820 conn(0x7f253006b110 msgr2=0x7f253006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:26.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.032+0000 7f2534c27640 1 -- 172.21.15.32:0/3295462820 shutdown_connections 2024-04-24T23:06:26.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.032+0000 7f2534c27640 1 -- 172.21.15.32:0/3295462820 wait complete. 2024-04-24T23:06:26.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.034+0000 7f2534c27640 1 Processor -- start 2024-04-24T23:06:26.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.034+0000 7f2534c27640 1 -- start start 2024-04-24T23:06:26.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.034+0000 7f2534c27640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f251c003b70 0x7f2530150650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:26.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.034+0000 7f2534c27640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25300b1260 0x7f25300ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:26.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.034+0000 7f2534c27640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f25300aff60 0x7f25300b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:26.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.034+0000 7f2534c27640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f25301650e0 con 0x7f251c003b70 2024-04-24T23:06:26.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.034+0000 7f2534c27640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f25300b3fb0 con 0x7f25300b1260 2024-04-24T23:06:26.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.034+0000 7f2534c27640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f2530165260 con 0x7f25300aff60 2024-04-24T23:06:26.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.034+0000 7f252e59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f251c003b70 0x7f2530150650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:26.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.035+0000 7f252e59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f251c003b70 0x7f2530150650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53626/0 (socket says 172.21.15.32:53626) 2024-04-24T23:06:26.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.035+0000 7f252e59c640 1 -- 172.21.15.32:0/1810350650 learned_addr learned my addr 172.21.15.32:0/1810350650 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:26.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.035+0000 7f2527fff640 1 --2- 172.21.15.32:0/1810350650 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25300b1260 0x7f25300ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:26.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.035+0000 7f252ed9d640 1 --2- 172.21.15.32:0/1810350650 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f25300aff60 0x7f25300b03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:26.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.035+0000 7f252e59c640 1 -- 172.21.15.32:0/1810350650 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f25300aff60 msgr2=0x7f25300b03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:26.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.035+0000 7f252e59c640 1 --2- 172.21.15.32:0/1810350650 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f25300aff60 0x7f25300b03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:26.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.035+0000 7f252e59c640 1 -- 172.21.15.32:0/1810350650 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25300b1260 msgr2=0x7f25300ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:26.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.035+0000 7f252e59c640 1 --2- 172.21.15.32:0/1810350650 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25300b1260 0x7f25300ceeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:26.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.035+0000 7f252e59c640 1 -- 172.21.15.32:0/1810350650 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2530150f30 con 0x7f251c003b70 2024-04-24T23:06:26.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.035+0000 7f252e59c640 1 --2- 172.21.15.32:0/1810350650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f251c003b70 0x7f2530150650 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f2520005040 tx=0x7f25200012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:26.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.035+0000 7f2526ffd640 1 -- 172.21.15.32:0/1810350650 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2520040020 con 0x7f251c003b70 2024-04-24T23:06:26.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.035+0000 7f2526ffd640 1 -- 172.21.15.32:0/1810350650 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2520005480 con 0x7f251c003b70 2024-04-24T23:06:26.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.035+0000 7f2534c27640 1 -- 172.21.15.32:0/1810350650 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2530151210 con 0x7f251c003b70 2024-04-24T23:06:26.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.035+0000 7f2526ffd640 1 -- 172.21.15.32:0/1810350650 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25200014f0 con 0x7f251c003b70 2024-04-24T23:06:26.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.035+0000 7f2534c27640 1 -- 172.21.15.32:0/1810350650 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25301517d0 con 0x7f251c003b70 2024-04-24T23:06:26.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.036+0000 7f2534c27640 1 -- 172.21.15.32:0/1810350650 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2530068430 con 0x7f251c003b70 2024-04-24T23:06:26.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.037+0000 7f2526ffd640 1 -- 172.21.15.32:0/1810350650 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2520043020 con 0x7f251c003b70 2024-04-24T23:06:26.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.037+0000 7f2526ffd640 1 --2- 172.21.15.32:0/1810350650 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f251006be60 0x7f251006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:26.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.037+0000 7f2526ffd640 1 -- 172.21.15.32:0/1810350650 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4943+0+0 (secure 0 0 0) 0x7f25200aac50 con 0x7f251c003b70 2024-04-24T23:06:26.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.037+0000 7f2527fff640 1 --2- 172.21.15.32:0/1810350650 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f251006be60 0x7f251006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:26.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.038+0000 7f2527fff640 1 --2- 172.21.15.32:0/1810350650 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f251006be60 0x7f251006e320 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f25300b11f0 tx=0x7f2518009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:26.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.042+0000 7f2526ffd640 1 -- 172.21.15.32:0/1810350650 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f25200b0020 con 0x7f251c003b70 2024-04-24T23:06:26.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.225+0000 7f2534c27640 1 -- 172.21.15.32:0/1810350650 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f25300b5670 con 0x7f251c003b70 2024-04-24T23:06:26.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.228+0000 7f2526ffd640 1 -- 172.21.15.32:0/1810350650 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7f252004a030 con 0x7f251c003b70 2024-04-24T23:06:26.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.231+0000 7f2534c27640 1 -- 172.21.15.32:0/1810350650 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f251006be60 msgr2=0x7f251006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:26.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.231+0000 7f2534c27640 1 --2- 172.21.15.32:0/1810350650 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f251006be60 0x7f251006e320 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f25300b11f0 tx=0x7f2518009040 comp rx=0 tx=0).stop 2024-04-24T23:06:26.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.231+0000 7f2534c27640 1 -- 172.21.15.32:0/1810350650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f251c003b70 msgr2=0x7f2530150650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:26.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.231+0000 7f2534c27640 1 --2- 172.21.15.32:0/1810350650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f251c003b70 0x7f2530150650 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f2520005040 tx=0x7f25200012d0 comp rx=0 tx=0).stop 2024-04-24T23:06:26.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.231+0000 7f2534c27640 1 -- 172.21.15.32:0/1810350650 shutdown_connections 2024-04-24T23:06:26.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.231+0000 7f2534c27640 1 --2- 172.21.15.32:0/1810350650 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f251006be60 0x7f251006e320 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:26.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.231+0000 7f2534c27640 1 --2- 172.21.15.32:0/1810350650 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f25300aff60 0x7f25300b03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:26.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.231+0000 7f2534c27640 1 --2- 172.21.15.32:0/1810350650 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25300b1260 0x7f25300ceeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:26.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.231+0000 7f2534c27640 1 --2- 172.21.15.32:0/1810350650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f251c003b70 0x7f2530150650 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:26.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.231+0000 7f2534c27640 1 -- 172.21.15.32:0/1810350650 >> 172.21.15.32:0/1810350650 conn(0x7f253006b110 msgr2=0x7f253015b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:26.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.231+0000 7f2534c27640 1 -- 172.21.15.32:0/1810350650 shutdown_connections 2024-04-24T23:06:26.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.231+0000 7f2534c27640 1 -- 172.21.15.32:0/1810350650 wait complete. 2024-04-24T23:06:26.244 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-24T23:06:26.245 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:06:26.289 DEBUG:teuthology.orchestra.run.smithi121:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:06:26.344 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-24T23:06:26.344 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:06:26.344 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:06:26.360 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=413 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T23:06:26.360 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:06:26.360 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:06:26.380 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3271 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T23:06:26.381 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:06:26.381 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:06:26.416 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=413 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T23:06:26.417 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:06:26.417 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:06:26.437 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3271 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T23:06:26.437 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-24T23:06:26.438 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:06:26.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.555+0000 7f3dadf29640 1 Processor -- start 2024-04-24T23:06:26.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.555+0000 7f3dadf29640 1 -- start start 2024-04-24T23:06:26.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.555+0000 7f3dadf29640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3da815e9b0 0x7f3da80d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:26.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.555+0000 7f3dadf29640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3da806b820 0x7f3da80b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:26.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.555+0000 7f3dadf29640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3da80d3170 con 0x7f3da80d4210 2024-04-24T23:06:26.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.555+0000 7f3dadf29640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f3da80d3450 con 0x7f3da815e9b0 2024-04-24T23:06:26.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.555+0000 7f3dadf29640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f3da80d3730 con 0x7f3da806b820 2024-04-24T23:06:26.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.556+0000 7f3da7fff640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3da806b820 0x7f3da80b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:26.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.556+0000 7f3da77fe640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3da815e9b0 0x7f3da80d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:26.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.556+0000 7f3da6ffd640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f3da80d4210 0x7f3da806b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:45360/0 (socket says 172.21.15.32:45360) 2024-04-24T23:06:26.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.556+0000 7f3da6ffd640 1 -- 172.21.15.32:0/726131212 learned_addr learned my addr 172.21.15.32:0/726131212 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:06:26.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.556+0000 7f3da77fe640 1 -- 172.21.15.32:0/726131212 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3da806b820 msgr2=0x7f3da80b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:26.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.556+0000 7f3da77fe640 1 --2- 172.21.15.32:0/726131212 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3da806b820 0x7f3da80b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:26.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.556+0000 7f3da77fe640 1 -- 172.21.15.32:0/726131212 >> v1:172.21.15.121:6789/0 conn(0x7f3da80d4210 legacy=0x7f3da806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:26.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.556+0000 7f3da77fe640 1 -- 172.21.15.32:0/726131212 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3da80d3a10 con 0x7f3da815e9b0 2024-04-24T23:06:26.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.556+0000 7f3da7fff640 1 --2- 172.21.15.32:0/726131212 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3da806b820 0x7f3da80b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:06:26.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.557+0000 7f3da77fe640 1 --2- 172.21.15.32:0/726131212 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3da815e9b0 0x7f3da80d2b90 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3d980097a0 tx=0x7f3d9802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4179d7d6c6ac7143 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:26.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.557+0000 7f3da67fc640 1 -- 172.21.15.32:0/726131212 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d98037070 con 0x7f3da815e9b0 2024-04-24T23:06:26.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.557+0000 7f3da67fc640 1 -- 172.21.15.32:0/726131212 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d98032440 con 0x7f3da815e9b0 2024-04-24T23:06:26.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.557+0000 7f3da67fc640 1 -- 172.21.15.32:0/726131212 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d98032810 con 0x7f3da815e9b0 2024-04-24T23:06:26.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.558+0000 7f3dadf29640 1 -- 172.21.15.32:0/726131212 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3da815e9b0 msgr2=0x7f3da80d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:26.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.558+0000 7f3dadf29640 1 --2- 172.21.15.32:0/726131212 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3da815e9b0 0x7f3da80d2b90 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3d980097a0 tx=0x7f3d9802a180 comp rx=0 tx=0).stop 2024-04-24T23:06:26.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.558+0000 7f3dadf29640 1 -- 172.21.15.32:0/726131212 shutdown_connections 2024-04-24T23:06:26.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.558+0000 7f3dadf29640 1 --2- 172.21.15.32:0/726131212 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3da806b820 0x7f3da80b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:26.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.558+0000 7f3dadf29640 1 --2- 172.21.15.32:0/726131212 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3da815e9b0 0x7f3da80d2b90 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:26.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.558+0000 7f3dadf29640 1 -- 172.21.15.32:0/726131212 >> 172.21.15.32:0/726131212 conn(0x7f3da80d11d0 msgr2=0x7f3da80d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:26.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.558+0000 7f3dadf29640 1 -- 172.21.15.32:0/726131212 shutdown_connections 2024-04-24T23:06:26.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.558+0000 7f3dadf29640 1 -- 172.21.15.32:0/726131212 wait complete. 2024-04-24T23:06:26.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.559+0000 7f3dadf29640 1 Processor -- start 2024-04-24T23:06:26.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.559+0000 7f3dadf29640 1 -- start start 2024-04-24T23:06:26.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.560+0000 7f3dadf29640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3da806b820 0x7f3da81f40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:26.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.560+0000 7f3dadf29640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3da80d4210 0x7f3da80ce080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:26.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.560+0000 7f3dadf29640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3da815e9b0 0x7f3da815fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:26.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.560+0000 7f3dadf29640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3da80ba0e0 con 0x7f3da80d4210 2024-04-24T23:06:26.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.560+0000 7f3dadf29640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3da80d2400 con 0x7f3da806b820 2024-04-24T23:06:26.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.560+0000 7f3dadf29640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f3da80ba260 con 0x7f3da815e9b0 2024-04-24T23:06:26.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.560+0000 7f3da6ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3da80d4210 0x7f3da80ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:26.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.560+0000 7f3da6ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3da80d4210 0x7f3da80ce080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53634/0 (socket says 172.21.15.32:53634) 2024-04-24T23:06:26.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.560+0000 7f3da6ffd640 1 -- 172.21.15.32:0/139694127 learned_addr learned my addr 172.21.15.32:0/139694127 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:26.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.560+0000 7f3da77fe640 1 --2- 172.21.15.32:0/139694127 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3da806b820 0x7f3da81f40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:26.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.560+0000 7f3da7fff640 1 --2- 172.21.15.32:0/139694127 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3da815e9b0 0x7f3da815fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:26.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.560+0000 7f3da6ffd640 1 -- 172.21.15.32:0/139694127 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3da815e9b0 msgr2=0x7f3da815fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:26.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.560+0000 7f3da6ffd640 1 --2- 172.21.15.32:0/139694127 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3da815e9b0 0x7f3da815fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:26.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.560+0000 7f3da6ffd640 1 -- 172.21.15.32:0/139694127 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3da806b820 msgr2=0x7f3da81f40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:26.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.560+0000 7f3da6ffd640 1 --2- 172.21.15.32:0/139694127 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3da806b820 0x7f3da81f40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:26.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.560+0000 7f3da6ffd640 1 -- 172.21.15.32:0/139694127 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3da81606a0 con 0x7f3da80d4210 2024-04-24T23:06:26.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.560+0000 7f3da6ffd640 1 --2- 172.21.15.32:0/139694127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3da80d4210 0x7f3da80ce080 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c00c430 tx=0x7f3d9c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:26.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.560+0000 7f3d8ffff640 1 -- 172.21.15.32:0/139694127 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c017070 con 0x7f3da80d4210 2024-04-24T23:06:26.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.561+0000 7f3dadf29640 1 -- 172.21.15.32:0/139694127 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3da8160980 con 0x7f3da80d4210 2024-04-24T23:06:26.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.561+0000 7f3d8ffff640 1 -- 172.21.15.32:0/139694127 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d9c005ce0 con 0x7f3da80d4210 2024-04-24T23:06:26.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.561+0000 7f3d8ffff640 1 -- 172.21.15.32:0/139694127 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c0072f0 con 0x7f3da80d4210 2024-04-24T23:06:26.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.561+0000 7f3dadf29640 1 -- 172.21.15.32:0/139694127 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3da80ce5c0 con 0x7f3da80d4210 2024-04-24T23:06:26.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.562+0000 7f3dadf29640 1 -- 172.21.15.32:0/139694127 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3da8068430 con 0x7f3da80d4210 2024-04-24T23:06:26.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.562+0000 7f3d8ffff640 1 -- 172.21.15.32:0/139694127 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3d9c01a070 con 0x7f3da80d4210 2024-04-24T23:06:26.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.562+0000 7f3d8ffff640 1 --2- 172.21.15.32:0/139694127 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3d9406be30 0x7f3d9406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:26.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.562+0000 7f3d8ffff640 1 -- 172.21.15.32:0/139694127 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4943+0+0 (secure 0 0 0) 0x7f3d9c089ef0 con 0x7f3da80d4210 2024-04-24T23:06:26.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.563+0000 7f3da77fe640 1 --2- 172.21.15.32:0/139694127 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3d9406be30 0x7f3d9406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:26.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.563+0000 7f3da77fe640 1 --2- 172.21.15.32:0/139694127 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3d9406be30 0x7f3d9406e2f0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f3d98034000 tx=0x7f3d9802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:26.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.568+0000 7f3d8ffff640 1 -- 172.21.15.32:0/139694127 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3d9c0584d0 con 0x7f3da80d4210 2024-04-24T23:06:26.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.787+0000 7f3dadf29640 1 -- 172.21.15.32:0/139694127 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3da80d2860 con 0x7f3da80d4210 2024-04-24T23:06:26.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.787+0000 7f3d8ffff640 1 -- 172.21.15.32:0/139694127 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+3875 (secure 0 0 0) 0x7f3d9c05c7d0 con 0x7f3da80d4210 2024-04-24T23:06:26.789 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:06:26.789 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":60,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3361752496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3361752496},{"type":"v1","addr":"172.21.15.121:6837","nonce":3361752496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":6799,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3813453544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3813453544},{"type":"v1","addr":"172.21.15.121:6836","nonce":3813453544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7779,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/1352085651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1352085651},{"type":"v1","addr":"172.21.15.32:6833","nonce":1352085651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":8205,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3759332396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3759332396},{"type":"v1","addr":"172.21.15.32:6835","nonce":3759332396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[]} 2024-04-24T23:06:26.790 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 60 2024-04-24T23:06:26.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.789+0000 7f3dadf29640 1 -- 172.21.15.32:0/139694127 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3d9406be30 msgr2=0x7f3d9406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:26.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.789+0000 7f3dadf29640 1 --2- 172.21.15.32:0/139694127 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3d9406be30 0x7f3d9406e2f0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f3d98034000 tx=0x7f3d9802e000 comp rx=0 tx=0).stop 2024-04-24T23:06:26.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.789+0000 7f3dadf29640 1 -- 172.21.15.32:0/139694127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3da80d4210 msgr2=0x7f3da80ce080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:26.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.789+0000 7f3dadf29640 1 --2- 172.21.15.32:0/139694127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3da80d4210 0x7f3da80ce080 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c00c430 tx=0x7f3d9c00c8f0 comp rx=0 tx=0).stop 2024-04-24T23:06:26.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.789+0000 7f3dadf29640 1 -- 172.21.15.32:0/139694127 shutdown_connections 2024-04-24T23:06:26.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.789+0000 7f3dadf29640 1 --2- 172.21.15.32:0/139694127 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3d9406be30 0x7f3d9406e2f0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:26.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.789+0000 7f3dadf29640 1 --2- 172.21.15.32:0/139694127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3da80d4210 0x7f3da80ce080 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:26.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.789+0000 7f3dadf29640 1 --2- 172.21.15.32:0/139694127 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3da815e9b0 0x7f3da815fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:26.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.789+0000 7f3dadf29640 1 --2- 172.21.15.32:0/139694127 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3da806b820 0x7f3da81f40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:26.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.789+0000 7f3dadf29640 1 -- 172.21.15.32:0/139694127 >> 172.21.15.32:0/139694127 conn(0x7f3da80d11d0 msgr2=0x7f3da80d15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:26.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.790+0000 7f3dadf29640 1 -- 172.21.15.32:0/139694127 shutdown_connections 2024-04-24T23:06:26.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.790+0000 7f3dadf29640 1 -- 172.21.15.32:0/139694127 wait complete. 2024-04-24T23:06:26.802 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-24T23:06:26.802 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-24T23:06:26.803 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-24T23:06:26.803 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:06:26.803 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-24T23:06:26.803 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-24T23:06:26.803 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-24T23:06:26.803 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:06:26.804 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-24T23:06:26.804 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-24T23:06:26.804 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-24T23:06:26.804 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:06:26.804 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-24T23:06:26.804 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-24T23:06:26.805 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-24T23:06:26.805 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:06:26.812 INFO:tasks.ceph.mds.a:Stopped 2024-04-24T23:06:26.812 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-24T23:06:26.814 INFO:tasks.ceph.mds.c:Stopped 2024-04-24T23:06:26.814 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-24T23:06:26.816 INFO:tasks.ceph.mds.b:Stopped 2024-04-24T23:06:26.816 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-24T23:06:26.817 INFO:tasks.ceph.mds.d:Stopped 2024-04-24T23:06:26.817 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-24T23:06:26.855 INFO:tasks.ceph.mds.a:Started 2024-04-24T23:06:26.855 DEBUG:teuthology.parallel:result is None 2024-04-24T23:06:26.855 INFO:tasks.ceph.mds.c:Started 2024-04-24T23:06:26.856 DEBUG:teuthology.parallel:result is None 2024-04-24T23:06:26.858 INFO:tasks.ceph.mds.b:Started 2024-04-24T23:06:26.859 DEBUG:teuthology.parallel:result is None 2024-04-24T23:06:26.859 INFO:tasks.ceph.mds.d:Started 2024-04-24T23:06:26.859 DEBUG:teuthology.parallel:result is None 2024-04-24T23:06:26.859 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-24T23:06:26.930 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2024-04-24T23:06:26.931 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2024-04-24T23:06:26.936 INFO:tasks.ceph.mds.d.smithi121.stdout:starting mds.d at 2024-04-24T23:06:26.936 INFO:tasks.ceph.mds.b.smithi121.stdout:starting mds.b at 2024-04-24T23:06:26.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.965+0000 7fd1c1a4d640 1 Processor -- start 2024-04-24T23:06:26.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.965+0000 7fd1c1a4d640 1 -- start start 2024-04-24T23:06:26.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.966+0000 7fd1c1a4d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd1bc06b820 0x7fd1bc0d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:26.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.966+0000 7fd1c1a4d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1bc0d3170 con 0x7fd1bc0d4210 2024-04-24T23:06:26.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.966+0000 7fd1c1a4d640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1bc0d3450 con 0x7fd1bc15e9b0 2024-04-24T23:06:26.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.966+0000 7fd1c1a4d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fd1bc0d3730 con 0x7fd1bc06b820 2024-04-24T23:06:26.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.966+0000 7fd1ba7fc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd1bc06b820 0x7fd1bc0d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:26.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.966+0000 7fd1ba7fc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd1bc06b820 0x7fd1bc0d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53692/0 (socket says 172.21.15.32:53692) 2024-04-24T23:06:26.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.966+0000 7fd1ba7fc640 1 -- 172.21.15.32:0/3748853944 learned_addr learned my addr 172.21.15.32:0/3748853944 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:26.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.966+0000 7fd1bb7fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd1bc0d4210 0x7fd1bc0b78f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42772/0 (socket says 172.21.15.32:42772) 2024-04-24T23:06:26.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.966+0000 7fd1b9ffb640 1 -- 172.21.15.32:0/3748853944 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2167967995 0 0) 0x7fd1bc0d3170 con 0x7fd1bc0d4210 2024-04-24T23:06:26.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.966+0000 7fd1ba7fc640 1 -- 172.21.15.32:0/3748853944 >> v1:172.21.15.121:6789/0 conn(0x7fd1bc15e9b0 legacy=0x7fd1bc06b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:26.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.966+0000 7fd1ba7fc640 1 -- 172.21.15.32:0/3748853944 >> v1:172.21.15.32:6789/0 conn(0x7fd1bc0d4210 legacy=0x7fd1bc0b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:26.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.967+0000 7fd1ba7fc640 1 -- 172.21.15.32:0/3748853944 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1bc0d3a10 con 0x7fd1bc06b820 2024-04-24T23:06:26.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.967+0000 7fd1ba7fc640 1 --2- 172.21.15.32:0/3748853944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1bc06b820 0x7fd1bc0d2b90 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0009870 tx=0x7fd1b002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2698ccd958feec9d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:26.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.967+0000 7fd1b9ffb640 1 -- 172.21.15.32:0/3748853944 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b0037070 con 0x7fd1bc06b820 2024-04-24T23:06:26.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.967+0000 7fd1b9ffb640 1 -- 172.21.15.32:0/3748853944 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1b0032440 con 0x7fd1bc06b820 2024-04-24T23:06:26.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.967+0000 7fd1b9ffb640 1 -- 172.21.15.32:0/3748853944 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b0032810 con 0x7fd1bc06b820 2024-04-24T23:06:26.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.968+0000 7fd1c1a4d640 1 -- 172.21.15.32:0/3748853944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1bc06b820 msgr2=0x7fd1bc0d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:26.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.968+0000 7fd1c1a4d640 1 --2- 172.21.15.32:0/3748853944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1bc06b820 0x7fd1bc0d2b90 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0009870 tx=0x7fd1b002a270 comp rx=0 tx=0).stop 2024-04-24T23:06:26.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.968+0000 7fd1c1a4d640 1 -- 172.21.15.32:0/3748853944 shutdown_connections 2024-04-24T23:06:26.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.968+0000 7fd1c1a4d640 1 --2- 172.21.15.32:0/3748853944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1bc06b820 0x7fd1bc0d2b90 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:26.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.968+0000 7fd1c1a4d640 1 -- 172.21.15.32:0/3748853944 >> 172.21.15.32:0/3748853944 conn(0x7fd1bc0d11d0 msgr2=0x7fd1bc0d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:26.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.968+0000 7fd1c1a4d640 1 -- 172.21.15.32:0/3748853944 shutdown_connections 2024-04-24T23:06:26.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.968+0000 7fd1c1a4d640 1 -- 172.21.15.32:0/3748853944 wait complete. 2024-04-24T23:06:26.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.969+0000 7fd1c1a4d640 1 Processor -- start 2024-04-24T23:06:26.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.969+0000 7fd1c1a4d640 1 -- start start 2024-04-24T23:06:26.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.970+0000 7fd1c1a4d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1bc06b820 0x7fd1bc15a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:26.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.970+0000 7fd1c1a4d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd1bc0d4210 0x7fd1bc0ca620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:26.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.970+0000 7fd1c1a4d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1bc15e9b0 0x7fd1bc0ccd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:26.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.970+0000 7fd1c1a4d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd1bc0ba0e0 con 0x7fd1bc06b820 2024-04-24T23:06:26.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.970+0000 7fd1c1a4d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd1bc0d2400 con 0x7fd1bc15e9b0 2024-04-24T23:06:26.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.970+0000 7fd1c1a4d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd1bc0ba260 con 0x7fd1bc0d4210 2024-04-24T23:06:26.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.970+0000 7fd1baffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1bc06b820 0x7fd1bc15a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:26.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.970+0000 7fd1baffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1bc06b820 0x7fd1bc15a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53702/0 (socket says 172.21.15.32:53702) 2024-04-24T23:06:26.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.970+0000 7fd1baffd640 1 -- 172.21.15.32:0/3617670542 learned_addr learned my addr 172.21.15.32:0/3617670542 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:26.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.970+0000 7fd1ba7fc640 1 --2- 172.21.15.32:0/3617670542 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd1bc0d4210 0x7fd1bc0ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:26.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.970+0000 7fd1bb7fe640 1 --2- 172.21.15.32:0/3617670542 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1bc15e9b0 0x7fd1bc0ccd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:26.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.971+0000 7fd1baffd640 1 -- 172.21.15.32:0/3617670542 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd1bc0d4210 msgr2=0x7fd1bc0ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:26.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.971+0000 7fd1baffd640 1 --2- 172.21.15.32:0/3617670542 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd1bc0d4210 0x7fd1bc0ca620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:26.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.971+0000 7fd1baffd640 1 -- 172.21.15.32:0/3617670542 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1bc15e9b0 msgr2=0x7fd1bc0ccd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:26.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.971+0000 7fd1baffd640 1 --2- 172.21.15.32:0/3617670542 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1bc15e9b0 0x7fd1bc0ccd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:26.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.971+0000 7fd1baffd640 1 -- 172.21.15.32:0/3617670542 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1bc0c8320 con 0x7fd1bc06b820 2024-04-24T23:06:26.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.971+0000 7fd1baffd640 1 --2- 172.21.15.32:0/3617670542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1bc06b820 0x7fd1bc15a5d0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac00c400 tx=0x7fd1ac00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:26.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.971+0000 7fd1a3fff640 1 -- 172.21.15.32:0/3617670542 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac017070 con 0x7fd1bc06b820 2024-04-24T23:06:26.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.971+0000 7fd1a3fff640 1 -- 172.21.15.32:0/3617670542 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1ac0052d0 con 0x7fd1bc06b820 2024-04-24T23:06:26.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.971+0000 7fd1a3fff640 1 -- 172.21.15.32:0/3617670542 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac003830 con 0x7fd1bc06b820 2024-04-24T23:06:26.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.971+0000 7fd1c1a4d640 1 -- 172.21.15.32:0/3617670542 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1bc0cd2a0 con 0x7fd1bc06b820 2024-04-24T23:06:26.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.971+0000 7fd1c1a4d640 1 -- 172.21.15.32:0/3617670542 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1bc0c85c0 con 0x7fd1bc06b820 2024-04-24T23:06:26.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.972+0000 7fd1c1a4d640 1 -- 172.21.15.32:0/3617670542 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1bc068430 con 0x7fd1bc06b820 2024-04-24T23:06:26.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.973+0000 7fd1a3fff640 1 -- 172.21.15.32:0/3617670542 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd1ac01a070 con 0x7fd1bc06b820 2024-04-24T23:06:26.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.973+0000 7fd1a3fff640 1 --2- 172.21.15.32:0/3617670542 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd1a806be30 0x7fd1a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:26.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.973+0000 7fd1a3fff640 1 -- 172.21.15.32:0/3617670542 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4943+0+0 (secure 0 0 0) 0x7fd1ac089e10 con 0x7fd1bc06b820 2024-04-24T23:06:26.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.973+0000 7fd1ba7fc640 1 --2- 172.21.15.32:0/3617670542 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd1a806be30 0x7fd1a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:26.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.974+0000 7fd1ba7fc640 1 --2- 172.21.15.32:0/3617670542 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd1a806be30 0x7fd1a806e2f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0034000 tx=0x7fd1b002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:26.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:26.977+0000 7fd1a3fff640 1 -- 172.21.15.32:0/3617670542 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd1ac058470 con 0x7fd1bc06b820 2024-04-24T23:06:27.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:27.171+0000 7fd1c1a4d640 1 -- 172.21.15.32:0/3617670542 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fd1bc074df0 con 0x7fd1bc06b820 2024-04-24T23:06:27.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:27.914+0000 7fd1a3fff640 1 -- 172.21.15.32:0/3617670542 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v150) ==== 97+0+0 (secure 0 0 0) 0x7fd1ac05c770 con 0x7fd1bc06b820 2024-04-24T23:06:27.916 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blocklist entries 2024-04-24T23:06:27.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:27.917+0000 7fd1c1a4d640 1 -- 172.21.15.32:0/3617670542 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd1a806be30 msgr2=0x7fd1a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:27.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:27.917+0000 7fd1c1a4d640 1 --2- 172.21.15.32:0/3617670542 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd1a806be30 0x7fd1a806e2f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0034000 tx=0x7fd1b002e000 comp rx=0 tx=0).stop 2024-04-24T23:06:27.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:27.917+0000 7fd1c1a4d640 1 -- 172.21.15.32:0/3617670542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1bc06b820 msgr2=0x7fd1bc15a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:27.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:27.917+0000 7fd1c1a4d640 1 --2- 172.21.15.32:0/3617670542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1bc06b820 0x7fd1bc15a5d0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac00c400 tx=0x7fd1ac00c8c0 comp rx=0 tx=0).stop 2024-04-24T23:06:27.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:27.917+0000 7fd1c1a4d640 1 -- 172.21.15.32:0/3617670542 shutdown_connections 2024-04-24T23:06:27.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:27.917+0000 7fd1c1a4d640 1 --2- 172.21.15.32:0/3617670542 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd1a806be30 0x7fd1a806e2f0 secure :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0034000 tx=0x7fd1b002e000 comp rx=0 tx=0).stop 2024-04-24T23:06:27.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:27.918+0000 7fd1c1a4d640 1 --2- 172.21.15.32:0/3617670542 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1bc15e9b0 0x7fd1bc0ccd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:27.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:27.918+0000 7fd1c1a4d640 1 --2- 172.21.15.32:0/3617670542 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd1bc0d4210 0x7fd1bc0ca620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:27.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:27.918+0000 7fd1c1a4d640 1 --2- 172.21.15.32:0/3617670542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1bc06b820 0x7fd1bc15a5d0 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:27.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:27.918+0000 7fd1c1a4d640 1 -- 172.21.15.32:0/3617670542 >> 172.21.15.32:0/3617670542 conn(0x7fd1bc0d11d0 msgr2=0x7fd1bc06b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:27.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:27.918+0000 7fd1c1a4d640 1 -- 172.21.15.32:0/3617670542 shutdown_connections 2024-04-24T23:06:27.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:27.918+0000 7fd1c1a4d640 1 -- 172.21.15.32:0/3617670542 wait complete. 2024-04-24T23:06:27.932 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-24T23:06:28.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.038+0000 7f689f7d5640 1 Processor -- start 2024-04-24T23:06:28.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.038+0000 7f689f7d5640 1 -- start start 2024-04-24T23:06:28.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.039+0000 7f689f7d5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6898072cf0 0x7f689806c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:28.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.039+0000 7f689f7d5640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6898071770 0x7f689806cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:28.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.039+0000 7f689f7d5640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f689815c990 con 0x7f689815c5b0 2024-04-24T23:06:28.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.039+0000 7f689f7d5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f68980b4740 con 0x7f6898072cf0 2024-04-24T23:06:28.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.039+0000 7f689f7d5640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f6898072860 con 0x7f6898071770 2024-04-24T23:06:28.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.039+0000 7f689cd70640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6898072cf0 0x7f689806c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:28.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.039+0000 7f689d571640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f689815c5b0 0x7f68980b4030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42778/0 (socket says 172.21.15.32:42778) 2024-04-24T23:06:28.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.039+0000 7f689d571640 1 -- 172.21.15.32:0/3152992373 learned_addr learned my addr 172.21.15.32:0/3152992373 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:06:28.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.039+0000 7f689dd72640 1 --2- 172.21.15.32:0/3152992373 >> v2:172.21.15.121:3300/0 conn(0x7f6898071770 0x7f689806cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:28.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.040+0000 7f689cd70640 1 -- 172.21.15.32:0/3152992373 >> v2:172.21.15.121:3300/0 conn(0x7f6898071770 msgr2=0x7f689806cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:28.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.040+0000 7f689cd70640 1 --2- 172.21.15.32:0/3152992373 >> v2:172.21.15.121:3300/0 conn(0x7f6898071770 0x7f689806cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:28.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.040+0000 7f689cd70640 1 -- 172.21.15.32:0/3152992373 >> v1:172.21.15.32:6789/0 conn(0x7f689815c5b0 legacy=0x7f68980b4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:28.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.040+0000 7f688ffff640 1 -- 172.21.15.32:0/3152992373 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3899439368 0 0) 0x7f6890001110 con 0x7f689815c5b0 2024-04-24T23:06:28.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.040+0000 7f689cd70640 1 -- 172.21.15.32:0/3152992373 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6898071c50 con 0x7f6898072cf0 2024-04-24T23:06:28.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.040+0000 7f689cd70640 1 --2- 172.21.15.32:0/3152992373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6898072cf0 0x7f689806c5f0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f68980aff50 tx=0x7f688802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71dc230f8f94ce04 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:28.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.040+0000 7f688ffff640 1 -- 172.21.15.32:0/3152992373 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6888037070 con 0x7f6898072cf0 2024-04-24T23:06:28.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.041+0000 7f688ffff640 1 -- 172.21.15.32:0/3152992373 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6888032440 con 0x7f6898072cf0 2024-04-24T23:06:28.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.041+0000 7f688ffff640 1 -- 172.21.15.32:0/3152992373 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688802ae60 con 0x7f6898072cf0 2024-04-24T23:06:28.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.041+0000 7f689f7d5640 1 -- 172.21.15.32:0/3152992373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6898072cf0 msgr2=0x7f689806c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:28.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.041+0000 7f689f7d5640 1 --2- 172.21.15.32:0/3152992373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6898072cf0 0x7f689806c5f0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f68980aff50 tx=0x7f688802a110 comp rx=0 tx=0).stop 2024-04-24T23:06:28.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.041+0000 7f689f7d5640 1 -- 172.21.15.32:0/3152992373 shutdown_connections 2024-04-24T23:06:28.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.041+0000 7f689f7d5640 1 --2- 172.21.15.32:0/3152992373 >> v2:172.21.15.121:3300/0 conn(0x7f6898071770 0x7f689806cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:28.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.041+0000 7f689f7d5640 1 --2- 172.21.15.32:0/3152992373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6898072cf0 0x7f689806c5f0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:28.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.041+0000 7f689f7d5640 1 -- 172.21.15.32:0/3152992373 >> 172.21.15.32:0/3152992373 conn(0x7f689806b110 msgr2=0x7f689806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:28.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.041+0000 7f689f7d5640 1 -- 172.21.15.32:0/3152992373 shutdown_connections 2024-04-24T23:06:28.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.041+0000 7f689f7d5640 1 -- 172.21.15.32:0/3152992373 wait complete. 2024-04-24T23:06:28.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.043+0000 7f689f7d5640 1 Processor -- start 2024-04-24T23:06:28.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.043+0000 7f689f7d5640 1 -- start start 2024-04-24T23:06:28.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.043+0000 7f689f7d5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6898071770 0x7f689815efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:28.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.043+0000 7f689f7d5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6898072cf0 0x7f68980ce230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:28.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.043+0000 7f689f7d5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f689815c5b0 0x7f68980cf270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:28.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.043+0000 7f689f7d5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f68980724f0 con 0x7f689815c5b0 2024-04-24T23:06:28.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.043+0000 7f689f7d5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6898072210 con 0x7f6898072cf0 2024-04-24T23:06:28.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.043+0000 7f689f7d5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f68980b4b40 con 0x7f6898071770 2024-04-24T23:06:28.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.043+0000 7f689dd72640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f689815c5b0 0x7f68980cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:28.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.043+0000 7f689dd72640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f689815c5b0 0x7f68980cf270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53718/0 (socket says 172.21.15.32:53718) 2024-04-24T23:06:28.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.043+0000 7f689dd72640 1 -- 172.21.15.32:0/3842840889 learned_addr learned my addr 172.21.15.32:0/3842840889 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:28.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.044+0000 7f689d571640 1 --2- 172.21.15.32:0/3842840889 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6898071770 0x7f689815efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:28.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.044+0000 7f689cd70640 1 --2- 172.21.15.32:0/3842840889 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6898072cf0 0x7f68980ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:28.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.044+0000 7f689dd72640 1 -- 172.21.15.32:0/3842840889 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6898071770 msgr2=0x7f689815efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:28.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.044+0000 7f689dd72640 1 --2- 172.21.15.32:0/3842840889 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6898071770 0x7f689815efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:28.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.044+0000 7f689dd72640 1 -- 172.21.15.32:0/3842840889 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6898072cf0 msgr2=0x7f68980ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:28.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.044+0000 7f689dd72640 1 --2- 172.21.15.32:0/3842840889 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6898072cf0 0x7f68980ce230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:28.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.044+0000 7f689dd72640 1 -- 172.21.15.32:0/3842840889 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6898150710 con 0x7f689815c5b0 2024-04-24T23:06:28.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.044+0000 7f689dd72640 1 --2- 172.21.15.32:0/3842840889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f689815c5b0 0x7f68980cf270 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f689400efc0 tx=0x7f689400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:28.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.044+0000 7f688dffb640 1 -- 172.21.15.32:0/3842840889 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6894018020 con 0x7f689815c5b0 2024-04-24T23:06:28.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.044+0000 7f688dffb640 1 -- 172.21.15.32:0/3842840889 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6894014480 con 0x7f689815c5b0 2024-04-24T23:06:28.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.044+0000 7f689f7d5640 1 -- 172.21.15.32:0/3842840889 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f689815f4e0 con 0x7f689815c5b0 2024-04-24T23:06:28.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.044+0000 7f688dffb640 1 -- 172.21.15.32:0/3842840889 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6894014870 con 0x7f689815c5b0 2024-04-24T23:06:28.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.044+0000 7f689f7d5640 1 -- 172.21.15.32:0/3842840889 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6898150430 con 0x7f689815c5b0 2024-04-24T23:06:28.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.045+0000 7f689f7d5640 1 -- 172.21.15.32:0/3842840889 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6898068430 con 0x7f689815c5b0 2024-04-24T23:06:28.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.046+0000 7f688dffb640 1 -- 172.21.15.32:0/3842840889 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6894005d10 con 0x7f689815c5b0 2024-04-24T23:06:28.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.046+0000 7f688dffb640 1 --2- 172.21.15.32:0/3842840889 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f687c06be30 0x7f687c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:28.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.047+0000 7f688dffb640 1 -- 172.21.15.32:0/3842840889 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 4857+0+0 (secure 0 0 0) 0x7f689408ba80 con 0x7f689815c5b0 2024-04-24T23:06:28.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.047+0000 7f689d571640 1 --2- 172.21.15.32:0/3842840889 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f687c06be30 0x7f687c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:28.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.047+0000 7f689d571640 1 --2- 172.21.15.32:0/3842840889 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f687c06be30 0x7f687c06e2f0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f6890000f60 tx=0x7f6890007c00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:28.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.051+0000 7f688dffb640 1 -- 172.21.15.32:0/3842840889 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f689408de20 con 0x7f689815c5b0 2024-04-24T23:06:28.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.301+0000 7f689f7d5640 1 -- 172.21.15.32:0/3842840889 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f68980b5c60 con 0x7f689815c5b0 2024-04-24T23:06:28.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.302+0000 7f688dffb640 1 -- 172.21.15.32:0/3842840889 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) ==== 80+0+5304 (secure 0 0 0) 0x7f6894022030 con 0x7f689815c5b0 2024-04-24T23:06:28.304 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:06:28.304 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:06:28.304 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2024-04-24T23:06:28.304 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:28.304 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2024-04-24T23:06:28.304 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmXRdxAxAA/EYEn1C38XhG7DC+0X4y7A==", 2024-04-24T23:06:28.304 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:06:28.304 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:06:28.304 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-24T23:06:28.304 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-24T23:06:28.304 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:06:28.305 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:28.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:28.305 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:28.305 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2024-04-24T23:06:28.305 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmak7bDhAA0he+Jp/gkkrfBjYKSObjcw==", 2024-04-24T23:06:28.305 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:06:28.305 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:06:28.305 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-24T23:06:28.305 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-24T23:06:28.305 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:06:28.305 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:28.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:28.305 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:28.305 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.c", 2024-04-24T23:06:28.306 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmZXFdChAA8TwwkA5EIXHBzPsQSFvXcQ==", 2024-04-24T23:06:28.306 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:06:28.306 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:06:28.306 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-24T23:06:28.306 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-24T23:06:28.306 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:06:28.306 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:28.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:28.306 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:28.306 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.d", 2024-04-24T23:06:28.306 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmLbGuFRAAe91fZlulziWdWiDdLBLWNA==", 2024-04-24T23:06:28.306 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:06:28.306 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:06:28.307 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-24T23:06:28.307 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-24T23:06:28.307 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:06:28.307 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:28.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:28.307 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:28.307 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2024-04-24T23:06:28.307 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCfjilmhTwoOBAAig98xa3a2qnv4rriUO9VCQ==", 2024-04-24T23:06:28.307 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:06:28.307 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:06:28.307 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:06:28.307 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:06:28.307 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:28.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:28.308 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:28.308 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2024-04-24T23:06:28.308 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCgjilmwRaAAxAAUqDOyR9HV6DEnmbyn3WAjA==", 2024-04-24T23:06:28.308 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:06:28.308 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:06:28.308 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:06:28.308 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:06:28.308 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:28.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:28.308 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:28.308 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2024-04-24T23:06:28.308 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCgjilmDl2qDBAAF1bhTOaSOc612wmFRd8+XA==", 2024-04-24T23:06:28.309 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:06:28.309 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:06:28.309 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:06:28.309 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:06:28.309 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:28.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:28.309 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:28.309 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2024-04-24T23:06:28.309 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCgjilmAxHwFRAA9a0v/YwngRxq1vmMRsmEkQ==", 2024-04-24T23:06:28.310 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:06:28.310 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:06:28.310 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:06:28.310 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:06:28.310 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:28.310 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:28.310 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:28.310 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2024-04-24T23:06:28.310 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCjjilmDPmJIxAASZlEo1ZJHOhIAFGukIj1tQ==", 2024-04-24T23:06:28.310 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:06:28.310 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:06:28.310 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:06:28.310 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:06:28.310 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:28.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:28.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:28.311 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2024-04-24T23:06:28.311 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCjjilmgcpuKhAAedSq0ubNB30s3bz0fY38hw==", 2024-04-24T23:06:28.311 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:06:28.311 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:06:28.311 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:06:28.311 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:06:28.311 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:28.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:28.312 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:28.312 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2024-04-24T23:06:28.312 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCjjilmFv2VMxAAM+mUc4I0sjawy2uwuUEKYg==", 2024-04-24T23:06:28.312 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:06:28.312 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:06:28.312 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:06:28.312 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:06:28.312 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:28.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:28.313 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:28.313 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2024-04-24T23:06:28.313 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCkjilmLvEbARAAnaOvQ5Ee0clN/A9SVoKuBQ==", 2024-04-24T23:06:28.313 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:06:28.313 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:06:28.313 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:06:28.313 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:06:28.313 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:28.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:28.313 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:28.313 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2024-04-24T23:06:28.313 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilm5oeiGRAAdLXwwqrYhV3HHo7Dw/2ZqQ==", 2024-04-24T23:06:28.314 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:06:28.314 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:06:28.314 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2024-04-24T23:06:28.314 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-24T23:06:28.314 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:28.314 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:28.314 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:28.314 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2024-04-24T23:06:28.315 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmyWBDIhAARtVP4Unk3HHlQYZ2QZ3GzQ==", 2024-04-24T23:06:28.315 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:06:28.315 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:06:28.315 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-24T23:06:28.315 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-24T23:06:28.315 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-24T23:06:28.315 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:28.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:28.315 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:28.315 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.2", 2024-04-24T23:06:28.315 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilm4QElHxAAXgE316aRCosxX7XER5gtKQ==", 2024-04-24T23:06:28.315 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:06:28.316 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:06:28.316 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-24T23:06:28.316 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-24T23:06:28.316 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-24T23:06:28.316 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:28.316 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:28.316 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:28.316 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.3", 2024-04-24T23:06:28.316 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmMZihJxAAoWaAAFY3vVfZnC4/GoT0+g==", 2024-04-24T23:06:28.317 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:06:28.317 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:06:28.317 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-24T23:06:28.317 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-24T23:06:28.317 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-24T23:06:28.317 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:28.317 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:28.317 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:28.317 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2024-04-24T23:06:28.317 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCdjilmdVXeIxAAwGGaAfshx4pHcoc5+3kRww==", 2024-04-24T23:06:28.318 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:06:28.318 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2024-04-24T23:06:28.318 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-24T23:06:28.318 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-24T23:06:28.318 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:06:28.318 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:28.318 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:28.318 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:28.318 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2024-04-24T23:06:28.318 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmn3FJABAAGnvNAoNLzK20cQF6beihmw==", 2024-04-24T23:06:28.319 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:06:28.319 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2024-04-24T23:06:28.319 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2024-04-24T23:06:28.319 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:06:28.319 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:28.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:06:28.319 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:28.319 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2024-04-24T23:06:28.319 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCdjilmMqs2OBAAOe3bLpnCq+HK2EbeXnwY8Q==", 2024-04-24T23:06:28.319 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:06:28.320 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2024-04-24T23:06:28.320 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2024-04-24T23:06:28.320 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:06:28.320 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:28.320 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:28.320 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:06:28.320 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:06:28.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.304+0000 7f689f7d5640 1 -- 172.21.15.32:0/3842840889 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f687c06be30 msgr2=0x7f687c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:28.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.304+0000 7f689f7d5640 1 --2- 172.21.15.32:0/3842840889 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f687c06be30 0x7f687c06e2f0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f6890000f60 tx=0x7f6890007c00 comp rx=0 tx=0).stop 2024-04-24T23:06:28.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.304+0000 7f689f7d5640 1 -- 172.21.15.32:0/3842840889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f689815c5b0 msgr2=0x7f68980cf270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:28.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.304+0000 7f689f7d5640 1 --2- 172.21.15.32:0/3842840889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f689815c5b0 0x7f68980cf270 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f689400efc0 tx=0x7f689400c490 comp rx=0 tx=0).stop 2024-04-24T23:06:28.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.304+0000 7f689f7d5640 1 -- 172.21.15.32:0/3842840889 shutdown_connections 2024-04-24T23:06:28.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.304+0000 7f689f7d5640 1 --2- 172.21.15.32:0/3842840889 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f687c06be30 0x7f687c06e2f0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:28.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.304+0000 7f689f7d5640 1 --2- 172.21.15.32:0/3842840889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f689815c5b0 0x7f68980cf270 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:28.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.304+0000 7f689f7d5640 1 --2- 172.21.15.32:0/3842840889 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6898072cf0 0x7f68980ce230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:28.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.304+0000 7f689f7d5640 1 --2- 172.21.15.32:0/3842840889 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6898071770 0x7f689815efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:28.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.304+0000 7f689f7d5640 1 -- 172.21.15.32:0/3842840889 >> 172.21.15.32:0/3842840889 conn(0x7f689806b110 msgr2=0x7f689806c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:28.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.304+0000 7f689f7d5640 1 -- 172.21.15.32:0/3842840889 shutdown_connections 2024-04-24T23:06:28.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.304+0000 7f689f7d5640 1 -- 172.21.15.32:0/3842840889 wait complete. 2024-04-24T23:06:28.324 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:06:28.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.425+0000 7f4426672640 1 Processor -- start 2024-04-24T23:06:28.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.425+0000 7f4426672640 1 -- start start 2024-04-24T23:06:28.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.426+0000 7f4426672640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f442006b820 0x7f44200ce630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:28.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.426+0000 7f4426672640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44200afdd0 con 0x7f44201476d0 2024-04-24T23:06:28.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.426+0000 7f4426672640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44200af010 con 0x7f44200cfb70 2024-04-24T23:06:28.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.426+0000 7f4426672640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f44200aedc0 con 0x7f442006b820 2024-04-24T23:06:28.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.426+0000 7f441ffff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f44201476d0 0x7f442006b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42782/0 (socket says 172.21.15.32:42782) 2024-04-24T23:06:28.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.426+0000 7f441ffff640 1 -- 172.21.15.32:0/813905476 learned_addr learned my addr 172.21.15.32:0/813905476 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:06:28.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.426+0000 7f441effd640 1 -- 172.21.15.32:0/813905476 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2991294768 0 0) 0x7f44200afdd0 con 0x7f44201476d0 2024-04-24T23:06:28.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.426+0000 7f441effd640 1 -- 172.21.15.32:0/813905476 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4408003520 con 0x7f44201476d0 2024-04-24T23:06:28.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.426+0000 7f441f7fe640 1 --2- 172.21.15.32:0/813905476 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f442006b820 0x7f44200ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:28.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.426+0000 7f441effd640 1 -- 172.21.15.32:0/813905476 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1515184171 0 0) 0x7f4408003520 con 0x7f44201476d0 2024-04-24T23:06:28.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.426+0000 7f441effd640 1 -- 172.21.15.32:0/813905476 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f442006b820 msgr2=0x7f44200ce630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:28.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.426+0000 7f441effd640 1 --2- 172.21.15.32:0/813905476 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f442006b820 0x7f44200ce630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:28.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.426+0000 7f441effd640 1 -- 172.21.15.32:0/813905476 >> v1:172.21.15.121:6789/0 conn(0x7f44200cfb70 legacy=0x7f442006d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:28.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.426+0000 7f441effd640 1 -- 172.21.15.32:0/813905476 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44200af2f0 con 0x7f44201476d0 2024-04-24T23:06:28.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.426+0000 7f441effd640 1 -- 172.21.15.32:0/813905476 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f4410001670 con 0x7f44201476d0 2024-04-24T23:06:28.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.426+0000 7f441effd640 1 -- 172.21.15.32:0/813905476 >> v1:172.21.15.32:6789/0 conn(0x7f44201476d0 legacy=0x7f442006b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:28.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.426+0000 7f441effd640 1 --2- 172.21.15.32:0/813905476 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f442006b820 0x7f44080039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:28.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.426+0000 7f441effd640 1 --2- 172.21.15.32:0/813905476 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4408003f60 0x7f4408004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:28.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.426+0000 7f441effd640 1 --2- 172.21.15.32:0/813905476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4408004d40 0x7f44080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:28.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.426+0000 7f441effd640 1 -- 172.21.15.32:0/813905476 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f44200aedc0 con 0x7f4408004d40 2024-04-24T23:06:28.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.426+0000 7f441effd640 1 -- 172.21.15.32:0/813905476 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f440800bb50 con 0x7f4408003f60 2024-04-24T23:06:28.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.426+0000 7f441effd640 1 -- 172.21.15.32:0/813905476 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f440800bd50 con 0x7f442006b820 2024-04-24T23:06:28.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.427+0000 7f4424c0f640 1 --2- 172.21.15.32:0/813905476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4408004d40 0x7f44080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:28.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.427+0000 7f441f7fe640 1 --2- 172.21.15.32:0/813905476 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4408003f60 0x7f4408004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:28.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.427+0000 7f441ffff640 1 --2- 172.21.15.32:0/813905476 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f442006b820 0x7f44080039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:28.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.427+0000 7f4424c0f640 1 -- 172.21.15.32:0/813905476 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f442006b820 msgr2=0x7f44080039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:28.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.427+0000 7f4424c0f640 1 --2- 172.21.15.32:0/813905476 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f442006b820 0x7f44080039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:28.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.427+0000 7f4424c0f640 1 -- 172.21.15.32:0/813905476 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4408003f60 msgr2=0x7f4408004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:28.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.427+0000 7f4424c0f640 1 --2- 172.21.15.32:0/813905476 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4408003f60 0x7f4408004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:28.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.427+0000 7f4424c0f640 1 -- 172.21.15.32:0/813905476 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4408003520 con 0x7f4408004d40 2024-04-24T23:06:28.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.428+0000 7f4424c0f640 1 --2- 172.21.15.32:0/813905476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4408004d40 0x7f44080051c0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f4418001e50 tx=0x7f4408005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aacec450ae3cad2a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:28.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.428+0000 7f441effd640 1 -- 172.21.15.32:0/813905476 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4418001a70 con 0x7f4408004d40 2024-04-24T23:06:28.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.428+0000 7f441effd640 1 -- 172.21.15.32:0/813905476 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4418003970 con 0x7f4408004d40 2024-04-24T23:06:28.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.428+0000 7f441effd640 1 -- 172.21.15.32:0/813905476 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4418003d60 con 0x7f4408004d40 2024-04-24T23:06:28.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.428+0000 7f4426672640 1 -- 172.21.15.32:0/813905476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4408004d40 msgr2=0x7f44080051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:28.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.428+0000 7f4426672640 1 --2- 172.21.15.32:0/813905476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4408004d40 0x7f44080051c0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f4418001e50 tx=0x7f4408005830 comp rx=0 tx=0).stop 2024-04-24T23:06:28.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.428+0000 7f4424c0f640 1 -- 172.21.15.32:0/813905476 reap_dead start 2024-04-24T23:06:28.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.428+0000 7f4426672640 1 -- 172.21.15.32:0/813905476 shutdown_connections 2024-04-24T23:06:28.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.428+0000 7f4426672640 1 -- 172.21.15.32:0/813905476 >> 172.21.15.32:0/813905476 conn(0x7f44200d0730 msgr2=0x7f44200d0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:28.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.428+0000 7f4426672640 1 -- 172.21.15.32:0/813905476 shutdown_connections 2024-04-24T23:06:28.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.428+0000 7f4426672640 1 -- 172.21.15.32:0/813905476 wait complete. 2024-04-24T23:06:28.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.429+0000 7f4426672640 1 Processor -- start 2024-04-24T23:06:28.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.430+0000 7f4426672640 1 -- start start 2024-04-24T23:06:28.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.430+0000 7f4426672640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4420160f90 0x7f4420070c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:28.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.430+0000 7f4426672640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4420161370 0x7f44200722d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:28.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.430+0000 7f4426672640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44200b0790 0x7f44200b2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:28.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.430+0000 7f4426672640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4420147ab0 con 0x7f44200b0790 2024-04-24T23:06:28.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.430+0000 7f4426672640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f44200af940 con 0x7f4420160f90 2024-04-24T23:06:28.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.430+0000 7f4426672640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f4420148470 con 0x7f4420161370 2024-04-24T23:06:28.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.430+0000 7f4424c0f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44200b0790 0x7f44200b2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:28.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.430+0000 7f4424c0f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44200b0790 0x7f44200b2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53726/0 (socket says 172.21.15.32:53726) 2024-04-24T23:06:28.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.430+0000 7f4424c0f640 1 -- 172.21.15.32:0/1827094922 learned_addr learned my addr 172.21.15.32:0/1827094922 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:28.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.431+0000 7f441ffff640 1 --2- 172.21.15.32:0/1827094922 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4420160f90 0x7f4420070c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:28.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.431+0000 7f441f7fe640 1 --2- 172.21.15.32:0/1827094922 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4420161370 0x7f44200722d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:28.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.431+0000 7f4424c0f640 1 -- 172.21.15.32:0/1827094922 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4420161370 msgr2=0x7f44200722d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:28.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.431+0000 7f4424c0f640 1 --2- 172.21.15.32:0/1827094922 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4420161370 0x7f44200722d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:28.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.431+0000 7f4424c0f640 1 -- 172.21.15.32:0/1827094922 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4420160f90 msgr2=0x7f4420070c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:28.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.431+0000 7f4424c0f640 1 --2- 172.21.15.32:0/1827094922 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4420160f90 0x7f4420070c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:28.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.431+0000 7f4424c0f640 1 -- 172.21.15.32:0/1827094922 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f442013fa00 con 0x7f44200b0790 2024-04-24T23:06:28.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.431+0000 7f441ffff640 1 --2- 172.21.15.32:0/1827094922 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4420160f90 0x7f4420070c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:06:28.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.431+0000 7f4424c0f640 1 --2- 172.21.15.32:0/1827094922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44200b0790 0x7f44200b2060 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f441802a920 tx=0x7f44180014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:28.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.431+0000 7f441cff9640 1 -- 172.21.15.32:0/1827094922 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4418001a70 con 0x7f44200b0790 2024-04-24T23:06:28.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.431+0000 7f441cff9640 1 -- 172.21.15.32:0/1827094922 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4418041070 con 0x7f44200b0790 2024-04-24T23:06:28.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.431+0000 7f4426672640 1 -- 172.21.15.32:0/1827094922 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4420071230 con 0x7f44200b0790 2024-04-24T23:06:28.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.431+0000 7f441cff9640 1 -- 172.21.15.32:0/1827094922 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4418007db0 con 0x7f44200b0790 2024-04-24T23:06:28.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.431+0000 7f4426672640 1 -- 172.21.15.32:0/1827094922 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44200717f0 con 0x7f44200b0790 2024-04-24T23:06:28.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.432+0000 7f4426672640 1 -- 172.21.15.32:0/1827094922 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4420068430 con 0x7f44200b0790 2024-04-24T23:06:28.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.433+0000 7f441cff9640 1 -- 172.21.15.32:0/1827094922 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f441803a070 con 0x7f44200b0790 2024-04-24T23:06:28.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.433+0000 7f441cff9640 1 --2- 172.21.15.32:0/1827094922 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f440c06be30 0x7f440c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:28.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.433+0000 7f441cff9640 1 -- 172.21.15.32:0/1827094922 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 4857+0+0 (secure 0 0 0) 0x7f44180aa370 con 0x7f44200b0790 2024-04-24T23:06:28.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.433+0000 7f441ffff640 1 --2- 172.21.15.32:0/1827094922 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f440c06be30 0x7f440c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:28.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.434+0000 7f441ffff640 1 --2- 172.21.15.32:0/1827094922 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f440c06be30 0x7f440c06e2f0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f4410007db0 tx=0x7f4410009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:28.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.437+0000 7f441cff9640 1 -- 172.21.15.32:0/1827094922 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4418078fe0 con 0x7f44200b0790 2024-04-24T23:06:28.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.623+0000 7f4426672640 1 -- 172.21.15.32:0/1827094922 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4420143b00 con 0x7f44200b0790 2024-04-24T23:06:28.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.624+0000 7f441cff9640 1 -- 172.21.15.32:0/1827094922 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v152) ==== 74+0+12914 (secure 0 0 0) 0x7f44180788e0 con 0x7f44200b0790 2024-04-24T23:06:28.625 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:06:28.626 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":152,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:06:27.924983+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:06:28.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.626+0000 7f4426672640 1 -- 172.21.15.32:0/1827094922 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f440c06be30 msgr2=0x7f440c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:28.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.626+0000 7f4426672640 1 --2- 172.21.15.32:0/1827094922 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f440c06be30 0x7f440c06e2f0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f4410007db0 tx=0x7f4410009040 comp rx=0 tx=0).stop 2024-04-24T23:06:28.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.626+0000 7f4426672640 1 -- 172.21.15.32:0/1827094922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44200b0790 msgr2=0x7f44200b2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:28.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.626+0000 7f4426672640 1 --2- 172.21.15.32:0/1827094922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44200b0790 0x7f44200b2060 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f441802a920 tx=0x7f44180014a0 comp rx=0 tx=0).stop 2024-04-24T23:06:28.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.626+0000 7f4426672640 1 -- 172.21.15.32:0/1827094922 shutdown_connections 2024-04-24T23:06:28.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.626+0000 7f4426672640 1 --2- 172.21.15.32:0/1827094922 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f440c06be30 0x7f440c06e2f0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:28.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.626+0000 7f4426672640 1 --2- 172.21.15.32:0/1827094922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44200b0790 0x7f44200b2060 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:28.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.626+0000 7f4426672640 1 --2- 172.21.15.32:0/1827094922 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4420161370 0x7f44200722d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:28.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.626+0000 7f4426672640 1 --2- 172.21.15.32:0/1827094922 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4420160f90 0x7f4420070c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:28.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.626+0000 7f4426672640 1 -- 172.21.15.32:0/1827094922 >> 172.21.15.32:0/1827094922 conn(0x7f44200d0730 msgr2=0x7f442006d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:28.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.626+0000 7f4426672640 1 -- 172.21.15.32:0/1827094922 shutdown_connections 2024-04-24T23:06:28.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.626+0000 7f4426672640 1 -- 172.21.15.32:0/1827094922 wait complete. 2024-04-24T23:06:28.639 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:06:28.639 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:06:28.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.741+0000 7f7153d09640 1 Processor -- start 2024-04-24T23:06:28.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.741+0000 7f7153d09640 1 -- start start 2024-04-24T23:06:28.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.742+0000 7f7153d09640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f714c162da0 0x7f714c0cf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:28.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.742+0000 7f7153d09640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f714c06cd00 0x7f714c0b3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:28.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.742+0000 7f7153d09640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f714c0cfe50 con 0x7f714c0cf990 2024-04-24T23:06:28.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.742+0000 7f7153d09640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f714c166900 con 0x7f714c162da0 2024-04-24T23:06:28.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.742+0000 7f7153d09640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f714c166be0 con 0x7f714c06cd00 2024-04-24T23:06:28.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.742+0000 7f7151aa5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f714c162da0 0x7f714c0cf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:28.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.742+0000 7f71512a4640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f714c0cf990 0x7f714c06c5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42798/0 (socket says 172.21.15.32:42798) 2024-04-24T23:06:28.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.742+0000 7f7151aa5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f714c162da0 0x7f714c0cf450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53728/0 (socket says 172.21.15.32:53728) 2024-04-24T23:06:28.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.742+0000 7f71512a4640 1 -- 172.21.15.32:0/3495206423 learned_addr learned my addr 172.21.15.32:0/3495206423 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:06:28.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.742+0000 7f71522a6640 1 --2- 172.21.15.32:0/3495206423 >> v2:172.21.15.121:3300/0 conn(0x7f714c06cd00 0x7f714c0b3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:28.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.742+0000 7f7151aa5640 1 -- 172.21.15.32:0/3495206423 >> v2:172.21.15.121:3300/0 conn(0x7f714c06cd00 msgr2=0x7f714c0b3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:28.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.742+0000 7f7151aa5640 1 --2- 172.21.15.32:0/3495206423 >> v2:172.21.15.121:3300/0 conn(0x7f714c06cd00 0x7f714c0b3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:28.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.742+0000 7f7151aa5640 1 -- 172.21.15.32:0/3495206423 >> v1:172.21.15.32:6789/0 conn(0x7f714c0cf990 legacy=0x7f714c06c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:28.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.742+0000 7f7150aa3640 1 -- 172.21.15.32:0/3495206423 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 712486882 0 0) 0x7f714c0cfe50 con 0x7f714c0cf990 2024-04-24T23:06:28.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.742+0000 7f7151aa5640 1 -- 172.21.15.32:0/3495206423 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f714c166ec0 con 0x7f714c162da0 2024-04-24T23:06:28.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.743+0000 7f7151aa5640 1 --2- 172.21.15.32:0/3495206423 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f714c162da0 0x7f714c0cf450 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f714c06e520 tx=0x7f714802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5232aa9c2ff48001 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:28.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.743+0000 7f7150aa3640 1 -- 172.21.15.32:0/3495206423 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7148037070 con 0x7f714c162da0 2024-04-24T23:06:28.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.743+0000 7f7150aa3640 1 -- 172.21.15.32:0/3495206423 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7148032440 con 0x7f714c162da0 2024-04-24T23:06:28.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.743+0000 7f7150aa3640 1 -- 172.21.15.32:0/3495206423 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714802ae60 con 0x7f714c162da0 2024-04-24T23:06:28.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.744+0000 7f7153d09640 1 -- 172.21.15.32:0/3495206423 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f714c162da0 msgr2=0x7f714c0cf450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:28.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.744+0000 7f7153d09640 1 --2- 172.21.15.32:0/3495206423 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f714c162da0 0x7f714c0cf450 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f714c06e520 tx=0x7f714802a110 comp rx=0 tx=0).stop 2024-04-24T23:06:28.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.744+0000 7f7153d09640 1 -- 172.21.15.32:0/3495206423 shutdown_connections 2024-04-24T23:06:28.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.744+0000 7f7153d09640 1 --2- 172.21.15.32:0/3495206423 >> v2:172.21.15.121:3300/0 conn(0x7f714c06cd00 0x7f714c0b3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:28.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.744+0000 7f7153d09640 1 --2- 172.21.15.32:0/3495206423 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f714c162da0 0x7f714c0cf450 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:28.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.744+0000 7f7153d09640 1 -- 172.21.15.32:0/3495206423 >> 172.21.15.32:0/3495206423 conn(0x7f714c06b110 msgr2=0x7f714c06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:28.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.744+0000 7f7153d09640 1 -- 172.21.15.32:0/3495206423 shutdown_connections 2024-04-24T23:06:28.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.744+0000 7f7153d09640 1 -- 172.21.15.32:0/3495206423 wait complete. 2024-04-24T23:06:28.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.745+0000 7f7153d09640 1 Processor -- start 2024-04-24T23:06:28.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.745+0000 7f7153d09640 1 -- start start 2024-04-24T23:06:28.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.746+0000 7f7153d09640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f714c06cd00 0x7f714c15fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:28.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.746+0000 7f7153d09640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f714c0cf990 0x7f714c160b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:28.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.746+0000 7f7153d09640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f714c162da0 0x7f714c161200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:28.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.746+0000 7f7153d09640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f714c1673e0 con 0x7f714c06cd00 2024-04-24T23:06:28.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.746+0000 7f7153d09640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f714c0afcf0 con 0x7f714c162da0 2024-04-24T23:06:28.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.746+0000 7f7153d09640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f714c167560 con 0x7f714c0cf990 2024-04-24T23:06:28.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.746+0000 7f7151aa5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f714c06cd00 0x7f714c15fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:28.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.746+0000 7f7151aa5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f714c06cd00 0x7f714c15fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53738/0 (socket says 172.21.15.32:53738) 2024-04-24T23:06:28.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.746+0000 7f7151aa5640 1 -- 172.21.15.32:0/4149365913 learned_addr learned my addr 172.21.15.32:0/4149365913 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:28.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.746+0000 7f71522a6640 1 --2- 172.21.15.32:0/4149365913 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f714c162da0 0x7f714c161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:28.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.746+0000 7f71512a4640 1 --2- 172.21.15.32:0/4149365913 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f714c0cf990 0x7f714c160b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:28.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.746+0000 7f7151aa5640 1 -- 172.21.15.32:0/4149365913 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f714c0cf990 msgr2=0x7f714c160b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:28.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.746+0000 7f7151aa5640 1 --2- 172.21.15.32:0/4149365913 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f714c0cf990 0x7f714c160b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:28.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.746+0000 7f7151aa5640 1 -- 172.21.15.32:0/4149365913 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f714c162da0 msgr2=0x7f714c161200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:28.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.746+0000 7f7151aa5640 1 --2- 172.21.15.32:0/4149365913 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f714c162da0 0x7f714c161200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:28.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.746+0000 7f7151aa5640 1 -- 172.21.15.32:0/4149365913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f714c160100 con 0x7f714c06cd00 2024-04-24T23:06:28.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.746+0000 7f7151aa5640 1 --2- 172.21.15.32:0/4149365913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f714c06cd00 0x7f714c15fa70 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f7148000c00 tx=0x7f71480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:28.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.747+0000 7f71427fc640 1 -- 172.21.15.32:0/4149365913 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7148037050 con 0x7f714c06cd00 2024-04-24T23:06:28.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.747+0000 7f71427fc640 1 -- 172.21.15.32:0/4149365913 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7148038040 con 0x7f714c06cd00 2024-04-24T23:06:28.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.747+0000 7f7153d09640 1 -- 172.21.15.32:0/4149365913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f714c163220 con 0x7f714c06cd00 2024-04-24T23:06:28.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.747+0000 7f71427fc640 1 -- 172.21.15.32:0/4149365913 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7148004470 con 0x7f714c06cd00 2024-04-24T23:06:28.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.747+0000 7f7153d09640 1 -- 172.21.15.32:0/4149365913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f714c1637e0 con 0x7f714c06cd00 2024-04-24T23:06:28.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.748+0000 7f7153d09640 1 -- 172.21.15.32:0/4149365913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f714c068430 con 0x7f714c06cd00 2024-04-24T23:06:28.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.748+0000 7f71427fc640 1 -- 172.21.15.32:0/4149365913 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7148030900 con 0x7f714c06cd00 2024-04-24T23:06:28.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.748+0000 7f71427fc640 1 --2- 172.21.15.32:0/4149365913 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f713006be30 0x7f713006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:28.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.749+0000 7f71427fc640 1 -- 172.21.15.32:0/4149365913 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 4857+0+0 (secure 0 0 0) 0x7f71480ab500 con 0x7f714c06cd00 2024-04-24T23:06:28.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.749+0000 7f71512a4640 1 --2- 172.21.15.32:0/4149365913 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f713006be30 0x7f713006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:28.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.749+0000 7f71512a4640 1 --2- 172.21.15.32:0/4149365913 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f713006be30 0x7f713006e2f0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f714c0b0680 tx=0x7f713c007d50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:28.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.753+0000 7f71427fc640 1 -- 172.21.15.32:0/4149365913 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7148079d60 con 0x7f714c06cd00 2024-04-24T23:06:28.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.935+0000 7f7153d09640 1 -- 172.21.15.32:0/4149365913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f714c0737a0 con 0x7f714c06cd00 2024-04-24T23:06:28.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.935+0000 7f71427fc640 1 -- 172.21.15.32:0/4149365913 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v152) ==== 93+0+10 (secure 0 0 0) 0x7f714807e060 con 0x7f714c06cd00 2024-04-24T23:06:28.938 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T23:06:28.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.938+0000 7f7153d09640 1 -- 172.21.15.32:0/4149365913 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f713006be30 msgr2=0x7f713006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:28.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.938+0000 7f7153d09640 1 --2- 172.21.15.32:0/4149365913 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f713006be30 0x7f713006e2f0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f714c0b0680 tx=0x7f713c007d50 comp rx=0 tx=0).stop 2024-04-24T23:06:28.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.938+0000 7f7153d09640 1 -- 172.21.15.32:0/4149365913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f714c06cd00 msgr2=0x7f714c15fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:28.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.938+0000 7f7153d09640 1 --2- 172.21.15.32:0/4149365913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f714c06cd00 0x7f714c15fa70 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f7148000c00 tx=0x7f71480046c0 comp rx=0 tx=0).stop 2024-04-24T23:06:28.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.938+0000 7f7153d09640 1 -- 172.21.15.32:0/4149365913 shutdown_connections 2024-04-24T23:06:28.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.938+0000 7f7153d09640 1 --2- 172.21.15.32:0/4149365913 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f713006be30 0x7f713006e2f0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:28.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.938+0000 7f7153d09640 1 --2- 172.21.15.32:0/4149365913 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f714c162da0 0x7f714c161200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:28.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.938+0000 7f7153d09640 1 --2- 172.21.15.32:0/4149365913 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f714c0cf990 0x7f714c160b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:28.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.938+0000 7f7153d09640 1 --2- 172.21.15.32:0/4149365913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f714c06cd00 0x7f714c15fa70 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:28.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.938+0000 7f7153d09640 1 -- 172.21.15.32:0/4149365913 >> 172.21.15.32:0/4149365913 conn(0x7f714c06b110 msgr2=0x7f714c0b1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:28.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.938+0000 7f7153d09640 1 -- 172.21.15.32:0/4149365913 shutdown_connections 2024-04-24T23:06:28.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:28.938+0000 7f7153d09640 1 -- 172.21.15.32:0/4149365913 wait complete. 2024-04-24T23:06:28.950 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-24T23:06:29.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.049+0000 7f075be38640 1 Processor -- start 2024-04-24T23:06:29.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.049+0000 7f075be38640 1 -- start start 2024-04-24T23:06:29.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.050+0000 7f075be38640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f075415e9b0 0x7f07540d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:29.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.050+0000 7f075be38640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f07540d4210 0x7f075406b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:29.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.050+0000 7f075be38640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07540d3170 con 0x7f07540ba0e0 2024-04-24T23:06:29.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.050+0000 7f075be38640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f07540b8900 con 0x7f075415e9b0 2024-04-24T23:06:29.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.050+0000 7f075be38640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f07540d3450 con 0x7f07540d4210 2024-04-24T23:06:29.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.051+0000 7f07593d3640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f07540d4210 0x7f075406b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:29.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.051+0000 7f075a3d5640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f07540ba0e0 0x7f0754071730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:45402/0 (socket says 172.21.15.32:45402) 2024-04-24T23:06:29.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.051+0000 7f07593d3640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f07540d4210 0x7f075406b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.32:60622/0 (socket says 172.21.15.32:60622) 2024-04-24T23:06:29.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.051+0000 7f075a3d5640 1 -- 172.21.15.32:0/2614994075 learned_addr learned my addr 172.21.15.32:0/2614994075 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:06:29.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.051+0000 7f0759bd4640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f075415e9b0 0x7f07540d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:29.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.051+0000 7f07593d3640 1 -- 172.21.15.32:0/2614994075 >> v2:172.21.15.121:3300/0 conn(0x7f075415e9b0 msgr2=0x7f07540d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:29.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.051+0000 7f07593d3640 1 --2- 172.21.15.32:0/2614994075 >> v2:172.21.15.121:3300/0 conn(0x7f075415e9b0 0x7f07540d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:29.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.051+0000 7f07593d3640 1 -- 172.21.15.32:0/2614994075 >> v1:172.21.15.121:6789/0 conn(0x7f07540ba0e0 legacy=0x7f0754071730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:29.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.051+0000 7f07593d3640 1 -- 172.21.15.32:0/2614994075 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07540d3730 con 0x7f07540d4210 2024-04-24T23:06:29.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.051+0000 7f0758bd2640 1 -- 172.21.15.32:0/2614994075 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1531902547 0 0) 0x7f07540d3170 con 0x7f07540ba0e0 2024-04-24T23:06:29.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.051+0000 7f0759bd4640 1 --2- 172.21.15.32:0/2614994075 >> v2:172.21.15.121:3300/0 conn(0x7f075415e9b0 0x7f07540d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:06:29.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.052+0000 7f07593d3640 1 --2- 172.21.15.32:0/2614994075 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f07540d4210 0x7f075406b110 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f07480097a0 tx=0x7f074802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff7bba3baa14618b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:29.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.052+0000 7f0758bd2640 1 -- 172.21.15.32:0/2614994075 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0748037070 con 0x7f07540d4210 2024-04-24T23:06:29.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.052+0000 7f0758bd2640 1 -- 172.21.15.32:0/2614994075 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0748032440 con 0x7f07540d4210 2024-04-24T23:06:29.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.052+0000 7f0758bd2640 1 -- 172.21.15.32:0/2614994075 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0748032810 con 0x7f07540d4210 2024-04-24T23:06:29.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.053+0000 7f075be38640 1 -- 172.21.15.32:0/2614994075 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f07540d4210 msgr2=0x7f075406b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:29.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.053+0000 7f075be38640 1 --2- 172.21.15.32:0/2614994075 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f07540d4210 0x7f075406b110 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f07480097a0 tx=0x7f074802a180 comp rx=0 tx=0).stop 2024-04-24T23:06:29.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.053+0000 7f075be38640 1 -- 172.21.15.32:0/2614994075 shutdown_connections 2024-04-24T23:06:29.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.053+0000 7f075be38640 1 --2- 172.21.15.32:0/2614994075 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f07540d4210 0x7f075406b110 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:29.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.053+0000 7f075be38640 1 --2- 172.21.15.32:0/2614994075 >> v2:172.21.15.121:3300/0 conn(0x7f075415e9b0 0x7f07540d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:29.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.053+0000 7f075be38640 1 -- 172.21.15.32:0/2614994075 >> 172.21.15.32:0/2614994075 conn(0x7f07540d11d0 msgr2=0x7f07540d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:29.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.053+0000 7f075be38640 1 -- 172.21.15.32:0/2614994075 shutdown_connections 2024-04-24T23:06:29.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.053+0000 7f075be38640 1 -- 172.21.15.32:0/2614994075 wait complete. 2024-04-24T23:06:29.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.054+0000 7f075be38640 1 Processor -- start 2024-04-24T23:06:29.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.054+0000 7f075be38640 1 -- start start 2024-04-24T23:06:29.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.055+0000 7f075be38640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f07540ba0e0 0x7f075415d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:29.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.055+0000 7f075be38640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f07540d4210 0x7f075415d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:29.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.055+0000 7f075be38640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f075415e9b0 0x7f075415fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:29.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.055+0000 7f075be38640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f07540d2400 con 0x7f075415e9b0 2024-04-24T23:06:29.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.055+0000 7f075be38640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f07540d3cf0 con 0x7f07540ba0e0 2024-04-24T23:06:29.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.055+0000 7f075be38640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f075406b650 con 0x7f07540d4210 2024-04-24T23:06:29.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.055+0000 7f075a3d5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f075415e9b0 0x7f075415fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:29.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.055+0000 7f075a3d5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f075415e9b0 0x7f075415fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53740/0 (socket says 172.21.15.32:53740) 2024-04-24T23:06:29.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.055+0000 7f075a3d5640 1 -- 172.21.15.32:0/2816772525 learned_addr learned my addr 172.21.15.32:0/2816772525 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:29.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.055+0000 7f0759bd4640 1 --2- 172.21.15.32:0/2816772525 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f07540ba0e0 0x7f075415d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:29.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.055+0000 7f07593d3640 1 --2- 172.21.15.32:0/2816772525 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f07540d4210 0x7f075415d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:29.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.055+0000 7f075a3d5640 1 -- 172.21.15.32:0/2816772525 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f07540d4210 msgr2=0x7f075415d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:29.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.055+0000 7f075a3d5640 1 --2- 172.21.15.32:0/2816772525 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f07540d4210 0x7f075415d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:29.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.055+0000 7f075a3d5640 1 -- 172.21.15.32:0/2816772525 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f07540ba0e0 msgr2=0x7f075415d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:29.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.055+0000 7f075a3d5640 1 --2- 172.21.15.32:0/2816772525 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f07540ba0e0 0x7f075415d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:29.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.055+0000 7f075a3d5640 1 -- 172.21.15.32:0/2816772525 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07541476a0 con 0x7f075415e9b0 2024-04-24T23:06:29.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.055+0000 7f0759bd4640 1 --2- 172.21.15.32:0/2816772525 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f07540ba0e0 0x7f075415d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:06:29.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.055+0000 7f07593d3640 1 --2- 172.21.15.32:0/2816772525 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f07540d4210 0x7f075415d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:06:29.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.055+0000 7f075a3d5640 1 --2- 172.21.15.32:0/2816772525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f075415e9b0 0x7f075415fde0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f074c00c7a0 tx=0x7f074c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:29.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.055+0000 7f07527fc640 1 -- 172.21.15.32:0/2816772525 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074c018070 con 0x7f075415e9b0 2024-04-24T23:06:29.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.055+0000 7f07527fc640 1 -- 172.21.15.32:0/2816772525 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f074c00a640 con 0x7f075415e9b0 2024-04-24T23:06:29.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.055+0000 7f07527fc640 1 -- 172.21.15.32:0/2816772525 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074c00aab0 con 0x7f075415e9b0 2024-04-24T23:06:29.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.056+0000 7f075be38640 1 -- 172.21.15.32:0/2816772525 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07540cce00 con 0x7f075415e9b0 2024-04-24T23:06:29.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.056+0000 7f075be38640 1 -- 172.21.15.32:0/2816772525 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07541609c0 con 0x7f075415e9b0 2024-04-24T23:06:29.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.056+0000 7f075be38640 1 -- 172.21.15.32:0/2816772525 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0754162110 con 0x7f075415e9b0 2024-04-24T23:06:29.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.057+0000 7f07527fc640 1 -- 172.21.15.32:0/2816772525 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f074c01b070 con 0x7f075415e9b0 2024-04-24T23:06:29.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.057+0000 7f07527fc640 1 --2- 172.21.15.32:0/2816772525 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f074006be30 0x7f074006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:29.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.057+0000 7f07527fc640 1 -- 172.21.15.32:0/2816772525 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 4857+0+0 (secure 0 0 0) 0x7f074c08a160 con 0x7f075415e9b0 2024-04-24T23:06:29.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.058+0000 7f0759bd4640 1 --2- 172.21.15.32:0/2816772525 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f074006be30 0x7f074006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:29.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.058+0000 7f0759bd4640 1 --2- 172.21.15.32:0/2816772525 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f074006be30 0x7f074006e2f0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f0744009af0 tx=0x7f0744008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:29.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.062+0000 7f07527fc640 1 -- 172.21.15.32:0/2816772525 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0754162110 con 0x7f075415e9b0 2024-04-24T23:06:29.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.249+0000 7f075be38640 1 -- 172.21.15.32:0/2816772525 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f075415d250 con 0x7f075415e9b0 2024-04-24T23:06:29.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.249+0000 7f07527fc640 1 -- 172.21.15.32:0/2816772525 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v152) ==== 84+0+65 (secure 0 0 0) 0x7f07541476a0 con 0x7f075415e9b0 2024-04-24T23:06:29.251 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:06:29.251 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2024-04-24T23:06:29.251 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:06:29.251 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2024-04-24T23:06:29.251 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": ".mgr" 2024-04-24T23:06:29.252 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:06:29.252 INFO:teuthology.orchestra.run.smithi032.stdout:] 2024-04-24T23:06:29.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.251+0000 7f075be38640 1 -- 172.21.15.32:0/2816772525 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f074006be30 msgr2=0x7f074006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:29.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.251+0000 7f075be38640 1 --2- 172.21.15.32:0/2816772525 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f074006be30 0x7f074006e2f0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f0744009af0 tx=0x7f0744008040 comp rx=0 tx=0).stop 2024-04-24T23:06:29.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.251+0000 7f075be38640 1 -- 172.21.15.32:0/2816772525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f075415e9b0 msgr2=0x7f075415fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:29.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.251+0000 7f075be38640 1 --2- 172.21.15.32:0/2816772525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f075415e9b0 0x7f075415fde0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f074c00c7a0 tx=0x7f074c00cc60 comp rx=0 tx=0).stop 2024-04-24T23:06:29.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.251+0000 7f075be38640 1 -- 172.21.15.32:0/2816772525 shutdown_connections 2024-04-24T23:06:29.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.251+0000 7f075be38640 1 --2- 172.21.15.32:0/2816772525 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f074006be30 0x7f074006e2f0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:29.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.251+0000 7f075be38640 1 --2- 172.21.15.32:0/2816772525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f075415e9b0 0x7f075415fde0 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:29.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.251+0000 7f075be38640 1 --2- 172.21.15.32:0/2816772525 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f07540d4210 0x7f075415d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:29.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.251+0000 7f075be38640 1 --2- 172.21.15.32:0/2816772525 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f07540ba0e0 0x7f075415d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:29.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.251+0000 7f075be38640 1 -- 172.21.15.32:0/2816772525 >> 172.21.15.32:0/2816772525 conn(0x7f07540d11d0 msgr2=0x7f07540d15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:29.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.251+0000 7f075be38640 1 -- 172.21.15.32:0/2816772525 shutdown_connections 2024-04-24T23:06:29.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.251+0000 7f075be38640 1 -- 172.21.15.32:0/2816772525 wait complete. 2024-04-24T23:06:29.264 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-24T23:06:29.264 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-24T23:06:29.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.367+0000 7f91be237640 1 Processor -- start 2024-04-24T23:06:29.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.367+0000 7f91be237640 1 -- start start 2024-04-24T23:06:29.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.367+0000 7f91be237640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f91b815b580 0x7f91b80cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:29.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.368+0000 7f91be237640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91b815bde0 con 0x7f91b80b58e0 2024-04-24T23:06:29.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.368+0000 7f91be237640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91b815c0c0 con 0x7f91b815b960 2024-04-24T23:06:29.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.368+0000 7f91be237640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f91b815c9f0 con 0x7f91b815b580 2024-04-24T23:06:29.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.368+0000 7f91b77fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f91b815b580 0x7f91b80cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:29.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.368+0000 7f91b7fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f91b80b58e0 0x7f91b815f340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42810/0 (socket says 172.21.15.32:42810) 2024-04-24T23:06:29.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.368+0000 7f91b7fff640 1 -- 172.21.15.32:0/3144516360 learned_addr learned my addr 172.21.15.32:0/3144516360 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:06:29.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.368+0000 7f91b67fc640 1 -- 172.21.15.32:0/3144516360 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3352805283 0 0) 0x7f91b815bde0 con 0x7f91b80b58e0 2024-04-24T23:06:29.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.368+0000 7f91b67fc640 1 -- 172.21.15.32:0/3144516360 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f91a0003520 con 0x7f91b80b58e0 2024-04-24T23:06:29.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.368+0000 7f91b77fe640 1 -- 172.21.15.32:0/3144516360 >> v1:172.21.15.121:6789/0 conn(0x7f91b815b960 legacy=0x7f91b806cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:29.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.368+0000 7f91b67fc640 1 -- 172.21.15.32:0/3144516360 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 887613778 0 0) 0x7f91b815c0c0 con 0x7f91b815b960 2024-04-24T23:06:29.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.368+0000 7f91b77fe640 1 -- 172.21.15.32:0/3144516360 >> v1:172.21.15.32:6789/0 conn(0x7f91b80b58e0 legacy=0x7f91b815f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:29.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.368+0000 7f91b77fe640 1 -- 172.21.15.32:0/3144516360 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91b815c3a0 con 0x7f91b815b580 2024-04-24T23:06:29.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.369+0000 7f91b77fe640 1 --2- 172.21.15.32:0/3144516360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91b815b580 0x7f91b80cde80 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f91a8009d90 tx=0x7f91a802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1a6f7e8ab2255b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:29.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.369+0000 7f91b67fc640 1 -- 172.21.15.32:0/3144516360 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a8007740 con 0x7f91b815b580 2024-04-24T23:06:29.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.369+0000 7f91b67fc640 1 -- 172.21.15.32:0/3144516360 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91a80078e0 con 0x7f91b815b580 2024-04-24T23:06:29.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.369+0000 7f91b67fc640 1 -- 172.21.15.32:0/3144516360 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a8007cb0 con 0x7f91b815b580 2024-04-24T23:06:29.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.369+0000 7f91be237640 1 -- 172.21.15.32:0/3144516360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91b815b580 msgr2=0x7f91b80cde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:29.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.369+0000 7f91be237640 1 --2- 172.21.15.32:0/3144516360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91b815b580 0x7f91b80cde80 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f91a8009d90 tx=0x7f91a802a6f0 comp rx=0 tx=0).stop 2024-04-24T23:06:29.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.369+0000 7f91be237640 1 -- 172.21.15.32:0/3144516360 shutdown_connections 2024-04-24T23:06:29.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.369+0000 7f91be237640 1 --2- 172.21.15.32:0/3144516360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91b815b580 0x7f91b80cde80 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:29.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.369+0000 7f91be237640 1 -- 172.21.15.32:0/3144516360 >> 172.21.15.32:0/3144516360 conn(0x7f91b80b1f10 msgr2=0x7f91b80b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:29.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.370+0000 7f91be237640 1 -- 172.21.15.32:0/3144516360 shutdown_connections 2024-04-24T23:06:29.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.370+0000 7f91be237640 1 -- 172.21.15.32:0/3144516360 wait complete. 2024-04-24T23:06:29.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.371+0000 7f91be237640 1 Processor -- start 2024-04-24T23:06:29.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.371+0000 7f91be237640 1 -- start start 2024-04-24T23:06:29.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.371+0000 7f91be237640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91b80b58e0 0x7f91b815d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:29.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.371+0000 7f91be237640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f91b815b580 0x7f91b815db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:29.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.371+0000 7f91be237640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91b815b960 0x7f91b81415d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:29.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.371+0000 7f91be237640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f91b806c7a0 con 0x7f91b815b960 2024-04-24T23:06:29.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.371+0000 7f91be237640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f91b80ce3c0 con 0x7f91b80b58e0 2024-04-24T23:06:29.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.371+0000 7f91be237640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f91b806c920 con 0x7f91b815b580 2024-04-24T23:06:29.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.372+0000 7f91b7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91b815b960 0x7f91b81415d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:29.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.372+0000 7f91b7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91b815b960 0x7f91b81415d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53762/0 (socket says 172.21.15.32:53762) 2024-04-24T23:06:29.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.372+0000 7f91b7fff640 1 -- 172.21.15.32:0/597192082 learned_addr learned my addr 172.21.15.32:0/597192082 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:29.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.372+0000 7f91b77fe640 1 --2- 172.21.15.32:0/597192082 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91b80b58e0 0x7f91b815d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:29.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.372+0000 7f91b6ffd640 1 --2- 172.21.15.32:0/597192082 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f91b815b580 0x7f91b815db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:29.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.372+0000 7f91b7fff640 1 -- 172.21.15.32:0/597192082 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f91b815b580 msgr2=0x7f91b815db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:29.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.372+0000 7f91b7fff640 1 --2- 172.21.15.32:0/597192082 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f91b815b580 0x7f91b815db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:29.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.372+0000 7f91b7fff640 1 -- 172.21.15.32:0/597192082 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91b80b58e0 msgr2=0x7f91b815d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:29.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.372+0000 7f91b7fff640 1 --2- 172.21.15.32:0/597192082 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91b80b58e0 0x7f91b815d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:29.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.372+0000 7f91b7fff640 1 -- 172.21.15.32:0/597192082 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91b81f0550 con 0x7f91b815b960 2024-04-24T23:06:29.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.372+0000 7f91b6ffd640 1 --2- 172.21.15.32:0/597192082 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f91b815b580 0x7f91b815db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:06:29.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.372+0000 7f91b7fff640 1 --2- 172.21.15.32:0/597192082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91b815b960 0x7f91b81415d0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f91b000c3f0 tx=0x7f91b000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:29.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.372+0000 7f919ffff640 1 -- 172.21.15.32:0/597192082 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b0017070 con 0x7f91b815b960 2024-04-24T23:06:29.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.372+0000 7f919ffff640 1 -- 172.21.15.32:0/597192082 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91b0005ce0 con 0x7f91b815b960 2024-04-24T23:06:29.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.372+0000 7f91be237640 1 -- 172.21.15.32:0/597192082 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91b81f0830 con 0x7f91b815b960 2024-04-24T23:06:29.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.372+0000 7f919ffff640 1 -- 172.21.15.32:0/597192082 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b0007310 con 0x7f91b815b960 2024-04-24T23:06:29.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.372+0000 7f91be237640 1 -- 172.21.15.32:0/597192082 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91b81f0df0 con 0x7f91b815b960 2024-04-24T23:06:29.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.374+0000 7f91be237640 1 -- 172.21.15.32:0/597192082 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91b8068430 con 0x7f91b815b960 2024-04-24T23:06:29.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.374+0000 7f919ffff640 1 -- 172.21.15.32:0/597192082 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f91b001a070 con 0x7f91b815b960 2024-04-24T23:06:29.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.374+0000 7f919ffff640 1 --2- 172.21.15.32:0/597192082 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f91a406be30 0x7f91a406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:29.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.374+0000 7f919ffff640 1 -- 172.21.15.32:0/597192082 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 4857+0+0 (secure 0 0 0) 0x7f91b0089e40 con 0x7f91b815b960 2024-04-24T23:06:29.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.375+0000 7f91b77fe640 1 --2- 172.21.15.32:0/597192082 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f91a406be30 0x7f91a406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:29.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.375+0000 7f91b77fe640 1 --2- 172.21.15.32:0/597192082 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f91a406be30 0x7f91a406e2f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f91a8034000 tx=0x7f91a802f000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:29.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.379+0000 7f919ffff640 1 -- 172.21.15.32:0/597192082 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f91b0058bd0 con 0x7f91b815b960 2024-04-24T23:06:29.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.560+0000 7f91be237640 1 -- 172.21.15.32:0/597192082 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f91b80af4f0 con 0x7f91b815b960 2024-04-24T23:06:29.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.916+0000 7f919ffff640 1 -- 172.21.15.32:0/597192082 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v153) ==== 138+0+0 (secure 0 0 0) 0x7f91b00584d0 con 0x7f91b815b960 2024-04-24T23:06:29.918 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2024-04-24T23:06:29.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.919+0000 7f91be237640 1 -- 172.21.15.32:0/597192082 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f91a406be30 msgr2=0x7f91a406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:29.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.919+0000 7f91be237640 1 --2- 172.21.15.32:0/597192082 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f91a406be30 0x7f91a406e2f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f91a8034000 tx=0x7f91a802f000 comp rx=0 tx=0).stop 2024-04-24T23:06:29.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.919+0000 7f91be237640 1 -- 172.21.15.32:0/597192082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91b815b960 msgr2=0x7f91b81415d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:29.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.919+0000 7f91be237640 1 --2- 172.21.15.32:0/597192082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91b815b960 0x7f91b81415d0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f91b000c3f0 tx=0x7f91b000c8b0 comp rx=0 tx=0).stop 2024-04-24T23:06:29.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.919+0000 7f91be237640 1 -- 172.21.15.32:0/597192082 shutdown_connections 2024-04-24T23:06:29.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.919+0000 7f91be237640 1 --2- 172.21.15.32:0/597192082 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f91a406be30 0x7f91a406e2f0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:29.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.919+0000 7f91be237640 1 --2- 172.21.15.32:0/597192082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91b815b960 0x7f91b81415d0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:29.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.919+0000 7f91be237640 1 --2- 172.21.15.32:0/597192082 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f91b815b580 0x7f91b815db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:29.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.919+0000 7f91be237640 1 --2- 172.21.15.32:0/597192082 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91b80b58e0 0x7f91b815d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:29.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.919+0000 7f91be237640 1 -- 172.21.15.32:0/597192082 >> 172.21.15.32:0/597192082 conn(0x7f91b80b1f10 msgr2=0x7f91b80ce570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:29.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.919+0000 7f91be237640 1 -- 172.21.15.32:0/597192082 shutdown_connections 2024-04-24T23:06:29.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:29.919+0000 7f91be237640 1 -- 172.21.15.32:0/597192082 wait complete. 2024-04-24T23:06:29.934 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-24T23:06:30.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.042+0000 7fd1b81fb640 1 Processor -- start 2024-04-24T23:06:30.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.042+0000 7fd1b81fb640 1 -- start start 2024-04-24T23:06:30.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.043+0000 7fd1b81fb640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd1b006c5f0 0x7fd1b006c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:30.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.043+0000 7fd1b81fb640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd1b00b44d0 0x7fd1b00b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:30.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.043+0000 7fd1b81fb640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd1b00af890 0x7fd1b00cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:30.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.043+0000 7fd1b81fb640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fd1b00b6430 con 0x7fd1b006c5f0 2024-04-24T23:06:30.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.043+0000 7fd1b81fb640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fd1b0156f80 con 0x7fd1b00af890 2024-04-24T23:06:30.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.043+0000 7fd1b81fb640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd1b00b6150 con 0x7fd1b00b44d0 2024-04-24T23:06:30.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.043+0000 7fd1b5f97640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd1b006c5f0 0x7fd1b006c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:30.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.043+0000 7fd1b5f97640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd1b006c5f0 0x7fd1b006c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48400/0 (socket says 172.21.15.32:48400) 2024-04-24T23:06:30.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.043+0000 7fd1b5f97640 1 -- 172.21.15.32:0/743780001 learned_addr learned my addr 172.21.15.32:0/743780001 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:30.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.043+0000 7fd1b6798640 1 --2- 172.21.15.32:0/743780001 >> v2:172.21.15.121:3300/0 conn(0x7fd1b00af890 0x7fd1b00cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:30.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.043+0000 7fd1b5796640 1 --2- 172.21.15.32:0/743780001 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd1b00b44d0 0x7fd1b00b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:30.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.043+0000 7fd1b5f97640 1 -- 172.21.15.32:0/743780001 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd1b00b44d0 msgr2=0x7fd1b00b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:30.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.043+0000 7fd1b5f97640 1 --2- 172.21.15.32:0/743780001 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd1b00b44d0 0x7fd1b00b5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:30.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.043+0000 7fd1b5f97640 1 -- 172.21.15.32:0/743780001 >> v2:172.21.15.121:3300/0 conn(0x7fd1b00af890 msgr2=0x7fd1b00cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:30.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.043+0000 7fd1b5f97640 1 --2- 172.21.15.32:0/743780001 >> v2:172.21.15.121:3300/0 conn(0x7fd1b00af890 0x7fd1b00cef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:30.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.043+0000 7fd1b5f97640 1 -- 172.21.15.32:0/743780001 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1b00b39f0 con 0x7fd1b006c5f0 2024-04-24T23:06:30.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.044+0000 7fd1b5f97640 1 --2- 172.21.15.32:0/743780001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1b006c5f0 0x7fd1b006c9d0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fd1b00aff30 tx=0x7fd1ac02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae9517b7c23e81cf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:30.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.044+0000 7fd1b4f95640 1 -- 172.21.15.32:0/743780001 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac037070 con 0x7fd1b006c5f0 2024-04-24T23:06:30.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.044+0000 7fd1b4f95640 1 -- 172.21.15.32:0/743780001 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1ac032440 con 0x7fd1b006c5f0 2024-04-24T23:06:30.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.044+0000 7fd1b4f95640 1 -- 172.21.15.32:0/743780001 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac032810 con 0x7fd1b006c5f0 2024-04-24T23:06:30.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.045+0000 7fd1b81fb640 1 -- 172.21.15.32:0/743780001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1b006c5f0 msgr2=0x7fd1b006c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:30.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.045+0000 7fd1b81fb640 1 --2- 172.21.15.32:0/743780001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1b006c5f0 0x7fd1b006c9d0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fd1b00aff30 tx=0x7fd1ac02a140 comp rx=0 tx=0).stop 2024-04-24T23:06:30.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.045+0000 7fd1b81fb640 1 -- 172.21.15.32:0/743780001 shutdown_connections 2024-04-24T23:06:30.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.045+0000 7fd1b81fb640 1 --2- 172.21.15.32:0/743780001 >> v2:172.21.15.121:3300/0 conn(0x7fd1b00af890 0x7fd1b00cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:30.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.045+0000 7fd1b81fb640 1 --2- 172.21.15.32:0/743780001 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd1b00b44d0 0x7fd1b00b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:30.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.045+0000 7fd1b81fb640 1 --2- 172.21.15.32:0/743780001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1b006c5f0 0x7fd1b006c9d0 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:30.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.045+0000 7fd1b81fb640 1 -- 172.21.15.32:0/743780001 >> 172.21.15.32:0/743780001 conn(0x7fd1b006b110 msgr2=0x7fd1b006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:30.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.045+0000 7fd1b81fb640 1 -- 172.21.15.32:0/743780001 shutdown_connections 2024-04-24T23:06:30.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.045+0000 7fd1b81fb640 1 -- 172.21.15.32:0/743780001 wait complete. 2024-04-24T23:06:30.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.046+0000 7fd1b81fb640 1 Processor -- start 2024-04-24T23:06:30.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.046+0000 7fd1b81fb640 1 -- start start 2024-04-24T23:06:30.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.047+0000 7fd1b81fb640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1b006c5f0 0x7fd1b01583c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:30.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.047+0000 7fd1b81fb640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd1b00af890 0x7fd1b01f7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:30.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.047+0000 7fd1b81fb640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1b00b44d0 0x7fd1b0159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:30.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.047+0000 7fd1b81fb640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b006cf10 con 0x7fd1b00b44d0 2024-04-24T23:06:30.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.047+0000 7fd1b81fb640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b00b3fb0 con 0x7fd1b006c5f0 2024-04-24T23:06:30.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.047+0000 7fd1b81fb640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd1b00b5dd0 con 0x7fd1b00af890 2024-04-24T23:06:30.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.047+0000 7fd1b6798640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1b00b44d0 0x7fd1b0159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:30.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.047+0000 7fd1b6798640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1b00b44d0 0x7fd1b0159ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48408/0 (socket says 172.21.15.32:48408) 2024-04-24T23:06:30.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.047+0000 7fd1b6798640 1 -- 172.21.15.32:0/1809090102 learned_addr learned my addr 172.21.15.32:0/1809090102 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:30.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.047+0000 7fd1b5f97640 1 --2- 172.21.15.32:0/1809090102 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1b006c5f0 0x7fd1b01583c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:30.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.047+0000 7fd1b5796640 1 --2- 172.21.15.32:0/1809090102 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd1b00af890 0x7fd1b01f7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:30.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.047+0000 7fd1b6798640 1 -- 172.21.15.32:0/1809090102 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd1b00af890 msgr2=0x7fd1b01f7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:30.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.047+0000 7fd1b6798640 1 --2- 172.21.15.32:0/1809090102 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd1b00af890 0x7fd1b01f7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:30.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.047+0000 7fd1b6798640 1 -- 172.21.15.32:0/1809090102 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1b006c5f0 msgr2=0x7fd1b01583c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:30.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.047+0000 7fd1b6798640 1 --2- 172.21.15.32:0/1809090102 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1b006c5f0 0x7fd1b01583c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:30.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.047+0000 7fd1b6798640 1 -- 172.21.15.32:0/1809090102 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1b01f77b0 con 0x7fd1b00b44d0 2024-04-24T23:06:30.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.047+0000 7fd1b5796640 1 --2- 172.21.15.32:0/1809090102 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd1b00af890 0x7fd1b01f7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:06:30.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.047+0000 7fd1b5f97640 1 --2- 172.21.15.32:0/1809090102 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1b006c5f0 0x7fd1b01583c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:06:30.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.048+0000 7fd1b6798640 1 --2- 172.21.15.32:0/1809090102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1b00b44d0 0x7fd1b0159ee0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8007e60 tx=0x7fd1a800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:30.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.048+0000 7fd1a67fc640 1 -- 172.21.15.32:0/1809090102 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a8014420 con 0x7fd1b00b44d0 2024-04-24T23:06:30.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.048+0000 7fd1a67fc640 1 -- 172.21.15.32:0/1809090102 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1a8019070 con 0x7fd1b00b44d0 2024-04-24T23:06:30.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.048+0000 7fd1b81fb640 1 -- 172.21.15.32:0/1809090102 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1b01589a0 con 0x7fd1b00b44d0 2024-04-24T23:06:30.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.048+0000 7fd1a67fc640 1 -- 172.21.15.32:0/1809090102 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a8014820 con 0x7fd1b00b44d0 2024-04-24T23:06:30.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.048+0000 7fd1b81fb640 1 -- 172.21.15.32:0/1809090102 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1b0158f60 con 0x7fd1b00b44d0 2024-04-24T23:06:30.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.049+0000 7fd1b81fb640 1 -- 172.21.15.32:0/1809090102 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1b0068430 con 0x7fd1b00b44d0 2024-04-24T23:06:30.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.049+0000 7fd1a67fc640 1 -- 172.21.15.32:0/1809090102 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd1a80149c0 con 0x7fd1b00b44d0 2024-04-24T23:06:30.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.050+0000 7fd1a67fc640 1 --2- 172.21.15.32:0/1809090102 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd19406be30 0x7fd19406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:30.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.050+0000 7fd1a67fc640 1 -- 172.21.15.32:0/1809090102 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5227+0+0 (secure 0 0 0) 0x7fd1a808b4f0 con 0x7fd1b00b44d0 2024-04-24T23:06:30.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.050+0000 7fd1b5f97640 1 --2- 172.21.15.32:0/1809090102 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd19406be30 0x7fd19406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:30.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.051+0000 7fd1b5f97640 1 --2- 172.21.15.32:0/1809090102 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd19406be30 0x7fd19406e2f0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac034000 tx=0x7fd1ac02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:30.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.054+0000 7fd1a67fc640 1 -- 172.21.15.32:0/1809090102 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd1a805a080 con 0x7fd1b00b44d0 2024-04-24T23:06:30.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.236+0000 7fd1b81fb640 1 -- 172.21.15.32:0/1809090102 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fd1b00b5930 con 0x7fd1b00b44d0 2024-04-24T23:06:30.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.919+0000 7fd1a67fc640 1 -- 172.21.15.32:0/1809090102 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v154) ==== 170+0+0 (secure 0 0 0) 0x7fd1a8059980 con 0x7fd1b00b44d0 2024-04-24T23:06:30.921 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2024-04-24T23:06:30.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.922+0000 7fd1b81fb640 1 -- 172.21.15.32:0/1809090102 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd19406be30 msgr2=0x7fd19406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:30.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.922+0000 7fd1b81fb640 1 --2- 172.21.15.32:0/1809090102 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd19406be30 0x7fd19406e2f0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac034000 tx=0x7fd1ac02e000 comp rx=0 tx=0).stop 2024-04-24T23:06:30.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.922+0000 7fd1b81fb640 1 -- 172.21.15.32:0/1809090102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1b00b44d0 msgr2=0x7fd1b0159ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:30.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.922+0000 7fd1b81fb640 1 --2- 172.21.15.32:0/1809090102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1b00b44d0 0x7fd1b0159ee0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8007e60 tx=0x7fd1a800e4b0 comp rx=0 tx=0).stop 2024-04-24T23:06:30.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.922+0000 7fd1b81fb640 1 -- 172.21.15.32:0/1809090102 shutdown_connections 2024-04-24T23:06:30.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.922+0000 7fd1b81fb640 1 --2- 172.21.15.32:0/1809090102 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd19406be30 0x7fd19406e2f0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:30.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.922+0000 7fd1b81fb640 1 --2- 172.21.15.32:0/1809090102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1b00b44d0 0x7fd1b0159ee0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:30.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.922+0000 7fd1b81fb640 1 --2- 172.21.15.32:0/1809090102 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd1b00af890 0x7fd1b01f7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:30.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.922+0000 7fd1b81fb640 1 --2- 172.21.15.32:0/1809090102 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1b006c5f0 0x7fd1b01583c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:30.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.922+0000 7fd1b81fb640 1 -- 172.21.15.32:0/1809090102 >> 172.21.15.32:0/1809090102 conn(0x7fd1b006b110 msgr2=0x7fd1b00aff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:30.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.922+0000 7fd1b81fb640 1 -- 172.21.15.32:0/1809090102 shutdown_connections 2024-04-24T23:06:30.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:30.922+0000 7fd1b81fb640 1 -- 172.21.15.32:0/1809090102 wait complete. 2024-04-24T23:06:30.944 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-24T23:06:31.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.052+0000 7f23165a2640 1 Processor -- start 2024-04-24T23:06:31.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.052+0000 7f23165a2640 1 -- start start 2024-04-24T23:06:31.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.052+0000 7f23165a2640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f231015e9b0 0x7f23100d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:31.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.052+0000 7f23165a2640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f231006b820 0x7f23100b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:31.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.052+0000 7f23165a2640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23100d3170 con 0x7f23100d4210 2024-04-24T23:06:31.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.052+0000 7f23165a2640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f23100d3450 con 0x7f231015e9b0 2024-04-24T23:06:31.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.052+0000 7f23165a2640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f23100d3730 con 0x7f231006b820 2024-04-24T23:06:31.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.053+0000 7f2314b3f640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f231006b820 0x7f23100b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:31.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.053+0000 7f230ffff640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f231015e9b0 0x7f23100d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:31.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.053+0000 7f2314b3f640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f231006b820 0x7f23100b78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.32:56568/0 (socket says 172.21.15.32:56568) 2024-04-24T23:06:31.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.053+0000 7f2314b3f640 1 -- 172.21.15.32:0/3464732706 learned_addr learned my addr 172.21.15.32:0/3464732706 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:31.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.053+0000 7f2314b3f640 1 -- 172.21.15.32:0/3464732706 >> v2:172.21.15.121:3300/0 conn(0x7f231015e9b0 msgr2=0x7f23100d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:31.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.053+0000 7f2314b3f640 1 --2- 172.21.15.32:0/3464732706 >> v2:172.21.15.121:3300/0 conn(0x7f231015e9b0 0x7f23100d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:31.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.053+0000 7f2314b3f640 1 -- 172.21.15.32:0/3464732706 >> v1:172.21.15.121:6789/0 conn(0x7f23100d4210 legacy=0x7f231006b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:31.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.053+0000 7f2314b3f640 1 -- 172.21.15.32:0/3464732706 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23100d3a10 con 0x7f231006b820 2024-04-24T23:06:31.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.054+0000 7f2314b3f640 1 --2- 172.21.15.32:0/3464732706 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f231006b820 0x7f23100b78f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2308009e50 tx=0x7f230802a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbb0e22564ddf0c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:31.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.054+0000 7f230effd640 1 -- 172.21.15.32:0/3464732706 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2308036070 con 0x7f231006b820 2024-04-24T23:06:31.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.054+0000 7f230effd640 1 -- 172.21.15.32:0/3464732706 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2308031600 con 0x7f231006b820 2024-04-24T23:06:31.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.054+0000 7f230effd640 1 -- 172.21.15.32:0/3464732706 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23080319d0 con 0x7f231006b820 2024-04-24T23:06:31.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.055+0000 7f23165a2640 1 -- 172.21.15.32:0/3464732706 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f231006b820 msgr2=0x7f23100b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:31.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.055+0000 7f23165a2640 1 --2- 172.21.15.32:0/3464732706 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f231006b820 0x7f23100b78f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2308009e50 tx=0x7f230802a710 comp rx=0 tx=0).stop 2024-04-24T23:06:31.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.055+0000 7f23165a2640 1 -- 172.21.15.32:0/3464732706 shutdown_connections 2024-04-24T23:06:31.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.055+0000 7f23165a2640 1 --2- 172.21.15.32:0/3464732706 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f231006b820 0x7f23100b78f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:31.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.055+0000 7f23165a2640 1 --2- 172.21.15.32:0/3464732706 >> v2:172.21.15.121:3300/0 conn(0x7f231015e9b0 0x7f23100d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:31.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.055+0000 7f23165a2640 1 -- 172.21.15.32:0/3464732706 >> 172.21.15.32:0/3464732706 conn(0x7f23100d11d0 msgr2=0x7f23100d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:31.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.055+0000 7f23165a2640 1 -- 172.21.15.32:0/3464732706 shutdown_connections 2024-04-24T23:06:31.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.055+0000 7f23165a2640 1 -- 172.21.15.32:0/3464732706 wait complete. 2024-04-24T23:06:31.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.056+0000 7f23165a2640 1 Processor -- start 2024-04-24T23:06:31.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.056+0000 7f23165a2640 1 -- start start 2024-04-24T23:06:31.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.057+0000 7f23165a2640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f231006b820 0x7f23101542c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:31.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.057+0000 7f23165a2640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f23100d4210 0x7f23101f7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:31.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.057+0000 7f23165a2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f231015e9b0 0x7f23101f8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:31.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.057+0000 7f23165a2640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f23100ba0e0 con 0x7f231015e9b0 2024-04-24T23:06:31.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.057+0000 7f23165a2640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f23100d2400 con 0x7f231006b820 2024-04-24T23:06:31.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.057+0000 7f23165a2640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f23100ba260 con 0x7f23100d4210 2024-04-24T23:06:31.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.057+0000 7f2314b3f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f231015e9b0 0x7f23101f8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:31.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.057+0000 7f2314b3f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f231015e9b0 0x7f23101f8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48412/0 (socket says 172.21.15.32:48412) 2024-04-24T23:06:31.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.057+0000 7f2314b3f640 1 -- 172.21.15.32:0/2893867529 learned_addr learned my addr 172.21.15.32:0/2893867529 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:31.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.057+0000 7f230ffff640 1 --2- 172.21.15.32:0/2893867529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f231006b820 0x7f23101542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:31.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.057+0000 7f230f7fe640 1 --2- 172.21.15.32:0/2893867529 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f23100d4210 0x7f23101f7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:31.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.057+0000 7f2314b3f640 1 -- 172.21.15.32:0/2893867529 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f23100d4210 msgr2=0x7f23101f7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:31.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.057+0000 7f2314b3f640 1 --2- 172.21.15.32:0/2893867529 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f23100d4210 0x7f23101f7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:31.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.057+0000 7f2314b3f640 1 -- 172.21.15.32:0/2893867529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f231006b820 msgr2=0x7f23101542c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:31.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.057+0000 7f2314b3f640 1 --2- 172.21.15.32:0/2893867529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f231006b820 0x7f23101542c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:31.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.057+0000 7f2314b3f640 1 -- 172.21.15.32:0/2893867529 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2310153260 con 0x7f231015e9b0 2024-04-24T23:06:31.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.058+0000 7f2314b3f640 1 --2- 172.21.15.32:0/2893867529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f231015e9b0 0x7f23101f8140 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f2308034040 tx=0x7f2308004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:31.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.058+0000 7f230cff9640 1 -- 172.21.15.32:0/2893867529 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2308036050 con 0x7f231015e9b0 2024-04-24T23:06:31.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.058+0000 7f230cff9640 1 -- 172.21.15.32:0/2893867529 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2308042070 con 0x7f231015e9b0 2024-04-24T23:06:31.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.058+0000 7f23165a2640 1 -- 172.21.15.32:0/2893867529 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2310153500 con 0x7f231015e9b0 2024-04-24T23:06:31.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.058+0000 7f230cff9640 1 -- 172.21.15.32:0/2893867529 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2308009510 con 0x7f231015e9b0 2024-04-24T23:06:31.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.058+0000 7f23165a2640 1 -- 172.21.15.32:0/2893867529 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2310153ac0 con 0x7f231015e9b0 2024-04-24T23:06:31.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.059+0000 7f23165a2640 1 -- 172.21.15.32:0/2893867529 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2310068430 con 0x7f231015e9b0 2024-04-24T23:06:31.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.060+0000 7f230cff9640 1 -- 172.21.15.32:0/2893867529 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2308007bf0 con 0x7f231015e9b0 2024-04-24T23:06:31.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.060+0000 7f230cff9640 1 --2- 172.21.15.32:0/2893867529 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f22fc06be30 0x7f22fc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:31.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.060+0000 7f230cff9640 1 -- 172.21.15.32:0/2893867529 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5609+0+0 (secure 0 0 0) 0x7f23080ac7a0 con 0x7f231015e9b0 2024-04-24T23:06:31.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.061+0000 7f230ffff640 1 --2- 172.21.15.32:0/2893867529 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f22fc06be30 0x7f22fc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:31.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.061+0000 7f230ffff640 1 --2- 172.21.15.32:0/2893867529 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f22fc06be30 0x7f22fc06e2f0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f23100d26a0 tx=0x7f2300008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:31.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.064+0000 7f230cff9640 1 -- 172.21.15.32:0/2893867529 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f230803a360 con 0x7f231015e9b0 2024-04-24T23:06:31.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.290+0000 7f23165a2640 1 -- 172.21.15.32:0/2893867529 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f231015fbc0 con 0x7f231015e9b0 2024-04-24T23:06:31.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.942+0000 7f230cff9640 1 -- 172.21.15.32:0/2893867529 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v63) ==== 174+0+0 (secure 0 0 0) 0x7f23080aae20 con 0x7f231015e9b0 2024-04-24T23:06:31.945 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 18 and data pool 19 2024-04-24T23:06:31.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.951+0000 7f23165a2640 1 -- 172.21.15.32:0/2893867529 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f22fc06be30 msgr2=0x7f22fc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:31.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.951+0000 7f23165a2640 1 --2- 172.21.15.32:0/2893867529 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f22fc06be30 0x7f22fc06e2f0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f23100d26a0 tx=0x7f2300008040 comp rx=0 tx=0).stop 2024-04-24T23:06:31.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.951+0000 7f23165a2640 1 -- 172.21.15.32:0/2893867529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f231015e9b0 msgr2=0x7f23101f8140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:31.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.951+0000 7f23165a2640 1 --2- 172.21.15.32:0/2893867529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f231015e9b0 0x7f23101f8140 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f2308034040 tx=0x7f2308004a40 comp rx=0 tx=0).stop 2024-04-24T23:06:31.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.954+0000 7f23165a2640 1 -- 172.21.15.32:0/2893867529 shutdown_connections 2024-04-24T23:06:31.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.954+0000 7f23165a2640 1 --2- 172.21.15.32:0/2893867529 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f22fc06be30 0x7f22fc06e2f0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:31.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.954+0000 7f23165a2640 1 --2- 172.21.15.32:0/2893867529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f231015e9b0 0x7f23101f8140 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:31.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.954+0000 7f23165a2640 1 --2- 172.21.15.32:0/2893867529 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f23100d4210 0x7f23101f7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:31.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.954+0000 7f23165a2640 1 --2- 172.21.15.32:0/2893867529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f231006b820 0x7f23101542c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:31.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.954+0000 7f23165a2640 1 -- 172.21.15.32:0/2893867529 >> 172.21.15.32:0/2893867529 conn(0x7f23100d11d0 msgr2=0x7f23100b92e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:31.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.955+0000 7f23165a2640 1 -- 172.21.15.32:0/2893867529 shutdown_connections 2024-04-24T23:06:31.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:31.955+0000 7f23165a2640 1 -- 172.21.15.32:0/2893867529 wait complete. 2024-04-24T23:06:31.975 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:06:32.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.078+0000 7f71ecccd640 1 Processor -- start 2024-04-24T23:06:32.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.078+0000 7f71ecccd640 1 -- start start 2024-04-24T23:06:32.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.079+0000 7f71ecccd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f71e806b820 0x7f71e80d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:32.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.079+0000 7f71ecccd640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71e80d4210 0x7f71e80b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:32.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.079+0000 7f71ecccd640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71e80d3170 con 0x7f71e815e9b0 2024-04-24T23:06:32.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.079+0000 7f71ecccd640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f71e80d3450 con 0x7f71e806b820 2024-04-24T23:06:32.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.079+0000 7f71ecccd640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f71e80d3730 con 0x7f71e80d4210 2024-04-24T23:06:32.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.079+0000 7f71e5d9b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f71e806b820 0x7f71e80d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:32.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.080+0000 7f71e5d9b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f71e806b820 0x7f71e80d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48428/0 (socket says 172.21.15.32:48428) 2024-04-24T23:06:32.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.080+0000 7f71e5d9b640 1 -- 172.21.15.32:0/2015445401 learned_addr learned my addr 172.21.15.32:0/2015445401 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:32.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.080+0000 7f71e659c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f71e815e9b0 0x7f71e806b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33090/0 (socket says 172.21.15.32:33090) 2024-04-24T23:06:32.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.080+0000 7f71e5d9b640 1 -- 172.21.15.32:0/2015445401 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71e80d4210 msgr2=0x7f71e80b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:32.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.080+0000 7f71e6d9d640 1 --2- 172.21.15.32:0/2015445401 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71e80d4210 0x7f71e80b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:32.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.080+0000 7f71e5d9b640 1 --2- 172.21.15.32:0/2015445401 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71e80d4210 0x7f71e80b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:32.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.080+0000 7f71e559a640 1 -- 172.21.15.32:0/2015445401 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1092422719 0 0) 0x7f71e80d3170 con 0x7f71e815e9b0 2024-04-24T23:06:32.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.080+0000 7f71e5d9b640 1 -- 172.21.15.32:0/2015445401 >> v1:172.21.15.32:6789/0 conn(0x7f71e815e9b0 legacy=0x7f71e806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:32.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.080+0000 7f71e5d9b640 1 -- 172.21.15.32:0/2015445401 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71e80d3a10 con 0x7f71e806b820 2024-04-24T23:06:32.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.080+0000 7f71e6d9d640 1 --2- 172.21.15.32:0/2015445401 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71e80d4210 0x7f71e80b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:06:32.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.081+0000 7f71e5d9b640 1 --2- 172.21.15.32:0/2015445401 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71e806b820 0x7f71e80d2b90 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f71d0009870 tx=0x7f71d002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b371517edb8b4b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:32.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.081+0000 7f71e559a640 1 -- 172.21.15.32:0/2015445401 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d0037070 con 0x7f71e806b820 2024-04-24T23:06:32.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.081+0000 7f71e559a640 1 -- 172.21.15.32:0/2015445401 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71d0032440 con 0x7f71e806b820 2024-04-24T23:06:32.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.081+0000 7f71e559a640 1 -- 172.21.15.32:0/2015445401 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d0032810 con 0x7f71e806b820 2024-04-24T23:06:32.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.081+0000 7f71ecccd640 1 -- 172.21.15.32:0/2015445401 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71e806b820 msgr2=0x7f71e80d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:32.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.081+0000 7f71ecccd640 1 --2- 172.21.15.32:0/2015445401 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71e806b820 0x7f71e80d2b90 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f71d0009870 tx=0x7f71d002a270 comp rx=0 tx=0).stop 2024-04-24T23:06:32.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.081+0000 7f71ecccd640 1 -- 172.21.15.32:0/2015445401 shutdown_connections 2024-04-24T23:06:32.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.081+0000 7f71ecccd640 1 --2- 172.21.15.32:0/2015445401 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71e806b820 0x7f71e80d2b90 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:32.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.081+0000 7f71ecccd640 1 --2- 172.21.15.32:0/2015445401 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71e80d4210 0x7f71e80b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:32.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.081+0000 7f71ecccd640 1 -- 172.21.15.32:0/2015445401 >> 172.21.15.32:0/2015445401 conn(0x7f71e80d11d0 msgr2=0x7f71e80d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:32.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.081+0000 7f71ecccd640 1 -- 172.21.15.32:0/2015445401 shutdown_connections 2024-04-24T23:06:32.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.081+0000 7f71ecccd640 1 -- 172.21.15.32:0/2015445401 wait complete. 2024-04-24T23:06:32.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.083+0000 7f71ecccd640 1 Processor -- start 2024-04-24T23:06:32.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.083+0000 7f71ecccd640 1 -- start start 2024-04-24T23:06:32.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.083+0000 7f71ecccd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71e806b820 0x7f71e81f45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:32.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.083+0000 7f71ecccd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71e80d4210 0x7f71e8148700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:32.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.083+0000 7f71ecccd640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71e815e9b0 0x7f71e80cce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:32.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.083+0000 7f71ecccd640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f71e80ba0e0 con 0x7f71e806b820 2024-04-24T23:06:32.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.083+0000 7f71ecccd640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f71e80d2400 con 0x7f71e80d4210 2024-04-24T23:06:32.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.083+0000 7f71ecccd640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f71e80ba260 con 0x7f71e815e9b0 2024-04-24T23:06:32.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.083+0000 7f71e659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71e806b820 0x7f71e81f45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:32.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.084+0000 7f71e659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71e806b820 0x7f71e81f45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48438/0 (socket says 172.21.15.32:48438) 2024-04-24T23:06:32.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.084+0000 7f71e659c640 1 -- 172.21.15.32:0/167272438 learned_addr learned my addr 172.21.15.32:0/167272438 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:32.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.084+0000 7f71e6d9d640 1 --2- 172.21.15.32:0/167272438 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71e815e9b0 0x7f71e80cce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:32.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.084+0000 7f71e5d9b640 1 --2- 172.21.15.32:0/167272438 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71e80d4210 0x7f71e8148700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:32.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.084+0000 7f71e659c640 1 -- 172.21.15.32:0/167272438 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71e815e9b0 msgr2=0x7f71e80cce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:32.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.084+0000 7f71e659c640 1 --2- 172.21.15.32:0/167272438 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71e815e9b0 0x7f71e80cce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:32.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.084+0000 7f71e659c640 1 -- 172.21.15.32:0/167272438 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71e80d4210 msgr2=0x7f71e8148700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:32.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.084+0000 7f71e659c640 1 --2- 172.21.15.32:0/167272438 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71e80d4210 0x7f71e8148700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:32.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.084+0000 7f71e659c640 1 -- 172.21.15.32:0/167272438 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71e81490e0 con 0x7f71e806b820 2024-04-24T23:06:32.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.084+0000 7f71e6d9d640 1 --2- 172.21.15.32:0/167272438 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71e815e9b0 0x7f71e80cce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:06:32.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.084+0000 7f71e659c640 1 --2- 172.21.15.32:0/167272438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71e806b820 0x7f71e81f45c0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f71e000c500 tx=0x7f71e000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:32.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.084+0000 7f71deffd640 1 -- 172.21.15.32:0/167272438 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e0018070 con 0x7f71e806b820 2024-04-24T23:06:32.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.084+0000 7f71deffd640 1 -- 172.21.15.32:0/167272438 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71e0007580 con 0x7f71e806b820 2024-04-24T23:06:32.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.084+0000 7f71deffd640 1 -- 172.21.15.32:0/167272438 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e0004a20 con 0x7f71e806b820 2024-04-24T23:06:32.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.084+0000 7f71ecccd640 1 -- 172.21.15.32:0/167272438 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71e81493c0 con 0x7f71e806b820 2024-04-24T23:06:32.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.084+0000 7f71ecccd640 1 -- 172.21.15.32:0/167272438 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71e8149980 con 0x7f71e806b820 2024-04-24T23:06:32.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.085+0000 7f71ecccd640 1 -- 172.21.15.32:0/167272438 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71e8068430 con 0x7f71e806b820 2024-04-24T23:06:32.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.086+0000 7f71deffd640 1 -- 172.21.15.32:0/167272438 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f71e001b070 con 0x7f71e806b820 2024-04-24T23:06:32.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.086+0000 7f71deffd640 1 --2- 172.21.15.32:0/167272438 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f71c806be30 0x7f71c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:32.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.087+0000 7f71deffd640 1 -- 172.21.15.32:0/167272438 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 5709+0+0 (secure 0 0 0) 0x7f71e008a130 con 0x7f71e806b820 2024-04-24T23:06:32.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.087+0000 7f71e5d9b640 1 --2- 172.21.15.32:0/167272438 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f71c806be30 0x7f71c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:32.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.087+0000 7f71e5d9b640 1 --2- 172.21.15.32:0/167272438 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f71c806be30 0x7f71c806e2f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f71d0034000 tx=0x7f71d002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:32.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.091+0000 7f71deffd640 1 -- 172.21.15.32:0/167272438 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f71e0058480 con 0x7f71e806b820 2024-04-24T23:06:32.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.276+0000 7f71ecccd640 1 -- 172.21.15.32:0/167272438 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f71e80b9a20 con 0x7f71e806b820 2024-04-24T23:06:32.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.277+0000 7f71deffd640 1 -- 172.21.15.32:0/167272438 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v155) ==== 74+0+16653 (secure 0 0 0) 0x7f71e005c780 con 0x7f71e806b820 2024-04-24T23:06:32.279 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:06:32.279 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":155,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:06:31.291302+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:06:29.591736+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-24T23:06:30.267629+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:06:32.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.279+0000 7f71ecccd640 1 -- 172.21.15.32:0/167272438 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f71c806be30 msgr2=0x7f71c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:32.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.279+0000 7f71ecccd640 1 --2- 172.21.15.32:0/167272438 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f71c806be30 0x7f71c806e2f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f71d0034000 tx=0x7f71d002e000 comp rx=0 tx=0).stop 2024-04-24T23:06:32.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.279+0000 7f71ecccd640 1 -- 172.21.15.32:0/167272438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71e806b820 msgr2=0x7f71e81f45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:32.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.279+0000 7f71ecccd640 1 --2- 172.21.15.32:0/167272438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71e806b820 0x7f71e81f45c0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f71e000c500 tx=0x7f71e000c9c0 comp rx=0 tx=0).stop 2024-04-24T23:06:32.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.279+0000 7f71ecccd640 1 -- 172.21.15.32:0/167272438 shutdown_connections 2024-04-24T23:06:32.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.279+0000 7f71ecccd640 1 --2- 172.21.15.32:0/167272438 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f71c806be30 0x7f71c806e2f0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:32.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.279+0000 7f71ecccd640 1 --2- 172.21.15.32:0/167272438 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71e815e9b0 0x7f71e80cce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:32.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.279+0000 7f71ecccd640 1 --2- 172.21.15.32:0/167272438 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71e80d4210 0x7f71e8148700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:32.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.279+0000 7f71ecccd640 1 --2- 172.21.15.32:0/167272438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71e806b820 0x7f71e81f45c0 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:32.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.279+0000 7f71ecccd640 1 -- 172.21.15.32:0/167272438 >> 172.21.15.32:0/167272438 conn(0x7f71e80d11d0 msgr2=0x7f71e815d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:32.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.279+0000 7f71ecccd640 1 -- 172.21.15.32:0/167272438 shutdown_connections 2024-04-24T23:06:32.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.279+0000 7f71ecccd640 1 -- 172.21.15.32:0/167272438 wait complete. 2024-04-24T23:06:32.296 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:06:32.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.395+0000 7f27aba1c640 1 Processor -- start 2024-04-24T23:06:32.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.395+0000 7f27aba1c640 1 -- start start 2024-04-24T23:06:32.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.396+0000 7f27aba1c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f27a40b5560 0x7f27a415e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:32.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.396+0000 7f27aba1c640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f27a40cd3c0 0x7f27a415ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:32.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.396+0000 7f27aba1c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f27a40b5940 0x7f27a40b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:32.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.396+0000 7f27aba1c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f27a40cd910 con 0x7f27a40b5560 2024-04-24T23:06:32.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.396+0000 7f27aba1c640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f27a40cdbf0 con 0x7f27a40cd3c0 2024-04-24T23:06:32.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.396+0000 7f27aba1c640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f27a40cded0 con 0x7f27a40b5940 2024-04-24T23:06:32.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.396+0000 7f27a97b8640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f27a40b5560 0x7f27a415e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:32.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.396+0000 7f27a97b8640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f27a40b5560 0x7f27a415e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48446/0 (socket says 172.21.15.32:48446) 2024-04-24T23:06:32.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.396+0000 7f27a97b8640 1 -- 172.21.15.32:0/1807596691 learned_addr learned my addr 172.21.15.32:0/1807596691 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:32.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.396+0000 7f27a9fb9640 1 --2- 172.21.15.32:0/1807596691 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f27a40b5940 0x7f27a40b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:32.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.396+0000 7f27a8fb7640 1 --2- 172.21.15.32:0/1807596691 >> v2:172.21.15.121:3300/0 conn(0x7f27a40cd3c0 0x7f27a415ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:32.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.396+0000 7f27a97b8640 1 -- 172.21.15.32:0/1807596691 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f27a40b5940 msgr2=0x7f27a40b5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:32.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.396+0000 7f27a97b8640 1 --2- 172.21.15.32:0/1807596691 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f27a40b5940 0x7f27a40b5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:32.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.396+0000 7f27a97b8640 1 -- 172.21.15.32:0/1807596691 >> v2:172.21.15.121:3300/0 conn(0x7f27a40cd3c0 msgr2=0x7f27a415ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:32.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.396+0000 7f27a97b8640 1 --2- 172.21.15.32:0/1807596691 >> v2:172.21.15.121:3300/0 conn(0x7f27a40cd3c0 0x7f27a415ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:32.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.396+0000 7f27a97b8640 1 -- 172.21.15.32:0/1807596691 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27a40ce1b0 con 0x7f27a40b5560 2024-04-24T23:06:32.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.396+0000 7f27a8fb7640 1 --2- 172.21.15.32:0/1807596691 >> v2:172.21.15.121:3300/0 conn(0x7f27a40cd3c0 0x7f27a415ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:06:32.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.396+0000 7f27a9fb9640 1 --2- 172.21.15.32:0/1807596691 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f27a40b5940 0x7f27a40b5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:06:32.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.397+0000 7f27a97b8640 1 --2- 172.21.15.32:0/1807596691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27a40b5560 0x7f27a415e6c0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f27a40ccb60 tx=0x7f27a002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38ba7de0d4534387 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:32.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.397+0000 7f279bfff640 1 -- 172.21.15.32:0/1807596691 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a0037070 con 0x7f27a40b5560 2024-04-24T23:06:32.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.397+0000 7f279bfff640 1 -- 172.21.15.32:0/1807596691 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27a0032440 con 0x7f27a40b5560 2024-04-24T23:06:32.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.397+0000 7f279bfff640 1 -- 172.21.15.32:0/1807596691 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a0032810 con 0x7f27a40b5560 2024-04-24T23:06:32.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.397+0000 7f27aba1c640 1 -- 172.21.15.32:0/1807596691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27a40b5560 msgr2=0x7f27a415e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:32.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.397+0000 7f27aba1c640 1 --2- 172.21.15.32:0/1807596691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27a40b5560 0x7f27a415e6c0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f27a40ccb60 tx=0x7f27a002a1e0 comp rx=0 tx=0).stop 2024-04-24T23:06:32.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.397+0000 7f27aba1c640 1 -- 172.21.15.32:0/1807596691 shutdown_connections 2024-04-24T23:06:32.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.397+0000 7f27aba1c640 1 --2- 172.21.15.32:0/1807596691 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f27a40b5940 0x7f27a40b5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:32.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.397+0000 7f27aba1c640 1 --2- 172.21.15.32:0/1807596691 >> v2:172.21.15.121:3300/0 conn(0x7f27a40cd3c0 0x7f27a415ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:32.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.397+0000 7f27aba1c640 1 --2- 172.21.15.32:0/1807596691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27a40b5560 0x7f27a415e6c0 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:32.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.398+0000 7f27aba1c640 1 -- 172.21.15.32:0/1807596691 >> 172.21.15.32:0/1807596691 conn(0x7f27a406b110 msgr2=0x7f27a406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:32.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.398+0000 7f27aba1c640 1 -- 172.21.15.32:0/1807596691 shutdown_connections 2024-04-24T23:06:32.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.398+0000 7f27aba1c640 1 -- 172.21.15.32:0/1807596691 wait complete. 2024-04-24T23:06:32.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.399+0000 7f27aba1c640 1 Processor -- start 2024-04-24T23:06:32.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.399+0000 7f27aba1c640 1 -- start start 2024-04-24T23:06:32.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.400+0000 7f27aba1c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27a40b5560 0x7f27a41581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:32.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.400+0000 7f27aba1c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27a40b5940 0x7f27a413ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:32.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.400+0000 7f27aba1c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f27a40cd3c0 0x7f27a40ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:32.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.400+0000 7f27aba1c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f27a40b62f0 con 0x7f27a40b5560 2024-04-24T23:06:32.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.400+0000 7f27aba1c640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f27a40b4420 con 0x7f27a40b5940 2024-04-24T23:06:32.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.400+0000 7f27aba1c640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f27a4142640 con 0x7f27a40cd3c0 2024-04-24T23:06:32.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.400+0000 7f27a97b8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27a40b5560 0x7f27a41581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:32.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.400+0000 7f27a97b8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27a40b5560 0x7f27a41581d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48448/0 (socket says 172.21.15.32:48448) 2024-04-24T23:06:32.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.400+0000 7f27a97b8640 1 -- 172.21.15.32:0/1123684584 learned_addr learned my addr 172.21.15.32:0/1123684584 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:32.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.400+0000 7f27a8fb7640 1 --2- 172.21.15.32:0/1123684584 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27a40b5940 0x7f27a413ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:32.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.400+0000 7f27a9fb9640 1 --2- 172.21.15.32:0/1123684584 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f27a40cd3c0 0x7f27a40ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:32.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.400+0000 7f27a97b8640 1 -- 172.21.15.32:0/1123684584 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f27a40cd3c0 msgr2=0x7f27a40ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:32.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.400+0000 7f27a97b8640 1 --2- 172.21.15.32:0/1123684584 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f27a40cd3c0 0x7f27a40ae8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:32.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.400+0000 7f27a97b8640 1 -- 172.21.15.32:0/1123684584 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27a40b5940 msgr2=0x7f27a413ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:32.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.400+0000 7f27a97b8640 1 --2- 172.21.15.32:0/1123684584 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27a40b5940 0x7f27a413ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:32.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.400+0000 7f27a97b8640 1 -- 172.21.15.32:0/1123684584 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27a4157410 con 0x7f27a40b5560 2024-04-24T23:06:32.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.400+0000 7f27a97b8640 1 --2- 172.21.15.32:0/1123684584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27a40b5560 0x7f27a41581d0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f27a002a220 tx=0x7f27a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:32.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.400+0000 7f2799ffb640 1 -- 172.21.15.32:0/1123684584 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a0041020 con 0x7f27a40b5560 2024-04-24T23:06:32.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.400+0000 7f2799ffb640 1 -- 172.21.15.32:0/1123684584 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27a00306c0 con 0x7f27a40b5560 2024-04-24T23:06:32.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.400+0000 7f27aba1c640 1 -- 172.21.15.32:0/1123684584 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27a41576f0 con 0x7f27a40b5560 2024-04-24T23:06:32.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.400+0000 7f2799ffb640 1 -- 172.21.15.32:0/1123684584 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a0037040 con 0x7f27a40b5560 2024-04-24T23:06:32.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.401+0000 7f27aba1c640 1 -- 172.21.15.32:0/1123684584 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27a4157cb0 con 0x7f27a40b5560 2024-04-24T23:06:32.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.402+0000 7f27aba1c640 1 -- 172.21.15.32:0/1123684584 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27a4068430 con 0x7f27a40b5560 2024-04-24T23:06:32.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.402+0000 7f2799ffb640 1 -- 172.21.15.32:0/1123684584 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f27a0044020 con 0x7f27a40b5560 2024-04-24T23:06:32.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.402+0000 7f2799ffb640 1 --2- 172.21.15.32:0/1123684584 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f278806be30 0x7f278806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:32.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.403+0000 7f2799ffb640 1 -- 172.21.15.32:0/1123684584 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 5709+0+0 (secure 0 0 0) 0x7f27a00ab6b0 con 0x7f27a40b5560 2024-04-24T23:06:32.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.403+0000 7f27a8fb7640 1 --2- 172.21.15.32:0/1123684584 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f278806be30 0x7f278806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:32.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.403+0000 7f27a8fb7640 1 --2- 172.21.15.32:0/1123684584 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f278806be30 0x7f278806e2f0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f27a414f740 tx=0x7f2794008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:32.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.407+0000 7f2799ffb640 1 -- 172.21.15.32:0/1123684584 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f27a004b030 con 0x7f27a40b5560 2024-04-24T23:06:32.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.590+0000 7f27aba1c640 1 -- 172.21.15.32:0/1123684584 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f27a4152fe0 con 0x7f27a40b5560 2024-04-24T23:06:32.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.591+0000 7f2799ffb640 1 -- 172.21.15.32:0/1123684584 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v155) ==== 74+0+16653 (secure 0 0 0) 0x7f27a007a050 con 0x7f27a40b5560 2024-04-24T23:06:32.592 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:06:32.593 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":155,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:06:31.291302+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:06:29.591736+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-24T23:06:30.267629+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:06:32.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.593+0000 7f27aba1c640 1 -- 172.21.15.32:0/1123684584 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f278806be30 msgr2=0x7f278806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:32.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.593+0000 7f27aba1c640 1 --2- 172.21.15.32:0/1123684584 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f278806be30 0x7f278806e2f0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f27a414f740 tx=0x7f2794008040 comp rx=0 tx=0).stop 2024-04-24T23:06:32.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.593+0000 7f27aba1c640 1 -- 172.21.15.32:0/1123684584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27a40b5560 msgr2=0x7f27a41581d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:32.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.593+0000 7f27aba1c640 1 --2- 172.21.15.32:0/1123684584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27a40b5560 0x7f27a41581d0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f27a002a220 tx=0x7f27a00046c0 comp rx=0 tx=0).stop 2024-04-24T23:06:32.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.593+0000 7f27aba1c640 1 -- 172.21.15.32:0/1123684584 shutdown_connections 2024-04-24T23:06:32.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.593+0000 7f27aba1c640 1 --2- 172.21.15.32:0/1123684584 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f278806be30 0x7f278806e2f0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:32.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.593+0000 7f27aba1c640 1 --2- 172.21.15.32:0/1123684584 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f27a40cd3c0 0x7f27a40ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:32.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.593+0000 7f27aba1c640 1 --2- 172.21.15.32:0/1123684584 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27a40b5940 0x7f27a413ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:32.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.593+0000 7f27aba1c640 1 --2- 172.21.15.32:0/1123684584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27a40b5560 0x7f27a41581d0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:32.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.593+0000 7f27aba1c640 1 -- 172.21.15.32:0/1123684584 >> 172.21.15.32:0/1123684584 conn(0x7f27a406b110 msgr2=0x7f27a40b0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:32.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.593+0000 7f27aba1c640 1 -- 172.21.15.32:0/1123684584 shutdown_connections 2024-04-24T23:06:32.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.593+0000 7f27aba1c640 1 -- 172.21.15.32:0/1123684584 wait complete. 2024-04-24T23:06:32.606 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-24T23:06:32.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.707+0000 7f0d45483640 1 Processor -- start 2024-04-24T23:06:32.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.708+0000 7f0d45483640 1 -- start start 2024-04-24T23:06:32.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.708+0000 7f0d45483640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d4015edd0 0x7f0d400b5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:32.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.708+0000 7f0d45483640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0d400cd3c0 0x7f0d400b5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:32.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.708+0000 7f0d45483640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d400cd910 con 0x7f0d400b5560 2024-04-24T23:06:32.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.708+0000 7f0d45483640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f0d400cdbf0 con 0x7f0d400cd3c0 2024-04-24T23:06:32.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.708+0000 7f0d45483640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f0d400cded0 con 0x7f0d4015edd0 2024-04-24T23:06:32.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.708+0000 7f0d3effd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0d400b5560 0x7f0d4015e6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33092/0 (socket says 172.21.15.32:33092) 2024-04-24T23:06:32.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.708+0000 7f0d3effd640 1 -- 172.21.15.32:0/653534634 learned_addr learned my addr 172.21.15.32:0/653534634 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:06:32.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.709+0000 7f0d3f7fe640 1 --2- 172.21.15.32:0/653534634 >> v2:172.21.15.121:3300/0 conn(0x7f0d400cd3c0 0x7f0d400b5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:32.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.709+0000 7f0d3dffb640 1 -- 172.21.15.32:0/653534634 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2312648543 0 0) 0x7f0d400cd910 con 0x7f0d400b5560 2024-04-24T23:06:32.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.709+0000 7f0d3dffb640 1 -- 172.21.15.32:0/653534634 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0d28003520 con 0x7f0d400b5560 2024-04-24T23:06:32.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.709+0000 7f0d3e7fc640 1 --2- 172.21.15.32:0/653534634 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d4015edd0 0x7f0d400b5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:32.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.709+0000 7f0d3dffb640 1 -- 172.21.15.32:0/653534634 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 496744167 0 0) 0x7f0d28003520 con 0x7f0d400b5560 2024-04-24T23:06:32.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.709+0000 7f0d3dffb640 1 -- 172.21.15.32:0/653534634 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d4015edd0 msgr2=0x7f0d400b5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:32.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.709+0000 7f0d3dffb640 1 --2- 172.21.15.32:0/653534634 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d4015edd0 0x7f0d400b5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:32.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.709+0000 7f0d3dffb640 1 -- 172.21.15.32:0/653534634 >> v2:172.21.15.121:3300/0 conn(0x7f0d400cd3c0 msgr2=0x7f0d400b5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:32.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.709+0000 7f0d3dffb640 1 --2- 172.21.15.32:0/653534634 >> v2:172.21.15.121:3300/0 conn(0x7f0d400cd3c0 0x7f0d400b5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:32.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.709+0000 7f0d3dffb640 1 -- 172.21.15.32:0/653534634 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d400ce1b0 con 0x7f0d400b5560 2024-04-24T23:06:32.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.709+0000 7f0d3dffb640 1 -- 172.21.15.32:0/653534634 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f0d38001670 con 0x7f0d400b5560 2024-04-24T23:06:32.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.709+0000 7f0d3dffb640 1 -- 172.21.15.32:0/653534634 >> v1:172.21.15.32:6789/0 conn(0x7f0d400b5560 legacy=0x7f0d4015e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:32.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.709+0000 7f0d3dffb640 1 --2- 172.21.15.32:0/653534634 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d280039b0 0x7f0d28003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:32.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.709+0000 7f0d3dffb640 1 --2- 172.21.15.32:0/653534634 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d4015edd0 0x7f0d28004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:32.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.709+0000 7f0d3dffb640 1 --2- 172.21.15.32:0/653534634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d28004d40 0x7f0d280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:32.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.709+0000 7f0d3dffb640 1 -- 172.21.15.32:0/653534634 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0d400cdbf0 con 0x7f0d28004d40 2024-04-24T23:06:32.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.709+0000 7f0d3dffb640 1 -- 172.21.15.32:0/653534634 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0d400cded0 con 0x7f0d280039b0 2024-04-24T23:06:32.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.709+0000 7f0d3dffb640 1 -- 172.21.15.32:0/653534634 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f0d2800b9c0 con 0x7f0d4015edd0 2024-04-24T23:06:32.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.709+0000 7f0d3f7fe640 1 --2- 172.21.15.32:0/653534634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d28004d40 0x7f0d280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:32.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.709+0000 7f0d3effd640 1 --2- 172.21.15.32:0/653534634 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d280039b0 0x7f0d28003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:32.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.709+0000 7f0d3e7fc640 1 --2- 172.21.15.32:0/653534634 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d4015edd0 0x7f0d28004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:32.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.710+0000 7f0d3f7fe640 1 -- 172.21.15.32:0/653534634 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d4015edd0 msgr2=0x7f0d28004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:32.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.710+0000 7f0d3f7fe640 1 --2- 172.21.15.32:0/653534634 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d4015edd0 0x7f0d28004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:32.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.710+0000 7f0d3f7fe640 1 -- 172.21.15.32:0/653534634 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d280039b0 msgr2=0x7f0d28003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:32.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.710+0000 7f0d3f7fe640 1 --2- 172.21.15.32:0/653534634 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d280039b0 0x7f0d28003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:32.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.710+0000 7f0d3f7fe640 1 -- 172.21.15.32:0/653534634 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d28003520 con 0x7f0d28004d40 2024-04-24T23:06:32.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.710+0000 7f0d3f7fe640 1 --2- 172.21.15.32:0/653534634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d28004d40 0x7f0d280051c0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f0d3000e170 tx=0x7f0d28005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8383acb9e5752c90 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:32.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.710+0000 7f0d3dffb640 1 -- 172.21.15.32:0/653534634 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d30034020 con 0x7f0d28004d40 2024-04-24T23:06:32.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.710+0000 7f0d3dffb640 1 -- 172.21.15.32:0/653534634 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d30030430 con 0x7f0d28004d40 2024-04-24T23:06:32.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.710+0000 7f0d3dffb640 1 -- 172.21.15.32:0/653534634 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d30009c10 con 0x7f0d28004d40 2024-04-24T23:06:32.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.711+0000 7f0d45483640 1 -- 172.21.15.32:0/653534634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d28004d40 msgr2=0x7f0d280051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:32.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.711+0000 7f0d45483640 1 --2- 172.21.15.32:0/653534634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d28004d40 0x7f0d280051c0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f0d3000e170 tx=0x7f0d28005830 comp rx=0 tx=0).stop 2024-04-24T23:06:32.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.711+0000 7f0d3f7fe640 1 -- 172.21.15.32:0/653534634 reap_dead start 2024-04-24T23:06:32.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.711+0000 7f0d45483640 1 -- 172.21.15.32:0/653534634 shutdown_connections 2024-04-24T23:06:32.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.711+0000 7f0d45483640 1 -- 172.21.15.32:0/653534634 >> 172.21.15.32:0/653534634 conn(0x7f0d4006b110 msgr2=0x7f0d4006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:32.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.711+0000 7f0d45483640 1 -- 172.21.15.32:0/653534634 shutdown_connections 2024-04-24T23:06:32.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.711+0000 7f0d45483640 1 -- 172.21.15.32:0/653534634 wait complete. 2024-04-24T23:06:32.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.712+0000 7f0d45483640 1 Processor -- start 2024-04-24T23:06:32.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.712+0000 7f0d45483640 1 -- start start 2024-04-24T23:06:32.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.713+0000 7f0d45483640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d28004d40 0x7f0d40158cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:32.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.713+0000 7f0d45483640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d40155fc0 0x7f0d4014ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:32.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.713+0000 7f0d45483640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d40072440 0x7f0d400728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:32.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.713+0000 7f0d45483640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0d400b5940 con 0x7f0d40072440 2024-04-24T23:06:32.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.713+0000 7f0d45483640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0d400b63c0 con 0x7f0d28004d40 2024-04-24T23:06:32.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.713+0000 7f0d45483640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f0d400b5ac0 con 0x7f0d40155fc0 2024-04-24T23:06:32.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.713+0000 7f0d3f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d40072440 0x7f0d400728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:32.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.713+0000 7f0d3f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d40072440 0x7f0d400728a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48454/0 (socket says 172.21.15.32:48454) 2024-04-24T23:06:32.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.713+0000 7f0d3f7fe640 1 -- 172.21.15.32:0/382192983 learned_addr learned my addr 172.21.15.32:0/382192983 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:32.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.713+0000 7f0d3effd640 1 --2- 172.21.15.32:0/382192983 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d28004d40 0x7f0d40158cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:32.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.713+0000 7f0d3e7fc640 1 --2- 172.21.15.32:0/382192983 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d40155fc0 0x7f0d4014ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:32.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.713+0000 7f0d3f7fe640 1 -- 172.21.15.32:0/382192983 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d40155fc0 msgr2=0x7f0d4014ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:32.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.713+0000 7f0d3f7fe640 1 --2- 172.21.15.32:0/382192983 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d40155fc0 0x7f0d4014ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:32.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.713+0000 7f0d3f7fe640 1 -- 172.21.15.32:0/382192983 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d28004d40 msgr2=0x7f0d40158cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:32.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.713+0000 7f0d3f7fe640 1 --2- 172.21.15.32:0/382192983 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d28004d40 0x7f0d40158cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:32.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.714+0000 7f0d3f7fe640 1 -- 172.21.15.32:0/382192983 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d401592d0 con 0x7f0d40072440 2024-04-24T23:06:32.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.714+0000 7f0d3e7fc640 1 --2- 172.21.15.32:0/382192983 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d40155fc0 0x7f0d4014ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:06:32.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.714+0000 7f0d3f7fe640 1 --2- 172.21.15.32:0/382192983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d40072440 0x7f0d400728a0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f0d3000e170 tx=0x7f0d30009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:32.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.714+0000 7f0d277fe640 1 -- 172.21.15.32:0/382192983 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d30034040 con 0x7f0d40072440 2024-04-24T23:06:32.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.714+0000 7f0d277fe640 1 -- 172.21.15.32:0/382192983 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d30003b90 con 0x7f0d40072440 2024-04-24T23:06:32.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.714+0000 7f0d45483640 1 -- 172.21.15.32:0/382192983 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d401595b0 con 0x7f0d40072440 2024-04-24T23:06:32.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.714+0000 7f0d277fe640 1 -- 172.21.15.32:0/382192983 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d30009c10 con 0x7f0d40072440 2024-04-24T23:06:32.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.714+0000 7f0d45483640 1 -- 172.21.15.32:0/382192983 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d40159b70 con 0x7f0d40072440 2024-04-24T23:06:32.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.715+0000 7f0d45483640 1 -- 172.21.15.32:0/382192983 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d4015f1f0 con 0x7f0d40072440 2024-04-24T23:06:32.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.716+0000 7f0d277fe640 1 -- 172.21.15.32:0/382192983 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0d30037020 con 0x7f0d40072440 2024-04-24T23:06:32.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.716+0000 7f0d277fe640 1 --2- 172.21.15.32:0/382192983 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0d2c06be30 0x7f0d2c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:32.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.716+0000 7f0d277fe640 1 -- 172.21.15.32:0/382192983 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 5709+0+0 (secure 0 0 0) 0x7f0d300abd40 con 0x7f0d40072440 2024-04-24T23:06:32.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.716+0000 7f0d3effd640 1 --2- 172.21.15.32:0/382192983 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0d2c06be30 0x7f0d2c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:32.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.717+0000 7f0d3effd640 1 --2- 172.21.15.32:0/382192983 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0d2c06be30 0x7f0d2c06e2f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0d38001d30 tx=0x7f0d38007970 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:32.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.720+0000 7f0d277fe640 1 -- 172.21.15.32:0/382192983 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0d4015f1f0 con 0x7f0d40072440 2024-04-24T23:06:32.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.940+0000 7f0d45483640 1 -- 172.21.15.32:0/382192983 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f0d40068430 con 0x7f0d40072440 2024-04-24T23:06:32.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.945+0000 7f0d277fe640 1 -- 172.21.15.32:0/382192983 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v64) ==== 118+0+0 (secure 0 0 0) 0x7f0d30049060 con 0x7f0d40072440 2024-04-24T23:06:32.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.947+0000 7f0d45483640 1 -- 172.21.15.32:0/382192983 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0d2c06be30 msgr2=0x7f0d2c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:32.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.947+0000 7f0d45483640 1 --2- 172.21.15.32:0/382192983 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0d2c06be30 0x7f0d2c06e2f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0d38001d30 tx=0x7f0d38007970 comp rx=0 tx=0).stop 2024-04-24T23:06:32.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.947+0000 7f0d45483640 1 -- 172.21.15.32:0/382192983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d40072440 msgr2=0x7f0d400728a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:32.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.947+0000 7f0d45483640 1 --2- 172.21.15.32:0/382192983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d40072440 0x7f0d400728a0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f0d3000e170 tx=0x7f0d30009750 comp rx=0 tx=0).stop 2024-04-24T23:06:32.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.947+0000 7f0d45483640 1 -- 172.21.15.32:0/382192983 shutdown_connections 2024-04-24T23:06:32.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.947+0000 7f0d45483640 1 --2- 172.21.15.32:0/382192983 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0d2c06be30 0x7f0d2c06e2f0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:32.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.947+0000 7f0d45483640 1 --2- 172.21.15.32:0/382192983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d40072440 0x7f0d400728a0 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:32.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.947+0000 7f0d45483640 1 --2- 172.21.15.32:0/382192983 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d40155fc0 0x7f0d4014ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:32.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.947+0000 7f0d45483640 1 --2- 172.21.15.32:0/382192983 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d28004d40 0x7f0d40158cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:32.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.947+0000 7f0d45483640 1 -- 172.21.15.32:0/382192983 >> 172.21.15.32:0/382192983 conn(0x7f0d4006b110 msgr2=0x7f0d40154770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:32.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.947+0000 7f0d45483640 1 -- 172.21.15.32:0/382192983 shutdown_connections 2024-04-24T23:06:32.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:32.947+0000 7f0d45483640 1 -- 172.21.15.32:0/382192983 wait complete. 2024-04-24T23:06:32.960 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-24T23:06:32.961 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-24T23:06:33.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.062+0000 7f159d4ba640 1 Processor -- start 2024-04-24T23:06:33.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.062+0000 7f159d4ba640 1 -- start start 2024-04-24T23:06:33.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.063+0000 7f159d4ba640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1598162e10 0x7f15980ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:33.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.063+0000 7f159d4ba640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15980cd270 con 0x7f15980b5af0 2024-04-24T23:06:33.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.063+0000 7f159d4ba640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1598071610 con 0x7f1598162320 2024-04-24T23:06:33.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.063+0000 7f159d4ba640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f15980b5660 con 0x7f1598162e10 2024-04-24T23:06:33.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.063+0000 7f15967fc640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f15980b5af0 0x7f159815ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33104/0 (socket says 172.21.15.32:33104) 2024-04-24T23:06:33.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.063+0000 7f15967fc640 1 -- 172.21.15.32:0/3696546948 learned_addr learned my addr 172.21.15.32:0/3696546948 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:06:33.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.063+0000 7f15977fe640 1 --2- 172.21.15.32:0/3696546948 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1598162e10 0x7f15980ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:33.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.063+0000 7f1595ffb640 1 -- 172.21.15.32:0/3696546948 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3708395697 0 0) 0x7f15980cd270 con 0x7f15980b5af0 2024-04-24T23:06:33.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.064+0000 7f1595ffb640 1 -- 172.21.15.32:0/3696546948 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1580003520 con 0x7f15980b5af0 2024-04-24T23:06:33.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.064+0000 7f1595ffb640 1 -- 172.21.15.32:0/3696546948 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 801844427 0 0) 0x7f1598071610 con 0x7f1598162320 2024-04-24T23:06:33.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.064+0000 7f1595ffb640 1 -- 172.21.15.32:0/3696546948 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f15980cd270 con 0x7f1598162320 2024-04-24T23:06:33.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.064+0000 7f15977fe640 1 -- 172.21.15.32:0/3696546948 >> v1:172.21.15.121:6789/0 conn(0x7f1598162320 legacy=0x7f1598162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:33.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.064+0000 7f15977fe640 1 -- 172.21.15.32:0/3696546948 >> v1:172.21.15.32:6789/0 conn(0x7f15980b5af0 legacy=0x7f159815ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:33.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.064+0000 7f15977fe640 1 -- 172.21.15.32:0/3696546948 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15980718f0 con 0x7f1598162e10 2024-04-24T23:06:33.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.064+0000 7f15977fe640 1 --2- 172.21.15.32:0/3696546948 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1598162e10 0x7f15980ccc90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1590009dc0 tx=0x7f159002a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40e746669be2164d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:33.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.065+0000 7f1595ffb640 1 -- 172.21.15.32:0/3696546948 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159002ae60 con 0x7f1598162e10 2024-04-24T23:06:33.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.065+0000 7f1595ffb640 1 -- 172.21.15.32:0/3696546948 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1590007740 con 0x7f1598162e10 2024-04-24T23:06:33.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.065+0000 7f1595ffb640 1 -- 172.21.15.32:0/3696546948 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1590007b10 con 0x7f1598162e10 2024-04-24T23:06:33.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.065+0000 7f159d4ba640 1 -- 172.21.15.32:0/3696546948 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1598162e10 msgr2=0x7f15980ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:33.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.065+0000 7f159d4ba640 1 --2- 172.21.15.32:0/3696546948 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1598162e10 0x7f15980ccc90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1590009dc0 tx=0x7f159002a640 comp rx=0 tx=0).stop 2024-04-24T23:06:33.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.065+0000 7f159d4ba640 1 -- 172.21.15.32:0/3696546948 shutdown_connections 2024-04-24T23:06:33.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.065+0000 7f159d4ba640 1 --2- 172.21.15.32:0/3696546948 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1598162e10 0x7f15980ccc90 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:33.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.065+0000 7f159d4ba640 1 -- 172.21.15.32:0/3696546948 >> 172.21.15.32:0/3696546948 conn(0x7f15980d0ae0 msgr2=0x7f15980d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:33.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.065+0000 7f159d4ba640 1 -- 172.21.15.32:0/3696546948 shutdown_connections 2024-04-24T23:06:33.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.065+0000 7f159d4ba640 1 -- 172.21.15.32:0/3696546948 wait complete. 2024-04-24T23:06:33.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.066+0000 7f159d4ba640 1 Processor -- start 2024-04-24T23:06:33.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.067+0000 7f159d4ba640 1 -- start start 2024-04-24T23:06:33.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.067+0000 7f159d4ba640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f15980b5af0 0x7f15981584d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:33.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.067+0000 7f159d4ba640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1598162320 0x7f15980cfc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:33.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.067+0000 7f159d4ba640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1598162e10 0x7f15981f4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:33.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.067+0000 7f159d4ba640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1598071bd0 con 0x7f1598162320 2024-04-24T23:06:33.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.067+0000 7f159d4ba640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1598067950 con 0x7f1598162e10 2024-04-24T23:06:33.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.067+0000 7f159d4ba640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f1598071d50 con 0x7f15980b5af0 2024-04-24T23:06:33.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.067+0000 7f15967fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1598162320 0x7f15980cfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:33.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.067+0000 7f15967fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1598162320 0x7f15980cfc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48468/0 (socket says 172.21.15.32:48468) 2024-04-24T23:06:33.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.067+0000 7f15967fc640 1 -- 172.21.15.32:0/2831514972 learned_addr learned my addr 172.21.15.32:0/2831514972 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:33.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.067+0000 7f15977fe640 1 --2- 172.21.15.32:0/2831514972 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1598162e10 0x7f15981f4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:33.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.067+0000 7f1596ffd640 1 --2- 172.21.15.32:0/2831514972 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f15980b5af0 0x7f15981584d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:33.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.068+0000 7f15967fc640 1 -- 172.21.15.32:0/2831514972 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f15980b5af0 msgr2=0x7f15981584d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:33.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.068+0000 7f15967fc640 1 --2- 172.21.15.32:0/2831514972 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f15980b5af0 0x7f15981584d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:33.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.068+0000 7f15967fc640 1 -- 172.21.15.32:0/2831514972 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1598162e10 msgr2=0x7f15981f4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:33.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.068+0000 7f15967fc640 1 --2- 172.21.15.32:0/2831514972 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1598162e10 0x7f15981f4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:33.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.068+0000 7f15967fc640 1 -- 172.21.15.32:0/2831514972 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1598158bb0 con 0x7f1598162320 2024-04-24T23:06:33.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.068+0000 7f15977fe640 1 --2- 172.21.15.32:0/2831514972 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1598162e10 0x7f15981f4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:06:33.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.068+0000 7f15967fc640 1 --2- 172.21.15.32:0/2831514972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1598162320 0x7f15980cfc90 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f158c00c500 tx=0x7f158c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:33.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.068+0000 7f157f7fe640 1 -- 172.21.15.32:0/2831514972 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158c017070 con 0x7f1598162320 2024-04-24T23:06:33.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.068+0000 7f159d4ba640 1 -- 172.21.15.32:0/2831514972 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1598158e90 con 0x7f1598162320 2024-04-24T23:06:33.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.068+0000 7f157f7fe640 1 -- 172.21.15.32:0/2831514972 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f158c005ce0 con 0x7f1598162320 2024-04-24T23:06:33.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.068+0000 7f157f7fe640 1 -- 172.21.15.32:0/2831514972 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158c0072f0 con 0x7f1598162320 2024-04-24T23:06:33.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.068+0000 7f159d4ba640 1 -- 172.21.15.32:0/2831514972 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1598159450 con 0x7f1598162320 2024-04-24T23:06:33.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.069+0000 7f159d4ba640 1 -- 172.21.15.32:0/2831514972 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15980690e0 con 0x7f1598162320 2024-04-24T23:06:33.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.070+0000 7f157f7fe640 1 -- 172.21.15.32:0/2831514972 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f158c01a070 con 0x7f1598162320 2024-04-24T23:06:33.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.070+0000 7f157f7fe640 1 --2- 172.21.15.32:0/2831514972 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f158406be30 0x7f158406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:33.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.070+0000 7f157f7fe640 1 -- 172.21.15.32:0/2831514972 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7f158c08a1d0 con 0x7f1598162320 2024-04-24T23:06:33.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.070+0000 7f1596ffd640 1 --2- 172.21.15.32:0/2831514972 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f158406be30 0x7f158406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:33.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.071+0000 7f1596ffd640 1 --2- 172.21.15.32:0/2831514972 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f158406be30 0x7f158406e2f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f1588007fd0 tx=0x7f1588009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:33.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.074+0000 7f157f7fe640 1 -- 172.21.15.32:0/2831514972 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f15980690e0 con 0x7f1598162320 2024-04-24T23:06:33.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.295+0000 7f159d4ba640 1 -- 172.21.15.32:0/2831514972 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f1598068430 con 0x7f1598162320 2024-04-24T23:06:33.918 INFO:tasks.ceph.mds.a.smithi032.stderr:2024-04-24T23:06:33.916+0000 7fb814aaa640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T23:06:33.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.950+0000 7f157f7fe640 1 -- 172.21.15.32:0/2831514972 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v65) ==== 122+0+0 (secure 0 0 0) 0x7f158c058540 con 0x7f1598162320 2024-04-24T23:06:33.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.953+0000 7f159d4ba640 1 -- 172.21.15.32:0/2831514972 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f158406be30 msgr2=0x7f158406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:33.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.953+0000 7f159d4ba640 1 --2- 172.21.15.32:0/2831514972 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f158406be30 0x7f158406e2f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f1588007fd0 tx=0x7f1588009040 comp rx=0 tx=0).stop 2024-04-24T23:06:33.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.953+0000 7f159d4ba640 1 -- 172.21.15.32:0/2831514972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1598162320 msgr2=0x7f15980cfc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:33.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.953+0000 7f159d4ba640 1 --2- 172.21.15.32:0/2831514972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1598162320 0x7f15980cfc90 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f158c00c500 tx=0x7f158c00c9c0 comp rx=0 tx=0).stop 2024-04-24T23:06:33.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.953+0000 7f159d4ba640 1 -- 172.21.15.32:0/2831514972 shutdown_connections 2024-04-24T23:06:33.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.953+0000 7f159d4ba640 1 --2- 172.21.15.32:0/2831514972 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f158406be30 0x7f158406e2f0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:33.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.953+0000 7f159d4ba640 1 --2- 172.21.15.32:0/2831514972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1598162320 0x7f15980cfc90 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:33.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.953+0000 7f159d4ba640 1 --2- 172.21.15.32:0/2831514972 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1598162e10 0x7f15981f4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:33.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.953+0000 7f159d4ba640 1 --2- 172.21.15.32:0/2831514972 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f15980b5af0 0x7f15981584d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:33.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.953+0000 7f159d4ba640 1 -- 172.21.15.32:0/2831514972 >> 172.21.15.32:0/2831514972 conn(0x7f15980d0ae0 msgr2=0x7f159815b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:33.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.953+0000 7f159d4ba640 1 -- 172.21.15.32:0/2831514972 shutdown_connections 2024-04-24T23:06:33.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:33.953+0000 7f159d4ba640 1 -- 172.21.15.32:0/2831514972 wait complete. 2024-04-24T23:06:33.967 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-24T23:06:33.967 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:06:34.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.071+0000 7f78d504d640 1 Processor -- start 2024-04-24T23:06:34.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.071+0000 7f78d504d640 1 -- start start 2024-04-24T23:06:34.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.072+0000 7f78d504d640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f78d006dc60 0x7f78d0169570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:34.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.072+0000 7f78d504d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78d00b6d10 con 0x7f78d006ba30 2024-04-24T23:06:34.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.072+0000 7f78d504d640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78d015ef10 con 0x7f78d0158dd0 2024-04-24T23:06:34.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.072+0000 7f78d504d640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f78d015f1f0 con 0x7f78d006dc60 2024-04-24T23:06:34.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.072+0000 7f78c7fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f78d006ba30 0x7f78d006d550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33120/0 (socket says 172.21.15.32:33120) 2024-04-24T23:06:34.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.072+0000 7f78c7fff640 1 -- 172.21.15.32:0/1688744312 learned_addr learned my addr 172.21.15.32:0/1688744312 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:06:34.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.072+0000 7f78cf59e640 1 --2- 172.21.15.32:0/1688744312 >> v2:172.21.15.121:3300/0 conn(0x7f78d006dc60 0x7f78d0169570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:34.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.072+0000 7f78ce59c640 1 -- 172.21.15.32:0/1688744312 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2797890326 0 0) 0x7f78d00b6d10 con 0x7f78d006ba30 2024-04-24T23:06:34.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.072+0000 7f78ce59c640 1 -- 172.21.15.32:0/1688744312 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f78bc003520 con 0x7f78d006ba30 2024-04-24T23:06:34.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.072+0000 7f78ce59c640 1 -- 172.21.15.32:0/1688744312 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 660144458 0 0) 0x7f78bc003520 con 0x7f78d006ba30 2024-04-24T23:06:34.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.072+0000 7f78ce59c640 1 -- 172.21.15.32:0/1688744312 >> v2:172.21.15.121:3300/0 conn(0x7f78d006dc60 msgr2=0x7f78d0169570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:34.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.072+0000 7f78ce59c640 1 --2- 172.21.15.32:0/1688744312 >> v2:172.21.15.121:3300/0 conn(0x7f78d006dc60 0x7f78d0169570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:34.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.072+0000 7f78ce59c640 1 -- 172.21.15.32:0/1688744312 >> v1:172.21.15.121:6789/0 conn(0x7f78d0158dd0 legacy=0x7f78d006b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:34.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.072+0000 7f78ce59c640 1 -- 172.21.15.32:0/1688744312 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78d015f4d0 con 0x7f78d006ba30 2024-04-24T23:06:34.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.072+0000 7f78ce59c640 1 -- 172.21.15.32:0/1688744312 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f78b8001670 con 0x7f78d006ba30 2024-04-24T23:06:34.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.073+0000 7f78ce59c640 1 -- 172.21.15.32:0/1688744312 >> v1:172.21.15.32:6789/0 conn(0x7f78d006ba30 legacy=0x7f78d006d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:34.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.073+0000 7f78ce59c640 1 --2- 172.21.15.32:0/1688744312 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78bc0039b0 0x7f78bc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:34.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.073+0000 7f78ce59c640 1 --2- 172.21.15.32:0/1688744312 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f78bc0043b0 0x7f78bc004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:34.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.073+0000 7f78ce59c640 1 --2- 172.21.15.32:0/1688744312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78bc005080 0x7f78bc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:34.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.073+0000 7f78ce59c640 1 -- 172.21.15.32:0/1688744312 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f78d015f1f0 con 0x7f78bc005080 2024-04-24T23:06:34.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.073+0000 7f78ce59c640 1 -- 172.21.15.32:0/1688744312 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc00bfe0 con 0x7f78bc0039b0 2024-04-24T23:06:34.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.073+0000 7f78ce59c640 1 -- 172.21.15.32:0/1688744312 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f78bc00c1e0 con 0x7f78bc0043b0 2024-04-24T23:06:34.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.073+0000 7f78cf59e640 1 --2- 172.21.15.32:0/1688744312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78bc005080 0x7f78bc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:34.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.073+0000 7f78ced9d640 1 --2- 172.21.15.32:0/1688744312 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78bc0039b0 0x7f78bc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:34.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.073+0000 7f78c7fff640 1 --2- 172.21.15.32:0/1688744312 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f78bc0043b0 0x7f78bc004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:34.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.073+0000 7f78cf59e640 1 -- 172.21.15.32:0/1688744312 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f78bc0043b0 msgr2=0x7f78bc004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:34.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.073+0000 7f78cf59e640 1 --2- 172.21.15.32:0/1688744312 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f78bc0043b0 0x7f78bc004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:34.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.073+0000 7f78cf59e640 1 -- 172.21.15.32:0/1688744312 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78bc0039b0 msgr2=0x7f78bc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:34.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.073+0000 7f78cf59e640 1 --2- 172.21.15.32:0/1688744312 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78bc0039b0 0x7f78bc003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:34.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.073+0000 7f78cf59e640 1 -- 172.21.15.32:0/1688744312 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78bc003520 con 0x7f78bc005080 2024-04-24T23:06:34.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.074+0000 7f78cf59e640 1 -- 172.21.15.32:0/1688744312 reap_dead start 2024-04-24T23:06:34.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.074+0000 7f78cf59e640 1 --2- 172.21.15.32:0/1688744312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78bc005080 0x7f78bc00b9c0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f78c8009fc0 tx=0x7f78bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=492c8b9a7251bbd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:34.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.074+0000 7f78ce59c640 1 -- 172.21.15.32:0/1688744312 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c8035020 con 0x7f78bc005080 2024-04-24T23:06:34.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.074+0000 7f78ce59c640 1 -- 172.21.15.32:0/1688744312 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78c8003bd0 con 0x7f78bc005080 2024-04-24T23:06:34.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.074+0000 7f78ce59c640 1 -- 172.21.15.32:0/1688744312 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c8009c10 con 0x7f78bc005080 2024-04-24T23:06:34.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.074+0000 7f78d504d640 1 -- 172.21.15.32:0/1688744312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78bc005080 msgr2=0x7f78bc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:34.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.074+0000 7f78d504d640 1 --2- 172.21.15.32:0/1688744312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78bc005080 0x7f78bc00b9c0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f78c8009fc0 tx=0x7f78bc005830 comp rx=0 tx=0).stop 2024-04-24T23:06:34.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.074+0000 7f78d504d640 1 -- 172.21.15.32:0/1688744312 shutdown_connections 2024-04-24T23:06:34.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.074+0000 7f78d504d640 1 --2- 172.21.15.32:0/1688744312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78bc005080 0x7f78bc00b9c0 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:34.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.074+0000 7f78d504d640 1 -- 172.21.15.32:0/1688744312 >> 172.21.15.32:0/1688744312 conn(0x7f78d006e800 msgr2=0x7f78d006ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:34.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.075+0000 7f78d504d640 1 -- 172.21.15.32:0/1688744312 shutdown_connections 2024-04-24T23:06:34.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.075+0000 7f78d504d640 1 -- 172.21.15.32:0/1688744312 wait complete. 2024-04-24T23:06:34.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.076+0000 7f78d504d640 1 Processor -- start 2024-04-24T23:06:34.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.076+0000 7f78d504d640 1 -- start start 2024-04-24T23:06:34.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.077+0000 7f78d504d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f78bc005080 0x7f78d0146ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:34.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.077+0000 7f78d504d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78d01387b0 0x7f78d01f3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:34.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.077+0000 7f78d504d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78d0160b40 0x7f78d0160fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:34.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.077+0000 7f78d504d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f78d006bfe0 con 0x7f78d01387b0 2024-04-24T23:06:34.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.077+0000 7f78d504d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f78d015fcd0 con 0x7f78d0160b40 2024-04-24T23:06:34.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.077+0000 7f78d504d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f78d01591b0 con 0x7f78bc005080 2024-04-24T23:06:34.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.077+0000 7f78c7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78d01387b0 0x7f78d01f3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:34.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.077+0000 7f78c7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78d01387b0 0x7f78d01f3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48484/0 (socket says 172.21.15.32:48484) 2024-04-24T23:06:34.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.077+0000 7f78c7fff640 1 -- 172.21.15.32:0/732319943 learned_addr learned my addr 172.21.15.32:0/732319943 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:34.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.077+0000 7f78ced9d640 1 --2- 172.21.15.32:0/732319943 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f78bc005080 0x7f78d0146ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:34.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.077+0000 7f78cf59e640 1 --2- 172.21.15.32:0/732319943 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78d0160b40 0x7f78d0160fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:34.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.077+0000 7f78c7fff640 1 -- 172.21.15.32:0/732319943 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f78bc005080 msgr2=0x7f78d0146ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:34.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.077+0000 7f78c7fff640 1 --2- 172.21.15.32:0/732319943 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f78bc005080 0x7f78d0146ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:34.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.077+0000 7f78c7fff640 1 -- 172.21.15.32:0/732319943 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78d0160b40 msgr2=0x7f78d0160fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:34.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.077+0000 7f78c7fff640 1 --2- 172.21.15.32:0/732319943 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78d0160b40 0x7f78d0160fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:34.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.077+0000 7f78c7fff640 1 -- 172.21.15.32:0/732319943 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78d01f7a40 con 0x7f78d01387b0 2024-04-24T23:06:34.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.077+0000 7f78c7fff640 1 --2- 172.21.15.32:0/732319943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78d01387b0 0x7f78d01f3bb0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f78b80072c0 tx=0x7f78b8007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:34.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.077+0000 7f78c77fe640 1 -- 172.21.15.32:0/732319943 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b800f940 con 0x7f78d01387b0 2024-04-24T23:06:34.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.077+0000 7f78c77fe640 1 -- 172.21.15.32:0/732319943 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78b8018070 con 0x7f78d01387b0 2024-04-24T23:06:34.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.077+0000 7f78c77fe640 1 -- 172.21.15.32:0/732319943 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b800fd40 con 0x7f78d01387b0 2024-04-24T23:06:34.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.077+0000 7f78d504d640 1 -- 172.21.15.32:0/732319943 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78d01f7d20 con 0x7f78d01387b0 2024-04-24T23:06:34.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.077+0000 7f78d504d640 1 -- 172.21.15.32:0/732319943 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78d01f82e0 con 0x7f78d01387b0 2024-04-24T23:06:34.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.078+0000 7f78d504d640 1 -- 172.21.15.32:0/732319943 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78d0068430 con 0x7f78d01387b0 2024-04-24T23:06:34.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.079+0000 7f78c77fe640 1 -- 172.21.15.32:0/732319943 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f78b801d460 con 0x7f78d01387b0 2024-04-24T23:06:34.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.079+0000 7f78c77fe640 1 --2- 172.21.15.32:0/732319943 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f78b006be60 0x7f78b006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:34.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.079+0000 7f78c77fe640 1 -- 172.21.15.32:0/732319943 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7f78b808aeb0 con 0x7f78d01387b0 2024-04-24T23:06:34.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.080+0000 7f78ced9d640 1 --2- 172.21.15.32:0/732319943 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f78b006be60 0x7f78b006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:34.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.080+0000 7f78ced9d640 1 --2- 172.21.15.32:0/732319943 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f78b006be60 0x7f78b006e320 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f78c0003640 tx=0x7f78c0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:34.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.084+0000 7f78c77fe640 1 -- 172.21.15.32:0/732319943 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f78b80112b0 con 0x7f78d01387b0 2024-04-24T23:06:34.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.306+0000 7f78d504d640 1 -- 172.21.15.32:0/732319943 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78d0160620 con 0x7f78d01387b0 2024-04-24T23:06:34.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.307+0000 7f78c77fe640 1 -- 172.21.15.32:0/732319943 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+5186 (secure 0 0 0) 0x7f78b8059860 con 0x7f78d01387b0 2024-04-24T23:06:34.308 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:06:34.309 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":65,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7120,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/1196305174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1196305174},{"type":"v1","addr":"172.21.15.121:6836","nonce":1196305174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7123,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3015902884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3015902884},{"type":"v1","addr":"172.21.15.121:6837","nonce":3015902884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":8280,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/1033273628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1033273628},{"type":"v1","addr":"172.21.15.32:6833","nonce":1033273628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:06:31.291599+0000","modified":"2024-04-24T23:06:33.296657+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8286},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8286":{"gid":8286,"name":"a","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/984603360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":984603360},{"type":"v1","addr":"172.21.15.32:6835","nonce":984603360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8286,"qdb_cluster":[8286]},"id":9}]} 2024-04-24T23:06:34.309 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 65 2024-04-24T23:06:34.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.309+0000 7f78d504d640 1 -- 172.21.15.32:0/732319943 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f78b006be60 msgr2=0x7f78b006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:34.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.309+0000 7f78d504d640 1 --2- 172.21.15.32:0/732319943 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f78b006be60 0x7f78b006e320 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f78c0003640 tx=0x7f78c0009040 comp rx=0 tx=0).stop 2024-04-24T23:06:34.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.309+0000 7f78d504d640 1 -- 172.21.15.32:0/732319943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78d01387b0 msgr2=0x7f78d01f3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:34.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.309+0000 7f78d504d640 1 --2- 172.21.15.32:0/732319943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78d01387b0 0x7f78d01f3bb0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f78b80072c0 tx=0x7f78b8007780 comp rx=0 tx=0).stop 2024-04-24T23:06:34.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.309+0000 7f78d504d640 1 -- 172.21.15.32:0/732319943 shutdown_connections 2024-04-24T23:06:34.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.309+0000 7f78d504d640 1 --2- 172.21.15.32:0/732319943 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f78b006be60 0x7f78b006e320 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:34.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.309+0000 7f78d504d640 1 --2- 172.21.15.32:0/732319943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78d01387b0 0x7f78d01f3bb0 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:34.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.309+0000 7f78d504d640 1 --2- 172.21.15.32:0/732319943 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78d0160b40 0x7f78d0160fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:34.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.309+0000 7f78d504d640 1 --2- 172.21.15.32:0/732319943 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f78bc005080 0x7f78d0146ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:34.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.309+0000 7f78d504d640 1 -- 172.21.15.32:0/732319943 >> 172.21.15.32:0/732319943 conn(0x7f78d006e800 msgr2=0x7f78d0143f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:34.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.309+0000 7f78d504d640 1 -- 172.21.15.32:0/732319943 shutdown_connections 2024-04-24T23:06:34.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.309+0000 7f78d504d640 1 -- 172.21.15.32:0/732319943 wait complete. 2024-04-24T23:06:34.322 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:06:34.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.421+0000 7f5d13238640 1 Processor -- start 2024-04-24T23:06:34.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.421+0000 7f5d13238640 1 -- start start 2024-04-24T23:06:34.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.421+0000 7f5d13238640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5d0c0b44d0 0x7f5d0c0b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:34.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.421+0000 7f5d13238640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5d0c0af890 0x7f5d0c15cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:34.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.421+0000 7f5d13238640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d0c0b6430 con 0x7f5d0c06c5f0 2024-04-24T23:06:34.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.421+0000 7f5d13238640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f5d0c156f80 con 0x7f5d0c0af890 2024-04-24T23:06:34.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.421+0000 7f5d13238640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f5d0c0b6150 con 0x7f5d0c0b44d0 2024-04-24T23:06:34.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.422+0000 7f5d0bfff640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5d0c0b44d0 0x7f5d0c0b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:34.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.422+0000 7f5d117d5640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5d0c0af890 0x7f5d0c15cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:34.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.422+0000 7f5d117d5640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5d0c0af890 0x7f5d0c15cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.32:40328/0 (socket says 172.21.15.32:40328) 2024-04-24T23:06:34.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.422+0000 7f5d0bfff640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5d0c0b44d0 0x7f5d0c0b5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.32:56704/0 (socket says 172.21.15.32:56704) 2024-04-24T23:06:34.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.422+0000 7f5d117d5640 1 -- 172.21.15.32:0/296632021 learned_addr learned my addr 172.21.15.32:0/296632021 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:34.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.422+0000 7f5d117d5640 1 -- 172.21.15.32:0/296632021 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5d0c0b44d0 msgr2=0x7f5d0c0b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:34.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.422+0000 7f5d117d5640 1 --2- 172.21.15.32:0/296632021 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5d0c0b44d0 0x7f5d0c0b5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:34.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.422+0000 7f5d117d5640 1 -- 172.21.15.32:0/296632021 >> v1:172.21.15.121:6789/0 conn(0x7f5d0c06c5f0 legacy=0x7f5d0c06c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:34.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.422+0000 7f5d117d5640 1 -- 172.21.15.32:0/296632021 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d0c0b39f0 con 0x7f5d0c0af890 2024-04-24T23:06:34.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.422+0000 7f5d0b7fe640 1 -- 172.21.15.32:0/296632021 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 219692226 0 0) 0x7f5d0c0b6430 con 0x7f5d0c06c5f0 2024-04-24T23:06:34.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.423+0000 7f5d117d5640 1 --2- 172.21.15.32:0/296632021 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d0c0af890 0x7f5d0c15cac0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f5d04009e50 tx=0x7f5d0402a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66883f255bb422d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:34.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.423+0000 7f5d0b7fe640 1 -- 172.21.15.32:0/296632021 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d04036070 con 0x7f5d0c0af890 2024-04-24T23:06:34.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.423+0000 7f5d0b7fe640 1 -- 172.21.15.32:0/296632021 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d04031600 con 0x7f5d0c0af890 2024-04-24T23:06:34.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.423+0000 7f5d0b7fe640 1 -- 172.21.15.32:0/296632021 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d040319d0 con 0x7f5d0c0af890 2024-04-24T23:06:34.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.424+0000 7f5d13238640 1 -- 172.21.15.32:0/296632021 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d0c0af890 msgr2=0x7f5d0c15cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:34.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.424+0000 7f5d13238640 1 --2- 172.21.15.32:0/296632021 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d0c0af890 0x7f5d0c15cac0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f5d04009e50 tx=0x7f5d0402a680 comp rx=0 tx=0).stop 2024-04-24T23:06:34.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.424+0000 7f5d13238640 1 -- 172.21.15.32:0/296632021 shutdown_connections 2024-04-24T23:06:34.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.424+0000 7f5d13238640 1 --2- 172.21.15.32:0/296632021 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d0c0af890 0x7f5d0c15cac0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:34.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.424+0000 7f5d13238640 1 --2- 172.21.15.32:0/296632021 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5d0c0b44d0 0x7f5d0c0b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:34.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.424+0000 7f5d13238640 1 -- 172.21.15.32:0/296632021 >> 172.21.15.32:0/296632021 conn(0x7f5d0c06b110 msgr2=0x7f5d0c06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:34.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.424+0000 7f5d13238640 1 -- 172.21.15.32:0/296632021 shutdown_connections 2024-04-24T23:06:34.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.424+0000 7f5d13238640 1 -- 172.21.15.32:0/296632021 wait complete. 2024-04-24T23:06:34.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.425+0000 7f5d13238640 1 Processor -- start 2024-04-24T23:06:34.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.425+0000 7f5d13238640 1 -- start start 2024-04-24T23:06:34.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.426+0000 7f5d13238640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5d0c06c5f0 0x7f5d0c151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:34.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.426+0000 7f5d13238640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d0c0af890 0x7f5d0c0ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:34.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.426+0000 7f5d13238640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d0c0b44d0 0x7f5d0c0aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:34.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.426+0000 7f5d13238640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0c0b5dd0 con 0x7f5d0c0af890 2024-04-24T23:06:34.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.426+0000 7f5d13238640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0c0b3fb0 con 0x7f5d0c0b44d0 2024-04-24T23:06:34.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.426+0000 7f5d13238640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f5d0c15d000 con 0x7f5d0c06c5f0 2024-04-24T23:06:34.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.426+0000 7f5d0bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d0c0af890 0x7f5d0c0ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:34.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.426+0000 7f5d0bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d0c0af890 0x7f5d0c0ae8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48486/0 (socket says 172.21.15.32:48486) 2024-04-24T23:06:34.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.426+0000 7f5d0bfff640 1 -- 172.21.15.32:0/478017731 learned_addr learned my addr 172.21.15.32:0/478017731 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:34.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.426+0000 7f5d117d5640 1 --2- 172.21.15.32:0/478017731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d0c0b44d0 0x7f5d0c0aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:34.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.426+0000 7f5d10fd4640 1 --2- 172.21.15.32:0/478017731 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5d0c06c5f0 0x7f5d0c151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:34.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.426+0000 7f5d0bfff640 1 -- 172.21.15.32:0/478017731 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5d0c06c5f0 msgr2=0x7f5d0c151ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:34.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.426+0000 7f5d0bfff640 1 --2- 172.21.15.32:0/478017731 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5d0c06c5f0 0x7f5d0c151ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:34.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.426+0000 7f5d0bfff640 1 -- 172.21.15.32:0/478017731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d0c0b44d0 msgr2=0x7f5d0c0aedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:34.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.426+0000 7f5d0bfff640 1 --2- 172.21.15.32:0/478017731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d0c0b44d0 0x7f5d0c0aedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:34.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.426+0000 7f5d0bfff640 1 -- 172.21.15.32:0/478017731 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d0c0afe80 con 0x7f5d0c0af890 2024-04-24T23:06:34.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.427+0000 7f5d0bfff640 1 --2- 172.21.15.32:0/478017731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d0c0af890 0x7f5d0c0ae8b0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f5cf800e8a0 tx=0x7f5cf800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:34.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.427+0000 7f5d097fa640 1 -- 172.21.15.32:0/478017731 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf8017070 con 0x7f5d0c0af890 2024-04-24T23:06:34.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.427+0000 7f5d097fa640 1 -- 172.21.15.32:0/478017731 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cf800c940 con 0x7f5d0c0af890 2024-04-24T23:06:34.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.427+0000 7f5d13238640 1 -- 172.21.15.32:0/478017731 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d0c0b0120 con 0x7f5d0c0af890 2024-04-24T23:06:34.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.427+0000 7f5d097fa640 1 -- 172.21.15.32:0/478017731 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf800cd30 con 0x7f5d0c0af890 2024-04-24T23:06:34.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.427+0000 7f5d13238640 1 -- 172.21.15.32:0/478017731 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d0c0b06e0 con 0x7f5d0c0af890 2024-04-24T23:06:34.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.428+0000 7f5d13238640 1 -- 172.21.15.32:0/478017731 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d0c068430 con 0x7f5d0c0af890 2024-04-24T23:06:34.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.428+0000 7f5d097fa640 1 -- 172.21.15.32:0/478017731 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5cf801a070 con 0x7f5d0c0af890 2024-04-24T23:06:34.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.429+0000 7f5d097fa640 1 --2- 172.21.15.32:0/478017731 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5cf006be60 0x7f5cf006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:34.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.429+0000 7f5d097fa640 1 -- 172.21.15.32:0/478017731 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7f5cf808ac20 con 0x7f5d0c0af890 2024-04-24T23:06:34.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.429+0000 7f5d10fd4640 1 --2- 172.21.15.32:0/478017731 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5cf006be60 0x7f5cf006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:34.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.430+0000 7f5d10fd4640 1 --2- 172.21.15.32:0/478017731 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5cf006be60 0x7f5cf006e320 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f5d00000f60 tx=0x7f5d00009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:34.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.433+0000 7f5d097fa640 1 -- 172.21.15.32:0/478017731 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5cf8058f80 con 0x7f5d0c0af890 2024-04-24T23:06:34.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.617+0000 7f5d13238640 1 -- 172.21.15.32:0/478017731 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5d0c0b5670 con 0x7f5d0c0af890 2024-04-24T23:06:34.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.618+0000 7f5d097fa640 1 -- 172.21.15.32:0/478017731 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) ==== 74+0+16640 (secure 0 0 0) 0x7f5cf805d280 con 0x7f5d0c0af890 2024-04-24T23:06:34.620 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:06:34.620 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":156,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:06:32.932580+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:06:29.591736+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-24T23:06:30.267629+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:06:34.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.620+0000 7f5d13238640 1 -- 172.21.15.32:0/478017731 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5cf006be60 msgr2=0x7f5cf006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:34.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.620+0000 7f5d13238640 1 --2- 172.21.15.32:0/478017731 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5cf006be60 0x7f5cf006e320 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f5d00000f60 tx=0x7f5d00009040 comp rx=0 tx=0).stop 2024-04-24T23:06:34.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.620+0000 7f5d13238640 1 -- 172.21.15.32:0/478017731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d0c0af890 msgr2=0x7f5d0c0ae8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:34.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.620+0000 7f5d13238640 1 --2- 172.21.15.32:0/478017731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d0c0af890 0x7f5d0c0ae8b0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f5cf800e8a0 tx=0x7f5cf800ed60 comp rx=0 tx=0).stop 2024-04-24T23:06:34.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.620+0000 7f5d13238640 1 -- 172.21.15.32:0/478017731 shutdown_connections 2024-04-24T23:06:34.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.620+0000 7f5d13238640 1 --2- 172.21.15.32:0/478017731 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5cf006be60 0x7f5cf006e320 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:34.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.620+0000 7f5d13238640 1 --2- 172.21.15.32:0/478017731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d0c0af890 0x7f5d0c0ae8b0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:34.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.620+0000 7f5d13238640 1 --2- 172.21.15.32:0/478017731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d0c0b44d0 0x7f5d0c0aedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:34.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.620+0000 7f5d13238640 1 --2- 172.21.15.32:0/478017731 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5d0c06c5f0 0x7f5d0c151ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:34.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.620+0000 7f5d13238640 1 -- 172.21.15.32:0/478017731 >> 172.21.15.32:0/478017731 conn(0x7f5d0c06b110 msgr2=0x7f5d0c155580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:34.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.620+0000 7f5d13238640 1 -- 172.21.15.32:0/478017731 shutdown_connections 2024-04-24T23:06:34.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.620+0000 7f5d13238640 1 -- 172.21.15.32:0/478017731 wait complete. 2024-04-24T23:06:34.633 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-24T23:06:34.633 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:06:34.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.732+0000 7f6ff6bce640 1 Processor -- start 2024-04-24T23:06:34.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.732+0000 7f6ff6bce640 1 -- start start 2024-04-24T23:06:34.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.733+0000 7f6ff6bce640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6ff015b380 0x7f6ff006c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:34.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.733+0000 7f6ff6bce640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6ff015cb00 0x7f6ff006cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:34.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.733+0000 7f6ff6bce640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ff00ce430 0x7f6ff00ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:34.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.733+0000 7f6ff6bce640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f6ff00b4520 con 0x7f6ff015b380 2024-04-24T23:06:34.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.733+0000 7f6ff6bce640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f6ff00b4800 con 0x7f6ff015cb00 2024-04-24T23:06:34.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.733+0000 7f6ff6bce640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f6ff00b4ae0 con 0x7f6ff00ce430 2024-04-24T23:06:34.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.733+0000 7f6ff496a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6ff015b380 0x7f6ff006c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:34.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.733+0000 7f6ff496a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6ff015b380 0x7f6ff006c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48496/0 (socket says 172.21.15.32:48496) 2024-04-24T23:06:34.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.733+0000 7f6ff496a640 1 -- 172.21.15.32:0/3706667971 learned_addr learned my addr 172.21.15.32:0/3706667971 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:34.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.733+0000 7f6feffff640 1 --2- 172.21.15.32:0/3706667971 >> v2:172.21.15.121:3300/0 conn(0x7f6ff015cb00 0x7f6ff006cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:34.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.733+0000 7f6ff516b640 1 --2- 172.21.15.32:0/3706667971 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ff00ce430 0x7f6ff00ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:34.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.733+0000 7f6ff496a640 1 -- 172.21.15.32:0/3706667971 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ff00ce430 msgr2=0x7f6ff00ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:34.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.733+0000 7f6ff496a640 1 --2- 172.21.15.32:0/3706667971 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ff00ce430 0x7f6ff00ce860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:34.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.733+0000 7f6ff496a640 1 -- 172.21.15.32:0/3706667971 >> v2:172.21.15.121:3300/0 conn(0x7f6ff015cb00 msgr2=0x7f6ff006cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:34.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.733+0000 7f6ff496a640 1 --2- 172.21.15.32:0/3706667971 >> v2:172.21.15.121:3300/0 conn(0x7f6ff015cb00 0x7f6ff006cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:34.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.733+0000 7f6ff496a640 1 -- 172.21.15.32:0/3706667971 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ff00b4dc0 con 0x7f6ff015b380 2024-04-24T23:06:34.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.734+0000 7f6ff496a640 1 --2- 172.21.15.32:0/3706667971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ff015b380 0x7f6ff006c5f0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f6ff00d12e0 tx=0x7f6fe402a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d041f61af4045aea server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:34.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.734+0000 7f6fef7fe640 1 -- 172.21.15.32:0/3706667971 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe4037070 con 0x7f6ff015b380 2024-04-24T23:06:34.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.734+0000 7f6fef7fe640 1 -- 172.21.15.32:0/3706667971 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fe402ae00 con 0x7f6ff015b380 2024-04-24T23:06:34.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.734+0000 7f6fef7fe640 1 -- 172.21.15.32:0/3706667971 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe4032660 con 0x7f6ff015b380 2024-04-24T23:06:34.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.735+0000 7f6ff6bce640 1 -- 172.21.15.32:0/3706667971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ff015b380 msgr2=0x7f6ff006c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:34.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.735+0000 7f6ff6bce640 1 --2- 172.21.15.32:0/3706667971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ff015b380 0x7f6ff006c5f0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f6ff00d12e0 tx=0x7f6fe402a0b0 comp rx=0 tx=0).stop 2024-04-24T23:06:34.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.735+0000 7f6ff6bce640 1 -- 172.21.15.32:0/3706667971 shutdown_connections 2024-04-24T23:06:34.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.735+0000 7f6ff6bce640 1 --2- 172.21.15.32:0/3706667971 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ff00ce430 0x7f6ff00ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:34.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.735+0000 7f6ff6bce640 1 --2- 172.21.15.32:0/3706667971 >> v2:172.21.15.121:3300/0 conn(0x7f6ff015cb00 0x7f6ff006cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:34.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.735+0000 7f6ff6bce640 1 --2- 172.21.15.32:0/3706667971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ff015b380 0x7f6ff006c5f0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:34.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.735+0000 7f6ff6bce640 1 -- 172.21.15.32:0/3706667971 >> 172.21.15.32:0/3706667971 conn(0x7f6ff006b110 msgr2=0x7f6ff006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:34.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.735+0000 7f6ff6bce640 1 -- 172.21.15.32:0/3706667971 shutdown_connections 2024-04-24T23:06:34.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.735+0000 7f6ff6bce640 1 -- 172.21.15.32:0/3706667971 wait complete. 2024-04-24T23:06:34.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.736+0000 7f6ff6bce640 1 Processor -- start 2024-04-24T23:06:34.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.736+0000 7f6ff6bce640 1 -- start start 2024-04-24T23:06:34.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.737+0000 7f6ff6bce640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ff00ce430 0x7f6ff0147110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:34.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.737+0000 7f6ff6bce640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ff015b380 0x7f6ff01541f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:34.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.737+0000 7f6ff6bce640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ff015cb00 0x7f6ff015da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:34.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.737+0000 7f6ff6bce640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff00ceda0 con 0x7f6ff015cb00 2024-04-24T23:06:34.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.737+0000 7f6ff6bce640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff015b760 con 0x7f6ff00ce430 2024-04-24T23:06:34.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.737+0000 7f6ff6bce640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f6ff00cef20 con 0x7f6ff015b380 2024-04-24T23:06:34.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.737+0000 7f6ff516b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ff015cb00 0x7f6ff015da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:34.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.737+0000 7f6ff516b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ff015cb00 0x7f6ff015da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48510/0 (socket says 172.21.15.32:48510) 2024-04-24T23:06:34.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.737+0000 7f6ff516b640 1 -- 172.21.15.32:0/3119483074 learned_addr learned my addr 172.21.15.32:0/3119483074 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:34.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.737+0000 7f6feffff640 1 --2- 172.21.15.32:0/3119483074 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ff015b380 0x7f6ff01541f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:34.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.737+0000 7f6ff496a640 1 --2- 172.21.15.32:0/3119483074 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ff00ce430 0x7f6ff0147110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:34.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.737+0000 7f6ff516b640 1 -- 172.21.15.32:0/3119483074 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ff015b380 msgr2=0x7f6ff01541f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:34.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.737+0000 7f6ff516b640 1 --2- 172.21.15.32:0/3119483074 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ff015b380 0x7f6ff01541f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:34.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.737+0000 7f6ff516b640 1 -- 172.21.15.32:0/3119483074 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ff00ce430 msgr2=0x7f6ff0147110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:34.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.737+0000 7f6ff516b640 1 --2- 172.21.15.32:0/3119483074 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ff00ce430 0x7f6ff0147110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:34.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.737+0000 7f6ff516b640 1 -- 172.21.15.32:0/3119483074 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ff014f380 con 0x7f6ff015cb00 2024-04-24T23:06:34.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.737+0000 7f6ff496a640 1 --2- 172.21.15.32:0/3119483074 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ff00ce430 0x7f6ff0147110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:06:34.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.737+0000 7f6feffff640 1 --2- 172.21.15.32:0/3119483074 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ff015b380 0x7f6ff01541f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:06:34.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.737+0000 7f6ff516b640 1 --2- 172.21.15.32:0/3119483074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ff015cb00 0x7f6ff015da70 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8007e90 tx=0x7f6fe800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:34.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.737+0000 7f6fed7fa640 1 -- 172.21.15.32:0/3119483074 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe8014420 con 0x7f6ff015cb00 2024-04-24T23:06:34.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.738+0000 7f6fed7fa640 1 -- 172.21.15.32:0/3119483074 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fe8019070 con 0x7f6ff015cb00 2024-04-24T23:06:34.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.738+0000 7f6ff6bce640 1 -- 172.21.15.32:0/3119483074 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ff014f5f0 con 0x7f6ff015cb00 2024-04-24T23:06:34.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.738+0000 7f6fed7fa640 1 -- 172.21.15.32:0/3119483074 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe8014820 con 0x7f6ff015cb00 2024-04-24T23:06:34.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.738+0000 7f6ff6bce640 1 -- 172.21.15.32:0/3119483074 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ff014fbb0 con 0x7f6ff015cb00 2024-04-24T23:06:34.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.739+0000 7f6ff6bce640 1 -- 172.21.15.32:0/3119483074 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ff0068430 con 0x7f6ff015cb00 2024-04-24T23:06:34.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.739+0000 7f6fed7fa640 1 -- 172.21.15.32:0/3119483074 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6fe80149c0 con 0x7f6ff015cb00 2024-04-24T23:06:34.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.739+0000 7f6fed7fa640 1 --2- 172.21.15.32:0/3119483074 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6fd406be60 0x7f6fd406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:34.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.740+0000 7f6fed7fa640 1 -- 172.21.15.32:0/3119483074 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7f6fe808b510 con 0x7f6ff015cb00 2024-04-24T23:06:34.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.740+0000 7f6ff496a640 1 --2- 172.21.15.32:0/3119483074 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6fd406be60 0x7f6fd406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:34.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.740+0000 7f6ff496a640 1 --2- 172.21.15.32:0/3119483074 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6fd406be60 0x7f6fd406e320 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4034000 tx=0x7f6fe402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:34.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.744+0000 7f6fed7fa640 1 -- 172.21.15.32:0/3119483074 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6fe8059f50 con 0x7f6ff015cb00 2024-04-24T23:06:34.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.922+0000 7f6ff6bce640 1 -- 172.21.15.32:0/3119483074 --> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6ff00afd60 con 0x7f6fd406be60 2024-04-24T23:06:34.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.929+0000 7f6fed7fa640 1 -- 172.21.15.32:0/3119483074 <== mgr.4108 v2:172.21.15.121:6832/1848157943 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247929 (secure 0 0 0) 0x7f6fe407e890 con 0x7f6fd406be60 2024-04-24T23:06:34.933 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:06:34.935 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-24T23:06:34.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.932+0000 7f6ff6bce640 1 -- 172.21.15.32:0/3119483074 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6fd406be60 msgr2=0x7f6fd406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:34.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.932+0000 7f6ff6bce640 1 --2- 172.21.15.32:0/3119483074 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6fd406be60 0x7f6fd406e320 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4034000 tx=0x7f6fe402e000 comp rx=0 tx=0).stop 2024-04-24T23:06:34.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.932+0000 7f6ff6bce640 1 -- 172.21.15.32:0/3119483074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ff015cb00 msgr2=0x7f6ff015da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:34.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.932+0000 7f6ff6bce640 1 --2- 172.21.15.32:0/3119483074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ff015cb00 0x7f6ff015da70 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8007e90 tx=0x7f6fe800e4b0 comp rx=0 tx=0).stop 2024-04-24T23:06:34.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.932+0000 7f6ff6bce640 1 -- 172.21.15.32:0/3119483074 shutdown_connections 2024-04-24T23:06:34.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.932+0000 7f6ff6bce640 1 --2- 172.21.15.32:0/3119483074 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6fd406be60 0x7f6fd406e320 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:34.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.932+0000 7f6ff6bce640 1 --2- 172.21.15.32:0/3119483074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ff015cb00 0x7f6ff015da70 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:34.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.932+0000 7f6ff6bce640 1 --2- 172.21.15.32:0/3119483074 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ff015b380 0x7f6ff01541f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:34.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.932+0000 7f6ff6bce640 1 --2- 172.21.15.32:0/3119483074 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ff00ce430 0x7f6ff0147110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:34.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.932+0000 7f6ff6bce640 1 -- 172.21.15.32:0/3119483074 >> 172.21.15.32:0/3119483074 conn(0x7f6ff006b110 msgr2=0x7f6ff00b54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:34.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.932+0000 7f6ff6bce640 1 -- 172.21.15.32:0/3119483074 shutdown_connections 2024-04-24T23:06:34.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:34.932+0000 7f6ff6bce640 1 -- 172.21.15.32:0/3119483074 wait complete. 2024-04-24T23:06:34.945 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":378,"stamp":"2024-04-24T23:06:33.232206+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":96,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":7884,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1554615,"statfs":{"total":1600000000,"available":1591925868,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000921"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.950189+0000","last_change":"2024-04-24T23:06:31.949729+0000","last_active":"2024-04-24T23:06:31.950189+0000","last_peered":"2024-04-24T23:06:31.950189+0000","last_clean":"2024-04-24T23:06:31.950189+0000","last_became_active":"2024-04-24T23:06:31.949221+0000","last_became_peered":"2024-04-24T23:06:31.949221+0000","last_unstale":"2024-04-24T23:06:31.950189+0000","last_undegraded":"2024-04-24T23:06:31.950189+0000","last_fullsized":"2024-04-24T23:06:31.950189+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:00:33.666522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.950377+0000","last_change":"2024-04-24T23:06:31.950265+0000","last_active":"2024-04-24T23:06:31.950377+0000","last_peered":"2024-04-24T23:06:31.950377+0000","last_clean":"2024-04-24T23:06:31.950377+0000","last_became_active":"2024-04-24T23:06:31.948691+0000","last_became_peered":"2024-04-24T23:06:31.948691+0000","last_unstale":"2024-04-24T23:06:31.950377+0000","last_undegraded":"2024-04-24T23:06:31.950377+0000","last_fullsized":"2024-04-24T23:06:31.950377+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:48:10.318356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.950192+0000","last_change":"2024-04-24T23:06:31.949468+0000","last_active":"2024-04-24T23:06:31.950192+0000","last_peered":"2024-04-24T23:06:31.950192+0000","last_clean":"2024-04-24T23:06:31.950192+0000","last_became_active":"2024-04-24T23:06:31.949152+0000","last_became_peered":"2024-04-24T23:06:31.949152+0000","last_unstale":"2024-04-24T23:06:31.950192+0000","last_undegraded":"2024-04-24T23:06:31.950192+0000","last_fullsized":"2024-04-24T23:06:31.950192+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:25.089775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.949366+0000","last_change":"2024-04-24T23:06:31.949242+0000","last_active":"2024-04-24T23:06:31.949366+0000","last_peered":"2024-04-24T23:06:31.949366+0000","last_clean":"2024-04-24T23:06:31.949366+0000","last_became_active":"2024-04-24T23:06:31.949002+0000","last_became_peered":"2024-04-24T23:06:31.949002+0000","last_unstale":"2024-04-24T23:06:31.949366+0000","last_undegraded":"2024-04-24T23:06:31.949366+0000","last_fullsized":"2024-04-24T23:06:31.949366+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:29:58.088447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.952340+0000","last_change":"2024-04-24T23:06:31.951849+0000","last_active":"2024-04-24T23:06:31.952340+0000","last_peered":"2024-04-24T23:06:31.952340+0000","last_clean":"2024-04-24T23:06:31.952340+0000","last_became_active":"2024-04-24T23:06:31.951398+0000","last_became_peered":"2024-04-24T23:06:31.951398+0000","last_unstale":"2024-04-24T23:06:31.952340+0000","last_undegraded":"2024-04-24T23:06:31.952340+0000","last_fullsized":"2024-04-24T23:06:31.952340+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:10:23.370363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.953387+0000","last_change":"2024-04-24T23:06:31.953253+0000","last_active":"2024-04-24T23:06:31.953387+0000","last_peered":"2024-04-24T23:06:31.953387+0000","last_clean":"2024-04-24T23:06:31.953387+0000","last_became_active":"2024-04-24T23:06:31.952847+0000","last_became_peered":"2024-04-24T23:06:31.952847+0000","last_unstale":"2024-04-24T23:06:31.953387+0000","last_undegraded":"2024-04-24T23:06:31.953387+0000","last_fullsized":"2024-04-24T23:06:31.953387+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:05:24.712535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.953083+0000","last_change":"2024-04-24T23:06:31.952957+0000","last_active":"2024-04-24T23:06:31.953083+0000","last_peered":"2024-04-24T23:06:31.953083+0000","last_clean":"2024-04-24T23:06:31.953083+0000","last_became_active":"2024-04-24T23:06:31.952499+0000","last_became_peered":"2024-04-24T23:06:31.952499+0000","last_unstale":"2024-04-24T23:06:31.953083+0000","last_undegraded":"2024-04-24T23:06:31.953083+0000","last_fullsized":"2024-04-24T23:06:31.953083+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:23:43.601408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.951570+0000","last_change":"2024-04-24T23:06:31.951259+0000","last_active":"2024-04-24T23:06:31.951570+0000","last_peered":"2024-04-24T23:06:31.951570+0000","last_clean":"2024-04-24T23:06:31.951570+0000","last_became_active":"2024-04-24T23:06:31.950019+0000","last_became_peered":"2024-04-24T23:06:31.950019+0000","last_unstale":"2024-04-24T23:06:31.951570+0000","last_undegraded":"2024-04-24T23:06:31.951570+0000","last_fullsized":"2024-04-24T23:06:31.951570+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:11:15.995269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.954474+0000","last_change":"2024-04-24T23:06:31.954317+0000","last_active":"2024-04-24T23:06:31.954474+0000","last_peered":"2024-04-24T23:06:31.954474+0000","last_clean":"2024-04-24T23:06:31.954474+0000","last_became_active":"2024-04-24T23:06:31.953759+0000","last_became_peered":"2024-04-24T23:06:31.953759+0000","last_unstale":"2024-04-24T23:06:31.954474+0000","last_undegraded":"2024-04-24T23:06:31.954474+0000","last_fullsized":"2024-04-24T23:06:31.954474+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:44:52.482584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.948776+0000","last_change":"2024-04-24T23:06:31.948657+0000","last_active":"2024-04-24T23:06:31.948776+0000","last_peered":"2024-04-24T23:06:31.948776+0000","last_clean":"2024-04-24T23:06:31.948776+0000","last_became_active":"2024-04-24T23:06:31.948408+0000","last_became_peered":"2024-04-24T23:06:31.948408+0000","last_unstale":"2024-04-24T23:06:31.948776+0000","last_undegraded":"2024-04-24T23:06:31.948776+0000","last_fullsized":"2024-04-24T23:06:31.948776+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:48:04.449827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.941096+0000","last_change":"2024-04-24T23:06:31.940951+0000","last_active":"2024-04-24T23:06:31.941096+0000","last_peered":"2024-04-24T23:06:31.941096+0000","last_clean":"2024-04-24T23:06:31.941096+0000","last_became_active":"2024-04-24T23:06:31.940650+0000","last_became_peered":"2024-04-24T23:06:31.940650+0000","last_unstale":"2024-04-24T23:06:31.941096+0000","last_undegraded":"2024-04-24T23:06:31.941096+0000","last_fullsized":"2024-04-24T23:06:31.941096+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:06:19.925551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.952541+0000","last_change":"2024-04-24T23:06:31.951451+0000","last_active":"2024-04-24T23:06:31.952541+0000","last_peered":"2024-04-24T23:06:31.952541+0000","last_clean":"2024-04-24T23:06:31.952541+0000","last_became_active":"2024-04-24T23:06:31.951227+0000","last_became_peered":"2024-04-24T23:06:31.951227+0000","last_unstale":"2024-04-24T23:06:31.952541+0000","last_undegraded":"2024-04-24T23:06:31.952541+0000","last_fullsized":"2024-04-24T23:06:31.952541+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:50:42.991995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.954912+0000","last_change":"2024-04-24T23:06:31.954828+0000","last_active":"2024-04-24T23:06:31.954912+0000","last_peered":"2024-04-24T23:06:31.954912+0000","last_clean":"2024-04-24T23:06:31.954912+0000","last_became_active":"2024-04-24T23:06:31.954662+0000","last_became_peered":"2024-04-24T23:06:31.954662+0000","last_unstale":"2024-04-24T23:06:31.954912+0000","last_undegraded":"2024-04-24T23:06:31.954912+0000","last_fullsized":"2024-04-24T23:06:31.954912+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:13:35.661141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.951672+0000","last_change":"2024-04-24T23:06:31.951563+0000","last_active":"2024-04-24T23:06:31.951672+0000","last_peered":"2024-04-24T23:06:31.951672+0000","last_clean":"2024-04-24T23:06:31.951672+0000","last_became_active":"2024-04-24T23:06:31.951181+0000","last_became_peered":"2024-04-24T23:06:31.951181+0000","last_unstale":"2024-04-24T23:06:31.951672+0000","last_undegraded":"2024-04-24T23:06:31.951672+0000","last_fullsized":"2024-04-24T23:06:31.951672+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:38:39.800063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.946562+0000","last_change":"2024-04-24T23:06:31.946438+0000","last_active":"2024-04-24T23:06:31.946562+0000","last_peered":"2024-04-24T23:06:31.946562+0000","last_clean":"2024-04-24T23:06:31.946562+0000","last_became_active":"2024-04-24T23:06:31.945718+0000","last_became_peered":"2024-04-24T23:06:31.945718+0000","last_unstale":"2024-04-24T23:06:31.946562+0000","last_undegraded":"2024-04-24T23:06:31.946562+0000","last_fullsized":"2024-04-24T23:06:31.946562+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:20:42.499371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.940313+0000","last_change":"2024-04-24T23:06:31.940065+0000","last_active":"2024-04-24T23:06:31.940313+0000","last_peered":"2024-04-24T23:06:31.940313+0000","last_clean":"2024-04-24T23:06:31.940313+0000","last_became_active":"2024-04-24T23:06:31.939773+0000","last_became_peered":"2024-04-24T23:06:31.939773+0000","last_unstale":"2024-04-24T23:06:31.940313+0000","last_undegraded":"2024-04-24T23:06:31.940313+0000","last_fullsized":"2024-04-24T23:06:31.940313+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:00.690415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.937892+0000","last_change":"2024-04-24T23:06:31.937777+0000","last_active":"2024-04-24T23:06:31.937892+0000","last_peered":"2024-04-24T23:06:31.937892+0000","last_clean":"2024-04-24T23:06:31.937892+0000","last_became_active":"2024-04-24T23:06:31.937541+0000","last_became_peered":"2024-04-24T23:06:31.937541+0000","last_unstale":"2024-04-24T23:06:31.937892+0000","last_undegraded":"2024-04-24T23:06:31.937892+0000","last_fullsized":"2024-04-24T23:06:31.937892+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:37:36.370336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.951445+0000","last_change":"2024-04-24T23:06:31.951318+0000","last_active":"2024-04-24T23:06:31.951445+0000","last_peered":"2024-04-24T23:06:31.951445+0000","last_clean":"2024-04-24T23:06:31.951445+0000","last_became_active":"2024-04-24T23:06:31.950991+0000","last_became_peered":"2024-04-24T23:06:31.950991+0000","last_unstale":"2024-04-24T23:06:31.951445+0000","last_undegraded":"2024-04-24T23:06:31.951445+0000","last_fullsized":"2024-04-24T23:06:31.951445+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:03:23.326244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.947332+0000","last_change":"2024-04-24T23:06:31.947181+0000","last_active":"2024-04-24T23:06:31.947332+0000","last_peered":"2024-04-24T23:06:31.947332+0000","last_clean":"2024-04-24T23:06:31.947332+0000","last_became_active":"2024-04-24T23:06:31.946631+0000","last_became_peered":"2024-04-24T23:06:31.946631+0000","last_unstale":"2024-04-24T23:06:31.947332+0000","last_undegraded":"2024-04-24T23:06:31.947332+0000","last_fullsized":"2024-04-24T23:06:31.947332+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:17:24.084994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.946186+0000","last_change":"2024-04-24T23:06:31.945982+0000","last_active":"2024-04-24T23:06:31.946186+0000","last_peered":"2024-04-24T23:06:31.946186+0000","last_clean":"2024-04-24T23:06:31.946186+0000","last_became_active":"2024-04-24T23:06:31.944899+0000","last_became_peered":"2024-04-24T23:06:31.944899+0000","last_unstale":"2024-04-24T23:06:31.946186+0000","last_undegraded":"2024-04-24T23:06:31.946186+0000","last_fullsized":"2024-04-24T23:06:31.946186+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:40:17.005051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.951573+0000","last_change":"2024-04-24T23:06:31.950843+0000","last_active":"2024-04-24T23:06:31.951573+0000","last_peered":"2024-04-24T23:06:31.951573+0000","last_clean":"2024-04-24T23:06:31.951573+0000","last_became_active":"2024-04-24T23:06:31.949890+0000","last_became_peered":"2024-04-24T23:06:31.949890+0000","last_unstale":"2024-04-24T23:06:31.951573+0000","last_undegraded":"2024-04-24T23:06:31.951573+0000","last_fullsized":"2024-04-24T23:06:31.951573+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:55:20.855052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.949042+0000","last_change":"2024-04-24T23:06:31.948931+0000","last_active":"2024-04-24T23:06:31.949042+0000","last_peered":"2024-04-24T23:06:31.949042+0000","last_clean":"2024-04-24T23:06:31.949042+0000","last_became_active":"2024-04-24T23:06:31.947254+0000","last_became_peered":"2024-04-24T23:06:31.947254+0000","last_unstale":"2024-04-24T23:06:31.949042+0000","last_undegraded":"2024-04-24T23:06:31.949042+0000","last_fullsized":"2024-04-24T23:06:31.949042+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:48:02.259154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.956578+0000","last_change":"2024-04-24T23:06:31.955813+0000","last_active":"2024-04-24T23:06:31.956578+0000","last_peered":"2024-04-24T23:06:31.956578+0000","last_clean":"2024-04-24T23:06:31.956578+0000","last_became_active":"2024-04-24T23:06:31.954874+0000","last_became_peered":"2024-04-24T23:06:31.954874+0000","last_unstale":"2024-04-24T23:06:31.956578+0000","last_undegraded":"2024-04-24T23:06:31.956578+0000","last_fullsized":"2024-04-24T23:06:31.956578+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:02:26.392545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.952664+0000","last_change":"2024-04-24T23:06:31.952538+0000","last_active":"2024-04-24T23:06:31.952664+0000","last_peered":"2024-04-24T23:06:31.952664+0000","last_clean":"2024-04-24T23:06:31.952664+0000","last_became_active":"2024-04-24T23:06:31.952299+0000","last_became_peered":"2024-04-24T23:06:31.952299+0000","last_unstale":"2024-04-24T23:06:31.952664+0000","last_undegraded":"2024-04-24T23:06:31.952664+0000","last_fullsized":"2024-04-24T23:06:31.952664+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:10:26.871816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.944164+0000","last_change":"2024-04-24T23:06:31.944040+0000","last_active":"2024-04-24T23:06:31.944164+0000","last_peered":"2024-04-24T23:06:31.944164+0000","last_clean":"2024-04-24T23:06:31.944164+0000","last_became_active":"2024-04-24T23:06:31.943258+0000","last_became_peered":"2024-04-24T23:06:31.943258+0000","last_unstale":"2024-04-24T23:06:31.944164+0000","last_undegraded":"2024-04-24T23:06:31.944164+0000","last_fullsized":"2024-04-24T23:06:31.944164+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:54:01.279196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.950153+0000","last_change":"2024-04-24T23:06:31.949490+0000","last_active":"2024-04-24T23:06:31.950153+0000","last_peered":"2024-04-24T23:06:31.950153+0000","last_clean":"2024-04-24T23:06:31.950153+0000","last_became_active":"2024-04-24T23:06:31.947623+0000","last_became_peered":"2024-04-24T23:06:31.947623+0000","last_unstale":"2024-04-24T23:06:31.950153+0000","last_undegraded":"2024-04-24T23:06:31.950153+0000","last_fullsized":"2024-04-24T23:06:31.950153+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:34:27.906820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.945972+0000","last_change":"2024-04-24T23:06:31.945838+0000","last_active":"2024-04-24T23:06:31.945972+0000","last_peered":"2024-04-24T23:06:31.945972+0000","last_clean":"2024-04-24T23:06:31.945972+0000","last_became_active":"2024-04-24T23:06:31.945599+0000","last_became_peered":"2024-04-24T23:06:31.945599+0000","last_unstale":"2024-04-24T23:06:31.945972+0000","last_undegraded":"2024-04-24T23:06:31.945972+0000","last_fullsized":"2024-04-24T23:06:31.945972+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:54:30.678507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.954524+0000","last_change":"2024-04-24T23:06:31.954417+0000","last_active":"2024-04-24T23:06:31.954524+0000","last_peered":"2024-04-24T23:06:31.954524+0000","last_clean":"2024-04-24T23:06:31.954524+0000","last_became_active":"2024-04-24T23:06:31.952867+0000","last_became_peered":"2024-04-24T23:06:31.952867+0000","last_unstale":"2024-04-24T23:06:31.954524+0000","last_undegraded":"2024-04-24T23:06:31.954524+0000","last_fullsized":"2024-04-24T23:06:31.954524+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:37:04.031161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.946282+0000","last_change":"2024-04-24T23:06:31.945326+0000","last_active":"2024-04-24T23:06:31.946282+0000","last_peered":"2024-04-24T23:06:31.946282+0000","last_clean":"2024-04-24T23:06:31.946282+0000","last_became_active":"2024-04-24T23:06:31.945094+0000","last_became_peered":"2024-04-24T23:06:31.945094+0000","last_unstale":"2024-04-24T23:06:31.946282+0000","last_undegraded":"2024-04-24T23:06:31.946282+0000","last_fullsized":"2024-04-24T23:06:31.946282+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:56:33.583507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.950173+0000","last_change":"2024-04-24T23:06:31.949666+0000","last_active":"2024-04-24T23:06:31.950173+0000","last_peered":"2024-04-24T23:06:31.950173+0000","last_clean":"2024-04-24T23:06:31.950173+0000","last_became_active":"2024-04-24T23:06:31.949394+0000","last_became_peered":"2024-04-24T23:06:31.949394+0000","last_unstale":"2024-04-24T23:06:31.950173+0000","last_undegraded":"2024-04-24T23:06:31.950173+0000","last_fullsized":"2024-04-24T23:06:31.950173+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:51:33.586275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.948849+0000","last_change":"2024-04-24T23:06:31.947130+0000","last_active":"2024-04-24T23:06:31.948849+0000","last_peered":"2024-04-24T23:06:31.948849+0000","last_clean":"2024-04-24T23:06:31.948849+0000","last_became_active":"2024-04-24T23:06:31.946785+0000","last_became_peered":"2024-04-24T23:06:31.946785+0000","last_unstale":"2024-04-24T23:06:31.948849+0000","last_undegraded":"2024-04-24T23:06:31.948849+0000","last_fullsized":"2024-04-24T23:06:31.948849+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:10:26.367289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.950478+0000","last_change":"2024-04-24T23:06:31.950355+0000","last_active":"2024-04-24T23:06:31.950478+0000","last_peered":"2024-04-24T23:06:31.950478+0000","last_clean":"2024-04-24T23:06:31.950478+0000","last_became_active":"2024-04-24T23:06:31.950121+0000","last_became_peered":"2024-04-24T23:06:31.950121+0000","last_unstale":"2024-04-24T23:06:31.950478+0000","last_undegraded":"2024-04-24T23:06:31.950478+0000","last_fullsized":"2024-04-24T23:06:31.950478+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:58:46.701785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.945255+0000","last_change":"2024-04-24T23:06:31.944917+0000","last_active":"2024-04-24T23:06:31.945255+0000","last_peered":"2024-04-24T23:06:31.945255+0000","last_clean":"2024-04-24T23:06:31.945255+0000","last_became_active":"2024-04-24T23:06:31.944641+0000","last_became_peered":"2024-04-24T23:06:31.944641+0000","last_unstale":"2024-04-24T23:06:31.945255+0000","last_undegraded":"2024-04-24T23:06:31.945255+0000","last_fullsized":"2024-04-24T23:06:31.945255+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:57:50.603164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1f","version":"155'4","reported_seq":16,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.963364+0000","last_change":"2024-04-24T23:06:30.930128+0000","last_active":"2024-04-24T23:06:31.963364+0000","last_peered":"2024-04-24T23:06:31.963364+0000","last_clean":"2024-04-24T23:06:31.963364+0000","last_became_active":"2024-04-24T23:06:30.929383+0000","last_became_peered":"2024-04-24T23:06:30.929383+0000","last_unstale":"2024-04-24T23:06:31.963364+0000","last_undegraded":"2024-04-24T23:06:31.963364+0000","last_fullsized":"2024-04-24T23:06:31.963364+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:53:36.932051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.937907+0000","last_change":"2024-04-24T23:06:31.937822+0000","last_active":"2024-04-24T23:06:31.937907+0000","last_peered":"2024-04-24T23:06:31.937907+0000","last_clean":"2024-04-24T23:06:31.937907+0000","last_became_active":"2024-04-24T23:06:31.937596+0000","last_became_peered":"2024-04-24T23:06:31.937596+0000","last_unstale":"2024-04-24T23:06:31.937907+0000","last_undegraded":"2024-04-24T23:06:31.937907+0000","last_fullsized":"2024-04-24T23:06:31.937907+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:29:46.851064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1e","version":"155'1","reported_seq":13,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.965589+0000","last_change":"2024-04-24T23:06:30.935179+0000","last_active":"2024-04-24T23:06:31.965589+0000","last_peered":"2024-04-24T23:06:31.965589+0000","last_clean":"2024-04-24T23:06:31.965589+0000","last_became_active":"2024-04-24T23:06:30.934892+0000","last_became_peered":"2024-04-24T23:06:30.934892+0000","last_unstale":"2024-04-24T23:06:31.965589+0000","last_undegraded":"2024-04-24T23:06:31.965589+0000","last_fullsized":"2024-04-24T23:06:31.965589+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:36.115975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.945417+0000","last_change":"2024-04-24T23:06:31.945287+0000","last_active":"2024-04-24T23:06:31.945417+0000","last_peered":"2024-04-24T23:06:31.945417+0000","last_clean":"2024-04-24T23:06:31.945417+0000","last_became_active":"2024-04-24T23:06:31.945044+0000","last_became_peered":"2024-04-24T23:06:31.945044+0000","last_unstale":"2024-04-24T23:06:31.945417+0000","last_undegraded":"2024-04-24T23:06:31.945417+0000","last_fullsized":"2024-04-24T23:06:31.945417+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:27.707116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1d","version":"155'2","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.963188+0000","last_change":"2024-04-24T23:06:30.933020+0000","last_active":"2024-04-24T23:06:31.963188+0000","last_peered":"2024-04-24T23:06:31.963188+0000","last_clean":"2024-04-24T23:06:31.963188+0000","last_became_active":"2024-04-24T23:06:30.932290+0000","last_became_peered":"2024-04-24T23:06:30.932290+0000","last_unstale":"2024-04-24T23:06:31.963188+0000","last_undegraded":"2024-04-24T23:06:31.963188+0000","last_fullsized":"2024-04-24T23:06:31.963188+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:09.649102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.945330+0000","last_change":"2024-04-24T23:06:31.945178+0000","last_active":"2024-04-24T23:06:31.945330+0000","last_peered":"2024-04-24T23:06:31.945330+0000","last_clean":"2024-04-24T23:06:31.945330+0000","last_became_active":"2024-04-24T23:06:31.944927+0000","last_became_peered":"2024-04-24T23:06:31.944927+0000","last_unstale":"2024-04-24T23:06:31.945330+0000","last_undegraded":"2024-04-24T23:06:31.945330+0000","last_fullsized":"2024-04-24T23:06:31.945330+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:45:12.542275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1c","version":"155'2","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.962666+0000","last_change":"2024-04-24T23:06:30.932975+0000","last_active":"2024-04-24T23:06:31.962666+0000","last_peered":"2024-04-24T23:06:31.962666+0000","last_clean":"2024-04-24T23:06:31.962666+0000","last_became_active":"2024-04-24T23:06:30.932724+0000","last_became_peered":"2024-04-24T23:06:30.932724+0000","last_unstale":"2024-04-24T23:06:31.962666+0000","last_undegraded":"2024-04-24T23:06:31.962666+0000","last_fullsized":"2024-04-24T23:06:31.962666+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:07:33.232218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.954026+0000","last_change":"2024-04-24T23:06:31.953879+0000","last_active":"2024-04-24T23:06:31.954026+0000","last_peered":"2024-04-24T23:06:31.954026+0000","last_clean":"2024-04-24T23:06:31.954026+0000","last_became_active":"2024-04-24T23:06:31.953612+0000","last_became_peered":"2024-04-24T23:06:31.953612+0000","last_unstale":"2024-04-24T23:06:31.954026+0000","last_undegraded":"2024-04-24T23:06:31.954026+0000","last_fullsized":"2024-04-24T23:06:31.954026+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:29:32.344819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1b","version":"155'1","reported_seq":13,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.961789+0000","last_change":"2024-04-24T23:06:30.938313+0000","last_active":"2024-04-24T23:06:31.961789+0000","last_peered":"2024-04-24T23:06:31.961789+0000","last_clean":"2024-04-24T23:06:31.961789+0000","last_became_active":"2024-04-24T23:06:30.937647+0000","last_became_peered":"2024-04-24T23:06:30.937647+0000","last_unstale":"2024-04-24T23:06:31.961789+0000","last_undegraded":"2024-04-24T23:06:31.961789+0000","last_fullsized":"2024-04-24T23:06:31.961789+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:07:57.792089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.951530+0000","last_change":"2024-04-24T23:06:31.949219+0000","last_active":"2024-04-24T23:06:31.951530+0000","last_peered":"2024-04-24T23:06:31.951530+0000","last_clean":"2024-04-24T23:06:31.951530+0000","last_became_active":"2024-04-24T23:06:31.948075+0000","last_became_peered":"2024-04-24T23:06:31.948075+0000","last_unstale":"2024-04-24T23:06:31.951530+0000","last_undegraded":"2024-04-24T23:06:31.951530+0000","last_fullsized":"2024-04-24T23:06:31.951530+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:25:21.049448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"155'1","reported_seq":13,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.960126+0000","last_change":"2024-04-24T23:06:30.931776+0000","last_active":"2024-04-24T23:06:31.960126+0000","last_peered":"2024-04-24T23:06:31.960126+0000","last_clean":"2024-04-24T23:06:31.960126+0000","last_became_active":"2024-04-24T23:06:30.931553+0000","last_became_peered":"2024-04-24T23:06:30.931553+0000","last_unstale":"2024-04-24T23:06:31.960126+0000","last_undegraded":"2024-04-24T23:06:31.960126+0000","last_fullsized":"2024-04-24T23:06:31.960126+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:10:37.924519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.946402+0000","last_change":"2024-04-24T23:06:31.946274+0000","last_active":"2024-04-24T23:06:31.946402+0000","last_peered":"2024-04-24T23:06:31.946402+0000","last_clean":"2024-04-24T23:06:31.946402+0000","last_became_active":"2024-04-24T23:06:31.945966+0000","last_became_peered":"2024-04-24T23:06:31.945966+0000","last_unstale":"2024-04-24T23:06:31.946402+0000","last_undegraded":"2024-04-24T23:06:31.946402+0000","last_fullsized":"2024-04-24T23:06:31.946402+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:48.803786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.936839+0000","last_change":"2024-04-24T23:06:30.926908+0000","last_active":"2024-04-24T23:06:31.936839+0000","last_peered":"2024-04-24T23:06:31.936839+0000","last_clean":"2024-04-24T23:06:31.936839+0000","last_became_active":"2024-04-24T23:06:30.925544+0000","last_became_peered":"2024-04-24T23:06:30.925544+0000","last_unstale":"2024-04-24T23:06:31.936839+0000","last_undegraded":"2024-04-24T23:06:31.936839+0000","last_fullsized":"2024-04-24T23:06:31.936839+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:49:25.469065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.943031+0000","last_change":"2024-04-24T23:06:31.942900+0000","last_active":"2024-04-24T23:06:31.943031+0000","last_peered":"2024-04-24T23:06:31.943031+0000","last_clean":"2024-04-24T23:06:31.943031+0000","last_became_active":"2024-04-24T23:06:31.942661+0000","last_became_peered":"2024-04-24T23:06:31.942661+0000","last_unstale":"2024-04-24T23:06:31.943031+0000","last_undegraded":"2024-04-24T23:06:31.943031+0000","last_fullsized":"2024-04-24T23:06:31.943031+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:53.045519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.944091+0000","last_change":"2024-04-24T23:06:30.935423+0000","last_active":"2024-04-24T23:06:31.944091+0000","last_peered":"2024-04-24T23:06:31.944091+0000","last_clean":"2024-04-24T23:06:31.944091+0000","last_became_active":"2024-04-24T23:06:30.935168+0000","last_became_peered":"2024-04-24T23:06:30.935168+0000","last_unstale":"2024-04-24T23:06:31.944091+0000","last_undegraded":"2024-04-24T23:06:31.944091+0000","last_fullsized":"2024-04-24T23:06:31.944091+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:50:31.858373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.949378+0000","last_change":"2024-04-24T23:06:31.949115+0000","last_active":"2024-04-24T23:06:31.949378+0000","last_peered":"2024-04-24T23:06:31.949378+0000","last_clean":"2024-04-24T23:06:31.949378+0000","last_became_active":"2024-04-24T23:06:31.947551+0000","last_became_peered":"2024-04-24T23:06:31.947551+0000","last_unstale":"2024-04-24T23:06:31.949378+0000","last_undegraded":"2024-04-24T23:06:31.949378+0000","last_fullsized":"2024-04-24T23:06:31.949378+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:25.397488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.944077+0000","last_change":"2024-04-24T23:06:30.933074+0000","last_active":"2024-04-24T23:06:31.944077+0000","last_peered":"2024-04-24T23:06:31.944077+0000","last_clean":"2024-04-24T23:06:31.944077+0000","last_became_active":"2024-04-24T23:06:30.932772+0000","last_became_peered":"2024-04-24T23:06:30.932772+0000","last_unstale":"2024-04-24T23:06:31.944077+0000","last_undegraded":"2024-04-24T23:06:31.944077+0000","last_fullsized":"2024-04-24T23:06:31.944077+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:18:01.691756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.950021+0000","last_change":"2024-04-24T23:06:31.949759+0000","last_active":"2024-04-24T23:06:31.950021+0000","last_peered":"2024-04-24T23:06:31.950021+0000","last_clean":"2024-04-24T23:06:31.950021+0000","last_became_active":"2024-04-24T23:06:31.949521+0000","last_became_peered":"2024-04-24T23:06:31.949521+0000","last_unstale":"2024-04-24T23:06:31.950021+0000","last_undegraded":"2024-04-24T23:06:31.950021+0000","last_fullsized":"2024-04-24T23:06:31.950021+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:55.349838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.946387+0000","last_change":"2024-04-24T23:06:30.931620+0000","last_active":"2024-04-24T23:06:31.946387+0000","last_peered":"2024-04-24T23:06:31.946387+0000","last_clean":"2024-04-24T23:06:31.946387+0000","last_became_active":"2024-04-24T23:06:30.931343+0000","last_became_peered":"2024-04-24T23:06:30.931343+0000","last_unstale":"2024-04-24T23:06:31.946387+0000","last_undegraded":"2024-04-24T23:06:31.946387+0000","last_fullsized":"2024-04-24T23:06:31.946387+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:02:57.038500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.951547+0000","last_change":"2024-04-24T23:06:31.948344+0000","last_active":"2024-04-24T23:06:31.951547+0000","last_peered":"2024-04-24T23:06:31.951547+0000","last_clean":"2024-04-24T23:06:31.951547+0000","last_became_active":"2024-04-24T23:06:31.948053+0000","last_became_peered":"2024-04-24T23:06:31.948053+0000","last_unstale":"2024-04-24T23:06:31.951547+0000","last_undegraded":"2024-04-24T23:06:31.951547+0000","last_fullsized":"2024-04-24T23:06:31.951547+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:04:07.117896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.15","version":"155'3","reported_seq":15,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.963705+0000","last_change":"2024-04-24T23:06:30.929875+0000","last_active":"2024-04-24T23:06:31.963705+0000","last_peered":"2024-04-24T23:06:31.963705+0000","last_clean":"2024-04-24T23:06:31.963705+0000","last_became_active":"2024-04-24T23:06:30.929051+0000","last_became_peered":"2024-04-24T23:06:30.929051+0000","last_unstale":"2024-04-24T23:06:31.963705+0000","last_undegraded":"2024-04-24T23:06:31.963705+0000","last_fullsized":"2024-04-24T23:06:31.963705+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:17:00.281627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.954238+0000","last_change":"2024-04-24T23:06:31.954113+0000","last_active":"2024-04-24T23:06:31.954238+0000","last_peered":"2024-04-24T23:06:31.954238+0000","last_clean":"2024-04-24T23:06:31.954238+0000","last_became_active":"2024-04-24T23:06:31.952536+0000","last_became_peered":"2024-04-24T23:06:31.952536+0000","last_unstale":"2024-04-24T23:06:31.954238+0000","last_undegraded":"2024-04-24T23:06:31.954238+0000","last_fullsized":"2024-04-24T23:06:31.954238+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:50:22.152102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.14","version":"155'6","reported_seq":18,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.966311+0000","last_change":"2024-04-24T23:06:30.935668+0000","last_active":"2024-04-24T23:06:31.966311+0000","last_peered":"2024-04-24T23:06:31.966311+0000","last_clean":"2024-04-24T23:06:31.966311+0000","last_became_active":"2024-04-24T23:06:30.935438+0000","last_became_peered":"2024-04-24T23:06:30.935438+0000","last_unstale":"2024-04-24T23:06:31.966311+0000","last_undegraded":"2024-04-24T23:06:31.966311+0000","last_fullsized":"2024-04-24T23:06:31.966311+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:27:55.856001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.947274+0000","last_change":"2024-04-24T23:06:31.946314+0000","last_active":"2024-04-24T23:06:31.947274+0000","last_peered":"2024-04-24T23:06:31.947274+0000","last_clean":"2024-04-24T23:06:31.947274+0000","last_became_active":"2024-04-24T23:06:31.946000+0000","last_became_peered":"2024-04-24T23:06:31.946000+0000","last_unstale":"2024-04-24T23:06:31.947274+0000","last_undegraded":"2024-04-24T23:06:31.947274+0000","last_fullsized":"2024-04-24T23:06:31.947274+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:44:16.658014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.938323+0000","last_change":"2024-04-24T23:06:30.927557+0000","last_active":"2024-04-24T23:06:31.938323+0000","last_peered":"2024-04-24T23:06:31.938323+0000","last_clean":"2024-04-24T23:06:31.938323+0000","last_became_active":"2024-04-24T23:06:30.927310+0000","last_became_peered":"2024-04-24T23:06:30.927310+0000","last_unstale":"2024-04-24T23:06:31.938323+0000","last_undegraded":"2024-04-24T23:06:31.938323+0000","last_fullsized":"2024-04-24T23:06:31.938323+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:01.699089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.955727+0000","last_change":"2024-04-24T23:06:31.955645+0000","last_active":"2024-04-24T23:06:31.955727+0000","last_peered":"2024-04-24T23:06:31.955727+0000","last_clean":"2024-04-24T23:06:31.955727+0000","last_became_active":"2024-04-24T23:06:31.955426+0000","last_became_peered":"2024-04-24T23:06:31.955426+0000","last_unstale":"2024-04-24T23:06:31.955727+0000","last_undegraded":"2024-04-24T23:06:31.955727+0000","last_fullsized":"2024-04-24T23:06:31.955727+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:32.529668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.933746+0000","last_change":"2024-04-24T23:06:30.933053+0000","last_active":"2024-04-24T23:06:31.933746+0000","last_peered":"2024-04-24T23:06:31.933746+0000","last_clean":"2024-04-24T23:06:31.933746+0000","last_became_active":"2024-04-24T23:06:30.932798+0000","last_became_peered":"2024-04-24T23:06:30.932798+0000","last_unstale":"2024-04-24T23:06:31.933746+0000","last_undegraded":"2024-04-24T23:06:31.933746+0000","last_fullsized":"2024-04-24T23:06:31.933746+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:33:56.873969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.951513+0000","last_change":"2024-04-24T23:06:31.948644+0000","last_active":"2024-04-24T23:06:31.951513+0000","last_peered":"2024-04-24T23:06:31.951513+0000","last_clean":"2024-04-24T23:06:31.951513+0000","last_became_active":"2024-04-24T23:06:31.947352+0000","last_became_peered":"2024-04-24T23:06:31.947352+0000","last_unstale":"2024-04-24T23:06:31.951513+0000","last_undegraded":"2024-04-24T23:06:31.951513+0000","last_fullsized":"2024-04-24T23:06:31.951513+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:36:45.428256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.10","version":"155'8","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.965701+0000","last_change":"2024-04-24T23:06:30.934955+0000","last_active":"2024-04-24T23:06:31.965701+0000","last_peered":"2024-04-24T23:06:31.965701+0000","last_clean":"2024-04-24T23:06:31.965701+0000","last_became_active":"2024-04-24T23:06:30.934676+0000","last_became_peered":"2024-04-24T23:06:30.934676+0000","last_unstale":"2024-04-24T23:06:31.965701+0000","last_undegraded":"2024-04-24T23:06:31.965701+0000","last_fullsized":"2024-04-24T23:06:31.965701+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:54.015309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.948106+0000","last_change":"2024-04-24T23:06:31.946127+0000","last_active":"2024-04-24T23:06:31.948106+0000","last_peered":"2024-04-24T23:06:31.948106+0000","last_clean":"2024-04-24T23:06:31.948106+0000","last_became_active":"2024-04-24T23:06:31.945872+0000","last_became_peered":"2024-04-24T23:06:31.945872+0000","last_unstale":"2024-04-24T23:06:31.948106+0000","last_undegraded":"2024-04-24T23:06:31.948106+0000","last_fullsized":"2024-04-24T23:06:31.948106+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:59:47.230611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.f","version":"155'1","reported_seq":13,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.963852+0000","last_change":"2024-04-24T23:06:30.937833+0000","last_active":"2024-04-24T23:06:31.963852+0000","last_peered":"2024-04-24T23:06:31.963852+0000","last_clean":"2024-04-24T23:06:31.963852+0000","last_became_active":"2024-04-24T23:06:30.937440+0000","last_became_peered":"2024-04-24T23:06:30.937440+0000","last_unstale":"2024-04-24T23:06:31.963852+0000","last_undegraded":"2024-04-24T23:06:31.963852+0000","last_fullsized":"2024-04-24T23:06:31.963852+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:59.062468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.942414+0000","last_change":"2024-04-24T23:06:31.942215+0000","last_active":"2024-04-24T23:06:31.942414+0000","last_peered":"2024-04-24T23:06:31.942414+0000","last_clean":"2024-04-24T23:06:31.942414+0000","last_became_active":"2024-04-24T23:06:31.941952+0000","last_became_peered":"2024-04-24T23:06:31.941952+0000","last_unstale":"2024-04-24T23:06:31.942414+0000","last_undegraded":"2024-04-24T23:06:31.942414+0000","last_fullsized":"2024-04-24T23:06:31.942414+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:33:01.134266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"155'2","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.963508+0000","last_change":"2024-04-24T23:06:30.932587+0000","last_active":"2024-04-24T23:06:31.963508+0000","last_peered":"2024-04-24T23:06:31.963508+0000","last_clean":"2024-04-24T23:06:31.963508+0000","last_became_active":"2024-04-24T23:06:30.931956+0000","last_became_peered":"2024-04-24T23:06:30.931956+0000","last_unstale":"2024-04-24T23:06:31.963508+0000","last_undegraded":"2024-04-24T23:06:31.963508+0000","last_fullsized":"2024-04-24T23:06:31.963508+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:16:54.161845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.952811+0000","last_change":"2024-04-24T23:06:31.952633+0000","last_active":"2024-04-24T23:06:31.952811+0000","last_peered":"2024-04-24T23:06:31.952811+0000","last_clean":"2024-04-24T23:06:31.952811+0000","last_became_active":"2024-04-24T23:06:31.952328+0000","last_became_peered":"2024-04-24T23:06:31.952328+0000","last_unstale":"2024-04-24T23:06:31.952811+0000","last_undegraded":"2024-04-24T23:06:31.952811+0000","last_fullsized":"2024-04-24T23:06:31.952811+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:38:59.660215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"155'1","reported_seq":13,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.964484+0000","last_change":"2024-04-24T23:06:30.932688+0000","last_active":"2024-04-24T23:06:31.964484+0000","last_peered":"2024-04-24T23:06:31.964484+0000","last_clean":"2024-04-24T23:06:31.964484+0000","last_became_active":"2024-04-24T23:06:30.932045+0000","last_became_peered":"2024-04-24T23:06:30.932045+0000","last_unstale":"2024-04-24T23:06:31.964484+0000","last_undegraded":"2024-04-24T23:06:31.964484+0000","last_fullsized":"2024-04-24T23:06:31.964484+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:14.603813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.947253+0000","last_change":"2024-04-24T23:06:31.944546+0000","last_active":"2024-04-24T23:06:31.947253+0000","last_peered":"2024-04-24T23:06:31.947253+0000","last_clean":"2024-04-24T23:06:31.947253+0000","last_became_active":"2024-04-24T23:06:31.944296+0000","last_became_peered":"2024-04-24T23:06:31.944296+0000","last_unstale":"2024-04-24T23:06:31.947253+0000","last_undegraded":"2024-04-24T23:06:31.947253+0000","last_fullsized":"2024-04-24T23:06:31.947253+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:47:36.471230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.941735+0000","last_change":"2024-04-24T23:06:30.937951+0000","last_active":"2024-04-24T23:06:31.941735+0000","last_peered":"2024-04-24T23:06:31.941735+0000","last_clean":"2024-04-24T23:06:31.941735+0000","last_became_active":"2024-04-24T23:06:30.937694+0000","last_became_peered":"2024-04-24T23:06:30.937694+0000","last_unstale":"2024-04-24T23:06:31.941735+0000","last_undegraded":"2024-04-24T23:06:31.941735+0000","last_fullsized":"2024-04-24T23:06:31.941735+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:18:58.007323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.947453+0000","last_change":"2024-04-24T23:06:31.947359+0000","last_active":"2024-04-24T23:06:31.947453+0000","last_peered":"2024-04-24T23:06:31.947453+0000","last_clean":"2024-04-24T23:06:31.947453+0000","last_became_active":"2024-04-24T23:06:31.946957+0000","last_became_peered":"2024-04-24T23:06:31.946957+0000","last_unstale":"2024-04-24T23:06:31.947453+0000","last_undegraded":"2024-04-24T23:06:31.947453+0000","last_fullsized":"2024-04-24T23:06:31.947453+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:32:35.998113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"155'2","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.965067+0000","last_change":"2024-04-24T23:06:30.938258+0000","last_active":"2024-04-24T23:06:31.965067+0000","last_peered":"2024-04-24T23:06:31.965067+0000","last_clean":"2024-04-24T23:06:31.965067+0000","last_became_active":"2024-04-24T23:06:30.937310+0000","last_became_peered":"2024-04-24T23:06:30.937310+0000","last_unstale":"2024-04-24T23:06:31.965067+0000","last_undegraded":"2024-04-24T23:06:31.965067+0000","last_fullsized":"2024-04-24T23:06:31.965067+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:14:52.090177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.959807+0000","last_change":"2024-04-24T23:06:31.959041+0000","last_active":"2024-04-24T23:06:31.959807+0000","last_peered":"2024-04-24T23:06:31.959807+0000","last_clean":"2024-04-24T23:06:31.959807+0000","last_became_active":"2024-04-24T23:06:31.958763+0000","last_became_peered":"2024-04-24T23:06:31.958763+0000","last_unstale":"2024-04-24T23:06:31.959807+0000","last_undegraded":"2024-04-24T23:06:31.959807+0000","last_fullsized":"2024-04-24T23:06:31.959807+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:26:44.239233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.950410+0000","last_change":"2024-04-24T23:06:30.932619+0000","last_active":"2024-04-24T23:06:31.950410+0000","last_peered":"2024-04-24T23:06:31.950410+0000","last_clean":"2024-04-24T23:06:31.950410+0000","last_became_active":"2024-04-24T23:06:30.932391+0000","last_became_peered":"2024-04-24T23:06:30.932391+0000","last_unstale":"2024-04-24T23:06:31.950410+0000","last_undegraded":"2024-04-24T23:06:31.950410+0000","last_fullsized":"2024-04-24T23:06:31.950410+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:01:43.325055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.947296+0000","last_change":"2024-04-24T23:06:31.946711+0000","last_active":"2024-04-24T23:06:31.947296+0000","last_peered":"2024-04-24T23:06:31.947296+0000","last_clean":"2024-04-24T23:06:31.947296+0000","last_became_active":"2024-04-24T23:06:31.946473+0000","last_became_peered":"2024-04-24T23:06:31.946473+0000","last_unstale":"2024-04-24T23:06:31.947296+0000","last_undegraded":"2024-04-24T23:06:31.947296+0000","last_fullsized":"2024-04-24T23:06:31.947296+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:07:17.716644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.942596+0000","last_change":"2024-04-24T23:06:30.933492+0000","last_active":"2024-04-24T23:06:31.942596+0000","last_peered":"2024-04-24T23:06:31.942596+0000","last_clean":"2024-04-24T23:06:31.942596+0000","last_became_active":"2024-04-24T23:06:30.933064+0000","last_became_peered":"2024-04-24T23:06:30.933064+0000","last_unstale":"2024-04-24T23:06:31.942596+0000","last_undegraded":"2024-04-24T23:06:31.942596+0000","last_fullsized":"2024-04-24T23:06:31.942596+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:10:25.036860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.951492+0000","last_change":"2024-04-24T23:06:31.948427+0000","last_active":"2024-04-24T23:06:31.951492+0000","last_peered":"2024-04-24T23:06:31.951492+0000","last_clean":"2024-04-24T23:06:31.951492+0000","last_became_active":"2024-04-24T23:06:31.948013+0000","last_became_peered":"2024-04-24T23:06:31.948013+0000","last_unstale":"2024-04-24T23:06:31.951492+0000","last_undegraded":"2024-04-24T23:06:31.951492+0000","last_fullsized":"2024-04-24T23:06:31.951492+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:19:03.084992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.934213+0000","last_change":"2024-04-24T23:06:30.935259+0000","last_active":"2024-04-24T23:06:31.934213+0000","last_peered":"2024-04-24T23:06:31.934213+0000","last_clean":"2024-04-24T23:06:31.934213+0000","last_became_active":"2024-04-24T23:06:30.934988+0000","last_became_peered":"2024-04-24T23:06:30.934988+0000","last_unstale":"2024-04-24T23:06:31.934213+0000","last_undegraded":"2024-04-24T23:06:31.934213+0000","last_fullsized":"2024-04-24T23:06:31.934213+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:18:26.082861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.954120+0000","last_change":"2024-04-24T23:06:31.953108+0000","last_active":"2024-04-24T23:06:31.954120+0000","last_peered":"2024-04-24T23:06:31.954120+0000","last_clean":"2024-04-24T23:06:31.954120+0000","last_became_active":"2024-04-24T23:06:31.952200+0000","last_became_peered":"2024-04-24T23:06:31.952200+0000","last_unstale":"2024-04-24T23:06:31.954120+0000","last_undegraded":"2024-04-24T23:06:31.954120+0000","last_fullsized":"2024-04-24T23:06:31.954120+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:48:41.214953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.7","version":"155'2","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.963409+0000","last_change":"2024-04-24T23:06:30.933292+0000","last_active":"2024-04-24T23:06:31.963409+0000","last_peered":"2024-04-24T23:06:31.963409+0000","last_clean":"2024-04-24T23:06:31.963409+0000","last_became_active":"2024-04-24T23:06:30.933071+0000","last_became_peered":"2024-04-24T23:06:30.933071+0000","last_unstale":"2024-04-24T23:06:31.963409+0000","last_undegraded":"2024-04-24T23:06:31.963409+0000","last_fullsized":"2024-04-24T23:06:31.963409+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:25:58.333171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.954383+0000","last_change":"2024-04-24T23:06:31.954265+0000","last_active":"2024-04-24T23:06:31.954383+0000","last_peered":"2024-04-24T23:06:31.954383+0000","last_clean":"2024-04-24T23:06:31.954383+0000","last_became_active":"2024-04-24T23:06:31.953898+0000","last_became_peered":"2024-04-24T23:06:31.953898+0000","last_unstale":"2024-04-24T23:06:31.954383+0000","last_undegraded":"2024-04-24T23:06:31.954383+0000","last_fullsized":"2024-04-24T23:06:31.954383+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:24.237447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.940746+0000","last_change":"2024-04-24T23:06:30.936290+0000","last_active":"2024-04-24T23:06:31.940746+0000","last_peered":"2024-04-24T23:06:31.940746+0000","last_clean":"2024-04-24T23:06:31.940746+0000","last_became_active":"2024-04-24T23:06:30.936049+0000","last_became_peered":"2024-04-24T23:06:30.936049+0000","last_unstale":"2024-04-24T23:06:31.940746+0000","last_undegraded":"2024-04-24T23:06:31.940746+0000","last_fullsized":"2024-04-24T23:06:31.940746+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:51:40.516708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.950171+0000","last_change":"2024-04-24T23:06:31.949930+0000","last_active":"2024-04-24T23:06:31.950171+0000","last_peered":"2024-04-24T23:06:31.950171+0000","last_clean":"2024-04-24T23:06:31.950171+0000","last_became_active":"2024-04-24T23:06:31.949636+0000","last_became_peered":"2024-04-24T23:06:31.949636+0000","last_unstale":"2024-04-24T23:06:31.950171+0000","last_undegraded":"2024-04-24T23:06:31.950171+0000","last_fullsized":"2024-04-24T23:06:31.950171+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:21:54.964154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.950551+0000","last_change":"2024-04-24T23:06:30.935624+0000","last_active":"2024-04-24T23:06:31.950551+0000","last_peered":"2024-04-24T23:06:31.950551+0000","last_clean":"2024-04-24T23:06:31.950551+0000","last_became_active":"2024-04-24T23:06:30.935387+0000","last_became_peered":"2024-04-24T23:06:30.935387+0000","last_unstale":"2024-04-24T23:06:31.950551+0000","last_undegraded":"2024-04-24T23:06:31.950551+0000","last_fullsized":"2024-04-24T23:06:31.950551+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:02.529511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.954682+0000","last_change":"2024-04-24T23:06:31.954556+0000","last_active":"2024-04-24T23:06:31.954682+0000","last_peered":"2024-04-24T23:06:31.954682+0000","last_clean":"2024-04-24T23:06:31.954682+0000","last_became_active":"2024-04-24T23:06:31.954074+0000","last_became_peered":"2024-04-24T23:06:31.954074+0000","last_unstale":"2024-04-24T23:06:31.954682+0000","last_undegraded":"2024-04-24T23:06:31.954682+0000","last_fullsized":"2024-04-24T23:06:31.954682+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:45:24.545531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.942756+0000","last_change":"2024-04-24T23:06:30.936106+0000","last_active":"2024-04-24T23:06:31.942756+0000","last_peered":"2024-04-24T23:06:31.942756+0000","last_clean":"2024-04-24T23:06:31.942756+0000","last_became_active":"2024-04-24T23:06:30.935856+0000","last_became_peered":"2024-04-24T23:06:30.935856+0000","last_unstale":"2024-04-24T23:06:31.942756+0000","last_undegraded":"2024-04-24T23:06:31.942756+0000","last_fullsized":"2024-04-24T23:06:31.942756+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:27.344320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.951591+0000","last_change":"2024-04-24T23:06:31.950936+0000","last_active":"2024-04-24T23:06:31.951591+0000","last_peered":"2024-04-24T23:06:31.951591+0000","last_clean":"2024-04-24T23:06:31.951591+0000","last_became_active":"2024-04-24T23:06:31.950659+0000","last_became_peered":"2024-04-24T23:06:31.950659+0000","last_unstale":"2024-04-24T23:06:31.951591+0000","last_undegraded":"2024-04-24T23:06:31.951591+0000","last_fullsized":"2024-04-24T23:06:31.951591+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:19.609533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.938973+0000","last_change":"2024-04-24T23:06:30.934663+0000","last_active":"2024-04-24T23:06:31.938973+0000","last_peered":"2024-04-24T23:06:31.938973+0000","last_clean":"2024-04-24T23:06:31.938973+0000","last_became_active":"2024-04-24T23:06:30.934266+0000","last_became_peered":"2024-04-24T23:06:30.934266+0000","last_unstale":"2024-04-24T23:06:31.938973+0000","last_undegraded":"2024-04-24T23:06:31.938973+0000","last_fullsized":"2024-04-24T23:06:31.938973+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:17:40.071951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.948032+0000","last_change":"2024-04-24T23:06:31.947912+0000","last_active":"2024-04-24T23:06:31.948032+0000","last_peered":"2024-04-24T23:06:31.948032+0000","last_clean":"2024-04-24T23:06:31.948032+0000","last_became_active":"2024-04-24T23:06:31.947626+0000","last_became_peered":"2024-04-24T23:06:31.947626+0000","last_unstale":"2024-04-24T23:06:31.948032+0000","last_undegraded":"2024-04-24T23:06:31.948032+0000","last_fullsized":"2024-04-24T23:06:31.948032+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:57:10.290493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.944557+0000","last_change":"2024-04-24T23:06:30.930687+0000","last_active":"2024-04-24T23:06:31.944557+0000","last_peered":"2024-04-24T23:06:31.944557+0000","last_clean":"2024-04-24T23:06:31.944557+0000","last_became_active":"2024-04-24T23:06:30.930397+0000","last_became_peered":"2024-04-24T23:06:30.930397+0000","last_unstale":"2024-04-24T23:06:31.944557+0000","last_undegraded":"2024-04-24T23:06:31.944557+0000","last_fullsized":"2024-04-24T23:06:31.944557+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:44:14.691545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.953737+0000","last_change":"2024-04-24T23:06:31.953569+0000","last_active":"2024-04-24T23:06:31.953737+0000","last_peered":"2024-04-24T23:06:31.953737+0000","last_clean":"2024-04-24T23:06:31.953737+0000","last_became_active":"2024-04-24T23:06:31.953288+0000","last_became_peered":"2024-04-24T23:06:31.953288+0000","last_unstale":"2024-04-24T23:06:31.953737+0000","last_undegraded":"2024-04-24T23:06:31.953737+0000","last_fullsized":"2024-04-24T23:06:31.953737+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:00:30.751504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.934224+0000","last_change":"2024-04-24T23:06:30.936327+0000","last_active":"2024-04-24T23:06:31.934224+0000","last_peered":"2024-04-24T23:06:31.934224+0000","last_clean":"2024-04-24T23:06:31.934224+0000","last_became_active":"2024-04-24T23:06:30.936101+0000","last_became_peered":"2024-04-24T23:06:30.936101+0000","last_unstale":"2024-04-24T23:06:31.934224+0000","last_undegraded":"2024-04-24T23:06:31.934224+0000","last_fullsized":"2024-04-24T23:06:31.934224+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:32:49.703476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.953081+0000","last_change":"2024-04-24T23:06:31.952973+0000","last_active":"2024-04-24T23:06:31.953081+0000","last_peered":"2024-04-24T23:06:31.953081+0000","last_clean":"2024-04-24T23:06:31.953081+0000","last_became_active":"2024-04-24T23:06:31.952686+0000","last_became_peered":"2024-04-24T23:06:31.952686+0000","last_unstale":"2024-04-24T23:06:31.953081+0000","last_undegraded":"2024-04-24T23:06:31.953081+0000","last_fullsized":"2024-04-24T23:06:31.953081+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:48:39.686094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.940501+0000","last_change":"2024-04-24T23:06:30.930903+0000","last_active":"2024-04-24T23:06:31.940501+0000","last_peered":"2024-04-24T23:06:31.940501+0000","last_clean":"2024-04-24T23:06:31.940501+0000","last_became_active":"2024-04-24T23:06:30.929827+0000","last_became_peered":"2024-04-24T23:06:30.929827+0000","last_unstale":"2024-04-24T23:06:31.940501+0000","last_undegraded":"2024-04-24T23:06:31.940501+0000","last_fullsized":"2024-04-24T23:06:31.940501+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:39:10.700191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.938693+0000","last_change":"2024-04-24T23:06:31.938600+0000","last_active":"2024-04-24T23:06:31.938693+0000","last_peered":"2024-04-24T23:06:31.938693+0000","last_clean":"2024-04-24T23:06:31.938693+0000","last_became_active":"2024-04-24T23:06:31.938437+0000","last_became_peered":"2024-04-24T23:06:31.938437+0000","last_unstale":"2024-04-24T23:06:31.938693+0000","last_undegraded":"2024-04-24T23:06:31.938693+0000","last_fullsized":"2024-04-24T23:06:31.938693+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:17.686935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.13","version":"155'2","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.963961+0000","last_change":"2024-04-24T23:06:30.931011+0000","last_active":"2024-04-24T23:06:31.963961+0000","last_peered":"2024-04-24T23:06:31.963961+0000","last_clean":"2024-04-24T23:06:31.963961+0000","last_became_active":"2024-04-24T23:06:30.930694+0000","last_became_peered":"2024-04-24T23:06:30.930694+0000","last_unstale":"2024-04-24T23:06:31.963961+0000","last_undegraded":"2024-04-24T23:06:31.963961+0000","last_fullsized":"2024-04-24T23:06:31.963961+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:47.503208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":435,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.942268+0000","last_change":"2024-04-24T22:58:55.138810+0000","last_active":"2024-04-24T23:06:31.942268+0000","last_peered":"2024-04-24T23:06:31.942268+0000","last_clean":"2024-04-24T23:06:31.942268+0000","last_became_active":"2024-04-24T22:58:55.138627+0000","last_became_peered":"2024-04-24T22:58:55.138627+0000","last_unstale":"2024-04-24T23:06:31.942268+0000","last_undegraded":"2024-04-24T23:06:31.942268+0000","last_fullsized":"2024-04-24T23:06:31.942268+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_clean_scrub_stamp":"2024-04-24T22:58:54.127235+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:51.383182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574942,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":839,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194473,"statfs":{"total":200000000,"available":199140432,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574942,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":839,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194473,"statfs":{"total":200000000,"available":199140432,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574942,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1415,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193896,"statfs":{"total":200000000,"available":198550064,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574942,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":839,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194473,"statfs":{"total":200000000,"available":199140432,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574942,"num_pgs":16,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":839,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194473,"statfs":{"total":200000000,"available":199140432,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574942,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":839,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194473,"statfs":{"total":200000000,"available":199140432,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1424,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193888,"statfs":{"total":200000000,"available":198541655,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574942,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":847,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194464,"statfs":{"total":200000000,"available":199131989,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-24T23:06:34.949 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:06:35.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.084+0000 7fe140363640 1 Processor -- start 2024-04-24T23:06:35.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.084+0000 7fe140363640 1 -- start start 2024-04-24T23:06:35.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.085+0000 7fe140363640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe13815e9b0 0x7fe1380d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:35.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.085+0000 7fe140363640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe13806b820 0x7fe1380b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:35.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.085+0000 7fe140363640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1380d3170 con 0x7fe1380d4210 2024-04-24T23:06:35.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.085+0000 7fe140363640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fe1380d3450 con 0x7fe13815e9b0 2024-04-24T23:06:35.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.085+0000 7fe140363640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fe1380d3730 con 0x7fe13806b820 2024-04-24T23:06:35.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.085+0000 7fe13e0ff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe13815e9b0 0x7fe1380d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:35.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.085+0000 7fe13d8fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe1380d4210 0x7fe13806b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33124/0 (socket says 172.21.15.32:33124) 2024-04-24T23:06:35.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.085+0000 7fe13d8fe640 1 -- 172.21.15.32:0/3313214807 learned_addr learned my addr 172.21.15.32:0/3313214807 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:06:35.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.085+0000 7fe13e0ff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe13815e9b0 0x7fe1380d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48524/0 (socket says 172.21.15.32:48524) 2024-04-24T23:06:35.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.086+0000 7fe13e0ff640 1 -- 172.21.15.32:0/3313214807 >> v2:172.21.15.121:3300/0 conn(0x7fe13806b820 msgr2=0x7fe1380b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:35.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.086+0000 7fe13e900640 1 --2- 172.21.15.32:0/3313214807 >> v2:172.21.15.121:3300/0 conn(0x7fe13806b820 0x7fe1380b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:35.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.086+0000 7fe13e0ff640 1 --2- 172.21.15.32:0/3313214807 >> v2:172.21.15.121:3300/0 conn(0x7fe13806b820 0x7fe1380b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:35.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.086+0000 7fe13d0fd640 1 -- 172.21.15.32:0/3313214807 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 772472164 0 0) 0x7fe1380d3170 con 0x7fe1380d4210 2024-04-24T23:06:35.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.086+0000 7fe13e0ff640 1 -- 172.21.15.32:0/3313214807 >> v1:172.21.15.32:6789/0 conn(0x7fe1380d4210 legacy=0x7fe13806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:35.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.086+0000 7fe13e0ff640 1 -- 172.21.15.32:0/3313214807 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1380d3a10 con 0x7fe13815e9b0 2024-04-24T23:06:35.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.086+0000 7fe13e900640 1 --2- 172.21.15.32:0/3313214807 >> v2:172.21.15.121:3300/0 conn(0x7fe13806b820 0x7fe1380b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:06:35.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.086+0000 7fe13e0ff640 1 --2- 172.21.15.32:0/3313214807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe13815e9b0 0x7fe1380d2b90 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fe1381622f0 tx=0x7fe13002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b081e3b7fd15bd6e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:35.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.087+0000 7fe13d0fd640 1 -- 172.21.15.32:0/3313214807 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe130037070 con 0x7fe13815e9b0 2024-04-24T23:06:35.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.087+0000 7fe13d0fd640 1 -- 172.21.15.32:0/3313214807 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe130032440 con 0x7fe13815e9b0 2024-04-24T23:06:35.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.087+0000 7fe13d0fd640 1 -- 172.21.15.32:0/3313214807 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13002ae60 con 0x7fe13815e9b0 2024-04-24T23:06:35.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.087+0000 7fe140363640 1 -- 172.21.15.32:0/3313214807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe13815e9b0 msgr2=0x7fe1380d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:35.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.087+0000 7fe140363640 1 --2- 172.21.15.32:0/3313214807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe13815e9b0 0x7fe1380d2b90 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fe1381622f0 tx=0x7fe13002a110 comp rx=0 tx=0).stop 2024-04-24T23:06:35.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.087+0000 7fe140363640 1 -- 172.21.15.32:0/3313214807 shutdown_connections 2024-04-24T23:06:35.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.087+0000 7fe140363640 1 --2- 172.21.15.32:0/3313214807 >> v2:172.21.15.121:3300/0 conn(0x7fe13806b820 0x7fe1380b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:35.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.087+0000 7fe140363640 1 --2- 172.21.15.32:0/3313214807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe13815e9b0 0x7fe1380d2b90 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:35.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.087+0000 7fe140363640 1 -- 172.21.15.32:0/3313214807 >> 172.21.15.32:0/3313214807 conn(0x7fe1380d11d0 msgr2=0x7fe1380d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:35.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.087+0000 7fe140363640 1 -- 172.21.15.32:0/3313214807 shutdown_connections 2024-04-24T23:06:35.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.087+0000 7fe140363640 1 -- 172.21.15.32:0/3313214807 wait complete. 2024-04-24T23:06:35.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.088+0000 7fe140363640 1 Processor -- start 2024-04-24T23:06:35.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.088+0000 7fe140363640 1 -- start start 2024-04-24T23:06:35.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.089+0000 7fe140363640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe13806b820 0x7fe1381549c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:35.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.089+0000 7fe140363640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe1380d4210 0x7fe138159b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:35.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.089+0000 7fe140363640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe13815e9b0 0x7fe1381f3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:35.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.089+0000 7fe140363640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe1380ba0e0 con 0x7fe13806b820 2024-04-24T23:06:35.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.089+0000 7fe140363640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe1380d2400 con 0x7fe13815e9b0 2024-04-24T23:06:35.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.089+0000 7fe140363640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe1380ba260 con 0x7fe1380d4210 2024-04-24T23:06:35.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.089+0000 7fe13e0ff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe13806b820 0x7fe1381549c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:35.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.089+0000 7fe13e0ff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe13806b820 0x7fe1381549c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48526/0 (socket says 172.21.15.32:48526) 2024-04-24T23:06:35.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.089+0000 7fe13e0ff640 1 -- 172.21.15.32:0/3173495210 learned_addr learned my addr 172.21.15.32:0/3173495210 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:35.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.089+0000 7fe13e0ff640 1 -- 172.21.15.32:0/3173495210 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe1380d4210 msgr2=0x7fe138159b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:35.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.089+0000 7fe13e0ff640 1 --2- 172.21.15.32:0/3173495210 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe1380d4210 0x7fe138159b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:35.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.089+0000 7fe13e0ff640 1 -- 172.21.15.32:0/3173495210 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe13815e9b0 msgr2=0x7fe1381f3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:35.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.089+0000 7fe13e0ff640 1 --2- 172.21.15.32:0/3173495210 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe13815e9b0 0x7fe1381f3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:35.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.090+0000 7fe13e0ff640 1 -- 172.21.15.32:0/3173495210 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1381f4780 con 0x7fe13806b820 2024-04-24T23:06:35.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.090+0000 7fe13e0ff640 1 --2- 172.21.15.32:0/3173495210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe13806b820 0x7fe1381549c0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fe130000c00 tx=0x7fe1300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:35.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.090+0000 7fe12effd640 1 -- 172.21.15.32:0/3173495210 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe130037050 con 0x7fe13806b820 2024-04-24T23:06:35.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.090+0000 7fe12effd640 1 -- 172.21.15.32:0/3173495210 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe130038040 con 0x7fe13806b820 2024-04-24T23:06:35.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.090+0000 7fe140363640 1 -- 172.21.15.32:0/3173495210 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1381f4a60 con 0x7fe13806b820 2024-04-24T23:06:35.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.090+0000 7fe12effd640 1 -- 172.21.15.32:0/3173495210 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe130004450 con 0x7fe13806b820 2024-04-24T23:06:35.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.090+0000 7fe140363640 1 -- 172.21.15.32:0/3173495210 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe138156910 con 0x7fe13806b820 2024-04-24T23:06:35.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.091+0000 7fe140363640 1 -- 172.21.15.32:0/3173495210 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe138156b10 con 0x7fe13806b820 2024-04-24T23:06:35.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.091+0000 7fe12effd640 1 -- 172.21.15.32:0/3173495210 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe130030980 con 0x7fe13806b820 2024-04-24T23:06:35.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.092+0000 7fe12effd640 1 --2- 172.21.15.32:0/3173495210 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe11c06be30 0x7fe11c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:35.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.092+0000 7fe12effd640 1 -- 172.21.15.32:0/3173495210 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7fe1300aba20 con 0x7fe13806b820 2024-04-24T23:06:35.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.092+0000 7fe13d8fe640 1 --2- 172.21.15.32:0/3173495210 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe11c06be30 0x7fe11c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:35.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.093+0000 7fe13d8fe640 1 --2- 172.21.15.32:0/3173495210 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe11c06be30 0x7fe11c06e2f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fe128007fd0 tx=0x7fe128007920 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:35.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.096+0000 7fe12effd640 1 -- 172.21.15.32:0/3173495210 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe13007a4f0 con 0x7fe13806b820 2024-04-24T23:06:35.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.274+0000 7fe140363640 1 -- 172.21.15.32:0/3173495210 --> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe138068430 con 0x7fe11c06be30 2024-04-24T23:06:35.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.280+0000 7fe12effd640 1 -- 172.21.15.32:0/3173495210 <== mgr.4108 v2:172.21.15.121:6832/1848157943 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247929 (secure 0 0 0) 0x7fe138068430 con 0x7fe11c06be30 2024-04-24T23:06:35.284 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:06:35.285 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-24T23:06:35.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.283+0000 7fe140363640 1 -- 172.21.15.32:0/3173495210 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe11c06be30 msgr2=0x7fe11c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:35.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.283+0000 7fe140363640 1 --2- 172.21.15.32:0/3173495210 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe11c06be30 0x7fe11c06e2f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fe128007fd0 tx=0x7fe128007920 comp rx=0 tx=0).stop 2024-04-24T23:06:35.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.283+0000 7fe140363640 1 -- 172.21.15.32:0/3173495210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe13806b820 msgr2=0x7fe1381549c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:35.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.283+0000 7fe140363640 1 --2- 172.21.15.32:0/3173495210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe13806b820 0x7fe1381549c0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fe130000c00 tx=0x7fe1300046c0 comp rx=0 tx=0).stop 2024-04-24T23:06:35.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.283+0000 7fe140363640 1 -- 172.21.15.32:0/3173495210 shutdown_connections 2024-04-24T23:06:35.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.283+0000 7fe140363640 1 --2- 172.21.15.32:0/3173495210 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe11c06be30 0x7fe11c06e2f0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:35.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.283+0000 7fe140363640 1 --2- 172.21.15.32:0/3173495210 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe13815e9b0 0x7fe1381f3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:35.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.283+0000 7fe140363640 1 --2- 172.21.15.32:0/3173495210 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe1380d4210 0x7fe138159b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:35.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.283+0000 7fe140363640 1 --2- 172.21.15.32:0/3173495210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe13806b820 0x7fe1381549c0 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:35.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.283+0000 7fe140363640 1 -- 172.21.15.32:0/3173495210 >> 172.21.15.32:0/3173495210 conn(0x7fe1380d11d0 msgr2=0x7fe138155cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:35.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.283+0000 7fe140363640 1 -- 172.21.15.32:0/3173495210 shutdown_connections 2024-04-24T23:06:35.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.283+0000 7fe140363640 1 -- 172.21.15.32:0/3173495210 wait complete. 2024-04-24T23:06:35.296 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":379,"stamp":"2024-04-24T23:06:35.232687+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":96,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":7884,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1554615,"statfs":{"total":1600000000,"available":1591925868,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.312184"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.950189+0000","last_change":"2024-04-24T23:06:31.949729+0000","last_active":"2024-04-24T23:06:31.950189+0000","last_peered":"2024-04-24T23:06:31.950189+0000","last_clean":"2024-04-24T23:06:31.950189+0000","last_became_active":"2024-04-24T23:06:31.949221+0000","last_became_peered":"2024-04-24T23:06:31.949221+0000","last_unstale":"2024-04-24T23:06:31.950189+0000","last_undegraded":"2024-04-24T23:06:31.950189+0000","last_fullsized":"2024-04-24T23:06:31.950189+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:00:33.666522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.950377+0000","last_change":"2024-04-24T23:06:31.950265+0000","last_active":"2024-04-24T23:06:31.950377+0000","last_peered":"2024-04-24T23:06:31.950377+0000","last_clean":"2024-04-24T23:06:31.950377+0000","last_became_active":"2024-04-24T23:06:31.948691+0000","last_became_peered":"2024-04-24T23:06:31.948691+0000","last_unstale":"2024-04-24T23:06:31.950377+0000","last_undegraded":"2024-04-24T23:06:31.950377+0000","last_fullsized":"2024-04-24T23:06:31.950377+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:48:10.318356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.950192+0000","last_change":"2024-04-24T23:06:31.949468+0000","last_active":"2024-04-24T23:06:31.950192+0000","last_peered":"2024-04-24T23:06:31.950192+0000","last_clean":"2024-04-24T23:06:31.950192+0000","last_became_active":"2024-04-24T23:06:31.949152+0000","last_became_peered":"2024-04-24T23:06:31.949152+0000","last_unstale":"2024-04-24T23:06:31.950192+0000","last_undegraded":"2024-04-24T23:06:31.950192+0000","last_fullsized":"2024-04-24T23:06:31.950192+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:25.089775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.949366+0000","last_change":"2024-04-24T23:06:31.949242+0000","last_active":"2024-04-24T23:06:31.949366+0000","last_peered":"2024-04-24T23:06:31.949366+0000","last_clean":"2024-04-24T23:06:31.949366+0000","last_became_active":"2024-04-24T23:06:31.949002+0000","last_became_peered":"2024-04-24T23:06:31.949002+0000","last_unstale":"2024-04-24T23:06:31.949366+0000","last_undegraded":"2024-04-24T23:06:31.949366+0000","last_fullsized":"2024-04-24T23:06:31.949366+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:29:58.088447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.952340+0000","last_change":"2024-04-24T23:06:31.951849+0000","last_active":"2024-04-24T23:06:31.952340+0000","last_peered":"2024-04-24T23:06:31.952340+0000","last_clean":"2024-04-24T23:06:31.952340+0000","last_became_active":"2024-04-24T23:06:31.951398+0000","last_became_peered":"2024-04-24T23:06:31.951398+0000","last_unstale":"2024-04-24T23:06:31.952340+0000","last_undegraded":"2024-04-24T23:06:31.952340+0000","last_fullsized":"2024-04-24T23:06:31.952340+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:10:23.370363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.953387+0000","last_change":"2024-04-24T23:06:31.953253+0000","last_active":"2024-04-24T23:06:31.953387+0000","last_peered":"2024-04-24T23:06:31.953387+0000","last_clean":"2024-04-24T23:06:31.953387+0000","last_became_active":"2024-04-24T23:06:31.952847+0000","last_became_peered":"2024-04-24T23:06:31.952847+0000","last_unstale":"2024-04-24T23:06:31.953387+0000","last_undegraded":"2024-04-24T23:06:31.953387+0000","last_fullsized":"2024-04-24T23:06:31.953387+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:05:24.712535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.953083+0000","last_change":"2024-04-24T23:06:31.952957+0000","last_active":"2024-04-24T23:06:31.953083+0000","last_peered":"2024-04-24T23:06:31.953083+0000","last_clean":"2024-04-24T23:06:31.953083+0000","last_became_active":"2024-04-24T23:06:31.952499+0000","last_became_peered":"2024-04-24T23:06:31.952499+0000","last_unstale":"2024-04-24T23:06:31.953083+0000","last_undegraded":"2024-04-24T23:06:31.953083+0000","last_fullsized":"2024-04-24T23:06:31.953083+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:23:43.601408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.951570+0000","last_change":"2024-04-24T23:06:31.951259+0000","last_active":"2024-04-24T23:06:31.951570+0000","last_peered":"2024-04-24T23:06:31.951570+0000","last_clean":"2024-04-24T23:06:31.951570+0000","last_became_active":"2024-04-24T23:06:31.950019+0000","last_became_peered":"2024-04-24T23:06:31.950019+0000","last_unstale":"2024-04-24T23:06:31.951570+0000","last_undegraded":"2024-04-24T23:06:31.951570+0000","last_fullsized":"2024-04-24T23:06:31.951570+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:11:15.995269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.954474+0000","last_change":"2024-04-24T23:06:31.954317+0000","last_active":"2024-04-24T23:06:31.954474+0000","last_peered":"2024-04-24T23:06:31.954474+0000","last_clean":"2024-04-24T23:06:31.954474+0000","last_became_active":"2024-04-24T23:06:31.953759+0000","last_became_peered":"2024-04-24T23:06:31.953759+0000","last_unstale":"2024-04-24T23:06:31.954474+0000","last_undegraded":"2024-04-24T23:06:31.954474+0000","last_fullsized":"2024-04-24T23:06:31.954474+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:44:52.482584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.948776+0000","last_change":"2024-04-24T23:06:31.948657+0000","last_active":"2024-04-24T23:06:31.948776+0000","last_peered":"2024-04-24T23:06:31.948776+0000","last_clean":"2024-04-24T23:06:31.948776+0000","last_became_active":"2024-04-24T23:06:31.948408+0000","last_became_peered":"2024-04-24T23:06:31.948408+0000","last_unstale":"2024-04-24T23:06:31.948776+0000","last_undegraded":"2024-04-24T23:06:31.948776+0000","last_fullsized":"2024-04-24T23:06:31.948776+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:48:04.449827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.941096+0000","last_change":"2024-04-24T23:06:31.940951+0000","last_active":"2024-04-24T23:06:31.941096+0000","last_peered":"2024-04-24T23:06:31.941096+0000","last_clean":"2024-04-24T23:06:31.941096+0000","last_became_active":"2024-04-24T23:06:31.940650+0000","last_became_peered":"2024-04-24T23:06:31.940650+0000","last_unstale":"2024-04-24T23:06:31.941096+0000","last_undegraded":"2024-04-24T23:06:31.941096+0000","last_fullsized":"2024-04-24T23:06:31.941096+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:06:19.925551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.952541+0000","last_change":"2024-04-24T23:06:31.951451+0000","last_active":"2024-04-24T23:06:31.952541+0000","last_peered":"2024-04-24T23:06:31.952541+0000","last_clean":"2024-04-24T23:06:31.952541+0000","last_became_active":"2024-04-24T23:06:31.951227+0000","last_became_peered":"2024-04-24T23:06:31.951227+0000","last_unstale":"2024-04-24T23:06:31.952541+0000","last_undegraded":"2024-04-24T23:06:31.952541+0000","last_fullsized":"2024-04-24T23:06:31.952541+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:50:42.991995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.954912+0000","last_change":"2024-04-24T23:06:31.954828+0000","last_active":"2024-04-24T23:06:31.954912+0000","last_peered":"2024-04-24T23:06:31.954912+0000","last_clean":"2024-04-24T23:06:31.954912+0000","last_became_active":"2024-04-24T23:06:31.954662+0000","last_became_peered":"2024-04-24T23:06:31.954662+0000","last_unstale":"2024-04-24T23:06:31.954912+0000","last_undegraded":"2024-04-24T23:06:31.954912+0000","last_fullsized":"2024-04-24T23:06:31.954912+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:13:35.661141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.951672+0000","last_change":"2024-04-24T23:06:31.951563+0000","last_active":"2024-04-24T23:06:31.951672+0000","last_peered":"2024-04-24T23:06:31.951672+0000","last_clean":"2024-04-24T23:06:31.951672+0000","last_became_active":"2024-04-24T23:06:31.951181+0000","last_became_peered":"2024-04-24T23:06:31.951181+0000","last_unstale":"2024-04-24T23:06:31.951672+0000","last_undegraded":"2024-04-24T23:06:31.951672+0000","last_fullsized":"2024-04-24T23:06:31.951672+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:38:39.800063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.946562+0000","last_change":"2024-04-24T23:06:31.946438+0000","last_active":"2024-04-24T23:06:31.946562+0000","last_peered":"2024-04-24T23:06:31.946562+0000","last_clean":"2024-04-24T23:06:31.946562+0000","last_became_active":"2024-04-24T23:06:31.945718+0000","last_became_peered":"2024-04-24T23:06:31.945718+0000","last_unstale":"2024-04-24T23:06:31.946562+0000","last_undegraded":"2024-04-24T23:06:31.946562+0000","last_fullsized":"2024-04-24T23:06:31.946562+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:20:42.499371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.940313+0000","last_change":"2024-04-24T23:06:31.940065+0000","last_active":"2024-04-24T23:06:31.940313+0000","last_peered":"2024-04-24T23:06:31.940313+0000","last_clean":"2024-04-24T23:06:31.940313+0000","last_became_active":"2024-04-24T23:06:31.939773+0000","last_became_peered":"2024-04-24T23:06:31.939773+0000","last_unstale":"2024-04-24T23:06:31.940313+0000","last_undegraded":"2024-04-24T23:06:31.940313+0000","last_fullsized":"2024-04-24T23:06:31.940313+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:00.690415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.937892+0000","last_change":"2024-04-24T23:06:31.937777+0000","last_active":"2024-04-24T23:06:31.937892+0000","last_peered":"2024-04-24T23:06:31.937892+0000","last_clean":"2024-04-24T23:06:31.937892+0000","last_became_active":"2024-04-24T23:06:31.937541+0000","last_became_peered":"2024-04-24T23:06:31.937541+0000","last_unstale":"2024-04-24T23:06:31.937892+0000","last_undegraded":"2024-04-24T23:06:31.937892+0000","last_fullsized":"2024-04-24T23:06:31.937892+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:37:36.370336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.951445+0000","last_change":"2024-04-24T23:06:31.951318+0000","last_active":"2024-04-24T23:06:31.951445+0000","last_peered":"2024-04-24T23:06:31.951445+0000","last_clean":"2024-04-24T23:06:31.951445+0000","last_became_active":"2024-04-24T23:06:31.950991+0000","last_became_peered":"2024-04-24T23:06:31.950991+0000","last_unstale":"2024-04-24T23:06:31.951445+0000","last_undegraded":"2024-04-24T23:06:31.951445+0000","last_fullsized":"2024-04-24T23:06:31.951445+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:03:23.326244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.947332+0000","last_change":"2024-04-24T23:06:31.947181+0000","last_active":"2024-04-24T23:06:31.947332+0000","last_peered":"2024-04-24T23:06:31.947332+0000","last_clean":"2024-04-24T23:06:31.947332+0000","last_became_active":"2024-04-24T23:06:31.946631+0000","last_became_peered":"2024-04-24T23:06:31.946631+0000","last_unstale":"2024-04-24T23:06:31.947332+0000","last_undegraded":"2024-04-24T23:06:31.947332+0000","last_fullsized":"2024-04-24T23:06:31.947332+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:17:24.084994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.946186+0000","last_change":"2024-04-24T23:06:31.945982+0000","last_active":"2024-04-24T23:06:31.946186+0000","last_peered":"2024-04-24T23:06:31.946186+0000","last_clean":"2024-04-24T23:06:31.946186+0000","last_became_active":"2024-04-24T23:06:31.944899+0000","last_became_peered":"2024-04-24T23:06:31.944899+0000","last_unstale":"2024-04-24T23:06:31.946186+0000","last_undegraded":"2024-04-24T23:06:31.946186+0000","last_fullsized":"2024-04-24T23:06:31.946186+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:40:17.005051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.951573+0000","last_change":"2024-04-24T23:06:31.950843+0000","last_active":"2024-04-24T23:06:31.951573+0000","last_peered":"2024-04-24T23:06:31.951573+0000","last_clean":"2024-04-24T23:06:31.951573+0000","last_became_active":"2024-04-24T23:06:31.949890+0000","last_became_peered":"2024-04-24T23:06:31.949890+0000","last_unstale":"2024-04-24T23:06:31.951573+0000","last_undegraded":"2024-04-24T23:06:31.951573+0000","last_fullsized":"2024-04-24T23:06:31.951573+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:55:20.855052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.949042+0000","last_change":"2024-04-24T23:06:31.948931+0000","last_active":"2024-04-24T23:06:31.949042+0000","last_peered":"2024-04-24T23:06:31.949042+0000","last_clean":"2024-04-24T23:06:31.949042+0000","last_became_active":"2024-04-24T23:06:31.947254+0000","last_became_peered":"2024-04-24T23:06:31.947254+0000","last_unstale":"2024-04-24T23:06:31.949042+0000","last_undegraded":"2024-04-24T23:06:31.949042+0000","last_fullsized":"2024-04-24T23:06:31.949042+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:48:02.259154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.956578+0000","last_change":"2024-04-24T23:06:31.955813+0000","last_active":"2024-04-24T23:06:31.956578+0000","last_peered":"2024-04-24T23:06:31.956578+0000","last_clean":"2024-04-24T23:06:31.956578+0000","last_became_active":"2024-04-24T23:06:31.954874+0000","last_became_peered":"2024-04-24T23:06:31.954874+0000","last_unstale":"2024-04-24T23:06:31.956578+0000","last_undegraded":"2024-04-24T23:06:31.956578+0000","last_fullsized":"2024-04-24T23:06:31.956578+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:02:26.392545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.952664+0000","last_change":"2024-04-24T23:06:31.952538+0000","last_active":"2024-04-24T23:06:31.952664+0000","last_peered":"2024-04-24T23:06:31.952664+0000","last_clean":"2024-04-24T23:06:31.952664+0000","last_became_active":"2024-04-24T23:06:31.952299+0000","last_became_peered":"2024-04-24T23:06:31.952299+0000","last_unstale":"2024-04-24T23:06:31.952664+0000","last_undegraded":"2024-04-24T23:06:31.952664+0000","last_fullsized":"2024-04-24T23:06:31.952664+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:10:26.871816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.944164+0000","last_change":"2024-04-24T23:06:31.944040+0000","last_active":"2024-04-24T23:06:31.944164+0000","last_peered":"2024-04-24T23:06:31.944164+0000","last_clean":"2024-04-24T23:06:31.944164+0000","last_became_active":"2024-04-24T23:06:31.943258+0000","last_became_peered":"2024-04-24T23:06:31.943258+0000","last_unstale":"2024-04-24T23:06:31.944164+0000","last_undegraded":"2024-04-24T23:06:31.944164+0000","last_fullsized":"2024-04-24T23:06:31.944164+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:54:01.279196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.950153+0000","last_change":"2024-04-24T23:06:31.949490+0000","last_active":"2024-04-24T23:06:31.950153+0000","last_peered":"2024-04-24T23:06:31.950153+0000","last_clean":"2024-04-24T23:06:31.950153+0000","last_became_active":"2024-04-24T23:06:31.947623+0000","last_became_peered":"2024-04-24T23:06:31.947623+0000","last_unstale":"2024-04-24T23:06:31.950153+0000","last_undegraded":"2024-04-24T23:06:31.950153+0000","last_fullsized":"2024-04-24T23:06:31.950153+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:34:27.906820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.945972+0000","last_change":"2024-04-24T23:06:31.945838+0000","last_active":"2024-04-24T23:06:31.945972+0000","last_peered":"2024-04-24T23:06:31.945972+0000","last_clean":"2024-04-24T23:06:31.945972+0000","last_became_active":"2024-04-24T23:06:31.945599+0000","last_became_peered":"2024-04-24T23:06:31.945599+0000","last_unstale":"2024-04-24T23:06:31.945972+0000","last_undegraded":"2024-04-24T23:06:31.945972+0000","last_fullsized":"2024-04-24T23:06:31.945972+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:54:30.678507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.954524+0000","last_change":"2024-04-24T23:06:31.954417+0000","last_active":"2024-04-24T23:06:31.954524+0000","last_peered":"2024-04-24T23:06:31.954524+0000","last_clean":"2024-04-24T23:06:31.954524+0000","last_became_active":"2024-04-24T23:06:31.952867+0000","last_became_peered":"2024-04-24T23:06:31.952867+0000","last_unstale":"2024-04-24T23:06:31.954524+0000","last_undegraded":"2024-04-24T23:06:31.954524+0000","last_fullsized":"2024-04-24T23:06:31.954524+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:37:04.031161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.946282+0000","last_change":"2024-04-24T23:06:31.945326+0000","last_active":"2024-04-24T23:06:31.946282+0000","last_peered":"2024-04-24T23:06:31.946282+0000","last_clean":"2024-04-24T23:06:31.946282+0000","last_became_active":"2024-04-24T23:06:31.945094+0000","last_became_peered":"2024-04-24T23:06:31.945094+0000","last_unstale":"2024-04-24T23:06:31.946282+0000","last_undegraded":"2024-04-24T23:06:31.946282+0000","last_fullsized":"2024-04-24T23:06:31.946282+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:56:33.583507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.950173+0000","last_change":"2024-04-24T23:06:31.949666+0000","last_active":"2024-04-24T23:06:31.950173+0000","last_peered":"2024-04-24T23:06:31.950173+0000","last_clean":"2024-04-24T23:06:31.950173+0000","last_became_active":"2024-04-24T23:06:31.949394+0000","last_became_peered":"2024-04-24T23:06:31.949394+0000","last_unstale":"2024-04-24T23:06:31.950173+0000","last_undegraded":"2024-04-24T23:06:31.950173+0000","last_fullsized":"2024-04-24T23:06:31.950173+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:51:33.586275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.948849+0000","last_change":"2024-04-24T23:06:31.947130+0000","last_active":"2024-04-24T23:06:31.948849+0000","last_peered":"2024-04-24T23:06:31.948849+0000","last_clean":"2024-04-24T23:06:31.948849+0000","last_became_active":"2024-04-24T23:06:31.946785+0000","last_became_peered":"2024-04-24T23:06:31.946785+0000","last_unstale":"2024-04-24T23:06:31.948849+0000","last_undegraded":"2024-04-24T23:06:31.948849+0000","last_fullsized":"2024-04-24T23:06:31.948849+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:10:26.367289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.950478+0000","last_change":"2024-04-24T23:06:31.950355+0000","last_active":"2024-04-24T23:06:31.950478+0000","last_peered":"2024-04-24T23:06:31.950478+0000","last_clean":"2024-04-24T23:06:31.950478+0000","last_became_active":"2024-04-24T23:06:31.950121+0000","last_became_peered":"2024-04-24T23:06:31.950121+0000","last_unstale":"2024-04-24T23:06:31.950478+0000","last_undegraded":"2024-04-24T23:06:31.950478+0000","last_fullsized":"2024-04-24T23:06:31.950478+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:58:46.701785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.945255+0000","last_change":"2024-04-24T23:06:31.944917+0000","last_active":"2024-04-24T23:06:31.945255+0000","last_peered":"2024-04-24T23:06:31.945255+0000","last_clean":"2024-04-24T23:06:31.945255+0000","last_became_active":"2024-04-24T23:06:31.944641+0000","last_became_peered":"2024-04-24T23:06:31.944641+0000","last_unstale":"2024-04-24T23:06:31.945255+0000","last_undegraded":"2024-04-24T23:06:31.945255+0000","last_fullsized":"2024-04-24T23:06:31.945255+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:57:50.603164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1f","version":"155'4","reported_seq":16,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.963364+0000","last_change":"2024-04-24T23:06:30.930128+0000","last_active":"2024-04-24T23:06:31.963364+0000","last_peered":"2024-04-24T23:06:31.963364+0000","last_clean":"2024-04-24T23:06:31.963364+0000","last_became_active":"2024-04-24T23:06:30.929383+0000","last_became_peered":"2024-04-24T23:06:30.929383+0000","last_unstale":"2024-04-24T23:06:31.963364+0000","last_undegraded":"2024-04-24T23:06:31.963364+0000","last_fullsized":"2024-04-24T23:06:31.963364+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:53:36.932051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.937907+0000","last_change":"2024-04-24T23:06:31.937822+0000","last_active":"2024-04-24T23:06:31.937907+0000","last_peered":"2024-04-24T23:06:31.937907+0000","last_clean":"2024-04-24T23:06:31.937907+0000","last_became_active":"2024-04-24T23:06:31.937596+0000","last_became_peered":"2024-04-24T23:06:31.937596+0000","last_unstale":"2024-04-24T23:06:31.937907+0000","last_undegraded":"2024-04-24T23:06:31.937907+0000","last_fullsized":"2024-04-24T23:06:31.937907+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:29:46.851064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1e","version":"155'1","reported_seq":13,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.965589+0000","last_change":"2024-04-24T23:06:30.935179+0000","last_active":"2024-04-24T23:06:31.965589+0000","last_peered":"2024-04-24T23:06:31.965589+0000","last_clean":"2024-04-24T23:06:31.965589+0000","last_became_active":"2024-04-24T23:06:30.934892+0000","last_became_peered":"2024-04-24T23:06:30.934892+0000","last_unstale":"2024-04-24T23:06:31.965589+0000","last_undegraded":"2024-04-24T23:06:31.965589+0000","last_fullsized":"2024-04-24T23:06:31.965589+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:36.115975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.945417+0000","last_change":"2024-04-24T23:06:31.945287+0000","last_active":"2024-04-24T23:06:31.945417+0000","last_peered":"2024-04-24T23:06:31.945417+0000","last_clean":"2024-04-24T23:06:31.945417+0000","last_became_active":"2024-04-24T23:06:31.945044+0000","last_became_peered":"2024-04-24T23:06:31.945044+0000","last_unstale":"2024-04-24T23:06:31.945417+0000","last_undegraded":"2024-04-24T23:06:31.945417+0000","last_fullsized":"2024-04-24T23:06:31.945417+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:27.707116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1d","version":"155'2","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.963188+0000","last_change":"2024-04-24T23:06:30.933020+0000","last_active":"2024-04-24T23:06:31.963188+0000","last_peered":"2024-04-24T23:06:31.963188+0000","last_clean":"2024-04-24T23:06:31.963188+0000","last_became_active":"2024-04-24T23:06:30.932290+0000","last_became_peered":"2024-04-24T23:06:30.932290+0000","last_unstale":"2024-04-24T23:06:31.963188+0000","last_undegraded":"2024-04-24T23:06:31.963188+0000","last_fullsized":"2024-04-24T23:06:31.963188+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:09.649102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.945330+0000","last_change":"2024-04-24T23:06:31.945178+0000","last_active":"2024-04-24T23:06:31.945330+0000","last_peered":"2024-04-24T23:06:31.945330+0000","last_clean":"2024-04-24T23:06:31.945330+0000","last_became_active":"2024-04-24T23:06:31.944927+0000","last_became_peered":"2024-04-24T23:06:31.944927+0000","last_unstale":"2024-04-24T23:06:31.945330+0000","last_undegraded":"2024-04-24T23:06:31.945330+0000","last_fullsized":"2024-04-24T23:06:31.945330+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:45:12.542275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1c","version":"155'2","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.962666+0000","last_change":"2024-04-24T23:06:30.932975+0000","last_active":"2024-04-24T23:06:31.962666+0000","last_peered":"2024-04-24T23:06:31.962666+0000","last_clean":"2024-04-24T23:06:31.962666+0000","last_became_active":"2024-04-24T23:06:30.932724+0000","last_became_peered":"2024-04-24T23:06:30.932724+0000","last_unstale":"2024-04-24T23:06:31.962666+0000","last_undegraded":"2024-04-24T23:06:31.962666+0000","last_fullsized":"2024-04-24T23:06:31.962666+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:07:33.232218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.954026+0000","last_change":"2024-04-24T23:06:31.953879+0000","last_active":"2024-04-24T23:06:31.954026+0000","last_peered":"2024-04-24T23:06:31.954026+0000","last_clean":"2024-04-24T23:06:31.954026+0000","last_became_active":"2024-04-24T23:06:31.953612+0000","last_became_peered":"2024-04-24T23:06:31.953612+0000","last_unstale":"2024-04-24T23:06:31.954026+0000","last_undegraded":"2024-04-24T23:06:31.954026+0000","last_fullsized":"2024-04-24T23:06:31.954026+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:29:32.344819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1b","version":"155'1","reported_seq":13,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.961789+0000","last_change":"2024-04-24T23:06:30.938313+0000","last_active":"2024-04-24T23:06:31.961789+0000","last_peered":"2024-04-24T23:06:31.961789+0000","last_clean":"2024-04-24T23:06:31.961789+0000","last_became_active":"2024-04-24T23:06:30.937647+0000","last_became_peered":"2024-04-24T23:06:30.937647+0000","last_unstale":"2024-04-24T23:06:31.961789+0000","last_undegraded":"2024-04-24T23:06:31.961789+0000","last_fullsized":"2024-04-24T23:06:31.961789+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:07:57.792089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.951530+0000","last_change":"2024-04-24T23:06:31.949219+0000","last_active":"2024-04-24T23:06:31.951530+0000","last_peered":"2024-04-24T23:06:31.951530+0000","last_clean":"2024-04-24T23:06:31.951530+0000","last_became_active":"2024-04-24T23:06:31.948075+0000","last_became_peered":"2024-04-24T23:06:31.948075+0000","last_unstale":"2024-04-24T23:06:31.951530+0000","last_undegraded":"2024-04-24T23:06:31.951530+0000","last_fullsized":"2024-04-24T23:06:31.951530+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:25:21.049448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"155'1","reported_seq":13,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.960126+0000","last_change":"2024-04-24T23:06:30.931776+0000","last_active":"2024-04-24T23:06:31.960126+0000","last_peered":"2024-04-24T23:06:31.960126+0000","last_clean":"2024-04-24T23:06:31.960126+0000","last_became_active":"2024-04-24T23:06:30.931553+0000","last_became_peered":"2024-04-24T23:06:30.931553+0000","last_unstale":"2024-04-24T23:06:31.960126+0000","last_undegraded":"2024-04-24T23:06:31.960126+0000","last_fullsized":"2024-04-24T23:06:31.960126+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:10:37.924519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.946402+0000","last_change":"2024-04-24T23:06:31.946274+0000","last_active":"2024-04-24T23:06:31.946402+0000","last_peered":"2024-04-24T23:06:31.946402+0000","last_clean":"2024-04-24T23:06:31.946402+0000","last_became_active":"2024-04-24T23:06:31.945966+0000","last_became_peered":"2024-04-24T23:06:31.945966+0000","last_unstale":"2024-04-24T23:06:31.946402+0000","last_undegraded":"2024-04-24T23:06:31.946402+0000","last_fullsized":"2024-04-24T23:06:31.946402+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:48.803786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.936839+0000","last_change":"2024-04-24T23:06:30.926908+0000","last_active":"2024-04-24T23:06:31.936839+0000","last_peered":"2024-04-24T23:06:31.936839+0000","last_clean":"2024-04-24T23:06:31.936839+0000","last_became_active":"2024-04-24T23:06:30.925544+0000","last_became_peered":"2024-04-24T23:06:30.925544+0000","last_unstale":"2024-04-24T23:06:31.936839+0000","last_undegraded":"2024-04-24T23:06:31.936839+0000","last_fullsized":"2024-04-24T23:06:31.936839+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:49:25.469065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.943031+0000","last_change":"2024-04-24T23:06:31.942900+0000","last_active":"2024-04-24T23:06:31.943031+0000","last_peered":"2024-04-24T23:06:31.943031+0000","last_clean":"2024-04-24T23:06:31.943031+0000","last_became_active":"2024-04-24T23:06:31.942661+0000","last_became_peered":"2024-04-24T23:06:31.942661+0000","last_unstale":"2024-04-24T23:06:31.943031+0000","last_undegraded":"2024-04-24T23:06:31.943031+0000","last_fullsized":"2024-04-24T23:06:31.943031+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:53.045519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.944091+0000","last_change":"2024-04-24T23:06:30.935423+0000","last_active":"2024-04-24T23:06:31.944091+0000","last_peered":"2024-04-24T23:06:31.944091+0000","last_clean":"2024-04-24T23:06:31.944091+0000","last_became_active":"2024-04-24T23:06:30.935168+0000","last_became_peered":"2024-04-24T23:06:30.935168+0000","last_unstale":"2024-04-24T23:06:31.944091+0000","last_undegraded":"2024-04-24T23:06:31.944091+0000","last_fullsized":"2024-04-24T23:06:31.944091+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:50:31.858373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.949378+0000","last_change":"2024-04-24T23:06:31.949115+0000","last_active":"2024-04-24T23:06:31.949378+0000","last_peered":"2024-04-24T23:06:31.949378+0000","last_clean":"2024-04-24T23:06:31.949378+0000","last_became_active":"2024-04-24T23:06:31.947551+0000","last_became_peered":"2024-04-24T23:06:31.947551+0000","last_unstale":"2024-04-24T23:06:31.949378+0000","last_undegraded":"2024-04-24T23:06:31.949378+0000","last_fullsized":"2024-04-24T23:06:31.949378+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:25.397488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.944077+0000","last_change":"2024-04-24T23:06:30.933074+0000","last_active":"2024-04-24T23:06:31.944077+0000","last_peered":"2024-04-24T23:06:31.944077+0000","last_clean":"2024-04-24T23:06:31.944077+0000","last_became_active":"2024-04-24T23:06:30.932772+0000","last_became_peered":"2024-04-24T23:06:30.932772+0000","last_unstale":"2024-04-24T23:06:31.944077+0000","last_undegraded":"2024-04-24T23:06:31.944077+0000","last_fullsized":"2024-04-24T23:06:31.944077+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:18:01.691756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.950021+0000","last_change":"2024-04-24T23:06:31.949759+0000","last_active":"2024-04-24T23:06:31.950021+0000","last_peered":"2024-04-24T23:06:31.950021+0000","last_clean":"2024-04-24T23:06:31.950021+0000","last_became_active":"2024-04-24T23:06:31.949521+0000","last_became_peered":"2024-04-24T23:06:31.949521+0000","last_unstale":"2024-04-24T23:06:31.950021+0000","last_undegraded":"2024-04-24T23:06:31.950021+0000","last_fullsized":"2024-04-24T23:06:31.950021+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:55.349838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.946387+0000","last_change":"2024-04-24T23:06:30.931620+0000","last_active":"2024-04-24T23:06:31.946387+0000","last_peered":"2024-04-24T23:06:31.946387+0000","last_clean":"2024-04-24T23:06:31.946387+0000","last_became_active":"2024-04-24T23:06:30.931343+0000","last_became_peered":"2024-04-24T23:06:30.931343+0000","last_unstale":"2024-04-24T23:06:31.946387+0000","last_undegraded":"2024-04-24T23:06:31.946387+0000","last_fullsized":"2024-04-24T23:06:31.946387+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:02:57.038500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.951547+0000","last_change":"2024-04-24T23:06:31.948344+0000","last_active":"2024-04-24T23:06:31.951547+0000","last_peered":"2024-04-24T23:06:31.951547+0000","last_clean":"2024-04-24T23:06:31.951547+0000","last_became_active":"2024-04-24T23:06:31.948053+0000","last_became_peered":"2024-04-24T23:06:31.948053+0000","last_unstale":"2024-04-24T23:06:31.951547+0000","last_undegraded":"2024-04-24T23:06:31.951547+0000","last_fullsized":"2024-04-24T23:06:31.951547+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:04:07.117896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.15","version":"155'3","reported_seq":15,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.963705+0000","last_change":"2024-04-24T23:06:30.929875+0000","last_active":"2024-04-24T23:06:31.963705+0000","last_peered":"2024-04-24T23:06:31.963705+0000","last_clean":"2024-04-24T23:06:31.963705+0000","last_became_active":"2024-04-24T23:06:30.929051+0000","last_became_peered":"2024-04-24T23:06:30.929051+0000","last_unstale":"2024-04-24T23:06:31.963705+0000","last_undegraded":"2024-04-24T23:06:31.963705+0000","last_fullsized":"2024-04-24T23:06:31.963705+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:17:00.281627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.954238+0000","last_change":"2024-04-24T23:06:31.954113+0000","last_active":"2024-04-24T23:06:31.954238+0000","last_peered":"2024-04-24T23:06:31.954238+0000","last_clean":"2024-04-24T23:06:31.954238+0000","last_became_active":"2024-04-24T23:06:31.952536+0000","last_became_peered":"2024-04-24T23:06:31.952536+0000","last_unstale":"2024-04-24T23:06:31.954238+0000","last_undegraded":"2024-04-24T23:06:31.954238+0000","last_fullsized":"2024-04-24T23:06:31.954238+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:50:22.152102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.14","version":"155'6","reported_seq":18,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.966311+0000","last_change":"2024-04-24T23:06:30.935668+0000","last_active":"2024-04-24T23:06:31.966311+0000","last_peered":"2024-04-24T23:06:31.966311+0000","last_clean":"2024-04-24T23:06:31.966311+0000","last_became_active":"2024-04-24T23:06:30.935438+0000","last_became_peered":"2024-04-24T23:06:30.935438+0000","last_unstale":"2024-04-24T23:06:31.966311+0000","last_undegraded":"2024-04-24T23:06:31.966311+0000","last_fullsized":"2024-04-24T23:06:31.966311+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:27:55.856001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.947274+0000","last_change":"2024-04-24T23:06:31.946314+0000","last_active":"2024-04-24T23:06:31.947274+0000","last_peered":"2024-04-24T23:06:31.947274+0000","last_clean":"2024-04-24T23:06:31.947274+0000","last_became_active":"2024-04-24T23:06:31.946000+0000","last_became_peered":"2024-04-24T23:06:31.946000+0000","last_unstale":"2024-04-24T23:06:31.947274+0000","last_undegraded":"2024-04-24T23:06:31.947274+0000","last_fullsized":"2024-04-24T23:06:31.947274+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:44:16.658014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.938323+0000","last_change":"2024-04-24T23:06:30.927557+0000","last_active":"2024-04-24T23:06:31.938323+0000","last_peered":"2024-04-24T23:06:31.938323+0000","last_clean":"2024-04-24T23:06:31.938323+0000","last_became_active":"2024-04-24T23:06:30.927310+0000","last_became_peered":"2024-04-24T23:06:30.927310+0000","last_unstale":"2024-04-24T23:06:31.938323+0000","last_undegraded":"2024-04-24T23:06:31.938323+0000","last_fullsized":"2024-04-24T23:06:31.938323+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:01.699089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.955727+0000","last_change":"2024-04-24T23:06:31.955645+0000","last_active":"2024-04-24T23:06:31.955727+0000","last_peered":"2024-04-24T23:06:31.955727+0000","last_clean":"2024-04-24T23:06:31.955727+0000","last_became_active":"2024-04-24T23:06:31.955426+0000","last_became_peered":"2024-04-24T23:06:31.955426+0000","last_unstale":"2024-04-24T23:06:31.955727+0000","last_undegraded":"2024-04-24T23:06:31.955727+0000","last_fullsized":"2024-04-24T23:06:31.955727+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:32.529668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.933746+0000","last_change":"2024-04-24T23:06:30.933053+0000","last_active":"2024-04-24T23:06:31.933746+0000","last_peered":"2024-04-24T23:06:31.933746+0000","last_clean":"2024-04-24T23:06:31.933746+0000","last_became_active":"2024-04-24T23:06:30.932798+0000","last_became_peered":"2024-04-24T23:06:30.932798+0000","last_unstale":"2024-04-24T23:06:31.933746+0000","last_undegraded":"2024-04-24T23:06:31.933746+0000","last_fullsized":"2024-04-24T23:06:31.933746+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:33:56.873969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.951513+0000","last_change":"2024-04-24T23:06:31.948644+0000","last_active":"2024-04-24T23:06:31.951513+0000","last_peered":"2024-04-24T23:06:31.951513+0000","last_clean":"2024-04-24T23:06:31.951513+0000","last_became_active":"2024-04-24T23:06:31.947352+0000","last_became_peered":"2024-04-24T23:06:31.947352+0000","last_unstale":"2024-04-24T23:06:31.951513+0000","last_undegraded":"2024-04-24T23:06:31.951513+0000","last_fullsized":"2024-04-24T23:06:31.951513+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:36:45.428256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.10","version":"155'8","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.965701+0000","last_change":"2024-04-24T23:06:30.934955+0000","last_active":"2024-04-24T23:06:31.965701+0000","last_peered":"2024-04-24T23:06:31.965701+0000","last_clean":"2024-04-24T23:06:31.965701+0000","last_became_active":"2024-04-24T23:06:30.934676+0000","last_became_peered":"2024-04-24T23:06:30.934676+0000","last_unstale":"2024-04-24T23:06:31.965701+0000","last_undegraded":"2024-04-24T23:06:31.965701+0000","last_fullsized":"2024-04-24T23:06:31.965701+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:54.015309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.948106+0000","last_change":"2024-04-24T23:06:31.946127+0000","last_active":"2024-04-24T23:06:31.948106+0000","last_peered":"2024-04-24T23:06:31.948106+0000","last_clean":"2024-04-24T23:06:31.948106+0000","last_became_active":"2024-04-24T23:06:31.945872+0000","last_became_peered":"2024-04-24T23:06:31.945872+0000","last_unstale":"2024-04-24T23:06:31.948106+0000","last_undegraded":"2024-04-24T23:06:31.948106+0000","last_fullsized":"2024-04-24T23:06:31.948106+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:59:47.230611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.f","version":"155'1","reported_seq":13,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.963852+0000","last_change":"2024-04-24T23:06:30.937833+0000","last_active":"2024-04-24T23:06:31.963852+0000","last_peered":"2024-04-24T23:06:31.963852+0000","last_clean":"2024-04-24T23:06:31.963852+0000","last_became_active":"2024-04-24T23:06:30.937440+0000","last_became_peered":"2024-04-24T23:06:30.937440+0000","last_unstale":"2024-04-24T23:06:31.963852+0000","last_undegraded":"2024-04-24T23:06:31.963852+0000","last_fullsized":"2024-04-24T23:06:31.963852+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:59.062468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.942414+0000","last_change":"2024-04-24T23:06:31.942215+0000","last_active":"2024-04-24T23:06:31.942414+0000","last_peered":"2024-04-24T23:06:31.942414+0000","last_clean":"2024-04-24T23:06:31.942414+0000","last_became_active":"2024-04-24T23:06:31.941952+0000","last_became_peered":"2024-04-24T23:06:31.941952+0000","last_unstale":"2024-04-24T23:06:31.942414+0000","last_undegraded":"2024-04-24T23:06:31.942414+0000","last_fullsized":"2024-04-24T23:06:31.942414+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:33:01.134266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"155'2","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.963508+0000","last_change":"2024-04-24T23:06:30.932587+0000","last_active":"2024-04-24T23:06:31.963508+0000","last_peered":"2024-04-24T23:06:31.963508+0000","last_clean":"2024-04-24T23:06:31.963508+0000","last_became_active":"2024-04-24T23:06:30.931956+0000","last_became_peered":"2024-04-24T23:06:30.931956+0000","last_unstale":"2024-04-24T23:06:31.963508+0000","last_undegraded":"2024-04-24T23:06:31.963508+0000","last_fullsized":"2024-04-24T23:06:31.963508+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:16:54.161845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.952811+0000","last_change":"2024-04-24T23:06:31.952633+0000","last_active":"2024-04-24T23:06:31.952811+0000","last_peered":"2024-04-24T23:06:31.952811+0000","last_clean":"2024-04-24T23:06:31.952811+0000","last_became_active":"2024-04-24T23:06:31.952328+0000","last_became_peered":"2024-04-24T23:06:31.952328+0000","last_unstale":"2024-04-24T23:06:31.952811+0000","last_undegraded":"2024-04-24T23:06:31.952811+0000","last_fullsized":"2024-04-24T23:06:31.952811+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:38:59.660215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"155'1","reported_seq":13,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.964484+0000","last_change":"2024-04-24T23:06:30.932688+0000","last_active":"2024-04-24T23:06:31.964484+0000","last_peered":"2024-04-24T23:06:31.964484+0000","last_clean":"2024-04-24T23:06:31.964484+0000","last_became_active":"2024-04-24T23:06:30.932045+0000","last_became_peered":"2024-04-24T23:06:30.932045+0000","last_unstale":"2024-04-24T23:06:31.964484+0000","last_undegraded":"2024-04-24T23:06:31.964484+0000","last_fullsized":"2024-04-24T23:06:31.964484+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:14.603813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.947253+0000","last_change":"2024-04-24T23:06:31.944546+0000","last_active":"2024-04-24T23:06:31.947253+0000","last_peered":"2024-04-24T23:06:31.947253+0000","last_clean":"2024-04-24T23:06:31.947253+0000","last_became_active":"2024-04-24T23:06:31.944296+0000","last_became_peered":"2024-04-24T23:06:31.944296+0000","last_unstale":"2024-04-24T23:06:31.947253+0000","last_undegraded":"2024-04-24T23:06:31.947253+0000","last_fullsized":"2024-04-24T23:06:31.947253+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:47:36.471230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.941735+0000","last_change":"2024-04-24T23:06:30.937951+0000","last_active":"2024-04-24T23:06:31.941735+0000","last_peered":"2024-04-24T23:06:31.941735+0000","last_clean":"2024-04-24T23:06:31.941735+0000","last_became_active":"2024-04-24T23:06:30.937694+0000","last_became_peered":"2024-04-24T23:06:30.937694+0000","last_unstale":"2024-04-24T23:06:31.941735+0000","last_undegraded":"2024-04-24T23:06:31.941735+0000","last_fullsized":"2024-04-24T23:06:31.941735+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:18:58.007323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.947453+0000","last_change":"2024-04-24T23:06:31.947359+0000","last_active":"2024-04-24T23:06:31.947453+0000","last_peered":"2024-04-24T23:06:31.947453+0000","last_clean":"2024-04-24T23:06:31.947453+0000","last_became_active":"2024-04-24T23:06:31.946957+0000","last_became_peered":"2024-04-24T23:06:31.946957+0000","last_unstale":"2024-04-24T23:06:31.947453+0000","last_undegraded":"2024-04-24T23:06:31.947453+0000","last_fullsized":"2024-04-24T23:06:31.947453+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:32:35.998113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"155'2","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.965067+0000","last_change":"2024-04-24T23:06:30.938258+0000","last_active":"2024-04-24T23:06:31.965067+0000","last_peered":"2024-04-24T23:06:31.965067+0000","last_clean":"2024-04-24T23:06:31.965067+0000","last_became_active":"2024-04-24T23:06:30.937310+0000","last_became_peered":"2024-04-24T23:06:30.937310+0000","last_unstale":"2024-04-24T23:06:31.965067+0000","last_undegraded":"2024-04-24T23:06:31.965067+0000","last_fullsized":"2024-04-24T23:06:31.965067+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:14:52.090177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.959807+0000","last_change":"2024-04-24T23:06:31.959041+0000","last_active":"2024-04-24T23:06:31.959807+0000","last_peered":"2024-04-24T23:06:31.959807+0000","last_clean":"2024-04-24T23:06:31.959807+0000","last_became_active":"2024-04-24T23:06:31.958763+0000","last_became_peered":"2024-04-24T23:06:31.958763+0000","last_unstale":"2024-04-24T23:06:31.959807+0000","last_undegraded":"2024-04-24T23:06:31.959807+0000","last_fullsized":"2024-04-24T23:06:31.959807+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:26:44.239233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.950410+0000","last_change":"2024-04-24T23:06:30.932619+0000","last_active":"2024-04-24T23:06:31.950410+0000","last_peered":"2024-04-24T23:06:31.950410+0000","last_clean":"2024-04-24T23:06:31.950410+0000","last_became_active":"2024-04-24T23:06:30.932391+0000","last_became_peered":"2024-04-24T23:06:30.932391+0000","last_unstale":"2024-04-24T23:06:31.950410+0000","last_undegraded":"2024-04-24T23:06:31.950410+0000","last_fullsized":"2024-04-24T23:06:31.950410+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:01:43.325055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.947296+0000","last_change":"2024-04-24T23:06:31.946711+0000","last_active":"2024-04-24T23:06:31.947296+0000","last_peered":"2024-04-24T23:06:31.947296+0000","last_clean":"2024-04-24T23:06:31.947296+0000","last_became_active":"2024-04-24T23:06:31.946473+0000","last_became_peered":"2024-04-24T23:06:31.946473+0000","last_unstale":"2024-04-24T23:06:31.947296+0000","last_undegraded":"2024-04-24T23:06:31.947296+0000","last_fullsized":"2024-04-24T23:06:31.947296+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:07:17.716644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.942596+0000","last_change":"2024-04-24T23:06:30.933492+0000","last_active":"2024-04-24T23:06:31.942596+0000","last_peered":"2024-04-24T23:06:31.942596+0000","last_clean":"2024-04-24T23:06:31.942596+0000","last_became_active":"2024-04-24T23:06:30.933064+0000","last_became_peered":"2024-04-24T23:06:30.933064+0000","last_unstale":"2024-04-24T23:06:31.942596+0000","last_undegraded":"2024-04-24T23:06:31.942596+0000","last_fullsized":"2024-04-24T23:06:31.942596+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:10:25.036860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.951492+0000","last_change":"2024-04-24T23:06:31.948427+0000","last_active":"2024-04-24T23:06:31.951492+0000","last_peered":"2024-04-24T23:06:31.951492+0000","last_clean":"2024-04-24T23:06:31.951492+0000","last_became_active":"2024-04-24T23:06:31.948013+0000","last_became_peered":"2024-04-24T23:06:31.948013+0000","last_unstale":"2024-04-24T23:06:31.951492+0000","last_undegraded":"2024-04-24T23:06:31.951492+0000","last_fullsized":"2024-04-24T23:06:31.951492+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:19:03.084992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.934213+0000","last_change":"2024-04-24T23:06:30.935259+0000","last_active":"2024-04-24T23:06:31.934213+0000","last_peered":"2024-04-24T23:06:31.934213+0000","last_clean":"2024-04-24T23:06:31.934213+0000","last_became_active":"2024-04-24T23:06:30.934988+0000","last_became_peered":"2024-04-24T23:06:30.934988+0000","last_unstale":"2024-04-24T23:06:31.934213+0000","last_undegraded":"2024-04-24T23:06:31.934213+0000","last_fullsized":"2024-04-24T23:06:31.934213+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:18:26.082861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.954120+0000","last_change":"2024-04-24T23:06:31.953108+0000","last_active":"2024-04-24T23:06:31.954120+0000","last_peered":"2024-04-24T23:06:31.954120+0000","last_clean":"2024-04-24T23:06:31.954120+0000","last_became_active":"2024-04-24T23:06:31.952200+0000","last_became_peered":"2024-04-24T23:06:31.952200+0000","last_unstale":"2024-04-24T23:06:31.954120+0000","last_undegraded":"2024-04-24T23:06:31.954120+0000","last_fullsized":"2024-04-24T23:06:31.954120+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:48:41.214953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.7","version":"155'2","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.963409+0000","last_change":"2024-04-24T23:06:30.933292+0000","last_active":"2024-04-24T23:06:31.963409+0000","last_peered":"2024-04-24T23:06:31.963409+0000","last_clean":"2024-04-24T23:06:31.963409+0000","last_became_active":"2024-04-24T23:06:30.933071+0000","last_became_peered":"2024-04-24T23:06:30.933071+0000","last_unstale":"2024-04-24T23:06:31.963409+0000","last_undegraded":"2024-04-24T23:06:31.963409+0000","last_fullsized":"2024-04-24T23:06:31.963409+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:25:58.333171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.954383+0000","last_change":"2024-04-24T23:06:31.954265+0000","last_active":"2024-04-24T23:06:31.954383+0000","last_peered":"2024-04-24T23:06:31.954383+0000","last_clean":"2024-04-24T23:06:31.954383+0000","last_became_active":"2024-04-24T23:06:31.953898+0000","last_became_peered":"2024-04-24T23:06:31.953898+0000","last_unstale":"2024-04-24T23:06:31.954383+0000","last_undegraded":"2024-04-24T23:06:31.954383+0000","last_fullsized":"2024-04-24T23:06:31.954383+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:24.237447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.940746+0000","last_change":"2024-04-24T23:06:30.936290+0000","last_active":"2024-04-24T23:06:31.940746+0000","last_peered":"2024-04-24T23:06:31.940746+0000","last_clean":"2024-04-24T23:06:31.940746+0000","last_became_active":"2024-04-24T23:06:30.936049+0000","last_became_peered":"2024-04-24T23:06:30.936049+0000","last_unstale":"2024-04-24T23:06:31.940746+0000","last_undegraded":"2024-04-24T23:06:31.940746+0000","last_fullsized":"2024-04-24T23:06:31.940746+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:51:40.516708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.950171+0000","last_change":"2024-04-24T23:06:31.949930+0000","last_active":"2024-04-24T23:06:31.950171+0000","last_peered":"2024-04-24T23:06:31.950171+0000","last_clean":"2024-04-24T23:06:31.950171+0000","last_became_active":"2024-04-24T23:06:31.949636+0000","last_became_peered":"2024-04-24T23:06:31.949636+0000","last_unstale":"2024-04-24T23:06:31.950171+0000","last_undegraded":"2024-04-24T23:06:31.950171+0000","last_fullsized":"2024-04-24T23:06:31.950171+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:21:54.964154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.950551+0000","last_change":"2024-04-24T23:06:30.935624+0000","last_active":"2024-04-24T23:06:31.950551+0000","last_peered":"2024-04-24T23:06:31.950551+0000","last_clean":"2024-04-24T23:06:31.950551+0000","last_became_active":"2024-04-24T23:06:30.935387+0000","last_became_peered":"2024-04-24T23:06:30.935387+0000","last_unstale":"2024-04-24T23:06:31.950551+0000","last_undegraded":"2024-04-24T23:06:31.950551+0000","last_fullsized":"2024-04-24T23:06:31.950551+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:02.529511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.954682+0000","last_change":"2024-04-24T23:06:31.954556+0000","last_active":"2024-04-24T23:06:31.954682+0000","last_peered":"2024-04-24T23:06:31.954682+0000","last_clean":"2024-04-24T23:06:31.954682+0000","last_became_active":"2024-04-24T23:06:31.954074+0000","last_became_peered":"2024-04-24T23:06:31.954074+0000","last_unstale":"2024-04-24T23:06:31.954682+0000","last_undegraded":"2024-04-24T23:06:31.954682+0000","last_fullsized":"2024-04-24T23:06:31.954682+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:45:24.545531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.942756+0000","last_change":"2024-04-24T23:06:30.936106+0000","last_active":"2024-04-24T23:06:31.942756+0000","last_peered":"2024-04-24T23:06:31.942756+0000","last_clean":"2024-04-24T23:06:31.942756+0000","last_became_active":"2024-04-24T23:06:30.935856+0000","last_became_peered":"2024-04-24T23:06:30.935856+0000","last_unstale":"2024-04-24T23:06:31.942756+0000","last_undegraded":"2024-04-24T23:06:31.942756+0000","last_fullsized":"2024-04-24T23:06:31.942756+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:27.344320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.951591+0000","last_change":"2024-04-24T23:06:31.950936+0000","last_active":"2024-04-24T23:06:31.951591+0000","last_peered":"2024-04-24T23:06:31.951591+0000","last_clean":"2024-04-24T23:06:31.951591+0000","last_became_active":"2024-04-24T23:06:31.950659+0000","last_became_peered":"2024-04-24T23:06:31.950659+0000","last_unstale":"2024-04-24T23:06:31.951591+0000","last_undegraded":"2024-04-24T23:06:31.951591+0000","last_fullsized":"2024-04-24T23:06:31.951591+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:19.609533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.938973+0000","last_change":"2024-04-24T23:06:30.934663+0000","last_active":"2024-04-24T23:06:31.938973+0000","last_peered":"2024-04-24T23:06:31.938973+0000","last_clean":"2024-04-24T23:06:31.938973+0000","last_became_active":"2024-04-24T23:06:30.934266+0000","last_became_peered":"2024-04-24T23:06:30.934266+0000","last_unstale":"2024-04-24T23:06:31.938973+0000","last_undegraded":"2024-04-24T23:06:31.938973+0000","last_fullsized":"2024-04-24T23:06:31.938973+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:17:40.071951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.948032+0000","last_change":"2024-04-24T23:06:31.947912+0000","last_active":"2024-04-24T23:06:31.948032+0000","last_peered":"2024-04-24T23:06:31.948032+0000","last_clean":"2024-04-24T23:06:31.948032+0000","last_became_active":"2024-04-24T23:06:31.947626+0000","last_became_peered":"2024-04-24T23:06:31.947626+0000","last_unstale":"2024-04-24T23:06:31.948032+0000","last_undegraded":"2024-04-24T23:06:31.948032+0000","last_fullsized":"2024-04-24T23:06:31.948032+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:57:10.290493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.944557+0000","last_change":"2024-04-24T23:06:30.930687+0000","last_active":"2024-04-24T23:06:31.944557+0000","last_peered":"2024-04-24T23:06:31.944557+0000","last_clean":"2024-04-24T23:06:31.944557+0000","last_became_active":"2024-04-24T23:06:30.930397+0000","last_became_peered":"2024-04-24T23:06:30.930397+0000","last_unstale":"2024-04-24T23:06:31.944557+0000","last_undegraded":"2024-04-24T23:06:31.944557+0000","last_fullsized":"2024-04-24T23:06:31.944557+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:44:14.691545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.953737+0000","last_change":"2024-04-24T23:06:31.953569+0000","last_active":"2024-04-24T23:06:31.953737+0000","last_peered":"2024-04-24T23:06:31.953737+0000","last_clean":"2024-04-24T23:06:31.953737+0000","last_became_active":"2024-04-24T23:06:31.953288+0000","last_became_peered":"2024-04-24T23:06:31.953288+0000","last_unstale":"2024-04-24T23:06:31.953737+0000","last_undegraded":"2024-04-24T23:06:31.953737+0000","last_fullsized":"2024-04-24T23:06:31.953737+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:00:30.751504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.934224+0000","last_change":"2024-04-24T23:06:30.936327+0000","last_active":"2024-04-24T23:06:31.934224+0000","last_peered":"2024-04-24T23:06:31.934224+0000","last_clean":"2024-04-24T23:06:31.934224+0000","last_became_active":"2024-04-24T23:06:30.936101+0000","last_became_peered":"2024-04-24T23:06:30.936101+0000","last_unstale":"2024-04-24T23:06:31.934224+0000","last_undegraded":"2024-04-24T23:06:31.934224+0000","last_fullsized":"2024-04-24T23:06:31.934224+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:32:49.703476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.953081+0000","last_change":"2024-04-24T23:06:31.952973+0000","last_active":"2024-04-24T23:06:31.953081+0000","last_peered":"2024-04-24T23:06:31.953081+0000","last_clean":"2024-04-24T23:06:31.953081+0000","last_became_active":"2024-04-24T23:06:31.952686+0000","last_became_peered":"2024-04-24T23:06:31.952686+0000","last_unstale":"2024-04-24T23:06:31.953081+0000","last_undegraded":"2024-04-24T23:06:31.953081+0000","last_fullsized":"2024-04-24T23:06:31.953081+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:48:39.686094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.940501+0000","last_change":"2024-04-24T23:06:30.930903+0000","last_active":"2024-04-24T23:06:31.940501+0000","last_peered":"2024-04-24T23:06:31.940501+0000","last_clean":"2024-04-24T23:06:31.940501+0000","last_became_active":"2024-04-24T23:06:30.929827+0000","last_became_peered":"2024-04-24T23:06:30.929827+0000","last_unstale":"2024-04-24T23:06:31.940501+0000","last_undegraded":"2024-04-24T23:06:31.940501+0000","last_fullsized":"2024-04-24T23:06:31.940501+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:39:10.700191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.938693+0000","last_change":"2024-04-24T23:06:31.938600+0000","last_active":"2024-04-24T23:06:31.938693+0000","last_peered":"2024-04-24T23:06:31.938693+0000","last_clean":"2024-04-24T23:06:31.938693+0000","last_became_active":"2024-04-24T23:06:31.938437+0000","last_became_peered":"2024-04-24T23:06:31.938437+0000","last_unstale":"2024-04-24T23:06:31.938693+0000","last_undegraded":"2024-04-24T23:06:31.938693+0000","last_fullsized":"2024-04-24T23:06:31.938693+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:30.917649+0000","last_clean_scrub_stamp":"2024-04-24T23:06:30.917649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:17.686935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.13","version":"155'2","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.963961+0000","last_change":"2024-04-24T23:06:30.931011+0000","last_active":"2024-04-24T23:06:31.963961+0000","last_peered":"2024-04-24T23:06:31.963961+0000","last_clean":"2024-04-24T23:06:31.963961+0000","last_became_active":"2024-04-24T23:06:30.930694+0000","last_became_peered":"2024-04-24T23:06:30.930694+0000","last_unstale":"2024-04-24T23:06:31.963961+0000","last_undegraded":"2024-04-24T23:06:31.963961+0000","last_fullsized":"2024-04-24T23:06:31.963961+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:29.913544+0000","last_clean_scrub_stamp":"2024-04-24T23:06:29.913544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:47.503208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":435,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-24T23:06:31.942268+0000","last_change":"2024-04-24T22:58:55.138810+0000","last_active":"2024-04-24T23:06:31.942268+0000","last_peered":"2024-04-24T23:06:31.942268+0000","last_clean":"2024-04-24T23:06:31.942268+0000","last_became_active":"2024-04-24T22:58:55.138627+0000","last_became_peered":"2024-04-24T22:58:55.138627+0000","last_unstale":"2024-04-24T23:06:31.942268+0000","last_undegraded":"2024-04-24T23:06:31.942268+0000","last_fullsized":"2024-04-24T23:06:31.942268+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_clean_scrub_stamp":"2024-04-24T22:58:54.127235+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:51.383182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574942,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":839,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194473,"statfs":{"total":200000000,"available":199140432,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574942,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":839,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194473,"statfs":{"total":200000000,"available":199140432,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574942,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1415,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193896,"statfs":{"total":200000000,"available":198550064,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574942,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":839,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194473,"statfs":{"total":200000000,"available":199140432,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574942,"num_pgs":16,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":839,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194473,"statfs":{"total":200000000,"available":199140432,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574942,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":839,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194473,"statfs":{"total":200000000,"available":199140432,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1424,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193888,"statfs":{"total":200000000,"available":198541655,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574942,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":847,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194464,"statfs":{"total":200000000,"available":199131989,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-24T23:06:35.300 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-24T23:06:35.300 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-24T23:06:35.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.443+0000 7fa545812640 1 Processor -- start 2024-04-24T23:06:35.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.443+0000 7fa545812640 1 -- start start 2024-04-24T23:06:35.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.444+0000 7fa545812640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa540162e10 0x7fa5400ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:35.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.444+0000 7fa545812640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5400cd270 con 0x7fa540162320 2024-04-24T23:06:35.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.444+0000 7fa545812640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa540071610 con 0x7fa5400b5af0 2024-04-24T23:06:35.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.444+0000 7fa545812640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fa5400b5660 con 0x7fa540162e10 2024-04-24T23:06:35.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.444+0000 7fa53f7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa540162e10 0x7fa5400ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:35.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.444+0000 7fa53effd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa540162320 0x7fa540162700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33126/0 (socket says 172.21.15.32:33126) 2024-04-24T23:06:35.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.444+0000 7fa53effd640 1 -- 172.21.15.32:0/1983447370 learned_addr learned my addr 172.21.15.32:0/1983447370 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:06:35.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.444+0000 7fa53f7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa540162e10 0x7fa5400ccc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48538/0 (socket says 172.21.15.32:48538) 2024-04-24T23:06:35.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.444+0000 7fa53f7fe640 1 -- 172.21.15.32:0/1983447370 >> v1:172.21.15.121:6789/0 conn(0x7fa5400b5af0 legacy=0x7fa54015ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:35.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.444+0000 7fa53f7fe640 1 -- 172.21.15.32:0/1983447370 >> v1:172.21.15.32:6789/0 conn(0x7fa540162320 legacy=0x7fa540162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:35.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.444+0000 7fa53dffb640 1 -- 172.21.15.32:0/1983447370 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1801493508 0 0) 0x7fa5400cd270 con 0x7fa540162320 2024-04-24T23:06:35.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.444+0000 7fa53f7fe640 1 -- 172.21.15.32:0/1983447370 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5400718f0 con 0x7fa540162e10 2024-04-24T23:06:35.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.445+0000 7fa53f7fe640 1 --2- 172.21.15.32:0/1983447370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa540162e10 0x7fa5400ccc90 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fa530009d90 tx=0x7fa53002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c89b0e56552b851e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:35.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.445+0000 7fa53dffb640 1 -- 172.21.15.32:0/1983447370 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa530007740 con 0x7fa540162e10 2024-04-24T23:06:35.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.445+0000 7fa53dffb640 1 -- 172.21.15.32:0/1983447370 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5300078e0 con 0x7fa540162e10 2024-04-24T23:06:35.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.445+0000 7fa53dffb640 1 -- 172.21.15.32:0/1983447370 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa530007cb0 con 0x7fa540162e10 2024-04-24T23:06:35.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.446+0000 7fa545812640 1 -- 172.21.15.32:0/1983447370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa540162e10 msgr2=0x7fa5400ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:35.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.446+0000 7fa545812640 1 --2- 172.21.15.32:0/1983447370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa540162e10 0x7fa5400ccc90 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fa530009d90 tx=0x7fa53002a660 comp rx=0 tx=0).stop 2024-04-24T23:06:35.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.446+0000 7fa545812640 1 -- 172.21.15.32:0/1983447370 shutdown_connections 2024-04-24T23:06:35.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.446+0000 7fa545812640 1 --2- 172.21.15.32:0/1983447370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa540162e10 0x7fa5400ccc90 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:35.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.446+0000 7fa545812640 1 -- 172.21.15.32:0/1983447370 >> 172.21.15.32:0/1983447370 conn(0x7fa5400d0ae0 msgr2=0x7fa5400d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:35.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.446+0000 7fa545812640 1 -- 172.21.15.32:0/1983447370 shutdown_connections 2024-04-24T23:06:35.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.446+0000 7fa545812640 1 -- 172.21.15.32:0/1983447370 wait complete. 2024-04-24T23:06:35.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.447+0000 7fa545812640 1 Processor -- start 2024-04-24T23:06:35.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.447+0000 7fa545812640 1 -- start start 2024-04-24T23:06:35.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.448+0000 7fa545812640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5400b5af0 0x7fa54014e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:35.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.448+0000 7fa545812640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa540162320 0x7fa5400cefd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:35.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.448+0000 7fa545812640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa540162e10 0x7fa5400cf510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:35.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.448+0000 7fa545812640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa540071bd0 con 0x7fa540162320 2024-04-24T23:06:35.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.448+0000 7fa545812640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa540067950 con 0x7fa5400b5af0 2024-04-24T23:06:35.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.448+0000 7fa545812640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa540071d50 con 0x7fa540162e10 2024-04-24T23:06:35.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.448+0000 7fa53e7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa540162320 0x7fa5400cefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:35.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.448+0000 7fa53e7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa540162320 0x7fa5400cefd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48548/0 (socket says 172.21.15.32:48548) 2024-04-24T23:06:35.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.448+0000 7fa53e7fc640 1 -- 172.21.15.32:0/1619380360 learned_addr learned my addr 172.21.15.32:0/1619380360 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:35.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.448+0000 7fa53f7fe640 1 --2- 172.21.15.32:0/1619380360 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa540162e10 0x7fa5400cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:35.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.448+0000 7fa53effd640 1 --2- 172.21.15.32:0/1619380360 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5400b5af0 0x7fa54014e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:35.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.448+0000 7fa53e7fc640 1 -- 172.21.15.32:0/1619380360 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa540162e10 msgr2=0x7fa5400cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:35.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.448+0000 7fa53e7fc640 1 --2- 172.21.15.32:0/1619380360 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa540162e10 0x7fa5400cf510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:35.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.448+0000 7fa53e7fc640 1 -- 172.21.15.32:0/1619380360 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5400b5af0 msgr2=0x7fa54014e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:35.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.448+0000 7fa53e7fc640 1 --2- 172.21.15.32:0/1619380360 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5400b5af0 0x7fa54014e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:35.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.448+0000 7fa53e7fc640 1 -- 172.21.15.32:0/1619380360 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5400cd590 con 0x7fa540162320 2024-04-24T23:06:35.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.448+0000 7fa53effd640 1 --2- 172.21.15.32:0/1619380360 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5400b5af0 0x7fa54014e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:06:35.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.448+0000 7fa53e7fc640 1 --2- 172.21.15.32:0/1619380360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa540162320 0x7fa5400cefd0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fa52800c450 tx=0x7fa52800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:35.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.449+0000 7fa537fff640 1 -- 172.21.15.32:0/1619380360 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5280052d0 con 0x7fa540162320 2024-04-24T23:06:35.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.449+0000 7fa537fff640 1 -- 172.21.15.32:0/1619380360 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa528005470 con 0x7fa540162320 2024-04-24T23:06:35.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.449+0000 7fa545812640 1 -- 172.21.15.32:0/1619380360 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5400cd870 con 0x7fa540162320 2024-04-24T23:06:35.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.449+0000 7fa537fff640 1 -- 172.21.15.32:0/1619380360 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa528012870 con 0x7fa540162320 2024-04-24T23:06:35.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.449+0000 7fa545812640 1 -- 172.21.15.32:0/1619380360 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5400cde30 con 0x7fa540162320 2024-04-24T23:06:35.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.450+0000 7fa545812640 1 -- 172.21.15.32:0/1619380360 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa540068430 con 0x7fa540162320 2024-04-24T23:06:35.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.450+0000 7fa537fff640 1 -- 172.21.15.32:0/1619380360 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa528012a10 con 0x7fa540162320 2024-04-24T23:06:35.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.450+0000 7fa537fff640 1 --2- 172.21.15.32:0/1619380360 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa52006be30 0x7fa52006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:35.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.451+0000 7fa537fff640 1 -- 172.21.15.32:0/1619380360 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7fa52808aa00 con 0x7fa540162320 2024-04-24T23:06:35.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.451+0000 7fa53effd640 1 --2- 172.21.15.32:0/1619380360 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa52006be30 0x7fa52006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:35.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.451+0000 7fa53effd640 1 --2- 172.21.15.32:0/1619380360 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa52006be30 0x7fa52006e2f0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fa538000f60 tx=0x7fa538009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:35.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.455+0000 7fa537fff640 1 -- 172.21.15.32:0/1619380360 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa528059440 con 0x7fa540162320 2024-04-24T23:06:35.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.708+0000 7fa545812640 1 -- 172.21.15.32:0/1619380360 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fa5400d0850 con 0x7fa540162320 2024-04-24T23:06:35.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.710+0000 7fa537fff640 1 -- 172.21.15.32:0/1619380360 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) ==== 190+0+147 (secure 0 0 0) 0x7fa528058d40 con 0x7fa540162320 2024-04-24T23:06:35.711 INFO:teuthology.orchestra.run.smithi032.stdout:[client.0] 2024-04-24T23:06:35.711 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQCejilm5oeiGRAAdLXwwqrYhV3HHo7Dw/2ZqQ== 2024-04-24T23:06:35.711 INFO:teuthology.orchestra.run.smithi032.stdout: caps mds = "allow" 2024-04-24T23:06:35.711 INFO:teuthology.orchestra.run.smithi032.stdout: caps mon = "allow r" 2024-04-24T23:06:35.711 INFO:teuthology.orchestra.run.smithi032.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-24T23:06:35.712 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2024-04-24T23:06:35.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.712+0000 7fa545812640 1 -- 172.21.15.32:0/1619380360 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa52006be30 msgr2=0x7fa52006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:35.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.712+0000 7fa545812640 1 --2- 172.21.15.32:0/1619380360 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa52006be30 0x7fa52006e2f0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fa538000f60 tx=0x7fa538009040 comp rx=0 tx=0).stop 2024-04-24T23:06:35.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.712+0000 7fa545812640 1 -- 172.21.15.32:0/1619380360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa540162320 msgr2=0x7fa5400cefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:35.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.712+0000 7fa545812640 1 --2- 172.21.15.32:0/1619380360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa540162320 0x7fa5400cefd0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fa52800c450 tx=0x7fa52800c910 comp rx=0 tx=0).stop 2024-04-24T23:06:35.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.712+0000 7fa545812640 1 -- 172.21.15.32:0/1619380360 shutdown_connections 2024-04-24T23:06:35.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.712+0000 7fa545812640 1 --2- 172.21.15.32:0/1619380360 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa52006be30 0x7fa52006e2f0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:35.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.712+0000 7fa545812640 1 --2- 172.21.15.32:0/1619380360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa540162320 0x7fa5400cefd0 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:35.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.712+0000 7fa545812640 1 --2- 172.21.15.32:0/1619380360 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa540162e10 0x7fa5400cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:35.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.712+0000 7fa545812640 1 --2- 172.21.15.32:0/1619380360 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5400b5af0 0x7fa54014e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:35.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.712+0000 7fa545812640 1 -- 172.21.15.32:0/1619380360 >> 172.21.15.32:0/1619380360 conn(0x7fa5400d0ae0 msgr2=0x7fa5400b43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:35.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.712+0000 7fa545812640 1 -- 172.21.15.32:0/1619380360 shutdown_connections 2024-04-24T23:06:35.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.712+0000 7fa545812640 1 -- 172.21.15.32:0/1619380360 wait complete. 2024-04-24T23:06:35.724 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:06:35.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.823+0000 7f30da6ba640 1 Processor -- start 2024-04-24T23:06:35.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.823+0000 7f30da6ba640 1 -- start start 2024-04-24T23:06:35.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.824+0000 7f30da6ba640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f30d4158dd0 0x7f30d406b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:35.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.824+0000 7f30da6ba640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f30d406dc60 0x7f30d40b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:35.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.824+0000 7f30da6ba640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30d406bd10 con 0x7f30d406b860 2024-04-24T23:06:35.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.824+0000 7f30da6ba640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f30d415ef10 con 0x7f30d4158dd0 2024-04-24T23:06:35.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.824+0000 7f30da6ba640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f30d415f1f0 con 0x7f30d406dc60 2024-04-24T23:06:35.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.824+0000 7f30d3fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f30d4158dd0 0x7f30d406b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:35.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.824+0000 7f30d3fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f30d4158dd0 0x7f30d406b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48558/0 (socket says 172.21.15.32:48558) 2024-04-24T23:06:35.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.824+0000 7f30d3fff640 1 -- 172.21.15.32:0/3951995550 learned_addr learned my addr 172.21.15.32:0/3951995550 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:35.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.824+0000 7f30d8c57640 1 --2- 172.21.15.32:0/3951995550 >> v2:172.21.15.121:3300/0 conn(0x7f30d406dc60 0x7f30d40b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:35.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.825+0000 7f30d3fff640 1 -- 172.21.15.32:0/3951995550 >> v2:172.21.15.121:3300/0 conn(0x7f30d406dc60 msgr2=0x7f30d40b5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:35.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.825+0000 7f30d3fff640 1 --2- 172.21.15.32:0/3951995550 >> v2:172.21.15.121:3300/0 conn(0x7f30d406dc60 0x7f30d40b5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:35.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.825+0000 7f30d3fff640 1 -- 172.21.15.32:0/3951995550 >> v1:172.21.15.121:6789/0 conn(0x7f30d406b860 legacy=0x7f30d406d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:35.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.825+0000 7f30d3fff640 1 -- 172.21.15.32:0/3951995550 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30d415f4d0 con 0x7f30d4158dd0 2024-04-24T23:06:35.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.825+0000 7f30d3fff640 1 --2- 172.21.15.32:0/3951995550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30d4158dd0 0x7f30d406b320 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f30d40b2170 tx=0x7f30bc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21966bc62ea21efc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:35.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.825+0000 7f30d2ffd640 1 -- 172.21.15.32:0/3951995550 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30bc037070 con 0x7f30d4158dd0 2024-04-24T23:06:35.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.826+0000 7f30d2ffd640 1 -- 172.21.15.32:0/3951995550 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30bc032440 con 0x7f30d4158dd0 2024-04-24T23:06:35.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.826+0000 7f30d2ffd640 1 -- 172.21.15.32:0/3951995550 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30bc032810 con 0x7f30d4158dd0 2024-04-24T23:06:35.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.826+0000 7f30da6ba640 1 -- 172.21.15.32:0/3951995550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30d4158dd0 msgr2=0x7f30d406b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:35.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.826+0000 7f30da6ba640 1 --2- 172.21.15.32:0/3951995550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30d4158dd0 0x7f30d406b320 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f30d40b2170 tx=0x7f30bc02a230 comp rx=0 tx=0).stop 2024-04-24T23:06:35.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.826+0000 7f30da6ba640 1 -- 172.21.15.32:0/3951995550 shutdown_connections 2024-04-24T23:06:35.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.826+0000 7f30da6ba640 1 --2- 172.21.15.32:0/3951995550 >> v2:172.21.15.121:3300/0 conn(0x7f30d406dc60 0x7f30d40b5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:35.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.826+0000 7f30da6ba640 1 --2- 172.21.15.32:0/3951995550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30d4158dd0 0x7f30d406b320 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:35.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.826+0000 7f30da6ba640 1 -- 172.21.15.32:0/3951995550 >> 172.21.15.32:0/3951995550 conn(0x7f30d406e800 msgr2=0x7f30d406ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:35.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.826+0000 7f30da6ba640 1 -- 172.21.15.32:0/3951995550 shutdown_connections 2024-04-24T23:06:35.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.826+0000 7f30da6ba640 1 -- 172.21.15.32:0/3951995550 wait complete. 2024-04-24T23:06:35.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.827+0000 7f30da6ba640 1 Processor -- start 2024-04-24T23:06:35.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.827+0000 7f30da6ba640 1 -- start start 2024-04-24T23:06:35.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.828+0000 7f30da6ba640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f30d406b860 0x7f30d4144200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:35.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.828+0000 7f30da6ba640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f30d406dc60 0x7f30d4139c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:35.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.828+0000 7f30da6ba640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30d4158dd0 0x7f30d41496d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:35.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.828+0000 7f30da6ba640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f30d406bfe0 con 0x7f30d4158dd0 2024-04-24T23:06:35.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.828+0000 7f30da6ba640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f30d415fcd0 con 0x7f30d406b860 2024-04-24T23:06:35.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.828+0000 7f30da6ba640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f30d41591b0 con 0x7f30d406dc60 2024-04-24T23:06:35.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.828+0000 7f30d8c57640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30d4158dd0 0x7f30d41496d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:35.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.828+0000 7f30d8c57640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30d4158dd0 0x7f30d41496d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48568/0 (socket says 172.21.15.32:48568) 2024-04-24T23:06:35.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.828+0000 7f30d8c57640 1 -- 172.21.15.32:0/3445146997 learned_addr learned my addr 172.21.15.32:0/3445146997 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:35.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.828+0000 7f30d3fff640 1 --2- 172.21.15.32:0/3445146997 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f30d406b860 0x7f30d4144200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:35.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.828+0000 7f30d37fe640 1 --2- 172.21.15.32:0/3445146997 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f30d406dc60 0x7f30d4139c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:35.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.829+0000 7f30d8c57640 1 -- 172.21.15.32:0/3445146997 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f30d406dc60 msgr2=0x7f30d4139c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:35.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.829+0000 7f30d8c57640 1 --2- 172.21.15.32:0/3445146997 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f30d406dc60 0x7f30d4139c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:35.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.829+0000 7f30d8c57640 1 -- 172.21.15.32:0/3445146997 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f30d406b860 msgr2=0x7f30d4144200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:35.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.829+0000 7f30d8c57640 1 --2- 172.21.15.32:0/3445146997 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f30d406b860 0x7f30d4144200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:35.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.829+0000 7f30d8c57640 1 -- 172.21.15.32:0/3445146997 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30d4148630 con 0x7f30d4158dd0 2024-04-24T23:06:35.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.829+0000 7f30d8c57640 1 --2- 172.21.15.32:0/3445146997 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30d4158dd0 0x7f30d41496d0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f30cc007e40 tx=0x7f30cc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:35.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.829+0000 7f30d0ff9640 1 -- 172.21.15.32:0/3445146997 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30cc014620 con 0x7f30d4158dd0 2024-04-24T23:06:35.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.829+0000 7f30d0ff9640 1 -- 172.21.15.32:0/3445146997 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30cc019070 con 0x7f30d4158dd0 2024-04-24T23:06:35.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.829+0000 7f30da6ba640 1 -- 172.21.15.32:0/3445146997 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30d4148910 con 0x7f30d4158dd0 2024-04-24T23:06:35.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.829+0000 7f30d0ff9640 1 -- 172.21.15.32:0/3445146997 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30cc014a20 con 0x7f30d4158dd0 2024-04-24T23:06:35.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.829+0000 7f30da6ba640 1 -- 172.21.15.32:0/3445146997 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30d4148ed0 con 0x7f30d4158dd0 2024-04-24T23:06:35.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.830+0000 7f30da6ba640 1 -- 172.21.15.32:0/3445146997 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30d4068430 con 0x7f30d4158dd0 2024-04-24T23:06:35.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.830+0000 7f30d0ff9640 1 -- 172.21.15.32:0/3445146997 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f30cc014bc0 con 0x7f30d4158dd0 2024-04-24T23:06:35.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.831+0000 7f30d0ff9640 1 --2- 172.21.15.32:0/3445146997 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f30b406be30 0x7f30b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:35.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.831+0000 7f30d0ff9640 1 -- 172.21.15.32:0/3445146997 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7f30cc08b620 con 0x7f30d4158dd0 2024-04-24T23:06:35.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.831+0000 7f30d3fff640 1 --2- 172.21.15.32:0/3445146997 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f30b406be30 0x7f30b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:35.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.832+0000 7f30d3fff640 1 --2- 172.21.15.32:0/3445146997 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f30b406be30 0x7f30b406e2f0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f30bc034000 tx=0x7f30bc02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:35.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:35.835+0000 7f30d0ff9640 1 -- 172.21.15.32:0/3445146997 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f30cc059980 con 0x7f30d4158dd0 2024-04-24T23:06:36.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.056+0000 7f30da6ba640 1 -- 172.21.15.32:0/3445146997 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30d414e1b0 con 0x7f30d4158dd0 2024-04-24T23:06:36.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.056+0000 7f30d0ff9640 1 -- 172.21.15.32:0/3445146997 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+5186 (secure 0 0 0) 0x7f30cc05dc80 con 0x7f30d4158dd0 2024-04-24T23:06:36.058 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:06:36.058 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":65,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7120,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/1196305174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1196305174},{"type":"v1","addr":"172.21.15.121:6836","nonce":1196305174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7123,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3015902884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3015902884},{"type":"v1","addr":"172.21.15.121:6837","nonce":3015902884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":8280,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/1033273628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1033273628},{"type":"v1","addr":"172.21.15.32:6833","nonce":1033273628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:06:31.291599+0000","modified":"2024-04-24T23:06:33.296657+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8286},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8286":{"gid":8286,"name":"a","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/984603360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":984603360},{"type":"v1","addr":"172.21.15.32:6835","nonce":984603360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8286,"qdb_cluster":[8286]},"id":9}]} 2024-04-24T23:06:36.058 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 65 2024-04-24T23:06:36.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.058+0000 7f30da6ba640 1 -- 172.21.15.32:0/3445146997 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f30b406be30 msgr2=0x7f30b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:36.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.058+0000 7f30da6ba640 1 --2- 172.21.15.32:0/3445146997 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f30b406be30 0x7f30b406e2f0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f30bc034000 tx=0x7f30bc02e000 comp rx=0 tx=0).stop 2024-04-24T23:06:36.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.058+0000 7f30da6ba640 1 -- 172.21.15.32:0/3445146997 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30d4158dd0 msgr2=0x7f30d41496d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:36.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.058+0000 7f30da6ba640 1 --2- 172.21.15.32:0/3445146997 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30d4158dd0 0x7f30d41496d0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f30cc007e40 tx=0x7f30cc00e4b0 comp rx=0 tx=0).stop 2024-04-24T23:06:36.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.058+0000 7f30da6ba640 1 -- 172.21.15.32:0/3445146997 shutdown_connections 2024-04-24T23:06:36.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.058+0000 7f30da6ba640 1 --2- 172.21.15.32:0/3445146997 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f30b406be30 0x7f30b406e2f0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:36.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.058+0000 7f30da6ba640 1 --2- 172.21.15.32:0/3445146997 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30d4158dd0 0x7f30d41496d0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:36.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.058+0000 7f30da6ba640 1 --2- 172.21.15.32:0/3445146997 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f30d406dc60 0x7f30d4139c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:36.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.058+0000 7f30da6ba640 1 --2- 172.21.15.32:0/3445146997 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f30d406b860 0x7f30d4144200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:36.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.058+0000 7f30da6ba640 1 -- 172.21.15.32:0/3445146997 >> 172.21.15.32:0/3445146997 conn(0x7f30d406e800 msgr2=0x7f30d41556c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:36.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.058+0000 7f30da6ba640 1 -- 172.21.15.32:0/3445146997 shutdown_connections 2024-04-24T23:06:36.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.058+0000 7f30da6ba640 1 -- 172.21.15.32:0/3445146997 wait complete. 2024-04-24T23:06:36.474 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 65, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T23:06:31.291599+0000', 'modified': '2024-04-24T23:06:33.296657+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8286}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8286': {'gid': 8286, 'name': 'a', 'rank': 0, 'incarnation': 63, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/984603360', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 984603360}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 984603360}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8286, 'qdb_cluster': [8286]} 2024-04-24T23:06:36.474 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-24T23:06:36.475 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-24T23:06:36.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.578+0000 7f22ae2df640 1 Processor -- start 2024-04-24T23:06:36.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.578+0000 7f22ae2df640 1 -- start start 2024-04-24T23:06:36.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.579+0000 7f22ae2df640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f22a80b44d0 0x7f22a80b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:36.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.579+0000 7f22ae2df640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22a80b6430 con 0x7f22a80af890 2024-04-24T23:06:36.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.579+0000 7f22ae2df640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22a8156f80 con 0x7f22a806c5f0 2024-04-24T23:06:36.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.579+0000 7f22ae2df640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f22a80b6150 con 0x7f22a80b44d0 2024-04-24T23:06:36.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.579+0000 7f22a77fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f22a80b44d0 0x7f22a80b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:36.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.579+0000 7f22ac87c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f22a80af890 0x7f22a815cac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33140/0 (socket says 172.21.15.32:33140) 2024-04-24T23:06:36.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.579+0000 7f22a77fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f22a80b44d0 0x7f22a80b5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48578/0 (socket says 172.21.15.32:48578) 2024-04-24T23:06:36.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.579+0000 7f22ac87c640 1 -- 172.21.15.32:0/291837966 learned_addr learned my addr 172.21.15.32:0/291837966 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:06:36.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.579+0000 7f22a77fe640 1 -- 172.21.15.32:0/291837966 >> v1:172.21.15.121:6789/0 conn(0x7f22a806c5f0 legacy=0x7f22a806c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:36.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.579+0000 7f22a77fe640 1 -- 172.21.15.32:0/291837966 >> v1:172.21.15.32:6789/0 conn(0x7f22a80af890 legacy=0x7f22a815cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:36.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.579+0000 7f22a77fe640 1 -- 172.21.15.32:0/291837966 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a80b39f0 con 0x7f22a80b44d0 2024-04-24T23:06:36.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.579+0000 7f22a6ffd640 1 -- 172.21.15.32:0/291837966 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4130489780 0 0) 0x7f22a80b6430 con 0x7f22a80af890 2024-04-24T23:06:36.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.580+0000 7f22a77fe640 1 --2- 172.21.15.32:0/291837966 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22a80b44d0 0x7f22a80b5890 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f22980097a0 tx=0x7f229802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7d752b8d39664d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:36.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.580+0000 7f22a6ffd640 1 -- 172.21.15.32:0/291837966 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2298037070 con 0x7f22a80b44d0 2024-04-24T23:06:36.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.580+0000 7f22a6ffd640 1 -- 172.21.15.32:0/291837966 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2298032440 con 0x7f22a80b44d0 2024-04-24T23:06:36.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.580+0000 7f22a6ffd640 1 -- 172.21.15.32:0/291837966 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2298032810 con 0x7f22a80b44d0 2024-04-24T23:06:36.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.580+0000 7f22ae2df640 1 -- 172.21.15.32:0/291837966 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22a80b44d0 msgr2=0x7f22a80b5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:36.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.580+0000 7f22ae2df640 1 --2- 172.21.15.32:0/291837966 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22a80b44d0 0x7f22a80b5890 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f22980097a0 tx=0x7f229802a1e0 comp rx=0 tx=0).stop 2024-04-24T23:06:36.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.581+0000 7f22ae2df640 1 -- 172.21.15.32:0/291837966 shutdown_connections 2024-04-24T23:06:36.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.581+0000 7f22ae2df640 1 --2- 172.21.15.32:0/291837966 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22a80b44d0 0x7f22a80b5890 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:36.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.581+0000 7f22ae2df640 1 -- 172.21.15.32:0/291837966 >> 172.21.15.32:0/291837966 conn(0x7f22a806b110 msgr2=0x7f22a806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:36.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.581+0000 7f22ae2df640 1 -- 172.21.15.32:0/291837966 shutdown_connections 2024-04-24T23:06:36.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.581+0000 7f22ae2df640 1 -- 172.21.15.32:0/291837966 wait complete. 2024-04-24T23:06:36.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.582+0000 7f22ae2df640 1 Processor -- start 2024-04-24T23:06:36.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.582+0000 7f22ae2df640 1 -- start start 2024-04-24T23:06:36.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.583+0000 7f22ae2df640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22a806c5f0 0x7f22a8140180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:36.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.583+0000 7f22ae2df640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22a80af890 0x7f22a8153830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:36.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.583+0000 7f22ae2df640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22a80b44d0 0x7f22a8153d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:36.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.583+0000 7f22ae2df640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f22a80b5dd0 con 0x7f22a80af890 2024-04-24T23:06:36.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.583+0000 7f22ae2df640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f22a80b3fb0 con 0x7f22a80b44d0 2024-04-24T23:06:36.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.583+0000 7f22ae2df640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f22a815d1d0 con 0x7f22a806c5f0 2024-04-24T23:06:36.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.583+0000 7f22a77fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22a80af890 0x7f22a8153830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:36.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.583+0000 7f22a77fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22a80af890 0x7f22a8153830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48584/0 (socket says 172.21.15.32:48584) 2024-04-24T23:06:36.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.583+0000 7f22a77fe640 1 -- 172.21.15.32:0/1133281365 learned_addr learned my addr 172.21.15.32:0/1133281365 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:36.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.583+0000 7f22a7fff640 1 --2- 172.21.15.32:0/1133281365 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22a806c5f0 0x7f22a8140180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:36.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.583+0000 7f22a77fe640 1 -- 172.21.15.32:0/1133281365 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22a806c5f0 msgr2=0x7f22a8140180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:36.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.583+0000 7f22ac87c640 1 --2- 172.21.15.32:0/1133281365 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22a80b44d0 0x7f22a8153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:36.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.583+0000 7f22a77fe640 1 --2- 172.21.15.32:0/1133281365 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22a806c5f0 0x7f22a8140180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:36.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.583+0000 7f22a77fe640 1 -- 172.21.15.32:0/1133281365 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22a80b44d0 msgr2=0x7f22a8153d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:36.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.583+0000 7f22a77fe640 1 --2- 172.21.15.32:0/1133281365 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22a80b44d0 0x7f22a8153d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:36.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.583+0000 7f22a77fe640 1 -- 172.21.15.32:0/1133281365 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a813f0e0 con 0x7f22a80af890 2024-04-24T23:06:36.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.583+0000 7f22a7fff640 1 --2- 172.21.15.32:0/1133281365 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22a806c5f0 0x7f22a8140180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:06:36.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.583+0000 7f22ac87c640 1 --2- 172.21.15.32:0/1133281365 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22a80b44d0 0x7f22a8153d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:06:36.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.583+0000 7f22a77fe640 1 --2- 172.21.15.32:0/1133281365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22a80af890 0x7f22a8153830 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f22980097a0 tx=0x7f22980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:36.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.583+0000 7f22a4ff9640 1 -- 172.21.15.32:0/1133281365 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2298037040 con 0x7f22a80af890 2024-04-24T23:06:36.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.583+0000 7f22a4ff9640 1 -- 172.21.15.32:0/1133281365 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2298004250 con 0x7f22a80af890 2024-04-24T23:06:36.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.583+0000 7f22ae2df640 1 -- 172.21.15.32:0/1133281365 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22a813f3c0 con 0x7f22a80af890 2024-04-24T23:06:36.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.583+0000 7f22a4ff9640 1 -- 172.21.15.32:0/1133281365 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2298032e20 con 0x7f22a80af890 2024-04-24T23:06:36.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.583+0000 7f22ae2df640 1 -- 172.21.15.32:0/1133281365 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22a813f980 con 0x7f22a80af890 2024-04-24T23:06:36.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.585+0000 7f22a4ff9640 1 -- 172.21.15.32:0/1133281365 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2298030770 con 0x7f22a80af890 2024-04-24T23:06:36.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.585+0000 7f22a4ff9640 1 --2- 172.21.15.32:0/1133281365 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f229406be30 0x7f229406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:36.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.585+0000 7f22a4ff9640 1 -- 172.21.15.32:0/1133281365 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7f22980ac1a0 con 0x7f22a80af890 2024-04-24T23:06:36.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.586+0000 7f22a7fff640 1 --2- 172.21.15.32:0/1133281365 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f229406be30 0x7f229406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:36.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.586+0000 7f22a7fff640 1 --2- 172.21.15.32:0/1133281365 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f229406be30 0x7f229406e2f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f229c003eb0 tx=0x7f229c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:36.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.586+0000 7f22b0fe0740 1 Processor -- start 2024-04-24T23:06:36.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.586+0000 7f22b0fe0740 1 -- start start 2024-04-24T23:06:36.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.587+0000 7f22b0fe0740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56176acf1e50 0x56176acf5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:36.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.587+0000 7f22b0fe0740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56176acc9b60 0x56176acf2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:36.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.587+0000 7f22b0fe0740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56176acf2770 0x56176acf5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:36.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.587+0000 7f22b0fe0740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56176ac2df00 con 0x56176acc9b60 2024-04-24T23:06:36.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.587+0000 7f22b0fe0740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x56176abd6750 con 0x56176acf2770 2024-04-24T23:06:36.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.587+0000 7f22b0fe0740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x56176ac29ec0 con 0x56176acf1e50 2024-04-24T23:06:36.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.587+0000 7f22a77fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56176acc9b60 0x56176acf2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:36.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.587+0000 7f22a77fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56176acc9b60 0x56176acf2230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48600/0 (socket says 172.21.15.32:48600) 2024-04-24T23:06:36.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.587+0000 7f22a77fe640 1 -- 172.21.15.32:0/2504315939 learned_addr learned my addr 172.21.15.32:0/2504315939 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:36.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.587+0000 7f22ac87c640 1 --2- 172.21.15.32:0/2504315939 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56176acf2770 0x56176acf5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:36.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.587+0000 7f22a7fff640 1 --2- 172.21.15.32:0/2504315939 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56176acf1e50 0x56176acf5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:36.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.587+0000 7f22a77fe640 1 -- 172.21.15.32:0/2504315939 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56176acf1e50 msgr2=0x56176acf5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:36.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.587+0000 7f22a77fe640 1 --2- 172.21.15.32:0/2504315939 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56176acf1e50 0x56176acf5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:36.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.587+0000 7f22a77fe640 1 -- 172.21.15.32:0/2504315939 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56176acf2770 msgr2=0x56176acf5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:36.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.587+0000 7f22a77fe640 1 --2- 172.21.15.32:0/2504315939 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56176acf2770 0x56176acf5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:36.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.587+0000 7f22a77fe640 1 -- 172.21.15.32:0/2504315939 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56176a9422d0 con 0x56176acc9b60 2024-04-24T23:06:36.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.587+0000 7f22ac87c640 1 --2- 172.21.15.32:0/2504315939 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56176acf2770 0x56176acf5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:06:36.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.587+0000 7f22a77fe640 1 --2- 172.21.15.32:0/2504315939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56176acc9b60 0x56176acf2230 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f2298035040 tx=0x7f22980093c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2e56b3c9c33b9ae server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:36.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.588+0000 7f228dffb640 1 -- 172.21.15.32:0/2504315939 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229807a3b0 con 0x56176acc9b60 2024-04-24T23:06:36.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.588+0000 7f228dffb640 1 -- 172.21.15.32:0/2504315939 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f229807e760 con 0x56176acc9b60 2024-04-24T23:06:36.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.588+0000 7f228dffb640 1 -- 172.21.15.32:0/2504315939 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2298082f90 con 0x56176acc9b60 2024-04-24T23:06:36.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.588+0000 7f22b0fe0740 1 -- 172.21.15.32:0/2504315939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56176acc9b60 msgr2=0x56176acf2230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:36.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.588+0000 7f22b0fe0740 1 --2- 172.21.15.32:0/2504315939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56176acc9b60 0x56176acf2230 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f2298035040 tx=0x7f22980093c0 comp rx=0 tx=0).stop 2024-04-24T23:06:36.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.588+0000 7f22b0fe0740 1 -- 172.21.15.32:0/2504315939 shutdown_connections 2024-04-24T23:06:36.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.588+0000 7f22b0fe0740 1 --2- 172.21.15.32:0/2504315939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56176acc9b60 0x56176acf2230 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:36.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.588+0000 7f22b0fe0740 1 --2- 172.21.15.32:0/2504315939 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56176acf2770 0x56176acf5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:36.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.588+0000 7f22b0fe0740 1 --2- 172.21.15.32:0/2504315939 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56176acf1e50 0x56176acf5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:36.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.588+0000 7f22b0fe0740 1 -- 172.21.15.32:0/2504315939 >> 172.21.15.32:0/2504315939 conn(0x56176acc7760 msgr2=0x56176ad10000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:36.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.588+0000 7f22b0fe0740 1 -- 172.21.15.32:0/2504315939 shutdown_connections 2024-04-24T23:06:36.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.588+0000 7f22b0fe0740 1 -- 172.21.15.32:0/2504315939 wait complete. 2024-04-24T23:06:36.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.589+0000 7f22b0fe0740 1 -- start start 2024-04-24T23:06:36.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.589+0000 7f22b0fe0740 1 Processor -- start 2024-04-24T23:06:36.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.590+0000 7f22b0fe0740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56176acf1e50 0x56176acf8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:36.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.590+0000 7f22b0fe0740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56176acf2770 0x56176ace7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:36.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.590+0000 7f22b0fe0740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56176ace81e0 0x56176ad136c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:36.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.590+0000 7f22b0fe0740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56176ac2df00 con 0x56176ace81e0 2024-04-24T23:06:36.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.590+0000 7f22b0fe0740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x56176abe9cf0 con 0x56176acf2770 2024-04-24T23:06:36.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.590+0000 7f22b0fe0740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x56176acc5ab0 con 0x56176acf1e50 2024-04-24T23:06:36.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.590+0000 7f22b0fe0740 10 client.0 _ll_register_callbacks cb 0x56176abe96e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:06:36.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.590+0000 7f22ac87c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56176ace81e0 0x56176ad136c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:36.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.590+0000 7f22ac87c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56176ace81e0 0x56176ad136c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48614/0 (socket says 172.21.15.32:48614) 2024-04-24T23:06:36.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.590+0000 7f22ac87c640 1 -- 172.21.15.32:0/1502027826 learned_addr learned my addr 172.21.15.32:0/1502027826 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:36.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.590+0000 7f22a77fe640 1 --2- 172.21.15.32:0/1502027826 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56176acf2770 0x56176ace7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:36.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.590+0000 7f22ac87c640 1 -- 172.21.15.32:0/1502027826 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56176acf1e50 msgr2=0x56176acf8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:36.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.590+0000 7f22a7fff640 1 --2- 172.21.15.32:0/1502027826 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56176acf1e50 0x56176acf8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:36.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.590+0000 7f22ac87c640 1 --2- 172.21.15.32:0/1502027826 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56176acf1e50 0x56176acf8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:36.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.590+0000 7f22ac87c640 1 -- 172.21.15.32:0/1502027826 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56176acf2770 msgr2=0x56176ace7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:36.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.590+0000 7f22ac87c640 1 --2- 172.21.15.32:0/1502027826 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56176acf2770 0x56176ace7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:36.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.590+0000 7f22ac87c640 1 -- 172.21.15.32:0/1502027826 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56176aa0c6b0 con 0x56176ace81e0 2024-04-24T23:06:36.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.590+0000 7f22a77fe640 1 --2- 172.21.15.32:0/1502027826 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56176acf2770 0x56176ace7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:06:36.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.590+0000 7f22a7fff640 1 --2- 172.21.15.32:0/1502027826 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56176acf1e50 0x56176acf8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:06:36.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.590+0000 7f22ac87c640 1 --2- 172.21.15.32:0/1502027826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56176ace81e0 0x56176ad136c0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f22a0000f80 tx=0x7f22a000e950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd061d1a64ac5bf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:36.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.590+0000 7f2286ffd640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-24T23:06:36.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.591+0000 7f2286ffd640 1 -- 172.21.15.32:0/1502027826 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a0019060 con 0x56176ace81e0 2024-04-24T23:06:36.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.591+0000 7f2286ffd640 1 -- 172.21.15.32:0/1502027826 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7f22a0014af0 con 0x56176ace81e0 2024-04-24T23:06:36.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.591+0000 7f22b0fe0740 1 -- 172.21.15.32:0/1502027826 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f229807e760 con 0x56176ace81e0 2024-04-24T23:06:36.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.591+0000 7f2286ffd640 1 -- 172.21.15.32:0/1502027826 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f22a0014cf0 con 0x56176ace81e0 2024-04-24T23:06:36.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.591+0000 7f22b0fe0740 10 client.8427 fetch_fsmap learned FSMap version 65 2024-04-24T23:06:36.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.591+0000 7f22b0fe0740 1 -- 172.21.15.32:0/1502027826 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) -- 0x56176ace9170 con 0x56176ace81e0 2024-04-24T23:06:36.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.591+0000 7f2286ffd640 1 -- 172.21.15.32:0/1502027826 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f229807e760 con 0x56176ace81e0 2024-04-24T23:06:36.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.591+0000 7f2286ffd640 1 -- 172.21.15.32:0/1502027826 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 65) ==== 3494+0+0 (secure 0 0 0) 0x7f22a00070d0 con 0x56176ace81e0 2024-04-24T23:06:36.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.591+0000 7f22b0fe0740 10 client.8427 fetch_fsmap finished waiting for FSMap version 65 2024-04-24T23:06:36.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.591+0000 7f22b0fe0740 10 client.8427 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T23:06:36.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.591+0000 7f22b0fe0740 20 client.8427 populate_metadata read hostname 'smithi032' 2024-04-24T23:06:36.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.591+0000 7f22b0fe0740 1 --2- 172.21.15.32:0/1502027826 >> [v2:172.21.15.32:6834/984603360,v1:172.21.15.32:6835/984603360] conn(0x56176ad114c0 0x56176ad24cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:36.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.591+0000 7f22b0fe0740 4 client.8427 mds_command: new command op to 8286 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:06:36.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.591+0000 7f22b0fe0740 1 -- 172.21.15.32:0/1502027826 --> [v2:172.21.15.32:6834/984603360,v1:172.21.15.32:6835/984603360] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56176ac3b440 con 0x56176ad114c0 2024-04-24T23:06:36.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.591+0000 7f22a7fff640 1 --2- 172.21.15.32:0/1502027826 >> [v2:172.21.15.32:6834/984603360,v1:172.21.15.32:6835/984603360] conn(0x56176ad114c0 0x56176ad24cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:36.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.591+0000 7f22a7fff640 1 --2- 172.21.15.32:0/1502027826 >> [v2:172.21.15.32:6834/984603360,v1:172.21.15.32:6835/984603360] conn(0x56176ad114c0 0x56176ad24cd0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c298eed70270673f server_cookie=326e646d114e8c3 in_seq=0 out_seq=0 2024-04-24T23:06:36.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.591+0000 7f2286ffd640 10 client.8427 ms_handle_connect on v2:172.21.15.32:6834/984603360 2024-04-24T23:06:36.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.592+0000 7f2286ffd640 1 -- 172.21.15.32:0/1502027826 <== mds.0 v2:172.21.15.32:6834/984603360 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56176ac3b440 con 0x56176ad114c0 2024-04-24T23:06:36.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.592+0000 7f2286ffd640 10 client.8427 handle_command_reply: tid=0 2024-04-24T23:06:36.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.592+0000 7f22b0fe0740 1 client.8427 shutdown 2024-04-24T23:06:36.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.592+0000 7f22b0fe0740 1 -- 172.21.15.32:0/1502027826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56176ace81e0 msgr2=0x56176ad136c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:36.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.592+0000 7f22b0fe0740 1 --2- 172.21.15.32:0/1502027826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56176ace81e0 0x56176ad136c0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f22a0000f80 tx=0x7f22a000e950 comp rx=0 tx=0).stop 2024-04-24T23:06:36.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.592+0000 7f22b0fe0740 1 -- 172.21.15.32:0/1502027826 shutdown_connections 2024-04-24T23:06:36.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.592+0000 7f22b0fe0740 1 --2- 172.21.15.32:0/1502027826 >> [v2:172.21.15.32:6834/984603360,v1:172.21.15.32:6835/984603360] conn(0x56176ad114c0 0x56176ad24cd0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:36.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.592+0000 7f22b0fe0740 1 --2- 172.21.15.32:0/1502027826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56176ace81e0 0x56176ad136c0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:36.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.592+0000 7f22b0fe0740 1 --2- 172.21.15.32:0/1502027826 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56176acf2770 0x56176ace7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:36.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.592+0000 7f2286ffd640 0 client.8427 ms_handle_reset on v2:172.21.15.32:6834/984603360 2024-04-24T23:06:36.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.592+0000 7f22b0fe0740 1 --2- 172.21.15.32:0/1502027826 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56176acf1e50 0x56176acf8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:36.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.592+0000 7f22b0fe0740 1 -- 172.21.15.32:0/1502027826 >> 172.21.15.32:0/1502027826 conn(0x56176acc9b60 msgr2=0x56176acc4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:36.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.592+0000 7f22b0fe0740 1 -- 172.21.15.32:0/1502027826 shutdown_connections 2024-04-24T23:06:36.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.592+0000 7f22b0fe0740 1 -- 172.21.15.32:0/1502027826 wait complete. 2024-04-24T23:06:36.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.593+0000 7f22b0fe0740 20 client.8427 trim_cache size 0 max 16384 2024-04-24T23:06:36.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.611+0000 7f22b0fe0740 1 Processor -- start 2024-04-24T23:06:36.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.611+0000 7f22b0fe0740 1 -- start start 2024-04-24T23:06:36.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.611+0000 7f22b0fe0740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56176ace81e0 0x56176acc47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:36.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.611+0000 7f22b0fe0740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56176acf1e50 0x56176ace7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:36.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.611+0000 7f22b0fe0740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56176acf2770 0x56176acc8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:36.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.611+0000 7f22b0fe0740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f22a80cec30 con 0x56176acf2770 2024-04-24T23:06:36.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.611+0000 7f22b0fe0740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f22a80cdfc0 con 0x56176ace81e0 2024-04-24T23:06:36.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.611+0000 7f22b0fe0740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f22a80ce5e0 con 0x56176acf1e50 2024-04-24T23:06:36.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.612+0000 7f22ac87c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56176acf2770 0x56176acc8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:36.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.612+0000 7f22ac87c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56176acf2770 0x56176acc8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48620/0 (socket says 172.21.15.32:48620) 2024-04-24T23:06:36.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.612+0000 7f22ac87c640 1 -- 172.21.15.32:0/1807019794 learned_addr learned my addr 172.21.15.32:0/1807019794 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:36.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.612+0000 7f22ac87c640 1 -- 172.21.15.32:0/1807019794 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56176acf1e50 msgr2=0x56176ace7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:36.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.612+0000 7f22ac87c640 1 --2- 172.21.15.32:0/1807019794 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56176acf1e50 0x56176ace7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:36.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.612+0000 7f22ac87c640 1 -- 172.21.15.32:0/1807019794 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56176ace81e0 msgr2=0x56176acc47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:36.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.612+0000 7f22a7fff640 1 --2- 172.21.15.32:0/1807019794 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56176ace81e0 0x56176acc47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:36.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.612+0000 7f22ac87c640 1 --2- 172.21.15.32:0/1807019794 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56176ace81e0 0x56176acc47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:36.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.612+0000 7f22ac87c640 1 -- 172.21.15.32:0/1807019794 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56176abd19e0 con 0x56176acf2770 2024-04-24T23:06:36.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.612+0000 7f22a7fff640 1 --2- 172.21.15.32:0/1807019794 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56176ace81e0 0x56176acc47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:06:36.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.612+0000 7f22ac87c640 1 --2- 172.21.15.32:0/1807019794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56176acf2770 0x56176acc8d00 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f22a0007b10 tx=0x7f22a0006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af34e4ae3651e534 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:36.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.612+0000 7f228d7fa640 1 -- 172.21.15.32:0/1807019794 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a002c070 con 0x56176acf2770 2024-04-24T23:06:36.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.612+0000 7f228d7fa640 1 -- 172.21.15.32:0/1807019794 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22a0001760 con 0x56176acf2770 2024-04-24T23:06:36.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.612+0000 7f228d7fa640 1 -- 172.21.15.32:0/1807019794 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a0007670 con 0x56176acf2770 2024-04-24T23:06:36.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.613+0000 7f22b0fe0740 1 -- 172.21.15.32:0/1807019794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56176acf2770 msgr2=0x56176acc8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:36.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.613+0000 7f22b0fe0740 1 --2- 172.21.15.32:0/1807019794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56176acf2770 0x56176acc8d00 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f22a0007b10 tx=0x7f22a0006000 comp rx=0 tx=0).stop 2024-04-24T23:06:36.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.613+0000 7f22b0fe0740 1 -- 172.21.15.32:0/1807019794 shutdown_connections 2024-04-24T23:06:36.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.613+0000 7f22b0fe0740 1 --2- 172.21.15.32:0/1807019794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56176acf2770 0x56176acc8d00 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:36.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.613+0000 7f22b0fe0740 1 --2- 172.21.15.32:0/1807019794 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56176acf1e50 0x56176ace7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:36.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.613+0000 7f22b0fe0740 1 --2- 172.21.15.32:0/1807019794 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56176ace81e0 0x56176acc47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:36.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.613+0000 7f22b0fe0740 1 -- 172.21.15.32:0/1807019794 >> 172.21.15.32:0/1807019794 conn(0x56176acc9b60 msgr2=0x56176acf0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:36.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.613+0000 7f22b0fe0740 1 -- 172.21.15.32:0/1807019794 shutdown_connections 2024-04-24T23:06:36.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.613+0000 7f22b0fe0740 1 -- 172.21.15.32:0/1807019794 wait complete. 2024-04-24T23:06:36.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.613+0000 7f22b0fe0740 1 -- start start 2024-04-24T23:06:36.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.614+0000 7f22b0fe0740 1 Processor -- start 2024-04-24T23:06:36.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.614+0000 7f22b0fe0740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56176ace81e0 0x56176ad91df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:36.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.614+0000 7f22b0fe0740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56176acf1e50 0x56176ad0fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:36.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.614+0000 7f22b0fe0740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56176acf2770 0x56176ad10310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:36.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.614+0000 7f22b0fe0740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56176ad11340 con 0x56176ace81e0 2024-04-24T23:06:36.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.614+0000 7f22b0fe0740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x56176ad256f0 con 0x56176acf1e50 2024-04-24T23:06:36.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.614+0000 7f22b0fe0740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x56176ad11990 con 0x56176acf2770 2024-04-24T23:06:36.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.614+0000 7f22b0fe0740 10 client.0 _ll_register_callbacks cb 0x7f22a0002080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:06:36.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.614+0000 7f22a7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56176ace81e0 0x56176ad91df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:36.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.614+0000 7f22a7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56176ace81e0 0x56176ad91df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48628/0 (socket says 172.21.15.32:48628) 2024-04-24T23:06:36.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.614+0000 7f22a7fff640 1 -- 172.21.15.32:0/3450191367 learned_addr learned my addr 172.21.15.32:0/3450191367 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:36.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.615+0000 7f22ac87c640 1 --2- 172.21.15.32:0/3450191367 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56176acf2770 0x56176ad10310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:36.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.615+0000 7f22a77fe640 1 --2- 172.21.15.32:0/3450191367 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56176acf1e50 0x56176ad0fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:36.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.615+0000 7f22a7fff640 1 -- 172.21.15.32:0/3450191367 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56176acf2770 msgr2=0x56176ad10310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:36.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.615+0000 7f22a7fff640 1 --2- 172.21.15.32:0/3450191367 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56176acf2770 0x56176ad10310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:36.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.615+0000 7f22a7fff640 1 -- 172.21.15.32:0/3450191367 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56176acf1e50 msgr2=0x56176ad0fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:36.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.615+0000 7f22a7fff640 1 --2- 172.21.15.32:0/3450191367 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56176acf1e50 0x56176ad0fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:36.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.615+0000 7f22a7fff640 1 -- 172.21.15.32:0/3450191367 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56176ad2d4b0 con 0x56176ace81e0 2024-04-24T23:06:36.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.615+0000 7f22ac87c640 1 --2- 172.21.15.32:0/3450191367 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56176acf2770 0x56176ad10310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:06:36.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.615+0000 7f22a77fe640 1 --2- 172.21.15.32:0/3450191367 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56176acf1e50 0x56176ad0fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:06:36.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.615+0000 7f22a7fff640 1 --2- 172.21.15.32:0/3450191367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56176ace81e0 0x56176ad91df0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f229c00d920 tx=0x7f229c0036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b09af8ca5a3fe46 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:36.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.615+0000 7f2286ffd640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-24T23:06:36.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.615+0000 7f2286ffd640 1 -- 172.21.15.32:0/3450191367 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c007250 con 0x56176ace81e0 2024-04-24T23:06:36.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.615+0000 7f22b0fe0740 1 -- 172.21.15.32:0/3450191367 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22a0001760 con 0x56176ace81e0 2024-04-24T23:06:36.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.615+0000 7f2286ffd640 1 -- 172.21.15.32:0/3450191367 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7f229c0073f0 con 0x56176ace81e0 2024-04-24T23:06:36.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.615+0000 7f2286ffd640 1 -- 172.21.15.32:0/3450191367 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f229c032050 con 0x56176ace81e0 2024-04-24T23:06:36.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.615+0000 7f22b0fe0740 10 client.8433 fetch_fsmap learned FSMap version 65 2024-04-24T23:06:36.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.615+0000 7f22b0fe0740 1 -- 172.21.15.32:0/3450191367 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) -- 0x56176ad25060 con 0x56176ace81e0 2024-04-24T23:06:36.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.615+0000 7f2286ffd640 1 -- 172.21.15.32:0/3450191367 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22a0001760 con 0x56176ace81e0 2024-04-24T23:06:36.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.615+0000 7f2286ffd640 1 -- 172.21.15.32:0/3450191367 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 65) ==== 3494+0+0 (secure 0 0 0) 0x7f229c035020 con 0x56176ace81e0 2024-04-24T23:06:36.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.615+0000 7f22b0fe0740 10 client.8433 fetch_fsmap finished waiting for FSMap version 65 2024-04-24T23:06:36.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.616+0000 7f22b0fe0740 10 client.8433 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T23:06:36.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.616+0000 7f22b0fe0740 20 client.8433 populate_metadata read hostname 'smithi032' 2024-04-24T23:06:36.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.616+0000 7f22b0fe0740 1 --2- 172.21.15.32:0/3450191367 >> [v2:172.21.15.32:6834/984603360,v1:172.21.15.32:6835/984603360] conn(0x56176ad1e980 0x56176ad1ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:36.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.616+0000 7f22b0fe0740 4 client.8433 mds_command: new command op to 8286 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:06:36.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.616+0000 7f22b0fe0740 1 -- 172.21.15.32:0/3450191367 --> [v2:172.21.15.32:6834/984603360,v1:172.21.15.32:6835/984603360] -- command(tid 0: {"prefix": "status"}) -- 0x56176ac35790 con 0x56176ad1e980 2024-04-24T23:06:36.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.616+0000 7f22a77fe640 1 --2- 172.21.15.32:0/3450191367 >> [v2:172.21.15.32:6834/984603360,v1:172.21.15.32:6835/984603360] conn(0x56176ad1e980 0x56176ad1ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:36.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.616+0000 7f22a77fe640 1 --2- 172.21.15.32:0/3450191367 >> [v2:172.21.15.32:6834/984603360,v1:172.21.15.32:6835/984603360] conn(0x56176ad1e980 0x56176ad1ed60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eacf85a99166227d server_cookie=2616a3bb9854f111 in_seq=0 out_seq=0 2024-04-24T23:06:36.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.616+0000 7f2286ffd640 10 client.8433 ms_handle_connect on v2:172.21.15.32:6834/984603360 2024-04-24T23:06:36.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.616+0000 7f2286ffd640 1 -- 172.21.15.32:0/3450191367 <== mds.0 v2:172.21.15.32:6834/984603360 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x56176ac35790 con 0x56176ad1e980 2024-04-24T23:06:36.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.616+0000 7f2286ffd640 10 client.8433 handle_command_reply: tid=0 2024-04-24T23:06:36.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.616+0000 7f22b0fe0740 1 client.8433 shutdown 2024-04-24T23:06:36.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.616+0000 7f22b0fe0740 1 -- 172.21.15.32:0/3450191367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56176ace81e0 msgr2=0x56176ad91df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:36.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.616+0000 7f22b0fe0740 1 --2- 172.21.15.32:0/3450191367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56176ace81e0 0x56176ad91df0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f229c00d920 tx=0x7f229c0036f0 comp rx=0 tx=0).stop 2024-04-24T23:06:36.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.616+0000 7f22b0fe0740 1 -- 172.21.15.32:0/3450191367 shutdown_connections 2024-04-24T23:06:36.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.616+0000 7f22b0fe0740 1 --2- 172.21.15.32:0/3450191367 >> [v2:172.21.15.32:6834/984603360,v1:172.21.15.32:6835/984603360] conn(0x56176ad1e980 0x56176ad1ed60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:36.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.616+0000 7f22b0fe0740 1 --2- 172.21.15.32:0/3450191367 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56176acf2770 0x56176ad10310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:36.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.616+0000 7f22b0fe0740 1 --2- 172.21.15.32:0/3450191367 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56176acf1e50 0x56176ad0fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:36.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.616+0000 7f2286ffd640 0 client.8433 ms_handle_reset on v2:172.21.15.32:6834/984603360 2024-04-24T23:06:36.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.616+0000 7f22b0fe0740 1 --2- 172.21.15.32:0/3450191367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56176ace81e0 0x56176ad91df0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:36.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.616+0000 7f22b0fe0740 1 -- 172.21.15.32:0/3450191367 >> 172.21.15.32:0/3450191367 conn(0x56176acc9b60 msgr2=0x56176acc5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:36.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.616+0000 7f22b0fe0740 1 -- 172.21.15.32:0/3450191367 shutdown_connections 2024-04-24T23:06:36.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.616+0000 7f22b0fe0740 1 -- 172.21.15.32:0/3450191367 wait complete. 2024-04-24T23:06:36.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.617+0000 7f22b0fe0740 20 client.8433 trim_cache size 0 max 16384 2024-04-24T23:06:36.625 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:06:36.625 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", 2024-04-24T23:06:36.625 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2024-04-24T23:06:36.625 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 8286, 2024-04-24T23:06:36.626 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-04-24T23:06:36.626 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-04-24T23:06:36.626 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-04-24T23:06:36.626 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 4.6731744329999998, 2024-04-24T23:06:36.626 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 65, 2024-04-24T23:06:36.626 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 156, 2024-04-24T23:06:36.626 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 156, 2024-04-24T23:06:36.626 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 9.6838547459999997 2024-04-24T23:06:36.626 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:06:36.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.617+0000 7f22ae2df640 1 -- 172.21.15.32:0/1133281365 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f229406be30 msgr2=0x7f229406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:36.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.617+0000 7f22ae2df640 1 --2- 172.21.15.32:0/1133281365 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f229406be30 0x7f229406e2f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f229c003eb0 tx=0x7f229c009040 comp rx=0 tx=0).stop 2024-04-24T23:06:36.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.617+0000 7f22ae2df640 1 -- 172.21.15.32:0/1133281365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22a80af890 msgr2=0x7f22a8153830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:36.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.617+0000 7f22ae2df640 1 --2- 172.21.15.32:0/1133281365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22a80af890 0x7f22a8153830 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f22980097a0 tx=0x7f22980046c0 comp rx=0 tx=0).stop 2024-04-24T23:06:36.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.617+0000 7f22ae2df640 1 -- 172.21.15.32:0/1133281365 shutdown_connections 2024-04-24T23:06:36.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.617+0000 7f22ae2df640 1 --2- 172.21.15.32:0/1133281365 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f229406be30 0x7f229406e2f0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:36.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.617+0000 7f22ae2df640 1 --2- 172.21.15.32:0/1133281365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22a80af890 0x7f22a8153830 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:36.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.617+0000 7f22ae2df640 1 --2- 172.21.15.32:0/1133281365 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22a80b44d0 0x7f22a8153d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:36.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.617+0000 7f22ae2df640 1 --2- 172.21.15.32:0/1133281365 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22a806c5f0 0x7f22a8140180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:36.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.617+0000 7f22ae2df640 1 -- 172.21.15.32:0/1133281365 >> 172.21.15.32:0/1133281365 conn(0x7f22a806b110 msgr2=0x7f22a8066ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:36.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.617+0000 7f22ae2df640 1 -- 172.21.15.32:0/1133281365 shutdown_connections 2024-04-24T23:06:36.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.617+0000 7f22ae2df640 1 -- 172.21.15.32:0/1133281365 wait complete. 2024-04-24T23:06:36.629 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T23:06:36.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.729+0000 7f275e1b0640 1 Processor -- start 2024-04-24T23:06:36.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.729+0000 7f275e1b0640 1 -- start start 2024-04-24T23:06:36.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.730+0000 7f275e1b0640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f275815cb00 0x7f27580ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:36.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.730+0000 7f275e1b0640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27580b4520 con 0x7f275806cd00 2024-04-24T23:06:36.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.730+0000 7f275e1b0640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27580b4800 con 0x7f275815b380 2024-04-24T23:06:36.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.730+0000 7f275e1b0640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f27580b4ae0 con 0x7f275815cb00 2024-04-24T23:06:36.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.730+0000 7f2756ffd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f275806cd00 0x7f27580ce430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33146/0 (socket says 172.21.15.32:33146) 2024-04-24T23:06:36.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.730+0000 7f2756ffd640 1 -- 172.21.15.32:0/4073274903 learned_addr learned my addr 172.21.15.32:0/4073274903 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:06:36.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.730+0000 7f2757fff640 1 --2- 172.21.15.32:0/4073274903 >> v2:172.21.15.121:3300/0 conn(0x7f275815cb00 0x7f27580ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:36.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.730+0000 7f27567fc640 1 -- 172.21.15.32:0/4073274903 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3534615698 0 0) 0x7f27580b4520 con 0x7f275806cd00 2024-04-24T23:06:36.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.730+0000 7f27567fc640 1 -- 172.21.15.32:0/4073274903 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2740003520 con 0x7f275806cd00 2024-04-24T23:06:36.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.730+0000 7f27567fc640 1 -- 172.21.15.32:0/4073274903 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4052129260 0 0) 0x7f2740003520 con 0x7f275806cd00 2024-04-24T23:06:36.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.731+0000 7f27567fc640 1 -- 172.21.15.32:0/4073274903 >> v2:172.21.15.121:3300/0 conn(0x7f275815cb00 msgr2=0x7f27580ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:36.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.731+0000 7f27567fc640 1 --2- 172.21.15.32:0/4073274903 >> v2:172.21.15.121:3300/0 conn(0x7f275815cb00 0x7f27580ceb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:36.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.731+0000 7f27567fc640 1 -- 172.21.15.32:0/4073274903 >> v1:172.21.15.121:6789/0 conn(0x7f275815b380 legacy=0x7f275806c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:36.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.731+0000 7f27567fc640 1 -- 172.21.15.32:0/4073274903 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27580b4dc0 con 0x7f275806cd00 2024-04-24T23:06:36.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.731+0000 7f27567fc640 1 -- 172.21.15.32:0/4073274903 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f274c001670 con 0x7f275806cd00 2024-04-24T23:06:36.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.731+0000 7f27567fc640 1 -- 172.21.15.32:0/4073274903 >> v1:172.21.15.32:6789/0 conn(0x7f275806cd00 legacy=0x7f27580ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:36.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.731+0000 7f27567fc640 1 --2- 172.21.15.32:0/4073274903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27400039b0 0x7f2740003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:36.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.731+0000 7f27567fc640 1 --2- 172.21.15.32:0/4073274903 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2740004800 0x7f2740004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:36.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.731+0000 7f27567fc640 1 --2- 172.21.15.32:0/4073274903 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27400051c0 0x7f274000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:36.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.731+0000 7f27567fc640 1 -- 172.21.15.32:0/4073274903 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f27580b4ae0 con 0x7f27400039b0 2024-04-24T23:06:36.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.731+0000 7f27567fc640 1 -- 172.21.15.32:0/4073274903 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f274000c160 con 0x7f27400051c0 2024-04-24T23:06:36.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.731+0000 7f27567fc640 1 -- 172.21.15.32:0/4073274903 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f274000c360 con 0x7f2740004800 2024-04-24T23:06:36.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.731+0000 7f27577fe640 1 --2- 172.21.15.32:0/4073274903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27400039b0 0x7f2740003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:36.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.731+0000 7f2756ffd640 1 --2- 172.21.15.32:0/4073274903 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2740004800 0x7f2740004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:36.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.731+0000 7f2757fff640 1 --2- 172.21.15.32:0/4073274903 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27400051c0 0x7f274000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:36.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.731+0000 7f27577fe640 1 -- 172.21.15.32:0/4073274903 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2740004800 msgr2=0x7f2740004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:36.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.731+0000 7f27577fe640 1 --2- 172.21.15.32:0/4073274903 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2740004800 0x7f2740004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:36.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.731+0000 7f27577fe640 1 -- 172.21.15.32:0/4073274903 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27400051c0 msgr2=0x7f274000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:36.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.731+0000 7f27577fe640 1 --2- 172.21.15.32:0/4073274903 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27400051c0 0x7f274000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:36.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.731+0000 7f27577fe640 1 -- 172.21.15.32:0/4073274903 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2740003520 con 0x7f27400039b0 2024-04-24T23:06:36.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.731+0000 7f2757fff640 1 -- 172.21.15.32:0/4073274903 reap_dead start 2024-04-24T23:06:36.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.732+0000 7f27577fe640 1 --2- 172.21.15.32:0/4073274903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27400039b0 0x7f2740003dd0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f2748001750 tx=0x7f2740005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=843de19c64585e48 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:36.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.732+0000 7f27567fc640 1 -- 172.21.15.32:0/4073274903 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2748001370 con 0x7f27400039b0 2024-04-24T23:06:36.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.732+0000 7f27567fc640 1 -- 172.21.15.32:0/4073274903 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27480052d0 con 0x7f27400039b0 2024-04-24T23:06:36.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.732+0000 7f27567fc640 1 -- 172.21.15.32:0/4073274903 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274802abc0 con 0x7f27400039b0 2024-04-24T23:06:36.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.732+0000 7f275e1b0640 1 -- 172.21.15.32:0/4073274903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27400039b0 msgr2=0x7f2740003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:36.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.732+0000 7f275e1b0640 1 --2- 172.21.15.32:0/4073274903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27400039b0 0x7f2740003dd0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f2748001750 tx=0x7f2740005830 comp rx=0 tx=0).stop 2024-04-24T23:06:36.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.733+0000 7f275e1b0640 1 -- 172.21.15.32:0/4073274903 shutdown_connections 2024-04-24T23:06:36.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.733+0000 7f275e1b0640 1 --2- 172.21.15.32:0/4073274903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27400039b0 0x7f2740003dd0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:36.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.733+0000 7f275e1b0640 1 -- 172.21.15.32:0/4073274903 >> 172.21.15.32:0/4073274903 conn(0x7f275806b110 msgr2=0x7f275806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:36.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.733+0000 7f275e1b0640 1 -- 172.21.15.32:0/4073274903 shutdown_connections 2024-04-24T23:06:36.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.733+0000 7f275e1b0640 1 -- 172.21.15.32:0/4073274903 wait complete. 2024-04-24T23:06:36.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.734+0000 7f275e1b0640 1 Processor -- start 2024-04-24T23:06:36.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.734+0000 7f275e1b0640 1 -- start start 2024-04-24T23:06:36.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.735+0000 7f275e1b0640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f27400039b0 0x7f27581f8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:36.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.735+0000 7f275e1b0640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f275815c100 0x7f27581468e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:36.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.735+0000 7f275e1b0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f275815c580 0x7f2758146e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:36.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.735+0000 7f275e1b0640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f275806c5f0 con 0x7f275815c580 2024-04-24T23:06:36.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.735+0000 7f275e1b0640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f275815b760 con 0x7f275815c100 2024-04-24T23:06:36.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.735+0000 7f275e1b0640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f275806c770 con 0x7f27400039b0 2024-04-24T23:06:36.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.735+0000 7f2757fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f275815c580 0x7f2758146e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:36.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.735+0000 7f2757fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f275815c580 0x7f2758146e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48652/0 (socket says 172.21.15.32:48652) 2024-04-24T23:06:36.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.735+0000 7f2757fff640 1 -- 172.21.15.32:0/817273356 learned_addr learned my addr 172.21.15.32:0/817273356 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:36.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.735+0000 7f2756ffd640 1 --2- 172.21.15.32:0/817273356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f275815c100 0x7f27581468e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:36.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.735+0000 7f27577fe640 1 --2- 172.21.15.32:0/817273356 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f27400039b0 0x7f27581f8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:36.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.735+0000 7f2757fff640 1 -- 172.21.15.32:0/817273356 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f27400039b0 msgr2=0x7f27581f8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:36.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.735+0000 7f2757fff640 1 --2- 172.21.15.32:0/817273356 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f27400039b0 0x7f27581f8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:36.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.735+0000 7f2757fff640 1 -- 172.21.15.32:0/817273356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f275815c100 msgr2=0x7f27581468e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:36.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.735+0000 7f2757fff640 1 --2- 172.21.15.32:0/817273356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f275815c100 0x7f27581468e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:36.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.735+0000 7f2757fff640 1 -- 172.21.15.32:0/817273356 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27581f8e40 con 0x7f275815c580 2024-04-24T23:06:36.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.735+0000 7f27577fe640 1 --2- 172.21.15.32:0/817273356 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f27400039b0 0x7f27581f8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:06:36.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.735+0000 7f2757fff640 1 --2- 172.21.15.32:0/817273356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f275815c580 0x7f2758146e20 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f2758072ea0 tx=0x7f275000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:36.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.735+0000 7f273ffff640 1 -- 172.21.15.32:0/817273356 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275000fd80 con 0x7f275815c580 2024-04-24T23:06:36.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.735+0000 7f273ffff640 1 -- 172.21.15.32:0/817273356 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2750015430 con 0x7f275815c580 2024-04-24T23:06:36.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.735+0000 7f273ffff640 1 -- 172.21.15.32:0/817273356 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2750015820 con 0x7f275815c580 2024-04-24T23:06:36.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.735+0000 7f275e1b0640 1 -- 172.21.15.32:0/817273356 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27580ab140 con 0x7f275815c580 2024-04-24T23:06:36.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.736+0000 7f275e1b0640 1 -- 172.21.15.32:0/817273356 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2758157a90 con 0x7f275815c580 2024-04-24T23:06:36.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.736+0000 7f275e1b0640 1 -- 172.21.15.32:0/817273356 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27580ab660 con 0x7f275815c580 2024-04-24T23:06:36.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.737+0000 7f273ffff640 1 -- 172.21.15.32:0/817273356 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f27500159c0 con 0x7f275815c580 2024-04-24T23:06:36.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.737+0000 7f273ffff640 1 --2- 172.21.15.32:0/817273356 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f274406be30 0x7f274406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:36.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.737+0000 7f273ffff640 1 -- 172.21.15.32:0/817273356 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7f275008c080 con 0x7f275815c580 2024-04-24T23:06:36.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.738+0000 7f27577fe640 1 --2- 172.21.15.32:0/817273356 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f274406be30 0x7f274406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:36.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.738+0000 7f27577fe640 1 --2- 172.21.15.32:0/817273356 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f274406be30 0x7f274406e2f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f2748032000 tx=0x7f274802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:36.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.742+0000 7f273ffff640 1 -- 172.21.15.32:0/817273356 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f27581f8e40 con 0x7f275815c580 2024-04-24T23:06:36.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.963+0000 7f275e1b0640 1 -- 172.21.15.32:0/817273356 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f27580afd60 con 0x7f275815c580 2024-04-24T23:06:36.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.965+0000 7f273ffff640 1 -- 172.21.15.32:0/817273356 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v65) ==== 53+0+73 (secure 0 0 0) 0x7f2750019030 con 0x7f275815c580 2024-04-24T23:06:36.966 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T23:06:36.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.967+0000 7f275e1b0640 1 -- 172.21.15.32:0/817273356 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f274406be30 msgr2=0x7f274406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:36.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.967+0000 7f275e1b0640 1 --2- 172.21.15.32:0/817273356 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f274406be30 0x7f274406e2f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f2748032000 tx=0x7f274802e000 comp rx=0 tx=0).stop 2024-04-24T23:06:36.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.967+0000 7f275e1b0640 1 -- 172.21.15.32:0/817273356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f275815c580 msgr2=0x7f2758146e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:36.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.967+0000 7f275e1b0640 1 --2- 172.21.15.32:0/817273356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f275815c580 0x7f2758146e20 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f2758072ea0 tx=0x7f275000f580 comp rx=0 tx=0).stop 2024-04-24T23:06:36.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.967+0000 7f275e1b0640 1 -- 172.21.15.32:0/817273356 shutdown_connections 2024-04-24T23:06:36.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.967+0000 7f275e1b0640 1 --2- 172.21.15.32:0/817273356 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f274406be30 0x7f274406e2f0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:36.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.967+0000 7f275e1b0640 1 --2- 172.21.15.32:0/817273356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f275815c580 0x7f2758146e20 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:36.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.967+0000 7f275e1b0640 1 --2- 172.21.15.32:0/817273356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f275815c100 0x7f27581468e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:36.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.967+0000 7f275e1b0640 1 --2- 172.21.15.32:0/817273356 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f27400039b0 0x7f27581f8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:36.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.967+0000 7f275e1b0640 1 -- 172.21.15.32:0/817273356 >> 172.21.15.32:0/817273356 conn(0x7f275806b110 msgr2=0x7f27580b33f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:36.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.967+0000 7f275e1b0640 1 -- 172.21.15.32:0/817273356 shutdown_connections 2024-04-24T23:06:36.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:36.967+0000 7f275e1b0640 1 -- 172.21.15.32:0/817273356 wait complete. 2024-04-24T23:06:36.979 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-24T23:06:36.980 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T23:06:36.980 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2024-04-24T23:06:36.980 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-24T23:06:36.980 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T23:06:36.980 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-24T23:06:36.980 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-24T23:06:36.980 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T23:06:36.980 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:06:36.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:06:36.980 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2024-04-24T23:06:36.996 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T23:06:36.996 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T23:06:36.996 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T23:06:36.996 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:06:36.996 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2024-04-24T23:06:36.997 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:06:36.997 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:06:36.997 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2024-04-24T23:06:36.997 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2024-04-24T23:06:36.997 INFO:teuthology.orchestra.run.smithi032.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:06:36.997 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2024-04-24T23:06:36.997 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2024-04-24T23:06:36.997 INFO:teuthology.orchestra.run.smithi032.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T23:06:36.998 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:06:36.998 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2024-04-24T23:06:36.998 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-24T23:06:36.998 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 42311sec preferred_lft 42311sec 2024-04-24T23:06:36.998 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link noprefixroute 2024-04-24T23:06:36.998 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:06:36.998 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:06:36.998 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:06:36.998 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2024-04-24T23:06:36.998 INFO:teuthology.orchestra.run.smithi032.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T23:06:36.998 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 72:d4:96:00:43:8c brd ff:ff:ff:ff:ff:ff 2024-04-24T23:06:36.998 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-24T23:06:36.998 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:06:36.998 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::f4c6:20ff:fe22:2647/64 scope link 2024-04-24T23:06:36.999 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:06:36.999 INFO:teuthology.orchestra.run.smithi032.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:06:36.999 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 72:d4:96:00:43:8c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T23:06:36.999 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::70d4:96ff:fe00:438c/64 scope link 2024-04-24T23:06:36.999 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:06:36.999 INFO:teuthology.orchestra.run.smithi032.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:06:36.999 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether e6:6b:2a:90:1a:f8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T23:06:36.999 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::e46b:2aff:fe90:1af8/64 scope link 2024-04-24T23:06:36.999 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:06:36.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:06:36.999 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T23:06:37.059 INFO:teuthology.orchestra.run.smithi032.stdout:1 2024-04-24T23:06:37.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:06:37.061 DEBUG:teuthology.orchestra.run.smithi032:> route 2024-04-24T23:06:37.117 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2024-04-24T23:06:37.118 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T23:06:37.118 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-24T23:06:37.118 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-24T23:06:37.118 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T23:06:37.119 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:06:37.119 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-24T23:06:37.119 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-24T23:06:37.119 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-24T23:06:37.119 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-24T23:06:37.119 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-24T23:06:37.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:06:37.233 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:06:37.235 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:06:37.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:06:37.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:06:37.245 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-04-24T23:06:37.340 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-24T23:06:37.340 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-24T23:06:37.340 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:06:37.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:06:37.340 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2024-04-24T23:06:37.405 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:06:37.405 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-24T23:06:37.461 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:06:37.461 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:06:37.461 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-24T23:06:37.516 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-24T23:06:37.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:06:37.516 DEBUG:teuthology.orchestra.run.smithi032:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:06:37.598 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:06:37.599 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-24T23:06:37.613 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:06:37.613 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-24T23:06:37.613 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.121:3300,172.21.15.121:3301" 2024-04-24T23:06:37.613 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: trying mount with new device syntax: 0@eb1b7a01-6f65-4534-9a12-83011862ba51.cephfs=/ 2024-04-24T23:06:37.613 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.32:3300/172.21.15.121:3300/172.21.15.121:3301" will pass to kernel 2024-04-24T23:06:37.614 INFO:teuthology.orchestra.run.smithi032.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-24T23:06:37.614 INFO:teuthology.orchestra.run.smithi032.stdout: You just mounted a file system that supports labels which does not 2024-04-24T23:06:37.614 INFO:teuthology.orchestra.run.smithi032.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-24T23:06:37.614 INFO:teuthology.orchestra.run.smithi032.stdout: applications will generate AVC messages and not be allowed access to 2024-04-24T23:06:37.614 INFO:teuthology.orchestra.run.smithi032.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-24T23:06:37.616 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-24T23:06:37.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:06:37.616 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-24T23:06:37.679 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:06:37.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:06:37.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:06:37.781 INFO:teuthology.orchestra.run.smithi032.stdout:client6557 2024-04-24T23:06:37.781 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:06:37.781 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:06:37.797 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T23:06:37.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.936+0000 7ff40b264640 1 Processor -- start 2024-04-24T23:06:37.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.936+0000 7ff40b264640 1 -- start start 2024-04-24T23:06:37.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.937+0000 7ff40b264640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff404074210 0x7ff40406c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:37.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.937+0000 7ff40b264640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4040b0770 con 0x7ff4040ae140 2024-04-24T23:06:37.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.937+0000 7ff40b264640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4040b1010 con 0x7ff4040b0b80 2024-04-24T23:06:37.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.937+0000 7ff40b264640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff4040b1c20 con 0x7ff404074210 2024-04-24T23:06:37.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.937+0000 7ff409801640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff4040ae140 0x7ff4040b20b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33148/0 (socket says 172.21.15.32:33148) 2024-04-24T23:06:37.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.937+0000 7ff409801640 1 -- 172.21.15.32:0/3737345102 learned_addr learned my addr 172.21.15.32:0/3737345102 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:06:37.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.937+0000 7ff409000640 1 --2- 172.21.15.32:0/3737345102 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff404074210 0x7ff40406c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:37.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.938+0000 7ff4037fe640 1 -- 172.21.15.32:0/3737345102 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 569983790 0 0) 0x7ff4040b0770 con 0x7ff4040ae140 2024-04-24T23:06:37.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.938+0000 7ff4037fe640 1 -- 172.21.15.32:0/3737345102 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff3ec003520 con 0x7ff4040ae140 2024-04-24T23:06:37.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.938+0000 7ff4037fe640 1 -- 172.21.15.32:0/3737345102 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2243408553 0 0) 0x7ff4040b1010 con 0x7ff4040b0b80 2024-04-24T23:06:37.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.938+0000 7ff4037fe640 1 -- 172.21.15.32:0/3737345102 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff4040b0770 con 0x7ff4040b0b80 2024-04-24T23:06:37.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.938+0000 7ff409000640 1 -- 172.21.15.32:0/3737345102 >> v1:172.21.15.121:6789/0 conn(0x7ff4040b0b80 legacy=0x7ff4040745f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:37.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.938+0000 7ff409000640 1 -- 172.21.15.32:0/3737345102 >> v1:172.21.15.32:6789/0 conn(0x7ff4040ae140 legacy=0x7ff4040b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:37.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.938+0000 7ff409000640 1 -- 172.21.15.32:0/3737345102 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4040b12f0 con 0x7ff404074210 2024-04-24T23:06:37.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.939+0000 7ff409000640 1 --2- 172.21.15.32:0/3737345102 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff404074210 0x7ff40406c480 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4009dc0 tx=0x7ff3f402a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a6e40798edf3992 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:37.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.939+0000 7ff4037fe640 1 -- 172.21.15.32:0/3737345102 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f4007740 con 0x7ff404074210 2024-04-24T23:06:37.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.939+0000 7ff4037fe640 1 -- 172.21.15.32:0/3737345102 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3f40078e0 con 0x7ff404074210 2024-04-24T23:06:37.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.939+0000 7ff4037fe640 1 -- 172.21.15.32:0/3737345102 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f4007c90 con 0x7ff404074210 2024-04-24T23:06:37.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.939+0000 7ff40b264640 1 -- 172.21.15.32:0/3737345102 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff404074210 msgr2=0x7ff40406c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:37.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.939+0000 7ff40b264640 1 --2- 172.21.15.32:0/3737345102 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff404074210 0x7ff40406c480 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4009dc0 tx=0x7ff3f402a6d0 comp rx=0 tx=0).stop 2024-04-24T23:06:37.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.939+0000 7ff40b264640 1 -- 172.21.15.32:0/3737345102 shutdown_connections 2024-04-24T23:06:37.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.939+0000 7ff40b264640 1 --2- 172.21.15.32:0/3737345102 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff404074210 0x7ff40406c480 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:37.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.939+0000 7ff40b264640 1 -- 172.21.15.32:0/3737345102 >> 172.21.15.32:0/3737345102 conn(0x7ff4040b2e60 msgr2=0x7ff4040b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:37.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.939+0000 7ff40b264640 1 -- 172.21.15.32:0/3737345102 shutdown_connections 2024-04-24T23:06:37.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.940+0000 7ff40b264640 1 -- 172.21.15.32:0/3737345102 wait complete. 2024-04-24T23:06:37.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.941+0000 7ff40b264640 1 Processor -- start 2024-04-24T23:06:37.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.941+0000 7ff40b264640 1 -- start start 2024-04-24T23:06:37.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.942+0000 7ff40b264640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff404074210 0x7ff4041f8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:37.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.942+0000 7ff40b264640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4040ae140 0x7ff4040af570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:37.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.942+0000 7ff40b264640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4041e8300 0x7ff4040afab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:37.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.942+0000 7ff40b264640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff4040b3ba0 con 0x7ff4041e8300 2024-04-24T23:06:37.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.942+0000 7ff40b264640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff4040b18b0 con 0x7ff4040ae140 2024-04-24T23:06:37.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.942+0000 7ff40b264640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff4040b3d20 con 0x7ff404074210 2024-04-24T23:06:37.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.942+0000 7ff409801640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4041e8300 0x7ff4040afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:37.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.942+0000 7ff409801640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4041e8300 0x7ff4040afab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48666/0 (socket says 172.21.15.32:48666) 2024-04-24T23:06:37.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.942+0000 7ff409801640 1 -- 172.21.15.32:0/1420564869 learned_addr learned my addr 172.21.15.32:0/1420564869 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:37.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.942+0000 7ff409000640 1 --2- 172.21.15.32:0/1420564869 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff404074210 0x7ff4041f8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:37.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.942+0000 7ff403fff640 1 --2- 172.21.15.32:0/1420564869 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4040ae140 0x7ff4040af570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:37.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.942+0000 7ff409801640 1 -- 172.21.15.32:0/1420564869 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff404074210 msgr2=0x7ff4041f8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:37.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.942+0000 7ff409801640 1 --2- 172.21.15.32:0/1420564869 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff404074210 0x7ff4041f8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:37.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.942+0000 7ff409801640 1 -- 172.21.15.32:0/1420564869 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4040ae140 msgr2=0x7ff4040af570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:37.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.942+0000 7ff409801640 1 --2- 172.21.15.32:0/1420564869 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4040ae140 0x7ff4040af570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:37.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.942+0000 7ff409801640 1 -- 172.21.15.32:0/1420564869 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4041ea1d0 con 0x7ff4041e8300 2024-04-24T23:06:37.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.942+0000 7ff409801640 1 --2- 172.21.15.32:0/1420564869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4041e8300 0x7ff4040afab0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc00c460 tx=0x7ff3fc00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:37.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.942+0000 7ff4017fa640 1 -- 172.21.15.32:0/1420564869 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc017070 con 0x7ff4041e8300 2024-04-24T23:06:37.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.942+0000 7ff4017fa640 1 -- 172.21.15.32:0/1420564869 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3fc005ce0 con 0x7ff4041e8300 2024-04-24T23:06:37.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.942+0000 7ff4017fa640 1 -- 172.21.15.32:0/1420564869 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc007290 con 0x7ff4041e8300 2024-04-24T23:06:37.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.942+0000 7ff40b264640 1 -- 172.21.15.32:0/1420564869 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4041e9360 con 0x7ff4041e8300 2024-04-24T23:06:37.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.942+0000 7ff40b264640 1 -- 172.21.15.32:0/1420564869 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4041e8ac0 con 0x7ff4041e8300 2024-04-24T23:06:37.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.943+0000 7ff40b264640 1 -- 172.21.15.32:0/1420564869 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff404068430 con 0x7ff4041e8300 2024-04-24T23:06:37.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.944+0000 7ff4017fa640 1 -- 172.21.15.32:0/1420564869 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff3fc01a070 con 0x7ff4041e8300 2024-04-24T23:06:37.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.944+0000 7ff4017fa640 1 --2- 172.21.15.32:0/1420564869 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff3f006be30 0x7ff3f006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:37.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.945+0000 7ff4017fa640 1 -- 172.21.15.32:0/1420564869 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7ff3fc08a160 con 0x7ff4041e8300 2024-04-24T23:06:37.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.945+0000 7ff409000640 1 --2- 172.21.15.32:0/1420564869 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff3f006be30 0x7ff3f006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:37.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.945+0000 7ff409000640 1 --2- 172.21.15.32:0/1420564869 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff3f006be30 0x7ff3f006e2f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4034000 tx=0x7ff3f4033040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:37.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:37.949+0000 7ff4017fa640 1 -- 172.21.15.32:0/1420564869 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff3fc058ba0 con 0x7ff4041e8300 2024-04-24T23:06:38.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:38.169+0000 7ff40b264640 1 -- 172.21.15.32:0/1420564869 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff40406b220 con 0x7ff4041e8300 2024-04-24T23:06:38.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:38.169+0000 7ff4017fa640 1 -- 172.21.15.32:0/1420564869 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v65) ==== 53+0+73 (secure 0 0 0) 0x7ff3fc0584a0 con 0x7ff4041e8300 2024-04-24T23:06:38.261 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T23:06:38.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:38.171+0000 7ff40b264640 1 -- 172.21.15.32:0/1420564869 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff3f006be30 msgr2=0x7ff3f006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:38.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:38.171+0000 7ff40b264640 1 --2- 172.21.15.32:0/1420564869 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff3f006be30 0x7ff3f006e2f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4034000 tx=0x7ff3f4033040 comp rx=0 tx=0).stop 2024-04-24T23:06:38.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:38.171+0000 7ff40b264640 1 -- 172.21.15.32:0/1420564869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4041e8300 msgr2=0x7ff4040afab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:38.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:38.171+0000 7ff40b264640 1 --2- 172.21.15.32:0/1420564869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4041e8300 0x7ff4040afab0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc00c460 tx=0x7ff3fc00c920 comp rx=0 tx=0).stop 2024-04-24T23:06:38.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:38.171+0000 7ff40b264640 1 -- 172.21.15.32:0/1420564869 shutdown_connections 2024-04-24T23:06:38.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:38.171+0000 7ff40b264640 1 --2- 172.21.15.32:0/1420564869 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff3f006be30 0x7ff3f006e2f0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:38.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:38.171+0000 7ff40b264640 1 --2- 172.21.15.32:0/1420564869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4041e8300 0x7ff4040afab0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:38.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:38.171+0000 7ff40b264640 1 --2- 172.21.15.32:0/1420564869 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4040ae140 0x7ff4040af570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:38.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:38.171+0000 7ff40b264640 1 --2- 172.21.15.32:0/1420564869 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff404074210 0x7ff4041f8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:38.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:38.171+0000 7ff40b264640 1 -- 172.21.15.32:0/1420564869 >> 172.21.15.32:0/1420564869 conn(0x7ff4040b2e60 msgr2=0x7ff40406e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:38.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:38.171+0000 7ff40b264640 1 -- 172.21.15.32:0/1420564869 shutdown_connections 2024-04-24T23:06:38.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:38.171+0000 7ff40b264640 1 -- 172.21.15.32:0/1420564869 wait complete. 2024-04-24T23:06:38.264 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-24T23:06:38.264 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T23:06:38.265 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi121.front.sepia.ceph.com 2024-04-24T23:06:38.265 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-24T23:06:38.265 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-24T23:06:38.265 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-24T23:06:38.265 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-24T23:06:38.265 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T23:06:38.265 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-24T23:06:38.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:06:38.265 DEBUG:teuthology.orchestra.run.smithi121:> ip addr 2024-04-24T23:06:38.285 INFO:teuthology.orchestra.run.smithi121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T23:06:38.285 INFO:teuthology.orchestra.run.smithi121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T23:06:38.285 INFO:teuthology.orchestra.run.smithi121.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T23:06:38.285 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:06:38.285 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 ::1/128 scope host 2024-04-24T23:06:38.285 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:06:38.286 INFO:teuthology.orchestra.run.smithi121.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:06:38.286 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:82 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:06:38.286 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f0 2024-04-24T23:06:38.286 INFO:teuthology.orchestra.run.smithi121.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:06:38.286 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:83 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:06:38.287 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f1 2024-04-24T23:06:38.287 INFO:teuthology.orchestra.run.smithi121.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:06:38.287 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a2 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:06:38.287 INFO:teuthology.orchestra.run.smithi121.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T23:06:38.287 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a3 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:06:38.287 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.21.15.121/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-24T23:06:38.287 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft 42333sec preferred_lft 42333sec 2024-04-24T23:06:38.287 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::93dd:8489:4369:e127/64 scope link noprefixroute 2024-04-24T23:06:38.287 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:06:38.288 INFO:teuthology.orchestra.run.smithi121.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T23:06:38.288 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether c6:c0:c7:45:ae:ba brd ff:ff:ff:ff:ff:ff 2024-04-24T23:06:38.288 INFO:teuthology.orchestra.run.smithi121.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-24T23:06:38.288 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:06:38.288 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::80fe:d2ff:fe13:d1/64 scope link 2024-04-24T23:06:38.288 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:06:38.289 INFO:teuthology.orchestra.run.smithi121.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:06:38.289 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether e2:ca:b9:e9:9e:bb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T23:06:38.289 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::e0ca:b9ff:fee9:9ebb/64 scope link 2024-04-24T23:06:38.289 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:06:38.289 INFO:teuthology.orchestra.run.smithi121.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:06:38.289 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether c6:c0:c7:45:ae:ba brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-24T23:06:38.289 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::c4c0:c7ff:fe45:aeba/64 scope link 2024-04-24T23:06:38.289 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:06:38.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:06:38.290 DEBUG:teuthology.orchestra.run.smithi121:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T23:06:38.353 INFO:teuthology.orchestra.run.smithi121.stdout:1 2024-04-24T23:06:38.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:06:38.354 DEBUG:teuthology.orchestra.run.smithi121:> route 2024-04-24T23:06:38.415 INFO:teuthology.orchestra.run.smithi121.stdout:Kernel IP routing table 2024-04-24T23:06:38.415 INFO:teuthology.orchestra.run.smithi121.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T23:06:38.415 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-24T23:06:38.415 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-24T23:06:38.415 INFO:teuthology.orchestra.run.smithi121.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T23:06:38.416 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:06:38.416 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2024-04-24T23:06:38.416 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-24T23:06:38.416 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-24T23:06:38.416 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-24T23:06:38.416 DEBUG:teuthology.orchestra.run.smithi121:> ') 2024-04-24T23:06:38.509 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T23:06:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:06:38.541 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T23:06:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:06:38.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:06:38.548 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2024-04-24T23:06:38.564 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-24T23:06:38.564 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-24T23:06:38.564 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-24T23:06:38.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:06:38.565 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set brx.0 up 2024-04-24T23:06:38.628 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:06:38.628 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-24T23:06:38.683 INFO:teuthology.orchestra.run.smithi121.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-24T23:06:38.684 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:06:38.684 DEBUG:teuthology.orchestra.run.smithi121:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-24T23:06:38.738 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-24T23:06:38.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:06:38.738 DEBUG:teuthology.orchestra.run.smithi121:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:06:38.822 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:06:38.822 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-24T23:06:38.839 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:06:38.839 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-24T23:06:38.839 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.121:3300,172.21.15.121:3301" 2024-04-24T23:06:38.839 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: trying mount with new device syntax: 1@eb1b7a01-6f65-4534-9a12-83011862ba51.cephfs=/ 2024-04-24T23:06:38.839 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.32:3300/172.21.15.121:3300/172.21.15.121:3301" will pass to kernel 2024-04-24T23:06:38.840 INFO:teuthology.orchestra.run.smithi121.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-24T23:06:38.840 INFO:teuthology.orchestra.run.smithi121.stdout: You just mounted a file system that supports labels which does not 2024-04-24T23:06:38.840 INFO:teuthology.orchestra.run.smithi121.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-24T23:06:38.840 INFO:teuthology.orchestra.run.smithi121.stdout: applications will generate AVC messages and not be allowed access to 2024-04-24T23:06:38.840 INFO:teuthology.orchestra.run.smithi121.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-24T23:06:38.841 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-24T23:06:38.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:06:38.841 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-24T23:06:38.905 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:06:38.994 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T23:06:38 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:06:38.995 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T23:06:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:06:39.001 INFO:teuthology.orchestra.run.smithi121.stdout:client7249 2024-04-24T23:06:39.002 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:06:39.002 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:06:39.018 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:06:39.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.128+0000 7fddc3d9e640 1 Processor -- start 2024-04-24T23:06:39.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.128+0000 7fddc3d9e640 1 -- start start 2024-04-24T23:06:39.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.129+0000 7fddc3d9e640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fddbc06cd00 0x7fddbc0ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:39.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.129+0000 7fddc3d9e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddbc0b4520 con 0x7fddbc15b380 2024-04-24T23:06:39.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.129+0000 7fddc3d9e640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddbc0b4800 con 0x7fddbc15cb00 2024-04-24T23:06:39.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.129+0000 7fddc3d9e640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fddbc0b4ae0 con 0x7fddbc06cd00 2024-04-24T23:06:39.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.129+0000 7fddc1b3a640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fddbc15b380 0x7fddbc06c5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33150/0 (socket says 172.21.15.32:33150) 2024-04-24T23:06:39.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.129+0000 7fddc1b3a640 1 -- 172.21.15.32:0/2479598339 learned_addr learned my addr 172.21.15.32:0/2479598339 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:06:39.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.129+0000 7fddc1339640 1 --2- 172.21.15.32:0/2479598339 >> v2:172.21.15.121:3300/0 conn(0x7fddbc06cd00 0x7fddbc0ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:39.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.129+0000 7fddc0b38640 1 -- 172.21.15.32:0/2479598339 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1848246120 0 0) 0x7fddbc0b4520 con 0x7fddbc15b380 2024-04-24T23:06:39.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.129+0000 7fddc0b38640 1 -- 172.21.15.32:0/2479598339 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdda8003520 con 0x7fddbc15b380 2024-04-24T23:06:39.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.130+0000 7fddc0b38640 1 -- 172.21.15.32:0/2479598339 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3317631975 0 0) 0x7fddbc0b4800 con 0x7fddbc15cb00 2024-04-24T23:06:39.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.130+0000 7fddc0b38640 1 -- 172.21.15.32:0/2479598339 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fddbc0b4520 con 0x7fddbc15cb00 2024-04-24T23:06:39.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.130+0000 7fddc1339640 1 -- 172.21.15.32:0/2479598339 >> v1:172.21.15.121:6789/0 conn(0x7fddbc15cb00 legacy=0x7fddbc0ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:39.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.130+0000 7fddc1339640 1 -- 172.21.15.32:0/2479598339 >> v1:172.21.15.32:6789/0 conn(0x7fddbc15b380 legacy=0x7fddbc06c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:39.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.130+0000 7fddc0b38640 1 -- 172.21.15.32:0/2479598339 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4141416919 0 0) 0x7fdda8003520 con 0x7fddbc15b380 2024-04-24T23:06:39.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.130+0000 7fddc1339640 1 -- 172.21.15.32:0/2479598339 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddbc0b4dc0 con 0x7fddbc06cd00 2024-04-24T23:06:39.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.130+0000 7fddc1339640 1 --2- 172.21.15.32:0/2479598339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fddbc06cd00 0x7fddbc0ce430 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fddac0097a0 tx=0x7fddac02a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c0a2acbced827fa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:39.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.131+0000 7fddc0b38640 1 -- 172.21.15.32:0/2479598339 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac037070 con 0x7fddbc06cd00 2024-04-24T23:06:39.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.131+0000 7fddc0b38640 1 -- 172.21.15.32:0/2479598339 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddac02abc0 con 0x7fddbc06cd00 2024-04-24T23:06:39.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.131+0000 7fddc0b38640 1 -- 172.21.15.32:0/2479598339 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb8001670 con 0x7fddbc06cd00 2024-04-24T23:06:39.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.131+0000 7fddc3d9e640 1 -- 172.21.15.32:0/2479598339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fddbc06cd00 msgr2=0x7fddbc0ce430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:39.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.131+0000 7fddc3d9e640 1 --2- 172.21.15.32:0/2479598339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fddbc06cd00 0x7fddbc0ce430 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fddac0097a0 tx=0x7fddac02a130 comp rx=0 tx=0).stop 2024-04-24T23:06:39.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.131+0000 7fddc3d9e640 1 -- 172.21.15.32:0/2479598339 shutdown_connections 2024-04-24T23:06:39.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.131+0000 7fddc3d9e640 1 --2- 172.21.15.32:0/2479598339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fddbc06cd00 0x7fddbc0ce430 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:39.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.131+0000 7fddc3d9e640 1 -- 172.21.15.32:0/2479598339 >> 172.21.15.32:0/2479598339 conn(0x7fddbc06b110 msgr2=0x7fddbc06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:39.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.131+0000 7fddc3d9e640 1 -- 172.21.15.32:0/2479598339 shutdown_connections 2024-04-24T23:06:39.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.131+0000 7fddc3d9e640 1 -- 172.21.15.32:0/2479598339 wait complete. 2024-04-24T23:06:39.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.133+0000 7fddc3d9e640 1 Processor -- start 2024-04-24T23:06:39.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.133+0000 7fddc3d9e640 1 -- start start 2024-04-24T23:06:39.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.133+0000 7fddc3d9e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fddbc06cd00 0x7fddbc147080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:39.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.133+0000 7fddc3d9e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fddbc15b380 0x7fddbc0ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:39.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.133+0000 7fddc3d9e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddbc15cb00 0x7fddbc0aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:39.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.133+0000 7fddc3d9e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fddbc0b55c0 con 0x7fddbc15cb00 2024-04-24T23:06:39.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.133+0000 7fddc3d9e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fddbc15b760 con 0x7fddbc06cd00 2024-04-24T23:06:39.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.133+0000 7fddc3d9e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fddbc0b5740 con 0x7fddbc15b380 2024-04-24T23:06:39.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.134+0000 7fddc233b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddbc15cb00 0x7fddbc0aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:39.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.134+0000 7fddc233b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddbc15cb00 0x7fddbc0aedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48670/0 (socket says 172.21.15.32:48670) 2024-04-24T23:06:39.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.134+0000 7fddc233b640 1 -- 172.21.15.32:0/3312935236 learned_addr learned my addr 172.21.15.32:0/3312935236 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:39.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.134+0000 7fddc1b3a640 1 --2- 172.21.15.32:0/3312935236 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fddbc06cd00 0x7fddbc147080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:39.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.134+0000 7fddc1339640 1 --2- 172.21.15.32:0/3312935236 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fddbc15b380 0x7fddbc0ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:39.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.134+0000 7fddc233b640 1 -- 172.21.15.32:0/3312935236 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fddbc15b380 msgr2=0x7fddbc0ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:39.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.134+0000 7fddc233b640 1 --2- 172.21.15.32:0/3312935236 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fddbc15b380 0x7fddbc0ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:39.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.134+0000 7fddc233b640 1 -- 172.21.15.32:0/3312935236 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fddbc06cd00 msgr2=0x7fddbc147080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:39.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.134+0000 7fddc233b640 1 --2- 172.21.15.32:0/3312935236 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fddbc06cd00 0x7fddbc147080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:39.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.134+0000 7fddc233b640 1 -- 172.21.15.32:0/3312935236 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddbc1485a0 con 0x7fddbc15cb00 2024-04-24T23:06:39.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.134+0000 7fddc233b640 1 --2- 172.21.15.32:0/3312935236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddbc15cb00 0x7fddbc0aedf0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fddb400c6e0 tx=0x7fddb400cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:39.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.134+0000 7fddb27fc640 1 -- 172.21.15.32:0/3312935236 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb4003b70 con 0x7fddbc15cb00 2024-04-24T23:06:39.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.134+0000 7fddb27fc640 1 -- 172.21.15.32:0/3312935236 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddb4003d10 con 0x7fddbc15cb00 2024-04-24T23:06:39.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.134+0000 7fddc3d9e640 1 -- 172.21.15.32:0/3312935236 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddbc071370 con 0x7fddbc15cb00 2024-04-24T23:06:39.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.134+0000 7fddb27fc640 1 -- 172.21.15.32:0/3312935236 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb40052d0 con 0x7fddbc15cb00 2024-04-24T23:06:39.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.134+0000 7fddc3d9e640 1 -- 172.21.15.32:0/3312935236 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddbc147ac0 con 0x7fddbc15cb00 2024-04-24T23:06:39.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.135+0000 7fddc3d9e640 1 -- 172.21.15.32:0/3312935236 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddbc068430 con 0x7fddbc15cb00 2024-04-24T23:06:39.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.136+0000 7fddb27fc640 1 -- 172.21.15.32:0/3312935236 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fddb4005470 con 0x7fddbc15cb00 2024-04-24T23:06:39.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.136+0000 7fddb27fc640 1 --2- 172.21.15.32:0/3312935236 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdda006be30 0x7fdda006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:39.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.136+0000 7fddb27fc640 1 -- 172.21.15.32:0/3312935236 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7fddb408ac60 con 0x7fddbc15cb00 2024-04-24T23:06:39.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.136+0000 7fddc1b3a640 1 --2- 172.21.15.32:0/3312935236 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdda006be30 0x7fdda006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:39.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.137+0000 7fddc1b3a640 1 --2- 172.21.15.32:0/3312935236 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdda006be30 0x7fdda006e2f0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fddb80017e0 tx=0x7fddb8001610 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:39.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.141+0000 7fddb27fc640 1 -- 172.21.15.32:0/3312935236 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fddb4058f40 con 0x7fddbc15cb00 2024-04-24T23:06:39.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.359+0000 7fddc3d9e640 1 -- 172.21.15.32:0/3312935236 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fddbc0afd60 con 0x7fddbc15cb00 2024-04-24T23:06:39.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.360+0000 7fddb27fc640 1 -- 172.21.15.32:0/3312935236 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+5186 (secure 0 0 0) 0x7fddb405d240 con 0x7fddbc15cb00 2024-04-24T23:06:39.361 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:06:39.362 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":65,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7120,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/1196305174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1196305174},{"type":"v1","addr":"172.21.15.121:6836","nonce":1196305174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7123,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3015902884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3015902884},{"type":"v1","addr":"172.21.15.121:6837","nonce":3015902884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":8280,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/1033273628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1033273628},{"type":"v1","addr":"172.21.15.32:6833","nonce":1033273628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:06:31.291599+0000","modified":"2024-04-24T23:06:33.296657+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8286},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8286":{"gid":8286,"name":"a","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/984603360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":984603360},{"type":"v1","addr":"172.21.15.32:6835","nonce":984603360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8286,"qdb_cluster":[8286]},"id":9}]} 2024-04-24T23:06:39.362 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 65 2024-04-24T23:06:39.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.362+0000 7fddc3d9e640 1 -- 172.21.15.32:0/3312935236 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdda006be30 msgr2=0x7fdda006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:39.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.362+0000 7fddc3d9e640 1 --2- 172.21.15.32:0/3312935236 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdda006be30 0x7fdda006e2f0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fddb80017e0 tx=0x7fddb8001610 comp rx=0 tx=0).stop 2024-04-24T23:06:39.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.362+0000 7fddc3d9e640 1 -- 172.21.15.32:0/3312935236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddbc15cb00 msgr2=0x7fddbc0aedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:39.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.362+0000 7fddc3d9e640 1 --2- 172.21.15.32:0/3312935236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddbc15cb00 0x7fddbc0aedf0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fddb400c6e0 tx=0x7fddb400cba0 comp rx=0 tx=0).stop 2024-04-24T23:06:39.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.362+0000 7fddc3d9e640 1 -- 172.21.15.32:0/3312935236 shutdown_connections 2024-04-24T23:06:39.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.362+0000 7fddc3d9e640 1 --2- 172.21.15.32:0/3312935236 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdda006be30 0x7fdda006e2f0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:39.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.362+0000 7fddc3d9e640 1 --2- 172.21.15.32:0/3312935236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddbc15cb00 0x7fddbc0aedf0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:39.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.362+0000 7fddc3d9e640 1 --2- 172.21.15.32:0/3312935236 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fddbc15b380 0x7fddbc0ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:39.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.362+0000 7fddc3d9e640 1 --2- 172.21.15.32:0/3312935236 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fddbc06cd00 0x7fddbc147080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:39.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.362+0000 7fddc3d9e640 1 -- 172.21.15.32:0/3312935236 >> 172.21.15.32:0/3312935236 conn(0x7fddbc06b110 msgr2=0x7fddbc157380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:39.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.362+0000 7fddc3d9e640 1 -- 172.21.15.32:0/3312935236 shutdown_connections 2024-04-24T23:06:39.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.362+0000 7fddc3d9e640 1 -- 172.21.15.32:0/3312935236 wait complete. 2024-04-24T23:06:39.374 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-04-24T23:06:39.494 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","whoami":0,"id":8286,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":7.5410169260000002,"mdsmap_epoch":65,"osdmap_epoch":156,"osdmap_epoch_barrier":156,"uptime":12.551699077} 2024-04-24T23:06:39.495 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", "fs_name": "cephfs", "id": 8286, "mdsmap_epoch": 65, "osdmap_epoch": 156, "osdmap_epoch_barrier": 156, "rank_uptime": 7.541016926, "state": "up:active", "uptime": 12.551699077, "want_state": "up:active", "whoami": 0 } 2024-04-24T23:06:39.495 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2024-04-24T23:06:39.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.598+0000 7f2a20cb2640 1 Processor -- start 2024-04-24T23:06:39.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.598+0000 7f2a20cb2640 1 -- start start 2024-04-24T23:06:39.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.599+0000 7f2a20cb2640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2a1c0b9ff0 0x7f2a1c0d40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:39.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.599+0000 7f2a20cb2640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2a1c0b6450 0x7f2a1c0b6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:39.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.599+0000 7f2a20cb2640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a1c0d3000 con 0x7f2a1c15e770 2024-04-24T23:06:39.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.599+0000 7f2a20cb2640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f2a1c0d32e0 con 0x7f2a1c0b9ff0 2024-04-24T23:06:39.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.599+0000 7f2a20cb2640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f2a1c0d35c0 con 0x7f2a1c0b6450 2024-04-24T23:06:39.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.599+0000 7f2a19d9b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2a1c0b9ff0 0x7f2a1c0d40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:39.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.599+0000 7f2a19d9b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2a1c0b9ff0 0x7f2a1c0d40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48672/0 (socket says 172.21.15.32:48672) 2024-04-24T23:06:39.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.599+0000 7f2a19d9b640 1 -- 172.21.15.32:0/563515781 learned_addr learned my addr 172.21.15.32:0/563515781 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:39.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.600+0000 7f2a1ad9d640 1 --2- 172.21.15.32:0/563515781 >> v2:172.21.15.121:3300/0 conn(0x7f2a1c0b6450 0x7f2a1c0b6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:39.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.600+0000 7f2a19d9b640 1 -- 172.21.15.32:0/563515781 >> v2:172.21.15.121:3300/0 conn(0x7f2a1c0b6450 msgr2=0x7f2a1c0b6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:39.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.600+0000 7f2a19d9b640 1 --2- 172.21.15.32:0/563515781 >> v2:172.21.15.121:3300/0 conn(0x7f2a1c0b6450 0x7f2a1c0b6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:39.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.600+0000 7f2a19d9b640 1 -- 172.21.15.32:0/563515781 >> v1:172.21.15.121:6789/0 conn(0x7f2a1c15e770 legacy=0x7f2a1c0d2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:06:39.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.600+0000 7f2a19d9b640 1 -- 172.21.15.32:0/563515781 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a1c0d38a0 con 0x7f2a1c0b9ff0 2024-04-24T23:06:39.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.600+0000 7f2a19d9b640 1 --2- 172.21.15.32:0/563515781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a1c0b9ff0 0x7f2a1c0d40a0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c0d0ab0 tx=0x7f2a0402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae9e5168fd6f03a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:39.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.601+0000 7f2a1959a640 1 -- 172.21.15.32:0/563515781 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a04037070 con 0x7f2a1c0b9ff0 2024-04-24T23:06:39.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.601+0000 7f2a1959a640 1 -- 172.21.15.32:0/563515781 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a04032440 con 0x7f2a1c0b9ff0 2024-04-24T23:06:39.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.601+0000 7f2a1959a640 1 -- 172.21.15.32:0/563515781 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a04032810 con 0x7f2a1c0b9ff0 2024-04-24T23:06:39.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.601+0000 7f2a20cb2640 1 -- 172.21.15.32:0/563515781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a1c0b9ff0 msgr2=0x7f2a1c0d40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:39.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.601+0000 7f2a20cb2640 1 --2- 172.21.15.32:0/563515781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a1c0b9ff0 0x7f2a1c0d40a0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c0d0ab0 tx=0x7f2a0402a230 comp rx=0 tx=0).stop 2024-04-24T23:06:39.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.601+0000 7f2a20cb2640 1 -- 172.21.15.32:0/563515781 shutdown_connections 2024-04-24T23:06:39.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.601+0000 7f2a20cb2640 1 --2- 172.21.15.32:0/563515781 >> v2:172.21.15.121:3300/0 conn(0x7f2a1c0b6450 0x7f2a1c0b6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:39.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.601+0000 7f2a20cb2640 1 --2- 172.21.15.32:0/563515781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a1c0b9ff0 0x7f2a1c0d40a0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:39.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.601+0000 7f2a20cb2640 1 -- 172.21.15.32:0/563515781 >> 172.21.15.32:0/563515781 conn(0x7f2a1c071910 msgr2=0x7f2a1c071d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:39.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.601+0000 7f2a20cb2640 1 -- 172.21.15.32:0/563515781 shutdown_connections 2024-04-24T23:06:39.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.601+0000 7f2a20cb2640 1 -- 172.21.15.32:0/563515781 wait complete. 2024-04-24T23:06:39.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.603+0000 7f2a20cb2640 1 Processor -- start 2024-04-24T23:06:39.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.603+0000 7f2a20cb2640 1 -- start start 2024-04-24T23:06:39.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.603+0000 7f2a20cb2640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2a1c0b6450 0x7f2a1c0ca790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:39.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.603+0000 7f2a20cb2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a1c0b9ff0 0x7f2a1c158490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:39.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.603+0000 7f2a20cb2640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2a1c15e770 0x7f2a1c1589d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:39.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.603+0000 7f2a20cb2640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1c15ecd0 con 0x7f2a1c0b9ff0 2024-04-24T23:06:39.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.603+0000 7f2a20cb2640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1c15eb50 con 0x7f2a1c15e770 2024-04-24T23:06:39.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.603+0000 7f2a20cb2640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f2a1c0b7790 con 0x7f2a1c0b6450 2024-04-24T23:06:39.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.603+0000 7f2a19d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a1c0b9ff0 0x7f2a1c158490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:39.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.603+0000 7f2a19d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a1c0b9ff0 0x7f2a1c158490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48676/0 (socket says 172.21.15.32:48676) 2024-04-24T23:06:39.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.603+0000 7f2a19d9b640 1 -- 172.21.15.32:0/3618354337 learned_addr learned my addr 172.21.15.32:0/3618354337 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:06:39.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.604+0000 7f2a1ad9d640 1 --2- 172.21.15.32:0/3618354337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2a1c15e770 0x7f2a1c1589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:39.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.604+0000 7f2a1a59c640 1 --2- 172.21.15.32:0/3618354337 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2a1c0b6450 0x7f2a1c0ca790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:39.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.604+0000 7f2a19d9b640 1 -- 172.21.15.32:0/3618354337 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2a1c0b6450 msgr2=0x7f2a1c0ca790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:39.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.604+0000 7f2a19d9b640 1 --2- 172.21.15.32:0/3618354337 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2a1c0b6450 0x7f2a1c0ca790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:39.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.604+0000 7f2a19d9b640 1 -- 172.21.15.32:0/3618354337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2a1c15e770 msgr2=0x7f2a1c1589d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:39.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.604+0000 7f2a19d9b640 1 --2- 172.21.15.32:0/3618354337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2a1c15e770 0x7f2a1c1589d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:39.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.604+0000 7f2a19d9b640 1 -- 172.21.15.32:0/3618354337 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a1c0cacd0 con 0x7f2a1c0b9ff0 2024-04-24T23:06:39.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.604+0000 7f2a19d9b640 1 --2- 172.21.15.32:0/3618354337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a1c0b9ff0 0x7f2a1c158490 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f2a04000c00 tx=0x7f2a040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:39.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.604+0000 7f2a12ffd640 1 -- 172.21.15.32:0/3618354337 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a04041020 con 0x7f2a1c0b9ff0 2024-04-24T23:06:39.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.604+0000 7f2a12ffd640 1 -- 172.21.15.32:0/3618354337 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a040306c0 con 0x7f2a1c0b9ff0 2024-04-24T23:06:39.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.604+0000 7f2a20cb2640 1 -- 172.21.15.32:0/3618354337 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a1c0cb9c0 con 0x7f2a1c0b9ff0 2024-04-24T23:06:39.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.604+0000 7f2a12ffd640 1 -- 172.21.15.32:0/3618354337 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a04037040 con 0x7f2a1c0b9ff0 2024-04-24T23:06:39.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.604+0000 7f2a20cb2640 1 -- 172.21.15.32:0/3618354337 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a1c1592e0 con 0x7f2a1c0b9ff0 2024-04-24T23:06:39.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.605+0000 7f2a20cb2640 1 -- 172.21.15.32:0/3618354337 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a1c06bd10 con 0x7f2a1c0b9ff0 2024-04-24T23:06:39.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.606+0000 7f2a12ffd640 1 -- 172.21.15.32:0/3618354337 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2a04044020 con 0x7f2a1c0b9ff0 2024-04-24T23:06:39.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.606+0000 7f2a12ffd640 1 --2- 172.21.15.32:0/3618354337 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f29fc06be30 0x7f29fc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:39.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.606+0000 7f2a12ffd640 1 -- 172.21.15.32:0/3618354337 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5709+0+0 (secure 0 0 0) 0x7f2a040ab200 con 0x7f2a1c0b9ff0 2024-04-24T23:06:39.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.607+0000 7f2a1a59c640 1 --2- 172.21.15.32:0/3618354337 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f29fc06be30 0x7f29fc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:39.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.607+0000 7f2a1a59c640 1 --2- 172.21.15.32:0/3618354337 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f29fc06be30 0x7f29fc06e2f0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c1f7d60 tx=0x7f2a14009060 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:39.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.611+0000 7f2a12ffd640 1 -- 172.21.15.32:0/3618354337 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2a1c06bd10 con 0x7f2a1c0b9ff0 2024-04-24T23:06:39.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.796+0000 7f2a20cb2640 1 -- 172.21.15.32:0/3618354337 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) -- 0x7f2a1c068430 con 0x7f2a1c0b9ff0 2024-04-24T23:06:39.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.974+0000 7f2a12ffd640 1 -- 172.21.15.32:0/3618354337 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v157) ==== 181+0+0 (secure 0 0 0) 0x7f2a0404b070 con 0x7f2a1c0b9ff0 2024-04-24T23:06:39.976 INFO:teuthology.orchestra.run.smithi032.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2024-04-24T23:06:39.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.976+0000 7f2a20cb2640 1 -- 172.21.15.32:0/3618354337 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f29fc06be30 msgr2=0x7f29fc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:39.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.976+0000 7f2a20cb2640 1 --2- 172.21.15.32:0/3618354337 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f29fc06be30 0x7f29fc06e2f0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c1f7d60 tx=0x7f2a14009060 comp rx=0 tx=0).stop 2024-04-24T23:06:39.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.976+0000 7f2a20cb2640 1 -- 172.21.15.32:0/3618354337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a1c0b9ff0 msgr2=0x7f2a1c158490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:39.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.976+0000 7f2a20cb2640 1 --2- 172.21.15.32:0/3618354337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a1c0b9ff0 0x7f2a1c158490 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f2a04000c00 tx=0x7f2a040046c0 comp rx=0 tx=0).stop 2024-04-24T23:06:39.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.976+0000 7f2a20cb2640 1 -- 172.21.15.32:0/3618354337 shutdown_connections 2024-04-24T23:06:39.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.976+0000 7f2a20cb2640 1 --2- 172.21.15.32:0/3618354337 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f29fc06be30 0x7f29fc06e2f0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:39.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.976+0000 7f2a20cb2640 1 --2- 172.21.15.32:0/3618354337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a1c0b9ff0 0x7f2a1c158490 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:39.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.976+0000 7f2a20cb2640 1 --2- 172.21.15.32:0/3618354337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2a1c15e770 0x7f2a1c1589d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:39.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.976+0000 7f2a20cb2640 1 --2- 172.21.15.32:0/3618354337 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2a1c0b6450 0x7f2a1c0ca790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:39.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.977+0000 7f2a20cb2640 1 -- 172.21.15.32:0/3618354337 >> 172.21.15.32:0/3618354337 conn(0x7f2a1c071910 msgr2=0x7f2a1c0b5500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:39.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.977+0000 7f2a20cb2640 1 -- 172.21.15.32:0/3618354337 shutdown_connections 2024-04-24T23:06:39.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:06:39.977+0000 7f2a20cb2640 1 -- 172.21.15.32:0/3618354337 wait complete. 2024-04-24T23:06:39.989 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-04-24T23:06:40.103 INFO:teuthology.orchestra.run.smithi032.stdout:{"osd_mon_report_interval":"5"} 2024-04-24T23:06:40.103 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-04-24T23:06:40.104 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2024-04-24T23:06:40.211 INFO:teuthology.orchestra.run.smithi032.stdout:{"mon_tick_interval":"5"} 2024-04-24T23:06:40.212 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2024-04-24T23:06:40.212 INFO:tasks.cephfs.test_full:pool capacity 33554432, 32MB should be enough to fill it 2024-04-24T23:06:40.212 DEBUG:teuthology.orchestra.run.smithi032:> stdin-killer -- python3 -c ' 2024-04-24T23:06:40.212 DEBUG:teuthology.orchestra.run.smithi032:> import time 2024-04-24T23:06:40.212 DEBUG:teuthology.orchestra.run.smithi032:> import datetime 2024-04-24T23:06:40.212 DEBUG:teuthology.orchestra.run.smithi032:> import subprocess 2024-04-24T23:06:40.212 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-24T23:06:40.212 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-24T23:06:40.212 DEBUG:teuthology.orchestra.run.smithi032:> # Write some buffered data through before going full, all should be well 2024-04-24T23:06:40.212 DEBUG:teuthology.orchestra.run.smithi032:> print("writing some data through which we expect to succeed") 2024-04-24T23:06:40.212 DEBUG:teuthology.orchestra.run.smithi032:> bytes = 0 2024-04-24T23:06:40.212 DEBUG:teuthology.orchestra.run.smithi032:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2024-04-24T23:06:40.213 DEBUG:teuthology.orchestra.run.smithi032:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2024-04-24T23:06:40.213 DEBUG:teuthology.orchestra.run.smithi032:> bytes += os.write(f, b'"'"'a'"'"' * 512 * 1024) 2024-04-24T23:06:40.213 DEBUG:teuthology.orchestra.run.smithi032:> os.fsync(f) 2024-04-24T23:06:40.213 DEBUG:teuthology.orchestra.run.smithi032:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2024-04-24T23:06:40.213 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-24T23:06:40.213 DEBUG:teuthology.orchestra.run.smithi032:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2024-04-24T23:06:40.213 DEBUG:teuthology.orchestra.run.smithi032:> # writing until we see an error from fsync. As long as we'"'"'re doing 2024-04-24T23:06:40.213 DEBUG:teuthology.orchestra.run.smithi032:> # buffered IO, the error should always only appear from fsync and not 2024-04-24T23:06:40.213 DEBUG:teuthology.orchestra.run.smithi032:> # from write 2024-04-24T23:06:40.213 DEBUG:teuthology.orchestra.run.smithi032:> full = False 2024-04-24T23:06:40.213 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-24T23:06:40.213 DEBUG:teuthology.orchestra.run.smithi032:> for n in range(0, int(32 * 0.9)): 2024-04-24T23:06:40.213 DEBUG:teuthology.orchestra.run.smithi032:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2024-04-24T23:06:40.213 DEBUG:teuthology.orchestra.run.smithi032:> print("wrote {0} bytes via buffered write, may repeat".format(bytes)) 2024-04-24T23:06:40.213 DEBUG:teuthology.orchestra.run.smithi032:> print("done writing {0} bytes".format(bytes)) 2024-04-24T23:06:40.213 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-24T23:06:40.213 DEBUG:teuthology.orchestra.run.smithi032:> # OK, now we should sneak in under the full condition 2024-04-24T23:06:40.214 DEBUG:teuthology.orchestra.run.smithi032:> # due to the time it takes the OSDs to report to the 2024-04-24T23:06:40.214 DEBUG:teuthology.orchestra.run.smithi032:> # mons, and get a successful fsync on our full-making data 2024-04-24T23:06:40.214 DEBUG:teuthology.orchestra.run.smithi032:> os.fsync(f) 2024-04-24T23:06:40.214 DEBUG:teuthology.orchestra.run.smithi032:> print("successfully fsync'"'"'ed prior to getting full state reported") 2024-04-24T23:06:40.214 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-24T23:06:40.214 DEBUG:teuthology.orchestra.run.smithi032:> # buffered write, add more dirty data to the buffer 2024-04-24T23:06:40.214 DEBUG:teuthology.orchestra.run.smithi032:> print("starting buffered write") 2024-04-24T23:06:40.214 DEBUG:teuthology.orchestra.run.smithi032:> try: 2024-04-24T23:06:40.214 DEBUG:teuthology.orchestra.run.smithi032:> for n in range(0, int(32 * 0.2)): 2024-04-24T23:06:40.214 DEBUG:teuthology.orchestra.run.smithi032:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2024-04-24T23:06:40.214 DEBUG:teuthology.orchestra.run.smithi032:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2024-04-24T23:06:40.214 DEBUG:teuthology.orchestra.run.smithi032:> time.sleep(15.0) 2024-04-24T23:06:40.214 DEBUG:teuthology.orchestra.run.smithi032:> except OSError: 2024-04-24T23:06:40.214 DEBUG:teuthology.orchestra.run.smithi032:> pass; 2024-04-24T23:06:40.214 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-24T23:06:40.215 DEBUG:teuthology.orchestra.run.smithi032:> print("wrote, now waiting 30s and then doing a close we expect to fail") 2024-04-24T23:06:40.215 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-24T23:06:40.215 DEBUG:teuthology.orchestra.run.smithi032:> # Wait long enough for a background flush that should fail 2024-04-24T23:06:40.215 DEBUG:teuthology.orchestra.run.smithi032:> time.sleep(30) 2024-04-24T23:06:40.215 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-24T23:06:40.215 DEBUG:teuthology.orchestra.run.smithi032:> if False: 2024-04-24T23:06:40.215 DEBUG:teuthology.orchestra.run.smithi032:> # ...and check that the failed background flush is reflected in fclose 2024-04-24T23:06:40.215 DEBUG:teuthology.orchestra.run.smithi032:> try: 2024-04-24T23:06:40.215 DEBUG:teuthology.orchestra.run.smithi032:> os.close(f) 2024-04-24T23:06:40.215 DEBUG:teuthology.orchestra.run.smithi032:> except OSError: 2024-04-24T23:06:40.215 DEBUG:teuthology.orchestra.run.smithi032:> print("close() returned an error as expected") 2024-04-24T23:06:40.215 DEBUG:teuthology.orchestra.run.smithi032:> else: 2024-04-24T23:06:40.215 DEBUG:teuthology.orchestra.run.smithi032:> raise RuntimeError("close() failed to raise error") 2024-04-24T23:06:40.216 DEBUG:teuthology.orchestra.run.smithi032:> else: 2024-04-24T23:06:40.216 DEBUG:teuthology.orchestra.run.smithi032:> # The kernel cephfs client does not raise errors on fclose 2024-04-24T23:06:40.216 DEBUG:teuthology.orchestra.run.smithi032:> os.close(f) 2024-04-24T23:06:40.216 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-24T23:06:40.216 DEBUG:teuthology.orchestra.run.smithi032:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2024-04-24T23:06:40.216 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-24T23:07:07.320 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:07:07.323 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:07:37.408 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:07:37.412 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:08:07.552 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:08:07.556 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:08:37.613 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:08:37.617 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:08:42.089 INFO:teuthology.orchestra.run.smithi032.stdout:writing some data through which we expect to succeed 2024-04-24T23:08:42.090 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed data successfully, will now attempt to fill fs 2024-04-24T23:08:42.090 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 1572864 bytes via buffered write, may repeat 2024-04-24T23:08:42.090 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 2621440 bytes via buffered write, may repeat 2024-04-24T23:08:42.090 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 3670016 bytes via buffered write, may repeat 2024-04-24T23:08:42.090 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 4718592 bytes via buffered write, may repeat 2024-04-24T23:08:42.090 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 5767168 bytes via buffered write, may repeat 2024-04-24T23:08:42.090 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 6815744 bytes via buffered write, may repeat 2024-04-24T23:08:42.090 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 7864320 bytes via buffered write, may repeat 2024-04-24T23:08:42.090 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 8912896 bytes via buffered write, may repeat 2024-04-24T23:08:42.090 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 9961472 bytes via buffered write, may repeat 2024-04-24T23:08:42.090 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 11010048 bytes via buffered write, may repeat 2024-04-24T23:08:42.091 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 12058624 bytes via buffered write, may repeat 2024-04-24T23:08:42.091 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 13107200 bytes via buffered write, may repeat 2024-04-24T23:08:42.091 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 14155776 bytes via buffered write, may repeat 2024-04-24T23:08:42.091 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 15204352 bytes via buffered write, may repeat 2024-04-24T23:08:42.091 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 16252928 bytes via buffered write, may repeat 2024-04-24T23:08:42.091 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 17301504 bytes via buffered write, may repeat 2024-04-24T23:08:42.091 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 18350080 bytes via buffered write, may repeat 2024-04-24T23:08:42.091 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 19398656 bytes via buffered write, may repeat 2024-04-24T23:08:42.091 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 20447232 bytes via buffered write, may repeat 2024-04-24T23:08:42.091 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 21495808 bytes via buffered write, may repeat 2024-04-24T23:08:42.091 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 22544384 bytes via buffered write, may repeat 2024-04-24T23:08:42.091 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 23592960 bytes via buffered write, may repeat 2024-04-24T23:08:42.091 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 24641536 bytes via buffered write, may repeat 2024-04-24T23:08:42.092 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 25690112 bytes via buffered write, may repeat 2024-04-24T23:08:42.092 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 26738688 bytes via buffered write, may repeat 2024-04-24T23:08:42.092 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 27787264 bytes via buffered write, may repeat 2024-04-24T23:08:42.092 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 28835840 bytes via buffered write, may repeat 2024-04-24T23:08:42.092 INFO:teuthology.orchestra.run.smithi032.stdout:wrote 29884416 bytes via buffered write, may repeat 2024-04-24T23:08:42.092 INFO:teuthology.orchestra.run.smithi032.stdout:done writing 29884416 bytes 2024-04-24T23:08:42.092 INFO:teuthology.orchestra.run.smithi032.stdout:successfully fsync'ed prior to getting full state reported 2024-04-24T23:08:42.092 INFO:teuthology.orchestra.run.smithi032.stdout:starting buffered write 2024-04-24T23:08:42.092 INFO:teuthology.orchestra.run.smithi032.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-04-24T23:08:42.092 INFO:teuthology.orchestra.run.smithi032.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-04-24T23:08:42.092 INFO:teuthology.orchestra.run.smithi032.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-04-24T23:08:42.092 INFO:teuthology.orchestra.run.smithi032.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-04-24T23:08:42.092 INFO:teuthology.orchestra.run.smithi032.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-04-24T23:08:42.092 INFO:teuthology.orchestra.run.smithi032.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-04-24T23:08:42.092 INFO:teuthology.orchestra.run.smithi032.stdout:wrote, now waiting 30s and then doing a close we expect to fail 2024-04-24T23:08:42.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:08:42.103 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-24T23:08:42.103 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:08:42.148 DEBUG:teuthology.orchestra.run.smithi121:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:08:42.189 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-24T23:08:42.189 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:08:42.189 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:08:42.189 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:08:42.207 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:08:42.207 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:08:42.265 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:08:42.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.408+0000 7f2ee0bce640 1 Processor -- start 2024-04-24T23:08:42.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.408+0000 7f2ee0bce640 1 -- start start 2024-04-24T23:08:42.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.409+0000 7f2ee0bce640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2edc15fd60 0x7f2edc06c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:42.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.409+0000 7f2ee0bce640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2edc06cb30 0x7f2edc0cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:42.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.409+0000 7f2ee0bce640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2edc0b46c0 con 0x7f2edc15b480 2024-04-24T23:08:42.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.409+0000 7f2ee0bce640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f2edc0b4ff0 con 0x7f2edc06cb30 2024-04-24T23:08:42.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.409+0000 7f2ee0bce640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f2edc0b49a0 con 0x7f2edc15fd60 2024-04-24T23:08:42.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.409+0000 7f2edad9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2edc06cb30 0x7f2edc0cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:42.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.409+0000 7f2edad9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2edc06cb30 0x7f2edc0cdf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43468/0 (socket says 172.21.15.32:43468) 2024-04-24T23:08:42.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.409+0000 7f2edad9d640 1 -- 172.21.15.32:0/2676797233 learned_addr learned my addr 172.21.15.32:0/2676797233 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:42.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.409+0000 7f2ed9d9b640 1 --2- 172.21.15.32:0/2676797233 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2edc15fd60 0x7f2edc06c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:42.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.409+0000 7f2edad9d640 1 -- 172.21.15.32:0/2676797233 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2edc15fd60 msgr2=0x7f2edc06c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:42.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.409+0000 7f2edad9d640 1 --2- 172.21.15.32:0/2676797233 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2edc15fd60 0x7f2edc06c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:42.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.409+0000 7f2edad9d640 1 -- 172.21.15.32:0/2676797233 >> v1:172.21.15.121:6789/0 conn(0x7f2edc15b480 legacy=0x7f2edc0b5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:42.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.409+0000 7f2edad9d640 1 -- 172.21.15.32:0/2676797233 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2edc0b4c80 con 0x7f2edc06cb30 2024-04-24T23:08:42.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.410+0000 7f2edad9d640 1 --2- 172.21.15.32:0/2676797233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2edc06cb30 0x7f2edc0cdf80 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc009e50 tx=0x7f2ecc02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53e440800cfd81a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:42.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.410+0000 7f2ed959a640 1 -- 172.21.15.32:0/2676797233 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc036070 con 0x7f2edc06cb30 2024-04-24T23:08:42.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.410+0000 7f2ed959a640 1 -- 172.21.15.32:0/2676797233 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ecc031600 con 0x7f2edc06cb30 2024-04-24T23:08:42.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.410+0000 7f2ed959a640 1 -- 172.21.15.32:0/2676797233 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc0319d0 con 0x7f2edc06cb30 2024-04-24T23:08:42.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.411+0000 7f2ee0bce640 1 -- 172.21.15.32:0/2676797233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2edc06cb30 msgr2=0x7f2edc0cdf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:42.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.411+0000 7f2ee0bce640 1 --2- 172.21.15.32:0/2676797233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2edc06cb30 0x7f2edc0cdf80 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc009e50 tx=0x7f2ecc02a710 comp rx=0 tx=0).stop 2024-04-24T23:08:42.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.411+0000 7f2ee0bce640 1 -- 172.21.15.32:0/2676797233 shutdown_connections 2024-04-24T23:08:42.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.411+0000 7f2ee0bce640 1 --2- 172.21.15.32:0/2676797233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2edc06cb30 0x7f2edc0cdf80 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:42.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.411+0000 7f2ee0bce640 1 --2- 172.21.15.32:0/2676797233 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2edc15fd60 0x7f2edc06c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:42.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.411+0000 7f2ee0bce640 1 -- 172.21.15.32:0/2676797233 >> 172.21.15.32:0/2676797233 conn(0x7f2edc06b110 msgr2=0x7f2edc06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:42.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.411+0000 7f2ee0bce640 1 -- 172.21.15.32:0/2676797233 shutdown_connections 2024-04-24T23:08:42.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.411+0000 7f2ee0bce640 1 -- 172.21.15.32:0/2676797233 wait complete. 2024-04-24T23:08:42.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.412+0000 7f2ee0bce640 1 Processor -- start 2024-04-24T23:08:42.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.412+0000 7f2ee0bce640 1 -- start start 2024-04-24T23:08:42.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.413+0000 7f2ee0bce640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2edc06cb30 0x7f2edc0aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:42.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.413+0000 7f2ee0bce640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2edc15b480 0x7f2edc071af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:42.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.413+0000 7f2ee0bce640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2edc15fd60 0x7f2edc072030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:42.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.413+0000 7f2ee0bce640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2edc15b860 con 0x7f2edc15fd60 2024-04-24T23:08:42.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.413+0000 7f2ee0bce640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2edc0ce4c0 con 0x7f2edc15b480 2024-04-24T23:08:42.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.413+0000 7f2ee0bce640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f2edc15c790 con 0x7f2edc06cb30 2024-04-24T23:08:42.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.413+0000 7f2edad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2edc15fd60 0x7f2edc072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:42.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.413+0000 7f2edad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2edc15fd60 0x7f2edc072030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43480/0 (socket says 172.21.15.32:43480) 2024-04-24T23:08:42.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.413+0000 7f2edad9d640 1 -- 172.21.15.32:0/3458041973 learned_addr learned my addr 172.21.15.32:0/3458041973 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:42.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.413+0000 7f2ed9d9b640 1 --2- 172.21.15.32:0/3458041973 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2edc15b480 0x7f2edc071af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:42.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.413+0000 7f2eda59c640 1 --2- 172.21.15.32:0/3458041973 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2edc06cb30 0x7f2edc0aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:42.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.413+0000 7f2edad9d640 1 -- 172.21.15.32:0/3458041973 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2edc06cb30 msgr2=0x7f2edc0aeb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:42.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.413+0000 7f2edad9d640 1 --2- 172.21.15.32:0/3458041973 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2edc06cb30 0x7f2edc0aeb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:42.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.413+0000 7f2edad9d640 1 -- 172.21.15.32:0/3458041973 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2edc15b480 msgr2=0x7f2edc071af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:42.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.413+0000 7f2edad9d640 1 --2- 172.21.15.32:0/3458041973 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2edc15b480 0x7f2edc071af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:42.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.413+0000 7f2edad9d640 1 -- 172.21.15.32:0/3458041973 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2edc15ca30 con 0x7f2edc15fd60 2024-04-24T23:08:42.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.414+0000 7f2edad9d640 1 --2- 172.21.15.32:0/3458041973 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2edc15fd60 0x7f2edc072030 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc034040 tx=0x7f2ecc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:42.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.414+0000 7f2ecaffd640 1 -- 172.21.15.32:0/3458041973 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc03f020 con 0x7f2edc15fd60 2024-04-24T23:08:42.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.414+0000 7f2ecaffd640 1 -- 172.21.15.32:0/3458041973 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ecc037040 con 0x7f2edc15fd60 2024-04-24T23:08:42.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.414+0000 7f2ee0bce640 1 -- 172.21.15.32:0/3458041973 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2edc15cd10 con 0x7f2edc15fd60 2024-04-24T23:08:42.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.414+0000 7f2ecaffd640 1 -- 172.21.15.32:0/3458041973 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc036070 con 0x7f2edc15fd60 2024-04-24T23:08:42.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.414+0000 7f2ee0bce640 1 -- 172.21.15.32:0/3458041973 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2edc15d2d0 con 0x7f2edc15fd60 2024-04-24T23:08:42.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.415+0000 7f2ee0bce640 1 -- 172.21.15.32:0/3458041973 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2edc068430 con 0x7f2edc15fd60 2024-04-24T23:08:42.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.415+0000 7f2ecaffd640 1 -- 172.21.15.32:0/3458041973 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2ecc042020 con 0x7f2edc15fd60 2024-04-24T23:08:42.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.416+0000 7f2ecaffd640 1 --2- 172.21.15.32:0/3458041973 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2ebc06be30 0x7f2ebc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:42.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.416+0000 7f2ecaffd640 1 -- 172.21.15.32:0/3458041973 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7f2ecc0ab340 con 0x7f2edc15fd60 2024-04-24T23:08:42.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.416+0000 7f2ecaffd640 1 -- 172.21.15.32:0/3458041973 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=159}) -- 0x7f2ebc073960 con 0x7f2edc15fd60 2024-04-24T23:08:42.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.416+0000 7f2eda59c640 1 --2- 172.21.15.32:0/3458041973 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2ebc06be30 0x7f2ebc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:42.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.417+0000 7f2eda59c640 1 --2- 172.21.15.32:0/3458041973 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2ebc06be30 0x7f2ebc06e2f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4001640 tx=0x7f2ed4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:42.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.420+0000 7f2ecaffd640 1 -- 172.21.15.32:0/3458041973 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2ecc049030 con 0x7f2edc15fd60 2024-04-24T23:08:42.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.604+0000 7f2ee0bce640 1 -- 172.21.15.32:0/3458041973 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2edc15d4a0 con 0x7f2edc15fd60 2024-04-24T23:08:42.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.605+0000 7f2ecaffd640 1 -- 172.21.15.32:0/3458041973 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v158) ==== 74+0+16666 (secure 0 0 0) 0x7f2ecc079cf0 con 0x7f2edc15fd60 2024-04-24T23:08:42.607 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:08:42.607 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":158,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:08:06.831677+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:06:29.591736+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-24T23:06:30.267629+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:08:42.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.607+0000 7f2ee0bce640 1 -- 172.21.15.32:0/3458041973 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2ebc06be30 msgr2=0x7f2ebc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:42.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.607+0000 7f2ee0bce640 1 --2- 172.21.15.32:0/3458041973 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2ebc06be30 0x7f2ebc06e2f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4001640 tx=0x7f2ed4009040 comp rx=0 tx=0).stop 2024-04-24T23:08:42.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.608+0000 7f2ee0bce640 1 -- 172.21.15.32:0/3458041973 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2edc15fd60 msgr2=0x7f2edc072030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:42.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.608+0000 7f2ee0bce640 1 --2- 172.21.15.32:0/3458041973 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2edc15fd60 0x7f2edc072030 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc034040 tx=0x7f2ecc004a40 comp rx=0 tx=0).stop 2024-04-24T23:08:42.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.608+0000 7f2ee0bce640 1 -- 172.21.15.32:0/3458041973 shutdown_connections 2024-04-24T23:08:42.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.608+0000 7f2ee0bce640 1 --2- 172.21.15.32:0/3458041973 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2ebc06be30 0x7f2ebc06e2f0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:42.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.608+0000 7f2ee0bce640 1 --2- 172.21.15.32:0/3458041973 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2edc15fd60 0x7f2edc072030 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:42.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.608+0000 7f2ee0bce640 1 --2- 172.21.15.32:0/3458041973 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2edc15b480 0x7f2edc071af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:42.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.608+0000 7f2ee0bce640 1 --2- 172.21.15.32:0/3458041973 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2edc06cb30 0x7f2edc0aeb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:42.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.608+0000 7f2ee0bce640 1 -- 172.21.15.32:0/3458041973 >> 172.21.15.32:0/3458041973 conn(0x7f2edc06b110 msgr2=0x7f2edc0b37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:42.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.608+0000 7f2ee0bce640 1 -- 172.21.15.32:0/3458041973 shutdown_connections 2024-04-24T23:08:42.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.608+0000 7f2ee0bce640 1 -- 172.21.15.32:0/3458041973 wait complete. 2024-04-24T23:08:42.620 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:06:29.591736+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '155', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:06:30.267629+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '158', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:08:42.621 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:08:42.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.725+0000 7f1b540a5640 1 Processor -- start 2024-04-24T23:08:42.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.725+0000 7f1b540a5640 1 -- start start 2024-04-24T23:08:42.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.726+0000 7f1b540a5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1b4c074210 0x7f1b4c06c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:42.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.726+0000 7f1b540a5640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1b4c0b0b80 0x7f1b4c0745f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:42.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.726+0000 7f1b540a5640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b4c0b0770 con 0x7f1b4c0ae140 2024-04-24T23:08:42.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.726+0000 7f1b540a5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f1b4c0b1c20 con 0x7f1b4c074210 2024-04-24T23:08:42.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.726+0000 7f1b540a5640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f1b4c074b30 con 0x7f1b4c0b0b80 2024-04-24T23:08:42.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.726+0000 7f1b51e41640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1b4c074210 0x7f1b4c06c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:42.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.726+0000 7f1b52642640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1b4c0ae140 0x7f1b4c0b20b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52302/0 (socket says 172.21.15.32:52302) 2024-04-24T23:08:42.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.726+0000 7f1b51e41640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1b4c074210 0x7f1b4c06c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43492/0 (socket says 172.21.15.32:43492) 2024-04-24T23:08:42.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.726+0000 7f1b52642640 1 -- 172.21.15.32:0/35132921 learned_addr learned my addr 172.21.15.32:0/35132921 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:08:42.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.726+0000 7f1b51640640 1 --2- 172.21.15.32:0/35132921 >> v2:172.21.15.121:3300/0 conn(0x7f1b4c0b0b80 0x7f1b4c0745f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:42.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.726+0000 7f1b51e41640 1 -- 172.21.15.32:0/35132921 >> v2:172.21.15.121:3300/0 conn(0x7f1b4c0b0b80 msgr2=0x7f1b4c0745f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:42.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.726+0000 7f1b51e41640 1 --2- 172.21.15.32:0/35132921 >> v2:172.21.15.121:3300/0 conn(0x7f1b4c0b0b80 0x7f1b4c0745f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:42.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.726+0000 7f1b51e41640 1 -- 172.21.15.32:0/35132921 >> v1:172.21.15.32:6789/0 conn(0x7f1b4c0ae140 legacy=0x7f1b4c0b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:42.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.726+0000 7f1b51e41640 1 -- 172.21.15.32:0/35132921 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b4c0b1010 con 0x7f1b4c074210 2024-04-24T23:08:42.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.727+0000 7f1b51e41640 1 --2- 172.21.15.32:0/35132921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b4c074210 0x7f1b4c06c480 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c0722d0 tx=0x7f1b4802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d6652a63be02bf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:42.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.727+0000 7f1b50e3f640 1 -- 172.21.15.32:0/35132921 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b48037070 con 0x7f1b4c074210 2024-04-24T23:08:42.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.727+0000 7f1b50e3f640 1 -- 172.21.15.32:0/35132921 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b48032440 con 0x7f1b4c074210 2024-04-24T23:08:42.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.727+0000 7f1b50e3f640 1 -- 172.21.15.32:0/35132921 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b48032810 con 0x7f1b4c074210 2024-04-24T23:08:42.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.727+0000 7f1b540a5640 1 -- 172.21.15.32:0/35132921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b4c074210 msgr2=0x7f1b4c06c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:42.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.727+0000 7f1b540a5640 1 --2- 172.21.15.32:0/35132921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b4c074210 0x7f1b4c06c480 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c0722d0 tx=0x7f1b4802a1a0 comp rx=0 tx=0).stop 2024-04-24T23:08:42.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.727+0000 7f1b540a5640 1 -- 172.21.15.32:0/35132921 shutdown_connections 2024-04-24T23:08:42.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.727+0000 7f1b540a5640 1 --2- 172.21.15.32:0/35132921 >> v2:172.21.15.121:3300/0 conn(0x7f1b4c0b0b80 0x7f1b4c0745f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:42.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.728+0000 7f1b540a5640 1 --2- 172.21.15.32:0/35132921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b4c074210 0x7f1b4c06c480 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:42.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.728+0000 7f1b540a5640 1 -- 172.21.15.32:0/35132921 >> 172.21.15.32:0/35132921 conn(0x7f1b4c0b2e60 msgr2=0x7f1b4c0b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:42.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.728+0000 7f1b540a5640 1 -- 172.21.15.32:0/35132921 shutdown_connections 2024-04-24T23:08:42.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.728+0000 7f1b540a5640 1 -- 172.21.15.32:0/35132921 wait complete. 2024-04-24T23:08:42.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.729+0000 7f1b540a5640 1 Processor -- start 2024-04-24T23:08:42.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.729+0000 7f1b540a5640 1 -- start start 2024-04-24T23:08:42.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.729+0000 7f1b540a5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b4c074210 0x7f1b4c0cb020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:42.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.729+0000 7f1b540a5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b4c0ae140 0x7f1b4c15d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:42.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.729+0000 7f1b540a5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b4c15dd80 0x7f1b4c15e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:42.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.729+0000 7f1b540a5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4c0b18b0 con 0x7f1b4c0ae140 2024-04-24T23:08:42.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.729+0000 7f1b540a5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4c0b15d0 con 0x7f1b4c15dd80 2024-04-24T23:08:42.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.729+0000 7f1b540a5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f1b4c0b3ba0 con 0x7f1b4c074210 2024-04-24T23:08:42.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.730+0000 7f1b51640640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b4c0ae140 0x7f1b4c15d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:42.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.730+0000 7f1b51640640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b4c0ae140 0x7f1b4c15d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43504/0 (socket says 172.21.15.32:43504) 2024-04-24T23:08:42.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.730+0000 7f1b51640640 1 -- 172.21.15.32:0/673947024 learned_addr learned my addr 172.21.15.32:0/673947024 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:42.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.730+0000 7f1b51e41640 1 --2- 172.21.15.32:0/673947024 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b4c074210 0x7f1b4c0cb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:42.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.730+0000 7f1b52642640 1 --2- 172.21.15.32:0/673947024 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b4c15dd80 0x7f1b4c15e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:42.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.730+0000 7f1b51640640 1 -- 172.21.15.32:0/673947024 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b4c074210 msgr2=0x7f1b4c0cb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:42.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.730+0000 7f1b51640640 1 --2- 172.21.15.32:0/673947024 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b4c074210 0x7f1b4c0cb020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:42.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.730+0000 7f1b51640640 1 -- 172.21.15.32:0/673947024 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b4c15dd80 msgr2=0x7f1b4c15e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:42.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.730+0000 7f1b51640640 1 --2- 172.21.15.32:0/673947024 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b4c15dd80 0x7f1b4c15e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:42.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.730+0000 7f1b51640640 1 -- 172.21.15.32:0/673947024 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b4c0d1d10 con 0x7f1b4c0ae140 2024-04-24T23:08:42.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.730+0000 7f1b51640640 1 --2- 172.21.15.32:0/673947024 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b4c0ae140 0x7f1b4c15d720 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f1b4000c7f0 tx=0x7f1b4000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:42.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.730+0000 7f1b3e7fc640 1 -- 172.21.15.32:0/673947024 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b40015070 con 0x7f1b4c0ae140 2024-04-24T23:08:42.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.730+0000 7f1b3e7fc640 1 -- 172.21.15.32:0/673947024 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b40004240 con 0x7f1b4c0ae140 2024-04-24T23:08:42.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.730+0000 7f1b540a5640 1 -- 172.21.15.32:0/673947024 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b4c0c9f80 con 0x7f1b4c0ae140 2024-04-24T23:08:42.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.730+0000 7f1b3e7fc640 1 -- 172.21.15.32:0/673947024 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b40007620 con 0x7f1b4c0ae140 2024-04-24T23:08:42.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.730+0000 7f1b540a5640 1 -- 172.21.15.32:0/673947024 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b4c0ca540 con 0x7f1b4c0ae140 2024-04-24T23:08:42.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.732+0000 7f1b540a5640 1 -- 172.21.15.32:0/673947024 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b4c068430 con 0x7f1b4c0ae140 2024-04-24T23:08:42.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.732+0000 7f1b3e7fc640 1 -- 172.21.15.32:0/673947024 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1b400079d0 con 0x7f1b4c0ae140 2024-04-24T23:08:42.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.732+0000 7f1b3e7fc640 1 --2- 172.21.15.32:0/673947024 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1b3006be60 0x7f1b3006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:42.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.733+0000 7f1b3e7fc640 1 -- 172.21.15.32:0/673947024 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7f1b4008ae20 con 0x7f1b4c0ae140 2024-04-24T23:08:42.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.733+0000 7f1b51e41640 1 --2- 172.21.15.32:0/673947024 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1b3006be60 0x7f1b3006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:42.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.733+0000 7f1b3e7fc640 1 -- 172.21.15.32:0/673947024 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=159}) -- 0x7f1b300739f0 con 0x7f1b4c0ae140 2024-04-24T23:08:42.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.733+0000 7f1b51e41640 1 --2- 172.21.15.32:0/673947024 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1b3006be60 0x7f1b3006e320 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1b48034000 tx=0x7f1b4802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:42.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.737+0000 7f1b3e7fc640 1 -- 172.21.15.32:0/673947024 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1b40059210 con 0x7f1b4c0ae140 2024-04-24T23:08:42.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.923+0000 7f1b540a5640 1 -- 172.21.15.32:0/673947024 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1b4c0b3980 con 0x7f1b4c0ae140 2024-04-24T23:08:42.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.924+0000 7f1b3e7fc640 1 -- 172.21.15.32:0/673947024 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v158) ==== 93+0+10 (secure 0 0 0) 0x7f1b4005d510 con 0x7f1b4c0ae140 2024-04-24T23:08:42.925 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T23:08:42.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.926+0000 7f1b540a5640 1 -- 172.21.15.32:0/673947024 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1b3006be60 msgr2=0x7f1b3006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:42.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.926+0000 7f1b540a5640 1 --2- 172.21.15.32:0/673947024 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1b3006be60 0x7f1b3006e320 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1b48034000 tx=0x7f1b4802e000 comp rx=0 tx=0).stop 2024-04-24T23:08:42.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.926+0000 7f1b540a5640 1 -- 172.21.15.32:0/673947024 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b4c0ae140 msgr2=0x7f1b4c15d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:42.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.926+0000 7f1b540a5640 1 --2- 172.21.15.32:0/673947024 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b4c0ae140 0x7f1b4c15d720 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f1b4000c7f0 tx=0x7f1b4000ccb0 comp rx=0 tx=0).stop 2024-04-24T23:08:42.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.926+0000 7f1b540a5640 1 -- 172.21.15.32:0/673947024 shutdown_connections 2024-04-24T23:08:42.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.926+0000 7f1b540a5640 1 --2- 172.21.15.32:0/673947024 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1b3006be60 0x7f1b3006e320 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:42.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.926+0000 7f1b540a5640 1 --2- 172.21.15.32:0/673947024 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b4c0ae140 0x7f1b4c15d720 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:42.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.926+0000 7f1b540a5640 1 --2- 172.21.15.32:0/673947024 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b4c15dd80 0x7f1b4c15e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:42.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.926+0000 7f1b540a5640 1 --2- 172.21.15.32:0/673947024 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b4c074210 0x7f1b4c0cb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:42.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.926+0000 7f1b540a5640 1 -- 172.21.15.32:0/673947024 >> 172.21.15.32:0/673947024 conn(0x7f1b4c0b2e60 msgr2=0x7f1b4c06e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:42.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.926+0000 7f1b540a5640 1 -- 172.21.15.32:0/673947024 shutdown_connections 2024-04-24T23:08:42.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:42.926+0000 7f1b540a5640 1 -- 172.21.15.32:0/673947024 wait complete. 2024-04-24T23:08:42.938 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:08:43.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.040+0000 7f457731d640 1 Processor -- start 2024-04-24T23:08:43.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.040+0000 7f457731d640 1 -- start start 2024-04-24T23:08:43.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.041+0000 7f457731d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f457015e9e0 0x7f45700d40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:43.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.041+0000 7f457731d640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f457006b110 0x7f457006b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:43.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.041+0000 7f457731d640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45700d3010 con 0x7f45700ba390 2024-04-24T23:08:43.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.041+0000 7f457731d640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f45700d2db0 con 0x7f457006b110 2024-04-24T23:08:43.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.041+0000 7f457731d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f45700d32f0 con 0x7f457015e9e0 2024-04-24T23:08:43.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.042+0000 7f45750b9640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f457015e9e0 0x7f45700d40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:43.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.042+0000 7f45758ba640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f457006b110 0x7f457006b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:43.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.042+0000 7f45750b9640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f457015e9e0 0x7f45700d40b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.32:48104/0 (socket says 172.21.15.32:48104) 2024-04-24T23:08:43.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.042+0000 7f45748b8640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f45700ba390 0x7f45700d26a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:51312/0 (socket says 172.21.15.32:51312) 2024-04-24T23:08:43.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.042+0000 7f45750b9640 1 -- 172.21.15.32:0/2059536962 learned_addr learned my addr 172.21.15.32:0/2059536962 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:43.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.042+0000 7f45758ba640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f457006b110 0x7f457006b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.32:42794/0 (socket says 172.21.15.32:42794) 2024-04-24T23:08:43.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.042+0000 7f45758ba640 1 -- 172.21.15.32:0/2059536962 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f457015e9e0 msgr2=0x7f45700d40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:43.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.042+0000 7f45758ba640 1 --2- 172.21.15.32:0/2059536962 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f457015e9e0 0x7f45700d40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:43.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.042+0000 7f45758ba640 1 -- 172.21.15.32:0/2059536962 >> v1:172.21.15.121:6789/0 conn(0x7f45700ba390 legacy=0x7f45700d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:43.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.042+0000 7f45758ba640 1 -- 172.21.15.32:0/2059536962 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45700d35d0 con 0x7f457006b110 2024-04-24T23:08:43.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.042+0000 7f45750b9640 1 --2- 172.21.15.32:0/2059536962 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f457015e9e0 0x7f45700d40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:08:43.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.043+0000 7f45758ba640 1 --2- 172.21.15.32:0/2059536962 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f457006b110 0x7f457006b5c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f4568009d90 tx=0x7f456802a560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a40f0e5720bc17b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:43.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.043+0000 7f4567fff640 1 -- 172.21.15.32:0/2059536962 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4568007740 con 0x7f457006b110 2024-04-24T23:08:43.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.043+0000 7f4567fff640 1 -- 172.21.15.32:0/2059536962 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45680078e0 con 0x7f457006b110 2024-04-24T23:08:43.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.043+0000 7f4567fff640 1 -- 172.21.15.32:0/2059536962 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4568007cb0 con 0x7f457006b110 2024-04-24T23:08:43.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.044+0000 7f457731d640 1 -- 172.21.15.32:0/2059536962 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f457006b110 msgr2=0x7f457006b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:43.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.044+0000 7f457731d640 1 --2- 172.21.15.32:0/2059536962 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f457006b110 0x7f457006b5c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f4568009d90 tx=0x7f456802a560 comp rx=0 tx=0).stop 2024-04-24T23:08:43.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.044+0000 7f457731d640 1 -- 172.21.15.32:0/2059536962 shutdown_connections 2024-04-24T23:08:43.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.044+0000 7f457731d640 1 --2- 172.21.15.32:0/2059536962 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f457006b110 0x7f457006b5c0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:43.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.044+0000 7f457731d640 1 --2- 172.21.15.32:0/2059536962 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f457015e9e0 0x7f45700d40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:43.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.044+0000 7f457731d640 1 -- 172.21.15.32:0/2059536962 >> 172.21.15.32:0/2059536962 conn(0x7f45700d12e0 msgr2=0x7f45700d16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:43.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.044+0000 7f457731d640 1 -- 172.21.15.32:0/2059536962 shutdown_connections 2024-04-24T23:08:43.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.044+0000 7f457731d640 1 -- 172.21.15.32:0/2059536962 wait complete. 2024-04-24T23:08:43.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.045+0000 7f457731d640 1 Processor -- start 2024-04-24T23:08:43.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.045+0000 7f457731d640 1 -- start start 2024-04-24T23:08:43.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.046+0000 7f457731d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f457006b110 0x7f4570150720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:43.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.046+0000 7f457731d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45700ba390 0x7f4570158d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:43.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.046+0000 7f457731d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f457015e9e0 0x7f4570156e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:43.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.046+0000 7f457731d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f45700b7640 con 0x7f457006b110 2024-04-24T23:08:43.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.046+0000 7f457731d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f45700d3b90 con 0x7f45700ba390 2024-04-24T23:08:43.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.046+0000 7f457731d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f45700b77c0 con 0x7f457015e9e0 2024-04-24T23:08:43.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.046+0000 7f45750b9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f457006b110 0x7f4570150720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:43.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.046+0000 7f45750b9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f457006b110 0x7f4570150720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43520/0 (socket says 172.21.15.32:43520) 2024-04-24T23:08:43.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.046+0000 7f45750b9640 1 -- 172.21.15.32:0/3125346025 learned_addr learned my addr 172.21.15.32:0/3125346025 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:43.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.046+0000 7f45748b8640 1 --2- 172.21.15.32:0/3125346025 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45700ba390 0x7f4570158d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:43.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.046+0000 7f45758ba640 1 --2- 172.21.15.32:0/3125346025 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f457015e9e0 0x7f4570156e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:43.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.046+0000 7f45750b9640 1 -- 172.21.15.32:0/3125346025 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f457015e9e0 msgr2=0x7f4570156e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:43.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.046+0000 7f45750b9640 1 --2- 172.21.15.32:0/3125346025 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f457015e9e0 0x7f4570156e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:43.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.046+0000 7f45750b9640 1 -- 172.21.15.32:0/3125346025 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45700ba390 msgr2=0x7f4570158d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:43.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.046+0000 7f45750b9640 1 --2- 172.21.15.32:0/3125346025 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45700ba390 0x7f4570158d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:43.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.046+0000 7f45750b9640 1 -- 172.21.15.32:0/3125346025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4570150c60 con 0x7f457006b110 2024-04-24T23:08:43.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.046+0000 7f45750b9640 1 --2- 172.21.15.32:0/3125346025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f457006b110 0x7f4570150720 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f456c00e9b0 tx=0x7f456c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:43.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.046+0000 7f4565ffb640 1 -- 172.21.15.32:0/3125346025 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456c017070 con 0x7f457006b110 2024-04-24T23:08:43.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.046+0000 7f4565ffb640 1 -- 172.21.15.32:0/3125346025 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f456c00c9a0 con 0x7f457006b110 2024-04-24T23:08:43.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.046+0000 7f457731d640 1 -- 172.21.15.32:0/3125346025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45701597d0 con 0x7f457006b110 2024-04-24T23:08:43.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.046+0000 7f4565ffb640 1 -- 172.21.15.32:0/3125346025 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456c00cd50 con 0x7f457006b110 2024-04-24T23:08:43.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.047+0000 7f457731d640 1 -- 172.21.15.32:0/3125346025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4570160690 con 0x7f457006b110 2024-04-24T23:08:43.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.048+0000 7f457731d640 1 -- 172.21.15.32:0/3125346025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4570068430 con 0x7f457006b110 2024-04-24T23:08:43.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.048+0000 7f4565ffb640 1 -- 172.21.15.32:0/3125346025 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f456c01a070 con 0x7f457006b110 2024-04-24T23:08:43.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.049+0000 7f4565ffb640 1 --2- 172.21.15.32:0/3125346025 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f455406be30 0x7f455406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:43.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.049+0000 7f4565ffb640 1 -- 172.21.15.32:0/3125346025 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7f456c08ad80 con 0x7f457006b110 2024-04-24T23:08:43.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.049+0000 7f4565ffb640 1 -- 172.21.15.32:0/3125346025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=159}) -- 0x7f4554073960 con 0x7f457006b110 2024-04-24T23:08:43.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.049+0000 7f45748b8640 1 --2- 172.21.15.32:0/3125346025 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f455406be30 0x7f455406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:43.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.049+0000 7f45748b8640 1 --2- 172.21.15.32:0/3125346025 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f455406be30 0x7f455406e2f0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f45700cea80 tx=0x7f4560000bc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:43.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.053+0000 7f4565ffb640 1 -- 172.21.15.32:0/3125346025 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f456c058fc0 con 0x7f457006b110 2024-04-24T23:08:43.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.233+0000 7f457731d640 1 -- 172.21.15.32:0/3125346025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f45700d1be0 con 0x7f457006b110 2024-04-24T23:08:43.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.233+0000 7f4565ffb640 1 -- 172.21.15.32:0/3125346025 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v158) ==== 104+0+11 (secure 0 0 0) 0x7f456c05d2c0 con 0x7f457006b110 2024-04-24T23:08:43.235 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-24T23:08:43.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.235+0000 7f457731d640 1 -- 172.21.15.32:0/3125346025 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f455406be30 msgr2=0x7f455406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:43.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.235+0000 7f457731d640 1 --2- 172.21.15.32:0/3125346025 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f455406be30 0x7f455406e2f0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f45700cea80 tx=0x7f4560000bc0 comp rx=0 tx=0).stop 2024-04-24T23:08:43.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.235+0000 7f457731d640 1 -- 172.21.15.32:0/3125346025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f457006b110 msgr2=0x7f4570150720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:43.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.235+0000 7f457731d640 1 --2- 172.21.15.32:0/3125346025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f457006b110 0x7f4570150720 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f456c00e9b0 tx=0x7f456c00ee70 comp rx=0 tx=0).stop 2024-04-24T23:08:43.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.235+0000 7f457731d640 1 -- 172.21.15.32:0/3125346025 shutdown_connections 2024-04-24T23:08:43.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.235+0000 7f457731d640 1 --2- 172.21.15.32:0/3125346025 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f455406be30 0x7f455406e2f0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:43.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.235+0000 7f457731d640 1 --2- 172.21.15.32:0/3125346025 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f457015e9e0 0x7f4570156e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:43.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.235+0000 7f457731d640 1 --2- 172.21.15.32:0/3125346025 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45700ba390 0x7f4570158d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:43.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.235+0000 7f457731d640 1 --2- 172.21.15.32:0/3125346025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f457006b110 0x7f4570150720 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:43.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.235+0000 7f457731d640 1 -- 172.21.15.32:0/3125346025 >> 172.21.15.32:0/3125346025 conn(0x7f45700d12e0 msgr2=0x7f45700d26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:43.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.235+0000 7f457731d640 1 -- 172.21.15.32:0/3125346025 shutdown_connections 2024-04-24T23:08:43.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.235+0000 7f457731d640 1 -- 172.21.15.32:0/3125346025 wait complete. 2024-04-24T23:08:43.247 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:08:43.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.349+0000 7fc4fdbe9640 1 Processor -- start 2024-04-24T23:08:43.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.349+0000 7fc4fdbe9640 1 -- start start 2024-04-24T23:08:43.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.350+0000 7fc4fdbe9640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc4f806cd00 0x7fc4f80ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:43.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.350+0000 7fc4fdbe9640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc4f815cb00 0x7fc4f80ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:43.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.350+0000 7fc4fdbe9640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4f80b4520 con 0x7fc4f815b380 2024-04-24T23:08:43.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.350+0000 7fc4fdbe9640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fc4f80b4800 con 0x7fc4f806cd00 2024-04-24T23:08:43.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.350+0000 7fc4fdbe9640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc4f80b4ae0 con 0x7fc4f815cb00 2024-04-24T23:08:43.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.350+0000 7fc4f7fff640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc4f815cb00 0x7fc4f80ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:43.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.351+0000 7fc4f6ffd640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc4f806cd00 0x7fc4f80ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:43.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.351+0000 7fc4f7fff640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc4f815cb00 0x7fc4f80ce970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.32:48124/0 (socket says 172.21.15.32:48124) 2024-04-24T23:08:43.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.351+0000 7fc4f77fe640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fc4f815b380 0x7fc4f806c5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:51316/0 (socket says 172.21.15.32:51316) 2024-04-24T23:08:43.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.351+0000 7fc4f7fff640 1 -- 172.21.15.32:0/3237848382 learned_addr learned my addr 172.21.15.32:0/3237848382 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:43.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.351+0000 7fc4f7fff640 1 -- 172.21.15.32:0/3237848382 >> v2:172.21.15.121:3300/0 conn(0x7fc4f806cd00 msgr2=0x7fc4f80ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:43.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.351+0000 7fc4f7fff640 1 --2- 172.21.15.32:0/3237848382 >> v2:172.21.15.121:3300/0 conn(0x7fc4f806cd00 0x7fc4f80ce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:43.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.351+0000 7fc4f7fff640 1 -- 172.21.15.32:0/3237848382 >> v1:172.21.15.121:6789/0 conn(0x7fc4f815b380 legacy=0x7fc4f806c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:43.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.351+0000 7fc4f7fff640 1 -- 172.21.15.32:0/3237848382 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f80b4dc0 con 0x7fc4f815cb00 2024-04-24T23:08:43.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.351+0000 7fc4f6ffd640 1 --2- 172.21.15.32:0/3237848382 >> v2:172.21.15.121:3300/0 conn(0x7fc4f806cd00 0x7fc4f80ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:08:43.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.352+0000 7fc4f7fff640 1 --2- 172.21.15.32:0/3237848382 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc4f815cb00 0x7fc4f80ce970 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0009e50 tx=0x7fc4f002a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ede97b963fbf4188 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:43.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.352+0000 7fc4f67fc640 1 -- 172.21.15.32:0/3237848382 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f0036070 con 0x7fc4f815cb00 2024-04-24T23:08:43.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.352+0000 7fc4f67fc640 1 -- 172.21.15.32:0/3237848382 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4f0031600 con 0x7fc4f815cb00 2024-04-24T23:08:43.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.352+0000 7fc4f67fc640 1 -- 172.21.15.32:0/3237848382 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f00319d0 con 0x7fc4f815cb00 2024-04-24T23:08:43.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.352+0000 7fc4fdbe9640 1 -- 172.21.15.32:0/3237848382 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc4f815cb00 msgr2=0x7fc4f80ce970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:43.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.352+0000 7fc4fdbe9640 1 --2- 172.21.15.32:0/3237848382 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc4f815cb00 0x7fc4f80ce970 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0009e50 tx=0x7fc4f002a680 comp rx=0 tx=0).stop 2024-04-24T23:08:43.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.352+0000 7fc4fdbe9640 1 -- 172.21.15.32:0/3237848382 shutdown_connections 2024-04-24T23:08:43.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.352+0000 7fc4fdbe9640 1 --2- 172.21.15.32:0/3237848382 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc4f815cb00 0x7fc4f80ce970 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:43.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.352+0000 7fc4fdbe9640 1 --2- 172.21.15.32:0/3237848382 >> v2:172.21.15.121:3300/0 conn(0x7fc4f806cd00 0x7fc4f80ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:43.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.352+0000 7fc4fdbe9640 1 -- 172.21.15.32:0/3237848382 >> 172.21.15.32:0/3237848382 conn(0x7fc4f806b110 msgr2=0x7fc4f806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:43.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.353+0000 7fc4fdbe9640 1 -- 172.21.15.32:0/3237848382 shutdown_connections 2024-04-24T23:08:43.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.353+0000 7fc4fdbe9640 1 -- 172.21.15.32:0/3237848382 wait complete. 2024-04-24T23:08:43.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.354+0000 7fc4fdbe9640 1 Processor -- start 2024-04-24T23:08:43.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.354+0000 7fc4fdbe9640 1 -- start start 2024-04-24T23:08:43.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.354+0000 7fc4fdbe9640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc4f806cd00 0x7fc4f814dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:43.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.354+0000 7fc4fdbe9640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4f815b380 0x7fc4f814eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:43.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.354+0000 7fc4fdbe9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4f815cb00 0x7fc4f814f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:43.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.355+0000 7fc4fdbe9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f815b760 con 0x7fc4f815cb00 2024-04-24T23:08:43.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.355+0000 7fc4fdbe9640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f80ceeb0 con 0x7fc4f815b380 2024-04-24T23:08:43.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.355+0000 7fc4fdbe9640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc4f80b55c0 con 0x7fc4f806cd00 2024-04-24T23:08:43.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.355+0000 7fc4f7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4f815cb00 0x7fc4f814f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:43.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.355+0000 7fc4f7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4f815cb00 0x7fc4f814f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43530/0 (socket says 172.21.15.32:43530) 2024-04-24T23:08:43.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.355+0000 7fc4f7fff640 1 -- 172.21.15.32:0/1739659044 learned_addr learned my addr 172.21.15.32:0/1739659044 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:43.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.355+0000 7fc4f77fe640 1 --2- 172.21.15.32:0/1739659044 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc4f806cd00 0x7fc4f814dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:43.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.355+0000 7fc4f6ffd640 1 --2- 172.21.15.32:0/1739659044 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4f815b380 0x7fc4f814eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:43.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.355+0000 7fc4f7fff640 1 -- 172.21.15.32:0/1739659044 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc4f806cd00 msgr2=0x7fc4f814dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:43.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.355+0000 7fc4f7fff640 1 --2- 172.21.15.32:0/1739659044 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc4f806cd00 0x7fc4f814dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:43.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.355+0000 7fc4f7fff640 1 -- 172.21.15.32:0/1739659044 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4f815b380 msgr2=0x7fc4f814eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:43.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.355+0000 7fc4f7fff640 1 --2- 172.21.15.32:0/1739659044 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4f815b380 0x7fc4f814eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:43.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.355+0000 7fc4f7fff640 1 -- 172.21.15.32:0/1739659044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f8072620 con 0x7fc4f815cb00 2024-04-24T23:08:43.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.355+0000 7fc4f7fff640 1 --2- 172.21.15.32:0/1739659044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4f815cb00 0x7fc4f814f0c0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0034040 tx=0x7fc4f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:43.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.355+0000 7fc4dffff640 1 -- 172.21.15.32:0/1739659044 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f0036050 con 0x7fc4f815cb00 2024-04-24T23:08:43.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.355+0000 7fc4dffff640 1 -- 172.21.15.32:0/1739659044 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4f0042070 con 0x7fc4f815cb00 2024-04-24T23:08:43.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.355+0000 7fc4fdbe9640 1 -- 172.21.15.32:0/1739659044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4f8072340 con 0x7fc4f815cb00 2024-04-24T23:08:43.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.355+0000 7fc4dffff640 1 -- 172.21.15.32:0/1739659044 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f0009510 con 0x7fc4f815cb00 2024-04-24T23:08:43.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.355+0000 7fc4fdbe9640 1 -- 172.21.15.32:0/1739659044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4f8150770 con 0x7fc4f815cb00 2024-04-24T23:08:43.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.357+0000 7fc4fdbe9640 1 -- 172.21.15.32:0/1739659044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4f8068430 con 0x7fc4f815cb00 2024-04-24T23:08:43.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.357+0000 7fc4dffff640 1 -- 172.21.15.32:0/1739659044 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc4f0007c70 con 0x7fc4f815cb00 2024-04-24T23:08:43.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.357+0000 7fc4dffff640 1 --2- 172.21.15.32:0/1739659044 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc4e406be30 0x7fc4e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:43.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.358+0000 7fc4dffff640 1 -- 172.21.15.32:0/1739659044 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7fc4f00abb90 con 0x7fc4f815cb00 2024-04-24T23:08:43.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.358+0000 7fc4dffff640 1 -- 172.21.15.32:0/1739659044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=159}) -- 0x7fc4e4073960 con 0x7fc4f815cb00 2024-04-24T23:08:43.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.358+0000 7fc4f77fe640 1 --2- 172.21.15.32:0/1739659044 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc4e406be30 0x7fc4e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:43.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.358+0000 7fc4f77fe640 1 --2- 172.21.15.32:0/1739659044 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc4e406be30 0x7fc4e406e2f0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc4f814b1a0 tx=0x7fc4e8007e80 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:43.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.362+0000 7fc4dffff640 1 -- 172.21.15.32:0/1739659044 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc4f0079f00 con 0x7fc4f815cb00 2024-04-24T23:08:43.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.544+0000 7fc4fdbe9640 1 -- 172.21.15.32:0/1739659044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc4f806c000 con 0x7fc4f815cb00 2024-04-24T23:08:43.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.544+0000 7fc4dffff640 1 -- 172.21.15.32:0/1739659044 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v158) ==== 100+0+11 (secure 0 0 0) 0x7fc4f007e200 con 0x7fc4f815cb00 2024-04-24T23:08:43.546 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-24T23:08:43.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.546+0000 7fc4fdbe9640 1 -- 172.21.15.32:0/1739659044 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc4e406be30 msgr2=0x7fc4e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:43.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.546+0000 7fc4fdbe9640 1 --2- 172.21.15.32:0/1739659044 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc4e406be30 0x7fc4e406e2f0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc4f814b1a0 tx=0x7fc4e8007e80 comp rx=0 tx=0).stop 2024-04-24T23:08:43.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.546+0000 7fc4fdbe9640 1 -- 172.21.15.32:0/1739659044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4f815cb00 msgr2=0x7fc4f814f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:43.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.546+0000 7fc4fdbe9640 1 --2- 172.21.15.32:0/1739659044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4f815cb00 0x7fc4f814f0c0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0034040 tx=0x7fc4f0004a40 comp rx=0 tx=0).stop 2024-04-24T23:08:43.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.546+0000 7fc4fdbe9640 1 -- 172.21.15.32:0/1739659044 shutdown_connections 2024-04-24T23:08:43.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.546+0000 7fc4fdbe9640 1 --2- 172.21.15.32:0/1739659044 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc4e406be30 0x7fc4e406e2f0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:43.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.546+0000 7fc4fdbe9640 1 --2- 172.21.15.32:0/1739659044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4f815cb00 0x7fc4f814f0c0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:43.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.546+0000 7fc4fdbe9640 1 --2- 172.21.15.32:0/1739659044 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4f815b380 0x7fc4f814eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:43.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.546+0000 7fc4fdbe9640 1 --2- 172.21.15.32:0/1739659044 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc4f806cd00 0x7fc4f814dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:43.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.546+0000 7fc4fdbe9640 1 -- 172.21.15.32:0/1739659044 >> 172.21.15.32:0/1739659044 conn(0x7fc4f806b110 msgr2=0x7fc4f815b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:43.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.546+0000 7fc4fdbe9640 1 -- 172.21.15.32:0/1739659044 shutdown_connections 2024-04-24T23:08:43.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.546+0000 7fc4fdbe9640 1 -- 172.21.15.32:0/1739659044 wait complete. 2024-04-24T23:08:43.559 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T23:08:43.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.659+0000 7ffb52676640 1 Processor -- start 2024-04-24T23:08:43.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.659+0000 7ffb52676640 1 -- start start 2024-04-24T23:08:43.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.660+0000 7ffb52676640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffb4c0cf9a0 0x7ffb4c15ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:43.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.660+0000 7ffb52676640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ffb4c0cfd80 0x7ffb4c06deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:43.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.660+0000 7ffb52676640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb4c06e3f0 0x7ffb4c0ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:43.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.660+0000 7ffb52676640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7ffb4c06da20 con 0x7ffb4c0cf9a0 2024-04-24T23:08:43.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.660+0000 7ffb52676640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7ffb4c15f680 con 0x7ffb4c0cfd80 2024-04-24T23:08:43.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.660+0000 7ffb52676640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ffb4c15f960 con 0x7ffb4c06e3f0 2024-04-24T23:08:43.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.660+0000 7ffb4bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffb4c0cf9a0 0x7ffb4c15ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:43.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.660+0000 7ffb4bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffb4c0cf9a0 0x7ffb4c15ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43536/0 (socket says 172.21.15.32:43536) 2024-04-24T23:08:43.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.660+0000 7ffb4bfff640 1 -- 172.21.15.32:0/2955421833 learned_addr learned my addr 172.21.15.32:0/2955421833 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:43.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.660+0000 7ffb50c13640 1 --2- 172.21.15.32:0/2955421833 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb4c06e3f0 0x7ffb4c0ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:43.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.660+0000 7ffb4b7fe640 1 --2- 172.21.15.32:0/2955421833 >> v2:172.21.15.121:3300/0 conn(0x7ffb4c0cfd80 0x7ffb4c06deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:43.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.660+0000 7ffb4bfff640 1 -- 172.21.15.32:0/2955421833 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb4c06e3f0 msgr2=0x7ffb4c0ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:43.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.660+0000 7ffb4bfff640 1 --2- 172.21.15.32:0/2955421833 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb4c06e3f0 0x7ffb4c0ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:43.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.660+0000 7ffb4bfff640 1 -- 172.21.15.32:0/2955421833 >> v2:172.21.15.121:3300/0 conn(0x7ffb4c0cfd80 msgr2=0x7ffb4c06deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:43.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.660+0000 7ffb4bfff640 1 --2- 172.21.15.32:0/2955421833 >> v2:172.21.15.121:3300/0 conn(0x7ffb4c0cfd80 0x7ffb4c06deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:43.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.660+0000 7ffb4bfff640 1 -- 172.21.15.32:0/2955421833 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb4c15fc40 con 0x7ffb4c0cf9a0 2024-04-24T23:08:43.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.660+0000 7ffb50c13640 1 --2- 172.21.15.32:0/2955421833 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb4c06e3f0 0x7ffb4c0ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:08:43.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.660+0000 7ffb4b7fe640 1 --2- 172.21.15.32:0/2955421833 >> v2:172.21.15.121:3300/0 conn(0x7ffb4c0cfd80 0x7ffb4c06deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:08:43.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.661+0000 7ffb4bfff640 1 --2- 172.21.15.32:0/2955421833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb4c0cf9a0 0x7ffb4c15ee50 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c0ceba0 tx=0x7ffb3c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bc73837f95c4ced server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:43.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.661+0000 7ffb4affd640 1 -- 172.21.15.32:0/2955421833 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c037070 con 0x7ffb4c0cf9a0 2024-04-24T23:08:43.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.661+0000 7ffb4affd640 1 -- 172.21.15.32:0/2955421833 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb3c032440 con 0x7ffb4c0cf9a0 2024-04-24T23:08:43.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.661+0000 7ffb4affd640 1 -- 172.21.15.32:0/2955421833 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c032810 con 0x7ffb4c0cf9a0 2024-04-24T23:08:43.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.661+0000 7ffb52676640 1 -- 172.21.15.32:0/2955421833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb4c0cf9a0 msgr2=0x7ffb4c15ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:43.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.661+0000 7ffb52676640 1 --2- 172.21.15.32:0/2955421833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb4c0cf9a0 0x7ffb4c15ee50 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c0ceba0 tx=0x7ffb3c02a1e0 comp rx=0 tx=0).stop 2024-04-24T23:08:43.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.661+0000 7ffb52676640 1 -- 172.21.15.32:0/2955421833 shutdown_connections 2024-04-24T23:08:43.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.661+0000 7ffb52676640 1 --2- 172.21.15.32:0/2955421833 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb4c06e3f0 0x7ffb4c0ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:43.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.661+0000 7ffb52676640 1 --2- 172.21.15.32:0/2955421833 >> v2:172.21.15.121:3300/0 conn(0x7ffb4c0cfd80 0x7ffb4c06deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:43.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.661+0000 7ffb52676640 1 --2- 172.21.15.32:0/2955421833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb4c0cf9a0 0x7ffb4c15ee50 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:43.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.661+0000 7ffb52676640 1 -- 172.21.15.32:0/2955421833 >> 172.21.15.32:0/2955421833 conn(0x7ffb4c0d0730 msgr2=0x7ffb4c0d0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:43.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.662+0000 7ffb52676640 1 -- 172.21.15.32:0/2955421833 shutdown_connections 2024-04-24T23:08:43.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.662+0000 7ffb52676640 1 -- 172.21.15.32:0/2955421833 wait complete. 2024-04-24T23:08:43.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.663+0000 7ffb52676640 1 Processor -- start 2024-04-24T23:08:43.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.663+0000 7ffb52676640 1 -- start start 2024-04-24T23:08:43.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.664+0000 7ffb52676640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb4c06e3f0 0x7ffb4c1452f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:43.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.664+0000 7ffb52676640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb4c0cf9a0 0x7ffb4c0b5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:43.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.664+0000 7ffb52676640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb4c0cfd80 0x7ffb4c0b5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:43.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.664+0000 7ffb52676640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ffb4c0afcf0 con 0x7ffb4c06e3f0 2024-04-24T23:08:43.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.664+0000 7ffb52676640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ffb4c06d610 con 0x7ffb4c0cf9a0 2024-04-24T23:08:43.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.664+0000 7ffb52676640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ffb4c0afe70 con 0x7ffb4c0cfd80 2024-04-24T23:08:43.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.664+0000 7ffb4bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb4c06e3f0 0x7ffb4c1452f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:43.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.664+0000 7ffb4bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb4c06e3f0 0x7ffb4c1452f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43548/0 (socket says 172.21.15.32:43548) 2024-04-24T23:08:43.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.664+0000 7ffb4bfff640 1 -- 172.21.15.32:0/2103176151 learned_addr learned my addr 172.21.15.32:0/2103176151 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:43.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.664+0000 7ffb50c13640 1 --2- 172.21.15.32:0/2103176151 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb4c0cfd80 0x7ffb4c0b5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:43.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.664+0000 7ffb4b7fe640 1 --2- 172.21.15.32:0/2103176151 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb4c0cf9a0 0x7ffb4c0b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:43.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.664+0000 7ffb4bfff640 1 -- 172.21.15.32:0/2103176151 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb4c0cfd80 msgr2=0x7ffb4c0b5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:43.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.664+0000 7ffb4bfff640 1 --2- 172.21.15.32:0/2103176151 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb4c0cfd80 0x7ffb4c0b5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:43.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.664+0000 7ffb4bfff640 1 -- 172.21.15.32:0/2103176151 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb4c0cf9a0 msgr2=0x7ffb4c0b5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:43.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.664+0000 7ffb4bfff640 1 --2- 172.21.15.32:0/2103176151 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb4c0cf9a0 0x7ffb4c0b5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:43.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.664+0000 7ffb4bfff640 1 -- 172.21.15.32:0/2103176151 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb4c0b1bb0 con 0x7ffb4c06e3f0 2024-04-24T23:08:43.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.664+0000 7ffb4bfff640 1 --2- 172.21.15.32:0/2103176151 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb4c06e3f0 0x7ffb4c1452f0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c02a220 tx=0x7ffb3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:43.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.664+0000 7ffb48ff9640 1 -- 172.21.15.32:0/2103176151 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c037040 con 0x7ffb4c06e3f0 2024-04-24T23:08:43.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.664+0000 7ffb48ff9640 1 -- 172.21.15.32:0/2103176151 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb3c042070 con 0x7ffb4c06e3f0 2024-04-24T23:08:43.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.664+0000 7ffb48ff9640 1 -- 172.21.15.32:0/2103176151 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c030bb0 con 0x7ffb4c06e3f0 2024-04-24T23:08:43.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.664+0000 7ffb52676640 1 -- 172.21.15.32:0/2103176151 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb4c145b40 con 0x7ffb4c06e3f0 2024-04-24T23:08:43.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.664+0000 7ffb52676640 1 -- 172.21.15.32:0/2103176151 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb4c146100 con 0x7ffb4c06e3f0 2024-04-24T23:08:43.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.666+0000 7ffb52676640 1 -- 172.21.15.32:0/2103176151 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb4c068430 con 0x7ffb4c06e3f0 2024-04-24T23:08:43.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.666+0000 7ffb48ff9640 1 -- 172.21.15.32:0/2103176151 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffb3c047460 con 0x7ffb4c06e3f0 2024-04-24T23:08:43.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.666+0000 7ffb48ff9640 1 --2- 172.21.15.32:0/2103176151 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ffb3806be30 0x7ffb3806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:43.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.667+0000 7ffb48ff9640 1 -- 172.21.15.32:0/2103176151 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7ffb3c0ab8e0 con 0x7ffb4c06e3f0 2024-04-24T23:08:43.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.667+0000 7ffb48ff9640 1 -- 172.21.15.32:0/2103176151 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=159}) -- 0x7ffb38073960 con 0x7ffb4c06e3f0 2024-04-24T23:08:43.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.667+0000 7ffb4b7fe640 1 --2- 172.21.15.32:0/2103176151 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ffb3806be30 0x7ffb3806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:43.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.667+0000 7ffb4b7fe640 1 --2- 172.21.15.32:0/2103176151 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ffb3806be30 0x7ffb3806e2f0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c06dc80 tx=0x7ffb40008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:43.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.671+0000 7ffb48ff9640 1 -- 172.21.15.32:0/2103176151 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ffb3c079cd0 con 0x7ffb4c06e3f0 2024-04-24T23:08:43.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.864+0000 7ffb52676640 1 -- 172.21.15.32:0/2103176151 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ffb4c06b860 con 0x7ffb4c06e3f0 2024-04-24T23:08:43.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.865+0000 7ffb48ff9640 1 -- 172.21.15.32:0/2103176151 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v158) ==== 80+0+0 (secure 0 0 0) 0x7ffb3c07dfd0 con 0x7ffb4c06e3f0 2024-04-24T23:08:43.867 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2024-04-24T23:08:43.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.867+0000 7ffb52676640 1 -- 172.21.15.32:0/2103176151 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ffb3806be30 msgr2=0x7ffb3806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:43.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.867+0000 7ffb52676640 1 --2- 172.21.15.32:0/2103176151 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ffb3806be30 0x7ffb3806e2f0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c06dc80 tx=0x7ffb40008040 comp rx=0 tx=0).stop 2024-04-24T23:08:43.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.867+0000 7ffb52676640 1 -- 172.21.15.32:0/2103176151 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb4c06e3f0 msgr2=0x7ffb4c1452f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:43.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.867+0000 7ffb52676640 1 --2- 172.21.15.32:0/2103176151 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb4c06e3f0 0x7ffb4c1452f0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c02a220 tx=0x7ffb3c0046c0 comp rx=0 tx=0).stop 2024-04-24T23:08:43.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.867+0000 7ffb52676640 1 -- 172.21.15.32:0/2103176151 shutdown_connections 2024-04-24T23:08:43.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.867+0000 7ffb52676640 1 --2- 172.21.15.32:0/2103176151 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ffb3806be30 0x7ffb3806e2f0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:43.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.867+0000 7ffb52676640 1 --2- 172.21.15.32:0/2103176151 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb4c0cfd80 0x7ffb4c0b5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:43.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.867+0000 7ffb52676640 1 --2- 172.21.15.32:0/2103176151 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb4c0cf9a0 0x7ffb4c0b5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:43.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.867+0000 7ffb52676640 1 --2- 172.21.15.32:0/2103176151 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb4c06e3f0 0x7ffb4c1452f0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:43.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.867+0000 7ffb52676640 1 -- 172.21.15.32:0/2103176151 >> 172.21.15.32:0/2103176151 conn(0x7ffb4c0d0730 msgr2=0x7ffb4c070dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:43.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.867+0000 7ffb52676640 1 -- 172.21.15.32:0/2103176151 shutdown_connections 2024-04-24T23:08:43.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:43.867+0000 7ffb52676640 1 -- 172.21.15.32:0/2103176151 wait complete. 2024-04-24T23:08:43.879 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-24T23:08:43.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:08:43.880 DEBUG:teuthology.orchestra.run.smithi032:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-24T23:08:43.909 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2024-04-24T23:08:43.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:08:43.909 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-24T23:08:43.965 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:08:43.965 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:08:43.965 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:08:43.984 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:08:43.984 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:08:44.044 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:08:44.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.144+0000 7f0d34415640 1 Processor -- start 2024-04-24T23:08:44.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.144+0000 7f0d34415640 1 -- start start 2024-04-24T23:08:44.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.145+0000 7f0d34415640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0d2c15e9b0 0x7f0d2c0d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:44.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.145+0000 7f0d34415640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d2c0d4210 0x7f0d2c06b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:44.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.145+0000 7f0d34415640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d2c0d3170 con 0x7f0d2c0ba0e0 2024-04-24T23:08:44.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.145+0000 7f0d34415640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f0d2c0b8900 con 0x7f0d2c15e9b0 2024-04-24T23:08:44.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.145+0000 7f0d34415640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f0d2c0d3450 con 0x7f0d2c0d4210 2024-04-24T23:08:44.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.145+0000 7f0d321b1640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0d2c15e9b0 0x7f0d2c0d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:44.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.145+0000 7f0d321b1640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0d2c15e9b0 0x7f0d2c0d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43552/0 (socket says 172.21.15.32:43552) 2024-04-24T23:08:44.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.145+0000 7f0d321b1640 1 -- 172.21.15.32:0/3298655018 learned_addr learned my addr 172.21.15.32:0/3298655018 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:44.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.145+0000 7f0d329b2640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0d2c0ba0e0 0x7f0d2c071730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52314/0 (socket says 172.21.15.32:52314) 2024-04-24T23:08:44.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.146+0000 7f0d321b1640 1 -- 172.21.15.32:0/3298655018 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d2c0d4210 msgr2=0x7f0d2c06b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:44.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.146+0000 7f0d319b0640 1 --2- 172.21.15.32:0/3298655018 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d2c0d4210 0x7f0d2c06b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:44.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.146+0000 7f0d321b1640 1 --2- 172.21.15.32:0/3298655018 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d2c0d4210 0x7f0d2c06b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:44.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.146+0000 7f0d321b1640 1 -- 172.21.15.32:0/3298655018 >> v1:172.21.15.32:6789/0 conn(0x7f0d2c0ba0e0 legacy=0x7f0d2c071730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:44.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.146+0000 7f0d321b1640 1 -- 172.21.15.32:0/3298655018 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d2c0d3730 con 0x7f0d2c15e9b0 2024-04-24T23:08:44.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.146+0000 7f0d311af640 1 -- 172.21.15.32:0/3298655018 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2291127207 0 0) 0x7f0d2c0d3170 con 0x7f0d2c0ba0e0 2024-04-24T23:08:44.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.146+0000 7f0d321b1640 1 --2- 172.21.15.32:0/3298655018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d2c15e9b0 0x7f0d2c0d2b90 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c009e50 tx=0x7f0d1c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80f8c18d12f56998 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:44.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.147+0000 7f0d311af640 1 -- 172.21.15.32:0/3298655018 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c036070 con 0x7f0d2c15e9b0 2024-04-24T23:08:44.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.147+0000 7f0d311af640 1 -- 172.21.15.32:0/3298655018 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d1c031600 con 0x7f0d2c15e9b0 2024-04-24T23:08:44.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.147+0000 7f0d311af640 1 -- 172.21.15.32:0/3298655018 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c0319d0 con 0x7f0d2c15e9b0 2024-04-24T23:08:44.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.147+0000 7f0d34415640 1 -- 172.21.15.32:0/3298655018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d2c15e9b0 msgr2=0x7f0d2c0d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:44.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.147+0000 7f0d34415640 1 --2- 172.21.15.32:0/3298655018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d2c15e9b0 0x7f0d2c0d2b90 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c009e50 tx=0x7f0d1c02a710 comp rx=0 tx=0).stop 2024-04-24T23:08:44.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.147+0000 7f0d34415640 1 -- 172.21.15.32:0/3298655018 shutdown_connections 2024-04-24T23:08:44.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.147+0000 7f0d34415640 1 --2- 172.21.15.32:0/3298655018 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d2c0d4210 0x7f0d2c06b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:44.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.147+0000 7f0d34415640 1 --2- 172.21.15.32:0/3298655018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d2c15e9b0 0x7f0d2c0d2b90 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:44.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.147+0000 7f0d34415640 1 -- 172.21.15.32:0/3298655018 >> 172.21.15.32:0/3298655018 conn(0x7f0d2c0d11d0 msgr2=0x7f0d2c0d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:44.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.147+0000 7f0d34415640 1 -- 172.21.15.32:0/3298655018 shutdown_connections 2024-04-24T23:08:44.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.147+0000 7f0d34415640 1 -- 172.21.15.32:0/3298655018 wait complete. 2024-04-24T23:08:44.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.148+0000 7f0d34415640 1 Processor -- start 2024-04-24T23:08:44.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.148+0000 7f0d34415640 1 -- start start 2024-04-24T23:08:44.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.149+0000 7f0d34415640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d2c0ba0e0 0x7f0d2c1f7130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:44.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.149+0000 7f0d34415640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d2c0d4210 0x7f0d2c154bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:44.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.149+0000 7f0d34415640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d2c15e9b0 0x7f0d2c155130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:44.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.149+0000 7f0d34415640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0d2c0d2400 con 0x7f0d2c15e9b0 2024-04-24T23:08:44.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.149+0000 7f0d34415640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0d2c0d3cf0 con 0x7f0d2c0d4210 2024-04-24T23:08:44.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.149+0000 7f0d34415640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f0d2c06b650 con 0x7f0d2c0ba0e0 2024-04-24T23:08:44.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.149+0000 7f0d329b2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d2c15e9b0 0x7f0d2c155130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:44.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.149+0000 7f0d329b2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d2c15e9b0 0x7f0d2c155130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43564/0 (socket says 172.21.15.32:43564) 2024-04-24T23:08:44.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.149+0000 7f0d329b2640 1 -- 172.21.15.32:0/327536784 learned_addr learned my addr 172.21.15.32:0/327536784 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:44.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.149+0000 7f0d321b1640 1 --2- 172.21.15.32:0/327536784 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d2c0ba0e0 0x7f0d2c1f7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:44.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.149+0000 7f0d319b0640 1 --2- 172.21.15.32:0/327536784 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d2c0d4210 0x7f0d2c154bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:44.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.150+0000 7f0d329b2640 1 -- 172.21.15.32:0/327536784 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d2c0ba0e0 msgr2=0x7f0d2c1f7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:44.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.150+0000 7f0d329b2640 1 --2- 172.21.15.32:0/327536784 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d2c0ba0e0 0x7f0d2c1f7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:44.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.150+0000 7f0d329b2640 1 -- 172.21.15.32:0/327536784 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d2c0d4210 msgr2=0x7f0d2c154bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:44.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.150+0000 7f0d329b2640 1 --2- 172.21.15.32:0/327536784 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d2c0d4210 0x7f0d2c154bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:44.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.150+0000 7f0d329b2640 1 -- 172.21.15.32:0/327536784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d2c161f80 con 0x7f0d2c15e9b0 2024-04-24T23:08:44.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.150+0000 7f0d321b1640 1 --2- 172.21.15.32:0/327536784 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d2c0ba0e0 0x7f0d2c1f7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:08:44.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.150+0000 7f0d319b0640 1 --2- 172.21.15.32:0/327536784 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d2c0d4210 0x7f0d2c154bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:08:44.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.150+0000 7f0d329b2640 1 --2- 172.21.15.32:0/327536784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d2c15e9b0 0x7f0d2c155130 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f0d2400c480 tx=0x7f0d2400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:44.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.150+0000 7f0d2affd640 1 -- 172.21.15.32:0/327536784 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d24018070 con 0x7f0d2c15e9b0 2024-04-24T23:08:44.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.150+0000 7f0d2affd640 1 -- 172.21.15.32:0/327536784 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d24007500 con 0x7f0d2c15e9b0 2024-04-24T23:08:44.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.150+0000 7f0d34415640 1 -- 172.21.15.32:0/327536784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d2c15ed90 con 0x7f0d2c15e9b0 2024-04-24T23:08:44.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.150+0000 7f0d2affd640 1 -- 172.21.15.32:0/327536784 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d24004a20 con 0x7f0d2c15e9b0 2024-04-24T23:08:44.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.150+0000 7f0d34415640 1 -- 172.21.15.32:0/327536784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d2c144cf0 con 0x7f0d2c15e9b0 2024-04-24T23:08:44.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.151+0000 7f0d34415640 1 -- 172.21.15.32:0/327536784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d2c068430 con 0x7f0d2c15e9b0 2024-04-24T23:08:44.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.151+0000 7f0d2affd640 1 -- 172.21.15.32:0/327536784 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0d2401b070 con 0x7f0d2c15e9b0 2024-04-24T23:08:44.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.152+0000 7f0d2affd640 1 --2- 172.21.15.32:0/327536784 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0d1806be30 0x7f0d1806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:44.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.152+0000 7f0d2affd640 1 -- 172.21.15.32:0/327536784 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7f0d2408a0d0 con 0x7f0d2c15e9b0 2024-04-24T23:08:44.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.152+0000 7f0d2affd640 1 -- 172.21.15.32:0/327536784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=159}) -- 0x7f0d18073960 con 0x7f0d2c15e9b0 2024-04-24T23:08:44.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.152+0000 7f0d321b1640 1 --2- 172.21.15.32:0/327536784 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0d1806be30 0x7f0d1806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:44.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.153+0000 7f0d321b1640 1 --2- 172.21.15.32:0/327536784 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0d1806be30 0x7f0d1806e2f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c033000 tx=0x7f0d1c009000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:44.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.156+0000 7f0d2affd640 1 -- 172.21.15.32:0/327536784 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0d2c15ed90 con 0x7f0d2c15e9b0 2024-04-24T23:08:44.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.338+0000 7f0d34415640 1 -- 172.21.15.32:0/327536784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0d2c071e20 con 0x7f0d2c15e9b0 2024-04-24T23:08:44.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.340+0000 7f0d2affd640 1 -- 172.21.15.32:0/327536784 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v158) ==== 74+0+16666 (secure 0 0 0) 0x7f0d24058b10 con 0x7f0d2c15e9b0 2024-04-24T23:08:44.341 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:08:44.341 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":158,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:08:06.831677+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:06:29.591736+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-24T23:06:30.267629+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:08:44.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.342+0000 7f0d34415640 1 -- 172.21.15.32:0/327536784 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0d1806be30 msgr2=0x7f0d1806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:44.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.342+0000 7f0d34415640 1 --2- 172.21.15.32:0/327536784 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0d1806be30 0x7f0d1806e2f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c033000 tx=0x7f0d1c009000 comp rx=0 tx=0).stop 2024-04-24T23:08:44.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.342+0000 7f0d34415640 1 -- 172.21.15.32:0/327536784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d2c15e9b0 msgr2=0x7f0d2c155130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:44.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.342+0000 7f0d34415640 1 --2- 172.21.15.32:0/327536784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d2c15e9b0 0x7f0d2c155130 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f0d2400c480 tx=0x7f0d2400c940 comp rx=0 tx=0).stop 2024-04-24T23:08:44.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.342+0000 7f0d34415640 1 -- 172.21.15.32:0/327536784 shutdown_connections 2024-04-24T23:08:44.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.342+0000 7f0d34415640 1 --2- 172.21.15.32:0/327536784 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0d1806be30 0x7f0d1806e2f0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:44.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.342+0000 7f0d34415640 1 --2- 172.21.15.32:0/327536784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d2c15e9b0 0x7f0d2c155130 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:44.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.342+0000 7f0d34415640 1 --2- 172.21.15.32:0/327536784 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d2c0d4210 0x7f0d2c154bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:44.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.342+0000 7f0d34415640 1 --2- 172.21.15.32:0/327536784 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d2c0ba0e0 0x7f0d2c1f7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:44.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.342+0000 7f0d34415640 1 -- 172.21.15.32:0/327536784 >> 172.21.15.32:0/327536784 conn(0x7f0d2c0d11d0 msgr2=0x7f0d2c0d15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:44.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.342+0000 7f0d34415640 1 -- 172.21.15.32:0/327536784 shutdown_connections 2024-04-24T23:08:44.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.342+0000 7f0d34415640 1 -- 172.21.15.32:0/327536784 wait complete. 2024-04-24T23:08:44.354 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:06:29.591736+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '155', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:06:30.267629+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '158', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:08:44.355 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:08:44.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.457+0000 7f6de3438640 1 Processor -- start 2024-04-24T23:08:44.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.457+0000 7f6de3438640 1 -- start start 2024-04-24T23:08:44.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.458+0000 7f6de3438640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6ddc15b380 0x7f6ddc06c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:44.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.458+0000 7f6de3438640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6ddc15cb00 0x7f6ddc06cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:44.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.458+0000 7f6de3438640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ddc0ce430 0x7f6ddc0ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:44.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.458+0000 7f6de3438640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f6ddc0b4520 con 0x7f6ddc15cb00 2024-04-24T23:08:44.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.458+0000 7f6de3438640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f6ddc0b4800 con 0x7f6ddc15b380 2024-04-24T23:08:44.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.458+0000 7f6de3438640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f6ddc0b4ae0 con 0x7f6ddc0ce430 2024-04-24T23:08:44.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.458+0000 7f6de09d3640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6ddc15cb00 0x7f6ddc06cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:44.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.458+0000 7f6de09d3640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6ddc15cb00 0x7f6ddc06cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43566/0 (socket says 172.21.15.32:43566) 2024-04-24T23:08:44.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.458+0000 7f6de09d3640 1 -- 172.21.15.32:0/4224864666 learned_addr learned my addr 172.21.15.32:0/4224864666 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:44.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.458+0000 7f6de11d4640 1 --2- 172.21.15.32:0/4224864666 >> v2:172.21.15.121:3300/0 conn(0x7f6ddc15b380 0x7f6ddc06c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:44.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.458+0000 7f6de19d5640 1 --2- 172.21.15.32:0/4224864666 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ddc0ce430 0x7f6ddc0ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:44.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.458+0000 7f6de09d3640 1 -- 172.21.15.32:0/4224864666 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ddc0ce430 msgr2=0x7f6ddc0ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:44.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.458+0000 7f6de09d3640 1 --2- 172.21.15.32:0/4224864666 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ddc0ce430 0x7f6ddc0ce860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:44.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.458+0000 7f6de09d3640 1 -- 172.21.15.32:0/4224864666 >> v2:172.21.15.121:3300/0 conn(0x7f6ddc15b380 msgr2=0x7f6ddc06c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:44.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.458+0000 7f6de09d3640 1 --2- 172.21.15.32:0/4224864666 >> v2:172.21.15.121:3300/0 conn(0x7f6ddc15b380 0x7f6ddc06c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:44.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.458+0000 7f6de09d3640 1 -- 172.21.15.32:0/4224864666 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ddc0b4dc0 con 0x7f6ddc15cb00 2024-04-24T23:08:44.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.458+0000 7f6de19d5640 1 --2- 172.21.15.32:0/4224864666 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ddc0ce430 0x7f6ddc0ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:08:44.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.459+0000 7f6de09d3640 1 --2- 172.21.15.32:0/4224864666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ddc15cb00 0x7f6ddc06cb30 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc06eed0 tx=0x7f6dcc02a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95281b7323a62cc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:44.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.459+0000 7f6dd3fff640 1 -- 172.21.15.32:0/4224864666 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc037070 con 0x7f6ddc15cb00 2024-04-24T23:08:44.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.459+0000 7f6dd3fff640 1 -- 172.21.15.32:0/4224864666 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dcc02ae00 con 0x7f6ddc15cb00 2024-04-24T23:08:44.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.459+0000 7f6dd3fff640 1 -- 172.21.15.32:0/4224864666 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc032660 con 0x7f6ddc15cb00 2024-04-24T23:08:44.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.460+0000 7f6de3438640 1 -- 172.21.15.32:0/4224864666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ddc15cb00 msgr2=0x7f6ddc06cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:44.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.460+0000 7f6de3438640 1 --2- 172.21.15.32:0/4224864666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ddc15cb00 0x7f6ddc06cb30 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc06eed0 tx=0x7f6dcc02a0b0 comp rx=0 tx=0).stop 2024-04-24T23:08:44.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.460+0000 7f6de3438640 1 -- 172.21.15.32:0/4224864666 shutdown_connections 2024-04-24T23:08:44.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.460+0000 7f6de3438640 1 --2- 172.21.15.32:0/4224864666 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ddc0ce430 0x7f6ddc0ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:44.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.460+0000 7f6de3438640 1 --2- 172.21.15.32:0/4224864666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ddc15cb00 0x7f6ddc06cb30 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:44.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.460+0000 7f6de3438640 1 --2- 172.21.15.32:0/4224864666 >> v2:172.21.15.121:3300/0 conn(0x7f6ddc15b380 0x7f6ddc06c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:44.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.460+0000 7f6de3438640 1 -- 172.21.15.32:0/4224864666 >> 172.21.15.32:0/4224864666 conn(0x7f6ddc06b110 msgr2=0x7f6ddc06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:44.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.460+0000 7f6de3438640 1 -- 172.21.15.32:0/4224864666 shutdown_connections 2024-04-24T23:08:44.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.460+0000 7f6de3438640 1 -- 172.21.15.32:0/4224864666 wait complete. 2024-04-24T23:08:44.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.461+0000 7f6de3438640 1 Processor -- start 2024-04-24T23:08:44.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.461+0000 7f6de3438640 1 -- start start 2024-04-24T23:08:44.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.462+0000 7f6de3438640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ddc0ce430 0x7f6ddc152dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:44.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.462+0000 7f6de3438640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ddc15b380 0x7f6ddc13d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:44.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.462+0000 7f6de3438640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ddc15cb00 0x7f6ddc13fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:44.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.462+0000 7f6de3438640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6ddc0ceda0 con 0x7f6ddc15cb00 2024-04-24T23:08:44.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.462+0000 7f6de3438640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6ddc15b760 con 0x7f6ddc15b380 2024-04-24T23:08:44.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.462+0000 7f6de3438640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f6ddc0cef20 con 0x7f6ddc0ce430 2024-04-24T23:08:44.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.462+0000 7f6de19d5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ddc15cb00 0x7f6ddc13fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:44.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.462+0000 7f6de19d5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ddc15cb00 0x7f6ddc13fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43578/0 (socket says 172.21.15.32:43578) 2024-04-24T23:08:44.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.462+0000 7f6de19d5640 1 -- 172.21.15.32:0/1871924780 learned_addr learned my addr 172.21.15.32:0/1871924780 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:44.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.462+0000 7f6de09d3640 1 --2- 172.21.15.32:0/1871924780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ddc15b380 0x7f6ddc13d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:44.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.462+0000 7f6de11d4640 1 --2- 172.21.15.32:0/1871924780 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ddc0ce430 0x7f6ddc152dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:44.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.462+0000 7f6de19d5640 1 -- 172.21.15.32:0/1871924780 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ddc0ce430 msgr2=0x7f6ddc152dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:44.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.462+0000 7f6de19d5640 1 --2- 172.21.15.32:0/1871924780 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ddc0ce430 0x7f6ddc152dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:44.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.462+0000 7f6de19d5640 1 -- 172.21.15.32:0/1871924780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ddc15b380 msgr2=0x7f6ddc13d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:44.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.462+0000 7f6de19d5640 1 --2- 172.21.15.32:0/1871924780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ddc15b380 0x7f6ddc13d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:44.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.462+0000 7f6de19d5640 1 -- 172.21.15.32:0/1871924780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ddc13d8b0 con 0x7f6ddc15cb00 2024-04-24T23:08:44.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.462+0000 7f6de19d5640 1 --2- 172.21.15.32:0/1871924780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ddc15cb00 0x7f6ddc13fbc0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4007e70 tx=0x7f6dd400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:44.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.462+0000 7f6dd1ffb640 1 -- 172.21.15.32:0/1871924780 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd401a070 con 0x7f6ddc15cb00 2024-04-24T23:08:44.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.462+0000 7f6dd1ffb640 1 -- 172.21.15.32:0/1871924780 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dd4003bb0 con 0x7f6ddc15cb00 2024-04-24T23:08:44.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.462+0000 7f6de3438640 1 -- 172.21.15.32:0/1871924780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ddc140100 con 0x7f6ddc15cb00 2024-04-24T23:08:44.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.462+0000 7f6dd1ffb640 1 -- 172.21.15.32:0/1871924780 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd4004030 con 0x7f6ddc15cb00 2024-04-24T23:08:44.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.463+0000 7f6de3438640 1 -- 172.21.15.32:0/1871924780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ddc140660 con 0x7f6ddc15cb00 2024-04-24T23:08:44.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.463+0000 7f6de3438640 1 -- 172.21.15.32:0/1871924780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ddc068430 con 0x7f6ddc15cb00 2024-04-24T23:08:44.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.464+0000 7f6dd1ffb640 1 -- 172.21.15.32:0/1871924780 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6dd401d070 con 0x7f6ddc15cb00 2024-04-24T23:08:44.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.464+0000 7f6dd1ffb640 1 --2- 172.21.15.32:0/1871924780 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6dc006be30 0x7f6dc006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:44.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.465+0000 7f6dd1ffb640 1 -- 172.21.15.32:0/1871924780 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7f6dd408b0b0 con 0x7f6ddc15cb00 2024-04-24T23:08:44.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.465+0000 7f6dd1ffb640 1 -- 172.21.15.32:0/1871924780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=159}) -- 0x7f6dc0073960 con 0x7f6ddc15cb00 2024-04-24T23:08:44.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.465+0000 7f6de11d4640 1 --2- 172.21.15.32:0/1871924780 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6dc006be30 0x7f6dc006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:44.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.465+0000 7f6de11d4640 1 --2- 172.21.15.32:0/1871924780 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6dc006be30 0x7f6dc006e2f0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8009a20 tx=0x7f6dd8009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:44.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.469+0000 7f6dd1ffb640 1 -- 172.21.15.32:0/1871924780 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6ddc140100 con 0x7f6ddc15cb00 2024-04-24T23:08:44.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.649+0000 7f6de3438640 1 -- 172.21.15.32:0/1871924780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6ddc0afd60 con 0x7f6ddc15cb00 2024-04-24T23:08:44.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.650+0000 7f6dd1ffb640 1 -- 172.21.15.32:0/1871924780 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v158) ==== 93+0+10 (secure 0 0 0) 0x7f6dd4010070 con 0x7f6ddc15cb00 2024-04-24T23:08:44.652 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T23:08:44.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.652+0000 7f6de3438640 1 -- 172.21.15.32:0/1871924780 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6dc006be30 msgr2=0x7f6dc006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:44.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.652+0000 7f6de3438640 1 --2- 172.21.15.32:0/1871924780 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6dc006be30 0x7f6dc006e2f0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8009a20 tx=0x7f6dd8009190 comp rx=0 tx=0).stop 2024-04-24T23:08:44.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.652+0000 7f6de3438640 1 -- 172.21.15.32:0/1871924780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ddc15cb00 msgr2=0x7f6ddc13fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:44.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.652+0000 7f6de3438640 1 --2- 172.21.15.32:0/1871924780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ddc15cb00 0x7f6ddc13fbc0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4007e70 tx=0x7f6dd400d4a0 comp rx=0 tx=0).stop 2024-04-24T23:08:44.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.652+0000 7f6de3438640 1 -- 172.21.15.32:0/1871924780 shutdown_connections 2024-04-24T23:08:44.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.652+0000 7f6de3438640 1 --2- 172.21.15.32:0/1871924780 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6dc006be30 0x7f6dc006e2f0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:44.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.652+0000 7f6de3438640 1 --2- 172.21.15.32:0/1871924780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ddc15cb00 0x7f6ddc13fbc0 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:44.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.652+0000 7f6de3438640 1 --2- 172.21.15.32:0/1871924780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ddc15b380 0x7f6ddc13d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:44.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.652+0000 7f6de3438640 1 --2- 172.21.15.32:0/1871924780 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ddc0ce430 0x7f6ddc152dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:44.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.652+0000 7f6de3438640 1 -- 172.21.15.32:0/1871924780 >> 172.21.15.32:0/1871924780 conn(0x7f6ddc06b110 msgr2=0x7f6ddc15d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:44.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.652+0000 7f6de3438640 1 -- 172.21.15.32:0/1871924780 shutdown_connections 2024-04-24T23:08:44.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.652+0000 7f6de3438640 1 -- 172.21.15.32:0/1871924780 wait complete. 2024-04-24T23:08:44.664 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:08:44.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.766+0000 7f776e693640 1 Processor -- start 2024-04-24T23:08:44.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.766+0000 7f776e693640 1 -- start start 2024-04-24T23:08:44.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.767+0000 7f776e693640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f77680b5af0 0x7f77680ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:44.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.767+0000 7f776e693640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7768162e10 0x7f776815ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:44.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.767+0000 7f776e693640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77680cd270 con 0x7f7768162320 2024-04-24T23:08:44.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.767+0000 7f776e693640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f77680b5660 con 0x7f7768162e10 2024-04-24T23:08:44.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.767+0000 7f776e693640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f7768071610 con 0x7f77680b5af0 2024-04-24T23:08:44.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.767+0000 7f776cc30640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7768162e10 0x7f776815ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:44.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.767+0000 7f776cc30640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7768162e10 0x7f776815ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43582/0 (socket says 172.21.15.32:43582) 2024-04-24T23:08:44.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.767+0000 7f776cc30640 1 -- 172.21.15.32:0/3201532218 learned_addr learned my addr 172.21.15.32:0/3201532218 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:44.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.767+0000 7f77677fe640 1 --2- 172.21.15.32:0/3201532218 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f77680b5af0 0x7f77680ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:44.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.767+0000 7f776cc30640 1 -- 172.21.15.32:0/3201532218 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f77680b5af0 msgr2=0x7f77680ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:44.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.767+0000 7f776cc30640 1 --2- 172.21.15.32:0/3201532218 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f77680b5af0 0x7f77680ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:44.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.767+0000 7f776cc30640 1 -- 172.21.15.32:0/3201532218 >> v1:172.21.15.121:6789/0 conn(0x7f7768162320 legacy=0x7f7768162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:44.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.767+0000 7f776cc30640 1 -- 172.21.15.32:0/3201532218 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77680718f0 con 0x7f7768162e10 2024-04-24T23:08:44.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.768+0000 7f776cc30640 1 --2- 172.21.15.32:0/3201532218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7768162e10 0x7f776815ed30 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f7760009e50 tx=0x7f776002a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95fff6fb57627c9b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:44.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.768+0000 7f7766ffd640 1 -- 172.21.15.32:0/3201532218 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7760036070 con 0x7f7768162e10 2024-04-24T23:08:44.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.768+0000 7f7766ffd640 1 -- 172.21.15.32:0/3201532218 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7760031600 con 0x7f7768162e10 2024-04-24T23:08:44.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.768+0000 7f7766ffd640 1 -- 172.21.15.32:0/3201532218 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77600319d0 con 0x7f7768162e10 2024-04-24T23:08:44.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.769+0000 7f776e693640 1 -- 172.21.15.32:0/3201532218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7768162e10 msgr2=0x7f776815ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:44.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.769+0000 7f776e693640 1 --2- 172.21.15.32:0/3201532218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7768162e10 0x7f776815ed30 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f7760009e50 tx=0x7f776002a5b0 comp rx=0 tx=0).stop 2024-04-24T23:08:44.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.769+0000 7f776e693640 1 -- 172.21.15.32:0/3201532218 shutdown_connections 2024-04-24T23:08:44.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.769+0000 7f776e693640 1 --2- 172.21.15.32:0/3201532218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7768162e10 0x7f776815ed30 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:44.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.769+0000 7f776e693640 1 --2- 172.21.15.32:0/3201532218 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f77680b5af0 0x7f77680ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:44.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.769+0000 7f776e693640 1 -- 172.21.15.32:0/3201532218 >> 172.21.15.32:0/3201532218 conn(0x7f77680d0ae0 msgr2=0x7f77680d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:44.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.769+0000 7f776e693640 1 -- 172.21.15.32:0/3201532218 shutdown_connections 2024-04-24T23:08:44.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.769+0000 7f776e693640 1 -- 172.21.15.32:0/3201532218 wait complete. 2024-04-24T23:08:44.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.770+0000 7f776e693640 1 Processor -- start 2024-04-24T23:08:44.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.770+0000 7f776e693640 1 -- start start 2024-04-24T23:08:44.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.771+0000 7f776e693640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f77680b5af0 0x7f77680cfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:44.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.771+0000 7f776e693640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7768162320 0x7f7768151f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:44.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.771+0000 7f776e693640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7768162e10 0x7f7768161b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:44.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.771+0000 7f776e693640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7768071bd0 con 0x7f7768162320 2024-04-24T23:08:44.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.771+0000 7f776e693640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7768067950 con 0x7f7768162e10 2024-04-24T23:08:44.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.771+0000 7f776e693640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f7768071d50 con 0x7f77680b5af0 2024-04-24T23:08:44.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.771+0000 7f77677fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7768162320 0x7f7768151f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:44.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.771+0000 7f77677fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7768162320 0x7f7768151f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43594/0 (socket says 172.21.15.32:43594) 2024-04-24T23:08:44.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.771+0000 7f77677fe640 1 -- 172.21.15.32:0/3589761570 learned_addr learned my addr 172.21.15.32:0/3589761570 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:44.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.771+0000 7f776cc30640 1 --2- 172.21.15.32:0/3589761570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7768162e10 0x7f7768161b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:44.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.771+0000 7f7767fff640 1 --2- 172.21.15.32:0/3589761570 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f77680b5af0 0x7f77680cfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:44.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.771+0000 7f77677fe640 1 -- 172.21.15.32:0/3589761570 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f77680b5af0 msgr2=0x7f77680cfeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:44.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.771+0000 7f77677fe640 1 --2- 172.21.15.32:0/3589761570 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f77680b5af0 0x7f77680cfeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:44.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.771+0000 7f77677fe640 1 -- 172.21.15.32:0/3589761570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7768162e10 msgr2=0x7f7768161b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:44.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.771+0000 7f77677fe640 1 --2- 172.21.15.32:0/3589761570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7768162e10 0x7f7768161b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:44.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.771+0000 7f77677fe640 1 -- 172.21.15.32:0/3589761570 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77680cd620 con 0x7f7768162320 2024-04-24T23:08:44.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.772+0000 7f77677fe640 1 --2- 172.21.15.32:0/3589761570 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7768162320 0x7f7768151f80 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f775800d7d0 tx=0x7f775800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:44.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.772+0000 7f7764ff9640 1 -- 172.21.15.32:0/3589761570 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775800b6c0 con 0x7f7768162320 2024-04-24T23:08:44.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.772+0000 7f7764ff9640 1 -- 172.21.15.32:0/3589761570 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7758019070 con 0x7f7768162320 2024-04-24T23:08:44.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.772+0000 7f7764ff9640 1 -- 172.21.15.32:0/3589761570 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775800bac0 con 0x7f7768162320 2024-04-24T23:08:44.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.772+0000 7f776e693640 1 -- 172.21.15.32:0/3589761570 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77680cd900 con 0x7f7768162320 2024-04-24T23:08:44.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.772+0000 7f776e693640 1 -- 172.21.15.32:0/3589761570 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77680cdec0 con 0x7f7768162320 2024-04-24T23:08:44.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.773+0000 7f776e693640 1 -- 172.21.15.32:0/3589761570 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7768068430 con 0x7f7768162320 2024-04-24T23:08:44.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.773+0000 7f7764ff9640 1 -- 172.21.15.32:0/3589761570 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f775800bc60 con 0x7f7768162320 2024-04-24T23:08:44.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.774+0000 7f7764ff9640 1 --2- 172.21.15.32:0/3589761570 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f774806be30 0x7f774806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:44.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.774+0000 7f7764ff9640 1 -- 172.21.15.32:0/3589761570 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7f775808b140 con 0x7f7768162320 2024-04-24T23:08:44.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.774+0000 7f7767fff640 1 --2- 172.21.15.32:0/3589761570 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f774806be30 0x7f774806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:44.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.774+0000 7f7764ff9640 1 -- 172.21.15.32:0/3589761570 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=159}) -- 0x7f7748073960 con 0x7f7768162320 2024-04-24T23:08:44.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.774+0000 7f7767fff640 1 --2- 172.21.15.32:0/3589761570 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f774806be30 0x7f774806e2f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f7750004f60 tx=0x7f7750007e30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:44.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.778+0000 7f7764ff9640 1 -- 172.21.15.32:0/3589761570 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7758059530 con 0x7f7768162320 2024-04-24T23:08:44.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.962+0000 7f776e693640 1 -- 172.21.15.32:0/3589761570 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7768067730 con 0x7f7768162320 2024-04-24T23:08:44.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.962+0000 7f7764ff9640 1 -- 172.21.15.32:0/3589761570 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v158) ==== 104+0+11 (secure 0 0 0) 0x7f775805d830 con 0x7f7768162320 2024-04-24T23:08:44.964 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-24T23:08:44.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.964+0000 7f776e693640 1 -- 172.21.15.32:0/3589761570 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f774806be30 msgr2=0x7f774806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:44.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.964+0000 7f776e693640 1 --2- 172.21.15.32:0/3589761570 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f774806be30 0x7f774806e2f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f7750004f60 tx=0x7f7750007e30 comp rx=0 tx=0).stop 2024-04-24T23:08:44.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.964+0000 7f776e693640 1 -- 172.21.15.32:0/3589761570 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7768162320 msgr2=0x7f7768151f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:44.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.964+0000 7f776e693640 1 --2- 172.21.15.32:0/3589761570 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7768162320 0x7f7768151f80 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f775800d7d0 tx=0x7f775800dc90 comp rx=0 tx=0).stop 2024-04-24T23:08:44.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.964+0000 7f776e693640 1 -- 172.21.15.32:0/3589761570 shutdown_connections 2024-04-24T23:08:44.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.964+0000 7f776e693640 1 --2- 172.21.15.32:0/3589761570 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f774806be30 0x7f774806e2f0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:44.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.964+0000 7f776e693640 1 --2- 172.21.15.32:0/3589761570 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7768162320 0x7f7768151f80 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:44.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.964+0000 7f776e693640 1 --2- 172.21.15.32:0/3589761570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7768162e10 0x7f7768161b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:44.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.964+0000 7f776e693640 1 --2- 172.21.15.32:0/3589761570 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f77680b5af0 0x7f77680cfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:44.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.964+0000 7f776e693640 1 -- 172.21.15.32:0/3589761570 >> 172.21.15.32:0/3589761570 conn(0x7f77680d0ae0 msgr2=0x7f776815eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:44.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.965+0000 7f776e693640 1 -- 172.21.15.32:0/3589761570 shutdown_connections 2024-04-24T23:08:44.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:44.965+0000 7f776e693640 1 -- 172.21.15.32:0/3589761570 wait complete. 2024-04-24T23:08:44.977 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:08:45.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.079+0000 7f0e5a97e640 1 Processor -- start 2024-04-24T23:08:45.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.079+0000 7f0e5a97e640 1 -- start start 2024-04-24T23:08:45.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.080+0000 7f0e5a97e640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0e5406c480 0x7f0e540b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:45.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.080+0000 7f0e5a97e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e540b0770 con 0x7f0e540b0b80 2024-04-24T23:08:45.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.080+0000 7f0e5a97e640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e540b1010 con 0x7f0e54074210 2024-04-24T23:08:45.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.080+0000 7f0e5a97e640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f0e5406c8d0 con 0x7f0e5406c480 2024-04-24T23:08:45.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.080+0000 7f0e537fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0e540b0b80 0x7f0e540ae140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52318/0 (socket says 172.21.15.32:52318) 2024-04-24T23:08:45.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.080+0000 7f0e537fe640 1 -- 172.21.15.32:0/1075053324 learned_addr learned my addr 172.21.15.32:0/1075053324 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:08:45.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.080+0000 7f0e58f1b640 1 --2- 172.21.15.32:0/1075053324 >> v2:172.21.15.121:3300/0 conn(0x7f0e5406c480 0x7f0e540b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:45.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.080+0000 7f0e52ffd640 1 -- 172.21.15.32:0/1075053324 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4020555970 0 0) 0x7f0e540b0770 con 0x7f0e540b0b80 2024-04-24T23:08:45.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.080+0000 7f0e52ffd640 1 -- 172.21.15.32:0/1075053324 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e40003520 con 0x7f0e540b0b80 2024-04-24T23:08:45.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.081+0000 7f0e52ffd640 1 -- 172.21.15.32:0/1075053324 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2599995337 0 0) 0x7f0e40003520 con 0x7f0e540b0b80 2024-04-24T23:08:45.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.081+0000 7f0e52ffd640 1 -- 172.21.15.32:0/1075053324 >> v2:172.21.15.121:3300/0 conn(0x7f0e5406c480 msgr2=0x7f0e540b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:45.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.081+0000 7f0e52ffd640 1 --2- 172.21.15.32:0/1075053324 >> v2:172.21.15.121:3300/0 conn(0x7f0e5406c480 0x7f0e540b20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:45.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.081+0000 7f0e52ffd640 1 -- 172.21.15.32:0/1075053324 >> v1:172.21.15.121:6789/0 conn(0x7f0e54074210 legacy=0x7f0e540745f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:45.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.081+0000 7f0e52ffd640 1 -- 172.21.15.32:0/1075053324 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e540b12f0 con 0x7f0e540b0b80 2024-04-24T23:08:45.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.081+0000 7f0e52ffd640 1 -- 172.21.15.32:0/1075053324 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f0e44001670 con 0x7f0e540b0b80 2024-04-24T23:08:45.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.081+0000 7f0e52ffd640 1 -- 172.21.15.32:0/1075053324 >> v1:172.21.15.32:6789/0 conn(0x7f0e540b0b80 legacy=0x7f0e540ae140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:45.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.081+0000 7f0e52ffd640 1 --2- 172.21.15.32:0/1075053324 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e400039b0 0x7f0e40003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:45.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.081+0000 7f0e52ffd640 1 --2- 172.21.15.32:0/1075053324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e40004800 0x7f0e40004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:45.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.081+0000 7f0e52ffd640 1 --2- 172.21.15.32:0/1075053324 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e400051c0 0x7f0e4000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:45.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.081+0000 7f0e52ffd640 1 -- 172.21.15.32:0/1075053324 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4000c030 con 0x7f0e40004800 2024-04-24T23:08:45.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.081+0000 7f0e52ffd640 1 -- 172.21.15.32:0/1075053324 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4000c1e0 con 0x7f0e400039b0 2024-04-24T23:08:45.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.081+0000 7f0e52ffd640 1 -- 172.21.15.32:0/1075053324 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f0e4000c3e0 con 0x7f0e400051c0 2024-04-24T23:08:45.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.081+0000 7f0e537fe640 1 --2- 172.21.15.32:0/1075053324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e40004800 0x7f0e40004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:45.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.081+0000 7f0e53fff640 1 --2- 172.21.15.32:0/1075053324 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e400039b0 0x7f0e40003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:45.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.081+0000 7f0e58f1b640 1 --2- 172.21.15.32:0/1075053324 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e400051c0 0x7f0e4000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:45.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.081+0000 7f0e537fe640 1 -- 172.21.15.32:0/1075053324 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e400051c0 msgr2=0x7f0e4000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:45.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.081+0000 7f0e537fe640 1 --2- 172.21.15.32:0/1075053324 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e400051c0 0x7f0e4000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:45.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.081+0000 7f0e537fe640 1 -- 172.21.15.32:0/1075053324 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e400039b0 msgr2=0x7f0e40003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:45.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.081+0000 7f0e537fe640 1 --2- 172.21.15.32:0/1075053324 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e400039b0 0x7f0e40003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:45.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.081+0000 7f0e537fe640 1 -- 172.21.15.32:0/1075053324 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e40003520 con 0x7f0e40004800 2024-04-24T23:08:45.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.081+0000 7f0e58f1b640 1 -- 172.21.15.32:0/1075053324 reap_dead start 2024-04-24T23:08:45.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.082+0000 7f0e537fe640 1 --2- 172.21.15.32:0/1075053324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e40004800 0x7f0e40004c80 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f0e4400c240 tx=0x7f0e40005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98bb47307b92cdba server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:45.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.082+0000 7f0e52ffd640 1 -- 172.21.15.32:0/1075053324 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e44007d10 con 0x7f0e40004800 2024-04-24T23:08:45.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.082+0000 7f0e52ffd640 1 -- 172.21.15.32:0/1075053324 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e44005ad0 con 0x7f0e40004800 2024-04-24T23:08:45.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.082+0000 7f0e52ffd640 1 -- 172.21.15.32:0/1075053324 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e44007070 con 0x7f0e40004800 2024-04-24T23:08:45.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.082+0000 7f0e5a97e640 1 -- 172.21.15.32:0/1075053324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e40004800 msgr2=0x7f0e40004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:45.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.082+0000 7f0e5a97e640 1 --2- 172.21.15.32:0/1075053324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e40004800 0x7f0e40004c80 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f0e4400c240 tx=0x7f0e40005830 comp rx=0 tx=0).stop 2024-04-24T23:08:45.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.083+0000 7f0e5a97e640 1 -- 172.21.15.32:0/1075053324 shutdown_connections 2024-04-24T23:08:45.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.083+0000 7f0e5a97e640 1 --2- 172.21.15.32:0/1075053324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e40004800 0x7f0e40004c80 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:45.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.083+0000 7f0e5a97e640 1 -- 172.21.15.32:0/1075053324 >> 172.21.15.32:0/1075053324 conn(0x7f0e540b2e60 msgr2=0x7f0e540b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:45.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.083+0000 7f0e5a97e640 1 -- 172.21.15.32:0/1075053324 shutdown_connections 2024-04-24T23:08:45.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.083+0000 7f0e5a97e640 1 -- 172.21.15.32:0/1075053324 wait complete. 2024-04-24T23:08:45.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.084+0000 7f0e5a97e640 1 Processor -- start 2024-04-24T23:08:45.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.084+0000 7f0e5a97e640 1 -- start start 2024-04-24T23:08:45.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.085+0000 7f0e5a97e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e40004800 0x7f0e541f5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:45.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.085+0000 7f0e5a97e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e540ce0d0 0x7f0e5409a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:45.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.085+0000 7f0e5a97e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e541f7090 0x7f0e541f7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:45.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.085+0000 7f0e5a97e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0e540b18b0 con 0x7f0e540ce0d0 2024-04-24T23:08:45.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.085+0000 7f0e5a97e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0e540b1c20 con 0x7f0e40004800 2024-04-24T23:08:45.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.085+0000 7f0e5a97e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f0e540b3ba0 con 0x7f0e541f7090 2024-04-24T23:08:45.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.085+0000 7f0e537fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e540ce0d0 0x7f0e5409a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:45.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.085+0000 7f0e537fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e540ce0d0 0x7f0e5409a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43614/0 (socket says 172.21.15.32:43614) 2024-04-24T23:08:45.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.085+0000 7f0e537fe640 1 -- 172.21.15.32:0/2128253970 learned_addr learned my addr 172.21.15.32:0/2128253970 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:45.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.085+0000 7f0e58f1b640 1 --2- 172.21.15.32:0/2128253970 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e541f7090 0x7f0e541f7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:45.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.085+0000 7f0e53fff640 1 --2- 172.21.15.32:0/2128253970 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e40004800 0x7f0e541f5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:45.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.085+0000 7f0e537fe640 1 -- 172.21.15.32:0/2128253970 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e541f7090 msgr2=0x7f0e541f7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:45.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.085+0000 7f0e537fe640 1 --2- 172.21.15.32:0/2128253970 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e541f7090 0x7f0e541f7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:45.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.085+0000 7f0e537fe640 1 -- 172.21.15.32:0/2128253970 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e40004800 msgr2=0x7f0e541f5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:45.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.085+0000 7f0e537fe640 1 --2- 172.21.15.32:0/2128253970 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e40004800 0x7f0e541f5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:45.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.085+0000 7f0e537fe640 1 -- 172.21.15.32:0/2128253970 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e541f5ff0 con 0x7f0e540ce0d0 2024-04-24T23:08:45.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.085+0000 7f0e537fe640 1 --2- 172.21.15.32:0/2128253970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e540ce0d0 0x7f0e5409a860 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f0e4400c240 tx=0x7f0e540b1b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:45.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.085+0000 7f0e50ff9640 1 -- 172.21.15.32:0/2128253970 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4403f020 con 0x7f0e540ce0d0 2024-04-24T23:08:45.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.086+0000 7f0e50ff9640 1 -- 172.21.15.32:0/2128253970 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e44001470 con 0x7f0e540ce0d0 2024-04-24T23:08:45.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.086+0000 7f0e50ff9640 1 -- 172.21.15.32:0/2128253970 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e440049e0 con 0x7f0e540ce0d0 2024-04-24T23:08:45.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.086+0000 7f0e5a97e640 1 -- 172.21.15.32:0/2128253970 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e541f62d0 con 0x7f0e540ce0d0 2024-04-24T23:08:45.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.086+0000 7f0e5a97e640 1 -- 172.21.15.32:0/2128253970 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e541f6890 con 0x7f0e540ce0d0 2024-04-24T23:08:45.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.087+0000 7f0e5a97e640 1 -- 172.21.15.32:0/2128253970 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e54068430 con 0x7f0e540ce0d0 2024-04-24T23:08:45.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.087+0000 7f0e50ff9640 1 -- 172.21.15.32:0/2128253970 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0e44042020 con 0x7f0e540ce0d0 2024-04-24T23:08:45.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.088+0000 7f0e50ff9640 1 --2- 172.21.15.32:0/2128253970 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0e3406be30 0x7f0e3406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:45.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.088+0000 7f0e50ff9640 1 -- 172.21.15.32:0/2128253970 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7f0e440aa8d0 con 0x7f0e540ce0d0 2024-04-24T23:08:45.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.088+0000 7f0e53fff640 1 --2- 172.21.15.32:0/2128253970 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0e3406be30 0x7f0e3406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:45.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.088+0000 7f0e50ff9640 1 -- 172.21.15.32:0/2128253970 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=159}) -- 0x7f0e340739c0 con 0x7f0e540ce0d0 2024-04-24T23:08:45.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.088+0000 7f0e53fff640 1 --2- 172.21.15.32:0/2128253970 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0e3406be30 0x7f0e3406e2f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c000c00 tx=0x7f0e3c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:45.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.092+0000 7f0e50ff9640 1 -- 172.21.15.32:0/2128253970 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0e44078c40 con 0x7f0e540ce0d0 2024-04-24T23:08:45.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.276+0000 7f0e5a97e640 1 -- 172.21.15.32:0/2128253970 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0e540b3980 con 0x7f0e540ce0d0 2024-04-24T23:08:45.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.277+0000 7f0e50ff9640 1 -- 172.21.15.32:0/2128253970 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v158) ==== 100+0+11 (secure 0 0 0) 0x7f0e4407cf40 con 0x7f0e540ce0d0 2024-04-24T23:08:45.278 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-24T23:08:45.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.279+0000 7f0e5a97e640 1 -- 172.21.15.32:0/2128253970 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0e3406be30 msgr2=0x7f0e3406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:45.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.279+0000 7f0e5a97e640 1 --2- 172.21.15.32:0/2128253970 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0e3406be30 0x7f0e3406e2f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c000c00 tx=0x7f0e3c009040 comp rx=0 tx=0).stop 2024-04-24T23:08:45.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.279+0000 7f0e5a97e640 1 -- 172.21.15.32:0/2128253970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e540ce0d0 msgr2=0x7f0e5409a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:45.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.279+0000 7f0e5a97e640 1 --2- 172.21.15.32:0/2128253970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e540ce0d0 0x7f0e5409a860 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f0e4400c240 tx=0x7f0e540b1b90 comp rx=0 tx=0).stop 2024-04-24T23:08:45.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.279+0000 7f0e5a97e640 1 -- 172.21.15.32:0/2128253970 shutdown_connections 2024-04-24T23:08:45.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.279+0000 7f0e5a97e640 1 --2- 172.21.15.32:0/2128253970 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0e3406be30 0x7f0e3406e2f0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:45.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.279+0000 7f0e5a97e640 1 --2- 172.21.15.32:0/2128253970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e540ce0d0 0x7f0e5409a860 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:45.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.279+0000 7f0e5a97e640 1 --2- 172.21.15.32:0/2128253970 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e541f7090 0x7f0e541f7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:45.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.279+0000 7f0e5a97e640 1 --2- 172.21.15.32:0/2128253970 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e40004800 0x7f0e541f5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:45.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.279+0000 7f0e5a97e640 1 -- 172.21.15.32:0/2128253970 >> 172.21.15.32:0/2128253970 conn(0x7f0e540b2e60 msgr2=0x7f0e540b1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:45.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.279+0000 7f0e5a97e640 1 -- 172.21.15.32:0/2128253970 shutdown_connections 2024-04-24T23:08:45.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.279+0000 7f0e5a97e640 1 -- 172.21.15.32:0/2128253970 wait complete. 2024-04-24T23:08:45.291 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T23:08:45.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.391+0000 7fa3bd3a5640 1 Processor -- start 2024-04-24T23:08:45.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.392+0000 7fa3bd3a5640 1 -- start start 2024-04-24T23:08:45.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.392+0000 7fa3bd3a5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa3b815e9b0 0x7fa3b80d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:45.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.392+0000 7fa3bd3a5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa3b80d4210 0x7fa3b806b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:45.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.392+0000 7fa3bd3a5640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3b80d3170 con 0x7fa3b80ba0e0 2024-04-24T23:08:45.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.392+0000 7fa3bd3a5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fa3b80b8900 con 0x7fa3b815e9b0 2024-04-24T23:08:45.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.392+0000 7fa3bd3a5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa3b80d3450 con 0x7fa3b80d4210 2024-04-24T23:08:45.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.392+0000 7fa3b6ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa3b815e9b0 0x7fa3b80d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:45.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.393+0000 7fa3b6ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa3b815e9b0 0x7fa3b80d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43630/0 (socket says 172.21.15.32:43630) 2024-04-24T23:08:45.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.393+0000 7fa3b6ffd640 1 -- 172.21.15.32:0/506873109 learned_addr learned my addr 172.21.15.32:0/506873109 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:45.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.393+0000 7fa3b67fc640 1 --2- 172.21.15.32:0/506873109 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa3b80d4210 0x7fa3b806b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:45.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.393+0000 7fa3b6ffd640 1 -- 172.21.15.32:0/506873109 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa3b80d4210 msgr2=0x7fa3b806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:45.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.393+0000 7fa3b6ffd640 1 --2- 172.21.15.32:0/506873109 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa3b80d4210 0x7fa3b806b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:45.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.393+0000 7fa3b6ffd640 1 -- 172.21.15.32:0/506873109 >> v1:172.21.15.121:6789/0 conn(0x7fa3b80ba0e0 legacy=0x7fa3b8071730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:45.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.393+0000 7fa3b6ffd640 1 -- 172.21.15.32:0/506873109 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3b80d3730 con 0x7fa3b815e9b0 2024-04-24T23:08:45.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.393+0000 7fa3b67fc640 1 --2- 172.21.15.32:0/506873109 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa3b80d4210 0x7fa3b806b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:08:45.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.394+0000 7fa3b6ffd640 1 --2- 172.21.15.32:0/506873109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3b815e9b0 0x7fa3b80d2b90 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fa3a8009870 tx=0x7fa3a802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11d375e1dc730d44 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:45.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.394+0000 7fa3b5ffb640 1 -- 172.21.15.32:0/506873109 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a8037070 con 0x7fa3b815e9b0 2024-04-24T23:08:45.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.394+0000 7fa3b5ffb640 1 -- 172.21.15.32:0/506873109 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3a8032440 con 0x7fa3b815e9b0 2024-04-24T23:08:45.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.394+0000 7fa3b5ffb640 1 -- 172.21.15.32:0/506873109 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a8032810 con 0x7fa3b815e9b0 2024-04-24T23:08:45.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.394+0000 7fa3bd3a5640 1 -- 172.21.15.32:0/506873109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3b815e9b0 msgr2=0x7fa3b80d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:45.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.394+0000 7fa3bd3a5640 1 --2- 172.21.15.32:0/506873109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3b815e9b0 0x7fa3b80d2b90 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fa3a8009870 tx=0x7fa3a802a210 comp rx=0 tx=0).stop 2024-04-24T23:08:45.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.394+0000 7fa3bd3a5640 1 -- 172.21.15.32:0/506873109 shutdown_connections 2024-04-24T23:08:45.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.394+0000 7fa3bd3a5640 1 --2- 172.21.15.32:0/506873109 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa3b80d4210 0x7fa3b806b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:45.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.394+0000 7fa3bd3a5640 1 --2- 172.21.15.32:0/506873109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3b815e9b0 0x7fa3b80d2b90 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:45.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.394+0000 7fa3bd3a5640 1 -- 172.21.15.32:0/506873109 >> 172.21.15.32:0/506873109 conn(0x7fa3b80d11d0 msgr2=0x7fa3b80d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:45.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.394+0000 7fa3bd3a5640 1 -- 172.21.15.32:0/506873109 shutdown_connections 2024-04-24T23:08:45.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.394+0000 7fa3bd3a5640 1 -- 172.21.15.32:0/506873109 wait complete. 2024-04-24T23:08:45.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.396+0000 7fa3bd3a5640 1 Processor -- start 2024-04-24T23:08:45.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.396+0000 7fa3bd3a5640 1 -- start start 2024-04-24T23:08:45.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.396+0000 7fa3bd3a5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa3b80ba0e0 0x7fa3b8152c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:45.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.396+0000 7fa3bd3a5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa3b80d4210 0x7fa3b81f4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:45.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.396+0000 7fa3bd3a5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3b815e9b0 0x7fa3b81f5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:45.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.396+0000 7fa3bd3a5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b80d2400 con 0x7fa3b815e9b0 2024-04-24T23:08:45.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.396+0000 7fa3bd3a5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b80d3cf0 con 0x7fa3b80d4210 2024-04-24T23:08:45.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.396+0000 7fa3bd3a5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa3b806b650 con 0x7fa3b80ba0e0 2024-04-24T23:08:45.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.396+0000 7fa3b77fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3b815e9b0 0x7fa3b81f5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:45.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.396+0000 7fa3b77fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3b815e9b0 0x7fa3b81f5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43646/0 (socket says 172.21.15.32:43646) 2024-04-24T23:08:45.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.397+0000 7fa3b77fe640 1 -- 172.21.15.32:0/2228004198 learned_addr learned my addr 172.21.15.32:0/2228004198 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:45.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.397+0000 7fa3b6ffd640 1 --2- 172.21.15.32:0/2228004198 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa3b80ba0e0 0x7fa3b8152c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:45.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.397+0000 7fa3b67fc640 1 --2- 172.21.15.32:0/2228004198 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa3b80d4210 0x7fa3b81f4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:45.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.397+0000 7fa3b77fe640 1 -- 172.21.15.32:0/2228004198 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa3b80ba0e0 msgr2=0x7fa3b8152c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:45.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.397+0000 7fa3b77fe640 1 --2- 172.21.15.32:0/2228004198 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa3b80ba0e0 0x7fa3b8152c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:45.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.397+0000 7fa3b77fe640 1 -- 172.21.15.32:0/2228004198 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa3b80d4210 msgr2=0x7fa3b81f4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:45.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.397+0000 7fa3b77fe640 1 --2- 172.21.15.32:0/2228004198 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa3b80d4210 0x7fa3b81f4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:45.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.397+0000 7fa3b77fe640 1 -- 172.21.15.32:0/2228004198 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3b81f5690 con 0x7fa3b815e9b0 2024-04-24T23:08:45.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.397+0000 7fa3b77fe640 1 --2- 172.21.15.32:0/2228004198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3b815e9b0 0x7fa3b81f5150 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fa3b000c780 tx=0x7fa3b000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:45.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.397+0000 7fa39f7fe640 1 -- 172.21.15.32:0/2228004198 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b0004b60 con 0x7fa3b815e9b0 2024-04-24T23:08:45.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.397+0000 7fa39f7fe640 1 -- 172.21.15.32:0/2228004198 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3b0004d00 con 0x7fa3b815e9b0 2024-04-24T23:08:45.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.397+0000 7fa39f7fe640 1 -- 172.21.15.32:0/2228004198 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b000aa90 con 0x7fa3b815e9b0 2024-04-24T23:08:45.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.397+0000 7fa3bd3a5640 1 -- 172.21.15.32:0/2228004198 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3b8153280 con 0x7fa3b815e9b0 2024-04-24T23:08:45.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.397+0000 7fa3bd3a5640 1 -- 172.21.15.32:0/2228004198 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3b8153840 con 0x7fa3b815e9b0 2024-04-24T23:08:45.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.398+0000 7fa3bd3a5640 1 -- 172.21.15.32:0/2228004198 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3b8068430 con 0x7fa3b815e9b0 2024-04-24T23:08:45.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.399+0000 7fa39f7fe640 1 -- 172.21.15.32:0/2228004198 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa3b000ac30 con 0x7fa3b815e9b0 2024-04-24T23:08:45.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.399+0000 7fa39f7fe640 1 --2- 172.21.15.32:0/2228004198 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa3a406be30 0x7fa3a406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:45.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.399+0000 7fa39f7fe640 1 -- 172.21.15.32:0/2228004198 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7fa3b008afd0 con 0x7fa3b815e9b0 2024-04-24T23:08:45.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.399+0000 7fa39f7fe640 1 -- 172.21.15.32:0/2228004198 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=159}) -- 0x7fa3a4073960 con 0x7fa3b815e9b0 2024-04-24T23:08:45.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.399+0000 7fa3b6ffd640 1 --2- 172.21.15.32:0/2228004198 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa3a406be30 0x7fa3a406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:45.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.400+0000 7fa3b6ffd640 1 --2- 172.21.15.32:0/2228004198 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa3a406be30 0x7fa3a406e2f0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fa3a8034000 tx=0x7fa3a802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:45.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.403+0000 7fa39f7fe640 1 -- 172.21.15.32:0/2228004198 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa3b0020070 con 0x7fa3b815e9b0 2024-04-24T23:08:45.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.598+0000 7fa3bd3a5640 1 -- 172.21.15.32:0/2228004198 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa3b8072160 con 0x7fa3b815e9b0 2024-04-24T23:08:45.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.598+0000 7fa39f7fe640 1 -- 172.21.15.32:0/2228004198 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v158) ==== 80+0+0 (secure 0 0 0) 0x7fa3b0059330 con 0x7fa3b815e9b0 2024-04-24T23:08:45.600 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2024-04-24T23:08:45.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.600+0000 7fa3bd3a5640 1 -- 172.21.15.32:0/2228004198 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa3a406be30 msgr2=0x7fa3a406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:45.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.600+0000 7fa3bd3a5640 1 --2- 172.21.15.32:0/2228004198 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa3a406be30 0x7fa3a406e2f0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fa3a8034000 tx=0x7fa3a802e000 comp rx=0 tx=0).stop 2024-04-24T23:08:45.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.600+0000 7fa3bd3a5640 1 -- 172.21.15.32:0/2228004198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3b815e9b0 msgr2=0x7fa3b81f5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:45.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.600+0000 7fa3bd3a5640 1 --2- 172.21.15.32:0/2228004198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3b815e9b0 0x7fa3b81f5150 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fa3b000c780 tx=0x7fa3b000cc40 comp rx=0 tx=0).stop 2024-04-24T23:08:45.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.600+0000 7fa3bd3a5640 1 -- 172.21.15.32:0/2228004198 shutdown_connections 2024-04-24T23:08:45.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.600+0000 7fa3bd3a5640 1 --2- 172.21.15.32:0/2228004198 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa3a406be30 0x7fa3a406e2f0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:45.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.600+0000 7fa3bd3a5640 1 --2- 172.21.15.32:0/2228004198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3b815e9b0 0x7fa3b81f5150 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:45.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.600+0000 7fa3bd3a5640 1 --2- 172.21.15.32:0/2228004198 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa3b80d4210 0x7fa3b81f4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:45.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.600+0000 7fa3bd3a5640 1 --2- 172.21.15.32:0/2228004198 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa3b80ba0e0 0x7fa3b8152c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:45.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.600+0000 7fa3bd3a5640 1 -- 172.21.15.32:0/2228004198 >> 172.21.15.32:0/2228004198 conn(0x7fa3b80d11d0 msgr2=0x7fa3b80cabb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:45.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.600+0000 7fa3bd3a5640 1 -- 172.21.15.32:0/2228004198 shutdown_connections 2024-04-24T23:08:45.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.600+0000 7fa3bd3a5640 1 -- 172.21.15.32:0/2228004198 wait complete. 2024-04-24T23:08:45.612 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-24T23:08:45.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:08:45.613 DEBUG:teuthology.orchestra.run.smithi121:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-24T23:08:45.644 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2024-04-24T23:08:45.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:08:45.644 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-24T23:08:45.702 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:08:45.702 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:08:45.702 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:08:45.718 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=413 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T23:08:45.718 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:08:45.718 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:08:45.718 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:08:45.759 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3271 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T23:08:45.760 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:08:45.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.858+0000 7f637fc9a640 1 Processor -- start 2024-04-24T23:08:45.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.859+0000 7f637fc9a640 1 -- start start 2024-04-24T23:08:45.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.859+0000 7f637fc9a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f637815e9e0 0x7f63780d40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:45.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.859+0000 7f637fc9a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f63780ba390 0x7f63780d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:45.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.859+0000 7f637fc9a640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63780d3010 con 0x7f637806b110 2024-04-24T23:08:45.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.859+0000 7f637fc9a640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f63780d32f0 con 0x7f63780ba390 2024-04-24T23:08:45.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.859+0000 7f637fc9a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f63780d35d0 con 0x7f637815e9e0 2024-04-24T23:08:45.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.859+0000 7f637d235640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f63780ba390 0x7f63780d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:45.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.860+0000 7f637d235640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f63780ba390 0x7f63780d26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43654/0 (socket says 172.21.15.32:43654) 2024-04-24T23:08:45.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.860+0000 7f637d235640 1 -- 172.21.15.32:0/1433259684 learned_addr learned my addr 172.21.15.32:0/1433259684 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:45.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.860+0000 7f637d235640 1 -- 172.21.15.32:0/1433259684 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f637815e9e0 msgr2=0x7f63780d40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:45.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.860+0000 7f637d235640 1 --2- 172.21.15.32:0/1433259684 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f637815e9e0 0x7f63780d40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:45.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.860+0000 7f637d235640 1 -- 172.21.15.32:0/1433259684 >> v1:172.21.15.121:6789/0 conn(0x7f637806b110 legacy=0x7f6378071730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:45.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.860+0000 7f637d235640 1 -- 172.21.15.32:0/1433259684 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63780d38b0 con 0x7f63780ba390 2024-04-24T23:08:45.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.861+0000 7f637d235640 1 --2- 172.21.15.32:0/1433259684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63780ba390 0x7f63780d26a0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f6368009870 tx=0x7f636802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20f76e5c24ff9fb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:45.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.861+0000 7f637ca34640 1 -- 172.21.15.32:0/1433259684 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6368037070 con 0x7f63780ba390 2024-04-24T23:08:45.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.861+0000 7f637ca34640 1 -- 172.21.15.32:0/1433259684 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6368032440 con 0x7f63780ba390 2024-04-24T23:08:45.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.861+0000 7f637ca34640 1 -- 172.21.15.32:0/1433259684 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6368032810 con 0x7f63780ba390 2024-04-24T23:08:45.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.861+0000 7f637fc9a640 1 -- 172.21.15.32:0/1433259684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63780ba390 msgr2=0x7f63780d26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:45.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.861+0000 7f637fc9a640 1 --2- 172.21.15.32:0/1433259684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63780ba390 0x7f63780d26a0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f6368009870 tx=0x7f636802a210 comp rx=0 tx=0).stop 2024-04-24T23:08:45.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.861+0000 7f637fc9a640 1 -- 172.21.15.32:0/1433259684 shutdown_connections 2024-04-24T23:08:45.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.861+0000 7f637fc9a640 1 --2- 172.21.15.32:0/1433259684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63780ba390 0x7f63780d26a0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:45.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.861+0000 7f637fc9a640 1 --2- 172.21.15.32:0/1433259684 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f637815e9e0 0x7f63780d40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:45.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.861+0000 7f637fc9a640 1 -- 172.21.15.32:0/1433259684 >> 172.21.15.32:0/1433259684 conn(0x7f63780d12e0 msgr2=0x7f63780d16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:45.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.861+0000 7f637fc9a640 1 -- 172.21.15.32:0/1433259684 shutdown_connections 2024-04-24T23:08:45.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.861+0000 7f637fc9a640 1 -- 172.21.15.32:0/1433259684 wait complete. 2024-04-24T23:08:45.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.862+0000 7f637fc9a640 1 Processor -- start 2024-04-24T23:08:45.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.863+0000 7f637fc9a640 1 -- start start 2024-04-24T23:08:45.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.863+0000 7f637fc9a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f637806b110 0x7f637815f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:45.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.863+0000 7f637fc9a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f63780ba390 0x7f63780c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:45.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.863+0000 7f637fc9a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f637815e9e0 0x7f63780ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:45.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.863+0000 7f637fc9a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f63780d2d60 con 0x7f637806b110 2024-04-24T23:08:45.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.863+0000 7f637fc9a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f63780d2be0 con 0x7f63780ba390 2024-04-24T23:08:45.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.863+0000 7f637fc9a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f637806b5c0 con 0x7f637815e9e0 2024-04-24T23:08:45.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.863+0000 7f637da36640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f637806b110 0x7f637815f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:45.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.863+0000 7f637da36640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f637806b110 0x7f637815f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43662/0 (socket says 172.21.15.32:43662) 2024-04-24T23:08:45.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.863+0000 7f637da36640 1 -- 172.21.15.32:0/1732505780 learned_addr learned my addr 172.21.15.32:0/1732505780 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:45.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.864+0000 7f637d235640 1 --2- 172.21.15.32:0/1732505780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f63780ba390 0x7f63780c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:45.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.864+0000 7f637e237640 1 --2- 172.21.15.32:0/1732505780 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f637815e9e0 0x7f63780ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:45.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.864+0000 7f637da36640 1 -- 172.21.15.32:0/1732505780 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f637815e9e0 msgr2=0x7f63780ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:45.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.864+0000 7f637da36640 1 --2- 172.21.15.32:0/1732505780 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f637815e9e0 0x7f63780ca2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:45.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.864+0000 7f637da36640 1 -- 172.21.15.32:0/1732505780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f63780ba390 msgr2=0x7f63780c9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:45.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.864+0000 7f637da36640 1 --2- 172.21.15.32:0/1732505780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f63780ba390 0x7f63780c9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:45.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.864+0000 7f637da36640 1 -- 172.21.15.32:0/1732505780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63780cbe40 con 0x7f637806b110 2024-04-24T23:08:45.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.864+0000 7f637d235640 1 --2- 172.21.15.32:0/1732505780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f63780ba390 0x7f63780c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:08:45.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.864+0000 7f637e237640 1 --2- 172.21.15.32:0/1732505780 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f637815e9e0 0x7f63780ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:08:45.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.864+0000 7f637da36640 1 --2- 172.21.15.32:0/1732505780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f637806b110 0x7f637815f030 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f637400b410 tx=0x7f637400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:45.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.864+0000 7f636e7fc640 1 -- 172.21.15.32:0/1732505780 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6374015020 con 0x7f637806b110 2024-04-24T23:08:45.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.864+0000 7f636e7fc640 1 -- 172.21.15.32:0/1732505780 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6374011850 con 0x7f637806b110 2024-04-24T23:08:45.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.864+0000 7f636e7fc640 1 -- 172.21.15.32:0/1732505780 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6374011c40 con 0x7f637806b110 2024-04-24T23:08:45.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.864+0000 7f637fc9a640 1 -- 172.21.15.32:0/1732505780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63780cc120 con 0x7f637806b110 2024-04-24T23:08:45.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.864+0000 7f637fc9a640 1 -- 172.21.15.32:0/1732505780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63780cc6e0 con 0x7f637806b110 2024-04-24T23:08:45.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.865+0000 7f637fc9a640 1 -- 172.21.15.32:0/1732505780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6378068430 con 0x7f637806b110 2024-04-24T23:08:45.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.866+0000 7f636e7fc640 1 -- 172.21.15.32:0/1732505780 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6374018020 con 0x7f637806b110 2024-04-24T23:08:45.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.866+0000 7f636e7fc640 1 --2- 172.21.15.32:0/1732505780 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f635c06be30 0x7f635c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:45.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.866+0000 7f636e7fc640 1 -- 172.21.15.32:0/1732505780 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7f637408ad10 con 0x7f637806b110 2024-04-24T23:08:45.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.866+0000 7f636e7fc640 1 -- 172.21.15.32:0/1732505780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=159}) -- 0x7f635c073960 con 0x7f637806b110 2024-04-24T23:08:45.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.867+0000 7f637d235640 1 --2- 172.21.15.32:0/1732505780 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f635c06be30 0x7f635c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:45.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.867+0000 7f637d235640 1 --2- 172.21.15.32:0/1732505780 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f635c06be30 0x7f635c06e2f0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f6368034000 tx=0x7f636802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:45.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:45.871+0000 7f636e7fc640 1 -- 172.21.15.32:0/1732505780 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f637401f020 con 0x7f637806b110 2024-04-24T23:08:46.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.090+0000 7f637fc9a640 1 -- 172.21.15.32:0/1732505780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f63780cfd20 con 0x7f637806b110 2024-04-24T23:08:46.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.091+0000 7f636e7fc640 1 -- 172.21.15.32:0/1732505780 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+5186 (secure 0 0 0) 0x7f63740172c0 con 0x7f637806b110 2024-04-24T23:08:46.092 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:08:46.093 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":65,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7120,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/1196305174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1196305174},{"type":"v1","addr":"172.21.15.121:6836","nonce":1196305174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7123,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3015902884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3015902884},{"type":"v1","addr":"172.21.15.121:6837","nonce":3015902884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":8280,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/1033273628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1033273628},{"type":"v1","addr":"172.21.15.32:6833","nonce":1033273628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:06:31.291599+0000","modified":"2024-04-24T23:06:33.296657+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8286},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8286":{"gid":8286,"name":"a","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/984603360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":984603360},{"type":"v1","addr":"172.21.15.32:6835","nonce":984603360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8286,"qdb_cluster":[8286]},"id":9}]} 2024-04-24T23:08:46.093 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 65 2024-04-24T23:08:46.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.092+0000 7f637fc9a640 1 -- 172.21.15.32:0/1732505780 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f635c06be30 msgr2=0x7f635c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:46.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.093+0000 7f637fc9a640 1 --2- 172.21.15.32:0/1732505780 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f635c06be30 0x7f635c06e2f0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f6368034000 tx=0x7f636802e000 comp rx=0 tx=0).stop 2024-04-24T23:08:46.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.093+0000 7f637fc9a640 1 -- 172.21.15.32:0/1732505780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f637806b110 msgr2=0x7f637815f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:46.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.093+0000 7f637fc9a640 1 --2- 172.21.15.32:0/1732505780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f637806b110 0x7f637815f030 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f637400b410 tx=0x7f637400b8d0 comp rx=0 tx=0).stop 2024-04-24T23:08:46.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.093+0000 7f637fc9a640 1 -- 172.21.15.32:0/1732505780 shutdown_connections 2024-04-24T23:08:46.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.093+0000 7f637fc9a640 1 --2- 172.21.15.32:0/1732505780 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f635c06be30 0x7f635c06e2f0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:46.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.093+0000 7f637fc9a640 1 --2- 172.21.15.32:0/1732505780 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f637815e9e0 0x7f63780ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:46.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.093+0000 7f637fc9a640 1 --2- 172.21.15.32:0/1732505780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f63780ba390 0x7f63780c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:46.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.093+0000 7f637fc9a640 1 --2- 172.21.15.32:0/1732505780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f637806b110 0x7f637815f030 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:46.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.093+0000 7f637fc9a640 1 -- 172.21.15.32:0/1732505780 >> 172.21.15.32:0/1732505780 conn(0x7f63780d12e0 msgr2=0x7f63780b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:46.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.093+0000 7f637fc9a640 1 -- 172.21.15.32:0/1732505780 shutdown_connections 2024-04-24T23:08:46.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.093+0000 7f637fc9a640 1 -- 172.21.15.32:0/1732505780 wait complete. 2024-04-24T23:08:46.105 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:08:46.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.204+0000 7f77de163640 1 Processor -- start 2024-04-24T23:08:46.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.204+0000 7f77de163640 1 -- start start 2024-04-24T23:08:46.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.205+0000 7f77de163640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f77d80af890 0x7f77d81670f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:46.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.205+0000 7f77de163640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77d80b6430 con 0x7f77d806c5f0 2024-04-24T23:08:46.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.205+0000 7f77de163640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77d8156f80 con 0x7f77d80b44d0 2024-04-24T23:08:46.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.205+0000 7f77de163640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f77d80b6150 con 0x7f77d80af890 2024-04-24T23:08:46.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.205+0000 7f77d7fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f77d80af890 0x7f77d81670f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:46.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.205+0000 7f77d77fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f77d806c5f0 0x7f77d806c9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52334/0 (socket says 172.21.15.32:52334) 2024-04-24T23:08:46.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.205+0000 7f77d7fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f77d80af890 0x7f77d81670f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43674/0 (socket says 172.21.15.32:43674) 2024-04-24T23:08:46.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.205+0000 7f77d77fe640 1 -- 172.21.15.32:0/3463214334 learned_addr learned my addr 172.21.15.32:0/3463214334 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:08:46.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.205+0000 7f77d7fff640 1 -- 172.21.15.32:0/3463214334 >> v1:172.21.15.121:6789/0 conn(0x7f77d80b44d0 legacy=0x7f77d80b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:46.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.205+0000 7f77d7fff640 1 -- 172.21.15.32:0/3463214334 >> v1:172.21.15.32:6789/0 conn(0x7f77d806c5f0 legacy=0x7f77d806c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:46.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.205+0000 7f77d7fff640 1 -- 172.21.15.32:0/3463214334 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d80b39f0 con 0x7f77d80af890 2024-04-24T23:08:46.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.205+0000 7f77d67fc640 1 -- 172.21.15.32:0/3463214334 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2334724566 0 0) 0x7f77d80b6430 con 0x7f77d806c5f0 2024-04-24T23:08:46.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.206+0000 7f77d7fff640 1 --2- 172.21.15.32:0/3463214334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77d80af890 0x7f77d81670f0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f77d0009d90 tx=0x7f77d002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44ed08b6b7fb358d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:46.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.206+0000 7f77d67fc640 1 -- 172.21.15.32:0/3463214334 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d0007740 con 0x7f77d80af890 2024-04-24T23:08:46.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.206+0000 7f77d67fc640 1 -- 172.21.15.32:0/3463214334 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77d00078e0 con 0x7f77d80af890 2024-04-24T23:08:46.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.206+0000 7f77d67fc640 1 -- 172.21.15.32:0/3463214334 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d0007cb0 con 0x7f77d80af890 2024-04-24T23:08:46.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.207+0000 7f77de163640 1 -- 172.21.15.32:0/3463214334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77d80af890 msgr2=0x7f77d81670f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:46.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.207+0000 7f77de163640 1 --2- 172.21.15.32:0/3463214334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77d80af890 0x7f77d81670f0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f77d0009d90 tx=0x7f77d002a6f0 comp rx=0 tx=0).stop 2024-04-24T23:08:46.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.207+0000 7f77de163640 1 -- 172.21.15.32:0/3463214334 shutdown_connections 2024-04-24T23:08:46.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.207+0000 7f77de163640 1 --2- 172.21.15.32:0/3463214334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77d80af890 0x7f77d81670f0 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:46.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.207+0000 7f77de163640 1 -- 172.21.15.32:0/3463214334 >> 172.21.15.32:0/3463214334 conn(0x7f77d806b110 msgr2=0x7f77d806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:46.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.207+0000 7f77de163640 1 -- 172.21.15.32:0/3463214334 shutdown_connections 2024-04-24T23:08:46.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.207+0000 7f77de163640 1 -- 172.21.15.32:0/3463214334 wait complete. 2024-04-24T23:08:46.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.208+0000 7f77de163640 1 Processor -- start 2024-04-24T23:08:46.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.208+0000 7f77de163640 1 -- start start 2024-04-24T23:08:46.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.209+0000 7f77de163640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77d806c5f0 0x7f77d8153a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:46.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.209+0000 7f77de163640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f77d80af890 0x7f77d813f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:46.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.209+0000 7f77de163640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77d80b44d0 0x7f77d80724a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:46.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.209+0000 7f77de163640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f77d8167630 con 0x7f77d806c5f0 2024-04-24T23:08:46.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.209+0000 7f77de163640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f77d80b3fb0 con 0x7f77d80b44d0 2024-04-24T23:08:46.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.209+0000 7f77de163640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f77d81677b0 con 0x7f77d80af890 2024-04-24T23:08:46.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.209+0000 7f77d77fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77d806c5f0 0x7f77d8153a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:46.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.209+0000 7f77d77fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77d806c5f0 0x7f77d8153a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43684/0 (socket says 172.21.15.32:43684) 2024-04-24T23:08:46.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.209+0000 7f77d77fe640 1 -- 172.21.15.32:0/298275783 learned_addr learned my addr 172.21.15.32:0/298275783 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:46.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.209+0000 7f77d77fe640 1 -- 172.21.15.32:0/298275783 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f77d80af890 msgr2=0x7f77d813f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:46.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.209+0000 7f77d77fe640 1 --2- 172.21.15.32:0/298275783 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f77d80af890 0x7f77d813f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:46.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.209+0000 7f77d77fe640 1 -- 172.21.15.32:0/298275783 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77d80b44d0 msgr2=0x7f77d80724a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:46.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.209+0000 7f77d77fe640 1 --2- 172.21.15.32:0/298275783 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77d80b44d0 0x7f77d80724a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:46.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.209+0000 7f77d77fe640 1 -- 172.21.15.32:0/298275783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d813fae0 con 0x7f77d806c5f0 2024-04-24T23:08:46.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.209+0000 7f77d77fe640 1 --2- 172.21.15.32:0/298275783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77d806c5f0 0x7f77d8153a60 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f77c800c4a0 tx=0x7f77c800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:46.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.210+0000 7f77bffff640 1 -- 172.21.15.32:0/298275783 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c8018070 con 0x7f77d806c5f0 2024-04-24T23:08:46.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.210+0000 7f77bffff640 1 -- 172.21.15.32:0/298275783 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77c80074e0 con 0x7f77d806c5f0 2024-04-24T23:08:46.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.210+0000 7f77de163640 1 -- 172.21.15.32:0/298275783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77d8154040 con 0x7f77d806c5f0 2024-04-24T23:08:46.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.210+0000 7f77bffff640 1 -- 172.21.15.32:0/298275783 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c8004a20 con 0x7f77d806c5f0 2024-04-24T23:08:46.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.210+0000 7f77de163640 1 -- 172.21.15.32:0/298275783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77d8154600 con 0x7f77d806c5f0 2024-04-24T23:08:46.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.211+0000 7f77de163640 1 -- 172.21.15.32:0/298275783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77d8068430 con 0x7f77d806c5f0 2024-04-24T23:08:46.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.211+0000 7f77bffff640 1 -- 172.21.15.32:0/298275783 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f77c801b070 con 0x7f77d806c5f0 2024-04-24T23:08:46.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.211+0000 7f77bffff640 1 --2- 172.21.15.32:0/298275783 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f77c406be60 0x7f77c406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:46.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.212+0000 7f77bffff640 1 -- 172.21.15.32:0/298275783 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7f77c808a100 con 0x7f77d806c5f0 2024-04-24T23:08:46.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.212+0000 7f77bffff640 1 -- 172.21.15.32:0/298275783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=159}) -- 0x7f77c40739f0 con 0x7f77d806c5f0 2024-04-24T23:08:46.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.212+0000 7f77d6ffd640 1 --2- 172.21.15.32:0/298275783 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f77c406be60 0x7f77c406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:46.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.212+0000 7f77d6ffd640 1 --2- 172.21.15.32:0/298275783 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f77c406be60 0x7f77c406e320 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f77cc000f60 tx=0x7f77cc0079b0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:46.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.216+0000 7f77bffff640 1 -- 172.21.15.32:0/298275783 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f77d813fae0 con 0x7f77d806c5f0 2024-04-24T23:08:46.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.398+0000 7f77de163640 1 -- 172.21.15.32:0/298275783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f77d80b5670 con 0x7f77d806c5f0 2024-04-24T23:08:46.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.399+0000 7f77bffff640 1 -- 172.21.15.32:0/298275783 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v158) ==== 74+0+16666 (secure 0 0 0) 0x7f77c8058b40 con 0x7f77d806c5f0 2024-04-24T23:08:46.401 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:08:46.401 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":158,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:08:06.831677+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:06:29.591736+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-24T23:06:30.267629+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:08:46.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.401+0000 7f77de163640 1 -- 172.21.15.32:0/298275783 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f77c406be60 msgr2=0x7f77c406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:46.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.401+0000 7f77de163640 1 --2- 172.21.15.32:0/298275783 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f77c406be60 0x7f77c406e320 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f77cc000f60 tx=0x7f77cc0079b0 comp rx=0 tx=0).stop 2024-04-24T23:08:46.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.401+0000 7f77de163640 1 -- 172.21.15.32:0/298275783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77d806c5f0 msgr2=0x7f77d8153a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:46.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.401+0000 7f77de163640 1 --2- 172.21.15.32:0/298275783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77d806c5f0 0x7f77d8153a60 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f77c800c4a0 tx=0x7f77c800c960 comp rx=0 tx=0).stop 2024-04-24T23:08:46.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.401+0000 7f77de163640 1 -- 172.21.15.32:0/298275783 shutdown_connections 2024-04-24T23:08:46.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.401+0000 7f77de163640 1 --2- 172.21.15.32:0/298275783 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f77c406be60 0x7f77c406e320 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:46.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.401+0000 7f77de163640 1 --2- 172.21.15.32:0/298275783 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77d80b44d0 0x7f77d80724a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:46.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.401+0000 7f77de163640 1 --2- 172.21.15.32:0/298275783 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f77d80af890 0x7f77d813f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:46.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.401+0000 7f77de163640 1 --2- 172.21.15.32:0/298275783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77d806c5f0 0x7f77d8153a60 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:46.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.401+0000 7f77de163640 1 -- 172.21.15.32:0/298275783 >> 172.21.15.32:0/298275783 conn(0x7f77d806b110 msgr2=0x7f77d806c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:46.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.401+0000 7f77de163640 1 -- 172.21.15.32:0/298275783 shutdown_connections 2024-04-24T23:08:46.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.401+0000 7f77de163640 1 -- 172.21.15.32:0/298275783 wait complete. 2024-04-24T23:08:46.414 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:06:29.591736+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '155', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:06:30.267629+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '158', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:08:46.415 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:08:46.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.516+0000 7fcaddfd5640 1 Processor -- start 2024-04-24T23:08:46.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.516+0000 7fcaddfd5640 1 -- start start 2024-04-24T23:08:46.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.517+0000 7fcaddfd5640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fcad815c5b0 0x7fcad80b4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:46.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.517+0000 7fcaddfd5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcad806cd00 0x7fcad80b4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:46.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.517+0000 7fcaddfd5640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcad815c990 con 0x7fcad8072cf0 2024-04-24T23:08:46.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.517+0000 7fcaddfd5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fcad8072860 con 0x7fcad806cd00 2024-04-24T23:08:46.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.517+0000 7fcaddfd5640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fcad8071c50 con 0x7fcad815c5b0 2024-04-24T23:08:46.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.517+0000 7fcad7fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcad806cd00 0x7fcad80b4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:46.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.517+0000 7fcad7fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcad806cd00 0x7fcad80b4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43696/0 (socket says 172.21.15.32:43696) 2024-04-24T23:08:46.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.517+0000 7fcad7fff640 1 -- 172.21.15.32:0/218260031 learned_addr learned my addr 172.21.15.32:0/218260031 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:46.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.517+0000 7fcad77fe640 1 --2- 172.21.15.32:0/218260031 >> v2:172.21.15.121:3300/0 conn(0x7fcad815c5b0 0x7fcad80b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:46.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.518+0000 7fcad7fff640 1 -- 172.21.15.32:0/218260031 >> v2:172.21.15.121:3300/0 conn(0x7fcad815c5b0 msgr2=0x7fcad80b4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:46.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.518+0000 7fcad7fff640 1 --2- 172.21.15.32:0/218260031 >> v2:172.21.15.121:3300/0 conn(0x7fcad815c5b0 0x7fcad80b4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:46.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.518+0000 7fcad7fff640 1 -- 172.21.15.32:0/218260031 >> v1:172.21.15.121:6789/0 conn(0x7fcad8072cf0 legacy=0x7fcad806c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:46.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.518+0000 7fcad7fff640 1 -- 172.21.15.32:0/218260031 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcad8071f30 con 0x7fcad806cd00 2024-04-24T23:08:46.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.518+0000 7fcad7fff640 1 --2- 172.21.15.32:0/218260031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcad806cd00 0x7fcad80b4b40 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fcad8067850 tx=0x7fcad002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97fa4fe90541bbc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:46.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.518+0000 7fcad67fc640 1 -- 172.21.15.32:0/218260031 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad0036070 con 0x7fcad806cd00 2024-04-24T23:08:46.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.518+0000 7fcad67fc640 1 -- 172.21.15.32:0/218260031 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcad0031600 con 0x7fcad806cd00 2024-04-24T23:08:46.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.518+0000 7fcad67fc640 1 -- 172.21.15.32:0/218260031 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad00319d0 con 0x7fcad806cd00 2024-04-24T23:08:46.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.519+0000 7fcaddfd5640 1 -- 172.21.15.32:0/218260031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcad806cd00 msgr2=0x7fcad80b4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:46.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.519+0000 7fcaddfd5640 1 --2- 172.21.15.32:0/218260031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcad806cd00 0x7fcad80b4b40 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fcad8067850 tx=0x7fcad002a660 comp rx=0 tx=0).stop 2024-04-24T23:08:46.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.519+0000 7fcaddfd5640 1 -- 172.21.15.32:0/218260031 shutdown_connections 2024-04-24T23:08:46.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.519+0000 7fcaddfd5640 1 --2- 172.21.15.32:0/218260031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcad806cd00 0x7fcad80b4b40 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:46.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.519+0000 7fcaddfd5640 1 --2- 172.21.15.32:0/218260031 >> v2:172.21.15.121:3300/0 conn(0x7fcad815c5b0 0x7fcad80b4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:46.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.519+0000 7fcaddfd5640 1 -- 172.21.15.32:0/218260031 >> 172.21.15.32:0/218260031 conn(0x7fcad806b110 msgr2=0x7fcad806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:46.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.519+0000 7fcaddfd5640 1 -- 172.21.15.32:0/218260031 shutdown_connections 2024-04-24T23:08:46.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.519+0000 7fcaddfd5640 1 -- 172.21.15.32:0/218260031 wait complete. 2024-04-24T23:08:46.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.520+0000 7fcaddfd5640 1 Processor -- start 2024-04-24T23:08:46.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.520+0000 7fcaddfd5640 1 -- start start 2024-04-24T23:08:46.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.521+0000 7fcaddfd5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcad806cd00 0x7fcad814a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:46.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.521+0000 7fcaddfd5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcad8072cf0 0x7fcad814d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:46.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.521+0000 7fcaddfd5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcad815c5b0 0x7fcad814b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:46.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.521+0000 7fcaddfd5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcad80b4570 con 0x7fcad8072cf0 2024-04-24T23:08:46.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.521+0000 7fcaddfd5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fcad80724f0 con 0x7fcad806cd00 2024-04-24T23:08:46.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.521+0000 7fcaddfd5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fcad80b46f0 con 0x7fcad815c5b0 2024-04-24T23:08:46.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.521+0000 7fcad6ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcad8072cf0 0x7fcad814d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:46.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.521+0000 7fcad6ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcad8072cf0 0x7fcad814d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43698/0 (socket says 172.21.15.32:43698) 2024-04-24T23:08:46.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.521+0000 7fcad6ffd640 1 -- 172.21.15.32:0/2771215163 learned_addr learned my addr 172.21.15.32:0/2771215163 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:46.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.521+0000 7fcad77fe640 1 --2- 172.21.15.32:0/2771215163 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcad806cd00 0x7fcad814a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:46.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.521+0000 7fcad7fff640 1 --2- 172.21.15.32:0/2771215163 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcad815c5b0 0x7fcad814b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:46.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.521+0000 7fcad6ffd640 1 -- 172.21.15.32:0/2771215163 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcad815c5b0 msgr2=0x7fcad814b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:46.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.521+0000 7fcad6ffd640 1 --2- 172.21.15.32:0/2771215163 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcad815c5b0 0x7fcad814b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:46.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.521+0000 7fcad6ffd640 1 -- 172.21.15.32:0/2771215163 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcad806cd00 msgr2=0x7fcad814a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:46.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.522+0000 7fcad6ffd640 1 --2- 172.21.15.32:0/2771215163 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcad806cd00 0x7fcad814a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:46.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.522+0000 7fcad6ffd640 1 -- 172.21.15.32:0/2771215163 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcad814dbc0 con 0x7fcad8072cf0 2024-04-24T23:08:46.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.522+0000 7fcad77fe640 1 --2- 172.21.15.32:0/2771215163 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcad806cd00 0x7fcad814a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:08:46.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.522+0000 7fcad7fff640 1 --2- 172.21.15.32:0/2771215163 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcad815c5b0 0x7fcad814b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:08:46.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.522+0000 7fcad6ffd640 1 --2- 172.21.15.32:0/2771215163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcad8072cf0 0x7fcad814d120 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fcacc00c430 tx=0x7fcacc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:46.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.522+0000 7fcabffff640 1 -- 172.21.15.32:0/2771215163 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcacc011790 con 0x7fcad8072cf0 2024-04-24T23:08:46.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.522+0000 7fcabffff640 1 -- 172.21.15.32:0/2771215163 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcacc017070 con 0x7fcad8072cf0 2024-04-24T23:08:46.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.522+0000 7fcaddfd5640 1 -- 172.21.15.32:0/2771215163 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcad814dea0 con 0x7fcad8072cf0 2024-04-24T23:08:46.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.522+0000 7fcabffff640 1 -- 172.21.15.32:0/2771215163 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcacc011bb0 con 0x7fcad8072cf0 2024-04-24T23:08:46.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.522+0000 7fcaddfd5640 1 -- 172.21.15.32:0/2771215163 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcad80cf080 con 0x7fcad8072cf0 2024-04-24T23:08:46.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.523+0000 7fcaddfd5640 1 -- 172.21.15.32:0/2771215163 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcad80cf280 con 0x7fcad8072cf0 2024-04-24T23:08:46.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.523+0000 7fcabffff640 1 -- 172.21.15.32:0/2771215163 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcacc01c460 con 0x7fcad8072cf0 2024-04-24T23:08:46.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.524+0000 7fcabffff640 1 --2- 172.21.15.32:0/2771215163 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcac406be30 0x7fcac406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:46.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.524+0000 7fcabffff640 1 -- 172.21.15.32:0/2771215163 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7fcacc08acd0 con 0x7fcad8072cf0 2024-04-24T23:08:46.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.524+0000 7fcabffff640 1 -- 172.21.15.32:0/2771215163 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=159}) -- 0x7fcac4073960 con 0x7fcad8072cf0 2024-04-24T23:08:46.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.524+0000 7fcad77fe640 1 --2- 172.21.15.32:0/2771215163 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcac406be30 0x7fcac406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:46.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.524+0000 7fcad77fe640 1 --2- 172.21.15.32:0/2771215163 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcac406be30 0x7fcac406e2f0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fcac8006fd0 tx=0x7fcac8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:46.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.528+0000 7fcabffff640 1 -- 172.21.15.32:0/2771215163 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcad80cf280 con 0x7fcad8072cf0 2024-04-24T23:08:46.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.710+0000 7fcaddfd5640 1 -- 172.21.15.32:0/2771215163 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcad806bc10 con 0x7fcad8072cf0 2024-04-24T23:08:46.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.710+0000 7fcabffff640 1 -- 172.21.15.32:0/2771215163 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v158) ==== 93+0+10 (secure 0 0 0) 0x7fcacc059680 con 0x7fcad8072cf0 2024-04-24T23:08:46.712 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T23:08:46.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.712+0000 7fcaddfd5640 1 -- 172.21.15.32:0/2771215163 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcac406be30 msgr2=0x7fcac406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:46.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.712+0000 7fcaddfd5640 1 --2- 172.21.15.32:0/2771215163 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcac406be30 0x7fcac406e2f0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fcac8006fd0 tx=0x7fcac8008040 comp rx=0 tx=0).stop 2024-04-24T23:08:46.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.712+0000 7fcaddfd5640 1 -- 172.21.15.32:0/2771215163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcad8072cf0 msgr2=0x7fcad814d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:46.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.712+0000 7fcaddfd5640 1 --2- 172.21.15.32:0/2771215163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcad8072cf0 0x7fcad814d120 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fcacc00c430 tx=0x7fcacc00c8f0 comp rx=0 tx=0).stop 2024-04-24T23:08:46.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.712+0000 7fcaddfd5640 1 -- 172.21.15.32:0/2771215163 shutdown_connections 2024-04-24T23:08:46.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.712+0000 7fcaddfd5640 1 --2- 172.21.15.32:0/2771215163 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fcac406be30 0x7fcac406e2f0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:46.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.712+0000 7fcaddfd5640 1 --2- 172.21.15.32:0/2771215163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcad8072cf0 0x7fcad814d120 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:46.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.712+0000 7fcaddfd5640 1 --2- 172.21.15.32:0/2771215163 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcad815c5b0 0x7fcad814b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:46.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.712+0000 7fcaddfd5640 1 --2- 172.21.15.32:0/2771215163 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcad806cd00 0x7fcad814a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:46.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.712+0000 7fcaddfd5640 1 -- 172.21.15.32:0/2771215163 >> 172.21.15.32:0/2771215163 conn(0x7fcad806b110 msgr2=0x7fcad80cc500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:46.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.712+0000 7fcaddfd5640 1 -- 172.21.15.32:0/2771215163 shutdown_connections 2024-04-24T23:08:46.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.712+0000 7fcaddfd5640 1 -- 172.21.15.32:0/2771215163 wait complete. 2024-04-24T23:08:46.725 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:08:46.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.826+0000 7f5688eda640 1 Processor -- start 2024-04-24T23:08:46.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.826+0000 7f5688eda640 1 -- start start 2024-04-24T23:08:46.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.827+0000 7f5688eda640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f568415b380 0x7f568406c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:46.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.827+0000 7f5688eda640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f568415cb00 0x7f568406cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:46.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.827+0000 7f5688eda640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f56840ce430 0x7f56840ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:46.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.827+0000 7f5688eda640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f56840b4520 con 0x7f56840ce430 2024-04-24T23:08:46.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.827+0000 7f5688eda640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f56840b4800 con 0x7f568415cb00 2024-04-24T23:08:46.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.827+0000 7f5688eda640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f56840b4ae0 con 0x7f568415b380 2024-04-24T23:08:46.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.827+0000 7f5682d9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f56840ce430 0x7f56840ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:46.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.828+0000 7f5682d9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f56840ce430 0x7f56840ce880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43708/0 (socket says 172.21.15.32:43708) 2024-04-24T23:08:46.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.828+0000 7f5682d9d640 1 -- 172.21.15.32:0/24203493 learned_addr learned my addr 172.21.15.32:0/24203493 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:46.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.828+0000 7f5681d9b640 1 --2- 172.21.15.32:0/24203493 >> v2:172.21.15.121:3300/0 conn(0x7f568415cb00 0x7f568406cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:46.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.828+0000 7f568259c640 1 --2- 172.21.15.32:0/24203493 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f568415b380 0x7f568406c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:46.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.828+0000 7f5682d9d640 1 -- 172.21.15.32:0/24203493 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f568415b380 msgr2=0x7f568406c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:46.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.828+0000 7f5682d9d640 1 --2- 172.21.15.32:0/24203493 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f568415b380 0x7f568406c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:46.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.828+0000 7f5682d9d640 1 -- 172.21.15.32:0/24203493 >> v2:172.21.15.121:3300/0 conn(0x7f568415cb00 msgr2=0x7f568406cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:46.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.828+0000 7f5682d9d640 1 --2- 172.21.15.32:0/24203493 >> v2:172.21.15.121:3300/0 conn(0x7f568415cb00 0x7f568406cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:46.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.828+0000 7f5682d9d640 1 -- 172.21.15.32:0/24203493 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56840b4dc0 con 0x7f56840ce430 2024-04-24T23:08:46.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.828+0000 7f568259c640 1 --2- 172.21.15.32:0/24203493 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f568415b380 0x7f568406c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:08:46.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.828+0000 7f5682d9d640 1 --2- 172.21.15.32:0/24203493 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56840ce430 0x7f56840ce880 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f568406d830 tx=0x7f567402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9766f645f3cb6914 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:46.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.829+0000 7f568159a640 1 -- 172.21.15.32:0/24203493 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5674036070 con 0x7f56840ce430 2024-04-24T23:08:46.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.829+0000 7f568159a640 1 -- 172.21.15.32:0/24203493 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5674031600 con 0x7f56840ce430 2024-04-24T23:08:46.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.829+0000 7f568159a640 1 -- 172.21.15.32:0/24203493 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56740319d0 con 0x7f56840ce430 2024-04-24T23:08:46.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.829+0000 7f5688eda640 1 -- 172.21.15.32:0/24203493 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56840ce430 msgr2=0x7f56840ce880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:46.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.829+0000 7f5688eda640 1 --2- 172.21.15.32:0/24203493 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56840ce430 0x7f56840ce880 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f568406d830 tx=0x7f567402a610 comp rx=0 tx=0).stop 2024-04-24T23:08:46.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.829+0000 7f5688eda640 1 -- 172.21.15.32:0/24203493 shutdown_connections 2024-04-24T23:08:46.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.829+0000 7f5688eda640 1 --2- 172.21.15.32:0/24203493 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56840ce430 0x7f56840ce880 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:46.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.829+0000 7f5688eda640 1 --2- 172.21.15.32:0/24203493 >> v2:172.21.15.121:3300/0 conn(0x7f568415cb00 0x7f568406cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:46.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.829+0000 7f5688eda640 1 --2- 172.21.15.32:0/24203493 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f568415b380 0x7f568406c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:46.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.829+0000 7f5688eda640 1 -- 172.21.15.32:0/24203493 >> 172.21.15.32:0/24203493 conn(0x7f568406b110 msgr2=0x7f568406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:46.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.829+0000 7f5688eda640 1 -- 172.21.15.32:0/24203493 shutdown_connections 2024-04-24T23:08:46.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.829+0000 7f5688eda640 1 -- 172.21.15.32:0/24203493 wait complete. 2024-04-24T23:08:46.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.831+0000 7f5688eda640 1 Processor -- start 2024-04-24T23:08:46.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.831+0000 7f5688eda640 1 -- start start 2024-04-24T23:08:46.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.831+0000 7f5688eda640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56840ce430 0x7f5684141f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:46.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.831+0000 7f5688eda640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f568415b380 0x7f5684071140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:46.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.831+0000 7f5688eda640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f568415cb00 0x7f5684157560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:46.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.831+0000 7f5688eda640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f56840cedc0 con 0x7f56840ce430 2024-04-24T23:08:46.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.831+0000 7f5688eda640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f568415b760 con 0x7f568415cb00 2024-04-24T23:08:46.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.831+0000 7f5688eda640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f56840cef40 con 0x7f568415b380 2024-04-24T23:08:46.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.831+0000 7f568259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56840ce430 0x7f5684141f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:46.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.831+0000 7f568259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56840ce430 0x7f5684141f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43720/0 (socket says 172.21.15.32:43720) 2024-04-24T23:08:46.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.831+0000 7f568259c640 1 -- 172.21.15.32:0/2895846526 learned_addr learned my addr 172.21.15.32:0/2895846526 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:46.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.832+0000 7f5681d9b640 1 --2- 172.21.15.32:0/2895846526 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f568415b380 0x7f5684071140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:46.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.832+0000 7f5682d9d640 1 --2- 172.21.15.32:0/2895846526 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f568415cb00 0x7f5684157560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:46.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.832+0000 7f568259c640 1 -- 172.21.15.32:0/2895846526 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f568415b380 msgr2=0x7f5684071140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:46.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.832+0000 7f568259c640 1 --2- 172.21.15.32:0/2895846526 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f568415b380 0x7f5684071140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:46.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.832+0000 7f568259c640 1 -- 172.21.15.32:0/2895846526 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f568415cb00 msgr2=0x7f5684157560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:46.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.832+0000 7f568259c640 1 --2- 172.21.15.32:0/2895846526 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f568415cb00 0x7f5684157560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:46.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.832+0000 7f568259c640 1 -- 172.21.15.32:0/2895846526 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56841f3a10 con 0x7f56840ce430 2024-04-24T23:08:46.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.832+0000 7f5681d9b640 1 --2- 172.21.15.32:0/2895846526 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f568415b380 0x7f5684071140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:08:46.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.832+0000 7f5682d9d640 1 --2- 172.21.15.32:0/2895846526 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f568415cb00 0x7f5684157560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:08:46.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.832+0000 7f568259c640 1 --2- 172.21.15.32:0/2895846526 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56840ce430 0x7f5684141f10 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f567c00e730 tx=0x7f567c00ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:46.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.832+0000 7f5672ffd640 1 -- 172.21.15.32:0/2895846526 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c00f040 con 0x7f56840ce430 2024-04-24T23:08:46.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.832+0000 7f5672ffd640 1 -- 172.21.15.32:0/2895846526 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f567c004230 con 0x7f56840ce430 2024-04-24T23:08:46.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.832+0000 7f5688eda640 1 -- 172.21.15.32:0/2895846526 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56841f3cf0 con 0x7f56840ce430 2024-04-24T23:08:46.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.832+0000 7f5672ffd640 1 -- 172.21.15.32:0/2895846526 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c00ee40 con 0x7f56840ce430 2024-04-24T23:08:46.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.832+0000 7f5688eda640 1 -- 172.21.15.32:0/2895846526 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56841f42b0 con 0x7f56840ce430 2024-04-24T23:08:46.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.833+0000 7f5688eda640 1 -- 172.21.15.32:0/2895846526 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5684068430 con 0x7f56840ce430 2024-04-24T23:08:46.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.834+0000 7f5672ffd640 1 -- 172.21.15.32:0/2895846526 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f567c00c6d0 con 0x7f56840ce430 2024-04-24T23:08:46.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.834+0000 7f5672ffd640 1 --2- 172.21.15.32:0/2895846526 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f566406be30 0x7f566406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:46.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.834+0000 7f5672ffd640 1 -- 172.21.15.32:0/2895846526 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7f567c08aaa0 con 0x7f56840ce430 2024-04-24T23:08:46.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.834+0000 7f5681d9b640 1 --2- 172.21.15.32:0/2895846526 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f566406be30 0x7f566406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:46.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.834+0000 7f5672ffd640 1 -- 172.21.15.32:0/2895846526 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=159}) -- 0x7f5664073960 con 0x7f56840ce430 2024-04-24T23:08:46.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.835+0000 7f5681d9b640 1 --2- 172.21.15.32:0/2895846526 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f566406be30 0x7f566406e2f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f5684156ce0 tx=0x7f5678008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:46.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:46.838+0000 7f5672ffd640 1 -- 172.21.15.32:0/2895846526 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f567c017020 con 0x7f56840ce430 2024-04-24T23:08:47.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.026+0000 7f5688eda640 1 -- 172.21.15.32:0/2895846526 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f568406c000 con 0x7f56840ce430 2024-04-24T23:08:47.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.027+0000 7f5672ffd640 1 -- 172.21.15.32:0/2895846526 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v158) ==== 104+0+11 (secure 0 0 0) 0x7f567c0594e0 con 0x7f56840ce430 2024-04-24T23:08:47.029 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-24T23:08:47.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.029+0000 7f5688eda640 1 -- 172.21.15.32:0/2895846526 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f566406be30 msgr2=0x7f566406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:47.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.029+0000 7f5688eda640 1 --2- 172.21.15.32:0/2895846526 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f566406be30 0x7f566406e2f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f5684156ce0 tx=0x7f5678008040 comp rx=0 tx=0).stop 2024-04-24T23:08:47.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.029+0000 7f5688eda640 1 -- 172.21.15.32:0/2895846526 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56840ce430 msgr2=0x7f5684141f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:47.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.029+0000 7f5688eda640 1 --2- 172.21.15.32:0/2895846526 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56840ce430 0x7f5684141f10 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f567c00e730 tx=0x7f567c00ebf0 comp rx=0 tx=0).stop 2024-04-24T23:08:47.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.029+0000 7f5688eda640 1 -- 172.21.15.32:0/2895846526 shutdown_connections 2024-04-24T23:08:47.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.029+0000 7f5688eda640 1 --2- 172.21.15.32:0/2895846526 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f566406be30 0x7f566406e2f0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:47.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.029+0000 7f5688eda640 1 --2- 172.21.15.32:0/2895846526 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f568415cb00 0x7f5684157560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:47.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.029+0000 7f5688eda640 1 --2- 172.21.15.32:0/2895846526 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f568415b380 0x7f5684071140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:47.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.029+0000 7f5688eda640 1 --2- 172.21.15.32:0/2895846526 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56840ce430 0x7f5684141f10 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:47.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.029+0000 7f5688eda640 1 -- 172.21.15.32:0/2895846526 >> 172.21.15.32:0/2895846526 conn(0x7f568406b110 msgr2=0x7f568406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:47.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.029+0000 7f5688eda640 1 -- 172.21.15.32:0/2895846526 shutdown_connections 2024-04-24T23:08:47.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.029+0000 7f5688eda640 1 -- 172.21.15.32:0/2895846526 wait complete. 2024-04-24T23:08:47.042 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:08:47.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.144+0000 7ff34d163640 1 Processor -- start 2024-04-24T23:08:47.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.144+0000 7ff34d163640 1 -- start start 2024-04-24T23:08:47.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.145+0000 7ff34d163640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff34806c480 0x7ff3480b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:47.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.145+0000 7ff34d163640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3480b0770 con 0x7ff348074210 2024-04-24T23:08:47.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.145+0000 7ff34d163640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3480b1010 con 0x7ff3480b0b80 2024-04-24T23:08:47.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.145+0000 7ff34d163640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7ff34806c8d0 con 0x7ff34806c480 2024-04-24T23:08:47.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.145+0000 7ff346d9d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff348074210 0x7ff3480745f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52338/0 (socket says 172.21.15.32:52338) 2024-04-24T23:08:47.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.145+0000 7ff346d9d640 1 -- 172.21.15.32:0/3584986979 learned_addr learned my addr 172.21.15.32:0/3584986979 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:08:47.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.145+0000 7ff34759e640 1 --2- 172.21.15.32:0/3584986979 >> v2:172.21.15.121:3300/0 conn(0x7ff34806c480 0x7ff3480b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:47.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.145+0000 7ff34659c640 1 -- 172.21.15.32:0/3584986979 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 704942113 0 0) 0x7ff3480b0770 con 0x7ff348074210 2024-04-24T23:08:47.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.145+0000 7ff34659c640 1 -- 172.21.15.32:0/3584986979 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff334003520 con 0x7ff348074210 2024-04-24T23:08:47.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.145+0000 7ff34659c640 1 -- 172.21.15.32:0/3584986979 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3518256014 0 0) 0x7ff3480b1010 con 0x7ff3480b0b80 2024-04-24T23:08:47.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.145+0000 7ff34659c640 1 -- 172.21.15.32:0/3584986979 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff3480b0770 con 0x7ff3480b0b80 2024-04-24T23:08:47.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.145+0000 7ff34659c640 1 -- 172.21.15.32:0/3584986979 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2805797695 0 0) 0x7ff334003520 con 0x7ff348074210 2024-04-24T23:08:47.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.145+0000 7ff34659c640 1 -- 172.21.15.32:0/3584986979 >> v2:172.21.15.121:3300/0 conn(0x7ff34806c480 msgr2=0x7ff3480b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:47.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.145+0000 7ff34659c640 1 --2- 172.21.15.32:0/3584986979 >> v2:172.21.15.121:3300/0 conn(0x7ff34806c480 0x7ff3480b20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:47.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.145+0000 7ff34659c640 1 -- 172.21.15.32:0/3584986979 >> v1:172.21.15.121:6789/0 conn(0x7ff3480b0b80 legacy=0x7ff3480ae140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:47.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.145+0000 7ff34659c640 1 -- 172.21.15.32:0/3584986979 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3480b12f0 con 0x7ff348074210 2024-04-24T23:08:47.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.145+0000 7ff34659c640 1 -- 172.21.15.32:0/3584986979 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7ff3380015f0 con 0x7ff348074210 2024-04-24T23:08:47.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.145+0000 7ff34659c640 1 -- 172.21.15.32:0/3584986979 >> v1:172.21.15.32:6789/0 conn(0x7ff348074210 legacy=0x7ff3480745f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:47.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.146+0000 7ff34659c640 1 --2- 172.21.15.32:0/3584986979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff334003b90 0x7ff334003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:47.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.146+0000 7ff34659c640 1 --2- 172.21.15.32:0/3584986979 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff334004800 0x7ff334004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:47.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.146+0000 7ff34659c640 1 --2- 172.21.15.32:0/3584986979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3340051c0 0x7ff33400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:47.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.146+0000 7ff34659c640 1 -- 172.21.15.32:0/3584986979 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff33400c090 con 0x7ff3340051c0 2024-04-24T23:08:47.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.146+0000 7ff34659c640 1 -- 172.21.15.32:0/3584986979 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff33400c210 con 0x7ff334003b90 2024-04-24T23:08:47.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.146+0000 7ff34659c640 1 -- 172.21.15.32:0/3584986979 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff33400c410 con 0x7ff334004800 2024-04-24T23:08:47.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.146+0000 7ff34759e640 1 --2- 172.21.15.32:0/3584986979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3340051c0 0x7ff33400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:47.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.146+0000 7ff346d9d640 1 --2- 172.21.15.32:0/3584986979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff334003b90 0x7ff334003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:47.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.146+0000 7ff33ffff640 1 --2- 172.21.15.32:0/3584986979 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff334004800 0x7ff334004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:47.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.146+0000 7ff34759e640 1 -- 172.21.15.32:0/3584986979 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff334004800 msgr2=0x7ff334004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:47.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.146+0000 7ff34759e640 1 --2- 172.21.15.32:0/3584986979 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff334004800 0x7ff334004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:47.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.146+0000 7ff34759e640 1 -- 172.21.15.32:0/3584986979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff334003b90 msgr2=0x7ff334003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:47.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.146+0000 7ff34759e640 1 --2- 172.21.15.32:0/3584986979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff334003b90 0x7ff334003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:47.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.146+0000 7ff34759e640 1 -- 172.21.15.32:0/3584986979 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff334003520 con 0x7ff3340051c0 2024-04-24T23:08:47.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.147+0000 7ff34759e640 1 -- 172.21.15.32:0/3584986979 reap_dead start 2024-04-24T23:08:47.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.147+0000 7ff34759e640 1 --2- 172.21.15.32:0/3584986979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3340051c0 0x7ff33400b9c0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7ff340009fc0 tx=0x7ff334005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89049ca3f77f8b7f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:47.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.147+0000 7ff34659c640 1 -- 172.21.15.32:0/3584986979 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff340035020 con 0x7ff3340051c0 2024-04-24T23:08:47.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.147+0000 7ff34659c640 1 -- 172.21.15.32:0/3584986979 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff340003bd0 con 0x7ff3340051c0 2024-04-24T23:08:47.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.147+0000 7ff34659c640 1 -- 172.21.15.32:0/3584986979 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff340009c10 con 0x7ff3340051c0 2024-04-24T23:08:47.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.147+0000 7ff34d163640 1 -- 172.21.15.32:0/3584986979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3340051c0 msgr2=0x7ff33400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:47.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.147+0000 7ff34d163640 1 --2- 172.21.15.32:0/3584986979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3340051c0 0x7ff33400b9c0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7ff340009fc0 tx=0x7ff334005830 comp rx=0 tx=0).stop 2024-04-24T23:08:47.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.147+0000 7ff34d163640 1 -- 172.21.15.32:0/3584986979 shutdown_connections 2024-04-24T23:08:47.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.147+0000 7ff34d163640 1 --2- 172.21.15.32:0/3584986979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3340051c0 0x7ff33400b9c0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:47.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.147+0000 7ff34d163640 1 -- 172.21.15.32:0/3584986979 >> 172.21.15.32:0/3584986979 conn(0x7ff3480b2e60 msgr2=0x7ff3480b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:47.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.147+0000 7ff34d163640 1 -- 172.21.15.32:0/3584986979 shutdown_connections 2024-04-24T23:08:47.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.147+0000 7ff34d163640 1 -- 172.21.15.32:0/3584986979 wait complete. 2024-04-24T23:08:47.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.149+0000 7ff34d163640 1 Processor -- start 2024-04-24T23:08:47.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.149+0000 7ff34d163640 1 -- start start 2024-04-24T23:08:47.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.149+0000 7ff34d163640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3340051c0 0x7ff3480c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:47.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.149+0000 7ff34d163640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff34815d830 0x7ff34809a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:47.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.149+0000 7ff34d163640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff34815eb70 0x7ff34815eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:47.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.149+0000 7ff34d163640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff3480b18b0 con 0x7ff3340051c0 2024-04-24T23:08:47.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.149+0000 7ff34d163640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff3480b1c20 con 0x7ff34815d830 2024-04-24T23:08:47.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.149+0000 7ff34d163640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff3480b3ba0 con 0x7ff34815eb70 2024-04-24T23:08:47.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.149+0000 7ff346d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3340051c0 0x7ff3480c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:47.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.150+0000 7ff346d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3340051c0 0x7ff3480c9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43732/0 (socket says 172.21.15.32:43732) 2024-04-24T23:08:47.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.150+0000 7ff346d9d640 1 -- 172.21.15.32:0/2716631144 learned_addr learned my addr 172.21.15.32:0/2716631144 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:47.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.150+0000 7ff34759e640 1 --2- 172.21.15.32:0/2716631144 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff34815eb70 0x7ff34815eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:47.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.150+0000 7ff33ffff640 1 --2- 172.21.15.32:0/2716631144 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff34815d830 0x7ff34809a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:47.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.150+0000 7ff346d9d640 1 -- 172.21.15.32:0/2716631144 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff34815eb70 msgr2=0x7ff34815eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:47.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.150+0000 7ff346d9d640 1 --2- 172.21.15.32:0/2716631144 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff34815eb70 0x7ff34815eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:47.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.150+0000 7ff346d9d640 1 -- 172.21.15.32:0/2716631144 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff34815d830 msgr2=0x7ff34809a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:47.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.150+0000 7ff346d9d640 1 --2- 172.21.15.32:0/2716631144 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff34815d830 0x7ff34809a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:47.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.150+0000 7ff346d9d640 1 -- 172.21.15.32:0/2716631144 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3480ca5c0 con 0x7ff3340051c0 2024-04-24T23:08:47.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.150+0000 7ff346d9d640 1 --2- 172.21.15.32:0/2716631144 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3340051c0 0x7ff3480c9e20 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7ff338004ec0 tx=0x7ff338007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:47.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.150+0000 7ff33f7fe640 1 -- 172.21.15.32:0/2716631144 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33801a070 con 0x7ff3340051c0 2024-04-24T23:08:47.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.150+0000 7ff33f7fe640 1 -- 172.21.15.32:0/2716631144 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff33800f7b0 con 0x7ff3340051c0 2024-04-24T23:08:47.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.150+0000 7ff34d163640 1 -- 172.21.15.32:0/2716631144 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3480ca8a0 con 0x7ff3340051c0 2024-04-24T23:08:47.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.150+0000 7ff33f7fe640 1 -- 172.21.15.32:0/2716631144 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33800fb80 con 0x7ff3340051c0 2024-04-24T23:08:47.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.150+0000 7ff34d163640 1 -- 172.21.15.32:0/2716631144 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3480cc2c0 con 0x7ff3340051c0 2024-04-24T23:08:47.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.151+0000 7ff34d163640 1 -- 172.21.15.32:0/2716631144 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3480cc4c0 con 0x7ff3340051c0 2024-04-24T23:08:47.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.152+0000 7ff33f7fe640 1 -- 172.21.15.32:0/2716631144 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff33801d070 con 0x7ff3340051c0 2024-04-24T23:08:47.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.152+0000 7ff33f7fe640 1 --2- 172.21.15.32:0/2716631144 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff32806be60 0x7ff32806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:47.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.152+0000 7ff33f7fe640 1 -- 172.21.15.32:0/2716631144 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7ff33808a6e0 con 0x7ff3340051c0 2024-04-24T23:08:47.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.152+0000 7ff33f7fe640 1 -- 172.21.15.32:0/2716631144 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=159}) -- 0x7ff3280739f0 con 0x7ff3340051c0 2024-04-24T23:08:47.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.152+0000 7ff33ffff640 1 --2- 172.21.15.32:0/2716631144 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff32806be60 0x7ff32806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:47.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.153+0000 7ff33ffff640 1 --2- 172.21.15.32:0/2716631144 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff32806be60 0x7ff32806e320 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7ff3480b01a0 tx=0x7ff330009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:47.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.156+0000 7ff33f7fe640 1 -- 172.21.15.32:0/2716631144 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff3480cc4c0 con 0x7ff3340051c0 2024-04-24T23:08:47.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.339+0000 7ff34d163640 1 -- 172.21.15.32:0/2716631144 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff348068430 con 0x7ff3340051c0 2024-04-24T23:08:47.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.340+0000 7ff33f7fe640 1 -- 172.21.15.32:0/2716631144 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v158) ==== 100+0+11 (secure 0 0 0) 0x7ff338059120 con 0x7ff3340051c0 2024-04-24T23:08:47.341 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-24T23:08:47.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.341+0000 7ff34d163640 1 -- 172.21.15.32:0/2716631144 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff32806be60 msgr2=0x7ff32806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:47.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.341+0000 7ff34d163640 1 --2- 172.21.15.32:0/2716631144 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff32806be60 0x7ff32806e320 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7ff3480b01a0 tx=0x7ff330009040 comp rx=0 tx=0).stop 2024-04-24T23:08:47.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.341+0000 7ff34d163640 1 -- 172.21.15.32:0/2716631144 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3340051c0 msgr2=0x7ff3480c9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:47.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.341+0000 7ff34d163640 1 --2- 172.21.15.32:0/2716631144 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3340051c0 0x7ff3480c9e20 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7ff338004ec0 tx=0x7ff338007430 comp rx=0 tx=0).stop 2024-04-24T23:08:47.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.342+0000 7ff34d163640 1 -- 172.21.15.32:0/2716631144 shutdown_connections 2024-04-24T23:08:47.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.342+0000 7ff34d163640 1 --2- 172.21.15.32:0/2716631144 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff32806be60 0x7ff32806e320 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:47.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.342+0000 7ff34d163640 1 --2- 172.21.15.32:0/2716631144 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff34815eb70 0x7ff34815eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:47.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.342+0000 7ff34d163640 1 --2- 172.21.15.32:0/2716631144 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff34815d830 0x7ff34809a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:47.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.342+0000 7ff34d163640 1 --2- 172.21.15.32:0/2716631144 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3340051c0 0x7ff3480c9e20 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:47.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.342+0000 7ff34d163640 1 -- 172.21.15.32:0/2716631144 >> 172.21.15.32:0/2716631144 conn(0x7ff3480b2e60 msgr2=0x7ff3480b20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:47.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.342+0000 7ff34d163640 1 -- 172.21.15.32:0/2716631144 shutdown_connections 2024-04-24T23:08:47.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.342+0000 7ff34d163640 1 -- 172.21.15.32:0/2716631144 wait complete. 2024-04-24T23:08:47.354 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:08:47.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.453+0000 7f659c0a5640 1 Processor -- start 2024-04-24T23:08:47.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.454+0000 7f659c0a5640 1 -- start start 2024-04-24T23:08:47.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.454+0000 7f659c0a5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f659406b820 0x7f65940d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:47.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.454+0000 7f659c0a5640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f65940d4210 0x7f65940b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:47.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.454+0000 7f659c0a5640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65940d3170 con 0x7f659415e9b0 2024-04-24T23:08:47.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.454+0000 7f659c0a5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f65940d3450 con 0x7f659406b820 2024-04-24T23:08:47.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.454+0000 7f659c0a5640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f65940d3730 con 0x7f65940d4210 2024-04-24T23:08:47.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.454+0000 7f6599e41640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f659415e9b0 0x7f659406b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52340/0 (socket says 172.21.15.32:52340) 2024-04-24T23:08:47.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.454+0000 7f6599e41640 1 -- 172.21.15.32:0/4134237966 learned_addr learned my addr 172.21.15.32:0/4134237966 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:08:47.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.454+0000 7f6599640640 1 --2- 172.21.15.32:0/4134237966 >> v2:172.21.15.32:3300/0 conn(0x7f659406b820 0x7f65940d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:47.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.455+0000 7f659a642640 1 --2- 172.21.15.32:0/4134237966 >> v2:172.21.15.121:3300/0 conn(0x7f65940d4210 0x7f65940b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:47.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.455+0000 7f6599640640 1 -- 172.21.15.32:0/4134237966 >> v2:172.21.15.121:3300/0 conn(0x7f65940d4210 msgr2=0x7f65940b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:47.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.455+0000 7f6599640640 1 --2- 172.21.15.32:0/4134237966 >> v2:172.21.15.121:3300/0 conn(0x7f65940d4210 0x7f65940b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:47.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.455+0000 7f6599640640 1 -- 172.21.15.32:0/4134237966 >> v1:172.21.15.32:6789/0 conn(0x7f659415e9b0 legacy=0x7f659406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:47.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.455+0000 7f6599640640 1 -- 172.21.15.32:0/4134237966 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65940d3a10 con 0x7f659406b820 2024-04-24T23:08:47.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.455+0000 7f6598e3f640 1 -- 172.21.15.32:0/4134237966 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 994868215 0 0) 0x7f65940d3170 con 0x7f659415e9b0 2024-04-24T23:08:47.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.455+0000 7f659a642640 1 --2- 172.21.15.32:0/4134237966 >> v2:172.21.15.121:3300/0 conn(0x7f65940d4210 0x7f65940b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:08:47.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.455+0000 7f6599640640 1 --2- 172.21.15.32:0/4134237966 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f659406b820 0x7f65940d2b90 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f65941622f0 tx=0x7f658802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ac35805d283f7d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:47.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.456+0000 7f6598e3f640 1 -- 172.21.15.32:0/4134237966 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6588037070 con 0x7f659406b820 2024-04-24T23:08:47.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.456+0000 7f6598e3f640 1 -- 172.21.15.32:0/4134237966 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6588032440 con 0x7f659406b820 2024-04-24T23:08:47.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.456+0000 7f6598e3f640 1 -- 172.21.15.32:0/4134237966 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f658802ae60 con 0x7f659406b820 2024-04-24T23:08:47.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.456+0000 7f659c0a5640 1 -- 172.21.15.32:0/4134237966 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f659406b820 msgr2=0x7f65940d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:47.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.456+0000 7f659c0a5640 1 --2- 172.21.15.32:0/4134237966 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f659406b820 0x7f65940d2b90 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f65941622f0 tx=0x7f658802a110 comp rx=0 tx=0).stop 2024-04-24T23:08:47.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.456+0000 7f659c0a5640 1 -- 172.21.15.32:0/4134237966 shutdown_connections 2024-04-24T23:08:47.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.456+0000 7f659c0a5640 1 --2- 172.21.15.32:0/4134237966 >> v2:172.21.15.121:3300/0 conn(0x7f65940d4210 0x7f65940b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:47.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.456+0000 7f659c0a5640 1 --2- 172.21.15.32:0/4134237966 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f659406b820 0x7f65940d2b90 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:47.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.456+0000 7f659c0a5640 1 -- 172.21.15.32:0/4134237966 >> 172.21.15.32:0/4134237966 conn(0x7f65940d11d0 msgr2=0x7f65940d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:47.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.456+0000 7f659c0a5640 1 -- 172.21.15.32:0/4134237966 shutdown_connections 2024-04-24T23:08:47.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.456+0000 7f659c0a5640 1 -- 172.21.15.32:0/4134237966 wait complete. 2024-04-24T23:08:47.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.457+0000 7f659c0a5640 1 Processor -- start 2024-04-24T23:08:47.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.457+0000 7f659c0a5640 1 -- start start 2024-04-24T23:08:47.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.458+0000 7f659c0a5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f659406b820 0x7f659414e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:47.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.458+0000 7f659c0a5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65940d4210 0x7f659415ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:47.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.458+0000 7f659c0a5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f659415e9b0 0x7f65941605b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:47.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.458+0000 7f659c0a5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f65940ba0e0 con 0x7f65940d4210 2024-04-24T23:08:47.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.458+0000 7f659c0a5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f65940d2400 con 0x7f659406b820 2024-04-24T23:08:47.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.458+0000 7f659c0a5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f65940ba260 con 0x7f659415e9b0 2024-04-24T23:08:47.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.458+0000 7f6599640640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65940d4210 0x7f659415ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:47.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.458+0000 7f6599640640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65940d4210 0x7f659415ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43752/0 (socket says 172.21.15.32:43752) 2024-04-24T23:08:47.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.458+0000 7f6599640640 1 -- 172.21.15.32:0/2712497257 learned_addr learned my addr 172.21.15.32:0/2712497257 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:47.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.458+0000 7f6599e41640 1 --2- 172.21.15.32:0/2712497257 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f659406b820 0x7f659414e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:47.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.458+0000 7f659a642640 1 --2- 172.21.15.32:0/2712497257 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f659415e9b0 0x7f65941605b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:47.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.459+0000 7f6599640640 1 -- 172.21.15.32:0/2712497257 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f659415e9b0 msgr2=0x7f65941605b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:47.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.459+0000 7f6599640640 1 --2- 172.21.15.32:0/2712497257 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f659415e9b0 0x7f65941605b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:47.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.459+0000 7f6599640640 1 -- 172.21.15.32:0/2712497257 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f659406b820 msgr2=0x7f659414e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:47.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.459+0000 7f6599640640 1 --2- 172.21.15.32:0/2712497257 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f659406b820 0x7f659414e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:47.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.459+0000 7f6599640640 1 -- 172.21.15.32:0/2712497257 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f659415f770 con 0x7f65940d4210 2024-04-24T23:08:47.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.459+0000 7f6599640640 1 --2- 172.21.15.32:0/2712497257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65940d4210 0x7f659415ed90 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f6588009890 tx=0x7f65880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:47.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.459+0000 7f65927fc640 1 -- 172.21.15.32:0/2712497257 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6588037050 con 0x7f65940d4210 2024-04-24T23:08:47.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.459+0000 7f65927fc640 1 -- 172.21.15.32:0/2712497257 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6588038040 con 0x7f65940d4210 2024-04-24T23:08:47.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.459+0000 7f65927fc640 1 -- 172.21.15.32:0/2712497257 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6588004470 con 0x7f65940d4210 2024-04-24T23:08:47.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.459+0000 7f659c0a5640 1 -- 172.21.15.32:0/2712497257 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f659415fa50 con 0x7f65940d4210 2024-04-24T23:08:47.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.459+0000 7f659c0a5640 1 -- 172.21.15.32:0/2712497257 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6594160010 con 0x7f65940d4210 2024-04-24T23:08:47.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.460+0000 7f659c0a5640 1 -- 172.21.15.32:0/2712497257 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6594068430 con 0x7f65940d4210 2024-04-24T23:08:47.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.460+0000 7f65927fc640 1 -- 172.21.15.32:0/2712497257 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6588030900 con 0x7f65940d4210 2024-04-24T23:08:47.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.461+0000 7f65927fc640 1 --2- 172.21.15.32:0/2712497257 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f658006be30 0x7f658006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:47.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.461+0000 7f65927fc640 1 -- 172.21.15.32:0/2712497257 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7f65880ab9c0 con 0x7f65940d4210 2024-04-24T23:08:47.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.461+0000 7f6599e41640 1 --2- 172.21.15.32:0/2712497257 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f658006be30 0x7f658006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:47.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.461+0000 7f65927fc640 1 -- 172.21.15.32:0/2712497257 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=159}) -- 0x7f6580073960 con 0x7f65940d4210 2024-04-24T23:08:47.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.462+0000 7f6599e41640 1 --2- 172.21.15.32:0/2712497257 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f658006be30 0x7f658006e2f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f658c000f60 tx=0x7f658c007c00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:47.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.465+0000 7f65927fc640 1 -- 172.21.15.32:0/2712497257 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f658807a480 con 0x7f65940d4210 2024-04-24T23:08:47.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.686+0000 7f659c0a5640 1 -- 172.21.15.32:0/2712497257 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f65940728d0 con 0x7f65940d4210 2024-04-24T23:08:47.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.687+0000 7f65927fc640 1 -- 172.21.15.32:0/2712497257 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+5186 (secure 0 0 0) 0x7f6588079d80 con 0x7f65940d4210 2024-04-24T23:08:47.689 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:08:47.689 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":65,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7120,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/1196305174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1196305174},{"type":"v1","addr":"172.21.15.121:6836","nonce":1196305174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7123,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3015902884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3015902884},{"type":"v1","addr":"172.21.15.121:6837","nonce":3015902884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":8280,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/1033273628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1033273628},{"type":"v1","addr":"172.21.15.32:6833","nonce":1033273628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:06:31.291599+0000","modified":"2024-04-24T23:06:33.296657+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8286},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8286":{"gid":8286,"name":"a","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/984603360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":984603360},{"type":"v1","addr":"172.21.15.32:6835","nonce":984603360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8286,"qdb_cluster":[8286]},"id":9}]} 2024-04-24T23:08:47.690 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 65 2024-04-24T23:08:47.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.689+0000 7f659c0a5640 1 -- 172.21.15.32:0/2712497257 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f658006be30 msgr2=0x7f658006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:47.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.689+0000 7f659c0a5640 1 --2- 172.21.15.32:0/2712497257 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f658006be30 0x7f658006e2f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f658c000f60 tx=0x7f658c007c00 comp rx=0 tx=0).stop 2024-04-24T23:08:47.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.689+0000 7f659c0a5640 1 -- 172.21.15.32:0/2712497257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65940d4210 msgr2=0x7f659415ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:47.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.689+0000 7f659c0a5640 1 --2- 172.21.15.32:0/2712497257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65940d4210 0x7f659415ed90 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f6588009890 tx=0x7f65880046c0 comp rx=0 tx=0).stop 2024-04-24T23:08:47.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.689+0000 7f659c0a5640 1 -- 172.21.15.32:0/2712497257 shutdown_connections 2024-04-24T23:08:47.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.689+0000 7f659c0a5640 1 --2- 172.21.15.32:0/2712497257 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f658006be30 0x7f658006e2f0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:47.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.689+0000 7f659c0a5640 1 --2- 172.21.15.32:0/2712497257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65940d4210 0x7f659415ed90 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:47.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.689+0000 7f659c0a5640 1 --2- 172.21.15.32:0/2712497257 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f659415e9b0 0x7f65941605b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:47.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.689+0000 7f659c0a5640 1 --2- 172.21.15.32:0/2712497257 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f659406b820 0x7f659414e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:47.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.689+0000 7f659c0a5640 1 -- 172.21.15.32:0/2712497257 >> 172.21.15.32:0/2712497257 conn(0x7f65940d11d0 msgr2=0x7f65940b7b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:47.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.689+0000 7f659c0a5640 1 -- 172.21.15.32:0/2712497257 shutdown_connections 2024-04-24T23:08:47.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.689+0000 7f659c0a5640 1 -- 172.21.15.32:0/2712497257 wait complete. 2024-04-24T23:08:47.702 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:08:47.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.801+0000 7f2fc8acd640 1 Processor -- start 2024-04-24T23:08:47.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.801+0000 7f2fc8acd640 1 -- start start 2024-04-24T23:08:47.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.802+0000 7f2fc8acd640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2fc415e9b0 0x7f2fc40d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:47.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.802+0000 7f2fc8acd640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fc40d3170 con 0x7f2fc40d4210 2024-04-24T23:08:47.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.802+0000 7f2fc8acd640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fc40d3450 con 0x7f2fc406b820 2024-04-24T23:08:47.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.802+0000 7f2fc8acd640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f2fc40d3730 con 0x7f2fc415e9b0 2024-04-24T23:08:47.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.802+0000 7f2fc1d9b640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2fc40d4210 0x7f2fc406b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52350/0 (socket says 172.21.15.32:52350) 2024-04-24T23:08:47.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.802+0000 7f2fc1d9b640 1 -- 172.21.15.32:0/1123823598 learned_addr learned my addr 172.21.15.32:0/1123823598 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:08:47.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.802+0000 7f2fc259c640 1 --2- 172.21.15.32:0/1123823598 >> v2:172.21.15.121:3300/0 conn(0x7f2fc415e9b0 0x7f2fc40d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:47.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.803+0000 7f2fc159a640 1 -- 172.21.15.32:0/1123823598 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1255341841 0 0) 0x7f2fc40d3170 con 0x7f2fc40d4210 2024-04-24T23:08:47.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.803+0000 7f2fc159a640 1 -- 172.21.15.32:0/1123823598 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2fac003520 con 0x7f2fc40d4210 2024-04-24T23:08:47.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.803+0000 7f2fc259c640 1 -- 172.21.15.32:0/1123823598 >> v1:172.21.15.121:6789/0 conn(0x7f2fc406b820 legacy=0x7f2fc40b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:47.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.803+0000 7f2fc259c640 1 -- 172.21.15.32:0/1123823598 >> v1:172.21.15.32:6789/0 conn(0x7f2fc40d4210 legacy=0x7f2fc406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:47.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.803+0000 7f2fc259c640 1 -- 172.21.15.32:0/1123823598 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fc40d3a10 con 0x7f2fc415e9b0 2024-04-24T23:08:47.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.804+0000 7f2fc259c640 1 --2- 172.21.15.32:0/1123823598 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2fc415e9b0 0x7f2fc40d2b90 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc0097a0 tx=0x7f2fbc02a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cf8ddc1396c8afe server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:47.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.804+0000 7f2fc159a640 1 -- 172.21.15.32:0/1123823598 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc037070 con 0x7f2fc415e9b0 2024-04-24T23:08:47.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.804+0000 7f2fc159a640 1 -- 172.21.15.32:0/1123823598 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fbc032440 con 0x7f2fc415e9b0 2024-04-24T23:08:47.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.804+0000 7f2fc159a640 1 -- 172.21.15.32:0/1123823598 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc0327f0 con 0x7f2fc415e9b0 2024-04-24T23:08:47.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.804+0000 7f2fc8acd640 1 -- 172.21.15.32:0/1123823598 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2fc415e9b0 msgr2=0x7f2fc40d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:47.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.804+0000 7f2fc8acd640 1 --2- 172.21.15.32:0/1123823598 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2fc415e9b0 0x7f2fc40d2b90 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc0097a0 tx=0x7f2fbc02a1a0 comp rx=0 tx=0).stop 2024-04-24T23:08:47.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.804+0000 7f2fc8acd640 1 -- 172.21.15.32:0/1123823598 shutdown_connections 2024-04-24T23:08:47.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.804+0000 7f2fc8acd640 1 --2- 172.21.15.32:0/1123823598 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2fc415e9b0 0x7f2fc40d2b90 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:47.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.804+0000 7f2fc8acd640 1 -- 172.21.15.32:0/1123823598 >> 172.21.15.32:0/1123823598 conn(0x7f2fc40d11d0 msgr2=0x7f2fc40d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:47.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.804+0000 7f2fc8acd640 1 -- 172.21.15.32:0/1123823598 shutdown_connections 2024-04-24T23:08:47.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.804+0000 7f2fc8acd640 1 -- 172.21.15.32:0/1123823598 wait complete. 2024-04-24T23:08:47.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.806+0000 7f2fc8acd640 1 Processor -- start 2024-04-24T23:08:47.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.806+0000 7f2fc8acd640 1 -- start start 2024-04-24T23:08:47.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.806+0000 7f2fc8acd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fc406b820 0x7f2fc414d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:47.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.807+0000 7f2fc8acd640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2fc40d4210 0x7f2fc4071730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:47.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.807+0000 7f2fc8acd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2fc415e9b0 0x7f2fc4071c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:47.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.807+0000 7f2fc8acd640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc40ba0e0 con 0x7f2fc406b820 2024-04-24T23:08:47.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.807+0000 7f2fc8acd640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc40d2400 con 0x7f2fc415e9b0 2024-04-24T23:08:47.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.807+0000 7f2fc8acd640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f2fc40ba260 con 0x7f2fc40d4210 2024-04-24T23:08:47.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.807+0000 7f2fc259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fc406b820 0x7f2fc414d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:47.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.807+0000 7f2fc259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fc406b820 0x7f2fc414d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43768/0 (socket says 172.21.15.32:43768) 2024-04-24T23:08:47.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.807+0000 7f2fc259c640 1 -- 172.21.15.32:0/3392014314 learned_addr learned my addr 172.21.15.32:0/3392014314 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:47.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.807+0000 7f2fc2d9d640 1 --2- 172.21.15.32:0/3392014314 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2fc415e9b0 0x7f2fc4071c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:47.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.807+0000 7f2fc1d9b640 1 --2- 172.21.15.32:0/3392014314 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2fc40d4210 0x7f2fc4071730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:47.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.807+0000 7f2fc259c640 1 -- 172.21.15.32:0/3392014314 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2fc40d4210 msgr2=0x7f2fc4071730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:47.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.807+0000 7f2fc259c640 1 --2- 172.21.15.32:0/3392014314 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2fc40d4210 0x7f2fc4071730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:47.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.807+0000 7f2fc259c640 1 -- 172.21.15.32:0/3392014314 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2fc415e9b0 msgr2=0x7f2fc4071c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:47.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.807+0000 7f2fc259c640 1 --2- 172.21.15.32:0/3392014314 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2fc415e9b0 0x7f2fc4071c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:47.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.807+0000 7f2fc259c640 1 -- 172.21.15.32:0/3392014314 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fc415a620 con 0x7f2fc406b820 2024-04-24T23:08:47.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.807+0000 7f2fc2d9d640 1 --2- 172.21.15.32:0/3392014314 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2fc415e9b0 0x7f2fc4071c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:08:47.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.807+0000 7f2fc1d9b640 1 --2- 172.21.15.32:0/3392014314 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2fc40d4210 0x7f2fc4071730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:08:47.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.807+0000 7f2fc259c640 1 --2- 172.21.15.32:0/3392014314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fc406b820 0x7f2fc414d950 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc0097a0 tx=0x7f2fbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:47.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.807+0000 7f2fb2ffd640 1 -- 172.21.15.32:0/3392014314 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc037050 con 0x7f2fc406b820 2024-04-24T23:08:47.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.807+0000 7f2fb2ffd640 1 -- 172.21.15.32:0/3392014314 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fbc038040 con 0x7f2fc406b820 2024-04-24T23:08:47.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.807+0000 7f2fc8acd640 1 -- 172.21.15.32:0/3392014314 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fc415a900 con 0x7f2fc406b820 2024-04-24T23:08:47.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.807+0000 7f2fb2ffd640 1 -- 172.21.15.32:0/3392014314 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc004470 con 0x7f2fc406b820 2024-04-24T23:08:47.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.807+0000 7f2fc8acd640 1 -- 172.21.15.32:0/3392014314 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fc415aec0 con 0x7f2fc406b820 2024-04-24T23:08:47.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.809+0000 7f2fc8acd640 1 -- 172.21.15.32:0/3392014314 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fc4068430 con 0x7f2fc406b820 2024-04-24T23:08:47.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.809+0000 7f2fb2ffd640 1 -- 172.21.15.32:0/3392014314 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2fbc03c6d0 con 0x7f2fc406b820 2024-04-24T23:08:47.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.809+0000 7f2fb2ffd640 1 --2- 172.21.15.32:0/3392014314 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2fa406be60 0x7f2fa406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:47.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.809+0000 7f2fb2ffd640 1 -- 172.21.15.32:0/3392014314 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7f2fbc0ab8a0 con 0x7f2fc406b820 2024-04-24T23:08:47.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.810+0000 7f2fb2ffd640 1 -- 172.21.15.32:0/3392014314 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=159}) -- 0x7f2fa4073990 con 0x7f2fc406b820 2024-04-24T23:08:47.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.810+0000 7f2fc1d9b640 1 --2- 172.21.15.32:0/3392014314 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2fa406be60 0x7f2fa406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:47.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.810+0000 7f2fc1d9b640 1 --2- 172.21.15.32:0/3392014314 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2fa406be60 0x7f2fa406e320 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4154690 tx=0x7f2fb8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:47.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:47.814+0000 7f2fb2ffd640 1 -- 172.21.15.32:0/3392014314 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2fbc07a250 con 0x7f2fc406b820 2024-04-24T23:08:48.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.000+0000 7f2fc8acd640 1 -- 172.21.15.32:0/3392014314 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2fc4072540 con 0x7f2fc406b820 2024-04-24T23:08:48.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.001+0000 7f2fb2ffd640 1 -- 172.21.15.32:0/3392014314 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v158) ==== 74+0+16666 (secure 0 0 0) 0x7f2fbc079b50 con 0x7f2fc406b820 2024-04-24T23:08:48.003 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:08:48.003 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":158,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:08:06.831677+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:06:29.591736+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-24T23:06:30.267629+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:08:48.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.003+0000 7f2fc8acd640 1 -- 172.21.15.32:0/3392014314 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2fa406be60 msgr2=0x7f2fa406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:48.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.003+0000 7f2fc8acd640 1 --2- 172.21.15.32:0/3392014314 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2fa406be60 0x7f2fa406e320 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4154690 tx=0x7f2fb8009040 comp rx=0 tx=0).stop 2024-04-24T23:08:48.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.003+0000 7f2fc8acd640 1 -- 172.21.15.32:0/3392014314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fc406b820 msgr2=0x7f2fc414d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:48.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.003+0000 7f2fc8acd640 1 --2- 172.21.15.32:0/3392014314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fc406b820 0x7f2fc414d950 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc0097a0 tx=0x7f2fbc0046c0 comp rx=0 tx=0).stop 2024-04-24T23:08:48.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.003+0000 7f2fc8acd640 1 -- 172.21.15.32:0/3392014314 shutdown_connections 2024-04-24T23:08:48.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.003+0000 7f2fc8acd640 1 --2- 172.21.15.32:0/3392014314 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2fa406be60 0x7f2fa406e320 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:48.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.003+0000 7f2fc8acd640 1 --2- 172.21.15.32:0/3392014314 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2fc415e9b0 0x7f2fc4071c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:48.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.003+0000 7f2fc8acd640 1 --2- 172.21.15.32:0/3392014314 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2fc40d4210 0x7f2fc4071730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:48.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.003+0000 7f2fc8acd640 1 --2- 172.21.15.32:0/3392014314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fc406b820 0x7f2fc414d950 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:48.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.003+0000 7f2fc8acd640 1 -- 172.21.15.32:0/3392014314 >> 172.21.15.32:0/3392014314 conn(0x7f2fc40d11d0 msgr2=0x7f2fc40b78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:48.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.003+0000 7f2fc8acd640 1 -- 172.21.15.32:0/3392014314 shutdown_connections 2024-04-24T23:08:48.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.003+0000 7f2fc8acd640 1 -- 172.21.15.32:0/3392014314 wait complete. 2024-04-24T23:08:48.016 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-24T23:08:48.016 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:08:48.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.115+0000 7f32be9a9640 1 Processor -- start 2024-04-24T23:08:48.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.115+0000 7f32be9a9640 1 -- start start 2024-04-24T23:08:48.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.116+0000 7f32be9a9640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f32b815e9b0 0x7f32b80d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:48.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.116+0000 7f32be9a9640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f32b806b820 0x7f32b80b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:48.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.116+0000 7f32be9a9640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32b80d3170 con 0x7f32b80d4210 2024-04-24T23:08:48.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.116+0000 7f32be9a9640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f32b80d3450 con 0x7f32b806b820 2024-04-24T23:08:48.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.116+0000 7f32be9a9640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f32b80d3730 con 0x7f32b815e9b0 2024-04-24T23:08:48.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.116+0000 7f32b77fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f32b80d4210 0x7f32b806b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52366/0 (socket says 172.21.15.32:52366) 2024-04-24T23:08:48.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.116+0000 7f32b77fe640 1 -- 172.21.15.32:0/847149841 learned_addr learned my addr 172.21.15.32:0/847149841 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:08:48.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.117+0000 7f32b6ffd640 1 -- 172.21.15.32:0/847149841 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4235081258 0 0) 0x7f32b80d3170 con 0x7f32b80d4210 2024-04-24T23:08:48.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.117+0000 7f32bcf46640 1 --2- 172.21.15.32:0/847149841 >> v2:172.21.15.121:3300/0 conn(0x7f32b806b820 0x7f32b80b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:48.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.117+0000 7f32b7fff640 1 --2- 172.21.15.32:0/847149841 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f32b815e9b0 0x7f32b80d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:48.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.117+0000 7f32b6ffd640 1 -- 172.21.15.32:0/847149841 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32a0003520 con 0x7f32b80d4210 2024-04-24T23:08:48.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.117+0000 7f32b6ffd640 1 -- 172.21.15.32:0/847149841 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4088943424 0 0) 0x7f32a0003520 con 0x7f32b80d4210 2024-04-24T23:08:48.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.117+0000 7f32b6ffd640 1 -- 172.21.15.32:0/847149841 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f32b815e9b0 msgr2=0x7f32b80d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:48.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.117+0000 7f32b6ffd640 1 --2- 172.21.15.32:0/847149841 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f32b815e9b0 0x7f32b80d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:48.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.117+0000 7f32b6ffd640 1 -- 172.21.15.32:0/847149841 >> v2:172.21.15.121:3300/0 conn(0x7f32b806b820 msgr2=0x7f32b80b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:48.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.117+0000 7f32b6ffd640 1 --2- 172.21.15.32:0/847149841 >> v2:172.21.15.121:3300/0 conn(0x7f32b806b820 0x7f32b80b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:48.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.117+0000 7f32b6ffd640 1 -- 172.21.15.32:0/847149841 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b80d3a10 con 0x7f32b80d4210 2024-04-24T23:08:48.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.117+0000 7f32b6ffd640 1 -- 172.21.15.32:0/847149841 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f32ac001670 con 0x7f32b80d4210 2024-04-24T23:08:48.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.117+0000 7f32b6ffd640 1 -- 172.21.15.32:0/847149841 >> v1:172.21.15.32:6789/0 conn(0x7f32b80d4210 legacy=0x7f32b806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:48.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.117+0000 7f32b6ffd640 1 --2- 172.21.15.32:0/847149841 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f32b815e9b0 0x7f32a00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:48.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.117+0000 7f32b6ffd640 1 --2- 172.21.15.32:0/847149841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32a0003f30 0x7f32a0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:48.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.117+0000 7f32b6ffd640 1 --2- 172.21.15.32:0/847149841 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32a0004d40 0x7f32a00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:48.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.117+0000 7f32b6ffd640 1 -- 172.21.15.32:0/847149841 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f32b80d3450 con 0x7f32a0003f30 2024-04-24T23:08:48.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.117+0000 7f32b6ffd640 1 -- 172.21.15.32:0/847149841 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f32b80d3730 con 0x7f32a0004d40 2024-04-24T23:08:48.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.117+0000 7f32b6ffd640 1 -- 172.21.15.32:0/847149841 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f32a000b9c0 con 0x7f32b815e9b0 2024-04-24T23:08:48.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.117+0000 7f32b77fe640 1 --2- 172.21.15.32:0/847149841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32a0003f30 0x7f32a0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:48.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.117+0000 7f32b7fff640 1 --2- 172.21.15.32:0/847149841 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f32b815e9b0 0x7f32a00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:48.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.117+0000 7f32bcf46640 1 --2- 172.21.15.32:0/847149841 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32a0004d40 0x7f32a00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:48.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.117+0000 7f32b77fe640 1 -- 172.21.15.32:0/847149841 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f32b815e9b0 msgr2=0x7f32a00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:48.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.117+0000 7f32b77fe640 1 --2- 172.21.15.32:0/847149841 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f32b815e9b0 0x7f32a00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:48.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.118+0000 7f32b77fe640 1 -- 172.21.15.32:0/847149841 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32a0004d40 msgr2=0x7f32a00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:48.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.118+0000 7f32b77fe640 1 --2- 172.21.15.32:0/847149841 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32a0004d40 0x7f32a00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:48.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.118+0000 7f32b77fe640 1 -- 172.21.15.32:0/847149841 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32a0003520 con 0x7f32a0003f30 2024-04-24T23:08:48.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.118+0000 7f32bcf46640 1 --2- 172.21.15.32:0/847149841 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32a0004d40 0x7f32a00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:08:48.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.118+0000 7f32b77fe640 1 --2- 172.21.15.32:0/847149841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32a0003f30 0x7f32a0004800 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f32ac00c240 tx=0x7f32a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9b74990b9a3e6ba server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:48.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.118+0000 7f32b6ffd640 1 -- 172.21.15.32:0/847149841 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ac007d30 con 0x7f32a0003f30 2024-04-24T23:08:48.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.118+0000 7f32b6ffd640 1 -- 172.21.15.32:0/847149841 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32ac005ad0 con 0x7f32a0003f30 2024-04-24T23:08:48.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.118+0000 7f32b6ffd640 1 -- 172.21.15.32:0/847149841 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ac007070 con 0x7f32a0003f30 2024-04-24T23:08:48.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.119+0000 7f32be9a9640 1 -- 172.21.15.32:0/847149841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32a0003f30 msgr2=0x7f32a0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:48.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.119+0000 7f32be9a9640 1 --2- 172.21.15.32:0/847149841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32a0003f30 0x7f32a0004800 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f32ac00c240 tx=0x7f32a0005830 comp rx=0 tx=0).stop 2024-04-24T23:08:48.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.119+0000 7f32bcf46640 1 -- 172.21.15.32:0/847149841 reap_dead start 2024-04-24T23:08:48.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.119+0000 7f32be9a9640 1 -- 172.21.15.32:0/847149841 shutdown_connections 2024-04-24T23:08:48.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.119+0000 7f32be9a9640 1 -- 172.21.15.32:0/847149841 >> 172.21.15.32:0/847149841 conn(0x7f32b80d11d0 msgr2=0x7f32b80d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:48.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.119+0000 7f32be9a9640 1 -- 172.21.15.32:0/847149841 shutdown_connections 2024-04-24T23:08:48.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.119+0000 7f32be9a9640 1 -- 172.21.15.32:0/847149841 wait complete. 2024-04-24T23:08:48.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.120+0000 7f32be9a9640 1 Processor -- start 2024-04-24T23:08:48.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.120+0000 7f32be9a9640 1 -- start start 2024-04-24T23:08:48.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.121+0000 7f32be9a9640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f32a0003f30 0x7f32b8157dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:48.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.121+0000 7f32be9a9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32b815f7a0 0x7f32b80ca220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:48.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.121+0000 7f32be9a9640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32b8160d60 0x7f32b81611e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:48.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.121+0000 7f32be9a9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f32b80ba0e0 con 0x7f32b815f7a0 2024-04-24T23:08:48.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.121+0000 7f32be9a9640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f32b80d2400 con 0x7f32b8160d60 2024-04-24T23:08:48.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.121+0000 7f32be9a9640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f32b80ba260 con 0x7f32a0003f30 2024-04-24T23:08:48.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.121+0000 7f32b77fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32b815f7a0 0x7f32b80ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:48.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.121+0000 7f32b77fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32b815f7a0 0x7f32b80ca220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43778/0 (socket says 172.21.15.32:43778) 2024-04-24T23:08:48.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.121+0000 7f32b77fe640 1 -- 172.21.15.32:0/1014084403 learned_addr learned my addr 172.21.15.32:0/1014084403 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:48.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.121+0000 7f32bcf46640 1 --2- 172.21.15.32:0/1014084403 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32b8160d60 0x7f32b81611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:48.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.121+0000 7f32b7fff640 1 --2- 172.21.15.32:0/1014084403 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f32a0003f30 0x7f32b8157dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:48.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.121+0000 7f32b77fe640 1 -- 172.21.15.32:0/1014084403 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f32a0003f30 msgr2=0x7f32b8157dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:48.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.121+0000 7f32b77fe640 1 --2- 172.21.15.32:0/1014084403 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f32a0003f30 0x7f32b8157dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:48.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.122+0000 7f32b77fe640 1 -- 172.21.15.32:0/1014084403 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32b8160d60 msgr2=0x7f32b81611e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:48.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.122+0000 7f32b77fe640 1 --2- 172.21.15.32:0/1014084403 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32b8160d60 0x7f32b81611e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:48.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.122+0000 7f32b77fe640 1 -- 172.21.15.32:0/1014084403 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b815fd60 con 0x7f32b815f7a0 2024-04-24T23:08:48.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.122+0000 7f32bcf46640 1 --2- 172.21.15.32:0/1014084403 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32b8160d60 0x7f32b81611e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:08:48.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.122+0000 7f32b77fe640 1 --2- 172.21.15.32:0/1014084403 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32b815f7a0 0x7f32b80ca220 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f32ac034040 tx=0x7f32b80d3be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:48.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.122+0000 7f32b4ff9640 1 -- 172.21.15.32:0/1014084403 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ac042070 con 0x7f32b815f7a0 2024-04-24T23:08:48.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.122+0000 7f32b4ff9640 1 -- 172.21.15.32:0/1014084403 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32ac007480 con 0x7f32b815f7a0 2024-04-24T23:08:48.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.122+0000 7f32be9a9640 1 -- 172.21.15.32:0/1014084403 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32b8160040 con 0x7f32b815f7a0 2024-04-24T23:08:48.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.122+0000 7f32b4ff9640 1 -- 172.21.15.32:0/1014084403 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ac001c70 con 0x7f32b815f7a0 2024-04-24T23:08:48.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.122+0000 7f32be9a9640 1 -- 172.21.15.32:0/1014084403 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32b8160600 con 0x7f32b815f7a0 2024-04-24T23:08:48.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.123+0000 7f32be9a9640 1 -- 172.21.15.32:0/1014084403 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32b8068430 con 0x7f32b815f7a0 2024-04-24T23:08:48.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.124+0000 7f32b4ff9640 1 -- 172.21.15.32:0/1014084403 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f32ac045070 con 0x7f32b815f7a0 2024-04-24T23:08:48.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.124+0000 7f32b4ff9640 1 --2- 172.21.15.32:0/1014084403 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f32a406be60 0x7f32a406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:48.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.124+0000 7f32b4ff9640 1 -- 172.21.15.32:0/1014084403 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7f32ac0abb00 con 0x7f32b815f7a0 2024-04-24T23:08:48.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.124+0000 7f32b4ff9640 1 -- 172.21.15.32:0/1014084403 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=159}) -- 0x7f32a4073990 con 0x7f32b815f7a0 2024-04-24T23:08:48.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.124+0000 7f32b7fff640 1 --2- 172.21.15.32:0/1014084403 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f32a406be60 0x7f32a406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:48.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.125+0000 7f32b7fff640 1 --2- 172.21.15.32:0/1014084403 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f32a406be60 0x7f32a406e320 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f32a8009a20 tx=0x7f32a8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:48.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.128+0000 7f32b4ff9640 1 -- 172.21.15.32:0/1014084403 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f32ac04c030 con 0x7f32b815f7a0 2024-04-24T23:08:48.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.347+0000 7f32be9a9640 1 -- 172.21.15.32:0/1014084403 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f32b81552f0 con 0x7f32b815f7a0 2024-04-24T23:08:48.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.347+0000 7f32b4ff9640 1 -- 172.21.15.32:0/1014084403 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+5186 (secure 0 0 0) 0x7f32ac07a550 con 0x7f32b815f7a0 2024-04-24T23:08:48.349 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:08:48.349 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":65,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7120,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/1196305174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1196305174},{"type":"v1","addr":"172.21.15.121:6836","nonce":1196305174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7123,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3015902884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3015902884},{"type":"v1","addr":"172.21.15.121:6837","nonce":3015902884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":8280,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/1033273628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1033273628},{"type":"v1","addr":"172.21.15.32:6833","nonce":1033273628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:06:31.291599+0000","modified":"2024-04-24T23:06:33.296657+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8286},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8286":{"gid":8286,"name":"a","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/984603360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":984603360},{"type":"v1","addr":"172.21.15.32:6835","nonce":984603360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8286,"qdb_cluster":[8286]},"id":9}]} 2024-04-24T23:08:48.350 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 65 2024-04-24T23:08:48.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.349+0000 7f32be9a9640 1 -- 172.21.15.32:0/1014084403 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f32a406be60 msgr2=0x7f32a406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:48.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.349+0000 7f32be9a9640 1 --2- 172.21.15.32:0/1014084403 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f32a406be60 0x7f32a406e320 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f32a8009a20 tx=0x7f32a8008040 comp rx=0 tx=0).stop 2024-04-24T23:08:48.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.349+0000 7f32be9a9640 1 -- 172.21.15.32:0/1014084403 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32b815f7a0 msgr2=0x7f32b80ca220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:48.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.349+0000 7f32be9a9640 1 --2- 172.21.15.32:0/1014084403 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32b815f7a0 0x7f32b80ca220 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f32ac034040 tx=0x7f32b80d3be0 comp rx=0 tx=0).stop 2024-04-24T23:08:48.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.349+0000 7f32be9a9640 1 -- 172.21.15.32:0/1014084403 shutdown_connections 2024-04-24T23:08:48.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.349+0000 7f32be9a9640 1 --2- 172.21.15.32:0/1014084403 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f32a406be60 0x7f32a406e320 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:48.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.349+0000 7f32be9a9640 1 --2- 172.21.15.32:0/1014084403 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32b815f7a0 0x7f32b80ca220 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:48.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.349+0000 7f32be9a9640 1 --2- 172.21.15.32:0/1014084403 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32b8160d60 0x7f32b81611e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:48.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.349+0000 7f32be9a9640 1 --2- 172.21.15.32:0/1014084403 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f32a0003f30 0x7f32b8157dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:48.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.349+0000 7f32be9a9640 1 -- 172.21.15.32:0/1014084403 >> 172.21.15.32:0/1014084403 conn(0x7f32b80d11d0 msgr2=0x7f32b80b82c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:48.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.349+0000 7f32be9a9640 1 -- 172.21.15.32:0/1014084403 shutdown_connections 2024-04-24T23:08:48.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.350+0000 7f32be9a9640 1 -- 172.21.15.32:0/1014084403 wait complete. 2024-04-24T23:08:48.362 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:08:48.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.461+0000 7f4ee7e9e640 1 Processor -- start 2024-04-24T23:08:48.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.462+0000 7f4ee7e9e640 1 -- start start 2024-04-24T23:08:48.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.462+0000 7f4ee7e9e640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4ee015b380 0x7f4ee006c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:48.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.462+0000 7f4ee7e9e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ee00cef50 con 0x7f4ee015cb00 2024-04-24T23:08:48.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.463+0000 7f4ee7e9e640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ee00b4520 con 0x7f4ee00ceb40 2024-04-24T23:08:48.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.463+0000 7f4ee7e9e640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f4ee00b4800 con 0x7f4ee015b380 2024-04-24T23:08:48.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.463+0000 7f4ee5439640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4ee015cb00 0x7f4ee00ce430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52370/0 (socket says 172.21.15.32:52370) 2024-04-24T23:08:48.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.463+0000 7f4ee5439640 1 -- 172.21.15.32:0/683275423 learned_addr learned my addr 172.21.15.32:0/683275423 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:08:48.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.463+0000 7f4ee5c3a640 1 --2- 172.21.15.32:0/683275423 >> v2:172.21.15.121:3300/0 conn(0x7f4ee015b380 0x7f4ee006c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:48.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.463+0000 7f4ee4c38640 1 -- 172.21.15.32:0/683275423 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3675790887 0 0) 0x7f4ed0001110 con 0x7f4ee015cb00 2024-04-24T23:08:48.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.463+0000 7f4ee4c38640 1 -- 172.21.15.32:0/683275423 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ecc003520 con 0x7f4ee015cb00 2024-04-24T23:08:48.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.463+0000 7f4ee4c38640 1 -- 172.21.15.32:0/683275423 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1797714575 0 0) 0x7f4ecc003520 con 0x7f4ee015cb00 2024-04-24T23:08:48.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.463+0000 7f4ee4c38640 1 -- 172.21.15.32:0/683275423 >> v2:172.21.15.121:3300/0 conn(0x7f4ee015b380 msgr2=0x7f4ee006c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:48.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.463+0000 7f4ee4c38640 1 --2- 172.21.15.32:0/683275423 >> v2:172.21.15.121:3300/0 conn(0x7f4ee015b380 0x7f4ee006c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:48.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.463+0000 7f4ee4c38640 1 -- 172.21.15.32:0/683275423 >> v1:172.21.15.121:6789/0 conn(0x7f4ee00ceb40 legacy=0x7f4ee00b55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:48.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.463+0000 7f4ee4c38640 1 -- 172.21.15.32:0/683275423 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee00b4ae0 con 0x7f4ee015cb00 2024-04-24T23:08:48.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.463+0000 7f4ee4c38640 1 -- 172.21.15.32:0/683275423 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f4ed0001840 con 0x7f4ee015cb00 2024-04-24T23:08:48.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.463+0000 7f4ee4c38640 1 -- 172.21.15.32:0/683275423 >> v1:172.21.15.32:6789/0 conn(0x7f4ee015cb00 legacy=0x7f4ee00ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:48.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.463+0000 7f4ee4c38640 1 --2- 172.21.15.32:0/683275423 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ecc0039b0 0x7f4ecc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:48.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.463+0000 7f4ee4c38640 1 --2- 172.21.15.32:0/683275423 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ecc004800 0x7f4ecc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:48.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.463+0000 7f4ee4c38640 1 --2- 172.21.15.32:0/683275423 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ecc0052e0 0x7f4ecc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:48.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.463+0000 7f4ee4c38640 1 -- 172.21.15.32:0/683275423 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee00b4800 con 0x7f4ecc0052e0 2024-04-24T23:08:48.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.463+0000 7f4ee4c38640 1 -- 172.21.15.32:0/683275423 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4ecc00c1b0 con 0x7f4ecc004800 2024-04-24T23:08:48.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.463+0000 7f4ee4c38640 1 -- 172.21.15.32:0/683275423 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f4ecc00c3b0 con 0x7f4ecc0039b0 2024-04-24T23:08:48.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.463+0000 7f4ee643b640 1 --2- 172.21.15.32:0/683275423 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ecc0052e0 0x7f4ecc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:48.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.464+0000 7f4ee5c3a640 1 --2- 172.21.15.32:0/683275423 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ecc0039b0 0x7f4ecc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:48.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.464+0000 7f4ee5439640 1 --2- 172.21.15.32:0/683275423 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ecc004800 0x7f4ecc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:48.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.464+0000 7f4ee643b640 1 -- 172.21.15.32:0/683275423 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ecc0039b0 msgr2=0x7f4ecc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:48.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.464+0000 7f4ee643b640 1 --2- 172.21.15.32:0/683275423 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ecc0039b0 0x7f4ecc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:48.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.464+0000 7f4ee643b640 1 -- 172.21.15.32:0/683275423 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ecc004800 msgr2=0x7f4ecc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:48.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.464+0000 7f4ee643b640 1 --2- 172.21.15.32:0/683275423 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ecc004800 0x7f4ecc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:48.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.464+0000 7f4ee643b640 1 -- 172.21.15.32:0/683275423 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ecc003520 con 0x7f4ecc0052e0 2024-04-24T23:08:48.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.464+0000 7f4ee643b640 1 -- 172.21.15.32:0/683275423 reap_dead start 2024-04-24T23:08:48.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.464+0000 7f4ee643b640 1 --2- 172.21.15.32:0/683275423 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ecc0052e0 0x7f4ecc00b9c0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8001e20 tx=0x7f4ecc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=745e42645f3a5e29 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:48.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.465+0000 7f4ee4c38640 1 -- 172.21.15.32:0/683275423 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed8001a70 con 0x7f4ecc0052e0 2024-04-24T23:08:48.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.465+0000 7f4ee4c38640 1 -- 172.21.15.32:0/683275423 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ed802ae40 con 0x7f4ecc0052e0 2024-04-24T23:08:48.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.465+0000 7f4ee4c38640 1 -- 172.21.15.32:0/683275423 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed8003bb0 con 0x7f4ecc0052e0 2024-04-24T23:08:48.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.465+0000 7f4ee7e9e640 1 -- 172.21.15.32:0/683275423 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ecc0052e0 msgr2=0x7f4ecc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:48.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.465+0000 7f4ee7e9e640 1 --2- 172.21.15.32:0/683275423 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ecc0052e0 0x7f4ecc00b9c0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8001e20 tx=0x7f4ecc005830 comp rx=0 tx=0).stop 2024-04-24T23:08:48.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.465+0000 7f4ee7e9e640 1 -- 172.21.15.32:0/683275423 shutdown_connections 2024-04-24T23:08:48.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.465+0000 7f4ee7e9e640 1 --2- 172.21.15.32:0/683275423 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ecc0052e0 0x7f4ecc00b9c0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:48.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.465+0000 7f4ee7e9e640 1 -- 172.21.15.32:0/683275423 >> 172.21.15.32:0/683275423 conn(0x7f4ee006b110 msgr2=0x7f4ee006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:48.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.465+0000 7f4ee7e9e640 1 -- 172.21.15.32:0/683275423 shutdown_connections 2024-04-24T23:08:48.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.465+0000 7f4ee7e9e640 1 -- 172.21.15.32:0/683275423 wait complete. 2024-04-24T23:08:48.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.466+0000 7f4ee7e9e640 1 Processor -- start 2024-04-24T23:08:48.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.466+0000 7f4ee7e9e640 1 -- start start 2024-04-24T23:08:48.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.467+0000 7f4ee7e9e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ecc0052e0 0x7f4ee014aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:48.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.467+0000 7f4ee7e9e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ee0154200 0x7f4ee01f3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:48.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.467+0000 7f4ee7e9e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ee01f4040 0x7f4ee014c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:48.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.467+0000 7f4ee7e9e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee015b760 con 0x7f4ecc0052e0 2024-04-24T23:08:48.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.467+0000 7f4ee7e9e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee00b50a0 con 0x7f4ee01f4040 2024-04-24T23:08:48.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.467+0000 7f4ee7e9e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f4ee00ce430 con 0x7f4ee0154200 2024-04-24T23:08:48.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.467+0000 7f4ee5c3a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ecc0052e0 0x7f4ee014aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:48.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.467+0000 7f4ee5c3a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ecc0052e0 0x7f4ee014aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43782/0 (socket says 172.21.15.32:43782) 2024-04-24T23:08:48.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.467+0000 7f4ee5c3a640 1 -- 172.21.15.32:0/2981259844 learned_addr learned my addr 172.21.15.32:0/2981259844 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:48.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.467+0000 7f4ee5439640 1 --2- 172.21.15.32:0/2981259844 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ee0154200 0x7f4ee01f3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:48.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.467+0000 7f4ee5c3a640 1 -- 172.21.15.32:0/2981259844 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ee0154200 msgr2=0x7f4ee01f3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:48.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.467+0000 7f4ee5c3a640 1 --2- 172.21.15.32:0/2981259844 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ee0154200 0x7f4ee01f3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:48.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.467+0000 7f4ee643b640 1 --2- 172.21.15.32:0/2981259844 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ee01f4040 0x7f4ee014c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:48.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.467+0000 7f4ee5c3a640 1 -- 172.21.15.32:0/2981259844 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ee01f4040 msgr2=0x7f4ee014c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:48.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.467+0000 7f4ee5c3a640 1 --2- 172.21.15.32:0/2981259844 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ee01f4040 0x7f4ee014c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:48.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.468+0000 7f4ee5c3a640 1 -- 172.21.15.32:0/2981259844 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee014eb30 con 0x7f4ecc0052e0 2024-04-24T23:08:48.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.468+0000 7f4ee5c3a640 1 --2- 172.21.15.32:0/2981259844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ecc0052e0 0x7f4ee014aa70 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f4edc011940 tx=0x7f4edc011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:48.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.468+0000 7f4ed67fc640 1 -- 172.21.15.32:0/2981259844 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4edc00faf0 con 0x7f4ecc0052e0 2024-04-24T23:08:48.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.468+0000 7f4ed67fc640 1 -- 172.21.15.32:0/2981259844 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4edc00fc90 con 0x7f4ecc0052e0 2024-04-24T23:08:48.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.468+0000 7f4ee7e9e640 1 -- 172.21.15.32:0/2981259844 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ee014ee10 con 0x7f4ecc0052e0 2024-04-24T23:08:48.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.468+0000 7f4ed67fc640 1 -- 172.21.15.32:0/2981259844 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4edc014440 con 0x7f4ecc0052e0 2024-04-24T23:08:48.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.468+0000 7f4ee7e9e640 1 -- 172.21.15.32:0/2981259844 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ee0154680 con 0x7f4ecc0052e0 2024-04-24T23:08:48.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.469+0000 7f4ee7e9e640 1 -- 172.21.15.32:0/2981259844 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ee014efe0 con 0x7f4ecc0052e0 2024-04-24T23:08:48.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.469+0000 7f4ed67fc640 1 -- 172.21.15.32:0/2981259844 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4edc012040 con 0x7f4ecc0052e0 2024-04-24T23:08:48.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.470+0000 7f4ed67fc640 1 --2- 172.21.15.32:0/2981259844 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4ec406be30 0x7f4ec406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:48.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.470+0000 7f4ed67fc640 1 -- 172.21.15.32:0/2981259844 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7f4edc08b550 con 0x7f4ecc0052e0 2024-04-24T23:08:48.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.470+0000 7f4ed67fc640 1 -- 172.21.15.32:0/2981259844 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=159}) -- 0x7f4ec4073960 con 0x7f4ecc0052e0 2024-04-24T23:08:48.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.470+0000 7f4ee5439640 1 --2- 172.21.15.32:0/2981259844 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4ec406be30 0x7f4ec406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:48.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.471+0000 7f4ee5439640 1 --2- 172.21.15.32:0/2981259844 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4ec406be30 0x7f4ec406e2f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f4ee015ab50 tx=0x7f4ed0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:48.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.474+0000 7f4ed67fc640 1 -- 172.21.15.32:0/2981259844 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4ee014efe0 con 0x7f4ecc0052e0 2024-04-24T23:08:48.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.693+0000 7f4ee7e9e640 1 -- 172.21.15.32:0/2981259844 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ee0153550 con 0x7f4ecc0052e0 2024-04-24T23:08:48.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.693+0000 7f4ed67fc640 1 -- 172.21.15.32:0/2981259844 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+5186 (secure 0 0 0) 0x7f4edc019030 con 0x7f4ecc0052e0 2024-04-24T23:08:48.695 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:08:48.695 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":65,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7120,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/1196305174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1196305174},{"type":"v1","addr":"172.21.15.121:6836","nonce":1196305174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7123,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3015902884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3015902884},{"type":"v1","addr":"172.21.15.121:6837","nonce":3015902884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":8280,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/1033273628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1033273628},{"type":"v1","addr":"172.21.15.32:6833","nonce":1033273628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:06:31.291599+0000","modified":"2024-04-24T23:06:33.296657+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8286},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8286":{"gid":8286,"name":"a","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/984603360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":984603360},{"type":"v1","addr":"172.21.15.32:6835","nonce":984603360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8286,"qdb_cluster":[8286]},"id":9}]} 2024-04-24T23:08:48.695 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 65 2024-04-24T23:08:48.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.695+0000 7f4ee7e9e640 1 -- 172.21.15.32:0/2981259844 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4ec406be30 msgr2=0x7f4ec406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:48.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.695+0000 7f4ee7e9e640 1 --2- 172.21.15.32:0/2981259844 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4ec406be30 0x7f4ec406e2f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f4ee015ab50 tx=0x7f4ed0009040 comp rx=0 tx=0).stop 2024-04-24T23:08:48.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.695+0000 7f4ee7e9e640 1 -- 172.21.15.32:0/2981259844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ecc0052e0 msgr2=0x7f4ee014aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:48.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.695+0000 7f4ee7e9e640 1 --2- 172.21.15.32:0/2981259844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ecc0052e0 0x7f4ee014aa70 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f4edc011940 tx=0x7f4edc011e00 comp rx=0 tx=0).stop 2024-04-24T23:08:48.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.695+0000 7f4ee7e9e640 1 -- 172.21.15.32:0/2981259844 shutdown_connections 2024-04-24T23:08:48.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.695+0000 7f4ee7e9e640 1 --2- 172.21.15.32:0/2981259844 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4ec406be30 0x7f4ec406e2f0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:48.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.695+0000 7f4ee7e9e640 1 --2- 172.21.15.32:0/2981259844 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ee01f4040 0x7f4ee014c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:48.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.695+0000 7f4ee7e9e640 1 --2- 172.21.15.32:0/2981259844 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ee0154200 0x7f4ee01f3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:48.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.695+0000 7f4ee7e9e640 1 --2- 172.21.15.32:0/2981259844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ecc0052e0 0x7f4ee014aa70 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:48.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.695+0000 7f4ee7e9e640 1 -- 172.21.15.32:0/2981259844 >> 172.21.15.32:0/2981259844 conn(0x7f4ee006b110 msgr2=0x7f4ee0154c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:48.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.695+0000 7f4ee7e9e640 1 -- 172.21.15.32:0/2981259844 shutdown_connections 2024-04-24T23:08:48.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.695+0000 7f4ee7e9e640 1 -- 172.21.15.32:0/2981259844 wait complete. 2024-04-24T23:08:48.707 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:08:48.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.806+0000 7f974a876640 1 Processor -- start 2024-04-24T23:08:48.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.806+0000 7f974a876640 1 -- start start 2024-04-24T23:08:48.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.807+0000 7f974a876640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f974415e9b0 0x7f97440d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:48.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.807+0000 7f974a876640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f97440d4210 0x7f974406b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:48.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.807+0000 7f974a876640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f97440ba0e0 0x7f974406b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:48.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.807+0000 7f974a876640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f97440d3170 con 0x7f97440ba0e0 2024-04-24T23:08:48.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.807+0000 7f974a876640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f97440d3450 con 0x7f974415e9b0 2024-04-24T23:08:48.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.807+0000 7f974a876640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f97440d3730 con 0x7f97440d4210 2024-04-24T23:08:48.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.807+0000 7f9748e13640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f97440ba0e0 0x7f974406b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:48.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.807+0000 7f9748e13640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f97440ba0e0 0x7f974406b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43792/0 (socket says 172.21.15.32:43792) 2024-04-24T23:08:48.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.807+0000 7f9748e13640 1 -- 172.21.15.32:0/1282244849 learned_addr learned my addr 172.21.15.32:0/1282244849 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:48.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.808+0000 7f9743fff640 1 --2- 172.21.15.32:0/1282244849 >> v2:172.21.15.121:3300/0 conn(0x7f974415e9b0 0x7f97440d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:48.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.808+0000 7f97437fe640 1 --2- 172.21.15.32:0/1282244849 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f97440d4210 0x7f974406b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:48.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.808+0000 7f9748e13640 1 -- 172.21.15.32:0/1282244849 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f97440d4210 msgr2=0x7f974406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:48.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.808+0000 7f9748e13640 1 --2- 172.21.15.32:0/1282244849 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f97440d4210 0x7f974406b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:48.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.808+0000 7f9748e13640 1 -- 172.21.15.32:0/1282244849 >> v2:172.21.15.121:3300/0 conn(0x7f974415e9b0 msgr2=0x7f97440d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:48.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.808+0000 7f9748e13640 1 --2- 172.21.15.32:0/1282244849 >> v2:172.21.15.121:3300/0 conn(0x7f974415e9b0 0x7f97440d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:48.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.808+0000 7f9748e13640 1 -- 172.21.15.32:0/1282244849 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97440d3a10 con 0x7f97440ba0e0 2024-04-24T23:08:48.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.808+0000 7f9743fff640 1 --2- 172.21.15.32:0/1282244849 >> v2:172.21.15.121:3300/0 conn(0x7f974415e9b0 0x7f97440d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:08:48.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.808+0000 7f97437fe640 1 --2- 172.21.15.32:0/1282244849 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f97440d4210 0x7f974406b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:08:48.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.808+0000 7f9748e13640 1 --2- 172.21.15.32:0/1282244849 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97440ba0e0 0x7f974406b650 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f974406be20 tx=0x7f973c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=864f4434e2d2e9fc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:48.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.809+0000 7f9742ffd640 1 -- 172.21.15.32:0/1282244849 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973c036070 con 0x7f97440ba0e0 2024-04-24T23:08:48.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.809+0000 7f9742ffd640 1 -- 172.21.15.32:0/1282244849 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f973c031600 con 0x7f97440ba0e0 2024-04-24T23:08:48.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.809+0000 7f9742ffd640 1 -- 172.21.15.32:0/1282244849 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973c0319d0 con 0x7f97440ba0e0 2024-04-24T23:08:48.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.809+0000 7f974a876640 1 -- 172.21.15.32:0/1282244849 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97440ba0e0 msgr2=0x7f974406b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:48.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.809+0000 7f974a876640 1 --2- 172.21.15.32:0/1282244849 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97440ba0e0 0x7f974406b650 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f974406be20 tx=0x7f973c02a6a0 comp rx=0 tx=0).stop 2024-04-24T23:08:48.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.809+0000 7f974a876640 1 -- 172.21.15.32:0/1282244849 shutdown_connections 2024-04-24T23:08:48.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.809+0000 7f974a876640 1 --2- 172.21.15.32:0/1282244849 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97440ba0e0 0x7f974406b650 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:48.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.809+0000 7f974a876640 1 --2- 172.21.15.32:0/1282244849 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f97440d4210 0x7f974406b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:48.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.809+0000 7f974a876640 1 --2- 172.21.15.32:0/1282244849 >> v2:172.21.15.121:3300/0 conn(0x7f974415e9b0 0x7f97440d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:48.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.809+0000 7f974a876640 1 -- 172.21.15.32:0/1282244849 >> 172.21.15.32:0/1282244849 conn(0x7f97440d11d0 msgr2=0x7f97440d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:48.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.809+0000 7f974a876640 1 -- 172.21.15.32:0/1282244849 shutdown_connections 2024-04-24T23:08:48.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.809+0000 7f974a876640 1 -- 172.21.15.32:0/1282244849 wait complete. 2024-04-24T23:08:48.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.810+0000 7f974a876640 1 Processor -- start 2024-04-24T23:08:48.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.811+0000 7f974a876640 1 -- start start 2024-04-24T23:08:48.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.811+0000 7f974a876640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f97440ba0e0 0x7f97441e97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:48.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.811+0000 7f974a876640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f97440d4210 0x7f97440cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:48.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.811+0000 7f974a876640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f974415e9b0 0x7f97440ce430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:48.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.811+0000 7f974a876640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9744071730 con 0x7f974415e9b0 2024-04-24T23:08:48.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.811+0000 7f974a876640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f97440d2400 con 0x7f97440ba0e0 2024-04-24T23:08:48.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.811+0000 7f974a876640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f97440718b0 con 0x7f97440d4210 2024-04-24T23:08:48.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.811+0000 7f9748e13640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f974415e9b0 0x7f97440ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:48.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.811+0000 7f9748e13640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f974415e9b0 0x7f97440ce430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43802/0 (socket says 172.21.15.32:43802) 2024-04-24T23:08:48.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.811+0000 7f9748e13640 1 -- 172.21.15.32:0/3185878147 learned_addr learned my addr 172.21.15.32:0/3185878147 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:48.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.812+0000 7f97437fe640 1 --2- 172.21.15.32:0/3185878147 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f97440d4210 0x7f97440cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:48.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.812+0000 7f9743fff640 1 --2- 172.21.15.32:0/3185878147 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f97440ba0e0 0x7f97441e97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:48.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.812+0000 7f9748e13640 1 -- 172.21.15.32:0/3185878147 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f97440d4210 msgr2=0x7f97440cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:48.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.812+0000 7f9748e13640 1 --2- 172.21.15.32:0/3185878147 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f97440d4210 0x7f97440cdef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:48.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.812+0000 7f9748e13640 1 -- 172.21.15.32:0/3185878147 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f97440ba0e0 msgr2=0x7f97441e97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:48.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.812+0000 7f9748e13640 1 --2- 172.21.15.32:0/3185878147 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f97440ba0e0 0x7f97441e97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:48.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.812+0000 7f9748e13640 1 -- 172.21.15.32:0/3185878147 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97441f41c0 con 0x7f974415e9b0 2024-04-24T23:08:48.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.812+0000 7f9748e13640 1 --2- 172.21.15.32:0/3185878147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f974415e9b0 0x7f97440ce430 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f973c009f40 tx=0x7f973c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:48.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.812+0000 7f9740ff9640 1 -- 172.21.15.32:0/3185878147 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973c036050 con 0x7f974415e9b0 2024-04-24T23:08:48.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.812+0000 7f9740ff9640 1 -- 172.21.15.32:0/3185878147 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f973c042070 con 0x7f974415e9b0 2024-04-24T23:08:48.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.812+0000 7f9740ff9640 1 -- 172.21.15.32:0/3185878147 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973c009510 con 0x7f974415e9b0 2024-04-24T23:08:48.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.812+0000 7f974a876640 1 -- 172.21.15.32:0/3185878147 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97441f44a0 con 0x7f974415e9b0 2024-04-24T23:08:48.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.812+0000 7f974a876640 1 -- 172.21.15.32:0/3185878147 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97441f4a60 con 0x7f974415e9b0 2024-04-24T23:08:48.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.813+0000 7f974a876640 1 -- 172.21.15.32:0/3185878147 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9744068430 con 0x7f974415e9b0 2024-04-24T23:08:48.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.814+0000 7f9740ff9640 1 -- 172.21.15.32:0/3185878147 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f973c007bd0 con 0x7f974415e9b0 2024-04-24T23:08:48.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.814+0000 7f9740ff9640 1 --2- 172.21.15.32:0/3185878147 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f973006be30 0x7f973006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:48.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.814+0000 7f9740ff9640 1 -- 172.21.15.32:0/3185878147 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7f973c0abe50 con 0x7f974415e9b0 2024-04-24T23:08:48.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.814+0000 7f9740ff9640 1 -- 172.21.15.32:0/3185878147 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=159}) -- 0x7f97300739c0 con 0x7f974415e9b0 2024-04-24T23:08:48.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.814+0000 7f9743fff640 1 --2- 172.21.15.32:0/3185878147 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f973006be30 0x7f973006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:48.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.815+0000 7f9743fff640 1 --2- 172.21.15.32:0/3185878147 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f973006be30 0x7f973006e2f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f97441553a0 tx=0x7f9734008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:48.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:48.818+0000 7f9740ff9640 1 -- 172.21.15.32:0/3185878147 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f973c07a800 con 0x7f974415e9b0 2024-04-24T23:08:49.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.002+0000 7f974a876640 1 -- 172.21.15.32:0/3185878147 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f97440d17c0 con 0x7f974415e9b0 2024-04-24T23:08:49.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.003+0000 7f9740ff9640 1 -- 172.21.15.32:0/3185878147 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v158) ==== 74+0+16666 (secure 0 0 0) 0x7f973c07a100 con 0x7f974415e9b0 2024-04-24T23:08:49.005 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:08:49.005 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":158,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:08:06.831677+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:06:29.591736+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-24T23:06:30.267629+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:08:49.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.005+0000 7f974a876640 1 -- 172.21.15.32:0/3185878147 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f973006be30 msgr2=0x7f973006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:49.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.005+0000 7f974a876640 1 --2- 172.21.15.32:0/3185878147 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f973006be30 0x7f973006e2f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f97441553a0 tx=0x7f9734008040 comp rx=0 tx=0).stop 2024-04-24T23:08:49.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.005+0000 7f974a876640 1 -- 172.21.15.32:0/3185878147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f974415e9b0 msgr2=0x7f97440ce430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:49.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.005+0000 7f974a876640 1 --2- 172.21.15.32:0/3185878147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f974415e9b0 0x7f97440ce430 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f973c009f40 tx=0x7f973c004a40 comp rx=0 tx=0).stop 2024-04-24T23:08:49.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.005+0000 7f974a876640 1 -- 172.21.15.32:0/3185878147 shutdown_connections 2024-04-24T23:08:49.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.005+0000 7f974a876640 1 --2- 172.21.15.32:0/3185878147 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f973006be30 0x7f973006e2f0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:49.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.005+0000 7f974a876640 1 --2- 172.21.15.32:0/3185878147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f974415e9b0 0x7f97440ce430 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:49.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.005+0000 7f974a876640 1 --2- 172.21.15.32:0/3185878147 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f97440d4210 0x7f97440cdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:49.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.005+0000 7f974a876640 1 --2- 172.21.15.32:0/3185878147 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f97440ba0e0 0x7f97441e97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:49.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.005+0000 7f974a876640 1 -- 172.21.15.32:0/3185878147 >> 172.21.15.32:0/3185878147 conn(0x7f97440d11d0 msgr2=0x7f974415a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:49.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.005+0000 7f974a876640 1 -- 172.21.15.32:0/3185878147 shutdown_connections 2024-04-24T23:08:49.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.005+0000 7f974a876640 1 -- 172.21.15.32:0/3185878147 wait complete. 2024-04-24T23:08:49.019 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-24T23:08:49.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.119+0000 7fb8201e7640 1 Processor -- start 2024-04-24T23:08:49.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.119+0000 7fb8201e7640 1 -- start start 2024-04-24T23:08:49.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.120+0000 7fb8201e7640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb81815e9b0 0x7fb8180d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:49.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.120+0000 7fb8201e7640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8180d3170 con 0x7fb8180d4210 2024-04-24T23:08:49.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.120+0000 7fb8201e7640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8180d3450 con 0x7fb81806b820 2024-04-24T23:08:49.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.120+0000 7fb8201e7640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fb8180d3730 con 0x7fb81815e9b0 2024-04-24T23:08:49.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.120+0000 7fb81d782640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb8180d4210 0x7fb81806b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52378/0 (socket says 172.21.15.32:52378) 2024-04-24T23:08:49.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.120+0000 7fb81d782640 1 -- 172.21.15.32:0/500462862 learned_addr learned my addr 172.21.15.32:0/500462862 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:08:49.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.121+0000 7fb81cf81640 1 -- 172.21.15.32:0/500462862 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1767779753 0 0) 0x7fb8180d3170 con 0x7fb8180d4210 2024-04-24T23:08:49.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.121+0000 7fb81cf81640 1 -- 172.21.15.32:0/500462862 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb800003520 con 0x7fb8180d4210 2024-04-24T23:08:49.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.121+0000 7fb81df83640 1 --2- 172.21.15.32:0/500462862 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb81815e9b0 0x7fb8180d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:49.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.121+0000 7fb81cf81640 1 -- 172.21.15.32:0/500462862 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1451631920 0 0) 0x7fb800003520 con 0x7fb8180d4210 2024-04-24T23:08:49.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.121+0000 7fb81cf81640 1 -- 172.21.15.32:0/500462862 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb81815e9b0 msgr2=0x7fb8180d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:49.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.121+0000 7fb81cf81640 1 --2- 172.21.15.32:0/500462862 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb81815e9b0 0x7fb8180d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:49.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.121+0000 7fb81cf81640 1 -- 172.21.15.32:0/500462862 >> v1:172.21.15.121:6789/0 conn(0x7fb81806b820 legacy=0x7fb8180b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:49.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.121+0000 7fb81cf81640 1 -- 172.21.15.32:0/500462862 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8180d3a10 con 0x7fb8180d4210 2024-04-24T23:08:49.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.121+0000 7fb81cf81640 1 -- 172.21.15.32:0/500462862 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7fb80c001670 con 0x7fb8180d4210 2024-04-24T23:08:49.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.121+0000 7fb81cf81640 1 -- 172.21.15.32:0/500462862 >> v1:172.21.15.32:6789/0 conn(0x7fb8180d4210 legacy=0x7fb81806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:49.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.121+0000 7fb81cf81640 1 --2- 172.21.15.32:0/500462862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8000039e0 0x7fb800003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:49.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.121+0000 7fb81cf81640 1 --2- 172.21.15.32:0/500462862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb800004800 0x7fb800004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:49.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.121+0000 7fb81cf81640 1 --2- 172.21.15.32:0/500462862 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb81815e9b0 0x7fb8000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:49.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.121+0000 7fb81cf81640 1 -- 172.21.15.32:0/500462862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb8180d3730 con 0x7fb800004800 2024-04-24T23:08:49.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.121+0000 7fb81cf81640 1 -- 172.21.15.32:0/500462862 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb80000ba70 con 0x7fb8000039e0 2024-04-24T23:08:49.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.121+0000 7fb81cf81640 1 -- 172.21.15.32:0/500462862 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fb80000bc70 con 0x7fb81815e9b0 2024-04-24T23:08:49.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.121+0000 7fb81d782640 1 --2- 172.21.15.32:0/500462862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb800004800 0x7fb800004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:49.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.121+0000 7fb81df83640 1 --2- 172.21.15.32:0/500462862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8000039e0 0x7fb800003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:49.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.121+0000 7fb81e784640 1 --2- 172.21.15.32:0/500462862 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb81815e9b0 0x7fb8000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:49.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.121+0000 7fb81d782640 1 -- 172.21.15.32:0/500462862 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb81815e9b0 msgr2=0x7fb8000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:49.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.121+0000 7fb81d782640 1 --2- 172.21.15.32:0/500462862 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb81815e9b0 0x7fb8000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:49.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.121+0000 7fb81d782640 1 -- 172.21.15.32:0/500462862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8000039e0 msgr2=0x7fb800003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:49.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.121+0000 7fb81d782640 1 --2- 172.21.15.32:0/500462862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8000039e0 0x7fb800003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:49.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.121+0000 7fb81d782640 1 -- 172.21.15.32:0/500462862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb800003520 con 0x7fb800004800 2024-04-24T23:08:49.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.121+0000 7fb81e784640 1 --2- 172.21.15.32:0/500462862 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb81815e9b0 0x7fb8000051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:08:49.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.122+0000 7fb81d782640 1 --2- 172.21.15.32:0/500462862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb800004800 0x7fb800004c80 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fb80c00c210 tx=0x7fb800005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb8a0408f5f926df server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:49.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.122+0000 7fb81cf81640 1 -- 172.21.15.32:0/500462862 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80c007cf0 con 0x7fb800004800 2024-04-24T23:08:49.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.122+0000 7fb81cf81640 1 -- 172.21.15.32:0/500462862 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb80c005af0 con 0x7fb800004800 2024-04-24T23:08:49.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.122+0000 7fb81cf81640 1 -- 172.21.15.32:0/500462862 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80c007070 con 0x7fb800004800 2024-04-24T23:08:49.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.123+0000 7fb8201e7640 1 -- 172.21.15.32:0/500462862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb800004800 msgr2=0x7fb800004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:49.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.123+0000 7fb8201e7640 1 --2- 172.21.15.32:0/500462862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb800004800 0x7fb800004c80 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fb80c00c210 tx=0x7fb800005830 comp rx=0 tx=0).stop 2024-04-24T23:08:49.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.123+0000 7fb81e784640 1 -- 172.21.15.32:0/500462862 reap_dead start 2024-04-24T23:08:49.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.123+0000 7fb8201e7640 1 -- 172.21.15.32:0/500462862 shutdown_connections 2024-04-24T23:08:49.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.123+0000 7fb8201e7640 1 -- 172.21.15.32:0/500462862 >> 172.21.15.32:0/500462862 conn(0x7fb8180d11d0 msgr2=0x7fb8180d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:49.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.123+0000 7fb8201e7640 1 -- 172.21.15.32:0/500462862 shutdown_connections 2024-04-24T23:08:49.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.123+0000 7fb8201e7640 1 -- 172.21.15.32:0/500462862 wait complete. 2024-04-24T23:08:49.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.124+0000 7fb8201e7640 1 Processor -- start 2024-04-24T23:08:49.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.124+0000 7fb8201e7640 1 -- start start 2024-04-24T23:08:49.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.125+0000 7fb8201e7640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb800004800 0x7fb81815ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:49.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.125+0000 7fb8201e7640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8180b5850 0x7fb8181f1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:49.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.125+0000 7fb8201e7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8181f4f20 0x7fb818158450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:49.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.125+0000 7fb8201e7640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb8180ba0e0 con 0x7fb8181f4f20 2024-04-24T23:08:49.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.125+0000 7fb8201e7640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb8180d2400 con 0x7fb8180b5850 2024-04-24T23:08:49.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.125+0000 7fb8201e7640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fb8180ba260 con 0x7fb800004800 2024-04-24T23:08:49.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.125+0000 7fb81e784640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8181f4f20 0x7fb818158450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:49.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.125+0000 7fb81e784640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8181f4f20 0x7fb818158450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43826/0 (socket says 172.21.15.32:43826) 2024-04-24T23:08:49.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.125+0000 7fb81e784640 1 -- 172.21.15.32:0/3057198788 learned_addr learned my addr 172.21.15.32:0/3057198788 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:49.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.125+0000 7fb81d782640 1 --2- 172.21.15.32:0/3057198788 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8180b5850 0x7fb8181f1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:49.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.125+0000 7fb81df83640 1 --2- 172.21.15.32:0/3057198788 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb800004800 0x7fb81815ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:49.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.125+0000 7fb81e784640 1 -- 172.21.15.32:0/3057198788 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb800004800 msgr2=0x7fb81815ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:49.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.125+0000 7fb81e784640 1 --2- 172.21.15.32:0/3057198788 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb800004800 0x7fb81815ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:49.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.125+0000 7fb81e784640 1 -- 172.21.15.32:0/3057198788 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8180b5850 msgr2=0x7fb8181f1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:49.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.125+0000 7fb81e784640 1 --2- 172.21.15.32:0/3057198788 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8180b5850 0x7fb8181f1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:49.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.125+0000 7fb81e784640 1 -- 172.21.15.32:0/3057198788 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8181617f0 con 0x7fb8181f4f20 2024-04-24T23:08:49.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.126+0000 7fb81e784640 1 --2- 172.21.15.32:0/3057198788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8181f4f20 0x7fb818158450 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fb81806bdf0 tx=0x7fb81000e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:49.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.126+0000 7fb8167fc640 1 -- 172.21.15.32:0/3057198788 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb810019070 con 0x7fb8181f4f20 2024-04-24T23:08:49.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.126+0000 7fb8167fc640 1 -- 172.21.15.32:0/3057198788 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8100052e0 con 0x7fb8181f4f20 2024-04-24T23:08:49.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.126+0000 7fb8167fc640 1 -- 172.21.15.32:0/3057198788 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb810003830 con 0x7fb8181f4f20 2024-04-24T23:08:49.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.126+0000 7fb8201e7640 1 -- 172.21.15.32:0/3057198788 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb818159330 con 0x7fb8181f4f20 2024-04-24T23:08:49.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.126+0000 7fb8201e7640 1 -- 172.21.15.32:0/3057198788 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb818161d40 con 0x7fb8181f4f20 2024-04-24T23:08:49.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.127+0000 7fb8201e7640 1 -- 172.21.15.32:0/3057198788 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb818068430 con 0x7fb8181f4f20 2024-04-24T23:08:49.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.127+0000 7fb8167fc640 1 -- 172.21.15.32:0/3057198788 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb81001c070 con 0x7fb8181f4f20 2024-04-24T23:08:49.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.128+0000 7fb8167fc640 1 --2- 172.21.15.32:0/3057198788 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb80406be30 0x7fb80406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:49.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.128+0000 7fb8167fc640 1 -- 172.21.15.32:0/3057198788 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7fb81008a280 con 0x7fb8181f4f20 2024-04-24T23:08:49.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.128+0000 7fb81df83640 1 --2- 172.21.15.32:0/3057198788 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb80406be30 0x7fb80406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:49.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.128+0000 7fb8167fc640 1 -- 172.21.15.32:0/3057198788 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=159}) -- 0x7fb804073960 con 0x7fb8181f4f20 2024-04-24T23:08:49.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.128+0000 7fb81df83640 1 --2- 172.21.15.32:0/3057198788 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb80406be30 0x7fb80406e2f0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fb818158040 tx=0x7fb80800b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:49.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.132+0000 7fb8167fc640 1 -- 172.21.15.32:0/3057198788 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb818159330 con 0x7fb8181f4f20 2024-04-24T23:08:49.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.354+0000 7fb8201e7640 1 -- 172.21.15.32:0/3057198788 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fb8180b8c00 con 0x7fb8181f4f20 2024-04-24T23:08:49.356 INFO:tasks.ceph.mon.a.smithi032.stderr:2024-04-24T23:08:49.354+0000 7f3b809c6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T23:08:49.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.356+0000 7fb8167fc640 1 -- 172.21.15.32:0/3057198788 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v66) ==== 161+0+0 (secure 0 0 0) 0x7fb810058540 con 0x7fb8181f4f20 2024-04-24T23:08:49.357 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-24T23:08:49.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.356+0000 7fb8167fc640 1 -- 172.21.15.32:0/3057198788 <== mon.0 v2:172.21.15.32:3300/0 8 ==== osd_map(159..159 src has 1..159) ==== 374+0+0 (secure 0 0 0) 0x7fb8100be2d0 con 0x7fb8181f4f20 2024-04-24T23:08:49.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.356+0000 7fb8167fc640 1 -- 172.21.15.32:0/3057198788 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=160}) -- 0x7fb8040743a0 con 0x7fb8181f4f20 2024-04-24T23:08:49.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.359+0000 7fb8201e7640 1 -- 172.21.15.32:0/3057198788 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb80406be30 msgr2=0x7fb80406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:49.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.359+0000 7fb8201e7640 1 --2- 172.21.15.32:0/3057198788 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb80406be30 0x7fb80406e2f0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fb818158040 tx=0x7fb80800b040 comp rx=0 tx=0).stop 2024-04-24T23:08:49.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.359+0000 7fb8201e7640 1 -- 172.21.15.32:0/3057198788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8181f4f20 msgr2=0x7fb818158450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:49.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.359+0000 7fb8201e7640 1 --2- 172.21.15.32:0/3057198788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8181f4f20 0x7fb818158450 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fb81806bdf0 tx=0x7fb81000e820 comp rx=0 tx=0).stop 2024-04-24T23:08:49.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.360+0000 7fb8201e7640 1 -- 172.21.15.32:0/3057198788 shutdown_connections 2024-04-24T23:08:49.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.360+0000 7fb8201e7640 1 --2- 172.21.15.32:0/3057198788 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb80406be30 0x7fb80406e2f0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:49.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.360+0000 7fb8201e7640 1 --2- 172.21.15.32:0/3057198788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8181f4f20 0x7fb818158450 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:49.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.360+0000 7fb8201e7640 1 --2- 172.21.15.32:0/3057198788 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8180b5850 0x7fb8181f1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:49.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.360+0000 7fb8201e7640 1 --2- 172.21.15.32:0/3057198788 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb800004800 0x7fb81815ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:49.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.360+0000 7fb8201e7640 1 -- 172.21.15.32:0/3057198788 >> 172.21.15.32:0/3057198788 conn(0x7fb8180d11d0 msgr2=0x7fb81815c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:49.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.360+0000 7fb8201e7640 1 -- 172.21.15.32:0/3057198788 shutdown_connections 2024-04-24T23:08:49.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.360+0000 7fb8201e7640 1 -- 172.21.15.32:0/3057198788 wait complete. 2024-04-24T23:08:49.373 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-24T23:08:49.396 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2024-04-24T23:08:49.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.475+0000 7f81a3ba9640 1 Processor -- start 2024-04-24T23:08:49.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.476+0000 7f81a3ba9640 1 -- start start 2024-04-24T23:08:49.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.476+0000 7f81a3ba9640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f819c15e9b0 0x7f819c0d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:49.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.476+0000 7f81a3ba9640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f819c0d4210 0x7f819c06b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:49.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.476+0000 7f81a3ba9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f819c0ba0e0 0x7f819c06b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:49.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.476+0000 7f81a3ba9640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f819c0d3170 con 0x7f819c0ba0e0 2024-04-24T23:08:49.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.476+0000 7f81a3ba9640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f819c0d3450 con 0x7f819c0d4210 2024-04-24T23:08:49.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.476+0000 7f81a3ba9640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f819c0d3730 con 0x7f819c15e9b0 2024-04-24T23:08:49.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.476+0000 7f81a2146640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f819c0ba0e0 0x7f819c06b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:49.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.477+0000 7f81a2146640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f819c0ba0e0 0x7f819c06b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43850/0 (socket says 172.21.15.32:43850) 2024-04-24T23:08:49.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.477+0000 7f81a2146640 1 -- 172.21.15.32:0/525363311 learned_addr learned my addr 172.21.15.32:0/525363311 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:49.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.477+0000 7f81a1144640 1 --2- 172.21.15.32:0/525363311 >> v2:172.21.15.121:3300/0 conn(0x7f819c0d4210 0x7f819c06b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:49.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.477+0000 7f81a1945640 1 --2- 172.21.15.32:0/525363311 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f819c15e9b0 0x7f819c0d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:49.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.477+0000 7f81a2146640 1 -- 172.21.15.32:0/525363311 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f819c15e9b0 msgr2=0x7f819c0d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:49.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.477+0000 7f81a2146640 1 --2- 172.21.15.32:0/525363311 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f819c15e9b0 0x7f819c0d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:49.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.477+0000 7f81a2146640 1 -- 172.21.15.32:0/525363311 >> v2:172.21.15.121:3300/0 conn(0x7f819c0d4210 msgr2=0x7f819c06b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:49.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.477+0000 7f81a2146640 1 --2- 172.21.15.32:0/525363311 >> v2:172.21.15.121:3300/0 conn(0x7f819c0d4210 0x7f819c06b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:49.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.477+0000 7f81a2146640 1 -- 172.21.15.32:0/525363311 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f819c0d3a10 con 0x7f819c0ba0e0 2024-04-24T23:08:49.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.477+0000 7f81a1144640 1 --2- 172.21.15.32:0/525363311 >> v2:172.21.15.121:3300/0 conn(0x7f819c0d4210 0x7f819c06b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:08:49.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.478+0000 7f81a2146640 1 --2- 172.21.15.32:0/525363311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f819c0ba0e0 0x7f819c06b650 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f819c06bc40 tx=0x7f818c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=306bfa70c05eeca2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:49.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.478+0000 7f81a0943640 1 -- 172.21.15.32:0/525363311 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818c036070 con 0x7f819c0ba0e0 2024-04-24T23:08:49.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.478+0000 7f81a0943640 1 -- 172.21.15.32:0/525363311 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f818c031600 con 0x7f819c0ba0e0 2024-04-24T23:08:49.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.478+0000 7f81a0943640 1 -- 172.21.15.32:0/525363311 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818c0319d0 con 0x7f819c0ba0e0 2024-04-24T23:08:49.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.478+0000 7f81a3ba9640 1 -- 172.21.15.32:0/525363311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f819c0ba0e0 msgr2=0x7f819c06b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:49.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.478+0000 7f81a3ba9640 1 --2- 172.21.15.32:0/525363311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f819c0ba0e0 0x7f819c06b650 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f819c06bc40 tx=0x7f818c02a6a0 comp rx=0 tx=0).stop 2024-04-24T23:08:49.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.478+0000 7f81a3ba9640 1 -- 172.21.15.32:0/525363311 shutdown_connections 2024-04-24T23:08:49.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.478+0000 7f81a3ba9640 1 --2- 172.21.15.32:0/525363311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f819c0ba0e0 0x7f819c06b650 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:49.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.478+0000 7f81a3ba9640 1 --2- 172.21.15.32:0/525363311 >> v2:172.21.15.121:3300/0 conn(0x7f819c0d4210 0x7f819c06b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:49.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.478+0000 7f81a3ba9640 1 --2- 172.21.15.32:0/525363311 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f819c15e9b0 0x7f819c0d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:49.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.478+0000 7f81a3ba9640 1 -- 172.21.15.32:0/525363311 >> 172.21.15.32:0/525363311 conn(0x7f819c0d11d0 msgr2=0x7f819c0d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:49.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.478+0000 7f81a3ba9640 1 -- 172.21.15.32:0/525363311 shutdown_connections 2024-04-24T23:08:49.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.478+0000 7f81a3ba9640 1 -- 172.21.15.32:0/525363311 wait complete. 2024-04-24T23:08:49.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.480+0000 7f81a3ba9640 1 Processor -- start 2024-04-24T23:08:49.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.480+0000 7f81a3ba9640 1 -- start start 2024-04-24T23:08:49.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.480+0000 7f81a3ba9640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f819c0ba0e0 0x7f819c1501f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:49.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.480+0000 7f81a3ba9640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f819c0d4210 0x7f819c0ccd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:49.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.480+0000 7f81a3ba9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f819c15e9b0 0x7f819c0cd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:49.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.480+0000 7f81a3ba9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f819c071730 con 0x7f819c15e9b0 2024-04-24T23:08:49.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.480+0000 7f81a3ba9640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f819c0d2400 con 0x7f819c0d4210 2024-04-24T23:08:49.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.480+0000 7f81a3ba9640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f819c0718b0 con 0x7f819c0ba0e0 2024-04-24T23:08:49.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.480+0000 7f81a2146640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f819c15e9b0 0x7f819c0cd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:49.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.480+0000 7f81a2146640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f819c15e9b0 0x7f819c0cd2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43866/0 (socket says 172.21.15.32:43866) 2024-04-24T23:08:49.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.480+0000 7f81a2146640 1 -- 172.21.15.32:0/2886159004 learned_addr learned my addr 172.21.15.32:0/2886159004 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:49.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.480+0000 7f81a1945640 1 --2- 172.21.15.32:0/2886159004 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f819c0ba0e0 0x7f819c1501f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:49.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.481+0000 7f81a1144640 1 --2- 172.21.15.32:0/2886159004 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f819c0d4210 0x7f819c0ccd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:49.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.481+0000 7f81a2146640 1 -- 172.21.15.32:0/2886159004 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f819c0ba0e0 msgr2=0x7f819c1501f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:49.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.481+0000 7f81a2146640 1 --2- 172.21.15.32:0/2886159004 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f819c0ba0e0 0x7f819c1501f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:49.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.481+0000 7f81a2146640 1 -- 172.21.15.32:0/2886159004 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f819c0d4210 msgr2=0x7f819c0ccd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:49.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.481+0000 7f81a2146640 1 --2- 172.21.15.32:0/2886159004 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f819c0d4210 0x7f819c0ccd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:49.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.481+0000 7f81a2146640 1 -- 172.21.15.32:0/2886159004 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f819c161c60 con 0x7f819c15e9b0 2024-04-24T23:08:49.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.481+0000 7f81a2146640 1 --2- 172.21.15.32:0/2886159004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f819c15e9b0 0x7f819c0cd2c0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f818c009f40 tx=0x7f818c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:49.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.481+0000 7f819a7fc640 1 -- 172.21.15.32:0/2886159004 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818c036050 con 0x7f819c15e9b0 2024-04-24T23:08:49.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.481+0000 7f819a7fc640 1 -- 172.21.15.32:0/2886159004 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f818c042070 con 0x7f819c15e9b0 2024-04-24T23:08:49.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.481+0000 7f819a7fc640 1 -- 172.21.15.32:0/2886159004 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818c009510 con 0x7f819c15e9b0 2024-04-24T23:08:49.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.481+0000 7f81a3ba9640 1 -- 172.21.15.32:0/2886159004 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f819c0c83b0 con 0x7f819c15e9b0 2024-04-24T23:08:49.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.481+0000 7f81a3ba9640 1 -- 172.21.15.32:0/2886159004 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f819c0c8930 con 0x7f819c15e9b0 2024-04-24T23:08:49.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.482+0000 7f81a3ba9640 1 -- 172.21.15.32:0/2886159004 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f819c15e210 con 0x7f819c15e9b0 2024-04-24T23:08:49.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.483+0000 7f819a7fc640 1 -- 172.21.15.32:0/2886159004 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f818c007bd0 con 0x7f819c15e9b0 2024-04-24T23:08:49.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.483+0000 7f819a7fc640 1 --2- 172.21.15.32:0/2886159004 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f818806be60 0x7f818806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:49.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.483+0000 7f819a7fc640 1 -- 172.21.15.32:0/2886159004 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 5795+0+0 (secure 0 0 0) 0x7f818c0abbc0 con 0x7f819c15e9b0 2024-04-24T23:08:49.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.483+0000 7f81a1945640 1 --2- 172.21.15.32:0/2886159004 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f818806be60 0x7f818806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:49.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.484+0000 7f819a7fc640 1 -- 172.21.15.32:0/2886159004 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=160}) -- 0x7f8188073a90 con 0x7f819c15e9b0 2024-04-24T23:08:49.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.484+0000 7f81a1945640 1 --2- 172.21.15.32:0/2886159004 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f818806be60 0x7f818806e320 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f819c15d480 tx=0x7f8194008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:49.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.488+0000 7f819a7fc640 1 -- 172.21.15.32:0/2886159004 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f819c15e210 con 0x7f819c15e9b0 2024-04-24T23:08:49.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:49.708+0000 7f81a3ba9640 1 -- 172.21.15.32:0/2886159004 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f819c0b7500 con 0x7f819c15e9b0 2024-04-24T23:08:50.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.357+0000 7f819a7fc640 1 -- 172.21.15.32:0/2886159004 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v67) ==== 104+0+0 (secure 0 0 0) 0x7f819c0b7500 con 0x7f819c15e9b0 2024-04-24T23:08:50.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.360+0000 7f81a3ba9640 1 -- 172.21.15.32:0/2886159004 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f818806be60 msgr2=0x7f818806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:50.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.360+0000 7f81a3ba9640 1 --2- 172.21.15.32:0/2886159004 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f818806be60 0x7f818806e320 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f819c15d480 tx=0x7f8194008040 comp rx=0 tx=0).stop 2024-04-24T23:08:50.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.360+0000 7f81a3ba9640 1 -- 172.21.15.32:0/2886159004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f819c15e9b0 msgr2=0x7f819c0cd2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:50.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.360+0000 7f81a3ba9640 1 --2- 172.21.15.32:0/2886159004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f819c15e9b0 0x7f819c0cd2c0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f818c009f40 tx=0x7f818c004a40 comp rx=0 tx=0).stop 2024-04-24T23:08:50.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.360+0000 7f81a3ba9640 1 -- 172.21.15.32:0/2886159004 shutdown_connections 2024-04-24T23:08:50.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.360+0000 7f81a3ba9640 1 --2- 172.21.15.32:0/2886159004 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f818806be60 0x7f818806e320 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:50.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.360+0000 7f81a3ba9640 1 --2- 172.21.15.32:0/2886159004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f819c15e9b0 0x7f819c0cd2c0 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:50.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.360+0000 7f81a3ba9640 1 --2- 172.21.15.32:0/2886159004 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f819c0d4210 0x7f819c0ccd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:50.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.360+0000 7f81a3ba9640 1 --2- 172.21.15.32:0/2886159004 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f819c0ba0e0 0x7f819c1501f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:50.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.360+0000 7f81a3ba9640 1 -- 172.21.15.32:0/2886159004 >> 172.21.15.32:0/2886159004 conn(0x7f819c0d11d0 msgr2=0x7f819c159a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:50.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.360+0000 7f81a3ba9640 1 -- 172.21.15.32:0/2886159004 shutdown_connections 2024-04-24T23:08:50.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.360+0000 7f81a3ba9640 1 -- 172.21.15.32:0/2886159004 wait complete. 2024-04-24T23:08:50.375 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-24T23:08:50.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.524+0000 7f1b1d47b640 1 Processor -- start 2024-04-24T23:08:50.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.524+0000 7f1b1d47b640 1 -- start start 2024-04-24T23:08:50.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.525+0000 7f1b1d47b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1b18161c30 0x7f1b18067980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:50.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.525+0000 7f1b1d47b640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b1806cdc0 0x7f1b180afa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:50.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.525+0000 7f1b1d47b640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b18071620 con 0x7f1b1806d200 2024-04-24T23:08:50.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.525+0000 7f1b1d47b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f1b180af5a0 con 0x7f1b18161c30 2024-04-24T23:08:50.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.525+0000 7f1b1d47b640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f1b180af270 con 0x7f1b1806cdc0 2024-04-24T23:08:50.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.525+0000 7f1b16ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1b18161c30 0x7f1b18067980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:50.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.525+0000 7f1b16ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1b18161c30 0x7f1b18067980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45328/0 (socket says 172.21.15.32:45328) 2024-04-24T23:08:50.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.525+0000 7f1b16ffd640 1 -- 172.21.15.32:0/2806929466 learned_addr learned my addr 172.21.15.32:0/2806929466 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:50.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.525+0000 7f1b0ffff640 1 --2- 172.21.15.32:0/2806929466 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b1806cdc0 0x7f1b180afa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:50.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.525+0000 7f1b16ffd640 1 -- 172.21.15.32:0/2806929466 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b1806cdc0 msgr2=0x7f1b180afa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:50.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.525+0000 7f1b16ffd640 1 --2- 172.21.15.32:0/2806929466 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b1806cdc0 0x7f1b180afa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:50.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.525+0000 7f1b16ffd640 1 -- 172.21.15.32:0/2806929466 >> v1:172.21.15.121:6789/0 conn(0x7f1b1806d200 legacy=0x7f1b180aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:50.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.525+0000 7f1b16ffd640 1 -- 172.21.15.32:0/2806929466 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b180718c0 con 0x7f1b18161c30 2024-04-24T23:08:50.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.525+0000 7f1b0ffff640 1 --2- 172.21.15.32:0/2806929466 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b1806cdc0 0x7f1b180afa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:08:50.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.526+0000 7f1b16ffd640 1 --2- 172.21.15.32:0/2806929466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b18161c30 0x7f1b18067980 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f1b08009870 tx=0x7f1b0802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d86d20bc6759de1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:50.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.526+0000 7f1b167fc640 1 -- 172.21.15.32:0/2806929466 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b08037070 con 0x7f1b18161c30 2024-04-24T23:08:50.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.526+0000 7f1b167fc640 1 -- 172.21.15.32:0/2806929466 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b08032440 con 0x7f1b18161c30 2024-04-24T23:08:50.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.526+0000 7f1b167fc640 1 -- 172.21.15.32:0/2806929466 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b08032810 con 0x7f1b18161c30 2024-04-24T23:08:50.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.527+0000 7f1b1d47b640 1 -- 172.21.15.32:0/2806929466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b18161c30 msgr2=0x7f1b18067980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:50.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.527+0000 7f1b1d47b640 1 --2- 172.21.15.32:0/2806929466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b18161c30 0x7f1b18067980 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f1b08009870 tx=0x7f1b0802a210 comp rx=0 tx=0).stop 2024-04-24T23:08:50.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.527+0000 7f1b1d47b640 1 -- 172.21.15.32:0/2806929466 shutdown_connections 2024-04-24T23:08:50.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.527+0000 7f1b1d47b640 1 --2- 172.21.15.32:0/2806929466 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b1806cdc0 0x7f1b180afa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:50.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.527+0000 7f1b1d47b640 1 --2- 172.21.15.32:0/2806929466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b18161c30 0x7f1b18067980 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:50.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.527+0000 7f1b1d47b640 1 -- 172.21.15.32:0/2806929466 >> 172.21.15.32:0/2806929466 conn(0x7f1b180d0a60 msgr2=0x7f1b180d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:50.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.527+0000 7f1b1d47b640 1 -- 172.21.15.32:0/2806929466 shutdown_connections 2024-04-24T23:08:50.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.527+0000 7f1b1d47b640 1 -- 172.21.15.32:0/2806929466 wait complete. 2024-04-24T23:08:50.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.528+0000 7f1b1d47b640 1 Processor -- start 2024-04-24T23:08:50.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.528+0000 7f1b1d47b640 1 -- start start 2024-04-24T23:08:50.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.529+0000 7f1b1d47b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b1806cdc0 0x7f1b181547d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:50.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.529+0000 7f1b1d47b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b1806d200 0x7f1b18159770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:50.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.529+0000 7f1b1d47b640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b18161c30 0x7f1b181529c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:50.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.529+0000 7f1b1d47b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1b18070d60 con 0x7f1b1806d200 2024-04-24T23:08:50.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.529+0000 7f1b1d47b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1b18162010 con 0x7f1b1806cdc0 2024-04-24T23:08:50.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.529+0000 7f1b1d47b640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f1b18070ee0 con 0x7f1b18161c30 2024-04-24T23:08:50.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.529+0000 7f1b0ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b1806d200 0x7f1b18159770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:50.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.529+0000 7f1b0ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b1806d200 0x7f1b18159770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45336/0 (socket says 172.21.15.32:45336) 2024-04-24T23:08:50.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.529+0000 7f1b0ffff640 1 -- 172.21.15.32:0/481044981 learned_addr learned my addr 172.21.15.32:0/481044981 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:50.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.529+0000 7f1b16ffd640 1 --2- 172.21.15.32:0/481044981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b1806cdc0 0x7f1b181547d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:50.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.529+0000 7f1b0ffff640 1 -- 172.21.15.32:0/481044981 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b18161c30 msgr2=0x7f1b181529c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:50.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.529+0000 7f1b177fe640 1 --2- 172.21.15.32:0/481044981 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b18161c30 0x7f1b181529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:50.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.529+0000 7f1b0ffff640 1 --2- 172.21.15.32:0/481044981 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b18161c30 0x7f1b181529c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:50.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.529+0000 7f1b0ffff640 1 -- 172.21.15.32:0/481044981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b1806cdc0 msgr2=0x7f1b181547d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:50.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.529+0000 7f1b0ffff640 1 --2- 172.21.15.32:0/481044981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b1806cdc0 0x7f1b181547d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:50.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.529+0000 7f1b0ffff640 1 -- 172.21.15.32:0/481044981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b18157070 con 0x7f1b1806d200 2024-04-24T23:08:50.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.529+0000 7f1b16ffd640 1 --2- 172.21.15.32:0/481044981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b1806cdc0 0x7f1b181547d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:08:50.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.529+0000 7f1b177fe640 1 --2- 172.21.15.32:0/481044981 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b18161c30 0x7f1b181529c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:08:50.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.529+0000 7f1b0ffff640 1 --2- 172.21.15.32:0/481044981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b1806d200 0x7f1b18159770 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f1b0000c820 tx=0x7f1b0000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:50.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.530+0000 7f1b0f7fe640 1 -- 172.21.15.32:0/481044981 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b00015070 con 0x7f1b1806d200 2024-04-24T23:08:50.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.530+0000 7f1b0f7fe640 1 -- 172.21.15.32:0/481044981 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b00007a30 con 0x7f1b1806d200 2024-04-24T23:08:50.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.530+0000 7f1b1d47b640 1 -- 172.21.15.32:0/481044981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b18154eb0 con 0x7f1b1806d200 2024-04-24T23:08:50.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.530+0000 7f1b0f7fe640 1 -- 172.21.15.32:0/481044981 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b00007e00 con 0x7f1b1806d200 2024-04-24T23:08:50.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.530+0000 7f1b1d47b640 1 -- 172.21.15.32:0/481044981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b18155470 con 0x7f1b1806d200 2024-04-24T23:08:50.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.531+0000 7f1b1d47b640 1 -- 172.21.15.32:0/481044981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b18068430 con 0x7f1b1806d200 2024-04-24T23:08:50.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.531+0000 7f1b0f7fe640 1 -- 172.21.15.32:0/481044981 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1b00018070 con 0x7f1b1806d200 2024-04-24T23:08:50.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.531+0000 7f1b0f7fe640 1 --2- 172.21.15.32:0/481044981 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1af806be30 0x7f1af806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:50.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.532+0000 7f1b0f7fe640 1 -- 172.21.15.32:0/481044981 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 5795+0+0 (secure 0 0 0) 0x7f1b0008ae20 con 0x7f1b1806d200 2024-04-24T23:08:50.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.532+0000 7f1b0f7fe640 1 -- 172.21.15.32:0/481044981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=160}) -- 0x7f1af8073a60 con 0x7f1b1806d200 2024-04-24T23:08:50.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.532+0000 7f1b16ffd640 1 --2- 172.21.15.32:0/481044981 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1af806be30 0x7f1af806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:50.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.532+0000 7f1b16ffd640 1 --2- 172.21.15.32:0/481044981 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1af806be30 0x7f1af806e2f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f1b08034000 tx=0x7f1b0802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:50.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.536+0000 7f1b0f7fe640 1 -- 172.21.15.32:0/481044981 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1b00059800 con 0x7f1b1806d200 2024-04-24T23:08:50.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:50.724+0000 7f1b1d47b640 1 -- 172.21.15.32:0/481044981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1b180b4d50 con 0x7f1b1806d200 2024-04-24T23:08:51.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.359+0000 7f1b0f7fe640 1 -- 172.21.15.32:0/481044981 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v160) ==== 181+0+0 (secure 0 0 0) 0x7f1b00059100 con 0x7f1b1806d200 2024-04-24T23:08:51.361 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2024-04-24T23:08:51.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.359+0000 7f1b0f7fe640 1 -- 172.21.15.32:0/481044981 <== mon.0 v2:172.21.15.32:3300/0 8 ==== osd_map(160..160 src has 1..160) ==== 296+0+0 (secure 0 0 0) 0x7f1b00089400 con 0x7f1b1806d200 2024-04-24T23:08:51.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.359+0000 7f1b0f7fe640 1 -- 172.21.15.32:0/481044981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=161}) -- 0x7f1af8071e90 con 0x7f1b1806d200 2024-04-24T23:08:51.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.365+0000 7f1b1d47b640 1 -- 172.21.15.32:0/481044981 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1af806be30 msgr2=0x7f1af806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:51.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.365+0000 7f1b1d47b640 1 --2- 172.21.15.32:0/481044981 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1af806be30 0x7f1af806e2f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f1b08034000 tx=0x7f1b0802e000 comp rx=0 tx=0).stop 2024-04-24T23:08:51.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.365+0000 7f1b1d47b640 1 -- 172.21.15.32:0/481044981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b1806d200 msgr2=0x7f1b18159770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:51.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.365+0000 7f1b1d47b640 1 --2- 172.21.15.32:0/481044981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b1806d200 0x7f1b18159770 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f1b0000c820 tx=0x7f1b0000cc20 comp rx=0 tx=0).stop 2024-04-24T23:08:51.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.366+0000 7f1b1d47b640 1 -- 172.21.15.32:0/481044981 shutdown_connections 2024-04-24T23:08:51.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.366+0000 7f1b1d47b640 1 --2- 172.21.15.32:0/481044981 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1af806be30 0x7f1af806e2f0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:51.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.366+0000 7f1b1d47b640 1 --2- 172.21.15.32:0/481044981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b1806d200 0x7f1b18159770 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:51.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.366+0000 7f1b1d47b640 1 --2- 172.21.15.32:0/481044981 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b18161c30 0x7f1b181529c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:51.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.366+0000 7f1b1d47b640 1 --2- 172.21.15.32:0/481044981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b1806cdc0 0x7f1b181547d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:51.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.366+0000 7f1b1d47b640 1 -- 172.21.15.32:0/481044981 >> 172.21.15.32:0/481044981 conn(0x7f1b180d0a60 msgr2=0x7f1b180b0db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:51.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.366+0000 7f1b1d47b640 1 -- 172.21.15.32:0/481044981 shutdown_connections 2024-04-24T23:08:51.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.366+0000 7f1b1d47b640 1 -- 172.21.15.32:0/481044981 wait complete. 2024-04-24T23:08:51.381 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-24T23:08:51.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.488+0000 7f90b5e12640 1 Processor -- start 2024-04-24T23:08:51.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.488+0000 7f90b5e12640 1 -- start start 2024-04-24T23:08:51.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.489+0000 7f90b5e12640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f90b00b9ff0 0x7f90b00d40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:51.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.489+0000 7f90b5e12640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f90b00b6450 0x7f90b00b68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:51.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.489+0000 7f90b5e12640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90b00d3000 con 0x7f90b015e770 2024-04-24T23:08:51.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.489+0000 7f90b5e12640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f90b00d32e0 con 0x7f90b00b9ff0 2024-04-24T23:08:51.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.489+0000 7f90b5e12640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f90b00d35c0 con 0x7f90b00b6450 2024-04-24T23:08:51.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.489+0000 7f90af7fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f90b015e770 0x7f90b00d2700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54922/0 (socket says 172.21.15.32:54922) 2024-04-24T23:08:51.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.489+0000 7f90af7fe640 1 -- 172.21.15.32:0/2977915070 learned_addr learned my addr 172.21.15.32:0/2977915070 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:08:51.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.490+0000 7f90affff640 1 --2- 172.21.15.32:0/2977915070 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f90b00b6450 0x7f90b00b68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:51.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.490+0000 7f90aeffd640 1 --2- 172.21.15.32:0/2977915070 >> v2:172.21.15.121:3300/0 conn(0x7f90b00b9ff0 0x7f90b00d40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:51.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.490+0000 7f90ae7fc640 1 -- 172.21.15.32:0/2977915070 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3224566164 0 0) 0x7f90b00d3000 con 0x7f90b015e770 2024-04-24T23:08:51.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.490+0000 7f90ae7fc640 1 -- 172.21.15.32:0/2977915070 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9098003520 con 0x7f90b015e770 2024-04-24T23:08:51.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.490+0000 7f90ae7fc640 1 -- 172.21.15.32:0/2977915070 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 588298427 0 0) 0x7f9098003520 con 0x7f90b015e770 2024-04-24T23:08:51.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.490+0000 7f90ae7fc640 1 -- 172.21.15.32:0/2977915070 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f90b00b6450 msgr2=0x7f90b00b68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:51.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.490+0000 7f90ae7fc640 1 --2- 172.21.15.32:0/2977915070 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f90b00b6450 0x7f90b00b68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:51.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.490+0000 7f90ae7fc640 1 -- 172.21.15.32:0/2977915070 >> v2:172.21.15.121:3300/0 conn(0x7f90b00b9ff0 msgr2=0x7f90b00d40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:51.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.490+0000 7f90ae7fc640 1 --2- 172.21.15.32:0/2977915070 >> v2:172.21.15.121:3300/0 conn(0x7f90b00b9ff0 0x7f90b00d40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:51.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.490+0000 7f90ae7fc640 1 -- 172.21.15.32:0/2977915070 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b00d38a0 con 0x7f90b015e770 2024-04-24T23:08:51.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.490+0000 7f90ae7fc640 1 -- 172.21.15.32:0/2977915070 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f90a0001670 con 0x7f90b015e770 2024-04-24T23:08:51.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.490+0000 7f90ae7fc640 1 -- 172.21.15.32:0/2977915070 >> v1:172.21.15.32:6789/0 conn(0x7f90b015e770 legacy=0x7f90b00d2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:51.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.490+0000 7f90ae7fc640 1 --2- 172.21.15.32:0/2977915070 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f90b00b6450 0x7f90980039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:51.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.490+0000 7f90ae7fc640 1 --2- 172.21.15.32:0/2977915070 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9098003f30 0x7f9098004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:51.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.490+0000 7f90ae7fc640 1 --2- 172.21.15.32:0/2977915070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9098004d40 0x7f90980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:51.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.490+0000 7f90ae7fc640 1 -- 172.21.15.32:0/2977915070 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f90b00d32e0 con 0x7f9098004d40 2024-04-24T23:08:51.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.490+0000 7f90ae7fc640 1 -- 172.21.15.32:0/2977915070 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f90b00d35c0 con 0x7f9098003f30 2024-04-24T23:08:51.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.490+0000 7f90ae7fc640 1 -- 172.21.15.32:0/2977915070 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f909800b9c0 con 0x7f90b00b6450 2024-04-24T23:08:51.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.490+0000 7f90af7fe640 1 --2- 172.21.15.32:0/2977915070 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f90b00b6450 0x7f90980039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:51.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.490+0000 7f90affff640 1 --2- 172.21.15.32:0/2977915070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9098004d40 0x7f90980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:51.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.490+0000 7f90aeffd640 1 --2- 172.21.15.32:0/2977915070 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9098003f30 0x7f9098004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:51.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.490+0000 7f90affff640 1 -- 172.21.15.32:0/2977915070 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f90b00b6450 msgr2=0x7f90980039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:51.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.490+0000 7f90affff640 1 --2- 172.21.15.32:0/2977915070 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f90b00b6450 0x7f90980039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:51.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.490+0000 7f90affff640 1 -- 172.21.15.32:0/2977915070 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9098003f30 msgr2=0x7f9098004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:51.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.490+0000 7f90affff640 1 --2- 172.21.15.32:0/2977915070 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9098003f30 0x7f9098004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:51.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.490+0000 7f90affff640 1 -- 172.21.15.32:0/2977915070 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9098003520 con 0x7f9098004d40 2024-04-24T23:08:51.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.491+0000 7f90affff640 1 --2- 172.21.15.32:0/2977915070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9098004d40 0x7f90980051c0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f90a8009fc0 tx=0x7f9098005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac3441d984d01333 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:51.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.491+0000 7f90ae7fc640 1 -- 172.21.15.32:0/2977915070 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a8009c10 con 0x7f9098004d40 2024-04-24T23:08:51.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.491+0000 7f90ae7fc640 1 -- 172.21.15.32:0/2977915070 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90a8036070 con 0x7f9098004d40 2024-04-24T23:08:51.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.491+0000 7f90ae7fc640 1 -- 172.21.15.32:0/2977915070 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a8031670 con 0x7f9098004d40 2024-04-24T23:08:51.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.492+0000 7f90b5e12640 1 -- 172.21.15.32:0/2977915070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9098004d40 msgr2=0x7f90980051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:51.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.492+0000 7f90b5e12640 1 --2- 172.21.15.32:0/2977915070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9098004d40 0x7f90980051c0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f90a8009fc0 tx=0x7f9098005830 comp rx=0 tx=0).stop 2024-04-24T23:08:51.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.492+0000 7f90affff640 1 -- 172.21.15.32:0/2977915070 reap_dead start 2024-04-24T23:08:51.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.492+0000 7f90b5e12640 1 -- 172.21.15.32:0/2977915070 shutdown_connections 2024-04-24T23:08:51.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.492+0000 7f90b5e12640 1 -- 172.21.15.32:0/2977915070 >> 172.21.15.32:0/2977915070 conn(0x7f90b0071910 msgr2=0x7f90b0071d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:51.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.492+0000 7f90b5e12640 1 -- 172.21.15.32:0/2977915070 shutdown_connections 2024-04-24T23:08:51.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.492+0000 7f90b5e12640 1 -- 172.21.15.32:0/2977915070 wait complete. 2024-04-24T23:08:51.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.493+0000 7f90b5e12640 1 Processor -- start 2024-04-24T23:08:51.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.493+0000 7f90b5e12640 1 -- start start 2024-04-24T23:08:51.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.494+0000 7f90b5e12640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90b01447d0 0x7f90b00cc710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:51.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.494+0000 7f90b5e12640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f90b00ccc50 0x7f90b0154910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:51.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.494+0000 7f90b5e12640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90b0154e50 0x7f90b01f70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:51.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.494+0000 7f90b5e12640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f90b015ecd0 con 0x7f90b0154e50 2024-04-24T23:08:51.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.494+0000 7f90b5e12640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f90b015eb50 con 0x7f90b01447d0 2024-04-24T23:08:51.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.494+0000 7f90b5e12640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f90b00b68d0 con 0x7f90b00ccc50 2024-04-24T23:08:51.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.494+0000 7f90affff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90b0154e50 0x7f90b01f70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:51.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.494+0000 7f90affff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90b0154e50 0x7f90b01f70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45354/0 (socket says 172.21.15.32:45354) 2024-04-24T23:08:51.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.494+0000 7f90affff640 1 -- 172.21.15.32:0/3966638034 learned_addr learned my addr 172.21.15.32:0/3966638034 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:51.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.494+0000 7f90af7fe640 1 --2- 172.21.15.32:0/3966638034 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90b01447d0 0x7f90b00cc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:51.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.494+0000 7f90aeffd640 1 --2- 172.21.15.32:0/3966638034 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f90b00ccc50 0x7f90b0154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:51.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.494+0000 7f90affff640 1 -- 172.21.15.32:0/3966638034 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f90b00ccc50 msgr2=0x7f90b0154910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:51.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.494+0000 7f90affff640 1 --2- 172.21.15.32:0/3966638034 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f90b00ccc50 0x7f90b0154910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:51.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.494+0000 7f90affff640 1 -- 172.21.15.32:0/3966638034 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90b01447d0 msgr2=0x7f90b00cc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:51.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.494+0000 7f90affff640 1 --2- 172.21.15.32:0/3966638034 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90b01447d0 0x7f90b00cc710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:51.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.494+0000 7f90affff640 1 -- 172.21.15.32:0/3966638034 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b00cd1d0 con 0x7f90b0154e50 2024-04-24T23:08:51.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.495+0000 7f90affff640 1 --2- 172.21.15.32:0/3966638034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90b0154e50 0x7f90b01f70d0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f90a800e230 tx=0x7f90a8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:51.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.495+0000 7f9097fff640 1 -- 172.21.15.32:0/3966638034 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a8009c10 con 0x7f90b0154e50 2024-04-24T23:08:51.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.495+0000 7f9097fff640 1 -- 172.21.15.32:0/3966638034 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90a8031dd0 con 0x7f90b0154e50 2024-04-24T23:08:51.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.495+0000 7f90b5e12640 1 -- 172.21.15.32:0/3966638034 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90b00cd4b0 con 0x7f90b0154e50 2024-04-24T23:08:51.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.495+0000 7f9097fff640 1 -- 172.21.15.32:0/3966638034 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a8003bb0 con 0x7f90b0154e50 2024-04-24T23:08:51.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.495+0000 7f90b5e12640 1 -- 172.21.15.32:0/3966638034 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90b00cda70 con 0x7f90b0154e50 2024-04-24T23:08:51.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.496+0000 7f90b5e12640 1 -- 172.21.15.32:0/3966638034 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90b0068430 con 0x7f90b0154e50 2024-04-24T23:08:51.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.496+0000 7f9097fff640 1 -- 172.21.15.32:0/3966638034 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f90a8005d10 con 0x7f90b0154e50 2024-04-24T23:08:51.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.497+0000 7f9097fff640 1 --2- 172.21.15.32:0/3966638034 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f909c06be30 0x7f909c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:51.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.497+0000 7f9097fff640 1 -- 172.21.15.32:0/3966638034 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 5357+0+0 (secure 0 0 0) 0x7f90a80aba20 con 0x7f90b0154e50 2024-04-24T23:08:51.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.497+0000 7f9097fff640 1 -- 172.21.15.32:0/3966638034 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=161}) -- 0x7f909c072c00 con 0x7f90b0154e50 2024-04-24T23:08:51.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.497+0000 7f90af7fe640 1 --2- 172.21.15.32:0/3966638034 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f909c06be30 0x7f909c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:51.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.498+0000 7f90af7fe640 1 --2- 172.21.15.32:0/3966638034 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f909c06be30 0x7f909c06e2f0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f90a0004840 tx=0x7f90a00078f0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:51.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.501+0000 7f9097fff640 1 -- 172.21.15.32:0/3966638034 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f90a8049060 con 0x7f90b0154e50 2024-04-24T23:08:51.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.683+0000 7f90b5e12640 1 -- 172.21.15.32:0/3966638034 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f90b00ce7e0 con 0x7f90b0154e50 2024-04-24T23:08:51.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.840+0000 7f9097fff640 1 -- 172.21.15.32:0/3966638034 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v161) ==== 169+0+0 (secure 0 0 0) 0x7f90a807a5f0 con 0x7f90b0154e50 2024-04-24T23:08:51.842 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2024-04-24T23:08:51.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.841+0000 7f9097fff640 1 -- 172.21.15.32:0/3966638034 <== mon.0 v2:172.21.15.32:3300/0 8 ==== osd_map(161..161 src has 1..161) ==== 687+0+0 (secure 0 0 0) 0x7f90a80abe00 con 0x7f90b0154e50 2024-04-24T23:08:51.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.841+0000 7f9097fff640 1 -- 172.21.15.32:0/3966638034 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=162}) -- 0x7f909c0719f0 con 0x7f90b0154e50 2024-04-24T23:08:51.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.844+0000 7f90b5e12640 1 -- 172.21.15.32:0/3966638034 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f909c06be30 msgr2=0x7f909c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:51.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.844+0000 7f90b5e12640 1 --2- 172.21.15.32:0/3966638034 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f909c06be30 0x7f909c06e2f0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f90a0004840 tx=0x7f90a00078f0 comp rx=0 tx=0).stop 2024-04-24T23:08:51.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.845+0000 7f90b5e12640 1 -- 172.21.15.32:0/3966638034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90b0154e50 msgr2=0x7f90b01f70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:51.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.845+0000 7f90b5e12640 1 --2- 172.21.15.32:0/3966638034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90b0154e50 0x7f90b01f70d0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f90a800e230 tx=0x7f90a8009750 comp rx=0 tx=0).stop 2024-04-24T23:08:51.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.846+0000 7f90b5e12640 1 -- 172.21.15.32:0/3966638034 shutdown_connections 2024-04-24T23:08:51.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.846+0000 7f90b5e12640 1 --2- 172.21.15.32:0/3966638034 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f909c06be30 0x7f909c06e2f0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:51.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.846+0000 7f90b5e12640 1 --2- 172.21.15.32:0/3966638034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90b0154e50 0x7f90b01f70d0 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:51.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.846+0000 7f90b5e12640 1 --2- 172.21.15.32:0/3966638034 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f90b00ccc50 0x7f90b0154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:51.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.846+0000 7f90b5e12640 1 --2- 172.21.15.32:0/3966638034 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90b01447d0 0x7f90b00cc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:51.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.846+0000 7f90b5e12640 1 -- 172.21.15.32:0/3966638034 >> 172.21.15.32:0/3966638034 conn(0x7f90b0071910 msgr2=0x7f90b00b8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:51.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.848+0000 7f90b5e12640 1 -- 172.21.15.32:0/3966638034 shutdown_connections 2024-04-24T23:08:51.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.848+0000 7f90b5e12640 1 -- 172.21.15.32:0/3966638034 wait complete. 2024-04-24T23:08:51.863 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-24T23:08:51.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.965+0000 7f782346b640 1 Processor -- start 2024-04-24T23:08:51.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.965+0000 7f782346b640 1 -- start start 2024-04-24T23:08:51.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.966+0000 7f782346b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f781c162e10 0x7f781c0ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:51.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.966+0000 7f782346b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f781c0cd270 con 0x7f781c0b5af0 2024-04-24T23:08:51.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.966+0000 7f782346b640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f781c071610 con 0x7f781c162320 2024-04-24T23:08:51.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.966+0000 7f782346b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f781c0b5660 con 0x7f781c162e10 2024-04-24T23:08:51.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.966+0000 7f7821a08640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f781c162e10 0x7f781c0ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:51.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.966+0000 7f7821a08640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f781c162e10 0x7f781c0ccc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45366/0 (socket says 172.21.15.32:45366) 2024-04-24T23:08:51.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.966+0000 7f7820a06640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f781c0b5af0 0x7f781c15ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54938/0 (socket says 172.21.15.32:54938) 2024-04-24T23:08:51.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.966+0000 7f7821a08640 1 -- 172.21.15.32:0/1661571137 learned_addr learned my addr 172.21.15.32:0/1661571137 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:51.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.966+0000 7f7821a08640 1 -- 172.21.15.32:0/1661571137 >> v1:172.21.15.121:6789/0 conn(0x7f781c162320 legacy=0x7f781c162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:51.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.966+0000 7f781bfff640 1 -- 172.21.15.32:0/1661571137 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2721668164 0 0) 0x7f781c0cd270 con 0x7f781c0b5af0 2024-04-24T23:08:51.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.967+0000 7f7821a08640 1 -- 172.21.15.32:0/1661571137 >> v1:172.21.15.32:6789/0 conn(0x7f781c0b5af0 legacy=0x7f781c15ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:51.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.967+0000 7f7821a08640 1 -- 172.21.15.32:0/1661571137 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f781c0718f0 con 0x7f781c162e10 2024-04-24T23:08:51.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.967+0000 7f7821a08640 1 --2- 172.21.15.32:0/1661571137 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f781c162e10 0x7f781c0ccc90 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f780c009e50 tx=0x7f780c02a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf82c252c53f76a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:51.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.967+0000 7f781bfff640 1 -- 172.21.15.32:0/1661571137 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780c036070 con 0x7f781c162e10 2024-04-24T23:08:51.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.967+0000 7f781bfff640 1 -- 172.21.15.32:0/1661571137 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f780c031600 con 0x7f781c162e10 2024-04-24T23:08:51.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.967+0000 7f781bfff640 1 -- 172.21.15.32:0/1661571137 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780c0319d0 con 0x7f781c162e10 2024-04-24T23:08:51.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.968+0000 7f782346b640 1 -- 172.21.15.32:0/1661571137 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f781c162e10 msgr2=0x7f781c0ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:51.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.968+0000 7f782346b640 1 --2- 172.21.15.32:0/1661571137 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f781c162e10 0x7f781c0ccc90 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f780c009e50 tx=0x7f780c02a6d0 comp rx=0 tx=0).stop 2024-04-24T23:08:51.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.968+0000 7f782346b640 1 -- 172.21.15.32:0/1661571137 shutdown_connections 2024-04-24T23:08:51.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.968+0000 7f782346b640 1 --2- 172.21.15.32:0/1661571137 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f781c162e10 0x7f781c0ccc90 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:51.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.968+0000 7f782346b640 1 -- 172.21.15.32:0/1661571137 >> 172.21.15.32:0/1661571137 conn(0x7f781c0d0ae0 msgr2=0x7f781c0d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:51.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.968+0000 7f782346b640 1 -- 172.21.15.32:0/1661571137 shutdown_connections 2024-04-24T23:08:51.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.968+0000 7f782346b640 1 -- 172.21.15.32:0/1661571137 wait complete. 2024-04-24T23:08:51.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.969+0000 7f782346b640 1 Processor -- start 2024-04-24T23:08:51.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.969+0000 7f782346b640 1 -- start start 2024-04-24T23:08:51.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.970+0000 7f782346b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f781c0b5af0 0x7f781c06ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:51.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.970+0000 7f782346b640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f781c162320 0x7f781c0ce450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:51.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.970+0000 7f782346b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f781c162e10 0x7f781c15b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:51.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.970+0000 7f782346b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f781c071bd0 con 0x7f781c0b5af0 2024-04-24T23:08:51.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.970+0000 7f782346b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f781c067950 con 0x7f781c162e10 2024-04-24T23:08:51.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.970+0000 7f782346b640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f781c071d50 con 0x7f781c162320 2024-04-24T23:08:51.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.970+0000 7f7821207640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f781c0b5af0 0x7f781c06ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:51.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.970+0000 7f7821207640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f781c0b5af0 0x7f781c06ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45372/0 (socket says 172.21.15.32:45372) 2024-04-24T23:08:51.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.970+0000 7f7821207640 1 -- 172.21.15.32:0/2267821607 learned_addr learned my addr 172.21.15.32:0/2267821607 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:51.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.970+0000 7f7821a08640 1 --2- 172.21.15.32:0/2267821607 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f781c162e10 0x7f781c15b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:51.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.970+0000 7f7820a06640 1 --2- 172.21.15.32:0/2267821607 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f781c162320 0x7f781c0ce450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:51.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.970+0000 7f7821207640 1 -- 172.21.15.32:0/2267821607 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f781c162320 msgr2=0x7f781c0ce450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:51.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.970+0000 7f7821207640 1 --2- 172.21.15.32:0/2267821607 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f781c162320 0x7f781c0ce450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:51.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.970+0000 7f7821207640 1 -- 172.21.15.32:0/2267821607 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f781c162e10 msgr2=0x7f781c15b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:51.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.970+0000 7f7821207640 1 --2- 172.21.15.32:0/2267821607 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f781c162e10 0x7f781c15b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:51.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.970+0000 7f7821207640 1 -- 172.21.15.32:0/2267821607 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f781c06efb0 con 0x7f781c0b5af0 2024-04-24T23:08:51.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.970+0000 7f7821a08640 1 --2- 172.21.15.32:0/2267821607 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f781c162e10 0x7f781c15b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:08:51.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.970+0000 7f7820a06640 1 --2- 172.21.15.32:0/2267821607 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f781c162320 0x7f781c0ce450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:08:51.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.971+0000 7f7821207640 1 --2- 172.21.15.32:0/2267821607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f781c0b5af0 0x7f781c06ea70 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f781400c460 tx=0x7f781400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:51.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.971+0000 7f7819ffb640 1 -- 172.21.15.32:0/2267821607 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7814016020 con 0x7f781c0b5af0 2024-04-24T23:08:51.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.971+0000 7f7819ffb640 1 -- 172.21.15.32:0/2267821607 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7814012870 con 0x7f781c0b5af0 2024-04-24T23:08:51.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.971+0000 7f7819ffb640 1 -- 172.21.15.32:0/2267821607 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7814012c60 con 0x7f781c0b5af0 2024-04-24T23:08:51.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.971+0000 7f782346b640 1 -- 172.21.15.32:0/2267821607 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f781c0cd690 con 0x7f781c0b5af0 2024-04-24T23:08:51.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.971+0000 7f782346b640 1 -- 172.21.15.32:0/2267821607 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f781c0cdc50 con 0x7f781c0b5af0 2024-04-24T23:08:51.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.972+0000 7f782346b640 1 -- 172.21.15.32:0/2267821607 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f781c068430 con 0x7f781c0b5af0 2024-04-24T23:08:51.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.972+0000 7f7819ffb640 1 -- 172.21.15.32:0/2267821607 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7814019020 con 0x7f781c0b5af0 2024-04-24T23:08:51.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.973+0000 7f7819ffb640 1 --2- 172.21.15.32:0/2267821607 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f780806be30 0x7f780806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:51.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.973+0000 7f7819ffb640 1 -- 172.21.15.32:0/2267821607 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 4943+0+0 (secure 0 0 0) 0x7f781408a580 con 0x7f781c0b5af0 2024-04-24T23:08:51.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.973+0000 7f7820a06640 1 --2- 172.21.15.32:0/2267821607 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f780806be30 0x7f780806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:51.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.973+0000 7f7820a06640 1 --2- 172.21.15.32:0/2267821607 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f780806be30 0x7f780806e2f0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f781c15ba90 tx=0x7f7810009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:51.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:51.977+0000 7f7819ffb640 1 -- 172.21.15.32:0/2267821607 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7814020070 con 0x7f781c0b5af0 2024-04-24T23:08:52.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.161+0000 7f782346b640 1 -- 172.21.15.32:0/2267821607 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f781c067730 con 0x7f781c0b5af0 2024-04-24T23:08:52.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.164+0000 7f7819ffb640 1 -- 172.21.15.32:0/2267821607 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) ==== 104+0+0 (secure 0 0 0) 0x7f78140592c0 con 0x7f781c0b5af0 2024-04-24T23:08:52.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.166+0000 7f782346b640 1 -- 172.21.15.32:0/2267821607 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f780806be30 msgr2=0x7f780806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:52.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.166+0000 7f782346b640 1 --2- 172.21.15.32:0/2267821607 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f780806be30 0x7f780806e2f0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f781c15ba90 tx=0x7f7810009040 comp rx=0 tx=0).stop 2024-04-24T23:08:52.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.166+0000 7f782346b640 1 -- 172.21.15.32:0/2267821607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f781c0b5af0 msgr2=0x7f781c06ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:52.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.166+0000 7f782346b640 1 --2- 172.21.15.32:0/2267821607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f781c0b5af0 0x7f781c06ea70 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f781400c460 tx=0x7f781400c920 comp rx=0 tx=0).stop 2024-04-24T23:08:52.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.166+0000 7f782346b640 1 -- 172.21.15.32:0/2267821607 shutdown_connections 2024-04-24T23:08:52.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.166+0000 7f782346b640 1 --2- 172.21.15.32:0/2267821607 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f780806be30 0x7f780806e2f0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:52.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.166+0000 7f782346b640 1 --2- 172.21.15.32:0/2267821607 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f781c162e10 0x7f781c15b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:52.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.166+0000 7f782346b640 1 --2- 172.21.15.32:0/2267821607 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f781c162320 0x7f781c0ce450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:52.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.166+0000 7f782346b640 1 --2- 172.21.15.32:0/2267821607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f781c0b5af0 0x7f781c06ea70 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:52.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.166+0000 7f782346b640 1 -- 172.21.15.32:0/2267821607 >> 172.21.15.32:0/2267821607 conn(0x7f781c0d0ae0 msgr2=0x7f781c0b0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:52.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.166+0000 7f782346b640 1 -- 172.21.15.32:0/2267821607 shutdown_connections 2024-04-24T23:08:52.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.166+0000 7f782346b640 1 -- 172.21.15.32:0/2267821607 wait complete. 2024-04-24T23:08:52.179 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose' 2024-04-24T23:08:52.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.279+0000 7fe61d36a640 1 Processor -- start 2024-04-24T23:08:52.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.279+0000 7fe61d36a640 1 -- start start 2024-04-24T23:08:52.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.280+0000 7fe61d36a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe61815e9b0 0x7fe6180d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:52.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.280+0000 7fe61d36a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6180d4210 0x7fe61806b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:52.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.280+0000 7fe61d36a640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6180d3170 con 0x7fe6180ba0e0 2024-04-24T23:08:52.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.280+0000 7fe61d36a640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fe6180b8900 con 0x7fe61815e9b0 2024-04-24T23:08:52.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.280+0000 7fe61d36a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe6180d3450 con 0x7fe6180d4210 2024-04-24T23:08:52.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.280+0000 7fe616ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe61815e9b0 0x7fe6180d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:52.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.280+0000 7fe6177fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe6180ba0e0 0x7fe618071730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54950/0 (socket says 172.21.15.32:54950) 2024-04-24T23:08:52.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.280+0000 7fe616ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe61815e9b0 0x7fe6180d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45388/0 (socket says 172.21.15.32:45388) 2024-04-24T23:08:52.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.280+0000 7fe6177fe640 1 -- 172.21.15.32:0/3512443222 learned_addr learned my addr 172.21.15.32:0/3512443222 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:08:52.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.280+0000 7fe6167fc640 1 --2- 172.21.15.32:0/3512443222 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6180d4210 0x7fe61806b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:52.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.281+0000 7fe616ffd640 1 -- 172.21.15.32:0/3512443222 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6180d4210 msgr2=0x7fe61806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:52.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.281+0000 7fe616ffd640 1 --2- 172.21.15.32:0/3512443222 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6180d4210 0x7fe61806b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:52.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.281+0000 7fe616ffd640 1 -- 172.21.15.32:0/3512443222 >> v1:172.21.15.32:6789/0 conn(0x7fe6180ba0e0 legacy=0x7fe618071730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:52.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.281+0000 7fe615ffb640 1 -- 172.21.15.32:0/3512443222 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1674165349 0 0) 0x7fe6180d3170 con 0x7fe6180ba0e0 2024-04-24T23:08:52.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.281+0000 7fe616ffd640 1 -- 172.21.15.32:0/3512443222 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6180d3730 con 0x7fe61815e9b0 2024-04-24T23:08:52.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.281+0000 7fe616ffd640 1 --2- 172.21.15.32:0/3512443222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe61815e9b0 0x7fe6180d2b90 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fe6100097a0 tx=0x7fe61002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c030c989053eb23b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:52.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.281+0000 7fe615ffb640 1 -- 172.21.15.32:0/3512443222 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe610037070 con 0x7fe61815e9b0 2024-04-24T23:08:52.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.281+0000 7fe615ffb640 1 -- 172.21.15.32:0/3512443222 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe610032440 con 0x7fe61815e9b0 2024-04-24T23:08:52.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.281+0000 7fe615ffb640 1 -- 172.21.15.32:0/3512443222 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe610032810 con 0x7fe61815e9b0 2024-04-24T23:08:52.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.282+0000 7fe61d36a640 1 -- 172.21.15.32:0/3512443222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe61815e9b0 msgr2=0x7fe6180d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:52.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.282+0000 7fe61d36a640 1 --2- 172.21.15.32:0/3512443222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe61815e9b0 0x7fe6180d2b90 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fe6100097a0 tx=0x7fe61002a180 comp rx=0 tx=0).stop 2024-04-24T23:08:52.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.282+0000 7fe61d36a640 1 -- 172.21.15.32:0/3512443222 shutdown_connections 2024-04-24T23:08:52.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.282+0000 7fe61d36a640 1 --2- 172.21.15.32:0/3512443222 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6180d4210 0x7fe61806b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:52.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.282+0000 7fe61d36a640 1 --2- 172.21.15.32:0/3512443222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe61815e9b0 0x7fe6180d2b90 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:52.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.282+0000 7fe61d36a640 1 -- 172.21.15.32:0/3512443222 >> 172.21.15.32:0/3512443222 conn(0x7fe6180d11d0 msgr2=0x7fe6180d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:52.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.282+0000 7fe61d36a640 1 -- 172.21.15.32:0/3512443222 shutdown_connections 2024-04-24T23:08:52.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.282+0000 7fe61d36a640 1 -- 172.21.15.32:0/3512443222 wait complete. 2024-04-24T23:08:52.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.283+0000 7fe61d36a640 1 Processor -- start 2024-04-24T23:08:52.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.283+0000 7fe61d36a640 1 -- start start 2024-04-24T23:08:52.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.284+0000 7fe61d36a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6180ba0e0 0x7fe618153f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:52.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.284+0000 7fe61d36a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6180d4210 0x7fe6181f4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:52.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.284+0000 7fe61d36a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe61815e9b0 0x7fe6181f45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:52.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.284+0000 7fe61d36a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe6180d2400 con 0x7fe6180d4210 2024-04-24T23:08:52.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.284+0000 7fe61d36a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe6180d3cf0 con 0x7fe6180ba0e0 2024-04-24T23:08:52.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.284+0000 7fe61d36a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe61806b650 con 0x7fe61815e9b0 2024-04-24T23:08:52.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.284+0000 7fe6167fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6180d4210 0x7fe6181f4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:52.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.284+0000 7fe6167fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6180d4210 0x7fe6181f4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45402/0 (socket says 172.21.15.32:45402) 2024-04-24T23:08:52.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.284+0000 7fe6167fc640 1 -- 172.21.15.32:0/1970184582 learned_addr learned my addr 172.21.15.32:0/1970184582 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:52.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.284+0000 7fe6177fe640 1 --2- 172.21.15.32:0/1970184582 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe61815e9b0 0x7fe6181f45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:52.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.284+0000 7fe616ffd640 1 --2- 172.21.15.32:0/1970184582 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6180ba0e0 0x7fe618153f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:52.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.284+0000 7fe6167fc640 1 -- 172.21.15.32:0/1970184582 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe61815e9b0 msgr2=0x7fe6181f45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:52.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.284+0000 7fe6167fc640 1 --2- 172.21.15.32:0/1970184582 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe61815e9b0 0x7fe6181f45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:52.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.284+0000 7fe6167fc640 1 -- 172.21.15.32:0/1970184582 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6180ba0e0 msgr2=0x7fe618153f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:52.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.284+0000 7fe6167fc640 1 --2- 172.21.15.32:0/1970184582 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6180ba0e0 0x7fe618153f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:52.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.284+0000 7fe6167fc640 1 -- 172.21.15.32:0/1970184582 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe618160aa0 con 0x7fe6180d4210 2024-04-24T23:08:52.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.285+0000 7fe6167fc640 1 --2- 172.21.15.32:0/1970184582 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6180d4210 0x7fe6181f4060 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fe60c00d800 tx=0x7fe60c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:52.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.285+0000 7fe5ff7fe640 1 -- 172.21.15.32:0/1970184582 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60c00b6c0 con 0x7fe6180d4210 2024-04-24T23:08:52.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.285+0000 7fe5ff7fe640 1 -- 172.21.15.32:0/1970184582 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe60c019070 con 0x7fe6180d4210 2024-04-24T23:08:52.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.285+0000 7fe61d36a640 1 -- 172.21.15.32:0/1970184582 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe618160d80 con 0x7fe6180d4210 2024-04-24T23:08:52.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.285+0000 7fe5ff7fe640 1 -- 172.21.15.32:0/1970184582 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60c00ba80 con 0x7fe6180d4210 2024-04-24T23:08:52.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.285+0000 7fe61d36a640 1 -- 172.21.15.32:0/1970184582 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe618161340 con 0x7fe6180d4210 2024-04-24T23:08:52.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.286+0000 7fe61d36a640 1 -- 172.21.15.32:0/1970184582 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe618068430 con 0x7fe6180d4210 2024-04-24T23:08:52.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.286+0000 7fe5ff7fe640 1 -- 172.21.15.32:0/1970184582 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe60c00bc20 con 0x7fe6180d4210 2024-04-24T23:08:52.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.287+0000 7fe5ff7fe640 1 --2- 172.21.15.32:0/1970184582 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe60406be30 0x7fe60406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:52.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.287+0000 7fe5ff7fe640 1 -- 172.21.15.32:0/1970184582 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 4943+0+0 (secure 0 0 0) 0x7fe60c08b030 con 0x7fe6180d4210 2024-04-24T23:08:52.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.287+0000 7fe616ffd640 1 --2- 172.21.15.32:0/1970184582 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe60406be30 0x7fe60406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:52.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.287+0000 7fe616ffd640 1 --2- 172.21.15.32:0/1970184582 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe60406be30 0x7fe60406e2f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fe610034000 tx=0x7fe61002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:52.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.291+0000 7fe5ff7fe640 1 -- 172.21.15.32:0/1970184582 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe60c059ce0 con 0x7fe6180d4210 2024-04-24T23:08:52.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:52.548+0000 7fe61d36a640 1 -- 172.21.15.32:0/1970184582 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]} v 0) -- 0x7fe6180cf350 con 0x7fe6180d4210 2024-04-24T23:08:53.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.367+0000 7fe5ff7fe640 1 -- 172.21.15.32:0/1970184582 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]}]=0 v474) ==== 132+0+0 (secure 0 0 0) 0x7fe60c0595e0 con 0x7fe6180d4210 2024-04-24T23:08:53.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.371+0000 7fe61d36a640 1 -- 172.21.15.32:0/1970184582 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe60406be30 msgr2=0x7fe60406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:53.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.371+0000 7fe61d36a640 1 --2- 172.21.15.32:0/1970184582 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe60406be30 0x7fe60406e2f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fe610034000 tx=0x7fe61002e000 comp rx=0 tx=0).stop 2024-04-24T23:08:53.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.371+0000 7fe61d36a640 1 -- 172.21.15.32:0/1970184582 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6180d4210 msgr2=0x7fe6181f4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:53.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.371+0000 7fe61d36a640 1 --2- 172.21.15.32:0/1970184582 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6180d4210 0x7fe6181f4060 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fe60c00d800 tx=0x7fe60c00dcc0 comp rx=0 tx=0).stop 2024-04-24T23:08:53.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.371+0000 7fe61d36a640 1 -- 172.21.15.32:0/1970184582 shutdown_connections 2024-04-24T23:08:53.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.371+0000 7fe61d36a640 1 --2- 172.21.15.32:0/1970184582 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe60406be30 0x7fe60406e2f0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:53.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.371+0000 7fe61d36a640 1 --2- 172.21.15.32:0/1970184582 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6180d4210 0x7fe6181f4060 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:53.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.371+0000 7fe61d36a640 1 --2- 172.21.15.32:0/1970184582 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe61815e9b0 0x7fe6181f45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:53.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.371+0000 7fe61d36a640 1 --2- 172.21.15.32:0/1970184582 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6180ba0e0 0x7fe618153f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:53.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.371+0000 7fe61d36a640 1 -- 172.21.15.32:0/1970184582 >> 172.21.15.32:0/1970184582 conn(0x7fe6180d11d0 msgr2=0x7fe6180cfc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:53.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.371+0000 7fe61d36a640 1 -- 172.21.15.32:0/1970184582 shutdown_connections 2024-04-24T23:08:53.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.371+0000 7fe61d36a640 1 -- 172.21.15.32:0/1970184582 wait complete. 2024-04-24T23:08:53.386 INFO:tasks.cephfs_test_runner:test_full_fclose (tasks.cephfs.test_full.TestQuotaFull) ... ok 2024-04-24T23:08:53.386 INFO:tasks.cephfs_test_runner:Starting test: test_full_fsync (tasks.cephfs.test_full.TestQuotaFull) That when the full flag is encountered during asynchronous 2024-04-24T23:08:53.387 INFO:tasks.cephfs_test_runner:test_full_fsync (tasks.cephfs.test_full.TestQuotaFull) 2024-04-24T23:08:53.387 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:08:53.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.493+0000 7f4730fb0640 1 Processor -- start 2024-04-24T23:08:53.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.493+0000 7f4730fb0640 1 -- start start 2024-04-24T23:08:53.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.494+0000 7f4730fb0640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f472c0b44d0 0x7f472c0b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:53.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.494+0000 7f4730fb0640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f472c0af890 0x7f472c15cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:53.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.494+0000 7f4730fb0640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f472c0b6430 con 0x7f472c06c5f0 2024-04-24T23:08:53.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.494+0000 7f4730fb0640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f472c156f80 con 0x7f472c0b44d0 2024-04-24T23:08:53.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.494+0000 7f4730fb0640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f472c0b6150 con 0x7f472c0af890 2024-04-24T23:08:53.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.494+0000 7f4723fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f472c0b44d0 0x7f472c0b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:53.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.494+0000 7f4723fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f472c0b44d0 0x7f472c0b5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45408/0 (socket says 172.21.15.32:45408) 2024-04-24T23:08:53.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.494+0000 7f4723fff640 1 -- 172.21.15.32:0/2957323252 learned_addr learned my addr 172.21.15.32:0/2957323252 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:53.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.494+0000 7f472ad9d640 1 --2- 172.21.15.32:0/2957323252 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f472c0af890 0x7f472c15cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:53.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.494+0000 7f4723fff640 1 -- 172.21.15.32:0/2957323252 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f472c0af890 msgr2=0x7f472c15cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:53.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.494+0000 7f4723fff640 1 --2- 172.21.15.32:0/2957323252 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f472c0af890 0x7f472c15cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:53.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.494+0000 7f4723fff640 1 -- 172.21.15.32:0/2957323252 >> v1:172.21.15.121:6789/0 conn(0x7f472c06c5f0 legacy=0x7f472c06c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:53.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.494+0000 7f4723fff640 1 -- 172.21.15.32:0/2957323252 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f472c0b39f0 con 0x7f472c0b44d0 2024-04-24T23:08:53.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.495+0000 7f4723fff640 1 --2- 172.21.15.32:0/2957323252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f472c0b44d0 0x7f472c0b5890 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f4714009870 tx=0x7f471402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a0d126138319e3f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:53.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.495+0000 7f4729d9b640 1 -- 172.21.15.32:0/2957323252 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4714037070 con 0x7f472c0b44d0 2024-04-24T23:08:53.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.495+0000 7f4729d9b640 1 -- 172.21.15.32:0/2957323252 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4714032440 con 0x7f472c0b44d0 2024-04-24T23:08:53.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.495+0000 7f4729d9b640 1 -- 172.21.15.32:0/2957323252 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4714032810 con 0x7f472c0b44d0 2024-04-24T23:08:53.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.496+0000 7f4730fb0640 1 -- 172.21.15.32:0/2957323252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f472c0b44d0 msgr2=0x7f472c0b5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:53.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.496+0000 7f4730fb0640 1 --2- 172.21.15.32:0/2957323252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f472c0b44d0 0x7f472c0b5890 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f4714009870 tx=0x7f471402a1c0 comp rx=0 tx=0).stop 2024-04-24T23:08:53.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.496+0000 7f4730fb0640 1 -- 172.21.15.32:0/2957323252 shutdown_connections 2024-04-24T23:08:53.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.496+0000 7f4730fb0640 1 --2- 172.21.15.32:0/2957323252 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f472c0af890 0x7f472c15cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:53.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.496+0000 7f4730fb0640 1 --2- 172.21.15.32:0/2957323252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f472c0b44d0 0x7f472c0b5890 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:53.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.496+0000 7f4730fb0640 1 -- 172.21.15.32:0/2957323252 >> 172.21.15.32:0/2957323252 conn(0x7f472c06b110 msgr2=0x7f472c06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:53.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.496+0000 7f4730fb0640 1 -- 172.21.15.32:0/2957323252 shutdown_connections 2024-04-24T23:08:53.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.496+0000 7f4730fb0640 1 -- 172.21.15.32:0/2957323252 wait complete. 2024-04-24T23:08:53.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.497+0000 7f4730fb0640 1 Processor -- start 2024-04-24T23:08:53.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.497+0000 7f4730fb0640 1 -- start start 2024-04-24T23:08:53.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.498+0000 7f4730fb0640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f472c06c5f0 0x7f472c0adce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:53.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.498+0000 7f4730fb0640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f472c0af890 0x7f472c0abcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:53.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.498+0000 7f4730fb0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f472c0b44d0 0x7f472c0ac1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:53.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.498+0000 7f4730fb0640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f472c0b5dd0 con 0x7f472c0b44d0 2024-04-24T23:08:53.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.498+0000 7f4730fb0640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f472c0b3fb0 con 0x7f472c06c5f0 2024-04-24T23:08:53.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.498+0000 7f4730fb0640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f472c15d000 con 0x7f472c0af890 2024-04-24T23:08:53.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.498+0000 7f472ad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f472c0b44d0 0x7f472c0ac1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:53.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.498+0000 7f472ad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f472c0b44d0 0x7f472c0ac1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45412/0 (socket says 172.21.15.32:45412) 2024-04-24T23:08:53.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.498+0000 7f472ad9d640 1 -- 172.21.15.32:0/1374179423 learned_addr learned my addr 172.21.15.32:0/1374179423 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:53.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.498+0000 7f472a59c640 1 --2- 172.21.15.32:0/1374179423 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f472c06c5f0 0x7f472c0adce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:53.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.498+0000 7f4723fff640 1 --2- 172.21.15.32:0/1374179423 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f472c0af890 0x7f472c0abcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:53.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.498+0000 7f472ad9d640 1 -- 172.21.15.32:0/1374179423 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f472c0af890 msgr2=0x7f472c0abcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:53.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.498+0000 7f472ad9d640 1 --2- 172.21.15.32:0/1374179423 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f472c0af890 0x7f472c0abcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:53.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.498+0000 7f472ad9d640 1 -- 172.21.15.32:0/1374179423 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f472c06c5f0 msgr2=0x7f472c0adce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:53.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.498+0000 7f472ad9d640 1 --2- 172.21.15.32:0/1374179423 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f472c06c5f0 0x7f472c0adce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:53.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.499+0000 7f472ad9d640 1 -- 172.21.15.32:0/1374179423 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f472c0ad930 con 0x7f472c0b44d0 2024-04-24T23:08:53.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.499+0000 7f472ad9d640 1 --2- 172.21.15.32:0/1374179423 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f472c0b44d0 0x7f472c0ac1f0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f4724007e70 tx=0x7f472400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:53.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.499+0000 7f4722ffd640 1 -- 172.21.15.32:0/1374179423 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4724014620 con 0x7f472c0b44d0 2024-04-24T23:08:53.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.499+0000 7f4722ffd640 1 -- 172.21.15.32:0/1374179423 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4724019070 con 0x7f472c0b44d0 2024-04-24T23:08:53.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.499+0000 7f4730fb0640 1 -- 172.21.15.32:0/1374179423 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f472c1e80a0 con 0x7f472c0b44d0 2024-04-24T23:08:53.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.499+0000 7f4722ffd640 1 -- 172.21.15.32:0/1374179423 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4724014a20 con 0x7f472c0b44d0 2024-04-24T23:08:53.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.499+0000 7f4730fb0640 1 -- 172.21.15.32:0/1374179423 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f472c0acb70 con 0x7f472c0b44d0 2024-04-24T23:08:53.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.500+0000 7f4730fb0640 1 -- 172.21.15.32:0/1374179423 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f472c068430 con 0x7f472c0b44d0 2024-04-24T23:08:53.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.500+0000 7f4722ffd640 1 -- 172.21.15.32:0/1374179423 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4724014bc0 con 0x7f472c0b44d0 2024-04-24T23:08:53.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.501+0000 7f4722ffd640 1 --2- 172.21.15.32:0/1374179423 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f470c06be30 0x7f470c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:53.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.501+0000 7f4722ffd640 1 -- 172.21.15.32:0/1374179423 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 4943+0+0 (secure 0 0 0) 0x7f472408b3f0 con 0x7f472c0b44d0 2024-04-24T23:08:53.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.501+0000 7f472a59c640 1 --2- 172.21.15.32:0/1374179423 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f470c06be30 0x7f470c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:53.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.502+0000 7f472a59c640 1 --2- 172.21.15.32:0/1374179423 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f470c06be30 0x7f470c06e2f0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f471c000f60 tx=0x7f471c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:53.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.505+0000 7f4722ffd640 1 -- 172.21.15.32:0/1374179423 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f472c1e80a0 con 0x7f472c0b44d0 2024-04-24T23:08:53.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.689+0000 7f4730fb0640 1 -- 172.21.15.32:0/1374179423 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f472c0b5670 con 0x7f472c0b44d0 2024-04-24T23:08:53.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.690+0000 7f4722ffd640 1 -- 172.21.15.32:0/1374179423 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v161) ==== 74+0+13041 (secure 0 0 0) 0x7f47240599b0 con 0x7f472c0b44d0 2024-04-24T23:08:53.691 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:08:53.692 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":161,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:08:51.840276+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6835/984603360":"2024-04-25T23:08:49.355529+0000","172.21.15.32:6834/984603360":"2024-04-25T23:08:49.355529+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:08:53.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.692+0000 7f4730fb0640 1 -- 172.21.15.32:0/1374179423 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f470c06be30 msgr2=0x7f470c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:53.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.692+0000 7f4730fb0640 1 --2- 172.21.15.32:0/1374179423 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f470c06be30 0x7f470c06e2f0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f471c000f60 tx=0x7f471c009040 comp rx=0 tx=0).stop 2024-04-24T23:08:53.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.692+0000 7f4730fb0640 1 -- 172.21.15.32:0/1374179423 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f472c0b44d0 msgr2=0x7f472c0ac1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:53.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.692+0000 7f4730fb0640 1 --2- 172.21.15.32:0/1374179423 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f472c0b44d0 0x7f472c0ac1f0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f4724007e70 tx=0x7f472400e4b0 comp rx=0 tx=0).stop 2024-04-24T23:08:53.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.692+0000 7f4730fb0640 1 -- 172.21.15.32:0/1374179423 shutdown_connections 2024-04-24T23:08:53.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.692+0000 7f4730fb0640 1 --2- 172.21.15.32:0/1374179423 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f470c06be30 0x7f470c06e2f0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:53.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.692+0000 7f4730fb0640 1 --2- 172.21.15.32:0/1374179423 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f472c0b44d0 0x7f472c0ac1f0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:53.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.692+0000 7f4730fb0640 1 --2- 172.21.15.32:0/1374179423 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f472c0af890 0x7f472c0abcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:53.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.692+0000 7f4730fb0640 1 --2- 172.21.15.32:0/1374179423 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f472c06c5f0 0x7f472c0adce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:53.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.692+0000 7f4730fb0640 1 -- 172.21.15.32:0/1374179423 >> 172.21.15.32:0/1374179423 conn(0x7f472c06b110 msgr2=0x7f472c15f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:53.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.692+0000 7f4730fb0640 1 -- 172.21.15.32:0/1374179423 shutdown_connections 2024-04-24T23:08:53.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.692+0000 7f4730fb0640 1 -- 172.21.15.32:0/1374179423 wait complete. 2024-04-24T23:08:53.705 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:08:53.705 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:08:53.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.808+0000 7fe87277e640 1 Processor -- start 2024-04-24T23:08:53.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.808+0000 7fe87277e640 1 -- start start 2024-04-24T23:08:53.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.809+0000 7fe87277e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe86c158dd0 0x7fe86c06b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:53.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.809+0000 7fe87277e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe86c06dc60 0x7fe86c0b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:53.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.809+0000 7fe87277e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe86c15ef10 con 0x7fe86c06b860 2024-04-24T23:08:53.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.809+0000 7fe87277e640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fe86c15f1f0 con 0x7fe86c06dc60 2024-04-24T23:08:53.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.809+0000 7fe87277e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe86c15f4d0 con 0x7fe86c158dd0 2024-04-24T23:08:53.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.809+0000 7fe870d1b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe86c06dc60 0x7fe86c0b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:53.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.809+0000 7fe870d1b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe86c06dc60 0x7fe86c0b5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45428/0 (socket says 172.21.15.32:45428) 2024-04-24T23:08:53.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.809+0000 7fe86b7fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe86c06b860 0x7fe86c06d550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54956/0 (socket says 172.21.15.32:54956) 2024-04-24T23:08:53.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.809+0000 7fe870d1b640 1 -- 172.21.15.32:0/1266107938 learned_addr learned my addr 172.21.15.32:0/1266107938 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:53.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.809+0000 7fe86bfff640 1 --2- 172.21.15.32:0/1266107938 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe86c158dd0 0x7fe86c06b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:53.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.809+0000 7fe870d1b640 1 -- 172.21.15.32:0/1266107938 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe86c158dd0 msgr2=0x7fe86c06b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:53.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.809+0000 7fe870d1b640 1 --2- 172.21.15.32:0/1266107938 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe86c158dd0 0x7fe86c06b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:53.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.809+0000 7fe870d1b640 1 -- 172.21.15.32:0/1266107938 >> v1:172.21.15.32:6789/0 conn(0x7fe86c06b860 legacy=0x7fe86c06d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:53.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.809+0000 7fe870d1b640 1 -- 172.21.15.32:0/1266107938 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe86c15f7b0 con 0x7fe86c06dc60 2024-04-24T23:08:53.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.809+0000 7fe86affd640 1 -- 172.21.15.32:0/1266107938 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3604848289 0 0) 0x7fe86c15ef10 con 0x7fe86c06b860 2024-04-24T23:08:53.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.810+0000 7fe870d1b640 1 --2- 172.21.15.32:0/1266107938 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe86c06dc60 0x7fe86c0b5d00 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fe864009e50 tx=0x7fe86402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a40e84935a0c40f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:53.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.810+0000 7fe86affd640 1 -- 172.21.15.32:0/1266107938 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe864036070 con 0x7fe86c06dc60 2024-04-24T23:08:53.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.810+0000 7fe86affd640 1 -- 172.21.15.32:0/1266107938 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe864031600 con 0x7fe86c06dc60 2024-04-24T23:08:53.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.810+0000 7fe86affd640 1 -- 172.21.15.32:0/1266107938 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8640319d0 con 0x7fe86c06dc60 2024-04-24T23:08:53.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.811+0000 7fe87277e640 1 -- 172.21.15.32:0/1266107938 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe86c06dc60 msgr2=0x7fe86c0b5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:53.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.811+0000 7fe87277e640 1 --2- 172.21.15.32:0/1266107938 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe86c06dc60 0x7fe86c0b5d00 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fe864009e50 tx=0x7fe86402a710 comp rx=0 tx=0).stop 2024-04-24T23:08:53.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.811+0000 7fe87277e640 1 -- 172.21.15.32:0/1266107938 shutdown_connections 2024-04-24T23:08:53.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.811+0000 7fe87277e640 1 --2- 172.21.15.32:0/1266107938 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe86c06dc60 0x7fe86c0b5d00 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:53.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.811+0000 7fe87277e640 1 --2- 172.21.15.32:0/1266107938 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe86c158dd0 0x7fe86c06b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:53.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.811+0000 7fe87277e640 1 -- 172.21.15.32:0/1266107938 >> 172.21.15.32:0/1266107938 conn(0x7fe86c06e800 msgr2=0x7fe86c06ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:53.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.811+0000 7fe87277e640 1 -- 172.21.15.32:0/1266107938 shutdown_connections 2024-04-24T23:08:53.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.811+0000 7fe87277e640 1 -- 172.21.15.32:0/1266107938 wait complete. 2024-04-24T23:08:53.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.812+0000 7fe87277e640 1 Processor -- start 2024-04-24T23:08:53.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.812+0000 7fe87277e640 1 -- start start 2024-04-24T23:08:53.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.813+0000 7fe87277e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe86c06b860 0x7fe86c15e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:53.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.813+0000 7fe87277e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe86c06dc60 0x7fe86c072480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:53.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.813+0000 7fe87277e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe86c158dd0 0x7fe86c0729c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:53.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.813+0000 7fe87277e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe86c06bfe0 con 0x7fe86c06dc60 2024-04-24T23:08:53.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.813+0000 7fe87277e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe86c15fcd0 con 0x7fe86c158dd0 2024-04-24T23:08:53.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.813+0000 7fe87277e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe86c1591b0 con 0x7fe86c06b860 2024-04-24T23:08:53.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.813+0000 7fe86b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe86c06dc60 0x7fe86c072480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:53.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.813+0000 7fe86b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe86c06dc60 0x7fe86c072480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45436/0 (socket says 172.21.15.32:45436) 2024-04-24T23:08:53.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.813+0000 7fe86b7fe640 1 -- 172.21.15.32:0/4112363560 learned_addr learned my addr 172.21.15.32:0/4112363560 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:53.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.813+0000 7fe870d1b640 1 --2- 172.21.15.32:0/4112363560 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe86c158dd0 0x7fe86c0729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:53.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.813+0000 7fe86bfff640 1 --2- 172.21.15.32:0/4112363560 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe86c06b860 0x7fe86c15e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:53.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.813+0000 7fe86b7fe640 1 -- 172.21.15.32:0/4112363560 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe86c06b860 msgr2=0x7fe86c15e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:53.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.813+0000 7fe86b7fe640 1 --2- 172.21.15.32:0/4112363560 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe86c06b860 0x7fe86c15e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:53.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.813+0000 7fe86b7fe640 1 -- 172.21.15.32:0/4112363560 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe86c158dd0 msgr2=0x7fe86c0729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:53.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.813+0000 7fe86b7fe640 1 --2- 172.21.15.32:0/4112363560 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe86c158dd0 0x7fe86c0729c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:53.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.813+0000 7fe86b7fe640 1 -- 172.21.15.32:0/4112363560 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe86c13f940 con 0x7fe86c06dc60 2024-04-24T23:08:53.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.813+0000 7fe870d1b640 1 --2- 172.21.15.32:0/4112363560 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe86c158dd0 0x7fe86c0729c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:08:53.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.813+0000 7fe86b7fe640 1 --2- 172.21.15.32:0/4112363560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe86c06dc60 0x7fe86c072480 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fe85c00c4d0 tx=0x7fe85c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:53.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.814+0000 7fe868ff9640 1 -- 172.21.15.32:0/4112363560 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85c018070 con 0x7fe86c06dc60 2024-04-24T23:08:53.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.814+0000 7fe868ff9640 1 -- 172.21.15.32:0/4112363560 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe85c007500 con 0x7fe86c06dc60 2024-04-24T23:08:53.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.814+0000 7fe87277e640 1 -- 172.21.15.32:0/4112363560 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe86c13fc20 con 0x7fe86c06dc60 2024-04-24T23:08:53.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.814+0000 7fe868ff9640 1 -- 172.21.15.32:0/4112363560 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85c004a20 con 0x7fe86c06dc60 2024-04-24T23:08:53.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.814+0000 7fe87277e640 1 -- 172.21.15.32:0/4112363560 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe86c1401e0 con 0x7fe86c06dc60 2024-04-24T23:08:53.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.815+0000 7fe87277e640 1 -- 172.21.15.32:0/4112363560 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe86c068430 con 0x7fe86c06dc60 2024-04-24T23:08:53.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.815+0000 7fe868ff9640 1 -- 172.21.15.32:0/4112363560 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe85c01b070 con 0x7fe86c06dc60 2024-04-24T23:08:53.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.816+0000 7fe868ff9640 1 --2- 172.21.15.32:0/4112363560 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe85806be30 0x7fe85806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:53.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.816+0000 7fe868ff9640 1 -- 172.21.15.32:0/4112363560 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 4943+0+0 (secure 0 0 0) 0x7fe85c089ee0 con 0x7fe86c06dc60 2024-04-24T23:08:53.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.816+0000 7fe86bfff640 1 --2- 172.21.15.32:0/4112363560 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe85806be30 0x7fe85806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:53.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.816+0000 7fe86bfff640 1 --2- 172.21.15.32:0/4112363560 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe85806be30 0x7fe85806e2f0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fe860006fd0 tx=0x7fe860008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:53.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:53.820+0000 7fe868ff9640 1 -- 172.21.15.32:0/4112363560 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe85c058b90 con 0x7fe86c06dc60 2024-04-24T23:08:54.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.000+0000 7fe87277e640 1 -- 172.21.15.32:0/4112363560 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe86c158400 con 0x7fe86c06dc60 2024-04-24T23:08:54.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.000+0000 7fe868ff9640 1 -- 172.21.15.32:0/4112363560 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v161) ==== 93+0+10 (secure 0 0 0) 0x7fe85c058490 con 0x7fe86c06dc60 2024-04-24T23:08:54.002 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T23:08:54.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.002+0000 7fe87277e640 1 -- 172.21.15.32:0/4112363560 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe85806be30 msgr2=0x7fe85806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:54.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.002+0000 7fe87277e640 1 --2- 172.21.15.32:0/4112363560 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe85806be30 0x7fe85806e2f0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fe860006fd0 tx=0x7fe860008040 comp rx=0 tx=0).stop 2024-04-24T23:08:54.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.002+0000 7fe87277e640 1 -- 172.21.15.32:0/4112363560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe86c06dc60 msgr2=0x7fe86c072480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:54.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.002+0000 7fe87277e640 1 --2- 172.21.15.32:0/4112363560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe86c06dc60 0x7fe86c072480 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fe85c00c4d0 tx=0x7fe85c00c990 comp rx=0 tx=0).stop 2024-04-24T23:08:54.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.002+0000 7fe87277e640 1 -- 172.21.15.32:0/4112363560 shutdown_connections 2024-04-24T23:08:54.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.002+0000 7fe87277e640 1 --2- 172.21.15.32:0/4112363560 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe85806be30 0x7fe85806e2f0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:54.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.002+0000 7fe87277e640 1 --2- 172.21.15.32:0/4112363560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe86c06dc60 0x7fe86c072480 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:54.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.002+0000 7fe87277e640 1 --2- 172.21.15.32:0/4112363560 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe86c158dd0 0x7fe86c0729c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:54.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.002+0000 7fe87277e640 1 --2- 172.21.15.32:0/4112363560 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe86c06b860 0x7fe86c15e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:54.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.002+0000 7fe87277e640 1 -- 172.21.15.32:0/4112363560 >> 172.21.15.32:0/4112363560 conn(0x7fe86c06e800 msgr2=0x7fe86c06ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:54.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.002+0000 7fe87277e640 1 -- 172.21.15.32:0/4112363560 shutdown_connections 2024-04-24T23:08:54.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.002+0000 7fe87277e640 1 -- 172.21.15.32:0/4112363560 wait complete. 2024-04-24T23:08:54.015 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync' 2024-04-24T23:08:54.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.114+0000 7faf98741640 1 Processor -- start 2024-04-24T23:08:54.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.115+0000 7faf98741640 1 -- start start 2024-04-24T23:08:54.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.115+0000 7faf98741640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7faf9015edd0 0x7faf900b5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:54.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.115+0000 7faf98741640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faf900cd3c0 0x7faf900b5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:54.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.115+0000 7faf98741640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf900cd910 con 0x7faf900b5560 2024-04-24T23:08:54.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.115+0000 7faf98741640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7faf900cdbf0 con 0x7faf900cd3c0 2024-04-24T23:08:54.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.115+0000 7faf98741640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7faf900cded0 con 0x7faf9015edd0 2024-04-24T23:08:54.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.116+0000 7faf96cde640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faf900cd3c0 0x7faf900b5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:54.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.116+0000 7faf96cde640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faf900cd3c0 0x7faf900b5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45450/0 (socket says 172.21.15.32:45450) 2024-04-24T23:08:54.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.116+0000 7faf964dd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7faf900b5560 0x7faf9015e6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54962/0 (socket says 172.21.15.32:54962) 2024-04-24T23:08:54.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.116+0000 7faf96cde640 1 -- 172.21.15.32:0/3157245472 learned_addr learned my addr 172.21.15.32:0/3157245472 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:54.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.116+0000 7faf95cdc640 1 --2- 172.21.15.32:0/3157245472 >> v2:172.21.15.121:3300/0 conn(0x7faf9015edd0 0x7faf900b5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:54.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.116+0000 7faf96cde640 1 -- 172.21.15.32:0/3157245472 >> v2:172.21.15.121:3300/0 conn(0x7faf9015edd0 msgr2=0x7faf900b5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:54.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.116+0000 7faf96cde640 1 --2- 172.21.15.32:0/3157245472 >> v2:172.21.15.121:3300/0 conn(0x7faf9015edd0 0x7faf900b5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:54.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.116+0000 7faf96cde640 1 -- 172.21.15.32:0/3157245472 >> v1:172.21.15.32:6789/0 conn(0x7faf900b5560 legacy=0x7faf9015e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:54.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.116+0000 7faf96cde640 1 -- 172.21.15.32:0/3157245472 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf900ce1b0 con 0x7faf900cd3c0 2024-04-24T23:08:54.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.116+0000 7faf954db640 1 -- 172.21.15.32:0/3157245472 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1589696875 0 0) 0x7faf900cd910 con 0x7faf900b5560 2024-04-24T23:08:54.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.116+0000 7faf95cdc640 1 --2- 172.21.15.32:0/3157245472 >> v2:172.21.15.121:3300/0 conn(0x7faf9015edd0 0x7faf900b5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:08:54.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.117+0000 7faf96cde640 1 --2- 172.21.15.32:0/3157245472 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf900cd3c0 0x7faf900b5e80 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7faf9006ced0 tx=0x7faf8802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b9c6fb288630c42 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:54.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.117+0000 7faf954db640 1 -- 172.21.15.32:0/3157245472 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf88036070 con 0x7faf900cd3c0 2024-04-24T23:08:54.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.117+0000 7faf954db640 1 -- 172.21.15.32:0/3157245472 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf88031600 con 0x7faf900cd3c0 2024-04-24T23:08:54.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.117+0000 7faf954db640 1 -- 172.21.15.32:0/3157245472 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf880319d0 con 0x7faf900cd3c0 2024-04-24T23:08:54.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.117+0000 7faf98741640 1 -- 172.21.15.32:0/3157245472 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf900cd3c0 msgr2=0x7faf900b5e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:54.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.117+0000 7faf98741640 1 --2- 172.21.15.32:0/3157245472 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf900cd3c0 0x7faf900b5e80 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7faf9006ced0 tx=0x7faf8802a730 comp rx=0 tx=0).stop 2024-04-24T23:08:54.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.117+0000 7faf98741640 1 -- 172.21.15.32:0/3157245472 shutdown_connections 2024-04-24T23:08:54.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.117+0000 7faf98741640 1 --2- 172.21.15.32:0/3157245472 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf900cd3c0 0x7faf900b5e80 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:54.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.117+0000 7faf98741640 1 --2- 172.21.15.32:0/3157245472 >> v2:172.21.15.121:3300/0 conn(0x7faf9015edd0 0x7faf900b5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:54.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.117+0000 7faf98741640 1 -- 172.21.15.32:0/3157245472 >> 172.21.15.32:0/3157245472 conn(0x7faf9006b110 msgr2=0x7faf9006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:54.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.117+0000 7faf98741640 1 -- 172.21.15.32:0/3157245472 shutdown_connections 2024-04-24T23:08:54.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.117+0000 7faf98741640 1 -- 172.21.15.32:0/3157245472 wait complete. 2024-04-24T23:08:54.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.119+0000 7faf98741640 1 Processor -- start 2024-04-24T23:08:54.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.119+0000 7faf98741640 1 -- start start 2024-04-24T23:08:54.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.119+0000 7faf98741640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf900b5560 0x7faf901ef8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:54.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.119+0000 7faf98741640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf900cd3c0 0x7faf90150dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:54.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.119+0000 7faf98741640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faf9015edd0 0x7faf90157bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:54.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.119+0000 7faf98741640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7faf9006c5f0 con 0x7faf900b5560 2024-04-24T23:08:54.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.119+0000 7faf98741640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7faf900b63c0 con 0x7faf900cd3c0 2024-04-24T23:08:54.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.119+0000 7faf98741640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7faf9006c770 con 0x7faf9015edd0 2024-04-24T23:08:54.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.119+0000 7faf964dd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf900b5560 0x7faf901ef8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:54.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.120+0000 7faf964dd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf900b5560 0x7faf901ef8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45462/0 (socket says 172.21.15.32:45462) 2024-04-24T23:08:54.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.120+0000 7faf964dd640 1 -- 172.21.15.32:0/1306011753 learned_addr learned my addr 172.21.15.32:0/1306011753 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:54.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.120+0000 7faf95cdc640 1 --2- 172.21.15.32:0/1306011753 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf900cd3c0 0x7faf90150dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:54.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.120+0000 7faf96cde640 1 --2- 172.21.15.32:0/1306011753 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faf9015edd0 0x7faf90157bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:54.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.120+0000 7faf964dd640 1 -- 172.21.15.32:0/1306011753 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faf9015edd0 msgr2=0x7faf90157bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:54.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.120+0000 7faf964dd640 1 --2- 172.21.15.32:0/1306011753 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faf9015edd0 0x7faf90157bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:54.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.120+0000 7faf964dd640 1 -- 172.21.15.32:0/1306011753 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf900cd3c0 msgr2=0x7faf90150dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:54.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.120+0000 7faf964dd640 1 --2- 172.21.15.32:0/1306011753 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf900cd3c0 0x7faf90150dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:54.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.120+0000 7faf964dd640 1 -- 172.21.15.32:0/1306011753 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf90151790 con 0x7faf900b5560 2024-04-24T23:08:54.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.120+0000 7faf964dd640 1 --2- 172.21.15.32:0/1306011753 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf900b5560 0x7faf901ef8e0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7faf8000c500 tx=0x7faf8000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:54.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.120+0000 7faf86ffd640 1 -- 172.21.15.32:0/1306011753 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf80018070 con 0x7faf900b5560 2024-04-24T23:08:54.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.120+0000 7faf86ffd640 1 -- 172.21.15.32:0/1306011753 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf80007580 con 0x7faf900b5560 2024-04-24T23:08:54.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.120+0000 7faf98741640 1 -- 172.21.15.32:0/1306011753 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf90151f80 con 0x7faf900b5560 2024-04-24T23:08:54.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.120+0000 7faf86ffd640 1 -- 172.21.15.32:0/1306011753 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf80004a20 con 0x7faf900b5560 2024-04-24T23:08:54.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.120+0000 7faf98741640 1 -- 172.21.15.32:0/1306011753 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf90151a70 con 0x7faf900b5560 2024-04-24T23:08:54.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.121+0000 7faf98741640 1 -- 172.21.15.32:0/1306011753 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf90068430 con 0x7faf900b5560 2024-04-24T23:08:54.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.122+0000 7faf86ffd640 1 -- 172.21.15.32:0/1306011753 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faf8001b070 con 0x7faf900b5560 2024-04-24T23:08:54.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.122+0000 7faf86ffd640 1 --2- 172.21.15.32:0/1306011753 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7faf7406be30 0x7faf7406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:54.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.122+0000 7faf86ffd640 1 -- 172.21.15.32:0/1306011753 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 4943+0+0 (secure 0 0 0) 0x7faf80089de0 con 0x7faf900b5560 2024-04-24T23:08:54.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.122+0000 7faf95cdc640 1 --2- 172.21.15.32:0/1306011753 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7faf7406be30 0x7faf7406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:54.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.123+0000 7faf95cdc640 1 --2- 172.21.15.32:0/1306011753 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7faf7406be30 0x7faf7406e2f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7faf90158760 tx=0x7faf8c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:54.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.127+0000 7faf86ffd640 1 -- 172.21.15.32:0/1306011753 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7faf80058b20 con 0x7faf900b5560 2024-04-24T23:08:54.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:54.384+0000 7faf98741640 1 -- 172.21.15.32:0/1306011753 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]} v 0) -- 0x7faf900cee70 con 0x7faf900b5560 2024-04-24T23:08:55.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.372+0000 7faf86ffd640 1 -- 172.21.15.32:0/1306011753 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]}]=0 v476) ==== 134+0+0 (secure 0 0 0) 0x7faf80058420 con 0x7faf900b5560 2024-04-24T23:08:55.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.376+0000 7faf98741640 1 -- 172.21.15.32:0/1306011753 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7faf7406be30 msgr2=0x7faf7406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:55.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.376+0000 7faf98741640 1 --2- 172.21.15.32:0/1306011753 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7faf7406be30 0x7faf7406e2f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7faf90158760 tx=0x7faf8c008040 comp rx=0 tx=0).stop 2024-04-24T23:08:55.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.376+0000 7faf98741640 1 -- 172.21.15.32:0/1306011753 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf900b5560 msgr2=0x7faf901ef8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:55.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.376+0000 7faf98741640 1 --2- 172.21.15.32:0/1306011753 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf900b5560 0x7faf901ef8e0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7faf8000c500 tx=0x7faf8000c9c0 comp rx=0 tx=0).stop 2024-04-24T23:08:55.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.376+0000 7faf98741640 1 -- 172.21.15.32:0/1306011753 shutdown_connections 2024-04-24T23:08:55.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.376+0000 7faf98741640 1 --2- 172.21.15.32:0/1306011753 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7faf7406be30 0x7faf7406e2f0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:55.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.376+0000 7faf98741640 1 --2- 172.21.15.32:0/1306011753 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faf9015edd0 0x7faf90157bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:55.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.376+0000 7faf98741640 1 --2- 172.21.15.32:0/1306011753 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf900cd3c0 0x7faf90150dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:55.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.376+0000 7faf98741640 1 --2- 172.21.15.32:0/1306011753 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf900b5560 0x7faf901ef8e0 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:55.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.376+0000 7faf98741640 1 -- 172.21.15.32:0/1306011753 >> 172.21.15.32:0/1306011753 conn(0x7faf9006b110 msgr2=0x7faf901549a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:55.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.376+0000 7faf98741640 1 -- 172.21.15.32:0/1306011753 shutdown_connections 2024-04-24T23:08:55.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.376+0000 7faf98741640 1 -- 172.21.15.32:0/1306011753 wait complete. 2024-04-24T23:08:55.392 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-24T23:08:55.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.499+0000 7f245bae1640 1 Processor -- start 2024-04-24T23:08:55.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.500+0000 7f245bae1640 1 -- start start 2024-04-24T23:08:55.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.500+0000 7f245bae1640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f24540d4210 0x7f24540d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:55.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.500+0000 7f245bae1640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24540d3170 con 0x7f245406b820 2024-04-24T23:08:55.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.500+0000 7f245bae1640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24540d3450 con 0x7f245415e9b0 2024-04-24T23:08:55.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.500+0000 7f245bae1640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f24540d3730 con 0x7f24540d4210 2024-04-24T23:08:55.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.500+0000 7f245907c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f245406b820 0x7f24540b78f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54964/0 (socket says 172.21.15.32:54964) 2024-04-24T23:08:55.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.500+0000 7f245907c640 1 -- 172.21.15.32:0/394044342 learned_addr learned my addr 172.21.15.32:0/394044342 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:08:55.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.501+0000 7f245a07e640 1 --2- 172.21.15.32:0/394044342 >> v2:172.21.15.121:3300/0 conn(0x7f24540d4210 0x7f24540d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:55.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.501+0000 7f245887b640 1 -- 172.21.15.32:0/394044342 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1089949868 0 0) 0x7f24540d3170 con 0x7f245406b820 2024-04-24T23:08:55.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.501+0000 7f245887b640 1 -- 172.21.15.32:0/394044342 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f243c003520 con 0x7f245406b820 2024-04-24T23:08:55.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.501+0000 7f245887b640 1 -- 172.21.15.32:0/394044342 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2335956897 0 0) 0x7f24540d3450 con 0x7f245415e9b0 2024-04-24T23:08:55.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.501+0000 7f245887b640 1 -- 172.21.15.32:0/394044342 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f24540d3170 con 0x7f245415e9b0 2024-04-24T23:08:55.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.501+0000 7f245a07e640 1 -- 172.21.15.32:0/394044342 >> v1:172.21.15.121:6789/0 conn(0x7f245415e9b0 legacy=0x7f245406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:55.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.501+0000 7f245a07e640 1 -- 172.21.15.32:0/394044342 >> v1:172.21.15.32:6789/0 conn(0x7f245406b820 legacy=0x7f24540b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:55.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.501+0000 7f245a07e640 1 -- 172.21.15.32:0/394044342 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24540d3a10 con 0x7f24540d4210 2024-04-24T23:08:55.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.502+0000 7f245a07e640 1 --2- 172.21.15.32:0/394044342 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24540d4210 0x7f24540d2b90 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f2444009d90 tx=0x7f244402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=add2f193b09b0ea0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:55.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.502+0000 7f245887b640 1 -- 172.21.15.32:0/394044342 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2444007740 con 0x7f24540d4210 2024-04-24T23:08:55.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.502+0000 7f245887b640 1 -- 172.21.15.32:0/394044342 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24440078e0 con 0x7f24540d4210 2024-04-24T23:08:55.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.502+0000 7f245887b640 1 -- 172.21.15.32:0/394044342 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2444007cb0 con 0x7f24540d4210 2024-04-24T23:08:55.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.502+0000 7f245bae1640 1 -- 172.21.15.32:0/394044342 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24540d4210 msgr2=0x7f24540d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:55.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.502+0000 7f245bae1640 1 --2- 172.21.15.32:0/394044342 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24540d4210 0x7f24540d2b90 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f2444009d90 tx=0x7f244402a660 comp rx=0 tx=0).stop 2024-04-24T23:08:55.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.503+0000 7f245bae1640 1 -- 172.21.15.32:0/394044342 shutdown_connections 2024-04-24T23:08:55.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.503+0000 7f245bae1640 1 --2- 172.21.15.32:0/394044342 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24540d4210 0x7f24540d2b90 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:55.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.503+0000 7f245bae1640 1 -- 172.21.15.32:0/394044342 >> 172.21.15.32:0/394044342 conn(0x7f24540d11d0 msgr2=0x7f24540d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:55.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.503+0000 7f245bae1640 1 -- 172.21.15.32:0/394044342 shutdown_connections 2024-04-24T23:08:55.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.503+0000 7f245bae1640 1 -- 172.21.15.32:0/394044342 wait complete. 2024-04-24T23:08:55.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.504+0000 7f245bae1640 1 Processor -- start 2024-04-24T23:08:55.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.504+0000 7f245bae1640 1 -- start start 2024-04-24T23:08:55.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.505+0000 7f245bae1640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f245406b820 0x7f24541e4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:55.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.505+0000 7f245bae1640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24540d4210 0x7f24540cd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:55.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.505+0000 7f245bae1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f245415e9b0 0x7f245415b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:55.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.505+0000 7f245bae1640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f24540ba0e0 con 0x7f245415e9b0 2024-04-24T23:08:55.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.505+0000 7f245bae1640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f24540d2400 con 0x7f24540d4210 2024-04-24T23:08:55.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.505+0000 7f245bae1640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f24540ba260 con 0x7f245406b820 2024-04-24T23:08:55.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.505+0000 7f245a07e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f245415e9b0 0x7f245415b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:55.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.505+0000 7f245a07e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f245415e9b0 0x7f245415b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45474/0 (socket says 172.21.15.32:45474) 2024-04-24T23:08:55.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.505+0000 7f245a07e640 1 -- 172.21.15.32:0/993013471 learned_addr learned my addr 172.21.15.32:0/993013471 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:55.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.505+0000 7f245907c640 1 --2- 172.21.15.32:0/993013471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24540d4210 0x7f24540cd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:55.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.505+0000 7f245987d640 1 --2- 172.21.15.32:0/993013471 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f245406b820 0x7f24541e4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:55.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.505+0000 7f245a07e640 1 -- 172.21.15.32:0/993013471 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f245406b820 msgr2=0x7f24541e4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:55.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.505+0000 7f245a07e640 1 --2- 172.21.15.32:0/993013471 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f245406b820 0x7f24541e4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:55.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.505+0000 7f245a07e640 1 -- 172.21.15.32:0/993013471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24540d4210 msgr2=0x7f24540cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:55.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.505+0000 7f245a07e640 1 --2- 172.21.15.32:0/993013471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24540d4210 0x7f24540cd6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:55.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.505+0000 7f245a07e640 1 -- 172.21.15.32:0/993013471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24540cdf40 con 0x7f245415e9b0 2024-04-24T23:08:55.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.505+0000 7f245987d640 1 --2- 172.21.15.32:0/993013471 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f245406b820 0x7f24541e4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:08:55.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.505+0000 7f245907c640 1 --2- 172.21.15.32:0/993013471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24540d4210 0x7f24540cd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:08:55.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.505+0000 7f245a07e640 1 --2- 172.21.15.32:0/993013471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f245415e9b0 0x7f245415b2f0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f2444009eb0 tx=0x7f2444004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:55.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.505+0000 7f24527fc640 1 -- 172.21.15.32:0/993013471 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2444040070 con 0x7f245415e9b0 2024-04-24T23:08:55.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.505+0000 7f24527fc640 1 -- 172.21.15.32:0/993013471 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2444031bc0 con 0x7f245415e9b0 2024-04-24T23:08:55.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.505+0000 7f245bae1640 1 -- 172.21.15.32:0/993013471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24540ce220 con 0x7f245415e9b0 2024-04-24T23:08:55.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.505+0000 7f24527fc640 1 -- 172.21.15.32:0/993013471 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2444009450 con 0x7f245415e9b0 2024-04-24T23:08:55.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.505+0000 7f245bae1640 1 -- 172.21.15.32:0/993013471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24540ce7e0 con 0x7f245415e9b0 2024-04-24T23:08:55.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.507+0000 7f245bae1640 1 -- 172.21.15.32:0/993013471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2454068430 con 0x7f245415e9b0 2024-04-24T23:08:55.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.507+0000 7f24527fc640 1 -- 172.21.15.32:0/993013471 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2444039040 con 0x7f245415e9b0 2024-04-24T23:08:55.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.507+0000 7f24527fc640 1 --2- 172.21.15.32:0/993013471 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f244006be30 0x7f244006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:55.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.507+0000 7f24527fc640 1 -- 172.21.15.32:0/993013471 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 4943+0+0 (secure 0 0 0) 0x7f24440ac2d0 con 0x7f245415e9b0 2024-04-24T23:08:55.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.508+0000 7f245987d640 1 --2- 172.21.15.32:0/993013471 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f244006be30 0x7f244006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:55.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.508+0000 7f245987d640 1 --2- 172.21.15.32:0/993013471 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f244006be30 0x7f244006e2f0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f244c001610 tx=0x7f244c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:55.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.512+0000 7f24527fc640 1 -- 172.21.15.32:0/993013471 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f244407b010 con 0x7f245415e9b0 2024-04-24T23:08:55.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.695+0000 7f245bae1640 1 -- 172.21.15.32:0/993013471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f24540d3340 con 0x7f245415e9b0 2024-04-24T23:08:55.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.698+0000 7f24527fc640 1 -- 172.21.15.32:0/993013471 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7f244407a910 con 0x7f245415e9b0 2024-04-24T23:08:55.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.700+0000 7f245bae1640 1 -- 172.21.15.32:0/993013471 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f244006be30 msgr2=0x7f244006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:55.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.700+0000 7f245bae1640 1 --2- 172.21.15.32:0/993013471 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f244006be30 0x7f244006e2f0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f244c001610 tx=0x7f244c009040 comp rx=0 tx=0).stop 2024-04-24T23:08:55.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.700+0000 7f245bae1640 1 -- 172.21.15.32:0/993013471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f245415e9b0 msgr2=0x7f245415b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:55.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.700+0000 7f245bae1640 1 --2- 172.21.15.32:0/993013471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f245415e9b0 0x7f245415b2f0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f2444009eb0 tx=0x7f2444004a40 comp rx=0 tx=0).stop 2024-04-24T23:08:55.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.700+0000 7f245bae1640 1 -- 172.21.15.32:0/993013471 shutdown_connections 2024-04-24T23:08:55.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.700+0000 7f245bae1640 1 --2- 172.21.15.32:0/993013471 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f244006be30 0x7f244006e2f0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:55.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.700+0000 7f245bae1640 1 --2- 172.21.15.32:0/993013471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f245415e9b0 0x7f245415b2f0 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:55.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.700+0000 7f245bae1640 1 --2- 172.21.15.32:0/993013471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24540d4210 0x7f24540cd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:55.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.700+0000 7f245bae1640 1 --2- 172.21.15.32:0/993013471 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f245406b820 0x7f24541e4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:55.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.700+0000 7f245bae1640 1 -- 172.21.15.32:0/993013471 >> 172.21.15.32:0/993013471 conn(0x7f24540d11d0 msgr2=0x7f245406b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:55.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.700+0000 7f245bae1640 1 -- 172.21.15.32:0/993013471 shutdown_connections 2024-04-24T23:08:55.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:55.700+0000 7f245bae1640 1 -- 172.21.15.32:0/993013471 wait complete. 2024-04-24T23:08:55.713 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-24T23:08:55.714 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:08:55.752 DEBUG:teuthology.orchestra.run.smithi121:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:08:55.797 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-24T23:08:55.797 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:08:55.798 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:08:55.813 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=413 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T23:08:55.813 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:08:55.813 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:08:55.830 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3271 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T23:08:55.831 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:08:55.831 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:08:55.868 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=413 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T23:08:55.869 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:08:55.869 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:08:55.886 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3271 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T23:08:55.886 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-24T23:08:55.886 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:08:56.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.007+0000 7ff7cab32640 1 Processor -- start 2024-04-24T23:08:56.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.008+0000 7ff7cab32640 1 -- start start 2024-04-24T23:08:56.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.008+0000 7ff7cab32640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff7c406b820 0x7ff7c40d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:56.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.008+0000 7ff7cab32640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7c40d3170 con 0x7ff7c40d4210 2024-04-24T23:08:56.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.008+0000 7ff7cab32640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7c40d3450 con 0x7ff7c415e9b0 2024-04-24T23:08:56.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.008+0000 7ff7cab32640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7ff7c40d3730 con 0x7ff7c406b820 2024-04-24T23:08:56.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.008+0000 7ff7c90cf640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff7c40d4210 0x7ff7c40b78f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54976/0 (socket says 172.21.15.32:54976) 2024-04-24T23:08:56.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.009+0000 7ff7c90cf640 1 -- 172.21.15.32:0/1394450709 learned_addr learned my addr 172.21.15.32:0/1394450709 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:08:56.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.009+0000 7ff7c3fff640 1 --2- 172.21.15.32:0/1394450709 >> v2:172.21.15.121:3300/0 conn(0x7ff7c406b820 0x7ff7c40d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:56.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.009+0000 7ff7c37fe640 1 -- 172.21.15.32:0/1394450709 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1872877865 0 0) 0x7ff7c40d3170 con 0x7ff7c40d4210 2024-04-24T23:08:56.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.009+0000 7ff7c37fe640 1 -- 172.21.15.32:0/1394450709 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff7b4003520 con 0x7ff7c40d4210 2024-04-24T23:08:56.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.009+0000 7ff7c37fe640 1 -- 172.21.15.32:0/1394450709 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1630164379 0 0) 0x7ff7b4003520 con 0x7ff7c40d4210 2024-04-24T23:08:56.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.009+0000 7ff7c37fe640 1 -- 172.21.15.32:0/1394450709 >> v2:172.21.15.121:3300/0 conn(0x7ff7c406b820 msgr2=0x7ff7c40d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:56.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.009+0000 7ff7c37fe640 1 --2- 172.21.15.32:0/1394450709 >> v2:172.21.15.121:3300/0 conn(0x7ff7c406b820 0x7ff7c40d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:56.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.009+0000 7ff7c37fe640 1 -- 172.21.15.32:0/1394450709 >> v1:172.21.15.121:6789/0 conn(0x7ff7c415e9b0 legacy=0x7ff7c406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:56.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.009+0000 7ff7c37fe640 1 -- 172.21.15.32:0/1394450709 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7c40d3a10 con 0x7ff7c40d4210 2024-04-24T23:08:56.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.009+0000 7ff7c37fe640 1 -- 172.21.15.32:0/1394450709 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7ff7bc0019f0 con 0x7ff7c40d4210 2024-04-24T23:08:56.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.009+0000 7ff7c37fe640 1 -- 172.21.15.32:0/1394450709 >> v1:172.21.15.32:6789/0 conn(0x7ff7c40d4210 legacy=0x7ff7c40b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:56.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.009+0000 7ff7c37fe640 1 --2- 172.21.15.32:0/1394450709 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7b40039b0 0x7ff7b4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:56.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.009+0000 7ff7c37fe640 1 --2- 172.21.15.32:0/1394450709 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7b4004800 0x7ff7b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:56.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.009+0000 7ff7c37fe640 1 --2- 172.21.15.32:0/1394450709 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7b40051c0 0x7ff7b400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:56.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.009+0000 7ff7c37fe640 1 -- 172.21.15.32:0/1394450709 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c40d3730 con 0x7ff7b40039b0 2024-04-24T23:08:56.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.009+0000 7ff7c37fe640 1 -- 172.21.15.32:0/1394450709 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b400c160 con 0x7ff7b40051c0 2024-04-24T23:08:56.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.009+0000 7ff7c37fe640 1 -- 172.21.15.32:0/1394450709 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff7b400c360 con 0x7ff7b4004800 2024-04-24T23:08:56.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.009+0000 7ff7c88ce640 1 --2- 172.21.15.32:0/1394450709 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7b40039b0 0x7ff7b4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:56.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.009+0000 7ff7c3fff640 1 --2- 172.21.15.32:0/1394450709 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7b4004800 0x7ff7b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:56.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.009+0000 7ff7c90cf640 1 --2- 172.21.15.32:0/1394450709 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7b40051c0 0x7ff7b400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:56.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.010+0000 7ff7c88ce640 1 -- 172.21.15.32:0/1394450709 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7b4004800 msgr2=0x7ff7b4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:56.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.010+0000 7ff7c88ce640 1 --2- 172.21.15.32:0/1394450709 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7b4004800 0x7ff7b4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:56.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.010+0000 7ff7c88ce640 1 -- 172.21.15.32:0/1394450709 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7b40051c0 msgr2=0x7ff7b400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:56.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.010+0000 7ff7c88ce640 1 --2- 172.21.15.32:0/1394450709 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7b40051c0 0x7ff7b400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:56.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.010+0000 7ff7c88ce640 1 -- 172.21.15.32:0/1394450709 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7b4003520 con 0x7ff7b40039b0 2024-04-24T23:08:56.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.010+0000 7ff7c90cf640 1 -- 172.21.15.32:0/1394450709 reap_dead start 2024-04-24T23:08:56.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.010+0000 7ff7c88ce640 1 --2- 172.21.15.32:0/1394450709 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7b40039b0 0x7ff7b4003dd0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8001750 tx=0x7ff7b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1b2d781c1fd4994 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:56.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.010+0000 7ff7c37fe640 1 -- 172.21.15.32:0/1394450709 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b8001370 con 0x7ff7b40039b0 2024-04-24T23:08:56.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.010+0000 7ff7c37fe640 1 -- 172.21.15.32:0/1394450709 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7b80052d0 con 0x7ff7b40039b0 2024-04-24T23:08:56.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.010+0000 7ff7c37fe640 1 -- 172.21.15.32:0/1394450709 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b802abc0 con 0x7ff7b40039b0 2024-04-24T23:08:56.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.011+0000 7ff7cab32640 1 -- 172.21.15.32:0/1394450709 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7b40039b0 msgr2=0x7ff7b4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:56.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.011+0000 7ff7cab32640 1 --2- 172.21.15.32:0/1394450709 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7b40039b0 0x7ff7b4003dd0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8001750 tx=0x7ff7b4005830 comp rx=0 tx=0).stop 2024-04-24T23:08:56.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.011+0000 7ff7cab32640 1 -- 172.21.15.32:0/1394450709 shutdown_connections 2024-04-24T23:08:56.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.011+0000 7ff7cab32640 1 --2- 172.21.15.32:0/1394450709 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7b40039b0 0x7ff7b4003dd0 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:56.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.011+0000 7ff7cab32640 1 -- 172.21.15.32:0/1394450709 >> 172.21.15.32:0/1394450709 conn(0x7ff7c40d11d0 msgr2=0x7ff7c40d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:56.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.011+0000 7ff7cab32640 1 -- 172.21.15.32:0/1394450709 shutdown_connections 2024-04-24T23:08:56.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.011+0000 7ff7cab32640 1 -- 172.21.15.32:0/1394450709 wait complete. 2024-04-24T23:08:56.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.012+0000 7ff7cab32640 1 Processor -- start 2024-04-24T23:08:56.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.012+0000 7ff7cab32640 1 -- start start 2024-04-24T23:08:56.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.013+0000 7ff7cab32640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7b40039b0 0x7ff7c40cd740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:56.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.013+0000 7ff7cab32640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7c414fc10 0x7ff7c4160e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:56.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.013+0000 7ff7cab32640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7c41f4600 0x7ff7c41f4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:56.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.013+0000 7ff7cab32640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c40ba0e0 con 0x7ff7c41f4600 2024-04-24T23:08:56.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.013+0000 7ff7cab32640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c40d2400 con 0x7ff7b40039b0 2024-04-24T23:08:56.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.013+0000 7ff7cab32640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff7c40ba260 con 0x7ff7c414fc10 2024-04-24T23:08:56.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.013+0000 7ff7c90cf640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7c41f4600 0x7ff7c41f4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:56.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.013+0000 7ff7c90cf640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7c41f4600 0x7ff7c41f4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45496/0 (socket says 172.21.15.32:45496) 2024-04-24T23:08:56.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.013+0000 7ff7c90cf640 1 -- 172.21.15.32:0/3465390758 learned_addr learned my addr 172.21.15.32:0/3465390758 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:56.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.014+0000 7ff7c90cf640 1 -- 172.21.15.32:0/3465390758 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7c414fc10 msgr2=0x7ff7c4160e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:56.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.014+0000 7ff7c90cf640 1 --2- 172.21.15.32:0/3465390758 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7c414fc10 0x7ff7c4160e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:56.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.014+0000 7ff7c90cf640 1 -- 172.21.15.32:0/3465390758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7b40039b0 msgr2=0x7ff7c40cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:56.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.014+0000 7ff7c90cf640 1 --2- 172.21.15.32:0/3465390758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7b40039b0 0x7ff7c40cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:56.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.014+0000 7ff7c90cf640 1 -- 172.21.15.32:0/3465390758 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7c40cc6a0 con 0x7ff7c41f4600 2024-04-24T23:08:56.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.014+0000 7ff7c90cf640 1 --2- 172.21.15.32:0/3465390758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7c41f4600 0x7ff7c41f4a80 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7ff7c40d0930 tx=0x7ff7bc007b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:56.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.014+0000 7ff7c17fa640 1 -- 172.21.15.32:0/3465390758 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc00aac0 con 0x7ff7c41f4600 2024-04-24T23:08:56.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.014+0000 7ff7c17fa640 1 -- 172.21.15.32:0/3465390758 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7bc019070 con 0x7ff7c41f4600 2024-04-24T23:08:56.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.014+0000 7ff7cab32640 1 -- 172.21.15.32:0/3465390758 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7c40cc980 con 0x7ff7c41f4600 2024-04-24T23:08:56.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.014+0000 7ff7c17fa640 1 -- 172.21.15.32:0/3465390758 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc0133f0 con 0x7ff7c41f4600 2024-04-24T23:08:56.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.014+0000 7ff7cab32640 1 -- 172.21.15.32:0/3465390758 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7c40ccf40 con 0x7ff7c41f4600 2024-04-24T23:08:56.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.015+0000 7ff7cab32640 1 -- 172.21.15.32:0/3465390758 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7c4068430 con 0x7ff7c41f4600 2024-04-24T23:08:56.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.015+0000 7ff7c17fa640 1 -- 172.21.15.32:0/3465390758 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff7bc013590 con 0x7ff7c41f4600 2024-04-24T23:08:56.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.016+0000 7ff7c17fa640 1 --2- 172.21.15.32:0/3465390758 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff7a806be60 0x7ff7a806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:56.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.016+0000 7ff7c17fa640 1 -- 172.21.15.32:0/3465390758 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 4943+0+0 (secure 0 0 0) 0x7ff7bc08af30 con 0x7ff7c41f4600 2024-04-24T23:08:56.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.016+0000 7ff7c88ce640 1 --2- 172.21.15.32:0/3465390758 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff7a806be60 0x7ff7a806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:56.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.017+0000 7ff7c88ce640 1 --2- 172.21.15.32:0/3465390758 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff7a806be60 0x7ff7a806e320 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8032000 tx=0x7ff7b802c000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:56.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.020+0000 7ff7c17fa640 1 -- 172.21.15.32:0/3465390758 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff7bc059c10 con 0x7ff7c41f4600 2024-04-24T23:08:56.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.241+0000 7ff7cab32640 1 -- 172.21.15.32:0/3465390758 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7c415dee0 con 0x7ff7c41f4600 2024-04-24T23:08:56.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.242+0000 7ff7c17fa640 1 -- 172.21.15.32:0/3465390758 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+3872 (secure 0 0 0) 0x7ff7bc059510 con 0x7ff7c41f4600 2024-04-24T23:08:56.244 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:08:56.245 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":67,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7120,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/1196305174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1196305174},{"type":"v1","addr":"172.21.15.121:6836","nonce":1196305174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7123,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3015902884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3015902884},{"type":"v1","addr":"172.21.15.121:6837","nonce":3015902884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":8280,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/1033273628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1033273628},{"type":"v1","addr":"172.21.15.32:6833","nonce":1033273628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":8595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/933272515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":933272515},{"type":"v1","addr":"172.21.15.32:6835","nonce":933272515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[]} 2024-04-24T23:08:56.245 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 67 2024-04-24T23:08:56.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.244+0000 7ff7cab32640 1 -- 172.21.15.32:0/3465390758 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff7a806be60 msgr2=0x7ff7a806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:56.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.244+0000 7ff7cab32640 1 --2- 172.21.15.32:0/3465390758 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff7a806be60 0x7ff7a806e320 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8032000 tx=0x7ff7b802c000 comp rx=0 tx=0).stop 2024-04-24T23:08:56.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.244+0000 7ff7cab32640 1 -- 172.21.15.32:0/3465390758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7c41f4600 msgr2=0x7ff7c41f4a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:56.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.244+0000 7ff7cab32640 1 --2- 172.21.15.32:0/3465390758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7c41f4600 0x7ff7c41f4a80 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7ff7c40d0930 tx=0x7ff7bc007b00 comp rx=0 tx=0).stop 2024-04-24T23:08:56.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.244+0000 7ff7cab32640 1 -- 172.21.15.32:0/3465390758 shutdown_connections 2024-04-24T23:08:56.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.244+0000 7ff7cab32640 1 --2- 172.21.15.32:0/3465390758 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff7a806be60 0x7ff7a806e320 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:56.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.244+0000 7ff7cab32640 1 --2- 172.21.15.32:0/3465390758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7c41f4600 0x7ff7c41f4a80 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:56.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.244+0000 7ff7cab32640 1 --2- 172.21.15.32:0/3465390758 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7c414fc10 0x7ff7c4160e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:56.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.244+0000 7ff7cab32640 1 --2- 172.21.15.32:0/3465390758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7b40039b0 0x7ff7c40cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:56.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.244+0000 7ff7cab32640 1 -- 172.21.15.32:0/3465390758 >> 172.21.15.32:0/3465390758 conn(0x7ff7c40d11d0 msgr2=0x7ff7c415d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:56.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.244+0000 7ff7cab32640 1 -- 172.21.15.32:0/3465390758 shutdown_connections 2024-04-24T23:08:56.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.244+0000 7ff7cab32640 1 -- 172.21.15.32:0/3465390758 wait complete. 2024-04-24T23:08:56.258 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-24T23:08:56.258 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-24T23:08:56.258 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-24T23:08:56.258 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:08:56.258 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-24T23:08:56.258 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-24T23:08:56.258 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-24T23:08:56.259 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:08:56.259 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-24T23:08:56.259 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-24T23:08:56.259 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-24T23:08:56.259 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:08:56.259 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-24T23:08:56.259 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-24T23:08:56.260 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-24T23:08:56.260 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:08:56.267 INFO:tasks.ceph.mds.a:Stopped 2024-04-24T23:08:56.268 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-24T23:08:56.269 INFO:tasks.ceph.mds.c:Stopped 2024-04-24T23:08:56.269 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-24T23:08:56.271 INFO:tasks.ceph.mds.d:Stopped 2024-04-24T23:08:56.271 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-24T23:08:56.271 INFO:tasks.ceph.mds.b:Stopped 2024-04-24T23:08:56.272 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-24T23:08:56.310 INFO:tasks.ceph.mds.a:Started 2024-04-24T23:08:56.310 DEBUG:teuthology.parallel:result is None 2024-04-24T23:08:56.311 INFO:tasks.ceph.mds.c:Started 2024-04-24T23:08:56.311 DEBUG:teuthology.parallel:result is None 2024-04-24T23:08:56.313 INFO:tasks.ceph.mds.d:Started 2024-04-24T23:08:56.314 DEBUG:teuthology.parallel:result is None 2024-04-24T23:08:56.314 INFO:tasks.ceph.mds.b:Started 2024-04-24T23:08:56.314 DEBUG:teuthology.parallel:result is None 2024-04-24T23:08:56.315 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-24T23:08:56.386 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2024-04-24T23:08:56.387 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2024-04-24T23:08:56.393 INFO:tasks.ceph.mds.b.smithi121.stdout:starting mds.b at 2024-04-24T23:08:56.393 INFO:tasks.ceph.mds.d.smithi121.stdout:starting mds.d at 2024-04-24T23:08:56.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.418+0000 7f7e4079e640 1 Processor -- start 2024-04-24T23:08:56.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.418+0000 7f7e4079e640 1 -- start start 2024-04-24T23:08:56.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.419+0000 7f7e4079e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e3815e9b0 0x7f7e380d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:56.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.419+0000 7f7e4079e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7e3806b820 0x7f7e380b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:56.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.419+0000 7f7e3ed3b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7e3806b820 0x7f7e380b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:56.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.419+0000 7f7e3ed3b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7e3806b820 0x7f7e380b78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45544/0 (socket says 172.21.15.32:45544) 2024-04-24T23:08:56.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.419+0000 7f7e3dd39640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7e380d4210 0x7f7e3806b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55004/0 (socket says 172.21.15.32:55004) 2024-04-24T23:08:56.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.419+0000 7f7e3ed3b640 1 -- 172.21.15.32:0/173787963 learned_addr learned my addr 172.21.15.32:0/173787963 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:56.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.419+0000 7f7e4079e640 1 -- 172.21.15.32:0/173787963 --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e380d3170 con 0x7f7e380d4210 2024-04-24T23:08:56.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.419+0000 7f7e4079e640 1 -- 172.21.15.32:0/173787963 --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f7e380d3450 con 0x7f7e3806b820 2024-04-24T23:08:56.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.419+0000 7f7e3e53a640 1 --2- 172.21.15.32:0/173787963 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e3815e9b0 0x7f7e380d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:56.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.419+0000 7f7e4079e640 1 -- 172.21.15.32:0/173787963 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f7e380d3730 con 0x7f7e3815e9b0 2024-04-24T23:08:56.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.419+0000 7f7e3d538640 1 -- 172.21.15.32:0/173787963 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2390021832 0 0) 0x7f7e380d3170 con 0x7f7e380d4210 2024-04-24T23:08:56.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.420+0000 7f7e3d538640 1 -- 172.21.15.32:0/173787963 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e24003520 con 0x7f7e380d4210 2024-04-24T23:08:56.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.420+0000 7f7e3e53a640 1 -- 172.21.15.32:0/173787963 >> v2:172.21.15.32:3300/0 conn(0x7f7e3806b820 msgr2=0x7f7e380b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:56.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.420+0000 7f7e3e53a640 1 --2- 172.21.15.32:0/173787963 >> v2:172.21.15.32:3300/0 conn(0x7f7e3806b820 0x7f7e380b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:56.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.420+0000 7f7e3e53a640 1 -- 172.21.15.32:0/173787963 >> v1:172.21.15.32:6789/0 conn(0x7f7e380d4210 legacy=0x7f7e3806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:56.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.420+0000 7f7e3e53a640 1 -- 172.21.15.32:0/173787963 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e380d3a10 con 0x7f7e3815e9b0 2024-04-24T23:08:56.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.420+0000 7f7e3e53a640 1 --2- 172.21.15.32:0/173787963 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e3815e9b0 0x7f7e380d2b90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7e340097a0 tx=0x7f7e3402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75f6d88b6a2bbc12 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:56.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.421+0000 7f7e3d538640 1 -- 172.21.15.32:0/173787963 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e34037070 con 0x7f7e3815e9b0 2024-04-24T23:08:56.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.421+0000 7f7e3d538640 1 -- 172.21.15.32:0/173787963 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e34032440 con 0x7f7e3815e9b0 2024-04-24T23:08:56.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.421+0000 7f7e3d538640 1 -- 172.21.15.32:0/173787963 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e34032810 con 0x7f7e3815e9b0 2024-04-24T23:08:56.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.421+0000 7f7e4079e640 1 -- 172.21.15.32:0/173787963 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e3815e9b0 msgr2=0x7f7e380d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:56.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.421+0000 7f7e4079e640 1 --2- 172.21.15.32:0/173787963 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e3815e9b0 0x7f7e380d2b90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7e340097a0 tx=0x7f7e3402a180 comp rx=0 tx=0).stop 2024-04-24T23:08:56.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.421+0000 7f7e4079e640 1 -- 172.21.15.32:0/173787963 shutdown_connections 2024-04-24T23:08:56.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.421+0000 7f7e4079e640 1 --2- 172.21.15.32:0/173787963 >> v2:172.21.15.32:3300/0 conn(0x7f7e3806b820 0x7f7e380b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:56.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.421+0000 7f7e4079e640 1 --2- 172.21.15.32:0/173787963 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e3815e9b0 0x7f7e380d2b90 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:56.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.421+0000 7f7e4079e640 1 -- 172.21.15.32:0/173787963 >> 172.21.15.32:0/173787963 conn(0x7f7e380d11d0 msgr2=0x7f7e380d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:56.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.421+0000 7f7e4079e640 1 -- 172.21.15.32:0/173787963 shutdown_connections 2024-04-24T23:08:56.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.421+0000 7f7e4079e640 1 -- 172.21.15.32:0/173787963 wait complete. 2024-04-24T23:08:56.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.422+0000 7f7e4079e640 1 Processor -- start 2024-04-24T23:08:56.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.422+0000 7f7e4079e640 1 -- start start 2024-04-24T23:08:56.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.423+0000 7f7e4079e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e3806b820 0x7f7e3815d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:56.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.423+0000 7f7e4079e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e380d4210 0x7f7e380cb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:56.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.423+0000 7f7e4079e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e3815e9b0 0x7f7e380cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:56.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.423+0000 7f7e4079e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7e380ba0e0 con 0x7f7e3806b820 2024-04-24T23:08:56.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.423+0000 7f7e4079e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7e380d2400 con 0x7f7e380d4210 2024-04-24T23:08:56.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.423+0000 7f7e4079e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f7e380ba260 con 0x7f7e3815e9b0 2024-04-24T23:08:56.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.423+0000 7f7e3e53a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e3806b820 0x7f7e3815d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:56.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.423+0000 7f7e3e53a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e3806b820 0x7f7e3815d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45550/0 (socket says 172.21.15.32:45550) 2024-04-24T23:08:56.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.423+0000 7f7e3e53a640 1 -- 172.21.15.32:0/424684234 learned_addr learned my addr 172.21.15.32:0/424684234 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:56.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.423+0000 7f7e3ed3b640 1 --2- 172.21.15.32:0/424684234 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e3815e9b0 0x7f7e380cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:56.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.423+0000 7f7e3dd39640 1 --2- 172.21.15.32:0/424684234 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e380d4210 0x7f7e380cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:56.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.423+0000 7f7e3e53a640 1 -- 172.21.15.32:0/424684234 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e3815e9b0 msgr2=0x7f7e380cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:56.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.423+0000 7f7e3e53a640 1 --2- 172.21.15.32:0/424684234 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e3815e9b0 0x7f7e380cbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:56.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.423+0000 7f7e3e53a640 1 -- 172.21.15.32:0/424684234 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e380d4210 msgr2=0x7f7e380cb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:56.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.423+0000 7f7e3e53a640 1 --2- 172.21.15.32:0/424684234 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e380d4210 0x7f7e380cb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:56.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.424+0000 7f7e3e53a640 1 -- 172.21.15.32:0/424684234 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e380cc310 con 0x7f7e3806b820 2024-04-24T23:08:56.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.424+0000 7f7e3ed3b640 1 --2- 172.21.15.32:0/424684234 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e3815e9b0 0x7f7e380cbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:08:56.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.424+0000 7f7e3dd39640 1 --2- 172.21.15.32:0/424684234 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e380d4210 0x7f7e380cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:08:56.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.424+0000 7f7e3e53a640 1 --2- 172.21.15.32:0/424684234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e3806b820 0x7f7e3815d5a0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f7e340098c0 tx=0x7f7e340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:56.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.424+0000 7f7e2effd640 1 -- 172.21.15.32:0/424684234 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e34041020 con 0x7f7e3806b820 2024-04-24T23:08:56.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.424+0000 7f7e2effd640 1 -- 172.21.15.32:0/424684234 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e3403d640 con 0x7f7e3806b820 2024-04-24T23:08:56.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.424+0000 7f7e4079e640 1 -- 172.21.15.32:0/424684234 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e380cce60 con 0x7f7e3806b820 2024-04-24T23:08:56.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.424+0000 7f7e2effd640 1 -- 172.21.15.32:0/424684234 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e34037040 con 0x7f7e3806b820 2024-04-24T23:08:56.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.424+0000 7f7e4079e640 1 -- 172.21.15.32:0/424684234 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e380ccb80 con 0x7f7e3806b820 2024-04-24T23:08:56.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.425+0000 7f7e2cff9640 1 -- 172.21.15.32:0/424684234 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e38068430 con 0x7f7e3806b820 2024-04-24T23:08:56.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.425+0000 7f7e2effd640 1 -- 172.21.15.32:0/424684234 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7e34044020 con 0x7f7e3806b820 2024-04-24T23:08:56.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.426+0000 7f7e2effd640 1 --2- 172.21.15.32:0/424684234 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7e1c06be30 0x7f7e1c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:56.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.426+0000 7f7e2effd640 1 -- 172.21.15.32:0/424684234 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 4943+0+0 (secure 0 0 0) 0x7f7e340ab350 con 0x7f7e3806b820 2024-04-24T23:08:56.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.426+0000 7f7e3dd39640 1 --2- 172.21.15.32:0/424684234 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7e1c06be30 0x7f7e1c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:56.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.427+0000 7f7e3dd39640 1 --2- 172.21.15.32:0/424684234 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7e1c06be30 0x7f7e1c06e2f0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f7e380cfde0 tx=0x7f7e28007b90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:56.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.430+0000 7f7e2effd640 1 -- 172.21.15.32:0/424684234 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7e3404b030 con 0x7f7e3806b820 2024-04-24T23:08:56.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:56.624+0000 7f7e2cff9640 1 -- 172.21.15.32:0/424684234 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f7e380cfe10 con 0x7f7e3806b820 2024-04-24T23:08:57.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.389+0000 7f7e2effd640 1 -- 172.21.15.32:0/424684234 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v163) ==== 97+0+0 (secure 0 0 0) 0x7f7e34079f70 con 0x7f7e3806b820 2024-04-24T23:08:57.392 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blocklist entries 2024-04-24T23:08:57.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.392+0000 7f7e2cff9640 1 -- 172.21.15.32:0/424684234 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7e1c06be30 msgr2=0x7f7e1c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:57.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.392+0000 7f7e2cff9640 1 --2- 172.21.15.32:0/424684234 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7e1c06be30 0x7f7e1c06e2f0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f7e380cfde0 tx=0x7f7e28007b90 comp rx=0 tx=0).stop 2024-04-24T23:08:57.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.392+0000 7f7e2cff9640 1 -- 172.21.15.32:0/424684234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e3806b820 msgr2=0x7f7e3815d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:57.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.392+0000 7f7e2cff9640 1 --2- 172.21.15.32:0/424684234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e3806b820 0x7f7e3815d5a0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f7e340098c0 tx=0x7f7e340046c0 comp rx=0 tx=0).stop 2024-04-24T23:08:57.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.392+0000 7f7e2cff9640 1 -- 172.21.15.32:0/424684234 shutdown_connections 2024-04-24T23:08:57.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.392+0000 7f7e2cff9640 1 --2- 172.21.15.32:0/424684234 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7e1c06be30 0x7f7e1c06e2f0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:57.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.392+0000 7f7e2cff9640 1 --2- 172.21.15.32:0/424684234 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e3815e9b0 0x7f7e380cbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:57.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.392+0000 7f7e2cff9640 1 --2- 172.21.15.32:0/424684234 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e380d4210 0x7f7e380cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:57.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.392+0000 7f7e2cff9640 1 --2- 172.21.15.32:0/424684234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e3806b820 0x7f7e3815d5a0 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:57.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.392+0000 7f7e2cff9640 1 -- 172.21.15.32:0/424684234 >> 172.21.15.32:0/424684234 conn(0x7f7e380d11d0 msgr2=0x7f7e380ca290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:57.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.392+0000 7f7e2cff9640 1 -- 172.21.15.32:0/424684234 shutdown_connections 2024-04-24T23:08:57.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.392+0000 7f7e2cff9640 1 -- 172.21.15.32:0/424684234 wait complete. 2024-04-24T23:08:57.408 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-24T23:08:57.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.511+0000 7fba7f5e7640 1 Processor -- start 2024-04-24T23:08:57.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.512+0000 7fba7f5e7640 1 -- start start 2024-04-24T23:08:57.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.512+0000 7fba7f5e7640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fba7815e9b0 0x7fba780d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:57.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.512+0000 7fba7f5e7640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba780d3170 con 0x7fba7806b820 2024-04-24T23:08:57.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.512+0000 7fba7f5e7640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba780d3450 con 0x7fba780d4210 2024-04-24T23:08:57.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.512+0000 7fba7f5e7640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fba780d3730 con 0x7fba7815e9b0 2024-04-24T23:08:57.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.512+0000 7fba7d383640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fba7815e9b0 0x7fba780d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:57.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.513+0000 7fba7db84640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fba7806b820 0x7fba780b78f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55020/0 (socket says 172.21.15.32:55020) 2024-04-24T23:08:57.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.513+0000 7fba7db84640 1 -- 172.21.15.32:0/2028674424 learned_addr learned my addr 172.21.15.32:0/2028674424 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:08:57.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.513+0000 7fba7d383640 1 -- 172.21.15.32:0/2028674424 >> v1:172.21.15.121:6789/0 conn(0x7fba780d4210 legacy=0x7fba7806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:57.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.513+0000 7fba7d383640 1 -- 172.21.15.32:0/2028674424 >> v1:172.21.15.32:6789/0 conn(0x7fba7806b820 legacy=0x7fba780b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:57.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.513+0000 7fba7d383640 1 -- 172.21.15.32:0/2028674424 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba780d3a10 con 0x7fba7815e9b0 2024-04-24T23:08:57.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.513+0000 7fba6bfff640 1 -- 172.21.15.32:0/2028674424 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3342040430 0 0) 0x7fba780d3170 con 0x7fba7806b820 2024-04-24T23:08:57.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.513+0000 7fba7d383640 1 --2- 172.21.15.32:0/2028674424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba7815e9b0 0x7fba780d2b90 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fba740097a0 tx=0x7fba7402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fcadaa3de8677da server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:57.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.514+0000 7fba6bfff640 1 -- 172.21.15.32:0/2028674424 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba74037070 con 0x7fba7815e9b0 2024-04-24T23:08:57.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.514+0000 7fba6bfff640 1 -- 172.21.15.32:0/2028674424 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba7402ae50 con 0x7fba7815e9b0 2024-04-24T23:08:57.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.514+0000 7fba6bfff640 1 -- 172.21.15.32:0/2028674424 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba74032660 con 0x7fba7815e9b0 2024-04-24T23:08:57.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.514+0000 7fba7f5e7640 1 -- 172.21.15.32:0/2028674424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba7815e9b0 msgr2=0x7fba780d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:57.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.514+0000 7fba7f5e7640 1 --2- 172.21.15.32:0/2028674424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba7815e9b0 0x7fba780d2b90 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fba740097a0 tx=0x7fba7402a140 comp rx=0 tx=0).stop 2024-04-24T23:08:57.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.514+0000 7fba7f5e7640 1 -- 172.21.15.32:0/2028674424 shutdown_connections 2024-04-24T23:08:57.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.514+0000 7fba7f5e7640 1 --2- 172.21.15.32:0/2028674424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba7815e9b0 0x7fba780d2b90 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:57.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.514+0000 7fba7f5e7640 1 -- 172.21.15.32:0/2028674424 >> 172.21.15.32:0/2028674424 conn(0x7fba780d11d0 msgr2=0x7fba780d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:57.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.514+0000 7fba7f5e7640 1 -- 172.21.15.32:0/2028674424 shutdown_connections 2024-04-24T23:08:57.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.514+0000 7fba7f5e7640 1 -- 172.21.15.32:0/2028674424 wait complete. 2024-04-24T23:08:57.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.515+0000 7fba7f5e7640 1 Processor -- start 2024-04-24T23:08:57.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.516+0000 7fba7f5e7640 1 -- start start 2024-04-24T23:08:57.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.516+0000 7fba7f5e7640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fba7806b820 0x7fba781f4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:57.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.516+0000 7fba7f5e7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba780d4210 0x7fba781574b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:57.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.516+0000 7fba7f5e7640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fba7815e9b0 0x7fba781579f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:57.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.516+0000 7fba7f5e7640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fba780ba0e0 con 0x7fba780d4210 2024-04-24T23:08:57.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.516+0000 7fba7f5e7640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fba780d2400 con 0x7fba7806b820 2024-04-24T23:08:57.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.516+0000 7fba7f5e7640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fba780ba260 con 0x7fba7815e9b0 2024-04-24T23:08:57.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.516+0000 7fba7cb82640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba780d4210 0x7fba781574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:57.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.516+0000 7fba7cb82640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba780d4210 0x7fba781574b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45562/0 (socket says 172.21.15.32:45562) 2024-04-24T23:08:57.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.516+0000 7fba7cb82640 1 -- 172.21.15.32:0/567220882 learned_addr learned my addr 172.21.15.32:0/567220882 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:57.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.517+0000 7fba7d383640 1 --2- 172.21.15.32:0/567220882 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fba7806b820 0x7fba781f4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:57.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.517+0000 7fba7db84640 1 --2- 172.21.15.32:0/567220882 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fba7815e9b0 0x7fba781579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:57.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.517+0000 7fba7cb82640 1 -- 172.21.15.32:0/567220882 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fba7815e9b0 msgr2=0x7fba781579f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:57.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.517+0000 7fba7cb82640 1 --2- 172.21.15.32:0/567220882 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fba7815e9b0 0x7fba781579f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:57.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.517+0000 7fba7cb82640 1 -- 172.21.15.32:0/567220882 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fba7806b820 msgr2=0x7fba781f4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:57.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.517+0000 7fba7cb82640 1 --2- 172.21.15.32:0/567220882 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fba7806b820 0x7fba781f4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:57.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.517+0000 7fba7cb82640 1 -- 172.21.15.32:0/567220882 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba78158070 con 0x7fba780d4210 2024-04-24T23:08:57.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.517+0000 7fba7d383640 1 --2- 172.21.15.32:0/567220882 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fba7806b820 0x7fba781f4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:08:57.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.517+0000 7fba7cb82640 1 --2- 172.21.15.32:0/567220882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba780d4210 0x7fba781574b0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fba6c00c440 tx=0x7fba6c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:57.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.517+0000 7fba69ffb640 1 -- 172.21.15.32:0/567220882 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c017070 con 0x7fba780d4210 2024-04-24T23:08:57.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.517+0000 7fba69ffb640 1 -- 172.21.15.32:0/567220882 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba6c0035f0 con 0x7fba780d4210 2024-04-24T23:08:57.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.517+0000 7fba7f5e7640 1 -- 172.21.15.32:0/567220882 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba780cb9c0 con 0x7fba780d4210 2024-04-24T23:08:57.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.517+0000 7fba69ffb640 1 -- 172.21.15.32:0/567220882 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c0039e0 con 0x7fba780d4210 2024-04-24T23:08:57.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.517+0000 7fba7f5e7640 1 -- 172.21.15.32:0/567220882 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba780cbf80 con 0x7fba780d4210 2024-04-24T23:08:57.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.518+0000 7fba7f5e7640 1 -- 172.21.15.32:0/567220882 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba78068430 con 0x7fba780d4210 2024-04-24T23:08:57.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.519+0000 7fba69ffb640 1 -- 172.21.15.32:0/567220882 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fba6c01a070 con 0x7fba780d4210 2024-04-24T23:08:57.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.519+0000 7fba69ffb640 1 --2- 172.21.15.32:0/567220882 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fba5c06be30 0x7fba5c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:57.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.519+0000 7fba69ffb640 1 -- 172.21.15.32:0/567220882 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4857+0+0 (secure 0 0 0) 0x7fba6c089e00 con 0x7fba780d4210 2024-04-24T23:08:57.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.519+0000 7fba7d383640 1 --2- 172.21.15.32:0/567220882 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fba5c06be30 0x7fba5c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:57.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.520+0000 7fba7d383640 1 --2- 172.21.15.32:0/567220882 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fba5c06be30 0x7fba5c06e2f0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fba74034000 tx=0x7fba7402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:57.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.523+0000 7fba69ffb640 1 -- 172.21.15.32:0/567220882 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fba6c058b90 con 0x7fba780d4210 2024-04-24T23:08:57.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.784+0000 7fba7f5e7640 1 -- 172.21.15.32:0/567220882 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fba780cae30 con 0x7fba780d4210 2024-04-24T23:08:57.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.784+0000 7fba69ffb640 1 -- 172.21.15.32:0/567220882 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) ==== 80+0+5304 (secure 0 0 0) 0x7fba6c0010c0 con 0x7fba780d4210 2024-04-24T23:08:57.787 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:08:57.787 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:08:57.787 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2024-04-24T23:08:57.787 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:08:57.787 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2024-04-24T23:08:57.787 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmXRdxAxAA/EYEn1C38XhG7DC+0X4y7A==", 2024-04-24T23:08:57.788 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:08:57.788 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:08:57.788 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-24T23:08:57.788 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-24T23:08:57.788 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:08:57.788 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:08:57.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:08:57.788 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:08:57.788 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2024-04-24T23:08:57.788 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmak7bDhAA0he+Jp/gkkrfBjYKSObjcw==", 2024-04-24T23:08:57.788 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:08:57.788 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:08:57.788 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-24T23:08:57.789 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-24T23:08:57.789 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:08:57.789 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:08:57.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:08:57.789 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:08:57.789 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.c", 2024-04-24T23:08:57.789 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmZXFdChAA8TwwkA5EIXHBzPsQSFvXcQ==", 2024-04-24T23:08:57.789 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:08:57.789 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:08:57.789 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-24T23:08:57.789 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-24T23:08:57.789 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:08:57.789 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:08:57.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:08:57.790 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:08:57.790 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.d", 2024-04-24T23:08:57.790 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmLbGuFRAAe91fZlulziWdWiDdLBLWNA==", 2024-04-24T23:08:57.790 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:08:57.790 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:08:57.790 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-24T23:08:57.790 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-24T23:08:57.790 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:08:57.790 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:08:57.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:08:57.790 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:08:57.790 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2024-04-24T23:08:57.791 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCfjilmhTwoOBAAig98xa3a2qnv4rriUO9VCQ==", 2024-04-24T23:08:57.791 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:08:57.791 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:08:57.791 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:08:57.791 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:08:57.791 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:08:57.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:08:57.791 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:08:57.791 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2024-04-24T23:08:57.791 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCgjilmwRaAAxAAUqDOyR9HV6DEnmbyn3WAjA==", 2024-04-24T23:08:57.791 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:08:57.791 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:08:57.791 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:08:57.791 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:08:57.791 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:08:57.792 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:08:57.792 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:08:57.792 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2024-04-24T23:08:57.792 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCgjilmDl2qDBAAF1bhTOaSOc612wmFRd8+XA==", 2024-04-24T23:08:57.792 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:08:57.792 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:08:57.792 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:08:57.792 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:08:57.792 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:08:57.792 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:08:57.792 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:08:57.792 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2024-04-24T23:08:57.792 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCgjilmAxHwFRAA9a0v/YwngRxq1vmMRsmEkQ==", 2024-04-24T23:08:57.793 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:08:57.793 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:08:57.793 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:08:57.793 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:08:57.793 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:08:57.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:08:57.793 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:08:57.793 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2024-04-24T23:08:57.793 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCjjilmDPmJIxAASZlEo1ZJHOhIAFGukIj1tQ==", 2024-04-24T23:08:57.793 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:08:57.793 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:08:57.793 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:08:57.793 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:08:57.794 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:08:57.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:08:57.794 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:08:57.794 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2024-04-24T23:08:57.794 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCjjilmgcpuKhAAedSq0ubNB30s3bz0fY38hw==", 2024-04-24T23:08:57.794 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:08:57.794 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:08:57.794 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:08:57.794 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:08:57.794 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:08:57.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:08:57.794 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:08:57.794 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2024-04-24T23:08:57.794 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCjjilmFv2VMxAAM+mUc4I0sjawy2uwuUEKYg==", 2024-04-24T23:08:57.795 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:08:57.795 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:08:57.795 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:08:57.795 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:08:57.795 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:08:57.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:08:57.795 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:08:57.795 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2024-04-24T23:08:57.795 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCkjilmLvEbARAAnaOvQ5Ee0clN/A9SVoKuBQ==", 2024-04-24T23:08:57.795 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:08:57.795 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:08:57.795 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:08:57.795 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:08:57.796 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:08:57.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:08:57.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:08:57.796 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2024-04-24T23:08:57.796 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilm5oeiGRAAdLXwwqrYhV3HHo7Dw/2ZqQ==", 2024-04-24T23:08:57.796 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:08:57.796 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:08:57.796 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2024-04-24T23:08:57.796 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-24T23:08:57.796 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:08:57.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:08:57.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:08:57.796 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2024-04-24T23:08:57.797 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmyWBDIhAARtVP4Unk3HHlQYZ2QZ3GzQ==", 2024-04-24T23:08:57.797 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:08:57.797 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:08:57.797 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-24T23:08:57.797 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-24T23:08:57.797 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-24T23:08:57.797 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:08:57.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:08:57.797 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:08:57.797 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.2", 2024-04-24T23:08:57.797 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilm4QElHxAAXgE316aRCosxX7XER5gtKQ==", 2024-04-24T23:08:57.797 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:08:57.797 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:08:57.797 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-24T23:08:57.798 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-24T23:08:57.798 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-24T23:08:57.798 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:08:57.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:08:57.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:08:57.798 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.3", 2024-04-24T23:08:57.798 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmMZihJxAAoWaAAFY3vVfZnC4/GoT0+g==", 2024-04-24T23:08:57.798 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:08:57.798 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:08:57.798 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-24T23:08:57.798 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-24T23:08:57.798 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-24T23:08:57.798 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:08:57.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:08:57.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:08:57.799 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2024-04-24T23:08:57.799 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCdjilmdVXeIxAAwGGaAfshx4pHcoc5+3kRww==", 2024-04-24T23:08:57.799 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:08:57.799 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2024-04-24T23:08:57.799 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-24T23:08:57.799 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-24T23:08:57.799 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:08:57.799 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:08:57.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:08:57.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:08:57.799 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2024-04-24T23:08:57.799 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmn3FJABAAGnvNAoNLzK20cQF6beihmw==", 2024-04-24T23:08:57.799 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:08:57.800 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2024-04-24T23:08:57.800 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2024-04-24T23:08:57.800 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:08:57.800 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:08:57.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:08:57.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:08:57.800 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2024-04-24T23:08:57.800 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCdjilmMqs2OBAAOe3bLpnCq+HK2EbeXnwY8Q==", 2024-04-24T23:08:57.800 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:08:57.800 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2024-04-24T23:08:57.800 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2024-04-24T23:08:57.800 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:08:57.800 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:08:57.800 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:08:57.800 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:08:57.801 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:08:57.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.787+0000 7fba7f5e7640 1 -- 172.21.15.32:0/567220882 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fba5c06be30 msgr2=0x7fba5c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:57.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.787+0000 7fba7f5e7640 1 --2- 172.21.15.32:0/567220882 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fba5c06be30 0x7fba5c06e2f0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fba74034000 tx=0x7fba7402e000 comp rx=0 tx=0).stop 2024-04-24T23:08:57.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.787+0000 7fba7f5e7640 1 -- 172.21.15.32:0/567220882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba780d4210 msgr2=0x7fba781574b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:57.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.787+0000 7fba7f5e7640 1 --2- 172.21.15.32:0/567220882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba780d4210 0x7fba781574b0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fba6c00c440 tx=0x7fba6c00c900 comp rx=0 tx=0).stop 2024-04-24T23:08:57.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.787+0000 7fba7f5e7640 1 -- 172.21.15.32:0/567220882 shutdown_connections 2024-04-24T23:08:57.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.787+0000 7fba7f5e7640 1 --2- 172.21.15.32:0/567220882 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fba5c06be30 0x7fba5c06e2f0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:57.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.787+0000 7fba7f5e7640 1 --2- 172.21.15.32:0/567220882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba780d4210 0x7fba781574b0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:57.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.787+0000 7fba7f5e7640 1 --2- 172.21.15.32:0/567220882 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fba7815e9b0 0x7fba781579f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:57.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.787+0000 7fba7f5e7640 1 --2- 172.21.15.32:0/567220882 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fba7806b820 0x7fba781f4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:57.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.787+0000 7fba7f5e7640 1 -- 172.21.15.32:0/567220882 >> 172.21.15.32:0/567220882 conn(0x7fba780d11d0 msgr2=0x7fba780b9260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:57.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.787+0000 7fba7f5e7640 1 -- 172.21.15.32:0/567220882 shutdown_connections 2024-04-24T23:08:57.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.787+0000 7fba7f5e7640 1 -- 172.21.15.32:0/567220882 wait complete. 2024-04-24T23:08:57.803 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:08:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.901+0000 7f36e5012640 1 Processor -- start 2024-04-24T23:08:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.901+0000 7f36e5012640 1 -- start start 2024-04-24T23:08:57.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.902+0000 7f36e5012640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f36e015cb00 0x7f36e00ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:57.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.902+0000 7f36e5012640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36e00b4520 con 0x7f36e015b380 2024-04-24T23:08:57.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.902+0000 7f36e5012640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36e00b4800 con 0x7f36e006cd00 2024-04-24T23:08:57.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.902+0000 7f36e5012640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f36e00b4ae0 con 0x7f36e015cb00 2024-04-24T23:08:57.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.902+0000 7f36ded9d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f36e015b380 0x7f36e006c5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55032/0 (socket says 172.21.15.32:55032) 2024-04-24T23:08:57.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.902+0000 7f36ded9d640 1 -- 172.21.15.32:0/373821651 learned_addr learned my addr 172.21.15.32:0/373821651 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:08:57.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.902+0000 7f36df59e640 1 --2- 172.21.15.32:0/373821651 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f36e015cb00 0x7f36e00ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:57.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.902+0000 7f36ddd9b640 1 -- 172.21.15.32:0/373821651 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3010574432 0 0) 0x7f36e00b4520 con 0x7f36e015b380 2024-04-24T23:08:57.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.902+0000 7f36ddd9b640 1 -- 172.21.15.32:0/373821651 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f36cc003520 con 0x7f36e015b380 2024-04-24T23:08:57.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.902+0000 7f36ddd9b640 1 -- 172.21.15.32:0/373821651 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2730039921 0 0) 0x7f36e00b4800 con 0x7f36e006cd00 2024-04-24T23:08:57.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.902+0000 7f36ddd9b640 1 -- 172.21.15.32:0/373821651 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f36e00b4520 con 0x7f36e006cd00 2024-04-24T23:08:57.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.903+0000 7f36ddd9b640 1 -- 172.21.15.32:0/373821651 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2795772839 0 0) 0x7f36cc003520 con 0x7f36e015b380 2024-04-24T23:08:57.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.903+0000 7f36ddd9b640 1 -- 172.21.15.32:0/373821651 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f36e015cb00 msgr2=0x7f36e00ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:57.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.903+0000 7f36ddd9b640 1 --2- 172.21.15.32:0/373821651 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f36e015cb00 0x7f36e00ceb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:57.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.903+0000 7f36ddd9b640 1 -- 172.21.15.32:0/373821651 >> v1:172.21.15.121:6789/0 conn(0x7f36e006cd00 legacy=0x7f36e00ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:57.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.903+0000 7f36ddd9b640 1 -- 172.21.15.32:0/373821651 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36e00b4dc0 con 0x7f36e015b380 2024-04-24T23:08:57.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.903+0000 7f36ddd9b640 1 -- 172.21.15.32:0/373821651 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f36d8001670 con 0x7f36e015b380 2024-04-24T23:08:57.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.903+0000 7f36df59e640 1 --2- 172.21.15.32:0/373821651 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f36e015cb00 0x7f36e00ceb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:08:57.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.903+0000 7f36ddd9b640 1 -- 172.21.15.32:0/373821651 >> v1:172.21.15.32:6789/0 conn(0x7f36e015b380 legacy=0x7f36e006c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:57.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.903+0000 7f36ddd9b640 1 --2- 172.21.15.32:0/373821651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36cc003b70 0x7f36cc003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:57.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.903+0000 7f36ddd9b640 1 --2- 172.21.15.32:0/373821651 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f36cc004800 0x7f36cc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:57.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.903+0000 7f36ddd9b640 1 --2- 172.21.15.32:0/373821651 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f36e015cb00 0x7f36cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:57.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.903+0000 7f36ddd9b640 1 -- 172.21.15.32:0/373821651 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f36e00b4ae0 con 0x7f36cc003b70 2024-04-24T23:08:57.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.903+0000 7f36ddd9b640 1 -- 172.21.15.32:0/373821651 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f36cc00baa0 con 0x7f36cc004800 2024-04-24T23:08:57.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.903+0000 7f36ddd9b640 1 -- 172.21.15.32:0/373821651 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f36cc00bca0 con 0x7f36e015cb00 2024-04-24T23:08:57.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.903+0000 7f36ded9d640 1 --2- 172.21.15.32:0/373821651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36cc003b70 0x7f36cc003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:57.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.903+0000 7f36de59c640 1 --2- 172.21.15.32:0/373821651 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f36cc004800 0x7f36cc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:57.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.903+0000 7f36df59e640 1 --2- 172.21.15.32:0/373821651 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f36e015cb00 0x7f36cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:57.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.903+0000 7f36ded9d640 1 -- 172.21.15.32:0/373821651 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f36e015cb00 msgr2=0x7f36cc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:57.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.903+0000 7f36ded9d640 1 --2- 172.21.15.32:0/373821651 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f36e015cb00 0x7f36cc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:57.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.903+0000 7f36ded9d640 1 -- 172.21.15.32:0/373821651 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f36cc004800 msgr2=0x7f36cc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:57.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.903+0000 7f36ded9d640 1 --2- 172.21.15.32:0/373821651 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f36cc004800 0x7f36cc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:57.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.903+0000 7f36ded9d640 1 -- 172.21.15.32:0/373821651 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36cc003520 con 0x7f36cc003b70 2024-04-24T23:08:57.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.904+0000 7f36ded9d640 1 --2- 172.21.15.32:0/373821651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36cc003b70 0x7f36cc003f70 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f36d8001eb0 tx=0x7f36cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=567ebd31db74a5c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:57.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.904+0000 7f36ddd9b640 1 -- 172.21.15.32:0/373821651 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d8007ce0 con 0x7f36cc003b70 2024-04-24T23:08:57.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.904+0000 7f36ddd9b640 1 -- 172.21.15.32:0/373821651 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36d8005ad0 con 0x7f36cc003b70 2024-04-24T23:08:57.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.904+0000 7f36ddd9b640 1 -- 172.21.15.32:0/373821651 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d8007070 con 0x7f36cc003b70 2024-04-24T23:08:57.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.904+0000 7f36e5012640 1 -- 172.21.15.32:0/373821651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36cc003b70 msgr2=0x7f36cc003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:57.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.904+0000 7f36e5012640 1 --2- 172.21.15.32:0/373821651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36cc003b70 0x7f36cc003f70 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f36d8001eb0 tx=0x7f36cc005830 comp rx=0 tx=0).stop 2024-04-24T23:08:57.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.904+0000 7f36df59e640 1 -- 172.21.15.32:0/373821651 reap_dead start 2024-04-24T23:08:57.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.904+0000 7f36e5012640 1 -- 172.21.15.32:0/373821651 shutdown_connections 2024-04-24T23:08:57.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.904+0000 7f36e5012640 1 -- 172.21.15.32:0/373821651 >> 172.21.15.32:0/373821651 conn(0x7f36e006b110 msgr2=0x7f36e006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:57.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.904+0000 7f36e5012640 1 -- 172.21.15.32:0/373821651 shutdown_connections 2024-04-24T23:08:57.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.904+0000 7f36e5012640 1 -- 172.21.15.32:0/373821651 wait complete. 2024-04-24T23:08:57.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.906+0000 7f36e5012640 1 Processor -- start 2024-04-24T23:08:57.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.906+0000 7f36e5012640 1 -- start start 2024-04-24T23:08:57.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.907+0000 7f36e5012640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f36cc003b70 0x7f36e0149c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:57.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.907+0000 7f36e5012640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f36e0071e80 0x7f36e00b1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:57.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.907+0000 7f36e5012640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36e00b2ff0 0x7f36e00b3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:57.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.907+0000 7f36e5012640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f36e00ceb40 con 0x7f36e00b2ff0 2024-04-24T23:08:57.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.907+0000 7f36e5012640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f36e015b760 con 0x7f36e0071e80 2024-04-24T23:08:57.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.907+0000 7f36e5012640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f36e00cecc0 con 0x7f36cc003b70 2024-04-24T23:08:57.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.907+0000 7f36df59e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36e00b2ff0 0x7f36e00b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:57.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.907+0000 7f36df59e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36e00b2ff0 0x7f36e00b3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45582/0 (socket says 172.21.15.32:45582) 2024-04-24T23:08:57.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.907+0000 7f36df59e640 1 -- 172.21.15.32:0/530184441 learned_addr learned my addr 172.21.15.32:0/530184441 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:57.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.907+0000 7f36de59c640 1 --2- 172.21.15.32:0/530184441 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f36e0071e80 0x7f36e00b1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:57.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.907+0000 7f36ded9d640 1 --2- 172.21.15.32:0/530184441 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f36cc003b70 0x7f36e0149c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:57.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.907+0000 7f36df59e640 1 -- 172.21.15.32:0/530184441 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f36cc003b70 msgr2=0x7f36e0149c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:57.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.907+0000 7f36df59e640 1 --2- 172.21.15.32:0/530184441 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f36cc003b70 0x7f36e0149c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:57.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.907+0000 7f36df59e640 1 -- 172.21.15.32:0/530184441 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f36e0071e80 msgr2=0x7f36e00b1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:57.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.907+0000 7f36df59e640 1 --2- 172.21.15.32:0/530184441 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f36e0071e80 0x7f36e00b1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:57.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.907+0000 7f36df59e640 1 -- 172.21.15.32:0/530184441 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36e0070820 con 0x7f36e00b2ff0 2024-04-24T23:08:57.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.907+0000 7f36df59e640 1 --2- 172.21.15.32:0/530184441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36e00b2ff0 0x7f36e00b3470 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f36cc004780 tx=0x7f36d000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:57.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.907+0000 7f36d77fe640 1 -- 172.21.15.32:0/530184441 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d0003990 con 0x7f36e00b2ff0 2024-04-24T23:08:57.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.907+0000 7f36d77fe640 1 -- 172.21.15.32:0/530184441 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36d0003b30 con 0x7f36e00b2ff0 2024-04-24T23:08:57.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.907+0000 7f36e5012640 1 -- 172.21.15.32:0/530184441 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36e0070b00 con 0x7f36e00b2ff0 2024-04-24T23:08:57.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.907+0000 7f36d77fe640 1 -- 172.21.15.32:0/530184441 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d0004020 con 0x7f36e00b2ff0 2024-04-24T23:08:57.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.907+0000 7f36e5012640 1 -- 172.21.15.32:0/530184441 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36e00715e0 con 0x7f36e00b2ff0 2024-04-24T23:08:57.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.908+0000 7f36e5012640 1 -- 172.21.15.32:0/530184441 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36e0068430 con 0x7f36e00b2ff0 2024-04-24T23:08:57.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.909+0000 7f36d77fe640 1 -- 172.21.15.32:0/530184441 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f36d00041c0 con 0x7f36e00b2ff0 2024-04-24T23:08:57.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.909+0000 7f36d77fe640 1 --2- 172.21.15.32:0/530184441 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f36c006be30 0x7f36c006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:57.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.910+0000 7f36d77fe640 1 -- 172.21.15.32:0/530184441 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4857+0+0 (secure 0 0 0) 0x7f36d008b5b0 con 0x7f36e00b2ff0 2024-04-24T23:08:57.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.910+0000 7f36ded9d640 1 --2- 172.21.15.32:0/530184441 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f36c006be30 0x7f36c006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:57.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.910+0000 7f36ded9d640 1 --2- 172.21.15.32:0/530184441 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f36c006be30 0x7f36c006e2f0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f36d8032000 tx=0x7f36d8034040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:57.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:57.914+0000 7f36d77fe640 1 -- 172.21.15.32:0/530184441 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f36d0019030 con 0x7f36e00b2ff0 2024-04-24T23:08:58.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.097+0000 7f36e5012640 1 -- 172.21.15.32:0/530184441 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f36e006c000 con 0x7f36e00b2ff0 2024-04-24T23:08:58.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.098+0000 7f36d77fe640 1 -- 172.21.15.32:0/530184441 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v165) ==== 74+0+12914 (secure 0 0 0) 0x7f36d0010040 con 0x7f36e00b2ff0 2024-04-24T23:08:58.100 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:08:58.100 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":165,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:08:57.397308+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:08:58.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.100+0000 7f36e5012640 1 -- 172.21.15.32:0/530184441 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f36c006be30 msgr2=0x7f36c006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:58.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.100+0000 7f36e5012640 1 --2- 172.21.15.32:0/530184441 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f36c006be30 0x7f36c006e2f0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f36d8032000 tx=0x7f36d8034040 comp rx=0 tx=0).stop 2024-04-24T23:08:58.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.100+0000 7f36e5012640 1 -- 172.21.15.32:0/530184441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36e00b2ff0 msgr2=0x7f36e00b3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:58.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.100+0000 7f36e5012640 1 --2- 172.21.15.32:0/530184441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36e00b2ff0 0x7f36e00b3470 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f36cc004780 tx=0x7f36d000f580 comp rx=0 tx=0).stop 2024-04-24T23:08:58.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.100+0000 7f36e5012640 1 -- 172.21.15.32:0/530184441 shutdown_connections 2024-04-24T23:08:58.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.100+0000 7f36e5012640 1 --2- 172.21.15.32:0/530184441 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f36c006be30 0x7f36c006e2f0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:58.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.100+0000 7f36e5012640 1 --2- 172.21.15.32:0/530184441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36e00b2ff0 0x7f36e00b3470 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:58.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.100+0000 7f36e5012640 1 --2- 172.21.15.32:0/530184441 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f36e0071e80 0x7f36e00b1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:58.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.100+0000 7f36e5012640 1 --2- 172.21.15.32:0/530184441 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f36cc003b70 0x7f36e0149c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:58.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.100+0000 7f36e5012640 1 -- 172.21.15.32:0/530184441 >> 172.21.15.32:0/530184441 conn(0x7f36e006b110 msgr2=0x7f36e00cf240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:58.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.100+0000 7f36e5012640 1 -- 172.21.15.32:0/530184441 shutdown_connections 2024-04-24T23:08:58.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.100+0000 7f36e5012640 1 -- 172.21.15.32:0/530184441 wait complete. 2024-04-24T23:08:58.114 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:08:58.114 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:08:58.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.215+0000 7f020a190640 1 Processor -- start 2024-04-24T23:08:58.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.215+0000 7f020a190640 1 -- start start 2024-04-24T23:08:58.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.215+0000 7f020a190640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f020406c5f0 0x7f020406c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:58.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.215+0000 7f020a190640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f02040af890 0x7f020415cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:58.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.215+0000 7f020a190640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02040b6430 con 0x7f02040b44d0 2024-04-24T23:08:58.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.215+0000 7f020a190640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f0204156f80 con 0x7f020406c5f0 2024-04-24T23:08:58.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.215+0000 7f020a190640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f02040b6150 con 0x7f02040af890 2024-04-24T23:08:58.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.216+0000 7f02037fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f020406c5f0 0x7f020406c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:58.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.216+0000 7f02037fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f020406c5f0 0x7f020406c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45584/0 (socket says 172.21.15.32:45584) 2024-04-24T23:08:58.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.216+0000 7f02037fe640 1 -- 172.21.15.32:0/2168602200 learned_addr learned my addr 172.21.15.32:0/2168602200 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:58.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.216+0000 7f0203fff640 1 --2- 172.21.15.32:0/2168602200 >> v2:172.21.15.121:3300/0 conn(0x7f02040af890 0x7f020415cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:58.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.216+0000 7f02037fe640 1 -- 172.21.15.32:0/2168602200 >> v2:172.21.15.121:3300/0 conn(0x7f02040af890 msgr2=0x7f020415cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:58.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.216+0000 7f02037fe640 1 --2- 172.21.15.32:0/2168602200 >> v2:172.21.15.121:3300/0 conn(0x7f02040af890 0x7f020415cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:58.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.216+0000 7f02037fe640 1 -- 172.21.15.32:0/2168602200 >> v1:172.21.15.121:6789/0 conn(0x7f02040b44d0 legacy=0x7f02040b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:58.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.216+0000 7f02037fe640 1 -- 172.21.15.32:0/2168602200 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02040b39f0 con 0x7f020406c5f0 2024-04-24T23:08:58.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.216+0000 7f0203fff640 1 --2- 172.21.15.32:0/2168602200 >> v2:172.21.15.121:3300/0 conn(0x7f02040af890 0x7f020415cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:08:58.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.217+0000 7f02037fe640 1 --2- 172.21.15.32:0/2168602200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f020406c5f0 0x7f020406c9d0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f0204073960 tx=0x7f01f402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b062630755238e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:58.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.217+0000 7f02027fc640 1 -- 172.21.15.32:0/2168602200 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f4037070 con 0x7f020406c5f0 2024-04-24T23:08:58.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.217+0000 7f02027fc640 1 -- 172.21.15.32:0/2168602200 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01f4032440 con 0x7f020406c5f0 2024-04-24T23:08:58.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.217+0000 7f02027fc640 1 -- 172.21.15.32:0/2168602200 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f4032810 con 0x7f020406c5f0 2024-04-24T23:08:58.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.217+0000 7f020a190640 1 -- 172.21.15.32:0/2168602200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f020406c5f0 msgr2=0x7f020406c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:58.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.217+0000 7f020a190640 1 --2- 172.21.15.32:0/2168602200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f020406c5f0 0x7f020406c9d0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f0204073960 tx=0x7f01f402a270 comp rx=0 tx=0).stop 2024-04-24T23:08:58.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.217+0000 7f020a190640 1 -- 172.21.15.32:0/2168602200 shutdown_connections 2024-04-24T23:08:58.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.217+0000 7f020a190640 1 --2- 172.21.15.32:0/2168602200 >> v2:172.21.15.121:3300/0 conn(0x7f02040af890 0x7f020415cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:58.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.217+0000 7f020a190640 1 --2- 172.21.15.32:0/2168602200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f020406c5f0 0x7f020406c9d0 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:58.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.217+0000 7f020a190640 1 -- 172.21.15.32:0/2168602200 >> 172.21.15.32:0/2168602200 conn(0x7f020406b110 msgr2=0x7f020406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:58.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.217+0000 7f020a190640 1 -- 172.21.15.32:0/2168602200 shutdown_connections 2024-04-24T23:08:58.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.217+0000 7f020a190640 1 -- 172.21.15.32:0/2168602200 wait complete. 2024-04-24T23:08:58.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.219+0000 7f020a190640 1 Processor -- start 2024-04-24T23:08:58.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.219+0000 7f020a190640 1 -- start start 2024-04-24T23:08:58.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.219+0000 7f020a190640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f020406c5f0 0x7f0204153050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:58.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.219+0000 7f020a190640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02040af890 0x7f0204146c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:58.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.220+0000 7f020a190640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02040b44d0 0x7f0204151760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:58.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.220+0000 7f020a190640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f020406cf10 con 0x7f02040af890 2024-04-24T23:08:58.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.220+0000 7f020a190640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f02040b3fb0 con 0x7f020406c5f0 2024-04-24T23:08:58.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.220+0000 7f020a190640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f020415d000 con 0x7f02040b44d0 2024-04-24T23:08:58.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.220+0000 7f0202ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02040af890 0x7f0204146c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:58.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.220+0000 7f0202ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02040af890 0x7f0204146c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45592/0 (socket says 172.21.15.32:45592) 2024-04-24T23:08:58.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.220+0000 7f0202ffd640 1 -- 172.21.15.32:0/1524236771 learned_addr learned my addr 172.21.15.32:0/1524236771 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:58.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.220+0000 7f02037fe640 1 --2- 172.21.15.32:0/1524236771 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f020406c5f0 0x7f0204153050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:58.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.220+0000 7f0203fff640 1 --2- 172.21.15.32:0/1524236771 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02040b44d0 0x7f0204151760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:58.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.220+0000 7f0202ffd640 1 -- 172.21.15.32:0/1524236771 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02040b44d0 msgr2=0x7f0204151760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:58.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.220+0000 7f0202ffd640 1 --2- 172.21.15.32:0/1524236771 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02040b44d0 0x7f0204151760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:58.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.220+0000 7f0202ffd640 1 -- 172.21.15.32:0/1524236771 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f020406c5f0 msgr2=0x7f0204153050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:58.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.220+0000 7f0202ffd640 1 --2- 172.21.15.32:0/1524236771 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f020406c5f0 0x7f0204153050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:58.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.220+0000 7f0202ffd640 1 -- 172.21.15.32:0/1524236771 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0204151fb0 con 0x7f02040af890 2024-04-24T23:08:58.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.220+0000 7f02037fe640 1 --2- 172.21.15.32:0/1524236771 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f020406c5f0 0x7f0204153050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:08:58.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.220+0000 7f0202ffd640 1 --2- 172.21.15.32:0/1524236771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02040af890 0x7f0204146c80 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f01f800c3f0 tx=0x7f01f800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:58.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.220+0000 7f01ebfff640 1 -- 172.21.15.32:0/1524236771 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f80052d0 con 0x7f02040af890 2024-04-24T23:08:58.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.220+0000 7f01ebfff640 1 -- 172.21.15.32:0/1524236771 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01f8005490 con 0x7f02040af890 2024-04-24T23:08:58.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.220+0000 7f01ebfff640 1 -- 172.21.15.32:0/1524236771 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f8007260 con 0x7f02040af890 2024-04-24T23:08:58.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.220+0000 7f020a190640 1 -- 172.21.15.32:0/1524236771 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0204152290 con 0x7f02040af890 2024-04-24T23:08:58.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.220+0000 7f020a190640 1 -- 172.21.15.32:0/1524236771 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0204152850 con 0x7f02040af890 2024-04-24T23:08:58.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.221+0000 7f020a190640 1 -- 172.21.15.32:0/1524236771 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0204068430 con 0x7f02040af890 2024-04-24T23:08:58.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.222+0000 7f01ebfff640 1 -- 172.21.15.32:0/1524236771 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f01f8007400 con 0x7f02040af890 2024-04-24T23:08:58.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.222+0000 7f01ebfff640 1 --2- 172.21.15.32:0/1524236771 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f01f006be30 0x7f01f006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:58.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.222+0000 7f01ebfff640 1 -- 172.21.15.32:0/1524236771 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4857+0+0 (secure 0 0 0) 0x7f01f808a050 con 0x7f02040af890 2024-04-24T23:08:58.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.222+0000 7f02037fe640 1 --2- 172.21.15.32:0/1524236771 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f01f006be30 0x7f01f006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:58.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.223+0000 7f02037fe640 1 --2- 172.21.15.32:0/1524236771 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f01f006be30 0x7f01f006e2f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f01f4034000 tx=0x7f01f402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:58.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.226+0000 7f01ebfff640 1 -- 172.21.15.32:0/1524236771 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f01f8058ca0 con 0x7f02040af890 2024-04-24T23:08:58.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.412+0000 7f020a190640 1 -- 172.21.15.32:0/1524236771 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f020415fc70 con 0x7f02040af890 2024-04-24T23:08:58.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.413+0000 7f01ebfff640 1 -- 172.21.15.32:0/1524236771 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v165) ==== 93+0+10 (secure 0 0 0) 0x7f01f80585a0 con 0x7f02040af890 2024-04-24T23:08:58.415 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T23:08:58.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.415+0000 7f020a190640 1 -- 172.21.15.32:0/1524236771 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f01f006be30 msgr2=0x7f01f006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:58.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.415+0000 7f020a190640 1 --2- 172.21.15.32:0/1524236771 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f01f006be30 0x7f01f006e2f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f01f4034000 tx=0x7f01f402e000 comp rx=0 tx=0).stop 2024-04-24T23:08:58.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.415+0000 7f020a190640 1 -- 172.21.15.32:0/1524236771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02040af890 msgr2=0x7f0204146c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:58.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.415+0000 7f020a190640 1 --2- 172.21.15.32:0/1524236771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02040af890 0x7f0204146c80 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f01f800c3f0 tx=0x7f01f800c7f0 comp rx=0 tx=0).stop 2024-04-24T23:08:58.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.415+0000 7f020a190640 1 -- 172.21.15.32:0/1524236771 shutdown_connections 2024-04-24T23:08:58.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.415+0000 7f020a190640 1 --2- 172.21.15.32:0/1524236771 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f01f006be30 0x7f01f006e2f0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:58.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.415+0000 7f020a190640 1 --2- 172.21.15.32:0/1524236771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02040af890 0x7f0204146c80 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:58.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.415+0000 7f020a190640 1 --2- 172.21.15.32:0/1524236771 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02040b44d0 0x7f0204151760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:58.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.415+0000 7f020a190640 1 --2- 172.21.15.32:0/1524236771 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f020406c5f0 0x7f0204153050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:58.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.415+0000 7f020a190640 1 -- 172.21.15.32:0/1524236771 >> 172.21.15.32:0/1524236771 conn(0x7f020406b110 msgr2=0x7f020415a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:58.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.415+0000 7f020a190640 1 -- 172.21.15.32:0/1524236771 shutdown_connections 2024-04-24T23:08:58.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.415+0000 7f020a190640 1 -- 172.21.15.32:0/1524236771 wait complete. 2024-04-24T23:08:58.429 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-24T23:08:58.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.527+0000 7f2b7ef42640 1 Processor -- start 2024-04-24T23:08:58.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.527+0000 7f2b7ef42640 1 -- start start 2024-04-24T23:08:58.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.529+0000 7f2b7ef42640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2b7806cd00 0x7f2b780ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:58.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.529+0000 7f2b7ef42640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2b7815cb00 0x7f2b780ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:58.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.529+0000 7f2b7ef42640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b780b4520 con 0x7f2b7815b380 2024-04-24T23:08:58.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.529+0000 7f2b7ef42640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f2b780b4800 con 0x7f2b7815cb00 2024-04-24T23:08:58.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.529+0000 7f2b7ef42640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f2b780b4ae0 con 0x7f2b7806cd00 2024-04-24T23:08:58.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.529+0000 7f2b7d4df640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2b7815cb00 0x7f2b780ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:58.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.529+0000 7f2b77fff640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2b7806cd00 0x7f2b780ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:58.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.529+0000 7f2b7d4df640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2b7815cb00 0x7f2b780ce970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.32:38182/0 (socket says 172.21.15.32:38182) 2024-04-24T23:08:58.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.529+0000 7f2b7d4df640 1 -- 172.21.15.32:0/1097702162 learned_addr learned my addr 172.21.15.32:0/1097702162 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:58.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.529+0000 7f2b77fff640 1 -- 172.21.15.32:0/1097702162 >> v2:172.21.15.121:3300/0 conn(0x7f2b7815cb00 msgr2=0x7f2b780ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:58.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.529+0000 7f2b77fff640 1 --2- 172.21.15.32:0/1097702162 >> v2:172.21.15.121:3300/0 conn(0x7f2b7815cb00 0x7f2b780ce970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:58.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.529+0000 7f2b77fff640 1 -- 172.21.15.32:0/1097702162 >> v1:172.21.15.121:6789/0 conn(0x7f2b7815b380 legacy=0x7f2b7806c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:58.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.529+0000 7f2b77fff640 1 -- 172.21.15.32:0/1097702162 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b780b4dc0 con 0x7f2b7806cd00 2024-04-24T23:08:58.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.529+0000 7f2b777fe640 1 -- 172.21.15.32:0/1097702162 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1727357487 0 0) 0x7f2b780b4520 con 0x7f2b7815b380 2024-04-24T23:08:58.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.529+0000 7f2b7d4df640 1 --2- 172.21.15.32:0/1097702162 >> v2:172.21.15.121:3300/0 conn(0x7f2b7815cb00 0x7f2b780ce970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:08:58.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.530+0000 7f2b77fff640 1 --2- 172.21.15.32:0/1097702162 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2b7806cd00 0x7f2b780ce430 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c0097a0 tx=0x7f2b6c02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0d8dac4f6a47c7a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:58.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.530+0000 7f2b777fe640 1 -- 172.21.15.32:0/1097702162 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c037070 con 0x7f2b7806cd00 2024-04-24T23:08:58.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.530+0000 7f2b777fe640 1 -- 172.21.15.32:0/1097702162 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b6c02ae50 con 0x7f2b7806cd00 2024-04-24T23:08:58.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.530+0000 7f2b777fe640 1 -- 172.21.15.32:0/1097702162 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c032660 con 0x7f2b7806cd00 2024-04-24T23:08:58.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.531+0000 7f2b7ef42640 1 -- 172.21.15.32:0/1097702162 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2b7806cd00 msgr2=0x7f2b780ce430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:58.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.531+0000 7f2b7ef42640 1 --2- 172.21.15.32:0/1097702162 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2b7806cd00 0x7f2b780ce430 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c0097a0 tx=0x7f2b6c02a100 comp rx=0 tx=0).stop 2024-04-24T23:08:58.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.531+0000 7f2b7ef42640 1 -- 172.21.15.32:0/1097702162 shutdown_connections 2024-04-24T23:08:58.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.531+0000 7f2b7ef42640 1 --2- 172.21.15.32:0/1097702162 >> v2:172.21.15.121:3300/0 conn(0x7f2b7815cb00 0x7f2b780ce970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:58.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.531+0000 7f2b7ef42640 1 --2- 172.21.15.32:0/1097702162 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2b7806cd00 0x7f2b780ce430 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:58.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.531+0000 7f2b7ef42640 1 -- 172.21.15.32:0/1097702162 >> 172.21.15.32:0/1097702162 conn(0x7f2b7806b110 msgr2=0x7f2b7806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:58.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.531+0000 7f2b7ef42640 1 -- 172.21.15.32:0/1097702162 shutdown_connections 2024-04-24T23:08:58.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.531+0000 7f2b7ef42640 1 -- 172.21.15.32:0/1097702162 wait complete. 2024-04-24T23:08:58.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.532+0000 7f2b7ef42640 1 Processor -- start 2024-04-24T23:08:58.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.532+0000 7f2b7ef42640 1 -- start start 2024-04-24T23:08:58.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.533+0000 7f2b7ef42640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2b7806cd00 0x7f2b781478c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:58.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.533+0000 7f2b7ef42640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b7815b380 0x7f2b78143b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:58.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.533+0000 7f2b7ef42640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2b7815cb00 0x7f2b7815d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:58.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.533+0000 7f2b7ef42640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7815b760 con 0x7f2b7815b380 2024-04-24T23:08:58.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.533+0000 7f2b7ef42640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2b780ceeb0 con 0x7f2b7806cd00 2024-04-24T23:08:58.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.533+0000 7f2b7ef42640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f2b780b55c0 con 0x7f2b7815cb00 2024-04-24T23:08:58.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.533+0000 7f2b77fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b7815b380 0x7f2b78143b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:58.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.533+0000 7f2b77fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b7815b380 0x7f2b78143b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45608/0 (socket says 172.21.15.32:45608) 2024-04-24T23:08:58.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.533+0000 7f2b77fff640 1 -- 172.21.15.32:0/1914189874 learned_addr learned my addr 172.21.15.32:0/1914189874 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:58.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.533+0000 7f2b7ccde640 1 --2- 172.21.15.32:0/1914189874 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2b7806cd00 0x7f2b781478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:58.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.533+0000 7f2b7d4df640 1 --2- 172.21.15.32:0/1914189874 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2b7815cb00 0x7f2b7815d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:58.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.533+0000 7f2b77fff640 1 -- 172.21.15.32:0/1914189874 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2b7815cb00 msgr2=0x7f2b7815d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:58.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.533+0000 7f2b77fff640 1 --2- 172.21.15.32:0/1914189874 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2b7815cb00 0x7f2b7815d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:58.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.533+0000 7f2b77fff640 1 -- 172.21.15.32:0/1914189874 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2b7806cd00 msgr2=0x7f2b781478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:58.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.533+0000 7f2b77fff640 1 --2- 172.21.15.32:0/1914189874 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2b7806cd00 0x7f2b781478c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:58.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.533+0000 7f2b77fff640 1 -- 172.21.15.32:0/1914189874 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b7814ed40 con 0x7f2b7815b380 2024-04-24T23:08:58.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.534+0000 7f2b77fff640 1 --2- 172.21.15.32:0/1914189874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b7815b380 0x7f2b78143b90 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c0097a0 tx=0x7f2b6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:58.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.534+0000 7f2b757fa640 1 -- 172.21.15.32:0/1914189874 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c037040 con 0x7f2b7815b380 2024-04-24T23:08:58.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.534+0000 7f2b757fa640 1 -- 172.21.15.32:0/1914189874 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b6c004030 con 0x7f2b7815b380 2024-04-24T23:08:58.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.534+0000 7f2b7ef42640 1 -- 172.21.15.32:0/1914189874 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b7814f020 con 0x7f2b7815b380 2024-04-24T23:08:58.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.534+0000 7f2b757fa640 1 -- 172.21.15.32:0/1914189874 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c004420 con 0x7f2b7815b380 2024-04-24T23:08:58.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.534+0000 7f2b7ef42640 1 -- 172.21.15.32:0/1914189874 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b7814f5e0 con 0x7f2b7815b380 2024-04-24T23:08:58.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.535+0000 7f2b7ef42640 1 -- 172.21.15.32:0/1914189874 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b78068430 con 0x7f2b7815b380 2024-04-24T23:08:58.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.535+0000 7f2b757fa640 1 -- 172.21.15.32:0/1914189874 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2b6c003610 con 0x7f2b7815b380 2024-04-24T23:08:58.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.536+0000 7f2b757fa640 1 --2- 172.21.15.32:0/1914189874 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2b6406be60 0x7f2b6406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:58.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.536+0000 7f2b757fa640 1 -- 172.21.15.32:0/1914189874 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4857+0+0 (secure 0 0 0) 0x7f2b6c0ab7a0 con 0x7f2b7815b380 2024-04-24T23:08:58.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.536+0000 7f2b7ccde640 1 --2- 172.21.15.32:0/1914189874 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2b6406be60 0x7f2b6406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:58.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.536+0000 7f2b7ccde640 1 --2- 172.21.15.32:0/1914189874 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2b6406be60 0x7f2b6406e320 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f2b70007fd0 tx=0x7f2b70009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:58.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.540+0000 7f2b757fa640 1 -- 172.21.15.32:0/1914189874 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2b6c07a400 con 0x7f2b7815b380 2024-04-24T23:08:58.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.729+0000 7f2b7ef42640 1 -- 172.21.15.32:0/1914189874 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f2b78067510 con 0x7f2b7815b380 2024-04-24T23:08:58.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.730+0000 7f2b757fa640 1 -- 172.21.15.32:0/1914189874 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v165) ==== 84+0+65 (secure 0 0 0) 0x7f2b6c079d00 con 0x7f2b7815b380 2024-04-24T23:08:58.732 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:08:58.733 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2024-04-24T23:08:58.733 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:08:58.733 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2024-04-24T23:08:58.733 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": ".mgr" 2024-04-24T23:08:58.733 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:08:58.733 INFO:teuthology.orchestra.run.smithi032.stdout:] 2024-04-24T23:08:58.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.732+0000 7f2b7ef42640 1 -- 172.21.15.32:0/1914189874 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2b6406be60 msgr2=0x7f2b6406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:58.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.732+0000 7f2b7ef42640 1 --2- 172.21.15.32:0/1914189874 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2b6406be60 0x7f2b6406e320 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f2b70007fd0 tx=0x7f2b70009040 comp rx=0 tx=0).stop 2024-04-24T23:08:58.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.732+0000 7f2b7ef42640 1 -- 172.21.15.32:0/1914189874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b7815b380 msgr2=0x7f2b78143b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:58.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.732+0000 7f2b7ef42640 1 --2- 172.21.15.32:0/1914189874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b7815b380 0x7f2b78143b90 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c0097a0 tx=0x7f2b6c0046c0 comp rx=0 tx=0).stop 2024-04-24T23:08:58.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.732+0000 7f2b7ef42640 1 -- 172.21.15.32:0/1914189874 shutdown_connections 2024-04-24T23:08:58.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.732+0000 7f2b7ef42640 1 --2- 172.21.15.32:0/1914189874 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2b6406be60 0x7f2b6406e320 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:58.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.732+0000 7f2b7ef42640 1 --2- 172.21.15.32:0/1914189874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b7815b380 0x7f2b78143b90 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:58.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.732+0000 7f2b7ef42640 1 --2- 172.21.15.32:0/1914189874 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2b7815cb00 0x7f2b7815d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:58.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.732+0000 7f2b7ef42640 1 --2- 172.21.15.32:0/1914189874 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2b7806cd00 0x7f2b781478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:58.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.732+0000 7f2b7ef42640 1 -- 172.21.15.32:0/1914189874 >> 172.21.15.32:0/1914189874 conn(0x7f2b7806b110 msgr2=0x7f2b7815b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:58.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.732+0000 7f2b7ef42640 1 -- 172.21.15.32:0/1914189874 shutdown_connections 2024-04-24T23:08:58.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.732+0000 7f2b7ef42640 1 -- 172.21.15.32:0/1914189874 wait complete. 2024-04-24T23:08:58.745 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-24T23:08:58.746 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-24T23:08:58.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.847+0000 7fc818fc8640 1 Processor -- start 2024-04-24T23:08:58.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.847+0000 7fc818fc8640 1 -- start start 2024-04-24T23:08:58.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.848+0000 7fc818fc8640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc81415b580 0x7fc8140cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:58.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.848+0000 7fc818fc8640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc81415bde0 con 0x7fc8140b58e0 2024-04-24T23:08:58.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.848+0000 7fc818fc8640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc81415c0c0 con 0x7fc81415b960 2024-04-24T23:08:58.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.848+0000 7fc818fc8640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc81415c9f0 con 0x7fc81415b580 2024-04-24T23:08:58.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.848+0000 7fc812d9d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc8140b58e0 0x7fc81415f340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55040/0 (socket says 172.21.15.32:55040) 2024-04-24T23:08:58.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.848+0000 7fc812d9d640 1 -- 172.21.15.32:0/4111470869 learned_addr learned my addr 172.21.15.32:0/4111470869 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:08:58.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.848+0000 7fc81259c640 1 --2- 172.21.15.32:0/4111470869 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc81415b580 0x7fc8140cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:58.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.848+0000 7fc81159a640 1 -- 172.21.15.32:0/4111470869 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 799952510 0 0) 0x7fc81415bde0 con 0x7fc8140b58e0 2024-04-24T23:08:58.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.848+0000 7fc81159a640 1 -- 172.21.15.32:0/4111470869 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc7fc003520 con 0x7fc8140b58e0 2024-04-24T23:08:58.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.848+0000 7fc81159a640 1 -- 172.21.15.32:0/4111470869 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3474481134 0 0) 0x7fc7fc003520 con 0x7fc8140b58e0 2024-04-24T23:08:58.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.848+0000 7fc81159a640 1 -- 172.21.15.32:0/4111470869 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc81415b580 msgr2=0x7fc8140cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:58.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.848+0000 7fc81159a640 1 --2- 172.21.15.32:0/4111470869 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc81415b580 0x7fc8140cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:58.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.848+0000 7fc81159a640 1 -- 172.21.15.32:0/4111470869 >> v1:172.21.15.121:6789/0 conn(0x7fc81415b960 legacy=0x7fc81406cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:58.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.848+0000 7fc81159a640 1 -- 172.21.15.32:0/4111470869 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc81415c3a0 con 0x7fc8140b58e0 2024-04-24T23:08:58.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.848+0000 7fc81159a640 1 -- 172.21.15.32:0/4111470869 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7fc80c0019f0 con 0x7fc8140b58e0 2024-04-24T23:08:58.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.848+0000 7fc81159a640 1 -- 172.21.15.32:0/4111470869 >> v1:172.21.15.32:6789/0 conn(0x7fc8140b58e0 legacy=0x7fc81415f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:58.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.848+0000 7fc81159a640 1 --2- 172.21.15.32:0/4111470869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7fc0039e0 0x7fc7fc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:58.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.849+0000 7fc81159a640 1 --2- 172.21.15.32:0/4111470869 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc81415b580 0x7fc7fc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:58.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.849+0000 7fc81159a640 1 --2- 172.21.15.32:0/4111470869 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7fc004d40 0x7fc7fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:58.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.849+0000 7fc81159a640 1 -- 172.21.15.32:0/4111470869 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc81415c9f0 con 0x7fc7fc0039e0 2024-04-24T23:08:58.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.849+0000 7fc81159a640 1 -- 172.21.15.32:0/4111470869 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc7fc00ba70 con 0x7fc7fc004d40 2024-04-24T23:08:58.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.849+0000 7fc81159a640 1 -- 172.21.15.32:0/4111470869 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc7fc00de80 con 0x7fc81415b580 2024-04-24T23:08:58.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.849+0000 7fc81259c640 1 --2- 172.21.15.32:0/4111470869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7fc0039e0 0x7fc7fc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:58.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.849+0000 7fc811d9b640 1 --2- 172.21.15.32:0/4111470869 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc81415b580 0x7fc7fc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:58.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.849+0000 7fc812d9d640 1 --2- 172.21.15.32:0/4111470869 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7fc004d40 0x7fc7fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:58.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.849+0000 7fc81259c640 1 -- 172.21.15.32:0/4111470869 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc81415b580 msgr2=0x7fc7fc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:58.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.849+0000 7fc81259c640 1 --2- 172.21.15.32:0/4111470869 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc81415b580 0x7fc7fc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:58.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.849+0000 7fc81259c640 1 -- 172.21.15.32:0/4111470869 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7fc004d40 msgr2=0x7fc7fc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:58.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.849+0000 7fc81259c640 1 --2- 172.21.15.32:0/4111470869 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7fc004d40 0x7fc7fc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:58.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.849+0000 7fc81259c640 1 -- 172.21.15.32:0/4111470869 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7fc003520 con 0x7fc7fc0039e0 2024-04-24T23:08:58.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.850+0000 7fc81259c640 1 --2- 172.21.15.32:0/4111470869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7fc0039e0 0x7fc7fc003e20 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fc804009a90 tx=0x7fc7fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2accd0712e1a415 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:58.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.850+0000 7fc81159a640 1 -- 172.21.15.32:0/4111470869 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc804036070 con 0x7fc7fc0039e0 2024-04-24T23:08:58.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.850+0000 7fc81159a640 1 -- 172.21.15.32:0/4111470869 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc804031400 con 0x7fc7fc0039e0 2024-04-24T23:08:58.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.850+0000 7fc81159a640 1 -- 172.21.15.32:0/4111470869 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc804009690 con 0x7fc7fc0039e0 2024-04-24T23:08:58.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.850+0000 7fc818fc8640 1 -- 172.21.15.32:0/4111470869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7fc0039e0 msgr2=0x7fc7fc003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:58.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.850+0000 7fc818fc8640 1 --2- 172.21.15.32:0/4111470869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7fc0039e0 0x7fc7fc003e20 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fc804009a90 tx=0x7fc7fc005830 comp rx=0 tx=0).stop 2024-04-24T23:08:58.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.850+0000 7fc812d9d640 1 -- 172.21.15.32:0/4111470869 reap_dead start 2024-04-24T23:08:58.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.850+0000 7fc818fc8640 1 -- 172.21.15.32:0/4111470869 shutdown_connections 2024-04-24T23:08:58.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.850+0000 7fc818fc8640 1 -- 172.21.15.32:0/4111470869 >> 172.21.15.32:0/4111470869 conn(0x7fc8140b1f10 msgr2=0x7fc8140b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:58.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.851+0000 7fc818fc8640 1 -- 172.21.15.32:0/4111470869 shutdown_connections 2024-04-24T23:08:58.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.851+0000 7fc818fc8640 1 -- 172.21.15.32:0/4111470869 wait complete. 2024-04-24T23:08:58.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.852+0000 7fc818fc8640 1 Processor -- start 2024-04-24T23:08:58.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.852+0000 7fc818fc8640 1 -- start start 2024-04-24T23:08:58.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.853+0000 7fc818fc8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7fc0039e0 0x7fc814158ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:58.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.853+0000 7fc818fc8640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc8141f4840 0x7fc814072320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:58.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.853+0000 7fc818fc8640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8141571e0 0x7fc8140ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:58.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.853+0000 7fc818fc8640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc81406c920 con 0x7fc7fc0039e0 2024-04-24T23:08:58.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.853+0000 7fc818fc8640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc81406c7a0 con 0x7fc8141571e0 2024-04-24T23:08:58.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.853+0000 7fc818fc8640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc81415fa50 con 0x7fc8141f4840 2024-04-24T23:08:58.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.853+0000 7fc81259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7fc0039e0 0x7fc814158ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:58.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.853+0000 7fc81259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7fc0039e0 0x7fc814158ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45626/0 (socket says 172.21.15.32:45626) 2024-04-24T23:08:58.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.853+0000 7fc81259c640 1 -- 172.21.15.32:0/1238138354 learned_addr learned my addr 172.21.15.32:0/1238138354 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:58.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.853+0000 7fc811d9b640 1 --2- 172.21.15.32:0/1238138354 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc8141f4840 0x7fc814072320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:58.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.853+0000 7fc812d9d640 1 --2- 172.21.15.32:0/1238138354 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8141571e0 0x7fc8140ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:58.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.853+0000 7fc81259c640 1 -- 172.21.15.32:0/1238138354 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc8141f4840 msgr2=0x7fc814072320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:58.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.853+0000 7fc81259c640 1 --2- 172.21.15.32:0/1238138354 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc8141f4840 0x7fc814072320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:58.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.853+0000 7fc81259c640 1 -- 172.21.15.32:0/1238138354 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8141571e0 msgr2=0x7fc8140ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:58.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.853+0000 7fc81259c640 1 --2- 172.21.15.32:0/1238138354 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8141571e0 0x7fc8140ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:58.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.853+0000 7fc81259c640 1 -- 172.21.15.32:0/1238138354 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8141f7e30 con 0x7fc7fc0039e0 2024-04-24T23:08:58.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.853+0000 7fc81259c640 1 --2- 172.21.15.32:0/1238138354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7fc0039e0 0x7fc814158ca0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fc804009bb0 tx=0x7fc804004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:58.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.853+0000 7fc802ffd640 1 -- 172.21.15.32:0/1238138354 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc804040020 con 0x7fc7fc0039e0 2024-04-24T23:08:58.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.853+0000 7fc802ffd640 1 -- 172.21.15.32:0/1238138354 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc804003940 con 0x7fc7fc0039e0 2024-04-24T23:08:58.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.853+0000 7fc818fc8640 1 -- 172.21.15.32:0/1238138354 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc814072860 con 0x7fc7fc0039e0 2024-04-24T23:08:58.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.853+0000 7fc802ffd640 1 -- 172.21.15.32:0/1238138354 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc804036040 con 0x7fc7fc0039e0 2024-04-24T23:08:58.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.853+0000 7fc818fc8640 1 -- 172.21.15.32:0/1238138354 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8141f5090 con 0x7fc7fc0039e0 2024-04-24T23:08:58.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.855+0000 7fc818fc8640 1 -- 172.21.15.32:0/1238138354 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc814068430 con 0x7fc7fc0039e0 2024-04-24T23:08:58.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.855+0000 7fc802ffd640 1 -- 172.21.15.32:0/1238138354 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc804043020 con 0x7fc7fc0039e0 2024-04-24T23:08:58.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.856+0000 7fc802ffd640 1 --2- 172.21.15.32:0/1238138354 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc7f406be30 0x7fc7f406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:58.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.856+0000 7fc802ffd640 1 -- 172.21.15.32:0/1238138354 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4857+0+0 (secure 0 0 0) 0x7fc8040aabc0 con 0x7fc7fc0039e0 2024-04-24T23:08:58.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.856+0000 7fc811d9b640 1 --2- 172.21.15.32:0/1238138354 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc7f406be30 0x7fc7f406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:58.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.856+0000 7fc811d9b640 1 --2- 172.21.15.32:0/1238138354 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc7f406be30 0x7fc7f406e2f0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fc8080017e0 tx=0x7fc808009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:58.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:58.860+0000 7fc802ffd640 1 -- 172.21.15.32:0/1238138354 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc80404a070 con 0x7fc7fc0039e0 2024-04-24T23:08:59.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.043+0000 7fc818fc8640 1 -- 172.21.15.32:0/1238138354 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fc814158350 con 0x7fc7fc0039e0 2024-04-24T23:08:59.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.391+0000 7fc802ffd640 1 -- 172.21.15.32:0/1238138354 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v166) ==== 138+0+0 (secure 0 0 0) 0x7fc804079270 con 0x7fc7fc0039e0 2024-04-24T23:08:59.393 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2024-04-24T23:08:59.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.394+0000 7fc818fc8640 1 -- 172.21.15.32:0/1238138354 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc7f406be30 msgr2=0x7fc7f406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:59.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.394+0000 7fc818fc8640 1 --2- 172.21.15.32:0/1238138354 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc7f406be30 0x7fc7f406e2f0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fc8080017e0 tx=0x7fc808009040 comp rx=0 tx=0).stop 2024-04-24T23:08:59.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.394+0000 7fc818fc8640 1 -- 172.21.15.32:0/1238138354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7fc0039e0 msgr2=0x7fc814158ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:59.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.394+0000 7fc818fc8640 1 --2- 172.21.15.32:0/1238138354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7fc0039e0 0x7fc814158ca0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fc804009bb0 tx=0x7fc804004730 comp rx=0 tx=0).stop 2024-04-24T23:08:59.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.396+0000 7fc818fc8640 1 -- 172.21.15.32:0/1238138354 shutdown_connections 2024-04-24T23:08:59.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.396+0000 7fc818fc8640 1 --2- 172.21.15.32:0/1238138354 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc7f406be30 0x7fc7f406e2f0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:59.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.396+0000 7fc818fc8640 1 --2- 172.21.15.32:0/1238138354 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8141571e0 0x7fc8140ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:59.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.396+0000 7fc818fc8640 1 --2- 172.21.15.32:0/1238138354 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc8141f4840 0x7fc814072320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:59.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.396+0000 7fc818fc8640 1 --2- 172.21.15.32:0/1238138354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7fc0039e0 0x7fc814158ca0 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:59.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.396+0000 7fc818fc8640 1 -- 172.21.15.32:0/1238138354 >> 172.21.15.32:0/1238138354 conn(0x7fc8140b1f10 msgr2=0x7fc814066ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:59.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.396+0000 7fc818fc8640 1 -- 172.21.15.32:0/1238138354 shutdown_connections 2024-04-24T23:08:59.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.396+0000 7fc818fc8640 1 -- 172.21.15.32:0/1238138354 wait complete. 2024-04-24T23:08:59.410 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-24T23:08:59.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.519+0000 7f2e248a5640 1 Processor -- start 2024-04-24T23:08:59.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.519+0000 7f2e248a5640 1 -- start start 2024-04-24T23:08:59.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.519+0000 7f2e248a5640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2e2015b380 0x7f2e2006c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:59.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.519+0000 7f2e248a5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e2015cb00 0x7f2e2006cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:59.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.519+0000 7f2e248a5640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e200b4520 con 0x7f2e200ce430 2024-04-24T23:08:59.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.519+0000 7f2e248a5640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f2e200cef90 con 0x7f2e2015b380 2024-04-24T23:08:59.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.519+0000 7f2e248a5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f2e200b4800 con 0x7f2e2015cb00 2024-04-24T23:08:59.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.520+0000 7f2e17fff640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e2015cb00 0x7f2e2006cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:59.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.520+0000 7f2e1e59c640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2e2015b380 0x7f2e2006c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:59.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.520+0000 7f2e17fff640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e2015cb00 0x7f2e2006cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.32:36302/0 (socket says 172.21.15.32:36302) 2024-04-24T23:08:59.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.520+0000 7f2e17fff640 1 -- 172.21.15.32:0/1296866287 learned_addr learned my addr 172.21.15.32:0/1296866287 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:59.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.520+0000 7f2e17fff640 1 -- 172.21.15.32:0/1296866287 >> v2:172.21.15.121:3300/0 conn(0x7f2e2015b380 msgr2=0x7f2e2006c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:08:59.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.520+0000 7f2e17fff640 1 --2- 172.21.15.32:0/1296866287 >> v2:172.21.15.121:3300/0 conn(0x7f2e2015b380 0x7f2e2006c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:59.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.520+0000 7f2e17fff640 1 -- 172.21.15.32:0/1296866287 >> v1:172.21.15.121:6789/0 conn(0x7f2e200ce430 legacy=0x7f2e200ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:59.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.520+0000 7f2e17fff640 1 -- 172.21.15.32:0/1296866287 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e200b4ae0 con 0x7f2e2015cb00 2024-04-24T23:08:59.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.520+0000 7f2e1dd9b640 1 -- 172.21.15.32:0/1296866287 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3522804795 0 0) 0x7f2e200b4520 con 0x7f2e200ce430 2024-04-24T23:08:59.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.520+0000 7f2e1e59c640 1 --2- 172.21.15.32:0/1296866287 >> v2:172.21.15.121:3300/0 conn(0x7f2e2015b380 0x7f2e2006c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:08:59.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.521+0000 7f2e17fff640 1 --2- 172.21.15.32:0/1296866287 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e2015cb00 0x7f2e2006cb30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2e08009870 tx=0x7f2e0802a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=371a6aa8f2380bac server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:59.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.521+0000 7f2e1dd9b640 1 -- 172.21.15.32:0/1296866287 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e08037070 con 0x7f2e2015cb00 2024-04-24T23:08:59.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.521+0000 7f2e1dd9b640 1 -- 172.21.15.32:0/1296866287 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e08032440 con 0x7f2e2015cb00 2024-04-24T23:08:59.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.521+0000 7f2e1dd9b640 1 -- 172.21.15.32:0/1296866287 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e08032810 con 0x7f2e2015cb00 2024-04-24T23:08:59.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.522+0000 7f2e248a5640 1 -- 172.21.15.32:0/1296866287 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e2015cb00 msgr2=0x7f2e2006cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:59.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.522+0000 7f2e248a5640 1 --2- 172.21.15.32:0/1296866287 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e2015cb00 0x7f2e2006cb30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2e08009870 tx=0x7f2e0802a1c0 comp rx=0 tx=0).stop 2024-04-24T23:08:59.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.522+0000 7f2e248a5640 1 -- 172.21.15.32:0/1296866287 shutdown_connections 2024-04-24T23:08:59.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.522+0000 7f2e248a5640 1 --2- 172.21.15.32:0/1296866287 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e2015cb00 0x7f2e2006cb30 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:59.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.522+0000 7f2e248a5640 1 --2- 172.21.15.32:0/1296866287 >> v2:172.21.15.121:3300/0 conn(0x7f2e2015b380 0x7f2e2006c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:59.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.522+0000 7f2e248a5640 1 -- 172.21.15.32:0/1296866287 >> 172.21.15.32:0/1296866287 conn(0x7f2e2006b110 msgr2=0x7f2e2006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:59.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.522+0000 7f2e248a5640 1 -- 172.21.15.32:0/1296866287 shutdown_connections 2024-04-24T23:08:59.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.522+0000 7f2e248a5640 1 -- 172.21.15.32:0/1296866287 wait complete. 2024-04-24T23:08:59.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.523+0000 7f2e248a5640 1 Processor -- start 2024-04-24T23:08:59.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.523+0000 7f2e248a5640 1 -- start start 2024-04-24T23:08:59.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.524+0000 7f2e248a5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e200ce430 0x7f2e20148150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:59.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.524+0000 7f2e248a5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e2015b380 0x7f2e200ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:59.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.524+0000 7f2e248a5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e2015cb00 0x7f2e200aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:59.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.524+0000 7f2e248a5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2015b760 con 0x7f2e2015cb00 2024-04-24T23:08:59.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.524+0000 7f2e248a5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2e200b50a0 con 0x7f2e200ce430 2024-04-24T23:08:59.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.524+0000 7f2e248a5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f2e200b55c0 con 0x7f2e2015b380 2024-04-24T23:08:59.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.524+0000 7f2e1ed9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e2015cb00 0x7f2e200aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:59.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.524+0000 7f2e1ed9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e2015cb00 0x7f2e200aedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45640/0 (socket says 172.21.15.32:45640) 2024-04-24T23:08:59.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.524+0000 7f2e1ed9d640 1 -- 172.21.15.32:0/2788360635 learned_addr learned my addr 172.21.15.32:0/2788360635 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:08:59.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.524+0000 7f2e1e59c640 1 --2- 172.21.15.32:0/2788360635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e200ce430 0x7f2e20148150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:59.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.524+0000 7f2e17fff640 1 --2- 172.21.15.32:0/2788360635 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e2015b380 0x7f2e200ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:59.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.524+0000 7f2e1ed9d640 1 -- 172.21.15.32:0/2788360635 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e2015b380 msgr2=0x7f2e200ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:59.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.524+0000 7f2e1ed9d640 1 --2- 172.21.15.32:0/2788360635 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e2015b380 0x7f2e200ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:59.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.524+0000 7f2e1ed9d640 1 -- 172.21.15.32:0/2788360635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e200ce430 msgr2=0x7f2e20148150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:59.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.524+0000 7f2e1ed9d640 1 --2- 172.21.15.32:0/2788360635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e200ce430 0x7f2e20148150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:59.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.524+0000 7f2e1ed9d640 1 -- 172.21.15.32:0/2788360635 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e200716d0 con 0x7f2e2015cb00 2024-04-24T23:08:59.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.524+0000 7f2e1e59c640 1 --2- 172.21.15.32:0/2788360635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e200ce430 0x7f2e20148150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:08:59.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.524+0000 7f2e17fff640 1 --2- 172.21.15.32:0/2788360635 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e2015b380 0x7f2e200ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:08:59.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.524+0000 7f2e1ed9d640 1 --2- 172.21.15.32:0/2788360635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e2015cb00 0x7f2e200aedf0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f2e1800c760 tx=0x7f2e1800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:59.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.525+0000 7f2e16ffd640 1 -- 172.21.15.32:0/2788360635 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e18003da0 con 0x7f2e2015cb00 2024-04-24T23:08:59.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.525+0000 7f2e16ffd640 1 -- 172.21.15.32:0/2788360635 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e18005070 con 0x7f2e2015cb00 2024-04-24T23:08:59.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.525+0000 7f2e248a5640 1 -- 172.21.15.32:0/2788360635 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e200719b0 con 0x7f2e2015cb00 2024-04-24T23:08:59.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.525+0000 7f2e16ffd640 1 -- 172.21.15.32:0/2788360635 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e18005460 con 0x7f2e2015cb00 2024-04-24T23:08:59.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.525+0000 7f2e248a5640 1 -- 172.21.15.32:0/2788360635 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e20071f70 con 0x7f2e2015cb00 2024-04-24T23:08:59.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.526+0000 7f2e248a5640 1 -- 172.21.15.32:0/2788360635 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e20068430 con 0x7f2e2015cb00 2024-04-24T23:08:59.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.526+0000 7f2e16ffd640 1 -- 172.21.15.32:0/2788360635 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2e18005600 con 0x7f2e2015cb00 2024-04-24T23:08:59.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.527+0000 7f2e16ffd640 1 --2- 172.21.15.32:0/2788360635 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2e0006be30 0x7f2e0006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:59.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.527+0000 7f2e16ffd640 1 -- 172.21.15.32:0/2788360635 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 5227+0+0 (secure 0 0 0) 0x7f2e1808abb0 con 0x7f2e2015cb00 2024-04-24T23:08:59.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.527+0000 7f2e1e59c640 1 --2- 172.21.15.32:0/2788360635 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2e0006be30 0x7f2e0006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:59.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.527+0000 7f2e1e59c640 1 --2- 172.21.15.32:0/2788360635 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2e0006be30 0x7f2e0006e2f0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f2e10009560 tx=0x7f2e10009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:59.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.531+0000 7f2e16ffd640 1 -- 172.21.15.32:0/2788360635 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2e18058fd0 con 0x7f2e2015cb00 2024-04-24T23:08:59.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:08:59.714+0000 7f2e248a5640 1 -- 172.21.15.32:0/2788360635 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f2e2006c000 con 0x7f2e2015cb00 2024-04-24T23:09:00.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.394+0000 7f2e16ffd640 1 -- 172.21.15.32:0/2788360635 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v167) ==== 170+0+0 (secure 0 0 0) 0x7f2e1805d2d0 con 0x7f2e2015cb00 2024-04-24T23:09:00.396 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2024-04-24T23:09:00.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.399+0000 7f2e248a5640 1 -- 172.21.15.32:0/2788360635 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2e0006be30 msgr2=0x7f2e0006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:00.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.399+0000 7f2e248a5640 1 --2- 172.21.15.32:0/2788360635 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2e0006be30 0x7f2e0006e2f0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f2e10009560 tx=0x7f2e10009190 comp rx=0 tx=0).stop 2024-04-24T23:09:00.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.399+0000 7f2e248a5640 1 -- 172.21.15.32:0/2788360635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e2015cb00 msgr2=0x7f2e200aedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:00.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.399+0000 7f2e248a5640 1 --2- 172.21.15.32:0/2788360635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e2015cb00 0x7f2e200aedf0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f2e1800c760 tx=0x7f2e1800cc20 comp rx=0 tx=0).stop 2024-04-24T23:09:00.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.401+0000 7f2e248a5640 1 -- 172.21.15.32:0/2788360635 shutdown_connections 2024-04-24T23:09:00.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.401+0000 7f2e248a5640 1 --2- 172.21.15.32:0/2788360635 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2e0006be30 0x7f2e0006e2f0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:00.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.401+0000 7f2e248a5640 1 --2- 172.21.15.32:0/2788360635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e2015cb00 0x7f2e200aedf0 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:00.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.401+0000 7f2e248a5640 1 --2- 172.21.15.32:0/2788360635 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e2015b380 0x7f2e200ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:00.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.401+0000 7f2e248a5640 1 --2- 172.21.15.32:0/2788360635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e200ce430 0x7f2e20148150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:00.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.402+0000 7f2e248a5640 1 -- 172.21.15.32:0/2788360635 >> 172.21.15.32:0/2788360635 conn(0x7f2e2006b110 msgr2=0x7f2e2006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:00.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.406+0000 7f2e248a5640 1 -- 172.21.15.32:0/2788360635 shutdown_connections 2024-04-24T23:09:00.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.406+0000 7f2e248a5640 1 -- 172.21.15.32:0/2788360635 wait complete. 2024-04-24T23:09:00.424 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-24T23:09:00.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.531+0000 7f80fcf28640 1 Processor -- start 2024-04-24T23:09:00.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.531+0000 7f80fcf28640 1 -- start start 2024-04-24T23:09:00.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.532+0000 7f80fcf28640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f80f815fd60 0x7f80f806c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:00.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.532+0000 7f80fcf28640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f80f806cb30 0x7f80f80cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:00.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.532+0000 7f80fcf28640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80f80b46c0 con 0x7f80f815b480 2024-04-24T23:09:00.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.532+0000 7f80fcf28640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f80f80b4ff0 con 0x7f80f806cb30 2024-04-24T23:09:00.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.532+0000 7f80fcf28640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f80f80b49a0 con 0x7f80f815fd60 2024-04-24T23:09:00.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.532+0000 7f80f6d9d640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f80f806cb30 0x7f80f80cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:00.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.532+0000 7f80f659c640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f80f815b480 0x7f80f80b5c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:53830/0 (socket says 172.21.15.32:53830) 2024-04-24T23:09:00.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.532+0000 7f80f5d9b640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f80f815fd60 0x7f80f806c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:00.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.532+0000 7f80f659c640 1 -- 172.21.15.32:0/966709670 learned_addr learned my addr 172.21.15.32:0/966709670 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:09:00.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.532+0000 7f80f6d9d640 1 -- 172.21.15.32:0/966709670 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f80f815fd60 msgr2=0x7f80f806c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:09:00.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.532+0000 7f80f6d9d640 1 --2- 172.21.15.32:0/966709670 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f80f815fd60 0x7f80f806c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:00.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.532+0000 7f80f6d9d640 1 -- 172.21.15.32:0/966709670 >> v1:172.21.15.121:6789/0 conn(0x7f80f815b480 legacy=0x7f80f80b5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:00.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.532+0000 7f80f6d9d640 1 -- 172.21.15.32:0/966709670 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80f80b4c80 con 0x7f80f806cb30 2024-04-24T23:09:00.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.533+0000 7f80f6d9d640 1 --2- 172.21.15.32:0/966709670 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80f806cb30 0x7f80f80cdf80 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f80f0009d90 tx=0x7f80f002a560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6177e5ed8752ab60 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:00.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.533+0000 7f80f559a640 1 -- 172.21.15.32:0/966709670 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f0007740 con 0x7f80f806cb30 2024-04-24T23:09:00.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.533+0000 7f80f559a640 1 -- 172.21.15.32:0/966709670 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80f00078e0 con 0x7f80f806cb30 2024-04-24T23:09:00.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.533+0000 7f80f559a640 1 -- 172.21.15.32:0/966709670 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f0007cb0 con 0x7f80f806cb30 2024-04-24T23:09:00.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.534+0000 7f80fcf28640 1 -- 172.21.15.32:0/966709670 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80f806cb30 msgr2=0x7f80f80cdf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:00.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.534+0000 7f80fcf28640 1 --2- 172.21.15.32:0/966709670 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80f806cb30 0x7f80f80cdf80 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f80f0009d90 tx=0x7f80f002a560 comp rx=0 tx=0).stop 2024-04-24T23:09:00.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.534+0000 7f80fcf28640 1 -- 172.21.15.32:0/966709670 shutdown_connections 2024-04-24T23:09:00.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.534+0000 7f80fcf28640 1 --2- 172.21.15.32:0/966709670 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80f806cb30 0x7f80f80cdf80 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:00.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.534+0000 7f80fcf28640 1 --2- 172.21.15.32:0/966709670 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f80f815fd60 0x7f80f806c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:00.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.534+0000 7f80fcf28640 1 -- 172.21.15.32:0/966709670 >> 172.21.15.32:0/966709670 conn(0x7f80f806b110 msgr2=0x7f80f806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:00.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.534+0000 7f80fcf28640 1 -- 172.21.15.32:0/966709670 shutdown_connections 2024-04-24T23:09:00.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.534+0000 7f80fcf28640 1 -- 172.21.15.32:0/966709670 wait complete. 2024-04-24T23:09:00.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.535+0000 7f80fcf28640 1 Processor -- start 2024-04-24T23:09:00.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.535+0000 7f80fcf28640 1 -- start start 2024-04-24T23:09:00.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.536+0000 7f80fcf28640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f80f806cb30 0x7f80f815c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:00.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.536+0000 7f80fcf28640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80f815b480 0x7f80f8145c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:00.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.536+0000 7f80fcf28640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80f815fd60 0x7f80f8155980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:00.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.536+0000 7f80fcf28640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f80f815b860 con 0x7f80f815b480 2024-04-24T23:09:00.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.536+0000 7f80fcf28640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f80f80ce4c0 con 0x7f80f815fd60 2024-04-24T23:09:00.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.536+0000 7f80fcf28640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f80f815c3a0 con 0x7f80f806cb30 2024-04-24T23:09:00.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.536+0000 7f80f5d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80f815b480 0x7f80f8145c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:00.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.536+0000 7f80f5d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80f815b480 0x7f80f8145c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40164/0 (socket says 172.21.15.32:40164) 2024-04-24T23:09:00.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.536+0000 7f80f5d9b640 1 -- 172.21.15.32:0/4013553198 learned_addr learned my addr 172.21.15.32:0/4013553198 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:09:00.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.536+0000 7f80f6d9d640 1 --2- 172.21.15.32:0/4013553198 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80f815fd60 0x7f80f8155980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:00.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.537+0000 7f80f659c640 1 --2- 172.21.15.32:0/4013553198 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f80f806cb30 0x7f80f815c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:00.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.537+0000 7f80f5d9b640 1 -- 172.21.15.32:0/4013553198 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f80f806cb30 msgr2=0x7f80f815c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:00.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.537+0000 7f80f5d9b640 1 --2- 172.21.15.32:0/4013553198 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f80f806cb30 0x7f80f815c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:00.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.537+0000 7f80f5d9b640 1 -- 172.21.15.32:0/4013553198 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80f815fd60 msgr2=0x7f80f8155980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:00.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.537+0000 7f80f5d9b640 1 --2- 172.21.15.32:0/4013553198 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80f815fd60 0x7f80f8155980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:00.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.537+0000 7f80f5d9b640 1 -- 172.21.15.32:0/4013553198 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80f81492e0 con 0x7f80f815b480 2024-04-24T23:09:00.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.537+0000 7f80f6d9d640 1 --2- 172.21.15.32:0/4013553198 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80f815fd60 0x7f80f8155980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:09:00.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.537+0000 7f80f659c640 1 --2- 172.21.15.32:0/4013553198 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f80f806cb30 0x7f80f815c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:09:00.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.537+0000 7f80f5d9b640 1 --2- 172.21.15.32:0/4013553198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80f815b480 0x7f80f8145c70 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f80ec00e870 tx=0x7f80ec00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:00.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.537+0000 7f80e6ffd640 1 -- 172.21.15.32:0/4013553198 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ec017070 con 0x7f80f815b480 2024-04-24T23:09:00.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.537+0000 7f80e6ffd640 1 -- 172.21.15.32:0/4013553198 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80ec00c940 con 0x7f80f815b480 2024-04-24T23:09:00.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.537+0000 7f80fcf28640 1 -- 172.21.15.32:0/4013553198 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80f815bd50 con 0x7f80f815b480 2024-04-24T23:09:00.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.537+0000 7f80e6ffd640 1 -- 172.21.15.32:0/4013553198 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ec00cd30 con 0x7f80f815b480 2024-04-24T23:09:00.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.537+0000 7f80fcf28640 1 -- 172.21.15.32:0/4013553198 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80f81461b0 con 0x7f80f815b480 2024-04-24T23:09:00.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.538+0000 7f80fcf28640 1 -- 172.21.15.32:0/4013553198 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80f8068430 con 0x7f80f815b480 2024-04-24T23:09:00.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.539+0000 7f80e6ffd640 1 -- 172.21.15.32:0/4013553198 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f80ec01a070 con 0x7f80f815b480 2024-04-24T23:09:00.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.539+0000 7f80e6ffd640 1 --2- 172.21.15.32:0/4013553198 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f80d806be30 0x7f80d806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:00.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.539+0000 7f80e6ffd640 1 -- 172.21.15.32:0/4013553198 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5609+0+0 (secure 0 0 0) 0x7f80ec08b070 con 0x7f80f815b480 2024-04-24T23:09:00.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.539+0000 7f80f659c640 1 --2- 172.21.15.32:0/4013553198 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f80d806be30 0x7f80d806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:00.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.540+0000 7f80f659c640 1 --2- 172.21.15.32:0/4013553198 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f80d806be30 0x7f80d806e2f0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f80e8007fd0 tx=0x7f80e8007d40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:00.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.544+0000 7f80e6ffd640 1 -- 172.21.15.32:0/4013553198 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f80ec059a80 con 0x7f80f815b480 2024-04-24T23:09:00.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:00.768+0000 7f80fcf28640 1 -- 172.21.15.32:0/4013553198 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f80f80b6520 con 0x7f80f815b480 2024-04-24T23:09:01.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.427+0000 7f80e6ffd640 1 -- 172.21.15.32:0/4013553198 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 20 and data pool 21 v70) ==== 174+0+0 (secure 0 0 0) 0x7f80ec059380 con 0x7f80f815b480 2024-04-24T23:09:01.431 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 20 and data pool 21 2024-04-24T23:09:01.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.436+0000 7f80fcf28640 1 -- 172.21.15.32:0/4013553198 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f80d806be30 msgr2=0x7f80d806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:01.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.436+0000 7f80fcf28640 1 --2- 172.21.15.32:0/4013553198 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f80d806be30 0x7f80d806e2f0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f80e8007fd0 tx=0x7f80e8007d40 comp rx=0 tx=0).stop 2024-04-24T23:09:01.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.436+0000 7f80fcf28640 1 -- 172.21.15.32:0/4013553198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80f815b480 msgr2=0x7f80f8145c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:01.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.436+0000 7f80fcf28640 1 --2- 172.21.15.32:0/4013553198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80f815b480 0x7f80f8145c70 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f80ec00e870 tx=0x7f80ec00ed30 comp rx=0 tx=0).stop 2024-04-24T23:09:01.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.436+0000 7f80fcf28640 1 -- 172.21.15.32:0/4013553198 shutdown_connections 2024-04-24T23:09:01.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.436+0000 7f80fcf28640 1 --2- 172.21.15.32:0/4013553198 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f80d806be30 0x7f80d806e2f0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:01.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.436+0000 7f80fcf28640 1 --2- 172.21.15.32:0/4013553198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80f815b480 0x7f80f8145c70 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:01.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.436+0000 7f80fcf28640 1 --2- 172.21.15.32:0/4013553198 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80f815fd60 0x7f80f8155980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:01.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.436+0000 7f80fcf28640 1 --2- 172.21.15.32:0/4013553198 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f80f806cb30 0x7f80f815c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:01.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.436+0000 7f80fcf28640 1 -- 172.21.15.32:0/4013553198 >> 172.21.15.32:0/4013553198 conn(0x7f80f806b110 msgr2=0x7f80f80cc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:01.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.436+0000 7f80fcf28640 1 -- 172.21.15.32:0/4013553198 shutdown_connections 2024-04-24T23:09:01.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.437+0000 7f80fcf28640 1 -- 172.21.15.32:0/4013553198 wait complete. 2024-04-24T23:09:01.451 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:09:01.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.554+0000 7f6c55177640 1 Processor -- start 2024-04-24T23:09:01.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.554+0000 7f6c55177640 1 -- start start 2024-04-24T23:09:01.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.555+0000 7f6c55177640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6c5006b820 0x7f6c500d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:01.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.555+0000 7f6c55177640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6c500d4210 0x7f6c500b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:01.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.555+0000 7f6c55177640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c500d3170 con 0x7f6c5015e9b0 2024-04-24T23:09:01.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.555+0000 7f6c55177640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f6c500d3450 con 0x7f6c5006b820 2024-04-24T23:09:01.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.555+0000 7f6c55177640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f6c500d3730 con 0x7f6c500d4210 2024-04-24T23:09:01.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.555+0000 7f6c4e59c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6c5006b820 0x7f6c500d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:01.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.555+0000 7f6c4ed9d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6c5015e9b0 0x7f6c5006b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40586/0 (socket says 172.21.15.32:40586) 2024-04-24T23:09:01.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.555+0000 7f6c4ed9d640 1 -- 172.21.15.32:0/3163412393 learned_addr learned my addr 172.21.15.32:0/3163412393 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:09:01.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.555+0000 7f6c4f59e640 1 --2- 172.21.15.32:0/3163412393 >> v2:172.21.15.121:3300/0 conn(0x7f6c500d4210 0x7f6c500b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:01.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.556+0000 7f6c4e59c640 1 -- 172.21.15.32:0/3163412393 >> v2:172.21.15.121:3300/0 conn(0x7f6c500d4210 msgr2=0x7f6c500b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:09:01.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.556+0000 7f6c4e59c640 1 --2- 172.21.15.32:0/3163412393 >> v2:172.21.15.121:3300/0 conn(0x7f6c500d4210 0x7f6c500b78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:01.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.556+0000 7f6c4e59c640 1 -- 172.21.15.32:0/3163412393 >> v1:172.21.15.32:6789/0 conn(0x7f6c5015e9b0 legacy=0x7f6c5006b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:01.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.556+0000 7f6c4e59c640 1 -- 172.21.15.32:0/3163412393 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c500d3a10 con 0x7f6c5006b820 2024-04-24T23:09:01.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.556+0000 7f6c4e59c640 1 --2- 172.21.15.32:0/3163412393 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c5006b820 0x7f6c500d2b90 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f6c501622f0 tx=0x7f6c3802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a596be1000d7d7ab server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:01.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.556+0000 7f6c4dd9b640 1 -- 172.21.15.32:0/3163412393 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c38037070 con 0x7f6c5006b820 2024-04-24T23:09:01.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.556+0000 7f6c4dd9b640 1 -- 172.21.15.32:0/3163412393 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c38032440 con 0x7f6c5006b820 2024-04-24T23:09:01.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.556+0000 7f6c4dd9b640 1 -- 172.21.15.32:0/3163412393 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3802ae60 con 0x7f6c5006b820 2024-04-24T23:09:01.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.557+0000 7f6c55177640 1 -- 172.21.15.32:0/3163412393 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c5006b820 msgr2=0x7f6c500d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:01.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.557+0000 7f6c55177640 1 --2- 172.21.15.32:0/3163412393 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c5006b820 0x7f6c500d2b90 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f6c501622f0 tx=0x7f6c3802a110 comp rx=0 tx=0).stop 2024-04-24T23:09:01.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.557+0000 7f6c55177640 1 -- 172.21.15.32:0/3163412393 shutdown_connections 2024-04-24T23:09:01.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.557+0000 7f6c55177640 1 --2- 172.21.15.32:0/3163412393 >> v2:172.21.15.121:3300/0 conn(0x7f6c500d4210 0x7f6c500b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:01.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.557+0000 7f6c55177640 1 --2- 172.21.15.32:0/3163412393 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c5006b820 0x7f6c500d2b90 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:01.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.557+0000 7f6c55177640 1 -- 172.21.15.32:0/3163412393 >> 172.21.15.32:0/3163412393 conn(0x7f6c500d11d0 msgr2=0x7f6c500d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:01.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.557+0000 7f6c55177640 1 -- 172.21.15.32:0/3163412393 shutdown_connections 2024-04-24T23:09:01.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.557+0000 7f6c55177640 1 -- 172.21.15.32:0/3163412393 wait complete. 2024-04-24T23:09:01.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.558+0000 7f6c55177640 1 Processor -- start 2024-04-24T23:09:01.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.558+0000 7f6c55177640 1 -- start start 2024-04-24T23:09:01.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.559+0000 7f6c55177640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c5006b820 0x7f6c500ca460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:01.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.559+0000 7f6c55177640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c500d4210 0x7f6c500cd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:01.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.559+0000 7f6c55177640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6c5015e9b0 0x7f6c500cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:01.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.559+0000 7f6c55177640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6c500ba0e0 con 0x7f6c5006b820 2024-04-24T23:09:01.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.559+0000 7f6c55177640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6c500d2400 con 0x7f6c500d4210 2024-04-24T23:09:01.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.559+0000 7f6c55177640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f6c500ba260 con 0x7f6c5015e9b0 2024-04-24T23:09:01.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.559+0000 7f6c4ed9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c5006b820 0x7f6c500ca460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:01.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.559+0000 7f6c4ed9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c5006b820 0x7f6c500ca460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40178/0 (socket says 172.21.15.32:40178) 2024-04-24T23:09:01.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.559+0000 7f6c4ed9d640 1 -- 172.21.15.32:0/770605014 learned_addr learned my addr 172.21.15.32:0/770605014 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:09:01.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.560+0000 7f6c4e59c640 1 --2- 172.21.15.32:0/770605014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c500d4210 0x7f6c500cd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:01.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.560+0000 7f6c4ed9d640 1 -- 172.21.15.32:0/770605014 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6c5015e9b0 msgr2=0x7f6c500cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:01.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.560+0000 7f6c4ed9d640 1 --2- 172.21.15.32:0/770605014 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6c5015e9b0 0x7f6c500cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:01.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.560+0000 7f6c4ed9d640 1 -- 172.21.15.32:0/770605014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c500d4210 msgr2=0x7f6c500cd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:01.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.560+0000 7f6c4ed9d640 1 --2- 172.21.15.32:0/770605014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c500d4210 0x7f6c500cd9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:01.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.560+0000 7f6c4ed9d640 1 -- 172.21.15.32:0/770605014 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c501f8850 con 0x7f6c5006b820 2024-04-24T23:09:01.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.560+0000 7f6c4ed9d640 1 --2- 172.21.15.32:0/770605014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c5006b820 0x7f6c500ca460 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f6c4800c3c0 tx=0x7f6c4800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:01.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.560+0000 7f6c477fe640 1 -- 172.21.15.32:0/770605014 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c48017070 con 0x7f6c5006b820 2024-04-24T23:09:01.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.560+0000 7f6c477fe640 1 -- 172.21.15.32:0/770605014 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c48005ce0 con 0x7f6c5006b820 2024-04-24T23:09:01.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.560+0000 7f6c55177640 1 -- 172.21.15.32:0/770605014 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c50157b30 con 0x7f6c5006b820 2024-04-24T23:09:01.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.560+0000 7f6c477fe640 1 -- 172.21.15.32:0/770605014 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c480072f0 con 0x7f6c5006b820 2024-04-24T23:09:01.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.560+0000 7f6c55177640 1 -- 172.21.15.32:0/770605014 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c500ca9a0 con 0x7f6c5006b820 2024-04-24T23:09:01.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.561+0000 7f6c55177640 1 -- 172.21.15.32:0/770605014 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c50068430 con 0x7f6c5006b820 2024-04-24T23:09:01.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.562+0000 7f6c477fe640 1 -- 172.21.15.32:0/770605014 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6c4801a070 con 0x7f6c5006b820 2024-04-24T23:09:01.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.562+0000 7f6c477fe640 1 --2- 172.21.15.32:0/770605014 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6c3006be60 0x7f6c3006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:01.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.562+0000 7f6c477fe640 1 -- 172.21.15.32:0/770605014 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5709+0+0 (secure 0 0 0) 0x7f6c4808a1c0 con 0x7f6c5006b820 2024-04-24T23:09:01.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.562+0000 7f6c4e59c640 1 --2- 172.21.15.32:0/770605014 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6c3006be60 0x7f6c3006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:01.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.563+0000 7f6c4e59c640 1 --2- 172.21.15.32:0/770605014 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6c3006be60 0x7f6c3006e320 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f6c38034000 tx=0x7f6c3802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:01.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.567+0000 7f6c477fe640 1 -- 172.21.15.32:0/770605014 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6c48058b70 con 0x7f6c5006b820 2024-04-24T23:09:01.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.750+0000 7f6c55177640 1 -- 172.21.15.32:0/770605014 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6c500d16b0 con 0x7f6c5006b820 2024-04-24T23:09:01.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.751+0000 7f6c477fe640 1 -- 172.21.15.32:0/770605014 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v168) ==== 74+0+16705 (secure 0 0 0) 0x7f6c48058470 con 0x7f6c5006b820 2024-04-24T23:09:01.753 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:09:01.753 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":168,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:09:00.769996+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:08:59.076360+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-24T23:08:59.745667+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:09:01.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.754+0000 7f6c55177640 1 -- 172.21.15.32:0/770605014 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6c3006be60 msgr2=0x7f6c3006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:01.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.754+0000 7f6c55177640 1 --2- 172.21.15.32:0/770605014 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6c3006be60 0x7f6c3006e320 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f6c38034000 tx=0x7f6c3802e000 comp rx=0 tx=0).stop 2024-04-24T23:09:01.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.754+0000 7f6c55177640 1 -- 172.21.15.32:0/770605014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c5006b820 msgr2=0x7f6c500ca460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:01.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.754+0000 7f6c55177640 1 --2- 172.21.15.32:0/770605014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c5006b820 0x7f6c500ca460 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f6c4800c3c0 tx=0x7f6c4800c880 comp rx=0 tx=0).stop 2024-04-24T23:09:01.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.754+0000 7f6c55177640 1 -- 172.21.15.32:0/770605014 shutdown_connections 2024-04-24T23:09:01.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.754+0000 7f6c55177640 1 --2- 172.21.15.32:0/770605014 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6c3006be60 0x7f6c3006e320 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:01.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.754+0000 7f6c55177640 1 --2- 172.21.15.32:0/770605014 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6c5015e9b0 0x7f6c500cdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:01.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.754+0000 7f6c55177640 1 --2- 172.21.15.32:0/770605014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c500d4210 0x7f6c500cd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:01.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.754+0000 7f6c55177640 1 --2- 172.21.15.32:0/770605014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c5006b820 0x7f6c500ca460 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:01.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.754+0000 7f6c55177640 1 -- 172.21.15.32:0/770605014 >> 172.21.15.32:0/770605014 conn(0x7f6c500d11d0 msgr2=0x7f6c500b7b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:01.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.754+0000 7f6c55177640 1 -- 172.21.15.32:0/770605014 shutdown_connections 2024-04-24T23:09:01.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.754+0000 7f6c55177640 1 -- 172.21.15.32:0/770605014 wait complete. 2024-04-24T23:09:01.767 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:09:01.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.867+0000 7f15dce46640 1 Processor -- start 2024-04-24T23:09:01.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.867+0000 7f15dce46640 1 -- start start 2024-04-24T23:09:01.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.868+0000 7f15dce46640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f15d80b5af0 0x7f15d80ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:01.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.868+0000 7f15dce46640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15d80cd270 con 0x7f15d8162320 2024-04-24T23:09:01.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.868+0000 7f15dce46640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15d8071610 con 0x7f15d8162e10 2024-04-24T23:09:01.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.868+0000 7f15dce46640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f15d80b5660 con 0x7f15d80b5af0 2024-04-24T23:09:01.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.868+0000 7f15d659c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f15d8162320 0x7f15d8162700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40592/0 (socket says 172.21.15.32:40592) 2024-04-24T23:09:01.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.868+0000 7f15d659c640 1 -- 172.21.15.32:0/16007141 learned_addr learned my addr 172.21.15.32:0/16007141 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:09:01.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.868+0000 7f15d5d9b640 1 -- 172.21.15.32:0/16007141 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1010215777 0 0) 0x7f15d80cd270 con 0x7f15d8162320 2024-04-24T23:09:01.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.868+0000 7f15d5d9b640 1 -- 172.21.15.32:0/16007141 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f15c4003520 con 0x7f15d8162320 2024-04-24T23:09:01.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.868+0000 7f15cffff640 1 --2- 172.21.15.32:0/16007141 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f15d80b5af0 0x7f15d80ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:01.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.868+0000 7f15d5d9b640 1 -- 172.21.15.32:0/16007141 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3929570312 0 0) 0x7f15c4003520 con 0x7f15d8162320 2024-04-24T23:09:01.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.868+0000 7f15d5d9b640 1 -- 172.21.15.32:0/16007141 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f15d80b5af0 msgr2=0x7f15d80ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:09:01.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.868+0000 7f15d5d9b640 1 --2- 172.21.15.32:0/16007141 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f15d80b5af0 0x7f15d80ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:01.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.868+0000 7f15d5d9b640 1 -- 172.21.15.32:0/16007141 >> v1:172.21.15.121:6789/0 conn(0x7f15d8162e10 legacy=0x7f15d815ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:01.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.868+0000 7f15d5d9b640 1 -- 172.21.15.32:0/16007141 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15d80718f0 con 0x7f15d8162320 2024-04-24T23:09:01.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.868+0000 7f15d5d9b640 1 -- 172.21.15.32:0/16007141 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f15c8001670 con 0x7f15d8162320 2024-04-24T23:09:01.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.868+0000 7f15d5d9b640 1 -- 172.21.15.32:0/16007141 >> v1:172.21.15.32:6789/0 conn(0x7f15d8162320 legacy=0x7f15d8162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:01.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.868+0000 7f15d5d9b640 1 --2- 172.21.15.32:0/16007141 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f15d80b5af0 0x7f15c40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:01.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.868+0000 7f15d5d9b640 1 --2- 172.21.15.32:0/16007141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15c4003f60 0x7f15c4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:01.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.868+0000 7f15d5d9b640 1 --2- 172.21.15.32:0/16007141 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15c4004d40 0x7f15c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:01.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.868+0000 7f15d5d9b640 1 -- 172.21.15.32:0/16007141 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f15d80b5660 con 0x7f15c4003f60 2024-04-24T23:09:01.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.868+0000 7f15d5d9b640 1 -- 172.21.15.32:0/16007141 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f15c400bc00 con 0x7f15c4004d40 2024-04-24T23:09:01.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.868+0000 7f15d5d9b640 1 -- 172.21.15.32:0/16007141 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f15c400be00 con 0x7f15d80b5af0 2024-04-24T23:09:01.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.869+0000 7f15cffff640 1 --2- 172.21.15.32:0/16007141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15c4003f60 0x7f15c4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:01.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.869+0000 7f15d659c640 1 --2- 172.21.15.32:0/16007141 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f15d80b5af0 0x7f15c40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:01.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.869+0000 7f15d6d9d640 1 --2- 172.21.15.32:0/16007141 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15c4004d40 0x7f15c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:01.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.869+0000 7f15cffff640 1 -- 172.21.15.32:0/16007141 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f15d80b5af0 msgr2=0x7f15c40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:09:01.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.869+0000 7f15cffff640 1 --2- 172.21.15.32:0/16007141 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f15d80b5af0 0x7f15c40039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:01.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.869+0000 7f15cffff640 1 -- 172.21.15.32:0/16007141 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15c4004d40 msgr2=0x7f15c40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:09:01.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.869+0000 7f15cffff640 1 --2- 172.21.15.32:0/16007141 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15c4004d40 0x7f15c40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:01.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.869+0000 7f15cffff640 1 -- 172.21.15.32:0/16007141 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15c4003520 con 0x7f15c4003f60 2024-04-24T23:09:01.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.870+0000 7f15cffff640 1 --2- 172.21.15.32:0/16007141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15c4003f60 0x7f15c4004800 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f15c0009a90 tx=0x7f15c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bee5a31eb100388 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:01.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.870+0000 7f15d5d9b640 1 -- 172.21.15.32:0/16007141 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c0036070 con 0x7f15c4003f60 2024-04-24T23:09:01.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.870+0000 7f15d5d9b640 1 -- 172.21.15.32:0/16007141 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15c0031400 con 0x7f15c4003f60 2024-04-24T23:09:01.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.870+0000 7f15d5d9b640 1 -- 172.21.15.32:0/16007141 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c0009690 con 0x7f15c4003f60 2024-04-24T23:09:01.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.870+0000 7f15dce46640 1 -- 172.21.15.32:0/16007141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15c4003f60 msgr2=0x7f15c4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:01.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.870+0000 7f15dce46640 1 --2- 172.21.15.32:0/16007141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15c4003f60 0x7f15c4004800 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f15c0009a90 tx=0x7f15c4005830 comp rx=0 tx=0).stop 2024-04-24T23:09:01.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.870+0000 7f15d6d9d640 1 -- 172.21.15.32:0/16007141 reap_dead start 2024-04-24T23:09:01.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.870+0000 7f15dce46640 1 -- 172.21.15.32:0/16007141 shutdown_connections 2024-04-24T23:09:01.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.870+0000 7f15dce46640 1 -- 172.21.15.32:0/16007141 >> 172.21.15.32:0/16007141 conn(0x7f15d80d0ae0 msgr2=0x7f15d80d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:01.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.870+0000 7f15dce46640 1 -- 172.21.15.32:0/16007141 shutdown_connections 2024-04-24T23:09:01.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.870+0000 7f15dce46640 1 -- 172.21.15.32:0/16007141 wait complete. 2024-04-24T23:09:01.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.872+0000 7f15dce46640 1 Processor -- start 2024-04-24T23:09:01.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.872+0000 7f15dce46640 1 -- start start 2024-04-24T23:09:01.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.872+0000 7f15dce46640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f15c4003f60 0x7f15d8150820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:01.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.872+0000 7f15dce46640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15d8155060 0x7f15d8152360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:01.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.872+0000 7f15dce46640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15d81528a0 0x7f15d806dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:01.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.872+0000 7f15dce46640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f15d8071bd0 con 0x7f15d8155060 2024-04-24T23:09:01.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.872+0000 7f15dce46640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f15d8067950 con 0x7f15d81528a0 2024-04-24T23:09:01.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.872+0000 7f15dce46640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f15d8071d50 con 0x7f15c4003f60 2024-04-24T23:09:01.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.873+0000 7f15cffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15d8155060 0x7f15d8152360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:01.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.873+0000 7f15cffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15d8155060 0x7f15d8152360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40198/0 (socket says 172.21.15.32:40198) 2024-04-24T23:09:01.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.873+0000 7f15cffff640 1 -- 172.21.15.32:0/3052681685 learned_addr learned my addr 172.21.15.32:0/3052681685 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:09:01.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.873+0000 7f15d659c640 1 --2- 172.21.15.32:0/3052681685 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f15c4003f60 0x7f15d8150820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:01.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.873+0000 7f15d6d9d640 1 --2- 172.21.15.32:0/3052681685 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15d81528a0 0x7f15d806dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:01.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.873+0000 7f15cffff640 1 -- 172.21.15.32:0/3052681685 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f15c4003f60 msgr2=0x7f15d8150820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:01.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.873+0000 7f15cffff640 1 --2- 172.21.15.32:0/3052681685 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f15c4003f60 0x7f15d8150820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:01.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.873+0000 7f15cffff640 1 -- 172.21.15.32:0/3052681685 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15d81528a0 msgr2=0x7f15d806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:01.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.873+0000 7f15cffff640 1 --2- 172.21.15.32:0/3052681685 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15d81528a0 0x7f15d806dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:01.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.873+0000 7f15cffff640 1 -- 172.21.15.32:0/3052681685 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15d8152db0 con 0x7f15d8155060 2024-04-24T23:09:01.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.873+0000 7f15cffff640 1 --2- 172.21.15.32:0/3052681685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15d8155060 0x7f15d8152360 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f15c0000c00 tx=0x7f15c0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:01.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.873+0000 7f15ceffd640 1 -- 172.21.15.32:0/3052681685 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c0040020 con 0x7f15d8155060 2024-04-24T23:09:01.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.873+0000 7f15ceffd640 1 -- 172.21.15.32:0/3052681685 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15c0003c60 con 0x7f15d8155060 2024-04-24T23:09:01.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.873+0000 7f15dce46640 1 -- 172.21.15.32:0/3052681685 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15d80ce720 con 0x7f15d8155060 2024-04-24T23:09:01.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.873+0000 7f15ceffd640 1 -- 172.21.15.32:0/3052681685 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c0036040 con 0x7f15d8155060 2024-04-24T23:09:01.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.873+0000 7f15dce46640 1 -- 172.21.15.32:0/3052681685 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15d80cece0 con 0x7f15d8155060 2024-04-24T23:09:01.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.874+0000 7f15dce46640 1 -- 172.21.15.32:0/3052681685 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15d8068430 con 0x7f15d8155060 2024-04-24T23:09:01.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.875+0000 7f15ceffd640 1 -- 172.21.15.32:0/3052681685 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f15c0043020 con 0x7f15d8155060 2024-04-24T23:09:01.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.875+0000 7f15ceffd640 1 --2- 172.21.15.32:0/3052681685 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f15b806be30 0x7f15b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:01.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.875+0000 7f15ceffd640 1 -- 172.21.15.32:0/3052681685 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5709+0+0 (secure 0 0 0) 0x7f15c00ab1e0 con 0x7f15d8155060 2024-04-24T23:09:01.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.875+0000 7f15d659c640 1 --2- 172.21.15.32:0/3052681685 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f15b806be30 0x7f15b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:01.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.876+0000 7f15d659c640 1 --2- 172.21.15.32:0/3052681685 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f15b806be30 0x7f15b806e2f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f15d806e9d0 tx=0x7f15c8001c10 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:01.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:01.879+0000 7f15ceffd640 1 -- 172.21.15.32:0/3052681685 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f15c004a030 con 0x7f15d8155060 2024-04-24T23:09:02.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.062+0000 7f15dce46640 1 -- 172.21.15.32:0/3052681685 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f15d8067730 con 0x7f15d8155060 2024-04-24T23:09:02.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.063+0000 7f15ceffd640 1 -- 172.21.15.32:0/3052681685 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v168) ==== 74+0+16705 (secure 0 0 0) 0x7f15c00044e0 con 0x7f15d8155060 2024-04-24T23:09:02.065 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:09:02.065 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":168,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:09:00.769996+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:08:59.076360+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-24T23:08:59.745667+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:09:02.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.065+0000 7f15dce46640 1 -- 172.21.15.32:0/3052681685 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f15b806be30 msgr2=0x7f15b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:02.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.065+0000 7f15dce46640 1 --2- 172.21.15.32:0/3052681685 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f15b806be30 0x7f15b806e2f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f15d806e9d0 tx=0x7f15c8001c10 comp rx=0 tx=0).stop 2024-04-24T23:09:02.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.065+0000 7f15dce46640 1 -- 172.21.15.32:0/3052681685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15d8155060 msgr2=0x7f15d8152360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:02.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.065+0000 7f15dce46640 1 --2- 172.21.15.32:0/3052681685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15d8155060 0x7f15d8152360 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f15c0000c00 tx=0x7f15c0004730 comp rx=0 tx=0).stop 2024-04-24T23:09:02.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.065+0000 7f15dce46640 1 -- 172.21.15.32:0/3052681685 shutdown_connections 2024-04-24T23:09:02.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.065+0000 7f15dce46640 1 --2- 172.21.15.32:0/3052681685 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f15b806be30 0x7f15b806e2f0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:02.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.065+0000 7f15dce46640 1 --2- 172.21.15.32:0/3052681685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15d8155060 0x7f15d8152360 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:02.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.065+0000 7f15dce46640 1 --2- 172.21.15.32:0/3052681685 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15d81528a0 0x7f15d806dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:02.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.065+0000 7f15dce46640 1 --2- 172.21.15.32:0/3052681685 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f15c4003f60 0x7f15d8150820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:02.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.065+0000 7f15dce46640 1 -- 172.21.15.32:0/3052681685 >> 172.21.15.32:0/3052681685 conn(0x7f15d80d0ae0 msgr2=0x7f15d80b0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:02.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.065+0000 7f15dce46640 1 -- 172.21.15.32:0/3052681685 shutdown_connections 2024-04-24T23:09:02.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.065+0000 7f15dce46640 1 -- 172.21.15.32:0/3052681685 wait complete. 2024-04-24T23:09:02.078 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-24T23:09:02.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.180+0000 7f9849038640 1 Processor -- start 2024-04-24T23:09:02.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.180+0000 7f9849038640 1 -- start start 2024-04-24T23:09:02.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.181+0000 7f9849038640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f984406b820 0x7f98440d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:02.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.181+0000 7f9849038640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98440d3170 con 0x7f984415e9b0 2024-04-24T23:09:02.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.181+0000 7f9849038640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98440d3450 con 0x7f98440d4210 2024-04-24T23:09:02.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.181+0000 7f9849038640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f98440d3730 con 0x7f984406b820 2024-04-24T23:09:02.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.181+0000 7f9842d9d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f984415e9b0 0x7f984406b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40602/0 (socket says 172.21.15.32:40602) 2024-04-24T23:09:02.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.181+0000 7f9842d9d640 1 -- 172.21.15.32:0/3119668124 learned_addr learned my addr 172.21.15.32:0/3119668124 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:09:02.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.181+0000 7f984259c640 1 --2- 172.21.15.32:0/3119668124 >> v2:172.21.15.121:3300/0 conn(0x7f984406b820 0x7f98440d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:02.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.181+0000 7f9841d9b640 1 -- 172.21.15.32:0/3119668124 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1610172794 0 0) 0x7f98440d3170 con 0x7f984415e9b0 2024-04-24T23:09:02.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.181+0000 7f9841d9b640 1 -- 172.21.15.32:0/3119668124 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9830003520 con 0x7f984415e9b0 2024-04-24T23:09:02.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.181+0000 7f9841d9b640 1 -- 172.21.15.32:0/3119668124 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 326030609 0 0) 0x7f98440d3450 con 0x7f98440d4210 2024-04-24T23:09:02.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.181+0000 7f9841d9b640 1 -- 172.21.15.32:0/3119668124 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f98440d3170 con 0x7f98440d4210 2024-04-24T23:09:02.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.181+0000 7f984259c640 1 -- 172.21.15.32:0/3119668124 >> v1:172.21.15.121:6789/0 conn(0x7f98440d4210 legacy=0x7f98440b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:02.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.181+0000 7f984259c640 1 -- 172.21.15.32:0/3119668124 >> v1:172.21.15.32:6789/0 conn(0x7f984415e9b0 legacy=0x7f984406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:02.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.181+0000 7f984259c640 1 -- 172.21.15.32:0/3119668124 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98440d3a10 con 0x7f984406b820 2024-04-24T23:09:02.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.182+0000 7f984259c640 1 --2- 172.21.15.32:0/3119668124 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f984406b820 0x7f98440d2b90 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f98340097a0 tx=0x7f983402a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2dfe6ee1fe4a65ce server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:02.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.182+0000 7f9841d9b640 1 -- 172.21.15.32:0/3119668124 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9834037070 con 0x7f984406b820 2024-04-24T23:09:02.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.182+0000 7f9841d9b640 1 -- 172.21.15.32:0/3119668124 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9834032440 con 0x7f984406b820 2024-04-24T23:09:02.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.182+0000 7f9841d9b640 1 -- 172.21.15.32:0/3119668124 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9834032810 con 0x7f984406b820 2024-04-24T23:09:02.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.183+0000 7f9849038640 1 -- 172.21.15.32:0/3119668124 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f984406b820 msgr2=0x7f98440d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:02.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.183+0000 7f9849038640 1 --2- 172.21.15.32:0/3119668124 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f984406b820 0x7f98440d2b90 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f98340097a0 tx=0x7f983402a1a0 comp rx=0 tx=0).stop 2024-04-24T23:09:02.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.183+0000 7f9849038640 1 -- 172.21.15.32:0/3119668124 shutdown_connections 2024-04-24T23:09:02.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.183+0000 7f9849038640 1 --2- 172.21.15.32:0/3119668124 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f984406b820 0x7f98440d2b90 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:02.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.183+0000 7f9849038640 1 -- 172.21.15.32:0/3119668124 >> 172.21.15.32:0/3119668124 conn(0x7f98440d11d0 msgr2=0x7f98440d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:02.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.183+0000 7f9849038640 1 -- 172.21.15.32:0/3119668124 shutdown_connections 2024-04-24T23:09:02.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.183+0000 7f9849038640 1 -- 172.21.15.32:0/3119668124 wait complete. 2024-04-24T23:09:02.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.184+0000 7f9849038640 1 Processor -- start 2024-04-24T23:09:02.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.184+0000 7f9849038640 1 -- start start 2024-04-24T23:09:02.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.185+0000 7f9849038640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f984406b820 0x7f9844158730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:02.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.185+0000 7f9849038640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98440d4210 0x7f9844158c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:02.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.185+0000 7f9849038640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f984415e9b0 0x7f98441e3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:02.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.185+0000 7f9849038640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f98440ba0e0 con 0x7f98440d4210 2024-04-24T23:09:02.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.185+0000 7f9849038640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f98440d2400 con 0x7f984406b820 2024-04-24T23:09:02.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.185+0000 7f9849038640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f98440ba260 con 0x7f984415e9b0 2024-04-24T23:09:02.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.185+0000 7f984259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98440d4210 0x7f9844158c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:02.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.185+0000 7f984259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98440d4210 0x7f9844158c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40208/0 (socket says 172.21.15.32:40208) 2024-04-24T23:09:02.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.185+0000 7f984259c640 1 -- 172.21.15.32:0/3527274184 learned_addr learned my addr 172.21.15.32:0/3527274184 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:09:02.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.185+0000 7f9842d9d640 1 --2- 172.21.15.32:0/3527274184 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f984406b820 0x7f9844158730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:02.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.185+0000 7f984259c640 1 -- 172.21.15.32:0/3527274184 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f984415e9b0 msgr2=0x7f98441e3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:02.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.185+0000 7f984259c640 1 --2- 172.21.15.32:0/3527274184 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f984415e9b0 0x7f98441e3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:02.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.185+0000 7f984259c640 1 -- 172.21.15.32:0/3527274184 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f984406b820 msgr2=0x7f9844158730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:02.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.185+0000 7f984259c640 1 --2- 172.21.15.32:0/3527274184 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f984406b820 0x7f9844158730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:02.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.185+0000 7f984259c640 1 -- 172.21.15.32:0/3527274184 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98441e46b0 con 0x7f98440d4210 2024-04-24T23:09:02.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.186+0000 7f984259c640 1 --2- 172.21.15.32:0/3527274184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98440d4210 0x7f9844158c70 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f9834035040 tx=0x7f98340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:02.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.186+0000 7f983b7fe640 1 -- 172.21.15.32:0/3527274184 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9834041030 con 0x7f98440d4210 2024-04-24T23:09:02.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.186+0000 7f983b7fe640 1 -- 172.21.15.32:0/3527274184 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f983403d4e0 con 0x7f98440d4210 2024-04-24T23:09:02.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.186+0000 7f983b7fe640 1 -- 172.21.15.32:0/3527274184 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9834037040 con 0x7f98440d4210 2024-04-24T23:09:02.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.186+0000 7f9849038640 1 -- 172.21.15.32:0/3527274184 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98441e4990 con 0x7f98440d4210 2024-04-24T23:09:02.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.186+0000 7f9849038640 1 -- 172.21.15.32:0/3527274184 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98441f5310 con 0x7f98440d4210 2024-04-24T23:09:02.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.187+0000 7f9849038640 1 -- 172.21.15.32:0/3527274184 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98441e4b60 con 0x7f98440d4210 2024-04-24T23:09:02.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.187+0000 7f983b7fe640 1 -- 172.21.15.32:0/3527274184 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9834044070 con 0x7f98440d4210 2024-04-24T23:09:02.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.188+0000 7f983b7fe640 1 --2- 172.21.15.32:0/3527274184 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f982406be60 0x7f982406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:02.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.188+0000 7f983b7fe640 1 -- 172.21.15.32:0/3527274184 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5709+0+0 (secure 0 0 0) 0x7f98340ab460 con 0x7f98440d4210 2024-04-24T23:09:02.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.188+0000 7f9842d9d640 1 --2- 172.21.15.32:0/3527274184 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f982406be60 0x7f982406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:02.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.189+0000 7f9842d9d640 1 --2- 172.21.15.32:0/3527274184 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f982406be60 0x7f982406e320 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f982c0011e0 tx=0x7f982c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:02.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.192+0000 7f983b7fe640 1 -- 172.21.15.32:0/3527274184 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9834079e20 con 0x7f98440d4210 2024-04-24T23:09:02.394 INFO:tasks.ceph.mds.c.smithi032.stderr:2024-04-24T23:09:02.393+0000 7fc4d4d3b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T23:09:02.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:02.414+0000 7f9849038640 1 -- 172.21.15.32:0/3527274184 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f9844068430 con 0x7f98440d4210 2024-04-24T23:09:03.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.413+0000 7f983b7fe640 1 -- 172.21.15.32:0/3527274184 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v72) ==== 118+0+0 (secure 0 0 0) 0x7f9834079720 con 0x7f98440d4210 2024-04-24T23:09:03.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.416+0000 7f9849038640 1 -- 172.21.15.32:0/3527274184 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f982406be60 msgr2=0x7f982406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:03.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.416+0000 7f9849038640 1 --2- 172.21.15.32:0/3527274184 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f982406be60 0x7f982406e320 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f982c0011e0 tx=0x7f982c009040 comp rx=0 tx=0).stop 2024-04-24T23:09:03.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.416+0000 7f9849038640 1 -- 172.21.15.32:0/3527274184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98440d4210 msgr2=0x7f9844158c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:03.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.416+0000 7f9849038640 1 --2- 172.21.15.32:0/3527274184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98440d4210 0x7f9844158c70 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f9834035040 tx=0x7f98340046c0 comp rx=0 tx=0).stop 2024-04-24T23:09:03.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.416+0000 7f9849038640 1 -- 172.21.15.32:0/3527274184 shutdown_connections 2024-04-24T23:09:03.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.416+0000 7f9849038640 1 --2- 172.21.15.32:0/3527274184 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f982406be60 0x7f982406e320 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:03.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.416+0000 7f9849038640 1 --2- 172.21.15.32:0/3527274184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98440d4210 0x7f9844158c70 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:03.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.416+0000 7f9849038640 1 --2- 172.21.15.32:0/3527274184 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f984415e9b0 0x7f98441e3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:03.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.416+0000 7f9849038640 1 --2- 172.21.15.32:0/3527274184 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f984406b820 0x7f9844158730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:03.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.416+0000 7f9849038640 1 -- 172.21.15.32:0/3527274184 >> 172.21.15.32:0/3527274184 conn(0x7f98440d11d0 msgr2=0x7f9844071730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:03.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.416+0000 7f9849038640 1 -- 172.21.15.32:0/3527274184 shutdown_connections 2024-04-24T23:09:03.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.416+0000 7f9849038640 1 -- 172.21.15.32:0/3527274184 wait complete. 2024-04-24T23:09:03.431 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-24T23:09:03.431 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-24T23:09:03.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.540+0000 7ff96ee83640 1 Processor -- start 2024-04-24T23:09:03.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.540+0000 7ff96ee83640 1 -- start start 2024-04-24T23:09:03.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.541+0000 7ff96ee83640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff96815b380 0x7ff96806c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:03.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.541+0000 7ff96ee83640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9680b4520 con 0x7ff96815cb00 2024-04-24T23:09:03.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.541+0000 7ff96ee83640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9680b4800 con 0x7ff9680ceb40 2024-04-24T23:09:03.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.541+0000 7ff96ee83640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff9680cef90 con 0x7ff96815b380 2024-04-24T23:09:03.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.541+0000 7ff967fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff96815cb00 0x7ff9680ce430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40608/0 (socket says 172.21.15.32:40608) 2024-04-24T23:09:03.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.541+0000 7ff967fff640 1 -- 172.21.15.32:0/3760941105 learned_addr learned my addr 172.21.15.32:0/3760941105 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:09:03.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.541+0000 7ff9677fe640 1 -- 172.21.15.32:0/3760941105 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1007723489 0 0) 0x7ff9680b4520 con 0x7ff96815cb00 2024-04-24T23:09:03.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.541+0000 7ff9677fe640 1 -- 172.21.15.32:0/3760941105 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff958003520 con 0x7ff96815cb00 2024-04-24T23:09:03.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.541+0000 7ff96cc1f640 1 --2- 172.21.15.32:0/3760941105 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff96815b380 0x7ff96806c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:03.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.541+0000 7ff9677fe640 1 -- 172.21.15.32:0/3760941105 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3679447642 0 0) 0x7ff958003520 con 0x7ff96815cb00 2024-04-24T23:09:03.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.541+0000 7ff9677fe640 1 -- 172.21.15.32:0/3760941105 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff96815b380 msgr2=0x7ff96806c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:09:03.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.541+0000 7ff9677fe640 1 --2- 172.21.15.32:0/3760941105 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff96815b380 0x7ff96806c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:03.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.541+0000 7ff9677fe640 1 -- 172.21.15.32:0/3760941105 >> v1:172.21.15.121:6789/0 conn(0x7ff9680ceb40 legacy=0x7ff9680b55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:03.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.541+0000 7ff9677fe640 1 -- 172.21.15.32:0/3760941105 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9680b4ae0 con 0x7ff96815cb00 2024-04-24T23:09:03.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.541+0000 7ff9677fe640 1 -- 172.21.15.32:0/3760941105 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7ff954001670 con 0x7ff96815cb00 2024-04-24T23:09:03.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.541+0000 7ff9677fe640 1 -- 172.21.15.32:0/3760941105 >> v1:172.21.15.32:6789/0 conn(0x7ff96815cb00 legacy=0x7ff9680ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:03.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.541+0000 7ff9677fe640 1 --2- 172.21.15.32:0/3760941105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9580039e0 0x7ff958003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:03.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.541+0000 7ff9677fe640 1 --2- 172.21.15.32:0/3760941105 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff958004800 0x7ff958004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:03.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.541+0000 7ff9677fe640 1 --2- 172.21.15.32:0/3760941105 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff96815b380 0x7ff95800dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:03.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.541+0000 7ff9677fe640 1 -- 172.21.15.32:0/3760941105 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff9680cef90 con 0x7ff9580039e0 2024-04-24T23:09:03.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.541+0000 7ff9677fe640 1 -- 172.21.15.32:0/3760941105 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff958005520 con 0x7ff958004800 2024-04-24T23:09:03.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.541+0000 7ff9677fe640 1 -- 172.21.15.32:0/3760941105 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff95800e110 con 0x7ff96815b380 2024-04-24T23:09:03.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.541+0000 7ff96cc1f640 1 --2- 172.21.15.32:0/3760941105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9580039e0 0x7ff958003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:03.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.542+0000 7ff967fff640 1 --2- 172.21.15.32:0/3760941105 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff958004800 0x7ff958004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:03.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.542+0000 7ff96d420640 1 --2- 172.21.15.32:0/3760941105 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff96815b380 0x7ff95800dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:03.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.542+0000 7ff96cc1f640 1 -- 172.21.15.32:0/3760941105 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff96815b380 msgr2=0x7ff95800dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:09:03.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.542+0000 7ff96cc1f640 1 --2- 172.21.15.32:0/3760941105 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff96815b380 0x7ff95800dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:03.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.542+0000 7ff96cc1f640 1 -- 172.21.15.32:0/3760941105 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff958004800 msgr2=0x7ff958004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:09:03.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.542+0000 7ff96cc1f640 1 --2- 172.21.15.32:0/3760941105 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff958004800 0x7ff958004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:03.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.542+0000 7ff96cc1f640 1 -- 172.21.15.32:0/3760941105 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff958003520 con 0x7ff9580039e0 2024-04-24T23:09:03.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.542+0000 7ff96cc1f640 1 --2- 172.21.15.32:0/3760941105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9580039e0 0x7ff958003e00 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7ff95c00ba30 tx=0x7ff958005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60108d993d3fe402 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:03.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.543+0000 7ff9677fe640 1 -- 172.21.15.32:0/3760941105 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c030070 con 0x7ff9580039e0 2024-04-24T23:09:03.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.543+0000 7ff9677fe640 1 -- 172.21.15.32:0/3760941105 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff95c02ade0 con 0x7ff9580039e0 2024-04-24T23:09:03.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.543+0000 7ff9677fe640 1 -- 172.21.15.32:0/3760941105 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c00b600 con 0x7ff9580039e0 2024-04-24T23:09:03.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.543+0000 7ff96ee83640 1 -- 172.21.15.32:0/3760941105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9580039e0 msgr2=0x7ff958003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:03.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.543+0000 7ff96ee83640 1 --2- 172.21.15.32:0/3760941105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9580039e0 0x7ff958003e00 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7ff95c00ba30 tx=0x7ff958005830 comp rx=0 tx=0).stop 2024-04-24T23:09:03.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.543+0000 7ff96d420640 1 -- 172.21.15.32:0/3760941105 reap_dead start 2024-04-24T23:09:03.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.543+0000 7ff96ee83640 1 -- 172.21.15.32:0/3760941105 shutdown_connections 2024-04-24T23:09:03.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.543+0000 7ff96ee83640 1 -- 172.21.15.32:0/3760941105 >> 172.21.15.32:0/3760941105 conn(0x7ff96806b110 msgr2=0x7ff96806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:03.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.543+0000 7ff96ee83640 1 -- 172.21.15.32:0/3760941105 shutdown_connections 2024-04-24T23:09:03.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.543+0000 7ff96ee83640 1 -- 172.21.15.32:0/3760941105 wait complete. 2024-04-24T23:09:03.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.544+0000 7ff96ee83640 1 Processor -- start 2024-04-24T23:09:03.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.544+0000 7ff96ee83640 1 -- start start 2024-04-24T23:09:03.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.545+0000 7ff96ee83640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9580039e0 0x7ff968146d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:03.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.545+0000 7ff96ee83640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9680ae8b0 0x7ff9680aed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:03.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.545+0000 7ff96ee83640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff968070680 0x7ff968070b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:03.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.545+0000 7ff96ee83640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff96815b760 con 0x7ff9580039e0 2024-04-24T23:09:03.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.545+0000 7ff96ee83640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff9680b50a0 con 0x7ff9680ae8b0 2024-04-24T23:09:03.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.545+0000 7ff96ee83640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff9680b5cd0 con 0x7ff968070680 2024-04-24T23:09:03.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.545+0000 7ff96cc1f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9580039e0 0x7ff968146d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:03.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.545+0000 7ff96cc1f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9580039e0 0x7ff968146d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40218/0 (socket says 172.21.15.32:40218) 2024-04-24T23:09:03.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.545+0000 7ff96cc1f640 1 -- 172.21.15.32:0/2337344291 learned_addr learned my addr 172.21.15.32:0/2337344291 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:09:03.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.546+0000 7ff96d420640 1 --2- 172.21.15.32:0/2337344291 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff968070680 0x7ff968070b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:03.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.546+0000 7ff967fff640 1 --2- 172.21.15.32:0/2337344291 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9680ae8b0 0x7ff9680aed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:03.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.546+0000 7ff96cc1f640 1 -- 172.21.15.32:0/2337344291 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff968070680 msgr2=0x7ff968070b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:03.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.546+0000 7ff96cc1f640 1 --2- 172.21.15.32:0/2337344291 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff968070680 0x7ff968070b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:03.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.546+0000 7ff96cc1f640 1 -- 172.21.15.32:0/2337344291 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9680ae8b0 msgr2=0x7ff9680aed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:03.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.546+0000 7ff96cc1f640 1 --2- 172.21.15.32:0/2337344291 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9680ae8b0 0x7ff9680aed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:03.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.546+0000 7ff96cc1f640 1 -- 172.21.15.32:0/2337344291 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff968071040 con 0x7ff9580039e0 2024-04-24T23:09:03.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.546+0000 7ff96cc1f640 1 --2- 172.21.15.32:0/2337344291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9580039e0 0x7ff968146d80 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7ff95c02a8d0 tx=0x7ff95c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:03.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.546+0000 7ff9657fa640 1 -- 172.21.15.32:0/2337344291 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c043030 con 0x7ff9580039e0 2024-04-24T23:09:03.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.546+0000 7ff9657fa640 1 -- 172.21.15.32:0/2337344291 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff95c02abe0 con 0x7ff9580039e0 2024-04-24T23:09:03.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.546+0000 7ff96ee83640 1 -- 172.21.15.32:0/2337344291 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9681472c0 con 0x7ff9580039e0 2024-04-24T23:09:03.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.546+0000 7ff9657fa640 1 -- 172.21.15.32:0/2337344291 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c00b600 con 0x7ff9580039e0 2024-04-24T23:09:03.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.546+0000 7ff96ee83640 1 -- 172.21.15.32:0/2337344291 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff968071590 con 0x7ff9580039e0 2024-04-24T23:09:03.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.547+0000 7ff96ee83640 1 -- 172.21.15.32:0/2337344291 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff968068430 con 0x7ff9580039e0 2024-04-24T23:09:03.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.548+0000 7ff9657fa640 1 -- 172.21.15.32:0/2337344291 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff95c046070 con 0x7ff9580039e0 2024-04-24T23:09:03.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.548+0000 7ff9657fa640 1 --2- 172.21.15.32:0/2337344291 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff94c06be30 0x7ff94c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:03.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.548+0000 7ff9657fa640 1 -- 172.21.15.32:0/2337344291 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 5709+0+0 (secure 0 0 0) 0x7ff95c0ab860 con 0x7ff9580039e0 2024-04-24T23:09:03.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.548+0000 7ff967fff640 1 --2- 172.21.15.32:0/2337344291 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff94c06be30 0x7ff94c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:03.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.549+0000 7ff967fff640 1 --2- 172.21.15.32:0/2337344291 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff94c06be30 0x7ff94c06e2f0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7ff954001cc0 tx=0x7ff954009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:03.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.552+0000 7ff9657fa640 1 -- 172.21.15.32:0/2337344291 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff9681472c0 con 0x7ff9580039e0 2024-04-24T23:09:03.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:03.773+0000 7ff96ee83640 1 -- 172.21.15.32:0/2337344291 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7ff96806c000 con 0x7ff9580039e0 2024-04-24T23:09:04.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.426+0000 7ff9657fa640 1 -- 172.21.15.32:0/2337344291 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v73) ==== 122+0+0 (secure 0 0 0) 0x7ff95c0ab020 con 0x7ff9580039e0 2024-04-24T23:09:04.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.429+0000 7ff96ee83640 1 -- 172.21.15.32:0/2337344291 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff94c06be30 msgr2=0x7ff94c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:04.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.429+0000 7ff96ee83640 1 --2- 172.21.15.32:0/2337344291 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff94c06be30 0x7ff94c06e2f0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7ff954001cc0 tx=0x7ff954009040 comp rx=0 tx=0).stop 2024-04-24T23:09:04.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.429+0000 7ff96ee83640 1 -- 172.21.15.32:0/2337344291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9580039e0 msgr2=0x7ff968146d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:04.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.429+0000 7ff96ee83640 1 --2- 172.21.15.32:0/2337344291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9580039e0 0x7ff968146d80 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7ff95c02a8d0 tx=0x7ff95c0046c0 comp rx=0 tx=0).stop 2024-04-24T23:09:04.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.429+0000 7ff96ee83640 1 -- 172.21.15.32:0/2337344291 shutdown_connections 2024-04-24T23:09:04.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.429+0000 7ff96ee83640 1 --2- 172.21.15.32:0/2337344291 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff94c06be30 0x7ff94c06e2f0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:04.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.429+0000 7ff96ee83640 1 --2- 172.21.15.32:0/2337344291 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff968070680 0x7ff968070b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:04.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.429+0000 7ff96ee83640 1 --2- 172.21.15.32:0/2337344291 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9680ae8b0 0x7ff9680aed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:04.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.429+0000 7ff96ee83640 1 --2- 172.21.15.32:0/2337344291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9580039e0 0x7ff968146d80 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:04.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.429+0000 7ff96ee83640 1 -- 172.21.15.32:0/2337344291 >> 172.21.15.32:0/2337344291 conn(0x7ff96806b110 msgr2=0x7ff9680cd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:04.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.429+0000 7ff96ee83640 1 -- 172.21.15.32:0/2337344291 shutdown_connections 2024-04-24T23:09:04.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.429+0000 7ff96ee83640 1 -- 172.21.15.32:0/2337344291 wait complete. 2024-04-24T23:09:04.445 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-24T23:09:04.445 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:09:04.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.550+0000 7f578efe9640 1 Processor -- start 2024-04-24T23:09:04.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.551+0000 7f578efe9640 1 -- start start 2024-04-24T23:09:04.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.551+0000 7f578efe9640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f578806cd00 0x7f57880ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:04.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.551+0000 7f578efe9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f578815cb00 0x7f57880ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:04.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.551+0000 7f578efe9640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57880b4520 con 0x7f578815b380 2024-04-24T23:09:04.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.551+0000 7f578efe9640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f57880b4800 con 0x7f578815cb00 2024-04-24T23:09:04.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.551+0000 7f578efe9640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f57880b4ae0 con 0x7f578806cd00 2024-04-24T23:09:04.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.552+0000 7f578d586640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f578815cb00 0x7f57880ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:04.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.552+0000 7f578cd85640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f578815b380 0x7f578806c5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40616/0 (socket says 172.21.15.32:40616) 2024-04-24T23:09:04.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.552+0000 7f578d586640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f578815cb00 0x7f57880ce970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40220/0 (socket says 172.21.15.32:40220) 2024-04-24T23:09:04.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.552+0000 7f578cd85640 1 -- 172.21.15.32:0/3873363449 learned_addr learned my addr 172.21.15.32:0/3873363449 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:09:04.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.552+0000 7f578d586640 1 -- 172.21.15.32:0/3873363449 >> v2:172.21.15.121:3300/0 conn(0x7f578806cd00 msgr2=0x7f57880ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:09:04.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.552+0000 7f5787fff640 1 --2- 172.21.15.32:0/3873363449 >> v2:172.21.15.121:3300/0 conn(0x7f578806cd00 0x7f57880ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:04.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.552+0000 7f578d586640 1 --2- 172.21.15.32:0/3873363449 >> v2:172.21.15.121:3300/0 conn(0x7f578806cd00 0x7f57880ce430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:04.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.552+0000 7f57877fe640 1 -- 172.21.15.32:0/3873363449 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2399005988 0 0) 0x7f57880b4520 con 0x7f578815b380 2024-04-24T23:09:04.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.552+0000 7f578d586640 1 -- 172.21.15.32:0/3873363449 >> v1:172.21.15.32:6789/0 conn(0x7f578815b380 legacy=0x7f578806c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:04.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.552+0000 7f578d586640 1 -- 172.21.15.32:0/3873363449 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57880b4dc0 con 0x7f578815cb00 2024-04-24T23:09:04.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.553+0000 7f578d586640 1 --2- 172.21.15.32:0/3873363449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f578815cb00 0x7f57880ce970 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f57880b33c0 tx=0x7f578002c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=105e9e01a634c360 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:04.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.553+0000 7f57877fe640 1 -- 172.21.15.32:0/3873363449 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5780009540 con 0x7f578815cb00 2024-04-24T23:09:04.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.553+0000 7f57877fe640 1 -- 172.21.15.32:0/3873363449 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57800096e0 con 0x7f578815cb00 2024-04-24T23:09:04.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.553+0000 7f57877fe640 1 -- 172.21.15.32:0/3873363449 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5780009ab0 con 0x7f578815cb00 2024-04-24T23:09:04.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.553+0000 7f578efe9640 1 -- 172.21.15.32:0/3873363449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f578815cb00 msgr2=0x7f57880ce970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:04.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.553+0000 7f578efe9640 1 --2- 172.21.15.32:0/3873363449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f578815cb00 0x7f57880ce970 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f57880b33c0 tx=0x7f578002c7f0 comp rx=0 tx=0).stop 2024-04-24T23:09:04.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.553+0000 7f578efe9640 1 -- 172.21.15.32:0/3873363449 shutdown_connections 2024-04-24T23:09:04.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.553+0000 7f578efe9640 1 --2- 172.21.15.32:0/3873363449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f578815cb00 0x7f57880ce970 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:04.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.553+0000 7f578efe9640 1 --2- 172.21.15.32:0/3873363449 >> v2:172.21.15.121:3300/0 conn(0x7f578806cd00 0x7f57880ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:04.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.553+0000 7f578efe9640 1 -- 172.21.15.32:0/3873363449 >> 172.21.15.32:0/3873363449 conn(0x7f578806b110 msgr2=0x7f578806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:04.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.553+0000 7f578efe9640 1 -- 172.21.15.32:0/3873363449 shutdown_connections 2024-04-24T23:09:04.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.553+0000 7f578efe9640 1 -- 172.21.15.32:0/3873363449 wait complete. 2024-04-24T23:09:04.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.555+0000 7f578efe9640 1 Processor -- start 2024-04-24T23:09:04.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.555+0000 7f578efe9640 1 -- start start 2024-04-24T23:09:04.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.555+0000 7f578efe9640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f578806cd00 0x7f57880717e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:04.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.555+0000 7f578efe9640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f578815b380 0x7f5788147d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:04.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.555+0000 7f578efe9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f578815cb00 0x7f5788159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:04.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.555+0000 7f578efe9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f578815b760 con 0x7f578815cb00 2024-04-24T23:09:04.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.555+0000 7f578efe9640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f57880ceeb0 con 0x7f578815b380 2024-04-24T23:09:04.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.555+0000 7f578efe9640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f57880b55c0 con 0x7f578806cd00 2024-04-24T23:09:04.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.556+0000 7f578d586640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f578815cb00 0x7f5788159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:04.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.556+0000 7f578d586640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f578815cb00 0x7f5788159cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40222/0 (socket says 172.21.15.32:40222) 2024-04-24T23:09:04.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.556+0000 7f578d586640 1 -- 172.21.15.32:0/2896739639 learned_addr learned my addr 172.21.15.32:0/2896739639 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:09:04.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.556+0000 7f5787fff640 1 --2- 172.21.15.32:0/2896739639 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f578815b380 0x7f5788147d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:04.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.556+0000 7f578cd85640 1 --2- 172.21.15.32:0/2896739639 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f578806cd00 0x7f57880717e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:04.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.556+0000 7f578d586640 1 -- 172.21.15.32:0/2896739639 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f578806cd00 msgr2=0x7f57880717e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:04.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.556+0000 7f578d586640 1 --2- 172.21.15.32:0/2896739639 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f578806cd00 0x7f57880717e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:04.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.556+0000 7f578d586640 1 -- 172.21.15.32:0/2896739639 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f578815b380 msgr2=0x7f5788147d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:04.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.556+0000 7f578d586640 1 --2- 172.21.15.32:0/2896739639 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f578815b380 0x7f5788147d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:04.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.556+0000 7f578d586640 1 -- 172.21.15.32:0/2896739639 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f578815a4e0 con 0x7f578815cb00 2024-04-24T23:09:04.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.556+0000 7f578d586640 1 --2- 172.21.15.32:0/2896739639 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f578815cb00 0x7f5788159cb0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f578000d140 tx=0x7f5780004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:04.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.556+0000 7f57857fa640 1 -- 172.21.15.32:0/2896739639 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5780043030 con 0x7f578815cb00 2024-04-24T23:09:04.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.556+0000 7f57857fa640 1 -- 172.21.15.32:0/2896739639 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5780035640 con 0x7f578815cb00 2024-04-24T23:09:04.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.556+0000 7f578efe9640 1 -- 172.21.15.32:0/2896739639 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f578815a7c0 con 0x7f578815cb00 2024-04-24T23:09:04.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.556+0000 7f57857fa640 1 -- 172.21.15.32:0/2896739639 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5780035a10 con 0x7f578815cb00 2024-04-24T23:09:04.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.556+0000 7f578efe9640 1 -- 172.21.15.32:0/2896739639 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5788152ad0 con 0x7f578815cb00 2024-04-24T23:09:04.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.558+0000 7f578efe9640 1 -- 172.21.15.32:0/2896739639 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5788068430 con 0x7f578815cb00 2024-04-24T23:09:04.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.558+0000 7f57857fa640 1 -- 172.21.15.32:0/2896739639 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5780046070 con 0x7f578815cb00 2024-04-24T23:09:04.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.558+0000 7f57857fa640 1 --2- 172.21.15.32:0/2896739639 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f576c06be30 0x7f576c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:04.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.558+0000 7f57857fa640 1 -- 172.21.15.32:0/2896739639 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 5709+0+0 (secure 0 0 0) 0x7f57800abf00 con 0x7f578815cb00 2024-04-24T23:09:04.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.559+0000 7f578cd85640 1 --2- 172.21.15.32:0/2896739639 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f576c06be30 0x7f576c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:04.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.559+0000 7f578cd85640 1 --2- 172.21.15.32:0/2896739639 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f576c06be30 0x7f576c06e2f0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f577c000f60 tx=0x7f577c007d50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:04.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.562+0000 7f57857fa640 1 -- 172.21.15.32:0/2896739639 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f578004d070 con 0x7f578815cb00 2024-04-24T23:09:04.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.786+0000 7f578efe9640 1 -- 172.21.15.32:0/2896739639 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f578806c000 con 0x7f578815cb00 2024-04-24T23:09:04.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.787+0000 7f57857fa640 1 -- 172.21.15.32:0/2896739639 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+5185 (secure 0 0 0) 0x7f578007a380 con 0x7f578815cb00 2024-04-24T23:09:04.788 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:09:04.788 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":73,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6695,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/91086102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":91086102},{"type":"v1","addr":"172.21.15.121:6836","nonce":91086102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":6698,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1443743127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1443743127},{"type":"v1","addr":"172.21.15.121:6837","nonce":1443743127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":7423,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/4008773444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":4008773444},{"type":"v1","addr":"172.21.15.32:6835","nonce":4008773444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:09:00.770308+0000","modified":"2024-04-24T23:09:03.774877+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8679},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8679":{"gid":8679,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6834/4273183179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":4273183179},{"type":"v1","addr":"172.21.15.32:6834","nonce":4273183179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8679,"qdb_cluster":[8679]},"id":10}]} 2024-04-24T23:09:04.789 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 73 2024-04-24T23:09:04.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.789+0000 7f578efe9640 1 -- 172.21.15.32:0/2896739639 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f576c06be30 msgr2=0x7f576c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:04.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.789+0000 7f578efe9640 1 --2- 172.21.15.32:0/2896739639 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f576c06be30 0x7f576c06e2f0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f577c000f60 tx=0x7f577c007d50 comp rx=0 tx=0).stop 2024-04-24T23:09:04.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.789+0000 7f578efe9640 1 -- 172.21.15.32:0/2896739639 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f578815cb00 msgr2=0x7f5788159cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:04.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.789+0000 7f578efe9640 1 --2- 172.21.15.32:0/2896739639 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f578815cb00 0x7f5788159cb0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f578000d140 tx=0x7f5780004a40 comp rx=0 tx=0).stop 2024-04-24T23:09:04.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.789+0000 7f578efe9640 1 -- 172.21.15.32:0/2896739639 shutdown_connections 2024-04-24T23:09:04.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.789+0000 7f578efe9640 1 --2- 172.21.15.32:0/2896739639 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f576c06be30 0x7f576c06e2f0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:04.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.789+0000 7f578efe9640 1 --2- 172.21.15.32:0/2896739639 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f578815cb00 0x7f5788159cb0 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:04.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.789+0000 7f578efe9640 1 --2- 172.21.15.32:0/2896739639 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f578815b380 0x7f5788147d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:04.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.789+0000 7f578efe9640 1 --2- 172.21.15.32:0/2896739639 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f578806cd00 0x7f57880717e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:04.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.789+0000 7f578efe9640 1 -- 172.21.15.32:0/2896739639 >> 172.21.15.32:0/2896739639 conn(0x7f578806b110 msgr2=0x7f578815cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:04.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.789+0000 7f578efe9640 1 -- 172.21.15.32:0/2896739639 shutdown_connections 2024-04-24T23:09:04.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.789+0000 7f578efe9640 1 -- 172.21.15.32:0/2896739639 wait complete. 2024-04-24T23:09:04.801 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:09:04.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.901+0000 7f0b2a609640 1 Processor -- start 2024-04-24T23:09:04.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.901+0000 7f0b2a609640 1 -- start start 2024-04-24T23:09:04.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.901+0000 7f0b2a609640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0b2415e9b0 0x7f0b240d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:04.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.901+0000 7f0b2a609640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0b240d4210 0x7f0b2406b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:04.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.901+0000 7f0b2a609640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b240d3170 con 0x7f0b240ba0e0 2024-04-24T23:09:04.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.901+0000 7f0b2a609640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f0b240b8900 con 0x7f0b2415e9b0 2024-04-24T23:09:04.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.901+0000 7f0b2a609640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f0b240d3450 con 0x7f0b240d4210 2024-04-24T23:09:04.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.902+0000 7f0b23fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0b2415e9b0 0x7f0b240d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:04.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.902+0000 7f0b23fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0b2415e9b0 0x7f0b240d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40232/0 (socket says 172.21.15.32:40232) 2024-04-24T23:09:04.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.902+0000 7f0b23fff640 1 -- 172.21.15.32:0/3356119929 learned_addr learned my addr 172.21.15.32:0/3356119929 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:09:04.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.902+0000 7f0b237fe640 1 --2- 172.21.15.32:0/3356119929 >> v2:172.21.15.121:3300/0 conn(0x7f0b240d4210 0x7f0b2406b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:04.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.902+0000 7f0b23fff640 1 -- 172.21.15.32:0/3356119929 >> v2:172.21.15.121:3300/0 conn(0x7f0b240d4210 msgr2=0x7f0b2406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:09:04.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.902+0000 7f0b23fff640 1 --2- 172.21.15.32:0/3356119929 >> v2:172.21.15.121:3300/0 conn(0x7f0b240d4210 0x7f0b2406b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:04.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.902+0000 7f0b23fff640 1 -- 172.21.15.32:0/3356119929 >> v1:172.21.15.121:6789/0 conn(0x7f0b240ba0e0 legacy=0x7f0b24071730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:09:04.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.902+0000 7f0b23fff640 1 -- 172.21.15.32:0/3356119929 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b240d3730 con 0x7f0b2415e9b0 2024-04-24T23:09:04.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.903+0000 7f0b23fff640 1 --2- 172.21.15.32:0/3356119929 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b2415e9b0 0x7f0b240d2b90 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f0b240730f0 tx=0x7f0b1802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88f814fc0a755ce2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:04.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.903+0000 7f0b22ffd640 1 -- 172.21.15.32:0/3356119929 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b18037070 con 0x7f0b2415e9b0 2024-04-24T23:09:04.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.903+0000 7f0b22ffd640 1 -- 172.21.15.32:0/3356119929 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b18032440 con 0x7f0b2415e9b0 2024-04-24T23:09:04.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.903+0000 7f0b22ffd640 1 -- 172.21.15.32:0/3356119929 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b18032810 con 0x7f0b2415e9b0 2024-04-24T23:09:04.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.903+0000 7f0b2a609640 1 -- 172.21.15.32:0/3356119929 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b2415e9b0 msgr2=0x7f0b240d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:04.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.903+0000 7f0b2a609640 1 --2- 172.21.15.32:0/3356119929 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b2415e9b0 0x7f0b240d2b90 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f0b240730f0 tx=0x7f0b1802a1a0 comp rx=0 tx=0).stop 2024-04-24T23:09:04.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.903+0000 7f0b2a609640 1 -- 172.21.15.32:0/3356119929 shutdown_connections 2024-04-24T23:09:04.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.903+0000 7f0b2a609640 1 --2- 172.21.15.32:0/3356119929 >> v2:172.21.15.121:3300/0 conn(0x7f0b240d4210 0x7f0b2406b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:04.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.903+0000 7f0b2a609640 1 --2- 172.21.15.32:0/3356119929 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b2415e9b0 0x7f0b240d2b90 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:04.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.903+0000 7f0b2a609640 1 -- 172.21.15.32:0/3356119929 >> 172.21.15.32:0/3356119929 conn(0x7f0b240d11d0 msgr2=0x7f0b240d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:04.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.903+0000 7f0b2a609640 1 -- 172.21.15.32:0/3356119929 shutdown_connections 2024-04-24T23:09:04.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.903+0000 7f0b2a609640 1 -- 172.21.15.32:0/3356119929 wait complete. 2024-04-24T23:09:04.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.904+0000 7f0b2a609640 1 Processor -- start 2024-04-24T23:09:04.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.905+0000 7f0b2a609640 1 -- start start 2024-04-24T23:09:04.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.905+0000 7f0b2a609640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0b240ba0e0 0x7f0b24144d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:04.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.905+0000 7f0b2a609640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b240d4210 0x7f0b241422e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:04.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.905+0000 7f0b2a609640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b2415e9b0 0x7f0b241619e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:04.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.905+0000 7f0b2a609640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0b240d2400 con 0x7f0b240d4210 2024-04-24T23:09:04.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.905+0000 7f0b2a609640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0b240d3cf0 con 0x7f0b2415e9b0 2024-04-24T23:09:04.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.905+0000 7f0b2a609640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f0b2406b650 con 0x7f0b240ba0e0 2024-04-24T23:09:04.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.905+0000 7f0b237fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b240d4210 0x7f0b241422e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:04.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.906+0000 7f0b237fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b240d4210 0x7f0b241422e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40236/0 (socket says 172.21.15.32:40236) 2024-04-24T23:09:04.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.906+0000 7f0b237fe640 1 -- 172.21.15.32:0/982653196 learned_addr learned my addr 172.21.15.32:0/982653196 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:09:04.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.906+0000 7f0b28ba6640 1 --2- 172.21.15.32:0/982653196 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b2415e9b0 0x7f0b241619e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:04.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.906+0000 7f0b23fff640 1 --2- 172.21.15.32:0/982653196 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0b240ba0e0 0x7f0b24144d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:04.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.906+0000 7f0b237fe640 1 -- 172.21.15.32:0/982653196 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0b240ba0e0 msgr2=0x7f0b24144d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:04.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.906+0000 7f0b237fe640 1 --2- 172.21.15.32:0/982653196 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0b240ba0e0 0x7f0b24144d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:04.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.906+0000 7f0b237fe640 1 -- 172.21.15.32:0/982653196 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b2415e9b0 msgr2=0x7f0b241619e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:04.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.906+0000 7f0b237fe640 1 --2- 172.21.15.32:0/982653196 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b2415e9b0 0x7f0b241619e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:04.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.906+0000 7f0b237fe640 1 -- 172.21.15.32:0/982653196 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b24152b80 con 0x7f0b240d4210 2024-04-24T23:09:04.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.906+0000 7f0b237fe640 1 --2- 172.21.15.32:0/982653196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b240d4210 0x7f0b241422e0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f0b1400d800 tx=0x7f0b1400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:04.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.906+0000 7f0b20ff9640 1 -- 172.21.15.32:0/982653196 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1400b660 con 0x7f0b240d4210 2024-04-24T23:09:04.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.906+0000 7f0b20ff9640 1 -- 172.21.15.32:0/982653196 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b14019070 con 0x7f0b240d4210 2024-04-24T23:09:04.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.906+0000 7f0b2a609640 1 -- 172.21.15.32:0/982653196 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b241452c0 con 0x7f0b240d4210 2024-04-24T23:09:04.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.906+0000 7f0b20ff9640 1 -- 172.21.15.32:0/982653196 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1400ba20 con 0x7f0b240d4210 2024-04-24T23:09:04.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.906+0000 7f0b2a609640 1 -- 172.21.15.32:0/982653196 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b24152e00 con 0x7f0b240d4210 2024-04-24T23:09:04.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.907+0000 7f0b2a609640 1 -- 172.21.15.32:0/982653196 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b24068430 con 0x7f0b240d4210 2024-04-24T23:09:04.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.908+0000 7f0b20ff9640 1 -- 172.21.15.32:0/982653196 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0b1400bbc0 con 0x7f0b240d4210 2024-04-24T23:09:04.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.908+0000 7f0b20ff9640 1 --2- 172.21.15.32:0/982653196 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0b1006be60 0x7f0b1006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:04.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.908+0000 7f0b20ff9640 1 -- 172.21.15.32:0/982653196 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 5709+0+0 (secure 0 0 0) 0x7f0b1408b2a0 con 0x7f0b240d4210 2024-04-24T23:09:04.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.908+0000 7f0b23fff640 1 --2- 172.21.15.32:0/982653196 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0b1006be60 0x7f0b1006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:04.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.909+0000 7f0b23fff640 1 --2- 172.21.15.32:0/982653196 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0b1006be60 0x7f0b1006e320 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f0b18034000 tx=0x7f0b1802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:04.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:04.912+0000 7f0b20ff9640 1 -- 172.21.15.32:0/982653196 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0b241452c0 con 0x7f0b240d4210 2024-04-24T23:09:05.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.096+0000 7f0b2a609640 1 -- 172.21.15.32:0/982653196 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0b240cb0b0 con 0x7f0b240d4210 2024-04-24T23:09:05.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.097+0000 7f0b20ff9640 1 -- 172.21.15.32:0/982653196 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v169) ==== 74+0+16692 (secure 0 0 0) 0x7f0b24152b80 con 0x7f0b240d4210 2024-04-24T23:09:05.099 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:09:05.099 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":169,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:09:02.410894+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:08:59.076360+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-24T23:08:59.745667+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:09:05.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.099+0000 7f0b2a609640 1 -- 172.21.15.32:0/982653196 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0b1006be60 msgr2=0x7f0b1006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:05.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.099+0000 7f0b2a609640 1 --2- 172.21.15.32:0/982653196 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0b1006be60 0x7f0b1006e320 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f0b18034000 tx=0x7f0b1802e000 comp rx=0 tx=0).stop 2024-04-24T23:09:05.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.099+0000 7f0b2a609640 1 -- 172.21.15.32:0/982653196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b240d4210 msgr2=0x7f0b241422e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:05.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.099+0000 7f0b2a609640 1 --2- 172.21.15.32:0/982653196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b240d4210 0x7f0b241422e0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f0b1400d800 tx=0x7f0b1400dcc0 comp rx=0 tx=0).stop 2024-04-24T23:09:05.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.099+0000 7f0b2a609640 1 -- 172.21.15.32:0/982653196 shutdown_connections 2024-04-24T23:09:05.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.099+0000 7f0b2a609640 1 --2- 172.21.15.32:0/982653196 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0b1006be60 0x7f0b1006e320 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:05.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.099+0000 7f0b2a609640 1 --2- 172.21.15.32:0/982653196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b240d4210 0x7f0b241422e0 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:05.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.099+0000 7f0b2a609640 1 --2- 172.21.15.32:0/982653196 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b2415e9b0 0x7f0b241619e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:05.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.099+0000 7f0b2a609640 1 --2- 172.21.15.32:0/982653196 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0b240ba0e0 0x7f0b24144d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:05.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.099+0000 7f0b2a609640 1 -- 172.21.15.32:0/982653196 >> 172.21.15.32:0/982653196 conn(0x7f0b240d11d0 msgr2=0x7f0b240cf7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:05.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.099+0000 7f0b2a609640 1 -- 172.21.15.32:0/982653196 shutdown_connections 2024-04-24T23:09:05.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.099+0000 7f0b2a609640 1 -- 172.21.15.32:0/982653196 wait complete. 2024-04-24T23:09:05.112 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-24T23:09:05.112 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:09:05.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.211+0000 7f652c8fc640 1 Processor -- start 2024-04-24T23:09:05.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.212+0000 7f652c8fc640 1 -- start start 2024-04-24T23:09:05.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.212+0000 7f652c8fc640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f652806cd00 0x7f6528166440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:05.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.212+0000 7f652c8fc640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65280cef90 con 0x7f652815b480 2024-04-24T23:09:05.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.213+0000 7f652c8fc640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65280b46c0 con 0x7f652815fd60 2024-04-24T23:09:05.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.213+0000 7f652c8fc640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f65280b4ff0 con 0x7f652806cd00 2024-04-24T23:09:05.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.213+0000 7f652659c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f652815b480 0x7f65280b5c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40626/0 (socket says 172.21.15.32:40626) 2024-04-24T23:09:05.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.213+0000 7f652659c640 1 -- 172.21.15.32:0/4056252982 learned_addr learned my addr 172.21.15.32:0/4056252982 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:09:05.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.213+0000 7f6526d9d640 1 --2- 172.21.15.32:0/4056252982 >> v2:172.21.15.121:3300/0 conn(0x7f652806cd00 0x7f6528166440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:05.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.213+0000 7f652559a640 1 -- 172.21.15.32:0/4056252982 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2330809533 0 0) 0x7f65280cef90 con 0x7f652815b480 2024-04-24T23:09:05.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.213+0000 7f652559a640 1 -- 172.21.15.32:0/4056252982 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6510003520 con 0x7f652815b480 2024-04-24T23:09:05.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.213+0000 7f652559a640 1 -- 172.21.15.32:0/4056252982 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3095371258 0 0) 0x7f6510003520 con 0x7f652815b480 2024-04-24T23:09:05.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.213+0000 7f652559a640 1 -- 172.21.15.32:0/4056252982 >> v2:172.21.15.121:3300/0 conn(0x7f652806cd00 msgr2=0x7f6528166440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:09:05.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.213+0000 7f652559a640 1 --2- 172.21.15.32:0/4056252982 >> v2:172.21.15.121:3300/0 conn(0x7f652806cd00 0x7f6528166440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:05.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.213+0000 7f652559a640 1 -- 172.21.15.32:0/4056252982 >> v1:172.21.15.121:6789/0 conn(0x7f652815fd60 legacy=0x7f652806c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:05.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.213+0000 7f652559a640 1 -- 172.21.15.32:0/4056252982 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65280b49a0 con 0x7f652815b480 2024-04-24T23:09:05.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.213+0000 7f652559a640 1 -- 172.21.15.32:0/4056252982 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f6518001670 con 0x7f652815b480 2024-04-24T23:09:05.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.213+0000 7f652559a640 1 -- 172.21.15.32:0/4056252982 >> v1:172.21.15.32:6789/0 conn(0x7f652815b480 legacy=0x7f65280b5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:05.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.213+0000 7f652559a640 1 --2- 172.21.15.32:0/4056252982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65100039b0 0x7f6510003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:05.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.213+0000 7f652559a640 1 --2- 172.21.15.32:0/4056252982 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6510004800 0x7f6510004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:05.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.213+0000 7f652559a640 1 --2- 172.21.15.32:0/4056252982 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65100051c0 0x7f651000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:05.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.213+0000 7f652559a640 1 -- 172.21.15.32:0/4056252982 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f65280b4ff0 con 0x7f65100039b0 2024-04-24T23:09:05.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.213+0000 7f652559a640 1 -- 172.21.15.32:0/4056252982 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f651000c060 con 0x7f65100051c0 2024-04-24T23:09:05.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.213+0000 7f652559a640 1 -- 172.21.15.32:0/4056252982 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f651000c260 con 0x7f6510004800 2024-04-24T23:09:05.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.213+0000 7f652659c640 1 --2- 172.21.15.32:0/4056252982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65100039b0 0x7f6510003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:05.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.214+0000 7f6526d9d640 1 --2- 172.21.15.32:0/4056252982 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65100051c0 0x7f651000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:05.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.214+0000 7f6525d9b640 1 --2- 172.21.15.32:0/4056252982 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6510004800 0x7f6510004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:05.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.214+0000 7f652659c640 1 -- 172.21.15.32:0/4056252982 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6510004800 msgr2=0x7f6510004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:09:05.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.214+0000 7f652659c640 1 --2- 172.21.15.32:0/4056252982 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6510004800 0x7f6510004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:05.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.214+0000 7f652659c640 1 -- 172.21.15.32:0/4056252982 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65100051c0 msgr2=0x7f651000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:09:05.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.214+0000 7f652659c640 1 --2- 172.21.15.32:0/4056252982 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65100051c0 0x7f651000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:05.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.214+0000 7f652659c640 1 -- 172.21.15.32:0/4056252982 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6510003520 con 0x7f65100039b0 2024-04-24T23:09:05.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.214+0000 7f6526d9d640 1 -- 172.21.15.32:0/4056252982 reap_dead start 2024-04-24T23:09:05.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.214+0000 7f652659c640 1 --2- 172.21.15.32:0/4056252982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65100039b0 0x7f6510003dd0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f651800c240 tx=0x7f6510005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f4390e03cac3b1f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:05.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.215+0000 7f652559a640 1 -- 172.21.15.32:0/4056252982 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6518007cf0 con 0x7f65100039b0 2024-04-24T23:09:05.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.215+0000 7f652559a640 1 -- 172.21.15.32:0/4056252982 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6518005af0 con 0x7f65100039b0 2024-04-24T23:09:05.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.215+0000 7f652559a640 1 -- 172.21.15.32:0/4056252982 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6518007070 con 0x7f65100039b0 2024-04-24T23:09:05.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.215+0000 7f652c8fc640 1 -- 172.21.15.32:0/4056252982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65100039b0 msgr2=0x7f6510003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:05.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.215+0000 7f652c8fc640 1 --2- 172.21.15.32:0/4056252982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65100039b0 0x7f6510003dd0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f651800c240 tx=0x7f6510005830 comp rx=0 tx=0).stop 2024-04-24T23:09:05.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.215+0000 7f652c8fc640 1 -- 172.21.15.32:0/4056252982 shutdown_connections 2024-04-24T23:09:05.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.215+0000 7f652c8fc640 1 --2- 172.21.15.32:0/4056252982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65100039b0 0x7f6510003dd0 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:05.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.215+0000 7f652c8fc640 1 -- 172.21.15.32:0/4056252982 >> 172.21.15.32:0/4056252982 conn(0x7f652806b110 msgr2=0x7f652806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:05.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.215+0000 7f652c8fc640 1 -- 172.21.15.32:0/4056252982 shutdown_connections 2024-04-24T23:09:05.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.215+0000 7f652c8fc640 1 -- 172.21.15.32:0/4056252982 wait complete. 2024-04-24T23:09:05.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.217+0000 7f652c8fc640 1 Processor -- start 2024-04-24T23:09:05.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.217+0000 7f652c8fc640 1 -- start start 2024-04-24T23:09:05.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.217+0000 7f652c8fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65100039b0 0x7f65280cd510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:05.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.217+0000 7f652c8fc640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f652814e710 0x7f652814aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:05.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.217+0000 7f652c8fc640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6528071d80 0x7f6528072230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:05.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.217+0000 7f652c8fc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f652815b860 con 0x7f65100039b0 2024-04-24T23:09:05.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.217+0000 7f652c8fc640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f65280b5480 con 0x7f6528071d80 2024-04-24T23:09:05.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.217+0000 7f652c8fc640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f652814b020 con 0x7f652814e710 2024-04-24T23:09:05.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.217+0000 7f652659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65100039b0 0x7f65280cd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:05.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.217+0000 7f652659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65100039b0 0x7f65280cd510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40254/0 (socket says 172.21.15.32:40254) 2024-04-24T23:09:05.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.217+0000 7f652659c640 1 -- 172.21.15.32:0/2891056366 learned_addr learned my addr 172.21.15.32:0/2891056366 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:09:05.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.218+0000 7f6525d9b640 1 --2- 172.21.15.32:0/2891056366 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f652814e710 0x7f652814aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:05.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.218+0000 7f6526d9d640 1 --2- 172.21.15.32:0/2891056366 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6528071d80 0x7f6528072230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:05.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.218+0000 7f652659c640 1 -- 172.21.15.32:0/2891056366 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f652814e710 msgr2=0x7f652814aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:05.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.218+0000 7f652659c640 1 --2- 172.21.15.32:0/2891056366 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f652814e710 0x7f652814aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:05.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.218+0000 7f652659c640 1 -- 172.21.15.32:0/2891056366 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6528071d80 msgr2=0x7f6528072230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:05.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.218+0000 7f652659c640 1 --2- 172.21.15.32:0/2891056366 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6528071d80 0x7f6528072230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:05.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.218+0000 7f652659c640 1 -- 172.21.15.32:0/2891056366 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65280cdaf0 con 0x7f65100039b0 2024-04-24T23:09:05.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.218+0000 7f652659c640 1 --2- 172.21.15.32:0/2891056366 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65100039b0 0x7f65280cd510 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f651800c330 tx=0x7f65280b4b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:05.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.218+0000 7f6516ffd640 1 -- 172.21.15.32:0/2891056366 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6518001c30 con 0x7f65100039b0 2024-04-24T23:09:05.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.218+0000 7f6516ffd640 1 -- 172.21.15.32:0/2891056366 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6518042070 con 0x7f65100039b0 2024-04-24T23:09:05.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.218+0000 7f6516ffd640 1 -- 172.21.15.32:0/2891056366 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6518007070 con 0x7f65100039b0 2024-04-24T23:09:05.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.218+0000 7f652c8fc640 1 -- 172.21.15.32:0/2891056366 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65281f1ee0 con 0x7f65100039b0 2024-04-24T23:09:05.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.218+0000 7f652c8fc640 1 -- 172.21.15.32:0/2891056366 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65281f24a0 con 0x7f65100039b0 2024-04-24T23:09:05.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.219+0000 7f652c8fc640 1 -- 172.21.15.32:0/2891056366 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f652814eaf0 con 0x7f65100039b0 2024-04-24T23:09:05.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.220+0000 7f6516ffd640 1 -- 172.21.15.32:0/2891056366 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6518007210 con 0x7f65100039b0 2024-04-24T23:09:05.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.220+0000 7f6516ffd640 1 --2- 172.21.15.32:0/2891056366 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f650806be60 0x7f650806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:05.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.220+0000 7f6516ffd640 1 -- 172.21.15.32:0/2891056366 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 5709+0+0 (secure 0 0 0) 0x7f65180aac90 con 0x7f65100039b0 2024-04-24T23:09:05.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.220+0000 7f6525d9b640 1 --2- 172.21.15.32:0/2891056366 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f650806be60 0x7f650806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:05.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.221+0000 7f6525d9b640 1 --2- 172.21.15.32:0/2891056366 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f650806be60 0x7f650806e320 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f65281524e0 tx=0x7f651c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:05.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.224+0000 7f6516ffd640 1 -- 172.21.15.32:0/2891056366 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6518078ff0 con 0x7f65100039b0 2024-04-24T23:09:05.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.402+0000 7f652c8fc640 1 -- 172.21.15.32:0/2891056366 --> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6528068430 con 0x7f650806be60 2024-04-24T23:09:05.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.409+0000 7f6516ffd640 1 -- 172.21.15.32:0/2891056366 <== mgr.4108 v2:172.21.15.121:6832/1848157943 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247926 (secure 0 0 0) 0x7f6528068430 con 0x7f650806be60 2024-04-24T23:09:05.413 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:09:05.415 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-24T23:09:05.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.412+0000 7f652c8fc640 1 -- 172.21.15.32:0/2891056366 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f650806be60 msgr2=0x7f650806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:05.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.412+0000 7f652c8fc640 1 --2- 172.21.15.32:0/2891056366 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f650806be60 0x7f650806e320 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f65281524e0 tx=0x7f651c009040 comp rx=0 tx=0).stop 2024-04-24T23:09:05.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.412+0000 7f652c8fc640 1 -- 172.21.15.32:0/2891056366 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65100039b0 msgr2=0x7f65280cd510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:05.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.412+0000 7f652c8fc640 1 --2- 172.21.15.32:0/2891056366 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65100039b0 0x7f65280cd510 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f651800c330 tx=0x7f65280b4b70 comp rx=0 tx=0).stop 2024-04-24T23:09:05.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.412+0000 7f652c8fc640 1 -- 172.21.15.32:0/2891056366 shutdown_connections 2024-04-24T23:09:05.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.412+0000 7f652c8fc640 1 --2- 172.21.15.32:0/2891056366 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f650806be60 0x7f650806e320 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:05.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.412+0000 7f652c8fc640 1 --2- 172.21.15.32:0/2891056366 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6528071d80 0x7f6528072230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:05.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.412+0000 7f652c8fc640 1 --2- 172.21.15.32:0/2891056366 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f652814e710 0x7f652814aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:05.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.412+0000 7f652c8fc640 1 --2- 172.21.15.32:0/2891056366 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65100039b0 0x7f65280cd510 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:05.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.412+0000 7f652c8fc640 1 -- 172.21.15.32:0/2891056366 >> 172.21.15.32:0/2891056366 conn(0x7f652806b110 msgr2=0x7f652815edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:05.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.412+0000 7f652c8fc640 1 -- 172.21.15.32:0/2891056366 shutdown_connections 2024-04-24T23:09:05.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.412+0000 7f652c8fc640 1 -- 172.21.15.32:0/2891056366 wait complete. 2024-04-24T23:09:05.425 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":467,"stamp":"2024-04-24T23:09:05.270206+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":112,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":8514,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1553985,"statfs":{"total":1600000000,"available":1591280947,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.876501"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.424485+0000","last_change":"2024-04-24T23:09:01.424376+0000","last_active":"2024-04-24T23:09:01.424485+0000","last_peered":"2024-04-24T23:09:01.424485+0000","last_clean":"2024-04-24T23:09:01.424485+0000","last_became_active":"2024-04-24T23:09:01.423583+0000","last_became_peered":"2024-04-24T23:09:01.423583+0000","last_unstale":"2024-04-24T23:09:01.424485+0000","last_undegraded":"2024-04-24T23:09:01.424485+0000","last_fullsized":"2024-04-24T23:09:01.424485+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:47:13.405533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.427226+0000","last_change":"2024-04-24T23:09:01.427034+0000","last_active":"2024-04-24T23:09:01.427226+0000","last_peered":"2024-04-24T23:09:01.427226+0000","last_clean":"2024-04-24T23:09:01.427226+0000","last_became_active":"2024-04-24T23:09:01.426768+0000","last_became_peered":"2024-04-24T23:09:01.426768+0000","last_unstale":"2024-04-24T23:09:01.427226+0000","last_undegraded":"2024-04-24T23:09:01.427226+0000","last_fullsized":"2024-04-24T23:09:01.427226+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:04.492603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.418136+0000","last_change":"2024-04-24T23:09:01.417701+0000","last_active":"2024-04-24T23:09:01.418136+0000","last_peered":"2024-04-24T23:09:01.418136+0000","last_clean":"2024-04-24T23:09:01.418136+0000","last_became_active":"2024-04-24T23:09:01.417023+0000","last_became_peered":"2024-04-24T23:09:01.417023+0000","last_unstale":"2024-04-24T23:09:01.418136+0000","last_undegraded":"2024-04-24T23:09:01.418136+0000","last_fullsized":"2024-04-24T23:09:01.418136+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:49:51.435440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.418558+0000","last_change":"2024-04-24T23:09:01.418187+0000","last_active":"2024-04-24T23:09:01.418558+0000","last_peered":"2024-04-24T23:09:01.418558+0000","last_clean":"2024-04-24T23:09:01.418558+0000","last_became_active":"2024-04-24T23:09:01.417852+0000","last_became_peered":"2024-04-24T23:09:01.417852+0000","last_unstale":"2024-04-24T23:09:01.418558+0000","last_undegraded":"2024-04-24T23:09:01.418558+0000","last_fullsized":"2024-04-24T23:09:01.418558+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:49:06.192899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.424014+0000","last_change":"2024-04-24T23:09:01.423879+0000","last_active":"2024-04-24T23:09:01.424014+0000","last_peered":"2024-04-24T23:09:01.424014+0000","last_clean":"2024-04-24T23:09:01.424014+0000","last_became_active":"2024-04-24T23:09:01.423554+0000","last_became_peered":"2024-04-24T23:09:01.423554+0000","last_unstale":"2024-04-24T23:09:01.424014+0000","last_undegraded":"2024-04-24T23:09:01.424014+0000","last_fullsized":"2024-04-24T23:09:01.424014+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:59.289070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.424293+0000","last_change":"2024-04-24T23:09:01.424162+0000","last_active":"2024-04-24T23:09:01.424293+0000","last_peered":"2024-04-24T23:09:01.424293+0000","last_clean":"2024-04-24T23:09:01.424293+0000","last_became_active":"2024-04-24T23:09:01.423581+0000","last_became_peered":"2024-04-24T23:09:01.423581+0000","last_unstale":"2024-04-24T23:09:01.424293+0000","last_undegraded":"2024-04-24T23:09:01.424293+0000","last_fullsized":"2024-04-24T23:09:01.424293+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:20:04.895778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.418682+0000","last_change":"2024-04-24T23:09:01.418567+0000","last_active":"2024-04-24T23:09:01.418682+0000","last_peered":"2024-04-24T23:09:01.418682+0000","last_clean":"2024-04-24T23:09:01.418682+0000","last_became_active":"2024-04-24T23:09:01.418263+0000","last_became_peered":"2024-04-24T23:09:01.418263+0000","last_unstale":"2024-04-24T23:09:01.418682+0000","last_undegraded":"2024-04-24T23:09:01.418682+0000","last_fullsized":"2024-04-24T23:09:01.418682+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:37.532986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.428738+0000","last_change":"2024-04-24T23:09:01.428631+0000","last_active":"2024-04-24T23:09:01.428738+0000","last_peered":"2024-04-24T23:09:01.428738+0000","last_clean":"2024-04-24T23:09:01.428738+0000","last_became_active":"2024-04-24T23:09:01.428378+0000","last_became_peered":"2024-04-24T23:09:01.428378+0000","last_unstale":"2024-04-24T23:09:01.428738+0000","last_undegraded":"2024-04-24T23:09:01.428738+0000","last_fullsized":"2024-04-24T23:09:01.428738+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:24.669943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.422743+0000","last_change":"2024-04-24T23:09:01.422401+0000","last_active":"2024-04-24T23:09:01.422743+0000","last_peered":"2024-04-24T23:09:01.422743+0000","last_clean":"2024-04-24T23:09:01.422743+0000","last_became_active":"2024-04-24T23:09:01.422096+0000","last_became_peered":"2024-04-24T23:09:01.422096+0000","last_unstale":"2024-04-24T23:09:01.422743+0000","last_undegraded":"2024-04-24T23:09:01.422743+0000","last_fullsized":"2024-04-24T23:09:01.422743+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:33:27.756672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.421571+0000","last_change":"2024-04-24T23:09:01.421429+0000","last_active":"2024-04-24T23:09:01.421571+0000","last_peered":"2024-04-24T23:09:01.421571+0000","last_clean":"2024-04-24T23:09:01.421571+0000","last_became_active":"2024-04-24T23:09:01.421202+0000","last_became_peered":"2024-04-24T23:09:01.421202+0000","last_unstale":"2024-04-24T23:09:01.421571+0000","last_undegraded":"2024-04-24T23:09:01.421571+0000","last_fullsized":"2024-04-24T23:09:01.421571+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:18.877710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.424533+0000","last_change":"2024-04-24T23:09:01.424376+0000","last_active":"2024-04-24T23:09:01.424533+0000","last_peered":"2024-04-24T23:09:01.424533+0000","last_clean":"2024-04-24T23:09:01.424533+0000","last_became_active":"2024-04-24T23:09:01.423934+0000","last_became_peered":"2024-04-24T23:09:01.423934+0000","last_unstale":"2024-04-24T23:09:01.424533+0000","last_undegraded":"2024-04-24T23:09:01.424533+0000","last_fullsized":"2024-04-24T23:09:01.424533+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:11.242842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.430893+0000","last_change":"2024-04-24T23:09:01.430677+0000","last_active":"2024-04-24T23:09:01.430893+0000","last_peered":"2024-04-24T23:09:01.430893+0000","last_clean":"2024-04-24T23:09:01.430893+0000","last_became_active":"2024-04-24T23:09:01.430257+0000","last_became_peered":"2024-04-24T23:09:01.430257+0000","last_unstale":"2024-04-24T23:09:01.430893+0000","last_undegraded":"2024-04-24T23:09:01.430893+0000","last_fullsized":"2024-04-24T23:09:01.430893+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:26.256633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.425215+0000","last_change":"2024-04-24T23:09:01.425094+0000","last_active":"2024-04-24T23:09:01.425215+0000","last_peered":"2024-04-24T23:09:01.425215+0000","last_clean":"2024-04-24T23:09:01.425215+0000","last_became_active":"2024-04-24T23:09:01.424860+0000","last_became_peered":"2024-04-24T23:09:01.424860+0000","last_unstale":"2024-04-24T23:09:01.425215+0000","last_undegraded":"2024-04-24T23:09:01.425215+0000","last_fullsized":"2024-04-24T23:09:01.425215+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:43:45.589467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.428824+0000","last_change":"2024-04-24T23:09:01.428710+0000","last_active":"2024-04-24T23:09:01.428824+0000","last_peered":"2024-04-24T23:09:01.428824+0000","last_clean":"2024-04-24T23:09:01.428824+0000","last_became_active":"2024-04-24T23:09:01.427986+0000","last_became_peered":"2024-04-24T23:09:01.427986+0000","last_unstale":"2024-04-24T23:09:01.428824+0000","last_undegraded":"2024-04-24T23:09:01.428824+0000","last_fullsized":"2024-04-24T23:09:01.428824+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:52:28.859209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.428087+0000","last_change":"2024-04-24T23:09:01.427719+0000","last_active":"2024-04-24T23:09:01.428087+0000","last_peered":"2024-04-24T23:09:01.428087+0000","last_clean":"2024-04-24T23:09:01.428087+0000","last_became_active":"2024-04-24T23:09:01.427489+0000","last_became_peered":"2024-04-24T23:09:01.427489+0000","last_unstale":"2024-04-24T23:09:01.428087+0000","last_undegraded":"2024-04-24T23:09:01.428087+0000","last_fullsized":"2024-04-24T23:09:01.428087+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:42:11.174669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.423056+0000","last_change":"2024-04-24T23:09:01.422929+0000","last_active":"2024-04-24T23:09:01.423056+0000","last_peered":"2024-04-24T23:09:01.423056+0000","last_clean":"2024-04-24T23:09:01.423056+0000","last_became_active":"2024-04-24T23:09:01.422705+0000","last_became_peered":"2024-04-24T23:09:01.422705+0000","last_unstale":"2024-04-24T23:09:01.423056+0000","last_undegraded":"2024-04-24T23:09:01.423056+0000","last_fullsized":"2024-04-24T23:09:01.423056+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:53:56.337235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.418156+0000","last_change":"2024-04-24T23:09:01.417978+0000","last_active":"2024-04-24T23:09:01.418156+0000","last_peered":"2024-04-24T23:09:01.418156+0000","last_clean":"2024-04-24T23:09:01.418156+0000","last_became_active":"2024-04-24T23:09:01.417718+0000","last_became_peered":"2024-04-24T23:09:01.417718+0000","last_unstale":"2024-04-24T23:09:01.418156+0000","last_undegraded":"2024-04-24T23:09:01.418156+0000","last_fullsized":"2024-04-24T23:09:01.418156+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:05.376272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.424489+0000","last_change":"2024-04-24T23:09:01.424377+0000","last_active":"2024-04-24T23:09:01.424489+0000","last_peered":"2024-04-24T23:09:01.424489+0000","last_clean":"2024-04-24T23:09:01.424489+0000","last_became_active":"2024-04-24T23:09:01.424138+0000","last_became_peered":"2024-04-24T23:09:01.424138+0000","last_unstale":"2024-04-24T23:09:01.424489+0000","last_undegraded":"2024-04-24T23:09:01.424489+0000","last_fullsized":"2024-04-24T23:09:01.424489+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:33:35.741652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.425452+0000","last_change":"2024-04-24T23:09:01.425339+0000","last_active":"2024-04-24T23:09:01.425452+0000","last_peered":"2024-04-24T23:09:01.425452+0000","last_clean":"2024-04-24T23:09:01.425452+0000","last_became_active":"2024-04-24T23:09:01.425085+0000","last_became_peered":"2024-04-24T23:09:01.425085+0000","last_unstale":"2024-04-24T23:09:01.425452+0000","last_undegraded":"2024-04-24T23:09:01.425452+0000","last_fullsized":"2024-04-24T23:09:01.425452+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:42:33.414609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.424084+0000","last_change":"2024-04-24T23:09:01.423830+0000","last_active":"2024-04-24T23:09:01.424084+0000","last_peered":"2024-04-24T23:09:01.424084+0000","last_clean":"2024-04-24T23:09:01.424084+0000","last_became_active":"2024-04-24T23:09:01.423577+0000","last_became_peered":"2024-04-24T23:09:01.423577+0000","last_unstale":"2024-04-24T23:09:01.424084+0000","last_undegraded":"2024-04-24T23:09:01.424084+0000","last_fullsized":"2024-04-24T23:09:01.424084+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:55.093440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.418112+0000","last_change":"2024-04-24T23:09:01.417570+0000","last_active":"2024-04-24T23:09:01.418112+0000","last_peered":"2024-04-24T23:09:01.418112+0000","last_clean":"2024-04-24T23:09:01.418112+0000","last_became_active":"2024-04-24T23:09:01.417313+0000","last_became_peered":"2024-04-24T23:09:01.417313+0000","last_unstale":"2024-04-24T23:09:01.418112+0000","last_undegraded":"2024-04-24T23:09:01.418112+0000","last_fullsized":"2024-04-24T23:09:01.418112+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:12.513190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.430873+0000","last_change":"2024-04-24T23:09:01.430483+0000","last_active":"2024-04-24T23:09:01.430873+0000","last_peered":"2024-04-24T23:09:01.430873+0000","last_clean":"2024-04-24T23:09:01.430873+0000","last_became_active":"2024-04-24T23:09:01.429720+0000","last_became_peered":"2024-04-24T23:09:01.429720+0000","last_unstale":"2024-04-24T23:09:01.430873+0000","last_undegraded":"2024-04-24T23:09:01.430873+0000","last_fullsized":"2024-04-24T23:09:01.430873+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:12:56.098330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.419192+0000","last_change":"2024-04-24T23:09:01.419061+0000","last_active":"2024-04-24T23:09:01.419192+0000","last_peered":"2024-04-24T23:09:01.419192+0000","last_clean":"2024-04-24T23:09:01.419192+0000","last_became_active":"2024-04-24T23:09:01.418812+0000","last_became_peered":"2024-04-24T23:09:01.418812+0000","last_unstale":"2024-04-24T23:09:01.419192+0000","last_undegraded":"2024-04-24T23:09:01.419192+0000","last_fullsized":"2024-04-24T23:09:01.419192+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:42:04.346496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.430723+0000","last_change":"2024-04-24T23:09:01.430595+0000","last_active":"2024-04-24T23:09:01.430723+0000","last_peered":"2024-04-24T23:09:01.430723+0000","last_clean":"2024-04-24T23:09:01.430723+0000","last_became_active":"2024-04-24T23:09:01.430363+0000","last_became_peered":"2024-04-24T23:09:01.430363+0000","last_unstale":"2024-04-24T23:09:01.430723+0000","last_undegraded":"2024-04-24T23:09:01.430723+0000","last_fullsized":"2024-04-24T23:09:01.430723+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:49:04.371757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.422425+0000","last_change":"2024-04-24T23:09:01.422312+0000","last_active":"2024-04-24T23:09:01.422425+0000","last_peered":"2024-04-24T23:09:01.422425+0000","last_clean":"2024-04-24T23:09:01.422425+0000","last_became_active":"2024-04-24T23:09:01.422089+0000","last_became_peered":"2024-04-24T23:09:01.422089+0000","last_unstale":"2024-04-24T23:09:01.422425+0000","last_undegraded":"2024-04-24T23:09:01.422425+0000","last_fullsized":"2024-04-24T23:09:01.422425+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:32:39.034275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.424032+0000","last_change":"2024-04-24T23:09:01.423315+0000","last_active":"2024-04-24T23:09:01.424032+0000","last_peered":"2024-04-24T23:09:01.424032+0000","last_clean":"2024-04-24T23:09:01.424032+0000","last_became_active":"2024-04-24T23:09:01.423073+0000","last_became_peered":"2024-04-24T23:09:01.423073+0000","last_unstale":"2024-04-24T23:09:01.424032+0000","last_undegraded":"2024-04-24T23:09:01.424032+0000","last_fullsized":"2024-04-24T23:09:01.424032+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:05:04.724031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.423608+0000","last_change":"2024-04-24T23:09:01.422911+0000","last_active":"2024-04-24T23:09:01.423608+0000","last_peered":"2024-04-24T23:09:01.423608+0000","last_clean":"2024-04-24T23:09:01.423608+0000","last_became_active":"2024-04-24T23:09:01.422610+0000","last_became_peered":"2024-04-24T23:09:01.422610+0000","last_unstale":"2024-04-24T23:09:01.423608+0000","last_undegraded":"2024-04-24T23:09:01.423608+0000","last_fullsized":"2024-04-24T23:09:01.423608+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:25.802602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.426596+0000","last_change":"2024-04-24T23:09:01.426447+0000","last_active":"2024-04-24T23:09:01.426596+0000","last_peered":"2024-04-24T23:09:01.426596+0000","last_clean":"2024-04-24T23:09:01.426596+0000","last_became_active":"2024-04-24T23:09:01.426213+0000","last_became_peered":"2024-04-24T23:09:01.426213+0000","last_unstale":"2024-04-24T23:09:01.426596+0000","last_undegraded":"2024-04-24T23:09:01.426596+0000","last_fullsized":"2024-04-24T23:09:01.426596+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:20:22.766564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.430058+0000","last_change":"2024-04-24T23:09:01.429889+0000","last_active":"2024-04-24T23:09:01.430058+0000","last_peered":"2024-04-24T23:09:01.430058+0000","last_clean":"2024-04-24T23:09:01.430058+0000","last_became_active":"2024-04-24T23:09:01.429664+0000","last_became_peered":"2024-04-24T23:09:01.429664+0000","last_unstale":"2024-04-24T23:09:01.430058+0000","last_undegraded":"2024-04-24T23:09:01.430058+0000","last_fullsized":"2024-04-24T23:09:01.430058+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:09:48.797884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.429584+0000","last_change":"2024-04-24T23:09:01.429438+0000","last_active":"2024-04-24T23:09:01.429584+0000","last_peered":"2024-04-24T23:09:01.429584+0000","last_clean":"2024-04-24T23:09:01.429584+0000","last_became_active":"2024-04-24T23:09:01.428969+0000","last_became_peered":"2024-04-24T23:09:01.428969+0000","last_unstale":"2024-04-24T23:09:01.429584+0000","last_undegraded":"2024-04-24T23:09:01.429584+0000","last_fullsized":"2024-04-24T23:09:01.429584+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:47.404287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.426223+0000","last_change":"2024-04-24T23:09:01.425483+0000","last_active":"2024-04-24T23:09:01.426223+0000","last_peered":"2024-04-24T23:09:01.426223+0000","last_clean":"2024-04-24T23:09:01.426223+0000","last_became_active":"2024-04-24T23:09:01.425206+0000","last_became_peered":"2024-04-24T23:09:01.425206+0000","last_unstale":"2024-04-24T23:09:01.426223+0000","last_undegraded":"2024-04-24T23:09:01.426223+0000","last_fullsized":"2024-04-24T23:09:01.426223+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:11:44.741451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.430735+0000","last_change":"2024-04-24T23:09:01.430604+0000","last_active":"2024-04-24T23:09:01.430735+0000","last_peered":"2024-04-24T23:09:01.430735+0000","last_clean":"2024-04-24T23:09:01.430735+0000","last_became_active":"2024-04-24T23:09:01.430371+0000","last_became_peered":"2024-04-24T23:09:01.430371+0000","last_unstale":"2024-04-24T23:09:01.430735+0000","last_undegraded":"2024-04-24T23:09:01.430735+0000","last_fullsized":"2024-04-24T23:09:01.430735+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:53:09.208422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.424508+0000","last_change":"2024-04-24T23:09:01.424409+0000","last_active":"2024-04-24T23:09:01.424508+0000","last_peered":"2024-04-24T23:09:01.424508+0000","last_clean":"2024-04-24T23:09:01.424508+0000","last_became_active":"2024-04-24T23:09:01.424183+0000","last_became_peered":"2024-04-24T23:09:01.424183+0000","last_unstale":"2024-04-24T23:09:01.424508+0000","last_undegraded":"2024-04-24T23:09:01.424508+0000","last_fullsized":"2024-04-24T23:09:01.424508+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:20.037915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"168'4","reported_seq":16,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.438609+0000","last_change":"2024-04-24T23:09:00.402015+0000","last_active":"2024-04-24T23:09:01.438609+0000","last_peered":"2024-04-24T23:09:01.438609+0000","last_clean":"2024-04-24T23:09:01.438609+0000","last_became_active":"2024-04-24T23:09:00.401741+0000","last_became_peered":"2024-04-24T23:09:00.401741+0000","last_unstale":"2024-04-24T23:09:01.438609+0000","last_undegraded":"2024-04-24T23:09:01.438609+0000","last_fullsized":"2024-04-24T23:09:01.438609+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:08:18.751223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.424145+0000","last_change":"2024-04-24T23:09:01.424016+0000","last_active":"2024-04-24T23:09:01.424145+0000","last_peered":"2024-04-24T23:09:01.424145+0000","last_clean":"2024-04-24T23:09:01.424145+0000","last_became_active":"2024-04-24T23:09:01.423774+0000","last_became_peered":"2024-04-24T23:09:01.423774+0000","last_unstale":"2024-04-24T23:09:01.424145+0000","last_undegraded":"2024-04-24T23:09:01.424145+0000","last_fullsized":"2024-04-24T23:09:01.424145+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:47:43.101190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.1e","version":"168'1","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.436864+0000","last_change":"2024-04-24T23:09:00.402853+0000","last_active":"2024-04-24T23:09:01.436864+0000","last_peered":"2024-04-24T23:09:01.436864+0000","last_clean":"2024-04-24T23:09:01.436864+0000","last_became_active":"2024-04-24T23:09:00.402244+0000","last_became_peered":"2024-04-24T23:09:00.402244+0000","last_unstale":"2024-04-24T23:09:01.436864+0000","last_undegraded":"2024-04-24T23:09:01.436864+0000","last_fullsized":"2024-04-24T23:09:01.436864+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:16:04.868002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.426678+0000","last_change":"2024-04-24T23:09:01.426562+0000","last_active":"2024-04-24T23:09:01.426678+0000","last_peered":"2024-04-24T23:09:01.426678+0000","last_clean":"2024-04-24T23:09:01.426678+0000","last_became_active":"2024-04-24T23:09:01.425730+0000","last_became_peered":"2024-04-24T23:09:01.425730+0000","last_unstale":"2024-04-24T23:09:01.426678+0000","last_undegraded":"2024-04-24T23:09:01.426678+0000","last_fullsized":"2024-04-24T23:09:01.426678+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:49:26.379338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.1d","version":"168'2","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.438129+0000","last_change":"2024-04-24T23:09:00.407212+0000","last_active":"2024-04-24T23:09:01.438129+0000","last_peered":"2024-04-24T23:09:01.438129+0000","last_clean":"2024-04-24T23:09:01.438129+0000","last_became_active":"2024-04-24T23:09:00.406687+0000","last_became_peered":"2024-04-24T23:09:00.406687+0000","last_unstale":"2024-04-24T23:09:01.438129+0000","last_undegraded":"2024-04-24T23:09:01.438129+0000","last_fullsized":"2024-04-24T23:09:01.438129+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:49:21.836047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.418827+0000","last_change":"2024-04-24T23:09:01.418687+0000","last_active":"2024-04-24T23:09:01.418827+0000","last_peered":"2024-04-24T23:09:01.418827+0000","last_clean":"2024-04-24T23:09:01.418827+0000","last_became_active":"2024-04-24T23:09:01.418440+0000","last_became_peered":"2024-04-24T23:09:01.418440+0000","last_unstale":"2024-04-24T23:09:01.418827+0000","last_undegraded":"2024-04-24T23:09:01.418827+0000","last_fullsized":"2024-04-24T23:09:01.418827+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:21:14.862296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1c","version":"168'2","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.437441+0000","last_change":"2024-04-24T23:09:00.403972+0000","last_active":"2024-04-24T23:09:01.437441+0000","last_peered":"2024-04-24T23:09:01.437441+0000","last_clean":"2024-04-24T23:09:01.437441+0000","last_became_active":"2024-04-24T23:09:00.403717+0000","last_became_peered":"2024-04-24T23:09:00.403717+0000","last_unstale":"2024-04-24T23:09:01.437441+0000","last_undegraded":"2024-04-24T23:09:01.437441+0000","last_fullsized":"2024-04-24T23:09:01.437441+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:02.508478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.428256+0000","last_change":"2024-04-24T23:09:01.428149+0000","last_active":"2024-04-24T23:09:01.428256+0000","last_peered":"2024-04-24T23:09:01.428256+0000","last_clean":"2024-04-24T23:09:01.428256+0000","last_became_active":"2024-04-24T23:09:01.427832+0000","last_became_peered":"2024-04-24T23:09:01.427832+0000","last_unstale":"2024-04-24T23:09:01.428256+0000","last_undegraded":"2024-04-24T23:09:01.428256+0000","last_fullsized":"2024-04-24T23:09:01.428256+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:35:30.340066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"168'1","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.436498+0000","last_change":"2024-04-24T23:09:00.409341+0000","last_active":"2024-04-24T23:09:01.436498+0000","last_peered":"2024-04-24T23:09:01.436498+0000","last_clean":"2024-04-24T23:09:01.436498+0000","last_became_active":"2024-04-24T23:09:00.409066+0000","last_became_peered":"2024-04-24T23:09:00.409066+0000","last_unstale":"2024-04-24T23:09:01.436498+0000","last_undegraded":"2024-04-24T23:09:01.436498+0000","last_fullsized":"2024-04-24T23:09:01.436498+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:21:23.449197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.420541+0000","last_change":"2024-04-24T23:09:01.420426+0000","last_active":"2024-04-24T23:09:01.420541+0000","last_peered":"2024-04-24T23:09:01.420541+0000","last_clean":"2024-04-24T23:09:01.420541+0000","last_became_active":"2024-04-24T23:09:01.420175+0000","last_became_peered":"2024-04-24T23:09:01.420175+0000","last_unstale":"2024-04-24T23:09:01.420541+0000","last_undegraded":"2024-04-24T23:09:01.420541+0000","last_fullsized":"2024-04-24T23:09:01.420541+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:30:47.293551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1a","version":"168'1","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.436871+0000","last_change":"2024-04-24T23:09:00.407115+0000","last_active":"2024-04-24T23:09:01.436871+0000","last_peered":"2024-04-24T23:09:01.436871+0000","last_clean":"2024-04-24T23:09:01.436871+0000","last_became_active":"2024-04-24T23:09:00.406871+0000","last_became_peered":"2024-04-24T23:09:00.406871+0000","last_unstale":"2024-04-24T23:09:01.436871+0000","last_undegraded":"2024-04-24T23:09:01.436871+0000","last_fullsized":"2024-04-24T23:09:01.436871+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:15.821843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.430032+0000","last_change":"2024-04-24T23:09:01.429910+0000","last_active":"2024-04-24T23:09:01.430032+0000","last_peered":"2024-04-24T23:09:01.430032+0000","last_clean":"2024-04-24T23:09:01.430032+0000","last_became_active":"2024-04-24T23:09:01.429669+0000","last_became_peered":"2024-04-24T23:09:01.429669+0000","last_unstale":"2024-04-24T23:09:01.430032+0000","last_undegraded":"2024-04-24T23:09:01.430032+0000","last_fullsized":"2024-04-24T23:09:01.430032+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:48:48.838559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.416647+0000","last_change":"2024-04-24T23:09:00.406851+0000","last_active":"2024-04-24T23:09:01.416647+0000","last_peered":"2024-04-24T23:09:01.416647+0000","last_clean":"2024-04-24T23:09:01.416647+0000","last_became_active":"2024-04-24T23:09:00.406602+0000","last_became_peered":"2024-04-24T23:09:00.406602+0000","last_unstale":"2024-04-24T23:09:01.416647+0000","last_undegraded":"2024-04-24T23:09:01.416647+0000","last_fullsized":"2024-04-24T23:09:01.416647+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:38.880892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.431000+0000","last_change":"2024-04-24T23:09:01.430861+0000","last_active":"2024-04-24T23:09:01.431000+0000","last_peered":"2024-04-24T23:09:01.431000+0000","last_clean":"2024-04-24T23:09:01.431000+0000","last_became_active":"2024-04-24T23:09:01.430112+0000","last_became_peered":"2024-04-24T23:09:01.430112+0000","last_unstale":"2024-04-24T23:09:01.431000+0000","last_undegraded":"2024-04-24T23:09:01.431000+0000","last_fullsized":"2024-04-24T23:09:01.431000+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:55:44.000350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.412594+0000","last_change":"2024-04-24T23:09:00.406935+0000","last_active":"2024-04-24T23:09:01.412594+0000","last_peered":"2024-04-24T23:09:01.412594+0000","last_clean":"2024-04-24T23:09:01.412594+0000","last_became_active":"2024-04-24T23:09:00.406356+0000","last_became_peered":"2024-04-24T23:09:00.406356+0000","last_unstale":"2024-04-24T23:09:01.412594+0000","last_undegraded":"2024-04-24T23:09:01.412594+0000","last_fullsized":"2024-04-24T23:09:01.412594+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:46:19.034727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.419658+0000","last_change":"2024-04-24T23:09:01.419495+0000","last_active":"2024-04-24T23:09:01.419658+0000","last_peered":"2024-04-24T23:09:01.419658+0000","last_clean":"2024-04-24T23:09:01.419658+0000","last_became_active":"2024-04-24T23:09:01.418998+0000","last_became_peered":"2024-04-24T23:09:01.418998+0000","last_unstale":"2024-04-24T23:09:01.419658+0000","last_undegraded":"2024-04-24T23:09:01.419658+0000","last_fullsized":"2024-04-24T23:09:01.419658+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:48:33.370702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.419143+0000","last_change":"2024-04-24T23:09:00.405813+0000","last_active":"2024-04-24T23:09:01.419143+0000","last_peered":"2024-04-24T23:09:01.419143+0000","last_clean":"2024-04-24T23:09:01.419143+0000","last_became_active":"2024-04-24T23:09:00.405573+0000","last_became_peered":"2024-04-24T23:09:00.405573+0000","last_unstale":"2024-04-24T23:09:01.419143+0000","last_undegraded":"2024-04-24T23:09:01.419143+0000","last_fullsized":"2024-04-24T23:09:01.419143+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:39:29.961818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.427724+0000","last_change":"2024-04-24T23:09:01.426812+0000","last_active":"2024-04-24T23:09:01.427724+0000","last_peered":"2024-04-24T23:09:01.427724+0000","last_clean":"2024-04-24T23:09:01.427724+0000","last_became_active":"2024-04-24T23:09:01.426576+0000","last_became_peered":"2024-04-24T23:09:01.426576+0000","last_unstale":"2024-04-24T23:09:01.427724+0000","last_undegraded":"2024-04-24T23:09:01.427724+0000","last_fullsized":"2024-04-24T23:09:01.427724+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:38:37.910180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.420741+0000","last_change":"2024-04-24T23:09:00.402438+0000","last_active":"2024-04-24T23:09:01.420741+0000","last_peered":"2024-04-24T23:09:01.420741+0000","last_clean":"2024-04-24T23:09:01.420741+0000","last_became_active":"2024-04-24T23:09:00.402193+0000","last_became_peered":"2024-04-24T23:09:00.402193+0000","last_unstale":"2024-04-24T23:09:01.420741+0000","last_undegraded":"2024-04-24T23:09:01.420741+0000","last_fullsized":"2024-04-24T23:09:01.420741+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:50:51.640234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.423037+0000","last_change":"2024-04-24T23:09:01.422295+0000","last_active":"2024-04-24T23:09:01.423037+0000","last_peered":"2024-04-24T23:09:01.423037+0000","last_clean":"2024-04-24T23:09:01.423037+0000","last_became_active":"2024-04-24T23:09:01.422069+0000","last_became_peered":"2024-04-24T23:09:01.422069+0000","last_unstale":"2024-04-24T23:09:01.423037+0000","last_undegraded":"2024-04-24T23:09:01.423037+0000","last_fullsized":"2024-04-24T23:09:01.423037+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:19:55.306564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.14","version":"168'6","reported_seq":18,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.438229+0000","last_change":"2024-04-24T23:09:00.404740+0000","last_active":"2024-04-24T23:09:01.438229+0000","last_peered":"2024-04-24T23:09:01.438229+0000","last_clean":"2024-04-24T23:09:01.438229+0000","last_became_active":"2024-04-24T23:09:00.404493+0000","last_became_peered":"2024-04-24T23:09:00.404493+0000","last_unstale":"2024-04-24T23:09:01.438229+0000","last_undegraded":"2024-04-24T23:09:01.438229+0000","last_fullsized":"2024-04-24T23:09:01.438229+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:41.386975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.429299+0000","last_change":"2024-04-24T23:09:01.429172+0000","last_active":"2024-04-24T23:09:01.429299+0000","last_peered":"2024-04-24T23:09:01.429299+0000","last_clean":"2024-04-24T23:09:01.429299+0000","last_became_active":"2024-04-24T23:09:01.428935+0000","last_became_peered":"2024-04-24T23:09:01.428935+0000","last_unstale":"2024-04-24T23:09:01.429299+0000","last_undegraded":"2024-04-24T23:09:01.429299+0000","last_fullsized":"2024-04-24T23:09:01.429299+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:38:07.310274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.13","version":"168'2","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.438486+0000","last_change":"2024-04-24T23:09:00.406010+0000","last_active":"2024-04-24T23:09:01.438486+0000","last_peered":"2024-04-24T23:09:01.438486+0000","last_clean":"2024-04-24T23:09:01.438486+0000","last_became_active":"2024-04-24T23:09:00.405708+0000","last_became_peered":"2024-04-24T23:09:00.405708+0000","last_unstale":"2024-04-24T23:09:01.438486+0000","last_undegraded":"2024-04-24T23:09:01.438486+0000","last_fullsized":"2024-04-24T23:09:01.438486+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:25:25.564334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.431023+0000","last_change":"2024-04-24T23:09:01.430886+0000","last_active":"2024-04-24T23:09:01.431023+0000","last_peered":"2024-04-24T23:09:01.431023+0000","last_clean":"2024-04-24T23:09:01.431023+0000","last_became_active":"2024-04-24T23:09:01.430627+0000","last_became_peered":"2024-04-24T23:09:01.430627+0000","last_unstale":"2024-04-24T23:09:01.431023+0000","last_undegraded":"2024-04-24T23:09:01.431023+0000","last_fullsized":"2024-04-24T23:09:01.431023+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:26:23.052957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.424151+0000","last_change":"2024-04-24T23:09:00.400745+0000","last_active":"2024-04-24T23:09:01.424151+0000","last_peered":"2024-04-24T23:09:01.424151+0000","last_clean":"2024-04-24T23:09:01.424151+0000","last_became_active":"2024-04-24T23:09:00.400503+0000","last_became_peered":"2024-04-24T23:09:00.400503+0000","last_unstale":"2024-04-24T23:09:01.424151+0000","last_undegraded":"2024-04-24T23:09:01.424151+0000","last_fullsized":"2024-04-24T23:09:01.424151+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:30.563473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.421132+0000","last_change":"2024-04-24T23:09:01.420868+0000","last_active":"2024-04-24T23:09:01.421132+0000","last_peered":"2024-04-24T23:09:01.421132+0000","last_clean":"2024-04-24T23:09:01.421132+0000","last_became_active":"2024-04-24T23:09:01.420382+0000","last_became_peered":"2024-04-24T23:09:01.420382+0000","last_unstale":"2024-04-24T23:09:01.421132+0000","last_undegraded":"2024-04-24T23:09:01.421132+0000","last_fullsized":"2024-04-24T23:09:01.421132+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:38:18.189981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.421445+0000","last_change":"2024-04-24T23:09:00.406688+0000","last_active":"2024-04-24T23:09:01.421445+0000","last_peered":"2024-04-24T23:09:01.421445+0000","last_clean":"2024-04-24T23:09:01.421445+0000","last_became_active":"2024-04-24T23:09:00.405818+0000","last_became_peered":"2024-04-24T23:09:00.405818+0000","last_unstale":"2024-04-24T23:09:01.421445+0000","last_undegraded":"2024-04-24T23:09:01.421445+0000","last_fullsized":"2024-04-24T23:09:01.421445+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:46:10.288129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.419378+0000","last_change":"2024-04-24T23:09:01.419251+0000","last_active":"2024-04-24T23:09:01.419378+0000","last_peered":"2024-04-24T23:09:01.419378+0000","last_clean":"2024-04-24T23:09:01.419378+0000","last_became_active":"2024-04-24T23:09:01.418645+0000","last_became_peered":"2024-04-24T23:09:01.418645+0000","last_unstale":"2024-04-24T23:09:01.419378+0000","last_undegraded":"2024-04-24T23:09:01.419378+0000","last_fullsized":"2024-04-24T23:09:01.419378+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:15.065149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.10","version":"168'8","reported_seq":20,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.439093+0000","last_change":"2024-04-24T23:09:00.408729+0000","last_active":"2024-04-24T23:09:01.439093+0000","last_peered":"2024-04-24T23:09:01.439093+0000","last_clean":"2024-04-24T23:09:01.439093+0000","last_became_active":"2024-04-24T23:09:00.408474+0000","last_became_peered":"2024-04-24T23:09:00.408474+0000","last_unstale":"2024-04-24T23:09:01.439093+0000","last_undegraded":"2024-04-24T23:09:01.439093+0000","last_fullsized":"2024-04-24T23:09:01.439093+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:40:05.356173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.431412+0000","last_change":"2024-04-24T23:09:01.431281+0000","last_active":"2024-04-24T23:09:01.431412+0000","last_peered":"2024-04-24T23:09:01.431412+0000","last_clean":"2024-04-24T23:09:01.431412+0000","last_became_active":"2024-04-24T23:09:01.431042+0000","last_became_peered":"2024-04-24T23:09:01.431042+0000","last_unstale":"2024-04-24T23:09:01.431412+0000","last_undegraded":"2024-04-24T23:09:01.431412+0000","last_fullsized":"2024-04-24T23:09:01.431412+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:58.596772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.f","version":"168'1","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.434992+0000","last_change":"2024-04-24T23:09:00.409619+0000","last_active":"2024-04-24T23:09:01.434992+0000","last_peered":"2024-04-24T23:09:01.434992+0000","last_clean":"2024-04-24T23:09:01.434992+0000","last_became_active":"2024-04-24T23:09:00.409047+0000","last_became_peered":"2024-04-24T23:09:00.409047+0000","last_unstale":"2024-04-24T23:09:01.434992+0000","last_undegraded":"2024-04-24T23:09:01.434992+0000","last_fullsized":"2024-04-24T23:09:01.434992+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:14:10.744522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.424958+0000","last_change":"2024-04-24T23:09:01.424823+0000","last_active":"2024-04-24T23:09:01.424958+0000","last_peered":"2024-04-24T23:09:01.424958+0000","last_clean":"2024-04-24T23:09:01.424958+0000","last_became_active":"2024-04-24T23:09:01.424573+0000","last_became_peered":"2024-04-24T23:09:01.424573+0000","last_unstale":"2024-04-24T23:09:01.424958+0000","last_undegraded":"2024-04-24T23:09:01.424958+0000","last_fullsized":"2024-04-24T23:09:01.424958+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:27:32.667126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.e","version":"168'2","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.436523+0000","last_change":"2024-04-24T23:09:00.408456+0000","last_active":"2024-04-24T23:09:01.436523+0000","last_peered":"2024-04-24T23:09:01.436523+0000","last_clean":"2024-04-24T23:09:01.436523+0000","last_became_active":"2024-04-24T23:09:00.408200+0000","last_became_peered":"2024-04-24T23:09:00.408200+0000","last_unstale":"2024-04-24T23:09:01.436523+0000","last_undegraded":"2024-04-24T23:09:01.436523+0000","last_fullsized":"2024-04-24T23:09:01.436523+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:09:27.358487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.428063+0000","last_change":"2024-04-24T23:09:01.426543+0000","last_active":"2024-04-24T23:09:01.428063+0000","last_peered":"2024-04-24T23:09:01.428063+0000","last_clean":"2024-04-24T23:09:01.428063+0000","last_became_active":"2024-04-24T23:09:01.426315+0000","last_became_peered":"2024-04-24T23:09:01.426315+0000","last_unstale":"2024-04-24T23:09:01.428063+0000","last_undegraded":"2024-04-24T23:09:01.428063+0000","last_fullsized":"2024-04-24T23:09:01.428063+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:56.311762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.d","version":"168'1","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.433839+0000","last_change":"2024-04-24T23:09:00.403417+0000","last_active":"2024-04-24T23:09:01.433839+0000","last_peered":"2024-04-24T23:09:01.433839+0000","last_clean":"2024-04-24T23:09:01.433839+0000","last_became_active":"2024-04-24T23:09:00.403135+0000","last_became_peered":"2024-04-24T23:09:00.403135+0000","last_unstale":"2024-04-24T23:09:01.433839+0000","last_undegraded":"2024-04-24T23:09:01.433839+0000","last_fullsized":"2024-04-24T23:09:01.433839+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:28:35.033779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.426415+0000","last_change":"2024-04-24T23:09:01.426223+0000","last_active":"2024-04-24T23:09:01.426415+0000","last_peered":"2024-04-24T23:09:01.426415+0000","last_clean":"2024-04-24T23:09:01.426415+0000","last_became_active":"2024-04-24T23:09:01.425413+0000","last_became_peered":"2024-04-24T23:09:01.425413+0000","last_unstale":"2024-04-24T23:09:01.426415+0000","last_undegraded":"2024-04-24T23:09:01.426415+0000","last_fullsized":"2024-04-24T23:09:01.426415+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:50:39.561897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.421580+0000","last_change":"2024-04-24T23:09:00.406318+0000","last_active":"2024-04-24T23:09:01.421580+0000","last_peered":"2024-04-24T23:09:01.421580+0000","last_clean":"2024-04-24T23:09:01.421580+0000","last_became_active":"2024-04-24T23:09:00.405447+0000","last_became_peered":"2024-04-24T23:09:00.405447+0000","last_unstale":"2024-04-24T23:09:01.421580+0000","last_undegraded":"2024-04-24T23:09:01.421580+0000","last_fullsized":"2024-04-24T23:09:01.421580+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:08:41.531441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.429038+0000","last_change":"2024-04-24T23:09:01.428925+0000","last_active":"2024-04-24T23:09:01.429038+0000","last_peered":"2024-04-24T23:09:01.429038+0000","last_clean":"2024-04-24T23:09:01.429038+0000","last_became_active":"2024-04-24T23:09:01.428358+0000","last_became_peered":"2024-04-24T23:09:01.428358+0000","last_unstale":"2024-04-24T23:09:01.429038+0000","last_undegraded":"2024-04-24T23:09:01.429038+0000","last_fullsized":"2024-04-24T23:09:01.429038+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:34:07.860555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.b","version":"168'2","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.437970+0000","last_change":"2024-04-24T23:09:00.398550+0000","last_active":"2024-04-24T23:09:01.437970+0000","last_peered":"2024-04-24T23:09:01.437970+0000","last_clean":"2024-04-24T23:09:01.437970+0000","last_became_active":"2024-04-24T23:09:00.398273+0000","last_became_peered":"2024-04-24T23:09:00.398273+0000","last_unstale":"2024-04-24T23:09:01.437970+0000","last_undegraded":"2024-04-24T23:09:01.437970+0000","last_fullsized":"2024-04-24T23:09:01.437970+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:03.390742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.425359+0000","last_change":"2024-04-24T23:09:01.425031+0000","last_active":"2024-04-24T23:09:01.425359+0000","last_peered":"2024-04-24T23:09:01.425359+0000","last_clean":"2024-04-24T23:09:01.425359+0000","last_became_active":"2024-04-24T23:09:01.424613+0000","last_became_peered":"2024-04-24T23:09:01.424613+0000","last_unstale":"2024-04-24T23:09:01.425359+0000","last_undegraded":"2024-04-24T23:09:01.425359+0000","last_fullsized":"2024-04-24T23:09:01.425359+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:50:14.494486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.416701+0000","last_change":"2024-04-24T23:09:00.410496+0000","last_active":"2024-04-24T23:09:01.416701+0000","last_peered":"2024-04-24T23:09:01.416701+0000","last_clean":"2024-04-24T23:09:01.416701+0000","last_became_active":"2024-04-24T23:09:00.410322+0000","last_became_peered":"2024-04-24T23:09:00.410322+0000","last_unstale":"2024-04-24T23:09:01.416701+0000","last_undegraded":"2024-04-24T23:09:01.416701+0000","last_fullsized":"2024-04-24T23:09:01.416701+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:56:51.579136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.428633+0000","last_change":"2024-04-24T23:09:01.428508+0000","last_active":"2024-04-24T23:09:01.428633+0000","last_peered":"2024-04-24T23:09:01.428633+0000","last_clean":"2024-04-24T23:09:01.428633+0000","last_became_active":"2024-04-24T23:09:01.428109+0000","last_became_peered":"2024-04-24T23:09:01.428109+0000","last_unstale":"2024-04-24T23:09:01.428633+0000","last_undegraded":"2024-04-24T23:09:01.428633+0000","last_fullsized":"2024-04-24T23:09:01.428633+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:10.182210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.415871+0000","last_change":"2024-04-24T23:09:00.407554+0000","last_active":"2024-04-24T23:09:01.415871+0000","last_peered":"2024-04-24T23:09:01.415871+0000","last_clean":"2024-04-24T23:09:01.415871+0000","last_became_active":"2024-04-24T23:09:00.407275+0000","last_became_peered":"2024-04-24T23:09:00.407275+0000","last_unstale":"2024-04-24T23:09:01.415871+0000","last_undegraded":"2024-04-24T23:09:01.415871+0000","last_fullsized":"2024-04-24T23:09:01.415871+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:12:26.213125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.425324+0000","last_change":"2024-04-24T23:09:01.424915+0000","last_active":"2024-04-24T23:09:01.425324+0000","last_peered":"2024-04-24T23:09:01.425324+0000","last_clean":"2024-04-24T23:09:01.425324+0000","last_became_active":"2024-04-24T23:09:01.424582+0000","last_became_peered":"2024-04-24T23:09:01.424582+0000","last_unstale":"2024-04-24T23:09:01.425324+0000","last_undegraded":"2024-04-24T23:09:01.425324+0000","last_fullsized":"2024-04-24T23:09:01.425324+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:21:28.971678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.416637+0000","last_change":"2024-04-24T23:09:00.409795+0000","last_active":"2024-04-24T23:09:01.416637+0000","last_peered":"2024-04-24T23:09:01.416637+0000","last_clean":"2024-04-24T23:09:01.416637+0000","last_became_active":"2024-04-24T23:09:00.409528+0000","last_became_peered":"2024-04-24T23:09:00.409528+0000","last_unstale":"2024-04-24T23:09:01.416637+0000","last_undegraded":"2024-04-24T23:09:01.416637+0000","last_fullsized":"2024-04-24T23:09:01.416637+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:02:00.452829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.420895+0000","last_change":"2024-04-24T23:09:01.420665+0000","last_active":"2024-04-24T23:09:01.420895+0000","last_peered":"2024-04-24T23:09:01.420895+0000","last_clean":"2024-04-24T23:09:01.420895+0000","last_became_active":"2024-04-24T23:09:01.419499+0000","last_became_peered":"2024-04-24T23:09:01.419499+0000","last_unstale":"2024-04-24T23:09:01.420895+0000","last_undegraded":"2024-04-24T23:09:01.420895+0000","last_fullsized":"2024-04-24T23:09:01.420895+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:01:43.519392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.7","version":"168'2","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.436788+0000","last_change":"2024-04-24T23:09:00.409447+0000","last_active":"2024-04-24T23:09:01.436788+0000","last_peered":"2024-04-24T23:09:01.436788+0000","last_clean":"2024-04-24T23:09:01.436788+0000","last_became_active":"2024-04-24T23:09:00.409143+0000","last_became_peered":"2024-04-24T23:09:00.409143+0000","last_unstale":"2024-04-24T23:09:01.436788+0000","last_undegraded":"2024-04-24T23:09:01.436788+0000","last_fullsized":"2024-04-24T23:09:01.436788+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:13:16.773673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.418065+0000","last_change":"2024-04-24T23:09:01.417943+0000","last_active":"2024-04-24T23:09:01.418065+0000","last_peered":"2024-04-24T23:09:01.418065+0000","last_clean":"2024-04-24T23:09:01.418065+0000","last_became_active":"2024-04-24T23:09:01.417682+0000","last_became_peered":"2024-04-24T23:09:01.417682+0000","last_unstale":"2024-04-24T23:09:01.418065+0000","last_undegraded":"2024-04-24T23:09:01.418065+0000","last_fullsized":"2024-04-24T23:09:01.418065+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:06:07.547684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.421222+0000","last_change":"2024-04-24T23:09:00.407797+0000","last_active":"2024-04-24T23:09:01.421222+0000","last_peered":"2024-04-24T23:09:01.421222+0000","last_clean":"2024-04-24T23:09:01.421222+0000","last_became_active":"2024-04-24T23:09:00.407106+0000","last_became_peered":"2024-04-24T23:09:00.407106+0000","last_unstale":"2024-04-24T23:09:01.421222+0000","last_undegraded":"2024-04-24T23:09:01.421222+0000","last_fullsized":"2024-04-24T23:09:01.421222+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:20:05.863066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.424007+0000","last_change":"2024-04-24T23:09:01.422935+0000","last_active":"2024-04-24T23:09:01.424007+0000","last_peered":"2024-04-24T23:09:01.424007+0000","last_clean":"2024-04-24T23:09:01.424007+0000","last_became_active":"2024-04-24T23:09:01.422640+0000","last_became_peered":"2024-04-24T23:09:01.422640+0000","last_unstale":"2024-04-24T23:09:01.424007+0000","last_undegraded":"2024-04-24T23:09:01.424007+0000","last_fullsized":"2024-04-24T23:09:01.424007+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:36:13.565647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.415428+0000","last_change":"2024-04-24T23:09:00.405664+0000","last_active":"2024-04-24T23:09:01.415428+0000","last_peered":"2024-04-24T23:09:01.415428+0000","last_clean":"2024-04-24T23:09:01.415428+0000","last_became_active":"2024-04-24T23:09:00.405425+0000","last_became_peered":"2024-04-24T23:09:00.405425+0000","last_unstale":"2024-04-24T23:09:01.415428+0000","last_undegraded":"2024-04-24T23:09:01.415428+0000","last_fullsized":"2024-04-24T23:09:01.415428+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:25:23.902454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.427147+0000","last_change":"2024-04-24T23:09:01.427033+0000","last_active":"2024-04-24T23:09:01.427147+0000","last_peered":"2024-04-24T23:09:01.427147+0000","last_clean":"2024-04-24T23:09:01.427147+0000","last_became_active":"2024-04-24T23:09:01.426729+0000","last_became_peered":"2024-04-24T23:09:01.426729+0000","last_unstale":"2024-04-24T23:09:01.427147+0000","last_undegraded":"2024-04-24T23:09:01.427147+0000","last_fullsized":"2024-04-24T23:09:01.427147+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:49:37.654240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.416082+0000","last_change":"2024-04-24T23:09:00.407757+0000","last_active":"2024-04-24T23:09:01.416082+0000","last_peered":"2024-04-24T23:09:01.416082+0000","last_clean":"2024-04-24T23:09:01.416082+0000","last_became_active":"2024-04-24T23:09:00.407438+0000","last_became_peered":"2024-04-24T23:09:00.407438+0000","last_unstale":"2024-04-24T23:09:01.416082+0000","last_undegraded":"2024-04-24T23:09:01.416082+0000","last_fullsized":"2024-04-24T23:09:01.416082+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:39:53.729104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.427523+0000","last_change":"2024-04-24T23:09:01.427358+0000","last_active":"2024-04-24T23:09:01.427523+0000","last_peered":"2024-04-24T23:09:01.427523+0000","last_clean":"2024-04-24T23:09:01.427523+0000","last_became_active":"2024-04-24T23:09:01.427120+0000","last_became_peered":"2024-04-24T23:09:01.427120+0000","last_unstale":"2024-04-24T23:09:01.427523+0000","last_undegraded":"2024-04-24T23:09:01.427523+0000","last_fullsized":"2024-04-24T23:09:01.427523+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:58:46.154130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.419889+0000","last_change":"2024-04-24T23:09:00.408417+0000","last_active":"2024-04-24T23:09:01.419889+0000","last_peered":"2024-04-24T23:09:01.419889+0000","last_clean":"2024-04-24T23:09:01.419889+0000","last_became_active":"2024-04-24T23:09:00.408152+0000","last_became_peered":"2024-04-24T23:09:00.408152+0000","last_unstale":"2024-04-24T23:09:01.419889+0000","last_undegraded":"2024-04-24T23:09:01.419889+0000","last_fullsized":"2024-04-24T23:09:01.419889+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:11:05.331214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.420061+0000","last_change":"2024-04-24T23:09:01.419933+0000","last_active":"2024-04-24T23:09:01.420061+0000","last_peered":"2024-04-24T23:09:01.420061+0000","last_clean":"2024-04-24T23:09:01.420061+0000","last_became_active":"2024-04-24T23:09:01.419679+0000","last_became_peered":"2024-04-24T23:09:01.419679+0000","last_unstale":"2024-04-24T23:09:01.420061+0000","last_undegraded":"2024-04-24T23:09:01.420061+0000","last_fullsized":"2024-04-24T23:09:01.420061+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:35:02.456274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.412395+0000","last_change":"2024-04-24T23:09:00.404758+0000","last_active":"2024-04-24T23:09:01.412395+0000","last_peered":"2024-04-24T23:09:01.412395+0000","last_clean":"2024-04-24T23:09:01.412395+0000","last_became_active":"2024-04-24T23:09:00.404524+0000","last_became_peered":"2024-04-24T23:09:00.404524+0000","last_unstale":"2024-04-24T23:09:01.412395+0000","last_undegraded":"2024-04-24T23:09:01.412395+0000","last_fullsized":"2024-04-24T23:09:01.412395+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:01:38.230637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.430764+0000","last_change":"2024-04-24T23:09:01.430622+0000","last_active":"2024-04-24T23:09:01.430764+0000","last_peered":"2024-04-24T23:09:01.430764+0000","last_clean":"2024-04-24T23:09:01.430764+0000","last_became_active":"2024-04-24T23:09:01.429682+0000","last_became_peered":"2024-04-24T23:09:01.429682+0000","last_unstale":"2024-04-24T23:09:01.430764+0000","last_undegraded":"2024-04-24T23:09:01.430764+0000","last_fullsized":"2024-04-24T23:09:01.430764+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:05:30.385966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.421681+0000","last_change":"2024-04-24T23:09:00.401728+0000","last_active":"2024-04-24T23:09:01.421681+0000","last_peered":"2024-04-24T23:09:01.421681+0000","last_clean":"2024-04-24T23:09:01.421681+0000","last_became_active":"2024-04-24T23:09:00.401483+0000","last_became_peered":"2024-04-24T23:09:00.401483+0000","last_unstale":"2024-04-24T23:09:01.421681+0000","last_undegraded":"2024-04-24T23:09:01.421681+0000","last_fullsized":"2024-04-24T23:09:01.421681+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:55:36.056034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.428674+0000","last_change":"2024-04-24T23:09:01.428553+0000","last_active":"2024-04-24T23:09:01.428674+0000","last_peered":"2024-04-24T23:09:01.428674+0000","last_clean":"2024-04-24T23:09:01.428674+0000","last_became_active":"2024-04-24T23:09:01.428317+0000","last_became_peered":"2024-04-24T23:09:01.428317+0000","last_unstale":"2024-04-24T23:09:01.428674+0000","last_undegraded":"2024-04-24T23:09:01.428674+0000","last_fullsized":"2024-04-24T23:09:01.428674+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:19:37.280708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.416802+0000","last_change":"2024-04-24T23:09:00.403098+0000","last_active":"2024-04-24T23:09:01.416802+0000","last_peered":"2024-04-24T23:09:01.416802+0000","last_clean":"2024-04-24T23:09:01.416802+0000","last_became_active":"2024-04-24T23:09:00.402604+0000","last_became_peered":"2024-04-24T23:09:00.402604+0000","last_unstale":"2024-04-24T23:09:01.416802+0000","last_undegraded":"2024-04-24T23:09:01.416802+0000","last_fullsized":"2024-04-24T23:09:01.416802+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:49:35.707261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.418373+0000","last_change":"2024-04-24T23:09:01.417867+0000","last_active":"2024-04-24T23:09:01.418373+0000","last_peered":"2024-04-24T23:09:01.418373+0000","last_clean":"2024-04-24T23:09:01.418373+0000","last_became_active":"2024-04-24T23:09:01.417624+0000","last_became_peered":"2024-04-24T23:09:01.417624+0000","last_unstale":"2024-04-24T23:09:01.418373+0000","last_undegraded":"2024-04-24T23:09:01.418373+0000","last_fullsized":"2024-04-24T23:09:01.418373+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:01.620061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.15","version":"168'3","reported_seq":15,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.437710+0000","last_change":"2024-04-24T23:09:00.402348+0000","last_active":"2024-04-24T23:09:01.437710+0000","last_peered":"2024-04-24T23:09:01.437710+0000","last_clean":"2024-04-24T23:09:01.437710+0000","last_became_active":"2024-04-24T23:09:00.402101+0000","last_became_peered":"2024-04-24T23:09:00.402101+0000","last_unstale":"2024-04-24T23:09:01.437710+0000","last_undegraded":"2024-04-24T23:09:01.437710+0000","last_fullsized":"2024-04-24T23:09:01.437710+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:54:29.329728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":462,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.421351+0000","last_change":"2024-04-24T22:58:55.138810+0000","last_active":"2024-04-24T23:09:01.421351+0000","last_peered":"2024-04-24T23:09:01.421351+0000","last_clean":"2024-04-24T23:09:01.421351+0000","last_became_active":"2024-04-24T22:58:55.138627+0000","last_became_peered":"2024-04-24T22:58:55.138627+0000","last_unstale":"2024-04-24T23:09:01.421351+0000","last_undegraded":"2024-04-24T23:09:01.421351+0000","last_fullsized":"2024-04-24T23:09:01.421351+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_clean_scrub_stamp":"2024-04-24T22:58:54.127235+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:51.383182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574972,"num_pgs":16,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":913,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194398,"statfs":{"total":200000000,"available":199064241,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574972,"num_pgs":12,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":920,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194392,"statfs":{"total":200000000,"available":199057466,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574972,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1498,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193813,"statfs":{"total":200000000,"available":198465292,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574972,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":922,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194389,"statfs":{"total":200000000,"available":199055250,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574972,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":921,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194391,"statfs":{"total":200000000,"available":199056850,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574972,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":920,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194392,"statfs":{"total":200000000,"available":199057444,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574972,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1497,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193815,"statfs":{"total":200000000,"available":198467028,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574972,"num_pgs":12,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":920,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194391,"statfs":{"total":200000000,"available":199057376,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-24T23:09:05.429 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:09:05.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.564+0000 7f9d993c8640 1 Processor -- start 2024-04-24T23:09:05.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.564+0000 7f9d993c8640 1 -- start start 2024-04-24T23:09:05.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.564+0000 7f9d993c8640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9d9406c5f0 0x7f9d9406c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:05.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.564+0000 7f9d993c8640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9d940af890 0x7f9d9415cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:05.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.564+0000 7f9d993c8640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d940b6430 con 0x7f9d940b44d0 2024-04-24T23:09:05.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.564+0000 7f9d993c8640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f9d94156f80 con 0x7f9d940af890 2024-04-24T23:09:05.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.564+0000 7f9d993c8640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f9d940b6150 con 0x7f9d9406c5f0 2024-04-24T23:09:05.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.565+0000 7f9d937fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9d940af890 0x7f9d9415cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:05.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.565+0000 7f9d937fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9d940af890 0x7f9d9415cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40268/0 (socket says 172.21.15.32:40268) 2024-04-24T23:09:05.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.565+0000 7f9d937fe640 1 -- 172.21.15.32:0/4265714354 learned_addr learned my addr 172.21.15.32:0/4265714354 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:09:05.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.565+0000 7f9d92ffd640 1 --2- 172.21.15.32:0/4265714354 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9d9406c5f0 0x7f9d9406c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:05.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.565+0000 7f9d937fe640 1 -- 172.21.15.32:0/4265714354 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9d9406c5f0 msgr2=0x7f9d9406c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:09:05.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.565+0000 7f9d937fe640 1 --2- 172.21.15.32:0/4265714354 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9d9406c5f0 0x7f9d9406c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:05.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.565+0000 7f9d937fe640 1 -- 172.21.15.32:0/4265714354 >> v1:172.21.15.121:6789/0 conn(0x7f9d940b44d0 legacy=0x7f9d940b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:09:05.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.565+0000 7f9d937fe640 1 -- 172.21.15.32:0/4265714354 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d940b39f0 con 0x7f9d940af890 2024-04-24T23:09:05.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.566+0000 7f9d937fe640 1 --2- 172.21.15.32:0/4265714354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d940af890 0x7f9d9415cac0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c009e50 tx=0x7f9d8c02a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea0967a7f6c6b0c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:05.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.566+0000 7f9d927fc640 1 -- 172.21.15.32:0/4265714354 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c036070 con 0x7f9d940af890 2024-04-24T23:09:05.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.566+0000 7f9d927fc640 1 -- 172.21.15.32:0/4265714354 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d8c031600 con 0x7f9d940af890 2024-04-24T23:09:05.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.566+0000 7f9d927fc640 1 -- 172.21.15.32:0/4265714354 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c0319d0 con 0x7f9d940af890 2024-04-24T23:09:05.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.566+0000 7f9d993c8640 1 -- 172.21.15.32:0/4265714354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d940af890 msgr2=0x7f9d9415cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:05.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.566+0000 7f9d993c8640 1 --2- 172.21.15.32:0/4265714354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d940af890 0x7f9d9415cac0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c009e50 tx=0x7f9d8c02a630 comp rx=0 tx=0).stop 2024-04-24T23:09:05.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.566+0000 7f9d993c8640 1 -- 172.21.15.32:0/4265714354 shutdown_connections 2024-04-24T23:09:05.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.566+0000 7f9d993c8640 1 --2- 172.21.15.32:0/4265714354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d940af890 0x7f9d9415cac0 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:05.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.566+0000 7f9d993c8640 1 --2- 172.21.15.32:0/4265714354 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9d9406c5f0 0x7f9d9406c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:05.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.566+0000 7f9d993c8640 1 -- 172.21.15.32:0/4265714354 >> 172.21.15.32:0/4265714354 conn(0x7f9d9406b110 msgr2=0x7f9d9406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:05.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.566+0000 7f9d993c8640 1 -- 172.21.15.32:0/4265714354 shutdown_connections 2024-04-24T23:09:05.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.566+0000 7f9d993c8640 1 -- 172.21.15.32:0/4265714354 wait complete. 2024-04-24T23:09:05.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.568+0000 7f9d993c8640 1 Processor -- start 2024-04-24T23:09:05.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.568+0000 7f9d993c8640 1 -- start start 2024-04-24T23:09:05.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.568+0000 7f9d993c8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d9406c5f0 0x7f9d94146ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:05.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.568+0000 7f9d993c8640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9d940af890 0x7f9d941f2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:05.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.568+0000 7f9d993c8640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d940b44d0 0x7f9d941f5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:05.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.568+0000 7f9d993c8640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9406cf10 con 0x7f9d9406c5f0 2024-04-24T23:09:05.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.568+0000 7f9d993c8640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9d940b3fb0 con 0x7f9d940b44d0 2024-04-24T23:09:05.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.568+0000 7f9d993c8640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f9d9415d000 con 0x7f9d940af890 2024-04-24T23:09:05.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.568+0000 7f9d92ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d9406c5f0 0x7f9d94146ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:05.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.568+0000 7f9d92ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d9406c5f0 0x7f9d94146ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40270/0 (socket says 172.21.15.32:40270) 2024-04-24T23:09:05.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.568+0000 7f9d92ffd640 1 -- 172.21.15.32:0/1099658077 learned_addr learned my addr 172.21.15.32:0/1099658077 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:09:05.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.569+0000 7f9d92ffd640 1 -- 172.21.15.32:0/1099658077 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9d940af890 msgr2=0x7f9d941f2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:05.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.569+0000 7f9d8bfff640 1 --2- 172.21.15.32:0/1099658077 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9d940af890 0x7f9d941f2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:05.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.569+0000 7f9d937fe640 1 --2- 172.21.15.32:0/1099658077 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d940b44d0 0x7f9d941f5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:05.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.569+0000 7f9d92ffd640 1 --2- 172.21.15.32:0/1099658077 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9d940af890 0x7f9d941f2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:05.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.569+0000 7f9d92ffd640 1 -- 172.21.15.32:0/1099658077 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d940b44d0 msgr2=0x7f9d941f5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:05.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.569+0000 7f9d92ffd640 1 --2- 172.21.15.32:0/1099658077 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d940b44d0 0x7f9d941f5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:05.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.569+0000 7f9d92ffd640 1 -- 172.21.15.32:0/1099658077 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d941f3440 con 0x7f9d9406c5f0 2024-04-24T23:09:05.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.569+0000 7f9d8bfff640 1 --2- 172.21.15.32:0/1099658077 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9d940af890 0x7f9d941f2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:09:05.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.569+0000 7f9d937fe640 1 --2- 172.21.15.32:0/1099658077 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d940b44d0 0x7f9d941f5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:09:05.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.569+0000 7f9d92ffd640 1 --2- 172.21.15.32:0/1099658077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d9406c5f0 0x7f9d94146ef0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f9d8400c7f0 tx=0x7f9d8400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:05.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.569+0000 7f9d8b7fe640 1 -- 172.21.15.32:0/1099658077 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d84015070 con 0x7f9d9406c5f0 2024-04-24T23:09:05.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.569+0000 7f9d8b7fe640 1 -- 172.21.15.32:0/1099658077 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d84004240 con 0x7f9d9406c5f0 2024-04-24T23:09:05.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.569+0000 7f9d993c8640 1 -- 172.21.15.32:0/1099658077 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d9415dd00 con 0x7f9d9406c5f0 2024-04-24T23:09:05.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.569+0000 7f9d8b7fe640 1 -- 172.21.15.32:0/1099658077 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d84007620 con 0x7f9d9406c5f0 2024-04-24T23:09:05.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.569+0000 7f9d993c8640 1 -- 172.21.15.32:0/1099658077 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d9415dfa0 con 0x7f9d9406c5f0 2024-04-24T23:09:05.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.570+0000 7f9d993c8640 1 -- 172.21.15.32:0/1099658077 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d94068430 con 0x7f9d9406c5f0 2024-04-24T23:09:05.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.570+0000 7f9d8b7fe640 1 -- 172.21.15.32:0/1099658077 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9d840079d0 con 0x7f9d9406c5f0 2024-04-24T23:09:05.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.571+0000 7f9d8b7fe640 1 --2- 172.21.15.32:0/1099658077 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9d7406be30 0x7f9d7406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:05.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.571+0000 7f9d8b7fe640 1 -- 172.21.15.32:0/1099658077 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 5709+0+0 (secure 0 0 0) 0x7f9d8408b1d0 con 0x7f9d9406c5f0 2024-04-24T23:09:05.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.571+0000 7f9d8bfff640 1 --2- 172.21.15.32:0/1099658077 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9d7406be30 0x7f9d7406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:05.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.572+0000 7f9d8bfff640 1 --2- 172.21.15.32:0/1099658077 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9d7406be30 0x7f9d7406e2f0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f9d9415bb60 tx=0x7f9d7c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:05.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.575+0000 7f9d8b7fe640 1 -- 172.21.15.32:0/1099658077 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9d941f3440 con 0x7f9d9406c5f0 2024-04-24T23:09:05.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.753+0000 7f9d993c8640 1 -- 172.21.15.32:0/1099658077 --> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9d940b5670 con 0x7f9d7406be30 2024-04-24T23:09:05.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.759+0000 7f9d8b7fe640 1 -- 172.21.15.32:0/1099658077 <== mgr.4108 v2:172.21.15.121:6832/1848157943 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247926 (secure 0 0 0) 0x7f9d940b5670 con 0x7f9d7406be30 2024-04-24T23:09:05.763 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:09:05.764 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-24T23:09:05.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.762+0000 7f9d993c8640 1 -- 172.21.15.32:0/1099658077 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9d7406be30 msgr2=0x7f9d7406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:05.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.762+0000 7f9d993c8640 1 --2- 172.21.15.32:0/1099658077 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9d7406be30 0x7f9d7406e2f0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f9d9415bb60 tx=0x7f9d7c008040 comp rx=0 tx=0).stop 2024-04-24T23:09:05.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.762+0000 7f9d993c8640 1 -- 172.21.15.32:0/1099658077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d9406c5f0 msgr2=0x7f9d94146ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:05.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.762+0000 7f9d993c8640 1 --2- 172.21.15.32:0/1099658077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d9406c5f0 0x7f9d94146ef0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f9d8400c7f0 tx=0x7f9d8400ccb0 comp rx=0 tx=0).stop 2024-04-24T23:09:05.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.762+0000 7f9d993c8640 1 -- 172.21.15.32:0/1099658077 shutdown_connections 2024-04-24T23:09:05.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.762+0000 7f9d993c8640 1 --2- 172.21.15.32:0/1099658077 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9d7406be30 0x7f9d7406e2f0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:05.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.762+0000 7f9d993c8640 1 --2- 172.21.15.32:0/1099658077 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d940b44d0 0x7f9d941f5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:05.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.762+0000 7f9d993c8640 1 --2- 172.21.15.32:0/1099658077 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9d940af890 0x7f9d941f2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:05.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.762+0000 7f9d993c8640 1 --2- 172.21.15.32:0/1099658077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d9406c5f0 0x7f9d94146ef0 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:05.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.762+0000 7f9d993c8640 1 -- 172.21.15.32:0/1099658077 >> 172.21.15.32:0/1099658077 conn(0x7f9d9406b110 msgr2=0x7f9d9406b650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:05.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.762+0000 7f9d993c8640 1 -- 172.21.15.32:0/1099658077 shutdown_connections 2024-04-24T23:09:05.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.762+0000 7f9d993c8640 1 -- 172.21.15.32:0/1099658077 wait complete. 2024-04-24T23:09:05.774 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":467,"stamp":"2024-04-24T23:09:05.270206+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":112,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":8514,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1553985,"statfs":{"total":1600000000,"available":1591280947,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.876501"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.424485+0000","last_change":"2024-04-24T23:09:01.424376+0000","last_active":"2024-04-24T23:09:01.424485+0000","last_peered":"2024-04-24T23:09:01.424485+0000","last_clean":"2024-04-24T23:09:01.424485+0000","last_became_active":"2024-04-24T23:09:01.423583+0000","last_became_peered":"2024-04-24T23:09:01.423583+0000","last_unstale":"2024-04-24T23:09:01.424485+0000","last_undegraded":"2024-04-24T23:09:01.424485+0000","last_fullsized":"2024-04-24T23:09:01.424485+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:47:13.405533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.427226+0000","last_change":"2024-04-24T23:09:01.427034+0000","last_active":"2024-04-24T23:09:01.427226+0000","last_peered":"2024-04-24T23:09:01.427226+0000","last_clean":"2024-04-24T23:09:01.427226+0000","last_became_active":"2024-04-24T23:09:01.426768+0000","last_became_peered":"2024-04-24T23:09:01.426768+0000","last_unstale":"2024-04-24T23:09:01.427226+0000","last_undegraded":"2024-04-24T23:09:01.427226+0000","last_fullsized":"2024-04-24T23:09:01.427226+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:04.492603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.418136+0000","last_change":"2024-04-24T23:09:01.417701+0000","last_active":"2024-04-24T23:09:01.418136+0000","last_peered":"2024-04-24T23:09:01.418136+0000","last_clean":"2024-04-24T23:09:01.418136+0000","last_became_active":"2024-04-24T23:09:01.417023+0000","last_became_peered":"2024-04-24T23:09:01.417023+0000","last_unstale":"2024-04-24T23:09:01.418136+0000","last_undegraded":"2024-04-24T23:09:01.418136+0000","last_fullsized":"2024-04-24T23:09:01.418136+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:49:51.435440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.418558+0000","last_change":"2024-04-24T23:09:01.418187+0000","last_active":"2024-04-24T23:09:01.418558+0000","last_peered":"2024-04-24T23:09:01.418558+0000","last_clean":"2024-04-24T23:09:01.418558+0000","last_became_active":"2024-04-24T23:09:01.417852+0000","last_became_peered":"2024-04-24T23:09:01.417852+0000","last_unstale":"2024-04-24T23:09:01.418558+0000","last_undegraded":"2024-04-24T23:09:01.418558+0000","last_fullsized":"2024-04-24T23:09:01.418558+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:49:06.192899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.424014+0000","last_change":"2024-04-24T23:09:01.423879+0000","last_active":"2024-04-24T23:09:01.424014+0000","last_peered":"2024-04-24T23:09:01.424014+0000","last_clean":"2024-04-24T23:09:01.424014+0000","last_became_active":"2024-04-24T23:09:01.423554+0000","last_became_peered":"2024-04-24T23:09:01.423554+0000","last_unstale":"2024-04-24T23:09:01.424014+0000","last_undegraded":"2024-04-24T23:09:01.424014+0000","last_fullsized":"2024-04-24T23:09:01.424014+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:59.289070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.424293+0000","last_change":"2024-04-24T23:09:01.424162+0000","last_active":"2024-04-24T23:09:01.424293+0000","last_peered":"2024-04-24T23:09:01.424293+0000","last_clean":"2024-04-24T23:09:01.424293+0000","last_became_active":"2024-04-24T23:09:01.423581+0000","last_became_peered":"2024-04-24T23:09:01.423581+0000","last_unstale":"2024-04-24T23:09:01.424293+0000","last_undegraded":"2024-04-24T23:09:01.424293+0000","last_fullsized":"2024-04-24T23:09:01.424293+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:20:04.895778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.418682+0000","last_change":"2024-04-24T23:09:01.418567+0000","last_active":"2024-04-24T23:09:01.418682+0000","last_peered":"2024-04-24T23:09:01.418682+0000","last_clean":"2024-04-24T23:09:01.418682+0000","last_became_active":"2024-04-24T23:09:01.418263+0000","last_became_peered":"2024-04-24T23:09:01.418263+0000","last_unstale":"2024-04-24T23:09:01.418682+0000","last_undegraded":"2024-04-24T23:09:01.418682+0000","last_fullsized":"2024-04-24T23:09:01.418682+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:37.532986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.428738+0000","last_change":"2024-04-24T23:09:01.428631+0000","last_active":"2024-04-24T23:09:01.428738+0000","last_peered":"2024-04-24T23:09:01.428738+0000","last_clean":"2024-04-24T23:09:01.428738+0000","last_became_active":"2024-04-24T23:09:01.428378+0000","last_became_peered":"2024-04-24T23:09:01.428378+0000","last_unstale":"2024-04-24T23:09:01.428738+0000","last_undegraded":"2024-04-24T23:09:01.428738+0000","last_fullsized":"2024-04-24T23:09:01.428738+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:24.669943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.422743+0000","last_change":"2024-04-24T23:09:01.422401+0000","last_active":"2024-04-24T23:09:01.422743+0000","last_peered":"2024-04-24T23:09:01.422743+0000","last_clean":"2024-04-24T23:09:01.422743+0000","last_became_active":"2024-04-24T23:09:01.422096+0000","last_became_peered":"2024-04-24T23:09:01.422096+0000","last_unstale":"2024-04-24T23:09:01.422743+0000","last_undegraded":"2024-04-24T23:09:01.422743+0000","last_fullsized":"2024-04-24T23:09:01.422743+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:33:27.756672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.421571+0000","last_change":"2024-04-24T23:09:01.421429+0000","last_active":"2024-04-24T23:09:01.421571+0000","last_peered":"2024-04-24T23:09:01.421571+0000","last_clean":"2024-04-24T23:09:01.421571+0000","last_became_active":"2024-04-24T23:09:01.421202+0000","last_became_peered":"2024-04-24T23:09:01.421202+0000","last_unstale":"2024-04-24T23:09:01.421571+0000","last_undegraded":"2024-04-24T23:09:01.421571+0000","last_fullsized":"2024-04-24T23:09:01.421571+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:18.877710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.424533+0000","last_change":"2024-04-24T23:09:01.424376+0000","last_active":"2024-04-24T23:09:01.424533+0000","last_peered":"2024-04-24T23:09:01.424533+0000","last_clean":"2024-04-24T23:09:01.424533+0000","last_became_active":"2024-04-24T23:09:01.423934+0000","last_became_peered":"2024-04-24T23:09:01.423934+0000","last_unstale":"2024-04-24T23:09:01.424533+0000","last_undegraded":"2024-04-24T23:09:01.424533+0000","last_fullsized":"2024-04-24T23:09:01.424533+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:11.242842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.430893+0000","last_change":"2024-04-24T23:09:01.430677+0000","last_active":"2024-04-24T23:09:01.430893+0000","last_peered":"2024-04-24T23:09:01.430893+0000","last_clean":"2024-04-24T23:09:01.430893+0000","last_became_active":"2024-04-24T23:09:01.430257+0000","last_became_peered":"2024-04-24T23:09:01.430257+0000","last_unstale":"2024-04-24T23:09:01.430893+0000","last_undegraded":"2024-04-24T23:09:01.430893+0000","last_fullsized":"2024-04-24T23:09:01.430893+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:26.256633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.425215+0000","last_change":"2024-04-24T23:09:01.425094+0000","last_active":"2024-04-24T23:09:01.425215+0000","last_peered":"2024-04-24T23:09:01.425215+0000","last_clean":"2024-04-24T23:09:01.425215+0000","last_became_active":"2024-04-24T23:09:01.424860+0000","last_became_peered":"2024-04-24T23:09:01.424860+0000","last_unstale":"2024-04-24T23:09:01.425215+0000","last_undegraded":"2024-04-24T23:09:01.425215+0000","last_fullsized":"2024-04-24T23:09:01.425215+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:43:45.589467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.428824+0000","last_change":"2024-04-24T23:09:01.428710+0000","last_active":"2024-04-24T23:09:01.428824+0000","last_peered":"2024-04-24T23:09:01.428824+0000","last_clean":"2024-04-24T23:09:01.428824+0000","last_became_active":"2024-04-24T23:09:01.427986+0000","last_became_peered":"2024-04-24T23:09:01.427986+0000","last_unstale":"2024-04-24T23:09:01.428824+0000","last_undegraded":"2024-04-24T23:09:01.428824+0000","last_fullsized":"2024-04-24T23:09:01.428824+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:52:28.859209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.428087+0000","last_change":"2024-04-24T23:09:01.427719+0000","last_active":"2024-04-24T23:09:01.428087+0000","last_peered":"2024-04-24T23:09:01.428087+0000","last_clean":"2024-04-24T23:09:01.428087+0000","last_became_active":"2024-04-24T23:09:01.427489+0000","last_became_peered":"2024-04-24T23:09:01.427489+0000","last_unstale":"2024-04-24T23:09:01.428087+0000","last_undegraded":"2024-04-24T23:09:01.428087+0000","last_fullsized":"2024-04-24T23:09:01.428087+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:42:11.174669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.423056+0000","last_change":"2024-04-24T23:09:01.422929+0000","last_active":"2024-04-24T23:09:01.423056+0000","last_peered":"2024-04-24T23:09:01.423056+0000","last_clean":"2024-04-24T23:09:01.423056+0000","last_became_active":"2024-04-24T23:09:01.422705+0000","last_became_peered":"2024-04-24T23:09:01.422705+0000","last_unstale":"2024-04-24T23:09:01.423056+0000","last_undegraded":"2024-04-24T23:09:01.423056+0000","last_fullsized":"2024-04-24T23:09:01.423056+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:53:56.337235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.418156+0000","last_change":"2024-04-24T23:09:01.417978+0000","last_active":"2024-04-24T23:09:01.418156+0000","last_peered":"2024-04-24T23:09:01.418156+0000","last_clean":"2024-04-24T23:09:01.418156+0000","last_became_active":"2024-04-24T23:09:01.417718+0000","last_became_peered":"2024-04-24T23:09:01.417718+0000","last_unstale":"2024-04-24T23:09:01.418156+0000","last_undegraded":"2024-04-24T23:09:01.418156+0000","last_fullsized":"2024-04-24T23:09:01.418156+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:05.376272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.424489+0000","last_change":"2024-04-24T23:09:01.424377+0000","last_active":"2024-04-24T23:09:01.424489+0000","last_peered":"2024-04-24T23:09:01.424489+0000","last_clean":"2024-04-24T23:09:01.424489+0000","last_became_active":"2024-04-24T23:09:01.424138+0000","last_became_peered":"2024-04-24T23:09:01.424138+0000","last_unstale":"2024-04-24T23:09:01.424489+0000","last_undegraded":"2024-04-24T23:09:01.424489+0000","last_fullsized":"2024-04-24T23:09:01.424489+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:33:35.741652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.425452+0000","last_change":"2024-04-24T23:09:01.425339+0000","last_active":"2024-04-24T23:09:01.425452+0000","last_peered":"2024-04-24T23:09:01.425452+0000","last_clean":"2024-04-24T23:09:01.425452+0000","last_became_active":"2024-04-24T23:09:01.425085+0000","last_became_peered":"2024-04-24T23:09:01.425085+0000","last_unstale":"2024-04-24T23:09:01.425452+0000","last_undegraded":"2024-04-24T23:09:01.425452+0000","last_fullsized":"2024-04-24T23:09:01.425452+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:42:33.414609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.424084+0000","last_change":"2024-04-24T23:09:01.423830+0000","last_active":"2024-04-24T23:09:01.424084+0000","last_peered":"2024-04-24T23:09:01.424084+0000","last_clean":"2024-04-24T23:09:01.424084+0000","last_became_active":"2024-04-24T23:09:01.423577+0000","last_became_peered":"2024-04-24T23:09:01.423577+0000","last_unstale":"2024-04-24T23:09:01.424084+0000","last_undegraded":"2024-04-24T23:09:01.424084+0000","last_fullsized":"2024-04-24T23:09:01.424084+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:55.093440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.418112+0000","last_change":"2024-04-24T23:09:01.417570+0000","last_active":"2024-04-24T23:09:01.418112+0000","last_peered":"2024-04-24T23:09:01.418112+0000","last_clean":"2024-04-24T23:09:01.418112+0000","last_became_active":"2024-04-24T23:09:01.417313+0000","last_became_peered":"2024-04-24T23:09:01.417313+0000","last_unstale":"2024-04-24T23:09:01.418112+0000","last_undegraded":"2024-04-24T23:09:01.418112+0000","last_fullsized":"2024-04-24T23:09:01.418112+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:12.513190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.430873+0000","last_change":"2024-04-24T23:09:01.430483+0000","last_active":"2024-04-24T23:09:01.430873+0000","last_peered":"2024-04-24T23:09:01.430873+0000","last_clean":"2024-04-24T23:09:01.430873+0000","last_became_active":"2024-04-24T23:09:01.429720+0000","last_became_peered":"2024-04-24T23:09:01.429720+0000","last_unstale":"2024-04-24T23:09:01.430873+0000","last_undegraded":"2024-04-24T23:09:01.430873+0000","last_fullsized":"2024-04-24T23:09:01.430873+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:12:56.098330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.419192+0000","last_change":"2024-04-24T23:09:01.419061+0000","last_active":"2024-04-24T23:09:01.419192+0000","last_peered":"2024-04-24T23:09:01.419192+0000","last_clean":"2024-04-24T23:09:01.419192+0000","last_became_active":"2024-04-24T23:09:01.418812+0000","last_became_peered":"2024-04-24T23:09:01.418812+0000","last_unstale":"2024-04-24T23:09:01.419192+0000","last_undegraded":"2024-04-24T23:09:01.419192+0000","last_fullsized":"2024-04-24T23:09:01.419192+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:42:04.346496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.430723+0000","last_change":"2024-04-24T23:09:01.430595+0000","last_active":"2024-04-24T23:09:01.430723+0000","last_peered":"2024-04-24T23:09:01.430723+0000","last_clean":"2024-04-24T23:09:01.430723+0000","last_became_active":"2024-04-24T23:09:01.430363+0000","last_became_peered":"2024-04-24T23:09:01.430363+0000","last_unstale":"2024-04-24T23:09:01.430723+0000","last_undegraded":"2024-04-24T23:09:01.430723+0000","last_fullsized":"2024-04-24T23:09:01.430723+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:49:04.371757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.422425+0000","last_change":"2024-04-24T23:09:01.422312+0000","last_active":"2024-04-24T23:09:01.422425+0000","last_peered":"2024-04-24T23:09:01.422425+0000","last_clean":"2024-04-24T23:09:01.422425+0000","last_became_active":"2024-04-24T23:09:01.422089+0000","last_became_peered":"2024-04-24T23:09:01.422089+0000","last_unstale":"2024-04-24T23:09:01.422425+0000","last_undegraded":"2024-04-24T23:09:01.422425+0000","last_fullsized":"2024-04-24T23:09:01.422425+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:32:39.034275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.424032+0000","last_change":"2024-04-24T23:09:01.423315+0000","last_active":"2024-04-24T23:09:01.424032+0000","last_peered":"2024-04-24T23:09:01.424032+0000","last_clean":"2024-04-24T23:09:01.424032+0000","last_became_active":"2024-04-24T23:09:01.423073+0000","last_became_peered":"2024-04-24T23:09:01.423073+0000","last_unstale":"2024-04-24T23:09:01.424032+0000","last_undegraded":"2024-04-24T23:09:01.424032+0000","last_fullsized":"2024-04-24T23:09:01.424032+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:05:04.724031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.423608+0000","last_change":"2024-04-24T23:09:01.422911+0000","last_active":"2024-04-24T23:09:01.423608+0000","last_peered":"2024-04-24T23:09:01.423608+0000","last_clean":"2024-04-24T23:09:01.423608+0000","last_became_active":"2024-04-24T23:09:01.422610+0000","last_became_peered":"2024-04-24T23:09:01.422610+0000","last_unstale":"2024-04-24T23:09:01.423608+0000","last_undegraded":"2024-04-24T23:09:01.423608+0000","last_fullsized":"2024-04-24T23:09:01.423608+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:25.802602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.426596+0000","last_change":"2024-04-24T23:09:01.426447+0000","last_active":"2024-04-24T23:09:01.426596+0000","last_peered":"2024-04-24T23:09:01.426596+0000","last_clean":"2024-04-24T23:09:01.426596+0000","last_became_active":"2024-04-24T23:09:01.426213+0000","last_became_peered":"2024-04-24T23:09:01.426213+0000","last_unstale":"2024-04-24T23:09:01.426596+0000","last_undegraded":"2024-04-24T23:09:01.426596+0000","last_fullsized":"2024-04-24T23:09:01.426596+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:20:22.766564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.430058+0000","last_change":"2024-04-24T23:09:01.429889+0000","last_active":"2024-04-24T23:09:01.430058+0000","last_peered":"2024-04-24T23:09:01.430058+0000","last_clean":"2024-04-24T23:09:01.430058+0000","last_became_active":"2024-04-24T23:09:01.429664+0000","last_became_peered":"2024-04-24T23:09:01.429664+0000","last_unstale":"2024-04-24T23:09:01.430058+0000","last_undegraded":"2024-04-24T23:09:01.430058+0000","last_fullsized":"2024-04-24T23:09:01.430058+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:09:48.797884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.429584+0000","last_change":"2024-04-24T23:09:01.429438+0000","last_active":"2024-04-24T23:09:01.429584+0000","last_peered":"2024-04-24T23:09:01.429584+0000","last_clean":"2024-04-24T23:09:01.429584+0000","last_became_active":"2024-04-24T23:09:01.428969+0000","last_became_peered":"2024-04-24T23:09:01.428969+0000","last_unstale":"2024-04-24T23:09:01.429584+0000","last_undegraded":"2024-04-24T23:09:01.429584+0000","last_fullsized":"2024-04-24T23:09:01.429584+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:47.404287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.426223+0000","last_change":"2024-04-24T23:09:01.425483+0000","last_active":"2024-04-24T23:09:01.426223+0000","last_peered":"2024-04-24T23:09:01.426223+0000","last_clean":"2024-04-24T23:09:01.426223+0000","last_became_active":"2024-04-24T23:09:01.425206+0000","last_became_peered":"2024-04-24T23:09:01.425206+0000","last_unstale":"2024-04-24T23:09:01.426223+0000","last_undegraded":"2024-04-24T23:09:01.426223+0000","last_fullsized":"2024-04-24T23:09:01.426223+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:11:44.741451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.430735+0000","last_change":"2024-04-24T23:09:01.430604+0000","last_active":"2024-04-24T23:09:01.430735+0000","last_peered":"2024-04-24T23:09:01.430735+0000","last_clean":"2024-04-24T23:09:01.430735+0000","last_became_active":"2024-04-24T23:09:01.430371+0000","last_became_peered":"2024-04-24T23:09:01.430371+0000","last_unstale":"2024-04-24T23:09:01.430735+0000","last_undegraded":"2024-04-24T23:09:01.430735+0000","last_fullsized":"2024-04-24T23:09:01.430735+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:53:09.208422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.424508+0000","last_change":"2024-04-24T23:09:01.424409+0000","last_active":"2024-04-24T23:09:01.424508+0000","last_peered":"2024-04-24T23:09:01.424508+0000","last_clean":"2024-04-24T23:09:01.424508+0000","last_became_active":"2024-04-24T23:09:01.424183+0000","last_became_peered":"2024-04-24T23:09:01.424183+0000","last_unstale":"2024-04-24T23:09:01.424508+0000","last_undegraded":"2024-04-24T23:09:01.424508+0000","last_fullsized":"2024-04-24T23:09:01.424508+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:20.037915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"168'4","reported_seq":16,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.438609+0000","last_change":"2024-04-24T23:09:00.402015+0000","last_active":"2024-04-24T23:09:01.438609+0000","last_peered":"2024-04-24T23:09:01.438609+0000","last_clean":"2024-04-24T23:09:01.438609+0000","last_became_active":"2024-04-24T23:09:00.401741+0000","last_became_peered":"2024-04-24T23:09:00.401741+0000","last_unstale":"2024-04-24T23:09:01.438609+0000","last_undegraded":"2024-04-24T23:09:01.438609+0000","last_fullsized":"2024-04-24T23:09:01.438609+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:08:18.751223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.424145+0000","last_change":"2024-04-24T23:09:01.424016+0000","last_active":"2024-04-24T23:09:01.424145+0000","last_peered":"2024-04-24T23:09:01.424145+0000","last_clean":"2024-04-24T23:09:01.424145+0000","last_became_active":"2024-04-24T23:09:01.423774+0000","last_became_peered":"2024-04-24T23:09:01.423774+0000","last_unstale":"2024-04-24T23:09:01.424145+0000","last_undegraded":"2024-04-24T23:09:01.424145+0000","last_fullsized":"2024-04-24T23:09:01.424145+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:47:43.101190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.1e","version":"168'1","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.436864+0000","last_change":"2024-04-24T23:09:00.402853+0000","last_active":"2024-04-24T23:09:01.436864+0000","last_peered":"2024-04-24T23:09:01.436864+0000","last_clean":"2024-04-24T23:09:01.436864+0000","last_became_active":"2024-04-24T23:09:00.402244+0000","last_became_peered":"2024-04-24T23:09:00.402244+0000","last_unstale":"2024-04-24T23:09:01.436864+0000","last_undegraded":"2024-04-24T23:09:01.436864+0000","last_fullsized":"2024-04-24T23:09:01.436864+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:16:04.868002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.426678+0000","last_change":"2024-04-24T23:09:01.426562+0000","last_active":"2024-04-24T23:09:01.426678+0000","last_peered":"2024-04-24T23:09:01.426678+0000","last_clean":"2024-04-24T23:09:01.426678+0000","last_became_active":"2024-04-24T23:09:01.425730+0000","last_became_peered":"2024-04-24T23:09:01.425730+0000","last_unstale":"2024-04-24T23:09:01.426678+0000","last_undegraded":"2024-04-24T23:09:01.426678+0000","last_fullsized":"2024-04-24T23:09:01.426678+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:49:26.379338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.1d","version":"168'2","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.438129+0000","last_change":"2024-04-24T23:09:00.407212+0000","last_active":"2024-04-24T23:09:01.438129+0000","last_peered":"2024-04-24T23:09:01.438129+0000","last_clean":"2024-04-24T23:09:01.438129+0000","last_became_active":"2024-04-24T23:09:00.406687+0000","last_became_peered":"2024-04-24T23:09:00.406687+0000","last_unstale":"2024-04-24T23:09:01.438129+0000","last_undegraded":"2024-04-24T23:09:01.438129+0000","last_fullsized":"2024-04-24T23:09:01.438129+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:49:21.836047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.418827+0000","last_change":"2024-04-24T23:09:01.418687+0000","last_active":"2024-04-24T23:09:01.418827+0000","last_peered":"2024-04-24T23:09:01.418827+0000","last_clean":"2024-04-24T23:09:01.418827+0000","last_became_active":"2024-04-24T23:09:01.418440+0000","last_became_peered":"2024-04-24T23:09:01.418440+0000","last_unstale":"2024-04-24T23:09:01.418827+0000","last_undegraded":"2024-04-24T23:09:01.418827+0000","last_fullsized":"2024-04-24T23:09:01.418827+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:21:14.862296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1c","version":"168'2","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.437441+0000","last_change":"2024-04-24T23:09:00.403972+0000","last_active":"2024-04-24T23:09:01.437441+0000","last_peered":"2024-04-24T23:09:01.437441+0000","last_clean":"2024-04-24T23:09:01.437441+0000","last_became_active":"2024-04-24T23:09:00.403717+0000","last_became_peered":"2024-04-24T23:09:00.403717+0000","last_unstale":"2024-04-24T23:09:01.437441+0000","last_undegraded":"2024-04-24T23:09:01.437441+0000","last_fullsized":"2024-04-24T23:09:01.437441+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:02.508478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.428256+0000","last_change":"2024-04-24T23:09:01.428149+0000","last_active":"2024-04-24T23:09:01.428256+0000","last_peered":"2024-04-24T23:09:01.428256+0000","last_clean":"2024-04-24T23:09:01.428256+0000","last_became_active":"2024-04-24T23:09:01.427832+0000","last_became_peered":"2024-04-24T23:09:01.427832+0000","last_unstale":"2024-04-24T23:09:01.428256+0000","last_undegraded":"2024-04-24T23:09:01.428256+0000","last_fullsized":"2024-04-24T23:09:01.428256+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:35:30.340066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"168'1","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.436498+0000","last_change":"2024-04-24T23:09:00.409341+0000","last_active":"2024-04-24T23:09:01.436498+0000","last_peered":"2024-04-24T23:09:01.436498+0000","last_clean":"2024-04-24T23:09:01.436498+0000","last_became_active":"2024-04-24T23:09:00.409066+0000","last_became_peered":"2024-04-24T23:09:00.409066+0000","last_unstale":"2024-04-24T23:09:01.436498+0000","last_undegraded":"2024-04-24T23:09:01.436498+0000","last_fullsized":"2024-04-24T23:09:01.436498+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:21:23.449197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.420541+0000","last_change":"2024-04-24T23:09:01.420426+0000","last_active":"2024-04-24T23:09:01.420541+0000","last_peered":"2024-04-24T23:09:01.420541+0000","last_clean":"2024-04-24T23:09:01.420541+0000","last_became_active":"2024-04-24T23:09:01.420175+0000","last_became_peered":"2024-04-24T23:09:01.420175+0000","last_unstale":"2024-04-24T23:09:01.420541+0000","last_undegraded":"2024-04-24T23:09:01.420541+0000","last_fullsized":"2024-04-24T23:09:01.420541+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:30:47.293551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1a","version":"168'1","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.436871+0000","last_change":"2024-04-24T23:09:00.407115+0000","last_active":"2024-04-24T23:09:01.436871+0000","last_peered":"2024-04-24T23:09:01.436871+0000","last_clean":"2024-04-24T23:09:01.436871+0000","last_became_active":"2024-04-24T23:09:00.406871+0000","last_became_peered":"2024-04-24T23:09:00.406871+0000","last_unstale":"2024-04-24T23:09:01.436871+0000","last_undegraded":"2024-04-24T23:09:01.436871+0000","last_fullsized":"2024-04-24T23:09:01.436871+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:15.821843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.430032+0000","last_change":"2024-04-24T23:09:01.429910+0000","last_active":"2024-04-24T23:09:01.430032+0000","last_peered":"2024-04-24T23:09:01.430032+0000","last_clean":"2024-04-24T23:09:01.430032+0000","last_became_active":"2024-04-24T23:09:01.429669+0000","last_became_peered":"2024-04-24T23:09:01.429669+0000","last_unstale":"2024-04-24T23:09:01.430032+0000","last_undegraded":"2024-04-24T23:09:01.430032+0000","last_fullsized":"2024-04-24T23:09:01.430032+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:48:48.838559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.416647+0000","last_change":"2024-04-24T23:09:00.406851+0000","last_active":"2024-04-24T23:09:01.416647+0000","last_peered":"2024-04-24T23:09:01.416647+0000","last_clean":"2024-04-24T23:09:01.416647+0000","last_became_active":"2024-04-24T23:09:00.406602+0000","last_became_peered":"2024-04-24T23:09:00.406602+0000","last_unstale":"2024-04-24T23:09:01.416647+0000","last_undegraded":"2024-04-24T23:09:01.416647+0000","last_fullsized":"2024-04-24T23:09:01.416647+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:38.880892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.431000+0000","last_change":"2024-04-24T23:09:01.430861+0000","last_active":"2024-04-24T23:09:01.431000+0000","last_peered":"2024-04-24T23:09:01.431000+0000","last_clean":"2024-04-24T23:09:01.431000+0000","last_became_active":"2024-04-24T23:09:01.430112+0000","last_became_peered":"2024-04-24T23:09:01.430112+0000","last_unstale":"2024-04-24T23:09:01.431000+0000","last_undegraded":"2024-04-24T23:09:01.431000+0000","last_fullsized":"2024-04-24T23:09:01.431000+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:55:44.000350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.412594+0000","last_change":"2024-04-24T23:09:00.406935+0000","last_active":"2024-04-24T23:09:01.412594+0000","last_peered":"2024-04-24T23:09:01.412594+0000","last_clean":"2024-04-24T23:09:01.412594+0000","last_became_active":"2024-04-24T23:09:00.406356+0000","last_became_peered":"2024-04-24T23:09:00.406356+0000","last_unstale":"2024-04-24T23:09:01.412594+0000","last_undegraded":"2024-04-24T23:09:01.412594+0000","last_fullsized":"2024-04-24T23:09:01.412594+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:46:19.034727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.419658+0000","last_change":"2024-04-24T23:09:01.419495+0000","last_active":"2024-04-24T23:09:01.419658+0000","last_peered":"2024-04-24T23:09:01.419658+0000","last_clean":"2024-04-24T23:09:01.419658+0000","last_became_active":"2024-04-24T23:09:01.418998+0000","last_became_peered":"2024-04-24T23:09:01.418998+0000","last_unstale":"2024-04-24T23:09:01.419658+0000","last_undegraded":"2024-04-24T23:09:01.419658+0000","last_fullsized":"2024-04-24T23:09:01.419658+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:48:33.370702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.419143+0000","last_change":"2024-04-24T23:09:00.405813+0000","last_active":"2024-04-24T23:09:01.419143+0000","last_peered":"2024-04-24T23:09:01.419143+0000","last_clean":"2024-04-24T23:09:01.419143+0000","last_became_active":"2024-04-24T23:09:00.405573+0000","last_became_peered":"2024-04-24T23:09:00.405573+0000","last_unstale":"2024-04-24T23:09:01.419143+0000","last_undegraded":"2024-04-24T23:09:01.419143+0000","last_fullsized":"2024-04-24T23:09:01.419143+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:39:29.961818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.427724+0000","last_change":"2024-04-24T23:09:01.426812+0000","last_active":"2024-04-24T23:09:01.427724+0000","last_peered":"2024-04-24T23:09:01.427724+0000","last_clean":"2024-04-24T23:09:01.427724+0000","last_became_active":"2024-04-24T23:09:01.426576+0000","last_became_peered":"2024-04-24T23:09:01.426576+0000","last_unstale":"2024-04-24T23:09:01.427724+0000","last_undegraded":"2024-04-24T23:09:01.427724+0000","last_fullsized":"2024-04-24T23:09:01.427724+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:38:37.910180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.420741+0000","last_change":"2024-04-24T23:09:00.402438+0000","last_active":"2024-04-24T23:09:01.420741+0000","last_peered":"2024-04-24T23:09:01.420741+0000","last_clean":"2024-04-24T23:09:01.420741+0000","last_became_active":"2024-04-24T23:09:00.402193+0000","last_became_peered":"2024-04-24T23:09:00.402193+0000","last_unstale":"2024-04-24T23:09:01.420741+0000","last_undegraded":"2024-04-24T23:09:01.420741+0000","last_fullsized":"2024-04-24T23:09:01.420741+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:50:51.640234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.423037+0000","last_change":"2024-04-24T23:09:01.422295+0000","last_active":"2024-04-24T23:09:01.423037+0000","last_peered":"2024-04-24T23:09:01.423037+0000","last_clean":"2024-04-24T23:09:01.423037+0000","last_became_active":"2024-04-24T23:09:01.422069+0000","last_became_peered":"2024-04-24T23:09:01.422069+0000","last_unstale":"2024-04-24T23:09:01.423037+0000","last_undegraded":"2024-04-24T23:09:01.423037+0000","last_fullsized":"2024-04-24T23:09:01.423037+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:19:55.306564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.14","version":"168'6","reported_seq":18,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.438229+0000","last_change":"2024-04-24T23:09:00.404740+0000","last_active":"2024-04-24T23:09:01.438229+0000","last_peered":"2024-04-24T23:09:01.438229+0000","last_clean":"2024-04-24T23:09:01.438229+0000","last_became_active":"2024-04-24T23:09:00.404493+0000","last_became_peered":"2024-04-24T23:09:00.404493+0000","last_unstale":"2024-04-24T23:09:01.438229+0000","last_undegraded":"2024-04-24T23:09:01.438229+0000","last_fullsized":"2024-04-24T23:09:01.438229+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:41.386975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.429299+0000","last_change":"2024-04-24T23:09:01.429172+0000","last_active":"2024-04-24T23:09:01.429299+0000","last_peered":"2024-04-24T23:09:01.429299+0000","last_clean":"2024-04-24T23:09:01.429299+0000","last_became_active":"2024-04-24T23:09:01.428935+0000","last_became_peered":"2024-04-24T23:09:01.428935+0000","last_unstale":"2024-04-24T23:09:01.429299+0000","last_undegraded":"2024-04-24T23:09:01.429299+0000","last_fullsized":"2024-04-24T23:09:01.429299+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:38:07.310274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.13","version":"168'2","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.438486+0000","last_change":"2024-04-24T23:09:00.406010+0000","last_active":"2024-04-24T23:09:01.438486+0000","last_peered":"2024-04-24T23:09:01.438486+0000","last_clean":"2024-04-24T23:09:01.438486+0000","last_became_active":"2024-04-24T23:09:00.405708+0000","last_became_peered":"2024-04-24T23:09:00.405708+0000","last_unstale":"2024-04-24T23:09:01.438486+0000","last_undegraded":"2024-04-24T23:09:01.438486+0000","last_fullsized":"2024-04-24T23:09:01.438486+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:25:25.564334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.431023+0000","last_change":"2024-04-24T23:09:01.430886+0000","last_active":"2024-04-24T23:09:01.431023+0000","last_peered":"2024-04-24T23:09:01.431023+0000","last_clean":"2024-04-24T23:09:01.431023+0000","last_became_active":"2024-04-24T23:09:01.430627+0000","last_became_peered":"2024-04-24T23:09:01.430627+0000","last_unstale":"2024-04-24T23:09:01.431023+0000","last_undegraded":"2024-04-24T23:09:01.431023+0000","last_fullsized":"2024-04-24T23:09:01.431023+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:26:23.052957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.424151+0000","last_change":"2024-04-24T23:09:00.400745+0000","last_active":"2024-04-24T23:09:01.424151+0000","last_peered":"2024-04-24T23:09:01.424151+0000","last_clean":"2024-04-24T23:09:01.424151+0000","last_became_active":"2024-04-24T23:09:00.400503+0000","last_became_peered":"2024-04-24T23:09:00.400503+0000","last_unstale":"2024-04-24T23:09:01.424151+0000","last_undegraded":"2024-04-24T23:09:01.424151+0000","last_fullsized":"2024-04-24T23:09:01.424151+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:30.563473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.421132+0000","last_change":"2024-04-24T23:09:01.420868+0000","last_active":"2024-04-24T23:09:01.421132+0000","last_peered":"2024-04-24T23:09:01.421132+0000","last_clean":"2024-04-24T23:09:01.421132+0000","last_became_active":"2024-04-24T23:09:01.420382+0000","last_became_peered":"2024-04-24T23:09:01.420382+0000","last_unstale":"2024-04-24T23:09:01.421132+0000","last_undegraded":"2024-04-24T23:09:01.421132+0000","last_fullsized":"2024-04-24T23:09:01.421132+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:38:18.189981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.421445+0000","last_change":"2024-04-24T23:09:00.406688+0000","last_active":"2024-04-24T23:09:01.421445+0000","last_peered":"2024-04-24T23:09:01.421445+0000","last_clean":"2024-04-24T23:09:01.421445+0000","last_became_active":"2024-04-24T23:09:00.405818+0000","last_became_peered":"2024-04-24T23:09:00.405818+0000","last_unstale":"2024-04-24T23:09:01.421445+0000","last_undegraded":"2024-04-24T23:09:01.421445+0000","last_fullsized":"2024-04-24T23:09:01.421445+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:46:10.288129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.419378+0000","last_change":"2024-04-24T23:09:01.419251+0000","last_active":"2024-04-24T23:09:01.419378+0000","last_peered":"2024-04-24T23:09:01.419378+0000","last_clean":"2024-04-24T23:09:01.419378+0000","last_became_active":"2024-04-24T23:09:01.418645+0000","last_became_peered":"2024-04-24T23:09:01.418645+0000","last_unstale":"2024-04-24T23:09:01.419378+0000","last_undegraded":"2024-04-24T23:09:01.419378+0000","last_fullsized":"2024-04-24T23:09:01.419378+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:15.065149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.10","version":"168'8","reported_seq":20,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.439093+0000","last_change":"2024-04-24T23:09:00.408729+0000","last_active":"2024-04-24T23:09:01.439093+0000","last_peered":"2024-04-24T23:09:01.439093+0000","last_clean":"2024-04-24T23:09:01.439093+0000","last_became_active":"2024-04-24T23:09:00.408474+0000","last_became_peered":"2024-04-24T23:09:00.408474+0000","last_unstale":"2024-04-24T23:09:01.439093+0000","last_undegraded":"2024-04-24T23:09:01.439093+0000","last_fullsized":"2024-04-24T23:09:01.439093+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:40:05.356173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.431412+0000","last_change":"2024-04-24T23:09:01.431281+0000","last_active":"2024-04-24T23:09:01.431412+0000","last_peered":"2024-04-24T23:09:01.431412+0000","last_clean":"2024-04-24T23:09:01.431412+0000","last_became_active":"2024-04-24T23:09:01.431042+0000","last_became_peered":"2024-04-24T23:09:01.431042+0000","last_unstale":"2024-04-24T23:09:01.431412+0000","last_undegraded":"2024-04-24T23:09:01.431412+0000","last_fullsized":"2024-04-24T23:09:01.431412+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:58.596772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.f","version":"168'1","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.434992+0000","last_change":"2024-04-24T23:09:00.409619+0000","last_active":"2024-04-24T23:09:01.434992+0000","last_peered":"2024-04-24T23:09:01.434992+0000","last_clean":"2024-04-24T23:09:01.434992+0000","last_became_active":"2024-04-24T23:09:00.409047+0000","last_became_peered":"2024-04-24T23:09:00.409047+0000","last_unstale":"2024-04-24T23:09:01.434992+0000","last_undegraded":"2024-04-24T23:09:01.434992+0000","last_fullsized":"2024-04-24T23:09:01.434992+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:14:10.744522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.424958+0000","last_change":"2024-04-24T23:09:01.424823+0000","last_active":"2024-04-24T23:09:01.424958+0000","last_peered":"2024-04-24T23:09:01.424958+0000","last_clean":"2024-04-24T23:09:01.424958+0000","last_became_active":"2024-04-24T23:09:01.424573+0000","last_became_peered":"2024-04-24T23:09:01.424573+0000","last_unstale":"2024-04-24T23:09:01.424958+0000","last_undegraded":"2024-04-24T23:09:01.424958+0000","last_fullsized":"2024-04-24T23:09:01.424958+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:27:32.667126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.e","version":"168'2","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.436523+0000","last_change":"2024-04-24T23:09:00.408456+0000","last_active":"2024-04-24T23:09:01.436523+0000","last_peered":"2024-04-24T23:09:01.436523+0000","last_clean":"2024-04-24T23:09:01.436523+0000","last_became_active":"2024-04-24T23:09:00.408200+0000","last_became_peered":"2024-04-24T23:09:00.408200+0000","last_unstale":"2024-04-24T23:09:01.436523+0000","last_undegraded":"2024-04-24T23:09:01.436523+0000","last_fullsized":"2024-04-24T23:09:01.436523+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:09:27.358487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.428063+0000","last_change":"2024-04-24T23:09:01.426543+0000","last_active":"2024-04-24T23:09:01.428063+0000","last_peered":"2024-04-24T23:09:01.428063+0000","last_clean":"2024-04-24T23:09:01.428063+0000","last_became_active":"2024-04-24T23:09:01.426315+0000","last_became_peered":"2024-04-24T23:09:01.426315+0000","last_unstale":"2024-04-24T23:09:01.428063+0000","last_undegraded":"2024-04-24T23:09:01.428063+0000","last_fullsized":"2024-04-24T23:09:01.428063+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:56.311762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.d","version":"168'1","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.433839+0000","last_change":"2024-04-24T23:09:00.403417+0000","last_active":"2024-04-24T23:09:01.433839+0000","last_peered":"2024-04-24T23:09:01.433839+0000","last_clean":"2024-04-24T23:09:01.433839+0000","last_became_active":"2024-04-24T23:09:00.403135+0000","last_became_peered":"2024-04-24T23:09:00.403135+0000","last_unstale":"2024-04-24T23:09:01.433839+0000","last_undegraded":"2024-04-24T23:09:01.433839+0000","last_fullsized":"2024-04-24T23:09:01.433839+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:28:35.033779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.426415+0000","last_change":"2024-04-24T23:09:01.426223+0000","last_active":"2024-04-24T23:09:01.426415+0000","last_peered":"2024-04-24T23:09:01.426415+0000","last_clean":"2024-04-24T23:09:01.426415+0000","last_became_active":"2024-04-24T23:09:01.425413+0000","last_became_peered":"2024-04-24T23:09:01.425413+0000","last_unstale":"2024-04-24T23:09:01.426415+0000","last_undegraded":"2024-04-24T23:09:01.426415+0000","last_fullsized":"2024-04-24T23:09:01.426415+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:50:39.561897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.421580+0000","last_change":"2024-04-24T23:09:00.406318+0000","last_active":"2024-04-24T23:09:01.421580+0000","last_peered":"2024-04-24T23:09:01.421580+0000","last_clean":"2024-04-24T23:09:01.421580+0000","last_became_active":"2024-04-24T23:09:00.405447+0000","last_became_peered":"2024-04-24T23:09:00.405447+0000","last_unstale":"2024-04-24T23:09:01.421580+0000","last_undegraded":"2024-04-24T23:09:01.421580+0000","last_fullsized":"2024-04-24T23:09:01.421580+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:08:41.531441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.429038+0000","last_change":"2024-04-24T23:09:01.428925+0000","last_active":"2024-04-24T23:09:01.429038+0000","last_peered":"2024-04-24T23:09:01.429038+0000","last_clean":"2024-04-24T23:09:01.429038+0000","last_became_active":"2024-04-24T23:09:01.428358+0000","last_became_peered":"2024-04-24T23:09:01.428358+0000","last_unstale":"2024-04-24T23:09:01.429038+0000","last_undegraded":"2024-04-24T23:09:01.429038+0000","last_fullsized":"2024-04-24T23:09:01.429038+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:34:07.860555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.b","version":"168'2","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.437970+0000","last_change":"2024-04-24T23:09:00.398550+0000","last_active":"2024-04-24T23:09:01.437970+0000","last_peered":"2024-04-24T23:09:01.437970+0000","last_clean":"2024-04-24T23:09:01.437970+0000","last_became_active":"2024-04-24T23:09:00.398273+0000","last_became_peered":"2024-04-24T23:09:00.398273+0000","last_unstale":"2024-04-24T23:09:01.437970+0000","last_undegraded":"2024-04-24T23:09:01.437970+0000","last_fullsized":"2024-04-24T23:09:01.437970+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:03.390742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.425359+0000","last_change":"2024-04-24T23:09:01.425031+0000","last_active":"2024-04-24T23:09:01.425359+0000","last_peered":"2024-04-24T23:09:01.425359+0000","last_clean":"2024-04-24T23:09:01.425359+0000","last_became_active":"2024-04-24T23:09:01.424613+0000","last_became_peered":"2024-04-24T23:09:01.424613+0000","last_unstale":"2024-04-24T23:09:01.425359+0000","last_undegraded":"2024-04-24T23:09:01.425359+0000","last_fullsized":"2024-04-24T23:09:01.425359+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:50:14.494486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.416701+0000","last_change":"2024-04-24T23:09:00.410496+0000","last_active":"2024-04-24T23:09:01.416701+0000","last_peered":"2024-04-24T23:09:01.416701+0000","last_clean":"2024-04-24T23:09:01.416701+0000","last_became_active":"2024-04-24T23:09:00.410322+0000","last_became_peered":"2024-04-24T23:09:00.410322+0000","last_unstale":"2024-04-24T23:09:01.416701+0000","last_undegraded":"2024-04-24T23:09:01.416701+0000","last_fullsized":"2024-04-24T23:09:01.416701+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:56:51.579136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.428633+0000","last_change":"2024-04-24T23:09:01.428508+0000","last_active":"2024-04-24T23:09:01.428633+0000","last_peered":"2024-04-24T23:09:01.428633+0000","last_clean":"2024-04-24T23:09:01.428633+0000","last_became_active":"2024-04-24T23:09:01.428109+0000","last_became_peered":"2024-04-24T23:09:01.428109+0000","last_unstale":"2024-04-24T23:09:01.428633+0000","last_undegraded":"2024-04-24T23:09:01.428633+0000","last_fullsized":"2024-04-24T23:09:01.428633+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:10.182210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.415871+0000","last_change":"2024-04-24T23:09:00.407554+0000","last_active":"2024-04-24T23:09:01.415871+0000","last_peered":"2024-04-24T23:09:01.415871+0000","last_clean":"2024-04-24T23:09:01.415871+0000","last_became_active":"2024-04-24T23:09:00.407275+0000","last_became_peered":"2024-04-24T23:09:00.407275+0000","last_unstale":"2024-04-24T23:09:01.415871+0000","last_undegraded":"2024-04-24T23:09:01.415871+0000","last_fullsized":"2024-04-24T23:09:01.415871+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:12:26.213125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.425324+0000","last_change":"2024-04-24T23:09:01.424915+0000","last_active":"2024-04-24T23:09:01.425324+0000","last_peered":"2024-04-24T23:09:01.425324+0000","last_clean":"2024-04-24T23:09:01.425324+0000","last_became_active":"2024-04-24T23:09:01.424582+0000","last_became_peered":"2024-04-24T23:09:01.424582+0000","last_unstale":"2024-04-24T23:09:01.425324+0000","last_undegraded":"2024-04-24T23:09:01.425324+0000","last_fullsized":"2024-04-24T23:09:01.425324+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:21:28.971678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.416637+0000","last_change":"2024-04-24T23:09:00.409795+0000","last_active":"2024-04-24T23:09:01.416637+0000","last_peered":"2024-04-24T23:09:01.416637+0000","last_clean":"2024-04-24T23:09:01.416637+0000","last_became_active":"2024-04-24T23:09:00.409528+0000","last_became_peered":"2024-04-24T23:09:00.409528+0000","last_unstale":"2024-04-24T23:09:01.416637+0000","last_undegraded":"2024-04-24T23:09:01.416637+0000","last_fullsized":"2024-04-24T23:09:01.416637+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:02:00.452829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.420895+0000","last_change":"2024-04-24T23:09:01.420665+0000","last_active":"2024-04-24T23:09:01.420895+0000","last_peered":"2024-04-24T23:09:01.420895+0000","last_clean":"2024-04-24T23:09:01.420895+0000","last_became_active":"2024-04-24T23:09:01.419499+0000","last_became_peered":"2024-04-24T23:09:01.419499+0000","last_unstale":"2024-04-24T23:09:01.420895+0000","last_undegraded":"2024-04-24T23:09:01.420895+0000","last_fullsized":"2024-04-24T23:09:01.420895+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:01:43.519392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.7","version":"168'2","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.436788+0000","last_change":"2024-04-24T23:09:00.409447+0000","last_active":"2024-04-24T23:09:01.436788+0000","last_peered":"2024-04-24T23:09:01.436788+0000","last_clean":"2024-04-24T23:09:01.436788+0000","last_became_active":"2024-04-24T23:09:00.409143+0000","last_became_peered":"2024-04-24T23:09:00.409143+0000","last_unstale":"2024-04-24T23:09:01.436788+0000","last_undegraded":"2024-04-24T23:09:01.436788+0000","last_fullsized":"2024-04-24T23:09:01.436788+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:13:16.773673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.418065+0000","last_change":"2024-04-24T23:09:01.417943+0000","last_active":"2024-04-24T23:09:01.418065+0000","last_peered":"2024-04-24T23:09:01.418065+0000","last_clean":"2024-04-24T23:09:01.418065+0000","last_became_active":"2024-04-24T23:09:01.417682+0000","last_became_peered":"2024-04-24T23:09:01.417682+0000","last_unstale":"2024-04-24T23:09:01.418065+0000","last_undegraded":"2024-04-24T23:09:01.418065+0000","last_fullsized":"2024-04-24T23:09:01.418065+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:06:07.547684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.421222+0000","last_change":"2024-04-24T23:09:00.407797+0000","last_active":"2024-04-24T23:09:01.421222+0000","last_peered":"2024-04-24T23:09:01.421222+0000","last_clean":"2024-04-24T23:09:01.421222+0000","last_became_active":"2024-04-24T23:09:00.407106+0000","last_became_peered":"2024-04-24T23:09:00.407106+0000","last_unstale":"2024-04-24T23:09:01.421222+0000","last_undegraded":"2024-04-24T23:09:01.421222+0000","last_fullsized":"2024-04-24T23:09:01.421222+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:20:05.863066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.424007+0000","last_change":"2024-04-24T23:09:01.422935+0000","last_active":"2024-04-24T23:09:01.424007+0000","last_peered":"2024-04-24T23:09:01.424007+0000","last_clean":"2024-04-24T23:09:01.424007+0000","last_became_active":"2024-04-24T23:09:01.422640+0000","last_became_peered":"2024-04-24T23:09:01.422640+0000","last_unstale":"2024-04-24T23:09:01.424007+0000","last_undegraded":"2024-04-24T23:09:01.424007+0000","last_fullsized":"2024-04-24T23:09:01.424007+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:36:13.565647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.415428+0000","last_change":"2024-04-24T23:09:00.405664+0000","last_active":"2024-04-24T23:09:01.415428+0000","last_peered":"2024-04-24T23:09:01.415428+0000","last_clean":"2024-04-24T23:09:01.415428+0000","last_became_active":"2024-04-24T23:09:00.405425+0000","last_became_peered":"2024-04-24T23:09:00.405425+0000","last_unstale":"2024-04-24T23:09:01.415428+0000","last_undegraded":"2024-04-24T23:09:01.415428+0000","last_fullsized":"2024-04-24T23:09:01.415428+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:25:23.902454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.427147+0000","last_change":"2024-04-24T23:09:01.427033+0000","last_active":"2024-04-24T23:09:01.427147+0000","last_peered":"2024-04-24T23:09:01.427147+0000","last_clean":"2024-04-24T23:09:01.427147+0000","last_became_active":"2024-04-24T23:09:01.426729+0000","last_became_peered":"2024-04-24T23:09:01.426729+0000","last_unstale":"2024-04-24T23:09:01.427147+0000","last_undegraded":"2024-04-24T23:09:01.427147+0000","last_fullsized":"2024-04-24T23:09:01.427147+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:49:37.654240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.416082+0000","last_change":"2024-04-24T23:09:00.407757+0000","last_active":"2024-04-24T23:09:01.416082+0000","last_peered":"2024-04-24T23:09:01.416082+0000","last_clean":"2024-04-24T23:09:01.416082+0000","last_became_active":"2024-04-24T23:09:00.407438+0000","last_became_peered":"2024-04-24T23:09:00.407438+0000","last_unstale":"2024-04-24T23:09:01.416082+0000","last_undegraded":"2024-04-24T23:09:01.416082+0000","last_fullsized":"2024-04-24T23:09:01.416082+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:39:53.729104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.427523+0000","last_change":"2024-04-24T23:09:01.427358+0000","last_active":"2024-04-24T23:09:01.427523+0000","last_peered":"2024-04-24T23:09:01.427523+0000","last_clean":"2024-04-24T23:09:01.427523+0000","last_became_active":"2024-04-24T23:09:01.427120+0000","last_became_peered":"2024-04-24T23:09:01.427120+0000","last_unstale":"2024-04-24T23:09:01.427523+0000","last_undegraded":"2024-04-24T23:09:01.427523+0000","last_fullsized":"2024-04-24T23:09:01.427523+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:58:46.154130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.419889+0000","last_change":"2024-04-24T23:09:00.408417+0000","last_active":"2024-04-24T23:09:01.419889+0000","last_peered":"2024-04-24T23:09:01.419889+0000","last_clean":"2024-04-24T23:09:01.419889+0000","last_became_active":"2024-04-24T23:09:00.408152+0000","last_became_peered":"2024-04-24T23:09:00.408152+0000","last_unstale":"2024-04-24T23:09:01.419889+0000","last_undegraded":"2024-04-24T23:09:01.419889+0000","last_fullsized":"2024-04-24T23:09:01.419889+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:11:05.331214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.420061+0000","last_change":"2024-04-24T23:09:01.419933+0000","last_active":"2024-04-24T23:09:01.420061+0000","last_peered":"2024-04-24T23:09:01.420061+0000","last_clean":"2024-04-24T23:09:01.420061+0000","last_became_active":"2024-04-24T23:09:01.419679+0000","last_became_peered":"2024-04-24T23:09:01.419679+0000","last_unstale":"2024-04-24T23:09:01.420061+0000","last_undegraded":"2024-04-24T23:09:01.420061+0000","last_fullsized":"2024-04-24T23:09:01.420061+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:35:02.456274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.412395+0000","last_change":"2024-04-24T23:09:00.404758+0000","last_active":"2024-04-24T23:09:01.412395+0000","last_peered":"2024-04-24T23:09:01.412395+0000","last_clean":"2024-04-24T23:09:01.412395+0000","last_became_active":"2024-04-24T23:09:00.404524+0000","last_became_peered":"2024-04-24T23:09:00.404524+0000","last_unstale":"2024-04-24T23:09:01.412395+0000","last_undegraded":"2024-04-24T23:09:01.412395+0000","last_fullsized":"2024-04-24T23:09:01.412395+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:01:38.230637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.430764+0000","last_change":"2024-04-24T23:09:01.430622+0000","last_active":"2024-04-24T23:09:01.430764+0000","last_peered":"2024-04-24T23:09:01.430764+0000","last_clean":"2024-04-24T23:09:01.430764+0000","last_became_active":"2024-04-24T23:09:01.429682+0000","last_became_peered":"2024-04-24T23:09:01.429682+0000","last_unstale":"2024-04-24T23:09:01.430764+0000","last_undegraded":"2024-04-24T23:09:01.430764+0000","last_fullsized":"2024-04-24T23:09:01.430764+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:05:30.385966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.421681+0000","last_change":"2024-04-24T23:09:00.401728+0000","last_active":"2024-04-24T23:09:01.421681+0000","last_peered":"2024-04-24T23:09:01.421681+0000","last_clean":"2024-04-24T23:09:01.421681+0000","last_became_active":"2024-04-24T23:09:00.401483+0000","last_became_peered":"2024-04-24T23:09:00.401483+0000","last_unstale":"2024-04-24T23:09:01.421681+0000","last_undegraded":"2024-04-24T23:09:01.421681+0000","last_fullsized":"2024-04-24T23:09:01.421681+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:55:36.056034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.428674+0000","last_change":"2024-04-24T23:09:01.428553+0000","last_active":"2024-04-24T23:09:01.428674+0000","last_peered":"2024-04-24T23:09:01.428674+0000","last_clean":"2024-04-24T23:09:01.428674+0000","last_became_active":"2024-04-24T23:09:01.428317+0000","last_became_peered":"2024-04-24T23:09:01.428317+0000","last_unstale":"2024-04-24T23:09:01.428674+0000","last_undegraded":"2024-04-24T23:09:01.428674+0000","last_fullsized":"2024-04-24T23:09:01.428674+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:19:37.280708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.416802+0000","last_change":"2024-04-24T23:09:00.403098+0000","last_active":"2024-04-24T23:09:01.416802+0000","last_peered":"2024-04-24T23:09:01.416802+0000","last_clean":"2024-04-24T23:09:01.416802+0000","last_became_active":"2024-04-24T23:09:00.402604+0000","last_became_peered":"2024-04-24T23:09:00.402604+0000","last_unstale":"2024-04-24T23:09:01.416802+0000","last_undegraded":"2024-04-24T23:09:01.416802+0000","last_fullsized":"2024-04-24T23:09:01.416802+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:49:35.707261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.418373+0000","last_change":"2024-04-24T23:09:01.417867+0000","last_active":"2024-04-24T23:09:01.418373+0000","last_peered":"2024-04-24T23:09:01.418373+0000","last_clean":"2024-04-24T23:09:01.418373+0000","last_became_active":"2024-04-24T23:09:01.417624+0000","last_became_peered":"2024-04-24T23:09:01.417624+0000","last_unstale":"2024-04-24T23:09:01.418373+0000","last_undegraded":"2024-04-24T23:09:01.418373+0000","last_fullsized":"2024-04-24T23:09:01.418373+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:09:00.392815+0000","last_clean_scrub_stamp":"2024-04-24T23:09:00.392815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:01.620061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.15","version":"168'3","reported_seq":15,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.437710+0000","last_change":"2024-04-24T23:09:00.402348+0000","last_active":"2024-04-24T23:09:01.437710+0000","last_peered":"2024-04-24T23:09:01.437710+0000","last_clean":"2024-04-24T23:09:01.437710+0000","last_became_active":"2024-04-24T23:09:00.402101+0000","last_became_peered":"2024-04-24T23:09:00.402101+0000","last_unstale":"2024-04-24T23:09:01.437710+0000","last_undegraded":"2024-04-24T23:09:01.437710+0000","last_fullsized":"2024-04-24T23:09:01.437710+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:59.388950+0000","last_clean_scrub_stamp":"2024-04-24T23:08:59.388950+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:54:29.329728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":462,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-24T23:09:01.421351+0000","last_change":"2024-04-24T22:58:55.138810+0000","last_active":"2024-04-24T23:09:01.421351+0000","last_peered":"2024-04-24T23:09:01.421351+0000","last_clean":"2024-04-24T23:09:01.421351+0000","last_became_active":"2024-04-24T22:58:55.138627+0000","last_became_peered":"2024-04-24T22:58:55.138627+0000","last_unstale":"2024-04-24T23:09:01.421351+0000","last_undegraded":"2024-04-24T23:09:01.421351+0000","last_fullsized":"2024-04-24T23:09:01.421351+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_clean_scrub_stamp":"2024-04-24T22:58:54.127235+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:51.383182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574972,"num_pgs":16,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":913,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194398,"statfs":{"total":200000000,"available":199064241,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574972,"num_pgs":12,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":920,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194392,"statfs":{"total":200000000,"available":199057466,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574972,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1498,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193813,"statfs":{"total":200000000,"available":198465292,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574972,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":922,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194389,"statfs":{"total":200000000,"available":199055250,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574972,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":921,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194391,"statfs":{"total":200000000,"available":199056850,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574972,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":920,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194392,"statfs":{"total":200000000,"available":199057444,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574972,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1497,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193815,"statfs":{"total":200000000,"available":198467028,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574972,"num_pgs":12,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":920,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194391,"statfs":{"total":200000000,"available":199057376,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-24T23:09:05.778 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-24T23:09:05.778 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-24T23:09:05.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.921+0000 7fa018da5640 1 Processor -- start 2024-04-24T23:09:05.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.921+0000 7fa018da5640 1 -- start start 2024-04-24T23:09:05.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.922+0000 7fa018da5640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa014158dd0 0x7fa01406b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:05.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.922+0000 7fa018da5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa01406b860 0x7fa01406d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:05.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.922+0000 7fa018da5640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0140b6d10 con 0x7fa01406da90 2024-04-24T23:09:05.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.922+0000 7fa018da5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fa01406bd10 con 0x7fa01406b860 2024-04-24T23:09:05.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.922+0000 7fa018da5640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fa01415ef10 con 0x7fa014158dd0 2024-04-24T23:09:05.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.922+0000 7fa011d9b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa01406b860 0x7fa01406d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:05.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.922+0000 7fa011d9b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa01406b860 0x7fa01406d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40272/0 (socket says 172.21.15.32:40272) 2024-04-24T23:09:05.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.922+0000 7fa011d9b640 1 -- 172.21.15.32:0/1504171690 learned_addr learned my addr 172.21.15.32:0/1504171690 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:09:05.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.922+0000 7fa01259c640 1 --2- 172.21.15.32:0/1504171690 >> v2:172.21.15.121:3300/0 conn(0x7fa014158dd0 0x7fa01406b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:05.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.922+0000 7fa011d9b640 1 -- 172.21.15.32:0/1504171690 >> v2:172.21.15.121:3300/0 conn(0x7fa014158dd0 msgr2=0x7fa01406b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:09:05.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.922+0000 7fa011d9b640 1 --2- 172.21.15.32:0/1504171690 >> v2:172.21.15.121:3300/0 conn(0x7fa014158dd0 0x7fa01406b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:05.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.922+0000 7fa011d9b640 1 -- 172.21.15.32:0/1504171690 >> v1:172.21.15.121:6789/0 conn(0x7fa01406da90 legacy=0x7fa014072580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:05.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.922+0000 7fa011d9b640 1 -- 172.21.15.32:0/1504171690 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa01415f1f0 con 0x7fa01406b860 2024-04-24T23:09:05.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.922+0000 7fa01259c640 1 --2- 172.21.15.32:0/1504171690 >> v2:172.21.15.121:3300/0 conn(0x7fa014158dd0 0x7fa01406b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:09:05.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.923+0000 7fa011d9b640 1 --2- 172.21.15.32:0/1504171690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa01406b860 0x7fa01406d550 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fa01406f3a0 tx=0x7fa00802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4adfc57726bc4ccc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:05.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.923+0000 7fa01159a640 1 -- 172.21.15.32:0/1504171690 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa008037070 con 0x7fa01406b860 2024-04-24T23:09:05.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.923+0000 7fa01159a640 1 -- 172.21.15.32:0/1504171690 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa008032440 con 0x7fa01406b860 2024-04-24T23:09:05.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.923+0000 7fa01159a640 1 -- 172.21.15.32:0/1504171690 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa008032810 con 0x7fa01406b860 2024-04-24T23:09:05.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.924+0000 7fa018da5640 1 -- 172.21.15.32:0/1504171690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa01406b860 msgr2=0x7fa01406d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:05.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.924+0000 7fa018da5640 1 --2- 172.21.15.32:0/1504171690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa01406b860 0x7fa01406d550 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fa01406f3a0 tx=0x7fa00802a270 comp rx=0 tx=0).stop 2024-04-24T23:09:05.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.924+0000 7fa018da5640 1 -- 172.21.15.32:0/1504171690 shutdown_connections 2024-04-24T23:09:05.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.924+0000 7fa018da5640 1 --2- 172.21.15.32:0/1504171690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa01406b860 0x7fa01406d550 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:05.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.924+0000 7fa018da5640 1 --2- 172.21.15.32:0/1504171690 >> v2:172.21.15.121:3300/0 conn(0x7fa014158dd0 0x7fa01406b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:05.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.924+0000 7fa018da5640 1 -- 172.21.15.32:0/1504171690 >> 172.21.15.32:0/1504171690 conn(0x7fa01406e800 msgr2=0x7fa01406ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:05.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.924+0000 7fa018da5640 1 -- 172.21.15.32:0/1504171690 shutdown_connections 2024-04-24T23:09:05.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.924+0000 7fa018da5640 1 -- 172.21.15.32:0/1504171690 wait complete. 2024-04-24T23:09:05.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.925+0000 7fa018da5640 1 Processor -- start 2024-04-24T23:09:05.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.925+0000 7fa018da5640 1 -- start start 2024-04-24T23:09:05.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.926+0000 7fa018da5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa01406b860 0x7fa014153980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:05.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.926+0000 7fa018da5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa01406da90 0x7fa0140b4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:05.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.926+0000 7fa018da5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa014158dd0 0x7fa014141a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:05.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.926+0000 7fa018da5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa01415fcd0 con 0x7fa01406da90 2024-04-24T23:09:05.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.926+0000 7fa018da5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa01415f7b0 con 0x7fa014158dd0 2024-04-24T23:09:05.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.926+0000 7fa018da5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa01406ded0 con 0x7fa01406b860 2024-04-24T23:09:05.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.926+0000 7fa011d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa01406da90 0x7fa0140b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:05.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.926+0000 7fa011d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa01406da90 0x7fa0140b4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40276/0 (socket says 172.21.15.32:40276) 2024-04-24T23:09:05.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.926+0000 7fa011d9b640 1 -- 172.21.15.32:0/4086756780 learned_addr learned my addr 172.21.15.32:0/4086756780 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:09:05.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.926+0000 7fa01259c640 1 --2- 172.21.15.32:0/4086756780 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa01406b860 0x7fa014153980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:05.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.926+0000 7fa012d9d640 1 --2- 172.21.15.32:0/4086756780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa014158dd0 0x7fa014141a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:05.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.926+0000 7fa011d9b640 1 -- 172.21.15.32:0/4086756780 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa01406b860 msgr2=0x7fa014153980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:05.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.926+0000 7fa011d9b640 1 --2- 172.21.15.32:0/4086756780 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa01406b860 0x7fa014153980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:05.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.926+0000 7fa011d9b640 1 -- 172.21.15.32:0/4086756780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa014158dd0 msgr2=0x7fa014141a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:05.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.926+0000 7fa011d9b640 1 --2- 172.21.15.32:0/4086756780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa014158dd0 0x7fa014141a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:05.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.926+0000 7fa011d9b640 1 -- 172.21.15.32:0/4086756780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0141f8880 con 0x7fa01406da90 2024-04-24T23:09:05.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.926+0000 7fa012d9d640 1 --2- 172.21.15.32:0/4086756780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa014158dd0 0x7fa014141a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:09:05.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.927+0000 7fa011d9b640 1 --2- 172.21.15.32:0/4086756780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa01406da90 0x7fa0140b4860 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fa00802a2b0 tx=0x7fa0080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:05.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.927+0000 7fa002ffd640 1 -- 172.21.15.32:0/4086756780 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa008041020 con 0x7fa01406da90 2024-04-24T23:09:05.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.927+0000 7fa002ffd640 1 -- 172.21.15.32:0/4086756780 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0080306c0 con 0x7fa01406da90 2024-04-24T23:09:05.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.927+0000 7fa018da5640 1 -- 172.21.15.32:0/4086756780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0141509b0 con 0x7fa01406da90 2024-04-24T23:09:05.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.927+0000 7fa002ffd640 1 -- 172.21.15.32:0/4086756780 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa008037040 con 0x7fa01406da90 2024-04-24T23:09:05.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.927+0000 7fa018da5640 1 -- 172.21.15.32:0/4086756780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa014150f70 con 0x7fa01406da90 2024-04-24T23:09:05.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.928+0000 7fa018da5640 1 -- 172.21.15.32:0/4086756780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa014068430 con 0x7fa01406da90 2024-04-24T23:09:05.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.928+0000 7fa002ffd640 1 -- 172.21.15.32:0/4086756780 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa008044020 con 0x7fa01406da90 2024-04-24T23:09:05.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.929+0000 7fa002ffd640 1 --2- 172.21.15.32:0/4086756780 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9ff406be60 0x7f9ff406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:05.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.929+0000 7fa002ffd640 1 -- 172.21.15.32:0/4086756780 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 5709+0+0 (secure 0 0 0) 0x7fa0080ab5f0 con 0x7fa01406da90 2024-04-24T23:09:05.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.929+0000 7fa01259c640 1 --2- 172.21.15.32:0/4086756780 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9ff406be60 0x7f9ff406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:05.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.930+0000 7fa01259c640 1 --2- 172.21.15.32:0/4086756780 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9ff406be60 0x7f9ff406e320 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fa00c006fd0 tx=0x7fa00c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:05.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:05.933+0000 7fa002ffd640 1 -- 172.21.15.32:0/4086756780 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa00804b030 con 0x7fa01406da90 2024-04-24T23:09:06.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.184+0000 7fa018da5640 1 -- 172.21.15.32:0/4086756780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fa014074e60 con 0x7fa01406da90 2024-04-24T23:09:06.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.187+0000 7fa002ffd640 1 -- 172.21.15.32:0/4086756780 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v22) ==== 190+0+147 (secure 0 0 0) 0x7fa008079fa0 con 0x7fa01406da90 2024-04-24T23:09:06.188 INFO:teuthology.orchestra.run.smithi032.stdout:[client.0] 2024-04-24T23:09:06.188 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQCejilm5oeiGRAAdLXwwqrYhV3HHo7Dw/2ZqQ== 2024-04-24T23:09:06.188 INFO:teuthology.orchestra.run.smithi032.stdout: caps mds = "allow" 2024-04-24T23:09:06.188 INFO:teuthology.orchestra.run.smithi032.stdout: caps mon = "allow r" 2024-04-24T23:09:06.189 INFO:teuthology.orchestra.run.smithi032.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-24T23:09:06.189 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2024-04-24T23:09:06.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.189+0000 7fa018da5640 1 -- 172.21.15.32:0/4086756780 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9ff406be60 msgr2=0x7f9ff406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:06.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.189+0000 7fa018da5640 1 --2- 172.21.15.32:0/4086756780 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9ff406be60 0x7f9ff406e320 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fa00c006fd0 tx=0x7fa00c008040 comp rx=0 tx=0).stop 2024-04-24T23:09:06.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.189+0000 7fa018da5640 1 -- 172.21.15.32:0/4086756780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa01406da90 msgr2=0x7fa0140b4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:06.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.189+0000 7fa018da5640 1 --2- 172.21.15.32:0/4086756780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa01406da90 0x7fa0140b4860 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fa00802a2b0 tx=0x7fa0080046c0 comp rx=0 tx=0).stop 2024-04-24T23:09:06.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.189+0000 7fa018da5640 1 -- 172.21.15.32:0/4086756780 shutdown_connections 2024-04-24T23:09:06.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.189+0000 7fa018da5640 1 --2- 172.21.15.32:0/4086756780 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9ff406be60 0x7f9ff406e320 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:06.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.189+0000 7fa018da5640 1 --2- 172.21.15.32:0/4086756780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa01406da90 0x7fa0140b4860 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:06.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.189+0000 7fa018da5640 1 --2- 172.21.15.32:0/4086756780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa014158dd0 0x7fa014141a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:06.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.189+0000 7fa018da5640 1 --2- 172.21.15.32:0/4086756780 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa01406b860 0x7fa014153980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:06.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.189+0000 7fa018da5640 1 -- 172.21.15.32:0/4086756780 >> 172.21.15.32:0/4086756780 conn(0x7fa01406e800 msgr2=0x7fa014072480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:06.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.189+0000 7fa018da5640 1 -- 172.21.15.32:0/4086756780 shutdown_connections 2024-04-24T23:09:06.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.189+0000 7fa018da5640 1 -- 172.21.15.32:0/4086756780 wait complete. 2024-04-24T23:09:06.201 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:09:06.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.300+0000 7f639a61c640 1 Processor -- start 2024-04-24T23:09:06.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.300+0000 7f639a61c640 1 -- start start 2024-04-24T23:09:06.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.301+0000 7f639a61c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f639415b480 0x7f63940b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:06.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.301+0000 7f639a61c640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f639406cd00 0x7f63940cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:06.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.301+0000 7f639a61c640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63940b46c0 con 0x7f639415fd60 2024-04-24T23:09:06.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.301+0000 7f639a61c640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f63940b4ff0 con 0x7f639406cd00 2024-04-24T23:09:06.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.301+0000 7f639a61c640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f63940b49a0 con 0x7f639415b480 2024-04-24T23:09:06.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.301+0000 7f63937fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f639415fd60 0x7f639406c5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40634/0 (socket says 172.21.15.32:40634) 2024-04-24T23:09:06.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.301+0000 7f63937fe640 1 -- 172.21.15.32:0/2341948661 learned_addr learned my addr 172.21.15.32:0/2341948661 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:09:06.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.301+0000 7f6393fff640 1 --2- 172.21.15.32:0/2341948661 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f639415b480 0x7f63940b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:06.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.301+0000 7f6398bb9640 1 --2- 172.21.15.32:0/2341948661 >> v2:172.21.15.121:3300/0 conn(0x7f639406cd00 0x7f63940cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:06.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.301+0000 7f6392ffd640 1 -- 172.21.15.32:0/2341948661 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3513515061 0 0) 0x7f63940b46c0 con 0x7f639415fd60 2024-04-24T23:09:06.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.301+0000 7f6392ffd640 1 -- 172.21.15.32:0/2341948661 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f637c005730 con 0x7f639415fd60 2024-04-24T23:09:06.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.302+0000 7f6392ffd640 1 -- 172.21.15.32:0/2341948661 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2204392450 0 0) 0x7f637c005730 con 0x7f639415fd60 2024-04-24T23:09:06.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.302+0000 7f6392ffd640 1 -- 172.21.15.32:0/2341948661 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f639415b480 msgr2=0x7f63940b5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:09:06.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.302+0000 7f6392ffd640 1 --2- 172.21.15.32:0/2341948661 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f639415b480 0x7f63940b5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:06.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.302+0000 7f6392ffd640 1 -- 172.21.15.32:0/2341948661 >> v2:172.21.15.121:3300/0 conn(0x7f639406cd00 msgr2=0x7f63940cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:09:06.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.302+0000 7f6392ffd640 1 --2- 172.21.15.32:0/2341948661 >> v2:172.21.15.121:3300/0 conn(0x7f639406cd00 0x7f63940cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:06.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.302+0000 7f6392ffd640 1 -- 172.21.15.32:0/2341948661 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63940b4c80 con 0x7f639415fd60 2024-04-24T23:09:06.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.302+0000 7f6392ffd640 1 -- 172.21.15.32:0/2341948661 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f6384001670 con 0x7f639415fd60 2024-04-24T23:09:06.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.302+0000 7f6392ffd640 1 -- 172.21.15.32:0/2341948661 >> v1:172.21.15.32:6789/0 conn(0x7f639415fd60 legacy=0x7f639406c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:06.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.302+0000 7f6392ffd640 1 --2- 172.21.15.32:0/2341948661 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f639415b480 0x7f637c005bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:06.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.302+0000 7f6392ffd640 1 --2- 172.21.15.32:0/2341948661 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f637c006140 0x7f637c006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:06.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.302+0000 7f6392ffd640 1 --2- 172.21.15.32:0/2341948661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f637c006f50 0x7f637c0073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:06.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.302+0000 7f6392ffd640 1 -- 172.21.15.32:0/2341948661 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f63940b4ff0 con 0x7f637c006f50 2024-04-24T23:09:06.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.302+0000 7f6392ffd640 1 -- 172.21.15.32:0/2341948661 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f63940b49a0 con 0x7f637c006140 2024-04-24T23:09:06.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.302+0000 7f6392ffd640 1 -- 172.21.15.32:0/2341948661 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f637c00dbd0 con 0x7f639415b480 2024-04-24T23:09:06.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.302+0000 7f6398bb9640 1 --2- 172.21.15.32:0/2341948661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f637c006f50 0x7f637c0073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:06.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.302+0000 7f6393fff640 1 --2- 172.21.15.32:0/2341948661 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f639415b480 0x7f637c005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:06.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.302+0000 7f63937fe640 1 --2- 172.21.15.32:0/2341948661 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f637c006140 0x7f637c006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:06.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.302+0000 7f6398bb9640 1 -- 172.21.15.32:0/2341948661 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f639415b480 msgr2=0x7f637c005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:09:06.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.302+0000 7f6398bb9640 1 --2- 172.21.15.32:0/2341948661 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f639415b480 0x7f637c005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:06.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.302+0000 7f6398bb9640 1 -- 172.21.15.32:0/2341948661 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f637c006140 msgr2=0x7f637c006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:09:06.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.302+0000 7f6398bb9640 1 --2- 172.21.15.32:0/2341948661 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f637c006140 0x7f637c006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:06.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.302+0000 7f6398bb9640 1 -- 172.21.15.32:0/2341948661 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f637c005730 con 0x7f637c006f50 2024-04-24T23:09:06.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.303+0000 7f6398bb9640 1 --2- 172.21.15.32:0/2341948661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f637c006f50 0x7f637c0073d0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f638c009fc0 tx=0x7f637c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6becbc9f57315a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:06.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.303+0000 7f6392ffd640 1 -- 172.21.15.32:0/2341948661 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638c009c10 con 0x7f637c006f50 2024-04-24T23:09:06.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.303+0000 7f6392ffd640 1 -- 172.21.15.32:0/2341948661 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f638c036070 con 0x7f637c006f50 2024-04-24T23:09:06.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.303+0000 7f6392ffd640 1 -- 172.21.15.32:0/2341948661 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638c02d070 con 0x7f637c006f50 2024-04-24T23:09:06.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.304+0000 7f639a61c640 1 -- 172.21.15.32:0/2341948661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f637c006f50 msgr2=0x7f637c0073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:06.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.304+0000 7f639a61c640 1 --2- 172.21.15.32:0/2341948661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f637c006f50 0x7f637c0073d0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f638c009fc0 tx=0x7f637c007a40 comp rx=0 tx=0).stop 2024-04-24T23:09:06.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.304+0000 7f6398bb9640 1 -- 172.21.15.32:0/2341948661 reap_dead start 2024-04-24T23:09:06.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.304+0000 7f639a61c640 1 -- 172.21.15.32:0/2341948661 shutdown_connections 2024-04-24T23:09:06.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.304+0000 7f639a61c640 1 -- 172.21.15.32:0/2341948661 >> 172.21.15.32:0/2341948661 conn(0x7f639406b110 msgr2=0x7f639406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:06.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.304+0000 7f639a61c640 1 -- 172.21.15.32:0/2341948661 shutdown_connections 2024-04-24T23:09:06.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.304+0000 7f639a61c640 1 -- 172.21.15.32:0/2341948661 wait complete. 2024-04-24T23:09:06.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.305+0000 7f639a61c640 1 Processor -- start 2024-04-24T23:09:06.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.305+0000 7f639a61c640 1 -- start start 2024-04-24T23:09:06.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.306+0000 7f639a61c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f637c006f50 0x7f639415e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:06.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.306+0000 7f639a61c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f639414ad30 0x7f639415c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:06.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.306+0000 7f639a61c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f639415cd80 0x7f639414bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:06.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.306+0000 7f639a61c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f639415b860 con 0x7f639415cd80 2024-04-24T23:09:06.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.306+0000 7f639a61c640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f63940b5c60 con 0x7f639414ad30 2024-04-24T23:09:06.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.306+0000 7f639a61c640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f639415e180 con 0x7f637c006f50 2024-04-24T23:09:06.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.306+0000 7f6398bb9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f639415cd80 0x7f639414bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:06.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.306+0000 7f6398bb9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f639415cd80 0x7f639414bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40290/0 (socket says 172.21.15.32:40290) 2024-04-24T23:09:06.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.306+0000 7f6398bb9640 1 -- 172.21.15.32:0/3077953736 learned_addr learned my addr 172.21.15.32:0/3077953736 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:09:06.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.306+0000 7f6393fff640 1 --2- 172.21.15.32:0/3077953736 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f637c006f50 0x7f639415e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:06.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.306+0000 7f63937fe640 1 --2- 172.21.15.32:0/3077953736 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f639414ad30 0x7f639415c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:06.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.306+0000 7f6398bb9640 1 -- 172.21.15.32:0/3077953736 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f637c006f50 msgr2=0x7f639415e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:06.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.306+0000 7f6398bb9640 1 --2- 172.21.15.32:0/3077953736 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f637c006f50 0x7f639415e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:06.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.306+0000 7f6398bb9640 1 -- 172.21.15.32:0/3077953736 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f639414ad30 msgr2=0x7f639415c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:06.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.306+0000 7f6398bb9640 1 --2- 172.21.15.32:0/3077953736 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f639414ad30 0x7f639415c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:06.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.306+0000 7f6398bb9640 1 -- 172.21.15.32:0/3077953736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f639415d570 con 0x7f639415cd80 2024-04-24T23:09:06.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.306+0000 7f63937fe640 1 --2- 172.21.15.32:0/3077953736 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f639414ad30 0x7f639415c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:09:06.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.306+0000 7f6398bb9640 1 --2- 172.21.15.32:0/3077953736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f639415cd80 0x7f639414bb30 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f638c00e230 tx=0x7f638c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:06.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.306+0000 7f6390ff9640 1 -- 172.21.15.32:0/3077953736 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638c009c10 con 0x7f639415cd80 2024-04-24T23:09:06.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.306+0000 7f6390ff9640 1 -- 172.21.15.32:0/3077953736 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f638c003990 con 0x7f639415cd80 2024-04-24T23:09:06.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.306+0000 7f639a61c640 1 -- 172.21.15.32:0/3077953736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f639415d850 con 0x7f639415cd80 2024-04-24T23:09:06.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.306+0000 7f6390ff9640 1 -- 172.21.15.32:0/3077953736 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638c003d80 con 0x7f639415cd80 2024-04-24T23:09:06.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.306+0000 7f639a61c640 1 -- 172.21.15.32:0/3077953736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f639415de10 con 0x7f639415cd80 2024-04-24T23:09:06.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.307+0000 7f639a61c640 1 -- 172.21.15.32:0/3077953736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f639415b9e0 con 0x7f639415cd80 2024-04-24T23:09:06.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.308+0000 7f6390ff9640 1 -- 172.21.15.32:0/3077953736 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f638c005d10 con 0x7f639415cd80 2024-04-24T23:09:06.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.308+0000 7f6390ff9640 1 --2- 172.21.15.32:0/3077953736 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f638006be30 0x7f638006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:06.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.308+0000 7f6390ff9640 1 -- 172.21.15.32:0/3077953736 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 5709+0+0 (secure 0 0 0) 0x7f638c0ac660 con 0x7f639415cd80 2024-04-24T23:09:06.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.309+0000 7f6393fff640 1 --2- 172.21.15.32:0/3077953736 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f638006be30 0x7f638006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:06.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.309+0000 7f6393fff640 1 --2- 172.21.15.32:0/3077953736 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f638006be30 0x7f638006e2f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f639415eb70 tx=0x7f6388009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:06.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.313+0000 7f6390ff9640 1 -- 172.21.15.32:0/3077953736 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f638c04a060 con 0x7f639415cd80 2024-04-24T23:09:06.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.533+0000 7f639a61c640 1 -- 172.21.15.32:0/3077953736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6394068430 con 0x7f639415cd80 2024-04-24T23:09:06.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.533+0000 7f6390ff9640 1 -- 172.21.15.32:0/3077953736 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+5185 (secure 0 0 0) 0x7f638c07b0a0 con 0x7f639415cd80 2024-04-24T23:09:06.535 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:09:06.535 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":73,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6695,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/91086102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":91086102},{"type":"v1","addr":"172.21.15.121:6836","nonce":91086102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":6698,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1443743127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1443743127},{"type":"v1","addr":"172.21.15.121:6837","nonce":1443743127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":7423,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/4008773444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":4008773444},{"type":"v1","addr":"172.21.15.32:6835","nonce":4008773444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:09:00.770308+0000","modified":"2024-04-24T23:09:03.774877+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8679},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8679":{"gid":8679,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6834/4273183179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":4273183179},{"type":"v1","addr":"172.21.15.32:6834","nonce":4273183179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8679,"qdb_cluster":[8679]},"id":10}]} 2024-04-24T23:09:06.535 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 73 2024-04-24T23:09:06.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.535+0000 7f639a61c640 1 -- 172.21.15.32:0/3077953736 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f638006be30 msgr2=0x7f638006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:06.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.535+0000 7f639a61c640 1 --2- 172.21.15.32:0/3077953736 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f638006be30 0x7f638006e2f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f639415eb70 tx=0x7f6388009190 comp rx=0 tx=0).stop 2024-04-24T23:09:06.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.535+0000 7f639a61c640 1 -- 172.21.15.32:0/3077953736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f639415cd80 msgr2=0x7f639414bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:06.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.535+0000 7f639a61c640 1 --2- 172.21.15.32:0/3077953736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f639415cd80 0x7f639414bb30 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f638c00e230 tx=0x7f638c009750 comp rx=0 tx=0).stop 2024-04-24T23:09:06.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.535+0000 7f639a61c640 1 -- 172.21.15.32:0/3077953736 shutdown_connections 2024-04-24T23:09:06.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.535+0000 7f639a61c640 1 --2- 172.21.15.32:0/3077953736 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f638006be30 0x7f638006e2f0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:06.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.535+0000 7f639a61c640 1 --2- 172.21.15.32:0/3077953736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f639415cd80 0x7f639414bb30 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:06.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.535+0000 7f639a61c640 1 --2- 172.21.15.32:0/3077953736 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f639414ad30 0x7f639415c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:06.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.535+0000 7f639a61c640 1 --2- 172.21.15.32:0/3077953736 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f637c006f50 0x7f639415e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:06.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.535+0000 7f639a61c640 1 -- 172.21.15.32:0/3077953736 >> 172.21.15.32:0/3077953736 conn(0x7f639406b110 msgr2=0x7f63940b3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:06.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.535+0000 7f639a61c640 1 -- 172.21.15.32:0/3077953736 shutdown_connections 2024-04-24T23:09:06.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:06.535+0000 7f639a61c640 1 -- 172.21.15.32:0/3077953736 wait complete. 2024-04-24T23:09:06.950 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 73, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T23:09:00.770308+0000', 'modified': '2024-04-24T23:09:03.774877+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8679}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8679': {'gid': 8679, 'name': 'c', 'rank': 0, 'incarnation': 70, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6834/4273183179', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6832', 'nonce': 4273183179}, {'type': 'v1', 'addr': '172.21.15.32:6834', 'nonce': 4273183179}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8679, 'qdb_cluster': [8679]} 2024-04-24T23:09:06.951 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-24T23:09:06.951 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-24T23:09:07.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.055+0000 7f7e58bab640 1 Processor -- start 2024-04-24T23:09:07.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.055+0000 7f7e58bab640 1 -- start start 2024-04-24T23:09:07.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.055+0000 7f7e58bab640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7e5415b380 0x7f7e5406c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:07.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.055+0000 7f7e58bab640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e540ceb40 0x7f7e5406cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:07.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.056+0000 7f7e58bab640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e540b4520 con 0x7f7e5415cb00 2024-04-24T23:09:07.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.056+0000 7f7e58bab640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f7e540cef70 con 0x7f7e5415b380 2024-04-24T23:09:07.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.056+0000 7f7e58bab640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f7e540b4800 con 0x7f7e540ceb40 2024-04-24T23:09:07.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.056+0000 7f7e51d9b640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7e5415cb00 0x7f7e540ce430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40644/0 (socket says 172.21.15.32:40644) 2024-04-24T23:09:07.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.056+0000 7f7e51d9b640 1 -- 172.21.15.32:0/4282272563 learned_addr learned my addr 172.21.15.32:0/4282272563 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:09:07.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.056+0000 7f7e5159a640 1 -- 172.21.15.32:0/4282272563 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 741606371 0 0) 0x7f7e540b4520 con 0x7f7e5415cb00 2024-04-24T23:09:07.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.056+0000 7f7e52d9d640 1 --2- 172.21.15.32:0/4282272563 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e540ceb40 0x7f7e5406cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:07.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.056+0000 7f7e5159a640 1 -- 172.21.15.32:0/4282272563 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e40005730 con 0x7f7e5415cb00 2024-04-24T23:09:07.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.056+0000 7f7e5259c640 1 --2- 172.21.15.32:0/4282272563 >> v2:172.21.15.121:3300/0 conn(0x7f7e5415b380 0x7f7e5406c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:07.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.056+0000 7f7e5159a640 1 -- 172.21.15.32:0/4282272563 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2696781080 0 0) 0x7f7e40005730 con 0x7f7e5415cb00 2024-04-24T23:09:07.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.056+0000 7f7e5159a640 1 -- 172.21.15.32:0/4282272563 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e540ceb40 msgr2=0x7f7e5406cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:09:07.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.056+0000 7f7e5159a640 1 --2- 172.21.15.32:0/4282272563 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e540ceb40 0x7f7e5406cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:07.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.056+0000 7f7e5159a640 1 -- 172.21.15.32:0/4282272563 >> v2:172.21.15.121:3300/0 conn(0x7f7e5415b380 msgr2=0x7f7e5406c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:09:07.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.056+0000 7f7e5159a640 1 --2- 172.21.15.32:0/4282272563 >> v2:172.21.15.121:3300/0 conn(0x7f7e5415b380 0x7f7e5406c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:07.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.056+0000 7f7e5159a640 1 -- 172.21.15.32:0/4282272563 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e540b4ae0 con 0x7f7e5415cb00 2024-04-24T23:09:07.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.056+0000 7f7e5159a640 1 -- 172.21.15.32:0/4282272563 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f7e3c001670 con 0x7f7e5415cb00 2024-04-24T23:09:07.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.056+0000 7f7e5159a640 1 -- 172.21.15.32:0/4282272563 >> v1:172.21.15.32:6789/0 conn(0x7f7e5415cb00 legacy=0x7f7e540ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:07.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.056+0000 7f7e5159a640 1 --2- 172.21.15.32:0/4282272563 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e540ceb40 0x7f7e40005bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:07.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.056+0000 7f7e5159a640 1 --2- 172.21.15.32:0/4282272563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e40006140 0x7f7e40006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:07.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.056+0000 7f7e5159a640 1 --2- 172.21.15.32:0/4282272563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e40006f50 0x7f7e400073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:07.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.056+0000 7f7e5159a640 1 -- 172.21.15.32:0/4282272563 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7e540cef70 con 0x7f7e40006f50 2024-04-24T23:09:07.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.056+0000 7f7e5159a640 1 -- 172.21.15.32:0/4282272563 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7e540b4800 con 0x7f7e40006140 2024-04-24T23:09:07.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.056+0000 7f7e5159a640 1 -- 172.21.15.32:0/4282272563 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f7e4000dcb0 con 0x7f7e540ceb40 2024-04-24T23:09:07.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.057+0000 7f7e52d9d640 1 --2- 172.21.15.32:0/4282272563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e40006f50 0x7f7e400073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:07.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.057+0000 7f7e51d9b640 1 --2- 172.21.15.32:0/4282272563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e40006140 0x7f7e40006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:07.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.057+0000 7f7e5259c640 1 --2- 172.21.15.32:0/4282272563 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e540ceb40 0x7f7e40005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:07.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.057+0000 7f7e52d9d640 1 -- 172.21.15.32:0/4282272563 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e540ceb40 msgr2=0x7f7e40005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:09:07.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.057+0000 7f7e52d9d640 1 --2- 172.21.15.32:0/4282272563 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e540ceb40 0x7f7e40005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:07.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.057+0000 7f7e52d9d640 1 -- 172.21.15.32:0/4282272563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e40006140 msgr2=0x7f7e40006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:09:07.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.057+0000 7f7e52d9d640 1 --2- 172.21.15.32:0/4282272563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e40006140 0x7f7e40006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:07.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.057+0000 7f7e52d9d640 1 -- 172.21.15.32:0/4282272563 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e40005730 con 0x7f7e40006f50 2024-04-24T23:09:07.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.057+0000 7f7e52d9d640 1 --2- 172.21.15.32:0/4282272563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e40006f50 0x7f7e400073d0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f7e44009fc0 tx=0x7f7e40007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f69cbca17dd97da server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:07.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.058+0000 7f7e5159a640 1 -- 172.21.15.32:0/4282272563 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e44009c10 con 0x7f7e40006f50 2024-04-24T23:09:07.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.058+0000 7f7e5159a640 1 -- 172.21.15.32:0/4282272563 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e44036070 con 0x7f7e40006f50 2024-04-24T23:09:07.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.058+0000 7f7e5159a640 1 -- 172.21.15.32:0/4282272563 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e44031880 con 0x7f7e40006f50 2024-04-24T23:09:07.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.058+0000 7f7e58bab640 1 -- 172.21.15.32:0/4282272563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e40006f50 msgr2=0x7f7e400073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:07.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.058+0000 7f7e58bab640 1 --2- 172.21.15.32:0/4282272563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e40006f50 0x7f7e400073d0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f7e44009fc0 tx=0x7f7e40007a40 comp rx=0 tx=0).stop 2024-04-24T23:09:07.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.058+0000 7f7e52d9d640 1 -- 172.21.15.32:0/4282272563 reap_dead start 2024-04-24T23:09:07.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.058+0000 7f7e58bab640 1 -- 172.21.15.32:0/4282272563 shutdown_connections 2024-04-24T23:09:07.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.058+0000 7f7e58bab640 1 -- 172.21.15.32:0/4282272563 >> 172.21.15.32:0/4282272563 conn(0x7f7e5406b110 msgr2=0x7f7e5406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:07.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.058+0000 7f7e58bab640 1 -- 172.21.15.32:0/4282272563 shutdown_connections 2024-04-24T23:09:07.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.058+0000 7f7e58bab640 1 -- 172.21.15.32:0/4282272563 wait complete. 2024-04-24T23:09:07.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.060+0000 7f7e58bab640 1 Processor -- start 2024-04-24T23:09:07.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.060+0000 7f7e58bab640 1 -- start start 2024-04-24T23:09:07.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.060+0000 7f7e58bab640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e40006f50 0x7f7e541440b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:07.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.060+0000 7f7e58bab640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e541529f0 0x7f7e54071eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:07.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.060+0000 7f7e58bab640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e5415d0b0 0x7f7e5415d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:07.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.060+0000 7f7e58bab640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5415b760 con 0x7f7e5415d0b0 2024-04-24T23:09:07.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.060+0000 7f7e58bab640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7e540b50a0 con 0x7f7e541529f0 2024-04-24T23:09:07.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.060+0000 7f7e58bab640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f7e5406cb30 con 0x7f7e40006f50 2024-04-24T23:09:07.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.060+0000 7f7e52d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e5415d0b0 0x7f7e5415d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:07.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.060+0000 7f7e52d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e5415d0b0 0x7f7e5415d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40312/0 (socket says 172.21.15.32:40312) 2024-04-24T23:09:07.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.060+0000 7f7e52d9d640 1 -- 172.21.15.32:0/1439124221 learned_addr learned my addr 172.21.15.32:0/1439124221 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:09:07.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.061+0000 7f7e52d9d640 1 -- 172.21.15.32:0/1439124221 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e40006f50 msgr2=0x7f7e541440b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:07.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.061+0000 7f7e52d9d640 1 --2- 172.21.15.32:0/1439124221 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e40006f50 0x7f7e541440b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:07.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.061+0000 7f7e52d9d640 1 -- 172.21.15.32:0/1439124221 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e541529f0 msgr2=0x7f7e54071eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:07.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.061+0000 7f7e52d9d640 1 --2- 172.21.15.32:0/1439124221 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e541529f0 0x7f7e54071eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:07.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.061+0000 7f7e52d9d640 1 -- 172.21.15.32:0/1439124221 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e54152f60 con 0x7f7e5415d0b0 2024-04-24T23:09:07.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.061+0000 7f7e52d9d640 1 --2- 172.21.15.32:0/1439124221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e5415d0b0 0x7f7e5415d530 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f7e4400e230 tx=0x7f7e44009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:07.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.061+0000 7f7e4affd640 1 -- 172.21.15.32:0/1439124221 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e44009c10 con 0x7f7e5415d0b0 2024-04-24T23:09:07.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.061+0000 7f7e4affd640 1 -- 172.21.15.32:0/1439124221 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e44031dd0 con 0x7f7e5415d0b0 2024-04-24T23:09:07.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.061+0000 7f7e4affd640 1 -- 172.21.15.32:0/1439124221 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e44003bd0 con 0x7f7e5415d0b0 2024-04-24T23:09:07.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.061+0000 7f7e58bab640 1 -- 172.21.15.32:0/1439124221 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e54153240 con 0x7f7e5415d0b0 2024-04-24T23:09:07.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.061+0000 7f7e58bab640 1 -- 172.21.15.32:0/1439124221 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e54153800 con 0x7f7e5415d0b0 2024-04-24T23:09:07.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.062+0000 7f7e4affd640 1 -- 172.21.15.32:0/1439124221 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7e44005d10 con 0x7f7e5415d0b0 2024-04-24T23:09:07.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.063+0000 7f7e4affd640 1 --2- 172.21.15.32:0/1439124221 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7e3406be60 0x7f7e3406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:07.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.063+0000 7f7e4affd640 1 -- 172.21.15.32:0/1439124221 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 5709+0+0 (secure 0 0 0) 0x7f7e440ac5d0 con 0x7f7e5415d0b0 2024-04-24T23:09:07.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.063+0000 7f7e5259c640 1 --2- 172.21.15.32:0/1439124221 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7e3406be60 0x7f7e3406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:07.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.064+0000 7f7e5259c640 1 --2- 172.21.15.32:0/1439124221 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7e3406be60 0x7f7e3406e320 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c006e80 tx=0x7f7e4c006df0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:07.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.064+0000 7f7e5b98a740 1 Processor -- start 2024-04-24T23:09:07.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.064+0000 7f7e5b98a740 1 -- start start 2024-04-24T23:09:07.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.065+0000 7f7e5b98a740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f2c54d9e50 0x55f2c54ddc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:07.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.065+0000 7f7e5b98a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2c54b1b60 0x55f2c54da230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:07.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.065+0000 7f7e5b98a740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f2c54da770 0x55f2c54dd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:07.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.065+0000 7f7e5b98a740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f2c5415f00 con 0x55f2c54d9e50 2024-04-24T23:09:07.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.065+0000 7f7e5b98a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55f2c53be750 con 0x55f2c54b1b60 2024-04-24T23:09:07.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.065+0000 7f7e5b98a740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x55f2c5411ec0 con 0x55f2c54da770 2024-04-24T23:09:07.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.065+0000 7f7e5259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f2c54d9e50 0x55f2c54ddc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:07.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.065+0000 7f7e5259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f2c54d9e50 0x55f2c54ddc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40318/0 (socket says 172.21.15.32:40318) 2024-04-24T23:09:07.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.065+0000 7f7e5259c640 1 -- 172.21.15.32:0/1919643674 learned_addr learned my addr 172.21.15.32:0/1919643674 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:09:07.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.065+0000 7f7e52d9d640 1 --2- 172.21.15.32:0/1919643674 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f2c54da770 0x55f2c54dd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:07.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.065+0000 7f7e51d9b640 1 --2- 172.21.15.32:0/1919643674 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2c54b1b60 0x55f2c54da230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:07.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.065+0000 7f7e5259c640 1 -- 172.21.15.32:0/1919643674 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f2c54da770 msgr2=0x55f2c54dd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:09:07.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.065+0000 7f7e5259c640 1 --2- 172.21.15.32:0/1919643674 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f2c54da770 0x55f2c54dd000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:07.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.065+0000 7f7e5259c640 1 -- 172.21.15.32:0/1919643674 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2c54b1b60 msgr2=0x55f2c54da230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:09:07.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.065+0000 7f7e5259c640 1 --2- 172.21.15.32:0/1919643674 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2c54b1b60 0x55f2c54da230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:07.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.065+0000 7f7e5259c640 1 -- 172.21.15.32:0/1919643674 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2c512a2d0 con 0x55f2c54d9e50 2024-04-24T23:09:07.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.065+0000 7f7e51d9b640 1 --2- 172.21.15.32:0/1919643674 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2c54b1b60 0x55f2c54da230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:09:07.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.065+0000 7f7e5259c640 1 --2- 172.21.15.32:0/1919643674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f2c54d9e50 0x55f2c54ddc50 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c004520 tx=0x7f7e4c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4c02f60f7cf0b39 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:07.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.065+0000 7f7e33fff640 1 -- 172.21.15.32:0/1919643674 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c01d050 con 0x55f2c54d9e50 2024-04-24T23:09:07.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.065+0000 7f7e33fff640 1 -- 172.21.15.32:0/1919643674 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e4c027070 con 0x55f2c54d9e50 2024-04-24T23:09:07.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.065+0000 7f7e33fff640 1 -- 172.21.15.32:0/1919643674 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c003a20 con 0x55f2c54d9e50 2024-04-24T23:09:07.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.066+0000 7f7e5b98a740 1 -- 172.21.15.32:0/1919643674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f2c54d9e50 msgr2=0x55f2c54ddc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:07.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.066+0000 7f7e5b98a740 1 --2- 172.21.15.32:0/1919643674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f2c54d9e50 0x55f2c54ddc50 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c004520 tx=0x7f7e4c014000 comp rx=0 tx=0).stop 2024-04-24T23:09:07.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.066+0000 7f7e5b98a740 1 -- 172.21.15.32:0/1919643674 shutdown_connections 2024-04-24T23:09:07.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.066+0000 7f7e5b98a740 1 --2- 172.21.15.32:0/1919643674 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f2c54da770 0x55f2c54dd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:07.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.066+0000 7f7e5b98a740 1 --2- 172.21.15.32:0/1919643674 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2c54b1b60 0x55f2c54da230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:07.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.066+0000 7f7e5b98a740 1 --2- 172.21.15.32:0/1919643674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f2c54d9e50 0x55f2c54ddc50 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:07.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.066+0000 7f7e5b98a740 1 -- 172.21.15.32:0/1919643674 >> 172.21.15.32:0/1919643674 conn(0x55f2c54af760 msgr2=0x55f2c54f8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:07.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.066+0000 7f7e5b98a740 1 -- 172.21.15.32:0/1919643674 shutdown_connections 2024-04-24T23:09:07.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.066+0000 7f7e5b98a740 1 -- 172.21.15.32:0/1919643674 wait complete. 2024-04-24T23:09:07.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.067+0000 7f7e5b98a740 1 -- start start 2024-04-24T23:09:07.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.067+0000 7f7e5b98a740 1 Processor -- start 2024-04-24T23:09:07.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.068+0000 7f7e5b98a740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f2c54d9e50 0x55f2c54d8420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:07.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.068+0000 7f7e5b98a740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f2c54da770 0x55f2c550cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:07.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.068+0000 7f7e5b98a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2c54d0510 0x55f2c550d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:07.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.068+0000 7f7e5b98a740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f2c54adab0 con 0x55f2c54d9e50 2024-04-24T23:09:07.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.068+0000 7f7e5b98a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55f2c53d1cf0 con 0x55f2c54d0510 2024-04-24T23:09:07.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.068+0000 7f7e5b98a740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x55f2c54ad4f0 con 0x55f2c54da770 2024-04-24T23:09:07.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.068+0000 7f7e5b98a740 10 client.0 _ll_register_callbacks cb 0x55f2c53d16e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:09:07.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.068+0000 7f7e5259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f2c54d9e50 0x55f2c54d8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:07.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.068+0000 7f7e5259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f2c54d9e50 0x55f2c54d8420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40324/0 (socket says 172.21.15.32:40324) 2024-04-24T23:09:07.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.068+0000 7f7e5259c640 1 -- 172.21.15.32:0/1128741815 learned_addr learned my addr 172.21.15.32:0/1128741815 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:09:07.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.068+0000 7f7e52d9d640 1 --2- 172.21.15.32:0/1128741815 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2c54d0510 0x55f2c550d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:07.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.068+0000 7f7e51d9b640 1 --2- 172.21.15.32:0/1128741815 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f2c54da770 0x55f2c550cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:07.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.068+0000 7f7e5259c640 1 -- 172.21.15.32:0/1128741815 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f2c54da770 msgr2=0x55f2c550cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:09:07.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.068+0000 7f7e5259c640 1 --2- 172.21.15.32:0/1128741815 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f2c54da770 0x55f2c550cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:07.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.068+0000 7f7e5259c640 1 -- 172.21.15.32:0/1128741815 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2c54d0510 msgr2=0x55f2c550d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:09:07.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.068+0000 7f7e5259c640 1 --2- 172.21.15.32:0/1128741815 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2c54d0510 0x55f2c550d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:07.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.068+0000 7f7e5259c640 1 -- 172.21.15.32:0/1128741815 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2c51f46b0 con 0x55f2c54d9e50 2024-04-24T23:09:07.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.068+0000 7f7e5259c640 1 --2- 172.21.15.32:0/1128741815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f2c54d9e50 0x55f2c54d8420 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c0044c0 tx=0x7f7e4c009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2af1e5e0328ec219 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:07.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.068+0000 7f7e317fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-24T23:09:07.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.068+0000 7f7e317fa640 1 -- 172.21.15.32:0/1128741815 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c035670 con 0x55f2c54d9e50 2024-04-24T23:09:07.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.069+0000 7f7e317fa640 1 -- 172.21.15.32:0/1128741815 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(169..169 src has 1..169) ==== 5709+0+0 (secure 0 0 0) 0x7f7e4c035810 con 0x55f2c54d9e50 2024-04-24T23:09:07.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.069+0000 7f7e5b98a740 1 -- 172.21.15.32:0/1128741815 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e4c027070 con 0x55f2c54d9e50 2024-04-24T23:09:07.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.069+0000 7f7e317fa640 1 -- 172.21.15.32:0/1128741815 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f7e4c01d020 con 0x55f2c54d9e50 2024-04-24T23:09:07.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.069+0000 7f7e5b98a740 10 client.8817 fetch_fsmap learned FSMap version 73 2024-04-24T23:09:07.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.069+0000 7f7e5b98a740 1 -- 172.21.15.32:0/1128741815 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) -- 0x55f2c550dc70 con 0x55f2c54d9e50 2024-04-24T23:09:07.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.069+0000 7f7e317fa640 1 -- 172.21.15.32:0/1128741815 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e4c027070 con 0x55f2c54d9e50 2024-04-24T23:09:07.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.069+0000 7f7e317fa640 1 -- 172.21.15.32:0/1128741815 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 73) ==== 3494+0+0 (secure 0 0 0) 0x7f7e4c035a10 con 0x55f2c54d9e50 2024-04-24T23:09:07.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.069+0000 7f7e5b98a740 10 client.8817 fetch_fsmap finished waiting for FSMap version 73 2024-04-24T23:09:07.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.069+0000 7f7e5b98a740 10 client.8817 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-24T23:09:07.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.069+0000 7f7e5b98a740 20 client.8817 populate_metadata read hostname 'smithi032' 2024-04-24T23:09:07.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.069+0000 7f7e5b98a740 1 --2- 172.21.15.32:0/1128741815 >> [v2:172.21.15.32:6832/4273183179,v1:172.21.15.32:6834/4273183179] conn(0x55f2c5518480 0x55f2c551a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:07.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.069+0000 7f7e5b98a740 4 client.8817 mds_command: new command op to 8679 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:09:07.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.069+0000 7f7e5b98a740 1 -- 172.21.15.32:0/1128741815 --> [v2:172.21.15.32:6832/4273183179,v1:172.21.15.32:6834/4273183179] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2c5423440 con 0x55f2c5518480 2024-04-24T23:09:07.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.069+0000 7f7e51d9b640 1 --2- 172.21.15.32:0/1128741815 >> [v2:172.21.15.32:6832/4273183179,v1:172.21.15.32:6834/4273183179] conn(0x55f2c5518480 0x55f2c551a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:07.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.069+0000 7f7e51d9b640 1 --2- 172.21.15.32:0/1128741815 >> [v2:172.21.15.32:6832/4273183179,v1:172.21.15.32:6834/4273183179] conn(0x55f2c5518480 0x55f2c551a870 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72477d919ded7e06 server_cookie=d762a32bbc061c24 in_seq=0 out_seq=0 2024-04-24T23:09:07.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.069+0000 7f7e317fa640 10 client.8817 ms_handle_connect on v2:172.21.15.32:6832/4273183179 2024-04-24T23:09:07.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.070+0000 7f7e317fa640 1 -- 172.21.15.32:0/1128741815 <== mds.0 v2:172.21.15.32:6832/4273183179 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f2c5423440 con 0x55f2c5518480 2024-04-24T23:09:07.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.070+0000 7f7e317fa640 10 client.8817 handle_command_reply: tid=0 2024-04-24T23:09:07.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.070+0000 7f7e5b98a740 1 client.8817 shutdown 2024-04-24T23:09:07.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.070+0000 7f7e5b98a740 1 -- 172.21.15.32:0/1128741815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f2c54d9e50 msgr2=0x55f2c54d8420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:07.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.070+0000 7f7e5b98a740 1 --2- 172.21.15.32:0/1128741815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f2c54d9e50 0x55f2c54d8420 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c0044c0 tx=0x7f7e4c009190 comp rx=0 tx=0).stop 2024-04-24T23:09:07.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.070+0000 7f7e5b98a740 1 -- 172.21.15.32:0/1128741815 shutdown_connections 2024-04-24T23:09:07.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.070+0000 7f7e5b98a740 1 --2- 172.21.15.32:0/1128741815 >> [v2:172.21.15.32:6832/4273183179,v1:172.21.15.32:6834/4273183179] conn(0x55f2c5518480 0x55f2c551a870 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:07.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.070+0000 7f7e5b98a740 1 --2- 172.21.15.32:0/1128741815 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2c54d0510 0x55f2c550d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:07.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.070+0000 7f7e5b98a740 1 --2- 172.21.15.32:0/1128741815 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f2c54da770 0x55f2c550cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:07.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.070+0000 7f7e317fa640 0 client.8817 ms_handle_reset on v2:172.21.15.32:6832/4273183179 2024-04-24T23:09:07.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.070+0000 7f7e5b98a740 1 --2- 172.21.15.32:0/1128741815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f2c54d9e50 0x55f2c54d8420 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:07.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.070+0000 7f7e5b98a740 1 -- 172.21.15.32:0/1128741815 >> 172.21.15.32:0/1128741815 conn(0x55f2c54b1b60 msgr2=0x55f2c54f88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:07.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.070+0000 7f7e5b98a740 1 -- 172.21.15.32:0/1128741815 shutdown_connections 2024-04-24T23:09:07.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.070+0000 7f7e5b98a740 1 -- 172.21.15.32:0/1128741815 wait complete. 2024-04-24T23:09:07.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.070+0000 7f7e5b98a740 20 client.8817 trim_cache size 0 max 16384 2024-04-24T23:09:07.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.089+0000 7f7e5b98a740 1 Processor -- start 2024-04-24T23:09:07.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.089+0000 7f7e5b98a740 1 -- start start 2024-04-24T23:09:07.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.089+0000 7f7e5b98a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2c54d0510 0x55f2c54b41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:07.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.089+0000 7f7e5b98a740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f2c54d9e50 0x55f2c551fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:07.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.089+0000 7f7e5b98a740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f2c54da770 0x55f2c5520130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:07.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.089+0000 7f7e5b98a740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7e541fbe60 con 0x55f2c54d9e50 2024-04-24T23:09:07.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.089+0000 7f7e5b98a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7e541493f0 con 0x55f2c54d0510 2024-04-24T23:09:07.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.089+0000 7f7e5b98a740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f7e54149a40 con 0x55f2c54da770 2024-04-24T23:09:07.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.089+0000 7f7e51d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f2c54d9e50 0x55f2c551fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:07.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.089+0000 7f7e51d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f2c54d9e50 0x55f2c551fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40332/0 (socket says 172.21.15.32:40332) 2024-04-24T23:09:07.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.089+0000 7f7e51d9b640 1 -- 172.21.15.32:0/2667235614 learned_addr learned my addr 172.21.15.32:0/2667235614 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:09:07.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.090+0000 7f7e51d9b640 1 -- 172.21.15.32:0/2667235614 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f2c54da770 msgr2=0x55f2c5520130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:09:07.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.090+0000 7f7e51d9b640 1 --2- 172.21.15.32:0/2667235614 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f2c54da770 0x55f2c5520130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:07.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.090+0000 7f7e51d9b640 1 -- 172.21.15.32:0/2667235614 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2c54d0510 msgr2=0x55f2c54b41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:09:07.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.090+0000 7f7e51d9b640 1 --2- 172.21.15.32:0/2667235614 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2c54d0510 0x55f2c54b41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:07.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.090+0000 7f7e51d9b640 1 -- 172.21.15.32:0/2667235614 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2c54b04a0 con 0x55f2c54d9e50 2024-04-24T23:09:07.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.090+0000 7f7e51d9b640 1 --2- 172.21.15.32:0/2667235614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f2c54d9e50 0x55f2c551fbf0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f7e5415e3a0 tx=0x7f7e3c004bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1515209fe7d17e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:07.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.090+0000 7f7e337fe640 1 -- 172.21.15.32:0/2667235614 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e3c004ce0 con 0x55f2c54d9e50 2024-04-24T23:09:07.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.090+0000 7f7e337fe640 1 -- 172.21.15.32:0/2667235614 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e3c0074f0 con 0x55f2c54d9e50 2024-04-24T23:09:07.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.090+0000 7f7e337fe640 1 -- 172.21.15.32:0/2667235614 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e3c016420 con 0x55f2c54d9e50 2024-04-24T23:09:07.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.090+0000 7f7e5b98a740 1 -- 172.21.15.32:0/2667235614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f2c54d9e50 msgr2=0x55f2c551fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:07.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.090+0000 7f7e5b98a740 1 --2- 172.21.15.32:0/2667235614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f2c54d9e50 0x55f2c551fbf0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f7e5415e3a0 tx=0x7f7e3c004bb0 comp rx=0 tx=0).stop 2024-04-24T23:09:07.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.091+0000 7f7e5b98a740 1 -- 172.21.15.32:0/2667235614 shutdown_connections 2024-04-24T23:09:07.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.091+0000 7f7e5b98a740 1 --2- 172.21.15.32:0/2667235614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f2c54d9e50 0x55f2c551fbf0 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:07.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.091+0000 7f7e5b98a740 1 --2- 172.21.15.32:0/2667235614 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f2c54da770 0x55f2c5520130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:07.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.091+0000 7f7e5b98a740 1 --2- 172.21.15.32:0/2667235614 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2c54d0510 0x55f2c54b41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:07.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.091+0000 7f7e5b98a740 1 -- 172.21.15.32:0/2667235614 >> 172.21.15.32:0/2667235614 conn(0x55f2c54b1b60 msgr2=0x55f2c54d81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:07.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.091+0000 7f7e5b98a740 1 -- 172.21.15.32:0/2667235614 shutdown_connections 2024-04-24T23:09:07.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.091+0000 7f7e5b98a740 1 -- 172.21.15.32:0/2667235614 wait complete. 2024-04-24T23:09:07.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.091+0000 7f7e5b98a740 1 -- start start 2024-04-24T23:09:07.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.091+0000 7f7e5b98a740 1 Processor -- start 2024-04-24T23:09:07.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.092+0000 7f7e5b98a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2c54d0510 0x55f2c54f7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:07.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.092+0000 7f7e5b98a740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f2c54d9e50 0x55f2c54f7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:07.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.092+0000 7f7e5b98a740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f2c54da770 0x55f2c54cee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:07.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.092+0000 7f7e5b98a740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f2c54e0eb0 con 0x55f2c54d9e50 2024-04-24T23:09:07.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.092+0000 7f7e5b98a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55f2c54e0c40 con 0x55f2c54d0510 2024-04-24T23:09:07.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.092+0000 7f7e5b98a740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x55f2c54fa4c0 con 0x55f2c54da770 2024-04-24T23:09:07.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.092+0000 7f7e5b98a740 10 client.0 _ll_register_callbacks cb 0x7f7e4c003c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:09:07.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.092+0000 7f7e51d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f2c54d9e50 0x55f2c54f7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:07.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.092+0000 7f7e51d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f2c54d9e50 0x55f2c54f7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40334/0 (socket says 172.21.15.32:40334) 2024-04-24T23:09:07.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.092+0000 7f7e51d9b640 1 -- 172.21.15.32:0/423698585 learned_addr learned my addr 172.21.15.32:0/423698585 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:09:07.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.092+0000 7f7e52d9d640 1 --2- 172.21.15.32:0/423698585 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f2c54da770 0x55f2c54cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:07.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.092+0000 7f7e5259c640 1 --2- 172.21.15.32:0/423698585 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2c54d0510 0x55f2c54f7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:07.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.093+0000 7f7e51d9b640 1 -- 172.21.15.32:0/423698585 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f2c54da770 msgr2=0x55f2c54cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:09:07.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.093+0000 7f7e51d9b640 1 --2- 172.21.15.32:0/423698585 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f2c54da770 0x55f2c54cee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:07.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.093+0000 7f7e51d9b640 1 -- 172.21.15.32:0/423698585 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2c54d0510 msgr2=0x55f2c54f7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:09:07.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.093+0000 7f7e51d9b640 1 --2- 172.21.15.32:0/423698585 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2c54d0510 0x55f2c54f7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:07.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.093+0000 7f7e51d9b640 1 -- 172.21.15.32:0/423698585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2c5520100 con 0x55f2c54d9e50 2024-04-24T23:09:07.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.093+0000 7f7e51d9b640 1 --2- 172.21.15.32:0/423698585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f2c54d9e50 0x55f2c54f7570 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c009010 tx=0x7f7e3c009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19ede535bb0f3fdc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:07.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.093+0000 7f7e317fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-24T23:09:07.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.093+0000 7f7e317fa640 1 -- 172.21.15.32:0/423698585 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e3c016620 con 0x55f2c54d9e50 2024-04-24T23:09:07.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.093+0000 7f7e317fa640 1 -- 172.21.15.32:0/423698585 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(169..169 src has 1..169) ==== 5709+0+0 (secure 0 0 0) 0x7f7e3c0167c0 con 0x55f2c54d9e50 2024-04-24T23:09:07.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.093+0000 7f7e5b98a740 1 -- 172.21.15.32:0/423698585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e3c0074f0 con 0x55f2c54d9e50 2024-04-24T23:09:07.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.093+0000 7f7e317fa640 1 -- 172.21.15.32:0/423698585 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f7e3c035050 con 0x55f2c54d9e50 2024-04-24T23:09:07.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.093+0000 7f7e5b98a740 10 client.8823 fetch_fsmap learned FSMap version 73 2024-04-24T23:09:07.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.093+0000 7f7e5b98a740 1 -- 172.21.15.32:0/423698585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) -- 0x55f2c54d8a70 con 0x55f2c54d9e50 2024-04-24T23:09:07.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.093+0000 7f7e317fa640 1 -- 172.21.15.32:0/423698585 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e3c0074f0 con 0x55f2c54d9e50 2024-04-24T23:09:07.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.093+0000 7f7e317fa640 1 -- 172.21.15.32:0/423698585 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 73) ==== 3494+0+0 (secure 0 0 0) 0x7f7e3c038020 con 0x55f2c54d9e50 2024-04-24T23:09:07.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.093+0000 7f7e5b98a740 10 client.8823 fetch_fsmap finished waiting for FSMap version 73 2024-04-24T23:09:07.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.093+0000 7f7e5b98a740 10 client.8823 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-24T23:09:07.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.093+0000 7f7e5b98a740 20 client.8823 populate_metadata read hostname 'smithi032' 2024-04-24T23:09:07.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.093+0000 7f7e5b98a740 1 --2- 172.21.15.32:0/423698585 >> [v2:172.21.15.32:6832/4273183179,v1:172.21.15.32:6834/4273183179] conn(0x55f2c54cf780 0x55f2c557a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:07.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.093+0000 7f7e5b98a740 4 client.8823 mds_command: new command op to 8679 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:09:07.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.093+0000 7f7e5b98a740 1 -- 172.21.15.32:0/423698585 --> [v2:172.21.15.32:6832/4273183179,v1:172.21.15.32:6834/4273183179] -- command(tid 0: {"prefix": "status"}) -- 0x55f2c541d790 con 0x55f2c54cf780 2024-04-24T23:09:07.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.093+0000 7f7e5259c640 1 --2- 172.21.15.32:0/423698585 >> [v2:172.21.15.32:6832/4273183179,v1:172.21.15.32:6834/4273183179] conn(0x55f2c54cf780 0x55f2c557a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:07.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.094+0000 7f7e317fa640 10 client.8823 ms_handle_connect on v2:172.21.15.32:6832/4273183179 2024-04-24T23:09:07.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.094+0000 7f7e5259c640 1 --2- 172.21.15.32:0/423698585 >> [v2:172.21.15.32:6832/4273183179,v1:172.21.15.32:6834/4273183179] conn(0x55f2c54cf780 0x55f2c557a130 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75a193c0af5239b2 server_cookie=55dff51e6d2cd500 in_seq=0 out_seq=0 2024-04-24T23:09:07.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.094+0000 7f7e317fa640 1 -- 172.21.15.32:0/423698585 <== mds.0 v2:172.21.15.32:6832/4273183179 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55f2c541d790 con 0x55f2c54cf780 2024-04-24T23:09:07.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.094+0000 7f7e317fa640 10 client.8823 handle_command_reply: tid=0 2024-04-24T23:09:07.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.094+0000 7f7e5b98a740 1 client.8823 shutdown 2024-04-24T23:09:07.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.094+0000 7f7e5b98a740 1 -- 172.21.15.32:0/423698585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f2c54d9e50 msgr2=0x55f2c54f7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:07.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.094+0000 7f7e5b98a740 1 --2- 172.21.15.32:0/423698585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f2c54d9e50 0x55f2c54f7570 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c009010 tx=0x7f7e3c009040 comp rx=0 tx=0).stop 2024-04-24T23:09:07.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.094+0000 7f7e5b98a740 1 -- 172.21.15.32:0/423698585 shutdown_connections 2024-04-24T23:09:07.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.094+0000 7f7e5b98a740 1 --2- 172.21.15.32:0/423698585 >> [v2:172.21.15.32:6832/4273183179,v1:172.21.15.32:6834/4273183179] conn(0x55f2c54cf780 0x55f2c557a130 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:07.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.094+0000 7f7e5b98a740 1 --2- 172.21.15.32:0/423698585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f2c54d9e50 0x55f2c54f7570 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:07.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.094+0000 7f7e5b98a740 1 --2- 172.21.15.32:0/423698585 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f2c54da770 0x55f2c54cee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:07.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.094+0000 7f7e5b98a740 1 --2- 172.21.15.32:0/423698585 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2c54d0510 0x55f2c54f7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:07.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.094+0000 7f7e317fa640 0 client.8823 ms_handle_reset on v2:172.21.15.32:6832/4273183179 2024-04-24T23:09:07.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.094+0000 7f7e5b98a740 1 -- 172.21.15.32:0/423698585 >> 172.21.15.32:0/423698585 conn(0x55f2c54b1b60 msgr2=0x55f2c54ac250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:07.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.094+0000 7f7e5b98a740 1 -- 172.21.15.32:0/423698585 shutdown_connections 2024-04-24T23:09:07.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.094+0000 7f7e5b98a740 1 -- 172.21.15.32:0/423698585 wait complete. 2024-04-24T23:09:07.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.094+0000 7f7e5b98a740 20 client.8823 trim_cache size 0 max 16384 2024-04-24T23:09:07.104 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:09:07.104 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", 2024-04-24T23:09:07.104 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2024-04-24T23:09:07.104 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 8679, 2024-04-24T23:09:07.104 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-04-24T23:09:07.104 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-04-24T23:09:07.105 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-04-24T23:09:07.105 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 5.6712343770000002, 2024-04-24T23:09:07.105 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 73, 2024-04-24T23:09:07.105 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 169, 2024-04-24T23:09:07.105 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 169, 2024-04-24T23:09:07.105 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 10.707356697 2024-04-24T23:09:07.105 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:09:07.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.095+0000 7f7e58bab640 1 -- 172.21.15.32:0/1439124221 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7e3406be60 msgr2=0x7f7e3406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:07.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.095+0000 7f7e58bab640 1 --2- 172.21.15.32:0/1439124221 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7e3406be60 0x7f7e3406e320 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c006e80 tx=0x7f7e4c006df0 comp rx=0 tx=0).stop 2024-04-24T23:09:07.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.095+0000 7f7e58bab640 1 -- 172.21.15.32:0/1439124221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e5415d0b0 msgr2=0x7f7e5415d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:07.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.095+0000 7f7e58bab640 1 --2- 172.21.15.32:0/1439124221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e5415d0b0 0x7f7e5415d530 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f7e4400e230 tx=0x7f7e44009750 comp rx=0 tx=0).stop 2024-04-24T23:09:07.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.095+0000 7f7e58bab640 1 -- 172.21.15.32:0/1439124221 shutdown_connections 2024-04-24T23:09:07.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.095+0000 7f7e58bab640 1 --2- 172.21.15.32:0/1439124221 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7e3406be60 0x7f7e3406e320 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:07.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.095+0000 7f7e58bab640 1 --2- 172.21.15.32:0/1439124221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e5415d0b0 0x7f7e5415d530 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:07.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.095+0000 7f7e58bab640 1 --2- 172.21.15.32:0/1439124221 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e541529f0 0x7f7e54071eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:07.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.095+0000 7f7e58bab640 1 --2- 172.21.15.32:0/1439124221 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e40006f50 0x7f7e541440b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:07.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.095+0000 7f7e58bab640 1 -- 172.21.15.32:0/1439124221 >> 172.21.15.32:0/1439124221 conn(0x7f7e5406b110 msgr2=0x7f7e540cf1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:07.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.095+0000 7f7e58bab640 1 -- 172.21.15.32:0/1439124221 shutdown_connections 2024-04-24T23:09:07.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.095+0000 7f7e58bab640 1 -- 172.21.15.32:0/1439124221 wait complete. 2024-04-24T23:09:07.108 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T23:09:07.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.207+0000 7f5adfa88640 1 Processor -- start 2024-04-24T23:09:07.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.208+0000 7f5adfa88640 1 -- start start 2024-04-24T23:09:07.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.208+0000 7f5adfa88640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5ad806b820 0x7f5ad80d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:07.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.208+0000 7f5adfa88640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5ad80d4210 0x7f5ad80b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:07.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.208+0000 7f5adfa88640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ad80d3170 con 0x7f5ad815e9b0 2024-04-24T23:09:07.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.208+0000 7f5adfa88640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f5ad80d3450 con 0x7f5ad80d4210 2024-04-24T23:09:07.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.208+0000 7f5adfa88640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f5ad80d3730 con 0x7f5ad806b820 2024-04-24T23:09:07.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.209+0000 7f5ade025640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5ad80d4210 0x7f5ad80b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:07.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.209+0000 7f5ade025640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5ad80d4210 0x7f5ad80b78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40346/0 (socket says 172.21.15.32:40346) 2024-04-24T23:09:07.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.209+0000 7f5ade025640 1 -- 172.21.15.32:0/4152683841 learned_addr learned my addr 172.21.15.32:0/4152683841 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:09:07.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.209+0000 7f5add023640 1 --2- 172.21.15.32:0/4152683841 >> v2:172.21.15.121:3300/0 conn(0x7f5ad806b820 0x7f5ad80d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:07.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.209+0000 7f5ade025640 1 -- 172.21.15.32:0/4152683841 >> v2:172.21.15.121:3300/0 conn(0x7f5ad806b820 msgr2=0x7f5ad80d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:09:07.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.209+0000 7f5ade025640 1 --2- 172.21.15.32:0/4152683841 >> v2:172.21.15.121:3300/0 conn(0x7f5ad806b820 0x7f5ad80d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:07.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.209+0000 7f5ade025640 1 -- 172.21.15.32:0/4152683841 >> v1:172.21.15.121:6789/0 conn(0x7f5ad815e9b0 legacy=0x7f5ad806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:07.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.209+0000 7f5ade025640 1 -- 172.21.15.32:0/4152683841 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ad80d3a10 con 0x7f5ad80d4210 2024-04-24T23:09:07.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.210+0000 7f5ade025640 1 --2- 172.21.15.32:0/4152683841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ad80d4210 0x7f5ad80b78f0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f5ad80730f0 tx=0x7f5ad002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0a7ccf6ce671b54 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:07.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.210+0000 7f5adc822640 1 -- 172.21.15.32:0/4152683841 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad0036070 con 0x7f5ad80d4210 2024-04-24T23:09:07.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.210+0000 7f5adc822640 1 -- 172.21.15.32:0/4152683841 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ad0031600 con 0x7f5ad80d4210 2024-04-24T23:09:07.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.210+0000 7f5adc822640 1 -- 172.21.15.32:0/4152683841 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad00319d0 con 0x7f5ad80d4210 2024-04-24T23:09:07.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.210+0000 7f5adfa88640 1 -- 172.21.15.32:0/4152683841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ad80d4210 msgr2=0x7f5ad80b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:07.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.210+0000 7f5adfa88640 1 --2- 172.21.15.32:0/4152683841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ad80d4210 0x7f5ad80b78f0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f5ad80730f0 tx=0x7f5ad002a6a0 comp rx=0 tx=0).stop 2024-04-24T23:09:07.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.210+0000 7f5adfa88640 1 -- 172.21.15.32:0/4152683841 shutdown_connections 2024-04-24T23:09:07.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.210+0000 7f5adfa88640 1 --2- 172.21.15.32:0/4152683841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ad80d4210 0x7f5ad80b78f0 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:07.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.210+0000 7f5adfa88640 1 --2- 172.21.15.32:0/4152683841 >> v2:172.21.15.121:3300/0 conn(0x7f5ad806b820 0x7f5ad80d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:07.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.210+0000 7f5adfa88640 1 -- 172.21.15.32:0/4152683841 >> 172.21.15.32:0/4152683841 conn(0x7f5ad80d11d0 msgr2=0x7f5ad80d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:07.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.210+0000 7f5adfa88640 1 -- 172.21.15.32:0/4152683841 shutdown_connections 2024-04-24T23:09:07.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.210+0000 7f5adfa88640 1 -- 172.21.15.32:0/4152683841 wait complete. 2024-04-24T23:09:07.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.212+0000 7f5adfa88640 1 Processor -- start 2024-04-24T23:09:07.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.212+0000 7f5adfa88640 1 -- start start 2024-04-24T23:09:07.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.212+0000 7f5adfa88640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ad806b820 0x7f5ad81e3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:07.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.212+0000 7f5adfa88640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ad80d4210 0x7f5ad80ca220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:07.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.212+0000 7f5adfa88640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5ad815e9b0 0x7f5ad81e44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:07.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.212+0000 7f5adfa88640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad80ba0e0 con 0x7f5ad806b820 2024-04-24T23:09:07.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.212+0000 7f5adfa88640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad80d2400 con 0x7f5ad80d4210 2024-04-24T23:09:07.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.212+0000 7f5adfa88640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f5ad80ba260 con 0x7f5ad815e9b0 2024-04-24T23:09:07.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.212+0000 7f5add824640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ad806b820 0x7f5ad81e3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:07.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.212+0000 7f5add824640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ad806b820 0x7f5ad81e3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40348/0 (socket says 172.21.15.32:40348) 2024-04-24T23:09:07.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.212+0000 7f5add824640 1 -- 172.21.15.32:0/732838658 learned_addr learned my addr 172.21.15.32:0/732838658 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:09:07.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.213+0000 7f5add023640 1 --2- 172.21.15.32:0/732838658 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ad80d4210 0x7f5ad80ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:07.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.213+0000 7f5ade025640 1 --2- 172.21.15.32:0/732838658 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5ad815e9b0 0x7f5ad81e44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:07.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.213+0000 7f5add824640 1 -- 172.21.15.32:0/732838658 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5ad815e9b0 msgr2=0x7f5ad81e44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:07.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.213+0000 7f5add824640 1 --2- 172.21.15.32:0/732838658 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5ad815e9b0 0x7f5ad81e44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:07.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.213+0000 7f5add824640 1 -- 172.21.15.32:0/732838658 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ad80d4210 msgr2=0x7f5ad80ca220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:07.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.213+0000 7f5add824640 1 --2- 172.21.15.32:0/732838658 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ad80d4210 0x7f5ad80ca220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:07.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.213+0000 7f5add824640 1 -- 172.21.15.32:0/732838658 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ad8148b00 con 0x7f5ad806b820 2024-04-24T23:09:07.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.213+0000 7f5ade025640 1 --2- 172.21.15.32:0/732838658 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5ad815e9b0 0x7f5ad81e44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:09:07.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.213+0000 7f5add824640 1 --2- 172.21.15.32:0/732838658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ad806b820 0x7f5ad81e3aa0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f5ad400c3f0 tx=0x7f5ad400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:07.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.213+0000 7f5ace7fc640 1 -- 172.21.15.32:0/732838658 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad40052d0 con 0x7f5ad806b820 2024-04-24T23:09:07.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.213+0000 7f5ace7fc640 1 -- 172.21.15.32:0/732838658 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ad4005490 con 0x7f5ad806b820 2024-04-24T23:09:07.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.213+0000 7f5adfa88640 1 -- 172.21.15.32:0/732838658 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ad8148de0 con 0x7f5ad806b820 2024-04-24T23:09:07.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.213+0000 7f5ace7fc640 1 -- 172.21.15.32:0/732838658 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad4007260 con 0x7f5ad806b820 2024-04-24T23:09:07.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.213+0000 7f5adfa88640 1 -- 172.21.15.32:0/732838658 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ad81493a0 con 0x7f5ad806b820 2024-04-24T23:09:07.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.214+0000 7f5adfa88640 1 -- 172.21.15.32:0/732838658 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ad8068430 con 0x7f5ad806b820 2024-04-24T23:09:07.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.215+0000 7f5ace7fc640 1 -- 172.21.15.32:0/732838658 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5ad4007400 con 0x7f5ad806b820 2024-04-24T23:09:07.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.215+0000 7f5ace7fc640 1 --2- 172.21.15.32:0/732838658 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5abc06be30 0x7f5abc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:07.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.215+0000 7f5ace7fc640 1 -- 172.21.15.32:0/732838658 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 5709+0+0 (secure 0 0 0) 0x7f5ad408a210 con 0x7f5ad806b820 2024-04-24T23:09:07.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.215+0000 7f5add023640 1 --2- 172.21.15.32:0/732838658 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5abc06be30 0x7f5abc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:07.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.216+0000 7f5add023640 1 --2- 172.21.15.32:0/732838658 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5abc06be30 0x7f5abc06e2f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f5ad80b8a00 tx=0x7f5ac8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:07.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.219+0000 7f5ace7fc640 1 -- 172.21.15.32:0/732838658 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5ad4058600 con 0x7f5ad806b820 2024-04-24T23:09:07.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.440+0000 7f5adfa88640 1 -- 172.21.15.32:0/732838658 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5ad806b240 con 0x7f5ad806b820 2024-04-24T23:09:07.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.441+0000 7f5ace7fc640 1 -- 172.21.15.32:0/732838658 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v73) ==== 53+0+73 (secure 0 0 0) 0x7f5ad405c900 con 0x7f5ad806b820 2024-04-24T23:09:07.442 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T23:09:07.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.443+0000 7f5adfa88640 1 -- 172.21.15.32:0/732838658 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5abc06be30 msgr2=0x7f5abc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:07.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.443+0000 7f5adfa88640 1 --2- 172.21.15.32:0/732838658 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5abc06be30 0x7f5abc06e2f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f5ad80b8a00 tx=0x7f5ac8008040 comp rx=0 tx=0).stop 2024-04-24T23:09:07.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.443+0000 7f5adfa88640 1 -- 172.21.15.32:0/732838658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ad806b820 msgr2=0x7f5ad81e3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:07.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.443+0000 7f5adfa88640 1 --2- 172.21.15.32:0/732838658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ad806b820 0x7f5ad81e3aa0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f5ad400c3f0 tx=0x7f5ad400c8b0 comp rx=0 tx=0).stop 2024-04-24T23:09:07.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.443+0000 7f5adfa88640 1 -- 172.21.15.32:0/732838658 shutdown_connections 2024-04-24T23:09:07.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.443+0000 7f5adfa88640 1 --2- 172.21.15.32:0/732838658 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5abc06be30 0x7f5abc06e2f0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:07.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.443+0000 7f5adfa88640 1 --2- 172.21.15.32:0/732838658 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5ad815e9b0 0x7f5ad81e44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:07.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.443+0000 7f5adfa88640 1 --2- 172.21.15.32:0/732838658 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ad80d4210 0x7f5ad80ca220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:07.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.443+0000 7f5adfa88640 1 --2- 172.21.15.32:0/732838658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ad806b820 0x7f5ad81e3aa0 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:07.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.443+0000 7f5adfa88640 1 -- 172.21.15.32:0/732838658 >> 172.21.15.32:0/732838658 conn(0x7f5ad80d11d0 msgr2=0x7f5ad80d15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:07.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.443+0000 7f5adfa88640 1 -- 172.21.15.32:0/732838658 shutdown_connections 2024-04-24T23:09:07.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07.443+0000 7f5adfa88640 1 -- 172.21.15.32:0/732838658 wait complete. 2024-04-24T23:09:07.455 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-24T23:09:07.455 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T23:09:07.455 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2024-04-24T23:09:07.456 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-24T23:09:07.456 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T23:09:07.456 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-24T23:09:07.456 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-24T23:09:07.456 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T23:09:07.456 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:09:07.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:09:07.456 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2024-04-24T23:09:07.472 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T23:09:07.472 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T23:09:07.472 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T23:09:07.472 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:09:07.472 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2024-04-24T23:09:07.472 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:09:07.472 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:09:07.472 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2024-04-24T23:09:07.472 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2024-04-24T23:09:07.472 INFO:teuthology.orchestra.run.smithi032.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:09:07.473 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2024-04-24T23:09:07.473 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2024-04-24T23:09:07.473 INFO:teuthology.orchestra.run.smithi032.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T23:09:07.473 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:09:07.473 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2024-04-24T23:09:07.473 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-24T23:09:07.473 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 42161sec preferred_lft 42161sec 2024-04-24T23:09:07.473 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link noprefixroute 2024-04-24T23:09:07.473 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:09:07.473 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:09:07.473 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:09:07.473 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2024-04-24T23:09:07.473 INFO:teuthology.orchestra.run.smithi032.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T23:09:07.473 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 72:d4:96:00:43:8c brd ff:ff:ff:ff:ff:ff 2024-04-24T23:09:07.473 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-24T23:09:07.474 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:09:07.474 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::f4c6:20ff:fe22:2647/64 scope link 2024-04-24T23:09:07.474 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:09:07.474 INFO:teuthology.orchestra.run.smithi032.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:09:07.474 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 72:d4:96:00:43:8c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T23:09:07.474 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::70d4:96ff:fe00:438c/64 scope link 2024-04-24T23:09:07.474 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:09:07.474 INFO:teuthology.orchestra.run.smithi032.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:09:07.474 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether e6:6b:2a:90:1a:f8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T23:09:07.474 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::e46b:2aff:fe90:1af8/64 scope link 2024-04-24T23:09:07.474 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:09:07.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:09:07.475 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T23:09:07.535 INFO:teuthology.orchestra.run.smithi032.stdout:1 2024-04-24T23:09:07.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:09:07.537 DEBUG:teuthology.orchestra.run.smithi032:> route 2024-04-24T23:09:07.593 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2024-04-24T23:09:07.593 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T23:09:07.593 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-24T23:09:07.593 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-24T23:09:07.593 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T23:09:07.594 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:09:07.594 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-24T23:09:07.594 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-24T23:09:07.594 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-24T23:09:07.594 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-24T23:09:07.594 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-24T23:09:07.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:09:07.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:09:07.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:09:07.720 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-04-24T23:09:07.735 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-24T23:09:07.735 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-24T23:09:07.735 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:09:07.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:09:07.736 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2024-04-24T23:09:07.759 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:09:07.778 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:09:07.800 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:09:07.800 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-24T23:09:07.874 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:09:07.875 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:09:07.875 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-24T23:09:07.930 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-24T23:09:07.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:09:07.930 DEBUG:teuthology.orchestra.run.smithi032:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:09:08.013 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:09:08.013 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-24T23:09:08.023 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:09:08.023 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-24T23:09:08.023 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.121:3300,172.21.15.121:3301" 2024-04-24T23:09:08.023 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: trying mount with new device syntax: 0@eb1b7a01-6f65-4534-9a12-83011862ba51.cephfs=/ 2024-04-24T23:09:08.023 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.32:3300/172.21.15.121:3300/172.21.15.121:3301" will pass to kernel 2024-04-24T23:09:08.024 INFO:teuthology.orchestra.run.smithi032.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-24T23:09:08.024 INFO:teuthology.orchestra.run.smithi032.stdout: You just mounted a file system that supports labels which does not 2024-04-24T23:09:08.024 INFO:teuthology.orchestra.run.smithi032.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-24T23:09:08.024 INFO:teuthology.orchestra.run.smithi032.stdout: applications will generate AVC messages and not be allowed access to 2024-04-24T23:09:08.025 INFO:teuthology.orchestra.run.smithi032.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-24T23:09:08.027 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-24T23:09:08.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:09:08.027 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-24T23:09:08.094 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:09:08.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:09:08.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:09:08.195 INFO:teuthology.orchestra.run.smithi032.stdout:client8832 2024-04-24T23:09:08.196 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:09:08.196 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:09:08.211 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T23:09:08.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.353+0000 7f6ef04ae640 1 Processor -- start 2024-04-24T23:09:08.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.353+0000 7f6ef04ae640 1 -- start start 2024-04-24T23:09:08.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.354+0000 7f6ef04ae640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ee815e9b0 0x7f6ee80d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:08.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.354+0000 7f6ef04ae640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6ee80d4210 0x7f6ee806b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:08.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.354+0000 7f6ef04ae640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ee80d3170 con 0x7f6ee80ba0e0 2024-04-24T23:09:08.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.354+0000 7f6ef04ae640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f6ee80b8900 con 0x7f6ee80d4210 2024-04-24T23:09:08.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.354+0000 7f6ef04ae640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f6ee80d3450 con 0x7f6ee815e9b0 2024-04-24T23:09:08.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.354+0000 7f6eeda49640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6ee80d4210 0x7f6ee806b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:08.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.354+0000 7f6eee24a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ee815e9b0 0x7f6ee80d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:08.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.354+0000 7f6eeea4b640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f6ee80ba0e0 0x7f6ee8071730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:53912/0 (socket says 172.21.15.32:53912) 2024-04-24T23:09:08.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.354+0000 7f6eeea4b640 1 -- 172.21.15.32:0/11089704 learned_addr learned my addr 172.21.15.32:0/11089704 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:09:08.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.354+0000 7f6eed248640 1 -- 172.21.15.32:0/11089704 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1542167005 0 0) 0x7f6ee80d3170 con 0x7f6ee80ba0e0 2024-04-24T23:09:08.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.354+0000 7f6eed248640 1 -- 172.21.15.32:0/11089704 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ed4003520 con 0x7f6ee80ba0e0 2024-04-24T23:09:08.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.354+0000 7f6eeda49640 1 -- 172.21.15.32:0/11089704 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ee815e9b0 msgr2=0x7f6ee80d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:09:08.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.355+0000 7f6eeda49640 1 --2- 172.21.15.32:0/11089704 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ee815e9b0 0x7f6ee80d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:08.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.355+0000 7f6eeda49640 1 -- 172.21.15.32:0/11089704 >> v1:172.21.15.121:6789/0 conn(0x7f6ee80ba0e0 legacy=0x7f6ee8071730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:08.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.355+0000 7f6eeda49640 1 -- 172.21.15.32:0/11089704 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ee80d3730 con 0x7f6ee80d4210 2024-04-24T23:09:08.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.355+0000 7f6eee24a640 1 --2- 172.21.15.32:0/11089704 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ee815e9b0 0x7f6ee80d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:09:08.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.355+0000 7f6eeda49640 1 --2- 172.21.15.32:0/11089704 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ee80d4210 0x7f6ee806b110 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f6ed80097a0 tx=0x7f6ed802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=949f757e8f3a160f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:08.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.356+0000 7f6eed248640 1 -- 172.21.15.32:0/11089704 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed8037070 con 0x7f6ee80d4210 2024-04-24T23:09:08.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.356+0000 7f6eed248640 1 -- 172.21.15.32:0/11089704 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ed8032440 con 0x7f6ee80d4210 2024-04-24T23:09:08.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.356+0000 7f6eed248640 1 -- 172.21.15.32:0/11089704 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed80327f0 con 0x7f6ee80d4210 2024-04-24T23:09:08.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.356+0000 7f6ef04ae640 1 -- 172.21.15.32:0/11089704 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ee80d4210 msgr2=0x7f6ee806b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:08.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.356+0000 7f6ef04ae640 1 --2- 172.21.15.32:0/11089704 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ee80d4210 0x7f6ee806b110 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f6ed80097a0 tx=0x7f6ed802a180 comp rx=0 tx=0).stop 2024-04-24T23:09:08.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.356+0000 7f6ef04ae640 1 -- 172.21.15.32:0/11089704 shutdown_connections 2024-04-24T23:09:08.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.356+0000 7f6ef04ae640 1 --2- 172.21.15.32:0/11089704 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ee80d4210 0x7f6ee806b110 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:08.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.356+0000 7f6ef04ae640 1 --2- 172.21.15.32:0/11089704 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ee815e9b0 0x7f6ee80d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:08.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.356+0000 7f6ef04ae640 1 -- 172.21.15.32:0/11089704 >> 172.21.15.32:0/11089704 conn(0x7f6ee80d11d0 msgr2=0x7f6ee80d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:08.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.356+0000 7f6ef04ae640 1 -- 172.21.15.32:0/11089704 shutdown_connections 2024-04-24T23:09:08.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.356+0000 7f6ef04ae640 1 -- 172.21.15.32:0/11089704 wait complete. 2024-04-24T23:09:08.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.357+0000 7f6ef04ae640 1 Processor -- start 2024-04-24T23:09:08.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.357+0000 7f6ef04ae640 1 -- start start 2024-04-24T23:09:08.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.358+0000 7f6ef04ae640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ee80ba0e0 0x7f6ee8154eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:08.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.358+0000 7f6ef04ae640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ee80d4210 0x7f6ee8159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:08.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.358+0000 7f6ef04ae640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ee815e9b0 0x7f6ee814b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:08.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.358+0000 7f6ef04ae640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee80d2400 con 0x7f6ee80d4210 2024-04-24T23:09:08.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.358+0000 7f6ef04ae640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee80d3cf0 con 0x7f6ee80ba0e0 2024-04-24T23:09:08.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.358+0000 7f6ef04ae640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f6ee806b650 con 0x7f6ee815e9b0 2024-04-24T23:09:08.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.358+0000 7f6eeda49640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ee80d4210 0x7f6ee8159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:08.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.358+0000 7f6eeda49640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ee80d4210 0x7f6ee8159ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40356/0 (socket says 172.21.15.32:40356) 2024-04-24T23:09:08.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.358+0000 7f6eeda49640 1 -- 172.21.15.32:0/1686833457 learned_addr learned my addr 172.21.15.32:0/1686833457 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:09:08.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.358+0000 7f6eee24a640 1 --2- 172.21.15.32:0/1686833457 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ee80ba0e0 0x7f6ee8154eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:08.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.358+0000 7f6eeea4b640 1 --2- 172.21.15.32:0/1686833457 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ee815e9b0 0x7f6ee814b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:08.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.359+0000 7f6eeda49640 1 -- 172.21.15.32:0/1686833457 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ee815e9b0 msgr2=0x7f6ee814b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:08.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.359+0000 7f6eeda49640 1 --2- 172.21.15.32:0/1686833457 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ee815e9b0 0x7f6ee814b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:08.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.359+0000 7f6eeda49640 1 -- 172.21.15.32:0/1686833457 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ee80ba0e0 msgr2=0x7f6ee8154eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:08.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.359+0000 7f6eeda49640 1 --2- 172.21.15.32:0/1686833457 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ee80ba0e0 0x7f6ee8154eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:08.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.359+0000 7f6eeda49640 1 -- 172.21.15.32:0/1686833457 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ee81f4900 con 0x7f6ee80d4210 2024-04-24T23:09:08.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.359+0000 7f6eee24a640 1 --2- 172.21.15.32:0/1686833457 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ee80ba0e0 0x7f6ee8154eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:09:08.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.359+0000 7f6eeda49640 1 --2- 172.21.15.32:0/1686833457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ee80d4210 0x7f6ee8159ca0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f6ed8035040 tx=0x7f6ed80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:08.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.359+0000 7f6edeffd640 1 -- 172.21.15.32:0/1686833457 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed8037040 con 0x7f6ee80d4210 2024-04-24T23:09:08.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.359+0000 7f6edeffd640 1 -- 172.21.15.32:0/1686833457 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ed80044c0 con 0x7f6ee80d4210 2024-04-24T23:09:08.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.359+0000 7f6ef04ae640 1 -- 172.21.15.32:0/1686833457 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ee81f4be0 con 0x7f6ee80d4210 2024-04-24T23:09:08.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.359+0000 7f6edeffd640 1 -- 172.21.15.32:0/1686833457 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed80325f0 con 0x7f6ee80d4210 2024-04-24T23:09:08.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.359+0000 7f6ef04ae640 1 -- 172.21.15.32:0/1686833457 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ee81f51a0 con 0x7f6ee80d4210 2024-04-24T23:09:08.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.360+0000 7f6ef04ae640 1 -- 172.21.15.32:0/1686833457 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ee8068430 con 0x7f6ee80d4210 2024-04-24T23:09:08.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.361+0000 7f6edeffd640 1 -- 172.21.15.32:0/1686833457 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6ed8044020 con 0x7f6ee80d4210 2024-04-24T23:09:08.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.361+0000 7f6edeffd640 1 --2- 172.21.15.32:0/1686833457 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6ecc06be30 0x7f6ecc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:08.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.362+0000 7f6edeffd640 1 -- 172.21.15.32:0/1686833457 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 5709+0+0 (secure 0 0 0) 0x7f6ed80aaf10 con 0x7f6ee80d4210 2024-04-24T23:09:08.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.362+0000 7f6eee24a640 1 --2- 172.21.15.32:0/1686833457 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6ecc06be30 0x7f6ecc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:08.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.362+0000 7f6eee24a640 1 --2- 172.21.15.32:0/1686833457 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6ecc06be30 0x7f6ecc06e2f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4009560 tx=0x7f6ee4009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:08.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.366+0000 7f6edeffd640 1 -- 172.21.15.32:0/1686833457 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6ed8079270 con 0x7f6ee80d4210 2024-04-24T23:09:08.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.593+0000 7f6ef04ae640 1 -- 172.21.15.32:0/1686833457 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6ee806ba20 con 0x7f6ee80d4210 2024-04-24T23:09:08.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.594+0000 7f6edeffd640 1 -- 172.21.15.32:0/1686833457 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v73) ==== 53+0+73 (secure 0 0 0) 0x7f6ed807d570 con 0x7f6ee80d4210 2024-04-24T23:09:08.595 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T23:09:08.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.596+0000 7f6ef04ae640 1 -- 172.21.15.32:0/1686833457 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6ecc06be30 msgr2=0x7f6ecc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:08.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.596+0000 7f6ef04ae640 1 --2- 172.21.15.32:0/1686833457 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6ecc06be30 0x7f6ecc06e2f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4009560 tx=0x7f6ee4009190 comp rx=0 tx=0).stop 2024-04-24T23:09:08.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.596+0000 7f6ef04ae640 1 -- 172.21.15.32:0/1686833457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ee80d4210 msgr2=0x7f6ee8159ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:08.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.596+0000 7f6ef04ae640 1 --2- 172.21.15.32:0/1686833457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ee80d4210 0x7f6ee8159ca0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f6ed8035040 tx=0x7f6ed80046c0 comp rx=0 tx=0).stop 2024-04-24T23:09:08.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.596+0000 7f6ef04ae640 1 -- 172.21.15.32:0/1686833457 shutdown_connections 2024-04-24T23:09:08.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.596+0000 7f6ef04ae640 1 --2- 172.21.15.32:0/1686833457 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6ecc06be30 0x7f6ecc06e2f0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:08.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.596+0000 7f6ef04ae640 1 --2- 172.21.15.32:0/1686833457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ee80d4210 0x7f6ee8159ca0 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:08.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.596+0000 7f6ef04ae640 1 --2- 172.21.15.32:0/1686833457 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ee815e9b0 0x7f6ee814b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:08.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.596+0000 7f6ef04ae640 1 --2- 172.21.15.32:0/1686833457 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ee80ba0e0 0x7f6ee8154eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:08.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.596+0000 7f6ef04ae640 1 -- 172.21.15.32:0/1686833457 >> 172.21.15.32:0/1686833457 conn(0x7f6ee80d11d0 msgr2=0x7f6ee80b8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:08.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.596+0000 7f6ef04ae640 1 -- 172.21.15.32:0/1686833457 shutdown_connections 2024-04-24T23:09:08.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:08.596+0000 7f6ef04ae640 1 -- 172.21.15.32:0/1686833457 wait complete. 2024-04-24T23:09:08.609 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-24T23:09:08.609 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T23:09:08.609 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi121.front.sepia.ceph.com 2024-04-24T23:09:08.609 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-24T23:09:08.609 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-24T23:09:08.609 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-24T23:09:08.609 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-24T23:09:08.610 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T23:09:08.610 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-24T23:09:08.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:09:08.610 DEBUG:teuthology.orchestra.run.smithi121:> ip addr 2024-04-24T23:09:08.629 INFO:teuthology.orchestra.run.smithi121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T23:09:08.629 INFO:teuthology.orchestra.run.smithi121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T23:09:08.630 INFO:teuthology.orchestra.run.smithi121.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T23:09:08.630 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:09:08.630 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 ::1/128 scope host 2024-04-24T23:09:08.630 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:09:08.630 INFO:teuthology.orchestra.run.smithi121.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:09:08.630 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:82 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:09:08.630 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f0 2024-04-24T23:09:08.630 INFO:teuthology.orchestra.run.smithi121.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:09:08.630 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:83 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:09:08.630 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f1 2024-04-24T23:09:08.630 INFO:teuthology.orchestra.run.smithi121.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:09:08.630 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a2 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:09:08.630 INFO:teuthology.orchestra.run.smithi121.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T23:09:08.630 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a3 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:09:08.631 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.21.15.121/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-24T23:09:08.631 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft 42182sec preferred_lft 42182sec 2024-04-24T23:09:08.631 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::93dd:8489:4369:e127/64 scope link noprefixroute 2024-04-24T23:09:08.631 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:09:08.631 INFO:teuthology.orchestra.run.smithi121.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T23:09:08.631 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether c6:c0:c7:45:ae:ba brd ff:ff:ff:ff:ff:ff 2024-04-24T23:09:08.631 INFO:teuthology.orchestra.run.smithi121.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-24T23:09:08.631 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:09:08.631 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::80fe:d2ff:fe13:d1/64 scope link 2024-04-24T23:09:08.631 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:09:08.631 INFO:teuthology.orchestra.run.smithi121.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:09:08.631 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether e2:ca:b9:e9:9e:bb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T23:09:08.631 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::e0ca:b9ff:fee9:9ebb/64 scope link 2024-04-24T23:09:08.631 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:09:08.632 INFO:teuthology.orchestra.run.smithi121.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:09:08.632 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether c6:c0:c7:45:ae:ba brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-24T23:09:08.632 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::c4c0:c7ff:fe45:aeba/64 scope link 2024-04-24T23:09:08.632 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:09:08.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:09:08.633 DEBUG:teuthology.orchestra.run.smithi121:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T23:09:08.696 INFO:teuthology.orchestra.run.smithi121.stdout:1 2024-04-24T23:09:08.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:09:08.698 DEBUG:teuthology.orchestra.run.smithi121:> route 2024-04-24T23:09:08.756 INFO:teuthology.orchestra.run.smithi121.stdout:Kernel IP routing table 2024-04-24T23:09:08.756 INFO:teuthology.orchestra.run.smithi121.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T23:09:08.756 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-24T23:09:08.756 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-24T23:09:08.757 INFO:teuthology.orchestra.run.smithi121.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T23:09:08.757 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:09:08.757 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2024-04-24T23:09:08.757 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-24T23:09:08.757 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-24T23:09:08.757 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-24T23:09:08.757 DEBUG:teuthology.orchestra.run.smithi121:> ') 2024-04-24T23:09:08.850 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T23:09:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:09:08.881 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T23:09:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:09:08.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:09:08.887 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2024-04-24T23:09:08.902 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-24T23:09:08.902 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-24T23:09:08.902 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-24T23:09:08.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:09:08.903 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set brx.0 up 2024-04-24T23:09:08.967 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:09:08.967 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-24T23:09:09.022 INFO:teuthology.orchestra.run.smithi121.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-24T23:09:09.022 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:09:09.023 DEBUG:teuthology.orchestra.run.smithi121:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-24T23:09:09.077 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-24T23:09:09.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:09:09.077 DEBUG:teuthology.orchestra.run.smithi121:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:09:09.159 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:09:09.159 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-24T23:09:09.171 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:09:09.171 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-24T23:09:09.171 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.121:3300,172.21.15.121:3301" 2024-04-24T23:09:09.171 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: trying mount with new device syntax: 1@eb1b7a01-6f65-4534-9a12-83011862ba51.cephfs=/ 2024-04-24T23:09:09.171 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.32:3300/172.21.15.121:3300/172.21.15.121:3301" will pass to kernel 2024-04-24T23:09:09.171 INFO:teuthology.orchestra.run.smithi121.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-24T23:09:09.171 INFO:teuthology.orchestra.run.smithi121.stdout: You just mounted a file system that supports labels which does not 2024-04-24T23:09:09.171 INFO:teuthology.orchestra.run.smithi121.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-24T23:09:09.171 INFO:teuthology.orchestra.run.smithi121.stdout: applications will generate AVC messages and not be allowed access to 2024-04-24T23:09:09.171 INFO:teuthology.orchestra.run.smithi121.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-24T23:09:09.172 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-24T23:09:09.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:09:09.173 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-24T23:09:09.237 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:09:09.325 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T23:09:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:09:09.326 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T23:09:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:09:09.332 INFO:teuthology.orchestra.run.smithi121.stdout:client8838 2024-04-24T23:09:09.333 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:09:09.333 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:09:09.348 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:09:09.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.452+0000 7fb3570f5640 1 Processor -- start 2024-04-24T23:09:09.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.452+0000 7fb3570f5640 1 -- start start 2024-04-24T23:09:09.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.453+0000 7fb3570f5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb35015edd0 0x7fb3500b5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:09.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.453+0000 7fb3570f5640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3500cd910 con 0x7fb3500b5560 2024-04-24T23:09:09.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.453+0000 7fb3570f5640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3500cdbf0 con 0x7fb3500cd3c0 2024-04-24T23:09:09.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.453+0000 7fb3570f5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fb3500cded0 con 0x7fb35015edd0 2024-04-24T23:09:09.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.453+0000 7fb34ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb35015edd0 0x7fb3500b5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:09.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.453+0000 7fb354e91640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb3500b5560 0x7fb35015e6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40656/0 (socket says 172.21.15.32:40656) 2024-04-24T23:09:09.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.453+0000 7fb354e91640 1 -- 172.21.15.32:0/1326152758 learned_addr learned my addr 172.21.15.32:0/1326152758 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:09:09.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.453+0000 7fb34ffff640 1 -- 172.21.15.32:0/1326152758 >> v1:172.21.15.121:6789/0 conn(0x7fb3500cd3c0 legacy=0x7fb3500b5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:09.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.453+0000 7fb34ffff640 1 -- 172.21.15.32:0/1326152758 >> v1:172.21.15.32:6789/0 conn(0x7fb3500b5560 legacy=0x7fb35015e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:09.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.453+0000 7fb34f7fe640 1 -- 172.21.15.32:0/1326152758 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3144563603 0 0) 0x7fb3500cd910 con 0x7fb3500b5560 2024-04-24T23:09:09.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.453+0000 7fb34ffff640 1 -- 172.21.15.32:0/1326152758 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3500ce1b0 con 0x7fb35015edd0 2024-04-24T23:09:09.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.454+0000 7fb34ffff640 1 --2- 172.21.15.32:0/1326152758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb35015edd0 0x7fb3500b5940 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fb33c0097a0 tx=0x7fb33c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbe0cbfd176121cf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:09.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.454+0000 7fb34f7fe640 1 -- 172.21.15.32:0/1326152758 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33c037070 con 0x7fb35015edd0 2024-04-24T23:09:09.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.454+0000 7fb34f7fe640 1 -- 172.21.15.32:0/1326152758 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb33c032440 con 0x7fb35015edd0 2024-04-24T23:09:09.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.454+0000 7fb34f7fe640 1 -- 172.21.15.32:0/1326152758 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33c032810 con 0x7fb35015edd0 2024-04-24T23:09:09.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.455+0000 7fb3570f5640 1 -- 172.21.15.32:0/1326152758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb35015edd0 msgr2=0x7fb3500b5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:09.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.455+0000 7fb3570f5640 1 --2- 172.21.15.32:0/1326152758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb35015edd0 0x7fb3500b5940 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fb33c0097a0 tx=0x7fb33c02a230 comp rx=0 tx=0).stop 2024-04-24T23:09:09.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.455+0000 7fb3570f5640 1 -- 172.21.15.32:0/1326152758 shutdown_connections 2024-04-24T23:09:09.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.455+0000 7fb3570f5640 1 --2- 172.21.15.32:0/1326152758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb35015edd0 0x7fb3500b5940 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:09.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.455+0000 7fb3570f5640 1 -- 172.21.15.32:0/1326152758 >> 172.21.15.32:0/1326152758 conn(0x7fb35006b110 msgr2=0x7fb35006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:09.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.455+0000 7fb3570f5640 1 -- 172.21.15.32:0/1326152758 shutdown_connections 2024-04-24T23:09:09.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.455+0000 7fb3570f5640 1 -- 172.21.15.32:0/1326152758 wait complete. 2024-04-24T23:09:09.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.456+0000 7fb3570f5640 1 Processor -- start 2024-04-24T23:09:09.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.456+0000 7fb3570f5640 1 -- start start 2024-04-24T23:09:09.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.457+0000 7fb3570f5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3500b5560 0x7fb350150c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:09.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.457+0000 7fb3570f5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3500cd3c0 0x7fb350150570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:09.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.457+0000 7fb3570f5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb35015edd0 0x7fb35014c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:09.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.457+0000 7fb3570f5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb3500b44a0 con 0x7fb3500b5560 2024-04-24T23:09:09.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.457+0000 7fb3570f5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb3500b4320 con 0x7fb3500cd3c0 2024-04-24T23:09:09.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.457+0000 7fb3570f5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fb35006c5f0 con 0x7fb35015edd0 2024-04-24T23:09:09.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.457+0000 7fb354e91640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3500b5560 0x7fb350150c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:09.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.457+0000 7fb354e91640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3500b5560 0x7fb350150c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40374/0 (socket says 172.21.15.32:40374) 2024-04-24T23:09:09.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.457+0000 7fb354e91640 1 -- 172.21.15.32:0/3075988379 learned_addr learned my addr 172.21.15.32:0/3075988379 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:09:09.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.457+0000 7fb34ffff640 1 --2- 172.21.15.32:0/3075988379 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3500cd3c0 0x7fb350150570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:09.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.457+0000 7fb355692640 1 --2- 172.21.15.32:0/3075988379 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb35015edd0 0x7fb35014c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:09.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.457+0000 7fb354e91640 1 -- 172.21.15.32:0/3075988379 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb35015edd0 msgr2=0x7fb35014c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:09.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.457+0000 7fb354e91640 1 --2- 172.21.15.32:0/3075988379 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb35015edd0 0x7fb35014c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:09.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.457+0000 7fb354e91640 1 -- 172.21.15.32:0/3075988379 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3500cd3c0 msgr2=0x7fb350150570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:09.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.457+0000 7fb354e91640 1 --2- 172.21.15.32:0/3075988379 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3500cd3c0 0x7fb350150570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:09.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.457+0000 7fb354e91640 1 -- 172.21.15.32:0/3075988379 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb35014d730 con 0x7fb3500b5560 2024-04-24T23:09:09.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.457+0000 7fb355692640 1 --2- 172.21.15.32:0/3075988379 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb35015edd0 0x7fb35014c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:09:09.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.457+0000 7fb354e91640 1 --2- 172.21.15.32:0/3075988379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3500b5560 0x7fb350150c10 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fb34400c4d0 tx=0x7fb34400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:09.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.458+0000 7fb34d7fa640 1 -- 172.21.15.32:0/3075988379 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb344018070 con 0x7fb3500b5560 2024-04-24T23:09:09.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.458+0000 7fb34d7fa640 1 -- 172.21.15.32:0/3075988379 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3440074e0 con 0x7fb3500b5560 2024-04-24T23:09:09.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.458+0000 7fb3570f5640 1 -- 172.21.15.32:0/3075988379 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3501516e0 con 0x7fb3500b5560 2024-04-24T23:09:09.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.458+0000 7fb34d7fa640 1 -- 172.21.15.32:0/3075988379 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb344004a20 con 0x7fb3500b5560 2024-04-24T23:09:09.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.458+0000 7fb3570f5640 1 -- 172.21.15.32:0/3075988379 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb350151ca0 con 0x7fb3500b5560 2024-04-24T23:09:09.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.459+0000 7fb3570f5640 1 -- 172.21.15.32:0/3075988379 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb350068430 con 0x7fb3500b5560 2024-04-24T23:09:09.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.459+0000 7fb34d7fa640 1 -- 172.21.15.32:0/3075988379 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb34401b070 con 0x7fb3500b5560 2024-04-24T23:09:09.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.459+0000 7fb34d7fa640 1 --2- 172.21.15.32:0/3075988379 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb33406be30 0x7fb33406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:09.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.460+0000 7fb34d7fa640 1 -- 172.21.15.32:0/3075988379 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 5709+0+0 (secure 0 0 0) 0x7fb34408a040 con 0x7fb3500b5560 2024-04-24T23:09:09.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.460+0000 7fb34ffff640 1 --2- 172.21.15.32:0/3075988379 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb33406be30 0x7fb33406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:09.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.460+0000 7fb34ffff640 1 --2- 172.21.15.32:0/3075988379 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb33406be30 0x7fb33406e2f0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fb33c034000 tx=0x7fb33c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:09.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.464+0000 7fb34d7fa640 1 -- 172.21.15.32:0/3075988379 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb344058430 con 0x7fb3500b5560 2024-04-24T23:09:09.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.712+0000 7fb3570f5640 1 -- 172.21.15.32:0/3075988379 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb35006d5f0 con 0x7fb3500b5560 2024-04-24T23:09:09.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.713+0000 7fb34d7fa640 1 -- 172.21.15.32:0/3075988379 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+5185 (secure 0 0 0) 0x7fb34405c730 con 0x7fb3500b5560 2024-04-24T23:09:09.715 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:09:09.715 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":73,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6695,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/91086102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":91086102},{"type":"v1","addr":"172.21.15.121:6836","nonce":91086102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":6698,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1443743127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1443743127},{"type":"v1","addr":"172.21.15.121:6837","nonce":1443743127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":7423,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/4008773444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":4008773444},{"type":"v1","addr":"172.21.15.32:6835","nonce":4008773444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:09:00.770308+0000","modified":"2024-04-24T23:09:03.774877+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8679},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8679":{"gid":8679,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6834/4273183179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":4273183179},{"type":"v1","addr":"172.21.15.32:6834","nonce":4273183179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8679,"qdb_cluster":[8679]},"id":10}]} 2024-04-24T23:09:09.715 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 73 2024-04-24T23:09:09.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.715+0000 7fb3570f5640 1 -- 172.21.15.32:0/3075988379 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb33406be30 msgr2=0x7fb33406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:09.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.715+0000 7fb3570f5640 1 --2- 172.21.15.32:0/3075988379 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb33406be30 0x7fb33406e2f0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fb33c034000 tx=0x7fb33c02e000 comp rx=0 tx=0).stop 2024-04-24T23:09:09.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.715+0000 7fb3570f5640 1 -- 172.21.15.32:0/3075988379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3500b5560 msgr2=0x7fb350150c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:09.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.715+0000 7fb3570f5640 1 --2- 172.21.15.32:0/3075988379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3500b5560 0x7fb350150c10 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fb34400c4d0 tx=0x7fb34400c990 comp rx=0 tx=0).stop 2024-04-24T23:09:09.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.715+0000 7fb3570f5640 1 -- 172.21.15.32:0/3075988379 shutdown_connections 2024-04-24T23:09:09.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.715+0000 7fb3570f5640 1 --2- 172.21.15.32:0/3075988379 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb33406be30 0x7fb33406e2f0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:09.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.715+0000 7fb3570f5640 1 --2- 172.21.15.32:0/3075988379 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb35015edd0 0x7fb35014c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:09.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.715+0000 7fb3570f5640 1 --2- 172.21.15.32:0/3075988379 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3500cd3c0 0x7fb350150570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:09.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.715+0000 7fb3570f5640 1 --2- 172.21.15.32:0/3075988379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3500b5560 0x7fb350150c10 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:09.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.715+0000 7fb3570f5640 1 -- 172.21.15.32:0/3075988379 >> 172.21.15.32:0/3075988379 conn(0x7fb35006b110 msgr2=0x7fb3500b0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:09.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.715+0000 7fb3570f5640 1 -- 172.21.15.32:0/3075988379 shutdown_connections 2024-04-24T23:09:09.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.715+0000 7fb3570f5640 1 -- 172.21.15.32:0/3075988379 wait complete. 2024-04-24T23:09:09.728 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2024-04-24T23:09:09.848 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","whoami":0,"id":8679,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":8.4148190950000004,"mdsmap_epoch":73,"osdmap_epoch":169,"osdmap_epoch_barrier":169,"uptime":13.450943709000001} 2024-04-24T23:09:09.848 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", "fs_name": "cephfs", "id": 8679, "mdsmap_epoch": 73, "osdmap_epoch": 169, "osdmap_epoch_barrier": 169, "rank_uptime": 8.414819095, "state": "up:active", "uptime": 13.450943709, "want_state": "up:active", "whoami": 0 } 2024-04-24T23:09:09.849 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2024-04-24T23:09:09.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.952+0000 7f3296f42640 1 Processor -- start 2024-04-24T23:09:09.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.952+0000 7f3296f42640 1 -- start start 2024-04-24T23:09:09.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.953+0000 7f3296f42640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f32900cd3c0 0x7f329015e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:09.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.953+0000 7f3296f42640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f329015ec80 0x7f32901513b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:09.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.953+0000 7f3296f42640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32900cd850 con 0x7f32900b57c0 2024-04-24T23:09:09.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.953+0000 7f3296f42640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f329015f0d0 con 0x7f329015ec80 2024-04-24T23:09:09.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.953+0000 7f3296f42640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f329015f5c0 con 0x7f32900cd3c0 2024-04-24T23:09:09.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.953+0000 7f32954df640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f329015ec80 0x7f32901513b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:09.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.953+0000 7f3294cde640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f32900b57c0 0x7f32900b5ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52744/0 (socket says 172.21.15.32:52744) 2024-04-24T23:09:09.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.953+0000 7f32954df640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f329015ec80 0x7f32901513b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32784/0 (socket says 172.21.15.32:32784) 2024-04-24T23:09:09.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.953+0000 7f3294cde640 1 -- 172.21.15.32:0/2481439697 learned_addr learned my addr 172.21.15.32:0/2481439697 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:09:09.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.953+0000 7f3287fff640 1 --2- 172.21.15.32:0/2481439697 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f32900cd3c0 0x7f329015e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:09.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.953+0000 7f32954df640 1 -- 172.21.15.32:0/2481439697 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f32900cd3c0 msgr2=0x7f329015e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:09:09.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.953+0000 7f32954df640 1 --2- 172.21.15.32:0/2481439697 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f32900cd3c0 0x7f329015e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:09.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.953+0000 7f32954df640 1 -- 172.21.15.32:0/2481439697 >> v1:172.21.15.32:6789/0 conn(0x7f32900b57c0 legacy=0x7f32900b5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:09.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.953+0000 7f32954df640 1 -- 172.21.15.32:0/2481439697 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32900cdb30 con 0x7f329015ec80 2024-04-24T23:09:09.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.953+0000 7f32877fe640 1 -- 172.21.15.32:0/2481439697 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 734905760 0 0) 0x7f32900cd850 con 0x7f32900b57c0 2024-04-24T23:09:09.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.954+0000 7f32954df640 1 --2- 172.21.15.32:0/2481439697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f329015ec80 0x7f32901513b0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f328800bfa0 tx=0x7f328802c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9c4deac50dd1d0c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:09.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.954+0000 7f32877fe640 1 -- 172.21.15.32:0/2481439697 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3288039070 con 0x7f329015ec80 2024-04-24T23:09:09.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.954+0000 7f32877fe640 1 -- 172.21.15.32:0/2481439697 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3288009540 con 0x7f329015ec80 2024-04-24T23:09:09.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.954+0000 7f32877fe640 1 -- 172.21.15.32:0/2481439697 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3288009910 con 0x7f329015ec80 2024-04-24T23:09:09.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.955+0000 7f3296f42640 1 -- 172.21.15.32:0/2481439697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f329015ec80 msgr2=0x7f32901513b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:09.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.955+0000 7f3296f42640 1 --2- 172.21.15.32:0/2481439697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f329015ec80 0x7f32901513b0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f328800bfa0 tx=0x7f328802c860 comp rx=0 tx=0).stop 2024-04-24T23:09:09.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.955+0000 7f3296f42640 1 -- 172.21.15.32:0/2481439697 shutdown_connections 2024-04-24T23:09:09.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.955+0000 7f3296f42640 1 --2- 172.21.15.32:0/2481439697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f329015ec80 0x7f32901513b0 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:09.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.955+0000 7f3296f42640 1 --2- 172.21.15.32:0/2481439697 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f32900cd3c0 0x7f329015e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:09.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.955+0000 7f3296f42640 1 -- 172.21.15.32:0/2481439697 >> 172.21.15.32:0/2481439697 conn(0x7f32900b1f10 msgr2=0x7f32900b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:09.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.955+0000 7f3296f42640 1 -- 172.21.15.32:0/2481439697 shutdown_connections 2024-04-24T23:09:09.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.955+0000 7f3296f42640 1 -- 172.21.15.32:0/2481439697 wait complete. 2024-04-24T23:09:09.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.956+0000 7f3296f42640 1 Processor -- start 2024-04-24T23:09:09.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.956+0000 7f3296f42640 1 -- start start 2024-04-24T23:09:09.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.957+0000 7f3296f42640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f32900b57c0 0x7f3290070250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:09.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.957+0000 7f3296f42640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f329015ec80 0x7f3290072c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:09.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.957+0000 7f3296f42640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f329014dbd0 0x7f32900b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:09.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.957+0000 7f3296f42640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f32900ce3d0 con 0x7f329015ec80 2024-04-24T23:09:09.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.957+0000 7f3296f42640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f32900ce0f0 con 0x7f329014dbd0 2024-04-24T23:09:09.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.957+0000 7f3296f42640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f32900b62b0 con 0x7f32900b57c0 2024-04-24T23:09:09.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.957+0000 7f3287fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f329015ec80 0x7f3290072c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:09.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.957+0000 7f3287fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f329015ec80 0x7f3290072c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32794/0 (socket says 172.21.15.32:32794) 2024-04-24T23:09:09.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.957+0000 7f3287fff640 1 -- 172.21.15.32:0/532618865 learned_addr learned my addr 172.21.15.32:0/532618865 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:09:09.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.957+0000 7f3294cde640 1 --2- 172.21.15.32:0/532618865 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f32900b57c0 0x7f3290070250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:09.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.957+0000 7f32954df640 1 --2- 172.21.15.32:0/532618865 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f329014dbd0 0x7f32900b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:09.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.957+0000 7f3287fff640 1 -- 172.21.15.32:0/532618865 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f32900b57c0 msgr2=0x7f3290070250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:09.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.957+0000 7f3287fff640 1 --2- 172.21.15.32:0/532618865 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f32900b57c0 0x7f3290070250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:09.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.957+0000 7f3287fff640 1 -- 172.21.15.32:0/532618865 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f329014dbd0 msgr2=0x7f32900b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:09.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.957+0000 7f3287fff640 1 --2- 172.21.15.32:0/532618865 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f329014dbd0 0x7f32900b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:09.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.957+0000 7f3287fff640 1 -- 172.21.15.32:0/532618865 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3290070950 con 0x7f329015ec80 2024-04-24T23:09:09.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.958+0000 7f3287fff640 1 --2- 172.21.15.32:0/532618865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f329015ec80 0x7f3290072c70 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f328000c820 tx=0x7f328000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:09.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.958+0000 7f32857fa640 1 -- 172.21.15.32:0/532618865 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3280015070 con 0x7f329015ec80 2024-04-24T23:09:09.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.958+0000 7f32857fa640 1 -- 172.21.15.32:0/532618865 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3280004240 con 0x7f329015ec80 2024-04-24T23:09:09.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.958+0000 7f3296f42640 1 -- 172.21.15.32:0/532618865 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3290070c30 con 0x7f329015ec80 2024-04-24T23:09:09.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.958+0000 7f32857fa640 1 -- 172.21.15.32:0/532618865 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328000ce10 con 0x7f329015ec80 2024-04-24T23:09:09.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.958+0000 7f3296f42640 1 -- 172.21.15.32:0/532618865 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32900711f0 con 0x7f329015ec80 2024-04-24T23:09:09.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.959+0000 7f3296f42640 1 -- 172.21.15.32:0/532618865 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32900691a0 con 0x7f329015ec80 2024-04-24T23:09:09.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.959+0000 7f32857fa640 1 -- 172.21.15.32:0/532618865 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3280007830 con 0x7f329015ec80 2024-04-24T23:09:09.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.960+0000 7f32857fa640 1 --2- 172.21.15.32:0/532618865 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f327c06be30 0x7f327c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:09.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.960+0000 7f32857fa640 1 -- 172.21.15.32:0/532618865 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 5709+0+0 (secure 0 0 0) 0x7f328008ac00 con 0x7f329015ec80 2024-04-24T23:09:09.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.960+0000 7f3294cde640 1 --2- 172.21.15.32:0/532618865 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f327c06be30 0x7f327c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:09.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.960+0000 7f3294cde640 1 --2- 172.21.15.32:0/532618865 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f327c06be30 0x7f327c06e2f0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f328c000f60 tx=0x7f328c007d50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:09.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:09.964+0000 7f32857fa640 1 -- 172.21.15.32:0/532618865 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f32900691a0 con 0x7f329015ec80 2024-04-24T23:09:10.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:10.147+0000 7f3296f42640 1 -- 172.21.15.32:0/532618865 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) -- 0x7f3290068430 con 0x7f329015ec80 2024-04-24T23:09:10.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:10.447+0000 7f32857fa640 1 -- 172.21.15.32:0/532618865 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v170) ==== 181+0+0 (secure 0 0 0) 0x7f3280091080 con 0x7f329015ec80 2024-04-24T23:09:10.449 INFO:teuthology.orchestra.run.smithi032.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2024-04-24T23:09:10.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:10.450+0000 7f3296f42640 1 -- 172.21.15.32:0/532618865 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f327c06be30 msgr2=0x7f327c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:10.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:10.450+0000 7f3296f42640 1 --2- 172.21.15.32:0/532618865 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f327c06be30 0x7f327c06e2f0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f328c000f60 tx=0x7f328c007d50 comp rx=0 tx=0).stop 2024-04-24T23:09:10.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:10.450+0000 7f3296f42640 1 -- 172.21.15.32:0/532618865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f329015ec80 msgr2=0x7f3290072c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:10.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:10.450+0000 7f3296f42640 1 --2- 172.21.15.32:0/532618865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f329015ec80 0x7f3290072c70 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f328000c820 tx=0x7f328000cce0 comp rx=0 tx=0).stop 2024-04-24T23:09:10.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:10.450+0000 7f3296f42640 1 -- 172.21.15.32:0/532618865 shutdown_connections 2024-04-24T23:09:10.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:10.450+0000 7f3296f42640 1 --2- 172.21.15.32:0/532618865 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f327c06be30 0x7f327c06e2f0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:10.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:10.450+0000 7f3296f42640 1 --2- 172.21.15.32:0/532618865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f329015ec80 0x7f3290072c70 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:10.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:10.450+0000 7f3296f42640 1 --2- 172.21.15.32:0/532618865 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f329014dbd0 0x7f32900b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:10.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:10.450+0000 7f3296f42640 1 --2- 172.21.15.32:0/532618865 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f32900b57c0 0x7f3290070250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:10.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:10.450+0000 7f3296f42640 1 -- 172.21.15.32:0/532618865 >> 172.21.15.32:0/532618865 conn(0x7f32900b1f10 msgr2=0x7f32900b0420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:10.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:10.450+0000 7f3296f42640 1 -- 172.21.15.32:0/532618865 shutdown_connections 2024-04-24T23:09:10.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:09:10.450+0000 7f3296f42640 1 -- 172.21.15.32:0/532618865 wait complete. 2024-04-24T23:09:10.463 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-04-24T23:09:10.578 INFO:teuthology.orchestra.run.smithi032.stdout:{"osd_mon_report_interval":"5"} 2024-04-24T23:09:10.578 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-04-24T23:09:10.578 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2024-04-24T23:09:10.685 INFO:teuthology.orchestra.run.smithi032.stdout:{"mon_tick_interval":"5"} 2024-04-24T23:09:10.686 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2024-04-24T23:09:10.686 INFO:tasks.cephfs.test_full:pool capacity 33554432, 32MB should be enough to fill it 2024-04-24T23:09:10.686 DEBUG:teuthology.orchestra.run.smithi032:> stdin-killer -- python3 -c ' 2024-04-24T23:09:10.686 DEBUG:teuthology.orchestra.run.smithi032:> import time 2024-04-24T23:09:10.686 DEBUG:teuthology.orchestra.run.smithi032:> import datetime 2024-04-24T23:09:10.686 DEBUG:teuthology.orchestra.run.smithi032:> import subprocess 2024-04-24T23:09:10.686 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-24T23:09:10.686 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-24T23:09:10.686 DEBUG:teuthology.orchestra.run.smithi032:> # Write some buffered data through before going full, all should be well 2024-04-24T23:09:10.686 DEBUG:teuthology.orchestra.run.smithi032:> print("writing some data through which we expect to succeed") 2024-04-24T23:09:10.686 DEBUG:teuthology.orchestra.run.smithi032:> bytes = 0 2024-04-24T23:09:10.687 DEBUG:teuthology.orchestra.run.smithi032:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2024-04-24T23:09:10.687 DEBUG:teuthology.orchestra.run.smithi032:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2024-04-24T23:09:10.687 DEBUG:teuthology.orchestra.run.smithi032:> bytes += os.write(f, b'"'"'a'"'"' * 4096) 2024-04-24T23:09:10.687 DEBUG:teuthology.orchestra.run.smithi032:> os.fsync(f) 2024-04-24T23:09:10.687 DEBUG:teuthology.orchestra.run.smithi032:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2024-04-24T23:09:10.687 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-24T23:09:10.687 DEBUG:teuthology.orchestra.run.smithi032:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2024-04-24T23:09:10.687 DEBUG:teuthology.orchestra.run.smithi032:> # writing until we see an error from fsync. As long as we'"'"'re doing 2024-04-24T23:09:10.687 DEBUG:teuthology.orchestra.run.smithi032:> # buffered IO, the error should always only appear from fsync and not 2024-04-24T23:09:10.687 DEBUG:teuthology.orchestra.run.smithi032:> # from write 2024-04-24T23:09:10.687 DEBUG:teuthology.orchestra.run.smithi032:> full = False 2024-04-24T23:09:10.687 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-24T23:09:10.687 DEBUG:teuthology.orchestra.run.smithi032:> for n in range(0, int(32 * 1.1)): 2024-04-24T23:09:10.687 DEBUG:teuthology.orchestra.run.smithi032:> try: 2024-04-24T23:09:10.687 DEBUG:teuthology.orchestra.run.smithi032:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2024-04-24T23:09:10.688 DEBUG:teuthology.orchestra.run.smithi032:> print("wrote bytes via buffered write, moving on to fsync") 2024-04-24T23:09:10.688 DEBUG:teuthology.orchestra.run.smithi032:> except OSError as e: 2024-04-24T23:09:10.688 DEBUG:teuthology.orchestra.run.smithi032:> if False: 2024-04-24T23:09:10.688 DEBUG:teuthology.orchestra.run.smithi032:> print("Unexpected error %s from write() instead of fsync()" % e) 2024-04-24T23:09:10.688 DEBUG:teuthology.orchestra.run.smithi032:> raise 2024-04-24T23:09:10.688 DEBUG:teuthology.orchestra.run.smithi032:> else: 2024-04-24T23:09:10.688 DEBUG:teuthology.orchestra.run.smithi032:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2024-04-24T23:09:10.688 DEBUG:teuthology.orchestra.run.smithi032:> full = True 2024-04-24T23:09:10.688 DEBUG:teuthology.orchestra.run.smithi032:> break 2024-04-24T23:09:10.688 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-24T23:09:10.688 DEBUG:teuthology.orchestra.run.smithi032:> try: 2024-04-24T23:09:10.688 DEBUG:teuthology.orchestra.run.smithi032:> os.fsync(f) 2024-04-24T23:09:10.688 DEBUG:teuthology.orchestra.run.smithi032:> print("fsync'"'"'ed successfully") 2024-04-24T23:09:10.688 DEBUG:teuthology.orchestra.run.smithi032:> except OSError as e: 2024-04-24T23:09:10.688 DEBUG:teuthology.orchestra.run.smithi032:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2024-04-24T23:09:10.688 DEBUG:teuthology.orchestra.run.smithi032:> full = True 2024-04-24T23:09:10.689 DEBUG:teuthology.orchestra.run.smithi032:> break 2024-04-24T23:09:10.689 DEBUG:teuthology.orchestra.run.smithi032:> else: 2024-04-24T23:09:10.689 DEBUG:teuthology.orchestra.run.smithi032:> print("Not full yet after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2024-04-24T23:09:10.689 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-24T23:09:10.689 DEBUG:teuthology.orchestra.run.smithi032:> if n > 32 * 0.9: 2024-04-24T23:09:10.689 DEBUG:teuthology.orchestra.run.smithi032:> # Be cautious in the last region where we expect to hit 2024-04-24T23:09:10.689 DEBUG:teuthology.orchestra.run.smithi032:> # the full condition, so that we don'"'"'t overshoot too dramatically 2024-04-24T23:09:10.689 DEBUG:teuthology.orchestra.run.smithi032:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2024-04-24T23:09:10.689 DEBUG:teuthology.orchestra.run.smithi032:> time.sleep(15.0) 2024-04-24T23:09:10.689 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-24T23:09:10.689 DEBUG:teuthology.orchestra.run.smithi032:> if not full: 2024-04-24T23:09:10.689 DEBUG:teuthology.orchestra.run.smithi032:> raise RuntimeError("Failed to reach fullness after writing %d bytes" % bytes) 2024-04-24T23:09:10.689 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-24T23:09:10.689 DEBUG:teuthology.orchestra.run.smithi032:> # close() should not raise an error because we already caught it in 2024-04-24T23:09:10.689 DEBUG:teuthology.orchestra.run.smithi032:> # fsync. There shouldn'"'"'t have been any more writeback errors 2024-04-24T23:09:10.689 DEBUG:teuthology.orchestra.run.smithi032:> # since then because all IOs got cancelled on the full flag. 2024-04-24T23:09:10.689 DEBUG:teuthology.orchestra.run.smithi032:> print("calling close") 2024-04-24T23:09:10.690 DEBUG:teuthology.orchestra.run.smithi032:> os.close(f) 2024-04-24T23:09:10.690 DEBUG:teuthology.orchestra.run.smithi032:> print("close() did not raise error") 2024-04-24T23:09:10.690 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-24T23:09:10.690 DEBUG:teuthology.orchestra.run.smithi032:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2024-04-24T23:09:10.690 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-24T23:09:37.855 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:09:37.858 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:10:07.917 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:10:07.921 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:10:11.569 INFO:teuthology.orchestra.run.smithi032.stdout:writing some data through which we expect to succeed 2024-04-24T23:10:11.569 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed data successfully, will now attempt to fill fs 2024-04-24T23:10:11.569 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:10:11.569 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:10:11.569 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 1.00 MB 2024-04-24T23:10:11.569 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:10:11.569 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:10:11.569 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 2.00 MB 2024-04-24T23:10:11.570 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:10:11.570 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:10:11.570 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 3.00 MB 2024-04-24T23:10:11.570 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:10:11.570 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:10:11.570 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 4.00 MB 2024-04-24T23:10:11.570 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:10:11.570 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:10:11.570 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 5.00 MB 2024-04-24T23:10:11.570 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:10:11.570 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:10:11.570 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 6.00 MB 2024-04-24T23:10:11.570 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:10:11.571 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:10:11.571 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 7.00 MB 2024-04-24T23:10:11.571 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:10:11.571 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:10:11.571 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 8.00 MB 2024-04-24T23:10:11.571 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:10:11.571 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:10:11.571 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 9.00 MB 2024-04-24T23:10:11.571 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:10:11.571 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:10:11.571 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 10.00 MB 2024-04-24T23:10:11.571 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:10:11.571 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:10:11.571 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 11.00 MB 2024-04-24T23:10:11.571 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:10:11.572 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:10:11.572 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 12.00 MB 2024-04-24T23:10:11.572 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:10:11.572 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:10:11.572 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 13.00 MB 2024-04-24T23:10:11.572 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:10:11.572 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:10:11.572 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 14.00 MB 2024-04-24T23:10:11.572 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:10:11.572 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:10:11.572 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 15.00 MB 2024-04-24T23:10:11.572 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:10:11.572 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:10:11.572 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 16.00 MB 2024-04-24T23:10:11.572 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:10:11.572 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:10:11.572 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 17.00 MB 2024-04-24T23:10:11.573 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:10:11.573 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:10:11.573 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 18.00 MB 2024-04-24T23:10:11.573 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:10:11.573 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:10:11.573 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 19.00 MB 2024-04-24T23:10:11.573 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:10:11.573 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:10:11.573 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 20.00 MB 2024-04-24T23:10:11.573 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:10:11.573 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:10:11.573 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 21.00 MB 2024-04-24T23:10:11.573 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:10:11.573 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:10:11.573 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 22.00 MB 2024-04-24T23:10:11.573 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:10:11.574 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:10:11.574 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 23.00 MB 2024-04-24T23:10:11.574 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:10:11.574 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:10:11.574 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 24.00 MB 2024-04-24T23:10:11.574 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:10:11.574 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:10:11.574 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 25.00 MB 2024-04-24T23:10:11.574 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:10:11.574 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:10:11.574 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 26.00 MB 2024-04-24T23:10:11.574 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:10:11.575 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:10:11.575 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 27.00 MB 2024-04-24T23:10:11.575 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:10:11.575 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:10:11.575 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 28.00 MB 2024-04-24T23:10:11.575 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:10:11.575 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:10:11.575 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 29.00 MB 2024-04-24T23:10:11.575 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:10:11.575 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:10:11.575 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 30.00 MB 2024-04-24T23:10:11.575 INFO:teuthology.orchestra.run.smithi032.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-04-24T23:10:11.575 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:10:11.575 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:10:11.575 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 31.00 MB 2024-04-24T23:10:11.576 INFO:teuthology.orchestra.run.smithi032.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-04-24T23:10:11.576 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:10:11.576 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:10:11.576 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 32.00 MB 2024-04-24T23:10:11.576 INFO:teuthology.orchestra.run.smithi032.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-04-24T23:10:11.576 INFO:teuthology.orchestra.run.smithi032.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-24T23:10:11.576 INFO:teuthology.orchestra.run.smithi032.stdout:fsync'ed successfully 2024-04-24T23:10:11.576 INFO:teuthology.orchestra.run.smithi032.stdout:Not full yet after 33.00 MB 2024-04-24T23:10:11.576 INFO:teuthology.orchestra.run.smithi032.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-04-24T23:10:11.576 INFO:teuthology.orchestra.run.smithi032.stdout:Reached fullness after 33.00 MB 2024-04-24T23:10:11.576 INFO:teuthology.orchestra.run.smithi032.stdout:calling close 2024-04-24T23:10:11.576 INFO:teuthology.orchestra.run.smithi032.stdout:close() did not raise error 2024-04-24T23:10:11.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:10:11.583 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-24T23:10:11.583 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:10:11.626 DEBUG:teuthology.orchestra.run.smithi121:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:10:11.668 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-24T23:10:11.668 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:10:11.669 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:10:11.669 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:10:11.687 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:10:11.687 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:10:11.745 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:10:11.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.892+0000 7f36a649e640 1 Processor -- start 2024-04-24T23:10:11.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.892+0000 7f36a649e640 1 -- start start 2024-04-24T23:10:11.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.893+0000 7f36a649e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f36a006c5f0 0x7f36a006c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:11.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.893+0000 7f36a649e640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f36a00b44d0 0x7f36a00b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:11.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.893+0000 7f36a649e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f36a00af890 0x7f36a00cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:11.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.893+0000 7f36a649e640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f36a00b6430 con 0x7f36a006c5f0 2024-04-24T23:10:11.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.893+0000 7f36a649e640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f36a0156f80 con 0x7f36a00b44d0 2024-04-24T23:10:11.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.893+0000 7f36a649e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f36a00b6150 con 0x7f36a00af890 2024-04-24T23:10:11.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.893+0000 7f369ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f36a006c5f0 0x7f36a006c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:11.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.893+0000 7f369ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f36a006c5f0 0x7f36a006c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34768/0 (socket says 172.21.15.32:34768) 2024-04-24T23:10:11.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.893+0000 7f369ffff640 1 -- 172.21.15.32:0/4235223948 learned_addr learned my addr 172.21.15.32:0/4235223948 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:11.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.893+0000 7f36a4a3b640 1 --2- 172.21.15.32:0/4235223948 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f36a00af890 0x7f36a00cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:11.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.893+0000 7f369f7fe640 1 --2- 172.21.15.32:0/4235223948 >> v2:172.21.15.121:3300/0 conn(0x7f36a00b44d0 0x7f36a00b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:11.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.893+0000 7f369ffff640 1 -- 172.21.15.32:0/4235223948 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f36a00af890 msgr2=0x7f36a00cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:11.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.893+0000 7f369ffff640 1 --2- 172.21.15.32:0/4235223948 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f36a00af890 0x7f36a00cef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:11.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.893+0000 7f369ffff640 1 -- 172.21.15.32:0/4235223948 >> v2:172.21.15.121:3300/0 conn(0x7f36a00b44d0 msgr2=0x7f36a00b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:11.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.893+0000 7f369ffff640 1 --2- 172.21.15.32:0/4235223948 >> v2:172.21.15.121:3300/0 conn(0x7f36a00b44d0 0x7f36a00b5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:11.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.893+0000 7f369ffff640 1 -- 172.21.15.32:0/4235223948 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36a00b39f0 con 0x7f36a006c5f0 2024-04-24T23:10:11.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.894+0000 7f369ffff640 1 --2- 172.21.15.32:0/4235223948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36a006c5f0 0x7f36a006c9d0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f36a0073080 tx=0x7f369002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d47ef9e0e695c87f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:11.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.894+0000 7f369effd640 1 -- 172.21.15.32:0/4235223948 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3690037070 con 0x7f36a006c5f0 2024-04-24T23:10:11.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.894+0000 7f369effd640 1 -- 172.21.15.32:0/4235223948 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3690032440 con 0x7f36a006c5f0 2024-04-24T23:10:11.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.894+0000 7f369effd640 1 -- 172.21.15.32:0/4235223948 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3690032810 con 0x7f36a006c5f0 2024-04-24T23:10:11.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.895+0000 7f36a649e640 1 -- 172.21.15.32:0/4235223948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36a006c5f0 msgr2=0x7f36a006c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:11.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.895+0000 7f36a649e640 1 --2- 172.21.15.32:0/4235223948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36a006c5f0 0x7f36a006c9d0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f36a0073080 tx=0x7f369002a140 comp rx=0 tx=0).stop 2024-04-24T23:10:11.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.895+0000 7f36a649e640 1 -- 172.21.15.32:0/4235223948 shutdown_connections 2024-04-24T23:10:11.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.895+0000 7f36a649e640 1 --2- 172.21.15.32:0/4235223948 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f36a00af890 0x7f36a00cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:11.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.895+0000 7f36a649e640 1 --2- 172.21.15.32:0/4235223948 >> v2:172.21.15.121:3300/0 conn(0x7f36a00b44d0 0x7f36a00b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:11.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.895+0000 7f36a649e640 1 --2- 172.21.15.32:0/4235223948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36a006c5f0 0x7f36a006c9d0 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:11.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.895+0000 7f36a649e640 1 -- 172.21.15.32:0/4235223948 >> 172.21.15.32:0/4235223948 conn(0x7f36a006b110 msgr2=0x7f36a006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:11.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.895+0000 7f36a649e640 1 -- 172.21.15.32:0/4235223948 shutdown_connections 2024-04-24T23:10:11.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.895+0000 7f36a649e640 1 -- 172.21.15.32:0/4235223948 wait complete. 2024-04-24T23:10:11.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.896+0000 7f36a649e640 1 Processor -- start 2024-04-24T23:10:11.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.896+0000 7f36a649e640 1 -- start start 2024-04-24T23:10:11.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.897+0000 7f36a649e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f36a006c5f0 0x7f36a00aef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:11.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.897+0000 7f36a649e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f36a00af890 0x7f36a00b12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:11.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.897+0000 7f36a649e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36a00b44d0 0x7f36a00b1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:11.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.897+0000 7f36a649e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f36a006cf10 con 0x7f36a00b44d0 2024-04-24T23:10:11.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.897+0000 7f36a649e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f36a00b3fb0 con 0x7f36a006c5f0 2024-04-24T23:10:11.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.897+0000 7f36a649e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f36a00b5dd0 con 0x7f36a00af890 2024-04-24T23:10:11.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.897+0000 7f36a4a3b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36a00b44d0 0x7f36a00b1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:11.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.897+0000 7f36a4a3b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36a00b44d0 0x7f36a00b1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34774/0 (socket says 172.21.15.32:34774) 2024-04-24T23:10:11.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.897+0000 7f36a4a3b640 1 -- 172.21.15.32:0/3713851320 learned_addr learned my addr 172.21.15.32:0/3713851320 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:11.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.897+0000 7f369f7fe640 1 --2- 172.21.15.32:0/3713851320 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f36a00af890 0x7f36a00b12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:11.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.897+0000 7f369ffff640 1 --2- 172.21.15.32:0/3713851320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f36a006c5f0 0x7f36a00aef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:11.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.897+0000 7f36a4a3b640 1 -- 172.21.15.32:0/3713851320 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f36a00af890 msgr2=0x7f36a00b12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:11.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.897+0000 7f36a4a3b640 1 --2- 172.21.15.32:0/3713851320 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f36a00af890 0x7f36a00b12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:11.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.897+0000 7f36a4a3b640 1 -- 172.21.15.32:0/3713851320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f36a006c5f0 msgr2=0x7f36a00aef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:11.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.897+0000 7f36a4a3b640 1 --2- 172.21.15.32:0/3713851320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f36a006c5f0 0x7f36a00aef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:11.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.897+0000 7f36a4a3b640 1 -- 172.21.15.32:0/3713851320 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36a00aa7d0 con 0x7f36a00b44d0 2024-04-24T23:10:11.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.898+0000 7f36a4a3b640 1 --2- 172.21.15.32:0/3713851320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36a00b44d0 0x7f36a00b1800 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f3698007e60 tx=0x7f369800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:11.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.898+0000 7f369cff9640 1 -- 172.21.15.32:0/3713851320 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3698014420 con 0x7f36a00b44d0 2024-04-24T23:10:11.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.898+0000 7f369cff9640 1 -- 172.21.15.32:0/3713851320 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3698019070 con 0x7f36a00b44d0 2024-04-24T23:10:11.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.898+0000 7f36a649e640 1 -- 172.21.15.32:0/3713851320 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36a00acdf0 con 0x7f36a00b44d0 2024-04-24T23:10:11.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.898+0000 7f369cff9640 1 -- 172.21.15.32:0/3713851320 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3698014820 con 0x7f36a00b44d0 2024-04-24T23:10:11.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.898+0000 7f36a649e640 1 -- 172.21.15.32:0/3713851320 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36a01e6c60 con 0x7f36a00b44d0 2024-04-24T23:10:11.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.899+0000 7f36a649e640 1 -- 172.21.15.32:0/3713851320 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36a0068430 con 0x7f36a00b44d0 2024-04-24T23:10:11.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.899+0000 7f369cff9640 1 -- 172.21.15.32:0/3713851320 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f36980149c0 con 0x7f36a00b44d0 2024-04-24T23:10:11.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.900+0000 7f369cff9640 1 --2- 172.21.15.32:0/3713851320 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f368006be30 0x7f368006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:11.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.900+0000 7f369ffff640 1 --2- 172.21.15.32:0/3713851320 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f368006be30 0x7f368006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:11.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.900+0000 7f369cff9640 1 -- 172.21.15.32:0/3713851320 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 5767+0+0 (secure 0 0 0) 0x7f3698089bb0 con 0x7f36a00b44d0 2024-04-24T23:10:11.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.900+0000 7f369cff9640 1 -- 172.21.15.32:0/3713851320 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=174}) -- 0x7f3680073a20 con 0x7f36a00b44d0 2024-04-24T23:10:11.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.901+0000 7f369ffff640 1 --2- 172.21.15.32:0/3713851320 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f368006be30 0x7f368006e2f0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f3690034000 tx=0x7f369002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:11.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:11.905+0000 7f369cff9640 1 -- 172.21.15.32:0/3713851320 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f36a00aa7d0 con 0x7f36a00b44d0 2024-04-24T23:10:12.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.088+0000 7f36a649e640 1 -- 172.21.15.32:0/3713851320 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f36a00b5670 con 0x7f36a00b44d0 2024-04-24T23:10:12.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.090+0000 7f369cff9640 1 -- 172.21.15.32:0/3713851320 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v173) ==== 74+0+16756 (secure 0 0 0) 0x7f36980598d0 con 0x7f36a00b44d0 2024-04-24T23:10:12.091 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:10:12.092 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":173,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:09:53.858501+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:08:59.076360+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-24T23:08:59.745667+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.1d","mappings":[{"from":2,"to":5}]},{"pgid":"21.39","mappings":[{"from":2,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:10:12.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.091+0000 7f36a649e640 1 -- 172.21.15.32:0/3713851320 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f368006be30 msgr2=0x7f368006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:12.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.091+0000 7f36a649e640 1 --2- 172.21.15.32:0/3713851320 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f368006be30 0x7f368006e2f0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f3690034000 tx=0x7f369002e000 comp rx=0 tx=0).stop 2024-04-24T23:10:12.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.091+0000 7f36a649e640 1 -- 172.21.15.32:0/3713851320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36a00b44d0 msgr2=0x7f36a00b1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:12.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.092+0000 7f36a649e640 1 --2- 172.21.15.32:0/3713851320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36a00b44d0 0x7f36a00b1800 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f3698007e60 tx=0x7f369800e4b0 comp rx=0 tx=0).stop 2024-04-24T23:10:12.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.092+0000 7f36a649e640 1 -- 172.21.15.32:0/3713851320 shutdown_connections 2024-04-24T23:10:12.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.092+0000 7f36a649e640 1 --2- 172.21.15.32:0/3713851320 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f368006be30 0x7f368006e2f0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:12.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.092+0000 7f36a649e640 1 --2- 172.21.15.32:0/3713851320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36a00b44d0 0x7f36a00b1800 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:12.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.092+0000 7f36a649e640 1 --2- 172.21.15.32:0/3713851320 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f36a00af890 0x7f36a00b12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:12.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.092+0000 7f36a649e640 1 --2- 172.21.15.32:0/3713851320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f36a006c5f0 0x7f36a00aef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:12.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.092+0000 7f36a649e640 1 -- 172.21.15.32:0/3713851320 >> 172.21.15.32:0/3713851320 conn(0x7f36a006b110 msgr2=0x7f36a0071750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:12.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.092+0000 7f36a649e640 1 -- 172.21.15.32:0/3713851320 shutdown_connections 2024-04-24T23:10:12.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.092+0000 7f36a649e640 1 -- 172.21.15.32:0/3713851320 wait complete. 2024-04-24T23:10:12.104 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:08:59.076360+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '168', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:08:59.745667+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '171', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:10:12.105 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:10:12.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.207+0000 7f506a914640 1 Processor -- start 2024-04-24T23:10:12.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.207+0000 7f506a914640 1 -- start start 2024-04-24T23:10:12.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.208+0000 7f506a914640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5064162320 0x7f50640ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:12.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.208+0000 7f506a914640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50640cd270 con 0x7f5064162e10 2024-04-24T23:10:12.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.208+0000 7f506a914640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5064071610 con 0x7f50640b5af0 2024-04-24T23:10:12.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.208+0000 7f506a914640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f50640b5660 con 0x7f5064162320 2024-04-24T23:10:12.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.208+0000 7f5068eb1640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5064162e10 0x7f506415ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47858/0 (socket says 172.21.15.32:47858) 2024-04-24T23:10:12.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.208+0000 7f5068eb1640 1 -- 172.21.15.32:0/1832315204 learned_addr learned my addr 172.21.15.32:0/1832315204 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:10:12.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.208+0000 7f5063fff640 1 --2- 172.21.15.32:0/1832315204 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5064162320 0x7f50640ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:12.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.208+0000 7f5062ffd640 1 -- 172.21.15.32:0/1832315204 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 827550602 0 0) 0x7f50640cd270 con 0x7f5064162e10 2024-04-24T23:10:12.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.208+0000 7f5062ffd640 1 -- 172.21.15.32:0/1832315204 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f504c003520 con 0x7f5064162e10 2024-04-24T23:10:12.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.209+0000 7f5062ffd640 1 -- 172.21.15.32:0/1832315204 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3595067845 0 0) 0x7f504c003520 con 0x7f5064162e10 2024-04-24T23:10:12.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.209+0000 7f5062ffd640 1 -- 172.21.15.32:0/1832315204 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5064162320 msgr2=0x7f50640ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:12.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.209+0000 7f5062ffd640 1 --2- 172.21.15.32:0/1832315204 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5064162320 0x7f50640ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:12.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.209+0000 7f5062ffd640 1 -- 172.21.15.32:0/1832315204 >> v1:172.21.15.121:6789/0 conn(0x7f50640b5af0 legacy=0x7f5064162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:12.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.209+0000 7f5062ffd640 1 -- 172.21.15.32:0/1832315204 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50640718f0 con 0x7f5064162e10 2024-04-24T23:10:12.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.209+0000 7f5062ffd640 1 -- 172.21.15.32:0/1832315204 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f505c0019f0 con 0x7f5064162e10 2024-04-24T23:10:12.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.209+0000 7f5062ffd640 1 -- 172.21.15.32:0/1832315204 >> v1:172.21.15.32:6789/0 conn(0x7f5064162e10 legacy=0x7f506415ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:12.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.209+0000 7f5062ffd640 1 --2- 172.21.15.32:0/1832315204 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f504c0039e0 0x7f504c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:12.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.209+0000 7f5062ffd640 1 --2- 172.21.15.32:0/1832315204 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5064162320 0x7f504c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:12.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.209+0000 7f5062ffd640 1 --2- 172.21.15.32:0/1832315204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f504c004d40 0x7f504c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:12.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.209+0000 7f5062ffd640 1 -- 172.21.15.32:0/1832315204 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f50640b5660 con 0x7f504c004d40 2024-04-24T23:10:12.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.209+0000 7f5062ffd640 1 -- 172.21.15.32:0/1832315204 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f504c00baf0 con 0x7f504c0039e0 2024-04-24T23:10:12.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.209+0000 7f5062ffd640 1 -- 172.21.15.32:0/1832315204 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f504c00bcf0 con 0x7f5064162320 2024-04-24T23:10:12.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.209+0000 7f5068eb1640 1 --2- 172.21.15.32:0/1832315204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f504c004d40 0x7f504c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:12.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.209+0000 7f50637fe640 1 --2- 172.21.15.32:0/1832315204 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5064162320 0x7f504c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:12.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.209+0000 7f5063fff640 1 --2- 172.21.15.32:0/1832315204 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f504c0039e0 0x7f504c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:12.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.209+0000 7f5068eb1640 1 -- 172.21.15.32:0/1832315204 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5064162320 msgr2=0x7f504c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:12.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.209+0000 7f5068eb1640 1 --2- 172.21.15.32:0/1832315204 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5064162320 0x7f504c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:12.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.209+0000 7f5068eb1640 1 -- 172.21.15.32:0/1832315204 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f504c0039e0 msgr2=0x7f504c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:12.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.209+0000 7f5068eb1640 1 --2- 172.21.15.32:0/1832315204 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f504c0039e0 0x7f504c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:12.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.209+0000 7f5068eb1640 1 -- 172.21.15.32:0/1832315204 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f504c003520 con 0x7f504c004d40 2024-04-24T23:10:12.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.210+0000 7f5068eb1640 1 --2- 172.21.15.32:0/1832315204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f504c004d40 0x7f504c0051c0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f505c00a850 tx=0x7f504c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3283d6f8111dc2cd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:12.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.210+0000 7f5062ffd640 1 -- 172.21.15.32:0/1832315204 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f505c00a4a0 con 0x7f504c004d40 2024-04-24T23:10:12.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.210+0000 7f5062ffd640 1 -- 172.21.15.32:0/1832315204 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f505c02f070 con 0x7f504c004d40 2024-04-24T23:10:12.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.210+0000 7f5062ffd640 1 -- 172.21.15.32:0/1832315204 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f505c02bce0 con 0x7f504c004d40 2024-04-24T23:10:12.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.211+0000 7f506a914640 1 -- 172.21.15.32:0/1832315204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f504c004d40 msgr2=0x7f504c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:12.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.211+0000 7f506a914640 1 --2- 172.21.15.32:0/1832315204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f504c004d40 0x7f504c0051c0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f505c00a850 tx=0x7f504c005830 comp rx=0 tx=0).stop 2024-04-24T23:10:12.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.211+0000 7f5068eb1640 1 -- 172.21.15.32:0/1832315204 reap_dead start 2024-04-24T23:10:12.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.211+0000 7f506a914640 1 -- 172.21.15.32:0/1832315204 shutdown_connections 2024-04-24T23:10:12.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.211+0000 7f506a914640 1 -- 172.21.15.32:0/1832315204 >> 172.21.15.32:0/1832315204 conn(0x7f50640d0ae0 msgr2=0x7f50640d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:12.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.211+0000 7f506a914640 1 -- 172.21.15.32:0/1832315204 shutdown_connections 2024-04-24T23:10:12.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.211+0000 7f506a914640 1 -- 172.21.15.32:0/1832315204 wait complete. 2024-04-24T23:10:12.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.212+0000 7f506a914640 1 Processor -- start 2024-04-24T23:10:12.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.212+0000 7f506a914640 1 -- start start 2024-04-24T23:10:12.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.213+0000 7f506a914640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5064150820 0x7f50641615c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:12.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.213+0000 7f506a914640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50640cd4f0 0x7f50641594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:12.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.213+0000 7f506a914640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f506415fea0 0x7f5064159a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:12.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.213+0000 7f506a914640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5064071bd0 con 0x7f50640cd4f0 2024-04-24T23:10:12.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.213+0000 7f506a914640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5064067950 con 0x7f506415fea0 2024-04-24T23:10:12.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.213+0000 7f506a914640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f5064071d50 con 0x7f5064150820 2024-04-24T23:10:12.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.213+0000 7f50637fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50640cd4f0 0x7f50641594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:12.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.213+0000 7f50637fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50640cd4f0 0x7f50641594c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34794/0 (socket says 172.21.15.32:34794) 2024-04-24T23:10:12.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.213+0000 7f50637fe640 1 -- 172.21.15.32:0/1127911704 learned_addr learned my addr 172.21.15.32:0/1127911704 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:12.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.213+0000 7f5063fff640 1 --2- 172.21.15.32:0/1127911704 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5064150820 0x7f50641615c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:12.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.213+0000 7f5068eb1640 1 --2- 172.21.15.32:0/1127911704 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f506415fea0 0x7f5064159a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:12.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.213+0000 7f50637fe640 1 -- 172.21.15.32:0/1127911704 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5064150820 msgr2=0x7f50641615c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:12.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.213+0000 7f50637fe640 1 --2- 172.21.15.32:0/1127911704 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5064150820 0x7f50641615c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:12.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.213+0000 7f50637fe640 1 -- 172.21.15.32:0/1127911704 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f506415fea0 msgr2=0x7f5064159a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:12.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.213+0000 7f50637fe640 1 --2- 172.21.15.32:0/1127911704 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f506415fea0 0x7f5064159a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:12.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.213+0000 7f50637fe640 1 -- 172.21.15.32:0/1127911704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50640cd9d0 con 0x7f50640cd4f0 2024-04-24T23:10:12.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.213+0000 7f5063fff640 1 --2- 172.21.15.32:0/1127911704 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5064150820 0x7f50641615c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:10:12.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.213+0000 7f5068eb1640 1 --2- 172.21.15.32:0/1127911704 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f506415fea0 0x7f5064159a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:10:12.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.213+0000 7f50637fe640 1 --2- 172.21.15.32:0/1127911704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50640cd4f0 0x7f50641594c0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f505400f820 tx=0x7f505400fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:12.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.214+0000 7f5060ff9640 1 -- 172.21.15.32:0/1127911704 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50540050d0 con 0x7f50640cd4f0 2024-04-24T23:10:12.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.214+0000 7f5060ff9640 1 -- 172.21.15.32:0/1127911704 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5054018070 con 0x7f50640cd4f0 2024-04-24T23:10:12.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.214+0000 7f5060ff9640 1 -- 172.21.15.32:0/1127911704 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5054005490 con 0x7f50640cd4f0 2024-04-24T23:10:12.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.214+0000 7f506a914640 1 -- 172.21.15.32:0/1127911704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50640cdcb0 con 0x7f50640cd4f0 2024-04-24T23:10:12.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.214+0000 7f506a914640 1 -- 172.21.15.32:0/1127911704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50640ce270 con 0x7f50640cd4f0 2024-04-24T23:10:12.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.215+0000 7f506a914640 1 -- 172.21.15.32:0/1127911704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5064151770 con 0x7f50640cd4f0 2024-04-24T23:10:12.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.215+0000 7f5060ff9640 1 -- 172.21.15.32:0/1127911704 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f505400d730 con 0x7f50640cd4f0 2024-04-24T23:10:12.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.215+0000 7f5060ff9640 1 --2- 172.21.15.32:0/1127911704 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f505006be60 0x7f505006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:12.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.216+0000 7f5060ff9640 1 -- 172.21.15.32:0/1127911704 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 5767+0+0 (secure 0 0 0) 0x7f505408ad80 con 0x7f50640cd4f0 2024-04-24T23:10:12.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.216+0000 7f5060ff9640 1 -- 172.21.15.32:0/1127911704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=174}) -- 0x7f5050073a50 con 0x7f50640cd4f0 2024-04-24T23:10:12.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.216+0000 7f5063fff640 1 --2- 172.21.15.32:0/1127911704 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f505006be60 0x7f505006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:12.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.216+0000 7f5063fff640 1 --2- 172.21.15.32:0/1127911704 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f505006be60 0x7f505006e320 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f5058009bc0 tx=0x7f5058008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:12.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.220+0000 7f5060ff9640 1 -- 172.21.15.32:0/1127911704 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5064151770 con 0x7f50640cd4f0 2024-04-24T23:10:12.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.400+0000 7f506a914640 1 -- 172.21.15.32:0/1127911704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5064068430 con 0x7f50640cd4f0 2024-04-24T23:10:12.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.401+0000 7f5060ff9640 1 -- 172.21.15.32:0/1127911704 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v173) ==== 93+0+10 (secure 0 0 0) 0x7f50540596f0 con 0x7f50640cd4f0 2024-04-24T23:10:12.403 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T23:10:12.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.403+0000 7f506a914640 1 -- 172.21.15.32:0/1127911704 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f505006be60 msgr2=0x7f505006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:12.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.403+0000 7f506a914640 1 --2- 172.21.15.32:0/1127911704 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f505006be60 0x7f505006e320 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f5058009bc0 tx=0x7f5058008040 comp rx=0 tx=0).stop 2024-04-24T23:10:12.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.403+0000 7f506a914640 1 -- 172.21.15.32:0/1127911704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50640cd4f0 msgr2=0x7f50641594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:12.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.403+0000 7f506a914640 1 --2- 172.21.15.32:0/1127911704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50640cd4f0 0x7f50641594c0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f505400f820 tx=0x7f505400fce0 comp rx=0 tx=0).stop 2024-04-24T23:10:12.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.403+0000 7f506a914640 1 -- 172.21.15.32:0/1127911704 shutdown_connections 2024-04-24T23:10:12.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.403+0000 7f506a914640 1 --2- 172.21.15.32:0/1127911704 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f505006be60 0x7f505006e320 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:12.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.403+0000 7f506a914640 1 --2- 172.21.15.32:0/1127911704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50640cd4f0 0x7f50641594c0 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:12.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.403+0000 7f506a914640 1 --2- 172.21.15.32:0/1127911704 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f506415fea0 0x7f5064159a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:12.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.403+0000 7f506a914640 1 --2- 172.21.15.32:0/1127911704 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5064150820 0x7f50641615c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:12.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.403+0000 7f506a914640 1 -- 172.21.15.32:0/1127911704 >> 172.21.15.32:0/1127911704 conn(0x7f50640d0ae0 msgr2=0x7f50640b0870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:12.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.403+0000 7f506a914640 1 -- 172.21.15.32:0/1127911704 shutdown_connections 2024-04-24T23:10:12.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.403+0000 7f506a914640 1 -- 172.21.15.32:0/1127911704 wait complete. 2024-04-24T23:10:12.416 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:10:12.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.518+0000 7fa8e269a640 1 Processor -- start 2024-04-24T23:10:12.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.518+0000 7fa8e269a640 1 -- start start 2024-04-24T23:10:12.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.519+0000 7fa8e269a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa8dc158dd0 0x7fa8dc06b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:12.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.519+0000 7fa8e269a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa8dc06dc60 0x7fa8dc0b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:12.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.519+0000 7fa8e269a640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8dc15ef10 con 0x7fa8dc06b860 2024-04-24T23:10:12.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.519+0000 7fa8e269a640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fa8dc15f1f0 con 0x7fa8dc06dc60 2024-04-24T23:10:12.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.519+0000 7fa8e269a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa8dc15f4d0 con 0x7fa8dc158dd0 2024-04-24T23:10:12.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.519+0000 7fa8e0c37640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa8dc06dc60 0x7fa8dc0b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:12.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.519+0000 7fa8e0c37640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa8dc06dc60 0x7fa8dc0b5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34800/0 (socket says 172.21.15.32:34800) 2024-04-24T23:10:12.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.519+0000 7fa8e0c37640 1 -- 172.21.15.32:0/460958704 learned_addr learned my addr 172.21.15.32:0/460958704 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:12.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.519+0000 7fa8db7fe640 1 --1- 172.21.15.32:0/460958704 >> v1:172.21.15.32:6789/0 conn(0x7fa8dc06b860 0x7fa8dc06d550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47864/0 (socket says 172.21.15.32:47864) 2024-04-24T23:10:12.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.519+0000 7fa8dbfff640 1 --2- 172.21.15.32:0/460958704 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa8dc158dd0 0x7fa8dc06b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:12.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.520+0000 7fa8e0c37640 1 -- 172.21.15.32:0/460958704 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa8dc158dd0 msgr2=0x7fa8dc06b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:12.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.520+0000 7fa8e0c37640 1 --2- 172.21.15.32:0/460958704 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa8dc158dd0 0x7fa8dc06b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:12.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.520+0000 7fa8e0c37640 1 -- 172.21.15.32:0/460958704 >> v1:172.21.15.32:6789/0 conn(0x7fa8dc06b860 legacy=0x7fa8dc06d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:12.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.520+0000 7fa8e0c37640 1 -- 172.21.15.32:0/460958704 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8dc15f7b0 con 0x7fa8dc06dc60 2024-04-24T23:10:12.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.520+0000 7fa8daffd640 1 -- 172.21.15.32:0/460958704 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2342010620 0 0) 0x7fa8dc15ef10 con 0x7fa8dc06b860 2024-04-24T23:10:12.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.520+0000 7fa8dbfff640 1 --2- 172.21.15.32:0/460958704 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa8dc158dd0 0x7fa8dc06b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:10:12.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.520+0000 7fa8e0c37640 1 --2- 172.21.15.32:0/460958704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8dc06dc60 0x7fa8dc0b5d00 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4009e50 tx=0x7fa8d402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f36c83556496830b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:12.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.520+0000 7fa8daffd640 1 -- 172.21.15.32:0/460958704 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d4036070 con 0x7fa8dc06dc60 2024-04-24T23:10:12.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.520+0000 7fa8daffd640 1 -- 172.21.15.32:0/460958704 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8d4031600 con 0x7fa8dc06dc60 2024-04-24T23:10:12.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.521+0000 7fa8daffd640 1 -- 172.21.15.32:0/460958704 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d40319d0 con 0x7fa8dc06dc60 2024-04-24T23:10:12.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.521+0000 7fa8e269a640 1 -- 172.21.15.32:0/460958704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8dc06dc60 msgr2=0x7fa8dc0b5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:12.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.521+0000 7fa8e269a640 1 --2- 172.21.15.32:0/460958704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8dc06dc60 0x7fa8dc0b5d00 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4009e50 tx=0x7fa8d402a710 comp rx=0 tx=0).stop 2024-04-24T23:10:12.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.521+0000 7fa8e269a640 1 -- 172.21.15.32:0/460958704 shutdown_connections 2024-04-24T23:10:12.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.521+0000 7fa8e269a640 1 --2- 172.21.15.32:0/460958704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8dc06dc60 0x7fa8dc0b5d00 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:12.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.521+0000 7fa8e269a640 1 --2- 172.21.15.32:0/460958704 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa8dc158dd0 0x7fa8dc06b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:12.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.521+0000 7fa8e269a640 1 -- 172.21.15.32:0/460958704 >> 172.21.15.32:0/460958704 conn(0x7fa8dc06e800 msgr2=0x7fa8dc06ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:12.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.521+0000 7fa8e269a640 1 -- 172.21.15.32:0/460958704 shutdown_connections 2024-04-24T23:10:12.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.521+0000 7fa8e269a640 1 -- 172.21.15.32:0/460958704 wait complete. 2024-04-24T23:10:12.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.522+0000 7fa8e269a640 1 Processor -- start 2024-04-24T23:10:12.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.522+0000 7fa8e269a640 1 -- start start 2024-04-24T23:10:12.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.523+0000 7fa8e269a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa8dc06b860 0x7fa8dc140d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:12.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.523+0000 7fa8e269a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8dc06dc60 0x7fa8dc1518d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:12.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.523+0000 7fa8e269a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa8dc158dd0 0x7fa8dc151e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:12.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.523+0000 7fa8e269a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa8dc06bfe0 con 0x7fa8dc06dc60 2024-04-24T23:10:12.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.523+0000 7fa8e269a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa8dc15fcd0 con 0x7fa8dc06b860 2024-04-24T23:10:12.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.523+0000 7fa8e269a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa8dc1591b0 con 0x7fa8dc158dd0 2024-04-24T23:10:12.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.523+0000 7fa8db7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8dc06dc60 0x7fa8dc1518d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:12.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.523+0000 7fa8db7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8dc06dc60 0x7fa8dc1518d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34804/0 (socket says 172.21.15.32:34804) 2024-04-24T23:10:12.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.523+0000 7fa8db7fe640 1 -- 172.21.15.32:0/1166758592 learned_addr learned my addr 172.21.15.32:0/1166758592 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:12.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.524+0000 7fa8e0c37640 1 --2- 172.21.15.32:0/1166758592 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa8dc158dd0 0x7fa8dc151e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:12.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.524+0000 7fa8dbfff640 1 --2- 172.21.15.32:0/1166758592 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa8dc06b860 0x7fa8dc140d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:12.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.524+0000 7fa8db7fe640 1 -- 172.21.15.32:0/1166758592 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa8dc158dd0 msgr2=0x7fa8dc151e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:12.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.524+0000 7fa8db7fe640 1 --2- 172.21.15.32:0/1166758592 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa8dc158dd0 0x7fa8dc151e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:12.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.524+0000 7fa8db7fe640 1 -- 172.21.15.32:0/1166758592 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa8dc06b860 msgr2=0x7fa8dc140d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:12.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.524+0000 7fa8db7fe640 1 --2- 172.21.15.32:0/1166758592 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa8dc06b860 0x7fa8dc140d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:12.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.524+0000 7fa8db7fe640 1 -- 172.21.15.32:0/1166758592 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8dc0b3980 con 0x7fa8dc06dc60 2024-04-24T23:10:12.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.524+0000 7fa8dbfff640 1 --2- 172.21.15.32:0/1166758592 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa8dc06b860 0x7fa8dc140d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:10:12.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.524+0000 7fa8e0c37640 1 --2- 172.21.15.32:0/1166758592 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa8dc158dd0 0x7fa8dc151e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:10:12.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.524+0000 7fa8db7fe640 1 --2- 172.21.15.32:0/1166758592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8dc06dc60 0x7fa8dc1518d0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fa8c400c4d0 tx=0x7fa8c400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:12.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.524+0000 7fa8d8ff9640 1 -- 172.21.15.32:0/1166758592 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4018070 con 0x7fa8dc06dc60 2024-04-24T23:10:12.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.524+0000 7fa8d8ff9640 1 -- 172.21.15.32:0/1166758592 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8c4007500 con 0x7fa8dc06dc60 2024-04-24T23:10:12.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.524+0000 7fa8e269a640 1 -- 172.21.15.32:0/1166758592 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8dc14fb10 con 0x7fa8dc06dc60 2024-04-24T23:10:12.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.524+0000 7fa8d8ff9640 1 -- 172.21.15.32:0/1166758592 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4004a20 con 0x7fa8dc06dc60 2024-04-24T23:10:12.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.524+0000 7fa8e269a640 1 -- 172.21.15.32:0/1166758592 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8dc1500d0 con 0x7fa8dc06dc60 2024-04-24T23:10:12.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.525+0000 7fa8e269a640 1 -- 172.21.15.32:0/1166758592 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8dc1502d0 con 0x7fa8dc06dc60 2024-04-24T23:10:12.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.526+0000 7fa8d8ff9640 1 -- 172.21.15.32:0/1166758592 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa8c401b070 con 0x7fa8dc06dc60 2024-04-24T23:10:12.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.526+0000 7fa8d8ff9640 1 --2- 172.21.15.32:0/1166758592 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa8bc06be30 0x7fa8bc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:12.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.526+0000 7fa8d8ff9640 1 -- 172.21.15.32:0/1166758592 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 5767+0+0 (secure 0 0 0) 0x7fa8c408a190 con 0x7fa8dc06dc60 2024-04-24T23:10:12.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.526+0000 7fa8d8ff9640 1 -- 172.21.15.32:0/1166758592 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=174}) -- 0x7fa8bc073a20 con 0x7fa8dc06dc60 2024-04-24T23:10:12.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.526+0000 7fa8dbfff640 1 --2- 172.21.15.32:0/1166758592 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa8bc06be30 0x7fa8bc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:12.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.527+0000 7fa8dbfff640 1 --2- 172.21.15.32:0/1166758592 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa8bc06be30 0x7fa8bc06e2f0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc006fd0 tx=0x7fa8cc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:12.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.531+0000 7fa8d8ff9640 1 -- 172.21.15.32:0/1166758592 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa8dc1502d0 con 0x7fa8dc06dc60 2024-04-24T23:10:12.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.713+0000 7fa8e269a640 1 -- 172.21.15.32:0/1166758592 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa8dc06ee80 con 0x7fa8dc06dc60 2024-04-24T23:10:12.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.714+0000 7fa8d8ff9640 1 -- 172.21.15.32:0/1166758592 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v173) ==== 104+0+11 (secure 0 0 0) 0x7fa8c4058b90 con 0x7fa8dc06dc60 2024-04-24T23:10:12.715 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-24T23:10:12.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.715+0000 7fa8e269a640 1 -- 172.21.15.32:0/1166758592 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa8bc06be30 msgr2=0x7fa8bc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:12.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.715+0000 7fa8e269a640 1 --2- 172.21.15.32:0/1166758592 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa8bc06be30 0x7fa8bc06e2f0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc006fd0 tx=0x7fa8cc008040 comp rx=0 tx=0).stop 2024-04-24T23:10:12.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.715+0000 7fa8e269a640 1 -- 172.21.15.32:0/1166758592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8dc06dc60 msgr2=0x7fa8dc1518d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:12.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.715+0000 7fa8e269a640 1 --2- 172.21.15.32:0/1166758592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8dc06dc60 0x7fa8dc1518d0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fa8c400c4d0 tx=0x7fa8c400c990 comp rx=0 tx=0).stop 2024-04-24T23:10:12.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.715+0000 7fa8e269a640 1 -- 172.21.15.32:0/1166758592 shutdown_connections 2024-04-24T23:10:12.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.715+0000 7fa8e269a640 1 --2- 172.21.15.32:0/1166758592 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa8bc06be30 0x7fa8bc06e2f0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:12.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.715+0000 7fa8e269a640 1 --2- 172.21.15.32:0/1166758592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8dc06dc60 0x7fa8dc1518d0 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:12.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.715+0000 7fa8e269a640 1 --2- 172.21.15.32:0/1166758592 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa8dc158dd0 0x7fa8dc151e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:12.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.715+0000 7fa8e269a640 1 --2- 172.21.15.32:0/1166758592 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa8dc06b860 0x7fa8dc140d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:12.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.715+0000 7fa8e269a640 1 -- 172.21.15.32:0/1166758592 >> 172.21.15.32:0/1166758592 conn(0x7fa8dc06e800 msgr2=0x7fa8dc1607b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:12.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.716+0000 7fa8e269a640 1 -- 172.21.15.32:0/1166758592 shutdown_connections 2024-04-24T23:10:12.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.716+0000 7fa8e269a640 1 -- 172.21.15.32:0/1166758592 wait complete. 2024-04-24T23:10:12.728 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:10:12.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.830+0000 7fe3b006d640 1 Processor -- start 2024-04-24T23:10:12.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.830+0000 7fe3b006d640 1 -- start start 2024-04-24T23:10:12.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.831+0000 7fe3b006d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3a806cd00 0x7fe3a8167130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:12.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.831+0000 7fe3b006d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3a815c990 con 0x7fe3a815c5b0 2024-04-24T23:10:12.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.831+0000 7fe3b006d640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3a8071c50 con 0x7fe3a8072cf0 2024-04-24T23:10:12.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.831+0000 7fe3b006d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe3a80b4740 con 0x7fe3a806cd00 2024-04-24T23:10:12.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.831+0000 7fe3ade09640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe3a815c5b0 0x7fe3a80b4030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47866/0 (socket says 172.21.15.32:47866) 2024-04-24T23:10:12.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.831+0000 7fe3ade09640 1 -- 172.21.15.32:0/2763080160 learned_addr learned my addr 172.21.15.32:0/2763080160 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:10:12.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.831+0000 7fe3ace07640 1 -- 172.21.15.32:0/2763080160 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4114129731 0 0) 0x7fe3a4001110 con 0x7fe3a815c5b0 2024-04-24T23:10:12.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.831+0000 7fe3ae60a640 1 --2- 172.21.15.32:0/2763080160 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3a806cd00 0x7fe3a8167130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:12.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.831+0000 7fe3ace07640 1 -- 172.21.15.32:0/2763080160 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe394003520 con 0x7fe3a815c5b0 2024-04-24T23:10:12.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.831+0000 7fe3ace07640 1 -- 172.21.15.32:0/2763080160 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3987469475 0 0) 0x7fe394003520 con 0x7fe3a815c5b0 2024-04-24T23:10:12.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.832+0000 7fe3ace07640 1 -- 172.21.15.32:0/2763080160 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3a806cd00 msgr2=0x7fe3a8167130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:12.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.832+0000 7fe3ace07640 1 --2- 172.21.15.32:0/2763080160 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3a806cd00 0x7fe3a8167130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:12.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.832+0000 7fe3ace07640 1 -- 172.21.15.32:0/2763080160 >> v1:172.21.15.121:6789/0 conn(0x7fe3a8072cf0 legacy=0x7fe3a806c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:12.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.832+0000 7fe3ace07640 1 -- 172.21.15.32:0/2763080160 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3a8071f30 con 0x7fe3a815c5b0 2024-04-24T23:10:12.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.832+0000 7fe3ace07640 1 -- 172.21.15.32:0/2763080160 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7fe3a4001840 con 0x7fe3a815c5b0 2024-04-24T23:10:12.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.832+0000 7fe3ace07640 1 -- 172.21.15.32:0/2763080160 >> v1:172.21.15.32:6789/0 conn(0x7fe3a815c5b0 legacy=0x7fe3a80b4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:12.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.832+0000 7fe3ace07640 1 --2- 172.21.15.32:0/2763080160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3940039e0 0x7fe394003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:12.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.832+0000 7fe3ace07640 1 --2- 172.21.15.32:0/2763080160 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3a806cd00 0x7fe394004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:12.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.832+0000 7fe3ace07640 1 --2- 172.21.15.32:0/2763080160 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe394004d40 0x7fe3940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:12.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.832+0000 7fe3ace07640 1 -- 172.21.15.32:0/2763080160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe39400ba70 con 0x7fe3940039e0 2024-04-24T23:10:12.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.832+0000 7fe3ace07640 1 -- 172.21.15.32:0/2763080160 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe39400bc20 con 0x7fe394004d40 2024-04-24T23:10:12.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.832+0000 7fe3ace07640 1 -- 172.21.15.32:0/2763080160 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe39400be20 con 0x7fe3a806cd00 2024-04-24T23:10:12.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.832+0000 7fe3ade09640 1 --2- 172.21.15.32:0/2763080160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3940039e0 0x7fe394003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:12.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.832+0000 7fe3ad608640 1 --2- 172.21.15.32:0/2763080160 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3a806cd00 0x7fe394004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:12.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.832+0000 7fe3ae60a640 1 --2- 172.21.15.32:0/2763080160 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe394004d40 0x7fe3940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:12.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.832+0000 7fe3ade09640 1 -- 172.21.15.32:0/2763080160 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3a806cd00 msgr2=0x7fe394004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:12.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.832+0000 7fe3ade09640 1 --2- 172.21.15.32:0/2763080160 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3a806cd00 0x7fe394004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:12.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.832+0000 7fe3ade09640 1 -- 172.21.15.32:0/2763080160 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe394004d40 msgr2=0x7fe3940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:12.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.832+0000 7fe3ade09640 1 --2- 172.21.15.32:0/2763080160 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe394004d40 0x7fe3940051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:12.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.832+0000 7fe3ade09640 1 -- 172.21.15.32:0/2763080160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe394003520 con 0x7fe3940039e0 2024-04-24T23:10:12.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.833+0000 7fe3ade09640 1 --2- 172.21.15.32:0/2763080160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3940039e0 0x7fe394003dc0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fe3a400c4c0 tx=0x7fe394005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4005de7e028e12b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:12.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.833+0000 7fe3ace07640 1 -- 172.21.15.32:0/2763080160 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a400c0e0 con 0x7fe3940039e0 2024-04-24T23:10:12.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.833+0000 7fe3ace07640 1 -- 172.21.15.32:0/2763080160 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3a4034070 con 0x7fe3940039e0 2024-04-24T23:10:12.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.833+0000 7fe3ace07640 1 -- 172.21.15.32:0/2763080160 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a4005d10 con 0x7fe3940039e0 2024-04-24T23:10:12.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.833+0000 7fe3b006d640 1 -- 172.21.15.32:0/2763080160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3940039e0 msgr2=0x7fe394003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:12.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.833+0000 7fe3b006d640 1 --2- 172.21.15.32:0/2763080160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3940039e0 0x7fe394003dc0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fe3a400c4c0 tx=0x7fe394005830 comp rx=0 tx=0).stop 2024-04-24T23:10:12.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.833+0000 7fe3ae60a640 1 -- 172.21.15.32:0/2763080160 reap_dead start 2024-04-24T23:10:12.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.833+0000 7fe3b006d640 1 -- 172.21.15.32:0/2763080160 shutdown_connections 2024-04-24T23:10:12.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.833+0000 7fe3b006d640 1 -- 172.21.15.32:0/2763080160 >> 172.21.15.32:0/2763080160 conn(0x7fe3a806b110 msgr2=0x7fe3a806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:12.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.834+0000 7fe3b006d640 1 -- 172.21.15.32:0/2763080160 shutdown_connections 2024-04-24T23:10:12.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.834+0000 7fe3b006d640 1 -- 172.21.15.32:0/2763080160 wait complete. 2024-04-24T23:10:12.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.835+0000 7fe3b006d640 1 Processor -- start 2024-04-24T23:10:12.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.835+0000 7fe3b006d640 1 -- start start 2024-04-24T23:10:12.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.835+0000 7fe3b006d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3940039e0 0x7fe3a8143cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:12.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.835+0000 7fe3b006d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3a815e760 0x7fe3a815cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:12.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.835+0000 7fe3b006d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3a815d0b0 0x7fe3a80ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:12.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.835+0000 7fe3b006d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a80724f0 con 0x7fe3940039e0 2024-04-24T23:10:12.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.835+0000 7fe3b006d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a8072860 con 0x7fe3a815e760 2024-04-24T23:10:12.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.835+0000 7fe3b006d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe3a8071770 con 0x7fe3a815d0b0 2024-04-24T23:10:12.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.836+0000 7fe3ade09640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3940039e0 0x7fe3a8143cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:12.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.836+0000 7fe3ade09640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3940039e0 0x7fe3a8143cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34824/0 (socket says 172.21.15.32:34824) 2024-04-24T23:10:12.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.836+0000 7fe3ade09640 1 -- 172.21.15.32:0/29789958 learned_addr learned my addr 172.21.15.32:0/29789958 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:12.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.836+0000 7fe3ad608640 1 --2- 172.21.15.32:0/29789958 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3a815e760 0x7fe3a815cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:12.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.836+0000 7fe3ae60a640 1 --2- 172.21.15.32:0/29789958 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3a815d0b0 0x7fe3a80ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:12.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.836+0000 7fe3ade09640 1 -- 172.21.15.32:0/29789958 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3a815d0b0 msgr2=0x7fe3a80ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:12.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.836+0000 7fe3ade09640 1 --2- 172.21.15.32:0/29789958 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3a815d0b0 0x7fe3a80ae8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:12.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.836+0000 7fe3ade09640 1 -- 172.21.15.32:0/29789958 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3a815e760 msgr2=0x7fe3a815cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:12.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.836+0000 7fe3ade09640 1 --2- 172.21.15.32:0/29789958 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3a815e760 0x7fe3a815cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:12.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.836+0000 7fe3ade09640 1 -- 172.21.15.32:0/29789958 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3a8144200 con 0x7fe3940039e0 2024-04-24T23:10:12.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.836+0000 7fe3ade09640 1 --2- 172.21.15.32:0/29789958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3940039e0 0x7fe3a8143cc0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fe3a400c5e0 tx=0x7fe3a8072100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:12.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.836+0000 7fe39e7fc640 1 -- 172.21.15.32:0/29789958 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a4005e40 con 0x7fe3940039e0 2024-04-24T23:10:12.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.836+0000 7fe39e7fc640 1 -- 172.21.15.32:0/29789958 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3a4007070 con 0x7fe3940039e0 2024-04-24T23:10:12.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.836+0000 7fe3b006d640 1 -- 172.21.15.32:0/29789958 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3a80cdb20 con 0x7fe3940039e0 2024-04-24T23:10:12.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.836+0000 7fe39e7fc640 1 -- 172.21.15.32:0/29789958 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a4007420 con 0x7fe3940039e0 2024-04-24T23:10:12.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.836+0000 7fe3b006d640 1 -- 172.21.15.32:0/29789958 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3a815d9a0 con 0x7fe3940039e0 2024-04-24T23:10:12.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.837+0000 7fe3b006d640 1 -- 172.21.15.32:0/29789958 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3a8068430 con 0x7fe3940039e0 2024-04-24T23:10:12.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.838+0000 7fe39e7fc640 1 -- 172.21.15.32:0/29789958 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe3a402e070 con 0x7fe3940039e0 2024-04-24T23:10:12.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.838+0000 7fe39e7fc640 1 --2- 172.21.15.32:0/29789958 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe38c06be30 0x7fe38c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:12.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.838+0000 7fe39e7fc640 1 -- 172.21.15.32:0/29789958 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 5767+0+0 (secure 0 0 0) 0x7fe3a40ab2a0 con 0x7fe3940039e0 2024-04-24T23:10:12.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.838+0000 7fe39e7fc640 1 -- 172.21.15.32:0/29789958 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=174}) -- 0x7fe38c073a80 con 0x7fe3940039e0 2024-04-24T23:10:12.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.838+0000 7fe3ad608640 1 --2- 172.21.15.32:0/29789958 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe38c06be30 0x7fe38c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:12.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.839+0000 7fe3ad608640 1 --2- 172.21.15.32:0/29789958 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe38c06be30 0x7fe38c06e2f0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8156470 tx=0x7fe398009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:12.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:12.842+0000 7fe39e7fc640 1 -- 172.21.15.32:0/29789958 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe3a8144200 con 0x7fe3940039e0 2024-04-24T23:10:13.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.024+0000 7fe3b006d640 1 -- 172.21.15.32:0/29789958 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe3a80b52b0 con 0x7fe3940039e0 2024-04-24T23:10:13.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.024+0000 7fe39e7fc640 1 -- 172.21.15.32:0/29789958 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v173) ==== 100+0+11 (secure 0 0 0) 0x7fe3a4040070 con 0x7fe3940039e0 2024-04-24T23:10:13.026 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-24T23:10:13.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.026+0000 7fe3b006d640 1 -- 172.21.15.32:0/29789958 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe38c06be30 msgr2=0x7fe38c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:13.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.026+0000 7fe3b006d640 1 --2- 172.21.15.32:0/29789958 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe38c06be30 0x7fe38c06e2f0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8156470 tx=0x7fe398009040 comp rx=0 tx=0).stop 2024-04-24T23:10:13.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.026+0000 7fe3b006d640 1 -- 172.21.15.32:0/29789958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3940039e0 msgr2=0x7fe3a8143cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:13.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.026+0000 7fe3b006d640 1 --2- 172.21.15.32:0/29789958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3940039e0 0x7fe3a8143cc0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fe3a400c5e0 tx=0x7fe3a8072100 comp rx=0 tx=0).stop 2024-04-24T23:10:13.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.026+0000 7fe3b006d640 1 -- 172.21.15.32:0/29789958 shutdown_connections 2024-04-24T23:10:13.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.026+0000 7fe3b006d640 1 --2- 172.21.15.32:0/29789958 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe38c06be30 0x7fe38c06e2f0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:13.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.026+0000 7fe3b006d640 1 --2- 172.21.15.32:0/29789958 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3a815d0b0 0x7fe3a80ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:13.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.026+0000 7fe3b006d640 1 --2- 172.21.15.32:0/29789958 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3a815e760 0x7fe3a815cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:13.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.026+0000 7fe3b006d640 1 --2- 172.21.15.32:0/29789958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3940039e0 0x7fe3a8143cc0 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:13.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.026+0000 7fe3b006d640 1 -- 172.21.15.32:0/29789958 >> 172.21.15.32:0/29789958 conn(0x7fe3a806b110 msgr2=0x7fe3a80b0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:13.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.026+0000 7fe3b006d640 1 -- 172.21.15.32:0/29789958 shutdown_connections 2024-04-24T23:10:13.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.026+0000 7fe3b006d640 1 -- 172.21.15.32:0/29789958 wait complete. 2024-04-24T23:10:13.039 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T23:10:13.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.139+0000 7ffa8889e640 1 Processor -- start 2024-04-24T23:10:13.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.139+0000 7ffa8889e640 1 -- start start 2024-04-24T23:10:13.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.140+0000 7ffa8889e640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ffa840b5560 0x7ffa8415e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:13.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.140+0000 7ffa8889e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa840cd910 con 0x7ffa840b6050 2024-04-24T23:10:13.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.140+0000 7ffa8889e640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa840cdbf0 con 0x7ffa840cd3c0 2024-04-24T23:10:13.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.140+0000 7ffa8889e640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7ffa840b6470 con 0x7ffa840b5560 2024-04-24T23:10:13.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.140+0000 7ffa82d9d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ffa840b6050 0x7ffa841514b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47878/0 (socket says 172.21.15.32:47878) 2024-04-24T23:10:13.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.140+0000 7ffa82d9d640 1 -- 172.21.15.32:0/813959443 learned_addr learned my addr 172.21.15.32:0/813959443 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:10:13.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.140+0000 7ffa8259c640 1 --2- 172.21.15.32:0/813959443 >> v2:172.21.15.121:3300/0 conn(0x7ffa840b5560 0x7ffa8415e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:13.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.140+0000 7ffa8159a640 1 -- 172.21.15.32:0/813959443 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 835895209 0 0) 0x7ffa840cd910 con 0x7ffa840b6050 2024-04-24T23:10:13.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.140+0000 7ffa8159a640 1 -- 172.21.15.32:0/813959443 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa70003440 con 0x7ffa840b6050 2024-04-24T23:10:13.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.141+0000 7ffa8159a640 1 -- 172.21.15.32:0/813959443 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3227028563 0 0) 0x7ffa70003440 con 0x7ffa840b6050 2024-04-24T23:10:13.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.141+0000 7ffa8159a640 1 -- 172.21.15.32:0/813959443 >> v2:172.21.15.121:3300/0 conn(0x7ffa840b5560 msgr2=0x7ffa8415e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:13.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.141+0000 7ffa8159a640 1 --2- 172.21.15.32:0/813959443 >> v2:172.21.15.121:3300/0 conn(0x7ffa840b5560 0x7ffa8415e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:13.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.141+0000 7ffa8159a640 1 -- 172.21.15.32:0/813959443 >> v1:172.21.15.121:6789/0 conn(0x7ffa840cd3c0 legacy=0x7ffa840b5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:13.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.141+0000 7ffa8159a640 1 -- 172.21.15.32:0/813959443 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa840cded0 con 0x7ffa840b6050 2024-04-24T23:10:13.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.141+0000 7ffa8159a640 1 -- 172.21.15.32:0/813959443 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7ffa740019f0 con 0x7ffa840b6050 2024-04-24T23:10:13.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.141+0000 7ffa8159a640 1 -- 172.21.15.32:0/813959443 >> v1:172.21.15.32:6789/0 conn(0x7ffa840b6050 legacy=0x7ffa841514b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:13.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.141+0000 7ffa8159a640 1 --2- 172.21.15.32:0/813959443 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa70003aa0 0x7ffa70003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:13.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.141+0000 7ffa8159a640 1 --2- 172.21.15.32:0/813959443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa70008850 0x7ffa7000acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:13.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.141+0000 7ffa8159a640 1 --2- 172.21.15.32:0/813959443 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa7000b4c0 0x7ffa7000d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:13.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.141+0000 7ffa8159a640 1 -- 172.21.15.32:0/813959443 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7000e230 con 0x7ffa70008850 2024-04-24T23:10:13.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.141+0000 7ffa8159a640 1 -- 172.21.15.32:0/813959443 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7000e3e0 con 0x7ffa70003aa0 2024-04-24T23:10:13.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.141+0000 7ffa8159a640 1 -- 172.21.15.32:0/813959443 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ffa7000e5e0 con 0x7ffa7000b4c0 2024-04-24T23:10:13.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.141+0000 7ffa81d9b640 1 --2- 172.21.15.32:0/813959443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa70008850 0x7ffa7000acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:13.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.141+0000 7ffa8259c640 1 --2- 172.21.15.32:0/813959443 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa70003aa0 0x7ffa70003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:13.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.141+0000 7ffa82d9d640 1 --2- 172.21.15.32:0/813959443 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa7000b4c0 0x7ffa7000d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:13.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.141+0000 7ffa81d9b640 1 -- 172.21.15.32:0/813959443 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa7000b4c0 msgr2=0x7ffa7000d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:13.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.141+0000 7ffa81d9b640 1 --2- 172.21.15.32:0/813959443 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa7000b4c0 0x7ffa7000d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:13.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.141+0000 7ffa81d9b640 1 -- 172.21.15.32:0/813959443 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa70003aa0 msgr2=0x7ffa70003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:13.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.141+0000 7ffa81d9b640 1 --2- 172.21.15.32:0/813959443 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa70003aa0 0x7ffa70003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:13.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.141+0000 7ffa81d9b640 1 -- 172.21.15.32:0/813959443 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa70003440 con 0x7ffa70008850 2024-04-24T23:10:13.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.141+0000 7ffa82d9d640 1 --2- 172.21.15.32:0/813959443 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa7000b4c0 0x7ffa7000d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:10:13.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.141+0000 7ffa82d9d640 1 -- 172.21.15.32:0/813959443 reap_dead start 2024-04-24T23:10:13.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.142+0000 7ffa81d9b640 1 --2- 172.21.15.32:0/813959443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa70008850 0x7ffa7000acc0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c001750 tx=0x7ffa70003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c17df6c763bcb64 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:13.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.142+0000 7ffa8159a640 1 -- 172.21.15.32:0/813959443 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c001370 con 0x7ffa70008850 2024-04-24T23:10:13.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.142+0000 7ffa8159a640 1 -- 172.21.15.32:0/813959443 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa6c0052d0 con 0x7ffa70008850 2024-04-24T23:10:13.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.142+0000 7ffa8159a640 1 -- 172.21.15.32:0/813959443 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c02abc0 con 0x7ffa70008850 2024-04-24T23:10:13.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.142+0000 7ffa8889e640 1 -- 172.21.15.32:0/813959443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa70008850 msgr2=0x7ffa7000acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:13.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.142+0000 7ffa8889e640 1 --2- 172.21.15.32:0/813959443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa70008850 0x7ffa7000acc0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c001750 tx=0x7ffa70003730 comp rx=0 tx=0).stop 2024-04-24T23:10:13.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.143+0000 7ffa8889e640 1 -- 172.21.15.32:0/813959443 shutdown_connections 2024-04-24T23:10:13.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.143+0000 7ffa8889e640 1 --2- 172.21.15.32:0/813959443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa70008850 0x7ffa7000acc0 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:13.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.143+0000 7ffa8889e640 1 -- 172.21.15.32:0/813959443 >> 172.21.15.32:0/813959443 conn(0x7ffa8406b110 msgr2=0x7ffa8406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:13.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.143+0000 7ffa8889e640 1 -- 172.21.15.32:0/813959443 shutdown_connections 2024-04-24T23:10:13.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.143+0000 7ffa8889e640 1 -- 172.21.15.32:0/813959443 wait complete. 2024-04-24T23:10:13.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.144+0000 7ffa8889e640 1 Processor -- start 2024-04-24T23:10:13.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.144+0000 7ffa8889e640 1 -- start start 2024-04-24T23:10:13.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.145+0000 7ffa8889e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa70008850 0x7ffa8415d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:13.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.145+0000 7ffa8889e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa84141b00 0x7ffa84071d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:13.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.145+0000 7ffa8889e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa84072280 0x7ffa84150b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:13.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.145+0000 7ffa8889e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ffa840b5940 con 0x7ffa70008850 2024-04-24T23:10:13.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.145+0000 7ffa8889e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ffa840ce490 con 0x7ffa84072280 2024-04-24T23:10:13.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.145+0000 7ffa8889e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ffa840b5ac0 con 0x7ffa84141b00 2024-04-24T23:10:13.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.145+0000 7ffa8259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa70008850 0x7ffa8415d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:13.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.145+0000 7ffa8259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa70008850 0x7ffa8415d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34842/0 (socket says 172.21.15.32:34842) 2024-04-24T23:10:13.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.145+0000 7ffa8259c640 1 -- 172.21.15.32:0/1391725861 learned_addr learned my addr 172.21.15.32:0/1391725861 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:13.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.145+0000 7ffa81d9b640 1 --2- 172.21.15.32:0/1391725861 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa84141b00 0x7ffa84071d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:13.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.145+0000 7ffa82d9d640 1 --2- 172.21.15.32:0/1391725861 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa84072280 0x7ffa84150b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:13.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.145+0000 7ffa8259c640 1 -- 172.21.15.32:0/1391725861 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa84141b00 msgr2=0x7ffa84071d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:13.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.145+0000 7ffa8259c640 1 --2- 172.21.15.32:0/1391725861 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa84141b00 0x7ffa84071d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:13.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.145+0000 7ffa8259c640 1 -- 172.21.15.32:0/1391725861 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa84072280 msgr2=0x7ffa84150b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:13.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.145+0000 7ffa8259c640 1 --2- 172.21.15.32:0/1391725861 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa84072280 0x7ffa84150b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:13.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.145+0000 7ffa8259c640 1 -- 172.21.15.32:0/1391725861 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa8415c2d0 con 0x7ffa70008850 2024-04-24T23:10:13.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.145+0000 7ffa82d9d640 1 --2- 172.21.15.32:0/1391725861 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa84072280 0x7ffa84150b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:10:13.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.145+0000 7ffa8259c640 1 --2- 172.21.15.32:0/1391725861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa70008850 0x7ffa8415d370 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c011aa0 tx=0x7ffa7c011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:13.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.145+0000 7ffa7affd640 1 -- 172.21.15.32:0/1391725861 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c00fb20 con 0x7ffa70008850 2024-04-24T23:10:13.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.145+0000 7ffa7affd640 1 -- 172.21.15.32:0/1391725861 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa7c00fcc0 con 0x7ffa70008850 2024-04-24T23:10:13.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.145+0000 7ffa8889e640 1 -- 172.21.15.32:0/1391725861 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa8415c5b0 con 0x7ffa70008850 2024-04-24T23:10:13.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.146+0000 7ffa7affd640 1 -- 172.21.15.32:0/1391725861 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c014660 con 0x7ffa70008850 2024-04-24T23:10:13.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.146+0000 7ffa8889e640 1 -- 172.21.15.32:0/1391725861 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa8415cb70 con 0x7ffa70008850 2024-04-24T23:10:13.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.146+0000 7ffa8889e640 1 -- 172.21.15.32:0/1391725861 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa84068430 con 0x7ffa70008850 2024-04-24T23:10:13.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.147+0000 7ffa7affd640 1 -- 172.21.15.32:0/1391725861 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffa7c012040 con 0x7ffa70008850 2024-04-24T23:10:13.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.147+0000 7ffa7affd640 1 --2- 172.21.15.32:0/1391725861 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ffa6406be60 0x7ffa6406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:13.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.147+0000 7ffa7affd640 1 -- 172.21.15.32:0/1391725861 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 5767+0+0 (secure 0 0 0) 0x7ffa7c08b4f0 con 0x7ffa70008850 2024-04-24T23:10:13.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.148+0000 7ffa7affd640 1 -- 172.21.15.32:0/1391725861 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=174}) -- 0x7ffa64073a50 con 0x7ffa70008850 2024-04-24T23:10:13.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.148+0000 7ffa81d9b640 1 --2- 172.21.15.32:0/1391725861 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ffa6406be60 0x7ffa6406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:13.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.148+0000 7ffa81d9b640 1 --2- 172.21.15.32:0/1391725861 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ffa6406be60 0x7ffa6406e320 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c032000 tx=0x7ffa6c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:13.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.152+0000 7ffa7affd640 1 -- 172.21.15.32:0/1391725861 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ffa7c019030 con 0x7ffa70008850 2024-04-24T23:10:13.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.350+0000 7ffa8889e640 1 -- 172.21.15.32:0/1391725861 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ffa840afa20 con 0x7ffa70008850 2024-04-24T23:10:13.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.350+0000 7ffa7affd640 1 -- 172.21.15.32:0/1391725861 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v173) ==== 80+0+0 (secure 0 0 0) 0x7ffa7c059e60 con 0x7ffa70008850 2024-04-24T23:10:13.352 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2024-04-24T23:10:13.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.352+0000 7ffa8889e640 1 -- 172.21.15.32:0/1391725861 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ffa6406be60 msgr2=0x7ffa6406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:13.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.352+0000 7ffa8889e640 1 --2- 172.21.15.32:0/1391725861 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ffa6406be60 0x7ffa6406e320 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c032000 tx=0x7ffa6c02e000 comp rx=0 tx=0).stop 2024-04-24T23:10:13.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.352+0000 7ffa8889e640 1 -- 172.21.15.32:0/1391725861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa70008850 msgr2=0x7ffa8415d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:13.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.352+0000 7ffa8889e640 1 --2- 172.21.15.32:0/1391725861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa70008850 0x7ffa8415d370 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c011aa0 tx=0x7ffa7c011f60 comp rx=0 tx=0).stop 2024-04-24T23:10:13.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.352+0000 7ffa8889e640 1 -- 172.21.15.32:0/1391725861 shutdown_connections 2024-04-24T23:10:13.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.352+0000 7ffa8889e640 1 --2- 172.21.15.32:0/1391725861 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ffa6406be60 0x7ffa6406e320 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:13.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.352+0000 7ffa8889e640 1 --2- 172.21.15.32:0/1391725861 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa84072280 0x7ffa84150b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:13.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.352+0000 7ffa8889e640 1 --2- 172.21.15.32:0/1391725861 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa84141b00 0x7ffa84071d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:13.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.352+0000 7ffa8889e640 1 --2- 172.21.15.32:0/1391725861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa70008850 0x7ffa8415d370 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:13.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.352+0000 7ffa8889e640 1 -- 172.21.15.32:0/1391725861 >> 172.21.15.32:0/1391725861 conn(0x7ffa8406b110 msgr2=0x7ffa840b0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:13.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.352+0000 7ffa8889e640 1 -- 172.21.15.32:0/1391725861 shutdown_connections 2024-04-24T23:10:13.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.352+0000 7ffa8889e640 1 -- 172.21.15.32:0/1391725861 wait complete. 2024-04-24T23:10:13.365 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-24T23:10:13.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:10:13.365 DEBUG:teuthology.orchestra.run.smithi032:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-24T23:10:13.394 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2024-04-24T23:10:13.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:10:13.395 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-24T23:10:13.451 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:10:13.451 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:10:13.451 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:10:13.470 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:10:13.470 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:10:13.528 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:10:13.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.627+0000 7f02462d6640 1 Processor -- start 2024-04-24T23:10:13.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.627+0000 7f02462d6640 1 -- start start 2024-04-24T23:10:13.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.628+0000 7f02462d6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f024006c5f0 0x7f024006c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:13.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.628+0000 7f02462d6640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02400b6430 con 0x7f02400af890 2024-04-24T23:10:13.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.628+0000 7f02462d6640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0240156f80 con 0x7f02400b44d0 2024-04-24T23:10:13.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.628+0000 7f02462d6640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f02400b6150 con 0x7f024006c5f0 2024-04-24T23:10:13.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.628+0000 7f023ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f024006c5f0 0x7f024006c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:13.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.628+0000 7f0244873640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f02400af890 0x7f024015cac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47890/0 (socket says 172.21.15.32:47890) 2024-04-24T23:10:13.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.628+0000 7f0244873640 1 -- 172.21.15.32:0/1970424615 learned_addr learned my addr 172.21.15.32:0/1970424615 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:10:13.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.628+0000 7f023ffff640 1 -- 172.21.15.32:0/1970424615 >> v1:172.21.15.121:6789/0 conn(0x7f02400b44d0 legacy=0x7f02400b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:13.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.628+0000 7f023ffff640 1 -- 172.21.15.32:0/1970424615 >> v1:172.21.15.32:6789/0 conn(0x7f02400af890 legacy=0x7f024015cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:13.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.628+0000 7f023ffff640 1 -- 172.21.15.32:0/1970424615 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02400b39f0 con 0x7f024006c5f0 2024-04-24T23:10:13.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.629+0000 7f023ffff640 1 --2- 172.21.15.32:0/1970424615 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f024006c5f0 0x7f024006c9d0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f02340097a0 tx=0x7f023402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e9ad415934afe98 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:13.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.629+0000 7f023effd640 1 -- 172.21.15.32:0/1970424615 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0234037070 con 0x7f024006c5f0 2024-04-24T23:10:13.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.629+0000 7f023effd640 1 -- 172.21.15.32:0/1970424615 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0234032440 con 0x7f024006c5f0 2024-04-24T23:10:13.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.629+0000 7f023effd640 1 -- 172.21.15.32:0/1970424615 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0234032810 con 0x7f024006c5f0 2024-04-24T23:10:13.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.630+0000 7f02462d6640 1 -- 172.21.15.32:0/1970424615 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f024006c5f0 msgr2=0x7f024006c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:13.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.630+0000 7f02462d6640 1 --2- 172.21.15.32:0/1970424615 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f024006c5f0 0x7f024006c9d0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f02340097a0 tx=0x7f023402a1e0 comp rx=0 tx=0).stop 2024-04-24T23:10:13.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.630+0000 7f02462d6640 1 -- 172.21.15.32:0/1970424615 shutdown_connections 2024-04-24T23:10:13.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.630+0000 7f02462d6640 1 --2- 172.21.15.32:0/1970424615 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f024006c5f0 0x7f024006c9d0 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:13.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.630+0000 7f02462d6640 1 -- 172.21.15.32:0/1970424615 >> 172.21.15.32:0/1970424615 conn(0x7f024006b110 msgr2=0x7f024006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:13.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.630+0000 7f02462d6640 1 -- 172.21.15.32:0/1970424615 shutdown_connections 2024-04-24T23:10:13.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.630+0000 7f02462d6640 1 -- 172.21.15.32:0/1970424615 wait complete. 2024-04-24T23:10:13.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.631+0000 7f02462d6640 1 Processor -- start 2024-04-24T23:10:13.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.631+0000 7f02462d6640 1 -- start start 2024-04-24T23:10:13.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.632+0000 7f02462d6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f024006c5f0 0x7f024013d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:13.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.632+0000 7f02462d6640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02400af890 0x7f02401e5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:13.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.632+0000 7f02462d6640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02400b44d0 0x7f024013b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:13.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.632+0000 7f02462d6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f024006cf10 con 0x7f024006c5f0 2024-04-24T23:10:13.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.632+0000 7f02462d6640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f02400b3fb0 con 0x7f02400b44d0 2024-04-24T23:10:13.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.632+0000 7f02462d6640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f024015d1d0 con 0x7f02400af890 2024-04-24T23:10:13.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.632+0000 7f023ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f024006c5f0 0x7f024013d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:13.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.632+0000 7f023ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f024006c5f0 0x7f024013d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34874/0 (socket says 172.21.15.32:34874) 2024-04-24T23:10:13.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.632+0000 7f023ffff640 1 -- 172.21.15.32:0/1217384123 learned_addr learned my addr 172.21.15.32:0/1217384123 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:13.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.632+0000 7f0244873640 1 --2- 172.21.15.32:0/1217384123 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02400b44d0 0x7f024013b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:13.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.632+0000 7f023f7fe640 1 --2- 172.21.15.32:0/1217384123 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02400af890 0x7f02401e5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:13.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.632+0000 7f023ffff640 1 -- 172.21.15.32:0/1217384123 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02400af890 msgr2=0x7f02401e5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:13.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.632+0000 7f023ffff640 1 --2- 172.21.15.32:0/1217384123 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02400af890 0x7f02401e5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:13.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.632+0000 7f023ffff640 1 -- 172.21.15.32:0/1217384123 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02400b44d0 msgr2=0x7f024013b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:13.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.632+0000 7f023ffff640 1 --2- 172.21.15.32:0/1217384123 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02400b44d0 0x7f024013b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:13.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.632+0000 7f023ffff640 1 -- 172.21.15.32:0/1217384123 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02401e58c0 con 0x7f024006c5f0 2024-04-24T23:10:13.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.632+0000 7f023ffff640 1 --2- 172.21.15.32:0/1217384123 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f024006c5f0 0x7f024013d320 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f0234000c00 tx=0x7f02340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:13.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.633+0000 7f023cff9640 1 -- 172.21.15.32:0/1217384123 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0234041070 con 0x7f024006c5f0 2024-04-24T23:10:13.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.633+0000 7f023cff9640 1 -- 172.21.15.32:0/1217384123 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f023403c930 con 0x7f024006c5f0 2024-04-24T23:10:13.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.633+0000 7f023cff9640 1 -- 172.21.15.32:0/1217384123 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0234037040 con 0x7f024006c5f0 2024-04-24T23:10:13.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.633+0000 7f02462d6640 1 -- 172.21.15.32:0/1217384123 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f024013c280 con 0x7f024006c5f0 2024-04-24T23:10:13.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.633+0000 7f02462d6640 1 -- 172.21.15.32:0/1217384123 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f024013c840 con 0x7f024006c5f0 2024-04-24T23:10:13.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.634+0000 7f02462d6640 1 -- 172.21.15.32:0/1217384123 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0240068430 con 0x7f024006c5f0 2024-04-24T23:10:13.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.634+0000 7f023cff9640 1 -- 172.21.15.32:0/1217384123 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0234044070 con 0x7f024006c5f0 2024-04-24T23:10:13.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.634+0000 7f023cff9640 1 --2- 172.21.15.32:0/1217384123 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f022c06be30 0x7f022c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:13.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.635+0000 7f023cff9640 1 -- 172.21.15.32:0/1217384123 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 5767+0+0 (secure 0 0 0) 0x7f02340abb50 con 0x7f024006c5f0 2024-04-24T23:10:13.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.635+0000 7f023f7fe640 1 --2- 172.21.15.32:0/1217384123 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f022c06be30 0x7f022c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:13.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.635+0000 7f023cff9640 1 -- 172.21.15.32:0/1217384123 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=174}) -- 0x7f022c073a20 con 0x7f024006c5f0 2024-04-24T23:10:13.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.635+0000 7f023f7fe640 1 --2- 172.21.15.32:0/1217384123 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f022c06be30 0x7f022c06e2f0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f02401e6a60 tx=0x7f0230008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:13.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.639+0000 7f023cff9640 1 -- 172.21.15.32:0/1217384123 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f023407a5e0 con 0x7f024006c5f0 2024-04-24T23:10:13.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.825+0000 7f02462d6640 1 -- 172.21.15.32:0/1217384123 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f024015ee10 con 0x7f024006c5f0 2024-04-24T23:10:13.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.826+0000 7f023cff9640 1 -- 172.21.15.32:0/1217384123 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v173) ==== 74+0+16756 (secure 0 0 0) 0x7f0234079ee0 con 0x7f024006c5f0 2024-04-24T23:10:13.828 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:10:13.828 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":173,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:09:53.858501+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:08:59.076360+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-24T23:08:59.745667+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.1d","mappings":[{"from":2,"to":5}]},{"pgid":"21.39","mappings":[{"from":2,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:10:13.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.828+0000 7f02462d6640 1 -- 172.21.15.32:0/1217384123 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f022c06be30 msgr2=0x7f022c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:13.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.828+0000 7f02462d6640 1 --2- 172.21.15.32:0/1217384123 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f022c06be30 0x7f022c06e2f0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f02401e6a60 tx=0x7f0230008040 comp rx=0 tx=0).stop 2024-04-24T23:10:13.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.828+0000 7f02462d6640 1 -- 172.21.15.32:0/1217384123 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f024006c5f0 msgr2=0x7f024013d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:13.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.828+0000 7f02462d6640 1 --2- 172.21.15.32:0/1217384123 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f024006c5f0 0x7f024013d320 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f0234000c00 tx=0x7f02340046c0 comp rx=0 tx=0).stop 2024-04-24T23:10:13.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.828+0000 7f02462d6640 1 -- 172.21.15.32:0/1217384123 shutdown_connections 2024-04-24T23:10:13.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.828+0000 7f02462d6640 1 --2- 172.21.15.32:0/1217384123 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f022c06be30 0x7f022c06e2f0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:13.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.828+0000 7f02462d6640 1 --2- 172.21.15.32:0/1217384123 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02400b44d0 0x7f024013b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:13.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.828+0000 7f02462d6640 1 --2- 172.21.15.32:0/1217384123 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02400af890 0x7f02401e5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:13.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.828+0000 7f02462d6640 1 --2- 172.21.15.32:0/1217384123 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f024006c5f0 0x7f024013d320 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:13.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.828+0000 7f02462d6640 1 -- 172.21.15.32:0/1217384123 >> 172.21.15.32:0/1217384123 conn(0x7f024006b110 msgr2=0x7f02400d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:13.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.828+0000 7f02462d6640 1 -- 172.21.15.32:0/1217384123 shutdown_connections 2024-04-24T23:10:13.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.828+0000 7f02462d6640 1 -- 172.21.15.32:0/1217384123 wait complete. 2024-04-24T23:10:13.843 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:08:59.076360+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '168', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:08:59.745667+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '171', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:10:13.843 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:10:13.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.947+0000 7f7c6aeca640 1 Processor -- start 2024-04-24T23:10:13.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.947+0000 7f7c6aeca640 1 -- start start 2024-04-24T23:10:13.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.948+0000 7f7c6aeca640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c6406b820 0x7f7c640d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:13.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.948+0000 7f7c6aeca640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7c640d4210 0x7f7c640b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:13.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.948+0000 7f7c6aeca640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c640d3170 con 0x7f7c6415e9b0 2024-04-24T23:10:13.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.948+0000 7f7c6aeca640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f7c640d3450 con 0x7f7c640d4210 2024-04-24T23:10:13.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.948+0000 7f7c6aeca640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f7c640d3730 con 0x7f7c6406b820 2024-04-24T23:10:13.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.949+0000 7f7c63fff640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c6406b820 0x7f7c640d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:13.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.949+0000 7f7c69467640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7c640d4210 0x7f7c640b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:13.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.949+0000 7f7c69467640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7c640d4210 0x7f7c640b78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.32:59176/0 (socket says 172.21.15.32:59176) 2024-04-24T23:10:13.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.949+0000 7f7c68c66640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f7c6415e9b0 0x7f7c6406b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:57280/0 (socket says 172.21.15.32:57280) 2024-04-24T23:10:13.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.949+0000 7f7c63fff640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c6406b820 0x7f7c640d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.32:43280/0 (socket says 172.21.15.32:43280) 2024-04-24T23:10:13.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.949+0000 7f7c69467640 1 -- 172.21.15.32:0/1338160373 learned_addr learned my addr 172.21.15.32:0/1338160373 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:13.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.949+0000 7f7c69467640 1 -- 172.21.15.32:0/1338160373 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c6406b820 msgr2=0x7f7c640d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:13.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.949+0000 7f7c69467640 1 --2- 172.21.15.32:0/1338160373 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c6406b820 0x7f7c640d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:13.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.949+0000 7f7c69467640 1 -- 172.21.15.32:0/1338160373 >> v1:172.21.15.121:6789/0 conn(0x7f7c6415e9b0 legacy=0x7f7c6406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:13.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.949+0000 7f7c69467640 1 -- 172.21.15.32:0/1338160373 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c640d3a10 con 0x7f7c640d4210 2024-04-24T23:10:13.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.950+0000 7f7c69467640 1 --2- 172.21.15.32:0/1338160373 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c640d4210 0x7f7c640b78f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f7c54009870 tx=0x7f7c5402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88d0614a2e089d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:13.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.950+0000 7f7c637fe640 1 -- 172.21.15.32:0/1338160373 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c54037070 con 0x7f7c640d4210 2024-04-24T23:10:13.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.950+0000 7f7c637fe640 1 -- 172.21.15.32:0/1338160373 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c54032440 con 0x7f7c640d4210 2024-04-24T23:10:13.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.950+0000 7f7c637fe640 1 -- 172.21.15.32:0/1338160373 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c54032810 con 0x7f7c640d4210 2024-04-24T23:10:13.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.951+0000 7f7c6aeca640 1 -- 172.21.15.32:0/1338160373 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c640d4210 msgr2=0x7f7c640b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:13.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.951+0000 7f7c6aeca640 1 --2- 172.21.15.32:0/1338160373 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c640d4210 0x7f7c640b78f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f7c54009870 tx=0x7f7c5402a270 comp rx=0 tx=0).stop 2024-04-24T23:10:13.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.951+0000 7f7c6aeca640 1 -- 172.21.15.32:0/1338160373 shutdown_connections 2024-04-24T23:10:13.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.951+0000 7f7c6aeca640 1 --2- 172.21.15.32:0/1338160373 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c640d4210 0x7f7c640b78f0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:13.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.951+0000 7f7c6aeca640 1 --2- 172.21.15.32:0/1338160373 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c6406b820 0x7f7c640d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:13.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.951+0000 7f7c6aeca640 1 -- 172.21.15.32:0/1338160373 >> 172.21.15.32:0/1338160373 conn(0x7f7c640d11d0 msgr2=0x7f7c640d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:13.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.951+0000 7f7c6aeca640 1 -- 172.21.15.32:0/1338160373 shutdown_connections 2024-04-24T23:10:13.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.951+0000 7f7c6aeca640 1 -- 172.21.15.32:0/1338160373 wait complete. 2024-04-24T23:10:13.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.952+0000 7f7c6aeca640 1 Processor -- start 2024-04-24T23:10:13.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.952+0000 7f7c6aeca640 1 -- start start 2024-04-24T23:10:13.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.953+0000 7f7c6aeca640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c6406b820 0x7f7c641f3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:13.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.953+0000 7f7c6aeca640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c640d4210 0x7f7c640ca230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:13.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.953+0000 7f7c6aeca640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c6415e9b0 0x7f7c640ca770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:13.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.953+0000 7f7c6aeca640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7c640ba0e0 con 0x7f7c6406b820 2024-04-24T23:10:13.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.953+0000 7f7c6aeca640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7c640d2400 con 0x7f7c640d4210 2024-04-24T23:10:13.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.953+0000 7f7c6aeca640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f7c640ba260 con 0x7f7c6415e9b0 2024-04-24T23:10:13.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.953+0000 7f7c68c66640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c6406b820 0x7f7c641f3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:13.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.953+0000 7f7c68c66640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c6406b820 0x7f7c641f3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34886/0 (socket says 172.21.15.32:34886) 2024-04-24T23:10:13.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.953+0000 7f7c68c66640 1 -- 172.21.15.32:0/3221340551 learned_addr learned my addr 172.21.15.32:0/3221340551 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:13.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.953+0000 7f7c69467640 1 --2- 172.21.15.32:0/3221340551 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c6415e9b0 0x7f7c640ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:13.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.953+0000 7f7c63fff640 1 --2- 172.21.15.32:0/3221340551 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c640d4210 0x7f7c640ca230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:13.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.953+0000 7f7c68c66640 1 -- 172.21.15.32:0/3221340551 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c6415e9b0 msgr2=0x7f7c640ca770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:13.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.953+0000 7f7c68c66640 1 --2- 172.21.15.32:0/3221340551 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c6415e9b0 0x7f7c640ca770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:13.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.953+0000 7f7c68c66640 1 -- 172.21.15.32:0/3221340551 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c640d4210 msgr2=0x7f7c640ca230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:13.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.953+0000 7f7c68c66640 1 --2- 172.21.15.32:0/3221340551 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c640d4210 0x7f7c640ca230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:13.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.953+0000 7f7c68c66640 1 -- 172.21.15.32:0/3221340551 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c640cadf0 con 0x7f7c6406b820 2024-04-24T23:10:13.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.954+0000 7f7c68c66640 1 --2- 172.21.15.32:0/3221340551 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c6406b820 0x7f7c641f3a60 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c00c730 tx=0x7f7c5c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:13.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.954+0000 7f7c617fa640 1 -- 172.21.15.32:0/3221340551 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c017070 con 0x7f7c6406b820 2024-04-24T23:10:13.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.954+0000 7f7c617fa640 1 -- 172.21.15.32:0/3221340551 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c5c00ce40 con 0x7f7c6406b820 2024-04-24T23:10:13.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.954+0000 7f7c617fa640 1 -- 172.21.15.32:0/3221340551 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c00a860 con 0x7f7c6406b820 2024-04-24T23:10:13.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.954+0000 7f7c6aeca640 1 -- 172.21.15.32:0/3221340551 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c641f4140 con 0x7f7c6406b820 2024-04-24T23:10:13.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.954+0000 7f7c6aeca640 1 -- 172.21.15.32:0/3221340551 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c64155160 con 0x7f7c6406b820 2024-04-24T23:10:13.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.955+0000 7f7c6aeca640 1 -- 172.21.15.32:0/3221340551 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c64068430 con 0x7f7c6406b820 2024-04-24T23:10:13.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.955+0000 7f7c617fa640 1 -- 172.21.15.32:0/3221340551 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7c5c01a070 con 0x7f7c6406b820 2024-04-24T23:10:13.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.956+0000 7f7c617fa640 1 --2- 172.21.15.32:0/3221340551 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7c5006be60 0x7f7c5006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:13.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.956+0000 7f7c617fa640 1 -- 172.21.15.32:0/3221340551 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 5767+0+0 (secure 0 0 0) 0x7f7c5c08a400 con 0x7f7c6406b820 2024-04-24T23:10:13.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.956+0000 7f7c617fa640 1 -- 172.21.15.32:0/3221340551 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=174}) -- 0x7f7c50073a50 con 0x7f7c6406b820 2024-04-24T23:10:13.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.956+0000 7f7c63fff640 1 --2- 172.21.15.32:0/3221340551 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7c5006be60 0x7f7c5006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:13.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.957+0000 7f7c63fff640 1 --2- 172.21.15.32:0/3221340551 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7c5006be60 0x7f7c5006e320 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f7c64153580 tx=0x7f7c58008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:13.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:13.960+0000 7f7c617fa640 1 -- 172.21.15.32:0/3221340551 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7c5c058e00 con 0x7f7c6406b820 2024-04-24T23:10:14.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.141+0000 7f7c6aeca640 1 -- 172.21.15.32:0/3221340551 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7c640b8a60 con 0x7f7c6406b820 2024-04-24T23:10:14.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.141+0000 7f7c617fa640 1 -- 172.21.15.32:0/3221340551 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v173) ==== 93+0+10 (secure 0 0 0) 0x7f7c5c058700 con 0x7f7c6406b820 2024-04-24T23:10:14.143 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T23:10:14.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.143+0000 7f7c6aeca640 1 -- 172.21.15.32:0/3221340551 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7c5006be60 msgr2=0x7f7c5006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:14.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.143+0000 7f7c6aeca640 1 --2- 172.21.15.32:0/3221340551 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7c5006be60 0x7f7c5006e320 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f7c64153580 tx=0x7f7c58008040 comp rx=0 tx=0).stop 2024-04-24T23:10:14.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.143+0000 7f7c6aeca640 1 -- 172.21.15.32:0/3221340551 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c6406b820 msgr2=0x7f7c641f3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:14.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.143+0000 7f7c6aeca640 1 --2- 172.21.15.32:0/3221340551 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c6406b820 0x7f7c641f3a60 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c00c730 tx=0x7f7c5c00cbf0 comp rx=0 tx=0).stop 2024-04-24T23:10:14.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.143+0000 7f7c6aeca640 1 -- 172.21.15.32:0/3221340551 shutdown_connections 2024-04-24T23:10:14.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.144+0000 7f7c6aeca640 1 --2- 172.21.15.32:0/3221340551 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7c5006be60 0x7f7c5006e320 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:14.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.144+0000 7f7c6aeca640 1 --2- 172.21.15.32:0/3221340551 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c6415e9b0 0x7f7c640ca770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:14.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.144+0000 7f7c6aeca640 1 --2- 172.21.15.32:0/3221340551 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c640d4210 0x7f7c640ca230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:14.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.144+0000 7f7c6aeca640 1 --2- 172.21.15.32:0/3221340551 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c6406b820 0x7f7c641f3a60 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:14.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.144+0000 7f7c6aeca640 1 -- 172.21.15.32:0/3221340551 >> 172.21.15.32:0/3221340551 conn(0x7f7c640d11d0 msgr2=0x7f7c6406b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:14.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.144+0000 7f7c6aeca640 1 -- 172.21.15.32:0/3221340551 shutdown_connections 2024-04-24T23:10:14.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.144+0000 7f7c6aeca640 1 -- 172.21.15.32:0/3221340551 wait complete. 2024-04-24T23:10:14.156 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:10:14.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.258+0000 7fb228ea5640 1 Processor -- start 2024-04-24T23:10:14.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.258+0000 7fb228ea5640 1 -- start start 2024-04-24T23:10:14.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.258+0000 7fb228ea5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb22406cd00 0x7fb224166440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:14.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.258+0000 7fb228ea5640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2240cef90 con 0x7fb22415fd60 2024-04-24T23:10:14.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.259+0000 7fb228ea5640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2240b46c0 con 0x7fb22415b480 2024-04-24T23:10:14.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.259+0000 7fb228ea5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fb2240b4ff0 con 0x7fb22406cd00 2024-04-24T23:10:14.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.259+0000 7fb221d9b640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb22415fd60 0x7fb22406c5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47900/0 (socket says 172.21.15.32:47900) 2024-04-24T23:10:14.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.259+0000 7fb221d9b640 1 -- 172.21.15.32:0/2034842547 learned_addr learned my addr 172.21.15.32:0/2034842547 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:10:14.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.259+0000 7fb222d9d640 1 --2- 172.21.15.32:0/2034842547 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb22406cd00 0x7fb224166440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:14.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.259+0000 7fb22159a640 1 -- 172.21.15.32:0/2034842547 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4040980135 0 0) 0x7fb2240cef90 con 0x7fb22415fd60 2024-04-24T23:10:14.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.259+0000 7fb22159a640 1 -- 172.21.15.32:0/2034842547 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb210003520 con 0x7fb22415fd60 2024-04-24T23:10:14.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.259+0000 7fb22159a640 1 -- 172.21.15.32:0/2034842547 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 422730173 0 0) 0x7fb210003520 con 0x7fb22415fd60 2024-04-24T23:10:14.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.259+0000 7fb22159a640 1 -- 172.21.15.32:0/2034842547 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb22406cd00 msgr2=0x7fb224166440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:14.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.259+0000 7fb22159a640 1 --2- 172.21.15.32:0/2034842547 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb22406cd00 0x7fb224166440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:14.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.259+0000 7fb22159a640 1 -- 172.21.15.32:0/2034842547 >> v1:172.21.15.121:6789/0 conn(0x7fb22415b480 legacy=0x7fb2240b5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:14.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.259+0000 7fb22159a640 1 -- 172.21.15.32:0/2034842547 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2240b49a0 con 0x7fb22415fd60 2024-04-24T23:10:14.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.259+0000 7fb22159a640 1 -- 172.21.15.32:0/2034842547 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7fb20c001670 con 0x7fb22415fd60 2024-04-24T23:10:14.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.259+0000 7fb22159a640 1 -- 172.21.15.32:0/2034842547 >> v1:172.21.15.32:6789/0 conn(0x7fb22415fd60 legacy=0x7fb22406c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:14.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.259+0000 7fb22159a640 1 --2- 172.21.15.32:0/2034842547 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2100039e0 0x7fb210003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:14.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.259+0000 7fb22159a640 1 --2- 172.21.15.32:0/2034842547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb210004800 0x7fb210004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:14.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.259+0000 7fb22159a640 1 --2- 172.21.15.32:0/2034842547 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb22406cd00 0x7fb2100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:14.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.259+0000 7fb22159a640 1 -- 172.21.15.32:0/2034842547 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb2240b4ff0 con 0x7fb2100039e0 2024-04-24T23:10:14.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.259+0000 7fb22159a640 1 -- 172.21.15.32:0/2034842547 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb21000baa0 con 0x7fb210004800 2024-04-24T23:10:14.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.259+0000 7fb22159a640 1 -- 172.21.15.32:0/2034842547 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fb21000bca0 con 0x7fb22406cd00 2024-04-24T23:10:14.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.259+0000 7fb22259c640 1 --2- 172.21.15.32:0/2034842547 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2100039e0 0x7fb210003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:14.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.259+0000 7fb222d9d640 1 --2- 172.21.15.32:0/2034842547 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb22406cd00 0x7fb2100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:14.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.259+0000 7fb221d9b640 1 --2- 172.21.15.32:0/2034842547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb210004800 0x7fb210004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:14.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.260+0000 7fb22259c640 1 -- 172.21.15.32:0/2034842547 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb22406cd00 msgr2=0x7fb2100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:14.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.260+0000 7fb22259c640 1 --2- 172.21.15.32:0/2034842547 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb22406cd00 0x7fb2100051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:14.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.260+0000 7fb22259c640 1 -- 172.21.15.32:0/2034842547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb210004800 msgr2=0x7fb210004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:14.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.260+0000 7fb22259c640 1 --2- 172.21.15.32:0/2034842547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb210004800 0x7fb210004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:14.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.260+0000 7fb22259c640 1 -- 172.21.15.32:0/2034842547 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb210003520 con 0x7fb2100039e0 2024-04-24T23:10:14.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.260+0000 7fb222d9d640 1 --2- 172.21.15.32:0/2034842547 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb22406cd00 0x7fb2100051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:10:14.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.260+0000 7fb22259c640 1 --2- 172.21.15.32:0/2034842547 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2100039e0 0x7fb210003e00 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fb21c001720 tx=0x7fb210005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b99175ab13454ff7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:14.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.260+0000 7fb22159a640 1 -- 172.21.15.32:0/2034842547 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb21c001370 con 0x7fb2100039e0 2024-04-24T23:10:14.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.260+0000 7fb22159a640 1 -- 172.21.15.32:0/2034842547 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb21c0052d0 con 0x7fb2100039e0 2024-04-24T23:10:14.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.260+0000 7fb22159a640 1 -- 172.21.15.32:0/2034842547 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb21c02ab90 con 0x7fb2100039e0 2024-04-24T23:10:14.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.261+0000 7fb228ea5640 1 -- 172.21.15.32:0/2034842547 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2100039e0 msgr2=0x7fb210003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:14.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.261+0000 7fb228ea5640 1 --2- 172.21.15.32:0/2034842547 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2100039e0 0x7fb210003e00 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fb21c001720 tx=0x7fb210005830 comp rx=0 tx=0).stop 2024-04-24T23:10:14.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.261+0000 7fb222d9d640 1 -- 172.21.15.32:0/2034842547 reap_dead start 2024-04-24T23:10:14.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.261+0000 7fb228ea5640 1 -- 172.21.15.32:0/2034842547 shutdown_connections 2024-04-24T23:10:14.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.261+0000 7fb228ea5640 1 -- 172.21.15.32:0/2034842547 >> 172.21.15.32:0/2034842547 conn(0x7fb22406b110 msgr2=0x7fb22406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:14.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.261+0000 7fb228ea5640 1 -- 172.21.15.32:0/2034842547 shutdown_connections 2024-04-24T23:10:14.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.261+0000 7fb228ea5640 1 -- 172.21.15.32:0/2034842547 wait complete. 2024-04-24T23:10:14.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.262+0000 7fb228ea5640 1 Processor -- start 2024-04-24T23:10:14.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.263+0000 7fb228ea5640 1 -- start start 2024-04-24T23:10:14.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.263+0000 7fb228ea5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2100039e0 0x7fb2240aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:14.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.263+0000 7fb228ea5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb224144c60 0x7fb2241450e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:14.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.263+0000 7fb228ea5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb22414b590 0x7fb22414ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:14.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.263+0000 7fb228ea5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb22415b860 con 0x7fb22414b590 2024-04-24T23:10:14.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.263+0000 7fb228ea5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb2240b5480 con 0x7fb2100039e0 2024-04-24T23:10:14.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.263+0000 7fb228ea5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fb22414c3b0 con 0x7fb224144c60 2024-04-24T23:10:14.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.263+0000 7fb222d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb22414b590 0x7fb22414ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:14.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.263+0000 7fb222d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb22414b590 0x7fb22414ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34900/0 (socket says 172.21.15.32:34900) 2024-04-24T23:10:14.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.263+0000 7fb222d9d640 1 -- 172.21.15.32:0/1639866907 learned_addr learned my addr 172.21.15.32:0/1639866907 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:14.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.264+0000 7fb22259c640 1 --2- 172.21.15.32:0/1639866907 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2100039e0 0x7fb2240aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:14.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.264+0000 7fb222d9d640 1 -- 172.21.15.32:0/1639866907 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb224144c60 msgr2=0x7fb2241450e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:14.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.264+0000 7fb221d9b640 1 --2- 172.21.15.32:0/1639866907 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb224144c60 0x7fb2241450e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:14.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.264+0000 7fb222d9d640 1 --2- 172.21.15.32:0/1639866907 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb224144c60 0x7fb2241450e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:14.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.264+0000 7fb222d9d640 1 -- 172.21.15.32:0/1639866907 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2100039e0 msgr2=0x7fb2240aeb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:14.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.264+0000 7fb222d9d640 1 --2- 172.21.15.32:0/1639866907 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2100039e0 0x7fb2240aeb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:14.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.264+0000 7fb222d9d640 1 -- 172.21.15.32:0/1639866907 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb22414c600 con 0x7fb22414b590 2024-04-24T23:10:14.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.264+0000 7fb222d9d640 1 --2- 172.21.15.32:0/1639866907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb22414b590 0x7fb22414ba10 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fb2240732c0 tx=0x7fb214009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:14.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.264+0000 7fb21affd640 1 -- 172.21.15.32:0/1639866907 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb214010e50 con 0x7fb22414b590 2024-04-24T23:10:14.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.264+0000 7fb21affd640 1 -- 172.21.15.32:0/1639866907 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb214003990 con 0x7fb22414b590 2024-04-24T23:10:14.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.264+0000 7fb228ea5640 1 -- 172.21.15.32:0/1639866907 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb22414c8e0 con 0x7fb22414b590 2024-04-24T23:10:14.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.264+0000 7fb21affd640 1 -- 172.21.15.32:0/1639866907 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb214003dc0 con 0x7fb22414b590 2024-04-24T23:10:14.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.264+0000 7fb228ea5640 1 -- 172.21.15.32:0/1639866907 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb22414bf50 con 0x7fb22414b590 2024-04-24T23:10:14.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.265+0000 7fb228ea5640 1 -- 172.21.15.32:0/1639866907 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb224068430 con 0x7fb22414b590 2024-04-24T23:10:14.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.266+0000 7fb21affd640 1 -- 172.21.15.32:0/1639866907 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb214005d10 con 0x7fb22414b590 2024-04-24T23:10:14.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.266+0000 7fb21affd640 1 --2- 172.21.15.32:0/1639866907 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb20406be30 0x7fb20406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:14.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.266+0000 7fb21affd640 1 -- 172.21.15.32:0/1639866907 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 5767+0+0 (secure 0 0 0) 0x7fb21408b5f0 con 0x7fb22414b590 2024-04-24T23:10:14.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.266+0000 7fb21affd640 1 -- 172.21.15.32:0/1639866907 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=174}) -- 0x7fb204073a20 con 0x7fb22414b590 2024-04-24T23:10:14.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.266+0000 7fb22259c640 1 --2- 172.21.15.32:0/1639866907 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb20406be30 0x7fb20406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:14.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.267+0000 7fb22259c640 1 --2- 172.21.15.32:0/1639866907 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb20406be30 0x7fb20406e2f0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fb21c032000 tx=0x7fb21c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:14.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.270+0000 7fb21affd640 1 -- 172.21.15.32:0/1639866907 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb214025020 con 0x7fb22414b590 2024-04-24T23:10:14.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.454+0000 7fb228ea5640 1 -- 172.21.15.32:0/1639866907 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb22406c000 con 0x7fb22414b590 2024-04-24T23:10:14.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.455+0000 7fb21affd640 1 -- 172.21.15.32:0/1639866907 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v173) ==== 104+0+11 (secure 0 0 0) 0x7fb214059f60 con 0x7fb22414b590 2024-04-24T23:10:14.456 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-24T23:10:14.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.457+0000 7fb228ea5640 1 -- 172.21.15.32:0/1639866907 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb20406be30 msgr2=0x7fb20406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:14.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.457+0000 7fb228ea5640 1 --2- 172.21.15.32:0/1639866907 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb20406be30 0x7fb20406e2f0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fb21c032000 tx=0x7fb21c02e000 comp rx=0 tx=0).stop 2024-04-24T23:10:14.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.457+0000 7fb228ea5640 1 -- 172.21.15.32:0/1639866907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb22414b590 msgr2=0x7fb22414ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:14.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.457+0000 7fb228ea5640 1 --2- 172.21.15.32:0/1639866907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb22414b590 0x7fb22414ba10 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fb2240732c0 tx=0x7fb214009f90 comp rx=0 tx=0).stop 2024-04-24T23:10:14.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.457+0000 7fb228ea5640 1 -- 172.21.15.32:0/1639866907 shutdown_connections 2024-04-24T23:10:14.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.457+0000 7fb228ea5640 1 --2- 172.21.15.32:0/1639866907 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb20406be30 0x7fb20406e2f0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:14.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.457+0000 7fb228ea5640 1 --2- 172.21.15.32:0/1639866907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb22414b590 0x7fb22414ba10 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:14.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.457+0000 7fb228ea5640 1 --2- 172.21.15.32:0/1639866907 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb224144c60 0x7fb2241450e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:14.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.457+0000 7fb228ea5640 1 --2- 172.21.15.32:0/1639866907 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2100039e0 0x7fb2240aeb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:14.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.457+0000 7fb228ea5640 1 -- 172.21.15.32:0/1639866907 >> 172.21.15.32:0/1639866907 conn(0x7fb22406b110 msgr2=0x7fb2240b11e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:14.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.457+0000 7fb228ea5640 1 -- 172.21.15.32:0/1639866907 shutdown_connections 2024-04-24T23:10:14.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.457+0000 7fb228ea5640 1 -- 172.21.15.32:0/1639866907 wait complete. 2024-04-24T23:10:14.469 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:10:14.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.571+0000 7f7518010640 1 Processor -- start 2024-04-24T23:10:14.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.571+0000 7f7518010640 1 -- start start 2024-04-24T23:10:14.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.572+0000 7f7518010640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f751006c5f0 0x7f751006c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:14.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.572+0000 7f7518010640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75100b6430 con 0x7f75100af890 2024-04-24T23:10:14.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.572+0000 7f7518010640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7510156f80 con 0x7f75100b44d0 2024-04-24T23:10:14.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.572+0000 7f7518010640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f75100b6150 con 0x7f751006c5f0 2024-04-24T23:10:14.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.572+0000 7f7515dac640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f751006c5f0 0x7f751006c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:14.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.572+0000 7f75165ad640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f75100af890 0x7f751015cac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47902/0 (socket says 172.21.15.32:47902) 2024-04-24T23:10:14.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.572+0000 7f75165ad640 1 -- 172.21.15.32:0/3683465976 learned_addr learned my addr 172.21.15.32:0/3683465976 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:10:14.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.572+0000 7f7515dac640 1 -- 172.21.15.32:0/3683465976 >> v1:172.21.15.121:6789/0 conn(0x7f75100b44d0 legacy=0x7f75100b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:14.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.572+0000 7f7515dac640 1 -- 172.21.15.32:0/3683465976 >> v1:172.21.15.32:6789/0 conn(0x7f75100af890 legacy=0x7f751015cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:14.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.572+0000 7f7515dac640 1 -- 172.21.15.32:0/3683465976 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75100b39f0 con 0x7f751006c5f0 2024-04-24T23:10:14.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.573+0000 7f7515dac640 1 --2- 172.21.15.32:0/3683465976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f751006c5f0 0x7f751006c9d0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f750c0097a0 tx=0x7f750c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa0e96379f188e2b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:14.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.573+0000 7f7514daa640 1 -- 172.21.15.32:0/3683465976 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750c037070 con 0x7f751006c5f0 2024-04-24T23:10:14.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.573+0000 7f7514daa640 1 -- 172.21.15.32:0/3683465976 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f750c032440 con 0x7f751006c5f0 2024-04-24T23:10:14.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.573+0000 7f7514daa640 1 -- 172.21.15.32:0/3683465976 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750c032810 con 0x7f751006c5f0 2024-04-24T23:10:14.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.574+0000 7f7518010640 1 -- 172.21.15.32:0/3683465976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f751006c5f0 msgr2=0x7f751006c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:14.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.574+0000 7f7518010640 1 --2- 172.21.15.32:0/3683465976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f751006c5f0 0x7f751006c9d0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f750c0097a0 tx=0x7f750c02a1e0 comp rx=0 tx=0).stop 2024-04-24T23:10:14.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.574+0000 7f7518010640 1 -- 172.21.15.32:0/3683465976 shutdown_connections 2024-04-24T23:10:14.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.574+0000 7f7518010640 1 --2- 172.21.15.32:0/3683465976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f751006c5f0 0x7f751006c9d0 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:14.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.574+0000 7f7518010640 1 -- 172.21.15.32:0/3683465976 >> 172.21.15.32:0/3683465976 conn(0x7f751006b110 msgr2=0x7f751006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:14.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.574+0000 7f7518010640 1 -- 172.21.15.32:0/3683465976 shutdown_connections 2024-04-24T23:10:14.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.574+0000 7f7518010640 1 -- 172.21.15.32:0/3683465976 wait complete. 2024-04-24T23:10:14.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.575+0000 7f7518010640 1 Processor -- start 2024-04-24T23:10:14.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.575+0000 7f7518010640 1 -- start start 2024-04-24T23:10:14.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.576+0000 7f7518010640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f751006c5f0 0x7f7510142d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:14.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.576+0000 7f7518010640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75100af890 0x7f75100aa4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:14.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.576+0000 7f7518010640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f75100b44d0 0x7f75100aa9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:14.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.576+0000 7f7518010640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f751006cf10 con 0x7f75100af890 2024-04-24T23:10:14.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.576+0000 7f7518010640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f75100b3fb0 con 0x7f751006c5f0 2024-04-24T23:10:14.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.576+0000 7f7518010640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f751015d1d0 con 0x7f75100b44d0 2024-04-24T23:10:14.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.576+0000 7f75155ab640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75100af890 0x7f75100aa4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:14.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.576+0000 7f75155ab640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75100af890 0x7f75100aa4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34910/0 (socket says 172.21.15.32:34910) 2024-04-24T23:10:14.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.576+0000 7f75155ab640 1 -- 172.21.15.32:0/1122400933 learned_addr learned my addr 172.21.15.32:0/1122400933 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:14.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.576+0000 7f7515dac640 1 --2- 172.21.15.32:0/1122400933 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f751006c5f0 0x7f7510142d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:14.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.576+0000 7f75165ad640 1 --2- 172.21.15.32:0/1122400933 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f75100b44d0 0x7f75100aa9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:14.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.576+0000 7f75155ab640 1 -- 172.21.15.32:0/1122400933 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f75100b44d0 msgr2=0x7f75100aa9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:14.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.576+0000 7f75155ab640 1 --2- 172.21.15.32:0/1122400933 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f75100b44d0 0x7f75100aa9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:14.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.576+0000 7f75155ab640 1 -- 172.21.15.32:0/1122400933 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f751006c5f0 msgr2=0x7f7510142d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:14.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.576+0000 7f75155ab640 1 --2- 172.21.15.32:0/1122400933 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f751006c5f0 0x7f7510142d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:14.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.576+0000 7f75155ab640 1 -- 172.21.15.32:0/1122400933 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f751014f420 con 0x7f75100af890 2024-04-24T23:10:14.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.576+0000 7f75155ab640 1 --2- 172.21.15.32:0/1122400933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75100af890 0x7f75100aa4a0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f7500009fd0 tx=0x7f750000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:14.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.576+0000 7f75067fc640 1 -- 172.21.15.32:0/1122400933 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7500015020 con 0x7f75100af890 2024-04-24T23:10:14.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.577+0000 7f75067fc640 1 -- 172.21.15.32:0/1122400933 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7500011830 con 0x7f75100af890 2024-04-24T23:10:14.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.577+0000 7f7518010640 1 -- 172.21.15.32:0/1122400933 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f751014f700 con 0x7f75100af890 2024-04-24T23:10:14.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.577+0000 7f75067fc640 1 -- 172.21.15.32:0/1122400933 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7500011c00 con 0x7f75100af890 2024-04-24T23:10:14.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.577+0000 7f7518010640 1 -- 172.21.15.32:0/1122400933 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f751014fcc0 con 0x7f75100af890 2024-04-24T23:10:14.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.577+0000 7f7518010640 1 -- 172.21.15.32:0/1122400933 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7510068430 con 0x7f75100af890 2024-04-24T23:10:14.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.578+0000 7f75067fc640 1 -- 172.21.15.32:0/1122400933 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7500018020 con 0x7f75100af890 2024-04-24T23:10:14.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.578+0000 7f75067fc640 1 --2- 172.21.15.32:0/1122400933 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f74f406be30 0x7f74f406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:14.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.578+0000 7f75067fc640 1 -- 172.21.15.32:0/1122400933 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 5767+0+0 (secure 0 0 0) 0x7f750008ac00 con 0x7f75100af890 2024-04-24T23:10:14.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.578+0000 7f75067fc640 1 -- 172.21.15.32:0/1122400933 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=174}) -- 0x7f74f4073a20 con 0x7f75100af890 2024-04-24T23:10:14.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.579+0000 7f7515dac640 1 --2- 172.21.15.32:0/1122400933 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f74f406be30 0x7f74f406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:14.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.579+0000 7f7515dac640 1 --2- 172.21.15.32:0/1122400933 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f74f406be30 0x7f74f406e2f0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f750c034000 tx=0x7f750c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:14.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.583+0000 7f75067fc640 1 -- 172.21.15.32:0/1122400933 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7500091080 con 0x7f75100af890 2024-04-24T23:10:14.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.768+0000 7f7518010640 1 -- 172.21.15.32:0/1122400933 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f75100b5670 con 0x7f75100af890 2024-04-24T23:10:14.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.768+0000 7f75067fc640 1 -- 172.21.15.32:0/1122400933 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v173) ==== 100+0+11 (secure 0 0 0) 0x7f750001f020 con 0x7f75100af890 2024-04-24T23:10:14.770 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-24T23:10:14.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.770+0000 7f7518010640 1 -- 172.21.15.32:0/1122400933 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f74f406be30 msgr2=0x7f74f406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:14.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.770+0000 7f7518010640 1 --2- 172.21.15.32:0/1122400933 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f74f406be30 0x7f74f406e2f0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f750c034000 tx=0x7f750c02e000 comp rx=0 tx=0).stop 2024-04-24T23:10:14.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.770+0000 7f7518010640 1 -- 172.21.15.32:0/1122400933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75100af890 msgr2=0x7f75100aa4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:14.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.770+0000 7f7518010640 1 --2- 172.21.15.32:0/1122400933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75100af890 0x7f75100aa4a0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f7500009fd0 tx=0x7f750000b870 comp rx=0 tx=0).stop 2024-04-24T23:10:14.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.770+0000 7f7518010640 1 -- 172.21.15.32:0/1122400933 shutdown_connections 2024-04-24T23:10:14.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.770+0000 7f7518010640 1 --2- 172.21.15.32:0/1122400933 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f74f406be30 0x7f74f406e2f0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:14.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.770+0000 7f7518010640 1 --2- 172.21.15.32:0/1122400933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75100af890 0x7f75100aa4a0 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:14.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.770+0000 7f7518010640 1 --2- 172.21.15.32:0/1122400933 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f75100b44d0 0x7f75100aa9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:14.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.770+0000 7f7518010640 1 --2- 172.21.15.32:0/1122400933 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f751006c5f0 0x7f7510142d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:14.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.770+0000 7f7518010640 1 -- 172.21.15.32:0/1122400933 >> 172.21.15.32:0/1122400933 conn(0x7f751006b110 msgr2=0x7f7510071960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:14.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.770+0000 7f7518010640 1 -- 172.21.15.32:0/1122400933 shutdown_connections 2024-04-24T23:10:14.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.770+0000 7f7518010640 1 -- 172.21.15.32:0/1122400933 wait complete. 2024-04-24T23:10:14.783 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T23:10:14.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.882+0000 7fb05c6fc640 1 Processor -- start 2024-04-24T23:10:14.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.882+0000 7fb05c6fc640 1 -- start start 2024-04-24T23:10:14.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.883+0000 7fb05c6fc640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb054162320 0x7fb0540ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:14.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.883+0000 7fb05c6fc640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb054162e10 0x7fb05415ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:14.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.883+0000 7fb05c6fc640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0540cd270 con 0x7fb0540b5af0 2024-04-24T23:10:14.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.883+0000 7fb05c6fc640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fb0540b5660 con 0x7fb054162320 2024-04-24T23:10:14.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.883+0000 7fb05c6fc640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fb054071610 con 0x7fb054162e10 2024-04-24T23:10:14.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.883+0000 7fb059c97640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb0540b5af0 0x7fb054162700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47914/0 (socket says 172.21.15.32:47914) 2024-04-24T23:10:14.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.883+0000 7fb059c97640 1 -- 172.21.15.32:0/3385415374 learned_addr learned my addr 172.21.15.32:0/3385415374 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:10:14.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.883+0000 7fb05a498640 1 --2- 172.21.15.32:0/3385415374 >> v2:172.21.15.121:3300/0 conn(0x7fb054162320 0x7fb0540ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:14.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.883+0000 7fb05ac99640 1 --2- 172.21.15.32:0/3385415374 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb054162e10 0x7fb05415ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:14.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.883+0000 7fb059496640 1 -- 172.21.15.32:0/3385415374 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 164361938 0 0) 0x7fb0540cd270 con 0x7fb0540b5af0 2024-04-24T23:10:14.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.883+0000 7fb059496640 1 -- 172.21.15.32:0/3385415374 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb03c003520 con 0x7fb0540b5af0 2024-04-24T23:10:14.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.884+0000 7fb059496640 1 -- 172.21.15.32:0/3385415374 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1679648889 0 0) 0x7fb03c003520 con 0x7fb0540b5af0 2024-04-24T23:10:14.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.884+0000 7fb05ac99640 1 -- 172.21.15.32:0/3385415374 >> v2:172.21.15.121:3300/0 conn(0x7fb054162320 msgr2=0x7fb0540ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:14.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.884+0000 7fb05ac99640 1 --2- 172.21.15.32:0/3385415374 >> v2:172.21.15.121:3300/0 conn(0x7fb054162320 0x7fb0540ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:14.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.884+0000 7fb05ac99640 1 -- 172.21.15.32:0/3385415374 >> v1:172.21.15.32:6789/0 conn(0x7fb0540b5af0 legacy=0x7fb054162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:14.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.884+0000 7fb05ac99640 1 -- 172.21.15.32:0/3385415374 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0540718f0 con 0x7fb054162e10 2024-04-24T23:10:14.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.884+0000 7fb05a498640 1 --2- 172.21.15.32:0/3385415374 >> v2:172.21.15.121:3300/0 conn(0x7fb054162320 0x7fb0540ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:10:14.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.885+0000 7fb05ac99640 1 --2- 172.21.15.32:0/3385415374 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb054162e10 0x7fb05415ed30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb044009d90 tx=0x7fb04402a4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57619723328e3afe server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:14.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.885+0000 7fb059496640 1 -- 172.21.15.32:0/3385415374 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb048001670 con 0x7fb054162e10 2024-04-24T23:10:14.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.885+0000 7fb059496640 1 -- 172.21.15.32:0/3385415374 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb044007540 con 0x7fb054162e10 2024-04-24T23:10:14.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.885+0000 7fb059496640 1 -- 172.21.15.32:0/3385415374 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0440078f0 con 0x7fb054162e10 2024-04-24T23:10:14.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.885+0000 7fb05c6fc640 1 -- 172.21.15.32:0/3385415374 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb054162e10 msgr2=0x7fb05415ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:14.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.885+0000 7fb05c6fc640 1 --2- 172.21.15.32:0/3385415374 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb054162e10 0x7fb05415ed30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb044009d90 tx=0x7fb04402a4d0 comp rx=0 tx=0).stop 2024-04-24T23:10:14.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.885+0000 7fb05c6fc640 1 -- 172.21.15.32:0/3385415374 shutdown_connections 2024-04-24T23:10:14.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.885+0000 7fb05c6fc640 1 --2- 172.21.15.32:0/3385415374 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb054162e10 0x7fb05415ed30 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:14.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.885+0000 7fb05c6fc640 1 --2- 172.21.15.32:0/3385415374 >> v2:172.21.15.121:3300/0 conn(0x7fb054162320 0x7fb0540ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:14.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.885+0000 7fb05c6fc640 1 -- 172.21.15.32:0/3385415374 >> 172.21.15.32:0/3385415374 conn(0x7fb0540d0ae0 msgr2=0x7fb0540d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:14.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.885+0000 7fb05c6fc640 1 -- 172.21.15.32:0/3385415374 shutdown_connections 2024-04-24T23:10:14.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.885+0000 7fb05c6fc640 1 -- 172.21.15.32:0/3385415374 wait complete. 2024-04-24T23:10:14.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.887+0000 7fb05c6fc640 1 Processor -- start 2024-04-24T23:10:14.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.887+0000 7fb05c6fc640 1 -- start start 2024-04-24T23:10:14.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.887+0000 7fb05c6fc640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb0540b5af0 0x7fb054142590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:14.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.887+0000 7fb05c6fc640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb054162320 0x7fb054157820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:14.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.887+0000 7fb05c6fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb054162e10 0x7fb054157d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:14.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.887+0000 7fb05c6fc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb054071bd0 con 0x7fb054162e10 2024-04-24T23:10:14.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.887+0000 7fb05c6fc640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb054067950 con 0x7fb054162320 2024-04-24T23:10:14.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.887+0000 7fb05c6fc640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fb054071d50 con 0x7fb0540b5af0 2024-04-24T23:10:14.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.887+0000 7fb05ac99640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb054162e10 0x7fb054157d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:14.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.887+0000 7fb05ac99640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb054162e10 0x7fb054157d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34924/0 (socket says 172.21.15.32:34924) 2024-04-24T23:10:14.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.887+0000 7fb05ac99640 1 -- 172.21.15.32:0/2889662069 learned_addr learned my addr 172.21.15.32:0/2889662069 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:14.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.888+0000 7fb05a498640 1 --2- 172.21.15.32:0/2889662069 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb0540b5af0 0x7fb054142590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:14.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.888+0000 7fb059c97640 1 --2- 172.21.15.32:0/2889662069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb054162320 0x7fb054157820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:14.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.888+0000 7fb05ac99640 1 -- 172.21.15.32:0/2889662069 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb0540b5af0 msgr2=0x7fb054142590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:14.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.888+0000 7fb05ac99640 1 --2- 172.21.15.32:0/2889662069 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb0540b5af0 0x7fb054142590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:14.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.888+0000 7fb05ac99640 1 -- 172.21.15.32:0/2889662069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb054162320 msgr2=0x7fb054157820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:14.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.888+0000 7fb05ac99640 1 --2- 172.21.15.32:0/2889662069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb054162320 0x7fb054157820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:14.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.888+0000 7fb05ac99640 1 -- 172.21.15.32:0/2889662069 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0541e81f0 con 0x7fb054162e10 2024-04-24T23:10:14.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.888+0000 7fb05a498640 1 --2- 172.21.15.32:0/2889662069 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb0540b5af0 0x7fb054142590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:10:14.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.888+0000 7fb059c97640 1 --2- 172.21.15.32:0/2889662069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb054162320 0x7fb054157820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:10:14.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.888+0000 7fb05ac99640 1 --2- 172.21.15.32:0/2889662069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb054162e10 0x7fb054157d60 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fb044009eb0 tx=0x7fb044004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:14.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.888+0000 7fb052ffd640 1 -- 172.21.15.32:0/2889662069 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb044040070 con 0x7fb054162e10 2024-04-24T23:10:14.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.888+0000 7fb052ffd640 1 -- 172.21.15.32:0/2889662069 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb044031c40 con 0x7fb054162e10 2024-04-24T23:10:14.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.888+0000 7fb05c6fc640 1 -- 172.21.15.32:0/2889662069 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0540ced30 con 0x7fb054162e10 2024-04-24T23:10:14.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.888+0000 7fb052ffd640 1 -- 172.21.15.32:0/2889662069 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb044009450 con 0x7fb054162e10 2024-04-24T23:10:14.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.888+0000 7fb05c6fc640 1 -- 172.21.15.32:0/2889662069 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0541e8790 con 0x7fb054162e10 2024-04-24T23:10:14.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.889+0000 7fb05c6fc640 1 -- 172.21.15.32:0/2889662069 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb054068430 con 0x7fb054162e10 2024-04-24T23:10:14.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.889+0000 7fb052ffd640 1 -- 172.21.15.32:0/2889662069 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb044039040 con 0x7fb054162e10 2024-04-24T23:10:14.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.890+0000 7fb052ffd640 1 --2- 172.21.15.32:0/2889662069 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb04006be30 0x7fb04006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:14.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.890+0000 7fb052ffd640 1 -- 172.21.15.32:0/2889662069 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 5767+0+0 (secure 0 0 0) 0x7fb0440ac2d0 con 0x7fb054162e10 2024-04-24T23:10:14.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.890+0000 7fb052ffd640 1 -- 172.21.15.32:0/2889662069 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=174}) -- 0x7fb040073a20 con 0x7fb054162e10 2024-04-24T23:10:14.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.890+0000 7fb05a498640 1 --2- 172.21.15.32:0/2889662069 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb04006be30 0x7fb04006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:14.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.891+0000 7fb05a498640 1 --2- 172.21.15.32:0/2889662069 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb04006be30 0x7fb04006e2f0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fb04c009ac0 tx=0x7fb04c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:14.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:14.894+0000 7fb052ffd640 1 -- 172.21.15.32:0/2889662069 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb04407acd0 con 0x7fb054162e10 2024-04-24T23:10:15.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.091+0000 7fb05c6fc640 1 -- 172.21.15.32:0/2889662069 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb054067730 con 0x7fb054162e10 2024-04-24T23:10:15.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.091+0000 7fb052ffd640 1 -- 172.21.15.32:0/2889662069 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v173) ==== 80+0+0 (secure 0 0 0) 0x7fb04407a5d0 con 0x7fb054162e10 2024-04-24T23:10:15.093 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2024-04-24T23:10:15.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.093+0000 7fb05c6fc640 1 -- 172.21.15.32:0/2889662069 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb04006be30 msgr2=0x7fb04006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:15.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.093+0000 7fb05c6fc640 1 --2- 172.21.15.32:0/2889662069 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb04006be30 0x7fb04006e2f0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fb04c009ac0 tx=0x7fb04c008040 comp rx=0 tx=0).stop 2024-04-24T23:10:15.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.093+0000 7fb05c6fc640 1 -- 172.21.15.32:0/2889662069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb054162e10 msgr2=0x7fb054157d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:15.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.093+0000 7fb05c6fc640 1 --2- 172.21.15.32:0/2889662069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb054162e10 0x7fb054157d60 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fb044009eb0 tx=0x7fb044004a40 comp rx=0 tx=0).stop 2024-04-24T23:10:15.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.093+0000 7fb05c6fc640 1 -- 172.21.15.32:0/2889662069 shutdown_connections 2024-04-24T23:10:15.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.093+0000 7fb05c6fc640 1 --2- 172.21.15.32:0/2889662069 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb04006be30 0x7fb04006e2f0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:15.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.093+0000 7fb05c6fc640 1 --2- 172.21.15.32:0/2889662069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb054162e10 0x7fb054157d60 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:15.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.093+0000 7fb05c6fc640 1 --2- 172.21.15.32:0/2889662069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb054162320 0x7fb054157820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:15.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.093+0000 7fb05c6fc640 1 --2- 172.21.15.32:0/2889662069 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb0540b5af0 0x7fb054142590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:15.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.093+0000 7fb05c6fc640 1 -- 172.21.15.32:0/2889662069 >> 172.21.15.32:0/2889662069 conn(0x7fb0540d0ae0 msgr2=0x7fb05415d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:15.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.093+0000 7fb05c6fc640 1 -- 172.21.15.32:0/2889662069 shutdown_connections 2024-04-24T23:10:15.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.093+0000 7fb05c6fc640 1 -- 172.21.15.32:0/2889662069 wait complete. 2024-04-24T23:10:15.106 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-24T23:10:15.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:10:15.106 DEBUG:teuthology.orchestra.run.smithi121:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-24T23:10:15.138 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2024-04-24T23:10:15.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:10:15.138 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-24T23:10:15.195 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:10:15.195 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:10:15.195 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:10:15.211 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=413 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T23:10:15.211 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:10:15.211 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:10:15.211 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:10:15.252 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3271 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T23:10:15.253 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:10:15.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.351+0000 7fd05f95a640 1 Processor -- start 2024-04-24T23:10:15.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.352+0000 7fd05f95a640 1 -- start start 2024-04-24T23:10:15.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.352+0000 7fd05f95a640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd05815e9b0 0x7fd0580d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:15.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.352+0000 7fd05f95a640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0580d3170 con 0x7fd0580d4210 2024-04-24T23:10:15.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.352+0000 7fd05f95a640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0580d3450 con 0x7fd05806b820 2024-04-24T23:10:15.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.352+0000 7fd05f95a640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fd0580d3730 con 0x7fd05815e9b0 2024-04-24T23:10:15.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.352+0000 7fd05cef5640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd0580d4210 0x7fd05806b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47922/0 (socket says 172.21.15.32:47922) 2024-04-24T23:10:15.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.353+0000 7fd05cef5640 1 -- 172.21.15.32:0/3801583571 learned_addr learned my addr 172.21.15.32:0/3801583571 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:10:15.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.353+0000 7fd05d6f6640 1 --2- 172.21.15.32:0/3801583571 >> v2:172.21.15.121:3300/0 conn(0x7fd05815e9b0 0x7fd0580d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:15.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.353+0000 7fd057fff640 1 -- 172.21.15.32:0/3801583571 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1931617949 0 0) 0x7fd0580d3170 con 0x7fd0580d4210 2024-04-24T23:10:15.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.353+0000 7fd057fff640 1 -- 172.21.15.32:0/3801583571 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd040003520 con 0x7fd0580d4210 2024-04-24T23:10:15.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.353+0000 7fd057fff640 1 -- 172.21.15.32:0/3801583571 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 964354660 0 0) 0x7fd040003520 con 0x7fd0580d4210 2024-04-24T23:10:15.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.353+0000 7fd057fff640 1 -- 172.21.15.32:0/3801583571 >> v2:172.21.15.121:3300/0 conn(0x7fd05815e9b0 msgr2=0x7fd0580d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:15.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.353+0000 7fd057fff640 1 --2- 172.21.15.32:0/3801583571 >> v2:172.21.15.121:3300/0 conn(0x7fd05815e9b0 0x7fd0580d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:15.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.353+0000 7fd057fff640 1 -- 172.21.15.32:0/3801583571 >> v1:172.21.15.121:6789/0 conn(0x7fd05806b820 legacy=0x7fd0580b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:15.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.353+0000 7fd057fff640 1 -- 172.21.15.32:0/3801583571 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0580d3a10 con 0x7fd0580d4210 2024-04-24T23:10:15.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.353+0000 7fd057fff640 1 -- 172.21.15.32:0/3801583571 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7fd04c001670 con 0x7fd0580d4210 2024-04-24T23:10:15.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.353+0000 7fd057fff640 1 -- 172.21.15.32:0/3801583571 >> v1:172.21.15.32:6789/0 conn(0x7fd0580d4210 legacy=0x7fd05806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:15.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.353+0000 7fd057fff640 1 --2- 172.21.15.32:0/3801583571 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0400039b0 0x7fd040003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:15.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.353+0000 7fd057fff640 1 --2- 172.21.15.32:0/3801583571 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd0400043b0 0x7fd040004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:15.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.353+0000 7fd057fff640 1 --2- 172.21.15.32:0/3801583571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd040005120 0x7fd04000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:15.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.353+0000 7fd057fff640 1 -- 172.21.15.32:0/3801583571 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd0580d3730 con 0x7fd040005120 2024-04-24T23:10:15.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.353+0000 7fd057fff640 1 -- 172.21.15.32:0/3801583571 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd04000c060 con 0x7fd0400039b0 2024-04-24T23:10:15.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.353+0000 7fd057fff640 1 -- 172.21.15.32:0/3801583571 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd04000c260 con 0x7fd0400043b0 2024-04-24T23:10:15.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.353+0000 7fd05def7640 1 --2- 172.21.15.32:0/3801583571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd040005120 0x7fd04000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:15.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.353+0000 7fd05d6f6640 1 --2- 172.21.15.32:0/3801583571 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0400039b0 0x7fd040003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:15.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.353+0000 7fd05cef5640 1 --2- 172.21.15.32:0/3801583571 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd0400043b0 0x7fd040004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:15.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.354+0000 7fd05d6f6640 1 -- 172.21.15.32:0/3801583571 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd0400043b0 msgr2=0x7fd040004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:15.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.354+0000 7fd05d6f6640 1 --2- 172.21.15.32:0/3801583571 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd0400043b0 0x7fd040004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:15.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.354+0000 7fd05d6f6640 1 -- 172.21.15.32:0/3801583571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd040005120 msgr2=0x7fd04000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:15.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.354+0000 7fd05d6f6640 1 --2- 172.21.15.32:0/3801583571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd040005120 0x7fd04000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:15.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.354+0000 7fd05d6f6640 1 -- 172.21.15.32:0/3801583571 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd040003520 con 0x7fd0400039b0 2024-04-24T23:10:15.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.354+0000 7fd05def7640 1 --2- 172.21.15.32:0/3801583571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd040005120 0x7fd04000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:10:15.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.354+0000 7fd05def7640 1 -- 172.21.15.32:0/3801583571 reap_dead start 2024-04-24T23:10:15.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.354+0000 7fd05d6f6640 1 --2- 172.21.15.32:0/3801583571 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0400039b0 0x7fd040003d90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fd050009ac0 tx=0x7fd040005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f4f51e403d6afdb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:15.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.354+0000 7fd057fff640 1 -- 172.21.15.32:0/3801583571 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd050036070 con 0x7fd0400039b0 2024-04-24T23:10:15.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.354+0000 7fd057fff640 1 -- 172.21.15.32:0/3801583571 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd050031400 con 0x7fd0400039b0 2024-04-24T23:10:15.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.355+0000 7fd057fff640 1 -- 172.21.15.32:0/3801583571 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd050009690 con 0x7fd0400039b0 2024-04-24T23:10:15.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.355+0000 7fd05f95a640 1 -- 172.21.15.32:0/3801583571 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0400039b0 msgr2=0x7fd040003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:15.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.355+0000 7fd05f95a640 1 --2- 172.21.15.32:0/3801583571 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0400039b0 0x7fd040003d90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fd050009ac0 tx=0x7fd040005830 comp rx=0 tx=0).stop 2024-04-24T23:10:15.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.355+0000 7fd05f95a640 1 -- 172.21.15.32:0/3801583571 shutdown_connections 2024-04-24T23:10:15.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.355+0000 7fd05f95a640 1 --2- 172.21.15.32:0/3801583571 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0400039b0 0x7fd040003d90 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:15.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.355+0000 7fd05f95a640 1 -- 172.21.15.32:0/3801583571 >> 172.21.15.32:0/3801583571 conn(0x7fd0580d11d0 msgr2=0x7fd0580d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:15.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.355+0000 7fd05f95a640 1 -- 172.21.15.32:0/3801583571 shutdown_connections 2024-04-24T23:10:15.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.355+0000 7fd05f95a640 1 -- 172.21.15.32:0/3801583571 wait complete. 2024-04-24T23:10:15.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.356+0000 7fd05f95a640 1 Processor -- start 2024-04-24T23:10:15.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.356+0000 7fd05f95a640 1 -- start start 2024-04-24T23:10:15.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.357+0000 7fd05f95a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd0400039b0 0x7fd058160710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:15.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.357+0000 7fd05f95a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd058159cc0 0x7fd058150a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:15.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.357+0000 7fd05f95a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd05815ed90 0x7fd0581613b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:15.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.357+0000 7fd05f95a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd0580ba0e0 con 0x7fd058159cc0 2024-04-24T23:10:15.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.357+0000 7fd05f95a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd0580d2400 con 0x7fd05815ed90 2024-04-24T23:10:15.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.357+0000 7fd05f95a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd0580ba260 con 0x7fd0400039b0 2024-04-24T23:10:15.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.357+0000 7fd05cef5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd058159cc0 0x7fd058150a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:15.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.357+0000 7fd05cef5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd058159cc0 0x7fd058150a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34944/0 (socket says 172.21.15.32:34944) 2024-04-24T23:10:15.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.357+0000 7fd05cef5640 1 -- 172.21.15.32:0/1990963923 learned_addr learned my addr 172.21.15.32:0/1990963923 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:15.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.357+0000 7fd05d6f6640 1 --2- 172.21.15.32:0/1990963923 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd0400039b0 0x7fd058160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:15.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.357+0000 7fd05def7640 1 --2- 172.21.15.32:0/1990963923 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd05815ed90 0x7fd0581613b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:15.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.357+0000 7fd05cef5640 1 -- 172.21.15.32:0/1990963923 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd0400039b0 msgr2=0x7fd058160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:15.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.357+0000 7fd05cef5640 1 --2- 172.21.15.32:0/1990963923 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd0400039b0 0x7fd058160710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:15.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.357+0000 7fd05cef5640 1 -- 172.21.15.32:0/1990963923 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd05815ed90 msgr2=0x7fd0581613b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:15.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.357+0000 7fd05cef5640 1 --2- 172.21.15.32:0/1990963923 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd05815ed90 0x7fd0581613b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:15.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.357+0000 7fd05cef5640 1 -- 172.21.15.32:0/1990963923 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd058150fc0 con 0x7fd058159cc0 2024-04-24T23:10:15.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.358+0000 7fd05cef5640 1 --2- 172.21.15.32:0/1990963923 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd058159cc0 0x7fd058150a80 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fd04c004fa0 tx=0x7fd04c0074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:15.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.358+0000 7fd055ffb640 1 -- 172.21.15.32:0/1990963923 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04c01a070 con 0x7fd058159cc0 2024-04-24T23:10:15.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.358+0000 7fd055ffb640 1 -- 172.21.15.32:0/1990963923 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd04c00f720 con 0x7fd058159cc0 2024-04-24T23:10:15.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.358+0000 7fd055ffb640 1 -- 172.21.15.32:0/1990963923 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04c00fb10 con 0x7fd058159cc0 2024-04-24T23:10:15.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.358+0000 7fd05f95a640 1 -- 172.21.15.32:0/1990963923 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd05815f670 con 0x7fd058159cc0 2024-04-24T23:10:15.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.358+0000 7fd05f95a640 1 -- 172.21.15.32:0/1990963923 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd05815fc30 con 0x7fd058159cc0 2024-04-24T23:10:15.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.359+0000 7fd05f95a640 1 -- 172.21.15.32:0/1990963923 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd058068430 con 0x7fd058159cc0 2024-04-24T23:10:15.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.359+0000 7fd055ffb640 1 -- 172.21.15.32:0/1990963923 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd04c00fcb0 con 0x7fd058159cc0 2024-04-24T23:10:15.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.360+0000 7fd055ffb640 1 --2- 172.21.15.32:0/1990963923 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd04406be60 0x7fd04406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:15.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.360+0000 7fd055ffb640 1 -- 172.21.15.32:0/1990963923 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 5767+0+0 (secure 0 0 0) 0x7fd04c08a420 con 0x7fd058159cc0 2024-04-24T23:10:15.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.360+0000 7fd055ffb640 1 -- 172.21.15.32:0/1990963923 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=174}) -- 0x7fd044073a50 con 0x7fd058159cc0 2024-04-24T23:10:15.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.360+0000 7fd05d6f6640 1 --2- 172.21.15.32:0/1990963923 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd04406be60 0x7fd04406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:15.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.360+0000 7fd05d6f6640 1 --2- 172.21.15.32:0/1990963923 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd04406be60 0x7fd04406e320 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fd050033000 tx=0x7fd050008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:15.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.364+0000 7fd055ffb640 1 -- 172.21.15.32:0/1990963923 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd04c058e50 con 0x7fd058159cc0 2024-04-24T23:10:15.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.589+0000 7fd05f95a640 1 -- 172.21.15.32:0/1990963923 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd0580cb0b0 con 0x7fd058159cc0 2024-04-24T23:10:15.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.589+0000 7fd055ffb640 1 -- 172.21.15.32:0/1990963923 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+5185 (secure 0 0 0) 0x7fd04c058750 con 0x7fd058159cc0 2024-04-24T23:10:15.591 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:10:15.591 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":73,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6695,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/91086102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":91086102},{"type":"v1","addr":"172.21.15.121:6836","nonce":91086102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":6698,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1443743127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1443743127},{"type":"v1","addr":"172.21.15.121:6837","nonce":1443743127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":7423,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/4008773444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":4008773444},{"type":"v1","addr":"172.21.15.32:6835","nonce":4008773444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:09:00.770308+0000","modified":"2024-04-24T23:09:03.774877+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8679},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8679":{"gid":8679,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6834/4273183179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":4273183179},{"type":"v1","addr":"172.21.15.32:6834","nonce":4273183179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8679,"qdb_cluster":[8679]},"id":10}]} 2024-04-24T23:10:15.591 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 73 2024-04-24T23:10:15.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.591+0000 7fd05f95a640 1 -- 172.21.15.32:0/1990963923 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd04406be60 msgr2=0x7fd04406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:15.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.591+0000 7fd05f95a640 1 --2- 172.21.15.32:0/1990963923 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd04406be60 0x7fd04406e320 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fd050033000 tx=0x7fd050008030 comp rx=0 tx=0).stop 2024-04-24T23:10:15.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.591+0000 7fd05f95a640 1 -- 172.21.15.32:0/1990963923 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd058159cc0 msgr2=0x7fd058150a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:15.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.591+0000 7fd05f95a640 1 --2- 172.21.15.32:0/1990963923 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd058159cc0 0x7fd058150a80 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fd04c004fa0 tx=0x7fd04c0074f0 comp rx=0 tx=0).stop 2024-04-24T23:10:15.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.591+0000 7fd05f95a640 1 -- 172.21.15.32:0/1990963923 shutdown_connections 2024-04-24T23:10:15.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.591+0000 7fd05f95a640 1 --2- 172.21.15.32:0/1990963923 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd04406be60 0x7fd04406e320 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:15.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.591+0000 7fd05f95a640 1 --2- 172.21.15.32:0/1990963923 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd058159cc0 0x7fd058150a80 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:15.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.591+0000 7fd05f95a640 1 --2- 172.21.15.32:0/1990963923 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd05815ed90 0x7fd0581613b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:15.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.591+0000 7fd05f95a640 1 --2- 172.21.15.32:0/1990963923 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd0400039b0 0x7fd058160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:15.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.591+0000 7fd05f95a640 1 -- 172.21.15.32:0/1990963923 >> 172.21.15.32:0/1990963923 conn(0x7fd0580d11d0 msgr2=0x7fd05806b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:15.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.591+0000 7fd05f95a640 1 -- 172.21.15.32:0/1990963923 shutdown_connections 2024-04-24T23:10:15.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.591+0000 7fd05f95a640 1 -- 172.21.15.32:0/1990963923 wait complete. 2024-04-24T23:10:15.604 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:10:15.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.703+0000 7f169bef5640 1 Processor -- start 2024-04-24T23:10:15.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.703+0000 7f169bef5640 1 -- start start 2024-04-24T23:10:15.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.704+0000 7f169bef5640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f169415b480 0x7f16940b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:15.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.704+0000 7f169bef5640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16940b46c0 con 0x7f169415fd60 2024-04-24T23:10:15.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.704+0000 7f169bef5640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16940b49a0 con 0x7f169406cd00 2024-04-24T23:10:15.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.704+0000 7f169bef5640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f16940b4ff0 con 0x7f169415b480 2024-04-24T23:10:15.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.704+0000 7f1699490640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f169415fd60 0x7f169406c5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47928/0 (socket says 172.21.15.32:47928) 2024-04-24T23:10:15.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.704+0000 7f1699490640 1 -- 172.21.15.32:0/2672969172 learned_addr learned my addr 172.21.15.32:0/2672969172 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:10:15.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.704+0000 7f1699c91640 1 --2- 172.21.15.32:0/2672969172 >> v2:172.21.15.121:3300/0 conn(0x7f169415b480 0x7f16940b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:15.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.704+0000 7f1698c8f640 1 -- 172.21.15.32:0/2672969172 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2829833532 0 0) 0x7f16940b46c0 con 0x7f169415fd60 2024-04-24T23:10:15.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.704+0000 7f1698c8f640 1 -- 172.21.15.32:0/2672969172 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1680003520 con 0x7f169415fd60 2024-04-24T23:10:15.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.704+0000 7f1699c91640 1 -- 172.21.15.32:0/2672969172 >> v1:172.21.15.121:6789/0 conn(0x7f169406cd00 legacy=0x7f16940cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:15.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.704+0000 7f1698c8f640 1 -- 172.21.15.32:0/2672969172 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1569353765 0 0) 0x7f16940b49a0 con 0x7f169406cd00 2024-04-24T23:10:15.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.704+0000 7f1699c91640 1 -- 172.21.15.32:0/2672969172 >> v1:172.21.15.32:6789/0 conn(0x7f169415fd60 legacy=0x7f169406c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:15.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.704+0000 7f1699c91640 1 -- 172.21.15.32:0/2672969172 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16940b4c80 con 0x7f169415b480 2024-04-24T23:10:15.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.705+0000 7f1699c91640 1 --2- 172.21.15.32:0/2672969172 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f169415b480 0x7f16940b5c60 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f16940b5170 tx=0x7f169002a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c88fc0fd8952dffe server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:15.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.705+0000 7f1698c8f640 1 -- 172.21.15.32:0/2672969172 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1690037070 con 0x7f169415b480 2024-04-24T23:10:15.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.705+0000 7f1698c8f640 1 -- 172.21.15.32:0/2672969172 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1690032440 con 0x7f169415b480 2024-04-24T23:10:15.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.705+0000 7f1698c8f640 1 -- 172.21.15.32:0/2672969172 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1690032810 con 0x7f169415b480 2024-04-24T23:10:15.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.706+0000 7f169bef5640 1 -- 172.21.15.32:0/2672969172 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f169415b480 msgr2=0x7f16940b5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:15.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.706+0000 7f169bef5640 1 --2- 172.21.15.32:0/2672969172 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f169415b480 0x7f16940b5c60 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f16940b5170 tx=0x7f169002a210 comp rx=0 tx=0).stop 2024-04-24T23:10:15.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.706+0000 7f169bef5640 1 -- 172.21.15.32:0/2672969172 shutdown_connections 2024-04-24T23:10:15.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.706+0000 7f169bef5640 1 --2- 172.21.15.32:0/2672969172 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f169415b480 0x7f16940b5c60 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:15.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.706+0000 7f169bef5640 1 -- 172.21.15.32:0/2672969172 >> 172.21.15.32:0/2672969172 conn(0x7f169406b110 msgr2=0x7f169406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:15.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.706+0000 7f169bef5640 1 -- 172.21.15.32:0/2672969172 shutdown_connections 2024-04-24T23:10:15.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.706+0000 7f169bef5640 1 -- 172.21.15.32:0/2672969172 wait complete. 2024-04-24T23:10:15.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.707+0000 7f169bef5640 1 Processor -- start 2024-04-24T23:10:15.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.707+0000 7f169bef5640 1 -- start start 2024-04-24T23:10:15.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.708+0000 7f169bef5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f169406cd00 0x7f1694158f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:15.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.708+0000 7f169bef5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f169415b480 0x7f16941f4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:15.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.708+0000 7f169bef5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f169415fd60 0x7f16941f45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:15.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.708+0000 7f169bef5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f169415b860 con 0x7f169406cd00 2024-04-24T23:10:15.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.708+0000 7f169bef5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f16940b5480 con 0x7f169415b480 2024-04-24T23:10:15.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.708+0000 7f169bef5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f169415d4b0 con 0x7f169415fd60 2024-04-24T23:10:15.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.708+0000 7f1699c91640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f169406cd00 0x7f1694158f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:15.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.708+0000 7f1699c91640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f169406cd00 0x7f1694158f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34954/0 (socket says 172.21.15.32:34954) 2024-04-24T23:10:15.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.708+0000 7f1699c91640 1 -- 172.21.15.32:0/4064798404 learned_addr learned my addr 172.21.15.32:0/4064798404 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:15.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.708+0000 7f169a492640 1 --2- 172.21.15.32:0/4064798404 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f169415fd60 0x7f16941f45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:15.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.708+0000 7f1699490640 1 --2- 172.21.15.32:0/4064798404 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f169415b480 0x7f16941f4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:15.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.708+0000 7f1699c91640 1 -- 172.21.15.32:0/4064798404 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f169415fd60 msgr2=0x7f16941f45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:15.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.708+0000 7f1699c91640 1 --2- 172.21.15.32:0/4064798404 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f169415fd60 0x7f16941f45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:15.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.708+0000 7f1699c91640 1 -- 172.21.15.32:0/4064798404 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f169415b480 msgr2=0x7f16941f4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:15.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.708+0000 7f1699c91640 1 --2- 172.21.15.32:0/4064798404 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f169415b480 0x7f16941f4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:15.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.708+0000 7f1699c91640 1 -- 172.21.15.32:0/4064798404 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f169415c8a0 con 0x7f169406cd00 2024-04-24T23:10:15.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.708+0000 7f1699490640 1 --2- 172.21.15.32:0/4064798404 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f169415b480 0x7f16941f4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:10:15.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.709+0000 7f1699c91640 1 --2- 172.21.15.32:0/4064798404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f169406cd00 0x7f1694158f00 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f1690035040 tx=0x7f16900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:15.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.709+0000 7f168a7fc640 1 -- 172.21.15.32:0/4064798404 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1690041070 con 0x7f169406cd00 2024-04-24T23:10:15.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.709+0000 7f168a7fc640 1 -- 172.21.15.32:0/4064798404 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f169003c890 con 0x7f169406cd00 2024-04-24T23:10:15.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.709+0000 7f169bef5640 1 -- 172.21.15.32:0/4064798404 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f169415cb80 con 0x7f169406cd00 2024-04-24T23:10:15.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.709+0000 7f168a7fc640 1 -- 172.21.15.32:0/4064798404 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1690037040 con 0x7f169406cd00 2024-04-24T23:10:15.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.709+0000 7f169bef5640 1 -- 172.21.15.32:0/4064798404 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f169415d140 con 0x7f169406cd00 2024-04-24T23:10:15.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.710+0000 7f169bef5640 1 -- 172.21.15.32:0/4064798404 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1694068430 con 0x7f169406cd00 2024-04-24T23:10:15.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.710+0000 7f168a7fc640 1 -- 172.21.15.32:0/4064798404 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1690044070 con 0x7f169406cd00 2024-04-24T23:10:15.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.711+0000 7f168a7fc640 1 --2- 172.21.15.32:0/4064798404 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f167806be60 0x7f167806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:15.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.711+0000 7f168a7fc640 1 -- 172.21.15.32:0/4064798404 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 5767+0+0 (secure 0 0 0) 0x7f16900abb30 con 0x7f169406cd00 2024-04-24T23:10:15.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.711+0000 7f168a7fc640 1 -- 172.21.15.32:0/4064798404 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=174}) -- 0x7f1678073a50 con 0x7f169406cd00 2024-04-24T23:10:15.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.711+0000 7f1699490640 1 --2- 172.21.15.32:0/4064798404 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f167806be60 0x7f167806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:15.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.711+0000 7f1699490640 1 --2- 172.21.15.32:0/4064798404 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f167806be60 0x7f167806e320 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f1684007fd0 tx=0x7f1684009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:15.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.715+0000 7f168a7fc640 1 -- 172.21.15.32:0/4064798404 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f169007a5c0 con 0x7f169406cd00 2024-04-24T23:10:15.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.897+0000 7f169bef5640 1 -- 172.21.15.32:0/4064798404 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f169406c000 con 0x7f169406cd00 2024-04-24T23:10:15.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.898+0000 7f168a7fc640 1 -- 172.21.15.32:0/4064798404 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v173) ==== 74+0+16756 (secure 0 0 0) 0x7f1690079ec0 con 0x7f169406cd00 2024-04-24T23:10:15.900 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:10:15.900 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":173,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:09:53.858501+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:08:59.076360+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-24T23:08:59.745667+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.1d","mappings":[{"from":2,"to":5}]},{"pgid":"21.39","mappings":[{"from":2,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:10:15.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.900+0000 7f169bef5640 1 -- 172.21.15.32:0/4064798404 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f167806be60 msgr2=0x7f167806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:15.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.900+0000 7f169bef5640 1 --2- 172.21.15.32:0/4064798404 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f167806be60 0x7f167806e320 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f1684007fd0 tx=0x7f1684009040 comp rx=0 tx=0).stop 2024-04-24T23:10:15.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.900+0000 7f169bef5640 1 -- 172.21.15.32:0/4064798404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f169406cd00 msgr2=0x7f1694158f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:15.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.900+0000 7f169bef5640 1 --2- 172.21.15.32:0/4064798404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f169406cd00 0x7f1694158f00 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f1690035040 tx=0x7f16900046c0 comp rx=0 tx=0).stop 2024-04-24T23:10:15.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.900+0000 7f169bef5640 1 -- 172.21.15.32:0/4064798404 shutdown_connections 2024-04-24T23:10:15.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.900+0000 7f169bef5640 1 --2- 172.21.15.32:0/4064798404 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f167806be60 0x7f167806e320 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:15.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.900+0000 7f169bef5640 1 --2- 172.21.15.32:0/4064798404 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f169415fd60 0x7f16941f45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:15.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.900+0000 7f169bef5640 1 --2- 172.21.15.32:0/4064798404 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f169415b480 0x7f16941f4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:15.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.900+0000 7f169bef5640 1 --2- 172.21.15.32:0/4064798404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f169406cd00 0x7f1694158f00 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:15.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.900+0000 7f169bef5640 1 -- 172.21.15.32:0/4064798404 >> 172.21.15.32:0/4064798404 conn(0x7f169406b110 msgr2=0x7f169406c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:15.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.900+0000 7f169bef5640 1 -- 172.21.15.32:0/4064798404 shutdown_connections 2024-04-24T23:10:15.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:15.901+0000 7f169bef5640 1 -- 172.21.15.32:0/4064798404 wait complete. 2024-04-24T23:10:15.913 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:08:59.076360+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '168', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:08:59.745667+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '171', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:10:15.914 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:10:16.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.015+0000 7f68bc238640 1 Processor -- start 2024-04-24T23:10:16.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.015+0000 7f68bc238640 1 -- start start 2024-04-24T23:10:16.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.016+0000 7f68bc238640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f68b415cc40 0x7f68b415d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:16.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.016+0000 7f68bc238640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f68b415d560 0x7f68b40cf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:16.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.016+0000 7f68bc238640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68b40b4ee0 con 0x7f68b40d05b0 2024-04-24T23:10:16.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.016+0000 7f68bc238640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f68b406cca0 con 0x7f68b415d560 2024-04-24T23:10:16.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.016+0000 7f68bc238640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f68b40ce5f0 con 0x7f68b415cc40 2024-04-24T23:10:16.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.016+0000 7f68ba7d5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f68b415d560 0x7f68b40cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:16.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.016+0000 7f68ba7d5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f68b415d560 0x7f68b40cf340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34960/0 (socket says 172.21.15.32:34960) 2024-04-24T23:10:16.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.016+0000 7f68ba7d5640 1 -- 172.21.15.32:0/1622218816 learned_addr learned my addr 172.21.15.32:0/1622218816 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:16.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.017+0000 7f68b9fd4640 1 --2- 172.21.15.32:0/1622218816 >> v2:172.21.15.121:3300/0 conn(0x7f68b415cc40 0x7f68b415d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:16.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.017+0000 7f68ba7d5640 1 -- 172.21.15.32:0/1622218816 >> v2:172.21.15.121:3300/0 conn(0x7f68b415cc40 msgr2=0x7f68b415d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:16.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.017+0000 7f68ba7d5640 1 --2- 172.21.15.32:0/1622218816 >> v2:172.21.15.121:3300/0 conn(0x7f68b415cc40 0x7f68b415d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.017+0000 7f68ba7d5640 1 -- 172.21.15.32:0/1622218816 >> v1:172.21.15.121:6789/0 conn(0x7f68b40d05b0 legacy=0x7f68b40cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:16.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.017+0000 7f68ba7d5640 1 -- 172.21.15.32:0/1622218816 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68b40b51c0 con 0x7f68b415d560 2024-04-24T23:10:16.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.017+0000 7f68ba7d5640 1 --2- 172.21.15.32:0/1622218816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68b415d560 0x7f68b40cf340 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f68b404f570 tx=0x7f68a402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4cd13ac345bbc03 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:16.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.018+0000 7f68b8fd2640 1 -- 172.21.15.32:0/1622218816 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a4036070 con 0x7f68b415d560 2024-04-24T23:10:16.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.018+0000 7f68b8fd2640 1 -- 172.21.15.32:0/1622218816 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68a4031600 con 0x7f68b415d560 2024-04-24T23:10:16.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.018+0000 7f68b8fd2640 1 -- 172.21.15.32:0/1622218816 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a40319d0 con 0x7f68b415d560 2024-04-24T23:10:16.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.018+0000 7f68bc238640 1 -- 172.21.15.32:0/1622218816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68b415d560 msgr2=0x7f68b40cf340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:16.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.018+0000 7f68bc238640 1 --2- 172.21.15.32:0/1622218816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68b415d560 0x7f68b40cf340 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f68b404f570 tx=0x7f68a402a730 comp rx=0 tx=0).stop 2024-04-24T23:10:16.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.018+0000 7f68bc238640 1 -- 172.21.15.32:0/1622218816 shutdown_connections 2024-04-24T23:10:16.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.018+0000 7f68bc238640 1 --2- 172.21.15.32:0/1622218816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68b415d560 0x7f68b40cf340 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.018+0000 7f68bc238640 1 --2- 172.21.15.32:0/1622218816 >> v2:172.21.15.121:3300/0 conn(0x7f68b415cc40 0x7f68b415d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.018+0000 7f68bc238640 1 -- 172.21.15.32:0/1622218816 >> 172.21.15.32:0/1622218816 conn(0x7f68b40d0ae0 msgr2=0x7f68b40d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:16.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.018+0000 7f68bc238640 1 -- 172.21.15.32:0/1622218816 shutdown_connections 2024-04-24T23:10:16.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.018+0000 7f68bc238640 1 -- 172.21.15.32:0/1622218816 wait complete. 2024-04-24T23:10:16.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.019+0000 7f68bc238640 1 Processor -- start 2024-04-24T23:10:16.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.019+0000 7f68bc238640 1 -- start start 2024-04-24T23:10:16.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.020+0000 7f68bc238640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68b40d05b0 0x7f68b413b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:16.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.020+0000 7f68bc238640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68b415cc40 0x7f68b415ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:16.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.020+0000 7f68bc238640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68b415d560 0x7f68b415f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:16.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.020+0000 7f68bc238640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f68b40b5a60 con 0x7f68b40d05b0 2024-04-24T23:10:16.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.020+0000 7f68bc238640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f68b40b5780 con 0x7f68b415cc40 2024-04-24T23:10:16.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.020+0000 7f68bc238640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f68b415d9b0 con 0x7f68b415d560 2024-04-24T23:10:16.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.020+0000 7f68b9fd4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68b40d05b0 0x7f68b413b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:16.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.020+0000 7f68b9fd4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68b40d05b0 0x7f68b413b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34970/0 (socket says 172.21.15.32:34970) 2024-04-24T23:10:16.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.020+0000 7f68b9fd4640 1 -- 172.21.15.32:0/3914576612 learned_addr learned my addr 172.21.15.32:0/3914576612 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:16.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.020+0000 7f68b97d3640 1 --2- 172.21.15.32:0/3914576612 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68b415cc40 0x7f68b415ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:16.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.020+0000 7f68ba7d5640 1 --2- 172.21.15.32:0/3914576612 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68b415d560 0x7f68b415f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:16.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.021+0000 7f68b9fd4640 1 -- 172.21.15.32:0/3914576612 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68b415d560 msgr2=0x7f68b415f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:16.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.021+0000 7f68b9fd4640 1 --2- 172.21.15.32:0/3914576612 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68b415d560 0x7f68b415f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.021+0000 7f68b9fd4640 1 -- 172.21.15.32:0/3914576612 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68b415cc40 msgr2=0x7f68b415ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:16.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.021+0000 7f68b9fd4640 1 --2- 172.21.15.32:0/3914576612 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68b415cc40 0x7f68b415ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.021+0000 7f68b9fd4640 1 -- 172.21.15.32:0/3914576612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68b41534e0 con 0x7f68b40d05b0 2024-04-24T23:10:16.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.021+0000 7f68ba7d5640 1 --2- 172.21.15.32:0/3914576612 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68b415d560 0x7f68b415f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:10:16.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.021+0000 7f68b9fd4640 1 --2- 172.21.15.32:0/3914576612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68b40d05b0 0x7f68b413b5b0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f68ac00d5d0 tx=0x7f68ac00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:16.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.021+0000 7f68b27fc640 1 -- 172.21.15.32:0/3914576612 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac0125f0 con 0x7f68b40d05b0 2024-04-24T23:10:16.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.021+0000 7f68b27fc640 1 -- 172.21.15.32:0/3914576612 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68ac019070 con 0x7f68b40d05b0 2024-04-24T23:10:16.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.021+0000 7f68bc238640 1 -- 172.21.15.32:0/3914576612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68b41537c0 con 0x7f68b40d05b0 2024-04-24T23:10:16.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.021+0000 7f68b27fc640 1 -- 172.21.15.32:0/3914576612 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac0129f0 con 0x7f68b40d05b0 2024-04-24T23:10:16.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.021+0000 7f68bc238640 1 -- 172.21.15.32:0/3914576612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68b415e130 con 0x7f68b40d05b0 2024-04-24T23:10:16.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.022+0000 7f68bc238640 1 -- 172.21.15.32:0/3914576612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68b4068430 con 0x7f68b40d05b0 2024-04-24T23:10:16.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.023+0000 7f68b27fc640 1 -- 172.21.15.32:0/3914576612 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f68ac012b90 con 0x7f68b40d05b0 2024-04-24T23:10:16.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.023+0000 7f68b27fc640 1 --2- 172.21.15.32:0/3914576612 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f68a006be30 0x7f68a006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:16.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.023+0000 7f68b27fc640 1 -- 172.21.15.32:0/3914576612 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 5767+0+0 (secure 0 0 0) 0x7f68ac08b260 con 0x7f68b40d05b0 2024-04-24T23:10:16.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.023+0000 7f68b27fc640 1 -- 172.21.15.32:0/3914576612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=174}) -- 0x7f68a0073a20 con 0x7f68b40d05b0 2024-04-24T23:10:16.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.023+0000 7f68b97d3640 1 --2- 172.21.15.32:0/3914576612 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f68a006be30 0x7f68a006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:16.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.024+0000 7f68b97d3640 1 --2- 172.21.15.32:0/3914576612 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f68a006be30 0x7f68a006e2f0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f68b41e5c50 tx=0x7f68a8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:16.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.027+0000 7f68b27fc640 1 -- 172.21.15.32:0/3914576612 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f68ac059bd0 con 0x7f68b40d05b0 2024-04-24T23:10:16.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.209+0000 7f68bc238640 1 -- 172.21.15.32:0/3914576612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f68b40d1600 con 0x7f68b40d05b0 2024-04-24T23:10:16.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.210+0000 7f68b27fc640 1 -- 172.21.15.32:0/3914576612 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v173) ==== 93+0+10 (secure 0 0 0) 0x7f68ac0594d0 con 0x7f68b40d05b0 2024-04-24T23:10:16.211 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T23:10:16.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.211+0000 7f68bc238640 1 -- 172.21.15.32:0/3914576612 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f68a006be30 msgr2=0x7f68a006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:16.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.211+0000 7f68bc238640 1 --2- 172.21.15.32:0/3914576612 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f68a006be30 0x7f68a006e2f0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f68b41e5c50 tx=0x7f68a8009040 comp rx=0 tx=0).stop 2024-04-24T23:10:16.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.211+0000 7f68bc238640 1 -- 172.21.15.32:0/3914576612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68b40d05b0 msgr2=0x7f68b413b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:16.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.211+0000 7f68bc238640 1 --2- 172.21.15.32:0/3914576612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68b40d05b0 0x7f68b413b5b0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f68ac00d5d0 tx=0x7f68ac00da90 comp rx=0 tx=0).stop 2024-04-24T23:10:16.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.212+0000 7f68bc238640 1 -- 172.21.15.32:0/3914576612 shutdown_connections 2024-04-24T23:10:16.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.212+0000 7f68bc238640 1 --2- 172.21.15.32:0/3914576612 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f68a006be30 0x7f68a006e2f0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.212+0000 7f68bc238640 1 --2- 172.21.15.32:0/3914576612 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68b415d560 0x7f68b415f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.212+0000 7f68bc238640 1 --2- 172.21.15.32:0/3914576612 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68b415cc40 0x7f68b415ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.212+0000 7f68bc238640 1 --2- 172.21.15.32:0/3914576612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68b40d05b0 0x7f68b413b5b0 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.212+0000 7f68bc238640 1 -- 172.21.15.32:0/3914576612 >> 172.21.15.32:0/3914576612 conn(0x7f68b40d0ae0 msgr2=0x7f68b40d0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:16.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.212+0000 7f68bc238640 1 -- 172.21.15.32:0/3914576612 shutdown_connections 2024-04-24T23:10:16.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.212+0000 7f68bc238640 1 -- 172.21.15.32:0/3914576612 wait complete. 2024-04-24T23:10:16.224 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:10:16.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.326+0000 7f9892061640 1 Processor -- start 2024-04-24T23:10:16.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.326+0000 7f9892061640 1 -- start start 2024-04-24T23:10:16.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.327+0000 7f9892061640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f988c0b5af0 0x7f988c0ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:16.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.327+0000 7f9892061640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f988c0cd270 con 0x7f988c162320 2024-04-24T23:10:16.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.327+0000 7f9892061640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f988c071610 con 0x7f988c162e10 2024-04-24T23:10:16.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.327+0000 7f9892061640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f988c0b5660 con 0x7f988c0b5af0 2024-04-24T23:10:16.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.327+0000 7f988b7fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f988c162320 0x7f988c162700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47940/0 (socket says 172.21.15.32:47940) 2024-04-24T23:10:16.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.327+0000 7f988b7fe640 1 -- 172.21.15.32:0/1008077928 learned_addr learned my addr 172.21.15.32:0/1008077928 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:10:16.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.327+0000 7f988affd640 1 --2- 172.21.15.32:0/1008077928 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f988c0b5af0 0x7f988c0ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:16.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.328+0000 7f988a7fc640 1 -- 172.21.15.32:0/1008077928 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3931554778 0 0) 0x7f988c0cd270 con 0x7f988c162320 2024-04-24T23:10:16.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.328+0000 7f988a7fc640 1 -- 172.21.15.32:0/1008077928 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9874003520 con 0x7f988c162320 2024-04-24T23:10:16.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.328+0000 7f988a7fc640 1 -- 172.21.15.32:0/1008077928 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2982342444 0 0) 0x7f988c071610 con 0x7f988c162e10 2024-04-24T23:10:16.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.328+0000 7f988a7fc640 1 -- 172.21.15.32:0/1008077928 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f988c0cd270 con 0x7f988c162e10 2024-04-24T23:10:16.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.328+0000 7f988a7fc640 1 -- 172.21.15.32:0/1008077928 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4166669035 0 0) 0x7f9874003520 con 0x7f988c162320 2024-04-24T23:10:16.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.328+0000 7f988affd640 1 -- 172.21.15.32:0/1008077928 >> v1:172.21.15.121:6789/0 conn(0x7f988c162e10 legacy=0x7f988c15ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:16.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.328+0000 7f988affd640 1 -- 172.21.15.32:0/1008077928 >> v1:172.21.15.32:6789/0 conn(0x7f988c162320 legacy=0x7f988c162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:16.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.328+0000 7f988affd640 1 -- 172.21.15.32:0/1008077928 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f988c0718f0 con 0x7f988c0b5af0 2024-04-24T23:10:16.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.329+0000 7f988affd640 1 --2- 172.21.15.32:0/1008077928 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f988c0b5af0 0x7f988c0ccc90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f98800097d0 tx=0x7f988002a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43c43cee81e8f48d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:16.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.329+0000 7f988a7fc640 1 -- 172.21.15.32:0/1008077928 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9880037070 con 0x7f988c0b5af0 2024-04-24T23:10:16.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.329+0000 7f988a7fc640 1 -- 172.21.15.32:0/1008077928 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f988002ab10 con 0x7f988c0b5af0 2024-04-24T23:10:16.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.329+0000 7f988a7fc640 1 -- 172.21.15.32:0/1008077928 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987c001670 con 0x7f988c0b5af0 2024-04-24T23:10:16.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.329+0000 7f9892061640 1 -- 172.21.15.32:0/1008077928 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f988c0b5af0 msgr2=0x7f988c0ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:16.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.329+0000 7f9892061640 1 --2- 172.21.15.32:0/1008077928 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f988c0b5af0 0x7f988c0ccc90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f98800097d0 tx=0x7f988002a080 comp rx=0 tx=0).stop 2024-04-24T23:10:16.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.329+0000 7f9892061640 1 -- 172.21.15.32:0/1008077928 shutdown_connections 2024-04-24T23:10:16.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.329+0000 7f9892061640 1 --2- 172.21.15.32:0/1008077928 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f988c0b5af0 0x7f988c0ccc90 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.329+0000 7f9892061640 1 -- 172.21.15.32:0/1008077928 >> 172.21.15.32:0/1008077928 conn(0x7f988c0d0ae0 msgr2=0x7f988c0d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:16.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.329+0000 7f9892061640 1 -- 172.21.15.32:0/1008077928 shutdown_connections 2024-04-24T23:10:16.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.329+0000 7f9892061640 1 -- 172.21.15.32:0/1008077928 wait complete. 2024-04-24T23:10:16.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.331+0000 7f9892061640 1 Processor -- start 2024-04-24T23:10:16.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.331+0000 7f9892061640 1 -- start start 2024-04-24T23:10:16.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.331+0000 7f9892061640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f988c0b5af0 0x7f988c1e8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:16.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.331+0000 7f9892061640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f988c162320 0x7f988c1ea020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:16.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.331+0000 7f9892061640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f988c162e10 0x7f988c15d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:16.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.331+0000 7f9892061640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f988c071bd0 con 0x7f988c162320 2024-04-24T23:10:16.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.331+0000 7f9892061640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f988c067950 con 0x7f988c0b5af0 2024-04-24T23:10:16.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.331+0000 7f9892061640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f988c071d50 con 0x7f988c162e10 2024-04-24T23:10:16.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.332+0000 7f988affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f988c162320 0x7f988c1ea020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:16.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.332+0000 7f988affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f988c162320 0x7f988c1ea020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34984/0 (socket says 172.21.15.32:34984) 2024-04-24T23:10:16.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.332+0000 7f988affd640 1 -- 172.21.15.32:0/681451417 learned_addr learned my addr 172.21.15.32:0/681451417 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:16.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.332+0000 7f988affd640 1 -- 172.21.15.32:0/681451417 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f988c162e10 msgr2=0x7f988c15d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:16.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.332+0000 7f988affd640 1 --2- 172.21.15.32:0/681451417 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f988c162e10 0x7f988c15d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.332+0000 7f988affd640 1 -- 172.21.15.32:0/681451417 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f988c0b5af0 msgr2=0x7f988c1e8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:16.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.332+0000 7f988affd640 1 --2- 172.21.15.32:0/681451417 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f988c0b5af0 0x7f988c1e8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.332+0000 7f988affd640 1 -- 172.21.15.32:0/681451417 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f988c1e9250 con 0x7f988c162320 2024-04-24T23:10:16.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.332+0000 7f988affd640 1 --2- 172.21.15.32:0/681451417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f988c162320 0x7f988c1ea020 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f9880035040 tx=0x7f98800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:16.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.332+0000 7f9873fff640 1 -- 172.21.15.32:0/681451417 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9880041020 con 0x7f988c162320 2024-04-24T23:10:16.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.332+0000 7f9873fff640 1 -- 172.21.15.32:0/681451417 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f988003d640 con 0x7f988c162320 2024-04-24T23:10:16.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.332+0000 7f9892061640 1 -- 172.21.15.32:0/681451417 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f988c1e9530 con 0x7f988c162320 2024-04-24T23:10:16.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.332+0000 7f9873fff640 1 -- 172.21.15.32:0/681451417 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9880037040 con 0x7f988c162320 2024-04-24T23:10:16.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.332+0000 7f9892061640 1 -- 172.21.15.32:0/681451417 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f988c0ce9a0 con 0x7f988c162320 2024-04-24T23:10:16.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.333+0000 7f9892061640 1 -- 172.21.15.32:0/681451417 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f988c068430 con 0x7f988c162320 2024-04-24T23:10:16.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.334+0000 7f9873fff640 1 -- 172.21.15.32:0/681451417 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9880044020 con 0x7f988c162320 2024-04-24T23:10:16.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.334+0000 7f9873fff640 1 --2- 172.21.15.32:0/681451417 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f987806be30 0x7f987806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:16.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.334+0000 7f9873fff640 1 -- 172.21.15.32:0/681451417 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 5767+0+0 (secure 0 0 0) 0x7f98800ab440 con 0x7f988c162320 2024-04-24T23:10:16.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.334+0000 7f9873fff640 1 -- 172.21.15.32:0/681451417 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=174}) -- 0x7f9878073a20 con 0x7f988c162320 2024-04-24T23:10:16.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.334+0000 7f988b7fe640 1 --2- 172.21.15.32:0/681451417 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f987806be30 0x7f987806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:16.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.335+0000 7f988b7fe640 1 --2- 172.21.15.32:0/681451417 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f987806be30 0x7f987806e2f0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f988c1621d0 tx=0x7f987c000f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:16.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.338+0000 7f9873fff640 1 -- 172.21.15.32:0/681451417 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f988004b030 con 0x7f988c162320 2024-04-24T23:10:16.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.520+0000 7f9892061640 1 -- 172.21.15.32:0/681451417 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f988c067730 con 0x7f988c162320 2024-04-24T23:10:16.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.520+0000 7f9873fff640 1 -- 172.21.15.32:0/681451417 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v173) ==== 104+0+11 (secure 0 0 0) 0x7f9880079e40 con 0x7f988c162320 2024-04-24T23:10:16.522 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-24T23:10:16.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.522+0000 7f9892061640 1 -- 172.21.15.32:0/681451417 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f987806be30 msgr2=0x7f987806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:16.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.522+0000 7f9892061640 1 --2- 172.21.15.32:0/681451417 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f987806be30 0x7f987806e2f0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f988c1621d0 tx=0x7f987c000f90 comp rx=0 tx=0).stop 2024-04-24T23:10:16.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.522+0000 7f9892061640 1 -- 172.21.15.32:0/681451417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f988c162320 msgr2=0x7f988c1ea020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:16.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.522+0000 7f9892061640 1 --2- 172.21.15.32:0/681451417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f988c162320 0x7f988c1ea020 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f9880035040 tx=0x7f98800046c0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.522+0000 7f9892061640 1 -- 172.21.15.32:0/681451417 shutdown_connections 2024-04-24T23:10:16.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.522+0000 7f9892061640 1 --2- 172.21.15.32:0/681451417 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f987806be30 0x7f987806e2f0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.522+0000 7f9892061640 1 --2- 172.21.15.32:0/681451417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f988c162320 0x7f988c1ea020 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.522+0000 7f9892061640 1 --2- 172.21.15.32:0/681451417 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f988c162e10 0x7f988c15d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.522+0000 7f9892061640 1 --2- 172.21.15.32:0/681451417 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f988c0b5af0 0x7f988c1e8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.522+0000 7f9892061640 1 -- 172.21.15.32:0/681451417 >> 172.21.15.32:0/681451417 conn(0x7f988c0d0ae0 msgr2=0x7f988c0b1340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:16.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.522+0000 7f9892061640 1 -- 172.21.15.32:0/681451417 shutdown_connections 2024-04-24T23:10:16.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.522+0000 7f9892061640 1 -- 172.21.15.32:0/681451417 wait complete. 2024-04-24T23:10:16.534 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:10:16.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.636+0000 7fdf211d5640 1 Processor -- start 2024-04-24T23:10:16.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.637+0000 7fdf211d5640 1 -- start start 2024-04-24T23:10:16.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.637+0000 7fdf211d5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdf1c06d390 0x7fdf1c06b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:16.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.637+0000 7fdf211d5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdf1c06b650 0x7fdf1c070d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:16.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.637+0000 7fdf211d5640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf1c06c7b0 con 0x7fdf1c15c7b0 2024-04-24T23:10:16.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.637+0000 7fdf211d5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fdf1c15cb90 con 0x7fdf1c06d390 2024-04-24T23:10:16.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.637+0000 7fdf211d5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fdf1c06ca90 con 0x7fdf1c06b650 2024-04-24T23:10:16.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.638+0000 7fdf1a59c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdf1c06d390 0x7fdf1c06b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:16.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.638+0000 7fdf1a59c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdf1c06d390 0x7fdf1c06b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34998/0 (socket says 172.21.15.32:34998) 2024-04-24T23:10:16.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.638+0000 7fdf1a59c640 1 -- 172.21.15.32:0/1049559350 learned_addr learned my addr 172.21.15.32:0/1049559350 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:16.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.638+0000 7fdf1ad9d640 1 --1- 172.21.15.32:0/1049559350 >> v1:172.21.15.32:6789/0 conn(0x7fdf1c15c7b0 0x7fdf1c0cdbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47956/0 (socket says 172.21.15.32:47956) 2024-04-24T23:10:16.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.638+0000 7fdf1b59e640 1 --2- 172.21.15.32:0/1049559350 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdf1c06b650 0x7fdf1c070d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:16.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.638+0000 7fdf1a59c640 1 -- 172.21.15.32:0/1049559350 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdf1c06b650 msgr2=0x7fdf1c070d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:16.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.638+0000 7fdf1a59c640 1 --2- 172.21.15.32:0/1049559350 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdf1c06b650 0x7fdf1c070d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.638+0000 7fdf1a59c640 1 -- 172.21.15.32:0/1049559350 >> v1:172.21.15.32:6789/0 conn(0x7fdf1c15c7b0 legacy=0x7fdf1c0cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:16.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.638+0000 7fdf1a59c640 1 -- 172.21.15.32:0/1049559350 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf1c06cd70 con 0x7fdf1c06d390 2024-04-24T23:10:16.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.638+0000 7fdf19d9b640 1 -- 172.21.15.32:0/1049559350 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4135182592 0 0) 0x7fdf1c06c7b0 con 0x7fdf1c15c7b0 2024-04-24T23:10:16.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.639+0000 7fdf1a59c640 1 --2- 172.21.15.32:0/1049559350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf1c06d390 0x7fdf1c06b110 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fdf10009850 tx=0x7fdf1002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb0df8fdb876c07c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:16.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.639+0000 7fdf19d9b640 1 -- 172.21.15.32:0/1049559350 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf10004030 con 0x7fdf1c06d390 2024-04-24T23:10:16.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.639+0000 7fdf19d9b640 1 -- 172.21.15.32:0/1049559350 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf100041d0 con 0x7fdf1c06d390 2024-04-24T23:10:16.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.639+0000 7fdf19d9b640 1 -- 172.21.15.32:0/1049559350 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf10003610 con 0x7fdf1c06d390 2024-04-24T23:10:16.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.639+0000 7fdf211d5640 1 -- 172.21.15.32:0/1049559350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf1c06d390 msgr2=0x7fdf1c06b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:16.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.639+0000 7fdf211d5640 1 --2- 172.21.15.32:0/1049559350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf1c06d390 0x7fdf1c06b110 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fdf10009850 tx=0x7fdf1002a230 comp rx=0 tx=0).stop 2024-04-24T23:10:16.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.639+0000 7fdf211d5640 1 -- 172.21.15.32:0/1049559350 shutdown_connections 2024-04-24T23:10:16.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.639+0000 7fdf211d5640 1 --2- 172.21.15.32:0/1049559350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf1c06d390 0x7fdf1c06b110 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.639+0000 7fdf211d5640 1 --2- 172.21.15.32:0/1049559350 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdf1c06b650 0x7fdf1c070d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.639+0000 7fdf211d5640 1 -- 172.21.15.32:0/1049559350 >> 172.21.15.32:0/1049559350 conn(0x7fdf1c0d0730 msgr2=0x7fdf1c0d0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:16.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.639+0000 7fdf211d5640 1 -- 172.21.15.32:0/1049559350 shutdown_connections 2024-04-24T23:10:16.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.639+0000 7fdf211d5640 1 -- 172.21.15.32:0/1049559350 wait complete. 2024-04-24T23:10:16.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.641+0000 7fdf211d5640 1 Processor -- start 2024-04-24T23:10:16.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.641+0000 7fdf211d5640 1 -- start start 2024-04-24T23:10:16.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.642+0000 7fdf211d5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdf1c06b650 0x7fdf1c14a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:16.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.642+0000 7fdf211d5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf1c06d390 0x7fdf1c15b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:16.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.642+0000 7fdf211d5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdf1c15c7b0 0x7fdf1c15b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:16.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.642+0000 7fdf211d5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c0d0450 con 0x7fdf1c06d390 2024-04-24T23:10:16.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.642+0000 7fdf211d5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c0d0170 con 0x7fdf1c15c7b0 2024-04-24T23:10:16.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.642+0000 7fdf211d5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fdf1c0afc30 con 0x7fdf1c06b650 2024-04-24T23:10:16.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.642+0000 7fdf1a59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf1c06d390 0x7fdf1c15b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:16.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.642+0000 7fdf1a59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf1c06d390 0x7fdf1c15b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35012/0 (socket says 172.21.15.32:35012) 2024-04-24T23:10:16.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.642+0000 7fdf1a59c640 1 -- 172.21.15.32:0/2761508752 learned_addr learned my addr 172.21.15.32:0/2761508752 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:16.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.642+0000 7fdf1ad9d640 1 --2- 172.21.15.32:0/2761508752 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdf1c06b650 0x7fdf1c14a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:16.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.642+0000 7fdf1b59e640 1 --2- 172.21.15.32:0/2761508752 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdf1c15c7b0 0x7fdf1c15b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:16.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.642+0000 7fdf1a59c640 1 -- 172.21.15.32:0/2761508752 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdf1c06b650 msgr2=0x7fdf1c14a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:16.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.642+0000 7fdf1a59c640 1 --2- 172.21.15.32:0/2761508752 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdf1c06b650 0x7fdf1c14a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.642+0000 7fdf1a59c640 1 -- 172.21.15.32:0/2761508752 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdf1c15c7b0 msgr2=0x7fdf1c15b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:16.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.642+0000 7fdf1a59c640 1 --2- 172.21.15.32:0/2761508752 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdf1c15c7b0 0x7fdf1c15b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.642+0000 7fdf1a59c640 1 -- 172.21.15.32:0/2761508752 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf1c154bc0 con 0x7fdf1c06d390 2024-04-24T23:10:16.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.642+0000 7fdf1a59c640 1 --2- 172.21.15.32:0/2761508752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf1c06d390 0x7fdf1c15b0e0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fdf10034040 tx=0x7fdf100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:16.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.642+0000 7fdf037fe640 1 -- 172.21.15.32:0/2761508752 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf10041030 con 0x7fdf1c06d390 2024-04-24T23:10:16.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.642+0000 7fdf037fe640 1 -- 172.21.15.32:0/2761508752 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf10003bd0 con 0x7fdf1c06d390 2024-04-24T23:10:16.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.642+0000 7fdf037fe640 1 -- 172.21.15.32:0/2761508752 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf10003da0 con 0x7fdf1c06d390 2024-04-24T23:10:16.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.643+0000 7fdf211d5640 1 -- 172.21.15.32:0/2761508752 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf1c154ea0 con 0x7fdf1c06d390 2024-04-24T23:10:16.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.643+0000 7fdf211d5640 1 -- 172.21.15.32:0/2761508752 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf1c155460 con 0x7fdf1c06d390 2024-04-24T23:10:16.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.644+0000 7fdf211d5640 1 -- 172.21.15.32:0/2761508752 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf1c068430 con 0x7fdf1c06d390 2024-04-24T23:10:16.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.644+0000 7fdf037fe640 1 -- 172.21.15.32:0/2761508752 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdf10044070 con 0x7fdf1c06d390 2024-04-24T23:10:16.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.644+0000 7fdf037fe640 1 --2- 172.21.15.32:0/2761508752 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdf0806be30 0x7fdf0806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:16.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.644+0000 7fdf037fe640 1 -- 172.21.15.32:0/2761508752 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 5767+0+0 (secure 0 0 0) 0x7fdf100ab820 con 0x7fdf1c06d390 2024-04-24T23:10:16.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.644+0000 7fdf037fe640 1 -- 172.21.15.32:0/2761508752 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=174}) -- 0x7fdf08073a20 con 0x7fdf1c06d390 2024-04-24T23:10:16.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.645+0000 7fdf1ad9d640 1 --2- 172.21.15.32:0/2761508752 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdf0806be30 0x7fdf0806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:16.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.645+0000 7fdf1ad9d640 1 --2- 172.21.15.32:0/2761508752 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdf0806be30 0x7fdf0806e2f0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fdf14007fd0 tx=0x7fdf14007de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:16.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.648+0000 7fdf037fe640 1 -- 172.21.15.32:0/2761508752 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdf10079b40 con 0x7fdf1c06d390 2024-04-24T23:10:16.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.831+0000 7fdf211d5640 1 -- 172.21.15.32:0/2761508752 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdf1c15bed0 con 0x7fdf1c06d390 2024-04-24T23:10:16.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.831+0000 7fdf037fe640 1 -- 172.21.15.32:0/2761508752 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v173) ==== 100+0+11 (secure 0 0 0) 0x7fdf1007de40 con 0x7fdf1c06d390 2024-04-24T23:10:16.833 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-24T23:10:16.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.833+0000 7fdf211d5640 1 -- 172.21.15.32:0/2761508752 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdf0806be30 msgr2=0x7fdf0806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:16.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.833+0000 7fdf211d5640 1 --2- 172.21.15.32:0/2761508752 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdf0806be30 0x7fdf0806e2f0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fdf14007fd0 tx=0x7fdf14007de0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.833+0000 7fdf211d5640 1 -- 172.21.15.32:0/2761508752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf1c06d390 msgr2=0x7fdf1c15b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:16.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.833+0000 7fdf211d5640 1 --2- 172.21.15.32:0/2761508752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf1c06d390 0x7fdf1c15b0e0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fdf10034040 tx=0x7fdf100046c0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.833+0000 7fdf211d5640 1 -- 172.21.15.32:0/2761508752 shutdown_connections 2024-04-24T23:10:16.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.833+0000 7fdf211d5640 1 --2- 172.21.15.32:0/2761508752 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdf0806be30 0x7fdf0806e2f0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.833+0000 7fdf211d5640 1 --2- 172.21.15.32:0/2761508752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf1c06d390 0x7fdf1c15b0e0 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.833+0000 7fdf211d5640 1 --2- 172.21.15.32:0/2761508752 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdf1c15c7b0 0x7fdf1c15b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.833+0000 7fdf211d5640 1 --2- 172.21.15.32:0/2761508752 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdf1c06b650 0x7fdf1c14a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.833+0000 7fdf211d5640 1 -- 172.21.15.32:0/2761508752 >> 172.21.15.32:0/2761508752 conn(0x7fdf1c0d0730 msgr2=0x7fdf1c0715c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:16.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.833+0000 7fdf211d5640 1 -- 172.21.15.32:0/2761508752 shutdown_connections 2024-04-24T23:10:16.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.833+0000 7fdf211d5640 1 -- 172.21.15.32:0/2761508752 wait complete. 2024-04-24T23:10:16.846 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:10:16.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.945+0000 7ff8cc4ba640 1 Processor -- start 2024-04-24T23:10:16.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.945+0000 7ff8cc4ba640 1 -- start start 2024-04-24T23:10:16.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.946+0000 7ff8cc4ba640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff8c406c5f0 0x7ff8c406c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:16.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.946+0000 7ff8cc4ba640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff8c40af890 0x7ff8c415cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:16.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.946+0000 7ff8cc4ba640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8c40b6430 con 0x7ff8c40b44d0 2024-04-24T23:10:16.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.946+0000 7ff8cc4ba640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7ff8c4156f80 con 0x7ff8c40af890 2024-04-24T23:10:16.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.946+0000 7ff8cc4ba640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff8c40b6150 con 0x7ff8c406c5f0 2024-04-24T23:10:16.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.946+0000 7ff8caa57640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff8c40af890 0x7ff8c415cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:16.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.946+0000 7ff8ca256640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff8c406c5f0 0x7ff8c406c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:16.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.946+0000 7ff8c9a55640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7ff8c40b44d0 0x7ff8c40b5890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:57344/0 (socket says 172.21.15.32:57344) 2024-04-24T23:10:16.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.946+0000 7ff8c9a55640 1 -- 172.21.15.32:0/3628469335 learned_addr learned my addr 172.21.15.32:0/3628469335 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:10:16.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.946+0000 7ff8caa57640 1 -- 172.21.15.32:0/3628469335 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff8c406c5f0 msgr2=0x7ff8c406c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:16.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.946+0000 7ff8caa57640 1 --2- 172.21.15.32:0/3628469335 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff8c406c5f0 0x7ff8c406c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.946+0000 7ff8caa57640 1 -- 172.21.15.32:0/3628469335 >> v1:172.21.15.121:6789/0 conn(0x7ff8c40b44d0 legacy=0x7ff8c40b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:16.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.946+0000 7ff8caa57640 1 -- 172.21.15.32:0/3628469335 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8c40b39f0 con 0x7ff8c40af890 2024-04-24T23:10:16.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.946+0000 7ff8ca256640 1 --2- 172.21.15.32:0/3628469335 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff8c406c5f0 0x7ff8c406c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:10:16.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.947+0000 7ff8caa57640 1 --2- 172.21.15.32:0/3628469335 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8c40af890 0x7ff8c415cac0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc009d90 tx=0x7ff8bc02a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=347e7926e48eb36b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:16.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.947+0000 7ff8c9254640 1 -- 172.21.15.32:0/3628469335 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc031600 con 0x7ff8c40af890 2024-04-24T23:10:16.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.947+0000 7ff8c9254640 1 -- 172.21.15.32:0/3628469335 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8bc0317a0 con 0x7ff8c40af890 2024-04-24T23:10:16.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.947+0000 7ff8c9254640 1 -- 172.21.15.32:0/3628469335 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc031b70 con 0x7ff8c40af890 2024-04-24T23:10:16.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.948+0000 7ff8cc4ba640 1 -- 172.21.15.32:0/3628469335 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8c40af890 msgr2=0x7ff8c415cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:16.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.948+0000 7ff8cc4ba640 1 --2- 172.21.15.32:0/3628469335 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8c40af890 0x7ff8c415cac0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc009d90 tx=0x7ff8bc02a5d0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.948+0000 7ff8cc4ba640 1 -- 172.21.15.32:0/3628469335 shutdown_connections 2024-04-24T23:10:16.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.948+0000 7ff8cc4ba640 1 --2- 172.21.15.32:0/3628469335 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8c40af890 0x7ff8c415cac0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.948+0000 7ff8cc4ba640 1 --2- 172.21.15.32:0/3628469335 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff8c406c5f0 0x7ff8c406c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.948+0000 7ff8cc4ba640 1 -- 172.21.15.32:0/3628469335 >> 172.21.15.32:0/3628469335 conn(0x7ff8c406b110 msgr2=0x7ff8c406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:16.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.948+0000 7ff8cc4ba640 1 -- 172.21.15.32:0/3628469335 shutdown_connections 2024-04-24T23:10:16.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.948+0000 7ff8cc4ba640 1 -- 172.21.15.32:0/3628469335 wait complete. 2024-04-24T23:10:16.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.949+0000 7ff8cc4ba640 1 Processor -- start 2024-04-24T23:10:16.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.949+0000 7ff8cc4ba640 1 -- start start 2024-04-24T23:10:16.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.950+0000 7ff8cc4ba640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8c406c5f0 0x7ff8c4143dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:16.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.950+0000 7ff8cc4ba640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8c40af890 0x7ff8c4158fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:16.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.950+0000 7ff8cc4ba640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff8c40b44d0 0x7ff8c40b0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:16.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.950+0000 7ff8cc4ba640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c406cf10 con 0x7ff8c40af890 2024-04-24T23:10:16.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.950+0000 7ff8cc4ba640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c40b3fb0 con 0x7ff8c406c5f0 2024-04-24T23:10:16.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.950+0000 7ff8cc4ba640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff8c415d000 con 0x7ff8c40b44d0 2024-04-24T23:10:16.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.950+0000 7ff8c9a55640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8c40af890 0x7ff8c4158fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:16.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.950+0000 7ff8c9a55640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8c40af890 0x7ff8c4158fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35022/0 (socket says 172.21.15.32:35022) 2024-04-24T23:10:16.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.950+0000 7ff8c9a55640 1 -- 172.21.15.32:0/2988051132 learned_addr learned my addr 172.21.15.32:0/2988051132 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:16.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.950+0000 7ff8ca256640 1 --2- 172.21.15.32:0/2988051132 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8c406c5f0 0x7ff8c4143dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:16.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.950+0000 7ff8caa57640 1 --2- 172.21.15.32:0/2988051132 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff8c40b44d0 0x7ff8c40b0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:16.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.950+0000 7ff8c9a55640 1 -- 172.21.15.32:0/2988051132 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff8c40b44d0 msgr2=0x7ff8c40b0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:16.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.950+0000 7ff8c9a55640 1 --2- 172.21.15.32:0/2988051132 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff8c40b44d0 0x7ff8c40b0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.950+0000 7ff8c9a55640 1 -- 172.21.15.32:0/2988051132 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8c406c5f0 msgr2=0x7ff8c4143dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:16.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.950+0000 7ff8c9a55640 1 --2- 172.21.15.32:0/2988051132 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8c406c5f0 0x7ff8c4143dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.950+0000 7ff8c9a55640 1 -- 172.21.15.32:0/2988051132 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8c4152300 con 0x7ff8c40af890 2024-04-24T23:10:16.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.951+0000 7ff8c9a55640 1 --2- 172.21.15.32:0/2988051132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8c40af890 0x7ff8c4158fa0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7ff8b400c3c0 tx=0x7ff8b400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:16.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.951+0000 7ff8baffd640 1 -- 172.21.15.32:0/2988051132 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b4017070 con 0x7ff8c40af890 2024-04-24T23:10:16.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.951+0000 7ff8baffd640 1 -- 172.21.15.32:0/2988051132 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8b4005ce0 con 0x7ff8c40af890 2024-04-24T23:10:16.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.951+0000 7ff8cc4ba640 1 -- 172.21.15.32:0/2988051132 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8c415de70 con 0x7ff8c40af890 2024-04-24T23:10:16.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.951+0000 7ff8baffd640 1 -- 172.21.15.32:0/2988051132 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b40072f0 con 0x7ff8c40af890 2024-04-24T23:10:16.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.951+0000 7ff8cc4ba640 1 -- 172.21.15.32:0/2988051132 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8c415e430 con 0x7ff8c40af890 2024-04-24T23:10:16.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.952+0000 7ff8cc4ba640 1 -- 172.21.15.32:0/2988051132 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8c4068430 con 0x7ff8c40af890 2024-04-24T23:10:16.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.952+0000 7ff8baffd640 1 -- 172.21.15.32:0/2988051132 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff8b401a070 con 0x7ff8c40af890 2024-04-24T23:10:16.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.953+0000 7ff8baffd640 1 --2- 172.21.15.32:0/2988051132 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff8a806be30 0x7ff8a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:16.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.953+0000 7ff8baffd640 1 -- 172.21.15.32:0/2988051132 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 5767+0+0 (secure 0 0 0) 0x7ff8b408a240 con 0x7ff8c40af890 2024-04-24T23:10:16.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.953+0000 7ff8ca256640 1 --2- 172.21.15.32:0/2988051132 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff8a806be30 0x7ff8a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:16.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.953+0000 7ff8baffd640 1 -- 172.21.15.32:0/2988051132 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=174}) -- 0x7ff8a8073a20 con 0x7ff8c40af890 2024-04-24T23:10:16.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.953+0000 7ff8ca256640 1 --2- 172.21.15.32:0/2988051132 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff8a806be30 0x7ff8a806e2f0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7ff8c0009560 tx=0x7ff8c0009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:16.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:16.957+0000 7ff8baffd640 1 -- 172.21.15.32:0/2988051132 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff8b40584c0 con 0x7ff8c40af890 2024-04-24T23:10:17.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.176+0000 7ff8cc4ba640 1 -- 172.21.15.32:0/2988051132 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff8c40b5670 con 0x7ff8c40af890 2024-04-24T23:10:17.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.177+0000 7ff8baffd640 1 -- 172.21.15.32:0/2988051132 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+5185 (secure 0 0 0) 0x7ff8b405c7c0 con 0x7ff8c40af890 2024-04-24T23:10:17.179 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:10:17.179 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":73,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6695,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/91086102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":91086102},{"type":"v1","addr":"172.21.15.121:6836","nonce":91086102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":6698,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1443743127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1443743127},{"type":"v1","addr":"172.21.15.121:6837","nonce":1443743127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":7423,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/4008773444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":4008773444},{"type":"v1","addr":"172.21.15.32:6835","nonce":4008773444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:09:00.770308+0000","modified":"2024-04-24T23:09:03.774877+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8679},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8679":{"gid":8679,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6834/4273183179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":4273183179},{"type":"v1","addr":"172.21.15.32:6834","nonce":4273183179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8679,"qdb_cluster":[8679]},"id":10}]} 2024-04-24T23:10:17.179 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 73 2024-04-24T23:10:17.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.179+0000 7ff8cc4ba640 1 -- 172.21.15.32:0/2988051132 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff8a806be30 msgr2=0x7ff8a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:17.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.179+0000 7ff8cc4ba640 1 --2- 172.21.15.32:0/2988051132 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff8a806be30 0x7ff8a806e2f0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7ff8c0009560 tx=0x7ff8c0009190 comp rx=0 tx=0).stop 2024-04-24T23:10:17.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.179+0000 7ff8cc4ba640 1 -- 172.21.15.32:0/2988051132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8c40af890 msgr2=0x7ff8c4158fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:17.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.179+0000 7ff8cc4ba640 1 --2- 172.21.15.32:0/2988051132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8c40af890 0x7ff8c4158fa0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7ff8b400c3c0 tx=0x7ff8b400c880 comp rx=0 tx=0).stop 2024-04-24T23:10:17.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.179+0000 7ff8cc4ba640 1 -- 172.21.15.32:0/2988051132 shutdown_connections 2024-04-24T23:10:17.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.179+0000 7ff8cc4ba640 1 --2- 172.21.15.32:0/2988051132 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff8a806be30 0x7ff8a806e2f0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.179+0000 7ff8cc4ba640 1 --2- 172.21.15.32:0/2988051132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8c40af890 0x7ff8c4158fa0 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.179+0000 7ff8cc4ba640 1 --2- 172.21.15.32:0/2988051132 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff8c40b44d0 0x7ff8c40b0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.179+0000 7ff8cc4ba640 1 --2- 172.21.15.32:0/2988051132 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8c406c5f0 0x7ff8c4143dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.179+0000 7ff8cc4ba640 1 -- 172.21.15.32:0/2988051132 >> 172.21.15.32:0/2988051132 conn(0x7ff8c406b110 msgr2=0x7ff8c4155e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:17.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.179+0000 7ff8cc4ba640 1 -- 172.21.15.32:0/2988051132 shutdown_connections 2024-04-24T23:10:17.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.179+0000 7ff8cc4ba640 1 -- 172.21.15.32:0/2988051132 wait complete. 2024-04-24T23:10:17.192 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:10:17.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.291+0000 7ff8720d4640 1 Processor -- start 2024-04-24T23:10:17.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.291+0000 7ff8720d4640 1 -- start start 2024-04-24T23:10:17.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.292+0000 7ff8720d4640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff86c0b5af0 0x7ff86c0ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:17.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.292+0000 7ff8720d4640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff86c162e10 0x7ff86c15ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:17.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.292+0000 7ff8720d4640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff86c0cd270 con 0x7ff86c162320 2024-04-24T23:10:17.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.292+0000 7ff8720d4640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7ff86c0b5660 con 0x7ff86c162e10 2024-04-24T23:10:17.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.292+0000 7ff8720d4640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7ff86c071610 con 0x7ff86c0b5af0 2024-04-24T23:10:17.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.292+0000 7ff86bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff86c162e10 0x7ff86c15ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:17.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.292+0000 7ff86bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff86c162e10 0x7ff86c15ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35032/0 (socket says 172.21.15.32:35032) 2024-04-24T23:10:17.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.292+0000 7ff86bfff640 1 -- 172.21.15.32:0/2592482432 learned_addr learned my addr 172.21.15.32:0/2592482432 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:17.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.292+0000 7ff86affd640 1 --2- 172.21.15.32:0/2592482432 >> v2:172.21.15.121:3300/0 conn(0x7ff86c0b5af0 0x7ff86c0ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:17.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.292+0000 7ff86bfff640 1 -- 172.21.15.32:0/2592482432 >> v2:172.21.15.121:3300/0 conn(0x7ff86c0b5af0 msgr2=0x7ff86c0ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:17.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.292+0000 7ff86bfff640 1 --2- 172.21.15.32:0/2592482432 >> v2:172.21.15.121:3300/0 conn(0x7ff86c0b5af0 0x7ff86c0ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.292+0000 7ff86bfff640 1 -- 172.21.15.32:0/2592482432 >> v1:172.21.15.121:6789/0 conn(0x7ff86c162320 legacy=0x7ff86c162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:17.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.292+0000 7ff86bfff640 1 -- 172.21.15.32:0/2592482432 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff86c0718f0 con 0x7ff86c162e10 2024-04-24T23:10:17.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.292+0000 7ff86affd640 1 --2- 172.21.15.32:0/2592482432 >> v2:172.21.15.121:3300/0 conn(0x7ff86c0b5af0 0x7ff86c0ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:10:17.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.293+0000 7ff86bfff640 1 --2- 172.21.15.32:0/2592482432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff86c162e10 0x7ff86c15ed30 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7ff86c0af8c0 tx=0x7ff86402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1c1250097c7ad59 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:17.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.293+0000 7ff86a7fc640 1 -- 172.21.15.32:0/2592482432 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff864036070 con 0x7ff86c162e10 2024-04-24T23:10:17.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.293+0000 7ff86a7fc640 1 -- 172.21.15.32:0/2592482432 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff864031600 con 0x7ff86c162e10 2024-04-24T23:10:17.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.293+0000 7ff86a7fc640 1 -- 172.21.15.32:0/2592482432 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8640319d0 con 0x7ff86c162e10 2024-04-24T23:10:17.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.293+0000 7ff8720d4640 1 -- 172.21.15.32:0/2592482432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff86c162e10 msgr2=0x7ff86c15ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:17.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.293+0000 7ff8720d4640 1 --2- 172.21.15.32:0/2592482432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff86c162e10 0x7ff86c15ed30 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7ff86c0af8c0 tx=0x7ff86402a660 comp rx=0 tx=0).stop 2024-04-24T23:10:17.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.293+0000 7ff8720d4640 1 -- 172.21.15.32:0/2592482432 shutdown_connections 2024-04-24T23:10:17.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.293+0000 7ff8720d4640 1 --2- 172.21.15.32:0/2592482432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff86c162e10 0x7ff86c15ed30 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.293+0000 7ff8720d4640 1 --2- 172.21.15.32:0/2592482432 >> v2:172.21.15.121:3300/0 conn(0x7ff86c0b5af0 0x7ff86c0ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.293+0000 7ff8720d4640 1 -- 172.21.15.32:0/2592482432 >> 172.21.15.32:0/2592482432 conn(0x7ff86c0d0ae0 msgr2=0x7ff86c0d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:17.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.294+0000 7ff8720d4640 1 -- 172.21.15.32:0/2592482432 shutdown_connections 2024-04-24T23:10:17.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.294+0000 7ff8720d4640 1 -- 172.21.15.32:0/2592482432 wait complete. 2024-04-24T23:10:17.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.295+0000 7ff8720d4640 1 Processor -- start 2024-04-24T23:10:17.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.295+0000 7ff8720d4640 1 -- start start 2024-04-24T23:10:17.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.295+0000 7ff8720d4640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff86c0b5af0 0x7ff86c0cded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:17.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.295+0000 7ff8720d4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff86c162320 0x7ff86c0af140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:17.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.295+0000 7ff8720d4640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff86c162e10 0x7ff86c0cc260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:17.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.295+0000 7ff8720d4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff86c071bd0 con 0x7ff86c162320 2024-04-24T23:10:17.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.295+0000 7ff8720d4640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff86c067950 con 0x7ff86c0b5af0 2024-04-24T23:10:17.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.295+0000 7ff8720d4640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff86c071d50 con 0x7ff86c162e10 2024-04-24T23:10:17.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.296+0000 7ff86affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff86c162320 0x7ff86c0af140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:17.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.296+0000 7ff86affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff86c162320 0x7ff86c0af140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35042/0 (socket says 172.21.15.32:35042) 2024-04-24T23:10:17.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.296+0000 7ff86affd640 1 -- 172.21.15.32:0/10017530 learned_addr learned my addr 172.21.15.32:0/10017530 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:17.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.296+0000 7ff86b7fe640 1 --2- 172.21.15.32:0/10017530 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff86c0b5af0 0x7ff86c0cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:17.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.296+0000 7ff86bfff640 1 --2- 172.21.15.32:0/10017530 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff86c162e10 0x7ff86c0cc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:17.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.296+0000 7ff86affd640 1 -- 172.21.15.32:0/10017530 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff86c162e10 msgr2=0x7ff86c0cc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:17.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.296+0000 7ff86affd640 1 --2- 172.21.15.32:0/10017530 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff86c162e10 0x7ff86c0cc260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.296+0000 7ff86affd640 1 -- 172.21.15.32:0/10017530 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff86c0b5af0 msgr2=0x7ff86c0cded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:17.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.296+0000 7ff86affd640 1 --2- 172.21.15.32:0/10017530 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff86c0b5af0 0x7ff86c0cded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.296+0000 7ff86affd640 1 -- 172.21.15.32:0/10017530 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff86c0af680 con 0x7ff86c162320 2024-04-24T23:10:17.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.296+0000 7ff86affd640 1 --2- 172.21.15.32:0/10017530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff86c162320 0x7ff86c0af140 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7ff86000c7f0 tx=0x7ff86000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:17.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.296+0000 7ff853fff640 1 -- 172.21.15.32:0/10017530 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff860015070 con 0x7ff86c162320 2024-04-24T23:10:17.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.296+0000 7ff853fff640 1 -- 172.21.15.32:0/10017530 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff860007a30 con 0x7ff86c162320 2024-04-24T23:10:17.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.296+0000 7ff8720d4640 1 -- 172.21.15.32:0/10017530 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff86c0cc7a0 con 0x7ff86c162320 2024-04-24T23:10:17.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.296+0000 7ff853fff640 1 -- 172.21.15.32:0/10017530 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff860007e00 con 0x7ff86c162320 2024-04-24T23:10:17.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.296+0000 7ff8720d4640 1 -- 172.21.15.32:0/10017530 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff86c0ce850 con 0x7ff86c162320 2024-04-24T23:10:17.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.297+0000 7ff8720d4640 1 -- 172.21.15.32:0/10017530 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff86c068430 con 0x7ff86c162320 2024-04-24T23:10:17.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.298+0000 7ff853fff640 1 -- 172.21.15.32:0/10017530 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff860018070 con 0x7ff86c162320 2024-04-24T23:10:17.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.298+0000 7ff853fff640 1 --2- 172.21.15.32:0/10017530 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff85806be30 0x7ff85806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:17.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.298+0000 7ff853fff640 1 -- 172.21.15.32:0/10017530 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 5767+0+0 (secure 0 0 0) 0x7ff86008ae10 con 0x7ff86c162320 2024-04-24T23:10:17.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.298+0000 7ff853fff640 1 -- 172.21.15.32:0/10017530 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=174}) -- 0x7ff858073a20 con 0x7ff86c162320 2024-04-24T23:10:17.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.298+0000 7ff86b7fe640 1 --2- 172.21.15.32:0/10017530 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff85806be30 0x7ff85806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:17.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.299+0000 7ff86b7fe640 1 --2- 172.21.15.32:0/10017530 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff85806be30 0x7ff85806e2f0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7ff85c004f60 tx=0x7ff85c007e30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:17.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.303+0000 7ff853fff640 1 -- 172.21.15.32:0/10017530 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff860059130 con 0x7ff86c162320 2024-04-24T23:10:17.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.490+0000 7ff8720d4640 1 -- 172.21.15.32:0/10017530 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff86c067730 con 0x7ff86c162320 2024-04-24T23:10:17.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.491+0000 7ff853fff640 1 -- 172.21.15.32:0/10017530 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v173) ==== 74+0+16756 (secure 0 0 0) 0x7ff86005d430 con 0x7ff86c162320 2024-04-24T23:10:17.493 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:10:17.493 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":173,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:09:53.858501+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:08:59.076360+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-24T23:08:59.745667+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.1d","mappings":[{"from":2,"to":5}]},{"pgid":"21.39","mappings":[{"from":2,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:10:17.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.493+0000 7ff8720d4640 1 -- 172.21.15.32:0/10017530 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff85806be30 msgr2=0x7ff85806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:17.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.493+0000 7ff8720d4640 1 --2- 172.21.15.32:0/10017530 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff85806be30 0x7ff85806e2f0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7ff85c004f60 tx=0x7ff85c007e30 comp rx=0 tx=0).stop 2024-04-24T23:10:17.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.493+0000 7ff8720d4640 1 -- 172.21.15.32:0/10017530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff86c162320 msgr2=0x7ff86c0af140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:17.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.493+0000 7ff8720d4640 1 --2- 172.21.15.32:0/10017530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff86c162320 0x7ff86c0af140 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7ff86000c7f0 tx=0x7ff86000cbf0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.493+0000 7ff8720d4640 1 -- 172.21.15.32:0/10017530 shutdown_connections 2024-04-24T23:10:17.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.493+0000 7ff8720d4640 1 --2- 172.21.15.32:0/10017530 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff85806be30 0x7ff85806e2f0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.493+0000 7ff8720d4640 1 --2- 172.21.15.32:0/10017530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff86c162320 0x7ff86c0af140 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.493+0000 7ff8720d4640 1 --2- 172.21.15.32:0/10017530 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff86c162e10 0x7ff86c0cc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.493+0000 7ff8720d4640 1 --2- 172.21.15.32:0/10017530 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff86c0b5af0 0x7ff86c0cded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.493+0000 7ff8720d4640 1 -- 172.21.15.32:0/10017530 >> 172.21.15.32:0/10017530 conn(0x7ff86c0d0ae0 msgr2=0x7ff86c06ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:17.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.493+0000 7ff8720d4640 1 -- 172.21.15.32:0/10017530 shutdown_connections 2024-04-24T23:10:17.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.493+0000 7ff8720d4640 1 -- 172.21.15.32:0/10017530 wait complete. 2024-04-24T23:10:17.506 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-24T23:10:17.506 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:10:17.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.605+0000 7fc782a9e640 1 Processor -- start 2024-04-24T23:10:17.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.605+0000 7fc782a9e640 1 -- start start 2024-04-24T23:10:17.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.606+0000 7fc782a9e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc77c06c5f0 0x7fc77c06c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:17.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.606+0000 7fc782a9e640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc77c0af890 0x7fc77c15cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:17.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.606+0000 7fc782a9e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc77c0b6430 con 0x7fc77c0b44d0 2024-04-24T23:10:17.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.606+0000 7fc782a9e640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fc77c156f80 con 0x7fc77c0af890 2024-04-24T23:10:17.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.606+0000 7fc782a9e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc77c0b6150 con 0x7fc77c06c5f0 2024-04-24T23:10:17.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.606+0000 7fc77bfff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc77c0b44d0 0x7fc77c0b5890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47972/0 (socket says 172.21.15.32:47972) 2024-04-24T23:10:17.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.606+0000 7fc77bfff640 1 -- 172.21.15.32:0/1745271322 learned_addr learned my addr 172.21.15.32:0/1745271322 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:10:17.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.606+0000 7fc78083a640 1 --2- 172.21.15.32:0/1745271322 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc77c06c5f0 0x7fc77c06c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:17.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.606+0000 7fc78103b640 1 --2- 172.21.15.32:0/1745271322 >> v2:172.21.15.121:3300/0 conn(0x7fc77c0af890 0x7fc77c15cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:17.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.606+0000 7fc77b7fe640 1 -- 172.21.15.32:0/1745271322 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1241449852 0 0) 0x7fc77c0b6430 con 0x7fc77c0b44d0 2024-04-24T23:10:17.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.606+0000 7fc77b7fe640 1 -- 172.21.15.32:0/1745271322 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc76c003520 con 0x7fc77c0b44d0 2024-04-24T23:10:17.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.606+0000 7fc77b7fe640 1 -- 172.21.15.32:0/1745271322 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3475597673 0 0) 0x7fc76c003520 con 0x7fc77c0b44d0 2024-04-24T23:10:17.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.607+0000 7fc78083a640 1 -- 172.21.15.32:0/1745271322 >> v2:172.21.15.121:3300/0 conn(0x7fc77c0af890 msgr2=0x7fc77c15cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:17.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.607+0000 7fc78083a640 1 --2- 172.21.15.32:0/1745271322 >> v2:172.21.15.121:3300/0 conn(0x7fc77c0af890 0x7fc77c15cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.607+0000 7fc78083a640 1 -- 172.21.15.32:0/1745271322 >> v1:172.21.15.32:6789/0 conn(0x7fc77c0b44d0 legacy=0x7fc77c0b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:17.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.607+0000 7fc78083a640 1 -- 172.21.15.32:0/1745271322 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc77c0b39f0 con 0x7fc77c06c5f0 2024-04-24T23:10:17.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.607+0000 7fc78083a640 1 --2- 172.21.15.32:0/1745271322 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc77c06c5f0 0x7fc77c06c9d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc7700097a0 tx=0x7fc77002a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=754c9e729d3c2529 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:17.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.607+0000 7fc77b7fe640 1 -- 172.21.15.32:0/1745271322 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc770037070 con 0x7fc77c06c5f0 2024-04-24T23:10:17.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.608+0000 7fc77b7fe640 1 -- 172.21.15.32:0/1745271322 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc77002ab10 con 0x7fc77c06c5f0 2024-04-24T23:10:17.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.608+0000 7fc77b7fe640 1 -- 172.21.15.32:0/1745271322 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc768001670 con 0x7fc77c06c5f0 2024-04-24T23:10:17.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.608+0000 7fc782a9e640 1 -- 172.21.15.32:0/1745271322 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc77c06c5f0 msgr2=0x7fc77c06c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:17.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.608+0000 7fc782a9e640 1 --2- 172.21.15.32:0/1745271322 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc77c06c5f0 0x7fc77c06c9d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc7700097a0 tx=0x7fc77002a080 comp rx=0 tx=0).stop 2024-04-24T23:10:17.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.608+0000 7fc782a9e640 1 -- 172.21.15.32:0/1745271322 shutdown_connections 2024-04-24T23:10:17.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.608+0000 7fc782a9e640 1 --2- 172.21.15.32:0/1745271322 >> v2:172.21.15.121:3300/0 conn(0x7fc77c0af890 0x7fc77c15cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.608+0000 7fc782a9e640 1 --2- 172.21.15.32:0/1745271322 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc77c06c5f0 0x7fc77c06c9d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.608+0000 7fc782a9e640 1 -- 172.21.15.32:0/1745271322 >> 172.21.15.32:0/1745271322 conn(0x7fc77c06b110 msgr2=0x7fc77c06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:17.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.608+0000 7fc782a9e640 1 -- 172.21.15.32:0/1745271322 shutdown_connections 2024-04-24T23:10:17.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.608+0000 7fc782a9e640 1 -- 172.21.15.32:0/1745271322 wait complete. 2024-04-24T23:10:17.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.609+0000 7fc782a9e640 1 Processor -- start 2024-04-24T23:10:17.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.609+0000 7fc782a9e640 1 -- start start 2024-04-24T23:10:17.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.610+0000 7fc782a9e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc77c06c5f0 0x7fc77c1f3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:17.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.610+0000 7fc782a9e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc77c0af890 0x7fc77c0b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:17.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.610+0000 7fc782a9e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc77c0b44d0 0x7fc77c0b1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:17.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.610+0000 7fc782a9e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc77c06cf10 con 0x7fc77c0af890 2024-04-24T23:10:17.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.610+0000 7fc782a9e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc77c0b3fb0 con 0x7fc77c06c5f0 2024-04-24T23:10:17.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.610+0000 7fc782a9e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc77c15d000 con 0x7fc77c0b44d0 2024-04-24T23:10:17.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.610+0000 7fc77bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc77c0af890 0x7fc77c0b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:17.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.610+0000 7fc77bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc77c0af890 0x7fc77c0b1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35056/0 (socket says 172.21.15.32:35056) 2024-04-24T23:10:17.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.610+0000 7fc77bfff640 1 -- 172.21.15.32:0/2718506080 learned_addr learned my addr 172.21.15.32:0/2718506080 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:17.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.611+0000 7fc78103b640 1 --2- 172.21.15.32:0/2718506080 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc77c0b44d0 0x7fc77c0b1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:17.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.611+0000 7fc78083a640 1 --2- 172.21.15.32:0/2718506080 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc77c06c5f0 0x7fc77c1f3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:17.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.611+0000 7fc77bfff640 1 -- 172.21.15.32:0/2718506080 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc77c0b44d0 msgr2=0x7fc77c0b1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:17.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.611+0000 7fc77bfff640 1 --2- 172.21.15.32:0/2718506080 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc77c0b44d0 0x7fc77c0b1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.611+0000 7fc77bfff640 1 -- 172.21.15.32:0/2718506080 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc77c06c5f0 msgr2=0x7fc77c1f3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:17.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.611+0000 7fc77bfff640 1 --2- 172.21.15.32:0/2718506080 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc77c06c5f0 0x7fc77c1f3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.611+0000 7fc77bfff640 1 -- 172.21.15.32:0/2718506080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc77c1e7a60 con 0x7fc77c0af890 2024-04-24T23:10:17.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.611+0000 7fc77bfff640 1 --2- 172.21.15.32:0/2718506080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc77c0af890 0x7fc77c0b1350 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fc76800c700 tx=0x7fc76800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:17.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.611+0000 7fc7797fa640 1 -- 172.21.15.32:0/2718506080 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc768017070 con 0x7fc77c0af890 2024-04-24T23:10:17.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.611+0000 7fc7797fa640 1 -- 172.21.15.32:0/2718506080 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc768005ad0 con 0x7fc77c0af890 2024-04-24T23:10:17.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.611+0000 7fc782a9e640 1 -- 172.21.15.32:0/2718506080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc77c1e7d40 con 0x7fc77c0af890 2024-04-24T23:10:17.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.611+0000 7fc7797fa640 1 -- 172.21.15.32:0/2718506080 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc768007070 con 0x7fc77c0af890 2024-04-24T23:10:17.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.611+0000 7fc782a9e640 1 -- 172.21.15.32:0/2718506080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc77c1e53a0 con 0x7fc77c0af890 2024-04-24T23:10:17.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.612+0000 7fc782a9e640 1 -- 172.21.15.32:0/2718506080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc77c068430 con 0x7fc77c0af890 2024-04-24T23:10:17.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.613+0000 7fc7797fa640 1 -- 172.21.15.32:0/2718506080 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc76801a070 con 0x7fc77c0af890 2024-04-24T23:10:17.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.613+0000 7fc7797fa640 1 --2- 172.21.15.32:0/2718506080 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc76006be30 0x7fc76006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:17.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.613+0000 7fc7797fa640 1 -- 172.21.15.32:0/2718506080 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 5767+0+0 (secure 0 0 0) 0x7fc76808a330 con 0x7fc77c0af890 2024-04-24T23:10:17.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.613+0000 7fc7797fa640 1 -- 172.21.15.32:0/2718506080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=174}) -- 0x7fc760073a20 con 0x7fc77c0af890 2024-04-24T23:10:17.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.613+0000 7fc78083a640 1 --2- 172.21.15.32:0/2718506080 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc76006be30 0x7fc76006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:17.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.614+0000 7fc78083a640 1 --2- 172.21.15.32:0/2718506080 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc76006be30 0x7fc76006e2f0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fc770034000 tx=0x7fc77002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:17.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.617+0000 7fc7797fa640 1 -- 172.21.15.32:0/2718506080 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc7680585b0 con 0x7fc77c0af890 2024-04-24T23:10:17.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.835+0000 7fc782a9e640 1 -- 172.21.15.32:0/2718506080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc77c1e7f10 con 0x7fc77c0af890 2024-04-24T23:10:17.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.836+0000 7fc7797fa640 1 -- 172.21.15.32:0/2718506080 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+5185 (secure 0 0 0) 0x7fc76805c8b0 con 0x7fc77c0af890 2024-04-24T23:10:17.837 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:10:17.838 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":73,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6695,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/91086102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":91086102},{"type":"v1","addr":"172.21.15.121:6836","nonce":91086102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":6698,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1443743127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1443743127},{"type":"v1","addr":"172.21.15.121:6837","nonce":1443743127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":7423,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/4008773444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":4008773444},{"type":"v1","addr":"172.21.15.32:6835","nonce":4008773444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:09:00.770308+0000","modified":"2024-04-24T23:09:03.774877+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8679},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8679":{"gid":8679,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6834/4273183179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":4273183179},{"type":"v1","addr":"172.21.15.32:6834","nonce":4273183179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8679,"qdb_cluster":[8679]},"id":10}]} 2024-04-24T23:10:17.838 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 73 2024-04-24T23:10:17.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.838+0000 7fc782a9e640 1 -- 172.21.15.32:0/2718506080 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc76006be30 msgr2=0x7fc76006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:17.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.838+0000 7fc782a9e640 1 --2- 172.21.15.32:0/2718506080 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc76006be30 0x7fc76006e2f0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fc770034000 tx=0x7fc77002e000 comp rx=0 tx=0).stop 2024-04-24T23:10:17.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.838+0000 7fc782a9e640 1 -- 172.21.15.32:0/2718506080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc77c0af890 msgr2=0x7fc77c0b1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:17.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.838+0000 7fc782a9e640 1 --2- 172.21.15.32:0/2718506080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc77c0af890 0x7fc77c0b1350 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fc76800c700 tx=0x7fc76800cbc0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.838+0000 7fc782a9e640 1 -- 172.21.15.32:0/2718506080 shutdown_connections 2024-04-24T23:10:17.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.838+0000 7fc782a9e640 1 --2- 172.21.15.32:0/2718506080 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc76006be30 0x7fc76006e2f0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.838+0000 7fc782a9e640 1 --2- 172.21.15.32:0/2718506080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc77c0af890 0x7fc77c0b1350 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.838+0000 7fc782a9e640 1 --2- 172.21.15.32:0/2718506080 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc77c0b44d0 0x7fc77c0b1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.838+0000 7fc782a9e640 1 --2- 172.21.15.32:0/2718506080 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc77c06c5f0 0x7fc77c1f3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.838+0000 7fc782a9e640 1 -- 172.21.15.32:0/2718506080 >> 172.21.15.32:0/2718506080 conn(0x7fc77c06b110 msgr2=0x7fc77c0d0190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:17.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.838+0000 7fc782a9e640 1 -- 172.21.15.32:0/2718506080 shutdown_connections 2024-04-24T23:10:17.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.838+0000 7fc782a9e640 1 -- 172.21.15.32:0/2718506080 wait complete. 2024-04-24T23:10:17.850 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:10:17.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.950+0000 7f0f24d1b640 1 Processor -- start 2024-04-24T23:10:17.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.950+0000 7f0f24d1b640 1 -- start start 2024-04-24T23:10:17.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.951+0000 7f0f24d1b640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0f2015c5b0 0x7f0f200b4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:17.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.951+0000 7f0f24d1b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0f20072cf0 0x7f0f2006c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:17.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.951+0000 7f0f24d1b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f2015c990 con 0x7f0f20071770 2024-04-24T23:10:17.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.951+0000 7f0f24d1b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f0f20072860 con 0x7f0f20072cf0 2024-04-24T23:10:17.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.951+0000 7f0f24d1b640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f0f20071c50 con 0x7f0f2015c5b0 2024-04-24T23:10:17.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.951+0000 7f0f1dd9b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0f20072cf0 0x7f0f2006c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:17.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.951+0000 7f0f1ed9d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0f20071770 0x7f0f200cef70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47986/0 (socket says 172.21.15.32:47986) 2024-04-24T23:10:17.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.951+0000 7f0f1ed9d640 1 -- 172.21.15.32:0/1396395311 learned_addr learned my addr 172.21.15.32:0/1396395311 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:10:17.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.952+0000 7f0f1e59c640 1 --2- 172.21.15.32:0/1396395311 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0f2015c5b0 0x7f0f200b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:17.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.952+0000 7f0f1dd9b640 1 -- 172.21.15.32:0/1396395311 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0f2015c5b0 msgr2=0x7f0f200b4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:17.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.952+0000 7f0f1dd9b640 1 --2- 172.21.15.32:0/1396395311 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0f2015c5b0 0x7f0f200b4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.952+0000 7f0f1dd9b640 1 -- 172.21.15.32:0/1396395311 >> v1:172.21.15.32:6789/0 conn(0x7f0f20071770 legacy=0x7f0f200cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:17.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.952+0000 7f0f1dd9b640 1 -- 172.21.15.32:0/1396395311 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f20071f30 con 0x7f0f20072cf0 2024-04-24T23:10:17.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.952+0000 7f0f1d59a640 1 -- 172.21.15.32:0/1396395311 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1770105845 0 0) 0x7f0f10001490 con 0x7f0f20071770 2024-04-24T23:10:17.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.952+0000 7f0f1dd9b640 1 --2- 172.21.15.32:0/1396395311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f20072cf0 0x7f0f2006c5f0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f0f080097a0 tx=0x7f0f0802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=801bdd274a3b3139 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:17.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.952+0000 7f0f1d59a640 1 -- 172.21.15.32:0/1396395311 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f08037070 con 0x7f0f20072cf0 2024-04-24T23:10:17.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.952+0000 7f0f1d59a640 1 -- 172.21.15.32:0/1396395311 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f0802ae50 con 0x7f0f20072cf0 2024-04-24T23:10:17.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.952+0000 7f0f1d59a640 1 -- 172.21.15.32:0/1396395311 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f08032660 con 0x7f0f20072cf0 2024-04-24T23:10:17.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.953+0000 7f0f24d1b640 1 -- 172.21.15.32:0/1396395311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f20072cf0 msgr2=0x7f0f2006c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:17.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.953+0000 7f0f24d1b640 1 --2- 172.21.15.32:0/1396395311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f20072cf0 0x7f0f2006c5f0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f0f080097a0 tx=0x7f0f0802a100 comp rx=0 tx=0).stop 2024-04-24T23:10:17.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.953+0000 7f0f24d1b640 1 -- 172.21.15.32:0/1396395311 shutdown_connections 2024-04-24T23:10:17.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.953+0000 7f0f24d1b640 1 --2- 172.21.15.32:0/1396395311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f20072cf0 0x7f0f2006c5f0 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.953+0000 7f0f24d1b640 1 --2- 172.21.15.32:0/1396395311 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0f2015c5b0 0x7f0f200b4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.953+0000 7f0f24d1b640 1 -- 172.21.15.32:0/1396395311 >> 172.21.15.32:0/1396395311 conn(0x7f0f2006b110 msgr2=0x7f0f2006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:17.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.953+0000 7f0f24d1b640 1 -- 172.21.15.32:0/1396395311 shutdown_connections 2024-04-24T23:10:17.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.953+0000 7f0f24d1b640 1 -- 172.21.15.32:0/1396395311 wait complete. 2024-04-24T23:10:17.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.954+0000 7f0f24d1b640 1 Processor -- start 2024-04-24T23:10:17.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.954+0000 7f0f24d1b640 1 -- start start 2024-04-24T23:10:17.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.955+0000 7f0f24d1b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f20071770 0x7f0f200ce710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:17.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.955+0000 7f0f24d1b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f20072cf0 0x7f0f20153520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:17.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.955+0000 7f0f24d1b640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0f2015c5b0 0x7f0f2006c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:17.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.955+0000 7f0f24d1b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0f200b4570 con 0x7f0f20072cf0 2024-04-24T23:10:17.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.955+0000 7f0f24d1b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0f200724f0 con 0x7f0f20071770 2024-04-24T23:10:17.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.955+0000 7f0f24d1b640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f0f200b46f0 con 0x7f0f2015c5b0 2024-04-24T23:10:17.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.955+0000 7f0f1dd9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f20072cf0 0x7f0f20153520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:17.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.955+0000 7f0f1dd9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f20072cf0 0x7f0f20153520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35076/0 (socket says 172.21.15.32:35076) 2024-04-24T23:10:17.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.955+0000 7f0f1dd9b640 1 -- 172.21.15.32:0/3463832420 learned_addr learned my addr 172.21.15.32:0/3463832420 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:17.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.955+0000 7f0f1e59c640 1 --2- 172.21.15.32:0/3463832420 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f20071770 0x7f0f200ce710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:17.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.955+0000 7f0f1ed9d640 1 --2- 172.21.15.32:0/3463832420 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0f2015c5b0 0x7f0f2006c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:17.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.956+0000 7f0f1dd9b640 1 -- 172.21.15.32:0/3463832420 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0f2015c5b0 msgr2=0x7f0f2006c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:17.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.956+0000 7f0f1dd9b640 1 --2- 172.21.15.32:0/3463832420 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0f2015c5b0 0x7f0f2006c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.956+0000 7f0f1dd9b640 1 -- 172.21.15.32:0/3463832420 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f20071770 msgr2=0x7f0f200ce710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:17.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.956+0000 7f0f1dd9b640 1 --2- 172.21.15.32:0/3463832420 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f20071770 0x7f0f200ce710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.956+0000 7f0f1dd9b640 1 -- 172.21.15.32:0/3463832420 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f200cecf0 con 0x7f0f20072cf0 2024-04-24T23:10:17.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.956+0000 7f0f1dd9b640 1 --2- 172.21.15.32:0/3463832420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f20072cf0 0x7f0f20153520 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f0f080097a0 tx=0x7f0f080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:17.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.956+0000 7f0f16ffd640 1 -- 172.21.15.32:0/3463832420 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f08037040 con 0x7f0f20072cf0 2024-04-24T23:10:17.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.956+0000 7f0f16ffd640 1 -- 172.21.15.32:0/3463832420 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f08004030 con 0x7f0f20072cf0 2024-04-24T23:10:17.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.956+0000 7f0f24d1b640 1 -- 172.21.15.32:0/3463832420 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f2015f420 con 0x7f0f20072cf0 2024-04-24T23:10:17.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.956+0000 7f0f16ffd640 1 -- 172.21.15.32:0/3463832420 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f08004420 con 0x7f0f20072cf0 2024-04-24T23:10:17.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.956+0000 7f0f24d1b640 1 -- 172.21.15.32:0/3463832420 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f2015f140 con 0x7f0f20072cf0 2024-04-24T23:10:17.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.957+0000 7f0f24d1b640 1 -- 172.21.15.32:0/3463832420 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f20068430 con 0x7f0f20072cf0 2024-04-24T23:10:17.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.958+0000 7f0f16ffd640 1 -- 172.21.15.32:0/3463832420 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0f08003610 con 0x7f0f20072cf0 2024-04-24T23:10:17.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.958+0000 7f0f16ffd640 1 --2- 172.21.15.32:0/3463832420 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0f0006be30 0x7f0f0006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:17.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.958+0000 7f0f16ffd640 1 -- 172.21.15.32:0/3463832420 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 5767+0+0 (secure 0 0 0) 0x7f0f080abb10 con 0x7f0f20072cf0 2024-04-24T23:10:17.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.958+0000 7f0f1e59c640 1 --2- 172.21.15.32:0/3463832420 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0f0006be30 0x7f0f0006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:17.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.958+0000 7f0f16ffd640 1 -- 172.21.15.32:0/3463832420 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=174}) -- 0x7f0f00073a20 con 0x7f0f20072cf0 2024-04-24T23:10:17.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.959+0000 7f0f1e59c640 1 --2- 172.21.15.32:0/3463832420 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0f0006be30 0x7f0f0006e2f0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f0f18006fd0 tx=0x7f0f18008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:17.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:17.963+0000 7f0f16ffd640 1 -- 172.21.15.32:0/3463832420 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0f0807a480 con 0x7f0f20072cf0 2024-04-24T23:10:18.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.185+0000 7f0f24d1b640 1 -- 172.21.15.32:0/3463832420 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f200b1ab0 con 0x7f0f20072cf0 2024-04-24T23:10:18.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.186+0000 7f0f16ffd640 1 -- 172.21.15.32:0/3463832420 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+5185 (secure 0 0 0) 0x7f0f08079d80 con 0x7f0f20072cf0 2024-04-24T23:10:18.188 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:10:18.188 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":73,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6695,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/91086102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":91086102},{"type":"v1","addr":"172.21.15.121:6836","nonce":91086102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":6698,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1443743127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1443743127},{"type":"v1","addr":"172.21.15.121:6837","nonce":1443743127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":7423,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/4008773444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":4008773444},{"type":"v1","addr":"172.21.15.32:6835","nonce":4008773444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:09:00.770308+0000","modified":"2024-04-24T23:09:03.774877+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8679},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8679":{"gid":8679,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6834/4273183179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":4273183179},{"type":"v1","addr":"172.21.15.32:6834","nonce":4273183179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8679,"qdb_cluster":[8679]},"id":10}]} 2024-04-24T23:10:18.188 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 73 2024-04-24T23:10:18.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.187+0000 7f0f24d1b640 1 -- 172.21.15.32:0/3463832420 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0f0006be30 msgr2=0x7f0f0006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:18.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.187+0000 7f0f24d1b640 1 --2- 172.21.15.32:0/3463832420 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0f0006be30 0x7f0f0006e2f0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f0f18006fd0 tx=0x7f0f18008040 comp rx=0 tx=0).stop 2024-04-24T23:10:18.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.188+0000 7f0f24d1b640 1 -- 172.21.15.32:0/3463832420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f20072cf0 msgr2=0x7f0f20153520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:18.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.188+0000 7f0f24d1b640 1 --2- 172.21.15.32:0/3463832420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f20072cf0 0x7f0f20153520 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f0f080097a0 tx=0x7f0f080046c0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.188+0000 7f0f24d1b640 1 -- 172.21.15.32:0/3463832420 shutdown_connections 2024-04-24T23:10:18.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.188+0000 7f0f24d1b640 1 --2- 172.21.15.32:0/3463832420 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f0f0006be30 0x7f0f0006e2f0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.188+0000 7f0f24d1b640 1 --2- 172.21.15.32:0/3463832420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f20072cf0 0x7f0f20153520 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.188+0000 7f0f24d1b640 1 --2- 172.21.15.32:0/3463832420 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0f2015c5b0 0x7f0f2006c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.188+0000 7f0f24d1b640 1 --2- 172.21.15.32:0/3463832420 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f20071770 0x7f0f200ce710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.188+0000 7f0f24d1b640 1 -- 172.21.15.32:0/3463832420 >> 172.21.15.32:0/3463832420 conn(0x7f0f2006b110 msgr2=0x7f0f200aff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:18.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.188+0000 7f0f24d1b640 1 -- 172.21.15.32:0/3463832420 shutdown_connections 2024-04-24T23:10:18.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.188+0000 7f0f24d1b640 1 -- 172.21.15.32:0/3463832420 wait complete. 2024-04-24T23:10:18.200 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:10:18.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.299+0000 7f289bf4c640 1 Processor -- start 2024-04-24T23:10:18.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.299+0000 7f289bf4c640 1 -- start start 2024-04-24T23:10:18.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.300+0000 7f289bf4c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f289406c5f0 0x7f289406c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:18.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.300+0000 7f289bf4c640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f28940b44d0 0x7f28940b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:18.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.300+0000 7f289bf4c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f28940af890 0x7f28940cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:18.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.300+0000 7f289bf4c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f28940b6430 con 0x7f289406c5f0 2024-04-24T23:10:18.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.300+0000 7f289bf4c640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f2894156f80 con 0x7f28940b44d0 2024-04-24T23:10:18.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.300+0000 7f289bf4c640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f28940b6150 con 0x7f28940af890 2024-04-24T23:10:18.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.300+0000 7f2899ce8640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f289406c5f0 0x7f289406c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:18.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.300+0000 7f2899ce8640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f289406c5f0 0x7f289406c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35092/0 (socket says 172.21.15.32:35092) 2024-04-24T23:10:18.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.300+0000 7f2899ce8640 1 -- 172.21.15.32:0/1349014274 learned_addr learned my addr 172.21.15.32:0/1349014274 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:18.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.300+0000 7f289a4e9640 1 --2- 172.21.15.32:0/1349014274 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f28940af890 0x7f28940cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:18.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.300+0000 7f28994e7640 1 --2- 172.21.15.32:0/1349014274 >> v2:172.21.15.121:3300/0 conn(0x7f28940b44d0 0x7f28940b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:18.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.300+0000 7f2899ce8640 1 -- 172.21.15.32:0/1349014274 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f28940af890 msgr2=0x7f28940cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:18.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.300+0000 7f2899ce8640 1 --2- 172.21.15.32:0/1349014274 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f28940af890 0x7f28940cef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.300+0000 7f2899ce8640 1 -- 172.21.15.32:0/1349014274 >> v2:172.21.15.121:3300/0 conn(0x7f28940b44d0 msgr2=0x7f28940b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:18.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.300+0000 7f2899ce8640 1 --2- 172.21.15.32:0/1349014274 >> v2:172.21.15.121:3300/0 conn(0x7f28940b44d0 0x7f28940b5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.300+0000 7f2899ce8640 1 -- 172.21.15.32:0/1349014274 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28940b39f0 con 0x7f289406c5f0 2024-04-24T23:10:18.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.300+0000 7f28994e7640 1 --2- 172.21.15.32:0/1349014274 >> v2:172.21.15.121:3300/0 conn(0x7f28940b44d0 0x7f28940b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:10:18.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.301+0000 7f2899ce8640 1 --2- 172.21.15.32:0/1349014274 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f289406c5f0 0x7f289406c9d0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f2894073080 tx=0x7f289002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f37885580d165f50 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:18.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.301+0000 7f2898ce6640 1 -- 172.21.15.32:0/1349014274 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2890037070 con 0x7f289406c5f0 2024-04-24T23:10:18.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.301+0000 7f2898ce6640 1 -- 172.21.15.32:0/1349014274 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2890032440 con 0x7f289406c5f0 2024-04-24T23:10:18.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.301+0000 7f2898ce6640 1 -- 172.21.15.32:0/1349014274 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2890032810 con 0x7f289406c5f0 2024-04-24T23:10:18.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.301+0000 7f289bf4c640 1 -- 172.21.15.32:0/1349014274 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f289406c5f0 msgr2=0x7f289406c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:18.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.301+0000 7f289bf4c640 1 --2- 172.21.15.32:0/1349014274 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f289406c5f0 0x7f289406c9d0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f2894073080 tx=0x7f289002a140 comp rx=0 tx=0).stop 2024-04-24T23:10:18.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.302+0000 7f289bf4c640 1 -- 172.21.15.32:0/1349014274 shutdown_connections 2024-04-24T23:10:18.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.302+0000 7f289bf4c640 1 --2- 172.21.15.32:0/1349014274 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f28940af890 0x7f28940cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.302+0000 7f289bf4c640 1 --2- 172.21.15.32:0/1349014274 >> v2:172.21.15.121:3300/0 conn(0x7f28940b44d0 0x7f28940b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.302+0000 7f289bf4c640 1 --2- 172.21.15.32:0/1349014274 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f289406c5f0 0x7f289406c9d0 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.302+0000 7f289bf4c640 1 -- 172.21.15.32:0/1349014274 >> 172.21.15.32:0/1349014274 conn(0x7f289406b110 msgr2=0x7f289406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:18.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.302+0000 7f289bf4c640 1 -- 172.21.15.32:0/1349014274 shutdown_connections 2024-04-24T23:10:18.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.302+0000 7f289bf4c640 1 -- 172.21.15.32:0/1349014274 wait complete. 2024-04-24T23:10:18.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.303+0000 7f289bf4c640 1 Processor -- start 2024-04-24T23:10:18.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.303+0000 7f289bf4c640 1 -- start start 2024-04-24T23:10:18.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.304+0000 7f289bf4c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f289406c5f0 0x7f28940aef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:18.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.304+0000 7f289bf4c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28940af890 0x7f28940b12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:18.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.304+0000 7f289bf4c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28940b44d0 0x7f28940b1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:18.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.304+0000 7f289bf4c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f289406cf10 con 0x7f28940af890 2024-04-24T23:10:18.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.304+0000 7f289bf4c640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f28940b3fb0 con 0x7f28940b44d0 2024-04-24T23:10:18.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.304+0000 7f289bf4c640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f28940b5dd0 con 0x7f289406c5f0 2024-04-24T23:10:18.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.304+0000 7f28994e7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28940af890 0x7f28940b12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:18.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.304+0000 7f28994e7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28940af890 0x7f28940b12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35098/0 (socket says 172.21.15.32:35098) 2024-04-24T23:10:18.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.304+0000 7f28994e7640 1 -- 172.21.15.32:0/1326794645 learned_addr learned my addr 172.21.15.32:0/1326794645 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:18.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.304+0000 7f2899ce8640 1 --2- 172.21.15.32:0/1326794645 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f289406c5f0 0x7f28940aef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:18.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.304+0000 7f289a4e9640 1 --2- 172.21.15.32:0/1326794645 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28940b44d0 0x7f28940b1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:18.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.304+0000 7f28994e7640 1 -- 172.21.15.32:0/1326794645 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f289406c5f0 msgr2=0x7f28940aef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:18.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.304+0000 7f28994e7640 1 --2- 172.21.15.32:0/1326794645 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f289406c5f0 0x7f28940aef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.304+0000 7f28994e7640 1 -- 172.21.15.32:0/1326794645 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28940b44d0 msgr2=0x7f28940b1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:18.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.304+0000 7f28994e7640 1 --2- 172.21.15.32:0/1326794645 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28940b44d0 0x7f28940b1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.304+0000 7f28994e7640 1 -- 172.21.15.32:0/1326794645 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28940aa7d0 con 0x7f28940af890 2024-04-24T23:10:18.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.304+0000 7f289a4e9640 1 --2- 172.21.15.32:0/1326794645 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28940b44d0 0x7f28940b1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:10:18.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.304+0000 7f28994e7640 1 --2- 172.21.15.32:0/1326794645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28940af890 0x7f28940b12c0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f288400e810 tx=0x7f288400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:18.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.304+0000 7f288a7fc640 1 -- 172.21.15.32:0/1326794645 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2884017070 con 0x7f28940af890 2024-04-24T23:10:18.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.304+0000 7f288a7fc640 1 -- 172.21.15.32:0/1326794645 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f288400c6d0 con 0x7f28940af890 2024-04-24T23:10:18.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.304+0000 7f289bf4c640 1 -- 172.21.15.32:0/1326794645 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28940acdf0 con 0x7f28940af890 2024-04-24T23:10:18.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.304+0000 7f288a7fc640 1 -- 172.21.15.32:0/1326794645 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288400f040 con 0x7f28940af890 2024-04-24T23:10:18.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.304+0000 7f289bf4c640 1 -- 172.21.15.32:0/1326794645 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28941e6c60 con 0x7f28940af890 2024-04-24T23:10:18.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.305+0000 7f289bf4c640 1 -- 172.21.15.32:0/1326794645 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2894068430 con 0x7f28940af890 2024-04-24T23:10:18.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.306+0000 7f288a7fc640 1 -- 172.21.15.32:0/1326794645 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f288401a070 con 0x7f28940af890 2024-04-24T23:10:18.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.306+0000 7f288a7fc640 1 --2- 172.21.15.32:0/1326794645 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f287806be30 0x7f287806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:18.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.307+0000 7f288a7fc640 1 -- 172.21.15.32:0/1326794645 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 5767+0+0 (secure 0 0 0) 0x7f288408aa30 con 0x7f28940af890 2024-04-24T23:10:18.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.307+0000 7f2899ce8640 1 --2- 172.21.15.32:0/1326794645 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f287806be30 0x7f287806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:18.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.307+0000 7f288a7fc640 1 -- 172.21.15.32:0/1326794645 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=174}) -- 0x7f2878073a20 con 0x7f28940af890 2024-04-24T23:10:18.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.307+0000 7f2899ce8640 1 --2- 172.21.15.32:0/1326794645 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f287806be30 0x7f287806e2f0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f2890034000 tx=0x7f289002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:18.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.311+0000 7f288a7fc640 1 -- 172.21.15.32:0/1326794645 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f28940aa7d0 con 0x7f28940af890 2024-04-24T23:10:18.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.492+0000 7f289bf4c640 1 -- 172.21.15.32:0/1326794645 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f28940b5280 con 0x7f28940af890 2024-04-24T23:10:18.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.493+0000 7f288a7fc640 1 -- 172.21.15.32:0/1326794645 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v173) ==== 74+0+16756 (secure 0 0 0) 0x7f28840593a0 con 0x7f28940af890 2024-04-24T23:10:18.495 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:10:18.495 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":173,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:09:53.858501+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:08:59.076360+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-24T23:08:59.745667+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.1d","mappings":[{"from":2,"to":5}]},{"pgid":"21.39","mappings":[{"from":2,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:10:18.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.495+0000 7f289bf4c640 1 -- 172.21.15.32:0/1326794645 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f287806be30 msgr2=0x7f287806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:18.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.495+0000 7f289bf4c640 1 --2- 172.21.15.32:0/1326794645 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f287806be30 0x7f287806e2f0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f2890034000 tx=0x7f289002e000 comp rx=0 tx=0).stop 2024-04-24T23:10:18.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.495+0000 7f289bf4c640 1 -- 172.21.15.32:0/1326794645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28940af890 msgr2=0x7f28940b12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:18.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.495+0000 7f289bf4c640 1 --2- 172.21.15.32:0/1326794645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28940af890 0x7f28940b12c0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f288400e810 tx=0x7f288400ecd0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.495+0000 7f289bf4c640 1 -- 172.21.15.32:0/1326794645 shutdown_connections 2024-04-24T23:10:18.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.495+0000 7f289bf4c640 1 --2- 172.21.15.32:0/1326794645 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f287806be30 0x7f287806e2f0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.495+0000 7f289bf4c640 1 --2- 172.21.15.32:0/1326794645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28940af890 0x7f28940b12c0 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.495+0000 7f289bf4c640 1 --2- 172.21.15.32:0/1326794645 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28940b44d0 0x7f28940b1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.495+0000 7f289bf4c640 1 --2- 172.21.15.32:0/1326794645 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f289406c5f0 0x7f28940aef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.495+0000 7f289bf4c640 1 -- 172.21.15.32:0/1326794645 >> 172.21.15.32:0/1326794645 conn(0x7f289406b110 msgr2=0x7f2894071750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:18.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.495+0000 7f289bf4c640 1 -- 172.21.15.32:0/1326794645 shutdown_connections 2024-04-24T23:10:18.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.495+0000 7f289bf4c640 1 -- 172.21.15.32:0/1326794645 wait complete. 2024-04-24T23:10:18.508 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-24T23:10:18.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.607+0000 7f8a9c84d640 1 Processor -- start 2024-04-24T23:10:18.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.607+0000 7f8a9c84d640 1 -- start start 2024-04-24T23:10:18.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.608+0000 7f8a9c84d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8a9806c5f0 0x7f8a9806c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:18.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.608+0000 7f8a9c84d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a980b6430 con 0x7f8a980b44d0 2024-04-24T23:10:18.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.608+0000 7f8a9c84d640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a98156f80 con 0x7f8a980af890 2024-04-24T23:10:18.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.608+0000 7f8a9c84d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f8a980b6150 con 0x7f8a9806c5f0 2024-04-24T23:10:18.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.608+0000 7f8a95d9b640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8a980b44d0 0x7f8a980b5890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47992/0 (socket says 172.21.15.32:47992) 2024-04-24T23:10:18.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.608+0000 7f8a95d9b640 1 -- 172.21.15.32:0/1081632211 learned_addr learned my addr 172.21.15.32:0/1081632211 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:10:18.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.608+0000 7f8a9659c640 1 --2- 172.21.15.32:0/1081632211 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8a9806c5f0 0x7f8a9806c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:18.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.608+0000 7f8a9559a640 1 -- 172.21.15.32:0/1081632211 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 550380452 0 0) 0x7f8a980b6430 con 0x7f8a980b44d0 2024-04-24T23:10:18.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.608+0000 7f8a9559a640 1 -- 172.21.15.32:0/1081632211 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8a80003520 con 0x7f8a980b44d0 2024-04-24T23:10:18.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.609+0000 7f8a9559a640 1 -- 172.21.15.32:0/1081632211 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 55649885 0 0) 0x7f8a80003520 con 0x7f8a980b44d0 2024-04-24T23:10:18.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.609+0000 7f8a9559a640 1 -- 172.21.15.32:0/1081632211 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8a9806c5f0 msgr2=0x7f8a9806c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:18.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.609+0000 7f8a9559a640 1 --2- 172.21.15.32:0/1081632211 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8a9806c5f0 0x7f8a9806c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.609+0000 7f8a9559a640 1 -- 172.21.15.32:0/1081632211 >> v1:172.21.15.121:6789/0 conn(0x7f8a980af890 legacy=0x7f8a9815cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:18.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.609+0000 7f8a9559a640 1 -- 172.21.15.32:0/1081632211 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a980b39f0 con 0x7f8a980b44d0 2024-04-24T23:10:18.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.609+0000 7f8a9559a640 1 -- 172.21.15.32:0/1081632211 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f8a8c001670 con 0x7f8a980b44d0 2024-04-24T23:10:18.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.609+0000 7f8a9559a640 1 -- 172.21.15.32:0/1081632211 >> v1:172.21.15.32:6789/0 conn(0x7f8a980b44d0 legacy=0x7f8a980b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:18.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.609+0000 7f8a9559a640 1 --2- 172.21.15.32:0/1081632211 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8a800039e0 0x7f8a80003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:18.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.609+0000 7f8a9559a640 1 --2- 172.21.15.32:0/1081632211 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a80004800 0x7f8a80004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:18.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.609+0000 7f8a9559a640 1 --2- 172.21.15.32:0/1081632211 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8a9806c5f0 0x7f8a80005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:18.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.609+0000 7f8a9559a640 1 -- 172.21.15.32:0/1081632211 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8a980b6150 con 0x7f8a80004800 2024-04-24T23:10:18.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.609+0000 7f8a9559a640 1 -- 172.21.15.32:0/1081632211 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8000baf0 con 0x7f8a800039e0 2024-04-24T23:10:18.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.609+0000 7f8a9559a640 1 -- 172.21.15.32:0/1081632211 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f8a8000bcf0 con 0x7f8a9806c5f0 2024-04-24T23:10:18.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.609+0000 7f8a95d9b640 1 --2- 172.21.15.32:0/1081632211 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a80004800 0x7f8a80004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:18.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.609+0000 7f8a9659c640 1 --2- 172.21.15.32:0/1081632211 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8a800039e0 0x7f8a80003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:18.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.609+0000 7f8a96d9d640 1 --2- 172.21.15.32:0/1081632211 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8a9806c5f0 0x7f8a80005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:18.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.609+0000 7f8a95d9b640 1 -- 172.21.15.32:0/1081632211 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8a9806c5f0 msgr2=0x7f8a80005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:18.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.609+0000 7f8a95d9b640 1 --2- 172.21.15.32:0/1081632211 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8a9806c5f0 0x7f8a80005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.609+0000 7f8a95d9b640 1 -- 172.21.15.32:0/1081632211 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8a800039e0 msgr2=0x7f8a80003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:18.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.609+0000 7f8a95d9b640 1 --2- 172.21.15.32:0/1081632211 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8a800039e0 0x7f8a80003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.609+0000 7f8a95d9b640 1 -- 172.21.15.32:0/1081632211 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a80003520 con 0x7f8a80004800 2024-04-24T23:10:18.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.609+0000 7f8a96d9d640 1 --2- 172.21.15.32:0/1081632211 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8a9806c5f0 0x7f8a80005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:10:18.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.610+0000 7f8a95d9b640 1 --2- 172.21.15.32:0/1081632211 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a80004800 0x7f8a80004c80 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c00c240 tx=0x7f8a80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=881c1347b5fd8053 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:18.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.610+0000 7f8a9559a640 1 -- 172.21.15.32:0/1081632211 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c007cf0 con 0x7f8a80004800 2024-04-24T23:10:18.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.610+0000 7f8a9559a640 1 -- 172.21.15.32:0/1081632211 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a8c0058d0 con 0x7f8a80004800 2024-04-24T23:10:18.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.610+0000 7f8a9559a640 1 -- 172.21.15.32:0/1081632211 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c005ce0 con 0x7f8a80004800 2024-04-24T23:10:18.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.610+0000 7f8a9c84d640 1 -- 172.21.15.32:0/1081632211 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a80004800 msgr2=0x7f8a80004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:18.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.610+0000 7f8a9c84d640 1 --2- 172.21.15.32:0/1081632211 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a80004800 0x7f8a80004c80 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c00c240 tx=0x7f8a80005830 comp rx=0 tx=0).stop 2024-04-24T23:10:18.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.610+0000 7f8a96d9d640 1 -- 172.21.15.32:0/1081632211 reap_dead start 2024-04-24T23:10:18.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.611+0000 7f8a9c84d640 1 -- 172.21.15.32:0/1081632211 shutdown_connections 2024-04-24T23:10:18.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.611+0000 7f8a9c84d640 1 -- 172.21.15.32:0/1081632211 >> 172.21.15.32:0/1081632211 conn(0x7f8a9806b110 msgr2=0x7f8a9806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:18.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.611+0000 7f8a9c84d640 1 -- 172.21.15.32:0/1081632211 shutdown_connections 2024-04-24T23:10:18.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.611+0000 7f8a9c84d640 1 -- 172.21.15.32:0/1081632211 wait complete. 2024-04-24T23:10:18.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.612+0000 7f8a9c84d640 1 Processor -- start 2024-04-24T23:10:18.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.612+0000 7f8a9c84d640 1 -- start start 2024-04-24T23:10:18.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.613+0000 7f8a9c84d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a80004800 0x7f8a98140ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:18.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.613+0000 7f8a9c84d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8a98154f10 0x7f8a981505c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:18.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.613+0000 7f8a9c84d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8a98143280 0x7f8a98143700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:18.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.613+0000 7f8a9c84d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8a9806c9d0 con 0x7f8a80004800 2024-04-24T23:10:18.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.613+0000 7f8a9c84d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8a980b3fb0 con 0x7f8a98154f10 2024-04-24T23:10:18.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.613+0000 7f8a9c84d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f8a9806cb50 con 0x7f8a98143280 2024-04-24T23:10:18.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.613+0000 7f8a9659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a80004800 0x7f8a98140ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:18.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.613+0000 7f8a9659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a80004800 0x7f8a98140ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35104/0 (socket says 172.21.15.32:35104) 2024-04-24T23:10:18.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.613+0000 7f8a9659c640 1 -- 172.21.15.32:0/1574380266 learned_addr learned my addr 172.21.15.32:0/1574380266 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:18.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.613+0000 7f8a95d9b640 1 --2- 172.21.15.32:0/1574380266 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8a98154f10 0x7f8a981505c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:18.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.613+0000 7f8a96d9d640 1 --2- 172.21.15.32:0/1574380266 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8a98143280 0x7f8a98143700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:18.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.613+0000 7f8a9659c640 1 -- 172.21.15.32:0/1574380266 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8a98143280 msgr2=0x7f8a98143700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:18.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.613+0000 7f8a9659c640 1 --2- 172.21.15.32:0/1574380266 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8a98143280 0x7f8a98143700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.613+0000 7f8a9659c640 1 -- 172.21.15.32:0/1574380266 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8a98154f10 msgr2=0x7f8a981505c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:18.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.613+0000 7f8a9659c640 1 --2- 172.21.15.32:0/1574380266 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8a98154f10 0x7f8a981505c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.613+0000 7f8a9659c640 1 -- 172.21.15.32:0/1574380266 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a98153440 con 0x7f8a80004800 2024-04-24T23:10:18.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.613+0000 7f8a96d9d640 1 --2- 172.21.15.32:0/1574380266 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8a98143280 0x7f8a98143700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:10:18.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.613+0000 7f8a95d9b640 1 --2- 172.21.15.32:0/1574380266 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8a98154f10 0x7f8a981505c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:10:18.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.613+0000 7f8a9659c640 1 --2- 172.21.15.32:0/1574380266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a80004800 0x7f8a98140ad0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f8a90011a00 tx=0x7f8a90011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:18.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.613+0000 7f8a86ffd640 1 -- 172.21.15.32:0/1574380266 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9000fb50 con 0x7f8a80004800 2024-04-24T23:10:18.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.613+0000 7f8a86ffd640 1 -- 172.21.15.32:0/1574380266 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a9000fcf0 con 0x7f8a80004800 2024-04-24T23:10:18.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.613+0000 7f8a9c84d640 1 -- 172.21.15.32:0/1574380266 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a98153720 con 0x7f8a80004800 2024-04-24T23:10:18.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.613+0000 7f8a86ffd640 1 -- 172.21.15.32:0/1574380266 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9000fea0 con 0x7f8a80004800 2024-04-24T23:10:18.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.613+0000 7f8a9c84d640 1 -- 172.21.15.32:0/1574380266 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a98143fc0 con 0x7f8a80004800 2024-04-24T23:10:18.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.615+0000 7f8a9c84d640 1 -- 172.21.15.32:0/1574380266 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a98068430 con 0x7f8a80004800 2024-04-24T23:10:18.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.615+0000 7f8a86ffd640 1 -- 172.21.15.32:0/1574380266 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8a90012040 con 0x7f8a80004800 2024-04-24T23:10:18.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.615+0000 7f8a86ffd640 1 --2- 172.21.15.32:0/1574380266 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8a7806be60 0x7f8a7806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:18.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.615+0000 7f8a86ffd640 1 -- 172.21.15.32:0/1574380266 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 5767+0+0 (secure 0 0 0) 0x7f8a9008b590 con 0x7f8a80004800 2024-04-24T23:10:18.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.615+0000 7f8a86ffd640 1 -- 172.21.15.32:0/1574380266 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=174}) -- 0x7f8a78073a50 con 0x7f8a80004800 2024-04-24T23:10:18.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.616+0000 7f8a95d9b640 1 --2- 172.21.15.32:0/1574380266 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8a7806be60 0x7f8a7806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:18.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.616+0000 7f8a95d9b640 1 --2- 172.21.15.32:0/1574380266 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8a7806be60 0x7f8a7806e320 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c033000 tx=0x7f8a8c009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:18.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.620+0000 7f8a86ffd640 1 -- 172.21.15.32:0/1574380266 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8a90019030 con 0x7f8a80004800 2024-04-24T23:10:18.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.846+0000 7f8a9c84d640 1 -- 172.21.15.32:0/1574380266 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f8a980b5670 con 0x7f8a80004800 2024-04-24T23:10:18.848 INFO:tasks.ceph.mon.a.smithi032.stderr:2024-04-24T23:10:18.847+0000 7f3b809c6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T23:10:18.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.848+0000 7f8a86ffd640 1 -- 172.21.15.32:0/1574380266 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v74) ==== 161+0+0 (secure 0 0 0) 0x7f8a900600d0 con 0x7f8a80004800 2024-04-24T23:10:18.850 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-24T23:10:18.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.849+0000 7f8a86ffd640 1 -- 172.21.15.32:0/1574380266 <== mon.0 v2:172.21.15.32:3300/0 8 ==== osd_map(174..174 src has 1..174) ==== 374+0+0 (secure 0 0 0) 0x7f8a900bf2d0 con 0x7f8a80004800 2024-04-24T23:10:18.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.849+0000 7f8a86ffd640 1 -- 172.21.15.32:0/1574380266 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=175}) -- 0x7f8a78074490 con 0x7f8a80004800 2024-04-24T23:10:18.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.852+0000 7f8a9c84d640 1 -- 172.21.15.32:0/1574380266 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8a7806be60 msgr2=0x7f8a7806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:18.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.852+0000 7f8a9c84d640 1 --2- 172.21.15.32:0/1574380266 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8a7806be60 0x7f8a7806e320 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c033000 tx=0x7f8a8c009030 comp rx=0 tx=0).stop 2024-04-24T23:10:18.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.853+0000 7f8a9c84d640 1 -- 172.21.15.32:0/1574380266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a80004800 msgr2=0x7f8a98140ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:18.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.853+0000 7f8a9c84d640 1 --2- 172.21.15.32:0/1574380266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a80004800 0x7f8a98140ad0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f8a90011a00 tx=0x7f8a90011ec0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.853+0000 7f8a9c84d640 1 -- 172.21.15.32:0/1574380266 shutdown_connections 2024-04-24T23:10:18.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.853+0000 7f8a9c84d640 1 --2- 172.21.15.32:0/1574380266 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f8a7806be60 0x7f8a7806e320 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.853+0000 7f8a9c84d640 1 --2- 172.21.15.32:0/1574380266 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8a98143280 0x7f8a98143700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.853+0000 7f8a9c84d640 1 --2- 172.21.15.32:0/1574380266 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8a98154f10 0x7f8a981505c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.853+0000 7f8a9c84d640 1 --2- 172.21.15.32:0/1574380266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a80004800 0x7f8a98140ad0 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.853+0000 7f8a9c84d640 1 -- 172.21.15.32:0/1574380266 >> 172.21.15.32:0/1574380266 conn(0x7f8a9806b110 msgr2=0x7f8a980b12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:18.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.853+0000 7f8a9c84d640 1 -- 172.21.15.32:0/1574380266 shutdown_connections 2024-04-24T23:10:18.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.853+0000 7f8a9c84d640 1 -- 172.21.15.32:0/1574380266 wait complete. 2024-04-24T23:10:18.866 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-24T23:10:18.890 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2024-04-24T23:10:18.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.969+0000 7f99b85e3640 1 Processor -- start 2024-04-24T23:10:18.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.969+0000 7f99b85e3640 1 -- start start 2024-04-24T23:10:18.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.970+0000 7f99b85e3640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f99b00b5560 0x7f99b015e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:18.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.970+0000 7f99b85e3640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99b00cd3c0 0x7f99b015ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:18.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.970+0000 7f99b85e3640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99b00cd910 con 0x7f99b00b5940 2024-04-24T23:10:18.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.970+0000 7f99b85e3640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f99b00cdbf0 con 0x7f99b00b5560 2024-04-24T23:10:18.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.970+0000 7f99b85e3640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f99b00cded0 con 0x7f99b00cd3c0 2024-04-24T23:10:18.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.970+0000 7f99b6b80640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f99b00b5940 0x7f99b00b5d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47996/0 (socket says 172.21.15.32:47996) 2024-04-24T23:10:18.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.970+0000 7f99b6b80640 1 -- 172.21.15.32:0/2722774497 learned_addr learned my addr 172.21.15.32:0/2722774497 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:10:18.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.970+0000 7f99b637f640 1 --2- 172.21.15.32:0/2722774497 >> v2:172.21.15.121:3300/0 conn(0x7f99b00b5560 0x7f99b015e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:18.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.970+0000 7f99b5b7e640 1 --2- 172.21.15.32:0/2722774497 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99b00cd3c0 0x7f99b015ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:18.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.970+0000 7f99b537d640 1 -- 172.21.15.32:0/2722774497 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1871163526 0 0) 0x7f99b00cd910 con 0x7f99b00b5940 2024-04-24T23:10:18.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.970+0000 7f99b537d640 1 -- 172.21.15.32:0/2722774497 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f999c003520 con 0x7f99b00b5940 2024-04-24T23:10:18.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.970+0000 7f99b537d640 1 -- 172.21.15.32:0/2722774497 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3300763102 0 0) 0x7f999c003520 con 0x7f99b00b5940 2024-04-24T23:10:18.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.970+0000 7f99b637f640 1 -- 172.21.15.32:0/2722774497 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99b00cd3c0 msgr2=0x7f99b015ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:18.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.970+0000 7f99b637f640 1 --2- 172.21.15.32:0/2722774497 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99b00cd3c0 0x7f99b015ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.970+0000 7f99b637f640 1 -- 172.21.15.32:0/2722774497 >> v1:172.21.15.32:6789/0 conn(0x7f99b00b5940 legacy=0x7f99b00b5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:18.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.970+0000 7f99b637f640 1 -- 172.21.15.32:0/2722774497 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99b00ce1b0 con 0x7f99b00b5560 2024-04-24T23:10:18.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.971+0000 7f99b637f640 1 --2- 172.21.15.32:0/2722774497 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99b00b5560 0x7f99b015e6c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f99ac00a840 tx=0x7f99ac02a3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85664fbb6a62116e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:18.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.971+0000 7f99b537d640 1 -- 172.21.15.32:0/2722774497 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ac037070 con 0x7f99b00b5560 2024-04-24T23:10:18.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.971+0000 7f99b537d640 1 -- 172.21.15.32:0/2722774497 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99ac02ac40 con 0x7f99b00b5560 2024-04-24T23:10:18.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.971+0000 7f99b537d640 1 -- 172.21.15.32:0/2722774497 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a80019f0 con 0x7f99b00b5560 2024-04-24T23:10:18.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.972+0000 7f99b85e3640 1 -- 172.21.15.32:0/2722774497 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99b00b5560 msgr2=0x7f99b015e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:18.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.972+0000 7f99b85e3640 1 --2- 172.21.15.32:0/2722774497 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99b00b5560 0x7f99b015e6c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f99ac00a840 tx=0x7f99ac02a3c0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.972+0000 7f99b85e3640 1 -- 172.21.15.32:0/2722774497 shutdown_connections 2024-04-24T23:10:18.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.972+0000 7f99b85e3640 1 --2- 172.21.15.32:0/2722774497 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99b00cd3c0 0x7f99b015ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.972+0000 7f99b85e3640 1 --2- 172.21.15.32:0/2722774497 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99b00b5560 0x7f99b015e6c0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.972+0000 7f99b85e3640 1 -- 172.21.15.32:0/2722774497 >> 172.21.15.32:0/2722774497 conn(0x7f99b006b110 msgr2=0x7f99b006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:18.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.972+0000 7f99b85e3640 1 -- 172.21.15.32:0/2722774497 shutdown_connections 2024-04-24T23:10:18.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.972+0000 7f99b85e3640 1 -- 172.21.15.32:0/2722774497 wait complete. 2024-04-24T23:10:18.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.973+0000 7f99b85e3640 1 Processor -- start 2024-04-24T23:10:18.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.973+0000 7f99b85e3640 1 -- start start 2024-04-24T23:10:18.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.974+0000 7f99b85e3640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99b00b5560 0x7f99b0070270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:18.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.974+0000 7f99b85e3640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99b00b5940 0x7f99b0157070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:18.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.974+0000 7f99b85e3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99b00cd3c0 0x7f99b0142680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:18.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.974+0000 7f99b85e3640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f99b006c770 con 0x7f99b00cd3c0 2024-04-24T23:10:18.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.974+0000 7f99b85e3640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f99b006c5f0 con 0x7f99b00b5940 2024-04-24T23:10:18.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.974+0000 7f99b85e3640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f99b006c8f0 con 0x7f99b00b5560 2024-04-24T23:10:18.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.974+0000 7f99b6b80640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99b00cd3c0 0x7f99b0142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:18.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.974+0000 7f99b6b80640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99b00cd3c0 0x7f99b0142680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35132/0 (socket says 172.21.15.32:35132) 2024-04-24T23:10:18.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.974+0000 7f99b6b80640 1 -- 172.21.15.32:0/1676899707 learned_addr learned my addr 172.21.15.32:0/1676899707 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:18.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.974+0000 7f99b637f640 1 --2- 172.21.15.32:0/1676899707 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99b00b5560 0x7f99b0070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:18.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.974+0000 7f99b5b7e640 1 --2- 172.21.15.32:0/1676899707 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99b00b5940 0x7f99b0157070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:18.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.974+0000 7f99b6b80640 1 -- 172.21.15.32:0/1676899707 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99b00b5560 msgr2=0x7f99b0070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:18.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.974+0000 7f99b6b80640 1 --2- 172.21.15.32:0/1676899707 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99b00b5560 0x7f99b0070270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.974+0000 7f99b6b80640 1 -- 172.21.15.32:0/1676899707 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99b00b5940 msgr2=0x7f99b0157070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:18.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.974+0000 7f99b6b80640 1 --2- 172.21.15.32:0/1676899707 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99b00b5940 0x7f99b0157070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:18.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.974+0000 7f99b6b80640 1 -- 172.21.15.32:0/1676899707 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99b00709e0 con 0x7f99b00cd3c0 2024-04-24T23:10:18.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.974+0000 7f99b5b7e640 1 --2- 172.21.15.32:0/1676899707 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99b00b5940 0x7f99b0157070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:10:18.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.974+0000 7f99b637f640 1 --2- 172.21.15.32:0/1676899707 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99b00b5560 0x7f99b0070270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:10:18.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.975+0000 7f99b6b80640 1 --2- 172.21.15.32:0/1676899707 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99b00cd3c0 0x7f99b0142680 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f99a8001b60 tx=0x7f99a800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:18.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.975+0000 7f99a6ffd640 1 -- 172.21.15.32:0/1676899707 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a8017070 con 0x7f99b00cd3c0 2024-04-24T23:10:18.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.975+0000 7f99a6ffd640 1 -- 172.21.15.32:0/1676899707 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99a8012400 con 0x7f99b00cd3c0 2024-04-24T23:10:18.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.975+0000 7f99b85e3640 1 -- 172.21.15.32:0/1676899707 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99b0070cc0 con 0x7f99b00cd3c0 2024-04-24T23:10:18.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.975+0000 7f99a6ffd640 1 -- 172.21.15.32:0/1676899707 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a80127b0 con 0x7f99b00cd3c0 2024-04-24T23:10:18.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.975+0000 7f99b85e3640 1 -- 172.21.15.32:0/1676899707 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99b0071280 con 0x7f99b00cd3c0 2024-04-24T23:10:18.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.976+0000 7f99b85e3640 1 -- 172.21.15.32:0/1676899707 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99b0068430 con 0x7f99b00cd3c0 2024-04-24T23:10:18.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.976+0000 7f99a6ffd640 1 -- 172.21.15.32:0/1676899707 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f99a801a070 con 0x7f99b00cd3c0 2024-04-24T23:10:18.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.977+0000 7f99a6ffd640 1 --2- 172.21.15.32:0/1676899707 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f999406be60 0x7f999406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:18.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.977+0000 7f99a6ffd640 1 -- 172.21.15.32:0/1676899707 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(174..174 src has 1..174) ==== 5853+0+0 (secure 0 0 0) 0x7f99a808ad30 con 0x7f99b00cd3c0 2024-04-24T23:10:18.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.977+0000 7f99a6ffd640 1 -- 172.21.15.32:0/1676899707 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=175}) -- 0x7f9994073b60 con 0x7f99b00cd3c0 2024-04-24T23:10:18.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.977+0000 7f99b637f640 1 --2- 172.21.15.32:0/1676899707 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f999406be60 0x7f999406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:18.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.977+0000 7f99b637f640 1 --2- 172.21.15.32:0/1676899707 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f999406be60 0x7f999406e320 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f99ac034000 tx=0x7f99ac008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:18.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:18.981+0000 7f99a6ffd640 1 -- 172.21.15.32:0/1676899707 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f99a8059000 con 0x7f99b00cd3c0 2024-04-24T23:10:19.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:19.207+0000 7f99b85e3640 1 -- 172.21.15.32:0/1676899707 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f99b01585a0 con 0x7f99b00cd3c0 2024-04-24T23:10:19.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:19.850+0000 7f99a6ffd640 1 -- 172.21.15.32:0/1676899707 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v75) ==== 104+0+0 (secure 0 0 0) 0x7f99a805d300 con 0x7f99b00cd3c0 2024-04-24T23:10:19.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:19.853+0000 7f99b85e3640 1 -- 172.21.15.32:0/1676899707 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f999406be60 msgr2=0x7f999406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:19.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:19.853+0000 7f99b85e3640 1 --2- 172.21.15.32:0/1676899707 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f999406be60 0x7f999406e320 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f99ac034000 tx=0x7f99ac008040 comp rx=0 tx=0).stop 2024-04-24T23:10:19.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:19.853+0000 7f99b85e3640 1 -- 172.21.15.32:0/1676899707 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99b00cd3c0 msgr2=0x7f99b0142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:19.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:19.853+0000 7f99b85e3640 1 --2- 172.21.15.32:0/1676899707 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99b00cd3c0 0x7f99b0142680 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f99a8001b60 tx=0x7f99a800a430 comp rx=0 tx=0).stop 2024-04-24T23:10:19.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:19.853+0000 7f99b85e3640 1 -- 172.21.15.32:0/1676899707 shutdown_connections 2024-04-24T23:10:19.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:19.853+0000 7f99b85e3640 1 --2- 172.21.15.32:0/1676899707 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f999406be60 0x7f999406e320 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:19.853+0000 7f99b85e3640 1 --2- 172.21.15.32:0/1676899707 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99b00cd3c0 0x7f99b0142680 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:19.853+0000 7f99b85e3640 1 --2- 172.21.15.32:0/1676899707 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99b00b5940 0x7f99b0157070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:19.853+0000 7f99b85e3640 1 --2- 172.21.15.32:0/1676899707 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99b00b5560 0x7f99b0070270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:19.853+0000 7f99b85e3640 1 -- 172.21.15.32:0/1676899707 >> 172.21.15.32:0/1676899707 conn(0x7f99b006b110 msgr2=0x7f99b0153590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:19.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:19.853+0000 7f99b85e3640 1 -- 172.21.15.32:0/1676899707 shutdown_connections 2024-04-24T23:10:19.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:19.853+0000 7f99b85e3640 1 -- 172.21.15.32:0/1676899707 wait complete. 2024-04-24T23:10:19.868 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-24T23:10:20.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.020+0000 7fe5567b7640 1 Processor -- start 2024-04-24T23:10:20.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.020+0000 7fe5567b7640 1 -- start start 2024-04-24T23:10:20.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.021+0000 7fe5567b7640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe5500b0140 0x7fe5501696f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:20.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.021+0000 7fe5567b7640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe550071620 con 0x7fe550067980 2024-04-24T23:10:20.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.021+0000 7fe5567b7640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5500af270 con 0x7fe550161c30 2024-04-24T23:10:20.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.021+0000 7fe5567b7640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fe5500af5a0 con 0x7fe5500b0140 2024-04-24T23:10:20.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.021+0000 7fe54f7fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe550067980 0x7fe5500afa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59866/0 (socket says 172.21.15.32:59866) 2024-04-24T23:10:20.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.021+0000 7fe54f7fe640 1 -- 172.21.15.32:0/772887535 learned_addr learned my addr 172.21.15.32:0/772887535 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:10:20.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.021+0000 7fe554d54640 1 --2- 172.21.15.32:0/772887535 >> v2:172.21.15.121:3300/0 conn(0x7fe5500b0140 0x7fe5501696f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:20.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.021+0000 7fe54effd640 1 -- 172.21.15.32:0/772887535 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 877290959 0 0) 0x7fe550071620 con 0x7fe550067980 2024-04-24T23:10:20.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.022+0000 7fe54effd640 1 -- 172.21.15.32:0/772887535 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe53c003520 con 0x7fe550067980 2024-04-24T23:10:20.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.022+0000 7fe54effd640 1 -- 172.21.15.32:0/772887535 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1483392460 0 0) 0x7fe53c003520 con 0x7fe550067980 2024-04-24T23:10:20.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.022+0000 7fe54effd640 1 -- 172.21.15.32:0/772887535 >> v2:172.21.15.121:3300/0 conn(0x7fe5500b0140 msgr2=0x7fe5501696f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:20.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.022+0000 7fe54effd640 1 --2- 172.21.15.32:0/772887535 >> v2:172.21.15.121:3300/0 conn(0x7fe5500b0140 0x7fe5501696f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:20.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.022+0000 7fe54effd640 1 -- 172.21.15.32:0/772887535 >> v1:172.21.15.121:6789/0 conn(0x7fe550161c30 legacy=0x7fe55006cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:20.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.022+0000 7fe54effd640 1 -- 172.21.15.32:0/772887535 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe55006d4d0 con 0x7fe550067980 2024-04-24T23:10:20.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.022+0000 7fe54effd640 1 -- 172.21.15.32:0/772887535 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7fe540001670 con 0x7fe550067980 2024-04-24T23:10:20.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.022+0000 7fe54effd640 1 -- 172.21.15.32:0/772887535 >> v1:172.21.15.32:6789/0 conn(0x7fe550067980 legacy=0x7fe5500afa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:20.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.022+0000 7fe54effd640 1 --2- 172.21.15.32:0/772887535 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe53c0039b0 0x7fe53c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:20.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.022+0000 7fe54effd640 1 --2- 172.21.15.32:0/772887535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe53c0043b0 0x7fe53c004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:20.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.022+0000 7fe54effd640 1 --2- 172.21.15.32:0/772887535 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe53c005080 0x7fe53c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:20.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.022+0000 7fe54effd640 1 -- 172.21.15.32:0/772887535 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe5500af5a0 con 0x7fe53c0043b0 2024-04-24T23:10:20.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.022+0000 7fe54effd640 1 -- 172.21.15.32:0/772887535 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe53c00bfe0 con 0x7fe53c0039b0 2024-04-24T23:10:20.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.022+0000 7fe54effd640 1 -- 172.21.15.32:0/772887535 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe53c00c1e0 con 0x7fe53c005080 2024-04-24T23:10:20.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.022+0000 7fe54f7fe640 1 --2- 172.21.15.32:0/772887535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe53c0043b0 0x7fe53c004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:20.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.022+0000 7fe54ffff640 1 --2- 172.21.15.32:0/772887535 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe53c0039b0 0x7fe53c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:20.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.022+0000 7fe554d54640 1 --2- 172.21.15.32:0/772887535 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe53c005080 0x7fe53c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:20.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.022+0000 7fe54f7fe640 1 -- 172.21.15.32:0/772887535 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe53c005080 msgr2=0x7fe53c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:20.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.022+0000 7fe54f7fe640 1 --2- 172.21.15.32:0/772887535 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe53c005080 0x7fe53c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:20.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.022+0000 7fe54f7fe640 1 -- 172.21.15.32:0/772887535 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe53c0039b0 msgr2=0x7fe53c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:20.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.022+0000 7fe54f7fe640 1 --2- 172.21.15.32:0/772887535 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe53c0039b0 0x7fe53c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:20.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.022+0000 7fe54f7fe640 1 -- 172.21.15.32:0/772887535 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe53c003520 con 0x7fe53c0043b0 2024-04-24T23:10:20.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.022+0000 7fe554d54640 1 --2- 172.21.15.32:0/772887535 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe53c005080 0x7fe53c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:10:20.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.022+0000 7fe554d54640 1 -- 172.21.15.32:0/772887535 reap_dead start 2024-04-24T23:10:20.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.023+0000 7fe54f7fe640 1 --2- 172.21.15.32:0/772887535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe53c0043b0 0x7fe53c004880 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fe54000c240 tx=0x7fe53c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=426da377ce58b1c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:20.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.023+0000 7fe54effd640 1 -- 172.21.15.32:0/772887535 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe540007d30 con 0x7fe53c0043b0 2024-04-24T23:10:20.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.023+0000 7fe54effd640 1 -- 172.21.15.32:0/772887535 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe540005af0 con 0x7fe53c0043b0 2024-04-24T23:10:20.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.023+0000 7fe54effd640 1 -- 172.21.15.32:0/772887535 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe540007070 con 0x7fe53c0043b0 2024-04-24T23:10:20.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.024+0000 7fe5567b7640 1 -- 172.21.15.32:0/772887535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe53c0043b0 msgr2=0x7fe53c004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:20.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.024+0000 7fe5567b7640 1 --2- 172.21.15.32:0/772887535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe53c0043b0 0x7fe53c004880 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fe54000c240 tx=0x7fe53c005830 comp rx=0 tx=0).stop 2024-04-24T23:10:20.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.024+0000 7fe5567b7640 1 -- 172.21.15.32:0/772887535 shutdown_connections 2024-04-24T23:10:20.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.024+0000 7fe5567b7640 1 --2- 172.21.15.32:0/772887535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe53c0043b0 0x7fe53c004880 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:20.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.024+0000 7fe5567b7640 1 -- 172.21.15.32:0/772887535 >> 172.21.15.32:0/772887535 conn(0x7fe5500d0a60 msgr2=0x7fe5500d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:20.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.024+0000 7fe5567b7640 1 -- 172.21.15.32:0/772887535 shutdown_connections 2024-04-24T23:10:20.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.024+0000 7fe5567b7640 1 -- 172.21.15.32:0/772887535 wait complete. 2024-04-24T23:10:20.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.025+0000 7fe5567b7640 1 Processor -- start 2024-04-24T23:10:20.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.025+0000 7fe5567b7640 1 -- start start 2024-04-24T23:10:20.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.026+0000 7fe5567b7640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe53c0043b0 0x7fe550152cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:20.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.026+0000 7fe5567b7640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5500ce190 0x7fe5500ce610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:20.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.026+0000 7fe5567b7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5501fc050 0x7fe5501fc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:20.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.026+0000 7fe5567b7640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe550162010 con 0x7fe5501fc050 2024-04-24T23:10:20.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.026+0000 7fe5567b7640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe550071ba0 con 0x7fe5500ce190 2024-04-24T23:10:20.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.026+0000 7fe5567b7640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe550070d60 con 0x7fe53c0043b0 2024-04-24T23:10:20.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.026+0000 7fe554d54640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5501fc050 0x7fe5501fc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:20.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.026+0000 7fe554d54640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5501fc050 0x7fe5501fc4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42178/0 (socket says 172.21.15.32:42178) 2024-04-24T23:10:20.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.026+0000 7fe554d54640 1 -- 172.21.15.32:0/222805992 learned_addr learned my addr 172.21.15.32:0/222805992 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:20.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.026+0000 7fe54ffff640 1 --2- 172.21.15.32:0/222805992 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe53c0043b0 0x7fe550152cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:20.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.026+0000 7fe54f7fe640 1 --2- 172.21.15.32:0/222805992 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5500ce190 0x7fe5500ce610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:20.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.026+0000 7fe554d54640 1 -- 172.21.15.32:0/222805992 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe53c0043b0 msgr2=0x7fe550152cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:20.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.026+0000 7fe554d54640 1 --2- 172.21.15.32:0/222805992 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe53c0043b0 0x7fe550152cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:20.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.026+0000 7fe554d54640 1 -- 172.21.15.32:0/222805992 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5500ce190 msgr2=0x7fe5500ce610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:20.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.026+0000 7fe554d54640 1 --2- 172.21.15.32:0/222805992 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5500ce190 0x7fe5500ce610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:20.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.026+0000 7fe554d54640 1 -- 172.21.15.32:0/222805992 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5501fcd60 con 0x7fe5501fc050 2024-04-24T23:10:20.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.026+0000 7fe54ffff640 1 --2- 172.21.15.32:0/222805992 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe53c0043b0 0x7fe550152cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:10:20.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.026+0000 7fe54f7fe640 1 --2- 172.21.15.32:0/222805992 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5500ce190 0x7fe5500ce610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:10:20.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.027+0000 7fe554d54640 1 --2- 172.21.15.32:0/222805992 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5501fc050 0x7fe5501fc4d0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fe550071160 tx=0x7fe548010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:20.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.027+0000 7fe54cff9640 1 -- 172.21.15.32:0/222805992 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe548010da0 con 0x7fe5501fc050 2024-04-24T23:10:20.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.027+0000 7fe54cff9640 1 -- 172.21.15.32:0/222805992 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe548003990 con 0x7fe5501fc050 2024-04-24T23:10:20.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.027+0000 7fe5567b7640 1 -- 172.21.15.32:0/222805992 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe55015f470 con 0x7fe5501fc050 2024-04-24T23:10:20.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.027+0000 7fe54cff9640 1 -- 172.21.15.32:0/222805992 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe548003dc0 con 0x7fe5501fc050 2024-04-24T23:10:20.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.027+0000 7fe5567b7640 1 -- 172.21.15.32:0/222805992 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5501fca10 con 0x7fe5501fc050 2024-04-24T23:10:20.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.028+0000 7fe5567b7640 1 -- 172.21.15.32:0/222805992 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe550068430 con 0x7fe5501fc050 2024-04-24T23:10:20.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.028+0000 7fe54cff9640 1 -- 172.21.15.32:0/222805992 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe54801e020 con 0x7fe5501fc050 2024-04-24T23:10:20.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.029+0000 7fe54cff9640 1 --2- 172.21.15.32:0/222805992 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe53006be30 0x7fe53006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:20.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.029+0000 7fe54cff9640 1 -- 172.21.15.32:0/222805992 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(174..174 src has 1..174) ==== 5853+0+0 (secure 0 0 0) 0x7fe54808b6b0 con 0x7fe5501fc050 2024-04-24T23:10:20.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.029+0000 7fe54cff9640 1 -- 172.21.15.32:0/222805992 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=175}) -- 0x7fe530073b30 con 0x7fe5501fc050 2024-04-24T23:10:20.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.029+0000 7fe54ffff640 1 --2- 172.21.15.32:0/222805992 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe53006be30 0x7fe53006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:20.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.029+0000 7fe54ffff640 1 --2- 172.21.15.32:0/222805992 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe53006be30 0x7fe53006e2f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fe5500af440 tx=0x7fe538009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:20.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.033+0000 7fe54cff9640 1 -- 172.21.15.32:0/222805992 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe548059fd0 con 0x7fe5501fc050 2024-04-24T23:10:20.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.219+0000 7fe5567b7640 1 -- 172.21.15.32:0/222805992 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe5500d18d0 con 0x7fe5501fc050 2024-04-24T23:10:20.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.928+0000 7fe54cff9640 1 -- 172.21.15.32:0/222805992 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v175) ==== 181+0+0 (secure 0 0 0) 0x7fe5480598d0 con 0x7fe5501fc050 2024-04-24T23:10:20.930 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2024-04-24T23:10:20.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.928+0000 7fe54cff9640 1 -- 172.21.15.32:0/222805992 <== mon.0 v2:172.21.15.32:3300/0 8 ==== osd_map(175..175 src has 1..175) ==== 296+0+0 (secure 0 0 0) 0x7fe5480bf2d0 con 0x7fe5501fc050 2024-04-24T23:10:20.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.928+0000 7fe54cff9640 1 -- 172.21.15.32:0/222805992 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=176}) -- 0x7fe530071f60 con 0x7fe5501fc050 2024-04-24T23:10:20.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.935+0000 7fe5567b7640 1 -- 172.21.15.32:0/222805992 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe53006be30 msgr2=0x7fe53006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:20.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.935+0000 7fe5567b7640 1 --2- 172.21.15.32:0/222805992 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe53006be30 0x7fe53006e2f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fe5500af440 tx=0x7fe538009040 comp rx=0 tx=0).stop 2024-04-24T23:10:20.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.935+0000 7fe5567b7640 1 -- 172.21.15.32:0/222805992 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5501fc050 msgr2=0x7fe5501fc4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:20.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.935+0000 7fe5567b7640 1 --2- 172.21.15.32:0/222805992 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5501fc050 0x7fe5501fc4d0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fe550071160 tx=0x7fe548010590 comp rx=0 tx=0).stop 2024-04-24T23:10:20.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.935+0000 7fe5567b7640 1 -- 172.21.15.32:0/222805992 shutdown_connections 2024-04-24T23:10:20.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.935+0000 7fe5567b7640 1 --2- 172.21.15.32:0/222805992 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe53006be30 0x7fe53006e2f0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:20.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.935+0000 7fe5567b7640 1 --2- 172.21.15.32:0/222805992 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5501fc050 0x7fe5501fc4d0 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:20.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.935+0000 7fe5567b7640 1 --2- 172.21.15.32:0/222805992 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5500ce190 0x7fe5500ce610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:20.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.935+0000 7fe5567b7640 1 --2- 172.21.15.32:0/222805992 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe53c0043b0 0x7fe550152cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:20.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.935+0000 7fe5567b7640 1 -- 172.21.15.32:0/222805992 >> 172.21.15.32:0/222805992 conn(0x7fe5500d0a60 msgr2=0x7fe5500b36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:20.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.936+0000 7fe5567b7640 1 -- 172.21.15.32:0/222805992 shutdown_connections 2024-04-24T23:10:20.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:20.936+0000 7fe5567b7640 1 -- 172.21.15.32:0/222805992 wait complete. 2024-04-24T23:10:20.951 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-24T23:10:21.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.060+0000 7f021cc09640 1 Processor -- start 2024-04-24T23:10:21.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.060+0000 7f021cc09640 1 -- start start 2024-04-24T23:10:21.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.061+0000 7f021cc09640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f02180b5390 0x7f02180b5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:21.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.061+0000 7f021cc09640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f02180b5d60 0x7f02180b36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:21.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.061+0000 7f021cc09640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0218160390 con 0x7f021815c420 2024-04-24T23:10:21.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.061+0000 7f021cc09640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f0218160140 con 0x7f02180b5d60 2024-04-24T23:10:21.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.061+0000 7f021cc09640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f0218160fa0 con 0x7f02180b5390 2024-04-24T23:10:21.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.061+0000 7f0216d9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f02180b5d60 0x7f02180b36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:21.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.061+0000 7f0216d9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f02180b5d60 0x7f02180b36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42192/0 (socket says 172.21.15.32:42192) 2024-04-24T23:10:21.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.061+0000 7f0216d9d640 1 -- 172.21.15.32:0/3108164590 learned_addr learned my addr 172.21.15.32:0/3108164590 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:21.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.061+0000 7f020dd9b640 1 --2- 172.21.15.32:0/3108164590 >> v2:172.21.15.121:3300/0 conn(0x7f02180b5390 0x7f02180b5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:21.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.061+0000 7f0216d9d640 1 -- 172.21.15.32:0/3108164590 >> v2:172.21.15.121:3300/0 conn(0x7f02180b5390 msgr2=0x7f02180b5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:21.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.061+0000 7f0216d9d640 1 --2- 172.21.15.32:0/3108164590 >> v2:172.21.15.121:3300/0 conn(0x7f02180b5390 0x7f02180b5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:21.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.061+0000 7f0216d9d640 1 -- 172.21.15.32:0/3108164590 >> v1:172.21.15.121:6789/0 conn(0x7f021815c420 legacy=0x7f02180af700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:21.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.061+0000 7f0216d9d640 1 -- 172.21.15.32:0/3108164590 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0218160670 con 0x7f02180b5d60 2024-04-24T23:10:21.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.061+0000 7f020dd9b640 1 --2- 172.21.15.32:0/3108164590 >> v2:172.21.15.121:3300/0 conn(0x7f02180b5390 0x7f02180b5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:10:21.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.062+0000 7f0216d9d640 1 --2- 172.21.15.32:0/3108164590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02180b5d60 0x7f02180b36d0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f0210009e50 tx=0x7f021002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9acddf7718df2961 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:21.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.062+0000 7f0215d9b640 1 -- 172.21.15.32:0/3108164590 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0210036070 con 0x7f02180b5d60 2024-04-24T23:10:21.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.062+0000 7f0215d9b640 1 -- 172.21.15.32:0/3108164590 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0210031600 con 0x7f02180b5d60 2024-04-24T23:10:21.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.062+0000 7f0215d9b640 1 -- 172.21.15.32:0/3108164590 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f021002ae60 con 0x7f02180b5d60 2024-04-24T23:10:21.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.063+0000 7f021cc09640 1 -- 172.21.15.32:0/3108164590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02180b5d60 msgr2=0x7f02180b36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:21.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.063+0000 7f021cc09640 1 --2- 172.21.15.32:0/3108164590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02180b5d60 0x7f02180b36d0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f0210009e50 tx=0x7f021002a680 comp rx=0 tx=0).stop 2024-04-24T23:10:21.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.063+0000 7f021cc09640 1 -- 172.21.15.32:0/3108164590 shutdown_connections 2024-04-24T23:10:21.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.063+0000 7f021cc09640 1 --2- 172.21.15.32:0/3108164590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02180b5d60 0x7f02180b36d0 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:21.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.063+0000 7f021cc09640 1 --2- 172.21.15.32:0/3108164590 >> v2:172.21.15.121:3300/0 conn(0x7f02180b5390 0x7f02180b5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:21.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.063+0000 7f021cc09640 1 -- 172.21.15.32:0/3108164590 >> 172.21.15.32:0/3108164590 conn(0x7f02180b1f10 msgr2=0x7f02180b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:21.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.063+0000 7f021cc09640 1 -- 172.21.15.32:0/3108164590 shutdown_connections 2024-04-24T23:10:21.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.063+0000 7f021cc09640 1 -- 172.21.15.32:0/3108164590 wait complete. 2024-04-24T23:10:21.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.064+0000 7f021cc09640 1 Processor -- start 2024-04-24T23:10:21.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.064+0000 7f021cc09640 1 -- start start 2024-04-24T23:10:21.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.065+0000 7f021cc09640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02180b5390 0x7f021814bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:21.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.065+0000 7f021cc09640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02180b5d60 0x7f02180ce870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:21.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.065+0000 7f021cc09640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f021815c420 0x7f02181f7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:21.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.065+0000 7f021cc09640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f02180717b0 con 0x7f021815c420 2024-04-24T23:10:21.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.065+0000 7f021cc09640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0218160c30 con 0x7f02180b5390 2024-04-24T23:10:21.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.065+0000 7f021cc09640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f0218071930 con 0x7f02180b5d60 2024-04-24T23:10:21.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.065+0000 7f0216d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f021815c420 0x7f02181f7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:21.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.065+0000 7f0216d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f021815c420 0x7f02181f7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42206/0 (socket says 172.21.15.32:42206) 2024-04-24T23:10:21.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.065+0000 7f0216d9d640 1 -- 172.21.15.32:0/419462369 learned_addr learned my addr 172.21.15.32:0/419462369 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:21.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.065+0000 7f020dd9b640 1 --2- 172.21.15.32:0/419462369 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02180b5d60 0x7f02180ce870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:21.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.065+0000 7f021659c640 1 --2- 172.21.15.32:0/419462369 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02180b5390 0x7f021814bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:21.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.065+0000 7f0216d9d640 1 -- 172.21.15.32:0/419462369 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02180b5d60 msgr2=0x7f02180ce870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:21.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.065+0000 7f0216d9d640 1 --2- 172.21.15.32:0/419462369 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02180b5d60 0x7f02180ce870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:21.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.065+0000 7f0216d9d640 1 -- 172.21.15.32:0/419462369 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02180b5390 msgr2=0x7f021814bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:21.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.065+0000 7f0216d9d640 1 --2- 172.21.15.32:0/419462369 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02180b5390 0x7f021814bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:21.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.065+0000 7f0216d9d640 1 -- 172.21.15.32:0/419462369 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02181f84c0 con 0x7f021815c420 2024-04-24T23:10:21.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.066+0000 7f0216d9d640 1 --2- 172.21.15.32:0/419462369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f021815c420 0x7f02181f7a20 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f0210009e50 tx=0x7f0210004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:21.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.066+0000 7f020f7fe640 1 -- 172.21.15.32:0/419462369 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0210036050 con 0x7f021815c420 2024-04-24T23:10:21.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.066+0000 7f020f7fe640 1 -- 172.21.15.32:0/419462369 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0210042070 con 0x7f021815c420 2024-04-24T23:10:21.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.066+0000 7f021cc09640 1 -- 172.21.15.32:0/419462369 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02181f87a0 con 0x7f021815c420 2024-04-24T23:10:21.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.066+0000 7f020f7fe640 1 -- 172.21.15.32:0/419462369 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0210009510 con 0x7f021815c420 2024-04-24T23:10:21.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.066+0000 7f021cc09640 1 -- 172.21.15.32:0/419462369 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02180ce070 con 0x7f021815c420 2024-04-24T23:10:21.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.067+0000 7f021cc09640 1 -- 172.21.15.32:0/419462369 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02180b5790 con 0x7f021815c420 2024-04-24T23:10:21.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.067+0000 7f020f7fe640 1 -- 172.21.15.32:0/419462369 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0210007bd0 con 0x7f021815c420 2024-04-24T23:10:21.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.068+0000 7f020f7fe640 1 --2- 172.21.15.32:0/419462369 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f01f806be30 0x7f01f806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:21.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.068+0000 7f020f7fe640 1 -- 172.21.15.32:0/419462369 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(175..175 src has 1..175) ==== 5415+0+0 (secure 0 0 0) 0x7f02100ab970 con 0x7f021815c420 2024-04-24T23:10:21.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.068+0000 7f020f7fe640 1 -- 172.21.15.32:0/419462369 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=176}) -- 0x7f01f8072cc0 con 0x7f021815c420 2024-04-24T23:10:21.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.068+0000 7f021659c640 1 --2- 172.21.15.32:0/419462369 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f01f806be30 0x7f01f806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:21.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.068+0000 7f021659c640 1 --2- 172.21.15.32:0/419462369 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f01f806be30 0x7f01f806e2f0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f0208006fd0 tx=0x7f0208008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:21.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.072+0000 7f020f7fe640 1 -- 172.21.15.32:0/419462369 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f02180b5790 con 0x7f021815c420 2024-04-24T23:10:21.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.255+0000 7f021cc09640 1 -- 172.21.15.32:0/419462369 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0218068430 con 0x7f021815c420 2024-04-24T23:10:21.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.854+0000 7f020f7fe640 1 -- 172.21.15.32:0/419462369 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v176) ==== 169+0+0 (secure 0 0 0) 0x7f0210045360 con 0x7f021815c420 2024-04-24T23:10:21.856 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2024-04-24T23:10:21.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.854+0000 7f020f7fe640 1 -- 172.21.15.32:0/419462369 <== mon.0 v2:172.21.15.32:3300/0 8 ==== osd_map(176..176 src has 1..176) ==== 721+0+0 (secure 0 0 0) 0x7f02100aa0d0 con 0x7f021815c420 2024-04-24T23:10:21.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.854+0000 7f020f7fe640 1 -- 172.21.15.32:0/419462369 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=177}) -- 0x7f01f8071a50 con 0x7f021815c420 2024-04-24T23:10:21.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.861+0000 7f021cc09640 1 -- 172.21.15.32:0/419462369 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f01f806be30 msgr2=0x7f01f806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:21.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.861+0000 7f021cc09640 1 --2- 172.21.15.32:0/419462369 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f01f806be30 0x7f01f806e2f0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f0208006fd0 tx=0x7f0208008040 comp rx=0 tx=0).stop 2024-04-24T23:10:21.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.861+0000 7f021cc09640 1 -- 172.21.15.32:0/419462369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f021815c420 msgr2=0x7f02181f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:21.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.861+0000 7f021cc09640 1 --2- 172.21.15.32:0/419462369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f021815c420 0x7f02181f7a20 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f0210009e50 tx=0x7f0210004a40 comp rx=0 tx=0).stop 2024-04-24T23:10:21.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.861+0000 7f021cc09640 1 -- 172.21.15.32:0/419462369 shutdown_connections 2024-04-24T23:10:21.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.861+0000 7f021cc09640 1 --2- 172.21.15.32:0/419462369 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f01f806be30 0x7f01f806e2f0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:21.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.861+0000 7f021cc09640 1 --2- 172.21.15.32:0/419462369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f021815c420 0x7f02181f7a20 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:21.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.861+0000 7f021cc09640 1 --2- 172.21.15.32:0/419462369 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02180b5d60 0x7f02180ce870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:21.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.861+0000 7f021cc09640 1 --2- 172.21.15.32:0/419462369 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02180b5390 0x7f021814bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:21.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.861+0000 7f021cc09640 1 -- 172.21.15.32:0/419462369 >> 172.21.15.32:0/419462369 conn(0x7f02180b1f10 msgr2=0x7f02180720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:21.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.863+0000 7f021cc09640 1 -- 172.21.15.32:0/419462369 shutdown_connections 2024-04-24T23:10:21.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.863+0000 7f021cc09640 1 -- 172.21.15.32:0/419462369 wait complete. 2024-04-24T23:10:21.879 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-24T23:10:21.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.986+0000 7f3da7c93640 1 Processor -- start 2024-04-24T23:10:21.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.986+0000 7f3da7c93640 1 -- start start 2024-04-24T23:10:21.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.987+0000 7f3da7c93640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3da00b5af0 0x7f3da00ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:21.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.987+0000 7f3da7c93640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3da00cd270 con 0x7f3da0162320 2024-04-24T23:10:21.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.987+0000 7f3da7c93640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3da0071610 con 0x7f3da0162e10 2024-04-24T23:10:21.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.987+0000 7f3da7c93640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f3da00b5660 con 0x7f3da00b5af0 2024-04-24T23:10:21.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.987+0000 7f3da5a2f640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3da0162320 0x7f3da0162700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59874/0 (socket says 172.21.15.32:59874) 2024-04-24T23:10:21.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.987+0000 7f3da5a2f640 1 -- 172.21.15.32:0/3095285885 learned_addr learned my addr 172.21.15.32:0/3095285885 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:10:21.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.987+0000 7f3da522e640 1 --2- 172.21.15.32:0/3095285885 >> v2:172.21.15.121:3300/0 conn(0x7f3da00b5af0 0x7f3da00ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:21.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.987+0000 7f3da4a2d640 1 -- 172.21.15.32:0/3095285885 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2302641402 0 0) 0x7f3da00cd270 con 0x7f3da0162320 2024-04-24T23:10:21.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.987+0000 7f3da4a2d640 1 -- 172.21.15.32:0/3095285885 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d88003520 con 0x7f3da0162320 2024-04-24T23:10:21.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.988+0000 7f3da4a2d640 1 -- 172.21.15.32:0/3095285885 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2650651687 0 0) 0x7f3da0071610 con 0x7f3da0162e10 2024-04-24T23:10:21.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.988+0000 7f3da4a2d640 1 -- 172.21.15.32:0/3095285885 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3da00cd270 con 0x7f3da0162e10 2024-04-24T23:10:21.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.988+0000 7f3da4a2d640 1 -- 172.21.15.32:0/3095285885 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4027403594 0 0) 0x7f3d88003520 con 0x7f3da0162320 2024-04-24T23:10:21.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.988+0000 7f3da4a2d640 1 -- 172.21.15.32:0/3095285885 >> v2:172.21.15.121:3300/0 conn(0x7f3da00b5af0 msgr2=0x7f3da00ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:21.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.988+0000 7f3da4a2d640 1 --2- 172.21.15.32:0/3095285885 >> v2:172.21.15.121:3300/0 conn(0x7f3da00b5af0 0x7f3da00ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:21.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.988+0000 7f3da4a2d640 1 -- 172.21.15.32:0/3095285885 >> v1:172.21.15.121:6789/0 conn(0x7f3da0162e10 legacy=0x7f3da015ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:21.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.988+0000 7f3da4a2d640 1 -- 172.21.15.32:0/3095285885 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3da00718f0 con 0x7f3da0162320 2024-04-24T23:10:21.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.988+0000 7f3da4a2d640 1 -- 172.21.15.32:0/3095285885 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f3d98001670 con 0x7f3da0162320 2024-04-24T23:10:21.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.988+0000 7f3da4a2d640 1 -- 172.21.15.32:0/3095285885 >> v1:172.21.15.32:6789/0 conn(0x7f3da0162320 legacy=0x7f3da0162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:21.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.988+0000 7f3da4a2d640 1 --2- 172.21.15.32:0/3095285885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d88003b40 0x7f3d88003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:21.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.988+0000 7f3da4a2d640 1 --2- 172.21.15.32:0/3095285885 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d88004800 0x7f3d88004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:21.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.988+0000 7f3da4a2d640 1 --2- 172.21.15.32:0/3095285885 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3d880051c0 0x7f3d8800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:21.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.988+0000 7f3da4a2d640 1 -- 172.21.15.32:0/3095285885 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3da00b5660 con 0x7f3d88003b40 2024-04-24T23:10:21.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.988+0000 7f3da4a2d640 1 -- 172.21.15.32:0/3095285885 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8800c0f0 con 0x7f3d88004800 2024-04-24T23:10:21.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.988+0000 7f3da4a2d640 1 -- 172.21.15.32:0/3095285885 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f3d8800c2f0 con 0x7f3d880051c0 2024-04-24T23:10:21.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.988+0000 7f3da5a2f640 1 --2- 172.21.15.32:0/3095285885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d88003b40 0x7f3d88003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:21.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.988+0000 7f3da522e640 1 --2- 172.21.15.32:0/3095285885 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d88004800 0x7f3d88004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:21.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.988+0000 7f3da6230640 1 --2- 172.21.15.32:0/3095285885 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3d880051c0 0x7f3d8800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:21.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.988+0000 7f3da5a2f640 1 -- 172.21.15.32:0/3095285885 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3d880051c0 msgr2=0x7f3d8800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:21.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.988+0000 7f3da5a2f640 1 --2- 172.21.15.32:0/3095285885 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3d880051c0 0x7f3d8800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:21.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.988+0000 7f3da5a2f640 1 -- 172.21.15.32:0/3095285885 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d88004800 msgr2=0x7f3d88004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:21.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.988+0000 7f3da5a2f640 1 --2- 172.21.15.32:0/3095285885 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d88004800 0x7f3d88004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:21.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.988+0000 7f3da5a2f640 1 -- 172.21.15.32:0/3095285885 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d88003520 con 0x7f3d88003b40 2024-04-24T23:10:21.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.988+0000 7f3da6230640 1 -- 172.21.15.32:0/3095285885 reap_dead start 2024-04-24T23:10:21.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.989+0000 7f3da5a2f640 1 --2- 172.21.15.32:0/3095285885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d88003b40 0x7f3d88003f40 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f3d9800c190 tx=0x7f3d88005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dea9c9f43f6ff310 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:21.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.989+0000 7f3da4a2d640 1 -- 172.21.15.32:0/3095285885 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d98007ce0 con 0x7f3d88003b40 2024-04-24T23:10:21.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.989+0000 7f3da4a2d640 1 -- 172.21.15.32:0/3095285885 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d98005ad0 con 0x7f3d88003b40 2024-04-24T23:10:21.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.989+0000 7f3da4a2d640 1 -- 172.21.15.32:0/3095285885 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d98007070 con 0x7f3d88003b40 2024-04-24T23:10:21.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.990+0000 7f3da7c93640 1 -- 172.21.15.32:0/3095285885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d88003b40 msgr2=0x7f3d88003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:21.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.990+0000 7f3da7c93640 1 --2- 172.21.15.32:0/3095285885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d88003b40 0x7f3d88003f40 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f3d9800c190 tx=0x7f3d88005830 comp rx=0 tx=0).stop 2024-04-24T23:10:21.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.990+0000 7f3da7c93640 1 -- 172.21.15.32:0/3095285885 shutdown_connections 2024-04-24T23:10:21.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.990+0000 7f3da7c93640 1 --2- 172.21.15.32:0/3095285885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d88003b40 0x7f3d88003f40 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:21.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.990+0000 7f3da7c93640 1 -- 172.21.15.32:0/3095285885 >> 172.21.15.32:0/3095285885 conn(0x7f3da00d0ae0 msgr2=0x7f3da00d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:21.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.990+0000 7f3da7c93640 1 -- 172.21.15.32:0/3095285885 shutdown_connections 2024-04-24T23:10:21.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.990+0000 7f3da7c93640 1 -- 172.21.15.32:0/3095285885 wait complete. 2024-04-24T23:10:21.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.991+0000 7f3da7c93640 1 Processor -- start 2024-04-24T23:10:21.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.991+0000 7f3da7c93640 1 -- start start 2024-04-24T23:10:21.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.992+0000 7f3da7c93640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d88003b40 0x7f3da006ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:21.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.992+0000 7f3da7c93640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3da0161230 0x7f3da00ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:21.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.992+0000 7f3da7c93640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3da01616b0 0x7f3da00ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:21.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.992+0000 7f3da7c93640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3da0071bd0 con 0x7f3da0161230 2024-04-24T23:10:21.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.992+0000 7f3da7c93640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3da0067950 con 0x7f3d88003b40 2024-04-24T23:10:21.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.992+0000 7f3da7c93640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f3da0071d50 con 0x7f3da01616b0 2024-04-24T23:10:21.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.992+0000 7f3da522e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3da0161230 0x7f3da00ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:21.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.992+0000 7f3da522e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3da0161230 0x7f3da00ce0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42218/0 (socket says 172.21.15.32:42218) 2024-04-24T23:10:21.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.992+0000 7f3da522e640 1 -- 172.21.15.32:0/2382643892 learned_addr learned my addr 172.21.15.32:0/2382643892 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:21.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.992+0000 7f3da522e640 1 -- 172.21.15.32:0/2382643892 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3da01616b0 msgr2=0x7f3da00ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:21.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.992+0000 7f3da522e640 1 --2- 172.21.15.32:0/2382643892 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3da01616b0 0x7f3da00ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:21.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.992+0000 7f3da522e640 1 -- 172.21.15.32:0/2382643892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d88003b40 msgr2=0x7f3da006ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:21.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.992+0000 7f3da5a2f640 1 --2- 172.21.15.32:0/2382643892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d88003b40 0x7f3da006ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:21.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.992+0000 7f3da522e640 1 --2- 172.21.15.32:0/2382643892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d88003b40 0x7f3da006ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:21.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.992+0000 7f3da522e640 1 -- 172.21.15.32:0/2382643892 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3da006de90 con 0x7f3da0161230 2024-04-24T23:10:21.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.992+0000 7f3da5a2f640 1 --2- 172.21.15.32:0/2382643892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d88003b40 0x7f3da006ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:10:22.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.992+0000 7f3da522e640 1 --2- 172.21.15.32:0/2382643892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3da0161230 0x7f3da00ce0a0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f3d94011940 tx=0x7f3d94011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:22.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.993+0000 7f3d9e7fc640 1 -- 172.21.15.32:0/2382643892 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9400fa90 con 0x7f3da0161230 2024-04-24T23:10:22.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.993+0000 7f3d9e7fc640 1 -- 172.21.15.32:0/2382643892 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d9400fc30 con 0x7f3da0161230 2024-04-24T23:10:22.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.993+0000 7f3da7c93640 1 -- 172.21.15.32:0/2382643892 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3da006e170 con 0x7f3da0161230 2024-04-24T23:10:22.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.993+0000 7f3d9e7fc640 1 -- 172.21.15.32:0/2382643892 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d94014440 con 0x7f3da0161230 2024-04-24T23:10:22.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.993+0000 7f3da7c93640 1 -- 172.21.15.32:0/2382643892 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3da006e730 con 0x7f3da0161230 2024-04-24T23:10:22.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.994+0000 7f3da7c93640 1 -- 172.21.15.32:0/2382643892 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3da0068430 con 0x7f3da0161230 2024-04-24T23:10:22.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.994+0000 7f3d9e7fc640 1 -- 172.21.15.32:0/2382643892 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3d94012040 con 0x7f3da0161230 2024-04-24T23:10:22.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.995+0000 7f3d9e7fc640 1 --2- 172.21.15.32:0/2382643892 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3d8c06be30 0x7f3d8c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:22.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.995+0000 7f3d9e7fc640 1 -- 172.21.15.32:0/2382643892 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(176..176 src has 1..176) ==== 4943+0+0 (secure 0 0 0) 0x7f3d9408b290 con 0x7f3da0161230 2024-04-24T23:10:22.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.995+0000 7f3da5a2f640 1 --2- 172.21.15.32:0/2382643892 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3d8c06be30 0x7f3d8c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:22.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.995+0000 7f3da5a2f640 1 --2- 172.21.15.32:0/2382643892 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3d8c06be30 0x7f3d8c06e2f0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f3d98032000 tx=0x7f3d98009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:22.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:21.999+0000 7f3d9e7fc640 1 -- 172.21.15.32:0/2382643892 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3d94019030 con 0x7f3da0161230 2024-04-24T23:10:22.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.184+0000 7f3da7c93640 1 -- 172.21.15.32:0/2382643892 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f3da01629b0 con 0x7f3da0161230 2024-04-24T23:10:22.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.187+0000 7f3d9e7fc640 1 -- 172.21.15.32:0/2382643892 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7f3d94059ea0 con 0x7f3da0161230 2024-04-24T23:10:22.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.189+0000 7f3da7c93640 1 -- 172.21.15.32:0/2382643892 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3d8c06be30 msgr2=0x7f3d8c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:22.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.189+0000 7f3da7c93640 1 --2- 172.21.15.32:0/2382643892 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3d8c06be30 0x7f3d8c06e2f0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f3d98032000 tx=0x7f3d98009030 comp rx=0 tx=0).stop 2024-04-24T23:10:22.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.189+0000 7f3da7c93640 1 -- 172.21.15.32:0/2382643892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3da0161230 msgr2=0x7f3da00ce0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:22.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.189+0000 7f3da7c93640 1 --2- 172.21.15.32:0/2382643892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3da0161230 0x7f3da00ce0a0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f3d94011940 tx=0x7f3d94011e00 comp rx=0 tx=0).stop 2024-04-24T23:10:22.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.189+0000 7f3da7c93640 1 -- 172.21.15.32:0/2382643892 shutdown_connections 2024-04-24T23:10:22.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.189+0000 7f3da7c93640 1 --2- 172.21.15.32:0/2382643892 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3d8c06be30 0x7f3d8c06e2f0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:22.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.189+0000 7f3da7c93640 1 --2- 172.21.15.32:0/2382643892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3da0161230 0x7f3da00ce0a0 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:22.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.189+0000 7f3da7c93640 1 --2- 172.21.15.32:0/2382643892 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3da01616b0 0x7f3da00ce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:22.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.189+0000 7f3da7c93640 1 --2- 172.21.15.32:0/2382643892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d88003b40 0x7f3da006ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:22.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.189+0000 7f3da7c93640 1 -- 172.21.15.32:0/2382643892 >> 172.21.15.32:0/2382643892 conn(0x7f3da00d0ae0 msgr2=0x7f3da015afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:22.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.189+0000 7f3da7c93640 1 -- 172.21.15.32:0/2382643892 shutdown_connections 2024-04-24T23:10:22.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.189+0000 7f3da7c93640 1 -- 172.21.15.32:0/2382643892 wait complete. 2024-04-24T23:10:22.202 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync' 2024-04-24T23:10:22.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.301+0000 7fdac0709640 1 Processor -- start 2024-04-24T23:10:22.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.301+0000 7fdac0709640 1 -- start start 2024-04-24T23:10:22.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.302+0000 7fdac0709640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdab806dc60 0x7fdab8169570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:22.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.302+0000 7fdac0709640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdab80b6d10 con 0x7fdab8158dd0 2024-04-24T23:10:22.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.302+0000 7fdac0709640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdab815ef10 con 0x7fdab806ba30 2024-04-24T23:10:22.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.302+0000 7fdac0709640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fdab815f1f0 con 0x7fdab806dc60 2024-04-24T23:10:22.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.302+0000 7fdabe4a5640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fdab8158dd0 0x7fdab806b320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59878/0 (socket says 172.21.15.32:59878) 2024-04-24T23:10:22.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.302+0000 7fdabe4a5640 1 -- 172.21.15.32:0/3167980080 learned_addr learned my addr 172.21.15.32:0/3167980080 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:10:22.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.302+0000 7fdabd4a3640 1 -- 172.21.15.32:0/3167980080 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3401965467 0 0) 0x7fdab80b6d10 con 0x7fdab8158dd0 2024-04-24T23:10:22.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.302+0000 7fdabeca6640 1 --2- 172.21.15.32:0/3167980080 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdab806dc60 0x7fdab8169570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:22.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.302+0000 7fdabd4a3640 1 -- 172.21.15.32:0/3167980080 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdaa0003520 con 0x7fdab8158dd0 2024-04-24T23:10:22.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.303+0000 7fdabd4a3640 1 -- 172.21.15.32:0/3167980080 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3434107824 0 0) 0x7fdaa0003520 con 0x7fdab8158dd0 2024-04-24T23:10:22.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.303+0000 7fdabd4a3640 1 -- 172.21.15.32:0/3167980080 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdab806dc60 msgr2=0x7fdab8169570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:22.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.303+0000 7fdabd4a3640 1 --2- 172.21.15.32:0/3167980080 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdab806dc60 0x7fdab8169570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:22.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.303+0000 7fdabd4a3640 1 -- 172.21.15.32:0/3167980080 >> v1:172.21.15.121:6789/0 conn(0x7fdab806ba30 legacy=0x7fdab806d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:22.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.303+0000 7fdabd4a3640 1 -- 172.21.15.32:0/3167980080 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdab815f4d0 con 0x7fdab8158dd0 2024-04-24T23:10:22.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.303+0000 7fdabd4a3640 1 -- 172.21.15.32:0/3167980080 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7fdab0001690 con 0x7fdab8158dd0 2024-04-24T23:10:22.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.303+0000 7fdabd4a3640 1 -- 172.21.15.32:0/3167980080 >> v1:172.21.15.32:6789/0 conn(0x7fdab8158dd0 legacy=0x7fdab806b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:22.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.303+0000 7fdabd4a3640 1 --2- 172.21.15.32:0/3167980080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaa00039e0 0x7fdaa0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:22.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.303+0000 7fdabd4a3640 1 --2- 172.21.15.32:0/3167980080 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdaa0004390 0x7fdaa0004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:22.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.303+0000 7fdabd4a3640 1 --2- 172.21.15.32:0/3167980080 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdab806dc60 0x7fdaa00050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:22.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.303+0000 7fdabd4a3640 1 -- 172.21.15.32:0/3167980080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdab815f1f0 con 0x7fdaa00039e0 2024-04-24T23:10:22.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.303+0000 7fdabd4a3640 1 -- 172.21.15.32:0/3167980080 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa000b9f0 con 0x7fdaa0004390 2024-04-24T23:10:22.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.303+0000 7fdabd4a3640 1 -- 172.21.15.32:0/3167980080 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fdaa000bbf0 con 0x7fdab806dc60 2024-04-24T23:10:22.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.303+0000 7fdabe4a5640 1 --2- 172.21.15.32:0/3167980080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaa00039e0 0x7fdaa0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:22.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.303+0000 7fdabdca4640 1 --2- 172.21.15.32:0/3167980080 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdaa0004390 0x7fdaa0004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:22.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.303+0000 7fdabeca6640 1 --2- 172.21.15.32:0/3167980080 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdab806dc60 0x7fdaa00050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:22.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.303+0000 7fdabe4a5640 1 -- 172.21.15.32:0/3167980080 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdab806dc60 msgr2=0x7fdaa00050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:22.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.303+0000 7fdabe4a5640 1 --2- 172.21.15.32:0/3167980080 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdab806dc60 0x7fdaa00050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:22.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.303+0000 7fdabe4a5640 1 -- 172.21.15.32:0/3167980080 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdaa0004390 msgr2=0x7fdaa0004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:22.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.303+0000 7fdabe4a5640 1 --2- 172.21.15.32:0/3167980080 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdaa0004390 0x7fdaa0004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:22.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.303+0000 7fdabe4a5640 1 -- 172.21.15.32:0/3167980080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaa0003520 con 0x7fdaa00039e0 2024-04-24T23:10:22.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.303+0000 7fdabdca4640 1 --2- 172.21.15.32:0/3167980080 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdaa0004390 0x7fdaa0004860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:10:22.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.304+0000 7fdabe4a5640 1 --2- 172.21.15.32:0/3167980080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaa00039e0 0x7fdaa0003dc0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7fdab000c210 tx=0x7fdaa0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2034aa47c25c7380 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:22.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.304+0000 7fdabd4a3640 1 -- 172.21.15.32:0/3167980080 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab0007d60 con 0x7fdaa00039e0 2024-04-24T23:10:22.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.304+0000 7fdabd4a3640 1 -- 172.21.15.32:0/3167980080 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdab0005af0 con 0x7fdaa00039e0 2024-04-24T23:10:22.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.304+0000 7fdabd4a3640 1 -- 172.21.15.32:0/3167980080 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab0007070 con 0x7fdaa00039e0 2024-04-24T23:10:22.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.304+0000 7fdac0709640 1 -- 172.21.15.32:0/3167980080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaa00039e0 msgr2=0x7fdaa0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:22.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.304+0000 7fdac0709640 1 --2- 172.21.15.32:0/3167980080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaa00039e0 0x7fdaa0003dc0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7fdab000c210 tx=0x7fdaa0005830 comp rx=0 tx=0).stop 2024-04-24T23:10:22.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.304+0000 7fdabeca6640 1 -- 172.21.15.32:0/3167980080 reap_dead start 2024-04-24T23:10:22.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.305+0000 7fdac0709640 1 -- 172.21.15.32:0/3167980080 shutdown_connections 2024-04-24T23:10:22.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.305+0000 7fdac0709640 1 -- 172.21.15.32:0/3167980080 >> 172.21.15.32:0/3167980080 conn(0x7fdab806e800 msgr2=0x7fdab806ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:22.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.305+0000 7fdac0709640 1 -- 172.21.15.32:0/3167980080 shutdown_connections 2024-04-24T23:10:22.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.305+0000 7fdac0709640 1 -- 172.21.15.32:0/3167980080 wait complete. 2024-04-24T23:10:22.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.306+0000 7fdac0709640 1 Processor -- start 2024-04-24T23:10:22.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.306+0000 7fdac0709640 1 -- start start 2024-04-24T23:10:22.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.306+0000 7fdac0709640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdaa00039e0 0x7fdab8146c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:22.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.306+0000 7fdac0709640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdab8157300 0x7fdab8142e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:22.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.306+0000 7fdac0709640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdab815c8c0 0x7fdab813ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:22.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.306+0000 7fdac0709640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdab806bfe0 con 0x7fdab8157300 2024-04-24T23:10:22.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.306+0000 7fdac0709640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdab815fcd0 con 0x7fdab815c8c0 2024-04-24T23:10:22.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.306+0000 7fdac0709640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fdab81591b0 con 0x7fdaa00039e0 2024-04-24T23:10:22.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.307+0000 7fdabdca4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdab8157300 0x7fdab8142e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:22.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.307+0000 7fdabdca4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdab8157300 0x7fdab8142e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42228/0 (socket says 172.21.15.32:42228) 2024-04-24T23:10:22.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.307+0000 7fdabdca4640 1 -- 172.21.15.32:0/4222300711 learned_addr learned my addr 172.21.15.32:0/4222300711 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:22.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.307+0000 7fdabe4a5640 1 --2- 172.21.15.32:0/4222300711 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdaa00039e0 0x7fdab8146c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:22.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.307+0000 7fdabdca4640 1 -- 172.21.15.32:0/4222300711 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdaa00039e0 msgr2=0x7fdab8146c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:22.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.307+0000 7fdabeca6640 1 --2- 172.21.15.32:0/4222300711 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdab815c8c0 0x7fdab813ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:22.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.307+0000 7fdabdca4640 1 --2- 172.21.15.32:0/4222300711 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdaa00039e0 0x7fdab8146c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:22.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.307+0000 7fdabdca4640 1 -- 172.21.15.32:0/4222300711 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdab815c8c0 msgr2=0x7fdab813ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:22.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.307+0000 7fdabdca4640 1 --2- 172.21.15.32:0/4222300711 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdab815c8c0 0x7fdab813ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:22.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.307+0000 7fdabdca4640 1 -- 172.21.15.32:0/4222300711 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdab8157870 con 0x7fdab8157300 2024-04-24T23:10:22.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.307+0000 7fdabeca6640 1 --2- 172.21.15.32:0/4222300711 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdab815c8c0 0x7fdab813ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:10:22.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.307+0000 7fdabdca4640 1 --2- 172.21.15.32:0/4222300711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdab8157300 0x7fdab8142e70 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fdaac00e5d0 tx=0x7fdaac00ea90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:22.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.307+0000 7fdab6ffd640 1 -- 172.21.15.32:0/4222300711 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaac0052e0 con 0x7fdab8157300 2024-04-24T23:10:22.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.307+0000 7fdab6ffd640 1 -- 172.21.15.32:0/4222300711 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaac018070 con 0x7fdab8157300 2024-04-24T23:10:22.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.307+0000 7fdac0709640 1 -- 172.21.15.32:0/4222300711 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdab8157b50 con 0x7fdab8157300 2024-04-24T23:10:22.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.307+0000 7fdab6ffd640 1 -- 172.21.15.32:0/4222300711 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaac003830 con 0x7fdab8157300 2024-04-24T23:10:22.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.307+0000 7fdac0709640 1 -- 172.21.15.32:0/4222300711 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdab8154b80 con 0x7fdab8157300 2024-04-24T23:10:22.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.308+0000 7fdac0709640 1 -- 172.21.15.32:0/4222300711 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdab8068430 con 0x7fdab8157300 2024-04-24T23:10:22.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.309+0000 7fdab6ffd640 1 -- 172.21.15.32:0/4222300711 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdaac0039d0 con 0x7fdab8157300 2024-04-24T23:10:22.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.309+0000 7fdab6ffd640 1 --2- 172.21.15.32:0/4222300711 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdaa406be60 0x7fdaa406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:22.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.309+0000 7fdab6ffd640 1 -- 172.21.15.32:0/4222300711 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(176..176 src has 1..176) ==== 4943+0+0 (secure 0 0 0) 0x7fdaac08a910 con 0x7fdab8157300 2024-04-24T23:10:22.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.309+0000 7fdabe4a5640 1 --2- 172.21.15.32:0/4222300711 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdaa406be60 0x7fdaa406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:22.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.310+0000 7fdabe4a5640 1 --2- 172.21.15.32:0/4222300711 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdaa406be60 0x7fdaa406e320 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fdab0033000 tx=0x7fdab002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:22.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.313+0000 7fdab6ffd640 1 -- 172.21.15.32:0/4222300711 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdaac0595c0 con 0x7fdab8157300 2024-04-24T23:10:22.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.575+0000 7fdac0709640 1 -- 172.21.15.32:0/4222300711 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]} v 0) -- 0x7fdab8068ca0 con 0x7fdab8157300 2024-04-24T23:10:22.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.940+0000 7fdab6ffd640 1 -- 172.21.15.32:0/4222300711 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]}]=0 v534) ==== 131+0+0 (secure 0 0 0) 0x7fdaac058ec0 con 0x7fdab8157300 2024-04-24T23:10:22.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.942+0000 7fdac0709640 1 -- 172.21.15.32:0/4222300711 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdaa406be60 msgr2=0x7fdaa406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:22.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.942+0000 7fdac0709640 1 --2- 172.21.15.32:0/4222300711 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdaa406be60 0x7fdaa406e320 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fdab0033000 tx=0x7fdab002e000 comp rx=0 tx=0).stop 2024-04-24T23:10:22.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.942+0000 7fdac0709640 1 -- 172.21.15.32:0/4222300711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdab8157300 msgr2=0x7fdab8142e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:22.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.942+0000 7fdac0709640 1 --2- 172.21.15.32:0/4222300711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdab8157300 0x7fdab8142e70 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fdaac00e5d0 tx=0x7fdaac00ea90 comp rx=0 tx=0).stop 2024-04-24T23:10:22.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.942+0000 7fdac0709640 1 -- 172.21.15.32:0/4222300711 shutdown_connections 2024-04-24T23:10:22.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.942+0000 7fdac0709640 1 --2- 172.21.15.32:0/4222300711 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fdaa406be60 0x7fdaa406e320 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:22.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.942+0000 7fdac0709640 1 --2- 172.21.15.32:0/4222300711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdab8157300 0x7fdab8142e70 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:22.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.942+0000 7fdac0709640 1 --2- 172.21.15.32:0/4222300711 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdab815c8c0 0x7fdab813ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:22.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.942+0000 7fdac0709640 1 --2- 172.21.15.32:0/4222300711 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdaa00039e0 0x7fdab8146c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:22.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.942+0000 7fdac0709640 1 -- 172.21.15.32:0/4222300711 >> 172.21.15.32:0/4222300711 conn(0x7fdab806e800 msgr2=0x7fdab8161800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:22.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.943+0000 7fdac0709640 1 -- 172.21.15.32:0/4222300711 shutdown_connections 2024-04-24T23:10:22.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:22.943+0000 7fdac0709640 1 -- 172.21.15.32:0/4222300711 wait complete. 2024-04-24T23:10:22.956 INFO:tasks.cephfs_test_runner:That when the full flag is encountered during asynchronous ... ok 2024-04-24T23:10:22.956 INFO:tasks.cephfs_test_runner:Starting test: test_full_same_file (tasks.cephfs.test_full.TestQuotaFull) 2024-04-24T23:10:22.956 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:10:23.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.057+0000 7f097f8d4640 1 Processor -- start 2024-04-24T23:10:23.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.057+0000 7f097f8d4640 1 -- start start 2024-04-24T23:10:23.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.058+0000 7f097f8d4640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f097806cd00 0x7f0978166440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:23.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.058+0000 7f097f8d4640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09780cef90 con 0x7f097815b480 2024-04-24T23:10:23.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.058+0000 7f097f8d4640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09780b46c0 con 0x7f097815fd60 2024-04-24T23:10:23.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.058+0000 7f097f8d4640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f09780b4ff0 con 0x7f097806cd00 2024-04-24T23:10:23.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.058+0000 7f097d670640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f097815b480 0x7f09780b5c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59882/0 (socket says 172.21.15.32:59882) 2024-04-24T23:10:23.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.058+0000 7f097d670640 1 -- 172.21.15.32:0/2639994928 learned_addr learned my addr 172.21.15.32:0/2639994928 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:10:23.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.058+0000 7f097de71640 1 --2- 172.21.15.32:0/2639994928 >> v2:172.21.15.121:3300/0 conn(0x7f097806cd00 0x7f0978166440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:23.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.058+0000 7f0977fff640 1 -- 172.21.15.32:0/2639994928 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3669248716 0 0) 0x7f09780cef90 con 0x7f097815b480 2024-04-24T23:10:23.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.059+0000 7f0977fff640 1 -- 172.21.15.32:0/2639994928 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0960003520 con 0x7f097815b480 2024-04-24T23:10:23.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.059+0000 7f0977fff640 1 -- 172.21.15.32:0/2639994928 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1773992152 0 0) 0x7f09780b46c0 con 0x7f097815fd60 2024-04-24T23:10:23.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.059+0000 7f0977fff640 1 -- 172.21.15.32:0/2639994928 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f09780cef90 con 0x7f097815fd60 2024-04-24T23:10:23.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.059+0000 7f0977fff640 1 -- 172.21.15.32:0/2639994928 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2621209620 0 0) 0x7f0960003520 con 0x7f097815b480 2024-04-24T23:10:23.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.059+0000 7f0977fff640 1 -- 172.21.15.32:0/2639994928 >> v2:172.21.15.121:3300/0 conn(0x7f097806cd00 msgr2=0x7f0978166440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:23.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.059+0000 7f0977fff640 1 --2- 172.21.15.32:0/2639994928 >> v2:172.21.15.121:3300/0 conn(0x7f097806cd00 0x7f0978166440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:23.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.059+0000 7f0977fff640 1 -- 172.21.15.32:0/2639994928 >> v1:172.21.15.121:6789/0 conn(0x7f097815fd60 legacy=0x7f097806c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:23.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.059+0000 7f0977fff640 1 -- 172.21.15.32:0/2639994928 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09780b49a0 con 0x7f097815b480 2024-04-24T23:10:23.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.059+0000 7f0977fff640 1 -- 172.21.15.32:0/2639994928 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f0968001970 con 0x7f097815b480 2024-04-24T23:10:23.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.059+0000 7f0977fff640 1 -- 172.21.15.32:0/2639994928 >> v1:172.21.15.32:6789/0 conn(0x7f097815b480 legacy=0x7f09780b5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:23.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.059+0000 7f0977fff640 1 --2- 172.21.15.32:0/2639994928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0960003b90 0x7f0960003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:23.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.059+0000 7f0977fff640 1 --2- 172.21.15.32:0/2639994928 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0960004800 0x7f0960004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:23.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.059+0000 7f0977fff640 1 --2- 172.21.15.32:0/2639994928 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09600051c0 0x7f096000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:23.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.059+0000 7f0977fff640 1 -- 172.21.15.32:0/2639994928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f09780b4ff0 con 0x7f0960003b90 2024-04-24T23:10:23.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.059+0000 7f0977fff640 1 -- 172.21.15.32:0/2639994928 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f096000c090 con 0x7f0960004800 2024-04-24T23:10:23.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.059+0000 7f0977fff640 1 -- 172.21.15.32:0/2639994928 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f096000c290 con 0x7f09600051c0 2024-04-24T23:10:23.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.059+0000 7f097d670640 1 --2- 172.21.15.32:0/2639994928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0960003b90 0x7f0960003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:23.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.059+0000 7f097ce6f640 1 --2- 172.21.15.32:0/2639994928 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0960004800 0x7f0960004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:23.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.059+0000 7f097de71640 1 --2- 172.21.15.32:0/2639994928 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09600051c0 0x7f096000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:23.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.059+0000 7f097d670640 1 -- 172.21.15.32:0/2639994928 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09600051c0 msgr2=0x7f096000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:23.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.059+0000 7f097d670640 1 --2- 172.21.15.32:0/2639994928 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09600051c0 0x7f096000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:23.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.059+0000 7f097d670640 1 -- 172.21.15.32:0/2639994928 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0960004800 msgr2=0x7f0960004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:23.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.059+0000 7f097d670640 1 --2- 172.21.15.32:0/2639994928 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0960004800 0x7f0960004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:23.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.059+0000 7f097d670640 1 -- 172.21.15.32:0/2639994928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0960003520 con 0x7f0960003b90 2024-04-24T23:10:23.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.059+0000 7f097de71640 1 -- 172.21.15.32:0/2639994928 reap_dead start 2024-04-24T23:10:23.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.060+0000 7f097d670640 1 --2- 172.21.15.32:0/2639994928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0960003b90 0x7f0960003f90 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f096800a6b0 tx=0x7f0960005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75341bc18b45cae6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:23.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.060+0000 7f0977fff640 1 -- 172.21.15.32:0/2639994928 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096800a2d0 con 0x7f0960003b90 2024-04-24T23:10:23.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.060+0000 7f0977fff640 1 -- 172.21.15.32:0/2639994928 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f096802f070 con 0x7f0960003b90 2024-04-24T23:10:23.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.060+0000 7f0977fff640 1 -- 172.21.15.32:0/2639994928 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096802bbd0 con 0x7f0960003b90 2024-04-24T23:10:23.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.061+0000 7f097f8d4640 1 -- 172.21.15.32:0/2639994928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0960003b90 msgr2=0x7f0960003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:23.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.061+0000 7f097f8d4640 1 --2- 172.21.15.32:0/2639994928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0960003b90 0x7f0960003f90 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f096800a6b0 tx=0x7f0960005830 comp rx=0 tx=0).stop 2024-04-24T23:10:23.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.061+0000 7f097f8d4640 1 -- 172.21.15.32:0/2639994928 shutdown_connections 2024-04-24T23:10:23.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.061+0000 7f097f8d4640 1 --2- 172.21.15.32:0/2639994928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0960003b90 0x7f0960003f90 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:23.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.061+0000 7f097f8d4640 1 -- 172.21.15.32:0/2639994928 >> 172.21.15.32:0/2639994928 conn(0x7f097806b110 msgr2=0x7f097806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:23.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.061+0000 7f097f8d4640 1 -- 172.21.15.32:0/2639994928 shutdown_connections 2024-04-24T23:10:23.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.061+0000 7f097f8d4640 1 -- 172.21.15.32:0/2639994928 wait complete. 2024-04-24T23:10:23.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.062+0000 7f097f8d4640 1 Processor -- start 2024-04-24T23:10:23.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.062+0000 7f097f8d4640 1 -- start start 2024-04-24T23:10:23.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.063+0000 7f097f8d4640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0960003b90 0x7f09781483b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:23.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.063+0000 7f097f8d4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f097815d410 0x7f09780ace80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:23.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.063+0000 7f097f8d4640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f097815d870 0x7f09780ad3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:23.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.063+0000 7f097f8d4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f097815b860 con 0x7f097815d410 2024-04-24T23:10:23.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.063+0000 7f097f8d4640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f09780b5480 con 0x7f097815d870 2024-04-24T23:10:23.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.063+0000 7f097f8d4640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f0978147310 con 0x7f0960003b90 2024-04-24T23:10:23.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.063+0000 7f097ce6f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f097815d410 0x7f09780ace80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:23.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.063+0000 7f097ce6f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f097815d410 0x7f09780ace80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42246/0 (socket says 172.21.15.32:42246) 2024-04-24T23:10:23.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.063+0000 7f097ce6f640 1 -- 172.21.15.32:0/1307045146 learned_addr learned my addr 172.21.15.32:0/1307045146 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:23.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.063+0000 7f097d670640 1 --2- 172.21.15.32:0/1307045146 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0960003b90 0x7f09781483b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:23.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.063+0000 7f097de71640 1 --2- 172.21.15.32:0/1307045146 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f097815d870 0x7f09780ad3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:23.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.063+0000 7f097ce6f640 1 -- 172.21.15.32:0/1307045146 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0960003b90 msgr2=0x7f09781483b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:23.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.063+0000 7f097ce6f640 1 --2- 172.21.15.32:0/1307045146 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0960003b90 0x7f09781483b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:23.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.063+0000 7f097ce6f640 1 -- 172.21.15.32:0/1307045146 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f097815d870 msgr2=0x7f09780ad3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:23.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.063+0000 7f097ce6f640 1 --2- 172.21.15.32:0/1307045146 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f097815d870 0x7f09780ad3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:23.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.063+0000 7f097ce6f640 1 -- 172.21.15.32:0/1307045146 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09781475f0 con 0x7f097815d410 2024-04-24T23:10:23.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.063+0000 7f097d670640 1 --2- 172.21.15.32:0/1307045146 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0960003b90 0x7f09781483b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:10:23.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.063+0000 7f097ce6f640 1 --2- 172.21.15.32:0/1307045146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f097815d410 0x7f09780ace80 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f096c00f7d0 tx=0x7f096c00fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:23.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.063+0000 7f0975ffb640 1 -- 172.21.15.32:0/1307045146 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096c01a070 con 0x7f097815d410 2024-04-24T23:10:23.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.063+0000 7f0975ffb640 1 -- 172.21.15.32:0/1307045146 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f096c005320 con 0x7f097815d410 2024-04-24T23:10:23.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.063+0000 7f097f8d4640 1 -- 172.21.15.32:0/1307045146 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09781478d0 con 0x7f097815d410 2024-04-24T23:10:23.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.064+0000 7f0975ffb640 1 -- 172.21.15.32:0/1307045146 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096c007270 con 0x7f097815d410 2024-04-24T23:10:23.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.064+0000 7f097f8d4640 1 -- 172.21.15.32:0/1307045146 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0978147e90 con 0x7f097815d410 2024-04-24T23:10:23.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.065+0000 7f097f8d4640 1 -- 172.21.15.32:0/1307045146 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09780cc480 con 0x7f097815d410 2024-04-24T23:10:23.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.065+0000 7f0975ffb640 1 -- 172.21.15.32:0/1307045146 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f096c01d070 con 0x7f097815d410 2024-04-24T23:10:23.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.065+0000 7f0975ffb640 1 --2- 172.21.15.32:0/1307045146 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f096406be60 0x7f096406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:23.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.066+0000 7f0975ffb640 1 -- 172.21.15.32:0/1307045146 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(176..176 src has 1..176) ==== 4943+0+0 (secure 0 0 0) 0x7f096c08a490 con 0x7f097815d410 2024-04-24T23:10:23.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.066+0000 7f097d670640 1 --2- 172.21.15.32:0/1307045146 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f096406be60 0x7f096406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:23.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.066+0000 7f097d670640 1 --2- 172.21.15.32:0/1307045146 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f096406be60 0x7f096406e320 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f0968033000 tx=0x7f0968038040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:23.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.070+0000 7f0975ffb640 1 -- 172.21.15.32:0/1307045146 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f096c0590b0 con 0x7f097815d410 2024-04-24T23:10:23.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.257+0000 7f097f8d4640 1 -- 172.21.15.32:0/1307045146 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f097806c000 con 0x7f097815d410 2024-04-24T23:10:23.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.257+0000 7f0975ffb640 1 -- 172.21.15.32:0/1307045146 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v176) ==== 74+0+13043 (secure 0 0 0) 0x7f096c0589b0 con 0x7f097815d410 2024-04-24T23:10:23.259 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:10:23.259 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":176,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:10:21.853437+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6834/4273183179":"2024-04-25T23:10:18.848186+0000","172.21.15.32:6832/4273183179":"2024-04-25T23:10:18.848186+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:10:23.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.259+0000 7f097f8d4640 1 -- 172.21.15.32:0/1307045146 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f096406be60 msgr2=0x7f096406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:23.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.259+0000 7f097f8d4640 1 --2- 172.21.15.32:0/1307045146 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f096406be60 0x7f096406e320 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f0968033000 tx=0x7f0968038040 comp rx=0 tx=0).stop 2024-04-24T23:10:23.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.259+0000 7f097f8d4640 1 -- 172.21.15.32:0/1307045146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f097815d410 msgr2=0x7f09780ace80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:23.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.259+0000 7f097f8d4640 1 --2- 172.21.15.32:0/1307045146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f097815d410 0x7f09780ace80 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f096c00f7d0 tx=0x7f096c00fc90 comp rx=0 tx=0).stop 2024-04-24T23:10:23.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.259+0000 7f097f8d4640 1 -- 172.21.15.32:0/1307045146 shutdown_connections 2024-04-24T23:10:23.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.260+0000 7f097f8d4640 1 --2- 172.21.15.32:0/1307045146 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f096406be60 0x7f096406e320 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:23.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.260+0000 7f097f8d4640 1 --2- 172.21.15.32:0/1307045146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f097815d410 0x7f09780ace80 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:23.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.260+0000 7f097f8d4640 1 --2- 172.21.15.32:0/1307045146 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f097815d870 0x7f09780ad3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:23.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.260+0000 7f097f8d4640 1 --2- 172.21.15.32:0/1307045146 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0960003b90 0x7f09781483b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:23.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.260+0000 7f097f8d4640 1 -- 172.21.15.32:0/1307045146 >> 172.21.15.32:0/1307045146 conn(0x7f097806b110 msgr2=0x7f09780727a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:23.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.260+0000 7f097f8d4640 1 -- 172.21.15.32:0/1307045146 shutdown_connections 2024-04-24T23:10:23.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.260+0000 7f097f8d4640 1 -- 172.21.15.32:0/1307045146 wait complete. 2024-04-24T23:10:23.272 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:10:23.272 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:10:23.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.375+0000 7f7106f09640 1 Processor -- start 2024-04-24T23:10:23.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.375+0000 7f7106f09640 1 -- start start 2024-04-24T23:10:23.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.376+0000 7f7106f09640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f710015b380 0x7f710006c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:23.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.376+0000 7f7106f09640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f710015cb00 0x7f710006cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:23.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.376+0000 7f7106f09640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71000ce430 0x7f71000ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:23.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.376+0000 7f7106f09640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f71000b4520 con 0x7f710015b380 2024-04-24T23:10:23.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.376+0000 7f7106f09640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f71000b4800 con 0x7f710015cb00 2024-04-24T23:10:23.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.376+0000 7f7106f09640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f71000b4ae0 con 0x7f71000ce430 2024-04-24T23:10:23.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.376+0000 7f7104ca5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f710015b380 0x7f710006c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:23.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.376+0000 7f7104ca5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f710015b380 0x7f710006c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42250/0 (socket says 172.21.15.32:42250) 2024-04-24T23:10:23.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.376+0000 7f7104ca5640 1 -- 172.21.15.32:0/487391848 learned_addr learned my addr 172.21.15.32:0/487391848 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:23.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.376+0000 7f71054a6640 1 --2- 172.21.15.32:0/487391848 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71000ce430 0x7f71000ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:23.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.376+0000 7f70fffff640 1 --2- 172.21.15.32:0/487391848 >> v2:172.21.15.121:3300/0 conn(0x7f710015cb00 0x7f710006cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:23.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.376+0000 7f7104ca5640 1 -- 172.21.15.32:0/487391848 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71000ce430 msgr2=0x7f71000ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:23.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.376+0000 7f7104ca5640 1 --2- 172.21.15.32:0/487391848 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71000ce430 0x7f71000ce860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:23.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.376+0000 7f7104ca5640 1 -- 172.21.15.32:0/487391848 >> v2:172.21.15.121:3300/0 conn(0x7f710015cb00 msgr2=0x7f710006cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:23.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.376+0000 7f7104ca5640 1 --2- 172.21.15.32:0/487391848 >> v2:172.21.15.121:3300/0 conn(0x7f710015cb00 0x7f710006cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:23.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.376+0000 7f7104ca5640 1 -- 172.21.15.32:0/487391848 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71000b4dc0 con 0x7f710015b380 2024-04-24T23:10:23.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.377+0000 7f7104ca5640 1 --2- 172.21.15.32:0/487391848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f710015b380 0x7f710006c5f0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f71000d12e0 tx=0x7f70f402a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff09a8fd2a486985 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:23.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.377+0000 7f70ff7fe640 1 -- 172.21.15.32:0/487391848 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f4037070 con 0x7f710015b380 2024-04-24T23:10:23.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.377+0000 7f70ff7fe640 1 -- 172.21.15.32:0/487391848 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70f402ae00 con 0x7f710015b380 2024-04-24T23:10:23.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.377+0000 7f70ff7fe640 1 -- 172.21.15.32:0/487391848 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f4032660 con 0x7f710015b380 2024-04-24T23:10:23.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.377+0000 7f7106f09640 1 -- 172.21.15.32:0/487391848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f710015b380 msgr2=0x7f710006c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:23.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.377+0000 7f7106f09640 1 --2- 172.21.15.32:0/487391848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f710015b380 0x7f710006c5f0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f71000d12e0 tx=0x7f70f402a0b0 comp rx=0 tx=0).stop 2024-04-24T23:10:23.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.377+0000 7f7106f09640 1 -- 172.21.15.32:0/487391848 shutdown_connections 2024-04-24T23:10:23.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.377+0000 7f7106f09640 1 --2- 172.21.15.32:0/487391848 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71000ce430 0x7f71000ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:23.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.377+0000 7f7106f09640 1 --2- 172.21.15.32:0/487391848 >> v2:172.21.15.121:3300/0 conn(0x7f710015cb00 0x7f710006cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:23.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.377+0000 7f7106f09640 1 --2- 172.21.15.32:0/487391848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f710015b380 0x7f710006c5f0 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:23.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.377+0000 7f7106f09640 1 -- 172.21.15.32:0/487391848 >> 172.21.15.32:0/487391848 conn(0x7f710006b110 msgr2=0x7f710006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:23.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.377+0000 7f7106f09640 1 -- 172.21.15.32:0/487391848 shutdown_connections 2024-04-24T23:10:23.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.377+0000 7f7106f09640 1 -- 172.21.15.32:0/487391848 wait complete. 2024-04-24T23:10:23.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.379+0000 7f7106f09640 1 Processor -- start 2024-04-24T23:10:23.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.379+0000 7f7106f09640 1 -- start start 2024-04-24T23:10:23.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.379+0000 7f7106f09640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71000ce430 0x7f7100153920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:23.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.379+0000 7f7106f09640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f710015b380 0x7f7100071100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:23.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.379+0000 7f7106f09640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f710015cb00 0x7f71001478b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:23.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.379+0000 7f7106f09640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f71000ceda0 con 0x7f71000ce430 2024-04-24T23:10:23.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.379+0000 7f7106f09640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f710015b760 con 0x7f710015b380 2024-04-24T23:10:23.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.379+0000 7f7106f09640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f71000cef20 con 0x7f710015cb00 2024-04-24T23:10:23.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.380+0000 7f7104ca5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71000ce430 0x7f7100153920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:23.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.380+0000 7f7104ca5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71000ce430 0x7f7100153920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42260/0 (socket says 172.21.15.32:42260) 2024-04-24T23:10:23.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.380+0000 7f7104ca5640 1 -- 172.21.15.32:0/1738690237 learned_addr learned my addr 172.21.15.32:0/1738690237 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:23.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.380+0000 7f71054a6640 1 --2- 172.21.15.32:0/1738690237 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f710015cb00 0x7f71001478b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:23.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.380+0000 7f70fffff640 1 --2- 172.21.15.32:0/1738690237 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f710015b380 0x7f7100071100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:23.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.380+0000 7f7104ca5640 1 -- 172.21.15.32:0/1738690237 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f710015cb00 msgr2=0x7f71001478b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:23.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.380+0000 7f7104ca5640 1 --2- 172.21.15.32:0/1738690237 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f710015cb00 0x7f71001478b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:23.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.380+0000 7f7104ca5640 1 -- 172.21.15.32:0/1738690237 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f710015b380 msgr2=0x7f7100071100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:23.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.380+0000 7f7104ca5640 1 --2- 172.21.15.32:0/1738690237 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f710015b380 0x7f7100071100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:23.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.380+0000 7f7104ca5640 1 -- 172.21.15.32:0/1738690237 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7100071640 con 0x7f71000ce430 2024-04-24T23:10:23.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.380+0000 7f7104ca5640 1 --2- 172.21.15.32:0/1738690237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71000ce430 0x7f7100153920 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f70f4000c00 tx=0x7f70f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:23.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.380+0000 7f70fd7fa640 1 -- 172.21.15.32:0/1738690237 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f4041030 con 0x7f71000ce430 2024-04-24T23:10:23.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.380+0000 7f70fd7fa640 1 -- 172.21.15.32:0/1738690237 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70f402ac00 con 0x7f71000ce430 2024-04-24T23:10:23.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.380+0000 7f7106f09640 1 -- 172.21.15.32:0/1738690237 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7100151430 con 0x7f71000ce430 2024-04-24T23:10:23.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.380+0000 7f70fd7fa640 1 -- 172.21.15.32:0/1738690237 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f4037040 con 0x7f71000ce430 2024-04-24T23:10:23.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.380+0000 7f7106f09640 1 -- 172.21.15.32:0/1738690237 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71001519f0 con 0x7f71000ce430 2024-04-24T23:10:23.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.381+0000 7f7106f09640 1 -- 172.21.15.32:0/1738690237 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7100068430 con 0x7f71000ce430 2024-04-24T23:10:23.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.382+0000 7f70fd7fa640 1 -- 172.21.15.32:0/1738690237 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f70f4044070 con 0x7f71000ce430 2024-04-24T23:10:23.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.382+0000 7f70fd7fa640 1 --2- 172.21.15.32:0/1738690237 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f70e406be30 0x7f70e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:23.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.382+0000 7f70fd7fa640 1 -- 172.21.15.32:0/1738690237 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(176..176 src has 1..176) ==== 4943+0+0 (secure 0 0 0) 0x7f70f40aaf80 con 0x7f71000ce430 2024-04-24T23:10:23.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.383+0000 7f70fffff640 1 --2- 172.21.15.32:0/1738690237 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f70e406be30 0x7f70e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:23.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.383+0000 7f70fffff640 1 --2- 172.21.15.32:0/1738690237 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f70e406be30 0x7f70e406e2f0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f7100150bc0 tx=0x7f70ec008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:23.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.387+0000 7f70fd7fa640 1 -- 172.21.15.32:0/1738690237 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f70f4079ba0 con 0x7f71000ce430 2024-04-24T23:10:23.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.568+0000 7f7106f09640 1 -- 172.21.15.32:0/1738690237 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f71000afd60 con 0x7f71000ce430 2024-04-24T23:10:23.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.568+0000 7f70fd7fa640 1 -- 172.21.15.32:0/1738690237 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v176) ==== 93+0+10 (secure 0 0 0) 0x7f70f40794a0 con 0x7f71000ce430 2024-04-24T23:10:23.570 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T23:10:23.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.570+0000 7f7106f09640 1 -- 172.21.15.32:0/1738690237 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f70e406be30 msgr2=0x7f70e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:23.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.570+0000 7f7106f09640 1 --2- 172.21.15.32:0/1738690237 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f70e406be30 0x7f70e406e2f0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f7100150bc0 tx=0x7f70ec008040 comp rx=0 tx=0).stop 2024-04-24T23:10:23.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.570+0000 7f7106f09640 1 -- 172.21.15.32:0/1738690237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71000ce430 msgr2=0x7f7100153920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:23.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.570+0000 7f7106f09640 1 --2- 172.21.15.32:0/1738690237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71000ce430 0x7f7100153920 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f70f4000c00 tx=0x7f70f40046c0 comp rx=0 tx=0).stop 2024-04-24T23:10:23.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.570+0000 7f7106f09640 1 -- 172.21.15.32:0/1738690237 shutdown_connections 2024-04-24T23:10:23.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.570+0000 7f7106f09640 1 --2- 172.21.15.32:0/1738690237 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f70e406be30 0x7f70e406e2f0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:23.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.570+0000 7f7106f09640 1 --2- 172.21.15.32:0/1738690237 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f710015cb00 0x7f71001478b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:23.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.570+0000 7f7106f09640 1 --2- 172.21.15.32:0/1738690237 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f710015b380 0x7f7100071100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:23.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.570+0000 7f7106f09640 1 --2- 172.21.15.32:0/1738690237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71000ce430 0x7f7100153920 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:23.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.570+0000 7f7106f09640 1 -- 172.21.15.32:0/1738690237 >> 172.21.15.32:0/1738690237 conn(0x7f710006b110 msgr2=0x7f7100152ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:23.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.570+0000 7f7106f09640 1 -- 172.21.15.32:0/1738690237 shutdown_connections 2024-04-24T23:10:23.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.570+0000 7f7106f09640 1 -- 172.21.15.32:0/1738690237 wait complete. 2024-04-24T23:10:23.582 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file' 2024-04-24T23:10:23.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.682+0000 7fe752eca640 1 Processor -- start 2024-04-24T23:10:23.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.682+0000 7fe752eca640 1 -- start start 2024-04-24T23:10:23.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.683+0000 7fe752eca640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe74c0b44d0 0x7fe74c0b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:23.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.683+0000 7fe752eca640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe74c0af890 0x7fe74c15cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:23.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.683+0000 7fe752eca640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe74c0b6430 con 0x7fe74c06c5f0 2024-04-24T23:10:23.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.683+0000 7fe752eca640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fe74c156f80 con 0x7fe74c0b44d0 2024-04-24T23:10:23.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.683+0000 7fe752eca640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe74c0b6150 con 0x7fe74c0af890 2024-04-24T23:10:23.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.683+0000 7fe750c66640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe74c06c5f0 0x7fe74c06c9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59894/0 (socket says 172.21.15.32:59894) 2024-04-24T23:10:23.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.683+0000 7fe750c66640 1 -- 172.21.15.32:0/2816582717 learned_addr learned my addr 172.21.15.32:0/2816582717 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:10:23.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.683+0000 7fe74bfff640 1 --2- 172.21.15.32:0/2816582717 >> v2:172.21.15.121:3300/0 conn(0x7fe74c0b44d0 0x7fe74c0b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:23.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.683+0000 7fe74b7fe640 1 -- 172.21.15.32:0/2816582717 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2791261259 0 0) 0x7fe74c0b6430 con 0x7fe74c06c5f0 2024-04-24T23:10:23.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.683+0000 7fe751467640 1 --2- 172.21.15.32:0/2816582717 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe74c0af890 0x7fe74c15cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:23.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.683+0000 7fe74b7fe640 1 -- 172.21.15.32:0/2816582717 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe73c003520 con 0x7fe74c06c5f0 2024-04-24T23:10:23.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.683+0000 7fe74b7fe640 1 -- 172.21.15.32:0/2816582717 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2145420549 0 0) 0x7fe73c003520 con 0x7fe74c06c5f0 2024-04-24T23:10:23.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.683+0000 7fe74b7fe640 1 -- 172.21.15.32:0/2816582717 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe74c0af890 msgr2=0x7fe74c15cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:23.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.683+0000 7fe74b7fe640 1 --2- 172.21.15.32:0/2816582717 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe74c0af890 0x7fe74c15cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:23.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.683+0000 7fe74b7fe640 1 -- 172.21.15.32:0/2816582717 >> v2:172.21.15.121:3300/0 conn(0x7fe74c0b44d0 msgr2=0x7fe74c0b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:23.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.683+0000 7fe74b7fe640 1 --2- 172.21.15.32:0/2816582717 >> v2:172.21.15.121:3300/0 conn(0x7fe74c0b44d0 0x7fe74c0b5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:23.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.683+0000 7fe74b7fe640 1 -- 172.21.15.32:0/2816582717 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe74c0b39f0 con 0x7fe74c06c5f0 2024-04-24T23:10:23.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.683+0000 7fe74b7fe640 1 -- 172.21.15.32:0/2816582717 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7fe740001690 con 0x7fe74c06c5f0 2024-04-24T23:10:23.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.684+0000 7fe74b7fe640 1 -- 172.21.15.32:0/2816582717 >> v1:172.21.15.32:6789/0 conn(0x7fe74c06c5f0 legacy=0x7fe74c06c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:23.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.684+0000 7fe74b7fe640 1 --2- 172.21.15.32:0/2816582717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe73c0039b0 0x7fe73c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:23.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.684+0000 7fe74b7fe640 1 --2- 172.21.15.32:0/2816582717 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe73c004800 0x7fe73c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:23.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.684+0000 7fe74b7fe640 1 --2- 172.21.15.32:0/2816582717 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe74c0af890 0x7fe73c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:23.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.684+0000 7fe74b7fe640 1 -- 172.21.15.32:0/2816582717 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe74c156f80 con 0x7fe73c0039b0 2024-04-24T23:10:23.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.684+0000 7fe74b7fe640 1 -- 172.21.15.32:0/2816582717 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe74c0b6150 con 0x7fe73c004800 2024-04-24T23:10:23.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.684+0000 7fe74b7fe640 1 -- 172.21.15.32:0/2816582717 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe73c00b9c0 con 0x7fe74c0af890 2024-04-24T23:10:23.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.684+0000 7fe750c66640 1 --2- 172.21.15.32:0/2816582717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe73c0039b0 0x7fe73c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:23.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.684+0000 7fe74bfff640 1 --2- 172.21.15.32:0/2816582717 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe73c004800 0x7fe73c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:23.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.684+0000 7fe751467640 1 --2- 172.21.15.32:0/2816582717 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe74c0af890 0x7fe73c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:23.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.684+0000 7fe750c66640 1 -- 172.21.15.32:0/2816582717 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe74c0af890 msgr2=0x7fe73c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:23.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.684+0000 7fe750c66640 1 --2- 172.21.15.32:0/2816582717 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe74c0af890 0x7fe73c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:23.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.684+0000 7fe750c66640 1 -- 172.21.15.32:0/2816582717 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe73c004800 msgr2=0x7fe73c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:23.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.684+0000 7fe750c66640 1 --2- 172.21.15.32:0/2816582717 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe73c004800 0x7fe73c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:23.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.684+0000 7fe750c66640 1 -- 172.21.15.32:0/2816582717 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe73c003520 con 0x7fe73c0039b0 2024-04-24T23:10:23.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.685+0000 7fe750c66640 1 --2- 172.21.15.32:0/2816582717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe73c0039b0 0x7fe73c003dd0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fe74000c210 tx=0x7fe73c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46264460dc0e73f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:23.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.685+0000 7fe74b7fe640 1 -- 172.21.15.32:0/2816582717 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe740007d50 con 0x7fe73c0039b0 2024-04-24T23:10:23.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.685+0000 7fe74b7fe640 1 -- 172.21.15.32:0/2816582717 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7400058d0 con 0x7fe73c0039b0 2024-04-24T23:10:23.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.685+0000 7fe74b7fe640 1 -- 172.21.15.32:0/2816582717 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe740005ca0 con 0x7fe73c0039b0 2024-04-24T23:10:23.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.685+0000 7fe752eca640 1 -- 172.21.15.32:0/2816582717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe73c0039b0 msgr2=0x7fe73c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:23.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.685+0000 7fe752eca640 1 --2- 172.21.15.32:0/2816582717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe73c0039b0 0x7fe73c003dd0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fe74000c210 tx=0x7fe73c005830 comp rx=0 tx=0).stop 2024-04-24T23:10:23.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.685+0000 7fe751467640 1 -- 172.21.15.32:0/2816582717 reap_dead start 2024-04-24T23:10:23.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.685+0000 7fe752eca640 1 -- 172.21.15.32:0/2816582717 shutdown_connections 2024-04-24T23:10:23.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.685+0000 7fe752eca640 1 -- 172.21.15.32:0/2816582717 >> 172.21.15.32:0/2816582717 conn(0x7fe74c06b110 msgr2=0x7fe74c06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:23.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.685+0000 7fe752eca640 1 -- 172.21.15.32:0/2816582717 shutdown_connections 2024-04-24T23:10:23.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.686+0000 7fe752eca640 1 -- 172.21.15.32:0/2816582717 wait complete. 2024-04-24T23:10:23.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.687+0000 7fe752eca640 1 Processor -- start 2024-04-24T23:10:23.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.687+0000 7fe752eca640 1 -- start start 2024-04-24T23:10:23.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.688+0000 7fe752eca640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe73c0039b0 0x7fe74c0b0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:23.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.688+0000 7fe752eca640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe74c15f520 0x7fe74c14ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:23.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.688+0000 7fe752eca640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe74c15f9a0 0x7fe74c14b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:23.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.688+0000 7fe752eca640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe74c0b5dd0 con 0x7fe73c0039b0 2024-04-24T23:10:23.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.688+0000 7fe752eca640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe74c0b3fb0 con 0x7fe74c15f9a0 2024-04-24T23:10:23.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.688+0000 7fe752eca640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe74c15cac0 con 0x7fe74c15f520 2024-04-24T23:10:23.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.688+0000 7fe750c66640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe73c0039b0 0x7fe74c0b0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:23.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.688+0000 7fe750c66640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe73c0039b0 0x7fe74c0b0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42288/0 (socket says 172.21.15.32:42288) 2024-04-24T23:10:23.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.688+0000 7fe750c66640 1 -- 172.21.15.32:0/3338640935 learned_addr learned my addr 172.21.15.32:0/3338640935 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:23.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.688+0000 7fe751467640 1 --2- 172.21.15.32:0/3338640935 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe74c15f9a0 0x7fe74c14b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:23.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.688+0000 7fe74bfff640 1 --2- 172.21.15.32:0/3338640935 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe74c15f520 0x7fe74c14ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:23.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.688+0000 7fe750c66640 1 -- 172.21.15.32:0/3338640935 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe74c15f520 msgr2=0x7fe74c14ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:23.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.688+0000 7fe750c66640 1 --2- 172.21.15.32:0/3338640935 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe74c15f520 0x7fe74c14ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:23.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.688+0000 7fe750c66640 1 -- 172.21.15.32:0/3338640935 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe74c15f9a0 msgr2=0x7fe74c14b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:23.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.688+0000 7fe750c66640 1 --2- 172.21.15.32:0/3338640935 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe74c15f9a0 0x7fe74c14b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:23.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.688+0000 7fe750c66640 1 -- 172.21.15.32:0/3338640935 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe74c158890 con 0x7fe73c0039b0 2024-04-24T23:10:23.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.688+0000 7fe750c66640 1 --2- 172.21.15.32:0/3338640935 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe73c0039b0 0x7fe74c0b0da0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fe74000c330 tx=0x7fe74c0b3bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:23.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.688+0000 7fe7497fa640 1 -- 172.21.15.32:0/3338640935 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe740043070 con 0x7fe73c0039b0 2024-04-24T23:10:23.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.688+0000 7fe7497fa640 1 -- 172.21.15.32:0/3338640935 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe740007070 con 0x7fe73c0039b0 2024-04-24T23:10:23.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.688+0000 7fe7497fa640 1 -- 172.21.15.32:0/3338640935 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe740038040 con 0x7fe73c0039b0 2024-04-24T23:10:23.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.688+0000 7fe752eca640 1 -- 172.21.15.32:0/3338640935 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe74c158b30 con 0x7fe73c0039b0 2024-04-24T23:10:23.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.689+0000 7fe752eca640 1 -- 172.21.15.32:0/3338640935 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe74c1590f0 con 0x7fe73c0039b0 2024-04-24T23:10:23.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.690+0000 7fe752eca640 1 -- 172.21.15.32:0/3338640935 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe74c068430 con 0x7fe73c0039b0 2024-04-24T23:10:23.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.690+0000 7fe7497fa640 1 -- 172.21.15.32:0/3338640935 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe740046070 con 0x7fe73c0039b0 2024-04-24T23:10:23.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.690+0000 7fe7497fa640 1 --2- 172.21.15.32:0/3338640935 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe73006be60 0x7fe73006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:23.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.691+0000 7fe7497fa640 1 -- 172.21.15.32:0/3338640935 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(176..176 src has 1..176) ==== 4943+0+0 (secure 0 0 0) 0x7fe7400aa4d0 con 0x7fe73c0039b0 2024-04-24T23:10:23.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.691+0000 7fe74bfff640 1 --2- 172.21.15.32:0/3338640935 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe73006be60 0x7fe73006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:23.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.691+0000 7fe74bfff640 1 --2- 172.21.15.32:0/3338640935 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe73006be60 0x7fe73006e320 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fe74c06cad0 tx=0x7fe738008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:23.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.695+0000 7fe7497fa640 1 -- 172.21.15.32:0/3338640935 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe740079210 con 0x7fe73c0039b0 2024-04-24T23:10:23.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:23.946+0000 7fe752eca640 1 -- 172.21.15.32:0/3338640935 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]} v 0) -- 0x7fe74c0b5670 con 0x7fe73c0039b0 2024-04-24T23:10:24.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:24.946+0000 7fe7497fa640 1 -- 172.21.15.32:0/3338640935 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]}]=0 v536) ==== 138+0+0 (secure 0 0 0) 0x7fe740078b10 con 0x7fe73c0039b0 2024-04-24T23:10:24.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:24.949+0000 7fe752eca640 1 -- 172.21.15.32:0/3338640935 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe73006be60 msgr2=0x7fe73006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:24.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:24.949+0000 7fe752eca640 1 --2- 172.21.15.32:0/3338640935 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe73006be60 0x7fe73006e320 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fe74c06cad0 tx=0x7fe738008040 comp rx=0 tx=0).stop 2024-04-24T23:10:24.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:24.949+0000 7fe752eca640 1 -- 172.21.15.32:0/3338640935 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe73c0039b0 msgr2=0x7fe74c0b0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:24.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:24.949+0000 7fe752eca640 1 --2- 172.21.15.32:0/3338640935 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe73c0039b0 0x7fe74c0b0da0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fe74000c330 tx=0x7fe74c0b3bc0 comp rx=0 tx=0).stop 2024-04-24T23:10:24.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:24.949+0000 7fe752eca640 1 -- 172.21.15.32:0/3338640935 shutdown_connections 2024-04-24T23:10:24.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:24.949+0000 7fe752eca640 1 --2- 172.21.15.32:0/3338640935 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe73006be60 0x7fe73006e320 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:24.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:24.949+0000 7fe752eca640 1 --2- 172.21.15.32:0/3338640935 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe74c15f9a0 0x7fe74c14b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:24.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:24.949+0000 7fe752eca640 1 --2- 172.21.15.32:0/3338640935 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe74c15f520 0x7fe74c14ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:24.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:24.949+0000 7fe752eca640 1 --2- 172.21.15.32:0/3338640935 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe73c0039b0 0x7fe74c0b0da0 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:24.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:24.949+0000 7fe752eca640 1 -- 172.21.15.32:0/3338640935 >> 172.21.15.32:0/3338640935 conn(0x7fe74c06b110 msgr2=0x7fe74c06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:24.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:24.949+0000 7fe752eca640 1 -- 172.21.15.32:0/3338640935 shutdown_connections 2024-04-24T23:10:24.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:24.949+0000 7fe752eca640 1 -- 172.21.15.32:0/3338640935 wait complete. 2024-04-24T23:10:24.964 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-24T23:10:25.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.072+0000 7f80f0d5a640 1 Processor -- start 2024-04-24T23:10:25.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.072+0000 7f80f0d5a640 1 -- start start 2024-04-24T23:10:25.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.073+0000 7f80f0d5a640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f80ec15b380 0x7f80ec06c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:25.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.073+0000 7f80f0d5a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f80ec0ceb40 0x7f80ec06cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:25.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.073+0000 7f80f0d5a640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80ec0b4520 con 0x7f80ec15cb00 2024-04-24T23:10:25.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.073+0000 7f80f0d5a640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f80ec0cef70 con 0x7f80ec15b380 2024-04-24T23:10:25.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.073+0000 7f80f0d5a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f80ec0b4800 con 0x7f80ec0ceb40 2024-04-24T23:10:25.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.073+0000 7f80e9d9b640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f80ec15cb00 0x7f80ec0ce430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59900/0 (socket says 172.21.15.32:59900) 2024-04-24T23:10:25.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.073+0000 7f80e9d9b640 1 -- 172.21.15.32:0/1233580438 learned_addr learned my addr 172.21.15.32:0/1233580438 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:10:25.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.073+0000 7f80ead9d640 1 --2- 172.21.15.32:0/1233580438 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f80ec0ceb40 0x7f80ec06cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:25.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.073+0000 7f80ea59c640 1 --2- 172.21.15.32:0/1233580438 >> v2:172.21.15.121:3300/0 conn(0x7f80ec15b380 0x7f80ec06c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:25.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.073+0000 7f80e959a640 1 -- 172.21.15.32:0/1233580438 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1476196006 0 0) 0x7f80ec0b4520 con 0x7f80ec15cb00 2024-04-24T23:10:25.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.073+0000 7f80e959a640 1 -- 172.21.15.32:0/1233580438 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f80d8005730 con 0x7f80ec15cb00 2024-04-24T23:10:25.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.074+0000 7f80e959a640 1 -- 172.21.15.32:0/1233580438 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1398677171 0 0) 0x7f80d8005730 con 0x7f80ec15cb00 2024-04-24T23:10:25.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.074+0000 7f80e959a640 1 -- 172.21.15.32:0/1233580438 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f80ec0ceb40 msgr2=0x7f80ec06cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:25.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.074+0000 7f80e959a640 1 --2- 172.21.15.32:0/1233580438 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f80ec0ceb40 0x7f80ec06cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:25.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.074+0000 7f80e959a640 1 -- 172.21.15.32:0/1233580438 >> v2:172.21.15.121:3300/0 conn(0x7f80ec15b380 msgr2=0x7f80ec06c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:25.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.074+0000 7f80e959a640 1 --2- 172.21.15.32:0/1233580438 >> v2:172.21.15.121:3300/0 conn(0x7f80ec15b380 0x7f80ec06c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:25.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.074+0000 7f80e959a640 1 -- 172.21.15.32:0/1233580438 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80ec0b4ae0 con 0x7f80ec15cb00 2024-04-24T23:10:25.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.074+0000 7f80e959a640 1 -- 172.21.15.32:0/1233580438 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f80d4001670 con 0x7f80ec15cb00 2024-04-24T23:10:25.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.074+0000 7f80e959a640 1 -- 172.21.15.32:0/1233580438 >> v1:172.21.15.32:6789/0 conn(0x7f80ec15cb00 legacy=0x7f80ec0ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:25.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.074+0000 7f80e959a640 1 --2- 172.21.15.32:0/1233580438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80d8005bc0 0x7f80d8005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:25.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.074+0000 7f80e959a640 1 --2- 172.21.15.32:0/1233580438 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f80ec0ceb40 0x7f80d8006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:25.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.074+0000 7f80e959a640 1 --2- 172.21.15.32:0/1233580438 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80d8006f50 0x7f80d80073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:25.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.074+0000 7f80e959a640 1 -- 172.21.15.32:0/1233580438 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f80ec0cef70 con 0x7f80d8005bc0 2024-04-24T23:10:25.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.074+0000 7f80e959a640 1 -- 172.21.15.32:0/1233580438 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f80ec0b4800 con 0x7f80d8006f50 2024-04-24T23:10:25.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.074+0000 7f80e959a640 1 -- 172.21.15.32:0/1233580438 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f80d800dc50 con 0x7f80ec0ceb40 2024-04-24T23:10:25.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.074+0000 7f80ea59c640 1 --2- 172.21.15.32:0/1233580438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80d8005bc0 0x7f80d8005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:25.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.074+0000 7f80ead9d640 1 --2- 172.21.15.32:0/1233580438 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80d8006f50 0x7f80d80073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:25.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.074+0000 7f80e9d9b640 1 --2- 172.21.15.32:0/1233580438 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f80ec0ceb40 0x7f80d8006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:25.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.074+0000 7f80ea59c640 1 -- 172.21.15.32:0/1233580438 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f80ec0ceb40 msgr2=0x7f80d8006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:25.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.074+0000 7f80ea59c640 1 --2- 172.21.15.32:0/1233580438 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f80ec0ceb40 0x7f80d8006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:25.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.074+0000 7f80ea59c640 1 -- 172.21.15.32:0/1233580438 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80d8006f50 msgr2=0x7f80d80073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:25.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.074+0000 7f80ea59c640 1 --2- 172.21.15.32:0/1233580438 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80d8006f50 0x7f80d80073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:25.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.074+0000 7f80ea59c640 1 -- 172.21.15.32:0/1233580438 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80d8005730 con 0x7f80d8005bc0 2024-04-24T23:10:25.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.075+0000 7f80ea59c640 1 --2- 172.21.15.32:0/1233580438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80d8005bc0 0x7f80d8005fe0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f80e4009fc0 tx=0x7f80d8007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6e4938bd59719db server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:25.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.075+0000 7f80e959a640 1 -- 172.21.15.32:0/1233580438 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e4009c10 con 0x7f80d8005bc0 2024-04-24T23:10:25.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.075+0000 7f80e959a640 1 -- 172.21.15.32:0/1233580438 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80e4036070 con 0x7f80d8005bc0 2024-04-24T23:10:25.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.075+0000 7f80e959a640 1 -- 172.21.15.32:0/1233580438 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e402d070 con 0x7f80d8005bc0 2024-04-24T23:10:25.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.076+0000 7f80f0d5a640 1 -- 172.21.15.32:0/1233580438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80d8005bc0 msgr2=0x7f80d8005fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:25.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.076+0000 7f80f0d5a640 1 --2- 172.21.15.32:0/1233580438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80d8005bc0 0x7f80d8005fe0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f80e4009fc0 tx=0x7f80d8007a40 comp rx=0 tx=0).stop 2024-04-24T23:10:25.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.076+0000 7f80ead9d640 1 -- 172.21.15.32:0/1233580438 reap_dead start 2024-04-24T23:10:25.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.076+0000 7f80f0d5a640 1 -- 172.21.15.32:0/1233580438 shutdown_connections 2024-04-24T23:10:25.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.076+0000 7f80f0d5a640 1 -- 172.21.15.32:0/1233580438 >> 172.21.15.32:0/1233580438 conn(0x7f80ec06b110 msgr2=0x7f80ec06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:25.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.076+0000 7f80f0d5a640 1 -- 172.21.15.32:0/1233580438 shutdown_connections 2024-04-24T23:10:25.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.076+0000 7f80f0d5a640 1 -- 172.21.15.32:0/1233580438 wait complete. 2024-04-24T23:10:25.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.077+0000 7f80f0d5a640 1 Processor -- start 2024-04-24T23:10:25.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.077+0000 7f80f0d5a640 1 -- start start 2024-04-24T23:10:25.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.078+0000 7f80f0d5a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f80d8005bc0 0x7f80ec14ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:25.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.078+0000 7f80f0d5a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80ec14c0a0 0x7f80ec070680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:25.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.078+0000 7f80f0d5a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80ec14c5f0 0x7f80ec14ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:25.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.078+0000 7f80f0d5a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f80ec15b760 con 0x7f80ec14c5f0 2024-04-24T23:10:25.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.078+0000 7f80f0d5a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f80ec0b50a0 con 0x7f80ec14c0a0 2024-04-24T23:10:25.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.078+0000 7f80f0d5a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f80ec06cb30 con 0x7f80d8005bc0 2024-04-24T23:10:25.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.078+0000 7f80ead9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80ec14c5f0 0x7f80ec14ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:25.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.078+0000 7f80ead9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80ec14c5f0 0x7f80ec14ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42294/0 (socket says 172.21.15.32:42294) 2024-04-24T23:10:25.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.078+0000 7f80ead9d640 1 -- 172.21.15.32:0/3859607012 learned_addr learned my addr 172.21.15.32:0/3859607012 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:25.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.078+0000 7f80ea59c640 1 --2- 172.21.15.32:0/3859607012 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f80d8005bc0 0x7f80ec14ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:25.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.078+0000 7f80e9d9b640 1 --2- 172.21.15.32:0/3859607012 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80ec14c0a0 0x7f80ec070680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:25.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.078+0000 7f80ead9d640 1 -- 172.21.15.32:0/3859607012 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f80d8005bc0 msgr2=0x7f80ec14ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:25.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.078+0000 7f80ead9d640 1 --2- 172.21.15.32:0/3859607012 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f80d8005bc0 0x7f80ec14ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:25.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.078+0000 7f80ead9d640 1 -- 172.21.15.32:0/3859607012 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80ec14c0a0 msgr2=0x7f80ec070680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:25.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.078+0000 7f80ead9d640 1 --2- 172.21.15.32:0/3859607012 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80ec14c0a0 0x7f80ec070680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:25.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.078+0000 7f80ead9d640 1 -- 172.21.15.32:0/3859607012 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80ec14b140 con 0x7f80ec14c5f0 2024-04-24T23:10:25.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.079+0000 7f80ead9d640 1 --2- 172.21.15.32:0/3859607012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80ec14c5f0 0x7f80ec14ca70 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f80dc011860 tx=0x7f80dc011d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:25.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.079+0000 7f80e2ffd640 1 -- 172.21.15.32:0/3859607012 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80dc004240 con 0x7f80ec14c5f0 2024-04-24T23:10:25.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.079+0000 7f80e2ffd640 1 -- 172.21.15.32:0/3859607012 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80dc01b070 con 0x7f80ec14c5f0 2024-04-24T23:10:25.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.079+0000 7f80e2ffd640 1 -- 172.21.15.32:0/3859607012 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80dc00f7a0 con 0x7f80ec14c5f0 2024-04-24T23:10:25.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.079+0000 7f80f0d5a640 1 -- 172.21.15.32:0/3859607012 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80ec070f80 con 0x7f80ec14c5f0 2024-04-24T23:10:25.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.079+0000 7f80f0d5a640 1 -- 172.21.15.32:0/3859607012 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80ec071540 con 0x7f80ec14c5f0 2024-04-24T23:10:25.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.080+0000 7f80f0d5a640 1 -- 172.21.15.32:0/3859607012 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80ec072020 con 0x7f80ec14c5f0 2024-04-24T23:10:25.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.080+0000 7f80e2ffd640 1 -- 172.21.15.32:0/3859607012 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f80dc012040 con 0x7f80ec14c5f0 2024-04-24T23:10:25.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.081+0000 7f80e2ffd640 1 --2- 172.21.15.32:0/3859607012 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f80cc06be60 0x7f80cc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:25.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.081+0000 7f80e2ffd640 1 -- 172.21.15.32:0/3859607012 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(176..176 src has 1..176) ==== 4943+0+0 (secure 0 0 0) 0x7f80dc08a830 con 0x7f80ec14c5f0 2024-04-24T23:10:25.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.081+0000 7f80ea59c640 1 --2- 172.21.15.32:0/3859607012 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f80cc06be60 0x7f80cc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:25.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.081+0000 7f80ea59c640 1 --2- 172.21.15.32:0/3859607012 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f80cc06be60 0x7f80cc06e320 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f80e4035000 tx=0x7f80e402f000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:25.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.085+0000 7f80e2ffd640 1 -- 172.21.15.32:0/3859607012 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f80dc058df0 con 0x7f80ec14c5f0 2024-04-24T23:10:25.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.271+0000 7f80f0d5a640 1 -- 172.21.15.32:0/3859607012 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f80ec068430 con 0x7f80ec14c5f0 2024-04-24T23:10:25.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.274+0000 7f80e2ffd640 1 -- 172.21.15.32:0/3859607012 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) ==== 122+0+0 (secure 0 0 0) 0x7f80dc05d0f0 con 0x7f80ec14c5f0 2024-04-24T23:10:25.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.276+0000 7f80f0d5a640 1 -- 172.21.15.32:0/3859607012 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f80cc06be60 msgr2=0x7f80cc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:25.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.276+0000 7f80f0d5a640 1 --2- 172.21.15.32:0/3859607012 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f80cc06be60 0x7f80cc06e320 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f80e4035000 tx=0x7f80e402f000 comp rx=0 tx=0).stop 2024-04-24T23:10:25.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.276+0000 7f80f0d5a640 1 -- 172.21.15.32:0/3859607012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80ec14c5f0 msgr2=0x7f80ec14ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:25.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.276+0000 7f80f0d5a640 1 --2- 172.21.15.32:0/3859607012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80ec14c5f0 0x7f80ec14ca70 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f80dc011860 tx=0x7f80dc011d20 comp rx=0 tx=0).stop 2024-04-24T23:10:25.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.276+0000 7f80f0d5a640 1 -- 172.21.15.32:0/3859607012 shutdown_connections 2024-04-24T23:10:25.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.276+0000 7f80f0d5a640 1 --2- 172.21.15.32:0/3859607012 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f80cc06be60 0x7f80cc06e320 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:25.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.276+0000 7f80f0d5a640 1 --2- 172.21.15.32:0/3859607012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80ec14c5f0 0x7f80ec14ca70 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:25.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.276+0000 7f80f0d5a640 1 --2- 172.21.15.32:0/3859607012 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80ec14c0a0 0x7f80ec070680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:25.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.276+0000 7f80f0d5a640 1 --2- 172.21.15.32:0/3859607012 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f80d8005bc0 0x7f80ec14ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:25.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.276+0000 7f80f0d5a640 1 -- 172.21.15.32:0/3859607012 >> 172.21.15.32:0/3859607012 conn(0x7f80ec06b110 msgr2=0x7f80ec0b3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:25.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.277+0000 7f80f0d5a640 1 -- 172.21.15.32:0/3859607012 shutdown_connections 2024-04-24T23:10:25.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.277+0000 7f80f0d5a640 1 -- 172.21.15.32:0/3859607012 wait complete. 2024-04-24T23:10:25.289 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-24T23:10:25.290 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:10:25.336 DEBUG:teuthology.orchestra.run.smithi121:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:10:25.376 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-24T23:10:25.376 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:10:25.376 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:10:25.391 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=413 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T23:10:25.391 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:10:25.392 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:10:25.433 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3271 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T23:10:25.433 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:10:25.433 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:10:25.449 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=413 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T23:10:25.449 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:10:25.449 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:10:25.490 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3271 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T23:10:25.490 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-24T23:10:25.490 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:10:25.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.589+0000 7f63c6fa2640 1 Processor -- start 2024-04-24T23:10:25.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.589+0000 7f63c6fa2640 1 -- start start 2024-04-24T23:10:25.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.590+0000 7f63c6fa2640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f63c015e9e0 0x7f63c00d40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:25.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.590+0000 7f63c6fa2640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f63c006b110 0x7f63c006b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:25.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.590+0000 7f63c6fa2640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63c00d3010 con 0x7f63c00ba390 2024-04-24T23:10:25.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.590+0000 7f63c6fa2640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f63c00d2db0 con 0x7f63c015e9e0 2024-04-24T23:10:25.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.590+0000 7f63c6fa2640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f63c00d32f0 con 0x7f63c006b110 2024-04-24T23:10:25.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.591+0000 7f63c4d3e640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f63c015e9e0 0x7f63c00d40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:25.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.591+0000 7f63c553f640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f63c006b110 0x7f63c006b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:25.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.591+0000 7f63bffff640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f63c00ba390 0x7f63c00d26a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:44920/0 (socket says 172.21.15.32:44920) 2024-04-24T23:10:25.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.591+0000 7f63bffff640 1 -- 172.21.15.32:0/876252955 learned_addr learned my addr 172.21.15.32:0/876252955 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:10:25.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.591+0000 7f63c553f640 1 -- 172.21.15.32:0/876252955 >> v2:172.21.15.121:3300/0 conn(0x7f63c015e9e0 msgr2=0x7f63c00d40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:25.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.591+0000 7f63c553f640 1 --2- 172.21.15.32:0/876252955 >> v2:172.21.15.121:3300/0 conn(0x7f63c015e9e0 0x7f63c00d40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:25.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.591+0000 7f63c553f640 1 -- 172.21.15.32:0/876252955 >> v1:172.21.15.121:6789/0 conn(0x7f63c00ba390 legacy=0x7f63c00d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:25.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.591+0000 7f63c553f640 1 -- 172.21.15.32:0/876252955 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63c00d35d0 con 0x7f63c006b110 2024-04-24T23:10:25.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.591+0000 7f63bf7fe640 1 -- 172.21.15.32:0/876252955 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2124536300 0 0) 0x7f63c00d3010 con 0x7f63c00ba390 2024-04-24T23:10:25.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.591+0000 7f63c4d3e640 1 --2- 172.21.15.32:0/876252955 >> v2:172.21.15.121:3300/0 conn(0x7f63c015e9e0 0x7f63c00d40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:10:25.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.592+0000 7f63c553f640 1 --2- 172.21.15.32:0/876252955 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f63c006b110 0x7f63c006b5c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f63b00097a0 tx=0x7f63b002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6eac1e721156dcb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:25.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.592+0000 7f63bf7fe640 1 -- 172.21.15.32:0/876252955 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b0037070 con 0x7f63c006b110 2024-04-24T23:10:25.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.592+0000 7f63bf7fe640 1 -- 172.21.15.32:0/876252955 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63b0032440 con 0x7f63c006b110 2024-04-24T23:10:25.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.592+0000 7f63bf7fe640 1 -- 172.21.15.32:0/876252955 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b0032810 con 0x7f63c006b110 2024-04-24T23:10:25.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.592+0000 7f63c6fa2640 1 -- 172.21.15.32:0/876252955 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f63c006b110 msgr2=0x7f63c006b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:25.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.592+0000 7f63c6fa2640 1 --2- 172.21.15.32:0/876252955 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f63c006b110 0x7f63c006b5c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f63b00097a0 tx=0x7f63b002a180 comp rx=0 tx=0).stop 2024-04-24T23:10:25.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.593+0000 7f63c6fa2640 1 -- 172.21.15.32:0/876252955 shutdown_connections 2024-04-24T23:10:25.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.593+0000 7f63c6fa2640 1 --2- 172.21.15.32:0/876252955 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f63c006b110 0x7f63c006b5c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:25.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.593+0000 7f63c6fa2640 1 --2- 172.21.15.32:0/876252955 >> v2:172.21.15.121:3300/0 conn(0x7f63c015e9e0 0x7f63c00d40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:25.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.593+0000 7f63c6fa2640 1 -- 172.21.15.32:0/876252955 >> 172.21.15.32:0/876252955 conn(0x7f63c00d12e0 msgr2=0x7f63c00d16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:25.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.593+0000 7f63c6fa2640 1 -- 172.21.15.32:0/876252955 shutdown_connections 2024-04-24T23:10:25.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.593+0000 7f63c6fa2640 1 -- 172.21.15.32:0/876252955 wait complete. 2024-04-24T23:10:25.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.594+0000 7f63c6fa2640 1 Processor -- start 2024-04-24T23:10:25.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.594+0000 7f63c6fa2640 1 -- start start 2024-04-24T23:10:25.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.595+0000 7f63c6fa2640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f63c006b110 0x7f63c00ca200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:25.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.595+0000 7f63c6fa2640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f63c00ba390 0x7f63c01514f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:25.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.595+0000 7f63c6fa2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63c015e9e0 0x7f63c0151a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:25.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.595+0000 7f63c6fa2640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f63c00b7640 con 0x7f63c015e9e0 2024-04-24T23:10:25.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.595+0000 7f63c6fa2640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f63c00d3b90 con 0x7f63c00ba390 2024-04-24T23:10:25.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.595+0000 7f63c6fa2640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f63c00b77c0 con 0x7f63c006b110 2024-04-24T23:10:25.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.595+0000 7f63c553f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63c015e9e0 0x7f63c0151a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:25.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.595+0000 7f63c553f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63c015e9e0 0x7f63c0151a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42300/0 (socket says 172.21.15.32:42300) 2024-04-24T23:10:25.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.595+0000 7f63c553f640 1 -- 172.21.15.32:0/4028495313 learned_addr learned my addr 172.21.15.32:0/4028495313 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:25.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.595+0000 7f63c4d3e640 1 --2- 172.21.15.32:0/4028495313 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f63c006b110 0x7f63c00ca200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:25.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.595+0000 7f63bffff640 1 --2- 172.21.15.32:0/4028495313 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f63c00ba390 0x7f63c01514f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:25.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.595+0000 7f63c553f640 1 -- 172.21.15.32:0/4028495313 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f63c006b110 msgr2=0x7f63c00ca200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:25.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.595+0000 7f63c553f640 1 --2- 172.21.15.32:0/4028495313 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f63c006b110 0x7f63c00ca200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:25.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.595+0000 7f63c553f640 1 -- 172.21.15.32:0/4028495313 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f63c00ba390 msgr2=0x7f63c01514f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:25.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.595+0000 7f63c553f640 1 --2- 172.21.15.32:0/4028495313 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f63c00ba390 0x7f63c01514f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:25.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.595+0000 7f63c553f640 1 -- 172.21.15.32:0/4028495313 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63c01522b0 con 0x7f63c015e9e0 2024-04-24T23:10:25.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.595+0000 7f63c553f640 1 --2- 172.21.15.32:0/4028495313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63c015e9e0 0x7f63c0151a30 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f63b0035040 tx=0x7f63b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:25.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.595+0000 7f63bd7fa640 1 -- 172.21.15.32:0/4028495313 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b0037040 con 0x7f63c015e9e0 2024-04-24T23:10:25.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.595+0000 7f63bd7fa640 1 -- 172.21.15.32:0/4028495313 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63b0042070 con 0x7f63c015e9e0 2024-04-24T23:10:25.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.595+0000 7f63c6fa2640 1 -- 172.21.15.32:0/4028495313 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63c0152590 con 0x7f63c015e9e0 2024-04-24T23:10:25.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.595+0000 7f63bd7fa640 1 -- 172.21.15.32:0/4028495313 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b00308a0 con 0x7f63c015e9e0 2024-04-24T23:10:25.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.596+0000 7f63c6fa2640 1 -- 172.21.15.32:0/4028495313 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63c0152b50 con 0x7f63c015e9e0 2024-04-24T23:10:25.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.597+0000 7f63c6fa2640 1 -- 172.21.15.32:0/4028495313 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63c0068430 con 0x7f63c015e9e0 2024-04-24T23:10:25.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.597+0000 7f63bd7fa640 1 -- 172.21.15.32:0/4028495313 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f63b0030a40 con 0x7f63c015e9e0 2024-04-24T23:10:25.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.597+0000 7f63bd7fa640 1 --2- 172.21.15.32:0/4028495313 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f63ac06be30 0x7f63ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:25.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.597+0000 7f63bd7fa640 1 -- 172.21.15.32:0/4028495313 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(176..176 src has 1..176) ==== 4943+0+0 (secure 0 0 0) 0x7f63b00ab260 con 0x7f63c015e9e0 2024-04-24T23:10:25.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.598+0000 7f63c4d3e640 1 --2- 172.21.15.32:0/4028495313 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f63ac06be30 0x7f63ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:25.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.598+0000 7f63c4d3e640 1 --2- 172.21.15.32:0/4028495313 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f63ac06be30 0x7f63ac06e2f0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f63b8009bd0 tx=0x7f63b800b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:25.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.601+0000 7f63bd7fa640 1 -- 172.21.15.32:0/4028495313 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f63b0079950 con 0x7f63c015e9e0 2024-04-24T23:10:25.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.820+0000 7f63c6fa2640 1 -- 172.21.15.32:0/4028495313 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f63c00b7250 con 0x7f63c015e9e0 2024-04-24T23:10:25.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.821+0000 7f63bd7fa640 1 -- 172.21.15.32:0/4028495313 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+3869 (secure 0 0 0) 0x7f63b007dc50 con 0x7f63c015e9e0 2024-04-24T23:10:25.823 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:10:25.823 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":75,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6695,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/91086102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":91086102},{"type":"v1","addr":"172.21.15.121:6836","nonce":91086102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":6698,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1443743127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1443743127},{"type":"v1","addr":"172.21.15.121:6837","nonce":1443743127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":7423,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/4008773444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":4008773444},{"type":"v1","addr":"172.21.15.32:6835","nonce":4008773444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":8994,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/3498990987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3498990987},{"type":"v1","addr":"172.21.15.32:6834","nonce":3498990987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75}],"filesystems":[]} 2024-04-24T23:10:25.823 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 75 2024-04-24T23:10:25.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.823+0000 7f63c6fa2640 1 -- 172.21.15.32:0/4028495313 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f63ac06be30 msgr2=0x7f63ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:25.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.823+0000 7f63c6fa2640 1 --2- 172.21.15.32:0/4028495313 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f63ac06be30 0x7f63ac06e2f0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f63b8009bd0 tx=0x7f63b800b040 comp rx=0 tx=0).stop 2024-04-24T23:10:25.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.823+0000 7f63c6fa2640 1 -- 172.21.15.32:0/4028495313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63c015e9e0 msgr2=0x7f63c0151a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:25.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.823+0000 7f63c6fa2640 1 --2- 172.21.15.32:0/4028495313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63c015e9e0 0x7f63c0151a30 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f63b0035040 tx=0x7f63b00046c0 comp rx=0 tx=0).stop 2024-04-24T23:10:25.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.823+0000 7f63c6fa2640 1 -- 172.21.15.32:0/4028495313 shutdown_connections 2024-04-24T23:10:25.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.823+0000 7f63c6fa2640 1 --2- 172.21.15.32:0/4028495313 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f63ac06be30 0x7f63ac06e2f0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:25.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.823+0000 7f63c6fa2640 1 --2- 172.21.15.32:0/4028495313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63c015e9e0 0x7f63c0151a30 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:25.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.823+0000 7f63c6fa2640 1 --2- 172.21.15.32:0/4028495313 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f63c00ba390 0x7f63c01514f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:25.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.823+0000 7f63c6fa2640 1 --2- 172.21.15.32:0/4028495313 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f63c006b110 0x7f63c00ca200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:25.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.823+0000 7f63c6fa2640 1 -- 172.21.15.32:0/4028495313 >> 172.21.15.32:0/4028495313 conn(0x7f63c00d12e0 msgr2=0x7f63c015d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:25.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.823+0000 7f63c6fa2640 1 -- 172.21.15.32:0/4028495313 shutdown_connections 2024-04-24T23:10:25.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:25.823+0000 7f63c6fa2640 1 -- 172.21.15.32:0/4028495313 wait complete. 2024-04-24T23:10:25.836 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-24T23:10:25.836 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-24T23:10:25.836 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-24T23:10:25.836 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:10:25.836 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-24T23:10:25.836 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-24T23:10:25.837 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-24T23:10:25.837 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:10:25.837 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-24T23:10:25.837 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-24T23:10:25.837 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-24T23:10:25.837 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:10:25.837 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-24T23:10:25.837 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-24T23:10:25.838 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-24T23:10:25.838 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:10:25.846 INFO:tasks.ceph.mds.a:Stopped 2024-04-24T23:10:25.846 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-24T23:10:25.847 INFO:tasks.ceph.mds.c:Stopped 2024-04-24T23:10:25.847 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-24T23:10:25.850 INFO:tasks.ceph.mds.b:Stopped 2024-04-24T23:10:25.850 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-24T23:10:25.889 INFO:tasks.ceph.mds.a:Started 2024-04-24T23:10:25.889 DEBUG:teuthology.parallel:result is None 2024-04-24T23:10:25.890 INFO:tasks.ceph.mds.c:Started 2024-04-24T23:10:25.890 DEBUG:teuthology.parallel:result is None 2024-04-24T23:10:25.893 INFO:tasks.ceph.mds.b:Started 2024-04-24T23:10:25.893 DEBUG:teuthology.parallel:result is None 2024-04-24T23:10:25.964 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2024-04-24T23:10:25.964 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2024-04-24T23:10:25.975 INFO:tasks.ceph.mds.b.smithi121.stdout:starting mds.b at 2024-04-24T23:10:31.850 INFO:tasks.ceph.mds.d:Stopped 2024-04-24T23:10:31.851 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-24T23:10:31.853 INFO:tasks.ceph.mds.d:Started 2024-04-24T23:10:31.854 DEBUG:teuthology.parallel:result is None 2024-04-24T23:10:31.854 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-24T23:10:31.936 INFO:tasks.ceph.mds.d.smithi121.stdout:starting mds.d at 2024-04-24T23:10:31.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.968+0000 7f6301400640 1 Processor -- start 2024-04-24T23:10:31.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.968+0000 7f6301400640 1 -- start start 2024-04-24T23:10:31.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.969+0000 7f6301400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62fc1474f0 0x7f62fc162320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:31.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.969+0000 7f6301400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f62fc147a00 0x7f62fc0cf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:31.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.969+0000 7f6301400640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62fc162900 con 0x7f62fc0d4840 2024-04-24T23:10:31.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.969+0000 7f6301400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f62fc0b7550 con 0x7f62fc147a00 2024-04-24T23:10:31.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.969+0000 7f6301400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f62fc162be0 con 0x7f62fc1474f0 2024-04-24T23:10:31.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.969+0000 7f62f3fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f62fc147a00 0x7f62fc0cf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:31.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.969+0000 7f62f3fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f62fc147a00 0x7f62fc0cf1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46710/0 (socket says 172.21.15.32:46710) 2024-04-24T23:10:31.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.969+0000 7f62f3fff640 1 -- 172.21.15.32:0/2487173724 learned_addr learned my addr 172.21.15.32:0/2487173724 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:31.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.969+0000 7f62faffd640 1 --2- 172.21.15.32:0/2487173724 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62fc1474f0 0x7f62fc162320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:31.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.969+0000 7f62f3fff640 1 -- 172.21.15.32:0/2487173724 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62fc1474f0 msgr2=0x7f62fc162320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:31.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.969+0000 7f62f3fff640 1 --2- 172.21.15.32:0/2487173724 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62fc1474f0 0x7f62fc162320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:31.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.969+0000 7f62f3fff640 1 -- 172.21.15.32:0/2487173724 >> v1:172.21.15.121:6789/0 conn(0x7f62fc0d4840 legacy=0x7f62fc0b79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:31.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.969+0000 7f62f3fff640 1 -- 172.21.15.32:0/2487173724 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62fc162ec0 con 0x7f62fc147a00 2024-04-24T23:10:31.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.970+0000 7f62f3fff640 1 --2- 172.21.15.32:0/2487173724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62fc147a00 0x7f62fc0cf1d0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f62e4009870 tx=0x7f62e402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca82b82dee0c2783 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:31.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.970+0000 7f62fa7fc640 1 -- 172.21.15.32:0/2487173724 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e4037070 con 0x7f62fc147a00 2024-04-24T23:10:31.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.970+0000 7f62fa7fc640 1 -- 172.21.15.32:0/2487173724 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62e4032440 con 0x7f62fc147a00 2024-04-24T23:10:31.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.970+0000 7f62fa7fc640 1 -- 172.21.15.32:0/2487173724 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e4032810 con 0x7f62fc147a00 2024-04-24T23:10:31.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.971+0000 7f6301400640 1 -- 172.21.15.32:0/2487173724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62fc147a00 msgr2=0x7f62fc0cf1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:31.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.971+0000 7f6301400640 1 --2- 172.21.15.32:0/2487173724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62fc147a00 0x7f62fc0cf1d0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f62e4009870 tx=0x7f62e402a270 comp rx=0 tx=0).stop 2024-04-24T23:10:31.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.971+0000 7f6301400640 1 -- 172.21.15.32:0/2487173724 shutdown_connections 2024-04-24T23:10:31.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.971+0000 7f6301400640 1 --2- 172.21.15.32:0/2487173724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62fc147a00 0x7f62fc0cf1d0 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:31.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.971+0000 7f6301400640 1 --2- 172.21.15.32:0/2487173724 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62fc1474f0 0x7f62fc162320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:31.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.971+0000 7f6301400640 1 -- 172.21.15.32:0/2487173724 >> 172.21.15.32:0/2487173724 conn(0x7f62fc074210 msgr2=0x7f62fc074620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:31.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.971+0000 7f6301400640 1 -- 172.21.15.32:0/2487173724 shutdown_connections 2024-04-24T23:10:31.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.971+0000 7f6301400640 1 -- 172.21.15.32:0/2487173724 wait complete. 2024-04-24T23:10:31.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.972+0000 7f6301400640 1 Processor -- start 2024-04-24T23:10:31.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.972+0000 7f6301400640 1 -- start start 2024-04-24T23:10:31.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.973+0000 7f6301400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62fc0d4840 0x7f62fc142fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:31.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.973+0000 7f6301400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62fc1474f0 0x7f62fc0cbf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:31.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.973+0000 7f6301400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62fc147a00 0x7f62fc0cda00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:31.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.973+0000 7f6301400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f62fc0cf710 con 0x7f62fc1474f0 2024-04-24T23:10:31.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.973+0000 7f6301400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f62fc148120 con 0x7f62fc0d4840 2024-04-24T23:10:31.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.973+0000 7f6301400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f62fc0cf890 con 0x7f62fc147a00 2024-04-24T23:10:31.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.973+0000 7f62f3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62fc1474f0 0x7f62fc0cbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:31.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.973+0000 7f62f3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62fc1474f0 0x7f62fc0cbf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46712/0 (socket says 172.21.15.32:46712) 2024-04-24T23:10:31.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.973+0000 7f62f3fff640 1 -- 172.21.15.32:0/2122477269 learned_addr learned my addr 172.21.15.32:0/2122477269 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:31.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.973+0000 7f62faffd640 1 --2- 172.21.15.32:0/2122477269 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62fc0d4840 0x7f62fc142fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:31.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.973+0000 7f62fb7fe640 1 --2- 172.21.15.32:0/2122477269 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62fc147a00 0x7f62fc0cda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:31.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.974+0000 7f62f3fff640 1 -- 172.21.15.32:0/2122477269 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62fc147a00 msgr2=0x7f62fc0cda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:31.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.974+0000 7f62f3fff640 1 --2- 172.21.15.32:0/2122477269 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62fc147a00 0x7f62fc0cda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:31.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.974+0000 7f62f3fff640 1 -- 172.21.15.32:0/2122477269 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62fc0d4840 msgr2=0x7f62fc142fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:31.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.974+0000 7f62f3fff640 1 --2- 172.21.15.32:0/2122477269 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62fc0d4840 0x7f62fc142fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:31.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.974+0000 7f62f3fff640 1 -- 172.21.15.32:0/2122477269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62fc0caec0 con 0x7f62fc1474f0 2024-04-24T23:10:31.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.974+0000 7f62f3fff640 1 --2- 172.21.15.32:0/2122477269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62fc1474f0 0x7f62fc0cbf60 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f62e402a2b0 tx=0x7f62e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:31.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.974+0000 7f62f37fe640 1 -- 172.21.15.32:0/2122477269 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e4041030 con 0x7f62fc1474f0 2024-04-24T23:10:31.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.974+0000 7f62f37fe640 1 -- 172.21.15.32:0/2122477269 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62e4030440 con 0x7f62fc1474f0 2024-04-24T23:10:31.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.974+0000 7f6301400640 1 -- 172.21.15.32:0/2122477269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62fc0cb1a0 con 0x7f62fc1474f0 2024-04-24T23:10:31.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.974+0000 7f62f37fe640 1 -- 172.21.15.32:0/2122477269 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e4037040 con 0x7f62fc1474f0 2024-04-24T23:10:31.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.974+0000 7f6301400640 1 -- 172.21.15.32:0/2122477269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62fc0cb760 con 0x7f62fc1474f0 2024-04-24T23:10:31.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.975+0000 7f6301400640 1 -- 172.21.15.32:0/2122477269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62fc0682b0 con 0x7f62fc1474f0 2024-04-24T23:10:31.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.975+0000 7f62f37fe640 1 -- 172.21.15.32:0/2122477269 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f62e4044070 con 0x7f62fc1474f0 2024-04-24T23:10:31.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.976+0000 7f62f37fe640 1 --2- 172.21.15.32:0/2122477269 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f62dc06be30 0x7f62dc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:31.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.976+0000 7f62f37fe640 1 -- 172.21.15.32:0/2122477269 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(180..180 src has 1..180) ==== 4943+0+0 (secure 0 0 0) 0x7f62e40ab110 con 0x7f62fc1474f0 2024-04-24T23:10:31.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.976+0000 7f62faffd640 1 --2- 172.21.15.32:0/2122477269 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f62dc06be30 0x7f62dc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:31.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.977+0000 7f62faffd640 1 --2- 172.21.15.32:0/2122477269 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f62dc06be30 0x7f62dc06e2f0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f62fc0ba6c0 tx=0x7f62ec009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:31.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:31.980+0000 7f62f37fe640 1 -- 172.21.15.32:0/2122477269 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f62e4079770 con 0x7f62fc1474f0 2024-04-24T23:10:32.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:32.175+0000 7f6301400640 1 -- 172.21.15.32:0/2122477269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f62fc074d40 con 0x7f62fc1474f0 2024-04-24T23:10:32.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:32.968+0000 7f62f37fe640 1 -- 172.21.15.32:0/2122477269 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v181) ==== 97+0+0 (secure 0 0 0) 0x7f62e407da70 con 0x7f62fc1474f0 2024-04-24T23:10:32.969 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blocklist entries 2024-04-24T23:10:32.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:32.971+0000 7f6301400640 1 -- 172.21.15.32:0/2122477269 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f62dc06be30 msgr2=0x7f62dc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:32.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:32.971+0000 7f6301400640 1 --2- 172.21.15.32:0/2122477269 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f62dc06be30 0x7f62dc06e2f0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f62fc0ba6c0 tx=0x7f62ec009190 comp rx=0 tx=0).stop 2024-04-24T23:10:32.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:32.971+0000 7f6301400640 1 -- 172.21.15.32:0/2122477269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62fc1474f0 msgr2=0x7f62fc0cbf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:32.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:32.971+0000 7f6301400640 1 --2- 172.21.15.32:0/2122477269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62fc1474f0 0x7f62fc0cbf60 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f62e402a2b0 tx=0x7f62e40046c0 comp rx=0 tx=0).stop 2024-04-24T23:10:32.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:32.971+0000 7f6301400640 1 -- 172.21.15.32:0/2122477269 shutdown_connections 2024-04-24T23:10:32.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:32.971+0000 7f6301400640 1 --2- 172.21.15.32:0/2122477269 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f62dc06be30 0x7f62dc06e2f0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:32.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:32.971+0000 7f6301400640 1 --2- 172.21.15.32:0/2122477269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62fc1474f0 0x7f62fc0cbf60 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:32.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:32.971+0000 7f6301400640 1 --2- 172.21.15.32:0/2122477269 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62fc147a00 0x7f62fc0cda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:32.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:32.971+0000 7f6301400640 1 --2- 172.21.15.32:0/2122477269 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62fc0d4840 0x7f62fc142fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:32.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:32.971+0000 7f6301400640 1 -- 172.21.15.32:0/2122477269 >> 172.21.15.32:0/2122477269 conn(0x7f62fc074210 msgr2=0x7f62fc0d3d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:32.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:32.971+0000 7f6301400640 1 -- 172.21.15.32:0/2122477269 shutdown_connections 2024-04-24T23:10:32.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:32.971+0000 7f6301400640 1 -- 172.21.15.32:0/2122477269 wait complete. 2024-04-24T23:10:32.986 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-24T23:10:33.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.091+0000 7fa696f63640 1 Processor -- start 2024-04-24T23:10:33.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.091+0000 7fa696f63640 1 -- start start 2024-04-24T23:10:33.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.092+0000 7fa696f63640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa6900b5af0 0x7fa6900ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:33.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.092+0000 7fa696f63640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa690162e10 0x7fa69015ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:33.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.092+0000 7fa696f63640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6900cd270 con 0x7fa690162320 2024-04-24T23:10:33.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.092+0000 7fa696f63640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fa6900b5660 con 0x7fa690162e10 2024-04-24T23:10:33.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.092+0000 7fa696f63640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fa690071610 con 0x7fa6900b5af0 2024-04-24T23:10:33.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.092+0000 7fa695500640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa690162e10 0x7fa69015ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:33.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.092+0000 7fa695500640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa690162e10 0x7fa69015ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46720/0 (socket says 172.21.15.32:46720) 2024-04-24T23:10:33.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.092+0000 7fa695500640 1 -- 172.21.15.32:0/4187213865 learned_addr learned my addr 172.21.15.32:0/4187213865 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:33.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.092+0000 7fa68ffff640 1 --2- 172.21.15.32:0/4187213865 >> v2:172.21.15.121:3300/0 conn(0x7fa6900b5af0 0x7fa6900ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:33.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.092+0000 7fa695500640 1 -- 172.21.15.32:0/4187213865 >> v2:172.21.15.121:3300/0 conn(0x7fa6900b5af0 msgr2=0x7fa6900ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:33.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.092+0000 7fa695500640 1 --2- 172.21.15.32:0/4187213865 >> v2:172.21.15.121:3300/0 conn(0x7fa6900b5af0 0x7fa6900ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:33.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.092+0000 7fa695500640 1 -- 172.21.15.32:0/4187213865 >> v1:172.21.15.32:6789/0 conn(0x7fa690162320 legacy=0x7fa690162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:33.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.092+0000 7fa695500640 1 -- 172.21.15.32:0/4187213865 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6900718f0 con 0x7fa690162e10 2024-04-24T23:10:33.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.093+0000 7fa695500640 1 --2- 172.21.15.32:0/4187213865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa690162e10 0x7fa69015ed30 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fa6900af8c0 tx=0x7fa68802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9cfc5f3e85ed680 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:33.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.093+0000 7fa68f7fe640 1 -- 172.21.15.32:0/4187213865 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa688036070 con 0x7fa690162e10 2024-04-24T23:10:33.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.093+0000 7fa68f7fe640 1 -- 172.21.15.32:0/4187213865 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa688031600 con 0x7fa690162e10 2024-04-24T23:10:33.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.093+0000 7fa68f7fe640 1 -- 172.21.15.32:0/4187213865 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6880319d0 con 0x7fa690162e10 2024-04-24T23:10:33.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.094+0000 7fa696f63640 1 -- 172.21.15.32:0/4187213865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa690162e10 msgr2=0x7fa69015ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:33.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.094+0000 7fa696f63640 1 --2- 172.21.15.32:0/4187213865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa690162e10 0x7fa69015ed30 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fa6900af8c0 tx=0x7fa68802a6a0 comp rx=0 tx=0).stop 2024-04-24T23:10:33.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.094+0000 7fa696f63640 1 -- 172.21.15.32:0/4187213865 shutdown_connections 2024-04-24T23:10:33.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.094+0000 7fa696f63640 1 --2- 172.21.15.32:0/4187213865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa690162e10 0x7fa69015ed30 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:33.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.094+0000 7fa696f63640 1 --2- 172.21.15.32:0/4187213865 >> v2:172.21.15.121:3300/0 conn(0x7fa6900b5af0 0x7fa6900ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:33.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.094+0000 7fa696f63640 1 -- 172.21.15.32:0/4187213865 >> 172.21.15.32:0/4187213865 conn(0x7fa6900d0ae0 msgr2=0x7fa6900d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:33.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.094+0000 7fa696f63640 1 -- 172.21.15.32:0/4187213865 shutdown_connections 2024-04-24T23:10:33.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.094+0000 7fa696f63640 1 -- 172.21.15.32:0/4187213865 wait complete. 2024-04-24T23:10:33.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.095+0000 7fa696f63640 1 Processor -- start 2024-04-24T23:10:33.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.095+0000 7fa696f63640 1 -- start start 2024-04-24T23:10:33.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.096+0000 7fa696f63640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa6900b5af0 0x7fa6900cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:33.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.096+0000 7fa696f63640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa690162320 0x7fa690149600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:33.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.096+0000 7fa696f63640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa690162e10 0x7fa69015f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:33.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.096+0000 7fa696f63640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa690071bd0 con 0x7fa690162320 2024-04-24T23:10:33.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.096+0000 7fa696f63640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa690067950 con 0x7fa690162e10 2024-04-24T23:10:33.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.096+0000 7fa696f63640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa690071d50 con 0x7fa6900b5af0 2024-04-24T23:10:33.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.096+0000 7fa68ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa690162320 0x7fa690149600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:33.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.096+0000 7fa68ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa690162320 0x7fa690149600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46722/0 (socket says 172.21.15.32:46722) 2024-04-24T23:10:33.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.096+0000 7fa68ffff640 1 -- 172.21.15.32:0/1016082335 learned_addr learned my addr 172.21.15.32:0/1016082335 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:33.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.097+0000 7fa695500640 1 --2- 172.21.15.32:0/1016082335 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa690162e10 0x7fa69015f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:33.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.097+0000 7fa68ffff640 1 -- 172.21.15.32:0/1016082335 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa6900b5af0 msgr2=0x7fa6900cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:33.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.097+0000 7fa694cff640 1 --2- 172.21.15.32:0/1016082335 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa6900b5af0 0x7fa6900cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:33.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.097+0000 7fa68ffff640 1 --2- 172.21.15.32:0/1016082335 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa6900b5af0 0x7fa6900cecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:33.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.097+0000 7fa68ffff640 1 -- 172.21.15.32:0/1016082335 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa690162e10 msgr2=0x7fa69015f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:33.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.097+0000 7fa68ffff640 1 --2- 172.21.15.32:0/1016082335 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa690162e10 0x7fa69015f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:33.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.097+0000 7fa68ffff640 1 -- 172.21.15.32:0/1016082335 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6901604e0 con 0x7fa690162320 2024-04-24T23:10:33.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.097+0000 7fa695500640 1 --2- 172.21.15.32:0/1016082335 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa690162e10 0x7fa69015f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:10:33.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.097+0000 7fa68ffff640 1 --2- 172.21.15.32:0/1016082335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa690162320 0x7fa690149600 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7fa67c00d830 tx=0x7fa67c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:33.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.097+0000 7fa68d7fa640 1 -- 172.21.15.32:0/1016082335 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67c00b6c0 con 0x7fa690162320 2024-04-24T23:10:33.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.097+0000 7fa68d7fa640 1 -- 172.21.15.32:0/1016082335 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa67c019070 con 0x7fa690162320 2024-04-24T23:10:33.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.097+0000 7fa696f63640 1 -- 172.21.15.32:0/1016082335 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa69015fc80 con 0x7fa690162320 2024-04-24T23:10:33.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.097+0000 7fa68d7fa640 1 -- 172.21.15.32:0/1016082335 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67c00ba80 con 0x7fa690162320 2024-04-24T23:10:33.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.097+0000 7fa696f63640 1 -- 172.21.15.32:0/1016082335 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa690160200 con 0x7fa690162320 2024-04-24T23:10:33.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.098+0000 7fa696f63640 1 -- 172.21.15.32:0/1016082335 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa69015c190 con 0x7fa690162320 2024-04-24T23:10:33.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.099+0000 7fa68d7fa640 1 -- 172.21.15.32:0/1016082335 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa67c01e460 con 0x7fa690162320 2024-04-24T23:10:33.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.099+0000 7fa68d7fa640 1 --2- 172.21.15.32:0/1016082335 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa67406be30 0x7fa67406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:33.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.099+0000 7fa68d7fa640 1 -- 172.21.15.32:0/1016082335 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(181..181 src has 1..181) ==== 4857+0+0 (secure 0 0 0) 0x7fa67c08b190 con 0x7fa690162320 2024-04-24T23:10:33.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.100+0000 7fa694cff640 1 --2- 172.21.15.32:0/1016082335 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa67406be30 0x7fa67406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:33.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.100+0000 7fa694cff640 1 --2- 172.21.15.32:0/1016082335 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa67406be30 0x7fa67406e2f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fa690158290 tx=0x7fa684009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:33.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.104+0000 7fa68d7fa640 1 -- 172.21.15.32:0/1016082335 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa69015c190 con 0x7fa690162320 2024-04-24T23:10:33.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.364+0000 7fa696f63640 1 -- 172.21.15.32:0/1016082335 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fa690068430 con 0x7fa690162320 2024-04-24T23:10:33.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.364+0000 7fa68d7fa640 1 -- 172.21.15.32:0/1016082335 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) ==== 80+0+5304 (secure 0 0 0) 0x7fa6901604e0 con 0x7fa690162320 2024-04-24T23:10:33.366 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:10:33.366 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:10:33.366 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2024-04-24T23:10:33.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:33.366 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2024-04-24T23:10:33.367 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmXRdxAxAA/EYEn1C38XhG7DC+0X4y7A==", 2024-04-24T23:10:33.367 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:10:33.367 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:10:33.367 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-24T23:10:33.367 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-24T23:10:33.367 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:10:33.367 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:33.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:33.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:33.367 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2024-04-24T23:10:33.367 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmak7bDhAA0he+Jp/gkkrfBjYKSObjcw==", 2024-04-24T23:10:33.367 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:10:33.368 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:10:33.368 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-24T23:10:33.368 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-24T23:10:33.368 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:10:33.368 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:33.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:33.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:33.368 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.c", 2024-04-24T23:10:33.368 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmZXFdChAA8TwwkA5EIXHBzPsQSFvXcQ==", 2024-04-24T23:10:33.368 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:10:33.368 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:10:33.368 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-24T23:10:33.368 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-24T23:10:33.368 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:10:33.368 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:33.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:33.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:33.369 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.d", 2024-04-24T23:10:33.369 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmLbGuFRAAe91fZlulziWdWiDdLBLWNA==", 2024-04-24T23:10:33.369 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:10:33.369 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:10:33.369 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-24T23:10:33.369 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-24T23:10:33.369 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:10:33.369 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:33.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:33.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:33.369 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2024-04-24T23:10:33.369 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCfjilmhTwoOBAAig98xa3a2qnv4rriUO9VCQ==", 2024-04-24T23:10:33.369 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:10:33.370 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:10:33.370 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:10:33.370 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:10:33.370 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:33.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:33.370 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:33.370 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2024-04-24T23:10:33.370 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCgjilmwRaAAxAAUqDOyR9HV6DEnmbyn3WAjA==", 2024-04-24T23:10:33.370 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:10:33.370 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:10:33.370 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:10:33.370 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:10:33.370 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:33.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:33.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:33.371 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2024-04-24T23:10:33.371 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCgjilmDl2qDBAAF1bhTOaSOc612wmFRd8+XA==", 2024-04-24T23:10:33.371 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:10:33.371 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:10:33.371 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:10:33.371 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:10:33.371 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:33.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:33.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:33.371 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2024-04-24T23:10:33.371 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCgjilmAxHwFRAA9a0v/YwngRxq1vmMRsmEkQ==", 2024-04-24T23:10:33.371 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:10:33.372 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:10:33.372 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:10:33.372 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:10:33.372 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:33.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:33.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:33.372 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2024-04-24T23:10:33.372 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCjjilmDPmJIxAASZlEo1ZJHOhIAFGukIj1tQ==", 2024-04-24T23:10:33.372 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:10:33.372 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:10:33.372 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:10:33.372 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:10:33.372 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:33.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:33.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:33.373 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2024-04-24T23:10:33.373 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCjjilmgcpuKhAAedSq0ubNB30s3bz0fY38hw==", 2024-04-24T23:10:33.373 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:10:33.373 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:10:33.373 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:10:33.373 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:10:33.373 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:33.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:33.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:33.373 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2024-04-24T23:10:33.373 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCjjilmFv2VMxAAM+mUc4I0sjawy2uwuUEKYg==", 2024-04-24T23:10:33.373 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:10:33.373 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:10:33.373 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:10:33.374 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:10:33.374 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:33.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:33.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:33.374 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2024-04-24T23:10:33.374 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCkjilmLvEbARAAnaOvQ5Ee0clN/A9SVoKuBQ==", 2024-04-24T23:10:33.374 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:10:33.374 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-24T23:10:33.374 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-24T23:10:33.374 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:10:33.374 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:33.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:33.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:33.375 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2024-04-24T23:10:33.375 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilm5oeiGRAAdLXwwqrYhV3HHo7Dw/2ZqQ==", 2024-04-24T23:10:33.375 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:10:33.375 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:10:33.375 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2024-04-24T23:10:33.375 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-24T23:10:33.375 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:33.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:33.375 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:33.375 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2024-04-24T23:10:33.375 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmyWBDIhAARtVP4Unk3HHlQYZ2QZ3GzQ==", 2024-04-24T23:10:33.375 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:10:33.375 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:10:33.375 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-24T23:10:33.375 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-24T23:10:33.376 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-24T23:10:33.376 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:33.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:33.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:33.376 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.2", 2024-04-24T23:10:33.376 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilm4QElHxAAXgE316aRCosxX7XER5gtKQ==", 2024-04-24T23:10:33.376 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:10:33.376 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:10:33.376 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-24T23:10:33.376 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-24T23:10:33.376 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-24T23:10:33.376 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:33.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:33.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:33.377 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.3", 2024-04-24T23:10:33.377 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmMZihJxAAoWaAAFY3vVfZnC4/GoT0+g==", 2024-04-24T23:10:33.377 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:10:33.377 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-24T23:10:33.377 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-24T23:10:33.377 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-24T23:10:33.377 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-24T23:10:33.377 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:33.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:33.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:33.377 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2024-04-24T23:10:33.377 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCdjilmdVXeIxAAwGGaAfshx4pHcoc5+3kRww==", 2024-04-24T23:10:33.378 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:10:33.378 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2024-04-24T23:10:33.378 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-24T23:10:33.378 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-24T23:10:33.378 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:10:33.378 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:33.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:33.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:33.378 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2024-04-24T23:10:33.378 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCejilmn3FJABAAGnvNAoNLzK20cQF6beihmw==", 2024-04-24T23:10:33.378 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:10:33.378 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2024-04-24T23:10:33.378 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2024-04-24T23:10:33.378 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:10:33.378 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:33.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:33.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:33.379 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2024-04-24T23:10:33.379 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCdjilmMqs2OBAAOe3bLpnCq+HK2EbeXnwY8Q==", 2024-04-24T23:10:33.379 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-24T23:10:33.379 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2024-04-24T23:10:33.379 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2024-04-24T23:10:33.379 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-24T23:10:33.379 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:33.379 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:33.379 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:33.379 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:10:33.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.366+0000 7fa696f63640 1 -- 172.21.15.32:0/1016082335 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa67406be30 msgr2=0x7fa67406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:33.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.366+0000 7fa696f63640 1 --2- 172.21.15.32:0/1016082335 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa67406be30 0x7fa67406e2f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fa690158290 tx=0x7fa684009040 comp rx=0 tx=0).stop 2024-04-24T23:10:33.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.366+0000 7fa696f63640 1 -- 172.21.15.32:0/1016082335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa690162320 msgr2=0x7fa690149600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:33.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.366+0000 7fa696f63640 1 --2- 172.21.15.32:0/1016082335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa690162320 0x7fa690149600 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7fa67c00d830 tx=0x7fa67c00dcf0 comp rx=0 tx=0).stop 2024-04-24T23:10:33.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.366+0000 7fa696f63640 1 -- 172.21.15.32:0/1016082335 shutdown_connections 2024-04-24T23:10:33.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.366+0000 7fa696f63640 1 --2- 172.21.15.32:0/1016082335 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa67406be30 0x7fa67406e2f0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:33.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.366+0000 7fa696f63640 1 --2- 172.21.15.32:0/1016082335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa690162320 0x7fa690149600 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:33.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.366+0000 7fa696f63640 1 --2- 172.21.15.32:0/1016082335 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa690162e10 0x7fa69015f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:33.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.366+0000 7fa696f63640 1 --2- 172.21.15.32:0/1016082335 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa6900b5af0 0x7fa6900cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:33.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.367+0000 7fa696f63640 1 -- 172.21.15.32:0/1016082335 >> 172.21.15.32:0/1016082335 conn(0x7fa6900d0ae0 msgr2=0x7fa6900b0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:33.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.367+0000 7fa696f63640 1 -- 172.21.15.32:0/1016082335 shutdown_connections 2024-04-24T23:10:33.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.367+0000 7fa696f63640 1 -- 172.21.15.32:0/1016082335 wait complete. 2024-04-24T23:10:33.382 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:10:33.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.481+0000 7f44ffa93640 1 Processor -- start 2024-04-24T23:10:33.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.481+0000 7f44ffa93640 1 -- start start 2024-04-24T23:10:33.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.482+0000 7f44ffa93640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f44f815d730 0x7f44f8167910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:33.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.482+0000 7f44ffa93640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44f80b4ee0 con 0x7f44f815cc40 2024-04-24T23:10:33.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.482+0000 7f44ffa93640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44f80b51c0 con 0x7f44f80d05b0 2024-04-24T23:10:33.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.482+0000 7f44ffa93640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f44f806cca0 con 0x7f44f815d730 2024-04-24T23:10:33.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.482+0000 7f44fd82f640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f44f815cc40 0x7f44f815d020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37906/0 (socket says 172.21.15.32:37906) 2024-04-24T23:10:33.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.482+0000 7f44fd82f640 1 -- 172.21.15.32:0/80061427 learned_addr learned my addr 172.21.15.32:0/80061427 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:10:33.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.482+0000 7f44fe030640 1 --2- 172.21.15.32:0/80061427 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f44f815d730 0x7f44f8167910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:33.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.482+0000 7f44fc82d640 1 -- 172.21.15.32:0/80061427 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 974843598 0 0) 0x7f44f80b4ee0 con 0x7f44f815cc40 2024-04-24T23:10:33.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.482+0000 7f44fc82d640 1 -- 172.21.15.32:0/80061427 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f44e0003520 con 0x7f44f815cc40 2024-04-24T23:10:33.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.482+0000 7f44fc82d640 1 -- 172.21.15.32:0/80061427 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1679558560 0 0) 0x7f44e0003520 con 0x7f44f815cc40 2024-04-24T23:10:33.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.482+0000 7f44fc82d640 1 -- 172.21.15.32:0/80061427 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f44f815d730 msgr2=0x7f44f8167910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:33.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.482+0000 7f44fc82d640 1 --2- 172.21.15.32:0/80061427 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f44f815d730 0x7f44f8167910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:33.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.482+0000 7f44fc82d640 1 -- 172.21.15.32:0/80061427 >> v1:172.21.15.121:6789/0 conn(0x7f44f80d05b0 legacy=0x7f44f81641e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:33.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.482+0000 7f44fc82d640 1 -- 172.21.15.32:0/80061427 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44f80b54a0 con 0x7f44f815cc40 2024-04-24T23:10:33.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.482+0000 7f44fc82d640 1 -- 172.21.15.32:0/80061427 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f44f0001670 con 0x7f44f815cc40 2024-04-24T23:10:33.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.482+0000 7f44fc82d640 1 -- 172.21.15.32:0/80061427 >> v1:172.21.15.32:6789/0 conn(0x7f44f815cc40 legacy=0x7f44f815d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:33.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.482+0000 7f44fc82d640 1 --2- 172.21.15.32:0/80061427 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f44f815d730 0x7f44e00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:33.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.482+0000 7f44fc82d640 1 --2- 172.21.15.32:0/80061427 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44e0003f20 0x7f44e0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:33.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.482+0000 7f44fc82d640 1 --2- 172.21.15.32:0/80061427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44e0004d40 0x7f44e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:33.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.482+0000 7f44fc82d640 1 -- 172.21.15.32:0/80061427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f44e000baa0 con 0x7f44e0004d40 2024-04-24T23:10:33.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.482+0000 7f44fc82d640 1 -- 172.21.15.32:0/80061427 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f44e000bc50 con 0x7f44e0003f20 2024-04-24T23:10:33.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.482+0000 7f44fc82d640 1 -- 172.21.15.32:0/80061427 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f44e000be50 con 0x7f44f815d730 2024-04-24T23:10:33.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.483+0000 7f44fe030640 1 --2- 172.21.15.32:0/80061427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44e0004d40 0x7f44e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:33.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.483+0000 7f44fd82f640 1 --2- 172.21.15.32:0/80061427 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f44f815d730 0x7f44e00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:33.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.483+0000 7f44fd02e640 1 --2- 172.21.15.32:0/80061427 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44e0003f20 0x7f44e0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:33.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.483+0000 7f44fe030640 1 -- 172.21.15.32:0/80061427 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f44f815d730 msgr2=0x7f44e00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:33.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.483+0000 7f44fe030640 1 --2- 172.21.15.32:0/80061427 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f44f815d730 0x7f44e00039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:33.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.483+0000 7f44fe030640 1 -- 172.21.15.32:0/80061427 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44e0003f20 msgr2=0x7f44e0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:33.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.483+0000 7f44fe030640 1 --2- 172.21.15.32:0/80061427 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44e0003f20 0x7f44e0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:33.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.483+0000 7f44fe030640 1 -- 172.21.15.32:0/80061427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44e0003520 con 0x7f44e0004d40 2024-04-24T23:10:33.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.484+0000 7f44fe030640 1 --2- 172.21.15.32:0/80061427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44e0004d40 0x7f44e00051c0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f44e8009fc0 tx=0x7f44e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76fc786596988022 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:33.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.484+0000 7f44fc82d640 1 -- 172.21.15.32:0/80061427 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e8009c10 con 0x7f44e0004d40 2024-04-24T23:10:33.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.484+0000 7f44fc82d640 1 -- 172.21.15.32:0/80061427 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44e8036070 con 0x7f44e0004d40 2024-04-24T23:10:33.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.484+0000 7f44fc82d640 1 -- 172.21.15.32:0/80061427 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e8031880 con 0x7f44e0004d40 2024-04-24T23:10:33.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.484+0000 7f44ffa93640 1 -- 172.21.15.32:0/80061427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44e0004d40 msgr2=0x7f44e00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:33.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.484+0000 7f44ffa93640 1 --2- 172.21.15.32:0/80061427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44e0004d40 0x7f44e00051c0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f44e8009fc0 tx=0x7f44e0005830 comp rx=0 tx=0).stop 2024-04-24T23:10:33.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.484+0000 7f44fe030640 1 -- 172.21.15.32:0/80061427 reap_dead start 2024-04-24T23:10:33.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.484+0000 7f44ffa93640 1 -- 172.21.15.32:0/80061427 shutdown_connections 2024-04-24T23:10:33.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.484+0000 7f44ffa93640 1 -- 172.21.15.32:0/80061427 >> 172.21.15.32:0/80061427 conn(0x7f44f80d0ae0 msgr2=0x7f44f80d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:33.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.484+0000 7f44ffa93640 1 -- 172.21.15.32:0/80061427 shutdown_connections 2024-04-24T23:10:33.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.484+0000 7f44ffa93640 1 -- 172.21.15.32:0/80061427 wait complete. 2024-04-24T23:10:33.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.485+0000 7f44ffa93640 1 Processor -- start 2024-04-24T23:10:33.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.486+0000 7f44ffa93640 1 -- start start 2024-04-24T23:10:33.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.486+0000 7f44ffa93640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f44f8159280 0x7f44f81e5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:33.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.486+0000 7f44ffa93640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44f8159660 0x7f44f8157ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:33.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.486+0000 7f44ffa93640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44f81581e0 0x7f44f8143100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:33.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.486+0000 7f44ffa93640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f44f80b5a60 con 0x7f44f8159660 2024-04-24T23:10:33.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.486+0000 7f44ffa93640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f44f80ce5f0 con 0x7f44f81581e0 2024-04-24T23:10:33.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.486+0000 7f44ffa93640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f44f8067940 con 0x7f44f8159280 2024-04-24T23:10:33.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.486+0000 7f44fd02e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44f8159660 0x7f44f8157ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:33.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.486+0000 7f44fd02e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44f8159660 0x7f44f8157ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46750/0 (socket says 172.21.15.32:46750) 2024-04-24T23:10:33.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.486+0000 7f44fd02e640 1 -- 172.21.15.32:0/1691521138 learned_addr learned my addr 172.21.15.32:0/1691521138 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:33.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.487+0000 7f44fd82f640 1 --2- 172.21.15.32:0/1691521138 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f44f8159280 0x7f44f81e5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:33.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.487+0000 7f44fe030640 1 --2- 172.21.15.32:0/1691521138 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44f81581e0 0x7f44f8143100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:33.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.487+0000 7f44fd02e640 1 -- 172.21.15.32:0/1691521138 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f44f8159280 msgr2=0x7f44f81e5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:33.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.487+0000 7f44fd02e640 1 --2- 172.21.15.32:0/1691521138 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f44f8159280 0x7f44f81e5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:33.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.487+0000 7f44fd02e640 1 -- 172.21.15.32:0/1691521138 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44f81581e0 msgr2=0x7f44f8143100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:33.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.487+0000 7f44fd02e640 1 --2- 172.21.15.32:0/1691521138 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44f81581e0 0x7f44f8143100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:33.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.487+0000 7f44fd02e640 1 -- 172.21.15.32:0/1691521138 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44f815b230 con 0x7f44f8159660 2024-04-24T23:10:33.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.487+0000 7f44fe030640 1 --2- 172.21.15.32:0/1691521138 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44f81581e0 0x7f44f8143100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:10:33.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.487+0000 7f44fd02e640 1 --2- 172.21.15.32:0/1691521138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44f8159660 0x7f44f8157ca0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f44ec00f750 tx=0x7f44ec00fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:33.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.487+0000 7f44f67fc640 1 -- 172.21.15.32:0/1691521138 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ec00fe60 con 0x7f44f8159660 2024-04-24T23:10:33.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.487+0000 7f44f67fc640 1 -- 172.21.15.32:0/1691521138 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44ec018070 con 0x7f44f8159660 2024-04-24T23:10:33.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.487+0000 7f44ffa93640 1 -- 172.21.15.32:0/1691521138 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44f8158740 con 0x7f44f8159660 2024-04-24T23:10:33.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.487+0000 7f44f67fc640 1 -- 172.21.15.32:0/1691521138 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ec00d730 con 0x7f44f8159660 2024-04-24T23:10:33.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.487+0000 7f44ffa93640 1 -- 172.21.15.32:0/1691521138 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44f815b7b0 con 0x7f44f8159660 2024-04-24T23:10:33.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.488+0000 7f44ffa93640 1 -- 172.21.15.32:0/1691521138 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44f815b9b0 con 0x7f44f8159660 2024-04-24T23:10:33.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.489+0000 7f44f67fc640 1 -- 172.21.15.32:0/1691521138 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f44ec00d8d0 con 0x7f44f8159660 2024-04-24T23:10:33.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.489+0000 7f44f67fc640 1 --2- 172.21.15.32:0/1691521138 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f44e406be60 0x7f44e406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:33.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.489+0000 7f44f67fc640 1 -- 172.21.15.32:0/1691521138 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(181..181 src has 1..181) ==== 4857+0+0 (secure 0 0 0) 0x7f44ec08a9d0 con 0x7f44f8159660 2024-04-24T23:10:33.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.489+0000 7f44fd82f640 1 --2- 172.21.15.32:0/1691521138 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f44e406be60 0x7f44e406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:33.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.490+0000 7f44fd82f640 1 --2- 172.21.15.32:0/1691521138 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f44e406be60 0x7f44e406e320 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f44f0007d70 tx=0x7f44f0000f80 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:33.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.493+0000 7f44f67fc640 1 -- 172.21.15.32:0/1691521138 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f44f815b9b0 con 0x7f44f8159660 2024-04-24T23:10:33.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.677+0000 7f44ffa93640 1 -- 172.21.15.32:0/1691521138 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f44f8068430 con 0x7f44f8159660 2024-04-24T23:10:33.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.678+0000 7f44f67fc640 1 -- 172.21.15.32:0/1691521138 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v181) ==== 74+0+12914 (secure 0 0 0) 0x7f44ec0596d0 con 0x7f44f8159660 2024-04-24T23:10:33.680 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:10:33.680 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":181,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:10:32.967050+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:10:33.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.680+0000 7f44ffa93640 1 -- 172.21.15.32:0/1691521138 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f44e406be60 msgr2=0x7f44e406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:33.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.680+0000 7f44ffa93640 1 --2- 172.21.15.32:0/1691521138 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f44e406be60 0x7f44e406e320 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f44f0007d70 tx=0x7f44f0000f80 comp rx=0 tx=0).stop 2024-04-24T23:10:33.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.680+0000 7f44ffa93640 1 -- 172.21.15.32:0/1691521138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44f8159660 msgr2=0x7f44f8157ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:33.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.680+0000 7f44ffa93640 1 --2- 172.21.15.32:0/1691521138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44f8159660 0x7f44f8157ca0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f44ec00f750 tx=0x7f44ec00fc10 comp rx=0 tx=0).stop 2024-04-24T23:10:33.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.680+0000 7f44ffa93640 1 -- 172.21.15.32:0/1691521138 shutdown_connections 2024-04-24T23:10:33.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.680+0000 7f44ffa93640 1 --2- 172.21.15.32:0/1691521138 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f44e406be60 0x7f44e406e320 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:33.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.680+0000 7f44ffa93640 1 --2- 172.21.15.32:0/1691521138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44f8159660 0x7f44f8157ca0 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:33.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.680+0000 7f44ffa93640 1 --2- 172.21.15.32:0/1691521138 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44f81581e0 0x7f44f8143100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:33.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.680+0000 7f44ffa93640 1 --2- 172.21.15.32:0/1691521138 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f44f8159280 0x7f44f81e5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:33.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.680+0000 7f44ffa93640 1 -- 172.21.15.32:0/1691521138 >> 172.21.15.32:0/1691521138 conn(0x7f44f80d0ae0 msgr2=0x7f44f80b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:33.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.680+0000 7f44ffa93640 1 -- 172.21.15.32:0/1691521138 shutdown_connections 2024-04-24T23:10:33.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.680+0000 7f44ffa93640 1 -- 172.21.15.32:0/1691521138 wait complete. 2024-04-24T23:10:33.693 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:10:33.693 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:10:33.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.795+0000 7f792f1a2640 1 Processor -- start 2024-04-24T23:10:33.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.795+0000 7f792f1a2640 1 -- start start 2024-04-24T23:10:33.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.796+0000 7f792f1a2640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f792806cd00 0x7f7928167130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:33.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.796+0000 7f792f1a2640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f792815c990 con 0x7f7928072cf0 2024-04-24T23:10:33.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.796+0000 7f792f1a2640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7928071c50 con 0x7f792815c5b0 2024-04-24T23:10:33.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.796+0000 7f792f1a2640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f79280b4740 con 0x7f792806cd00 2024-04-24T23:10:33.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.796+0000 7f7927fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7928072cf0 0x7f792806c5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37908/0 (socket says 172.21.15.32:37908) 2024-04-24T23:10:33.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.796+0000 7f7927fff640 1 -- 172.21.15.32:0/335027631 learned_addr learned my addr 172.21.15.32:0/335027631 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:10:33.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.796+0000 7f792d73f640 1 --2- 172.21.15.32:0/335027631 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f792806cd00 0x7f7928167130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:33.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.796+0000 7f79277fe640 1 -- 172.21.15.32:0/335027631 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1382758033 0 0) 0x7f7914001110 con 0x7f7928072cf0 2024-04-24T23:10:33.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.796+0000 7f79277fe640 1 -- 172.21.15.32:0/335027631 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7918003520 con 0x7f7928072cf0 2024-04-24T23:10:33.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.796+0000 7f79277fe640 1 -- 172.21.15.32:0/335027631 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 527025937 0 0) 0x7f7918003520 con 0x7f7928072cf0 2024-04-24T23:10:33.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.796+0000 7f79277fe640 1 -- 172.21.15.32:0/335027631 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f792806cd00 msgr2=0x7f7928167130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:33.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.796+0000 7f79277fe640 1 --2- 172.21.15.32:0/335027631 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f792806cd00 0x7f7928167130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:33.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.796+0000 7f79277fe640 1 -- 172.21.15.32:0/335027631 >> v1:172.21.15.121:6789/0 conn(0x7f792815c5b0 legacy=0x7f79280b4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:33.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.796+0000 7f79277fe640 1 -- 172.21.15.32:0/335027631 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7928071f30 con 0x7f7928072cf0 2024-04-24T23:10:33.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.796+0000 7f79277fe640 1 -- 172.21.15.32:0/335027631 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f7914001840 con 0x7f7928072cf0 2024-04-24T23:10:33.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.796+0000 7f79277fe640 1 -- 172.21.15.32:0/335027631 >> v1:172.21.15.32:6789/0 conn(0x7f7928072cf0 legacy=0x7f792806c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:33.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.796+0000 7f79277fe640 1 --2- 172.21.15.32:0/335027631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79180039e0 0x7f7918003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:33.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.796+0000 7f79277fe640 1 --2- 172.21.15.32:0/335027631 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7918004800 0x7f7918004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:33.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.796+0000 7f79277fe640 1 --2- 172.21.15.32:0/335027631 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f792806cd00 0x7f7918005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:33.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.796+0000 7f79277fe640 1 -- 172.21.15.32:0/335027631 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f791800baf0 con 0x7f79180039e0 2024-04-24T23:10:33.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.796+0000 7f79277fe640 1 -- 172.21.15.32:0/335027631 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f791800bca0 con 0x7f7918004800 2024-04-24T23:10:33.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.796+0000 7f79277fe640 1 -- 172.21.15.32:0/335027631 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f791800bea0 con 0x7f792806cd00 2024-04-24T23:10:33.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.797+0000 7f792cf3e640 1 --2- 172.21.15.32:0/335027631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79180039e0 0x7f7918003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:33.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.797+0000 7f7927fff640 1 --2- 172.21.15.32:0/335027631 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7918004800 0x7f7918004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:33.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.797+0000 7f792d73f640 1 --2- 172.21.15.32:0/335027631 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f792806cd00 0x7f7918005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:33.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.797+0000 7f792cf3e640 1 -- 172.21.15.32:0/335027631 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f792806cd00 msgr2=0x7f7918005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:33.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.797+0000 7f792cf3e640 1 --2- 172.21.15.32:0/335027631 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f792806cd00 0x7f7918005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:33.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.797+0000 7f792cf3e640 1 -- 172.21.15.32:0/335027631 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7918004800 msgr2=0x7f7918004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:33.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.797+0000 7f792cf3e640 1 --2- 172.21.15.32:0/335027631 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7918004800 0x7f7918004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:33.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.797+0000 7f792cf3e640 1 -- 172.21.15.32:0/335027631 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7918003520 con 0x7f79180039e0 2024-04-24T23:10:33.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.798+0000 7f792cf3e640 1 --2- 172.21.15.32:0/335027631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79180039e0 0x7f7918003e00 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f791c001750 tx=0x7f7918005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5c103956ea3e50d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:33.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.798+0000 7f79277fe640 1 -- 172.21.15.32:0/335027631 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791c001370 con 0x7f79180039e0 2024-04-24T23:10:33.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.798+0000 7f79277fe640 1 -- 172.21.15.32:0/335027631 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f791c0052d0 con 0x7f79180039e0 2024-04-24T23:10:33.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.798+0000 7f79277fe640 1 -- 172.21.15.32:0/335027631 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791c02abe0 con 0x7f79180039e0 2024-04-24T23:10:33.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.798+0000 7f792f1a2640 1 -- 172.21.15.32:0/335027631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79180039e0 msgr2=0x7f7918003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:33.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.798+0000 7f792f1a2640 1 --2- 172.21.15.32:0/335027631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79180039e0 0x7f7918003e00 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f791c001750 tx=0x7f7918005830 comp rx=0 tx=0).stop 2024-04-24T23:10:33.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.798+0000 7f792d73f640 1 -- 172.21.15.32:0/335027631 reap_dead start 2024-04-24T23:10:33.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.798+0000 7f792f1a2640 1 -- 172.21.15.32:0/335027631 shutdown_connections 2024-04-24T23:10:33.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.798+0000 7f792f1a2640 1 -- 172.21.15.32:0/335027631 >> 172.21.15.32:0/335027631 conn(0x7f792806b110 msgr2=0x7f792806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:33.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.798+0000 7f792f1a2640 1 -- 172.21.15.32:0/335027631 shutdown_connections 2024-04-24T23:10:33.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.798+0000 7f792f1a2640 1 -- 172.21.15.32:0/335027631 wait complete. 2024-04-24T23:10:33.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.799+0000 7f792f1a2640 1 Processor -- start 2024-04-24T23:10:33.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.799+0000 7f792f1a2640 1 -- start start 2024-04-24T23:10:33.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.800+0000 7f792f1a2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79180039e0 0x7f792814df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:33.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.800+0000 7f792f1a2640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f79280cd990 0x7f792815e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:33.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.800+0000 7f792f1a2640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f792815d7f0 0x7f792815dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:33.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.800+0000 7f792f1a2640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f79280724f0 con 0x7f79180039e0 2024-04-24T23:10:33.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.800+0000 7f792f1a2640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7928072860 con 0x7f792815d7f0 2024-04-24T23:10:33.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.800+0000 7f792f1a2640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f7928071770 con 0x7f79280cd990 2024-04-24T23:10:33.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.800+0000 7f792cf3e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79180039e0 0x7f792814df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:33.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.800+0000 7f792cf3e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79180039e0 0x7f792814df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46778/0 (socket says 172.21.15.32:46778) 2024-04-24T23:10:33.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.800+0000 7f792cf3e640 1 -- 172.21.15.32:0/666553980 learned_addr learned my addr 172.21.15.32:0/666553980 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:33.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.801+0000 7f7927fff640 1 --2- 172.21.15.32:0/666553980 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f79280cd990 0x7f792815e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:33.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.801+0000 7f792d73f640 1 --2- 172.21.15.32:0/666553980 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f792815d7f0 0x7f792815dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:33.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.801+0000 7f792cf3e640 1 -- 172.21.15.32:0/666553980 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f79280cd990 msgr2=0x7f792815e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:33.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.801+0000 7f792cf3e640 1 --2- 172.21.15.32:0/666553980 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f79280cd990 0x7f792815e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:33.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.801+0000 7f792cf3e640 1 -- 172.21.15.32:0/666553980 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f792815d7f0 msgr2=0x7f792815dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:33.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.801+0000 7f792cf3e640 1 --2- 172.21.15.32:0/666553980 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f792815d7f0 0x7f792815dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:33.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.801+0000 7f792cf3e640 1 -- 172.21.15.32:0/666553980 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f792814e4c0 con 0x7f79180039e0 2024-04-24T23:10:33.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.801+0000 7f792d73f640 1 --2- 172.21.15.32:0/666553980 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f792815d7f0 0x7f792815dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:10:33.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.801+0000 7f7927fff640 1 --2- 172.21.15.32:0/666553980 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f79280cd990 0x7f792815e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:10:33.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.801+0000 7f792cf3e640 1 --2- 172.21.15.32:0/666553980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79180039e0 0x7f792814df80 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f791c008fd0 tx=0x7f791c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:33.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.801+0000 7f79257fa640 1 -- 172.21.15.32:0/666553980 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791c037040 con 0x7f79180039e0 2024-04-24T23:10:33.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.801+0000 7f79257fa640 1 -- 172.21.15.32:0/666553980 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f791c005480 con 0x7f79180039e0 2024-04-24T23:10:33.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.801+0000 7f792f1a2640 1 -- 172.21.15.32:0/666553980 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f792815ec90 con 0x7f79180039e0 2024-04-24T23:10:33.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.801+0000 7f79257fa640 1 -- 172.21.15.32:0/666553980 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791c007a30 con 0x7f79180039e0 2024-04-24T23:10:33.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.801+0000 7f792f1a2640 1 -- 172.21.15.32:0/666553980 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f792815cff0 con 0x7f79180039e0 2024-04-24T23:10:33.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.802+0000 7f792f1a2640 1 -- 172.21.15.32:0/666553980 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7928068430 con 0x7f79180039e0 2024-04-24T23:10:33.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.803+0000 7f79257fa640 1 -- 172.21.15.32:0/666553980 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f791c044020 con 0x7f79180039e0 2024-04-24T23:10:33.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.803+0000 7f79257fa640 1 --2- 172.21.15.32:0/666553980 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f790c06be30 0x7f790c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:33.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.803+0000 7f79257fa640 1 -- 172.21.15.32:0/666553980 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(181..181 src has 1..181) ==== 4857+0+0 (secure 0 0 0) 0x7f791c0aa610 con 0x7f79180039e0 2024-04-24T23:10:33.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.803+0000 7f7927fff640 1 --2- 172.21.15.32:0/666553980 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f790c06be30 0x7f790c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:33.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.804+0000 7f7927fff640 1 --2- 172.21.15.32:0/666553980 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f790c06be30 0x7f790c06e2f0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f79280aec80 tx=0x7f7914009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:33.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.807+0000 7f79257fa640 1 -- 172.21.15.32:0/666553980 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f791c04b020 con 0x7f79180039e0 2024-04-24T23:10:33.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.991+0000 7f792f1a2640 1 -- 172.21.15.32:0/666553980 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f79280b1dd0 con 0x7f79180039e0 2024-04-24T23:10:33.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.991+0000 7f79257fa640 1 -- 172.21.15.32:0/666553980 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v181) ==== 93+0+10 (secure 0 0 0) 0x7f791c079310 con 0x7f79180039e0 2024-04-24T23:10:33.993 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T23:10:33.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.993+0000 7f792f1a2640 1 -- 172.21.15.32:0/666553980 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f790c06be30 msgr2=0x7f790c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:33.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.993+0000 7f792f1a2640 1 --2- 172.21.15.32:0/666553980 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f790c06be30 0x7f790c06e2f0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f79280aec80 tx=0x7f7914009040 comp rx=0 tx=0).stop 2024-04-24T23:10:33.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.993+0000 7f792f1a2640 1 -- 172.21.15.32:0/666553980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79180039e0 msgr2=0x7f792814df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:33.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.993+0000 7f792f1a2640 1 --2- 172.21.15.32:0/666553980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79180039e0 0x7f792814df80 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f791c008fd0 tx=0x7f791c001120 comp rx=0 tx=0).stop 2024-04-24T23:10:33.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.993+0000 7f792f1a2640 1 -- 172.21.15.32:0/666553980 shutdown_connections 2024-04-24T23:10:33.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.993+0000 7f792f1a2640 1 --2- 172.21.15.32:0/666553980 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f790c06be30 0x7f790c06e2f0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:33.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.993+0000 7f792f1a2640 1 --2- 172.21.15.32:0/666553980 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f792815d7f0 0x7f792815dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:33.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.993+0000 7f792f1a2640 1 --2- 172.21.15.32:0/666553980 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f79280cd990 0x7f792815e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:33.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.993+0000 7f792f1a2640 1 --2- 172.21.15.32:0/666553980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79180039e0 0x7f792814df80 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:33.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.993+0000 7f792f1a2640 1 -- 172.21.15.32:0/666553980 >> 172.21.15.32:0/666553980 conn(0x7f792806b110 msgr2=0x7f792815a590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:33.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.993+0000 7f792f1a2640 1 -- 172.21.15.32:0/666553980 shutdown_connections 2024-04-24T23:10:33.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:33.993+0000 7f792f1a2640 1 -- 172.21.15.32:0/666553980 wait complete. 2024-04-24T23:10:34.006 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-24T23:10:34.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.104+0000 7f4ca7d90640 1 Processor -- start 2024-04-24T23:10:34.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.104+0000 7f4ca7d90640 1 -- start start 2024-04-24T23:10:34.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.105+0000 7f4ca7d90640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ca00b0b80 0x7f4ca006c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:34.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.105+0000 7f4ca7d90640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4ca00ae140 0x7f4ca00b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:34.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.105+0000 7f4ca7d90640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ca00b0770 con 0x7f4ca0074210 2024-04-24T23:10:34.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.105+0000 7f4ca7d90640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f4ca00b1c20 con 0x7f4ca00ae140 2024-04-24T23:10:34.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.105+0000 7f4ca7d90640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f4ca00b1010 con 0x7f4ca00b0b80 2024-04-24T23:10:34.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.105+0000 7f4ca632d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4ca00ae140 0x7f4ca00b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:34.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.105+0000 7f4ca632d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4ca00ae140 0x7f4ca00b20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46784/0 (socket says 172.21.15.32:46784) 2024-04-24T23:10:34.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.105+0000 7f4ca632d640 1 -- 172.21.15.32:0/3758059579 learned_addr learned my addr 172.21.15.32:0/3758059579 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:34.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.105+0000 7f4ca532b640 1 --2- 172.21.15.32:0/3758059579 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ca00b0b80 0x7f4ca006c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:34.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.106+0000 7f4ca632d640 1 -- 172.21.15.32:0/3758059579 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ca00b0b80 msgr2=0x7f4ca006c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:34.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.106+0000 7f4ca632d640 1 --2- 172.21.15.32:0/3758059579 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ca00b0b80 0x7f4ca006c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:34.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.106+0000 7f4ca632d640 1 -- 172.21.15.32:0/3758059579 >> v1:172.21.15.121:6789/0 conn(0x7f4ca0074210 legacy=0x7f4ca00745f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:34.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.106+0000 7f4ca632d640 1 -- 172.21.15.32:0/3758059579 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ca00b12f0 con 0x7f4ca00ae140 2024-04-24T23:10:34.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.106+0000 7f4ca632d640 1 --2- 172.21.15.32:0/3758059579 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ca00ae140 0x7f4ca00b20b0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f4c98009870 tx=0x7f4c9802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b0fc78445a98bbf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:34.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.106+0000 7f4ca4b2a640 1 -- 172.21.15.32:0/3758059579 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c98037070 con 0x7f4ca00ae140 2024-04-24T23:10:34.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.107+0000 7f4ca4b2a640 1 -- 172.21.15.32:0/3758059579 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c98032440 con 0x7f4ca00ae140 2024-04-24T23:10:34.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.107+0000 7f4ca4b2a640 1 -- 172.21.15.32:0/3758059579 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c98032810 con 0x7f4ca00ae140 2024-04-24T23:10:34.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.107+0000 7f4ca7d90640 1 -- 172.21.15.32:0/3758059579 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ca00ae140 msgr2=0x7f4ca00b20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:34.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.107+0000 7f4ca7d90640 1 --2- 172.21.15.32:0/3758059579 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ca00ae140 0x7f4ca00b20b0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f4c98009870 tx=0x7f4c9802a270 comp rx=0 tx=0).stop 2024-04-24T23:10:34.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.107+0000 7f4ca7d90640 1 -- 172.21.15.32:0/3758059579 shutdown_connections 2024-04-24T23:10:34.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.107+0000 7f4ca7d90640 1 --2- 172.21.15.32:0/3758059579 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ca00ae140 0x7f4ca00b20b0 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:34.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.107+0000 7f4ca7d90640 1 --2- 172.21.15.32:0/3758059579 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ca00b0b80 0x7f4ca006c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:34.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.107+0000 7f4ca7d90640 1 -- 172.21.15.32:0/3758059579 >> 172.21.15.32:0/3758059579 conn(0x7f4ca00b2e60 msgr2=0x7f4ca00b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:34.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.107+0000 7f4ca7d90640 1 -- 172.21.15.32:0/3758059579 shutdown_connections 2024-04-24T23:10:34.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.107+0000 7f4ca7d90640 1 -- 172.21.15.32:0/3758059579 wait complete. 2024-04-24T23:10:34.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.108+0000 7f4ca7d90640 1 Processor -- start 2024-04-24T23:10:34.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.108+0000 7f4ca7d90640 1 -- start start 2024-04-24T23:10:34.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.109+0000 7f4ca7d90640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ca0074210 0x7f4ca015de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:34.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.109+0000 7f4ca7d90640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ca00ae140 0x7f4ca00aee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:34.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.109+0000 7f4ca7d90640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ca00af370 0x7f4ca00c67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:34.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.109+0000 7f4ca7d90640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca00b3ba0 con 0x7f4ca00af370 2024-04-24T23:10:34.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.109+0000 7f4ca7d90640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca00b18b0 con 0x7f4ca00ae140 2024-04-24T23:10:34.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.109+0000 7f4ca7d90640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f4ca00b3d20 con 0x7f4ca0074210 2024-04-24T23:10:34.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.109+0000 7f4ca632d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ca00af370 0x7f4ca00c67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:34.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.109+0000 7f4ca632d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ca00af370 0x7f4ca00c67d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46800/0 (socket says 172.21.15.32:46800) 2024-04-24T23:10:34.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.109+0000 7f4ca632d640 1 -- 172.21.15.32:0/3713404404 learned_addr learned my addr 172.21.15.32:0/3713404404 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:34.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.109+0000 7f4ca5b2c640 1 --2- 172.21.15.32:0/3713404404 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ca0074210 0x7f4ca015de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:34.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.109+0000 7f4ca532b640 1 --2- 172.21.15.32:0/3713404404 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ca00ae140 0x7f4ca00aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:34.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.109+0000 7f4ca632d640 1 -- 172.21.15.32:0/3713404404 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ca0074210 msgr2=0x7f4ca015de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:34.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.109+0000 7f4ca632d640 1 --2- 172.21.15.32:0/3713404404 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ca0074210 0x7f4ca015de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:34.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.110+0000 7f4ca632d640 1 -- 172.21.15.32:0/3713404404 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ca00ae140 msgr2=0x7f4ca00aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:34.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.110+0000 7f4ca632d640 1 --2- 172.21.15.32:0/3713404404 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ca00ae140 0x7f4ca00aee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:34.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.110+0000 7f4ca632d640 1 -- 172.21.15.32:0/3713404404 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ca00c6210 con 0x7f4ca00af370 2024-04-24T23:10:34.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.110+0000 7f4ca632d640 1 --2- 172.21.15.32:0/3713404404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ca00af370 0x7f4ca00c67d0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f4c9802a2b0 tx=0x7f4c980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:34.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.110+0000 7f4c9e7fc640 1 -- 172.21.15.32:0/3713404404 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c98041020 con 0x7f4ca00af370 2024-04-24T23:10:34.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.110+0000 7f4c9e7fc640 1 -- 172.21.15.32:0/3713404404 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c9803d640 con 0x7f4ca00af370 2024-04-24T23:10:34.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.110+0000 7f4ca7d90640 1 -- 172.21.15.32:0/3713404404 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ca00c7cb0 con 0x7f4ca00af370 2024-04-24T23:10:34.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.110+0000 7f4c9e7fc640 1 -- 172.21.15.32:0/3713404404 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c98037040 con 0x7f4ca00af370 2024-04-24T23:10:34.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.110+0000 7f4ca7d90640 1 -- 172.21.15.32:0/3713404404 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ca00c76f0 con 0x7f4ca00af370 2024-04-24T23:10:34.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.111+0000 7f4ca7d90640 1 -- 172.21.15.32:0/3713404404 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ca0068430 con 0x7f4ca00af370 2024-04-24T23:10:34.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.111+0000 7f4c9e7fc640 1 -- 172.21.15.32:0/3713404404 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4c98044020 con 0x7f4ca00af370 2024-04-24T23:10:34.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.112+0000 7f4c9e7fc640 1 --2- 172.21.15.32:0/3713404404 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4c8c06be60 0x7f4c8c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:34.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.112+0000 7f4c9e7fc640 1 -- 172.21.15.32:0/3713404404 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(181..181 src has 1..181) ==== 4857+0+0 (secure 0 0 0) 0x7f4c980ab1e0 con 0x7f4ca00af370 2024-04-24T23:10:34.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.112+0000 7f4ca5b2c640 1 --2- 172.21.15.32:0/3713404404 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4c8c06be60 0x7f4c8c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:34.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.113+0000 7f4ca5b2c640 1 --2- 172.21.15.32:0/3713404404 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4c8c06be60 0x7f4c8c06e320 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f4c90001d40 tx=0x7f4c90009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:34.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.116+0000 7f4c9e7fc640 1 -- 172.21.15.32:0/3713404404 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4c9804b030 con 0x7f4ca00af370 2024-04-24T23:10:34.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.300+0000 7f4ca7d90640 1 -- 172.21.15.32:0/3713404404 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f4ca006b220 con 0x7f4ca00af370 2024-04-24T23:10:34.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.301+0000 7f4c9e7fc640 1 -- 172.21.15.32:0/3713404404 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v181) ==== 84+0+65 (secure 0 0 0) 0x7f4ca00c6210 con 0x7f4ca00af370 2024-04-24T23:10:34.302 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:10:34.302 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2024-04-24T23:10:34.302 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:34.303 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2024-04-24T23:10:34.303 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": ".mgr" 2024-04-24T23:10:34.303 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:34.303 INFO:teuthology.orchestra.run.smithi032.stdout:] 2024-04-24T23:10:34.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.302+0000 7f4ca7d90640 1 -- 172.21.15.32:0/3713404404 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4c8c06be60 msgr2=0x7f4c8c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:34.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.302+0000 7f4ca7d90640 1 --2- 172.21.15.32:0/3713404404 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4c8c06be60 0x7f4c8c06e320 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f4c90001d40 tx=0x7f4c90009040 comp rx=0 tx=0).stop 2024-04-24T23:10:34.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.302+0000 7f4ca7d90640 1 -- 172.21.15.32:0/3713404404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ca00af370 msgr2=0x7f4ca00c67d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:34.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.302+0000 7f4ca7d90640 1 --2- 172.21.15.32:0/3713404404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ca00af370 0x7f4ca00c67d0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f4c9802a2b0 tx=0x7f4c980046c0 comp rx=0 tx=0).stop 2024-04-24T23:10:34.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.302+0000 7f4ca7d90640 1 -- 172.21.15.32:0/3713404404 shutdown_connections 2024-04-24T23:10:34.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.302+0000 7f4ca7d90640 1 --2- 172.21.15.32:0/3713404404 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f4c8c06be60 0x7f4c8c06e320 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:34.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.302+0000 7f4ca7d90640 1 --2- 172.21.15.32:0/3713404404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ca00af370 0x7f4ca00c67d0 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:34.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.302+0000 7f4ca7d90640 1 --2- 172.21.15.32:0/3713404404 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ca00ae140 0x7f4ca00aee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:34.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.302+0000 7f4ca7d90640 1 --2- 172.21.15.32:0/3713404404 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ca0074210 0x7f4ca015de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:34.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.302+0000 7f4ca7d90640 1 -- 172.21.15.32:0/3713404404 >> 172.21.15.32:0/3713404404 conn(0x7f4ca00b2e60 msgr2=0x7f4ca00b3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:34.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.302+0000 7f4ca7d90640 1 -- 172.21.15.32:0/3713404404 shutdown_connections 2024-04-24T23:10:34.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.303+0000 7f4ca7d90640 1 -- 172.21.15.32:0/3713404404 wait complete. 2024-04-24T23:10:34.317 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-24T23:10:34.317 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-24T23:10:34.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.420+0000 7f994889e640 1 Processor -- start 2024-04-24T23:10:34.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.421+0000 7f994889e640 1 -- start start 2024-04-24T23:10:34.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.421+0000 7f994889e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f994415e770 0x7f99440d40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:34.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.421+0000 7f994889e640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f99440b6450 0x7f99440b68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:34.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.421+0000 7f994889e640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99440d3000 con 0x7f99440b9ff0 2024-04-24T23:10:34.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.421+0000 7f994889e640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f99440b6df0 con 0x7f99440b6450 2024-04-24T23:10:34.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.421+0000 7f994889e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f99440d32e0 con 0x7f994415e770 2024-04-24T23:10:34.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.422+0000 7f9942d9d640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f99440b6450 0x7f99440b68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:34.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.422+0000 7f994259c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f994415e770 0x7f99440d40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:34.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.422+0000 7f993bfff640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f99440b9ff0 0x7f99440d2700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:40102/0 (socket says 172.21.15.32:40102) 2024-04-24T23:10:34.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.422+0000 7f994259c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f994415e770 0x7f99440d40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.32:51482/0 (socket says 172.21.15.32:51482) 2024-04-24T23:10:34.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.422+0000 7f9942d9d640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f99440b6450 0x7f99440b68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.32:48740/0 (socket says 172.21.15.32:48740) 2024-04-24T23:10:34.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.422+0000 7f993bfff640 1 -- 172.21.15.32:0/1134818165 learned_addr learned my addr 172.21.15.32:0/1134818165 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:10:34.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.422+0000 7f994259c640 1 -- 172.21.15.32:0/1134818165 >> v2:172.21.15.121:3300/0 conn(0x7f99440b6450 msgr2=0x7f99440b68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:34.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.422+0000 7f994259c640 1 --2- 172.21.15.32:0/1134818165 >> v2:172.21.15.121:3300/0 conn(0x7f99440b6450 0x7f99440b68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:34.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.422+0000 7f994259c640 1 -- 172.21.15.32:0/1134818165 >> v1:172.21.15.121:6789/0 conn(0x7f99440b9ff0 legacy=0x7f99440d2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:34.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.422+0000 7f994259c640 1 -- 172.21.15.32:0/1134818165 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99440d35c0 con 0x7f994415e770 2024-04-24T23:10:34.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.423+0000 7f994259c640 1 --2- 172.21.15.32:0/1134818165 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f994415e770 0x7f99440d40a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f99340097a0 tx=0x7f993402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1010ff9957f038e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:34.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.423+0000 7f9941d9b640 1 -- 172.21.15.32:0/1134818165 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9934037070 con 0x7f994415e770 2024-04-24T23:10:34.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.423+0000 7f9941d9b640 1 -- 172.21.15.32:0/1134818165 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9934032440 con 0x7f994415e770 2024-04-24T23:10:34.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.423+0000 7f9941d9b640 1 -- 172.21.15.32:0/1134818165 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9934032810 con 0x7f994415e770 2024-04-24T23:10:34.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.423+0000 7f994889e640 1 -- 172.21.15.32:0/1134818165 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f994415e770 msgr2=0x7f99440d40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:34.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.424+0000 7f994889e640 1 --2- 172.21.15.32:0/1134818165 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f994415e770 0x7f99440d40a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f99340097a0 tx=0x7f993402a180 comp rx=0 tx=0).stop 2024-04-24T23:10:34.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.424+0000 7f994889e640 1 -- 172.21.15.32:0/1134818165 shutdown_connections 2024-04-24T23:10:34.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.424+0000 7f994889e640 1 --2- 172.21.15.32:0/1134818165 >> v2:172.21.15.121:3300/0 conn(0x7f99440b6450 0x7f99440b68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:34.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.424+0000 7f994889e640 1 --2- 172.21.15.32:0/1134818165 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f994415e770 0x7f99440d40a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:34.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.424+0000 7f994889e640 1 -- 172.21.15.32:0/1134818165 >> 172.21.15.32:0/1134818165 conn(0x7f9944071910 msgr2=0x7f9944071d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:34.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.424+0000 7f994889e640 1 -- 172.21.15.32:0/1134818165 shutdown_connections 2024-04-24T23:10:34.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.424+0000 7f994889e640 1 -- 172.21.15.32:0/1134818165 wait complete. 2024-04-24T23:10:34.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.425+0000 7f994889e640 1 Processor -- start 2024-04-24T23:10:34.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.425+0000 7f994889e640 1 -- start start 2024-04-24T23:10:34.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.426+0000 7f994889e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99440b6450 0x7f99441f44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:34.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.426+0000 7f994889e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99440b9ff0 0x7f9944161760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:34.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.426+0000 7f994889e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f994415e770 0x7f9944156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:34.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.426+0000 7f994889e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f994415eb50 con 0x7f99440b9ff0 2024-04-24T23:10:34.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.426+0000 7f994889e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f99440d3b80 con 0x7f994415e770 2024-04-24T23:10:34.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.426+0000 7f994889e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f994415ecd0 con 0x7f99440b6450 2024-04-24T23:10:34.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.426+0000 7f993bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99440b9ff0 0x7f9944161760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:34.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.426+0000 7f994259c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99440b6450 0x7f99441f44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:34.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.426+0000 7f994259c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99440b6450 0x7f99441f44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.32:51496/0 (socket says 172.21.15.32:51496) 2024-04-24T23:10:34.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.426+0000 7f994259c640 1 -- 172.21.15.32:0/6070001 learned_addr learned my addr 172.21.15.32:0/6070001 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:34.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.426+0000 7f9942d9d640 1 --2- 172.21.15.32:0/6070001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f994415e770 0x7f9944156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:34.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.426+0000 7f994259c640 1 -- 172.21.15.32:0/6070001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f994415e770 msgr2=0x7f9944156b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:34.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.426+0000 7f994259c640 1 --2- 172.21.15.32:0/6070001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f994415e770 0x7f9944156b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:34.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.426+0000 7f994259c640 1 -- 172.21.15.32:0/6070001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99440b9ff0 msgr2=0x7f9944161760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:34.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.426+0000 7f994259c640 1 --2- 172.21.15.32:0/6070001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99440b9ff0 0x7f9944161760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:34.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.426+0000 7f994259c640 1 -- 172.21.15.32:0/6070001 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99441609a0 con 0x7f99440b6450 2024-04-24T23:10:34.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.426+0000 7f993bfff640 1 --2- 172.21.15.32:0/6070001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99440b9ff0 0x7f9944161760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:10:34.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.426+0000 7f994259c640 1 --2- 172.21.15.32:0/6070001 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99440b6450 0x7f99441f44b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9934035040 tx=0x7f99340046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:34.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.427+0000 7f993affd640 1 -- 172.21.15.32:0/6070001 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9934037040 con 0x7f99440b6450 2024-04-24T23:10:34.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.427+0000 7f993affd640 1 -- 172.21.15.32:0/6070001 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9934030500 con 0x7f99440b6450 2024-04-24T23:10:34.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.427+0000 7f994889e640 1 -- 172.21.15.32:0/6070001 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9944160c80 con 0x7f99440b6450 2024-04-24T23:10:34.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.427+0000 7f993affd640 1 -- 172.21.15.32:0/6070001 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9934030910 con 0x7f99440b6450 2024-04-24T23:10:34.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.427+0000 7f994889e640 1 -- 172.21.15.32:0/6070001 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9944161ca0 con 0x7f99440b6450 2024-04-24T23:10:34.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.428+0000 7f994889e640 1 -- 172.21.15.32:0/6070001 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9944068430 con 0x7f99440b6450 2024-04-24T23:10:34.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.428+0000 7f993affd640 1 -- 172.21.15.32:0/6070001 <== mon.2 v2:172.21.15.121:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9934044020 con 0x7f99440b6450 2024-04-24T23:10:34.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.429+0000 7f993affd640 1 --2- 172.21.15.32:0/6070001 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f992406be30 0x7f992406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:34.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.429+0000 7f993affd640 1 -- 172.21.15.32:0/6070001 <== mon.2 v2:172.21.15.121:3301/0 5 ==== osd_map(181..181 src has 1..181) ==== 4857+0+0 (secure 0 0 0) 0x7f99340aad00 con 0x7f99440b6450 2024-04-24T23:10:34.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.429+0000 7f993bfff640 1 --2- 172.21.15.32:0/6070001 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f992406be30 0x7f992406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:34.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.429+0000 7f993bfff640 1 --2- 172.21.15.32:0/6070001 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f992406be30 0x7f992406e2f0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f99440cfbd0 tx=0x7f992c007e80 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:34.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.434+0000 7f993affd640 1 -- 172.21.15.32:0/6070001 <== mon.2 v2:172.21.15.121:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9934079310 con 0x7f99440b6450 2024-04-24T23:10:34.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.617+0000 7f994889e640 1 -- 172.21.15.32:0/6070001 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f99440c7740 con 0x7f99440b6450 2024-04-24T23:10:34.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.982+0000 7f993affd640 1 -- 172.21.15.32:0/6070001 <== mon.2 v2:172.21.15.121:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v182) ==== 138+0+0 (secure 0 0 0) 0x7f993407d610 con 0x7f99440b6450 2024-04-24T23:10:34.984 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2024-04-24T23:10:34.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.985+0000 7f994889e640 1 -- 172.21.15.32:0/6070001 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f992406be30 msgr2=0x7f992406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:34.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.985+0000 7f994889e640 1 --2- 172.21.15.32:0/6070001 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f992406be30 0x7f992406e2f0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f99440cfbd0 tx=0x7f992c007e80 comp rx=0 tx=0).stop 2024-04-24T23:10:34.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.985+0000 7f994889e640 1 -- 172.21.15.32:0/6070001 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99440b6450 msgr2=0x7f99441f44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:34.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.985+0000 7f994889e640 1 --2- 172.21.15.32:0/6070001 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99440b6450 0x7f99441f44b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9934035040 tx=0x7f99340046c0 comp rx=0 tx=0).stop 2024-04-24T23:10:34.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.985+0000 7f994889e640 1 -- 172.21.15.32:0/6070001 shutdown_connections 2024-04-24T23:10:34.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.985+0000 7f994889e640 1 --2- 172.21.15.32:0/6070001 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f992406be30 0x7f992406e2f0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:34.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.985+0000 7f994889e640 1 --2- 172.21.15.32:0/6070001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99440b9ff0 0x7f9944161760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:34.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.985+0000 7f994889e640 1 --2- 172.21.15.32:0/6070001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f994415e770 0x7f9944156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:34.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.986+0000 7f994889e640 1 --2- 172.21.15.32:0/6070001 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99440b6450 0x7f99441f44b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:34.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.986+0000 7f994889e640 1 -- 172.21.15.32:0/6070001 >> 172.21.15.32:0/6070001 conn(0x7f9944071910 msgr2=0x7f994415b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:34.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.986+0000 7f994889e640 1 -- 172.21.15.32:0/6070001 shutdown_connections 2024-04-24T23:10:34.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:34.986+0000 7f994889e640 1 -- 172.21.15.32:0/6070001 wait complete. 2024-04-24T23:10:34.999 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-24T23:10:35.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.107+0000 7f87eec8c640 1 Processor -- start 2024-04-24T23:10:35.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.107+0000 7f87eec8c640 1 -- start start 2024-04-24T23:10:35.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.108+0000 7f87eec8c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87e806cd00 0x7f87e80ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:35.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.108+0000 7f87eec8c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f87e815cb00 0x7f87e80ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:35.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.108+0000 7f87eec8c640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87e80b4520 con 0x7f87e815b380 2024-04-24T23:10:35.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.108+0000 7f87eec8c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f87e80b4800 con 0x7f87e815cb00 2024-04-24T23:10:35.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.108+0000 7f87eec8c640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f87e80b4ae0 con 0x7f87e806cd00 2024-04-24T23:10:35.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.108+0000 7f87ed229640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f87e815cb00 0x7f87e80ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:35.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.108+0000 7f87ed229640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f87e815cb00 0x7f87e80ce970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46822/0 (socket says 172.21.15.32:46822) 2024-04-24T23:10:35.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.108+0000 7f87ed229640 1 -- 172.21.15.32:0/1408781084 learned_addr learned my addr 172.21.15.32:0/1408781084 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:35.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.108+0000 7f87e7fff640 1 --2- 172.21.15.32:0/1408781084 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87e806cd00 0x7f87e80ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:35.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.108+0000 7f87ed229640 1 -- 172.21.15.32:0/1408781084 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87e806cd00 msgr2=0x7f87e80ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:35.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.108+0000 7f87ed229640 1 --2- 172.21.15.32:0/1408781084 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87e806cd00 0x7f87e80ce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:35.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.108+0000 7f87ed229640 1 -- 172.21.15.32:0/1408781084 >> v1:172.21.15.121:6789/0 conn(0x7f87e815b380 legacy=0x7f87e806c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:35.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.108+0000 7f87ed229640 1 -- 172.21.15.32:0/1408781084 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87e80b4dc0 con 0x7f87e815cb00 2024-04-24T23:10:35.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.109+0000 7f87ed229640 1 --2- 172.21.15.32:0/1408781084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87e815cb00 0x7f87e80ce970 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f87e0009e50 tx=0x7f87e002a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ebd450b5d0c91a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:35.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.109+0000 7f87e77fe640 1 -- 172.21.15.32:0/1408781084 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e0036070 con 0x7f87e815cb00 2024-04-24T23:10:35.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.109+0000 7f87e77fe640 1 -- 172.21.15.32:0/1408781084 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87e0031600 con 0x7f87e815cb00 2024-04-24T23:10:35.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.109+0000 7f87e77fe640 1 -- 172.21.15.32:0/1408781084 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e00319d0 con 0x7f87e815cb00 2024-04-24T23:10:35.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.110+0000 7f87eec8c640 1 -- 172.21.15.32:0/1408781084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87e815cb00 msgr2=0x7f87e80ce970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:35.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.110+0000 7f87eec8c640 1 --2- 172.21.15.32:0/1408781084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87e815cb00 0x7f87e80ce970 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f87e0009e50 tx=0x7f87e002a630 comp rx=0 tx=0).stop 2024-04-24T23:10:35.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.110+0000 7f87eec8c640 1 -- 172.21.15.32:0/1408781084 shutdown_connections 2024-04-24T23:10:35.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.110+0000 7f87eec8c640 1 --2- 172.21.15.32:0/1408781084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87e815cb00 0x7f87e80ce970 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:35.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.110+0000 7f87eec8c640 1 --2- 172.21.15.32:0/1408781084 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87e806cd00 0x7f87e80ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:35.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.110+0000 7f87eec8c640 1 -- 172.21.15.32:0/1408781084 >> 172.21.15.32:0/1408781084 conn(0x7f87e806b110 msgr2=0x7f87e806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:35.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.110+0000 7f87eec8c640 1 -- 172.21.15.32:0/1408781084 shutdown_connections 2024-04-24T23:10:35.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.110+0000 7f87eec8c640 1 -- 172.21.15.32:0/1408781084 wait complete. 2024-04-24T23:10:35.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.111+0000 7f87eec8c640 1 Processor -- start 2024-04-24T23:10:35.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.111+0000 7f87eec8c640 1 -- start start 2024-04-24T23:10:35.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.112+0000 7f87eec8c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87e806cd00 0x7f87e8071a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:35.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.112+0000 7f87eec8c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87e815b380 0x7f87e814cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:35.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.112+0000 7f87eec8c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87e815cb00 0x7f87e8149cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:35.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.112+0000 7f87eec8c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f87e815b760 con 0x7f87e815cb00 2024-04-24T23:10:35.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.112+0000 7f87eec8c640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f87e80ceeb0 con 0x7f87e806cd00 2024-04-24T23:10:35.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.112+0000 7f87eec8c640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f87e80b55c0 con 0x7f87e815b380 2024-04-24T23:10:35.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.112+0000 7f87ed229640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87e815cb00 0x7f87e8149cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:35.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.112+0000 7f87ed229640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87e815cb00 0x7f87e8149cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46826/0 (socket says 172.21.15.32:46826) 2024-04-24T23:10:35.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.112+0000 7f87ed229640 1 -- 172.21.15.32:0/3315732667 learned_addr learned my addr 172.21.15.32:0/3315732667 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:35.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.112+0000 7f87eca28640 1 --2- 172.21.15.32:0/3315732667 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87e806cd00 0x7f87e8071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:35.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.112+0000 7f87e7fff640 1 --2- 172.21.15.32:0/3315732667 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87e815b380 0x7f87e814cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:35.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.112+0000 7f87ed229640 1 -- 172.21.15.32:0/3315732667 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87e815b380 msgr2=0x7f87e814cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:35.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.112+0000 7f87ed229640 1 --2- 172.21.15.32:0/3315732667 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87e815b380 0x7f87e814cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:35.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.112+0000 7f87ed229640 1 -- 172.21.15.32:0/3315732667 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87e806cd00 msgr2=0x7f87e8071a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:35.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.112+0000 7f87ed229640 1 --2- 172.21.15.32:0/3315732667 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87e806cd00 0x7f87e8071a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:35.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.112+0000 7f87ed229640 1 -- 172.21.15.32:0/3315732667 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87e8146220 con 0x7f87e815cb00 2024-04-24T23:10:35.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.112+0000 7f87eca28640 1 --2- 172.21.15.32:0/3315732667 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87e806cd00 0x7f87e8071a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:10:35.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.113+0000 7f87ed229640 1 --2- 172.21.15.32:0/3315732667 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87e815cb00 0x7f87e8149cd0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f87e0009e50 tx=0x7f87e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:35.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.113+0000 7f87e57fa640 1 -- 172.21.15.32:0/3315732667 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e0036050 con 0x7f87e815cb00 2024-04-24T23:10:35.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.113+0000 7f87e57fa640 1 -- 172.21.15.32:0/3315732667 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87e0042070 con 0x7f87e815cb00 2024-04-24T23:10:35.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.113+0000 7f87eec8c640 1 -- 172.21.15.32:0/3315732667 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87e81477f0 con 0x7f87e815cb00 2024-04-24T23:10:35.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.113+0000 7f87e57fa640 1 -- 172.21.15.32:0/3315732667 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e0009510 con 0x7f87e815cb00 2024-04-24T23:10:35.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.113+0000 7f87eec8c640 1 -- 172.21.15.32:0/3315732667 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87e814c340 con 0x7f87e815cb00 2024-04-24T23:10:35.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.114+0000 7f87eec8c640 1 -- 172.21.15.32:0/3315732667 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87e8068430 con 0x7f87e815cb00 2024-04-24T23:10:35.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.115+0000 7f87e57fa640 1 -- 172.21.15.32:0/3315732667 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f87e0007bd0 con 0x7f87e815cb00 2024-04-24T23:10:35.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.115+0000 7f87e57fa640 1 --2- 172.21.15.32:0/3315732667 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f87d406be30 0x7f87d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:35.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.115+0000 7f87e57fa640 1 -- 172.21.15.32:0/3315732667 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(182..182 src has 1..182) ==== 5227+0+0 (secure 0 0 0) 0x7f87e00abba0 con 0x7f87e815cb00 2024-04-24T23:10:35.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.115+0000 7f87eca28640 1 --2- 172.21.15.32:0/3315732667 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f87d406be30 0x7f87d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:35.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.116+0000 7f87eca28640 1 --2- 172.21.15.32:0/3315732667 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f87d406be30 0x7f87d406e2f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f87d8003eb0 tx=0x7f87d8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:35.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.119+0000 7f87e57fa640 1 -- 172.21.15.32:0/3315732667 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f87e007a6b0 con 0x7f87e815cb00 2024-04-24T23:10:35.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.308+0000 7f87eec8c640 1 -- 172.21.15.32:0/3315732667 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f87e8067260 con 0x7f87e815cb00 2024-04-24T23:10:35.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.981+0000 7f87e57fa640 1 -- 172.21.15.32:0/3315732667 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v183) ==== 170+0+0 (secure 0 0 0) 0x7f87e0079fb0 con 0x7f87e815cb00 2024-04-24T23:10:35.983 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2024-04-24T23:10:35.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.984+0000 7f87eec8c640 1 -- 172.21.15.32:0/3315732667 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f87d406be30 msgr2=0x7f87d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:35.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.984+0000 7f87eec8c640 1 --2- 172.21.15.32:0/3315732667 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f87d406be30 0x7f87d406e2f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f87d8003eb0 tx=0x7f87d8009040 comp rx=0 tx=0).stop 2024-04-24T23:10:35.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.984+0000 7f87eec8c640 1 -- 172.21.15.32:0/3315732667 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87e815cb00 msgr2=0x7f87e8149cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:35.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.984+0000 7f87eec8c640 1 --2- 172.21.15.32:0/3315732667 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87e815cb00 0x7f87e8149cd0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f87e0009e50 tx=0x7f87e0004a40 comp rx=0 tx=0).stop 2024-04-24T23:10:35.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.984+0000 7f87eec8c640 1 -- 172.21.15.32:0/3315732667 shutdown_connections 2024-04-24T23:10:35.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.984+0000 7f87eec8c640 1 --2- 172.21.15.32:0/3315732667 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f87d406be30 0x7f87d406e2f0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:35.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.984+0000 7f87eec8c640 1 --2- 172.21.15.32:0/3315732667 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87e815cb00 0x7f87e8149cd0 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:35.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.984+0000 7f87eec8c640 1 --2- 172.21.15.32:0/3315732667 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87e815b380 0x7f87e814cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:35.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.984+0000 7f87eec8c640 1 --2- 172.21.15.32:0/3315732667 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87e806cd00 0x7f87e8071a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:35.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.984+0000 7f87eec8c640 1 -- 172.21.15.32:0/3315732667 >> 172.21.15.32:0/3315732667 conn(0x7f87e806b110 msgr2=0x7f87e80ce360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:35.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.984+0000 7f87eec8c640 1 -- 172.21.15.32:0/3315732667 shutdown_connections 2024-04-24T23:10:35.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:35.985+0000 7f87eec8c640 1 -- 172.21.15.32:0/3315732667 wait complete. 2024-04-24T23:10:36.011 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-24T23:10:36.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.119+0000 7fa95d334640 1 Processor -- start 2024-04-24T23:10:36.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.119+0000 7fa95d334640 1 -- start start 2024-04-24T23:10:36.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.119+0000 7fa95d334640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9580b0b80 0x7fa95806c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:36.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.119+0000 7fa95d334640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9580b0770 con 0x7fa958074210 2024-04-24T23:10:36.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.119+0000 7fa95d334640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9580b1010 con 0x7fa9580ae140 2024-04-24T23:10:36.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.119+0000 7fa95d334640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa9580b1c20 con 0x7fa9580b0b80 2024-04-24T23:10:36.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.120+0000 7fa956ffd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa958074210 0x7fa9580745f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37914/0 (socket says 172.21.15.32:37914) 2024-04-24T23:10:36.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.120+0000 7fa956ffd640 1 -- 172.21.15.32:0/785054116 learned_addr learned my addr 172.21.15.32:0/785054116 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:10:36.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.120+0000 7fa9567fc640 1 --2- 172.21.15.32:0/785054116 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9580b0b80 0x7fa95806c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:36.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.120+0000 7fa955ffb640 1 -- 172.21.15.32:0/785054116 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2258316050 0 0) 0x7fa9580b0770 con 0x7fa958074210 2024-04-24T23:10:36.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.120+0000 7fa955ffb640 1 -- 172.21.15.32:0/785054116 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa940003520 con 0x7fa958074210 2024-04-24T23:10:36.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.120+0000 7fa955ffb640 1 -- 172.21.15.32:0/785054116 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1103932343 0 0) 0x7fa9580b1010 con 0x7fa9580ae140 2024-04-24T23:10:36.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.120+0000 7fa955ffb640 1 -- 172.21.15.32:0/785054116 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa9580b0770 con 0x7fa9580ae140 2024-04-24T23:10:36.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.120+0000 7fa955ffb640 1 -- 172.21.15.32:0/785054116 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3691939710 0 0) 0x7fa940003520 con 0x7fa958074210 2024-04-24T23:10:36.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.120+0000 7fa955ffb640 1 -- 172.21.15.32:0/785054116 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9580b0b80 msgr2=0x7fa95806c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:36.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.120+0000 7fa955ffb640 1 --2- 172.21.15.32:0/785054116 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9580b0b80 0x7fa95806c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:36.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.120+0000 7fa955ffb640 1 -- 172.21.15.32:0/785054116 >> v1:172.21.15.121:6789/0 conn(0x7fa9580ae140 legacy=0x7fa9580b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:36.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.120+0000 7fa955ffb640 1 -- 172.21.15.32:0/785054116 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9580b12f0 con 0x7fa958074210 2024-04-24T23:10:36.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.120+0000 7fa955ffb640 1 -- 172.21.15.32:0/785054116 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7fa948001670 con 0x7fa958074210 2024-04-24T23:10:36.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.120+0000 7fa955ffb640 1 -- 172.21.15.32:0/785054116 >> v1:172.21.15.32:6789/0 conn(0x7fa958074210 legacy=0x7fa9580745f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:36.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.120+0000 7fa955ffb640 1 --2- 172.21.15.32:0/785054116 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa940003b70 0x7fa940003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:36.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.120+0000 7fa955ffb640 1 --2- 172.21.15.32:0/785054116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa940004800 0x7fa940004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:36.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.120+0000 7fa955ffb640 1 --2- 172.21.15.32:0/785054116 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9400051c0 0x7fa94000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:36.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.120+0000 7fa955ffb640 1 -- 172.21.15.32:0/785054116 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa9580b1c20 con 0x7fa940004800 2024-04-24T23:10:36.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.120+0000 7fa955ffb640 1 -- 172.21.15.32:0/785054116 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa94000bf00 con 0x7fa9400051c0 2024-04-24T23:10:36.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.120+0000 7fa955ffb640 1 -- 172.21.15.32:0/785054116 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa94000c080 con 0x7fa940003b70 2024-04-24T23:10:36.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.120+0000 7fa9567fc640 1 --2- 172.21.15.32:0/785054116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa940004800 0x7fa940004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:36.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.120+0000 7fa956ffd640 1 --2- 172.21.15.32:0/785054116 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa940003b70 0x7fa940003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:36.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.121+0000 7fa9577fe640 1 --2- 172.21.15.32:0/785054116 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9400051c0 0x7fa94000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:36.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.121+0000 7fa9567fc640 1 -- 172.21.15.32:0/785054116 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa940003b70 msgr2=0x7fa940003f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:36.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.121+0000 7fa9567fc640 1 --2- 172.21.15.32:0/785054116 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa940003b70 0x7fa940003f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:36.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.121+0000 7fa9567fc640 1 -- 172.21.15.32:0/785054116 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9400051c0 msgr2=0x7fa94000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:36.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.121+0000 7fa9567fc640 1 --2- 172.21.15.32:0/785054116 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9400051c0 0x7fa94000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:36.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.121+0000 7fa9567fc640 1 -- 172.21.15.32:0/785054116 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa940003520 con 0x7fa940004800 2024-04-24T23:10:36.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.121+0000 7fa9577fe640 1 --2- 172.21.15.32:0/785054116 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9400051c0 0x7fa94000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:10:36.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.121+0000 7fa9567fc640 1 --2- 172.21.15.32:0/785054116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa940004800 0x7fa940004c80 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fa94c009ac0 tx=0x7fa940005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=400c3d11c6cae3da server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:36.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.121+0000 7fa955ffb640 1 -- 172.21.15.32:0/785054116 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c036070 con 0x7fa940004800 2024-04-24T23:10:36.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.122+0000 7fa955ffb640 1 -- 172.21.15.32:0/785054116 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa94c031400 con 0x7fa940004800 2024-04-24T23:10:36.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.122+0000 7fa955ffb640 1 -- 172.21.15.32:0/785054116 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c009690 con 0x7fa940004800 2024-04-24T23:10:36.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.122+0000 7fa95d334640 1 -- 172.21.15.32:0/785054116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa940004800 msgr2=0x7fa940004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:36.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.122+0000 7fa95d334640 1 --2- 172.21.15.32:0/785054116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa940004800 0x7fa940004c80 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fa94c009ac0 tx=0x7fa940005830 comp rx=0 tx=0).stop 2024-04-24T23:10:36.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.122+0000 7fa9577fe640 1 -- 172.21.15.32:0/785054116 reap_dead start 2024-04-24T23:10:36.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.122+0000 7fa95d334640 1 -- 172.21.15.32:0/785054116 shutdown_connections 2024-04-24T23:10:36.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.122+0000 7fa95d334640 1 -- 172.21.15.32:0/785054116 >> 172.21.15.32:0/785054116 conn(0x7fa9580b2e60 msgr2=0x7fa9580b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:36.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.122+0000 7fa95d334640 1 -- 172.21.15.32:0/785054116 shutdown_connections 2024-04-24T23:10:36.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.122+0000 7fa95d334640 1 -- 172.21.15.32:0/785054116 wait complete. 2024-04-24T23:10:36.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.123+0000 7fa95d334640 1 Processor -- start 2024-04-24T23:10:36.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.123+0000 7fa95d334640 1 -- start start 2024-04-24T23:10:36.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.124+0000 7fa95d334640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa940004800 0x7fa95809edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:36.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.124+0000 7fa95d334640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9581f81d0 0x7fa95815e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:36.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.124+0000 7fa95d334640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9580c83c0 0x7fa9581f7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:36.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.124+0000 7fa95d334640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa9580b3ba0 con 0x7fa9580c83c0 2024-04-24T23:10:36.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.124+0000 7fa95d334640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa9580b18b0 con 0x7fa9581f81d0 2024-04-24T23:10:36.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.124+0000 7fa95d334640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa9580b3d20 con 0x7fa940004800 2024-04-24T23:10:36.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.124+0000 7fa9577fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9580c83c0 0x7fa9581f7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:36.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.124+0000 7fa9577fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9580c83c0 0x7fa9581f7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46838/0 (socket says 172.21.15.32:46838) 2024-04-24T23:10:36.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.124+0000 7fa9577fe640 1 -- 172.21.15.32:0/988420354 learned_addr learned my addr 172.21.15.32:0/988420354 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:36.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.124+0000 7fa956ffd640 1 --2- 172.21.15.32:0/988420354 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa940004800 0x7fa95809edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:36.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.125+0000 7fa9567fc640 1 --2- 172.21.15.32:0/988420354 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9581f81d0 0x7fa95815e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:36.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.125+0000 7fa9577fe640 1 -- 172.21.15.32:0/988420354 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa940004800 msgr2=0x7fa95809edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:36.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.125+0000 7fa9577fe640 1 --2- 172.21.15.32:0/988420354 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa940004800 0x7fa95809edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:36.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.125+0000 7fa9577fe640 1 -- 172.21.15.32:0/988420354 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9581f81d0 msgr2=0x7fa95815e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:36.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.125+0000 7fa9577fe640 1 --2- 172.21.15.32:0/988420354 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9581f81d0 0x7fa95815e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:36.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.125+0000 7fa9577fe640 1 -- 172.21.15.32:0/988420354 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9581f4b60 con 0x7fa9580c83c0 2024-04-24T23:10:36.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.125+0000 7fa9577fe640 1 --2- 172.21.15.32:0/988420354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9580c83c0 0x7fa9581f7630 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7fa9580ae520 tx=0x7fa95000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:36.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.125+0000 7fa93f7fe640 1 -- 172.21.15.32:0/988420354 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95000ee50 con 0x7fa9580c83c0 2024-04-24T23:10:36.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.125+0000 7fa93f7fe640 1 -- 172.21.15.32:0/988420354 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa950003b80 con 0x7fa9580c83c0 2024-04-24T23:10:36.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.125+0000 7fa95d334640 1 -- 172.21.15.32:0/988420354 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9581f4e40 con 0x7fa9580c83c0 2024-04-24T23:10:36.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.125+0000 7fa93f7fe640 1 -- 172.21.15.32:0/988420354 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa950003d70 con 0x7fa9580c83c0 2024-04-24T23:10:36.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.125+0000 7fa95d334640 1 -- 172.21.15.32:0/988420354 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9580caff0 con 0x7fa9580c83c0 2024-04-24T23:10:36.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.126+0000 7fa95d334640 1 -- 172.21.15.32:0/988420354 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa958068430 con 0x7fa9580c83c0 2024-04-24T23:10:36.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.127+0000 7fa93f7fe640 1 -- 172.21.15.32:0/988420354 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa95001a020 con 0x7fa9580c83c0 2024-04-24T23:10:36.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.127+0000 7fa93f7fe640 1 --2- 172.21.15.32:0/988420354 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa94406be60 0x7fa94406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:36.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.127+0000 7fa93f7fe640 1 -- 172.21.15.32:0/988420354 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 5609+0+0 (secure 0 0 0) 0x7fa95008a140 con 0x7fa9580c83c0 2024-04-24T23:10:36.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.127+0000 7fa956ffd640 1 --2- 172.21.15.32:0/988420354 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa94406be60 0x7fa94406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:36.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.128+0000 7fa956ffd640 1 --2- 172.21.15.32:0/988420354 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa94406be60 0x7fa94406e320 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fa9480011e0 tx=0x7fa948001e50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:36.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.132+0000 7fa93f7fe640 1 -- 172.21.15.32:0/988420354 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa950021020 con 0x7fa9580c83c0 2024-04-24T23:10:36.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.352+0000 7fa95d334640 1 -- 172.21.15.32:0/988420354 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fa95806b220 con 0x7fa9580c83c0 2024-04-24T23:10:36.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.991+0000 7fa93f7fe640 1 -- 172.21.15.32:0/988420354 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 22 and data pool 23 v78) ==== 174+0+0 (secure 0 0 0) 0x7fa950058500 con 0x7fa9580c83c0 2024-04-24T23:10:36.994 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 22 and data pool 23 2024-04-24T23:10:36.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.996+0000 7fa95d334640 1 -- 172.21.15.32:0/988420354 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa94406be60 msgr2=0x7fa94406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:36.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.996+0000 7fa95d334640 1 --2- 172.21.15.32:0/988420354 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa94406be60 0x7fa94406e320 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fa9480011e0 tx=0x7fa948001e50 comp rx=0 tx=0).stop 2024-04-24T23:10:36.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.996+0000 7fa95d334640 1 -- 172.21.15.32:0/988420354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9580c83c0 msgr2=0x7fa9581f7630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:36.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:36.996+0000 7fa95d334640 1 --2- 172.21.15.32:0/988420354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9580c83c0 0x7fa9581f7630 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7fa9580ae520 tx=0x7fa95000ec00 comp rx=0 tx=0).stop 2024-04-24T23:10:37.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.000+0000 7fa95d334640 1 -- 172.21.15.32:0/988420354 shutdown_connections 2024-04-24T23:10:37.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.000+0000 7fa95d334640 1 --2- 172.21.15.32:0/988420354 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa94406be60 0x7fa94406e320 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:37.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.000+0000 7fa95d334640 1 --2- 172.21.15.32:0/988420354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9580c83c0 0x7fa9581f7630 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:37.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.000+0000 7fa95d334640 1 --2- 172.21.15.32:0/988420354 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9581f81d0 0x7fa95815e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:37.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.000+0000 7fa95d334640 1 --2- 172.21.15.32:0/988420354 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa940004800 0x7fa95809edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:37.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.000+0000 7fa95d334640 1 -- 172.21.15.32:0/988420354 >> 172.21.15.32:0/988420354 conn(0x7fa9580b2e60 msgr2=0x7fa9580b20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:37.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.000+0000 7fa95d334640 1 -- 172.21.15.32:0/988420354 shutdown_connections 2024-04-24T23:10:37.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.000+0000 7fa95d334640 1 -- 172.21.15.32:0/988420354 wait complete. 2024-04-24T23:10:37.028 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:10:37.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.130+0000 7f14b2653640 1 Processor -- start 2024-04-24T23:10:37.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.130+0000 7f14b2653640 1 -- start start 2024-04-24T23:10:37.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.131+0000 7f14b2653640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f14ac15e9b0 0x7f14ac0d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:37.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.131+0000 7f14b2653640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f14ac0d4210 0x7f14ac06b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:37.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.131+0000 7f14b2653640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14ac0d3170 con 0x7f14ac0ba0e0 2024-04-24T23:10:37.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.131+0000 7f14b2653640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f14ac0b8900 con 0x7f14ac15e9b0 2024-04-24T23:10:37.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.131+0000 7f14b2653640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f14ac0d3450 con 0x7f14ac0d4210 2024-04-24T23:10:37.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.131+0000 7f14abfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f14ac15e9b0 0x7f14ac0d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:37.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.131+0000 7f14abfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f14ac15e9b0 0x7f14ac0d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46854/0 (socket says 172.21.15.32:46854) 2024-04-24T23:10:37.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.131+0000 7f14abfff640 1 -- 172.21.15.32:0/340357394 learned_addr learned my addr 172.21.15.32:0/340357394 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:37.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.131+0000 7f14b0bf0640 1 --1- 172.21.15.32:0/340357394 >> v1:172.21.15.32:6789/0 conn(0x7f14ac0ba0e0 0x7f14ac071730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37920/0 (socket says 172.21.15.32:37920) 2024-04-24T23:10:37.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.132+0000 7f14ab7fe640 1 --2- 172.21.15.32:0/340357394 >> v2:172.21.15.121:3300/0 conn(0x7f14ac0d4210 0x7f14ac06b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:37.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.132+0000 7f14abfff640 1 -- 172.21.15.32:0/340357394 >> v2:172.21.15.121:3300/0 conn(0x7f14ac0d4210 msgr2=0x7f14ac06b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:37.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.132+0000 7f14abfff640 1 --2- 172.21.15.32:0/340357394 >> v2:172.21.15.121:3300/0 conn(0x7f14ac0d4210 0x7f14ac06b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:37.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.132+0000 7f14abfff640 1 -- 172.21.15.32:0/340357394 >> v1:172.21.15.32:6789/0 conn(0x7f14ac0ba0e0 legacy=0x7f14ac071730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:37.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.132+0000 7f14abfff640 1 -- 172.21.15.32:0/340357394 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14ac0d3730 con 0x7f14ac15e9b0 2024-04-24T23:10:37.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.132+0000 7f14abfff640 1 --2- 172.21.15.32:0/340357394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14ac15e9b0 0x7f14ac0d2b90 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f14ac0730f0 tx=0x7f149402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=113d81a2f51e3574 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:37.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.133+0000 7f14aaffd640 1 -- 172.21.15.32:0/340357394 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1494037070 con 0x7f14ac15e9b0 2024-04-24T23:10:37.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.133+0000 7f14aaffd640 1 -- 172.21.15.32:0/340357394 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f149402ad90 con 0x7f14ac15e9b0 2024-04-24T23:10:37.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.133+0000 7f14aaffd640 1 -- 172.21.15.32:0/340357394 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1494032660 con 0x7f14ac15e9b0 2024-04-24T23:10:37.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.133+0000 7f14b2653640 1 -- 172.21.15.32:0/340357394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14ac15e9b0 msgr2=0x7f14ac0d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:37.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.133+0000 7f14b2653640 1 --2- 172.21.15.32:0/340357394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14ac15e9b0 0x7f14ac0d2b90 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f14ac0730f0 tx=0x7f149402a100 comp rx=0 tx=0).stop 2024-04-24T23:10:37.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.133+0000 7f14b2653640 1 -- 172.21.15.32:0/340357394 shutdown_connections 2024-04-24T23:10:37.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.133+0000 7f14b2653640 1 --2- 172.21.15.32:0/340357394 >> v2:172.21.15.121:3300/0 conn(0x7f14ac0d4210 0x7f14ac06b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:37.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.133+0000 7f14b2653640 1 --2- 172.21.15.32:0/340357394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14ac15e9b0 0x7f14ac0d2b90 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:37.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.133+0000 7f14b2653640 1 -- 172.21.15.32:0/340357394 >> 172.21.15.32:0/340357394 conn(0x7f14ac0d11d0 msgr2=0x7f14ac0d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:37.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.133+0000 7f14b2653640 1 -- 172.21.15.32:0/340357394 shutdown_connections 2024-04-24T23:10:37.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.133+0000 7f14b2653640 1 -- 172.21.15.32:0/340357394 wait complete. 2024-04-24T23:10:37.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.134+0000 7f14b2653640 1 Processor -- start 2024-04-24T23:10:37.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.135+0000 7f14b2653640 1 -- start start 2024-04-24T23:10:37.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.135+0000 7f14b2653640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14ac0ba0e0 0x7f14ac160de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:37.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.135+0000 7f14b2653640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14ac0d4210 0x7f14ac1539e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:37.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.135+0000 7f14b2653640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14ac15e9b0 0x7f14ac153f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:37.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.135+0000 7f14b2653640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f14ac0d2400 con 0x7f14ac0ba0e0 2024-04-24T23:10:37.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.135+0000 7f14b2653640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f14ac0d3cf0 con 0x7f14ac0d4210 2024-04-24T23:10:37.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.135+0000 7f14b2653640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f14ac06b650 con 0x7f14ac15e9b0 2024-04-24T23:10:37.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.135+0000 7f14abfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14ac0ba0e0 0x7f14ac160de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:37.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.135+0000 7f14abfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14ac0ba0e0 0x7f14ac160de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46862/0 (socket says 172.21.15.32:46862) 2024-04-24T23:10:37.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.135+0000 7f14abfff640 1 -- 172.21.15.32:0/1667214745 learned_addr learned my addr 172.21.15.32:0/1667214745 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:37.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.136+0000 7f14ab7fe640 1 --2- 172.21.15.32:0/1667214745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14ac0d4210 0x7f14ac1539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:37.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.136+0000 7f14b0bf0640 1 --2- 172.21.15.32:0/1667214745 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14ac15e9b0 0x7f14ac153f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:37.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.136+0000 7f14abfff640 1 -- 172.21.15.32:0/1667214745 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14ac15e9b0 msgr2=0x7f14ac153f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:37.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.136+0000 7f14abfff640 1 --2- 172.21.15.32:0/1667214745 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14ac15e9b0 0x7f14ac153f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:37.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.136+0000 7f14abfff640 1 -- 172.21.15.32:0/1667214745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14ac0d4210 msgr2=0x7f14ac1539e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:37.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.136+0000 7f14abfff640 1 --2- 172.21.15.32:0/1667214745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14ac0d4210 0x7f14ac1539e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:37.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.136+0000 7f14abfff640 1 -- 172.21.15.32:0/1667214745 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14ac0cc4c0 con 0x7f14ac0ba0e0 2024-04-24T23:10:37.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.136+0000 7f14abfff640 1 --2- 172.21.15.32:0/1667214745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14ac0ba0e0 0x7f14ac160de0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f1494000c00 tx=0x7f14940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:37.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.136+0000 7f14a8ff9640 1 -- 172.21.15.32:0/1667214745 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1494037040 con 0x7f14ac0ba0e0 2024-04-24T23:10:37.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.136+0000 7f14a8ff9640 1 -- 172.21.15.32:0/1667214745 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1494042070 con 0x7f14ac0ba0e0 2024-04-24T23:10:37.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.136+0000 7f14b2653640 1 -- 172.21.15.32:0/1667214745 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14ac0cc7a0 con 0x7f14ac0ba0e0 2024-04-24T23:10:37.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.136+0000 7f14a8ff9640 1 -- 172.21.15.32:0/1667214745 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149403b670 con 0x7f14ac0ba0e0 2024-04-24T23:10:37.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.136+0000 7f14b2653640 1 -- 172.21.15.32:0/1667214745 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14ac0ccd60 con 0x7f14ac0ba0e0 2024-04-24T23:10:37.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.137+0000 7f14b2653640 1 -- 172.21.15.32:0/1667214745 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14ac0b7500 con 0x7f14ac0ba0e0 2024-04-24T23:10:37.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.138+0000 7f14a8ff9640 1 -- 172.21.15.32:0/1667214745 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f149403b810 con 0x7f14ac0ba0e0 2024-04-24T23:10:37.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.138+0000 7f14a8ff9640 1 --2- 172.21.15.32:0/1667214745 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f148c06be30 0x7f148c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:37.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.138+0000 7f14a8ff9640 1 -- 172.21.15.32:0/1667214745 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(184..184 src has 1..184) ==== 5709+0+0 (secure 0 0 0) 0x7f14940ab3c0 con 0x7f14ac0ba0e0 2024-04-24T23:10:37.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.138+0000 7f14ab7fe640 1 --2- 172.21.15.32:0/1667214745 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f148c06be30 0x7f148c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:37.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.139+0000 7f14ab7fe640 1 --2- 172.21.15.32:0/1667214745 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f148c06be30 0x7f148c06e2f0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f14ac0725a0 tx=0x7f149c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:37.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.143+0000 7f14a8ff9640 1 -- 172.21.15.32:0/1667214745 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f14ac0b7500 con 0x7f14ac0ba0e0 2024-04-24T23:10:37.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.326+0000 7f14b2653640 1 -- 172.21.15.32:0/1667214745 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f14ac068430 con 0x7f14ac0ba0e0 2024-04-24T23:10:37.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.327+0000 7f14a8ff9640 1 -- 172.21.15.32:0/1667214745 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v184) ==== 74+0+16705 (secure 0 0 0) 0x7f14940a8160 con 0x7f14ac0ba0e0 2024-04-24T23:10:37.330 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:10:37.330 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":184,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:10:36.353456+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:10:34.649562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-24T23:10:35.339604+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:10:37.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.329+0000 7f14b2653640 1 -- 172.21.15.32:0/1667214745 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f148c06be30 msgr2=0x7f148c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:37.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.329+0000 7f14b2653640 1 --2- 172.21.15.32:0/1667214745 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f148c06be30 0x7f148c06e2f0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f14ac0725a0 tx=0x7f149c008040 comp rx=0 tx=0).stop 2024-04-24T23:10:37.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.329+0000 7f14b2653640 1 -- 172.21.15.32:0/1667214745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14ac0ba0e0 msgr2=0x7f14ac160de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:37.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.329+0000 7f14b2653640 1 --2- 172.21.15.32:0/1667214745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14ac0ba0e0 0x7f14ac160de0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f1494000c00 tx=0x7f14940046c0 comp rx=0 tx=0).stop 2024-04-24T23:10:37.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.329+0000 7f14b2653640 1 -- 172.21.15.32:0/1667214745 shutdown_connections 2024-04-24T23:10:37.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.329+0000 7f14b2653640 1 --2- 172.21.15.32:0/1667214745 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f148c06be30 0x7f148c06e2f0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:37.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.329+0000 7f14b2653640 1 --2- 172.21.15.32:0/1667214745 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14ac15e9b0 0x7f14ac153f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:37.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.329+0000 7f14b2653640 1 --2- 172.21.15.32:0/1667214745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14ac0d4210 0x7f14ac1539e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:37.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.329+0000 7f14b2653640 1 --2- 172.21.15.32:0/1667214745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14ac0ba0e0 0x7f14ac160de0 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:37.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.329+0000 7f14b2653640 1 -- 172.21.15.32:0/1667214745 >> 172.21.15.32:0/1667214745 conn(0x7f14ac0d11d0 msgr2=0x7f14ac0cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:37.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.329+0000 7f14b2653640 1 -- 172.21.15.32:0/1667214745 shutdown_connections 2024-04-24T23:10:37.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.330+0000 7f14b2653640 1 -- 172.21.15.32:0/1667214745 wait complete. 2024-04-24T23:10:37.343 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:10:37.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.442+0000 7fd126968640 1 Processor -- start 2024-04-24T23:10:37.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.442+0000 7fd126968640 1 -- start start 2024-04-24T23:10:37.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.443+0000 7fd126968640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd12015fd60 0x7fd12006c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:37.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.443+0000 7fd126968640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd12006cb30 0x7fd1200cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:37.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.443+0000 7fd126968640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1200b46c0 con 0x7fd12015b480 2024-04-24T23:10:37.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.443+0000 7fd126968640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fd1200b4ff0 con 0x7fd12015fd60 2024-04-24T23:10:37.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.443+0000 7fd126968640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd1200b49a0 con 0x7fd12006cb30 2024-04-24T23:10:37.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.443+0000 7fd11f7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd12015fd60 0x7fd12006c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:37.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.443+0000 7fd11ffff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd12015b480 0x7fd1200b5c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37928/0 (socket says 172.21.15.32:37928) 2024-04-24T23:10:37.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.443+0000 7fd11f7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd12015fd60 0x7fd12006c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46868/0 (socket says 172.21.15.32:46868) 2024-04-24T23:10:37.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.443+0000 7fd11ffff640 1 -- 172.21.15.32:0/1986799506 learned_addr learned my addr 172.21.15.32:0/1986799506 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:10:37.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.443+0000 7fd124f05640 1 --2- 172.21.15.32:0/1986799506 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd12006cb30 0x7fd1200cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:37.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.443+0000 7fd11f7fe640 1 -- 172.21.15.32:0/1986799506 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd12006cb30 msgr2=0x7fd1200cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:37.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.443+0000 7fd11f7fe640 1 --2- 172.21.15.32:0/1986799506 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd12006cb30 0x7fd1200cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:37.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.443+0000 7fd11f7fe640 1 -- 172.21.15.32:0/1986799506 >> v1:172.21.15.32:6789/0 conn(0x7fd12015b480 legacy=0x7fd1200b5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:37.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.443+0000 7fd11f7fe640 1 -- 172.21.15.32:0/1986799506 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1200b4c80 con 0x7fd12015fd60 2024-04-24T23:10:37.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.443+0000 7fd11effd640 1 -- 172.21.15.32:0/1986799506 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3551500036 0 0) 0x7fd1200b46c0 con 0x7fd12015b480 2024-04-24T23:10:37.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.443+0000 7fd124f05640 1 --2- 172.21.15.32:0/1986799506 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd12006cb30 0x7fd1200cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:10:37.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.444+0000 7fd11f7fe640 1 --2- 172.21.15.32:0/1986799506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd12015fd60 0x7fd12006c5f0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fd1140097a0 tx=0x7fd11402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83b9b27119ffba53 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:37.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.444+0000 7fd11effd640 1 -- 172.21.15.32:0/1986799506 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd114037070 con 0x7fd12015fd60 2024-04-24T23:10:37.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.444+0000 7fd11effd640 1 -- 172.21.15.32:0/1986799506 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd114032440 con 0x7fd12015fd60 2024-04-24T23:10:37.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.444+0000 7fd11effd640 1 -- 172.21.15.32:0/1986799506 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd114032810 con 0x7fd12015fd60 2024-04-24T23:10:37.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.444+0000 7fd126968640 1 -- 172.21.15.32:0/1986799506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd12015fd60 msgr2=0x7fd12006c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:37.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.444+0000 7fd126968640 1 --2- 172.21.15.32:0/1986799506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd12015fd60 0x7fd12006c5f0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fd1140097a0 tx=0x7fd11402a180 comp rx=0 tx=0).stop 2024-04-24T23:10:37.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.444+0000 7fd126968640 1 -- 172.21.15.32:0/1986799506 shutdown_connections 2024-04-24T23:10:37.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.444+0000 7fd126968640 1 --2- 172.21.15.32:0/1986799506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd12015fd60 0x7fd12006c5f0 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:37.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.444+0000 7fd126968640 1 --2- 172.21.15.32:0/1986799506 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd12006cb30 0x7fd1200cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:37.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.444+0000 7fd126968640 1 -- 172.21.15.32:0/1986799506 >> 172.21.15.32:0/1986799506 conn(0x7fd12006b110 msgr2=0x7fd12006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:37.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.445+0000 7fd126968640 1 -- 172.21.15.32:0/1986799506 shutdown_connections 2024-04-24T23:10:37.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.445+0000 7fd126968640 1 -- 172.21.15.32:0/1986799506 wait complete. 2024-04-24T23:10:37.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.446+0000 7fd126968640 1 Processor -- start 2024-04-24T23:10:37.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.446+0000 7fd126968640 1 -- start start 2024-04-24T23:10:37.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.447+0000 7fd126968640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd12006cb30 0x7fd12015df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:37.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.447+0000 7fd126968640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd12015b480 0x7fd12015cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:37.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.447+0000 7fd126968640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd12015fd60 0x7fd12015d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:37.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.447+0000 7fd126968640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd12015b860 con 0x7fd12006cb30 2024-04-24T23:10:37.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.447+0000 7fd126968640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd1200ce4c0 con 0x7fd12015b480 2024-04-24T23:10:37.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.447+0000 7fd126968640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd1201432c0 con 0x7fd12015fd60 2024-04-24T23:10:37.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.447+0000 7fd11ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd12006cb30 0x7fd12015df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:37.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.447+0000 7fd11ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd12006cb30 0x7fd12015df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46876/0 (socket says 172.21.15.32:46876) 2024-04-24T23:10:37.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.447+0000 7fd11ffff640 1 -- 172.21.15.32:0/670035591 learned_addr learned my addr 172.21.15.32:0/670035591 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:37.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.447+0000 7fd124f05640 1 --2- 172.21.15.32:0/670035591 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd12015fd60 0x7fd12015d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:37.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.447+0000 7fd11f7fe640 1 --2- 172.21.15.32:0/670035591 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd12015b480 0x7fd12015cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:37.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.447+0000 7fd11ffff640 1 -- 172.21.15.32:0/670035591 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd12015fd60 msgr2=0x7fd12015d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:37.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.447+0000 7fd11ffff640 1 --2- 172.21.15.32:0/670035591 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd12015fd60 0x7fd12015d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:37.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.447+0000 7fd11ffff640 1 -- 172.21.15.32:0/670035591 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd12015b480 msgr2=0x7fd12015cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:37.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.447+0000 7fd11ffff640 1 --2- 172.21.15.32:0/670035591 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd12015b480 0x7fd12015cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:37.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.447+0000 7fd11ffff640 1 -- 172.21.15.32:0/670035591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd12015aae0 con 0x7fd12006cb30 2024-04-24T23:10:37.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.447+0000 7fd11ffff640 1 --2- 172.21.15.32:0/670035591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd12006cb30 0x7fd12015df00 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7fd11000e960 tx=0x7fd11000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:37.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.447+0000 7fd11cff9640 1 -- 172.21.15.32:0/670035591 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1100072e0 con 0x7fd12006cb30 2024-04-24T23:10:37.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.447+0000 7fd11cff9640 1 -- 172.21.15.32:0/670035591 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd110016020 con 0x7fd12006cb30 2024-04-24T23:10:37.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.447+0000 7fd126968640 1 -- 172.21.15.32:0/670035591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd12015a800 con 0x7fd12006cb30 2024-04-24T23:10:37.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.447+0000 7fd11cff9640 1 -- 172.21.15.32:0/670035591 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11000c850 con 0x7fd12006cb30 2024-04-24T23:10:37.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.447+0000 7fd126968640 1 -- 172.21.15.32:0/670035591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd120142f50 con 0x7fd12006cb30 2024-04-24T23:10:37.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.449+0000 7fd126968640 1 -- 172.21.15.32:0/670035591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd120068430 con 0x7fd12006cb30 2024-04-24T23:10:37.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.449+0000 7fd11cff9640 1 -- 172.21.15.32:0/670035591 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd11000c9f0 con 0x7fd12006cb30 2024-04-24T23:10:37.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.449+0000 7fd11cff9640 1 --2- 172.21.15.32:0/670035591 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd10c06be30 0x7fd10c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:37.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.449+0000 7fd11cff9640 1 -- 172.21.15.32:0/670035591 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(184..184 src has 1..184) ==== 5709+0+0 (secure 0 0 0) 0x7fd11008d3f0 con 0x7fd12006cb30 2024-04-24T23:10:37.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.450+0000 7fd11f7fe640 1 --2- 172.21.15.32:0/670035591 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd10c06be30 0x7fd10c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:37.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.450+0000 7fd11f7fe640 1 --2- 172.21.15.32:0/670035591 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd10c06be30 0x7fd10c06e2f0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fd114034000 tx=0x7fd11402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:37.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.454+0000 7fd11cff9640 1 -- 172.21.15.32:0/670035591 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd110020020 con 0x7fd12006cb30 2024-04-24T23:10:37.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.639+0000 7fd126968640 1 -- 172.21.15.32:0/670035591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd12006c000 con 0x7fd12006cb30 2024-04-24T23:10:37.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.640+0000 7fd11cff9640 1 -- 172.21.15.32:0/670035591 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v184) ==== 74+0+16705 (secure 0 0 0) 0x7fd11005b750 con 0x7fd12006cb30 2024-04-24T23:10:37.643 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:10:37.643 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":184,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:10:36.353456+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:10:34.649562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-24T23:10:35.339604+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:10:37.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.642+0000 7fd126968640 1 -- 172.21.15.32:0/670035591 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd10c06be30 msgr2=0x7fd10c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:37.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.642+0000 7fd126968640 1 --2- 172.21.15.32:0/670035591 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd10c06be30 0x7fd10c06e2f0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fd114034000 tx=0x7fd11402e000 comp rx=0 tx=0).stop 2024-04-24T23:10:37.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.642+0000 7fd126968640 1 -- 172.21.15.32:0/670035591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd12006cb30 msgr2=0x7fd12015df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:37.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.642+0000 7fd126968640 1 --2- 172.21.15.32:0/670035591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd12006cb30 0x7fd12015df00 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7fd11000e960 tx=0x7fd11000ee20 comp rx=0 tx=0).stop 2024-04-24T23:10:37.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.642+0000 7fd126968640 1 -- 172.21.15.32:0/670035591 shutdown_connections 2024-04-24T23:10:37.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.642+0000 7fd126968640 1 --2- 172.21.15.32:0/670035591 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd10c06be30 0x7fd10c06e2f0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:37.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.642+0000 7fd126968640 1 --2- 172.21.15.32:0/670035591 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd12015fd60 0x7fd12015d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:37.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.642+0000 7fd126968640 1 --2- 172.21.15.32:0/670035591 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd12015b480 0x7fd12015cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:37.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.642+0000 7fd126968640 1 --2- 172.21.15.32:0/670035591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd12006cb30 0x7fd12015df00 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:37.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.642+0000 7fd126968640 1 -- 172.21.15.32:0/670035591 >> 172.21.15.32:0/670035591 conn(0x7fd12006b110 msgr2=0x7fd120072810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:37.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.642+0000 7fd126968640 1 -- 172.21.15.32:0/670035591 shutdown_connections 2024-04-24T23:10:37.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.642+0000 7fd126968640 1 -- 172.21.15.32:0/670035591 wait complete. 2024-04-24T23:10:37.656 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-24T23:10:37.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.758+0000 7f6195800640 1 Processor -- start 2024-04-24T23:10:37.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.758+0000 7f6195800640 1 -- start start 2024-04-24T23:10:37.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.759+0000 7f6195800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f619015b4d0 0x7f6190072f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:37.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.759+0000 7f6195800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6190073460 0x7f61900b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:37.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.759+0000 7f6195800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f61900b23e0 0x7f619015eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:37.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.759+0000 7f6195800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f619015b8b0 con 0x7f619015b4d0 2024-04-24T23:10:37.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.759+0000 7f6195800640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f61900ce100 con 0x7f61900b23e0 2024-04-24T23:10:37.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.759+0000 7f6195800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f61900ce3e0 con 0x7f6190073460 2024-04-24T23:10:37.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.759+0000 7f618effd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f619015b4d0 0x7f6190072f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:37.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.759+0000 7f618effd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f619015b4d0 0x7f6190072f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46888/0 (socket says 172.21.15.32:46888) 2024-04-24T23:10:37.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.759+0000 7f618effd640 1 -- 172.21.15.32:0/2352532552 learned_addr learned my addr 172.21.15.32:0/2352532552 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:37.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.759+0000 7f6187fff640 1 --2- 172.21.15.32:0/2352532552 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6190073460 0x7f61900b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:37.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.759+0000 7f618f7fe640 1 --2- 172.21.15.32:0/2352532552 >> v2:172.21.15.121:3300/0 conn(0x7f61900b23e0 0x7f619015eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:37.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.759+0000 7f618effd640 1 -- 172.21.15.32:0/2352532552 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6190073460 msgr2=0x7f61900b1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:37.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.759+0000 7f618effd640 1 --2- 172.21.15.32:0/2352532552 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6190073460 0x7f61900b1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:37.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.759+0000 7f618effd640 1 -- 172.21.15.32:0/2352532552 >> v2:172.21.15.121:3300/0 conn(0x7f61900b23e0 msgr2=0x7f619015eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:37.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.759+0000 7f618effd640 1 --2- 172.21.15.32:0/2352532552 >> v2:172.21.15.121:3300/0 conn(0x7f61900b23e0 0x7f619015eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:37.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.759+0000 7f618effd640 1 -- 172.21.15.32:0/2352532552 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61900ce6c0 con 0x7f619015b4d0 2024-04-24T23:10:37.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.759+0000 7f618f7fe640 1 --2- 172.21.15.32:0/2352532552 >> v2:172.21.15.121:3300/0 conn(0x7f61900b23e0 0x7f619015eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:10:37.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.760+0000 7f618effd640 1 --2- 172.21.15.32:0/2352532552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f619015b4d0 0x7f6190072f20 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f61900cc480 tx=0x7f618002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0801f4ff513bc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:37.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.760+0000 7f618e7fc640 1 -- 172.21.15.32:0/2352532552 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6180037070 con 0x7f619015b4d0 2024-04-24T23:10:37.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.760+0000 7f618e7fc640 1 -- 172.21.15.32:0/2352532552 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6180032440 con 0x7f619015b4d0 2024-04-24T23:10:37.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.760+0000 7f618e7fc640 1 -- 172.21.15.32:0/2352532552 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6180032810 con 0x7f619015b4d0 2024-04-24T23:10:37.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.761+0000 7f6195800640 1 -- 172.21.15.32:0/2352532552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f619015b4d0 msgr2=0x7f6190072f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:37.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.761+0000 7f6195800640 1 --2- 172.21.15.32:0/2352532552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f619015b4d0 0x7f6190072f20 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f61900cc480 tx=0x7f618002a140 comp rx=0 tx=0).stop 2024-04-24T23:10:37.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.761+0000 7f6195800640 1 -- 172.21.15.32:0/2352532552 shutdown_connections 2024-04-24T23:10:37.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.761+0000 7f6195800640 1 --2- 172.21.15.32:0/2352532552 >> v2:172.21.15.121:3300/0 conn(0x7f61900b23e0 0x7f619015eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:37.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.761+0000 7f6195800640 1 --2- 172.21.15.32:0/2352532552 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6190073460 0x7f61900b1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:37.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.761+0000 7f6195800640 1 --2- 172.21.15.32:0/2352532552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f619015b4d0 0x7f6190072f20 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:37.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.761+0000 7f6195800640 1 -- 172.21.15.32:0/2352532552 >> 172.21.15.32:0/2352532552 conn(0x7f6190074840 msgr2=0x7f619006b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:37.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.761+0000 7f6195800640 1 -- 172.21.15.32:0/2352532552 shutdown_connections 2024-04-24T23:10:37.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.761+0000 7f6195800640 1 -- 172.21.15.32:0/2352532552 wait complete. 2024-04-24T23:10:37.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.762+0000 7f6195800640 1 Processor -- start 2024-04-24T23:10:37.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.762+0000 7f6195800640 1 -- start start 2024-04-24T23:10:37.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.763+0000 7f6195800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6190073460 0x7f619015ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:37.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.763+0000 7f6195800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61900b23e0 0x7f6190152660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:37.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.763+0000 7f6195800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f619015b4d0 0x7f619013aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:37.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.763+0000 7f6195800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f619015f530 con 0x7f61900b23e0 2024-04-24T23:10:37.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.763+0000 7f6195800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f61900cec80 con 0x7f6190073460 2024-04-24T23:10:37.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.763+0000 7f6195800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f619015f6b0 con 0x7f619015b4d0 2024-04-24T23:10:37.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.763+0000 7f6187fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61900b23e0 0x7f6190152660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:37.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.763+0000 7f6187fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61900b23e0 0x7f6190152660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46902/0 (socket says 172.21.15.32:46902) 2024-04-24T23:10:37.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.763+0000 7f6187fff640 1 -- 172.21.15.32:0/2994949047 learned_addr learned my addr 172.21.15.32:0/2994949047 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:37.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.763+0000 7f618effd640 1 --2- 172.21.15.32:0/2994949047 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6190073460 0x7f619015ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:37.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.763+0000 7f618f7fe640 1 --2- 172.21.15.32:0/2994949047 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f619015b4d0 0x7f619013aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:37.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.763+0000 7f6187fff640 1 -- 172.21.15.32:0/2994949047 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f619015b4d0 msgr2=0x7f619013aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:37.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.763+0000 7f6187fff640 1 --2- 172.21.15.32:0/2994949047 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f619015b4d0 0x7f619013aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:37.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.763+0000 7f6187fff640 1 -- 172.21.15.32:0/2994949047 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6190073460 msgr2=0x7f619015ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:37.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.763+0000 7f6187fff640 1 --2- 172.21.15.32:0/2994949047 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6190073460 0x7f619015ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:37.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.763+0000 7f6187fff640 1 -- 172.21.15.32:0/2994949047 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f619015d680 con 0x7f61900b23e0 2024-04-24T23:10:37.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.763+0000 7f6187fff640 1 --2- 172.21.15.32:0/2994949047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61900b23e0 0x7f6190152660 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f617800c7f0 tx=0x7f617800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:37.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.764+0000 7f61877fe640 1 -- 172.21.15.32:0/2994949047 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6178015070 con 0x7f61900b23e0 2024-04-24T23:10:37.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.764+0000 7f61877fe640 1 -- 172.21.15.32:0/2994949047 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6178004220 con 0x7f61900b23e0 2024-04-24T23:10:37.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.764+0000 7f61877fe640 1 -- 172.21.15.32:0/2994949047 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6178007620 con 0x7f61900b23e0 2024-04-24T23:10:37.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.764+0000 7f6195800640 1 -- 172.21.15.32:0/2994949047 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f619015c010 con 0x7f61900b23e0 2024-04-24T23:10:37.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.764+0000 7f6195800640 1 -- 172.21.15.32:0/2994949047 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f619015c5d0 con 0x7f61900b23e0 2024-04-24T23:10:37.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.765+0000 7f6195800640 1 -- 172.21.15.32:0/2994949047 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61900682b0 con 0x7f61900b23e0 2024-04-24T23:10:37.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.765+0000 7f61877fe640 1 -- 172.21.15.32:0/2994949047 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f61780079d0 con 0x7f61900b23e0 2024-04-24T23:10:37.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.766+0000 7f61877fe640 1 --2- 172.21.15.32:0/2994949047 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f617006be60 0x7f617006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:37.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.766+0000 7f61877fe640 1 -- 172.21.15.32:0/2994949047 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(184..184 src has 1..184) ==== 5709+0+0 (secure 0 0 0) 0x7f617808ae50 con 0x7f61900b23e0 2024-04-24T23:10:37.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.766+0000 7f618effd640 1 --2- 172.21.15.32:0/2994949047 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f617006be60 0x7f617006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:37.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.766+0000 7f618effd640 1 --2- 172.21.15.32:0/2994949047 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f617006be60 0x7f617006e320 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f6180034000 tx=0x7f618002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:37.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.770+0000 7f61877fe640 1 -- 172.21.15.32:0/2994949047 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f619015d680 con 0x7f61900b23e0 2024-04-24T23:10:37.986 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:10:37.989 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:10:37.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:37.992+0000 7f6195800640 1 -- 172.21.15.32:0/2994949047 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f619015ed60 con 0x7f61900b23e0 2024-04-24T23:10:38.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.009+0000 7f61877fe640 1 -- 172.21.15.32:0/2994949047 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v79) ==== 118+0+0 (secure 0 0 0) 0x7f6178059120 con 0x7f61900b23e0 2024-04-24T23:10:38.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.011+0000 7f6195800640 1 -- 172.21.15.32:0/2994949047 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f617006be60 msgr2=0x7f617006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:38.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.011+0000 7f6195800640 1 --2- 172.21.15.32:0/2994949047 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f617006be60 0x7f617006e320 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f6180034000 tx=0x7f618002e000 comp rx=0 tx=0).stop 2024-04-24T23:10:38.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.011+0000 7f6195800640 1 -- 172.21.15.32:0/2994949047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61900b23e0 msgr2=0x7f6190152660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:38.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.011+0000 7f6195800640 1 --2- 172.21.15.32:0/2994949047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61900b23e0 0x7f6190152660 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f617800c7f0 tx=0x7f617800ccb0 comp rx=0 tx=0).stop 2024-04-24T23:10:38.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.011+0000 7f6195800640 1 -- 172.21.15.32:0/2994949047 shutdown_connections 2024-04-24T23:10:38.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.011+0000 7f6195800640 1 --2- 172.21.15.32:0/2994949047 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f617006be60 0x7f617006e320 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:38.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.011+0000 7f6195800640 1 --2- 172.21.15.32:0/2994949047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61900b23e0 0x7f6190152660 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:38.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.011+0000 7f6195800640 1 --2- 172.21.15.32:0/2994949047 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f619015b4d0 0x7f619013aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:38.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.011+0000 7f6195800640 1 --2- 172.21.15.32:0/2994949047 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6190073460 0x7f619015ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:38.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.011+0000 7f6195800640 1 -- 172.21.15.32:0/2994949047 >> 172.21.15.32:0/2994949047 conn(0x7f6190074840 msgr2=0x7f619014f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:38.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.011+0000 7f6195800640 1 -- 172.21.15.32:0/2994949047 shutdown_connections 2024-04-24T23:10:38.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.011+0000 7f6195800640 1 -- 172.21.15.32:0/2994949047 wait complete. 2024-04-24T23:10:38.025 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-24T23:10:38.025 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-24T23:10:38.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.202+0000 7f9c7386b640 1 Processor -- start 2024-04-24T23:10:38.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.202+0000 7f9c7386b640 1 -- start start 2024-04-24T23:10:38.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.203+0000 7f9c7386b640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9c6c15b380 0x7f9c6c06c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:38.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.203+0000 7f9c7386b640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c6c15cb00 0x7f9c6c06cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:38.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.203+0000 7f9c7386b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9c6c0ce430 0x7f9c6c0ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:38.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.203+0000 7f9c7386b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f9c6c0b4520 con 0x7f9c6c0ce430 2024-04-24T23:10:38.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.203+0000 7f9c7386b640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f9c6c0b4800 con 0x7f9c6c15b380 2024-04-24T23:10:38.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.203+0000 7f9c7386b640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f9c6c0b4ae0 con 0x7f9c6c15cb00 2024-04-24T23:10:38.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.203+0000 7f9c71e08640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9c6c0ce430 0x7f9c6c0ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:38.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.203+0000 7f9c71e08640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9c6c0ce430 0x7f9c6c0ce880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46916/0 (socket says 172.21.15.32:46916) 2024-04-24T23:10:38.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.203+0000 7f9c71e08640 1 -- 172.21.15.32:0/658564954 learned_addr learned my addr 172.21.15.32:0/658564954 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:38.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.203+0000 7f9c70e06640 1 --2- 172.21.15.32:0/658564954 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c6c15cb00 0x7f9c6c06cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:38.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.203+0000 7f9c71607640 1 --2- 172.21.15.32:0/658564954 >> v2:172.21.15.121:3300/0 conn(0x7f9c6c15b380 0x7f9c6c06c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:38.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.203+0000 7f9c71e08640 1 -- 172.21.15.32:0/658564954 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c6c15cb00 msgr2=0x7f9c6c06cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:38.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.203+0000 7f9c71e08640 1 --2- 172.21.15.32:0/658564954 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c6c15cb00 0x7f9c6c06cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:38.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.203+0000 7f9c71e08640 1 -- 172.21.15.32:0/658564954 >> v2:172.21.15.121:3300/0 conn(0x7f9c6c15b380 msgr2=0x7f9c6c06c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:38.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.203+0000 7f9c71e08640 1 --2- 172.21.15.32:0/658564954 >> v2:172.21.15.121:3300/0 conn(0x7f9c6c15b380 0x7f9c6c06c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:38.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.203+0000 7f9c71e08640 1 -- 172.21.15.32:0/658564954 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c6c0b4dc0 con 0x7f9c6c0ce430 2024-04-24T23:10:38.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.203+0000 7f9c70e06640 1 --2- 172.21.15.32:0/658564954 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c6c15cb00 0x7f9c6c06cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:10:38.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.204+0000 7f9c71e08640 1 --2- 172.21.15.32:0/658564954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c6c0ce430 0x7f9c6c0ce880 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c06eed0 tx=0x7f9c6402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcc6e53bd8cfccc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:38.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.204+0000 7f9c63fff640 1 -- 172.21.15.32:0/658564954 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c64036070 con 0x7f9c6c0ce430 2024-04-24T23:10:38.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.204+0000 7f9c63fff640 1 -- 172.21.15.32:0/658564954 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c64031600 con 0x7f9c6c0ce430 2024-04-24T23:10:38.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.204+0000 7f9c63fff640 1 -- 172.21.15.32:0/658564954 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c640319d0 con 0x7f9c6c0ce430 2024-04-24T23:10:38.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.205+0000 7f9c7386b640 1 -- 172.21.15.32:0/658564954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c6c0ce430 msgr2=0x7f9c6c0ce880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:38.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.205+0000 7f9c7386b640 1 --2- 172.21.15.32:0/658564954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c6c0ce430 0x7f9c6c0ce880 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c06eed0 tx=0x7f9c6402a610 comp rx=0 tx=0).stop 2024-04-24T23:10:38.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.205+0000 7f9c7386b640 1 -- 172.21.15.32:0/658564954 shutdown_connections 2024-04-24T23:10:38.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.205+0000 7f9c7386b640 1 --2- 172.21.15.32:0/658564954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c6c0ce430 0x7f9c6c0ce880 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:38.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.205+0000 7f9c7386b640 1 --2- 172.21.15.32:0/658564954 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c6c15cb00 0x7f9c6c06cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:38.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.205+0000 7f9c7386b640 1 --2- 172.21.15.32:0/658564954 >> v2:172.21.15.121:3300/0 conn(0x7f9c6c15b380 0x7f9c6c06c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:38.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.205+0000 7f9c7386b640 1 -- 172.21.15.32:0/658564954 >> 172.21.15.32:0/658564954 conn(0x7f9c6c06b110 msgr2=0x7f9c6c06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:38.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.205+0000 7f9c7386b640 1 -- 172.21.15.32:0/658564954 shutdown_connections 2024-04-24T23:10:38.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.205+0000 7f9c7386b640 1 -- 172.21.15.32:0/658564954 wait complete. 2024-04-24T23:10:38.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.206+0000 7f9c7386b640 1 Processor -- start 2024-04-24T23:10:38.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.206+0000 7f9c7386b640 1 -- start start 2024-04-24T23:10:38.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.207+0000 7f9c7386b640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c6c0ce430 0x7f9c6c147b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:38.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.207+0000 7f9c7386b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c6c15b380 0x7f9c6c1445a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:38.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.207+0000 7f9c7386b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c6c15cb00 0x7f9c6c1588a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:38.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.207+0000 7f9c7386b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9c6c0cedc0 con 0x7f9c6c15b380 2024-04-24T23:10:38.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.207+0000 7f9c7386b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9c6c15b760 con 0x7f9c6c15cb00 2024-04-24T23:10:38.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.207+0000 7f9c7386b640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f9c6c0cef40 con 0x7f9c6c0ce430 2024-04-24T23:10:38.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.207+0000 7f9c70e06640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c6c15b380 0x7f9c6c1445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:38.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.207+0000 7f9c70e06640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c6c15b380 0x7f9c6c1445a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46920/0 (socket says 172.21.15.32:46920) 2024-04-24T23:10:38.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.207+0000 7f9c70e06640 1 -- 172.21.15.32:0/2022647621 learned_addr learned my addr 172.21.15.32:0/2022647621 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:38.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.207+0000 7f9c71607640 1 --2- 172.21.15.32:0/2022647621 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c6c0ce430 0x7f9c6c147b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:38.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.207+0000 7f9c71e08640 1 --2- 172.21.15.32:0/2022647621 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c6c15cb00 0x7f9c6c1588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:38.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.207+0000 7f9c70e06640 1 -- 172.21.15.32:0/2022647621 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c6c0ce430 msgr2=0x7f9c6c147b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:38.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.207+0000 7f9c70e06640 1 --2- 172.21.15.32:0/2022647621 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c6c0ce430 0x7f9c6c147b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:38.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.207+0000 7f9c70e06640 1 -- 172.21.15.32:0/2022647621 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c6c15cb00 msgr2=0x7f9c6c1588a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:38.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.207+0000 7f9c70e06640 1 --2- 172.21.15.32:0/2022647621 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c6c15cb00 0x7f9c6c1588a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:38.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.207+0000 7f9c70e06640 1 -- 172.21.15.32:0/2022647621 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c6c1591f0 con 0x7f9c6c15b380 2024-04-24T23:10:38.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.207+0000 7f9c70e06640 1 --2- 172.21.15.32:0/2022647621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c6c15b380 0x7f9c6c1445a0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c00e730 tx=0x7f9c5c00ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:38.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.207+0000 7f9c61ffb640 1 -- 172.21.15.32:0/2022647621 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c00f040 con 0x7f9c6c15b380 2024-04-24T23:10:38.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.207+0000 7f9c61ffb640 1 -- 172.21.15.32:0/2022647621 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c5c004230 con 0x7f9c6c15b380 2024-04-24T23:10:38.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.207+0000 7f9c7386b640 1 -- 172.21.15.32:0/2022647621 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c6c159530 con 0x7f9c6c15b380 2024-04-24T23:10:38.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.207+0000 7f9c61ffb640 1 -- 172.21.15.32:0/2022647621 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c00ee40 con 0x7f9c6c15b380 2024-04-24T23:10:38.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.208+0000 7f9c7386b640 1 -- 172.21.15.32:0/2022647621 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c6c159d70 con 0x7f9c6c15b380 2024-04-24T23:10:38.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.209+0000 7f9c7386b640 1 -- 172.21.15.32:0/2022647621 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c6c15a2f0 con 0x7f9c6c15b380 2024-04-24T23:10:38.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.209+0000 7f9c61ffb640 1 -- 172.21.15.32:0/2022647621 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9c5c00c6d0 con 0x7f9c6c15b380 2024-04-24T23:10:38.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.209+0000 7f9c61ffb640 1 --2- 172.21.15.32:0/2022647621 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9c5006be30 0x7f9c5006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:38.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.210+0000 7f9c61ffb640 1 -- 172.21.15.32:0/2022647621 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(185..185 src has 1..185) ==== 5709+0+0 (secure 0 0 0) 0x7f9c5c08a980 con 0x7f9c6c15b380 2024-04-24T23:10:38.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.210+0000 7f9c71607640 1 --2- 172.21.15.32:0/2022647621 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9c5006be30 0x7f9c5006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:38.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.210+0000 7f9c71607640 1 --2- 172.21.15.32:0/2022647621 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9c5006be30 0x7f9c5006e2f0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f9c68006fd0 tx=0x7f9c68008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:38.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.214+0000 7f9c61ffb640 1 -- 172.21.15.32:0/2022647621 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9c6c159530 con 0x7f9c6c15b380 2024-04-24T23:10:38.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:38.434+0000 7f9c7386b640 1 -- 172.21.15.32:0/2022647621 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f9c6c068430 con 0x7f9c6c15b380 2024-04-24T23:10:39.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.010+0000 7f9c61ffb640 1 -- 172.21.15.32:0/2022647621 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v80) ==== 122+0+0 (secure 0 0 0) 0x7f9c5c017020 con 0x7f9c6c15b380 2024-04-24T23:10:39.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.013+0000 7f9c7386b640 1 -- 172.21.15.32:0/2022647621 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9c5006be30 msgr2=0x7f9c5006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:39.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.013+0000 7f9c7386b640 1 --2- 172.21.15.32:0/2022647621 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9c5006be30 0x7f9c5006e2f0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f9c68006fd0 tx=0x7f9c68008040 comp rx=0 tx=0).stop 2024-04-24T23:10:39.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.013+0000 7f9c7386b640 1 -- 172.21.15.32:0/2022647621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c6c15b380 msgr2=0x7f9c6c1445a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:39.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.013+0000 7f9c7386b640 1 --2- 172.21.15.32:0/2022647621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c6c15b380 0x7f9c6c1445a0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c00e730 tx=0x7f9c5c00ebf0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.013+0000 7f9c7386b640 1 -- 172.21.15.32:0/2022647621 shutdown_connections 2024-04-24T23:10:39.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.013+0000 7f9c7386b640 1 --2- 172.21.15.32:0/2022647621 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9c5006be30 0x7f9c5006e2f0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.013+0000 7f9c7386b640 1 --2- 172.21.15.32:0/2022647621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c6c15b380 0x7f9c6c1445a0 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.013+0000 7f9c7386b640 1 --2- 172.21.15.32:0/2022647621 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c6c15cb00 0x7f9c6c1588a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.013+0000 7f9c7386b640 1 --2- 172.21.15.32:0/2022647621 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c6c0ce430 0x7f9c6c147b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.013+0000 7f9c7386b640 1 -- 172.21.15.32:0/2022647621 >> 172.21.15.32:0/2022647621 conn(0x7f9c6c06b110 msgr2=0x7f9c6c15d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:39.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.013+0000 7f9c7386b640 1 -- 172.21.15.32:0/2022647621 shutdown_connections 2024-04-24T23:10:39.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.013+0000 7f9c7386b640 1 -- 172.21.15.32:0/2022647621 wait complete. 2024-04-24T23:10:39.028 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-24T23:10:39.028 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:10:39.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.130+0000 7f855d4cd640 1 Processor -- start 2024-04-24T23:10:39.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.130+0000 7f855d4cd640 1 -- start start 2024-04-24T23:10:39.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.131+0000 7f855d4cd640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8558158dd0 0x7f855806b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:39.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.131+0000 7f855d4cd640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f855806b860 0x7f855806d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:39.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.131+0000 7f855d4cd640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85580b6d10 con 0x7f855806da90 2024-04-24T23:10:39.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.131+0000 7f855d4cd640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f855815ef10 con 0x7f855806b860 2024-04-24T23:10:39.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.131+0000 7f855d4cd640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f855815f1f0 con 0x7f8558158dd0 2024-04-24T23:10:39.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.131+0000 7f854ffff640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f855806b860 0x7f855806d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:39.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.131+0000 7f8556ffd640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8558158dd0 0x7f855806b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:39.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.131+0000 7f854ffff640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f855806b860 0x7f855806d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.32:48872/0 (socket says 172.21.15.32:48872) 2024-04-24T23:10:39.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.131+0000 7f8556ffd640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8558158dd0 0x7f855806b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.32:51616/0 (socket says 172.21.15.32:51616) 2024-04-24T23:10:39.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.131+0000 7f85577fe640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f855806da90 0x7f8558072580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:40140/0 (socket says 172.21.15.32:40140) 2024-04-24T23:10:39.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.131+0000 7f854ffff640 1 -- 172.21.15.32:0/579616155 learned_addr learned my addr 172.21.15.32:0/579616155 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:39.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.132+0000 7f8556ffd640 1 -- 172.21.15.32:0/579616155 >> v2:172.21.15.121:3300/0 conn(0x7f855806b860 msgr2=0x7f855806d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:39.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.132+0000 7f8556ffd640 1 --2- 172.21.15.32:0/579616155 >> v2:172.21.15.121:3300/0 conn(0x7f855806b860 0x7f855806d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.132+0000 7f8556ffd640 1 -- 172.21.15.32:0/579616155 >> v1:172.21.15.121:6789/0 conn(0x7f855806da90 legacy=0x7f8558072580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:39.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.132+0000 7f8556ffd640 1 -- 172.21.15.32:0/579616155 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f855815f4d0 con 0x7f8558158dd0 2024-04-24T23:10:39.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.132+0000 7f854ffff640 1 --2- 172.21.15.32:0/579616155 >> v2:172.21.15.121:3300/0 conn(0x7f855806b860 0x7f855806d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:10:39.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.132+0000 7f8556ffd640 1 --2- 172.21.15.32:0/579616155 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8558158dd0 0x7f855806b320 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f85480097a0 tx=0x7f854802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85bb9bb0a8e8a73d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:39.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.133+0000 7f85567fc640 1 -- 172.21.15.32:0/579616155 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8548037070 con 0x7f8558158dd0 2024-04-24T23:10:39.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.133+0000 7f85567fc640 1 -- 172.21.15.32:0/579616155 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8548032440 con 0x7f8558158dd0 2024-04-24T23:10:39.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.133+0000 7f85567fc640 1 -- 172.21.15.32:0/579616155 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8548032810 con 0x7f8558158dd0 2024-04-24T23:10:39.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.133+0000 7f855d4cd640 1 -- 172.21.15.32:0/579616155 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8558158dd0 msgr2=0x7f855806b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:39.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.133+0000 7f855d4cd640 1 --2- 172.21.15.32:0/579616155 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8558158dd0 0x7f855806b320 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f85480097a0 tx=0x7f854802a180 comp rx=0 tx=0).stop 2024-04-24T23:10:39.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.133+0000 7f855d4cd640 1 -- 172.21.15.32:0/579616155 shutdown_connections 2024-04-24T23:10:39.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.133+0000 7f855d4cd640 1 --2- 172.21.15.32:0/579616155 >> v2:172.21.15.121:3300/0 conn(0x7f855806b860 0x7f855806d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.133+0000 7f855d4cd640 1 --2- 172.21.15.32:0/579616155 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8558158dd0 0x7f855806b320 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.133+0000 7f855d4cd640 1 -- 172.21.15.32:0/579616155 >> 172.21.15.32:0/579616155 conn(0x7f855806e800 msgr2=0x7f855806ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:39.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.133+0000 7f855d4cd640 1 -- 172.21.15.32:0/579616155 shutdown_connections 2024-04-24T23:10:39.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.133+0000 7f855d4cd640 1 -- 172.21.15.32:0/579616155 wait complete. 2024-04-24T23:10:39.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.134+0000 7f855d4cd640 1 Processor -- start 2024-04-24T23:10:39.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.134+0000 7f855d4cd640 1 -- start start 2024-04-24T23:10:39.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.135+0000 7f855d4cd640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f855806b860 0x7f8558155b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:39.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.135+0000 7f855d4cd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f855806da90 0x7f855814cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:39.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.135+0000 7f855d4cd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8558158dd0 0x7f855815c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:39.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.135+0000 7f855d4cd640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f855815fcd0 con 0x7f8558158dd0 2024-04-24T23:10:39.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.135+0000 7f855d4cd640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f855806bce0 con 0x7f855806da90 2024-04-24T23:10:39.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.135+0000 7f855d4cd640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f855806ded0 con 0x7f855806b860 2024-04-24T23:10:39.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.135+0000 7f85577fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8558158dd0 0x7f855815c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:39.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.135+0000 7f85577fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8558158dd0 0x7f855815c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46922/0 (socket says 172.21.15.32:46922) 2024-04-24T23:10:39.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.135+0000 7f85577fe640 1 -- 172.21.15.32:0/508014701 learned_addr learned my addr 172.21.15.32:0/508014701 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:39.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.135+0000 7f854ffff640 1 --2- 172.21.15.32:0/508014701 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f855806da90 0x7f855814cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:39.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.135+0000 7f8556ffd640 1 --2- 172.21.15.32:0/508014701 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f855806b860 0x7f8558155b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:39.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.136+0000 7f85577fe640 1 -- 172.21.15.32:0/508014701 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f855806b860 msgr2=0x7f8558155b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:39.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.136+0000 7f85577fe640 1 --2- 172.21.15.32:0/508014701 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f855806b860 0x7f8558155b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.136+0000 7f85577fe640 1 -- 172.21.15.32:0/508014701 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f855806da90 msgr2=0x7f855814cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:39.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.136+0000 7f85577fe640 1 --2- 172.21.15.32:0/508014701 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f855806da90 0x7f855814cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.136+0000 7f85577fe640 1 -- 172.21.15.32:0/508014701 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f855815b730 con 0x7f8558158dd0 2024-04-24T23:10:39.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.136+0000 7f85577fe640 1 --2- 172.21.15.32:0/508014701 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8558158dd0 0x7f855815c7d0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f855000c780 tx=0x7f855000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:39.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.136+0000 7f854f7fe640 1 -- 172.21.15.32:0/508014701 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8550017070 con 0x7f8558158dd0 2024-04-24T23:10:39.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.136+0000 7f854f7fe640 1 -- 172.21.15.32:0/508014701 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f855000a640 con 0x7f8558158dd0 2024-04-24T23:10:39.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.136+0000 7f854f7fe640 1 -- 172.21.15.32:0/508014701 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855000aa30 con 0x7f8558158dd0 2024-04-24T23:10:39.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.136+0000 7f855d4cd640 1 -- 172.21.15.32:0/508014701 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f855815ba10 con 0x7f8558158dd0 2024-04-24T23:10:39.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.136+0000 7f855d4cd640 1 -- 172.21.15.32:0/508014701 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f855815bfd0 con 0x7f8558158dd0 2024-04-24T23:10:39.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.137+0000 7f855d4cd640 1 -- 172.21.15.32:0/508014701 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8558068430 con 0x7f8558158dd0 2024-04-24T23:10:39.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.137+0000 7f854f7fe640 1 -- 172.21.15.32:0/508014701 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f855001a070 con 0x7f8558158dd0 2024-04-24T23:10:39.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.138+0000 7f854f7fe640 1 --2- 172.21.15.32:0/508014701 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f853806be60 0x7f853806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:39.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.138+0000 7f854f7fe640 1 -- 172.21.15.32:0/508014701 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(185..185 src has 1..185) ==== 5709+0+0 (secure 0 0 0) 0x7f855008a3e0 con 0x7f8558158dd0 2024-04-24T23:10:39.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.138+0000 7f8556ffd640 1 --2- 172.21.15.32:0/508014701 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f853806be60 0x7f853806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:39.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.138+0000 7f8556ffd640 1 --2- 172.21.15.32:0/508014701 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f853806be60 0x7f853806e320 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f8548034000 tx=0x7f854802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:39.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.142+0000 7f854f7fe640 1 -- 172.21.15.32:0/508014701 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8550058730 con 0x7f8558158dd0 2024-04-24T23:10:39.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.361+0000 7f855d4cd640 1 -- 172.21.15.32:0/508014701 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8558068a60 con 0x7f8558158dd0 2024-04-24T23:10:39.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.361+0000 7f854f7fe640 1 -- 172.21.15.32:0/508014701 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) ==== 94+0+5185 (secure 0 0 0) 0x7f855005ca30 con 0x7f8558158dd0 2024-04-24T23:10:39.364 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:10:39.364 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":80,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6941,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/416518156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":416518156},{"type":"v1","addr":"172.21.15.121:6837","nonce":416518156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7762,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1506667233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1506667233},{"type":"v1","addr":"172.21.15.121:6835","nonce":1506667233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":9063,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/2215079101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":2215079101},{"type":"v1","addr":"172.21.15.32:6834","nonce":2215079101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:10:36.353761+0000","modified":"2024-04-24T23:10:38.436407+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9066},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9066":{"gid":9066,"name":"a","rank":0,"incarnation":78,"state":"up:active","state_seq":4,"addr":"172.21.15.32:6835/901353412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":901353412},{"type":"v1","addr":"172.21.15.32:6835","nonce":901353412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9066,"qdb_cluster":[9066]},"id":11}]} 2024-04-24T23:10:39.364 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 80 2024-04-24T23:10:39.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.363+0000 7f855d4cd640 1 -- 172.21.15.32:0/508014701 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f853806be60 msgr2=0x7f853806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:39.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.363+0000 7f855d4cd640 1 --2- 172.21.15.32:0/508014701 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f853806be60 0x7f853806e320 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f8548034000 tx=0x7f854802e000 comp rx=0 tx=0).stop 2024-04-24T23:10:39.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.363+0000 7f855d4cd640 1 -- 172.21.15.32:0/508014701 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8558158dd0 msgr2=0x7f855815c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:39.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.363+0000 7f855d4cd640 1 --2- 172.21.15.32:0/508014701 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8558158dd0 0x7f855815c7d0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f855000c780 tx=0x7f855000cc40 comp rx=0 tx=0).stop 2024-04-24T23:10:39.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.363+0000 7f855d4cd640 1 -- 172.21.15.32:0/508014701 shutdown_connections 2024-04-24T23:10:39.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.363+0000 7f855d4cd640 1 --2- 172.21.15.32:0/508014701 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f853806be60 0x7f853806e320 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.363+0000 7f855d4cd640 1 --2- 172.21.15.32:0/508014701 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8558158dd0 0x7f855815c7d0 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.363+0000 7f855d4cd640 1 --2- 172.21.15.32:0/508014701 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f855806da90 0x7f855814cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.363+0000 7f855d4cd640 1 --2- 172.21.15.32:0/508014701 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f855806b860 0x7f8558155b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.363+0000 7f855d4cd640 1 -- 172.21.15.32:0/508014701 >> 172.21.15.32:0/508014701 conn(0x7f855806e800 msgr2=0x7f8558154b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:39.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.363+0000 7f855d4cd640 1 -- 172.21.15.32:0/508014701 shutdown_connections 2024-04-24T23:10:39.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.363+0000 7f855d4cd640 1 -- 172.21.15.32:0/508014701 wait complete. 2024-04-24T23:10:39.377 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:10:39.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.475+0000 7f9e187e7640 1 Processor -- start 2024-04-24T23:10:39.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.475+0000 7f9e187e7640 1 -- start start 2024-04-24T23:10:39.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.476+0000 7f9e187e7640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9e1006b820 0x7f9e100d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:39.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.476+0000 7f9e187e7640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e100d4210 0x7f9e100b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:39.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.476+0000 7f9e187e7640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e100d3170 con 0x7f9e1015e9b0 2024-04-24T23:10:39.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.476+0000 7f9e187e7640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f9e100d3450 con 0x7f9e1006b820 2024-04-24T23:10:39.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.476+0000 7f9e187e7640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f9e100d3730 con 0x7f9e100d4210 2024-04-24T23:10:39.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.477+0000 7f9e16583640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9e1015e9b0 0x7f9e1006b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37942/0 (socket says 172.21.15.32:37942) 2024-04-24T23:10:39.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.477+0000 7f9e15d82640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9e1006b820 0x7f9e100d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:39.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.477+0000 7f9e16583640 1 -- 172.21.15.32:0/1766422095 learned_addr learned my addr 172.21.15.32:0/1766422095 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:10:39.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.477+0000 7f9e16d84640 1 --2- 172.21.15.32:0/1766422095 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e100d4210 0x7f9e100b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:39.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.477+0000 7f9e15d82640 1 -- 172.21.15.32:0/1766422095 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e100d4210 msgr2=0x7f9e100b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:39.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.477+0000 7f9e15d82640 1 --2- 172.21.15.32:0/1766422095 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e100d4210 0x7f9e100b78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.477+0000 7f9e15581640 1 -- 172.21.15.32:0/1766422095 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2156260483 0 0) 0x7f9e100d3170 con 0x7f9e1015e9b0 2024-04-24T23:10:39.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.477+0000 7f9e15d82640 1 -- 172.21.15.32:0/1766422095 >> v1:172.21.15.32:6789/0 conn(0x7f9e1015e9b0 legacy=0x7f9e1006b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:39.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.477+0000 7f9e15d82640 1 -- 172.21.15.32:0/1766422095 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e100d3a10 con 0x7f9e1006b820 2024-04-24T23:10:39.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.478+0000 7f9e15d82640 1 --2- 172.21.15.32:0/1766422095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e1006b820 0x7f9e100d2b90 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f9e000097a0 tx=0x7f9e0002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0b3098131a2505 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:39.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.478+0000 7f9e15581640 1 -- 172.21.15.32:0/1766422095 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e00037070 con 0x7f9e1006b820 2024-04-24T23:10:39.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.478+0000 7f9e15581640 1 -- 172.21.15.32:0/1766422095 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e00032440 con 0x7f9e1006b820 2024-04-24T23:10:39.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.478+0000 7f9e15581640 1 -- 172.21.15.32:0/1766422095 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e00032810 con 0x7f9e1006b820 2024-04-24T23:10:39.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.478+0000 7f9e187e7640 1 -- 172.21.15.32:0/1766422095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e1006b820 msgr2=0x7f9e100d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:39.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.478+0000 7f9e187e7640 1 --2- 172.21.15.32:0/1766422095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e1006b820 0x7f9e100d2b90 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f9e000097a0 tx=0x7f9e0002a180 comp rx=0 tx=0).stop 2024-04-24T23:10:39.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.478+0000 7f9e187e7640 1 -- 172.21.15.32:0/1766422095 shutdown_connections 2024-04-24T23:10:39.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.478+0000 7f9e187e7640 1 --2- 172.21.15.32:0/1766422095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e1006b820 0x7f9e100d2b90 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.478+0000 7f9e187e7640 1 --2- 172.21.15.32:0/1766422095 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e100d4210 0x7f9e100b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.478+0000 7f9e187e7640 1 -- 172.21.15.32:0/1766422095 >> 172.21.15.32:0/1766422095 conn(0x7f9e100d11d0 msgr2=0x7f9e100d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:39.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.478+0000 7f9e187e7640 1 -- 172.21.15.32:0/1766422095 shutdown_connections 2024-04-24T23:10:39.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.478+0000 7f9e187e7640 1 -- 172.21.15.32:0/1766422095 wait complete. 2024-04-24T23:10:39.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.480+0000 7f9e187e7640 1 Processor -- start 2024-04-24T23:10:39.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.480+0000 7f9e187e7640 1 -- start start 2024-04-24T23:10:39.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.480+0000 7f9e187e7640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e1006b820 0x7f9e100ce3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:39.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.480+0000 7f9e187e7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e100d4210 0x7f9e100cb820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:39.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.480+0000 7f9e187e7640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e1015e9b0 0x7f9e1015b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:39.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.480+0000 7f9e187e7640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9e100ba0e0 con 0x7f9e100d4210 2024-04-24T23:10:39.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.480+0000 7f9e187e7640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9e100d2400 con 0x7f9e1006b820 2024-04-24T23:10:39.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.480+0000 7f9e187e7640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f9e100ba260 con 0x7f9e1015e9b0 2024-04-24T23:10:39.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.480+0000 7f9e15d82640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e100d4210 0x7f9e100cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:39.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.480+0000 7f9e15d82640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e100d4210 0x7f9e100cb820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46934/0 (socket says 172.21.15.32:46934) 2024-04-24T23:10:39.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.480+0000 7f9e15d82640 1 -- 172.21.15.32:0/707153394 learned_addr learned my addr 172.21.15.32:0/707153394 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:39.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.481+0000 7f9e16d84640 1 --2- 172.21.15.32:0/707153394 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e1015e9b0 0x7f9e1015b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:39.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.481+0000 7f9e16583640 1 --2- 172.21.15.32:0/707153394 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e1006b820 0x7f9e100ce3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:39.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.481+0000 7f9e15d82640 1 -- 172.21.15.32:0/707153394 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e1015e9b0 msgr2=0x7f9e1015b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:39.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.481+0000 7f9e15d82640 1 --2- 172.21.15.32:0/707153394 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e1015e9b0 0x7f9e1015b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.481+0000 7f9e15d82640 1 -- 172.21.15.32:0/707153394 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e1006b820 msgr2=0x7f9e100ce3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:39.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.481+0000 7f9e15d82640 1 --2- 172.21.15.32:0/707153394 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e1006b820 0x7f9e100ce3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.481+0000 7f9e15d82640 1 -- 172.21.15.32:0/707153394 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e100ce920 con 0x7f9e100d4210 2024-04-24T23:10:39.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.481+0000 7f9e15d82640 1 --2- 172.21.15.32:0/707153394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e100d4210 0x7f9e100cb820 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f9e00035040 tx=0x7f9e000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:39.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.481+0000 7f9e06ffd640 1 -- 172.21.15.32:0/707153394 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e00037040 con 0x7f9e100d4210 2024-04-24T23:10:39.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.481+0000 7f9e06ffd640 1 -- 172.21.15.32:0/707153394 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e00030770 con 0x7f9e100d4210 2024-04-24T23:10:39.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.481+0000 7f9e06ffd640 1 -- 172.21.15.32:0/707153394 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e00030b80 con 0x7f9e100d4210 2024-04-24T23:10:39.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.481+0000 7f9e187e7640 1 -- 172.21.15.32:0/707153394 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e100cc270 con 0x7f9e100d4210 2024-04-24T23:10:39.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.481+0000 7f9e187e7640 1 -- 172.21.15.32:0/707153394 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e100cc830 con 0x7f9e100d4210 2024-04-24T23:10:39.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.482+0000 7f9e187e7640 1 -- 172.21.15.32:0/707153394 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e10068430 con 0x7f9e100d4210 2024-04-24T23:10:39.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.483+0000 7f9e06ffd640 1 -- 172.21.15.32:0/707153394 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9e00044020 con 0x7f9e100d4210 2024-04-24T23:10:39.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.483+0000 7f9e06ffd640 1 --2- 172.21.15.32:0/707153394 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9df406be30 0x7f9df406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:39.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.483+0000 7f9e06ffd640 1 -- 172.21.15.32:0/707153394 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(185..185 src has 1..185) ==== 5709+0+0 (secure 0 0 0) 0x7f9e000ab400 con 0x7f9e100d4210 2024-04-24T23:10:39.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.483+0000 7f9e16583640 1 --2- 172.21.15.32:0/707153394 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9df406be30 0x7f9df406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:39.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.484+0000 7f9e16583640 1 --2- 172.21.15.32:0/707153394 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9df406be30 0x7f9df406e2f0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f9e08000f60 tx=0x7f9e08007c00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:39.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.488+0000 7f9e06ffd640 1 -- 172.21.15.32:0/707153394 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9e00032e20 con 0x7f9e100d4210 2024-04-24T23:10:39.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.672+0000 7f9e187e7640 1 -- 172.21.15.32:0/707153394 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9e100d16b0 con 0x7f9e100d4210 2024-04-24T23:10:39.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.673+0000 7f9e06ffd640 1 -- 172.21.15.32:0/707153394 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v185) ==== 74+0+16692 (secure 0 0 0) 0x7f9e00079e40 con 0x7f9e100d4210 2024-04-24T23:10:39.676 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:10:39.676 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":185,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:10:37.993580+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:10:34.649562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-24T23:10:35.339604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"185","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:10:39.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.675+0000 7f9e187e7640 1 -- 172.21.15.32:0/707153394 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9df406be30 msgr2=0x7f9df406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:39.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.675+0000 7f9e187e7640 1 --2- 172.21.15.32:0/707153394 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9df406be30 0x7f9df406e2f0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f9e08000f60 tx=0x7f9e08007c00 comp rx=0 tx=0).stop 2024-04-24T23:10:39.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.675+0000 7f9e187e7640 1 -- 172.21.15.32:0/707153394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e100d4210 msgr2=0x7f9e100cb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:39.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.675+0000 7f9e187e7640 1 --2- 172.21.15.32:0/707153394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e100d4210 0x7f9e100cb820 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f9e00035040 tx=0x7f9e000046c0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.675+0000 7f9e187e7640 1 -- 172.21.15.32:0/707153394 shutdown_connections 2024-04-24T23:10:39.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.675+0000 7f9e187e7640 1 --2- 172.21.15.32:0/707153394 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9df406be30 0x7f9df406e2f0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.675+0000 7f9e187e7640 1 --2- 172.21.15.32:0/707153394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e100d4210 0x7f9e100cb820 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.675+0000 7f9e187e7640 1 --2- 172.21.15.32:0/707153394 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e1015e9b0 0x7f9e1015b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.675+0000 7f9e187e7640 1 --2- 172.21.15.32:0/707153394 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e1006b820 0x7f9e100ce3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.675+0000 7f9e187e7640 1 -- 172.21.15.32:0/707153394 >> 172.21.15.32:0/707153394 conn(0x7f9e100d11d0 msgr2=0x7f9e100cefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:39.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.675+0000 7f9e187e7640 1 -- 172.21.15.32:0/707153394 shutdown_connections 2024-04-24T23:10:39.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.675+0000 7f9e187e7640 1 -- 172.21.15.32:0/707153394 wait complete. 2024-04-24T23:10:39.689 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-24T23:10:39.689 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:10:39.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.786+0000 7f713eada640 1 Processor -- start 2024-04-24T23:10:39.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.786+0000 7f713eada640 1 -- start start 2024-04-24T23:10:39.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.787+0000 7f713eada640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f713815e9b0 0x7f71380d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:39.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.787+0000 7f713eada640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f713806b820 0x7f71380b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:39.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.787+0000 7f713eada640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71380d3170 con 0x7f71380d4210 2024-04-24T23:10:39.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.787+0000 7f713eada640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f71380d3450 con 0x7f713806b820 2024-04-24T23:10:39.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.787+0000 7f713eada640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f71380d3730 con 0x7f713815e9b0 2024-04-24T23:10:39.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.788+0000 7f713c876640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f713815e9b0 0x7f71380d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:39.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.788+0000 7f713d077640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f713806b820 0x7f71380b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:39.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.788+0000 7f713d077640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f713806b820 0x7f71380b78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.32:55944/0 (socket says 172.21.15.32:55944) 2024-04-24T23:10:39.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.788+0000 7f713c876640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f713815e9b0 0x7f71380d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.32:38554/0 (socket says 172.21.15.32:38554) 2024-04-24T23:10:39.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.788+0000 7f713d077640 1 -- 172.21.15.32:0/1938617786 learned_addr learned my addr 172.21.15.32:0/1938617786 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:39.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.788+0000 7f7137fff640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f71380d4210 0x7f713806b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:55248/0 (socket says 172.21.15.32:55248) 2024-04-24T23:10:39.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.788+0000 7f713d077640 1 -- 172.21.15.32:0/1938617786 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f713815e9b0 msgr2=0x7f71380d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:39.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.788+0000 7f713d077640 1 --2- 172.21.15.32:0/1938617786 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f713815e9b0 0x7f71380d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.788+0000 7f713d077640 1 -- 172.21.15.32:0/1938617786 >> v1:172.21.15.121:6789/0 conn(0x7f71380d4210 legacy=0x7f713806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:39.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.788+0000 7f713d077640 1 -- 172.21.15.32:0/1938617786 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71380d3a10 con 0x7f713806b820 2024-04-24T23:10:39.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.788+0000 7f713c876640 1 --2- 172.21.15.32:0/1938617786 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f713815e9b0 0x7f71380d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:10:39.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.789+0000 7f713d077640 1 --2- 172.21.15.32:0/1938617786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f713806b820 0x7f71380b78f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f7130009e50 tx=0x7f713002a730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26278b5543bb7001 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:39.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.789+0000 7f71377fe640 1 -- 172.21.15.32:0/1938617786 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7130036070 con 0x7f713806b820 2024-04-24T23:10:39.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.789+0000 7f71377fe640 1 -- 172.21.15.32:0/1938617786 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7130031600 con 0x7f713806b820 2024-04-24T23:10:39.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.789+0000 7f71377fe640 1 -- 172.21.15.32:0/1938617786 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71300319d0 con 0x7f713806b820 2024-04-24T23:10:39.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.789+0000 7f713eada640 1 -- 172.21.15.32:0/1938617786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f713806b820 msgr2=0x7f71380b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:39.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.789+0000 7f713eada640 1 --2- 172.21.15.32:0/1938617786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f713806b820 0x7f71380b78f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f7130009e50 tx=0x7f713002a730 comp rx=0 tx=0).stop 2024-04-24T23:10:39.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.789+0000 7f713eada640 1 -- 172.21.15.32:0/1938617786 shutdown_connections 2024-04-24T23:10:39.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.789+0000 7f713eada640 1 --2- 172.21.15.32:0/1938617786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f713806b820 0x7f71380b78f0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.789+0000 7f713eada640 1 --2- 172.21.15.32:0/1938617786 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f713815e9b0 0x7f71380d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.789+0000 7f713eada640 1 -- 172.21.15.32:0/1938617786 >> 172.21.15.32:0/1938617786 conn(0x7f71380d11d0 msgr2=0x7f71380d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:39.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.789+0000 7f713eada640 1 -- 172.21.15.32:0/1938617786 shutdown_connections 2024-04-24T23:10:39.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.789+0000 7f713eada640 1 -- 172.21.15.32:0/1938617786 wait complete. 2024-04-24T23:10:39.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.791+0000 7f713eada640 1 Processor -- start 2024-04-24T23:10:39.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.791+0000 7f713eada640 1 -- start start 2024-04-24T23:10:39.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.791+0000 7f713eada640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f713806b820 0x7f7138153d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:39.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.792+0000 7f713eada640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71380d4210 0x7f71380ce240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:39.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.792+0000 7f713eada640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f713815e9b0 0x7f71380ce780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:39.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.792+0000 7f713eada640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f71380ba0e0 con 0x7f713806b820 2024-04-24T23:10:39.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.792+0000 7f713eada640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f71380d2400 con 0x7f71380d4210 2024-04-24T23:10:39.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.792+0000 7f713eada640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f71380ba260 con 0x7f713815e9b0 2024-04-24T23:10:39.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.792+0000 7f713c876640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f713806b820 0x7f7138153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:39.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.792+0000 7f713c876640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f713806b820 0x7f7138153d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48700/0 (socket says 172.21.15.32:48700) 2024-04-24T23:10:39.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.792+0000 7f713c876640 1 -- 172.21.15.32:0/627798234 learned_addr learned my addr 172.21.15.32:0/627798234 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:39.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.792+0000 7f7137fff640 1 --2- 172.21.15.32:0/627798234 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71380d4210 0x7f71380ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:39.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.792+0000 7f713d077640 1 --2- 172.21.15.32:0/627798234 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f713815e9b0 0x7f71380ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:39.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.792+0000 7f713c876640 1 -- 172.21.15.32:0/627798234 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f713815e9b0 msgr2=0x7f71380ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:39.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.792+0000 7f713c876640 1 --2- 172.21.15.32:0/627798234 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f713815e9b0 0x7f71380ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.792+0000 7f713c876640 1 -- 172.21.15.32:0/627798234 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71380d4210 msgr2=0x7f71380ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:39.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.792+0000 7f713c876640 1 --2- 172.21.15.32:0/627798234 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71380d4210 0x7f71380ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.792+0000 7f713c876640 1 -- 172.21.15.32:0/627798234 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f713815eec0 con 0x7f713806b820 2024-04-24T23:10:39.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.792+0000 7f7137fff640 1 --2- 172.21.15.32:0/627798234 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71380d4210 0x7f71380ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:10:39.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.792+0000 7f713d077640 1 --2- 172.21.15.32:0/627798234 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f713815e9b0 0x7f71380ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:10:39.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.792+0000 7f713c876640 1 --2- 172.21.15.32:0/627798234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f713806b820 0x7f7138153d70 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f712c00e960 tx=0x7f712c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:39.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.792+0000 7f71357fa640 1 -- 172.21.15.32:0/627798234 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712c017070 con 0x7f713806b820 2024-04-24T23:10:39.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.792+0000 7f71357fa640 1 -- 172.21.15.32:0/627798234 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f712c00ca00 con 0x7f713806b820 2024-04-24T23:10:39.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.793+0000 7f713eada640 1 -- 172.21.15.32:0/627798234 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f713815f1a0 con 0x7f713806b820 2024-04-24T23:10:39.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.793+0000 7f71357fa640 1 -- 172.21.15.32:0/627798234 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712c00cdd0 con 0x7f713806b820 2024-04-24T23:10:39.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.793+0000 7f713eada640 1 -- 172.21.15.32:0/627798234 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f713815f760 con 0x7f713806b820 2024-04-24T23:10:39.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.793+0000 7f713eada640 1 -- 172.21.15.32:0/627798234 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7138068430 con 0x7f713806b820 2024-04-24T23:10:39.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.794+0000 7f71357fa640 1 -- 172.21.15.32:0/627798234 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f712c01a070 con 0x7f713806b820 2024-04-24T23:10:39.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.794+0000 7f71357fa640 1 --2- 172.21.15.32:0/627798234 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f711c06be30 0x7f711c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:39.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.795+0000 7f71357fa640 1 -- 172.21.15.32:0/627798234 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(185..185 src has 1..185) ==== 5709+0+0 (secure 0 0 0) 0x7f712c08af90 con 0x7f713806b820 2024-04-24T23:10:39.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.795+0000 7f7137fff640 1 --2- 172.21.15.32:0/627798234 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f711c06be30 0x7f711c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:39.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.795+0000 7f7137fff640 1 --2- 172.21.15.32:0/627798234 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f711c06be30 0x7f711c06e2f0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f71381f7d40 tx=0x7f7124007e80 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:39.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.799+0000 7f71357fa640 1 -- 172.21.15.32:0/627798234 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f712c059380 con 0x7f713806b820 2024-04-24T23:10:39.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.977+0000 7f713eada640 1 -- 172.21.15.32:0/627798234 --> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f71380d2680 con 0x7f711c06be30 2024-04-24T23:10:39.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.984+0000 7f71357fa640 1 -- 172.21.15.32:0/627798234 <== mgr.4108 v2:172.21.15.121:6832/1848157943 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247925 (secure 0 0 0) 0x7f71380d2680 con 0x7f711c06be30 2024-04-24T23:10:39.989 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:10:39.991 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-24T23:10:39.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.987+0000 7f713eada640 1 -- 172.21.15.32:0/627798234 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f711c06be30 msgr2=0x7f711c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:39.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.987+0000 7f713eada640 1 --2- 172.21.15.32:0/627798234 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f711c06be30 0x7f711c06e2f0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f71381f7d40 tx=0x7f7124007e80 comp rx=0 tx=0).stop 2024-04-24T23:10:39.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.987+0000 7f713eada640 1 -- 172.21.15.32:0/627798234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f713806b820 msgr2=0x7f7138153d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:39.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.987+0000 7f713eada640 1 --2- 172.21.15.32:0/627798234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f713806b820 0x7f7138153d70 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f712c00e960 tx=0x7f712c00ee20 comp rx=0 tx=0).stop 2024-04-24T23:10:39.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.987+0000 7f713eada640 1 -- 172.21.15.32:0/627798234 shutdown_connections 2024-04-24T23:10:39.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.987+0000 7f713eada640 1 --2- 172.21.15.32:0/627798234 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f711c06be30 0x7f711c06e2f0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.987+0000 7f713eada640 1 --2- 172.21.15.32:0/627798234 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f713815e9b0 0x7f71380ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.987+0000 7f713eada640 1 --2- 172.21.15.32:0/627798234 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71380d4210 0x7f71380ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.987+0000 7f713eada640 1 --2- 172.21.15.32:0/627798234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f713806b820 0x7f7138153d70 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.987+0000 7f713eada640 1 -- 172.21.15.32:0/627798234 >> 172.21.15.32:0/627798234 conn(0x7f71380d11d0 msgr2=0x7f71380b78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:39.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.987+0000 7f713eada640 1 -- 172.21.15.32:0/627798234 shutdown_connections 2024-04-24T23:10:39.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:39.987+0000 7f713eada640 1 -- 172.21.15.32:0/627798234 wait complete. 2024-04-24T23:10:40.001 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":530,"stamp":"2024-04-24T23:10:39.293865+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":9195,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1553304,"statfs":{"total":1600000000,"available":1590583944,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.313509"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.008754+0000","last_change":"2024-04-24T23:10:37.008622+0000","last_active":"2024-04-24T23:10:37.008754+0000","last_peered":"2024-04-24T23:10:37.008754+0000","last_clean":"2024-04-24T23:10:37.008754+0000","last_became_active":"2024-04-24T23:10:37.008368+0000","last_became_peered":"2024-04-24T23:10:37.008368+0000","last_unstale":"2024-04-24T23:10:37.008754+0000","last_undegraded":"2024-04-24T23:10:37.008754+0000","last_fullsized":"2024-04-24T23:10:37.008754+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:55:05.387326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.008920+0000","last_change":"2024-04-24T23:10:37.008790+0000","last_active":"2024-04-24T23:10:37.008920+0000","last_peered":"2024-04-24T23:10:37.008920+0000","last_clean":"2024-04-24T23:10:37.008920+0000","last_became_active":"2024-04-24T23:10:37.008476+0000","last_became_peered":"2024-04-24T23:10:37.008476+0000","last_unstale":"2024-04-24T23:10:37.008920+0000","last_undegraded":"2024-04-24T23:10:37.008920+0000","last_fullsized":"2024-04-24T23:10:37.008920+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:56:53.705009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.004588+0000","last_change":"2024-04-24T23:10:37.003644+0000","last_active":"2024-04-24T23:10:37.004588+0000","last_peered":"2024-04-24T23:10:37.004588+0000","last_clean":"2024-04-24T23:10:37.004588+0000","last_became_active":"2024-04-24T23:10:37.002896+0000","last_became_peered":"2024-04-24T23:10:37.002896+0000","last_unstale":"2024-04-24T23:10:37.004588+0000","last_undegraded":"2024-04-24T23:10:37.004588+0000","last_fullsized":"2024-04-24T23:10:37.004588+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:48:23.839497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.009975+0000","last_change":"2024-04-24T23:10:37.009792+0000","last_active":"2024-04-24T23:10:37.009975+0000","last_peered":"2024-04-24T23:10:37.009975+0000","last_clean":"2024-04-24T23:10:37.009975+0000","last_became_active":"2024-04-24T23:10:37.009387+0000","last_became_peered":"2024-04-24T23:10:37.009387+0000","last_unstale":"2024-04-24T23:10:37.009975+0000","last_undegraded":"2024-04-24T23:10:37.009975+0000","last_fullsized":"2024-04-24T23:10:37.009975+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:30:28.867316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.009901+0000","last_change":"2024-04-24T23:10:37.009744+0000","last_active":"2024-04-24T23:10:37.009901+0000","last_peered":"2024-04-24T23:10:37.009901+0000","last_clean":"2024-04-24T23:10:37.009901+0000","last_became_active":"2024-04-24T23:10:37.009304+0000","last_became_peered":"2024-04-24T23:10:37.009304+0000","last_unstale":"2024-04-24T23:10:37.009901+0000","last_undegraded":"2024-04-24T23:10:37.009901+0000","last_fullsized":"2024-04-24T23:10:37.009901+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:28:42.260245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.008046+0000","last_change":"2024-04-24T23:10:37.006773+0000","last_active":"2024-04-24T23:10:37.008046+0000","last_peered":"2024-04-24T23:10:37.008046+0000","last_clean":"2024-04-24T23:10:37.008046+0000","last_became_active":"2024-04-24T23:10:37.006525+0000","last_became_peered":"2024-04-24T23:10:37.006525+0000","last_unstale":"2024-04-24T23:10:37.008046+0000","last_undegraded":"2024-04-24T23:10:37.008046+0000","last_fullsized":"2024-04-24T23:10:37.008046+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:41.030023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.010206+0000","last_change":"2024-04-24T23:10:37.009945+0000","last_active":"2024-04-24T23:10:37.010206+0000","last_peered":"2024-04-24T23:10:37.010206+0000","last_clean":"2024-04-24T23:10:37.010206+0000","last_became_active":"2024-04-24T23:10:37.009386+0000","last_became_peered":"2024-04-24T23:10:37.009386+0000","last_unstale":"2024-04-24T23:10:37.010206+0000","last_undegraded":"2024-04-24T23:10:37.010206+0000","last_fullsized":"2024-04-24T23:10:37.010206+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:02.652113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.011722+0000","last_change":"2024-04-24T23:10:37.011533+0000","last_active":"2024-04-24T23:10:37.011722+0000","last_peered":"2024-04-24T23:10:37.011722+0000","last_clean":"2024-04-24T23:10:37.011722+0000","last_became_active":"2024-04-24T23:10:37.011288+0000","last_became_peered":"2024-04-24T23:10:37.011288+0000","last_unstale":"2024-04-24T23:10:37.011722+0000","last_undegraded":"2024-04-24T23:10:37.011722+0000","last_fullsized":"2024-04-24T23:10:37.011722+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:07:19.425606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.016535+0000","last_change":"2024-04-24T23:10:37.015856+0000","last_active":"2024-04-24T23:10:37.016535+0000","last_peered":"2024-04-24T23:10:37.016535+0000","last_clean":"2024-04-24T23:10:37.016535+0000","last_became_active":"2024-04-24T23:10:37.015508+0000","last_became_peered":"2024-04-24T23:10:37.015508+0000","last_unstale":"2024-04-24T23:10:37.016535+0000","last_undegraded":"2024-04-24T23:10:37.016535+0000","last_fullsized":"2024-04-24T23:10:37.016535+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:39:42.709982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.008907+0000","last_change":"2024-04-24T23:10:37.008780+0000","last_active":"2024-04-24T23:10:37.008907+0000","last_peered":"2024-04-24T23:10:37.008907+0000","last_clean":"2024-04-24T23:10:37.008907+0000","last_became_active":"2024-04-24T23:10:37.008514+0000","last_became_peered":"2024-04-24T23:10:37.008514+0000","last_unstale":"2024-04-24T23:10:37.008907+0000","last_undegraded":"2024-04-24T23:10:37.008907+0000","last_fullsized":"2024-04-24T23:10:37.008907+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:12:20.165575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.005233+0000","last_change":"2024-04-24T23:10:37.005104+0000","last_active":"2024-04-24T23:10:37.005233+0000","last_peered":"2024-04-24T23:10:37.005233+0000","last_clean":"2024-04-24T23:10:37.005233+0000","last_became_active":"2024-04-24T23:10:37.004857+0000","last_became_peered":"2024-04-24T23:10:37.004857+0000","last_unstale":"2024-04-24T23:10:37.005233+0000","last_undegraded":"2024-04-24T23:10:37.005233+0000","last_fullsized":"2024-04-24T23:10:37.005233+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:57:57.500385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.010067+0000","last_change":"2024-04-24T23:10:37.009954+0000","last_active":"2024-04-24T23:10:37.010067+0000","last_peered":"2024-04-24T23:10:37.010067+0000","last_clean":"2024-04-24T23:10:37.010067+0000","last_became_active":"2024-04-24T23:10:37.007786+0000","last_became_peered":"2024-04-24T23:10:37.007786+0000","last_unstale":"2024-04-24T23:10:37.010067+0000","last_undegraded":"2024-04-24T23:10:37.010067+0000","last_fullsized":"2024-04-24T23:10:37.010067+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:39.879189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.009807+0000","last_change":"2024-04-24T23:10:37.009622+0000","last_active":"2024-04-24T23:10:37.009807+0000","last_peered":"2024-04-24T23:10:37.009807+0000","last_clean":"2024-04-24T23:10:37.009807+0000","last_became_active":"2024-04-24T23:10:37.007721+0000","last_became_peered":"2024-04-24T23:10:37.007721+0000","last_unstale":"2024-04-24T23:10:37.009807+0000","last_undegraded":"2024-04-24T23:10:37.009807+0000","last_fullsized":"2024-04-24T23:10:37.009807+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:06:48.308222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.010533+0000","last_change":"2024-04-24T23:10:37.010378+0000","last_active":"2024-04-24T23:10:37.010533+0000","last_peered":"2024-04-24T23:10:37.010533+0000","last_clean":"2024-04-24T23:10:37.010533+0000","last_became_active":"2024-04-24T23:10:37.010023+0000","last_became_peered":"2024-04-24T23:10:37.010023+0000","last_unstale":"2024-04-24T23:10:37.010533+0000","last_undegraded":"2024-04-24T23:10:37.010533+0000","last_fullsized":"2024-04-24T23:10:37.010533+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:21:01.576041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.012711+0000","last_change":"2024-04-24T23:10:37.012604+0000","last_active":"2024-04-24T23:10:37.012711+0000","last_peered":"2024-04-24T23:10:37.012711+0000","last_clean":"2024-04-24T23:10:37.012711+0000","last_became_active":"2024-04-24T23:10:37.012372+0000","last_became_peered":"2024-04-24T23:10:37.012372+0000","last_unstale":"2024-04-24T23:10:37.012711+0000","last_undegraded":"2024-04-24T23:10:37.012711+0000","last_fullsized":"2024-04-24T23:10:37.012711+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:28:29.997978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.003221+0000","last_change":"2024-04-24T23:10:37.003103+0000","last_active":"2024-04-24T23:10:37.003221+0000","last_peered":"2024-04-24T23:10:37.003221+0000","last_clean":"2024-04-24T23:10:37.003221+0000","last_became_active":"2024-04-24T23:10:37.002582+0000","last_became_peered":"2024-04-24T23:10:37.002582+0000","last_unstale":"2024-04-24T23:10:37.003221+0000","last_undegraded":"2024-04-24T23:10:37.003221+0000","last_fullsized":"2024-04-24T23:10:37.003221+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:03:25.100801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.013354+0000","last_change":"2024-04-24T23:10:37.012815+0000","last_active":"2024-04-24T23:10:37.013354+0000","last_peered":"2024-04-24T23:10:37.013354+0000","last_clean":"2024-04-24T23:10:37.013354+0000","last_became_active":"2024-04-24T23:10:37.012583+0000","last_became_peered":"2024-04-24T23:10:37.012583+0000","last_unstale":"2024-04-24T23:10:37.013354+0000","last_undegraded":"2024-04-24T23:10:37.013354+0000","last_fullsized":"2024-04-24T23:10:37.013354+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:07:38.090091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.012884+0000","last_change":"2024-04-24T23:10:37.012711+0000","last_active":"2024-04-24T23:10:37.012884+0000","last_peered":"2024-04-24T23:10:37.012884+0000","last_clean":"2024-04-24T23:10:37.012884+0000","last_became_active":"2024-04-24T23:10:37.012414+0000","last_became_peered":"2024-04-24T23:10:37.012414+0000","last_unstale":"2024-04-24T23:10:37.012884+0000","last_undegraded":"2024-04-24T23:10:37.012884+0000","last_fullsized":"2024-04-24T23:10:37.012884+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:00:56.631780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.002673+0000","last_change":"2024-04-24T23:10:37.002555+0000","last_active":"2024-04-24T23:10:37.002673+0000","last_peered":"2024-04-24T23:10:37.002673+0000","last_clean":"2024-04-24T23:10:37.002673+0000","last_became_active":"2024-04-24T23:10:37.002275+0000","last_became_peered":"2024-04-24T23:10:37.002275+0000","last_unstale":"2024-04-24T23:10:37.002673+0000","last_undegraded":"2024-04-24T23:10:37.002673+0000","last_fullsized":"2024-04-24T23:10:37.002673+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:24:18.212326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.008241+0000","last_change":"2024-04-24T23:10:37.007457+0000","last_active":"2024-04-24T23:10:37.008241+0000","last_peered":"2024-04-24T23:10:37.008241+0000","last_clean":"2024-04-24T23:10:37.008241+0000","last_became_active":"2024-04-24T23:10:37.007076+0000","last_became_peered":"2024-04-24T23:10:37.007076+0000","last_unstale":"2024-04-24T23:10:37.008241+0000","last_undegraded":"2024-04-24T23:10:37.008241+0000","last_fullsized":"2024-04-24T23:10:37.008241+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:12:31.454186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.006069+0000","last_change":"2024-04-24T23:10:37.005931+0000","last_active":"2024-04-24T23:10:37.006069+0000","last_peered":"2024-04-24T23:10:37.006069+0000","last_clean":"2024-04-24T23:10:37.006069+0000","last_became_active":"2024-04-24T23:10:37.005686+0000","last_became_peered":"2024-04-24T23:10:37.005686+0000","last_unstale":"2024-04-24T23:10:37.006069+0000","last_undegraded":"2024-04-24T23:10:37.006069+0000","last_fullsized":"2024-04-24T23:10:37.006069+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:49:21.491856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.003772+0000","last_change":"2024-04-24T23:10:37.003623+0000","last_active":"2024-04-24T23:10:37.003772+0000","last_peered":"2024-04-24T23:10:37.003772+0000","last_clean":"2024-04-24T23:10:37.003772+0000","last_became_active":"2024-04-24T23:10:37.003389+0000","last_became_peered":"2024-04-24T23:10:37.003389+0000","last_unstale":"2024-04-24T23:10:37.003772+0000","last_undegraded":"2024-04-24T23:10:37.003772+0000","last_fullsized":"2024-04-24T23:10:37.003772+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:30:22.461469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.004072+0000","last_change":"2024-04-24T23:10:37.003210+0000","last_active":"2024-04-24T23:10:37.004072+0000","last_peered":"2024-04-24T23:10:37.004072+0000","last_clean":"2024-04-24T23:10:37.004072+0000","last_became_active":"2024-04-24T23:10:37.002992+0000","last_became_peered":"2024-04-24T23:10:37.002992+0000","last_unstale":"2024-04-24T23:10:37.004072+0000","last_undegraded":"2024-04-24T23:10:37.004072+0000","last_fullsized":"2024-04-24T23:10:37.004072+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:59:25.663501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.008803+0000","last_change":"2024-04-24T23:10:37.008678+0000","last_active":"2024-04-24T23:10:37.008803+0000","last_peered":"2024-04-24T23:10:37.008803+0000","last_clean":"2024-04-24T23:10:37.008803+0000","last_became_active":"2024-04-24T23:10:37.008369+0000","last_became_peered":"2024-04-24T23:10:37.008369+0000","last_unstale":"2024-04-24T23:10:37.008803+0000","last_undegraded":"2024-04-24T23:10:37.008803+0000","last_fullsized":"2024-04-24T23:10:37.008803+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:59.274844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.006841+0000","last_change":"2024-04-24T23:10:37.006675+0000","last_active":"2024-04-24T23:10:37.006841+0000","last_peered":"2024-04-24T23:10:37.006841+0000","last_clean":"2024-04-24T23:10:37.006841+0000","last_became_active":"2024-04-24T23:10:37.006427+0000","last_became_peered":"2024-04-24T23:10:37.006427+0000","last_unstale":"2024-04-24T23:10:37.006841+0000","last_undegraded":"2024-04-24T23:10:37.006841+0000","last_fullsized":"2024-04-24T23:10:37.006841+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:39:10.133178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.006747+0000","last_change":"2024-04-24T23:10:37.006627+0000","last_active":"2024-04-24T23:10:37.006747+0000","last_peered":"2024-04-24T23:10:37.006747+0000","last_clean":"2024-04-24T23:10:37.006747+0000","last_became_active":"2024-04-24T23:10:37.006152+0000","last_became_peered":"2024-04-24T23:10:37.006152+0000","last_unstale":"2024-04-24T23:10:37.006747+0000","last_undegraded":"2024-04-24T23:10:37.006747+0000","last_fullsized":"2024-04-24T23:10:37.006747+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:22:28.640301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.007353+0000","last_change":"2024-04-24T23:10:37.007203+0000","last_active":"2024-04-24T23:10:37.007353+0000","last_peered":"2024-04-24T23:10:37.007353+0000","last_clean":"2024-04-24T23:10:37.007353+0000","last_became_active":"2024-04-24T23:10:37.006960+0000","last_became_peered":"2024-04-24T23:10:37.006960+0000","last_unstale":"2024-04-24T23:10:37.007353+0000","last_undegraded":"2024-04-24T23:10:37.007353+0000","last_fullsized":"2024-04-24T23:10:37.007353+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:03:26.361025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.001013+0000","last_change":"2024-04-24T23:10:37.000127+0000","last_active":"2024-04-24T23:10:37.001013+0000","last_peered":"2024-04-24T23:10:37.001013+0000","last_clean":"2024-04-24T23:10:37.001013+0000","last_became_active":"2024-04-24T23:10:36.999864+0000","last_became_peered":"2024-04-24T23:10:36.999864+0000","last_unstale":"2024-04-24T23:10:37.001013+0000","last_undegraded":"2024-04-24T23:10:37.001013+0000","last_fullsized":"2024-04-24T23:10:37.001013+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:12:04.154083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.004671+0000","last_change":"2024-04-24T23:10:37.004555+0000","last_active":"2024-04-24T23:10:37.004671+0000","last_peered":"2024-04-24T23:10:37.004671+0000","last_clean":"2024-04-24T23:10:37.004671+0000","last_became_active":"2024-04-24T23:10:37.004124+0000","last_became_peered":"2024-04-24T23:10:37.004124+0000","last_unstale":"2024-04-24T23:10:37.004671+0000","last_undegraded":"2024-04-24T23:10:37.004671+0000","last_fullsized":"2024-04-24T23:10:37.004671+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:02:52.094502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.010007+0000","last_change":"2024-04-24T23:10:37.009886+0000","last_active":"2024-04-24T23:10:37.010007+0000","last_peered":"2024-04-24T23:10:37.010007+0000","last_clean":"2024-04-24T23:10:37.010007+0000","last_became_active":"2024-04-24T23:10:37.009659+0000","last_became_peered":"2024-04-24T23:10:37.009659+0000","last_unstale":"2024-04-24T23:10:37.010007+0000","last_undegraded":"2024-04-24T23:10:37.010007+0000","last_fullsized":"2024-04-24T23:10:37.010007+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:33:17.919519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.002308+0000","last_change":"2024-04-24T23:10:37.002178+0000","last_active":"2024-04-24T23:10:37.002308+0000","last_peered":"2024-04-24T23:10:37.002308+0000","last_clean":"2024-04-24T23:10:37.002308+0000","last_became_active":"2024-04-24T23:10:37.001712+0000","last_became_peered":"2024-04-24T23:10:37.001712+0000","last_unstale":"2024-04-24T23:10:37.002308+0000","last_undegraded":"2024-04-24T23:10:37.002308+0000","last_fullsized":"2024-04-24T23:10:37.002308+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:57.343615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.013180+0000","last_change":"2024-04-24T23:10:37.011598+0000","last_active":"2024-04-24T23:10:37.013180+0000","last_peered":"2024-04-24T23:10:37.013180+0000","last_clean":"2024-04-24T23:10:37.013180+0000","last_became_active":"2024-04-24T23:10:37.010555+0000","last_became_peered":"2024-04-24T23:10:37.010555+0000","last_unstale":"2024-04-24T23:10:37.013180+0000","last_undegraded":"2024-04-24T23:10:37.013180+0000","last_fullsized":"2024-04-24T23:10:37.013180+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:42:00.527747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.009958+0000","last_change":"2024-04-24T23:10:37.009610+0000","last_active":"2024-04-24T23:10:37.009958+0000","last_peered":"2024-04-24T23:10:37.009958+0000","last_clean":"2024-04-24T23:10:37.009958+0000","last_became_active":"2024-04-24T23:10:37.009093+0000","last_became_peered":"2024-04-24T23:10:37.009093+0000","last_unstale":"2024-04-24T23:10:37.009958+0000","last_undegraded":"2024-04-24T23:10:37.009958+0000","last_fullsized":"2024-04-24T23:10:37.009958+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:31:22.414206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1f","version":"184'4","reported_seq":16,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.015558+0000","last_change":"2024-04-24T23:10:35.999854+0000","last_active":"2024-04-24T23:10:37.015558+0000","last_peered":"2024-04-24T23:10:37.015558+0000","last_clean":"2024-04-24T23:10:37.015558+0000","last_became_active":"2024-04-24T23:10:35.999014+0000","last_became_peered":"2024-04-24T23:10:35.999014+0000","last_unstale":"2024-04-24T23:10:37.015558+0000","last_undegraded":"2024-04-24T23:10:37.015558+0000","last_fullsized":"2024-04-24T23:10:37.015558+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:43.248061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.001355+0000","last_change":"2024-04-24T23:10:36.999675+0000","last_active":"2024-04-24T23:10:37.001355+0000","last_peered":"2024-04-24T23:10:37.001355+0000","last_clean":"2024-04-24T23:10:37.001355+0000","last_became_active":"2024-04-24T23:10:36.999403+0000","last_became_peered":"2024-04-24T23:10:36.999403+0000","last_unstale":"2024-04-24T23:10:37.001355+0000","last_undegraded":"2024-04-24T23:10:37.001355+0000","last_fullsized":"2024-04-24T23:10:37.001355+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:57:03.693700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1e","version":"184'1","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.018293+0000","last_change":"2024-04-24T23:10:35.994643+0000","last_active":"2024-04-24T23:10:37.018293+0000","last_peered":"2024-04-24T23:10:37.018293+0000","last_clean":"2024-04-24T23:10:37.018293+0000","last_became_active":"2024-04-24T23:10:35.994386+0000","last_became_peered":"2024-04-24T23:10:35.994386+0000","last_unstale":"2024-04-24T23:10:37.018293+0000","last_undegraded":"2024-04-24T23:10:37.018293+0000","last_fullsized":"2024-04-24T23:10:37.018293+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:54:14.516810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.008066+0000","last_change":"2024-04-24T23:10:37.007736+0000","last_active":"2024-04-24T23:10:37.008066+0000","last_peered":"2024-04-24T23:10:37.008066+0000","last_clean":"2024-04-24T23:10:37.008066+0000","last_became_active":"2024-04-24T23:10:37.007470+0000","last_became_peered":"2024-04-24T23:10:37.007470+0000","last_unstale":"2024-04-24T23:10:37.008066+0000","last_undegraded":"2024-04-24T23:10:37.008066+0000","last_fullsized":"2024-04-24T23:10:37.008066+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:58:49.251613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1d","version":"184'2","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.017929+0000","last_change":"2024-04-24T23:10:35.991120+0000","last_active":"2024-04-24T23:10:37.017929+0000","last_peered":"2024-04-24T23:10:37.017929+0000","last_clean":"2024-04-24T23:10:37.017929+0000","last_became_active":"2024-04-24T23:10:35.990834+0000","last_became_peered":"2024-04-24T23:10:35.990834+0000","last_unstale":"2024-04-24T23:10:37.017929+0000","last_undegraded":"2024-04-24T23:10:37.017929+0000","last_fullsized":"2024-04-24T23:10:37.017929+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:25.979175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.009475+0000","last_change":"2024-04-24T23:10:37.009362+0000","last_active":"2024-04-24T23:10:37.009475+0000","last_peered":"2024-04-24T23:10:37.009475+0000","last_clean":"2024-04-24T23:10:37.009475+0000","last_became_active":"2024-04-24T23:10:37.009129+0000","last_became_peered":"2024-04-24T23:10:37.009129+0000","last_unstale":"2024-04-24T23:10:37.009475+0000","last_undegraded":"2024-04-24T23:10:37.009475+0000","last_fullsized":"2024-04-24T23:10:37.009475+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:39:07.039159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1c","version":"184'2","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.018553+0000","last_change":"2024-04-24T23:10:35.993193+0000","last_active":"2024-04-24T23:10:37.018553+0000","last_peered":"2024-04-24T23:10:37.018553+0000","last_clean":"2024-04-24T23:10:37.018553+0000","last_became_active":"2024-04-24T23:10:35.992755+0000","last_became_peered":"2024-04-24T23:10:35.992755+0000","last_unstale":"2024-04-24T23:10:37.018553+0000","last_undegraded":"2024-04-24T23:10:37.018553+0000","last_fullsized":"2024-04-24T23:10:37.018553+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:47:11.159920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.007820+0000","last_change":"2024-04-24T23:10:37.007712+0000","last_active":"2024-04-24T23:10:37.007820+0000","last_peered":"2024-04-24T23:10:37.007820+0000","last_clean":"2024-04-24T23:10:37.007820+0000","last_became_active":"2024-04-24T23:10:37.007484+0000","last_became_peered":"2024-04-24T23:10:37.007484+0000","last_unstale":"2024-04-24T23:10:37.007820+0000","last_undegraded":"2024-04-24T23:10:37.007820+0000","last_fullsized":"2024-04-24T23:10:37.007820+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:34:02.283055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1b","version":"184'1","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.012228+0000","last_change":"2024-04-24T23:10:35.997388+0000","last_active":"2024-04-24T23:10:37.012228+0000","last_peered":"2024-04-24T23:10:37.012228+0000","last_clean":"2024-04-24T23:10:37.012228+0000","last_became_active":"2024-04-24T23:10:35.997015+0000","last_became_peered":"2024-04-24T23:10:35.997015+0000","last_unstale":"2024-04-24T23:10:37.012228+0000","last_undegraded":"2024-04-24T23:10:37.012228+0000","last_fullsized":"2024-04-24T23:10:37.012228+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:13:45.188236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.013276+0000","last_change":"2024-04-24T23:10:37.012065+0000","last_active":"2024-04-24T23:10:37.013276+0000","last_peered":"2024-04-24T23:10:37.013276+0000","last_clean":"2024-04-24T23:10:37.013276+0000","last_became_active":"2024-04-24T23:10:37.010900+0000","last_became_peered":"2024-04-24T23:10:37.010900+0000","last_unstale":"2024-04-24T23:10:37.013276+0000","last_undegraded":"2024-04-24T23:10:37.013276+0000","last_fullsized":"2024-04-24T23:10:37.013276+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:50:31.616773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1a","version":"184'1","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.016561+0000","last_change":"2024-04-24T23:10:35.999978+0000","last_active":"2024-04-24T23:10:37.016561+0000","last_peered":"2024-04-24T23:10:37.016561+0000","last_clean":"2024-04-24T23:10:37.016561+0000","last_became_active":"2024-04-24T23:10:35.999727+0000","last_became_peered":"2024-04-24T23:10:35.999727+0000","last_unstale":"2024-04-24T23:10:37.016561+0000","last_undegraded":"2024-04-24T23:10:37.016561+0000","last_fullsized":"2024-04-24T23:10:37.016561+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:12.119593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.004113+0000","last_change":"2024-04-24T23:10:37.003815+0000","last_active":"2024-04-24T23:10:37.004113+0000","last_peered":"2024-04-24T23:10:37.004113+0000","last_clean":"2024-04-24T23:10:37.004113+0000","last_became_active":"2024-04-24T23:10:37.003211+0000","last_became_peered":"2024-04-24T23:10:37.003211+0000","last_unstale":"2024-04-24T23:10:37.004113+0000","last_undegraded":"2024-04-24T23:10:37.004113+0000","last_fullsized":"2024-04-24T23:10:37.004113+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:20.867936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:36.996680+0000","last_change":"2024-04-24T23:10:35.997843+0000","last_active":"2024-04-24T23:10:36.996680+0000","last_peered":"2024-04-24T23:10:36.996680+0000","last_clean":"2024-04-24T23:10:36.996680+0000","last_became_active":"2024-04-24T23:10:35.997332+0000","last_became_peered":"2024-04-24T23:10:35.997332+0000","last_unstale":"2024-04-24T23:10:36.996680+0000","last_undegraded":"2024-04-24T23:10:36.996680+0000","last_fullsized":"2024-04-24T23:10:36.996680+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:42:50.314310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.007074+0000","last_change":"2024-04-24T23:10:37.006964+0000","last_active":"2024-04-24T23:10:37.007074+0000","last_peered":"2024-04-24T23:10:37.007074+0000","last_clean":"2024-04-24T23:10:37.007074+0000","last_became_active":"2024-04-24T23:10:37.006728+0000","last_became_peered":"2024-04-24T23:10:37.006728+0000","last_unstale":"2024-04-24T23:10:37.007074+0000","last_undegraded":"2024-04-24T23:10:37.007074+0000","last_fullsized":"2024-04-24T23:10:37.007074+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:10:29.200164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:36.997332+0000","last_change":"2024-04-24T23:10:35.985591+0000","last_active":"2024-04-24T23:10:36.997332+0000","last_peered":"2024-04-24T23:10:36.997332+0000","last_clean":"2024-04-24T23:10:36.997332+0000","last_became_active":"2024-04-24T23:10:35.985337+0000","last_became_peered":"2024-04-24T23:10:35.985337+0000","last_unstale":"2024-04-24T23:10:36.997332+0000","last_undegraded":"2024-04-24T23:10:36.997332+0000","last_fullsized":"2024-04-24T23:10:36.997332+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:14.594260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.005856+0000","last_change":"2024-04-24T23:10:37.005672+0000","last_active":"2024-04-24T23:10:37.005856+0000","last_peered":"2024-04-24T23:10:37.005856+0000","last_clean":"2024-04-24T23:10:37.005856+0000","last_became_active":"2024-04-24T23:10:37.005401+0000","last_became_peered":"2024-04-24T23:10:37.005401+0000","last_unstale":"2024-04-24T23:10:37.005856+0000","last_undegraded":"2024-04-24T23:10:37.005856+0000","last_fullsized":"2024-04-24T23:10:37.005856+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:55:25.145785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.003796+0000","last_change":"2024-04-24T23:10:35.996000+0000","last_active":"2024-04-24T23:10:37.003796+0000","last_peered":"2024-04-24T23:10:37.003796+0000","last_clean":"2024-04-24T23:10:37.003796+0000","last_became_active":"2024-04-24T23:10:35.995715+0000","last_became_peered":"2024-04-24T23:10:35.995715+0000","last_unstale":"2024-04-24T23:10:37.003796+0000","last_undegraded":"2024-04-24T23:10:37.003796+0000","last_fullsized":"2024-04-24T23:10:37.003796+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:27:34.034027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.001756+0000","last_change":"2024-04-24T23:10:37.001632+0000","last_active":"2024-04-24T23:10:37.001756+0000","last_peered":"2024-04-24T23:10:37.001756+0000","last_clean":"2024-04-24T23:10:37.001756+0000","last_became_active":"2024-04-24T23:10:37.001364+0000","last_became_peered":"2024-04-24T23:10:37.001364+0000","last_unstale":"2024-04-24T23:10:37.001756+0000","last_undegraded":"2024-04-24T23:10:37.001756+0000","last_fullsized":"2024-04-24T23:10:37.001756+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:59.508332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.15","version":"184'3","reported_seq":15,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.018434+0000","last_change":"2024-04-24T23:10:35.994171+0000","last_active":"2024-04-24T23:10:37.018434+0000","last_peered":"2024-04-24T23:10:37.018434+0000","last_clean":"2024-04-24T23:10:37.018434+0000","last_became_active":"2024-04-24T23:10:35.993292+0000","last_became_peered":"2024-04-24T23:10:35.993292+0000","last_unstale":"2024-04-24T23:10:37.018434+0000","last_undegraded":"2024-04-24T23:10:37.018434+0000","last_fullsized":"2024-04-24T23:10:37.018434+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:35:21.260712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.005112+0000","last_change":"2024-04-24T23:10:37.004851+0000","last_active":"2024-04-24T23:10:37.005112+0000","last_peered":"2024-04-24T23:10:37.005112+0000","last_clean":"2024-04-24T23:10:37.005112+0000","last_became_active":"2024-04-24T23:10:37.003252+0000","last_became_peered":"2024-04-24T23:10:37.003252+0000","last_unstale":"2024-04-24T23:10:37.005112+0000","last_undegraded":"2024-04-24T23:10:37.005112+0000","last_fullsized":"2024-04-24T23:10:37.005112+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:35:59.820604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.14","version":"184'6","reported_seq":18,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.020761+0000","last_change":"2024-04-24T23:10:35.994209+0000","last_active":"2024-04-24T23:10:37.020761+0000","last_peered":"2024-04-24T23:10:37.020761+0000","last_clean":"2024-04-24T23:10:37.020761+0000","last_became_active":"2024-04-24T23:10:35.993951+0000","last_became_peered":"2024-04-24T23:10:35.993951+0000","last_unstale":"2024-04-24T23:10:37.020761+0000","last_undegraded":"2024-04-24T23:10:37.020761+0000","last_fullsized":"2024-04-24T23:10:37.020761+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:32:21.495029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.003016+0000","last_change":"2024-04-24T23:10:37.002884+0000","last_active":"2024-04-24T23:10:37.003016+0000","last_peered":"2024-04-24T23:10:37.003016+0000","last_clean":"2024-04-24T23:10:37.003016+0000","last_became_active":"2024-04-24T23:10:37.002227+0000","last_became_peered":"2024-04-24T23:10:37.002227+0000","last_unstale":"2024-04-24T23:10:37.003016+0000","last_undegraded":"2024-04-24T23:10:37.003016+0000","last_fullsized":"2024-04-24T23:10:37.003016+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:15.391781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"184'2","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.016833+0000","last_change":"2024-04-24T23:10:35.988868+0000","last_active":"2024-04-24T23:10:37.016833+0000","last_peered":"2024-04-24T23:10:37.016833+0000","last_clean":"2024-04-24T23:10:37.016833+0000","last_became_active":"2024-04-24T23:10:35.988583+0000","last_became_peered":"2024-04-24T23:10:35.988583+0000","last_unstale":"2024-04-24T23:10:37.016833+0000","last_undegraded":"2024-04-24T23:10:37.016833+0000","last_fullsized":"2024-04-24T23:10:37.016833+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:27.111193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.004095+0000","last_change":"2024-04-24T23:10:37.003518+0000","last_active":"2024-04-24T23:10:37.004095+0000","last_peered":"2024-04-24T23:10:37.004095+0000","last_clean":"2024-04-24T23:10:37.004095+0000","last_became_active":"2024-04-24T23:10:37.002874+0000","last_became_peered":"2024-04-24T23:10:37.002874+0000","last_unstale":"2024-04-24T23:10:37.004095+0000","last_undegraded":"2024-04-24T23:10:37.004095+0000","last_fullsized":"2024-04-24T23:10:37.004095+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:58.728830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.002886+0000","last_change":"2024-04-24T23:10:35.994364+0000","last_active":"2024-04-24T23:10:37.002886+0000","last_peered":"2024-04-24T23:10:37.002886+0000","last_clean":"2024-04-24T23:10:37.002886+0000","last_became_active":"2024-04-24T23:10:35.994132+0000","last_became_peered":"2024-04-24T23:10:35.994132+0000","last_unstale":"2024-04-24T23:10:37.002886+0000","last_undegraded":"2024-04-24T23:10:37.002886+0000","last_fullsized":"2024-04-24T23:10:37.002886+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:36.783109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.005091+0000","last_change":"2024-04-24T23:10:37.004961+0000","last_active":"2024-04-24T23:10:37.005091+0000","last_peered":"2024-04-24T23:10:37.005091+0000","last_clean":"2024-04-24T23:10:37.005091+0000","last_became_active":"2024-04-24T23:10:37.004696+0000","last_became_peered":"2024-04-24T23:10:37.004696+0000","last_unstale":"2024-04-24T23:10:37.005091+0000","last_undegraded":"2024-04-24T23:10:37.005091+0000","last_fullsized":"2024-04-24T23:10:37.005091+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:07.927715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:36.997185+0000","last_change":"2024-04-24T23:10:35.997166+0000","last_active":"2024-04-24T23:10:36.997185+0000","last_peered":"2024-04-24T23:10:36.997185+0000","last_clean":"2024-04-24T23:10:36.997185+0000","last_became_active":"2024-04-24T23:10:35.996865+0000","last_became_peered":"2024-04-24T23:10:35.996865+0000","last_unstale":"2024-04-24T23:10:36.997185+0000","last_undegraded":"2024-04-24T23:10:36.997185+0000","last_fullsized":"2024-04-24T23:10:36.997185+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:32:12.340301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.011683+0000","last_change":"2024-04-24T23:10:37.011217+0000","last_active":"2024-04-24T23:10:37.011683+0000","last_peered":"2024-04-24T23:10:37.011683+0000","last_clean":"2024-04-24T23:10:37.011683+0000","last_became_active":"2024-04-24T23:10:37.010877+0000","last_became_peered":"2024-04-24T23:10:37.010877+0000","last_unstale":"2024-04-24T23:10:37.011683+0000","last_undegraded":"2024-04-24T23:10:37.011683+0000","last_fullsized":"2024-04-24T23:10:37.011683+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:07:27.021813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.10","version":"184'8","reported_seq":20,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.019188+0000","last_change":"2024-04-24T23:10:35.993657+0000","last_active":"2024-04-24T23:10:37.019188+0000","last_peered":"2024-04-24T23:10:37.019188+0000","last_clean":"2024-04-24T23:10:37.019188+0000","last_became_active":"2024-04-24T23:10:35.993411+0000","last_became_peered":"2024-04-24T23:10:35.993411+0000","last_unstale":"2024-04-24T23:10:37.019188+0000","last_undegraded":"2024-04-24T23:10:37.019188+0000","last_fullsized":"2024-04-24T23:10:37.019188+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:16:59.608136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:36.999530+0000","last_change":"2024-04-24T23:10:36.999386+0000","last_active":"2024-04-24T23:10:36.999530+0000","last_peered":"2024-04-24T23:10:36.999530+0000","last_clean":"2024-04-24T23:10:36.999530+0000","last_became_active":"2024-04-24T23:10:36.999099+0000","last_became_peered":"2024-04-24T23:10:36.999099+0000","last_unstale":"2024-04-24T23:10:36.999530+0000","last_undegraded":"2024-04-24T23:10:36.999530+0000","last_fullsized":"2024-04-24T23:10:36.999530+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:16.757940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.f","version":"184'1","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.019033+0000","last_change":"2024-04-24T23:10:35.992022+0000","last_active":"2024-04-24T23:10:37.019033+0000","last_peered":"2024-04-24T23:10:37.019033+0000","last_clean":"2024-04-24T23:10:37.019033+0000","last_became_active":"2024-04-24T23:10:35.991766+0000","last_became_peered":"2024-04-24T23:10:35.991766+0000","last_unstale":"2024-04-24T23:10:37.019033+0000","last_undegraded":"2024-04-24T23:10:37.019033+0000","last_fullsized":"2024-04-24T23:10:37.019033+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:08.684645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.005023+0000","last_change":"2024-04-24T23:10:37.003252+0000","last_active":"2024-04-24T23:10:37.005023+0000","last_peered":"2024-04-24T23:10:37.005023+0000","last_clean":"2024-04-24T23:10:37.005023+0000","last_became_active":"2024-04-24T23:10:37.002995+0000","last_became_peered":"2024-04-24T23:10:37.002995+0000","last_unstale":"2024-04-24T23:10:37.005023+0000","last_undegraded":"2024-04-24T23:10:37.005023+0000","last_fullsized":"2024-04-24T23:10:37.005023+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:50:44.121167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.e","version":"184'2","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.015921+0000","last_change":"2024-04-24T23:10:35.997234+0000","last_active":"2024-04-24T23:10:37.015921+0000","last_peered":"2024-04-24T23:10:37.015921+0000","last_clean":"2024-04-24T23:10:37.015921+0000","last_became_active":"2024-04-24T23:10:35.996764+0000","last_became_peered":"2024-04-24T23:10:35.996764+0000","last_unstale":"2024-04-24T23:10:37.015921+0000","last_undegraded":"2024-04-24T23:10:37.015921+0000","last_fullsized":"2024-04-24T23:10:37.015921+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:24:22.875084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.009939+0000","last_change":"2024-04-24T23:10:37.009349+0000","last_active":"2024-04-24T23:10:37.009939+0000","last_peered":"2024-04-24T23:10:37.009939+0000","last_clean":"2024-04-24T23:10:37.009939+0000","last_became_active":"2024-04-24T23:10:37.008333+0000","last_became_peered":"2024-04-24T23:10:37.008333+0000","last_unstale":"2024-04-24T23:10:37.009939+0000","last_undegraded":"2024-04-24T23:10:37.009939+0000","last_fullsized":"2024-04-24T23:10:37.009939+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:27.286371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.d","version":"184'1","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.017311+0000","last_change":"2024-04-24T23:10:35.995970+0000","last_active":"2024-04-24T23:10:37.017311+0000","last_peered":"2024-04-24T23:10:37.017311+0000","last_clean":"2024-04-24T23:10:37.017311+0000","last_became_active":"2024-04-24T23:10:35.995720+0000","last_became_peered":"2024-04-24T23:10:35.995720+0000","last_unstale":"2024-04-24T23:10:37.017311+0000","last_undegraded":"2024-04-24T23:10:37.017311+0000","last_fullsized":"2024-04-24T23:10:37.017311+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:00:40.007853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.000420+0000","last_change":"2024-04-24T23:10:37.000295+0000","last_active":"2024-04-24T23:10:37.000420+0000","last_peered":"2024-04-24T23:10:37.000420+0000","last_clean":"2024-04-24T23:10:37.000420+0000","last_became_active":"2024-04-24T23:10:37.000003+0000","last_became_peered":"2024-04-24T23:10:37.000003+0000","last_unstale":"2024-04-24T23:10:37.000420+0000","last_undegraded":"2024-04-24T23:10:37.000420+0000","last_fullsized":"2024-04-24T23:10:37.000420+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:51:03.078892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.000638+0000","last_change":"2024-04-24T23:10:35.992761+0000","last_active":"2024-04-24T23:10:37.000638+0000","last_peered":"2024-04-24T23:10:37.000638+0000","last_clean":"2024-04-24T23:10:37.000638+0000","last_became_active":"2024-04-24T23:10:35.992279+0000","last_became_peered":"2024-04-24T23:10:35.992279+0000","last_unstale":"2024-04-24T23:10:37.000638+0000","last_undegraded":"2024-04-24T23:10:37.000638+0000","last_fullsized":"2024-04-24T23:10:37.000638+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:28.574407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.007228+0000","last_change":"2024-04-24T23:10:37.007104+0000","last_active":"2024-04-24T23:10:37.007228+0000","last_peered":"2024-04-24T23:10:37.007228+0000","last_clean":"2024-04-24T23:10:37.007228+0000","last_became_active":"2024-04-24T23:10:37.006673+0000","last_became_peered":"2024-04-24T23:10:37.006673+0000","last_unstale":"2024-04-24T23:10:37.007228+0000","last_undegraded":"2024-04-24T23:10:37.007228+0000","last_fullsized":"2024-04-24T23:10:37.007228+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:22:53.848440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.b","version":"184'2","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.019449+0000","last_change":"2024-04-24T23:10:35.994431+0000","last_active":"2024-04-24T23:10:37.019449+0000","last_peered":"2024-04-24T23:10:37.019449+0000","last_clean":"2024-04-24T23:10:37.019449+0000","last_became_active":"2024-04-24T23:10:35.993715+0000","last_became_peered":"2024-04-24T23:10:35.993715+0000","last_unstale":"2024-04-24T23:10:37.019449+0000","last_undegraded":"2024-04-24T23:10:37.019449+0000","last_fullsized":"2024-04-24T23:10:37.019449+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:46:01.336615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.006798+0000","last_change":"2024-04-24T23:10:37.006673+0000","last_active":"2024-04-24T23:10:37.006798+0000","last_peered":"2024-04-24T23:10:37.006798+0000","last_clean":"2024-04-24T23:10:37.006798+0000","last_became_active":"2024-04-24T23:10:37.005912+0000","last_became_peered":"2024-04-24T23:10:37.005912+0000","last_unstale":"2024-04-24T23:10:37.006798+0000","last_undegraded":"2024-04-24T23:10:37.006798+0000","last_fullsized":"2024-04-24T23:10:37.006798+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:12:40.140042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:36.999798+0000","last_change":"2024-04-24T23:10:35.991747+0000","last_active":"2024-04-24T23:10:36.999798+0000","last_peered":"2024-04-24T23:10:36.999798+0000","last_clean":"2024-04-24T23:10:36.999798+0000","last_became_active":"2024-04-24T23:10:35.991462+0000","last_became_peered":"2024-04-24T23:10:35.991462+0000","last_unstale":"2024-04-24T23:10:36.999798+0000","last_undegraded":"2024-04-24T23:10:36.999798+0000","last_fullsized":"2024-04-24T23:10:36.999798+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:41:26.919041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.001479+0000","last_change":"2024-04-24T23:10:37.001155+0000","last_active":"2024-04-24T23:10:37.001479+0000","last_peered":"2024-04-24T23:10:37.001479+0000","last_clean":"2024-04-24T23:10:37.001479+0000","last_became_active":"2024-04-24T23:10:37.000633+0000","last_became_peered":"2024-04-24T23:10:37.000633+0000","last_unstale":"2024-04-24T23:10:37.001479+0000","last_undegraded":"2024-04-24T23:10:37.001479+0000","last_fullsized":"2024-04-24T23:10:37.001479+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:06:15.691725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.005354+0000","last_change":"2024-04-24T23:10:35.993906+0000","last_active":"2024-04-24T23:10:37.005354+0000","last_peered":"2024-04-24T23:10:37.005354+0000","last_clean":"2024-04-24T23:10:37.005354+0000","last_became_active":"2024-04-24T23:10:35.993660+0000","last_became_peered":"2024-04-24T23:10:35.993660+0000","last_unstale":"2024-04-24T23:10:37.005354+0000","last_undegraded":"2024-04-24T23:10:37.005354+0000","last_fullsized":"2024-04-24T23:10:37.005354+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:26:18.754981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.008658+0000","last_change":"2024-04-24T23:10:37.008472+0000","last_active":"2024-04-24T23:10:37.008658+0000","last_peered":"2024-04-24T23:10:37.008658+0000","last_clean":"2024-04-24T23:10:37.008658+0000","last_became_active":"2024-04-24T23:10:37.005340+0000","last_became_peered":"2024-04-24T23:10:37.005340+0000","last_unstale":"2024-04-24T23:10:37.008658+0000","last_undegraded":"2024-04-24T23:10:37.008658+0000","last_fullsized":"2024-04-24T23:10:37.008658+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:13:06.121421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:36.999405+0000","last_change":"2024-04-24T23:10:35.993107+0000","last_active":"2024-04-24T23:10:36.999405+0000","last_peered":"2024-04-24T23:10:36.999405+0000","last_clean":"2024-04-24T23:10:36.999405+0000","last_became_active":"2024-04-24T23:10:35.992154+0000","last_became_peered":"2024-04-24T23:10:35.992154+0000","last_unstale":"2024-04-24T23:10:36.999405+0000","last_undegraded":"2024-04-24T23:10:36.999405+0000","last_fullsized":"2024-04-24T23:10:36.999405+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:25:37.239074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.002443+0000","last_change":"2024-04-24T23:10:37.002111+0000","last_active":"2024-04-24T23:10:37.002443+0000","last_peered":"2024-04-24T23:10:37.002443+0000","last_clean":"2024-04-24T23:10:37.002443+0000","last_became_active":"2024-04-24T23:10:37.001191+0000","last_became_peered":"2024-04-24T23:10:37.001191+0000","last_unstale":"2024-04-24T23:10:37.002443+0000","last_undegraded":"2024-04-24T23:10:37.002443+0000","last_fullsized":"2024-04-24T23:10:37.002443+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:03:20.261932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.7","version":"184'2","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.015509+0000","last_change":"2024-04-24T23:10:35.994324+0000","last_active":"2024-04-24T23:10:37.015509+0000","last_peered":"2024-04-24T23:10:37.015509+0000","last_clean":"2024-04-24T23:10:37.015509+0000","last_became_active":"2024-04-24T23:10:35.993636+0000","last_became_peered":"2024-04-24T23:10:35.993636+0000","last_unstale":"2024-04-24T23:10:37.015509+0000","last_undegraded":"2024-04-24T23:10:37.015509+0000","last_fullsized":"2024-04-24T23:10:37.015509+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:35:43.319355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.011773+0000","last_change":"2024-04-24T23:10:37.011651+0000","last_active":"2024-04-24T23:10:37.011773+0000","last_peered":"2024-04-24T23:10:37.011773+0000","last_clean":"2024-04-24T23:10:37.011773+0000","last_became_active":"2024-04-24T23:10:37.011357+0000","last_became_peered":"2024-04-24T23:10:37.011357+0000","last_unstale":"2024-04-24T23:10:37.011773+0000","last_undegraded":"2024-04-24T23:10:37.011773+0000","last_fullsized":"2024-04-24T23:10:37.011773+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:40.903388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.001590+0000","last_change":"2024-04-24T23:10:36.000285+0000","last_active":"2024-04-24T23:10:37.001590+0000","last_peered":"2024-04-24T23:10:37.001590+0000","last_clean":"2024-04-24T23:10:37.001590+0000","last_became_active":"2024-04-24T23:10:35.999990+0000","last_became_peered":"2024-04-24T23:10:35.999990+0000","last_unstale":"2024-04-24T23:10:37.001590+0000","last_undegraded":"2024-04-24T23:10:37.001590+0000","last_fullsized":"2024-04-24T23:10:37.001590+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:14:36.496334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.007177+0000","last_change":"2024-04-24T23:10:37.007038+0000","last_active":"2024-04-24T23:10:37.007177+0000","last_peered":"2024-04-24T23:10:37.007177+0000","last_clean":"2024-04-24T23:10:37.007177+0000","last_became_active":"2024-04-24T23:10:37.006653+0000","last_became_peered":"2024-04-24T23:10:37.006653+0000","last_unstale":"2024-04-24T23:10:37.007177+0000","last_undegraded":"2024-04-24T23:10:37.007177+0000","last_fullsized":"2024-04-24T23:10:37.007177+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:08:20.983386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:36.997902+0000","last_change":"2024-04-24T23:10:35.998896+0000","last_active":"2024-04-24T23:10:36.997902+0000","last_peered":"2024-04-24T23:10:36.997902+0000","last_clean":"2024-04-24T23:10:36.997902+0000","last_became_active":"2024-04-24T23:10:35.998652+0000","last_became_peered":"2024-04-24T23:10:35.998652+0000","last_unstale":"2024-04-24T23:10:36.997902+0000","last_undegraded":"2024-04-24T23:10:36.997902+0000","last_fullsized":"2024-04-24T23:10:36.997902+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:27:53.829534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.012552+0000","last_change":"2024-04-24T23:10:37.012136+0000","last_active":"2024-04-24T23:10:37.012552+0000","last_peered":"2024-04-24T23:10:37.012552+0000","last_clean":"2024-04-24T23:10:37.012552+0000","last_became_active":"2024-04-24T23:10:37.011826+0000","last_became_peered":"2024-04-24T23:10:37.011826+0000","last_unstale":"2024-04-24T23:10:37.012552+0000","last_undegraded":"2024-04-24T23:10:37.012552+0000","last_fullsized":"2024-04-24T23:10:37.012552+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:44.366747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:36.997478+0000","last_change":"2024-04-24T23:10:35.992950+0000","last_active":"2024-04-24T23:10:36.997478+0000","last_peered":"2024-04-24T23:10:36.997478+0000","last_clean":"2024-04-24T23:10:36.997478+0000","last_became_active":"2024-04-24T23:10:35.992642+0000","last_became_peered":"2024-04-24T23:10:35.992642+0000","last_unstale":"2024-04-24T23:10:36.997478+0000","last_undegraded":"2024-04-24T23:10:36.997478+0000","last_fullsized":"2024-04-24T23:10:36.997478+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:15:04.388968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.013335+0000","last_change":"2024-04-24T23:10:37.012259+0000","last_active":"2024-04-24T23:10:37.013335+0000","last_peered":"2024-04-24T23:10:37.013335+0000","last_clean":"2024-04-24T23:10:37.013335+0000","last_became_active":"2024-04-24T23:10:37.011297+0000","last_became_peered":"2024-04-24T23:10:37.011297+0000","last_unstale":"2024-04-24T23:10:37.013335+0000","last_undegraded":"2024-04-24T23:10:37.013335+0000","last_fullsized":"2024-04-24T23:10:37.013335+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:18:13.287164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.000121+0000","last_change":"2024-04-24T23:10:35.991426+0000","last_active":"2024-04-24T23:10:37.000121+0000","last_peered":"2024-04-24T23:10:37.000121+0000","last_clean":"2024-04-24T23:10:37.000121+0000","last_became_active":"2024-04-24T23:10:35.990882+0000","last_became_peered":"2024-04-24T23:10:35.990882+0000","last_unstale":"2024-04-24T23:10:37.000121+0000","last_undegraded":"2024-04-24T23:10:37.000121+0000","last_fullsized":"2024-04-24T23:10:37.000121+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:11:29.917193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.001375+0000","last_change":"2024-04-24T23:10:37.000966+0000","last_active":"2024-04-24T23:10:37.001375+0000","last_peered":"2024-04-24T23:10:37.001375+0000","last_clean":"2024-04-24T23:10:37.001375+0000","last_became_active":"2024-04-24T23:10:37.000311+0000","last_became_peered":"2024-04-24T23:10:37.000311+0000","last_unstale":"2024-04-24T23:10:37.001375+0000","last_undegraded":"2024-04-24T23:10:37.001375+0000","last_fullsized":"2024-04-24T23:10:37.001375+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:28:03.147405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:36.997401+0000","last_change":"2024-04-24T23:10:35.991371+0000","last_active":"2024-04-24T23:10:36.997401+0000","last_peered":"2024-04-24T23:10:36.997401+0000","last_clean":"2024-04-24T23:10:36.997401+0000","last_became_active":"2024-04-24T23:10:35.991125+0000","last_became_peered":"2024-04-24T23:10:35.991125+0000","last_unstale":"2024-04-24T23:10:36.997401+0000","last_undegraded":"2024-04-24T23:10:36.997401+0000","last_fullsized":"2024-04-24T23:10:36.997401+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:38:36.911527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.009165+0000","last_change":"2024-04-24T23:10:37.009050+0000","last_active":"2024-04-24T23:10:37.009165+0000","last_peered":"2024-04-24T23:10:37.009165+0000","last_clean":"2024-04-24T23:10:37.009165+0000","last_became_active":"2024-04-24T23:10:37.008799+0000","last_became_peered":"2024-04-24T23:10:37.008799+0000","last_unstale":"2024-04-24T23:10:37.009165+0000","last_undegraded":"2024-04-24T23:10:37.009165+0000","last_fullsized":"2024-04-24T23:10:37.009165+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:56:27.011581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:36.998492+0000","last_change":"2024-04-24T23:10:36.000319+0000","last_active":"2024-04-24T23:10:36.998492+0000","last_peered":"2024-04-24T23:10:36.998492+0000","last_clean":"2024-04-24T23:10:36.998492+0000","last_became_active":"2024-04-24T23:10:35.999236+0000","last_became_peered":"2024-04-24T23:10:35.999236+0000","last_unstale":"2024-04-24T23:10:36.998492+0000","last_undegraded":"2024-04-24T23:10:36.998492+0000","last_fullsized":"2024-04-24T23:10:36.998492+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:50:26.335560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.002060+0000","last_change":"2024-04-24T23:10:37.001779+0000","last_active":"2024-04-24T23:10:37.002060+0000","last_peered":"2024-04-24T23:10:37.002060+0000","last_clean":"2024-04-24T23:10:37.002060+0000","last_became_active":"2024-04-24T23:10:37.001547+0000","last_became_peered":"2024-04-24T23:10:37.001547+0000","last_unstale":"2024-04-24T23:10:37.002060+0000","last_undegraded":"2024-04-24T23:10:37.002060+0000","last_fullsized":"2024-04-24T23:10:37.002060+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:47.163526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:36.998136+0000","last_change":"2024-04-24T23:10:35.997601+0000","last_active":"2024-04-24T23:10:36.998136+0000","last_peered":"2024-04-24T23:10:36.998136+0000","last_clean":"2024-04-24T23:10:36.998136+0000","last_became_active":"2024-04-24T23:10:35.996990+0000","last_became_peered":"2024-04-24T23:10:35.996990+0000","last_unstale":"2024-04-24T23:10:36.998136+0000","last_undegraded":"2024-04-24T23:10:36.998136+0000","last_fullsized":"2024-04-24T23:10:36.998136+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:54:26.006604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.010351+0000","last_change":"2024-04-24T23:10:37.010222+0000","last_active":"2024-04-24T23:10:37.010351+0000","last_peered":"2024-04-24T23:10:37.010351+0000","last_clean":"2024-04-24T23:10:37.010351+0000","last_became_active":"2024-04-24T23:10:37.009699+0000","last_became_peered":"2024-04-24T23:10:37.009699+0000","last_unstale":"2024-04-24T23:10:37.010351+0000","last_undegraded":"2024-04-24T23:10:37.010351+0000","last_fullsized":"2024-04-24T23:10:37.010351+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:24.214918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:36.999796+0000","last_change":"2024-04-24T23:10:35.991208+0000","last_active":"2024-04-24T23:10:36.999796+0000","last_peered":"2024-04-24T23:10:36.999796+0000","last_clean":"2024-04-24T23:10:36.999796+0000","last_became_active":"2024-04-24T23:10:35.990551+0000","last_became_peered":"2024-04-24T23:10:35.990551+0000","last_unstale":"2024-04-24T23:10:36.999796+0000","last_undegraded":"2024-04-24T23:10:36.999796+0000","last_fullsized":"2024-04-24T23:10:36.999796+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:29:38.358276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":494,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.001116+0000","last_change":"2024-04-24T22:58:55.138810+0000","last_active":"2024-04-24T23:10:37.001116+0000","last_peered":"2024-04-24T23:10:37.001116+0000","last_clean":"2024-04-24T23:10:37.001116+0000","last_became_active":"2024-04-24T22:58:55.138627+0000","last_became_peered":"2024-04-24T22:58:55.138627+0000","last_unstale":"2024-04-24T23:10:37.001116+0000","last_undegraded":"2024-04-24T23:10:37.001116+0000","last_fullsized":"2024-04-24T23:10:37.001116+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_clean_scrub_stamp":"2024-04-24T22:58:54.127235+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:51.383182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574991,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1005,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194307,"statfs":{"total":200000000,"available":198970585,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574991,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1005,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194307,"statfs":{"total":200000000,"available":198970585,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574991,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1581,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193730,"statfs":{"total":200000000,"available":198380217,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574991,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1005,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194307,"statfs":{"total":200000000,"available":198970585,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574991,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1005,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194307,"statfs":{"total":200000000,"available":198970585,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574991,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1005,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194307,"statfs":{"total":200000000,"available":198970585,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574991,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1581,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193730,"statfs":{"total":200000000,"available":198380217,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574991,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1005,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194307,"statfs":{"total":200000000,"available":198970585,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-24T23:10:40.005 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:10:40.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.140+0000 7f2876ca5640 1 Processor -- start 2024-04-24T23:10:40.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.140+0000 7f2876ca5640 1 -- start start 2024-04-24T23:10:40.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.141+0000 7f2876ca5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f287006cd00 0x7f28700ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:40.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.141+0000 7f2876ca5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f287015cb00 0x7f28700ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:40.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.141+0000 7f2876ca5640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28700b4520 con 0x7f287015b380 2024-04-24T23:10:40.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.141+0000 7f2876ca5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f28700b4800 con 0x7f287015cb00 2024-04-24T23:10:40.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.141+0000 7f2876ca5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f28700b4ae0 con 0x7f287006cd00 2024-04-24T23:10:40.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.141+0000 7f2875242640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f287015cb00 0x7f28700ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:40.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.141+0000 7f2874a41640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f287015b380 0x7f287006c5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58604/0 (socket says 172.21.15.32:58604) 2024-04-24T23:10:40.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.141+0000 7f2874a41640 1 -- 172.21.15.32:0/3717661739 learned_addr learned my addr 172.21.15.32:0/3717661739 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:10:40.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.142+0000 7f286ffff640 1 --2- 172.21.15.32:0/3717661739 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f287006cd00 0x7f28700ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:40.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.142+0000 7f2875242640 1 -- 172.21.15.32:0/3717661739 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f287006cd00 msgr2=0x7f28700ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:40.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.142+0000 7f2875242640 1 --2- 172.21.15.32:0/3717661739 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f287006cd00 0x7f28700ce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.142+0000 7f2875242640 1 -- 172.21.15.32:0/3717661739 >> v1:172.21.15.32:6789/0 conn(0x7f287015b380 legacy=0x7f287006c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:40.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.142+0000 7f286f7fe640 1 -- 172.21.15.32:0/3717661739 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 278150504 0 0) 0x7f28700b4520 con 0x7f287015b380 2024-04-24T23:10:40.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.142+0000 7f2875242640 1 -- 172.21.15.32:0/3717661739 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28700b4dc0 con 0x7f287015cb00 2024-04-24T23:10:40.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.142+0000 7f2875242640 1 --2- 172.21.15.32:0/3717661739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f287015cb00 0x7f28700ce970 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f28600097a0 tx=0x7f286002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f0889dc57051751 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:40.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.142+0000 7f286f7fe640 1 -- 172.21.15.32:0/3717661739 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2860037070 con 0x7f287015cb00 2024-04-24T23:10:40.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.143+0000 7f286f7fe640 1 -- 172.21.15.32:0/3717661739 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f286002ae50 con 0x7f287015cb00 2024-04-24T23:10:40.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.143+0000 7f286f7fe640 1 -- 172.21.15.32:0/3717661739 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2860032660 con 0x7f287015cb00 2024-04-24T23:10:40.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.143+0000 7f2876ca5640 1 -- 172.21.15.32:0/3717661739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f287015cb00 msgr2=0x7f28700ce970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:40.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.143+0000 7f2876ca5640 1 --2- 172.21.15.32:0/3717661739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f287015cb00 0x7f28700ce970 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f28600097a0 tx=0x7f286002a100 comp rx=0 tx=0).stop 2024-04-24T23:10:40.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.143+0000 7f2876ca5640 1 -- 172.21.15.32:0/3717661739 shutdown_connections 2024-04-24T23:10:40.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.143+0000 7f2876ca5640 1 --2- 172.21.15.32:0/3717661739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f287015cb00 0x7f28700ce970 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.143+0000 7f2876ca5640 1 --2- 172.21.15.32:0/3717661739 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f287006cd00 0x7f28700ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.143+0000 7f2876ca5640 1 -- 172.21.15.32:0/3717661739 >> 172.21.15.32:0/3717661739 conn(0x7f287006b110 msgr2=0x7f287006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:40.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.143+0000 7f2876ca5640 1 -- 172.21.15.32:0/3717661739 shutdown_connections 2024-04-24T23:10:40.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.143+0000 7f2876ca5640 1 -- 172.21.15.32:0/3717661739 wait complete. 2024-04-24T23:10:40.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.144+0000 7f2876ca5640 1 Processor -- start 2024-04-24T23:10:40.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.144+0000 7f2876ca5640 1 -- start start 2024-04-24T23:10:40.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.145+0000 7f2876ca5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f287006cd00 0x7f2870145120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:40.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.145+0000 7f2876ca5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f287015b380 0x7f287014f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:40.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.145+0000 7f2876ca5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f287015cb00 0x7f287014aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:40.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.145+0000 7f2876ca5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f287015b760 con 0x7f287006cd00 2024-04-24T23:10:40.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.145+0000 7f2876ca5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f28700ceeb0 con 0x7f287015b380 2024-04-24T23:10:40.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.145+0000 7f2876ca5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f28700b55c0 con 0x7f287015cb00 2024-04-24T23:10:40.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.145+0000 7f2874a41640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f287006cd00 0x7f2870145120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:40.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.145+0000 7f2874a41640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f287006cd00 0x7f2870145120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48728/0 (socket says 172.21.15.32:48728) 2024-04-24T23:10:40.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.145+0000 7f2874a41640 1 -- 172.21.15.32:0/138297846 learned_addr learned my addr 172.21.15.32:0/138297846 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:40.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.145+0000 7f2875242640 1 --2- 172.21.15.32:0/138297846 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f287015cb00 0x7f287014aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:40.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.145+0000 7f286ffff640 1 --2- 172.21.15.32:0/138297846 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f287015b380 0x7f287014f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:40.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.145+0000 7f2874a41640 1 -- 172.21.15.32:0/138297846 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f287015cb00 msgr2=0x7f287014aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:40.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.145+0000 7f2874a41640 1 --2- 172.21.15.32:0/138297846 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f287015cb00 0x7f287014aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.145+0000 7f2874a41640 1 -- 172.21.15.32:0/138297846 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f287015b380 msgr2=0x7f287014f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:40.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.146+0000 7f2874a41640 1 --2- 172.21.15.32:0/138297846 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f287015b380 0x7f287014f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.146+0000 7f2874a41640 1 -- 172.21.15.32:0/138297846 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2870158a10 con 0x7f287006cd00 2024-04-24T23:10:40.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.146+0000 7f2874a41640 1 --2- 172.21.15.32:0/138297846 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f287006cd00 0x7f2870145120 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f286800c8c0 tx=0x7f286800cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:40.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.146+0000 7f286d7fa640 1 -- 172.21.15.32:0/138297846 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2868018070 con 0x7f287006cd00 2024-04-24T23:10:40.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.146+0000 7f286d7fa640 1 -- 172.21.15.32:0/138297846 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f286800a640 con 0x7f287006cd00 2024-04-24T23:10:40.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.146+0000 7f2876ca5640 1 -- 172.21.15.32:0/138297846 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f287014fa00 con 0x7f287006cd00 2024-04-24T23:10:40.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.146+0000 7f286d7fa640 1 -- 172.21.15.32:0/138297846 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f286800aa30 con 0x7f287006cd00 2024-04-24T23:10:40.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.146+0000 7f2876ca5640 1 -- 172.21.15.32:0/138297846 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2870158c80 con 0x7f287006cd00 2024-04-24T23:10:40.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.147+0000 7f2876ca5640 1 -- 172.21.15.32:0/138297846 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2870068430 con 0x7f287006cd00 2024-04-24T23:10:40.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.147+0000 7f286d7fa640 1 -- 172.21.15.32:0/138297846 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f286801b070 con 0x7f287006cd00 2024-04-24T23:10:40.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.148+0000 7f286d7fa640 1 --2- 172.21.15.32:0/138297846 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f285c06be30 0x7f285c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:40.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.148+0000 7f286d7fa640 1 -- 172.21.15.32:0/138297846 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(185..185 src has 1..185) ==== 5709+0+0 (secure 0 0 0) 0x7f286808a4e0 con 0x7f287006cd00 2024-04-24T23:10:40.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.148+0000 7f286ffff640 1 --2- 172.21.15.32:0/138297846 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f285c06be30 0x7f285c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:40.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.149+0000 7f286ffff640 1 --2- 172.21.15.32:0/138297846 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f285c06be30 0x7f285c06e2f0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f287015f630 tx=0x7f2864008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:40.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.153+0000 7f286d7fa640 1 -- 172.21.15.32:0/138297846 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2868058f20 con 0x7f287006cd00 2024-04-24T23:10:40.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.332+0000 7f2876ca5640 1 -- 172.21.15.32:0/138297846 --> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f28700afd60 con 0x7f285c06be30 2024-04-24T23:10:40.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.339+0000 7f286d7fa640 1 -- 172.21.15.32:0/138297846 <== mgr.4108 v2:172.21.15.121:6832/1848157943 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247925 (secure 0 0 0) 0x7f28700afd60 con 0x7f285c06be30 2024-04-24T23:10:40.344 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:10:40.346 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-24T23:10:40.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.342+0000 7f2876ca5640 1 -- 172.21.15.32:0/138297846 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f285c06be30 msgr2=0x7f285c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:40.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.342+0000 7f2876ca5640 1 --2- 172.21.15.32:0/138297846 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f285c06be30 0x7f285c06e2f0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f287015f630 tx=0x7f2864008040 comp rx=0 tx=0).stop 2024-04-24T23:10:40.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.342+0000 7f2876ca5640 1 -- 172.21.15.32:0/138297846 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f287006cd00 msgr2=0x7f2870145120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:40.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.342+0000 7f2876ca5640 1 --2- 172.21.15.32:0/138297846 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f287006cd00 0x7f2870145120 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f286800c8c0 tx=0x7f286800cd80 comp rx=0 tx=0).stop 2024-04-24T23:10:40.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.342+0000 7f2876ca5640 1 -- 172.21.15.32:0/138297846 shutdown_connections 2024-04-24T23:10:40.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.342+0000 7f2876ca5640 1 --2- 172.21.15.32:0/138297846 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f285c06be30 0x7f285c06e2f0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.342+0000 7f2876ca5640 1 --2- 172.21.15.32:0/138297846 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f287015cb00 0x7f287014aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.342+0000 7f2876ca5640 1 --2- 172.21.15.32:0/138297846 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f287015b380 0x7f287014f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.342+0000 7f2876ca5640 1 --2- 172.21.15.32:0/138297846 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f287006cd00 0x7f2870145120 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.342+0000 7f2876ca5640 1 -- 172.21.15.32:0/138297846 >> 172.21.15.32:0/138297846 conn(0x7f287006b110 msgr2=0x7f28700ce360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:40.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.342+0000 7f2876ca5640 1 -- 172.21.15.32:0/138297846 shutdown_connections 2024-04-24T23:10:40.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.342+0000 7f2876ca5640 1 -- 172.21.15.32:0/138297846 wait complete. 2024-04-24T23:10:40.356 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":530,"stamp":"2024-04-24T23:10:39.293865+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":9195,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1553304,"statfs":{"total":1600000000,"available":1590583944,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.313509"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.008754+0000","last_change":"2024-04-24T23:10:37.008622+0000","last_active":"2024-04-24T23:10:37.008754+0000","last_peered":"2024-04-24T23:10:37.008754+0000","last_clean":"2024-04-24T23:10:37.008754+0000","last_became_active":"2024-04-24T23:10:37.008368+0000","last_became_peered":"2024-04-24T23:10:37.008368+0000","last_unstale":"2024-04-24T23:10:37.008754+0000","last_undegraded":"2024-04-24T23:10:37.008754+0000","last_fullsized":"2024-04-24T23:10:37.008754+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:55:05.387326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.008920+0000","last_change":"2024-04-24T23:10:37.008790+0000","last_active":"2024-04-24T23:10:37.008920+0000","last_peered":"2024-04-24T23:10:37.008920+0000","last_clean":"2024-04-24T23:10:37.008920+0000","last_became_active":"2024-04-24T23:10:37.008476+0000","last_became_peered":"2024-04-24T23:10:37.008476+0000","last_unstale":"2024-04-24T23:10:37.008920+0000","last_undegraded":"2024-04-24T23:10:37.008920+0000","last_fullsized":"2024-04-24T23:10:37.008920+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:56:53.705009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.004588+0000","last_change":"2024-04-24T23:10:37.003644+0000","last_active":"2024-04-24T23:10:37.004588+0000","last_peered":"2024-04-24T23:10:37.004588+0000","last_clean":"2024-04-24T23:10:37.004588+0000","last_became_active":"2024-04-24T23:10:37.002896+0000","last_became_peered":"2024-04-24T23:10:37.002896+0000","last_unstale":"2024-04-24T23:10:37.004588+0000","last_undegraded":"2024-04-24T23:10:37.004588+0000","last_fullsized":"2024-04-24T23:10:37.004588+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:48:23.839497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.009975+0000","last_change":"2024-04-24T23:10:37.009792+0000","last_active":"2024-04-24T23:10:37.009975+0000","last_peered":"2024-04-24T23:10:37.009975+0000","last_clean":"2024-04-24T23:10:37.009975+0000","last_became_active":"2024-04-24T23:10:37.009387+0000","last_became_peered":"2024-04-24T23:10:37.009387+0000","last_unstale":"2024-04-24T23:10:37.009975+0000","last_undegraded":"2024-04-24T23:10:37.009975+0000","last_fullsized":"2024-04-24T23:10:37.009975+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:30:28.867316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.009901+0000","last_change":"2024-04-24T23:10:37.009744+0000","last_active":"2024-04-24T23:10:37.009901+0000","last_peered":"2024-04-24T23:10:37.009901+0000","last_clean":"2024-04-24T23:10:37.009901+0000","last_became_active":"2024-04-24T23:10:37.009304+0000","last_became_peered":"2024-04-24T23:10:37.009304+0000","last_unstale":"2024-04-24T23:10:37.009901+0000","last_undegraded":"2024-04-24T23:10:37.009901+0000","last_fullsized":"2024-04-24T23:10:37.009901+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:28:42.260245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.008046+0000","last_change":"2024-04-24T23:10:37.006773+0000","last_active":"2024-04-24T23:10:37.008046+0000","last_peered":"2024-04-24T23:10:37.008046+0000","last_clean":"2024-04-24T23:10:37.008046+0000","last_became_active":"2024-04-24T23:10:37.006525+0000","last_became_peered":"2024-04-24T23:10:37.006525+0000","last_unstale":"2024-04-24T23:10:37.008046+0000","last_undegraded":"2024-04-24T23:10:37.008046+0000","last_fullsized":"2024-04-24T23:10:37.008046+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:41.030023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.010206+0000","last_change":"2024-04-24T23:10:37.009945+0000","last_active":"2024-04-24T23:10:37.010206+0000","last_peered":"2024-04-24T23:10:37.010206+0000","last_clean":"2024-04-24T23:10:37.010206+0000","last_became_active":"2024-04-24T23:10:37.009386+0000","last_became_peered":"2024-04-24T23:10:37.009386+0000","last_unstale":"2024-04-24T23:10:37.010206+0000","last_undegraded":"2024-04-24T23:10:37.010206+0000","last_fullsized":"2024-04-24T23:10:37.010206+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:02.652113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.011722+0000","last_change":"2024-04-24T23:10:37.011533+0000","last_active":"2024-04-24T23:10:37.011722+0000","last_peered":"2024-04-24T23:10:37.011722+0000","last_clean":"2024-04-24T23:10:37.011722+0000","last_became_active":"2024-04-24T23:10:37.011288+0000","last_became_peered":"2024-04-24T23:10:37.011288+0000","last_unstale":"2024-04-24T23:10:37.011722+0000","last_undegraded":"2024-04-24T23:10:37.011722+0000","last_fullsized":"2024-04-24T23:10:37.011722+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:07:19.425606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.016535+0000","last_change":"2024-04-24T23:10:37.015856+0000","last_active":"2024-04-24T23:10:37.016535+0000","last_peered":"2024-04-24T23:10:37.016535+0000","last_clean":"2024-04-24T23:10:37.016535+0000","last_became_active":"2024-04-24T23:10:37.015508+0000","last_became_peered":"2024-04-24T23:10:37.015508+0000","last_unstale":"2024-04-24T23:10:37.016535+0000","last_undegraded":"2024-04-24T23:10:37.016535+0000","last_fullsized":"2024-04-24T23:10:37.016535+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:39:42.709982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.008907+0000","last_change":"2024-04-24T23:10:37.008780+0000","last_active":"2024-04-24T23:10:37.008907+0000","last_peered":"2024-04-24T23:10:37.008907+0000","last_clean":"2024-04-24T23:10:37.008907+0000","last_became_active":"2024-04-24T23:10:37.008514+0000","last_became_peered":"2024-04-24T23:10:37.008514+0000","last_unstale":"2024-04-24T23:10:37.008907+0000","last_undegraded":"2024-04-24T23:10:37.008907+0000","last_fullsized":"2024-04-24T23:10:37.008907+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:12:20.165575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.005233+0000","last_change":"2024-04-24T23:10:37.005104+0000","last_active":"2024-04-24T23:10:37.005233+0000","last_peered":"2024-04-24T23:10:37.005233+0000","last_clean":"2024-04-24T23:10:37.005233+0000","last_became_active":"2024-04-24T23:10:37.004857+0000","last_became_peered":"2024-04-24T23:10:37.004857+0000","last_unstale":"2024-04-24T23:10:37.005233+0000","last_undegraded":"2024-04-24T23:10:37.005233+0000","last_fullsized":"2024-04-24T23:10:37.005233+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:57:57.500385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.010067+0000","last_change":"2024-04-24T23:10:37.009954+0000","last_active":"2024-04-24T23:10:37.010067+0000","last_peered":"2024-04-24T23:10:37.010067+0000","last_clean":"2024-04-24T23:10:37.010067+0000","last_became_active":"2024-04-24T23:10:37.007786+0000","last_became_peered":"2024-04-24T23:10:37.007786+0000","last_unstale":"2024-04-24T23:10:37.010067+0000","last_undegraded":"2024-04-24T23:10:37.010067+0000","last_fullsized":"2024-04-24T23:10:37.010067+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:39.879189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.009807+0000","last_change":"2024-04-24T23:10:37.009622+0000","last_active":"2024-04-24T23:10:37.009807+0000","last_peered":"2024-04-24T23:10:37.009807+0000","last_clean":"2024-04-24T23:10:37.009807+0000","last_became_active":"2024-04-24T23:10:37.007721+0000","last_became_peered":"2024-04-24T23:10:37.007721+0000","last_unstale":"2024-04-24T23:10:37.009807+0000","last_undegraded":"2024-04-24T23:10:37.009807+0000","last_fullsized":"2024-04-24T23:10:37.009807+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:06:48.308222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.010533+0000","last_change":"2024-04-24T23:10:37.010378+0000","last_active":"2024-04-24T23:10:37.010533+0000","last_peered":"2024-04-24T23:10:37.010533+0000","last_clean":"2024-04-24T23:10:37.010533+0000","last_became_active":"2024-04-24T23:10:37.010023+0000","last_became_peered":"2024-04-24T23:10:37.010023+0000","last_unstale":"2024-04-24T23:10:37.010533+0000","last_undegraded":"2024-04-24T23:10:37.010533+0000","last_fullsized":"2024-04-24T23:10:37.010533+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:21:01.576041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.012711+0000","last_change":"2024-04-24T23:10:37.012604+0000","last_active":"2024-04-24T23:10:37.012711+0000","last_peered":"2024-04-24T23:10:37.012711+0000","last_clean":"2024-04-24T23:10:37.012711+0000","last_became_active":"2024-04-24T23:10:37.012372+0000","last_became_peered":"2024-04-24T23:10:37.012372+0000","last_unstale":"2024-04-24T23:10:37.012711+0000","last_undegraded":"2024-04-24T23:10:37.012711+0000","last_fullsized":"2024-04-24T23:10:37.012711+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:28:29.997978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.003221+0000","last_change":"2024-04-24T23:10:37.003103+0000","last_active":"2024-04-24T23:10:37.003221+0000","last_peered":"2024-04-24T23:10:37.003221+0000","last_clean":"2024-04-24T23:10:37.003221+0000","last_became_active":"2024-04-24T23:10:37.002582+0000","last_became_peered":"2024-04-24T23:10:37.002582+0000","last_unstale":"2024-04-24T23:10:37.003221+0000","last_undegraded":"2024-04-24T23:10:37.003221+0000","last_fullsized":"2024-04-24T23:10:37.003221+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:03:25.100801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.013354+0000","last_change":"2024-04-24T23:10:37.012815+0000","last_active":"2024-04-24T23:10:37.013354+0000","last_peered":"2024-04-24T23:10:37.013354+0000","last_clean":"2024-04-24T23:10:37.013354+0000","last_became_active":"2024-04-24T23:10:37.012583+0000","last_became_peered":"2024-04-24T23:10:37.012583+0000","last_unstale":"2024-04-24T23:10:37.013354+0000","last_undegraded":"2024-04-24T23:10:37.013354+0000","last_fullsized":"2024-04-24T23:10:37.013354+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:07:38.090091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.012884+0000","last_change":"2024-04-24T23:10:37.012711+0000","last_active":"2024-04-24T23:10:37.012884+0000","last_peered":"2024-04-24T23:10:37.012884+0000","last_clean":"2024-04-24T23:10:37.012884+0000","last_became_active":"2024-04-24T23:10:37.012414+0000","last_became_peered":"2024-04-24T23:10:37.012414+0000","last_unstale":"2024-04-24T23:10:37.012884+0000","last_undegraded":"2024-04-24T23:10:37.012884+0000","last_fullsized":"2024-04-24T23:10:37.012884+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:00:56.631780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.002673+0000","last_change":"2024-04-24T23:10:37.002555+0000","last_active":"2024-04-24T23:10:37.002673+0000","last_peered":"2024-04-24T23:10:37.002673+0000","last_clean":"2024-04-24T23:10:37.002673+0000","last_became_active":"2024-04-24T23:10:37.002275+0000","last_became_peered":"2024-04-24T23:10:37.002275+0000","last_unstale":"2024-04-24T23:10:37.002673+0000","last_undegraded":"2024-04-24T23:10:37.002673+0000","last_fullsized":"2024-04-24T23:10:37.002673+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:24:18.212326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.008241+0000","last_change":"2024-04-24T23:10:37.007457+0000","last_active":"2024-04-24T23:10:37.008241+0000","last_peered":"2024-04-24T23:10:37.008241+0000","last_clean":"2024-04-24T23:10:37.008241+0000","last_became_active":"2024-04-24T23:10:37.007076+0000","last_became_peered":"2024-04-24T23:10:37.007076+0000","last_unstale":"2024-04-24T23:10:37.008241+0000","last_undegraded":"2024-04-24T23:10:37.008241+0000","last_fullsized":"2024-04-24T23:10:37.008241+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:12:31.454186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.006069+0000","last_change":"2024-04-24T23:10:37.005931+0000","last_active":"2024-04-24T23:10:37.006069+0000","last_peered":"2024-04-24T23:10:37.006069+0000","last_clean":"2024-04-24T23:10:37.006069+0000","last_became_active":"2024-04-24T23:10:37.005686+0000","last_became_peered":"2024-04-24T23:10:37.005686+0000","last_unstale":"2024-04-24T23:10:37.006069+0000","last_undegraded":"2024-04-24T23:10:37.006069+0000","last_fullsized":"2024-04-24T23:10:37.006069+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:49:21.491856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.003772+0000","last_change":"2024-04-24T23:10:37.003623+0000","last_active":"2024-04-24T23:10:37.003772+0000","last_peered":"2024-04-24T23:10:37.003772+0000","last_clean":"2024-04-24T23:10:37.003772+0000","last_became_active":"2024-04-24T23:10:37.003389+0000","last_became_peered":"2024-04-24T23:10:37.003389+0000","last_unstale":"2024-04-24T23:10:37.003772+0000","last_undegraded":"2024-04-24T23:10:37.003772+0000","last_fullsized":"2024-04-24T23:10:37.003772+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:30:22.461469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.004072+0000","last_change":"2024-04-24T23:10:37.003210+0000","last_active":"2024-04-24T23:10:37.004072+0000","last_peered":"2024-04-24T23:10:37.004072+0000","last_clean":"2024-04-24T23:10:37.004072+0000","last_became_active":"2024-04-24T23:10:37.002992+0000","last_became_peered":"2024-04-24T23:10:37.002992+0000","last_unstale":"2024-04-24T23:10:37.004072+0000","last_undegraded":"2024-04-24T23:10:37.004072+0000","last_fullsized":"2024-04-24T23:10:37.004072+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:59:25.663501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.008803+0000","last_change":"2024-04-24T23:10:37.008678+0000","last_active":"2024-04-24T23:10:37.008803+0000","last_peered":"2024-04-24T23:10:37.008803+0000","last_clean":"2024-04-24T23:10:37.008803+0000","last_became_active":"2024-04-24T23:10:37.008369+0000","last_became_peered":"2024-04-24T23:10:37.008369+0000","last_unstale":"2024-04-24T23:10:37.008803+0000","last_undegraded":"2024-04-24T23:10:37.008803+0000","last_fullsized":"2024-04-24T23:10:37.008803+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:59.274844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.006841+0000","last_change":"2024-04-24T23:10:37.006675+0000","last_active":"2024-04-24T23:10:37.006841+0000","last_peered":"2024-04-24T23:10:37.006841+0000","last_clean":"2024-04-24T23:10:37.006841+0000","last_became_active":"2024-04-24T23:10:37.006427+0000","last_became_peered":"2024-04-24T23:10:37.006427+0000","last_unstale":"2024-04-24T23:10:37.006841+0000","last_undegraded":"2024-04-24T23:10:37.006841+0000","last_fullsized":"2024-04-24T23:10:37.006841+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:39:10.133178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.006747+0000","last_change":"2024-04-24T23:10:37.006627+0000","last_active":"2024-04-24T23:10:37.006747+0000","last_peered":"2024-04-24T23:10:37.006747+0000","last_clean":"2024-04-24T23:10:37.006747+0000","last_became_active":"2024-04-24T23:10:37.006152+0000","last_became_peered":"2024-04-24T23:10:37.006152+0000","last_unstale":"2024-04-24T23:10:37.006747+0000","last_undegraded":"2024-04-24T23:10:37.006747+0000","last_fullsized":"2024-04-24T23:10:37.006747+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:22:28.640301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.007353+0000","last_change":"2024-04-24T23:10:37.007203+0000","last_active":"2024-04-24T23:10:37.007353+0000","last_peered":"2024-04-24T23:10:37.007353+0000","last_clean":"2024-04-24T23:10:37.007353+0000","last_became_active":"2024-04-24T23:10:37.006960+0000","last_became_peered":"2024-04-24T23:10:37.006960+0000","last_unstale":"2024-04-24T23:10:37.007353+0000","last_undegraded":"2024-04-24T23:10:37.007353+0000","last_fullsized":"2024-04-24T23:10:37.007353+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:03:26.361025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.001013+0000","last_change":"2024-04-24T23:10:37.000127+0000","last_active":"2024-04-24T23:10:37.001013+0000","last_peered":"2024-04-24T23:10:37.001013+0000","last_clean":"2024-04-24T23:10:37.001013+0000","last_became_active":"2024-04-24T23:10:36.999864+0000","last_became_peered":"2024-04-24T23:10:36.999864+0000","last_unstale":"2024-04-24T23:10:37.001013+0000","last_undegraded":"2024-04-24T23:10:37.001013+0000","last_fullsized":"2024-04-24T23:10:37.001013+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:12:04.154083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.004671+0000","last_change":"2024-04-24T23:10:37.004555+0000","last_active":"2024-04-24T23:10:37.004671+0000","last_peered":"2024-04-24T23:10:37.004671+0000","last_clean":"2024-04-24T23:10:37.004671+0000","last_became_active":"2024-04-24T23:10:37.004124+0000","last_became_peered":"2024-04-24T23:10:37.004124+0000","last_unstale":"2024-04-24T23:10:37.004671+0000","last_undegraded":"2024-04-24T23:10:37.004671+0000","last_fullsized":"2024-04-24T23:10:37.004671+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:02:52.094502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.010007+0000","last_change":"2024-04-24T23:10:37.009886+0000","last_active":"2024-04-24T23:10:37.010007+0000","last_peered":"2024-04-24T23:10:37.010007+0000","last_clean":"2024-04-24T23:10:37.010007+0000","last_became_active":"2024-04-24T23:10:37.009659+0000","last_became_peered":"2024-04-24T23:10:37.009659+0000","last_unstale":"2024-04-24T23:10:37.010007+0000","last_undegraded":"2024-04-24T23:10:37.010007+0000","last_fullsized":"2024-04-24T23:10:37.010007+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:33:17.919519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.002308+0000","last_change":"2024-04-24T23:10:37.002178+0000","last_active":"2024-04-24T23:10:37.002308+0000","last_peered":"2024-04-24T23:10:37.002308+0000","last_clean":"2024-04-24T23:10:37.002308+0000","last_became_active":"2024-04-24T23:10:37.001712+0000","last_became_peered":"2024-04-24T23:10:37.001712+0000","last_unstale":"2024-04-24T23:10:37.002308+0000","last_undegraded":"2024-04-24T23:10:37.002308+0000","last_fullsized":"2024-04-24T23:10:37.002308+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:57.343615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.013180+0000","last_change":"2024-04-24T23:10:37.011598+0000","last_active":"2024-04-24T23:10:37.013180+0000","last_peered":"2024-04-24T23:10:37.013180+0000","last_clean":"2024-04-24T23:10:37.013180+0000","last_became_active":"2024-04-24T23:10:37.010555+0000","last_became_peered":"2024-04-24T23:10:37.010555+0000","last_unstale":"2024-04-24T23:10:37.013180+0000","last_undegraded":"2024-04-24T23:10:37.013180+0000","last_fullsized":"2024-04-24T23:10:37.013180+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:42:00.527747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.009958+0000","last_change":"2024-04-24T23:10:37.009610+0000","last_active":"2024-04-24T23:10:37.009958+0000","last_peered":"2024-04-24T23:10:37.009958+0000","last_clean":"2024-04-24T23:10:37.009958+0000","last_became_active":"2024-04-24T23:10:37.009093+0000","last_became_peered":"2024-04-24T23:10:37.009093+0000","last_unstale":"2024-04-24T23:10:37.009958+0000","last_undegraded":"2024-04-24T23:10:37.009958+0000","last_fullsized":"2024-04-24T23:10:37.009958+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:31:22.414206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1f","version":"184'4","reported_seq":16,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.015558+0000","last_change":"2024-04-24T23:10:35.999854+0000","last_active":"2024-04-24T23:10:37.015558+0000","last_peered":"2024-04-24T23:10:37.015558+0000","last_clean":"2024-04-24T23:10:37.015558+0000","last_became_active":"2024-04-24T23:10:35.999014+0000","last_became_peered":"2024-04-24T23:10:35.999014+0000","last_unstale":"2024-04-24T23:10:37.015558+0000","last_undegraded":"2024-04-24T23:10:37.015558+0000","last_fullsized":"2024-04-24T23:10:37.015558+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:43.248061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.001355+0000","last_change":"2024-04-24T23:10:36.999675+0000","last_active":"2024-04-24T23:10:37.001355+0000","last_peered":"2024-04-24T23:10:37.001355+0000","last_clean":"2024-04-24T23:10:37.001355+0000","last_became_active":"2024-04-24T23:10:36.999403+0000","last_became_peered":"2024-04-24T23:10:36.999403+0000","last_unstale":"2024-04-24T23:10:37.001355+0000","last_undegraded":"2024-04-24T23:10:37.001355+0000","last_fullsized":"2024-04-24T23:10:37.001355+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:57:03.693700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1e","version":"184'1","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.018293+0000","last_change":"2024-04-24T23:10:35.994643+0000","last_active":"2024-04-24T23:10:37.018293+0000","last_peered":"2024-04-24T23:10:37.018293+0000","last_clean":"2024-04-24T23:10:37.018293+0000","last_became_active":"2024-04-24T23:10:35.994386+0000","last_became_peered":"2024-04-24T23:10:35.994386+0000","last_unstale":"2024-04-24T23:10:37.018293+0000","last_undegraded":"2024-04-24T23:10:37.018293+0000","last_fullsized":"2024-04-24T23:10:37.018293+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:54:14.516810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.008066+0000","last_change":"2024-04-24T23:10:37.007736+0000","last_active":"2024-04-24T23:10:37.008066+0000","last_peered":"2024-04-24T23:10:37.008066+0000","last_clean":"2024-04-24T23:10:37.008066+0000","last_became_active":"2024-04-24T23:10:37.007470+0000","last_became_peered":"2024-04-24T23:10:37.007470+0000","last_unstale":"2024-04-24T23:10:37.008066+0000","last_undegraded":"2024-04-24T23:10:37.008066+0000","last_fullsized":"2024-04-24T23:10:37.008066+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:58:49.251613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1d","version":"184'2","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.017929+0000","last_change":"2024-04-24T23:10:35.991120+0000","last_active":"2024-04-24T23:10:37.017929+0000","last_peered":"2024-04-24T23:10:37.017929+0000","last_clean":"2024-04-24T23:10:37.017929+0000","last_became_active":"2024-04-24T23:10:35.990834+0000","last_became_peered":"2024-04-24T23:10:35.990834+0000","last_unstale":"2024-04-24T23:10:37.017929+0000","last_undegraded":"2024-04-24T23:10:37.017929+0000","last_fullsized":"2024-04-24T23:10:37.017929+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:25.979175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.009475+0000","last_change":"2024-04-24T23:10:37.009362+0000","last_active":"2024-04-24T23:10:37.009475+0000","last_peered":"2024-04-24T23:10:37.009475+0000","last_clean":"2024-04-24T23:10:37.009475+0000","last_became_active":"2024-04-24T23:10:37.009129+0000","last_became_peered":"2024-04-24T23:10:37.009129+0000","last_unstale":"2024-04-24T23:10:37.009475+0000","last_undegraded":"2024-04-24T23:10:37.009475+0000","last_fullsized":"2024-04-24T23:10:37.009475+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:39:07.039159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1c","version":"184'2","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.018553+0000","last_change":"2024-04-24T23:10:35.993193+0000","last_active":"2024-04-24T23:10:37.018553+0000","last_peered":"2024-04-24T23:10:37.018553+0000","last_clean":"2024-04-24T23:10:37.018553+0000","last_became_active":"2024-04-24T23:10:35.992755+0000","last_became_peered":"2024-04-24T23:10:35.992755+0000","last_unstale":"2024-04-24T23:10:37.018553+0000","last_undegraded":"2024-04-24T23:10:37.018553+0000","last_fullsized":"2024-04-24T23:10:37.018553+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:47:11.159920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.007820+0000","last_change":"2024-04-24T23:10:37.007712+0000","last_active":"2024-04-24T23:10:37.007820+0000","last_peered":"2024-04-24T23:10:37.007820+0000","last_clean":"2024-04-24T23:10:37.007820+0000","last_became_active":"2024-04-24T23:10:37.007484+0000","last_became_peered":"2024-04-24T23:10:37.007484+0000","last_unstale":"2024-04-24T23:10:37.007820+0000","last_undegraded":"2024-04-24T23:10:37.007820+0000","last_fullsized":"2024-04-24T23:10:37.007820+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:34:02.283055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1b","version":"184'1","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.012228+0000","last_change":"2024-04-24T23:10:35.997388+0000","last_active":"2024-04-24T23:10:37.012228+0000","last_peered":"2024-04-24T23:10:37.012228+0000","last_clean":"2024-04-24T23:10:37.012228+0000","last_became_active":"2024-04-24T23:10:35.997015+0000","last_became_peered":"2024-04-24T23:10:35.997015+0000","last_unstale":"2024-04-24T23:10:37.012228+0000","last_undegraded":"2024-04-24T23:10:37.012228+0000","last_fullsized":"2024-04-24T23:10:37.012228+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:13:45.188236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.013276+0000","last_change":"2024-04-24T23:10:37.012065+0000","last_active":"2024-04-24T23:10:37.013276+0000","last_peered":"2024-04-24T23:10:37.013276+0000","last_clean":"2024-04-24T23:10:37.013276+0000","last_became_active":"2024-04-24T23:10:37.010900+0000","last_became_peered":"2024-04-24T23:10:37.010900+0000","last_unstale":"2024-04-24T23:10:37.013276+0000","last_undegraded":"2024-04-24T23:10:37.013276+0000","last_fullsized":"2024-04-24T23:10:37.013276+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:50:31.616773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1a","version":"184'1","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.016561+0000","last_change":"2024-04-24T23:10:35.999978+0000","last_active":"2024-04-24T23:10:37.016561+0000","last_peered":"2024-04-24T23:10:37.016561+0000","last_clean":"2024-04-24T23:10:37.016561+0000","last_became_active":"2024-04-24T23:10:35.999727+0000","last_became_peered":"2024-04-24T23:10:35.999727+0000","last_unstale":"2024-04-24T23:10:37.016561+0000","last_undegraded":"2024-04-24T23:10:37.016561+0000","last_fullsized":"2024-04-24T23:10:37.016561+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:12.119593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.004113+0000","last_change":"2024-04-24T23:10:37.003815+0000","last_active":"2024-04-24T23:10:37.004113+0000","last_peered":"2024-04-24T23:10:37.004113+0000","last_clean":"2024-04-24T23:10:37.004113+0000","last_became_active":"2024-04-24T23:10:37.003211+0000","last_became_peered":"2024-04-24T23:10:37.003211+0000","last_unstale":"2024-04-24T23:10:37.004113+0000","last_undegraded":"2024-04-24T23:10:37.004113+0000","last_fullsized":"2024-04-24T23:10:37.004113+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:20.867936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:36.996680+0000","last_change":"2024-04-24T23:10:35.997843+0000","last_active":"2024-04-24T23:10:36.996680+0000","last_peered":"2024-04-24T23:10:36.996680+0000","last_clean":"2024-04-24T23:10:36.996680+0000","last_became_active":"2024-04-24T23:10:35.997332+0000","last_became_peered":"2024-04-24T23:10:35.997332+0000","last_unstale":"2024-04-24T23:10:36.996680+0000","last_undegraded":"2024-04-24T23:10:36.996680+0000","last_fullsized":"2024-04-24T23:10:36.996680+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:42:50.314310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.007074+0000","last_change":"2024-04-24T23:10:37.006964+0000","last_active":"2024-04-24T23:10:37.007074+0000","last_peered":"2024-04-24T23:10:37.007074+0000","last_clean":"2024-04-24T23:10:37.007074+0000","last_became_active":"2024-04-24T23:10:37.006728+0000","last_became_peered":"2024-04-24T23:10:37.006728+0000","last_unstale":"2024-04-24T23:10:37.007074+0000","last_undegraded":"2024-04-24T23:10:37.007074+0000","last_fullsized":"2024-04-24T23:10:37.007074+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:10:29.200164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:36.997332+0000","last_change":"2024-04-24T23:10:35.985591+0000","last_active":"2024-04-24T23:10:36.997332+0000","last_peered":"2024-04-24T23:10:36.997332+0000","last_clean":"2024-04-24T23:10:36.997332+0000","last_became_active":"2024-04-24T23:10:35.985337+0000","last_became_peered":"2024-04-24T23:10:35.985337+0000","last_unstale":"2024-04-24T23:10:36.997332+0000","last_undegraded":"2024-04-24T23:10:36.997332+0000","last_fullsized":"2024-04-24T23:10:36.997332+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:14.594260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.005856+0000","last_change":"2024-04-24T23:10:37.005672+0000","last_active":"2024-04-24T23:10:37.005856+0000","last_peered":"2024-04-24T23:10:37.005856+0000","last_clean":"2024-04-24T23:10:37.005856+0000","last_became_active":"2024-04-24T23:10:37.005401+0000","last_became_peered":"2024-04-24T23:10:37.005401+0000","last_unstale":"2024-04-24T23:10:37.005856+0000","last_undegraded":"2024-04-24T23:10:37.005856+0000","last_fullsized":"2024-04-24T23:10:37.005856+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:55:25.145785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.003796+0000","last_change":"2024-04-24T23:10:35.996000+0000","last_active":"2024-04-24T23:10:37.003796+0000","last_peered":"2024-04-24T23:10:37.003796+0000","last_clean":"2024-04-24T23:10:37.003796+0000","last_became_active":"2024-04-24T23:10:35.995715+0000","last_became_peered":"2024-04-24T23:10:35.995715+0000","last_unstale":"2024-04-24T23:10:37.003796+0000","last_undegraded":"2024-04-24T23:10:37.003796+0000","last_fullsized":"2024-04-24T23:10:37.003796+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:27:34.034027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.001756+0000","last_change":"2024-04-24T23:10:37.001632+0000","last_active":"2024-04-24T23:10:37.001756+0000","last_peered":"2024-04-24T23:10:37.001756+0000","last_clean":"2024-04-24T23:10:37.001756+0000","last_became_active":"2024-04-24T23:10:37.001364+0000","last_became_peered":"2024-04-24T23:10:37.001364+0000","last_unstale":"2024-04-24T23:10:37.001756+0000","last_undegraded":"2024-04-24T23:10:37.001756+0000","last_fullsized":"2024-04-24T23:10:37.001756+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:59.508332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.15","version":"184'3","reported_seq":15,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.018434+0000","last_change":"2024-04-24T23:10:35.994171+0000","last_active":"2024-04-24T23:10:37.018434+0000","last_peered":"2024-04-24T23:10:37.018434+0000","last_clean":"2024-04-24T23:10:37.018434+0000","last_became_active":"2024-04-24T23:10:35.993292+0000","last_became_peered":"2024-04-24T23:10:35.993292+0000","last_unstale":"2024-04-24T23:10:37.018434+0000","last_undegraded":"2024-04-24T23:10:37.018434+0000","last_fullsized":"2024-04-24T23:10:37.018434+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:35:21.260712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.005112+0000","last_change":"2024-04-24T23:10:37.004851+0000","last_active":"2024-04-24T23:10:37.005112+0000","last_peered":"2024-04-24T23:10:37.005112+0000","last_clean":"2024-04-24T23:10:37.005112+0000","last_became_active":"2024-04-24T23:10:37.003252+0000","last_became_peered":"2024-04-24T23:10:37.003252+0000","last_unstale":"2024-04-24T23:10:37.005112+0000","last_undegraded":"2024-04-24T23:10:37.005112+0000","last_fullsized":"2024-04-24T23:10:37.005112+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:35:59.820604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.14","version":"184'6","reported_seq":18,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.020761+0000","last_change":"2024-04-24T23:10:35.994209+0000","last_active":"2024-04-24T23:10:37.020761+0000","last_peered":"2024-04-24T23:10:37.020761+0000","last_clean":"2024-04-24T23:10:37.020761+0000","last_became_active":"2024-04-24T23:10:35.993951+0000","last_became_peered":"2024-04-24T23:10:35.993951+0000","last_unstale":"2024-04-24T23:10:37.020761+0000","last_undegraded":"2024-04-24T23:10:37.020761+0000","last_fullsized":"2024-04-24T23:10:37.020761+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:32:21.495029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.003016+0000","last_change":"2024-04-24T23:10:37.002884+0000","last_active":"2024-04-24T23:10:37.003016+0000","last_peered":"2024-04-24T23:10:37.003016+0000","last_clean":"2024-04-24T23:10:37.003016+0000","last_became_active":"2024-04-24T23:10:37.002227+0000","last_became_peered":"2024-04-24T23:10:37.002227+0000","last_unstale":"2024-04-24T23:10:37.003016+0000","last_undegraded":"2024-04-24T23:10:37.003016+0000","last_fullsized":"2024-04-24T23:10:37.003016+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:15.391781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"184'2","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.016833+0000","last_change":"2024-04-24T23:10:35.988868+0000","last_active":"2024-04-24T23:10:37.016833+0000","last_peered":"2024-04-24T23:10:37.016833+0000","last_clean":"2024-04-24T23:10:37.016833+0000","last_became_active":"2024-04-24T23:10:35.988583+0000","last_became_peered":"2024-04-24T23:10:35.988583+0000","last_unstale":"2024-04-24T23:10:37.016833+0000","last_undegraded":"2024-04-24T23:10:37.016833+0000","last_fullsized":"2024-04-24T23:10:37.016833+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:27.111193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.004095+0000","last_change":"2024-04-24T23:10:37.003518+0000","last_active":"2024-04-24T23:10:37.004095+0000","last_peered":"2024-04-24T23:10:37.004095+0000","last_clean":"2024-04-24T23:10:37.004095+0000","last_became_active":"2024-04-24T23:10:37.002874+0000","last_became_peered":"2024-04-24T23:10:37.002874+0000","last_unstale":"2024-04-24T23:10:37.004095+0000","last_undegraded":"2024-04-24T23:10:37.004095+0000","last_fullsized":"2024-04-24T23:10:37.004095+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:58.728830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.002886+0000","last_change":"2024-04-24T23:10:35.994364+0000","last_active":"2024-04-24T23:10:37.002886+0000","last_peered":"2024-04-24T23:10:37.002886+0000","last_clean":"2024-04-24T23:10:37.002886+0000","last_became_active":"2024-04-24T23:10:35.994132+0000","last_became_peered":"2024-04-24T23:10:35.994132+0000","last_unstale":"2024-04-24T23:10:37.002886+0000","last_undegraded":"2024-04-24T23:10:37.002886+0000","last_fullsized":"2024-04-24T23:10:37.002886+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:36.783109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.005091+0000","last_change":"2024-04-24T23:10:37.004961+0000","last_active":"2024-04-24T23:10:37.005091+0000","last_peered":"2024-04-24T23:10:37.005091+0000","last_clean":"2024-04-24T23:10:37.005091+0000","last_became_active":"2024-04-24T23:10:37.004696+0000","last_became_peered":"2024-04-24T23:10:37.004696+0000","last_unstale":"2024-04-24T23:10:37.005091+0000","last_undegraded":"2024-04-24T23:10:37.005091+0000","last_fullsized":"2024-04-24T23:10:37.005091+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:07.927715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:36.997185+0000","last_change":"2024-04-24T23:10:35.997166+0000","last_active":"2024-04-24T23:10:36.997185+0000","last_peered":"2024-04-24T23:10:36.997185+0000","last_clean":"2024-04-24T23:10:36.997185+0000","last_became_active":"2024-04-24T23:10:35.996865+0000","last_became_peered":"2024-04-24T23:10:35.996865+0000","last_unstale":"2024-04-24T23:10:36.997185+0000","last_undegraded":"2024-04-24T23:10:36.997185+0000","last_fullsized":"2024-04-24T23:10:36.997185+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:32:12.340301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.011683+0000","last_change":"2024-04-24T23:10:37.011217+0000","last_active":"2024-04-24T23:10:37.011683+0000","last_peered":"2024-04-24T23:10:37.011683+0000","last_clean":"2024-04-24T23:10:37.011683+0000","last_became_active":"2024-04-24T23:10:37.010877+0000","last_became_peered":"2024-04-24T23:10:37.010877+0000","last_unstale":"2024-04-24T23:10:37.011683+0000","last_undegraded":"2024-04-24T23:10:37.011683+0000","last_fullsized":"2024-04-24T23:10:37.011683+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:07:27.021813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.10","version":"184'8","reported_seq":20,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.019188+0000","last_change":"2024-04-24T23:10:35.993657+0000","last_active":"2024-04-24T23:10:37.019188+0000","last_peered":"2024-04-24T23:10:37.019188+0000","last_clean":"2024-04-24T23:10:37.019188+0000","last_became_active":"2024-04-24T23:10:35.993411+0000","last_became_peered":"2024-04-24T23:10:35.993411+0000","last_unstale":"2024-04-24T23:10:37.019188+0000","last_undegraded":"2024-04-24T23:10:37.019188+0000","last_fullsized":"2024-04-24T23:10:37.019188+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:16:59.608136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:36.999530+0000","last_change":"2024-04-24T23:10:36.999386+0000","last_active":"2024-04-24T23:10:36.999530+0000","last_peered":"2024-04-24T23:10:36.999530+0000","last_clean":"2024-04-24T23:10:36.999530+0000","last_became_active":"2024-04-24T23:10:36.999099+0000","last_became_peered":"2024-04-24T23:10:36.999099+0000","last_unstale":"2024-04-24T23:10:36.999530+0000","last_undegraded":"2024-04-24T23:10:36.999530+0000","last_fullsized":"2024-04-24T23:10:36.999530+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:16.757940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.f","version":"184'1","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.019033+0000","last_change":"2024-04-24T23:10:35.992022+0000","last_active":"2024-04-24T23:10:37.019033+0000","last_peered":"2024-04-24T23:10:37.019033+0000","last_clean":"2024-04-24T23:10:37.019033+0000","last_became_active":"2024-04-24T23:10:35.991766+0000","last_became_peered":"2024-04-24T23:10:35.991766+0000","last_unstale":"2024-04-24T23:10:37.019033+0000","last_undegraded":"2024-04-24T23:10:37.019033+0000","last_fullsized":"2024-04-24T23:10:37.019033+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:08.684645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.005023+0000","last_change":"2024-04-24T23:10:37.003252+0000","last_active":"2024-04-24T23:10:37.005023+0000","last_peered":"2024-04-24T23:10:37.005023+0000","last_clean":"2024-04-24T23:10:37.005023+0000","last_became_active":"2024-04-24T23:10:37.002995+0000","last_became_peered":"2024-04-24T23:10:37.002995+0000","last_unstale":"2024-04-24T23:10:37.005023+0000","last_undegraded":"2024-04-24T23:10:37.005023+0000","last_fullsized":"2024-04-24T23:10:37.005023+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:50:44.121167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.e","version":"184'2","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.015921+0000","last_change":"2024-04-24T23:10:35.997234+0000","last_active":"2024-04-24T23:10:37.015921+0000","last_peered":"2024-04-24T23:10:37.015921+0000","last_clean":"2024-04-24T23:10:37.015921+0000","last_became_active":"2024-04-24T23:10:35.996764+0000","last_became_peered":"2024-04-24T23:10:35.996764+0000","last_unstale":"2024-04-24T23:10:37.015921+0000","last_undegraded":"2024-04-24T23:10:37.015921+0000","last_fullsized":"2024-04-24T23:10:37.015921+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:24:22.875084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.009939+0000","last_change":"2024-04-24T23:10:37.009349+0000","last_active":"2024-04-24T23:10:37.009939+0000","last_peered":"2024-04-24T23:10:37.009939+0000","last_clean":"2024-04-24T23:10:37.009939+0000","last_became_active":"2024-04-24T23:10:37.008333+0000","last_became_peered":"2024-04-24T23:10:37.008333+0000","last_unstale":"2024-04-24T23:10:37.009939+0000","last_undegraded":"2024-04-24T23:10:37.009939+0000","last_fullsized":"2024-04-24T23:10:37.009939+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:27.286371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.d","version":"184'1","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.017311+0000","last_change":"2024-04-24T23:10:35.995970+0000","last_active":"2024-04-24T23:10:37.017311+0000","last_peered":"2024-04-24T23:10:37.017311+0000","last_clean":"2024-04-24T23:10:37.017311+0000","last_became_active":"2024-04-24T23:10:35.995720+0000","last_became_peered":"2024-04-24T23:10:35.995720+0000","last_unstale":"2024-04-24T23:10:37.017311+0000","last_undegraded":"2024-04-24T23:10:37.017311+0000","last_fullsized":"2024-04-24T23:10:37.017311+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:00:40.007853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.000420+0000","last_change":"2024-04-24T23:10:37.000295+0000","last_active":"2024-04-24T23:10:37.000420+0000","last_peered":"2024-04-24T23:10:37.000420+0000","last_clean":"2024-04-24T23:10:37.000420+0000","last_became_active":"2024-04-24T23:10:37.000003+0000","last_became_peered":"2024-04-24T23:10:37.000003+0000","last_unstale":"2024-04-24T23:10:37.000420+0000","last_undegraded":"2024-04-24T23:10:37.000420+0000","last_fullsized":"2024-04-24T23:10:37.000420+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:51:03.078892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.000638+0000","last_change":"2024-04-24T23:10:35.992761+0000","last_active":"2024-04-24T23:10:37.000638+0000","last_peered":"2024-04-24T23:10:37.000638+0000","last_clean":"2024-04-24T23:10:37.000638+0000","last_became_active":"2024-04-24T23:10:35.992279+0000","last_became_peered":"2024-04-24T23:10:35.992279+0000","last_unstale":"2024-04-24T23:10:37.000638+0000","last_undegraded":"2024-04-24T23:10:37.000638+0000","last_fullsized":"2024-04-24T23:10:37.000638+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:28.574407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.007228+0000","last_change":"2024-04-24T23:10:37.007104+0000","last_active":"2024-04-24T23:10:37.007228+0000","last_peered":"2024-04-24T23:10:37.007228+0000","last_clean":"2024-04-24T23:10:37.007228+0000","last_became_active":"2024-04-24T23:10:37.006673+0000","last_became_peered":"2024-04-24T23:10:37.006673+0000","last_unstale":"2024-04-24T23:10:37.007228+0000","last_undegraded":"2024-04-24T23:10:37.007228+0000","last_fullsized":"2024-04-24T23:10:37.007228+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:22:53.848440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.b","version":"184'2","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.019449+0000","last_change":"2024-04-24T23:10:35.994431+0000","last_active":"2024-04-24T23:10:37.019449+0000","last_peered":"2024-04-24T23:10:37.019449+0000","last_clean":"2024-04-24T23:10:37.019449+0000","last_became_active":"2024-04-24T23:10:35.993715+0000","last_became_peered":"2024-04-24T23:10:35.993715+0000","last_unstale":"2024-04-24T23:10:37.019449+0000","last_undegraded":"2024-04-24T23:10:37.019449+0000","last_fullsized":"2024-04-24T23:10:37.019449+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:46:01.336615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.006798+0000","last_change":"2024-04-24T23:10:37.006673+0000","last_active":"2024-04-24T23:10:37.006798+0000","last_peered":"2024-04-24T23:10:37.006798+0000","last_clean":"2024-04-24T23:10:37.006798+0000","last_became_active":"2024-04-24T23:10:37.005912+0000","last_became_peered":"2024-04-24T23:10:37.005912+0000","last_unstale":"2024-04-24T23:10:37.006798+0000","last_undegraded":"2024-04-24T23:10:37.006798+0000","last_fullsized":"2024-04-24T23:10:37.006798+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:12:40.140042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:36.999798+0000","last_change":"2024-04-24T23:10:35.991747+0000","last_active":"2024-04-24T23:10:36.999798+0000","last_peered":"2024-04-24T23:10:36.999798+0000","last_clean":"2024-04-24T23:10:36.999798+0000","last_became_active":"2024-04-24T23:10:35.991462+0000","last_became_peered":"2024-04-24T23:10:35.991462+0000","last_unstale":"2024-04-24T23:10:36.999798+0000","last_undegraded":"2024-04-24T23:10:36.999798+0000","last_fullsized":"2024-04-24T23:10:36.999798+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:41:26.919041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.001479+0000","last_change":"2024-04-24T23:10:37.001155+0000","last_active":"2024-04-24T23:10:37.001479+0000","last_peered":"2024-04-24T23:10:37.001479+0000","last_clean":"2024-04-24T23:10:37.001479+0000","last_became_active":"2024-04-24T23:10:37.000633+0000","last_became_peered":"2024-04-24T23:10:37.000633+0000","last_unstale":"2024-04-24T23:10:37.001479+0000","last_undegraded":"2024-04-24T23:10:37.001479+0000","last_fullsized":"2024-04-24T23:10:37.001479+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:06:15.691725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.005354+0000","last_change":"2024-04-24T23:10:35.993906+0000","last_active":"2024-04-24T23:10:37.005354+0000","last_peered":"2024-04-24T23:10:37.005354+0000","last_clean":"2024-04-24T23:10:37.005354+0000","last_became_active":"2024-04-24T23:10:35.993660+0000","last_became_peered":"2024-04-24T23:10:35.993660+0000","last_unstale":"2024-04-24T23:10:37.005354+0000","last_undegraded":"2024-04-24T23:10:37.005354+0000","last_fullsized":"2024-04-24T23:10:37.005354+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:26:18.754981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.008658+0000","last_change":"2024-04-24T23:10:37.008472+0000","last_active":"2024-04-24T23:10:37.008658+0000","last_peered":"2024-04-24T23:10:37.008658+0000","last_clean":"2024-04-24T23:10:37.008658+0000","last_became_active":"2024-04-24T23:10:37.005340+0000","last_became_peered":"2024-04-24T23:10:37.005340+0000","last_unstale":"2024-04-24T23:10:37.008658+0000","last_undegraded":"2024-04-24T23:10:37.008658+0000","last_fullsized":"2024-04-24T23:10:37.008658+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:13:06.121421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:36.999405+0000","last_change":"2024-04-24T23:10:35.993107+0000","last_active":"2024-04-24T23:10:36.999405+0000","last_peered":"2024-04-24T23:10:36.999405+0000","last_clean":"2024-04-24T23:10:36.999405+0000","last_became_active":"2024-04-24T23:10:35.992154+0000","last_became_peered":"2024-04-24T23:10:35.992154+0000","last_unstale":"2024-04-24T23:10:36.999405+0000","last_undegraded":"2024-04-24T23:10:36.999405+0000","last_fullsized":"2024-04-24T23:10:36.999405+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:25:37.239074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.002443+0000","last_change":"2024-04-24T23:10:37.002111+0000","last_active":"2024-04-24T23:10:37.002443+0000","last_peered":"2024-04-24T23:10:37.002443+0000","last_clean":"2024-04-24T23:10:37.002443+0000","last_became_active":"2024-04-24T23:10:37.001191+0000","last_became_peered":"2024-04-24T23:10:37.001191+0000","last_unstale":"2024-04-24T23:10:37.002443+0000","last_undegraded":"2024-04-24T23:10:37.002443+0000","last_fullsized":"2024-04-24T23:10:37.002443+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:03:20.261932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.7","version":"184'2","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.015509+0000","last_change":"2024-04-24T23:10:35.994324+0000","last_active":"2024-04-24T23:10:37.015509+0000","last_peered":"2024-04-24T23:10:37.015509+0000","last_clean":"2024-04-24T23:10:37.015509+0000","last_became_active":"2024-04-24T23:10:35.993636+0000","last_became_peered":"2024-04-24T23:10:35.993636+0000","last_unstale":"2024-04-24T23:10:37.015509+0000","last_undegraded":"2024-04-24T23:10:37.015509+0000","last_fullsized":"2024-04-24T23:10:37.015509+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:35:43.319355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.011773+0000","last_change":"2024-04-24T23:10:37.011651+0000","last_active":"2024-04-24T23:10:37.011773+0000","last_peered":"2024-04-24T23:10:37.011773+0000","last_clean":"2024-04-24T23:10:37.011773+0000","last_became_active":"2024-04-24T23:10:37.011357+0000","last_became_peered":"2024-04-24T23:10:37.011357+0000","last_unstale":"2024-04-24T23:10:37.011773+0000","last_undegraded":"2024-04-24T23:10:37.011773+0000","last_fullsized":"2024-04-24T23:10:37.011773+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:40.903388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.001590+0000","last_change":"2024-04-24T23:10:36.000285+0000","last_active":"2024-04-24T23:10:37.001590+0000","last_peered":"2024-04-24T23:10:37.001590+0000","last_clean":"2024-04-24T23:10:37.001590+0000","last_became_active":"2024-04-24T23:10:35.999990+0000","last_became_peered":"2024-04-24T23:10:35.999990+0000","last_unstale":"2024-04-24T23:10:37.001590+0000","last_undegraded":"2024-04-24T23:10:37.001590+0000","last_fullsized":"2024-04-24T23:10:37.001590+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:14:36.496334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.007177+0000","last_change":"2024-04-24T23:10:37.007038+0000","last_active":"2024-04-24T23:10:37.007177+0000","last_peered":"2024-04-24T23:10:37.007177+0000","last_clean":"2024-04-24T23:10:37.007177+0000","last_became_active":"2024-04-24T23:10:37.006653+0000","last_became_peered":"2024-04-24T23:10:37.006653+0000","last_unstale":"2024-04-24T23:10:37.007177+0000","last_undegraded":"2024-04-24T23:10:37.007177+0000","last_fullsized":"2024-04-24T23:10:37.007177+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:08:20.983386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:36.997902+0000","last_change":"2024-04-24T23:10:35.998896+0000","last_active":"2024-04-24T23:10:36.997902+0000","last_peered":"2024-04-24T23:10:36.997902+0000","last_clean":"2024-04-24T23:10:36.997902+0000","last_became_active":"2024-04-24T23:10:35.998652+0000","last_became_peered":"2024-04-24T23:10:35.998652+0000","last_unstale":"2024-04-24T23:10:36.997902+0000","last_undegraded":"2024-04-24T23:10:36.997902+0000","last_fullsized":"2024-04-24T23:10:36.997902+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:27:53.829534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.012552+0000","last_change":"2024-04-24T23:10:37.012136+0000","last_active":"2024-04-24T23:10:37.012552+0000","last_peered":"2024-04-24T23:10:37.012552+0000","last_clean":"2024-04-24T23:10:37.012552+0000","last_became_active":"2024-04-24T23:10:37.011826+0000","last_became_peered":"2024-04-24T23:10:37.011826+0000","last_unstale":"2024-04-24T23:10:37.012552+0000","last_undegraded":"2024-04-24T23:10:37.012552+0000","last_fullsized":"2024-04-24T23:10:37.012552+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:44.366747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:36.997478+0000","last_change":"2024-04-24T23:10:35.992950+0000","last_active":"2024-04-24T23:10:36.997478+0000","last_peered":"2024-04-24T23:10:36.997478+0000","last_clean":"2024-04-24T23:10:36.997478+0000","last_became_active":"2024-04-24T23:10:35.992642+0000","last_became_peered":"2024-04-24T23:10:35.992642+0000","last_unstale":"2024-04-24T23:10:36.997478+0000","last_undegraded":"2024-04-24T23:10:36.997478+0000","last_fullsized":"2024-04-24T23:10:36.997478+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:15:04.388968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.013335+0000","last_change":"2024-04-24T23:10:37.012259+0000","last_active":"2024-04-24T23:10:37.013335+0000","last_peered":"2024-04-24T23:10:37.013335+0000","last_clean":"2024-04-24T23:10:37.013335+0000","last_became_active":"2024-04-24T23:10:37.011297+0000","last_became_peered":"2024-04-24T23:10:37.011297+0000","last_unstale":"2024-04-24T23:10:37.013335+0000","last_undegraded":"2024-04-24T23:10:37.013335+0000","last_fullsized":"2024-04-24T23:10:37.013335+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:18:13.287164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.000121+0000","last_change":"2024-04-24T23:10:35.991426+0000","last_active":"2024-04-24T23:10:37.000121+0000","last_peered":"2024-04-24T23:10:37.000121+0000","last_clean":"2024-04-24T23:10:37.000121+0000","last_became_active":"2024-04-24T23:10:35.990882+0000","last_became_peered":"2024-04-24T23:10:35.990882+0000","last_unstale":"2024-04-24T23:10:37.000121+0000","last_undegraded":"2024-04-24T23:10:37.000121+0000","last_fullsized":"2024-04-24T23:10:37.000121+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:11:29.917193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.001375+0000","last_change":"2024-04-24T23:10:37.000966+0000","last_active":"2024-04-24T23:10:37.001375+0000","last_peered":"2024-04-24T23:10:37.001375+0000","last_clean":"2024-04-24T23:10:37.001375+0000","last_became_active":"2024-04-24T23:10:37.000311+0000","last_became_peered":"2024-04-24T23:10:37.000311+0000","last_unstale":"2024-04-24T23:10:37.001375+0000","last_undegraded":"2024-04-24T23:10:37.001375+0000","last_fullsized":"2024-04-24T23:10:37.001375+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:28:03.147405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:36.997401+0000","last_change":"2024-04-24T23:10:35.991371+0000","last_active":"2024-04-24T23:10:36.997401+0000","last_peered":"2024-04-24T23:10:36.997401+0000","last_clean":"2024-04-24T23:10:36.997401+0000","last_became_active":"2024-04-24T23:10:35.991125+0000","last_became_peered":"2024-04-24T23:10:35.991125+0000","last_unstale":"2024-04-24T23:10:36.997401+0000","last_undegraded":"2024-04-24T23:10:36.997401+0000","last_fullsized":"2024-04-24T23:10:36.997401+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:38:36.911527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.009165+0000","last_change":"2024-04-24T23:10:37.009050+0000","last_active":"2024-04-24T23:10:37.009165+0000","last_peered":"2024-04-24T23:10:37.009165+0000","last_clean":"2024-04-24T23:10:37.009165+0000","last_became_active":"2024-04-24T23:10:37.008799+0000","last_became_peered":"2024-04-24T23:10:37.008799+0000","last_unstale":"2024-04-24T23:10:37.009165+0000","last_undegraded":"2024-04-24T23:10:37.009165+0000","last_fullsized":"2024-04-24T23:10:37.009165+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:56:27.011581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:36.998492+0000","last_change":"2024-04-24T23:10:36.000319+0000","last_active":"2024-04-24T23:10:36.998492+0000","last_peered":"2024-04-24T23:10:36.998492+0000","last_clean":"2024-04-24T23:10:36.998492+0000","last_became_active":"2024-04-24T23:10:35.999236+0000","last_became_peered":"2024-04-24T23:10:35.999236+0000","last_unstale":"2024-04-24T23:10:36.998492+0000","last_undegraded":"2024-04-24T23:10:36.998492+0000","last_fullsized":"2024-04-24T23:10:36.998492+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:50:26.335560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.002060+0000","last_change":"2024-04-24T23:10:37.001779+0000","last_active":"2024-04-24T23:10:37.002060+0000","last_peered":"2024-04-24T23:10:37.002060+0000","last_clean":"2024-04-24T23:10:37.002060+0000","last_became_active":"2024-04-24T23:10:37.001547+0000","last_became_peered":"2024-04-24T23:10:37.001547+0000","last_unstale":"2024-04-24T23:10:37.002060+0000","last_undegraded":"2024-04-24T23:10:37.002060+0000","last_fullsized":"2024-04-24T23:10:37.002060+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:47.163526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:36.998136+0000","last_change":"2024-04-24T23:10:35.997601+0000","last_active":"2024-04-24T23:10:36.998136+0000","last_peered":"2024-04-24T23:10:36.998136+0000","last_clean":"2024-04-24T23:10:36.998136+0000","last_became_active":"2024-04-24T23:10:35.996990+0000","last_became_peered":"2024-04-24T23:10:35.996990+0000","last_unstale":"2024-04-24T23:10:36.998136+0000","last_undegraded":"2024-04-24T23:10:36.998136+0000","last_fullsized":"2024-04-24T23:10:36.998136+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:54:26.006604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.010351+0000","last_change":"2024-04-24T23:10:37.010222+0000","last_active":"2024-04-24T23:10:37.010351+0000","last_peered":"2024-04-24T23:10:37.010351+0000","last_clean":"2024-04-24T23:10:37.010351+0000","last_became_active":"2024-04-24T23:10:37.009699+0000","last_became_peered":"2024-04-24T23:10:37.009699+0000","last_unstale":"2024-04-24T23:10:37.010351+0000","last_undegraded":"2024-04-24T23:10:37.010351+0000","last_fullsized":"2024-04-24T23:10:37.010351+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:35.979581+0000","last_clean_scrub_stamp":"2024-04-24T23:10:35.979581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:24.214918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:36.999796+0000","last_change":"2024-04-24T23:10:35.991208+0000","last_active":"2024-04-24T23:10:36.999796+0000","last_peered":"2024-04-24T23:10:36.999796+0000","last_clean":"2024-04-24T23:10:36.999796+0000","last_became_active":"2024-04-24T23:10:35.990551+0000","last_became_peered":"2024-04-24T23:10:35.990551+0000","last_unstale":"2024-04-24T23:10:36.999796+0000","last_undegraded":"2024-04-24T23:10:36.999796+0000","last_fullsized":"2024-04-24T23:10:36.999796+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:34.973494+0000","last_clean_scrub_stamp":"2024-04-24T23:10:34.973494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:29:38.358276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":494,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-24T23:10:37.001116+0000","last_change":"2024-04-24T22:58:55.138810+0000","last_active":"2024-04-24T23:10:37.001116+0000","last_peered":"2024-04-24T23:10:37.001116+0000","last_clean":"2024-04-24T23:10:37.001116+0000","last_became_active":"2024-04-24T22:58:55.138627+0000","last_became_peered":"2024-04-24T22:58:55.138627+0000","last_unstale":"2024-04-24T23:10:37.001116+0000","last_undegraded":"2024-04-24T23:10:37.001116+0000","last_fullsized":"2024-04-24T23:10:37.001116+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_clean_scrub_stamp":"2024-04-24T22:58:54.127235+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:51.383182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574991,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1005,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194307,"statfs":{"total":200000000,"available":198970585,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574991,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1005,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194307,"statfs":{"total":200000000,"available":198970585,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574991,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1581,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193730,"statfs":{"total":200000000,"available":198380217,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574991,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1005,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194307,"statfs":{"total":200000000,"available":198970585,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574991,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1005,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194307,"statfs":{"total":200000000,"available":198970585,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574991,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1005,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194307,"statfs":{"total":200000000,"available":198970585,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574991,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1581,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193730,"statfs":{"total":200000000,"available":198380217,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574991,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1005,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194307,"statfs":{"total":200000000,"available":198970585,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-24T23:10:40.360 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-24T23:10:40.360 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-24T23:10:40.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.501+0000 7f177c341640 1 Processor -- start 2024-04-24T23:10:40.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.501+0000 7f177c341640 1 -- start start 2024-04-24T23:10:40.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.503+0000 7f177c341640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1774074210 0x7f177406c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:40.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.503+0000 7f177c341640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f17740ae140 0x7f17740b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:40.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.503+0000 7f177c341640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17740b0770 con 0x7f17740b0b80 2024-04-24T23:10:40.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.503+0000 7f177c341640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f17740b1c20 con 0x7f17740ae140 2024-04-24T23:10:40.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.503+0000 7f177c341640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f17740b1010 con 0x7f1774074210 2024-04-24T23:10:40.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.503+0000 7f17798dc640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f17740b0b80 0x7f17740745f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58606/0 (socket says 172.21.15.32:58606) 2024-04-24T23:10:40.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.503+0000 7f17798dc640 1 -- 172.21.15.32:0/534795353 learned_addr learned my addr 172.21.15.32:0/534795353 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:10:40.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.503+0000 7f177a0dd640 1 --2- 172.21.15.32:0/534795353 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1774074210 0x7f177406c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:40.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.503+0000 7f177a8de640 1 --2- 172.21.15.32:0/534795353 >> v2:172.21.15.121:3300/0 conn(0x7f17740ae140 0x7f17740b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:40.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.503+0000 7f17790db640 1 -- 172.21.15.32:0/534795353 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1369395861 0 0) 0x7f17740b0770 con 0x7f17740b0b80 2024-04-24T23:10:40.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.503+0000 7f17790db640 1 -- 172.21.15.32:0/534795353 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f175c003520 con 0x7f17740b0b80 2024-04-24T23:10:40.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.503+0000 7f17790db640 1 -- 172.21.15.32:0/534795353 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1784036874 0 0) 0x7f175c003520 con 0x7f17740b0b80 2024-04-24T23:10:40.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.503+0000 7f17790db640 1 -- 172.21.15.32:0/534795353 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1774074210 msgr2=0x7f177406c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:40.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.503+0000 7f17790db640 1 --2- 172.21.15.32:0/534795353 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1774074210 0x7f177406c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.503+0000 7f17790db640 1 -- 172.21.15.32:0/534795353 >> v2:172.21.15.121:3300/0 conn(0x7f17740ae140 msgr2=0x7f17740b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:40.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.503+0000 7f17790db640 1 --2- 172.21.15.32:0/534795353 >> v2:172.21.15.121:3300/0 conn(0x7f17740ae140 0x7f17740b20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.503+0000 7f17790db640 1 -- 172.21.15.32:0/534795353 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17740b12f0 con 0x7f17740b0b80 2024-04-24T23:10:40.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.503+0000 7f17790db640 1 -- 172.21.15.32:0/534795353 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f1768001670 con 0x7f17740b0b80 2024-04-24T23:10:40.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.503+0000 7f17790db640 1 -- 172.21.15.32:0/534795353 >> v1:172.21.15.32:6789/0 conn(0x7f17740b0b80 legacy=0x7f17740745f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:40.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.503+0000 7f17790db640 1 --2- 172.21.15.32:0/534795353 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f175c0039b0 0x7f175c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:40.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.503+0000 7f17790db640 1 --2- 172.21.15.32:0/534795353 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1774074210 0x7f175c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:40.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.503+0000 7f17790db640 1 --2- 172.21.15.32:0/534795353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f175c004d40 0x7f175c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:40.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.503+0000 7f17790db640 1 -- 172.21.15.32:0/534795353 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f17740b1c20 con 0x7f175c004d40 2024-04-24T23:10:40.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.503+0000 7f17790db640 1 -- 172.21.15.32:0/534795353 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f17740b1010 con 0x7f175c0039b0 2024-04-24T23:10:40.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.503+0000 7f17790db640 1 -- 172.21.15.32:0/534795353 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f175c00b9c0 con 0x7f1774074210 2024-04-24T23:10:40.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.503+0000 7f177a8de640 1 --2- 172.21.15.32:0/534795353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f175c004d40 0x7f175c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:40.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.503+0000 7f17798dc640 1 --2- 172.21.15.32:0/534795353 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1774074210 0x7f175c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:40.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.503+0000 7f177a0dd640 1 --2- 172.21.15.32:0/534795353 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f175c0039b0 0x7f175c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:40.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.503+0000 7f177a8de640 1 -- 172.21.15.32:0/534795353 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1774074210 msgr2=0x7f175c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:40.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.503+0000 7f177a8de640 1 --2- 172.21.15.32:0/534795353 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1774074210 0x7f175c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.503+0000 7f177a8de640 1 -- 172.21.15.32:0/534795353 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f175c0039b0 msgr2=0x7f175c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:40.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.503+0000 7f177a8de640 1 --2- 172.21.15.32:0/534795353 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f175c0039b0 0x7f175c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.503+0000 7f177a8de640 1 -- 172.21.15.32:0/534795353 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f175c003520 con 0x7f175c004d40 2024-04-24T23:10:40.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.504+0000 7f177a8de640 1 --2- 172.21.15.32:0/534795353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f175c004d40 0x7f175c0051c0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f1764009fc0 tx=0x7f175c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e67ee03f8d710f6c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:40.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.504+0000 7f17790db640 1 -- 172.21.15.32:0/534795353 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1764009c10 con 0x7f175c004d40 2024-04-24T23:10:40.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.504+0000 7f17790db640 1 -- 172.21.15.32:0/534795353 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1764036070 con 0x7f175c004d40 2024-04-24T23:10:40.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.504+0000 7f17790db640 1 -- 172.21.15.32:0/534795353 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176402d070 con 0x7f175c004d40 2024-04-24T23:10:40.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.505+0000 7f177c341640 1 -- 172.21.15.32:0/534795353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f175c004d40 msgr2=0x7f175c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:40.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.505+0000 7f177c341640 1 --2- 172.21.15.32:0/534795353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f175c004d40 0x7f175c0051c0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f1764009fc0 tx=0x7f175c005830 comp rx=0 tx=0).stop 2024-04-24T23:10:40.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.505+0000 7f177a8de640 1 -- 172.21.15.32:0/534795353 reap_dead start 2024-04-24T23:10:40.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.505+0000 7f177c341640 1 -- 172.21.15.32:0/534795353 shutdown_connections 2024-04-24T23:10:40.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.505+0000 7f177c341640 1 -- 172.21.15.32:0/534795353 >> 172.21.15.32:0/534795353 conn(0x7f17740b2e60 msgr2=0x7f17740b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:40.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.505+0000 7f177c341640 1 -- 172.21.15.32:0/534795353 shutdown_connections 2024-04-24T23:10:40.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.505+0000 7f177c341640 1 -- 172.21.15.32:0/534795353 wait complete. 2024-04-24T23:10:40.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.507+0000 7f177c341640 1 Processor -- start 2024-04-24T23:10:40.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.507+0000 7f177c341640 1 -- start start 2024-04-24T23:10:40.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.508+0000 7f177c341640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f175c004d40 0x7f17741f75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:40.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.508+0000 7f177c341640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f17740c7ea0 0x7f1774159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:40.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.508+0000 7f177c341640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17740d11d0 0x7f177415a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:40.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.508+0000 7f177c341640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f17740b3ba0 con 0x7f17740d11d0 2024-04-24T23:10:40.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.508+0000 7f177c341640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f17740b18b0 con 0x7f17740c7ea0 2024-04-24T23:10:40.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.508+0000 7f177c341640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f17740b3d20 con 0x7f175c004d40 2024-04-24T23:10:40.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.508+0000 7f177a8de640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17740d11d0 0x7f177415a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:40.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.508+0000 7f177a8de640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17740d11d0 0x7f177415a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48754/0 (socket says 172.21.15.32:48754) 2024-04-24T23:10:40.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.508+0000 7f177a8de640 1 -- 172.21.15.32:0/2336743571 learned_addr learned my addr 172.21.15.32:0/2336743571 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:40.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.508+0000 7f177a0dd640 1 --2- 172.21.15.32:0/2336743571 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f175c004d40 0x7f17741f75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:40.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.508+0000 7f17798dc640 1 --2- 172.21.15.32:0/2336743571 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f17740c7ea0 0x7f1774159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:40.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.508+0000 7f177a8de640 1 -- 172.21.15.32:0/2336743571 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f175c004d40 msgr2=0x7f17741f75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:40.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.508+0000 7f177a8de640 1 --2- 172.21.15.32:0/2336743571 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f175c004d40 0x7f17741f75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.508+0000 7f177a8de640 1 -- 172.21.15.32:0/2336743571 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f17740c7ea0 msgr2=0x7f1774159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:40.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.508+0000 7f177a8de640 1 --2- 172.21.15.32:0/2336743571 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f17740c7ea0 0x7f1774159fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.508+0000 7f177a8de640 1 -- 172.21.15.32:0/2336743571 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17740c6e00 con 0x7f17740d11d0 2024-04-24T23:10:40.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.508+0000 7f17798dc640 1 --2- 172.21.15.32:0/2336743571 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f17740c7ea0 0x7f1774159fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:10:40.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.508+0000 7f177a8de640 1 --2- 172.21.15.32:0/2336743571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17740d11d0 0x7f177415a4f0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f1764033040 tx=0x7f1764009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:40.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.508+0000 7f1772ffd640 1 -- 172.21.15.32:0/2336743571 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1764009c10 con 0x7f17740d11d0 2024-04-24T23:10:40.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.508+0000 7f1772ffd640 1 -- 172.21.15.32:0/2336743571 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1764003bb0 con 0x7f17740d11d0 2024-04-24T23:10:40.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.508+0000 7f177c341640 1 -- 172.21.15.32:0/2336743571 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17740c70e0 con 0x7f17740d11d0 2024-04-24T23:10:40.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.508+0000 7f1772ffd640 1 -- 172.21.15.32:0/2336743571 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1764004020 con 0x7f17740d11d0 2024-04-24T23:10:40.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.508+0000 7f177c341640 1 -- 172.21.15.32:0/2336743571 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17740c76a0 con 0x7f17740d11d0 2024-04-24T23:10:40.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.508+0000 7f177c341640 1 -- 172.21.15.32:0/2336743571 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1774068430 con 0x7f17740d11d0 2024-04-24T23:10:40.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.509+0000 7f1772ffd640 1 -- 172.21.15.32:0/2336743571 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1764005d10 con 0x7f17740d11d0 2024-04-24T23:10:40.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.509+0000 7f1772ffd640 1 --2- 172.21.15.32:0/2336743571 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f176006be30 0x7f176006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:40.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.509+0000 7f1772ffd640 1 -- 172.21.15.32:0/2336743571 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(185..185 src has 1..185) ==== 5709+0+0 (secure 0 0 0) 0x7f17640ac490 con 0x7f17740d11d0 2024-04-24T23:10:40.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.510+0000 7f177a0dd640 1 --2- 172.21.15.32:0/2336743571 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f176006be30 0x7f176006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:40.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.510+0000 7f177a0dd640 1 --2- 172.21.15.32:0/2336743571 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f176006be30 0x7f176006e2f0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f176c009b40 tx=0x7f176c009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:40.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.515+0000 7f1772ffd640 1 -- 172.21.15.32:0/2336743571 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f176404a060 con 0x7f17740d11d0 2024-04-24T23:10:40.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.766+0000 7f177c341640 1 -- 172.21.15.32:0/2336743571 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f177406b220 con 0x7f17740d11d0 2024-04-24T23:10:40.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.768+0000 7f1772ffd640 1 -- 172.21.15.32:0/2336743571 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v23) ==== 190+0+147 (secure 0 0 0) 0x7f176407ada0 con 0x7f17740d11d0 2024-04-24T23:10:40.770 INFO:teuthology.orchestra.run.smithi032.stdout:[client.0] 2024-04-24T23:10:40.771 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQCejilm5oeiGRAAdLXwwqrYhV3HHo7Dw/2ZqQ== 2024-04-24T23:10:40.771 INFO:teuthology.orchestra.run.smithi032.stdout: caps mds = "allow" 2024-04-24T23:10:40.771 INFO:teuthology.orchestra.run.smithi032.stdout: caps mon = "allow r" 2024-04-24T23:10:40.771 INFO:teuthology.orchestra.run.smithi032.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-24T23:10:40.771 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2024-04-24T23:10:40.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.770+0000 7f177c341640 1 -- 172.21.15.32:0/2336743571 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f176006be30 msgr2=0x7f176006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:40.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.770+0000 7f177c341640 1 --2- 172.21.15.32:0/2336743571 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f176006be30 0x7f176006e2f0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f176c009b40 tx=0x7f176c009190 comp rx=0 tx=0).stop 2024-04-24T23:10:40.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.770+0000 7f177c341640 1 -- 172.21.15.32:0/2336743571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17740d11d0 msgr2=0x7f177415a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:40.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.770+0000 7f177c341640 1 --2- 172.21.15.32:0/2336743571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17740d11d0 0x7f177415a4f0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f1764033040 tx=0x7f1764009750 comp rx=0 tx=0).stop 2024-04-24T23:10:40.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.770+0000 7f177c341640 1 -- 172.21.15.32:0/2336743571 shutdown_connections 2024-04-24T23:10:40.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.770+0000 7f177c341640 1 --2- 172.21.15.32:0/2336743571 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f176006be30 0x7f176006e2f0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.770+0000 7f177c341640 1 --2- 172.21.15.32:0/2336743571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17740d11d0 0x7f177415a4f0 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.770+0000 7f177c341640 1 --2- 172.21.15.32:0/2336743571 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f17740c7ea0 0x7f1774159fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.770+0000 7f177c341640 1 --2- 172.21.15.32:0/2336743571 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f175c004d40 0x7f17741f75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.770+0000 7f177c341640 1 -- 172.21.15.32:0/2336743571 >> 172.21.15.32:0/2336743571 conn(0x7f17740b2e60 msgr2=0x7f177406d570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:40.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.770+0000 7f177c341640 1 -- 172.21.15.32:0/2336743571 shutdown_connections 2024-04-24T23:10:40.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.770+0000 7f177c341640 1 -- 172.21.15.32:0/2336743571 wait complete. 2024-04-24T23:10:40.783 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:10:40.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.883+0000 7f45631b7640 1 Processor -- start 2024-04-24T23:10:40.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.883+0000 7f45631b7640 1 -- start start 2024-04-24T23:10:40.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.883+0000 7f45631b7640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f455c15b480 0x7f455c0b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:40.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.883+0000 7f45631b7640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f455c0b46c0 con 0x7f455c15fd60 2024-04-24T23:10:40.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.883+0000 7f45631b7640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f455c0b49a0 con 0x7f455c06cd00 2024-04-24T23:10:40.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.883+0000 7f45631b7640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f455c0b4ff0 con 0x7f455c15b480 2024-04-24T23:10:40.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.883+0000 7f4560f53640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f455c15b480 0x7f455c0b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:40.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.883+0000 7f455bfff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f455c15fd60 0x7f455c06c5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58620/0 (socket says 172.21.15.32:58620) 2024-04-24T23:10:40.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.883+0000 7f455bfff640 1 -- 172.21.15.32:0/1384482872 learned_addr learned my addr 172.21.15.32:0/1384482872 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:10:40.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.883+0000 7f4560f53640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f455c15b480 0x7f455c0b5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48762/0 (socket says 172.21.15.32:48762) 2024-04-24T23:10:40.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.883+0000 7f4560f53640 1 -- 172.21.15.32:0/1384482872 >> v1:172.21.15.121:6789/0 conn(0x7f455c06cd00 legacy=0x7f455c0cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:40.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.883+0000 7f4560f53640 1 -- 172.21.15.32:0/1384482872 >> v1:172.21.15.32:6789/0 conn(0x7f455c15fd60 legacy=0x7f455c06c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:40.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.883+0000 7f4560f53640 1 -- 172.21.15.32:0/1384482872 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f455c0b4c80 con 0x7f455c15b480 2024-04-24T23:10:40.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.883+0000 7f455b7fe640 1 -- 172.21.15.32:0/1384482872 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2818718972 0 0) 0x7f455c0b46c0 con 0x7f455c15fd60 2024-04-24T23:10:40.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.884+0000 7f4560f53640 1 --2- 172.21.15.32:0/1384482872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f455c15b480 0x7f455c0b5c60 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f455c0b5170 tx=0x7f455002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fa552673ef63120 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:40.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.884+0000 7f455b7fe640 1 -- 172.21.15.32:0/1384482872 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4550037070 con 0x7f455c15b480 2024-04-24T23:10:40.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.884+0000 7f455b7fe640 1 -- 172.21.15.32:0/1384482872 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4550032440 con 0x7f455c15b480 2024-04-24T23:10:40.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.884+0000 7f455b7fe640 1 -- 172.21.15.32:0/1384482872 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4550032810 con 0x7f455c15b480 2024-04-24T23:10:40.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.885+0000 7f45631b7640 1 -- 172.21.15.32:0/1384482872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f455c15b480 msgr2=0x7f455c0b5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:40.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.885+0000 7f45631b7640 1 --2- 172.21.15.32:0/1384482872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f455c15b480 0x7f455c0b5c60 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f455c0b5170 tx=0x7f455002a1c0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.885+0000 7f45631b7640 1 -- 172.21.15.32:0/1384482872 shutdown_connections 2024-04-24T23:10:40.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.885+0000 7f45631b7640 1 --2- 172.21.15.32:0/1384482872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f455c15b480 0x7f455c0b5c60 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.885+0000 7f45631b7640 1 -- 172.21.15.32:0/1384482872 >> 172.21.15.32:0/1384482872 conn(0x7f455c06b110 msgr2=0x7f455c06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:40.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.885+0000 7f45631b7640 1 -- 172.21.15.32:0/1384482872 shutdown_connections 2024-04-24T23:10:40.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.885+0000 7f45631b7640 1 -- 172.21.15.32:0/1384482872 wait complete. 2024-04-24T23:10:40.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.886+0000 7f45631b7640 1 Processor -- start 2024-04-24T23:10:40.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.886+0000 7f45631b7640 1 -- start start 2024-04-24T23:10:40.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.888+0000 7f45631b7640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f455c06cd00 0x7f455c15d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:40.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.888+0000 7f45631b7640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f455c15b480 0x7f455c159720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:40.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.888+0000 7f45631b7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f455c15fd60 0x7f455c154bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:40.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.888+0000 7f45631b7640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f455c15b860 con 0x7f455c15fd60 2024-04-24T23:10:40.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.888+0000 7f45631b7640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f455c0b5480 con 0x7f455c15b480 2024-04-24T23:10:40.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.888+0000 7f45631b7640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f455c15dbd0 con 0x7f455c06cd00 2024-04-24T23:10:40.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.888+0000 7f4561754640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f455c15fd60 0x7f455c154bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:40.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.888+0000 7f4561754640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f455c15fd60 0x7f455c154bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48768/0 (socket says 172.21.15.32:48768) 2024-04-24T23:10:40.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.888+0000 7f4561754640 1 -- 172.21.15.32:0/3223726374 learned_addr learned my addr 172.21.15.32:0/3223726374 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:40.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.888+0000 7f4560f53640 1 --2- 172.21.15.32:0/3223726374 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f455c06cd00 0x7f455c15d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:40.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.888+0000 7f455bfff640 1 --2- 172.21.15.32:0/3223726374 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f455c15b480 0x7f455c159720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:40.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.888+0000 7f4561754640 1 -- 172.21.15.32:0/3223726374 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f455c06cd00 msgr2=0x7f455c15d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:40.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.888+0000 7f4561754640 1 --2- 172.21.15.32:0/3223726374 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f455c06cd00 0x7f455c15d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.888+0000 7f4561754640 1 -- 172.21.15.32:0/3223726374 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f455c15b480 msgr2=0x7f455c159720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:40.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.888+0000 7f4561754640 1 --2- 172.21.15.32:0/3223726374 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f455c15b480 0x7f455c159720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.888+0000 7f4561754640 1 -- 172.21.15.32:0/3223726374 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f455c15deb0 con 0x7f455c15fd60 2024-04-24T23:10:40.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.888+0000 7f455bfff640 1 --2- 172.21.15.32:0/3223726374 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f455c15b480 0x7f455c159720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:10:40.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.888+0000 7f4561754640 1 --2- 172.21.15.32:0/3223726374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f455c15fd60 0x7f455c154bc0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f455400c6d0 tx=0x7f455400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:40.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.888+0000 7f45597fa640 1 -- 172.21.15.32:0/3223726374 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4554016020 con 0x7f455c15fd60 2024-04-24T23:10:40.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.888+0000 7f45597fa640 1 -- 172.21.15.32:0/3223726374 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f455400a840 con 0x7f455c15fd60 2024-04-24T23:10:40.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.888+0000 7f45631b7640 1 -- 172.21.15.32:0/3223726374 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f455c15e190 con 0x7f455c15fd60 2024-04-24T23:10:40.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.888+0000 7f45597fa640 1 -- 172.21.15.32:0/3223726374 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455400ac70 con 0x7f455c15fd60 2024-04-24T23:10:40.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.888+0000 7f45631b7640 1 -- 172.21.15.32:0/3223726374 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f455c159c60 con 0x7f455c15fd60 2024-04-24T23:10:40.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.889+0000 7f45631b7640 1 -- 172.21.15.32:0/3223726374 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f455c15e580 con 0x7f455c15fd60 2024-04-24T23:10:40.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.889+0000 7f45597fa640 1 -- 172.21.15.32:0/3223726374 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4554019020 con 0x7f455c15fd60 2024-04-24T23:10:40.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.890+0000 7f45597fa640 1 --2- 172.21.15.32:0/3223726374 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f454006be30 0x7f454006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:40.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.890+0000 7f45597fa640 1 -- 172.21.15.32:0/3223726374 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(185..185 src has 1..185) ==== 5709+0+0 (secure 0 0 0) 0x7f455408b2a0 con 0x7f455c15fd60 2024-04-24T23:10:40.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.890+0000 7f4560f53640 1 --2- 172.21.15.32:0/3223726374 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f454006be30 0x7f454006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:40.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.890+0000 7f4560f53640 1 --2- 172.21.15.32:0/3223726374 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f454006be30 0x7f454006e2f0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f4550034000 tx=0x7f455002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:40.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:40.894+0000 7f45597fa640 1 -- 172.21.15.32:0/3223726374 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4554020020 con 0x7f455c15fd60 2024-04-24T23:10:41.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.117+0000 7f45631b7640 1 -- 172.21.15.32:0/3223726374 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f455c068430 con 0x7f455c15fd60 2024-04-24T23:10:41.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.118+0000 7f45597fa640 1 -- 172.21.15.32:0/3223726374 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) ==== 94+0+5185 (secure 0 0 0) 0x7f4554059c50 con 0x7f455c15fd60 2024-04-24T23:10:41.119 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:10:41.119 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":80,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6941,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/416518156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":416518156},{"type":"v1","addr":"172.21.15.121:6837","nonce":416518156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7762,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1506667233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1506667233},{"type":"v1","addr":"172.21.15.121:6835","nonce":1506667233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":9063,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/2215079101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":2215079101},{"type":"v1","addr":"172.21.15.32:6834","nonce":2215079101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:10:36.353761+0000","modified":"2024-04-24T23:10:38.436407+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9066},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9066":{"gid":9066,"name":"a","rank":0,"incarnation":78,"state":"up:active","state_seq":4,"addr":"172.21.15.32:6835/901353412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":901353412},{"type":"v1","addr":"172.21.15.32:6835","nonce":901353412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9066,"qdb_cluster":[9066]},"id":11}]} 2024-04-24T23:10:41.120 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 80 2024-04-24T23:10:41.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.119+0000 7f45631b7640 1 -- 172.21.15.32:0/3223726374 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f454006be30 msgr2=0x7f454006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:41.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.119+0000 7f45631b7640 1 --2- 172.21.15.32:0/3223726374 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f454006be30 0x7f454006e2f0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f4550034000 tx=0x7f455002e000 comp rx=0 tx=0).stop 2024-04-24T23:10:41.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.119+0000 7f45631b7640 1 -- 172.21.15.32:0/3223726374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f455c15fd60 msgr2=0x7f455c154bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:41.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.119+0000 7f45631b7640 1 --2- 172.21.15.32:0/3223726374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f455c15fd60 0x7f455c154bc0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f455400c6d0 tx=0x7f455400cb90 comp rx=0 tx=0).stop 2024-04-24T23:10:41.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.120+0000 7f45631b7640 1 -- 172.21.15.32:0/3223726374 shutdown_connections 2024-04-24T23:10:41.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.120+0000 7f45631b7640 1 --2- 172.21.15.32:0/3223726374 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f454006be30 0x7f454006e2f0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.120+0000 7f45631b7640 1 --2- 172.21.15.32:0/3223726374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f455c15fd60 0x7f455c154bc0 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.120+0000 7f45631b7640 1 --2- 172.21.15.32:0/3223726374 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f455c15b480 0x7f455c159720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.120+0000 7f45631b7640 1 --2- 172.21.15.32:0/3223726374 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f455c06cd00 0x7f455c15d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.120+0000 7f45631b7640 1 -- 172.21.15.32:0/3223726374 >> 172.21.15.32:0/3223726374 conn(0x7f455c06b110 msgr2=0x7f455c0b1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:41.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.120+0000 7f45631b7640 1 -- 172.21.15.32:0/3223726374 shutdown_connections 2024-04-24T23:10:41.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.120+0000 7f45631b7640 1 -- 172.21.15.32:0/3223726374 wait complete. 2024-04-24T23:10:41.535 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 80, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T23:10:36.353761+0000', 'modified': '2024-04-24T23:10:38.436407+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9066}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9066': {'gid': 9066, 'name': 'a', 'rank': 0, 'incarnation': 78, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.32:6835/901353412', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6833', 'nonce': 901353412}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 901353412}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [23], 'metadata_pool': 22, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9066, 'qdb_cluster': [9066]} 2024-04-24T23:10:41.535 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-24T23:10:41.535 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-24T23:10:41.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.639+0000 7f7e1e8a5640 1 Processor -- start 2024-04-24T23:10:41.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.639+0000 7f7e1e8a5640 1 -- start start 2024-04-24T23:10:41.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.640+0000 7f7e1e8a5640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7e18074210 0x7f7e1806c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:41.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.640+0000 7f7e1e8a5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7e180b0b80 0x7f7e180745f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:41.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.640+0000 7f7e1e8a5640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e180b0770 con 0x7f7e180ae140 2024-04-24T23:10:41.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.640+0000 7f7e1e8a5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f7e180b1c20 con 0x7f7e180b0b80 2024-04-24T23:10:41.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.640+0000 7f7e1e8a5640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f7e18074b30 con 0x7f7e18074210 2024-04-24T23:10:41.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.640+0000 7f7e177fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7e180b0b80 0x7f7e180745f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:41.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.640+0000 7f7e177fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7e180b0b80 0x7f7e180745f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48774/0 (socket says 172.21.15.32:48774) 2024-04-24T23:10:41.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.640+0000 7f7e177fe640 1 -- 172.21.15.32:0/2933370689 learned_addr learned my addr 172.21.15.32:0/2933370689 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:41.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.640+0000 7f7e177fe640 1 -- 172.21.15.32:0/2933370689 >> v2:172.21.15.121:3300/0 conn(0x7f7e18074210 msgr2=0x7f7e1806c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:41.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.640+0000 7f7e177fe640 1 --2- 172.21.15.32:0/2933370689 >> v2:172.21.15.121:3300/0 conn(0x7f7e18074210 0x7f7e1806c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.640+0000 7f7e177fe640 1 -- 172.21.15.32:0/2933370689 >> v1:172.21.15.32:6789/0 conn(0x7f7e180ae140 legacy=0x7f7e180b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:41.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.640+0000 7f7e177fe640 1 -- 172.21.15.32:0/2933370689 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e180b1010 con 0x7f7e180b0b80 2024-04-24T23:10:41.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.641+0000 7f7e177fe640 1 --2- 172.21.15.32:0/2933370689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e180b0b80 0x7f7e180745f0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f7e18073960 tx=0x7f7e0c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e74235c1baa94dc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:41.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.641+0000 7f7e16ffd640 1 -- 172.21.15.32:0/2933370689 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c037070 con 0x7f7e180b0b80 2024-04-24T23:10:41.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.641+0000 7f7e16ffd640 1 -- 172.21.15.32:0/2933370689 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e0c032440 con 0x7f7e180b0b80 2024-04-24T23:10:41.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.641+0000 7f7e16ffd640 1 -- 172.21.15.32:0/2933370689 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c032810 con 0x7f7e180b0b80 2024-04-24T23:10:41.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.642+0000 7f7e1e8a5640 1 -- 172.21.15.32:0/2933370689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e180b0b80 msgr2=0x7f7e180745f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:41.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.642+0000 7f7e1e8a5640 1 --2- 172.21.15.32:0/2933370689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e180b0b80 0x7f7e180745f0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f7e18073960 tx=0x7f7e0c02a270 comp rx=0 tx=0).stop 2024-04-24T23:10:41.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.642+0000 7f7e1e8a5640 1 -- 172.21.15.32:0/2933370689 shutdown_connections 2024-04-24T23:10:41.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.642+0000 7f7e1e8a5640 1 --2- 172.21.15.32:0/2933370689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e180b0b80 0x7f7e180745f0 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.642+0000 7f7e1e8a5640 1 --2- 172.21.15.32:0/2933370689 >> v2:172.21.15.121:3300/0 conn(0x7f7e18074210 0x7f7e1806c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.642+0000 7f7e1e8a5640 1 -- 172.21.15.32:0/2933370689 >> 172.21.15.32:0/2933370689 conn(0x7f7e180b2e60 msgr2=0x7f7e180b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:41.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.642+0000 7f7e1e8a5640 1 -- 172.21.15.32:0/2933370689 shutdown_connections 2024-04-24T23:10:41.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.642+0000 7f7e1e8a5640 1 -- 172.21.15.32:0/2933370689 wait complete. 2024-04-24T23:10:41.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.643+0000 7f7e1e8a5640 1 Processor -- start 2024-04-24T23:10:41.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.643+0000 7f7e1e8a5640 1 -- start start 2024-04-24T23:10:41.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.644+0000 7f7e1e8a5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e18074210 0x7f7e180d18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:41.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.644+0000 7f7e1e8a5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e180ae140 0x7f7e180aef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:41.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.644+0000 7f7e1e8a5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e1815d360 0x7f7e180af490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:41.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.644+0000 7f7e1e8a5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7e180b18b0 con 0x7f7e18074210 2024-04-24T23:10:41.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.644+0000 7f7e1e8a5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7e180b15d0 con 0x7f7e1815d360 2024-04-24T23:10:41.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.644+0000 7f7e1e8a5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f7e180b3ba0 con 0x7f7e180ae140 2024-04-24T23:10:41.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.644+0000 7f7e17fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e18074210 0x7f7e180d18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:41.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.644+0000 7f7e17fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e18074210 0x7f7e180d18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48786/0 (socket says 172.21.15.32:48786) 2024-04-24T23:10:41.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.644+0000 7f7e17fff640 1 -- 172.21.15.32:0/2722264240 learned_addr learned my addr 172.21.15.32:0/2722264240 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:41.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.644+0000 7f7e17fff640 1 -- 172.21.15.32:0/2722264240 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e180ae140 msgr2=0x7f7e180aef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:41.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.644+0000 7f7e177fe640 1 --2- 172.21.15.32:0/2722264240 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e180ae140 0x7f7e180aef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:41.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.644+0000 7f7e1ce42640 1 --2- 172.21.15.32:0/2722264240 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e1815d360 0x7f7e180af490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:41.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.644+0000 7f7e17fff640 1 --2- 172.21.15.32:0/2722264240 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e180ae140 0x7f7e180aef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.644+0000 7f7e17fff640 1 -- 172.21.15.32:0/2722264240 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e1815d360 msgr2=0x7f7e180af490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:41.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.644+0000 7f7e17fff640 1 --2- 172.21.15.32:0/2722264240 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e1815d360 0x7f7e180af490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.644+0000 7f7e17fff640 1 -- 172.21.15.32:0/2722264240 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e180c8560 con 0x7f7e18074210 2024-04-24T23:10:41.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.645+0000 7f7e17fff640 1 --2- 172.21.15.32:0/2722264240 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e18074210 0x7f7e180d18e0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f7e0800b440 tx=0x7f7e0800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:41.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.645+0000 7f7e14ff9640 1 -- 172.21.15.32:0/2722264240 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e08015020 con 0x7f7e18074210 2024-04-24T23:10:41.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.645+0000 7f7e14ff9640 1 -- 172.21.15.32:0/2722264240 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e08011890 con 0x7f7e18074210 2024-04-24T23:10:41.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.645+0000 7f7e1e8a5640 1 -- 172.21.15.32:0/2722264240 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e180c8840 con 0x7f7e18074210 2024-04-24T23:10:41.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.645+0000 7f7e14ff9640 1 -- 172.21.15.32:0/2722264240 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e08011c80 con 0x7f7e18074210 2024-04-24T23:10:41.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.645+0000 7f7e1e8a5640 1 -- 172.21.15.32:0/2722264240 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e180c8e00 con 0x7f7e18074210 2024-04-24T23:10:41.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.646+0000 7f7e14ff9640 1 -- 172.21.15.32:0/2722264240 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7e08018020 con 0x7f7e18074210 2024-04-24T23:10:41.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.647+0000 7f7e14ff9640 1 --2- 172.21.15.32:0/2722264240 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7e0406be60 0x7f7e0406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:41.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.647+0000 7f7e14ff9640 1 -- 172.21.15.32:0/2722264240 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(185..185 src has 1..185) ==== 5709+0+0 (secure 0 0 0) 0x7f7e0808af80 con 0x7f7e18074210 2024-04-24T23:10:41.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.647+0000 7f7e177fe640 1 --2- 172.21.15.32:0/2722264240 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7e0406be60 0x7f7e0406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:41.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.648+0000 7f7e177fe640 1 --2- 172.21.15.32:0/2722264240 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7e0406be60 0x7f7e0406e320 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c034000 tx=0x7f7e0c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:41.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.648+0000 7f7e215b5740 1 Processor -- start 2024-04-24T23:10:41.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.648+0000 7f7e215b5740 1 -- start start 2024-04-24T23:10:41.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.649+0000 7f7e215b5740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac114c1b60 0x55ac114744f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:41.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.649+0000 7f7e215b5740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ac114a4c80 0x55ac114a4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:41.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.649+0000 7f7e215b5740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ac114a4540 0x55ac11479350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:41.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.649+0000 7f7e215b5740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ac113dcf00 con 0x55ac114c1b60 2024-04-24T23:10:41.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.649+0000 7f7e215b5740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55ac11385750 con 0x55ac114a4c80 2024-04-24T23:10:41.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.649+0000 7f7e215b5740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x55ac113d8ec0 con 0x55ac114a4540 2024-04-24T23:10:41.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.649+0000 7f7e17fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac114c1b60 0x55ac114744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:41.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.649+0000 7f7e17fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac114c1b60 0x55ac114744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48796/0 (socket says 172.21.15.32:48796) 2024-04-24T23:10:41.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.649+0000 7f7e17fff640 1 -- 172.21.15.32:0/318971221 learned_addr learned my addr 172.21.15.32:0/318971221 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:41.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.649+0000 7f7e177fe640 1 --2- 172.21.15.32:0/318971221 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ac114a4c80 0x55ac114a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:41.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.649+0000 7f7e1ce42640 1 --2- 172.21.15.32:0/318971221 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ac114a4540 0x55ac11479350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:41.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.649+0000 7f7e17fff640 1 -- 172.21.15.32:0/318971221 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ac114a4540 msgr2=0x55ac11479350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:41.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.649+0000 7f7e17fff640 1 --2- 172.21.15.32:0/318971221 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ac114a4540 0x55ac11479350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.649+0000 7f7e17fff640 1 -- 172.21.15.32:0/318971221 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ac114a4c80 msgr2=0x55ac114a4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:41.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.649+0000 7f7e17fff640 1 --2- 172.21.15.32:0/318971221 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ac114a4c80 0x55ac114a4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.649+0000 7f7e17fff640 1 -- 172.21.15.32:0/318971221 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac110f12d0 con 0x55ac114c1b60 2024-04-24T23:10:41.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.649+0000 7f7e17fff640 1 --2- 172.21.15.32:0/318971221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac114c1b60 0x55ac114744f0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f7e080618d0 tx=0x7f7e080599e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c990d4fae0dd60cd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:41.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.649+0000 7f7dfdffb640 1 -- 172.21.15.32:0/318971221 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e08087070 con 0x55ac114c1b60 2024-04-24T23:10:41.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.649+0000 7f7dfdffb640 1 -- 172.21.15.32:0/318971221 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e08061e00 con 0x55ac114c1b60 2024-04-24T23:10:41.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.649+0000 7f7dfdffb640 1 -- 172.21.15.32:0/318971221 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e08071070 con 0x55ac114c1b60 2024-04-24T23:10:41.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.650+0000 7f7e215b5740 1 -- 172.21.15.32:0/318971221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac114c1b60 msgr2=0x55ac114744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:41.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.650+0000 7f7e215b5740 1 --2- 172.21.15.32:0/318971221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac114c1b60 0x55ac114744f0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f7e080618d0 tx=0x7f7e080599e0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.650+0000 7f7e215b5740 1 -- 172.21.15.32:0/318971221 shutdown_connections 2024-04-24T23:10:41.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.650+0000 7f7e215b5740 1 --2- 172.21.15.32:0/318971221 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ac114a4540 0x55ac11479350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.650+0000 7f7e215b5740 1 --2- 172.21.15.32:0/318971221 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ac114a4c80 0x55ac114a4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.650+0000 7f7e215b5740 1 --2- 172.21.15.32:0/318971221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac114c1b60 0x55ac114744f0 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.650+0000 7f7e215b5740 1 -- 172.21.15.32:0/318971221 >> 172.21.15.32:0/318971221 conn(0x55ac11476760 msgr2=0x55ac114bf000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:41.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.650+0000 7f7e215b5740 1 -- 172.21.15.32:0/318971221 shutdown_connections 2024-04-24T23:10:41.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.650+0000 7f7e215b5740 1 -- 172.21.15.32:0/318971221 wait complete. 2024-04-24T23:10:41.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.651+0000 7f7e215b5740 1 -- start start 2024-04-24T23:10:41.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.651+0000 7f7e215b5740 1 Processor -- start 2024-04-24T23:10:41.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.652+0000 7f7e215b5740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ac114a4c80 0x55ac114cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:41.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.652+0000 7f7e215b5740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac114c1b60 0x55ac11473650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:41.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.652+0000 7f7e215b5740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ac114a7120 0x55ac114b6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:41.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.652+0000 7f7e215b5740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ac11478910 con 0x55ac114c1b60 2024-04-24T23:10:41.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.652+0000 7f7e215b5740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55ac11398cf0 con 0x55ac114a4c80 2024-04-24T23:10:41.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.652+0000 7f7e215b5740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x55ac11477d00 con 0x55ac114a7120 2024-04-24T23:10:41.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.652+0000 7f7e215b5740 10 client.0 _ll_register_callbacks cb 0x55ac113986e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:10:41.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.652+0000 7f7e177fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac114c1b60 0x55ac11473650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:41.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.652+0000 7f7e177fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac114c1b60 0x55ac11473650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48812/0 (socket says 172.21.15.32:48812) 2024-04-24T23:10:41.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.652+0000 7f7e177fe640 1 -- 172.21.15.32:0/1510620550 learned_addr learned my addr 172.21.15.32:0/1510620550 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:41.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.652+0000 7f7e17fff640 1 --2- 172.21.15.32:0/1510620550 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ac114a4c80 0x55ac114cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:41.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.652+0000 7f7e1ce42640 1 --2- 172.21.15.32:0/1510620550 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ac114a7120 0x55ac114b6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:41.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.652+0000 7f7e177fe640 1 -- 172.21.15.32:0/1510620550 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ac114a7120 msgr2=0x55ac114b6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:41.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.652+0000 7f7e177fe640 1 --2- 172.21.15.32:0/1510620550 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ac114a7120 0x55ac114b6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.652+0000 7f7e177fe640 1 -- 172.21.15.32:0/1510620550 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ac114a4c80 msgr2=0x55ac114cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:41.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.652+0000 7f7e177fe640 1 --2- 172.21.15.32:0/1510620550 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ac114a4c80 0x55ac114cf0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.652+0000 7f7e177fe640 1 -- 172.21.15.32:0/1510620550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac111bb6b0 con 0x55ac114c1b60 2024-04-24T23:10:41.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.652+0000 7f7e177fe640 1 --2- 172.21.15.32:0/1510620550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac114c1b60 0x55ac11473650 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c02a860 tx=0x7f7e0c02c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae0841cc081128ac server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:41.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.652+0000 7f7df6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-24T23:10:41.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.652+0000 7f7df6ffd640 1 -- 172.21.15.32:0/1510620550 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c037040 con 0x55ac114c1b60 2024-04-24T23:10:41.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.652+0000 7f7df6ffd640 1 -- 172.21.15.32:0/1510620550 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(185..185 src has 1..185) ==== 5709+0+0 (secure 0 0 0) 0x7f7e0c03cd40 con 0x55ac114c1b60 2024-04-24T23:10:41.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.652+0000 7f7e215b5740 1 -- 172.21.15.32:0/1510620550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e08061e00 con 0x55ac114c1b60 2024-04-24T23:10:41.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.653+0000 7f7df6ffd640 1 -- 172.21.15.32:0/1510620550 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f7e0c056050 con 0x55ac114c1b60 2024-04-24T23:10:41.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.653+0000 7f7e215b5740 10 client.9210 fetch_fsmap learned FSMap version 80 2024-04-24T23:10:41.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.653+0000 7f7e215b5740 1 -- 172.21.15.32:0/1510620550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=2+}) -- 0x55ac114be660 con 0x55ac114c1b60 2024-04-24T23:10:41.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.653+0000 7f7df6ffd640 1 -- 172.21.15.32:0/1510620550 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e08061e00 con 0x55ac114c1b60 2024-04-24T23:10:41.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.653+0000 7f7df6ffd640 1 -- 172.21.15.32:0/1510620550 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 80) ==== 3494+0+0 (secure 0 0 0) 0x7f7e0c059020 con 0x55ac114c1b60 2024-04-24T23:10:41.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.653+0000 7f7e215b5740 10 client.9210 fetch_fsmap finished waiting for FSMap version 80 2024-04-24T23:10:41.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.653+0000 7f7e215b5740 10 client.9210 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T23:10:41.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.653+0000 7f7e215b5740 20 client.9210 populate_metadata read hostname 'smithi032' 2024-04-24T23:10:41.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.653+0000 7f7e215b5740 1 --2- 172.21.15.32:0/1510620550 >> [v2:172.21.15.32:6833/901353412,v1:172.21.15.32:6835/901353412] conn(0x55ac1149f890 0x55ac11479360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:41.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.653+0000 7f7e215b5740 4 client.9210 mds_command: new command op to 9066 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:10:41.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.653+0000 7f7e215b5740 1 -- 172.21.15.32:0/1510620550 --> [v2:172.21.15.32:6833/901353412,v1:172.21.15.32:6835/901353412] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac113ea440 con 0x55ac1149f890 2024-04-24T23:10:41.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.653+0000 7f7e17fff640 1 --2- 172.21.15.32:0/1510620550 >> [v2:172.21.15.32:6833/901353412,v1:172.21.15.32:6835/901353412] conn(0x55ac1149f890 0x55ac11479360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:41.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.653+0000 7f7e17fff640 1 --2- 172.21.15.32:0/1510620550 >> [v2:172.21.15.32:6833/901353412,v1:172.21.15.32:6835/901353412] conn(0x55ac1149f890 0x55ac11479360 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a70651f103c4bb2e server_cookie=f5cf63542dcac234 in_seq=0 out_seq=0 2024-04-24T23:10:41.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.653+0000 7f7df6ffd640 10 client.9210 ms_handle_connect on v2:172.21.15.32:6833/901353412 2024-04-24T23:10:41.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.654+0000 7f7df6ffd640 1 -- 172.21.15.32:0/1510620550 <== mds.0 v2:172.21.15.32:6833/901353412 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ac113ea440 con 0x55ac1149f890 2024-04-24T23:10:41.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.654+0000 7f7df6ffd640 10 client.9210 handle_command_reply: tid=0 2024-04-24T23:10:41.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.654+0000 7f7e215b5740 1 client.9210 shutdown 2024-04-24T23:10:41.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.654+0000 7f7e215b5740 1 -- 172.21.15.32:0/1510620550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac114c1b60 msgr2=0x55ac11473650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:41.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.654+0000 7f7e215b5740 1 --2- 172.21.15.32:0/1510620550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac114c1b60 0x55ac11473650 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c02a860 tx=0x7f7e0c02c070 comp rx=0 tx=0).stop 2024-04-24T23:10:41.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.654+0000 7f7e215b5740 1 -- 172.21.15.32:0/1510620550 shutdown_connections 2024-04-24T23:10:41.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.654+0000 7f7e215b5740 1 --2- 172.21.15.32:0/1510620550 >> [v2:172.21.15.32:6833/901353412,v1:172.21.15.32:6835/901353412] conn(0x55ac1149f890 0x55ac11479360 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.654+0000 7f7e215b5740 1 --2- 172.21.15.32:0/1510620550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac114c1b60 0x55ac11473650 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.654+0000 7f7e215b5740 1 --2- 172.21.15.32:0/1510620550 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ac114a7120 0x55ac114b6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.654+0000 7f7df6ffd640 0 client.9210 ms_handle_reset on v2:172.21.15.32:6833/901353412 2024-04-24T23:10:41.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.654+0000 7f7e215b5740 1 --2- 172.21.15.32:0/1510620550 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ac114a4c80 0x55ac114cf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.654+0000 7f7e215b5740 1 -- 172.21.15.32:0/1510620550 >> 172.21.15.32:0/1510620550 conn(0x55ac114a4540 msgr2=0x55ac114c1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:41.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.654+0000 7f7e215b5740 1 -- 172.21.15.32:0/1510620550 shutdown_connections 2024-04-24T23:10:41.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.654+0000 7f7e215b5740 1 -- 172.21.15.32:0/1510620550 wait complete. 2024-04-24T23:10:41.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.654+0000 7f7e215b5740 20 client.9210 trim_cache size 0 max 16384 2024-04-24T23:10:41.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.673+0000 7f7e215b5740 1 Processor -- start 2024-04-24T23:10:41.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.673+0000 7f7e215b5740 1 -- start start 2024-04-24T23:10:41.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.673+0000 7f7e215b5740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ac114a4c80 0x55ac1146f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:41.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.673+0000 7f7e215b5740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac114a7120 0x55ac114b7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:41.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.673+0000 7f7e215b5740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ac114c1b60 0x55ac11473650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:41.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.673+0000 7f7e215b5740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7e181fff00 con 0x55ac114a7120 2024-04-24T23:10:41.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.673+0000 7f7e215b5740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7e18161970 con 0x55ac114a4c80 2024-04-24T23:10:41.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.673+0000 7f7e215b5740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f7e181ff8b0 con 0x55ac114c1b60 2024-04-24T23:10:41.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.674+0000 7f7e177fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac114a7120 0x55ac114b7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:41.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.674+0000 7f7e177fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac114a7120 0x55ac114b7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48814/0 (socket says 172.21.15.32:48814) 2024-04-24T23:10:41.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.674+0000 7f7e177fe640 1 -- 172.21.15.32:0/1233340912 learned_addr learned my addr 172.21.15.32:0/1233340912 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:41.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.674+0000 7f7e1ce42640 1 --2- 172.21.15.32:0/1233340912 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ac114c1b60 0x55ac11473650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:41.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.674+0000 7f7e17fff640 1 --2- 172.21.15.32:0/1233340912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ac114a4c80 0x55ac1146f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:41.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.674+0000 7f7e177fe640 1 -- 172.21.15.32:0/1233340912 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ac114c1b60 msgr2=0x55ac11473650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:41.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.674+0000 7f7e177fe640 1 --2- 172.21.15.32:0/1233340912 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ac114c1b60 0x55ac11473650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.674+0000 7f7e177fe640 1 -- 172.21.15.32:0/1233340912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ac114a4c80 msgr2=0x55ac1146f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:41.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.674+0000 7f7e177fe640 1 --2- 172.21.15.32:0/1233340912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ac114a4c80 0x55ac1146f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.674+0000 7f7e177fe640 1 -- 172.21.15.32:0/1233340912 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac114cea40 con 0x55ac114a7120 2024-04-24T23:10:41.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.674+0000 7f7e1ce42640 1 --2- 172.21.15.32:0/1233340912 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ac114c1b60 0x55ac11473650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:10:41.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.674+0000 7f7e17fff640 1 --2- 172.21.15.32:0/1233340912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ac114a4c80 0x55ac1146f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:10:41.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.674+0000 7f7e177fe640 1 --2- 172.21.15.32:0/1233340912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac114a7120 0x55ac114b7330 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c009630 tx=0x7f7e0c03a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abb690ef0ef2ef69 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:41.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.674+0000 7f7dfd7fa640 1 -- 172.21.15.32:0/1233340912 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c037070 con 0x55ac114a7120 2024-04-24T23:10:41.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.674+0000 7f7dfd7fa640 1 -- 172.21.15.32:0/1233340912 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e0c04e070 con 0x55ac114a7120 2024-04-24T23:10:41.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.674+0000 7f7dfd7fa640 1 -- 172.21.15.32:0/1233340912 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c0493e0 con 0x55ac114a7120 2024-04-24T23:10:41.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.675+0000 7f7e215b5740 1 -- 172.21.15.32:0/1233340912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac114a7120 msgr2=0x55ac114b7330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:41.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.675+0000 7f7e215b5740 1 --2- 172.21.15.32:0/1233340912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac114a7120 0x55ac114b7330 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c009630 tx=0x7f7e0c03a000 comp rx=0 tx=0).stop 2024-04-24T23:10:41.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.675+0000 7f7e215b5740 1 -- 172.21.15.32:0/1233340912 shutdown_connections 2024-04-24T23:10:41.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.675+0000 7f7e215b5740 1 --2- 172.21.15.32:0/1233340912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac114a7120 0x55ac114b7330 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.675+0000 7f7e215b5740 1 --2- 172.21.15.32:0/1233340912 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ac114c1b60 0x55ac11473650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.675+0000 7f7e215b5740 1 --2- 172.21.15.32:0/1233340912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ac114a4c80 0x55ac1146f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.675+0000 7f7e215b5740 1 -- 172.21.15.32:0/1233340912 >> 172.21.15.32:0/1233340912 conn(0x55ac114a4540 msgr2=0x55ac114bddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:41.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.675+0000 7f7e215b5740 1 -- 172.21.15.32:0/1233340912 shutdown_connections 2024-04-24T23:10:41.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.675+0000 7f7e215b5740 1 -- 172.21.15.32:0/1233340912 wait complete. 2024-04-24T23:10:41.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.675+0000 7f7e215b5740 1 -- start start 2024-04-24T23:10:41.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.676+0000 7f7e215b5740 1 Processor -- start 2024-04-24T23:10:41.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.676+0000 7f7e215b5740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac114a4c80 0x55ac114b8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:41.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.676+0000 7f7e215b5740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ac114a7120 0x55ac114c0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:41.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.676+0000 7f7e215b5740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ac114c1b60 0x55ac114e1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:41.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.676+0000 7f7e215b5740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ac1149f710 con 0x55ac114a4c80 2024-04-24T23:10:41.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.676+0000 7f7e215b5740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55ac114cf690 con 0x55ac114a7120 2024-04-24T23:10:41.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.676+0000 7f7e215b5740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x55ac1149fd60 con 0x55ac114c1b60 2024-04-24T23:10:41.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.676+0000 7f7e215b5740 10 client.0 _ll_register_callbacks cb 0x7f7e0c032b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:10:41.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.676+0000 7f7e17fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac114a4c80 0x55ac114b8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:41.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.676+0000 7f7e17fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac114a4c80 0x55ac114b8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48820/0 (socket says 172.21.15.32:48820) 2024-04-24T23:10:41.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.676+0000 7f7e17fff640 1 -- 172.21.15.32:0/313474499 learned_addr learned my addr 172.21.15.32:0/313474499 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:41.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.677+0000 7f7e1ce42640 1 --2- 172.21.15.32:0/313474499 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ac114c1b60 0x55ac114e1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:41.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.677+0000 7f7e177fe640 1 --2- 172.21.15.32:0/313474499 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ac114a7120 0x55ac114c0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:41.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.677+0000 7f7e17fff640 1 -- 172.21.15.32:0/313474499 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ac114c1b60 msgr2=0x55ac114e1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:41.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.677+0000 7f7e17fff640 1 --2- 172.21.15.32:0/313474499 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ac114c1b60 0x55ac114e1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.677+0000 7f7e17fff640 1 -- 172.21.15.32:0/313474499 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ac114a7120 msgr2=0x55ac114c0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:41.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.677+0000 7f7e17fff640 1 --2- 172.21.15.32:0/313474499 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ac114a7120 0x55ac114c0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.677+0000 7f7e17fff640 1 -- 172.21.15.32:0/313474499 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac11474590 con 0x55ac114a4c80 2024-04-24T23:10:41.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.677+0000 7f7e1ce42640 1 --2- 172.21.15.32:0/313474499 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ac114c1b60 0x55ac114e1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:10:41.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.677+0000 7f7e177fe640 1 --2- 172.21.15.32:0/313474499 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ac114a7120 0x55ac114c0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:10:41.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.677+0000 7f7e17fff640 1 --2- 172.21.15.32:0/313474499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac114a4c80 0x55ac114b8e30 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f7e08006d50 tx=0x7f7e08006ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb0afd15b0b33e01 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:41.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.677+0000 7f7df6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-24T23:10:41.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.677+0000 7f7df6ffd640 1 -- 172.21.15.32:0/313474499 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e08077060 con 0x55ac114a4c80 2024-04-24T23:10:41.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.677+0000 7f7df6ffd640 1 -- 172.21.15.32:0/313474499 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(185..185 src has 1..185) ==== 5709+0+0 (secure 0 0 0) 0x7f7e080711e0 con 0x55ac114a4c80 2024-04-24T23:10:41.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.677+0000 7f7e215b5740 1 -- 172.21.15.32:0/313474499 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e0c04e070 con 0x55ac114a4c80 2024-04-24T23:10:41.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.677+0000 7f7df6ffd640 1 -- 172.21.15.32:0/313474499 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f7e08087040 con 0x55ac114a4c80 2024-04-24T23:10:41.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.677+0000 7f7e215b5740 10 client.9216 fetch_fsmap learned FSMap version 80 2024-04-24T23:10:41.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.677+0000 7f7e215b5740 1 -- 172.21.15.32:0/313474499 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=2+}) -- 0x55ac114e24b0 con 0x55ac114a4c80 2024-04-24T23:10:41.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.677+0000 7f7df6ffd640 1 -- 172.21.15.32:0/313474499 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e0c04e070 con 0x55ac114a4c80 2024-04-24T23:10:41.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.677+0000 7f7df6ffd640 1 -- 172.21.15.32:0/313474499 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 80) ==== 3494+0+0 (secure 0 0 0) 0x7f7e08055890 con 0x55ac114a4c80 2024-04-24T23:10:41.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.677+0000 7f7e215b5740 10 client.9216 fetch_fsmap finished waiting for FSMap version 80 2024-04-24T23:10:41.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.677+0000 7f7e215b5740 10 client.9216 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T23:10:41.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.677+0000 7f7e215b5740 20 client.9216 populate_metadata read hostname 'smithi032' 2024-04-24T23:10:41.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.677+0000 7f7e215b5740 1 --2- 172.21.15.32:0/313474499 >> [v2:172.21.15.32:6833/901353412,v1:172.21.15.32:6835/901353412] conn(0x55ac114b4dc0 0x55ac114c2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:41.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.677+0000 7f7e215b5740 4 client.9216 mds_command: new command op to 9066 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:10:41.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.677+0000 7f7e215b5740 1 -- 172.21.15.32:0/313474499 --> [v2:172.21.15.32:6833/901353412,v1:172.21.15.32:6835/901353412] -- command(tid 0: {"prefix": "status"}) -- 0x55ac113e4790 con 0x55ac114b4dc0 2024-04-24T23:10:41.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.678+0000 7f7e177fe640 1 --2- 172.21.15.32:0/313474499 >> [v2:172.21.15.32:6833/901353412,v1:172.21.15.32:6835/901353412] conn(0x55ac114b4dc0 0x55ac114c2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:41.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.678+0000 7f7e177fe640 1 --2- 172.21.15.32:0/313474499 >> [v2:172.21.15.32:6833/901353412,v1:172.21.15.32:6835/901353412] conn(0x55ac114b4dc0 0x55ac114c2420 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a51ff76116e9d5a server_cookie=f29df436f5a487e6 in_seq=0 out_seq=0 2024-04-24T23:10:41.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.678+0000 7f7df6ffd640 10 client.9216 ms_handle_connect on v2:172.21.15.32:6833/901353412 2024-04-24T23:10:41.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.678+0000 7f7df6ffd640 1 -- 172.21.15.32:0/313474499 <== mds.0 v2:172.21.15.32:6833/901353412 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x55ac113e4790 con 0x55ac114b4dc0 2024-04-24T23:10:41.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.678+0000 7f7df6ffd640 10 client.9216 handle_command_reply: tid=0 2024-04-24T23:10:41.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.678+0000 7f7e215b5740 1 client.9216 shutdown 2024-04-24T23:10:41.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.678+0000 7f7e215b5740 1 -- 172.21.15.32:0/313474499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac114a4c80 msgr2=0x55ac114b8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:41.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.678+0000 7f7e215b5740 1 --2- 172.21.15.32:0/313474499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac114a4c80 0x55ac114b8e30 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f7e08006d50 tx=0x7f7e08006ce0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.678+0000 7f7e215b5740 1 -- 172.21.15.32:0/313474499 shutdown_connections 2024-04-24T23:10:41.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.678+0000 7f7e215b5740 1 --2- 172.21.15.32:0/313474499 >> [v2:172.21.15.32:6833/901353412,v1:172.21.15.32:6835/901353412] conn(0x55ac114b4dc0 0x55ac114c2420 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.678+0000 7f7e215b5740 1 --2- 172.21.15.32:0/313474499 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ac114c1b60 0x55ac114e1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.678+0000 7f7e215b5740 1 --2- 172.21.15.32:0/313474499 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ac114a7120 0x55ac114c0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.678+0000 7f7df6ffd640 0 client.9216 ms_handle_reset on v2:172.21.15.32:6833/901353412 2024-04-24T23:10:41.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.678+0000 7f7e215b5740 1 --2- 172.21.15.32:0/313474499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac114a4c80 0x55ac114b8e30 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.678+0000 7f7e215b5740 1 -- 172.21.15.32:0/313474499 >> 172.21.15.32:0/313474499 conn(0x55ac114a4540 msgr2=0x55ac114a0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:41.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.678+0000 7f7e215b5740 1 -- 172.21.15.32:0/313474499 shutdown_connections 2024-04-24T23:10:41.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.678+0000 7f7e215b5740 1 -- 172.21.15.32:0/313474499 wait complete. 2024-04-24T23:10:41.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.678+0000 7f7e215b5740 20 client.9216 trim_cache size 0 max 16384 2024-04-24T23:10:41.688 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:10:41.688 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", 2024-04-24T23:10:41.688 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2024-04-24T23:10:41.688 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 9066, 2024-04-24T23:10:41.688 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-04-24T23:10:41.689 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-04-24T23:10:41.689 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-04-24T23:10:41.689 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 4.6851322460000002, 2024-04-24T23:10:41.689 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 80, 2024-04-24T23:10:41.689 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 185, 2024-04-24T23:10:41.689 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 185, 2024-04-24T23:10:41.689 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 15.713646279000001 2024-04-24T23:10:41.689 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:10:41.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.679+0000 7f7e1e8a5640 1 -- 172.21.15.32:0/2722264240 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7e0406be60 msgr2=0x7f7e0406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:41.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.679+0000 7f7e1e8a5640 1 --2- 172.21.15.32:0/2722264240 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7e0406be60 0x7f7e0406e320 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c034000 tx=0x7f7e0c02e000 comp rx=0 tx=0).stop 2024-04-24T23:10:41.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.679+0000 7f7e1e8a5640 1 -- 172.21.15.32:0/2722264240 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e18074210 msgr2=0x7f7e180d18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:41.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.679+0000 7f7e1e8a5640 1 --2- 172.21.15.32:0/2722264240 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e18074210 0x7f7e180d18e0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f7e0800b440 tx=0x7f7e0800b900 comp rx=0 tx=0).stop 2024-04-24T23:10:41.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.679+0000 7f7e1e8a5640 1 -- 172.21.15.32:0/2722264240 shutdown_connections 2024-04-24T23:10:41.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.679+0000 7f7e1e8a5640 1 --2- 172.21.15.32:0/2722264240 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7e0406be60 0x7f7e0406e320 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.679+0000 7f7e1e8a5640 1 --2- 172.21.15.32:0/2722264240 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e1815d360 0x7f7e180af490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.679+0000 7f7e1e8a5640 1 --2- 172.21.15.32:0/2722264240 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e180ae140 0x7f7e180aef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.679+0000 7f7e1e8a5640 1 --2- 172.21.15.32:0/2722264240 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e18074210 0x7f7e180d18e0 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.679+0000 7f7e1e8a5640 1 -- 172.21.15.32:0/2722264240 >> 172.21.15.32:0/2722264240 conn(0x7f7e180b2e60 msgr2=0x7f7e1806d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:41.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.679+0000 7f7e1e8a5640 1 -- 172.21.15.32:0/2722264240 shutdown_connections 2024-04-24T23:10:41.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.679+0000 7f7e1e8a5640 1 -- 172.21.15.32:0/2722264240 wait complete. 2024-04-24T23:10:41.693 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T23:10:41.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.792+0000 7f80172bd640 1 Processor -- start 2024-04-24T23:10:41.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.793+0000 7f80172bd640 1 -- start start 2024-04-24T23:10:41.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.793+0000 7f80172bd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f801006b820 0x7f80100d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:41.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.793+0000 7f80172bd640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f80100d4210 0x7f80100b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:41.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.793+0000 7f80172bd640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80100d3170 con 0x7f801015e9b0 2024-04-24T23:10:41.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.793+0000 7f80172bd640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f80100d3450 con 0x7f801006b820 2024-04-24T23:10:41.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.793+0000 7f80172bd640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f80100d3730 con 0x7f80100d4210 2024-04-24T23:10:41.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.794+0000 7f8014858640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f801006b820 0x7f80100d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:41.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.794+0000 7f8015059640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f801015e9b0 0x7f801006b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58632/0 (socket says 172.21.15.32:58632) 2024-04-24T23:10:41.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.794+0000 7f8014858640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f801006b820 0x7f80100d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48834/0 (socket says 172.21.15.32:48834) 2024-04-24T23:10:41.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.794+0000 7f8015059640 1 -- 172.21.15.32:0/4073593571 learned_addr learned my addr 172.21.15.32:0/4073593571 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:10:41.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.794+0000 7f801585a640 1 --2- 172.21.15.32:0/4073593571 >> v2:172.21.15.121:3300/0 conn(0x7f80100d4210 0x7f80100b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:41.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.794+0000 7f8014858640 1 -- 172.21.15.32:0/4073593571 >> v2:172.21.15.121:3300/0 conn(0x7f80100d4210 msgr2=0x7f80100b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:41.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.794+0000 7f8014858640 1 --2- 172.21.15.32:0/4073593571 >> v2:172.21.15.121:3300/0 conn(0x7f80100d4210 0x7f80100b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.794+0000 7f8014858640 1 -- 172.21.15.32:0/4073593571 >> v1:172.21.15.32:6789/0 conn(0x7f801015e9b0 legacy=0x7f801006b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:41.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.794+0000 7f8007fff640 1 -- 172.21.15.32:0/4073593571 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2542130988 0 0) 0x7f80100d3170 con 0x7f801015e9b0 2024-04-24T23:10:41.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.794+0000 7f8014858640 1 -- 172.21.15.32:0/4073593571 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80100d3a10 con 0x7f801006b820 2024-04-24T23:10:41.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.795+0000 7f8014858640 1 --2- 172.21.15.32:0/4073593571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f801006b820 0x7f80100d2b90 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f80101622f0 tx=0x7f800002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26969038b35344b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:41.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.795+0000 7f8007fff640 1 -- 172.21.15.32:0/4073593571 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8000037070 con 0x7f801006b820 2024-04-24T23:10:41.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.795+0000 7f8007fff640 1 -- 172.21.15.32:0/4073593571 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8000032440 con 0x7f801006b820 2024-04-24T23:10:41.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.795+0000 7f8007fff640 1 -- 172.21.15.32:0/4073593571 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800002ae60 con 0x7f801006b820 2024-04-24T23:10:41.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.795+0000 7f80172bd640 1 -- 172.21.15.32:0/4073593571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f801006b820 msgr2=0x7f80100d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:41.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.795+0000 7f80172bd640 1 --2- 172.21.15.32:0/4073593571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f801006b820 0x7f80100d2b90 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f80101622f0 tx=0x7f800002a110 comp rx=0 tx=0).stop 2024-04-24T23:10:41.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.795+0000 7f80172bd640 1 -- 172.21.15.32:0/4073593571 shutdown_connections 2024-04-24T23:10:41.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.795+0000 7f80172bd640 1 --2- 172.21.15.32:0/4073593571 >> v2:172.21.15.121:3300/0 conn(0x7f80100d4210 0x7f80100b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.795+0000 7f80172bd640 1 --2- 172.21.15.32:0/4073593571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f801006b820 0x7f80100d2b90 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.795+0000 7f80172bd640 1 -- 172.21.15.32:0/4073593571 >> 172.21.15.32:0/4073593571 conn(0x7f80100d11d0 msgr2=0x7f80100d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:41.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.795+0000 7f80172bd640 1 -- 172.21.15.32:0/4073593571 shutdown_connections 2024-04-24T23:10:41.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.795+0000 7f80172bd640 1 -- 172.21.15.32:0/4073593571 wait complete. 2024-04-24T23:10:41.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.797+0000 7f80172bd640 1 Processor -- start 2024-04-24T23:10:41.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.797+0000 7f80172bd640 1 -- start start 2024-04-24T23:10:41.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.797+0000 7f80172bd640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f801006b820 0x7f80100b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:41.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.797+0000 7f80172bd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80100d4210 0x7f80100b7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:41.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.797+0000 7f80172bd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f801015e9b0 0x7f80100cc870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:41.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.797+0000 7f80172bd640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f80100ba0e0 con 0x7f801015e9b0 2024-04-24T23:10:41.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.797+0000 7f80172bd640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f80100d2400 con 0x7f80100d4210 2024-04-24T23:10:41.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.797+0000 7f80172bd640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f80100ba260 con 0x7f801006b820 2024-04-24T23:10:41.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.797+0000 7f801585a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f801015e9b0 0x7f80100cc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:41.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.797+0000 7f801585a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f801015e9b0 0x7f80100cc870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48846/0 (socket says 172.21.15.32:48846) 2024-04-24T23:10:41.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.797+0000 7f801585a640 1 -- 172.21.15.32:0/3617757639 learned_addr learned my addr 172.21.15.32:0/3617757639 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:41.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.798+0000 7f8015059640 1 --2- 172.21.15.32:0/3617757639 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f801006b820 0x7f80100b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:41.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.798+0000 7f8014858640 1 --2- 172.21.15.32:0/3617757639 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80100d4210 0x7f80100b7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:41.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.798+0000 7f801585a640 1 -- 172.21.15.32:0/3617757639 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f801006b820 msgr2=0x7f80100b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:41.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.798+0000 7f801585a640 1 --2- 172.21.15.32:0/3617757639 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f801006b820 0x7f80100b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.798+0000 7f801585a640 1 -- 172.21.15.32:0/3617757639 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80100d4210 msgr2=0x7f80100b7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:41.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.798+0000 7f801585a640 1 --2- 172.21.15.32:0/3617757639 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80100d4210 0x7f80100b7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:41.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.798+0000 7f801585a640 1 -- 172.21.15.32:0/3617757639 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80101f4db0 con 0x7f801015e9b0 2024-04-24T23:10:41.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.798+0000 7f8015059640 1 --2- 172.21.15.32:0/3617757639 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f801006b820 0x7f80100b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:10:41.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.798+0000 7f801585a640 1 --2- 172.21.15.32:0/3617757639 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f801015e9b0 0x7f80100cc870 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f8008007e40 tx=0x7f800800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:41.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.798+0000 7f8005ffb640 1 -- 172.21.15.32:0/3617757639 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8008014620 con 0x7f801015e9b0 2024-04-24T23:10:41.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.798+0000 7f8005ffb640 1 -- 172.21.15.32:0/3617757639 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8008019070 con 0x7f801015e9b0 2024-04-24T23:10:41.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.798+0000 7f80172bd640 1 -- 172.21.15.32:0/3617757639 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80101f7810 con 0x7f801015e9b0 2024-04-24T23:10:41.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.798+0000 7f8005ffb640 1 -- 172.21.15.32:0/3617757639 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8008014a20 con 0x7f801015e9b0 2024-04-24T23:10:41.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.798+0000 7f80172bd640 1 -- 172.21.15.32:0/3617757639 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80101f7dd0 con 0x7f801015e9b0 2024-04-24T23:10:41.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.799+0000 7f80172bd640 1 -- 172.21.15.32:0/3617757639 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8010068430 con 0x7f801015e9b0 2024-04-24T23:10:41.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.800+0000 7f8005ffb640 1 -- 172.21.15.32:0/3617757639 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8008014bc0 con 0x7f801015e9b0 2024-04-24T23:10:41.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.800+0000 7f8005ffb640 1 --2- 172.21.15.32:0/3617757639 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7ff406be30 0x7f7ff406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:41.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.800+0000 7f8005ffb640 1 -- 172.21.15.32:0/3617757639 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(185..185 src has 1..185) ==== 5709+0+0 (secure 0 0 0) 0x7f800808b6f0 con 0x7f801015e9b0 2024-04-24T23:10:41.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.800+0000 7f8015059640 1 --2- 172.21.15.32:0/3617757639 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7ff406be30 0x7f7ff406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:41.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.801+0000 7f8015059640 1 --2- 172.21.15.32:0/3617757639 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7ff406be30 0x7f7ff406e2f0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f800c000f60 tx=0x7f800c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:41.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:41.804+0000 7f8005ffb640 1 -- 172.21.15.32:0/3617757639 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f800805a130 con 0x7f801015e9b0 2024-04-24T23:10:41.960 INFO:tasks.ceph.mds.a.smithi032.stderr:2024-04-24T23:10:41.959+0000 7f51253bb640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T23:10:42.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.029+0000 7f80172bd640 1 -- 172.21.15.32:0/3617757639 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f801015bb30 con 0x7f801015e9b0 2024-04-24T23:10:42.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.029+0000 7f8005ffb640 1 -- 172.21.15.32:0/3617757639 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v80) ==== 53+0+73 (secure 0 0 0) 0x7f8008059a30 con 0x7f801015e9b0 2024-04-24T23:10:42.031 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T23:10:42.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.031+0000 7f80172bd640 1 -- 172.21.15.32:0/3617757639 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7ff406be30 msgr2=0x7f7ff406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:42.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.031+0000 7f80172bd640 1 --2- 172.21.15.32:0/3617757639 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7ff406be30 0x7f7ff406e2f0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f800c000f60 tx=0x7f800c009040 comp rx=0 tx=0).stop 2024-04-24T23:10:42.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.031+0000 7f80172bd640 1 -- 172.21.15.32:0/3617757639 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f801015e9b0 msgr2=0x7f80100cc870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:42.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.031+0000 7f80172bd640 1 --2- 172.21.15.32:0/3617757639 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f801015e9b0 0x7f80100cc870 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f8008007e40 tx=0x7f800800e4b0 comp rx=0 tx=0).stop 2024-04-24T23:10:42.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.031+0000 7f80172bd640 1 -- 172.21.15.32:0/3617757639 shutdown_connections 2024-04-24T23:10:42.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.031+0000 7f80172bd640 1 --2- 172.21.15.32:0/3617757639 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7ff406be30 0x7f7ff406e2f0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:42.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.031+0000 7f80172bd640 1 --2- 172.21.15.32:0/3617757639 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f801015e9b0 0x7f80100cc870 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:42.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.031+0000 7f80172bd640 1 --2- 172.21.15.32:0/3617757639 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80100d4210 0x7f80100b7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:42.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.031+0000 7f80172bd640 1 --2- 172.21.15.32:0/3617757639 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f801006b820 0x7f80100b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:42.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.031+0000 7f80172bd640 1 -- 172.21.15.32:0/3617757639 >> 172.21.15.32:0/3617757639 conn(0x7f80100d11d0 msgr2=0x7f8010155af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:42.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.031+0000 7f80172bd640 1 -- 172.21.15.32:0/3617757639 shutdown_connections 2024-04-24T23:10:42.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.031+0000 7f80172bd640 1 -- 172.21.15.32:0/3617757639 wait complete. 2024-04-24T23:10:42.044 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-24T23:10:42.044 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T23:10:42.044 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2024-04-24T23:10:42.044 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-24T23:10:42.044 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T23:10:42.044 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-24T23:10:42.044 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-24T23:10:42.044 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T23:10:42.044 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:10:42.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:10:42.045 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2024-04-24T23:10:42.061 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T23:10:42.061 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T23:10:42.061 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T23:10:42.061 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:10:42.062 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2024-04-24T23:10:42.062 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:10:42.062 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:10:42.062 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2024-04-24T23:10:42.062 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2024-04-24T23:10:42.062 INFO:teuthology.orchestra.run.smithi032.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:10:42.062 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2024-04-24T23:10:42.062 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2024-04-24T23:10:42.062 INFO:teuthology.orchestra.run.smithi032.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T23:10:42.062 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:10:42.062 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2024-04-24T23:10:42.063 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-24T23:10:42.063 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 42066sec preferred_lft 42066sec 2024-04-24T23:10:42.063 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link noprefixroute 2024-04-24T23:10:42.063 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:10:42.063 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:10:42.063 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:10:42.063 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2024-04-24T23:10:42.063 INFO:teuthology.orchestra.run.smithi032.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T23:10:42.063 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 72:d4:96:00:43:8c brd ff:ff:ff:ff:ff:ff 2024-04-24T23:10:42.063 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-24T23:10:42.063 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:10:42.063 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::f4c6:20ff:fe22:2647/64 scope link 2024-04-24T23:10:42.063 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:10:42.063 INFO:teuthology.orchestra.run.smithi032.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:10:42.063 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 72:d4:96:00:43:8c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T23:10:42.064 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::70d4:96ff:fe00:438c/64 scope link 2024-04-24T23:10:42.064 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:10:42.064 INFO:teuthology.orchestra.run.smithi032.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:10:42.064 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether e6:6b:2a:90:1a:f8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T23:10:42.064 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::e46b:2aff:fe90:1af8/64 scope link 2024-04-24T23:10:42.064 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:10:42.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:10:42.064 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T23:10:42.124 INFO:teuthology.orchestra.run.smithi032.stdout:1 2024-04-24T23:10:42.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:10:42.126 DEBUG:teuthology.orchestra.run.smithi032:> route 2024-04-24T23:10:42.183 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2024-04-24T23:10:42.183 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T23:10:42.183 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-24T23:10:42.183 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-24T23:10:42.183 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T23:10:42.183 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:10:42.183 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-24T23:10:42.184 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-24T23:10:42.184 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-24T23:10:42.184 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-24T23:10:42.184 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-24T23:10:42.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:10:42.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:10:42.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:10:42.310 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-04-24T23:10:42.325 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-24T23:10:42.325 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-24T23:10:42.326 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:10:42.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:10:42.326 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2024-04-24T23:10:42.389 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:10:42.390 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-24T23:10:42.445 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:10:42.446 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:10:42.446 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-24T23:10:42.501 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-24T23:10:42.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:10:42.502 DEBUG:teuthology.orchestra.run.smithi032:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:10:42.584 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:10:42.584 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-24T23:10:42.592 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:10:42.592 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-24T23:10:42.592 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.121:3300,172.21.15.121:3301" 2024-04-24T23:10:42.592 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: trying mount with new device syntax: 0@eb1b7a01-6f65-4534-9a12-83011862ba51.cephfs=/ 2024-04-24T23:10:42.592 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.32:3300/172.21.15.121:3300/172.21.15.121:3301" will pass to kernel 2024-04-24T23:10:42.593 INFO:teuthology.orchestra.run.smithi032.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-24T23:10:42.593 INFO:teuthology.orchestra.run.smithi032.stdout: You just mounted a file system that supports labels which does not 2024-04-24T23:10:42.593 INFO:teuthology.orchestra.run.smithi032.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-24T23:10:42.593 INFO:teuthology.orchestra.run.smithi032.stdout: applications will generate AVC messages and not be allowed access to 2024-04-24T23:10:42.593 INFO:teuthology.orchestra.run.smithi032.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-24T23:10:42.595 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-24T23:10:42.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:10:42.595 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-24T23:10:42.660 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:10:42.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:10:42.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:10:42.762 INFO:teuthology.orchestra.run.smithi032.stdout:client7876 2024-04-24T23:10:42.762 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:10:42.762 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:10:42.778 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T23:10:42.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.917+0000 7f274e7a9640 1 Processor -- start 2024-04-24T23:10:42.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.918+0000 7f274e7a9640 1 -- start start 2024-04-24T23:10:42.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.918+0000 7f274e7a9640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f274806b820 0x7f27480d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:42.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.918+0000 7f274e7a9640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27480d3170 con 0x7f27480d4210 2024-04-24T23:10:42.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.918+0000 7f274e7a9640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27480d3450 con 0x7f274815e9b0 2024-04-24T23:10:42.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.918+0000 7f274e7a9640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f27480d3730 con 0x7f274806b820 2024-04-24T23:10:42.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.918+0000 7f274cd46640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f27480d4210 0x7f27480b78f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58640/0 (socket says 172.21.15.32:58640) 2024-04-24T23:10:42.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.918+0000 7f274cd46640 1 -- 172.21.15.32:0/3052815919 learned_addr learned my addr 172.21.15.32:0/3052815919 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:10:42.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.919+0000 7f27477fe640 1 --2- 172.21.15.32:0/3052815919 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f274806b820 0x7f27480d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:42.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.919+0000 7f2746ffd640 1 -- 172.21.15.32:0/3052815919 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2306139824 0 0) 0x7f27480d3170 con 0x7f27480d4210 2024-04-24T23:10:42.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.919+0000 7f2746ffd640 1 -- 172.21.15.32:0/3052815919 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2730003520 con 0x7f27480d4210 2024-04-24T23:10:42.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.919+0000 7f27477fe640 1 -- 172.21.15.32:0/3052815919 >> v1:172.21.15.121:6789/0 conn(0x7f274815e9b0 legacy=0x7f274806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:42.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.919+0000 7f27477fe640 1 -- 172.21.15.32:0/3052815919 >> v1:172.21.15.32:6789/0 conn(0x7f27480d4210 legacy=0x7f27480b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:42.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.919+0000 7f27477fe640 1 -- 172.21.15.32:0/3052815919 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27480d3a10 con 0x7f274806b820 2024-04-24T23:10:42.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.920+0000 7f27477fe640 1 --2- 172.21.15.32:0/3052815919 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f274806b820 0x7f27480d2b90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f273c0097d0 tx=0x7f273c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff628546bcb1b39f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:42.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.920+0000 7f2746ffd640 1 -- 172.21.15.32:0/3052815919 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273c037070 con 0x7f274806b820 2024-04-24T23:10:42.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.920+0000 7f2746ffd640 1 -- 172.21.15.32:0/3052815919 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f273c032440 con 0x7f274806b820 2024-04-24T23:10:42.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.920+0000 7f2746ffd640 1 -- 172.21.15.32:0/3052815919 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273c0327f0 con 0x7f274806b820 2024-04-24T23:10:42.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.920+0000 7f274e7a9640 1 -- 172.21.15.32:0/3052815919 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f274806b820 msgr2=0x7f27480d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:42.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.920+0000 7f274e7a9640 1 --2- 172.21.15.32:0/3052815919 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f274806b820 0x7f27480d2b90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f273c0097d0 tx=0x7f273c02a210 comp rx=0 tx=0).stop 2024-04-24T23:10:42.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.920+0000 7f274e7a9640 1 -- 172.21.15.32:0/3052815919 shutdown_connections 2024-04-24T23:10:42.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.920+0000 7f274e7a9640 1 --2- 172.21.15.32:0/3052815919 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f274806b820 0x7f27480d2b90 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:42.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.920+0000 7f274e7a9640 1 -- 172.21.15.32:0/3052815919 >> 172.21.15.32:0/3052815919 conn(0x7f27480d11d0 msgr2=0x7f27480d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:42.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.920+0000 7f274e7a9640 1 -- 172.21.15.32:0/3052815919 shutdown_connections 2024-04-24T23:10:42.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.920+0000 7f274e7a9640 1 -- 172.21.15.32:0/3052815919 wait complete. 2024-04-24T23:10:42.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.922+0000 7f274e7a9640 1 Processor -- start 2024-04-24T23:10:42.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.922+0000 7f274e7a9640 1 -- start start 2024-04-24T23:10:42.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.922+0000 7f274e7a9640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f274806b820 0x7f274814ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:42.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.922+0000 7f274e7a9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27480d4210 0x7f2748160da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:42.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.922+0000 7f274e7a9640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f274815e9b0 0x7f27481612e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:42.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.922+0000 7f274e7a9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f27480ba0e0 con 0x7f27480d4210 2024-04-24T23:10:42.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.922+0000 7f274e7a9640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f27480d2400 con 0x7f274806b820 2024-04-24T23:10:42.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.922+0000 7f274e7a9640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f27480ba260 con 0x7f274815e9b0 2024-04-24T23:10:42.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.923+0000 7f27477fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27480d4210 0x7f2748160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:42.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.923+0000 7f27477fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27480d4210 0x7f2748160da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48858/0 (socket says 172.21.15.32:48858) 2024-04-24T23:10:42.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.923+0000 7f27477fe640 1 -- 172.21.15.32:0/1359535292 learned_addr learned my addr 172.21.15.32:0/1359535292 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:42.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.923+0000 7f274cd46640 1 --2- 172.21.15.32:0/1359535292 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f274815e9b0 0x7f27481612e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:42.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.923+0000 7f27477fe640 1 -- 172.21.15.32:0/1359535292 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f274815e9b0 msgr2=0x7f27481612e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:42.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.923+0000 7f2747fff640 1 --2- 172.21.15.32:0/1359535292 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f274806b820 0x7f274814ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:42.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.923+0000 7f27477fe640 1 --2- 172.21.15.32:0/1359535292 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f274815e9b0 0x7f27481612e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:42.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.923+0000 7f27477fe640 1 -- 172.21.15.32:0/1359535292 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f274806b820 msgr2=0x7f274814ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:42.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.923+0000 7f27477fe640 1 --2- 172.21.15.32:0/1359535292 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f274806b820 0x7f274814ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:42.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.923+0000 7f27477fe640 1 -- 172.21.15.32:0/1359535292 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2748156dc0 con 0x7f27480d4210 2024-04-24T23:10:42.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.923+0000 7f2747fff640 1 --2- 172.21.15.32:0/1359535292 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f274806b820 0x7f274814ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:10:42.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.923+0000 7f27477fe640 1 --2- 172.21.15.32:0/1359535292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27480d4210 0x7f2748160da0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f273c035040 tx=0x7f273c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:42.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.923+0000 7f2744ff9640 1 -- 172.21.15.32:0/1359535292 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273c037050 con 0x7f27480d4210 2024-04-24T23:10:42.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.923+0000 7f2744ff9640 1 -- 172.21.15.32:0/1359535292 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f273c038040 con 0x7f27480d4210 2024-04-24T23:10:42.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.923+0000 7f274e7a9640 1 -- 172.21.15.32:0/1359535292 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27480ca000 con 0x7f27480d4210 2024-04-24T23:10:42.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.923+0000 7f2744ff9640 1 -- 172.21.15.32:0/1359535292 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273c004490 con 0x7f27480d4210 2024-04-24T23:10:42.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.923+0000 7f274e7a9640 1 -- 172.21.15.32:0/1359535292 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27480ca5c0 con 0x7f27480d4210 2024-04-24T23:10:42.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.924+0000 7f274e7a9640 1 -- 172.21.15.32:0/1359535292 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2748068430 con 0x7f27480d4210 2024-04-24T23:10:42.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.925+0000 7f2744ff9640 1 -- 172.21.15.32:0/1359535292 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f273c03c6d0 con 0x7f27480d4210 2024-04-24T23:10:42.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.925+0000 7f2744ff9640 1 --2- 172.21.15.32:0/1359535292 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f273406be30 0x7f273406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:42.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.925+0000 7f2744ff9640 1 -- 172.21.15.32:0/1359535292 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(185..185 src has 1..185) ==== 5709+0+0 (secure 0 0 0) 0x7f273c0ab810 con 0x7f27480d4210 2024-04-24T23:10:42.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.926+0000 7f2747fff640 1 --2- 172.21.15.32:0/1359535292 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f273406be30 0x7f273406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:42.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.926+0000 7f2747fff640 1 --2- 172.21.15.32:0/1359535292 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f273406be30 0x7f273406e2f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f27480b61a0 tx=0x7f2738009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:42.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:42.930+0000 7f2744ff9640 1 -- 172.21.15.32:0/1359535292 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f273c07a250 con 0x7f27480d4210 2024-04-24T23:10:43.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:43.152+0000 7f274e7a9640 1 -- 172.21.15.32:0/1359535292 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2748074df0 con 0x7f27480d4210 2024-04-24T23:10:43.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:43.153+0000 7f2744ff9640 1 -- 172.21.15.32:0/1359535292 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v80) ==== 53+0+73 (secure 0 0 0) 0x7f273c079b50 con 0x7f27480d4210 2024-04-24T23:10:43.154 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T23:10:43.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:43.154+0000 7f274e7a9640 1 -- 172.21.15.32:0/1359535292 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f273406be30 msgr2=0x7f273406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:43.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:43.154+0000 7f274e7a9640 1 --2- 172.21.15.32:0/1359535292 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f273406be30 0x7f273406e2f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f27480b61a0 tx=0x7f2738009040 comp rx=0 tx=0).stop 2024-04-24T23:10:43.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:43.154+0000 7f274e7a9640 1 -- 172.21.15.32:0/1359535292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27480d4210 msgr2=0x7f2748160da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:43.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:43.154+0000 7f274e7a9640 1 --2- 172.21.15.32:0/1359535292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27480d4210 0x7f2748160da0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f273c035040 tx=0x7f273c0046c0 comp rx=0 tx=0).stop 2024-04-24T23:10:43.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:43.154+0000 7f274e7a9640 1 -- 172.21.15.32:0/1359535292 shutdown_connections 2024-04-24T23:10:43.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:43.154+0000 7f274e7a9640 1 --2- 172.21.15.32:0/1359535292 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f273406be30 0x7f273406e2f0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:43.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:43.154+0000 7f274e7a9640 1 --2- 172.21.15.32:0/1359535292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27480d4210 0x7f2748160da0 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:43.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:43.154+0000 7f274e7a9640 1 --2- 172.21.15.32:0/1359535292 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f274815e9b0 0x7f27481612e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:43.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:43.154+0000 7f274e7a9640 1 --2- 172.21.15.32:0/1359535292 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f274806b820 0x7f274814ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:43.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:43.154+0000 7f274e7a9640 1 -- 172.21.15.32:0/1359535292 >> 172.21.15.32:0/1359535292 conn(0x7f27480d11d0 msgr2=0x7f27480b89f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:43.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:43.155+0000 7f274e7a9640 1 -- 172.21.15.32:0/1359535292 shutdown_connections 2024-04-24T23:10:43.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:43.155+0000 7f274e7a9640 1 -- 172.21.15.32:0/1359535292 wait complete. 2024-04-24T23:10:43.167 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-24T23:10:43.167 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T23:10:43.167 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi121.front.sepia.ceph.com 2024-04-24T23:10:43.167 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-24T23:10:43.167 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-24T23:10:43.167 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-24T23:10:43.167 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-24T23:10:43.167 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T23:10:43.167 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-24T23:10:43.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:10:43.168 DEBUG:teuthology.orchestra.run.smithi121:> ip addr 2024-04-24T23:10:43.186 INFO:teuthology.orchestra.run.smithi121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T23:10:43.186 INFO:teuthology.orchestra.run.smithi121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T23:10:43.187 INFO:teuthology.orchestra.run.smithi121.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T23:10:43.187 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:10:43.187 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 ::1/128 scope host 2024-04-24T23:10:43.187 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:10:43.187 INFO:teuthology.orchestra.run.smithi121.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:10:43.187 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:82 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:10:43.187 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f0 2024-04-24T23:10:43.187 INFO:teuthology.orchestra.run.smithi121.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:10:43.187 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:83 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:10:43.187 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f1 2024-04-24T23:10:43.187 INFO:teuthology.orchestra.run.smithi121.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:10:43.187 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a2 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:10:43.187 INFO:teuthology.orchestra.run.smithi121.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T23:10:43.188 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a3 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:10:43.188 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.21.15.121/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-24T23:10:43.188 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft 42088sec preferred_lft 42088sec 2024-04-24T23:10:43.188 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::93dd:8489:4369:e127/64 scope link noprefixroute 2024-04-24T23:10:43.188 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:10:43.188 INFO:teuthology.orchestra.run.smithi121.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T23:10:43.188 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether c6:c0:c7:45:ae:ba brd ff:ff:ff:ff:ff:ff 2024-04-24T23:10:43.188 INFO:teuthology.orchestra.run.smithi121.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-24T23:10:43.188 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:10:43.188 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::80fe:d2ff:fe13:d1/64 scope link 2024-04-24T23:10:43.188 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:10:43.188 INFO:teuthology.orchestra.run.smithi121.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:10:43.188 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether e2:ca:b9:e9:9e:bb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T23:10:43.188 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::e0ca:b9ff:fee9:9ebb/64 scope link 2024-04-24T23:10:43.188 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:10:43.189 INFO:teuthology.orchestra.run.smithi121.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:10:43.189 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether c6:c0:c7:45:ae:ba brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-24T23:10:43.189 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::c4c0:c7ff:fe45:aeba/64 scope link 2024-04-24T23:10:43.189 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:10:43.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:10:43.189 DEBUG:teuthology.orchestra.run.smithi121:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T23:10:43.253 INFO:teuthology.orchestra.run.smithi121.stdout:1 2024-04-24T23:10:43.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:10:43.255 DEBUG:teuthology.orchestra.run.smithi121:> route 2024-04-24T23:10:43.313 INFO:teuthology.orchestra.run.smithi121.stdout:Kernel IP routing table 2024-04-24T23:10:43.314 INFO:teuthology.orchestra.run.smithi121.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T23:10:43.314 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-24T23:10:43.314 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-24T23:10:43.314 INFO:teuthology.orchestra.run.smithi121.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T23:10:43.314 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:10:43.314 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2024-04-24T23:10:43.314 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-24T23:10:43.314 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-24T23:10:43.315 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-24T23:10:43.315 DEBUG:teuthology.orchestra.run.smithi121:> ') 2024-04-24T23:10:43.407 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T23:10:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:10:43.438 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T23:10:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:10:43.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:10:43.445 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2024-04-24T23:10:43.460 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-24T23:10:43.460 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-24T23:10:43.460 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-24T23:10:43.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:10:43.460 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set brx.0 up 2024-04-24T23:10:43.524 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:10:43.524 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-24T23:10:43.579 INFO:teuthology.orchestra.run.smithi121.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-24T23:10:43.580 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:10:43.580 DEBUG:teuthology.orchestra.run.smithi121:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-24T23:10:43.634 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-24T23:10:43.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:10:43.634 DEBUG:teuthology.orchestra.run.smithi121:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:10:43.715 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:10:43.716 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-24T23:10:43.722 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T23:10:43.722 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-24T23:10:43.722 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.121:3300,172.21.15.121:3301" 2024-04-24T23:10:43.722 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: trying mount with new device syntax: 1@eb1b7a01-6f65-4534-9a12-83011862ba51.cephfs=/ 2024-04-24T23:10:43.722 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.32:3300/172.21.15.121:3300/172.21.15.121:3301" will pass to kernel 2024-04-24T23:10:43.723 INFO:teuthology.orchestra.run.smithi121.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-24T23:10:43.723 INFO:teuthology.orchestra.run.smithi121.stdout: You just mounted a file system that supports labels which does not 2024-04-24T23:10:43.723 INFO:teuthology.orchestra.run.smithi121.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-24T23:10:43.723 INFO:teuthology.orchestra.run.smithi121.stdout: applications will generate AVC messages and not be allowed access to 2024-04-24T23:10:43.723 INFO:teuthology.orchestra.run.smithi121.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-24T23:10:43.725 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-24T23:10:43.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:10:43.725 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-24T23:10:43.789 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:10:43.877 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T23:10:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:10:43.878 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-24T23:10:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:10:43.884 INFO:teuthology.orchestra.run.smithi121.stdout:client7040 2024-04-24T23:10:43.884 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:10:43.884 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:10:43.899 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:10:44.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.012+0000 7f712dbc8640 1 Processor -- start 2024-04-24T23:10:44.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.013+0000 7f712dbc8640 1 -- start start 2024-04-24T23:10:44.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.013+0000 7f712dbc8640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f712806b820 0x7f71280d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:44.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.013+0000 7f712dbc8640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f71280d4210 0x7f71280b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:44.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.013+0000 7f712dbc8640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71280d3170 con 0x7f712815e9b0 2024-04-24T23:10:44.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.013+0000 7f712dbc8640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f71280d3450 con 0x7f71280d4210 2024-04-24T23:10:44.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.013+0000 7f712dbc8640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f71280d3730 con 0x7f712806b820 2024-04-24T23:10:44.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.013+0000 7f71277fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f712815e9b0 0x7f712806b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58654/0 (socket says 172.21.15.32:58654) 2024-04-24T23:10:44.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.013+0000 7f71277fe640 1 -- 172.21.15.32:0/504778878 learned_addr learned my addr 172.21.15.32:0/504778878 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:10:44.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.014+0000 7f7127fff640 1 --2- 172.21.15.32:0/504778878 >> v2:172.21.15.121:3300/0 conn(0x7f71280d4210 0x7f71280b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:44.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.014+0000 7f7126ffd640 1 --2- 172.21.15.32:0/504778878 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f712806b820 0x7f71280d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:44.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.014+0000 7f71267fc640 1 -- 172.21.15.32:0/504778878 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1591475272 0 0) 0x7f71280d3170 con 0x7f712815e9b0 2024-04-24T23:10:44.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.014+0000 7f71267fc640 1 -- 172.21.15.32:0/504778878 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7110003520 con 0x7f712815e9b0 2024-04-24T23:10:44.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.014+0000 7f71267fc640 1 -- 172.21.15.32:0/504778878 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2028788776 0 0) 0x7f7110003520 con 0x7f712815e9b0 2024-04-24T23:10:44.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.014+0000 7f71267fc640 1 -- 172.21.15.32:0/504778878 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f712806b820 msgr2=0x7f71280d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:44.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.014+0000 7f71267fc640 1 --2- 172.21.15.32:0/504778878 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f712806b820 0x7f71280d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:44.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.014+0000 7f71267fc640 1 -- 172.21.15.32:0/504778878 >> v2:172.21.15.121:3300/0 conn(0x7f71280d4210 msgr2=0x7f71280b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:44.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.014+0000 7f71267fc640 1 --2- 172.21.15.32:0/504778878 >> v2:172.21.15.121:3300/0 conn(0x7f71280d4210 0x7f71280b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:44.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.014+0000 7f71267fc640 1 -- 172.21.15.32:0/504778878 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71280d3a10 con 0x7f712815e9b0 2024-04-24T23:10:44.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.014+0000 7f71267fc640 1 -- 172.21.15.32:0/504778878 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f7118001670 con 0x7f712815e9b0 2024-04-24T23:10:44.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.014+0000 7f71267fc640 1 -- 172.21.15.32:0/504778878 >> v1:172.21.15.32:6789/0 conn(0x7f712815e9b0 legacy=0x7f712806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:44.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.014+0000 7f71267fc640 1 --2- 172.21.15.32:0/504778878 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71100039b0 0x7f7110003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:44.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.014+0000 7f71267fc640 1 --2- 172.21.15.32:0/504778878 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f712806b820 0x7f7110004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:44.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.014+0000 7f71267fc640 1 --2- 172.21.15.32:0/504778878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7110004d40 0x7f71100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:44.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.014+0000 7f71267fc640 1 -- 172.21.15.32:0/504778878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f71280d3450 con 0x7f7110004d40 2024-04-24T23:10:44.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.014+0000 7f71267fc640 1 -- 172.21.15.32:0/504778878 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f71280d3730 con 0x7f71100039b0 2024-04-24T23:10:44.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.014+0000 7f71267fc640 1 -- 172.21.15.32:0/504778878 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f711000b9c0 con 0x7f712806b820 2024-04-24T23:10:44.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.014+0000 7f71277fe640 1 --2- 172.21.15.32:0/504778878 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71100039b0 0x7f7110003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:44.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.014+0000 7f7126ffd640 1 --2- 172.21.15.32:0/504778878 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f712806b820 0x7f7110004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:44.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.014+0000 7f7127fff640 1 --2- 172.21.15.32:0/504778878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7110004d40 0x7f71100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:44.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.015+0000 7f71277fe640 1 -- 172.21.15.32:0/504778878 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f712806b820 msgr2=0x7f7110004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:44.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.015+0000 7f71277fe640 1 --2- 172.21.15.32:0/504778878 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f712806b820 0x7f7110004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:44.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.015+0000 7f71277fe640 1 -- 172.21.15.32:0/504778878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7110004d40 msgr2=0x7f71100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:44.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.015+0000 7f71277fe640 1 --2- 172.21.15.32:0/504778878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7110004d40 0x7f71100051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:44.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.015+0000 7f71277fe640 1 -- 172.21.15.32:0/504778878 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7110003520 con 0x7f71100039b0 2024-04-24T23:10:44.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.015+0000 7f7127fff640 1 --2- 172.21.15.32:0/504778878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7110004d40 0x7f71100051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:10:44.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.015+0000 7f71277fe640 1 --2- 172.21.15.32:0/504778878 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71100039b0 0x7f7110003dd0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f711800c240 tx=0x7f7110005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2f977592432739f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:44.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.016+0000 7f71267fc640 1 -- 172.21.15.32:0/504778878 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7118007d30 con 0x7f71100039b0 2024-04-24T23:10:44.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.016+0000 7f71267fc640 1 -- 172.21.15.32:0/504778878 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7118005ad0 con 0x7f71100039b0 2024-04-24T23:10:44.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.016+0000 7f71267fc640 1 -- 172.21.15.32:0/504778878 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7118007070 con 0x7f71100039b0 2024-04-24T23:10:44.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.016+0000 7f712dbc8640 1 -- 172.21.15.32:0/504778878 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71100039b0 msgr2=0x7f7110003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:44.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.016+0000 7f712dbc8640 1 --2- 172.21.15.32:0/504778878 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71100039b0 0x7f7110003dd0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f711800c240 tx=0x7f7110005830 comp rx=0 tx=0).stop 2024-04-24T23:10:44.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.016+0000 7f7127fff640 1 -- 172.21.15.32:0/504778878 reap_dead start 2024-04-24T23:10:44.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.016+0000 7f712dbc8640 1 -- 172.21.15.32:0/504778878 shutdown_connections 2024-04-24T23:10:44.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.016+0000 7f712dbc8640 1 -- 172.21.15.32:0/504778878 >> 172.21.15.32:0/504778878 conn(0x7f71280d11d0 msgr2=0x7f71280d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:44.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.016+0000 7f712dbc8640 1 -- 172.21.15.32:0/504778878 shutdown_connections 2024-04-24T23:10:44.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.016+0000 7f712dbc8640 1 -- 172.21.15.32:0/504778878 wait complete. 2024-04-24T23:10:44.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.017+0000 7f712dbc8640 1 Processor -- start 2024-04-24T23:10:44.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.018+0000 7f712dbc8640 1 -- start start 2024-04-24T23:10:44.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.018+0000 7f712dbc8640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71100039b0 0x7f7128151ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:44.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.018+0000 7f712dbc8640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71281570d0 0x7f7128157550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:44.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.018+0000 7f712dbc8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f712815bc20 0x7f712815c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:44.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.018+0000 7f712dbc8640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f71280ba0e0 con 0x7f712815bc20 2024-04-24T23:10:44.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.018+0000 7f712dbc8640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f71280d2400 con 0x7f71100039b0 2024-04-24T23:10:44.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.018+0000 7f712dbc8640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f71280ba260 con 0x7f71281570d0 2024-04-24T23:10:44.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.018+0000 7f7127fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f712815bc20 0x7f712815c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:44.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.018+0000 7f7127fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f712815bc20 0x7f712815c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48882/0 (socket says 172.21.15.32:48882) 2024-04-24T23:10:44.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.018+0000 7f7127fff640 1 -- 172.21.15.32:0/461291168 learned_addr learned my addr 172.21.15.32:0/461291168 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:44.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.019+0000 7f71277fe640 1 --2- 172.21.15.32:0/461291168 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71100039b0 0x7f7128151ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:44.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.019+0000 7f7126ffd640 1 --2- 172.21.15.32:0/461291168 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71281570d0 0x7f7128157550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:44.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.019+0000 7f7127fff640 1 -- 172.21.15.32:0/461291168 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71281570d0 msgr2=0x7f7128157550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:44.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.019+0000 7f7127fff640 1 --2- 172.21.15.32:0/461291168 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71281570d0 0x7f7128157550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:44.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.019+0000 7f7127fff640 1 -- 172.21.15.32:0/461291168 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71100039b0 msgr2=0x7f7128151ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:44.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.019+0000 7f7127fff640 1 --2- 172.21.15.32:0/461291168 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71100039b0 0x7f7128151ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:44.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.019+0000 7f7127fff640 1 -- 172.21.15.32:0/461291168 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71281f8370 con 0x7f712815bc20 2024-04-24T23:10:44.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.019+0000 7f7126ffd640 1 --2- 172.21.15.32:0/461291168 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71281570d0 0x7f7128157550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:10:44.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.019+0000 7f7127fff640 1 --2- 172.21.15.32:0/461291168 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f712815bc20 0x7f712815c0a0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f712806bdf0 tx=0x7f712000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:44.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.019+0000 7f710ffff640 1 -- 172.21.15.32:0/461291168 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7120009c10 con 0x7f712815bc20 2024-04-24T23:10:44.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.019+0000 7f710ffff640 1 -- 172.21.15.32:0/461291168 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7120003990 con 0x7f712815bc20 2024-04-24T23:10:44.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.019+0000 7f712dbc8640 1 -- 172.21.15.32:0/461291168 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71281f75b0 con 0x7f712815bc20 2024-04-24T23:10:44.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.019+0000 7f710ffff640 1 -- 172.21.15.32:0/461291168 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7120003d80 con 0x7f712815bc20 2024-04-24T23:10:44.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.019+0000 7f712dbc8640 1 -- 172.21.15.32:0/461291168 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71281f7b70 con 0x7f712815bc20 2024-04-24T23:10:44.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.020+0000 7f712dbc8640 1 -- 172.21.15.32:0/461291168 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7128068430 con 0x7f712815bc20 2024-04-24T23:10:44.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.021+0000 7f710ffff640 1 -- 172.21.15.32:0/461291168 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7120003f20 con 0x7f712815bc20 2024-04-24T23:10:44.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.021+0000 7f710ffff640 1 --2- 172.21.15.32:0/461291168 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f711406be30 0x7f711406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:44.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.021+0000 7f710ffff640 1 -- 172.21.15.32:0/461291168 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(185..185 src has 1..185) ==== 5709+0+0 (secure 0 0 0) 0x7f712008b870 con 0x7f712815bc20 2024-04-24T23:10:44.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.021+0000 7f71277fe640 1 --2- 172.21.15.32:0/461291168 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f711406be30 0x7f711406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:44.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.022+0000 7f71277fe640 1 --2- 172.21.15.32:0/461291168 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f711406be30 0x7f711406e2f0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f7118033000 tx=0x7f7118009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:44.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.025+0000 7f710ffff640 1 -- 172.21.15.32:0/461291168 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7120019030 con 0x7f712815bc20 2024-04-24T23:10:44.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.248+0000 7f712dbc8640 1 -- 172.21.15.32:0/461291168 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f71281526b0 con 0x7f712815bc20 2024-04-24T23:10:44.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.248+0000 7f710ffff640 1 -- 172.21.15.32:0/461291168 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) ==== 94+0+5185 (secure 0 0 0) 0x7f7120010040 con 0x7f712815bc20 2024-04-24T23:10:44.250 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:10:44.250 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":80,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6941,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/416518156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":416518156},{"type":"v1","addr":"172.21.15.121:6837","nonce":416518156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7762,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1506667233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1506667233},{"type":"v1","addr":"172.21.15.121:6835","nonce":1506667233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":9063,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/2215079101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":2215079101},{"type":"v1","addr":"172.21.15.32:6834","nonce":2215079101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:10:36.353761+0000","modified":"2024-04-24T23:10:38.436407+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9066},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9066":{"gid":9066,"name":"a","rank":0,"incarnation":78,"state":"up:active","state_seq":4,"addr":"172.21.15.32:6835/901353412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":901353412},{"type":"v1","addr":"172.21.15.32:6835","nonce":901353412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9066,"qdb_cluster":[9066]},"id":11}]} 2024-04-24T23:10:44.251 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 80 2024-04-24T23:10:44.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.250+0000 7f712dbc8640 1 -- 172.21.15.32:0/461291168 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f711406be30 msgr2=0x7f711406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:44.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.250+0000 7f712dbc8640 1 --2- 172.21.15.32:0/461291168 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f711406be30 0x7f711406e2f0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f7118033000 tx=0x7f7118009030 comp rx=0 tx=0).stop 2024-04-24T23:10:44.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.250+0000 7f712dbc8640 1 -- 172.21.15.32:0/461291168 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f712815bc20 msgr2=0x7f712815c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:44.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.250+0000 7f712dbc8640 1 --2- 172.21.15.32:0/461291168 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f712815bc20 0x7f712815c0a0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f712806bdf0 tx=0x7f712000f580 comp rx=0 tx=0).stop 2024-04-24T23:10:44.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.250+0000 7f712dbc8640 1 -- 172.21.15.32:0/461291168 shutdown_connections 2024-04-24T23:10:44.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.250+0000 7f712dbc8640 1 --2- 172.21.15.32:0/461291168 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f711406be30 0x7f711406e2f0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:44.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.250+0000 7f712dbc8640 1 --2- 172.21.15.32:0/461291168 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f712815bc20 0x7f712815c0a0 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:44.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.250+0000 7f712dbc8640 1 --2- 172.21.15.32:0/461291168 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71281570d0 0x7f7128157550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:44.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.250+0000 7f712dbc8640 1 --2- 172.21.15.32:0/461291168 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71100039b0 0x7f7128151ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:44.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.250+0000 7f712dbc8640 1 -- 172.21.15.32:0/461291168 >> 172.21.15.32:0/461291168 conn(0x7f71280d11d0 msgr2=0x7f7128071730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:44.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.250+0000 7f712dbc8640 1 -- 172.21.15.32:0/461291168 shutdown_connections 2024-04-24T23:10:44.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.250+0000 7f712dbc8640 1 -- 172.21.15.32:0/461291168 wait complete. 2024-04-24T23:10:44.263 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-04-24T23:10:44.383 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","whoami":0,"id":9066,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":7.3793606839999999,"mdsmap_epoch":80,"osdmap_epoch":185,"osdmap_epoch_barrier":185,"uptime":18.407876724000001} 2024-04-24T23:10:44.383 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", "fs_name": "cephfs", "id": 9066, "mdsmap_epoch": 80, "osdmap_epoch": 185, "osdmap_epoch_barrier": 185, "rank_uptime": 7.379360684, "state": "up:active", "uptime": 18.407876724, "want_state": "up:active", "whoami": 0 } 2024-04-24T23:10:44.383 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2024-04-24T23:10:44.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.486+0000 7f9feaba2640 1 Processor -- start 2024-04-24T23:10:44.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.486+0000 7f9feaba2640 1 -- start start 2024-04-24T23:10:44.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.487+0000 7f9feaba2640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9fe40b5ff0 0x7f9fe40cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:44.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.487+0000 7f9feaba2640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fe415bde0 con 0x7f9fe415b960 2024-04-24T23:10:44.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.487+0000 7f9feaba2640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fe415c0c0 con 0x7f9fe415b580 2024-04-24T23:10:44.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.487+0000 7f9feaba2640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f9fe415c9f0 con 0x7f9fe40b5ff0 2024-04-24T23:10:44.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.487+0000 7f9fe3fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9fe415b960 0x7f9fe40b58e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58664/0 (socket says 172.21.15.32:58664) 2024-04-24T23:10:44.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.487+0000 7f9fe3fff640 1 -- 172.21.15.32:0/3806067926 learned_addr learned my addr 172.21.15.32:0/3806067926 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:10:44.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.487+0000 7f9fe913f640 1 --2- 172.21.15.32:0/3806067926 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9fe40b5ff0 0x7f9fe40cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:44.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.487+0000 7f9fe37fe640 1 -- 172.21.15.32:0/3806067926 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2942501480 0 0) 0x7f9fe415bde0 con 0x7f9fe415b960 2024-04-24T23:10:44.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.487+0000 7f9fe37fe640 1 -- 172.21.15.32:0/3806067926 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9fd0003520 con 0x7f9fe415b960 2024-04-24T23:10:44.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.487+0000 7f9fe37fe640 1 -- 172.21.15.32:0/3806067926 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3441646824 0 0) 0x7f9fe415c0c0 con 0x7f9fe415b580 2024-04-24T23:10:44.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.487+0000 7f9fe37fe640 1 -- 172.21.15.32:0/3806067926 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9fe415bde0 con 0x7f9fe415b580 2024-04-24T23:10:44.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.488+0000 7f9fe913f640 1 -- 172.21.15.32:0/3806067926 >> v1:172.21.15.121:6789/0 conn(0x7f9fe415b580 legacy=0x7f9fe406cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:44.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.488+0000 7f9fe913f640 1 -- 172.21.15.32:0/3806067926 >> v1:172.21.15.32:6789/0 conn(0x7f9fe415b960 legacy=0x7f9fe40b58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:44.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.488+0000 7f9fe913f640 1 -- 172.21.15.32:0/3806067926 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fe415c3a0 con 0x7f9fe40b5ff0 2024-04-24T23:10:44.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.488+0000 7f9fe913f640 1 --2- 172.21.15.32:0/3806067926 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9fe40b5ff0 0x7f9fe40cde80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc0097d0 tx=0x7f9fcc02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d04a489438a4baa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:44.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.488+0000 7f9fe37fe640 1 -- 172.21.15.32:0/3806067926 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc037070 con 0x7f9fe40b5ff0 2024-04-24T23:10:44.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.488+0000 7f9fe37fe640 1 -- 172.21.15.32:0/3806067926 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fcc032440 con 0x7f9fe40b5ff0 2024-04-24T23:10:44.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.488+0000 7f9fe37fe640 1 -- 172.21.15.32:0/3806067926 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc032810 con 0x7f9fe40b5ff0 2024-04-24T23:10:44.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.489+0000 7f9feaba2640 1 -- 172.21.15.32:0/3806067926 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9fe40b5ff0 msgr2=0x7f9fe40cde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:44.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.489+0000 7f9feaba2640 1 --2- 172.21.15.32:0/3806067926 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9fe40b5ff0 0x7f9fe40cde80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc0097d0 tx=0x7f9fcc02a210 comp rx=0 tx=0).stop 2024-04-24T23:10:44.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.489+0000 7f9feaba2640 1 -- 172.21.15.32:0/3806067926 shutdown_connections 2024-04-24T23:10:44.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.489+0000 7f9feaba2640 1 --2- 172.21.15.32:0/3806067926 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9fe40b5ff0 0x7f9fe40cde80 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:44.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.489+0000 7f9feaba2640 1 -- 172.21.15.32:0/3806067926 >> 172.21.15.32:0/3806067926 conn(0x7f9fe40b1f10 msgr2=0x7f9fe40b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:44.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.489+0000 7f9feaba2640 1 -- 172.21.15.32:0/3806067926 shutdown_connections 2024-04-24T23:10:44.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.489+0000 7f9feaba2640 1 -- 172.21.15.32:0/3806067926 wait complete. 2024-04-24T23:10:44.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.490+0000 7f9feaba2640 1 Processor -- start 2024-04-24T23:10:44.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.490+0000 7f9feaba2640 1 -- start start 2024-04-24T23:10:44.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.491+0000 7f9feaba2640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9fe40b5ff0 0x7f9fe40b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:44.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.491+0000 7f9feaba2640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9fe415b580 0x7f9fe40b3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:44.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.491+0000 7f9feaba2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fe415b960 0x7f9fe40b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:44.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.491+0000 7f9feaba2640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe406c7a0 con 0x7f9fe415b960 2024-04-24T23:10:44.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.491+0000 7f9feaba2640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe40ce3c0 con 0x7f9fe40b5ff0 2024-04-24T23:10:44.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.491+0000 7f9feaba2640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f9fe406c920 con 0x7f9fe415b580 2024-04-24T23:10:44.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.491+0000 7f9fe913f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fe415b960 0x7f9fe40b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:44.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.491+0000 7f9fe913f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fe415b960 0x7f9fe40b3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48890/0 (socket says 172.21.15.32:48890) 2024-04-24T23:10:44.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.491+0000 7f9fe913f640 1 -- 172.21.15.32:0/2646074873 learned_addr learned my addr 172.21.15.32:0/2646074873 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:44.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.491+0000 7f9fe3fff640 1 --2- 172.21.15.32:0/2646074873 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9fe415b580 0x7f9fe40b3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:44.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.491+0000 7f9fe893e640 1 --2- 172.21.15.32:0/2646074873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9fe40b5ff0 0x7f9fe40b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:44.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.491+0000 7f9fe913f640 1 -- 172.21.15.32:0/2646074873 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9fe415b580 msgr2=0x7f9fe40b3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:44.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.491+0000 7f9fe913f640 1 --2- 172.21.15.32:0/2646074873 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9fe415b580 0x7f9fe40b3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:44.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.491+0000 7f9fe913f640 1 -- 172.21.15.32:0/2646074873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9fe40b5ff0 msgr2=0x7f9fe40b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:44.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.491+0000 7f9fe913f640 1 --2- 172.21.15.32:0/2646074873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9fe40b5ff0 0x7f9fe40b2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:44.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.491+0000 7f9fe913f640 1 -- 172.21.15.32:0/2646074873 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fe4157580 con 0x7f9fe415b960 2024-04-24T23:10:44.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.492+0000 7f9fe913f640 1 --2- 172.21.15.32:0/2646074873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fe415b960 0x7f9fe40b3a40 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc035040 tx=0x7f9fcc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:44.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.492+0000 7f9fe17fa640 1 -- 172.21.15.32:0/2646074873 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc041020 con 0x7f9fe415b960 2024-04-24T23:10:44.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.492+0000 7f9fe17fa640 1 -- 172.21.15.32:0/2646074873 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fcc03d640 con 0x7f9fe415b960 2024-04-24T23:10:44.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.492+0000 7f9feaba2640 1 -- 172.21.15.32:0/2646074873 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fe4158f40 con 0x7f9fe415b960 2024-04-24T23:10:44.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.492+0000 7f9fe17fa640 1 -- 172.21.15.32:0/2646074873 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc037040 con 0x7f9fe415b960 2024-04-24T23:10:44.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.492+0000 7f9feaba2640 1 -- 172.21.15.32:0/2646074873 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fe4157860 con 0x7f9fe415b960 2024-04-24T23:10:44.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.493+0000 7f9feaba2640 1 -- 172.21.15.32:0/2646074873 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fe4068430 con 0x7f9fe415b960 2024-04-24T23:10:44.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.493+0000 7f9fe17fa640 1 -- 172.21.15.32:0/2646074873 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9fcc044020 con 0x7f9fe415b960 2024-04-24T23:10:44.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.494+0000 7f9fe17fa640 1 --2- 172.21.15.32:0/2646074873 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9fc406be30 0x7f9fc406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:44.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.494+0000 7f9fe17fa640 1 -- 172.21.15.32:0/2646074873 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(185..185 src has 1..185) ==== 5709+0+0 (secure 0 0 0) 0x7f9fcc0ab4e0 con 0x7f9fe415b960 2024-04-24T23:10:44.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.494+0000 7f9fe893e640 1 --2- 172.21.15.32:0/2646074873 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9fc406be30 0x7f9fc406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:44.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.495+0000 7f9fe893e640 1 --2- 172.21.15.32:0/2646074873 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9fc406be30 0x7f9fc406e2f0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc001d50 tx=0x7f9fdc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:44.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.498+0000 7f9fe17fa640 1 -- 172.21.15.32:0/2646074873 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9fcc04b030 con 0x7f9fe415b960 2024-04-24T23:10:44.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:44.680+0000 7f9feaba2640 1 -- 172.21.15.32:0/2646074873 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) -- 0x7f9fe40cc9d0 con 0x7f9fe415b960 2024-04-24T23:10:45.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.035+0000 7f9fe17fa640 1 -- 172.21.15.32:0/2646074873 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v186) ==== 181+0+0 (secure 0 0 0) 0x7f9fcc079ea0 con 0x7f9fe415b960 2024-04-24T23:10:45.037 INFO:teuthology.orchestra.run.smithi032.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2024-04-24T23:10:45.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.038+0000 7f9feaba2640 1 -- 172.21.15.32:0/2646074873 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9fc406be30 msgr2=0x7f9fc406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:45.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.038+0000 7f9feaba2640 1 --2- 172.21.15.32:0/2646074873 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9fc406be30 0x7f9fc406e2f0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc001d50 tx=0x7f9fdc009040 comp rx=0 tx=0).stop 2024-04-24T23:10:45.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.038+0000 7f9feaba2640 1 -- 172.21.15.32:0/2646074873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fe415b960 msgr2=0x7f9fe40b3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:45.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.038+0000 7f9feaba2640 1 --2- 172.21.15.32:0/2646074873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fe415b960 0x7f9fe40b3a40 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc035040 tx=0x7f9fcc0046c0 comp rx=0 tx=0).stop 2024-04-24T23:10:45.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.038+0000 7f9feaba2640 1 -- 172.21.15.32:0/2646074873 shutdown_connections 2024-04-24T23:10:45.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.038+0000 7f9feaba2640 1 --2- 172.21.15.32:0/2646074873 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9fc406be30 0x7f9fc406e2f0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:45.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.038+0000 7f9feaba2640 1 --2- 172.21.15.32:0/2646074873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fe415b960 0x7f9fe40b3a40 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:45.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.038+0000 7f9feaba2640 1 --2- 172.21.15.32:0/2646074873 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9fe415b580 0x7f9fe40b3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:45.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.038+0000 7f9feaba2640 1 --2- 172.21.15.32:0/2646074873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9fe40b5ff0 0x7f9fe40b2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:45.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.038+0000 7f9feaba2640 1 -- 172.21.15.32:0/2646074873 >> 172.21.15.32:0/2646074873 conn(0x7f9fe40b1f10 msgr2=0x7f9fe4155c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:45.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.038+0000 7f9feaba2640 1 -- 172.21.15.32:0/2646074873 shutdown_connections 2024-04-24T23:10:45.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.038+0000 7f9feaba2640 1 -- 172.21.15.32:0/2646074873 wait complete. 2024-04-24T23:10:45.052 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-04-24T23:10:45.168 INFO:teuthology.orchestra.run.smithi032.stdout:{"osd_mon_report_interval":"5"} 2024-04-24T23:10:45.168 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-04-24T23:10:45.168 INFO:tasks.cephfs.test_full:Writing 32MB should fill this cluster 2024-04-24T23:10:45.169 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:10:45.169 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:10:45.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:10:45.184 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=16 seek=0) 2024-04-24T23:10:45.308 INFO:teuthology.orchestra.run.smithi032.stderr:16+0 records in 2024-04-24T23:10:45.308 INFO:teuthology.orchestra.run.smithi032.stderr:16+0 records out 2024-04-24T23:10:45.308 INFO:teuthology.orchestra.run.smithi032.stderr:16777216 bytes (17 MB, 16 MiB) copied, 0.0655201 s, 256 MB/s 2024-04-24T23:10:45.309 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:10:45.310 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:10:45.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:10:45.326 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=17 seek=0) 2024-04-24T23:10:45.448 INFO:teuthology.orchestra.run.smithi032.stderr:17+0 records in 2024-04-24T23:10:45.448 INFO:teuthology.orchestra.run.smithi032.stderr:17+0 records out 2024-04-24T23:10:45.448 INFO:teuthology.orchestra.run.smithi032.stderr:17825792 bytes (18 MB, 17 MiB) copied, 0.0648458 s, 275 MB/s 2024-04-24T23:10:45.448 INFO:tasks.cephfs.test_full:Writing file B succeeded (full status will happen soon) 2024-04-24T23:10:45.449 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-24T23:10:45.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.548+0000 7f289359e640 1 Processor -- start 2024-04-24T23:10:45.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.548+0000 7f289359e640 1 -- start start 2024-04-24T23:10:45.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.549+0000 7f289359e640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f289415c110 0x7f2894162120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:45.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.549+0000 7f289359e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f289406e430 0x7f28940d1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:45.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.549+0000 7f289359e640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f289415c4f0 con 0x7f289406ced0 2024-04-24T23:10:45.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.549+0000 7f289359e640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f289415ecd0 con 0x7f289415c110 2024-04-24T23:10:45.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.549+0000 7f289359e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f289406e960 con 0x7f289406e430 2024-04-24T23:10:45.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.549+0000 7f2892d9d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f289406e430 0x7f28940d1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:45.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.549+0000 7f2892d9d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f289406e430 0x7f28940d1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.32:38702/0 (socket says 172.21.15.32:38702) 2024-04-24T23:10:45.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.549+0000 7f2892d9d640 1 -- 172.21.15.32:0/561702493 learned_addr learned my addr 172.21.15.32:0/561702493 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:45.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.549+0000 7f289259c640 1 --2- 172.21.15.32:0/561702493 >> v2:172.21.15.121:3300/0 conn(0x7f289415c110 0x7f2894162120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:45.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.550+0000 7f2892d9d640 1 -- 172.21.15.32:0/561702493 >> v2:172.21.15.121:3300/0 conn(0x7f289415c110 msgr2=0x7f2894162120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:45.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.550+0000 7f2892d9d640 1 --2- 172.21.15.32:0/561702493 >> v2:172.21.15.121:3300/0 conn(0x7f289415c110 0x7f2894162120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:45.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.550+0000 7f2892d9d640 1 -- 172.21.15.32:0/561702493 >> v1:172.21.15.121:6789/0 conn(0x7f289406ced0 legacy=0x7f289415f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:45.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.550+0000 7f2892d9d640 1 -- 172.21.15.32:0/561702493 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f289406ec40 con 0x7f289406e430 2024-04-24T23:10:45.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.550+0000 7f2892d9d640 1 --2- 172.21.15.32:0/561702493 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f289406e430 0x7f28940d1f90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f288c009e50 tx=0x7f288c02a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63780255a9e41b7d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:45.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.551+0000 7f2891d9b640 1 -- 172.21.15.32:0/561702493 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288c036070 con 0x7f289406e430 2024-04-24T23:10:45.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.551+0000 7f2891d9b640 1 -- 172.21.15.32:0/561702493 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f288c031600 con 0x7f289406e430 2024-04-24T23:10:45.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.551+0000 7f2891d9b640 1 -- 172.21.15.32:0/561702493 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288c0319d0 con 0x7f289406e430 2024-04-24T23:10:45.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.551+0000 7f289359e640 1 -- 172.21.15.32:0/561702493 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f289406e430 msgr2=0x7f28940d1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:45.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.551+0000 7f289359e640 1 --2- 172.21.15.32:0/561702493 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f289406e430 0x7f28940d1f90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f288c009e50 tx=0x7f288c02a710 comp rx=0 tx=0).stop 2024-04-24T23:10:45.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.551+0000 7f289359e640 1 -- 172.21.15.32:0/561702493 shutdown_connections 2024-04-24T23:10:45.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.551+0000 7f289359e640 1 --2- 172.21.15.32:0/561702493 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f289406e430 0x7f28940d1f90 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:45.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.551+0000 7f289359e640 1 --2- 172.21.15.32:0/561702493 >> v2:172.21.15.121:3300/0 conn(0x7f289415c110 0x7f2894162120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:45.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.551+0000 7f289359e640 1 -- 172.21.15.32:0/561702493 >> 172.21.15.32:0/561702493 conn(0x7f28940b7d80 msgr2=0x7f289406d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:45.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.551+0000 7f289359e640 1 -- 172.21.15.32:0/561702493 shutdown_connections 2024-04-24T23:10:45.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.551+0000 7f289359e640 1 -- 172.21.15.32:0/561702493 wait complete. 2024-04-24T23:10:45.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.552+0000 7f289359e640 1 Processor -- start 2024-04-24T23:10:45.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.553+0000 7f289359e640 1 -- start start 2024-04-24T23:10:45.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.553+0000 7f289359e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f289406ced0 0x7f2894146c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:45.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.553+0000 7f289359e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f289406e430 0x7f2894159db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:45.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.553+0000 7f289359e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f289415c110 0x7f28940b2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:45.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.553+0000 7f289359e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f289415f870 con 0x7f289406ced0 2024-04-24T23:10:45.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.553+0000 7f289359e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f289406f200 con 0x7f289406e430 2024-04-24T23:10:45.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.553+0000 7f289359e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f289415f9f0 con 0x7f289415c110 2024-04-24T23:10:45.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.553+0000 7f289259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f289406ced0 0x7f2894146c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:45.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.553+0000 7f289259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f289406ced0 0x7f2894146c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48892/0 (socket says 172.21.15.32:48892) 2024-04-24T23:10:45.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.553+0000 7f289259c640 1 -- 172.21.15.32:0/1110601749 learned_addr learned my addr 172.21.15.32:0/1110601749 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:45.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.553+0000 7f2892d9d640 1 --2- 172.21.15.32:0/1110601749 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f289415c110 0x7f28940b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:45.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.554+0000 7f288bfff640 1 --2- 172.21.15.32:0/1110601749 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f289406e430 0x7f2894159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:45.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.554+0000 7f289259c640 1 -- 172.21.15.32:0/1110601749 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f289415c110 msgr2=0x7f28940b2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:45.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.554+0000 7f289259c640 1 --2- 172.21.15.32:0/1110601749 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f289415c110 0x7f28940b2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:45.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.554+0000 7f289259c640 1 -- 172.21.15.32:0/1110601749 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f289406e430 msgr2=0x7f2894159db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:45.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.554+0000 7f289259c640 1 --2- 172.21.15.32:0/1110601749 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f289406e430 0x7f2894159db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:45.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.554+0000 7f289259c640 1 -- 172.21.15.32:0/1110601749 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2894158670 con 0x7f289406ced0 2024-04-24T23:10:45.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.554+0000 7f289259c640 1 --2- 172.21.15.32:0/1110601749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f289406ced0 0x7f2894146c40 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f288400e870 tx=0x7f288400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:45.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.554+0000 7f288affd640 1 -- 172.21.15.32:0/1110601749 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2884017070 con 0x7f289406ced0 2024-04-24T23:10:45.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.554+0000 7f288affd640 1 -- 172.21.15.32:0/1110601749 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f288400c940 con 0x7f289406ced0 2024-04-24T23:10:45.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.554+0000 7f289359e640 1 -- 172.21.15.32:0/1110601749 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2894158950 con 0x7f289406ced0 2024-04-24T23:10:45.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.554+0000 7f288affd640 1 -- 172.21.15.32:0/1110601749 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288400cd30 con 0x7f289406ced0 2024-04-24T23:10:45.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.554+0000 7f289359e640 1 -- 172.21.15.32:0/1110601749 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2894158f10 con 0x7f289406ced0 2024-04-24T23:10:45.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.555+0000 7f289359e640 1 -- 172.21.15.32:0/1110601749 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28940682b0 con 0x7f289406ced0 2024-04-24T23:10:45.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.556+0000 7f288affd640 1 -- 172.21.15.32:0/1110601749 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f288401a070 con 0x7f289406ced0 2024-04-24T23:10:45.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.556+0000 7f288affd640 1 --2- 172.21.15.32:0/1110601749 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f287406be30 0x7f287406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:45.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.556+0000 7f288affd640 1 -- 172.21.15.32:0/1110601749 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 5709+0+0 (secure 0 0 0) 0x7f288408af90 con 0x7f289406ced0 2024-04-24T23:10:45.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.556+0000 7f288bfff640 1 --2- 172.21.15.32:0/1110601749 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f287406be30 0x7f287406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:45.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.557+0000 7f288bfff640 1 --2- 172.21.15.32:0/1110601749 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f287406be30 0x7f287406e2f0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f289406bc60 tx=0x7f287c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:45.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.560+0000 7f288affd640 1 -- 172.21.15.32:0/1110601749 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2884059380 con 0x7f289406ced0 2024-04-24T23:10:45.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.743+0000 7f289359e640 1 -- 172.21.15.32:0/1110601749 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f28940ae8b0 con 0x7f289406ced0 2024-04-24T23:10:45.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.745+0000 7f288affd640 1 -- 172.21.15.32:0/1110601749 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v186) ==== 81+0+34188 (secure 0 0 0) 0x7f288405d680 con 0x7f289406ced0 2024-04-24T23:10:45.747 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:10:45.747 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:10:45.747 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 186, 2024-04-24T23:10:45.747 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", 2024-04-24T23:10:45.747 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2024-04-24T22:58:46.964851+0000", 2024-04-24T23:10:45.747 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2024-04-24T23:10:45.031422+0000", 2024-04-24T23:10:45.747 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2024-04-24T22:58:52.048787+0000", 2024-04-24T23:10:45.747 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2024-04-24T22:58:49.045609+0000", 2024-04-24T23:10:45.747 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-24T23:10:45.747 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2024-04-24T23:10:45.747 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2024-04-24T23:10:45.748 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2024-04-24T23:10:45.748 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2024-04-24T23:10:45.748 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2024-04-24T23:10:45.748 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2024-04-24T23:10:45.748 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:10:45.748 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 4, 2024-04-24T23:10:45.748 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.69999998807907104, 2024-04-24T23:10:45.748 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-24T23:10:45.748 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-24T23:10:45.748 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2024-04-24T23:10:45.748 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 23, 2024-04-24T23:10:45.748 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2024-04-24T23:10:45.748 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "luminous", 2024-04-24T23:10:45.748 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "jewel", 2024-04-24T23:10:45.749 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "squid", 2024-04-24T23:10:45.749 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_crimson": false, 2024-04-24T23:10:45.749 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2024-04-24T23:10:45.749 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.749 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2024-04-24T23:10:45.749 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": ".mgr", 2024-04-24T23:10:45.749 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T22:58:53.171408+0000", 2024-04-24T23:10:45.749 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:10:45.749 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:10:45.749 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:10:45.749 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:10:45.749 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:10:45.749 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:10:45.749 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:10:45.749 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:10:45.750 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:10:45.750 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:10:45.750 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:10:45.750 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:10:45.750 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2024-04-24T23:10:45.750 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2024-04-24T23:10:45.750 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2024-04-24T23:10:45.750 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2024-04-24T23:10:45.750 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2024-04-24T23:10:45.750 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:10:45.750 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:10:45.750 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:10:45.750 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:10:45.750 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:10:45.751 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:10:45.751 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:10:45.751 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.751 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "113", 2024-04-24T23:10:45.751 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:10:45.751 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:10:45.751 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:10:45.751 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:10:45.751 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:10:45.751 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:10:45.751 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:10:45.751 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:10:45.751 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:10:45.752 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:10:45.752 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:10:45.752 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:10:45.752 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:10:45.752 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:10:45.752 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:10:45.752 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:10:45.752 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:10:45.752 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:10:45.752 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:10:45.752 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:10:45.752 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:10:45.752 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:10:45.753 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:10:45.753 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:10:45.753 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:10:45.753 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:10:45.753 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.753 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:10:45.753 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:10:45.753 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:10:45.753 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:10:45.753 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:10:45.753 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:10:45.753 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:10:45.753 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:10:45.753 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:10:45.754 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:10:45.754 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:10:45.754 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:10:45.754 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_max": 32, 2024-04-24T23:10:45.754 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2024-04-24T23:10:45.754 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.754 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:10:45.754 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": {} 2024-04-24T23:10:45.754 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.754 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:10:45.754 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:10:45.754 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 8, 2024-04-24T23:10:45.754 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 8, 2024-04-24T23:10:45.755 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.25, 2024-04-24T23:10:45.755 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-04-24T23:10:45.755 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-04-24T23:10:45.755 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:10:45.755 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:10:45.755 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:10:45.755 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:45.755 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.755 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.755 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 22, 2024-04-24T23:10:45.755 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2024-04-24T23:10:45.755 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:10:34.649562+0000", 2024-04-24T23:10:45.755 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:10:45.755 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:10:45.755 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:10:45.756 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:10:45.756 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:10:45.756 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:10:45.756 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:10:45.756 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:10:45.756 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:10:45.756 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:10:45.756 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:10:45.756 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:10:45.756 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 32, 2024-04-24T23:10:45.756 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 32, 2024-04-24T23:10:45.756 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 32, 2024-04-24T23:10:45.756 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 32, 2024-04-24T23:10:45.756 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 32, 2024-04-24T23:10:45.756 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:10:45.757 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:10:45.757 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:10:45.757 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:10:45.757 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:10:45.757 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:10:45.757 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:10:45.757 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.757 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "184", 2024-04-24T23:10:45.757 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:10:45.757 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:10:45.757 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:10:45.757 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:10:45.757 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:10:45.758 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:10:45.758 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:10:45.758 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:10:45.758 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:10:45.758 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:10:45.758 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:10:45.758 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:10:45.758 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:10:45.758 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:10:45.758 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:10:45.758 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:10:45.758 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:10:45.758 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:10:45.759 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:10:45.759 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:10:45.759 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:10:45.759 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:10:45.759 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:10:45.759 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:10:45.759 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:10:45.759 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:10:45.759 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.759 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:10:45.759 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:10:45.759 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:10:45.759 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:10:45.760 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:10:45.760 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:10:45.760 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:10:45.760 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:10:45.760 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:10:45.760 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:10:45.760 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:10:45.760 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:10:45.760 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2024-04-24T23:10:45.760 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2024-04-24T23:10:45.760 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2024-04-24T23:10:45.760 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.760 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:10:45.760 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:10:45.761 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2024-04-24T23:10:45.761 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:45.761 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.761 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:10:45.761 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:10:45.761 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.5, 2024-04-24T23:10:45.761 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.5, 2024-04-24T23:10:45.761 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:10:45.761 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.5, 2024-04-24T23:10:45.761 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.5, 2024-04-24T23:10:45.761 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:10:45.761 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:10:45.761 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:10:45.761 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:45.761 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.761 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.762 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 23, 2024-04-24T23:10:45.762 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2024-04-24T23:10:45.762 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:10:35.339604+0000", 2024-04-24T23:10:45.762 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:10:45.762 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:10:45.762 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:10:45.762 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:10:45.762 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:10:45.762 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:10:45.762 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:10:45.762 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:10:45.762 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:10:45.762 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:10:45.762 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:10:45.763 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:10:45.763 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 64, 2024-04-24T23:10:45.763 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 64, 2024-04-24T23:10:45.763 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 64, 2024-04-24T23:10:45.763 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 64, 2024-04-24T23:10:45.763 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 64, 2024-04-24T23:10:45.763 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:10:45.763 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:10:45.763 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:10:45.763 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:10:45.763 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:10:45.763 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:10:45.763 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:10:45.764 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.764 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "186", 2024-04-24T23:10:45.764 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:10:45.764 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:10:45.764 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:10:45.764 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:10:45.764 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:10:45.764 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:10:45.764 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:10:45.764 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:10:45.764 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:10:45.764 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 33554432, 2024-04-24T23:10:45.764 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:10:45.764 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:10:45.765 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:10:45.765 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:10:45.765 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:10:45.765 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:10:45.765 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:10:45.765 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:10:45.765 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:10:45.765 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:10:45.765 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:10:45.765 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:10:45.765 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:10:45.765 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:10:45.765 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:10:45.765 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:10:45.766 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.766 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:10:45.766 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:10:45.766 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:10:45.766 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:10:45.766 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:10:45.766 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:10:45.766 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:10:45.766 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:10:45.766 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:10:45.766 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:10:45.766 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:10:45.766 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:10:45.766 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 64, 2024-04-24T23:10:45.767 INFO:teuthology.orchestra.run.smithi032.stdout: "target_size_ratio": 0.89999997615814209 2024-04-24T23:10:45.767 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.767 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:10:45.767 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:10:45.767 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2024-04-24T23:10:45.767 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:45.767 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.767 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:10:45.767 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:10:45.767 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.3799999952316284, 2024-04-24T23:10:45.767 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.3799999952316284, 2024-04-24T23:10:45.767 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:10:45.767 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.3799999952316284, 2024-04-24T23:10:45.767 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.3799999952316284, 2024-04-24T23:10:45.768 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:10:45.768 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:10:45.768 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:10:45.768 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:45.768 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:45.768 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:10:45.768 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-04-24T23:10:45.768 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.768 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:10:45.768 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:10:45.768 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:10:45.768 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:10:45.769 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:10:45.769 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:10:45.769 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:10:45.769 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:10:45.769 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:10:45.769 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:10:45.769 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:10:45.769 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:10:45.769 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:10:45.769 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:45.769 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.769 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:45.769 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2024-04-24T23:10:45.769 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:10:45.770 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.770 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.770 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:45.770 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2024-04-24T23:10:45.770 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:10:45.770 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:45.770 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:45.770 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.770 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:10:45.770 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:45.770 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.770 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:45.770 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2024-04-24T23:10:45.771 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:10:45.771 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.771 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.771 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:45.771 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2024-04-24T23:10:45.771 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:10:45.771 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:45.771 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:45.771 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.771 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:10:45.771 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:45.771 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.771 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:45.771 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2024-04-24T23:10:45.771 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:10:45.772 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.772 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.772 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:45.772 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2024-04-24T23:10:45.772 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:10:45.772 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:45.772 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:45.772 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.772 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:10:45.772 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:45.772 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.773 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:45.773 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2024-04-24T23:10:45.773 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:10:45.773 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.773 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.773 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:45.773 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2024-04-24T23:10:45.773 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:10:45.773 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:45.774 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:45.774 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.774 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/1479395426", 2024-04-24T23:10:45.774 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/1479395426", 2024-04-24T23:10:45.774 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/1479395426", 2024-04-24T23:10:45.774 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/1479395426", 2024-04-24T23:10:45.774 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:10:45.774 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:10:45.774 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:10:45.774 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:45.774 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.774 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.775 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:10:45.775 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:10:45.775 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:10:45.775 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:10:45.775 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:10:45.775 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:10:45.775 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:10:45.775 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:10:45.775 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:10:45.775 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:10:45.775 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:10:45.775 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:10:45.775 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:10:45.776 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:45.776 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.776 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:45.776 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2024-04-24T23:10:45.776 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:10:45.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.776 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.776 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:45.776 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2024-04-24T23:10:45.776 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:10:45.776 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:45.776 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:45.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.776 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:10:45.776 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:45.777 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.777 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:45.777 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2024-04-24T23:10:45.777 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:10:45.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.777 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.777 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:45.777 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2024-04-24T23:10:45.777 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:10:45.777 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:45.777 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:45.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.777 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:10:45.777 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:45.778 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.778 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:45.778 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2024-04-24T23:10:45.778 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:10:45.778 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.778 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.778 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:45.778 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2024-04-24T23:10:45.778 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:10:45.778 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:45.778 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:45.778 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.778 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:10:45.778 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:45.778 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.779 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:45.779 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2024-04-24T23:10:45.779 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:10:45.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.779 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.779 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:45.779 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2024-04-24T23:10:45.779 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:10:45.779 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:45.779 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:45.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.779 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/1340633718", 2024-04-24T23:10:45.779 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/1340633718", 2024-04-24T23:10:45.779 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/1340633718", 2024-04-24T23:10:45.780 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/1340633718", 2024-04-24T23:10:45.780 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:10:45.780 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:10:45.780 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:10:45.780 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:45.780 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.780 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.780 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:10:45.780 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:10:45.780 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:10:45.780 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:10:45.780 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:10:45.780 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:10:45.780 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:10:45.781 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:10:45.781 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:10:45.781 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:10:45.781 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:10:45.781 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:10:45.781 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:10:45.781 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:45.781 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.781 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:45.781 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2024-04-24T23:10:45.781 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:10:45.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.781 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.781 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:45.782 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2024-04-24T23:10:45.782 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:10:45.782 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:45.782 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:45.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.782 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:10:45.782 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:45.782 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.782 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:45.782 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2024-04-24T23:10:45.782 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:10:45.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.782 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.783 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:45.783 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2024-04-24T23:10:45.783 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:10:45.783 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:45.783 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:45.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.783 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:10:45.783 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:45.783 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.783 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:45.783 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2024-04-24T23:10:45.783 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:10:45.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.783 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.784 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:45.784 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2024-04-24T23:10:45.784 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:10:45.784 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:45.784 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:45.784 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.784 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:10:45.784 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:45.784 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.784 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:45.784 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2024-04-24T23:10:45.784 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:10:45.784 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.785 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.785 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:45.785 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2024-04-24T23:10:45.785 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:10:45.785 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:45.785 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:45.785 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.785 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/3405899334", 2024-04-24T23:10:45.785 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6820/3405899334", 2024-04-24T23:10:45.785 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6828/3405899334", 2024-04-24T23:10:45.785 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6824/3405899334", 2024-04-24T23:10:45.785 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:10:45.785 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:10:45.785 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:10:45.786 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:45.786 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.786 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.786 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:10:45.786 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:10:45.786 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:10:45.786 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:10:45.786 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:10:45.786 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:10:45.786 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:10:45.786 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:10:45.786 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:10:45.786 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:10:45.786 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:10:45.787 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:10:45.787 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:10:45.787 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:45.787 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.787 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:45.787 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2024-04-24T23:10:45.787 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:10:45.787 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.787 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.787 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:45.787 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2024-04-24T23:10:45.787 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:10:45.787 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:45.787 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:45.787 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.787 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:10:45.788 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:45.788 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.788 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:45.788 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2024-04-24T23:10:45.788 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:10:45.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.788 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.788 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:45.788 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2024-04-24T23:10:45.788 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:10:45.788 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:45.788 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:45.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.788 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:10:45.789 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:45.789 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.789 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:45.789 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2024-04-24T23:10:45.789 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:10:45.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.789 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.789 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:45.789 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2024-04-24T23:10:45.789 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:10:45.789 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:45.789 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:45.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.790 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:10:45.790 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:45.790 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.790 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:45.790 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2024-04-24T23:10:45.790 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:10:45.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.790 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.790 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:45.790 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2024-04-24T23:10:45.790 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:10:45.790 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:45.790 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:45.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.791 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6821/3381057190", 2024-04-24T23:10:45.791 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6825/3381057190", 2024-04-24T23:10:45.791 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/3381057190", 2024-04-24T23:10:45.791 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/3381057190", 2024-04-24T23:10:45.791 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:10:45.791 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:10:45.791 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:10:45.791 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:45.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.791 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.791 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:10:45.791 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:10:45.791 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:10:45.791 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:10:45.791 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:10:45.792 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:10:45.792 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:10:45.792 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:10:45.792 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:10:45.792 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:10:45.792 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:10:45.792 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:10:45.792 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:10:45.792 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:45.792 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.792 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:45.792 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6800", 2024-04-24T23:10:45.792 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:10:45.792 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.793 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.793 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:45.793 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6801", 2024-04-24T23:10:45.793 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:10:45.793 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:45.793 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:45.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.793 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:10:45.793 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:45.793 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.793 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:45.793 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6802", 2024-04-24T23:10:45.793 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:10:45.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.794 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.794 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:45.794 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6803", 2024-04-24T23:10:45.794 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:10:45.794 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:45.794 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:45.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.794 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:10:45.794 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:45.794 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.794 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:45.794 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6806", 2024-04-24T23:10:45.794 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:10:45.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.795 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.795 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:45.795 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6807", 2024-04-24T23:10:45.795 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:10:45.795 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:45.795 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:45.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.795 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:10:45.795 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:45.795 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.795 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:45.796 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6804", 2024-04-24T23:10:45.796 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:10:45.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.796 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:45.796 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6805", 2024-04-24T23:10:45.796 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:10:45.796 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:45.796 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:45.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.796 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6801/4010553979", 2024-04-24T23:10:45.796 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6803/4010553979", 2024-04-24T23:10:45.796 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6807/4010553979", 2024-04-24T23:10:45.797 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6805/4010553979", 2024-04-24T23:10:45.797 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:10:45.797 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:10:45.797 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:10:45.797 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:45.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.797 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.797 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:10:45.797 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:10:45.797 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:10:45.797 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:10:45.797 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:10:45.797 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:10:45.797 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:10:45.798 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:10:45.798 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:10:45.798 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:10:45.798 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:10:45.798 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:10:45.798 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:10:45.798 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:45.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.798 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:45.798 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6808", 2024-04-24T23:10:45.798 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:10:45.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.798 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:45.799 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6809", 2024-04-24T23:10:45.799 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:10:45.799 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:45.799 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:45.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.799 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:10:45.799 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:45.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.799 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:45.799 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6810", 2024-04-24T23:10:45.799 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:10:45.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.801 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:45.801 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6811", 2024-04-24T23:10:45.801 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:10:45.801 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:45.802 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:45.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.802 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:10:45.802 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:45.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.802 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:45.802 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6814", 2024-04-24T23:10:45.802 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:10:45.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.803 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:45.803 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6815", 2024-04-24T23:10:45.803 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:10:45.803 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:45.803 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:45.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.803 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:10:45.803 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:45.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.803 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:45.803 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6812", 2024-04-24T23:10:45.803 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:10:45.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.804 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:45.804 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6813", 2024-04-24T23:10:45.804 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:10:45.804 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:45.804 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:45.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.804 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6809/3794084091", 2024-04-24T23:10:45.804 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6811/3794084091", 2024-04-24T23:10:45.804 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6815/3794084091", 2024-04-24T23:10:45.804 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6813/3794084091", 2024-04-24T23:10:45.804 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:10:45.804 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:10:45.804 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:10:45.804 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:45.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.805 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:10:45.805 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:10:45.805 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:10:45.805 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:10:45.805 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:10:45.805 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:10:45.805 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:10:45.805 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:10:45.805 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:10:45.805 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:10:45.805 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:10:45.805 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:10:45.805 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:10:45.805 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:45.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.806 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:45.806 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6823", 2024-04-24T23:10:45.806 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:10:45.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.806 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:45.806 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6825", 2024-04-24T23:10:45.806 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:10:45.806 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:45.806 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:45.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.806 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:10:45.807 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:45.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.807 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:45.807 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6826", 2024-04-24T23:10:45.807 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:10:45.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.807 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:45.807 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6827", 2024-04-24T23:10:45.807 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:10:45.807 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:45.807 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:45.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.807 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:10:45.807 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:45.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.808 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:45.808 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6830", 2024-04-24T23:10:45.808 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:10:45.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.808 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:45.808 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6831", 2024-04-24T23:10:45.808 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:10:45.808 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:45.808 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:45.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.808 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:10:45.808 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:45.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.809 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:45.809 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6828", 2024-04-24T23:10:45.809 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:10:45.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.809 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:45.809 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6829", 2024-04-24T23:10:45.809 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:10:45.809 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:45.809 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:45.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.809 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6825/1348640258", 2024-04-24T23:10:45.810 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6827/1348640258", 2024-04-24T23:10:45.810 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6831/1348640258", 2024-04-24T23:10:45.810 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6829/1348640258", 2024-04-24T23:10:45.810 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:10:45.810 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:10:45.810 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:10:45.810 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:45.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.810 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:10:45.810 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:10:45.810 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:10:45.810 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:10:45.810 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:10:45.811 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:10:45.811 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:10:45.811 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:10:45.811 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:10:45.811 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:10:45.811 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:10:45.811 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:10:45.811 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:10:45.811 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:45.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.811 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:45.811 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6816", 2024-04-24T23:10:45.812 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:10:45.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.812 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:45.812 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6817", 2024-04-24T23:10:45.812 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:10:45.812 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:45.812 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:45.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.812 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:10:45.812 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:45.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.813 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:45.813 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6818", 2024-04-24T23:10:45.813 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:10:45.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.813 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:45.813 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6819", 2024-04-24T23:10:45.813 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:10:45.813 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:45.813 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:45.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.813 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:10:45.813 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:45.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.814 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:45.814 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6822", 2024-04-24T23:10:45.814 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:10:45.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.814 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:45.814 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6824", 2024-04-24T23:10:45.814 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:10:45.814 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:45.814 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:45.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.814 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:10:45.815 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:45.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.815 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:45.815 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6820", 2024-04-24T23:10:45.815 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:10:45.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.815 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:45.815 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6821", 2024-04-24T23:10:45.815 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:10:45.815 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:45.815 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:45.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.815 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6817/3250005122", 2024-04-24T23:10:45.815 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6819/3250005122", 2024-04-24T23:10:45.815 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6824/3250005122", 2024-04-24T23:10:45.816 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6821/3250005122", 2024-04-24T23:10:45.816 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:10:45.816 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:10:45.816 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:10:45.816 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:45.816 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:45.816 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:10:45.816 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2024-04-24T23:10:45.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.816 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:10:45.816 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:10:45.816 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:10:45.816 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:10:45.816 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:10:45.816 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:10:45.816 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.156184+0000", 2024-04-24T23:10:45.817 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:10:45.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.817 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:10:45.817 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:10:45.817 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:10:45.817 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:10:45.817 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:10:45.817 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:10:45.817 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240774+0000", 2024-04-24T23:10:45.817 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:10:45.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.817 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:10:45.817 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:10:45.817 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:10:45.818 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:10:45.818 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:10:45.818 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:10:45.818 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240949+0000", 2024-04-24T23:10:45.818 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:10:45.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.818 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:10:45.818 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:10:45.818 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:10:45.818 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:10:45.818 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:10:45.818 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:10:45.818 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.171672+0000", 2024-04-24T23:10:45.819 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:10:45.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.819 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:10:45.819 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:10:45.819 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:10:45.819 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:10:45.819 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:10:45.819 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:10:45.819 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.220379+0000", 2024-04-24T23:10:45.819 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:10:45.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.819 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:10:45.819 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:10:45.820 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:10:45.820 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:10:45.820 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:10:45.820 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:10:45.820 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.204963+0000", 2024-04-24T23:10:45.820 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:10:45.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.820 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:10:45.820 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:10:45.820 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:10:45.820 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:10:45.820 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:10:45.820 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:10:45.820 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.239748+0000", 2024-04-24T23:10:45.820 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:10:45.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:45.821 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:10:45.821 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:10:45.821 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:10:45.821 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:10:45.821 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:10:45.821 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:10:45.821 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.154104+0000", 2024-04-24T23:10:45.821 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:10:45.821 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:45.821 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:10:45.821 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2024-04-24T23:10:45.821 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2024-04-24T23:10:45.821 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_primaries": [], 2024-04-24T23:10:45.821 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2024-04-24T23:10:45.822 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2024-04-24T23:10:45.822 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklist": {}, 2024-04-24T23:10:45.822 INFO:teuthology.orchestra.run.smithi032.stdout: "range_blocklist": {}, 2024-04-24T23:10:45.822 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2024-04-24T23:10:45.822 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2024-04-24T23:10:45.822 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2024-04-24T23:10:45.822 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2024-04-24T23:10:45.822 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2024-04-24T23:10:45.822 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2024-04-24T23:10:45.822 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2024-04-24T23:10:45.822 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:45.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:45.822 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2024-04-24T23:10:45.822 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2024-04-24T23:10:45.823 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2024-04-24T23:10:45.823 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2024-04-24T23:10:45.823 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {}, 2024-04-24T23:10:45.823 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode": { 2024-04-24T23:10:45.823 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_enabled": false, 2024-04-24T23:10:45.823 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_bucket_count": 0, 2024-04-24T23:10:45.823 INFO:teuthology.orchestra.run.smithi032.stdout: "degraded_stretch_mode": 0, 2024-04-24T23:10:45.823 INFO:teuthology.orchestra.run.smithi032.stdout: "recovering_stretch_mode": 0, 2024-04-24T23:10:45.823 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_bucket": 0 2024-04-24T23:10:45.823 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:45.823 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:10:45.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.746+0000 7f289359e640 1 -- 172.21.15.32:0/1110601749 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f287406be30 msgr2=0x7f287406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:45.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.746+0000 7f289359e640 1 --2- 172.21.15.32:0/1110601749 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f287406be30 0x7f287406e2f0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f289406bc60 tx=0x7f287c009040 comp rx=0 tx=0).stop 2024-04-24T23:10:45.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.746+0000 7f289359e640 1 -- 172.21.15.32:0/1110601749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f289406ced0 msgr2=0x7f2894146c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:45.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.746+0000 7f289359e640 1 --2- 172.21.15.32:0/1110601749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f289406ced0 0x7f2894146c40 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f288400e870 tx=0x7f288400ed30 comp rx=0 tx=0).stop 2024-04-24T23:10:45.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.746+0000 7f289359e640 1 -- 172.21.15.32:0/1110601749 shutdown_connections 2024-04-24T23:10:45.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.746+0000 7f289359e640 1 --2- 172.21.15.32:0/1110601749 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f287406be30 0x7f287406e2f0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:45.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.746+0000 7f289359e640 1 --2- 172.21.15.32:0/1110601749 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f289415c110 0x7f28940b2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:45.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.747+0000 7f289359e640 1 --2- 172.21.15.32:0/1110601749 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f289406e430 0x7f2894159db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:45.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.747+0000 7f289359e640 1 --2- 172.21.15.32:0/1110601749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f289406ced0 0x7f2894146c40 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:45.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.747+0000 7f289359e640 1 -- 172.21.15.32:0/1110601749 >> 172.21.15.32:0/1110601749 conn(0x7f28940b7d80 msgr2=0x7f28940d1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:45.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.747+0000 7f289359e640 1 -- 172.21.15.32:0/1110601749 shutdown_connections 2024-04-24T23:10:45.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:45.747+0000 7f289359e640 1 -- 172.21.15.32:0/1110601749 wait complete. 2024-04-24T23:10:45.826 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-04-24T23:10:50.828 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-24T23:10:50.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.940+0000 7f10d443f640 1 Processor -- start 2024-04-24T23:10:50.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.940+0000 7f10d443f640 1 -- start start 2024-04-24T23:10:50.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.941+0000 7f10d443f640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f10cc06c5f0 0x7f10cc06c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:50.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.941+0000 7f10d443f640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10cc0b6430 con 0x7f10cc0af890 2024-04-24T23:10:50.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.941+0000 7f10d443f640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10cc156f80 con 0x7f10cc0b44d0 2024-04-24T23:10:50.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.941+0000 7f10d443f640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f10cc0b6150 con 0x7f10cc06c5f0 2024-04-24T23:10:50.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.941+0000 7f10d29dc640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f10cc0af890 0x7f10cc15cac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46088/0 (socket says 172.21.15.32:46088) 2024-04-24T23:10:50.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.941+0000 7f10d29dc640 1 -- 172.21.15.32:0/1349730484 learned_addr learned my addr 172.21.15.32:0/1349730484 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:10:50.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.941+0000 7f10d21db640 1 --2- 172.21.15.32:0/1349730484 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f10cc06c5f0 0x7f10cc06c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:50.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.941+0000 7f10d11d9640 1 -- 172.21.15.32:0/1349730484 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3367223212 0 0) 0x7f10cc0b6430 con 0x7f10cc0af890 2024-04-24T23:10:50.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.941+0000 7f10d11d9640 1 -- 172.21.15.32:0/1349730484 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f10b8003520 con 0x7f10cc0af890 2024-04-24T23:10:50.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.941+0000 7f10d11d9640 1 -- 172.21.15.32:0/1349730484 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2370539782 0 0) 0x7f10b8003520 con 0x7f10cc0af890 2024-04-24T23:10:50.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.942+0000 7f10d11d9640 1 -- 172.21.15.32:0/1349730484 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f10cc06c5f0 msgr2=0x7f10cc06c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:50.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.942+0000 7f10d11d9640 1 --2- 172.21.15.32:0/1349730484 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f10cc06c5f0 0x7f10cc06c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:50.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.942+0000 7f10d11d9640 1 -- 172.21.15.32:0/1349730484 >> v1:172.21.15.121:6789/0 conn(0x7f10cc0b44d0 legacy=0x7f10cc0b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:50.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.942+0000 7f10d11d9640 1 -- 172.21.15.32:0/1349730484 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10cc0b39f0 con 0x7f10cc0af890 2024-04-24T23:10:50.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.942+0000 7f10d11d9640 1 -- 172.21.15.32:0/1349730484 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f10c40019f0 con 0x7f10cc0af890 2024-04-24T23:10:50.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.942+0000 7f10d11d9640 1 -- 172.21.15.32:0/1349730484 >> v1:172.21.15.32:6789/0 conn(0x7f10cc0af890 legacy=0x7f10cc15cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:50.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.942+0000 7f10d11d9640 1 --2- 172.21.15.32:0/1349730484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10b80039e0 0x7f10b8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:50.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.942+0000 7f10d11d9640 1 --2- 172.21.15.32:0/1349730484 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f10cc06c5f0 0x7f10b8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:50.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.942+0000 7f10d11d9640 1 --2- 172.21.15.32:0/1349730484 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10b8004d40 0x7f10b80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:50.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.942+0000 7f10d11d9640 1 -- 172.21.15.32:0/1349730484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc0b6150 con 0x7f10b80039e0 2024-04-24T23:10:50.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.942+0000 7f10d11d9640 1 -- 172.21.15.32:0/1349730484 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f10b800ba70 con 0x7f10b8004d40 2024-04-24T23:10:50.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.942+0000 7f10d11d9640 1 -- 172.21.15.32:0/1349730484 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f10b800bc70 con 0x7f10cc06c5f0 2024-04-24T23:10:50.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.942+0000 7f10d21db640 1 --2- 172.21.15.32:0/1349730484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10b80039e0 0x7f10b8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:50.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.942+0000 7f10d19da640 1 --2- 172.21.15.32:0/1349730484 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f10cc06c5f0 0x7f10b8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:50.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.942+0000 7f10d29dc640 1 --2- 172.21.15.32:0/1349730484 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10b8004d40 0x7f10b80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:50.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.942+0000 7f10d21db640 1 -- 172.21.15.32:0/1349730484 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f10cc06c5f0 msgr2=0x7f10b8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:50.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.942+0000 7f10d21db640 1 --2- 172.21.15.32:0/1349730484 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f10cc06c5f0 0x7f10b8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:50.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.942+0000 7f10d21db640 1 -- 172.21.15.32:0/1349730484 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10b8004d40 msgr2=0x7f10b80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:50.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.942+0000 7f10d21db640 1 --2- 172.21.15.32:0/1349730484 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10b8004d40 0x7f10b80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:50.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.942+0000 7f10d21db640 1 -- 172.21.15.32:0/1349730484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10b8003520 con 0x7f10b80039e0 2024-04-24T23:10:50.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.943+0000 7f10d21db640 1 --2- 172.21.15.32:0/1349730484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10b80039e0 0x7f10b8003dc0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f10c8009ac0 tx=0x7f10b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88fdda95003dc622 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:50.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.943+0000 7f10d11d9640 1 -- 172.21.15.32:0/1349730484 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c8036070 con 0x7f10b80039e0 2024-04-24T23:10:50.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.943+0000 7f10d11d9640 1 -- 172.21.15.32:0/1349730484 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10c8031400 con 0x7f10b80039e0 2024-04-24T23:10:50.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.943+0000 7f10d11d9640 1 -- 172.21.15.32:0/1349730484 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c8009690 con 0x7f10b80039e0 2024-04-24T23:10:50.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.944+0000 7f10d443f640 1 -- 172.21.15.32:0/1349730484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10b80039e0 msgr2=0x7f10b8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:50.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.944+0000 7f10d443f640 1 --2- 172.21.15.32:0/1349730484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10b80039e0 0x7f10b8003dc0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f10c8009ac0 tx=0x7f10b8005830 comp rx=0 tx=0).stop 2024-04-24T23:10:50.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.944+0000 7f10d29dc640 1 -- 172.21.15.32:0/1349730484 reap_dead start 2024-04-24T23:10:50.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.944+0000 7f10d443f640 1 -- 172.21.15.32:0/1349730484 shutdown_connections 2024-04-24T23:10:50.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.944+0000 7f10d443f640 1 -- 172.21.15.32:0/1349730484 >> 172.21.15.32:0/1349730484 conn(0x7f10cc06b110 msgr2=0x7f10cc06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:50.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.944+0000 7f10d443f640 1 -- 172.21.15.32:0/1349730484 shutdown_connections 2024-04-24T23:10:50.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.944+0000 7f10d443f640 1 -- 172.21.15.32:0/1349730484 wait complete. 2024-04-24T23:10:50.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.945+0000 7f10d443f640 1 Processor -- start 2024-04-24T23:10:50.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.945+0000 7f10d443f640 1 -- start start 2024-04-24T23:10:50.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.946+0000 7f10d443f640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f10b80039e0 0x7f10cc154570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:50.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.946+0000 7f10d443f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10cc154ab0 0x7f10cc15e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:50.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.946+0000 7f10d443f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10cc15ef30 0x7f10cc15f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:50.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.946+0000 7f10d443f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc06c9d0 con 0x7f10cc154ab0 2024-04-24T23:10:50.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.946+0000 7f10d443f640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc0b3fb0 con 0x7f10cc15ef30 2024-04-24T23:10:50.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.946+0000 7f10d443f640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f10cc06cb50 con 0x7f10b80039e0 2024-04-24T23:10:50.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.946+0000 7f10d19da640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10cc154ab0 0x7f10cc15e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:50.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.946+0000 7f10d19da640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10cc154ab0 0x7f10cc15e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41098/0 (socket says 172.21.15.32:41098) 2024-04-24T23:10:50.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.946+0000 7f10d19da640 1 -- 172.21.15.32:0/2088346625 learned_addr learned my addr 172.21.15.32:0/2088346625 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:50.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.946+0000 7f10d21db640 1 --2- 172.21.15.32:0/2088346625 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f10b80039e0 0x7f10cc154570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:50.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.946+0000 7f10d29dc640 1 --2- 172.21.15.32:0/2088346625 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10cc15ef30 0x7f10cc15f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:50.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.946+0000 7f10d19da640 1 -- 172.21.15.32:0/2088346625 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f10b80039e0 msgr2=0x7f10cc154570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:50.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.946+0000 7f10d19da640 1 --2- 172.21.15.32:0/2088346625 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f10b80039e0 0x7f10cc154570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:50.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.946+0000 7f10d19da640 1 -- 172.21.15.32:0/2088346625 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10cc15ef30 msgr2=0x7f10cc15f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:50.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.946+0000 7f10d19da640 1 --2- 172.21.15.32:0/2088346625 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10cc15ef30 0x7f10cc15f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:50.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.946+0000 7f10d19da640 1 -- 172.21.15.32:0/2088346625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10cc155580 con 0x7f10cc154ab0 2024-04-24T23:10:50.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.946+0000 7f10d29dc640 1 --2- 172.21.15.32:0/2088346625 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10cc15ef30 0x7f10cc15f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:10:50.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.947+0000 7f10d19da640 1 --2- 172.21.15.32:0/2088346625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10cc154ab0 0x7f10cc15e9f0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f10bc00f770 tx=0x7f10bc00fc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:50.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.947+0000 7f10c2ffd640 1 -- 172.21.15.32:0/2088346625 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10bc00fe80 con 0x7f10cc154ab0 2024-04-24T23:10:50.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.947+0000 7f10c2ffd640 1 -- 172.21.15.32:0/2088346625 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10bc018070 con 0x7f10cc154ab0 2024-04-24T23:10:50.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.947+0000 7f10c2ffd640 1 -- 172.21.15.32:0/2088346625 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10bc00d730 con 0x7f10cc154ab0 2024-04-24T23:10:50.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.947+0000 7f10d443f640 1 -- 172.21.15.32:0/2088346625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10cc1572b0 con 0x7f10cc154ab0 2024-04-24T23:10:50.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.947+0000 7f10d443f640 1 -- 172.21.15.32:0/2088346625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10cc157550 con 0x7f10cc154ab0 2024-04-24T23:10:50.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.948+0000 7f10d443f640 1 -- 172.21.15.32:0/2088346625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10cc068430 con 0x7f10cc154ab0 2024-04-24T23:10:50.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.948+0000 7f10c2ffd640 1 -- 172.21.15.32:0/2088346625 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f10bc00d8d0 con 0x7f10cc154ab0 2024-04-24T23:10:50.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.949+0000 7f10c2ffd640 1 --2- 172.21.15.32:0/2088346625 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f10b006be60 0x7f10b006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:50.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.949+0000 7f10c2ffd640 1 -- 172.21.15.32:0/2088346625 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 5709+0+0 (secure 0 0 0) 0x7f10bc08acc0 con 0x7f10cc154ab0 2024-04-24T23:10:50.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.949+0000 7f10d21db640 1 --2- 172.21.15.32:0/2088346625 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f10b006be60 0x7f10b006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:50.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.950+0000 7f10d21db640 1 --2- 172.21.15.32:0/2088346625 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f10b006be60 0x7f10b006e320 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f10c8033000 tx=0x7f10c8008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:50.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:50.953+0000 7f10c2ffd640 1 -- 172.21.15.32:0/2088346625 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f10bc059700 con 0x7f10cc154ab0 2024-04-24T23:10:51.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:51.143+0000 7f10d443f640 1 -- 172.21.15.32:0/2088346625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f10cc0b5670 con 0x7f10cc154ab0 2024-04-24T23:10:51.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:51.144+0000 7f10c2ffd640 1 -- 172.21.15.32:0/2088346625 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v186) ==== 81+0+34188 (secure 0 0 0) 0x7f10bc059000 con 0x7f10cc154ab0 2024-04-24T23:10:51.146 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:10:51.146 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:10:51.146 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 186, 2024-04-24T23:10:51.147 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", 2024-04-24T23:10:51.147 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2024-04-24T22:58:46.964851+0000", 2024-04-24T23:10:51.147 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2024-04-24T23:10:45.031422+0000", 2024-04-24T23:10:51.147 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2024-04-24T22:58:52.048787+0000", 2024-04-24T23:10:51.147 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2024-04-24T22:58:49.045609+0000", 2024-04-24T23:10:51.147 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-24T23:10:51.147 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2024-04-24T23:10:51.147 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2024-04-24T23:10:51.147 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2024-04-24T23:10:51.147 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2024-04-24T23:10:51.147 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2024-04-24T23:10:51.147 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2024-04-24T23:10:51.147 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:10:51.148 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 4, 2024-04-24T23:10:51.148 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.69999998807907104, 2024-04-24T23:10:51.148 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-24T23:10:51.148 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-24T23:10:51.148 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2024-04-24T23:10:51.148 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 23, 2024-04-24T23:10:51.148 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2024-04-24T23:10:51.148 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "luminous", 2024-04-24T23:10:51.148 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "jewel", 2024-04-24T23:10:51.148 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "squid", 2024-04-24T23:10:51.148 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_crimson": false, 2024-04-24T23:10:51.148 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2024-04-24T23:10:51.148 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.148 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2024-04-24T23:10:51.149 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": ".mgr", 2024-04-24T23:10:51.149 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T22:58:53.171408+0000", 2024-04-24T23:10:51.149 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:10:51.149 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:10:51.149 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:10:51.149 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:10:51.149 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:10:51.149 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:10:51.149 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:10:51.149 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:10:51.149 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:10:51.149 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:10:51.149 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:10:51.149 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:10:51.150 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2024-04-24T23:10:51.150 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2024-04-24T23:10:51.150 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2024-04-24T23:10:51.150 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2024-04-24T23:10:51.150 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2024-04-24T23:10:51.150 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:10:51.150 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:10:51.150 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:10:51.150 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:10:51.150 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:10:51.150 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:10:51.150 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:10:51.151 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.151 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "113", 2024-04-24T23:10:51.151 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:10:51.151 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:10:51.151 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:10:51.151 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:10:51.151 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:10:51.151 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:10:51.151 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:10:51.151 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:10:51.151 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:10:51.151 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:10:51.151 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:10:51.151 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:10:51.151 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:10:51.152 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:10:51.152 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:10:51.152 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:10:51.152 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:10:51.152 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:10:51.152 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:10:51.152 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:10:51.152 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:10:51.152 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:10:51.152 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:10:51.152 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:10:51.152 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:10:51.152 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:10:51.153 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.153 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:10:51.153 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:10:51.153 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:10:51.153 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:10:51.153 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:10:51.153 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:10:51.153 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:10:51.153 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:10:51.153 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:10:51.153 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:10:51.153 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:10:51.153 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:10:51.153 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_max": 32, 2024-04-24T23:10:51.153 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2024-04-24T23:10:51.154 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.154 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:10:51.154 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": {} 2024-04-24T23:10:51.154 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.154 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:10:51.154 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:10:51.154 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 8, 2024-04-24T23:10:51.154 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 8, 2024-04-24T23:10:51.154 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.25, 2024-04-24T23:10:51.154 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-04-24T23:10:51.154 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-04-24T23:10:51.154 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:10:51.154 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:10:51.155 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:10:51.155 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:51.155 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.155 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.155 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 22, 2024-04-24T23:10:51.155 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2024-04-24T23:10:51.155 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:10:34.649562+0000", 2024-04-24T23:10:51.155 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:10:51.155 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:10:51.155 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:10:51.156 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:10:51.156 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:10:51.156 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:10:51.156 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:10:51.156 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:10:51.156 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:10:51.156 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:10:51.156 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:10:51.156 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:10:51.156 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 32, 2024-04-24T23:10:51.156 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 32, 2024-04-24T23:10:51.156 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 32, 2024-04-24T23:10:51.156 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 32, 2024-04-24T23:10:51.157 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 32, 2024-04-24T23:10:51.157 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:10:51.157 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:10:51.157 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:10:51.157 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:10:51.157 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:10:51.157 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:10:51.157 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:10:51.157 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.157 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "184", 2024-04-24T23:10:51.157 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:10:51.157 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:10:51.157 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:10:51.157 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:10:51.157 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:10:51.158 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:10:51.158 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:10:51.158 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:10:51.158 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:10:51.158 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:10:51.158 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:10:51.158 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:10:51.158 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:10:51.158 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:10:51.158 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:10:51.158 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:10:51.158 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:10:51.158 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:10:51.158 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:10:51.159 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:10:51.159 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:10:51.159 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:10:51.159 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:10:51.159 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:10:51.159 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:10:51.159 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:10:51.159 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.159 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:10:51.159 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:10:51.159 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:10:51.159 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:10:51.159 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:10:51.160 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:10:51.160 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:10:51.160 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:10:51.160 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:10:51.160 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:10:51.160 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:10:51.160 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:10:51.160 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2024-04-24T23:10:51.160 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2024-04-24T23:10:51.160 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2024-04-24T23:10:51.160 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.160 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:10:51.160 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:10:51.160 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2024-04-24T23:10:51.160 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:51.161 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.161 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:10:51.161 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:10:51.161 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.5, 2024-04-24T23:10:51.161 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.5, 2024-04-24T23:10:51.161 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:10:51.161 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.5, 2024-04-24T23:10:51.161 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.5, 2024-04-24T23:10:51.161 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:10:51.161 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:10:51.161 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:10:51.161 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:51.161 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.161 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.162 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 23, 2024-04-24T23:10:51.162 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2024-04-24T23:10:51.162 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:10:35.339604+0000", 2024-04-24T23:10:51.162 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:10:51.162 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:10:51.162 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:10:51.162 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:10:51.162 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:10:51.162 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:10:51.162 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:10:51.162 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:10:51.162 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:10:51.162 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:10:51.162 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:10:51.163 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:10:51.163 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 64, 2024-04-24T23:10:51.163 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 64, 2024-04-24T23:10:51.163 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 64, 2024-04-24T23:10:51.163 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 64, 2024-04-24T23:10:51.163 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 64, 2024-04-24T23:10:51.163 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:10:51.163 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:10:51.163 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:10:51.163 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:10:51.163 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:10:51.163 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:10:51.163 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:10:51.163 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.164 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "186", 2024-04-24T23:10:51.164 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:10:51.164 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:10:51.164 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:10:51.164 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:10:51.164 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:10:51.164 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:10:51.164 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:10:51.164 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:10:51.164 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:10:51.164 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 33554432, 2024-04-24T23:10:51.164 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:10:51.164 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:10:51.164 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:10:51.165 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:10:51.165 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:10:51.165 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:10:51.165 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:10:51.165 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:10:51.165 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:10:51.165 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:10:51.165 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:10:51.165 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:10:51.165 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:10:51.165 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:10:51.165 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:10:51.165 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:10:51.165 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.165 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:10:51.166 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:10:51.166 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:10:51.166 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:10:51.166 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:10:51.166 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:10:51.166 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:10:51.166 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:10:51.166 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:10:51.166 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:10:51.166 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:10:51.166 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:10:51.166 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 64, 2024-04-24T23:10:51.166 INFO:teuthology.orchestra.run.smithi032.stdout: "target_size_ratio": 0.89999997615814209 2024-04-24T23:10:51.166 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.167 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:10:51.167 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:10:51.167 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2024-04-24T23:10:51.167 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:51.167 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.167 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:10:51.167 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:10:51.167 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.3799999952316284, 2024-04-24T23:10:51.167 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.3799999952316284, 2024-04-24T23:10:51.167 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:10:51.167 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.3799999952316284, 2024-04-24T23:10:51.167 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.3799999952316284, 2024-04-24T23:10:51.167 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:10:51.168 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:10:51.168 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:10:51.168 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:51.168 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:51.168 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:10:51.168 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-04-24T23:10:51.168 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.168 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:10:51.168 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:10:51.168 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:10:51.168 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:10:51.168 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:10:51.168 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:10:51.168 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:10:51.169 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:10:51.169 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:10:51.169 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:10:51.169 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:10:51.169 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:10:51.169 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:10:51.169 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:51.169 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.169 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:51.169 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2024-04-24T23:10:51.169 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:10:51.169 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.169 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.169 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:51.169 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2024-04-24T23:10:51.170 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:10:51.170 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:51.170 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:51.170 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.170 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:10:51.170 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:51.170 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.170 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:51.170 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2024-04-24T23:10:51.170 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:10:51.170 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.170 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.170 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:51.171 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2024-04-24T23:10:51.171 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:10:51.171 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:51.171 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:51.171 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.171 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:10:51.171 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:51.171 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.171 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:51.171 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2024-04-24T23:10:51.171 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:10:51.171 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.171 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.171 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:51.172 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2024-04-24T23:10:51.172 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:10:51.172 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:51.172 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:51.172 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.172 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:10:51.172 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:51.172 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.172 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:51.172 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2024-04-24T23:10:51.172 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:10:51.172 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.172 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.172 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:51.172 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2024-04-24T23:10:51.173 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:10:51.173 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:51.173 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:51.173 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.173 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/1479395426", 2024-04-24T23:10:51.173 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/1479395426", 2024-04-24T23:10:51.173 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/1479395426", 2024-04-24T23:10:51.173 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/1479395426", 2024-04-24T23:10:51.173 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:10:51.173 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:10:51.173 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:10:51.173 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:51.173 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.173 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.174 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:10:51.174 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:10:51.174 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:10:51.174 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:10:51.174 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:10:51.174 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:10:51.174 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:10:51.174 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:10:51.174 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:10:51.174 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:10:51.174 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:10:51.174 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:10:51.174 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:10:51.174 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:51.175 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.175 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:51.175 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2024-04-24T23:10:51.175 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:10:51.175 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.175 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.175 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:51.175 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2024-04-24T23:10:51.175 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:10:51.175 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:51.175 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:51.175 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.175 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:10:51.175 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:51.176 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.176 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:51.176 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2024-04-24T23:10:51.176 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:10:51.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.176 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.176 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:51.176 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2024-04-24T23:10:51.177 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:10:51.177 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:51.177 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:51.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.177 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:10:51.177 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:51.177 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.177 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:51.177 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2024-04-24T23:10:51.177 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:10:51.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.177 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.177 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:51.177 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2024-04-24T23:10:51.178 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:10:51.178 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:51.178 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:51.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.178 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:10:51.178 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:51.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.178 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:51.178 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2024-04-24T23:10:51.178 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:10:51.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.179 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:51.179 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2024-04-24T23:10:51.179 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:10:51.179 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:51.179 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:51.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.179 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/1340633718", 2024-04-24T23:10:51.179 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/1340633718", 2024-04-24T23:10:51.179 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/1340633718", 2024-04-24T23:10:51.179 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/1340633718", 2024-04-24T23:10:51.179 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:10:51.179 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:10:51.179 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:10:51.180 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:51.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.180 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.180 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:10:51.180 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:10:51.180 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:10:51.180 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:10:51.180 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:10:51.180 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:10:51.180 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:10:51.180 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:10:51.180 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:10:51.180 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:10:51.180 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:10:51.180 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:10:51.181 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:10:51.181 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:51.181 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.181 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:51.181 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2024-04-24T23:10:51.181 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:10:51.181 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.181 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.181 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:51.181 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2024-04-24T23:10:51.181 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:10:51.181 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:51.181 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:51.181 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.181 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:10:51.182 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:51.182 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.182 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:51.182 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2024-04-24T23:10:51.182 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:10:51.182 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.182 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.182 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:51.182 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2024-04-24T23:10:51.182 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:10:51.182 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:51.182 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:51.182 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.183 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:10:51.183 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:51.183 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.183 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:51.183 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2024-04-24T23:10:51.183 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:10:51.183 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.183 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.183 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:51.183 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2024-04-24T23:10:51.183 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:10:51.183 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:51.183 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:51.183 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.183 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:10:51.184 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:51.184 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.184 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:51.184 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2024-04-24T23:10:51.184 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:10:51.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.184 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.184 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:51.184 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2024-04-24T23:10:51.184 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:10:51.184 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:51.184 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:51.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.184 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/3405899334", 2024-04-24T23:10:51.185 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6820/3405899334", 2024-04-24T23:10:51.185 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6828/3405899334", 2024-04-24T23:10:51.185 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6824/3405899334", 2024-04-24T23:10:51.185 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:10:51.185 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:10:51.185 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:10:51.185 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:51.185 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.185 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.185 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:10:51.185 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:10:51.185 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:10:51.185 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:10:51.185 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:10:51.186 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:10:51.186 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:10:51.186 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:10:51.186 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:10:51.186 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:10:51.186 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:10:51.186 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:10:51.186 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:10:51.186 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:51.186 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.186 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:51.186 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2024-04-24T23:10:51.186 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:10:51.186 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.187 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.187 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:51.187 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2024-04-24T23:10:51.187 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:10:51.187 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:51.187 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:51.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.187 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:10:51.187 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:51.187 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.187 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:51.187 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2024-04-24T23:10:51.187 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:10:51.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.188 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.188 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:51.188 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2024-04-24T23:10:51.188 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:10:51.188 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:51.188 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:51.188 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.188 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:10:51.188 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:51.188 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.188 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:51.188 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2024-04-24T23:10:51.188 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:10:51.188 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.189 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.189 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:51.189 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2024-04-24T23:10:51.189 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:10:51.189 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:51.189 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:51.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.189 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:10:51.189 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:51.189 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.189 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:51.189 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2024-04-24T23:10:51.189 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:10:51.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.189 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.190 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:51.190 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2024-04-24T23:10:51.190 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:10:51.190 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:51.190 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:51.190 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.190 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6821/3381057190", 2024-04-24T23:10:51.190 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6825/3381057190", 2024-04-24T23:10:51.190 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/3381057190", 2024-04-24T23:10:51.190 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/3381057190", 2024-04-24T23:10:51.190 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:10:51.190 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:10:51.190 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:10:51.190 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:51.191 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.191 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.191 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:10:51.191 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:10:51.191 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:10:51.191 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:10:51.191 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:10:51.191 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:10:51.191 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:10:51.191 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:10:51.191 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:10:51.191 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:10:51.191 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:10:51.191 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:10:51.192 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:10:51.192 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:51.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.192 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:51.192 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6800", 2024-04-24T23:10:51.192 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:10:51.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.192 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:51.192 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6801", 2024-04-24T23:10:51.192 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:10:51.192 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:51.192 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:51.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.193 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:10:51.193 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:51.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.193 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:51.193 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6802", 2024-04-24T23:10:51.193 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:10:51.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.193 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:51.193 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6803", 2024-04-24T23:10:51.193 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:10:51.193 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:51.194 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:51.194 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.194 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:10:51.194 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:51.194 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.194 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:51.194 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6806", 2024-04-24T23:10:51.194 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:10:51.194 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.194 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.194 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:51.194 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6807", 2024-04-24T23:10:51.194 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:10:51.195 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:51.195 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:51.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.195 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:10:51.195 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:51.195 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.195 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:51.195 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6804", 2024-04-24T23:10:51.195 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:10:51.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.195 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.195 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:51.195 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6805", 2024-04-24T23:10:51.196 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:10:51.196 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:51.196 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:51.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.196 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6801/4010553979", 2024-04-24T23:10:51.196 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6803/4010553979", 2024-04-24T23:10:51.196 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6807/4010553979", 2024-04-24T23:10:51.196 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6805/4010553979", 2024-04-24T23:10:51.196 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:10:51.196 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:10:51.196 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:10:51.196 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:51.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.197 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.197 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:10:51.197 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:10:51.197 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:10:51.197 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:10:51.197 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:10:51.197 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:10:51.197 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:10:51.197 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:10:51.197 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:10:51.197 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:10:51.197 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:10:51.197 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:10:51.197 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:10:51.198 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:51.198 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.198 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:51.198 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6808", 2024-04-24T23:10:51.198 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:10:51.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.198 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.198 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:51.198 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6809", 2024-04-24T23:10:51.198 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:10:51.198 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:51.198 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:51.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.198 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:10:51.199 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:51.199 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.199 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:51.199 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6810", 2024-04-24T23:10:51.199 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:10:51.199 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.199 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.199 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:51.199 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6811", 2024-04-24T23:10:51.199 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:10:51.199 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:51.199 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:51.199 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.200 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:10:51.200 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:51.200 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.200 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:51.200 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6814", 2024-04-24T23:10:51.200 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:10:51.200 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.200 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.200 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:51.200 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6815", 2024-04-24T23:10:51.200 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:10:51.200 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:51.200 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:51.200 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.201 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:10:51.201 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:51.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.201 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:51.201 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6812", 2024-04-24T23:10:51.201 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:10:51.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.201 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:51.201 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6813", 2024-04-24T23:10:51.201 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:10:51.201 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:51.201 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:51.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.202 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6809/3794084091", 2024-04-24T23:10:51.202 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6811/3794084091", 2024-04-24T23:10:51.202 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6815/3794084091", 2024-04-24T23:10:51.202 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6813/3794084091", 2024-04-24T23:10:51.202 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:10:51.202 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:10:51.202 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:10:51.202 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:51.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.202 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:10:51.202 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:10:51.202 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:10:51.202 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:10:51.203 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:10:51.203 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:10:51.203 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:10:51.203 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:10:51.203 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:10:51.203 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:10:51.203 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:10:51.203 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:10:51.203 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:10:51.203 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:51.203 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.203 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:51.203 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6823", 2024-04-24T23:10:51.204 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:10:51.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.204 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.204 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:51.204 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6825", 2024-04-24T23:10:51.204 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:10:51.204 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:51.204 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:51.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.204 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:10:51.204 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:51.204 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.204 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:51.204 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6826", 2024-04-24T23:10:51.204 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:10:51.205 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.205 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:51.205 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6827", 2024-04-24T23:10:51.205 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:10:51.205 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:51.205 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:51.205 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.205 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:10:51.205 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:51.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.205 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:51.205 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6830", 2024-04-24T23:10:51.206 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:10:51.206 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.206 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.206 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:51.206 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6831", 2024-04-24T23:10:51.206 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:10:51.206 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:51.206 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:51.206 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.206 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:10:51.206 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:51.206 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.206 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:51.207 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6828", 2024-04-24T23:10:51.207 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:10:51.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.207 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.207 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:51.207 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6829", 2024-04-24T23:10:51.207 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:10:51.207 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:51.207 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:51.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.207 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6825/1348640258", 2024-04-24T23:10:51.207 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6827/1348640258", 2024-04-24T23:10:51.207 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6831/1348640258", 2024-04-24T23:10:51.207 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6829/1348640258", 2024-04-24T23:10:51.208 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:10:51.208 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:10:51.208 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:10:51.208 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:51.208 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.208 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.208 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:10:51.208 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:10:51.208 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:10:51.208 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:10:51.208 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:10:51.208 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:10:51.208 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:10:51.208 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:10:51.208 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:10:51.209 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:10:51.209 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:10:51.209 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:10:51.209 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:10:51.209 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:51.209 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.209 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:51.209 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6816", 2024-04-24T23:10:51.209 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:10:51.209 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.209 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.209 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:51.209 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6817", 2024-04-24T23:10:51.210 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:10:51.210 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:51.210 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:51.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.210 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:10:51.210 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:51.210 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.210 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:51.210 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6818", 2024-04-24T23:10:51.210 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:10:51.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.210 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.210 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:51.210 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6819", 2024-04-24T23:10:51.211 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:10:51.211 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:51.211 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:51.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.211 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:10:51.211 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:51.211 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.211 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:51.211 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6822", 2024-04-24T23:10:51.211 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:10:51.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.211 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.211 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:51.211 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6824", 2024-04-24T23:10:51.212 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:10:51.212 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:51.212 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:51.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.212 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:10:51.212 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:51.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.212 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:51.212 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6820", 2024-04-24T23:10:51.212 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:10:51.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.212 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:51.212 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6821", 2024-04-24T23:10:51.213 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:10:51.213 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:51.213 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:51.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.213 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6817/3250005122", 2024-04-24T23:10:51.213 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6819/3250005122", 2024-04-24T23:10:51.213 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6824/3250005122", 2024-04-24T23:10:51.213 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6821/3250005122", 2024-04-24T23:10:51.213 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:10:51.213 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:10:51.213 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:10:51.213 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:51.213 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:51.213 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:10:51.213 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2024-04-24T23:10:51.214 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.214 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:10:51.214 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:10:51.214 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:10:51.214 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:10:51.214 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:10:51.214 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:10:51.214 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.156184+0000", 2024-04-24T23:10:51.214 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:10:51.214 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.214 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.214 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:10:51.214 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:10:51.214 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:10:51.215 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:10:51.215 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:10:51.215 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:10:51.215 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240774+0000", 2024-04-24T23:10:51.215 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:10:51.215 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.215 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.215 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:10:51.215 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:10:51.215 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:10:51.215 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:10:51.215 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:10:51.215 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:10:51.215 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240949+0000", 2024-04-24T23:10:51.216 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:10:51.216 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.216 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.216 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:10:51.216 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:10:51.216 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:10:51.216 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:10:51.216 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:10:51.216 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:10:51.216 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.171672+0000", 2024-04-24T23:10:51.216 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:10:51.216 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.216 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.216 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:10:51.216 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:10:51.217 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:10:51.217 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:10:51.217 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:10:51.217 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:10:51.217 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.220379+0000", 2024-04-24T23:10:51.217 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:10:51.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.217 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.217 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:10:51.217 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:10:51.217 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:10:51.217 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:10:51.217 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:10:51.217 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:10:51.217 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.204963+0000", 2024-04-24T23:10:51.218 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:10:51.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.218 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:10:51.218 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:10:51.218 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:10:51.218 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:10:51.218 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:10:51.218 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:10:51.218 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.239748+0000", 2024-04-24T23:10:51.218 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:10:51.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:51.219 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:10:51.219 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:10:51.219 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:10:51.219 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:10:51.219 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:10:51.219 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:10:51.219 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.154104+0000", 2024-04-24T23:10:51.219 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:10:51.219 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:51.219 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:10:51.219 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2024-04-24T23:10:51.219 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2024-04-24T23:10:51.219 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_primaries": [], 2024-04-24T23:10:51.219 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2024-04-24T23:10:51.219 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2024-04-24T23:10:51.220 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklist": {}, 2024-04-24T23:10:51.220 INFO:teuthology.orchestra.run.smithi032.stdout: "range_blocklist": {}, 2024-04-24T23:10:51.220 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2024-04-24T23:10:51.220 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2024-04-24T23:10:51.220 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2024-04-24T23:10:51.220 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2024-04-24T23:10:51.220 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2024-04-24T23:10:51.220 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2024-04-24T23:10:51.220 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2024-04-24T23:10:51.220 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:51.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:51.220 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2024-04-24T23:10:51.220 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2024-04-24T23:10:51.220 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2024-04-24T23:10:51.220 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2024-04-24T23:10:51.221 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {}, 2024-04-24T23:10:51.221 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode": { 2024-04-24T23:10:51.221 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_enabled": false, 2024-04-24T23:10:51.221 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_bucket_count": 0, 2024-04-24T23:10:51.221 INFO:teuthology.orchestra.run.smithi032.stdout: "degraded_stretch_mode": 0, 2024-04-24T23:10:51.221 INFO:teuthology.orchestra.run.smithi032.stdout: "recovering_stretch_mode": 0, 2024-04-24T23:10:51.221 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_bucket": 0 2024-04-24T23:10:51.221 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:51.221 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:10:51.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:51.146+0000 7f10d443f640 1 -- 172.21.15.32:0/2088346625 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f10b006be60 msgr2=0x7f10b006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:51.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:51.146+0000 7f10d443f640 1 --2- 172.21.15.32:0/2088346625 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f10b006be60 0x7f10b006e320 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f10c8033000 tx=0x7f10c8008030 comp rx=0 tx=0).stop 2024-04-24T23:10:51.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:51.146+0000 7f10d443f640 1 -- 172.21.15.32:0/2088346625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10cc154ab0 msgr2=0x7f10cc15e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:51.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:51.146+0000 7f10d443f640 1 --2- 172.21.15.32:0/2088346625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10cc154ab0 0x7f10cc15e9f0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f10bc00f770 tx=0x7f10bc00fc30 comp rx=0 tx=0).stop 2024-04-24T23:10:51.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:51.146+0000 7f10d443f640 1 -- 172.21.15.32:0/2088346625 shutdown_connections 2024-04-24T23:10:51.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:51.146+0000 7f10d443f640 1 --2- 172.21.15.32:0/2088346625 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f10b006be60 0x7f10b006e320 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:51.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:51.146+0000 7f10d443f640 1 --2- 172.21.15.32:0/2088346625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10cc154ab0 0x7f10cc15e9f0 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:51.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:51.146+0000 7f10d443f640 1 --2- 172.21.15.32:0/2088346625 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10cc15ef30 0x7f10cc15f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:51.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:51.146+0000 7f10d443f640 1 --2- 172.21.15.32:0/2088346625 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f10b80039e0 0x7f10cc154570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:51.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:51.146+0000 7f10d443f640 1 -- 172.21.15.32:0/2088346625 >> 172.21.15.32:0/2088346625 conn(0x7f10cc06b110 msgr2=0x7f10cc0cc500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:51.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:51.147+0000 7f10d443f640 1 -- 172.21.15.32:0/2088346625 shutdown_connections 2024-04-24T23:10:51.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:51.147+0000 7f10d443f640 1 -- 172.21.15.32:0/2088346625 wait complete. 2024-04-24T23:10:51.224 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-04-24T23:10:56.225 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-24T23:10:56.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.336+0000 7f80b9e9a640 1 Processor -- start 2024-04-24T23:10:56.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.336+0000 7f80b9e9a640 1 -- start start 2024-04-24T23:10:56.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.337+0000 7f80b9e9a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f80b415e9b0 0x7f80b40d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:56.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.337+0000 7f80b9e9a640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f80b40d4210 0x7f80b406b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:56.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.337+0000 7f80b9e9a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f80b40ba0e0 0x7f80b406b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:56.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.337+0000 7f80b9e9a640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f80b40d3170 con 0x7f80b40ba0e0 2024-04-24T23:10:56.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.337+0000 7f80b9e9a640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f80b40d3450 con 0x7f80b40d4210 2024-04-24T23:10:56.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.337+0000 7f80b9e9a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f80b40d3730 con 0x7f80b415e9b0 2024-04-24T23:10:56.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.337+0000 7f80b3fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f80b40ba0e0 0x7f80b406b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:56.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.337+0000 7f80b3fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f80b40ba0e0 0x7f80b406b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41102/0 (socket says 172.21.15.32:41102) 2024-04-24T23:10:56.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.337+0000 7f80b3fff640 1 -- 172.21.15.32:0/2662293297 learned_addr learned my addr 172.21.15.32:0/2662293297 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:56.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.338+0000 7f80b37fe640 1 --2- 172.21.15.32:0/2662293297 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f80b415e9b0 0x7f80b40d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:56.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.338+0000 7f80b2ffd640 1 --2- 172.21.15.32:0/2662293297 >> v2:172.21.15.121:3300/0 conn(0x7f80b40d4210 0x7f80b406b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:56.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.338+0000 7f80b3fff640 1 -- 172.21.15.32:0/2662293297 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f80b415e9b0 msgr2=0x7f80b40d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:56.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.338+0000 7f80b3fff640 1 --2- 172.21.15.32:0/2662293297 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f80b415e9b0 0x7f80b40d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:56.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.338+0000 7f80b3fff640 1 -- 172.21.15.32:0/2662293297 >> v2:172.21.15.121:3300/0 conn(0x7f80b40d4210 msgr2=0x7f80b406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:56.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.338+0000 7f80b3fff640 1 --2- 172.21.15.32:0/2662293297 >> v2:172.21.15.121:3300/0 conn(0x7f80b40d4210 0x7f80b406b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:56.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.338+0000 7f80b3fff640 1 -- 172.21.15.32:0/2662293297 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80b40d3a10 con 0x7f80b40ba0e0 2024-04-24T23:10:56.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.339+0000 7f80b3fff640 1 --2- 172.21.15.32:0/2662293297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80b40ba0e0 0x7f80b406b650 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f80b406bc40 tx=0x7f80ac02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f295472482abade server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:56.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.339+0000 7f80b27fc640 1 -- 172.21.15.32:0/2662293297 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ac036070 con 0x7f80b40ba0e0 2024-04-24T23:10:56.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.339+0000 7f80b27fc640 1 -- 172.21.15.32:0/2662293297 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80ac031600 con 0x7f80b40ba0e0 2024-04-24T23:10:56.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.339+0000 7f80b27fc640 1 -- 172.21.15.32:0/2662293297 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ac0319d0 con 0x7f80b40ba0e0 2024-04-24T23:10:56.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.339+0000 7f80b9e9a640 1 -- 172.21.15.32:0/2662293297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80b40ba0e0 msgr2=0x7f80b406b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:56.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.339+0000 7f80b9e9a640 1 --2- 172.21.15.32:0/2662293297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80b40ba0e0 0x7f80b406b650 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f80b406bc40 tx=0x7f80ac02a6a0 comp rx=0 tx=0).stop 2024-04-24T23:10:56.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.339+0000 7f80b9e9a640 1 -- 172.21.15.32:0/2662293297 shutdown_connections 2024-04-24T23:10:56.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.339+0000 7f80b9e9a640 1 --2- 172.21.15.32:0/2662293297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80b40ba0e0 0x7f80b406b650 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:56.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.339+0000 7f80b9e9a640 1 --2- 172.21.15.32:0/2662293297 >> v2:172.21.15.121:3300/0 conn(0x7f80b40d4210 0x7f80b406b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:56.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.339+0000 7f80b9e9a640 1 --2- 172.21.15.32:0/2662293297 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f80b415e9b0 0x7f80b40d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:56.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.339+0000 7f80b9e9a640 1 -- 172.21.15.32:0/2662293297 >> 172.21.15.32:0/2662293297 conn(0x7f80b40d11d0 msgr2=0x7f80b40d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:56.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.339+0000 7f80b9e9a640 1 -- 172.21.15.32:0/2662293297 shutdown_connections 2024-04-24T23:10:56.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.339+0000 7f80b9e9a640 1 -- 172.21.15.32:0/2662293297 wait complete. 2024-04-24T23:10:56.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.341+0000 7f80b9e9a640 1 Processor -- start 2024-04-24T23:10:56.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.341+0000 7f80b9e9a640 1 -- start start 2024-04-24T23:10:56.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.342+0000 7f80b9e9a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80b40ba0e0 0x7f80b41501f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:56.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.342+0000 7f80b9e9a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80b40d4210 0x7f80b40ccd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:56.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.342+0000 7f80b9e9a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f80b415e9b0 0x7f80b40cd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:56.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.342+0000 7f80b9e9a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f80b4071730 con 0x7f80b40ba0e0 2024-04-24T23:10:56.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.342+0000 7f80b9e9a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f80b40d2400 con 0x7f80b40d4210 2024-04-24T23:10:56.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.342+0000 7f80b9e9a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f80b40718b0 con 0x7f80b415e9b0 2024-04-24T23:10:56.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.342+0000 7f80b37fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80b40ba0e0 0x7f80b41501f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:56.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.342+0000 7f80b37fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80b40ba0e0 0x7f80b41501f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41106/0 (socket says 172.21.15.32:41106) 2024-04-24T23:10:56.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.342+0000 7f80b37fe640 1 -- 172.21.15.32:0/312896415 learned_addr learned my addr 172.21.15.32:0/312896415 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:56.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.342+0000 7f80b2ffd640 1 --2- 172.21.15.32:0/312896415 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80b40d4210 0x7f80b40ccd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:56.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.342+0000 7f80b3fff640 1 --2- 172.21.15.32:0/312896415 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f80b415e9b0 0x7f80b40cd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:56.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.342+0000 7f80b37fe640 1 -- 172.21.15.32:0/312896415 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f80b415e9b0 msgr2=0x7f80b40cd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:56.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.342+0000 7f80b37fe640 1 --2- 172.21.15.32:0/312896415 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f80b415e9b0 0x7f80b40cd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:56.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.342+0000 7f80b37fe640 1 -- 172.21.15.32:0/312896415 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80b40d4210 msgr2=0x7f80b40ccd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:56.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.342+0000 7f80b37fe640 1 --2- 172.21.15.32:0/312896415 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80b40d4210 0x7f80b40ccd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:56.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.342+0000 7f80b37fe640 1 -- 172.21.15.32:0/312896415 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80b4161c60 con 0x7f80b40ba0e0 2024-04-24T23:10:56.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.342+0000 7f80b37fe640 1 --2- 172.21.15.32:0/312896415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80b40ba0e0 0x7f80b41501f0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f80a400e870 tx=0x7f80a400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:56.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.342+0000 7f809bfff640 1 -- 172.21.15.32:0/312896415 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a4017070 con 0x7f80b40ba0e0 2024-04-24T23:10:56.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.342+0000 7f809bfff640 1 -- 172.21.15.32:0/312896415 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80a400c940 con 0x7f80b40ba0e0 2024-04-24T23:10:56.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.342+0000 7f80b9e9a640 1 -- 172.21.15.32:0/312896415 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80b40c83b0 con 0x7f80b40ba0e0 2024-04-24T23:10:56.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.342+0000 7f809bfff640 1 -- 172.21.15.32:0/312896415 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a400cd30 con 0x7f80b40ba0e0 2024-04-24T23:10:56.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.343+0000 7f80b9e9a640 1 -- 172.21.15.32:0/312896415 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80b40c8930 con 0x7f80b40ba0e0 2024-04-24T23:10:56.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.343+0000 7f80b9e9a640 1 -- 172.21.15.32:0/312896415 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80b415e210 con 0x7f80b40ba0e0 2024-04-24T23:10:56.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.344+0000 7f809bfff640 1 -- 172.21.15.32:0/312896415 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f80a401a070 con 0x7f80b40ba0e0 2024-04-24T23:10:56.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.344+0000 7f809bfff640 1 --2- 172.21.15.32:0/312896415 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f80a006be60 0x7f80a006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:56.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.345+0000 7f809bfff640 1 -- 172.21.15.32:0/312896415 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(189..189 src has 1..189) ==== 5738+0+0 (secure 0 0 0) 0x7f80a408acd0 con 0x7f80b40ba0e0 2024-04-24T23:10:56.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.345+0000 7f809bfff640 1 -- 172.21.15.32:0/312896415 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=190}) -- 0x7f80a0073a50 con 0x7f80b40ba0e0 2024-04-24T23:10:56.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.345+0000 7f80b2ffd640 1 --2- 172.21.15.32:0/312896415 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f80a006be60 0x7f80a006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:56.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.345+0000 7f80b2ffd640 1 --2- 172.21.15.32:0/312896415 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f80a006be60 0x7f80a006e320 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f80b415a530 tx=0x7f80a8009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:56.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.349+0000 7f809bfff640 1 -- 172.21.15.32:0/312896415 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f80b415e210 con 0x7f80b40ba0e0 2024-04-24T23:10:56.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.532+0000 7f80b9e9a640 1 -- 172.21.15.32:0/312896415 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f80b4068430 con 0x7f80b40ba0e0 2024-04-24T23:10:56.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.533+0000 7f809bfff640 1 -- 172.21.15.32:0/312896415 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v189) ==== 81+0+34398 (secure 0 0 0) 0x7f80a4059020 con 0x7f80b40ba0e0 2024-04-24T23:10:56.535 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:10:56.535 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:10:56.535 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 189, 2024-04-24T23:10:56.535 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", 2024-04-24T23:10:56.535 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2024-04-24T22:58:46.964851+0000", 2024-04-24T23:10:56.536 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2024-04-24T23:10:54.060025+0000", 2024-04-24T23:10:56.536 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2024-04-24T22:58:52.048787+0000", 2024-04-24T23:10:56.536 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2024-04-24T22:58:49.045609+0000", 2024-04-24T23:10:56.536 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-24T23:10:56.536 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2024-04-24T23:10:56.536 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2024-04-24T23:10:56.536 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2024-04-24T23:10:56.536 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2024-04-24T23:10:56.536 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2024-04-24T23:10:56.536 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2024-04-24T23:10:56.536 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:10:56.536 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 4, 2024-04-24T23:10:56.536 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.69999998807907104, 2024-04-24T23:10:56.536 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-24T23:10:56.536 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-24T23:10:56.537 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2024-04-24T23:10:56.537 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 23, 2024-04-24T23:10:56.537 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2024-04-24T23:10:56.537 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "luminous", 2024-04-24T23:10:56.537 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "luminous", 2024-04-24T23:10:56.537 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "squid", 2024-04-24T23:10:56.537 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_crimson": false, 2024-04-24T23:10:56.537 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2024-04-24T23:10:56.537 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.537 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2024-04-24T23:10:56.537 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": ".mgr", 2024-04-24T23:10:56.537 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T22:58:53.171408+0000", 2024-04-24T23:10:56.537 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:10:56.537 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:10:56.538 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:10:56.538 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:10:56.538 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:10:56.538 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:10:56.538 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:10:56.538 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:10:56.538 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:10:56.538 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:10:56.538 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:10:56.538 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:10:56.538 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2024-04-24T23:10:56.538 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2024-04-24T23:10:56.538 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2024-04-24T23:10:56.539 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2024-04-24T23:10:56.539 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2024-04-24T23:10:56.539 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:10:56.539 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:10:56.539 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:10:56.539 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:10:56.539 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:10:56.539 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:10:56.539 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:10:56.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.539 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "113", 2024-04-24T23:10:56.539 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:10:56.539 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:10:56.539 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:10:56.540 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:10:56.540 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:10:56.540 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:10:56.540 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:10:56.540 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:10:56.540 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:10:56.540 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:10:56.540 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:10:56.540 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:10:56.540 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:10:56.540 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:10:56.540 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:10:56.540 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:10:56.540 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:10:56.541 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:10:56.541 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:10:56.541 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:10:56.541 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:10:56.541 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:10:56.541 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:10:56.541 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:10:56.541 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:10:56.541 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:10:56.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.541 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:10:56.541 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:10:56.541 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:10:56.542 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:10:56.542 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:10:56.542 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:10:56.542 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:10:56.542 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:10:56.542 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:10:56.542 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:10:56.542 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:10:56.542 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:10:56.542 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_max": 32, 2024-04-24T23:10:56.542 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2024-04-24T23:10:56.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.542 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:10:56.542 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": {} 2024-04-24T23:10:56.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.543 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:10:56.543 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:10:56.543 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 8, 2024-04-24T23:10:56.543 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 8, 2024-04-24T23:10:56.543 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.25, 2024-04-24T23:10:56.543 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-04-24T23:10:56.543 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-04-24T23:10:56.543 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:10:56.543 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:10:56.543 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:10:56.543 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:56.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.543 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 22, 2024-04-24T23:10:56.544 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2024-04-24T23:10:56.544 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:10:34.649562+0000", 2024-04-24T23:10:56.544 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:10:56.544 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:10:56.544 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:10:56.544 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:10:56.544 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:10:56.544 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:10:56.544 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:10:56.544 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:10:56.544 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:10:56.544 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:10:56.544 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:10:56.544 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:10:56.544 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 32, 2024-04-24T23:10:56.545 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 32, 2024-04-24T23:10:56.545 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 32, 2024-04-24T23:10:56.545 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 32, 2024-04-24T23:10:56.545 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 32, 2024-04-24T23:10:56.545 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:10:56.545 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:10:56.545 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:10:56.545 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:10:56.545 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:10:56.545 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:10:56.545 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:10:56.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.545 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "184", 2024-04-24T23:10:56.545 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:10:56.546 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:10:56.546 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:10:56.546 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:10:56.546 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:10:56.546 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:10:56.546 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:10:56.546 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:10:56.546 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:10:56.546 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:10:56.546 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:10:56.546 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:10:56.546 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:10:56.546 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:10:56.546 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:10:56.546 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:10:56.547 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:10:56.547 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:10:56.547 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:10:56.547 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:10:56.547 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:10:56.547 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:10:56.547 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:10:56.547 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:10:56.547 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:10:56.547 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:10:56.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.547 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:10:56.547 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:10:56.547 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:10:56.548 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:10:56.548 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:10:56.548 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:10:56.548 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:10:56.548 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:10:56.548 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:10:56.548 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:10:56.548 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:10:56.548 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:10:56.548 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2024-04-24T23:10:56.548 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2024-04-24T23:10:56.548 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2024-04-24T23:10:56.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.548 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:10:56.548 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:10:56.548 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2024-04-24T23:10:56.549 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:56.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.549 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:10:56.549 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:10:56.549 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.5, 2024-04-24T23:10:56.549 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.5, 2024-04-24T23:10:56.549 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:10:56.549 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.5, 2024-04-24T23:10:56.549 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.5, 2024-04-24T23:10:56.549 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:10:56.549 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:10:56.549 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:10:56.549 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:56.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.550 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 23, 2024-04-24T23:10:56.550 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2024-04-24T23:10:56.550 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:10:35.339604+0000", 2024-04-24T23:10:56.550 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1027, 2024-04-24T23:10:56.550 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,full,full_quota", 2024-04-24T23:10:56.550 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:10:56.550 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:10:56.550 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:10:56.550 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:10:56.550 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:10:56.550 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:10:56.550 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:10:56.550 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:10:56.551 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:10:56.551 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:10:56.551 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 64, 2024-04-24T23:10:56.551 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 64, 2024-04-24T23:10:56.551 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 64, 2024-04-24T23:10:56.551 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 64, 2024-04-24T23:10:56.551 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 64, 2024-04-24T23:10:56.551 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:10:56.551 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:10:56.551 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:10:56.551 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:10:56.551 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:10:56.551 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:10:56.551 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:10:56.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.552 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "187", 2024-04-24T23:10:56.552 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:10:56.552 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:10:56.552 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:10:56.552 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:10:56.552 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:10:56.552 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:10:56.552 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:10:56.552 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:10:56.552 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:10:56.552 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 33554432, 2024-04-24T23:10:56.552 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:10:56.552 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:10:56.552 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:10:56.553 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:10:56.553 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:10:56.553 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:10:56.553 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:10:56.553 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:10:56.553 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:10:56.553 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:10:56.553 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:10:56.553 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:10:56.553 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:10:56.553 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:10:56.553 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:10:56.553 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:10:56.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.554 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:10:56.554 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:10:56.554 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:10:56.554 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:10:56.554 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:10:56.554 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:10:56.554 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:10:56.554 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:10:56.554 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:10:56.554 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:10:56.554 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:10:56.554 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:10:56.554 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 64, 2024-04-24T23:10:56.554 INFO:teuthology.orchestra.run.smithi032.stdout: "target_size_ratio": 0.89999997615814209 2024-04-24T23:10:56.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.555 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:10:56.555 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:10:56.555 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2024-04-24T23:10:56.555 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:56.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.555 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:10:56.555 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:10:56.555 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.3799999952316284, 2024-04-24T23:10:56.555 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.3799999952316284, 2024-04-24T23:10:56.555 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:10:56.555 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.3799999952316284, 2024-04-24T23:10:56.555 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.3799999952316284, 2024-04-24T23:10:56.555 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:10:56.555 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:10:56.555 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:10:56.556 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:56.556 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:56.556 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:10:56.556 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-04-24T23:10:56.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.556 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:10:56.556 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:10:56.556 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:10:56.556 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:10:56.556 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:10:56.556 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:10:56.556 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:10:56.556 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:10:56.556 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:10:56.557 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:10:56.557 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:10:56.557 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:10:56.557 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:10:56.557 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:56.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.557 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:56.557 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2024-04-24T23:10:56.557 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:10:56.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.557 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:56.557 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2024-04-24T23:10:56.557 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:10:56.557 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:56.557 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:56.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.558 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:10:56.558 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:56.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.558 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:56.558 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2024-04-24T23:10:56.558 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:10:56.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.558 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:56.558 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2024-04-24T23:10:56.558 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:10:56.558 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:56.558 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:56.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.559 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:10:56.559 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:56.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.559 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:56.559 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2024-04-24T23:10:56.559 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:10:56.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.559 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:56.559 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2024-04-24T23:10:56.559 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:10:56.559 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:56.559 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:56.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.560 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:10:56.560 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:56.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.560 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:56.560 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2024-04-24T23:10:56.560 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:10:56.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.560 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:56.560 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2024-04-24T23:10:56.560 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:10:56.560 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:56.560 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:56.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.560 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/1479395426", 2024-04-24T23:10:56.561 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/1479395426", 2024-04-24T23:10:56.561 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/1479395426", 2024-04-24T23:10:56.561 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/1479395426", 2024-04-24T23:10:56.561 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:10:56.561 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:10:56.561 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:10:56.561 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:56.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.561 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:10:56.561 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:10:56.561 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:10:56.561 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:10:56.561 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:10:56.562 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:10:56.562 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:10:56.562 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:10:56.562 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:10:56.562 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:10:56.562 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:10:56.562 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:10:56.562 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:10:56.562 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:56.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.562 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:56.562 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2024-04-24T23:10:56.562 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:10:56.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.563 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:56.563 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2024-04-24T23:10:56.563 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:10:56.563 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:56.563 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:56.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.563 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:10:56.563 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:56.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.563 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:56.563 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2024-04-24T23:10:56.563 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:10:56.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.564 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:56.564 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2024-04-24T23:10:56.564 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:10:56.564 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:56.564 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:56.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.564 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:10:56.564 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:56.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.564 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:56.564 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2024-04-24T23:10:56.564 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:10:56.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.564 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:56.565 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2024-04-24T23:10:56.565 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:10:56.565 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:56.565 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:56.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.565 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:10:56.565 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:56.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.565 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:56.565 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2024-04-24T23:10:56.565 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:10:56.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.565 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:56.566 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2024-04-24T23:10:56.566 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:10:56.566 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:56.566 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:56.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.566 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/1340633718", 2024-04-24T23:10:56.566 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/1340633718", 2024-04-24T23:10:56.566 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/1340633718", 2024-04-24T23:10:56.566 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/1340633718", 2024-04-24T23:10:56.566 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:10:56.566 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:10:56.566 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:10:56.566 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:56.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.567 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:10:56.567 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:10:56.567 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:10:56.567 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:10:56.567 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:10:56.567 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:10:56.567 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:10:56.567 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:10:56.567 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:10:56.567 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:10:56.567 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:10:56.567 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:10:56.567 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:10:56.567 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:56.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.568 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:56.568 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2024-04-24T23:10:56.568 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:10:56.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.568 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:56.568 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2024-04-24T23:10:56.568 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:10:56.568 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:56.568 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:56.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.568 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:10:56.568 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:56.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.569 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:56.569 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2024-04-24T23:10:56.569 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:10:56.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.569 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:56.569 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2024-04-24T23:10:56.569 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:10:56.569 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:56.569 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:56.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.570 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:10:56.570 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:56.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.570 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:56.570 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2024-04-24T23:10:56.570 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:10:56.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.570 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:56.570 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2024-04-24T23:10:56.570 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:10:56.570 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:56.570 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:56.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.571 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:10:56.571 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:56.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.571 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:56.571 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2024-04-24T23:10:56.571 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:10:56.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.571 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:56.571 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2024-04-24T23:10:56.572 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:10:56.572 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:56.572 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:56.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.572 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/3405899334", 2024-04-24T23:10:56.572 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6820/3405899334", 2024-04-24T23:10:56.572 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6828/3405899334", 2024-04-24T23:10:56.572 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6824/3405899334", 2024-04-24T23:10:56.572 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:10:56.572 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:10:56.572 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:10:56.572 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:56.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.573 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:10:56.573 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:10:56.573 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:10:56.573 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:10:56.573 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:10:56.573 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:10:56.573 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:10:56.573 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:10:56.573 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:10:56.573 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 188, 2024-04-24T23:10:56.573 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:10:56.573 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:10:56.573 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:10:56.573 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:56.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.574 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:56.574 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2024-04-24T23:10:56.574 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:10:56.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.574 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:56.574 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2024-04-24T23:10:56.574 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:10:56.574 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:56.574 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:56.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.574 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:10:56.574 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:56.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.575 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:56.575 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2024-04-24T23:10:56.575 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:10:56.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.575 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:56.575 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2024-04-24T23:10:56.575 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:10:56.575 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:56.575 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:56.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.575 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:10:56.575 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:56.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.575 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:56.576 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2024-04-24T23:10:56.576 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:10:56.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.576 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:56.576 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2024-04-24T23:10:56.576 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:10:56.576 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:56.576 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:56.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.576 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:10:56.576 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:56.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.576 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:56.577 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2024-04-24T23:10:56.577 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:10:56.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.577 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:56.577 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2024-04-24T23:10:56.577 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:10:56.577 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:56.577 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:56.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.577 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6821/3381057190", 2024-04-24T23:10:56.577 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6825/3381057190", 2024-04-24T23:10:56.577 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/3381057190", 2024-04-24T23:10:56.577 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/3381057190", 2024-04-24T23:10:56.577 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:10:56.578 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:10:56.578 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:10:56.578 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:56.578 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.578 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.578 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:10:56.578 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:10:56.578 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:10:56.578 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:10:56.578 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:10:56.578 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:10:56.578 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:10:56.578 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:10:56.578 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:10:56.579 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:10:56.579 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:10:56.579 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:10:56.579 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:10:56.579 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:56.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.579 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:56.579 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6800", 2024-04-24T23:10:56.579 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:10:56.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.579 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:56.579 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6801", 2024-04-24T23:10:56.579 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:10:56.579 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:56.580 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:56.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.580 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:10:56.580 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:56.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.580 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:56.580 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6802", 2024-04-24T23:10:56.580 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:10:56.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.580 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:56.580 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6803", 2024-04-24T23:10:56.580 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:10:56.580 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:56.580 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:56.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.581 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:10:56.581 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:56.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.581 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:56.581 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6806", 2024-04-24T23:10:56.581 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:10:56.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.581 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:56.581 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6807", 2024-04-24T23:10:56.581 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:10:56.581 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:56.581 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:56.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.582 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:10:56.582 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:56.582 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.582 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:56.582 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6804", 2024-04-24T23:10:56.582 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:10:56.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.582 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.582 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:56.582 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6805", 2024-04-24T23:10:56.582 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:10:56.582 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:56.582 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:56.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.583 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6801/4010553979", 2024-04-24T23:10:56.583 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6803/4010553979", 2024-04-24T23:10:56.583 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6807/4010553979", 2024-04-24T23:10:56.583 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6805/4010553979", 2024-04-24T23:10:56.583 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:10:56.583 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:10:56.583 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:10:56.583 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:56.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.583 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:10:56.583 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:10:56.583 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:10:56.583 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:10:56.583 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:10:56.584 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:10:56.584 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:10:56.584 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:10:56.584 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:10:56.584 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:10:56.584 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:10:56.584 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:10:56.584 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:10:56.584 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:56.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.584 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:56.584 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6808", 2024-04-24T23:10:56.584 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:10:56.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.585 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:56.585 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6809", 2024-04-24T23:10:56.585 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:10:56.585 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:56.585 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:56.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.585 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:10:56.585 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:56.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.585 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:56.585 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6810", 2024-04-24T23:10:56.585 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:10:56.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.586 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:56.586 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6811", 2024-04-24T23:10:56.586 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:10:56.586 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:56.586 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:56.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.586 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:10:56.586 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:56.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.586 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:56.586 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6814", 2024-04-24T23:10:56.586 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:10:56.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.587 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:56.587 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6815", 2024-04-24T23:10:56.587 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:10:56.587 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:56.587 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:56.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.587 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:10:56.587 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:56.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.587 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:56.587 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6812", 2024-04-24T23:10:56.587 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:10:56.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.588 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:56.588 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6813", 2024-04-24T23:10:56.588 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:10:56.588 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:56.588 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:56.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.588 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6809/3794084091", 2024-04-24T23:10:56.588 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6811/3794084091", 2024-04-24T23:10:56.588 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6815/3794084091", 2024-04-24T23:10:56.588 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6813/3794084091", 2024-04-24T23:10:56.588 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:10:56.588 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:10:56.588 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:10:56.588 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:56.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.589 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:10:56.589 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:10:56.589 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:10:56.589 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:10:56.589 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:10:56.589 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:10:56.589 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:10:56.589 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:10:56.589 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:10:56.589 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:10:56.589 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:10:56.590 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:10:56.590 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:10:56.590 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:56.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.590 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:56.590 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6823", 2024-04-24T23:10:56.590 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:10:56.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.590 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:56.590 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6825", 2024-04-24T23:10:56.590 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:10:56.590 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:56.590 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:56.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.591 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:10:56.591 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:56.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.591 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:56.591 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6826", 2024-04-24T23:10:56.591 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:10:56.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.591 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:56.591 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6827", 2024-04-24T23:10:56.591 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:10:56.591 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:56.591 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:56.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.592 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:10:56.592 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:56.592 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.592 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:56.592 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6830", 2024-04-24T23:10:56.592 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:10:56.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.592 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.592 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:56.592 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6831", 2024-04-24T23:10:56.592 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:10:56.592 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:56.592 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:56.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.592 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:10:56.593 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:56.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.593 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:56.593 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6828", 2024-04-24T23:10:56.593 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:10:56.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.593 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:56.593 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6829", 2024-04-24T23:10:56.593 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:10:56.593 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:56.593 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:56.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.593 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6825/1348640258", 2024-04-24T23:10:56.593 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6827/1348640258", 2024-04-24T23:10:56.594 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6831/1348640258", 2024-04-24T23:10:56.594 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6829/1348640258", 2024-04-24T23:10:56.594 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:10:56.594 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:10:56.594 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:10:56.594 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:56.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.594 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:10:56.594 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:10:56.594 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:10:56.594 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:10:56.594 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:10:56.594 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:10:56.595 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:10:56.595 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:10:56.595 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:10:56.595 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:10:56.595 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:10:56.595 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:10:56.595 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:10:56.595 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:56.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.595 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:56.595 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6816", 2024-04-24T23:10:56.595 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:10:56.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.596 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:56.596 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6817", 2024-04-24T23:10:56.596 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:10:56.596 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:56.596 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:56.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.596 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:10:56.596 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:56.596 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.596 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:56.596 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6818", 2024-04-24T23:10:56.596 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:10:56.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.597 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:56.597 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6819", 2024-04-24T23:10:56.597 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:10:56.597 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:56.597 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:56.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.597 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:10:56.597 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:56.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.597 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:56.597 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6822", 2024-04-24T23:10:56.597 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:10:56.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.598 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.598 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:56.598 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6824", 2024-04-24T23:10:56.598 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:10:56.598 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:56.598 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:56.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.598 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:10:56.598 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:56.598 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.598 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:56.598 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6820", 2024-04-24T23:10:56.598 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:10:56.599 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.599 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.599 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:56.599 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6821", 2024-04-24T23:10:56.599 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:10:56.599 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:56.599 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:56.599 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.599 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6817/3250005122", 2024-04-24T23:10:56.599 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6819/3250005122", 2024-04-24T23:10:56.599 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6824/3250005122", 2024-04-24T23:10:56.599 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6821/3250005122", 2024-04-24T23:10:56.599 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:10:56.599 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:10:56.599 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:10:56.600 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:56.600 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:56.600 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:10:56.600 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2024-04-24T23:10:56.600 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.600 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:10:56.600 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:10:56.600 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:10:56.600 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:10:56.600 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:10:56.600 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:10:56.600 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.156184+0000", 2024-04-24T23:10:56.600 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:10:56.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.600 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.601 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:10:56.601 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:10:56.601 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:10:56.601 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:10:56.601 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:10:56.601 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:10:56.601 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240774+0000", 2024-04-24T23:10:56.601 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:10:56.601 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.601 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.601 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:10:56.601 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:10:56.601 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:10:56.601 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:10:56.601 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:10:56.602 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:10:56.602 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240949+0000", 2024-04-24T23:10:56.602 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:10:56.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.602 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.602 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:10:56.602 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:10:56.602 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:10:56.602 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:10:56.602 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:10:56.602 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:10:56.602 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.171672+0000", 2024-04-24T23:10:56.602 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:10:56.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.603 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.603 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:10:56.603 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:10:56.603 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:10:56.603 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:10:56.603 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:10:56.603 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:10:56.603 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.220379+0000", 2024-04-24T23:10:56.603 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:10:56.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.603 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.603 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:10:56.603 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:10:56.603 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:10:56.604 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:10:56.604 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:10:56.604 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:10:56.604 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.204963+0000", 2024-04-24T23:10:56.604 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:10:56.604 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.604 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.604 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:10:56.604 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:10:56.604 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:10:56.604 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:10:56.604 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:10:56.604 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:10:56.604 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.239748+0000", 2024-04-24T23:10:56.604 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:10:56.605 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.605 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:10:56.605 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:10:56.605 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:10:56.605 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:10:56.605 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:10:56.605 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:10:56.605 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.154104+0000", 2024-04-24T23:10:56.605 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:10:56.605 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:56.605 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:10:56.605 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2024-04-24T23:10:56.605 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [ 2024-04-24T23:10:56.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.606 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "23.6", 2024-04-24T23:10:56.606 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:10:56.606 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:56.606 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 5, 2024-04-24T23:10:56.606 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 3 2024-04-24T23:10:56.606 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:56.606 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:56.606 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:56.606 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:10:56.606 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_primaries": [], 2024-04-24T23:10:56.606 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2024-04-24T23:10:56.606 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2024-04-24T23:10:56.606 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklist": {}, 2024-04-24T23:10:56.606 INFO:teuthology.orchestra.run.smithi032.stdout: "range_blocklist": {}, 2024-04-24T23:10:56.606 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2024-04-24T23:10:56.607 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2024-04-24T23:10:56.607 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2024-04-24T23:10:56.607 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2024-04-24T23:10:56.607 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2024-04-24T23:10:56.607 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2024-04-24T23:10:56.607 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2024-04-24T23:10:56.607 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:56.607 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:56.607 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2024-04-24T23:10:56.607 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2024-04-24T23:10:56.607 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2024-04-24T23:10:56.607 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2024-04-24T23:10:56.607 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {}, 2024-04-24T23:10:56.607 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode": { 2024-04-24T23:10:56.607 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_enabled": false, 2024-04-24T23:10:56.608 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_bucket_count": 0, 2024-04-24T23:10:56.608 INFO:teuthology.orchestra.run.smithi032.stdout: "degraded_stretch_mode": 0, 2024-04-24T23:10:56.608 INFO:teuthology.orchestra.run.smithi032.stdout: "recovering_stretch_mode": 0, 2024-04-24T23:10:56.608 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_bucket": 0 2024-04-24T23:10:56.608 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:56.608 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:10:56.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.535+0000 7f80b9e9a640 1 -- 172.21.15.32:0/312896415 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f80a006be60 msgr2=0x7f80a006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:56.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.535+0000 7f80b9e9a640 1 --2- 172.21.15.32:0/312896415 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f80a006be60 0x7f80a006e320 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f80b415a530 tx=0x7f80a8009190 comp rx=0 tx=0).stop 2024-04-24T23:10:56.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.535+0000 7f80b9e9a640 1 -- 172.21.15.32:0/312896415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80b40ba0e0 msgr2=0x7f80b41501f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:56.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.535+0000 7f80b9e9a640 1 --2- 172.21.15.32:0/312896415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80b40ba0e0 0x7f80b41501f0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f80a400e870 tx=0x7f80a400ed30 comp rx=0 tx=0).stop 2024-04-24T23:10:56.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.535+0000 7f80b9e9a640 1 -- 172.21.15.32:0/312896415 shutdown_connections 2024-04-24T23:10:56.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.535+0000 7f80b9e9a640 1 --2- 172.21.15.32:0/312896415 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f80a006be60 0x7f80a006e320 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:56.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.535+0000 7f80b9e9a640 1 --2- 172.21.15.32:0/312896415 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f80b415e9b0 0x7f80b40cd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:56.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.535+0000 7f80b9e9a640 1 --2- 172.21.15.32:0/312896415 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80b40d4210 0x7f80b40ccd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:56.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.535+0000 7f80b9e9a640 1 --2- 172.21.15.32:0/312896415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80b40ba0e0 0x7f80b41501f0 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:56.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.535+0000 7f80b9e9a640 1 -- 172.21.15.32:0/312896415 >> 172.21.15.32:0/312896415 conn(0x7f80b40d11d0 msgr2=0x7f80b4159a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:56.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.535+0000 7f80b9e9a640 1 -- 172.21.15.32:0/312896415 shutdown_connections 2024-04-24T23:10:56.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.535+0000 7f80b9e9a640 1 -- 172.21.15.32:0/312896415 wait complete. 2024-04-24T23:10:56.611 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2024-04-24T23:10:56.611 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:10:56.611 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:10:56.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:10:56.627 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=16) 2024-04-24T23:10:56.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:10:56.826 INFO:teuthology.orchestra.run.smithi032.stderr:dd: fdatasync failed for 'large_file_b': No space left on device 2024-04-24T23:10:56.826 INFO:teuthology.orchestra.run.smithi032.stderr:50+0 records in 2024-04-24T23:10:56.826 INFO:teuthology.orchestra.run.smithi032.stderr:50+0 records out 2024-04-24T23:10:56.826 INFO:teuthology.orchestra.run.smithi032.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.139922 s, 375 MB/s 2024-04-24T23:10:56.827 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-24T23:10:56.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.925+0000 7f49f6bf9640 1 Processor -- start 2024-04-24T23:10:56.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.925+0000 7f49f6bf9640 1 -- start start 2024-04-24T23:10:56.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.926+0000 7f49f6bf9640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f49f015b380 0x7f49f006c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:56.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.926+0000 7f49f6bf9640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f49f015cb00 0x7f49f006cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:56.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.926+0000 7f49f6bf9640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49f00b4520 con 0x7f49f00ce430 2024-04-24T23:10:56.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.926+0000 7f49f6bf9640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f49f00cef90 con 0x7f49f015b380 2024-04-24T23:10:56.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.926+0000 7f49f6bf9640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f49f00b4800 con 0x7f49f015cb00 2024-04-24T23:10:56.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.926+0000 7f49f5196640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f49f00ce430 0x7f49f00ce880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46090/0 (socket says 172.21.15.32:46090) 2024-04-24T23:10:56.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.926+0000 7f49f5196640 1 -- 172.21.15.32:0/2238634969 learned_addr learned my addr 172.21.15.32:0/2238634969 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:10:56.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.927+0000 7f49f4995640 1 --2- 172.21.15.32:0/2238634969 >> v2:172.21.15.121:3300/0 conn(0x7f49f015b380 0x7f49f006c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:56.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.927+0000 7f49effff640 1 --2- 172.21.15.32:0/2238634969 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f49f015cb00 0x7f49f006cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:56.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.927+0000 7f49ef7fe640 1 -- 172.21.15.32:0/2238634969 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2223876035 0 0) 0x7f49f00b4520 con 0x7f49f00ce430 2024-04-24T23:10:56.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.927+0000 7f49ef7fe640 1 -- 172.21.15.32:0/2238634969 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f49d8003520 con 0x7f49f00ce430 2024-04-24T23:10:56.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.927+0000 7f49ef7fe640 1 -- 172.21.15.32:0/2238634969 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 640042435 0 0) 0x7f49d8003520 con 0x7f49f00ce430 2024-04-24T23:10:56.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.927+0000 7f49f4995640 1 -- 172.21.15.32:0/2238634969 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f49f015cb00 msgr2=0x7f49f006cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:56.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.927+0000 7f49f4995640 1 --2- 172.21.15.32:0/2238634969 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f49f015cb00 0x7f49f006cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:56.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.927+0000 7f49f4995640 1 -- 172.21.15.32:0/2238634969 >> v1:172.21.15.32:6789/0 conn(0x7f49f00ce430 legacy=0x7f49f00ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:56.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.927+0000 7f49f4995640 1 -- 172.21.15.32:0/2238634969 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49f00b4ae0 con 0x7f49f015b380 2024-04-24T23:10:56.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.927+0000 7f49effff640 1 --2- 172.21.15.32:0/2238634969 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f49f015cb00 0x7f49f006cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:10:56.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.928+0000 7f49f4995640 1 --2- 172.21.15.32:0/2238634969 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f49f015b380 0x7f49f006c5f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f49e8009d90 tx=0x7f49e802a560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89ff8b82284432ac server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:56.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.928+0000 7f49ef7fe640 1 -- 172.21.15.32:0/2238634969 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e0001670 con 0x7f49f015b380 2024-04-24T23:10:56.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.928+0000 7f49ef7fe640 1 -- 172.21.15.32:0/2238634969 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49e8007540 con 0x7f49f015b380 2024-04-24T23:10:56.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.928+0000 7f49ef7fe640 1 -- 172.21.15.32:0/2238634969 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e80078f0 con 0x7f49f015b380 2024-04-24T23:10:56.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.928+0000 7f49f6bf9640 1 -- 172.21.15.32:0/2238634969 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f49f015b380 msgr2=0x7f49f006c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:56.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.928+0000 7f49f6bf9640 1 --2- 172.21.15.32:0/2238634969 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f49f015b380 0x7f49f006c5f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f49e8009d90 tx=0x7f49e802a560 comp rx=0 tx=0).stop 2024-04-24T23:10:56.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.928+0000 7f49f6bf9640 1 -- 172.21.15.32:0/2238634969 shutdown_connections 2024-04-24T23:10:56.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.928+0000 7f49f6bf9640 1 --2- 172.21.15.32:0/2238634969 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f49f015cb00 0x7f49f006cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:56.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.928+0000 7f49f6bf9640 1 --2- 172.21.15.32:0/2238634969 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f49f015b380 0x7f49f006c5f0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:56.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.928+0000 7f49f6bf9640 1 -- 172.21.15.32:0/2238634969 >> 172.21.15.32:0/2238634969 conn(0x7f49f006b110 msgr2=0x7f49f006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:56.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.929+0000 7f49f6bf9640 1 -- 172.21.15.32:0/2238634969 shutdown_connections 2024-04-24T23:10:56.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.929+0000 7f49f6bf9640 1 -- 172.21.15.32:0/2238634969 wait complete. 2024-04-24T23:10:56.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.930+0000 7f49f6bf9640 1 Processor -- start 2024-04-24T23:10:56.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.930+0000 7f49f6bf9640 1 -- start start 2024-04-24T23:10:56.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.931+0000 7f49f6bf9640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f49f00ce430 0x7f49f014e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:56.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.931+0000 7f49f6bf9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49f015b380 0x7f49f014e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:56.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.931+0000 7f49f6bf9640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f49f015cb00 0x7f49f01e8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:56.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.931+0000 7f49f6bf9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f49f015b760 con 0x7f49f015b380 2024-04-24T23:10:56.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.931+0000 7f49f6bf9640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f49f00b50a0 con 0x7f49f00ce430 2024-04-24T23:10:56.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.931+0000 7f49f6bf9640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f49f00b55c0 con 0x7f49f015cb00 2024-04-24T23:10:56.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.931+0000 7f49effff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49f015b380 0x7f49f014e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:56.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.931+0000 7f49effff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49f015b380 0x7f49f014e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41120/0 (socket says 172.21.15.32:41120) 2024-04-24T23:10:56.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.931+0000 7f49effff640 1 -- 172.21.15.32:0/1014170630 learned_addr learned my addr 172.21.15.32:0/1014170630 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:56.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.931+0000 7f49f4995640 1 --2- 172.21.15.32:0/1014170630 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f49f00ce430 0x7f49f014e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:56.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.931+0000 7f49f5196640 1 --2- 172.21.15.32:0/1014170630 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f49f015cb00 0x7f49f01e8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:56.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.931+0000 7f49effff640 1 -- 172.21.15.32:0/1014170630 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f49f015cb00 msgr2=0x7f49f01e8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:56.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.931+0000 7f49effff640 1 --2- 172.21.15.32:0/1014170630 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f49f015cb00 0x7f49f01e8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:56.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.931+0000 7f49effff640 1 -- 172.21.15.32:0/1014170630 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f49f00ce430 msgr2=0x7f49f014e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:56.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.931+0000 7f49effff640 1 --2- 172.21.15.32:0/1014170630 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f49f00ce430 0x7f49f014e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:56.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.931+0000 7f49effff640 1 -- 172.21.15.32:0/1014170630 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49f0157cc0 con 0x7f49f015b380 2024-04-24T23:10:56.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.931+0000 7f49effff640 1 --2- 172.21.15.32:0/1014170630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49f015b380 0x7f49f014e0f0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f49e400e960 tx=0x7f49e400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:56.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.931+0000 7f49ed7fa640 1 -- 172.21.15.32:0/1014170630 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e4017070 con 0x7f49f015b380 2024-04-24T23:10:56.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.931+0000 7f49ed7fa640 1 -- 172.21.15.32:0/1014170630 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49e400c9a0 con 0x7f49f015b380 2024-04-24T23:10:56.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.931+0000 7f49f6bf9640 1 -- 172.21.15.32:0/1014170630 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49f0157fa0 con 0x7f49f015b380 2024-04-24T23:10:56.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.931+0000 7f49ed7fa640 1 -- 172.21.15.32:0/1014170630 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e400cd70 con 0x7f49f015b380 2024-04-24T23:10:56.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.932+0000 7f49f6bf9640 1 -- 172.21.15.32:0/1014170630 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49f0158560 con 0x7f49f015b380 2024-04-24T23:10:56.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.933+0000 7f49f6bf9640 1 -- 172.21.15.32:0/1014170630 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49f0068430 con 0x7f49f015b380 2024-04-24T23:10:56.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.933+0000 7f49ed7fa640 1 -- 172.21.15.32:0/1014170630 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f49e401a070 con 0x7f49f015b380 2024-04-24T23:10:56.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.933+0000 7f49ed7fa640 1 --2- 172.21.15.32:0/1014170630 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f49dc06be30 0x7f49dc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:56.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.934+0000 7f49ed7fa640 1 -- 172.21.15.32:0/1014170630 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(189..189 src has 1..189) ==== 5738+0+0 (secure 0 0 0) 0x7f49e408b000 con 0x7f49f015b380 2024-04-24T23:10:56.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.934+0000 7f49f4995640 1 --2- 172.21.15.32:0/1014170630 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f49dc06be30 0x7f49dc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:56.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.934+0000 7f49ed7fa640 1 -- 172.21.15.32:0/1014170630 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=190}) -- 0x7f49dc0739c0 con 0x7f49f015b380 2024-04-24T23:10:56.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.934+0000 7f49f4995640 1 --2- 172.21.15.32:0/1014170630 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f49dc06be30 0x7f49dc06e2f0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f49e8034000 tx=0x7f49e802f000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:56.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:56.938+0000 7f49ed7fa640 1 -- 172.21.15.32:0/1014170630 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f49e4059340 con 0x7f49f015b380 2024-04-24T23:10:57.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.122+0000 7f49f6bf9640 1 -- 172.21.15.32:0/1014170630 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f49f006c000 con 0x7f49f015b380 2024-04-24T23:10:57.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.124+0000 7f49ed7fa640 1 -- 172.21.15.32:0/1014170630 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v189) ==== 81+0+34398 (secure 0 0 0) 0x7f49e405d640 con 0x7f49f015b380 2024-04-24T23:10:57.126 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:10:57.126 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:10:57.126 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 189, 2024-04-24T23:10:57.126 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", 2024-04-24T23:10:57.126 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2024-04-24T22:58:46.964851+0000", 2024-04-24T23:10:57.127 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2024-04-24T23:10:54.060025+0000", 2024-04-24T23:10:57.127 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2024-04-24T22:58:52.048787+0000", 2024-04-24T23:10:57.127 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2024-04-24T22:58:49.045609+0000", 2024-04-24T23:10:57.127 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-24T23:10:57.127 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2024-04-24T23:10:57.127 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2024-04-24T23:10:57.127 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2024-04-24T23:10:57.127 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2024-04-24T23:10:57.127 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2024-04-24T23:10:57.128 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2024-04-24T23:10:57.128 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:10:57.128 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 4, 2024-04-24T23:10:57.128 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.69999998807907104, 2024-04-24T23:10:57.128 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-24T23:10:57.128 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-24T23:10:57.128 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2024-04-24T23:10:57.128 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 23, 2024-04-24T23:10:57.128 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2024-04-24T23:10:57.128 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "luminous", 2024-04-24T23:10:57.128 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "luminous", 2024-04-24T23:10:57.129 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "squid", 2024-04-24T23:10:57.129 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_crimson": false, 2024-04-24T23:10:57.129 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2024-04-24T23:10:57.129 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.129 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2024-04-24T23:10:57.129 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": ".mgr", 2024-04-24T23:10:57.129 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T22:58:53.171408+0000", 2024-04-24T23:10:57.129 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:10:57.129 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:10:57.129 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:10:57.129 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:10:57.129 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:10:57.129 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:10:57.129 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:10:57.130 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:10:57.130 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:10:57.130 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:10:57.130 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:10:57.130 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:10:57.130 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2024-04-24T23:10:57.130 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2024-04-24T23:10:57.130 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2024-04-24T23:10:57.130 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2024-04-24T23:10:57.130 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2024-04-24T23:10:57.130 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:10:57.130 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:10:57.130 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:10:57.131 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:10:57.131 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:10:57.131 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:10:57.131 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:10:57.131 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.131 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "113", 2024-04-24T23:10:57.131 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:10:57.131 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:10:57.131 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:10:57.131 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:10:57.131 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:10:57.131 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:10:57.131 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:10:57.132 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:10:57.132 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:10:57.132 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:10:57.132 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:10:57.132 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:10:57.132 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:10:57.132 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:10:57.132 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:10:57.132 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:10:57.133 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:10:57.133 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:10:57.133 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:10:57.133 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:10:57.133 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:10:57.133 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:10:57.133 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:10:57.133 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:10:57.133 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:10:57.133 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:10:57.133 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.133 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:10:57.133 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:10:57.134 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:10:57.134 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:10:57.134 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:10:57.134 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:10:57.134 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:10:57.134 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:10:57.134 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:10:57.134 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:10:57.134 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:10:57.134 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:10:57.135 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_max": 32, 2024-04-24T23:10:57.135 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2024-04-24T23:10:57.135 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.135 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:10:57.135 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": {} 2024-04-24T23:10:57.135 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.135 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:10:57.135 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:10:57.135 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 8, 2024-04-24T23:10:57.135 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 8, 2024-04-24T23:10:57.135 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.25, 2024-04-24T23:10:57.135 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-04-24T23:10:57.135 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-04-24T23:10:57.135 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:10:57.136 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:10:57.136 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:10:57.136 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:57.136 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.136 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.136 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 22, 2024-04-24T23:10:57.136 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2024-04-24T23:10:57.136 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:10:34.649562+0000", 2024-04-24T23:10:57.136 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:10:57.136 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:10:57.136 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:10:57.136 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:10:57.136 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:10:57.137 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:10:57.137 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:10:57.137 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:10:57.137 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:10:57.137 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:10:57.137 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:10:57.137 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:10:57.138 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 32, 2024-04-24T23:10:57.138 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 32, 2024-04-24T23:10:57.138 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 32, 2024-04-24T23:10:57.138 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 32, 2024-04-24T23:10:57.138 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 32, 2024-04-24T23:10:57.138 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:10:57.138 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:10:57.138 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:10:57.139 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:10:57.139 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:10:57.139 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:10:57.139 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:10:57.139 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.139 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "184", 2024-04-24T23:10:57.139 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:10:57.139 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:10:57.140 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:10:57.140 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:10:57.140 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:10:57.140 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:10:57.140 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:10:57.140 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:10:57.140 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:10:57.140 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:10:57.140 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:10:57.140 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:10:57.140 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:10:57.140 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:10:57.141 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:10:57.141 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:10:57.141 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:10:57.141 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:10:57.141 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:10:57.141 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:10:57.141 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:10:57.141 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:10:57.141 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:10:57.142 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:10:57.142 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:10:57.142 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:10:57.142 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.142 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:10:57.142 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:10:57.142 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:10:57.142 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:10:57.142 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:10:57.142 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:10:57.142 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:10:57.142 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:10:57.142 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:10:57.143 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:10:57.143 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:10:57.143 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:10:57.143 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2024-04-24T23:10:57.143 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2024-04-24T23:10:57.143 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2024-04-24T23:10:57.143 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.143 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:10:57.143 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:10:57.143 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2024-04-24T23:10:57.144 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:57.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.144 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:10:57.144 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:10:57.144 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.5, 2024-04-24T23:10:57.144 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.5, 2024-04-24T23:10:57.144 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:10:57.144 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.5, 2024-04-24T23:10:57.144 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.5, 2024-04-24T23:10:57.145 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:10:57.145 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:10:57.145 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:10:57.145 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:57.145 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.145 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.145 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 23, 2024-04-24T23:10:57.145 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2024-04-24T23:10:57.145 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:10:35.339604+0000", 2024-04-24T23:10:57.145 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1027, 2024-04-24T23:10:57.146 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,full,full_quota", 2024-04-24T23:10:57.146 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:10:57.146 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:10:57.146 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:10:57.146 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:10:57.146 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:10:57.146 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:10:57.146 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:10:57.146 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:10:57.146 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:10:57.146 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:10:57.146 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 64, 2024-04-24T23:10:57.146 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 64, 2024-04-24T23:10:57.147 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 64, 2024-04-24T23:10:57.147 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 64, 2024-04-24T23:10:57.147 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 64, 2024-04-24T23:10:57.147 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:10:57.147 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:10:57.147 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:10:57.147 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:10:57.147 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:10:57.147 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:10:57.147 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:10:57.147 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.147 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "187", 2024-04-24T23:10:57.147 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:10:57.147 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:10:57.148 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:10:57.148 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:10:57.148 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:10:57.148 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:10:57.148 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:10:57.148 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:10:57.148 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:10:57.148 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 33554432, 2024-04-24T23:10:57.148 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:10:57.148 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:10:57.148 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:10:57.148 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:10:57.149 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:10:57.149 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:10:57.149 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:10:57.149 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:10:57.149 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:10:57.149 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:10:57.149 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:10:57.149 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:10:57.149 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:10:57.149 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:10:57.149 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:10:57.149 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:10:57.149 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.149 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:10:57.150 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:10:57.150 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:10:57.150 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:10:57.150 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:10:57.150 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:10:57.150 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:10:57.150 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:10:57.150 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:10:57.150 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:10:57.150 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:10:57.150 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:10:57.150 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 64, 2024-04-24T23:10:57.150 INFO:teuthology.orchestra.run.smithi032.stdout: "target_size_ratio": 0.89999997615814209 2024-04-24T23:10:57.150 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.151 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:10:57.151 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:10:57.151 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2024-04-24T23:10:57.151 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:57.151 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.151 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:10:57.151 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:10:57.151 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.3799999952316284, 2024-04-24T23:10:57.151 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.3799999952316284, 2024-04-24T23:10:57.151 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:10:57.151 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.3799999952316284, 2024-04-24T23:10:57.151 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.3799999952316284, 2024-04-24T23:10:57.151 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:10:57.151 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:10:57.151 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:10:57.152 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:57.152 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:57.152 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:10:57.152 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-04-24T23:10:57.152 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.152 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:10:57.152 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:10:57.152 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:10:57.152 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:10:57.152 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:10:57.152 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:10:57.152 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:10:57.152 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:10:57.153 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:10:57.153 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:10:57.153 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:10:57.153 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:10:57.153 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:10:57.153 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:57.153 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.153 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:57.153 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2024-04-24T23:10:57.153 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:10:57.153 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.153 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.154 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:57.154 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2024-04-24T23:10:57.154 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:10:57.154 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:57.154 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:57.154 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.154 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:10:57.154 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:57.154 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.154 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:57.154 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2024-04-24T23:10:57.154 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:10:57.155 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.155 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.155 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:57.155 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2024-04-24T23:10:57.155 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:10:57.155 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:57.155 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:57.155 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.155 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:10:57.155 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:57.155 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.156 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:57.156 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2024-04-24T23:10:57.156 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:10:57.156 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.156 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.156 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:57.156 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2024-04-24T23:10:57.156 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:10:57.156 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:57.156 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:57.157 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.157 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:10:57.157 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:57.157 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.157 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:57.157 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2024-04-24T23:10:57.157 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:10:57.157 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.157 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.157 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:57.157 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2024-04-24T23:10:57.157 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:10:57.157 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:57.157 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:57.157 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.158 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/1479395426", 2024-04-24T23:10:57.158 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/1479395426", 2024-04-24T23:10:57.158 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/1479395426", 2024-04-24T23:10:57.158 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/1479395426", 2024-04-24T23:10:57.158 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:10:57.158 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:10:57.158 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:10:57.158 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:57.158 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.158 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.158 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:10:57.158 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:10:57.158 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:10:57.159 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:10:57.159 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:10:57.159 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:10:57.159 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:10:57.159 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:10:57.159 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:10:57.159 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:10:57.159 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:10:57.159 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:10:57.159 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:10:57.160 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:57.160 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.160 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:57.160 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2024-04-24T23:10:57.160 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:10:57.160 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.160 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.160 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:57.160 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2024-04-24T23:10:57.160 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:10:57.160 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:57.160 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:57.160 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.161 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:10:57.161 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:57.161 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.161 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:57.161 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2024-04-24T23:10:57.161 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:10:57.162 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.162 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.162 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:57.162 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2024-04-24T23:10:57.162 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:10:57.163 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:57.163 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:57.163 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.163 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:10:57.163 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:57.163 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.164 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:57.164 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2024-04-24T23:10:57.164 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:10:57.164 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.164 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.164 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:57.165 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2024-04-24T23:10:57.165 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:10:57.165 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:57.165 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:57.165 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.166 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:10:57.166 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:57.166 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.166 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:57.166 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2024-04-24T23:10:57.166 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:10:57.167 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.167 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.167 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:57.167 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2024-04-24T23:10:57.167 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:10:57.167 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:57.168 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:57.168 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.168 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/1340633718", 2024-04-24T23:10:57.168 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/1340633718", 2024-04-24T23:10:57.168 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/1340633718", 2024-04-24T23:10:57.168 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/1340633718", 2024-04-24T23:10:57.169 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:10:57.169 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:10:57.169 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:10:57.169 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:57.169 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.170 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.170 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:10:57.170 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:10:57.170 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:10:57.170 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:10:57.170 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:10:57.171 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:10:57.171 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:10:57.171 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:10:57.171 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:10:57.171 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:10:57.171 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:10:57.172 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:10:57.172 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:10:57.172 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:57.172 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.172 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:57.172 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2024-04-24T23:10:57.173 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:10:57.173 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.173 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.173 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:57.173 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2024-04-24T23:10:57.173 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:10:57.174 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:57.174 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:57.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.174 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:10:57.174 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:57.175 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.175 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:57.175 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2024-04-24T23:10:57.175 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:10:57.175 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.175 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.176 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:57.176 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2024-04-24T23:10:57.176 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:10:57.176 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:57.176 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:57.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.177 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:10:57.177 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:57.177 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.177 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:57.177 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2024-04-24T23:10:57.178 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:10:57.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.178 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:57.178 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2024-04-24T23:10:57.179 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:10:57.179 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:57.179 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:57.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.179 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:10:57.179 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:57.179 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.180 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:57.180 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2024-04-24T23:10:57.180 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:10:57.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.180 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.180 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:57.181 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2024-04-24T23:10:57.181 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:10:57.181 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:57.181 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:57.181 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.181 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/3405899334", 2024-04-24T23:10:57.182 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6820/3405899334", 2024-04-24T23:10:57.182 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6828/3405899334", 2024-04-24T23:10:57.182 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6824/3405899334", 2024-04-24T23:10:57.182 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:10:57.182 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:10:57.183 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:10:57.183 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:57.183 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.183 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.183 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:10:57.183 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:10:57.184 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:10:57.184 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:10:57.184 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:10:57.184 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:10:57.184 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:10:57.185 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:10:57.185 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:10:57.185 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 188, 2024-04-24T23:10:57.185 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:10:57.185 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:10:57.185 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:10:57.186 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:57.186 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.186 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:57.186 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2024-04-24T23:10:57.186 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:10:57.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.187 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.187 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:57.187 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2024-04-24T23:10:57.187 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:10:57.187 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:57.188 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:57.188 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.188 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:10:57.188 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:57.188 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.188 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:57.189 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2024-04-24T23:10:57.189 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:10:57.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.189 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.189 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:57.189 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2024-04-24T23:10:57.190 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:10:57.190 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:57.190 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:57.190 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.190 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:10:57.191 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:57.191 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.191 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:57.191 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2024-04-24T23:10:57.191 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:10:57.191 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.192 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:57.192 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2024-04-24T23:10:57.192 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:10:57.192 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:57.192 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:57.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.193 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:10:57.193 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:57.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.193 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:57.194 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2024-04-24T23:10:57.194 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:10:57.194 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.194 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.194 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:57.194 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2024-04-24T23:10:57.195 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:10:57.195 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:57.195 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:57.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.195 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6821/3381057190", 2024-04-24T23:10:57.196 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6825/3381057190", 2024-04-24T23:10:57.196 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/3381057190", 2024-04-24T23:10:57.196 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/3381057190", 2024-04-24T23:10:57.196 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:10:57.196 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:10:57.196 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:10:57.197 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:57.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.197 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.197 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:10:57.197 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:10:57.198 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:10:57.198 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:10:57.198 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:10:57.198 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:10:57.198 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:10:57.198 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:10:57.199 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:10:57.199 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:10:57.199 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:10:57.199 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:10:57.199 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:10:57.199 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:57.200 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.200 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:57.200 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6800", 2024-04-24T23:10:57.200 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:10:57.200 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.201 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:57.201 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6801", 2024-04-24T23:10:57.201 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:10:57.201 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:57.201 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:57.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.202 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:10:57.202 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:57.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.202 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:57.202 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6802", 2024-04-24T23:10:57.203 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:10:57.203 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.203 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.203 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:57.203 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6803", 2024-04-24T23:10:57.203 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:10:57.204 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:57.204 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:57.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.204 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:10:57.204 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:57.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.205 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:57.205 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6806", 2024-04-24T23:10:57.205 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:10:57.205 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.206 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:57.206 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6807", 2024-04-24T23:10:57.206 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:10:57.206 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:57.206 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:57.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.207 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:10:57.207 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:57.207 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.207 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:57.207 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6804", 2024-04-24T23:10:57.207 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:10:57.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.207 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.208 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:57.208 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6805", 2024-04-24T23:10:57.208 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:10:57.208 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:57.208 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:57.208 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.208 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6801/4010553979", 2024-04-24T23:10:57.208 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6803/4010553979", 2024-04-24T23:10:57.208 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6807/4010553979", 2024-04-24T23:10:57.208 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6805/4010553979", 2024-04-24T23:10:57.208 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:10:57.208 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:10:57.208 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:10:57.208 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:57.208 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.209 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.209 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:10:57.209 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:10:57.209 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:10:57.209 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:10:57.209 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:10:57.209 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:10:57.209 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:10:57.209 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:10:57.209 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:10:57.209 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:10:57.209 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:10:57.209 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:10:57.209 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:10:57.209 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:57.210 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.210 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:57.210 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6808", 2024-04-24T23:10:57.210 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:10:57.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.210 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.210 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:57.210 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6809", 2024-04-24T23:10:57.210 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:10:57.210 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:57.210 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:57.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.210 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:10:57.211 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:57.211 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.211 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:57.211 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6810", 2024-04-24T23:10:57.211 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:10:57.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.211 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.211 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:57.211 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6811", 2024-04-24T23:10:57.211 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:10:57.211 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:57.211 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:57.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.211 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:10:57.211 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:57.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.212 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:57.212 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6814", 2024-04-24T23:10:57.212 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:10:57.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.212 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:57.212 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6815", 2024-04-24T23:10:57.212 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:10:57.212 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:57.212 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:57.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.212 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:10:57.212 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:57.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.213 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:57.213 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6812", 2024-04-24T23:10:57.213 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:10:57.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.213 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.213 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:57.213 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6813", 2024-04-24T23:10:57.213 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:10:57.213 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:57.213 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:57.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.213 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6809/3794084091", 2024-04-24T23:10:57.213 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6811/3794084091", 2024-04-24T23:10:57.213 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6815/3794084091", 2024-04-24T23:10:57.213 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6813/3794084091", 2024-04-24T23:10:57.214 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:10:57.214 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:10:57.214 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:10:57.214 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:57.214 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.214 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.214 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:10:57.214 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:10:57.214 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:10:57.214 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:10:57.214 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:10:57.214 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:10:57.214 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:10:57.214 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:10:57.214 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:10:57.215 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:10:57.215 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:10:57.215 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:10:57.215 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:10:57.215 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:57.215 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.215 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:57.215 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6823", 2024-04-24T23:10:57.215 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:10:57.215 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.215 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.215 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:57.215 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6825", 2024-04-24T23:10:57.215 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:10:57.216 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:57.216 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:57.216 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.216 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:10:57.216 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:57.216 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.216 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:57.216 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6826", 2024-04-24T23:10:57.216 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:10:57.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.217 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.217 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:57.217 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6827", 2024-04-24T23:10:57.217 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:10:57.217 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:57.217 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:57.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.217 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:10:57.217 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:57.217 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.217 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:57.217 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6830", 2024-04-24T23:10:57.217 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:10:57.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.218 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:57.218 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6831", 2024-04-24T23:10:57.218 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:10:57.218 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:57.218 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:57.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.218 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:10:57.218 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:57.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.218 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:57.218 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6828", 2024-04-24T23:10:57.218 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:10:57.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.219 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.219 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:57.219 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6829", 2024-04-24T23:10:57.219 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:10:57.219 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:57.219 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:57.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.219 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6825/1348640258", 2024-04-24T23:10:57.219 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6827/1348640258", 2024-04-24T23:10:57.219 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6831/1348640258", 2024-04-24T23:10:57.219 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6829/1348640258", 2024-04-24T23:10:57.219 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:10:57.219 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:10:57.219 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:10:57.220 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:57.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.220 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.220 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:10:57.220 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:10:57.220 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:10:57.220 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:10:57.220 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:10:57.220 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:10:57.220 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:10:57.220 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:10:57.220 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:10:57.220 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:10:57.220 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:10:57.220 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:10:57.221 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:10:57.221 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:57.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.221 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:57.221 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6816", 2024-04-24T23:10:57.221 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:10:57.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.221 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:57.221 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6817", 2024-04-24T23:10:57.221 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:10:57.221 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:57.221 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:57.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.221 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:10:57.222 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:57.222 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.222 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:57.222 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6818", 2024-04-24T23:10:57.222 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:10:57.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.222 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.222 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:57.222 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6819", 2024-04-24T23:10:57.222 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:10:57.222 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:57.222 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:57.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.222 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:10:57.222 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:57.223 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.223 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:57.223 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6822", 2024-04-24T23:10:57.223 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:10:57.223 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.223 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.223 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:57.223 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6824", 2024-04-24T23:10:57.223 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:10:57.223 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:57.223 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:57.223 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.223 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:10:57.224 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:57.224 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.224 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:57.224 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6820", 2024-04-24T23:10:57.224 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:10:57.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.224 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.224 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:57.224 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6821", 2024-04-24T23:10:57.224 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:10:57.224 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:57.224 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:57.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.224 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6817/3250005122", 2024-04-24T23:10:57.225 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6819/3250005122", 2024-04-24T23:10:57.225 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6824/3250005122", 2024-04-24T23:10:57.225 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6821/3250005122", 2024-04-24T23:10:57.225 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:10:57.225 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:10:57.225 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:10:57.225 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:57.225 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:57.225 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:10:57.225 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2024-04-24T23:10:57.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.225 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:10:57.225 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:10:57.225 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:10:57.226 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:10:57.226 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:10:57.226 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:10:57.226 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.156184+0000", 2024-04-24T23:10:57.226 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:10:57.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.226 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:10:57.226 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:10:57.226 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:10:57.226 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:10:57.226 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:10:57.226 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:10:57.226 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240774+0000", 2024-04-24T23:10:57.227 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:10:57.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.227 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.227 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:10:57.227 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:10:57.227 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:10:57.227 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:10:57.227 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:10:57.227 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:10:57.227 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240949+0000", 2024-04-24T23:10:57.227 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:10:57.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.227 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.227 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:10:57.228 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:10:57.228 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:10:57.228 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:10:57.228 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:10:57.228 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:10:57.228 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.171672+0000", 2024-04-24T23:10:57.228 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:10:57.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.228 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:10:57.228 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:10:57.228 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:10:57.228 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:10:57.228 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:10:57.229 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:10:57.229 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.220379+0000", 2024-04-24T23:10:57.229 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:10:57.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.229 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.229 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:10:57.229 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:10:57.229 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:10:57.229 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:10:57.229 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:10:57.229 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:10:57.229 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.204963+0000", 2024-04-24T23:10:57.229 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:10:57.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.230 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:10:57.230 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:10:57.230 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:10:57.230 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:10:57.230 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:10:57.230 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:10:57.230 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.239748+0000", 2024-04-24T23:10:57.230 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:10:57.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.230 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:10:57.230 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:10:57.231 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:10:57.231 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:10:57.231 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:10:57.231 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:10:57.231 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.154104+0000", 2024-04-24T23:10:57.231 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:10:57.231 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:57.231 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:10:57.231 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2024-04-24T23:10:57.231 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [ 2024-04-24T23:10:57.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.231 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "23.6", 2024-04-24T23:10:57.231 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:10:57.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:57.231 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 5, 2024-04-24T23:10:57.232 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 3 2024-04-24T23:10:57.232 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:57.232 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:57.232 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:57.232 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:10:57.232 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_primaries": [], 2024-04-24T23:10:57.232 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2024-04-24T23:10:57.232 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2024-04-24T23:10:57.232 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklist": {}, 2024-04-24T23:10:57.232 INFO:teuthology.orchestra.run.smithi032.stdout: "range_blocklist": {}, 2024-04-24T23:10:57.232 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2024-04-24T23:10:57.232 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2024-04-24T23:10:57.232 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2024-04-24T23:10:57.232 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2024-04-24T23:10:57.232 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2024-04-24T23:10:57.232 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2024-04-24T23:10:57.233 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2024-04-24T23:10:57.233 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:57.233 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:57.233 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2024-04-24T23:10:57.233 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2024-04-24T23:10:57.233 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2024-04-24T23:10:57.233 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2024-04-24T23:10:57.233 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {}, 2024-04-24T23:10:57.233 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode": { 2024-04-24T23:10:57.233 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_enabled": false, 2024-04-24T23:10:57.233 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_bucket_count": 0, 2024-04-24T23:10:57.233 INFO:teuthology.orchestra.run.smithi032.stdout: "degraded_stretch_mode": 0, 2024-04-24T23:10:57.233 INFO:teuthology.orchestra.run.smithi032.stdout: "recovering_stretch_mode": 0, 2024-04-24T23:10:57.233 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_bucket": 0 2024-04-24T23:10:57.233 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:57.233 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:10:57.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.126+0000 7f49f6bf9640 1 -- 172.21.15.32:0/1014170630 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f49dc06be30 msgr2=0x7f49dc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:57.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.126+0000 7f49f6bf9640 1 --2- 172.21.15.32:0/1014170630 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f49dc06be30 0x7f49dc06e2f0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f49e8034000 tx=0x7f49e802f000 comp rx=0 tx=0).stop 2024-04-24T23:10:57.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.126+0000 7f49f6bf9640 1 -- 172.21.15.32:0/1014170630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49f015b380 msgr2=0x7f49f014e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:57.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.126+0000 7f49f6bf9640 1 --2- 172.21.15.32:0/1014170630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49f015b380 0x7f49f014e0f0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f49e400e960 tx=0x7f49e400ee20 comp rx=0 tx=0).stop 2024-04-24T23:10:57.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.126+0000 7f49f6bf9640 1 -- 172.21.15.32:0/1014170630 shutdown_connections 2024-04-24T23:10:57.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.126+0000 7f49f6bf9640 1 --2- 172.21.15.32:0/1014170630 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f49dc06be30 0x7f49dc06e2f0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:57.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.126+0000 7f49f6bf9640 1 --2- 172.21.15.32:0/1014170630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49f015b380 0x7f49f014e0f0 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:57.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.126+0000 7f49f6bf9640 1 --2- 172.21.15.32:0/1014170630 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f49f015cb00 0x7f49f01e8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:57.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.126+0000 7f49f6bf9640 1 --2- 172.21.15.32:0/1014170630 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f49f00ce430 0x7f49f014e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:57.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.126+0000 7f49f6bf9640 1 -- 172.21.15.32:0/1014170630 >> 172.21.15.32:0/1014170630 conn(0x7f49f006b110 msgr2=0x7f49f006b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:57.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.126+0000 7f49f6bf9640 1 -- 172.21.15.32:0/1014170630 shutdown_connections 2024-04-24T23:10:57.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.126+0000 7f49f6bf9640 1 -- 172.21.15.32:0/1014170630 wait complete. 2024-04-24T23:10:57.237 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:10:57.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.336+0000 7f6d6425d640 1 Processor -- start 2024-04-24T23:10:57.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.336+0000 7f6d6425d640 1 -- start start 2024-04-24T23:10:57.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.337+0000 7f6d6425d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6d5c0b5560 0x7f6d5c15e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:57.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.337+0000 7f6d6425d640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6d5c0cd3c0 0x7f6d5c15ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:57.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.337+0000 7f6d6425d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d5c0cd910 con 0x7f6d5c0b5940 2024-04-24T23:10:57.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.337+0000 7f6d6425d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f6d5c0b6470 con 0x7f6d5c0b5560 2024-04-24T23:10:57.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.337+0000 7f6d6425d640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f6d5c0cdbf0 con 0x7f6d5c0cd3c0 2024-04-24T23:10:57.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.337+0000 7f6d61ff9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6d5c0b5560 0x7f6d5c15e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:57.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.337+0000 7f6d627fa640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6d5c0b5940 0x7f6d5c0b5d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46092/0 (socket says 172.21.15.32:46092) 2024-04-24T23:10:57.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.337+0000 7f6d627fa640 1 -- 172.21.15.32:0/404573271 learned_addr learned my addr 172.21.15.32:0/404573271 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:10:57.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.337+0000 7f6d61ff9640 1 -- 172.21.15.32:0/404573271 >> v2:172.21.15.121:3300/0 conn(0x7f6d5c0cd3c0 msgr2=0x7f6d5c15ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:57.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.337+0000 7f6d61ff9640 1 --2- 172.21.15.32:0/404573271 >> v2:172.21.15.121:3300/0 conn(0x7f6d5c0cd3c0 0x7f6d5c15ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:57.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.337+0000 7f6d61ff9640 1 -- 172.21.15.32:0/404573271 >> v1:172.21.15.32:6789/0 conn(0x7f6d5c0b5940 legacy=0x7f6d5c0b5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:57.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.337+0000 7f6d60ff7640 1 -- 172.21.15.32:0/404573271 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2591400686 0 0) 0x7f6d5c0cd910 con 0x7f6d5c0b5940 2024-04-24T23:10:57.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.337+0000 7f6d61ff9640 1 -- 172.21.15.32:0/404573271 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d5c0cded0 con 0x7f6d5c0b5560 2024-04-24T23:10:57.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.338+0000 7f6d61ff9640 1 --2- 172.21.15.32:0/404573271 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d5c0b5560 0x7f6d5c15e6c0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c0ccb60 tx=0x7f6d5802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ebbdb38837caf9a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:57.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.338+0000 7f6d60ff7640 1 -- 172.21.15.32:0/404573271 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d58037070 con 0x7f6d5c0b5560 2024-04-24T23:10:57.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.338+0000 7f6d60ff7640 1 -- 172.21.15.32:0/404573271 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d58032440 con 0x7f6d5c0b5560 2024-04-24T23:10:57.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.338+0000 7f6d60ff7640 1 -- 172.21.15.32:0/404573271 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d58032810 con 0x7f6d5c0b5560 2024-04-24T23:10:57.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.339+0000 7f6d6425d640 1 -- 172.21.15.32:0/404573271 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d5c0b5560 msgr2=0x7f6d5c15e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:57.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.339+0000 7f6d6425d640 1 --2- 172.21.15.32:0/404573271 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d5c0b5560 0x7f6d5c15e6c0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c0ccb60 tx=0x7f6d5802a1a0 comp rx=0 tx=0).stop 2024-04-24T23:10:57.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.339+0000 7f6d6425d640 1 -- 172.21.15.32:0/404573271 shutdown_connections 2024-04-24T23:10:57.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.339+0000 7f6d6425d640 1 --2- 172.21.15.32:0/404573271 >> v2:172.21.15.121:3300/0 conn(0x7f6d5c0cd3c0 0x7f6d5c15ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:57.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.339+0000 7f6d6425d640 1 --2- 172.21.15.32:0/404573271 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d5c0b5560 0x7f6d5c15e6c0 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:57.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.339+0000 7f6d6425d640 1 -- 172.21.15.32:0/404573271 >> 172.21.15.32:0/404573271 conn(0x7f6d5c06b110 msgr2=0x7f6d5c06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:57.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.339+0000 7f6d6425d640 1 -- 172.21.15.32:0/404573271 shutdown_connections 2024-04-24T23:10:57.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.339+0000 7f6d6425d640 1 -- 172.21.15.32:0/404573271 wait complete. 2024-04-24T23:10:57.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.340+0000 7f6d6425d640 1 Processor -- start 2024-04-24T23:10:57.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.340+0000 7f6d6425d640 1 -- start start 2024-04-24T23:10:57.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.341+0000 7f6d6425d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6d5c0b5560 0x7f6d5c0b07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:57.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.341+0000 7f6d6425d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d5c0b5940 0x7f6d5c1575d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:57.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.341+0000 7f6d6425d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d5c0cd3c0 0x7f6d5c148690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:57.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.341+0000 7f6d6425d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5c06c5f0 con 0x7f6d5c0cd3c0 2024-04-24T23:10:57.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.341+0000 7f6d6425d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5c0ce490 con 0x7f6d5c0b5940 2024-04-24T23:10:57.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.341+0000 7f6d6425d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f6d5c06c770 con 0x7f6d5c0b5560 2024-04-24T23:10:57.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.341+0000 7f6d627fa640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d5c0cd3c0 0x7f6d5c148690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:57.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.341+0000 7f6d627fa640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d5c0cd3c0 0x7f6d5c148690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41136/0 (socket says 172.21.15.32:41136) 2024-04-24T23:10:57.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.341+0000 7f6d627fa640 1 -- 172.21.15.32:0/64098459 learned_addr learned my addr 172.21.15.32:0/64098459 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:57.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.341+0000 7f6d61ff9640 1 --2- 172.21.15.32:0/64098459 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6d5c0b5560 0x7f6d5c0b07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:57.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.341+0000 7f6d617f8640 1 --2- 172.21.15.32:0/64098459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d5c0b5940 0x7f6d5c1575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:57.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.341+0000 7f6d627fa640 1 -- 172.21.15.32:0/64098459 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6d5c0b5560 msgr2=0x7f6d5c0b07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:57.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.341+0000 7f6d627fa640 1 --2- 172.21.15.32:0/64098459 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6d5c0b5560 0x7f6d5c0b07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:57.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.341+0000 7f6d627fa640 1 -- 172.21.15.32:0/64098459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d5c0b5940 msgr2=0x7f6d5c1575d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:57.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.341+0000 7f6d627fa640 1 --2- 172.21.15.32:0/64098459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d5c0b5940 0x7f6d5c1575d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:57.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.341+0000 7f6d627fa640 1 -- 172.21.15.32:0/64098459 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d5c072000 con 0x7f6d5c0cd3c0 2024-04-24T23:10:57.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.341+0000 7f6d627fa640 1 --2- 172.21.15.32:0/64098459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d5c0cd3c0 0x7f6d5c148690 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f6d5400c7a0 tx=0x7f6d5400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:57.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.342+0000 7f6d527fc640 1 -- 172.21.15.32:0/64098459 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d54018070 con 0x7f6d5c0cd3c0 2024-04-24T23:10:57.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.342+0000 7f6d527fc640 1 -- 172.21.15.32:0/64098459 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d5400a640 con 0x7f6d5c0cd3c0 2024-04-24T23:10:57.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.342+0000 7f6d6425d640 1 -- 172.21.15.32:0/64098459 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d5c156af0 con 0x7f6d5c0cd3c0 2024-04-24T23:10:57.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.342+0000 7f6d527fc640 1 -- 172.21.15.32:0/64098459 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5400aab0 con 0x7f6d5c0cd3c0 2024-04-24T23:10:57.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.342+0000 7f6d6425d640 1 -- 172.21.15.32:0/64098459 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d5c1570b0 con 0x7f6d5c0cd3c0 2024-04-24T23:10:57.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.343+0000 7f6d6425d640 1 -- 172.21.15.32:0/64098459 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d5c1572b0 con 0x7f6d5c0cd3c0 2024-04-24T23:10:57.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.343+0000 7f6d527fc640 1 -- 172.21.15.32:0/64098459 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6d5401b070 con 0x7f6d5c0cd3c0 2024-04-24T23:10:57.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.344+0000 7f6d527fc640 1 --2- 172.21.15.32:0/64098459 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6d4006be30 0x7f6d4006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:57.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.344+0000 7f6d527fc640 1 -- 172.21.15.32:0/64098459 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(189..189 src has 1..189) ==== 5738+0+0 (secure 0 0 0) 0x7f6d5408a4a0 con 0x7f6d5c0cd3c0 2024-04-24T23:10:57.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.344+0000 7f6d527fc640 1 -- 172.21.15.32:0/64098459 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=190}) -- 0x7f6d40073a20 con 0x7f6d5c0cd3c0 2024-04-24T23:10:57.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.344+0000 7f6d61ff9640 1 --2- 172.21.15.32:0/64098459 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6d4006be30 0x7f6d4006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:57.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.345+0000 7f6d61ff9640 1 --2- 172.21.15.32:0/64098459 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6d4006be30 0x7f6d4006e2f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f6d58034000 tx=0x7f6d5802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:57.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.348+0000 7f6d527fc640 1 -- 172.21.15.32:0/64098459 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6d5c1572b0 con 0x7f6d5c0cd3c0 2024-04-24T23:10:57.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.568+0000 7f6d6425d640 1 -- 172.21.15.32:0/64098459 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6d5c068430 con 0x7f6d5c0cd3c0 2024-04-24T23:10:57.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.568+0000 7f6d527fc640 1 -- 172.21.15.32:0/64098459 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) ==== 94+0+5185 (secure 0 0 0) 0x7f6d54058740 con 0x7f6d5c0cd3c0 2024-04-24T23:10:57.570 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:10:57.570 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":80,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6941,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/416518156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":416518156},{"type":"v1","addr":"172.21.15.121:6837","nonce":416518156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7762,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1506667233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1506667233},{"type":"v1","addr":"172.21.15.121:6835","nonce":1506667233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":9063,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/2215079101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":2215079101},{"type":"v1","addr":"172.21.15.32:6834","nonce":2215079101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:10:36.353761+0000","modified":"2024-04-24T23:10:38.436407+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9066},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9066":{"gid":9066,"name":"a","rank":0,"incarnation":78,"state":"up:active","state_seq":4,"addr":"172.21.15.32:6835/901353412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":901353412},{"type":"v1","addr":"172.21.15.32:6835","nonce":901353412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9066,"qdb_cluster":[9066]},"id":11}]} 2024-04-24T23:10:57.570 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 80 2024-04-24T23:10:57.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.570+0000 7f6d6425d640 1 -- 172.21.15.32:0/64098459 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6d4006be30 msgr2=0x7f6d4006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:57.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.570+0000 7f6d6425d640 1 --2- 172.21.15.32:0/64098459 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6d4006be30 0x7f6d4006e2f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f6d58034000 tx=0x7f6d5802e000 comp rx=0 tx=0).stop 2024-04-24T23:10:57.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.570+0000 7f6d6425d640 1 -- 172.21.15.32:0/64098459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d5c0cd3c0 msgr2=0x7f6d5c148690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:57.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.570+0000 7f6d6425d640 1 --2- 172.21.15.32:0/64098459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d5c0cd3c0 0x7f6d5c148690 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f6d5400c7a0 tx=0x7f6d5400cc60 comp rx=0 tx=0).stop 2024-04-24T23:10:57.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.570+0000 7f6d6425d640 1 -- 172.21.15.32:0/64098459 shutdown_connections 2024-04-24T23:10:57.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.570+0000 7f6d6425d640 1 --2- 172.21.15.32:0/64098459 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6d4006be30 0x7f6d4006e2f0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:57.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.570+0000 7f6d6425d640 1 --2- 172.21.15.32:0/64098459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d5c0cd3c0 0x7f6d5c148690 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:57.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.570+0000 7f6d6425d640 1 --2- 172.21.15.32:0/64098459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d5c0b5940 0x7f6d5c1575d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:57.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.570+0000 7f6d6425d640 1 --2- 172.21.15.32:0/64098459 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6d5c0b5560 0x7f6d5c0b07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:57.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.570+0000 7f6d6425d640 1 -- 172.21.15.32:0/64098459 >> 172.21.15.32:0/64098459 conn(0x7f6d5c06b110 msgr2=0x7f6d5c14f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:57.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.570+0000 7f6d6425d640 1 -- 172.21.15.32:0/64098459 shutdown_connections 2024-04-24T23:10:57.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.570+0000 7f6d6425d640 1 -- 172.21.15.32:0/64098459 wait complete. 2024-04-24T23:10:57.583 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-04-24T23:10:57.701 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","whoami":0,"id":9066,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":20.69845759,"mdsmap_epoch":80,"osdmap_epoch":189,"osdmap_epoch_barrier":189,"uptime":31.726975005} 2024-04-24T23:10:57.702 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", "fs_name": "cephfs", "id": 9066, "mdsmap_epoch": 80, "osdmap_epoch": 189, "osdmap_epoch_barrier": 189, "rank_uptime": 20.69845759, "state": "up:active", "uptime": 31.726975005, "want_state": "up:active", "whoami": 0 } 2024-04-24T23:10:57.702 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-24T23:10:57.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:10:57.702 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2024-04-24T23:10:57.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:10:57.719 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2024-04-24T23:10:57.781 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-24T23:10:57.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.920+0000 7fa2f7f61640 1 Processor -- start 2024-04-24T23:10:57.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.920+0000 7fa2f7f61640 1 -- start start 2024-04-24T23:10:57.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.921+0000 7fa2f7f61640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2f015b380 0x7fa2f006c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:57.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.921+0000 7fa2f7f61640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa2f00ceb40 0x7fa2f006cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:57.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.921+0000 7fa2f7f61640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2f00b4520 con 0x7fa2f015cb00 2024-04-24T23:10:57.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.921+0000 7fa2f7f61640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fa2f00cef90 con 0x7fa2f00ceb40 2024-04-24T23:10:57.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.921+0000 7fa2f7f61640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa2f00b4800 con 0x7fa2f015b380 2024-04-24T23:10:57.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.921+0000 7fa2f54fc640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa2f015cb00 0x7fa2f00ce430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46104/0 (socket says 172.21.15.32:46104) 2024-04-24T23:10:57.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.921+0000 7fa2f54fc640 1 -- 172.21.15.32:0/177170753 learned_addr learned my addr 172.21.15.32:0/177170753 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:10:57.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.921+0000 7fa2f5cfd640 1 --2- 172.21.15.32:0/177170753 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2f015b380 0x7fa2f006c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:57.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.921+0000 7fa2f64fe640 1 --2- 172.21.15.32:0/177170753 >> v2:172.21.15.121:3300/0 conn(0x7fa2f00ceb40 0x7fa2f006cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:57.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.921+0000 7fa2f4cfb640 1 -- 172.21.15.32:0/177170753 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 443070441 0 0) 0x7fa2f00b4520 con 0x7fa2f015cb00 2024-04-24T23:10:57.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.921+0000 7fa2f4cfb640 1 -- 172.21.15.32:0/177170753 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa2dc003520 con 0x7fa2f015cb00 2024-04-24T23:10:57.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.921+0000 7fa2f5cfd640 1 -- 172.21.15.32:0/177170753 >> v2:172.21.15.121:3300/0 conn(0x7fa2f00ceb40 msgr2=0x7fa2f006cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:10:57.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.921+0000 7fa2f5cfd640 1 --2- 172.21.15.32:0/177170753 >> v2:172.21.15.121:3300/0 conn(0x7fa2f00ceb40 0x7fa2f006cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:57.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.921+0000 7fa2f5cfd640 1 -- 172.21.15.32:0/177170753 >> v1:172.21.15.32:6789/0 conn(0x7fa2f015cb00 legacy=0x7fa2f00ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:57.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.921+0000 7fa2f5cfd640 1 -- 172.21.15.32:0/177170753 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2f00b4ae0 con 0x7fa2f015b380 2024-04-24T23:10:57.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.922+0000 7fa2f5cfd640 1 --2- 172.21.15.32:0/177170753 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2f015b380 0x7fa2f006c5f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec0097a0 tx=0x7fa2ec02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed9599c83af4d9f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:57.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.922+0000 7fa2f4cfb640 1 -- 172.21.15.32:0/177170753 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec037070 con 0x7fa2f015b380 2024-04-24T23:10:57.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.922+0000 7fa2f4cfb640 1 -- 172.21.15.32:0/177170753 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2ec02ae50 con 0x7fa2f015b380 2024-04-24T23:10:57.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.922+0000 7fa2f4cfb640 1 -- 172.21.15.32:0/177170753 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec032660 con 0x7fa2f015b380 2024-04-24T23:10:57.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.923+0000 7fa2f7f61640 1 -- 172.21.15.32:0/177170753 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2f015b380 msgr2=0x7fa2f006c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:57.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.923+0000 7fa2f7f61640 1 --2- 172.21.15.32:0/177170753 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2f015b380 0x7fa2f006c5f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec0097a0 tx=0x7fa2ec02a100 comp rx=0 tx=0).stop 2024-04-24T23:10:57.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.923+0000 7fa2f7f61640 1 -- 172.21.15.32:0/177170753 shutdown_connections 2024-04-24T23:10:57.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.923+0000 7fa2f7f61640 1 --2- 172.21.15.32:0/177170753 >> v2:172.21.15.121:3300/0 conn(0x7fa2f00ceb40 0x7fa2f006cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:57.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.923+0000 7fa2f7f61640 1 --2- 172.21.15.32:0/177170753 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2f015b380 0x7fa2f006c5f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:57.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.923+0000 7fa2f7f61640 1 -- 172.21.15.32:0/177170753 >> 172.21.15.32:0/177170753 conn(0x7fa2f006b110 msgr2=0x7fa2f006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:57.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.923+0000 7fa2f7f61640 1 -- 172.21.15.32:0/177170753 shutdown_connections 2024-04-24T23:10:57.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.923+0000 7fa2f7f61640 1 -- 172.21.15.32:0/177170753 wait complete. 2024-04-24T23:10:57.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.924+0000 7fa2f7f61640 1 Processor -- start 2024-04-24T23:10:57.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.924+0000 7fa2f7f61640 1 -- start start 2024-04-24T23:10:57.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.925+0000 7fa2f7f61640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2f00ceb40 0x7fa2f0148da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:57.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.925+0000 7fa2f7f61640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2f015b380 0x7fa2f01e4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:57.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.925+0000 7fa2f7f61640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2f015cb00 0x7fa2f00b3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:57.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.925+0000 7fa2f7f61640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f015b760 con 0x7fa2f00ceb40 2024-04-24T23:10:57.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.925+0000 7fa2f7f61640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f00b50a0 con 0x7fa2f015b380 2024-04-24T23:10:57.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.925+0000 7fa2f7f61640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa2f00b55c0 con 0x7fa2f015cb00 2024-04-24T23:10:57.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.925+0000 7fa2f5cfd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2f00ceb40 0x7fa2f0148da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:57.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.925+0000 7fa2f5cfd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2f00ceb40 0x7fa2f0148da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41148/0 (socket says 172.21.15.32:41148) 2024-04-24T23:10:57.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.925+0000 7fa2f5cfd640 1 -- 172.21.15.32:0/3932340321 learned_addr learned my addr 172.21.15.32:0/3932340321 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:10:57.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.925+0000 7fa2f54fc640 1 --2- 172.21.15.32:0/3932340321 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2f015b380 0x7fa2f01e4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:57.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.925+0000 7fa2f64fe640 1 --2- 172.21.15.32:0/3932340321 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2f015cb00 0x7fa2f00b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:57.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.925+0000 7fa2f5cfd640 1 -- 172.21.15.32:0/3932340321 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2f015cb00 msgr2=0x7fa2f00b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:57.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.925+0000 7fa2f5cfd640 1 --2- 172.21.15.32:0/3932340321 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2f015cb00 0x7fa2f00b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:57.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.925+0000 7fa2f5cfd640 1 -- 172.21.15.32:0/3932340321 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2f015b380 msgr2=0x7fa2f01e4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:57.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.925+0000 7fa2f5cfd640 1 --2- 172.21.15.32:0/3932340321 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2f015b380 0x7fa2f01e4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:57.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.926+0000 7fa2f5cfd640 1 -- 172.21.15.32:0/3932340321 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2f01f7fd0 con 0x7fa2f00ceb40 2024-04-24T23:10:57.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.926+0000 7fa2f54fc640 1 --2- 172.21.15.32:0/3932340321 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2f015b380 0x7fa2f01e4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:10:57.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.926+0000 7fa2f5cfd640 1 --2- 172.21.15.32:0/3932340321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2f00ceb40 0x7fa2f0148da0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec035040 tx=0x7fa2ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:57.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.926+0000 7fa2e67fc640 1 -- 172.21.15.32:0/3932340321 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec041070 con 0x7fa2f00ceb40 2024-04-24T23:10:57.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.926+0000 7fa2e67fc640 1 -- 172.21.15.32:0/3932340321 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2ec03c460 con 0x7fa2f00ceb40 2024-04-24T23:10:57.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.926+0000 7fa2f7f61640 1 -- 172.21.15.32:0/3932340321 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2f01f82b0 con 0x7fa2f00ceb40 2024-04-24T23:10:57.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.926+0000 7fa2e67fc640 1 -- 172.21.15.32:0/3932340321 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec037040 con 0x7fa2f00ceb40 2024-04-24T23:10:57.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.926+0000 7fa2f7f61640 1 -- 172.21.15.32:0/3932340321 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2f01f8870 con 0x7fa2f00ceb40 2024-04-24T23:10:57.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.927+0000 7fa2f7f61640 1 -- 172.21.15.32:0/3932340321 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2f0068430 con 0x7fa2f00ceb40 2024-04-24T23:10:57.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.927+0000 7fa2e67fc640 1 -- 172.21.15.32:0/3932340321 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa2ec03a070 con 0x7fa2f00ceb40 2024-04-24T23:10:57.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.928+0000 7fa2e67fc640 1 --2- 172.21.15.32:0/3932340321 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa2d406be30 0x7fa2d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:57.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.928+0000 7fa2e67fc640 1 -- 172.21.15.32:0/3932340321 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(189..189 src has 1..189) ==== 5738+0+0 (secure 0 0 0) 0x7fa2ec0ab060 con 0x7fa2f00ceb40 2024-04-24T23:10:57.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.928+0000 7fa2e67fc640 1 -- 172.21.15.32:0/3932340321 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=190}) -- 0x7fa2d4073a20 con 0x7fa2f00ceb40 2024-04-24T23:10:57.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.928+0000 7fa2f54fc640 1 --2- 172.21.15.32:0/3932340321 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa2d406be30 0x7fa2d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:57.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.929+0000 7fa2f54fc640 1 --2- 172.21.15.32:0/3932340321 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa2d406be30 0x7fa2d406e2f0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fa2f00b1f40 tx=0x7fa2e0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:57.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:57.932+0000 7fa2e67fc640 1 -- 172.21.15.32:0/3932340321 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa2ec0799d0 con 0x7fa2f00ceb40 2024-04-24T23:10:58.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:58.119+0000 7fa2f7f61640 1 -- 172.21.15.32:0/3932340321 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7fa2f00b0ee0 con 0x7fa2f00ceb40 2024-04-24T23:10:58.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:58.120+0000 7fa2e67fc640 1 -- 172.21.15.32:0/3932340321 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v189) ==== 81+0+34398 (secure 0 0 0) 0x7fa2ec0792d0 con 0x7fa2f00ceb40 2024-04-24T23:10:58.122 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:10:58.122 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:10:58.122 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 189, 2024-04-24T23:10:58.122 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", 2024-04-24T23:10:58.122 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2024-04-24T22:58:46.964851+0000", 2024-04-24T23:10:58.123 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2024-04-24T23:10:54.060025+0000", 2024-04-24T23:10:58.123 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2024-04-24T22:58:52.048787+0000", 2024-04-24T23:10:58.123 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2024-04-24T22:58:49.045609+0000", 2024-04-24T23:10:58.123 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-24T23:10:58.123 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2024-04-24T23:10:58.123 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2024-04-24T23:10:58.123 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2024-04-24T23:10:58.123 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2024-04-24T23:10:58.123 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2024-04-24T23:10:58.123 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2024-04-24T23:10:58.123 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:10:58.123 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 4, 2024-04-24T23:10:58.123 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.69999998807907104, 2024-04-24T23:10:58.124 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-24T23:10:58.124 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-24T23:10:58.124 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2024-04-24T23:10:58.124 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 23, 2024-04-24T23:10:58.124 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2024-04-24T23:10:58.124 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "luminous", 2024-04-24T23:10:58.124 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "luminous", 2024-04-24T23:10:58.124 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "squid", 2024-04-24T23:10:58.124 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_crimson": false, 2024-04-24T23:10:58.124 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2024-04-24T23:10:58.124 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.124 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2024-04-24T23:10:58.124 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": ".mgr", 2024-04-24T23:10:58.124 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T22:58:53.171408+0000", 2024-04-24T23:10:58.124 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:10:58.124 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:10:58.125 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:10:58.125 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:10:58.125 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:10:58.125 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:10:58.125 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:10:58.125 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:10:58.125 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:10:58.125 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:10:58.125 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:10:58.125 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:10:58.125 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2024-04-24T23:10:58.125 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2024-04-24T23:10:58.125 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2024-04-24T23:10:58.125 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2024-04-24T23:10:58.125 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2024-04-24T23:10:58.125 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:10:58.126 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:10:58.126 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:10:58.126 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:10:58.126 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:10:58.126 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:10:58.126 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:10:58.126 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.126 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "113", 2024-04-24T23:10:58.126 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:10:58.127 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:10:58.127 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:10:58.127 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:10:58.127 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:10:58.127 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:10:58.127 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:10:58.127 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:10:58.127 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:10:58.127 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:10:58.127 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:10:58.127 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:10:58.127 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:10:58.127 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:10:58.127 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:10:58.128 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:10:58.128 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:10:58.128 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:10:58.128 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:10:58.128 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:10:58.128 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:10:58.128 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:10:58.128 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:10:58.128 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:10:58.128 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:10:58.128 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:10:58.128 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.128 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:10:58.129 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:10:58.129 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:10:58.129 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:10:58.129 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:10:58.129 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:10:58.129 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:10:58.129 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:10:58.129 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:10:58.129 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:10:58.129 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:10:58.129 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:10:58.130 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_max": 32, 2024-04-24T23:10:58.130 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2024-04-24T23:10:58.130 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.130 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:10:58.130 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": {} 2024-04-24T23:10:58.130 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.130 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:10:58.130 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:10:58.130 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 8, 2024-04-24T23:10:58.130 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 8, 2024-04-24T23:10:58.130 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.25, 2024-04-24T23:10:58.130 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-04-24T23:10:58.130 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-04-24T23:10:58.131 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:10:58.131 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:10:58.131 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:10:58.131 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:58.131 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.131 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.131 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 22, 2024-04-24T23:10:58.131 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2024-04-24T23:10:58.131 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:10:34.649562+0000", 2024-04-24T23:10:58.131 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:10:58.131 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:10:58.131 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:10:58.131 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:10:58.131 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:10:58.132 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:10:58.132 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:10:58.132 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:10:58.132 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:10:58.132 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:10:58.132 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:10:58.132 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:10:58.132 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 32, 2024-04-24T23:10:58.132 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 32, 2024-04-24T23:10:58.132 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 32, 2024-04-24T23:10:58.132 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 32, 2024-04-24T23:10:58.132 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 32, 2024-04-24T23:10:58.132 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:10:58.133 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:10:58.133 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:10:58.133 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:10:58.133 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:10:58.133 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:10:58.133 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:10:58.133 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.133 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "184", 2024-04-24T23:10:58.133 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:10:58.134 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:10:58.134 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:10:58.134 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:10:58.134 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:10:58.134 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:10:58.134 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:10:58.134 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:10:58.134 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:10:58.134 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:10:58.134 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:10:58.134 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:10:58.134 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:10:58.134 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:10:58.135 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:10:58.135 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:10:58.135 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:10:58.135 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:10:58.135 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:10:58.135 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:10:58.135 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:10:58.135 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:10:58.135 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:10:58.135 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:10:58.135 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:10:58.135 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:10:58.135 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.135 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:10:58.136 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:10:58.136 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:10:58.136 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:10:58.136 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:10:58.136 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:10:58.136 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:10:58.136 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:10:58.136 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:10:58.136 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:10:58.136 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:10:58.136 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:10:58.136 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2024-04-24T23:10:58.137 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2024-04-24T23:10:58.137 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2024-04-24T23:10:58.137 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.137 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:10:58.137 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:10:58.137 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2024-04-24T23:10:58.137 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:58.137 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.137 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:10:58.138 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:10:58.138 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.5, 2024-04-24T23:10:58.138 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.5, 2024-04-24T23:10:58.138 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:10:58.138 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.5, 2024-04-24T23:10:58.138 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.5, 2024-04-24T23:10:58.138 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:10:58.138 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:10:58.138 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:10:58.138 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:58.138 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.138 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.138 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 23, 2024-04-24T23:10:58.138 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2024-04-24T23:10:58.139 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:10:35.339604+0000", 2024-04-24T23:10:58.139 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1027, 2024-04-24T23:10:58.139 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,full,full_quota", 2024-04-24T23:10:58.139 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:10:58.139 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:10:58.139 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:10:58.139 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:10:58.139 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:10:58.139 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:10:58.139 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:10:58.139 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:10:58.139 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:10:58.139 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:10:58.139 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 64, 2024-04-24T23:10:58.139 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 64, 2024-04-24T23:10:58.140 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 64, 2024-04-24T23:10:58.140 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 64, 2024-04-24T23:10:58.140 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 64, 2024-04-24T23:10:58.140 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:10:58.140 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:10:58.140 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:10:58.140 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:10:58.140 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:10:58.140 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:10:58.140 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:10:58.140 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.140 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "187", 2024-04-24T23:10:58.140 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:10:58.140 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:10:58.141 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:10:58.141 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:10:58.141 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:10:58.141 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:10:58.141 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:10:58.141 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:10:58.141 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:10:58.141 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 33554432, 2024-04-24T23:10:58.141 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:10:58.141 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:10:58.141 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:10:58.141 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:10:58.141 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:10:58.141 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:10:58.141 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:10:58.142 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:10:58.142 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:10:58.142 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:10:58.142 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:10:58.142 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:10:58.142 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:10:58.142 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:10:58.142 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:10:58.142 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:10:58.142 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.142 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:10:58.142 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:10:58.142 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:10:58.143 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:10:58.143 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:10:58.143 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:10:58.143 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:10:58.143 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:10:58.143 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:10:58.143 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:10:58.143 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:10:58.143 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:10:58.143 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 64, 2024-04-24T23:10:58.143 INFO:teuthology.orchestra.run.smithi032.stdout: "target_size_ratio": 0.89999997615814209 2024-04-24T23:10:58.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.144 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:10:58.144 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:10:58.144 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2024-04-24T23:10:58.144 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:58.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.144 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:10:58.144 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:10:58.144 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.3799999952316284, 2024-04-24T23:10:58.144 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.3799999952316284, 2024-04-24T23:10:58.144 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:10:58.144 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.3799999952316284, 2024-04-24T23:10:58.144 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.3799999952316284, 2024-04-24T23:10:58.144 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:10:58.144 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:10:58.145 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:10:58.145 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:58.145 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:58.145 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:10:58.145 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-04-24T23:10:58.145 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.145 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:10:58.145 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:10:58.145 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:10:58.145 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:10:58.145 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:10:58.146 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:10:58.146 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:10:58.146 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:10:58.146 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:10:58.146 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:10:58.146 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:10:58.146 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:10:58.146 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:10:58.146 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:58.146 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.146 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:58.146 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2024-04-24T23:10:58.146 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:10:58.147 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.147 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.147 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:58.147 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2024-04-24T23:10:58.147 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:10:58.147 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:58.147 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:58.147 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.147 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:10:58.147 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:58.147 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.148 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:58.148 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2024-04-24T23:10:58.148 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:10:58.148 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.148 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.148 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:58.148 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2024-04-24T23:10:58.148 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:10:58.148 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:58.148 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:58.148 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.148 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:10:58.148 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:58.148 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.148 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:58.149 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2024-04-24T23:10:58.149 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:10:58.149 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.149 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.149 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:58.149 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2024-04-24T23:10:58.149 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:10:58.149 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:58.149 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:58.149 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.149 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:10:58.149 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:58.149 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.149 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:58.149 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2024-04-24T23:10:58.150 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:10:58.150 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.150 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.150 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:58.150 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2024-04-24T23:10:58.150 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:10:58.150 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:58.150 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:58.150 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.150 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/1479395426", 2024-04-24T23:10:58.150 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/1479395426", 2024-04-24T23:10:58.150 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/1479395426", 2024-04-24T23:10:58.150 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/1479395426", 2024-04-24T23:10:58.151 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:10:58.151 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:10:58.151 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:10:58.151 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:58.151 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.151 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.151 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:10:58.151 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:10:58.151 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:10:58.151 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:10:58.151 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:10:58.151 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:10:58.151 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:10:58.151 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:10:58.151 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:10:58.151 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:10:58.152 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:10:58.152 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:10:58.152 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:10:58.152 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:58.152 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.152 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:58.152 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2024-04-24T23:10:58.152 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:10:58.152 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.152 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.152 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:58.152 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2024-04-24T23:10:58.152 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:10:58.152 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:58.152 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:58.153 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.153 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:10:58.153 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:58.153 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.153 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:58.153 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2024-04-24T23:10:58.153 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:10:58.153 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.153 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.153 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:58.153 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2024-04-24T23:10:58.153 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:10:58.153 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:58.154 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:58.154 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.154 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:10:58.154 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:58.154 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.154 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:58.154 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2024-04-24T23:10:58.154 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:10:58.154 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.154 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.154 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:58.154 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2024-04-24T23:10:58.154 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:10:58.155 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:58.155 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:58.155 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.155 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:10:58.155 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:58.155 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.155 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:58.155 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2024-04-24T23:10:58.155 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:10:58.155 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.155 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.155 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:58.155 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2024-04-24T23:10:58.155 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:10:58.155 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:58.155 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:58.156 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.156 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/1340633718", 2024-04-24T23:10:58.156 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/1340633718", 2024-04-24T23:10:58.156 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/1340633718", 2024-04-24T23:10:58.156 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/1340633718", 2024-04-24T23:10:58.156 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:10:58.156 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:10:58.156 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:10:58.156 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:58.156 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.156 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.156 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:10:58.156 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:10:58.156 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:10:58.156 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:10:58.157 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:10:58.157 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:10:58.157 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:10:58.157 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:10:58.157 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:10:58.157 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:10:58.157 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:10:58.157 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:10:58.157 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:10:58.157 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:58.157 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.157 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:58.157 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2024-04-24T23:10:58.157 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:10:58.157 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.158 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.158 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:58.158 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2024-04-24T23:10:58.158 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:10:58.158 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:58.158 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:58.158 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.158 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:10:58.158 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:58.158 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.158 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:58.159 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2024-04-24T23:10:58.159 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:10:58.159 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.159 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.159 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:58.159 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2024-04-24T23:10:58.159 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:10:58.159 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:58.159 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:58.159 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.159 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:10:58.159 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:58.159 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.160 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:58.160 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2024-04-24T23:10:58.160 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:10:58.160 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.160 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.160 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:58.160 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2024-04-24T23:10:58.160 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:10:58.160 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:58.160 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:58.160 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.160 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:10:58.161 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:58.161 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.161 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:58.161 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2024-04-24T23:10:58.161 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:10:58.161 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.161 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.161 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:58.161 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2024-04-24T23:10:58.161 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:10:58.161 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:58.161 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:58.161 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.161 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/3405899334", 2024-04-24T23:10:58.162 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6820/3405899334", 2024-04-24T23:10:58.162 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6828/3405899334", 2024-04-24T23:10:58.162 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6824/3405899334", 2024-04-24T23:10:58.162 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:10:58.162 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:10:58.162 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:10:58.162 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:58.162 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.162 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.162 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:10:58.162 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:10:58.162 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:10:58.162 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:10:58.162 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:10:58.163 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:10:58.163 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:10:58.163 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:10:58.163 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:10:58.163 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 188, 2024-04-24T23:10:58.163 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:10:58.163 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:10:58.163 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:10:58.163 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:58.163 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.163 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:58.163 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2024-04-24T23:10:58.163 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:10:58.164 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.164 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.164 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:58.164 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2024-04-24T23:10:58.164 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:10:58.164 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:58.164 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:58.164 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.164 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:10:58.164 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:58.165 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.165 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:58.165 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2024-04-24T23:10:58.165 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:10:58.165 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.165 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.165 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:58.165 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2024-04-24T23:10:58.165 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:10:58.165 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:58.165 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:58.165 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.165 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:10:58.165 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:58.165 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.166 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:58.166 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2024-04-24T23:10:58.166 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:10:58.166 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.166 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.166 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:58.166 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2024-04-24T23:10:58.166 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:10:58.166 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:58.166 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:58.166 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.166 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:10:58.166 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:58.167 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.167 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:58.167 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2024-04-24T23:10:58.167 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:10:58.167 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.167 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.167 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:58.167 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2024-04-24T23:10:58.167 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:10:58.167 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:58.167 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:58.167 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.167 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6821/3381057190", 2024-04-24T23:10:58.167 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6825/3381057190", 2024-04-24T23:10:58.167 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/3381057190", 2024-04-24T23:10:58.168 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/3381057190", 2024-04-24T23:10:58.168 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:10:58.168 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:10:58.168 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:10:58.168 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:58.168 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.168 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.168 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:10:58.168 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:10:58.169 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:10:58.169 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:10:58.169 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:10:58.169 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:10:58.169 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:10:58.169 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:10:58.169 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:10:58.169 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:10:58.169 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:10:58.169 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:10:58.170 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:10:58.170 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:58.170 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.170 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:58.170 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6800", 2024-04-24T23:10:58.170 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:10:58.170 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.170 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.170 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:58.170 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6801", 2024-04-24T23:10:58.170 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:10:58.170 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:58.170 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:58.170 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.171 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:10:58.171 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:58.171 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.171 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:58.171 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6802", 2024-04-24T23:10:58.171 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:10:58.171 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.171 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.171 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:58.171 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6803", 2024-04-24T23:10:58.171 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:10:58.171 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:58.171 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:58.171 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.171 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:10:58.172 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:58.172 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.172 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:58.172 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6806", 2024-04-24T23:10:58.172 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:10:58.172 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.172 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.172 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:58.172 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6807", 2024-04-24T23:10:58.172 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:10:58.172 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:58.172 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:58.172 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.172 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:10:58.172 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:58.173 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.173 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:58.173 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6804", 2024-04-24T23:10:58.173 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:10:58.173 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.173 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.173 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:58.173 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6805", 2024-04-24T23:10:58.173 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:10:58.173 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:58.173 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:58.173 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.173 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6801/4010553979", 2024-04-24T23:10:58.173 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6803/4010553979", 2024-04-24T23:10:58.174 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6807/4010553979", 2024-04-24T23:10:58.174 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6805/4010553979", 2024-04-24T23:10:58.174 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:10:58.174 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:10:58.174 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:10:58.174 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:58.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.174 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.174 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:10:58.174 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:10:58.174 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:10:58.175 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:10:58.175 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:10:58.175 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:10:58.175 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:10:58.175 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:10:58.175 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:10:58.175 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:10:58.175 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:10:58.175 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:10:58.175 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:10:58.175 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:58.175 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.175 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:58.175 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6808", 2024-04-24T23:10:58.175 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:10:58.175 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.176 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.176 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:58.176 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6809", 2024-04-24T23:10:58.176 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:10:58.176 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:58.176 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:58.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.176 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:10:58.176 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:58.176 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.176 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:58.176 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6810", 2024-04-24T23:10:58.176 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:10:58.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.176 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.177 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:58.177 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6811", 2024-04-24T23:10:58.177 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:10:58.177 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:58.177 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:58.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.177 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:10:58.177 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:58.177 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.177 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:58.177 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6814", 2024-04-24T23:10:58.177 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:10:58.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.177 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.177 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:58.178 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6815", 2024-04-24T23:10:58.178 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:10:58.178 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:58.178 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:58.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.178 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:10:58.178 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:58.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.178 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:58.178 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6812", 2024-04-24T23:10:58.178 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:10:58.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.178 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:58.179 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6813", 2024-04-24T23:10:58.179 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:10:58.179 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:58.179 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:58.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.179 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6809/3794084091", 2024-04-24T23:10:58.179 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6811/3794084091", 2024-04-24T23:10:58.179 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6815/3794084091", 2024-04-24T23:10:58.179 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6813/3794084091", 2024-04-24T23:10:58.179 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:10:58.179 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:10:58.179 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:10:58.180 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:58.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.180 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.180 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:10:58.180 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:10:58.180 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:10:58.180 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:10:58.180 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:10:58.180 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:10:58.180 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:10:58.180 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:10:58.180 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:10:58.180 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:10:58.180 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:10:58.180 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:10:58.181 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:10:58.181 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:58.181 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.181 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:58.181 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6823", 2024-04-24T23:10:58.181 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:10:58.181 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.181 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.181 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:58.181 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6825", 2024-04-24T23:10:58.181 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:10:58.181 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:58.181 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:58.181 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.182 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:10:58.182 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:58.182 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.182 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:58.182 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6826", 2024-04-24T23:10:58.182 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:10:58.182 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.182 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.182 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:58.182 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6827", 2024-04-24T23:10:58.182 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:10:58.182 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:58.182 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:58.182 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.183 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:10:58.183 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:58.183 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.183 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:58.183 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6830", 2024-04-24T23:10:58.183 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:10:58.183 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.183 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.183 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:58.183 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6831", 2024-04-24T23:10:58.183 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:10:58.183 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:58.183 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:58.183 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.183 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:10:58.184 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:58.184 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.184 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:58.184 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6828", 2024-04-24T23:10:58.184 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:10:58.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.184 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.184 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:58.184 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6829", 2024-04-24T23:10:58.184 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:10:58.184 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:58.184 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:58.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.184 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6825/1348640258", 2024-04-24T23:10:58.185 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6827/1348640258", 2024-04-24T23:10:58.185 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6831/1348640258", 2024-04-24T23:10:58.185 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6829/1348640258", 2024-04-24T23:10:58.185 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:10:58.185 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:10:58.185 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:10:58.185 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:58.185 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.185 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.185 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:10:58.185 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:10:58.185 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:10:58.186 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:10:58.186 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:10:58.186 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:10:58.186 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:10:58.186 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:10:58.186 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:10:58.186 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:10:58.186 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:10:58.186 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:10:58.186 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:10:58.186 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:58.186 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.186 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:58.186 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6816", 2024-04-24T23:10:58.186 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:10:58.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.187 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.187 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:58.187 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6817", 2024-04-24T23:10:58.187 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:10:58.187 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:58.187 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:58.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.187 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:10:58.187 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:58.187 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.187 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:58.187 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6818", 2024-04-24T23:10:58.187 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:10:58.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.187 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.187 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:58.188 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6819", 2024-04-24T23:10:58.188 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:10:58.188 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:58.188 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:58.188 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.188 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:10:58.188 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:58.188 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.188 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:58.188 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6822", 2024-04-24T23:10:58.188 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:10:58.188 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.188 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.188 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:58.188 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6824", 2024-04-24T23:10:58.188 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:10:58.188 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:58.189 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:58.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.189 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:10:58.189 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:10:58.189 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.189 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:10:58.189 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6820", 2024-04-24T23:10:58.189 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:10:58.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.189 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.190 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:10:58.190 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6821", 2024-04-24T23:10:58.190 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:10:58.190 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:58.190 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:58.190 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.190 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6817/3250005122", 2024-04-24T23:10:58.190 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6819/3250005122", 2024-04-24T23:10:58.190 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6824/3250005122", 2024-04-24T23:10:58.190 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6821/3250005122", 2024-04-24T23:10:58.191 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:10:58.191 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:10:58.191 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:10:58.191 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:58.191 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:58.191 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:10:58.191 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2024-04-24T23:10:58.191 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.191 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:10:58.191 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:10:58.191 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:10:58.191 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:10:58.191 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:10:58.191 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:10:58.191 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.156184+0000", 2024-04-24T23:10:58.191 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:10:58.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.192 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:10:58.192 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:10:58.192 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:10:58.192 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:10:58.192 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:10:58.192 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:10:58.192 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240774+0000", 2024-04-24T23:10:58.192 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:10:58.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.192 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:10:58.192 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:10:58.192 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:10:58.192 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:10:58.193 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:10:58.193 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:10:58.193 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240949+0000", 2024-04-24T23:10:58.193 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:10:58.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.193 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:10:58.193 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:10:58.193 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:10:58.193 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:10:58.193 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:10:58.193 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:10:58.193 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.171672+0000", 2024-04-24T23:10:58.193 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:10:58.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.194 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.194 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:10:58.194 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:10:58.194 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:10:58.194 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:10:58.194 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:10:58.194 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:10:58.194 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.220379+0000", 2024-04-24T23:10:58.194 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:10:58.194 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.194 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.194 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:10:58.194 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:10:58.194 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:10:58.195 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:10:58.195 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:10:58.195 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:10:58.195 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.204963+0000", 2024-04-24T23:10:58.195 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:10:58.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.195 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.195 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:10:58.195 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:10:58.195 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:10:58.195 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:10:58.196 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:10:58.196 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:10:58.196 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.239748+0000", 2024-04-24T23:10:58.196 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:10:58.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.196 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:10:58.196 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:10:58.196 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:10:58.196 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:10:58.196 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:10:58.196 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:10:58.196 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.154104+0000", 2024-04-24T23:10:58.196 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:10:58.197 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:58.197 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:10:58.197 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2024-04-24T23:10:58.197 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [ 2024-04-24T23:10:58.197 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.197 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "23.6", 2024-04-24T23:10:58.197 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:10:58.197 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:10:58.197 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 5, 2024-04-24T23:10:58.197 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 3 2024-04-24T23:10:58.197 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:58.197 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:10:58.197 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:58.197 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:10:58.197 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_primaries": [], 2024-04-24T23:10:58.197 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2024-04-24T23:10:58.198 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2024-04-24T23:10:58.198 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklist": {}, 2024-04-24T23:10:58.198 INFO:teuthology.orchestra.run.smithi032.stdout: "range_blocklist": {}, 2024-04-24T23:10:58.198 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2024-04-24T23:10:58.198 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2024-04-24T23:10:58.198 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2024-04-24T23:10:58.198 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2024-04-24T23:10:58.198 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2024-04-24T23:10:58.198 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2024-04-24T23:10:58.198 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2024-04-24T23:10:58.198 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:58.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:10:58.198 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2024-04-24T23:10:58.198 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2024-04-24T23:10:58.199 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2024-04-24T23:10:58.199 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2024-04-24T23:10:58.199 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {}, 2024-04-24T23:10:58.199 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode": { 2024-04-24T23:10:58.199 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_enabled": false, 2024-04-24T23:10:58.199 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_bucket_count": 0, 2024-04-24T23:10:58.199 INFO:teuthology.orchestra.run.smithi032.stdout: "degraded_stretch_mode": 0, 2024-04-24T23:10:58.199 INFO:teuthology.orchestra.run.smithi032.stdout: "recovering_stretch_mode": 0, 2024-04-24T23:10:58.199 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_bucket": 0 2024-04-24T23:10:58.199 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:10:58.199 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:10:58.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:58.122+0000 7fa2f7f61640 1 -- 172.21.15.32:0/3932340321 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa2d406be30 msgr2=0x7fa2d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:58.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:58.122+0000 7fa2f7f61640 1 --2- 172.21.15.32:0/3932340321 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa2d406be30 0x7fa2d406e2f0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fa2f00b1f40 tx=0x7fa2e0009040 comp rx=0 tx=0).stop 2024-04-24T23:10:58.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:58.122+0000 7fa2f7f61640 1 -- 172.21.15.32:0/3932340321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2f00ceb40 msgr2=0x7fa2f0148da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:58.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:58.122+0000 7fa2f7f61640 1 --2- 172.21.15.32:0/3932340321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2f00ceb40 0x7fa2f0148da0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec035040 tx=0x7fa2ec0046c0 comp rx=0 tx=0).stop 2024-04-24T23:10:58.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:58.122+0000 7fa2f7f61640 1 -- 172.21.15.32:0/3932340321 shutdown_connections 2024-04-24T23:10:58.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:58.122+0000 7fa2f7f61640 1 --2- 172.21.15.32:0/3932340321 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa2d406be30 0x7fa2d406e2f0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:58.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:58.122+0000 7fa2f7f61640 1 --2- 172.21.15.32:0/3932340321 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2f015cb00 0x7fa2f00b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:58.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:58.122+0000 7fa2f7f61640 1 --2- 172.21.15.32:0/3932340321 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2f015b380 0x7fa2f01e4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:58.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:58.122+0000 7fa2f7f61640 1 --2- 172.21.15.32:0/3932340321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2f00ceb40 0x7fa2f0148da0 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:58.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:58.122+0000 7fa2f7f61640 1 -- 172.21.15.32:0/3932340321 >> 172.21.15.32:0/3932340321 conn(0x7fa2f006b110 msgr2=0x7fa2f015d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:58.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:58.122+0000 7fa2f7f61640 1 -- 172.21.15.32:0/3932340321 shutdown_connections 2024-04-24T23:10:58.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:10:58.122+0000 7fa2f7f61640 1 -- 172.21.15.32:0/3932340321 wait complete. 2024-04-24T23:10:58.202 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-04-24T23:11:03.203 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-24T23:11:03.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.316+0000 7f7b6b238640 1 Processor -- start 2024-04-24T23:11:03.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.316+0000 7f7b6b238640 1 -- start start 2024-04-24T23:11:03.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.317+0000 7f7b6b238640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7b6415b380 0x7f7b6406c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:03.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.317+0000 7f7b6b238640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7b640ceb40 0x7f7b6406cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:03.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.317+0000 7f7b6b238640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b640b4520 con 0x7f7b6415cb00 2024-04-24T23:11:03.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.317+0000 7f7b6b238640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f7b640cef90 con 0x7f7b640ceb40 2024-04-24T23:11:03.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.317+0000 7f7b6b238640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f7b640b4800 con 0x7f7b6415b380 2024-04-24T23:11:03.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.317+0000 7f7b68fd4640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7b6415b380 0x7f7b6406c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:03.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.317+0000 7f7b697d5640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7b640ceb40 0x7f7b6406cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:03.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.317+0000 7f7b63fff640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f7b6415cb00 0x7f7b640ce430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:49406/0 (socket says 172.21.15.32:49406) 2024-04-24T23:11:03.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.317+0000 7f7b63fff640 1 -- 172.21.15.32:0/344227449 learned_addr learned my addr 172.21.15.32:0/344227449 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:11:03.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.318+0000 7f7b637fe640 1 -- 172.21.15.32:0/344227449 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1759000125 0 0) 0x7f7b640b4520 con 0x7f7b6415cb00 2024-04-24T23:11:03.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.318+0000 7f7b637fe640 1 -- 172.21.15.32:0/344227449 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b54003520 con 0x7f7b6415cb00 2024-04-24T23:11:03.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.318+0000 7f7b697d5640 1 -- 172.21.15.32:0/344227449 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7b6415b380 msgr2=0x7f7b6406c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:03.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.318+0000 7f7b697d5640 1 --2- 172.21.15.32:0/344227449 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7b6415b380 0x7f7b6406c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:03.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.318+0000 7f7b697d5640 1 -- 172.21.15.32:0/344227449 >> v1:172.21.15.121:6789/0 conn(0x7f7b6415cb00 legacy=0x7f7b640ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:03.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.318+0000 7f7b697d5640 1 -- 172.21.15.32:0/344227449 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b640b4ae0 con 0x7f7b640ceb40 2024-04-24T23:11:03.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.318+0000 7f7b68fd4640 1 --2- 172.21.15.32:0/344227449 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7b6415b380 0x7f7b6406c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:11:03.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.319+0000 7f7b697d5640 1 --2- 172.21.15.32:0/344227449 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7b640ceb40 0x7f7b6406cb30 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c009d90 tx=0x7f7b5c02a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6ab4570d20e62c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:03.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.319+0000 7f7b637fe640 1 -- 172.21.15.32:0/344227449 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c031600 con 0x7f7b640ceb40 2024-04-24T23:11:03.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.319+0000 7f7b637fe640 1 -- 172.21.15.32:0/344227449 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b5c0317a0 con 0x7f7b640ceb40 2024-04-24T23:11:03.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.319+0000 7f7b637fe640 1 -- 172.21.15.32:0/344227449 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c031b70 con 0x7f7b640ceb40 2024-04-24T23:11:03.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.319+0000 7f7b6b238640 1 -- 172.21.15.32:0/344227449 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7b640ceb40 msgr2=0x7f7b6406cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:03.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.319+0000 7f7b6b238640 1 --2- 172.21.15.32:0/344227449 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7b640ceb40 0x7f7b6406cb30 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c009d90 tx=0x7f7b5c02a5d0 comp rx=0 tx=0).stop 2024-04-24T23:11:03.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.319+0000 7f7b6b238640 1 -- 172.21.15.32:0/344227449 shutdown_connections 2024-04-24T23:11:03.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.319+0000 7f7b6b238640 1 --2- 172.21.15.32:0/344227449 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7b640ceb40 0x7f7b6406cb30 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:03.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.319+0000 7f7b6b238640 1 --2- 172.21.15.32:0/344227449 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7b6415b380 0x7f7b6406c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:03.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.319+0000 7f7b6b238640 1 -- 172.21.15.32:0/344227449 >> 172.21.15.32:0/344227449 conn(0x7f7b6406b110 msgr2=0x7f7b6406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:03.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.319+0000 7f7b6b238640 1 -- 172.21.15.32:0/344227449 shutdown_connections 2024-04-24T23:11:03.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.319+0000 7f7b6b238640 1 -- 172.21.15.32:0/344227449 wait complete. 2024-04-24T23:11:03.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.321+0000 7f7b6b238640 1 Processor -- start 2024-04-24T23:11:03.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.321+0000 7f7b6b238640 1 -- start start 2024-04-24T23:11:03.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.321+0000 7f7b6b238640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7b640ceb40 0x7f7b64159e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:03.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.321+0000 7f7b6b238640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b6415b380 0x7f7b64149360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:03.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.321+0000 7f7b6b238640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7b6415cb00 0x7f7b64151720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:03.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.321+0000 7f7b6b238640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6415b760 con 0x7f7b6415b380 2024-04-24T23:11:03.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.321+0000 7f7b6b238640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7b640b50a0 con 0x7f7b6415cb00 2024-04-24T23:11:03.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.321+0000 7f7b6b238640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f7b640b55c0 con 0x7f7b640ceb40 2024-04-24T23:11:03.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.322+0000 7f7b63fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b6415b380 0x7f7b64149360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:03.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.322+0000 7f7b63fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b6415b380 0x7f7b64149360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35570/0 (socket says 172.21.15.32:35570) 2024-04-24T23:11:03.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.322+0000 7f7b63fff640 1 -- 172.21.15.32:0/4147960999 learned_addr learned my addr 172.21.15.32:0/4147960999 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:03.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.322+0000 7f7b697d5640 1 --2- 172.21.15.32:0/4147960999 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7b6415cb00 0x7f7b64151720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:03.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.322+0000 7f7b68fd4640 1 --2- 172.21.15.32:0/4147960999 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7b640ceb40 0x7f7b64159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:03.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.322+0000 7f7b63fff640 1 -- 172.21.15.32:0/4147960999 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7b640ceb40 msgr2=0x7f7b64159e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:03.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.322+0000 7f7b63fff640 1 --2- 172.21.15.32:0/4147960999 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7b640ceb40 0x7f7b64159e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:03.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.322+0000 7f7b63fff640 1 -- 172.21.15.32:0/4147960999 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7b6415cb00 msgr2=0x7f7b64151720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:03.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.322+0000 7f7b63fff640 1 --2- 172.21.15.32:0/4147960999 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7b6415cb00 0x7f7b64151720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:03.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.322+0000 7f7b63fff640 1 -- 172.21.15.32:0/4147960999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b64151fe0 con 0x7f7b6415b380 2024-04-24T23:11:03.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.322+0000 7f7b63fff640 1 --2- 172.21.15.32:0/4147960999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b6415b380 0x7f7b64149360 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f7b5000c3c0 tx=0x7f7b5000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:03.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.322+0000 7f7b617fa640 1 -- 172.21.15.32:0/4147960999 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b50017070 con 0x7f7b6415b380 2024-04-24T23:11:03.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.322+0000 7f7b617fa640 1 -- 172.21.15.32:0/4147960999 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b50005ce0 con 0x7f7b6415b380 2024-04-24T23:11:03.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.322+0000 7f7b617fa640 1 -- 172.21.15.32:0/4147960999 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b500072f0 con 0x7f7b6415b380 2024-04-24T23:11:03.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.322+0000 7f7b6b238640 1 -- 172.21.15.32:0/4147960999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b641522c0 con 0x7f7b6415b380 2024-04-24T23:11:03.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.323+0000 7f7b6b238640 1 -- 172.21.15.32:0/4147960999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b64152880 con 0x7f7b6415b380 2024-04-24T23:11:03.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.323+0000 7f7b6b238640 1 -- 172.21.15.32:0/4147960999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b64068430 con 0x7f7b6415b380 2024-04-24T23:11:03.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.324+0000 7f7b617fa640 1 -- 172.21.15.32:0/4147960999 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7b5001a070 con 0x7f7b6415b380 2024-04-24T23:11:03.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.325+0000 7f7b617fa640 1 --2- 172.21.15.32:0/4147960999 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7b4806be30 0x7f7b4806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:03.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.325+0000 7f7b617fa640 1 -- 172.21.15.32:0/4147960999 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(189..189 src has 1..189) ==== 5738+0+0 (secure 0 0 0) 0x7f7b5008a250 con 0x7f7b6415b380 2024-04-24T23:11:03.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.325+0000 7f7b68fd4640 1 --2- 172.21.15.32:0/4147960999 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7b4806be30 0x7f7b4806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:03.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.325+0000 7f7b617fa640 1 -- 172.21.15.32:0/4147960999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=190}) -- 0x7f7b480739c0 con 0x7f7b6415b380 2024-04-24T23:11:03.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.325+0000 7f7b68fd4640 1 --2- 172.21.15.32:0/4147960999 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7b4806be30 0x7f7b4806e2f0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f7b58009560 tx=0x7f7b58009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:03.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.329+0000 7f7b617fa640 1 -- 172.21.15.32:0/4147960999 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7b50058510 con 0x7f7b6415b380 2024-04-24T23:11:03.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.512+0000 7f7b6b238640 1 -- 172.21.15.32:0/4147960999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f7b6406c000 con 0x7f7b6415b380 2024-04-24T23:11:03.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.514+0000 7f7b617fa640 1 -- 172.21.15.32:0/4147960999 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v189) ==== 81+0+34398 (secure 0 0 0) 0x7f7b5005c810 con 0x7f7b6415b380 2024-04-24T23:11:03.516 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:11:03.516 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:11:03.516 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 189, 2024-04-24T23:11:03.516 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", 2024-04-24T23:11:03.516 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2024-04-24T22:58:46.964851+0000", 2024-04-24T23:11:03.516 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2024-04-24T23:10:54.060025+0000", 2024-04-24T23:11:03.516 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2024-04-24T22:58:52.048787+0000", 2024-04-24T23:11:03.516 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2024-04-24T22:58:49.045609+0000", 2024-04-24T23:11:03.516 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-24T23:11:03.516 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2024-04-24T23:11:03.516 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2024-04-24T23:11:03.516 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2024-04-24T23:11:03.517 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2024-04-24T23:11:03.517 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2024-04-24T23:11:03.517 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2024-04-24T23:11:03.517 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:11:03.517 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 4, 2024-04-24T23:11:03.517 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.69999998807907104, 2024-04-24T23:11:03.517 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-24T23:11:03.517 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-24T23:11:03.517 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2024-04-24T23:11:03.517 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 23, 2024-04-24T23:11:03.517 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2024-04-24T23:11:03.517 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "luminous", 2024-04-24T23:11:03.517 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "luminous", 2024-04-24T23:11:03.518 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "squid", 2024-04-24T23:11:03.518 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_crimson": false, 2024-04-24T23:11:03.518 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2024-04-24T23:11:03.518 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.518 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2024-04-24T23:11:03.518 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": ".mgr", 2024-04-24T23:11:03.518 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T22:58:53.171408+0000", 2024-04-24T23:11:03.518 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:11:03.518 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:11:03.518 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:11:03.518 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:11:03.518 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:11:03.518 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:11:03.518 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:11:03.518 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:11:03.519 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:11:03.519 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:11:03.519 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:11:03.519 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:11:03.519 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2024-04-24T23:11:03.519 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2024-04-24T23:11:03.519 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2024-04-24T23:11:03.519 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2024-04-24T23:11:03.519 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2024-04-24T23:11:03.519 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:11:03.519 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:11:03.519 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:11:03.519 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:11:03.519 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:11:03.519 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:11:03.520 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:11:03.520 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.520 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "113", 2024-04-24T23:11:03.520 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:11:03.520 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:11:03.520 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:11:03.520 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:11:03.520 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:11:03.520 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:11:03.520 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:11:03.520 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:11:03.520 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:11:03.520 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:11:03.520 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:11:03.521 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:11:03.521 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:11:03.521 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:11:03.521 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:11:03.521 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:11:03.521 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:11:03.521 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:11:03.521 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:11:03.521 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:11:03.521 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:11:03.521 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:11:03.521 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:11:03.521 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:11:03.521 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:11:03.521 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:11:03.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.522 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:11:03.522 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:11:03.522 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:11:03.522 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:11:03.522 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:11:03.522 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:11:03.522 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:11:03.522 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:11:03.522 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:11:03.522 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:11:03.522 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:11:03.522 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:11:03.522 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_max": 32, 2024-04-24T23:11:03.523 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2024-04-24T23:11:03.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.523 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:11:03.523 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": {} 2024-04-24T23:11:03.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.523 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:11:03.523 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:11:03.523 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 8, 2024-04-24T23:11:03.523 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 8, 2024-04-24T23:11:03.523 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.25, 2024-04-24T23:11:03.523 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-04-24T23:11:03.523 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-04-24T23:11:03.523 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:11:03.523 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:11:03.523 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:11:03.524 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:03.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.524 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.524 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 22, 2024-04-24T23:11:03.524 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2024-04-24T23:11:03.524 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:10:34.649562+0000", 2024-04-24T23:11:03.524 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:11:03.524 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:11:03.524 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:11:03.524 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:11:03.524 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:11:03.524 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:11:03.524 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:11:03.524 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:11:03.525 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:11:03.525 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:11:03.525 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:11:03.525 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:11:03.525 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 32, 2024-04-24T23:11:03.525 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 32, 2024-04-24T23:11:03.525 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 32, 2024-04-24T23:11:03.525 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 32, 2024-04-24T23:11:03.525 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 32, 2024-04-24T23:11:03.525 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:11:03.525 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:11:03.525 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:11:03.525 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:11:03.525 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:11:03.526 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:11:03.526 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:11:03.526 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.526 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "184", 2024-04-24T23:11:03.526 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:11:03.526 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:11:03.526 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:11:03.526 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:11:03.526 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:11:03.526 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:11:03.526 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:11:03.526 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:11:03.526 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:11:03.526 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:11:03.527 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:11:03.527 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:11:03.527 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:11:03.527 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:11:03.527 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:11:03.527 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:11:03.527 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:11:03.527 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:11:03.527 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:11:03.527 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:11:03.527 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:11:03.527 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:11:03.527 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:11:03.527 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:11:03.528 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:11:03.528 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:11:03.528 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.528 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:11:03.528 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:11:03.528 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:11:03.528 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:11:03.528 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:11:03.528 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:11:03.528 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:11:03.528 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:11:03.528 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:11:03.528 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:11:03.528 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:11:03.528 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:11:03.529 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2024-04-24T23:11:03.529 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2024-04-24T23:11:03.529 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2024-04-24T23:11:03.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.529 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:11:03.529 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:11:03.529 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2024-04-24T23:11:03.529 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:03.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.529 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:11:03.529 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:11:03.529 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.5, 2024-04-24T23:11:03.529 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.5, 2024-04-24T23:11:03.529 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:11:03.529 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.5, 2024-04-24T23:11:03.530 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.5, 2024-04-24T23:11:03.530 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:11:03.530 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:11:03.530 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:11:03.530 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:03.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.530 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 23, 2024-04-24T23:11:03.530 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2024-04-24T23:11:03.530 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:10:35.339604+0000", 2024-04-24T23:11:03.530 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1027, 2024-04-24T23:11:03.530 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,full,full_quota", 2024-04-24T23:11:03.530 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:11:03.530 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:11:03.531 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:11:03.531 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:11:03.531 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:11:03.531 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:11:03.531 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:11:03.531 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:11:03.531 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:11:03.531 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:11:03.531 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 64, 2024-04-24T23:11:03.531 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 64, 2024-04-24T23:11:03.531 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 64, 2024-04-24T23:11:03.531 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 64, 2024-04-24T23:11:03.531 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 64, 2024-04-24T23:11:03.531 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:11:03.531 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:11:03.532 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:11:03.532 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:11:03.532 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:11:03.532 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:11:03.532 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:11:03.532 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.532 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "187", 2024-04-24T23:11:03.532 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:11:03.532 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:11:03.532 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:11:03.532 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:11:03.532 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:11:03.532 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:11:03.532 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:11:03.533 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:11:03.533 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:11:03.533 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 33554432, 2024-04-24T23:11:03.533 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:11:03.533 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:11:03.533 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:11:03.533 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:11:03.533 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:11:03.533 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:11:03.533 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:11:03.533 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:11:03.533 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:11:03.533 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:11:03.533 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:11:03.533 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:11:03.534 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:11:03.534 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:11:03.534 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:11:03.534 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:11:03.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.534 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:11:03.534 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:11:03.534 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:11:03.534 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:11:03.534 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:11:03.534 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:11:03.534 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:11:03.534 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:11:03.534 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:11:03.535 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:11:03.535 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:11:03.535 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:11:03.535 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 64, 2024-04-24T23:11:03.535 INFO:teuthology.orchestra.run.smithi032.stdout: "target_size_ratio": 0.89999997615814209 2024-04-24T23:11:03.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.535 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:11:03.535 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:11:03.535 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2024-04-24T23:11:03.535 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:03.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.535 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:11:03.535 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:11:03.535 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.3799999952316284, 2024-04-24T23:11:03.535 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.3799999952316284, 2024-04-24T23:11:03.536 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:11:03.536 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.3799999952316284, 2024-04-24T23:11:03.536 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.3799999952316284, 2024-04-24T23:11:03.536 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:11:03.536 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:11:03.536 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:11:03.536 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:03.536 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:03.536 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:11:03.536 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-04-24T23:11:03.536 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.536 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:11:03.536 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:11:03.536 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:11:03.536 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:11:03.537 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:11:03.537 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:11:03.537 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:11:03.537 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:11:03.537 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:11:03.537 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:11:03.537 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:11:03.537 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:11:03.537 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:11:03.537 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:03.537 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.537 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:03.537 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2024-04-24T23:11:03.537 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:11:03.537 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.538 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:03.538 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2024-04-24T23:11:03.538 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:11:03.538 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:03.538 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:03.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.538 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:11:03.538 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:03.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.538 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:03.538 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2024-04-24T23:11:03.538 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:11:03.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.539 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:03.539 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2024-04-24T23:11:03.539 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:11:03.539 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:03.539 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:03.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.539 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:11:03.539 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:03.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.539 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:03.539 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2024-04-24T23:11:03.539 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:11:03.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.539 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:03.540 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2024-04-24T23:11:03.540 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:11:03.540 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:03.540 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:03.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.540 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:11:03.540 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:03.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.540 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:03.540 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2024-04-24T23:11:03.540 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:11:03.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.541 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:03.541 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2024-04-24T23:11:03.541 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:11:03.541 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:03.541 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:03.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.541 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/1479395426", 2024-04-24T23:11:03.541 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/1479395426", 2024-04-24T23:11:03.541 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/1479395426", 2024-04-24T23:11:03.541 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/1479395426", 2024-04-24T23:11:03.541 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:11:03.541 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:11:03.541 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:11:03.541 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:03.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.542 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.542 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:11:03.542 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:11:03.542 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:11:03.542 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:11:03.542 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:11:03.542 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:11:03.542 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:11:03.542 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:11:03.542 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:11:03.542 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:11:03.542 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:11:03.542 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:11:03.542 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:11:03.543 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:03.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.543 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:03.543 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2024-04-24T23:11:03.543 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:11:03.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.543 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:03.543 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2024-04-24T23:11:03.543 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:11:03.543 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:03.543 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:03.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.543 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:11:03.543 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:03.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.544 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:03.544 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2024-04-24T23:11:03.544 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:11:03.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.544 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:03.544 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2024-04-24T23:11:03.544 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:11:03.544 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:03.544 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:03.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.544 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:11:03.544 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:03.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.545 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:03.545 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2024-04-24T23:11:03.545 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:11:03.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.545 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:03.545 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2024-04-24T23:11:03.545 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:11:03.545 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:03.545 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:03.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.545 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:11:03.545 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:03.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.546 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:03.546 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2024-04-24T23:11:03.546 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:11:03.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.546 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:03.546 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2024-04-24T23:11:03.546 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:11:03.546 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:03.546 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:03.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.546 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/1340633718", 2024-04-24T23:11:03.546 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/1340633718", 2024-04-24T23:11:03.546 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/1340633718", 2024-04-24T23:11:03.546 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/1340633718", 2024-04-24T23:11:03.547 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:11:03.547 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:11:03.547 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:11:03.547 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:03.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.547 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:11:03.547 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:11:03.547 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:11:03.547 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:11:03.547 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:11:03.547 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:11:03.547 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:11:03.547 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:11:03.547 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:11:03.548 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:11:03.548 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:11:03.548 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:11:03.548 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:11:03.548 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:03.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.548 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:03.548 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2024-04-24T23:11:03.548 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:11:03.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.548 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:03.548 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2024-04-24T23:11:03.548 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:11:03.548 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:03.549 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:03.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.549 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:11:03.549 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:03.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.549 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:03.549 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2024-04-24T23:11:03.549 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:11:03.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.549 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:03.549 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2024-04-24T23:11:03.549 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:11:03.549 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:03.549 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:03.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.550 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:11:03.550 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:03.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.550 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:03.550 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2024-04-24T23:11:03.550 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:11:03.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.550 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:03.550 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2024-04-24T23:11:03.550 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:11:03.550 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:03.550 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:03.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.551 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:11:03.551 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:03.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.551 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:03.551 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2024-04-24T23:11:03.551 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:11:03.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.551 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:03.551 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2024-04-24T23:11:03.551 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:11:03.551 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:03.551 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:03.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.551 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/3405899334", 2024-04-24T23:11:03.552 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6820/3405899334", 2024-04-24T23:11:03.552 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6828/3405899334", 2024-04-24T23:11:03.552 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6824/3405899334", 2024-04-24T23:11:03.552 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:11:03.552 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:11:03.552 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:11:03.552 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:03.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.552 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:11:03.552 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:11:03.552 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:11:03.552 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:11:03.552 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:11:03.553 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:11:03.553 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:11:03.553 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:11:03.553 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:11:03.553 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 188, 2024-04-24T23:11:03.553 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:11:03.553 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:11:03.553 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:11:03.553 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:03.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.553 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:03.553 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2024-04-24T23:11:03.553 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:11:03.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.554 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:03.554 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2024-04-24T23:11:03.554 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:11:03.554 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:03.554 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:03.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.554 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:11:03.554 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:03.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.554 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:03.554 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2024-04-24T23:11:03.554 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:11:03.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.555 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:03.555 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2024-04-24T23:11:03.555 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:11:03.555 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:03.555 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:03.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.555 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:11:03.555 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:03.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.555 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:03.555 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2024-04-24T23:11:03.555 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:11:03.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.555 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:03.556 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2024-04-24T23:11:03.556 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:11:03.556 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:03.556 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:03.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.556 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:11:03.556 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:03.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.556 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:03.556 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2024-04-24T23:11:03.556 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:11:03.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.556 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:03.556 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2024-04-24T23:11:03.557 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:11:03.557 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:03.557 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:03.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.557 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6821/3381057190", 2024-04-24T23:11:03.557 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6825/3381057190", 2024-04-24T23:11:03.557 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/3381057190", 2024-04-24T23:11:03.557 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/3381057190", 2024-04-24T23:11:03.557 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:11:03.557 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:11:03.557 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:11:03.557 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:03.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.558 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:11:03.558 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:11:03.558 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:11:03.558 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:11:03.558 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:11:03.558 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:11:03.558 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:11:03.558 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:11:03.558 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:11:03.558 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:11:03.558 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:11:03.558 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:11:03.558 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:11:03.558 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:03.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.559 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:03.559 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6800", 2024-04-24T23:11:03.559 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:11:03.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.559 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:03.559 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6801", 2024-04-24T23:11:03.559 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:11:03.559 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:03.559 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:03.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.559 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:11:03.559 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:03.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.559 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:03.560 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6802", 2024-04-24T23:11:03.560 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:11:03.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.560 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:03.560 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6803", 2024-04-24T23:11:03.560 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:11:03.560 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:03.560 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:03.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.560 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:11:03.560 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:03.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.560 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:03.560 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6806", 2024-04-24T23:11:03.561 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:11:03.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.561 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:03.561 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6807", 2024-04-24T23:11:03.561 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:11:03.561 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:03.561 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:03.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.561 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:11:03.561 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:03.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.561 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:03.561 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6804", 2024-04-24T23:11:03.561 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:11:03.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.562 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:03.562 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6805", 2024-04-24T23:11:03.562 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:11:03.562 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:03.562 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:03.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.562 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6801/4010553979", 2024-04-24T23:11:03.562 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6803/4010553979", 2024-04-24T23:11:03.562 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6807/4010553979", 2024-04-24T23:11:03.562 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6805/4010553979", 2024-04-24T23:11:03.562 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:11:03.562 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:11:03.563 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:11:03.563 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:03.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.563 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:11:03.563 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:11:03.563 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:11:03.563 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:11:03.563 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:11:03.563 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:11:03.563 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:11:03.563 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:11:03.563 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:11:03.563 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:11:03.563 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:11:03.564 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:11:03.564 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:11:03.564 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:03.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.564 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:03.564 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6808", 2024-04-24T23:11:03.564 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:11:03.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.564 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:03.564 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6809", 2024-04-24T23:11:03.564 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:11:03.564 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:03.564 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:03.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.565 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:11:03.565 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:03.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.565 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:03.565 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6810", 2024-04-24T23:11:03.565 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:11:03.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.565 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:03.565 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6811", 2024-04-24T23:11:03.565 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:11:03.565 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:03.565 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:03.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.565 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:11:03.566 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:03.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.566 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:03.566 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6814", 2024-04-24T23:11:03.566 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:11:03.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.566 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:03.566 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6815", 2024-04-24T23:11:03.566 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:11:03.566 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:03.566 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:03.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.566 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:11:03.567 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:03.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.567 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:03.567 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6812", 2024-04-24T23:11:03.567 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:11:03.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.567 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:03.567 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6813", 2024-04-24T23:11:03.567 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:11:03.567 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:03.567 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:03.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.567 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6809/3794084091", 2024-04-24T23:11:03.567 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6811/3794084091", 2024-04-24T23:11:03.568 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6815/3794084091", 2024-04-24T23:11:03.568 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6813/3794084091", 2024-04-24T23:11:03.568 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:11:03.568 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:11:03.568 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:11:03.568 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:03.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.568 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:11:03.568 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:11:03.568 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:11:03.568 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:11:03.568 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:11:03.568 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:11:03.568 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:11:03.569 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:11:03.569 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:11:03.569 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:11:03.569 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:11:03.569 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:11:03.569 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:11:03.569 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:03.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.569 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:03.569 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6823", 2024-04-24T23:11:03.569 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:11:03.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.569 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:03.569 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6825", 2024-04-24T23:11:03.570 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:11:03.570 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:03.570 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:03.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.570 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:11:03.570 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:03.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.570 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:03.570 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6826", 2024-04-24T23:11:03.570 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:11:03.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.570 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:03.570 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6827", 2024-04-24T23:11:03.570 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:11:03.571 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:03.571 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:03.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.571 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:11:03.571 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:03.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.571 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:03.571 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6830", 2024-04-24T23:11:03.571 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:11:03.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.571 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:03.571 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6831", 2024-04-24T23:11:03.571 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:11:03.571 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:03.572 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:03.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.572 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:11:03.572 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:03.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.572 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:03.572 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6828", 2024-04-24T23:11:03.572 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:11:03.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.572 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:03.572 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6829", 2024-04-24T23:11:03.572 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:11:03.572 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:03.572 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:03.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.573 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6825/1348640258", 2024-04-24T23:11:03.573 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6827/1348640258", 2024-04-24T23:11:03.573 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6831/1348640258", 2024-04-24T23:11:03.573 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6829/1348640258", 2024-04-24T23:11:03.573 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:11:03.573 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:11:03.573 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:11:03.573 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:03.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.573 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:11:03.573 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:11:03.573 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:11:03.573 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:11:03.574 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:11:03.574 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:11:03.574 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:11:03.574 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:11:03.574 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:11:03.574 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:11:03.574 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:11:03.574 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:11:03.574 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:11:03.574 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:03.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.574 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:03.574 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6816", 2024-04-24T23:11:03.574 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:11:03.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.575 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:03.575 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6817", 2024-04-24T23:11:03.575 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:11:03.575 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:03.575 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:03.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.575 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:11:03.575 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:03.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.575 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:03.575 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6818", 2024-04-24T23:11:03.575 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:11:03.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.576 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:03.576 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6819", 2024-04-24T23:11:03.576 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:11:03.576 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:03.576 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:03.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.576 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:11:03.576 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:03.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.576 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:03.576 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6822", 2024-04-24T23:11:03.576 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:11:03.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.576 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:03.577 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6824", 2024-04-24T23:11:03.577 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:11:03.577 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:03.577 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:03.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.577 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:11:03.577 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:03.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.577 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:03.577 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6820", 2024-04-24T23:11:03.577 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:11:03.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.577 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:03.577 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6821", 2024-04-24T23:11:03.578 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:11:03.578 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:03.578 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:03.578 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.578 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6817/3250005122", 2024-04-24T23:11:03.578 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6819/3250005122", 2024-04-24T23:11:03.578 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6824/3250005122", 2024-04-24T23:11:03.578 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6821/3250005122", 2024-04-24T23:11:03.578 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:11:03.578 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:11:03.578 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:11:03.578 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:03.578 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:03.578 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:11:03.578 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2024-04-24T23:11:03.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.579 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:11:03.579 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:11:03.579 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:11:03.579 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:11:03.579 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:11:03.579 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:11:03.579 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.156184+0000", 2024-04-24T23:11:03.579 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:11:03.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.579 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:11:03.579 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:11:03.579 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:11:03.579 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:11:03.580 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:11:03.580 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:11:03.580 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240774+0000", 2024-04-24T23:11:03.580 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:11:03.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.580 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:11:03.580 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:11:03.580 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:11:03.580 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:11:03.580 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:11:03.580 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:11:03.580 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240949+0000", 2024-04-24T23:11:03.580 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:11:03.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.581 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:11:03.581 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:11:03.581 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:11:03.581 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:11:03.581 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:11:03.581 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:11:03.581 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.171672+0000", 2024-04-24T23:11:03.581 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:11:03.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.581 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:11:03.581 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:11:03.581 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:11:03.582 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:11:03.582 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:11:03.582 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:11:03.582 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.220379+0000", 2024-04-24T23:11:03.582 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:11:03.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.582 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.582 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:11:03.582 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:11:03.582 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:11:03.582 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:11:03.582 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:11:03.582 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:11:03.582 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.204963+0000", 2024-04-24T23:11:03.583 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:11:03.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.583 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:11:03.583 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:11:03.583 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:11:03.583 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:11:03.583 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:11:03.583 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:11:03.583 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.239748+0000", 2024-04-24T23:11:03.583 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:11:03.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.584 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:11:03.584 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:11:03.584 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:11:03.584 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:11:03.584 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:11:03.584 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:11:03.584 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.154104+0000", 2024-04-24T23:11:03.584 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:11:03.584 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:03.584 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:11:03.584 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2024-04-24T23:11:03.584 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [ 2024-04-24T23:11:03.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.585 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "23.6", 2024-04-24T23:11:03.585 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:11:03.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:03.585 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 5, 2024-04-24T23:11:03.585 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 3 2024-04-24T23:11:03.585 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:03.585 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:03.585 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:03.585 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:11:03.585 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_primaries": [], 2024-04-24T23:11:03.585 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2024-04-24T23:11:03.585 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2024-04-24T23:11:03.585 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklist": {}, 2024-04-24T23:11:03.585 INFO:teuthology.orchestra.run.smithi032.stdout: "range_blocklist": {}, 2024-04-24T23:11:03.586 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2024-04-24T23:11:03.586 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2024-04-24T23:11:03.586 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2024-04-24T23:11:03.586 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2024-04-24T23:11:03.586 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2024-04-24T23:11:03.586 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2024-04-24T23:11:03.586 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2024-04-24T23:11:03.586 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:03.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:03.586 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2024-04-24T23:11:03.586 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2024-04-24T23:11:03.586 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2024-04-24T23:11:03.586 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2024-04-24T23:11:03.586 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {}, 2024-04-24T23:11:03.587 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode": { 2024-04-24T23:11:03.587 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_enabled": false, 2024-04-24T23:11:03.587 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_bucket_count": 0, 2024-04-24T23:11:03.587 INFO:teuthology.orchestra.run.smithi032.stdout: "degraded_stretch_mode": 0, 2024-04-24T23:11:03.587 INFO:teuthology.orchestra.run.smithi032.stdout: "recovering_stretch_mode": 0, 2024-04-24T23:11:03.587 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_bucket": 0 2024-04-24T23:11:03.587 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:03.587 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:11:03.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.516+0000 7f7b6b238640 1 -- 172.21.15.32:0/4147960999 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7b4806be30 msgr2=0x7f7b4806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:03.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.516+0000 7f7b6b238640 1 --2- 172.21.15.32:0/4147960999 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7b4806be30 0x7f7b4806e2f0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f7b58009560 tx=0x7f7b58009190 comp rx=0 tx=0).stop 2024-04-24T23:11:03.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.516+0000 7f7b6b238640 1 -- 172.21.15.32:0/4147960999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b6415b380 msgr2=0x7f7b64149360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:03.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.516+0000 7f7b6b238640 1 --2- 172.21.15.32:0/4147960999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b6415b380 0x7f7b64149360 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f7b5000c3c0 tx=0x7f7b5000c880 comp rx=0 tx=0).stop 2024-04-24T23:11:03.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.516+0000 7f7b6b238640 1 -- 172.21.15.32:0/4147960999 shutdown_connections 2024-04-24T23:11:03.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.516+0000 7f7b6b238640 1 --2- 172.21.15.32:0/4147960999 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f7b4806be30 0x7f7b4806e2f0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:03.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.516+0000 7f7b6b238640 1 --2- 172.21.15.32:0/4147960999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b6415b380 0x7f7b64149360 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:03.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.516+0000 7f7b6b238640 1 --2- 172.21.15.32:0/4147960999 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7b6415cb00 0x7f7b64151720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:03.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.516+0000 7f7b6b238640 1 --2- 172.21.15.32:0/4147960999 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7b640ceb40 0x7f7b64159e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:03.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.516+0000 7f7b6b238640 1 -- 172.21.15.32:0/4147960999 >> 172.21.15.32:0/4147960999 conn(0x7f7b6406b110 msgr2=0x7f7b640cf1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:03.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.516+0000 7f7b6b238640 1 -- 172.21.15.32:0/4147960999 shutdown_connections 2024-04-24T23:11:03.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:03.516+0000 7f7b6b238640 1 -- 172.21.15.32:0/4147960999 wait complete. 2024-04-24T23:11:03.593 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-04-24T23:11:08.061 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:11:08.064 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:11:08.594 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-24T23:11:08.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.706+0000 7fc2c2637640 1 Processor -- start 2024-04-24T23:11:08.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.706+0000 7fc2c2637640 1 -- start start 2024-04-24T23:11:08.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.707+0000 7fc2c2637640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc2bc15b380 0x7fc2bc06c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:08.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.707+0000 7fc2c2637640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2bc0cef50 con 0x7fc2bc0ceb40 2024-04-24T23:11:08.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.707+0000 7fc2c2637640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2bc0b4520 con 0x7fc2bc15cb00 2024-04-24T23:11:08.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.707+0000 7fc2c2637640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fc2bc0b4800 con 0x7fc2bc15b380 2024-04-24T23:11:08.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.707+0000 7fc2bbfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc2bc15b380 0x7fc2bc06c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:08.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.707+0000 7fc2bbfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc2bc15b380 0x7fc2bc06c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35586/0 (socket says 172.21.15.32:35586) 2024-04-24T23:11:08.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.707+0000 7fc2c0bd4640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc2bc0ceb40 0x7fc2bc0b55c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35024/0 (socket says 172.21.15.32:35024) 2024-04-24T23:11:08.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.707+0000 7fc2bbfff640 1 -- 172.21.15.32:0/4171648331 learned_addr learned my addr 172.21.15.32:0/4171648331 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:08.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.708+0000 7fc2bbfff640 1 -- 172.21.15.32:0/4171648331 >> v1:172.21.15.121:6789/0 conn(0x7fc2bc15cb00 legacy=0x7fc2bc0ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:08.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.708+0000 7fc2bbfff640 1 -- 172.21.15.32:0/4171648331 >> v1:172.21.15.32:6789/0 conn(0x7fc2bc0ceb40 legacy=0x7fc2bc0b55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:08.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.708+0000 7fc2baffd640 1 -- 172.21.15.32:0/4171648331 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3909056191 0 0) 0x7fc2b4001490 con 0x7fc2bc0ceb40 2024-04-24T23:11:08.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.708+0000 7fc2bbfff640 1 -- 172.21.15.32:0/4171648331 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2bc0b4ae0 con 0x7fc2bc15b380 2024-04-24T23:11:08.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.708+0000 7fc2bbfff640 1 --2- 172.21.15.32:0/4171648331 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2bc15b380 0x7fc2bc06c5f0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fc2b000bb10 tx=0x7fc2b002c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12a12dccdd077905 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:08.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.709+0000 7fc2baffd640 1 -- 172.21.15.32:0/4171648331 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b0032710 con 0x7fc2bc15b380 2024-04-24T23:11:08.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.709+0000 7fc2baffd640 1 -- 172.21.15.32:0/4171648331 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2b00328b0 con 0x7fc2bc15b380 2024-04-24T23:11:08.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.709+0000 7fc2baffd640 1 -- 172.21.15.32:0/4171648331 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b0032ca0 con 0x7fc2bc15b380 2024-04-24T23:11:08.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.709+0000 7fc2c2637640 1 -- 172.21.15.32:0/4171648331 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2bc15b380 msgr2=0x7fc2bc06c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:08.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.709+0000 7fc2c2637640 1 --2- 172.21.15.32:0/4171648331 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2bc15b380 0x7fc2bc06c5f0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fc2b000bb10 tx=0x7fc2b002c850 comp rx=0 tx=0).stop 2024-04-24T23:11:08.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.709+0000 7fc2c2637640 1 -- 172.21.15.32:0/4171648331 shutdown_connections 2024-04-24T23:11:08.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.709+0000 7fc2c2637640 1 --2- 172.21.15.32:0/4171648331 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2bc15b380 0x7fc2bc06c5f0 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:08.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.709+0000 7fc2c2637640 1 -- 172.21.15.32:0/4171648331 >> 172.21.15.32:0/4171648331 conn(0x7fc2bc06b110 msgr2=0x7fc2bc06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:08.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.709+0000 7fc2c2637640 1 -- 172.21.15.32:0/4171648331 shutdown_connections 2024-04-24T23:11:08.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.709+0000 7fc2c2637640 1 -- 172.21.15.32:0/4171648331 wait complete. 2024-04-24T23:11:08.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.710+0000 7fc2c2637640 1 Processor -- start 2024-04-24T23:11:08.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.711+0000 7fc2c2637640 1 -- start start 2024-04-24T23:11:08.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.711+0000 7fc2c2637640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2bc0ceb40 0x7fc2bc147e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:08.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.711+0000 7fc2c2637640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc2bc15b380 0x7fc2bc15d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:08.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.711+0000 7fc2c2637640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc2bc15cb00 0x7fc2bc15dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:08.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.711+0000 7fc2c2637640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc2bc15b760 con 0x7fc2bc0ceb40 2024-04-24T23:11:08.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.711+0000 7fc2c2637640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc2bc0b50a0 con 0x7fc2bc15b380 2024-04-24T23:11:08.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.711+0000 7fc2c2637640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc2bc06cb30 con 0x7fc2bc15cb00 2024-04-24T23:11:08.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.711+0000 7fc2bbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2bc0ceb40 0x7fc2bc147e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:08.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.712+0000 7fc2bbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2bc0ceb40 0x7fc2bc147e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35602/0 (socket says 172.21.15.32:35602) 2024-04-24T23:11:08.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.712+0000 7fc2bbfff640 1 -- 172.21.15.32:0/1971323590 learned_addr learned my addr 172.21.15.32:0/1971323590 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:08.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.712+0000 7fc2bb7fe640 1 --2- 172.21.15.32:0/1971323590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc2bc15b380 0x7fc2bc15d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:08.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.712+0000 7fc2c0bd4640 1 --2- 172.21.15.32:0/1971323590 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc2bc15cb00 0x7fc2bc15dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:08.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.712+0000 7fc2bbfff640 1 -- 172.21.15.32:0/1971323590 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc2bc15cb00 msgr2=0x7fc2bc15dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:08.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.712+0000 7fc2bbfff640 1 --2- 172.21.15.32:0/1971323590 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc2bc15cb00 0x7fc2bc15dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:08.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.712+0000 7fc2bbfff640 1 -- 172.21.15.32:0/1971323590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc2bc15b380 msgr2=0x7fc2bc15d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:08.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.712+0000 7fc2bbfff640 1 --2- 172.21.15.32:0/1971323590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc2bc15b380 0x7fc2bc15d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:08.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.712+0000 7fc2bbfff640 1 -- 172.21.15.32:0/1971323590 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2bc154870 con 0x7fc2bc0ceb40 2024-04-24T23:11:08.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.712+0000 7fc2c0bd4640 1 --2- 172.21.15.32:0/1971323590 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc2bc15cb00 0x7fc2bc15dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:11:08.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.712+0000 7fc2bbfff640 1 --2- 172.21.15.32:0/1971323590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2bc0ceb40 0x7fc2bc147e70 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fc2b000b2c0 tx=0x7fc2b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:08.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.712+0000 7fc2b8ff9640 1 -- 172.21.15.32:0/1971323590 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b0041020 con 0x7fc2bc0ceb40 2024-04-24T23:11:08.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.712+0000 7fc2b8ff9640 1 -- 172.21.15.32:0/1971323590 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2b0003850 con 0x7fc2bc0ceb40 2024-04-24T23:11:08.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.712+0000 7fc2c2637640 1 -- 172.21.15.32:0/1971323590 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2bc1e4e30 con 0x7fc2bc0ceb40 2024-04-24T23:11:08.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.712+0000 7fc2b8ff9640 1 -- 172.21.15.32:0/1971323590 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b0003c40 con 0x7fc2bc0ceb40 2024-04-24T23:11:08.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.712+0000 7fc2c2637640 1 -- 172.21.15.32:0/1971323590 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2bc154b50 con 0x7fc2bc0ceb40 2024-04-24T23:11:08.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.713+0000 7fc2c2637640 1 -- 172.21.15.32:0/1971323590 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2bc068430 con 0x7fc2bc0ceb40 2024-04-24T23:11:08.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.714+0000 7fc2b8ff9640 1 -- 172.21.15.32:0/1971323590 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc2b0044020 con 0x7fc2bc0ceb40 2024-04-24T23:11:08.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.714+0000 7fc2b8ff9640 1 --2- 172.21.15.32:0/1971323590 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc2a806be30 0x7fc2a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:08.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.714+0000 7fc2b8ff9640 1 -- 172.21.15.32:0/1971323590 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(190..190 src has 1..190) ==== 5738+0+0 (secure 0 0 0) 0x7fc2b00aba50 con 0x7fc2bc0ceb40 2024-04-24T23:11:08.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.714+0000 7fc2bb7fe640 1 --2- 172.21.15.32:0/1971323590 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc2a806be30 0x7fc2a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:08.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.715+0000 7fc2bb7fe640 1 --2- 172.21.15.32:0/1971323590 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc2a806be30 0x7fc2a806e2f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac006fd0 tx=0x7fc2ac008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:08.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.719+0000 7fc2b8ff9640 1 -- 172.21.15.32:0/1971323590 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc2b004b030 con 0x7fc2bc0ceb40 2024-04-24T23:11:08.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.902+0000 7fc2c2637640 1 -- 172.21.15.32:0/1971323590 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7fc2bc0afd60 con 0x7fc2bc0ceb40 2024-04-24T23:11:08.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.904+0000 7fc2b8ff9640 1 -- 172.21.15.32:0/1971323590 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v190) ==== 81+0+34379 (secure 0 0 0) 0x7fc2b007a3e0 con 0x7fc2bc0ceb40 2024-04-24T23:11:08.906 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:11:08.906 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:11:08.906 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 190, 2024-04-24T23:11:08.906 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", 2024-04-24T23:11:08.906 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2024-04-24T22:58:46.964851+0000", 2024-04-24T23:11:08.906 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2024-04-24T23:11:06.861172+0000", 2024-04-24T23:11:08.906 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2024-04-24T22:58:52.048787+0000", 2024-04-24T23:11:08.906 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2024-04-24T22:58:49.045609+0000", 2024-04-24T23:11:08.907 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-24T23:11:08.907 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2024-04-24T23:11:08.907 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2024-04-24T23:11:08.907 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2024-04-24T23:11:08.907 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2024-04-24T23:11:08.907 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2024-04-24T23:11:08.907 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2024-04-24T23:11:08.907 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:11:08.907 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 4, 2024-04-24T23:11:08.907 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.69999998807907104, 2024-04-24T23:11:08.907 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-24T23:11:08.907 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-24T23:11:08.907 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2024-04-24T23:11:08.907 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 23, 2024-04-24T23:11:08.907 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2024-04-24T23:11:08.908 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "luminous", 2024-04-24T23:11:08.908 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "luminous", 2024-04-24T23:11:08.908 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "squid", 2024-04-24T23:11:08.908 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_crimson": false, 2024-04-24T23:11:08.908 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2024-04-24T23:11:08.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.908 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2024-04-24T23:11:08.908 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": ".mgr", 2024-04-24T23:11:08.908 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T22:58:53.171408+0000", 2024-04-24T23:11:08.908 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:11:08.908 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:11:08.908 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:11:08.908 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:11:08.908 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:11:08.908 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:11:08.909 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:11:08.909 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:11:08.909 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:11:08.909 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:11:08.909 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:11:08.909 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:11:08.909 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2024-04-24T23:11:08.909 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2024-04-24T23:11:08.909 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2024-04-24T23:11:08.909 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2024-04-24T23:11:08.909 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2024-04-24T23:11:08.909 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:11:08.909 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:11:08.909 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:11:08.909 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:11:08.910 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:11:08.910 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:11:08.910 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:11:08.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.910 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "113", 2024-04-24T23:11:08.910 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:11:08.910 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:11:08.910 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:11:08.910 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:11:08.910 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:11:08.910 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:11:08.910 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:11:08.910 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:11:08.910 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:11:08.911 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:11:08.911 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:11:08.911 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:11:08.911 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:11:08.911 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:11:08.911 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:11:08.911 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:11:08.911 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:11:08.911 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:11:08.911 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:11:08.911 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:11:08.911 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:11:08.911 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:11:08.911 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:11:08.911 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:11:08.912 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:11:08.912 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:11:08.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.912 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:11:08.912 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:11:08.912 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:11:08.912 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:11:08.912 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:11:08.912 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:11:08.912 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:11:08.912 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:11:08.912 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:11:08.912 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:11:08.912 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:11:08.912 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:11:08.913 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_max": 32, 2024-04-24T23:11:08.913 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2024-04-24T23:11:08.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.913 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:11:08.913 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": {} 2024-04-24T23:11:08.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.913 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:11:08.913 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:11:08.913 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 8, 2024-04-24T23:11:08.913 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 8, 2024-04-24T23:11:08.913 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.25, 2024-04-24T23:11:08.913 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-04-24T23:11:08.913 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-04-24T23:11:08.913 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:11:08.913 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:11:08.914 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:11:08.914 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:08.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.914 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 22, 2024-04-24T23:11:08.914 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2024-04-24T23:11:08.914 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:10:34.649562+0000", 2024-04-24T23:11:08.914 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:11:08.914 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:11:08.914 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:11:08.914 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:11:08.914 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:11:08.914 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:11:08.914 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:11:08.915 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:11:08.915 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:11:08.915 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:11:08.915 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:11:08.915 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:11:08.915 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 32, 2024-04-24T23:11:08.915 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 32, 2024-04-24T23:11:08.915 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 32, 2024-04-24T23:11:08.915 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 32, 2024-04-24T23:11:08.915 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 32, 2024-04-24T23:11:08.915 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:11:08.915 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:11:08.915 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:11:08.915 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:11:08.915 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:11:08.915 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:11:08.916 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:11:08.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.916 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "184", 2024-04-24T23:11:08.916 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:11:08.916 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:11:08.916 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:11:08.916 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:11:08.916 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:11:08.916 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:11:08.916 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:11:08.916 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:11:08.916 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:11:08.916 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:11:08.916 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:11:08.916 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:11:08.916 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:11:08.917 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:11:08.917 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:11:08.917 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:11:08.917 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:11:08.917 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:11:08.917 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:11:08.917 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:11:08.917 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:11:08.917 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:11:08.917 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:11:08.917 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:11:08.917 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:11:08.917 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:11:08.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.917 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:11:08.917 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:11:08.918 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:11:08.918 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:11:08.918 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:11:08.918 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:11:08.918 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:11:08.918 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:11:08.918 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:11:08.918 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:11:08.918 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:11:08.918 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:11:08.918 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2024-04-24T23:11:08.918 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2024-04-24T23:11:08.918 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2024-04-24T23:11:08.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.919 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:11:08.919 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:11:08.919 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2024-04-24T23:11:08.919 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:08.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.919 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:11:08.919 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:11:08.919 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.5, 2024-04-24T23:11:08.919 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.5, 2024-04-24T23:11:08.919 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:11:08.919 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.5, 2024-04-24T23:11:08.919 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.5, 2024-04-24T23:11:08.919 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:11:08.919 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:11:08.920 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:11:08.920 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:08.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.920 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 23, 2024-04-24T23:11:08.920 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2024-04-24T23:11:08.920 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:10:35.339604+0000", 2024-04-24T23:11:08.920 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:11:08.920 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:11:08.920 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:11:08.921 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:11:08.921 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:11:08.921 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:11:08.921 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:11:08.921 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:11:08.921 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:11:08.921 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:11:08.921 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:11:08.921 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:11:08.921 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 64, 2024-04-24T23:11:08.921 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 64, 2024-04-24T23:11:08.921 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 64, 2024-04-24T23:11:08.921 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 64, 2024-04-24T23:11:08.921 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 64, 2024-04-24T23:11:08.921 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:11:08.921 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:11:08.922 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:11:08.922 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:11:08.922 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:11:08.922 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:11:08.922 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:11:08.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.922 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "190", 2024-04-24T23:11:08.922 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:11:08.922 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:11:08.925 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:11:08.925 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:11:08.925 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:11:08.925 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:11:08.926 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:11:08.926 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:11:08.926 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:11:08.926 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 33554432, 2024-04-24T23:11:08.926 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:11:08.926 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:11:08.926 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:11:08.926 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:11:08.926 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:11:08.926 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:11:08.926 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:11:08.926 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:11:08.926 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:11:08.926 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:11:08.927 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:11:08.927 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:11:08.927 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:11:08.927 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:11:08.927 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:11:08.927 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:11:08.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.927 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:11:08.927 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:11:08.927 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:11:08.927 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:11:08.927 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:11:08.927 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:11:08.927 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:11:08.928 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:11:08.928 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:11:08.928 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:11:08.928 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:11:08.928 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:11:08.928 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 64, 2024-04-24T23:11:08.928 INFO:teuthology.orchestra.run.smithi032.stdout: "target_size_ratio": 0.89999997615814209 2024-04-24T23:11:08.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.928 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:11:08.928 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:11:08.928 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2024-04-24T23:11:08.928 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:08.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.928 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:11:08.928 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:11:08.928 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.3799999952316284, 2024-04-24T23:11:08.929 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.3799999952316284, 2024-04-24T23:11:08.929 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:11:08.929 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.3799999952316284, 2024-04-24T23:11:08.929 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.3799999952316284, 2024-04-24T23:11:08.929 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:11:08.929 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:11:08.929 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:11:08.929 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:08.929 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:08.929 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:11:08.929 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-04-24T23:11:08.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.929 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:11:08.929 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:11:08.929 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:11:08.930 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:11:08.930 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:11:08.930 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:11:08.930 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:11:08.930 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:11:08.930 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:11:08.930 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:11:08.930 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:11:08.930 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:11:08.930 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:11:08.930 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:08.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.930 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:08.930 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2024-04-24T23:11:08.931 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:11:08.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.931 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:08.931 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2024-04-24T23:11:08.931 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:11:08.931 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:08.931 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:08.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.931 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:11:08.931 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:08.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.931 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:08.931 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2024-04-24T23:11:08.931 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:11:08.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.932 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:08.932 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2024-04-24T23:11:08.932 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:11:08.932 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:08.932 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:08.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.932 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:11:08.932 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:08.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.932 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:08.932 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2024-04-24T23:11:08.932 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:11:08.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.933 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:08.933 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2024-04-24T23:11:08.933 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:11:08.933 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:08.933 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:08.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.933 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:11:08.933 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:08.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.933 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:08.933 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2024-04-24T23:11:08.933 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:11:08.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.933 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:08.934 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2024-04-24T23:11:08.934 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:11:08.934 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:08.934 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:08.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.934 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/1479395426", 2024-04-24T23:11:08.934 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/1479395426", 2024-04-24T23:11:08.934 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/1479395426", 2024-04-24T23:11:08.934 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/1479395426", 2024-04-24T23:11:08.934 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:11:08.934 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:11:08.934 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:11:08.934 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:08.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.935 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:11:08.935 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:11:08.935 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:11:08.935 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:11:08.935 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:11:08.935 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:11:08.935 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:11:08.935 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:11:08.935 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:11:08.935 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:11:08.935 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:11:08.935 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:11:08.935 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:11:08.935 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:08.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.936 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:08.936 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2024-04-24T23:11:08.936 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:11:08.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.936 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:08.936 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2024-04-24T23:11:08.936 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:11:08.936 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:08.936 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:08.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.936 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:11:08.936 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:08.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.936 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:08.937 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2024-04-24T23:11:08.937 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:11:08.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.937 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:08.937 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2024-04-24T23:11:08.937 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:11:08.937 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:08.937 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:08.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.937 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:11:08.937 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:08.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.937 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:08.937 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2024-04-24T23:11:08.938 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:11:08.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.938 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:08.938 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2024-04-24T23:11:08.938 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:11:08.938 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:08.938 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:08.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.938 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:11:08.938 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:08.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.938 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:08.938 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2024-04-24T23:11:08.939 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:11:08.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.939 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:08.939 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2024-04-24T23:11:08.939 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:11:08.939 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:08.939 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:08.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.939 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/1340633718", 2024-04-24T23:11:08.939 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/1340633718", 2024-04-24T23:11:08.939 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/1340633718", 2024-04-24T23:11:08.939 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/1340633718", 2024-04-24T23:11:08.939 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:11:08.939 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:11:08.940 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:11:08.940 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:08.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.940 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:11:08.940 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:11:08.940 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:11:08.940 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:11:08.940 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:11:08.940 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:11:08.940 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:11:08.940 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:11:08.940 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:11:08.940 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:11:08.940 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:11:08.940 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:11:08.941 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:11:08.941 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:08.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.941 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:08.941 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2024-04-24T23:11:08.941 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:11:08.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.941 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:08.941 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2024-04-24T23:11:08.941 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:11:08.941 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:08.941 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:08.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.941 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:11:08.942 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:08.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.942 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:08.942 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2024-04-24T23:11:08.942 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:11:08.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.942 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:08.942 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2024-04-24T23:11:08.942 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:11:08.942 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:08.942 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:08.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.942 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:11:08.943 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:08.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.943 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:08.943 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2024-04-24T23:11:08.943 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:11:08.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.943 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:08.943 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2024-04-24T23:11:08.943 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:11:08.943 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:08.943 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:08.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.943 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:11:08.943 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:08.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.944 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:08.944 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2024-04-24T23:11:08.944 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:11:08.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.944 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:08.944 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2024-04-24T23:11:08.944 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:11:08.944 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:08.944 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:08.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.944 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/3405899334", 2024-04-24T23:11:08.944 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6820/3405899334", 2024-04-24T23:11:08.944 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6828/3405899334", 2024-04-24T23:11:08.945 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6824/3405899334", 2024-04-24T23:11:08.945 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:11:08.945 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:11:08.945 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:11:08.945 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:08.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.945 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:11:08.945 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:11:08.945 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:11:08.945 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:11:08.945 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:11:08.945 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:11:08.945 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:11:08.945 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:11:08.946 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:11:08.946 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 188, 2024-04-24T23:11:08.946 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:11:08.946 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:11:08.946 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:11:08.946 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:08.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.946 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:08.946 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2024-04-24T23:11:08.946 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:11:08.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.946 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:08.946 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2024-04-24T23:11:08.946 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:11:08.947 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:08.947 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:08.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.947 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:11:08.947 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:08.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.947 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:08.947 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2024-04-24T23:11:08.947 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:11:08.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.947 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:08.947 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2024-04-24T23:11:08.947 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:11:08.948 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:08.948 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:08.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.948 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:11:08.948 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:08.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.948 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:08.948 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2024-04-24T23:11:08.948 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:11:08.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.948 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:08.948 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2024-04-24T23:11:08.948 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:11:08.948 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:08.948 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:08.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.949 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:11:08.949 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:08.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.949 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:08.949 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2024-04-24T23:11:08.949 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:11:08.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.949 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:08.949 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2024-04-24T23:11:08.949 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:11:08.949 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:08.949 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:08.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.949 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6821/3381057190", 2024-04-24T23:11:08.950 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6825/3381057190", 2024-04-24T23:11:08.950 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/3381057190", 2024-04-24T23:11:08.950 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/3381057190", 2024-04-24T23:11:08.950 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:11:08.950 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:11:08.950 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:11:08.950 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:08.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.950 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:11:08.950 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:11:08.950 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:11:08.950 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:11:08.950 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:11:08.951 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:11:08.951 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:11:08.951 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:11:08.951 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:11:08.951 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:11:08.951 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:11:08.951 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:11:08.951 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:11:08.951 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:08.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.951 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:08.951 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6800", 2024-04-24T23:11:08.951 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:11:08.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.952 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:08.952 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6801", 2024-04-24T23:11:08.952 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:11:08.952 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:08.952 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:08.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.952 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:11:08.952 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:08.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.952 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:08.952 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6802", 2024-04-24T23:11:08.953 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:11:08.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.953 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:08.953 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6803", 2024-04-24T23:11:08.953 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:11:08.953 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:08.953 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:08.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.953 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:11:08.953 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:08.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.953 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:08.953 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6806", 2024-04-24T23:11:08.954 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:11:08.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.954 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:08.954 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6807", 2024-04-24T23:11:08.954 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:11:08.954 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:08.954 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:08.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.954 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:11:08.954 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:08.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.954 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:08.954 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6804", 2024-04-24T23:11:08.955 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:11:08.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.955 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:08.955 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6805", 2024-04-24T23:11:08.955 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:11:08.955 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:08.955 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:08.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.955 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6801/4010553979", 2024-04-24T23:11:08.955 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6803/4010553979", 2024-04-24T23:11:08.955 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6807/4010553979", 2024-04-24T23:11:08.955 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6805/4010553979", 2024-04-24T23:11:08.955 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:11:08.956 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:11:08.956 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:11:08.956 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:08.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.956 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:11:08.956 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:11:08.956 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:11:08.956 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:11:08.956 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:11:08.956 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:11:08.956 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:11:08.956 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:11:08.956 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:11:08.957 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:11:08.957 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:11:08.957 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:11:08.957 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:11:08.957 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:08.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.957 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:08.957 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6808", 2024-04-24T23:11:08.957 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:11:08.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.957 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:08.957 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6809", 2024-04-24T23:11:08.957 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:11:08.957 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:08.958 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:08.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.958 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:11:08.958 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:08.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.958 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:08.958 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6810", 2024-04-24T23:11:08.958 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:11:08.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.958 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:08.958 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6811", 2024-04-24T23:11:08.958 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:11:08.958 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:08.959 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:08.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.959 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:11:08.959 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:08.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.959 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:08.959 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6814", 2024-04-24T23:11:08.959 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:11:08.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.959 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:08.959 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6815", 2024-04-24T23:11:08.959 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:11:08.959 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:08.960 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:08.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.960 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:11:08.960 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:08.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.960 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:08.960 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6812", 2024-04-24T23:11:08.960 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:11:08.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.960 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:08.960 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6813", 2024-04-24T23:11:08.960 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:11:08.960 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:08.961 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:08.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.961 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6809/3794084091", 2024-04-24T23:11:08.961 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6811/3794084091", 2024-04-24T23:11:08.961 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6815/3794084091", 2024-04-24T23:11:08.961 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6813/3794084091", 2024-04-24T23:11:08.961 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:11:08.961 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:11:08.961 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:11:08.961 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:08.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.961 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:11:08.961 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:11:08.962 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:11:08.962 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:11:08.962 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:11:08.962 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:11:08.962 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:11:08.962 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:11:08.962 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:11:08.962 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:11:08.962 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:11:08.962 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:11:08.962 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:11:08.962 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:08.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.962 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:08.962 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6823", 2024-04-24T23:11:08.963 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:11:08.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.963 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:08.963 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6825", 2024-04-24T23:11:08.963 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:11:08.963 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:08.963 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:08.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.963 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:11:08.963 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:08.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.963 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:08.963 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6826", 2024-04-24T23:11:08.963 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:11:08.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.964 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:08.964 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6827", 2024-04-24T23:11:08.964 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:11:08.964 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:08.964 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:08.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.964 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:11:08.964 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:08.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.964 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:08.964 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6830", 2024-04-24T23:11:08.964 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:11:08.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.965 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:08.965 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6831", 2024-04-24T23:11:08.965 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:11:08.965 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:08.965 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:08.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.965 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:11:08.965 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:08.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.965 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:08.965 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6828", 2024-04-24T23:11:08.965 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:11:08.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.966 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:08.966 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6829", 2024-04-24T23:11:08.966 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:11:08.966 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:08.966 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:08.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.966 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6825/1348640258", 2024-04-24T23:11:08.966 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6827/1348640258", 2024-04-24T23:11:08.966 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6831/1348640258", 2024-04-24T23:11:08.966 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6829/1348640258", 2024-04-24T23:11:08.966 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:11:08.966 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:11:08.966 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:11:08.966 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:08.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.967 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.967 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:11:08.967 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:11:08.967 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:11:08.967 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:11:08.967 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:11:08.967 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:11:08.967 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:11:08.967 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:11:08.967 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:11:08.967 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:11:08.967 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:11:08.967 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:11:08.967 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:11:08.967 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:08.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.968 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:08.968 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6816", 2024-04-24T23:11:08.968 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:11:08.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.968 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:08.968 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6817", 2024-04-24T23:11:08.968 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:11:08.968 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:08.968 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:08.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.968 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:11:08.968 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:08.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.968 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:08.969 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6818", 2024-04-24T23:11:08.969 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:11:08.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.969 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:08.969 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6819", 2024-04-24T23:11:08.969 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:11:08.969 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:08.969 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:08.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.969 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:11:08.969 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:08.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.969 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:08.969 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6822", 2024-04-24T23:11:08.970 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:11:08.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.970 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:08.970 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6824", 2024-04-24T23:11:08.970 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:11:08.970 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:08.970 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:08.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.970 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:11:08.970 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:08.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.970 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:08.970 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6820", 2024-04-24T23:11:08.971 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:11:08.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.971 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.971 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:08.971 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6821", 2024-04-24T23:11:08.971 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:11:08.971 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:08.971 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:08.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.971 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6817/3250005122", 2024-04-24T23:11:08.971 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6819/3250005122", 2024-04-24T23:11:08.971 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6824/3250005122", 2024-04-24T23:11:08.971 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6821/3250005122", 2024-04-24T23:11:08.971 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:11:08.971 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:11:08.972 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:11:08.972 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:08.972 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:08.972 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:11:08.972 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2024-04-24T23:11:08.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.972 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:11:08.972 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:11:08.972 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:11:08.972 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:11:08.972 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:11:08.972 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:11:08.972 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.156184+0000", 2024-04-24T23:11:08.972 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:11:08.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.973 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:11:08.973 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:11:08.973 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:11:08.973 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:11:08.973 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:11:08.973 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:11:08.973 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240774+0000", 2024-04-24T23:11:08.973 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:11:08.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.973 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:11:08.973 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:11:08.973 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:11:08.973 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:11:08.973 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:11:08.973 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:11:08.974 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240949+0000", 2024-04-24T23:11:08.974 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:11:08.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.974 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:11:08.974 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:11:08.974 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:11:08.974 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:11:08.974 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:11:08.974 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:11:08.974 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.171672+0000", 2024-04-24T23:11:08.974 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:11:08.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.974 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:11:08.975 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:11:08.975 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:11:08.975 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:11:08.975 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:11:08.975 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:11:08.975 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.220379+0000", 2024-04-24T23:11:08.975 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:11:08.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.975 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:11:08.975 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:11:08.975 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:11:08.975 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:11:08.975 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:11:08.975 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:11:08.976 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.204963+0000", 2024-04-24T23:11:08.976 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:11:08.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.976 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:11:08.976 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:11:08.976 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:11:08.976 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:11:08.976 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:11:08.976 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:11:08.976 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.239748+0000", 2024-04-24T23:11:08.976 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:11:08.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.977 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:11:08.977 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:11:08.977 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:11:08.977 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:11:08.977 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:11:08.977 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:11:08.977 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.154104+0000", 2024-04-24T23:11:08.977 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:11:08.977 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:08.977 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:11:08.977 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2024-04-24T23:11:08.977 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [ 2024-04-24T23:11:08.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.977 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "23.6", 2024-04-24T23:11:08.977 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:11:08.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:08.978 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 5, 2024-04-24T23:11:08.978 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 3 2024-04-24T23:11:08.978 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:08.978 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:08.978 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:08.978 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:11:08.978 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_primaries": [], 2024-04-24T23:11:08.978 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2024-04-24T23:11:08.978 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2024-04-24T23:11:08.978 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklist": {}, 2024-04-24T23:11:08.978 INFO:teuthology.orchestra.run.smithi032.stdout: "range_blocklist": {}, 2024-04-24T23:11:08.978 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2024-04-24T23:11:08.978 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2024-04-24T23:11:08.978 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2024-04-24T23:11:08.979 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2024-04-24T23:11:08.979 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2024-04-24T23:11:08.979 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2024-04-24T23:11:08.979 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2024-04-24T23:11:08.979 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:08.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:08.979 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2024-04-24T23:11:08.979 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2024-04-24T23:11:08.979 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2024-04-24T23:11:08.979 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2024-04-24T23:11:08.979 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {}, 2024-04-24T23:11:08.979 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode": { 2024-04-24T23:11:08.979 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_enabled": false, 2024-04-24T23:11:08.979 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_bucket_count": 0, 2024-04-24T23:11:08.979 INFO:teuthology.orchestra.run.smithi032.stdout: "degraded_stretch_mode": 0, 2024-04-24T23:11:08.980 INFO:teuthology.orchestra.run.smithi032.stdout: "recovering_stretch_mode": 0, 2024-04-24T23:11:08.980 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_bucket": 0 2024-04-24T23:11:08.980 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:08.980 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:11:08.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.906+0000 7fc2c2637640 1 -- 172.21.15.32:0/1971323590 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc2a806be30 msgr2=0x7fc2a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:08.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.906+0000 7fc2c2637640 1 --2- 172.21.15.32:0/1971323590 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc2a806be30 0x7fc2a806e2f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac006fd0 tx=0x7fc2ac008040 comp rx=0 tx=0).stop 2024-04-24T23:11:08.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.906+0000 7fc2c2637640 1 -- 172.21.15.32:0/1971323590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2bc0ceb40 msgr2=0x7fc2bc147e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:08.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.906+0000 7fc2c2637640 1 --2- 172.21.15.32:0/1971323590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2bc0ceb40 0x7fc2bc147e70 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fc2b000b2c0 tx=0x7fc2b00046c0 comp rx=0 tx=0).stop 2024-04-24T23:11:08.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.906+0000 7fc2c2637640 1 -- 172.21.15.32:0/1971323590 shutdown_connections 2024-04-24T23:11:08.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.906+0000 7fc2c2637640 1 --2- 172.21.15.32:0/1971323590 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc2a806be30 0x7fc2a806e2f0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:08.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.906+0000 7fc2c2637640 1 --2- 172.21.15.32:0/1971323590 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc2bc15cb00 0x7fc2bc15dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:08.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.906+0000 7fc2c2637640 1 --2- 172.21.15.32:0/1971323590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc2bc15b380 0x7fc2bc15d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:08.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.906+0000 7fc2c2637640 1 --2- 172.21.15.32:0/1971323590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2bc0ceb40 0x7fc2bc147e70 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:08.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.906+0000 7fc2c2637640 1 -- 172.21.15.32:0/1971323590 >> 172.21.15.32:0/1971323590 conn(0x7fc2bc06b110 msgr2=0x7fc2bc0b5fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:08.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.906+0000 7fc2c2637640 1 -- 172.21.15.32:0/1971323590 shutdown_connections 2024-04-24T23:11:08.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:08.906+0000 7fc2c2637640 1 -- 172.21.15.32:0/1971323590 wait complete. 2024-04-24T23:11:08.983 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2024-04-24T23:11:08.983 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-24T23:11:09.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.084+0000 7f934062a640 1 Processor -- start 2024-04-24T23:11:09.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.084+0000 7f934062a640 1 -- start start 2024-04-24T23:11:09.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.084+0000 7f934062a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9338162320 0x7f93380ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:09.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.085+0000 7f934062a640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93380cd270 con 0x7f9338162e10 2024-04-24T23:11:09.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.085+0000 7f934062a640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9338071610 con 0x7f93380b5af0 2024-04-24T23:11:09.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.085+0000 7f934062a640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f93380b5660 con 0x7f9338162320 2024-04-24T23:11:09.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.085+0000 7f933e3c6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9338162320 0x7f93380ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:09.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.085+0000 7f933ebc7640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9338162e10 0x7f933815ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35036/0 (socket says 172.21.15.32:35036) 2024-04-24T23:11:09.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.085+0000 7f933e3c6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9338162320 0x7f93380ccc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35616/0 (socket says 172.21.15.32:35616) 2024-04-24T23:11:09.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.085+0000 7f933ebc7640 1 -- 172.21.15.32:0/2746208361 learned_addr learned my addr 172.21.15.32:0/2746208361 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:11:09.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.085+0000 7f933e3c6640 1 -- 172.21.15.32:0/2746208361 >> v1:172.21.15.121:6789/0 conn(0x7f93380b5af0 legacy=0x7f9338162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:09.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.085+0000 7f933e3c6640 1 -- 172.21.15.32:0/2746208361 >> v1:172.21.15.32:6789/0 conn(0x7f9338162e10 legacy=0x7f933815ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:09.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.085+0000 7f933e3c6640 1 -- 172.21.15.32:0/2746208361 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93380718f0 con 0x7f9338162320 2024-04-24T23:11:09.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.085+0000 7f933d3c4640 1 -- 172.21.15.32:0/2746208361 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3681152779 0 0) 0x7f93380cd270 con 0x7f9338162e10 2024-04-24T23:11:09.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.086+0000 7f933e3c6640 1 --2- 172.21.15.32:0/2746208361 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9338162320 0x7f93380ccc90 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f933806c5b0 tx=0x7f933002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5804b67edac11e98 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:09.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.086+0000 7f933d3c4640 1 -- 172.21.15.32:0/2746208361 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9330037070 con 0x7f9338162320 2024-04-24T23:11:09.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.086+0000 7f933d3c4640 1 -- 172.21.15.32:0/2746208361 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f933002ae20 con 0x7f9338162320 2024-04-24T23:11:09.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.086+0000 7f933d3c4640 1 -- 172.21.15.32:0/2746208361 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9330032660 con 0x7f9338162320 2024-04-24T23:11:09.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.086+0000 7f934062a640 1 -- 172.21.15.32:0/2746208361 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9338162320 msgr2=0x7f93380ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:09.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.086+0000 7f934062a640 1 --2- 172.21.15.32:0/2746208361 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9338162320 0x7f93380ccc90 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f933806c5b0 tx=0x7f933002a110 comp rx=0 tx=0).stop 2024-04-24T23:11:09.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.086+0000 7f934062a640 1 -- 172.21.15.32:0/2746208361 shutdown_connections 2024-04-24T23:11:09.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.086+0000 7f934062a640 1 --2- 172.21.15.32:0/2746208361 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9338162320 0x7f93380ccc90 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:09.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.086+0000 7f934062a640 1 -- 172.21.15.32:0/2746208361 >> 172.21.15.32:0/2746208361 conn(0x7f93380d0ae0 msgr2=0x7f93380d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:09.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.087+0000 7f934062a640 1 -- 172.21.15.32:0/2746208361 shutdown_connections 2024-04-24T23:11:09.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.087+0000 7f934062a640 1 -- 172.21.15.32:0/2746208361 wait complete. 2024-04-24T23:11:09.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.088+0000 7f934062a640 1 Processor -- start 2024-04-24T23:11:09.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.088+0000 7f934062a640 1 -- start start 2024-04-24T23:11:09.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.088+0000 7f934062a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93380b5af0 0x7f93381e8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:09.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.088+0000 7f934062a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9338162320 0x7f933814e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:09.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.088+0000 7f934062a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9338162e10 0x7f93381f4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:09.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.088+0000 7f934062a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9338071bd0 con 0x7f9338162e10 2024-04-24T23:11:09.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.088+0000 7f934062a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9338067950 con 0x7f93380b5af0 2024-04-24T23:11:09.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.088+0000 7f934062a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f9338071d50 con 0x7f9338162320 2024-04-24T23:11:09.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.088+0000 7f933ebc7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9338162e10 0x7f93381f4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:09.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.088+0000 7f933ebc7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9338162e10 0x7f93381f4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35620/0 (socket says 172.21.15.32:35620) 2024-04-24T23:11:09.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.088+0000 7f933ebc7640 1 -- 172.21.15.32:0/1117762543 learned_addr learned my addr 172.21.15.32:0/1117762543 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:09.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.089+0000 7f933e3c6640 1 --2- 172.21.15.32:0/1117762543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93380b5af0 0x7f93381e8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:09.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.089+0000 7f933dbc5640 1 --2- 172.21.15.32:0/1117762543 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9338162320 0x7f933814e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:09.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.089+0000 7f933ebc7640 1 -- 172.21.15.32:0/1117762543 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9338162320 msgr2=0x7f933814e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:09.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.089+0000 7f933ebc7640 1 --2- 172.21.15.32:0/1117762543 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9338162320 0x7f933814e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:09.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.089+0000 7f933ebc7640 1 -- 172.21.15.32:0/1117762543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93380b5af0 msgr2=0x7f93381e8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:09.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.089+0000 7f933ebc7640 1 --2- 172.21.15.32:0/1117762543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93380b5af0 0x7f93381e8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:09.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.089+0000 7f933ebc7640 1 -- 172.21.15.32:0/1117762543 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93381e8990 con 0x7f9338162e10 2024-04-24T23:11:09.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.089+0000 7f933e3c6640 1 --2- 172.21.15.32:0/1117762543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93380b5af0 0x7f93381e8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:11:09.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.089+0000 7f933dbc5640 1 --2- 172.21.15.32:0/1117762543 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9338162320 0x7f933814e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:11:09.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.089+0000 7f933ebc7640 1 --2- 172.21.15.32:0/1117762543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9338162e10 0x7f93381f4c20 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f932800c700 tx=0x7f932800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:09.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.089+0000 7f9336ffd640 1 -- 172.21.15.32:0/1117762543 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9328018070 con 0x7f9338162e10 2024-04-24T23:11:09.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.089+0000 7f9336ffd640 1 -- 172.21.15.32:0/1117762543 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f932800ce10 con 0x7f9338162e10 2024-04-24T23:11:09.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.089+0000 7f934062a640 1 -- 172.21.15.32:0/1117762543 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93381e8cd0 con 0x7f9338162e10 2024-04-24T23:11:09.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.089+0000 7f9336ffd640 1 -- 172.21.15.32:0/1117762543 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932800a860 con 0x7f9338162e10 2024-04-24T23:11:09.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.089+0000 7f934062a640 1 -- 172.21.15.32:0/1117762543 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93381e9510 con 0x7f9338162e10 2024-04-24T23:11:09.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.090+0000 7f934062a640 1 -- 172.21.15.32:0/1117762543 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93380690e0 con 0x7f9338162e10 2024-04-24T23:11:09.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.091+0000 7f9336ffd640 1 -- 172.21.15.32:0/1117762543 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f932801b070 con 0x7f9338162e10 2024-04-24T23:11:09.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.091+0000 7f9336ffd640 1 --2- 172.21.15.32:0/1117762543 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f932406be30 0x7f932406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:09.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.091+0000 7f9336ffd640 1 -- 172.21.15.32:0/1117762543 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(190..190 src has 1..190) ==== 5738+0+0 (secure 0 0 0) 0x7f932808a530 con 0x7f9338162e10 2024-04-24T23:11:09.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.091+0000 7f933e3c6640 1 --2- 172.21.15.32:0/1117762543 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f932406be30 0x7f932406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:09.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.092+0000 7f933e3c6640 1 --2- 172.21.15.32:0/1117762543 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f932406be30 0x7f932406e2f0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f9330034000 tx=0x7f933002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:09.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.095+0000 7f9336ffd640 1 -- 172.21.15.32:0/1117762543 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f93380690e0 con 0x7f9338162e10 2024-04-24T23:11:09.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.279+0000 7f934062a640 1 -- 172.21.15.32:0/1117762543 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f93381e86e0 con 0x7f9338162e10 2024-04-24T23:11:09.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.280+0000 7f9336ffd640 1 -- 172.21.15.32:0/1117762543 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v190) ==== 81+0+34379 (secure 0 0 0) 0x7f93381e8cd0 con 0x7f9338162e10 2024-04-24T23:11:09.282 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:11:09.282 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:11:09.282 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 190, 2024-04-24T23:11:09.282 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", 2024-04-24T23:11:09.282 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2024-04-24T22:58:46.964851+0000", 2024-04-24T23:11:09.283 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2024-04-24T23:11:06.861172+0000", 2024-04-24T23:11:09.283 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2024-04-24T22:58:52.048787+0000", 2024-04-24T23:11:09.283 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2024-04-24T22:58:49.045609+0000", 2024-04-24T23:11:09.283 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-24T23:11:09.283 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2024-04-24T23:11:09.283 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2024-04-24T23:11:09.283 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2024-04-24T23:11:09.283 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2024-04-24T23:11:09.283 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2024-04-24T23:11:09.283 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2024-04-24T23:11:09.283 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:11:09.283 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 4, 2024-04-24T23:11:09.283 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.69999998807907104, 2024-04-24T23:11:09.283 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-24T23:11:09.284 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-24T23:11:09.284 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2024-04-24T23:11:09.284 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 23, 2024-04-24T23:11:09.284 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2024-04-24T23:11:09.284 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "luminous", 2024-04-24T23:11:09.284 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "luminous", 2024-04-24T23:11:09.284 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "squid", 2024-04-24T23:11:09.284 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_crimson": false, 2024-04-24T23:11:09.284 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2024-04-24T23:11:09.284 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.284 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2024-04-24T23:11:09.284 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": ".mgr", 2024-04-24T23:11:09.284 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T22:58:53.171408+0000", 2024-04-24T23:11:09.284 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:11:09.285 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:11:09.285 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:11:09.285 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:11:09.285 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:11:09.285 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:11:09.285 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:11:09.285 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:11:09.285 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:11:09.285 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:11:09.285 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:11:09.285 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:11:09.285 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2024-04-24T23:11:09.285 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2024-04-24T23:11:09.285 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2024-04-24T23:11:09.285 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2024-04-24T23:11:09.286 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2024-04-24T23:11:09.286 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:11:09.286 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:11:09.286 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:11:09.286 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:11:09.286 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:11:09.286 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:11:09.286 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:11:09.286 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.286 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "113", 2024-04-24T23:11:09.286 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:11:09.286 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:11:09.286 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:11:09.287 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:11:09.287 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:11:09.287 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:11:09.287 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:11:09.287 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:11:09.287 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:11:09.287 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:11:09.287 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:11:09.287 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:11:09.287 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:11:09.287 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:11:09.287 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:11:09.287 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:11:09.287 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:11:09.288 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:11:09.288 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:11:09.288 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:11:09.288 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:11:09.288 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:11:09.288 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:11:09.288 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:11:09.288 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:11:09.288 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:11:09.288 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.288 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:11:09.288 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:11:09.288 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:11:09.288 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:11:09.289 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:11:09.289 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:11:09.289 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:11:09.289 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:11:09.289 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:11:09.289 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:11:09.289 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:11:09.289 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:11:09.289 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_max": 32, 2024-04-24T23:11:09.289 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2024-04-24T23:11:09.289 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.289 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:11:09.289 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": {} 2024-04-24T23:11:09.290 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.290 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:11:09.290 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:11:09.290 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 8, 2024-04-24T23:11:09.290 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 8, 2024-04-24T23:11:09.290 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.25, 2024-04-24T23:11:09.290 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-04-24T23:11:09.290 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-04-24T23:11:09.290 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:11:09.290 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:11:09.290 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:11:09.290 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:09.290 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.290 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.291 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 22, 2024-04-24T23:11:09.291 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2024-04-24T23:11:09.291 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:10:34.649562+0000", 2024-04-24T23:11:09.291 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:11:09.291 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:11:09.291 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:11:09.291 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:11:09.291 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:11:09.291 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:11:09.291 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:11:09.291 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:11:09.291 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:11:09.291 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:11:09.291 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:11:09.292 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:11:09.292 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 32, 2024-04-24T23:11:09.292 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 32, 2024-04-24T23:11:09.292 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 32, 2024-04-24T23:11:09.292 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 32, 2024-04-24T23:11:09.292 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 32, 2024-04-24T23:11:09.292 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:11:09.292 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:11:09.292 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:11:09.292 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:11:09.292 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:11:09.292 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:11:09.292 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:11:09.292 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.293 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "184", 2024-04-24T23:11:09.293 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:11:09.293 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:11:09.293 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:11:09.293 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:11:09.293 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:11:09.293 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:11:09.293 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:11:09.293 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:11:09.293 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:11:09.293 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-24T23:11:09.293 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:11:09.293 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:11:09.293 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:11:09.294 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:11:09.294 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:11:09.294 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:11:09.294 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:11:09.294 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:11:09.294 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:11:09.294 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:11:09.294 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:11:09.294 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:11:09.294 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:11:09.294 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:11:09.294 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:11:09.294 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:11:09.295 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.295 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:11:09.295 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:11:09.295 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:11:09.295 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:11:09.295 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:11:09.295 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:11:09.295 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:11:09.295 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:11:09.295 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:11:09.295 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:11:09.295 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:11:09.295 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:11:09.295 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2024-04-24T23:11:09.296 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2024-04-24T23:11:09.296 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2024-04-24T23:11:09.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.296 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:11:09.296 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:11:09.296 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2024-04-24T23:11:09.296 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:09.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.296 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:11:09.296 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:11:09.296 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.5, 2024-04-24T23:11:09.296 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.5, 2024-04-24T23:11:09.296 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:11:09.296 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.5, 2024-04-24T23:11:09.297 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.5, 2024-04-24T23:11:09.297 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:11:09.297 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:11:09.297 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:11:09.297 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:09.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.297 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 23, 2024-04-24T23:11:09.297 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2024-04-24T23:11:09.297 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-24T23:10:35.339604+0000", 2024-04-24T23:11:09.297 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-24T23:11:09.297 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-24T23:11:09.297 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-24T23:11:09.298 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-24T23:11:09.298 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-24T23:11:09.298 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-24T23:11:09.298 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-24T23:11:09.298 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-24T23:11:09.298 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T23:11:09.298 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T23:11:09.298 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-24T23:11:09.298 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-24T23:11:09.298 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 64, 2024-04-24T23:11:09.298 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 64, 2024-04-24T23:11:09.298 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 64, 2024-04-24T23:11:09.298 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 64, 2024-04-24T23:11:09.298 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 64, 2024-04-24T23:11:09.299 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-24T23:11:09.299 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-24T23:11:09.299 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-24T23:11:09.299 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-24T23:11:09.299 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-24T23:11:09.299 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-24T23:11:09.299 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-24T23:11:09.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.299 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "190", 2024-04-24T23:11:09.299 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-24T23:11:09.299 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T23:11:09.299 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T23:11:09.300 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-24T23:11:09.300 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-24T23:11:09.300 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-24T23:11:09.300 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-24T23:11:09.300 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-24T23:11:09.300 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-24T23:11:09.300 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 33554432, 2024-04-24T23:11:09.300 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-24T23:11:09.300 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-24T23:11:09.300 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-24T23:11:09.300 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-24T23:11:09.300 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-24T23:11:09.300 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-24T23:11:09.301 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-24T23:11:09.301 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-24T23:11:09.301 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T23:11:09.301 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T23:11:09.301 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T23:11:09.301 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-24T23:11:09.301 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-24T23:11:09.301 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-24T23:11:09.301 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-24T23:11:09.301 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-24T23:11:09.301 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.301 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-24T23:11:09.301 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-24T23:11:09.302 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-24T23:11:09.302 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-24T23:11:09.302 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-24T23:11:09.302 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T23:11:09.302 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-24T23:11:09.302 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-24T23:11:09.302 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-24T23:11:09.302 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-24T23:11:09.302 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-24T23:11:09.302 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-24T23:11:09.302 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 64, 2024-04-24T23:11:09.302 INFO:teuthology.orchestra.run.smithi032.stdout: "target_size_ratio": 0.89999997615814209 2024-04-24T23:11:09.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.303 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-24T23:11:09.303 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-24T23:11:09.303 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2024-04-24T23:11:09.303 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:09.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.303 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-24T23:11:09.303 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-24T23:11:09.303 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.3799999952316284, 2024-04-24T23:11:09.303 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.3799999952316284, 2024-04-24T23:11:09.303 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-24T23:11:09.303 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.3799999952316284, 2024-04-24T23:11:09.303 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.3799999952316284, 2024-04-24T23:11:09.303 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-04-24T23:11:09.303 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-24T23:11:09.304 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-04-24T23:11:09.304 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:09.304 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:09.304 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:11:09.304 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-04-24T23:11:09.304 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.304 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:11:09.304 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:11:09.304 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:11:09.304 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:11:09.304 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:11:09.304 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:11:09.304 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:11:09.304 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:11:09.305 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:11:09.305 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:11:09.305 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:11:09.305 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:11:09.305 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:11:09.305 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:09.305 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.305 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:09.305 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2024-04-24T23:11:09.305 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:11:09.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.305 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.305 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:09.305 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2024-04-24T23:11:09.305 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:11:09.306 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:09.306 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:09.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.306 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:11:09.306 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:09.306 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.306 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:09.306 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2024-04-24T23:11:09.306 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:11:09.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.306 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.306 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:09.306 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2024-04-24T23:11:09.307 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:11:09.307 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:09.307 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:09.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.307 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:11:09.307 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:09.307 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.307 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:09.307 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2024-04-24T23:11:09.307 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:11:09.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.307 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.307 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:09.307 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2024-04-24T23:11:09.307 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:11:09.307 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:09.308 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:09.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.308 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:11:09.308 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:09.308 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.308 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:09.308 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2024-04-24T23:11:09.308 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:11:09.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.308 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.308 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:09.308 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2024-04-24T23:11:09.308 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1479395426 2024-04-24T23:11:09.308 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:09.308 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:09.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.309 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/1479395426", 2024-04-24T23:11:09.309 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/1479395426", 2024-04-24T23:11:09.309 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/1479395426", 2024-04-24T23:11:09.309 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/1479395426", 2024-04-24T23:11:09.309 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:11:09.309 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:11:09.309 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:11:09.309 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:09.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.309 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.309 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:11:09.309 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:11:09.310 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:11:09.310 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:11:09.310 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:11:09.310 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:11:09.310 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:11:09.310 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:11:09.310 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:11:09.310 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:11:09.310 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:11:09.310 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:11:09.310 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:11:09.310 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:09.310 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.310 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:09.311 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2024-04-24T23:11:09.311 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:11:09.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.311 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:09.311 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2024-04-24T23:11:09.311 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:11:09.311 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:09.311 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:09.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.311 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:11:09.311 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:09.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.311 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:09.312 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2024-04-24T23:11:09.312 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:11:09.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.312 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.312 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:09.312 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2024-04-24T23:11:09.312 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:11:09.312 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:09.312 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:09.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.312 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:11:09.312 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:09.312 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.313 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:09.313 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2024-04-24T23:11:09.313 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:11:09.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.313 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.313 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:09.313 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2024-04-24T23:11:09.313 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:11:09.313 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:09.313 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:09.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.313 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:11:09.313 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:09.313 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.313 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:09.314 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2024-04-24T23:11:09.314 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:11:09.314 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.314 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.314 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:09.314 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2024-04-24T23:11:09.314 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1340633718 2024-04-24T23:11:09.314 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:09.314 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:09.314 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.314 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/1340633718", 2024-04-24T23:11:09.314 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/1340633718", 2024-04-24T23:11:09.314 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/1340633718", 2024-04-24T23:11:09.314 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/1340633718", 2024-04-24T23:11:09.314 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:11:09.315 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:11:09.315 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:11:09.315 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:09.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.315 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.315 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:11:09.315 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:11:09.315 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:11:09.315 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:11:09.315 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:11:09.315 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:11:09.315 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:11:09.315 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:11:09.315 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:11:09.316 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:11:09.316 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:11:09.316 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:11:09.316 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:11:09.316 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:09.316 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.316 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:09.316 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2024-04-24T23:11:09.316 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:11:09.316 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.316 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.316 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:09.316 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2024-04-24T23:11:09.316 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:11:09.316 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:09.316 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:09.317 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.317 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:11:09.317 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:09.317 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.317 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:09.317 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2024-04-24T23:11:09.317 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:11:09.317 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.317 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.317 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:09.317 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2024-04-24T23:11:09.317 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:11:09.317 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:09.317 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:09.317 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.318 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:11:09.318 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:09.318 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.318 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:09.318 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2024-04-24T23:11:09.318 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:11:09.318 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.318 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.318 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:09.318 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2024-04-24T23:11:09.318 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:11:09.318 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:09.318 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:09.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.319 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:11:09.319 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:09.319 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.319 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:09.319 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2024-04-24T23:11:09.319 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:11:09.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.319 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.319 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:09.319 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2024-04-24T23:11:09.319 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3405899334 2024-04-24T23:11:09.319 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:09.319 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:09.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.320 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/3405899334", 2024-04-24T23:11:09.320 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6820/3405899334", 2024-04-24T23:11:09.320 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6828/3405899334", 2024-04-24T23:11:09.320 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6824/3405899334", 2024-04-24T23:11:09.320 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:11:09.320 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:11:09.320 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:11:09.320 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:09.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.320 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.320 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:11:09.320 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:11:09.321 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:11:09.321 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:11:09.321 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:11:09.321 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:11:09.321 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:11:09.321 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:11:09.321 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:11:09.321 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 188, 2024-04-24T23:11:09.321 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:11:09.321 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:11:09.321 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:11:09.321 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:09.321 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.321 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:09.321 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2024-04-24T23:11:09.322 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:11:09.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.322 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.322 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:09.322 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2024-04-24T23:11:09.322 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:11:09.322 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:09.322 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:09.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.322 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:11:09.322 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:09.323 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.323 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:09.323 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2024-04-24T23:11:09.323 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:11:09.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.323 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.323 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:09.323 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2024-04-24T23:11:09.323 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:11:09.323 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:09.323 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:09.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.323 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:11:09.324 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:09.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.324 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:09.324 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2024-04-24T23:11:09.324 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:11:09.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.324 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:09.324 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2024-04-24T23:11:09.324 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:11:09.324 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:09.324 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:09.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.324 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:11:09.324 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:09.325 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.325 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:09.325 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2024-04-24T23:11:09.325 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:11:09.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.325 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.325 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:09.325 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2024-04-24T23:11:09.325 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3381057190 2024-04-24T23:11:09.325 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:09.325 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:09.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.325 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6821/3381057190", 2024-04-24T23:11:09.325 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6825/3381057190", 2024-04-24T23:11:09.326 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/3381057190", 2024-04-24T23:11:09.326 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/3381057190", 2024-04-24T23:11:09.326 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:11:09.326 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:11:09.326 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:11:09.326 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:09.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.326 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.326 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:11:09.326 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:11:09.326 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:11:09.326 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:11:09.326 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:11:09.326 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:11:09.327 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:11:09.327 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:11:09.327 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:11:09.327 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:11:09.327 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:11:09.327 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:11:09.327 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:11:09.327 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:09.327 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.327 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:09.327 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6800", 2024-04-24T23:11:09.327 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:11:09.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.327 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.327 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:09.328 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6801", 2024-04-24T23:11:09.328 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:11:09.328 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:09.328 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:09.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.328 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:11:09.328 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:09.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.328 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:09.328 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6802", 2024-04-24T23:11:09.328 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:11:09.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.328 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:09.329 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6803", 2024-04-24T23:11:09.329 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:11:09.329 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:09.329 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:09.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.329 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:11:09.329 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:09.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.329 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:09.329 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6806", 2024-04-24T23:11:09.329 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:11:09.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.329 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:09.330 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6807", 2024-04-24T23:11:09.330 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:11:09.330 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:09.330 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:09.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.330 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:11:09.330 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:09.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.330 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:09.330 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6804", 2024-04-24T23:11:09.330 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:11:09.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.330 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:09.331 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6805", 2024-04-24T23:11:09.331 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4010553979 2024-04-24T23:11:09.331 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:09.333 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:09.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.333 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6801/4010553979", 2024-04-24T23:11:09.333 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6803/4010553979", 2024-04-24T23:11:09.333 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6807/4010553979", 2024-04-24T23:11:09.334 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6805/4010553979", 2024-04-24T23:11:09.334 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:11:09.334 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:11:09.334 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:11:09.334 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:09.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.334 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:11:09.334 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:11:09.334 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:11:09.334 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:11:09.334 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:11:09.334 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:11:09.334 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:11:09.335 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:11:09.335 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:11:09.335 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:11:09.335 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:11:09.335 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:11:09.335 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:11:09.335 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:09.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.335 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:09.335 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6808", 2024-04-24T23:11:09.335 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:11:09.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.335 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:09.336 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6809", 2024-04-24T23:11:09.336 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:11:09.336 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:09.336 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:09.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.336 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:11:09.336 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:09.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.336 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:09.336 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6810", 2024-04-24T23:11:09.336 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:11:09.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.336 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:09.337 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6811", 2024-04-24T23:11:09.337 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:11:09.337 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:09.337 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:09.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.337 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:11:09.337 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:09.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.337 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:09.337 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6814", 2024-04-24T23:11:09.337 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:11:09.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.337 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:09.338 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6815", 2024-04-24T23:11:09.338 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:11:09.338 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:09.338 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:09.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.338 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:11:09.338 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:09.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.338 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:09.338 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6812", 2024-04-24T23:11:09.338 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:11:09.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.338 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:09.339 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6813", 2024-04-24T23:11:09.339 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3794084091 2024-04-24T23:11:09.339 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:09.339 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:09.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.339 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6809/3794084091", 2024-04-24T23:11:09.339 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6811/3794084091", 2024-04-24T23:11:09.339 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6815/3794084091", 2024-04-24T23:11:09.339 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6813/3794084091", 2024-04-24T23:11:09.339 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:11:09.339 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:11:09.339 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:11:09.339 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:09.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.340 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:11:09.340 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:11:09.340 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:11:09.340 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:11:09.340 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:11:09.340 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:11:09.340 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:11:09.340 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:11:09.340 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:11:09.340 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:11:09.340 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:11:09.340 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:11:09.340 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:11:09.341 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:09.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.341 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:09.341 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6823", 2024-04-24T23:11:09.341 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:11:09.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.341 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:09.341 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6825", 2024-04-24T23:11:09.341 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:11:09.341 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:09.341 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:09.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.341 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:11:09.342 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:09.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.342 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:09.342 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6826", 2024-04-24T23:11:09.342 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:11:09.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.342 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:09.342 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6827", 2024-04-24T23:11:09.342 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:11:09.342 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:09.342 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:09.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.343 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:11:09.343 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:09.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.343 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:09.343 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6830", 2024-04-24T23:11:09.343 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:11:09.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.343 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:09.343 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6831", 2024-04-24T23:11:09.343 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:11:09.343 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:09.343 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:09.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.344 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:11:09.344 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:09.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.344 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:09.344 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6828", 2024-04-24T23:11:09.344 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:11:09.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.344 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:09.344 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6829", 2024-04-24T23:11:09.344 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1348640258 2024-04-24T23:11:09.344 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:09.344 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:09.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.345 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6825/1348640258", 2024-04-24T23:11:09.345 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6827/1348640258", 2024-04-24T23:11:09.345 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6831/1348640258", 2024-04-24T23:11:09.345 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6829/1348640258", 2024-04-24T23:11:09.345 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:11:09.345 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:11:09.345 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:11:09.345 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:09.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.345 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:11:09.345 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T23:11:09.345 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-24T23:11:09.345 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-24T23:11:09.346 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-24T23:11:09.346 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-24T23:11:09.346 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-24T23:11:09.346 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-24T23:11:09.346 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-04-24T23:11:09.346 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 183, 2024-04-24T23:11:09.346 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-24T23:11:09.346 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-24T23:11:09.346 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-24T23:11:09.346 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:09.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.346 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:09.346 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6816", 2024-04-24T23:11:09.347 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:11:09.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.347 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:09.347 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6817", 2024-04-24T23:11:09.347 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:11:09.347 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:09.347 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:09.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.347 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-24T23:11:09.347 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:09.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.347 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:09.347 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6818", 2024-04-24T23:11:09.347 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:11:09.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.348 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:09.348 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6819", 2024-04-24T23:11:09.348 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:11:09.348 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:09.348 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:09.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.348 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-24T23:11:09.348 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:09.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.348 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:09.348 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6822", 2024-04-24T23:11:09.348 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:11:09.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.349 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:09.349 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6824", 2024-04-24T23:11:09.349 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:11:09.349 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:09.349 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:09.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.349 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-24T23:11:09.349 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-24T23:11:09.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.349 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-24T23:11:09.349 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6820", 2024-04-24T23:11:09.349 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:11:09.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.350 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.350 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-24T23:11:09.350 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.121:6821", 2024-04-24T23:11:09.350 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3250005122 2024-04-24T23:11:09.350 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:09.350 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:09.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.350 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.121:6817/3250005122", 2024-04-24T23:11:09.350 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.121:6819/3250005122", 2024-04-24T23:11:09.350 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.121:6824/3250005122", 2024-04-24T23:11:09.350 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.121:6821/3250005122", 2024-04-24T23:11:09.350 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-24T23:11:09.350 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-24T23:11:09.351 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-24T23:11:09.351 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:09.351 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:09.351 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:11:09.351 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2024-04-24T23:11:09.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.351 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-24T23:11:09.351 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:11:09.351 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:11:09.351 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:11:09.351 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:11:09.351 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:11:09.351 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.156184+0000", 2024-04-24T23:11:09.351 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:11:09.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.352 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-24T23:11:09.352 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:11:09.352 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:11:09.352 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:11:09.352 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:11:09.352 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:11:09.352 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240774+0000", 2024-04-24T23:11:09.352 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:11:09.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.352 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-24T23:11:09.352 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:11:09.353 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:11:09.353 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:11:09.353 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:11:09.353 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:11:09.353 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.240949+0000", 2024-04-24T23:11:09.353 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:11:09.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.353 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-04-24T23:11:09.353 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:11:09.353 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:11:09.353 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:11:09.353 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:11:09.353 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:11:09.354 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.171672+0000", 2024-04-24T23:11:09.354 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:11:09.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.354 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-04-24T23:11:09.354 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:11:09.354 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:11:09.354 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:11:09.354 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:11:09.354 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:11:09.354 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.220379+0000", 2024-04-24T23:11:09.354 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:11:09.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.355 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-04-24T23:11:09.355 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:11:09.355 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:11:09.355 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:11:09.355 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:11:09.355 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:11:09.355 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.204963+0000", 2024-04-24T23:11:09.355 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:11:09.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.355 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.355 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-04-24T23:11:09.355 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:11:09.355 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:11:09.355 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:11:09.356 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:11:09.356 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:11:09.356 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.239748+0000", 2024-04-24T23:11:09.356 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:11:09.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.356 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.356 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-04-24T23:11:09.356 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-24T23:11:09.356 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-24T23:11:09.356 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-24T23:11:09.356 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-24T23:11:09.356 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-24T23:11:09.356 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-24T22:58:50.154104+0000", 2024-04-24T23:11:09.356 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-24T23:11:09.356 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:09.357 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:11:09.357 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2024-04-24T23:11:09.357 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [ 2024-04-24T23:11:09.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.357 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "23.6", 2024-04-24T23:11:09.357 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-04-24T23:11:09.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-24T23:11:09.357 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 5, 2024-04-24T23:11:09.357 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 3 2024-04-24T23:11:09.357 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:09.357 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-24T23:11:09.357 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:09.357 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-24T23:11:09.357 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_primaries": [], 2024-04-24T23:11:09.358 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2024-04-24T23:11:09.358 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2024-04-24T23:11:09.358 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklist": {}, 2024-04-24T23:11:09.358 INFO:teuthology.orchestra.run.smithi032.stdout: "range_blocklist": {}, 2024-04-24T23:11:09.358 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2024-04-24T23:11:09.358 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2024-04-24T23:11:09.358 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2024-04-24T23:11:09.358 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2024-04-24T23:11:09.358 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2024-04-24T23:11:09.358 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2024-04-24T23:11:09.358 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2024-04-24T23:11:09.358 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:09.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-24T23:11:09.359 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2024-04-24T23:11:09.359 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2024-04-24T23:11:09.359 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2024-04-24T23:11:09.359 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2024-04-24T23:11:09.359 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {}, 2024-04-24T23:11:09.359 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode": { 2024-04-24T23:11:09.359 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_enabled": false, 2024-04-24T23:11:09.359 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_bucket_count": 0, 2024-04-24T23:11:09.359 INFO:teuthology.orchestra.run.smithi032.stdout: "degraded_stretch_mode": 0, 2024-04-24T23:11:09.360 INFO:teuthology.orchestra.run.smithi032.stdout: "recovering_stretch_mode": 0, 2024-04-24T23:11:09.360 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_bucket": 0 2024-04-24T23:11:09.360 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-24T23:11:09.360 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:11:09.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.282+0000 7f934062a640 1 -- 172.21.15.32:0/1117762543 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f932406be30 msgr2=0x7f932406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:09.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.282+0000 7f934062a640 1 --2- 172.21.15.32:0/1117762543 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f932406be30 0x7f932406e2f0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f9330034000 tx=0x7f933002e000 comp rx=0 tx=0).stop 2024-04-24T23:11:09.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.282+0000 7f934062a640 1 -- 172.21.15.32:0/1117762543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9338162e10 msgr2=0x7f93381f4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:09.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.282+0000 7f934062a640 1 --2- 172.21.15.32:0/1117762543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9338162e10 0x7f93381f4c20 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f932800c700 tx=0x7f932800cbc0 comp rx=0 tx=0).stop 2024-04-24T23:11:09.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.282+0000 7f934062a640 1 -- 172.21.15.32:0/1117762543 shutdown_connections 2024-04-24T23:11:09.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.282+0000 7f934062a640 1 --2- 172.21.15.32:0/1117762543 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f932406be30 0x7f932406e2f0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:09.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.282+0000 7f934062a640 1 --2- 172.21.15.32:0/1117762543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9338162e10 0x7f93381f4c20 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:09.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.282+0000 7f934062a640 1 --2- 172.21.15.32:0/1117762543 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9338162320 0x7f933814e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:09.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.282+0000 7f934062a640 1 --2- 172.21.15.32:0/1117762543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93380b5af0 0x7f93381e8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:09.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.282+0000 7f934062a640 1 -- 172.21.15.32:0/1117762543 >> 172.21.15.32:0/1117762543 conn(0x7f93380d0ae0 msgr2=0x7f933815d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:09.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.282+0000 7f934062a640 1 -- 172.21.15.32:0/1117762543 shutdown_connections 2024-04-24T23:11:09.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.282+0000 7f934062a640 1 -- 172.21.15.32:0/1117762543 wait complete. 2024-04-24T23:11:09.363 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:11:09.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.463+0000 7fe3500f5640 1 Processor -- start 2024-04-24T23:11:09.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.463+0000 7fe3500f5640 1 -- start start 2024-04-24T23:11:09.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.464+0000 7fe3500f5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe34815e9b0 0x7fe3480d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:09.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.464+0000 7fe3500f5640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe3480d4210 0x7fe34806b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:09.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.464+0000 7fe3500f5640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3480d3170 con 0x7fe3480ba0e0 2024-04-24T23:11:09.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.464+0000 7fe3500f5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fe3480b8900 con 0x7fe34815e9b0 2024-04-24T23:11:09.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.464+0000 7fe3500f5640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fe3480d3450 con 0x7fe3480d4210 2024-04-24T23:11:09.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.464+0000 7fe34de91640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe34815e9b0 0x7fe3480d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:09.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.464+0000 7fe34de91640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe34815e9b0 0x7fe3480d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35624/0 (socket says 172.21.15.32:35624) 2024-04-24T23:11:09.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.464+0000 7fe34de91640 1 -- 172.21.15.32:0/1876648109 learned_addr learned my addr 172.21.15.32:0/1876648109 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:09.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.465+0000 7fe34de91640 1 -- 172.21.15.32:0/1876648109 >> v2:172.21.15.121:3300/0 conn(0x7fe3480d4210 msgr2=0x7fe34806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:09.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.465+0000 7fe34de91640 1 --2- 172.21.15.32:0/1876648109 >> v2:172.21.15.121:3300/0 conn(0x7fe3480d4210 0x7fe34806b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:09.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.465+0000 7fe34de91640 1 -- 172.21.15.32:0/1876648109 >> v1:172.21.15.121:6789/0 conn(0x7fe3480ba0e0 legacy=0x7fe348071730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:09.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.465+0000 7fe34de91640 1 -- 172.21.15.32:0/1876648109 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3480d3730 con 0x7fe34815e9b0 2024-04-24T23:11:09.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.465+0000 7fe34de91640 1 --2- 172.21.15.32:0/1876648109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe34815e9b0 0x7fe3480d2b90 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fe3480730f0 tx=0x7fe34402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dfd5287ec7625c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:09.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.466+0000 7fe34ce8f640 1 -- 172.21.15.32:0/1876648109 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe344037070 con 0x7fe34815e9b0 2024-04-24T23:11:09.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.466+0000 7fe34ce8f640 1 -- 172.21.15.32:0/1876648109 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe344032440 con 0x7fe34815e9b0 2024-04-24T23:11:09.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.466+0000 7fe34ce8f640 1 -- 172.21.15.32:0/1876648109 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe344032810 con 0x7fe34815e9b0 2024-04-24T23:11:09.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.466+0000 7fe3500f5640 1 -- 172.21.15.32:0/1876648109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe34815e9b0 msgr2=0x7fe3480d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:09.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.466+0000 7fe3500f5640 1 --2- 172.21.15.32:0/1876648109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe34815e9b0 0x7fe3480d2b90 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fe3480730f0 tx=0x7fe34402a1a0 comp rx=0 tx=0).stop 2024-04-24T23:11:09.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.466+0000 7fe3500f5640 1 -- 172.21.15.32:0/1876648109 shutdown_connections 2024-04-24T23:11:09.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.466+0000 7fe3500f5640 1 --2- 172.21.15.32:0/1876648109 >> v2:172.21.15.121:3300/0 conn(0x7fe3480d4210 0x7fe34806b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:09.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.466+0000 7fe3500f5640 1 --2- 172.21.15.32:0/1876648109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe34815e9b0 0x7fe3480d2b90 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:09.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.466+0000 7fe3500f5640 1 -- 172.21.15.32:0/1876648109 >> 172.21.15.32:0/1876648109 conn(0x7fe3480d11d0 msgr2=0x7fe3480d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:09.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.466+0000 7fe3500f5640 1 -- 172.21.15.32:0/1876648109 shutdown_connections 2024-04-24T23:11:09.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.466+0000 7fe3500f5640 1 -- 172.21.15.32:0/1876648109 wait complete. 2024-04-24T23:11:09.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.467+0000 7fe3500f5640 1 Processor -- start 2024-04-24T23:11:09.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.467+0000 7fe3500f5640 1 -- start start 2024-04-24T23:11:09.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.468+0000 7fe3500f5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3480ba0e0 0x7fe3481f73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:09.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.468+0000 7fe3500f5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3480d4210 0x7fe3480ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:09.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.468+0000 7fe3500f5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe34815e9b0 0x7fe3480c9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:09.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.468+0000 7fe3500f5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe3480d2400 con 0x7fe3480d4210 2024-04-24T23:11:09.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.468+0000 7fe3500f5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe3480d3cf0 con 0x7fe34815e9b0 2024-04-24T23:11:09.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.468+0000 7fe3500f5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe34806b650 con 0x7fe3480ba0e0 2024-04-24T23:11:09.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.468+0000 7fe34d690640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3480d4210 0x7fe3480ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:09.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.468+0000 7fe34d690640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3480d4210 0x7fe3480ccc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35628/0 (socket says 172.21.15.32:35628) 2024-04-24T23:11:09.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.468+0000 7fe34d690640 1 -- 172.21.15.32:0/2774633728 learned_addr learned my addr 172.21.15.32:0/2774633728 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:09.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.468+0000 7fe34d690640 1 -- 172.21.15.32:0/2774633728 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3480ba0e0 msgr2=0x7fe3481f73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:09.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.468+0000 7fe34d690640 1 --2- 172.21.15.32:0/2774633728 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3480ba0e0 0x7fe3481f73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:09.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.468+0000 7fe34e692640 1 --2- 172.21.15.32:0/2774633728 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe34815e9b0 0x7fe3480c9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:09.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.468+0000 7fe34d690640 1 -- 172.21.15.32:0/2774633728 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe34815e9b0 msgr2=0x7fe3480c9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:09.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.469+0000 7fe34d690640 1 --2- 172.21.15.32:0/2774633728 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe34815e9b0 0x7fe3480c9c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:09.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.469+0000 7fe34d690640 1 -- 172.21.15.32:0/2774633728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3481541d0 con 0x7fe3480d4210 2024-04-24T23:11:09.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.469+0000 7fe34d690640 1 --2- 172.21.15.32:0/2774633728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3480d4210 0x7fe3480ccc30 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fe33800b4a0 tx=0x7fe33800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:09.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.469+0000 7fe33e7fc640 1 -- 172.21.15.32:0/2774633728 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe338014070 con 0x7fe3480d4210 2024-04-24T23:11:09.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.469+0000 7fe33e7fc640 1 -- 172.21.15.32:0/2774633728 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe338010610 con 0x7fe3480d4210 2024-04-24T23:11:09.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.469+0000 7fe3500f5640 1 -- 172.21.15.32:0/2774633728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3480cbb90 con 0x7fe3480d4210 2024-04-24T23:11:09.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.469+0000 7fe33e7fc640 1 -- 172.21.15.32:0/2774633728 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe338010a00 con 0x7fe3480d4210 2024-04-24T23:11:09.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.469+0000 7fe3500f5640 1 -- 172.21.15.32:0/2774633728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3480cc150 con 0x7fe3480d4210 2024-04-24T23:11:09.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.470+0000 7fe3500f5640 1 -- 172.21.15.32:0/2774633728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe348068430 con 0x7fe3480d4210 2024-04-24T23:11:09.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.470+0000 7fe33e7fc640 1 -- 172.21.15.32:0/2774633728 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe338017070 con 0x7fe3480d4210 2024-04-24T23:11:09.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.471+0000 7fe33e7fc640 1 --2- 172.21.15.32:0/2774633728 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe32c06be30 0x7fe32c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:09.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.471+0000 7fe33e7fc640 1 -- 172.21.15.32:0/2774633728 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(190..190 src has 1..190) ==== 5738+0+0 (secure 0 0 0) 0x7fe33808a890 con 0x7fe3480d4210 2024-04-24T23:11:09.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.471+0000 7fe34de91640 1 --2- 172.21.15.32:0/2774633728 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe32c06be30 0x7fe32c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:09.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.471+0000 7fe34de91640 1 --2- 172.21.15.32:0/2774633728 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe32c06be30 0x7fe32c06e2f0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fe344034000 tx=0x7fe34402c000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:09.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.475+0000 7fe33e7fc640 1 -- 172.21.15.32:0/2774633728 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe3481541d0 con 0x7fe3480d4210 2024-04-24T23:11:09.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.694+0000 7fe3500f5640 1 -- 172.21.15.32:0/2774633728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe3480718b0 con 0x7fe3480d4210 2024-04-24T23:11:09.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.695+0000 7fe33e7fc640 1 -- 172.21.15.32:0/2774633728 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) ==== 94+0+5185 (secure 0 0 0) 0x7fe3380592b0 con 0x7fe3480d4210 2024-04-24T23:11:09.696 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:11:09.697 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":80,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6941,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/416518156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":416518156},{"type":"v1","addr":"172.21.15.121:6837","nonce":416518156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7762,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1506667233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1506667233},{"type":"v1","addr":"172.21.15.121:6835","nonce":1506667233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":9063,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/2215079101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":2215079101},{"type":"v1","addr":"172.21.15.32:6834","nonce":2215079101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:10:36.353761+0000","modified":"2024-04-24T23:10:38.436407+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9066},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9066":{"gid":9066,"name":"a","rank":0,"incarnation":78,"state":"up:active","state_seq":4,"addr":"172.21.15.32:6835/901353412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":901353412},{"type":"v1","addr":"172.21.15.32:6835","nonce":901353412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9066,"qdb_cluster":[9066]},"id":11}]} 2024-04-24T23:11:09.697 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 80 2024-04-24T23:11:09.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.696+0000 7fe3500f5640 1 -- 172.21.15.32:0/2774633728 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe32c06be30 msgr2=0x7fe32c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:09.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.696+0000 7fe3500f5640 1 --2- 172.21.15.32:0/2774633728 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe32c06be30 0x7fe32c06e2f0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fe344034000 tx=0x7fe34402c000 comp rx=0 tx=0).stop 2024-04-24T23:11:09.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.696+0000 7fe3500f5640 1 -- 172.21.15.32:0/2774633728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3480d4210 msgr2=0x7fe3480ccc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:09.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.696+0000 7fe3500f5640 1 --2- 172.21.15.32:0/2774633728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3480d4210 0x7fe3480ccc30 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fe33800b4a0 tx=0x7fe33800b960 comp rx=0 tx=0).stop 2024-04-24T23:11:09.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.697+0000 7fe3500f5640 1 -- 172.21.15.32:0/2774633728 shutdown_connections 2024-04-24T23:11:09.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.697+0000 7fe3500f5640 1 --2- 172.21.15.32:0/2774633728 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe32c06be30 0x7fe32c06e2f0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:09.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.697+0000 7fe3500f5640 1 --2- 172.21.15.32:0/2774633728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3480d4210 0x7fe3480ccc30 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:09.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.697+0000 7fe3500f5640 1 --2- 172.21.15.32:0/2774633728 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe34815e9b0 0x7fe3480c9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:09.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.697+0000 7fe3500f5640 1 --2- 172.21.15.32:0/2774633728 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3480ba0e0 0x7fe3481f73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:09.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.697+0000 7fe3500f5640 1 -- 172.21.15.32:0/2774633728 >> 172.21.15.32:0/2774633728 conn(0x7fe3480d11d0 msgr2=0x7fe3480720b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:09.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.697+0000 7fe3500f5640 1 -- 172.21.15.32:0/2774633728 shutdown_connections 2024-04-24T23:11:09.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:09.697+0000 7fe3500f5640 1 -- 172.21.15.32:0/2774633728 wait complete. 2024-04-24T23:11:09.709 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-04-24T23:11:09.829 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","whoami":0,"id":9066,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":32.825260966000002,"mdsmap_epoch":80,"osdmap_epoch":190,"osdmap_epoch_barrier":189,"uptime":43.853778695999999} 2024-04-24T23:11:09.830 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "eb1b7a01-6f65-4534-9a12-83011862ba51", "fs_name": "cephfs", "id": 9066, "mdsmap_epoch": 80, "osdmap_epoch": 190, "osdmap_epoch_barrier": 189, "rank_uptime": 32.825260966, "state": "up:active", "uptime": 43.853778696, "want_state": "up:active", "whoami": 0 } 2024-04-24T23:11:09.830 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-24T23:11:09.830 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:11:09.830 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:11:09.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:11:09.846 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2024-04-24T23:11:10.082 INFO:teuthology.orchestra.run.smithi032.stderr:50+0 records in 2024-04-24T23:11:10.082 INFO:teuthology.orchestra.run.smithi032.stderr:50+0 records out 2024-04-24T23:11:10.082 INFO:teuthology.orchestra.run.smithi032.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.180063 s, 291 MB/s 2024-04-24T23:11:10.083 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-24T23:11:10.083 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:11:10.138 DEBUG:teuthology.orchestra.run.smithi121:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:11:10.179 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-24T23:11:10.179 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:11:10.180 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:11:10.180 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:11:10.196 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:11:10.196 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:11:10.252 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:11:10.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.390+0000 7fc35d7d5640 1 Processor -- start 2024-04-24T23:11:10.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.390+0000 7fc35d7d5640 1 -- start start 2024-04-24T23:11:10.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.391+0000 7fc35d7d5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc358158dd0 0x7fc35806b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:10.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.391+0000 7fc35d7d5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc35806b860 0x7fc35806d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:10.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.391+0000 7fc35d7d5640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3580b6d10 con 0x7fc35806da90 2024-04-24T23:11:10.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.391+0000 7fc35d7d5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fc35815ef10 con 0x7fc35806b860 2024-04-24T23:11:10.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.391+0000 7fc35d7d5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc35815f1f0 con 0x7fc358158dd0 2024-04-24T23:11:10.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.391+0000 7fc3567fc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc35806b860 0x7fc35806d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:10.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.391+0000 7fc3577fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc35806da90 0x7fc358072580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35856/0 (socket says 172.21.15.32:35856) 2024-04-24T23:11:10.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.391+0000 7fc3567fc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc35806b860 0x7fc35806d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49086/0 (socket says 172.21.15.32:49086) 2024-04-24T23:11:10.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.391+0000 7fc3577fe640 1 -- 172.21.15.32:0/2607192360 learned_addr learned my addr 172.21.15.32:0/2607192360 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:11:10.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.391+0000 7fc356ffd640 1 --2- 172.21.15.32:0/2607192360 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc358158dd0 0x7fc35806b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:10.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.391+0000 7fc3567fc640 1 -- 172.21.15.32:0/2607192360 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc358158dd0 msgr2=0x7fc35806b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:10.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.391+0000 7fc3567fc640 1 --2- 172.21.15.32:0/2607192360 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc358158dd0 0x7fc35806b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:10.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.391+0000 7fc3567fc640 1 -- 172.21.15.32:0/2607192360 >> v1:172.21.15.32:6789/0 conn(0x7fc35806da90 legacy=0x7fc358072580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:10.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.392+0000 7fc3567fc640 1 -- 172.21.15.32:0/2607192360 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc35815f4d0 con 0x7fc35806b860 2024-04-24T23:11:10.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.392+0000 7fc355ffb640 1 -- 172.21.15.32:0/2607192360 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1356528469 0 0) 0x7fc3580b6d10 con 0x7fc35806da90 2024-04-24T23:11:10.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.392+0000 7fc3567fc640 1 --2- 172.21.15.32:0/2607192360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc35806b860 0x7fc35806d550 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fc34c0097a0 tx=0x7fc34c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5109a350db5d0223 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:10.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.392+0000 7fc355ffb640 1 -- 172.21.15.32:0/2607192360 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34c037070 con 0x7fc35806b860 2024-04-24T23:11:10.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.392+0000 7fc355ffb640 1 -- 172.21.15.32:0/2607192360 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc34c032440 con 0x7fc35806b860 2024-04-24T23:11:10.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.392+0000 7fc355ffb640 1 -- 172.21.15.32:0/2607192360 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34c032810 con 0x7fc35806b860 2024-04-24T23:11:10.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.393+0000 7fc35d7d5640 1 -- 172.21.15.32:0/2607192360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc35806b860 msgr2=0x7fc35806d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:10.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.393+0000 7fc35d7d5640 1 --2- 172.21.15.32:0/2607192360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc35806b860 0x7fc35806d550 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fc34c0097a0 tx=0x7fc34c02a180 comp rx=0 tx=0).stop 2024-04-24T23:11:10.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.393+0000 7fc35d7d5640 1 -- 172.21.15.32:0/2607192360 shutdown_connections 2024-04-24T23:11:10.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.393+0000 7fc35d7d5640 1 --2- 172.21.15.32:0/2607192360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc35806b860 0x7fc35806d550 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:10.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.393+0000 7fc35d7d5640 1 --2- 172.21.15.32:0/2607192360 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc358158dd0 0x7fc35806b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:10.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.393+0000 7fc35d7d5640 1 -- 172.21.15.32:0/2607192360 >> 172.21.15.32:0/2607192360 conn(0x7fc35806e800 msgr2=0x7fc35806ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:10.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.393+0000 7fc35d7d5640 1 -- 172.21.15.32:0/2607192360 shutdown_connections 2024-04-24T23:11:10.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.393+0000 7fc35d7d5640 1 -- 172.21.15.32:0/2607192360 wait complete. 2024-04-24T23:11:10.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.394+0000 7fc35d7d5640 1 Processor -- start 2024-04-24T23:11:10.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.394+0000 7fc35d7d5640 1 -- start start 2024-04-24T23:11:10.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.395+0000 7fc35d7d5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc35806b860 0x7fc35814df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:10.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.395+0000 7fc35d7d5640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc35806da90 0x7fc3581578e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:10.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.395+0000 7fc35d7d5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc358158dd0 0x7fc358160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:10.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.395+0000 7fc35d7d5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc35815fcd0 con 0x7fc358158dd0 2024-04-24T23:11:10.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.395+0000 7fc35d7d5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc35806bce0 con 0x7fc35806b860 2024-04-24T23:11:10.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.395+0000 7fc35d7d5640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc35806ded0 con 0x7fc35806da90 2024-04-24T23:11:10.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.395+0000 7fc3577fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc358158dd0 0x7fc358160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:10.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.395+0000 7fc3577fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc358158dd0 0x7fc358160ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49102/0 (socket says 172.21.15.32:49102) 2024-04-24T23:11:10.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.395+0000 7fc3577fe640 1 -- 172.21.15.32:0/1912485625 learned_addr learned my addr 172.21.15.32:0/1912485625 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:10.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.395+0000 7fc356ffd640 1 --2- 172.21.15.32:0/1912485625 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc35806b860 0x7fc35814df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:10.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.395+0000 7fc3567fc640 1 --2- 172.21.15.32:0/1912485625 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc35806da90 0x7fc3581578e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:10.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.396+0000 7fc3577fe640 1 -- 172.21.15.32:0/1912485625 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc35806da90 msgr2=0x7fc3581578e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:10.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.396+0000 7fc3577fe640 1 --2- 172.21.15.32:0/1912485625 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc35806da90 0x7fc3581578e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:10.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.396+0000 7fc3577fe640 1 -- 172.21.15.32:0/1912485625 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc35806b860 msgr2=0x7fc35814df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:10.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.396+0000 7fc3577fe640 1 --2- 172.21.15.32:0/1912485625 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc35806b860 0x7fc35814df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:10.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.396+0000 7fc3577fe640 1 -- 172.21.15.32:0/1912485625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3581585a0 con 0x7fc358158dd0 2024-04-24T23:11:10.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.396+0000 7fc356ffd640 1 --2- 172.21.15.32:0/1912485625 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc35806b860 0x7fc35814df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:11:10.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.396+0000 7fc3577fe640 1 --2- 172.21.15.32:0/1912485625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc358158dd0 0x7fc358160ae0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fc35000c7d0 tx=0x7fc35000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:10.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.396+0000 7fc33f7fe640 1 -- 172.21.15.32:0/1912485625 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc350018070 con 0x7fc358158dd0 2024-04-24T23:11:10.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.396+0000 7fc33f7fe640 1 -- 172.21.15.32:0/1912485625 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc35000a5e0 con 0x7fc358158dd0 2024-04-24T23:11:10.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.396+0000 7fc35d7d5640 1 -- 172.21.15.32:0/1912485625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3580b2500 con 0x7fc358158dd0 2024-04-24T23:11:10.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.396+0000 7fc33f7fe640 1 -- 172.21.15.32:0/1912485625 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc35000aa10 con 0x7fc358158dd0 2024-04-24T23:11:10.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.396+0000 7fc35d7d5640 1 -- 172.21.15.32:0/1912485625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3580b2ac0 con 0x7fc358158dd0 2024-04-24T23:11:10.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.397+0000 7fc35d7d5640 1 -- 172.21.15.32:0/1912485625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc358068430 con 0x7fc358158dd0 2024-04-24T23:11:10.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.397+0000 7fc33f7fe640 1 -- 172.21.15.32:0/1912485625 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc35001b070 con 0x7fc358158dd0 2024-04-24T23:11:10.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.398+0000 7fc33f7fe640 1 --2- 172.21.15.32:0/1912485625 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc34406be30 0x7fc34406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:10.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.398+0000 7fc33f7fe640 1 -- 172.21.15.32:0/1912485625 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(190..190 src has 1..190) ==== 5738+0+0 (secure 0 0 0) 0x7fc35008a400 con 0x7fc358158dd0 2024-04-24T23:11:10.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.398+0000 7fc356ffd640 1 --2- 172.21.15.32:0/1912485625 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc34406be30 0x7fc34406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:10.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.399+0000 7fc356ffd640 1 --2- 172.21.15.32:0/1912485625 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc34406be30 0x7fc34406e2f0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fc348006fd0 tx=0x7fc348008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:10.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.402+0000 7fc33f7fe640 1 -- 172.21.15.32:0/1912485625 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc3581585a0 con 0x7fc358158dd0 2024-04-24T23:11:10.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.586+0000 7fc35d7d5640 1 -- 172.21.15.32:0/1912485625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc358074e60 con 0x7fc358158dd0 2024-04-24T23:11:10.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.587+0000 7fc33f7fe640 1 -- 172.21.15.32:0/1912485625 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v190) ==== 74+0+16748 (secure 0 0 0) 0x7fc350058d90 con 0x7fc358158dd0 2024-04-24T23:11:10.589 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:11:10.589 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":190,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:11:06.861172+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:10:34.649562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-24T23:10:35.339604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"190","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"23.6","mappings":[{"from":5,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:11:10.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.589+0000 7fc35d7d5640 1 -- 172.21.15.32:0/1912485625 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc34406be30 msgr2=0x7fc34406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:10.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.589+0000 7fc35d7d5640 1 --2- 172.21.15.32:0/1912485625 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc34406be30 0x7fc34406e2f0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fc348006fd0 tx=0x7fc348008040 comp rx=0 tx=0).stop 2024-04-24T23:11:10.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.589+0000 7fc35d7d5640 1 -- 172.21.15.32:0/1912485625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc358158dd0 msgr2=0x7fc358160ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:10.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.589+0000 7fc35d7d5640 1 --2- 172.21.15.32:0/1912485625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc358158dd0 0x7fc358160ae0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fc35000c7d0 tx=0x7fc35000cc90 comp rx=0 tx=0).stop 2024-04-24T23:11:10.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.589+0000 7fc35d7d5640 1 -- 172.21.15.32:0/1912485625 shutdown_connections 2024-04-24T23:11:10.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.589+0000 7fc35d7d5640 1 --2- 172.21.15.32:0/1912485625 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc34406be30 0x7fc34406e2f0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:10.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.589+0000 7fc35d7d5640 1 --2- 172.21.15.32:0/1912485625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc358158dd0 0x7fc358160ae0 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:10.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.589+0000 7fc35d7d5640 1 --2- 172.21.15.32:0/1912485625 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc35806da90 0x7fc3581578e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:10.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.589+0000 7fc35d7d5640 1 --2- 172.21.15.32:0/1912485625 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc35806b860 0x7fc35814df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:10.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.589+0000 7fc35d7d5640 1 -- 172.21.15.32:0/1912485625 >> 172.21.15.32:0/1912485625 conn(0x7fc35806e800 msgr2=0x7fc3581525b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:10.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.589+0000 7fc35d7d5640 1 -- 172.21.15.32:0/1912485625 shutdown_connections 2024-04-24T23:11:10.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.589+0000 7fc35d7d5640 1 -- 172.21.15.32:0/1912485625 wait complete. 2024-04-24T23:11:10.602 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:10:34.649562+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '184', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:10:35.339604+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '190', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:11:10.602 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:11:10.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.703+0000 7fa070b89640 1 Processor -- start 2024-04-24T23:11:10.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.704+0000 7fa070b89640 1 -- start start 2024-04-24T23:11:10.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.704+0000 7fa070b89640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa06c15c5b0 0x7fa06c0b4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:10.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.705+0000 7fa070b89640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa06c15c990 con 0x7fa06c072cf0 2024-04-24T23:11:10.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.705+0000 7fa070b89640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa06c071c50 con 0x7fa06c06cd00 2024-04-24T23:11:10.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.705+0000 7fa070b89640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fa06c072860 con 0x7fa06c15c5b0 2024-04-24T23:11:10.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.705+0000 7fa069d9b640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa06c072cf0 0x7fa06c06c5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35862/0 (socket says 172.21.15.32:35862) 2024-04-24T23:11:10.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.705+0000 7fa069d9b640 1 -- 172.21.15.32:0/3615190722 learned_addr learned my addr 172.21.15.32:0/3615190722 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:11:10.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.705+0000 7fa06959a640 1 -- 172.21.15.32:0/3615190722 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1706814592 0 0) 0x7fa054001110 con 0x7fa06c072cf0 2024-04-24T23:11:10.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.705+0000 7fa06a59c640 1 --2- 172.21.15.32:0/3615190722 >> v2:172.21.15.121:3300/0 conn(0x7fa06c15c5b0 0x7fa06c0b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:10.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.705+0000 7fa06959a640 1 -- 172.21.15.32:0/3615190722 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa058003520 con 0x7fa06c072cf0 2024-04-24T23:11:10.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.705+0000 7fa06959a640 1 -- 172.21.15.32:0/3615190722 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1700839329 0 0) 0x7fa058003520 con 0x7fa06c072cf0 2024-04-24T23:11:10.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.705+0000 7fa06959a640 1 -- 172.21.15.32:0/3615190722 >> v2:172.21.15.121:3300/0 conn(0x7fa06c15c5b0 msgr2=0x7fa06c0b4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:10.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.705+0000 7fa06959a640 1 --2- 172.21.15.32:0/3615190722 >> v2:172.21.15.121:3300/0 conn(0x7fa06c15c5b0 0x7fa06c0b4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:10.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.705+0000 7fa06959a640 1 -- 172.21.15.32:0/3615190722 >> v1:172.21.15.121:6789/0 conn(0x7fa06c06cd00 legacy=0x7fa06c0b4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:10.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.705+0000 7fa06959a640 1 -- 172.21.15.32:0/3615190722 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa06c071f30 con 0x7fa06c072cf0 2024-04-24T23:11:10.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.705+0000 7fa06959a640 1 -- 172.21.15.32:0/3615190722 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7fa054001840 con 0x7fa06c072cf0 2024-04-24T23:11:10.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.705+0000 7fa06959a640 1 -- 172.21.15.32:0/3615190722 >> v1:172.21.15.32:6789/0 conn(0x7fa06c072cf0 legacy=0x7fa06c06c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:10.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.705+0000 7fa06959a640 1 --2- 172.21.15.32:0/3615190722 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa0580039b0 0x7fa058003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:10.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.705+0000 7fa06959a640 1 --2- 172.21.15.32:0/3615190722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0580043b0 0x7fa058004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:10.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.705+0000 7fa06959a640 1 --2- 172.21.15.32:0/3615190722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa058005120 0x7fa05800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:10.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.705+0000 7fa06959a640 1 -- 172.21.15.32:0/3615190722 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa06c072860 con 0x7fa0580043b0 2024-04-24T23:11:10.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.705+0000 7fa06959a640 1 -- 172.21.15.32:0/3615190722 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa05800c060 con 0x7fa058005120 2024-04-24T23:11:10.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.705+0000 7fa06959a640 1 -- 172.21.15.32:0/3615190722 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa05800c260 con 0x7fa0580039b0 2024-04-24T23:11:10.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.705+0000 7fa069d9b640 1 --2- 172.21.15.32:0/3615190722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0580043b0 0x7fa058004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:10.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.706+0000 7fa06a59c640 1 --2- 172.21.15.32:0/3615190722 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa0580039b0 0x7fa058003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:10.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.706+0000 7fa06ad9d640 1 --2- 172.21.15.32:0/3615190722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa058005120 0x7fa05800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:10.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.706+0000 7fa069d9b640 1 -- 172.21.15.32:0/3615190722 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa0580039b0 msgr2=0x7fa058003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:10.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.706+0000 7fa069d9b640 1 --2- 172.21.15.32:0/3615190722 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa0580039b0 0x7fa058003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:10.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.706+0000 7fa069d9b640 1 -- 172.21.15.32:0/3615190722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa058005120 msgr2=0x7fa05800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:10.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.706+0000 7fa069d9b640 1 --2- 172.21.15.32:0/3615190722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa058005120 0x7fa05800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:10.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.706+0000 7fa069d9b640 1 -- 172.21.15.32:0/3615190722 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa058003520 con 0x7fa0580043b0 2024-04-24T23:11:10.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.706+0000 7fa06ad9d640 1 -- 172.21.15.32:0/3615190722 reap_dead start 2024-04-24T23:11:10.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.706+0000 7fa069d9b640 1 --2- 172.21.15.32:0/3615190722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0580043b0 0x7fa058004880 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fa05400c4c0 tx=0x7fa058005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d616c9458b9c037 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:10.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.707+0000 7fa06959a640 1 -- 172.21.15.32:0/3615190722 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05400c0e0 con 0x7fa0580043b0 2024-04-24T23:11:10.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.707+0000 7fa06959a640 1 -- 172.21.15.32:0/3615190722 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa054034070 con 0x7fa0580043b0 2024-04-24T23:11:10.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.707+0000 7fa06959a640 1 -- 172.21.15.32:0/3615190722 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa054005d50 con 0x7fa0580043b0 2024-04-24T23:11:10.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.707+0000 7fa070b89640 1 -- 172.21.15.32:0/3615190722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0580043b0 msgr2=0x7fa058004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:10.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.707+0000 7fa070b89640 1 --2- 172.21.15.32:0/3615190722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0580043b0 0x7fa058004880 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fa05400c4c0 tx=0x7fa058005830 comp rx=0 tx=0).stop 2024-04-24T23:11:10.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.707+0000 7fa070b89640 1 -- 172.21.15.32:0/3615190722 shutdown_connections 2024-04-24T23:11:10.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.707+0000 7fa070b89640 1 --2- 172.21.15.32:0/3615190722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0580043b0 0x7fa058004880 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:10.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.707+0000 7fa070b89640 1 -- 172.21.15.32:0/3615190722 >> 172.21.15.32:0/3615190722 conn(0x7fa06c06b110 msgr2=0x7fa06c06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:10.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.707+0000 7fa070b89640 1 -- 172.21.15.32:0/3615190722 shutdown_connections 2024-04-24T23:11:10.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.707+0000 7fa070b89640 1 -- 172.21.15.32:0/3615190722 wait complete. 2024-04-24T23:11:10.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.709+0000 7fa070b89640 1 Processor -- start 2024-04-24T23:11:10.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.709+0000 7fa070b89640 1 -- start start 2024-04-24T23:11:10.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.709+0000 7fa070b89640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa0580043b0 0x7fa06c0b4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:10.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.709+0000 7fa070b89640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa06c0b30e0 0x7fa06c151000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:10.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.709+0000 7fa070b89640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa06c0b5810 0x7fa06c142860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:10.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.709+0000 7fa070b89640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa06c071770 con 0x7fa06c0b5810 2024-04-24T23:11:10.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.709+0000 7fa070b89640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa06c0724f0 con 0x7fa06c0b30e0 2024-04-24T23:11:10.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.709+0000 7fa070b89640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fa06c0718f0 con 0x7fa0580043b0 2024-04-24T23:11:10.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.709+0000 7fa06ad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa06c0b5810 0x7fa06c142860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:10.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.709+0000 7fa06ad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa06c0b5810 0x7fa06c142860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49112/0 (socket says 172.21.15.32:49112) 2024-04-24T23:11:10.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.709+0000 7fa06ad9d640 1 -- 172.21.15.32:0/1535086358 learned_addr learned my addr 172.21.15.32:0/1535086358 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:10.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.710+0000 7fa069d9b640 1 --2- 172.21.15.32:0/1535086358 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa06c0b30e0 0x7fa06c151000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:10.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.710+0000 7fa06a59c640 1 --2- 172.21.15.32:0/1535086358 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa0580043b0 0x7fa06c0b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:10.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.710+0000 7fa06ad9d640 1 -- 172.21.15.32:0/1535086358 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa0580043b0 msgr2=0x7fa06c0b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:10.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.710+0000 7fa06ad9d640 1 --2- 172.21.15.32:0/1535086358 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa0580043b0 0x7fa06c0b4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:10.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.710+0000 7fa06ad9d640 1 -- 172.21.15.32:0/1535086358 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa06c0b30e0 msgr2=0x7fa06c151000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:10.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.710+0000 7fa06ad9d640 1 --2- 172.21.15.32:0/1535086358 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa06c0b30e0 0x7fa06c151000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:10.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.710+0000 7fa06ad9d640 1 -- 172.21.15.32:0/1535086358 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa06c0b35a0 con 0x7fa06c0b5810 2024-04-24T23:11:10.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.710+0000 7fa069d9b640 1 --2- 172.21.15.32:0/1535086358 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa06c0b30e0 0x7fa06c151000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:11:10.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.710+0000 7fa06a59c640 1 --2- 172.21.15.32:0/1535086358 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa0580043b0 0x7fa06c0b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:11:10.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.710+0000 7fa06ad9d640 1 --2- 172.21.15.32:0/1535086358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa06c0b5810 0x7fa06c142860 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7fa06c0712e0 tx=0x7fa05c00fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:10.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.710+0000 7fa062ffd640 1 -- 172.21.15.32:0/1535086358 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c019070 con 0x7fa06c0b5810 2024-04-24T23:11:10.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.710+0000 7fa062ffd640 1 -- 172.21.15.32:0/1535086358 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa05c006e20 con 0x7fa06c0b5810 2024-04-24T23:11:10.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.710+0000 7fa070b89640 1 -- 172.21.15.32:0/1535086358 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa06c0b3880 con 0x7fa06c0b5810 2024-04-24T23:11:10.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.710+0000 7fa062ffd640 1 -- 172.21.15.32:0/1535086358 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c003b90 con 0x7fa06c0b5810 2024-04-24T23:11:10.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.710+0000 7fa070b89640 1 -- 172.21.15.32:0/1535086358 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa06c0b3e40 con 0x7fa06c0b5810 2024-04-24T23:11:10.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.711+0000 7fa070b89640 1 -- 172.21.15.32:0/1535086358 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa06c068430 con 0x7fa06c0b5810 2024-04-24T23:11:10.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.712+0000 7fa062ffd640 1 -- 172.21.15.32:0/1535086358 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa05c01c070 con 0x7fa06c0b5810 2024-04-24T23:11:10.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.712+0000 7fa062ffd640 1 --2- 172.21.15.32:0/1535086358 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa04c06be60 0x7fa04c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:10.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.712+0000 7fa062ffd640 1 -- 172.21.15.32:0/1535086358 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(190..190 src has 1..190) ==== 5738+0+0 (secure 0 0 0) 0x7fa05c08a1a0 con 0x7fa06c0b5810 2024-04-24T23:11:10.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.712+0000 7fa06a59c640 1 --2- 172.21.15.32:0/1535086358 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa04c06be60 0x7fa04c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:10.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.713+0000 7fa06a59c640 1 --2- 172.21.15.32:0/1535086358 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa04c06be60 0x7fa04c06e320 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fa06c0740e0 tx=0x7fa064008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:10.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.716+0000 7fa062ffd640 1 -- 172.21.15.32:0/1535086358 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa05c058b30 con 0x7fa06c0b5810 2024-04-24T23:11:10.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.899+0000 7fa070b89640 1 -- 172.21.15.32:0/1535086358 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa06c15a460 con 0x7fa06c0b5810 2024-04-24T23:11:10.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.900+0000 7fa062ffd640 1 -- 172.21.15.32:0/1535086358 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v190) ==== 93+0+10 (secure 0 0 0) 0x7fa05c058430 con 0x7fa06c0b5810 2024-04-24T23:11:10.902 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T23:11:10.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.902+0000 7fa070b89640 1 -- 172.21.15.32:0/1535086358 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa04c06be60 msgr2=0x7fa04c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:10.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.902+0000 7fa070b89640 1 --2- 172.21.15.32:0/1535086358 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa04c06be60 0x7fa04c06e320 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fa06c0740e0 tx=0x7fa064008040 comp rx=0 tx=0).stop 2024-04-24T23:11:10.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.902+0000 7fa070b89640 1 -- 172.21.15.32:0/1535086358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa06c0b5810 msgr2=0x7fa06c142860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:10.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.902+0000 7fa070b89640 1 --2- 172.21.15.32:0/1535086358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa06c0b5810 0x7fa06c142860 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7fa06c0712e0 tx=0x7fa05c00fd40 comp rx=0 tx=0).stop 2024-04-24T23:11:10.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.902+0000 7fa070b89640 1 -- 172.21.15.32:0/1535086358 shutdown_connections 2024-04-24T23:11:10.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.902+0000 7fa070b89640 1 --2- 172.21.15.32:0/1535086358 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fa04c06be60 0x7fa04c06e320 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:10.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.902+0000 7fa070b89640 1 --2- 172.21.15.32:0/1535086358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa06c0b5810 0x7fa06c142860 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:10.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.902+0000 7fa070b89640 1 --2- 172.21.15.32:0/1535086358 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa06c0b30e0 0x7fa06c151000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:10.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.902+0000 7fa070b89640 1 --2- 172.21.15.32:0/1535086358 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa0580043b0 0x7fa06c0b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:10.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.902+0000 7fa070b89640 1 -- 172.21.15.32:0/1535086358 >> 172.21.15.32:0/1535086358 conn(0x7fa06c06b110 msgr2=0x7fa06c1593f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:10.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.902+0000 7fa070b89640 1 -- 172.21.15.32:0/1535086358 shutdown_connections 2024-04-24T23:11:10.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:10.902+0000 7fa070b89640 1 -- 172.21.15.32:0/1535086358 wait complete. 2024-04-24T23:11:10.914 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:11:11.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.015+0000 7ffa03bdc640 1 Processor -- start 2024-04-24T23:11:11.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.016+0000 7ffa03bdc640 1 -- start start 2024-04-24T23:11:11.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.016+0000 7ffa03bdc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff9fc15c6f0 0x7ff9fc06c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:11.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.016+0000 7ffa03bdc640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff9fc06d2e0 0x7ff9fc06b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:11.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.017+0000 7ffa03bdc640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9fc0cf2f0 con 0x7ff9fc06d740 2024-04-24T23:11:11.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.017+0000 7ffa03bdc640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7ff9fc0afcd0 con 0x7ff9fc15c6f0 2024-04-24T23:11:11.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.017+0000 7ffa03bdc640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7ff9fc0d01d0 con 0x7ff9fc06d2e0 2024-04-24T23:11:11.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.017+0000 7ffa01978640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff9fc15c6f0 0x7ff9fc06c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:11.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.017+0000 7ffa01978640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff9fc15c6f0 0x7ff9fc06c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49126/0 (socket says 172.21.15.32:49126) 2024-04-24T23:11:11.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.017+0000 7ffa01978640 1 -- 172.21.15.32:0/1672321294 learned_addr learned my addr 172.21.15.32:0/1672321294 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:11.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.017+0000 7ffa01177640 1 --2- 172.21.15.32:0/1672321294 >> v2:172.21.15.121:3300/0 conn(0x7ff9fc06d2e0 0x7ff9fc06b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:11.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.017+0000 7ffa01978640 1 -- 172.21.15.32:0/1672321294 >> v2:172.21.15.121:3300/0 conn(0x7ff9fc06d2e0 msgr2=0x7ff9fc06b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:11.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.017+0000 7ffa01978640 1 --2- 172.21.15.32:0/1672321294 >> v2:172.21.15.121:3300/0 conn(0x7ff9fc06d2e0 0x7ff9fc06b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:11.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.017+0000 7ffa01978640 1 -- 172.21.15.32:0/1672321294 >> v1:172.21.15.121:6789/0 conn(0x7ff9fc06d740 legacy=0x7ff9fc070d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:11.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.017+0000 7ffa01978640 1 -- 172.21.15.32:0/1672321294 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9fc0cf590 con 0x7ff9fc15c6f0 2024-04-24T23:11:11.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.018+0000 7ffa01978640 1 --2- 172.21.15.32:0/1672321294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9fc15c6f0 0x7ff9fc06c7b0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc073170 tx=0x7ff9f802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b4ef7cccda2ec84 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:11.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.018+0000 7ffa00976640 1 -- 172.21.15.32:0/1672321294 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f8037070 con 0x7ff9fc15c6f0 2024-04-24T23:11:11.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.018+0000 7ffa00976640 1 -- 172.21.15.32:0/1672321294 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9f8032440 con 0x7ff9fc15c6f0 2024-04-24T23:11:11.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.018+0000 7ffa00976640 1 -- 172.21.15.32:0/1672321294 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f8032810 con 0x7ff9fc15c6f0 2024-04-24T23:11:11.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.018+0000 7ffa03bdc640 1 -- 172.21.15.32:0/1672321294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9fc15c6f0 msgr2=0x7ff9fc06c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:11.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.018+0000 7ffa03bdc640 1 --2- 172.21.15.32:0/1672321294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9fc15c6f0 0x7ff9fc06c7b0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc073170 tx=0x7ff9f802a1a0 comp rx=0 tx=0).stop 2024-04-24T23:11:11.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.018+0000 7ffa03bdc640 1 -- 172.21.15.32:0/1672321294 shutdown_connections 2024-04-24T23:11:11.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.018+0000 7ffa03bdc640 1 --2- 172.21.15.32:0/1672321294 >> v2:172.21.15.121:3300/0 conn(0x7ff9fc06d2e0 0x7ff9fc06b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:11.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.018+0000 7ffa03bdc640 1 --2- 172.21.15.32:0/1672321294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9fc15c6f0 0x7ff9fc06c7b0 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:11.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.018+0000 7ffa03bdc640 1 -- 172.21.15.32:0/1672321294 >> 172.21.15.32:0/1672321294 conn(0x7ff9fc0d0730 msgr2=0x7ff9fc0d0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:11.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.019+0000 7ffa03bdc640 1 -- 172.21.15.32:0/1672321294 shutdown_connections 2024-04-24T23:11:11.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.019+0000 7ffa03bdc640 1 -- 172.21.15.32:0/1672321294 wait complete. 2024-04-24T23:11:11.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.020+0000 7ffa03bdc640 1 Processor -- start 2024-04-24T23:11:11.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.020+0000 7ffa03bdc640 1 -- start start 2024-04-24T23:11:11.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.020+0000 7ffa03bdc640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9fc06d2e0 0x7ff9fc1f38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:11.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.020+0000 7ffa03bdc640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9fc06d740 0x7ff9fc157ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:11.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.020+0000 7ffa03bdc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9fc15c6f0 0x7ff9fc1f5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:11.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.020+0000 7ffa03bdc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff9fc0d04b0 con 0x7ff9fc15c6f0 2024-04-24T23:11:11.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.020+0000 7ffa03bdc640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff9fc15cad0 con 0x7ff9fc06d740 2024-04-24T23:11:11.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.020+0000 7ffa03bdc640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff9fc06ccf0 con 0x7ff9fc06d2e0 2024-04-24T23:11:11.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.020+0000 7ffa02179640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9fc15c6f0 0x7ff9fc1f5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:11.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.020+0000 7ffa02179640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9fc15c6f0 0x7ff9fc1f5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49136/0 (socket says 172.21.15.32:49136) 2024-04-24T23:11:11.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.020+0000 7ffa02179640 1 -- 172.21.15.32:0/883008174 learned_addr learned my addr 172.21.15.32:0/883008174 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:11.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.021+0000 7ffa01177640 1 --2- 172.21.15.32:0/883008174 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9fc06d740 0x7ff9fc157ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:11.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.021+0000 7ffa02179640 1 -- 172.21.15.32:0/883008174 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9fc06d2e0 msgr2=0x7ff9fc1f38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:11.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.021+0000 7ffa01978640 1 --2- 172.21.15.32:0/883008174 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9fc06d2e0 0x7ff9fc1f38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:11.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.021+0000 7ffa02179640 1 --2- 172.21.15.32:0/883008174 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9fc06d2e0 0x7ff9fc1f38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:11.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.021+0000 7ffa02179640 1 -- 172.21.15.32:0/883008174 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9fc06d740 msgr2=0x7ff9fc157ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:11.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.021+0000 7ffa02179640 1 --2- 172.21.15.32:0/883008174 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9fc06d740 0x7ff9fc157ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:11.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.021+0000 7ffa02179640 1 -- 172.21.15.32:0/883008174 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9fc1f36c0 con 0x7ff9fc15c6f0 2024-04-24T23:11:11.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.021+0000 7ffa01177640 1 --2- 172.21.15.32:0/883008174 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9fc06d740 0x7ff9fc157ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:11:11.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.021+0000 7ffa01978640 1 --2- 172.21.15.32:0/883008174 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9fc06d2e0 0x7ff9fc1f38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:11:11.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.021+0000 7ffa02179640 1 --2- 172.21.15.32:0/883008174 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9fc15c6f0 0x7ff9fc1f5050 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7ff9f400c780 tx=0x7ff9f400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:11.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.021+0000 7ff9f27fc640 1 -- 172.21.15.32:0/883008174 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f4004b60 con 0x7ff9fc15c6f0 2024-04-24T23:11:11.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.021+0000 7ff9f27fc640 1 -- 172.21.15.32:0/883008174 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9f4004d00 con 0x7ff9fc15c6f0 2024-04-24T23:11:11.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.021+0000 7ffa03bdc640 1 -- 172.21.15.32:0/883008174 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9fc14fb70 con 0x7ff9fc15c6f0 2024-04-24T23:11:11.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.021+0000 7ff9f27fc640 1 -- 172.21.15.32:0/883008174 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f400aa90 con 0x7ff9fc15c6f0 2024-04-24T23:11:11.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.021+0000 7ffa03bdc640 1 -- 172.21.15.32:0/883008174 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9fc150130 con 0x7ff9fc15c6f0 2024-04-24T23:11:11.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.022+0000 7ffa03bdc640 1 -- 172.21.15.32:0/883008174 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9fc068430 con 0x7ff9fc15c6f0 2024-04-24T23:11:11.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.023+0000 7ff9f27fc640 1 -- 172.21.15.32:0/883008174 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff9f400ac30 con 0x7ff9fc15c6f0 2024-04-24T23:11:11.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.023+0000 7ff9f27fc640 1 --2- 172.21.15.32:0/883008174 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff9e006be30 0x7ff9e006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:11.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.023+0000 7ff9f27fc640 1 -- 172.21.15.32:0/883008174 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(190..190 src has 1..190) ==== 5738+0+0 (secure 0 0 0) 0x7ff9f408b060 con 0x7ff9fc15c6f0 2024-04-24T23:11:11.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.023+0000 7ffa01978640 1 --2- 172.21.15.32:0/883008174 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff9e006be30 0x7ff9e006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:11.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.024+0000 7ffa01978640 1 --2- 172.21.15.32:0/883008174 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff9e006be30 0x7ff9e006e2f0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8034000 tx=0x7ff9f802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:11.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.027+0000 7ff9f27fc640 1 -- 172.21.15.32:0/883008174 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff9f4020070 con 0x7ff9fc15c6f0 2024-04-24T23:11:11.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.213+0000 7ffa03bdc640 1 -- 172.21.15.32:0/883008174 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff9fc15f170 con 0x7ff9fc15c6f0 2024-04-24T23:11:11.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.213+0000 7ff9f27fc640 1 -- 172.21.15.32:0/883008174 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v190) ==== 104+0+11 (secure 0 0 0) 0x7ff9f4059a80 con 0x7ff9fc15c6f0 2024-04-24T23:11:11.215 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-24T23:11:11.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.215+0000 7ffa03bdc640 1 -- 172.21.15.32:0/883008174 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff9e006be30 msgr2=0x7ff9e006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:11.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.215+0000 7ffa03bdc640 1 --2- 172.21.15.32:0/883008174 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff9e006be30 0x7ff9e006e2f0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8034000 tx=0x7ff9f802e000 comp rx=0 tx=0).stop 2024-04-24T23:11:11.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.215+0000 7ffa03bdc640 1 -- 172.21.15.32:0/883008174 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9fc15c6f0 msgr2=0x7ff9fc1f5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:11.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.215+0000 7ffa03bdc640 1 --2- 172.21.15.32:0/883008174 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9fc15c6f0 0x7ff9fc1f5050 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7ff9f400c780 tx=0x7ff9f400cc40 comp rx=0 tx=0).stop 2024-04-24T23:11:11.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.215+0000 7ffa03bdc640 1 -- 172.21.15.32:0/883008174 shutdown_connections 2024-04-24T23:11:11.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.215+0000 7ffa03bdc640 1 --2- 172.21.15.32:0/883008174 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff9e006be30 0x7ff9e006e2f0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:11.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.215+0000 7ffa03bdc640 1 --2- 172.21.15.32:0/883008174 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9fc15c6f0 0x7ff9fc1f5050 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:11.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.215+0000 7ffa03bdc640 1 --2- 172.21.15.32:0/883008174 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9fc06d740 0x7ff9fc157ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:11.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.215+0000 7ffa03bdc640 1 --2- 172.21.15.32:0/883008174 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9fc06d2e0 0x7ff9fc1f38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:11.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.215+0000 7ffa03bdc640 1 -- 172.21.15.32:0/883008174 >> 172.21.15.32:0/883008174 conn(0x7ff9fc0d0730 msgr2=0x7ff9fc15f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:11.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.215+0000 7ffa03bdc640 1 -- 172.21.15.32:0/883008174 shutdown_connections 2024-04-24T23:11:11.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.215+0000 7ffa03bdc640 1 -- 172.21.15.32:0/883008174 wait complete. 2024-04-24T23:11:11.227 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:11:11.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.329+0000 7f463cb90640 1 Processor -- start 2024-04-24T23:11:11.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.329+0000 7f463cb90640 1 -- start start 2024-04-24T23:11:11.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.330+0000 7f463cb90640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f46380d4210 0x7f46380d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:11.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.330+0000 7f463cb90640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46380d3170 con 0x7f463815e9b0 2024-04-24T23:11:11.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.330+0000 7f463cb90640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46380d3450 con 0x7f463806b820 2024-04-24T23:11:11.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.330+0000 7f463cb90640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f46380d3730 con 0x7f46380d4210 2024-04-24T23:11:11.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.330+0000 7f463659c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f463815e9b0 0x7f463806b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35870/0 (socket says 172.21.15.32:35870) 2024-04-24T23:11:11.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.330+0000 7f463659c640 1 -- 172.21.15.32:0/1167491092 learned_addr learned my addr 172.21.15.32:0/1167491092 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:11:11.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.331+0000 7f4636d9d640 1 --2- 172.21.15.32:0/1167491092 >> v2:172.21.15.121:3300/0 conn(0x7f46380d4210 0x7f46380d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:11.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.331+0000 7f463559a640 1 -- 172.21.15.32:0/1167491092 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1349854856 0 0) 0x7f46380d3170 con 0x7f463815e9b0 2024-04-24T23:11:11.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.331+0000 7f463559a640 1 -- 172.21.15.32:0/1167491092 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4624003520 con 0x7f463815e9b0 2024-04-24T23:11:11.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.331+0000 7f463559a640 1 -- 172.21.15.32:0/1167491092 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2909668440 0 0) 0x7f4624003520 con 0x7f463815e9b0 2024-04-24T23:11:11.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.331+0000 7f463559a640 1 -- 172.21.15.32:0/1167491092 >> v2:172.21.15.121:3300/0 conn(0x7f46380d4210 msgr2=0x7f46380d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:11.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.331+0000 7f463559a640 1 --2- 172.21.15.32:0/1167491092 >> v2:172.21.15.121:3300/0 conn(0x7f46380d4210 0x7f46380d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:11.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.331+0000 7f463559a640 1 -- 172.21.15.32:0/1167491092 >> v1:172.21.15.121:6789/0 conn(0x7f463806b820 legacy=0x7f46380b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:11.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.331+0000 7f463559a640 1 -- 172.21.15.32:0/1167491092 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46380d3a10 con 0x7f463815e9b0 2024-04-24T23:11:11.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.331+0000 7f463559a640 1 -- 172.21.15.32:0/1167491092 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f4630001670 con 0x7f463815e9b0 2024-04-24T23:11:11.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.331+0000 7f463559a640 1 -- 172.21.15.32:0/1167491092 >> v1:172.21.15.32:6789/0 conn(0x7f463815e9b0 legacy=0x7f463806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:11.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.331+0000 7f463559a640 1 --2- 172.21.15.32:0/1167491092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46240039b0 0x7f4624003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:11.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.331+0000 7f463559a640 1 --2- 172.21.15.32:0/1167491092 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4624004800 0x7f4624004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:11.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.331+0000 7f463559a640 1 --2- 172.21.15.32:0/1167491092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46240051c0 0x7f462400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:11.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.331+0000 7f463559a640 1 -- 172.21.15.32:0/1167491092 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f46380d3730 con 0x7f46240039b0 2024-04-24T23:11:11.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.331+0000 7f463559a640 1 -- 172.21.15.32:0/1167491092 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f462400c160 con 0x7f46240051c0 2024-04-24T23:11:11.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.331+0000 7f463559a640 1 -- 172.21.15.32:0/1167491092 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f462400c360 con 0x7f4624004800 2024-04-24T23:11:11.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.331+0000 7f463659c640 1 --2- 172.21.15.32:0/1167491092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46240039b0 0x7f4624003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:11.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.331+0000 7f4636d9d640 1 --2- 172.21.15.32:0/1167491092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46240051c0 0x7f462400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:11.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.331+0000 7f4635d9b640 1 --2- 172.21.15.32:0/1167491092 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4624004800 0x7f4624004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:11.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.332+0000 7f463659c640 1 -- 172.21.15.32:0/1167491092 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4624004800 msgr2=0x7f4624004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:11.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.332+0000 7f463659c640 1 --2- 172.21.15.32:0/1167491092 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4624004800 0x7f4624004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:11.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.332+0000 7f463659c640 1 -- 172.21.15.32:0/1167491092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46240051c0 msgr2=0x7f462400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:11.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.332+0000 7f463659c640 1 --2- 172.21.15.32:0/1167491092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46240051c0 0x7f462400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:11.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.332+0000 7f463659c640 1 -- 172.21.15.32:0/1167491092 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4624003520 con 0x7f46240039b0 2024-04-24T23:11:11.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.332+0000 7f4636d9d640 1 -- 172.21.15.32:0/1167491092 reap_dead start 2024-04-24T23:11:11.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.332+0000 7f463659c640 1 --2- 172.21.15.32:0/1167491092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46240039b0 0x7f4624003dd0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f463000c240 tx=0x7f4624005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38cbcb7d340ac856 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:11.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.332+0000 7f463559a640 1 -- 172.21.15.32:0/1167491092 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4630007cf0 con 0x7f46240039b0 2024-04-24T23:11:11.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.332+0000 7f463559a640 1 -- 172.21.15.32:0/1167491092 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4630005af0 con 0x7f46240039b0 2024-04-24T23:11:11.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.332+0000 7f463559a640 1 -- 172.21.15.32:0/1167491092 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4630007070 con 0x7f46240039b0 2024-04-24T23:11:11.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.333+0000 7f463cb90640 1 -- 172.21.15.32:0/1167491092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46240039b0 msgr2=0x7f4624003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:11.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.333+0000 7f463cb90640 1 --2- 172.21.15.32:0/1167491092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46240039b0 0x7f4624003dd0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f463000c240 tx=0x7f4624005830 comp rx=0 tx=0).stop 2024-04-24T23:11:11.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.333+0000 7f463cb90640 1 -- 172.21.15.32:0/1167491092 shutdown_connections 2024-04-24T23:11:11.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.333+0000 7f463cb90640 1 --2- 172.21.15.32:0/1167491092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46240039b0 0x7f4624003dd0 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:11.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.333+0000 7f463cb90640 1 -- 172.21.15.32:0/1167491092 >> 172.21.15.32:0/1167491092 conn(0x7f46380d11d0 msgr2=0x7f46380d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:11.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.333+0000 7f463cb90640 1 -- 172.21.15.32:0/1167491092 shutdown_connections 2024-04-24T23:11:11.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.333+0000 7f463cb90640 1 -- 172.21.15.32:0/1167491092 wait complete. 2024-04-24T23:11:11.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.334+0000 7f463cb90640 1 Processor -- start 2024-04-24T23:11:11.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.334+0000 7f463cb90640 1 -- start start 2024-04-24T23:11:11.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.335+0000 7f463cb90640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46240039b0 0x7f4638156820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:11.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.335+0000 7f463cb90640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46381618b0 0x7f46381f4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:11.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.335+0000 7f463cb90640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4638154610 0x7f463815f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:11.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.335+0000 7f463cb90640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f46380ba0e0 con 0x7f4638154610 2024-04-24T23:11:11.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.335+0000 7f463cb90640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f46380d2400 con 0x7f46381618b0 2024-04-24T23:11:11.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.335+0000 7f463cb90640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f46380ba260 con 0x7f46240039b0 2024-04-24T23:11:11.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.335+0000 7f4636d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4638154610 0x7f463815f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:11.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.335+0000 7f4636d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4638154610 0x7f463815f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49154/0 (socket says 172.21.15.32:49154) 2024-04-24T23:11:11.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.335+0000 7f4636d9d640 1 -- 172.21.15.32:0/3699303064 learned_addr learned my addr 172.21.15.32:0/3699303064 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:11.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.335+0000 7f4635d9b640 1 --2- 172.21.15.32:0/3699303064 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46381618b0 0x7f46381f4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:11.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.335+0000 7f4636d9d640 1 -- 172.21.15.32:0/3699303064 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46240039b0 msgr2=0x7f4638156820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:11.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.335+0000 7f463659c640 1 --2- 172.21.15.32:0/3699303064 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46240039b0 0x7f4638156820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:11.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.336+0000 7f4636d9d640 1 --2- 172.21.15.32:0/3699303064 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46240039b0 0x7f4638156820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:11.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.336+0000 7f4636d9d640 1 -- 172.21.15.32:0/3699303064 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46381618b0 msgr2=0x7f46381f4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:11.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.336+0000 7f4636d9d640 1 --2- 172.21.15.32:0/3699303064 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46381618b0 0x7f46381f4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:11.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.336+0000 7f4636d9d640 1 -- 172.21.15.32:0/3699303064 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46381550c0 con 0x7f4638154610 2024-04-24T23:11:11.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.336+0000 7f4635d9b640 1 --2- 172.21.15.32:0/3699303064 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46381618b0 0x7f46381f4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:11:11.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.336+0000 7f463659c640 1 --2- 172.21.15.32:0/3699303064 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46240039b0 0x7f4638156820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:11:11.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.336+0000 7f4636d9d640 1 --2- 172.21.15.32:0/3699303064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4638154610 0x7f463815f7b0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f46380d0930 tx=0x7f462800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:11.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.336+0000 7f462effd640 1 -- 172.21.15.32:0/3699303064 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462800fd80 con 0x7f4638154610 2024-04-24T23:11:11.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.336+0000 7f462effd640 1 -- 172.21.15.32:0/3699303064 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f462801d070 con 0x7f4638154610 2024-04-24T23:11:11.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.336+0000 7f463cb90640 1 -- 172.21.15.32:0/3699303064 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46381f5430 con 0x7f4638154610 2024-04-24T23:11:11.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.336+0000 7f462effd640 1 -- 172.21.15.32:0/3699303064 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4628015670 con 0x7f4638154610 2024-04-24T23:11:11.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.336+0000 7f463cb90640 1 -- 172.21.15.32:0/3699303064 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46380c9450 con 0x7f4638154610 2024-04-24T23:11:11.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.337+0000 7f463cb90640 1 -- 172.21.15.32:0/3699303064 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4638068430 con 0x7f4638154610 2024-04-24T23:11:11.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.337+0000 7f462effd640 1 -- 172.21.15.32:0/3699303064 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4628015810 con 0x7f4638154610 2024-04-24T23:11:11.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.338+0000 7f462effd640 1 --2- 172.21.15.32:0/3699303064 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f461806be30 0x7f461806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:11.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.338+0000 7f462effd640 1 -- 172.21.15.32:0/3699303064 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(190..190 src has 1..190) ==== 5738+0+0 (secure 0 0 0) 0x7f462808b970 con 0x7f4638154610 2024-04-24T23:11:11.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.338+0000 7f463659c640 1 --2- 172.21.15.32:0/3699303064 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f461806be30 0x7f461806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:11.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.338+0000 7f463659c640 1 --2- 172.21.15.32:0/3699303064 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f461806be30 0x7f461806e2f0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f4630032000 tx=0x7f4630009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:11.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.342+0000 7f462effd640 1 -- 172.21.15.32:0/3699303064 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4628027020 con 0x7f4638154610 2024-04-24T23:11:11.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.527+0000 7f463cb90640 1 -- 172.21.15.32:0/3699303064 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f46380b8010 con 0x7f4638154610 2024-04-24T23:11:11.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.527+0000 7f462effd640 1 -- 172.21.15.32:0/3699303064 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v190) ==== 100+0+11 (secure 0 0 0) 0x7f4628010040 con 0x7f4638154610 2024-04-24T23:11:11.529 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-24T23:11:11.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.529+0000 7f463cb90640 1 -- 172.21.15.32:0/3699303064 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f461806be30 msgr2=0x7f461806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:11.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.529+0000 7f463cb90640 1 --2- 172.21.15.32:0/3699303064 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f461806be30 0x7f461806e2f0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f4630032000 tx=0x7f4630009030 comp rx=0 tx=0).stop 2024-04-24T23:11:11.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.529+0000 7f463cb90640 1 -- 172.21.15.32:0/3699303064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4638154610 msgr2=0x7f463815f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:11.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.529+0000 7f463cb90640 1 --2- 172.21.15.32:0/3699303064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4638154610 0x7f463815f7b0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f46380d0930 tx=0x7f462800f580 comp rx=0 tx=0).stop 2024-04-24T23:11:11.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.529+0000 7f463cb90640 1 -- 172.21.15.32:0/3699303064 shutdown_connections 2024-04-24T23:11:11.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.529+0000 7f463cb90640 1 --2- 172.21.15.32:0/3699303064 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f461806be30 0x7f461806e2f0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:11.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.529+0000 7f463cb90640 1 --2- 172.21.15.32:0/3699303064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4638154610 0x7f463815f7b0 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:11.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.529+0000 7f463cb90640 1 --2- 172.21.15.32:0/3699303064 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46381618b0 0x7f46381f4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:11.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.529+0000 7f463cb90640 1 --2- 172.21.15.32:0/3699303064 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46240039b0 0x7f4638156820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:11.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.529+0000 7f463cb90640 1 -- 172.21.15.32:0/3699303064 >> 172.21.15.32:0/3699303064 conn(0x7f46380d11d0 msgr2=0x7f463815c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:11.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.530+0000 7f463cb90640 1 -- 172.21.15.32:0/3699303064 shutdown_connections 2024-04-24T23:11:11.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.530+0000 7f463cb90640 1 -- 172.21.15.32:0/3699303064 wait complete. 2024-04-24T23:11:11.542 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T23:11:11.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.642+0000 7f45d1563640 1 Processor -- start 2024-04-24T23:11:11.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.643+0000 7f45d1563640 1 -- start start 2024-04-24T23:11:11.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.643+0000 7f45d1563640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f45cc06ba30 0x7f45cc06d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:11.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.643+0000 7f45d1563640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f45cc06da90 0x7f45cc0b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:11.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.643+0000 7f45d1563640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45cc15ef10 con 0x7f45cc158dd0 2024-04-24T23:11:11.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.643+0000 7f45d1563640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f45cc15f1f0 con 0x7f45cc06ba30 2024-04-24T23:11:11.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.643+0000 7f45d1563640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f45cc15f4d0 con 0x7f45cc06da90 2024-04-24T23:11:11.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.643+0000 7f45ca7fc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f45cc06ba30 0x7f45cc06d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:11.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.643+0000 7f45caffd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f45cc158dd0 0x7f45cc06b320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35882/0 (socket says 172.21.15.32:35882) 2024-04-24T23:11:11.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.643+0000 7f45caffd640 1 -- 172.21.15.32:0/3507062178 learned_addr learned my addr 172.21.15.32:0/3507062178 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:11:11.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.644+0000 7f45cb7fe640 1 --2- 172.21.15.32:0/3507062178 >> v2:172.21.15.121:3300/0 conn(0x7f45cc06da90 0x7f45cc0b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:11.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.644+0000 7f45ca7fc640 1 -- 172.21.15.32:0/3507062178 >> v2:172.21.15.121:3300/0 conn(0x7f45cc06da90 msgr2=0x7f45cc0b5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:11.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.644+0000 7f45ca7fc640 1 --2- 172.21.15.32:0/3507062178 >> v2:172.21.15.121:3300/0 conn(0x7f45cc06da90 0x7f45cc0b5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:11.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.644+0000 7f45ca7fc640 1 -- 172.21.15.32:0/3507062178 >> v1:172.21.15.32:6789/0 conn(0x7f45cc158dd0 legacy=0x7f45cc06b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:11.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.644+0000 7f45ca7fc640 1 -- 172.21.15.32:0/3507062178 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45cc15f7b0 con 0x7f45cc06ba30 2024-04-24T23:11:11.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.644+0000 7f45c9ffb640 1 -- 172.21.15.32:0/3507062178 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1883085523 0 0) 0x7f45cc15ef10 con 0x7f45cc158dd0 2024-04-24T23:11:11.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.644+0000 7f45cb7fe640 1 --2- 172.21.15.32:0/3507062178 >> v2:172.21.15.121:3300/0 conn(0x7f45cc06da90 0x7f45cc0b5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:11:11.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.645+0000 7f45ca7fc640 1 --2- 172.21.15.32:0/3507062178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45cc06ba30 0x7f45cc06d550 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f45cc0b2170 tx=0x7f45b402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4017540476bd9d7e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:11.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.645+0000 7f45c9ffb640 1 -- 172.21.15.32:0/3507062178 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b4037070 con 0x7f45cc06ba30 2024-04-24T23:11:11.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.645+0000 7f45c9ffb640 1 -- 172.21.15.32:0/3507062178 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45b4032440 con 0x7f45cc06ba30 2024-04-24T23:11:11.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.645+0000 7f45c9ffb640 1 -- 172.21.15.32:0/3507062178 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b4032810 con 0x7f45cc06ba30 2024-04-24T23:11:11.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.645+0000 7f45d1563640 1 -- 172.21.15.32:0/3507062178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45cc06ba30 msgr2=0x7f45cc06d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:11.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.645+0000 7f45d1563640 1 --2- 172.21.15.32:0/3507062178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45cc06ba30 0x7f45cc06d550 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f45cc0b2170 tx=0x7f45b402a1a0 comp rx=0 tx=0).stop 2024-04-24T23:11:11.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.645+0000 7f45d1563640 1 -- 172.21.15.32:0/3507062178 shutdown_connections 2024-04-24T23:11:11.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.645+0000 7f45d1563640 1 --2- 172.21.15.32:0/3507062178 >> v2:172.21.15.121:3300/0 conn(0x7f45cc06da90 0x7f45cc0b5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:11.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.645+0000 7f45d1563640 1 --2- 172.21.15.32:0/3507062178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45cc06ba30 0x7f45cc06d550 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:11.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.645+0000 7f45d1563640 1 -- 172.21.15.32:0/3507062178 >> 172.21.15.32:0/3507062178 conn(0x7f45cc06e800 msgr2=0x7f45cc06ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:11.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.645+0000 7f45d1563640 1 -- 172.21.15.32:0/3507062178 shutdown_connections 2024-04-24T23:11:11.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.645+0000 7f45d1563640 1 -- 172.21.15.32:0/3507062178 wait complete. 2024-04-24T23:11:11.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.647+0000 7f45d1563640 1 Processor -- start 2024-04-24T23:11:11.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.647+0000 7f45d1563640 1 -- start start 2024-04-24T23:11:11.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.647+0000 7f45d1563640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45cc06ba30 0x7f45cc162350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:11.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.647+0000 7f45d1563640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45cc06da90 0x7f45cc162890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:11.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.647+0000 7f45d1563640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f45cc158dd0 0x7f45cc1f34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:11.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.647+0000 7f45d1563640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f45cc06bfe0 con 0x7f45cc06da90 2024-04-24T23:11:11.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.647+0000 7f45d1563640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f45cc06ded0 con 0x7f45cc06ba30 2024-04-24T23:11:11.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.647+0000 7f45d1563640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f45cc1591b0 con 0x7f45cc158dd0 2024-04-24T23:11:11.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.647+0000 7f45ca7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45cc06da90 0x7f45cc162890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:11.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.647+0000 7f45ca7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45cc06da90 0x7f45cc162890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49168/0 (socket says 172.21.15.32:49168) 2024-04-24T23:11:11.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.647+0000 7f45ca7fc640 1 -- 172.21.15.32:0/2459568126 learned_addr learned my addr 172.21.15.32:0/2459568126 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:11.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.648+0000 7f45caffd640 1 --2- 172.21.15.32:0/2459568126 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45cc06ba30 0x7f45cc162350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:11.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.648+0000 7f45ca7fc640 1 -- 172.21.15.32:0/2459568126 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f45cc158dd0 msgr2=0x7f45cc1f34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:11.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.648+0000 7f45ca7fc640 1 --2- 172.21.15.32:0/2459568126 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f45cc158dd0 0x7f45cc1f34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:11.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.648+0000 7f45ca7fc640 1 -- 172.21.15.32:0/2459568126 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45cc06ba30 msgr2=0x7f45cc162350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:11.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.648+0000 7f45ca7fc640 1 --2- 172.21.15.32:0/2459568126 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45cc06ba30 0x7f45cc162350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:11.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.648+0000 7f45ca7fc640 1 -- 172.21.15.32:0/2459568126 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45cc0b52a0 con 0x7f45cc06da90 2024-04-24T23:11:11.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.648+0000 7f45ca7fc640 1 --2- 172.21.15.32:0/2459568126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45cc06da90 0x7f45cc162890 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f45b4000c00 tx=0x7f45b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:11.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.648+0000 7f45c37fe640 1 -- 172.21.15.32:0/2459568126 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b4037050 con 0x7f45cc06da90 2024-04-24T23:11:11.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.648+0000 7f45c37fe640 1 -- 172.21.15.32:0/2459568126 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45b4038040 con 0x7f45cc06da90 2024-04-24T23:11:11.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.648+0000 7f45d1563640 1 -- 172.21.15.32:0/2459568126 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45cc13cd10 con 0x7f45cc06da90 2024-04-24T23:11:11.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.648+0000 7f45c37fe640 1 -- 172.21.15.32:0/2459568126 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b4004470 con 0x7f45cc06da90 2024-04-24T23:11:11.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.648+0000 7f45d1563640 1 -- 172.21.15.32:0/2459568126 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45cc0b4d70 con 0x7f45cc06da90 2024-04-24T23:11:11.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.649+0000 7f45d1563640 1 -- 172.21.15.32:0/2459568126 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45cc0b5030 con 0x7f45cc06da90 2024-04-24T23:11:11.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.650+0000 7f45c37fe640 1 -- 172.21.15.32:0/2459568126 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f45b4030900 con 0x7f45cc06da90 2024-04-24T23:11:11.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.650+0000 7f45c37fe640 1 --2- 172.21.15.32:0/2459568126 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f45ac06be30 0x7f45ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:11.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.650+0000 7f45c37fe640 1 -- 172.21.15.32:0/2459568126 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(190..190 src has 1..190) ==== 5738+0+0 (secure 0 0 0) 0x7f45b40abab0 con 0x7f45cc06da90 2024-04-24T23:11:11.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.650+0000 7f45caffd640 1 --2- 172.21.15.32:0/2459568126 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f45ac06be30 0x7f45ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:11.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.651+0000 7f45caffd640 1 --2- 172.21.15.32:0/2459568126 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f45ac06be30 0x7f45ac06e2f0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f45c4000f60 tx=0x7f45c4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:11.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.654+0000 7f45c37fe640 1 -- 172.21.15.32:0/2459568126 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f45b407a560 con 0x7f45cc06da90 2024-04-24T23:11:11.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.848+0000 7f45d1563640 1 -- 172.21.15.32:0/2459568126 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f45cc068430 con 0x7f45cc06da90 2024-04-24T23:11:11.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.849+0000 7f45c37fe640 1 -- 172.21.15.32:0/2459568126 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v190) ==== 80+0+0 (secure 0 0 0) 0x7f45b4079e60 con 0x7f45cc06da90 2024-04-24T23:11:11.851 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2024-04-24T23:11:11.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.851+0000 7f45d1563640 1 -- 172.21.15.32:0/2459568126 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f45ac06be30 msgr2=0x7f45ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:11.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.851+0000 7f45d1563640 1 --2- 172.21.15.32:0/2459568126 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f45ac06be30 0x7f45ac06e2f0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f45c4000f60 tx=0x7f45c4009040 comp rx=0 tx=0).stop 2024-04-24T23:11:11.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.851+0000 7f45d1563640 1 -- 172.21.15.32:0/2459568126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45cc06da90 msgr2=0x7f45cc162890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:11.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.851+0000 7f45d1563640 1 --2- 172.21.15.32:0/2459568126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45cc06da90 0x7f45cc162890 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f45b4000c00 tx=0x7f45b40046c0 comp rx=0 tx=0).stop 2024-04-24T23:11:11.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.851+0000 7f45d1563640 1 -- 172.21.15.32:0/2459568126 shutdown_connections 2024-04-24T23:11:11.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.851+0000 7f45d1563640 1 --2- 172.21.15.32:0/2459568126 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f45ac06be30 0x7f45ac06e2f0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:11.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.851+0000 7f45d1563640 1 --2- 172.21.15.32:0/2459568126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45cc06da90 0x7f45cc162890 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:11.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.851+0000 7f45d1563640 1 --2- 172.21.15.32:0/2459568126 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f45cc158dd0 0x7f45cc1f34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:11.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.851+0000 7f45d1563640 1 --2- 172.21.15.32:0/2459568126 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45cc06ba30 0x7f45cc162350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:11.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.851+0000 7f45d1563640 1 -- 172.21.15.32:0/2459568126 >> 172.21.15.32:0/2459568126 conn(0x7f45cc06e800 msgr2=0x7f45cc06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:11.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.851+0000 7f45d1563640 1 -- 172.21.15.32:0/2459568126 shutdown_connections 2024-04-24T23:11:11.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:11.851+0000 7f45d1563640 1 -- 172.21.15.32:0/2459568126 wait complete. 2024-04-24T23:11:11.863 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-24T23:11:11.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:11:11.863 DEBUG:teuthology.orchestra.run.smithi032:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-24T23:11:11.906 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2024-04-24T23:11:11.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:11:11.906 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-24T23:11:11.962 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:11:11.962 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:11:11.962 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:11:11.979 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:11:11.979 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:11:12.036 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:11:12.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.136+0000 7f228fa15640 1 Processor -- start 2024-04-24T23:11:12.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.136+0000 7f228fa15640 1 -- start start 2024-04-24T23:11:12.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.137+0000 7f228fa15640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2288158dd0 0x7f228806b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:12.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.137+0000 7f228fa15640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f228806b860 0x7f228806d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:12.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.137+0000 7f228fa15640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f228806da90 0x7f2288072580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:12.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.137+0000 7f228fa15640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f228815ef10 con 0x7f228806da90 2024-04-24T23:11:12.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.137+0000 7f228fa15640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f228815f1f0 con 0x7f2288158dd0 2024-04-24T23:11:12.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.137+0000 7f228fa15640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f228815f4d0 con 0x7f228806b860 2024-04-24T23:11:12.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.137+0000 7f228dfb2640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f228806da90 0x7f2288072580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:12.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.137+0000 7f228dfb2640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f228806da90 0x7f2288072580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49170/0 (socket says 172.21.15.32:49170) 2024-04-24T23:11:12.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.137+0000 7f228dfb2640 1 -- 172.21.15.32:0/720295510 learned_addr learned my addr 172.21.15.32:0/720295510 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:12.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.137+0000 7f228cfb0640 1 --2- 172.21.15.32:0/720295510 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f228806b860 0x7f228806d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:12.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.137+0000 7f228d7b1640 1 --2- 172.21.15.32:0/720295510 >> v2:172.21.15.121:3300/0 conn(0x7f2288158dd0 0x7f228806b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:12.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.137+0000 7f228dfb2640 1 -- 172.21.15.32:0/720295510 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f228806b860 msgr2=0x7f228806d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:12.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.137+0000 7f228dfb2640 1 --2- 172.21.15.32:0/720295510 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f228806b860 0x7f228806d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:12.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.137+0000 7f228dfb2640 1 -- 172.21.15.32:0/720295510 >> v2:172.21.15.121:3300/0 conn(0x7f2288158dd0 msgr2=0x7f228806b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:12.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.137+0000 7f228dfb2640 1 --2- 172.21.15.32:0/720295510 >> v2:172.21.15.121:3300/0 conn(0x7f2288158dd0 0x7f228806b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:12.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.137+0000 7f228dfb2640 1 -- 172.21.15.32:0/720295510 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f228815f7b0 con 0x7f228806da90 2024-04-24T23:11:12.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.138+0000 7f228dfb2640 1 --2- 172.21.15.32:0/720295510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f228806da90 0x7f2288072580 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f228806f3a0 tx=0x7f228002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8275503a3df93020 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:12.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.138+0000 7f227ffff640 1 -- 172.21.15.32:0/720295510 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2280036070 con 0x7f228806da90 2024-04-24T23:11:12.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.138+0000 7f227ffff640 1 -- 172.21.15.32:0/720295510 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2280031600 con 0x7f228806da90 2024-04-24T23:11:12.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.138+0000 7f227ffff640 1 -- 172.21.15.32:0/720295510 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22800319d0 con 0x7f228806da90 2024-04-24T23:11:12.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.139+0000 7f228fa15640 1 -- 172.21.15.32:0/720295510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f228806da90 msgr2=0x7f2288072580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:12.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.139+0000 7f228fa15640 1 --2- 172.21.15.32:0/720295510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f228806da90 0x7f2288072580 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f228806f3a0 tx=0x7f228002a6a0 comp rx=0 tx=0).stop 2024-04-24T23:11:12.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.139+0000 7f228fa15640 1 -- 172.21.15.32:0/720295510 shutdown_connections 2024-04-24T23:11:12.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.139+0000 7f228fa15640 1 --2- 172.21.15.32:0/720295510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f228806da90 0x7f2288072580 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:12.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.139+0000 7f228fa15640 1 --2- 172.21.15.32:0/720295510 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f228806b860 0x7f228806d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:12.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.139+0000 7f228fa15640 1 --2- 172.21.15.32:0/720295510 >> v2:172.21.15.121:3300/0 conn(0x7f2288158dd0 0x7f228806b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:12.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.139+0000 7f228fa15640 1 -- 172.21.15.32:0/720295510 >> 172.21.15.32:0/720295510 conn(0x7f228806e800 msgr2=0x7f228806ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:12.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.139+0000 7f228fa15640 1 -- 172.21.15.32:0/720295510 shutdown_connections 2024-04-24T23:11:12.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.139+0000 7f228fa15640 1 -- 172.21.15.32:0/720295510 wait complete. 2024-04-24T23:11:12.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.140+0000 7f228fa15640 1 Processor -- start 2024-04-24T23:11:12.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.140+0000 7f228fa15640 1 -- start start 2024-04-24T23:11:12.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.141+0000 7f228fa15640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f228806b860 0x7f228815e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:12.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.141+0000 7f228fa15640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f228806da90 0x7f228814bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:12.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.141+0000 7f228fa15640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2288158dd0 0x7f2288150d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:12.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.141+0000 7f228fa15640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f228806ded0 con 0x7f2288158dd0 2024-04-24T23:11:12.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.141+0000 7f228fa15640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f228815fcd0 con 0x7f228806da90 2024-04-24T23:11:12.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.141+0000 7f228fa15640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f228806bfe0 con 0x7f228806b860 2024-04-24T23:11:12.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.141+0000 7f228dfb2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2288158dd0 0x7f2288150d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:12.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.141+0000 7f228dfb2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2288158dd0 0x7f2288150d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49186/0 (socket says 172.21.15.32:49186) 2024-04-24T23:11:12.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.141+0000 7f228dfb2640 1 -- 172.21.15.32:0/1970434434 learned_addr learned my addr 172.21.15.32:0/1970434434 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:12.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.141+0000 7f228d7b1640 1 --2- 172.21.15.32:0/1970434434 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f228806b860 0x7f228815e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:12.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.141+0000 7f228cfb0640 1 --2- 172.21.15.32:0/1970434434 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f228806da90 0x7f228814bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:12.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.141+0000 7f228dfb2640 1 -- 172.21.15.32:0/1970434434 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f228806b860 msgr2=0x7f228815e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:12.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.141+0000 7f228dfb2640 1 --2- 172.21.15.32:0/1970434434 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f228806b860 0x7f228815e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:12.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.141+0000 7f228dfb2640 1 -- 172.21.15.32:0/1970434434 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f228806da90 msgr2=0x7f228814bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:12.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.141+0000 7f228dfb2640 1 --2- 172.21.15.32:0/1970434434 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f228806da90 0x7f228814bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:12.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.141+0000 7f228dfb2640 1 -- 172.21.15.32:0/1970434434 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f228814c490 con 0x7f2288158dd0 2024-04-24T23:11:12.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.141+0000 7f228dfb2640 1 --2- 172.21.15.32:0/1970434434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2288158dd0 0x7f2288150d90 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f2280034040 tx=0x7f2280004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:12.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.141+0000 7f227dffb640 1 -- 172.21.15.32:0/1970434434 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2280036050 con 0x7f2288158dd0 2024-04-24T23:11:12.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.141+0000 7f227dffb640 1 -- 172.21.15.32:0/1970434434 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2280042070 con 0x7f2288158dd0 2024-04-24T23:11:12.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.141+0000 7f227dffb640 1 -- 172.21.15.32:0/1970434434 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2280009510 con 0x7f2288158dd0 2024-04-24T23:11:12.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.141+0000 7f228fa15640 1 -- 172.21.15.32:0/1970434434 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22880b5050 con 0x7f2288158dd0 2024-04-24T23:11:12.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.142+0000 7f228fa15640 1 -- 172.21.15.32:0/1970434434 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2288154430 con 0x7f2288158dd0 2024-04-24T23:11:12.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.143+0000 7f228fa15640 1 -- 172.21.15.32:0/1970434434 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2288068430 con 0x7f2288158dd0 2024-04-24T23:11:12.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.143+0000 7f227dffb640 1 -- 172.21.15.32:0/1970434434 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2280007bd0 con 0x7f2288158dd0 2024-04-24T23:11:12.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.143+0000 7f227dffb640 1 --2- 172.21.15.32:0/1970434434 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f226c06be30 0x7f226c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:12.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.144+0000 7f227dffb640 1 -- 172.21.15.32:0/1970434434 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(190..190 src has 1..190) ==== 5738+0+0 (secure 0 0 0) 0x7f22800abd40 con 0x7f2288158dd0 2024-04-24T23:11:12.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.144+0000 7f228d7b1640 1 --2- 172.21.15.32:0/1970434434 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f226c06be30 0x7f226c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:12.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.144+0000 7f228d7b1640 1 --2- 172.21.15.32:0/1970434434 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f226c06be30 0x7f226c06e2f0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f228400aac0 tx=0x7f2284008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:12.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.148+0000 7f227dffb640 1 -- 172.21.15.32:0/1970434434 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2280079ff0 con 0x7f2288158dd0 2024-04-24T23:11:12.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.332+0000 7f228fa15640 1 -- 172.21.15.32:0/1970434434 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f228814d5d0 con 0x7f2288158dd0 2024-04-24T23:11:12.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.333+0000 7f227dffb640 1 -- 172.21.15.32:0/1970434434 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v190) ==== 74+0+16748 (secure 0 0 0) 0x7f228007e2f0 con 0x7f2288158dd0 2024-04-24T23:11:12.335 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:11:12.335 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":190,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:11:06.861172+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:10:34.649562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-24T23:10:35.339604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"190","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"23.6","mappings":[{"from":5,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:11:12.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.335+0000 7f228fa15640 1 -- 172.21.15.32:0/1970434434 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f226c06be30 msgr2=0x7f226c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:12.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.335+0000 7f228fa15640 1 --2- 172.21.15.32:0/1970434434 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f226c06be30 0x7f226c06e2f0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f228400aac0 tx=0x7f2284008040 comp rx=0 tx=0).stop 2024-04-24T23:11:12.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.335+0000 7f228fa15640 1 -- 172.21.15.32:0/1970434434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2288158dd0 msgr2=0x7f2288150d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:12.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.335+0000 7f228fa15640 1 --2- 172.21.15.32:0/1970434434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2288158dd0 0x7f2288150d90 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f2280034040 tx=0x7f2280004a40 comp rx=0 tx=0).stop 2024-04-24T23:11:12.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.335+0000 7f228fa15640 1 -- 172.21.15.32:0/1970434434 shutdown_connections 2024-04-24T23:11:12.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.335+0000 7f228fa15640 1 --2- 172.21.15.32:0/1970434434 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f226c06be30 0x7f226c06e2f0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:12.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.335+0000 7f228fa15640 1 --2- 172.21.15.32:0/1970434434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2288158dd0 0x7f2288150d90 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:12.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.335+0000 7f228fa15640 1 --2- 172.21.15.32:0/1970434434 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f228806da90 0x7f228814bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:12.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.335+0000 7f228fa15640 1 --2- 172.21.15.32:0/1970434434 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f228806b860 0x7f228815e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:12.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.335+0000 7f228fa15640 1 -- 172.21.15.32:0/1970434434 >> 172.21.15.32:0/1970434434 conn(0x7f228806e800 msgr2=0x7f2288160f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:12.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.335+0000 7f228fa15640 1 -- 172.21.15.32:0/1970434434 shutdown_connections 2024-04-24T23:11:12.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.335+0000 7f228fa15640 1 -- 172.21.15.32:0/1970434434 wait complete. 2024-04-24T23:11:12.348 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:10:34.649562+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '184', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:10:35.339604+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '190', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:11:12.348 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:11:12.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.450+0000 7f17fba1c640 1 Processor -- start 2024-04-24T23:11:12.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.450+0000 7f17fba1c640 1 -- start start 2024-04-24T23:11:12.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.451+0000 7f17fba1c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f17f415e9b0 0x7f17f40d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:12.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.451+0000 7f17fba1c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f17f40d4210 0x7f17f406b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:12.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.451+0000 7f17fba1c640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17f40d3170 con 0x7f17f40ba0e0 2024-04-24T23:11:12.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.451+0000 7f17fba1c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f17f40b8900 con 0x7f17f40d4210 2024-04-24T23:11:12.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.451+0000 7f17fba1c640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f17f40d3450 con 0x7f17f415e9b0 2024-04-24T23:11:12.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.451+0000 7f17f8fb7640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f17f40d4210 0x7f17f406b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:12.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.451+0000 7f17f8fb7640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f17f40d4210 0x7f17f406b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49198/0 (socket says 172.21.15.32:49198) 2024-04-24T23:11:12.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.451+0000 7f17f8fb7640 1 -- 172.21.15.32:0/2292839153 learned_addr learned my addr 172.21.15.32:0/2292839153 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:12.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.452+0000 7f17f8fb7640 1 -- 172.21.15.32:0/2292839153 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f17f415e9b0 msgr2=0x7f17f40d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:12.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.452+0000 7f17f8fb7640 1 --2- 172.21.15.32:0/2292839153 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f17f415e9b0 0x7f17f40d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:12.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.452+0000 7f17f8fb7640 1 -- 172.21.15.32:0/2292839153 >> v1:172.21.15.121:6789/0 conn(0x7f17f40ba0e0 legacy=0x7f17f4071730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:12.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.452+0000 7f17f8fb7640 1 -- 172.21.15.32:0/2292839153 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17f40d3730 con 0x7f17f40d4210 2024-04-24T23:11:12.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.452+0000 7f17f8fb7640 1 --2- 172.21.15.32:0/2292839153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17f40d4210 0x7f17f406b110 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f17e8009870 tx=0x7f17e802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f49073f79d1cf90a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:12.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.452+0000 7f17f3fff640 1 -- 172.21.15.32:0/2292839153 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e8037070 con 0x7f17f40d4210 2024-04-24T23:11:12.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.453+0000 7f17f3fff640 1 -- 172.21.15.32:0/2292839153 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17e8032440 con 0x7f17f40d4210 2024-04-24T23:11:12.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.453+0000 7f17f3fff640 1 -- 172.21.15.32:0/2292839153 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e8032810 con 0x7f17f40d4210 2024-04-24T23:11:12.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.453+0000 7f17fba1c640 1 -- 172.21.15.32:0/2292839153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17f40d4210 msgr2=0x7f17f406b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:12.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.453+0000 7f17fba1c640 1 --2- 172.21.15.32:0/2292839153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17f40d4210 0x7f17f406b110 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f17e8009870 tx=0x7f17e802a210 comp rx=0 tx=0).stop 2024-04-24T23:11:12.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.453+0000 7f17fba1c640 1 -- 172.21.15.32:0/2292839153 shutdown_connections 2024-04-24T23:11:12.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.453+0000 7f17fba1c640 1 --2- 172.21.15.32:0/2292839153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17f40d4210 0x7f17f406b110 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:12.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.453+0000 7f17fba1c640 1 --2- 172.21.15.32:0/2292839153 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f17f415e9b0 0x7f17f40d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:12.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.453+0000 7f17fba1c640 1 -- 172.21.15.32:0/2292839153 >> 172.21.15.32:0/2292839153 conn(0x7f17f40d11d0 msgr2=0x7f17f40d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:12.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.453+0000 7f17fba1c640 1 -- 172.21.15.32:0/2292839153 shutdown_connections 2024-04-24T23:11:12.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.453+0000 7f17fba1c640 1 -- 172.21.15.32:0/2292839153 wait complete. 2024-04-24T23:11:12.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.454+0000 7f17fba1c640 1 Processor -- start 2024-04-24T23:11:12.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.454+0000 7f17fba1c640 1 -- start start 2024-04-24T23:11:12.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.455+0000 7f17fba1c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17f40ba0e0 0x7f17f41f75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:12.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.455+0000 7f17fba1c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f17f40d4210 0x7f17f40cc380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:12.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.455+0000 7f17fba1c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f17f415e9b0 0x7f17f415f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:12.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.455+0000 7f17fba1c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f17f40d2400 con 0x7f17f40ba0e0 2024-04-24T23:11:12.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.455+0000 7f17fba1c640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f17f40d3cf0 con 0x7f17f415e9b0 2024-04-24T23:11:12.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.455+0000 7f17fba1c640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f17f406b650 con 0x7f17f40d4210 2024-04-24T23:11:12.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.455+0000 7f17f97b8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17f40ba0e0 0x7f17f41f75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:12.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.455+0000 7f17f97b8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17f40ba0e0 0x7f17f41f75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49208/0 (socket says 172.21.15.32:49208) 2024-04-24T23:11:12.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.455+0000 7f17f97b8640 1 -- 172.21.15.32:0/2247546981 learned_addr learned my addr 172.21.15.32:0/2247546981 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:12.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.455+0000 7f17f97b8640 1 -- 172.21.15.32:0/2247546981 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f17f40d4210 msgr2=0x7f17f40cc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:12.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.455+0000 7f17f8fb7640 1 --2- 172.21.15.32:0/2247546981 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f17f40d4210 0x7f17f40cc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:12.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.455+0000 7f17f9fb9640 1 --2- 172.21.15.32:0/2247546981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f17f415e9b0 0x7f17f415f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:12.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.455+0000 7f17f97b8640 1 --2- 172.21.15.32:0/2247546981 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f17f40d4210 0x7f17f40cc380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:12.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.455+0000 7f17f97b8640 1 -- 172.21.15.32:0/2247546981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f17f415e9b0 msgr2=0x7f17f415f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:12.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.455+0000 7f17f97b8640 1 --2- 172.21.15.32:0/2247546981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f17f415e9b0 0x7f17f415f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:12.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.455+0000 7f17f97b8640 1 -- 172.21.15.32:0/2247546981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17f40ca920 con 0x7f17f40ba0e0 2024-04-24T23:11:12.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.456+0000 7f17f97b8640 1 --2- 172.21.15.32:0/2247546981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17f40ba0e0 0x7f17f41f75f0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f17ec00b410 tx=0x7f17ec00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:12.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.456+0000 7f17f1ffb640 1 -- 172.21.15.32:0/2247546981 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ec015020 con 0x7f17f40ba0e0 2024-04-24T23:11:12.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.456+0000 7f17f1ffb640 1 -- 172.21.15.32:0/2247546981 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17ec011850 con 0x7f17f40ba0e0 2024-04-24T23:11:12.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.456+0000 7f17fba1c640 1 -- 172.21.15.32:0/2247546981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17f40c84d0 con 0x7f17f40ba0e0 2024-04-24T23:11:12.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.456+0000 7f17f1ffb640 1 -- 172.21.15.32:0/2247546981 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ec011c40 con 0x7f17f40ba0e0 2024-04-24T23:11:12.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.456+0000 7f17fba1c640 1 -- 172.21.15.32:0/2247546981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17f40c8a90 con 0x7f17f40ba0e0 2024-04-24T23:11:12.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.457+0000 7f17fba1c640 1 -- 172.21.15.32:0/2247546981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17f4068430 con 0x7f17f40ba0e0 2024-04-24T23:11:12.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.457+0000 7f17f1ffb640 1 -- 172.21.15.32:0/2247546981 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f17ec018020 con 0x7f17f40ba0e0 2024-04-24T23:11:12.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.458+0000 7f17f1ffb640 1 --2- 172.21.15.32:0/2247546981 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f17e006be30 0x7f17e006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:12.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.458+0000 7f17f1ffb640 1 -- 172.21.15.32:0/2247546981 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(190..190 src has 1..190) ==== 5738+0+0 (secure 0 0 0) 0x7f17ec08ad50 con 0x7f17f40ba0e0 2024-04-24T23:11:12.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.458+0000 7f17f8fb7640 1 --2- 172.21.15.32:0/2247546981 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f17e006be30 0x7f17e006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:12.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.459+0000 7f17f8fb7640 1 --2- 172.21.15.32:0/2247546981 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f17e006be30 0x7f17e006e2f0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f17e8034000 tx=0x7f17e802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:12.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.462+0000 7f17f1ffb640 1 -- 172.21.15.32:0/2247546981 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f17ec08e030 con 0x7f17f40ba0e0 2024-04-24T23:11:12.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.643+0000 7f17fba1c640 1 -- 172.21.15.32:0/2247546981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f17f406ba20 con 0x7f17f40ba0e0 2024-04-24T23:11:12.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.644+0000 7f17f1ffb640 1 -- 172.21.15.32:0/2247546981 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v190) ==== 93+0+10 (secure 0 0 0) 0x7f17ec01f020 con 0x7f17f40ba0e0 2024-04-24T23:11:12.645 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T23:11:12.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.646+0000 7f17fba1c640 1 -- 172.21.15.32:0/2247546981 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f17e006be30 msgr2=0x7f17e006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:12.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.646+0000 7f17fba1c640 1 --2- 172.21.15.32:0/2247546981 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f17e006be30 0x7f17e006e2f0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f17e8034000 tx=0x7f17e802e000 comp rx=0 tx=0).stop 2024-04-24T23:11:12.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.646+0000 7f17fba1c640 1 -- 172.21.15.32:0/2247546981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17f40ba0e0 msgr2=0x7f17f41f75f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:12.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.646+0000 7f17fba1c640 1 --2- 172.21.15.32:0/2247546981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17f40ba0e0 0x7f17f41f75f0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f17ec00b410 tx=0x7f17ec00b8d0 comp rx=0 tx=0).stop 2024-04-24T23:11:12.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.646+0000 7f17fba1c640 1 -- 172.21.15.32:0/2247546981 shutdown_connections 2024-04-24T23:11:12.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.646+0000 7f17fba1c640 1 --2- 172.21.15.32:0/2247546981 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f17e006be30 0x7f17e006e2f0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:12.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.646+0000 7f17fba1c640 1 --2- 172.21.15.32:0/2247546981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f17f415e9b0 0x7f17f415f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:12.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.646+0000 7f17fba1c640 1 --2- 172.21.15.32:0/2247546981 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f17f40d4210 0x7f17f40cc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:12.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.646+0000 7f17fba1c640 1 --2- 172.21.15.32:0/2247546981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17f40ba0e0 0x7f17f41f75f0 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:12.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.646+0000 7f17fba1c640 1 -- 172.21.15.32:0/2247546981 >> 172.21.15.32:0/2247546981 conn(0x7f17f40d11d0 msgr2=0x7f17f415cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:12.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.646+0000 7f17fba1c640 1 -- 172.21.15.32:0/2247546981 shutdown_connections 2024-04-24T23:11:12.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.646+0000 7f17fba1c640 1 -- 172.21.15.32:0/2247546981 wait complete. 2024-04-24T23:11:12.658 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:11:12.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.760+0000 7f249d293640 1 Processor -- start 2024-04-24T23:11:12.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.760+0000 7f249d293640 1 -- start start 2024-04-24T23:11:12.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.761+0000 7f249d293640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f24980b5af0 0x7f24980ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:12.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.761+0000 7f249d293640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2498162e10 0x7f249815ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:12.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.761+0000 7f249d293640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24980cd270 con 0x7f2498162320 2024-04-24T23:11:12.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.761+0000 7f249d293640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f24980b5660 con 0x7f24980b5af0 2024-04-24T23:11:12.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.761+0000 7f249d293640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f2498071610 con 0x7f2498162e10 2024-04-24T23:11:12.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.761+0000 7f248ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f24980b5af0 0x7f24980ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:12.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.761+0000 7f2496ffd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2498162320 0x7f2498162700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35886/0 (socket says 172.21.15.32:35886) 2024-04-24T23:11:12.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.761+0000 7f2496ffd640 1 -- 172.21.15.32:0/2884810071 learned_addr learned my addr 172.21.15.32:0/2884810071 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:11:12.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.761+0000 7f24977fe640 1 --2- 172.21.15.32:0/2884810071 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2498162e10 0x7f249815ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:12.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.761+0000 7f248ffff640 1 -- 172.21.15.32:0/2884810071 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2498162e10 msgr2=0x7f249815ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:12.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.761+0000 7f248ffff640 1 --2- 172.21.15.32:0/2884810071 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2498162e10 0x7f249815ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:12.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.761+0000 7f248ffff640 1 -- 172.21.15.32:0/2884810071 >> v1:172.21.15.32:6789/0 conn(0x7f2498162320 legacy=0x7f2498162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:12.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.761+0000 7f248ffff640 1 -- 172.21.15.32:0/2884810071 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24980718f0 con 0x7f24980b5af0 2024-04-24T23:11:12.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.761+0000 7f24967fc640 1 -- 172.21.15.32:0/2884810071 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2321110479 0 0) 0x7f24980cd270 con 0x7f2498162320 2024-04-24T23:11:12.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.761+0000 7f24977fe640 1 --2- 172.21.15.32:0/2884810071 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2498162e10 0x7f249815ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:11:12.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.762+0000 7f248ffff640 1 --2- 172.21.15.32:0/2884810071 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24980b5af0 0x7f24980ccc90 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f24800097a0 tx=0x7f248002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80259d194a458337 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:12.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.762+0000 7f24967fc640 1 -- 172.21.15.32:0/2884810071 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2480037070 con 0x7f24980b5af0 2024-04-24T23:11:12.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.762+0000 7f24967fc640 1 -- 172.21.15.32:0/2884810071 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f248002ae50 con 0x7f24980b5af0 2024-04-24T23:11:12.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.762+0000 7f24967fc640 1 -- 172.21.15.32:0/2884810071 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2480032660 con 0x7f24980b5af0 2024-04-24T23:11:12.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.763+0000 7f249d293640 1 -- 172.21.15.32:0/2884810071 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24980b5af0 msgr2=0x7f24980ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:12.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.763+0000 7f249d293640 1 --2- 172.21.15.32:0/2884810071 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24980b5af0 0x7f24980ccc90 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f24800097a0 tx=0x7f248002a100 comp rx=0 tx=0).stop 2024-04-24T23:11:12.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.763+0000 7f249d293640 1 -- 172.21.15.32:0/2884810071 shutdown_connections 2024-04-24T23:11:12.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.763+0000 7f249d293640 1 --2- 172.21.15.32:0/2884810071 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24980b5af0 0x7f24980ccc90 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:12.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.763+0000 7f249d293640 1 --2- 172.21.15.32:0/2884810071 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2498162e10 0x7f249815ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:12.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.763+0000 7f249d293640 1 -- 172.21.15.32:0/2884810071 >> 172.21.15.32:0/2884810071 conn(0x7f24980d0ae0 msgr2=0x7f24980d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:12.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.763+0000 7f249d293640 1 -- 172.21.15.32:0/2884810071 shutdown_connections 2024-04-24T23:11:12.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.763+0000 7f249d293640 1 -- 172.21.15.32:0/2884810071 wait complete. 2024-04-24T23:11:12.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.764+0000 7f249d293640 1 Processor -- start 2024-04-24T23:11:12.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.764+0000 7f249d293640 1 -- start start 2024-04-24T23:11:12.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.765+0000 7f249d293640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f24980b5af0 0x7f2498146560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:12.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.765+0000 7f249d293640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2498162320 0x7f249806e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:12.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.765+0000 7f249d293640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2498162e10 0x7f24981612f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:12.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.765+0000 7f249d293640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2498071bd0 con 0x7f2498162e10 2024-04-24T23:11:12.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.765+0000 7f249d293640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2498067950 con 0x7f2498162320 2024-04-24T23:11:12.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.765+0000 7f249d293640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f2498071d50 con 0x7f24980b5af0 2024-04-24T23:11:12.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.765+0000 7f24977fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2498162e10 0x7f24981612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:12.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.765+0000 7f24977fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2498162e10 0x7f24981612f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49240/0 (socket says 172.21.15.32:49240) 2024-04-24T23:11:12.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.765+0000 7f24977fe640 1 -- 172.21.15.32:0/1938352494 learned_addr learned my addr 172.21.15.32:0/1938352494 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:12.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.765+0000 7f2496ffd640 1 --2- 172.21.15.32:0/1938352494 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f24980b5af0 0x7f2498146560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:12.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.765+0000 7f248ffff640 1 --2- 172.21.15.32:0/1938352494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2498162320 0x7f249806e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:12.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.765+0000 7f24977fe640 1 -- 172.21.15.32:0/1938352494 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f24980b5af0 msgr2=0x7f2498146560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:12.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.765+0000 7f24977fe640 1 --2- 172.21.15.32:0/1938352494 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f24980b5af0 0x7f2498146560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:12.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.765+0000 7f24977fe640 1 -- 172.21.15.32:0/1938352494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2498162320 msgr2=0x7f249806e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:12.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.765+0000 7f24977fe640 1 --2- 172.21.15.32:0/1938352494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2498162320 0x7f249806e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:12.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.765+0000 7f24977fe640 1 -- 172.21.15.32:0/1938352494 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2498146aa0 con 0x7f2498162e10 2024-04-24T23:11:12.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.765+0000 7f24977fe640 1 --2- 172.21.15.32:0/1938352494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2498162e10 0x7f24981612f0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f249000cd50 tx=0x7f249000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:12.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.765+0000 7f248f7fe640 1 -- 172.21.15.32:0/1938352494 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249000f680 con 0x7f2498162e10 2024-04-24T23:11:12.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.765+0000 7f248f7fe640 1 -- 172.21.15.32:0/1938352494 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f249000f840 con 0x7f2498162e10 2024-04-24T23:11:12.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.765+0000 7f249d293640 1 -- 172.21.15.32:0/1938352494 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f249806ee60 con 0x7f2498162e10 2024-04-24T23:11:12.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.765+0000 7f248f7fe640 1 -- 172.21.15.32:0/1938352494 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249000fc30 con 0x7f2498162e10 2024-04-24T23:11:12.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.765+0000 7f249d293640 1 -- 172.21.15.32:0/1938352494 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24981f8250 con 0x7f2498162e10 2024-04-24T23:11:12.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.766+0000 7f249d293640 1 -- 172.21.15.32:0/1938352494 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f249815be60 con 0x7f2498162e10 2024-04-24T23:11:12.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.767+0000 7f248f7fe640 1 -- 172.21.15.32:0/1938352494 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f24900213f0 con 0x7f2498162e10 2024-04-24T23:11:12.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.767+0000 7f248f7fe640 1 --2- 172.21.15.32:0/1938352494 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f247806be60 0x7f247806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:12.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.767+0000 7f248f7fe640 1 -- 172.21.15.32:0/1938352494 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(190..190 src has 1..190) ==== 5738+0+0 (secure 0 0 0) 0x7f249008bb00 con 0x7f2498162e10 2024-04-24T23:11:12.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.767+0000 7f2496ffd640 1 --2- 172.21.15.32:0/1938352494 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f247806be60 0x7f247806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:12.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.768+0000 7f2496ffd640 1 --2- 172.21.15.32:0/1938352494 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f247806be60 0x7f247806e320 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f2488000f60 tx=0x7f2488007b50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:12.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.771+0000 7f248f7fe640 1 -- 172.21.15.32:0/1938352494 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f249815be60 con 0x7f2498162e10 2024-04-24T23:11:12.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.953+0000 7f249d293640 1 -- 172.21.15.32:0/1938352494 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f249806f090 con 0x7f2498162e10 2024-04-24T23:11:12.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.954+0000 7f248f7fe640 1 -- 172.21.15.32:0/1938352494 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v190) ==== 104+0+11 (secure 0 0 0) 0x7f2498146aa0 con 0x7f2498162e10 2024-04-24T23:11:12.955 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-24T23:11:12.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.956+0000 7f249d293640 1 -- 172.21.15.32:0/1938352494 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f247806be60 msgr2=0x7f247806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:12.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.956+0000 7f249d293640 1 --2- 172.21.15.32:0/1938352494 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f247806be60 0x7f247806e320 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f2488000f60 tx=0x7f2488007b50 comp rx=0 tx=0).stop 2024-04-24T23:11:12.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.956+0000 7f249d293640 1 -- 172.21.15.32:0/1938352494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2498162e10 msgr2=0x7f24981612f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:12.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.956+0000 7f249d293640 1 --2- 172.21.15.32:0/1938352494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2498162e10 0x7f24981612f0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f249000cd50 tx=0x7f249000a650 comp rx=0 tx=0).stop 2024-04-24T23:11:12.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.956+0000 7f249d293640 1 -- 172.21.15.32:0/1938352494 shutdown_connections 2024-04-24T23:11:12.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.956+0000 7f249d293640 1 --2- 172.21.15.32:0/1938352494 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f247806be60 0x7f247806e320 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:12.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.956+0000 7f249d293640 1 --2- 172.21.15.32:0/1938352494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2498162e10 0x7f24981612f0 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:12.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.956+0000 7f249d293640 1 --2- 172.21.15.32:0/1938352494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2498162320 0x7f249806e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:12.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.956+0000 7f249d293640 1 --2- 172.21.15.32:0/1938352494 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f24980b5af0 0x7f2498146560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:12.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.956+0000 7f249d293640 1 -- 172.21.15.32:0/1938352494 >> 172.21.15.32:0/1938352494 conn(0x7f24980d0ae0 msgr2=0x7f24980b0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:12.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.956+0000 7f249d293640 1 -- 172.21.15.32:0/1938352494 shutdown_connections 2024-04-24T23:11:12.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:12.956+0000 7f249d293640 1 -- 172.21.15.32:0/1938352494 wait complete. 2024-04-24T23:11:12.968 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:11:13.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.070+0000 7f2d80509640 1 Processor -- start 2024-04-24T23:11:13.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.070+0000 7f2d80509640 1 -- start start 2024-04-24T23:11:13.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.071+0000 7f2d80509640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2d78074210 0x7f2d7806c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:13.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.071+0000 7f2d80509640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2d780ae140 0x7f2d780b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:13.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.071+0000 7f2d80509640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d780b0770 con 0x7f2d780b0b80 2024-04-24T23:11:13.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.071+0000 7f2d80509640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f2d780b1c20 con 0x7f2d78074210 2024-04-24T23:11:13.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.071+0000 7f2d80509640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f2d780b1010 con 0x7f2d780ae140 2024-04-24T23:11:13.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.071+0000 7f2d7e2a5640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2d78074210 0x7f2d7806c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:13.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.071+0000 7f2d7eaa6640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2d780ae140 0x7f2d780b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:13.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.071+0000 7f2d7daa4640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f2d780b0b80 0x7f2d780745f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:46540/0 (socket says 172.21.15.32:46540) 2024-04-24T23:11:13.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.071+0000 7f2d7daa4640 1 -- 172.21.15.32:0/2873672826 learned_addr learned my addr 172.21.15.32:0/2873672826 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:11:13.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.071+0000 7f2d7e2a5640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2d78074210 0x7f2d7806c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.32:47094/0 (socket says 172.21.15.32:47094) 2024-04-24T23:11:13.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.072+0000 7f2d7e2a5640 1 -- 172.21.15.32:0/2873672826 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2d780ae140 msgr2=0x7f2d780b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:13.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.072+0000 7f2d7e2a5640 1 --2- 172.21.15.32:0/2873672826 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2d780ae140 0x7f2d780b20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:13.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.072+0000 7f2d7e2a5640 1 -- 172.21.15.32:0/2873672826 >> v1:172.21.15.121:6789/0 conn(0x7f2d780b0b80 legacy=0x7f2d780745f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:13.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.072+0000 7f2d7e2a5640 1 -- 172.21.15.32:0/2873672826 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d780b12f0 con 0x7f2d78074210 2024-04-24T23:11:13.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.072+0000 7f2d7eaa6640 1 --2- 172.21.15.32:0/2873672826 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2d780ae140 0x7f2d780b20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:11:13.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.072+0000 7f2d7e2a5640 1 --2- 172.21.15.32:0/2873672826 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d78074210 0x7f2d7806c480 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f2d740097a0 tx=0x7f2d7402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12d313aef416928c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:13.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.073+0000 7f2d7d2a3640 1 -- 172.21.15.32:0/2873672826 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d74037070 con 0x7f2d78074210 2024-04-24T23:11:13.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.073+0000 7f2d7d2a3640 1 -- 172.21.15.32:0/2873672826 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d74032440 con 0x7f2d78074210 2024-04-24T23:11:13.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.073+0000 7f2d7d2a3640 1 -- 172.21.15.32:0/2873672826 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d74032810 con 0x7f2d78074210 2024-04-24T23:11:13.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.073+0000 7f2d80509640 1 -- 172.21.15.32:0/2873672826 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d78074210 msgr2=0x7f2d7806c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:13.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.073+0000 7f2d80509640 1 --2- 172.21.15.32:0/2873672826 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d78074210 0x7f2d7806c480 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f2d740097a0 tx=0x7f2d7402a180 comp rx=0 tx=0).stop 2024-04-24T23:11:13.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.073+0000 7f2d80509640 1 -- 172.21.15.32:0/2873672826 shutdown_connections 2024-04-24T23:11:13.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.073+0000 7f2d80509640 1 --2- 172.21.15.32:0/2873672826 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2d780ae140 0x7f2d780b20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:13.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.073+0000 7f2d80509640 1 --2- 172.21.15.32:0/2873672826 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d78074210 0x7f2d7806c480 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:13.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.073+0000 7f2d80509640 1 -- 172.21.15.32:0/2873672826 >> 172.21.15.32:0/2873672826 conn(0x7f2d780b2e60 msgr2=0x7f2d780b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:13.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.073+0000 7f2d80509640 1 -- 172.21.15.32:0/2873672826 shutdown_connections 2024-04-24T23:11:13.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.073+0000 7f2d80509640 1 -- 172.21.15.32:0/2873672826 wait complete. 2024-04-24T23:11:13.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.074+0000 7f2d80509640 1 Processor -- start 2024-04-24T23:11:13.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.075+0000 7f2d80509640 1 -- start start 2024-04-24T23:11:13.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.075+0000 7f2d80509640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2d78074210 0x7f2d7815d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:13.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.075+0000 7f2d80509640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d780ae140 0x7f2d780af010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:13.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.075+0000 7f2d80509640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d780af550 0x7f2d780cac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:13.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.075+0000 7f2d80509640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2d780b3ba0 con 0x7f2d780af550 2024-04-24T23:11:13.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.075+0000 7f2d80509640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2d780b18b0 con 0x7f2d780ae140 2024-04-24T23:11:13.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.075+0000 7f2d80509640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f2d780b3d20 con 0x7f2d78074210 2024-04-24T23:11:13.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.075+0000 7f2d7eaa6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d780af550 0x7f2d780cac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:13.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.075+0000 7f2d7eaa6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d780af550 0x7f2d780cac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49246/0 (socket says 172.21.15.32:49246) 2024-04-24T23:11:13.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.075+0000 7f2d7eaa6640 1 -- 172.21.15.32:0/2708370118 learned_addr learned my addr 172.21.15.32:0/2708370118 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:13.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.076+0000 7f2d7daa4640 1 --2- 172.21.15.32:0/2708370118 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d780ae140 0x7f2d780af010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:13.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.076+0000 7f2d7e2a5640 1 --2- 172.21.15.32:0/2708370118 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2d78074210 0x7f2d7815d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:13.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.076+0000 7f2d7eaa6640 1 -- 172.21.15.32:0/2708370118 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2d78074210 msgr2=0x7f2d7815d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:13.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.076+0000 7f2d7eaa6640 1 --2- 172.21.15.32:0/2708370118 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2d78074210 0x7f2d7815d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:13.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.076+0000 7f2d7eaa6640 1 -- 172.21.15.32:0/2708370118 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d780ae140 msgr2=0x7f2d780af010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:13.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.076+0000 7f2d7eaa6640 1 --2- 172.21.15.32:0/2708370118 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d780ae140 0x7f2d780af010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:13.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.076+0000 7f2d7eaa6640 1 -- 172.21.15.32:0/2708370118 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d7815ec10 con 0x7f2d780af550 2024-04-24T23:11:13.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.076+0000 7f2d7eaa6640 1 --2- 172.21.15.32:0/2708370118 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d780af550 0x7f2d780cac40 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f2d7000efc0 tx=0x7f2d70009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:13.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.076+0000 7f2d6effd640 1 -- 172.21.15.32:0/2708370118 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d70018030 con 0x7f2d780af550 2024-04-24T23:11:13.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.076+0000 7f2d6effd640 1 -- 172.21.15.32:0/2708370118 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d7000cde0 con 0x7f2d780af550 2024-04-24T23:11:13.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.076+0000 7f2d80509640 1 -- 172.21.15.32:0/2708370118 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d780c9790 con 0x7f2d780af550 2024-04-24T23:11:13.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.076+0000 7f2d6effd640 1 -- 172.21.15.32:0/2708370118 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d70003bb0 con 0x7f2d780af550 2024-04-24T23:11:13.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.076+0000 7f2d80509640 1 -- 172.21.15.32:0/2708370118 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d7815f1b0 con 0x7f2d780af550 2024-04-24T23:11:13.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.077+0000 7f2d80509640 1 -- 172.21.15.32:0/2708370118 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d78068430 con 0x7f2d780af550 2024-04-24T23:11:13.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.078+0000 7f2d6effd640 1 -- 172.21.15.32:0/2708370118 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2d70005d10 con 0x7f2d780af550 2024-04-24T23:11:13.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.078+0000 7f2d6effd640 1 --2- 172.21.15.32:0/2708370118 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2d5c06be30 0x7f2d5c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:13.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.078+0000 7f2d6effd640 1 -- 172.21.15.32:0/2708370118 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(190..190 src has 1..190) ==== 5738+0+0 (secure 0 0 0) 0x7f2d7008b920 con 0x7f2d780af550 2024-04-24T23:11:13.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.078+0000 7f2d7e2a5640 1 --2- 172.21.15.32:0/2708370118 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2d5c06be30 0x7f2d5c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:13.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.079+0000 7f2d7e2a5640 1 --2- 172.21.15.32:0/2708370118 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2d5c06be30 0x7f2d5c06e2f0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f2d74034000 tx=0x7f2d7402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:13.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.082+0000 7f2d6effd640 1 -- 172.21.15.32:0/2708370118 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2d70022070 con 0x7f2d780af550 2024-04-24T23:11:13.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.265+0000 7f2d80509640 1 -- 172.21.15.32:0/2708370118 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2d7806b220 con 0x7f2d780af550 2024-04-24T23:11:13.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.266+0000 7f2d6effd640 1 -- 172.21.15.32:0/2708370118 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v190) ==== 100+0+11 (secure 0 0 0) 0x7f2d70010040 con 0x7f2d780af550 2024-04-24T23:11:13.267 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-24T23:11:13.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.267+0000 7f2d80509640 1 -- 172.21.15.32:0/2708370118 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2d5c06be30 msgr2=0x7f2d5c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:13.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.267+0000 7f2d80509640 1 --2- 172.21.15.32:0/2708370118 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2d5c06be30 0x7f2d5c06e2f0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f2d74034000 tx=0x7f2d7402e000 comp rx=0 tx=0).stop 2024-04-24T23:11:13.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.268+0000 7f2d80509640 1 -- 172.21.15.32:0/2708370118 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d780af550 msgr2=0x7f2d780cac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:13.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.268+0000 7f2d80509640 1 --2- 172.21.15.32:0/2708370118 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d780af550 0x7f2d780cac40 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f2d7000efc0 tx=0x7f2d70009f90 comp rx=0 tx=0).stop 2024-04-24T23:11:13.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.268+0000 7f2d80509640 1 -- 172.21.15.32:0/2708370118 shutdown_connections 2024-04-24T23:11:13.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.268+0000 7f2d80509640 1 --2- 172.21.15.32:0/2708370118 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2d5c06be30 0x7f2d5c06e2f0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:13.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.268+0000 7f2d80509640 1 --2- 172.21.15.32:0/2708370118 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d780af550 0x7f2d780cac40 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:13.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.268+0000 7f2d80509640 1 --2- 172.21.15.32:0/2708370118 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d780ae140 0x7f2d780af010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:13.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.268+0000 7f2d80509640 1 --2- 172.21.15.32:0/2708370118 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2d78074210 0x7f2d7815d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:13.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.268+0000 7f2d80509640 1 -- 172.21.15.32:0/2708370118 >> 172.21.15.32:0/2708370118 conn(0x7f2d780b2e60 msgr2=0x7f2d780b69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:13.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.268+0000 7f2d80509640 1 -- 172.21.15.32:0/2708370118 shutdown_connections 2024-04-24T23:11:13.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.268+0000 7f2d80509640 1 -- 172.21.15.32:0/2708370118 wait complete. 2024-04-24T23:11:13.280 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T23:11:13.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.380+0000 7f6f3d876640 1 Processor -- start 2024-04-24T23:11:13.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.380+0000 7f6f3d876640 1 -- start start 2024-04-24T23:11:13.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.381+0000 7f6f3d876640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f380d4210 0x7f6f380d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:13.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.381+0000 7f6f3d876640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f380d3170 con 0x7f6f3806b820 2024-04-24T23:11:13.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.381+0000 7f6f3d876640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f380d3450 con 0x7f6f3815e9b0 2024-04-24T23:11:13.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.381+0000 7f6f3d876640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f6f380d3730 con 0x7f6f380d4210 2024-04-24T23:11:13.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.381+0000 7f6f367fc640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6f3806b820 0x7f6f380b78f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35898/0 (socket says 172.21.15.32:35898) 2024-04-24T23:11:13.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.381+0000 7f6f367fc640 1 -- 172.21.15.32:0/2736470980 learned_addr learned my addr 172.21.15.32:0/2736470980 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:11:13.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.381+0000 7f6f377fe640 1 --2- 172.21.15.32:0/2736470980 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f380d4210 0x7f6f380d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:13.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.382+0000 7f6f35ffb640 1 -- 172.21.15.32:0/2736470980 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1532942569 0 0) 0x7f6f380d3170 con 0x7f6f3806b820 2024-04-24T23:11:13.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.382+0000 7f6f35ffb640 1 -- 172.21.15.32:0/2736470980 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6f24003520 con 0x7f6f3806b820 2024-04-24T23:11:13.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.382+0000 7f6f35ffb640 1 -- 172.21.15.32:0/2736470980 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3722742284 0 0) 0x7f6f24003520 con 0x7f6f3806b820 2024-04-24T23:11:13.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.382+0000 7f6f35ffb640 1 -- 172.21.15.32:0/2736470980 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f380d4210 msgr2=0x7f6f380d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:13.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.382+0000 7f6f35ffb640 1 --2- 172.21.15.32:0/2736470980 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f380d4210 0x7f6f380d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:13.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.382+0000 7f6f35ffb640 1 -- 172.21.15.32:0/2736470980 >> v1:172.21.15.121:6789/0 conn(0x7f6f3815e9b0 legacy=0x7f6f3806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:13.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.382+0000 7f6f35ffb640 1 -- 172.21.15.32:0/2736470980 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f380d3a10 con 0x7f6f3806b820 2024-04-24T23:11:13.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.382+0000 7f6f35ffb640 1 -- 172.21.15.32:0/2736470980 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f6f20001670 con 0x7f6f3806b820 2024-04-24T23:11:13.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.382+0000 7f6f35ffb640 1 -- 172.21.15.32:0/2736470980 >> v1:172.21.15.32:6789/0 conn(0x7f6f3806b820 legacy=0x7f6f380b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:13.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.382+0000 7f6f35ffb640 1 --2- 172.21.15.32:0/2736470980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f240039e0 0x7f6f24003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:13.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.382+0000 7f6f35ffb640 1 --2- 172.21.15.32:0/2736470980 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f380d4210 0x7f6f24004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:13.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.382+0000 7f6f35ffb640 1 --2- 172.21.15.32:0/2736470980 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f24004d40 0x7f6f240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:13.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.382+0000 7f6f35ffb640 1 -- 172.21.15.32:0/2736470980 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6f380d3730 con 0x7f6f240039e0 2024-04-24T23:11:13.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.382+0000 7f6f35ffb640 1 -- 172.21.15.32:0/2736470980 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6f2400b9f0 con 0x7f6f24004d40 2024-04-24T23:11:13.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.382+0000 7f6f35ffb640 1 -- 172.21.15.32:0/2736470980 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f6f2400bbf0 con 0x7f6f380d4210 2024-04-24T23:11:13.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.382+0000 7f6f36ffd640 1 --2- 172.21.15.32:0/2736470980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f240039e0 0x7f6f24003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:13.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.382+0000 7f6f367fc640 1 --2- 172.21.15.32:0/2736470980 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f380d4210 0x7f6f24004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:13.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.382+0000 7f6f377fe640 1 --2- 172.21.15.32:0/2736470980 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f24004d40 0x7f6f240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:13.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.382+0000 7f6f36ffd640 1 -- 172.21.15.32:0/2736470980 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f380d4210 msgr2=0x7f6f24004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:13.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.382+0000 7f6f36ffd640 1 --2- 172.21.15.32:0/2736470980 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f380d4210 0x7f6f24004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:13.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.382+0000 7f6f36ffd640 1 -- 172.21.15.32:0/2736470980 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f24004d40 msgr2=0x7f6f240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:13.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.382+0000 7f6f36ffd640 1 --2- 172.21.15.32:0/2736470980 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f24004d40 0x7f6f240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:13.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.382+0000 7f6f36ffd640 1 -- 172.21.15.32:0/2736470980 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f24003520 con 0x7f6f240039e0 2024-04-24T23:11:13.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.383+0000 7f6f36ffd640 1 --2- 172.21.15.32:0/2736470980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f240039e0 0x7f6f24003dc0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f6f30001860 tx=0x7f6f24005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a344a3d551cd4c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:13.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.383+0000 7f6f35ffb640 1 -- 172.21.15.32:0/2736470980 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f30037070 con 0x7f6f240039e0 2024-04-24T23:11:13.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.383+0000 7f6f35ffb640 1 -- 172.21.15.32:0/2736470980 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f3002ab60 con 0x7f6f240039e0 2024-04-24T23:11:13.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.383+0000 7f6f35ffb640 1 -- 172.21.15.32:0/2736470980 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f30001430 con 0x7f6f240039e0 2024-04-24T23:11:13.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.384+0000 7f6f3d876640 1 -- 172.21.15.32:0/2736470980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f240039e0 msgr2=0x7f6f24003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:13.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.384+0000 7f6f3d876640 1 --2- 172.21.15.32:0/2736470980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f240039e0 0x7f6f24003dc0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f6f30001860 tx=0x7f6f24005830 comp rx=0 tx=0).stop 2024-04-24T23:11:13.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.384+0000 7f6f377fe640 1 -- 172.21.15.32:0/2736470980 reap_dead start 2024-04-24T23:11:13.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.384+0000 7f6f3d876640 1 -- 172.21.15.32:0/2736470980 shutdown_connections 2024-04-24T23:11:13.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.384+0000 7f6f3d876640 1 -- 172.21.15.32:0/2736470980 >> 172.21.15.32:0/2736470980 conn(0x7f6f380d11d0 msgr2=0x7f6f380d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:13.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.384+0000 7f6f3d876640 1 -- 172.21.15.32:0/2736470980 shutdown_connections 2024-04-24T23:11:13.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.384+0000 7f6f3d876640 1 -- 172.21.15.32:0/2736470980 wait complete. 2024-04-24T23:11:13.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.385+0000 7f6f3d876640 1 Processor -- start 2024-04-24T23:11:13.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.385+0000 7f6f3d876640 1 -- start start 2024-04-24T23:11:13.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.386+0000 7f6f3d876640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f240039e0 0x7f6f380acf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:13.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.386+0000 7f6f3d876640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f380aba00 0x7f6f3814cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:13.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.386+0000 7f6f3d876640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f380cdac0 0x7f6f3814d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:13.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.386+0000 7f6f3d876640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6f380ba0e0 con 0x7f6f240039e0 2024-04-24T23:11:13.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.386+0000 7f6f3d876640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6f380d2400 con 0x7f6f380cdac0 2024-04-24T23:11:13.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.386+0000 7f6f3d876640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f6f380ba260 con 0x7f6f380aba00 2024-04-24T23:11:13.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.386+0000 7f6f36ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f240039e0 0x7f6f380acf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:13.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.386+0000 7f6f36ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f240039e0 0x7f6f380acf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49268/0 (socket says 172.21.15.32:49268) 2024-04-24T23:11:13.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.386+0000 7f6f36ffd640 1 -- 172.21.15.32:0/2643597634 learned_addr learned my addr 172.21.15.32:0/2643597634 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:13.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.386+0000 7f6f367fc640 1 --2- 172.21.15.32:0/2643597634 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f380aba00 0x7f6f3814cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:13.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.386+0000 7f6f377fe640 1 --2- 172.21.15.32:0/2643597634 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f380cdac0 0x7f6f3814d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:13.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.386+0000 7f6f36ffd640 1 -- 172.21.15.32:0/2643597634 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f380aba00 msgr2=0x7f6f3814cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:13.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.386+0000 7f6f36ffd640 1 --2- 172.21.15.32:0/2643597634 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f380aba00 0x7f6f3814cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:13.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.386+0000 7f6f36ffd640 1 -- 172.21.15.32:0/2643597634 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f380cdac0 msgr2=0x7f6f3814d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:13.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.386+0000 7f6f36ffd640 1 --2- 172.21.15.32:0/2643597634 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f380cdac0 0x7f6f3814d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:13.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.386+0000 7f6f36ffd640 1 -- 172.21.15.32:0/2643597634 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f380abec0 con 0x7f6f240039e0 2024-04-24T23:11:13.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.386+0000 7f6f377fe640 1 --2- 172.21.15.32:0/2643597634 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f380cdac0 0x7f6f3814d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:11:13.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.386+0000 7f6f36ffd640 1 --2- 172.21.15.32:0/2643597634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f240039e0 0x7f6f380acf60 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f6f300018c0 tx=0x7f6f30001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:13.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.387+0000 7f6f2ffff640 1 -- 172.21.15.32:0/2643597634 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3003f020 con 0x7f6f240039e0 2024-04-24T23:11:13.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.387+0000 7f6f2ffff640 1 -- 172.21.15.32:0/2643597634 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f30005270 con 0x7f6f240039e0 2024-04-24T23:11:13.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.387+0000 7f6f3d876640 1 -- 172.21.15.32:0/2643597634 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f380ac1a0 con 0x7f6f240039e0 2024-04-24T23:11:13.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.387+0000 7f6f2ffff640 1 -- 172.21.15.32:0/2643597634 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f30037040 con 0x7f6f240039e0 2024-04-24T23:11:13.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.387+0000 7f6f3d876640 1 -- 172.21.15.32:0/2643597634 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f380ac760 con 0x7f6f240039e0 2024-04-24T23:11:13.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.388+0000 7f6f3d876640 1 -- 172.21.15.32:0/2643597634 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f380690e0 con 0x7f6f240039e0 2024-04-24T23:11:13.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.388+0000 7f6f2ffff640 1 -- 172.21.15.32:0/2643597634 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6f30042020 con 0x7f6f240039e0 2024-04-24T23:11:13.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.388+0000 7f6f2ffff640 1 --2- 172.21.15.32:0/2643597634 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6f1806be60 0x7f6f1806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:13.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.389+0000 7f6f2ffff640 1 -- 172.21.15.32:0/2643597634 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(190..190 src has 1..190) ==== 5738+0+0 (secure 0 0 0) 0x7f6f300aabc0 con 0x7f6f240039e0 2024-04-24T23:11:13.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.389+0000 7f6f367fc640 1 --2- 172.21.15.32:0/2643597634 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6f1806be60 0x7f6f1806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:13.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.389+0000 7f6f367fc640 1 --2- 172.21.15.32:0/2643597634 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6f1806be60 0x7f6f1806e320 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f6f381f7bc0 tx=0x7f6f20009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:13.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.393+0000 7f6f2ffff640 1 -- 172.21.15.32:0/2643597634 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6f380690e0 con 0x7f6f240039e0 2024-04-24T23:11:13.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.587+0000 7f6f3d876640 1 -- 172.21.15.32:0/2643597634 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6f38068430 con 0x7f6f240039e0 2024-04-24T23:11:13.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.587+0000 7f6f2ffff640 1 -- 172.21.15.32:0/2643597634 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v190) ==== 80+0+0 (secure 0 0 0) 0x7f6f300ac080 con 0x7f6f240039e0 2024-04-24T23:11:13.589 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2024-04-24T23:11:13.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.589+0000 7f6f3d876640 1 -- 172.21.15.32:0/2643597634 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6f1806be60 msgr2=0x7f6f1806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:13.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.589+0000 7f6f3d876640 1 --2- 172.21.15.32:0/2643597634 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6f1806be60 0x7f6f1806e320 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f6f381f7bc0 tx=0x7f6f20009040 comp rx=0 tx=0).stop 2024-04-24T23:11:13.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.589+0000 7f6f3d876640 1 -- 172.21.15.32:0/2643597634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f240039e0 msgr2=0x7f6f380acf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:13.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.589+0000 7f6f3d876640 1 --2- 172.21.15.32:0/2643597634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f240039e0 0x7f6f380acf60 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f6f300018c0 tx=0x7f6f30001230 comp rx=0 tx=0).stop 2024-04-24T23:11:13.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.589+0000 7f6f3d876640 1 -- 172.21.15.32:0/2643597634 shutdown_connections 2024-04-24T23:11:13.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.589+0000 7f6f3d876640 1 --2- 172.21.15.32:0/2643597634 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f6f1806be60 0x7f6f1806e320 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:13.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.589+0000 7f6f3d876640 1 --2- 172.21.15.32:0/2643597634 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f380cdac0 0x7f6f3814d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:13.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.589+0000 7f6f3d876640 1 --2- 172.21.15.32:0/2643597634 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f380aba00 0x7f6f3814cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:13.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.589+0000 7f6f3d876640 1 --2- 172.21.15.32:0/2643597634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f240039e0 0x7f6f380acf60 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:13.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.589+0000 7f6f3d876640 1 -- 172.21.15.32:0/2643597634 >> 172.21.15.32:0/2643597634 conn(0x7f6f380d11d0 msgr2=0x7f6f380b81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:13.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.589+0000 7f6f3d876640 1 -- 172.21.15.32:0/2643597634 shutdown_connections 2024-04-24T23:11:13.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.589+0000 7f6f3d876640 1 -- 172.21.15.32:0/2643597634 wait complete. 2024-04-24T23:11:13.602 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-24T23:11:13.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:11:13.602 DEBUG:teuthology.orchestra.run.smithi121:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-24T23:11:13.634 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2024-04-24T23:11:13.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:11:13.634 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-24T23:11:13.692 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:11:13.692 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:11:13.692 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:11:13.708 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=413 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T23:11:13.709 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:11:13.709 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:11:13.709 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:11:13.750 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3271 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T23:11:13.750 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:11:13.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.849+0000 7ff534b90640 1 Processor -- start 2024-04-24T23:11:13.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.849+0000 7ff534b90640 1 -- start start 2024-04-24T23:11:13.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.850+0000 7ff534b90640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff53006b820 0x7ff5300d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:13.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.850+0000 7ff534b90640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff5300d4210 0x7ff5300b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:13.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.850+0000 7ff534b90640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5300d3170 con 0x7ff53015e9b0 2024-04-24T23:11:13.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.850+0000 7ff534b90640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7ff5300d3450 con 0x7ff5300d4210 2024-04-24T23:11:13.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.850+0000 7ff534b90640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff5300d3730 con 0x7ff53006b820 2024-04-24T23:11:13.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.850+0000 7ff52e59c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff53015e9b0 0x7ff53006b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35904/0 (socket says 172.21.15.32:35904) 2024-04-24T23:11:13.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.851+0000 7ff52e59c640 1 -- 172.21.15.32:0/852857487 learned_addr learned my addr 172.21.15.32:0/852857487 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:11:13.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.851+0000 7ff52d59a640 1 -- 172.21.15.32:0/852857487 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2417552625 0 0) 0x7ff5300d3170 con 0x7ff53015e9b0 2024-04-24T23:11:13.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.851+0000 7ff52d59a640 1 -- 172.21.15.32:0/852857487 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff518003520 con 0x7ff53015e9b0 2024-04-24T23:11:13.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.851+0000 7ff52dd9b640 1 --2- 172.21.15.32:0/852857487 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff53006b820 0x7ff5300d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:13.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.851+0000 7ff52ed9d640 1 --2- 172.21.15.32:0/852857487 >> v2:172.21.15.121:3300/0 conn(0x7ff5300d4210 0x7ff5300b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:13.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.851+0000 7ff52d59a640 1 -- 172.21.15.32:0/852857487 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3757557851 0 0) 0x7ff518003520 con 0x7ff53015e9b0 2024-04-24T23:11:13.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.851+0000 7ff52d59a640 1 -- 172.21.15.32:0/852857487 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff53006b820 msgr2=0x7ff5300d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:13.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.851+0000 7ff52d59a640 1 --2- 172.21.15.32:0/852857487 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff53006b820 0x7ff5300d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:13.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.851+0000 7ff52d59a640 1 -- 172.21.15.32:0/852857487 >> v2:172.21.15.121:3300/0 conn(0x7ff5300d4210 msgr2=0x7ff5300b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:13.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.851+0000 7ff52d59a640 1 --2- 172.21.15.32:0/852857487 >> v2:172.21.15.121:3300/0 conn(0x7ff5300d4210 0x7ff5300b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:13.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.851+0000 7ff52d59a640 1 -- 172.21.15.32:0/852857487 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5300d3a10 con 0x7ff53015e9b0 2024-04-24T23:11:13.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.851+0000 7ff52d59a640 1 -- 172.21.15.32:0/852857487 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7ff528001670 con 0x7ff53015e9b0 2024-04-24T23:11:13.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.851+0000 7ff52d59a640 1 -- 172.21.15.32:0/852857487 >> v1:172.21.15.32:6789/0 conn(0x7ff53015e9b0 legacy=0x7ff53006b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:13.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.851+0000 7ff52d59a640 1 --2- 172.21.15.32:0/852857487 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff5180039b0 0x7ff518003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:13.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.851+0000 7ff52d59a640 1 --2- 172.21.15.32:0/852857487 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff53006b820 0x7ff518004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:13.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.851+0000 7ff52d59a640 1 --2- 172.21.15.32:0/852857487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff518004d40 0x7ff5180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:13.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.851+0000 7ff52d59a640 1 -- 172.21.15.32:0/852857487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff5300d3450 con 0x7ff518004d40 2024-04-24T23:11:13.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.851+0000 7ff52d59a640 1 -- 172.21.15.32:0/852857487 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff5300d3730 con 0x7ff5180039b0 2024-04-24T23:11:13.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.851+0000 7ff52d59a640 1 -- 172.21.15.32:0/852857487 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff51800bac0 con 0x7ff53006b820 2024-04-24T23:11:13.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.851+0000 7ff52ed9d640 1 --2- 172.21.15.32:0/852857487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff518004d40 0x7ff5180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:13.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.851+0000 7ff52dd9b640 1 --2- 172.21.15.32:0/852857487 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff53006b820 0x7ff518004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:13.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.851+0000 7ff52e59c640 1 --2- 172.21.15.32:0/852857487 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff5180039b0 0x7ff518003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:13.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.852+0000 7ff52ed9d640 1 -- 172.21.15.32:0/852857487 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff53006b820 msgr2=0x7ff518004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:13.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.852+0000 7ff52ed9d640 1 --2- 172.21.15.32:0/852857487 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff53006b820 0x7ff518004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:13.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.852+0000 7ff52ed9d640 1 -- 172.21.15.32:0/852857487 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff5180039b0 msgr2=0x7ff518003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:13.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.852+0000 7ff52ed9d640 1 --2- 172.21.15.32:0/852857487 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff5180039b0 0x7ff518003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:13.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.852+0000 7ff52ed9d640 1 -- 172.21.15.32:0/852857487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff518003520 con 0x7ff518004d40 2024-04-24T23:11:13.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.852+0000 7ff52ed9d640 1 --2- 172.21.15.32:0/852857487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff518004d40 0x7ff5180051c0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7ff53006bdf0 tx=0x7ff518005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6ed76e0647d0dd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:13.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.852+0000 7ff52d59a640 1 -- 172.21.15.32:0/852857487 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff520033070 con 0x7ff518004d40 2024-04-24T23:11:13.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.852+0000 7ff52d59a640 1 -- 172.21.15.32:0/852857487 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff52002f650 con 0x7ff518004d40 2024-04-24T23:11:13.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.852+0000 7ff52d59a640 1 -- 172.21.15.32:0/852857487 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff520009be0 con 0x7ff518004d40 2024-04-24T23:11:13.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.853+0000 7ff534b90640 1 -- 172.21.15.32:0/852857487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff518004d40 msgr2=0x7ff5180051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:13.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.853+0000 7ff534b90640 1 --2- 172.21.15.32:0/852857487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff518004d40 0x7ff5180051c0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7ff53006bdf0 tx=0x7ff518005830 comp rx=0 tx=0).stop 2024-04-24T23:11:13.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.853+0000 7ff52ed9d640 1 -- 172.21.15.32:0/852857487 reap_dead start 2024-04-24T23:11:13.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.853+0000 7ff534b90640 1 -- 172.21.15.32:0/852857487 shutdown_connections 2024-04-24T23:11:13.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.853+0000 7ff534b90640 1 -- 172.21.15.32:0/852857487 >> 172.21.15.32:0/852857487 conn(0x7ff5300d11d0 msgr2=0x7ff5300d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:13.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.853+0000 7ff534b90640 1 -- 172.21.15.32:0/852857487 shutdown_connections 2024-04-24T23:11:13.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.853+0000 7ff534b90640 1 -- 172.21.15.32:0/852857487 wait complete. 2024-04-24T23:11:13.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.854+0000 7ff534b90640 1 Processor -- start 2024-04-24T23:11:13.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.854+0000 7ff534b90640 1 -- start start 2024-04-24T23:11:13.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.855+0000 7ff534b90640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff518004d40 0x7ff530150dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:13.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.855+0000 7ff534b90640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff530161130 0x7ff5301615b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:13.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.855+0000 7ff534b90640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff5300cd5c0 0x7ff5300cda40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:13.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.855+0000 7ff534b90640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff5300ba0e0 con 0x7ff530161130 2024-04-24T23:11:13.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.855+0000 7ff534b90640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff5300d2400 con 0x7ff518004d40 2024-04-24T23:11:13.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.855+0000 7ff534b90640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff5300ba260 con 0x7ff5300cd5c0 2024-04-24T23:11:13.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.855+0000 7ff52dd9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff530161130 0x7ff5301615b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:13.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.855+0000 7ff52dd9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff530161130 0x7ff5301615b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49284/0 (socket says 172.21.15.32:49284) 2024-04-24T23:11:13.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.855+0000 7ff52dd9b640 1 -- 172.21.15.32:0/919578653 learned_addr learned my addr 172.21.15.32:0/919578653 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:13.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.855+0000 7ff52ed9d640 1 --2- 172.21.15.32:0/919578653 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff5300cd5c0 0x7ff5300cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:13.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.856+0000 7ff52e59c640 1 --2- 172.21.15.32:0/919578653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff518004d40 0x7ff530150dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:13.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.856+0000 7ff52dd9b640 1 -- 172.21.15.32:0/919578653 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff5300cd5c0 msgr2=0x7ff5300cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:13.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.856+0000 7ff52dd9b640 1 --2- 172.21.15.32:0/919578653 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff5300cd5c0 0x7ff5300cda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:13.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.856+0000 7ff52dd9b640 1 -- 172.21.15.32:0/919578653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff518004d40 msgr2=0x7ff530150dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:13.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.856+0000 7ff52dd9b640 1 --2- 172.21.15.32:0/919578653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff518004d40 0x7ff530150dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:13.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.856+0000 7ff52dd9b640 1 -- 172.21.15.32:0/919578653 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5300cb930 con 0x7ff530161130 2024-04-24T23:11:13.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.856+0000 7ff52ed9d640 1 --2- 172.21.15.32:0/919578653 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff5300cd5c0 0x7ff5300cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:11:13.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.856+0000 7ff52dd9b640 1 --2- 172.21.15.32:0/919578653 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff530161130 0x7ff5301615b0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7ff52400b820 tx=0x7ff52400bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:13.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.856+0000 7ff51effd640 1 -- 172.21.15.32:0/919578653 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff524018070 con 0x7ff530161130 2024-04-24T23:11:13.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.856+0000 7ff51effd640 1 -- 172.21.15.32:0/919578653 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff52400be10 con 0x7ff530161130 2024-04-24T23:11:13.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.856+0000 7ff51effd640 1 -- 172.21.15.32:0/919578653 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff524010c60 con 0x7ff530161130 2024-04-24T23:11:13.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.856+0000 7ff534b90640 1 -- 172.21.15.32:0/919578653 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5300cbc10 con 0x7ff530161130 2024-04-24T23:11:13.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.856+0000 7ff534b90640 1 -- 172.21.15.32:0/919578653 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5300cc1d0 con 0x7ff530161130 2024-04-24T23:11:13.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.857+0000 7ff534b90640 1 -- 172.21.15.32:0/919578653 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff530068430 con 0x7ff530161130 2024-04-24T23:11:13.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.857+0000 7ff51effd640 1 -- 172.21.15.32:0/919578653 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff52401b070 con 0x7ff530161130 2024-04-24T23:11:13.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.858+0000 7ff51effd640 1 --2- 172.21.15.32:0/919578653 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff51006be30 0x7ff51006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:13.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.858+0000 7ff51effd640 1 -- 172.21.15.32:0/919578653 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(190..190 src has 1..190) ==== 5738+0+0 (secure 0 0 0) 0x7ff52408b270 con 0x7ff530161130 2024-04-24T23:11:13.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.858+0000 7ff52e59c640 1 --2- 172.21.15.32:0/919578653 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff51006be30 0x7ff51006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:13.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.859+0000 7ff52e59c640 1 --2- 172.21.15.32:0/919578653 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff51006be30 0x7ff51006e2f0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7ff528007bf0 tx=0x7ff528009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:13.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:13.862+0000 7ff51effd640 1 -- 172.21.15.32:0/919578653 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff524059c00 con 0x7ff530161130 2024-04-24T23:11:14.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.081+0000 7ff534b90640 1 -- 172.21.15.32:0/919578653 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff5300d2870 con 0x7ff530161130 2024-04-24T23:11:14.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.082+0000 7ff51effd640 1 -- 172.21.15.32:0/919578653 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) ==== 94+0+5185 (secure 0 0 0) 0x7ff524059500 con 0x7ff530161130 2024-04-24T23:11:14.084 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:11:14.084 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":80,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6941,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/416518156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":416518156},{"type":"v1","addr":"172.21.15.121:6837","nonce":416518156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7762,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1506667233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1506667233},{"type":"v1","addr":"172.21.15.121:6835","nonce":1506667233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":9063,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/2215079101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":2215079101},{"type":"v1","addr":"172.21.15.32:6834","nonce":2215079101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:10:36.353761+0000","modified":"2024-04-24T23:10:38.436407+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9066},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9066":{"gid":9066,"name":"a","rank":0,"incarnation":78,"state":"up:active","state_seq":4,"addr":"172.21.15.32:6835/901353412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":901353412},{"type":"v1","addr":"172.21.15.32:6835","nonce":901353412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9066,"qdb_cluster":[9066]},"id":11}]} 2024-04-24T23:11:14.084 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 80 2024-04-24T23:11:14.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.084+0000 7ff534b90640 1 -- 172.21.15.32:0/919578653 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff51006be30 msgr2=0x7ff51006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:14.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.084+0000 7ff534b90640 1 --2- 172.21.15.32:0/919578653 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff51006be30 0x7ff51006e2f0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7ff528007bf0 tx=0x7ff528009040 comp rx=0 tx=0).stop 2024-04-24T23:11:14.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.084+0000 7ff534b90640 1 -- 172.21.15.32:0/919578653 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff530161130 msgr2=0x7ff5301615b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:14.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.084+0000 7ff534b90640 1 --2- 172.21.15.32:0/919578653 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff530161130 0x7ff5301615b0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7ff52400b820 tx=0x7ff52400bce0 comp rx=0 tx=0).stop 2024-04-24T23:11:14.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.084+0000 7ff534b90640 1 -- 172.21.15.32:0/919578653 shutdown_connections 2024-04-24T23:11:14.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.084+0000 7ff534b90640 1 --2- 172.21.15.32:0/919578653 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff51006be30 0x7ff51006e2f0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:14.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.084+0000 7ff534b90640 1 --2- 172.21.15.32:0/919578653 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff530161130 0x7ff5301615b0 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:14.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.084+0000 7ff534b90640 1 --2- 172.21.15.32:0/919578653 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff5300cd5c0 0x7ff5300cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:14.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.084+0000 7ff534b90640 1 --2- 172.21.15.32:0/919578653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff518004d40 0x7ff530150dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:14.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.084+0000 7ff534b90640 1 -- 172.21.15.32:0/919578653 >> 172.21.15.32:0/919578653 conn(0x7ff5300d11d0 msgr2=0x7ff5300ca290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:14.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.084+0000 7ff534b90640 1 -- 172.21.15.32:0/919578653 shutdown_connections 2024-04-24T23:11:14.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.084+0000 7ff534b90640 1 -- 172.21.15.32:0/919578653 wait complete. 2024-04-24T23:11:14.097 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:11:14.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.196+0000 7f79cf807640 1 Processor -- start 2024-04-24T23:11:14.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.196+0000 7f79cf807640 1 -- start start 2024-04-24T23:11:14.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.197+0000 7f79cf807640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f79c806cd00 0x7f79c80ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:14.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.197+0000 7f79cf807640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f79c815cb00 0x7f79c80ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:14.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.197+0000 7f79cf807640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79c80b4520 con 0x7f79c815b380 2024-04-24T23:11:14.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.197+0000 7f79cf807640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f79c80b4800 con 0x7f79c806cd00 2024-04-24T23:11:14.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.197+0000 7f79cf807640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f79c80b4ae0 con 0x7f79c815cb00 2024-04-24T23:11:14.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.197+0000 7f79ccda2640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f79c806cd00 0x7f79c80ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:14.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.197+0000 7f79cd5a3640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f79c815b380 0x7f79c806c5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35916/0 (socket says 172.21.15.32:35916) 2024-04-24T23:11:14.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.197+0000 7f79cd5a3640 1 -- 172.21.15.32:0/1925106036 learned_addr learned my addr 172.21.15.32:0/1925106036 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:11:14.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.198+0000 7f79cdda4640 1 --2- 172.21.15.32:0/1925106036 >> v2:172.21.15.121:3300/0 conn(0x7f79c815cb00 0x7f79c80ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:14.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.198+0000 7f79ccda2640 1 -- 172.21.15.32:0/1925106036 >> v2:172.21.15.121:3300/0 conn(0x7f79c815cb00 msgr2=0x7f79c80ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:14.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.198+0000 7f79ccda2640 1 --2- 172.21.15.32:0/1925106036 >> v2:172.21.15.121:3300/0 conn(0x7f79c815cb00 0x7f79c80ce970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:14.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.198+0000 7f79ccda2640 1 -- 172.21.15.32:0/1925106036 >> v1:172.21.15.32:6789/0 conn(0x7f79c815b380 legacy=0x7f79c806c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:14.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.198+0000 7f79bffff640 1 -- 172.21.15.32:0/1925106036 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 61990408 0 0) 0x7f79c80b4520 con 0x7f79c815b380 2024-04-24T23:11:14.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.198+0000 7f79ccda2640 1 -- 172.21.15.32:0/1925106036 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79c80b4dc0 con 0x7f79c806cd00 2024-04-24T23:11:14.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.198+0000 7f79cdda4640 1 --2- 172.21.15.32:0/1925106036 >> v2:172.21.15.121:3300/0 conn(0x7f79c815cb00 0x7f79c80ce970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:11:14.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.198+0000 7f79ccda2640 1 --2- 172.21.15.32:0/1925106036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c806cd00 0x7f79c80ce430 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f79c80cdd20 tx=0x7f79c402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc233fac6bc98221 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:14.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.199+0000 7f79bffff640 1 -- 172.21.15.32:0/1925106036 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c4037070 con 0x7f79c806cd00 2024-04-24T23:11:14.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.199+0000 7f79bffff640 1 -- 172.21.15.32:0/1925106036 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79c4032440 con 0x7f79c806cd00 2024-04-24T23:11:14.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.199+0000 7f79bffff640 1 -- 172.21.15.32:0/1925106036 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c402ae60 con 0x7f79c806cd00 2024-04-24T23:11:14.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.199+0000 7f79cf807640 1 -- 172.21.15.32:0/1925106036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c806cd00 msgr2=0x7f79c80ce430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:14.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.199+0000 7f79cf807640 1 --2- 172.21.15.32:0/1925106036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c806cd00 0x7f79c80ce430 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f79c80cdd20 tx=0x7f79c402a110 comp rx=0 tx=0).stop 2024-04-24T23:11:14.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.199+0000 7f79cf807640 1 -- 172.21.15.32:0/1925106036 shutdown_connections 2024-04-24T23:11:14.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.199+0000 7f79cf807640 1 --2- 172.21.15.32:0/1925106036 >> v2:172.21.15.121:3300/0 conn(0x7f79c815cb00 0x7f79c80ce970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:14.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.199+0000 7f79cf807640 1 --2- 172.21.15.32:0/1925106036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c806cd00 0x7f79c80ce430 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:14.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.199+0000 7f79cf807640 1 -- 172.21.15.32:0/1925106036 >> 172.21.15.32:0/1925106036 conn(0x7f79c806b110 msgr2=0x7f79c806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:14.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.199+0000 7f79cf807640 1 -- 172.21.15.32:0/1925106036 shutdown_connections 2024-04-24T23:11:14.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.199+0000 7f79cf807640 1 -- 172.21.15.32:0/1925106036 wait complete. 2024-04-24T23:11:14.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.200+0000 7f79cf807640 1 Processor -- start 2024-04-24T23:11:14.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.201+0000 7f79cf807640 1 -- start start 2024-04-24T23:11:14.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.201+0000 7f79cf807640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c806cd00 0x7f79c81e4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:14.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.201+0000 7f79cf807640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79c815b380 0x7f79c8145720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:14.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.201+0000 7f79cf807640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f79c815cb00 0x7f79c8071f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:14.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.201+0000 7f79cf807640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f79c815b760 con 0x7f79c806cd00 2024-04-24T23:11:14.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.201+0000 7f79cf807640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f79c80ceeb0 con 0x7f79c815b380 2024-04-24T23:11:14.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.201+0000 7f79cf807640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f79c80b55c0 con 0x7f79c815cb00 2024-04-24T23:11:14.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.201+0000 7f79cd5a3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c806cd00 0x7f79c81e4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:14.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.201+0000 7f79cd5a3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c806cd00 0x7f79c81e4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49308/0 (socket says 172.21.15.32:49308) 2024-04-24T23:11:14.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.201+0000 7f79cd5a3640 1 -- 172.21.15.32:0/27721280 learned_addr learned my addr 172.21.15.32:0/27721280 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:14.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.202+0000 7f79ccda2640 1 --2- 172.21.15.32:0/27721280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79c815b380 0x7f79c8145720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:14.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.202+0000 7f79cdda4640 1 --2- 172.21.15.32:0/27721280 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f79c815cb00 0x7f79c8071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:14.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.202+0000 7f79cd5a3640 1 -- 172.21.15.32:0/27721280 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f79c815cb00 msgr2=0x7f79c8071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:14.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.202+0000 7f79cd5a3640 1 --2- 172.21.15.32:0/27721280 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f79c815cb00 0x7f79c8071f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:14.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.202+0000 7f79cd5a3640 1 -- 172.21.15.32:0/27721280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79c815b380 msgr2=0x7f79c8145720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:14.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.202+0000 7f79cd5a3640 1 --2- 172.21.15.32:0/27721280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79c815b380 0x7f79c8145720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:14.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.202+0000 7f79cd5a3640 1 -- 172.21.15.32:0/27721280 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79c8070e70 con 0x7f79c806cd00 2024-04-24T23:11:14.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.202+0000 7f79ccda2640 1 --2- 172.21.15.32:0/27721280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79c815b380 0x7f79c8145720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:11:14.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.202+0000 7f79cdda4640 1 --2- 172.21.15.32:0/27721280 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f79c815cb00 0x7f79c8071f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:11:14.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.202+0000 7f79cd5a3640 1 --2- 172.21.15.32:0/27721280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c806cd00 0x7f79c81e4d90 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f79b800c4d0 tx=0x7f79b800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:14.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.202+0000 7f79bdffb640 1 -- 172.21.15.32:0/27721280 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b8018070 con 0x7f79c806cd00 2024-04-24T23:11:14.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.202+0000 7f79bdffb640 1 -- 172.21.15.32:0/27721280 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79b8007500 con 0x7f79c806cd00 2024-04-24T23:11:14.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.202+0000 7f79bdffb640 1 -- 172.21.15.32:0/27721280 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b8004a20 con 0x7f79c806cd00 2024-04-24T23:11:14.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.202+0000 7f79cf807640 1 -- 172.21.15.32:0/27721280 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79c8071150 con 0x7f79c806cd00 2024-04-24T23:11:14.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.202+0000 7f79cf807640 1 -- 172.21.15.32:0/27721280 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79c8071710 con 0x7f79c806cd00 2024-04-24T23:11:14.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.203+0000 7f79cf807640 1 -- 172.21.15.32:0/27721280 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79c8068430 con 0x7f79c806cd00 2024-04-24T23:11:14.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.203+0000 7f79bdffb640 1 -- 172.21.15.32:0/27721280 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f79b801b070 con 0x7f79c806cd00 2024-04-24T23:11:14.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.204+0000 7f79bdffb640 1 --2- 172.21.15.32:0/27721280 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f79ac06be30 0x7f79ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:14.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.204+0000 7f79bdffb640 1 -- 172.21.15.32:0/27721280 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(190..190 src has 1..190) ==== 5738+0+0 (secure 0 0 0) 0x7f79b808a200 con 0x7f79c806cd00 2024-04-24T23:11:14.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.204+0000 7f79ccda2640 1 --2- 172.21.15.32:0/27721280 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f79ac06be30 0x7f79ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:14.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.205+0000 7f79ccda2640 1 --2- 172.21.15.32:0/27721280 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f79ac06be30 0x7f79ac06e2f0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f79c4034000 tx=0x7f79c402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:14.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.208+0000 7f79bdffb640 1 -- 172.21.15.32:0/27721280 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f79b8058b90 con 0x7f79c806cd00 2024-04-24T23:11:14.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.391+0000 7f79cf807640 1 -- 172.21.15.32:0/27721280 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f79c806c000 con 0x7f79c806cd00 2024-04-24T23:11:14.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.392+0000 7f79bdffb640 1 -- 172.21.15.32:0/27721280 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v190) ==== 74+0+16748 (secure 0 0 0) 0x7f79b8058490 con 0x7f79c806cd00 2024-04-24T23:11:14.394 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:11:14.394 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":190,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:11:06.861172+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:10:34.649562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-24T23:10:35.339604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"190","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"23.6","mappings":[{"from":5,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:11:14.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.394+0000 7f79cf807640 1 -- 172.21.15.32:0/27721280 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f79ac06be30 msgr2=0x7f79ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:14.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.394+0000 7f79cf807640 1 --2- 172.21.15.32:0/27721280 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f79ac06be30 0x7f79ac06e2f0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f79c4034000 tx=0x7f79c402e000 comp rx=0 tx=0).stop 2024-04-24T23:11:14.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.394+0000 7f79cf807640 1 -- 172.21.15.32:0/27721280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c806cd00 msgr2=0x7f79c81e4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:14.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.394+0000 7f79cf807640 1 --2- 172.21.15.32:0/27721280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c806cd00 0x7f79c81e4d90 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f79b800c4d0 tx=0x7f79b800c990 comp rx=0 tx=0).stop 2024-04-24T23:11:14.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.394+0000 7f79cf807640 1 -- 172.21.15.32:0/27721280 shutdown_connections 2024-04-24T23:11:14.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.394+0000 7f79cf807640 1 --2- 172.21.15.32:0/27721280 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f79ac06be30 0x7f79ac06e2f0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:14.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.394+0000 7f79cf807640 1 --2- 172.21.15.32:0/27721280 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f79c815cb00 0x7f79c8071f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:14.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.394+0000 7f79cf807640 1 --2- 172.21.15.32:0/27721280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79c815b380 0x7f79c8145720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:14.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.394+0000 7f79cf807640 1 --2- 172.21.15.32:0/27721280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c806cd00 0x7f79c81e4d90 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:14.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.394+0000 7f79cf807640 1 -- 172.21.15.32:0/27721280 >> 172.21.15.32:0/27721280 conn(0x7f79c806b110 msgr2=0x7f79c806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:14.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.394+0000 7f79cf807640 1 -- 172.21.15.32:0/27721280 shutdown_connections 2024-04-24T23:11:14.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.394+0000 7f79cf807640 1 -- 172.21.15.32:0/27721280 wait complete. 2024-04-24T23:11:14.407 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:58:53.171408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:10:34.649562+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '184', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:10:35.339604+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '190', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:11:14.407 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:11:14.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.508+0000 7f802835a640 1 Processor -- start 2024-04-24T23:11:14.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.508+0000 7f802835a640 1 -- start start 2024-04-24T23:11:14.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.509+0000 7f802835a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f802006b820 0x7f80200d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:14.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.509+0000 7f802835a640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f80200d4210 0x7f80200b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:14.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.509+0000 7f802835a640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80200d3170 con 0x7f802015e9b0 2024-04-24T23:11:14.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.509+0000 7f802835a640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f80200d3450 con 0x7f802006b820 2024-04-24T23:11:14.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.509+0000 7f802835a640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f80200d3730 con 0x7f80200d4210 2024-04-24T23:11:14.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.510+0000 7f80258f5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f802006b820 0x7f80200d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:14.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.510+0000 7f80260f6640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f802015e9b0 0x7f802006b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35930/0 (socket says 172.21.15.32:35930) 2024-04-24T23:11:14.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.510+0000 7f80260f6640 1 -- 172.21.15.32:0/3900742451 learned_addr learned my addr 172.21.15.32:0/3900742451 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:11:14.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.510+0000 7f80268f7640 1 --2- 172.21.15.32:0/3900742451 >> v2:172.21.15.121:3300/0 conn(0x7f80200d4210 0x7f80200b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:14.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.510+0000 7f80258f5640 1 -- 172.21.15.32:0/3900742451 >> v2:172.21.15.121:3300/0 conn(0x7f80200d4210 msgr2=0x7f80200b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:14.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.510+0000 7f80258f5640 1 --2- 172.21.15.32:0/3900742451 >> v2:172.21.15.121:3300/0 conn(0x7f80200d4210 0x7f80200b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:14.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.510+0000 7f80258f5640 1 -- 172.21.15.32:0/3900742451 >> v1:172.21.15.32:6789/0 conn(0x7f802015e9b0 legacy=0x7f802006b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:14.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.510+0000 7f80250f4640 1 -- 172.21.15.32:0/3900742451 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1067524179 0 0) 0x7f80200d3170 con 0x7f802015e9b0 2024-04-24T23:11:14.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.510+0000 7f80258f5640 1 -- 172.21.15.32:0/3900742451 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80200d3a10 con 0x7f802006b820 2024-04-24T23:11:14.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.510+0000 7f80268f7640 1 --2- 172.21.15.32:0/3900742451 >> v2:172.21.15.121:3300/0 conn(0x7f80200d4210 0x7f80200b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:11:14.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.511+0000 7f80258f5640 1 --2- 172.21.15.32:0/3900742451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f802006b820 0x7f80200d2b90 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f80201622f0 tx=0x7f801002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1dcc4fb62a066d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:14.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.511+0000 7f80250f4640 1 -- 172.21.15.32:0/3900742451 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8010037070 con 0x7f802006b820 2024-04-24T23:11:14.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.511+0000 7f80250f4640 1 -- 172.21.15.32:0/3900742451 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8010032440 con 0x7f802006b820 2024-04-24T23:11:14.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.511+0000 7f80250f4640 1 -- 172.21.15.32:0/3900742451 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801002ae60 con 0x7f802006b820 2024-04-24T23:11:14.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.511+0000 7f802835a640 1 -- 172.21.15.32:0/3900742451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f802006b820 msgr2=0x7f80200d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:14.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.511+0000 7f802835a640 1 --2- 172.21.15.32:0/3900742451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f802006b820 0x7f80200d2b90 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f80201622f0 tx=0x7f801002a110 comp rx=0 tx=0).stop 2024-04-24T23:11:14.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.511+0000 7f802835a640 1 -- 172.21.15.32:0/3900742451 shutdown_connections 2024-04-24T23:11:14.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.511+0000 7f802835a640 1 --2- 172.21.15.32:0/3900742451 >> v2:172.21.15.121:3300/0 conn(0x7f80200d4210 0x7f80200b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:14.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.511+0000 7f802835a640 1 --2- 172.21.15.32:0/3900742451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f802006b820 0x7f80200d2b90 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:14.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.511+0000 7f802835a640 1 -- 172.21.15.32:0/3900742451 >> 172.21.15.32:0/3900742451 conn(0x7f80200d11d0 msgr2=0x7f80200d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:14.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.511+0000 7f802835a640 1 -- 172.21.15.32:0/3900742451 shutdown_connections 2024-04-24T23:11:14.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.511+0000 7f802835a640 1 -- 172.21.15.32:0/3900742451 wait complete. 2024-04-24T23:11:14.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.512+0000 7f802835a640 1 Processor -- start 2024-04-24T23:11:14.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.513+0000 7f802835a640 1 -- start start 2024-04-24T23:11:14.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.513+0000 7f802835a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f802006b820 0x7f80200cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:14.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.513+0000 7f802835a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80200d4210 0x7f80201f77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:14.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.513+0000 7f802835a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f802015e9b0 0x7f80201f7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:14.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.513+0000 7f802835a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f80200ba0e0 con 0x7f802015e9b0 2024-04-24T23:11:14.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.513+0000 7f802835a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f80200d2400 con 0x7f80200d4210 2024-04-24T23:11:14.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.513+0000 7f802835a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f80200ba260 con 0x7f802006b820 2024-04-24T23:11:14.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.513+0000 7f80268f7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f802015e9b0 0x7f80201f7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:14.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.513+0000 7f80268f7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f802015e9b0 0x7f80201f7d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49330/0 (socket says 172.21.15.32:49330) 2024-04-24T23:11:14.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.513+0000 7f80268f7640 1 -- 172.21.15.32:0/4148176896 learned_addr learned my addr 172.21.15.32:0/4148176896 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:14.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.514+0000 7f80258f5640 1 --2- 172.21.15.32:0/4148176896 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80200d4210 0x7f80201f77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:14.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.514+0000 7f80260f6640 1 --2- 172.21.15.32:0/4148176896 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f802006b820 0x7f80200cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:14.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.514+0000 7f80268f7640 1 -- 172.21.15.32:0/4148176896 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f802006b820 msgr2=0x7f80200cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:14.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.514+0000 7f80268f7640 1 --2- 172.21.15.32:0/4148176896 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f802006b820 0x7f80200cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:14.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.514+0000 7f80268f7640 1 -- 172.21.15.32:0/4148176896 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80200d4210 msgr2=0x7f80201f77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:14.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.514+0000 7f80268f7640 1 --2- 172.21.15.32:0/4148176896 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80200d4210 0x7f80201f77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:14.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.514+0000 7f80268f7640 1 -- 172.21.15.32:0/4148176896 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80201f8460 con 0x7f802015e9b0 2024-04-24T23:11:14.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.514+0000 7f80258f5640 1 --2- 172.21.15.32:0/4148176896 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80200d4210 0x7f80201f77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:11:14.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.514+0000 7f80260f6640 1 --2- 172.21.15.32:0/4148176896 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f802006b820 0x7f80200cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:11:14.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.514+0000 7f80268f7640 1 --2- 172.21.15.32:0/4148176896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f802015e9b0 0x7f80201f7d00 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f8018007ef0 tx=0x7f801800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:14.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.514+0000 7f8016ffd640 1 -- 172.21.15.32:0/4148176896 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801801a070 con 0x7f802015e9b0 2024-04-24T23:11:14.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.514+0000 7f8016ffd640 1 -- 172.21.15.32:0/4148176896 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8018003b90 con 0x7f802015e9b0 2024-04-24T23:11:14.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.514+0000 7f802835a640 1 -- 172.21.15.32:0/4148176896 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80201f4370 con 0x7f802015e9b0 2024-04-24T23:11:14.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.514+0000 7f8016ffd640 1 -- 172.21.15.32:0/4148176896 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8018004030 con 0x7f802015e9b0 2024-04-24T23:11:14.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.514+0000 7f802835a640 1 -- 172.21.15.32:0/4148176896 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80200ce1a0 con 0x7f802015e9b0 2024-04-24T23:11:14.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.515+0000 7f802835a640 1 -- 172.21.15.32:0/4148176896 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8020068430 con 0x7f802015e9b0 2024-04-24T23:11:14.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.516+0000 7f8016ffd640 1 -- 172.21.15.32:0/4148176896 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f801801d070 con 0x7f802015e9b0 2024-04-24T23:11:14.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.516+0000 7f8016ffd640 1 --2- 172.21.15.32:0/4148176896 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f800406be30 0x7f800406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:14.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.516+0000 7f8016ffd640 1 -- 172.21.15.32:0/4148176896 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(190..190 src has 1..190) ==== 5738+0+0 (secure 0 0 0) 0x7f801808b0c0 con 0x7f802015e9b0 2024-04-24T23:11:14.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.516+0000 7f80260f6640 1 --2- 172.21.15.32:0/4148176896 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f800406be30 0x7f800406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:14.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.517+0000 7f80260f6640 1 --2- 172.21.15.32:0/4148176896 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f800406be30 0x7f800406e2f0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f801c000f60 tx=0x7f801c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:14.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.520+0000 7f8016ffd640 1 -- 172.21.15.32:0/4148176896 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8018010070 con 0x7f802015e9b0 2024-04-24T23:11:14.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.700+0000 7f802835a640 1 -- 172.21.15.32:0/4148176896 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f802006b600 con 0x7f802015e9b0 2024-04-24T23:11:14.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.701+0000 7f8016ffd640 1 -- 172.21.15.32:0/4148176896 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v190) ==== 93+0+10 (secure 0 0 0) 0x7f8018059400 con 0x7f802015e9b0 2024-04-24T23:11:14.703 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-24T23:11:14.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.703+0000 7f802835a640 1 -- 172.21.15.32:0/4148176896 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f800406be30 msgr2=0x7f800406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:14.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.703+0000 7f802835a640 1 --2- 172.21.15.32:0/4148176896 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f800406be30 0x7f800406e2f0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f801c000f60 tx=0x7f801c009040 comp rx=0 tx=0).stop 2024-04-24T23:11:14.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.703+0000 7f802835a640 1 -- 172.21.15.32:0/4148176896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f802015e9b0 msgr2=0x7f80201f7d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:14.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.703+0000 7f802835a640 1 --2- 172.21.15.32:0/4148176896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f802015e9b0 0x7f80201f7d00 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f8018007ef0 tx=0x7f801800d4a0 comp rx=0 tx=0).stop 2024-04-24T23:11:14.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.703+0000 7f802835a640 1 -- 172.21.15.32:0/4148176896 shutdown_connections 2024-04-24T23:11:14.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.703+0000 7f802835a640 1 --2- 172.21.15.32:0/4148176896 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f800406be30 0x7f800406e2f0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:14.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.703+0000 7f802835a640 1 --2- 172.21.15.32:0/4148176896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f802015e9b0 0x7f80201f7d00 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:14.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.703+0000 7f802835a640 1 --2- 172.21.15.32:0/4148176896 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80200d4210 0x7f80201f77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:14.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.703+0000 7f802835a640 1 --2- 172.21.15.32:0/4148176896 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f802006b820 0x7f80200cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:14.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.703+0000 7f802835a640 1 -- 172.21.15.32:0/4148176896 >> 172.21.15.32:0/4148176896 conn(0x7f80200d11d0 msgr2=0x7f80200c9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:14.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.703+0000 7f802835a640 1 -- 172.21.15.32:0/4148176896 shutdown_connections 2024-04-24T23:11:14.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.703+0000 7f802835a640 1 -- 172.21.15.32:0/4148176896 wait complete. 2024-04-24T23:11:14.715 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:11:14.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.817+0000 7f3c1ded4640 1 Processor -- start 2024-04-24T23:11:14.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.817+0000 7f3c1ded4640 1 -- start start 2024-04-24T23:11:14.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.818+0000 7f3c1ded4640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3c1806c5f0 0x7f3c1806c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:14.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.818+0000 7f3c1ded4640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3c180b44d0 0x7f3c180b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:14.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.818+0000 7f3c1ded4640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3c180af890 0x7f3c180cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:14.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.818+0000 7f3c1ded4640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f3c180b6430 con 0x7f3c180b44d0 2024-04-24T23:11:14.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.818+0000 7f3c1ded4640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f3c18156f80 con 0x7f3c1806c5f0 2024-04-24T23:11:14.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.818+0000 7f3c1ded4640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f3c180b6150 con 0x7f3c180af890 2024-04-24T23:11:14.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.818+0000 7f3c16ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3c180b44d0 0x7f3c180b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:14.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.818+0000 7f3c16ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3c180b44d0 0x7f3c180b5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49338/0 (socket says 172.21.15.32:49338) 2024-04-24T23:11:14.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.818+0000 7f3c16ffd640 1 -- 172.21.15.32:0/2014183438 learned_addr learned my addr 172.21.15.32:0/2014183438 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:14.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.818+0000 7f3c17fff640 1 --2- 172.21.15.32:0/2014183438 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3c180af890 0x7f3c180cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:14.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.818+0000 7f3c177fe640 1 --2- 172.21.15.32:0/2014183438 >> v2:172.21.15.121:3300/0 conn(0x7f3c1806c5f0 0x7f3c1806c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:14.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.818+0000 7f3c16ffd640 1 -- 172.21.15.32:0/2014183438 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3c180af890 msgr2=0x7f3c180cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:14.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.818+0000 7f3c16ffd640 1 --2- 172.21.15.32:0/2014183438 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3c180af890 0x7f3c180cef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:14.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.818+0000 7f3c16ffd640 1 -- 172.21.15.32:0/2014183438 >> v2:172.21.15.121:3300/0 conn(0x7f3c1806c5f0 msgr2=0x7f3c1806c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:14.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.818+0000 7f3c16ffd640 1 --2- 172.21.15.32:0/2014183438 >> v2:172.21.15.121:3300/0 conn(0x7f3c1806c5f0 0x7f3c1806c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:14.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.818+0000 7f3c16ffd640 1 -- 172.21.15.32:0/2014183438 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c180b39f0 con 0x7f3c180b44d0 2024-04-24T23:11:14.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.819+0000 7f3c16ffd640 1 --2- 172.21.15.32:0/2014183438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c180b44d0 0x7f3c180b5890 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f3c1806c1f0 tx=0x7f3c0c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=860c5999116d8645 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:14.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.819+0000 7f3c167fc640 1 -- 172.21.15.32:0/2014183438 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c037070 con 0x7f3c180b44d0 2024-04-24T23:11:14.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.819+0000 7f3c167fc640 1 -- 172.21.15.32:0/2014183438 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c0c032440 con 0x7f3c180b44d0 2024-04-24T23:11:14.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.819+0000 7f3c167fc640 1 -- 172.21.15.32:0/2014183438 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c032810 con 0x7f3c180b44d0 2024-04-24T23:11:14.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.820+0000 7f3c1ded4640 1 -- 172.21.15.32:0/2014183438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c180b44d0 msgr2=0x7f3c180b5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:14.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.820+0000 7f3c1ded4640 1 --2- 172.21.15.32:0/2014183438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c180b44d0 0x7f3c180b5890 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f3c1806c1f0 tx=0x7f3c0c02a140 comp rx=0 tx=0).stop 2024-04-24T23:11:14.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.820+0000 7f3c1ded4640 1 -- 172.21.15.32:0/2014183438 shutdown_connections 2024-04-24T23:11:14.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.820+0000 7f3c1ded4640 1 --2- 172.21.15.32:0/2014183438 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3c180af890 0x7f3c180cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:14.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.820+0000 7f3c1ded4640 1 --2- 172.21.15.32:0/2014183438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c180b44d0 0x7f3c180b5890 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:14.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.820+0000 7f3c1ded4640 1 --2- 172.21.15.32:0/2014183438 >> v2:172.21.15.121:3300/0 conn(0x7f3c1806c5f0 0x7f3c1806c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:14.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.820+0000 7f3c1ded4640 1 -- 172.21.15.32:0/2014183438 >> 172.21.15.32:0/2014183438 conn(0x7f3c1806b110 msgr2=0x7f3c1806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:14.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.820+0000 7f3c1ded4640 1 -- 172.21.15.32:0/2014183438 shutdown_connections 2024-04-24T23:11:14.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.820+0000 7f3c1ded4640 1 -- 172.21.15.32:0/2014183438 wait complete. 2024-04-24T23:11:14.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.821+0000 7f3c1ded4640 1 Processor -- start 2024-04-24T23:11:14.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.821+0000 7f3c1ded4640 1 -- start start 2024-04-24T23:11:14.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.822+0000 7f3c1ded4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c1806c5f0 0x7f3c181f5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:14.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.822+0000 7f3c1ded4640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3c180af890 0x7f3c18159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:14.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.822+0000 7f3c1ded4640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c180b44d0 0x7f3c1815b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:14.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.822+0000 7f3c1ded4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1806cf10 con 0x7f3c1806c5f0 2024-04-24T23:11:14.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.822+0000 7f3c1ded4640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3c180b3fb0 con 0x7f3c180b44d0 2024-04-24T23:11:14.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.822+0000 7f3c1ded4640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f3c180b5dd0 con 0x7f3c180af890 2024-04-24T23:11:14.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.822+0000 7f3c177fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c1806c5f0 0x7f3c181f5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:14.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.822+0000 7f3c177fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c1806c5f0 0x7f3c181f5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49346/0 (socket says 172.21.15.32:49346) 2024-04-24T23:11:14.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.822+0000 7f3c177fe640 1 -- 172.21.15.32:0/2354195699 learned_addr learned my addr 172.21.15.32:0/2354195699 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:14.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.822+0000 7f3c16ffd640 1 --2- 172.21.15.32:0/2354195699 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3c180af890 0x7f3c18159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:14.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.822+0000 7f3c17fff640 1 --2- 172.21.15.32:0/2354195699 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c180b44d0 0x7f3c1815b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:14.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.822+0000 7f3c177fe640 1 -- 172.21.15.32:0/2354195699 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3c180af890 msgr2=0x7f3c18159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:14.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.822+0000 7f3c177fe640 1 --2- 172.21.15.32:0/2354195699 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3c180af890 0x7f3c18159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:14.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.822+0000 7f3c177fe640 1 -- 172.21.15.32:0/2354195699 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c180b44d0 msgr2=0x7f3c1815b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:14.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.822+0000 7f3c177fe640 1 --2- 172.21.15.32:0/2354195699 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c180b44d0 0x7f3c1815b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:14.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.822+0000 7f3c177fe640 1 -- 172.21.15.32:0/2354195699 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c1815bba0 con 0x7f3c1806c5f0 2024-04-24T23:11:14.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.822+0000 7f3c17fff640 1 --2- 172.21.15.32:0/2354195699 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c180b44d0 0x7f3c1815b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:11:14.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.822+0000 7f3c177fe640 1 --2- 172.21.15.32:0/2354195699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c1806c5f0 0x7f3c181f5170 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f3c0800d5f0 tx=0x7f3c0800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:14.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.823+0000 7f3bfffff640 1 -- 172.21.15.32:0/2354195699 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c080125f0 con 0x7f3c1806c5f0 2024-04-24T23:11:14.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.823+0000 7f3bfffff640 1 -- 172.21.15.32:0/2354195699 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c08019070 con 0x7f3c1806c5f0 2024-04-24T23:11:14.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.823+0000 7f3c1ded4640 1 -- 172.21.15.32:0/2354195699 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c1815be80 con 0x7f3c1806c5f0 2024-04-24T23:11:14.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.823+0000 7f3bfffff640 1 -- 172.21.15.32:0/2354195699 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c080129d0 con 0x7f3c1806c5f0 2024-04-24T23:11:14.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.823+0000 7f3c1ded4640 1 -- 172.21.15.32:0/2354195699 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c18159b60 con 0x7f3c1806c5f0 2024-04-24T23:11:14.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.824+0000 7f3c1ded4640 1 -- 172.21.15.32:0/2354195699 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c18068430 con 0x7f3c1806c5f0 2024-04-24T23:11:14.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.824+0000 7f3bfffff640 1 -- 172.21.15.32:0/2354195699 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3c08012b70 con 0x7f3c1806c5f0 2024-04-24T23:11:14.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.825+0000 7f3bfffff640 1 --2- 172.21.15.32:0/2354195699 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3c0406be30 0x7f3c0406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:14.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.825+0000 7f3bfffff640 1 -- 172.21.15.32:0/2354195699 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(190..190 src has 1..190) ==== 5738+0+0 (secure 0 0 0) 0x7f3c0808b1d0 con 0x7f3c1806c5f0 2024-04-24T23:11:14.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.825+0000 7f3c16ffd640 1 --2- 172.21.15.32:0/2354195699 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3c0406be30 0x7f3c0406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:14.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.825+0000 7f3c16ffd640 1 --2- 172.21.15.32:0/2354195699 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3c0406be30 0x7f3c0406e2f0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c034000 tx=0x7f3c0c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:14.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:14.829+0000 7f3bfffff640 1 -- 172.21.15.32:0/2354195699 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3c08059b60 con 0x7f3c1806c5f0 2024-04-24T23:11:15.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.009+0000 7f3c1ded4640 1 -- 172.21.15.32:0/2354195699 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3c180b5670 con 0x7f3c1806c5f0 2024-04-24T23:11:15.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.010+0000 7f3bfffff640 1 -- 172.21.15.32:0/2354195699 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v190) ==== 104+0+11 (secure 0 0 0) 0x7f3c08059460 con 0x7f3c1806c5f0 2024-04-24T23:11:15.011 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-24T23:11:15.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.011+0000 7f3c1ded4640 1 -- 172.21.15.32:0/2354195699 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3c0406be30 msgr2=0x7f3c0406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:15.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.011+0000 7f3c1ded4640 1 --2- 172.21.15.32:0/2354195699 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3c0406be30 0x7f3c0406e2f0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c034000 tx=0x7f3c0c02e000 comp rx=0 tx=0).stop 2024-04-24T23:11:15.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.011+0000 7f3c1ded4640 1 -- 172.21.15.32:0/2354195699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c1806c5f0 msgr2=0x7f3c181f5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:15.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.011+0000 7f3c1ded4640 1 --2- 172.21.15.32:0/2354195699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c1806c5f0 0x7f3c181f5170 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f3c0800d5f0 tx=0x7f3c0800dab0 comp rx=0 tx=0).stop 2024-04-24T23:11:15.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.011+0000 7f3c1ded4640 1 -- 172.21.15.32:0/2354195699 shutdown_connections 2024-04-24T23:11:15.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.011+0000 7f3c1ded4640 1 --2- 172.21.15.32:0/2354195699 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3c0406be30 0x7f3c0406e2f0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:15.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.011+0000 7f3c1ded4640 1 --2- 172.21.15.32:0/2354195699 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c180b44d0 0x7f3c1815b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:15.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.011+0000 7f3c1ded4640 1 --2- 172.21.15.32:0/2354195699 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3c180af890 0x7f3c18159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:15.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.011+0000 7f3c1ded4640 1 --2- 172.21.15.32:0/2354195699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c1806c5f0 0x7f3c181f5170 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:15.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.011+0000 7f3c1ded4640 1 -- 172.21.15.32:0/2354195699 >> 172.21.15.32:0/2354195699 conn(0x7f3c1806b110 msgr2=0x7f3c18154d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:15.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.012+0000 7f3c1ded4640 1 -- 172.21.15.32:0/2354195699 shutdown_connections 2024-04-24T23:11:15.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.012+0000 7f3c1ded4640 1 -- 172.21.15.32:0/2354195699 wait complete. 2024-04-24T23:11:15.024 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:11:15.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.126+0000 7fc7d093b640 1 Processor -- start 2024-04-24T23:11:15.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.126+0000 7fc7d093b640 1 -- start start 2024-04-24T23:11:15.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.127+0000 7fc7d093b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc7cc15e810 0x7fc7cc0d1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:15.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.127+0000 7fc7d093b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7cc0b1df0 con 0x7fc7cc1623d0 2024-04-24T23:11:15.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.127+0000 7fc7d093b640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7cc0d2180 con 0x7fc7cc0b6bf0 2024-04-24T23:11:15.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.127+0000 7fc7d093b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fc7cc0b7970 con 0x7fc7cc15e810 2024-04-24T23:11:15.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.127+0000 7fc7ca59c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc7cc15e810 0x7fc7cc0d1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:15.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.127+0000 7fc7cad9d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc7cc1623d0 0x7fc7cc06e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35938/0 (socket says 172.21.15.32:35938) 2024-04-24T23:11:15.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.127+0000 7fc7cad9d640 1 -- 172.21.15.32:0/2886378372 learned_addr learned my addr 172.21.15.32:0/2886378372 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:11:15.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.127+0000 7fc7ca59c640 1 -- 172.21.15.32:0/2886378372 >> v1:172.21.15.121:6789/0 conn(0x7fc7cc0b6bf0 legacy=0x7fc7cc15f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:15.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.127+0000 7fc7ca59c640 1 -- 172.21.15.32:0/2886378372 >> v1:172.21.15.32:6789/0 conn(0x7fc7cc1623d0 legacy=0x7fc7cc06e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:15.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.127+0000 7fc7ca59c640 1 -- 172.21.15.32:0/2886378372 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7cc0b70d0 con 0x7fc7cc15e810 2024-04-24T23:11:15.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.127+0000 7fc7c9d9b640 1 -- 172.21.15.32:0/2886378372 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2000547319 0 0) 0x7fc7cc0b1df0 con 0x7fc7cc1623d0 2024-04-24T23:11:15.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.128+0000 7fc7ca59c640 1 --2- 172.21.15.32:0/2886378372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7cc15e810 0x7fc7cc0d1c40 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc0097a0 tx=0x7fc7bc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c74be7037335f55 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:15.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.128+0000 7fc7c9d9b640 1 -- 172.21.15.32:0/2886378372 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc037070 con 0x7fc7cc15e810 2024-04-24T23:11:15.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.128+0000 7fc7c9d9b640 1 -- 172.21.15.32:0/2886378372 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7bc032440 con 0x7fc7cc15e810 2024-04-24T23:11:15.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.128+0000 7fc7c9d9b640 1 -- 172.21.15.32:0/2886378372 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc032810 con 0x7fc7cc15e810 2024-04-24T23:11:15.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.129+0000 7fc7d093b640 1 -- 172.21.15.32:0/2886378372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7cc15e810 msgr2=0x7fc7cc0d1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:15.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.129+0000 7fc7d093b640 1 --2- 172.21.15.32:0/2886378372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7cc15e810 0x7fc7cc0d1c40 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc0097a0 tx=0x7fc7bc02a230 comp rx=0 tx=0).stop 2024-04-24T23:11:15.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.129+0000 7fc7d093b640 1 -- 172.21.15.32:0/2886378372 shutdown_connections 2024-04-24T23:11:15.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.129+0000 7fc7d093b640 1 --2- 172.21.15.32:0/2886378372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7cc15e810 0x7fc7cc0d1c40 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:15.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.129+0000 7fc7d093b640 1 -- 172.21.15.32:0/2886378372 >> 172.21.15.32:0/2886378372 conn(0x7fc7cc0d24c0 msgr2=0x7fc7cc0d28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:15.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.129+0000 7fc7d093b640 1 -- 172.21.15.32:0/2886378372 shutdown_connections 2024-04-24T23:11:15.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.129+0000 7fc7d093b640 1 -- 172.21.15.32:0/2886378372 wait complete. 2024-04-24T23:11:15.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.130+0000 7fc7d093b640 1 Processor -- start 2024-04-24T23:11:15.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.130+0000 7fc7d093b640 1 -- start start 2024-04-24T23:11:15.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.131+0000 7fc7d093b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7cc0b6bf0 0x7fc7cc155a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:15.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.131+0000 7fc7d093b640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc7cc15e810 0x7fc7cc0cfa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:15.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.131+0000 7fc7d093b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7cc1623d0 0x7fc7cc1573c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:15.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.131+0000 7fc7d093b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc7cc15f010 con 0x7fc7cc1623d0 2024-04-24T23:11:15.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.131+0000 7fc7d093b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc7cc0b7690 con 0x7fc7cc0b6bf0 2024-04-24T23:11:15.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.131+0000 7fc7d093b640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc7cc15fc40 con 0x7fc7cc15e810 2024-04-24T23:11:15.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.131+0000 7fc7cad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7cc1623d0 0x7fc7cc1573c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:15.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.131+0000 7fc7cad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7cc1623d0 0x7fc7cc1573c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49368/0 (socket says 172.21.15.32:49368) 2024-04-24T23:11:15.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.131+0000 7fc7cad9d640 1 -- 172.21.15.32:0/3140905761 learned_addr learned my addr 172.21.15.32:0/3140905761 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:15.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.131+0000 7fc7c1d9b640 1 --2- 172.21.15.32:0/3140905761 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc7cc15e810 0x7fc7cc0cfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:15.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.132+0000 7fc7ca59c640 1 --2- 172.21.15.32:0/3140905761 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7cc0b6bf0 0x7fc7cc155a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:15.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.132+0000 7fc7cad9d640 1 -- 172.21.15.32:0/3140905761 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc7cc15e810 msgr2=0x7fc7cc0cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:15.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.132+0000 7fc7cad9d640 1 --2- 172.21.15.32:0/3140905761 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc7cc15e810 0x7fc7cc0cfa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:15.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.132+0000 7fc7cad9d640 1 -- 172.21.15.32:0/3140905761 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7cc0b6bf0 msgr2=0x7fc7cc155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:15.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.132+0000 7fc7cad9d640 1 --2- 172.21.15.32:0/3140905761 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7cc0b6bf0 0x7fc7cc155a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:15.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.132+0000 7fc7cad9d640 1 -- 172.21.15.32:0/3140905761 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7cc156450 con 0x7fc7cc1623d0 2024-04-24T23:11:15.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.132+0000 7fc7c1d9b640 1 --2- 172.21.15.32:0/3140905761 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc7cc15e810 0x7fc7cc0cfa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:11:15.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.132+0000 7fc7ca59c640 1 --2- 172.21.15.32:0/3140905761 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7cc0b6bf0 0x7fc7cc155a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:11:15.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.132+0000 7fc7cad9d640 1 --2- 172.21.15.32:0/3140905761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7cc1623d0 0x7fc7cc1573c0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fc7c400c730 tx=0x7fc7c400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:15.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.132+0000 7fc7c37fe640 1 -- 172.21.15.32:0/3140905761 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c4018070 con 0x7fc7cc1623d0 2024-04-24T23:11:15.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.132+0000 7fc7c37fe640 1 -- 172.21.15.32:0/3140905761 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7c400ce40 con 0x7fc7cc1623d0 2024-04-24T23:11:15.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.132+0000 7fc7d093b640 1 -- 172.21.15.32:0/3140905761 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7cc155f80 con 0x7fc7cc1623d0 2024-04-24T23:11:15.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.132+0000 7fc7c37fe640 1 -- 172.21.15.32:0/3140905761 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c400a860 con 0x7fc7cc1623d0 2024-04-24T23:11:15.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.132+0000 7fc7d093b640 1 -- 172.21.15.32:0/3140905761 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7cc0af880 con 0x7fc7cc1623d0 2024-04-24T23:11:15.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.133+0000 7fc7d093b640 1 -- 172.21.15.32:0/3140905761 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7cc068430 con 0x7fc7cc1623d0 2024-04-24T23:11:15.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.133+0000 7fc7c37fe640 1 -- 172.21.15.32:0/3140905761 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc7c401b070 con 0x7fc7cc1623d0 2024-04-24T23:11:15.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.134+0000 7fc7c37fe640 1 --2- 172.21.15.32:0/3140905761 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc7ac06be60 0x7fc7ac06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:15.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.134+0000 7fc7c37fe640 1 -- 172.21.15.32:0/3140905761 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(190..190 src has 1..190) ==== 5738+0+0 (secure 0 0 0) 0x7fc7c408a430 con 0x7fc7cc1623d0 2024-04-24T23:11:15.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.134+0000 7fc7ca59c640 1 --2- 172.21.15.32:0/3140905761 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc7ac06be60 0x7fc7ac06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:15.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.135+0000 7fc7ca59c640 1 --2- 172.21.15.32:0/3140905761 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc7ac06be60 0x7fc7ac06e320 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc034000 tx=0x7fc7bc02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:15.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.138+0000 7fc7c37fe640 1 -- 172.21.15.32:0/3140905761 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc7c4058dc0 con 0x7fc7cc1623d0 2024-04-24T23:11:15.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.320+0000 7fc7d093b640 1 -- 172.21.15.32:0/3140905761 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc7cc15a460 con 0x7fc7cc1623d0 2024-04-24T23:11:15.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.320+0000 7fc7c37fe640 1 -- 172.21.15.32:0/3140905761 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v190) ==== 100+0+11 (secure 0 0 0) 0x7fc7c40586c0 con 0x7fc7cc1623d0 2024-04-24T23:11:15.322 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-24T23:11:15.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.322+0000 7fc7d093b640 1 -- 172.21.15.32:0/3140905761 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc7ac06be60 msgr2=0x7fc7ac06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:15.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.322+0000 7fc7d093b640 1 --2- 172.21.15.32:0/3140905761 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc7ac06be60 0x7fc7ac06e320 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc034000 tx=0x7fc7bc02e000 comp rx=0 tx=0).stop 2024-04-24T23:11:15.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.322+0000 7fc7d093b640 1 -- 172.21.15.32:0/3140905761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7cc1623d0 msgr2=0x7fc7cc1573c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:15.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.322+0000 7fc7d093b640 1 --2- 172.21.15.32:0/3140905761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7cc1623d0 0x7fc7cc1573c0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fc7c400c730 tx=0x7fc7c400cbf0 comp rx=0 tx=0).stop 2024-04-24T23:11:15.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.323+0000 7fc7d093b640 1 -- 172.21.15.32:0/3140905761 shutdown_connections 2024-04-24T23:11:15.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.323+0000 7fc7d093b640 1 --2- 172.21.15.32:0/3140905761 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc7ac06be60 0x7fc7ac06e320 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:15.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.323+0000 7fc7d093b640 1 --2- 172.21.15.32:0/3140905761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7cc1623d0 0x7fc7cc1573c0 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:15.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.323+0000 7fc7d093b640 1 --2- 172.21.15.32:0/3140905761 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc7cc15e810 0x7fc7cc0cfa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:15.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.323+0000 7fc7d093b640 1 --2- 172.21.15.32:0/3140905761 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7cc0b6bf0 0x7fc7cc155a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:15.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.323+0000 7fc7d093b640 1 -- 172.21.15.32:0/3140905761 >> 172.21.15.32:0/3140905761 conn(0x7fc7cc0d24c0 msgr2=0x7fc7cc0d28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:15.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.323+0000 7fc7d093b640 1 -- 172.21.15.32:0/3140905761 shutdown_connections 2024-04-24T23:11:15.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.323+0000 7fc7d093b640 1 -- 172.21.15.32:0/3140905761 wait complete. 2024-04-24T23:11:15.335 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:11:15.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.433+0000 7f5f29058640 1 Processor -- start 2024-04-24T23:11:15.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.433+0000 7f5f29058640 1 -- start start 2024-04-24T23:11:15.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.434+0000 7f5f29058640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5f2406cd00 0x7f5f240ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:15.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.434+0000 7f5f29058640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5f2415cb00 0x7f5f240ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:15.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.434+0000 7f5f29058640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f240b4520 con 0x7f5f2415b380 2024-04-24T23:11:15.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.434+0000 7f5f29058640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f5f240b4800 con 0x7f5f2406cd00 2024-04-24T23:11:15.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.434+0000 7f5f29058640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f5f240b4ae0 con 0x7f5f2415cb00 2024-04-24T23:11:15.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.434+0000 7f5f22d9d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5f2415b380 0x7f5f2406c5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35950/0 (socket says 172.21.15.32:35950) 2024-04-24T23:11:15.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.434+0000 7f5f22d9d640 1 -- 172.21.15.32:0/1805474554 learned_addr learned my addr 172.21.15.32:0/1805474554 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:11:15.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.435+0000 7f5f2359e640 1 --2- 172.21.15.32:0/1805474554 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5f2415cb00 0x7f5f240ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:15.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.435+0000 7f5f2259c640 1 --2- 172.21.15.32:0/1805474554 >> v2:172.21.15.121:3300/0 conn(0x7f5f2406cd00 0x7f5f240ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:15.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.435+0000 7f5f21d9b640 1 -- 172.21.15.32:0/1805474554 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3278444660 0 0) 0x7f5f240b4520 con 0x7f5f2415b380 2024-04-24T23:11:15.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.435+0000 7f5f21d9b640 1 -- 172.21.15.32:0/1805474554 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f0c003520 con 0x7f5f2415b380 2024-04-24T23:11:15.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.435+0000 7f5f21d9b640 1 -- 172.21.15.32:0/1805474554 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2367041690 0 0) 0x7f5f0c003520 con 0x7f5f2415b380 2024-04-24T23:11:15.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.435+0000 7f5f21d9b640 1 -- 172.21.15.32:0/1805474554 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5f2415cb00 msgr2=0x7f5f240ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:15.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.435+0000 7f5f21d9b640 1 --2- 172.21.15.32:0/1805474554 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5f2415cb00 0x7f5f240ce970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:15.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.435+0000 7f5f21d9b640 1 -- 172.21.15.32:0/1805474554 >> v2:172.21.15.121:3300/0 conn(0x7f5f2406cd00 msgr2=0x7f5f240ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:15.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.435+0000 7f5f21d9b640 1 --2- 172.21.15.32:0/1805474554 >> v2:172.21.15.121:3300/0 conn(0x7f5f2406cd00 0x7f5f240ce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:15.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.435+0000 7f5f21d9b640 1 -- 172.21.15.32:0/1805474554 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f240b4dc0 con 0x7f5f2415b380 2024-04-24T23:11:15.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.435+0000 7f5f21d9b640 1 -- 172.21.15.32:0/1805474554 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f5f14001670 con 0x7f5f2415b380 2024-04-24T23:11:15.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.435+0000 7f5f2359e640 1 --2- 172.21.15.32:0/1805474554 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5f2415cb00 0x7f5f240ce970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:11:15.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.435+0000 7f5f21d9b640 1 -- 172.21.15.32:0/1805474554 >> v1:172.21.15.32:6789/0 conn(0x7f5f2415b380 legacy=0x7f5f2406c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:15.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.435+0000 7f5f21d9b640 1 --2- 172.21.15.32:0/1805474554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f0c0039b0 0x7f5f0c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:15.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.435+0000 7f5f21d9b640 1 --2- 172.21.15.32:0/1805474554 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5f2415cb00 0x7f5f0c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:15.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.435+0000 7f5f21d9b640 1 --2- 172.21.15.32:0/1805474554 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f0c004d40 0x7f5f0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:15.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.435+0000 7f5f21d9b640 1 -- 172.21.15.32:0/1805474554 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5f240b4800 con 0x7f5f0c0039b0 2024-04-24T23:11:15.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.435+0000 7f5f21d9b640 1 -- 172.21.15.32:0/1805474554 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5f240b4ae0 con 0x7f5f0c004d40 2024-04-24T23:11:15.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.435+0000 7f5f21d9b640 1 -- 172.21.15.32:0/1805474554 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f5f0c00b9c0 con 0x7f5f2415cb00 2024-04-24T23:11:15.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.435+0000 7f5f22d9d640 1 --2- 172.21.15.32:0/1805474554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f0c0039b0 0x7f5f0c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:15.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.435+0000 7f5f2259c640 1 --2- 172.21.15.32:0/1805474554 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5f2415cb00 0x7f5f0c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:15.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.435+0000 7f5f2359e640 1 --2- 172.21.15.32:0/1805474554 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f0c004d40 0x7f5f0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:15.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.435+0000 7f5f22d9d640 1 -- 172.21.15.32:0/1805474554 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5f2415cb00 msgr2=0x7f5f0c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:15.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.435+0000 7f5f22d9d640 1 --2- 172.21.15.32:0/1805474554 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5f2415cb00 0x7f5f0c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:15.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.435+0000 7f5f22d9d640 1 -- 172.21.15.32:0/1805474554 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f0c004d40 msgr2=0x7f5f0c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:15.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.435+0000 7f5f22d9d640 1 --2- 172.21.15.32:0/1805474554 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f0c004d40 0x7f5f0c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:15.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.435+0000 7f5f22d9d640 1 -- 172.21.15.32:0/1805474554 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f0c003520 con 0x7f5f0c0039b0 2024-04-24T23:11:15.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.436+0000 7f5f22d9d640 1 --2- 172.21.15.32:0/1805474554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f0c0039b0 0x7f5f0c003dd0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f5f1400c240 tx=0x7f5f0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5602cbeecb2b9ee9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:15.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.436+0000 7f5f21d9b640 1 -- 172.21.15.32:0/1805474554 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f14007d30 con 0x7f5f0c0039b0 2024-04-24T23:11:15.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.436+0000 7f5f21d9b640 1 -- 172.21.15.32:0/1805474554 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f14005ad0 con 0x7f5f0c0039b0 2024-04-24T23:11:15.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.436+0000 7f5f21d9b640 1 -- 172.21.15.32:0/1805474554 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f14007070 con 0x7f5f0c0039b0 2024-04-24T23:11:15.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.437+0000 7f5f29058640 1 -- 172.21.15.32:0/1805474554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f0c0039b0 msgr2=0x7f5f0c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:15.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.437+0000 7f5f29058640 1 --2- 172.21.15.32:0/1805474554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f0c0039b0 0x7f5f0c003dd0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f5f1400c240 tx=0x7f5f0c005830 comp rx=0 tx=0).stop 2024-04-24T23:11:15.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.437+0000 7f5f2359e640 1 -- 172.21.15.32:0/1805474554 reap_dead start 2024-04-24T23:11:15.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.437+0000 7f5f29058640 1 -- 172.21.15.32:0/1805474554 shutdown_connections 2024-04-24T23:11:15.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.437+0000 7f5f29058640 1 -- 172.21.15.32:0/1805474554 >> 172.21.15.32:0/1805474554 conn(0x7f5f2406b110 msgr2=0x7f5f2406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:15.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.437+0000 7f5f29058640 1 -- 172.21.15.32:0/1805474554 shutdown_connections 2024-04-24T23:11:15.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.437+0000 7f5f29058640 1 -- 172.21.15.32:0/1805474554 wait complete. 2024-04-24T23:11:15.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.438+0000 7f5f29058640 1 Processor -- start 2024-04-24T23:11:15.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.438+0000 7f5f29058640 1 -- start start 2024-04-24T23:11:15.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.439+0000 7f5f29058640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f0c0039b0 0x7f5f2413e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:15.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.439+0000 7f5f29058640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5f240721f0 0x7f5f2414f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:15.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.439+0000 7f5f29058640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f240ae8b0 0x7f5f240aed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:15.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.439+0000 7f5f29058640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5f240ce970 con 0x7f5f0c0039b0 2024-04-24T23:11:15.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.439+0000 7f5f29058640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2415b760 con 0x7f5f240ae8b0 2024-04-24T23:11:15.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.439+0000 7f5f29058640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f5f240ceaf0 con 0x7f5f240721f0 2024-04-24T23:11:15.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.439+0000 7f5f22d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f0c0039b0 0x7f5f2413e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:15.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.439+0000 7f5f22d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f0c0039b0 0x7f5f2413e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49388/0 (socket says 172.21.15.32:49388) 2024-04-24T23:11:15.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.439+0000 7f5f22d9d640 1 -- 172.21.15.32:0/3342498146 learned_addr learned my addr 172.21.15.32:0/3342498146 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:15.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.439+0000 7f5f2259c640 1 --2- 172.21.15.32:0/3342498146 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5f240721f0 0x7f5f2414f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:15.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.439+0000 7f5f2359e640 1 --2- 172.21.15.32:0/3342498146 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f240ae8b0 0x7f5f240aed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:15.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.440+0000 7f5f22d9d640 1 -- 172.21.15.32:0/3342498146 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5f240721f0 msgr2=0x7f5f2414f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:15.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.440+0000 7f5f22d9d640 1 --2- 172.21.15.32:0/3342498146 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5f240721f0 0x7f5f2414f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:15.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.440+0000 7f5f22d9d640 1 -- 172.21.15.32:0/3342498146 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f240ae8b0 msgr2=0x7f5f240aed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:15.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.440+0000 7f5f22d9d640 1 --2- 172.21.15.32:0/3342498146 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f240ae8b0 0x7f5f240aed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:15.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.440+0000 7f5f22d9d640 1 -- 172.21.15.32:0/3342498146 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f24070680 con 0x7f5f0c0039b0 2024-04-24T23:11:15.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.440+0000 7f5f2259c640 1 --2- 172.21.15.32:0/3342498146 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5f240721f0 0x7f5f2414f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:11:15.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.440+0000 7f5f22d9d640 1 --2- 172.21.15.32:0/3342498146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f0c0039b0 0x7f5f2413e080 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f5f1400c360 tx=0x7f5f240b4f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:15.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.440+0000 7f5f0b7fe640 1 -- 172.21.15.32:0/3342498146 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f14042070 con 0x7f5f0c0039b0 2024-04-24T23:11:15.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.440+0000 7f5f0b7fe640 1 -- 172.21.15.32:0/3342498146 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f14007480 con 0x7f5f0c0039b0 2024-04-24T23:11:15.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.440+0000 7f5f0b7fe640 1 -- 172.21.15.32:0/3342498146 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f14001c70 con 0x7f5f0c0039b0 2024-04-24T23:11:15.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.440+0000 7f5f29058640 1 -- 172.21.15.32:0/3342498146 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f24070960 con 0x7f5f0c0039b0 2024-04-24T23:11:15.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.440+0000 7f5f29058640 1 -- 172.21.15.32:0/3342498146 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f24070f20 con 0x7f5f0c0039b0 2024-04-24T23:11:15.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.441+0000 7f5f29058640 1 -- 172.21.15.32:0/3342498146 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f240ceed0 con 0x7f5f0c0039b0 2024-04-24T23:11:15.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.441+0000 7f5f0b7fe640 1 -- 172.21.15.32:0/3342498146 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5f14045070 con 0x7f5f0c0039b0 2024-04-24T23:11:15.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.442+0000 7f5f0b7fe640 1 --2- 172.21.15.32:0/3342498146 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5f1006be60 0x7f5f1006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:15.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.442+0000 7f5f0b7fe640 1 -- 172.21.15.32:0/3342498146 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(190..190 src has 1..190) ==== 5738+0+0 (secure 0 0 0) 0x7f5f140ab930 con 0x7f5f0c0039b0 2024-04-24T23:11:15.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.442+0000 7f5f2259c640 1 --2- 172.21.15.32:0/3342498146 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5f1006be60 0x7f5f1006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:15.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.442+0000 7f5f2259c640 1 --2- 172.21.15.32:0/3342498146 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5f1006be60 0x7f5f1006e320 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f5f24072610 tx=0x7f5f1800d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:15.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.446+0000 7f5f0b7fe640 1 -- 172.21.15.32:0/3342498146 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5f240ceed0 con 0x7f5f0c0039b0 2024-04-24T23:11:15.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.664+0000 7f5f29058640 1 -- 172.21.15.32:0/3342498146 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5f2406c000 con 0x7f5f0c0039b0 2024-04-24T23:11:15.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.665+0000 7f5f0b7fe640 1 -- 172.21.15.32:0/3342498146 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) ==== 94+0+5185 (secure 0 0 0) 0x7f5f1407a3f0 con 0x7f5f0c0039b0 2024-04-24T23:11:15.667 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:11:15.667 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":80,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6941,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/416518156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":416518156},{"type":"v1","addr":"172.21.15.121:6837","nonce":416518156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7762,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1506667233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1506667233},{"type":"v1","addr":"172.21.15.121:6835","nonce":1506667233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":9063,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/2215079101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":2215079101},{"type":"v1","addr":"172.21.15.32:6834","nonce":2215079101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:10:36.353761+0000","modified":"2024-04-24T23:10:38.436407+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9066},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9066":{"gid":9066,"name":"a","rank":0,"incarnation":78,"state":"up:active","state_seq":4,"addr":"172.21.15.32:6835/901353412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":901353412},{"type":"v1","addr":"172.21.15.32:6835","nonce":901353412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9066,"qdb_cluster":[9066]},"id":11}]} 2024-04-24T23:11:15.667 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 80 2024-04-24T23:11:15.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.667+0000 7f5f29058640 1 -- 172.21.15.32:0/3342498146 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5f1006be60 msgr2=0x7f5f1006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:15.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.667+0000 7f5f29058640 1 --2- 172.21.15.32:0/3342498146 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5f1006be60 0x7f5f1006e320 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f5f24072610 tx=0x7f5f1800d040 comp rx=0 tx=0).stop 2024-04-24T23:11:15.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.667+0000 7f5f29058640 1 -- 172.21.15.32:0/3342498146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f0c0039b0 msgr2=0x7f5f2413e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:15.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.667+0000 7f5f29058640 1 --2- 172.21.15.32:0/3342498146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f0c0039b0 0x7f5f2413e080 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f5f1400c360 tx=0x7f5f240b4f90 comp rx=0 tx=0).stop 2024-04-24T23:11:15.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.667+0000 7f5f29058640 1 -- 172.21.15.32:0/3342498146 shutdown_connections 2024-04-24T23:11:15.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.667+0000 7f5f29058640 1 --2- 172.21.15.32:0/3342498146 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5f1006be60 0x7f5f1006e320 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:15.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.667+0000 7f5f29058640 1 --2- 172.21.15.32:0/3342498146 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f240ae8b0 0x7f5f240aed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:15.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.667+0000 7f5f29058640 1 --2- 172.21.15.32:0/3342498146 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5f240721f0 0x7f5f2414f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:15.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.667+0000 7f5f29058640 1 --2- 172.21.15.32:0/3342498146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f0c0039b0 0x7f5f2413e080 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:15.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.667+0000 7f5f29058640 1 -- 172.21.15.32:0/3342498146 >> 172.21.15.32:0/3342498146 conn(0x7f5f2406b110 msgr2=0x7f5f240b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:15.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.667+0000 7f5f29058640 1 -- 172.21.15.32:0/3342498146 shutdown_connections 2024-04-24T23:11:15.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.667+0000 7f5f29058640 1 -- 172.21.15.32:0/3342498146 wait complete. 2024-04-24T23:11:15.679 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:11:15.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.779+0000 7f865ad5a640 1 Processor -- start 2024-04-24T23:11:15.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.779+0000 7f865ad5a640 1 -- start start 2024-04-24T23:11:15.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.779+0000 7f865ad5a640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f865406c5f0 0x7f865406c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:15.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.780+0000 7f865ad5a640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86540b6430 con 0x7f86540b44d0 2024-04-24T23:11:15.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.780+0000 7f865ad5a640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8654156f80 con 0x7f86540af890 2024-04-24T23:11:15.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.780+0000 7f865ad5a640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f86540b6150 con 0x7f865406c5f0 2024-04-24T23:11:15.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.780+0000 7f8653fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f86540b44d0 0x7f86540b5890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35964/0 (socket says 172.21.15.32:35964) 2024-04-24T23:11:15.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.780+0000 7f8653fff640 1 -- 172.21.15.32:0/368158856 learned_addr learned my addr 172.21.15.32:0/368158856 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:11:15.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.780+0000 7f8658af6640 1 --2- 172.21.15.32:0/368158856 >> v2:172.21.15.121:3300/0 conn(0x7f865406c5f0 0x7f865406c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:15.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.780+0000 7f86537fe640 1 -- 172.21.15.32:0/368158856 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4291477168 0 0) 0x7f86540b6430 con 0x7f86540b44d0 2024-04-24T23:11:15.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.780+0000 7f86537fe640 1 -- 172.21.15.32:0/368158856 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8644003520 con 0x7f86540b44d0 2024-04-24T23:11:15.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.780+0000 7f86537fe640 1 -- 172.21.15.32:0/368158856 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 46405360 0 0) 0x7f8644003520 con 0x7f86540b44d0 2024-04-24T23:11:15.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.780+0000 7f86537fe640 1 -- 172.21.15.32:0/368158856 >> v2:172.21.15.121:3300/0 conn(0x7f865406c5f0 msgr2=0x7f865406c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:15.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.780+0000 7f86537fe640 1 --2- 172.21.15.32:0/368158856 >> v2:172.21.15.121:3300/0 conn(0x7f865406c5f0 0x7f865406c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:15.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.780+0000 7f86537fe640 1 -- 172.21.15.32:0/368158856 >> v1:172.21.15.121:6789/0 conn(0x7f86540af890 legacy=0x7f865415cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:15.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.780+0000 7f86537fe640 1 -- 172.21.15.32:0/368158856 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86540b39f0 con 0x7f86540b44d0 2024-04-24T23:11:15.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.780+0000 7f86537fe640 1 -- 172.21.15.32:0/368158856 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f8640001670 con 0x7f86540b44d0 2024-04-24T23:11:15.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.780+0000 7f86537fe640 1 -- 172.21.15.32:0/368158856 >> v1:172.21.15.32:6789/0 conn(0x7f86540b44d0 legacy=0x7f86540b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:15.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.780+0000 7f86537fe640 1 --2- 172.21.15.32:0/368158856 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86440039b0 0x7f8644003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:15.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.780+0000 7f86537fe640 1 --2- 172.21.15.32:0/368158856 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8644004800 0x7f8644004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:15.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.780+0000 7f86537fe640 1 --2- 172.21.15.32:0/368158856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86440051c0 0x7f864400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:15.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.780+0000 7f86537fe640 1 -- 172.21.15.32:0/368158856 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f86540b6150 con 0x7f86440051c0 2024-04-24T23:11:15.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.780+0000 7f86537fe640 1 -- 172.21.15.32:0/368158856 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f864400c060 con 0x7f86440039b0 2024-04-24T23:11:15.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.780+0000 7f86537fe640 1 -- 172.21.15.32:0/368158856 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f864400c260 con 0x7f8644004800 2024-04-24T23:11:15.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.780+0000 7f8658af6640 1 --2- 172.21.15.32:0/368158856 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86440039b0 0x7f8644003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:15.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.780+0000 7f86592f7640 1 --2- 172.21.15.32:0/368158856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86440051c0 0x7f864400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:15.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.781+0000 7f8653fff640 1 --2- 172.21.15.32:0/368158856 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8644004800 0x7f8644004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:15.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.781+0000 7f86592f7640 1 -- 172.21.15.32:0/368158856 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8644004800 msgr2=0x7f8644004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:15.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.781+0000 7f86592f7640 1 --2- 172.21.15.32:0/368158856 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8644004800 0x7f8644004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:15.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.781+0000 7f86592f7640 1 -- 172.21.15.32:0/368158856 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86440039b0 msgr2=0x7f8644003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:15.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.781+0000 7f86592f7640 1 --2- 172.21.15.32:0/368158856 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86440039b0 0x7f8644003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:15.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.781+0000 7f86592f7640 1 -- 172.21.15.32:0/368158856 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8644003520 con 0x7f86440051c0 2024-04-24T23:11:15.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.781+0000 7f86592f7640 1 -- 172.21.15.32:0/368158856 reap_dead start 2024-04-24T23:11:15.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.781+0000 7f86592f7640 1 --2- 172.21.15.32:0/368158856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86440051c0 0x7f864400b9c0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f864c001e50 tx=0x7f8644005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=febd700e931de8c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:15.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.782+0000 7f86537fe640 1 -- 172.21.15.32:0/368158856 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864c001a70 con 0x7f86440051c0 2024-04-24T23:11:15.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.782+0000 7f86537fe640 1 -- 172.21.15.32:0/368158856 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f864c003970 con 0x7f86440051c0 2024-04-24T23:11:15.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.782+0000 7f86537fe640 1 -- 172.21.15.32:0/368158856 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864c003d60 con 0x7f86440051c0 2024-04-24T23:11:15.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.782+0000 7f865ad5a640 1 -- 172.21.15.32:0/368158856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86440051c0 msgr2=0x7f864400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:15.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.782+0000 7f865ad5a640 1 --2- 172.21.15.32:0/368158856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86440051c0 0x7f864400b9c0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f864c001e50 tx=0x7f8644005830 comp rx=0 tx=0).stop 2024-04-24T23:11:15.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.782+0000 7f865ad5a640 1 -- 172.21.15.32:0/368158856 shutdown_connections 2024-04-24T23:11:15.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.782+0000 7f865ad5a640 1 --2- 172.21.15.32:0/368158856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86440051c0 0x7f864400b9c0 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:15.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.782+0000 7f865ad5a640 1 -- 172.21.15.32:0/368158856 >> 172.21.15.32:0/368158856 conn(0x7f865406b110 msgr2=0x7f865406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:15.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.782+0000 7f865ad5a640 1 -- 172.21.15.32:0/368158856 shutdown_connections 2024-04-24T23:11:15.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.782+0000 7f865ad5a640 1 -- 172.21.15.32:0/368158856 wait complete. 2024-04-24T23:11:15.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.783+0000 7f865ad5a640 1 Processor -- start 2024-04-24T23:11:15.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.783+0000 7f865ad5a640 1 -- start start 2024-04-24T23:11:15.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.784+0000 7f865ad5a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86440051c0 0x7f865414fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:15.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.784+0000 7f865ad5a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8654153ee0 0x7f865414cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:15.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.784+0000 7f865ad5a640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8654154340 0x7f865414d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:15.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.784+0000 7f865ad5a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f865406c9d0 con 0x7f8654153ee0 2024-04-24T23:11:15.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.784+0000 7f865ad5a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f86540b3fb0 con 0x7f86440051c0 2024-04-24T23:11:15.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.784+0000 7f865ad5a640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f865406cb50 con 0x7f8654154340 2024-04-24T23:11:15.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.784+0000 7f8653fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8654153ee0 0x7f865414cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:15.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.784+0000 7f8653fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8654153ee0 0x7f865414cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49416/0 (socket says 172.21.15.32:49416) 2024-04-24T23:11:15.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.784+0000 7f8653fff640 1 -- 172.21.15.32:0/3163679166 learned_addr learned my addr 172.21.15.32:0/3163679166 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:15.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.784+0000 7f86592f7640 1 --2- 172.21.15.32:0/3163679166 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8654154340 0x7f865414d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:15.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.784+0000 7f8658af6640 1 --2- 172.21.15.32:0/3163679166 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86440051c0 0x7f865414fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:15.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.785+0000 7f8653fff640 1 -- 172.21.15.32:0/3163679166 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8654154340 msgr2=0x7f865414d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:15.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.785+0000 7f8653fff640 1 --2- 172.21.15.32:0/3163679166 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8654154340 0x7f865414d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:15.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.785+0000 7f8653fff640 1 -- 172.21.15.32:0/3163679166 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86440051c0 msgr2=0x7f865414fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:15.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.785+0000 7f8653fff640 1 --2- 172.21.15.32:0/3163679166 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86440051c0 0x7f865414fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:15.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.785+0000 7f8653fff640 1 -- 172.21.15.32:0/3163679166 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8654151ef0 con 0x7f8654153ee0 2024-04-24T23:11:15.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.785+0000 7f8653fff640 1 --2- 172.21.15.32:0/3163679166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8654153ee0 0x7f865414cb70 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f8640001e70 tx=0x7f8640007660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:15.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.785+0000 7f86517fa640 1 -- 172.21.15.32:0/3163679166 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864000f960 con 0x7f8654153ee0 2024-04-24T23:11:15.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.785+0000 7f86517fa640 1 -- 172.21.15.32:0/3163679166 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8640018070 con 0x7f8654153ee0 2024-04-24T23:11:15.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.785+0000 7f865ad5a640 1 -- 172.21.15.32:0/3163679166 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86541521d0 con 0x7f8654153ee0 2024-04-24T23:11:15.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.785+0000 7f86517fa640 1 -- 172.21.15.32:0/3163679166 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864000fd60 con 0x7f8654153ee0 2024-04-24T23:11:15.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.785+0000 7f865ad5a640 1 -- 172.21.15.32:0/3163679166 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8654152790 con 0x7f8654153ee0 2024-04-24T23:11:15.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.786+0000 7f865ad5a640 1 -- 172.21.15.32:0/3163679166 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8654068430 con 0x7f8654153ee0 2024-04-24T23:11:15.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.787+0000 7f86517fa640 1 -- 172.21.15.32:0/3163679166 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f864001d460 con 0x7f8654153ee0 2024-04-24T23:11:15.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.787+0000 7f86517fa640 1 --2- 172.21.15.32:0/3163679166 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f863806be30 0x7f863806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:15.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.787+0000 7f86517fa640 1 -- 172.21.15.32:0/3163679166 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(190..190 src has 1..190) ==== 5738+0+0 (secure 0 0 0) 0x7f864008adb0 con 0x7f8654153ee0 2024-04-24T23:11:15.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.787+0000 7f8658af6640 1 --2- 172.21.15.32:0/3163679166 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f863806be30 0x7f863806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:15.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.788+0000 7f8658af6640 1 --2- 172.21.15.32:0/3163679166 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f863806be30 0x7f863806e2f0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f8648009b40 tx=0x7f8648009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:15.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.791+0000 7f86517fa640 1 -- 172.21.15.32:0/3163679166 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8640059740 con 0x7f8654153ee0 2024-04-24T23:11:15.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.974+0000 7f865ad5a640 1 -- 172.21.15.32:0/3163679166 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f86540b5670 con 0x7f8654153ee0 2024-04-24T23:11:15.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.975+0000 7f86517fa640 1 -- 172.21.15.32:0/3163679166 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v190) ==== 74+0+16748 (secure 0 0 0) 0x7f8640059040 con 0x7f8654153ee0 2024-04-24T23:11:15.977 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:11:15.977 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":190,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:11:06.861172+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:10:34.649562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-24T23:10:35.339604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"190","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"23.6","mappings":[{"from":5,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:11:15.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.978+0000 7f865ad5a640 1 -- 172.21.15.32:0/3163679166 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f863806be30 msgr2=0x7f863806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:15.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.978+0000 7f865ad5a640 1 --2- 172.21.15.32:0/3163679166 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f863806be30 0x7f863806e2f0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f8648009b40 tx=0x7f8648009190 comp rx=0 tx=0).stop 2024-04-24T23:11:15.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.978+0000 7f865ad5a640 1 -- 172.21.15.32:0/3163679166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8654153ee0 msgr2=0x7f865414cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:15.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.978+0000 7f865ad5a640 1 --2- 172.21.15.32:0/3163679166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8654153ee0 0x7f865414cb70 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f8640001e70 tx=0x7f8640007660 comp rx=0 tx=0).stop 2024-04-24T23:11:15.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.978+0000 7f865ad5a640 1 -- 172.21.15.32:0/3163679166 shutdown_connections 2024-04-24T23:11:15.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.978+0000 7f865ad5a640 1 --2- 172.21.15.32:0/3163679166 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f863806be30 0x7f863806e2f0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:15.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.978+0000 7f865ad5a640 1 --2- 172.21.15.32:0/3163679166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8654153ee0 0x7f865414cb70 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:15.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.978+0000 7f865ad5a640 1 --2- 172.21.15.32:0/3163679166 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8654154340 0x7f865414d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:15.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.978+0000 7f865ad5a640 1 --2- 172.21.15.32:0/3163679166 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86440051c0 0x7f865414fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:15.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.978+0000 7f865ad5a640 1 -- 172.21.15.32:0/3163679166 >> 172.21.15.32:0/3163679166 conn(0x7f865406b110 msgr2=0x7f86540d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:15.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.978+0000 7f865ad5a640 1 -- 172.21.15.32:0/3163679166 shutdown_connections 2024-04-24T23:11:15.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:15.978+0000 7f865ad5a640 1 -- 172.21.15.32:0/3163679166 wait complete. 2024-04-24T23:11:15.990 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-24T23:11:15.991 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:11:16.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.090+0000 7fd9d93c1640 1 Processor -- start 2024-04-24T23:11:16.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.091+0000 7fd9d93c1640 1 -- start start 2024-04-24T23:11:16.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.091+0000 7fd9d93c1640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd9d4162320 0x7fd9d40ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:16.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.091+0000 7fd9d93c1640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd9d40b5af0 0x7fd9d4162700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:16.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.091+0000 7fd9d93c1640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9d40cd270 con 0x7fd9d4162c40 2024-04-24T23:11:16.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.091+0000 7fd9d93c1640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fd9d40b5660 con 0x7fd9d40b5af0 2024-04-24T23:11:16.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.091+0000 7fd9d93c1640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd9d4071610 con 0x7fd9d4162320 2024-04-24T23:11:16.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.092+0000 7fd9d27fc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd9d40b5af0 0x7fd9d4162700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:16.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.092+0000 7fd9d27fc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd9d40b5af0 0x7fd9d4162700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49428/0 (socket says 172.21.15.32:49428) 2024-04-24T23:11:16.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.092+0000 7fd9d27fc640 1 -- 172.21.15.32:0/621900946 learned_addr learned my addr 172.21.15.32:0/621900946 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:16.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.092+0000 7fd9d37fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd9d4162c40 0x7fd9d40ae8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35980/0 (socket says 172.21.15.32:35980) 2024-04-24T23:11:16.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.092+0000 7fd9d2ffd640 1 --2- 172.21.15.32:0/621900946 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd9d4162320 0x7fd9d40ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:16.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.092+0000 7fd9d27fc640 1 -- 172.21.15.32:0/621900946 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd9d4162320 msgr2=0x7fd9d40ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:16.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.092+0000 7fd9d27fc640 1 --2- 172.21.15.32:0/621900946 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd9d4162320 0x7fd9d40ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:16.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.092+0000 7fd9d27fc640 1 -- 172.21.15.32:0/621900946 >> v1:172.21.15.32:6789/0 conn(0x7fd9d4162c40 legacy=0x7fd9d40ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:16.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.092+0000 7fd9d27fc640 1 -- 172.21.15.32:0/621900946 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9d40718f0 con 0x7fd9d40b5af0 2024-04-24T23:11:16.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.092+0000 7fd9d1ffb640 1 -- 172.21.15.32:0/621900946 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3020878683 0 0) 0x7fd9d40cd270 con 0x7fd9d4162c40 2024-04-24T23:11:16.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.093+0000 7fd9d27fc640 1 --2- 172.21.15.32:0/621900946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9d40b5af0 0x7fd9d4162700 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8009870 tx=0x7fd9c802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cd184d80b8d5ad6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:16.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.093+0000 7fd9d1ffb640 1 -- 172.21.15.32:0/621900946 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c8037070 con 0x7fd9d40b5af0 2024-04-24T23:11:16.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.093+0000 7fd9d1ffb640 1 -- 172.21.15.32:0/621900946 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9c8032440 con 0x7fd9d40b5af0 2024-04-24T23:11:16.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.093+0000 7fd9d1ffb640 1 -- 172.21.15.32:0/621900946 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c8032810 con 0x7fd9d40b5af0 2024-04-24T23:11:16.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.093+0000 7fd9d93c1640 1 -- 172.21.15.32:0/621900946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9d40b5af0 msgr2=0x7fd9d4162700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:16.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.093+0000 7fd9d93c1640 1 --2- 172.21.15.32:0/621900946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9d40b5af0 0x7fd9d4162700 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8009870 tx=0x7fd9c802a1c0 comp rx=0 tx=0).stop 2024-04-24T23:11:16.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.093+0000 7fd9d93c1640 1 -- 172.21.15.32:0/621900946 shutdown_connections 2024-04-24T23:11:16.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.093+0000 7fd9d93c1640 1 --2- 172.21.15.32:0/621900946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9d40b5af0 0x7fd9d4162700 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:16.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.093+0000 7fd9d93c1640 1 --2- 172.21.15.32:0/621900946 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd9d4162320 0x7fd9d40ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:16.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.093+0000 7fd9d93c1640 1 -- 172.21.15.32:0/621900946 >> 172.21.15.32:0/621900946 conn(0x7fd9d40d0ae0 msgr2=0x7fd9d40d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:16.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.093+0000 7fd9d93c1640 1 -- 172.21.15.32:0/621900946 shutdown_connections 2024-04-24T23:11:16.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.093+0000 7fd9d93c1640 1 -- 172.21.15.32:0/621900946 wait complete. 2024-04-24T23:11:16.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.095+0000 7fd9d93c1640 1 Processor -- start 2024-04-24T23:11:16.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.095+0000 7fd9d93c1640 1 -- start start 2024-04-24T23:11:16.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.095+0000 7fd9d93c1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9d40b5af0 0x7fd9d40ce5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:16.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.095+0000 7fd9d93c1640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9d4162320 0x7fd9d4146430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:16.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.095+0000 7fd9d93c1640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd9d4162c40 0x7fd9d415a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:16.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.095+0000 7fd9d93c1640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d4067950 con 0x7fd9d40b5af0 2024-04-24T23:11:16.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.095+0000 7fd9d93c1640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d415c630 con 0x7fd9d4162320 2024-04-24T23:11:16.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.095+0000 7fd9d93c1640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd9d4071bd0 con 0x7fd9d4162c40 2024-04-24T23:11:16.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.095+0000 7fd9d2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9d40b5af0 0x7fd9d40ce5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:16.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.095+0000 7fd9d2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9d40b5af0 0x7fd9d40ce5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49444/0 (socket says 172.21.15.32:49444) 2024-04-24T23:11:16.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.096+0000 7fd9d2ffd640 1 -- 172.21.15.32:0/2500777609 learned_addr learned my addr 172.21.15.32:0/2500777609 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:16.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.096+0000 7fd9d27fc640 1 --2- 172.21.15.32:0/2500777609 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9d4162320 0x7fd9d4146430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:16.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.096+0000 7fd9d37fe640 1 --2- 172.21.15.32:0/2500777609 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd9d4162c40 0x7fd9d415a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:16.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.096+0000 7fd9d2ffd640 1 -- 172.21.15.32:0/2500777609 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd9d4162c40 msgr2=0x7fd9d415a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:16.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.096+0000 7fd9d2ffd640 1 --2- 172.21.15.32:0/2500777609 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd9d4162c40 0x7fd9d415a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:16.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.096+0000 7fd9d2ffd640 1 -- 172.21.15.32:0/2500777609 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9d4162320 msgr2=0x7fd9d4146430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:16.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.096+0000 7fd9d2ffd640 1 --2- 172.21.15.32:0/2500777609 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9d4162320 0x7fd9d4146430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:16.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.096+0000 7fd9d2ffd640 1 -- 172.21.15.32:0/2500777609 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9d40cd590 con 0x7fd9d40b5af0 2024-04-24T23:11:16.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.096+0000 7fd9d27fc640 1 --2- 172.21.15.32:0/2500777609 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9d4162320 0x7fd9d4146430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:11:16.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.096+0000 7fd9d2ffd640 1 --2- 172.21.15.32:0/2500777609 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9d40b5af0 0x7fd9d40ce5f0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7fd9c400d800 tx=0x7fd9c400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:16.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.096+0000 7fd9bb7fe640 1 -- 172.21.15.32:0/2500777609 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c400b660 con 0x7fd9d40b5af0 2024-04-24T23:11:16.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.096+0000 7fd9bb7fe640 1 -- 172.21.15.32:0/2500777609 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9c4019070 con 0x7fd9d40b5af0 2024-04-24T23:11:16.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.096+0000 7fd9bb7fe640 1 -- 172.21.15.32:0/2500777609 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c400ba20 con 0x7fd9d40b5af0 2024-04-24T23:11:16.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.096+0000 7fd9d93c1640 1 -- 172.21.15.32:0/2500777609 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9d40cd830 con 0x7fd9d40b5af0 2024-04-24T23:11:16.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.096+0000 7fd9d93c1640 1 -- 172.21.15.32:0/2500777609 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9d40cddf0 con 0x7fd9d40b5af0 2024-04-24T23:11:16.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.097+0000 7fd9d93c1640 1 -- 172.21.15.32:0/2500777609 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9d4068430 con 0x7fd9d40b5af0 2024-04-24T23:11:16.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.098+0000 7fd9bb7fe640 1 -- 172.21.15.32:0/2500777609 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd9c400bbc0 con 0x7fd9d40b5af0 2024-04-24T23:11:16.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.098+0000 7fd9bb7fe640 1 --2- 172.21.15.32:0/2500777609 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd9c006be60 0x7fd9c006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:16.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.098+0000 7fd9bb7fe640 1 -- 172.21.15.32:0/2500777609 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(190..190 src has 1..190) ==== 5738+0+0 (secure 0 0 0) 0x7fd9c408b280 con 0x7fd9d40b5af0 2024-04-24T23:11:16.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.098+0000 7fd9d27fc640 1 --2- 172.21.15.32:0/2500777609 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd9c006be60 0x7fd9c006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:16.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.099+0000 7fd9d27fc640 1 --2- 172.21.15.32:0/2500777609 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd9c006be60 0x7fd9c006e320 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8034000 tx=0x7fd9c802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:16.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.102+0000 7fd9bb7fe640 1 -- 172.21.15.32:0/2500777609 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd9c4059b80 con 0x7fd9d40b5af0 2024-04-24T23:11:16.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.331+0000 7fd9d93c1640 1 -- 172.21.15.32:0/2500777609 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd9d40d0850 con 0x7fd9d40b5af0 2024-04-24T23:11:16.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.332+0000 7fd9bb7fe640 1 -- 172.21.15.32:0/2500777609 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) ==== 94+0+5185 (secure 0 0 0) 0x7fd9c4059480 con 0x7fd9d40b5af0 2024-04-24T23:11:16.334 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:11:16.334 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":80,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6941,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/416518156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":416518156},{"type":"v1","addr":"172.21.15.121:6837","nonce":416518156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7762,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1506667233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1506667233},{"type":"v1","addr":"172.21.15.121:6835","nonce":1506667233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":9063,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/2215079101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":2215079101},{"type":"v1","addr":"172.21.15.32:6834","nonce":2215079101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:10:36.353761+0000","modified":"2024-04-24T23:10:38.436407+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9066},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9066":{"gid":9066,"name":"a","rank":0,"incarnation":78,"state":"up:active","state_seq":4,"addr":"172.21.15.32:6835/901353412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":901353412},{"type":"v1","addr":"172.21.15.32:6835","nonce":901353412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9066,"qdb_cluster":[9066]},"id":11}]} 2024-04-24T23:11:16.334 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 80 2024-04-24T23:11:16.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.334+0000 7fd9d93c1640 1 -- 172.21.15.32:0/2500777609 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd9c006be60 msgr2=0x7fd9c006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:16.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.334+0000 7fd9d93c1640 1 --2- 172.21.15.32:0/2500777609 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd9c006be60 0x7fd9c006e320 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8034000 tx=0x7fd9c802e000 comp rx=0 tx=0).stop 2024-04-24T23:11:16.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.334+0000 7fd9d93c1640 1 -- 172.21.15.32:0/2500777609 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9d40b5af0 msgr2=0x7fd9d40ce5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:16.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.334+0000 7fd9d93c1640 1 --2- 172.21.15.32:0/2500777609 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9d40b5af0 0x7fd9d40ce5f0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7fd9c400d800 tx=0x7fd9c400dcc0 comp rx=0 tx=0).stop 2024-04-24T23:11:16.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.334+0000 7fd9d93c1640 1 -- 172.21.15.32:0/2500777609 shutdown_connections 2024-04-24T23:11:16.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.334+0000 7fd9d93c1640 1 --2- 172.21.15.32:0/2500777609 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd9c006be60 0x7fd9c006e320 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:16.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.334+0000 7fd9d93c1640 1 --2- 172.21.15.32:0/2500777609 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd9d4162c40 0x7fd9d415a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:16.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.334+0000 7fd9d93c1640 1 --2- 172.21.15.32:0/2500777609 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9d4162320 0x7fd9d4146430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:16.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.334+0000 7fd9d93c1640 1 --2- 172.21.15.32:0/2500777609 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9d40b5af0 0x7fd9d40ce5f0 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:16.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.334+0000 7fd9d93c1640 1 -- 172.21.15.32:0/2500777609 >> 172.21.15.32:0/2500777609 conn(0x7fd9d40d0ae0 msgr2=0x7fd9d40b1e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:16.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.334+0000 7fd9d93c1640 1 -- 172.21.15.32:0/2500777609 shutdown_connections 2024-04-24T23:11:16.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.334+0000 7fd9d93c1640 1 -- 172.21.15.32:0/2500777609 wait complete. 2024-04-24T23:11:16.347 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:11:16.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.446+0000 7f79f72ee640 1 Processor -- start 2024-04-24T23:11:16.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.446+0000 7f79f72ee640 1 -- start start 2024-04-24T23:11:16.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.447+0000 7f79f72ee640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f79f015b380 0x7f79f006c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:16.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.447+0000 7f79f72ee640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f79f015cb00 0x7f79f006cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:16.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.447+0000 7f79f72ee640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f79f00ce430 0x7f79f00ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:16.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.447+0000 7f79f72ee640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f79f00b4520 con 0x7f79f015cb00 2024-04-24T23:11:16.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.447+0000 7f79f72ee640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f79f00b4800 con 0x7f79f00ce430 2024-04-24T23:11:16.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.447+0000 7f79f72ee640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f79f00b4ae0 con 0x7f79f015b380 2024-04-24T23:11:16.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.447+0000 7f79f4889640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f79f015cb00 0x7f79f006cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:16.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.447+0000 7f79f4889640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f79f015cb00 0x7f79f006cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49456/0 (socket says 172.21.15.32:49456) 2024-04-24T23:11:16.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.447+0000 7f79f4889640 1 -- 172.21.15.32:0/34864010 learned_addr learned my addr 172.21.15.32:0/34864010 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:16.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.447+0000 7f79f588b640 1 --2- 172.21.15.32:0/34864010 >> v2:172.21.15.121:3300/0 conn(0x7f79f00ce430 0x7f79f00ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:16.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.447+0000 7f79f508a640 1 --2- 172.21.15.32:0/34864010 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f79f015b380 0x7f79f006c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:16.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.447+0000 7f79f4889640 1 -- 172.21.15.32:0/34864010 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f79f015b380 msgr2=0x7f79f006c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:16.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.447+0000 7f79f4889640 1 --2- 172.21.15.32:0/34864010 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f79f015b380 0x7f79f006c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:16.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.447+0000 7f79f4889640 1 -- 172.21.15.32:0/34864010 >> v2:172.21.15.121:3300/0 conn(0x7f79f00ce430 msgr2=0x7f79f00ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:16.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.447+0000 7f79f4889640 1 --2- 172.21.15.32:0/34864010 >> v2:172.21.15.121:3300/0 conn(0x7f79f00ce430 0x7f79f00ce880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:16.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.447+0000 7f79f4889640 1 -- 172.21.15.32:0/34864010 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79f00b4dc0 con 0x7f79f015cb00 2024-04-24T23:11:16.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.447+0000 7f79f588b640 1 --2- 172.21.15.32:0/34864010 >> v2:172.21.15.121:3300/0 conn(0x7f79f00ce430 0x7f79f00ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:11:16.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.447+0000 7f79f508a640 1 --2- 172.21.15.32:0/34864010 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f79f015b380 0x7f79f006c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:11:16.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.448+0000 7f79f4889640 1 --2- 172.21.15.32:0/34864010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79f015cb00 0x7f79f006cb30 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f79f0070130 tx=0x7f79e002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea94331ba5019b34 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:16.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.448+0000 7f79e7fff640 1 -- 172.21.15.32:0/34864010 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e0037070 con 0x7f79f015cb00 2024-04-24T23:11:16.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.448+0000 7f79e7fff640 1 -- 172.21.15.32:0/34864010 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79e0032440 con 0x7f79f015cb00 2024-04-24T23:11:16.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.448+0000 7f79e7fff640 1 -- 172.21.15.32:0/34864010 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e0032810 con 0x7f79f015cb00 2024-04-24T23:11:16.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.449+0000 7f79f72ee640 1 -- 172.21.15.32:0/34864010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79f015cb00 msgr2=0x7f79f006cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:16.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.449+0000 7f79f72ee640 1 --2- 172.21.15.32:0/34864010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79f015cb00 0x7f79f006cb30 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f79f0070130 tx=0x7f79e002a140 comp rx=0 tx=0).stop 2024-04-24T23:11:16.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.449+0000 7f79f72ee640 1 -- 172.21.15.32:0/34864010 shutdown_connections 2024-04-24T23:11:16.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.449+0000 7f79f72ee640 1 --2- 172.21.15.32:0/34864010 >> v2:172.21.15.121:3300/0 conn(0x7f79f00ce430 0x7f79f00ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:16.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.449+0000 7f79f72ee640 1 --2- 172.21.15.32:0/34864010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79f015cb00 0x7f79f006cb30 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:16.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.449+0000 7f79f72ee640 1 --2- 172.21.15.32:0/34864010 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f79f015b380 0x7f79f006c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:16.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.449+0000 7f79f72ee640 1 -- 172.21.15.32:0/34864010 >> 172.21.15.32:0/34864010 conn(0x7f79f006b110 msgr2=0x7f79f006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:16.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.449+0000 7f79f72ee640 1 -- 172.21.15.32:0/34864010 shutdown_connections 2024-04-24T23:11:16.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.449+0000 7f79f72ee640 1 -- 172.21.15.32:0/34864010 wait complete. 2024-04-24T23:11:16.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.450+0000 7f79f72ee640 1 Processor -- start 2024-04-24T23:11:16.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.450+0000 7f79f72ee640 1 -- start start 2024-04-24T23:11:16.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.451+0000 7f79f72ee640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79f00ce430 0x7f79f0142d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:16.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.451+0000 7f79f72ee640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79f015b380 0x7f79f0143790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:16.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.451+0000 7f79f72ee640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f79f015cb00 0x7f79f01564b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:16.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.451+0000 7f79f72ee640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f79f00cedc0 con 0x7f79f015b380 2024-04-24T23:11:16.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.451+0000 7f79f72ee640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f79f015b760 con 0x7f79f00ce430 2024-04-24T23:11:16.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.451+0000 7f79f72ee640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f79f00cef40 con 0x7f79f015cb00 2024-04-24T23:11:16.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.451+0000 7f79f4889640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79f015b380 0x7f79f0143790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:16.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.451+0000 7f79f4889640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79f015b380 0x7f79f0143790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49462/0 (socket says 172.21.15.32:49462) 2024-04-24T23:11:16.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.451+0000 7f79f4889640 1 -- 172.21.15.32:0/1521734756 learned_addr learned my addr 172.21.15.32:0/1521734756 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:16.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.451+0000 7f79f508a640 1 --2- 172.21.15.32:0/1521734756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79f00ce430 0x7f79f0142d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:16.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.451+0000 7f79f588b640 1 --2- 172.21.15.32:0/1521734756 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f79f015cb00 0x7f79f01564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:16.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.451+0000 7f79f4889640 1 -- 172.21.15.32:0/1521734756 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f79f015cb00 msgr2=0x7f79f01564b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:16.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.451+0000 7f79f4889640 1 --2- 172.21.15.32:0/1521734756 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f79f015cb00 0x7f79f01564b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:16.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.451+0000 7f79f4889640 1 -- 172.21.15.32:0/1521734756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79f00ce430 msgr2=0x7f79f0142d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:16.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.451+0000 7f79f4889640 1 --2- 172.21.15.32:0/1521734756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79f00ce430 0x7f79f0142d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:16.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.451+0000 7f79f4889640 1 -- 172.21.15.32:0/1521734756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79f0151af0 con 0x7f79f015b380 2024-04-24T23:11:16.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.451+0000 7f79f588b640 1 --2- 172.21.15.32:0/1521734756 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f79f015cb00 0x7f79f01564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:11:16.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.451+0000 7f79f4889640 1 --2- 172.21.15.32:0/1521734756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79f015b380 0x7f79f0143790 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f79e002a180 tx=0x7f79e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:16.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.451+0000 7f79e5ffb640 1 -- 172.21.15.32:0/1521734756 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e0037040 con 0x7f79f015b380 2024-04-24T23:11:16.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.451+0000 7f79e5ffb640 1 -- 172.21.15.32:0/1521734756 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79e0042070 con 0x7f79f015b380 2024-04-24T23:11:16.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.451+0000 7f79f72ee640 1 -- 172.21.15.32:0/1521734756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79f0151dd0 con 0x7f79f015b380 2024-04-24T23:11:16.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.451+0000 7f79e5ffb640 1 -- 172.21.15.32:0/1521734756 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e0030bb0 con 0x7f79f015b380 2024-04-24T23:11:16.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.452+0000 7f79f72ee640 1 -- 172.21.15.32:0/1521734756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79f0152390 con 0x7f79f015b380 2024-04-24T23:11:16.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.453+0000 7f79f72ee640 1 -- 172.21.15.32:0/1521734756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79f0068430 con 0x7f79f015b380 2024-04-24T23:11:16.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.453+0000 7f79e5ffb640 1 -- 172.21.15.32:0/1521734756 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f79e0047460 con 0x7f79f015b380 2024-04-24T23:11:16.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.453+0000 7f79e5ffb640 1 --2- 172.21.15.32:0/1521734756 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f79d406be30 0x7f79d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:16.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.453+0000 7f79e5ffb640 1 -- 172.21.15.32:0/1521734756 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(190..190 src has 1..190) ==== 5738+0+0 (secure 0 0 0) 0x7f79e00ab9e0 con 0x7f79f015b380 2024-04-24T23:11:16.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.454+0000 7f79f508a640 1 --2- 172.21.15.32:0/1521734756 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f79d406be30 0x7f79d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:16.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.454+0000 7f79f508a640 1 --2- 172.21.15.32:0/1521734756 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f79d406be30 0x7f79d406e2f0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f79ec00aae0 tx=0x7f79ec008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:16.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.458+0000 7f79e5ffb640 1 -- 172.21.15.32:0/1521734756 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f79e007a400 con 0x7f79f015b380 2024-04-24T23:11:16.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.678+0000 7f79f72ee640 1 -- 172.21.15.32:0/1521734756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f79f00afd60 con 0x7f79f015b380 2024-04-24T23:11:16.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.678+0000 7f79e5ffb640 1 -- 172.21.15.32:0/1521734756 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) ==== 94+0+5185 (secure 0 0 0) 0x7f79e0079d00 con 0x7f79f015b380 2024-04-24T23:11:16.680 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:11:16.681 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":80,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6941,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/416518156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":416518156},{"type":"v1","addr":"172.21.15.121:6837","nonce":416518156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7762,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1506667233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1506667233},{"type":"v1","addr":"172.21.15.121:6835","nonce":1506667233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":9063,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/2215079101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":2215079101},{"type":"v1","addr":"172.21.15.32:6834","nonce":2215079101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:10:36.353761+0000","modified":"2024-04-24T23:10:38.436407+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9066},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9066":{"gid":9066,"name":"a","rank":0,"incarnation":78,"state":"up:active","state_seq":4,"addr":"172.21.15.32:6835/901353412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":901353412},{"type":"v1","addr":"172.21.15.32:6835","nonce":901353412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9066,"qdb_cluster":[9066]},"id":11}]} 2024-04-24T23:11:16.681 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 80 2024-04-24T23:11:16.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.680+0000 7f79f72ee640 1 -- 172.21.15.32:0/1521734756 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f79d406be30 msgr2=0x7f79d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:16.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.680+0000 7f79f72ee640 1 --2- 172.21.15.32:0/1521734756 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f79d406be30 0x7f79d406e2f0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f79ec00aae0 tx=0x7f79ec008040 comp rx=0 tx=0).stop 2024-04-24T23:11:16.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.680+0000 7f79f72ee640 1 -- 172.21.15.32:0/1521734756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79f015b380 msgr2=0x7f79f0143790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:16.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.680+0000 7f79f72ee640 1 --2- 172.21.15.32:0/1521734756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79f015b380 0x7f79f0143790 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f79e002a180 tx=0x7f79e00046c0 comp rx=0 tx=0).stop 2024-04-24T23:11:16.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.680+0000 7f79f72ee640 1 -- 172.21.15.32:0/1521734756 shutdown_connections 2024-04-24T23:11:16.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.680+0000 7f79f72ee640 1 --2- 172.21.15.32:0/1521734756 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f79d406be30 0x7f79d406e2f0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:16.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.680+0000 7f79f72ee640 1 --2- 172.21.15.32:0/1521734756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79f015b380 0x7f79f0143790 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:16.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.680+0000 7f79f72ee640 1 --2- 172.21.15.32:0/1521734756 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f79f015cb00 0x7f79f01564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:16.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.680+0000 7f79f72ee640 1 --2- 172.21.15.32:0/1521734756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79f00ce430 0x7f79f0142d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:16.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.680+0000 7f79f72ee640 1 -- 172.21.15.32:0/1521734756 >> 172.21.15.32:0/1521734756 conn(0x7f79f006b110 msgr2=0x7f79f00b56a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:16.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.680+0000 7f79f72ee640 1 -- 172.21.15.32:0/1521734756 shutdown_connections 2024-04-24T23:11:16.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.680+0000 7f79f72ee640 1 -- 172.21.15.32:0/1521734756 wait complete. 2024-04-24T23:11:16.693 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:11:16.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.792+0000 7f75a2f16640 1 Processor -- start 2024-04-24T23:11:16.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.792+0000 7f75a2f16640 1 -- start start 2024-04-24T23:11:16.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.793+0000 7f75a2f16640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f759c15cc40 0x7f759c15d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:16.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.793+0000 7f75a2f16640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f759c0d05b0 0x7f759c15d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:16.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.793+0000 7f75a2f16640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f759c0cec30 0x7f759c0cf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:16.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.793+0000 7f75a2f16640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f759c06cca0 con 0x7f759c0cec30 2024-04-24T23:11:16.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.793+0000 7f75a2f16640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f759c0ce5f0 con 0x7f759c0d05b0 2024-04-24T23:11:16.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.793+0000 7f75a2f16640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f759c15daa0 con 0x7f759c15cc40 2024-04-24T23:11:16.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.793+0000 7f75a14b3640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f759c0cec30 0x7f759c0cf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:16.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.793+0000 7f75a14b3640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f759c0cec30 0x7f759c0cf0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49470/0 (socket says 172.21.15.32:49470) 2024-04-24T23:11:16.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.793+0000 7f75a14b3640 1 -- 172.21.15.32:0/2676805148 learned_addr learned my addr 172.21.15.32:0/2676805148 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:16.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.793+0000 7f75a0cb2640 1 --2- 172.21.15.32:0/2676805148 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f759c15cc40 0x7f759c15d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:16.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.793+0000 7f759bfff640 1 --2- 172.21.15.32:0/2676805148 >> v2:172.21.15.121:3300/0 conn(0x7f759c0d05b0 0x7f759c15d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:16.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.793+0000 7f75a14b3640 1 -- 172.21.15.32:0/2676805148 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f759c15cc40 msgr2=0x7f759c15d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:16.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.793+0000 7f75a14b3640 1 --2- 172.21.15.32:0/2676805148 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f759c15cc40 0x7f759c15d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:16.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.793+0000 7f75a14b3640 1 -- 172.21.15.32:0/2676805148 >> v2:172.21.15.121:3300/0 conn(0x7f759c0d05b0 msgr2=0x7f759c15d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:16.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.793+0000 7f75a14b3640 1 --2- 172.21.15.32:0/2676805148 >> v2:172.21.15.121:3300/0 conn(0x7f759c0d05b0 0x7f759c15d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:16.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.793+0000 7f75a14b3640 1 -- 172.21.15.32:0/2676805148 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f759c0b4ee0 con 0x7f759c0cec30 2024-04-24T23:11:16.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.794+0000 7f75a14b3640 1 --2- 172.21.15.32:0/2676805148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f759c0cec30 0x7f759c0cf0a0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f759c06cb70 tx=0x7f759402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d63d136eae9cd755 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:16.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.794+0000 7f759b7fe640 1 -- 172.21.15.32:0/2676805148 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7594036070 con 0x7f759c0cec30 2024-04-24T23:11:16.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.794+0000 7f759b7fe640 1 -- 172.21.15.32:0/2676805148 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7594031600 con 0x7f759c0cec30 2024-04-24T23:11:16.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.794+0000 7f759b7fe640 1 -- 172.21.15.32:0/2676805148 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75940319d0 con 0x7f759c0cec30 2024-04-24T23:11:16.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.794+0000 7f75a2f16640 1 -- 172.21.15.32:0/2676805148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f759c0cec30 msgr2=0x7f759c0cf0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:16.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.795+0000 7f75a2f16640 1 --2- 172.21.15.32:0/2676805148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f759c0cec30 0x7f759c0cf0a0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f759c06cb70 tx=0x7f759402a6a0 comp rx=0 tx=0).stop 2024-04-24T23:11:16.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.795+0000 7f75a2f16640 1 -- 172.21.15.32:0/2676805148 shutdown_connections 2024-04-24T23:11:16.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.795+0000 7f75a2f16640 1 --2- 172.21.15.32:0/2676805148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f759c0cec30 0x7f759c0cf0a0 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:16.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.795+0000 7f75a2f16640 1 --2- 172.21.15.32:0/2676805148 >> v2:172.21.15.121:3300/0 conn(0x7f759c0d05b0 0x7f759c15d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:16.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.795+0000 7f75a2f16640 1 --2- 172.21.15.32:0/2676805148 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f759c15cc40 0x7f759c15d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:16.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.795+0000 7f75a2f16640 1 -- 172.21.15.32:0/2676805148 >> 172.21.15.32:0/2676805148 conn(0x7f759c0d0ae0 msgr2=0x7f759c0d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:16.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.795+0000 7f75a2f16640 1 -- 172.21.15.32:0/2676805148 shutdown_connections 2024-04-24T23:11:16.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.795+0000 7f75a2f16640 1 -- 172.21.15.32:0/2676805148 wait complete. 2024-04-24T23:11:16.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.796+0000 7f75a2f16640 1 Processor -- start 2024-04-24T23:11:16.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.796+0000 7f75a2f16640 1 -- start start 2024-04-24T23:11:16.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.797+0000 7f75a2f16640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f759c0cec30 0x7f759c15bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:16.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.797+0000 7f75a2f16640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f759c0d05b0 0x7f759c1e56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:16.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.797+0000 7f75a2f16640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f759c15cc40 0x7f759c1e71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:16.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.797+0000 7f75a2f16640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f759c0b5780 con 0x7f759c15cc40 2024-04-24T23:11:16.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.797+0000 7f75a2f16640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f759c0b54a0 con 0x7f759c0d05b0 2024-04-24T23:11:16.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.797+0000 7f75a2f16640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f759c0b5a60 con 0x7f759c0cec30 2024-04-24T23:11:16.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.797+0000 7f75a14b3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f759c15cc40 0x7f759c1e71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:16.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.797+0000 7f75a14b3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f759c15cc40 0x7f759c1e71f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49482/0 (socket says 172.21.15.32:49482) 2024-04-24T23:11:16.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.797+0000 7f75a14b3640 1 -- 172.21.15.32:0/2616018310 learned_addr learned my addr 172.21.15.32:0/2616018310 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:16.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.797+0000 7f75a0cb2640 1 --2- 172.21.15.32:0/2616018310 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f759c0cec30 0x7f759c15bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:16.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.797+0000 7f759bfff640 1 --2- 172.21.15.32:0/2616018310 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f759c0d05b0 0x7f759c1e56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:16.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.797+0000 7f75a14b3640 1 -- 172.21.15.32:0/2616018310 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f759c0cec30 msgr2=0x7f759c15bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:16.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.797+0000 7f75a14b3640 1 --2- 172.21.15.32:0/2616018310 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f759c0cec30 0x7f759c15bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:16.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.797+0000 7f75a14b3640 1 -- 172.21.15.32:0/2616018310 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f759c0d05b0 msgr2=0x7f759c1e56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:16.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.797+0000 7f75a14b3640 1 --2- 172.21.15.32:0/2616018310 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f759c0d05b0 0x7f759c1e56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:16.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.797+0000 7f75a14b3640 1 -- 172.21.15.32:0/2616018310 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f759c15c4b0 con 0x7f759c15cc40 2024-04-24T23:11:16.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.797+0000 7f75a14b3640 1 --2- 172.21.15.32:0/2616018310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f759c15cc40 0x7f759c1e71f0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f7594009f40 tx=0x7f7594004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:16.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.797+0000 7f75997fa640 1 -- 172.21.15.32:0/2616018310 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7594036050 con 0x7f759c15cc40 2024-04-24T23:11:16.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.798+0000 7f75997fa640 1 -- 172.21.15.32:0/2616018310 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7594042070 con 0x7f759c15cc40 2024-04-24T23:11:16.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.798+0000 7f75a2f16640 1 -- 172.21.15.32:0/2616018310 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f759c1e5fc0 con 0x7f759c15cc40 2024-04-24T23:11:16.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.798+0000 7f75997fa640 1 -- 172.21.15.32:0/2616018310 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7594009510 con 0x7f759c15cc40 2024-04-24T23:11:16.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.798+0000 7f75a2f16640 1 -- 172.21.15.32:0/2616018310 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f759c1e6580 con 0x7f759c15cc40 2024-04-24T23:11:16.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.799+0000 7f75a2f16640 1 -- 172.21.15.32:0/2616018310 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f759c068430 con 0x7f759c15cc40 2024-04-24T23:11:16.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.799+0000 7f75997fa640 1 -- 172.21.15.32:0/2616018310 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7594007bf0 con 0x7f759c15cc40 2024-04-24T23:11:16.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.799+0000 7f75997fa640 1 --2- 172.21.15.32:0/2616018310 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f758006be30 0x7f758006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:16.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.800+0000 7f75997fa640 1 -- 172.21.15.32:0/2616018310 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(190..190 src has 1..190) ==== 5738+0+0 (secure 0 0 0) 0x7f75940ab9f0 con 0x7f759c15cc40 2024-04-24T23:11:16.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.800+0000 7f75a0cb2640 1 --2- 172.21.15.32:0/2616018310 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f758006be30 0x7f758006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:16.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.800+0000 7f75a0cb2640 1 --2- 172.21.15.32:0/2616018310 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f758006be30 0x7f758006e2f0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f759c1e6e50 tx=0x7f7590009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:16.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.804+0000 7f75997fa640 1 -- 172.21.15.32:0/2616018310 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7594079dc0 con 0x7f759c15cc40 2024-04-24T23:11:16.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.992+0000 7f75a2f16640 1 -- 172.21.15.32:0/2616018310 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f759c072f20 con 0x7f759c15cc40 2024-04-24T23:11:16.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.993+0000 7f75997fa640 1 -- 172.21.15.32:0/2616018310 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v191) ==== 74+0+16767 (secure 0 0 0) 0x7f759407e0c0 con 0x7f759c15cc40 2024-04-24T23:11:16.995 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:11:16.995 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":191,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:11:16.862965+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:10:34.649562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-24T23:10:35.339604+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"191","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"23.6","mappings":[{"from":5,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:11:16.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.995+0000 7f75a2f16640 1 -- 172.21.15.32:0/2616018310 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f758006be30 msgr2=0x7f758006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:16.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.995+0000 7f75a2f16640 1 --2- 172.21.15.32:0/2616018310 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f758006be30 0x7f758006e2f0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f759c1e6e50 tx=0x7f7590009190 comp rx=0 tx=0).stop 2024-04-24T23:11:16.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.995+0000 7f75a2f16640 1 -- 172.21.15.32:0/2616018310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f759c15cc40 msgr2=0x7f759c1e71f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:16.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.995+0000 7f75a2f16640 1 --2- 172.21.15.32:0/2616018310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f759c15cc40 0x7f759c1e71f0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f7594009f40 tx=0x7f7594004a40 comp rx=0 tx=0).stop 2024-04-24T23:11:16.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.995+0000 7f75a2f16640 1 -- 172.21.15.32:0/2616018310 shutdown_connections 2024-04-24T23:11:16.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.995+0000 7f75a2f16640 1 --2- 172.21.15.32:0/2616018310 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f758006be30 0x7f758006e2f0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:16.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.995+0000 7f75a2f16640 1 --2- 172.21.15.32:0/2616018310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f759c15cc40 0x7f759c1e71f0 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:16.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.995+0000 7f75a2f16640 1 --2- 172.21.15.32:0/2616018310 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f759c0d05b0 0x7f759c1e56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:16.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.995+0000 7f75a2f16640 1 --2- 172.21.15.32:0/2616018310 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f759c0cec30 0x7f759c15bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:16.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.995+0000 7f75a2f16640 1 -- 172.21.15.32:0/2616018310 >> 172.21.15.32:0/2616018310 conn(0x7f759c0d0ae0 msgr2=0x7f759c15f550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:16.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.995+0000 7f75a2f16640 1 -- 172.21.15.32:0/2616018310 shutdown_connections 2024-04-24T23:11:16.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:16.995+0000 7f75a2f16640 1 -- 172.21.15.32:0/2616018310 wait complete. 2024-04-24T23:11:17.008 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-24T23:11:17.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.108+0000 7f97afa9e640 1 Processor -- start 2024-04-24T23:11:17.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.109+0000 7f97afa9e640 1 -- start start 2024-04-24T23:11:17.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.109+0000 7f97afa9e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f97a806b820 0x7f97a80d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:17.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.109+0000 7f97afa9e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f97a80d4210 0x7f97a80b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:17.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.109+0000 7f97afa9e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97a80d3170 con 0x7f97a815e9b0 2024-04-24T23:11:17.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.109+0000 7f97afa9e640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f97a80d3450 con 0x7f97a80d4210 2024-04-24T23:11:17.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.109+0000 7f97afa9e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f97a80d3730 con 0x7f97a806b820 2024-04-24T23:11:17.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.110+0000 7f97ae03b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f97a80d4210 0x7f97a80b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:17.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.110+0000 7f97ad83a640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f97a815e9b0 0x7f97a806b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35986/0 (socket says 172.21.15.32:35986) 2024-04-24T23:11:17.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.110+0000 7f97ae03b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f97a80d4210 0x7f97a80b78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49488/0 (socket says 172.21.15.32:49488) 2024-04-24T23:11:17.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.110+0000 7f97ad83a640 1 -- 172.21.15.32:0/3957230868 learned_addr learned my addr 172.21.15.32:0/3957230868 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:11:17.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.110+0000 7f97ae03b640 1 -- 172.21.15.32:0/3957230868 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f97a806b820 msgr2=0x7f97a80d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:17.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.110+0000 7f97ae03b640 1 --2- 172.21.15.32:0/3957230868 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f97a806b820 0x7f97a80d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:17.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.110+0000 7f97ae03b640 1 -- 172.21.15.32:0/3957230868 >> v1:172.21.15.32:6789/0 conn(0x7f97a815e9b0 legacy=0x7f97a806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:17.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.110+0000 7f97ae03b640 1 -- 172.21.15.32:0/3957230868 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97a80d3a10 con 0x7f97a80d4210 2024-04-24T23:11:17.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.110+0000 7f97ac838640 1 -- 172.21.15.32:0/3957230868 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 509786787 0 0) 0x7f97a80d3170 con 0x7f97a815e9b0 2024-04-24T23:11:17.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.111+0000 7f97ae03b640 1 --2- 172.21.15.32:0/3957230868 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97a80d4210 0x7f97a80b78f0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f97a0009d90 tx=0x7f97a002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a34dfeb81de88b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:17.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.111+0000 7f97ac838640 1 -- 172.21.15.32:0/3957230868 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a0007740 con 0x7f97a80d4210 2024-04-24T23:11:17.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.111+0000 7f97ac838640 1 -- 172.21.15.32:0/3957230868 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97a00078e0 con 0x7f97a80d4210 2024-04-24T23:11:17.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.111+0000 7f97ac838640 1 -- 172.21.15.32:0/3957230868 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a0007cb0 con 0x7f97a80d4210 2024-04-24T23:11:17.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.111+0000 7f97afa9e640 1 -- 172.21.15.32:0/3957230868 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97a80d4210 msgr2=0x7f97a80b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:17.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.111+0000 7f97afa9e640 1 --2- 172.21.15.32:0/3957230868 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97a80d4210 0x7f97a80b78f0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f97a0009d90 tx=0x7f97a002a660 comp rx=0 tx=0).stop 2024-04-24T23:11:17.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.111+0000 7f97afa9e640 1 -- 172.21.15.32:0/3957230868 shutdown_connections 2024-04-24T23:11:17.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.111+0000 7f97afa9e640 1 --2- 172.21.15.32:0/3957230868 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97a80d4210 0x7f97a80b78f0 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:17.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.111+0000 7f97afa9e640 1 --2- 172.21.15.32:0/3957230868 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f97a806b820 0x7f97a80d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:17.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.111+0000 7f97afa9e640 1 -- 172.21.15.32:0/3957230868 >> 172.21.15.32:0/3957230868 conn(0x7f97a80d11d0 msgr2=0x7f97a80d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:17.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.111+0000 7f97afa9e640 1 -- 172.21.15.32:0/3957230868 shutdown_connections 2024-04-24T23:11:17.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.111+0000 7f97afa9e640 1 -- 172.21.15.32:0/3957230868 wait complete. 2024-04-24T23:11:17.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.112+0000 7f97afa9e640 1 Processor -- start 2024-04-24T23:11:17.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.113+0000 7f97afa9e640 1 -- start start 2024-04-24T23:11:17.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.113+0000 7f97afa9e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f97a806b820 0x7f97a81564b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:17.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.113+0000 7f97afa9e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f97a80d4210 0x7f97a815c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:17.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.113+0000 7f97afa9e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97a815e9b0 0x7f97a81f7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:17.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.113+0000 7f97afa9e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f97a80ba0e0 con 0x7f97a815e9b0 2024-04-24T23:11:17.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.113+0000 7f97afa9e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f97a80d2400 con 0x7f97a806b820 2024-04-24T23:11:17.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.113+0000 7f97afa9e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f97a80ba260 con 0x7f97a80d4210 2024-04-24T23:11:17.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.113+0000 7f97ae03b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97a815e9b0 0x7f97a81f7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:17.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.113+0000 7f97ae03b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97a815e9b0 0x7f97a81f7800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49490/0 (socket says 172.21.15.32:49490) 2024-04-24T23:11:17.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.113+0000 7f97ae03b640 1 -- 172.21.15.32:0/938013337 learned_addr learned my addr 172.21.15.32:0/938013337 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:17.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.114+0000 7f97ad83a640 1 --2- 172.21.15.32:0/938013337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f97a806b820 0x7f97a81564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:17.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.114+0000 7f97ae03b640 1 -- 172.21.15.32:0/938013337 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f97a80d4210 msgr2=0x7f97a815c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:17.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.114+0000 7f97ae03b640 1 --2- 172.21.15.32:0/938013337 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f97a80d4210 0x7f97a815c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:17.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.114+0000 7f97ae03b640 1 -- 172.21.15.32:0/938013337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f97a806b820 msgr2=0x7f97a81564b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:17.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.114+0000 7f97ae03b640 1 --2- 172.21.15.32:0/938013337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f97a806b820 0x7f97a81564b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:17.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.114+0000 7f97ae03b640 1 -- 172.21.15.32:0/938013337 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97a81569f0 con 0x7f97a815e9b0 2024-04-24T23:11:17.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.114+0000 7f97ae03b640 1 --2- 172.21.15.32:0/938013337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97a815e9b0 0x7f97a81f7800 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f97a0009d90 tx=0x7f97a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:17.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.114+0000 7f979e7fc640 1 -- 172.21.15.32:0/938013337 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a0040070 con 0x7f97a815e9b0 2024-04-24T23:11:17.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.114+0000 7f979e7fc640 1 -- 172.21.15.32:0/938013337 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97a0031c40 con 0x7f97a815e9b0 2024-04-24T23:11:17.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.114+0000 7f97afa9e640 1 -- 172.21.15.32:0/938013337 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97a815cb90 con 0x7f97a815e9b0 2024-04-24T23:11:17.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.114+0000 7f979e7fc640 1 -- 172.21.15.32:0/938013337 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a0003990 con 0x7f97a815e9b0 2024-04-24T23:11:17.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.114+0000 7f97afa9e640 1 -- 172.21.15.32:0/938013337 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97a80cd8b0 con 0x7f97a815e9b0 2024-04-24T23:11:17.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.115+0000 7f97afa9e640 1 -- 172.21.15.32:0/938013337 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97a8068430 con 0x7f97a815e9b0 2024-04-24T23:11:17.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.116+0000 7f979e7fc640 1 -- 172.21.15.32:0/938013337 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f97a0039040 con 0x7f97a815e9b0 2024-04-24T23:11:17.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.116+0000 7f979e7fc640 1 --2- 172.21.15.32:0/938013337 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f978c06be60 0x7f978c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:17.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.116+0000 7f979e7fc640 1 -- 172.21.15.32:0/938013337 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(191..191 src has 1..191) ==== 5738+0+0 (secure 0 0 0) 0x7f97a00ac1b0 con 0x7f97a815e9b0 2024-04-24T23:11:17.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.116+0000 7f979e7fc640 1 -- 172.21.15.32:0/938013337 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=192}) -- 0x7f978c073a50 con 0x7f97a815e9b0 2024-04-24T23:11:17.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.116+0000 7f97ad83a640 1 --2- 172.21.15.32:0/938013337 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f978c06be60 0x7f978c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:17.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.117+0000 7f97ad83a640 1 --2- 172.21.15.32:0/938013337 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f978c06be60 0x7f978c06e320 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f97a4000f60 tx=0x7f97a4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:17.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.121+0000 7f979e7fc640 1 -- 172.21.15.32:0/938013337 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f97a007ab40 con 0x7f97a815e9b0 2024-04-24T23:11:17.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.343+0000 7f97afa9e640 1 -- 172.21.15.32:0/938013337 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f97a815da80 con 0x7f97a815e9b0 2024-04-24T23:11:17.345 INFO:tasks.ceph.mon.a.smithi032.stderr:2024-04-24T23:11:17.343+0000 7f3b809c6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T23:11:17.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.347+0000 7f979e7fc640 1 -- 172.21.15.32:0/938013337 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v81) ==== 161+0+0 (secure 0 0 0) 0x7f97a007a440 con 0x7f97a815e9b0 2024-04-24T23:11:17.349 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-24T23:11:17.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.348+0000 7f979e7fc640 1 -- 172.21.15.32:0/938013337 <== mon.0 v2:172.21.15.32:3300/0 8 ==== osd_map(192..192 src has 1..192) ==== 374+0+0 (secure 0 0 0) 0x7f97a00e02d0 con 0x7f97a815e9b0 2024-04-24T23:11:17.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.348+0000 7f979e7fc640 1 -- 172.21.15.32:0/938013337 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=193}) -- 0x7f978c074350 con 0x7f97a815e9b0 2024-04-24T23:11:17.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.350+0000 7f97afa9e640 1 -- 172.21.15.32:0/938013337 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f978c06be60 msgr2=0x7f978c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:17.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.350+0000 7f97afa9e640 1 --2- 172.21.15.32:0/938013337 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f978c06be60 0x7f978c06e320 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f97a4000f60 tx=0x7f97a4009040 comp rx=0 tx=0).stop 2024-04-24T23:11:17.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.351+0000 7f97afa9e640 1 -- 172.21.15.32:0/938013337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97a815e9b0 msgr2=0x7f97a81f7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:17.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.351+0000 7f97afa9e640 1 --2- 172.21.15.32:0/938013337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97a815e9b0 0x7f97a81f7800 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f97a0009d90 tx=0x7f97a0004a40 comp rx=0 tx=0).stop 2024-04-24T23:11:17.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.351+0000 7f97afa9e640 1 -- 172.21.15.32:0/938013337 shutdown_connections 2024-04-24T23:11:17.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.351+0000 7f97afa9e640 1 --2- 172.21.15.32:0/938013337 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f978c06be60 0x7f978c06e320 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:17.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.351+0000 7f97afa9e640 1 --2- 172.21.15.32:0/938013337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97a815e9b0 0x7f97a81f7800 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:17.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.351+0000 7f97afa9e640 1 --2- 172.21.15.32:0/938013337 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f97a80d4210 0x7f97a815c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:17.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.351+0000 7f97afa9e640 1 --2- 172.21.15.32:0/938013337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f97a806b820 0x7f97a81564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:17.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.351+0000 7f97afa9e640 1 -- 172.21.15.32:0/938013337 >> 172.21.15.32:0/938013337 conn(0x7f97a80d11d0 msgr2=0x7f97a8072140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:17.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.351+0000 7f97afa9e640 1 -- 172.21.15.32:0/938013337 shutdown_connections 2024-04-24T23:11:17.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.351+0000 7f97afa9e640 1 -- 172.21.15.32:0/938013337 wait complete. 2024-04-24T23:11:17.355 INFO:tasks.ceph.mds.a.smithi032.stderr: -9664> 2024-04-24T23:10:41.959+0000 7f51253bb640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T23:11:17.365 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-24T23:11:17.389 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2024-04-24T23:11:17.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.467+0000 7f1fa1e10640 1 Processor -- start 2024-04-24T23:11:17.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.468+0000 7f1fa1e10640 1 -- start start 2024-04-24T23:11:17.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.468+0000 7f1fa1e10640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1f9c06b820 0x7f1f9c0d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:17.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.468+0000 7f1fa1e10640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f9c0d3170 con 0x7f1f9c0d4210 2024-04-24T23:11:17.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.468+0000 7f1fa1e10640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f9c0d3450 con 0x7f1f9c15e9b0 2024-04-24T23:11:17.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.468+0000 7f1fa1e10640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f1f9c0d3730 con 0x7f1f9c06b820 2024-04-24T23:11:17.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.468+0000 7f1f9bfff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1f9c0d4210 0x7f1f9c0b78f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36012/0 (socket says 172.21.15.32:36012) 2024-04-24T23:11:17.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.468+0000 7f1f9bfff640 1 -- 172.21.15.32:0/2732616769 learned_addr learned my addr 172.21.15.32:0/2732616769 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:11:17.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.469+0000 7f1f9affd640 1 --2- 172.21.15.32:0/2732616769 >> v2:172.21.15.121:3300/0 conn(0x7f1f9c06b820 0x7f1f9c0d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:17.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.469+0000 7f1f9a7fc640 1 -- 172.21.15.32:0/2732616769 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1483319889 0 0) 0x7f1f9c0d3170 con 0x7f1f9c0d4210 2024-04-24T23:11:17.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.469+0000 7f1f9a7fc640 1 -- 172.21.15.32:0/2732616769 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f84003520 con 0x7f1f9c0d4210 2024-04-24T23:11:17.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.469+0000 7f1f9a7fc640 1 -- 172.21.15.32:0/2732616769 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2072966543 0 0) 0x7f1f84003520 con 0x7f1f9c0d4210 2024-04-24T23:11:17.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.469+0000 7f1f9a7fc640 1 -- 172.21.15.32:0/2732616769 >> v2:172.21.15.121:3300/0 conn(0x7f1f9c06b820 msgr2=0x7f1f9c0d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:17.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.469+0000 7f1f9a7fc640 1 --2- 172.21.15.32:0/2732616769 >> v2:172.21.15.121:3300/0 conn(0x7f1f9c06b820 0x7f1f9c0d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:17.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.469+0000 7f1f9a7fc640 1 -- 172.21.15.32:0/2732616769 >> v1:172.21.15.121:6789/0 conn(0x7f1f9c15e9b0 legacy=0x7f1f9c06b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:17.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.469+0000 7f1f9a7fc640 1 -- 172.21.15.32:0/2732616769 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f9c0d3a10 con 0x7f1f9c0d4210 2024-04-24T23:11:17.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.469+0000 7f1f9a7fc640 1 -- 172.21.15.32:0/2732616769 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f1f940019f0 con 0x7f1f9c0d4210 2024-04-24T23:11:17.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.469+0000 7f1f9a7fc640 1 -- 172.21.15.32:0/2732616769 >> v1:172.21.15.32:6789/0 conn(0x7f1f9c0d4210 legacy=0x7f1f9c0b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:17.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.469+0000 7f1f9a7fc640 1 --2- 172.21.15.32:0/2732616769 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1f840039b0 0x7f1f84003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:17.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.469+0000 7f1f9a7fc640 1 --2- 172.21.15.32:0/2732616769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f840043b0 0x7f1f84004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:17.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.469+0000 7f1f9a7fc640 1 --2- 172.21.15.32:0/2732616769 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f84005120 0x7f1f8400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:17.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.469+0000 7f1f9a7fc640 1 -- 172.21.15.32:0/2732616769 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c0d3730 con 0x7f1f840043b0 2024-04-24T23:11:17.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.469+0000 7f1f9a7fc640 1 -- 172.21.15.32:0/2732616769 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8400c060 con 0x7f1f84005120 2024-04-24T23:11:17.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.469+0000 7f1f9a7fc640 1 -- 172.21.15.32:0/2732616769 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f1f8400c260 con 0x7f1f840039b0 2024-04-24T23:11:17.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.469+0000 7f1f9affd640 1 --2- 172.21.15.32:0/2732616769 >> v2:172.21.15.121:3300/0 conn(0x7f1f9c06b820 0x7f1f9c0d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:11:17.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.469+0000 7f1f9affd640 1 --2- 172.21.15.32:0/2732616769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f840043b0 0x7f1f84004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:17.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.469+0000 7f1f9b7fe640 1 --2- 172.21.15.32:0/2732616769 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1f840039b0 0x7f1f84003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:17.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.469+0000 7f1f9bfff640 1 --2- 172.21.15.32:0/2732616769 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f84005120 0x7f1f8400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:17.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.470+0000 7f1f9affd640 1 -- 172.21.15.32:0/2732616769 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1f840039b0 msgr2=0x7f1f84003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:17.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.470+0000 7f1f9affd640 1 --2- 172.21.15.32:0/2732616769 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1f840039b0 0x7f1f84003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:17.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.470+0000 7f1f9affd640 1 -- 172.21.15.32:0/2732616769 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f84005120 msgr2=0x7f1f8400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:17.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.470+0000 7f1f9affd640 1 --2- 172.21.15.32:0/2732616769 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f84005120 0x7f1f8400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:17.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.470+0000 7f1f9affd640 1 -- 172.21.15.32:0/2732616769 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f84003520 con 0x7f1f840043b0 2024-04-24T23:11:17.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.470+0000 7f1f9bfff640 1 -- 172.21.15.32:0/2732616769 reap_dead start 2024-04-24T23:11:17.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.470+0000 7f1f9affd640 1 --2- 172.21.15.32:0/2732616769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f840043b0 0x7f1f84004880 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f1f90009c70 tx=0x7f1f84005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fca2692e22e2ad1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:17.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.470+0000 7f1f9a7fc640 1 -- 172.21.15.32:0/2732616769 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f90009840 con 0x7f1f840043b0 2024-04-24T23:11:17.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.470+0000 7f1f9a7fc640 1 -- 172.21.15.32:0/2732616769 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f90004020 con 0x7f1f840043b0 2024-04-24T23:11:17.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.470+0000 7f1f9a7fc640 1 -- 172.21.15.32:0/2732616769 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f900043d0 con 0x7f1f840043b0 2024-04-24T23:11:17.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.471+0000 7f1fa1e10640 1 -- 172.21.15.32:0/2732616769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f840043b0 msgr2=0x7f1f84004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:17.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.471+0000 7f1fa1e10640 1 --2- 172.21.15.32:0/2732616769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f840043b0 0x7f1f84004880 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f1f90009c70 tx=0x7f1f84005830 comp rx=0 tx=0).stop 2024-04-24T23:11:17.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.471+0000 7f1fa1e10640 1 -- 172.21.15.32:0/2732616769 shutdown_connections 2024-04-24T23:11:17.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.471+0000 7f1fa1e10640 1 --2- 172.21.15.32:0/2732616769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f840043b0 0x7f1f84004880 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:17.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.471+0000 7f1fa1e10640 1 -- 172.21.15.32:0/2732616769 >> 172.21.15.32:0/2732616769 conn(0x7f1f9c0d11d0 msgr2=0x7f1f9c0d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:17.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.471+0000 7f1fa1e10640 1 -- 172.21.15.32:0/2732616769 shutdown_connections 2024-04-24T23:11:17.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.471+0000 7f1fa1e10640 1 -- 172.21.15.32:0/2732616769 wait complete. 2024-04-24T23:11:17.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.472+0000 7f1fa1e10640 1 Processor -- start 2024-04-24T23:11:17.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.472+0000 7f1fa1e10640 1 -- start start 2024-04-24T23:11:17.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.473+0000 7f1fa1e10640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1f840043b0 0x7f1f9c157e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:17.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.473+0000 7f1fa1e10640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f9c157210 0x7f1f9c1f3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:17.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.473+0000 7f1fa1e10640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f9c15ac90 0x7f1f9c0cd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:17.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.473+0000 7f1fa1e10640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c0ba0e0 con 0x7f1f9c15ac90 2024-04-24T23:11:17.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.473+0000 7f1fa1e10640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c0d2400 con 0x7f1f9c157210 2024-04-24T23:11:17.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.473+0000 7f1fa1e10640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f1f9c0ba260 con 0x7f1f840043b0 2024-04-24T23:11:17.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.473+0000 7f1f9bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f9c15ac90 0x7f1f9c0cd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:17.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.473+0000 7f1f9bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f9c15ac90 0x7f1f9c0cd9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49510/0 (socket says 172.21.15.32:49510) 2024-04-24T23:11:17.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.473+0000 7f1f9bfff640 1 -- 172.21.15.32:0/1807571031 learned_addr learned my addr 172.21.15.32:0/1807571031 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:17.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.473+0000 7f1f9b7fe640 1 --2- 172.21.15.32:0/1807571031 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1f840043b0 0x7f1f9c157e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:17.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.473+0000 7f1f9affd640 1 --2- 172.21.15.32:0/1807571031 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f9c157210 0x7f1f9c1f3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:17.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.474+0000 7f1f9bfff640 1 -- 172.21.15.32:0/1807571031 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1f840043b0 msgr2=0x7f1f9c157e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:17.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.474+0000 7f1f9bfff640 1 --2- 172.21.15.32:0/1807571031 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1f840043b0 0x7f1f9c157e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:17.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.474+0000 7f1f9bfff640 1 -- 172.21.15.32:0/1807571031 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f9c157210 msgr2=0x7f1f9c1f3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:17.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.474+0000 7f1f9bfff640 1 --2- 172.21.15.32:0/1807571031 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f9c157210 0x7f1f9c1f3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:17.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.474+0000 7f1f9bfff640 1 -- 172.21.15.32:0/1807571031 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f9c1481a0 con 0x7f1f9c15ac90 2024-04-24T23:11:17.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.474+0000 7f1f9b7fe640 1 --2- 172.21.15.32:0/1807571031 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1f840043b0 0x7f1f9c157e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:11:17.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.474+0000 7f1f9affd640 1 --2- 172.21.15.32:0/1807571031 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f9c157210 0x7f1f9c1f3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:11:17.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.474+0000 7f1f9bfff640 1 --2- 172.21.15.32:0/1807571031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f9c15ac90 0x7f1f9c0cd9f0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c06bdd0 tx=0x7f1f940079d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:17.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.474+0000 7f1f83fff640 1 -- 172.21.15.32:0/1807571031 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f9400aa40 con 0x7f1f9c15ac90 2024-04-24T23:11:17.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.474+0000 7f1f83fff640 1 -- 172.21.15.32:0/1807571031 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f94018070 con 0x7f1f9c15ac90 2024-04-24T23:11:17.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.474+0000 7f1fa1e10640 1 -- 172.21.15.32:0/1807571031 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f9c148440 con 0x7f1f9c15ac90 2024-04-24T23:11:17.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.474+0000 7f1f83fff640 1 -- 172.21.15.32:0/1807571031 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f9400ae00 con 0x7f1f9c15ac90 2024-04-24T23:11:17.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.474+0000 7f1fa1e10640 1 -- 172.21.15.32:0/1807571031 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f9c148a00 con 0x7f1f9c15ac90 2024-04-24T23:11:17.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.475+0000 7f1fa1e10640 1 -- 172.21.15.32:0/1807571031 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f9c068430 con 0x7f1f9c15ac90 2024-04-24T23:11:17.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.475+0000 7f1f83fff640 1 -- 172.21.15.32:0/1807571031 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1f9401d460 con 0x7f1f9c15ac90 2024-04-24T23:11:17.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.476+0000 7f1f83fff640 1 --2- 172.21.15.32:0/1807571031 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1f8806be60 0x7f1f8806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:17.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.476+0000 7f1f83fff640 1 -- 172.21.15.32:0/1807571031 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(192..192 src has 1..192) ==== 5824+0+0 (secure 0 0 0) 0x7f1f9408b360 con 0x7f1f9c15ac90 2024-04-24T23:11:17.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.476+0000 7f1f83fff640 1 -- 172.21.15.32:0/1807571031 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=193}) -- 0x7f1f88073af0 con 0x7f1f9c15ac90 2024-04-24T23:11:17.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.476+0000 7f1f9b7fe640 1 --2- 172.21.15.32:0/1807571031 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1f8806be60 0x7f1f8806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:17.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.476+0000 7f1f9b7fe640 1 --2- 172.21.15.32:0/1807571031 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1f8806be60 0x7f1f8806e320 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c003700 tx=0x7f1f8c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:17.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.480+0000 7f1f83fff640 1 -- 172.21.15.32:0/1807571031 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1f94059ca0 con 0x7f1f9c15ac90 2024-04-24T23:11:17.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:17.701+0000 7f1fa1e10640 1 -- 172.21.15.32:0/1807571031 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f1f9c0d2790 con 0x7f1f9c15ac90 2024-04-24T23:11:18.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.349+0000 7f1f83fff640 1 -- 172.21.15.32:0/1807571031 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v82) ==== 104+0+0 (secure 0 0 0) 0x7f1f940595a0 con 0x7f1f9c15ac90 2024-04-24T23:11:18.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.352+0000 7f1fa1e10640 1 -- 172.21.15.32:0/1807571031 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1f8806be60 msgr2=0x7f1f8806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:18.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.352+0000 7f1fa1e10640 1 --2- 172.21.15.32:0/1807571031 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1f8806be60 0x7f1f8806e320 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c003700 tx=0x7f1f8c009040 comp rx=0 tx=0).stop 2024-04-24T23:11:18.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.352+0000 7f1fa1e10640 1 -- 172.21.15.32:0/1807571031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f9c15ac90 msgr2=0x7f1f9c0cd9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:18.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.352+0000 7f1fa1e10640 1 --2- 172.21.15.32:0/1807571031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f9c15ac90 0x7f1f9c0cd9f0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c06bdd0 tx=0x7f1f940079d0 comp rx=0 tx=0).stop 2024-04-24T23:11:18.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.352+0000 7f1fa1e10640 1 -- 172.21.15.32:0/1807571031 shutdown_connections 2024-04-24T23:11:18.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.352+0000 7f1fa1e10640 1 --2- 172.21.15.32:0/1807571031 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1f8806be60 0x7f1f8806e320 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:18.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.352+0000 7f1fa1e10640 1 --2- 172.21.15.32:0/1807571031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f9c15ac90 0x7f1f9c0cd9f0 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:18.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.352+0000 7f1fa1e10640 1 --2- 172.21.15.32:0/1807571031 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f9c157210 0x7f1f9c1f3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:18.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.352+0000 7f1fa1e10640 1 --2- 172.21.15.32:0/1807571031 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1f840043b0 0x7f1f9c157e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:18.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.352+0000 7f1fa1e10640 1 -- 172.21.15.32:0/1807571031 >> 172.21.15.32:0/1807571031 conn(0x7f1f9c0d11d0 msgr2=0x7f1f9c15cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:18.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.352+0000 7f1fa1e10640 1 -- 172.21.15.32:0/1807571031 shutdown_connections 2024-04-24T23:11:18.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.352+0000 7f1fa1e10640 1 -- 172.21.15.32:0/1807571031 wait complete. 2024-04-24T23:11:18.368 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-24T23:11:18.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.521+0000 7ffa3bf42640 1 Processor -- start 2024-04-24T23:11:18.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.521+0000 7ffa3bf42640 1 -- start start 2024-04-24T23:11:18.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.522+0000 7ffa3bf42640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa340b5ff0 0x7ffa340cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:18.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.522+0000 7ffa3bf42640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa3415bde0 con 0x7ffa3415b960 2024-04-24T23:11:18.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.522+0000 7ffa3bf42640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa3415c0c0 con 0x7ffa3415b580 2024-04-24T23:11:18.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.522+0000 7ffa3bf42640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ffa3415c9f0 con 0x7ffa340b5ff0 2024-04-24T23:11:18.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.522+0000 7ffa394dd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ffa3415b960 0x7ffa340b58e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36024/0 (socket says 172.21.15.32:36024) 2024-04-24T23:11:18.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.522+0000 7ffa394dd640 1 -- 172.21.15.32:0/792446715 learned_addr learned my addr 172.21.15.32:0/792446715 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:11:18.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.522+0000 7ffa3a4df640 1 --2- 172.21.15.32:0/792446715 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa340b5ff0 0x7ffa340cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:18.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.522+0000 7ffa38cdc640 1 -- 172.21.15.32:0/792446715 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2035311305 0 0) 0x7ffa3415bde0 con 0x7ffa3415b960 2024-04-24T23:11:18.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.522+0000 7ffa38cdc640 1 -- 172.21.15.32:0/792446715 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa20003520 con 0x7ffa3415b960 2024-04-24T23:11:18.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.522+0000 7ffa38cdc640 1 -- 172.21.15.32:0/792446715 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 963737232 0 0) 0x7ffa20003520 con 0x7ffa3415b960 2024-04-24T23:11:18.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.522+0000 7ffa38cdc640 1 -- 172.21.15.32:0/792446715 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa340b5ff0 msgr2=0x7ffa340cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:18.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.522+0000 7ffa38cdc640 1 --2- 172.21.15.32:0/792446715 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa340b5ff0 0x7ffa340cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:18.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.522+0000 7ffa38cdc640 1 -- 172.21.15.32:0/792446715 >> v1:172.21.15.121:6789/0 conn(0x7ffa3415b580 legacy=0x7ffa3406cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:18.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.522+0000 7ffa38cdc640 1 -- 172.21.15.32:0/792446715 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa3415c3a0 con 0x7ffa3415b960 2024-04-24T23:11:18.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.522+0000 7ffa38cdc640 1 -- 172.21.15.32:0/792446715 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7ffa24001670 con 0x7ffa3415b960 2024-04-24T23:11:18.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.522+0000 7ffa38cdc640 1 -- 172.21.15.32:0/792446715 >> v1:172.21.15.32:6789/0 conn(0x7ffa3415b960 legacy=0x7ffa340b58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:18.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.523+0000 7ffa38cdc640 1 --2- 172.21.15.32:0/792446715 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa340b5ff0 0x7ffa200039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:18.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.523+0000 7ffa38cdc640 1 --2- 172.21.15.32:0/792446715 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa20003f20 0x7ffa20004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:18.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.523+0000 7ffa38cdc640 1 --2- 172.21.15.32:0/792446715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa20004d40 0x7ffa200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:18.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.523+0000 7ffa38cdc640 1 -- 172.21.15.32:0/792446715 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3415c9f0 con 0x7ffa20003f20 2024-04-24T23:11:18.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.523+0000 7ffa38cdc640 1 -- 172.21.15.32:0/792446715 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2000ba20 con 0x7ffa20004d40 2024-04-24T23:11:18.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.523+0000 7ffa38cdc640 1 -- 172.21.15.32:0/792446715 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ffa2000de30 con 0x7ffa340b5ff0 2024-04-24T23:11:18.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.523+0000 7ffa394dd640 1 --2- 172.21.15.32:0/792446715 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa20003f20 0x7ffa20004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:18.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.523+0000 7ffa39cde640 1 --2- 172.21.15.32:0/792446715 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa340b5ff0 0x7ffa200039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:18.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.523+0000 7ffa3a4df640 1 --2- 172.21.15.32:0/792446715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa20004d40 0x7ffa200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:18.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.523+0000 7ffa394dd640 1 -- 172.21.15.32:0/792446715 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa340b5ff0 msgr2=0x7ffa200039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:18.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.523+0000 7ffa394dd640 1 --2- 172.21.15.32:0/792446715 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa340b5ff0 0x7ffa200039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:18.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.523+0000 7ffa394dd640 1 -- 172.21.15.32:0/792446715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa20004d40 msgr2=0x7ffa200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:18.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.523+0000 7ffa394dd640 1 --2- 172.21.15.32:0/792446715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa20004d40 0x7ffa200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:18.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.523+0000 7ffa394dd640 1 -- 172.21.15.32:0/792446715 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa20003520 con 0x7ffa20003f20 2024-04-24T23:11:18.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.524+0000 7ffa394dd640 1 --2- 172.21.15.32:0/792446715 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa20003f20 0x7ffa20004800 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7ffa2400c210 tx=0x7ffa20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea09de04b37a3d00 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:18.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.524+0000 7ffa38cdc640 1 -- 172.21.15.32:0/792446715 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa24007d10 con 0x7ffa20003f20 2024-04-24T23:11:18.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.524+0000 7ffa38cdc640 1 -- 172.21.15.32:0/792446715 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa24005ad0 con 0x7ffa20003f20 2024-04-24T23:11:18.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.524+0000 7ffa38cdc640 1 -- 172.21.15.32:0/792446715 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa24007070 con 0x7ffa20003f20 2024-04-24T23:11:18.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.524+0000 7ffa3bf42640 1 -- 172.21.15.32:0/792446715 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa20003f20 msgr2=0x7ffa20004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:18.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.524+0000 7ffa3bf42640 1 --2- 172.21.15.32:0/792446715 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa20003f20 0x7ffa20004800 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7ffa2400c210 tx=0x7ffa20005830 comp rx=0 tx=0).stop 2024-04-24T23:11:18.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.524+0000 7ffa3a4df640 1 -- 172.21.15.32:0/792446715 reap_dead start 2024-04-24T23:11:18.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.524+0000 7ffa3bf42640 1 -- 172.21.15.32:0/792446715 shutdown_connections 2024-04-24T23:11:18.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.524+0000 7ffa3bf42640 1 -- 172.21.15.32:0/792446715 >> 172.21.15.32:0/792446715 conn(0x7ffa340b1f10 msgr2=0x7ffa340b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:18.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.525+0000 7ffa3bf42640 1 -- 172.21.15.32:0/792446715 shutdown_connections 2024-04-24T23:11:18.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.525+0000 7ffa3bf42640 1 -- 172.21.15.32:0/792446715 wait complete. 2024-04-24T23:11:18.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.526+0000 7ffa3bf42640 1 Processor -- start 2024-04-24T23:11:18.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.526+0000 7ffa3bf42640 1 -- start start 2024-04-24T23:11:18.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.526+0000 7ffa3bf42640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa20003f20 0x7ffa341f8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:18.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.526+0000 7ffa3bf42640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa34072220 0x7ffa3414e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:18.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.526+0000 7ffa3bf42640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa340ae8b0 0x7ffa340aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:18.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.526+0000 7ffa3bf42640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3406c920 con 0x7ffa20003f20 2024-04-24T23:11:18.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.526+0000 7ffa3bf42640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3406c7a0 con 0x7ffa34072220 2024-04-24T23:11:18.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.526+0000 7ffa3bf42640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ffa340cde80 con 0x7ffa340ae8b0 2024-04-24T23:11:18.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.527+0000 7ffa39cde640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa20003f20 0x7ffa341f8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:18.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.527+0000 7ffa39cde640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa20003f20 0x7ffa341f8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49528/0 (socket says 172.21.15.32:49528) 2024-04-24T23:11:18.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.527+0000 7ffa39cde640 1 -- 172.21.15.32:0/735260154 learned_addr learned my addr 172.21.15.32:0/735260154 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:18.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.527+0000 7ffa3a4df640 1 --2- 172.21.15.32:0/735260154 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa340ae8b0 0x7ffa340aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:18.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.527+0000 7ffa394dd640 1 --2- 172.21.15.32:0/735260154 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa34072220 0x7ffa3414e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:18.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.527+0000 7ffa39cde640 1 -- 172.21.15.32:0/735260154 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa340ae8b0 msgr2=0x7ffa340aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:18.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.527+0000 7ffa39cde640 1 --2- 172.21.15.32:0/735260154 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa340ae8b0 0x7ffa340aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:18.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.527+0000 7ffa39cde640 1 -- 172.21.15.32:0/735260154 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa34072220 msgr2=0x7ffa3414e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:18.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.527+0000 7ffa39cde640 1 --2- 172.21.15.32:0/735260154 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa34072220 0x7ffa3414e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:18.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.527+0000 7ffa39cde640 1 -- 172.21.15.32:0/735260154 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa340726e0 con 0x7ffa20003f20 2024-04-24T23:11:18.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.527+0000 7ffa39cde640 1 --2- 172.21.15.32:0/735260154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa20003f20 0x7ffa341f8900 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7ffa3000f7c0 tx=0x7ffa3000fc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:18.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.527+0000 7ffa2a7fc640 1 -- 172.21.15.32:0/735260154 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa300052e0 con 0x7ffa20003f20 2024-04-24T23:11:18.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.527+0000 7ffa2a7fc640 1 -- 172.21.15.32:0/735260154 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa30018070 con 0x7ffa20003f20 2024-04-24T23:11:18.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.527+0000 7ffa3bf42640 1 -- 172.21.15.32:0/735260154 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa340729c0 con 0x7ffa20003f20 2024-04-24T23:11:18.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.527+0000 7ffa2a7fc640 1 -- 172.21.15.32:0/735260154 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3000d970 con 0x7ffa20003f20 2024-04-24T23:11:18.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.527+0000 7ffa3bf42640 1 -- 172.21.15.32:0/735260154 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa34072f80 con 0x7ffa20003f20 2024-04-24T23:11:18.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.528+0000 7ffa3bf42640 1 -- 172.21.15.32:0/735260154 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa34068430 con 0x7ffa20003f20 2024-04-24T23:11:18.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.529+0000 7ffa2a7fc640 1 -- 172.21.15.32:0/735260154 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffa3000db10 con 0x7ffa20003f20 2024-04-24T23:11:18.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.529+0000 7ffa2a7fc640 1 --2- 172.21.15.32:0/735260154 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ffa1806be30 0x7ffa1806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:18.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.529+0000 7ffa2a7fc640 1 -- 172.21.15.32:0/735260154 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(192..192 src has 1..192) ==== 5824+0+0 (secure 0 0 0) 0x7ffa3008aea0 con 0x7ffa20003f20 2024-04-24T23:11:18.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.529+0000 7ffa2a7fc640 1 -- 172.21.15.32:0/735260154 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=193}) -- 0x7ffa18073ad0 con 0x7ffa20003f20 2024-04-24T23:11:18.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.530+0000 7ffa394dd640 1 --2- 172.21.15.32:0/735260154 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ffa1806be30 0x7ffa1806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:18.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.530+0000 7ffa394dd640 1 --2- 172.21.15.32:0/735260154 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ffa1806be30 0x7ffa1806e2f0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7ffa24032000 tx=0x7ffa24009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:18.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.533+0000 7ffa2a7fc640 1 -- 172.21.15.32:0/735260154 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ffa3001a310 con 0x7ffa20003f20 2024-04-24T23:11:18.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:18.720+0000 7ffa3bf42640 1 -- 172.21.15.32:0/735260154 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7ffa340cc1f0 con 0x7ffa20003f20 2024-04-24T23:11:19.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.351+0000 7ffa2a7fc640 1 -- 172.21.15.32:0/735260154 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v193) ==== 181+0+0 (secure 0 0 0) 0x7ffa300597e0 con 0x7ffa20003f20 2024-04-24T23:11:19.353 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2024-04-24T23:11:19.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.351+0000 7ffa2a7fc640 1 -- 172.21.15.32:0/735260154 <== mon.0 v2:172.21.15.32:3300/0 8 ==== osd_map(193..193 src has 1..193) ==== 296+0+0 (secure 0 0 0) 0x7ffa30089470 con 0x7ffa20003f20 2024-04-24T23:11:19.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.351+0000 7ffa2a7fc640 1 -- 172.21.15.32:0/735260154 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=194}) -- 0x7ffa18071f00 con 0x7ffa20003f20 2024-04-24T23:11:19.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.357+0000 7ffa3bf42640 1 -- 172.21.15.32:0/735260154 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ffa1806be30 msgr2=0x7ffa1806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:19.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.357+0000 7ffa3bf42640 1 --2- 172.21.15.32:0/735260154 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ffa1806be30 0x7ffa1806e2f0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7ffa24032000 tx=0x7ffa24009030 comp rx=0 tx=0).stop 2024-04-24T23:11:19.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.357+0000 7ffa3bf42640 1 -- 172.21.15.32:0/735260154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa20003f20 msgr2=0x7ffa341f8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:19.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.357+0000 7ffa3bf42640 1 --2- 172.21.15.32:0/735260154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa20003f20 0x7ffa341f8900 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7ffa3000f7c0 tx=0x7ffa3000fc80 comp rx=0 tx=0).stop 2024-04-24T23:11:19.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.359+0000 7ffa3bf42640 1 -- 172.21.15.32:0/735260154 shutdown_connections 2024-04-24T23:11:19.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.359+0000 7ffa3bf42640 1 --2- 172.21.15.32:0/735260154 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ffa1806be30 0x7ffa1806e2f0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:19.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.359+0000 7ffa3bf42640 1 --2- 172.21.15.32:0/735260154 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa340ae8b0 0x7ffa340aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:19.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.359+0000 7ffa3bf42640 1 --2- 172.21.15.32:0/735260154 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa34072220 0x7ffa3414e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:19.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.359+0000 7ffa3bf42640 1 --2- 172.21.15.32:0/735260154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa20003f20 0x7ffa341f8900 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:19.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.359+0000 7ffa3bf42640 1 -- 172.21.15.32:0/735260154 >> 172.21.15.32:0/735260154 conn(0x7ffa340b1f10 msgr2=0x7ffa34155550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:19.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.359+0000 7ffa3bf42640 1 -- 172.21.15.32:0/735260154 shutdown_connections 2024-04-24T23:11:19.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.359+0000 7ffa3bf42640 1 -- 172.21.15.32:0/735260154 wait complete. 2024-04-24T23:11:19.374 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-24T23:11:19.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.483+0000 7f1fbedce640 1 Processor -- start 2024-04-24T23:11:19.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.483+0000 7f1fbedce640 1 -- start start 2024-04-24T23:11:19.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.484+0000 7f1fbedce640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1fb815c4f0 0x7f1fb81601a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:19.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.484+0000 7f1fbedce640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1fb8067730 0x7f1fb81606e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:19.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.484+0000 7f1fbedce640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fb80b5cf0 con 0x7f1fb8160c20 2024-04-24T23:11:19.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.484+0000 7f1fbedce640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f1fb80b5860 con 0x7f1fb8067730 2024-04-24T23:11:19.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.484+0000 7f1fbedce640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f1fb80b4c50 con 0x7f1fb815c4f0 2024-04-24T23:11:19.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.484+0000 7f1fbd36b640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1fb8160c20 0x7f1fb815c8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36036/0 (socket says 172.21.15.32:36036) 2024-04-24T23:11:19.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.484+0000 7f1fbd36b640 1 -- 172.21.15.32:0/3612514386 learned_addr learned my addr 172.21.15.32:0/3612514386 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:11:19.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.484+0000 7f1fb7fff640 1 --2- 172.21.15.32:0/3612514386 >> v2:172.21.15.121:3300/0 conn(0x7f1fb8067730 0x7f1fb81606e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:19.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.484+0000 7f1fbcb6a640 1 --2- 172.21.15.32:0/3612514386 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1fb815c4f0 0x7f1fb81601a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:19.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.484+0000 7f1fb77fe640 1 -- 172.21.15.32:0/3612514386 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 356639029 0 0) 0x7f1fb80b5cf0 con 0x7f1fb8160c20 2024-04-24T23:11:19.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.484+0000 7f1fb77fe640 1 -- 172.21.15.32:0/3612514386 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1fa8003520 con 0x7f1fb8160c20 2024-04-24T23:11:19.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.485+0000 7f1fb7fff640 1 -- 172.21.15.32:0/3612514386 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1fb815c4f0 msgr2=0x7f1fb81601a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:19.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.485+0000 7f1fb7fff640 1 --2- 172.21.15.32:0/3612514386 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1fb815c4f0 0x7f1fb81601a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:19.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.485+0000 7f1fb77fe640 1 -- 172.21.15.32:0/3612514386 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3410078622 0 0) 0x7f1fa8003520 con 0x7f1fb8160c20 2024-04-24T23:11:19.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.485+0000 7f1fb7fff640 1 -- 172.21.15.32:0/3612514386 >> v1:172.21.15.32:6789/0 conn(0x7f1fb8160c20 legacy=0x7f1fb815c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:19.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.485+0000 7f1fb7fff640 1 -- 172.21.15.32:0/3612514386 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fb80b4f30 con 0x7f1fb8067730 2024-04-24T23:11:19.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.485+0000 7f1fbcb6a640 1 --2- 172.21.15.32:0/3612514386 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1fb815c4f0 0x7f1fb81601a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:11:19.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.485+0000 7f1fb7fff640 1 --2- 172.21.15.32:0/3612514386 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fb8067730 0x7f1fb81606e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f1fa40097a0 tx=0x7f1fa402a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d316564b72b6463 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:19.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.486+0000 7f1fb77fe640 1 -- 172.21.15.32:0/3612514386 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa4037070 con 0x7f1fb8067730 2024-04-24T23:11:19.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.486+0000 7f1fb77fe640 1 -- 172.21.15.32:0/3612514386 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fa402ab10 con 0x7f1fb8067730 2024-04-24T23:11:19.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.486+0000 7f1fb77fe640 1 -- 172.21.15.32:0/3612514386 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb00019f0 con 0x7f1fb8067730 2024-04-24T23:11:19.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.486+0000 7f1fbedce640 1 -- 172.21.15.32:0/3612514386 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fb8067730 msgr2=0x7f1fb81606e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:19.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.486+0000 7f1fbedce640 1 --2- 172.21.15.32:0/3612514386 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fb8067730 0x7f1fb81606e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f1fa40097a0 tx=0x7f1fa402a080 comp rx=0 tx=0).stop 2024-04-24T23:11:19.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.486+0000 7f1fbedce640 1 -- 172.21.15.32:0/3612514386 shutdown_connections 2024-04-24T23:11:19.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.486+0000 7f1fbedce640 1 --2- 172.21.15.32:0/3612514386 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fb8067730 0x7f1fb81606e0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:19.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.486+0000 7f1fbedce640 1 --2- 172.21.15.32:0/3612514386 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1fb815c4f0 0x7f1fb81601a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:19.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.486+0000 7f1fbedce640 1 -- 172.21.15.32:0/3612514386 >> 172.21.15.32:0/3612514386 conn(0x7f1fb80d0a60 msgr2=0x7f1fb80d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:19.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.486+0000 7f1fbedce640 1 -- 172.21.15.32:0/3612514386 shutdown_connections 2024-04-24T23:11:19.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.486+0000 7f1fbedce640 1 -- 172.21.15.32:0/3612514386 wait complete. 2024-04-24T23:11:19.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.488+0000 7f1fbedce640 1 Processor -- start 2024-04-24T23:11:19.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.488+0000 7f1fbedce640 1 -- start start 2024-04-24T23:11:19.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.488+0000 7f1fbedce640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fb8067730 0x7f1fb815cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:19.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.488+0000 7f1fbedce640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1fb815c4f0 0x7f1fb815e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:19.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.488+0000 7f1fbedce640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fb8160c20 0x7f1fb8153100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:19.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.488+0000 7f1fbedce640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb80d0780 con 0x7f1fb8067730 2024-04-24T23:11:19.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.488+0000 7f1fbedce640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb80b54f0 con 0x7f1fb8160c20 2024-04-24T23:11:19.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.488+0000 7f1fbedce640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f1fb80cf910 con 0x7f1fb815c4f0 2024-04-24T23:11:19.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.489+0000 7f1fbcb6a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fb8067730 0x7f1fb815cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:19.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.489+0000 7f1fbcb6a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fb8067730 0x7f1fb815cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49540/0 (socket says 172.21.15.32:49540) 2024-04-24T23:11:19.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.489+0000 7f1fbcb6a640 1 -- 172.21.15.32:0/549893449 learned_addr learned my addr 172.21.15.32:0/549893449 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:19.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.489+0000 7f1fb7fff640 1 --2- 172.21.15.32:0/549893449 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1fb815c4f0 0x7f1fb815e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:19.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.489+0000 7f1fbd36b640 1 --2- 172.21.15.32:0/549893449 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fb8160c20 0x7f1fb8153100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:19.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.489+0000 7f1fbcb6a640 1 -- 172.21.15.32:0/549893449 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1fb815c4f0 msgr2=0x7f1fb815e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:19.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.489+0000 7f1fbcb6a640 1 --2- 172.21.15.32:0/549893449 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1fb815c4f0 0x7f1fb815e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:19.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.489+0000 7f1fbcb6a640 1 -- 172.21.15.32:0/549893449 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fb8160c20 msgr2=0x7f1fb8153100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:19.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.489+0000 7f1fbcb6a640 1 --2- 172.21.15.32:0/549893449 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fb8160c20 0x7f1fb8153100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:19.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.489+0000 7f1fbcb6a640 1 -- 172.21.15.32:0/549893449 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fb815f0d0 con 0x7f1fb8067730 2024-04-24T23:11:19.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.489+0000 7f1fb7fff640 1 --2- 172.21.15.32:0/549893449 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1fb815c4f0 0x7f1fb815e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:11:19.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.489+0000 7f1fbd36b640 1 --2- 172.21.15.32:0/549893449 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fb8160c20 0x7f1fb8153100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:11:19.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.489+0000 7f1fbcb6a640 1 --2- 172.21.15.32:0/549893449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fb8067730 0x7f1fb815cd70 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f1fac00ebe0 tx=0x7f1fac00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:19.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.489+0000 7f1fb57fa640 1 -- 172.21.15.32:0/549893449 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fac017070 con 0x7f1fb8067730 2024-04-24T23:11:19.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.489+0000 7f1fb57fa640 1 -- 172.21.15.32:0/549893449 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fac00cc30 con 0x7f1fb8067730 2024-04-24T23:11:19.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.489+0000 7f1fbedce640 1 -- 172.21.15.32:0/549893449 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fb815f3b0 con 0x7f1fb8067730 2024-04-24T23:11:19.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.489+0000 7f1fb57fa640 1 -- 172.21.15.32:0/549893449 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fac013470 con 0x7f1fb8067730 2024-04-24T23:11:19.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.489+0000 7f1fbedce640 1 -- 172.21.15.32:0/549893449 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fb815f970 con 0x7f1fb8067730 2024-04-24T23:11:19.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.490+0000 7f1fbedce640 1 -- 172.21.15.32:0/549893449 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fb8068430 con 0x7f1fb8067730 2024-04-24T23:11:19.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.491+0000 7f1fb57fa640 1 -- 172.21.15.32:0/549893449 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1fac01a070 con 0x7f1fb8067730 2024-04-24T23:11:19.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.491+0000 7f1fb57fa640 1 --2- 172.21.15.32:0/549893449 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1f9c06be30 0x7f1f9c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:19.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.491+0000 7f1fb57fa640 1 -- 172.21.15.32:0/549893449 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(193..193 src has 1..193) ==== 5386+0+0 (secure 0 0 0) 0x7f1fac08b010 con 0x7f1fb8067730 2024-04-24T23:11:19.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.491+0000 7f1fb57fa640 1 -- 172.21.15.32:0/549893449 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=194}) -- 0x7f1f9c072c60 con 0x7f1fb8067730 2024-04-24T23:11:19.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.492+0000 7f1fb7fff640 1 --2- 172.21.15.32:0/549893449 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1f9c06be30 0x7f1f9c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:19.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.492+0000 7f1fb7fff640 1 --2- 172.21.15.32:0/549893449 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1f9c06be30 0x7f1f9c06e2f0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4034000 tx=0x7f1fa402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:19.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.495+0000 7f1fb57fa640 1 -- 172.21.15.32:0/549893449 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1fac059b00 con 0x7f1fb8067730 2024-04-24T23:11:19.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:19.677+0000 7f1fbedce640 1 -- 172.21.15.32:0/549893449 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1fb80d0200 con 0x7f1fb8067730 2024-04-24T23:11:20.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.352+0000 7f1fb57fa640 1 -- 172.21.15.32:0/549893449 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v194) ==== 169+0+0 (secure 0 0 0) 0x7f1fac059400 con 0x7f1fb8067730 2024-04-24T23:11:20.354 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2024-04-24T23:11:20.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.352+0000 7f1fb57fa640 1 -- 172.21.15.32:0/549893449 <== mon.0 v2:172.21.15.32:3300/0 8 ==== osd_map(194..194 src has 1..194) ==== 313+0+0 (secure 0 0 0) 0x7f1fac0bf2d0 con 0x7f1fb8067730 2024-04-24T23:11:20.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.352+0000 7f1fb57fa640 1 -- 172.21.15.32:0/549893449 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=195}) -- 0x7f1f9c0719f0 con 0x7f1fb8067730 2024-04-24T23:11:20.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.361+0000 7f1fbedce640 1 -- 172.21.15.32:0/549893449 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1f9c06be30 msgr2=0x7f1f9c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:20.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.361+0000 7f1fbedce640 1 --2- 172.21.15.32:0/549893449 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1f9c06be30 0x7f1f9c06e2f0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4034000 tx=0x7f1fa402e000 comp rx=0 tx=0).stop 2024-04-24T23:11:20.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.361+0000 7f1fbedce640 1 -- 172.21.15.32:0/549893449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fb8067730 msgr2=0x7f1fb815cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:20.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.361+0000 7f1fbedce640 1 --2- 172.21.15.32:0/549893449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fb8067730 0x7f1fb815cd70 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f1fac00ebe0 tx=0x7f1fac00c520 comp rx=0 tx=0).stop 2024-04-24T23:11:20.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.362+0000 7f1fbedce640 1 -- 172.21.15.32:0/549893449 shutdown_connections 2024-04-24T23:11:20.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.362+0000 7f1fbedce640 1 --2- 172.21.15.32:0/549893449 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f1f9c06be30 0x7f1f9c06e2f0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:20.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.362+0000 7f1fbedce640 1 --2- 172.21.15.32:0/549893449 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fb8160c20 0x7f1fb8153100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:20.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.362+0000 7f1fbedce640 1 --2- 172.21.15.32:0/549893449 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1fb815c4f0 0x7f1fb815e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:20.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.362+0000 7f1fbedce640 1 --2- 172.21.15.32:0/549893449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fb8067730 0x7f1fb815cd70 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:20.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.362+0000 7f1fbedce640 1 -- 172.21.15.32:0/549893449 >> 172.21.15.32:0/549893449 conn(0x7f1fb80d0a60 msgr2=0x7f1fb806cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:20.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.362+0000 7f1fbedce640 1 -- 172.21.15.32:0/549893449 shutdown_connections 2024-04-24T23:11:20.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.362+0000 7f1fbedce640 1 -- 172.21.15.32:0/549893449 wait complete. 2024-04-24T23:11:20.379 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-24T23:11:20.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.486+0000 7f26ca3c8640 1 Processor -- start 2024-04-24T23:11:20.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.486+0000 7f26ca3c8640 1 -- start start 2024-04-24T23:11:20.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.487+0000 7f26ca3c8640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f26c40b5af0 0x7f26c40ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:20.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.487+0000 7f26ca3c8640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f26c4162e10 0x7f26c415ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:20.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.487+0000 7f26ca3c8640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26c40cd270 con 0x7f26c4162320 2024-04-24T23:11:20.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.487+0000 7f26ca3c8640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f26c40b5660 con 0x7f26c40b5af0 2024-04-24T23:11:20.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.487+0000 7f26ca3c8640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f26c4071610 con 0x7f26c4162e10 2024-04-24T23:11:20.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.487+0000 7f26c37fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f26c40b5af0 0x7f26c40ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:20.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.487+0000 7f26c3fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f26c4162320 0x7f26c4162700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56308/0 (socket says 172.21.15.32:56308) 2024-04-24T23:11:20.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.487+0000 7f26c3fff640 1 -- 172.21.15.32:0/1255846548 learned_addr learned my addr 172.21.15.32:0/1255846548 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:11:20.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.487+0000 7f26c8965640 1 --2- 172.21.15.32:0/1255846548 >> v2:172.21.15.121:3300/0 conn(0x7f26c4162e10 0x7f26c415ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:20.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.487+0000 7f26c37fe640 1 -- 172.21.15.32:0/1255846548 >> v2:172.21.15.121:3300/0 conn(0x7f26c4162e10 msgr2=0x7f26c415ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:20.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.487+0000 7f26c37fe640 1 --2- 172.21.15.32:0/1255846548 >> v2:172.21.15.121:3300/0 conn(0x7f26c4162e10 0x7f26c415ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:20.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.487+0000 7f26c37fe640 1 -- 172.21.15.32:0/1255846548 >> v1:172.21.15.32:6789/0 conn(0x7f26c4162320 legacy=0x7f26c4162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:20.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.487+0000 7f26c37fe640 1 -- 172.21.15.32:0/1255846548 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26c40718f0 con 0x7f26c40b5af0 2024-04-24T23:11:20.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.488+0000 7f26c37fe640 1 --2- 172.21.15.32:0/1255846548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26c40b5af0 0x7f26c40ccc90 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f26c40d1bc0 tx=0x7f26b402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=334b8379781be589 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:20.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.488+0000 7f26c2ffd640 1 -- 172.21.15.32:0/1255846548 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b4037070 con 0x7f26c40b5af0 2024-04-24T23:11:20.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.488+0000 7f26c2ffd640 1 -- 172.21.15.32:0/1255846548 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26b4032440 con 0x7f26c40b5af0 2024-04-24T23:11:20.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.488+0000 7f26c2ffd640 1 -- 172.21.15.32:0/1255846548 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b402ae60 con 0x7f26c40b5af0 2024-04-24T23:11:20.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.489+0000 7f26ca3c8640 1 -- 172.21.15.32:0/1255846548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26c40b5af0 msgr2=0x7f26c40ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:20.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.489+0000 7f26ca3c8640 1 --2- 172.21.15.32:0/1255846548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26c40b5af0 0x7f26c40ccc90 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f26c40d1bc0 tx=0x7f26b402a110 comp rx=0 tx=0).stop 2024-04-24T23:11:20.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.489+0000 7f26ca3c8640 1 -- 172.21.15.32:0/1255846548 shutdown_connections 2024-04-24T23:11:20.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.489+0000 7f26ca3c8640 1 --2- 172.21.15.32:0/1255846548 >> v2:172.21.15.121:3300/0 conn(0x7f26c4162e10 0x7f26c415ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:20.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.489+0000 7f26ca3c8640 1 --2- 172.21.15.32:0/1255846548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26c40b5af0 0x7f26c40ccc90 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:20.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.489+0000 7f26ca3c8640 1 -- 172.21.15.32:0/1255846548 >> 172.21.15.32:0/1255846548 conn(0x7f26c40d0ae0 msgr2=0x7f26c40d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:20.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.489+0000 7f26ca3c8640 1 -- 172.21.15.32:0/1255846548 shutdown_connections 2024-04-24T23:11:20.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.489+0000 7f26ca3c8640 1 -- 172.21.15.32:0/1255846548 wait complete. 2024-04-24T23:11:20.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.490+0000 7f26ca3c8640 1 Processor -- start 2024-04-24T23:11:20.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.490+0000 7f26ca3c8640 1 -- start start 2024-04-24T23:11:20.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.491+0000 7f26ca3c8640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26c40b5af0 0x7f26c414e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:20.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.491+0000 7f26ca3c8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26c4162320 0x7f26c40c5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:20.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.491+0000 7f26ca3c8640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26c4162e10 0x7f26c414d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:20.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.491+0000 7f26ca3c8640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f26c4071bd0 con 0x7f26c4162320 2024-04-24T23:11:20.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.491+0000 7f26ca3c8640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f26c4067950 con 0x7f26c4162e10 2024-04-24T23:11:20.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.491+0000 7f26ca3c8640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f26c4071d50 con 0x7f26c40b5af0 2024-04-24T23:11:20.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.491+0000 7f26c37fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26c4162320 0x7f26c40c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:20.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.491+0000 7f26c37fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26c4162320 0x7f26c40c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54414/0 (socket says 172.21.15.32:54414) 2024-04-24T23:11:20.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.491+0000 7f26c37fe640 1 -- 172.21.15.32:0/38331765 learned_addr learned my addr 172.21.15.32:0/38331765 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:20.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.491+0000 7f26c37fe640 1 -- 172.21.15.32:0/38331765 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26c40b5af0 msgr2=0x7f26c414e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:20.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.491+0000 7f26c37fe640 1 --2- 172.21.15.32:0/38331765 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26c40b5af0 0x7f26c414e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:20.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.491+0000 7f26c37fe640 1 -- 172.21.15.32:0/38331765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26c4162e10 msgr2=0x7f26c414d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:20.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.491+0000 7f26c37fe640 1 --2- 172.21.15.32:0/38331765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26c4162e10 0x7f26c414d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:20.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.491+0000 7f26c37fe640 1 -- 172.21.15.32:0/38331765 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26c414e600 con 0x7f26c4162320 2024-04-24T23:11:20.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.491+0000 7f26c37fe640 1 --2- 172.21.15.32:0/38331765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26c4162320 0x7f26c40c5980 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f26b4009890 tx=0x7f26b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:20.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.492+0000 7f26c0ff9640 1 -- 172.21.15.32:0/38331765 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b4037040 con 0x7f26c4162320 2024-04-24T23:11:20.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.492+0000 7f26c0ff9640 1 -- 172.21.15.32:0/38331765 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26b4042070 con 0x7f26c4162320 2024-04-24T23:11:20.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.492+0000 7f26c0ff9640 1 -- 172.21.15.32:0/38331765 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b4030bd0 con 0x7f26c4162320 2024-04-24T23:11:20.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.492+0000 7f26ca3c8640 1 -- 172.21.15.32:0/38331765 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26c414e8e0 con 0x7f26c4162320 2024-04-24T23:11:20.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.492+0000 7f26ca3c8640 1 -- 172.21.15.32:0/38331765 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26c414eea0 con 0x7f26c4162320 2024-04-24T23:11:20.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.493+0000 7f26ca3c8640 1 -- 172.21.15.32:0/38331765 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26c4068430 con 0x7f26c4162320 2024-04-24T23:11:20.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.493+0000 7f26c0ff9640 1 -- 172.21.15.32:0/38331765 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f26b4047460 con 0x7f26c4162320 2024-04-24T23:11:20.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.493+0000 7f26c0ff9640 1 --2- 172.21.15.32:0/38331765 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f26b006be60 0x7f26b006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:20.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.494+0000 7f26c0ff9640 1 -- 172.21.15.32:0/38331765 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 4943+0+0 (secure 0 0 0) 0x7f26b40ab6e0 con 0x7f26c4162320 2024-04-24T23:11:20.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.494+0000 7f26c3fff640 1 --2- 172.21.15.32:0/38331765 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f26b006be60 0x7f26b006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:20.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.494+0000 7f26c3fff640 1 --2- 172.21.15.32:0/38331765 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f26b006be60 0x7f26b006e320 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f26b8000f60 tx=0x7f26b8007a80 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:20.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.498+0000 7f26c0ff9640 1 -- 172.21.15.32:0/38331765 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f26b4032e20 con 0x7f26c4162320 2024-04-24T23:11:20.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.681+0000 7f26ca3c8640 1 -- 172.21.15.32:0/38331765 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f26c4067730 con 0x7f26c4162320 2024-04-24T23:11:20.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.685+0000 7f26c0ff9640 1 -- 172.21.15.32:0/38331765 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) ==== 104+0+0 (secure 0 0 0) 0x7f26b407a390 con 0x7f26c4162320 2024-04-24T23:11:20.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.687+0000 7f26ca3c8640 1 -- 172.21.15.32:0/38331765 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f26b006be60 msgr2=0x7f26b006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:20.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.687+0000 7f26ca3c8640 1 --2- 172.21.15.32:0/38331765 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f26b006be60 0x7f26b006e320 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f26b8000f60 tx=0x7f26b8007a80 comp rx=0 tx=0).stop 2024-04-24T23:11:20.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.687+0000 7f26ca3c8640 1 -- 172.21.15.32:0/38331765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26c4162320 msgr2=0x7f26c40c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:20.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.687+0000 7f26ca3c8640 1 --2- 172.21.15.32:0/38331765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26c4162320 0x7f26c40c5980 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f26b4009890 tx=0x7f26b40046c0 comp rx=0 tx=0).stop 2024-04-24T23:11:20.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.687+0000 7f26ca3c8640 1 -- 172.21.15.32:0/38331765 shutdown_connections 2024-04-24T23:11:20.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.687+0000 7f26ca3c8640 1 --2- 172.21.15.32:0/38331765 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f26b006be60 0x7f26b006e320 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:20.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.687+0000 7f26ca3c8640 1 --2- 172.21.15.32:0/38331765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26c4162320 0x7f26c40c5980 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:20.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.687+0000 7f26ca3c8640 1 --2- 172.21.15.32:0/38331765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26c4162e10 0x7f26c414d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:20.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.687+0000 7f26ca3c8640 1 --2- 172.21.15.32:0/38331765 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26c40b5af0 0x7f26c414e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:20.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.687+0000 7f26ca3c8640 1 -- 172.21.15.32:0/38331765 >> 172.21.15.32:0/38331765 conn(0x7f26c40d0ae0 msgr2=0x7f26c415d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:20.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.687+0000 7f26ca3c8640 1 -- 172.21.15.32:0/38331765 shutdown_connections 2024-04-24T23:11:20.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.687+0000 7f26ca3c8640 1 -- 172.21.15.32:0/38331765 wait complete. 2024-04-24T23:11:20.843 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file' 2024-04-24T23:11:20.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.943+0000 7f2e9c2b2640 1 Processor -- start 2024-04-24T23:11:20.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.943+0000 7f2e9c2b2640 1 -- start start 2024-04-24T23:11:20.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.944+0000 7f2e9c2b2640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2e9415e9e0 0x7f2e940d40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:20.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.944+0000 7f2e9c2b2640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e940ba390 0x7f2e940d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:20.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.944+0000 7f2e9c2b2640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e940d3010 con 0x7f2e9406b110 2024-04-24T23:11:20.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.944+0000 7f2e9c2b2640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f2e940d32f0 con 0x7f2e9415e9e0 2024-04-24T23:11:20.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.944+0000 7f2e9c2b2640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f2e940d35d0 con 0x7f2e940ba390 2024-04-24T23:11:20.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.944+0000 7f2e9a84f640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2e9406b110 0x7f2e94071730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56316/0 (socket says 172.21.15.32:56316) 2024-04-24T23:11:20.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.944+0000 7f2e9a84f640 1 -- 172.21.15.32:0/107905748 learned_addr learned my addr 172.21.15.32:0/107905748 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:11:20.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.945+0000 7f2e9984d640 1 --2- 172.21.15.32:0/107905748 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e940ba390 0x7f2e940d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:20.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.945+0000 7f2e9a04e640 1 --2- 172.21.15.32:0/107905748 >> v2:172.21.15.121:3300/0 conn(0x7f2e9415e9e0 0x7f2e940d40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:20.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.945+0000 7f2e9904c640 1 -- 172.21.15.32:0/107905748 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3228933226 0 0) 0x7f2e940d3010 con 0x7f2e9406b110 2024-04-24T23:11:20.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.945+0000 7f2e9904c640 1 -- 172.21.15.32:0/107905748 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e7c003520 con 0x7f2e9406b110 2024-04-24T23:11:20.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.945+0000 7f2e9984d640 1 -- 172.21.15.32:0/107905748 >> v2:172.21.15.121:3300/0 conn(0x7f2e9415e9e0 msgr2=0x7f2e940d40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:20.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.945+0000 7f2e9984d640 1 --2- 172.21.15.32:0/107905748 >> v2:172.21.15.121:3300/0 conn(0x7f2e9415e9e0 0x7f2e940d40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:20.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.945+0000 7f2e9984d640 1 -- 172.21.15.32:0/107905748 >> v1:172.21.15.32:6789/0 conn(0x7f2e9406b110 legacy=0x7f2e94071730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:20.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.945+0000 7f2e9984d640 1 -- 172.21.15.32:0/107905748 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e940d38b0 con 0x7f2e940ba390 2024-04-24T23:11:20.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.945+0000 7f2e9a04e640 1 --2- 172.21.15.32:0/107905748 >> v2:172.21.15.121:3300/0 conn(0x7f2e9415e9e0 0x7f2e940d40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:11:20.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.946+0000 7f2e9984d640 1 --2- 172.21.15.32:0/107905748 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e940ba390 0x7f2e940d26a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2e880097a0 tx=0x7f2e8802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b01ee132b62cb1f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:20.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.946+0000 7f2e9904c640 1 -- 172.21.15.32:0/107905748 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e88037070 con 0x7f2e940ba390 2024-04-24T23:11:20.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.946+0000 7f2e9904c640 1 -- 172.21.15.32:0/107905748 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e88032440 con 0x7f2e940ba390 2024-04-24T23:11:20.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.946+0000 7f2e9904c640 1 -- 172.21.15.32:0/107905748 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e880327f0 con 0x7f2e940ba390 2024-04-24T23:11:20.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.946+0000 7f2e9c2b2640 1 -- 172.21.15.32:0/107905748 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e940ba390 msgr2=0x7f2e940d26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:20.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.946+0000 7f2e9c2b2640 1 --2- 172.21.15.32:0/107905748 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e940ba390 0x7f2e940d26a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2e880097a0 tx=0x7f2e8802a180 comp rx=0 tx=0).stop 2024-04-24T23:11:20.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.946+0000 7f2e9c2b2640 1 -- 172.21.15.32:0/107905748 shutdown_connections 2024-04-24T23:11:20.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.946+0000 7f2e9c2b2640 1 --2- 172.21.15.32:0/107905748 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e940ba390 0x7f2e940d26a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:20.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.946+0000 7f2e9c2b2640 1 --2- 172.21.15.32:0/107905748 >> v2:172.21.15.121:3300/0 conn(0x7f2e9415e9e0 0x7f2e940d40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:20.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.946+0000 7f2e9c2b2640 1 -- 172.21.15.32:0/107905748 >> 172.21.15.32:0/107905748 conn(0x7f2e940d12e0 msgr2=0x7f2e940d16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:20.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.947+0000 7f2e9c2b2640 1 -- 172.21.15.32:0/107905748 shutdown_connections 2024-04-24T23:11:20.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.947+0000 7f2e9c2b2640 1 -- 172.21.15.32:0/107905748 wait complete. 2024-04-24T23:11:20.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.948+0000 7f2e9c2b2640 1 Processor -- start 2024-04-24T23:11:20.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.948+0000 7f2e9c2b2640 1 -- start start 2024-04-24T23:11:20.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.948+0000 7f2e9c2b2640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e9406b110 0x7f2e941f52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:20.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.948+0000 7f2e9c2b2640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e940ba390 0x7f2e94154330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:20.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.948+0000 7f2e9c2b2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e9415e9e0 0x7f2e94154870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:20.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.949+0000 7f2e9c2b2640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2e940d2d60 con 0x7f2e9415e9e0 2024-04-24T23:11:20.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.949+0000 7f2e9c2b2640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2e940d2be0 con 0x7f2e9406b110 2024-04-24T23:11:20.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.949+0000 7f2e9c2b2640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f2e9406b5c0 con 0x7f2e940ba390 2024-04-24T23:11:20.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.949+0000 7f2e9a84f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e9415e9e0 0x7f2e94154870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:20.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.949+0000 7f2e9a84f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e9415e9e0 0x7f2e94154870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54416/0 (socket says 172.21.15.32:54416) 2024-04-24T23:11:20.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.949+0000 7f2e9a84f640 1 -- 172.21.15.32:0/3433863972 learned_addr learned my addr 172.21.15.32:0/3433863972 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:20.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.949+0000 7f2e9a04e640 1 --2- 172.21.15.32:0/3433863972 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e9406b110 0x7f2e941f52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:20.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.949+0000 7f2e9984d640 1 --2- 172.21.15.32:0/3433863972 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e940ba390 0x7f2e94154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:20.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.949+0000 7f2e9a84f640 1 -- 172.21.15.32:0/3433863972 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e940ba390 msgr2=0x7f2e94154330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:20.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.949+0000 7f2e9a84f640 1 --2- 172.21.15.32:0/3433863972 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e940ba390 0x7f2e94154330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:20.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.949+0000 7f2e9a84f640 1 -- 172.21.15.32:0/3433863972 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e9406b110 msgr2=0x7f2e941f52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:20.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.949+0000 7f2e9a84f640 1 --2- 172.21.15.32:0/3433863972 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e9406b110 0x7f2e941f52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:20.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.949+0000 7f2e9a84f640 1 -- 172.21.15.32:0/3433863972 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e94161600 con 0x7f2e9415e9e0 2024-04-24T23:11:20.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.949+0000 7f2e9a84f640 1 --2- 172.21.15.32:0/3433863972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e9415e9e0 0x7f2e94154870 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f2e8400cb40 tx=0x7f2e8400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:20.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.949+0000 7f2e92ffd640 1 -- 172.21.15.32:0/3433863972 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e84017070 con 0x7f2e9415e9e0 2024-04-24T23:11:20.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.949+0000 7f2e92ffd640 1 -- 172.21.15.32:0/3433863972 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e8400aca0 con 0x7f2e9415e9e0 2024-04-24T23:11:20.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.949+0000 7f2e9c2b2640 1 -- 172.21.15.32:0/3433863972 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e941618e0 con 0x7f2e9415e9e0 2024-04-24T23:11:20.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.949+0000 7f2e92ffd640 1 -- 172.21.15.32:0/3433863972 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e84012620 con 0x7f2e9415e9e0 2024-04-24T23:11:20.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.949+0000 7f2e9c2b2640 1 -- 172.21.15.32:0/3433863972 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e941f60c0 con 0x7f2e9415e9e0 2024-04-24T23:11:20.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.950+0000 7f2e9c2b2640 1 -- 172.21.15.32:0/3433863972 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e94068430 con 0x7f2e9415e9e0 2024-04-24T23:11:20.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.951+0000 7f2e92ffd640 1 -- 172.21.15.32:0/3433863972 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2e8401a070 con 0x7f2e9415e9e0 2024-04-24T23:11:20.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.951+0000 7f2e92ffd640 1 --2- 172.21.15.32:0/3433863972 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2e8006be30 0x7f2e8006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:20.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.952+0000 7f2e92ffd640 1 -- 172.21.15.32:0/3433863972 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 4943+0+0 (secure 0 0 0) 0x7f2e8408a900 con 0x7f2e9415e9e0 2024-04-24T23:11:20.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.952+0000 7f2e9a04e640 1 --2- 172.21.15.32:0/3433863972 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2e8006be30 0x7f2e8006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:20.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.952+0000 7f2e9a04e640 1 --2- 172.21.15.32:0/3433863972 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2e8006be30 0x7f2e8006e2f0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c009560 tx=0x7f2e8c009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:20.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:20.956+0000 7f2e92ffd640 1 -- 172.21.15.32:0/3433863972 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2e84058f50 con 0x7f2e9415e9e0 2024-04-24T23:11:21.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:21.209+0000 7f2e9c2b2640 1 -- 172.21.15.32:0/3433863972 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]} v 0) -- 0x7f2e940d4260 con 0x7f2e9415e9e0 2024-04-24T23:11:21.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:21.362+0000 7f2e92ffd640 1 -- 172.21.15.32:0/3433863972 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]}]=0 v584) ==== 135+0+0 (secure 0 0 0) 0x7f2e8405d250 con 0x7f2e9415e9e0 2024-04-24T23:11:21.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:21.364+0000 7f2e9c2b2640 1 -- 172.21.15.32:0/3433863972 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2e8006be30 msgr2=0x7f2e8006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:21.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:21.364+0000 7f2e9c2b2640 1 --2- 172.21.15.32:0/3433863972 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2e8006be30 0x7f2e8006e2f0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c009560 tx=0x7f2e8c009190 comp rx=0 tx=0).stop 2024-04-24T23:11:21.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:21.364+0000 7f2e9c2b2640 1 -- 172.21.15.32:0/3433863972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e9415e9e0 msgr2=0x7f2e94154870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:21.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:21.364+0000 7f2e9c2b2640 1 --2- 172.21.15.32:0/3433863972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e9415e9e0 0x7f2e94154870 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f2e8400cb40 tx=0x7f2e8400a060 comp rx=0 tx=0).stop 2024-04-24T23:11:21.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:21.364+0000 7f2e9c2b2640 1 -- 172.21.15.32:0/3433863972 shutdown_connections 2024-04-24T23:11:21.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:21.364+0000 7f2e9c2b2640 1 --2- 172.21.15.32:0/3433863972 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f2e8006be30 0x7f2e8006e2f0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:21.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:21.364+0000 7f2e9c2b2640 1 --2- 172.21.15.32:0/3433863972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e9415e9e0 0x7f2e94154870 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:21.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:21.364+0000 7f2e9c2b2640 1 --2- 172.21.15.32:0/3433863972 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e940ba390 0x7f2e94154330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:21.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:21.364+0000 7f2e9c2b2640 1 --2- 172.21.15.32:0/3433863972 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e9406b110 0x7f2e941f52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:21.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:21.364+0000 7f2e9c2b2640 1 -- 172.21.15.32:0/3433863972 >> 172.21.15.32:0/3433863972 conn(0x7f2e940d12e0 msgr2=0x7f2e94155d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:21.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:21.364+0000 7f2e9c2b2640 1 -- 172.21.15.32:0/3433863972 shutdown_connections 2024-04-24T23:11:21.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:21.364+0000 7f2e9c2b2640 1 -- 172.21.15.32:0/3433863972 wait complete. 2024-04-24T23:11:21.377 INFO:tasks.cephfs_test_runner:test_full_same_file (tasks.cephfs.test_full.TestQuotaFull) ... ok 2024-04-24T23:11:21.377 INFO:tasks.cephfs_test_runner: 2024-04-24T23:11:21.377 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-24T23:11:21.377 INFO:tasks.cephfs_test_runner:Ran 10 tests in 707.046s 2024-04-24T23:11:21.377 INFO:tasks.cephfs_test_runner: 2024-04-24T23:11:21.377 INFO:tasks.cephfs_test_runner:OK 2024-04-24T23:11:21.378 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-24T23:11:21.387 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-24T23:11:21.410 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-24T23:11:21.410 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:11:21.410 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:11:21.427 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=413 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T23:11:21.427 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-24T23:11:21.427 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:11:21.483 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=413 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T23:11:21.483 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:11:21.483 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:11:21.502 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3271 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T23:11:21.503 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-24T23:11:21.503 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:11:21.561 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3271 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T23:11:21.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:11:21.561 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-04-24T23:11:21.577 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-24T23:11:21.577 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-24T23:11:21.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:11:21.577 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T23:11:21.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:11:21.642 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T23:11:21.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:11:21.709 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link delete ceph-brx 2024-04-24T23:11:21.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:11:21.811 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-04-24T23:11:21.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:11:21.827 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link delete ceph-brx 2024-04-24T23:11:21.895 INFO:teuthology.orchestra.run.smithi032.stderr:Cannot find device "ceph-brx" 2024-04-24T23:11:21.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:11:21.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:11:21.897 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2024-04-24T23:11:21.916 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-24T23:11:21.916 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-24T23:11:21.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:11:21.916 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-24T23:11:21.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:11:21.985 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T23:11:22.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:11:22.052 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link delete ceph-brx 2024-04-24T23:11:22.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:11:22.148 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2024-04-24T23:11:22.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:11:22.163 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link delete ceph-brx 2024-04-24T23:11:22.231 INFO:teuthology.orchestra.run.smithi121.stderr:Cannot find device "ceph-brx" 2024-04-24T23:11:22.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:11:22.233 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-24T23:11:22.242 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-24T23:11:22.242 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:11:22.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.344+0000 7f99eed61640 1 Processor -- start 2024-04-24T23:11:22.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.344+0000 7f99eed61640 1 -- start start 2024-04-24T23:11:22.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.345+0000 7f99eed61640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99e806b820 0x7f99e80d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:22.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.345+0000 7f99eed61640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99e80d3170 con 0x7f99e80d4210 2024-04-24T23:11:22.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.345+0000 7f99eed61640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99e80d3450 con 0x7f99e815e9b0 2024-04-24T23:11:22.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.345+0000 7f99eed61640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f99e80d3730 con 0x7f99e806b820 2024-04-24T23:11:22.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.345+0000 7f99ed2fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f99e80d4210 0x7f99e80b78f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56328/0 (socket says 172.21.15.32:56328) 2024-04-24T23:11:22.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.345+0000 7f99ed2fe640 1 -- 172.21.15.32:0/792339246 learned_addr learned my addr 172.21.15.32:0/792339246 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:11:22.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.345+0000 7f99e7fff640 1 --2- 172.21.15.32:0/792339246 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99e806b820 0x7f99e80d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:22.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.346+0000 7f99e77fe640 1 -- 172.21.15.32:0/792339246 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3916284897 0 0) 0x7f99e80d3170 con 0x7f99e80d4210 2024-04-24T23:11:22.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.346+0000 7f99e77fe640 1 -- 172.21.15.32:0/792339246 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f99d0003520 con 0x7f99e80d4210 2024-04-24T23:11:22.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.346+0000 7f99e77fe640 1 -- 172.21.15.32:0/792339246 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 112864880 0 0) 0x7f99e80d3450 con 0x7f99e815e9b0 2024-04-24T23:11:22.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.346+0000 7f99e77fe640 1 -- 172.21.15.32:0/792339246 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f99e80d3170 con 0x7f99e815e9b0 2024-04-24T23:11:22.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.346+0000 7f99e7fff640 1 -- 172.21.15.32:0/792339246 >> v1:172.21.15.121:6789/0 conn(0x7f99e815e9b0 legacy=0x7f99e806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:22.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.346+0000 7f99e7fff640 1 -- 172.21.15.32:0/792339246 >> v1:172.21.15.32:6789/0 conn(0x7f99e80d4210 legacy=0x7f99e80b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:22.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.346+0000 7f99e7fff640 1 -- 172.21.15.32:0/792339246 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99e80d3a10 con 0x7f99e806b820 2024-04-24T23:11:22.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.347+0000 7f99e7fff640 1 --2- 172.21.15.32:0/792339246 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99e806b820 0x7f99e80d2b90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f99dc0097d0 tx=0x7f99dc02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25d4c65fd104e035 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:22.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.347+0000 7f99e77fe640 1 -- 172.21.15.32:0/792339246 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99dc037070 con 0x7f99e806b820 2024-04-24T23:11:22.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.347+0000 7f99e77fe640 1 -- 172.21.15.32:0/792339246 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99dc032440 con 0x7f99e806b820 2024-04-24T23:11:22.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.347+0000 7f99e77fe640 1 -- 172.21.15.32:0/792339246 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99dc032810 con 0x7f99e806b820 2024-04-24T23:11:22.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.347+0000 7f99eed61640 1 -- 172.21.15.32:0/792339246 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99e806b820 msgr2=0x7f99e80d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:22.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.347+0000 7f99eed61640 1 --2- 172.21.15.32:0/792339246 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99e806b820 0x7f99e80d2b90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f99dc0097d0 tx=0x7f99dc02a210 comp rx=0 tx=0).stop 2024-04-24T23:11:22.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.347+0000 7f99eed61640 1 -- 172.21.15.32:0/792339246 shutdown_connections 2024-04-24T23:11:22.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.347+0000 7f99eed61640 1 --2- 172.21.15.32:0/792339246 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99e806b820 0x7f99e80d2b90 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:22.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.347+0000 7f99eed61640 1 -- 172.21.15.32:0/792339246 >> 172.21.15.32:0/792339246 conn(0x7f99e80d11d0 msgr2=0x7f99e80d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:22.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.347+0000 7f99eed61640 1 -- 172.21.15.32:0/792339246 shutdown_connections 2024-04-24T23:11:22.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.347+0000 7f99eed61640 1 -- 172.21.15.32:0/792339246 wait complete. 2024-04-24T23:11:22.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.349+0000 7f99eed61640 1 Processor -- start 2024-04-24T23:11:22.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.349+0000 7f99eed61640 1 -- start start 2024-04-24T23:11:22.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.349+0000 7f99eed61640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99e806b820 0x7f99e80cab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:22.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.349+0000 7f99eed61640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99e80d4210 0x7f99e815f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:22.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.349+0000 7f99eed61640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99e815e9b0 0x7f99e80cd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:22.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.350+0000 7f99eed61640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f99e80ba0e0 con 0x7f99e815e9b0 2024-04-24T23:11:22.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.350+0000 7f99eed61640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f99e80d2400 con 0x7f99e80d4210 2024-04-24T23:11:22.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.350+0000 7f99eed61640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f99e80ba260 con 0x7f99e806b820 2024-04-24T23:11:22.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.350+0000 7f99ed2fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99e815e9b0 0x7f99e80cd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:22.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.350+0000 7f99ed2fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99e815e9b0 0x7f99e80cd9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54432/0 (socket says 172.21.15.32:54432) 2024-04-24T23:11:22.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.350+0000 7f99ed2fe640 1 -- 172.21.15.32:0/4235616090 learned_addr learned my addr 172.21.15.32:0/4235616090 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:22.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.350+0000 7f99e7fff640 1 --2- 172.21.15.32:0/4235616090 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99e80d4210 0x7f99e815f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:22.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.350+0000 7f99ecafd640 1 --2- 172.21.15.32:0/4235616090 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99e806b820 0x7f99e80cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:22.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.350+0000 7f99ed2fe640 1 -- 172.21.15.32:0/4235616090 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99e806b820 msgr2=0x7f99e80cab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:22.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.350+0000 7f99ed2fe640 1 --2- 172.21.15.32:0/4235616090 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99e806b820 0x7f99e80cab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:22.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.350+0000 7f99ed2fe640 1 -- 172.21.15.32:0/4235616090 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99e80d4210 msgr2=0x7f99e815f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:22.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.350+0000 7f99ed2fe640 1 --2- 172.21.15.32:0/4235616090 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99e80d4210 0x7f99e815f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:22.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.350+0000 7f99ed2fe640 1 -- 172.21.15.32:0/4235616090 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99e81601b0 con 0x7f99e815e9b0 2024-04-24T23:11:22.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.350+0000 7f99e7fff640 1 --2- 172.21.15.32:0/4235616090 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99e80d4210 0x7f99e815f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:11:22.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.350+0000 7f99ed2fe640 1 --2- 172.21.15.32:0/4235616090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99e815e9b0 0x7f99e80cd9f0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f99e000cb10 tx=0x7f99e000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:22.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.350+0000 7f99e57fa640 1 -- 172.21.15.32:0/4235616090 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e0017070 con 0x7f99e815e9b0 2024-04-24T23:11:22.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.350+0000 7f99e57fa640 1 -- 172.21.15.32:0/4235616090 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99e000ac70 con 0x7f99e815e9b0 2024-04-24T23:11:22.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.350+0000 7f99eed61640 1 -- 172.21.15.32:0/4235616090 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99e81f7910 con 0x7f99e815e9b0 2024-04-24T23:11:22.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.350+0000 7f99e57fa640 1 -- 172.21.15.32:0/4235616090 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e0012640 con 0x7f99e815e9b0 2024-04-24T23:11:22.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.350+0000 7f99eed61640 1 -- 172.21.15.32:0/4235616090 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99e80cb050 con 0x7f99e815e9b0 2024-04-24T23:11:22.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.351+0000 7f99eed61640 1 -- 172.21.15.32:0/4235616090 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99e81f7d00 con 0x7f99e815e9b0 2024-04-24T23:11:22.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.352+0000 7f99e57fa640 1 -- 172.21.15.32:0/4235616090 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f99e001a070 con 0x7f99e815e9b0 2024-04-24T23:11:22.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.352+0000 7f99e57fa640 1 --2- 172.21.15.32:0/4235616090 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f99d406be30 0x7f99d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:22.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.353+0000 7f99e57fa640 1 -- 172.21.15.32:0/4235616090 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 4943+0+0 (secure 0 0 0) 0x7f99e008a7f0 con 0x7f99e815e9b0 2024-04-24T23:11:22.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.353+0000 7f99ecafd640 1 --2- 172.21.15.32:0/4235616090 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f99d406be30 0x7f99d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:22.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.353+0000 7f99ecafd640 1 --2- 172.21.15.32:0/4235616090 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f99d406be30 0x7f99d406e2f0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f99d8007fd0 tx=0x7f99d8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:22.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.357+0000 7f99e57fa640 1 -- 172.21.15.32:0/4235616090 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f99e000ae20 con 0x7f99e815e9b0 2024-04-24T23:11:22.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.540+0000 7f99eed61640 1 -- 172.21.15.32:0/4235616090 --> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f99e8068430 con 0x7f99d406be30 2024-04-24T23:11:22.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.542+0000 7f99e57fa640 1 -- 172.21.15.32:0/4235616090 <== mgr.4108 v2:172.21.15.121:6832/1848157943 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+12600 (secure 0 0 0) 0x7f99e8068430 con 0x7f99d406be30 2024-04-24T23:11:22.544 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:11:22.544 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-24T23:11:22.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.544+0000 7f99eed61640 1 -- 172.21.15.32:0/4235616090 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f99d406be30 msgr2=0x7f99d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:22.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.544+0000 7f99eed61640 1 --2- 172.21.15.32:0/4235616090 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f99d406be30 0x7f99d406e2f0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f99d8007fd0 tx=0x7f99d8009040 comp rx=0 tx=0).stop 2024-04-24T23:11:22.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.544+0000 7f99eed61640 1 -- 172.21.15.32:0/4235616090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99e815e9b0 msgr2=0x7f99e80cd9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:22.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.544+0000 7f99eed61640 1 --2- 172.21.15.32:0/4235616090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99e815e9b0 0x7f99e80cd9f0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f99e000cb10 tx=0x7f99e000a060 comp rx=0 tx=0).stop 2024-04-24T23:11:22.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.544+0000 7f99eed61640 1 -- 172.21.15.32:0/4235616090 shutdown_connections 2024-04-24T23:11:22.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.544+0000 7f99eed61640 1 --2- 172.21.15.32:0/4235616090 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f99d406be30 0x7f99d406e2f0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:22.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.544+0000 7f99eed61640 1 --2- 172.21.15.32:0/4235616090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99e815e9b0 0x7f99e80cd9f0 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:22.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.544+0000 7f99eed61640 1 --2- 172.21.15.32:0/4235616090 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99e80d4210 0x7f99e815f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:22.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.544+0000 7f99eed61640 1 --2- 172.21.15.32:0/4235616090 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99e806b820 0x7f99e80cab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:22.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.544+0000 7f99eed61640 1 -- 172.21.15.32:0/4235616090 >> 172.21.15.32:0/4235616090 conn(0x7f99e80d11d0 msgr2=0x7f99e8071fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:22.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.544+0000 7f99eed61640 1 -- 172.21.15.32:0/4235616090 shutdown_connections 2024-04-24T23:11:22.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.544+0000 7f99eed61640 1 -- 172.21.15.32:0/4235616090 wait complete. 2024-04-24T23:11:22.556 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":560,"stamp":"2024-04-24T23:11:21.304513+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":112073,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1450426,"statfs":{"total":1600000000,"available":1485237048,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-52491310,"num_objects":-36,"num_object_clones":0,"num_object_copies":-72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-36,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-131,"num_write_kb":-85088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.439471"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":509,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-04-24T23:11:16.866349+0000","last_change":"2024-04-24T22:58:55.138810+0000","last_active":"2024-04-24T23:11:16.866349+0000","last_peered":"2024-04-24T23:11:16.866349+0000","last_clean":"2024-04-24T23:11:16.866349+0000","last_became_active":"2024-04-24T22:58:55.138627+0000","last_became_peered":"2024-04-24T22:58:55.138627+0000","last_unstale":"2024-04-24T23:11:16.866349+0000","last_undegraded":"2024-04-24T23:11:16.866349+0000","last_fullsized":"2024-04-24T23:11:16.866349+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_clean_scrub_stamp":"2024-04-24T22:58:54.127235+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:51.383182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574999,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":5144,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":190168,"statfs":{"total":200000000,"available":194732364,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574999,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":17432,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":177880,"statfs":{"total":200000000,"available":182149216,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574999,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":22105,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":173207,"statfs":{"total":200000000,"available":177364108,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574999,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9240,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":186072,"statfs":{"total":200000000,"available":190538026,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574999,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":11354,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":183957,"statfs":{"total":200000000,"available":188372906,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574999,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":5144,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":190167,"statfs":{"total":200000000,"available":194731580,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574999,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":22165,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":173147,"statfs":{"total":200000000,"available":177302947,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574999,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":19486,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":175826,"statfs":{"total":200000000,"available":180045901,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-24T23:11:22.557 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:11:22.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.655+0000 7f9bf614f640 1 Processor -- start 2024-04-24T23:11:22.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.655+0000 7f9bf614f640 1 -- start start 2024-04-24T23:11:22.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.656+0000 7f9bf614f640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9bf0162320 0x7f9bf00ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:22.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.656+0000 7f9bf614f640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bf00cd270 con 0x7f9bf0162e10 2024-04-24T23:11:22.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.656+0000 7f9bf614f640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bf0071610 con 0x7f9bf00b5af0 2024-04-24T23:11:22.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.656+0000 7f9bf614f640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f9bf00b5660 con 0x7f9bf0162320 2024-04-24T23:11:22.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.656+0000 7f9bef7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9bf0162320 0x7f9bf00ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:22.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.656+0000 7f9beffff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9bf0162e10 0x7f9bf015ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56336/0 (socket says 172.21.15.32:56336) 2024-04-24T23:11:22.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.656+0000 7f9beffff640 1 -- 172.21.15.32:0/1517726428 learned_addr learned my addr 172.21.15.32:0/1517726428 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:11:22.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.656+0000 7f9bef7fe640 1 -- 172.21.15.32:0/1517726428 >> v1:172.21.15.121:6789/0 conn(0x7f9bf00b5af0 legacy=0x7f9bf0162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:22.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.656+0000 7f9bef7fe640 1 -- 172.21.15.32:0/1517726428 >> v1:172.21.15.32:6789/0 conn(0x7f9bf0162e10 legacy=0x7f9bf015ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:22.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.657+0000 7f9bee7fc640 1 -- 172.21.15.32:0/1517726428 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3613662175 0 0) 0x7f9bf00cd270 con 0x7f9bf0162e10 2024-04-24T23:11:22.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.657+0000 7f9bef7fe640 1 -- 172.21.15.32:0/1517726428 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf00718f0 con 0x7f9bf0162320 2024-04-24T23:11:22.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.657+0000 7f9bef7fe640 1 --2- 172.21.15.32:0/1517726428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bf0162320 0x7f9bf00ccc90 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f9be00097a0 tx=0x7f9be002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=158dfa4a621ed0cc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:22.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.657+0000 7f9bee7fc640 1 -- 172.21.15.32:0/1517726428 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be0037070 con 0x7f9bf0162320 2024-04-24T23:11:22.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.657+0000 7f9bee7fc640 1 -- 172.21.15.32:0/1517726428 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9be0032440 con 0x7f9bf0162320 2024-04-24T23:11:22.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.657+0000 7f9bee7fc640 1 -- 172.21.15.32:0/1517726428 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be0032810 con 0x7f9bf0162320 2024-04-24T23:11:22.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.658+0000 7f9bf614f640 1 -- 172.21.15.32:0/1517726428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bf0162320 msgr2=0x7f9bf00ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:22.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.658+0000 7f9bf614f640 1 --2- 172.21.15.32:0/1517726428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bf0162320 0x7f9bf00ccc90 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f9be00097a0 tx=0x7f9be002a1a0 comp rx=0 tx=0).stop 2024-04-24T23:11:22.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.658+0000 7f9bf614f640 1 -- 172.21.15.32:0/1517726428 shutdown_connections 2024-04-24T23:11:22.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.658+0000 7f9bf614f640 1 --2- 172.21.15.32:0/1517726428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bf0162320 0x7f9bf00ccc90 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:22.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.658+0000 7f9bf614f640 1 -- 172.21.15.32:0/1517726428 >> 172.21.15.32:0/1517726428 conn(0x7f9bf00d0ae0 msgr2=0x7f9bf00d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:22.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.658+0000 7f9bf614f640 1 -- 172.21.15.32:0/1517726428 shutdown_connections 2024-04-24T23:11:22.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.658+0000 7f9bf614f640 1 -- 172.21.15.32:0/1517726428 wait complete. 2024-04-24T23:11:22.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.659+0000 7f9bf614f640 1 Processor -- start 2024-04-24T23:11:22.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.659+0000 7f9bf614f640 1 -- start start 2024-04-24T23:11:22.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.660+0000 7f9bf614f640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9bf00b5af0 0x7f9bf00cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:22.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.660+0000 7f9bf614f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bf0162320 0x7f9bf006e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:22.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.660+0000 7f9bf614f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bf0162e10 0x7f9bf0157ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:22.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.660+0000 7f9bf614f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf0071bd0 con 0x7f9bf0162e10 2024-04-24T23:11:22.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.660+0000 7f9bf614f640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf0067950 con 0x7f9bf0162320 2024-04-24T23:11:22.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.660+0000 7f9bf614f640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f9bf0071d50 con 0x7f9bf00b5af0 2024-04-24T23:11:22.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.660+0000 7f9beffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bf0162e10 0x7f9bf0157ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:22.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.660+0000 7f9beffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bf0162e10 0x7f9bf0157ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54450/0 (socket says 172.21.15.32:54450) 2024-04-24T23:11:22.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.660+0000 7f9beffff640 1 -- 172.21.15.32:0/3935068364 learned_addr learned my addr 172.21.15.32:0/3935068364 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:22.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.660+0000 7f9bef7fe640 1 --2- 172.21.15.32:0/3935068364 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9bf00b5af0 0x7f9bf00cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:22.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.660+0000 7f9beeffd640 1 --2- 172.21.15.32:0/3935068364 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bf0162320 0x7f9bf006e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:22.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.661+0000 7f9beffff640 1 -- 172.21.15.32:0/3935068364 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9bf00b5af0 msgr2=0x7f9bf00cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:22.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.661+0000 7f9beffff640 1 --2- 172.21.15.32:0/3935068364 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9bf00b5af0 0x7f9bf00cd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:22.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.661+0000 7f9beffff640 1 -- 172.21.15.32:0/3935068364 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bf0162320 msgr2=0x7f9bf006e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:22.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.661+0000 7f9beffff640 1 --2- 172.21.15.32:0/3935068364 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bf0162320 0x7f9bf006e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:22.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.661+0000 7f9beffff640 1 -- 172.21.15.32:0/3935068364 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf00cdad0 con 0x7f9bf0162e10 2024-04-24T23:11:22.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.661+0000 7f9beffff640 1 --2- 172.21.15.32:0/3935068364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bf0162e10 0x7f9bf0157ec0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f9be800c700 tx=0x7f9be800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:22.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.661+0000 7f9bd7fff640 1 -- 172.21.15.32:0/3935068364 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be8018070 con 0x7f9bf0162e10 2024-04-24T23:11:22.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.661+0000 7f9bd7fff640 1 -- 172.21.15.32:0/3935068364 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9be800ce10 con 0x7f9bf0162e10 2024-04-24T23:11:22.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.661+0000 7f9bf614f640 1 -- 172.21.15.32:0/3935068364 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bf00cddb0 con 0x7f9bf0162e10 2024-04-24T23:11:22.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.661+0000 7f9bd7fff640 1 -- 172.21.15.32:0/3935068364 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be800a860 con 0x7f9bf0162e10 2024-04-24T23:11:22.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.661+0000 7f9bf614f640 1 -- 172.21.15.32:0/3935068364 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bf00ce650 con 0x7f9bf0162e10 2024-04-24T23:11:22.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.662+0000 7f9bf614f640 1 -- 172.21.15.32:0/3935068364 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bf00ce1a0 con 0x7f9bf0162e10 2024-04-24T23:11:22.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.663+0000 7f9bd7fff640 1 -- 172.21.15.32:0/3935068364 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9be801b070 con 0x7f9bf0162e10 2024-04-24T23:11:22.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.663+0000 7f9bd7fff640 1 --2- 172.21.15.32:0/3935068364 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9bdc06be30 0x7f9bdc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:22.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.663+0000 7f9bd7fff640 1 -- 172.21.15.32:0/3935068364 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 4943+0+0 (secure 0 0 0) 0x7f9be808a040 con 0x7f9bf0162e10 2024-04-24T23:11:22.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.663+0000 7f9bef7fe640 1 --2- 172.21.15.32:0/3935068364 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9bdc06be30 0x7f9bdc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:22.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.664+0000 7f9bef7fe640 1 --2- 172.21.15.32:0/3935068364 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9bdc06be30 0x7f9bdc06e2f0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f9be0034000 tx=0x7f9be002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:22.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.667+0000 7f9bd7fff640 1 -- 172.21.15.32:0/3935068364 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9be8058730 con 0x7f9bf0162e10 2024-04-24T23:11:22.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.848+0000 7f9bf614f640 1 -- 172.21.15.32:0/3935068364 --> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9bf0068430 con 0x7f9bdc06be30 2024-04-24T23:11:22.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.850+0000 7f9bd7fff640 1 -- 172.21.15.32:0/3935068364 <== mgr.4108 v2:172.21.15.121:6832/1848157943 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+12600 (secure 0 0 0) 0x7f9bf0068430 con 0x7f9bdc06be30 2024-04-24T23:11:22.851 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:11:22.852 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-24T23:11:22.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.852+0000 7f9bf614f640 1 -- 172.21.15.32:0/3935068364 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9bdc06be30 msgr2=0x7f9bdc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:22.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.852+0000 7f9bf614f640 1 --2- 172.21.15.32:0/3935068364 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9bdc06be30 0x7f9bdc06e2f0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f9be0034000 tx=0x7f9be002e000 comp rx=0 tx=0).stop 2024-04-24T23:11:22.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.852+0000 7f9bf614f640 1 -- 172.21.15.32:0/3935068364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bf0162e10 msgr2=0x7f9bf0157ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:22.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.852+0000 7f9bf614f640 1 --2- 172.21.15.32:0/3935068364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bf0162e10 0x7f9bf0157ec0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f9be800c700 tx=0x7f9be800cbc0 comp rx=0 tx=0).stop 2024-04-24T23:11:22.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.852+0000 7f9bf614f640 1 -- 172.21.15.32:0/3935068364 shutdown_connections 2024-04-24T23:11:22.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.852+0000 7f9bf614f640 1 --2- 172.21.15.32:0/3935068364 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9bdc06be30 0x7f9bdc06e2f0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:22.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.852+0000 7f9bf614f640 1 --2- 172.21.15.32:0/3935068364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bf0162e10 0x7f9bf0157ec0 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:22.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.852+0000 7f9bf614f640 1 --2- 172.21.15.32:0/3935068364 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bf0162320 0x7f9bf006e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:22.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.852+0000 7f9bf614f640 1 --2- 172.21.15.32:0/3935068364 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9bf00b5af0 0x7f9bf00cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:22.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.852+0000 7f9bf614f640 1 -- 172.21.15.32:0/3935068364 >> 172.21.15.32:0/3935068364 conn(0x7f9bf00d0ae0 msgr2=0x7f9bf015dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:22.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.852+0000 7f9bf614f640 1 -- 172.21.15.32:0/3935068364 shutdown_connections 2024-04-24T23:11:22.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.852+0000 7f9bf614f640 1 -- 172.21.15.32:0/3935068364 wait complete. 2024-04-24T23:11:22.864 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":560,"stamp":"2024-04-24T23:11:21.304513+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":112073,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1450426,"statfs":{"total":1600000000,"available":1485237048,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-52491310,"num_objects":-36,"num_object_clones":0,"num_object_copies":-72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-36,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-131,"num_write_kb":-85088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.439471"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":509,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-04-24T23:11:16.866349+0000","last_change":"2024-04-24T22:58:55.138810+0000","last_active":"2024-04-24T23:11:16.866349+0000","last_peered":"2024-04-24T23:11:16.866349+0000","last_clean":"2024-04-24T23:11:16.866349+0000","last_became_active":"2024-04-24T22:58:55.138627+0000","last_became_peered":"2024-04-24T22:58:55.138627+0000","last_unstale":"2024-04-24T23:11:16.866349+0000","last_undegraded":"2024-04-24T23:11:16.866349+0000","last_fullsized":"2024-04-24T23:11:16.866349+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_clean_scrub_stamp":"2024-04-24T22:58:54.127235+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:51.383182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574999,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":5144,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":190168,"statfs":{"total":200000000,"available":194732364,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574999,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":17432,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":177880,"statfs":{"total":200000000,"available":182149216,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574999,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":22105,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":173207,"statfs":{"total":200000000,"available":177364108,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574999,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9240,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":186072,"statfs":{"total":200000000,"available":190538026,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574999,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":11354,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":183957,"statfs":{"total":200000000,"available":188372906,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574999,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":5144,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":190167,"statfs":{"total":200000000,"available":194731580,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574999,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":22165,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":173147,"statfs":{"total":200000000,"available":177302947,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574999,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":19486,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":175826,"statfs":{"total":200000000,"available":180045901,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-24T23:11:22.865 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-24T23:11:22.865 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:11:22.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.964+0000 7f5f9b610640 1 Processor -- start 2024-04-24T23:11:22.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.964+0000 7f5f9b610640 1 -- start start 2024-04-24T23:11:22.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.965+0000 7f5f9b610640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5f941476d0 0x7f5f940ce630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:22.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.965+0000 7f5f9b610640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f940afdd0 con 0x7f5f9406b820 2024-04-24T23:11:22.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.965+0000 7f5f9b610640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f940af010 con 0x7f5f940cfb70 2024-04-24T23:11:22.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.965+0000 7f5f9b610640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f5f940aedc0 con 0x7f5f941476d0 2024-04-24T23:11:22.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.965+0000 7f5f99bad640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5f9406b820 0x7f5f9406d5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56340/0 (socket says 172.21.15.32:56340) 2024-04-24T23:11:22.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.965+0000 7f5f99bad640 1 -- 172.21.15.32:0/3154765802 learned_addr learned my addr 172.21.15.32:0/3154765802 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:11:22.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.965+0000 7f5f993ac640 1 --2- 172.21.15.32:0/3154765802 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5f941476d0 0x7f5f940ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:22.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.965+0000 7f5f8bfff640 1 -- 172.21.15.32:0/3154765802 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3270183784 0 0) 0x7f5f940afdd0 con 0x7f5f9406b820 2024-04-24T23:11:22.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.965+0000 7f5f8bfff640 1 -- 172.21.15.32:0/3154765802 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f80003520 con 0x7f5f9406b820 2024-04-24T23:11:22.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.965+0000 7f5f8bfff640 1 -- 172.21.15.32:0/3154765802 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2793319603 0 0) 0x7f5f940af010 con 0x7f5f940cfb70 2024-04-24T23:11:22.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.965+0000 7f5f8bfff640 1 -- 172.21.15.32:0/3154765802 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f940afdd0 con 0x7f5f940cfb70 2024-04-24T23:11:22.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.965+0000 7f5f8bfff640 1 -- 172.21.15.32:0/3154765802 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3304317652 0 0) 0x7f5f80003520 con 0x7f5f9406b820 2024-04-24T23:11:22.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.965+0000 7f5f8bfff640 1 -- 172.21.15.32:0/3154765802 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5f941476d0 msgr2=0x7f5f940ce630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:22.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.965+0000 7f5f8bfff640 1 --2- 172.21.15.32:0/3154765802 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5f941476d0 0x7f5f940ce630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:22.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.965+0000 7f5f8bfff640 1 -- 172.21.15.32:0/3154765802 >> v1:172.21.15.121:6789/0 conn(0x7f5f940cfb70 legacy=0x7f5f9406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:22.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.965+0000 7f5f8bfff640 1 -- 172.21.15.32:0/3154765802 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f940af2f0 con 0x7f5f9406b820 2024-04-24T23:11:22.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.965+0000 7f5f8bfff640 1 -- 172.21.15.32:0/3154765802 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f5f8c0019f0 con 0x7f5f9406b820 2024-04-24T23:11:22.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.965+0000 7f5f8bfff640 1 -- 172.21.15.32:0/3154765802 >> v1:172.21.15.32:6789/0 conn(0x7f5f9406b820 legacy=0x7f5f9406d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:22.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.965+0000 7f5f8bfff640 1 --2- 172.21.15.32:0/3154765802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f80003b70 0x7f5f80003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:22.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.965+0000 7f5f8bfff640 1 --2- 172.21.15.32:0/3154765802 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f80004800 0x7f5f80004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:22.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.966+0000 7f5f8bfff640 1 --2- 172.21.15.32:0/3154765802 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5f941476d0 0x7f5f80005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:22.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.966+0000 7f5f8bfff640 1 -- 172.21.15.32:0/3154765802 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5f940aedc0 con 0x7f5f80003b70 2024-04-24T23:11:22.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.966+0000 7f5f8bfff640 1 -- 172.21.15.32:0/3154765802 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8000bb50 con 0x7f5f80004800 2024-04-24T23:11:22.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.966+0000 7f5f8bfff640 1 -- 172.21.15.32:0/3154765802 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f5f8000bd20 con 0x7f5f941476d0 2024-04-24T23:11:22.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.966+0000 7f5f993ac640 1 --2- 172.21.15.32:0/3154765802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f80003b70 0x7f5f80003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:22.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.966+0000 7f5f98bab640 1 --2- 172.21.15.32:0/3154765802 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f80004800 0x7f5f80004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:22.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.966+0000 7f5f99bad640 1 --2- 172.21.15.32:0/3154765802 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5f941476d0 0x7f5f80005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:22.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.966+0000 7f5f993ac640 1 -- 172.21.15.32:0/3154765802 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5f941476d0 msgr2=0x7f5f80005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:22.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.966+0000 7f5f993ac640 1 --2- 172.21.15.32:0/3154765802 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5f941476d0 0x7f5f80005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:22.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.966+0000 7f5f993ac640 1 -- 172.21.15.32:0/3154765802 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f80004800 msgr2=0x7f5f80004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:22.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.966+0000 7f5f993ac640 1 --2- 172.21.15.32:0/3154765802 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f80004800 0x7f5f80004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:22.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.966+0000 7f5f993ac640 1 -- 172.21.15.32:0/3154765802 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f80003520 con 0x7f5f80003b70 2024-04-24T23:11:22.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.966+0000 7f5f99bad640 1 --2- 172.21.15.32:0/3154765802 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5f941476d0 0x7f5f80005220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:11:22.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.967+0000 7f5f993ac640 1 --2- 172.21.15.32:0/3154765802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f80003b70 0x7f5f80003f70 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f5f90009af0 tx=0x7f5f80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82e18796bd9586cc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:22.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.967+0000 7f5f8bfff640 1 -- 172.21.15.32:0/3154765802 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f90036070 con 0x7f5f80003b70 2024-04-24T23:11:22.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.967+0000 7f5f8bfff640 1 -- 172.21.15.32:0/3154765802 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f90031400 con 0x7f5f80003b70 2024-04-24T23:11:22.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.967+0000 7f5f8bfff640 1 -- 172.21.15.32:0/3154765802 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f90009690 con 0x7f5f80003b70 2024-04-24T23:11:22.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.967+0000 7f5f9b610640 1 -- 172.21.15.32:0/3154765802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f80003b70 msgr2=0x7f5f80003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:22.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.967+0000 7f5f9b610640 1 --2- 172.21.15.32:0/3154765802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f80003b70 0x7f5f80003f70 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f5f90009af0 tx=0x7f5f80005830 comp rx=0 tx=0).stop 2024-04-24T23:11:22.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.967+0000 7f5f99bad640 1 -- 172.21.15.32:0/3154765802 reap_dead start 2024-04-24T23:11:22.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.967+0000 7f5f9b610640 1 -- 172.21.15.32:0/3154765802 shutdown_connections 2024-04-24T23:11:22.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.967+0000 7f5f9b610640 1 -- 172.21.15.32:0/3154765802 >> 172.21.15.32:0/3154765802 conn(0x7f5f940d0730 msgr2=0x7f5f940d0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:22.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.967+0000 7f5f9b610640 1 -- 172.21.15.32:0/3154765802 shutdown_connections 2024-04-24T23:11:22.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.967+0000 7f5f9b610640 1 -- 172.21.15.32:0/3154765802 wait complete. 2024-04-24T23:11:22.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.968+0000 7f5f9b610640 1 Processor -- start 2024-04-24T23:11:22.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.969+0000 7f5f9b610640 1 -- start start 2024-04-24T23:11:22.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.969+0000 7f5f9b610640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f80003b70 0x7f5f94141aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:22.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.969+0000 7f5f9b610640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f9413e610 0x7f5f9413eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:22.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.969+0000 7f5f9b610640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5f9413f070 0x7f5f941510b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:22.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.969+0000 7f5f9b610640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5f94147ab0 con 0x7f5f80003b70 2024-04-24T23:11:22.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.969+0000 7f5f9b610640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5f940af940 con 0x7f5f9413e610 2024-04-24T23:11:22.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.969+0000 7f5f9b610640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f5f94148470 con 0x7f5f9413f070 2024-04-24T23:11:22.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.969+0000 7f5f993ac640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f80003b70 0x7f5f94141aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:22.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.969+0000 7f5f993ac640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f80003b70 0x7f5f94141aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54460/0 (socket says 172.21.15.32:54460) 2024-04-24T23:11:22.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.969+0000 7f5f993ac640 1 -- 172.21.15.32:0/4205683206 learned_addr learned my addr 172.21.15.32:0/4205683206 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:22.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.969+0000 7f5f99bad640 1 --2- 172.21.15.32:0/4205683206 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5f9413f070 0x7f5f941510b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:22.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.970+0000 7f5f98bab640 1 --2- 172.21.15.32:0/4205683206 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f9413e610 0x7f5f9413eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:22.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.970+0000 7f5f993ac640 1 -- 172.21.15.32:0/4205683206 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5f9413f070 msgr2=0x7f5f941510b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:22.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.970+0000 7f5f993ac640 1 --2- 172.21.15.32:0/4205683206 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5f9413f070 0x7f5f941510b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:22.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.970+0000 7f5f993ac640 1 -- 172.21.15.32:0/4205683206 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f9413e610 msgr2=0x7f5f9413eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:22.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.970+0000 7f5f993ac640 1 --2- 172.21.15.32:0/4205683206 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f9413e610 0x7f5f9413eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:22.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.970+0000 7f5f993ac640 1 -- 172.21.15.32:0/4205683206 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f94151e20 con 0x7f5f80003b70 2024-04-24T23:11:22.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.970+0000 7f5f98bab640 1 --2- 172.21.15.32:0/4205683206 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f9413e610 0x7f5f9413eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:11:22.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.970+0000 7f5f993ac640 1 --2- 172.21.15.32:0/4205683206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f80003b70 0x7f5f94141aa0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f5f90009af0 tx=0x7f5f90004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:22.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.970+0000 7f5f89ffb640 1 -- 172.21.15.32:0/4205683206 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f90040020 con 0x7f5f80003b70 2024-04-24T23:11:22.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.970+0000 7f5f89ffb640 1 -- 172.21.15.32:0/4205683206 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f90003b40 con 0x7f5f80003b70 2024-04-24T23:11:22.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.970+0000 7f5f9b610640 1 -- 172.21.15.32:0/4205683206 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f94140ce0 con 0x7f5f80003b70 2024-04-24T23:11:22.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.970+0000 7f5f89ffb640 1 -- 172.21.15.32:0/4205683206 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f90036040 con 0x7f5f80003b70 2024-04-24T23:11:22.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.970+0000 7f5f9b610640 1 -- 172.21.15.32:0/4205683206 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f941412a0 con 0x7f5f80003b70 2024-04-24T23:11:22.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.971+0000 7f5f9b610640 1 -- 172.21.15.32:0/4205683206 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f94068430 con 0x7f5f80003b70 2024-04-24T23:11:22.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.972+0000 7f5f89ffb640 1 -- 172.21.15.32:0/4205683206 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5f90043020 con 0x7f5f80003b70 2024-04-24T23:11:22.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.972+0000 7f5f89ffb640 1 --2- 172.21.15.32:0/4205683206 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5f7806be30 0x7f5f7806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:22.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.972+0000 7f5f89ffb640 1 -- 172.21.15.32:0/4205683206 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 4943+0+0 (secure 0 0 0) 0x7f5f900aad90 con 0x7f5f80003b70 2024-04-24T23:11:22.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.972+0000 7f5f98bab640 1 --2- 172.21.15.32:0/4205683206 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5f7806be30 0x7f5f7806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:22.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.973+0000 7f5f98bab640 1 --2- 172.21.15.32:0/4205683206 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5f7806be30 0x7f5f7806e2f0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f5f9415eb50 tx=0x7f5f84009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:22.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:22.976+0000 7f5f89ffb640 1 -- 172.21.15.32:0/4205683206 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5f9004a070 con 0x7f5f80003b70 2024-04-24T23:11:23.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.154+0000 7f5f9b610640 1 -- 172.21.15.32:0/4205683206 --> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5f9413f450 con 0x7f5f7806be30 2024-04-24T23:11:23.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.156+0000 7f5f89ffb640 1 -- 172.21.15.32:0/4205683206 <== mgr.4108 v2:172.21.15.121:6832/1848157943 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+12600 (secure 0 0 0) 0x7f5f9413f450 con 0x7f5f7806be30 2024-04-24T23:11:23.156 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:11:23.157 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-24T23:11:23.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.157+0000 7f5f9b610640 1 -- 172.21.15.32:0/4205683206 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5f7806be30 msgr2=0x7f5f7806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:23.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.157+0000 7f5f9b610640 1 --2- 172.21.15.32:0/4205683206 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5f7806be30 0x7f5f7806e2f0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f5f9415eb50 tx=0x7f5f84009040 comp rx=0 tx=0).stop 2024-04-24T23:11:23.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.157+0000 7f5f9b610640 1 -- 172.21.15.32:0/4205683206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f80003b70 msgr2=0x7f5f94141aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:23.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.157+0000 7f5f9b610640 1 --2- 172.21.15.32:0/4205683206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f80003b70 0x7f5f94141aa0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f5f90009af0 tx=0x7f5f90004730 comp rx=0 tx=0).stop 2024-04-24T23:11:23.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.157+0000 7f5f9b610640 1 -- 172.21.15.32:0/4205683206 shutdown_connections 2024-04-24T23:11:23.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.157+0000 7f5f9b610640 1 --2- 172.21.15.32:0/4205683206 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f5f7806be30 0x7f5f7806e2f0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:23.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.157+0000 7f5f9b610640 1 --2- 172.21.15.32:0/4205683206 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5f9413f070 0x7f5f941510b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:23.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.157+0000 7f5f9b610640 1 --2- 172.21.15.32:0/4205683206 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f9413e610 0x7f5f9413eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:23.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.157+0000 7f5f9b610640 1 --2- 172.21.15.32:0/4205683206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f80003b70 0x7f5f94141aa0 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:23.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.157+0000 7f5f9b610640 1 -- 172.21.15.32:0/4205683206 >> 172.21.15.32:0/4205683206 conn(0x7f5f940d0730 msgr2=0x7f5f940cef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:23.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.157+0000 7f5f9b610640 1 -- 172.21.15.32:0/4205683206 shutdown_connections 2024-04-24T23:11:23.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.157+0000 7f5f9b610640 1 -- 172.21.15.32:0/4205683206 wait complete. 2024-04-24T23:11:23.169 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":560,"stamp":"2024-04-24T23:11:21.304513+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":112073,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1450426,"statfs":{"total":1600000000,"available":1485237048,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-52491310,"num_objects":-36,"num_object_clones":0,"num_object_copies":-72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-36,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-131,"num_write_kb":-85088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.439471"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":509,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-04-24T23:11:16.866349+0000","last_change":"2024-04-24T22:58:55.138810+0000","last_active":"2024-04-24T23:11:16.866349+0000","last_peered":"2024-04-24T23:11:16.866349+0000","last_clean":"2024-04-24T23:11:16.866349+0000","last_became_active":"2024-04-24T22:58:55.138627+0000","last_became_peered":"2024-04-24T22:58:55.138627+0000","last_unstale":"2024-04-24T23:11:16.866349+0000","last_undegraded":"2024-04-24T23:11:16.866349+0000","last_fullsized":"2024-04-24T23:11:16.866349+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:58:54.127235+0000","last_clean_scrub_stamp":"2024-04-24T22:58:54.127235+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:51.383182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574999,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":5144,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":190168,"statfs":{"total":200000000,"available":194732364,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574999,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":17432,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":177880,"statfs":{"total":200000000,"available":182149216,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574999,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":22105,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":173207,"statfs":{"total":200000000,"available":177364108,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574999,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9240,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":186072,"statfs":{"total":200000000,"available":190538026,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574999,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":11354,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":183957,"statfs":{"total":200000000,"available":188372906,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574999,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":5144,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":190167,"statfs":{"total":200000000,"available":194731580,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574999,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":22165,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":173147,"statfs":{"total":200000000,"available":177302947,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574999,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":19486,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":175826,"statfs":{"total":200000000,"available":180045901,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-24T23:11:23.170 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:11:23.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.269+0000 7fb17ad14640 1 Processor -- start 2024-04-24T23:11:23.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.269+0000 7fb17ad14640 1 -- start start 2024-04-24T23:11:23.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.270+0000 7fb17ad14640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb17415edd0 0x7fb1740b5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:23.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.270+0000 7fb17ad14640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb1740cd3c0 0x7fb1740b5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:23.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.270+0000 7fb17ad14640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1740cd910 con 0x7fb1740b5560 2024-04-24T23:11:23.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.270+0000 7fb17ad14640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fb1740cdbf0 con 0x7fb1740cd3c0 2024-04-24T23:11:23.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.270+0000 7fb17ad14640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fb1740cded0 con 0x7fb17415edd0 2024-04-24T23:11:23.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.271+0000 7fb173fff640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb17415edd0 0x7fb1740b5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:23.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.271+0000 7fb1792b1640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb1740cd3c0 0x7fb1740b5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:23.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.271+0000 7fb178ab0640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fb1740b5560 0x7fb17415e6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:57878/0 (socket says 172.21.15.32:57878) 2024-04-24T23:11:23.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.271+0000 7fb178ab0640 1 -- 172.21.15.32:0/1428194561 learned_addr learned my addr 172.21.15.32:0/1428194561 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:11:23.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.271+0000 7fb1792b1640 1 -- 172.21.15.32:0/1428194561 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb17415edd0 msgr2=0x7fb1740b5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:23.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.271+0000 7fb1792b1640 1 --2- 172.21.15.32:0/1428194561 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb17415edd0 0x7fb1740b5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:23.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.271+0000 7fb1792b1640 1 -- 172.21.15.32:0/1428194561 >> v1:172.21.15.121:6789/0 conn(0x7fb1740b5560 legacy=0x7fb17415e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:23.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.271+0000 7fb1737fe640 1 -- 172.21.15.32:0/1428194561 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 461221187 0 0) 0x7fb1740cd910 con 0x7fb1740b5560 2024-04-24T23:11:23.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.271+0000 7fb1792b1640 1 -- 172.21.15.32:0/1428194561 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1740ce1b0 con 0x7fb1740cd3c0 2024-04-24T23:11:23.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.272+0000 7fb1792b1640 1 --2- 172.21.15.32:0/1428194561 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1740cd3c0 0x7fb1740b5e80 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fb15c0097a0 tx=0x7fb15c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf8ba64ef47be537 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:23.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.272+0000 7fb1737fe640 1 -- 172.21.15.32:0/1428194561 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c037070 con 0x7fb1740cd3c0 2024-04-24T23:11:23.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.272+0000 7fb1737fe640 1 -- 172.21.15.32:0/1428194561 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb15c032440 con 0x7fb1740cd3c0 2024-04-24T23:11:23.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.272+0000 7fb1737fe640 1 -- 172.21.15.32:0/1428194561 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c032810 con 0x7fb1740cd3c0 2024-04-24T23:11:23.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.273+0000 7fb17ad14640 1 -- 172.21.15.32:0/1428194561 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1740cd3c0 msgr2=0x7fb1740b5e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:23.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.273+0000 7fb17ad14640 1 --2- 172.21.15.32:0/1428194561 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1740cd3c0 0x7fb1740b5e80 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fb15c0097a0 tx=0x7fb15c02a180 comp rx=0 tx=0).stop 2024-04-24T23:11:23.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.273+0000 7fb17ad14640 1 -- 172.21.15.32:0/1428194561 shutdown_connections 2024-04-24T23:11:23.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.273+0000 7fb17ad14640 1 --2- 172.21.15.32:0/1428194561 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1740cd3c0 0x7fb1740b5e80 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:23.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.273+0000 7fb17ad14640 1 --2- 172.21.15.32:0/1428194561 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb17415edd0 0x7fb1740b5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:23.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.273+0000 7fb17ad14640 1 -- 172.21.15.32:0/1428194561 >> 172.21.15.32:0/1428194561 conn(0x7fb17406b110 msgr2=0x7fb17406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:23.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.273+0000 7fb17ad14640 1 -- 172.21.15.32:0/1428194561 shutdown_connections 2024-04-24T23:11:23.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.273+0000 7fb17ad14640 1 -- 172.21.15.32:0/1428194561 wait complete. 2024-04-24T23:11:23.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.274+0000 7fb17ad14640 1 Processor -- start 2024-04-24T23:11:23.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.274+0000 7fb17ad14640 1 -- start start 2024-04-24T23:11:23.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.275+0000 7fb17ad14640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1740b5560 0x7fb174156130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:23.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.275+0000 7fb17ad14640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb1740cd3c0 0x7fb17414ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:23.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.275+0000 7fb17ad14640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb17415edd0 0x7fb1741f47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:23.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.275+0000 7fb17ad14640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb17406c5f0 con 0x7fb1740b5560 2024-04-24T23:11:23.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.275+0000 7fb17ad14640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb1740b63c0 con 0x7fb17415edd0 2024-04-24T23:11:23.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.275+0000 7fb17ad14640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fb17406c770 con 0x7fb1740cd3c0 2024-04-24T23:11:23.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.275+0000 7fb178ab0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1740b5560 0x7fb174156130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:23.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.275+0000 7fb178ab0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1740b5560 0x7fb174156130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54476/0 (socket says 172.21.15.32:54476) 2024-04-24T23:11:23.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.275+0000 7fb178ab0640 1 -- 172.21.15.32:0/4054378492 learned_addr learned my addr 172.21.15.32:0/4054378492 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:23.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.275+0000 7fb173fff640 1 --2- 172.21.15.32:0/4054378492 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb1740cd3c0 0x7fb17414ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:23.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.275+0000 7fb1792b1640 1 --2- 172.21.15.32:0/4054378492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb17415edd0 0x7fb1741f47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:23.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.275+0000 7fb178ab0640 1 -- 172.21.15.32:0/4054378492 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb1740cd3c0 msgr2=0x7fb17414ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:23.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.275+0000 7fb178ab0640 1 --2- 172.21.15.32:0/4054378492 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb1740cd3c0 0x7fb17414ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:23.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.275+0000 7fb178ab0640 1 -- 172.21.15.32:0/4054378492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb17415edd0 msgr2=0x7fb1741f47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:23.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.275+0000 7fb178ab0640 1 --2- 172.21.15.32:0/4054378492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb17415edd0 0x7fb1741f47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:23.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.275+0000 7fb178ab0640 1 -- 172.21.15.32:0/4054378492 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb174158cf0 con 0x7fb1740b5560 2024-04-24T23:11:23.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.275+0000 7fb178ab0640 1 --2- 172.21.15.32:0/4054378492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1740b5560 0x7fb174156130 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fb16c00c8c0 tx=0x7fb16c00cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:23.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.276+0000 7fb1717fa640 1 -- 172.21.15.32:0/4054378492 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16c018070 con 0x7fb1740b5560 2024-04-24T23:11:23.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.276+0000 7fb1717fa640 1 -- 172.21.15.32:0/4054378492 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb16c00a640 con 0x7fb1740b5560 2024-04-24T23:11:23.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.276+0000 7fb17ad14640 1 -- 172.21.15.32:0/4054378492 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1741f5040 con 0x7fb1740b5560 2024-04-24T23:11:23.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.276+0000 7fb1717fa640 1 -- 172.21.15.32:0/4054378492 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16c00aa30 con 0x7fb1740b5560 2024-04-24T23:11:23.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.276+0000 7fb17ad14640 1 -- 172.21.15.32:0/4054378492 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1741f5600 con 0x7fb1740b5560 2024-04-24T23:11:23.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.277+0000 7fb17ad14640 1 -- 172.21.15.32:0/4054378492 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb174068430 con 0x7fb1740b5560 2024-04-24T23:11:23.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.277+0000 7fb1717fa640 1 -- 172.21.15.32:0/4054378492 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb16c00abd0 con 0x7fb1740b5560 2024-04-24T23:11:23.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.278+0000 7fb1717fa640 1 --2- 172.21.15.32:0/4054378492 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb15406be30 0x7fb15406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:23.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.278+0000 7fb1717fa640 1 -- 172.21.15.32:0/4054378492 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 4943+0+0 (secure 0 0 0) 0x7fb16c08a340 con 0x7fb1740b5560 2024-04-24T23:11:23.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.278+0000 7fb173fff640 1 --2- 172.21.15.32:0/4054378492 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb15406be30 0x7fb15406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:23.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.278+0000 7fb173fff640 1 --2- 172.21.15.32:0/4054378492 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb15406be30 0x7fb15406e2f0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fb1741f2370 tx=0x7fb16400a2a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:23.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.282+0000 7fb1717fa640 1 -- 172.21.15.32:0/4054378492 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb16c058880 con 0x7fb1740b5560 2024-04-24T23:11:23.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.468+0000 7fb17ad14640 1 -- 172.21.15.32:0/4054378492 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb17406d5f0 con 0x7fb1740b5560 2024-04-24T23:11:23.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.469+0000 7fb1717fa640 1 -- 172.21.15.32:0/4054378492 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v194) ==== 74+0+13041 (secure 0 0 0) 0x7fb16c05cb80 con 0x7fb1740b5560 2024-04-24T23:11:23.470 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:11:23.470 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":194,"fsid":"eb1b7a01-6f65-4534-9a12-83011862ba51","created":"2024-04-24T22:58:46.964851+0000","modified":"2024-04-24T23:11:20.352022+0000","last_up_change":"2024-04-24T22:58:52.048787+0000","last_in_change":"2024-04-24T22:58:49.045609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:58:53.171408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6801","nonce":1479395426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6803","nonce":1479395426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6807","nonce":1479395426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1479395426},{"type":"v1","addr":"172.21.15.32:6805","nonce":1479395426}]},"public_addr":"172.21.15.32:6801/1479395426","cluster_addr":"172.21.15.32:6803/1479395426","heartbeat_back_addr":"172.21.15.32:6807/1479395426","heartbeat_front_addr":"172.21.15.32:6805/1479395426","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6809","nonce":1340633718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6811","nonce":1340633718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6815","nonce":1340633718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1340633718},{"type":"v1","addr":"172.21.15.32:6813","nonce":1340633718}]},"public_addr":"172.21.15.32:6809/1340633718","cluster_addr":"172.21.15.32:6811/1340633718","heartbeat_back_addr":"172.21.15.32:6815/1340633718","heartbeat_front_addr":"172.21.15.32:6813/1340633718","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6817","nonce":3405899334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6820","nonce":3405899334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6828","nonce":3405899334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3405899334},{"type":"v1","addr":"172.21.15.32:6824","nonce":3405899334}]},"public_addr":"172.21.15.32:6817/3405899334","cluster_addr":"172.21.15.32:6820/3405899334","heartbeat_back_addr":"172.21.15.32:6828/3405899334","heartbeat_front_addr":"172.21.15.32:6824/3405899334","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6821","nonce":3381057190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6825","nonce":3381057190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6831","nonce":3381057190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":3381057190},{"type":"v1","addr":"172.21.15.32:6829","nonce":3381057190}]},"public_addr":"172.21.15.32:6821/3381057190","cluster_addr":"172.21.15.32:6825/3381057190","heartbeat_back_addr":"172.21.15.32:6831/3381057190","heartbeat_front_addr":"172.21.15.32:6829/3381057190","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6801","nonce":4010553979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6803","nonce":4010553979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6807","nonce":4010553979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4010553979},{"type":"v1","addr":"172.21.15.121:6805","nonce":4010553979}]},"public_addr":"172.21.15.121:6801/4010553979","cluster_addr":"172.21.15.121:6803/4010553979","heartbeat_back_addr":"172.21.15.121:6807/4010553979","heartbeat_front_addr":"172.21.15.121:6805/4010553979","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6809","nonce":3794084091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6811","nonce":3794084091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6815","nonce":3794084091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3794084091},{"type":"v1","addr":"172.21.15.121:6813","nonce":3794084091}]},"public_addr":"172.21.15.121:6809/3794084091","cluster_addr":"172.21.15.121:6811/3794084091","heartbeat_back_addr":"172.21.15.121:6815/3794084091","heartbeat_front_addr":"172.21.15.121:6813/3794084091","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6825","nonce":1348640258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6827","nonce":1348640258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6831","nonce":1348640258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1348640258},{"type":"v1","addr":"172.21.15.121:6829","nonce":1348640258}]},"public_addr":"172.21.15.121:6825/1348640258","cluster_addr":"172.21.15.121:6827/1348640258","heartbeat_back_addr":"172.21.15.121:6831/1348640258","heartbeat_front_addr":"172.21.15.121:6829/1348640258","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6817","nonce":3250005122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6819","nonce":3250005122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6824","nonce":3250005122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3250005122},{"type":"v1","addr":"172.21.15.121:6821","nonce":3250005122}]},"public_addr":"172.21.15.121:6817/3250005122","cluster_addr":"172.21.15.121:6819/3250005122","heartbeat_back_addr":"172.21.15.121:6824/3250005122","heartbeat_front_addr":"172.21.15.121:6821/3250005122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.156184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240774+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.240949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.171672+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.220379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.204963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.239748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:58:50.154104+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6835/901353412":"2024-04-25T23:11:17.344184+0000","172.21.15.32:6833/901353412":"2024-04-25T23:11:17.344184+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:11:23.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.471+0000 7fb17ad14640 1 -- 172.21.15.32:0/4054378492 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb15406be30 msgr2=0x7fb15406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:23.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.471+0000 7fb17ad14640 1 --2- 172.21.15.32:0/4054378492 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb15406be30 0x7fb15406e2f0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fb1741f2370 tx=0x7fb16400a2a0 comp rx=0 tx=0).stop 2024-04-24T23:11:23.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.471+0000 7fb17ad14640 1 -- 172.21.15.32:0/4054378492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1740b5560 msgr2=0x7fb174156130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:23.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.471+0000 7fb17ad14640 1 --2- 172.21.15.32:0/4054378492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1740b5560 0x7fb174156130 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fb16c00c8c0 tx=0x7fb16c00cd80 comp rx=0 tx=0).stop 2024-04-24T23:11:23.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.471+0000 7fb17ad14640 1 -- 172.21.15.32:0/4054378492 shutdown_connections 2024-04-24T23:11:23.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.471+0000 7fb17ad14640 1 --2- 172.21.15.32:0/4054378492 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb15406be30 0x7fb15406e2f0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:23.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.471+0000 7fb17ad14640 1 --2- 172.21.15.32:0/4054378492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb17415edd0 0x7fb1741f47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:23.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.471+0000 7fb17ad14640 1 --2- 172.21.15.32:0/4054378492 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb1740cd3c0 0x7fb17414ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:23.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.471+0000 7fb17ad14640 1 --2- 172.21.15.32:0/4054378492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1740b5560 0x7fb174156130 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:23.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.471+0000 7fb17ad14640 1 -- 172.21.15.32:0/4054378492 >> 172.21.15.32:0/4054378492 conn(0x7fb17406b110 msgr2=0x7fb1740b5940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:23.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.471+0000 7fb17ad14640 1 -- 172.21.15.32:0/4054378492 shutdown_connections 2024-04-24T23:11:23.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:23.471+0000 7fb17ad14640 1 -- 172.21.15.32:0/4054378492 wait complete. 2024-04-24T23:11:24.484 INFO:tasks.ceph:Scrubbing osd.0 2024-04-24T23:11:24.484 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-24T23:11:24.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.600+0000 7f3c6251b640 1 Processor -- start 2024-04-24T23:11:24.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.600+0000 7f3c6251b640 1 -- start start 2024-04-24T23:11:24.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.601+0000 7f3c6251b640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3c5c15e6e0 0x7f3c5c0b9f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:24.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.601+0000 7f3c6251b640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3c5c0cfa40 0x7f3c5c0d29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:24.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.601+0000 7f3c6251b640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c5c0b6e60 con 0x7f3c5c0b3870 2024-04-24T23:11:24.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.601+0000 7f3c6251b640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f3c5c0d12e0 con 0x7f3c5c0cfa40 2024-04-24T23:11:24.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.601+0000 7f3c6251b640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f3c5c0d15c0 con 0x7f3c5c15e6e0 2024-04-24T23:11:24.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.601+0000 7f3c60ab8640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3c5c0cfa40 0x7f3c5c0d29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:24.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.601+0000 7f3c5bfff640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3c5c15e6e0 0x7f3c5c0b9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:24.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.601+0000 7f3c60ab8640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3c5c0cfa40 0x7f3c5c0d29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.32:47182/0 (socket says 172.21.15.32:47182) 2024-04-24T23:11:24.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.601+0000 7f3c5b7fe640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f3c5c0b3870 0x7f3c5c0cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:57882/0 (socket says 172.21.15.32:57882) 2024-04-24T23:11:24.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.601+0000 7f3c60ab8640 1 -- 172.21.15.32:0/3771448178 learned_addr learned my addr 172.21.15.32:0/3771448178 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:24.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.601+0000 7f3c5bfff640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3c5c15e6e0 0x7f3c5c0b9f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.32:44044/0 (socket says 172.21.15.32:44044) 2024-04-24T23:11:24.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.602+0000 7f3c60ab8640 1 -- 172.21.15.32:0/3771448178 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3c5c15e6e0 msgr2=0x7f3c5c0b9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:24.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.602+0000 7f3c60ab8640 1 --2- 172.21.15.32:0/3771448178 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3c5c15e6e0 0x7f3c5c0b9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:24.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.602+0000 7f3c60ab8640 1 -- 172.21.15.32:0/3771448178 >> v1:172.21.15.121:6789/0 conn(0x7f3c5c0b3870 legacy=0x7f3c5c0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:24.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.602+0000 7f3c60ab8640 1 -- 172.21.15.32:0/3771448178 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c5c0d18a0 con 0x7f3c5c0cfa40 2024-04-24T23:11:24.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.603+0000 7f3c60ab8640 1 --2- 172.21.15.32:0/3771448178 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c5c0cfa40 0x7f3c5c0d29c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f3c54009e50 tx=0x7f3c5402a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c614dc16eeaabf52 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:24.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.603+0000 7f3c5affd640 1 -- 172.21.15.32:0/3771448178 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c54036070 con 0x7f3c5c0cfa40 2024-04-24T23:11:24.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.603+0000 7f3c5affd640 1 -- 172.21.15.32:0/3771448178 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c54031600 con 0x7f3c5c0cfa40 2024-04-24T23:11:24.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.603+0000 7f3c5affd640 1 -- 172.21.15.32:0/3771448178 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c540319d0 con 0x7f3c5c0cfa40 2024-04-24T23:11:24.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.603+0000 7f3c6251b640 1 -- 172.21.15.32:0/3771448178 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c5c0cfa40 msgr2=0x7f3c5c0d29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:24.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.603+0000 7f3c6251b640 1 --2- 172.21.15.32:0/3771448178 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c5c0cfa40 0x7f3c5c0d29c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f3c54009e50 tx=0x7f3c5402a710 comp rx=0 tx=0).stop 2024-04-24T23:11:24.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.603+0000 7f3c6251b640 1 -- 172.21.15.32:0/3771448178 shutdown_connections 2024-04-24T23:11:24.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.603+0000 7f3c6251b640 1 --2- 172.21.15.32:0/3771448178 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c5c0cfa40 0x7f3c5c0d29c0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:24.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.603+0000 7f3c6251b640 1 --2- 172.21.15.32:0/3771448178 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3c5c15e6e0 0x7f3c5c0b9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:24.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.603+0000 7f3c6251b640 1 -- 172.21.15.32:0/3771448178 >> 172.21.15.32:0/3771448178 conn(0x7f3c5c071910 msgr2=0x7f3c5c071d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:24.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.603+0000 7f3c6251b640 1 -- 172.21.15.32:0/3771448178 shutdown_connections 2024-04-24T23:11:24.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.603+0000 7f3c6251b640 1 -- 172.21.15.32:0/3771448178 wait complete. 2024-04-24T23:11:24.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.605+0000 7f3c6251b640 1 Processor -- start 2024-04-24T23:11:24.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.605+0000 7f3c6251b640 1 -- start start 2024-04-24T23:11:24.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.606+0000 7f3c6251b640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3c5c0b3870 0x7f3c5c15ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:24.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.606+0000 7f3c6251b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c5c0cfa40 0x7f3c5c153d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:24.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.606+0000 7f3c6251b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c5c15e6e0 0x7f3c5c15ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:24.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.606+0000 7f3c6251b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5c0d0fd0 con 0x7f3c5c0cfa40 2024-04-24T23:11:24.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.606+0000 7f3c6251b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5c0ba450 con 0x7f3c5c15e6e0 2024-04-24T23:11:24.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.606+0000 7f3c6251b640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f3c5c15eac0 con 0x7f3c5c0b3870 2024-04-24T23:11:24.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.606+0000 7f3c5b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c5c0cfa40 0x7f3c5c153d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:24.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.606+0000 7f3c5bfff640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3c5c0b3870 0x7f3c5c15ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:24.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.606+0000 7f3c5b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c5c0cfa40 0x7f3c5c153d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54490/0 (socket says 172.21.15.32:54490) 2024-04-24T23:11:24.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.606+0000 7f3c5b7fe640 1 -- 172.21.15.32:0/3558680492 learned_addr learned my addr 172.21.15.32:0/3558680492 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:24.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.606+0000 7f3c60ab8640 1 --2- 172.21.15.32:0/3558680492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c5c15e6e0 0x7f3c5c15ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:24.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.606+0000 7f3c5b7fe640 1 -- 172.21.15.32:0/3558680492 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3c5c0b3870 msgr2=0x7f3c5c15ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:24.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.606+0000 7f3c5b7fe640 1 --2- 172.21.15.32:0/3558680492 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3c5c0b3870 0x7f3c5c15ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:24.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.606+0000 7f3c5b7fe640 1 -- 172.21.15.32:0/3558680492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c5c15e6e0 msgr2=0x7f3c5c15ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:24.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.606+0000 7f3c5b7fe640 1 --2- 172.21.15.32:0/3558680492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c5c15e6e0 0x7f3c5c15ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:24.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.606+0000 7f3c5b7fe640 1 -- 172.21.15.32:0/3558680492 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c5c155390 con 0x7f3c5c0cfa40 2024-04-24T23:11:24.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.606+0000 7f3c5bfff640 1 --2- 172.21.15.32:0/3558680492 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3c5c0b3870 0x7f3c5c15ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:11:24.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.606+0000 7f3c5b7fe640 1 --2- 172.21.15.32:0/3558680492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c5c0cfa40 0x7f3c5c153d10 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f3c4400c400 tx=0x7f3c4400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:24.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.606+0000 7f3c58ff9640 1 -- 172.21.15.32:0/3558680492 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c440074e0 con 0x7f3c5c0cfa40 2024-04-24T23:11:24.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.607+0000 7f3c58ff9640 1 -- 172.21.15.32:0/3558680492 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c44007680 con 0x7f3c5c0cfa40 2024-04-24T23:11:24.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.607+0000 7f3c6251b640 1 -- 172.21.15.32:0/3558680492 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c5c155670 con 0x7f3c5c0cfa40 2024-04-24T23:11:24.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.607+0000 7f3c58ff9640 1 -- 172.21.15.32:0/3558680492 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c44004a00 con 0x7f3c5c0cfa40 2024-04-24T23:11:24.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.607+0000 7f3c6251b640 1 -- 172.21.15.32:0/3558680492 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c5c1591b0 con 0x7f3c5c0cfa40 2024-04-24T23:11:24.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.607+0000 7f3c6251b640 1 -- 172.21.15.32:0/3558680492 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3c5c068430 con 0x7f3c5c0cfa40 2024-04-24T23:11:24.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.608+0000 7f3c58ff9640 1 -- 172.21.15.32:0/3558680492 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3c4401a020 con 0x7f3c5c0cfa40 2024-04-24T23:11:24.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.608+0000 7f3c58ff9640 1 --2- 172.21.15.32:0/3558680492 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3c3c06be30 0x7f3c3c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:24.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.609+0000 7f3c58ff9640 1 -- 172.21.15.32:0/3558680492 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 4943+0+0 (secure 0 0 0) 0x7f3c4408a230 con 0x7f3c5c0cfa40 2024-04-24T23:11:24.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.609+0000 7f3c58ff9640 1 --2- 172.21.15.32:0/3558680492 >> [v2:172.21.15.32:6800/1479395426,v1:172.21.15.32:6801/1479395426] conn(0x7f3c3c071cf0 0x7f3c3c074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:24.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.609+0000 7f3c58ff9640 1 -- 172.21.15.32:0/3558680492 --> [v2:172.21.15.32:6800/1479395426,v1:172.21.15.32:6801/1479395426] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3c3c0747c0 con 0x7f3c3c071cf0 2024-04-24T23:11:24.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.609+0000 7f3c58ff9640 1 -- 172.21.15.32:0/3558680492 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=194) ==== 24+0+0 (secure 0 0 0) 0x7f3c44091050 con 0x7f3c5c0cfa40 2024-04-24T23:11:24.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.609+0000 7f3c5bfff640 1 --2- 172.21.15.32:0/3558680492 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3c3c06be30 0x7f3c3c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:24.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.609+0000 7f3c60ab8640 1 --2- 172.21.15.32:0/3558680492 >> [v2:172.21.15.32:6800/1479395426,v1:172.21.15.32:6801/1479395426] conn(0x7f3c3c071cf0 0x7f3c3c074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:24.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.609+0000 7f3c5bfff640 1 --2- 172.21.15.32:0/3558680492 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3c3c06be30 0x7f3c3c06e2f0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c1f7ec0 tx=0x7f3c4c009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:24.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.609+0000 7f3c60ab8640 1 --2- 172.21.15.32:0/3558680492 >> [v2:172.21.15.32:6800/1479395426,v1:172.21.15.32:6801/1479395426] conn(0x7f3c3c071cf0 0x7f3c3c074110 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:24.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.610+0000 7f3c58ff9640 1 -- 172.21.15.32:0/3558680492 <== osd.0 v2:172.21.15.32:6800/1479395426 1 ==== command_reply(tid 1: 0 ) ==== 8+0+25033 (crc 0 0 0) 0x7f3c3c0747c0 con 0x7f3c3c071cf0 2024-04-24T23:11:24.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.622+0000 7f3c6251b640 1 -- 172.21.15.32:0/3558680492 --> [v2:172.21.15.32:6800/1479395426,v1:172.21.15.32:6801/1479395426] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f3c5c0cf820 con 0x7f3c3c071cf0 2024-04-24T23:11:24.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.622+0000 7f3c58ff9640 1 -- 172.21.15.32:0/3558680492 <== osd.0 v2:172.21.15.32:6800/1479395426 2 ==== command_reply(tid 2: 0 ) ==== 8+0+565 (crc 0 0 0) 0x7f3c5c0cf820 con 0x7f3c3c071cf0 2024-04-24T23:11:24.624 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:11:24.624 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-04-24T23:11:24.624 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:11:24.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.623+0000 7f3c6251b640 1 -- 172.21.15.32:0/3558680492 >> [v2:172.21.15.32:6800/1479395426,v1:172.21.15.32:6801/1479395426] conn(0x7f3c3c071cf0 msgr2=0x7f3c3c074110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:24.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.623+0000 7f3c6251b640 1 --2- 172.21.15.32:0/3558680492 >> [v2:172.21.15.32:6800/1479395426,v1:172.21.15.32:6801/1479395426] conn(0x7f3c3c071cf0 0x7f3c3c074110 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:24.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.623+0000 7f3c6251b640 1 -- 172.21.15.32:0/3558680492 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3c3c06be30 msgr2=0x7f3c3c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:24.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.623+0000 7f3c6251b640 1 --2- 172.21.15.32:0/3558680492 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f3c3c06be30 0x7f3c3c06e2f0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c1f7ec0 tx=0x7f3c4c009190 comp rx=0 tx=0).stop 2024-04-24T23:11:24.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.623+0000 7f3c6251b640 1 -- 172.21.15.32:0/3558680492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c5c0cfa40 msgr2=0x7f3c5c153d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:24.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.623+0000 7f3c6251b640 1 --2- 172.21.15.32:0/3558680492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c5c0cfa40 0x7f3c5c153d10 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f3c4400c400 tx=0x7f3c4400c8c0 comp rx=0 tx=0).stop 2024-04-24T23:11:24.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.623+0000 7f3c60ab8640 1 -- 172.21.15.32:0/3558680492 reap_dead start 2024-04-24T23:11:24.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.623+0000 7f3c6251b640 1 -- 172.21.15.32:0/3558680492 shutdown_connections 2024-04-24T23:11:24.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.623+0000 7f3c6251b640 1 -- 172.21.15.32:0/3558680492 >> 172.21.15.32:0/3558680492 conn(0x7f3c5c071910 msgr2=0x7f3c5c0b5650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:24.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.623+0000 7f3c6251b640 1 -- 172.21.15.32:0/3558680492 shutdown_connections 2024-04-24T23:11:24.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.623+0000 7f3c6251b640 1 -- 172.21.15.32:0/3558680492 wait complete. 2024-04-24T23:11:24.635 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-24T23:11:24.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.736+0000 7fd9129a9640 1 Processor -- start 2024-04-24T23:11:24.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.736+0000 7fd9129a9640 1 -- start start 2024-04-24T23:11:24.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.737+0000 7fd9129a9640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd90c06b820 0x7fd90c0d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:24.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.737+0000 7fd9129a9640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd90c0d4210 0x7fd90c0b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:24.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.737+0000 7fd9129a9640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd90c0d3170 con 0x7fd90c15e9b0 2024-04-24T23:11:24.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.737+0000 7fd9129a9640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fd90c0d3450 con 0x7fd90c06b820 2024-04-24T23:11:24.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.737+0000 7fd9129a9640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd90c0d3730 con 0x7fd90c0d4210 2024-04-24T23:11:24.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.737+0000 7fd90b7fe640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd90c06b820 0x7fd90c0d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:24.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.737+0000 7fd910f46640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd90c0d4210 0x7fd90c0b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:24.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.738+0000 7fd90bfff640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fd90c15e9b0 0x7fd90c06b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:57896/0 (socket says 172.21.15.32:57896) 2024-04-24T23:11:24.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.738+0000 7fd90bfff640 1 -- 172.21.15.32:0/59639492 learned_addr learned my addr 172.21.15.32:0/59639492 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:11:24.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.738+0000 7fd90b7fe640 1 -- 172.21.15.32:0/59639492 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd90c0d4210 msgr2=0x7fd90c0b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:24.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.738+0000 7fd90b7fe640 1 --2- 172.21.15.32:0/59639492 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd90c0d4210 0x7fd90c0b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:24.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.738+0000 7fd90b7fe640 1 -- 172.21.15.32:0/59639492 >> v1:172.21.15.121:6789/0 conn(0x7fd90c15e9b0 legacy=0x7fd90c06b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:24.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.738+0000 7fd90b7fe640 1 -- 172.21.15.32:0/59639492 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd90c0d3a10 con 0x7fd90c06b820 2024-04-24T23:11:24.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.738+0000 7fd910f46640 1 --2- 172.21.15.32:0/59639492 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd90c0d4210 0x7fd90c0b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:11:24.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.739+0000 7fd90b7fe640 1 --2- 172.21.15.32:0/59639492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd90c06b820 0x7fd90c0d2b90 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fd9000097a0 tx=0x7fd90002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21327d48eeb4f5ce server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:24.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.739+0000 7fd90affd640 1 -- 172.21.15.32:0/59639492 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd900037070 con 0x7fd90c06b820 2024-04-24T23:11:24.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.739+0000 7fd90affd640 1 -- 172.21.15.32:0/59639492 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd900032440 con 0x7fd90c06b820 2024-04-24T23:11:24.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.739+0000 7fd90affd640 1 -- 172.21.15.32:0/59639492 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd900032810 con 0x7fd90c06b820 2024-04-24T23:11:24.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.739+0000 7fd9129a9640 1 -- 172.21.15.32:0/59639492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd90c06b820 msgr2=0x7fd90c0d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:24.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.739+0000 7fd9129a9640 1 --2- 172.21.15.32:0/59639492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd90c06b820 0x7fd90c0d2b90 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fd9000097a0 tx=0x7fd90002a180 comp rx=0 tx=0).stop 2024-04-24T23:11:24.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.739+0000 7fd9129a9640 1 -- 172.21.15.32:0/59639492 shutdown_connections 2024-04-24T23:11:24.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.739+0000 7fd9129a9640 1 --2- 172.21.15.32:0/59639492 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd90c0d4210 0x7fd90c0b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:24.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.739+0000 7fd9129a9640 1 --2- 172.21.15.32:0/59639492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd90c06b820 0x7fd90c0d2b90 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:24.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.739+0000 7fd9129a9640 1 -- 172.21.15.32:0/59639492 >> 172.21.15.32:0/59639492 conn(0x7fd90c0d11d0 msgr2=0x7fd90c0d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:24.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.739+0000 7fd9129a9640 1 -- 172.21.15.32:0/59639492 shutdown_connections 2024-04-24T23:11:24.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.739+0000 7fd9129a9640 1 -- 172.21.15.32:0/59639492 wait complete. 2024-04-24T23:11:24.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.741+0000 7fd9129a9640 1 Processor -- start 2024-04-24T23:11:24.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.741+0000 7fd9129a9640 1 -- start start 2024-04-24T23:11:24.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.741+0000 7fd9129a9640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd90c06b820 0x7fd90c154530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:24.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.741+0000 7fd9129a9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd90c0d4210 0x7fd90c1617f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:24.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.741+0000 7fd9129a9640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd90c15e9b0 0x7fd90c1579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:24.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.741+0000 7fd9129a9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd90c0ba0e0 con 0x7fd90c0d4210 2024-04-24T23:11:24.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.741+0000 7fd9129a9640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd90c0d2400 con 0x7fd90c06b820 2024-04-24T23:11:24.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.741+0000 7fd9129a9640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd90c0ba260 con 0x7fd90c15e9b0 2024-04-24T23:11:24.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.742+0000 7fd90b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd90c0d4210 0x7fd90c1617f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:24.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.742+0000 7fd90b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd90c0d4210 0x7fd90c1617f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54500/0 (socket says 172.21.15.32:54500) 2024-04-24T23:11:24.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.742+0000 7fd90b7fe640 1 -- 172.21.15.32:0/1696901847 learned_addr learned my addr 172.21.15.32:0/1696901847 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:24.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.742+0000 7fd910f46640 1 --2- 172.21.15.32:0/1696901847 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd90c15e9b0 0x7fd90c1579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:24.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.742+0000 7fd90bfff640 1 --2- 172.21.15.32:0/1696901847 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd90c06b820 0x7fd90c154530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:24.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.742+0000 7fd90b7fe640 1 -- 172.21.15.32:0/1696901847 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd90c15e9b0 msgr2=0x7fd90c1579c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:24.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.742+0000 7fd90b7fe640 1 --2- 172.21.15.32:0/1696901847 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd90c15e9b0 0x7fd90c1579c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:24.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.742+0000 7fd90b7fe640 1 -- 172.21.15.32:0/1696901847 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd90c06b820 msgr2=0x7fd90c154530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:24.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.742+0000 7fd90b7fe640 1 --2- 172.21.15.32:0/1696901847 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd90c06b820 0x7fd90c154530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:24.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.742+0000 7fd90b7fe640 1 -- 172.21.15.32:0/1696901847 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd90c156920 con 0x7fd90c0d4210 2024-04-24T23:11:24.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.742+0000 7fd90b7fe640 1 --2- 172.21.15.32:0/1696901847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd90c0d4210 0x7fd90c1617f0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7fd900035040 tx=0x7fd9000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:24.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.742+0000 7fd908ff9640 1 -- 172.21.15.32:0/1696901847 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd900037040 con 0x7fd90c0d4210 2024-04-24T23:11:24.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.742+0000 7fd908ff9640 1 -- 172.21.15.32:0/1696901847 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd900030500 con 0x7fd90c0d4210 2024-04-24T23:11:24.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.742+0000 7fd9129a9640 1 -- 172.21.15.32:0/1696901847 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd90c156c00 con 0x7fd90c0d4210 2024-04-24T23:11:24.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.742+0000 7fd908ff9640 1 -- 172.21.15.32:0/1696901847 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd900030910 con 0x7fd90c0d4210 2024-04-24T23:11:24.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.742+0000 7fd9129a9640 1 -- 172.21.15.32:0/1696901847 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd90c1571c0 con 0x7fd90c0d4210 2024-04-24T23:11:24.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.743+0000 7fd9129a9640 1 -- 172.21.15.32:0/1696901847 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd90c068430 con 0x7fd90c0d4210 2024-04-24T23:11:24.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.744+0000 7fd908ff9640 1 -- 172.21.15.32:0/1696901847 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd900044020 con 0x7fd90c0d4210 2024-04-24T23:11:24.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.744+0000 7fd908ff9640 1 --2- 172.21.15.32:0/1696901847 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd8f806be30 0x7fd8f806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:24.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.744+0000 7fd908ff9640 1 -- 172.21.15.32:0/1696901847 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 4943+0+0 (secure 0 0 0) 0x7fd9000aad50 con 0x7fd90c0d4210 2024-04-24T23:11:24.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.745+0000 7fd90bfff640 1 --2- 172.21.15.32:0/1696901847 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd8f806be30 0x7fd8f806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:24.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.745+0000 7fd90bfff640 1 --2- 172.21.15.32:0/1696901847 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd8f806be30 0x7fd8f806e2f0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc007fd0 tx=0x7fd8fc007e80 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:24.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.748+0000 7fd908ff9640 1 -- 172.21.15.32:0/1696901847 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd9000ae030 con 0x7fd90c0d4210 2024-04-24T23:11:24.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.934+0000 7fd9129a9640 1 -- 172.21.15.32:0/1696901847 --> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fd90c0b8b60 con 0x7fd8f806be30 2024-04-24T23:11:24.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.934+0000 7fd908ff9640 1 -- 172.21.15.32:0/1696901847 <== mgr.4108 v2:172.21.15.121:6832/1848157943 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fd90c0b8b60 con 0x7fd8f806be30 2024-04-24T23:11:24.936 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 0 to deep-scrub 2024-04-24T23:11:24.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.936+0000 7fd9129a9640 1 -- 172.21.15.32:0/1696901847 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd8f806be30 msgr2=0x7fd8f806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:24.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.936+0000 7fd9129a9640 1 --2- 172.21.15.32:0/1696901847 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd8f806be30 0x7fd8f806e2f0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc007fd0 tx=0x7fd8fc007e80 comp rx=0 tx=0).stop 2024-04-24T23:11:24.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.936+0000 7fd9129a9640 1 -- 172.21.15.32:0/1696901847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd90c0d4210 msgr2=0x7fd90c1617f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:24.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.936+0000 7fd9129a9640 1 --2- 172.21.15.32:0/1696901847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd90c0d4210 0x7fd90c1617f0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7fd900035040 tx=0x7fd9000046c0 comp rx=0 tx=0).stop 2024-04-24T23:11:24.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.936+0000 7fd9129a9640 1 -- 172.21.15.32:0/1696901847 shutdown_connections 2024-04-24T23:11:24.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.936+0000 7fd9129a9640 1 --2- 172.21.15.32:0/1696901847 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd8f806be30 0x7fd8f806e2f0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:24.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.936+0000 7fd9129a9640 1 --2- 172.21.15.32:0/1696901847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd90c0d4210 0x7fd90c1617f0 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:24.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.936+0000 7fd9129a9640 1 --2- 172.21.15.32:0/1696901847 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd90c15e9b0 0x7fd90c1579c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:24.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.936+0000 7fd9129a9640 1 --2- 172.21.15.32:0/1696901847 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd90c06b820 0x7fd90c154530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:24.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.936+0000 7fd9129a9640 1 -- 172.21.15.32:0/1696901847 >> 172.21.15.32:0/1696901847 conn(0x7fd90c0d11d0 msgr2=0x7fd90c0b78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:24.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.937+0000 7fd9129a9640 1 -- 172.21.15.32:0/1696901847 shutdown_connections 2024-04-24T23:11:24.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:24.937+0000 7fd9129a9640 1 -- 172.21.15.32:0/1696901847 wait complete. 2024-04-24T23:11:24.949 INFO:tasks.ceph:Scrubbing osd.1 2024-04-24T23:11:24.949 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-24T23:11:25.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.052+0000 7f94484ee640 1 Processor -- start 2024-04-24T23:11:25.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.052+0000 7f94484ee640 1 -- start start 2024-04-24T23:11:25.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.053+0000 7f94484ee640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f94400cd3c0 0x7f944015e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:25.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.053+0000 7f94484ee640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f944015ec80 0x7f944006c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:25.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.053+0000 7f94484ee640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94400cd850 con 0x7f94400b57c0 2024-04-24T23:11:25.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.053+0000 7f94484ee640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f944015f0c0 con 0x7f94400cd3c0 2024-04-24T23:11:25.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.053+0000 7f94484ee640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f944015f5c0 con 0x7f944015ec80 2024-04-24T23:11:25.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.053+0000 7f944628a640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f94400b57c0 0x7f94400b5ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56356/0 (socket says 172.21.15.32:56356) 2024-04-24T23:11:25.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.053+0000 7f944628a640 1 -- 172.21.15.32:0/2310488074 learned_addr learned my addr 172.21.15.32:0/2310488074 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:11:25.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.053+0000 7f9445a89640 1 --2- 172.21.15.32:0/2310488074 >> v2:172.21.15.121:3300/0 conn(0x7f94400cd3c0 0x7f944015e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:25.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.053+0000 7f9446a8b640 1 --2- 172.21.15.32:0/2310488074 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f944015ec80 0x7f944006c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:25.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.054+0000 7f9445288640 1 -- 172.21.15.32:0/2310488074 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2454548135 0 0) 0x7f94400cd850 con 0x7f94400b57c0 2024-04-24T23:11:25.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.054+0000 7f9445288640 1 -- 172.21.15.32:0/2310488074 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9428003520 con 0x7f94400b57c0 2024-04-24T23:11:25.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.054+0000 7f9445a89640 1 -- 172.21.15.32:0/2310488074 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f944015ec80 msgr2=0x7f944006c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:25.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.054+0000 7f9445a89640 1 --2- 172.21.15.32:0/2310488074 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f944015ec80 0x7f944006c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.054+0000 7f9445a89640 1 -- 172.21.15.32:0/2310488074 >> v1:172.21.15.32:6789/0 conn(0x7f94400b57c0 legacy=0x7f94400b5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:25.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.054+0000 7f9445a89640 1 -- 172.21.15.32:0/2310488074 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94400cdb30 con 0x7f94400cd3c0 2024-04-24T23:11:25.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.054+0000 7f9446a8b640 1 --2- 172.21.15.32:0/2310488074 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f944015ec80 0x7f944006c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:11:25.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.055+0000 7f9445a89640 1 --2- 172.21.15.32:0/2310488074 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94400cd3c0 0x7f944015e740 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f94340097a0 tx=0x7f943402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82094cb258f19261 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:25.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.055+0000 7f9445288640 1 -- 172.21.15.32:0/2310488074 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9434037070 con 0x7f94400cd3c0 2024-04-24T23:11:25.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.055+0000 7f9445288640 1 -- 172.21.15.32:0/2310488074 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9434032440 con 0x7f94400cd3c0 2024-04-24T23:11:25.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.055+0000 7f9445288640 1 -- 172.21.15.32:0/2310488074 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94340327f0 con 0x7f94400cd3c0 2024-04-24T23:11:25.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.055+0000 7f94484ee640 1 -- 172.21.15.32:0/2310488074 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94400cd3c0 msgr2=0x7f944015e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:25.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.055+0000 7f94484ee640 1 --2- 172.21.15.32:0/2310488074 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94400cd3c0 0x7f944015e740 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f94340097a0 tx=0x7f943402a180 comp rx=0 tx=0).stop 2024-04-24T23:11:25.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.055+0000 7f94484ee640 1 -- 172.21.15.32:0/2310488074 shutdown_connections 2024-04-24T23:11:25.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.055+0000 7f94484ee640 1 --2- 172.21.15.32:0/2310488074 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94400cd3c0 0x7f944015e740 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.055+0000 7f94484ee640 1 --2- 172.21.15.32:0/2310488074 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f944015ec80 0x7f944006c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.055+0000 7f94484ee640 1 -- 172.21.15.32:0/2310488074 >> 172.21.15.32:0/2310488074 conn(0x7f94400b1f10 msgr2=0x7f94400b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:25.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.055+0000 7f94484ee640 1 -- 172.21.15.32:0/2310488074 shutdown_connections 2024-04-24T23:11:25.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.055+0000 7f94484ee640 1 -- 172.21.15.32:0/2310488074 wait complete. 2024-04-24T23:11:25.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.057+0000 7f94484ee640 1 Processor -- start 2024-04-24T23:11:25.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.057+0000 7f94484ee640 1 -- start start 2024-04-24T23:11:25.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.057+0000 7f94484ee640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f94400b57c0 0x7f94401462f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:25.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.057+0000 7f94484ee640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f944015ec80 0x7f9440132890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:25.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.057+0000 7f94484ee640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f944014c680 0x7f944014de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:25.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.057+0000 7f94484ee640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f94400ce3d0 con 0x7f944014c680 2024-04-24T23:11:25.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.057+0000 7f94484ee640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f94400ce0f0 con 0x7f944015ec80 2024-04-24T23:11:25.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.057+0000 7f94484ee640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f94400b62b0 con 0x7f94400b57c0 2024-04-24T23:11:25.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.057+0000 7f9446a8b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f944014c680 0x7f944014de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:25.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.057+0000 7f9446a8b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f944014c680 0x7f944014de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54510/0 (socket says 172.21.15.32:54510) 2024-04-24T23:11:25.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.057+0000 7f9446a8b640 1 -- 172.21.15.32:0/3367991745 learned_addr learned my addr 172.21.15.32:0/3367991745 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:25.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.058+0000 7f944628a640 1 --2- 172.21.15.32:0/3367991745 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f94400b57c0 0x7f94401462f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:25.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.058+0000 7f9445a89640 1 --2- 172.21.15.32:0/3367991745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f944015ec80 0x7f9440132890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:25.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.058+0000 7f9446a8b640 1 -- 172.21.15.32:0/3367991745 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f94400b57c0 msgr2=0x7f94401462f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:25.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.058+0000 7f9446a8b640 1 --2- 172.21.15.32:0/3367991745 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f94400b57c0 0x7f94401462f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.058+0000 7f9446a8b640 1 -- 172.21.15.32:0/3367991745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f944015ec80 msgr2=0x7f9440132890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:25.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.058+0000 7f9446a8b640 1 --2- 172.21.15.32:0/3367991745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f944015ec80 0x7f9440132890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.058+0000 7f9446a8b640 1 -- 172.21.15.32:0/3367991745 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9440146830 con 0x7f944014c680 2024-04-24T23:11:25.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.058+0000 7f9446a8b640 1 --2- 172.21.15.32:0/3367991745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f944014c680 0x7f944014de90 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f9430009db0 tx=0x7f943000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:25.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.058+0000 7f943effd640 1 -- 172.21.15.32:0/3367991745 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9430018030 con 0x7f944014c680 2024-04-24T23:11:25.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.058+0000 7f943effd640 1 -- 172.21.15.32:0/3367991745 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f943000cd30 con 0x7f944014c680 2024-04-24T23:11:25.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.058+0000 7f94484ee640 1 -- 172.21.15.32:0/3367991745 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f944014e8e0 con 0x7f944014c680 2024-04-24T23:11:25.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.058+0000 7f943effd640 1 -- 172.21.15.32:0/3367991745 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9430003bb0 con 0x7f944014c680 2024-04-24T23:11:25.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.058+0000 7f94484ee640 1 -- 172.21.15.32:0/3367991745 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f944014eea0 con 0x7f944014c680 2024-04-24T23:11:25.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.059+0000 7f94484ee640 1 -- 172.21.15.32:0/3367991745 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9440068430 con 0x7f944014c680 2024-04-24T23:11:25.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.060+0000 7f943effd640 1 -- 172.21.15.32:0/3367991745 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9430005d10 con 0x7f944014c680 2024-04-24T23:11:25.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.060+0000 7f943effd640 1 --2- 172.21.15.32:0/3367991745 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f942c06be30 0x7f942c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:25.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.060+0000 7f943effd640 1 -- 172.21.15.32:0/3367991745 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 4943+0+0 (secure 0 0 0) 0x7f943008b300 con 0x7f944014c680 2024-04-24T23:11:25.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.060+0000 7f943effd640 1 --2- 172.21.15.32:0/3367991745 >> [v2:172.21.15.32:6808/1340633718,v1:172.21.15.32:6809/1340633718] conn(0x7f942c071cf0 0x7f942c074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:25.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.060+0000 7f943effd640 1 -- 172.21.15.32:0/3367991745 --> [v2:172.21.15.32:6808/1340633718,v1:172.21.15.32:6809/1340633718] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f943000be50 con 0x7f942c071cf0 2024-04-24T23:11:25.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.060+0000 7f943effd640 1 -- 172.21.15.32:0/3367991745 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=194) ==== 24+0+0 (secure 0 0 0) 0x7f9430092050 con 0x7f944014c680 2024-04-24T23:11:25.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.060+0000 7f944628a640 1 --2- 172.21.15.32:0/3367991745 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f942c06be30 0x7f942c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:25.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.060+0000 7f9445a89640 1 --2- 172.21.15.32:0/3367991745 >> [v2:172.21.15.32:6808/1340633718,v1:172.21.15.32:6809/1340633718] conn(0x7f942c071cf0 0x7f942c074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:25.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.061+0000 7f9445a89640 1 --2- 172.21.15.32:0/3367991745 >> [v2:172.21.15.32:6808/1340633718,v1:172.21.15.32:6809/1340633718] conn(0x7f942c071cf0 0x7f942c074110 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:25.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.061+0000 7f944628a640 1 --2- 172.21.15.32:0/3367991745 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f942c06be30 0x7f942c06e2f0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f9438005880 tx=0x7f9438009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:25.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.061+0000 7f943effd640 1 -- 172.21.15.32:0/3367991745 <== osd.1 v2:172.21.15.32:6808/1340633718 1 ==== command_reply(tid 1: 0 ) ==== 8+0+25033 (crc 0 0 0) 0x7f943000be50 con 0x7f942c071cf0 2024-04-24T23:11:25.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.073+0000 7f94484ee640 1 -- 172.21.15.32:0/3367991745 --> [v2:172.21.15.32:6808/1340633718,v1:172.21.15.32:6809/1340633718] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f944014f610 con 0x7f942c071cf0 2024-04-24T23:11:25.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.073+0000 7f943effd640 1 -- 172.21.15.32:0/3367991745 <== osd.1 v2:172.21.15.32:6808/1340633718 2 ==== command_reply(tid 2: 0 ) ==== 8+0+565 (crc 0 0 0) 0x7f944014f610 con 0x7f942c071cf0 2024-04-24T23:11:25.077 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:11:25.078 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-04-24T23:11:25.078 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:11:25.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.074+0000 7f94484ee640 1 -- 172.21.15.32:0/3367991745 >> [v2:172.21.15.32:6808/1340633718,v1:172.21.15.32:6809/1340633718] conn(0x7f942c071cf0 msgr2=0x7f942c074110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:25.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.074+0000 7f94484ee640 1 --2- 172.21.15.32:0/3367991745 >> [v2:172.21.15.32:6808/1340633718,v1:172.21.15.32:6809/1340633718] conn(0x7f942c071cf0 0x7f942c074110 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.074+0000 7f94484ee640 1 -- 172.21.15.32:0/3367991745 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f942c06be30 msgr2=0x7f942c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:25.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.074+0000 7f94484ee640 1 --2- 172.21.15.32:0/3367991745 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f942c06be30 0x7f942c06e2f0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f9438005880 tx=0x7f9438009040 comp rx=0 tx=0).stop 2024-04-24T23:11:25.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.074+0000 7f94484ee640 1 -- 172.21.15.32:0/3367991745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f944014c680 msgr2=0x7f944014de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:25.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.074+0000 7f94484ee640 1 --2- 172.21.15.32:0/3367991745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f944014c680 0x7f944014de90 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f9430009db0 tx=0x7f943000c490 comp rx=0 tx=0).stop 2024-04-24T23:11:25.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.074+0000 7f9446a8b640 1 -- 172.21.15.32:0/3367991745 reap_dead start 2024-04-24T23:11:25.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.074+0000 7f94484ee640 1 -- 172.21.15.32:0/3367991745 shutdown_connections 2024-04-24T23:11:25.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.074+0000 7f94484ee640 1 -- 172.21.15.32:0/3367991745 >> 172.21.15.32:0/3367991745 conn(0x7f94400b1f10 msgr2=0x7f944006c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:25.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.074+0000 7f94484ee640 1 -- 172.21.15.32:0/3367991745 shutdown_connections 2024-04-24T23:11:25.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.074+0000 7f94484ee640 1 -- 172.21.15.32:0/3367991745 wait complete. 2024-04-24T23:11:25.086 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-24T23:11:25.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.186+0000 7f5988b9e640 1 Processor -- start 2024-04-24T23:11:25.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.186+0000 7f5988b9e640 1 -- start start 2024-04-24T23:11:25.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.187+0000 7f5988b9e640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f598415b260 0x7f59840ceee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:25.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.187+0000 7f5988b9e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5984072820 0x7f5984072cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:25.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.187+0000 7f5988b9e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f59840ccc90 0x7f59840cd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:25.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.187+0000 7f5988b9e640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f598415b640 con 0x7f5984072820 2024-04-24T23:11:25.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.187+0000 7f5988b9e640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f59840ce120 con 0x7f598415b260 2024-04-24T23:11:25.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.187+0000 7f5988b9e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f59840ce400 con 0x7f59840ccc90 2024-04-24T23:11:25.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.187+0000 7f5982ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5984072820 0x7f5984072cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:25.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.187+0000 7f5982ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5984072820 0x7f5984072cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54512/0 (socket says 172.21.15.32:54512) 2024-04-24T23:11:25.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.187+0000 7f5982ffd640 1 -- 172.21.15.32:0/2193247278 learned_addr learned my addr 172.21.15.32:0/2193247278 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:25.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.187+0000 7f5983fff640 1 --2- 172.21.15.32:0/2193247278 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f59840ccc90 0x7f59840cd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:25.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.187+0000 7f59837fe640 1 --2- 172.21.15.32:0/2193247278 >> v2:172.21.15.121:3300/0 conn(0x7f598415b260 0x7f59840ceee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:25.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.187+0000 7f5982ffd640 1 -- 172.21.15.32:0/2193247278 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f59840ccc90 msgr2=0x7f59840cd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:25.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.187+0000 7f5982ffd640 1 --2- 172.21.15.32:0/2193247278 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f59840ccc90 0x7f59840cd120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.187+0000 7f5982ffd640 1 -- 172.21.15.32:0/2193247278 >> v2:172.21.15.121:3300/0 conn(0x7f598415b260 msgr2=0x7f59840ceee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:25.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.187+0000 7f5982ffd640 1 --2- 172.21.15.32:0/2193247278 >> v2:172.21.15.121:3300/0 conn(0x7f598415b260 0x7f59840ceee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.187+0000 7f5982ffd640 1 -- 172.21.15.32:0/2193247278 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59840ce6e0 con 0x7f5984072820 2024-04-24T23:11:25.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.187+0000 7f59837fe640 1 --2- 172.21.15.32:0/2193247278 >> v2:172.21.15.121:3300/0 conn(0x7f598415b260 0x7f59840ceee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:11:25.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.188+0000 7f5982ffd640 1 --2- 172.21.15.32:0/2193247278 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5984072820 0x7f5984072cb0 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f598406dcc0 tx=0x7f597802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31b0766e299d3c10 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:25.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.188+0000 7f59827fc640 1 -- 172.21.15.32:0/2193247278 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5978037070 con 0x7f5984072820 2024-04-24T23:11:25.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.188+0000 7f59827fc640 1 -- 172.21.15.32:0/2193247278 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5978032440 con 0x7f5984072820 2024-04-24T23:11:25.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.188+0000 7f59827fc640 1 -- 172.21.15.32:0/2193247278 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5978032810 con 0x7f5984072820 2024-04-24T23:11:25.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.189+0000 7f5988b9e640 1 -- 172.21.15.32:0/2193247278 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5984072820 msgr2=0x7f5984072cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:25.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.189+0000 7f5988b9e640 1 --2- 172.21.15.32:0/2193247278 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5984072820 0x7f5984072cb0 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f598406dcc0 tx=0x7f597802a1e0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.189+0000 7f5988b9e640 1 -- 172.21.15.32:0/2193247278 shutdown_connections 2024-04-24T23:11:25.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.189+0000 7f5988b9e640 1 --2- 172.21.15.32:0/2193247278 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f59840ccc90 0x7f59840cd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.189+0000 7f5988b9e640 1 --2- 172.21.15.32:0/2193247278 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5984072820 0x7f5984072cb0 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.189+0000 7f5988b9e640 1 --2- 172.21.15.32:0/2193247278 >> v2:172.21.15.121:3300/0 conn(0x7f598415b260 0x7f59840ceee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.189+0000 7f5988b9e640 1 -- 172.21.15.32:0/2193247278 >> 172.21.15.32:0/2193247278 conn(0x7f598406b2f0 msgr2=0x7f598406b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:25.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.189+0000 7f5988b9e640 1 -- 172.21.15.32:0/2193247278 shutdown_connections 2024-04-24T23:11:25.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.189+0000 7f5988b9e640 1 -- 172.21.15.32:0/2193247278 wait complete. 2024-04-24T23:11:25.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.190+0000 7f5988b9e640 1 Processor -- start 2024-04-24T23:11:25.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.190+0000 7f5988b9e640 1 -- start start 2024-04-24T23:11:25.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.191+0000 7f5988b9e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5984072820 0x7f5984145350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:25.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.191+0000 7f5988b9e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59840ccc90 0x7f5984144970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:25.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.191+0000 7f5988b9e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f598415b260 0x7f598415e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:25.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.191+0000 7f5988b9e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f59840af7f0 con 0x7f5984072820 2024-04-24T23:11:25.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.191+0000 7f5988b9e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f59840cf420 con 0x7f59840ccc90 2024-04-24T23:11:25.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.191+0000 7f5988b9e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f59840732c0 con 0x7f598415b260 2024-04-24T23:11:25.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.191+0000 7f59837fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5984072820 0x7f5984145350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:25.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.191+0000 7f59837fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5984072820 0x7f5984145350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54522/0 (socket says 172.21.15.32:54522) 2024-04-24T23:11:25.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.191+0000 7f59837fe640 1 -- 172.21.15.32:0/1364423000 learned_addr learned my addr 172.21.15.32:0/1364423000 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:25.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.191+0000 7f5983fff640 1 --2- 172.21.15.32:0/1364423000 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f598415b260 0x7f598415e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:25.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.191+0000 7f5982ffd640 1 --2- 172.21.15.32:0/1364423000 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59840ccc90 0x7f5984144970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:25.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.191+0000 7f59837fe640 1 -- 172.21.15.32:0/1364423000 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f598415b260 msgr2=0x7f598415e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:25.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.191+0000 7f59837fe640 1 --2- 172.21.15.32:0/1364423000 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f598415b260 0x7f598415e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.192+0000 7f59837fe640 1 -- 172.21.15.32:0/1364423000 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59840ccc90 msgr2=0x7f5984144970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:25.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.192+0000 7f59837fe640 1 --2- 172.21.15.32:0/1364423000 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59840ccc90 0x7f5984144970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.192+0000 7f59837fe640 1 -- 172.21.15.32:0/1364423000 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59840707f0 con 0x7f5984072820 2024-04-24T23:11:25.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.192+0000 7f5982ffd640 1 --2- 172.21.15.32:0/1364423000 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59840ccc90 0x7f5984144970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:11:25.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.192+0000 7f59837fe640 1 --2- 172.21.15.32:0/1364423000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5984072820 0x7f5984145350 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f597400d800 tx=0x7f597400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:25.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.192+0000 7f596bfff640 1 -- 172.21.15.32:0/1364423000 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5974004030 con 0x7f5984072820 2024-04-24T23:11:25.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.192+0000 7f596bfff640 1 -- 172.21.15.32:0/1364423000 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59740041d0 con 0x7f5984072820 2024-04-24T23:11:25.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.192+0000 7f596bfff640 1 -- 172.21.15.32:0/1364423000 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f597400b6c0 con 0x7f5984072820 2024-04-24T23:11:25.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.192+0000 7f5988b9e640 1 -- 172.21.15.32:0/1364423000 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5984070a90 con 0x7f5984072820 2024-04-24T23:11:25.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.192+0000 7f5988b9e640 1 -- 172.21.15.32:0/1364423000 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5984071050 con 0x7f5984072820 2024-04-24T23:11:25.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.193+0000 7f5988b9e640 1 -- 172.21.15.32:0/1364423000 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59840682b0 con 0x7f5984072820 2024-04-24T23:11:25.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.193+0000 7f596bfff640 1 -- 172.21.15.32:0/1364423000 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f597400b860 con 0x7f5984072820 2024-04-24T23:11:25.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.194+0000 7f596bfff640 1 --2- 172.21.15.32:0/1364423000 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f597006be30 0x7f597006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:25.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.194+0000 7f596bfff640 1 -- 172.21.15.32:0/1364423000 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 4943+0+0 (secure 0 0 0) 0x7f597408adc0 con 0x7f5984072820 2024-04-24T23:11:25.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.194+0000 7f5982ffd640 1 --2- 172.21.15.32:0/1364423000 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f597006be30 0x7f597006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:25.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.195+0000 7f5982ffd640 1 --2- 172.21.15.32:0/1364423000 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f597006be30 0x7f597006e2f0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f5978034000 tx=0x7f597802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:25.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.198+0000 7f596bfff640 1 -- 172.21.15.32:0/1364423000 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5974059a70 con 0x7f5984072820 2024-04-24T23:11:25.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.381+0000 7f5988b9e640 1 -- 172.21.15.32:0/1364423000 --> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f5984072fe0 con 0x7f597006be30 2024-04-24T23:11:25.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.381+0000 7f596bfff640 1 -- 172.21.15.32:0/1364423000 <== mgr.4108 v2:172.21.15.121:6832/1848157943 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f5984072fe0 con 0x7f597006be30 2024-04-24T23:11:25.383 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 1 to deep-scrub 2024-04-24T23:11:25.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.383+0000 7f5988b9e640 1 -- 172.21.15.32:0/1364423000 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f597006be30 msgr2=0x7f597006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:25.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.383+0000 7f5988b9e640 1 --2- 172.21.15.32:0/1364423000 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f597006be30 0x7f597006e2f0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f5978034000 tx=0x7f597802e000 comp rx=0 tx=0).stop 2024-04-24T23:11:25.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.383+0000 7f5988b9e640 1 -- 172.21.15.32:0/1364423000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5984072820 msgr2=0x7f5984145350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:25.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.383+0000 7f5988b9e640 1 --2- 172.21.15.32:0/1364423000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5984072820 0x7f5984145350 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f597400d800 tx=0x7f597400dcc0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.383+0000 7f5988b9e640 1 -- 172.21.15.32:0/1364423000 shutdown_connections 2024-04-24T23:11:25.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.383+0000 7f5988b9e640 1 --2- 172.21.15.32:0/1364423000 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f597006be30 0x7f597006e2f0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.383+0000 7f5988b9e640 1 --2- 172.21.15.32:0/1364423000 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f598415b260 0x7f598415e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.383+0000 7f5988b9e640 1 --2- 172.21.15.32:0/1364423000 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59840ccc90 0x7f5984144970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.383+0000 7f5988b9e640 1 --2- 172.21.15.32:0/1364423000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5984072820 0x7f5984145350 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.383+0000 7f5988b9e640 1 -- 172.21.15.32:0/1364423000 >> 172.21.15.32:0/1364423000 conn(0x7f598406b2f0 msgr2=0x7f5984159d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:25.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.384+0000 7f5988b9e640 1 -- 172.21.15.32:0/1364423000 shutdown_connections 2024-04-24T23:11:25.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.384+0000 7f5988b9e640 1 -- 172.21.15.32:0/1364423000 wait complete. 2024-04-24T23:11:25.396 INFO:tasks.ceph:Scrubbing osd.2 2024-04-24T23:11:25.396 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-24T23:11:25.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.500+0000 7fc47c076640 1 Processor -- start 2024-04-24T23:11:25.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.500+0000 7fc47c076640 1 -- start start 2024-04-24T23:11:25.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.501+0000 7fc47c076640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc47415b580 0x7fc4740cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:25.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.501+0000 7fc47c076640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc47415b960 0x7fc47406cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:25.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.501+0000 7fc47c076640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc47415bde0 con 0x7fc4740b58e0 2024-04-24T23:11:25.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.501+0000 7fc47c076640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fc47415c9f0 con 0x7fc47415b580 2024-04-24T23:11:25.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.501+0000 7fc47c076640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc4740ce3c0 con 0x7fc47415b960 2024-04-24T23:11:25.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.501+0000 7fc479611640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc47415b960 0x7fc47406cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:25.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.501+0000 7fc479e12640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc47415b580 0x7fc4740cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:25.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.501+0000 7fc479611640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc47415b960 0x7fc47406cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.32:44112/0 (socket says 172.21.15.32:44112) 2024-04-24T23:11:25.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.501+0000 7fc479611640 1 -- 172.21.15.32:0/4095533436 learned_addr learned my addr 172.21.15.32:0/4095533436 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:25.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.501+0000 7fc47a613640 1 --1- 172.21.15.32:0/4095533436 >> v1:172.21.15.121:6789/0 conn(0x7fc4740b58e0 0x7fc47415f340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:57912/0 (socket says 172.21.15.32:57912) 2024-04-24T23:11:25.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.501+0000 7fc479e12640 1 -- 172.21.15.32:0/4095533436 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc47415b960 msgr2=0x7fc47406cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:25.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.501+0000 7fc479e12640 1 --2- 172.21.15.32:0/4095533436 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc47415b960 0x7fc47406cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.501+0000 7fc479e12640 1 -- 172.21.15.32:0/4095533436 >> v1:172.21.15.121:6789/0 conn(0x7fc4740b58e0 legacy=0x7fc47415f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:25.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.501+0000 7fc479e12640 1 -- 172.21.15.32:0/4095533436 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc47415c0c0 con 0x7fc47415b580 2024-04-24T23:11:25.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.501+0000 7fc479611640 1 --2- 172.21.15.32:0/4095533436 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc47415b960 0x7fc47406cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:11:25.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.502+0000 7fc479e12640 1 --2- 172.21.15.32:0/4095533436 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc47415b580 0x7fc4740cde80 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fc4700097a0 tx=0x7fc47002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d79daa2e01c0299 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:25.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.502+0000 7fc478e10640 1 -- 172.21.15.32:0/4095533436 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc470037070 con 0x7fc47415b580 2024-04-24T23:11:25.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.502+0000 7fc478e10640 1 -- 172.21.15.32:0/4095533436 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc470032440 con 0x7fc47415b580 2024-04-24T23:11:25.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.502+0000 7fc478e10640 1 -- 172.21.15.32:0/4095533436 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc470032810 con 0x7fc47415b580 2024-04-24T23:11:25.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.503+0000 7fc47c076640 1 -- 172.21.15.32:0/4095533436 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc47415b580 msgr2=0x7fc4740cde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:25.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.503+0000 7fc47c076640 1 --2- 172.21.15.32:0/4095533436 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc47415b580 0x7fc4740cde80 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fc4700097a0 tx=0x7fc47002a180 comp rx=0 tx=0).stop 2024-04-24T23:11:25.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.503+0000 7fc47c076640 1 -- 172.21.15.32:0/4095533436 shutdown_connections 2024-04-24T23:11:25.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.503+0000 7fc47c076640 1 --2- 172.21.15.32:0/4095533436 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc47415b960 0x7fc47406cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.503+0000 7fc47c076640 1 --2- 172.21.15.32:0/4095533436 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc47415b580 0x7fc4740cde80 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.503+0000 7fc47c076640 1 -- 172.21.15.32:0/4095533436 >> 172.21.15.32:0/4095533436 conn(0x7fc4740b1f10 msgr2=0x7fc4740b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:25.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.503+0000 7fc47c076640 1 -- 172.21.15.32:0/4095533436 shutdown_connections 2024-04-24T23:11:25.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.503+0000 7fc47c076640 1 -- 172.21.15.32:0/4095533436 wait complete. 2024-04-24T23:11:25.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.504+0000 7fc47c076640 1 Processor -- start 2024-04-24T23:11:25.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.504+0000 7fc47c076640 1 -- start start 2024-04-24T23:11:25.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.505+0000 7fc47c076640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4740b58e0 0x7fc4740710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:25.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.505+0000 7fc47c076640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc47415b580 0x7fc474072e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:25.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.505+0000 7fc47c076640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc47415b960 0x7fc47415a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:25.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.505+0000 7fc47c076640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc47406c7a0 con 0x7fc47415b580 2024-04-24T23:11:25.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.505+0000 7fc47c076640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc47415c680 con 0x7fc4740b58e0 2024-04-24T23:11:25.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.505+0000 7fc47c076640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fc47406c920 con 0x7fc47415b960 2024-04-24T23:11:25.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.505+0000 7fc479611640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc47415b580 0x7fc474072e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:25.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.505+0000 7fc479611640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc47415b580 0x7fc474072e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54536/0 (socket says 172.21.15.32:54536) 2024-04-24T23:11:25.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.505+0000 7fc479611640 1 -- 172.21.15.32:0/2486040788 learned_addr learned my addr 172.21.15.32:0/2486040788 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:25.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.505+0000 7fc47a613640 1 --2- 172.21.15.32:0/2486040788 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc47415b960 0x7fc47415a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:25.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.505+0000 7fc479e12640 1 --2- 172.21.15.32:0/2486040788 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4740b58e0 0x7fc4740710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:25.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.505+0000 7fc479611640 1 -- 172.21.15.32:0/2486040788 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc47415b960 msgr2=0x7fc47415a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:25.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.505+0000 7fc479611640 1 --2- 172.21.15.32:0/2486040788 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc47415b960 0x7fc47415a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.505+0000 7fc479611640 1 -- 172.21.15.32:0/2486040788 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4740b58e0 msgr2=0x7fc4740710f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:25.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.505+0000 7fc479611640 1 --2- 172.21.15.32:0/2486040788 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4740b58e0 0x7fc4740710f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.505+0000 7fc479611640 1 -- 172.21.15.32:0/2486040788 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc47415aaa0 con 0x7fc47415b580 2024-04-24T23:11:25.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.505+0000 7fc47a613640 1 --2- 172.21.15.32:0/2486040788 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc47415b960 0x7fc47415a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:11:25.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.505+0000 7fc479e12640 1 --2- 172.21.15.32:0/2486040788 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4740b58e0 0x7fc4740710f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:11:25.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.506+0000 7fc479611640 1 --2- 172.21.15.32:0/2486040788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc47415b580 0x7fc474072e90 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7fc46400e9b0 tx=0x7fc46400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:25.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.506+0000 7fc46a7fc640 1 -- 172.21.15.32:0/2486040788 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc464017070 con 0x7fc47415b580 2024-04-24T23:11:25.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.506+0000 7fc46a7fc640 1 -- 172.21.15.32:0/2486040788 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc46400c9a0 con 0x7fc47415b580 2024-04-24T23:11:25.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.506+0000 7fc47c076640 1 -- 172.21.15.32:0/2486040788 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4740b3470 con 0x7fc47415b580 2024-04-24T23:11:25.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.506+0000 7fc46a7fc640 1 -- 172.21.15.32:0/2486040788 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46400cd50 con 0x7fc47415b580 2024-04-24T23:11:25.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.506+0000 7fc47c076640 1 -- 172.21.15.32:0/2486040788 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc474153230 con 0x7fc47415b580 2024-04-24T23:11:25.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.507+0000 7fc47c076640 1 -- 172.21.15.32:0/2486040788 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc474068430 con 0x7fc47415b580 2024-04-24T23:11:25.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.507+0000 7fc46a7fc640 1 -- 172.21.15.32:0/2486040788 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc46401a070 con 0x7fc47415b580 2024-04-24T23:11:25.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.508+0000 7fc46a7fc640 1 --2- 172.21.15.32:0/2486040788 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc45806be30 0x7fc45806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:25.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.508+0000 7fc46a7fc640 1 -- 172.21.15.32:0/2486040788 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 4943+0+0 (secure 0 0 0) 0x7fc46408b200 con 0x7fc47415b580 2024-04-24T23:11:25.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.508+0000 7fc46a7fc640 1 --2- 172.21.15.32:0/2486040788 >> [v2:172.21.15.32:6816/3405899334,v1:172.21.15.32:6817/3405899334] conn(0x7fc458071cf0 0x7fc458074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:25.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.508+0000 7fc46a7fc640 1 -- 172.21.15.32:0/2486040788 --> [v2:172.21.15.32:6816/3405899334,v1:172.21.15.32:6817/3405899334] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc464027ea0 con 0x7fc458071cf0 2024-04-24T23:11:25.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.508+0000 7fc46a7fc640 1 -- 172.21.15.32:0/2486040788 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=194) ==== 24+0+0 (secure 0 0 0) 0x7fc46408b690 con 0x7fc47415b580 2024-04-24T23:11:25.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.508+0000 7fc479e12640 1 --2- 172.21.15.32:0/2486040788 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc45806be30 0x7fc45806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:25.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.508+0000 7fc47a613640 1 --2- 172.21.15.32:0/2486040788 >> [v2:172.21.15.32:6816/3405899334,v1:172.21.15.32:6817/3405899334] conn(0x7fc458071cf0 0x7fc458074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:25.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.508+0000 7fc47a613640 1 --2- 172.21.15.32:0/2486040788 >> [v2:172.21.15.32:6816/3405899334,v1:172.21.15.32:6817/3405899334] conn(0x7fc458071cf0 0x7fc458074110 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:25.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.509+0000 7fc479e12640 1 --2- 172.21.15.32:0/2486040788 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc45806be30 0x7fc45806e2f0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fc470034000 tx=0x7fc47002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:25.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.509+0000 7fc46a7fc640 1 -- 172.21.15.32:0/2486040788 <== osd.2 v2:172.21.15.32:6816/3405899334 1 ==== command_reply(tid 1: 0 ) ==== 8+0+25033 (crc 0 0 0) 0x7fc464027ea0 con 0x7fc458071cf0 2024-04-24T23:11:25.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.521+0000 7fc47c076640 1 -- 172.21.15.32:0/2486040788 --> [v2:172.21.15.32:6816/3405899334,v1:172.21.15.32:6817/3405899334] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fc4740cc9d0 con 0x7fc458071cf0 2024-04-24T23:11:25.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.521+0000 7fc46a7fc640 1 -- 172.21.15.32:0/2486040788 <== osd.2 v2:172.21.15.32:6816/3405899334 2 ==== command_reply(tid 2: 0 ) ==== 8+0+565 (crc 0 0 0) 0x7fc4740cc9d0 con 0x7fc458071cf0 2024-04-24T23:11:25.523 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:11:25.524 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-04-24T23:11:25.524 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:11:25.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.522+0000 7fc47c076640 1 -- 172.21.15.32:0/2486040788 >> [v2:172.21.15.32:6816/3405899334,v1:172.21.15.32:6817/3405899334] conn(0x7fc458071cf0 msgr2=0x7fc458074110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:25.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.522+0000 7fc47c076640 1 --2- 172.21.15.32:0/2486040788 >> [v2:172.21.15.32:6816/3405899334,v1:172.21.15.32:6817/3405899334] conn(0x7fc458071cf0 0x7fc458074110 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.522+0000 7fc47c076640 1 -- 172.21.15.32:0/2486040788 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc45806be30 msgr2=0x7fc45806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:25.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.522+0000 7fc47c076640 1 --2- 172.21.15.32:0/2486040788 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fc45806be30 0x7fc45806e2f0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fc470034000 tx=0x7fc47002e000 comp rx=0 tx=0).stop 2024-04-24T23:11:25.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.522+0000 7fc47c076640 1 -- 172.21.15.32:0/2486040788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc47415b580 msgr2=0x7fc474072e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:25.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.522+0000 7fc47c076640 1 --2- 172.21.15.32:0/2486040788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc47415b580 0x7fc474072e90 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7fc46400e9b0 tx=0x7fc46400ee70 comp rx=0 tx=0).stop 2024-04-24T23:11:25.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.522+0000 7fc47a613640 1 -- 172.21.15.32:0/2486040788 reap_dead start 2024-04-24T23:11:25.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.522+0000 7fc47c076640 1 -- 172.21.15.32:0/2486040788 shutdown_connections 2024-04-24T23:11:25.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.522+0000 7fc47c076640 1 -- 172.21.15.32:0/2486040788 >> 172.21.15.32:0/2486040788 conn(0x7fc4740b1f10 msgr2=0x7fc47415f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:25.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.522+0000 7fc47c076640 1 -- 172.21.15.32:0/2486040788 shutdown_connections 2024-04-24T23:11:25.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.522+0000 7fc47c076640 1 -- 172.21.15.32:0/2486040788 wait complete. 2024-04-24T23:11:25.534 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-24T23:11:25.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.635+0000 7ff7c1fb6640 1 Processor -- start 2024-04-24T23:11:25.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.635+0000 7ff7c1fb6640 1 -- start start 2024-04-24T23:11:25.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.635+0000 7ff7c1fb6640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff7bc06c5f0 0x7ff7bc06c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:25.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.636+0000 7ff7c1fb6640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7bc0b44d0 0x7ff7bc0b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:25.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.636+0000 7ff7c1fb6640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7bc0b6430 con 0x7ff7bc0af890 2024-04-24T23:11:25.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.636+0000 7ff7c1fb6640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7ff7bc156f80 con 0x7ff7bc06c5f0 2024-04-24T23:11:25.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.636+0000 7ff7c1fb6640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff7bc0b6150 con 0x7ff7bc0b44d0 2024-04-24T23:11:25.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.636+0000 7ff7bbfff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff7bc0af890 0x7ff7bc0cef70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56368/0 (socket says 172.21.15.32:56368) 2024-04-24T23:11:25.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.636+0000 7ff7bbfff640 1 -- 172.21.15.32:0/3064031798 learned_addr learned my addr 172.21.15.32:0/3064031798 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:11:25.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.636+0000 7ff7baffd640 1 --2- 172.21.15.32:0/3064031798 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7bc0b44d0 0x7ff7bc0b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:25.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.636+0000 7ff7bb7fe640 1 --2- 172.21.15.32:0/3064031798 >> v2:172.21.15.121:3300/0 conn(0x7ff7bc06c5f0 0x7ff7bc06c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:25.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.636+0000 7ff7ba7fc640 1 -- 172.21.15.32:0/3064031798 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1625207182 0 0) 0x7ff7bc0b6430 con 0x7ff7bc0af890 2024-04-24T23:11:25.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.636+0000 7ff7ba7fc640 1 -- 172.21.15.32:0/3064031798 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff7a4003520 con 0x7ff7bc0af890 2024-04-24T23:11:25.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.636+0000 7ff7baffd640 1 -- 172.21.15.32:0/3064031798 >> v2:172.21.15.121:3300/0 conn(0x7ff7bc06c5f0 msgr2=0x7ff7bc06c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:25.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.636+0000 7ff7ba7fc640 1 -- 172.21.15.32:0/3064031798 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1277176919 0 0) 0x7ff7a4003520 con 0x7ff7bc0af890 2024-04-24T23:11:25.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.636+0000 7ff7baffd640 1 --2- 172.21.15.32:0/3064031798 >> v2:172.21.15.121:3300/0 conn(0x7ff7bc06c5f0 0x7ff7bc06c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.636+0000 7ff7baffd640 1 -- 172.21.15.32:0/3064031798 >> v1:172.21.15.32:6789/0 conn(0x7ff7bc0af890 legacy=0x7ff7bc0cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:25.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.636+0000 7ff7baffd640 1 -- 172.21.15.32:0/3064031798 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7bc0b39f0 con 0x7ff7bc0b44d0 2024-04-24T23:11:25.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.637+0000 7ff7baffd640 1 --2- 172.21.15.32:0/3064031798 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7bc0b44d0 0x7ff7bc0b5890 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac0097a0 tx=0x7ff7ac02a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8d50fa7d49cfde1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:25.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.637+0000 7ff7ba7fc640 1 -- 172.21.15.32:0/3064031798 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac037070 con 0x7ff7bc0b44d0 2024-04-24T23:11:25.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.637+0000 7ff7ba7fc640 1 -- 172.21.15.32:0/3064031798 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7ac02ab10 con 0x7ff7bc0b44d0 2024-04-24T23:11:25.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.637+0000 7ff7ba7fc640 1 -- 172.21.15.32:0/3064031798 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b40019f0 con 0x7ff7bc0b44d0 2024-04-24T23:11:25.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.638+0000 7ff7c1fb6640 1 -- 172.21.15.32:0/3064031798 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7bc0b44d0 msgr2=0x7ff7bc0b5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:25.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.638+0000 7ff7c1fb6640 1 --2- 172.21.15.32:0/3064031798 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7bc0b44d0 0x7ff7bc0b5890 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac0097a0 tx=0x7ff7ac02a080 comp rx=0 tx=0).stop 2024-04-24T23:11:25.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.638+0000 7ff7c1fb6640 1 -- 172.21.15.32:0/3064031798 shutdown_connections 2024-04-24T23:11:25.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.638+0000 7ff7c1fb6640 1 --2- 172.21.15.32:0/3064031798 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7bc0b44d0 0x7ff7bc0b5890 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.638+0000 7ff7c1fb6640 1 --2- 172.21.15.32:0/3064031798 >> v2:172.21.15.121:3300/0 conn(0x7ff7bc06c5f0 0x7ff7bc06c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.638+0000 7ff7c1fb6640 1 -- 172.21.15.32:0/3064031798 >> 172.21.15.32:0/3064031798 conn(0x7ff7bc06b110 msgr2=0x7ff7bc06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:25.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.638+0000 7ff7c1fb6640 1 -- 172.21.15.32:0/3064031798 shutdown_connections 2024-04-24T23:11:25.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.638+0000 7ff7c1fb6640 1 -- 172.21.15.32:0/3064031798 wait complete. 2024-04-24T23:11:25.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.639+0000 7ff7c1fb6640 1 Processor -- start 2024-04-24T23:11:25.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.639+0000 7ff7c1fb6640 1 -- start start 2024-04-24T23:11:25.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.640+0000 7ff7c1fb6640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7bc06c5f0 0x7ff7bc151b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:25.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.640+0000 7ff7c1fb6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7bc0af890 0x7ff7bc14c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:25.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.640+0000 7ff7c1fb6640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7bc0b44d0 0x7ff7bc14ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:25.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.640+0000 7ff7c1fb6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff7bc06cf10 con 0x7ff7bc0af890 2024-04-24T23:11:25.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.640+0000 7ff7c1fb6640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff7bc0b3fb0 con 0x7ff7bc06c5f0 2024-04-24T23:11:25.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.640+0000 7ff7c1fb6640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ff7bc0b5dd0 con 0x7ff7bc0b44d0 2024-04-24T23:11:25.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.640+0000 7ff7baffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7bc0af890 0x7ff7bc14c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:25.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.640+0000 7ff7baffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7bc0af890 0x7ff7bc14c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54552/0 (socket says 172.21.15.32:54552) 2024-04-24T23:11:25.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.640+0000 7ff7baffd640 1 -- 172.21.15.32:0/1080317674 learned_addr learned my addr 172.21.15.32:0/1080317674 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:25.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.640+0000 7ff7bbfff640 1 --2- 172.21.15.32:0/1080317674 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7bc0b44d0 0x7ff7bc14ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:25.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.640+0000 7ff7bb7fe640 1 --2- 172.21.15.32:0/1080317674 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7bc06c5f0 0x7ff7bc151b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:25.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.640+0000 7ff7baffd640 1 -- 172.21.15.32:0/1080317674 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7bc0b44d0 msgr2=0x7ff7bc14ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:25.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.640+0000 7ff7baffd640 1 --2- 172.21.15.32:0/1080317674 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7bc0b44d0 0x7ff7bc14ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.640+0000 7ff7baffd640 1 -- 172.21.15.32:0/1080317674 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7bc06c5f0 msgr2=0x7ff7bc151b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:25.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.640+0000 7ff7baffd640 1 --2- 172.21.15.32:0/1080317674 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7bc06c5f0 0x7ff7bc151b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.640+0000 7ff7baffd640 1 -- 172.21.15.32:0/1080317674 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7bc14cdf0 con 0x7ff7bc0af890 2024-04-24T23:11:25.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.640+0000 7ff7bb7fe640 1 --2- 172.21.15.32:0/1080317674 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7bc06c5f0 0x7ff7bc151b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:11:25.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.641+0000 7ff7baffd640 1 --2- 172.21.15.32:0/1080317674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7bc0af890 0x7ff7bc14c680 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac035040 tx=0x7ff7ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:25.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.641+0000 7ff7a3fff640 1 -- 172.21.15.32:0/1080317674 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac037050 con 0x7ff7bc0af890 2024-04-24T23:11:25.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.641+0000 7ff7a3fff640 1 -- 172.21.15.32:0/1080317674 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7ac043070 con 0x7ff7bc0af890 2024-04-24T23:11:25.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.641+0000 7ff7c1fb6640 1 -- 172.21.15.32:0/1080317674 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7bc14d0d0 con 0x7ff7bc0af890 2024-04-24T23:11:25.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.641+0000 7ff7a3fff640 1 -- 172.21.15.32:0/1080317674 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac006e10 con 0x7ff7bc0af890 2024-04-24T23:11:25.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.641+0000 7ff7c1fb6640 1 -- 172.21.15.32:0/1080317674 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7bc14d690 con 0x7ff7bc0af890 2024-04-24T23:11:25.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.642+0000 7ff7c1fb6640 1 -- 172.21.15.32:0/1080317674 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7bc0b5670 con 0x7ff7bc0af890 2024-04-24T23:11:25.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.642+0000 7ff7a3fff640 1 -- 172.21.15.32:0/1080317674 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff7ac03ca10 con 0x7ff7bc0af890 2024-04-24T23:11:25.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.643+0000 7ff7a3fff640 1 --2- 172.21.15.32:0/1080317674 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff7a806be30 0x7ff7a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:25.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.643+0000 7ff7a3fff640 1 -- 172.21.15.32:0/1080317674 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 4943+0+0 (secure 0 0 0) 0x7ff7ac0ab5c0 con 0x7ff7bc0af890 2024-04-24T23:11:25.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.643+0000 7ff7bb7fe640 1 --2- 172.21.15.32:0/1080317674 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff7a806be30 0x7ff7a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:25.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.643+0000 7ff7bb7fe640 1 --2- 172.21.15.32:0/1080317674 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff7a806be30 0x7ff7a806e2f0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0009af0 tx=0x7ff7b0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:25.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.647+0000 7ff7a3fff640 1 -- 172.21.15.32:0/1080317674 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff7ac07a390 con 0x7ff7bc0af890 2024-04-24T23:11:25.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.832+0000 7ff7c1fb6640 1 -- 172.21.15.32:0/1080317674 --> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7ff7bc068410 con 0x7ff7a806be30 2024-04-24T23:11:25.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.833+0000 7ff7a3fff640 1 -- 172.21.15.32:0/1080317674 <== mgr.4108 v2:172.21.15.121:6832/1848157943 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7ff7bc068410 con 0x7ff7a806be30 2024-04-24T23:11:25.835 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 2 to deep-scrub 2024-04-24T23:11:25.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.835+0000 7ff7c1fb6640 1 -- 172.21.15.32:0/1080317674 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff7a806be30 msgr2=0x7ff7a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:25.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.835+0000 7ff7c1fb6640 1 --2- 172.21.15.32:0/1080317674 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff7a806be30 0x7ff7a806e2f0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0009af0 tx=0x7ff7b0008040 comp rx=0 tx=0).stop 2024-04-24T23:11:25.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.835+0000 7ff7c1fb6640 1 -- 172.21.15.32:0/1080317674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7bc0af890 msgr2=0x7ff7bc14c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:25.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.835+0000 7ff7c1fb6640 1 --2- 172.21.15.32:0/1080317674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7bc0af890 0x7ff7bc14c680 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac035040 tx=0x7ff7ac0046c0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.835+0000 7ff7c1fb6640 1 -- 172.21.15.32:0/1080317674 shutdown_connections 2024-04-24T23:11:25.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.835+0000 7ff7c1fb6640 1 --2- 172.21.15.32:0/1080317674 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ff7a806be30 0x7ff7a806e2f0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.835+0000 7ff7c1fb6640 1 --2- 172.21.15.32:0/1080317674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7bc0af890 0x7ff7bc14c680 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.835+0000 7ff7c1fb6640 1 --2- 172.21.15.32:0/1080317674 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7bc0b44d0 0x7ff7bc14ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.835+0000 7ff7c1fb6640 1 --2- 172.21.15.32:0/1080317674 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7bc06c5f0 0x7ff7bc151b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.835+0000 7ff7c1fb6640 1 -- 172.21.15.32:0/1080317674 >> 172.21.15.32:0/1080317674 conn(0x7ff7bc06b110 msgr2=0x7ff7bc15d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:25.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.835+0000 7ff7c1fb6640 1 -- 172.21.15.32:0/1080317674 shutdown_connections 2024-04-24T23:11:25.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.835+0000 7ff7c1fb6640 1 -- 172.21.15.32:0/1080317674 wait complete. 2024-04-24T23:11:25.848 INFO:tasks.ceph:Scrubbing osd.3 2024-04-24T23:11:25.848 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-24T23:11:25.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.951+0000 7f704b916640 1 Processor -- start 2024-04-24T23:11:25.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.951+0000 7f704b916640 1 -- start start 2024-04-24T23:11:25.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.952+0000 7f704b916640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f704415b960 0x7f70440cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:25.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.952+0000 7f704b916640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f704415bde0 con 0x7f704415b580 2024-04-24T23:11:25.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.952+0000 7f704b916640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f704415c0c0 con 0x7f70440b58e0 2024-04-24T23:11:25.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.952+0000 7f704b916640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f704415c9f0 con 0x7f704415b960 2024-04-24T23:11:25.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.952+0000 7f70496b2640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f704415b580 0x7f704406cb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56372/0 (socket says 172.21.15.32:56372) 2024-04-24T23:11:25.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.952+0000 7f70496b2640 1 -- 172.21.15.32:0/21859682 learned_addr learned my addr 172.21.15.32:0/21859682 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:11:25.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.952+0000 7f7048eb1640 1 --2- 172.21.15.32:0/21859682 >> v2:172.21.15.121:3300/0 conn(0x7f704415b960 0x7f70440cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:25.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.952+0000 7f7043fff640 1 -- 172.21.15.32:0/21859682 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2397039236 0 0) 0x7f704415bde0 con 0x7f704415b580 2024-04-24T23:11:25.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.952+0000 7f7043fff640 1 -- 172.21.15.32:0/21859682 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f702c003520 con 0x7f704415b580 2024-04-24T23:11:25.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.953+0000 7f7043fff640 1 -- 172.21.15.32:0/21859682 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3792061137 0 0) 0x7f702c003520 con 0x7f704415b580 2024-04-24T23:11:25.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.953+0000 7f7043fff640 1 -- 172.21.15.32:0/21859682 >> v2:172.21.15.121:3300/0 conn(0x7f704415b960 msgr2=0x7f70440cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:25.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.953+0000 7f7043fff640 1 --2- 172.21.15.32:0/21859682 >> v2:172.21.15.121:3300/0 conn(0x7f704415b960 0x7f70440cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.953+0000 7f7043fff640 1 -- 172.21.15.32:0/21859682 >> v1:172.21.15.121:6789/0 conn(0x7f70440b58e0 legacy=0x7f704415f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:25.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.953+0000 7f7043fff640 1 -- 172.21.15.32:0/21859682 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f704415c3a0 con 0x7f704415b580 2024-04-24T23:11:25.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.953+0000 7f7043fff640 1 -- 172.21.15.32:0/21859682 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f703c001670 con 0x7f704415b580 2024-04-24T23:11:25.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.953+0000 7f7043fff640 1 -- 172.21.15.32:0/21859682 >> v1:172.21.15.32:6789/0 conn(0x7f704415b580 legacy=0x7f704406cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:25.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.953+0000 7f7043fff640 1 --2- 172.21.15.32:0/21859682 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f702c0039b0 0x7f702c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:25.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.953+0000 7f7043fff640 1 --2- 172.21.15.32:0/21859682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f702c004800 0x7f702c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:25.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.953+0000 7f7043fff640 1 --2- 172.21.15.32:0/21859682 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f702c0051c0 0x7f702c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:25.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.953+0000 7f7043fff640 1 -- 172.21.15.32:0/21859682 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f704415c9f0 con 0x7f702c004800 2024-04-24T23:11:25.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.953+0000 7f7043fff640 1 -- 172.21.15.32:0/21859682 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f702c00c060 con 0x7f702c0051c0 2024-04-24T23:11:25.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.953+0000 7f7043fff640 1 -- 172.21.15.32:0/21859682 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f702c00c260 con 0x7f702c0039b0 2024-04-24T23:11:25.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.953+0000 7f7048eb1640 1 --2- 172.21.15.32:0/21859682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f702c004800 0x7f702c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:25.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.953+0000 7f70496b2640 1 --2- 172.21.15.32:0/21859682 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f702c0039b0 0x7f702c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:25.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.953+0000 7f7049eb3640 1 --2- 172.21.15.32:0/21859682 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f702c0051c0 0x7f702c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:25.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.953+0000 7f7048eb1640 1 -- 172.21.15.32:0/21859682 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f702c0039b0 msgr2=0x7f702c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:25.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.953+0000 7f7048eb1640 1 --2- 172.21.15.32:0/21859682 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f702c0039b0 0x7f702c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.953+0000 7f7048eb1640 1 -- 172.21.15.32:0/21859682 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f702c0051c0 msgr2=0x7f702c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:25.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.953+0000 7f7048eb1640 1 --2- 172.21.15.32:0/21859682 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f702c0051c0 0x7f702c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.953+0000 7f7048eb1640 1 -- 172.21.15.32:0/21859682 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f702c003520 con 0x7f702c004800 2024-04-24T23:11:25.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.953+0000 7f7049eb3640 1 --2- 172.21.15.32:0/21859682 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f702c0051c0 0x7f702c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:11:25.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.953+0000 7f7049eb3640 1 -- 172.21.15.32:0/21859682 reap_dead start 2024-04-24T23:11:25.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.954+0000 7f7048eb1640 1 --2- 172.21.15.32:0/21859682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f702c004800 0x7f702c004c80 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f7038009a90 tx=0x7f702c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=877b267c2e4cea5b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:25.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.954+0000 7f7043fff640 1 -- 172.21.15.32:0/21859682 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7038036070 con 0x7f702c004800 2024-04-24T23:11:25.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.954+0000 7f7043fff640 1 -- 172.21.15.32:0/21859682 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7038031400 con 0x7f702c004800 2024-04-24T23:11:25.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.954+0000 7f7043fff640 1 -- 172.21.15.32:0/21859682 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7038009690 con 0x7f702c004800 2024-04-24T23:11:25.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.954+0000 7f704b916640 1 -- 172.21.15.32:0/21859682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f702c004800 msgr2=0x7f702c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:25.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.954+0000 7f704b916640 1 --2- 172.21.15.32:0/21859682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f702c004800 0x7f702c004c80 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f7038009a90 tx=0x7f702c005830 comp rx=0 tx=0).stop 2024-04-24T23:11:25.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.955+0000 7f704b916640 1 -- 172.21.15.32:0/21859682 shutdown_connections 2024-04-24T23:11:25.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.955+0000 7f704b916640 1 --2- 172.21.15.32:0/21859682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f702c004800 0x7f702c004c80 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.955+0000 7f704b916640 1 -- 172.21.15.32:0/21859682 >> 172.21.15.32:0/21859682 conn(0x7f70440b1f10 msgr2=0x7f70440b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:25.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.955+0000 7f704b916640 1 -- 172.21.15.32:0/21859682 shutdown_connections 2024-04-24T23:11:25.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.955+0000 7f704b916640 1 -- 172.21.15.32:0/21859682 wait complete. 2024-04-24T23:11:25.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.956+0000 7f704b916640 1 Processor -- start 2024-04-24T23:11:25.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.956+0000 7f704b916640 1 -- start start 2024-04-24T23:11:25.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.957+0000 7f704b916640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f702c004800 0x7f7044157430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:25.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.957+0000 7f704b916640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7044152270 0x7f704415a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:25.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.957+0000 7f704b916640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7044147540 0x7f70441557b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:25.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.957+0000 7f704b916640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f704406c920 con 0x7f7044147540 2024-04-24T23:11:25.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.957+0000 7f704b916640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f704406c7a0 con 0x7f7044152270 2024-04-24T23:11:25.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.957+0000 7f704b916640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f70440cde80 con 0x7f702c004800 2024-04-24T23:11:25.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.957+0000 7f7049eb3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7044147540 0x7f70441557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:25.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.957+0000 7f7049eb3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7044147540 0x7f70441557b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54566/0 (socket says 172.21.15.32:54566) 2024-04-24T23:11:25.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.957+0000 7f7049eb3640 1 -- 172.21.15.32:0/3183035608 learned_addr learned my addr 172.21.15.32:0/3183035608 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:25.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.957+0000 7f70496b2640 1 --2- 172.21.15.32:0/3183035608 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f702c004800 0x7f7044157430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:25.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.957+0000 7f7048eb1640 1 --2- 172.21.15.32:0/3183035608 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7044152270 0x7f704415a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:25.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.957+0000 7f7049eb3640 1 -- 172.21.15.32:0/3183035608 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f702c004800 msgr2=0x7f7044157430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:25.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.957+0000 7f7049eb3640 1 --2- 172.21.15.32:0/3183035608 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f702c004800 0x7f7044157430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.957+0000 7f7049eb3640 1 -- 172.21.15.32:0/3183035608 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7044152270 msgr2=0x7f704415a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:25.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.957+0000 7f7049eb3640 1 --2- 172.21.15.32:0/3183035608 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7044152270 0x7f704415a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.957+0000 7f7049eb3640 1 -- 172.21.15.32:0/3183035608 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70441514b0 con 0x7f7044147540 2024-04-24T23:11:25.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.957+0000 7f70496b2640 1 --2- 172.21.15.32:0/3183035608 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f702c004800 0x7f7044157430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:11:25.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.957+0000 7f7048eb1640 1 --2- 172.21.15.32:0/3183035608 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7044152270 0x7f704415a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:11:25.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.957+0000 7f7049eb3640 1 --2- 172.21.15.32:0/3183035608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7044147540 0x7f70441557b0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f70440cc410 tx=0x7f703400e8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:25.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.957+0000 7f7041ffb640 1 -- 172.21.15.32:0/3183035608 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7034019070 con 0x7f7044147540 2024-04-24T23:11:25.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.957+0000 7f7041ffb640 1 -- 172.21.15.32:0/3183035608 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7034003b80 con 0x7f7044147540 2024-04-24T23:11:25.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.957+0000 7f7041ffb640 1 -- 172.21.15.32:0/3183035608 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70340050d0 con 0x7f7044147540 2024-04-24T23:11:25.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.957+0000 7f704b916640 1 -- 172.21.15.32:0/3183035608 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7044151790 con 0x7f7044147540 2024-04-24T23:11:25.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.958+0000 7f704b916640 1 -- 172.21.15.32:0/3183035608 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70441479c0 con 0x7f7044147540 2024-04-24T23:11:25.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.958+0000 7f704b916640 1 -- 172.21.15.32:0/3183035608 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f704415a610 con 0x7f7044147540 2024-04-24T23:11:25.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.959+0000 7f7041ffb640 1 -- 172.21.15.32:0/3183035608 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f703401c070 con 0x7f7044147540 2024-04-24T23:11:25.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.959+0000 7f7041ffb640 1 --2- 172.21.15.32:0/3183035608 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f703006be60 0x7f703006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:25.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.959+0000 7f7041ffb640 1 -- 172.21.15.32:0/3183035608 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 4943+0+0 (secure 0 0 0) 0x7f7034089fe0 con 0x7f7044147540 2024-04-24T23:11:25.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.960+0000 7f7041ffb640 1 --2- 172.21.15.32:0/3183035608 >> [v2:172.21.15.32:6818/3381057190,v1:172.21.15.32:6821/3381057190] conn(0x7f7030071d20 0x7f7030074140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:25.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.960+0000 7f7041ffb640 1 -- 172.21.15.32:0/3183035608 --> [v2:172.21.15.32:6818/3381057190,v1:172.21.15.32:6821/3381057190] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f70300747f0 con 0x7f7030071d20 2024-04-24T23:11:25.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.960+0000 7f7041ffb640 1 -- 172.21.15.32:0/3183035608 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=194) ==== 24+0+0 (secure 0 0 0) 0x7f703408c5e0 con 0x7f7044147540 2024-04-24T23:11:25.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.960+0000 7f70496b2640 1 --2- 172.21.15.32:0/3183035608 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f703006be60 0x7f703006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:25.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.960+0000 7f7048eb1640 1 --2- 172.21.15.32:0/3183035608 >> [v2:172.21.15.32:6818/3381057190,v1:172.21.15.32:6821/3381057190] conn(0x7f7030071d20 0x7f7030074140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:25.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.960+0000 7f7048eb1640 1 --2- 172.21.15.32:0/3183035608 >> [v2:172.21.15.32:6818/3381057190,v1:172.21.15.32:6821/3381057190] conn(0x7f7030071d20 0x7f7030074140 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:25.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.960+0000 7f70496b2640 1 --2- 172.21.15.32:0/3183035608 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f703006be60 0x7f703006e320 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f703c007d50 tx=0x7f703c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:25.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.961+0000 7f7041ffb640 1 -- 172.21.15.32:0/3183035608 <== osd.3 v2:172.21.15.32:6818/3381057190 1 ==== command_reply(tid 1: 0 ) ==== 8+0+25033 (crc 0 0 0) 0x7f70300747f0 con 0x7f7030071d20 2024-04-24T23:11:25.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.973+0000 7f704b916640 1 -- 172.21.15.32:0/3183035608 --> [v2:172.21.15.32:6818/3381057190,v1:172.21.15.32:6821/3381057190] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f704415a860 con 0x7f7030071d20 2024-04-24T23:11:25.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.973+0000 7f7041ffb640 1 -- 172.21.15.32:0/3183035608 <== osd.3 v2:172.21.15.32:6818/3381057190 2 ==== command_reply(tid 2: 0 ) ==== 8+0+565 (crc 0 0 0) 0x7f70380315d0 con 0x7f7030071d20 2024-04-24T23:11:25.974 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:11:25.975 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-04-24T23:11:25.975 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:11:25.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.973+0000 7f704b916640 1 -- 172.21.15.32:0/3183035608 >> [v2:172.21.15.32:6818/3381057190,v1:172.21.15.32:6821/3381057190] conn(0x7f7030071d20 msgr2=0x7f7030074140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:25.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.973+0000 7f704b916640 1 --2- 172.21.15.32:0/3183035608 >> [v2:172.21.15.32:6818/3381057190,v1:172.21.15.32:6821/3381057190] conn(0x7f7030071d20 0x7f7030074140 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.973+0000 7f704b916640 1 -- 172.21.15.32:0/3183035608 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f703006be60 msgr2=0x7f703006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:25.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.973+0000 7f704b916640 1 --2- 172.21.15.32:0/3183035608 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f703006be60 0x7f703006e320 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f703c007d50 tx=0x7f703c009040 comp rx=0 tx=0).stop 2024-04-24T23:11:25.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.973+0000 7f704b916640 1 -- 172.21.15.32:0/3183035608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7044147540 msgr2=0x7f70441557b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:25.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.973+0000 7f704b916640 1 --2- 172.21.15.32:0/3183035608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7044147540 0x7f70441557b0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f70440cc410 tx=0x7f703400e8b0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.973+0000 7f7049eb3640 1 -- 172.21.15.32:0/3183035608 reap_dead start 2024-04-24T23:11:25.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.973+0000 7f704b916640 1 -- 172.21.15.32:0/3183035608 shutdown_connections 2024-04-24T23:11:25.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.973+0000 7f704b916640 1 -- 172.21.15.32:0/3183035608 >> 172.21.15.32:0/3183035608 conn(0x7f70440b1f10 msgr2=0x7f704415f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:25.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.973+0000 7f704b916640 1 -- 172.21.15.32:0/3183035608 shutdown_connections 2024-04-24T23:11:25.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:25.973+0000 7f704b916640 1 -- 172.21.15.32:0/3183035608 wait complete. 2024-04-24T23:11:25.985 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-24T23:11:26.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.085+0000 7ffa49693640 1 Processor -- start 2024-04-24T23:11:26.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.085+0000 7ffa49693640 1 -- start start 2024-04-24T23:11:26.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.085+0000 7ffa49693640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ffa44074210 0x7ffa4406c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:26.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.085+0000 7ffa49693640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa440ae140 0x7ffa440b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:26.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.085+0000 7ffa49693640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa440b0770 con 0x7ffa440b0b80 2024-04-24T23:11:26.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.085+0000 7ffa49693640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7ffa440b1c20 con 0x7ffa44074210 2024-04-24T23:11:26.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.085+0000 7ffa49693640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ffa440b1010 con 0x7ffa440ae140 2024-04-24T23:11:26.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.085+0000 7ffa427fc640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ffa440b0b80 0x7ffa440745f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56386/0 (socket says 172.21.15.32:56386) 2024-04-24T23:11:26.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.086+0000 7ffa427fc640 1 -- 172.21.15.32:0/1557160770 learned_addr learned my addr 172.21.15.32:0/1557160770 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:11:26.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.086+0000 7ffa42ffd640 1 --2- 172.21.15.32:0/1557160770 >> v2:172.21.15.121:3300/0 conn(0x7ffa44074210 0x7ffa4406c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:26.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.086+0000 7ffa437fe640 1 --2- 172.21.15.32:0/1557160770 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa440ae140 0x7ffa440b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:26.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.086+0000 7ffa41ffb640 1 -- 172.21.15.32:0/1557160770 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 357891475 0 0) 0x7ffa440b0770 con 0x7ffa440b0b80 2024-04-24T23:11:26.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.086+0000 7ffa41ffb640 1 -- 172.21.15.32:0/1557160770 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa30003520 con 0x7ffa440b0b80 2024-04-24T23:11:26.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.086+0000 7ffa41ffb640 1 -- 172.21.15.32:0/1557160770 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4140635800 0 0) 0x7ffa30003520 con 0x7ffa440b0b80 2024-04-24T23:11:26.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.086+0000 7ffa41ffb640 1 -- 172.21.15.32:0/1557160770 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa440ae140 msgr2=0x7ffa440b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:26.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.086+0000 7ffa41ffb640 1 --2- 172.21.15.32:0/1557160770 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa440ae140 0x7ffa440b20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:26.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.086+0000 7ffa41ffb640 1 -- 172.21.15.32:0/1557160770 >> v2:172.21.15.121:3300/0 conn(0x7ffa44074210 msgr2=0x7ffa4406c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:26.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.086+0000 7ffa41ffb640 1 --2- 172.21.15.32:0/1557160770 >> v2:172.21.15.121:3300/0 conn(0x7ffa44074210 0x7ffa4406c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:26.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.086+0000 7ffa41ffb640 1 -- 172.21.15.32:0/1557160770 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa440b12f0 con 0x7ffa440b0b80 2024-04-24T23:11:26.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.086+0000 7ffa41ffb640 1 -- 172.21.15.32:0/1557160770 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7ffa2c001670 con 0x7ffa440b0b80 2024-04-24T23:11:26.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.086+0000 7ffa41ffb640 1 -- 172.21.15.32:0/1557160770 >> v1:172.21.15.32:6789/0 conn(0x7ffa440b0b80 legacy=0x7ffa440745f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:26.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.086+0000 7ffa41ffb640 1 --2- 172.21.15.32:0/1557160770 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa440ae140 0x7ffa300039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:26.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.086+0000 7ffa41ffb640 1 --2- 172.21.15.32:0/1557160770 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa30003f30 0x7ffa30004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:26.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.086+0000 7ffa41ffb640 1 --2- 172.21.15.32:0/1557160770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa30004d40 0x7ffa300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:26.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.086+0000 7ffa41ffb640 1 -- 172.21.15.32:0/1557160770 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ffa440b1c20 con 0x7ffa30004d40 2024-04-24T23:11:26.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.086+0000 7ffa41ffb640 1 -- 172.21.15.32:0/1557160770 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ffa440b1010 con 0x7ffa30003f30 2024-04-24T23:11:26.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.086+0000 7ffa41ffb640 1 -- 172.21.15.32:0/1557160770 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ffa3000b9c0 con 0x7ffa440ae140 2024-04-24T23:11:26.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.086+0000 7ffa437fe640 1 --2- 172.21.15.32:0/1557160770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa30004d40 0x7ffa300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:26.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.086+0000 7ffa42ffd640 1 --2- 172.21.15.32:0/1557160770 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa440ae140 0x7ffa300039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:26.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.086+0000 7ffa427fc640 1 --2- 172.21.15.32:0/1557160770 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa30003f30 0x7ffa30004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:26.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.087+0000 7ffa437fe640 1 -- 172.21.15.32:0/1557160770 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa440ae140 msgr2=0x7ffa300039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:26.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.087+0000 7ffa437fe640 1 --2- 172.21.15.32:0/1557160770 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa440ae140 0x7ffa300039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:26.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.087+0000 7ffa437fe640 1 -- 172.21.15.32:0/1557160770 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa30003f30 msgr2=0x7ffa30004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:26.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.087+0000 7ffa437fe640 1 --2- 172.21.15.32:0/1557160770 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa30003f30 0x7ffa30004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:26.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.087+0000 7ffa437fe640 1 -- 172.21.15.32:0/1557160770 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa30003520 con 0x7ffa30004d40 2024-04-24T23:11:26.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.087+0000 7ffa437fe640 1 --2- 172.21.15.32:0/1557160770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa30004d40 0x7ffa300051c0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7ffa34009fc0 tx=0x7ffa30005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f0e6754590a7d6f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:26.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.087+0000 7ffa41ffb640 1 -- 172.21.15.32:0/1557160770 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa34009c10 con 0x7ffa30004d40 2024-04-24T23:11:26.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.088+0000 7ffa41ffb640 1 -- 172.21.15.32:0/1557160770 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa34036070 con 0x7ffa30004d40 2024-04-24T23:11:26.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.088+0000 7ffa41ffb640 1 -- 172.21.15.32:0/1557160770 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa34031670 con 0x7ffa30004d40 2024-04-24T23:11:26.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.088+0000 7ffa49693640 1 -- 172.21.15.32:0/1557160770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa30004d40 msgr2=0x7ffa300051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:26.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.088+0000 7ffa49693640 1 --2- 172.21.15.32:0/1557160770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa30004d40 0x7ffa300051c0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7ffa34009fc0 tx=0x7ffa30005830 comp rx=0 tx=0).stop 2024-04-24T23:11:26.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.088+0000 7ffa437fe640 1 -- 172.21.15.32:0/1557160770 reap_dead start 2024-04-24T23:11:26.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.088+0000 7ffa49693640 1 -- 172.21.15.32:0/1557160770 shutdown_connections 2024-04-24T23:11:26.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.088+0000 7ffa49693640 1 -- 172.21.15.32:0/1557160770 >> 172.21.15.32:0/1557160770 conn(0x7ffa440b2e60 msgr2=0x7ffa440b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:26.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.088+0000 7ffa49693640 1 -- 172.21.15.32:0/1557160770 shutdown_connections 2024-04-24T23:11:26.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.088+0000 7ffa49693640 1 -- 172.21.15.32:0/1557160770 wait complete. 2024-04-24T23:11:26.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.089+0000 7ffa49693640 1 Processor -- start 2024-04-24T23:11:26.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.089+0000 7ffa49693640 1 -- start start 2024-04-24T23:11:26.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.090+0000 7ffa49693640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa4415e0b0 0x7ffa4409d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:26.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.090+0000 7ffa49693640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa4415e490 0x7ffa441f45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:26.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.090+0000 7ffa49693640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa441f4b10 0x7ffa440c7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:26.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.090+0000 7ffa49693640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ffa440b3ba0 con 0x7ffa4415e0b0 2024-04-24T23:11:26.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.090+0000 7ffa49693640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ffa440b18b0 con 0x7ffa441f4b10 2024-04-24T23:11:26.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.090+0000 7ffa49693640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ffa440b3d20 con 0x7ffa4415e490 2024-04-24T23:11:26.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.090+0000 7ffa42ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa4415e0b0 0x7ffa4409d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:26.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.090+0000 7ffa42ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa4415e0b0 0x7ffa4409d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54580/0 (socket says 172.21.15.32:54580) 2024-04-24T23:11:26.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.090+0000 7ffa42ffd640 1 -- 172.21.15.32:0/1603829956 learned_addr learned my addr 172.21.15.32:0/1603829956 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:26.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.090+0000 7ffa437fe640 1 --2- 172.21.15.32:0/1603829956 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa441f4b10 0x7ffa440c7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:26.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.090+0000 7ffa427fc640 1 --2- 172.21.15.32:0/1603829956 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa4415e490 0x7ffa441f45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:26.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.090+0000 7ffa42ffd640 1 -- 172.21.15.32:0/1603829956 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa4415e490 msgr2=0x7ffa441f45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:26.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.090+0000 7ffa42ffd640 1 --2- 172.21.15.32:0/1603829956 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa4415e490 0x7ffa441f45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:26.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.091+0000 7ffa42ffd640 1 -- 172.21.15.32:0/1603829956 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa441f4b10 msgr2=0x7ffa440c7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:26.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.091+0000 7ffa42ffd640 1 --2- 172.21.15.32:0/1603829956 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa441f4b10 0x7ffa440c7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:26.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.091+0000 7ffa42ffd640 1 -- 172.21.15.32:0/1603829956 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa441f5110 con 0x7ffa4415e0b0 2024-04-24T23:11:26.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.091+0000 7ffa437fe640 1 --2- 172.21.15.32:0/1603829956 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa441f4b10 0x7ffa440c7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:11:26.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.091+0000 7ffa427fc640 1 --2- 172.21.15.32:0/1603829956 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa4415e490 0x7ffa441f45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:11:26.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.091+0000 7ffa42ffd640 1 --2- 172.21.15.32:0/1603829956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa4415e0b0 0x7ffa4409d420 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c011aa0 tx=0x7ffa3c011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:26.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.091+0000 7ffa3b7fe640 1 -- 172.21.15.32:0/1603829956 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c00fb80 con 0x7ffa4415e0b0 2024-04-24T23:11:26.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.091+0000 7ffa3b7fe640 1 -- 172.21.15.32:0/1603829956 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa3c00fd20 con 0x7ffa4415e0b0 2024-04-24T23:11:26.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.091+0000 7ffa49693640 1 -- 172.21.15.32:0/1603829956 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa441f53f0 con 0x7ffa4415e0b0 2024-04-24T23:11:26.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.091+0000 7ffa3b7fe640 1 -- 172.21.15.32:0/1603829956 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c014660 con 0x7ffa4415e0b0 2024-04-24T23:11:26.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.091+0000 7ffa49693640 1 -- 172.21.15.32:0/1603829956 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa44161530 con 0x7ffa4415e0b0 2024-04-24T23:11:26.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.092+0000 7ffa49693640 1 -- 172.21.15.32:0/1603829956 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa44068430 con 0x7ffa4415e0b0 2024-04-24T23:11:26.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.092+0000 7ffa3b7fe640 1 -- 172.21.15.32:0/1603829956 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffa3c012040 con 0x7ffa4415e0b0 2024-04-24T23:11:26.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.093+0000 7ffa3b7fe640 1 --2- 172.21.15.32:0/1603829956 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ffa2406be30 0x7ffa2406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:26.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.093+0000 7ffa3b7fe640 1 -- 172.21.15.32:0/1603829956 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 4943+0+0 (secure 0 0 0) 0x7ffa3c08b330 con 0x7ffa4415e0b0 2024-04-24T23:11:26.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.093+0000 7ffa427fc640 1 --2- 172.21.15.32:0/1603829956 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ffa2406be30 0x7ffa2406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:26.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.094+0000 7ffa427fc640 1 --2- 172.21.15.32:0/1603829956 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ffa2406be30 0x7ffa2406e2f0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7ffa4415e8b0 tx=0x7ffa2c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:26.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.097+0000 7ffa3b7fe640 1 -- 172.21.15.32:0/1603829956 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ffa3c019030 con 0x7ffa4415e0b0 2024-04-24T23:11:26.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.286+0000 7ffa49693640 1 -- 172.21.15.32:0/1603829956 --> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7ffa4406b220 con 0x7ffa2406be30 2024-04-24T23:11:26.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.286+0000 7ffa3b7fe640 1 -- 172.21.15.32:0/1603829956 <== mgr.4108 v2:172.21.15.121:6832/1848157943 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7ffa4406b220 con 0x7ffa2406be30 2024-04-24T23:11:26.288 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 3 to deep-scrub 2024-04-24T23:11:26.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.289+0000 7ffa49693640 1 -- 172.21.15.32:0/1603829956 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ffa2406be30 msgr2=0x7ffa2406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:26.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.289+0000 7ffa49693640 1 --2- 172.21.15.32:0/1603829956 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ffa2406be30 0x7ffa2406e2f0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7ffa4415e8b0 tx=0x7ffa2c009040 comp rx=0 tx=0).stop 2024-04-24T23:11:26.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.289+0000 7ffa49693640 1 -- 172.21.15.32:0/1603829956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa4415e0b0 msgr2=0x7ffa4409d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:26.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.289+0000 7ffa49693640 1 --2- 172.21.15.32:0/1603829956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa4415e0b0 0x7ffa4409d420 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c011aa0 tx=0x7ffa3c011f60 comp rx=0 tx=0).stop 2024-04-24T23:11:26.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.289+0000 7ffa49693640 1 -- 172.21.15.32:0/1603829956 shutdown_connections 2024-04-24T23:11:26.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.289+0000 7ffa49693640 1 --2- 172.21.15.32:0/1603829956 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ffa2406be30 0x7ffa2406e2f0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:26.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.289+0000 7ffa49693640 1 --2- 172.21.15.32:0/1603829956 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa441f4b10 0x7ffa440c7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:26.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.289+0000 7ffa49693640 1 --2- 172.21.15.32:0/1603829956 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa4415e490 0x7ffa441f45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:26.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.289+0000 7ffa49693640 1 --2- 172.21.15.32:0/1603829956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa4415e0b0 0x7ffa4409d420 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:26.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.289+0000 7ffa49693640 1 -- 172.21.15.32:0/1603829956 >> 172.21.15.32:0/1603829956 conn(0x7ffa440b2e60 msgr2=0x7ffa4406e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:26.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.289+0000 7ffa49693640 1 -- 172.21.15.32:0/1603829956 shutdown_connections 2024-04-24T23:11:26.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.289+0000 7ffa49693640 1 -- 172.21.15.32:0/1603829956 wait complete. 2024-04-24T23:11:26.301 INFO:tasks.ceph:Scrubbing osd.4 2024-04-24T23:11:26.301 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-24T23:11:26.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.405+0000 7ffbf3a5d640 1 Processor -- start 2024-04-24T23:11:26.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.405+0000 7ffbf3a5d640 1 -- start start 2024-04-24T23:11:26.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.406+0000 7ffbf3a5d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffbec15b580 0x7ffbec0cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:26.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.406+0000 7ffbf3a5d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffbec15b960 0x7ffbec06cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:26.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.406+0000 7ffbf3a5d640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ffbec0b58e0 0x7ffbec0b5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:26.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.406+0000 7ffbf3a5d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7ffbec0b6270 con 0x7ffbec15b960 2024-04-24T23:11:26.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.406+0000 7ffbf3a5d640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7ffbec15c9f0 con 0x7ffbec0b58e0 2024-04-24T23:11:26.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.406+0000 7ffbf3a5d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ffbec0ce3c0 con 0x7ffbec15b580 2024-04-24T23:11:26.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.406+0000 7ffbf0ff8640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffbec15b960 0x7ffbec06cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:26.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.406+0000 7ffbf0ff8640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffbec15b960 0x7ffbec06cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54588/0 (socket says 172.21.15.32:54588) 2024-04-24T23:11:26.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.406+0000 7ffbf0ff8640 1 -- 172.21.15.32:0/1578484219 learned_addr learned my addr 172.21.15.32:0/1578484219 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:26.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.406+0000 7ffbf17f9640 1 --2- 172.21.15.32:0/1578484219 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffbec15b580 0x7ffbec0cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:26.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.406+0000 7ffbf1ffa640 1 --2- 172.21.15.32:0/1578484219 >> v2:172.21.15.121:3300/0 conn(0x7ffbec0b58e0 0x7ffbec0b5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:26.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.406+0000 7ffbf0ff8640 1 -- 172.21.15.32:0/1578484219 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffbec15b580 msgr2=0x7ffbec0cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:26.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.406+0000 7ffbf0ff8640 1 --2- 172.21.15.32:0/1578484219 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffbec15b580 0x7ffbec0cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:26.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.406+0000 7ffbf0ff8640 1 -- 172.21.15.32:0/1578484219 >> v2:172.21.15.121:3300/0 conn(0x7ffbec0b58e0 msgr2=0x7ffbec0b5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:26.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.406+0000 7ffbf0ff8640 1 --2- 172.21.15.32:0/1578484219 >> v2:172.21.15.121:3300/0 conn(0x7ffbec0b58e0 0x7ffbec0b5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:26.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.406+0000 7ffbf0ff8640 1 -- 172.21.15.32:0/1578484219 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbec15bde0 con 0x7ffbec15b960 2024-04-24T23:11:26.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.407+0000 7ffbf0ff8640 1 --2- 172.21.15.32:0/1578484219 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbec15b960 0x7ffbec06cb50 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7ffbec0cc980 tx=0x7ffbdc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa4a90bd01a26081 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:26.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.407+0000 7ffbe3fff640 1 -- 172.21.15.32:0/1578484219 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc037070 con 0x7ffbec15b960 2024-04-24T23:11:26.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.407+0000 7ffbe3fff640 1 -- 172.21.15.32:0/1578484219 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbdc032440 con 0x7ffbec15b960 2024-04-24T23:11:26.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.407+0000 7ffbe3fff640 1 -- 172.21.15.32:0/1578484219 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc032810 con 0x7ffbec15b960 2024-04-24T23:11:26.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.408+0000 7ffbf3a5d640 1 -- 172.21.15.32:0/1578484219 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbec15b960 msgr2=0x7ffbec06cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:26.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.408+0000 7ffbf3a5d640 1 --2- 172.21.15.32:0/1578484219 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbec15b960 0x7ffbec06cb50 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7ffbec0cc980 tx=0x7ffbdc02a1e0 comp rx=0 tx=0).stop 2024-04-24T23:11:26.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.408+0000 7ffbf3a5d640 1 -- 172.21.15.32:0/1578484219 shutdown_connections 2024-04-24T23:11:26.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.408+0000 7ffbf3a5d640 1 --2- 172.21.15.32:0/1578484219 >> v2:172.21.15.121:3300/0 conn(0x7ffbec0b58e0 0x7ffbec0b5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:26.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.408+0000 7ffbf3a5d640 1 --2- 172.21.15.32:0/1578484219 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbec15b960 0x7ffbec06cb50 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:26.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.408+0000 7ffbf3a5d640 1 --2- 172.21.15.32:0/1578484219 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffbec15b580 0x7ffbec0cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:26.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.408+0000 7ffbf3a5d640 1 -- 172.21.15.32:0/1578484219 >> 172.21.15.32:0/1578484219 conn(0x7ffbec0b1f10 msgr2=0x7ffbec0b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:26.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.408+0000 7ffbf3a5d640 1 -- 172.21.15.32:0/1578484219 shutdown_connections 2024-04-24T23:11:26.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.408+0000 7ffbf3a5d640 1 -- 172.21.15.32:0/1578484219 wait complete. 2024-04-24T23:11:26.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.409+0000 7ffbf3a5d640 1 Processor -- start 2024-04-24T23:11:26.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.409+0000 7ffbf3a5d640 1 -- start start 2024-04-24T23:11:26.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.410+0000 7ffbf3a5d640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffbec0b58e0 0x7ffbec146cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:26.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.410+0000 7ffbf3a5d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbec15b580 0x7ffbec151440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:26.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.410+0000 7ffbf3a5d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffbec15b960 0x7ffbec0ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:26.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.410+0000 7ffbf3a5d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ffbec15c680 con 0x7ffbec15b580 2024-04-24T23:11:26.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.410+0000 7ffbf3a5d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ffbec15c3a0 con 0x7ffbec15b960 2024-04-24T23:11:26.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.410+0000 7ffbf3a5d640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7ffbec06c7a0 con 0x7ffbec0b58e0 2024-04-24T23:11:26.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.410+0000 7ffbf0ff8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbec15b580 0x7ffbec151440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:26.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.410+0000 7ffbf0ff8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbec15b580 0x7ffbec151440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54590/0 (socket says 172.21.15.32:54590) 2024-04-24T23:11:26.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.410+0000 7ffbf0ff8640 1 -- 172.21.15.32:0/737957966 learned_addr learned my addr 172.21.15.32:0/737957966 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:26.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.410+0000 7ffbf17f9640 1 --2- 172.21.15.32:0/737957966 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffbec0b58e0 0x7ffbec146cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:26.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.410+0000 7ffbf1ffa640 1 --2- 172.21.15.32:0/737957966 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffbec15b960 0x7ffbec0ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:26.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.410+0000 7ffbf0ff8640 1 -- 172.21.15.32:0/737957966 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffbec0b58e0 msgr2=0x7ffbec146cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:26.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.410+0000 7ffbf0ff8640 1 --2- 172.21.15.32:0/737957966 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffbec0b58e0 0x7ffbec146cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:26.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.410+0000 7ffbf0ff8640 1 -- 172.21.15.32:0/737957966 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffbec15b960 msgr2=0x7ffbec0ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:26.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.410+0000 7ffbf0ff8640 1 --2- 172.21.15.32:0/737957966 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffbec15b960 0x7ffbec0ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:26.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.410+0000 7ffbf0ff8640 1 -- 172.21.15.32:0/737957966 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbec151f40 con 0x7ffbec15b580 2024-04-24T23:11:26.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.410+0000 7ffbf0ff8640 1 --2- 172.21.15.32:0/737957966 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbec15b580 0x7ffbec151440 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc02a220 tx=0x7ffbdc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:26.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.410+0000 7ffbe1ffb640 1 -- 172.21.15.32:0/737957966 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc041020 con 0x7ffbec15b580 2024-04-24T23:11:26.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.410+0000 7ffbe1ffb640 1 -- 172.21.15.32:0/737957966 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbdc03d620 con 0x7ffbec15b580 2024-04-24T23:11:26.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.410+0000 7ffbf3a5d640 1 -- 172.21.15.32:0/737957966 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbec152220 con 0x7ffbec15b580 2024-04-24T23:11:26.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.410+0000 7ffbe1ffb640 1 -- 172.21.15.32:0/737957966 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc037040 con 0x7ffbec15b580 2024-04-24T23:11:26.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.411+0000 7ffbf3a5d640 1 -- 172.21.15.32:0/737957966 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbec1527e0 con 0x7ffbec15b580 2024-04-24T23:11:26.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.411+0000 7ffbf3a5d640 1 -- 172.21.15.32:0/737957966 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffbec068430 con 0x7ffbec15b580 2024-04-24T23:11:26.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.412+0000 7ffbe1ffb640 1 -- 172.21.15.32:0/737957966 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffbdc044020 con 0x7ffbec15b580 2024-04-24T23:11:26.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.412+0000 7ffbe1ffb640 1 --2- 172.21.15.32:0/737957966 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ffbd006be30 0x7ffbd006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:26.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.413+0000 7ffbe1ffb640 1 -- 172.21.15.32:0/737957966 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 4943+0+0 (secure 0 0 0) 0x7ffbdc0ab240 con 0x7ffbec15b580 2024-04-24T23:11:26.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.413+0000 7ffbf17f9640 1 --2- 172.21.15.32:0/737957966 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ffbd006be30 0x7ffbd006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:26.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.413+0000 7ffbe1ffb640 1 --2- 172.21.15.32:0/737957966 >> [v2:172.21.15.121:6800/4010553979,v1:172.21.15.121:6801/4010553979] conn(0x7ffbd0071cf0 0x7ffbd0074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:26.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.413+0000 7ffbe1ffb640 1 -- 172.21.15.32:0/737957966 --> [v2:172.21.15.121:6800/4010553979,v1:172.21.15.121:6801/4010553979] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffbdc02cea0 con 0x7ffbd0071cf0 2024-04-24T23:11:26.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.413+0000 7ffbe1ffb640 1 -- 172.21.15.32:0/737957966 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=194) ==== 24+0+0 (secure 0 0 0) 0x7ffbdc0b2050 con 0x7ffbec15b580 2024-04-24T23:11:26.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.413+0000 7ffbf1ffa640 1 --2- 172.21.15.32:0/737957966 >> [v2:172.21.15.121:6800/4010553979,v1:172.21.15.121:6801/4010553979] conn(0x7ffbd0071cf0 0x7ffbd0074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:26.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.413+0000 7ffbf17f9640 1 --2- 172.21.15.32:0/737957966 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ffbd006be30 0x7ffbd006e2f0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7ffbe8009ad0 tx=0x7ffbe8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:26.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.413+0000 7ffbf1ffa640 1 --2- 172.21.15.32:0/737957966 >> [v2:172.21.15.121:6800/4010553979,v1:172.21.15.121:6801/4010553979] conn(0x7ffbd0071cf0 0x7ffbd0074110 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:26.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.414+0000 7ffbe1ffb640 1 -- 172.21.15.32:0/737957966 <== osd.4 v2:172.21.15.121:6800/4010553979 1 ==== command_reply(tid 1: 0 ) ==== 8+0+25033 (crc 0 0 0) 0x7ffbdc02cea0 con 0x7ffbd0071cf0 2024-04-24T23:11:26.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.426+0000 7ffbf3a5d640 1 -- 172.21.15.32:0/737957966 --> [v2:172.21.15.121:6800/4010553979,v1:172.21.15.121:6801/4010553979] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ffbec0af4f0 con 0x7ffbd0071cf0 2024-04-24T23:11:26.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.426+0000 7ffbe1ffb640 1 -- 172.21.15.32:0/737957966 <== osd.4 v2:172.21.15.121:6800/4010553979 2 ==== command_reply(tid 2: 0 ) ==== 8+0+565 (crc 0 0 0) 0x7ffbec0af4f0 con 0x7ffbd0071cf0 2024-04-24T23:11:26.428 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:11:26.428 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-04-24T23:11:26.428 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:11:26.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.427+0000 7ffbf3a5d640 1 -- 172.21.15.32:0/737957966 >> [v2:172.21.15.121:6800/4010553979,v1:172.21.15.121:6801/4010553979] conn(0x7ffbd0071cf0 msgr2=0x7ffbd0074110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:26.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.427+0000 7ffbf3a5d640 1 --2- 172.21.15.32:0/737957966 >> [v2:172.21.15.121:6800/4010553979,v1:172.21.15.121:6801/4010553979] conn(0x7ffbd0071cf0 0x7ffbd0074110 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:26.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.427+0000 7ffbf3a5d640 1 -- 172.21.15.32:0/737957966 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ffbd006be30 msgr2=0x7ffbd006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:26.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.427+0000 7ffbf3a5d640 1 --2- 172.21.15.32:0/737957966 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7ffbd006be30 0x7ffbd006e2f0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7ffbe8009ad0 tx=0x7ffbe8008040 comp rx=0 tx=0).stop 2024-04-24T23:11:26.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.427+0000 7ffbf3a5d640 1 -- 172.21.15.32:0/737957966 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbec15b580 msgr2=0x7ffbec151440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:26.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.427+0000 7ffbf3a5d640 1 --2- 172.21.15.32:0/737957966 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbec15b580 0x7ffbec151440 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc02a220 tx=0x7ffbdc0046c0 comp rx=0 tx=0).stop 2024-04-24T23:11:26.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.427+0000 7ffbf1ffa640 1 -- 172.21.15.32:0/737957966 reap_dead start 2024-04-24T23:11:26.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.427+0000 7ffbf3a5d640 1 -- 172.21.15.32:0/737957966 shutdown_connections 2024-04-24T23:11:26.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.427+0000 7ffbf3a5d640 1 -- 172.21.15.32:0/737957966 >> 172.21.15.32:0/737957966 conn(0x7ffbec0b1f10 msgr2=0x7ffbec0ce540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:26.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.427+0000 7ffbf3a5d640 1 -- 172.21.15.32:0/737957966 shutdown_connections 2024-04-24T23:11:26.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.427+0000 7ffbf3a5d640 1 -- 172.21.15.32:0/737957966 wait complete. 2024-04-24T23:11:26.439 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-24T23:11:26.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.540+0000 7f7111596640 1 Processor -- start 2024-04-24T23:11:26.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.540+0000 7f7111596640 1 -- start start 2024-04-24T23:11:26.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.541+0000 7f7111596640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f710c06c5f0 0x7f710c06c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:26.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.541+0000 7f7111596640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f710c0b6430 con 0x7f710c0b44d0 2024-04-24T23:11:26.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.541+0000 7f7111596640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f710c156f80 con 0x7f710c0af890 2024-04-24T23:11:26.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.541+0000 7f7111596640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f710c0b6150 con 0x7f710c06c5f0 2024-04-24T23:11:26.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.541+0000 7f710affd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f710c06c5f0 0x7f710c06c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:26.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.541+0000 7f7103fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f710c0b44d0 0x7f710c0b5890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56392/0 (socket says 172.21.15.32:56392) 2024-04-24T23:11:26.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.541+0000 7f7103fff640 1 -- 172.21.15.32:0/575432064 learned_addr learned my addr 172.21.15.32:0/575432064 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:11:26.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.541+0000 7f710affd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f710c06c5f0 0x7f710c06c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54602/0 (socket says 172.21.15.32:54602) 2024-04-24T23:11:26.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.541+0000 7f710affd640 1 -- 172.21.15.32:0/575432064 >> v1:172.21.15.121:6789/0 conn(0x7f710c0af890 legacy=0x7f710c15cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:26.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.541+0000 7f710affd640 1 -- 172.21.15.32:0/575432064 >> v1:172.21.15.32:6789/0 conn(0x7f710c0b44d0 legacy=0x7f710c0b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:26.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.541+0000 7f710a7fc640 1 -- 172.21.15.32:0/575432064 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2243377747 0 0) 0x7f710c0b6430 con 0x7f710c0b44d0 2024-04-24T23:11:26.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.541+0000 7f710affd640 1 -- 172.21.15.32:0/575432064 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f710c0b39f0 con 0x7f710c06c5f0 2024-04-24T23:11:26.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.542+0000 7f710affd640 1 --2- 172.21.15.32:0/575432064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f710c06c5f0 0x7f710c06c9d0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f70fc0097a0 tx=0x7f70fc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0575e970eac54e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:26.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.542+0000 7f710a7fc640 1 -- 172.21.15.32:0/575432064 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70fc037070 con 0x7f710c06c5f0 2024-04-24T23:11:26.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.542+0000 7f710a7fc640 1 -- 172.21.15.32:0/575432064 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70fc032440 con 0x7f710c06c5f0 2024-04-24T23:11:26.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.542+0000 7f710a7fc640 1 -- 172.21.15.32:0/575432064 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70fc032810 con 0x7f710c06c5f0 2024-04-24T23:11:26.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.543+0000 7f7111596640 1 -- 172.21.15.32:0/575432064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f710c06c5f0 msgr2=0x7f710c06c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:26.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.543+0000 7f7111596640 1 --2- 172.21.15.32:0/575432064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f710c06c5f0 0x7f710c06c9d0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f70fc0097a0 tx=0x7f70fc02a1e0 comp rx=0 tx=0).stop 2024-04-24T23:11:26.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.543+0000 7f7111596640 1 -- 172.21.15.32:0/575432064 shutdown_connections 2024-04-24T23:11:26.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.543+0000 7f7111596640 1 --2- 172.21.15.32:0/575432064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f710c06c5f0 0x7f710c06c9d0 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:26.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.543+0000 7f7111596640 1 -- 172.21.15.32:0/575432064 >> 172.21.15.32:0/575432064 conn(0x7f710c06b110 msgr2=0x7f710c06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:26.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.543+0000 7f7111596640 1 -- 172.21.15.32:0/575432064 shutdown_connections 2024-04-24T23:11:26.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.543+0000 7f7111596640 1 -- 172.21.15.32:0/575432064 wait complete. 2024-04-24T23:11:26.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.544+0000 7f7111596640 1 Processor -- start 2024-04-24T23:11:26.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.544+0000 7f7111596640 1 -- start start 2024-04-24T23:11:26.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.545+0000 7f7111596640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f710c06c5f0 0x7f710c154c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:26.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.545+0000 7f7111596640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f710c0af890 0x7f710c1f4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:26.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.545+0000 7f7111596640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f710c0b44d0 0x7f710c15aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:26.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.545+0000 7f7111596640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f710c06cf10 con 0x7f710c0b44d0 2024-04-24T23:11:26.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.545+0000 7f7111596640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f710c0b3fb0 con 0x7f710c0af890 2024-04-24T23:11:26.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.545+0000 7f7111596640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f710c15d1d0 con 0x7f710c06c5f0 2024-04-24T23:11:26.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.545+0000 7f710b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f710c0b44d0 0x7f710c15aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:26.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.545+0000 7f710b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f710c0b44d0 0x7f710c15aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54614/0 (socket says 172.21.15.32:54614) 2024-04-24T23:11:26.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.545+0000 7f710b7fe640 1 -- 172.21.15.32:0/3979839134 learned_addr learned my addr 172.21.15.32:0/3979839134 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:26.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.545+0000 7f7103fff640 1 --2- 172.21.15.32:0/3979839134 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f710c0af890 0x7f710c1f4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:26.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.545+0000 7f710affd640 1 --2- 172.21.15.32:0/3979839134 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f710c06c5f0 0x7f710c154c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:26.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.545+0000 7f710b7fe640 1 -- 172.21.15.32:0/3979839134 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f710c06c5f0 msgr2=0x7f710c154c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:26.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.545+0000 7f710b7fe640 1 --2- 172.21.15.32:0/3979839134 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f710c06c5f0 0x7f710c154c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:26.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.545+0000 7f710b7fe640 1 -- 172.21.15.32:0/3979839134 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f710c0af890 msgr2=0x7f710c1f4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:26.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.545+0000 7f710b7fe640 1 --2- 172.21.15.32:0/3979839134 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f710c0af890 0x7f710c1f4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:26.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.545+0000 7f710b7fe640 1 -- 172.21.15.32:0/3979839134 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f710c15b940 con 0x7f710c0b44d0 2024-04-24T23:11:26.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.545+0000 7f710b7fe640 1 --2- 172.21.15.32:0/3979839134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f710c0b44d0 0x7f710c15aec0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f710400c780 tx=0x7f710400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:26.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.545+0000 7f71037fe640 1 -- 172.21.15.32:0/3979839134 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7104004b60 con 0x7f710c0b44d0 2024-04-24T23:11:26.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.545+0000 7f71037fe640 1 -- 172.21.15.32:0/3979839134 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7104004d00 con 0x7f710c0b44d0 2024-04-24T23:11:26.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.545+0000 7f7111596640 1 -- 172.21.15.32:0/3979839134 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f710c15bc20 con 0x7f710c0b44d0 2024-04-24T23:11:26.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.545+0000 7f71037fe640 1 -- 172.21.15.32:0/3979839134 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710400aa90 con 0x7f710c0b44d0 2024-04-24T23:11:26.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.545+0000 7f7111596640 1 -- 172.21.15.32:0/3979839134 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f710c0cdcf0 con 0x7f710c0b44d0 2024-04-24T23:11:26.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.546+0000 7f7111596640 1 -- 172.21.15.32:0/3979839134 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f710c06e620 con 0x7f710c0b44d0 2024-04-24T23:11:26.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.547+0000 7f71037fe640 1 -- 172.21.15.32:0/3979839134 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f710400ac30 con 0x7f710c0b44d0 2024-04-24T23:11:26.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.547+0000 7f71037fe640 1 --2- 172.21.15.32:0/3979839134 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f70ec06be30 0x7f70ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:26.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.547+0000 7f71037fe640 1 -- 172.21.15.32:0/3979839134 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 4943+0+0 (secure 0 0 0) 0x7f710408af50 con 0x7f710c0b44d0 2024-04-24T23:11:26.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.548+0000 7f710affd640 1 --2- 172.21.15.32:0/3979839134 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f70ec06be30 0x7f70ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:26.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.548+0000 7f710affd640 1 --2- 172.21.15.32:0/3979839134 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f70ec06be30 0x7f70ec06e2f0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f70fc034000 tx=0x7f70fc02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:26.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.552+0000 7f71037fe640 1 -- 172.21.15.32:0/3979839134 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f710c06e620 con 0x7f710c0b44d0 2024-04-24T23:11:26.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.736+0000 7f7111596640 1 -- 172.21.15.32:0/3979839134 --> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f710c068430 con 0x7f70ec06be30 2024-04-24T23:11:26.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.737+0000 7f71037fe640 1 -- 172.21.15.32:0/3979839134 <== mgr.4108 v2:172.21.15.121:6832/1848157943 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f710c068430 con 0x7f70ec06be30 2024-04-24T23:11:26.739 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 4 to deep-scrub 2024-04-24T23:11:26.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.739+0000 7f7111596640 1 -- 172.21.15.32:0/3979839134 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f70ec06be30 msgr2=0x7f70ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:26.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.739+0000 7f7111596640 1 --2- 172.21.15.32:0/3979839134 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f70ec06be30 0x7f70ec06e2f0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f70fc034000 tx=0x7f70fc02e000 comp rx=0 tx=0).stop 2024-04-24T23:11:26.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.739+0000 7f7111596640 1 -- 172.21.15.32:0/3979839134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f710c0b44d0 msgr2=0x7f710c15aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:26.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.739+0000 7f7111596640 1 --2- 172.21.15.32:0/3979839134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f710c0b44d0 0x7f710c15aec0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f710400c780 tx=0x7f710400cc40 comp rx=0 tx=0).stop 2024-04-24T23:11:26.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.739+0000 7f7111596640 1 -- 172.21.15.32:0/3979839134 shutdown_connections 2024-04-24T23:11:26.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.739+0000 7f7111596640 1 --2- 172.21.15.32:0/3979839134 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f70ec06be30 0x7f70ec06e2f0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:26.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.739+0000 7f7111596640 1 --2- 172.21.15.32:0/3979839134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f710c0b44d0 0x7f710c15aec0 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:26.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.739+0000 7f7111596640 1 --2- 172.21.15.32:0/3979839134 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f710c0af890 0x7f710c1f4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:26.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.739+0000 7f7111596640 1 --2- 172.21.15.32:0/3979839134 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f710c06c5f0 0x7f710c154c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:26.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.739+0000 7f7111596640 1 -- 172.21.15.32:0/3979839134 >> 172.21.15.32:0/3979839134 conn(0x7f710c06b110 msgr2=0x7f710c0b4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:26.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.739+0000 7f7111596640 1 -- 172.21.15.32:0/3979839134 shutdown_connections 2024-04-24T23:11:26.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.739+0000 7f7111596640 1 -- 172.21.15.32:0/3979839134 wait complete. 2024-04-24T23:11:26.752 INFO:tasks.ceph:Scrubbing osd.5 2024-04-24T23:11:26.752 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-24T23:11:26.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.855+0000 7fd7c1293640 1 Processor -- start 2024-04-24T23:11:26.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.855+0000 7fd7c1293640 1 -- start start 2024-04-24T23:11:26.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.856+0000 7fd7c1293640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd7bc15c420 0x7fd7bc0af700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:26.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.856+0000 7fd7c1293640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7bc0afc40 con 0x7fd7bc0b5390 2024-04-24T23:11:26.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.856+0000 7fd7c1293640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7bc160390 con 0x7fd7bc0b5f30 2024-04-24T23:11:26.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.856+0000 7fd7c1293640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fd7bc160140 con 0x7fd7bc15c420 2024-04-24T23:11:26.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.856+0000 7fd7ba7fc640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd7bc0b5390 0x7fd7bc0b5820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56400/0 (socket says 172.21.15.32:56400) 2024-04-24T23:11:26.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.856+0000 7fd7ba7fc640 1 -- 172.21.15.32:0/2347004372 learned_addr learned my addr 172.21.15.32:0/2347004372 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:11:26.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.856+0000 7fd7baffd640 1 --2- 172.21.15.32:0/2347004372 >> v2:172.21.15.121:3300/0 conn(0x7fd7bc15c420 0x7fd7bc0af700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:26.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.856+0000 7fd7b9ffb640 1 -- 172.21.15.32:0/2347004372 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4219346979 0 0) 0x7fd7a4001110 con 0x7fd7bc0b5390 2024-04-24T23:11:26.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.856+0000 7fd7b9ffb640 1 -- 172.21.15.32:0/2347004372 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd7a8003520 con 0x7fd7bc0b5390 2024-04-24T23:11:26.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.857+0000 7fd7b9ffb640 1 -- 172.21.15.32:0/2347004372 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3883491356 0 0) 0x7fd7a8003520 con 0x7fd7bc0b5390 2024-04-24T23:11:26.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.857+0000 7fd7b9ffb640 1 -- 172.21.15.32:0/2347004372 >> v2:172.21.15.121:3300/0 conn(0x7fd7bc15c420 msgr2=0x7fd7bc0af700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:26.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.857+0000 7fd7b9ffb640 1 --2- 172.21.15.32:0/2347004372 >> v2:172.21.15.121:3300/0 conn(0x7fd7bc15c420 0x7fd7bc0af700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:26.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.857+0000 7fd7b9ffb640 1 -- 172.21.15.32:0/2347004372 >> v1:172.21.15.121:6789/0 conn(0x7fd7bc0b5f30 legacy=0x7fd7bc0b36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:26.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.857+0000 7fd7b9ffb640 1 -- 172.21.15.32:0/2347004372 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7bc160670 con 0x7fd7bc0b5390 2024-04-24T23:11:26.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.857+0000 7fd7b9ffb640 1 -- 172.21.15.32:0/2347004372 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7fd7a4001840 con 0x7fd7bc0b5390 2024-04-24T23:11:26.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.857+0000 7fd7b9ffb640 1 -- 172.21.15.32:0/2347004372 >> v1:172.21.15.32:6789/0 conn(0x7fd7bc0b5390 legacy=0x7fd7bc0b5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:26.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.857+0000 7fd7b9ffb640 1 --2- 172.21.15.32:0/2347004372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7a80039b0 0x7fd7a8003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:26.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.857+0000 7fd7b9ffb640 1 --2- 172.21.15.32:0/2347004372 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7a8004800 0x7fd7a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:26.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.857+0000 7fd7b9ffb640 1 --2- 172.21.15.32:0/2347004372 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7a8005210 0x7fd7a800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:26.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.857+0000 7fd7b9ffb640 1 -- 172.21.15.32:0/2347004372 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd7bc160140 con 0x7fd7a80039b0 2024-04-24T23:11:26.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.857+0000 7fd7b9ffb640 1 -- 172.21.15.32:0/2347004372 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a800c140 con 0x7fd7a8005210 2024-04-24T23:11:26.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.857+0000 7fd7b9ffb640 1 -- 172.21.15.32:0/2347004372 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd7a800c340 con 0x7fd7a8004800 2024-04-24T23:11:26.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.857+0000 7fd7baffd640 1 --2- 172.21.15.32:0/2347004372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7a80039b0 0x7fd7a8003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:26.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.857+0000 7fd7ba7fc640 1 --2- 172.21.15.32:0/2347004372 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7a8004800 0x7fd7a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:26.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.857+0000 7fd7bb7fe640 1 --2- 172.21.15.32:0/2347004372 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7a8005210 0x7fd7a800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:26.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.857+0000 7fd7baffd640 1 -- 172.21.15.32:0/2347004372 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7a8004800 msgr2=0x7fd7a8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:26.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.857+0000 7fd7baffd640 1 --2- 172.21.15.32:0/2347004372 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7a8004800 0x7fd7a8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:26.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.857+0000 7fd7baffd640 1 -- 172.21.15.32:0/2347004372 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7a8005210 msgr2=0x7fd7a800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:26.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.857+0000 7fd7baffd640 1 --2- 172.21.15.32:0/2347004372 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7a8005210 0x7fd7a800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:26.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.857+0000 7fd7baffd640 1 -- 172.21.15.32:0/2347004372 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7a8003520 con 0x7fd7a80039b0 2024-04-24T23:11:26.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.857+0000 7fd7bb7fe640 1 -- 172.21.15.32:0/2347004372 reap_dead start 2024-04-24T23:11:26.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.857+0000 7fd7ba7fc640 1 --2- 172.21.15.32:0/2347004372 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7a8004800 0x7fd7a8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:11:26.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.858+0000 7fd7baffd640 1 --2- 172.21.15.32:0/2347004372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7a80039b0 0x7fd7a8003df0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4009ac0 tx=0x7fd7a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edeb01b807d36d5e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:26.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.858+0000 7fd7b9ffb640 1 -- 172.21.15.32:0/2347004372 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b4036070 con 0x7fd7a80039b0 2024-04-24T23:11:26.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.858+0000 7fd7b9ffb640 1 -- 172.21.15.32:0/2347004372 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7b4031400 con 0x7fd7a80039b0 2024-04-24T23:11:26.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.858+0000 7fd7b9ffb640 1 -- 172.21.15.32:0/2347004372 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b4009690 con 0x7fd7a80039b0 2024-04-24T23:11:26.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.858+0000 7fd7c1293640 1 -- 172.21.15.32:0/2347004372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7a80039b0 msgr2=0x7fd7a8003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:26.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.858+0000 7fd7c1293640 1 --2- 172.21.15.32:0/2347004372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7a80039b0 0x7fd7a8003df0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4009ac0 tx=0x7fd7a8005830 comp rx=0 tx=0).stop 2024-04-24T23:11:26.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.858+0000 7fd7c1293640 1 -- 172.21.15.32:0/2347004372 shutdown_connections 2024-04-24T23:11:26.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.858+0000 7fd7c1293640 1 --2- 172.21.15.32:0/2347004372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7a80039b0 0x7fd7a8003df0 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:26.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.858+0000 7fd7c1293640 1 -- 172.21.15.32:0/2347004372 >> 172.21.15.32:0/2347004372 conn(0x7fd7bc0b1f10 msgr2=0x7fd7bc0b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:26.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.859+0000 7fd7c1293640 1 -- 172.21.15.32:0/2347004372 shutdown_connections 2024-04-24T23:11:26.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.859+0000 7fd7c1293640 1 -- 172.21.15.32:0/2347004372 wait complete. 2024-04-24T23:11:26.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.860+0000 7fd7c1293640 1 Processor -- start 2024-04-24T23:11:26.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.860+0000 7fd7c1293640 1 -- start start 2024-04-24T23:11:26.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.861+0000 7fd7c1293640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7a80039b0 0x7fd7bc14cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:26.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.861+0000 7fd7c1293640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7bc154270 0x7fd7bc1546f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:26.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.861+0000 7fd7c1293640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7bc06c5a0 0x7fd7bc06ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:26.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.861+0000 7fd7c1293640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd7bc160c30 con 0x7fd7a80039b0 2024-04-24T23:11:26.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.861+0000 7fd7c1293640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd7bc160fd0 con 0x7fd7bc154270 2024-04-24T23:11:26.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.861+0000 7fd7c1293640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fd7bc0717b0 con 0x7fd7bc06c5a0 2024-04-24T23:11:26.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.861+0000 7fd7baffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7a80039b0 0x7fd7bc14cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:26.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.861+0000 7fd7baffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7a80039b0 0x7fd7bc14cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54632/0 (socket says 172.21.15.32:54632) 2024-04-24T23:11:26.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.861+0000 7fd7baffd640 1 -- 172.21.15.32:0/3445172616 learned_addr learned my addr 172.21.15.32:0/3445172616 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:26.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.861+0000 7fd7bb7fe640 1 --2- 172.21.15.32:0/3445172616 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7bc06c5a0 0x7fd7bc06ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:26.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.861+0000 7fd7ba7fc640 1 --2- 172.21.15.32:0/3445172616 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7bc154270 0x7fd7bc1546f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:26.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.861+0000 7fd7baffd640 1 -- 172.21.15.32:0/3445172616 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7bc06c5a0 msgr2=0x7fd7bc06ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:26.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.861+0000 7fd7baffd640 1 --2- 172.21.15.32:0/3445172616 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7bc06c5a0 0x7fd7bc06ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:26.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.861+0000 7fd7baffd640 1 -- 172.21.15.32:0/3445172616 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7bc154270 msgr2=0x7fd7bc1546f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:26.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.861+0000 7fd7baffd640 1 --2- 172.21.15.32:0/3445172616 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7bc154270 0x7fd7bc1546f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:26.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.861+0000 7fd7baffd640 1 -- 172.21.15.32:0/3445172616 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7bc1f3010 con 0x7fd7a80039b0 2024-04-24T23:11:26.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.861+0000 7fd7baffd640 1 --2- 172.21.15.32:0/3445172616 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7a80039b0 0x7fd7bc14cf50 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4034040 tx=0x7fd7b4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:26.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.861+0000 7fd7b37fe640 1 -- 172.21.15.32:0/3445172616 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b403f020 con 0x7fd7a80039b0 2024-04-24T23:11:26.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.861+0000 7fd7b37fe640 1 -- 172.21.15.32:0/3445172616 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7b4037040 con 0x7fd7a80039b0 2024-04-24T23:11:26.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.861+0000 7fd7c1293640 1 -- 172.21.15.32:0/3445172616 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7bc1f32f0 con 0x7fd7a80039b0 2024-04-24T23:11:26.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.861+0000 7fd7b37fe640 1 -- 172.21.15.32:0/3445172616 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b4036070 con 0x7fd7a80039b0 2024-04-24T23:11:26.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.861+0000 7fd7c1293640 1 -- 172.21.15.32:0/3445172616 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7bc1f38b0 con 0x7fd7a80039b0 2024-04-24T23:11:26.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.862+0000 7fd7c1293640 1 -- 172.21.15.32:0/3445172616 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd7bc068430 con 0x7fd7a80039b0 2024-04-24T23:11:26.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.863+0000 7fd7b37fe640 1 -- 172.21.15.32:0/3445172616 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd7b4042020 con 0x7fd7a80039b0 2024-04-24T23:11:26.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.863+0000 7fd7b37fe640 1 --2- 172.21.15.32:0/3445172616 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd79c06be30 0x7fd79c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:26.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.863+0000 7fd7b37fe640 1 -- 172.21.15.32:0/3445172616 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 4943+0+0 (secure 0 0 0) 0x7fd7b40aab70 con 0x7fd7a80039b0 2024-04-24T23:11:26.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.863+0000 7fd7b37fe640 1 --2- 172.21.15.32:0/3445172616 >> [v2:172.21.15.121:6808/3794084091,v1:172.21.15.121:6809/3794084091] conn(0x7fd79c071cf0 0x7fd79c074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:26.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.863+0000 7fd7b37fe640 1 -- 172.21.15.32:0/3445172616 --> [v2:172.21.15.121:6808/3794084091,v1:172.21.15.121:6809/3794084091] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd7b4001d30 con 0x7fd79c071cf0 2024-04-24T23:11:26.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.863+0000 7fd7ba7fc640 1 --2- 172.21.15.32:0/3445172616 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd79c06be30 0x7fd79c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:26.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.863+0000 7fd7b37fe640 1 -- 172.21.15.32:0/3445172616 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=194) ==== 24+0+0 (secure 0 0 0) 0x7fd7b4075490 con 0x7fd7a80039b0 2024-04-24T23:11:26.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.864+0000 7fd7bb7fe640 1 --2- 172.21.15.32:0/3445172616 >> [v2:172.21.15.121:6808/3794084091,v1:172.21.15.121:6809/3794084091] conn(0x7fd79c071cf0 0x7fd79c074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:26.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.864+0000 7fd7ba7fc640 1 --2- 172.21.15.32:0/3445172616 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd79c06be30 0x7fd79c06e2f0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc0b1320 tx=0x7fd7a4001110 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:26.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.864+0000 7fd7bb7fe640 1 --2- 172.21.15.32:0/3445172616 >> [v2:172.21.15.121:6808/3794084091,v1:172.21.15.121:6809/3794084091] conn(0x7fd79c071cf0 0x7fd79c074110 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:26.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.865+0000 7fd7b37fe640 1 -- 172.21.15.32:0/3445172616 <== osd.5 v2:172.21.15.121:6808/3794084091 1 ==== command_reply(tid 1: 0 ) ==== 8+0+25033 (crc 0 0 0) 0x7fd7b4001d30 con 0x7fd79c071cf0 2024-04-24T23:11:26.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.877+0000 7fd7c1293640 1 -- 172.21.15.32:0/3445172616 --> [v2:172.21.15.121:6808/3794084091,v1:172.21.15.121:6809/3794084091] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd7bc0730d0 con 0x7fd79c071cf0 2024-04-24T23:11:26.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.877+0000 7fd7b37fe640 1 -- 172.21.15.32:0/3445172616 <== osd.5 v2:172.21.15.121:6808/3794084091 2 ==== command_reply(tid 2: 0 ) ==== 8+0+565 (crc 0 0 0) 0x7fd7bc0730d0 con 0x7fd79c071cf0 2024-04-24T23:11:26.879 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:11:26.879 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-04-24T23:11:26.879 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:11:26.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.878+0000 7fd7c1293640 1 -- 172.21.15.32:0/3445172616 >> [v2:172.21.15.121:6808/3794084091,v1:172.21.15.121:6809/3794084091] conn(0x7fd79c071cf0 msgr2=0x7fd79c074110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:26.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.878+0000 7fd7c1293640 1 --2- 172.21.15.32:0/3445172616 >> [v2:172.21.15.121:6808/3794084091,v1:172.21.15.121:6809/3794084091] conn(0x7fd79c071cf0 0x7fd79c074110 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:26.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.878+0000 7fd7c1293640 1 -- 172.21.15.32:0/3445172616 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd79c06be30 msgr2=0x7fd79c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:26.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.878+0000 7fd7c1293640 1 --2- 172.21.15.32:0/3445172616 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fd79c06be30 0x7fd79c06e2f0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc0b1320 tx=0x7fd7a4001110 comp rx=0 tx=0).stop 2024-04-24T23:11:26.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.878+0000 7fd7c1293640 1 -- 172.21.15.32:0/3445172616 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7a80039b0 msgr2=0x7fd7bc14cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:26.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.878+0000 7fd7c1293640 1 --2- 172.21.15.32:0/3445172616 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7a80039b0 0x7fd7bc14cf50 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4034040 tx=0x7fd7b4004730 comp rx=0 tx=0).stop 2024-04-24T23:11:26.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.878+0000 7fd7bb7fe640 1 -- 172.21.15.32:0/3445172616 reap_dead start 2024-04-24T23:11:26.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.878+0000 7fd7c1293640 1 -- 172.21.15.32:0/3445172616 shutdown_connections 2024-04-24T23:11:26.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.878+0000 7fd7c1293640 1 -- 172.21.15.32:0/3445172616 >> 172.21.15.32:0/3445172616 conn(0x7fd7bc0b1f10 msgr2=0x7fd7bc0afe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:26.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.878+0000 7fd7c1293640 1 -- 172.21.15.32:0/3445172616 shutdown_connections 2024-04-24T23:11:26.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.878+0000 7fd7c1293640 1 -- 172.21.15.32:0/3445172616 wait complete. 2024-04-24T23:11:26.890 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-24T23:11:26.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.989+0000 7f17605ee640 1 Processor -- start 2024-04-24T23:11:26.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.989+0000 7f17605ee640 1 -- start start 2024-04-24T23:11:26.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.990+0000 7f17605ee640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f175815b480 0x7f17580b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:26.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.990+0000 7f17605ee640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f175815fd60 0x7f175806c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:26.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.990+0000 7f17605ee640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17580cef90 con 0x7f175806cb30 2024-04-24T23:11:26.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.990+0000 7f17605ee640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f17580b4ff0 con 0x7f175815b480 2024-04-24T23:11:26.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.990+0000 7f17605ee640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f17580b46c0 con 0x7f175815fd60 2024-04-24T23:11:26.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.991+0000 7f175eb8b640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f175806cb30 0x7f17580ccc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56412/0 (socket says 172.21.15.32:56412) 2024-04-24T23:11:26.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.991+0000 7f175eb8b640 1 -- 172.21.15.32:0/4209744593 learned_addr learned my addr 172.21.15.32:0/4209744593 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:11:26.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.991+0000 7f175db89640 1 --2- 172.21.15.32:0/4209744593 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f175815fd60 0x7f175806c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:26.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.991+0000 7f175e38a640 1 --2- 172.21.15.32:0/4209744593 >> v2:172.21.15.121:3300/0 conn(0x7f175815b480 0x7f17580b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:26.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.991+0000 7f175d388640 1 -- 172.21.15.32:0/4209744593 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4097462365 0 0) 0x7f17580cef90 con 0x7f175806cb30 2024-04-24T23:11:26.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.991+0000 7f175d388640 1 -- 172.21.15.32:0/4209744593 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1744003520 con 0x7f175806cb30 2024-04-24T23:11:26.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.991+0000 7f175d388640 1 -- 172.21.15.32:0/4209744593 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 613040964 0 0) 0x7f1744003520 con 0x7f175806cb30 2024-04-24T23:11:26.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.991+0000 7f175db89640 1 -- 172.21.15.32:0/4209744593 >> v2:172.21.15.121:3300/0 conn(0x7f175815b480 msgr2=0x7f17580b5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:26.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.991+0000 7f175db89640 1 --2- 172.21.15.32:0/4209744593 >> v2:172.21.15.121:3300/0 conn(0x7f175815b480 0x7f17580b5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:26.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.991+0000 7f175db89640 1 -- 172.21.15.32:0/4209744593 >> v1:172.21.15.32:6789/0 conn(0x7f175806cb30 legacy=0x7f17580ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:26.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.991+0000 7f175db89640 1 -- 172.21.15.32:0/4209744593 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17580b49a0 con 0x7f175815fd60 2024-04-24T23:11:26.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.992+0000 7f175db89640 1 --2- 172.21.15.32:0/4209744593 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f175815fd60 0x7f175806c5f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f17480097a0 tx=0x7f174802a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd26813e61482d5f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:26.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.992+0000 7f175d388640 1 -- 172.21.15.32:0/4209744593 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1748037070 con 0x7f175815fd60 2024-04-24T23:11:26.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.992+0000 7f175d388640 1 -- 172.21.15.32:0/4209744593 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f174802aba0 con 0x7f175815fd60 2024-04-24T23:11:26.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.992+0000 7f175d388640 1 -- 172.21.15.32:0/4209744593 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17500019f0 con 0x7f175815fd60 2024-04-24T23:11:26.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.992+0000 7f17605ee640 1 -- 172.21.15.32:0/4209744593 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f175815fd60 msgr2=0x7f175806c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:26.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.993+0000 7f17605ee640 1 --2- 172.21.15.32:0/4209744593 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f175815fd60 0x7f175806c5f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f17480097a0 tx=0x7f174802a110 comp rx=0 tx=0).stop 2024-04-24T23:11:26.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.993+0000 7f17605ee640 1 -- 172.21.15.32:0/4209744593 shutdown_connections 2024-04-24T23:11:26.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.993+0000 7f17605ee640 1 --2- 172.21.15.32:0/4209744593 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f175815fd60 0x7f175806c5f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:26.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.993+0000 7f17605ee640 1 --2- 172.21.15.32:0/4209744593 >> v2:172.21.15.121:3300/0 conn(0x7f175815b480 0x7f17580b5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:26.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.993+0000 7f17605ee640 1 -- 172.21.15.32:0/4209744593 >> 172.21.15.32:0/4209744593 conn(0x7f175806b110 msgr2=0x7f175806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:26.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.993+0000 7f17605ee640 1 -- 172.21.15.32:0/4209744593 shutdown_connections 2024-04-24T23:11:26.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.993+0000 7f17605ee640 1 -- 172.21.15.32:0/4209744593 wait complete. 2024-04-24T23:11:26.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.994+0000 7f17605ee640 1 Processor -- start 2024-04-24T23:11:26.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.994+0000 7f17605ee640 1 -- start start 2024-04-24T23:11:26.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.994+0000 7f17605ee640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f175806cb30 0x7f1758149ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:26.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.994+0000 7f17605ee640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f175815b480 0x7f175815df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:26.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.995+0000 7f17605ee640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f175815fd60 0x7f175814cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:26.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.995+0000 7f17605ee640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f17580b5480 con 0x7f175815fd60 2024-04-24T23:11:26.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.995+0000 7f17605ee640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f17580b61a0 con 0x7f175806cb30 2024-04-24T23:11:26.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.995+0000 7f17605ee640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f175815b860 con 0x7f175815b480 2024-04-24T23:11:26.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.995+0000 7f175eb8b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f175815fd60 0x7f175814cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:26.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.995+0000 7f175eb8b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f175815fd60 0x7f175814cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54648/0 (socket says 172.21.15.32:54648) 2024-04-24T23:11:26.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.995+0000 7f175eb8b640 1 -- 172.21.15.32:0/3285956499 learned_addr learned my addr 172.21.15.32:0/3285956499 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:26.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.995+0000 7f175db89640 1 --2- 172.21.15.32:0/3285956499 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f175815b480 0x7f175815df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:26.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.995+0000 7f175e38a640 1 --2- 172.21.15.32:0/3285956499 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f175806cb30 0x7f1758149ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:26.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.995+0000 7f175eb8b640 1 -- 172.21.15.32:0/3285956499 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f175815b480 msgr2=0x7f175815df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:26.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.995+0000 7f175eb8b640 1 --2- 172.21.15.32:0/3285956499 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f175815b480 0x7f175815df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:26.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.995+0000 7f175eb8b640 1 -- 172.21.15.32:0/3285956499 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f175806cb30 msgr2=0x7f1758149ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:26.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.995+0000 7f175eb8b640 1 --2- 172.21.15.32:0/3285956499 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f175806cb30 0x7f1758149ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:26.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.995+0000 7f175eb8b640 1 -- 172.21.15.32:0/3285956499 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f175814a410 con 0x7f175815fd60 2024-04-24T23:11:26.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.995+0000 7f175eb8b640 1 --2- 172.21.15.32:0/3285956499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f175815fd60 0x7f175814cd50 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f1750001b60 tx=0x7f175000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:26.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.995+0000 7f174effd640 1 -- 172.21.15.32:0/3285956499 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1750017070 con 0x7f175815fd60 2024-04-24T23:11:26.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.995+0000 7f174effd640 1 -- 172.21.15.32:0/3285956499 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1750012400 con 0x7f175815fd60 2024-04-24T23:11:27.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.995+0000 7f17605ee640 1 -- 172.21.15.32:0/3285956499 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17581481f0 con 0x7f175815fd60 2024-04-24T23:11:27.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.995+0000 7f174effd640 1 -- 172.21.15.32:0/3285956499 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17500127b0 con 0x7f175815fd60 2024-04-24T23:11:27.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.995+0000 7f17605ee640 1 -- 172.21.15.32:0/3285956499 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17581487b0 con 0x7f175815fd60 2024-04-24T23:11:27.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.996+0000 7f17605ee640 1 -- 172.21.15.32:0/3285956499 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1758068430 con 0x7f175815fd60 2024-04-24T23:11:27.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.997+0000 7f174effd640 1 -- 172.21.15.32:0/3285956499 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f175001a070 con 0x7f175815fd60 2024-04-24T23:11:27.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.997+0000 7f174effd640 1 --2- 172.21.15.32:0/3285956499 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f173c06be30 0x7f173c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:27.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.998+0000 7f174effd640 1 -- 172.21.15.32:0/3285956499 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 4943+0+0 (secure 0 0 0) 0x7f175008a910 con 0x7f175815fd60 2024-04-24T23:11:27.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.998+0000 7f175e38a640 1 --2- 172.21.15.32:0/3285956499 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f173c06be30 0x7f173c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:27.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:26.998+0000 7f175e38a640 1 --2- 172.21.15.32:0/3285956499 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f173c06be30 0x7f173c06e2f0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f175815add0 tx=0x7f1754008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:27.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.002+0000 7f174effd640 1 -- 172.21.15.32:0/3285956499 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1750059000 con 0x7f175815fd60 2024-04-24T23:11:27.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.188+0000 7f17605ee640 1 -- 172.21.15.32:0/3285956499 --> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f175806c000 con 0x7f173c06be30 2024-04-24T23:11:27.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.189+0000 7f174effd640 1 -- 172.21.15.32:0/3285956499 <== mgr.4108 v2:172.21.15.121:6832/1848157943 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f175806c000 con 0x7f173c06be30 2024-04-24T23:11:27.191 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 5 to deep-scrub 2024-04-24T23:11:27.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.191+0000 7f17605ee640 1 -- 172.21.15.32:0/3285956499 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f173c06be30 msgr2=0x7f173c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:27.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.191+0000 7f17605ee640 1 --2- 172.21.15.32:0/3285956499 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f173c06be30 0x7f173c06e2f0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f175815add0 tx=0x7f1754008040 comp rx=0 tx=0).stop 2024-04-24T23:11:27.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.191+0000 7f17605ee640 1 -- 172.21.15.32:0/3285956499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f175815fd60 msgr2=0x7f175814cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:27.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.191+0000 7f17605ee640 1 --2- 172.21.15.32:0/3285956499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f175815fd60 0x7f175814cd50 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f1750001b60 tx=0x7f175000a430 comp rx=0 tx=0).stop 2024-04-24T23:11:27.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.192+0000 7f17605ee640 1 -- 172.21.15.32:0/3285956499 shutdown_connections 2024-04-24T23:11:27.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.192+0000 7f17605ee640 1 --2- 172.21.15.32:0/3285956499 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f173c06be30 0x7f173c06e2f0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:27.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.192+0000 7f17605ee640 1 --2- 172.21.15.32:0/3285956499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f175815fd60 0x7f175814cd50 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:27.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.192+0000 7f17605ee640 1 --2- 172.21.15.32:0/3285956499 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f175815b480 0x7f175815df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:27.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.192+0000 7f17605ee640 1 --2- 172.21.15.32:0/3285956499 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f175806cb30 0x7f1758149ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:27.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.192+0000 7f17605ee640 1 -- 172.21.15.32:0/3285956499 >> 172.21.15.32:0/3285956499 conn(0x7f175806b110 msgr2=0x7f17580aff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:27.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.192+0000 7f17605ee640 1 -- 172.21.15.32:0/3285956499 shutdown_connections 2024-04-24T23:11:27.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.192+0000 7f17605ee640 1 -- 172.21.15.32:0/3285956499 wait complete. 2024-04-24T23:11:27.204 INFO:tasks.ceph:Scrubbing osd.6 2024-04-24T23:11:27.204 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-24T23:11:27.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.307+0000 7f4148fa9640 1 Processor -- start 2024-04-24T23:11:27.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.308+0000 7f4148fa9640 1 -- start start 2024-04-24T23:11:27.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.308+0000 7f4148fa9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f414415b960 0x7f41440cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:27.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.308+0000 7f4148fa9640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f41440b58e0 0x7f41440b5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:27.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.308+0000 7f4148fa9640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f414415bde0 con 0x7f414415b580 2024-04-24T23:11:27.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.308+0000 7f4148fa9640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f414415c9f0 con 0x7f414415b960 2024-04-24T23:11:27.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.308+0000 7f4148fa9640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f41440ce3c0 con 0x7f41440b58e0 2024-04-24T23:11:27.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.309+0000 7f4141d9b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f414415b960 0x7f41440cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:27.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.309+0000 7f4141d9b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f414415b960 0x7f41440cde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54662/0 (socket says 172.21.15.32:54662) 2024-04-24T23:11:27.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.309+0000 7f4141d9b640 1 -- 172.21.15.32:0/534153972 learned_addr learned my addr 172.21.15.32:0/534153972 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:27.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.309+0000 7f4142d9d640 1 --2- 172.21.15.32:0/534153972 >> v2:172.21.15.121:3300/0 conn(0x7f41440b58e0 0x7f41440b5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:27.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.309+0000 7f4141d9b640 1 -- 172.21.15.32:0/534153972 >> v2:172.21.15.121:3300/0 conn(0x7f41440b58e0 msgr2=0x7f41440b5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:27.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.309+0000 7f4141d9b640 1 --2- 172.21.15.32:0/534153972 >> v2:172.21.15.121:3300/0 conn(0x7f41440b58e0 0x7f41440b5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:27.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.309+0000 7f4141d9b640 1 -- 172.21.15.32:0/534153972 >> v1:172.21.15.121:6789/0 conn(0x7f414415b580 legacy=0x7f414406cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:27.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.309+0000 7f4141d9b640 1 -- 172.21.15.32:0/534153972 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f414415c0c0 con 0x7f414415b960 2024-04-24T23:11:27.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.310+0000 7f4141d9b640 1 --2- 172.21.15.32:0/534153972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f414415b960 0x7f41440cde80 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f41440b2d30 tx=0x7f413802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af1503a65cf40269 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:27.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.310+0000 7f414159a640 1 -- 172.21.15.32:0/534153972 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4138037070 con 0x7f414415b960 2024-04-24T23:11:27.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.310+0000 7f414159a640 1 -- 172.21.15.32:0/534153972 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4138032440 con 0x7f414415b960 2024-04-24T23:11:27.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.310+0000 7f414159a640 1 -- 172.21.15.32:0/534153972 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4138032810 con 0x7f414415b960 2024-04-24T23:11:27.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.310+0000 7f4148fa9640 1 -- 172.21.15.32:0/534153972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f414415b960 msgr2=0x7f41440cde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:27.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.310+0000 7f4148fa9640 1 --2- 172.21.15.32:0/534153972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f414415b960 0x7f41440cde80 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f41440b2d30 tx=0x7f413802a230 comp rx=0 tx=0).stop 2024-04-24T23:11:27.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.310+0000 7f4148fa9640 1 -- 172.21.15.32:0/534153972 shutdown_connections 2024-04-24T23:11:27.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.310+0000 7f4148fa9640 1 --2- 172.21.15.32:0/534153972 >> v2:172.21.15.121:3300/0 conn(0x7f41440b58e0 0x7f41440b5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:27.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.310+0000 7f4148fa9640 1 --2- 172.21.15.32:0/534153972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f414415b960 0x7f41440cde80 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:27.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.310+0000 7f4148fa9640 1 -- 172.21.15.32:0/534153972 >> 172.21.15.32:0/534153972 conn(0x7f41440b1f10 msgr2=0x7f41440b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:27.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.311+0000 7f4148fa9640 1 -- 172.21.15.32:0/534153972 shutdown_connections 2024-04-24T23:11:27.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.311+0000 7f4148fa9640 1 -- 172.21.15.32:0/534153972 wait complete. 2024-04-24T23:11:27.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.312+0000 7f4148fa9640 1 Processor -- start 2024-04-24T23:11:27.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.312+0000 7f4148fa9640 1 -- start start 2024-04-24T23:11:27.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.313+0000 7f4148fa9640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f41440b58e0 0x7f4144143770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:27.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.313+0000 7f4148fa9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f414415b580 0x7f4144071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:27.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.313+0000 7f4148fa9640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f414415b960 0x7f41440ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:27.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.313+0000 7f4148fa9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f414415c680 con 0x7f414415b580 2024-04-24T23:11:27.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.313+0000 7f4148fa9640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f41440b6260 con 0x7f414415b960 2024-04-24T23:11:27.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.313+0000 7f4148fa9640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f414406c7a0 con 0x7f41440b58e0 2024-04-24T23:11:27.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.313+0000 7f4141d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f414415b580 0x7f4144071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:27.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.313+0000 7f4141d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f414415b580 0x7f4144071bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54676/0 (socket says 172.21.15.32:54676) 2024-04-24T23:11:27.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.313+0000 7f4141d9b640 1 -- 172.21.15.32:0/1724769141 learned_addr learned my addr 172.21.15.32:0/1724769141 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:27.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.313+0000 7f414259c640 1 --2- 172.21.15.32:0/1724769141 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f41440b58e0 0x7f4144143770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:27.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.313+0000 7f4142d9d640 1 --2- 172.21.15.32:0/1724769141 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f414415b960 0x7f41440ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:27.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.313+0000 7f4141d9b640 1 -- 172.21.15.32:0/1724769141 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f41440b58e0 msgr2=0x7f4144143770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:27.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.313+0000 7f4141d9b640 1 --2- 172.21.15.32:0/1724769141 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f41440b58e0 0x7f4144143770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:27.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.313+0000 7f4141d9b640 1 -- 172.21.15.32:0/1724769141 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f414415b960 msgr2=0x7f41440ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:27.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.313+0000 7f4141d9b640 1 --2- 172.21.15.32:0/1724769141 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f414415b960 0x7f41440ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:27.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.313+0000 7f4141d9b640 1 -- 172.21.15.32:0/1724769141 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41440726c0 con 0x7f414415b580 2024-04-24T23:11:27.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.313+0000 7f414259c640 1 --2- 172.21.15.32:0/1724769141 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f41440b58e0 0x7f4144143770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:11:27.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.313+0000 7f4141d9b640 1 --2- 172.21.15.32:0/1724769141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f414415b580 0x7f4144071bb0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f41380097c0 tx=0x7f41380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:27.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.313+0000 7f4132ffd640 1 -- 172.21.15.32:0/1724769141 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4138037040 con 0x7f414415b580 2024-04-24T23:11:27.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.313+0000 7f4132ffd640 1 -- 172.21.15.32:0/1724769141 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4138042070 con 0x7f414415b580 2024-04-24T23:11:27.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.313+0000 7f4148fa9640 1 -- 172.21.15.32:0/1724769141 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4144143cb0 con 0x7f414415b580 2024-04-24T23:11:27.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.313+0000 7f4132ffd640 1 -- 172.21.15.32:0/1724769141 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4138030b90 con 0x7f414415b580 2024-04-24T23:11:27.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.313+0000 7f4148fa9640 1 -- 172.21.15.32:0/1724769141 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4144151770 con 0x7f414415b580 2024-04-24T23:11:27.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.314+0000 7f4148fa9640 1 -- 172.21.15.32:0/1724769141 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4144068430 con 0x7f414415b580 2024-04-24T23:11:27.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.315+0000 7f4132ffd640 1 -- 172.21.15.32:0/1724769141 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4138047460 con 0x7f414415b580 2024-04-24T23:11:27.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.315+0000 7f4132ffd640 1 --2- 172.21.15.32:0/1724769141 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f412406be30 0x7f412406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:27.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.315+0000 7f4132ffd640 1 -- 172.21.15.32:0/1724769141 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 4943+0+0 (secure 0 0 0) 0x7f41380ab7d0 con 0x7f414415b580 2024-04-24T23:11:27.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.315+0000 7f4132ffd640 1 --2- 172.21.15.32:0/1724769141 >> [v2:172.21.15.121:6823/1348640258,v1:172.21.15.121:6825/1348640258] conn(0x7f4124071cf0 0x7f4124074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:27.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.315+0000 7f4132ffd640 1 -- 172.21.15.32:0/1724769141 --> [v2:172.21.15.121:6823/1348640258,v1:172.21.15.121:6825/1348640258] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f413802be50 con 0x7f4124071cf0 2024-04-24T23:11:27.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.315+0000 7f4132ffd640 1 -- 172.21.15.32:0/1724769141 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=194) ==== 24+0+0 (secure 0 0 0) 0x7f41380b02c0 con 0x7f414415b580 2024-04-24T23:11:27.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.316+0000 7f414259c640 1 --2- 172.21.15.32:0/1724769141 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f412406be30 0x7f412406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:27.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.316+0000 7f4142d9d640 1 --2- 172.21.15.32:0/1724769141 >> [v2:172.21.15.121:6823/1348640258,v1:172.21.15.121:6825/1348640258] conn(0x7f4124071cf0 0x7f4124074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:27.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.316+0000 7f4142d9d640 1 --2- 172.21.15.32:0/1724769141 >> [v2:172.21.15.121:6823/1348640258,v1:172.21.15.121:6825/1348640258] conn(0x7f4124071cf0 0x7f4124074110 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:27.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.316+0000 7f414259c640 1 --2- 172.21.15.32:0/1724769141 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f412406be30 0x7f412406e2f0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f4134001750 tx=0x7f4134009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:27.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.317+0000 7f4132ffd640 1 -- 172.21.15.32:0/1724769141 <== osd.6 v2:172.21.15.121:6823/1348640258 1 ==== command_reply(tid 1: 0 ) ==== 8+0+25033 (crc 0 0 0) 0x7f413802be50 con 0x7f4124071cf0 2024-04-24T23:11:27.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.328+0000 7f4148fa9640 1 -- 172.21.15.32:0/1724769141 --> [v2:172.21.15.121:6823/1348640258,v1:172.21.15.121:6825/1348640258] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f41440cc1f0 con 0x7f4124071cf0 2024-04-24T23:11:27.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.328+0000 7f4132ffd640 1 -- 172.21.15.32:0/1724769141 <== osd.6 v2:172.21.15.121:6823/1348640258 2 ==== command_reply(tid 2: 0 ) ==== 8+0+565 (crc 0 0 0) 0x7f41440cc1f0 con 0x7f4124071cf0 2024-04-24T23:11:27.331 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:11:27.331 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-04-24T23:11:27.331 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:11:27.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.329+0000 7f4148fa9640 1 -- 172.21.15.32:0/1724769141 >> [v2:172.21.15.121:6823/1348640258,v1:172.21.15.121:6825/1348640258] conn(0x7f4124071cf0 msgr2=0x7f4124074110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:27.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.329+0000 7f4148fa9640 1 --2- 172.21.15.32:0/1724769141 >> [v2:172.21.15.121:6823/1348640258,v1:172.21.15.121:6825/1348640258] conn(0x7f4124071cf0 0x7f4124074110 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:27.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.329+0000 7f4148fa9640 1 -- 172.21.15.32:0/1724769141 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f412406be30 msgr2=0x7f412406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:27.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.329+0000 7f4148fa9640 1 --2- 172.21.15.32:0/1724769141 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f412406be30 0x7f412406e2f0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f4134001750 tx=0x7f4134009040 comp rx=0 tx=0).stop 2024-04-24T23:11:27.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.329+0000 7f4148fa9640 1 -- 172.21.15.32:0/1724769141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f414415b580 msgr2=0x7f4144071bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:27.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.329+0000 7f4148fa9640 1 --2- 172.21.15.32:0/1724769141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f414415b580 0x7f4144071bb0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f41380097c0 tx=0x7f41380046c0 comp rx=0 tx=0).stop 2024-04-24T23:11:27.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.329+0000 7f4142d9d640 1 -- 172.21.15.32:0/1724769141 reap_dead start 2024-04-24T23:11:27.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.329+0000 7f4148fa9640 1 -- 172.21.15.32:0/1724769141 shutdown_connections 2024-04-24T23:11:27.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.329+0000 7f4148fa9640 1 -- 172.21.15.32:0/1724769141 >> 172.21.15.32:0/1724769141 conn(0x7f41440b1f10 msgr2=0x7f41440ce570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:27.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.329+0000 7f4148fa9640 1 -- 172.21.15.32:0/1724769141 shutdown_connections 2024-04-24T23:11:27.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.329+0000 7f4148fa9640 1 -- 172.21.15.32:0/1724769141 wait complete. 2024-04-24T23:11:27.342 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-24T23:11:27.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.441+0000 7fe2784bd640 1 Processor -- start 2024-04-24T23:11:27.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.441+0000 7fe2784bd640 1 -- start start 2024-04-24T23:11:27.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.442+0000 7fe2784bd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe2700b5af0 0x7fe2700ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:27.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.442+0000 7fe2784bd640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe270162e10 0x7fe27015ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:27.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.442+0000 7fe2784bd640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2700cd270 con 0x7fe270162320 2024-04-24T23:11:27.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.442+0000 7fe2784bd640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fe2700b5660 con 0x7fe2700b5af0 2024-04-24T23:11:27.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.442+0000 7fe2784bd640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe270071610 con 0x7fe270162e10 2024-04-24T23:11:27.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.442+0000 7fe275a58640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe2700b5af0 0x7fe2700ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:27.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.442+0000 7fe275a58640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe2700b5af0 0x7fe2700ccc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54692/0 (socket says 172.21.15.32:54692) 2024-04-24T23:11:27.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.442+0000 7fe275a58640 1 -- 172.21.15.32:0/1780344460 learned_addr learned my addr 172.21.15.32:0/1780344460 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:27.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.442+0000 7fe276a5a640 1 --2- 172.21.15.32:0/1780344460 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe270162e10 0x7fe27015ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:27.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.442+0000 7fe275a58640 1 -- 172.21.15.32:0/1780344460 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe270162e10 msgr2=0x7fe27015ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:27.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.442+0000 7fe275a58640 1 --2- 172.21.15.32:0/1780344460 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe270162e10 0x7fe27015ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:27.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.442+0000 7fe275a58640 1 -- 172.21.15.32:0/1780344460 >> v1:172.21.15.121:6789/0 conn(0x7fe270162320 legacy=0x7fe270162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:27.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.442+0000 7fe275a58640 1 -- 172.21.15.32:0/1780344460 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2700718f0 con 0x7fe2700b5af0 2024-04-24T23:11:27.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.443+0000 7fe275a58640 1 --2- 172.21.15.32:0/1780344460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2700b5af0 0x7fe2700ccc90 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7fe264009870 tx=0x7fe26402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ab5aac0aee68907 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:27.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.443+0000 7fe275257640 1 -- 172.21.15.32:0/1780344460 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe264037070 con 0x7fe2700b5af0 2024-04-24T23:11:27.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.443+0000 7fe275257640 1 -- 172.21.15.32:0/1780344460 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe264032440 con 0x7fe2700b5af0 2024-04-24T23:11:27.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.443+0000 7fe275257640 1 -- 172.21.15.32:0/1780344460 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe264032810 con 0x7fe2700b5af0 2024-04-24T23:11:27.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.444+0000 7fe2784bd640 1 -- 172.21.15.32:0/1780344460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2700b5af0 msgr2=0x7fe2700ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:27.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.444+0000 7fe2784bd640 1 --2- 172.21.15.32:0/1780344460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2700b5af0 0x7fe2700ccc90 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7fe264009870 tx=0x7fe26402a180 comp rx=0 tx=0).stop 2024-04-24T23:11:27.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.444+0000 7fe2784bd640 1 -- 172.21.15.32:0/1780344460 shutdown_connections 2024-04-24T23:11:27.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.444+0000 7fe2784bd640 1 --2- 172.21.15.32:0/1780344460 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe270162e10 0x7fe27015ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:27.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.444+0000 7fe2784bd640 1 --2- 172.21.15.32:0/1780344460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2700b5af0 0x7fe2700ccc90 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:27.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.444+0000 7fe2784bd640 1 -- 172.21.15.32:0/1780344460 >> 172.21.15.32:0/1780344460 conn(0x7fe2700d0ae0 msgr2=0x7fe2700d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:27.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.444+0000 7fe2784bd640 1 -- 172.21.15.32:0/1780344460 shutdown_connections 2024-04-24T23:11:27.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.444+0000 7fe2784bd640 1 -- 172.21.15.32:0/1780344460 wait complete. 2024-04-24T23:11:27.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.445+0000 7fe2784bd640 1 Processor -- start 2024-04-24T23:11:27.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.445+0000 7fe2784bd640 1 -- start start 2024-04-24T23:11:27.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.446+0000 7fe2784bd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2700b5af0 0x7fe2701613c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:27.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.446+0000 7fe2784bd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe270162320 0x7fe270150600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:27.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.446+0000 7fe2784bd640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe270162e10 0x7fe2700ce710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:27.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.446+0000 7fe2784bd640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe270071bd0 con 0x7fe270162320 2024-04-24T23:11:27.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.446+0000 7fe2784bd640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe270067950 con 0x7fe2700b5af0 2024-04-24T23:11:27.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.446+0000 7fe2784bd640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fe270071d50 con 0x7fe270162e10 2024-04-24T23:11:27.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.446+0000 7fe275a58640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe270162320 0x7fe270150600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:27.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.446+0000 7fe275a58640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe270162320 0x7fe270150600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54700/0 (socket says 172.21.15.32:54700) 2024-04-24T23:11:27.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.446+0000 7fe275a58640 1 -- 172.21.15.32:0/2205906363 learned_addr learned my addr 172.21.15.32:0/2205906363 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:27.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.446+0000 7fe276a5a640 1 --2- 172.21.15.32:0/2205906363 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe270162e10 0x7fe2700ce710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:27.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.446+0000 7fe276259640 1 --2- 172.21.15.32:0/2205906363 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2700b5af0 0x7fe2701613c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:27.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.446+0000 7fe275a58640 1 -- 172.21.15.32:0/2205906363 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe270162e10 msgr2=0x7fe2700ce710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:27.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.446+0000 7fe275a58640 1 --2- 172.21.15.32:0/2205906363 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe270162e10 0x7fe2700ce710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:27.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.446+0000 7fe275a58640 1 -- 172.21.15.32:0/2205906363 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2700b5af0 msgr2=0x7fe2701613c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:27.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.446+0000 7fe275a58640 1 --2- 172.21.15.32:0/2205906363 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2700b5af0 0x7fe2701613c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:27.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.446+0000 7fe275a58640 1 -- 172.21.15.32:0/2205906363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2700cec50 con 0x7fe270162320 2024-04-24T23:11:27.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.446+0000 7fe275a58640 1 --2- 172.21.15.32:0/2205906363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe270162320 0x7fe270150600 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fe264009870 tx=0x7fe2640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:27.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.447+0000 7fe26effd640 1 -- 172.21.15.32:0/2205906363 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe264041020 con 0x7fe270162320 2024-04-24T23:11:27.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.447+0000 7fe26effd640 1 -- 172.21.15.32:0/2205906363 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe264030970 con 0x7fe270162320 2024-04-24T23:11:27.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.447+0000 7fe2784bd640 1 -- 172.21.15.32:0/2205906363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2700cef00 con 0x7fe270162320 2024-04-24T23:11:27.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.447+0000 7fe26effd640 1 -- 172.21.15.32:0/2205906363 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe264037040 con 0x7fe270162320 2024-04-24T23:11:27.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.447+0000 7fe2784bd640 1 -- 172.21.15.32:0/2205906363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2700cf4c0 con 0x7fe270162320 2024-04-24T23:11:27.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.448+0000 7fe2784bd640 1 -- 172.21.15.32:0/2205906363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2700af960 con 0x7fe270162320 2024-04-24T23:11:27.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.448+0000 7fe26effd640 1 -- 172.21.15.32:0/2205906363 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe264044020 con 0x7fe270162320 2024-04-24T23:11:27.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.448+0000 7fe26effd640 1 --2- 172.21.15.32:0/2205906363 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe25c06be30 0x7fe25c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:27.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.449+0000 7fe26effd640 1 -- 172.21.15.32:0/2205906363 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 4943+0+0 (secure 0 0 0) 0x7fe2640ab490 con 0x7fe270162320 2024-04-24T23:11:27.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.449+0000 7fe276259640 1 --2- 172.21.15.32:0/2205906363 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe25c06be30 0x7fe25c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:27.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.449+0000 7fe276259640 1 --2- 172.21.15.32:0/2205906363 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe25c06be30 0x7fe25c06e2f0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fe268003d90 tx=0x7fe268009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:27.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.453+0000 7fe26effd640 1 -- 172.21.15.32:0/2205906363 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe2700af960 con 0x7fe270162320 2024-04-24T23:11:27.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.637+0000 7fe2784bd640 1 -- 172.21.15.32:0/2205906363 --> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7fe270068430 con 0x7fe25c06be30 2024-04-24T23:11:27.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.637+0000 7fe26effd640 1 -- 172.21.15.32:0/2205906363 <== mgr.4108 v2:172.21.15.121:6832/1848157943 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fe270068430 con 0x7fe25c06be30 2024-04-24T23:11:27.640 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 6 to deep-scrub 2024-04-24T23:11:27.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.639+0000 7fe2784bd640 1 -- 172.21.15.32:0/2205906363 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe25c06be30 msgr2=0x7fe25c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:27.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.639+0000 7fe2784bd640 1 --2- 172.21.15.32:0/2205906363 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe25c06be30 0x7fe25c06e2f0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fe268003d90 tx=0x7fe268009040 comp rx=0 tx=0).stop 2024-04-24T23:11:27.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.639+0000 7fe2784bd640 1 -- 172.21.15.32:0/2205906363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe270162320 msgr2=0x7fe270150600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:27.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.639+0000 7fe2784bd640 1 --2- 172.21.15.32:0/2205906363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe270162320 0x7fe270150600 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fe264009870 tx=0x7fe2640046c0 comp rx=0 tx=0).stop 2024-04-24T23:11:27.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.639+0000 7fe2784bd640 1 -- 172.21.15.32:0/2205906363 shutdown_connections 2024-04-24T23:11:27.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.639+0000 7fe2784bd640 1 --2- 172.21.15.32:0/2205906363 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fe25c06be30 0x7fe25c06e2f0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:27.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.639+0000 7fe2784bd640 1 --2- 172.21.15.32:0/2205906363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe270162320 0x7fe270150600 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:27.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.639+0000 7fe2784bd640 1 --2- 172.21.15.32:0/2205906363 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe270162e10 0x7fe2700ce710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:27.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.639+0000 7fe2784bd640 1 --2- 172.21.15.32:0/2205906363 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2700b5af0 0x7fe2701613c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:27.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.639+0000 7fe2784bd640 1 -- 172.21.15.32:0/2205906363 >> 172.21.15.32:0/2205906363 conn(0x7fe2700d0ae0 msgr2=0x7fe270162700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:27.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.639+0000 7fe2784bd640 1 -- 172.21.15.32:0/2205906363 shutdown_connections 2024-04-24T23:11:27.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.639+0000 7fe2784bd640 1 -- 172.21.15.32:0/2205906363 wait complete. 2024-04-24T23:11:27.653 INFO:tasks.ceph:Scrubbing osd.7 2024-04-24T23:11:27.653 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-24T23:11:27.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.755+0000 7f9eab177640 1 Processor -- start 2024-04-24T23:11:27.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.755+0000 7f9eab177640 1 -- start start 2024-04-24T23:11:27.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.756+0000 7f9eab177640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9ea40b5ff0 0x7f9ea40cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:27.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.756+0000 7f9eab177640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ea415bde0 con 0x7f9ea415b580 2024-04-24T23:11:27.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.756+0000 7f9eab177640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ea415c0c0 con 0x7f9ea415b960 2024-04-24T23:11:27.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.756+0000 7f9eab177640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f9ea415c9f0 con 0x7f9ea40b5ff0 2024-04-24T23:11:27.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.756+0000 7f9ea8f13640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9ea415b580 0x7f9ea406cb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56420/0 (socket says 172.21.15.32:56420) 2024-04-24T23:11:27.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.756+0000 7f9ea8f13640 1 -- 172.21.15.32:0/855236797 learned_addr learned my addr 172.21.15.32:0/855236797 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:11:27.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.756+0000 7f9ea9714640 1 --2- 172.21.15.32:0/855236797 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9ea40b5ff0 0x7f9ea40cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:27.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.756+0000 7f9ea37fe640 1 -- 172.21.15.32:0/855236797 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3930870235 0 0) 0x7f9ea415bde0 con 0x7f9ea415b580 2024-04-24T23:11:27.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.756+0000 7f9ea37fe640 1 -- 172.21.15.32:0/855236797 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9e94003520 con 0x7f9ea415b580 2024-04-24T23:11:27.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.756+0000 7f9ea37fe640 1 -- 172.21.15.32:0/855236797 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1295276961 0 0) 0x7f9ea415c0c0 con 0x7f9ea415b960 2024-04-24T23:11:27.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.756+0000 7f9ea37fe640 1 -- 172.21.15.32:0/855236797 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ea415bde0 con 0x7f9ea415b960 2024-04-24T23:11:27.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.756+0000 7f9ea37fe640 1 -- 172.21.15.32:0/855236797 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1787092142 0 0) 0x7f9e94003520 con 0x7f9ea415b580 2024-04-24T23:11:27.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.756+0000 7f9ea37fe640 1 -- 172.21.15.32:0/855236797 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9ea40b5ff0 msgr2=0x7f9ea40cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:27.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.756+0000 7f9ea37fe640 1 --2- 172.21.15.32:0/855236797 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9ea40b5ff0 0x7f9ea40cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:27.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.756+0000 7f9ea37fe640 1 -- 172.21.15.32:0/855236797 >> v1:172.21.15.121:6789/0 conn(0x7f9ea415b960 legacy=0x7f9ea40b58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:27.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.756+0000 7f9ea37fe640 1 -- 172.21.15.32:0/855236797 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ea415c3a0 con 0x7f9ea415b580 2024-04-24T23:11:27.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.757+0000 7f9ea37fe640 1 -- 172.21.15.32:0/855236797 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7f9e98001670 con 0x7f9ea415b580 2024-04-24T23:11:27.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.757+0000 7f9ea9714640 1 --2- 172.21.15.32:0/855236797 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9ea40b5ff0 0x7f9ea40cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:11:27.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.757+0000 7f9ea37fe640 1 -- 172.21.15.32:0/855236797 >> v1:172.21.15.32:6789/0 conn(0x7f9ea415b580 legacy=0x7f9ea406cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:27.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.757+0000 7f9ea37fe640 1 --2- 172.21.15.32:0/855236797 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9ea40b5ff0 0x7f9e94003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:27.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.757+0000 7f9ea37fe640 1 --2- 172.21.15.32:0/855236797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e940040b0 0x7f9e94004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:27.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.757+0000 7f9ea37fe640 1 --2- 172.21.15.32:0/855236797 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e94004d40 0x7f9e940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:27.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.757+0000 7f9ea37fe640 1 -- 172.21.15.32:0/855236797 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea415c9f0 con 0x7f9e940040b0 2024-04-24T23:11:27.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.757+0000 7f9ea37fe640 1 -- 172.21.15.32:0/855236797 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9400ba70 con 0x7f9e94004d40 2024-04-24T23:11:27.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.757+0000 7f9ea37fe640 1 -- 172.21.15.32:0/855236797 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f9e9400bc70 con 0x7f9ea40b5ff0 2024-04-24T23:11:27.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.757+0000 7f9ea3fff640 1 --2- 172.21.15.32:0/855236797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e940040b0 0x7f9e94004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:27.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.757+0000 7f9ea8f13640 1 --2- 172.21.15.32:0/855236797 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9ea40b5ff0 0x7f9e94003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:27.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.757+0000 7f9ea9714640 1 --2- 172.21.15.32:0/855236797 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e94004d40 0x7f9e940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:27.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.757+0000 7f9ea3fff640 1 -- 172.21.15.32:0/855236797 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9ea40b5ff0 msgr2=0x7f9e94003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:27.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.757+0000 7f9ea3fff640 1 --2- 172.21.15.32:0/855236797 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9ea40b5ff0 0x7f9e94003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:27.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.757+0000 7f9ea3fff640 1 -- 172.21.15.32:0/855236797 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e94004d40 msgr2=0x7f9e940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:27.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.757+0000 7f9ea3fff640 1 --2- 172.21.15.32:0/855236797 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e94004d40 0x7f9e940051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:27.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.757+0000 7f9ea3fff640 1 -- 172.21.15.32:0/855236797 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e94003520 con 0x7f9e940040b0 2024-04-24T23:11:27.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.758+0000 7f9ea3fff640 1 --2- 172.21.15.32:0/855236797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e940040b0 0x7f9e94004800 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f9e900018a0 tx=0x7f9e94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3547d8b1aaa831c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:27.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.758+0000 7f9ea37fe640 1 -- 172.21.15.32:0/855236797 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e900014f0 con 0x7f9e940040b0 2024-04-24T23:11:27.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.758+0000 7f9ea37fe640 1 -- 172.21.15.32:0/855236797 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e900052d0 con 0x7f9e940040b0 2024-04-24T23:11:27.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.758+0000 7f9ea37fe640 1 -- 172.21.15.32:0/855236797 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9002ac60 con 0x7f9e940040b0 2024-04-24T23:11:27.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.758+0000 7f9eab177640 1 -- 172.21.15.32:0/855236797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e940040b0 msgr2=0x7f9e94004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:27.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.758+0000 7f9eab177640 1 --2- 172.21.15.32:0/855236797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e940040b0 0x7f9e94004800 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f9e900018a0 tx=0x7f9e94005830 comp rx=0 tx=0).stop 2024-04-24T23:11:27.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.758+0000 7f9ea9714640 1 -- 172.21.15.32:0/855236797 reap_dead start 2024-04-24T23:11:27.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.759+0000 7f9eab177640 1 -- 172.21.15.32:0/855236797 shutdown_connections 2024-04-24T23:11:27.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.759+0000 7f9eab177640 1 -- 172.21.15.32:0/855236797 >> 172.21.15.32:0/855236797 conn(0x7f9ea40b1f10 msgr2=0x7f9ea40b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:27.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.759+0000 7f9eab177640 1 -- 172.21.15.32:0/855236797 shutdown_connections 2024-04-24T23:11:27.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.759+0000 7f9eab177640 1 -- 172.21.15.32:0/855236797 wait complete. 2024-04-24T23:11:27.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.760+0000 7f9eab177640 1 Processor -- start 2024-04-24T23:11:27.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.760+0000 7f9eab177640 1 -- start start 2024-04-24T23:11:27.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.761+0000 7f9eab177640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e940040b0 0x7f9ea414df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:27.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.761+0000 7f9eab177640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ea414fe00 0x7f9ea413f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:27.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.761+0000 7f9eab177640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ea413f7d0 0x7f9ea406caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:27.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.761+0000 7f9eab177640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea406c920 con 0x7f9ea413f7d0 2024-04-24T23:11:27.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.761+0000 7f9eab177640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea406c7a0 con 0x7f9ea414fe00 2024-04-24T23:11:27.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.761+0000 7f9eab177640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f9ea40cde80 con 0x7f9e940040b0 2024-04-24T23:11:27.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.761+0000 7f9ea9714640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ea413f7d0 0x7f9ea406caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:27.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.761+0000 7f9ea9714640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ea413f7d0 0x7f9ea406caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54722/0 (socket says 172.21.15.32:54722) 2024-04-24T23:11:27.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.761+0000 7f9ea9714640 1 -- 172.21.15.32:0/3853934085 learned_addr learned my addr 172.21.15.32:0/3853934085 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:27.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.761+0000 7f9ea3fff640 1 --2- 172.21.15.32:0/3853934085 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ea414fe00 0x7f9ea413f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:27.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.761+0000 7f9ea8f13640 1 --2- 172.21.15.32:0/3853934085 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e940040b0 0x7f9ea414df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:27.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.761+0000 7f9ea9714640 1 -- 172.21.15.32:0/3853934085 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e940040b0 msgr2=0x7f9ea414df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:27.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.761+0000 7f9ea9714640 1 --2- 172.21.15.32:0/3853934085 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e940040b0 0x7f9ea414df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:27.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.761+0000 7f9ea9714640 1 -- 172.21.15.32:0/3853934085 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ea414fe00 msgr2=0x7f9ea413f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:27.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.761+0000 7f9ea9714640 1 --2- 172.21.15.32:0/3853934085 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ea414fe00 0x7f9ea413f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:27.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.761+0000 7f9ea9714640 1 -- 172.21.15.32:0/3853934085 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ea41504c0 con 0x7f9ea413f7d0 2024-04-24T23:11:27.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.761+0000 7f9ea8f13640 1 --2- 172.21.15.32:0/3853934085 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e940040b0 0x7f9ea414df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:11:27.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.761+0000 7f9ea9714640 1 --2- 172.21.15.32:0/3853934085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ea413f7d0 0x7f9ea406caa0 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f9ea406ed60 tx=0x7f9e9c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:27.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.761+0000 7f9ea17fa640 1 -- 172.21.15.32:0/3853934085 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c00fd10 con 0x7f9ea413f7d0 2024-04-24T23:11:27.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.761+0000 7f9ea17fa640 1 -- 172.21.15.32:0/3853934085 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e9c003990 con 0x7f9ea413f7d0 2024-04-24T23:11:27.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.761+0000 7f9ea17fa640 1 -- 172.21.15.32:0/3853934085 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c003d80 con 0x7f9ea413f7d0 2024-04-24T23:11:27.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.761+0000 7f9eab177640 1 -- 172.21.15.32:0/3853934085 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ea41507a0 con 0x7f9ea413f7d0 2024-04-24T23:11:27.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.761+0000 7f9eab177640 1 -- 172.21.15.32:0/3853934085 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ea4150d60 con 0x7f9ea413f7d0 2024-04-24T23:11:27.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.762+0000 7f9eab177640 1 -- 172.21.15.32:0/3853934085 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9ea4150260 con 0x7f9ea413f7d0 2024-04-24T23:11:27.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.763+0000 7f9ea17fa640 1 -- 172.21.15.32:0/3853934085 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9e9c004030 con 0x7f9ea413f7d0 2024-04-24T23:11:27.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.763+0000 7f9ea17fa640 1 --2- 172.21.15.32:0/3853934085 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9e8806be60 0x7f9e8806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:27.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.763+0000 7f9ea17fa640 1 -- 172.21.15.32:0/3853934085 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 4943+0+0 (secure 0 0 0) 0x7f9e9c08b3c0 con 0x7f9ea413f7d0 2024-04-24T23:11:27.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.763+0000 7f9ea17fa640 1 --2- 172.21.15.32:0/3853934085 >> [v2:172.21.15.121:6816/3250005122,v1:172.21.15.121:6817/3250005122] conn(0x7f9e88071d20 0x7f9e88074140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:27.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.763+0000 7f9ea17fa640 1 -- 172.21.15.32:0/3853934085 --> [v2:172.21.15.121:6816/3250005122,v1:172.21.15.121:6817/3250005122] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9e9c01bec0 con 0x7f9e88071d20 2024-04-24T23:11:27.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.763+0000 7f9ea17fa640 1 -- 172.21.15.32:0/3853934085 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=194) ==== 24+0+0 (secure 0 0 0) 0x7f9e9c092050 con 0x7f9ea413f7d0 2024-04-24T23:11:27.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.763+0000 7f9ea8f13640 1 --2- 172.21.15.32:0/3853934085 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9e8806be60 0x7f9e8806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:27.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.764+0000 7f9ea3fff640 1 --2- 172.21.15.32:0/3853934085 >> [v2:172.21.15.121:6816/3250005122,v1:172.21.15.121:6817/3250005122] conn(0x7f9e88071d20 0x7f9e88074140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:27.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.764+0000 7f9ea8f13640 1 --2- 172.21.15.32:0/3853934085 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9e8806be60 0x7f9e8806e320 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f9e98001eb0 tx=0x7f9e98009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:27.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.764+0000 7f9ea3fff640 1 --2- 172.21.15.32:0/3853934085 >> [v2:172.21.15.121:6816/3250005122,v1:172.21.15.121:6817/3250005122] conn(0x7f9e88071d20 0x7f9e88074140 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:27.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.765+0000 7f9ea17fa640 1 -- 172.21.15.32:0/3853934085 <== osd.7 v2:172.21.15.121:6816/3250005122 1 ==== command_reply(tid 1: 0 ) ==== 8+0+25033 (crc 0 0 0) 0x7f9e9c01bec0 con 0x7f9e88071d20 2024-04-24T23:11:27.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.777+0000 7f9eab177640 1 -- 172.21.15.32:0/3853934085 --> [v2:172.21.15.121:6816/3250005122,v1:172.21.15.121:6817/3250005122] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f9ea40b1b40 con 0x7f9e88071d20 2024-04-24T23:11:27.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.777+0000 7f9ea17fa640 1 -- 172.21.15.32:0/3853934085 <== osd.7 v2:172.21.15.121:6816/3250005122 2 ==== command_reply(tid 2: 0 ) ==== 8+0+565 (crc 0 0 0) 0x7f9ea40b1b40 con 0x7f9e88071d20 2024-04-24T23:11:27.780 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-24T23:11:27.780 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-04-24T23:11:27.780 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-24T23:11:27.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.778+0000 7f9eab177640 1 -- 172.21.15.32:0/3853934085 >> [v2:172.21.15.121:6816/3250005122,v1:172.21.15.121:6817/3250005122] conn(0x7f9e88071d20 msgr2=0x7f9e88074140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:27.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.778+0000 7f9eab177640 1 --2- 172.21.15.32:0/3853934085 >> [v2:172.21.15.121:6816/3250005122,v1:172.21.15.121:6817/3250005122] conn(0x7f9e88071d20 0x7f9e88074140 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:27.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.778+0000 7f9eab177640 1 -- 172.21.15.32:0/3853934085 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9e8806be60 msgr2=0x7f9e8806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:27.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.778+0000 7f9eab177640 1 --2- 172.21.15.32:0/3853934085 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9e8806be60 0x7f9e8806e320 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f9e98001eb0 tx=0x7f9e98009040 comp rx=0 tx=0).stop 2024-04-24T23:11:27.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.778+0000 7f9eab177640 1 -- 172.21.15.32:0/3853934085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ea413f7d0 msgr2=0x7f9ea406caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:27.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.778+0000 7f9eab177640 1 --2- 172.21.15.32:0/3853934085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ea413f7d0 0x7f9ea406caa0 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f9ea406ed60 tx=0x7f9e9c009f90 comp rx=0 tx=0).stop 2024-04-24T23:11:27.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.778+0000 7f9ea9714640 1 -- 172.21.15.32:0/3853934085 reap_dead start 2024-04-24T23:11:27.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.778+0000 7f9eab177640 1 -- 172.21.15.32:0/3853934085 shutdown_connections 2024-04-24T23:11:27.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.778+0000 7f9eab177640 1 -- 172.21.15.32:0/3853934085 >> 172.21.15.32:0/3853934085 conn(0x7f9ea40b1f10 msgr2=0x7f9ea40cffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:27.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.778+0000 7f9eab177640 1 -- 172.21.15.32:0/3853934085 shutdown_connections 2024-04-24T23:11:27.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.778+0000 7f9eab177640 1 -- 172.21.15.32:0/3853934085 wait complete. 2024-04-24T23:11:27.791 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-24T23:11:27.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.889+0000 7f198af09640 1 Processor -- start 2024-04-24T23:11:27.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.889+0000 7f198af09640 1 -- start start 2024-04-24T23:11:27.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.890+0000 7f198af09640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1984158dd0 0x7f198406b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:27.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.890+0000 7f198af09640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f198406dc60 0x7f19840b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:27.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.890+0000 7f198af09640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f198406bd10 con 0x7f198406b860 2024-04-24T23:11:27.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.890+0000 7f198af09640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f198415ef10 con 0x7f1984158dd0 2024-04-24T23:11:27.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.890+0000 7f198af09640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f198415f1f0 con 0x7f198406dc60 2024-04-24T23:11:27.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.890+0000 7f19894a6640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f198406dc60 0x7f19840b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:27.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.890+0000 7f1988ca5640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1984158dd0 0x7f198406b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:27.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.890+0000 7f19894a6640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f198406dc60 0x7f19840b5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.32:44274/0 (socket says 172.21.15.32:44274) 2024-04-24T23:11:27.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.890+0000 7f1988ca5640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1984158dd0 0x7f198406b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.32:47426/0 (socket says 172.21.15.32:47426) 2024-04-24T23:11:27.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.890+0000 7f19894a6640 1 -- 172.21.15.32:0/2799990532 learned_addr learned my addr 172.21.15.32:0/2799990532 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:27.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.890+0000 7f1983fff640 1 --1- 172.21.15.32:0/2799990532 >> v1:172.21.15.121:6789/0 conn(0x7f198406b860 0x7f198406d550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.32:57966/0 (socket says 172.21.15.32:57966) 2024-04-24T23:11:27.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.891+0000 7f1988ca5640 1 -- 172.21.15.32:0/2799990532 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f198406dc60 msgr2=0x7f19840b5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:27.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.891+0000 7f1988ca5640 1 --2- 172.21.15.32:0/2799990532 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f198406dc60 0x7f19840b5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:27.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.891+0000 7f1988ca5640 1 -- 172.21.15.32:0/2799990532 >> v1:172.21.15.121:6789/0 conn(0x7f198406b860 legacy=0x7f198406d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:27.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.891+0000 7f1988ca5640 1 -- 172.21.15.32:0/2799990532 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f198415f4d0 con 0x7f1984158dd0 2024-04-24T23:11:27.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.891+0000 7f19837fe640 1 -- 172.21.15.32:0/2799990532 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1982733955 0 0) 0x7f198406bd10 con 0x7f198406b860 2024-04-24T23:11:27.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.891+0000 7f1988ca5640 1 --2- 172.21.15.32:0/2799990532 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1984158dd0 0x7f198406b320 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f197c009d90 tx=0x7f197c02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd433ddb9e1a44ec server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:27.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.892+0000 7f19837fe640 1 -- 172.21.15.32:0/2799990532 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197c007740 con 0x7f1984158dd0 2024-04-24T23:11:27.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.892+0000 7f19837fe640 1 -- 172.21.15.32:0/2799990532 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f197c0078e0 con 0x7f1984158dd0 2024-04-24T23:11:27.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.892+0000 7f19837fe640 1 -- 172.21.15.32:0/2799990532 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197c007cb0 con 0x7f1984158dd0 2024-04-24T23:11:27.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.892+0000 7f198af09640 1 -- 172.21.15.32:0/2799990532 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1984158dd0 msgr2=0x7f198406b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:27.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.892+0000 7f198af09640 1 --2- 172.21.15.32:0/2799990532 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1984158dd0 0x7f198406b320 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f197c009d90 tx=0x7f197c02a660 comp rx=0 tx=0).stop 2024-04-24T23:11:27.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.892+0000 7f198af09640 1 -- 172.21.15.32:0/2799990532 shutdown_connections 2024-04-24T23:11:27.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.892+0000 7f198af09640 1 --2- 172.21.15.32:0/2799990532 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f198406dc60 0x7f19840b5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:27.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.892+0000 7f198af09640 1 --2- 172.21.15.32:0/2799990532 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1984158dd0 0x7f198406b320 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:27.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.892+0000 7f198af09640 1 -- 172.21.15.32:0/2799990532 >> 172.21.15.32:0/2799990532 conn(0x7f198406e800 msgr2=0x7f198406ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:27.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.892+0000 7f198af09640 1 -- 172.21.15.32:0/2799990532 shutdown_connections 2024-04-24T23:11:27.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.892+0000 7f198af09640 1 -- 172.21.15.32:0/2799990532 wait complete. 2024-04-24T23:11:27.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.893+0000 7f198af09640 1 Processor -- start 2024-04-24T23:11:27.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.893+0000 7f198af09640 1 -- start start 2024-04-24T23:11:27.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.894+0000 7f198af09640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f198406b860 0x7f198415cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:27.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.894+0000 7f198af09640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f198406dc60 0x7f198413ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:27.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.894+0000 7f198af09640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1984158dd0 0x7f198415c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:27.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.894+0000 7f198af09640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f198406bfe0 con 0x7f198406b860 2024-04-24T23:11:27.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.894+0000 7f198af09640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f198415fcd0 con 0x7f1984158dd0 2024-04-24T23:11:27.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.894+0000 7f198af09640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f19841591b0 con 0x7f198406dc60 2024-04-24T23:11:27.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.894+0000 7f1988ca5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f198406b860 0x7f198415cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:27.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.894+0000 7f1988ca5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f198406b860 0x7f198415cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54734/0 (socket says 172.21.15.32:54734) 2024-04-24T23:11:27.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.894+0000 7f1988ca5640 1 -- 172.21.15.32:0/1687203261 learned_addr learned my addr 172.21.15.32:0/1687203261 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:27.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.895+0000 7f19894a6640 1 --2- 172.21.15.32:0/1687203261 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1984158dd0 0x7f198415c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:27.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.895+0000 7f1983fff640 1 --2- 172.21.15.32:0/1687203261 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f198406dc60 0x7f198413ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:27.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.895+0000 7f1988ca5640 1 -- 172.21.15.32:0/1687203261 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f198406dc60 msgr2=0x7f198413ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:27.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.895+0000 7f1988ca5640 1 --2- 172.21.15.32:0/1687203261 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f198406dc60 0x7f198413ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:27.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.895+0000 7f1988ca5640 1 -- 172.21.15.32:0/1687203261 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1984158dd0 msgr2=0x7f198415c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:27.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.895+0000 7f1988ca5640 1 --2- 172.21.15.32:0/1687203261 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1984158dd0 0x7f198415c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:27.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.895+0000 7f1988ca5640 1 -- 172.21.15.32:0/1687203261 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19841f42d0 con 0x7f198406b860 2024-04-24T23:11:27.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.895+0000 7f19894a6640 1 --2- 172.21.15.32:0/1687203261 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1984158dd0 0x7f198415c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:11:27.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.895+0000 7f1988ca5640 1 --2- 172.21.15.32:0/1687203261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f198406b860 0x7f198415cfe0 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f197c009d90 tx=0x7f197c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:27.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.895+0000 7f19817fa640 1 -- 172.21.15.32:0/1687203261 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197c03f070 con 0x7f198406b860 2024-04-24T23:11:27.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.895+0000 7f19817fa640 1 -- 172.21.15.32:0/1687203261 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f197c031c40 con 0x7f198406b860 2024-04-24T23:11:27.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.895+0000 7f198af09640 1 -- 172.21.15.32:0/1687203261 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19841f4590 con 0x7f198406b860 2024-04-24T23:11:27.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.895+0000 7f19817fa640 1 -- 172.21.15.32:0/1687203261 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197c009450 con 0x7f198406b860 2024-04-24T23:11:27.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.895+0000 7f198af09640 1 -- 172.21.15.32:0/1687203261 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19841f4b50 con 0x7f198406b860 2024-04-24T23:11:27.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.896+0000 7f198af09640 1 -- 172.21.15.32:0/1687203261 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1984068430 con 0x7f198406b860 2024-04-24T23:11:27.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.897+0000 7f19817fa640 1 -- 172.21.15.32:0/1687203261 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f197c043410 con 0x7f198406b860 2024-04-24T23:11:27.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.897+0000 7f19817fa640 1 --2- 172.21.15.32:0/1687203261 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f196406be60 0x7f196406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:27.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.897+0000 7f19817fa640 1 -- 172.21.15.32:0/1687203261 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 4943+0+0 (secure 0 0 0) 0x7f197c0abfb0 con 0x7f198406b860 2024-04-24T23:11:27.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.898+0000 7f1983fff640 1 --2- 172.21.15.32:0/1687203261 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f196406be60 0x7f196406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:27.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.898+0000 7f1983fff640 1 --2- 172.21.15.32:0/1687203261 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f196406be60 0x7f196406e320 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f19840b55c0 tx=0x7f1974007b50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:27.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:27.902+0000 7f19817fa640 1 -- 172.21.15.32:0/1687203261 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f197c07ac60 con 0x7f198406b860 2024-04-24T23:11:28.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.083+0000 7f198af09640 1 -- 172.21.15.32:0/1687203261 --> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f1984142510 con 0x7f196406be60 2024-04-24T23:11:28.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.084+0000 7f19817fa640 1 -- 172.21.15.32:0/1687203261 <== mgr.4108 v2:172.21.15.121:6832/1848157943 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f1984142510 con 0x7f196406be60 2024-04-24T23:11:28.086 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 7 to deep-scrub 2024-04-24T23:11:28.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.085+0000 7f198af09640 1 -- 172.21.15.32:0/1687203261 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f196406be60 msgr2=0x7f196406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:28.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.085+0000 7f198af09640 1 --2- 172.21.15.32:0/1687203261 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f196406be60 0x7f196406e320 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f19840b55c0 tx=0x7f1974007b50 comp rx=0 tx=0).stop 2024-04-24T23:11:28.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.085+0000 7f198af09640 1 -- 172.21.15.32:0/1687203261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f198406b860 msgr2=0x7f198415cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:28.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.085+0000 7f198af09640 1 --2- 172.21.15.32:0/1687203261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f198406b860 0x7f198415cfe0 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f197c009d90 tx=0x7f197c004a40 comp rx=0 tx=0).stop 2024-04-24T23:11:28.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.085+0000 7f198af09640 1 -- 172.21.15.32:0/1687203261 shutdown_connections 2024-04-24T23:11:28.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.085+0000 7f198af09640 1 --2- 172.21.15.32:0/1687203261 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f196406be60 0x7f196406e320 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:28.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.085+0000 7f198af09640 1 --2- 172.21.15.32:0/1687203261 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1984158dd0 0x7f198415c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:28.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.085+0000 7f198af09640 1 --2- 172.21.15.32:0/1687203261 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f198406dc60 0x7f198413ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:28.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.085+0000 7f198af09640 1 --2- 172.21.15.32:0/1687203261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f198406b860 0x7f198415cfe0 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:28.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.085+0000 7f198af09640 1 -- 172.21.15.32:0/1687203261 >> 172.21.15.32:0/1687203261 conn(0x7f198406e800 msgr2=0x7f1984157e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:28.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.085+0000 7f198af09640 1 -- 172.21.15.32:0/1687203261 shutdown_connections 2024-04-24T23:11:28.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.085+0000 7f198af09640 1 -- 172.21.15.32:0/1687203261 wait complete. 2024-04-24T23:11:28.099 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:11:28.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.197+0000 7f9c5a2c4640 1 Processor -- start 2024-04-24T23:11:28.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.197+0000 7f9c5a2c4640 1 -- start start 2024-04-24T23:11:28.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.198+0000 7f9c5a2c4640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c5406c5f0 0x7f9c5406c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:28.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.198+0000 7f9c5a2c4640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9c540af890 0x7f9c5415cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:28.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.198+0000 7f9c5a2c4640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c540b6430 con 0x7f9c540b44d0 2024-04-24T23:11:28.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.198+0000 7f9c5a2c4640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f9c54156f80 con 0x7f9c540af890 2024-04-24T23:11:28.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.198+0000 7f9c5a2c4640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f9c540b6150 con 0x7f9c5406c5f0 2024-04-24T23:11:28.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.198+0000 7f9c58861640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9c540af890 0x7f9c5415cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:28.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.198+0000 7f9c58861640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9c540af890 0x7f9c5415cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54750/0 (socket says 172.21.15.32:54750) 2024-04-24T23:11:28.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.198+0000 7f9c58861640 1 -- 172.21.15.32:0/2818713674 learned_addr learned my addr 172.21.15.32:0/2818713674 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:28.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.198+0000 7f9c537fe640 1 --1- 172.21.15.32:0/2818713674 >> v1:172.21.15.32:6789/0 conn(0x7f9c540b44d0 0x7f9c540b5890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56426/0 (socket says 172.21.15.32:56426) 2024-04-24T23:11:28.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.198+0000 7f9c53fff640 1 --2- 172.21.15.32:0/2818713674 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c5406c5f0 0x7f9c5406c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:28.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.198+0000 7f9c58861640 1 -- 172.21.15.32:0/2818713674 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c5406c5f0 msgr2=0x7f9c5406c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:28.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.198+0000 7f9c58861640 1 --2- 172.21.15.32:0/2818713674 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c5406c5f0 0x7f9c5406c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:28.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.198+0000 7f9c58861640 1 -- 172.21.15.32:0/2818713674 >> v1:172.21.15.32:6789/0 conn(0x7f9c540b44d0 legacy=0x7f9c540b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:28.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.198+0000 7f9c58861640 1 -- 172.21.15.32:0/2818713674 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c540b39f0 con 0x7f9c540af890 2024-04-24T23:11:28.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.198+0000 7f9c52ffd640 1 -- 172.21.15.32:0/2818713674 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2225500398 0 0) 0x7f9c540b6430 con 0x7f9c540b44d0 2024-04-24T23:11:28.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.199+0000 7f9c58861640 1 --2- 172.21.15.32:0/2818713674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c540af890 0x7f9c5415cac0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c009e50 tx=0x7f9c4c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e6a77e926243b62 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:28.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.199+0000 7f9c52ffd640 1 -- 172.21.15.32:0/2818713674 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c036070 con 0x7f9c540af890 2024-04-24T23:11:28.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.199+0000 7f9c52ffd640 1 -- 172.21.15.32:0/2818713674 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c4c031600 con 0x7f9c540af890 2024-04-24T23:11:28.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.199+0000 7f9c52ffd640 1 -- 172.21.15.32:0/2818713674 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c0319d0 con 0x7f9c540af890 2024-04-24T23:11:28.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.200+0000 7f9c5a2c4640 1 -- 172.21.15.32:0/2818713674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c540af890 msgr2=0x7f9c5415cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:28.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.200+0000 7f9c5a2c4640 1 --2- 172.21.15.32:0/2818713674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c540af890 0x7f9c5415cac0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c009e50 tx=0x7f9c4c02a680 comp rx=0 tx=0).stop 2024-04-24T23:11:28.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.200+0000 7f9c5a2c4640 1 -- 172.21.15.32:0/2818713674 shutdown_connections 2024-04-24T23:11:28.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.200+0000 7f9c5a2c4640 1 --2- 172.21.15.32:0/2818713674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c540af890 0x7f9c5415cac0 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:28.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.200+0000 7f9c5a2c4640 1 --2- 172.21.15.32:0/2818713674 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c5406c5f0 0x7f9c5406c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:28.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.200+0000 7f9c5a2c4640 1 -- 172.21.15.32:0/2818713674 >> 172.21.15.32:0/2818713674 conn(0x7f9c5406b110 msgr2=0x7f9c5406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:28.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.200+0000 7f9c5a2c4640 1 -- 172.21.15.32:0/2818713674 shutdown_connections 2024-04-24T23:11:28.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.200+0000 7f9c5a2c4640 1 -- 172.21.15.32:0/2818713674 wait complete. 2024-04-24T23:11:28.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.201+0000 7f9c5a2c4640 1 Processor -- start 2024-04-24T23:11:28.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.201+0000 7f9c5a2c4640 1 -- start start 2024-04-24T23:11:28.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.202+0000 7f9c5a2c4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c5406c5f0 0x7f9c5415bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:28.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.202+0000 7f9c5a2c4640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c540af890 0x7f9c5414ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:28.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.202+0000 7f9c5a2c4640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c540b44d0 0x7f9c540ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:28.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.202+0000 7f9c5a2c4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9c5406cf10 con 0x7f9c5406c5f0 2024-04-24T23:11:28.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.202+0000 7f9c5a2c4640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9c540b3fb0 con 0x7f9c540b44d0 2024-04-24T23:11:28.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.202+0000 7f9c5a2c4640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f9c5415d000 con 0x7f9c540af890 2024-04-24T23:11:28.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.202+0000 7f9c53fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c5406c5f0 0x7f9c5415bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:28.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.202+0000 7f9c53fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c5406c5f0 0x7f9c5415bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54756/0 (socket says 172.21.15.32:54756) 2024-04-24T23:11:28.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.202+0000 7f9c53fff640 1 -- 172.21.15.32:0/2662539021 learned_addr learned my addr 172.21.15.32:0/2662539021 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:28.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.202+0000 7f9c58861640 1 --2- 172.21.15.32:0/2662539021 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c540b44d0 0x7f9c540ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:28.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.202+0000 7f9c537fe640 1 --2- 172.21.15.32:0/2662539021 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c540af890 0x7f9c5414ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:28.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.202+0000 7f9c53fff640 1 -- 172.21.15.32:0/2662539021 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c540af890 msgr2=0x7f9c5414ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:28.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.202+0000 7f9c53fff640 1 --2- 172.21.15.32:0/2662539021 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c540af890 0x7f9c5414ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:28.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.202+0000 7f9c53fff640 1 -- 172.21.15.32:0/2662539021 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c540b44d0 msgr2=0x7f9c540ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:28.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.202+0000 7f9c53fff640 1 --2- 172.21.15.32:0/2662539021 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c540b44d0 0x7f9c540ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:28.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.202+0000 7f9c53fff640 1 -- 172.21.15.32:0/2662539021 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c5415c510 con 0x7f9c5406c5f0 2024-04-24T23:11:28.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.202+0000 7f9c537fe640 1 --2- 172.21.15.32:0/2662539021 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c540af890 0x7f9c5414ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:11:28.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.203+0000 7f9c53fff640 1 --2- 172.21.15.32:0/2662539021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c5406c5f0 0x7f9c5415bfd0 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f9c4400d800 tx=0x7f9c4400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:28.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.203+0000 7f9c50ff9640 1 -- 172.21.15.32:0/2662539021 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4400b6c0 con 0x7f9c5406c5f0 2024-04-24T23:11:28.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.203+0000 7f9c50ff9640 1 -- 172.21.15.32:0/2662539021 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c44019070 con 0x7f9c5406c5f0 2024-04-24T23:11:28.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.203+0000 7f9c5a2c4640 1 -- 172.21.15.32:0/2662539021 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c5415bd80 con 0x7f9c5406c5f0 2024-04-24T23:11:28.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.203+0000 7f9c50ff9640 1 -- 172.21.15.32:0/2662539021 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4400ba80 con 0x7f9c5406c5f0 2024-04-24T23:11:28.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.203+0000 7f9c5a2c4640 1 -- 172.21.15.32:0/2662539021 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c540b1210 con 0x7f9c5406c5f0 2024-04-24T23:11:28.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.204+0000 7f9c5a2c4640 1 -- 172.21.15.32:0/2662539021 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c54068430 con 0x7f9c5406c5f0 2024-04-24T23:11:28.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.204+0000 7f9c50ff9640 1 -- 172.21.15.32:0/2662539021 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9c4400bc20 con 0x7f9c5406c5f0 2024-04-24T23:11:28.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.205+0000 7f9c50ff9640 1 --2- 172.21.15.32:0/2662539021 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9c4006be30 0x7f9c4006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:28.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.205+0000 7f9c50ff9640 1 -- 172.21.15.32:0/2662539021 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 4943+0+0 (secure 0 0 0) 0x7f9c4408b020 con 0x7f9c5406c5f0 2024-04-24T23:11:28.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.205+0000 7f9c537fe640 1 --2- 172.21.15.32:0/2662539021 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9c4006be30 0x7f9c4006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:28.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.206+0000 7f9c537fe640 1 --2- 172.21.15.32:0/2662539021 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9c4006be30 0x7f9c4006e2f0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f9c54157690 tx=0x7f9c48007de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:28.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.209+0000 7f9c50ff9640 1 -- 172.21.15.32:0/2662539021 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9c44059c40 con 0x7f9c5406c5f0 2024-04-24T23:11:28.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.387+0000 7f9c5a2c4640 1 -- 172.21.15.32:0/2662539021 --> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9c540b5670 con 0x7f9c4006be30 2024-04-24T23:11:28.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.388+0000 7f9c50ff9640 1 -- 172.21.15.32:0/2662539021 <== mgr.4108 v2:172.21.15.121:6832/1848157943 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+12592 (secure 0 0 0) 0x7f9c540b5670 con 0x7f9c4006be30 2024-04-24T23:11:28.391 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:11:28.391 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-24T23:11:28.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.390+0000 7f9c5a2c4640 1 -- 172.21.15.32:0/2662539021 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9c4006be30 msgr2=0x7f9c4006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:28.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.390+0000 7f9c5a2c4640 1 --2- 172.21.15.32:0/2662539021 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9c4006be30 0x7f9c4006e2f0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f9c54157690 tx=0x7f9c48007de0 comp rx=0 tx=0).stop 2024-04-24T23:11:28.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.390+0000 7f9c5a2c4640 1 -- 172.21.15.32:0/2662539021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c5406c5f0 msgr2=0x7f9c5415bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:28.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.390+0000 7f9c5a2c4640 1 --2- 172.21.15.32:0/2662539021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c5406c5f0 0x7f9c5415bfd0 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f9c4400d800 tx=0x7f9c4400dcc0 comp rx=0 tx=0).stop 2024-04-24T23:11:28.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.390+0000 7f9c5a2c4640 1 -- 172.21.15.32:0/2662539021 shutdown_connections 2024-04-24T23:11:28.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.390+0000 7f9c5a2c4640 1 --2- 172.21.15.32:0/2662539021 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f9c4006be30 0x7f9c4006e2f0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:28.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.390+0000 7f9c5a2c4640 1 --2- 172.21.15.32:0/2662539021 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c540b44d0 0x7f9c540ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:28.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.390+0000 7f9c5a2c4640 1 --2- 172.21.15.32:0/2662539021 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c540af890 0x7f9c5414ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:28.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.390+0000 7f9c5a2c4640 1 --2- 172.21.15.32:0/2662539021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c5406c5f0 0x7f9c5415bfd0 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:28.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.390+0000 7f9c5a2c4640 1 -- 172.21.15.32:0/2662539021 >> 172.21.15.32:0/2662539021 conn(0x7f9c5406b110 msgr2=0x7f9c54066ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:28.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.390+0000 7f9c5a2c4640 1 -- 172.21.15.32:0/2662539021 shutdown_connections 2024-04-24T23:11:28.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.390+0000 7f9c5a2c4640 1 -- 172.21.15.32:0/2662539021 wait complete. 2024-04-24T23:11:28.403 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":563,"stamp":"2024-04-24T23:11:27.305696+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":9726,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1552773,"statfs":{"total":1600000000,"available":1590040184,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-52491451,"num_objects":-36,"num_object_clones":0,"num_object_copies":-72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-36,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-132,"num_write_kb":-85089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001548"},"pg_stats":[{"pgid":"1.0","version":"194'119","reported_seq":526,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T23:11:25.487196+0000","last_change":"2024-04-24T23:11:25.487219+0000","last_active":"2024-04-24T23:11:25.487196+0000","last_peered":"2024-04-24T23:11:25.487196+0000","last_clean":"2024-04-24T23:11:25.487196+0000","last_became_active":"2024-04-24T22:58:55.138627+0000","last_became_peered":"2024-04-24T22:58:55.138627+0000","last_unstale":"2024-04-24T23:11:25.487196+0000","last_undegraded":"2024-04-24T23:11:25.487196+0000","last_fullsized":"2024-04-24T23:11:25.487196+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"194'119","last_scrub_stamp":"2024-04-24T23:11:25.487153+0000","last_deep_scrub":"194'119","last_deep_scrub_stamp":"2024-04-24T23:11:25.487153+0000","last_clean_scrub_stamp":"2024-04-24T23:11:25.487153+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:16.078162+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575001,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1071,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194240,"statfs":{"total":200000000,"available":198902615,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575001,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1071,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194240,"statfs":{"total":200000000,"available":198902615,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575001,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1648,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193664,"statfs":{"total":200000000,"available":198312247,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575001,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1071,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194240,"statfs":{"total":200000000,"available":198902615,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575001,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1071,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194240,"statfs":{"total":200000000,"available":198902615,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575001,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1071,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194240,"statfs":{"total":200000000,"available":198902615,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575001,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1648,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193664,"statfs":{"total":200000000,"available":198312247,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575001,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1071,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194240,"statfs":{"total":200000000,"available":198902615,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-24T23:11:28.407 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-24T23:11:28.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.505+0000 7fb9af82c640 1 Processor -- start 2024-04-24T23:11:28.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.506+0000 7fb9af82c640 1 -- start start 2024-04-24T23:11:28.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.506+0000 7fb9af82c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb9a80af890 0x7fb9a81670f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:28.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.506+0000 7fb9af82c640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9a80b6430 con 0x7fb9a806c5f0 2024-04-24T23:11:28.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.506+0000 7fb9af82c640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9a8156f80 con 0x7fb9a80b44d0 2024-04-24T23:11:28.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.506+0000 7fb9af82c640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fb9a80b6150 con 0x7fb9a80af890 2024-04-24T23:11:28.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.507+0000 7fb9ad5c8640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb9a806c5f0 0x7fb9a806c9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56434/0 (socket says 172.21.15.32:56434) 2024-04-24T23:11:28.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.507+0000 7fb9ad5c8640 1 -- 172.21.15.32:0/2282832756 learned_addr learned my addr 172.21.15.32:0/2282832756 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:11:28.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.507+0000 7fb9addc9640 1 --2- 172.21.15.32:0/2282832756 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb9a80af890 0x7fb9a81670f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:28.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.507+0000 7fb99ffff640 1 -- 172.21.15.32:0/2282832756 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4136481559 0 0) 0x7fb9a80b6430 con 0x7fb9a806c5f0 2024-04-24T23:11:28.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.507+0000 7fb99ffff640 1 -- 172.21.15.32:0/2282832756 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb994003520 con 0x7fb9a806c5f0 2024-04-24T23:11:28.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.507+0000 7fb99ffff640 1 -- 172.21.15.32:0/2282832756 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2519723574 0 0) 0x7fb994003520 con 0x7fb9a806c5f0 2024-04-24T23:11:28.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.507+0000 7fb99ffff640 1 -- 172.21.15.32:0/2282832756 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb9a80af890 msgr2=0x7fb9a81670f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:28.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.507+0000 7fb99ffff640 1 --2- 172.21.15.32:0/2282832756 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb9a80af890 0x7fb9a81670f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:28.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.507+0000 7fb99ffff640 1 -- 172.21.15.32:0/2282832756 >> v1:172.21.15.121:6789/0 conn(0x7fb9a80b44d0 legacy=0x7fb9a80b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:28.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.507+0000 7fb99ffff640 1 -- 172.21.15.32:0/2282832756 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a80b39f0 con 0x7fb9a806c5f0 2024-04-24T23:11:28.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.507+0000 7fb99ffff640 1 -- 172.21.15.32:0/2282832756 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1942340391 0 0) 0x7fb9a4001690 con 0x7fb9a806c5f0 2024-04-24T23:11:28.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.507+0000 7fb99ffff640 1 -- 172.21.15.32:0/2282832756 >> v1:172.21.15.32:6789/0 conn(0x7fb9a806c5f0 legacy=0x7fb9a806c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:28.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.507+0000 7fb99ffff640 1 --2- 172.21.15.32:0/2282832756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9940039e0 0x7fb994003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:28.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.507+0000 7fb99ffff640 1 --2- 172.21.15.32:0/2282832756 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb9a80af890 0x7fb994004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:28.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.507+0000 7fb99ffff640 1 --2- 172.21.15.32:0/2282832756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb994004d40 0x7fb9940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:28.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.507+0000 7fb99ffff640 1 -- 172.21.15.32:0/2282832756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a80b6150 con 0x7fb994004d40 2024-04-24T23:11:28.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.507+0000 7fb99ffff640 1 -- 172.21.15.32:0/2282832756 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb99400ba70 con 0x7fb9940039e0 2024-04-24T23:11:28.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.507+0000 7fb99ffff640 1 -- 172.21.15.32:0/2282832756 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fb99400bc70 con 0x7fb9a80af890 2024-04-24T23:11:28.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.507+0000 7fb9addc9640 1 --2- 172.21.15.32:0/2282832756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb994004d40 0x7fb9940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:28.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.507+0000 7fb9ad5c8640 1 --2- 172.21.15.32:0/2282832756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9940039e0 0x7fb994003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:28.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.507+0000 7fb9acdc7640 1 --2- 172.21.15.32:0/2282832756 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb9a80af890 0x7fb994004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:28.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.508+0000 7fb9addc9640 1 -- 172.21.15.32:0/2282832756 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb9a80af890 msgr2=0x7fb994004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:28.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.508+0000 7fb9addc9640 1 --2- 172.21.15.32:0/2282832756 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb9a80af890 0x7fb994004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:28.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.508+0000 7fb9addc9640 1 -- 172.21.15.32:0/2282832756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9940039e0 msgr2=0x7fb994003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:28.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.508+0000 7fb9addc9640 1 --2- 172.21.15.32:0/2282832756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9940039e0 0x7fb994003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:28.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.508+0000 7fb9addc9640 1 -- 172.21.15.32:0/2282832756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb994003520 con 0x7fb994004d40 2024-04-24T23:11:28.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.508+0000 7fb9addc9640 1 --2- 172.21.15.32:0/2282832756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb994004d40 0x7fb9940051c0 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0009fc0 tx=0x7fb994005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65bf7f75e795a913 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:28.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.508+0000 7fb99ffff640 1 -- 172.21.15.32:0/2282832756 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a0009c10 con 0x7fb994004d40 2024-04-24T23:11:28.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.508+0000 7fb99ffff640 1 -- 172.21.15.32:0/2282832756 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9a0036070 con 0x7fb994004d40 2024-04-24T23:11:28.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.508+0000 7fb99ffff640 1 -- 172.21.15.32:0/2282832756 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a0031880 con 0x7fb994004d40 2024-04-24T23:11:28.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.509+0000 7fb9af82c640 1 -- 172.21.15.32:0/2282832756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb994004d40 msgr2=0x7fb9940051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:28.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.509+0000 7fb9af82c640 1 --2- 172.21.15.32:0/2282832756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb994004d40 0x7fb9940051c0 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0009fc0 tx=0x7fb994005830 comp rx=0 tx=0).stop 2024-04-24T23:11:28.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.509+0000 7fb9addc9640 1 -- 172.21.15.32:0/2282832756 reap_dead start 2024-04-24T23:11:28.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.509+0000 7fb9af82c640 1 -- 172.21.15.32:0/2282832756 shutdown_connections 2024-04-24T23:11:28.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.509+0000 7fb9af82c640 1 -- 172.21.15.32:0/2282832756 >> 172.21.15.32:0/2282832756 conn(0x7fb9a806b110 msgr2=0x7fb9a806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:28.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.509+0000 7fb9af82c640 1 -- 172.21.15.32:0/2282832756 shutdown_connections 2024-04-24T23:11:28.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.509+0000 7fb9af82c640 1 -- 172.21.15.32:0/2282832756 wait complete. 2024-04-24T23:11:28.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.510+0000 7fb9af82c640 1 Processor -- start 2024-04-24T23:11:28.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.510+0000 7fb9af82c640 1 -- start start 2024-04-24T23:11:28.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.511+0000 7fb9af82c640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb994004d40 0x7fb9a814de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:28.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.511+0000 7fb9af82c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9a81537c0 0x7fb9a8151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:28.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.511+0000 7fb9af82c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9a8153c40 0x7fb9a81520f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:28.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.511+0000 7fb9af82c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a81650e0 con 0x7fb9a81537c0 2024-04-24T23:11:28.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.511+0000 7fb9af82c640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a80b3fb0 con 0x7fb9a8153c40 2024-04-24T23:11:28.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.511+0000 7fb9af82c640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7fb9a8165260 con 0x7fb994004d40 2024-04-24T23:11:28.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.511+0000 7fb9acdc7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9a81537c0 0x7fb9a8151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:28.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.511+0000 7fb9acdc7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9a81537c0 0x7fb9a8151bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54784/0 (socket says 172.21.15.32:54784) 2024-04-24T23:11:28.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.511+0000 7fb9acdc7640 1 -- 172.21.15.32:0/1247953740 learned_addr learned my addr 172.21.15.32:0/1247953740 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:28.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.511+0000 7fb9ad5c8640 1 --2- 172.21.15.32:0/1247953740 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb994004d40 0x7fb9a814de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:28.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.511+0000 7fb9addc9640 1 --2- 172.21.15.32:0/1247953740 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9a8153c40 0x7fb9a81520f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:28.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.512+0000 7fb9acdc7640 1 -- 172.21.15.32:0/1247953740 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb994004d40 msgr2=0x7fb9a814de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:28.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.512+0000 7fb9acdc7640 1 --2- 172.21.15.32:0/1247953740 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb994004d40 0x7fb9a814de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:28.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.512+0000 7fb9acdc7640 1 -- 172.21.15.32:0/1247953740 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9a8153c40 msgr2=0x7fb9a81520f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:28.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.512+0000 7fb9acdc7640 1 --2- 172.21.15.32:0/1247953740 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9a8153c40 0x7fb9a81520f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:28.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.512+0000 7fb9acdc7640 1 -- 172.21.15.32:0/1247953740 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a814e3a0 con 0x7fb9a81537c0 2024-04-24T23:11:28.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.512+0000 7fb9acdc7640 1 --2- 172.21.15.32:0/1247953740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9a81537c0 0x7fb9a8151bb0 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7fb99800f7f0 tx=0x7fb99800fcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:28.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.512+0000 7fb99dffb640 1 -- 172.21.15.32:0/1247953740 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99800fea0 con 0x7fb9a81537c0 2024-04-24T23:11:28.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.512+0000 7fb99dffb640 1 -- 172.21.15.32:0/1247953740 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb998018070 con 0x7fb9a81537c0 2024-04-24T23:11:28.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.512+0000 7fb9af82c640 1 -- 172.21.15.32:0/1247953740 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9a8152720 con 0x7fb9a81537c0 2024-04-24T23:11:28.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.512+0000 7fb99dffb640 1 -- 172.21.15.32:0/1247953740 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99800d730 con 0x7fb9a81537c0 2024-04-24T23:11:28.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.512+0000 7fb9af82c640 1 -- 172.21.15.32:0/1247953740 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9a8152ce0 con 0x7fb9a81537c0 2024-04-24T23:11:28.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.513+0000 7fb9af82c640 1 -- 172.21.15.32:0/1247953740 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9a8068430 con 0x7fb9a81537c0 2024-04-24T23:11:28.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.514+0000 7fb99dffb640 1 -- 172.21.15.32:0/1247953740 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb99800d8d0 con 0x7fb9a81537c0 2024-04-24T23:11:28.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.514+0000 7fb99dffb640 1 --2- 172.21.15.32:0/1247953740 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb98c06be60 0x7fb98c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:28.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.514+0000 7fb99dffb640 1 -- 172.21.15.32:0/1247953740 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 4943+0+0 (secure 0 0 0) 0x7fb99808a8a0 con 0x7fb9a81537c0 2024-04-24T23:11:28.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.514+0000 7fb9ad5c8640 1 --2- 172.21.15.32:0/1247953740 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb98c06be60 0x7fb98c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:28.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.515+0000 7fb9ad5c8640 1 --2- 172.21.15.32:0/1247953740 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb98c06be60 0x7fb98c06e320 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fb9a40010e0 tx=0x7fb9a40078f0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:28.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.518+0000 7fb99dffb640 1 -- 172.21.15.32:0/1247953740 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb9a814e3a0 con 0x7fb9a81537c0 2024-04-24T23:11:28.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.703+0000 7fb9af82c640 1 -- 172.21.15.32:0/1247953740 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7fb9a80b5670 con 0x7fb9a81537c0 2024-04-24T23:11:28.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.706+0000 7fb99dffb640 1 -- 172.21.15.32:0/1247953740 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v22)=0 v22) ==== 123+0+0 (secure 0 0 0) 0x7fb998058f90 con 0x7fb9a81537c0 2024-04-24T23:11:28.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.706+0000 7fb99dffb640 1 -- 172.21.15.32:0/1247953740 <== mon.0 v2:172.21.15.32:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fb99805d290 con 0x7fb9a81537c0 2024-04-24T23:11:28.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.708+0000 7fb9af82c640 1 -- 172.21.15.32:0/1247953740 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb98c06be60 msgr2=0x7fb98c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:28.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.708+0000 7fb9af82c640 1 --2- 172.21.15.32:0/1247953740 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb98c06be60 0x7fb98c06e320 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fb9a40010e0 tx=0x7fb9a40078f0 comp rx=0 tx=0).stop 2024-04-24T23:11:28.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.708+0000 7fb9af82c640 1 -- 172.21.15.32:0/1247953740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9a81537c0 msgr2=0x7fb9a8151bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:28.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.708+0000 7fb9af82c640 1 --2- 172.21.15.32:0/1247953740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9a81537c0 0x7fb9a8151bb0 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7fb99800f7f0 tx=0x7fb99800fcb0 comp rx=0 tx=0).stop 2024-04-24T23:11:28.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.708+0000 7fb9af82c640 1 -- 172.21.15.32:0/1247953740 shutdown_connections 2024-04-24T23:11:28.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.708+0000 7fb9af82c640 1 --2- 172.21.15.32:0/1247953740 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7fb98c06be60 0x7fb98c06e320 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:28.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.708+0000 7fb9af82c640 1 --2- 172.21.15.32:0/1247953740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9a81537c0 0x7fb9a8151bb0 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:28.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.708+0000 7fb9af82c640 1 --2- 172.21.15.32:0/1247953740 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9a8153c40 0x7fb9a81520f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:28.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.708+0000 7fb9af82c640 1 --2- 172.21.15.32:0/1247953740 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb994004d40 0x7fb9a814de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:28.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.709+0000 7fb9af82c640 1 -- 172.21.15.32:0/1247953740 >> 172.21.15.32:0/1247953740 conn(0x7fb9a806b110 msgr2=0x7fb9a80cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:28.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.709+0000 7fb9af82c640 1 -- 172.21.15.32:0/1247953740 shutdown_connections 2024-04-24T23:11:28.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.709+0000 7fb9af82c640 1 -- 172.21.15.32:0/1247953740 wait complete. 2024-04-24T23:11:28.722 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-24T23:11:28.722 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:11:28.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.820+0000 7f605a49e640 1 Processor -- start 2024-04-24T23:11:28.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.820+0000 7f605a49e640 1 -- start start 2024-04-24T23:11:28.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.821+0000 7f605a49e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6054074210 0x7f605406c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:28.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.821+0000 7f605a49e640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f60540ae140 0x7f60540b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:28.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.821+0000 7f605a49e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60540b0770 con 0x7f60540b0b80 2024-04-24T23:11:28.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.821+0000 7f605a49e640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f60540b1c20 con 0x7f60540ae140 2024-04-24T23:11:28.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.821+0000 7f605a49e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f60540b1010 con 0x7f6054074210 2024-04-24T23:11:28.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.821+0000 7f60537fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f60540b0b80 0x7f60540745f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56440/0 (socket says 172.21.15.32:56440) 2024-04-24T23:11:28.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.821+0000 7f60537fe640 1 -- 172.21.15.32:0/533512569 learned_addr learned my addr 172.21.15.32:0/533512569 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-24T23:11:28.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.821+0000 7f6053fff640 1 --2- 172.21.15.32:0/533512569 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6054074210 0x7f605406c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:28.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.821+0000 7f6058a3b640 1 --2- 172.21.15.32:0/533512569 >> v2:172.21.15.121:3300/0 conn(0x7f60540ae140 0x7f60540b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:28.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.822+0000 7f6052ffd640 1 -- 172.21.15.32:0/533512569 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2238555933 0 0) 0x7f60540b0770 con 0x7f60540b0b80 2024-04-24T23:11:28.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.822+0000 7f6052ffd640 1 -- 172.21.15.32:0/533512569 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6040003520 con 0x7f60540b0b80 2024-04-24T23:11:28.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.822+0000 7f6053fff640 1 -- 172.21.15.32:0/533512569 >> v2:172.21.15.121:3300/0 conn(0x7f60540ae140 msgr2=0x7f60540b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:11:28.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.822+0000 7f6053fff640 1 --2- 172.21.15.32:0/533512569 >> v2:172.21.15.121:3300/0 conn(0x7f60540ae140 0x7f60540b20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:28.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.822+0000 7f6053fff640 1 -- 172.21.15.32:0/533512569 >> v1:172.21.15.32:6789/0 conn(0x7f60540b0b80 legacy=0x7f60540745f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:28.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.822+0000 7f6053fff640 1 -- 172.21.15.32:0/533512569 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60540b12f0 con 0x7f6054074210 2024-04-24T23:11:28.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.822+0000 7f6058a3b640 1 --2- 172.21.15.32:0/533512569 >> v2:172.21.15.121:3300/0 conn(0x7f60540ae140 0x7f60540b20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:11:28.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.823+0000 7f6053fff640 1 --2- 172.21.15.32:0/533512569 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6054074210 0x7f605406c480 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f60440097a0 tx=0x7f604402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=220695e4b49be7c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:28.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.823+0000 7f6052ffd640 1 -- 172.21.15.32:0/533512569 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6044037070 con 0x7f6054074210 2024-04-24T23:11:28.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.823+0000 7f6052ffd640 1 -- 172.21.15.32:0/533512569 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f6044032440 con 0x7f6054074210 2024-04-24T23:11:28.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.823+0000 7f6052ffd640 1 -- 172.21.15.32:0/533512569 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6044032830 con 0x7f6054074210 2024-04-24T23:11:28.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.823+0000 7f605a49e640 1 -- 172.21.15.32:0/533512569 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6054074210 msgr2=0x7f605406c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:28.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.823+0000 7f605a49e640 1 --2- 172.21.15.32:0/533512569 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6054074210 0x7f605406c480 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f60440097a0 tx=0x7f604402a180 comp rx=0 tx=0).stop 2024-04-24T23:11:28.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.823+0000 7f605a49e640 1 -- 172.21.15.32:0/533512569 shutdown_connections 2024-04-24T23:11:28.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.823+0000 7f605a49e640 1 --2- 172.21.15.32:0/533512569 >> v2:172.21.15.121:3300/0 conn(0x7f60540ae140 0x7f60540b20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:28.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.823+0000 7f605a49e640 1 --2- 172.21.15.32:0/533512569 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6054074210 0x7f605406c480 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:28.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.823+0000 7f605a49e640 1 -- 172.21.15.32:0/533512569 >> 172.21.15.32:0/533512569 conn(0x7f60540b2e60 msgr2=0x7f60540b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:28.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.823+0000 7f605a49e640 1 -- 172.21.15.32:0/533512569 shutdown_connections 2024-04-24T23:11:28.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.823+0000 7f605a49e640 1 -- 172.21.15.32:0/533512569 wait complete. 2024-04-24T23:11:28.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.824+0000 7f605a49e640 1 Processor -- start 2024-04-24T23:11:28.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.825+0000 7f605a49e640 1 -- start start 2024-04-24T23:11:28.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.825+0000 7f605a49e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6054074210 0x7f60541585a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:28.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.825+0000 7f605a49e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f60540ae140 0x7f60540cd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:28.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.825+0000 7f605a49e640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f60540cdf30 0x7f605415fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:28.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.825+0000 7f605a49e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f60540b3ba0 con 0x7f6054074210 2024-04-24T23:11:28.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.825+0000 7f605a49e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f60540b18b0 con 0x7f60540ae140 2024-04-24T23:11:28.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.825+0000 7f605a49e640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 -- 0x7f60540b3d20 con 0x7f60540cdf30 2024-04-24T23:11:28.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.825+0000 7f6053fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6054074210 0x7f60541585a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:28.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.826+0000 7f6053fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6054074210 0x7f60541585a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54798/0 (socket says 172.21.15.32:54798) 2024-04-24T23:11:28.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.826+0000 7f6053fff640 1 -- 172.21.15.32:0/82230746 learned_addr learned my addr 172.21.15.32:0/82230746 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-24T23:11:28.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.826+0000 7f60537fe640 1 --2- 172.21.15.32:0/82230746 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f60540ae140 0x7f60540cd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:28.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.826+0000 7f6058a3b640 1 --2- 172.21.15.32:0/82230746 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f60540cdf30 0x7f605415fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:28.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.826+0000 7f6053fff640 1 -- 172.21.15.32:0/82230746 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f60540cdf30 msgr2=0x7f605415fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:28.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.826+0000 7f6053fff640 1 --2- 172.21.15.32:0/82230746 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f60540cdf30 0x7f605415fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:28.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.826+0000 7f6053fff640 1 -- 172.21.15.32:0/82230746 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f60540ae140 msgr2=0x7f60540cd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:28.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.826+0000 7f6053fff640 1 --2- 172.21.15.32:0/82230746 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f60540ae140 0x7f60540cd9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:28.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.826+0000 7f6053fff640 1 -- 172.21.15.32:0/82230746 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60540c8560 con 0x7f6054074210 2024-04-24T23:11:28.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.826+0000 7f6053fff640 1 --2- 172.21.15.32:0/82230746 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6054074210 0x7f60541585a0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f6044000c00 tx=0x7f60440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:28.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.826+0000 7f6050ff9640 1 -- 172.21.15.32:0/82230746 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6044041030 con 0x7f6054074210 2024-04-24T23:11:28.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.826+0000 7f6050ff9640 1 -- 172.21.15.32:0/82230746 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f60440304c0 con 0x7f6054074210 2024-04-24T23:11:28.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.826+0000 7f605a49e640 1 -- 172.21.15.32:0/82230746 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60540c8840 con 0x7f6054074210 2024-04-24T23:11:28.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.826+0000 7f6050ff9640 1 -- 172.21.15.32:0/82230746 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6044037040 con 0x7f6054074210 2024-04-24T23:11:28.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.826+0000 7f605a49e640 1 -- 172.21.15.32:0/82230746 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60540c8e00 con 0x7f6054074210 2024-04-24T23:11:28.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.827+0000 7f605a49e640 1 -- 172.21.15.32:0/82230746 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60540b56b0 con 0x7f6054074210 2024-04-24T23:11:28.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.828+0000 7f6050ff9640 1 -- 172.21.15.32:0/82230746 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f60440308b0 con 0x7f6054074210 2024-04-24T23:11:28.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.828+0000 7f6050ff9640 1 --2- 172.21.15.32:0/82230746 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f603406be60 0x7f603406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:28.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.828+0000 7f6050ff9640 1 -- 172.21.15.32:0/82230746 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 4943+0+0 (secure 0 0 0) 0x7f60440aaf60 con 0x7f6054074210 2024-04-24T23:11:28.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.828+0000 7f60537fe640 1 --2- 172.21.15.32:0/82230746 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f603406be60 0x7f603406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:28.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.829+0000 7f60537fe640 1 --2- 172.21.15.32:0/82230746 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f603406be60 0x7f603406e320 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f60540d0fe0 tx=0x7f603c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:28.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:28.833+0000 7f6050ff9640 1 -- 172.21.15.32:0/82230746 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f60540b56b0 con 0x7f6054074210 2024-04-24T23:11:29.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:29.052+0000 7f605a49e640 1 -- 172.21.15.32:0/82230746 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6054068430 con 0x7f6054074210 2024-04-24T23:11:29.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:29.053+0000 7f6050ff9640 1 -- 172.21.15.32:0/82230746 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+3869 (secure 0 0 0) 0x7f604404b020 con 0x7f6054074210 2024-04-24T23:11:29.055 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-24T23:11:29.055 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":82,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6941,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/416518156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":416518156},{"type":"v1","addr":"172.21.15.121:6837","nonce":416518156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7762,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1506667233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1506667233},{"type":"v1","addr":"172.21.15.121:6835","nonce":1506667233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":9063,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/2215079101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":2215079101},{"type":"v1","addr":"172.21.15.32:6834","nonce":2215079101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":9459,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/962598736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":962598736},{"type":"v1","addr":"172.21.15.32:6835","nonce":962598736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":82}],"filesystems":[]} 2024-04-24T23:11:29.056 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 82 2024-04-24T23:11:29.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:29.054+0000 7f605a49e640 1 -- 172.21.15.32:0/82230746 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f603406be60 msgr2=0x7f603406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:29.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:29.054+0000 7f605a49e640 1 --2- 172.21.15.32:0/82230746 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f603406be60 0x7f603406e320 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f60540d0fe0 tx=0x7f603c009040 comp rx=0 tx=0).stop 2024-04-24T23:11:29.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:29.054+0000 7f605a49e640 1 -- 172.21.15.32:0/82230746 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6054074210 msgr2=0x7f60541585a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:29.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:29.054+0000 7f605a49e640 1 --2- 172.21.15.32:0/82230746 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6054074210 0x7f60541585a0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f6044000c00 tx=0x7f60440046c0 comp rx=0 tx=0).stop 2024-04-24T23:11:29.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:29.054+0000 7f605a49e640 1 -- 172.21.15.32:0/82230746 shutdown_connections 2024-04-24T23:11:29.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:29.054+0000 7f605a49e640 1 --2- 172.21.15.32:0/82230746 >> [v2:172.21.15.121:6832/1848157943,v1:172.21.15.121:6833/1848157943] conn(0x7f603406be60 0x7f603406e320 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:29.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:29.054+0000 7f605a49e640 1 --2- 172.21.15.32:0/82230746 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f60540cdf30 0x7f605415fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:29.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:29.054+0000 7f605a49e640 1 --2- 172.21.15.32:0/82230746 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f60540ae140 0x7f60540cd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:29.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:29.054+0000 7f605a49e640 1 --2- 172.21.15.32:0/82230746 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6054074210 0x7f60541585a0 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:29.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:29.054+0000 7f605a49e640 1 -- 172.21.15.32:0/82230746 >> 172.21.15.32:0/82230746 conn(0x7f60540b2e60 msgr2=0x7f605406e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:29.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:29.055+0000 7f605a49e640 1 -- 172.21.15.32:0/82230746 shutdown_connections 2024-04-24T23:11:29.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-24T23:11:29.055+0000 7f605a49e640 1 -- 172.21.15.32:0/82230746 wait complete. 2024-04-24T23:11:29.068 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-24T23:11:29.068 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-24T23:11:29.068 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-24T23:11:29.068 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:11:29.077 INFO:tasks.ceph.mds.a:Stopped 2024-04-24T23:11:29.077 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-24T23:11:29.077 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:11:29.126 INFO:tasks.ceph.mds.c:Stopped 2024-04-24T23:11:29.126 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-24T23:11:29.127 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:11:29.138 INFO:tasks.ceph.mds.b:Stopped 2024-04-24T23:11:29.139 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-24T23:11:29.139 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:11:29.191 INFO:tasks.ceph.mds.d:Stopped 2024-04-24T23:11:29.191 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-24T23:11:29.191 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-24T23:11:29.191 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:11:29.208 INFO:tasks.ceph.osd.0:Stopped 2024-04-24T23:11:29.208 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-24T23:11:29.208 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:11:29.265 INFO:tasks.ceph.osd.1:Stopped 2024-04-24T23:11:29.265 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-24T23:11:29.265 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:11:29.323 INFO:tasks.ceph.osd.2:Stopped 2024-04-24T23:11:29.323 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-24T23:11:29.323 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:11:29.380 INFO:tasks.ceph.osd.3:Stopped 2024-04-24T23:11:29.381 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-24T23:11:29.381 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:11:29.400 INFO:tasks.ceph.osd.4:Stopped 2024-04-24T23:11:29.400 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-24T23:11:29.400 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:11:29.462 INFO:tasks.ceph.osd.5:Stopped 2024-04-24T23:11:29.462 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-24T23:11:29.463 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:11:29.526 INFO:tasks.ceph.osd.6:Stopped 2024-04-24T23:11:29.526 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-24T23:11:29.527 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:11:29.588 INFO:tasks.ceph.osd.7:Stopped 2024-04-24T23:11:29.588 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-24T23:11:29.588 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-24T23:11:29.588 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:11:29.613 INFO:tasks.ceph.mgr.y:Stopped 2024-04-24T23:11:29.613 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-24T23:11:29.613 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:11:29.656 INFO:tasks.ceph.mgr.x:Stopped 2024-04-24T23:11:29.657 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-24T23:11:29.657 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-24T23:11:29.657 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:11:29.668 INFO:tasks.ceph.mon.a:Stopped 2024-04-24T23:11:29.668 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-24T23:11:29.668 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:11:29.710 INFO:tasks.ceph.mon.b:Stopped 2024-04-24T23:11:29.710 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-24T23:11:29.710 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:11:29.763 INFO:tasks.ceph.mon.c:Stopped 2024-04-24T23:11:29.763 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-24T23:11:29.763 DEBUG:teuthology.orchestra.run.smithi032:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | egrep -v PG_RECOVERY_FULL | egrep -v PG_DEGRADED | head -n 1 2024-04-24T23:11:29.801 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi032.front.sepia.ceph.com 2024-04-24T23:11:29.802 DEBUG:teuthology.orchestra.run.smithi032:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-24T23:11:30.109 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi032.front.sepia.ceph.com 2024-04-24T23:11:30.109 DEBUG:teuthology.orchestra.run.smithi032:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-24T23:11:30.191 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi032.front.sepia.ceph.com 2024-04-24T23:11:30.191 DEBUG:teuthology.orchestra.run.smithi032:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-24T23:11:30.249 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi032.front.sepia.ceph.com 2024-04-24T23:11:30.249 DEBUG:teuthology.orchestra.run.smithi032:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-24T23:11:30.307 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi121.front.sepia.ceph.com 2024-04-24T23:11:30.308 DEBUG:teuthology.orchestra.run.smithi121:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-24T23:11:30.683 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi121.front.sepia.ceph.com 2024-04-24T23:11:30.683 DEBUG:teuthology.orchestra.run.smithi121:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-24T23:11:30.731 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi121.front.sepia.ceph.com 2024-04-24T23:11:30.731 DEBUG:teuthology.orchestra.run.smithi121:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-24T23:11:30.780 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi121.front.sepia.ceph.com 2024-04-24T23:11:30.780 DEBUG:teuthology.orchestra.run.smithi121:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-24T23:11:30.821 INFO:tasks.ceph:Archiving mon data... 2024-04-24T23:11:30.822 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672173/data/mon.a.tgz 2024-04-24T23:11:30.822 DEBUG:teuthology.orchestra.run.smithi032:> mktemp 2024-04-24T23:11:30.839 INFO:teuthology.orchestra.run.smithi032.stdout:/tmp/tmp.HH8x6URaXa 2024-04-24T23:11:30.839 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.HH8x6URaXa 2024-04-24T23:11:31.010 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0666 /tmp/tmp.HH8x6URaXa 2024-04-24T23:11:31.096 DEBUG:teuthology.orchestra.remote:smithi032:/tmp/tmp.HH8x6URaXa is 481KB 2024-04-24T23:11:31.165 DEBUG:teuthology.orchestra.run.smithi032:> rm -fr /tmp/tmp.HH8x6URaXa 2024-04-24T23:11:31.180 DEBUG:teuthology.misc:Transferring archived files from smithi121:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672173/data/mon.b.tgz 2024-04-24T23:11:31.180 DEBUG:teuthology.orchestra.run.smithi121:> mktemp 2024-04-24T23:11:31.197 INFO:teuthology.orchestra.run.smithi121.stdout:/tmp/tmp.PRpIKGCEIO 2024-04-24T23:11:31.197 DEBUG:teuthology.orchestra.run.smithi121:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.PRpIKGCEIO 2024-04-24T23:11:31.393 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 0666 /tmp/tmp.PRpIKGCEIO 2024-04-24T23:11:31.474 DEBUG:teuthology.orchestra.remote:smithi121:/tmp/tmp.PRpIKGCEIO is 501KB 2024-04-24T23:11:31.570 DEBUG:teuthology.orchestra.run.smithi121:> rm -fr /tmp/tmp.PRpIKGCEIO 2024-04-24T23:11:31.584 DEBUG:teuthology.misc:Transferring archived files from smithi121:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672173/data/mon.c.tgz 2024-04-24T23:11:31.584 DEBUG:teuthology.orchestra.run.smithi121:> mktemp 2024-04-24T23:11:31.638 INFO:teuthology.orchestra.run.smithi121.stdout:/tmp/tmp.WBTcbRpwr7 2024-04-24T23:11:31.638 DEBUG:teuthology.orchestra.run.smithi121:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.WBTcbRpwr7 2024-04-24T23:11:31.797 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 0666 /tmp/tmp.WBTcbRpwr7 2024-04-24T23:11:31.878 DEBUG:teuthology.orchestra.remote:smithi121:/tmp/tmp.WBTcbRpwr7 is 501KB 2024-04-24T23:11:31.975 DEBUG:teuthology.orchestra.run.smithi121:> rm -fr /tmp/tmp.WBTcbRpwr7 2024-04-24T23:11:31.989 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-24T23:11:31.989 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-24T23:11:31.992 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-24T23:11:32.078 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-24T23:11:32.105 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-24T23:11:32.106 INFO:tasks.ceph:Archiving crash dumps... 2024-04-24T23:11:32.107 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672173/remote/smithi032/crash 2024-04-24T23:11:32.108 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-24T23:11:32.148 DEBUG:teuthology.misc:Transferring archived files from smithi121:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672173/remote/smithi121/crash 2024-04-24T23:11:32.149 DEBUG:teuthology.orchestra.run.smithi121:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-24T23:11:32.171 INFO:tasks.ceph:Shutting down logrotate 2024-04-24T23:11:32.171 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:11:32.193 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:11:32.269 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-24T23:11:32.297 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-24T23:11:32.320 INFO:tasks.ceph:Compressing logs... 2024-04-24T23:11:32.321 DEBUG:teuthology.orchestra.run.smithi032:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-24T23:11:32.340 DEBUG:teuthology.orchestra.run.smithi121:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-24T23:11:32.365 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43208.log 2024-04-24T23:11:32.365 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50076.log 2024-04-24T23:11:32.365 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35125.log 2024-04-24T23:11:32.365 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45905.log 88.7% 2024-04-24T23:11:32.366 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43208.log.gz 2024-04-24T23:11:32.366 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38179.log 2024-04-24T23:11:32.417 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.35320.log 2024-04-24T23:11:32.417 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-24T23:11:32.417 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.1.35320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30206.log 2024-04-24T23:11:32.417 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.35320.log.gz 2024-04-24T23:11:32.417 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-24T23:11:32.417 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.30206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.35819.log 2024-04-24T23:11:32.417 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph.audit.log: gzip -5 87.7% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.30206.log.gz /var/log/ceph/ceph-client.admin.29955.log 2024-04-24T23:11:32.417 INFO:teuthology.orchestra.run.smithi121.stderr: 2024-04-24T23:11:32.417 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.1.35819.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.35819.log.gz 2024-04-24T23:11:32.417 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.36906.log 2024-04-24T23:11:32.417 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.29955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30246.log 2024-04-24T23:11:32.417 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.1.36906.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.29955.log.gz 2024-04-24T23:11:32.417 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-24T23:11:32.418 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.36906.log.gz 2024-04-24T23:11:32.418 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.30246.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-24T23:11:32.418 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-osd.7.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.30246.log.gz 2024-04-24T23:11:32.418 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-24T23:11:32.418 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.31186.log 2024-04-24T23:11:32.418 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-24T23:11:32.418 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.1.31186.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.1.31186.log.gz 2024-04-24T23:11:32.418 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/ceph-client.admin.29995.log 2024-04-24T23:11:32.418 INFO:teuthology.orchestra.run.smithi121.stderr: 92.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-24T23:11:32.418 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-24T23:11:32.418 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.29995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34823.log 2024-04-24T23:11:32.418 INFO:teuthology.orchestra.run.smithi121.stderr: 87.3%/var/log/ceph/ceph-mgr.x.log: -- replaced with /var/log/ceph/ceph-client.admin.29995.log.gz 2024-04-24T23:11:32.418 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-24T23:11:32.418 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33273.log 2024-04-24T23:11:32.418 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.1.34823.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34823.log.gz 2024-04-24T23:11:32.419 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-mon.c.log: 93.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-24T23:11:32.419 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.36388.log 2024-04-24T23:11:32.419 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.1.33273.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.1.33273.log.gz 2024-04-24T23:11:32.419 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-24T23:11:32.419 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.1.36388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34342.log 2024-04-24T23:11:32.419 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.36388.log.gz 2024-04-24T23:11:32.419 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33845.log 2024-04-24T23:11:32.419 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.1.34342.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34342.log.gz 2024-04-24T23:11:32.419 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.32200.log 2024-04-24T23:11:32.419 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.1.33845.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.33845.log.gz 2024-04-24T23:11:32.419 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.32754.log 2024-04-24T23:11:32.419 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.1.32200.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.32200.log.gz 2024-04-24T23:11:32.419 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30035.log 2024-04-24T23:11:32.419 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.1.32754.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.32754.log.gz 2024-04-24T23:11:32.419 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30079.log 2024-04-24T23:11:32.420 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.30035.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.30035.log.gz 2024-04-24T23:11:32.420 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30126.log 2024-04-24T23:11:32.420 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.30079.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.30079.log.gz 2024-04-24T23:11:32.420 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.31584.log 2024-04-24T23:11:32.420 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.30126.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.30126.log.gz 2024-04-24T23:11:32.420 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30166.log 2024-04-24T23:11:32.420 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.3.31584.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.31584.log.gz 2024-04-24T23:11:32.420 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-24T23:11:32.420 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.30166.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.30166.log.gz 2024-04-24T23:11:32.420 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-24T23:11:32.435 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-mon.b.log: 92.5% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-24T23:11:32.529 INFO:teuthology.orchestra.run.smithi121.stderr: 93.5% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-24T23:11:32.561 INFO:teuthology.orchestra.run.smithi121.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-24T23:11:32.751 INFO:teuthology.orchestra.run.smithi121.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-24T23:11:32.782 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65032.log 2024-04-24T23:11:32.782 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40644.log 2024-04-24T23:11:32.782 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.35125.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50076.log.gz 2024-04-24T23:11:32.782 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45905.log: /var/log/ceph/ceph-client.admin.38179.log: gzip/var/log/ceph/ceph-client.admin.65032.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.51023.log 2024-04-24T23:11:32.782 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45905.log.gz 2024-04-24T23:11:32.782 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% 89.4% -- replaced with /var/log/ceph/ceph-client.admin.38179.log.gz -- replaced with /var/log/ceph/ceph-client.admin.35125.log.gz 2024-04-24T23:11:32.782 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-04-24T23:11:32.782 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.40644.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.65032.log.gz 2024-04-24T23:11:32.783 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33262.log 2024-04-24T23:11:32.783 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40644.log.gz 2024-04-24T23:11:32.783 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62694.log 2024-04-24T23:11:32.783 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.33262.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51023.log.gz 2024-04-24T23:11:32.783 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41442.log 2024-04-24T23:11:32.783 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33262.log.gz 2024-04-24T23:11:32.783 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.62694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54476.log 2024-04-24T23:11:32.783 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41442.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.62694.log.gz 2024-04-24T23:11:32.783 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70675.log 2024-04-24T23:11:32.783 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.54476.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41442.log.gz 2024-04-24T23:11:32.783 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44077.log 2024-04-24T23:11:32.783 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54476.log.gz 2024-04-24T23:11:32.784 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.70675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31056.log 2024-04-24T23:11:32.784 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.70675.log.gz 2024-04-24T23:11:32.784 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36655.log 2024-04-24T23:11:32.784 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44077.log.gz 2024-04-24T23:11:32.784 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.31056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52408.log 2024-04-24T23:11:32.784 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.31056.log.gz 2024-04-24T23:11:32.784 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.36655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33431.log 2024-04-24T23:11:32.784 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36655.log.gz 2024-04-24T23:11:32.784 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33136.log 2024-04-24T23:11:32.784 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52408.log.gz 2024-04-24T23:11:32.784 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.33431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64206.log 2024-04-24T23:11:32.785 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.33431.log.gz 2024-04-24T23:11:32.785 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.33136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37795.log 2024-04-24T23:11:32.785 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.64206.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33136.log.gz 2024-04-24T23:11:32.785 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34619.log 2024-04-24T23:11:32.785 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.64206.log.gz 2024-04-24T23:11:32.785 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64164.log 2024-04-24T23:11:32.785 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37795.log.gz 2024-04-24T23:11:32.785 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.34619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45030.log 2024-04-24T23:11:32.785 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34619.log.gz 2024-04-24T23:11:32.785 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.64164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40706.log 2024-04-24T23:11:32.785 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.64164.log.gz 2024-04-24T23:11:32.786 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50416.log 2024-04-24T23:11:32.786 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.40706.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45030.log.gz 2024-04-24T23:11:32.786 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69291.log 2024-04-24T23:11:32.786 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40706.log.gz 2024-04-24T23:11:32.786 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52612.log 2024-04-24T23:11:32.786 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50416.log.gz 2024-04-24T23:11:32.786 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.69291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53571.log 2024-04-24T23:11:32.786 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.69291.log.gz 2024-04-24T23:11:32.786 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69039.log 2024-04-24T23:11:32.786 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52612.log.gz 2024-04-24T23:11:32.786 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.53571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55763.log 2024-04-24T23:11:32.787 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53571.log.gz 2024-04-24T23:11:32.787 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.69039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52366.log 2024-04-24T23:11:32.787 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.55763.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.69039.log.gz 2024-04-24T23:11:32.787 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42832.log 2024-04-24T23:11:32.787 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55763.log.gz 2024-04-24T23:11:32.787 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63815.log 2024-04-24T23:11:32.787 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52366.log.gz 2024-04-24T23:11:32.787 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53319.log 2024-04-24T23:11:32.787 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.63815.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42832.log.gz 2024-04-24T23:11:32.787 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41089.log 2024-04-24T23:11:32.787 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.63815.log.gz 2024-04-24T23:11:32.787 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.53319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31540.log 2024-04-24T23:11:32.788 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41089.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53319.log.gz 2024-04-24T23:11:32.788 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40182.log 2024-04-24T23:11:32.788 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41089.log.gz 2024-04-24T23:11:32.788 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.31540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44689.log 2024-04-24T23:11:32.788 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.31540.log.gz 2024-04-24T23:11:32.788 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.40182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44605.log 2024-04-24T23:11:32.788 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40182.log.gz 2024-04-24T23:11:32.788 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32342.log 2024-04-24T23:11:32.788 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44689.log.gz 2024-04-24T23:11:32.788 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43412.log 2024-04-24T23:11:32.788 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44605.log.gz 2024-04-24T23:11:32.789 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.32342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48935.log 2024-04-24T23:11:32.789 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6%/var/log/ceph/ceph-client.admin.43412.log: -- replaced with /var/log/ceph/ceph-client.admin.32342.log.gz 2024-04-24T23:11:32.789 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44818.log 2024-04-24T23:11:32.789 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43412.log.gz 2024-04-24T23:11:32.789 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38533.log 2024-04-24T23:11:32.789 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44818.log: gzip 88.2% -5 -- replaced with /var/log/ceph/ceph-client.admin.48935.log.gz --verbose 2024-04-24T23:11:32.789 INFO:teuthology.orchestra.run.smithi032.stderr: -- /var/log/ceph/ceph-client.admin.32687.log 2024-04-24T23:11:32.789 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.38533.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44818.log.gz 2024-04-24T23:11:32.789 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42213.log 2024-04-24T23:11:32.789 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.32687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46319.log 2024-04-24T23:11:32.789 INFO:teuthology.orchestra.run.smithi032.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.38533.log.gz 2024-04-24T23:11:32.789 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.32687.log.gz 2024-04-24T23:11:32.789 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62342.log 2024-04-24T23:11:32.790 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42213.log.gz 2024-04-24T23:11:32.790 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38347.log 2024-04-24T23:11:32.790 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.62342.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46319.log.gz 2024-04-24T23:11:32.790 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32351.log 2024-04-24T23:11:32.790 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62342.log.gz 2024-04-24T23:11:32.790 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.38347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53090.log 2024-04-24T23:11:32.790 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.32351.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38347.log.gz 2024-04-24T23:11:32.790 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35378.log 2024-04-24T23:11:32.790 INFO:teuthology.orchestra.run.smithi032.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.32351.log.gz 2024-04-24T23:11:32.791 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.53090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46151.log 2024-04-24T23:11:32.791 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53090.log.gz 2024-04-24T23:11:32.791 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.35378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47801.log 2024-04-24T23:11:32.791 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46151.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35378.log.gz 2024-04-24T23:11:32.791 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60728.log 2024-04-24T23:11:32.791 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46151.log.gz 2024-04-24T23:11:32.791 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67160.log 2024-04-24T23:11:32.791 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47801.log.gz 2024-04-24T23:11:32.791 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.60728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40098.log 2024-04-24T23:11:32.791 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60728.log.gz 2024-04-24T23:11:32.792 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.67160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38449.log 2024-04-24T23:11:32.792 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.40098.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.67160.log.gz 2024-04-24T23:11:32.792 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51427.log 2024-04-24T23:11:32.792 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40098.log.gz 2024-04-24T23:11:32.792 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.38449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55517.log 2024-04-24T23:11:32.792 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38449.log.gz 2024-04-24T23:11:32.792 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53445.log 2024-04-24T23:11:32.792 INFO:teuthology.orchestra.run.smithi032.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.51427.log.gz 2024-04-24T23:11:32.792 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.55517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40518.log 2024-04-24T23:11:32.792 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.55517.log.gz 2024-04-24T23:11:32.793 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.53445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65320.log 2024-04-24T23:11:32.793 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53445.log.gz 2024-04-24T23:11:32.793 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.40518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32548.log 2024-04-24T23:11:32.793 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.65320.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40518.log.gz 2024-04-24T23:11:32.793 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55136.log 2024-04-24T23:11:32.793 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.65320.log.gz 2024-04-24T23:11:32.793 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.32548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33220.log 2024-04-24T23:11:32.793 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.32548.log.gz 2024-04-24T23:11:32.793 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.55136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38221.log 2024-04-24T23:11:32.793 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55136.log.gz 2024-04-24T23:11:32.794 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.33220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43848.log 2024-04-24T23:11:32.794 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33220.log.gz 2024-04-24T23:11:32.794 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.38221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47759.log 2024-04-24T23:11:32.794 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38221.log.gz 2024-04-24T23:11:32.794 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32352.log 2024-04-24T23:11:32.794 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43848.log.gz 2024-04-24T23:11:32.794 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45657.log 2024-04-24T23:11:32.794 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.47759.log.gz 2024-04-24T23:11:32.795 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.32352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49097.log 2024-04-24T23:11:32.795 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.32352.log.gz 2024-04-24T23:11:32.795 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70425.log 2024-04-24T23:11:32.795 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45657.log.gz 2024-04-24T23:11:32.795 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53829.log 2024-04-24T23:11:32.795 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49097.log.gz 2024-04-24T23:11:32.795 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.70425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57320.log 2024-04-24T23:11:32.795 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.70425.log.gz 2024-04-24T23:11:32.795 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.53829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47847.log 2024-04-24T23:11:32.795 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53829.log.gz 2024-04-24T23:11:32.795 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.57320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68608.log 2024-04-24T23:11:32.796 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57320.log.gz 2024-04-24T23:11:32.796 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30764.log 2024-04-24T23:11:32.796 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47847.log.gz 2024-04-24T23:11:32.796 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.68608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55559.log 2024-04-24T23:11:32.796 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61291.log 2024-04-24T23:11:32.796 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.68608.log.gz 2024-04-24T23:11:32.796 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.30764.log: /var/log/ceph/ceph-client.admin.55559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40224.log 2024-04-24T23:11:32.796 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.30764.log.gz 2024-04-24T23:11:32.796 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.55559.log.gz 2024-04-24T23:11:32.796 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.61291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69165.log 2024-04-24T23:11:32.796 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.40224.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61291.log.gz 2024-04-24T23:11:32.797 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33304.log 2024-04-24T23:11:32.797 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40224.log.gz 2024-04-24T23:11:32.797 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.69165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58067.log 2024-04-24T23:11:32.797 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.33304.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.69165.log.gz 2024-04-24T23:11:32.797 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49323.log 2024-04-24T23:11:32.797 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33304.log.gz/var/log/ceph/ceph-client.0.58067.log: 2024-04-24T23:11:32.797 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51990.log 2024-04-24T23:11:32.797 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.58067.log.gz 2024-04-24T23:11:32.797 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49013.log 2024-04-24T23:11:32.798 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49323.log.gz 2024-04-24T23:11:32.798 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56806.log 2024-04-24T23:11:32.798 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49013.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51990.log.gz 2024-04-24T23:11:32.798 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56512.log 2024-04-24T23:11:32.798 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49013.log.gz 2024-04-24T23:11:32.798 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.56806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37045.log 2024-04-24T23:11:32.798 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.56512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37501.log 2024-04-24T23:11:32.798 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56806.log.gz 2024-04-24T23:11:32.798 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40602.log 2024-04-24T23:11:32.798 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.56512.log.gz 2024-04-24T23:11:32.799 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37045.log.gz 2024-04-24T23:11:32.799 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54434.log 2024-04-24T23:11:32.799 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.40602.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37501.log.gz 2024-04-24T23:11:32.799 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70131.log 2024-04-24T23:11:32.799 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.54434.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40602.log.gz 2024-04-24T23:11:32.799 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52032.log 2024-04-24T23:11:32.799 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54434.log.gz 2024-04-24T23:11:32.799 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.70131.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-24T23:11:32.799 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.70131.log.gz 2024-04-24T23:11:32.799 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66069.log 2024-04-24T23:11:32.799 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52032.log.gz 2024-04-24T23:11:32.800 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38389.log 2024-04-24T23:11:32.800 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.66069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69123.log 2024-04-24T23:11:32.800 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.38389.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.66069.log.gz 2024-04-24T23:11:32.800 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48099.log 2024-04-24T23:11:32.800 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38389.log.gz 2024-04-24T23:11:32.800 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.69123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68813.log 2024-04-24T23:11:32.800 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33515.log 2024-04-24T23:11:32.800 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.69123.log.gz 2024-04-24T23:11:32.800 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48099.log.gz 2024-04-24T23:11:32.801 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.68813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57232.log 2024-04-24T23:11:32.801 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.68813.log.gz 2024-04-24T23:11:32.801 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.33515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42520.log 2024-04-24T23:11:32.801 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.57232.log: 86.6% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-24T23:11:32.801 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53361.log 2024-04-24T23:11:32.801 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57232.log.gz 2024-04-24T23:11:32.801 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43512.log 2024-04-24T23:11:32.801 INFO:teuthology.orchestra.run.smithi032.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.33515.log.gz 2024-04-24T23:11:32.801 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42520.log.gz 2024-04-24T23:11:32.802 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.53361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69879.log 2024-04-24T23:11:32.802 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53361.log.gz 2024-04-24T23:11:32.802 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42874.log 2024-04-24T23:11:32.802 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43512.log.gz 2024-04-24T23:11:32.802 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.69879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36445.log 2024-04-24T23:11:32.802 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44245.log 2024-04-24T23:11:32.802 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.69879.log.gz 2024-04-24T23:11:32.802 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42874.log.gz 2024-04-24T23:11:32.802 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.36445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65985.log 2024-04-24T23:11:32.803 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44245.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36445.log.gz 2024-04-24T23:11:32.803 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39542.log 2024-04-24T23:11:32.803 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44245.log.gz 2024-04-24T23:11:32.803 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.65985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70467.log 2024-04-24T23:11:32.803 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.65985.log.gz 2024-04-24T23:11:32.803 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.39542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61788.log 2024-04-24T23:11:32.803 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39542.log.gz 2024-04-24T23:11:32.803 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.70467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36283.log 2024-04-24T23:11:32.803 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.70467.log.gz 2024-04-24T23:11:32.803 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.61788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62904.log 2024-04-24T23:11:32.803 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61788.log.gz 2024-04-24T23:11:32.803 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.36283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38095.log 2024-04-24T23:11:32.804 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36283.log.gz 2024-04-24T23:11:32.804 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.62904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47664.log 2024-04-24T23:11:32.804 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.38095.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62904.log.gz 2024-04-24T23:11:32.804 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59945.log 2024-04-24T23:11:32.804 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38095.log.gz 2024-04-24T23:11:32.804 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54626.log 2024-04-24T23:11:32.804 INFO:teuthology.orchestra.run.smithi032.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47664.log.gz 2024-04-24T23:11:32.804 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.59945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45240.log 2024-04-24T23:11:32.804 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.59945.log.gz 2024-04-24T23:11:32.804 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.54626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47049.log 2024-04-24T23:11:32.804 INFO:teuthology.orchestra.run.smithi032.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.54626.log.gz 2024-04-24T23:11:32.804 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59903.log 2024-04-24T23:11:32.804 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40560.log 2024-04-24T23:11:32.804 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45240.log.gz 2024-04-24T23:11:32.805 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47049.log.gz 2024-04-24T23:11:32.805 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.59903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65897.log 2024-04-24T23:11:32.805 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59903.log.gz 2024-04-24T23:11:32.805 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.28565.log 2024-04-24T23:11:32.805 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.40560.log: /var/log/ceph/ceph-client.admin.65897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69478.log 2024-04-24T23:11:32.805 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40560.log.gz 2024-04-24T23:11:32.805 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.65897.log.gz 2024-04-24T23:11:32.805 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph.tmp-client.admin.28565.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-24T23:11:32.805 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.28565.log.gz 2024-04-24T23:11:32.805 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.69478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56469.log 2024-04-24T23:11:32.805 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.69478.log.gz 2024-04-24T23:11:32.805 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36613.log 2024-04-24T23:11:32.806 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-osd.2.log: /var/log/ceph/ceph-client.admin.56469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66621.log 2024-04-24T23:11:32.806 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.36613.log: 87.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.56469.log.gz 2024-04-24T23:11:32.806 INFO:teuthology.orchestra.run.smithi032.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.34955.log 2024-04-24T23:11:32.806 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36613.log.gz 2024-04-24T23:11:32.806 INFO:teuthology.orchestra.run.smithi032.stderr:gzip/var/log/ceph/ceph-client.admin.66621.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.63954.log 2024-04-24T23:11:32.806 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.34955.log: gzip 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66621.log.gz -5 2024-04-24T23:11:32.806 INFO:teuthology.orchestra.run.smithi032.stderr: --verbose -- /var/log/ceph/ceph-client.admin.38137.log 2024-04-24T23:11:32.806 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34955.log.gz 2024-04-24T23:11:32.806 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.63954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31750.log 2024-04-24T23:11:32.807 INFO:teuthology.orchestra.run.smithi032.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.63954.log.gz 2024-04-24T23:11:32.807 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.38137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40392.log 2024-04-24T23:11:32.807 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.31750.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38137.log.gz 2024-04-24T23:11:32.807 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47973.log 2024-04-24T23:11:32.807 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.31750.log.gz 2024-04-24T23:11:32.807 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.40392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47007.log 2024-04-24T23:11:32.807 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40392.log.gz 2024-04-24T23:11:32.807 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40308.log 2024-04-24T23:11:32.808 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47007.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47973.log.gz 2024-04-24T23:11:32.808 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33347.log 2024-04-24T23:11:32.808 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47007.log.gz 2024-04-24T23:11:32.808 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.40308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49139.log 2024-04-24T23:11:32.808 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40308.log.gz 2024-04-24T23:11:32.808 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.33347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69249.log 2024-04-24T23:11:32.808 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49139.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33347.log.gz 2024-04-24T23:11:32.808 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67809.log 2024-04-24T23:11:32.808 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49139.log.gz 2024-04-24T23:11:32.809 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.69249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58146.log 2024-04-24T23:11:32.809 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.69249.log.gz 2024-04-24T23:11:32.809 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.67809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32005.log 2024-04-24T23:11:32.809 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.58146.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.67809.log.gz 2024-04-24T23:11:32.809 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35522.log 2024-04-24T23:11:32.809 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.58146.log.gz 2024-04-24T23:11:32.809 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.32005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41880.log 2024-04-24T23:11:32.809 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.32005.log.gz 2024-04-24T23:11:32.809 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.35522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68855.log 2024-04-24T23:11:32.810 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41880.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35522.log.gz 2024-04-24T23:11:32.810 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63477.log 2024-04-24T23:11:32.810 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.68855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46755.log 2024-04-24T23:11:32.810 INFO:teuthology.orchestra.run.smithi032.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.41880.log.gz 2024-04-24T23:11:32.810 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.68855.log.gz 2024-04-24T23:11:32.810 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.63477.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.63477.log.gz 2024-04-24T23:11:32.810 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36367.log 2024-04-24T23:11:32.810 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31792.log 2024-04-24T23:11:32.810 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46755.log.gz 2024-04-24T23:11:32.810 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.36367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60258.log 2024-04-24T23:11:32.810 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.31792.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36367.log.gz 2024-04-24T23:11:32.810 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32007.log 2024-04-24T23:11:32.810 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.31792.log.gz 2024-04-24T23:11:32.811 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.60258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57276.log 2024-04-24T23:11:32.811 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.60258.log.gz 2024-04-24T23:11:32.811 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.32007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36697.log 2024-04-24T23:11:32.811 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.57276.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.32007.log.gz 2024-04-24T23:11:32.811 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62862.log 2024-04-24T23:11:32.811 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57276.log.gz 2024-04-24T23:11:32.811 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.36697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31414.log 2024-04-24T23:11:32.811 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36697.log.gz 2024-04-24T23:11:32.811 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.62862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51348.log 2024-04-24T23:11:32.811 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.31414.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.62862.log.gz 2024-04-24T23:11:32.811 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64290.log 2024-04-24T23:11:32.811 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.31414.log.gz 2024-04-24T23:11:32.812 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.0.51348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68148.log 2024-04-24T23:11:32.812 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.51348.log.gz 2024-04-24T23:11:32.812 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.64290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37884.log 2024-04-24T23:11:32.812 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.68148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39131.log 2024-04-24T23:11:32.812 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.68148.log.gz 2024-04-24T23:11:32.812 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37884.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.64290.log.gz 2024-04-24T23:11:32.812 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37171.log 2024-04-24T23:11:32.812 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53048.log 2024-04-24T23:11:32.812 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.39131.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37884.log.gz 2024-04-24T23:11:32.812 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63771.log 2024-04-24T23:11:32.812 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39131.log.gz 2024-04-24T23:11:32.812 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37171.log.gz 2024-04-24T23:11:32.813 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.53048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60003.log 2024-04-24T23:11:32.813 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.63771.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53048.log.gz 2024-04-24T23:11:32.813 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54560.log 2024-04-24T23:11:32.813 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.63771.log.gz 2024-04-24T23:11:32.813 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.60003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43554.log 2024-04-24T23:11:32.813 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60003.log.gz 2024-04-24T23:11:32.813 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.54560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40140.log 2024-04-24T23:11:32.813 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47889.log 2024-04-24T23:11:32.813 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.40140.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43554.log.gz 2024-04-24T23:11:32.813 INFO:teuthology.orchestra.run.smithi032.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.54560.log.gz 2024-04-24T23:11:32.813 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57106.log 2024-04-24T23:11:32.813 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40140.log.gz 2024-04-24T23:11:32.814 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62138.log 2024-04-24T23:11:32.814 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47889.log.gz 2024-04-24T23:11:32.814 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.57106.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-24T23:11:32.814 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.62138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39416.log 2024-04-24T23:11:32.814 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57106.log.gz 2024-04-24T23:11:32.814 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.62138.log.gz 2024-04-24T23:11:32.814 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68771.log 2024-04-24T23:11:32.814 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.39416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59341.log 2024-04-24T23:11:32.814 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39416.log.gz 2024-04-24T23:11:32.814 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.68771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63302.log 2024-04-24T23:11:32.814 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50332.log 2024-04-24T23:11:32.814 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.59341.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.68771.log.gz 2024-04-24T23:11:32.815 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.63302.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.59341.log.gz 2024-04-24T23:11:32.815 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50728.log 2024-04-24T23:11:32.815 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.63302.log.gz 2024-04-24T23:11:32.815 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43370.log 2024-04-24T23:11:32.815 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50332.log: /var/log/ceph/ceph-client.admin.50728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43722.log 2024-04-24T23:11:32.815 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50332.log.gz 2024-04-24T23:11:32.815 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43370.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50728.log.gz 2024-04-24T23:11:32.815 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66027.log 2024-04-24T23:11:32.815 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43370.log.gz 2024-04-24T23:11:32.815 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62820.log 2024-04-24T23:11:32.815 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43722.log.gz 2024-04-24T23:11:32.815 INFO:teuthology.orchestra.run.smithi032.stderr: 89.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-24T23:11:32.816 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65194.log 2024-04-24T23:11:32.816 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.66027.log: /var/log/ceph/ceph-client.admin.62820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31312.log 2024-04-24T23:11:32.816 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.66027.log.gz 2024-04-24T23:11:32.816 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.65194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59735.log 2024-04-24T23:11:32.816 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.62820.log.gz 2024-04-24T23:11:32.816 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.65194.log.gz 2024-04-24T23:11:32.816 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.31312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34703.log 2024-04-24T23:11:32.816 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.31312.log.gz 2024-04-24T23:11:32.816 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.59735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52838.log 2024-04-24T23:11:32.816 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.34703.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.59735.log.gz 2024-04-24T23:11:32.816 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53132.log 2024-04-24T23:11:32.816 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34703.log.gz 2024-04-24T23:11:32.816 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57645.log 2024-04-24T23:11:32.817 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.53132.log: gzip -5 --verbose -- 87.9% /var/log/ceph/ceph-client.admin.54182.log 2024-04-24T23:11:32.817 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52838.log.gz 2024-04-24T23:11:32.817 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.57645.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53132.log.gz 2024-04-24T23:11:32.817 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49365.log 2024-04-24T23:11:32.817 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.54182.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57645.log.gz 2024-04-24T23:11:32.817 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48141.log 2024-04-24T23:11:32.817 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54182.log.gz 2024-04-24T23:11:32.817 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57371.log 2024-04-24T23:11:32.817 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48141.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49365.log.gz 2024-04-24T23:11:32.817 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66537.log 2024-04-24T23:11:32.817 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.57371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30370.log 2024-04-24T23:11:32.817 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48141.log.gz 2024-04-24T23:11:32.817 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.66537.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57371.log.gz 2024-04-24T23:11:32.818 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65504.log 2024-04-24T23:11:32.818 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.30370.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.66537.log.gz 2024-04-24T23:11:32.818 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35737.log 2024-04-24T23:11:32.818 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.30370.log.gz 2024-04-24T23:11:32.818 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.65504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48576.log 2024-04-24T23:11:32.818 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.35737.log: gzip -5 88.6% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.65504.log.gz 2024-04-24T23:11:32.818 INFO:teuthology.orchestra.run.smithi032.stderr: /var/log/ceph/ceph-client.admin.62096.log 2024-04-24T23:11:32.818 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48576.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35737.log.gz 2024-04-24T23:11:32.818 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39888.log 2024-04-24T23:11:32.818 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48576.log.gz 2024-04-24T23:11:32.818 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.62096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69562.log 2024-04-24T23:11:32.818 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.62096.log.gz 2024-04-24T23:11:32.818 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.39888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60951.log 2024-04-24T23:11:32.819 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.69562.log: gzip -5 --verbose -- 88.2% /var/log/ceph/ceph-client.admin.57002.log 2024-04-24T23:11:32.819 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39888.log.gz 2024-04-24T23:11:32.819 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.69562.log.gz 2024-04-24T23:11:32.819 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.60951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51660.log 2024-04-24T23:11:32.819 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.57002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31582.log 2024-04-24T23:11:32.819 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.60951.log.gz 2024-04-24T23:11:32.819 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57002.log.gz 2024-04-24T23:11:32.819 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63397.log 2024-04-24T23:11:32.819 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51660.log.gz 2024-04-24T23:11:32.819 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.31582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42733.log 2024-04-24T23:11:32.819 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.63397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33094.log 2024-04-24T23:11:32.819 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.31582.log.gz 2024-04-24T23:11:32.819 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42733.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.63397.log.gz 2024-04-24T23:11:32.820 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41670.log 2024-04-24T23:11:32.820 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42733.log.gz 2024-04-24T23:11:32.820 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.33094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60132.log 2024-04-24T23:11:32.820 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41670.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33094.log.gz 2024-04-24T23:11:32.820 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52922.log 2024-04-24T23:11:32.820 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41670.log.gz 2024-04-24T23:11:32.820 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.60132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49659.log 2024-04-24T23:11:32.820 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.60132.log.gz 2024-04-24T23:11:32.820 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32884.log 2024-04-24T23:11:32.820 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49659.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52922.log.gz 2024-04-24T23:11:32.820 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38010.log 2024-04-24T23:11:32.820 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49659.log.gz 2024-04-24T23:11:32.820 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.32884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52450.log 2024-04-24T23:11:32.821 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.32884.log.gz 2024-04-24T23:11:32.821 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.38010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34997.log 2024-04-24T23:11:32.821 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55889.log 2024-04-24T23:11:32.821 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38010.log.gz 2024-04-24T23:11:32.821 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.34997.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52450.log.gz 2024-04-24T23:11:32.821 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46587.log 2024-04-24T23:11:32.821 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34997.log.gz 2024-04-24T23:11:32.821 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.55889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68250.log 2024-04-24T23:11:32.821 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37087.log 2024-04-24T23:11:32.821 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55889.log.gz 2024-04-24T23:11:32.821 INFO:teuthology.orchestra.run.smithi032.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46587.log.gz 2024-04-24T23:11:32.821 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.68250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56409.log 2024-04-24T23:11:32.821 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.68250.log.gz 2024-04-24T23:11:32.822 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36487.log 2024-04-24T23:11:32.822 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.56409.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37087.log.gz 2024-04-24T23:11:32.822 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49449.log 2024-04-24T23:11:32.822 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56409.log.gz 2024-04-24T23:11:32.822 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.36487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49181.log 2024-04-24T23:11:32.822 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36487.log.gz 2024-04-24T23:11:32.822 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47290.log 2024-04-24T23:11:32.822 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49181.log: gzip -5 88.7% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.49449.log.gz /var/log/ceph/ceph-client.admin.31142.log 2024-04-24T23:11:32.822 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-04-24T23:11:32.822 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47290.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49181.log.gz 2024-04-24T23:11:32.822 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56764.log 2024-04-24T23:11:32.822 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47290.log.gz 2024-04-24T23:11:32.822 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.31142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60770.log 2024-04-24T23:11:32.823 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.56764.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.31142.log.gz 2024-04-24T23:11:32.823 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56596.log 2024-04-24T23:11:32.823 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.60770.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56764.log.gz 2024-04-24T23:11:32.823 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41946.log 2024-04-24T23:11:32.823 INFO:teuthology.orchestra.run.smithi032.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60770.log.gz 2024-04-24T23:11:32.823 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.56596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39500.log 2024-04-24T23:11:32.823 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58676.log 88.3% 2024-04-24T23:11:32.823 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.56596.log.gz 2024-04-24T23:11:32.823 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41946.log.gz 2024-04-24T23:11:32.823 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.39500.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-24T23:11:32.823 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39500.log.gz 2024-04-24T23:11:32.823 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.58676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40962.log 2024-04-24T23:11:32.823 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33473.log 2024-04-24T23:11:32.824 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58676.log.gz 2024-04-24T23:11:32.824 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.40962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57813.log 2024-04-24T23:11:32.824 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40962.log.gz 2024-04-24T23:11:32.824 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.33473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59777.log 2024-04-24T23:11:32.824 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.57813.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33473.log.gz 2024-04-24T23:11:32.824 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32688.log 2024-04-24T23:11:32.824 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32350.log 2024-04-24T23:11:32.824 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.59777.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.57813.log.gz 2024-04-24T23:11:32.824 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.32688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31456.log 2024-04-24T23:11:32.824 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59777.log.gz 2024-04-24T23:11:32.825 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43328.log 2024-04-24T23:11:32.825 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.32350.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.32688.log.gz 2024-04-24T23:11:32.825 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.31456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66844.log 2024-04-24T23:11:32.825 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.32350.log.gz 2024-04-24T23:11:32.825 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.31456.log.gz 2024-04-24T23:11:32.825 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52324.log 2024-04-24T23:11:32.825 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43328.log: /var/log/ceph/ceph-client.admin.66844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41312.log 2024-04-24T23:11:32.825 INFO:teuthology.orchestra.run.smithi032.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.43328.log.gz 2024-04-24T23:11:32.825 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.66844.log.gz 2024-04-24T23:11:32.825 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55847.log 2024-04-24T23:11:32.825 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70509.log 2024-04-24T23:11:32.825 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41312.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.52324.log.gz 2024-04-24T23:11:32.826 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.55847.log: gzip 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41312.log.gz -5 2024-04-24T23:11:32.826 INFO:teuthology.orchestra.run.smithi032.stderr: --verbose -- /var/log/ceph/ceph-client.admin.30668.log 2024-04-24T23:11:32.826 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.55847.log.gz 2024-04-24T23:11:32.826 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62610.log 2024-04-24T23:11:32.826 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.70509.log: /var/log/ceph/ceph-client.admin.30668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37415.log 2024-04-24T23:11:32.826 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.30668.log.gz 2024-04-24T23:11:32.826 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.70509.log.gz 2024-04-24T23:11:32.826 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.62610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62526.log 2024-04-24T23:11:32.826 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53667.log 2024-04-24T23:11:32.826 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37415.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.62610.log.gz 2024-04-24T23:11:32.826 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.62526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58821.log 2024-04-24T23:11:32.826 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37415.log.gz 2024-04-24T23:11:32.827 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41046.log 2024-04-24T23:11:32.827 INFO:teuthology.orchestra.run.smithi032.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.62526.log.gz 2024-04-24T23:11:32.827 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.53667.log: /var/log/ceph/ceph-client.admin.58821.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53667.log.gz 2024-04-24T23:11:32.827 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56554.log 2024-04-24T23:11:32.827 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.58821.log.gz 2024-04-24T23:11:32.827 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69963.log 2024-04-24T23:11:32.827 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41046.log: /var/log/ceph/ceph-client.admin.56554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43596.log 2024-04-24T23:11:32.827 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41046.log.gz 2024-04-24T23:11:32.827 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56554.log.gz 2024-04-24T23:11:32.827 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.69963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57771.log 2024-04-24T23:11:32.827 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46073.log 2024-04-24T23:11:32.827 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43596.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.69963.log.gz 2024-04-24T23:11:32.828 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.57771.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43596.log.gz 2024-04-24T23:11:32.828 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58462.log 2024-04-24T23:11:32.828 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70633.log 2024-04-24T23:11:32.828 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46073.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57771.log.gz 2024-04-24T23:11:32.828 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.58462.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46073.log.gz 2024-04-24T23:11:32.828 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57729.log 2024-04-24T23:11:32.828 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37839.log 2024-04-24T23:11:32.828 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6%/var/log/ceph/ceph-client.admin.70633.log: -- replaced with /var/log/ceph/ceph-client.admin.58462.log.gz 2024-04-24T23:11:32.828 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.57729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60858.log 2024-04-24T23:11:32.828 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.70633.log.gz 2024-04-24T23:11:32.828 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49407.log 2024-04-24T23:11:32.828 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37839.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57729.log.gz 2024-04-24T23:11:32.829 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.60858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39704.log 2024-04-24T23:11:32.829 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37839.log.gz 2024-04-24T23:11:32.829 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68010.log 2024-04-24T23:11:32.829 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49407.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60858.log.gz 2024-04-24T23:11:32.829 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.39704.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49407.log.gz 2024-04-24T23:11:32.829 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44904.log 2024-04-24T23:11:32.829 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43286.log 2024-04-24T23:11:32.829 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39704.log.gz 2024-04-24T23:11:32.829 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.68010.log: /var/log/ceph/ceph-client.admin.44904.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.68010.log.gz 2024-04-24T23:11:32.829 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57455.log 2024-04-24T23:11:32.829 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56902.log 2024-04-24T23:11:32.829 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44904.log.gz/var/log/ceph/ceph-client.admin.43286.log: 2024-04-24T23:11:32.830 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.57455.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43286.log.gz 2024-04-24T23:11:32.830 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65278.log 2024-04-24T23:11:32.830 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57455.log.gz 2024-04-24T23:11:32.830 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35294.log 2024-04-24T23:11:32.830 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.56902.log: /var/log/ceph/ceph-client.admin.65278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37968.log 2024-04-24T23:11:32.830 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56902.log.gz 2024-04-24T23:11:32.830 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.35294.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65278.log.gz 2024-04-24T23:11:32.830 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44119.log 2024-04-24T23:11:32.830 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-24T23:11:32.830 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37968.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.35294.log.gz 2024-04-24T23:11:32.830 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44119.log: gzip -5 --verbose -- 88.1% /var/log/ceph/ceph-client.admin.37585.log -- replaced with /var/log/ceph/ceph-client.admin.37968.log.gz 2024-04-24T23:11:32.830 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-04-24T23:11:32.831 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44119.log.gz 2024-04-24T23:11:32.831 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51159.log 2024-04-24T23:11:32.831 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52074.log 2024-04-24T23:11:32.831 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51159.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37585.log.gz 2024-04-24T23:11:32.831 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66195.log 2024-04-24T23:11:32.831 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51159.log.gz 2024-04-24T23:11:32.831 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32347.log 2024-04-24T23:11:32.831 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52074.log.gz 2024-04-24T23:11:32.831 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.66195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70341.log 2024-04-24T23:11:32.831 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.32347.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.66195.log.gz 2024-04-24T23:11:32.831 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50896.log 2024-04-24T23:11:32.831 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.32347.log.gz 2024-04-24T23:11:32.831 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.70341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57879.log 2024-04-24T23:11:32.832 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50589.log 2024-04-24T23:11:32.832 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50896.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.70341.log.gz 2024-04-24T23:11:32.832 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.57879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38866.log 2024-04-24T23:11:32.832 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50896.log.gz 2024-04-24T23:11:32.832 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55094.log 2024-04-24T23:11:32.832 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57879.log.gz 2024-04-24T23:11:32.832 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50589.log.gz 2024-04-24T23:11:32.832 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.38866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67034.log 2024-04-24T23:11:32.832 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47195.log 2024-04-24T23:11:32.832 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.55094.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38866.log.gz 2024-04-24T23:11:32.832 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.67034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34787.log 2024-04-24T23:11:32.832 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55094.log.gz 2024-04-24T23:11:32.833 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.67034.log.gz 2024-04-24T23:11:32.833 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50980.log 2024-04-24T23:11:32.833 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.34787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53277.log 2024-04-24T23:11:32.833 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47195.log.gz 2024-04-24T23:11:32.833 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50980.log: gzip -5 --verbose -- 88.0% /var/log/ceph/ceph-client.admin.61249.log 2024-04-24T23:11:32.833 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34787.log.gz 2024-04-24T23:11:32.833 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50980.log.gz 2024-04-24T23:11:32.833 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.53277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35081.log 2024-04-24T23:11:32.833 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53277.log.gz 2024-04-24T23:11:32.833 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67202.log 2024-04-24T23:11:32.833 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.61249.log: /var/log/ceph/ceph-client.admin.35081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59861.log 2024-04-24T23:11:32.834 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.61249.log.gz 2024-04-24T23:11:32.834 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35081.log.gz 2024-04-24T23:11:32.834 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.67202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54392.log 2024-04-24T23:11:32.834 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30845.log 2024-04-24T23:11:32.834 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.59861.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.67202.log.gz 2024-04-24T23:11:32.834 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.54392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36911.log 2024-04-24T23:11:32.834 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.59861.log.gz 2024-04-24T23:11:32.834 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54392.log.gz 2024-04-24T23:11:32.834 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.30845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69081.log 2024-04-24T23:11:32.834 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.36911.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-24T23:11:32.834 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.30845.log.gz 2024-04-24T23:11:32.835 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.69081.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36911.log.gz 2024-04-24T23:11:32.835 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-24T23:11:32.835 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.69081.log.gz 2024-04-24T23:11:32.835 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60814.log 2024-04-24T23:11:32.835 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52570.log 2024-04-24T23:11:32.835 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.60814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44524.log 2024-04-24T23:11:32.835 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52570.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.60814.log.gz 2024-04-24T23:11:32.835 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60993.log 2024-04-24T23:11:32.835 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52570.log.gz 2024-04-24T23:11:32.835 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46235.log 2024-04-24T23:11:32.835 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44524.log: /var/log/ceph/ceph-client.admin.60993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67076.log 2024-04-24T23:11:32.835 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46235.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44524.log.gz 89.1% -- replaced with /var/log/ceph/ceph-client.admin.60993.log.gz 2024-04-24T23:11:32.836 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-04-24T23:11:32.836 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46235.log.gz 2024-04-24T23:11:32.836 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59651.log 2024-04-24T23:11:32.836 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.67076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44732.log 2024-04-24T23:11:32.836 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.59651.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.67076.log.gz 2024-04-24T23:11:32.836 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70299.log 2024-04-24T23:11:32.836 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.59651.log.gz 2024-04-24T23:11:32.836 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30930.log 2024-04-24T23:11:32.836 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44732.log: /var/log/ceph/ceph-client.admin.70299.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44732.log.gz 2024-04-24T23:11:32.836 INFO:teuthology.orchestra.run.smithi032.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.70299.log.gz 2024-04-24T23:11:32.836 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66751.log 2024-04-24T23:11:32.836 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.30930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51469.log 2024-04-24T23:11:32.837 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.66751.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.30930.log.gz 2024-04-24T23:11:32.837 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63049.log 2024-04-24T23:11:32.837 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69436.log 2024-04-24T23:11:32.837 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.66751.log.gz 2024-04-24T23:11:32.837 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51469.log: gzip/var/log/ceph/ceph-client.admin.63049.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.34871.log 2024-04-24T23:11:32.837 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.69436.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.63049.log.gz 2024-04-24T23:11:32.837 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51469.log.gz 2024-04-24T23:11:32.837 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.69436.log.gz 2024-04-24T23:11:32.837 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70593.log 2024-04-24T23:11:32.838 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.34871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38305.log 2024-04-24T23:11:32.838 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.70593.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34871.log.gz 2024-04-24T23:11:32.838 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53871.log 2024-04-24T23:11:32.838 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.70593.log.gz 2024-04-24T23:11:32.838 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39830.log 2024-04-24T23:11:32.838 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.38305.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-24T23:11:32.838 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.53871.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38305.log.gz 2024-04-24T23:11:32.838 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.39830.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53871.log.gz 2024-04-24T23:11:32.838 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39830.log.gz 2024-04-24T23:11:32.839 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63727.log 2024-04-24T23:11:32.839 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67310.log 2024-04-24T23:11:32.839 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.63727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45072.log 2024-04-24T23:11:32.839 INFO:teuthology.orchestra.run.smithi032.stderr: 94.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-24T23:11:32.839 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.63727.log.gz 2024-04-24T23:11:32.839 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.67310.log: gzip -5 --verbose -- 88.4% -- replaced with /var/log/ceph/ceph-client.admin.67310.log.gz 2024-04-24T23:11:32.839 INFO:teuthology.orchestra.run.smithi032.stderr: /var/log/ceph/ceph-client.admin.46713.log 2024-04-24T23:11:32.839 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60048.log 2024-04-24T23:11:32.839 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45072.log.gz 2024-04-24T23:11:32.840 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37002.log 2024-04-24T23:11:32.840 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46713.log.gz 2024-04-24T23:11:32.840 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.60048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43932.log 2024-04-24T23:11:32.840 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60048.log.gz 2024-04-24T23:11:32.840 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59221.log 2024-04-24T23:11:32.840 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37002.log.gz 2024-04-24T23:11:32.840 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64757.log 2024-04-24T23:11:32.840 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31098.log 2024-04-24T23:11:32.840 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.59221.log: 88.8%/var/log/ceph/ceph-client.admin.64757.log: -- replaced with /var/log/ceph/ceph-client.admin.43932.log.gz 2024-04-24T23:11:32.840 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59221.log.gz 88.0% -- replaced with /var/log/ceph/ceph-client.admin.64757.log.gz 2024-04-24T23:11:32.841 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55685.log 2024-04-24T23:11:32.841 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-04-24T23:11:32.841 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.31098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44161.log 2024-04-24T23:11:32.841 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.55685.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.31098.log.gz 2024-04-24T23:11:32.841 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60643.log 2024-04-24T23:11:32.841 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55685.log.gz/var/log/ceph/ceph-client.admin.44161.log: 2024-04-24T23:11:32.841 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69207.log 2024-04-24T23:11:32.841 INFO:teuthology.orchestra.run.smithi032.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44161.log.gz 2024-04-24T23:11:32.842 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66707.log 2024-04-24T23:11:32.842 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.60643.log: /var/log/ceph/ceph-client.admin.69207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49239.log 2024-04-24T23:11:32.842 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.69207.log.gz 2024-04-24T23:11:32.842 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.66707.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.60643.log.gz 2024-04-24T23:11:32.842 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46629.log 2024-04-24T23:11:32.842 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.66707.log.gz 2024-04-24T23:11:32.842 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54935.log 2024-04-24T23:11:32.842 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2%/var/log/ceph/ceph-client.admin.46629.log: -- replaced with /var/log/ceph/ceph-client.admin.49239.log.gz 2024-04-24T23:11:32.842 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64948.log 2024-04-24T23:11:32.842 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46629.log.gz 2024-04-24T23:11:32.843 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46031.log 2024-04-24T23:11:32.843 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.54935.log: gzip 87.7% -5 --verbose -- /var/log/ceph/ceph-client.admin.70551.log 2024-04-24T23:11:32.843 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54935.log.gz/var/log/ceph/ceph-client.admin.64948.log: 2024-04-24T23:11:32.843 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68208.log 2024-04-24T23:11:32.843 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.64948.log.gz 2024-04-24T23:11:32.843 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46031.log: /var/log/ceph/ceph-client.admin.70551.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46031.log.gz 2024-04-24T23:11:32.843 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.70551.log.gz 2024-04-24T23:11:32.843 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45390.log 2024-04-24T23:11:32.843 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47620.log 2024-04-24T23:11:32.843 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.68208.log: /var/log/ceph/ceph-client.admin.45390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50459.log 2024-04-24T23:11:32.843 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47620.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.68208.log.gz 2024-04-24T23:11:32.843 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45390.log.gz 2024-04-24T23:11:32.844 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57585.log 2024-04-24T23:11:32.844 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47620.log.gz 2024-04-24T23:11:32.844 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68440.log 2024-04-24T23:11:32.844 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48455.log 2024-04-24T23:11:32.844 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.57585.log: 89.1%/var/log/ceph/ceph-client.admin.68440.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.57585.log.gz 2024-04-24T23:11:32.844 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50459.log.gz 2024-04-24T23:11:32.844 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61726.log 2024-04-24T23:11:32.844 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.68440.log.gz 2024-04-24T23:11:32.844 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.0.48455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40014.log 2024-04-24T23:11:32.844 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.48455.log.gz 2024-04-24T23:11:32.844 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67577.log 2024-04-24T23:11:32.845 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.61726.log: /var/log/ceph/ceph-client.admin.40014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58188.log 2024-04-24T23:11:32.845 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.67577.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.61726.log.gz 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40014.log.gz 2024-04-24T23:11:32.845 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-04-24T23:11:32.845 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.67577.log.gz 2024-04-24T23:11:32.845 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60518.log 2024-04-24T23:11:32.845 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.58188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49888.log 2024-04-24T23:11:32.845 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58188.log.gz 2024-04-24T23:11:32.845 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.60518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48893.log 2024-04-24T23:11:32.845 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.60518.log.gz 2024-04-24T23:11:32.845 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63600.log 2024-04-24T23:11:32.845 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49888.log.gz 2024-04-24T23:11:32.846 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54308.log 2024-04-24T23:11:32.846 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48893.log.gz 2024-04-24T23:11:32.846 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.63600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68524.log 2024-04-24T23:11:32.846 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.63600.log.gz 2024-04-24T23:11:32.846 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.54308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61081.log 2024-04-24T23:11:32.846 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54308.log.gz 2024-04-24T23:11:32.846 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67244.log 2024-04-24T23:11:32.846 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.68524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70383.log 2024-04-24T23:11:32.846 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.61081.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.68524.log.gz 2024-04-24T23:11:32.846 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.67244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62568.log 2024-04-24T23:11:32.846 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.61081.log.gz/var/log/ceph/ceph-client.admin.70383.log: 2024-04-24T23:11:32.846 INFO:teuthology.orchestra.run.smithi032.stderr: 91.2% 87.5% -- replaced with /var/log/ceph/ceph-client.admin.70383.log.gz 2024-04-24T23:11:32.847 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39458.log 2024-04-24T23:11:32.847 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.67244.log.gz 2024-04-24T23:11:32.847 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.62568.log: -- /var/log/ceph/ceph-client.admin.47494.log 2024-04-24T23:11:32.847 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.39458.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62568.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46503.log 2024-04-24T23:11:32.847 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-04-24T23:11:32.847 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39458.log.gz 2024-04-24T23:11:32.847 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45156.log 2024-04-24T23:11:32.847 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47494.log: /var/log/ceph/ceph-client.admin.46503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42691.log 2024-04-24T23:11:32.847 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45156.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.47494.log.gz 2024-04-24T23:11:32.847 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46503.log.gz 2024-04-24T23:11:32.847 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37627.log 2024-04-24T23:11:32.848 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45156.log.gz 2024-04-24T23:11:32.848 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.42691.log: -- /var/log/ceph/ceph-client.admin.32926.log 2024-04-24T23:11:32.848 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37627.log: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69795.log 2024-04-24T23:11:32.848 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42691.log.gz 2024-04-24T23:11:32.848 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37627.log.gz 2024-04-24T23:11:32.848 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.32926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58420.log 2024-04-24T23:11:32.848 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.69795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38263.log 2024-04-24T23:11:32.848 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32926.log.gz 2024-04-24T23:11:32.848 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.58420.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.69795.log.gz 2024-04-24T23:11:32.848 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37293.log 2024-04-24T23:11:32.848 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58420.log.gz 2024-04-24T23:11:32.849 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52796.log 2024-04-24T23:11:32.849 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.38263.log: /var/log/ceph/ceph-client.admin.37293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50251.log 2024-04-24T23:11:32.849 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52796.log: 88.1% 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38263.log.gz 2024-04-24T23:11:32.849 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37293.log.gz 2024-04-24T23:11:32.849 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52796.log.gz 2024-04-24T23:11:32.849 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52964.log 2024-04-24T23:11:32.849 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31666.log 2024-04-24T23:11:32.849 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55989.log 2024-04-24T23:11:32.849 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52964.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50251.log.gz 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52964.log.gz 2024-04-24T23:11:32.849 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.31666.log: 2024-04-24T23:11:32.849 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- 89.1% -- replaced with /var/log/ceph/ceph-client.admin.31666.log.gz 2024-04-24T23:11:32.849 INFO:teuthology.orchestra.run.smithi032.stderr: /var/log/ceph/ceph-client.admin.69391.log 2024-04-24T23:11:32.850 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.55989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68687.log 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55989.log.gz 2024-04-24T23:11:32.850 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-04-24T23:11:32.850 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.69391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55805.log 2024-04-24T23:11:32.850 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.69391.log.gz 2024-04-24T23:11:32.850 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.68687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51093.log 89.5% -- replaced with /var/log/ceph/ceph-client.admin.68687.log.gz 2024-04-24T23:11:32.850 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-04-24T23:11:32.850 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.55805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44774.log 2024-04-24T23:11:32.850 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8%/var/log/ceph/ceph-client.admin.51093.log: -- replaced with /var/log/ceph/ceph-client.admin.55805.log.gz 2024-04-24T23:11:32.850 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47091.log 2024-04-24T23:11:32.850 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67681.log 2024-04-24T23:11:32.850 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44774.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.51093.log.gz 2024-04-24T23:11:32.851 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47091.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44774.log.gz 2024-04-24T23:11:32.851 INFO:teuthology.orchestra.run.smithi032.stderr:gzip 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47091.log.gz 2024-04-24T23:11:32.851 INFO:teuthology.orchestra.run.smithi032.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.61039.log 2024-04-24T23:11:32.851 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.67681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54041.log 2024-04-24T23:11:32.851 INFO:teuthology.orchestra.run.smithi032.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.67681.log.gz 2024-04-24T23:11:32.851 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.61039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60300.log 2024-04-24T23:11:32.851 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.61039.log.gz 2024-04-24T23:11:32.851 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-24T23:11:32.851 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.54041.log: /var/log/ceph/ceph-client.admin.60300.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54041.log.gz 2024-04-24T23:11:32.851 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.60300.log.gz 2024-04-24T23:11:32.851 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63996.log 2024-04-24T23:11:32.852 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37457.log 2024-04-24T23:11:32.852 INFO:teuthology.orchestra.run.smithi032.stderr:gzip/var/log/ceph/ceph-client.admin.63996.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.68566.log 2024-04-24T23:11:32.852 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37457.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.63996.log.gz 2024-04-24T23:11:32.852 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54224.log 2024-04-24T23:11:32.852 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37457.log.gz 2024-04-24T23:11:32.852 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.68566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52754.log 2024-04-24T23:11:32.852 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.68566.log.gz/var/log/ceph/ceph-client.admin.54224.log: 2024-04-24T23:11:32.852 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36960.log 2024-04-24T23:11:32.852 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9%/var/log/ceph/ceph-client.admin.52754.log: -- replaced with /var/log/ceph/ceph-client.admin.54224.log.gz 2024-04-24T23:11:32.852 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40881.log 2024-04-24T23:11:32.852 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52754.log.gz 2024-04-24T23:11:32.852 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.36960.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36960.log.gz 2024-04-24T23:11:32.852 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35420.log 2024-04-24T23:11:32.853 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.40881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49575.log 2024-04-24T23:11:32.853 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40881.log.gz 2024-04-24T23:11:32.853 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.35420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58378.log 2024-04-24T23:11:32.853 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49575.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35420.log.gz 2024-04-24T23:11:32.853 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49575.log.gz 2024-04-24T23:11:32.853 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63091.log 2024-04-24T23:11:32.853 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.58378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62258.log 2024-04-24T23:11:32.853 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58378.log.gz 2024-04-24T23:11:32.853 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.63091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49717.log 2024-04-24T23:11:32.853 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.62258.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.63091.log.gz 2024-04-24T23:11:32.853 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.62258.log.gz 2024-04-24T23:11:32.854 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45198.log 2024-04-24T23:11:32.854 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51948.log 2024-04-24T23:11:32.854 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49717.log.gz 2024-04-24T23:11:32.854 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45198.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.45198.log.gz 2024-04-24T23:11:32.854 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66153.log 2024-04-24T23:11:32.854 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64694.log 2024-04-24T23:11:32.854 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51948.log: /var/log/ceph/ceph-client.admin.66153.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51948.log.gz 2024-04-24T23:11:32.854 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37669.log 2024-04-24T23:11:32.854 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.66153.log.gz 2024-04-24T23:11:32.854 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.64694.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.64694.log.gz 2024-04-24T23:11:32.854 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53487.log 2024-04-24T23:11:32.854 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64038.log 2024-04-24T23:11:32.855 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37669.log: /var/log/ceph/ceph-client.admin.53487.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37669.log.gz 2024-04-24T23:11:32.855 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53487.log.gz 2024-04-24T23:11:32.855 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47536.log 2024-04-24T23:11:32.855 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.64038.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.64038.log.gz 2024-04-24T23:11:32.855 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67851.log 2024-04-24T23:11:32.855 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38908.log 2024-04-24T23:11:32.855 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47536.log: /var/log/ceph/ceph-client.admin.67851.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47536.log.gz 2024-04-24T23:11:32.855 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58905.log 2024-04-24T23:11:32.855 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.67851.log.gz 2024-04-24T23:11:32.855 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.38908.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.38908.log.gz 2024-04-24T23:11:32.855 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70005.log 2024-04-24T23:11:32.856 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38950.log 2024-04-24T23:11:32.856 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.58905.log: /var/log/ceph/ceph-client.admin.70005.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.58905.log.gz 2024-04-24T23:11:32.857 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54350.log 2024-04-24T23:11:32.857 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.70005.log.gz 2024-04-24T23:11:32.857 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.38950.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38950.log.gz 2024-04-24T23:11:32.857 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32343.log 2024-04-24T23:11:32.857 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36025.log 2024-04-24T23:11:32.857 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.54350.log: /var/log/ceph/ceph-client.admin.32343.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54350.log.gz 2024-04-24T23:11:32.858 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.32343.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39662.log 2024-04-24T23:11:32.858 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-04-24T23:11:32.858 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.36025.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36025.log.gz 2024-04-24T23:11:32.858 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65672.log 2024-04-24T23:11:32.858 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69520.log 2024-04-24T23:11:32.858 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.39662.log: /var/log/ceph/ceph-client.admin.65672.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39662.log.gz 2024-04-24T23:11:32.858 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.65672.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35167.log 2024-04-24T23:11:32.858 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-04-24T23:11:32.858 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.69520.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.69520.log.gz 2024-04-24T23:11:32.858 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56199.log 2024-04-24T23:11:32.858 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63217.log 2024-04-24T23:11:32.858 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.35167.log: /var/log/ceph/ceph-client.admin.56199.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35167.log.gz 2024-04-24T23:11:32.859 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56199.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63260.log 2024-04-24T23:11:32.859 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-04-24T23:11:32.859 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.63217.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.63217.log.gz 2024-04-24T23:11:32.859 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43990.log 2024-04-24T23:11:32.859 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31498.log 2024-04-24T23:11:32.859 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.63260.log: /var/log/ceph/ceph-client.admin.43990.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.63260.log.gz 2024-04-24T23:11:32.859 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56073.log 2024-04-24T23:11:32.859 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43990.log.gz 2024-04-24T23:11:32.859 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.31498.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.31498.log.gz 2024-04-24T23:11:32.859 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39972.log 2024-04-24T23:11:32.859 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52654.log 2024-04-24T23:11:32.859 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.56073.log: /var/log/ceph/ceph-client.admin.39972.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.56073.log.gz 2024-04-24T23:11:32.860 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39972.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32011.log 2024-04-24T23:11:32.860 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-04-24T23:11:32.860 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52654.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52654.log.gz 2024-04-24T23:11:32.860 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62778.log 2024-04-24T23:11:32.860 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70215.log 2024-04-24T23:11:32.860 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.32011.log: /var/log/ceph/ceph-client.admin.62778.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32011.log.gz 2024-04-24T23:11:32.860 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.62778.log.gz 2024-04-24T23:11:32.860 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44444.log 2024-04-24T23:11:32.860 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.70215.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.70215.log.gz 2024-04-24T23:11:32.860 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35462.log 2024-04-24T23:11:32.860 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70173.log 2024-04-24T23:11:32.860 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44444.log: /var/log/ceph/ceph-client.admin.35462.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44444.log.gz 2024-04-24T23:11:32.861 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31624.log 2024-04-24T23:11:32.861 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35462.log.gz 2024-04-24T23:11:32.861 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.70173.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.70173.log.gz 2024-04-24T23:11:32.861 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49972.log 2024-04-24T23:11:32.861 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50938.log 2024-04-24T23:11:32.861 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.31624.log: /var/log/ceph/ceph-client.admin.49972.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.31624.log.gz 2024-04-24T23:11:32.861 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55196.log 2024-04-24T23:11:32.861 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49972.log.gz 2024-04-24T23:11:32.861 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50938.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50938.log.gz 2024-04-24T23:11:32.861 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31708.log 2024-04-24T23:11:32.861 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63175.log 2024-04-24T23:11:32.861 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.55196.log: /var/log/ceph/ceph-client.admin.31708.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.55196.log.gz 2024-04-24T23:11:32.862 INFO:teuthology.orchestra.run.smithi032.stderr: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46419.log 2024-04-24T23:11:32.862 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.31708.log.gz 2024-04-24T23:11:32.862 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.63175.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.63175.log.gz 2024-04-24T23:11:32.862 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64990.log 2024-04-24T23:11:32.862 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49281.log 2024-04-24T23:11:32.862 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46419.log: /var/log/ceph/ceph-client.admin.64990.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46419.log.gz 2024-04-24T23:11:32.862 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.64990.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35039.log 2024-04-24T23:11:32.862 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-04-24T23:11:32.862 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49281.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49281.log.gz 2024-04-24T23:11:32.862 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.61605.log 2024-04-24T23:11:32.862 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68997.log 2024-04-24T23:11:32.862 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.35039.log: /var/log/ceph/ceph-client.0.61605.log: 0.0% 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35039.log.gz 2024-04-24T23:11:32.862 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.0.61605.log.gz 2024-04-24T23:11:32.863 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33389.log 2024-04-24T23:11:32.863 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41586.log 2024-04-24T23:11:32.863 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.68997.log: /var/log/ceph/ceph-client.admin.33389.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.68997.log.gz 2024-04-24T23:11:32.863 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33389.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59467.log 2024-04-24T23:11:32.863 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-04-24T23:11:32.863 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41586.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41586.log.gz 2024-04-24T23:11:32.863 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34913.log 2024-04-24T23:11:32.863 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59179.log 2024-04-24T23:11:32.863 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.59467.log: /var/log/ceph/ceph-client.admin.34913.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.59467.log.gz 2024-04-24T23:11:32.863 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44349.log 2024-04-24T23:11:32.863 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34913.log.gz 2024-04-24T23:11:32.863 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.59179.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59179.log.gz 2024-04-24T23:11:32.864 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62652.log 2024-04-24T23:11:32.864 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56283.log 2024-04-24T23:11:32.864 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44349.log: /var/log/ceph/ceph-client.admin.62652.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44349.log.gz 2024-04-24T23:11:32.864 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32016.log 2024-04-24T23:11:32.864 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.62652.log.gz 2024-04-24T23:11:32.864 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.56283.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.56283.log.gz 2024-04-24T23:11:32.864 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31372.log 2024-04-24T23:11:32.864 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37753.log 2024-04-24T23:11:32.864 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.32016.log: /var/log/ceph/ceph-client.admin.31372.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.32016.log.gz 2024-04-24T23:11:32.864 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.31372.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62384.log 2024-04-24T23:11:32.864 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-04-24T23:11:32.864 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37753.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37753.log.gz 2024-04-24T23:11:32.865 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62180.log 2024-04-24T23:11:32.865 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48015.log 2024-04-24T23:11:32.865 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.62384.log: /var/log/ceph/ceph-client.admin.62180.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.62384.log.gz 2024-04-24T23:11:32.865 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.62180.log.gz 2024-04-24T23:11:32.865 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32014.log 2024-04-24T23:11:32.865 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48015.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48015.log.gz 2024-04-24T23:11:32.865 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36781.log 2024-04-24T23:11:32.865 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37129.log 2024-04-24T23:11:32.865 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.32014.log: /var/log/ceph/ceph-client.admin.36781.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.32014.log.gz 2024-04-24T23:11:32.865 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50501.log 2024-04-24T23:11:32.865 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36781.log.gz 2024-04-24T23:11:32.865 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37129.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37129.log.gz 2024-04-24T23:11:32.866 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39047.log 2024-04-24T23:11:32.866 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53403.log 2024-04-24T23:11:32.866 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50501.log: /var/log/ceph/ceph-client.admin.39047.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50501.log.gz 2024-04-24T23:11:32.866 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.39047.log.gz 2024-04-24T23:11:32.866 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53232.log 2024-04-24T23:11:32.866 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.53403.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53403.log.gz 2024-04-24T23:11:32.866 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59693.log 2024-04-24T23:11:32.866 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45114.log 2024-04-24T23:11:32.866 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.53232.log: /var/log/ceph/ceph-client.admin.59693.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53232.log.gz 2024-04-24T23:11:32.866 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.59693.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.48267.log 2024-04-24T23:11:32.866 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-04-24T23:11:32.866 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45114.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45114.log.gz 2024-04-24T23:11:32.867 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36325.log 2024-04-24T23:11:32.867 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.67498.log 2024-04-24T23:11:32.867 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48267.log: /var/log/ceph/ceph-client.admin.36325.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48267.log.gz 2024-04-24T23:11:32.867 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36325.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54997.log 2024-04-24T23:11:32.867 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-04-24T23:11:32.867 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.0.67498.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.67498.log.gz 2024-04-24T23:11:32.867 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59819.log 2024-04-24T23:11:32.867 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42134.log 2024-04-24T23:11:32.867 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.54997.log: /var/log/ceph/ceph-client.admin.59819.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54997.log.gz 2024-04-24T23:11:32.867 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61333.log 2024-04-24T23:11:32.868 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.59819.log.gz 2024-04-24T23:11:32.868 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.0.42134.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.42134.log.gz 2024-04-24T23:11:32.868 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45578.log 2024-04-24T23:11:32.868 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40801.log 2024-04-24T23:11:32.868 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.61333.log: /var/log/ceph/ceph-client.0.45578.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.45578.log.gz 2024-04-24T23:11:32.868 INFO:teuthology.orchestra.run.smithi032.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.61333.log.gz 2024-04-24T23:11:32.868 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36241.log 2024-04-24T23:11:32.868 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41484.log 2024-04-24T23:11:32.868 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.40801.log: /var/log/ceph/ceph-client.admin.36241.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40801.log.gz 2024-04-24T23:11:32.868 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36241.log.gz 2024-04-24T23:11:32.868 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55601.log 2024-04-24T23:11:32.868 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.41484.log: -- /var/log/ceph/ceph-client.admin.46461.log 2024-04-24T23:11:32.869 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41484.log.gz 2024-04-24T23:11:32.869 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.55601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56680.log 2024-04-24T23:11:32.869 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46461.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55601.log.gz 2024-04-24T23:11:32.869 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46461.log.gz 2024-04-24T23:11:32.869 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32010.log 2024-04-24T23:11:32.869 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56638.log 2024-04-24T23:11:32.869 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.56680.log: /var/log/ceph/ceph-client.admin.32010.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.56680.log.gz 2024-04-24T23:11:32.869 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.32010.log.gz 2024-04-24T23:11:32.869 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69837.log 2024-04-24T23:11:32.869 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46193.log 2024-04-24T23:11:32.869 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.56638.log: /var/log/ceph/ceph-client.admin.69837.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56638.log.gz 2024-04-24T23:11:32.869 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65714.log 2024-04-24T23:11:32.869 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.69837.log.gz 2024-04-24T23:11:32.870 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46193.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46193.log.gz 2024-04-24T23:11:32.870 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31834.log 2024-04-24T23:11:32.870 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51777.log 2024-04-24T23:11:32.870 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.65714.log: /var/log/ceph/ceph-client.admin.31834.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.65714.log.gz 2024-04-24T23:11:32.870 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68292.log 2024-04-24T23:11:32.870 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.31834.log.gz 2024-04-24T23:11:32.870 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66992.log 2024-04-24T23:11:32.870 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.68292.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51777.log.gz 2024-04-24T23:11:32.870 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.68292.log.gz 2024-04-24T23:11:32.870 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38052.log 2024-04-24T23:11:32.870 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41838.log 2024-04-24T23:11:32.870 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.66992.log: /var/log/ceph/ceph-client.admin.38052.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.66992.log.gz 2024-04-24T23:11:32.871 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.38052.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.52282.log 2024-04-24T23:11:32.871 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-04-24T23:11:32.871 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66333.log 2024-04-24T23:11:32.871 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41838.log: /var/log/ceph/ceph-client.admin.52282.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41838.log.gz 2024-04-24T23:11:32.871 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49491.log 2024-04-24T23:11:32.871 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52282.log.gz 2024-04-24T23:11:32.871 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.66333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.34443.log 2024-04-24T23:11:32.871 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49491.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.66333.log.gz 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49491.log.gz 2024-04-24T23:11:32.871 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-04-24T23:11:32.871 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60216.log 2024-04-24T23:11:32.871 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53955.log 2024-04-24T23:11:32.871 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.2.34443.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.34443.log.gz 2024-04-24T23:11:32.872 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.60216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30972.log 2024-04-24T23:11:32.872 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.53955.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60216.log.gz 2024-04-24T23:11:32.872 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53955.log.gz 2024-04-24T23:11:32.872 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41796.log 2024-04-24T23:11:32.872 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.30972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60600.log 2024-04-24T23:11:32.872 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.30972.log.gz 2024-04-24T23:11:32.872 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41754.log 2024-04-24T23:11:32.872 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.60600.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41796.log.gz 2024-04-24T23:11:32.872 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.60600.log.gz 2024-04-24T23:11:32.872 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66237.log 2024-04-24T23:11:32.872 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63908.log 2024-04-24T23:11:32.872 INFO:teuthology.orchestra.run.smithi032.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41754.log.gz 2024-04-24T23:11:32.872 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.66237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45324.log 2024-04-24T23:11:32.873 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.63908.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.66237.log.gz 2024-04-24T23:11:32.873 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.63908.log.gz 2024-04-24T23:11:32.873 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63866.log 2024-04-24T23:11:32.873 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38787.log 2024-04-24T23:11:32.873 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45324.log: /var/log/ceph/ceph-client.admin.63866.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.63866.log.gz 2024-04-24T23:11:32.873 INFO:teuthology.orchestra.run.smithi032.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.45324.log.gz 2024-04-24T23:11:32.873 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59095.log 2024-04-24T23:11:32.873 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.0.38787.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.38787.log.gz 2024-04-24T23:11:32.873 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52116.log 2024-04-24T23:11:32.873 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41712.log 2024-04-24T23:11:32.873 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.59095.log: /var/log/ceph/ceph-client.admin.52116.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.59095.log.gz 2024-04-24T23:11:32.873 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52116.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64356.log 2024-04-24T23:11:32.873 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-04-24T23:11:32.874 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48534.log 2024-04-24T23:11:32.874 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41712.log: /var/log/ceph/ceph-client.admin.64356.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.64356.log.gz 2024-04-24T23:11:32.874 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41712.log.gz 2024-04-24T23:11:32.874 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47708.log 2024-04-24T23:11:32.874 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43124.log 2024-04-24T23:11:32.874 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48534.log.gz 2024-04-24T23:11:32.874 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68106.log 2024-04-24T23:11:32.874 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43124.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47708.log.gz 2024-04-24T23:11:32.874 INFO:teuthology.orchestra.run.smithi032.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43124.log.gz 2024-04-24T23:11:32.874 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57501.log 2024-04-24T23:11:32.874 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.68106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52528.log 2024-04-24T23:11:32.875 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.68106.log.gz 2024-04-24T23:11:32.875 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.57501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43680.log 2024-04-24T23:11:32.875 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52528.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.57501.log.gz 2024-04-24T23:11:32.875 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52528.log.gz 2024-04-24T23:11:32.875 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32686.log 2024-04-24T23:11:32.875 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47133.log 2024-04-24T23:11:32.875 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43680.log: /var/log/ceph/ceph-client.admin.32686.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43680.log.gz 2024-04-24T23:11:32.875 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.32686.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44287.log 2024-04-24T23:11:32.875 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-04-24T23:11:32.875 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51617.log 2024-04-24T23:11:32.875 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47133.log: /var/log/ceph/ceph-client.admin.44287.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47133.log.gz 2024-04-24T23:11:32.875 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44287.log.gz 2024-04-24T23:11:32.875 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34139.log 2024-04-24T23:11:32.876 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65943.log 2024-04-24T23:11:32.876 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51617.log.gz 2024-04-24T23:11:32.876 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.34139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55293.log 2024-04-24T23:11:32.876 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.65943.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34139.log.gz 2024-04-24T23:11:32.876 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65943.log.gz 2024-04-24T23:11:32.876 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49055.log 2024-04-24T23:11:32.876 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.55293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39584.log 2024-04-24T23:11:32.876 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55293.log.gz 2024-04-24T23:11:32.876 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56325.log 2024-04-24T23:11:32.876 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.39584.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49055.log.gz 2024-04-24T23:11:32.876 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39584.log.gz 2024-04-24T23:11:32.876 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55643.log 2024-04-24T23:11:32.877 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54136.log 2024-04-24T23:11:32.877 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.56325.log: /var/log/ceph/ceph-client.admin.55643.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56325.log.gz 2024-04-24T23:11:32.877 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55643.log.gz 2024-04-24T23:11:32.877 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50770.log 2024-04-24T23:11:32.877 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.54136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37213.log 2024-04-24T23:11:32.877 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54136.log.gz 2024-04-24T23:11:32.877 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65462.log 2024-04-24T23:11:32.877 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37213.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50770.log.gz 2024-04-24T23:11:32.877 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37213.log.gz 2024-04-24T23:11:32.877 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57190.log 2024-04-24T23:11:32.877 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58579.log 2024-04-24T23:11:32.877 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.65462.log: /var/log/ceph/ceph-client.admin.57190.log: 93.0% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-24T23:11:32.877 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.65462.log.gz 2024-04-24T23:11:32.877 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57190.log.gz 2024-04-24T23:11:32.878 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45282.log 2024-04-24T23:11:32.878 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35941.log 2024-04-24T23:11:32.878 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.58579.log: /var/log/ceph/ceph-client.admin.45282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65756.log 2024-04-24T23:11:32.878 INFO:teuthology.orchestra.run.smithi032.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.58579.log.gz 2024-04-24T23:11:32.878 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.35941.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45282.log.gz 2024-04-24T23:11:32.878 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35941.log.gz 2024-04-24T23:11:32.878 INFO:teuthology.orchestra.run.smithi032.stderr: --verbose -- /var/log/ceph/ceph-client.admin.43764.log 2024-04-24T23:11:32.878 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.65756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57543.log 2024-04-24T23:11:32.878 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.65756.log.gz 2024-04-24T23:11:32.878 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55931.log 2024-04-24T23:11:32.878 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8%/var/log/ceph/ceph-client.admin.57543.log: -- replaced with /var/log/ceph/ceph-client.admin.43764.log.gz 2024-04-24T23:11:32.878 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36823.log 2024-04-24T23:11:32.878 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57543.log.gz 2024-04-24T23:11:32.879 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.55931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67619.log 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55931.log.gz 2024-04-24T23:11:32.879 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-04-24T23:11:32.879 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.36823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65798.log 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36823.log.gz 2024-04-24T23:11:32.879 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-04-24T23:11:32.879 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.67619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66950.log 2024-04-24T23:11:32.879 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.67619.log.gz/var/log/ceph/ceph-client.admin.65798.log: 2024-04-24T23:11:32.879 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56031.log 2024-04-24T23:11:32.879 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.65798.log.gz 2024-04-24T23:11:32.879 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.66950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32968.log 2024-04-24T23:11:32.879 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.66950.log.gz/var/log/ceph/ceph-client.admin.56031.log: 2024-04-24T23:11:32.879 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62484.log 2024-04-24T23:11:32.879 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.56031.log.gz 2024-04-24T23:11:32.880 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.32968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42445.log 2024-04-24T23:11:32.880 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.32968.log.gz 2024-04-24T23:11:32.880 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67118.log 2024-04-24T23:11:32.880 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.62484.log: /var/log/ceph/ceph-client.admin.42445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65074.log 2024-04-24T23:11:32.880 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.67118.log: 87.7% 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62484.log.gz 2024-04-24T23:11:32.880 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42445.log.gz 2024-04-24T23:11:32.880 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.67118.log.gz 2024-04-24T23:11:32.880 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39746.log 2024-04-24T23:11:32.880 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.65074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60174.log 2024-04-24T23:11:32.880 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.39746.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.65074.log.gz 2024-04-24T23:11:32.880 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51906.log 2024-04-24T23:11:32.880 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46881.log 2024-04-24T23:11:32.881 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0%/var/log/ceph/ceph-client.admin.60174.log: -- replaced with /var/log/ceph/ceph-client.admin.39746.log.gz 2024-04-24T23:11:32.881 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31228.log 2024-04-24T23:11:32.881 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.60174.log.gz 2024-04-24T23:11:32.881 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51906.log.gz/var/log/ceph/ceph-client.admin.46881.log: 2024-04-24T23:11:32.881 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35983.log 2024-04-24T23:11:32.881 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.31228.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46881.log.gz 2024-04-24T23:11:32.881 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48057.log 2024-04-24T23:11:32.881 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.31228.log.gz 2024-04-24T23:11:32.881 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42562.log 2024-04-24T23:11:32.881 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.35983.log: /var/log/ceph/ceph-client.admin.48057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45947.log 2024-04-24T23:11:32.881 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42562.log: 88.2% 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48057.log.gz 2024-04-24T23:11:32.881 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35983.log.gz 2024-04-24T23:11:32.882 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35779.log 2024-04-24T23:11:32.882 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42562.log.gz 2024-04-24T23:11:32.882 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45699.log 2024-04-24T23:11:32.882 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- 87.7% /var/log/ceph/ceph-client.admin.59053.log 2024-04-24T23:11:32.882 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45947.log.gz 2024-04-24T23:11:32.882 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.35779.log: gzip/var/log/ceph/ceph-client.admin.45699.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.57413.log 2024-04-24T23:11:32.882 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.59053.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45699.log.gz 2024-04-24T23:11:32.882 INFO:teuthology.orchestra.run.smithi032.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35779.log.gz 2024-04-24T23:11:32.882 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.59053.log.gz 2024-04-24T23:11:32.882 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34829.log 2024-04-24T23:11:32.882 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.57413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62426.log 2024-04-24T23:11:32.882 INFO:teuthology.orchestra.run.smithi032.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.57413.log.gz 2024-04-24T23:11:32.883 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.34829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54814.log 2024-04-24T23:11:32.883 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34829.log.gz 2024-04-24T23:11:32.883 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.62426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58537.log 2024-04-24T23:11:32.883 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.62426.log.gz 2024-04-24T23:11:32.883 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.0.54814.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.54814.log.gz 2024-04-24T23:11:32.883 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59425.log 2024-04-24T23:11:32.883 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.58537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69333.log 2024-04-24T23:11:32.883 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58537.log.gz 2024-04-24T23:11:32.883 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.59425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59525.log 87.8% -- replaced with /var/log/ceph/ceph-client.admin.59425.log.gz 2024-04-24T23:11:32.883 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-04-24T23:11:32.883 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.69333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58863.log 2024-04-24T23:11:32.883 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.69333.log.gz 2024-04-24T23:11:32.884 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.59525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64642.log 2024-04-24T23:11:32.884 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.59525.log.gz 2024-04-24T23:11:32.884 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.58863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63685.log 2024-04-24T23:11:32.884 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34745.log 2024-04-24T23:11:32.884 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58863.log.gz 2024-04-24T23:11:32.884 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.64642.log: /var/log/ceph/ceph-client.admin.63685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52880.log 2024-04-24T23:11:32.884 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.34745.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.64642.log.gz 2024-04-24T23:11:32.884 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.63685.log.gz 2024-04-24T23:11:32.884 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60342.log 2024-04-24T23:11:32.884 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34745.log.gz 2024-04-24T23:11:32.884 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62962.log 2024-04-24T23:11:32.884 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52880.log: /var/log/ceph/ceph-client.admin.60342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46923.log 2024-04-24T23:11:32.885 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.62962.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52880.log.gz 2024-04-24T23:11:32.885 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60342.log.gz 2024-04-24T23:11:32.885 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.62962.log.gzgzip 2024-04-24T23:11:32.885 INFO:teuthology.orchestra.run.smithi032.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.59383.log 2024-04-24T23:11:32.885 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36067.log 2024-04-24T23:11:32.885 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46545.log 2024-04-24T23:11:32.885 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.59383.log: 88.1%/var/log/ceph/ceph-client.admin.36067.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46923.log.gz 2024-04-24T23:11:32.885 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.59383.log.gz 2024-04-24T23:11:32.885 INFO:teuthology.orchestra.run.smithi032.stderr:gzip 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36067.log.gz 2024-04-24T23:11:32.885 INFO:teuthology.orchestra.run.smithi032.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.37711.log 2024-04-24T23:11:32.885 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64122.log 2024-04-24T23:11:32.885 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46545.log.gz 2024-04-24T23:11:32.886 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57687.log 88.8% 2024-04-24T23:11:32.886 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37711.log.gz 2024-04-24T23:11:32.886 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.64122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36199.log 2024-04-24T23:11:32.886 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.64122.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45989.log 2024-04-24T23:11:32.886 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-04-24T23:11:32.886 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.57687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61684.log 2024-04-24T23:11:32.886 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.36199.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57687.log.gz/var/log/ceph/ceph-client.admin.45989.log: 2024-04-24T23:11:32.886 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36199.log.gz 2024-04-24T23:11:32.886 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45989.log.gz 2024-04-24T23:11:32.886 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.34003.log 2024-04-24T23:11:32.887 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.61684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46277.log 2024-04-24T23:11:32.887 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9%/var/log/ceph/ceph-client.0.34003.log: -- replaced with /var/log/ceph/ceph-client.admin.61684.log.gz 2024-04-24T23:11:32.887 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.34003.log.gz 2024-04-24T23:11:32.887 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65546.log 2024-04-24T23:11:32.887 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33178.log 2024-04-24T23:11:32.887 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53529.log 2024-04-24T23:11:32.887 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.65546.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46277.log.gz 2024-04-24T23:11:32.887 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.33178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65420.log 2024-04-24T23:11:32.887 INFO:teuthology.orchestra.run.smithi032.stderr: 87.5% 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33178.log.gz 2024-04-24T23:11:32.887 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.65546.log.gz 2024-04-24T23:11:32.888 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.53529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40434.log 2024-04-24T23:11:32.888 INFO:teuthology.orchestra.run.smithi032.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53529.log.gz 2024-04-24T23:11:32.888 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43638.log 2024-04-24T23:11:32.888 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.65420.log: /var/log/ceph/ceph-client.admin.40434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48767.log 2024-04-24T23:11:32.888 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40434.log.gz 2024-04-24T23:11:32.888 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43638.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65420.log.gz 2024-04-24T23:11:32.888 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- 88.9% /var/log/ceph/ceph-client.admin.36571.log 2024-04-24T23:11:32.888 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43638.log.gz 2024-04-24T23:11:32.888 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53913.log 2024-04-24T23:11:32.888 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3%/var/log/ceph/ceph-client.admin.36571.log: -- replaced with /var/log/ceph/ceph-client.admin.48767.log.gz 2024-04-24T23:11:32.888 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65116.log 2024-04-24T23:11:32.888 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36571.log.gz 2024-04-24T23:11:32.888 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58760.log 2024-04-24T23:11:32.889 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.53913.log: /var/log/ceph/ceph-client.admin.65116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30887.log 2024-04-24T23:11:32.889 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.58760.log: 88.3% 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53913.log.gz 2024-04-24T23:11:32.889 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.65116.log.gz 2024-04-24T23:11:32.889 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58760.log.gz 2024-04-24T23:11:32.889 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56157.log 2024-04-24T23:11:32.889 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.30887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49846.log 2024-04-24T23:11:32.889 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.56157.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.30887.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70089.log 2024-04-24T23:11:32.889 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-04-24T23:11:32.889 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56157.log.gz 2024-04-24T23:11:32.889 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54085.log 2024-04-24T23:11:32.889 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49846.log: /var/log/ceph/ceph-client.admin.70089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39788.log 2024-04-24T23:11:32.889 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49846.log.gz 2024-04-24T23:11:32.889 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.54085.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.70089.log.gz 2024-04-24T23:11:32.890 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.54085.log.gz 2024-04-24T23:11:32.890 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38599.log 2024-04-24T23:11:32.890 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.39788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33653.log 2024-04-24T23:11:32.890 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39788.log.gz 2024-04-24T23:11:32.890 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50812.log 2024-04-24T23:11:32.890 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.38599.log: /var/log/ceph/ceph-client.admin.33653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44647.log 2024-04-24T23:11:32.890 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50812.log: 87.7% 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33653.log.gz 2024-04-24T23:11:32.890 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38599.log.gz 2024-04-24T23:11:32.890 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50812.log.gz 2024-04-24T23:11:32.890 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61123.log 2024-04-24T23:11:32.890 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63007.log 2024-04-24T23:11:32.890 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49533.log 2024-04-24T23:11:32.891 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.61123.log: 88.9%/var/log/ceph/ceph-client.admin.63007.log: -- replaced with /var/log/ceph/ceph-client.admin.44647.log.gz 2024-04-24T23:11:32.891 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 88.9% -- replaced with /var/log/ceph/ceph-client.admin.63007.log.gz 2024-04-24T23:11:32.891 INFO:teuthology.orchestra.run.smithi032.stderr: --verbose -- /var/log/ceph/ceph-client.admin.32015.log 2024-04-24T23:11:32.891 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49533.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61123.log.gz 2024-04-24T23:11:32.891 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50686.log 2024-04-24T23:11:32.891 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49533.log.gz 2024-04-24T23:11:32.891 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47578.log 2024-04-24T23:11:32.891 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.32015.log: /var/log/ceph/ceph-client.admin.50686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36529.log 2024-04-24T23:11:32.891 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47578.log: 88.0% 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50686.log.gz 2024-04-24T23:11:32.891 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.32015.log.gz 2024-04-24T23:11:32.891 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47578.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67968.log 2024-04-24T23:11:32.891 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-04-24T23:11:32.892 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53006.log 2024-04-24T23:11:32.892 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.36529.log: /var/log/ceph/ceph-client.admin.67968.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36529.log.gz 2024-04-24T23:11:32.892 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44988.log 2024-04-24T23:11:32.892 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.67968.log.gz 2024-04-24T23:11:32.892 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.53006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52696.log 2024-04-24T23:11:32.892 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56241.log 2024-04-24T23:11:32.892 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1%/var/log/ceph/ceph-client.admin.44988.log: -- replaced with /var/log/ceph/ceph-client.admin.53006.log.gz 2024-04-24T23:11:32.892 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52696.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44988.log.gz 2024-04-24T23:11:32.892 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65840.log 2024-04-24T23:11:32.892 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52696.log.gz 2024-04-24T23:11:32.892 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5/var/log/ceph/ceph-client.admin.56241.log: --verbose -- /var/log/ceph/ceph-client.admin.41270.log 2024-04-24T23:11:32.892 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.65840.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.56241.log.gz 2024-04-24T23:11:32.892 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32012.log 2024-04-24T23:11:32.893 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.65840.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32680.log 2024-04-24T23:11:32.893 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-04-24T23:11:32.893 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41270.log: /var/log/ceph/ceph-client.admin.32012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43890.log 2024-04-24T23:11:32.893 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.32680.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.32012.log.gz 2024-04-24T23:11:32.893 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41270.log.gz 2024-04-24T23:11:32.893 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 89.0% -- replaced with /var/log/ceph/ceph-client.admin.32680.log.gz 2024-04-24T23:11:32.893 INFO:teuthology.orchestra.run.smithi032.stderr: --verbose -- /var/log/ceph/ceph-client.admin.33052.log 2024-04-24T23:11:32.893 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65630.log 2024-04-24T23:11:32.893 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43890.log.gz 2024-04-24T23:11:32.893 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.33052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32682.log 2024-04-24T23:11:32.893 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.33052.log.gz/var/log/ceph/ceph-client.admin.65630.log: 2024-04-24T23:11:32.893 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35252.log 2024-04-24T23:11:32.894 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.65630.log.gz 2024-04-24T23:11:32.894 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51735.log 2024-04-24T23:11:32.894 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.32682.log: /var/log/ceph/ceph-client.admin.35252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46671.log 2024-04-24T23:11:32.894 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51735.log: 87.9% 88.9% -- replaced with /var/log/ceph/ceph-client.admin.32682.log.gz 2024-04-24T23:11:32.894 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35252.log.gz 2024-04-24T23:11:32.894 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51735.log.gz 2024-04-24T23:11:32.894 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66579.log 2024-04-24T23:11:32.894 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56367.log 2024-04-24T23:11:32.894 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46671.log: /var/log/ceph/ceph-client.admin.66579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40350.log 2024-04-24T23:11:32.894 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.56367.log: 88.9% 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46671.log.gz 2024-04-24T23:11:32.894 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.66579.log.gz 2024-04-24T23:11:32.894 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56367.log.gz 2024-04-24T23:11:32.895 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53747.log 2024-04-24T23:11:32.895 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50640.log 2024-04-24T23:11:32.895 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.40350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48201.log 2024-04-24T23:11:32.895 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.53747.log: 88.7% 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53747.log.gz 2024-04-24T23:11:32.895 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50640.log: -- replaced with /var/log/ceph/ceph-client.admin.40350.log.gz 2024-04-24T23:11:32.895 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50640.log.gz 2024-04-24T23:11:32.895 INFO:teuthology.orchestra.run.smithi032.stderr: --verbose -- /var/log/ceph/ceph-client.admin.30619.log 2024-04-24T23:11:32.895 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69921.log 2024-04-24T23:11:32.895 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.30619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43454.log 2024-04-24T23:11:32.895 INFO:teuthology.orchestra.run.smithi032.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.48201.log.gz 2024-04-24T23:11:32.895 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.30619.log.gz 2024-04-24T23:11:32.895 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.69921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43166.log 2024-04-24T23:11:32.895 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49617.log 2024-04-24T23:11:32.896 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.69921.log.gz 2024-04-24T23:11:32.896 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43166.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43454.log.gz 2024-04-24T23:11:32.896 INFO:teuthology.orchestra.run.smithi032.stderr:gzip 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43166.log.gz 2024-04-24T23:11:32.896 INFO:teuthology.orchestra.run.smithi032.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.33010.log 2024-04-24T23:11:32.896 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65362.log 2024-04-24T23:11:32.896 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49617.log.gz 2024-04-24T23:11:32.896 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.33010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58718.log 2024-04-24T23:11:32.896 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33010.log.gz 2024-04-24T23:11:32.896 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.65362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42403.log 2024-04-24T23:11:32.896 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.65362.log.gz 2024-04-24T23:11:32.896 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.58718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39089.log 2024-04-24T23:11:32.896 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58718.log.gz 2024-04-24T23:11:32.896 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37373.log 2024-04-24T23:11:32.897 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62012.log 2024-04-24T23:11:32.897 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.39089.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42403.log.gz 2024-04-24T23:11:32.897 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34661.log 2024-04-24T23:11:32.897 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39089.log.gz 2024-04-24T23:11:32.897 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.62012.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37373.log.gz 2024-04-24T23:11:32.897 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62012.log.gz 2024-04-24T23:11:32.897 INFO:teuthology.orchestra.run.smithi032.stderr: --verbose -- /var/log/ceph/ceph-client.admin.49762.log 2024-04-24T23:11:32.897 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.34661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53997.log 2024-04-24T23:11:32.897 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34661.log.gz 2024-04-24T23:11:32.897 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64248.log 2024-04-24T23:11:32.897 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49762.log.gz 2024-04-24T23:11:32.897 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.53997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62300.log 2024-04-24T23:11:32.898 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53997.log.gz 2024-04-24T23:11:32.898 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46839.log 2024-04-24T23:11:32.898 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.64248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69753.log 2024-04-24T23:11:32.898 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.62300.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.64248.log.gz 2024-04-24T23:11:32.898 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46839.log: gzip -5 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46839.log.gz 2024-04-24T23:11:32.898 INFO:teuthology.orchestra.run.smithi032.stderr: --verbose -- /var/log/ceph/ceph-client.admin.68955.log 2024-04-24T23:11:32.898 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.69753.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.62300.log.gz 2024-04-24T23:11:32.898 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-24T23:11:32.898 INFO:teuthology.orchestra.run.smithi032.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.69753.log.gz 2024-04-24T23:11:32.898 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44862.log 2024-04-24T23:11:32.898 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.68955.log: /var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48851.log 2024-04-24T23:11:32.899 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44862.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.68955.log.gz 2024-04-24T23:11:32.899 INFO:teuthology.orchestra.run.smithi032.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44862.log.gz 2024-04-24T23:11:32.899 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68729.log 2024-04-24T23:11:32.899 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50171.log 2024-04-24T23:11:32.899 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48851.log: /var/log/ceph/ceph-client.admin.68729.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48851.log.gz 2024-04-24T23:11:32.899 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41400.log 2024-04-24T23:11:32.899 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.68729.log.gz 2024-04-24T23:11:32.899 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54518.log 2024-04-24T23:11:32.899 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41400.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50171.log.gz 2024-04-24T23:11:32.899 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41400.log.gz 2024-04-24T23:11:32.899 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40056.log 2024-04-24T23:11:32.899 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50854.log 2024-04-24T23:11:32.900 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.54518.log: /var/log/ceph/ceph-client.admin.40056.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54518.log.gz 2024-04-24T23:11:32.900 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40056.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42255.log 2024-04-24T23:11:32.900 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-04-24T23:11:32.900 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.64562.log 2024-04-24T23:11:32.900 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50854.log: /var/log/ceph/ceph-client.admin.42255.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50854.log.gz 2024-04-24T23:11:32.900 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41175.log 2024-04-24T23:11:32.900 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42255.log.gz 2024-04-24T23:11:32.900 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.0.64562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60685.log 2024-04-24T23:11:32.900 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41175.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.64562.log.gz 2024-04-24T23:11:32.900 INFO:teuthology.orchestra.run.smithi032.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41175.log.gz 2024-04-24T23:11:32.900 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37543.log 2024-04-24T23:11:32.900 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43082.log 2024-04-24T23:11:32.901 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37543.log: /var/log/ceph/ceph-client.admin.60685.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37543.log.gz 88.3% -- replaced with /var/log/ceph/ceph-client.admin.60685.log.gz 2024-04-24T23:11:32.901 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-04-24T23:11:32.901 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47931.log 2024-04-24T23:11:32.901 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62736.log 2024-04-24T23:11:32.901 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43082.log: /var/log/ceph/ceph-client.admin.47931.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43082.log.gz 2024-04-24T23:11:32.901 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47931.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44203.log 2024-04-24T23:11:32.901 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-04-24T23:11:32.901 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.62736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31186.log 2024-04-24T23:11:32.901 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44203.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44203.log.gz 2024-04-24T23:11:32.901 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.62736.log.gz 2024-04-24T23:11:32.901 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68913.log 2024-04-24T23:11:32.901 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.31186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42790.log 2024-04-24T23:11:32.902 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.31186.log.gz/var/log/ceph/ceph-client.admin.68913.log: 2024-04-24T23:11:32.902 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40266.log 2024-04-24T23:11:32.902 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42790.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.68913.log.gz 2024-04-24T23:11:32.902 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42790.log.gz 2024-04-24T23:11:32.902 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32348.log 2024-04-24T23:11:32.902 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50545.log 2024-04-24T23:11:32.902 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.40266.log: /var/log/ceph/ceph-client.admin.32348.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40266.log.gz 2024-04-24T23:11:32.902 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.32348.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31014.log 2024-04-24T23:11:32.902 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-04-24T23:11:32.902 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59137.log 2024-04-24T23:11:32.902 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50545.log: /var/log/ceph/ceph-client.admin.31014.log: 87.9% 88.2% -- replaced with /var/log/ceph/ceph-client.admin.31014.log.gz 2024-04-24T23:11:32.903 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50545.log.gz 2024-04-24T23:11:32.903 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60090.log 2024-04-24T23:11:32.903 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.59137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36739.log 2024-04-24T23:11:32.903 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59137.log.gz 2024-04-24T23:11:32.903 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.60090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66111.log 2024-04-24T23:11:32.903 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.36739.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.60090.log.gz 2024-04-24T23:11:32.903 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36739.log.gz 2024-04-24T23:11:32.903 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46377.log 2024-04-24T23:11:32.903 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.66111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43806.log 2024-04-24T23:11:32.903 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.66111.log.gz 2024-04-24T23:11:32.903 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63133.log 2024-04-24T23:11:32.903 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43806.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46377.log.gz 2024-04-24T23:11:32.904 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43806.log.gz 2024-04-24T23:11:32.904 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31270.log 2024-04-24T23:11:32.904 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41526.log 2024-04-24T23:11:32.904 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.63133.log: /var/log/ceph/ceph-client.admin.31270.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.63133.log.gz 2024-04-24T23:11:32.904 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.31270.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60438.log 2024-04-24T23:11:32.904 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-04-24T23:11:32.904 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61165.log 2024-04-24T23:11:32.904 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41526.log: /var/log/ceph/ceph-client.admin.60438.log: 88.3% 88.3% -- replaced with /var/log/ceph/ceph-client.admin.60438.log.gz 2024-04-24T23:11:32.904 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41526.log.gz 2024-04-24T23:11:32.904 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32685.log 2024-04-24T23:11:32.905 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.61165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49930.log 2024-04-24T23:11:32.905 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.61165.log.gz 2024-04-24T23:11:32.905 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.32685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47410.log 2024-04-24T23:11:32.905 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49930.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.32685.log.gz 2024-04-24T23:11:32.905 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49930.log.gz 2024-04-24T23:11:32.905 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64080.log 2024-04-24T23:11:32.905 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50374.log 2024-04-24T23:11:32.905 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47410.log: /var/log/ceph/ceph-client.admin.64080.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47410.log.gz 2024-04-24T23:11:32.905 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.64080.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53174.log 2024-04-24T23:11:32.905 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-04-24T23:11:32.905 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68482.log 2024-04-24T23:11:32.906 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.53174.log: /var/log/ceph/ceph-client.admin.50374.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53174.log.gz 2024-04-24T23:11:32.906 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50374.log.gz 2024-04-24T23:11:32.906 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66802.log 2024-04-24T23:11:32.906 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.68482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40476.log 2024-04-24T23:11:32.906 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.68482.log.gz 2024-04-24T23:11:32.906 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.66802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57064.log 2024-04-24T23:11:32.906 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.40476.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.66802.log.gz 2024-04-24T23:11:32.906 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40476.log.gz 2024-04-24T23:11:32.906 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35863.log 2024-04-24T23:11:32.906 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50014.log 2024-04-24T23:11:32.906 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.57064.log: /var/log/ceph/ceph-client.admin.35863.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.57064.log.gz 2024-04-24T23:11:32.906 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.35863.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41628.log 2024-04-24T23:11:32.906 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-04-24T23:11:32.907 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61399.log 2024-04-24T23:11:32.907 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41628.log: /var/log/ceph/ceph-client.admin.50014.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50014.log.gz 2024-04-24T23:11:32.907 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41628.log.gz 2024-04-24T23:11:32.907 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30106.log 2024-04-24T23:11:32.907 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.61399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35209.log 2024-04-24T23:11:32.907 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.61399.log.gz/var/log/ceph/ceph-client.admin.30106.log: 2024-04-24T23:11:32.907 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39930.log 2024-04-24T23:11:32.907 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.35209.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.30106.log.gz 2024-04-24T23:11:32.907 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35209.log.gz 2024-04-24T23:11:32.907 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63558.log 2024-04-24T23:11:32.907 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.39930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49804.log 2024-04-24T23:11:32.907 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39930.log.gz 2024-04-24T23:11:32.908 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.63558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41004.log 2024-04-24T23:11:32.908 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49804.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.63558.log.gz 2024-04-24T23:11:32.908 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49804.log.gz 2024-04-24T23:11:32.908 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41354.log 2024-04-24T23:11:32.908 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32336.log 2024-04-24T23:11:32.908 INFO:teuthology.orchestra.run.smithi032.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.41004.log.gz 2024-04-24T23:11:32.908 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46797.log 2024-04-24T23:11:32.908 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.32336.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41354.log.gz 2024-04-24T23:11:32.908 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.32336.log.gz 2024-04-24T23:11:32.908 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67893.log 2024-04-24T23:11:32.908 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56722.log 2024-04-24T23:11:32.908 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46797.log: /var/log/ceph/ceph-client.admin.67893.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46797.log.gz 2024-04-24T23:11:32.909 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.67893.log.gz 2024-04-24T23:11:32.909 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57148.log 2024-04-24T23:11:32.909 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35336.log 2024-04-24T23:11:32.909 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.56722.log: /var/log/ceph/ceph-client.admin.57148.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56722.log.gz 2024-04-24T23:11:32.909 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57148.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46965.log 2024-04-24T23:11:32.909 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-04-24T23:11:32.909 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.35336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59567.log 2024-04-24T23:11:32.909 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35336.log.gz 2024-04-24T23:11:32.909 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59609.log 2024-04-24T23:11:32.909 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.59567.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46965.log.gz 2024-04-24T23:11:32.909 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.59567.log.gz 2024-04-24T23:11:32.909 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70047.log 2024-04-24T23:11:32.910 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.59609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47452.log 2024-04-24T23:11:32.910 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59609.log.gz 2024-04-24T23:11:32.910 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.70047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35821.log 2024-04-24T23:11:32.910 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47452.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.70047.log.gz 2024-04-24T23:11:32.910 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47452.log.gz 2024-04-24T23:11:32.910 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66453.log 2024-04-24T23:11:32.910 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65236.log 2024-04-24T23:11:32.910 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.35821.log: /var/log/ceph/ceph-client.admin.66453.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35821.log.gz 2024-04-24T23:11:32.910 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66453.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63642.log 2024-04-24T23:11:32.910 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-04-24T23:11:32.910 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54266.log 2024-04-24T23:11:32.911 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.63642.log: /var/log/ceph/ceph-client.admin.65236.log: 88.5% 88.7% -- replaced with /var/log/ceph/ceph-client.admin.63642.log.gz 2024-04-24T23:11:32.911 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.65236.log.gz 2024-04-24T23:11:32.911 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44035.log 2024-04-24T23:11:32.911 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54893.log 2024-04-24T23:11:32.911 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.54266.log: /var/log/ceph/ceph-client.admin.44035.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54266.log.gz 2024-04-24T23:11:32.911 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36109.log 2024-04-24T23:11:32.911 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.54893.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44035.log.gz 2024-04-24T23:11:32.911 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39228.log 2024-04-24T23:11:32.911 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.54893.log.gz 2024-04-24T23:11:32.911 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.36109.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36109.log.gz 2024-04-24T23:11:32.911 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58250.log 2024-04-24T23:11:32.911 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32681.log 2024-04-24T23:11:32.912 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.39228.log: /var/log/ceph/ceph-client.admin.58250.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39228.log.gz 2024-04-24T23:11:32.912 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58250.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37926.log 2024-04-24T23:11:32.912 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-04-24T23:11:32.912 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41131.log 2024-04-24T23:11:32.912 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.32681.log: /var/log/ceph/ceph-client.admin.37926.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.32681.log.gz 2024-04-24T23:11:32.912 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66886.log 2024-04-24T23:11:32.912 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37926.log.gz 2024-04-24T23:11:32.912 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41219.log 2024-04-24T23:11:32.912 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.66886.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41131.log.gz 2024-04-24T23:11:32.912 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.66886.log.gz 2024-04-24T23:11:32.912 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44946.log 2024-04-24T23:11:32.912 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43040.log 2024-04-24T23:11:32.913 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41219.log: /var/log/ceph/ceph-client.admin.44946.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41219.log.gz 2024-04-24T23:11:32.913 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44946.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35584.log 2024-04-24T23:11:32.913 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-04-24T23:11:32.913 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62054.log 2024-04-24T23:11:32.913 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43040.log: /var/log/ceph/ceph-client.admin.35584.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35584.log.gz 2024-04-24T23:11:32.913 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43040.log.gz 2024-04-24T23:11:32.913 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70257.log 2024-04-24T23:11:32.913 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.62054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34577.log 2024-04-24T23:11:32.913 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.62054.log.gz 2024-04-24T23:11:32.913 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.70257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56115.log 2024-04-24T23:11:32.913 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.34577.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.70257.log.gz 2024-04-24T23:11:32.913 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34577.log.gz 2024-04-24T23:11:32.913 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36866.log 2024-04-24T23:11:32.914 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66495.log 2024-04-24T23:11:32.914 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.56115.log: /var/log/ceph/ceph-client.admin.36866.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56115.log.gz 2024-04-24T23:11:32.914 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38491.log 2024-04-24T23:11:32.914 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6%/var/log/ceph/ceph-client.admin.66495.log: -- replaced with /var/log/ceph/ceph-client.admin.36866.log.gz 2024-04-24T23:11:32.914 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66495.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35695.log 2024-04-24T23:11:32.914 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-04-24T23:11:32.914 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65588.log 2024-04-24T23:11:32.914 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.38491.log: /var/log/ceph/ceph-client.admin.35695.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35695.log.gz 2024-04-24T23:11:32.914 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38491.log.gz 2024-04-24T23:11:32.914 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61207.log 2024-04-24T23:11:32.914 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.65588.log: -- /var/log/ceph/ceph-client.admin.60909.log 2024-04-24T23:11:32.915 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.65588.log.gz 2024-04-24T23:11:32.915 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.61207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59299.log 2024-04-24T23:11:32.915 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.60909.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.61207.log.gz 2024-04-24T23:11:32.915 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60909.log.gz 2024-04-24T23:11:32.915 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48809.log 2024-04-24T23:11:32.915 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.59299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66663.log 2024-04-24T23:11:32.915 INFO:teuthology.orchestra.run.smithi032.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.59299.log.gz 2024-04-24T23:11:32.915 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48809.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48809.log.gz 2024-04-24T23:11:32.915 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.66663.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.66663.log.gz 2024-04-24T23:11:32.916 INFO:teuthology.orchestra.run.smithi121.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-24T23:11:33.228 INFO:teuthology.orchestra.run.smithi121.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-24T23:11:33.236 INFO:teuthology.orchestra.run.smithi032.stderr: 93.9% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-24T23:11:33.239 INFO:teuthology.orchestra.run.smithi121.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-24T23:11:33.256 INFO:teuthology.orchestra.run.smithi032.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-24T23:11:33.313 INFO:teuthology.orchestra.run.smithi121.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-24T23:11:33.322 INFO:teuthology.orchestra.run.smithi032.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-24T23:11:33.375 INFO:teuthology.orchestra.run.smithi121.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-24T23:11:33.377 INFO:teuthology.orchestra.run.smithi121.stderr: 2024-04-24T23:11:33.377 INFO:teuthology.orchestra.run.smithi121.stderr:real 0m1.002s 2024-04-24T23:11:33.377 INFO:teuthology.orchestra.run.smithi121.stderr:user 0m4.486s 2024-04-24T23:11:33.377 INFO:teuthology.orchestra.run.smithi121.stderr:sys 0m0.185s 2024-04-24T23:11:33.469 INFO:teuthology.orchestra.run.smithi032.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-24T23:11:33.490 INFO:teuthology.orchestra.run.smithi032.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-24T23:11:33.506 INFO:teuthology.orchestra.run.smithi032.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-24T23:11:33.508 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-04-24T23:11:33.509 INFO:teuthology.orchestra.run.smithi032.stderr:real 0m1.155s 2024-04-24T23:11:33.509 INFO:teuthology.orchestra.run.smithi032.stderr:user 0m5.698s 2024-04-24T23:11:33.509 INFO:teuthology.orchestra.run.smithi032.stderr:sys 0m0.627s 2024-04-24T23:11:33.509 INFO:tasks.ceph:Archiving logs... 2024-04-24T23:11:33.509 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672173/remote/smithi032/log 2024-04-24T23:11:33.510 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-24T23:11:34.804 DEBUG:teuthology.misc:Transferring archived files from smithi121:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672173/remote/smithi121/log 2024-04-24T23:11:34.805 DEBUG:teuthology.orchestra.run.smithi121:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-24T23:11:35.421 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-24T23:11:35.431 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-24T23:11:35.431 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-24T23:11:35.434 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-24T23:11:35.535 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-24T23:11:35.545 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-24T23:11:35.545 DEBUG:teuthology.orchestra.run.smithi032:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T23:11:35.548 DEBUG:teuthology.orchestra.run.smithi121:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T23:11:35.563 INFO:teuthology.orchestra.run.smithi032.stderr:bash: line 1: ntpq: command not found 2024-04-24T23:11:35.564 INFO:teuthology.orchestra.run.smithi121.stderr:bash: line 1: ntpq: command not found 2024-04-24T23:11:35.568 INFO:teuthology.orchestra.run.smithi032.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T23:11:35.568 INFO:teuthology.orchestra.run.smithi032.stdout:=============================================================================== 2024-04-24T23:11:35.569 INFO:teuthology.orchestra.run.smithi032.stdout:^- hv01.front.sepia.ceph.com 2 6 377 33 +1279us[+1279us] +/- 570ms 2024-04-24T23:11:35.569 INFO:teuthology.orchestra.run.smithi032.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 34 +74us[ +74us] +/- 53ms 2024-04-24T23:11:35.569 INFO:teuthology.orchestra.run.smithi032.stdout:^* hv03.front.sepia.ceph.com 3 6 377 34 -60us[ -46us] +/- 61ms 2024-04-24T23:11:35.569 INFO:teuthology.orchestra.run.smithi032.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:11:35.570 INFO:teuthology.orchestra.run.smithi121.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T23:11:35.570 INFO:teuthology.orchestra.run.smithi121.stdout:=============================================================================== 2024-04-24T23:11:35.570 INFO:teuthology.orchestra.run.smithi121.stdout:^- hv01.front.sepia.ceph.com 2 6 377 33 +1262us[+1262us] +/- 570ms 2024-04-24T23:11:35.570 INFO:teuthology.orchestra.run.smithi121.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 33 +63us[ +63us] +/- 53ms 2024-04-24T23:11:35.571 INFO:teuthology.orchestra.run.smithi121.stdout:^* hv03.front.sepia.ceph.com 3 6 377 34 -69us[ -63us] +/- 61ms 2024-04-24T23:11:35.571 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:11:35.571 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-24T23:11:35.580 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-24T23:11:35.581 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-24T23:11:35.589 DEBUG:teuthology.orchestra.run.smithi032:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-24T23:11:35.703 DEBUG:teuthology.orchestra.run.smithi121:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-24T23:11:35.772 DEBUG:teuthology.orchestra.run.smithi032:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T23:11:35.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:11:35.804 DEBUG:teuthology.orchestra.run.smithi121:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T23:11:35.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:11:35.832 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-24T23:11:35.842 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-24T23:11:35.850 INFO:teuthology.task.internal:Duration was 1095.448203 seconds 2024-04-24T23:11:35.850 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-24T23:11:35.859 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-24T23:11:35.859 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-24T23:11:35.862 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-24T23:11:35.900 INFO:teuthology.orchestra.run.smithi032.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T23:11:35.910 INFO:teuthology.orchestra.run.smithi121.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T23:11:36.151 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-24T23:11:36.151 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi032.front.sepia.ceph.com 2024-04-24T23:11:36.151 DEBUG:teuthology.orchestra.run.smithi032:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-24T23:11:36.194 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi121.front.sepia.ceph.com 2024-04-24T23:11:36.194 DEBUG:teuthology.orchestra.run.smithi121:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-24T23:11:36.216 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-24T23:11:36.216 DEBUG:teuthology.orchestra.run.smithi032:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-24T23:11:36.237 DEBUG:teuthology.orchestra.run.smithi121:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-24T23:11:36.282 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-24T23:11:36.282 DEBUG:teuthology.orchestra.run.smithi032:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-24T23:11:36.304 DEBUG:teuthology.orchestra.run.smithi121:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-24T23:11:36.436 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-24T23:11:36.446 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-24T23:11:36.446 DEBUG:teuthology.orchestra.run.smithi032:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-24T23:11:36.470 DEBUG:teuthology.orchestra.run.smithi121:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-24T23:11:36.494 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-24T23:11:36.503 DEBUG:teuthology.orchestra.run.smithi032:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-24T23:11:36.513 DEBUG:teuthology.orchestra.run.smithi121:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-24T23:11:36.534 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern = core 2024-04-24T23:11:36.556 INFO:teuthology.orchestra.run.smithi121.stdout:kernel.core_pattern = core 2024-04-24T23:11:36.568 DEBUG:teuthology.orchestra.run.smithi032:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-24T23:11:36.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:11:36.602 DEBUG:teuthology.orchestra.run.smithi121:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-24T23:11:36.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:11:36.622 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-24T23:11:36.633 INFO:teuthology.task.internal:Transferring archived files... 2024-04-24T23:11:36.634 DEBUG:teuthology.misc:Transferring archived files from smithi032:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672173/remote/smithi032 2024-04-24T23:11:36.634 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-24T23:11:36.677 DEBUG:teuthology.misc:Transferring archived files from smithi121:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672173/remote/smithi121 2024-04-24T23:11:36.677 DEBUG:teuthology.orchestra.run.smithi121:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-24T23:11:36.709 INFO:teuthology.task.internal:Removing archive directory... 2024-04-24T23:11:36.709 DEBUG:teuthology.orchestra.run.smithi032:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-24T23:11:36.713 DEBUG:teuthology.orchestra.run.smithi121:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-24T23:11:36.758 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-24T23:11:36.770 INFO:teuthology.task.internal:Not uploading archives. 2024-04-24T23:11:36.770 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-24T23:11:36.781 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-24T23:11:36.781 DEBUG:teuthology.orchestra.run.smithi032:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-24T23:11:36.783 DEBUG:teuthology.orchestra.run.smithi121:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-24T23:11:36.796 INFO:teuthology.orchestra.run.smithi032.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 24 23:11 /home/ubuntu/cephtest 2024-04-24T23:11:36.812 INFO:teuthology.orchestra.run.smithi121.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 24 23:11 /home/ubuntu/cephtest 2024-04-24T23:11:36.813 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-24T23:11:36.824 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-24T23:11:36.848 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/mds-full} duration: 1095.4482033252716 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-04-24T23:11:36.848 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-24T23:11:36.937 INFO:teuthology.run:pass